2023-11-09T03:32:46.210 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-09T03:32:46.214 DEBUG:teuthology.run:Teuthology command: teuthology --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snapshots} --archive /home/teuthworker/archive/pdonnell-2023-11-09_03:21:01-fs:functional-main-distro-default-smithi/7452464 --name pdonnell-2023-11-09_03:21:01-fs:functional-main-distro-default-smithi --verbose --owner scheduled_pdonnell@teuthology -- /home/teuthworker/archive/pdonnell-2023-11-09_03:21:01-fs:functional-main-distro-default-smithi/7452464/orig.config.yaml 2023-11-09T03:32:46.259 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-09T03:32:46.342 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2023-11-09_03:21:01-fs:functional-main-distro-default-smithi/7452464 branch: main description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snapshots} email: pdonnell@redhat.com first_in_suite: false job_id: '7452464' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2023-11-09_03:21:01-fs:functional-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - evicting unresponsive client - RECENT_CRASH sha1: 7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs check-counter: dry_run: true install: ceph: flavor: default sha1: 7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20231108.195322 sha1: 8dae6b9fa1760040e7156daa9d03f77642df4f90 owner: scheduled_pdonnell@teuthology priority: 50 repo: https://github.com/ceph/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 5927 sha1: 7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4 sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-batrick-testing-20231108.195322 suite_path: /home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph.git suite_sha1: 8dae6b9fa1760040e7156daa9d03f77642df4f90 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi076.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBSaOfkkKDMrAw8lm/2WHJKaNe6HUq/VfeTji9wjOUR84c6zvChUurbrU6h/utzarVeqdBjqZcioe/t0+54KMAU= smithi113.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK1EN9hrd6xzeNxf/hBCFS2ygei1vHqiKEq9qLM07K+yU9nQIGDVWXPapz/JUuXhIDcePlEle4hhoRzs9FOUUMg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - exec: mon.a: - ceph config set mgr mgr/crash/warn_recent_interval 0 - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_snapshots teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 6899cd26fceddb2fec83dc1a1349394b28c8998e timestamp: 2023-11-09_03:21:01 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3590714 2023-11-09T03:32:46.343 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa; will attempt to use it 2023-11-09T03:32:46.343 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks 2023-11-09T03:32:46.344 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-09T03:32:46.346 INFO:teuthology.task.internal:Checking packages... 2023-11-09T03:32:46.371 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4' 2023-11-09T03:32:46.371 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-09T03:32:46.371 INFO:teuthology.packaging:ref: None 2023-11-09T03:32:46.372 INFO:teuthology.packaging:tag: None 2023-11-09T03:32:46.372 INFO:teuthology.packaging:branch: main 2023-11-09T03:32:46.372 INFO:teuthology.packaging:sha1: 7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4 2023-11-09T03:32:46.373 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2023-11-09T03:32:46.772 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7258.g96f65fd8 2023-11-09T03:32:46.773 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-09T03:32:46.779 INFO:teuthology.task.internal:no buildpackages task found 2023-11-09T03:32:46.780 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-09T03:32:46.793 INFO:teuthology.task.internal:Saving configuration 2023-11-09T03:32:46.810 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-09T03:32:46.817 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-09T03:32:46.851 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi076.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-11-09_03:21:01-fs:functional-main-distro-default-smithi/7452464', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-09 03:27:25.666687', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBSaOfkkKDMrAw8lm/2WHJKaNe6HUq/VfeTji9wjOUR84c6zvChUurbrU6h/utzarVeqdBjqZcioe/t0+54KMAU='} 2023-11-09T03:32:46.878 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi113.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-11-09_03:21:01-fs:functional-main-distro-default-smithi/7452464', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-09 03:27:25.668533', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK1EN9hrd6xzeNxf/hBCFS2ygei1vHqiKEq9qLM07K+yU9nQIGDVWXPapz/JUuXhIDcePlEle4hhoRzs9FOUUMg='} 2023-11-09T03:32:46.879 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-09T03:32:46.885 INFO:teuthology.task.internal:roles: ubuntu@smithi076.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-09T03:32:46.885 INFO:teuthology.task.internal:roles: ubuntu@smithi113.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-09T03:32:46.886 INFO:teuthology.run_tasks:Running task console_log... 2023-11-09T03:32:46.968 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f48f0fffb80>, signals=[15]) 2023-11-09T03:32:46.969 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-09T03:32:46.976 INFO:teuthology.task.internal:Opening connections... 2023-11-09T03:32:46.976 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T03:32:46.978 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-09T03:32:47.063 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi113.front.sepia.ceph.com 2023-11-09T03:32:47.064 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-09T03:32:47.150 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-09T03:32:47.156 DEBUG:teuthology.orchestra.run.smithi076:> uname -m 2023-11-09T03:32:47.174 INFO:teuthology.orchestra.run.smithi076.stdout:x86_64 2023-11-09T03:32:47.174 DEBUG:teuthology.orchestra.run.smithi076:> cat /etc/os-release 2023-11-09T03:32:47.230 INFO:teuthology.orchestra.run.smithi076.stdout:NAME="CentOS Stream" 2023-11-09T03:32:47.230 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION="9" 2023-11-09T03:32:47.230 INFO:teuthology.orchestra.run.smithi076.stdout:ID="centos" 2023-11-09T03:32:47.231 INFO:teuthology.orchestra.run.smithi076.stdout:ID_LIKE="rhel fedora" 2023-11-09T03:32:47.231 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION_ID="9" 2023-11-09T03:32:47.231 INFO:teuthology.orchestra.run.smithi076.stdout:PLATFORM_ID="platform:el9" 2023-11-09T03:32:47.231 INFO:teuthology.orchestra.run.smithi076.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-09T03:32:47.231 INFO:teuthology.orchestra.run.smithi076.stdout:ANSI_COLOR="0;31" 2023-11-09T03:32:47.232 INFO:teuthology.orchestra.run.smithi076.stdout:LOGO="fedora-logo-icon" 2023-11-09T03:32:47.232 INFO:teuthology.orchestra.run.smithi076.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-09T03:32:47.232 INFO:teuthology.orchestra.run.smithi076.stdout:HOME_URL="https://centos.org/" 2023-11-09T03:32:47.232 INFO:teuthology.orchestra.run.smithi076.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-09T03:32:47.233 INFO:teuthology.orchestra.run.smithi076.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-09T03:32:47.233 INFO:teuthology.orchestra.run.smithi076.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-09T03:32:47.233 INFO:teuthology.lock.ops:Updating smithi076.front.sepia.ceph.com on lock server 2023-11-09T03:32:47.258 DEBUG:teuthology.orchestra.run.smithi113:> uname -m 2023-11-09T03:32:47.276 INFO:teuthology.orchestra.run.smithi113.stdout:x86_64 2023-11-09T03:32:47.276 DEBUG:teuthology.orchestra.run.smithi113:> cat /etc/os-release 2023-11-09T03:32:47.333 INFO:teuthology.orchestra.run.smithi113.stdout:NAME="CentOS Stream" 2023-11-09T03:32:47.333 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION="9" 2023-11-09T03:32:47.334 INFO:teuthology.orchestra.run.smithi113.stdout:ID="centos" 2023-11-09T03:32:47.334 INFO:teuthology.orchestra.run.smithi113.stdout:ID_LIKE="rhel fedora" 2023-11-09T03:32:47.334 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION_ID="9" 2023-11-09T03:32:47.335 INFO:teuthology.orchestra.run.smithi113.stdout:PLATFORM_ID="platform:el9" 2023-11-09T03:32:47.335 INFO:teuthology.orchestra.run.smithi113.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-09T03:32:47.335 INFO:teuthology.orchestra.run.smithi113.stdout:ANSI_COLOR="0;31" 2023-11-09T03:32:47.335 INFO:teuthology.orchestra.run.smithi113.stdout:LOGO="fedora-logo-icon" 2023-11-09T03:32:47.335 INFO:teuthology.orchestra.run.smithi113.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-09T03:32:47.336 INFO:teuthology.orchestra.run.smithi113.stdout:HOME_URL="https://centos.org/" 2023-11-09T03:32:47.336 INFO:teuthology.orchestra.run.smithi113.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-09T03:32:47.336 INFO:teuthology.orchestra.run.smithi113.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-09T03:32:47.336 INFO:teuthology.orchestra.run.smithi113.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-09T03:32:47.336 INFO:teuthology.lock.ops:Updating smithi113.front.sepia.ceph.com on lock server 2023-11-09T03:32:47.358 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-09T03:32:47.368 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-09T03:32:47.374 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-09T03:32:47.374 DEBUG:teuthology.orchestra.run.smithi076:> test '!' -e /home/ubuntu/cephtest 2023-11-09T03:32:47.377 DEBUG:teuthology.orchestra.run.smithi113:> test '!' -e /home/ubuntu/cephtest 2023-11-09T03:32:47.393 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-09T03:32:47.399 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-09T03:32:47.400 DEBUG:teuthology.orchestra.run.smithi076:> test -z $(ls -A /var/lib/ceph) 2023-11-09T03:32:47.433 DEBUG:teuthology.orchestra.run.smithi113:> test -z $(ls -A /var/lib/ceph) 2023-11-09T03:32:47.543 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-09T03:32:47.599 INFO:teuthology.run_tasks:Running task kernel... 2023-11-09T03:32:47.614 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-11-09T03:32:47.615 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-11-09T03:32:47.616 DEBUG:teuthology.orchestra.run.smithi076:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-09T03:32:47.616 DEBUG:teuthology.orchestra.run.smithi113:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-09T03:32:47.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:32:47.634 DEBUG:teuthology.orchestra.run.smithi076:> uname -r 2023-11-09T03:32:47.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:32:47.635 DEBUG:teuthology.orchestra.run.smithi113:> uname -r 2023-11-09T03:32:47.691 INFO:teuthology.orchestra.run.smithi076.stdout:5.14.0-361.el9.x86_64 2023-11-09T03:32:47.694 INFO:teuthology.task.kernel:Running kernel on smithi076: 5.14.0-361.el9.x86_64 2023-11-09T03:32:47.694 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum install -y kernel 2023-11-09T03:32:47.695 INFO:teuthology.orchestra.run.smithi113.stdout:5.14.0-361.el9.x86_64 2023-11-09T03:32:47.696 INFO:teuthology.task.kernel:Running kernel on smithi113: 5.14.0-361.el9.x86_64 2023-11-09T03:32:47.696 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum install -y kernel 2023-11-09T03:32:49.867 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS-9 - AppStream 13 MB/s | 6.3 MB 00:00 2023-11-09T03:32:49.869 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS-9 - AppStream 12 MB/s | 6.3 MB 00:00 2023-11-09T03:32:51.251 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS-9 - BaseOS 6.1 MB/s | 2.0 MB 00:00 2023-11-09T03:32:51.579 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS-9 - BaseOS 3.1 MB/s | 2.0 MB 00:00 2023-11-09T03:32:52.108 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 7.9 MB 00:00 2023-11-09T03:32:53.086 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - BaseOS 6.7 MB/s | 7.9 MB 00:01 2023-11-09T03:32:55.101 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - AppStream 9.3 MB/s | 18 MB 00:01 2023-11-09T03:32:58.137 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - AppStream 4.3 MB/s | 18 MB 00:04 2023-11-09T03:32:59.994 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 5.8 MB 00:01 2023-11-09T03:33:01.570 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 15 kB 00:00 2023-11-09T03:33:02.012 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - CRB 12 MB/s | 5.8 MB 00:00 2023-11-09T03:33:02.799 INFO:teuthology.orchestra.run.smithi076.stdout:Extra Packages for Enterprise Linux 18 MB/s | 19 MB 00:01 2023-11-09T03:33:03.479 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 15 kB 00:00 2023-11-09T03:33:07.397 INFO:teuthology.orchestra.run.smithi076.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2023-11-09T03:33:09.704 INFO:teuthology.orchestra.run.smithi076.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-11-09T03:33:09.705 INFO:teuthology.orchestra.run.smithi076.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2023-11-09T03:33:09.794 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2023-11-09T03:33:09.800 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2023-11-09T03:33:09.800 INFO:teuthology.orchestra.run.smithi076.stdout: Package Arch Version Repository Size 2023-11-09T03:33:09.800 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2023-11-09T03:33:09.800 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2023-11-09T03:33:09.801 INFO:teuthology.orchestra.run.smithi076.stdout: kernel x86_64 5.14.0-382.el9 CentOS-BaseOS 5.4 M 2023-11-09T03:33:09.801 INFO:teuthology.orchestra.run.smithi076.stdout:Installing dependencies: 2023-11-09T03:33:09.801 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-core x86_64 5.14.0-382.el9 CentOS-BaseOS 20 M 2023-11-09T03:33:09.801 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-modules x86_64 5.14.0-382.el9 CentOS-BaseOS 38 M 2023-11-09T03:33:09.802 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-modules-core x86_64 5.14.0-382.el9 CentOS-BaseOS 32 M 2023-11-09T03:33:09.802 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:33:09.802 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2023-11-09T03:33:09.802 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2023-11-09T03:33:09.802 INFO:teuthology.orchestra.run.smithi076.stdout:Install 4 Packages 2023-11-09T03:33:09.803 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:33:09.803 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 95 M 2023-11-09T03:33:09.804 INFO:teuthology.orchestra.run.smithi076.stdout:Installed size: 121 M 2023-11-09T03:33:09.804 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2023-11-09T03:33:10.040 INFO:teuthology.orchestra.run.smithi076.stdout:(1/4): kernel-5.14.0-382.el9.x86_64.rpm 23 MB/s | 5.4 MB 00:00 2023-11-09T03:33:10.316 INFO:teuthology.orchestra.run.smithi076.stdout:(2/4): kernel-core-5.14.0-382.el9.x86_64.rpm 39 MB/s | 20 MB 00:00 2023-11-09T03:33:10.707 INFO:teuthology.orchestra.run.smithi076.stdout:(3/4): kernel-modules-core-5.14.0-382.el9.x86_6 49 MB/s | 32 MB 00:00 2023-11-09T03:33:11.083 INFO:teuthology.orchestra.run.smithi076.stdout:(4/4): kernel-modules-5.14.0-382.el9.x86_64.rpm 29 MB/s | 38 MB 00:01 2023-11-09T03:33:11.086 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2023-11-09T03:33:11.086 INFO:teuthology.orchestra.run.smithi076.stdout:Total 74 MB/s | 95 MB 00:01 2023-11-09T03:33:11.087 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2023-11-09T03:33:11.472 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2023-11-09T03:33:11.472 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2023-11-09T03:33:12.477 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2023-11-09T03:33:12.477 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2023-11-09T03:33:14.854 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2023-11-09T03:33:15.522 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : kernel-modules-core-5.14.0-382.el9.x86_64 1/4 2023-11-09T03:33:15.672 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : kernel-core-5.14.0-382.el9.x86_64 2/4 2023-11-09T03:33:18.384 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-core-5.14.0-382.el9.x86_64 2/4 2023-11-09T03:33:18.578 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : kernel-modules-5.14.0-382.el9.x86_64 3/4 2023-11-09T03:33:19.286 INFO:teuthology.orchestra.run.smithi113.stdout:Extra Packages for Enterprise Linux 1.2 MB/s | 19 MB 00:15 2023-11-09T03:33:23.149 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-modules-5.14.0-382.el9.x86_64 3/4 2023-11-09T03:33:23.256 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : kernel-5.14.0-382.el9.x86_64 4/4 2023-11-09T03:33:23.902 INFO:teuthology.orchestra.run.smithi113.stdout:lab-extras 46 kB/s | 1.7 kB 00:00 2023-11-09T03:33:26.218 INFO:teuthology.orchestra.run.smithi113.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-11-09T03:33:26.218 INFO:teuthology.orchestra.run.smithi113.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2023-11-09T03:33:26.306 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2023-11-09T03:33:26.311 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2023-11-09T03:33:26.311 INFO:teuthology.orchestra.run.smithi113.stdout: Package Arch Version Repository Size 2023-11-09T03:33:26.311 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2023-11-09T03:33:26.312 INFO:teuthology.orchestra.run.smithi113.stdout:Installing: 2023-11-09T03:33:26.312 INFO:teuthology.orchestra.run.smithi113.stdout: kernel x86_64 5.14.0-382.el9 CentOS-BaseOS 5.4 M 2023-11-09T03:33:26.312 INFO:teuthology.orchestra.run.smithi113.stdout:Installing dependencies: 2023-11-09T03:33:26.312 INFO:teuthology.orchestra.run.smithi113.stdout: kernel-core x86_64 5.14.0-382.el9 CentOS-BaseOS 20 M 2023-11-09T03:33:26.312 INFO:teuthology.orchestra.run.smithi113.stdout: kernel-modules x86_64 5.14.0-382.el9 CentOS-BaseOS 38 M 2023-11-09T03:33:26.313 INFO:teuthology.orchestra.run.smithi113.stdout: kernel-modules-core x86_64 5.14.0-382.el9 CentOS-BaseOS 32 M 2023-11-09T03:33:26.313 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:33:26.313 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2023-11-09T03:33:26.313 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2023-11-09T03:33:26.313 INFO:teuthology.orchestra.run.smithi113.stdout:Install 4 Packages 2023-11-09T03:33:26.314 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:33:26.315 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 95 M 2023-11-09T03:33:26.315 INFO:teuthology.orchestra.run.smithi113.stdout:Installed size: 121 M 2023-11-09T03:33:26.316 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2023-11-09T03:33:26.570 INFO:teuthology.orchestra.run.smithi113.stdout:(1/4): kernel-5.14.0-382.el9.x86_64.rpm 21 MB/s | 5.4 MB 00:00 2023-11-09T03:33:26.804 INFO:teuthology.orchestra.run.smithi113.stdout:(2/4): kernel-core-5.14.0-382.el9.x86_64.rpm 41 MB/s | 20 MB 00:00 2023-11-09T03:33:27.204 INFO:teuthology.orchestra.run.smithi113.stdout:(3/4): kernel-modules-5.14.0-382.el9.x86_64.rpm 42 MB/s | 38 MB 00:00 2023-11-09T03:33:27.529 INFO:teuthology.orchestra.run.smithi113.stdout:(4/4): kernel-modules-core-5.14.0-382.el9.x86_6 34 MB/s | 32 MB 00:00 2023-11-09T03:33:27.532 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2023-11-09T03:33:27.533 INFO:teuthology.orchestra.run.smithi113.stdout:Total 78 MB/s | 95 MB 00:01 2023-11-09T03:33:27.533 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2023-11-09T03:33:27.868 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-modules-core-5.14.0-382.el9.x86_64 4/4 2023-11-09T03:33:27.897 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2023-11-09T03:33:27.897 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2023-11-09T03:33:28.906 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2023-11-09T03:33:28.907 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2023-11-09T03:33:31.274 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2023-11-09T03:33:31.914 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : kernel-modules-core-5.14.0-382.el9.x86_64 1/4 2023-11-09T03:33:32.038 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : kernel-core-5.14.0-382.el9.x86_64 2/4 2023-11-09T03:33:34.764 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: kernel-core-5.14.0-382.el9.x86_64 2/4 2023-11-09T03:33:34.940 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : kernel-modules-5.14.0-382.el9.x86_64 3/4 2023-11-09T03:33:39.534 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: kernel-modules-5.14.0-382.el9.x86_64 3/4 2023-11-09T03:33:39.641 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : kernel-5.14.0-382.el9.x86_64 4/4 2023-11-09T03:33:44.237 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: kernel-modules-core-5.14.0-382.el9.x86_64 4/4 2023-11-09T03:33:57.067 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-core-5.14.0-382.el9.x86_64 4/4 2023-11-09T03:33:57.108 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-modules-5.14.0-382.el9.x86_64 4/4 2023-11-09T03:33:57.530 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-5.14.0-382.el9.x86_64 4/4 2023-11-09T03:33:57.530 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : kernel-5.14.0-382.el9.x86_64 1/4 2023-11-09T03:33:57.531 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : kernel-core-5.14.0-382.el9.x86_64 2/4 2023-11-09T03:33:57.531 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : kernel-modules-5.14.0-382.el9.x86_64 3/4 2023-11-09T03:33:57.958 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : kernel-modules-core-5.14.0-382.el9.x86_64 4/4 2023-11-09T03:33:57.959 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:33:57.959 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2023-11-09T03:33:57.959 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-5.14.0-382.el9.x86_64 2023-11-09T03:33:57.959 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-core-5.14.0-382.el9.x86_64 2023-11-09T03:33:57.959 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-modules-5.14.0-382.el9.x86_64 2023-11-09T03:33:57.960 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-modules-core-5.14.0-382.el9.x86_64 2023-11-09T03:33:57.960 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:33:57.960 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2023-11-09T03:33:58.469 DEBUG:teuthology.orchestra.run.smithi076:> rpm -q kernel | sort -rV | head -n 1 2023-11-09T03:33:58.588 INFO:teuthology.orchestra.run.smithi076.stdout:kernel-5.14.0-382.el9.x86_64 2023-11-09T03:33:58.589 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-382.el9.x86_64 2023-11-09T03:33:58.589 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-382.el9.x86_64 2023-11-09T03:33:58.589 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-11-09T03:33:58.589 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-11-09T03:33:58.590 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-11-09T03:33:58.590 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi076.front.sepia.ceph.com, path=None, version=distro) 2023-11-09T03:33:58.590 DEBUG:teuthology.orchestra.run.smithi076:> rpm -q kernel | sort -rV | head -n 1 2023-11-09T03:33:58.695 INFO:teuthology.orchestra.run.smithi076.stdout:kernel-5.14.0-382.el9.x86_64 2023-11-09T03:33:58.696 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-382.el9.x86_64 2023-11-09T03:33:58.696 DEBUG:teuthology.orchestra.run.smithi076:> sudo rpm -qi grub2-tools 2023-11-09T03:33:58.733 INFO:teuthology.orchestra.run.smithi076.stdout:Name : grub2-tools 2023-11-09T03:33:58.733 INFO:teuthology.orchestra.run.smithi076.stdout:Epoch : 1 2023-11-09T03:33:58.733 INFO:teuthology.orchestra.run.smithi076.stdout:Version : 2.06 2023-11-09T03:33:58.734 INFO:teuthology.orchestra.run.smithi076.stdout:Release : 68.el9 2023-11-09T03:33:58.734 INFO:teuthology.orchestra.run.smithi076.stdout:Architecture: x86_64 2023-11-09T03:33:58.734 INFO:teuthology.orchestra.run.smithi076.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-11-09T03:33:58.734 INFO:teuthology.orchestra.run.smithi076.stdout:Group : Unspecified 2023-11-09T03:33:58.735 INFO:teuthology.orchestra.run.smithi076.stdout:Size : 8264254 2023-11-09T03:33:58.735 INFO:teuthology.orchestra.run.smithi076.stdout:License : GPLv3+ 2023-11-09T03:33:58.735 INFO:teuthology.orchestra.run.smithi076.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-11-09T03:33:58.735 INFO:teuthology.orchestra.run.smithi076.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-11-09T03:33:58.735 INFO:teuthology.orchestra.run.smithi076.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-11-09T03:33:58.736 INFO:teuthology.orchestra.run.smithi076.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-11-09T03:33:58.736 INFO:teuthology.orchestra.run.smithi076.stdout:Packager : builder@centos.org 2023-11-09T03:33:58.736 INFO:teuthology.orchestra.run.smithi076.stdout:Vendor : CentOS 2023-11-09T03:33:58.736 INFO:teuthology.orchestra.run.smithi076.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-09T03:33:58.736 INFO:teuthology.orchestra.run.smithi076.stdout:Summary : Support tools for GRUB. 2023-11-09T03:33:58.737 INFO:teuthology.orchestra.run.smithi076.stdout:Description : 2023-11-09T03:33:58.737 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:33:58.737 INFO:teuthology.orchestra.run.smithi076.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-09T03:33:58.737 INFO:teuthology.orchestra.run.smithi076.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-09T03:33:58.738 INFO:teuthology.orchestra.run.smithi076.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-09T03:33:58.738 INFO:teuthology.orchestra.run.smithi076.stdout:hardware devices. 2023-11-09T03:33:58.738 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:33:58.738 INFO:teuthology.orchestra.run.smithi076.stdout:This subpackage provides tools for support of all platforms. 2023-11-09T03:33:58.740 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-09T03:33:58.740 INFO:teuthology.task.kernel:Updating grub on smithi076 to boot 5.14.0-382.el9.x86_64 2023-11-09T03:33:58.741 DEBUG:teuthology.orchestra.run.smithi076:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-09T03:33:59.353 INFO:teuthology.orchestra.run.smithi076.stderr:Generating grub configuration file ... 2023-11-09T03:34:00.691 INFO:teuthology.orchestra.run.smithi076.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-11-09T03:34:00.722 INFO:teuthology.orchestra.run.smithi076.stderr:done 2023-11-09T03:34:00.725 DEBUG:teuthology.orchestra.run.smithi076:> mktemp 2023-11-09T03:34:00.741 INFO:teuthology.orchestra.run.smithi076.stdout:/tmp/tmp.MbjoMcFB3g 2023-11-09T03:34:00.742 DEBUG:teuthology.orchestra.run.smithi076:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.MbjoMcFB3g 2023-11-09T03:34:00.812 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0666 /tmp/tmp.MbjoMcFB3g 2023-11-09T03:34:00.945 DEBUG:teuthology.orchestra.remote:smithi076:/tmp/tmp.MbjoMcFB3g is 6KB 2023-11-09T03:34:00.993 DEBUG:teuthology.orchestra.run.smithi076:> rm -fr /tmp/tmp.MbjoMcFB3g 2023-11-09T03:34:01.008 DEBUG:teuthology.orchestra.run.smithi076:> sudo /bin/ls /boot/loader/entries || true 2023-11-09T03:34:01.085 INFO:teuthology.orchestra.run.smithi076.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-11-09T03:34:01.085 INFO:teuthology.orchestra.run.smithi076.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-11-09T03:34:01.086 INFO:teuthology.orchestra.run.smithi076.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-11-09T03:34:01.086 INFO:teuthology.orchestra.run.smithi076.stdout:15949e3c88704300882482eddfd7baae-5.14.0-382.el9.x86_64.conf 2023-11-09T03:34:01.087 DEBUG:teuthology.orchestra.run.smithi076:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-382.el9.x86_64 2023-11-09T03:34:01.244 DEBUG:teuthology.orchestra.run.smithi076:> sudo shutdown -r now 2023-11-09T03:34:13.296 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: kernel-core-5.14.0-382.el9.x86_64 4/4 2023-11-09T03:34:13.339 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: kernel-modules-5.14.0-382.el9.x86_64 4/4 2023-11-09T03:34:13.705 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: kernel-5.14.0-382.el9.x86_64 4/4 2023-11-09T03:34:13.706 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : kernel-5.14.0-382.el9.x86_64 1/4 2023-11-09T03:34:13.706 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : kernel-core-5.14.0-382.el9.x86_64 2/4 2023-11-09T03:34:13.706 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : kernel-modules-5.14.0-382.el9.x86_64 3/4 2023-11-09T03:34:14.085 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : kernel-modules-core-5.14.0-382.el9.x86_64 4/4 2023-11-09T03:34:14.085 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:34:14.085 INFO:teuthology.orchestra.run.smithi113.stdout:Installed: 2023-11-09T03:34:14.086 INFO:teuthology.orchestra.run.smithi113.stdout: kernel-5.14.0-382.el9.x86_64 2023-11-09T03:34:14.086 INFO:teuthology.orchestra.run.smithi113.stdout: kernel-core-5.14.0-382.el9.x86_64 2023-11-09T03:34:14.086 INFO:teuthology.orchestra.run.smithi113.stdout: kernel-modules-5.14.0-382.el9.x86_64 2023-11-09T03:34:14.086 INFO:teuthology.orchestra.run.smithi113.stdout: kernel-modules-core-5.14.0-382.el9.x86_64 2023-11-09T03:34:14.086 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:34:14.087 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2023-11-09T03:34:14.561 DEBUG:teuthology.orchestra.run.smithi113:> rpm -q kernel | sort -rV | head -n 1 2023-11-09T03:34:14.699 INFO:teuthology.orchestra.run.smithi113.stdout:kernel-5.14.0-382.el9.x86_64 2023-11-09T03:34:14.699 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-382.el9.x86_64 2023-11-09T03:34:14.699 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-382.el9.x86_64 2023-11-09T03:34:14.700 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-11-09T03:34:14.700 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-11-09T03:34:14.700 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-11-09T03:34:14.700 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi113.front.sepia.ceph.com, path=None, version=distro) 2023-11-09T03:34:14.701 DEBUG:teuthology.orchestra.run.smithi113:> rpm -q kernel | sort -rV | head -n 1 2023-11-09T03:34:14.806 INFO:teuthology.orchestra.run.smithi113.stdout:kernel-5.14.0-382.el9.x86_64 2023-11-09T03:34:14.807 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-382.el9.x86_64 2023-11-09T03:34:14.808 DEBUG:teuthology.orchestra.run.smithi113:> sudo rpm -qi grub2-tools 2023-11-09T03:34:14.850 INFO:teuthology.orchestra.run.smithi113.stdout:Name : grub2-tools 2023-11-09T03:34:14.851 INFO:teuthology.orchestra.run.smithi113.stdout:Epoch : 1 2023-11-09T03:34:14.851 INFO:teuthology.orchestra.run.smithi113.stdout:Version : 2.06 2023-11-09T03:34:14.851 INFO:teuthology.orchestra.run.smithi113.stdout:Release : 68.el9 2023-11-09T03:34:14.851 INFO:teuthology.orchestra.run.smithi113.stdout:Architecture: x86_64 2023-11-09T03:34:14.852 INFO:teuthology.orchestra.run.smithi113.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-11-09T03:34:14.852 INFO:teuthology.orchestra.run.smithi113.stdout:Group : Unspecified 2023-11-09T03:34:14.852 INFO:teuthology.orchestra.run.smithi113.stdout:Size : 8264254 2023-11-09T03:34:14.852 INFO:teuthology.orchestra.run.smithi113.stdout:License : GPLv3+ 2023-11-09T03:34:14.852 INFO:teuthology.orchestra.run.smithi113.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-11-09T03:34:14.853 INFO:teuthology.orchestra.run.smithi113.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-11-09T03:34:14.853 INFO:teuthology.orchestra.run.smithi113.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-11-09T03:34:14.853 INFO:teuthology.orchestra.run.smithi113.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-11-09T03:34:14.853 INFO:teuthology.orchestra.run.smithi113.stdout:Packager : builder@centos.org 2023-11-09T03:34:14.853 INFO:teuthology.orchestra.run.smithi113.stdout:Vendor : CentOS 2023-11-09T03:34:14.854 INFO:teuthology.orchestra.run.smithi113.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-09T03:34:14.854 INFO:teuthology.orchestra.run.smithi113.stdout:Summary : Support tools for GRUB. 2023-11-09T03:34:14.854 INFO:teuthology.orchestra.run.smithi113.stdout:Description : 2023-11-09T03:34:14.854 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:34:14.854 INFO:teuthology.orchestra.run.smithi113.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-09T03:34:14.855 INFO:teuthology.orchestra.run.smithi113.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-09T03:34:14.856 INFO:teuthology.orchestra.run.smithi113.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-09T03:34:14.856 INFO:teuthology.orchestra.run.smithi113.stdout:hardware devices. 2023-11-09T03:34:14.856 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:34:14.857 INFO:teuthology.orchestra.run.smithi113.stdout:This subpackage provides tools for support of all platforms. 2023-11-09T03:34:14.858 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-09T03:34:14.858 INFO:teuthology.task.kernel:Updating grub on smithi113 to boot 5.14.0-382.el9.x86_64 2023-11-09T03:34:14.858 DEBUG:teuthology.orchestra.run.smithi113:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-09T03:34:15.429 INFO:teuthology.orchestra.run.smithi113.stderr:Generating grub configuration file ... 2023-11-09T03:34:16.764 INFO:teuthology.orchestra.run.smithi113.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-11-09T03:34:16.819 INFO:teuthology.orchestra.run.smithi113.stderr:done 2023-11-09T03:34:16.821 DEBUG:teuthology.orchestra.run.smithi113:> mktemp 2023-11-09T03:34:16.836 INFO:teuthology.orchestra.run.smithi113.stdout:/tmp/tmp.Ect0syDIbG 2023-11-09T03:34:16.838 DEBUG:teuthology.orchestra.run.smithi113:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Ect0syDIbG 2023-11-09T03:34:16.905 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0666 /tmp/tmp.Ect0syDIbG 2023-11-09T03:34:17.045 DEBUG:teuthology.orchestra.remote:smithi113:/tmp/tmp.Ect0syDIbG is 6KB 2023-11-09T03:34:17.097 DEBUG:teuthology.orchestra.run.smithi113:> rm -fr /tmp/tmp.Ect0syDIbG 2023-11-09T03:34:17.112 DEBUG:teuthology.orchestra.run.smithi113:> sudo /bin/ls /boot/loader/entries || true 2023-11-09T03:34:17.177 INFO:teuthology.orchestra.run.smithi113.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-11-09T03:34:17.177 INFO:teuthology.orchestra.run.smithi113.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-11-09T03:34:17.177 INFO:teuthology.orchestra.run.smithi113.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-11-09T03:34:17.178 INFO:teuthology.orchestra.run.smithi113.stdout:15949e3c88704300882482eddfd7baae-5.14.0-382.el9.x86_64.conf 2023-11-09T03:34:17.179 DEBUG:teuthology.orchestra.run.smithi113:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-382.el9.x86_64 2023-11-09T03:34:17.327 DEBUG:teuthology.orchestra.run.smithi113:> sudo shutdown -r now 2023-11-09T03:34:31.258 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-11-09T03:34:31.259 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2023-11-09T03:34:31.260 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-09T03:34:47.346 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-11-09T03:34:47.347 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2023-11-09T03:34:47.347 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-09T03:35:05.854 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.76 2023-11-09T03:35:14.862 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2023-11-09T03:35:14.863 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-09T03:35:21.978 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.113 2023-11-09T03:35:30.986 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2023-11-09T03:35:30.987 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-09T03:35:34.046 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.113 2023-11-09T03:35:46.058 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2023-11-09T03:35:46.059 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-09T03:36:04.474 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.113 2023-11-09T03:36:14.876 DEBUG:teuthology.orchestra.remote:timed out 2023-11-09T03:36:19.478 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2023-11-09T03:36:19.479 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-09T03:36:19.868 DEBUG:teuthology.orchestra.run.smithi113:> true 2023-11-09T03:36:20.241 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi113.front.sepia.ceph.com' 2023-11-09T03:36:20.241 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-382.el9.x86_64"... 2023-11-09T03:36:20.242 DEBUG:teuthology.orchestra.run.smithi113:> uname -r 2023-11-09T03:36:20.256 INFO:teuthology.orchestra.run.smithi113.stdout:5.14.0-382.el9.x86_64 2023-11-09T03:36:20.257 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-382.el9.x86_64 vs 5.14.0-382.el9.x86_64 2023-11-09T03:36:20.258 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-09T03:36:20.258 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-09T03:36:21.259 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-11-09T03:36:21.260 DEBUG:teuthology.orchestra.run.smithi113:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-09T03:36:21.330 INFO:teuthology.orchestra.run.smithi113.stdout:ttyS1 2023-11-09T03:36:21.375 DEBUG:teuthology.parallel:result is None 2023-11-09T03:36:26.878 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2023-11-09T03:36:26.879 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-09T03:36:27.319 DEBUG:teuthology.orchestra.run.smithi076:> true 2023-11-09T03:36:27.657 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi076.front.sepia.ceph.com' 2023-11-09T03:36:27.657 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-382.el9.x86_64"... 2023-11-09T03:36:27.658 DEBUG:teuthology.orchestra.run.smithi076:> uname -r 2023-11-09T03:36:27.672 INFO:teuthology.orchestra.run.smithi076.stdout:5.14.0-382.el9.x86_64 2023-11-09T03:36:27.672 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-382.el9.x86_64 vs 5.14.0-382.el9.x86_64 2023-11-09T03:36:27.673 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-09T03:36:27.673 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-09T03:36:28.673 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-11-09T03:36:28.674 DEBUG:teuthology.orchestra.run.smithi076:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-09T03:36:28.751 INFO:teuthology.orchestra.run.smithi076.stdout:ttyS1 2023-11-09T03:36:28.795 DEBUG:teuthology.parallel:result is None 2023-11-09T03:36:28.796 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-09T03:36:28.804 INFO:teuthology.task.internal:Creating test directory... 2023-11-09T03:36:28.804 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-09T03:36:28.807 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-09T03:36:28.826 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-09T03:36:28.834 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-09T03:36:28.845 INFO:teuthology.task.internal:Creating archive directory... 2023-11-09T03:36:28.846 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-09T03:36:28.865 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-09T03:36:28.991 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-09T03:36:29.000 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-09T03:36:29.000 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-09T03:36:29.003 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-09T03:36:29.036 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-09T03:36:29.038 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-09T03:36:29.050 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-09T03:36:29.051 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-09T03:36:29.053 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-09T03:36:29.060 INFO:teuthology.task.internal:Configuring sudo... 2023-11-09T03:36:29.060 DEBUG:teuthology.orchestra.run.smithi076:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-09T03:36:29.095 DEBUG:teuthology.orchestra.run.smithi113:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-09T03:36:29.126 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-09T03:36:29.137 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-09T03:36:29.138 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-09T03:36:29.165 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-09T03:36:29.183 DEBUG:teuthology.orchestra.run.smithi076:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-09T03:36:29.262 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-09T03:36:29.315 DEBUG:teuthology.orchestra.run.smithi076:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-09T03:36:29.399 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-09T03:36:29.440 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:36:29.440 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-09T03:36:29.506 DEBUG:teuthology.orchestra.run.smithi113:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-09T03:36:29.551 DEBUG:teuthology.orchestra.run.smithi113:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-09T03:36:29.607 DEBUG:teuthology.orchestra.run.smithi113:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-09T03:36:29.689 DEBUG:teuthology.orchestra.run.smithi113:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-09T03:36:29.736 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:36:29.736 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-09T03:36:29.803 DEBUG:teuthology.orchestra.run.smithi076:> sudo service rsyslog restart 2023-11-09T03:36:29.807 DEBUG:teuthology.orchestra.run.smithi113:> sudo service rsyslog restart 2023-11-09T03:36:29.853 INFO:teuthology.orchestra.run.smithi076.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-09T03:36:29.899 INFO:teuthology.orchestra.run.smithi113.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-09T03:36:30.253 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-09T03:36:30.264 INFO:teuthology.task.internal:Starting timer... 2023-11-09T03:36:30.265 INFO:teuthology.run_tasks:Running task pcp... 2023-11-09T03:36:30.278 INFO:teuthology.run_tasks:Running task selinux... 2023-11-09T03:36:30.294 DEBUG:teuthology.orchestra.run.smithi076:> sudo service auditd rotate 2023-11-09T03:36:30.358 INFO:teuthology.orchestra.run.smithi076.stdout:Rotating logs: 2023-11-09T03:36:30.359 DEBUG:teuthology.orchestra.run.smithi113:> sudo service auditd rotate 2023-11-09T03:36:30.435 INFO:teuthology.orchestra.run.smithi113.stdout:Rotating logs: 2023-11-09T03:36:30.437 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-09T03:36:30.437 DEBUG:teuthology.orchestra.run.smithi076:> /usr/sbin/getenforce 2023-11-09T03:36:30.461 INFO:teuthology.orchestra.run.smithi076.stdout:Permissive 2023-11-09T03:36:30.462 DEBUG:teuthology.orchestra.run.smithi113:> /usr/sbin/getenforce 2023-11-09T03:36:30.495 INFO:teuthology.orchestra.run.smithi113.stdout:Permissive 2023-11-09T03:36:30.496 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi076.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi113.front.sepia.ceph.com': 'permissive'} 2023-11-09T03:36:30.496 DEBUG:teuthology.orchestra.run.smithi076:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-09T03:36:30.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:36:30.529 DEBUG:teuthology.orchestra.run.smithi113:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-09T03:36:30.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:36:30.563 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-09T03:36:30.563 DEBUG:teuthology.orchestra.run.smithi076:> sudo /usr/sbin/setenforce permissive 2023-11-09T03:36:30.605 DEBUG:teuthology.orchestra.run.smithi113:> sudo /usr/sbin/setenforce permissive 2023-11-09T03:36:30.639 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-09T03:36:30.655 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-11-09T03:36:30.662 INFO:teuthology.repo_utils:Fetching main from origin 2023-11-09T03:36:30.780 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-09T03:36:30.797 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-09T03:36:30.799 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi076.front.sepia.ceph.com,smithi113.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-09T03:42:01.210 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi076.front.sepia.ceph.com'), Remote(name='ubuntu@smithi113.front.sepia.ceph.com')] 2023-11-09T03:42:01.211 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2023-11-09T03:42:01.212 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-09T03:42:01.296 DEBUG:teuthology.orchestra.run.smithi076:> true 2023-11-09T03:42:01.369 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi076.front.sepia.ceph.com' 2023-11-09T03:42:01.370 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2023-11-09T03:42:01.370 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-09T03:42:01.453 DEBUG:teuthology.orchestra.run.smithi113:> true 2023-11-09T03:42:01.522 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi113.front.sepia.ceph.com' 2023-11-09T03:42:01.522 INFO:teuthology.run_tasks:Running task clock... 2023-11-09T03:42:01.534 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-09T03:42:01.535 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-09T03:42:01.535 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-09T03:42:01.538 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-09T03:42:01.538 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-09T03:42:01.566 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-09T03:42:01.583 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-09T03:42:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-09T03:42:01.608 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-09T03:42:01.612 INFO:teuthology.orchestra.run.smithi076.stderr:sudo: ntpd: command not found 2023-11-09T03:42:01.625 INFO:teuthology.orchestra.run.smithi076.stdout:506 Cannot talk to daemon 2023-11-09T03:42:01.637 INFO:teuthology.orchestra.run.smithi113.stderr:sudo: ntpd: command not found 2023-11-09T03:42:01.642 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-09T03:42:01.650 INFO:teuthology.orchestra.run.smithi113.stdout:506 Cannot talk to daemon 2023-11-09T03:42:01.658 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-09T03:42:01.667 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-09T03:42:01.683 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-09T03:42:01.701 INFO:teuthology.orchestra.run.smithi076.stderr:bash: line 1: ntpq: command not found 2023-11-09T03:42:01.704 INFO:teuthology.orchestra.run.smithi076.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-09T03:42:01.705 INFO:teuthology.orchestra.run.smithi076.stdout:=============================================================================== 2023-11-09T03:42:01.705 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-09T03:42:01.705 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-09T03:42:01.705 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-09T03:42:01.705 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-09T03:42:01.726 INFO:teuthology.orchestra.run.smithi113.stderr:bash: line 1: ntpq: command not found 2023-11-09T03:42:01.729 INFO:teuthology.orchestra.run.smithi113.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-09T03:42:01.729 INFO:teuthology.orchestra.run.smithi113.stdout:=============================================================================== 2023-11-09T03:42:01.729 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-09T03:42:01.730 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-09T03:42:01.730 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-09T03:42:01.730 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-09T03:42:01.731 INFO:teuthology.run_tasks:Running task install... 2023-11-09T03:42:01.739 DEBUG:teuthology.task.install:project ceph 2023-11-09T03:42:01.740 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4'}} 2023-11-09T03:42:01.740 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4'} 2023-11-09T03:42:01.740 INFO:teuthology.task.install:Using flavor: default 2023-11-09T03:42:01.769 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-09T03:42:01.770 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-09T03:42:01.771 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4', 'tag': None, 'wait_for_package': False} 2023-11-09T03:42:01.771 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4 2023-11-09T03:42:01.773 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4', 'tag': None, 'wait_for_package': False} 2023-11-09T03:42:01.774 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4 2023-11-09T03:42:01.920 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/main/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/centos/9/flavors/default/ 2023-11-09T03:42:01.920 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7035.g7532011a 2023-11-09T03:42:01.990 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/main/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/centos/9/flavors/default/ 2023-11-09T03:42:01.990 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7035.g7532011a 2023-11-09T03:42:02.248 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/main/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/main/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/main/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-09T03:42:02.249 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:42:02.249 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-09T03:42:02.278 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-09T03:42:02.279 DEBUG:teuthology.orchestra.run.smithi113:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-09T03:42:02.307 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/main/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/main/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/main/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-09T03:42:02.307 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:42:02.308 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-09T03:42:02.336 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-09T03:42:02.336 DEBUG:teuthology.orchestra.run.smithi076:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-09T03:42:02.348 DEBUG:teuthology.orchestra.run.smithi113:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-09T03:42:02.405 DEBUG:teuthology.orchestra.run.smithi076:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-09T03:42:02.429 DEBUG:teuthology.orchestra.run.smithi113:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-09T03:42:02.485 DEBUG:teuthology.orchestra.run.smithi076:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-09T03:42:02.496 INFO:teuthology.orchestra.run.smithi113.stdout:check_obsoletes = 1 2023-11-09T03:42:02.497 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum clean all 2023-11-09T03:42:02.551 INFO:teuthology.orchestra.run.smithi076.stdout:check_obsoletes = 1 2023-11-09T03:42:02.553 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum clean all 2023-11-09T03:42:02.736 INFO:teuthology.orchestra.run.smithi113.stdout:72 files removed 2023-11-09T03:42:02.763 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-09T03:42:02.790 INFO:teuthology.orchestra.run.smithi076.stdout:72 files removed 2023-11-09T03:42:02.817 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-09T03:42:03.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph packages for x86_64 109 kB/s | 78 kB 00:00 2023-11-09T03:42:03.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph packages for x86_64 99 kB/s | 78 kB 00:00 2023-11-09T03:42:04.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph noarch packages 16 kB/s | 9.4 kB 00:00 2023-11-09T03:42:04.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph noarch packages 15 kB/s | 9.4 kB 00:00 2023-11-09T03:42:04.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2023-11-09T03:42:04.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2023-11-09T03:42:05.405 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS-9 - AppStream 16 MB/s | 6.3 MB 00:00 2023-11-09T03:42:05.425 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS-9 - AppStream 14 MB/s | 6.3 MB 00:00 2023-11-09T03:42:06.909 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS-9 - BaseOS 4.7 MB/s | 2.0 MB 00:00 2023-11-09T03:42:07.131 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS-9 - BaseOS 3.0 MB/s | 2.0 MB 00:00 2023-11-09T03:42:07.658 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - BaseOS 20 MB/s | 7.9 MB 00:00 2023-11-09T03:42:10.340 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - BaseOS 2.7 MB/s | 7.9 MB 00:02 2023-11-09T03:42:10.442 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - AppStream 11 MB/s | 18 MB 00:01 2023-11-09T03:42:13.239 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - AppStream 9.9 MB/s | 18 MB 00:01 2023-11-09T03:42:14.660 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - CRB 7.3 MB/s | 5.8 MB 00:00 2023-11-09T03:42:16.502 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 15 kB 00:00 2023-11-09T03:42:18.082 INFO:teuthology.orchestra.run.smithi113.stdout:Extra Packages for Enterprise Linux 14 MB/s | 19 MB 00:01 2023-11-09T03:42:19.215 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - CRB 2.2 MB/s | 5.8 MB 00:02 2023-11-09T03:42:21.049 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 15 kB 00:00 2023-11-09T03:42:22.762 INFO:teuthology.orchestra.run.smithi113.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2023-11-09T03:42:23.042 INFO:teuthology.orchestra.run.smithi076.stdout:Extra Packages for Enterprise Linux 10 MB/s | 19 MB 00:01 2023-11-09T03:42:25.356 INFO:teuthology.orchestra.run.smithi113.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-09T03:42:25.356 INFO:teuthology.orchestra.run.smithi113.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-09T03:42:25.367 INFO:teuthology.orchestra.run.smithi113.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-11-09T03:42:25.368 INFO:teuthology.orchestra.run.smithi113.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-11-09T03:42:25.369 INFO:teuthology.orchestra.run.smithi113.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-11-09T03:42:25.370 INFO:teuthology.orchestra.run.smithi113.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-11-09T03:42:25.372 INFO:teuthology.orchestra.run.smithi113.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-11-09T03:42:25.374 INFO:teuthology.orchestra.run.smithi113.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-11-09T03:42:25.375 INFO:teuthology.orchestra.run.smithi113.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-11-09T03:42:25.444 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2023-11-09T03:42:25.454 INFO:teuthology.orchestra.run.smithi113.stdout:============================================================================================ 2023-11-09T03:42:25.454 INFO:teuthology.orchestra.run.smithi113.stdout: Package Arch Version Repository Size 2023-11-09T03:42:25.455 INFO:teuthology.orchestra.run.smithi113.stdout:============================================================================================ 2023-11-09T03:42:25.455 INFO:teuthology.orchestra.run.smithi113.stdout:Installing: 2023-11-09T03:42:25.455 INFO:teuthology.orchestra.run.smithi113.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-11-09T03:42:25.455 INFO:teuthology.orchestra.run.smithi113.stdout: ceph x86_64 2:18.0.0-7035.g7532011a.el9 ceph 6.5 k 2023-11-09T03:42:25.455 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-base x86_64 2:18.0.0-7035.g7532011a.el9 ceph 5.2 M 2023-11-09T03:42:25.456 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-fuse x86_64 2:18.0.0-7035.g7532011a.el9 ceph 859 k 2023-11-09T03:42:25.456 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7035.g7532011a.el9 ceph 140 k 2023-11-09T03:42:25.456 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr x86_64 2:18.0.0-7035.g7532011a.el9 ceph 1.5 M 2023-11-09T03:42:25.456 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 132 k 2023-11-09T03:42:25.456 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 1.8 M 2023-11-09T03:42:25.457 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 7.4 M 2023-11-09T03:42:25.457 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-rook noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 48 k 2023-11-09T03:42:25.457 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-radosgw x86_64 2:18.0.0-7035.g7532011a.el9 ceph 8.7 M 2023-11-09T03:42:25.457 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-test x86_64 2:18.0.0-7035.g7532011a.el9 ceph 41 M 2023-11-09T03:42:25.457 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-volume noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 262 k 2023-11-09T03:42:25.458 INFO:teuthology.orchestra.run.smithi113.stdout: cephadm noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 270 k 2023-11-09T03:42:25.458 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-mirror x86_64 2:18.0.0-7035.g7532011a.el9 ceph 217 k 2023-11-09T03:42:25.458 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-top noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 18 k 2023-11-09T03:42:25.458 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 appstream 22 k 2023-11-09T03:42:25.458 INFO:teuthology.orchestra.run.smithi113.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-11-09T03:42:25.459 INFO:teuthology.orchestra.run.smithi113.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-11-09T03:42:25.459 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs-devel x86_64 2:18.0.0-7035.g7532011a.el9 ceph 31 k 2023-11-09T03:42:25.459 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs2 x86_64 2:18.0.0-7035.g7532011a.el9 ceph 718 k 2023-11-09T03:42:25.459 INFO:teuthology.orchestra.run.smithi113.stdout: librados-devel x86_64 2:18.0.0-7035.g7532011a.el9 ceph 126 k 2023-11-09T03:42:25.459 INFO:teuthology.orchestra.run.smithi113.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-11-09T03:42:25.460 INFO:teuthology.orchestra.run.smithi113.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2023-11-09T03:42:25.460 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-devel x86_64 1:3.0.7-24.el9 appstream 4.1 M 2023-11-09T03:42:25.460 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-11-09T03:42:25.460 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-11-09T03:42:25.460 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-11-09T03:42:25.461 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cephfs x86_64 2:18.0.0-7035.g7532011a.el9 ceph 162 k 2023-11-09T03:42:25.461 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rados x86_64 2:18.0.0-7035.g7532011a.el9 ceph 320 k 2023-11-09T03:42:25.461 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rbd x86_64 2:18.0.0-7035.g7532011a.el9 ceph 299 k 2023-11-09T03:42:25.461 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rgw x86_64 2:18.0.0-7035.g7532011a.el9 ceph 99 k 2023-11-09T03:42:25.461 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-fuse x86_64 2:18.0.0-7035.g7532011a.el9 ceph 84 k 2023-11-09T03:42:25.462 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-mirror x86_64 2:18.0.0-7035.g7532011a.el9 ceph 3.0 M 2023-11-09T03:42:25.462 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-nbd x86_64 2:18.0.0-7035.g7532011a.el9 ceph 169 k 2023-11-09T03:42:25.462 INFO:teuthology.orchestra.run.smithi113.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 appstream 86 k 2023-11-09T03:42:25.462 INFO:teuthology.orchestra.run.smithi113.stdout:Upgrading: 2023-11-09T03:42:25.462 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager x86_64 1:1.45.6-1.el9 CentOS-BaseOS 2.2 M 2023-11-09T03:42:25.463 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-libnm x86_64 1:1.45.6-1.el9 CentOS-BaseOS 1.8 M 2023-11-09T03:42:25.463 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-team x86_64 1:1.45.6-1.el9 CentOS-BaseOS 42 k 2023-11-09T03:42:25.463 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-tui x86_64 1:1.45.6-1.el9 CentOS-BaseOS 245 k 2023-11-09T03:42:25.463 INFO:teuthology.orchestra.run.smithi113.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-11-09T03:42:25.463 INFO:teuthology.orchestra.run.smithi113.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-11-09T03:42:25.464 INFO:teuthology.orchestra.run.smithi113.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-11-09T03:42:25.464 INFO:teuthology.orchestra.run.smithi113.stdout: librados2 x86_64 2:18.0.0-7035.g7532011a.el9 ceph 3.3 M 2023-11-09T03:42:25.464 INFO:teuthology.orchestra.run.smithi113.stdout: librbd1 x86_64 2:18.0.0-7035.g7532011a.el9 ceph 3.0 M 2023-11-09T03:42:25.464 INFO:teuthology.orchestra.run.smithi113.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-11-09T03:42:25.465 INFO:teuthology.orchestra.run.smithi113.stdout:Installing dependencies: 2023-11-09T03:42:25.465 INFO:teuthology.orchestra.run.smithi113.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2023-11-09T03:42:25.465 INFO:teuthology.orchestra.run.smithi113.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-11-09T03:42:25.465 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-common x86_64 2:18.0.0-7035.g7532011a.el9 ceph 19 M 2023-11-09T03:42:25.465 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 23 k 2023-11-09T03:42:25.465 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mds x86_64 2:18.0.0-7035.g7532011a.el9 ceph 2.1 M 2023-11-09T03:42:25.466 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 243 k 2023-11-09T03:42:25.466 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mon x86_64 2:18.0.0-7035.g7532011a.el9 ceph 4.4 M 2023-11-09T03:42:25.466 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-osd x86_64 2:18.0.0-7035.g7532011a.el9 ceph 18 M 2023-11-09T03:42:25.466 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 15 k 2023-11-09T03:42:25.466 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-selinux x86_64 2:18.0.0-7035.g7532011a.el9 ceph 24 k 2023-11-09T03:42:25.467 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-11-09T03:42:25.467 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-11-09T03:42:25.467 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-11-09T03:42:25.467 INFO:teuthology.orchestra.run.smithi113.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-11-09T03:42:25.467 INFO:teuthology.orchestra.run.smithi113.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-11-09T03:42:25.468 INFO:teuthology.orchestra.run.smithi113.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-11-09T03:42:25.468 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-11-09T03:42:25.468 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-11-09T03:42:25.468 INFO:teuthology.orchestra.run.smithi113.stdout: libcephsqlite x86_64 2:18.0.0-7035.g7532011a.el9 ceph 163 k 2023-11-09T03:42:25.469 INFO:teuthology.orchestra.run.smithi113.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-11-09T03:42:25.469 INFO:teuthology.orchestra.run.smithi113.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-11-09T03:42:25.469 INFO:teuthology.orchestra.run.smithi113.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-11-09T03:42:25.469 INFO:teuthology.orchestra.run.smithi113.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-11-09T03:42:25.469 INFO:teuthology.orchestra.run.smithi113.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-11-09T03:42:25.470 INFO:teuthology.orchestra.run.smithi113.stdout: libradosstriper1 x86_64 2:18.0.0-7035.g7532011a.el9 ceph 476 k 2023-11-09T03:42:25.470 INFO:teuthology.orchestra.run.smithi113.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-11-09T03:42:25.470 INFO:teuthology.orchestra.run.smithi113.stdout: librgw2 x86_64 2:18.0.0-7035.g7532011a.el9 ceph 4.8 M 2023-11-09T03:42:25.470 INFO:teuthology.orchestra.run.smithi113.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-11-09T03:42:25.470 INFO:teuthology.orchestra.run.smithi113.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-11-09T03:42:25.471 INFO:teuthology.orchestra.run.smithi113.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-11-09T03:42:25.471 INFO:teuthology.orchestra.run.smithi113.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-11-09T03:42:25.471 INFO:teuthology.orchestra.run.smithi113.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-11-09T03:42:25.471 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-11-09T03:42:25.471 INFO:teuthology.orchestra.run.smithi113.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-11-09T03:42:25.472 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-11-09T03:42:25.472 INFO:teuthology.orchestra.run.smithi113.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-11-09T03:42:25.472 INFO:teuthology.orchestra.run.smithi113.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-11-09T03:42:25.472 INFO:teuthology.orchestra.run.smithi113.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-11-09T03:42:25.472 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-11-09T03:42:25.473 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-argparse x86_64 2:18.0.0-7035.g7532011a.el9 ceph 45 k 2023-11-09T03:42:25.473 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-common x86_64 2:18.0.0-7035.g7532011a.el9 ceph 127 k 2023-11-09T03:42:25.473 INFO:teuthology.orchestra.run.smithi113.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-11-09T03:42:25.473 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-11-09T03:42:25.473 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-11-09T03:42:25.474 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-11-09T03:42:25.474 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-11-09T03:42:25.474 INFO:teuthology.orchestra.run.smithi113.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-11-09T03:42:25.474 INFO:teuthology.orchestra.run.smithi113.stdout: python3-google-auth noarch 1:2.23.3-1.el9 epel 197 k 2023-11-09T03:42:25.474 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-11-09T03:42:25.475 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-11-09T03:42:25.475 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-11-09T03:42:25.475 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-11-09T03:42:25.475 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-11-09T03:42:25.475 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-11-09T03:42:25.476 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-11-09T03:42:25.476 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-11-09T03:42:25.476 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-11-09T03:42:25.476 INFO:teuthology.orchestra.run.smithi113.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-11-09T03:42:25.476 INFO:teuthology.orchestra.run.smithi113.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-11-09T03:42:25.476 INFO:teuthology.orchestra.run.smithi113.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-11-09T03:42:25.477 INFO:teuthology.orchestra.run.smithi113.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-11-09T03:42:25.477 INFO:teuthology.orchestra.run.smithi113.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-11-09T03:42:25.477 INFO:teuthology.orchestra.run.smithi113.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-11-09T03:42:25.477 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-11-09T03:42:25.477 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-11-09T03:42:25.477 INFO:teuthology.orchestra.run.smithi113.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-11-09T03:42:25.478 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-11-09T03:42:25.478 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-11-09T03:42:25.478 INFO:teuthology.orchestra.run.smithi113.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-11-09T03:42:25.478 INFO:teuthology.orchestra.run.smithi113.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-11-09T03:42:25.478 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-11-09T03:42:25.479 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-11-09T03:42:25.479 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-11-09T03:42:25.479 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-11-09T03:42:25.479 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-11-09T03:42:25.479 INFO:teuthology.orchestra.run.smithi113.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-11-09T03:42:25.480 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-11-09T03:42:25.480 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-11-09T03:42:25.480 INFO:teuthology.orchestra.run.smithi113.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-11-09T03:42:25.480 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-11-09T03:42:25.480 INFO:teuthology.orchestra.run.smithi113.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-11-09T03:42:25.480 INFO:teuthology.orchestra.run.smithi113.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-11-09T03:42:25.481 INFO:teuthology.orchestra.run.smithi113.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-11-09T03:42:25.481 INFO:teuthology.orchestra.run.smithi113.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-11-09T03:42:25.481 INFO:teuthology.orchestra.run.smithi113.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-11-09T03:42:25.481 INFO:teuthology.orchestra.run.smithi113.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-11-09T03:42:25.481 INFO:teuthology.orchestra.run.smithi113.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-11-09T03:42:25.482 INFO:teuthology.orchestra.run.smithi113.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-11-09T03:42:25.482 INFO:teuthology.orchestra.run.smithi113.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-11-09T03:42:25.482 INFO:teuthology.orchestra.run.smithi113.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-11-09T03:42:25.482 INFO:teuthology.orchestra.run.smithi113.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-11-09T03:42:25.482 INFO:teuthology.orchestra.run.smithi113.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-11-09T03:42:25.483 INFO:teuthology.orchestra.run.smithi113.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-11-09T03:42:25.483 INFO:teuthology.orchestra.run.smithi113.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-11-09T03:42:25.483 INFO:teuthology.orchestra.run.smithi113.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2023-11-09T03:42:25.483 INFO:teuthology.orchestra.run.smithi113.stdout:Installing weak dependencies: 2023-11-09T03:42:25.483 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-11-09T03:42:25.484 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:42:25.484 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2023-11-09T03:42:25.484 INFO:teuthology.orchestra.run.smithi113.stdout:============================================================================================ 2023-11-09T03:42:25.484 INFO:teuthology.orchestra.run.smithi113.stdout:Install 132 Packages 2023-11-09T03:42:25.484 INFO:teuthology.orchestra.run.smithi113.stdout:Upgrade 10 Packages 2023-11-09T03:42:25.485 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:42:25.485 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 200 M 2023-11-09T03:42:25.485 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2023-11-09T03:42:25.980 INFO:teuthology.orchestra.run.smithi113.stdout:(1/142): ceph-18.0.0-7035.g7532011a.el9.x86_64. 43 kB/s | 6.5 kB 00:00 2023-11-09T03:42:26.497 INFO:teuthology.orchestra.run.smithi113.stdout:(2/142): ceph-fuse-18.0.0-7035.g7532011a.el9.x8 1.6 MB/s | 859 kB 00:00 2023-11-09T03:42:26.564 INFO:teuthology.orchestra.run.smithi113.stdout:(3/142): ceph-immutable-object-cache-18.0.0-703 2.1 MB/s | 140 kB 00:00 2023-11-09T03:42:26.906 INFO:teuthology.orchestra.run.smithi113.stdout:(4/142): ceph-mds-18.0.0-7035.g7532011a.el9.x86 6.1 MB/s | 2.1 MB 00:00 2023-11-09T03:42:27.098 INFO:teuthology.orchestra.run.smithi113.stdout:(5/142): ceph-mgr-18.0.0-7035.g7532011a.el9.x86 7.6 MB/s | 1.5 MB 00:00 2023-11-09T03:42:27.431 INFO:teuthology.orchestra.run.smithi113.stdout:(6/142): ceph-base-18.0.0-7035.g7532011a.el9.x8 3.3 MB/s | 5.2 MB 00:01 2023-11-09T03:42:27.600 INFO:teuthology.orchestra.run.smithi076.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2023-11-09T03:42:27.673 INFO:teuthology.orchestra.run.smithi113.stdout:(7/142): ceph-mon-18.0.0-7035.g7532011a.el9.x86 7.6 MB/s | 4.4 MB 00:00 2023-11-09T03:42:28.623 INFO:teuthology.orchestra.run.smithi113.stdout:(8/142): ceph-radosgw-18.0.0-7035.g7532011a.el9 9.2 MB/s | 8.7 MB 00:00 2023-11-09T03:42:28.690 INFO:teuthology.orchestra.run.smithi113.stdout:(9/142): ceph-selinux-18.0.0-7035.g7532011a.el9 362 kB/s | 24 kB 00:00 2023-11-09T03:42:30.178 INFO:teuthology.orchestra.run.smithi076.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-09T03:42:30.179 INFO:teuthology.orchestra.run.smithi076.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-09T03:42:30.192 INFO:teuthology.orchestra.run.smithi076.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-11-09T03:42:30.193 INFO:teuthology.orchestra.run.smithi076.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-11-09T03:42:30.193 INFO:teuthology.orchestra.run.smithi076.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-11-09T03:42:30.194 INFO:teuthology.orchestra.run.smithi076.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-11-09T03:42:30.196 INFO:teuthology.orchestra.run.smithi076.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-11-09T03:42:30.201 INFO:teuthology.orchestra.run.smithi076.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-11-09T03:42:30.202 INFO:teuthology.orchestra.run.smithi076.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-11-09T03:42:30.269 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2023-11-09T03:42:30.288 INFO:teuthology.orchestra.run.smithi076.stdout:============================================================================================ 2023-11-09T03:42:30.288 INFO:teuthology.orchestra.run.smithi076.stdout: Package Arch Version Repository Size 2023-11-09T03:42:30.288 INFO:teuthology.orchestra.run.smithi076.stdout:============================================================================================ 2023-11-09T03:42:30.289 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2023-11-09T03:42:30.289 INFO:teuthology.orchestra.run.smithi076.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-11-09T03:42:30.289 INFO:teuthology.orchestra.run.smithi076.stdout: ceph x86_64 2:18.0.0-7035.g7532011a.el9 ceph 6.5 k 2023-11-09T03:42:30.289 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-base x86_64 2:18.0.0-7035.g7532011a.el9 ceph 5.2 M 2023-11-09T03:42:30.289 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-fuse x86_64 2:18.0.0-7035.g7532011a.el9 ceph 859 k 2023-11-09T03:42:30.290 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7035.g7532011a.el9 ceph 140 k 2023-11-09T03:42:30.290 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr x86_64 2:18.0.0-7035.g7532011a.el9 ceph 1.5 M 2023-11-09T03:42:30.290 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 132 k 2023-11-09T03:42:30.290 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 1.8 M 2023-11-09T03:42:30.290 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 7.4 M 2023-11-09T03:42:30.291 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-rook noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 48 k 2023-11-09T03:42:30.291 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-radosgw x86_64 2:18.0.0-7035.g7532011a.el9 ceph 8.7 M 2023-11-09T03:42:30.291 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-test x86_64 2:18.0.0-7035.g7532011a.el9 ceph 41 M 2023-11-09T03:42:30.291 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-volume noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 262 k 2023-11-09T03:42:30.291 INFO:teuthology.orchestra.run.smithi076.stdout: cephadm noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 270 k 2023-11-09T03:42:30.292 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-mirror x86_64 2:18.0.0-7035.g7532011a.el9 ceph 217 k 2023-11-09T03:42:30.292 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-top noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 18 k 2023-11-09T03:42:30.292 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 appstream 22 k 2023-11-09T03:42:30.292 INFO:teuthology.orchestra.run.smithi076.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-11-09T03:42:30.292 INFO:teuthology.orchestra.run.smithi076.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-11-09T03:42:30.293 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs-devel x86_64 2:18.0.0-7035.g7532011a.el9 ceph 31 k 2023-11-09T03:42:30.293 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs2 x86_64 2:18.0.0-7035.g7532011a.el9 ceph 718 k 2023-11-09T03:42:30.293 INFO:teuthology.orchestra.run.smithi076.stdout: librados-devel x86_64 2:18.0.0-7035.g7532011a.el9 ceph 126 k 2023-11-09T03:42:30.293 INFO:teuthology.orchestra.run.smithi076.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-11-09T03:42:30.293 INFO:teuthology.orchestra.run.smithi076.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2023-11-09T03:42:30.294 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-devel x86_64 1:3.0.7-24.el9 appstream 4.1 M 2023-11-09T03:42:30.294 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-11-09T03:42:30.294 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-11-09T03:42:30.294 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-11-09T03:42:30.294 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cephfs x86_64 2:18.0.0-7035.g7532011a.el9 ceph 162 k 2023-11-09T03:42:30.295 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rados x86_64 2:18.0.0-7035.g7532011a.el9 ceph 320 k 2023-11-09T03:42:30.295 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rbd x86_64 2:18.0.0-7035.g7532011a.el9 ceph 299 k 2023-11-09T03:42:30.295 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rgw x86_64 2:18.0.0-7035.g7532011a.el9 ceph 99 k 2023-11-09T03:42:30.295 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-fuse x86_64 2:18.0.0-7035.g7532011a.el9 ceph 84 k 2023-11-09T03:42:30.295 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-mirror x86_64 2:18.0.0-7035.g7532011a.el9 ceph 3.0 M 2023-11-09T03:42:30.295 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-nbd x86_64 2:18.0.0-7035.g7532011a.el9 ceph 169 k 2023-11-09T03:42:30.296 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 appstream 86 k 2023-11-09T03:42:30.296 INFO:teuthology.orchestra.run.smithi076.stdout:Upgrading: 2023-11-09T03:42:30.296 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager x86_64 1:1.45.6-1.el9 CentOS-BaseOS 2.2 M 2023-11-09T03:42:30.296 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-libnm x86_64 1:1.45.6-1.el9 CentOS-BaseOS 1.8 M 2023-11-09T03:42:30.296 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-team x86_64 1:1.45.6-1.el9 CentOS-BaseOS 42 k 2023-11-09T03:42:30.297 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-tui x86_64 1:1.45.6-1.el9 CentOS-BaseOS 245 k 2023-11-09T03:42:30.297 INFO:teuthology.orchestra.run.smithi076.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-11-09T03:42:30.297 INFO:teuthology.orchestra.run.smithi076.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-11-09T03:42:30.297 INFO:teuthology.orchestra.run.smithi076.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-11-09T03:42:30.297 INFO:teuthology.orchestra.run.smithi076.stdout: librados2 x86_64 2:18.0.0-7035.g7532011a.el9 ceph 3.3 M 2023-11-09T03:42:30.298 INFO:teuthology.orchestra.run.smithi076.stdout: librbd1 x86_64 2:18.0.0-7035.g7532011a.el9 ceph 3.0 M 2023-11-09T03:42:30.298 INFO:teuthology.orchestra.run.smithi076.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-11-09T03:42:30.298 INFO:teuthology.orchestra.run.smithi076.stdout:Installing dependencies: 2023-11-09T03:42:30.298 INFO:teuthology.orchestra.run.smithi076.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2023-11-09T03:42:30.298 INFO:teuthology.orchestra.run.smithi076.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-11-09T03:42:30.299 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-common x86_64 2:18.0.0-7035.g7532011a.el9 ceph 19 M 2023-11-09T03:42:30.299 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 23 k 2023-11-09T03:42:30.299 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mds x86_64 2:18.0.0-7035.g7532011a.el9 ceph 2.1 M 2023-11-09T03:42:30.299 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 243 k 2023-11-09T03:42:30.300 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mon x86_64 2:18.0.0-7035.g7532011a.el9 ceph 4.4 M 2023-11-09T03:42:30.300 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-osd x86_64 2:18.0.0-7035.g7532011a.el9 ceph 18 M 2023-11-09T03:42:30.300 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 15 k 2023-11-09T03:42:30.300 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-selinux x86_64 2:18.0.0-7035.g7532011a.el9 ceph 24 k 2023-11-09T03:42:30.300 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-11-09T03:42:30.301 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-11-09T03:42:30.301 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-11-09T03:42:30.301 INFO:teuthology.orchestra.run.smithi076.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-11-09T03:42:30.301 INFO:teuthology.orchestra.run.smithi076.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-11-09T03:42:30.301 INFO:teuthology.orchestra.run.smithi076.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-11-09T03:42:30.302 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-11-09T03:42:30.302 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-11-09T03:42:30.302 INFO:teuthology.orchestra.run.smithi076.stdout: libcephsqlite x86_64 2:18.0.0-7035.g7532011a.el9 ceph 163 k 2023-11-09T03:42:30.302 INFO:teuthology.orchestra.run.smithi076.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-11-09T03:42:30.302 INFO:teuthology.orchestra.run.smithi076.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-11-09T03:42:30.303 INFO:teuthology.orchestra.run.smithi076.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-11-09T03:42:30.303 INFO:teuthology.orchestra.run.smithi076.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-11-09T03:42:30.303 INFO:teuthology.orchestra.run.smithi076.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-11-09T03:42:30.303 INFO:teuthology.orchestra.run.smithi076.stdout: libradosstriper1 x86_64 2:18.0.0-7035.g7532011a.el9 ceph 476 k 2023-11-09T03:42:30.303 INFO:teuthology.orchestra.run.smithi076.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-11-09T03:42:30.304 INFO:teuthology.orchestra.run.smithi076.stdout: librgw2 x86_64 2:18.0.0-7035.g7532011a.el9 ceph 4.8 M 2023-11-09T03:42:30.304 INFO:teuthology.orchestra.run.smithi076.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-11-09T03:42:30.304 INFO:teuthology.orchestra.run.smithi076.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-11-09T03:42:30.304 INFO:teuthology.orchestra.run.smithi076.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-11-09T03:42:30.304 INFO:teuthology.orchestra.run.smithi076.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-11-09T03:42:30.305 INFO:teuthology.orchestra.run.smithi076.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-11-09T03:42:30.305 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-11-09T03:42:30.305 INFO:teuthology.orchestra.run.smithi076.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-11-09T03:42:30.305 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-11-09T03:42:30.305 INFO:teuthology.orchestra.run.smithi076.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-11-09T03:42:30.306 INFO:teuthology.orchestra.run.smithi076.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-11-09T03:42:30.306 INFO:teuthology.orchestra.run.smithi076.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-11-09T03:42:30.306 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-11-09T03:42:30.306 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-argparse x86_64 2:18.0.0-7035.g7532011a.el9 ceph 45 k 2023-11-09T03:42:30.306 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-common x86_64 2:18.0.0-7035.g7532011a.el9 ceph 127 k 2023-11-09T03:42:30.307 INFO:teuthology.orchestra.run.smithi076.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-11-09T03:42:30.307 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-11-09T03:42:30.307 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-11-09T03:42:30.307 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-11-09T03:42:30.307 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-11-09T03:42:30.308 INFO:teuthology.orchestra.run.smithi076.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-11-09T03:42:30.308 INFO:teuthology.orchestra.run.smithi076.stdout: python3-google-auth noarch 1:2.23.3-1.el9 epel 197 k 2023-11-09T03:42:30.308 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-11-09T03:42:30.308 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-11-09T03:42:30.308 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-11-09T03:42:30.309 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-11-09T03:42:30.309 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-11-09T03:42:30.309 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-11-09T03:42:30.309 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-11-09T03:42:30.309 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-11-09T03:42:30.310 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-11-09T03:42:30.310 INFO:teuthology.orchestra.run.smithi076.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-11-09T03:42:30.310 INFO:teuthology.orchestra.run.smithi076.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-11-09T03:42:30.310 INFO:teuthology.orchestra.run.smithi076.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-11-09T03:42:30.310 INFO:teuthology.orchestra.run.smithi076.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-11-09T03:42:30.311 INFO:teuthology.orchestra.run.smithi076.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-11-09T03:42:30.311 INFO:teuthology.orchestra.run.smithi076.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-11-09T03:42:30.311 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-11-09T03:42:30.311 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-11-09T03:42:30.311 INFO:teuthology.orchestra.run.smithi076.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-11-09T03:42:30.312 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-11-09T03:42:30.312 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-11-09T03:42:30.312 INFO:teuthology.orchestra.run.smithi076.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-11-09T03:42:30.312 INFO:teuthology.orchestra.run.smithi076.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-11-09T03:42:30.312 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-11-09T03:42:30.313 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-11-09T03:42:30.313 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-11-09T03:42:30.313 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-11-09T03:42:30.313 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-11-09T03:42:30.313 INFO:teuthology.orchestra.run.smithi076.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-11-09T03:42:30.313 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-11-09T03:42:30.314 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-11-09T03:42:30.314 INFO:teuthology.orchestra.run.smithi076.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-11-09T03:42:30.314 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-11-09T03:42:30.314 INFO:teuthology.orchestra.run.smithi076.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-11-09T03:42:30.315 INFO:teuthology.orchestra.run.smithi076.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-11-09T03:42:30.315 INFO:teuthology.orchestra.run.smithi076.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-11-09T03:42:30.315 INFO:teuthology.orchestra.run.smithi076.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-11-09T03:42:30.315 INFO:teuthology.orchestra.run.smithi076.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-11-09T03:42:30.315 INFO:teuthology.orchestra.run.smithi076.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-11-09T03:42:30.315 INFO:teuthology.orchestra.run.smithi076.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-11-09T03:42:30.316 INFO:teuthology.orchestra.run.smithi076.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-11-09T03:42:30.316 INFO:teuthology.orchestra.run.smithi076.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-11-09T03:42:30.316 INFO:teuthology.orchestra.run.smithi076.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-11-09T03:42:30.316 INFO:teuthology.orchestra.run.smithi076.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-11-09T03:42:30.316 INFO:teuthology.orchestra.run.smithi076.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-11-09T03:42:30.317 INFO:teuthology.orchestra.run.smithi076.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-11-09T03:42:30.317 INFO:teuthology.orchestra.run.smithi076.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-11-09T03:42:30.317 INFO:teuthology.orchestra.run.smithi076.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2023-11-09T03:42:30.317 INFO:teuthology.orchestra.run.smithi076.stdout:Installing weak dependencies: 2023-11-09T03:42:30.317 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-11-09T03:42:30.318 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:42:30.318 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2023-11-09T03:42:30.318 INFO:teuthology.orchestra.run.smithi076.stdout:============================================================================================ 2023-11-09T03:42:30.318 INFO:teuthology.orchestra.run.smithi076.stdout:Install 132 Packages 2023-11-09T03:42:30.318 INFO:teuthology.orchestra.run.smithi076.stdout:Upgrade 10 Packages 2023-11-09T03:42:30.319 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:42:30.319 INFO:teuthology.orchestra.run.smithi113.stdout:(10/142): ceph-common-18.0.0-7035.g7532011a.el9 4.3 MB/s | 19 MB 00:04 2023-11-09T03:42:30.320 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 200 M 2023-11-09T03:42:30.320 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2023-11-09T03:42:30.399 INFO:teuthology.orchestra.run.smithi113.stdout:(11/142): cephfs-mirror-18.0.0-7035.g7532011a.e 2.0 MB/s | 217 kB 00:00 2023-11-09T03:42:30.574 INFO:teuthology.orchestra.run.smithi113.stdout:(12/142): ceph-osd-18.0.0-7035.g7532011a.el9.x8 5.6 MB/s | 18 MB 00:03 2023-11-09T03:42:30.599 INFO:teuthology.orchestra.run.smithi113.stdout:(13/142): libcephfs-devel-18.0.0-7035.g7532011a 156 kB/s | 31 kB 00:00 2023-11-09T03:42:30.808 INFO:teuthology.orchestra.run.smithi113.stdout:(14/142): libcephsqlite-18.0.0-7035.g7532011a.e 780 kB/s | 163 kB 00:00 2023-11-09T03:42:30.833 INFO:teuthology.orchestra.run.smithi113.stdout:(15/142): libcephfs2-18.0.0-7035.g7532011a.el9. 2.7 MB/s | 718 kB 00:00 2023-11-09T03:42:30.892 INFO:teuthology.orchestra.run.smithi113.stdout:(16/142): librados-devel-18.0.0-7035.g7532011a. 1.5 MB/s | 126 kB 00:00 2023-11-09T03:42:30.893 INFO:teuthology.orchestra.run.smithi076.stdout:(1/142): ceph-18.0.0-7035.g7532011a.el9.x86_64. 26 kB/s | 6.5 kB 00:00 2023-11-09T03:42:31.034 INFO:teuthology.orchestra.run.smithi113.stdout:(17/142): libradosstriper1-18.0.0-7035.g7532011 2.3 MB/s | 476 kB 00:00 2023-11-09T03:42:31.309 INFO:teuthology.orchestra.run.smithi076.stdout:(2/142): ceph-fuse-18.0.0-7035.g7532011a.el9.x8 2.0 MB/s | 859 kB 00:00 2023-11-09T03:42:31.377 INFO:teuthology.orchestra.run.smithi076.stdout:(3/142): ceph-immutable-object-cache-18.0.0-703 2.1 MB/s | 140 kB 00:00 2023-11-09T03:42:31.751 INFO:teuthology.orchestra.run.smithi113.stdout:(18/142): python3-ceph-argparse-18.0.0-7035.g75 63 kB/s | 45 kB 00:00 2023-11-09T03:42:31.842 INFO:teuthology.orchestra.run.smithi113.stdout:(19/142): librgw2-18.0.0-7035.g7532011a.el9.x86 5.0 MB/s | 4.8 MB 00:00 2023-11-09T03:42:31.852 INFO:teuthology.orchestra.run.smithi076.stdout:(4/142): ceph-mds-18.0.0-7035.g7532011a.el9.x86 4.4 MB/s | 2.1 MB 00:00 2023-11-09T03:42:31.868 INFO:teuthology.orchestra.run.smithi113.stdout:(20/142): python3-ceph-common-18.0.0-7035.g7532 1.1 MB/s | 127 kB 00:00 2023-11-09T03:42:31.910 INFO:teuthology.orchestra.run.smithi113.stdout:(21/142): python3-cephfs-18.0.0-7035.g7532011a. 2.4 MB/s | 162 kB 00:00 2023-11-09T03:42:32.068 INFO:teuthology.orchestra.run.smithi113.stdout:(22/142): python3-rados-18.0.0-7035.g7532011a.e 1.6 MB/s | 320 kB 00:00 2023-11-09T03:42:32.102 INFO:teuthology.orchestra.run.smithi113.stdout:(23/142): python3-rbd-18.0.0-7035.g7532011a.el9 1.5 MB/s | 299 kB 00:00 2023-11-09T03:42:32.186 INFO:teuthology.orchestra.run.smithi076.stdout:(5/142): ceph-mgr-18.0.0-7035.g7532011a.el9.x86 4.4 MB/s | 1.5 MB 00:00 2023-11-09T03:42:32.186 INFO:teuthology.orchestra.run.smithi113.stdout:(24/142): python3-rgw-18.0.0-7035.g7532011a.el9 850 kB/s | 99 kB 00:00 2023-11-09T03:42:32.211 INFO:teuthology.orchestra.run.smithi113.stdout:(25/142): rbd-fuse-18.0.0-7035.g7532011a.el9.x8 776 kB/s | 84 kB 00:00 2023-11-09T03:42:32.561 INFO:teuthology.orchestra.run.smithi113.stdout:(26/142): rbd-nbd-18.0.0-7035.g7532011a.el9.x86 484 kB/s | 169 kB 00:00 2023-11-09T03:42:32.661 INFO:teuthology.orchestra.run.smithi113.stdout:(27/142): ceph-grafana-dashboards-18.0.0-7035.g 232 kB/s | 23 kB 00:00 2023-11-09T03:42:32.787 INFO:teuthology.orchestra.run.smithi113.stdout:(28/142): ceph-mgr-cephadm-18.0.0-7035.g7532011 1.0 MB/s | 132 kB 00:00 2023-11-09T03:42:33.179 INFO:teuthology.orchestra.run.smithi113.stdout:(29/142): rbd-mirror-18.0.0-7035.g7532011a.el9. 3.0 MB/s | 3.0 MB 00:00 2023-11-09T03:42:33.269 INFO:teuthology.orchestra.run.smithi076.stdout:(6/142): ceph-base-18.0.0-7035.g7532011a.el9.x8 2.0 MB/s | 5.2 MB 00:02 2023-11-09T03:42:33.287 INFO:teuthology.orchestra.run.smithi113.stdout:(30/142): ceph-mgr-dashboard-18.0.0-7035.g75320 3.5 MB/s | 1.8 MB 00:00 2023-11-09T03:42:33.413 INFO:teuthology.orchestra.run.smithi113.stdout:(31/142): ceph-mgr-modules-core-18.0.0-7035.g75 1.9 MB/s | 243 kB 00:00 2023-11-09T03:42:33.488 INFO:teuthology.orchestra.run.smithi113.stdout:(32/142): ceph-mgr-rook-18.0.0-7035.g7532011a.e 637 kB/s | 48 kB 00:00 2023-11-09T03:42:33.503 INFO:teuthology.orchestra.run.smithi076.stdout:(7/142): ceph-mon-18.0.0-7035.g7532011a.el9.x86 3.3 MB/s | 4.4 MB 00:01 2023-11-09T03:42:34.438 INFO:teuthology.orchestra.run.smithi113.stdout:(33/142): ceph-prometheus-alerts-18.0.0-7035.g7 16 kB/s | 15 kB 00:00 2023-11-09T03:42:34.722 INFO:teuthology.orchestra.run.smithi113.stdout:(34/142): ceph-volume-18.0.0-7035.g7532011a.el9 926 kB/s | 262 kB 00:00 2023-11-09T03:42:34.855 INFO:teuthology.orchestra.run.smithi113.stdout:(35/142): cephadm-18.0.0-7035.g7532011a.el9.noa 2.0 MB/s | 270 kB 00:00 2023-11-09T03:42:34.956 INFO:teuthology.orchestra.run.smithi113.stdout:(36/142): cephfs-top-18.0.0-7035.g7532011a.el9. 183 kB/s | 18 kB 00:00 2023-11-09T03:42:35.081 INFO:teuthology.orchestra.run.smithi113.stdout:(37/142): bison-3.7.4-5.el9.x86_64.rpm 7.2 MB/s | 920 kB 00:00 2023-11-09T03:42:35.117 INFO:teuthology.orchestra.run.smithi113.stdout:(38/142): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2023-11-09T03:42:35.148 INFO:teuthology.orchestra.run.smithi113.stdout:(39/142): flex-2.6.4-9.el9.x86_64.rpm 9.0 MB/s | 307 kB 00:00 2023-11-09T03:42:35.182 INFO:teuthology.orchestra.run.smithi113.stdout:(40/142): flexiblas-3.0.4-8.el9.x86_64.rpm 953 kB/s | 32 kB 00:00 2023-11-09T03:42:35.257 INFO:teuthology.orchestra.run.smithi113.stdout:(41/142): flexiblas-netlib-3.0.4-8.el9.x86_64.r 40 MB/s | 3.0 MB 00:00 2023-11-09T03:42:35.291 INFO:teuthology.orchestra.run.smithi113.stdout:(42/142): flexiblas-openblas-openmp-3.0.4-8.el9 514 kB/s | 17 kB 00:00 2023-11-09T03:42:35.325 INFO:teuthology.orchestra.run.smithi113.stdout:(43/142): jq-1.6-16.el9.x86_64.rpm 5.5 MB/s | 186 kB 00:00 2023-11-09T03:42:35.358 INFO:teuthology.orchestra.run.smithi113.stdout:(44/142): libaio-devel-0.3.111-13.el9.x86_64.rp 302 kB/s | 10 kB 00:00 2023-11-09T03:42:35.392 INFO:teuthology.orchestra.run.smithi113.stdout:(45/142): libpmemobj-1.12.1-1.el9.x86_64.rpm 4.7 MB/s | 159 kB 00:00 2023-11-09T03:42:35.434 INFO:teuthology.orchestra.run.smithi113.stdout:(46/142): librdkafka-1.6.1-102.el9.x86_64.rpm 15 MB/s | 661 kB 00:00 2023-11-09T03:42:35.468 INFO:teuthology.orchestra.run.smithi113.stdout:(47/142): libtool-2.4.6-45.el9.x86_64.rpm 17 MB/s | 578 kB 00:00 2023-11-09T03:42:35.493 INFO:teuthology.orchestra.run.smithi113.stdout:(48/142): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.8 MB/s | 46 kB 00:00 2023-11-09T03:42:35.526 INFO:teuthology.orchestra.run.smithi113.stdout:(49/142): lttng-ust-2.12.0-6.el9.x86_64.rpm 8.3 MB/s | 282 kB 00:00 2023-11-09T03:42:35.560 INFO:teuthology.orchestra.run.smithi113.stdout:(50/142): oniguruma-6.9.6-1.el9.5.x86_64.rpm 6.3 MB/s | 217 kB 00:00 2023-11-09T03:42:35.585 INFO:teuthology.orchestra.run.smithi113.stdout:(51/142): openblas-0.3.21-2.el9.x86_64.rpm 1.3 MB/s | 34 kB 00:00 2023-11-09T03:42:35.669 INFO:teuthology.orchestra.run.smithi113.stdout:(52/142): openblas-openmp-0.3.21-2.el9.x86_64.r 57 MB/s | 4.7 MB 00:00 2023-11-09T03:42:35.719 INFO:teuthology.orchestra.run.smithi113.stdout:(53/142): postgresql-13.11-1.el9.x86_64.rpm 30 MB/s | 1.5 MB 00:00 2023-11-09T03:42:35.770 INFO:teuthology.orchestra.run.smithi113.stdout:(54/142): postgresql-contrib-13.11-1.el9.x86_64 16 MB/s | 811 kB 00:00 2023-11-09T03:42:35.804 INFO:teuthology.orchestra.run.smithi113.stdout:(55/142): postgresql-private-libs-13.11-1.el9.x 4.0 MB/s | 134 kB 00:00 2023-11-09T03:42:35.853 INFO:teuthology.orchestra.run.smithi076.stdout:(8/142): ceph-radosgw-18.0.0-7035.g7532011a.el9 3.7 MB/s | 8.7 MB 00:02 2023-11-09T03:42:35.903 INFO:teuthology.orchestra.run.smithi076.stdout:(9/142): ceph-selinux-18.0.0-7035.g7532011a.el9 484 kB/s | 24 kB 00:00 2023-11-09T03:42:35.904 INFO:teuthology.orchestra.run.smithi113.stdout:(56/142): postgresql-server-13.11-1.el9.x86_64. 57 MB/s | 5.7 MB 00:00 2023-11-09T03:42:35.996 INFO:teuthology.orchestra.run.smithi113.stdout:(57/142): python3-babel-2.9.1-2.el9.noarch.rpm 63 MB/s | 5.8 MB 00:00 2023-11-09T03:42:36.030 INFO:teuthology.orchestra.run.smithi113.stdout:(58/142): python3-devel-3.9.17-2.el9.x86_64.rpm 6.1 MB/s | 205 kB 00:00 2023-11-09T03:42:36.063 INFO:teuthology.orchestra.run.smithi113.stdout:(59/142): python3-jinja2-2.11.3-4.el9.noarch.rp 6.7 MB/s | 228 kB 00:00 2023-11-09T03:42:36.155 INFO:teuthology.orchestra.run.smithi113.stdout:(60/142): ceph-mgr-diskprediction-local-18.0.0- 2.5 MB/s | 7.4 MB 00:02 2023-11-09T03:42:36.180 INFO:teuthology.orchestra.run.smithi113.stdout:(61/142): python3-jsonpatch-1.21-16.el9.noarch. 209 kB/s | 24 kB 00:00 2023-11-09T03:42:36.206 INFO:teuthology.orchestra.run.smithi113.stdout:(62/142): python3-jsonpointer-2.0-4.el9.noarch. 361 kB/s | 18 kB 00:00 2023-11-09T03:42:36.231 INFO:teuthology.orchestra.run.smithi113.stdout:(63/142): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2023-11-09T03:42:36.256 INFO:teuthology.orchestra.run.smithi113.stdout:(64/142): python3-markupsafe-1.1.1-12.el9.x86_6 631 kB/s | 32 kB 00:00 2023-11-09T03:42:36.340 INFO:teuthology.orchestra.run.smithi113.stdout:(65/142): python3-numpy-1.20.1-5.el9.x86_64.rpm 44 MB/s | 4.8 MB 00:00 2023-11-09T03:42:36.366 INFO:teuthology.orchestra.run.smithi113.stdout:(66/142): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 217 kB 00:00 2023-11-09T03:42:36.391 INFO:teuthology.orchestra.run.smithi113.stdout:(67/142): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2023-11-09T03:42:36.416 INFO:teuthology.orchestra.run.smithi113.stdout:(68/142): python3-prettytable-0.7.2-27.el9.noar 827 kB/s | 41 kB 00:00 2023-11-09T03:42:36.457 INFO:teuthology.orchestra.run.smithi113.stdout:(69/142): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.0 MB/s | 132 kB 00:00 2023-11-09T03:42:36.482 INFO:teuthology.orchestra.run.smithi113.stdout:(70/142): python3-pyasn1-modules-0.4.8-6.el9.no 3.1 MB/s | 211 kB 00:00 2023-11-09T03:42:36.508 INFO:teuthology.orchestra.run.smithi113.stdout:(71/142): python3-requests-oauthlib-1.3.0-12.el 856 kB/s | 43 kB 00:00 2023-11-09T03:42:36.558 INFO:teuthology.orchestra.run.smithi113.stdout:(72/142): python3-toml-0.10.2-6.el9.noarch.rpm 817 kB/s | 39 kB 00:00 2023-11-09T03:42:36.742 INFO:teuthology.orchestra.run.smithi113.stdout:(73/142): python3-scipy-1.6.2-8.el9.x86_64.rpm 60 MB/s | 16 MB 00:00 2023-11-09T03:42:36.767 INFO:teuthology.orchestra.run.smithi113.stdout:(74/142): socat-1.7.4.1-5.el9.x86_64.rpm 1.4 MB/s | 300 kB 00:00 2023-11-09T03:42:36.792 INFO:teuthology.orchestra.run.smithi113.stdout:(75/142): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2023-11-09T03:42:36.818 INFO:teuthology.orchestra.run.smithi113.stdout:(76/142): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2023-11-09T03:42:36.843 INFO:teuthology.orchestra.run.smithi113.stdout:(77/142): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2023-11-09T03:42:36.877 INFO:teuthology.orchestra.run.smithi113.stdout:(78/142): libgfortran-11.4.1-2.3.el9.x86_64.rpm 13 MB/s | 803 kB 00:00 2023-11-09T03:42:36.902 INFO:teuthology.orchestra.run.smithi113.stdout:(79/142): libquadmath-11.4.1-2.3.el9.x86_64.rpm 3.4 MB/s | 195 kB 00:00 2023-11-09T03:42:36.928 INFO:teuthology.orchestra.run.smithi113.stdout:(80/142): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2023-11-09T03:42:36.953 INFO:teuthology.orchestra.run.smithi113.stdout:(81/142): python3-ply-3.11-14.el9.noarch.rpm 2.0 MB/s | 103 kB 00:00 2023-11-09T03:42:36.978 INFO:teuthology.orchestra.run.smithi113.stdout:(82/142): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2023-11-09T03:42:37.003 INFO:teuthology.orchestra.run.smithi113.stdout:(83/142): python3-requests-2.25.1-7.el9.noarch. 2.2 MB/s | 113 kB 00:00 2023-11-09T03:42:37.029 INFO:teuthology.orchestra.run.smithi113.stdout:(84/142): python3-urllib3-1.26.5-4.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2023-11-09T03:42:37.329 INFO:teuthology.orchestra.run.smithi113.stdout:(85/142): python3-cryptography-36.0.1-4.el9.x86 3.8 MB/s | 1.2 MB 00:00 2023-11-09T03:42:37.896 INFO:teuthology.orchestra.run.smithi113.stdout:(86/142): ceph-test-18.0.0-7035.g7532011a.el9.x 4.5 MB/s | 41 MB 00:09 2023-11-09T03:42:38.363 INFO:teuthology.orchestra.run.smithi113.stdout:(87/142): automake-1.16.2-8.el9.noarch.rpm 516 kB/s | 689 kB 00:01 2023-11-09T03:42:38.413 INFO:teuthology.orchestra.run.smithi113.stdout:(88/142): elfutils-libelf-devel-0.189-3.el9.x86 21 kB/s | 22 kB 00:01 2023-11-09T03:42:38.449 INFO:teuthology.orchestra.run.smithi076.stdout:(10/142): ceph-osd-18.0.0-7035.g7532011a.el9.x8 3.4 MB/s | 18 MB 00:05 2023-11-09T03:42:38.720 INFO:teuthology.orchestra.run.smithi076.stdout:(11/142): ceph-common-18.0.0-7035.g7532011a.el9 2.4 MB/s | 19 MB 00:08 2023-11-09T03:42:38.754 INFO:teuthology.orchestra.run.smithi076.stdout:(12/142): cephfs-mirror-18.0.0-7035.g7532011a.e 714 kB/s | 217 kB 00:00 2023-11-09T03:42:38.763 INFO:teuthology.orchestra.run.smithi113.stdout:(89/142): librabbitmq-0.11.0-7.el9.x86_64.rpm 54 kB/s | 45 kB 00:00 2023-11-09T03:42:38.779 INFO:teuthology.orchestra.run.smithi076.stdout:(13/142): libcephfs-devel-18.0.0-7035.g7532011a 534 kB/s | 31 kB 00:00 2023-11-09T03:42:38.862 INFO:teuthology.orchestra.run.smithi076.stdout:(14/142): libcephsqlite-18.0.0-7035.g7532011a.e 1.9 MB/s | 163 kB 00:00 2023-11-09T03:42:38.946 INFO:teuthology.orchestra.run.smithi076.stdout:(15/142): librados-devel-18.0.0-7035.g7532011a. 1.5 MB/s | 126 kB 00:00 2023-11-09T03:42:38.988 INFO:teuthology.orchestra.run.smithi076.stdout:(16/142): libcephfs2-18.0.0-7035.g7532011a.el9. 3.0 MB/s | 718 kB 00:00 2023-11-09T03:42:39.064 INFO:teuthology.orchestra.run.smithi113.stdout:(90/142): libuuid-devel-2.37.4-15.el9.x86_64.rp 37 kB/s | 26 kB 00:00 2023-11-09T03:42:39.172 INFO:teuthology.orchestra.run.smithi076.stdout:(17/142): libradosstriper1-18.0.0-7035.g7532011 2.1 MB/s | 476 kB 00:00 2023-11-09T03:42:39.255 INFO:teuthology.orchestra.run.smithi076.stdout:(18/142): python3-ceph-argparse-18.0.0-7035.g75 543 kB/s | 45 kB 00:00 2023-11-09T03:42:39.355 INFO:teuthology.orchestra.run.smithi076.stdout:(19/142): python3-ceph-common-18.0.0-7035.g7532 1.2 MB/s | 127 kB 00:00 2023-11-09T03:42:39.456 INFO:teuthology.orchestra.run.smithi076.stdout:(20/142): python3-cephfs-18.0.0-7035.g7532011a. 1.6 MB/s | 162 kB 00:00 2023-11-09T03:42:39.464 INFO:teuthology.orchestra.run.smithi113.stdout:(91/142): python3-pytz-2021.1-5.el9.noarch.rpm 73 kB/s | 51 kB 00:00 2023-11-09T03:42:39.564 INFO:teuthology.orchestra.run.smithi076.stdout:(21/142): python3-rados-18.0.0-7035.g7532011a.e 2.9 MB/s | 320 kB 00:00 2023-11-09T03:42:39.639 INFO:teuthology.orchestra.run.smithi113.stdout:(92/142): openssl-devel-3.0.7-24.el9.x86_64.rpm 3.4 MB/s | 4.1 MB 00:01 2023-11-09T03:42:39.681 INFO:teuthology.orchestra.run.smithi076.stdout:(22/142): python3-rbd-18.0.0-7035.g7532011a.el9 2.5 MB/s | 299 kB 00:00 2023-11-09T03:42:39.765 INFO:teuthology.orchestra.run.smithi076.stdout:(23/142): python3-rgw-18.0.0-7035.g7532011a.el9 1.2 MB/s | 99 kB 00:00 2023-11-09T03:42:39.790 INFO:teuthology.orchestra.run.smithi113.stdout:(93/142): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 119 kB/s | 86 kB 00:00 2023-11-09T03:42:39.832 INFO:teuthology.orchestra.run.smithi076.stdout:(24/142): rbd-fuse-18.0.0-7035.g7532011a.el9.x8 1.2 MB/s | 84 kB 00:00 2023-11-09T03:42:39.982 INFO:teuthology.orchestra.run.smithi113.stdout:(94/142): gperftools-libs-2.9.1-2.el9.x86_64.rp 904 kB/s | 309 kB 00:00 2023-11-09T03:42:40.098 INFO:teuthology.orchestra.run.smithi113.stdout:(95/142): libarrow-doc-9.0.0-9.el9.noarch.rpm 209 kB/s | 24 kB 00:00 2023-11-09T03:42:40.132 INFO:teuthology.orchestra.run.smithi076.stdout:(25/142): librgw2-18.0.0-7035.g7532011a.el9.x86 4.2 MB/s | 4.8 MB 00:01 2023-11-09T03:42:40.165 INFO:teuthology.orchestra.run.smithi113.stdout:(96/142): zlib-devel-1.2.11-40.el9.x86_64.rpm 64 kB/s | 45 kB 00:00 2023-11-09T03:42:40.190 INFO:teuthology.orchestra.run.smithi076.stdout:(26/142): rbd-nbd-18.0.0-7035.g7532011a.el9.x86 2.8 MB/s | 169 kB 00:00 2023-11-09T03:42:40.257 INFO:teuthology.orchestra.run.smithi076.stdout:(27/142): ceph-grafana-dashboards-18.0.0-7035.g 348 kB/s | 23 kB 00:00 2023-11-09T03:42:40.274 INFO:teuthology.orchestra.run.smithi113.stdout:(97/142): liboath-2.6.7-2.el9.x86_64.rpm 277 kB/s | 49 kB 00:00 2023-11-09T03:42:40.316 INFO:teuthology.orchestra.run.smithi113.stdout:(98/142): libunwind-1.6.2-1.el9.x86_64.rpm 448 kB/s | 67 kB 00:00 2023-11-09T03:42:40.341 INFO:teuthology.orchestra.run.smithi076.stdout:(28/142): ceph-mgr-cephadm-18.0.0-7035.g7532011 1.6 MB/s | 132 kB 00:00 2023-11-09T03:42:40.458 INFO:teuthology.orchestra.run.smithi113.stdout:(99/142): libarrow-9.0.0-9.el9.x86_64.rpm 6.6 MB/s | 4.4 MB 00:00 2023-11-09T03:42:40.583 INFO:teuthology.orchestra.run.smithi076.stdout:(29/142): rbd-mirror-18.0.0-7035.g7532011a.el9. 4.0 MB/s | 3.0 MB 00:00 2023-11-09T03:42:40.616 INFO:teuthology.orchestra.run.smithi113.stdout:(100/142): python3-asyncssh-2.13.2-3.el9.noarch 1.8 MB/s | 548 kB 00:00 2023-11-09T03:42:40.642 INFO:teuthology.orchestra.run.smithi113.stdout:(101/142): python3-bcrypt-3.2.2-1.el9.x86_64.rp 237 kB/s | 43 kB 00:00 2023-11-09T03:42:40.717 INFO:teuthology.orchestra.run.smithi076.stdout:(30/142): ceph-mgr-dashboard-18.0.0-7035.g75320 4.7 MB/s | 1.8 MB 00:00 2023-11-09T03:42:40.725 INFO:teuthology.orchestra.run.smithi113.stdout:(102/142): parquet-libs-9.0.0-9.el9.x86_64.rpm 1.8 MB/s | 837 kB 00:00 2023-11-09T03:42:40.751 INFO:teuthology.orchestra.run.smithi113.stdout:(103/142): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2023-11-09T03:42:40.776 INFO:teuthology.orchestra.run.smithi113.stdout:(104/142): python3-cachetools-4.2.4-1.el9.noarc 202 kB/s | 32 kB 00:00 2023-11-09T03:42:40.809 INFO:teuthology.orchestra.run.smithi076.stdout:(31/142): ceph-mgr-modules-core-18.0.0-7035.g75 2.6 MB/s | 243 kB 00:00 2023-11-09T03:42:40.879 INFO:teuthology.orchestra.run.smithi076.stdout:(32/142): ceph-mgr-rook-18.0.0-7035.g7532011a.e 719 kB/s | 48 kB 00:00 2023-11-09T03:42:40.926 INFO:teuthology.orchestra.run.smithi113.stdout:(105/142): python3-cheroot-8.6.0-4.el9.noarch.r 856 kB/s | 172 kB 00:00 2023-11-09T03:42:40.934 INFO:teuthology.orchestra.run.smithi076.stdout:(33/142): ceph-prometheus-alerts-18.0.0-7035.g7 259 kB/s | 15 kB 00:00 2023-11-09T03:42:41.002 INFO:teuthology.orchestra.run.smithi113.stdout:(106/142): python3-cherrypy-18.6.1-2.el9.noarch 1.4 MB/s | 358 kB 00:00 2023-11-09T03:42:41.018 INFO:teuthology.orchestra.run.smithi113.stdout:(107/142): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2023-11-09T03:42:41.043 INFO:teuthology.orchestra.run.smithi076.stdout:(34/142): ceph-volume-18.0.0-7035.g7532011a.el9 2.4 MB/s | 262 kB 00:00 2023-11-09T03:42:41.069 INFO:teuthology.orchestra.run.smithi113.stdout:(108/142): python3-google-auth-2.23.3-1.el9.noa 674 kB/s | 197 kB 00:00 2023-11-09T03:42:41.127 INFO:teuthology.orchestra.run.smithi113.stdout:(109/142): python3-jaraco-classes-3.2.1-5.el9.n 141 kB/s | 18 kB 00:00 2023-11-09T03:42:41.143 INFO:teuthology.orchestra.run.smithi076.stdout:(35/142): cephadm-18.0.0-7035.g7532011a.el9.noa 2.6 MB/s | 270 kB 00:00 2023-11-09T03:42:41.161 INFO:teuthology.orchestra.run.smithi113.stdout:(110/142): python3-jaraco-collections-3.0.0-8.e 163 kB/s | 23 kB 00:00 2023-11-09T03:42:41.195 INFO:teuthology.orchestra.run.smithi113.stdout:(111/142): python3-jaraco-functools-3.5.0-2.el9 155 kB/s | 19 kB 00:00 2023-11-09T03:42:41.202 INFO:teuthology.orchestra.run.smithi076.stdout:(36/142): cephfs-top-18.0.0-7035.g7532011a.el9. 313 kB/s | 18 kB 00:00 2023-11-09T03:42:41.236 INFO:teuthology.orchestra.run.smithi113.stdout:(112/142): python3-jwt+crypto-2.4.0-1.el9.noarc 120 kB/s | 9.0 kB 00:00 2023-11-09T03:42:41.262 INFO:teuthology.orchestra.run.smithi113.stdout:(113/142): python3-jaraco-text-3.2.0-6.el9.noar 146 kB/s | 20 kB 00:00 2023-11-09T03:42:41.327 INFO:teuthology.orchestra.run.smithi076.stdout:(37/142): bison-3.7.4-5.el9.x86_64.rpm 7.2 MB/s | 920 kB 00:00 2023-11-09T03:42:41.360 INFO:teuthology.orchestra.run.smithi076.stdout:(38/142): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2023-11-09T03:42:41.362 INFO:teuthology.orchestra.run.smithi113.stdout:(114/142): python3-jwt-2.4.0-1.el9.noarch.rpm 245 kB/s | 41 kB 00:00 2023-11-09T03:42:41.411 INFO:teuthology.orchestra.run.smithi076.stdout:(39/142): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2023-11-09T03:42:41.429 INFO:teuthology.orchestra.run.smithi113.stdout:(115/142): python3-logutils-0.3.5-21.el9.noarch 278 kB/s | 46 kB 00:00 2023-11-09T03:42:41.436 INFO:teuthology.orchestra.run.smithi076.stdout:(40/142): flexiblas-3.0.4-8.el9.x86_64.rpm 1.2 MB/s | 32 kB 00:00 2023-11-09T03:42:41.511 INFO:teuthology.orchestra.run.smithi076.stdout:(41/142): flexiblas-netlib-3.0.4-8.el9.x86_64.r 40 MB/s | 3.0 MB 00:00 2023-11-09T03:42:41.513 INFO:teuthology.orchestra.run.smithi113.stdout:(116/142): python3-more-itertools-8.12.0-2.el9. 525 kB/s | 79 kB 00:00 2023-11-09T03:42:41.537 INFO:teuthology.orchestra.run.smithi076.stdout:(42/142): flexiblas-openblas-openmp-3.0.4-8.el9 687 kB/s | 17 kB 00:00 2023-11-09T03:42:41.546 INFO:teuthology.orchestra.run.smithi113.stdout:(117/142): python3-kubernetes-26.1.0-2.el9.noar 3.3 MB/s | 1.0 MB 00:00 2023-11-09T03:42:41.570 INFO:teuthology.orchestra.run.smithi076.stdout:(43/142): jq-1.6-16.el9.x86_64.rpm 5.4 MB/s | 186 kB 00:00 2023-11-09T03:42:41.571 INFO:teuthology.orchestra.run.smithi113.stdout:(118/142): python3-natsort-7.1.1-5.el9.noarch.r 406 kB/s | 58 kB 00:00 2023-11-09T03:42:41.604 INFO:teuthology.orchestra.run.smithi076.stdout:(44/142): libaio-devel-0.3.111-13.el9.x86_64.rp 304 kB/s | 10 kB 00:00 2023-11-09T03:42:41.638 INFO:teuthology.orchestra.run.smithi113.stdout:(119/142): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2023-11-09T03:42:41.722 INFO:teuthology.orchestra.run.smithi113.stdout:(120/142): python3-pyOpenSSL-21.0.0-1.el9.noarc 599 kB/s | 90 kB 00:00 2023-11-09T03:42:41.748 INFO:teuthology.orchestra.run.smithi113.stdout:(121/142): python3-pecan-1.4.2-3.el9.noarch.rpm 1.1 MB/s | 272 kB 00:00 2023-11-09T03:42:41.761 INFO:teuthology.orchestra.run.smithi113.stdout:(122/142): python3-repoze-lru-0.7-16.el9.noarch 250 kB/s | 31 kB 00:00 2023-11-09T03:42:41.928 INFO:teuthology.orchestra.run.smithi113.stdout:(123/142): python3-routes-2.5.1-5.el9.noarch.rp 916 kB/s | 188 kB 00:00 2023-11-09T03:42:41.945 INFO:teuthology.orchestra.run.smithi113.stdout:(124/142): python3-tempora-5.0.0-2.el9.noarch.r 195 kB/s | 36 kB 00:00 2023-11-09T03:42:41.971 INFO:teuthology.orchestra.run.smithi113.stdout:(125/142): python3-rsa-4.9-2.el9.noarch.rpm 265 kB/s | 59 kB 00:00 2023-11-09T03:42:41.987 INFO:teuthology.orchestra.run.smithi076.stdout:(45/142): ceph-test-18.0.0-7035.g7532011a.el9.x 6.8 MB/s | 41 MB 00:06 2023-11-09T03:42:42.013 INFO:teuthology.orchestra.run.smithi076.stdout:(46/142): libpmemobj-1.12.1-1.el9.x86_64.rpm 389 kB/s | 159 kB 00:00 2023-11-09T03:42:42.046 INFO:teuthology.orchestra.run.smithi076.stdout:(47/142): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2023-11-09T03:42:42.071 INFO:teuthology.orchestra.run.smithi113.stdout:(126/142): python3-typing-extensions-4.4.0-2.el 359 kB/s | 51 kB 00:00 2023-11-09T03:42:42.071 INFO:teuthology.orchestra.run.smithi076.stdout:(48/142): libtool-2.4.6-45.el9.x86_64.rpm 9.8 MB/s | 578 kB 00:00 2023-11-09T03:42:42.089 INFO:teuthology.orchestra.run.smithi076.stdout:(49/142): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2023-11-09T03:42:42.114 INFO:teuthology.orchestra.run.smithi076.stdout:(50/142): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2023-11-09T03:42:42.139 INFO:teuthology.orchestra.run.smithi076.stdout:(51/142): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2023-11-09T03:42:42.156 INFO:teuthology.orchestra.run.smithi076.stdout:(52/142): openblas-0.3.21-2.el9.x86_64.rpm 826 kB/s | 34 kB 00:00 2023-11-09T03:42:42.196 INFO:teuthology.orchestra.run.smithi113.stdout:(127/142): python3-websocket-client-1.2.3-2.el9 398 kB/s | 90 kB 00:00 2023-11-09T03:42:42.244 INFO:teuthology.orchestra.run.smithi076.stdout:(53/142): openblas-openmp-0.3.21-2.el9.x86_64.r 45 MB/s | 4.7 MB 00:00 2023-11-09T03:42:42.254 INFO:teuthology.orchestra.run.smithi113.stdout:(128/142): python3-webob-1.8.7-6.el9.noarch.rpm 743 kB/s | 230 kB 00:00 2023-11-09T03:42:42.286 INFO:teuthology.orchestra.run.smithi076.stdout:(54/142): postgresql-13.11-1.el9.x86_64.rpm 12 MB/s | 1.5 MB 00:00 2023-11-09T03:42:42.322 INFO:teuthology.orchestra.run.smithi113.stdout:(129/142): python3-werkzeug-2.0.3-3.el9.1.noarc 1.7 MB/s | 427 kB 00:00 2023-11-09T03:42:42.328 INFO:teuthology.orchestra.run.smithi076.stdout:(55/142): postgresql-contrib-13.11-1.el9.x86_64 9.8 MB/s | 811 kB 00:00 2023-11-09T03:42:42.338 INFO:teuthology.orchestra.run.smithi113.stdout:(130/142): python3-zc-lockfile-2.0-10.el9.noarc 141 kB/s | 20 kB 00:00 2023-11-09T03:42:42.354 INFO:teuthology.orchestra.run.smithi076.stdout:(56/142): postgresql-private-libs-13.11-1.el9.x 2.0 MB/s | 134 kB 00:00 2023-11-09T03:42:42.438 INFO:teuthology.orchestra.run.smithi076.stdout:(57/142): postgresql-server-13.11-1.el9.x86_64. 52 MB/s | 5.7 MB 00:00 2023-11-09T03:42:42.530 INFO:teuthology.orchestra.run.smithi076.stdout:(58/142): ceph-mgr-diskprediction-local-18.0.0- 3.8 MB/s | 7.4 MB 00:01 2023-11-09T03:42:42.530 INFO:teuthology.orchestra.run.smithi113.stdout:(131/142): re2-20211101-3.el9.x86_64.rpm 696 kB/s | 192 kB 00:00 2023-11-09T03:42:42.597 INFO:teuthology.orchestra.run.smithi076.stdout:(59/142): python3-babel-2.9.1-2.el9.noarch.rpm 24 MB/s | 5.8 MB 00:00 2023-11-09T03:42:42.622 INFO:teuthology.orchestra.run.smithi076.stdout:(60/142): python3-devel-3.9.17-2.el9.x86_64.rpm 1.1 MB/s | 205 kB 00:00 2023-11-09T03:42:42.655 INFO:teuthology.orchestra.run.smithi076.stdout:(61/142): python3-jinja2-2.11.3-4.el9.noarch.rp 1.8 MB/s | 228 kB 00:00 2023-11-09T03:42:42.672 INFO:teuthology.orchestra.run.smithi076.stdout:(62/142): python3-jsonpatch-1.21-16.el9.noarch. 492 kB/s | 24 kB 00:00 2023-11-09T03:42:42.689 INFO:teuthology.orchestra.run.smithi076.stdout:(63/142): python3-jsonpointer-2.0-4.el9.noarch. 272 kB/s | 18 kB 00:00 2023-11-09T03:42:42.714 INFO:teuthology.orchestra.run.smithi076.stdout:(64/142): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2023-11-09T03:42:42.731 INFO:teuthology.orchestra.run.smithi113.stdout:(132/142): librados2-18.0.0-7035.g7532011a.el9. 8.5 MB/s | 3.3 MB 00:00 2023-11-09T03:42:42.731 INFO:teuthology.orchestra.run.smithi076.stdout:(65/142): python3-markupsafe-1.1.1-12.el9.x86_6 764 kB/s | 32 kB 00:00 2023-11-09T03:42:42.773 INFO:teuthology.orchestra.run.smithi113.stdout:(133/142): thrift-0.15.0-2.el9.x86_64.rpm 3.5 MB/s | 1.6 MB 00:00 2023-11-09T03:42:42.798 INFO:teuthology.orchestra.run.smithi113.stdout:(134/142): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.0 MB/s | 68 kB 00:00 2023-11-09T03:42:42.806 INFO:teuthology.orchestra.run.smithi076.stdout:(66/142): python3-numpy-1.20.1-5.el9.x86_64.rpm 41 MB/s | 4.8 MB 00:00 2023-11-09T03:42:42.832 INFO:teuthology.orchestra.run.smithi076.stdout:(67/142): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.9 MB/s | 217 kB 00:00 2023-11-09T03:42:42.848 INFO:teuthology.orchestra.run.smithi113.stdout:(135/142): NetworkManager-1.45.6-1.el9.x86_64.r 30 MB/s | 2.2 MB 00:00 2023-11-09T03:42:42.857 INFO:teuthology.orchestra.run.smithi076.stdout:(68/142): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 174 kB 00:00 2023-11-09T03:42:42.874 INFO:teuthology.orchestra.run.smithi076.stdout:(69/142): python3-prettytable-0.7.2-27.el9.noar 622 kB/s | 41 kB 00:00 2023-11-09T03:42:42.890 INFO:teuthology.orchestra.run.smithi113.stdout:(136/142): NetworkManager-libnm-1.45.6-1.el9.x8 20 MB/s | 1.8 MB 00:00 2023-11-09T03:42:42.899 INFO:teuthology.orchestra.run.smithi076.stdout:(70/142): python3-pyasn1-modules-0.4.8-6.el9.no 5.0 MB/s | 211 kB 00:00 2023-11-09T03:42:42.915 INFO:teuthology.orchestra.run.smithi113.stdout:(137/142): NetworkManager-team-1.45.6-1.el9.x86 656 kB/s | 42 kB 00:00 2023-11-09T03:42:42.932 INFO:teuthology.orchestra.run.smithi076.stdout:(71/142): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2023-11-09T03:42:42.949 INFO:teuthology.orchestra.run.smithi113.stdout:(138/142): NetworkManager-tui-1.45.6-1.el9.x86_ 4.1 MB/s | 245 kB 00:00 2023-11-09T03:42:42.950 INFO:teuthology.orchestra.run.smithi076.stdout:(72/142): python3-requests-oauthlib-1.3.0-12.el 570 kB/s | 43 kB 00:00 2023-11-09T03:42:42.974 INFO:teuthology.orchestra.run.smithi113.stdout:(139/142): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 69 kB 00:00 2023-11-09T03:42:43.000 INFO:teuthology.orchestra.run.smithi113.stdout:(140/142): libacl-2.3.1-4.el9.x86_64.rpm 439 kB/s | 22 kB 00:00 2023-11-09T03:42:43.025 INFO:teuthology.orchestra.run.smithi113.stdout:(141/142): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2023-11-09T03:42:43.158 INFO:teuthology.orchestra.run.smithi076.stdout:(73/142): python3-toml-0.10.2-6.el9.noarch.rpm 187 kB/s | 39 kB 00:00 2023-11-09T03:42:43.267 INFO:teuthology.orchestra.run.smithi113.stdout:(142/142): librbd1-18.0.0-7035.g7532011a.el9.x8 4.1 MB/s | 3.0 MB 00:00 2023-11-09T03:42:43.271 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2023-11-09T03:42:43.271 INFO:teuthology.orchestra.run.smithi113.stdout:Total 11 MB/s | 200 MB 00:17 2023-11-09T03:42:43.325 INFO:teuthology.orchestra.run.smithi076.stdout:(74/142): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2023-11-09T03:42:43.350 INFO:teuthology.orchestra.run.smithi076.stdout:(75/142): socat-1.7.4.1-5.el9.x86_64.rpm 753 kB/s | 300 kB 00:00 2023-11-09T03:42:43.367 INFO:teuthology.orchestra.run.smithi076.stdout:(76/142): uuid-1.6.2-55.el9.x86_64.rpm 272 kB/s | 56 kB 00:00 2023-11-09T03:42:43.384 INFO:teuthology.orchestra.run.smithi076.stdout:(77/142): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.9 MB/s | 63 kB 00:00 2023-11-09T03:42:43.399 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2023-11-09T03:42:43.409 INFO:teuthology.orchestra.run.smithi076.stdout:(78/142): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2023-11-09T03:42:43.451 INFO:teuthology.orchestra.run.smithi076.stdout:(79/142): libgfortran-11.4.1-2.3.el9.x86_64.rpm 9.4 MB/s | 803 kB 00:00 2023-11-09T03:42:43.476 INFO:teuthology.orchestra.run.smithi076.stdout:(80/142): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.6 MB/s | 241 kB 00:00 2023-11-09T03:42:43.527 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2023-11-09T03:42:43.527 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2023-11-09T03:42:43.552 INFO:teuthology.orchestra.run.smithi076.stdout:(81/142): libquadmath-11.4.1-2.3.el9.x86_64.rpm 1.4 MB/s | 195 kB 00:00 2023-11-09T03:42:43.577 INFO:teuthology.orchestra.run.smithi076.stdout:(82/142): python3-ply-3.11-14.el9.noarch.rpm 823 kB/s | 103 kB 00:00 2023-11-09T03:42:43.602 INFO:teuthology.orchestra.run.smithi076.stdout:(83/142): python3-pycparser-2.20-6.el9.noarch.r 992 kB/s | 124 kB 00:00 2023-11-09T03:42:43.627 INFO:teuthology.orchestra.run.smithi076.stdout:(84/142): python3-requests-2.25.1-7.el9.noarch. 2.2 MB/s | 113 kB 00:00 2023-11-09T03:42:43.653 INFO:teuthology.orchestra.run.smithi076.stdout:(85/142): python3-urllib3-1.26.5-4.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2023-11-09T03:42:44.278 INFO:teuthology.orchestra.run.smithi076.stdout:(86/142): elfutils-libelf-devel-0.189-3.el9.x86 36 kB/s | 22 kB 00:00 2023-11-09T03:42:44.594 INFO:teuthology.orchestra.run.smithi076.stdout:(87/142): python3-cryptography-36.0.1-4.el9.x86 1.2 MB/s | 1.2 MB 00:00 2023-11-09T03:42:44.653 INFO:teuthology.orchestra.run.smithi076.stdout:(88/142): automake-1.16.2-8.el9.noarch.rpm 689 kB/s | 689 kB 00:01 2023-11-09T03:42:44.953 INFO:teuthology.orchestra.run.smithi076.stdout:(89/142): librabbitmq-0.11.0-7.el9.x86_64.rpm 67 kB/s | 45 kB 00:00 2023-11-09T03:42:45.187 INFO:teuthology.orchestra.run.smithi076.stdout:(90/142): libuuid-devel-2.37.4-15.el9.x86_64.rp 43 kB/s | 26 kB 00:00 2023-11-09T03:42:45.318 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2023-11-09T03:42:45.318 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2023-11-09T03:42:45.654 INFO:teuthology.orchestra.run.smithi076.stdout:(91/142): python3-pytz-2021.1-5.el9.noarch.rpm 73 kB/s | 51 kB 00:00 2023-11-09T03:42:45.938 INFO:teuthology.orchestra.run.smithi076.stdout:(92/142): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 119 kB/s | 86 kB 00:00 2023-11-09T03:42:46.330 INFO:teuthology.orchestra.run.smithi076.stdout:(93/142): zlib-devel-1.2.11-40.el9.x86_64.rpm 67 kB/s | 45 kB 00:00 2023-11-09T03:42:46.796 INFO:teuthology.orchestra.run.smithi076.stdout:(94/142): openssl-devel-3.0.7-24.el9.x86_64.rpm 1.9 MB/s | 4.1 MB 00:02 2023-11-09T03:42:47.972 INFO:teuthology.orchestra.run.smithi076.stdout:(95/142): gperftools-libs-2.9.1-2.el9.x86_64.rp 152 kB/s | 309 kB 00:02 2023-11-09T03:42:48.080 INFO:teuthology.orchestra.run.smithi076.stdout:(96/142): liboath-2.6.7-2.el9.x86_64.rpm 449 kB/s | 49 kB 00:00 2023-11-09T03:42:48.214 INFO:teuthology.orchestra.run.smithi076.stdout:(97/142): libunwind-1.6.2-1.el9.x86_64.rpm 505 kB/s | 67 kB 00:00 2023-11-09T03:42:48.239 INFO:teuthology.orchestra.run.smithi076.stdout:(98/142): libarrow-doc-9.0.0-9.el9.noarch.rpm 17 kB/s | 24 kB 00:01 2023-11-09T03:42:49.381 INFO:teuthology.orchestra.run.smithi076.stdout:(99/142): parquet-libs-9.0.0-9.el9.x86_64.rpm 718 kB/s | 837 kB 00:01 2023-11-09T03:42:49.756 INFO:teuthology.orchestra.run.smithi076.stdout:(100/142): python3-bcrypt-3.2.2-1.el9.x86_64.rp 116 kB/s | 43 kB 00:00 2023-11-09T03:42:49.856 INFO:teuthology.orchestra.run.smithi076.stdout:(101/142): python3-cachetools-4.2.4-1.el9.noarc 322 kB/s | 32 kB 00:00 2023-11-09T03:42:49.940 INFO:teuthology.orchestra.run.smithi076.stdout:(102/142): python3-certifi-2021.10.8-2.el9.noar 180 kB/s | 15 kB 00:00 2023-11-09T03:42:50.240 INFO:teuthology.orchestra.run.smithi076.stdout:(103/142): python3-cheroot-8.6.0-4.el9.noarch.r 572 kB/s | 172 kB 00:00 2023-11-09T03:42:50.565 INFO:teuthology.orchestra.run.smithi076.stdout:(104/142): python3-cherrypy-18.6.1-2.el9.noarch 1.1 MB/s | 358 kB 00:00 2023-11-09T03:42:50.665 INFO:teuthology.orchestra.run.smithi076.stdout:(105/142): python3-google-auth-2.23.3-1.el9.noa 1.9 MB/s | 197 kB 00:00 2023-11-09T03:42:50.710 INFO:teuthology.orchestra.run.smithi076.stdout:(106/142): python3-jaraco-8.2.1-3.el9.noarch.rp 245 kB/s | 11 kB 00:00 2023-11-09T03:42:50.743 INFO:teuthology.orchestra.run.smithi076.stdout:(107/142): python3-jaraco-classes-3.2.1-5.el9.n 533 kB/s | 18 kB 00:00 2023-11-09T03:42:50.785 INFO:teuthology.orchestra.run.smithi076.stdout:(108/142): python3-jaraco-collections-3.0.0-8.e 555 kB/s | 23 kB 00:00 2023-11-09T03:42:50.798 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2023-11-09T03:42:50.827 INFO:teuthology.orchestra.run.smithi076.stdout:(109/142): python3-jaraco-functools-3.5.0-2.el9 467 kB/s | 19 kB 00:00 2023-11-09T03:42:50.869 INFO:teuthology.orchestra.run.smithi076.stdout:(110/142): python3-jaraco-text-3.2.0-6.el9.noar 473 kB/s | 20 kB 00:00 2023-11-09T03:42:50.877 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2023-11-09T03:42:50.910 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2023-11-09T03:42:50.911 INFO:teuthology.orchestra.run.smithi076.stdout:(111/142): python3-jwt+crypto-2.4.0-1.el9.noarc 217 kB/s | 9.0 kB 00:00 2023-11-09T03:42:50.969 INFO:teuthology.orchestra.run.smithi076.stdout:(112/142): python3-jwt-2.4.0-1.el9.noarch.rpm 702 kB/s | 41 kB 00:00 2023-11-09T03:42:51.071 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2023-11-09T03:42:51.236 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/152 2023-11-09T03:42:51.536 INFO:teuthology.orchestra.run.smithi076.stdout:(113/142): python3-kubernetes-26.1.0-2.el9.noar 1.8 MB/s | 1.0 MB 00:00 2023-11-09T03:42:51.556 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 5/152 2023-11-09T03:42:51.586 INFO:teuthology.orchestra.run.smithi076.stdout:(114/142): python3-logutils-0.3.5-21.el9.noarch 927 kB/s | 46 kB 00:00 2023-11-09T03:42:51.620 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : librados2-2:18.0.0-7035.g7532011a.el9.x86_64 6/152 2023-11-09T03:42:51.662 INFO:teuthology.orchestra.run.smithi076.stdout:(115/142): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2023-11-09T03:42:51.740 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librados2-2:18.0.0-7035.g7532011a.el9.x86_64 6/152 2023-11-09T03:42:51.789 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephfs2-2:18.0.0-7035.g7532011a.el9.x86_64 7/152 2023-11-09T03:42:51.880 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libcephfs2-2:18.0.0-7035.g7532011a.el9.x86_64 7/152 2023-11-09T03:42:51.922 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rados-2:18.0.0-7035.g7532011a.el9.x86_64 8/152 2023-11-09T03:42:51.960 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/152 2023-11-09T03:42:51.979 INFO:teuthology.orchestra.run.smithi076.stdout:(116/142): python3-natsort-7.1.1-5.el9.noarch.r 182 kB/s | 58 kB 00:00 2023-11-09T03:42:52.095 INFO:teuthology.orchestra.run.smithi076.stdout:(117/142): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2023-11-09T03:42:52.124 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 10/152 2023-11-09T03:42:52.182 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-13.11-1.el9.x86_64 11/152 2023-11-09T03:42:52.202 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephsqlite-2:18.0.0-7035.g7532011a.el9.x86_64 12/152 2023-11-09T03:42:52.229 INFO:teuthology.orchestra.run.smithi076.stdout:(118/142): python3-asyncssh-2.13.2-3.el9.noarch 137 kB/s | 548 kB 00:03 2023-11-09T03:42:52.302 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7035.g7532011a.el9.x86_64 12/152 2023-11-09T03:42:52.313 INFO:teuthology.orchestra.run.smithi076.stdout:(119/142): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2023-11-09T03:42:52.327 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libradosstriper1-2:18.0.0-7035.g7532011a.el9.x86 13/152 2023-11-09T03:42:52.371 INFO:teuthology.orchestra.run.smithi076.stdout:(120/142): python3-repoze-lru-0.7-16.el9.noarch 527 kB/s | 31 kB 00:00 2023-11-09T03:42:52.390 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7035.g7532011a.el9.x86 13/152 2023-11-09T03:42:52.397 INFO:teuthology.orchestra.run.smithi076.stdout:(121/142): python3-portend-3.1.0-2.el9.noarch.r 54 kB/s | 16 kB 00:00 2023-11-09T03:42:52.464 INFO:teuthology.orchestra.run.smithi076.stdout:(122/142): python3-rsa-4.9-2.el9.noarch.rpm 888 kB/s | 59 kB 00:00 2023-11-09T03:42:52.547 INFO:teuthology.orchestra.run.smithi076.stdout:(123/142): python3-routes-2.5.1-5.el9.noarch.rp 1.0 MB/s | 188 kB 00:00 2023-11-09T03:42:52.563 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 14/152 2023-11-09T03:42:52.572 INFO:teuthology.orchestra.run.smithi076.stdout:(124/142): python3-tempora-5.0.0-2.el9.noarch.r 329 kB/s | 36 kB 00:00 2023-11-09T03:42:52.606 INFO:teuthology.orchestra.run.smithi076.stdout:(125/142): python3-typing-extensions-4.4.0-2.el 872 kB/s | 51 kB 00:00 2023-11-09T03:42:52.682 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 15/152 2023-11-09T03:42:52.690 INFO:teuthology.orchestra.run.smithi076.stdout:(126/142): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2023-11-09T03:42:52.742 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 15/152 2023-11-09T03:42:52.841 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 16/152 2023-11-09T03:42:52.874 INFO:teuthology.orchestra.run.smithi076.stdout:(127/142): python3-werkzeug-2.0.3-3.el9.1.noarc 2.3 MB/s | 427 kB 00:00 2023-11-09T03:42:52.885 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : NetworkManager-1:1.45.6-1.el9.x86_64 16/152 2023-11-09T03:42:53.163 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 16/152 2023-11-09T03:42:53.165 INFO:teuthology.orchestra.run.smithi076.stdout:(128/142): python3-zc-lockfile-2.0-10.el9.noarc 68 kB/s | 20 kB 00:00 2023-11-09T03:42:53.198 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/152 2023-11-09T03:42:53.232 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/152 2023-11-09T03:42:53.241 INFO:teuthology.orchestra.run.smithi076.stdout:(129/142): python3-webob-1.8.7-6.el9.noarch.rpm 344 kB/s | 230 kB 00:00 2023-11-09T03:42:53.301 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/152 2023-11-09T03:42:53.354 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 20/152 2023-11-09T03:42:53.385 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 21/152 2023-11-09T03:42:53.428 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 22/152 2023-11-09T03:42:53.499 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 23/152 2023-11-09T03:42:53.541 INFO:teuthology.orchestra.run.smithi076.stdout:(130/142): re2-20211101-3.el9.x86_64.rpm 511 kB/s | 192 kB 00:00 2023-11-09T03:42:53.542 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/152 2023-11-09T03:42:53.570 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/152 2023-11-09T03:42:53.639 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/152 2023-11-09T03:42:53.675 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ceph-common-2:18.0.0-7035.g7532011a.el9. 27/152 2023-11-09T03:42:53.710 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ceph-argparse-2:18.0.0-7035.g7532011a.el 28/152 2023-11-09T03:42:53.784 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cephfs-2:18.0.0-7035.g7532011a.el9.x86_6 29/152 2023-11-09T03:42:53.908 INFO:teuthology.orchestra.run.smithi076.stdout:(131/142): librados2-18.0.0-7035.g7532011a.el9. 9.0 MB/s | 3.3 MB 00:00 2023-11-09T03:42:53.927 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/152 2023-11-09T03:42:53.988 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/152 2023-11-09T03:42:54.167 INFO:teuthology.orchestra.run.smithi076.stdout:(132/142): librbd1-18.0.0-7035.g7532011a.el9.x8 12 MB/s | 3.0 MB 00:00 2023-11-09T03:42:54.201 INFO:teuthology.orchestra.run.smithi076.stdout:(133/142): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2023-11-09T03:42:54.259 INFO:teuthology.orchestra.run.smithi076.stdout:(134/142): NetworkManager-1.45.6-1.el9.x86_64.r 38 MB/s | 2.2 MB 00:00 2023-11-09T03:42:54.318 INFO:teuthology.orchestra.run.smithi076.stdout:(135/142): NetworkManager-libnm-1.45.6-1.el9.x8 30 MB/s | 1.8 MB 00:00 2023-11-09T03:42:54.351 INFO:teuthology.orchestra.run.smithi076.stdout:(136/142): NetworkManager-team-1.45.6-1.el9.x86 1.2 MB/s | 42 kB 00:00 2023-11-09T03:42:54.385 INFO:teuthology.orchestra.run.smithi076.stdout:(137/142): NetworkManager-tui-1.45.6-1.el9.x86_ 7.2 MB/s | 245 kB 00:00 2023-11-09T03:42:54.419 INFO:teuthology.orchestra.run.smithi076.stdout:(138/142): acl-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 69 kB 00:00 2023-11-09T03:42:54.452 INFO:teuthology.orchestra.run.smithi076.stdout:(139/142): libacl-2.3.1-4.el9.x86_64.rpm 660 kB/s | 22 kB 00:00 2023-11-09T03:42:54.494 INFO:teuthology.orchestra.run.smithi076.stdout:(140/142): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2023-11-09T03:42:54.516 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/152 2023-11-09T03:42:54.599 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/152 2023-11-09T03:42:54.637 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 34/152 2023-11-09T03:42:54.676 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librados-devel-2:18.0.0-7035.g7532011a.el9.x86_6 35/152 2023-11-09T03:42:54.705 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/152 2023-11-09T03:42:54.812 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : re2-1:20211101-3.el9.x86_64 37/152 2023-11-09T03:42:54.859 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 38/152 2023-11-09T03:42:54.916 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 39/152 2023-11-09T03:42:54.988 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 40/152 2023-11-09T03:42:54.994 INFO:teuthology.orchestra.run.smithi076.stdout:(141/142): thrift-0.15.0-2.el9.x86_64.rpm 924 kB/s | 1.6 MB 00:01 2023-11-09T03:42:55.027 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 41/152 2023-11-09T03:42:55.063 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 42/152 2023-11-09T03:42:55.111 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 43/152 2023-11-09T03:42:55.155 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 44/152 2023-11-09T03:42:55.198 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 45/152 2023-11-09T03:42:55.334 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 46/152 2023-11-09T03:42:55.391 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 47/152 2023-11-09T03:42:55.433 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 48/152 2023-11-09T03:42:55.527 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 49/152 2023-11-09T03:42:55.567 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-google-auth-1:2.23.3-1.el9.noarch 50/152 2023-11-09T03:42:55.607 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 51/152 2023-11-09T03:42:55.640 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 52/152 2023-11-09T03:42:55.666 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 53/152 2023-11-09T03:42:55.694 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 54/152 2023-11-09T03:42:55.791 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 55/152 2023-11-09T03:42:55.827 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : automake-1.16.2-8.el9.noarch 56/152 2023-11-09T03:42:55.876 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ply-3.11-14.el9.noarch 57/152 2023-11-09T03:42:55.940 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 58/152 2023-11-09T03:42:56.137 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 59/152 2023-11-09T03:42:56.265 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 60/152 2023-11-09T03:42:56.405 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 61/152 2023-11-09T03:42:56.451 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 62/152 2023-11-09T03:42:56.480 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 63/152 2023-11-09T03:42:56.514 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 64/152 2023-11-09T03:42:56.546 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 65/152 2023-11-09T03:42:56.579 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : fuse-2.9.9-15.el9.x86_64 66/152 2023-11-09T03:42:56.613 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 67/152 2023-11-09T03:42:56.658 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : uuid-1.6.2-55.el9.x86_64 68/152 2023-11-09T03:42:56.693 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 69/152 2023-11-09T03:42:56.723 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 70/152 2023-11-09T03:42:56.800 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 71/152 2023-11-09T03:42:56.841 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 72/152 2023-11-09T03:42:56.875 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/152 2023-11-09T03:42:56.908 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/152 2023-11-09T03:42:56.934 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/152 2023-11-09T03:42:57.022 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/152 2023-11-09T03:42:57.062 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 77/152 2023-11-09T03:42:57.163 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 78/152 2023-11-09T03:42:57.213 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2023-11-09T03:42:57.946 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2023-11-09T03:42:57.993 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2023-11-09T03:42:58.027 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2023-11-09T03:42:58.125 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2023-11-09T03:42:58.169 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 84/152 2023-11-09T03:42:58.346 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openblas-0.3.21-2.el9.x86_64 85/152 2023-11-09T03:42:58.375 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 86/152 2023-11-09T03:42:58.453 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 87/152 2023-11-09T03:42:59.098 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 88/152 2023-11-09T03:42:59.218 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 89/152 2023-11-09T03:43:00.472 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 90/152 2023-11-09T03:43:00.554 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 91/152 2023-11-09T03:43:00.592 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 92/152 2023-11-09T03:43:00.625 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : jq-1.6-16.el9.x86_64 93/152 2023-11-09T03:43:00.652 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2023-11-09T03:43:00.958 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 95/152 2023-11-09T03:43:00.982 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : librbd1-2:18.0.0-7035.g7532011a.el9.x86_64 96/152 2023-11-09T03:43:01.073 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librbd1-2:18.0.0-7035.g7532011a.el9.x86_64 96/152 2023-11-09T03:43:01.103 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rbd-2:18.0.0-7035.g7532011a.el9.x86_64 97/152 2023-11-09T03:43:01.139 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 98/152 2023-11-09T03:43:01.567 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 99/152 2023-11-09T03:43:01.601 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librgw2-2:18.0.0-7035.g7532011a.el9.x86_64 100/152 2023-11-09T03:43:01.666 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librgw2-2:18.0.0-7035.g7532011a.el9.x86_64 100/152 2023-11-09T03:43:01.695 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rgw-2:18.0.0-7035.g7532011a.el9.x86_64 101/152 2023-11-09T03:43:03.772 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-common-2:18.0.0-7035.g7532011a.el9.x86_64 102/152 2023-11-09T03:43:03.808 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-common-2:18.0.0-7035.g7532011a.el9.x86_64 102/152 2023-11-09T03:43:04.353 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-common-2:18.0.0-7035.g7532011a.el9.x86_64 102/152 2023-11-09T03:43:04.395 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-base-2:18.0.0-7035.g7532011a.el9.x86_64 103/152 2023-11-09T03:43:04.482 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-base-2:18.0.0-7035.g7532011a.el9.x86_64 103/152 2023-11-09T03:43:04.483 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-11-09T03:43:04.483 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-11-09T03:43:04.483 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:43:04.521 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-selinux-2:18.0.0-7035.g7532011a.el9.x86_64 104/152 2023-11-09T03:43:15.726 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7035.g7532011a.el9.x86_64 104/152 2023-11-09T03:43:15.726 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /sys 2023-11-09T03:43:15.726 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /proc 2023-11-09T03:43:15.727 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /mnt 2023-11-09T03:43:15.727 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /var/tmp 2023-11-09T03:43:15.727 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /home 2023-11-09T03:43:15.727 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /root 2023-11-09T03:43:15.727 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /tmp 2023-11-09T03:43:15.727 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:43:15.900 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-volume-2:18.0.0-7035.g7532011a.el9.noarch 105/152 2023-11-09T03:43:15.916 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-volume-2:18.0.0-7035.g7532011a.el9.noarch 105/152 2023-11-09T03:43:15.916 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-09T03:43:15.917 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-11-09T03:43:15.917 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:43:17.503 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-osd-2:18.0.0-7035.g7532011a.el9.x86_64 106/152 2023-11-09T03:43:17.529 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-osd-2:18.0.0-7035.g7532011a.el9.x86_64 106/152 2023-11-09T03:43:17.529 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-09T03:43:17.529 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-11-09T03:43:17.530 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-09T03:43:17.530 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-09T03:43:17.530 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:43:17.746 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mds-2:18.0.0-7035.g7532011a.el9.x86_64 107/152 2023-11-09T03:43:17.768 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mds-2:18.0.0-7035.g7532011a.el9.x86_64 107/152 2023-11-09T03:43:17.768 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-09T03:43:17.769 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-11-09T03:43:17.769 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-09T03:43:17.769 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-09T03:43:17.769 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:43:18.213 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mon-2:18.0.0-7035.g7532011a.el9.x86_64 108/152 2023-11-09T03:43:18.235 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mon-2:18.0.0-7035.g7532011a.el9.x86_64 108/152 2023-11-09T03:43:18.236 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-09T03:43:18.236 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-11-09T03:43:18.236 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-09T03:43:18.236 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-09T03:43:18.236 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:43:18.628 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: cephadm-2:18.0.0-7035.g7532011a.el9.noarch 109/152 2023-11-09T03:43:18.655 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephadm-2:18.0.0-7035.g7532011a.el9.noarch 109/152 2023-11-09T03:43:18.684 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7035.g7532011a.e 110/152 2023-11-09T03:43:18.746 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7035.g7532011a. 111/152 2023-11-09T03:43:18.770 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7035.g7532011a.el9.noa 112/152 2023-11-09T03:43:18.990 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7035.g7532011a.el9.noa 112/152 2023-11-09T03:43:19.021 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7035.g7532011a.el9.n 113/152 2023-11-09T03:43:19.979 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7035.g7532011a.el9.n 113/152 2023-11-09T03:43:20.005 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7035.g753 114/152 2023-11-09T03:43:20.104 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7035.g753 114/152 2023-11-09T03:43:20.252 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7035.g7532011a.el 115/152 2023-11-09T03:43:20.282 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-2:18.0.0-7035.g7532011a.el9.x86_64 116/152 2023-11-09T03:43:20.305 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7035.g7532011a.el9.x86_64 116/152 2023-11-09T03:43:20.305 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-09T03:43:20.305 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-11-09T03:43:20.305 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-09T03:43:20.306 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-09T03:43:20.306 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:43:20.349 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-rook-2:18.0.0-7035.g7532011a.el9.noarch 117/152 2023-11-09T03:43:20.360 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7035.g7532011a.el9.noarch 117/152 2023-11-09T03:43:21.145 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-2:18.0.0-7035.g7532011a.el9.x86_64 118/152 2023-11-09T03:43:21.175 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-radosgw-2:18.0.0-7035.g7532011a.el9.x86_64 119/152 2023-11-09T03:43:21.198 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7035.g7532011a.el9.x86_64 119/152 2023-11-09T03:43:21.198 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-09T03:43:21.198 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-11-09T03:43:21.198 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-09T03:43:21.199 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-09T03:43:21.199 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:43:21.242 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7035.g75320 120/152 2023-11-09T03:43:21.264 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7035.g75320 120/152 2023-11-09T03:43:21.264 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-09T03:43:21.264 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-11-09T03:43:21.265 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:43:21.318 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephfs-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 121/152 2023-11-09T03:43:21.340 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 121/152 2023-11-09T03:43:21.341 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-09T03:43:21.341 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-11-09T03:43:21.341 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-09T03:43:21.341 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-09T03:43:21.342 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:43:21.636 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 122/152 2023-11-09T03:43:21.659 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 122/152 2023-11-09T03:43:21.659 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-09T03:43:21.660 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-11-09T03:43:21.660 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-09T03:43:21.660 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-09T03:43:21.660 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:43:25.471 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-test-2:18.0.0-7035.g7532011a.el9.x86_64 123/152 2023-11-09T03:43:25.513 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-fuse-2:18.0.0-7035.g7532011a.el9.x86_64 124/152 2023-11-09T03:43:25.543 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-nbd-2:18.0.0-7035.g7532011a.el9.x86_64 125/152 2023-11-09T03:43:25.728 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 126/152 2023-11-09T03:43:25.842 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 127/152 2023-11-09T03:43:25.909 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-fuse-2:18.0.0-7035.g7532011a.el9.x86_64 128/152 2023-11-09T03:43:25.952 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libtool-2.4.6-45.el9.x86_64 129/152 2023-11-09T03:43:25.985 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 130/152 2023-11-09T03:43:26.007 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephfs-devel-2:18.0.0-7035.g7532011a.el9.x86_ 131/152 2023-11-09T03:43:26.037 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : NetworkManager-team-1:1.45.6-1.el9.x86_64 132/152 2023-11-09T03:43:26.108 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : NetworkManager-tui-1:1.45.6-1.el9.x86_64 133/152 2023-11-09T03:43:26.167 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 134/152 2023-11-09T03:43:26.194 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 135/152 2023-11-09T03:43:26.759 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 136/152 2023-11-09T03:43:26.794 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 136/152 2023-11-09T03:43:26.814 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 136/152 2023-11-09T03:43:26.867 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephfs-top-2:18.0.0-7035.g7532011a.el9.noarch 137/152 2023-11-09T03:43:27.720 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 138/152 2023-11-09T03:43:27.816 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 139/152 2023-11-09T03:43:27.856 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 140/152 2023-11-09T03:43:27.962 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flex-2.6.4-9.el9.x86_64 141/152 2023-11-09T03:43:27.994 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : bison-3.7.4-5.el9.x86_64 142/152 2023-11-09T03:43:28.020 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 143/152 2023-11-09T03:43:28.021 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 144/152 2023-11-09T03:43:28.096 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 144/152 2023-11-09T03:43:28.121 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 145/152 2023-11-09T03:43:28.146 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 146/152 2023-11-09T03:43:28.171 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 147/152 2023-11-09T03:43:28.178 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 148/152 2023-11-09T03:43:28.178 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 148/152 2023-11-09T03:43:28.374 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 148/152 2023-11-09T03:43:28.375 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 149/152 2023-11-09T03:43:28.457 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 149/152 2023-11-09T03:43:28.482 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 150/152 2023-11-09T03:43:28.482 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 151/152 2023-11-09T03:43:28.567 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 151/152 2023-11-09T03:43:28.631 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 152/152 2023-11-09T03:43:33.235 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 152/152 2023-11-09T03:43:33.235 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-2:18.0.0-7035.g7532011a.el9.x86_64 1/152 2023-11-09T03:43:33.235 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-base-2:18.0.0-7035.g7532011a.el9.x86_64 2/152 2023-11-09T03:43:33.236 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-common-2:18.0.0-7035.g7532011a.el9.x86_64 3/152 2023-11-09T03:43:33.236 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-fuse-2:18.0.0-7035.g7532011a.el9.x86_64 4/152 2023-11-09T03:43:33.236 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7035.g75320 5/152 2023-11-09T03:43:33.236 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mds-2:18.0.0-7035.g7532011a.el9.x86_64 6/152 2023-11-09T03:43:33.236 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-2:18.0.0-7035.g7532011a.el9.x86_64 7/152 2023-11-09T03:43:33.237 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mon-2:18.0.0-7035.g7532011a.el9.x86_64 8/152 2023-11-09T03:43:33.237 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-osd-2:18.0.0-7035.g7532011a.el9.x86_64 9/152 2023-11-09T03:43:33.237 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-radosgw-2:18.0.0-7035.g7532011a.el9.x86_64 10/152 2023-11-09T03:43:33.237 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-selinux-2:18.0.0-7035.g7532011a.el9.x86_64 11/152 2023-11-09T03:43:33.237 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-test-2:18.0.0-7035.g7532011a.el9.x86_64 12/152 2023-11-09T03:43:33.238 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephfs-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 13/152 2023-11-09T03:43:33.238 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephfs-devel-2:18.0.0-7035.g7532011a.el9.x86_ 14/152 2023-11-09T03:43:33.238 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephfs2-2:18.0.0-7035.g7532011a.el9.x86_64 15/152 2023-11-09T03:43:33.238 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephsqlite-2:18.0.0-7035.g7532011a.el9.x86_64 16/152 2023-11-09T03:43:33.238 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados-devel-2:18.0.0-7035.g7532011a.el9.x86_6 17/152 2023-11-09T03:43:33.239 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libradosstriper1-2:18.0.0-7035.g7532011a.el9.x86 18/152 2023-11-09T03:43:33.239 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librgw2-2:18.0.0-7035.g7532011a.el9.x86_64 19/152 2023-11-09T03:43:33.239 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7035.g7532011a.el 20/152 2023-11-09T03:43:33.239 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ceph-common-2:18.0.0-7035.g7532011a.el9. 21/152 2023-11-09T03:43:33.239 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cephfs-2:18.0.0-7035.g7532011a.el9.x86_6 22/152 2023-11-09T03:43:33.240 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rados-2:18.0.0-7035.g7532011a.el9.x86_64 23/152 2023-11-09T03:43:33.240 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rbd-2:18.0.0-7035.g7532011a.el9.x86_64 24/152 2023-11-09T03:43:33.240 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rgw-2:18.0.0-7035.g7532011a.el9.x86_64 25/152 2023-11-09T03:43:33.240 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-fuse-2:18.0.0-7035.g7532011a.el9.x86_64 26/152 2023-11-09T03:43:33.240 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 27/152 2023-11-09T03:43:33.241 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-nbd-2:18.0.0-7035.g7532011a.el9.x86_64 28/152 2023-11-09T03:43:33.241 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7035.g7532011a. 29/152 2023-11-09T03:43:33.241 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7035.g7532011a.el9.noa 30/152 2023-11-09T03:43:33.241 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7035.g7532011a.el9.n 31/152 2023-11-09T03:43:33.241 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7035.g753 32/152 2023-11-09T03:43:33.241 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7035.g7532011a.el 33/152 2023-11-09T03:43:33.242 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7035.g7532011a.el9.noarch 34/152 2023-11-09T03:43:33.242 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7035.g7532011a.e 35/152 2023-11-09T03:43:33.242 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-volume-2:18.0.0-7035.g7532011a.el9.noarch 36/152 2023-11-09T03:43:33.242 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephadm-2:18.0.0-7035.g7532011a.el9.noarch 37/152 2023-11-09T03:43:33.243 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephfs-top-2:18.0.0-7035.g7532011a.el9.noarch 38/152 2023-11-09T03:43:33.243 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : bison-3.7.4-5.el9.x86_64 39/152 2023-11-09T03:43:33.243 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 40/152 2023-11-09T03:43:33.243 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/152 2023-11-09T03:43:33.243 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/152 2023-11-09T03:43:33.243 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/152 2023-11-09T03:43:33.244 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/152 2023-11-09T03:43:33.244 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : jq-1.6-16.el9.x86_64 45/152 2023-11-09T03:43:33.244 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 46/152 2023-11-09T03:43:33.244 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 47/152 2023-11-09T03:43:33.244 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 48/152 2023-11-09T03:43:33.245 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 49/152 2023-11-09T03:43:33.245 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 50/152 2023-11-09T03:43:33.245 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 51/152 2023-11-09T03:43:33.245 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 52/152 2023-11-09T03:43:33.245 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 53/152 2023-11-09T03:43:33.246 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 54/152 2023-11-09T03:43:33.246 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-13.11-1.el9.x86_64 55/152 2023-11-09T03:43:33.246 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 56/152 2023-11-09T03:43:33.246 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 57/152 2023-11-09T03:43:33.246 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 58/152 2023-11-09T03:43:33.246 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 59/152 2023-11-09T03:43:33.247 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 60/152 2023-11-09T03:43:33.247 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 61/152 2023-11-09T03:43:33.247 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 62/152 2023-11-09T03:43:33.247 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 63/152 2023-11-09T03:43:33.247 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 64/152 2023-11-09T03:43:33.248 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 65/152 2023-11-09T03:43:33.248 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 66/152 2023-11-09T03:43:33.248 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 67/152 2023-11-09T03:43:33.248 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 68/152 2023-11-09T03:43:33.248 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/152 2023-11-09T03:43:33.249 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 70/152 2023-11-09T03:43:33.249 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 71/152 2023-11-09T03:43:33.249 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 72/152 2023-11-09T03:43:33.249 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 73/152 2023-11-09T03:43:33.249 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 74/152 2023-11-09T03:43:33.250 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 75/152 2023-11-09T03:43:33.250 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 76/152 2023-11-09T03:43:33.250 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 77/152 2023-11-09T03:43:33.250 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 78/152 2023-11-09T03:43:33.250 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 79/152 2023-11-09T03:43:33.250 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 80/152 2023-11-09T03:43:33.251 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 81/152 2023-11-09T03:43:33.251 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ply-3.11-14.el9.noarch 82/152 2023-11-09T03:43:33.251 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 83/152 2023-11-09T03:43:33.251 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 84/152 2023-11-09T03:43:33.251 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 85/152 2023-11-09T03:43:33.252 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 86/152 2023-11-09T03:43:33.252 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : automake-1.16.2-8.el9.noarch 87/152 2023-11-09T03:43:33.252 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 88/152 2023-11-09T03:43:33.252 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 89/152 2023-11-09T03:43:33.252 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 90/152 2023-11-09T03:43:33.253 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 91/152 2023-11-09T03:43:33.253 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/152 2023-11-09T03:43:33.253 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 93/152 2023-11-09T03:43:33.253 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 94/152 2023-11-09T03:43:33.253 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 95/152 2023-11-09T03:43:33.254 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 96/152 2023-11-09T03:43:33.254 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 97/152 2023-11-09T03:43:33.254 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/152 2023-11-09T03:43:33.254 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/152 2023-11-09T03:43:33.254 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 100/152 2023-11-09T03:43:33.255 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 101/152 2023-11-09T03:43:33.255 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/152 2023-11-09T03:43:33.255 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/152 2023-11-09T03:43:33.255 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 104/152 2023-11-09T03:43:33.255 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/152 2023-11-09T03:43:33.255 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/152 2023-11-09T03:43:33.256 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-google-auth-1:2.23.3-1.el9.noarch 107/152 2023-11-09T03:43:33.256 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/152 2023-11-09T03:43:33.256 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/152 2023-11-09T03:43:33.256 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/152 2023-11-09T03:43:33.256 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/152 2023-11-09T03:43:33.257 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/152 2023-11-09T03:43:33.257 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 113/152 2023-11-09T03:43:33.257 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 114/152 2023-11-09T03:43:33.257 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/152 2023-11-09T03:43:33.257 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/152 2023-11-09T03:43:33.258 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/152 2023-11-09T03:43:33.258 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/152 2023-11-09T03:43:33.258 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/152 2023-11-09T03:43:33.258 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/152 2023-11-09T03:43:33.258 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/152 2023-11-09T03:43:33.259 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/152 2023-11-09T03:43:33.259 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/152 2023-11-09T03:43:33.259 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/152 2023-11-09T03:43:33.260 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/152 2023-11-09T03:43:33.260 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/152 2023-11-09T03:43:33.260 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/152 2023-11-09T03:43:33.260 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/152 2023-11-09T03:43:33.260 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/152 2023-11-09T03:43:33.260 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/152 2023-11-09T03:43:33.261 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/152 2023-11-09T03:43:33.261 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/152 2023-11-09T03:43:33.261 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados2-2:18.0.0-7035.g7532011a.el9.x86_64 133/152 2023-11-09T03:43:33.261 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/152 2023-11-09T03:43:33.261 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librbd1-2:18.0.0-7035.g7532011a.el9.x86_64 135/152 2023-11-09T03:43:33.262 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/152 2023-11-09T03:43:33.262 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 137/152 2023-11-09T03:43:33.262 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 138/152 2023-11-09T03:43:33.262 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-1:1.45.6-1.el9.x86_64 139/152 2023-11-09T03:43:33.262 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/152 2023-11-09T03:43:33.263 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 141/152 2023-11-09T03:43:33.263 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/152 2023-11-09T03:43:33.263 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-team-1:1.45.6-1.el9.x86_64 143/152 2023-11-09T03:43:33.263 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/152 2023-11-09T03:43:33.263 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-tui-1:1.45.6-1.el9.x86_64 145/152 2023-11-09T03:43:33.264 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/152 2023-11-09T03:43:33.264 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/152 2023-11-09T03:43:33.264 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/152 2023-11-09T03:43:33.264 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 149/152 2023-11-09T03:43:33.264 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 150/152 2023-11-09T03:43:33.265 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 151/152 2023-11-09T03:43:34.699 INFO:teuthology.orchestra.run.smithi076.stdout:(142/142): libarrow-9.0.0-9.el9.x86_64.rpm 94 kB/s | 4.4 MB 00:48 2023-11-09T03:43:34.703 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2023-11-09T03:43:34.703 INFO:teuthology.orchestra.run.smithi076.stdout:Total 3.1 MB/s | 200 MB 01:04 2023-11-09T03:43:34.824 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2023-11-09T03:43:34.950 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2023-11-09T03:43:34.951 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2023-11-09T03:43:35.778 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 152/152 2023-11-09T03:43:35.779 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:43:35.779 INFO:teuthology.orchestra.run.smithi113.stdout:Upgraded: 2023-11-09T03:43:35.779 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-1:1.45.6-1.el9.x86_64 2023-11-09T03:43:35.779 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 2023-11-09T03:43:35.780 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-team-1:1.45.6-1.el9.x86_64 2023-11-09T03:43:35.780 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-tui-1:1.45.6-1.el9.x86_64 2023-11-09T03:43:35.780 INFO:teuthology.orchestra.run.smithi113.stdout: acl-2.3.1-4.el9.x86_64 2023-11-09T03:43:35.780 INFO:teuthology.orchestra.run.smithi113.stdout: libacl-2.3.1-4.el9.x86_64 2023-11-09T03:43:35.780 INFO:teuthology.orchestra.run.smithi113.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-11-09T03:43:35.781 INFO:teuthology.orchestra.run.smithi113.stdout: librados2-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.781 INFO:teuthology.orchestra.run.smithi113.stdout: librbd1-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.781 INFO:teuthology.orchestra.run.smithi113.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-11-09T03:43:35.781 INFO:teuthology.orchestra.run.smithi113.stdout:Installed: 2023-11-09T03:43:35.781 INFO:teuthology.orchestra.run.smithi113.stdout: automake-1.16.2-8.el9.noarch 2023-11-09T03:43:35.782 INFO:teuthology.orchestra.run.smithi113.stdout: bison-3.7.4-5.el9.x86_64 2023-11-09T03:43:35.782 INFO:teuthology.orchestra.run.smithi113.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-11-09T03:43:35.782 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.782 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-base-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.782 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-common-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.783 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-fuse-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.783 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-grafana-dashboards-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-09T03:43:35.783 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-immutable-object-cache-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.783 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mds-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.783 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.784 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-cephadm-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-09T03:43:35.784 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-dashboard-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-09T03:43:35.784 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-09T03:43:35.784 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-modules-core-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-09T03:43:35.785 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-rook-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-09T03:43:35.785 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mon-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.785 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-osd-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.785 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-prometheus-alerts-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-09T03:43:35.785 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-radosgw-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.786 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-selinux-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.786 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-test-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.786 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-volume-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-09T03:43:35.786 INFO:teuthology.orchestra.run.smithi113.stdout: cephadm-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-09T03:43:35.786 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.787 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-top-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-09T03:43:35.787 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-11-09T03:43:35.787 INFO:teuthology.orchestra.run.smithi113.stdout: flex-2.6.4-9.el9.x86_64 2023-11-09T03:43:35.787 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-11-09T03:43:35.787 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-11-09T03:43:35.788 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-11-09T03:43:35.788 INFO:teuthology.orchestra.run.smithi113.stdout: fuse-2.9.9-15.el9.x86_64 2023-11-09T03:43:35.788 INFO:teuthology.orchestra.run.smithi113.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-11-09T03:43:35.788 INFO:teuthology.orchestra.run.smithi113.stdout: jq-1.6-16.el9.x86_64 2023-11-09T03:43:35.788 INFO:teuthology.orchestra.run.smithi113.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-11-09T03:43:35.789 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-9.0.0-9.el9.x86_64 2023-11-09T03:43:35.789 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-11-09T03:43:35.789 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs-devel-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.789 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs2-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.789 INFO:teuthology.orchestra.run.smithi113.stdout: libcephsqlite-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.790 INFO:teuthology.orchestra.run.smithi113.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-11-09T03:43:35.790 INFO:teuthology.orchestra.run.smithi113.stdout: liboath-2.6.7-2.el9.x86_64 2023-11-09T03:43:35.790 INFO:teuthology.orchestra.run.smithi113.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-11-09T03:43:35.790 INFO:teuthology.orchestra.run.smithi113.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-11-09T03:43:35.790 INFO:teuthology.orchestra.run.smithi113.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-11-09T03:43:35.791 INFO:teuthology.orchestra.run.smithi113.stdout: librados-devel-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.791 INFO:teuthology.orchestra.run.smithi113.stdout: libradosstriper1-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.791 INFO:teuthology.orchestra.run.smithi113.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-11-09T03:43:35.791 INFO:teuthology.orchestra.run.smithi113.stdout: librgw2-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.791 INFO:teuthology.orchestra.run.smithi113.stdout: libtool-2.4.6-45.el9.x86_64 2023-11-09T03:43:35.792 INFO:teuthology.orchestra.run.smithi113.stdout: libunwind-1.6.2-1.el9.x86_64 2023-11-09T03:43:35.792 INFO:teuthology.orchestra.run.smithi113.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-11-09T03:43:35.792 INFO:teuthology.orchestra.run.smithi113.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-11-09T03:43:35.792 INFO:teuthology.orchestra.run.smithi113.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-11-09T03:43:35.792 INFO:teuthology.orchestra.run.smithi113.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-11-09T03:43:35.792 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-0.3.21-2.el9.x86_64 2023-11-09T03:43:35.793 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-11-09T03:43:35.793 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-11-09T03:43:35.793 INFO:teuthology.orchestra.run.smithi113.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-11-09T03:43:35.793 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-13.11-1.el9.x86_64 2023-11-09T03:43:35.793 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-11-09T03:43:35.794 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-11-09T03:43:35.794 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-server-13.11-1.el9.x86_64 2023-11-09T03:43:35.794 INFO:teuthology.orchestra.run.smithi113.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-11-09T03:43:35.794 INFO:teuthology.orchestra.run.smithi113.stdout: python3-babel-2.9.1-2.el9.noarch 2023-11-09T03:43:35.795 INFO:teuthology.orchestra.run.smithi113.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-11-09T03:43:35.795 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-11-09T03:43:35.795 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-argparse-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.795 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-common-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.795 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cephfs-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.795 INFO:teuthology.orchestra.run.smithi113.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-11-09T03:43:35.796 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-11-09T03:43:35.796 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-11-09T03:43:35.796 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-11-09T03:43:35.796 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-11-09T03:43:35.796 INFO:teuthology.orchestra.run.smithi113.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-11-09T03:43:35.797 INFO:teuthology.orchestra.run.smithi113.stdout: python3-google-auth-1:2.23.3-1.el9.noarch 2023-11-09T03:43:35.797 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-11-09T03:43:35.797 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-11-09T03:43:35.797 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-11-09T03:43:35.797 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-11-09T03:43:35.798 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-11-09T03:43:35.798 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-11-09T03:43:35.798 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-11-09T03:43:35.798 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-11-09T03:43:35.798 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-11-09T03:43:35.799 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-11-09T03:43:35.799 INFO:teuthology.orchestra.run.smithi113.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-11-09T03:43:35.799 INFO:teuthology.orchestra.run.smithi113.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-11-09T03:43:35.799 INFO:teuthology.orchestra.run.smithi113.stdout: python3-mako-1.1.4-6.el9.noarch 2023-11-09T03:43:35.799 INFO:teuthology.orchestra.run.smithi113.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-11-09T03:43:35.800 INFO:teuthology.orchestra.run.smithi113.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-11-09T03:43:35.800 INFO:teuthology.orchestra.run.smithi113.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-11-09T03:43:35.800 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-11-09T03:43:35.800 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-11-09T03:43:35.800 INFO:teuthology.orchestra.run.smithi113.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-11-09T03:43:35.801 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-11-09T03:43:35.801 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ply-3.11-14.el9.noarch 2023-11-09T03:43:35.801 INFO:teuthology.orchestra.run.smithi113.stdout: python3-portend-3.1.0-2.el9.noarch 2023-11-09T03:43:35.801 INFO:teuthology.orchestra.run.smithi113.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-11-09T03:43:35.801 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-11-09T03:43:35.802 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-11-09T03:43:35.802 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-11-09T03:43:35.802 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pycparser-2.20-6.el9.noarch 2023-11-09T03:43:35.802 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pytz-2021.1-5.el9.noarch 2023-11-09T03:43:35.802 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rados-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.803 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rbd-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.803 INFO:teuthology.orchestra.run.smithi113.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-11-09T03:43:35.803 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-2.25.1-7.el9.noarch 2023-11-09T03:43:35.803 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-11-09T03:43:35.803 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rgw-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.804 INFO:teuthology.orchestra.run.smithi113.stdout: python3-routes-2.5.1-5.el9.noarch 2023-11-09T03:43:35.804 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rsa-4.9-2.el9.noarch 2023-11-09T03:43:35.804 INFO:teuthology.orchestra.run.smithi113.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-11-09T03:43:35.804 INFO:teuthology.orchestra.run.smithi113.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-11-09T03:43:35.804 INFO:teuthology.orchestra.run.smithi113.stdout: python3-toml-0.10.2-6.el9.noarch 2023-11-09T03:43:35.804 INFO:teuthology.orchestra.run.smithi113.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-11-09T03:43:35.805 INFO:teuthology.orchestra.run.smithi113.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-11-09T03:43:35.805 INFO:teuthology.orchestra.run.smithi113.stdout: python3-webob-1.8.7-6.el9.noarch 2023-11-09T03:43:35.805 INFO:teuthology.orchestra.run.smithi113.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-11-09T03:43:35.805 INFO:teuthology.orchestra.run.smithi113.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-11-09T03:43:35.805 INFO:teuthology.orchestra.run.smithi113.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-11-09T03:43:35.806 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-fuse-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.806 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.806 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-nbd-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:43:35.806 INFO:teuthology.orchestra.run.smithi113.stdout: re2-1:20211101-3.el9.x86_64 2023-11-09T03:43:35.806 INFO:teuthology.orchestra.run.smithi113.stdout: socat-1.7.4.1-5.el9.x86_64 2023-11-09T03:43:35.807 INFO:teuthology.orchestra.run.smithi113.stdout: thrift-0.15.0-2.el9.x86_64 2023-11-09T03:43:35.807 INFO:teuthology.orchestra.run.smithi113.stdout: uuid-1.6.2-55.el9.x86_64 2023-11-09T03:43:35.807 INFO:teuthology.orchestra.run.smithi113.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-11-09T03:43:35.807 INFO:teuthology.orchestra.run.smithi113.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-11-09T03:43:35.808 INFO:teuthology.orchestra.run.smithi113.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-11-09T03:43:35.808 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:43:35.808 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2023-11-09T03:43:36.173 DEBUG:teuthology.parallel:result is None 2023-11-09T03:43:36.814 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2023-11-09T03:43:36.815 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2023-11-09T03:43:42.325 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2023-11-09T03:43:42.393 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2023-11-09T03:43:42.418 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2023-11-09T03:43:42.445 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2023-11-09T03:43:42.496 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/152 2023-11-09T03:43:42.820 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 5/152 2023-11-09T03:43:42.895 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : librados2-2:18.0.0-7035.g7532011a.el9.x86_64 6/152 2023-11-09T03:43:43.014 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librados2-2:18.0.0-7035.g7532011a.el9.x86_64 6/152 2023-11-09T03:43:43.038 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephfs2-2:18.0.0-7035.g7532011a.el9.x86_64 7/152 2023-11-09T03:43:43.111 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libcephfs2-2:18.0.0-7035.g7532011a.el9.x86_64 7/152 2023-11-09T03:43:43.163 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rados-2:18.0.0-7035.g7532011a.el9.x86_64 8/152 2023-11-09T03:43:43.202 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/152 2023-11-09T03:43:43.363 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 10/152 2023-11-09T03:43:43.419 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-13.11-1.el9.x86_64 11/152 2023-11-09T03:43:43.451 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephsqlite-2:18.0.0-7035.g7532011a.el9.x86_64 12/152 2023-11-09T03:43:43.536 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7035.g7532011a.el9.x86_64 12/152 2023-11-09T03:43:43.560 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libradosstriper1-2:18.0.0-7035.g7532011a.el9.x86 13/152 2023-11-09T03:43:43.605 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7035.g7532011a.el9.x86 13/152 2023-11-09T03:43:43.783 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 14/152 2023-11-09T03:43:43.906 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 15/152 2023-11-09T03:43:43.974 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 15/152 2023-11-09T03:43:44.076 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 16/152 2023-11-09T03:43:44.134 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : NetworkManager-1:1.45.6-1.el9.x86_64 16/152 2023-11-09T03:43:44.384 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 16/152 2023-11-09T03:43:44.422 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/152 2023-11-09T03:43:44.456 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/152 2023-11-09T03:43:44.533 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/152 2023-11-09T03:43:44.586 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 20/152 2023-11-09T03:43:44.625 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 21/152 2023-11-09T03:43:44.661 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 22/152 2023-11-09T03:43:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 23/152 2023-11-09T03:43:44.767 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/152 2023-11-09T03:43:44.794 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/152 2023-11-09T03:43:44.864 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/152 2023-11-09T03:43:44.899 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ceph-common-2:18.0.0-7035.g7532011a.el9. 27/152 2023-11-09T03:43:44.957 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ceph-argparse-2:18.0.0-7035.g7532011a.el 28/152 2023-11-09T03:43:45.042 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cephfs-2:18.0.0-7035.g7532011a.el9.x86_6 29/152 2023-11-09T03:43:45.184 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/152 2023-11-09T03:43:45.238 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/152 2023-11-09T03:43:45.787 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/152 2023-11-09T03:43:45.874 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/152 2023-11-09T03:43:45.920 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 34/152 2023-11-09T03:43:45.957 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librados-devel-2:18.0.0-7035.g7532011a.el9.x86_6 35/152 2023-11-09T03:43:45.987 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/152 2023-11-09T03:43:46.094 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : re2-1:20211101-3.el9.x86_64 37/152 2023-11-09T03:43:46.133 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 38/152 2023-11-09T03:43:46.189 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 39/152 2023-11-09T03:43:46.263 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 40/152 2023-11-09T03:43:46.292 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 41/152 2023-11-09T03:43:46.329 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 42/152 2023-11-09T03:43:46.375 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 43/152 2023-11-09T03:43:46.429 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 44/152 2023-11-09T03:43:46.479 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 45/152 2023-11-09T03:43:46.613 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 46/152 2023-11-09T03:43:46.656 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 47/152 2023-11-09T03:43:46.697 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 48/152 2023-11-09T03:43:46.792 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 49/152 2023-11-09T03:43:46.824 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-google-auth-1:2.23.3-1.el9.noarch 50/152 2023-11-09T03:43:46.864 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 51/152 2023-11-09T03:43:46.897 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 52/152 2023-11-09T03:43:46.923 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 53/152 2023-11-09T03:43:46.951 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 54/152 2023-11-09T03:43:47.038 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 55/152 2023-11-09T03:43:47.084 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : automake-1.16.2-8.el9.noarch 56/152 2023-11-09T03:43:47.132 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ply-3.11-14.el9.noarch 57/152 2023-11-09T03:43:47.188 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 58/152 2023-11-09T03:43:47.390 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 59/152 2023-11-09T03:43:47.446 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 60/152 2023-11-09T03:43:47.576 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 61/152 2023-11-09T03:43:47.624 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 62/152 2023-11-09T03:43:47.653 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 63/152 2023-11-09T03:43:47.687 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 64/152 2023-11-09T03:43:47.720 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 65/152 2023-11-09T03:43:47.753 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : fuse-2.9.9-15.el9.x86_64 66/152 2023-11-09T03:43:47.787 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 67/152 2023-11-09T03:43:47.839 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : uuid-1.6.2-55.el9.x86_64 68/152 2023-11-09T03:43:47.874 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 69/152 2023-11-09T03:43:47.904 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 70/152 2023-11-09T03:43:47.981 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 71/152 2023-11-09T03:43:48.023 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 72/152 2023-11-09T03:43:48.056 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/152 2023-11-09T03:43:48.098 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/152 2023-11-09T03:43:48.132 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/152 2023-11-09T03:43:48.222 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/152 2023-11-09T03:43:48.260 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 77/152 2023-11-09T03:43:48.362 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 78/152 2023-11-09T03:43:48.419 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2023-11-09T03:43:49.152 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2023-11-09T03:43:49.199 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2023-11-09T03:43:49.253 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2023-11-09T03:43:49.357 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2023-11-09T03:43:49.392 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 84/152 2023-11-09T03:43:49.569 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openblas-0.3.21-2.el9.x86_64 85/152 2023-11-09T03:43:49.598 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 86/152 2023-11-09T03:43:49.677 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 87/152 2023-11-09T03:43:50.346 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 88/152 2023-11-09T03:43:50.533 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 89/152 2023-11-09T03:43:51.814 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 90/152 2023-11-09T03:43:51.901 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 91/152 2023-11-09T03:43:51.932 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 92/152 2023-11-09T03:43:51.971 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : jq-1.6-16.el9.x86_64 93/152 2023-11-09T03:43:51.999 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2023-11-09T03:43:52.306 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 95/152 2023-11-09T03:43:52.337 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : librbd1-2:18.0.0-7035.g7532011a.el9.x86_64 96/152 2023-11-09T03:43:52.429 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librbd1-2:18.0.0-7035.g7532011a.el9.x86_64 96/152 2023-11-09T03:43:52.458 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rbd-2:18.0.0-7035.g7532011a.el9.x86_64 97/152 2023-11-09T03:43:52.495 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 98/152 2023-11-09T03:43:52.922 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 99/152 2023-11-09T03:43:52.948 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librgw2-2:18.0.0-7035.g7532011a.el9.x86_64 100/152 2023-11-09T03:43:53.020 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librgw2-2:18.0.0-7035.g7532011a.el9.x86_64 100/152 2023-11-09T03:43:53.050 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rgw-2:18.0.0-7035.g7532011a.el9.x86_64 101/152 2023-11-09T03:43:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-common-2:18.0.0-7035.g7532011a.el9.x86_64 102/152 2023-11-09T03:43:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-common-2:18.0.0-7035.g7532011a.el9.x86_64 102/152 2023-11-09T03:43:55.778 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-common-2:18.0.0-7035.g7532011a.el9.x86_64 102/152 2023-11-09T03:43:55.816 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-base-2:18.0.0-7035.g7532011a.el9.x86_64 103/152 2023-11-09T03:43:55.886 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-base-2:18.0.0-7035.g7532011a.el9.x86_64 103/152 2023-11-09T03:43:55.887 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-11-09T03:43:55.887 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-11-09T03:43:55.887 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:43:55.918 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-selinux-2:18.0.0-7035.g7532011a.el9.x86_64 104/152 2023-11-09T03:44:07.326 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7035.g7532011a.el9.x86_64 104/152 2023-11-09T03:44:07.326 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /sys 2023-11-09T03:44:07.327 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /proc 2023-11-09T03:44:07.327 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /mnt 2023-11-09T03:44:07.327 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /var/tmp 2023-11-09T03:44:07.327 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /home 2023-11-09T03:44:07.327 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /root 2023-11-09T03:44:07.328 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /tmp 2023-11-09T03:44:07.328 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:07.488 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-volume-2:18.0.0-7035.g7532011a.el9.noarch 105/152 2023-11-09T03:44:07.503 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-volume-2:18.0.0-7035.g7532011a.el9.noarch 105/152 2023-11-09T03:44:07.503 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-09T03:44:07.503 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-11-09T03:44:07.503 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:09.066 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-osd-2:18.0.0-7035.g7532011a.el9.x86_64 106/152 2023-11-09T03:44:09.091 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-osd-2:18.0.0-7035.g7532011a.el9.x86_64 106/152 2023-11-09T03:44:09.092 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-09T03:44:09.092 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-11-09T03:44:09.092 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-09T03:44:09.092 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-09T03:44:09.093 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:09.300 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mds-2:18.0.0-7035.g7532011a.el9.x86_64 107/152 2023-11-09T03:44:09.322 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mds-2:18.0.0-7035.g7532011a.el9.x86_64 107/152 2023-11-09T03:44:09.322 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-09T03:44:09.323 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-11-09T03:44:09.323 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-09T03:44:09.323 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-09T03:44:09.323 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:09.768 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mon-2:18.0.0-7035.g7532011a.el9.x86_64 108/152 2023-11-09T03:44:09.790 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mon-2:18.0.0-7035.g7532011a.el9.x86_64 108/152 2023-11-09T03:44:09.791 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-09T03:44:09.791 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-11-09T03:44:09.791 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-09T03:44:09.791 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-09T03:44:09.792 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:10.116 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: cephadm-2:18.0.0-7035.g7532011a.el9.noarch 109/152 2023-11-09T03:44:10.142 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephadm-2:18.0.0-7035.g7532011a.el9.noarch 109/152 2023-11-09T03:44:10.170 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7035.g7532011a.e 110/152 2023-11-09T03:44:10.232 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7035.g7532011a. 111/152 2023-11-09T03:44:10.273 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7035.g7532011a.el9.noa 112/152 2023-11-09T03:44:10.494 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7035.g7532011a.el9.noa 112/152 2023-11-09T03:44:10.524 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7035.g7532011a.el9.n 113/152 2023-11-09T03:44:11.485 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7035.g7532011a.el9.n 113/152 2023-11-09T03:44:11.516 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7035.g753 114/152 2023-11-09T03:44:11.616 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7035.g753 114/152 2023-11-09T03:44:11.764 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7035.g7532011a.el 115/152 2023-11-09T03:44:11.793 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-2:18.0.0-7035.g7532011a.el9.x86_64 116/152 2023-11-09T03:44:11.816 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7035.g7532011a.el9.x86_64 116/152 2023-11-09T03:44:11.817 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-09T03:44:11.817 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-11-09T03:44:11.817 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-09T03:44:11.817 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-09T03:44:11.817 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:11.869 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-rook-2:18.0.0-7035.g7532011a.el9.noarch 117/152 2023-11-09T03:44:11.879 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7035.g7532011a.el9.noarch 117/152 2023-11-09T03:44:12.677 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-2:18.0.0-7035.g7532011a.el9.x86_64 118/152 2023-11-09T03:44:12.712 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-radosgw-2:18.0.0-7035.g7532011a.el9.x86_64 119/152 2023-11-09T03:44:12.734 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7035.g7532011a.el9.x86_64 119/152 2023-11-09T03:44:12.734 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-09T03:44:12.735 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-11-09T03:44:12.735 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-09T03:44:12.735 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-09T03:44:12.735 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:12.771 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7035.g75320 120/152 2023-11-09T03:44:12.791 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7035.g75320 120/152 2023-11-09T03:44:12.792 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-09T03:44:12.792 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-11-09T03:44:12.792 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:12.838 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephfs-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 121/152 2023-11-09T03:44:12.860 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 121/152 2023-11-09T03:44:12.861 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-09T03:44:12.861 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-11-09T03:44:12.861 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-09T03:44:12.861 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-09T03:44:12.861 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:13.184 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 122/152 2023-11-09T03:44:13.206 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 122/152 2023-11-09T03:44:13.207 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-09T03:44:13.207 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-11-09T03:44:13.207 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-09T03:44:13.208 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-09T03:44:13.208 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:17.016 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-test-2:18.0.0-7035.g7532011a.el9.x86_64 123/152 2023-11-09T03:44:17.058 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-fuse-2:18.0.0-7035.g7532011a.el9.x86_64 124/152 2023-11-09T03:44:17.088 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-nbd-2:18.0.0-7035.g7532011a.el9.x86_64 125/152 2023-11-09T03:44:17.288 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 126/152 2023-11-09T03:44:17.395 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 127/152 2023-11-09T03:44:17.448 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-fuse-2:18.0.0-7035.g7532011a.el9.x86_64 128/152 2023-11-09T03:44:17.489 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libtool-2.4.6-45.el9.x86_64 129/152 2023-11-09T03:44:17.522 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 130/152 2023-11-09T03:44:17.544 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephfs-devel-2:18.0.0-7035.g7532011a.el9.x86_ 131/152 2023-11-09T03:44:17.582 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : NetworkManager-team-1:1.45.6-1.el9.x86_64 132/152 2023-11-09T03:44:17.688 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : NetworkManager-tui-1:1.45.6-1.el9.x86_64 133/152 2023-11-09T03:44:17.767 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 134/152 2023-11-09T03:44:17.800 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 135/152 2023-11-09T03:44:18.428 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 136/152 2023-11-09T03:44:18.466 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 136/152 2023-11-09T03:44:18.486 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 136/152 2023-11-09T03:44:18.541 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephfs-top-2:18.0.0-7035.g7532011a.el9.noarch 137/152 2023-11-09T03:44:19.406 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 138/152 2023-11-09T03:44:19.496 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 139/152 2023-11-09T03:44:19.537 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 140/152 2023-11-09T03:44:19.625 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flex-2.6.4-9.el9.x86_64 141/152 2023-11-09T03:44:19.658 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : bison-3.7.4-5.el9.x86_64 142/152 2023-11-09T03:44:19.684 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 143/152 2023-11-09T03:44:19.684 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 144/152 2023-11-09T03:44:19.768 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 144/152 2023-11-09T03:44:19.793 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 145/152 2023-11-09T03:44:19.818 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 146/152 2023-11-09T03:44:19.844 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 147/152 2023-11-09T03:44:19.850 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 148/152 2023-11-09T03:44:19.850 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 148/152 2023-11-09T03:44:20.030 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 148/152 2023-11-09T03:44:20.030 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 149/152 2023-11-09T03:44:20.113 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 149/152 2023-11-09T03:44:20.137 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 150/152 2023-11-09T03:44:20.138 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 151/152 2023-11-09T03:44:20.223 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 151/152 2023-11-09T03:44:20.288 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 152/152 2023-11-09T03:44:24.974 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 152/152 2023-11-09T03:44:24.974 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-2:18.0.0-7035.g7532011a.el9.x86_64 1/152 2023-11-09T03:44:24.974 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-base-2:18.0.0-7035.g7532011a.el9.x86_64 2/152 2023-11-09T03:44:24.975 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-common-2:18.0.0-7035.g7532011a.el9.x86_64 3/152 2023-11-09T03:44:24.975 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-fuse-2:18.0.0-7035.g7532011a.el9.x86_64 4/152 2023-11-09T03:44:24.975 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7035.g75320 5/152 2023-11-09T03:44:24.975 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mds-2:18.0.0-7035.g7532011a.el9.x86_64 6/152 2023-11-09T03:44:24.975 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-2:18.0.0-7035.g7532011a.el9.x86_64 7/152 2023-11-09T03:44:24.976 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mon-2:18.0.0-7035.g7532011a.el9.x86_64 8/152 2023-11-09T03:44:24.976 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-osd-2:18.0.0-7035.g7532011a.el9.x86_64 9/152 2023-11-09T03:44:24.976 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-radosgw-2:18.0.0-7035.g7532011a.el9.x86_64 10/152 2023-11-09T03:44:24.976 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-selinux-2:18.0.0-7035.g7532011a.el9.x86_64 11/152 2023-11-09T03:44:24.976 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-test-2:18.0.0-7035.g7532011a.el9.x86_64 12/152 2023-11-09T03:44:24.977 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephfs-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 13/152 2023-11-09T03:44:24.977 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephfs-devel-2:18.0.0-7035.g7532011a.el9.x86_ 14/152 2023-11-09T03:44:24.977 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephfs2-2:18.0.0-7035.g7532011a.el9.x86_64 15/152 2023-11-09T03:44:24.977 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephsqlite-2:18.0.0-7035.g7532011a.el9.x86_64 16/152 2023-11-09T03:44:24.977 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados-devel-2:18.0.0-7035.g7532011a.el9.x86_6 17/152 2023-11-09T03:44:24.978 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libradosstriper1-2:18.0.0-7035.g7532011a.el9.x86 18/152 2023-11-09T03:44:24.978 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librgw2-2:18.0.0-7035.g7532011a.el9.x86_64 19/152 2023-11-09T03:44:24.978 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7035.g7532011a.el 20/152 2023-11-09T03:44:24.978 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ceph-common-2:18.0.0-7035.g7532011a.el9. 21/152 2023-11-09T03:44:24.978 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cephfs-2:18.0.0-7035.g7532011a.el9.x86_6 22/152 2023-11-09T03:44:24.979 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rados-2:18.0.0-7035.g7532011a.el9.x86_64 23/152 2023-11-09T03:44:24.979 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rbd-2:18.0.0-7035.g7532011a.el9.x86_64 24/152 2023-11-09T03:44:24.979 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rgw-2:18.0.0-7035.g7532011a.el9.x86_64 25/152 2023-11-09T03:44:24.979 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-fuse-2:18.0.0-7035.g7532011a.el9.x86_64 26/152 2023-11-09T03:44:24.979 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 27/152 2023-11-09T03:44:24.980 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-nbd-2:18.0.0-7035.g7532011a.el9.x86_64 28/152 2023-11-09T03:44:24.980 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7035.g7532011a. 29/152 2023-11-09T03:44:24.991 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7035.g7532011a.el9.noa 30/152 2023-11-09T03:44:24.991 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7035.g7532011a.el9.n 31/152 2023-11-09T03:44:24.991 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7035.g753 32/152 2023-11-09T03:44:24.992 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7035.g7532011a.el 33/152 2023-11-09T03:44:24.992 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7035.g7532011a.el9.noarch 34/152 2023-11-09T03:44:24.992 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7035.g7532011a.e 35/152 2023-11-09T03:44:24.992 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-volume-2:18.0.0-7035.g7532011a.el9.noarch 36/152 2023-11-09T03:44:24.992 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephadm-2:18.0.0-7035.g7532011a.el9.noarch 37/152 2023-11-09T03:44:24.993 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephfs-top-2:18.0.0-7035.g7532011a.el9.noarch 38/152 2023-11-09T03:44:24.993 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : bison-3.7.4-5.el9.x86_64 39/152 2023-11-09T03:44:24.993 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 40/152 2023-11-09T03:44:24.993 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/152 2023-11-09T03:44:24.993 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/152 2023-11-09T03:44:24.994 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/152 2023-11-09T03:44:24.994 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/152 2023-11-09T03:44:24.994 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : jq-1.6-16.el9.x86_64 45/152 2023-11-09T03:44:24.994 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 46/152 2023-11-09T03:44:24.994 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 47/152 2023-11-09T03:44:24.995 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 48/152 2023-11-09T03:44:24.995 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 49/152 2023-11-09T03:44:24.995 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 50/152 2023-11-09T03:44:24.995 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 51/152 2023-11-09T03:44:24.995 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 52/152 2023-11-09T03:44:24.996 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 53/152 2023-11-09T03:44:24.996 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 54/152 2023-11-09T03:44:24.996 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-13.11-1.el9.x86_64 55/152 2023-11-09T03:44:24.996 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 56/152 2023-11-09T03:44:24.996 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 57/152 2023-11-09T03:44:24.997 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 58/152 2023-11-09T03:44:24.997 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 59/152 2023-11-09T03:44:24.997 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 60/152 2023-11-09T03:44:24.997 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 61/152 2023-11-09T03:44:24.997 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 62/152 2023-11-09T03:44:24.998 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 63/152 2023-11-09T03:44:24.998 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 64/152 2023-11-09T03:44:24.998 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 65/152 2023-11-09T03:44:24.998 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 66/152 2023-11-09T03:44:24.998 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 67/152 2023-11-09T03:44:24.999 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 68/152 2023-11-09T03:44:24.999 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/152 2023-11-09T03:44:24.999 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 70/152 2023-11-09T03:44:24.999 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 71/152 2023-11-09T03:44:25.000 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 72/152 2023-11-09T03:44:25.000 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 73/152 2023-11-09T03:44:25.000 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 74/152 2023-11-09T03:44:25.000 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 75/152 2023-11-09T03:44:25.000 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 76/152 2023-11-09T03:44:25.001 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 77/152 2023-11-09T03:44:25.001 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 78/152 2023-11-09T03:44:25.001 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 79/152 2023-11-09T03:44:25.001 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 80/152 2023-11-09T03:44:25.002 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 81/152 2023-11-09T03:44:25.002 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ply-3.11-14.el9.noarch 82/152 2023-11-09T03:44:25.002 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 83/152 2023-11-09T03:44:25.002 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 84/152 2023-11-09T03:44:25.002 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 85/152 2023-11-09T03:44:25.003 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 86/152 2023-11-09T03:44:25.003 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : automake-1.16.2-8.el9.noarch 87/152 2023-11-09T03:44:25.003 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 88/152 2023-11-09T03:44:25.003 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 89/152 2023-11-09T03:44:25.003 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 90/152 2023-11-09T03:44:25.004 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 91/152 2023-11-09T03:44:25.004 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/152 2023-11-09T03:44:25.004 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 93/152 2023-11-09T03:44:25.004 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 94/152 2023-11-09T03:44:25.004 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 95/152 2023-11-09T03:44:25.004 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 96/152 2023-11-09T03:44:25.005 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 97/152 2023-11-09T03:44:25.005 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/152 2023-11-09T03:44:25.005 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/152 2023-11-09T03:44:25.005 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 100/152 2023-11-09T03:44:25.005 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 101/152 2023-11-09T03:44:25.006 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/152 2023-11-09T03:44:25.006 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/152 2023-11-09T03:44:25.006 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 104/152 2023-11-09T03:44:25.006 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/152 2023-11-09T03:44:25.006 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/152 2023-11-09T03:44:25.007 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-google-auth-1:2.23.3-1.el9.noarch 107/152 2023-11-09T03:44:25.007 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/152 2023-11-09T03:44:25.007 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/152 2023-11-09T03:44:25.007 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/152 2023-11-09T03:44:25.007 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/152 2023-11-09T03:44:25.008 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/152 2023-11-09T03:44:25.008 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 113/152 2023-11-09T03:44:25.008 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 114/152 2023-11-09T03:44:25.008 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/152 2023-11-09T03:44:25.008 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/152 2023-11-09T03:44:25.009 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/152 2023-11-09T03:44:25.009 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/152 2023-11-09T03:44:25.009 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/152 2023-11-09T03:44:25.009 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/152 2023-11-09T03:44:25.009 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/152 2023-11-09T03:44:25.009 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/152 2023-11-09T03:44:25.010 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/152 2023-11-09T03:44:25.010 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/152 2023-11-09T03:44:25.010 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/152 2023-11-09T03:44:25.010 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/152 2023-11-09T03:44:25.010 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/152 2023-11-09T03:44:25.011 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/152 2023-11-09T03:44:25.011 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/152 2023-11-09T03:44:25.011 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/152 2023-11-09T03:44:25.011 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/152 2023-11-09T03:44:25.011 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/152 2023-11-09T03:44:25.012 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados2-2:18.0.0-7035.g7532011a.el9.x86_64 133/152 2023-11-09T03:44:25.012 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/152 2023-11-09T03:44:25.012 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librbd1-2:18.0.0-7035.g7532011a.el9.x86_64 135/152 2023-11-09T03:44:25.012 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/152 2023-11-09T03:44:25.012 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 137/152 2023-11-09T03:44:25.012 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 138/152 2023-11-09T03:44:25.013 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-1:1.45.6-1.el9.x86_64 139/152 2023-11-09T03:44:25.013 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/152 2023-11-09T03:44:25.013 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 141/152 2023-11-09T03:44:25.013 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/152 2023-11-09T03:44:25.013 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-team-1:1.45.6-1.el9.x86_64 143/152 2023-11-09T03:44:25.014 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/152 2023-11-09T03:44:25.014 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-tui-1:1.45.6-1.el9.x86_64 145/152 2023-11-09T03:44:25.014 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/152 2023-11-09T03:44:25.014 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/152 2023-11-09T03:44:25.014 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/152 2023-11-09T03:44:25.015 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 149/152 2023-11-09T03:44:25.015 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 150/152 2023-11-09T03:44:25.015 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 151/152 2023-11-09T03:44:27.807 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 152/152 2023-11-09T03:44:27.807 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:27.807 INFO:teuthology.orchestra.run.smithi076.stdout:Upgraded: 2023-11-09T03:44:27.807 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-1:1.45.6-1.el9.x86_64 2023-11-09T03:44:27.808 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 2023-11-09T03:44:27.808 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-team-1:1.45.6-1.el9.x86_64 2023-11-09T03:44:27.808 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-tui-1:1.45.6-1.el9.x86_64 2023-11-09T03:44:27.808 INFO:teuthology.orchestra.run.smithi076.stdout: acl-2.3.1-4.el9.x86_64 2023-11-09T03:44:27.809 INFO:teuthology.orchestra.run.smithi076.stdout: libacl-2.3.1-4.el9.x86_64 2023-11-09T03:44:27.809 INFO:teuthology.orchestra.run.smithi076.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-11-09T03:44:27.809 INFO:teuthology.orchestra.run.smithi076.stdout: librados2-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.809 INFO:teuthology.orchestra.run.smithi076.stdout: librbd1-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.809 INFO:teuthology.orchestra.run.smithi076.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-11-09T03:44:27.809 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2023-11-09T03:44:27.810 INFO:teuthology.orchestra.run.smithi076.stdout: automake-1.16.2-8.el9.noarch 2023-11-09T03:44:27.810 INFO:teuthology.orchestra.run.smithi076.stdout: bison-3.7.4-5.el9.x86_64 2023-11-09T03:44:27.810 INFO:teuthology.orchestra.run.smithi076.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-11-09T03:44:27.810 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.810 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-base-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.811 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-common-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.811 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-fuse-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.811 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-grafana-dashboards-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-09T03:44:27.811 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-immutable-object-cache-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.811 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mds-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.812 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.812 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-cephadm-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-09T03:44:27.812 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-dashboard-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-09T03:44:27.812 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-09T03:44:27.812 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-modules-core-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-09T03:44:27.813 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-rook-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-09T03:44:27.813 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mon-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.813 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-osd-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.813 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-prometheus-alerts-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-09T03:44:27.813 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-radosgw-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.814 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-selinux-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.814 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-test-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.814 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-volume-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-09T03:44:27.814 INFO:teuthology.orchestra.run.smithi076.stdout: cephadm-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-09T03:44:27.814 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.815 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-top-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-09T03:44:27.815 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-11-09T03:44:27.815 INFO:teuthology.orchestra.run.smithi076.stdout: flex-2.6.4-9.el9.x86_64 2023-11-09T03:44:27.815 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-11-09T03:44:27.815 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-11-09T03:44:27.816 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-11-09T03:44:27.816 INFO:teuthology.orchestra.run.smithi076.stdout: fuse-2.9.9-15.el9.x86_64 2023-11-09T03:44:27.816 INFO:teuthology.orchestra.run.smithi076.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-11-09T03:44:27.816 INFO:teuthology.orchestra.run.smithi076.stdout: jq-1.6-16.el9.x86_64 2023-11-09T03:44:27.816 INFO:teuthology.orchestra.run.smithi076.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-11-09T03:44:27.817 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-9.0.0-9.el9.x86_64 2023-11-09T03:44:27.817 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-11-09T03:44:27.817 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs-devel-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.817 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs2-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.817 INFO:teuthology.orchestra.run.smithi076.stdout: libcephsqlite-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.818 INFO:teuthology.orchestra.run.smithi076.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-11-09T03:44:27.818 INFO:teuthology.orchestra.run.smithi076.stdout: liboath-2.6.7-2.el9.x86_64 2023-11-09T03:44:27.818 INFO:teuthology.orchestra.run.smithi076.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-11-09T03:44:27.818 INFO:teuthology.orchestra.run.smithi076.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-11-09T03:44:27.818 INFO:teuthology.orchestra.run.smithi076.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-11-09T03:44:27.819 INFO:teuthology.orchestra.run.smithi076.stdout: librados-devel-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.819 INFO:teuthology.orchestra.run.smithi076.stdout: libradosstriper1-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.819 INFO:teuthology.orchestra.run.smithi076.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-11-09T03:44:27.819 INFO:teuthology.orchestra.run.smithi076.stdout: librgw2-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.819 INFO:teuthology.orchestra.run.smithi076.stdout: libtool-2.4.6-45.el9.x86_64 2023-11-09T03:44:27.820 INFO:teuthology.orchestra.run.smithi076.stdout: libunwind-1.6.2-1.el9.x86_64 2023-11-09T03:44:27.820 INFO:teuthology.orchestra.run.smithi076.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-11-09T03:44:27.820 INFO:teuthology.orchestra.run.smithi076.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-11-09T03:44:27.820 INFO:teuthology.orchestra.run.smithi076.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-11-09T03:44:27.820 INFO:teuthology.orchestra.run.smithi076.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-11-09T03:44:27.821 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-0.3.21-2.el9.x86_64 2023-11-09T03:44:27.821 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-11-09T03:44:27.821 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-11-09T03:44:27.821 INFO:teuthology.orchestra.run.smithi076.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-11-09T03:44:27.821 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-13.11-1.el9.x86_64 2023-11-09T03:44:27.822 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-11-09T03:44:27.822 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-11-09T03:44:27.822 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-server-13.11-1.el9.x86_64 2023-11-09T03:44:27.822 INFO:teuthology.orchestra.run.smithi076.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-11-09T03:44:27.822 INFO:teuthology.orchestra.run.smithi076.stdout: python3-babel-2.9.1-2.el9.noarch 2023-11-09T03:44:27.823 INFO:teuthology.orchestra.run.smithi076.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-11-09T03:44:27.823 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-11-09T03:44:27.823 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-argparse-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.823 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-common-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.823 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cephfs-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.824 INFO:teuthology.orchestra.run.smithi076.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-11-09T03:44:27.824 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-11-09T03:44:27.824 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-11-09T03:44:27.824 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-11-09T03:44:27.824 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-11-09T03:44:27.824 INFO:teuthology.orchestra.run.smithi076.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-11-09T03:44:27.825 INFO:teuthology.orchestra.run.smithi076.stdout: python3-google-auth-1:2.23.3-1.el9.noarch 2023-11-09T03:44:27.825 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-11-09T03:44:27.825 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-11-09T03:44:27.825 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-11-09T03:44:27.825 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-11-09T03:44:27.826 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-11-09T03:44:27.826 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-11-09T03:44:27.826 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-11-09T03:44:27.826 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-11-09T03:44:27.826 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-11-09T03:44:27.826 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-11-09T03:44:27.827 INFO:teuthology.orchestra.run.smithi076.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-11-09T03:44:27.827 INFO:teuthology.orchestra.run.smithi076.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-11-09T03:44:27.827 INFO:teuthology.orchestra.run.smithi076.stdout: python3-mako-1.1.4-6.el9.noarch 2023-11-09T03:44:27.827 INFO:teuthology.orchestra.run.smithi076.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-11-09T03:44:27.827 INFO:teuthology.orchestra.run.smithi076.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-11-09T03:44:27.828 INFO:teuthology.orchestra.run.smithi076.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-11-09T03:44:27.828 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-11-09T03:44:27.828 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-11-09T03:44:27.828 INFO:teuthology.orchestra.run.smithi076.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-11-09T03:44:27.828 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-11-09T03:44:27.829 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ply-3.11-14.el9.noarch 2023-11-09T03:44:27.829 INFO:teuthology.orchestra.run.smithi076.stdout: python3-portend-3.1.0-2.el9.noarch 2023-11-09T03:44:27.829 INFO:teuthology.orchestra.run.smithi076.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-11-09T03:44:27.829 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-11-09T03:44:27.829 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-11-09T03:44:27.830 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-11-09T03:44:27.830 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pycparser-2.20-6.el9.noarch 2023-11-09T03:44:27.830 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pytz-2021.1-5.el9.noarch 2023-11-09T03:44:27.830 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rados-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.830 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rbd-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.831 INFO:teuthology.orchestra.run.smithi076.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-11-09T03:44:27.831 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-2.25.1-7.el9.noarch 2023-11-09T03:44:27.831 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-11-09T03:44:27.831 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rgw-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.831 INFO:teuthology.orchestra.run.smithi076.stdout: python3-routes-2.5.1-5.el9.noarch 2023-11-09T03:44:27.832 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rsa-4.9-2.el9.noarch 2023-11-09T03:44:27.832 INFO:teuthology.orchestra.run.smithi076.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-11-09T03:44:27.832 INFO:teuthology.orchestra.run.smithi076.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-11-09T03:44:27.832 INFO:teuthology.orchestra.run.smithi076.stdout: python3-toml-0.10.2-6.el9.noarch 2023-11-09T03:44:27.832 INFO:teuthology.orchestra.run.smithi076.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-11-09T03:44:27.833 INFO:teuthology.orchestra.run.smithi076.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-11-09T03:44:27.833 INFO:teuthology.orchestra.run.smithi076.stdout: python3-webob-1.8.7-6.el9.noarch 2023-11-09T03:44:27.833 INFO:teuthology.orchestra.run.smithi076.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-11-09T03:44:27.833 INFO:teuthology.orchestra.run.smithi076.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-11-09T03:44:27.833 INFO:teuthology.orchestra.run.smithi076.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-11-09T03:44:27.833 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-fuse-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.834 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.834 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-nbd-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-09T03:44:27.834 INFO:teuthology.orchestra.run.smithi076.stdout: re2-1:20211101-3.el9.x86_64 2023-11-09T03:44:27.834 INFO:teuthology.orchestra.run.smithi076.stdout: socat-1.7.4.1-5.el9.x86_64 2023-11-09T03:44:27.834 INFO:teuthology.orchestra.run.smithi076.stdout: thrift-0.15.0-2.el9.x86_64 2023-11-09T03:44:27.835 INFO:teuthology.orchestra.run.smithi076.stdout: uuid-1.6.2-55.el9.x86_64 2023-11-09T03:44:27.835 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-11-09T03:44:27.835 INFO:teuthology.orchestra.run.smithi076.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-11-09T03:44:27.835 INFO:teuthology.orchestra.run.smithi076.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-11-09T03:44:27.836 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:27.836 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2023-11-09T03:44:28.275 DEBUG:teuthology.parallel:result is None 2023-11-09T03:44:28.276 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4 2023-11-09T03:44:28.500 DEBUG:teuthology.orchestra.run.smithi076:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-09T03:44:28.522 INFO:teuthology.orchestra.run.smithi076.stdout:18.0.0-7035.g7532011a.el9 2023-11-09T03:44:28.523 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7035.g7532011a.el9 2023-11-09T03:44:28.523 INFO:teuthology.task.install:The correct ceph version 18.0.0-7035.g7532011a is installed. 2023-11-09T03:44:28.524 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4 2023-11-09T03:44:28.739 DEBUG:teuthology.orchestra.run.smithi113:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-09T03:44:28.764 INFO:teuthology.orchestra.run.smithi113.stdout:18.0.0-7035.g7532011a.el9 2023-11-09T03:44:28.765 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7035.g7532011a.el9 2023-11-09T03:44:28.765 INFO:teuthology.task.install:The correct ceph version 18.0.0-7035.g7532011a is installed. 2023-11-09T03:44:28.766 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-09T03:44:28.779 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:44:28.779 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-09T03:44:28.812 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:44:28.812 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-09T03:44:28.845 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-09T03:44:28.846 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:44:28.846 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/daemon-helper 2023-11-09T03:44:28.879 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-09T03:44:28.947 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:44:28.947 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/daemon-helper 2023-11-09T03:44:28.976 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-09T03:44:29.045 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-09T03:44:29.045 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:44:29.046 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-09T03:44:29.073 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-09T03:44:29.139 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:44:29.140 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-09T03:44:29.167 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-09T03:44:29.235 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-09T03:44:29.236 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:44:29.236 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/stdin-killer 2023-11-09T03:44:29.263 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-09T03:44:29.329 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:44:29.330 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/stdin-killer 2023-11-09T03:44:29.356 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-09T03:44:29.423 INFO:teuthology.run_tasks:Running task ceph... 2023-11-09T03:44:29.678 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-09T03:44:29.679 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 777 /var/log/ceph 2023-11-09T03:44:29.681 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 777 /var/log/ceph 2023-11-09T03:44:29.708 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-09T03:44:29.708 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-09T03:44:29.748 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-09T03:44:29.775 INFO:tasks.ceph:Creating extra log directories... 2023-11-09T03:44:29.775 DEBUG:teuthology.orchestra.run.smithi076:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-09T03:44:29.815 DEBUG:teuthology.orchestra.run.smithi113:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-09T03:44:29.853 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-09T03:44:29.876 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-09T03:44:29.888 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-09T03:44:29.888 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:44:29.889 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-09T03:44:29.904 DEBUG:teuthology.orchestra.run.smithi076:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-09T03:44:30.022 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-09T03:44:30.049 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:44:30.050 DEBUG:teuthology.orchestra.run.smithi113:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-09T03:44:30.065 DEBUG:teuthology.orchestra.run.smithi113:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-09T03:44:30.180 DEBUG:teuthology.orchestra.run.smithi113:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-09T03:44:30.213 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-09T03:44:30.213 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'RECENT_CRASH'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-09T03:44:30.213 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2023-11-09_03:21:01-fs:functional-main-distro-default-smithi/7452464', 'branch': 'main', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snapshots}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7452464', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2023-11-09_03:21:01-fs:functional-main-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'RECENT_CRASH'], 'sha1': '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'check-counter': {'dry_run': True}, 'install': {'ceph': {'flavor': 'default', 'sha1': '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-batrick-testing-20231108.195322', 'sha1': '8dae6b9fa1760040e7156daa9d03f77642df4f90'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 50, 'repo': 'https://github.com/ceph/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 5927, 'sha1': '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-batrick-testing-20231108.195322', 'suite_path': '/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/batrick/ceph.git', 'suite_sha1': '8dae6b9fa1760040e7156daa9d03f77642df4f90', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi076.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBSaOfkkKDMrAw8lm/2WHJKaNe6HUq/VfeTji9wjOUR84c6zvChUurbrU6h/utzarVeqdBjqZcioe/t0+54KMAU=', 'smithi113.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK1EN9hrd6xzeNxf/hBCFS2ygei1vHqiKEq9qLM07K+yU9nQIGDVWXPapz/JUuXhIDcePlEle4hhoRzs9FOUUMg='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'RECENT_CRASH'], 'sha1': '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'exec': {'mon.a': ['ceph config set mgr mgr/crash/warn_recent_interval 0']}}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_snapshots']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '6899cd26fceddb2fec83dc1a1349394b28c8998e', 'timestamp': '2023-11-09_03:21:01', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3590714'} 2023-11-09T03:44:30.214 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-09T03:44:30.217 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-09T03:44:30.262 DEBUG:teuthology.orchestra.run.smithi076:> sudo install -d -m0777 -- /var/run/ceph 2023-11-09T03:44:30.272 DEBUG:teuthology.orchestra.run.smithi113:> sudo install -d -m0777 -- /var/run/ceph 2023-11-09T03:44:30.329 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:44:30.329 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/scratch_devs of=/dev/stdout 2023-11-09T03:44:30.352 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-09T03:44:30.352 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_1 2023-11-09T03:44:30.406 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-09T03:44:30.406 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-09T03:44:30.407 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 662 Links: 1 2023-11-09T03:44:30.407 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-09T03:44:30.407 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2023-11-09T03:44:30.408 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2023-11-09 03:44:20.624034296 +0000 2023-11-09T03:44:30.408 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2023-11-09 03:41:36.191878683 +0000 2023-11-09T03:44:30.408 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2023-11-09 03:41:36.191878683 +0000 2023-11-09T03:44:30.408 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2023-11-09 03:41:36.191878683 +0000 2023-11-09T03:44:30.409 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-09T03:44:30.472 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2023-11-09T03:44:30.472 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2023-11-09T03:44:30.472 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000210448 s, 2.4 MB/s 2023-11-09T03:44:30.473 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-09T03:44:30.529 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_2 2023-11-09T03:44:30.583 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-09T03:44:30.583 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-09T03:44:30.583 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 681 Links: 1 2023-11-09T03:44:30.584 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-09T03:44:30.584 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2023-11-09T03:44:30.584 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2023-11-09 03:44:20.624034296 +0000 2023-11-09T03:44:30.584 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2023-11-09 03:41:36.688863200 +0000 2023-11-09T03:44:30.585 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2023-11-09 03:41:36.688863200 +0000 2023-11-09T03:44:30.585 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2023-11-09 03:41:36.688863200 +0000 2023-11-09T03:44:30.585 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-09T03:44:30.649 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2023-11-09T03:44:30.650 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2023-11-09T03:44:30.650 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000210931 s, 2.4 MB/s 2023-11-09T03:44:30.651 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-09T03:44:30.706 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_3 2023-11-09T03:44:30.760 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-09T03:44:30.760 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-09T03:44:30.761 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 700 Links: 1 2023-11-09T03:44:30.761 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-09T03:44:30.761 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2023-11-09T03:44:30.761 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2023-11-09 03:44:20.624034296 +0000 2023-11-09T03:44:30.761 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2023-11-09 03:41:37.187847655 +0000 2023-11-09T03:44:30.762 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2023-11-09 03:41:37.187847655 +0000 2023-11-09T03:44:30.762 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2023-11-09 03:41:37.187847655 +0000 2023-11-09T03:44:30.762 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-09T03:44:30.826 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2023-11-09T03:44:30.826 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2023-11-09T03:44:30.826 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000187103 s, 2.7 MB/s 2023-11-09T03:44:30.827 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-09T03:44:30.883 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_4 2023-11-09T03:44:30.937 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-09T03:44:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-09T03:44:30.938 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 719 Links: 1 2023-11-09T03:44:30.938 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-09T03:44:30.938 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2023-11-09T03:44:30.938 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2023-11-09 03:44:20.625034265 +0000 2023-11-09T03:44:30.939 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2023-11-09 03:41:37.691831954 +0000 2023-11-09T03:44:30.939 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2023-11-09 03:41:37.691831954 +0000 2023-11-09T03:44:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2023-11-09 03:41:37.691831954 +0000 2023-11-09T03:44:30.941 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-09T03:44:31.003 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2023-11-09T03:44:31.003 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2023-11-09T03:44:31.003 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000235507 s, 2.2 MB/s 2023-11-09T03:44:31.004 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-09T03:44:31.060 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-09T03:44:31.061 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:44:31.061 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/scratch_devs of=/dev/stdout 2023-11-09T03:44:31.078 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-09T03:44:31.078 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_1 2023-11-09T03:44:31.134 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-09T03:44:31.134 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-09T03:44:31.134 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 663 Links: 1 2023-11-09T03:44:31.134 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-09T03:44:31.135 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2023-11-09T03:44:31.135 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2023-11-09 03:43:28.990724650 +0000 2023-11-09T03:44:31.135 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2023-11-09 03:41:36.193858428 +0000 2023-11-09T03:44:31.135 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2023-11-09 03:41:36.193858428 +0000 2023-11-09T03:44:31.135 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2023-11-09 03:41:36.193858428 +0000 2023-11-09T03:44:31.136 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-09T03:44:31.200 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2023-11-09T03:44:31.201 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2023-11-09T03:44:31.201 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000196842 s, 2.6 MB/s 2023-11-09T03:44:31.202 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-09T03:44:31.258 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_2 2023-11-09T03:44:31.314 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-09T03:44:31.314 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-09T03:44:31.314 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 682 Links: 1 2023-11-09T03:44:31.314 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-09T03:44:31.315 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2023-11-09T03:44:31.315 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2023-11-09 03:43:28.990724650 +0000 2023-11-09T03:44:31.315 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2023-11-09 03:41:36.693847809 +0000 2023-11-09T03:44:31.315 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2023-11-09 03:41:36.693847809 +0000 2023-11-09T03:44:31.315 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2023-11-09 03:41:36.693847809 +0000 2023-11-09T03:44:31.316 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-09T03:44:31.380 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2023-11-09T03:44:31.380 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2023-11-09T03:44:31.380 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000241916 s, 2.1 MB/s 2023-11-09T03:44:31.381 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-09T03:44:31.437 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_3 2023-11-09T03:44:31.493 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-09T03:44:31.493 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-09T03:44:31.493 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 703 Links: 1 2023-11-09T03:44:31.493 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-09T03:44:31.494 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2023-11-09T03:44:31.494 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2023-11-09 03:43:28.990724650 +0000 2023-11-09T03:44:31.494 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2023-11-09 03:41:37.193837190 +0000 2023-11-09T03:44:31.494 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2023-11-09 03:41:37.193837190 +0000 2023-11-09T03:44:31.494 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2023-11-09 03:41:37.193837190 +0000 2023-11-09T03:44:31.495 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-09T03:44:31.558 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2023-11-09T03:44:31.558 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2023-11-09T03:44:31.559 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000229842 s, 2.2 MB/s 2023-11-09T03:44:31.562 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-09T03:44:31.618 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_4 2023-11-09T03:44:31.673 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-09T03:44:31.673 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-09T03:44:31.673 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 719 Links: 1 2023-11-09T03:44:31.674 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-09T03:44:31.674 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2023-11-09T03:44:31.674 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2023-11-09 03:43:28.991724629 +0000 2023-11-09T03:44:31.674 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2023-11-09 03:41:37.708826252 +0000 2023-11-09T03:44:31.675 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2023-11-09 03:41:37.708826252 +0000 2023-11-09T03:44:31.675 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2023-11-09 03:41:37.708826252 +0000 2023-11-09T03:44:31.675 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-09T03:44:31.738 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2023-11-09T03:44:31.739 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2023-11-09T03:44:31.739 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000185938 s, 2.8 MB/s 2023-11-09T03:44:31.740 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-09T03:44:31.794 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-09T03:44:31.795 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi076.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi113.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-09T03:44:31.795 INFO:tasks.ceph:Generating config... 2023-11-09T03:44:31.815 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-09T03:44:31.815 INFO:tasks.ceph:[client] debug client = 20 2023-11-09T03:44:31.815 INFO:tasks.ceph:[client] debug ms = 1 2023-11-09T03:44:31.815 INFO:tasks.ceph:[client] pid file = 2023-11-09T03:44:31.816 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-09T03:44:31.816 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-09T03:44:31.816 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-09T03:44:31.816 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-09T03:44:31.817 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-09T03:44:31.817 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-09T03:44:31.817 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-09T03:44:31.817 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-09T03:44:31.818 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-09T03:44:31.818 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-09T03:44:31.818 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-09T03:44:31.818 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-09T03:44:31.819 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-09T03:44:31.819 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-09T03:44:31.819 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-09T03:44:31.819 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-09T03:44:31.820 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-09T03:44:31.820 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-09T03:44:31.820 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-09T03:44:31.820 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-09T03:44:31.821 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-11-09T03:44:31.821 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-11-09T03:44:31.821 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-09T03:44:31.821 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-09T03:44:31.822 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-11-09T03:44:31.822 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-09T03:44:31.822 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-09T03:44:31.822 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-09T03:44:31.823 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-09T03:44:31.823 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-09T03:44:31.823 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-09T03:44:31.823 INFO:tasks.ceph:Setting up mon.a... 2023-11-09T03:44:31.824 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-09T03:44:31.876 INFO:teuthology.orchestra.run.smithi076.stdout:creating /etc/ceph/ceph.keyring 2023-11-09T03:44:31.878 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-09T03:44:31.923 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-09T03:44:31.950 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.76'), ('mon.b', '172.21.15.113'), ('mon.c', '[v2:172.21.15.113:3301,v1:172.21.15.113:6790]')] 2023-11-09T03:44:31.951 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.76,172.21.15.113,[v2:172.21.15.113:3301,v1:172.21.15.113:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-09T03:44:31.952 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:44:31.952 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-09T03:44:32.004 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.76 --add b 172.21.15.113 --addv c '[v2:172.21.15.113:3301,v1:172.21.15.113:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-09T03:44:32.089 INFO:teuthology.orchestra.run.smithi076.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-09T03:44:32.090 INFO:teuthology.orchestra.run.smithi076.stdout:monmaptool: generated fsid c213b26f-d0da-45d5-8e34-bb75c096d4f2 2023-11-09T03:44:32.090 INFO:teuthology.orchestra.run.smithi076.stdout:setting min_mon_release = pacific 2023-11-09T03:44:32.090 INFO:teuthology.orchestra.run.smithi076.stdout:epoch 0 2023-11-09T03:44:32.090 INFO:teuthology.orchestra.run.smithi076.stdout:fsid c213b26f-d0da-45d5-8e34-bb75c096d4f2 2023-11-09T03:44:32.091 INFO:teuthology.orchestra.run.smithi076.stdout:last_changed 2023-11-09T03:44:32.091267+0000 2023-11-09T03:44:32.091 INFO:teuthology.orchestra.run.smithi076.stdout:created 2023-11-09T03:44:32.091267+0000 2023-11-09T03:44:32.091 INFO:teuthology.orchestra.run.smithi076.stdout:min_mon_release 16 (pacific) 2023-11-09T03:44:32.091 INFO:teuthology.orchestra.run.smithi076.stdout:election_strategy: 1 2023-11-09T03:44:32.091 INFO:teuthology.orchestra.run.smithi076.stdout:0: [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon.a 2023-11-09T03:44:32.091 INFO:teuthology.orchestra.run.smithi076.stdout:1: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.b 2023-11-09T03:44:32.092 INFO:teuthology.orchestra.run.smithi076.stdout:2: [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] mon.c 2023-11-09T03:44:32.092 INFO:teuthology.orchestra.run.smithi076.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-09T03:44:32.093 DEBUG:teuthology.orchestra.run.smithi076:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-09T03:44:32.107 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c213b26f-d0da-45d5-8e34-bb75c096d4f2... 2023-11-09T03:44:32.108 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-09T03:44:32.150 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-09T03:44:32.198 INFO:teuthology.orchestra.run.smithi076.stdout:[global] 2023-11-09T03:44:32.198 INFO:teuthology.orchestra.run.smithi076.stdout: chdir = "" 2023-11-09T03:44:32.198 INFO:teuthology.orchestra.run.smithi076.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-09T03:44:32.198 INFO:teuthology.orchestra.run.smithi076.stdout: auth supported = cephx 2023-11-09T03:44:32.199 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.199 INFO:teuthology.orchestra.run.smithi076.stdout: filestore xattr use omap = true 2023-11-09T03:44:32.199 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.199 INFO:teuthology.orchestra.run.smithi076.stdout: mon clock drift allowed = 1.000 2023-11-09T03:44:32.199 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.200 INFO:teuthology.orchestra.run.smithi076.stdout: osd crush chooseleaf type = 0 2023-11-09T03:44:32.200 INFO:teuthology.orchestra.run.smithi076.stdout: auth debug = true 2023-11-09T03:44:32.200 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.200 INFO:teuthology.orchestra.run.smithi076.stdout: ms die on old message = true 2023-11-09T03:44:32.200 INFO:teuthology.orchestra.run.smithi076.stdout: ms die on bug = true 2023-11-09T03:44:32.200 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.201 INFO:teuthology.orchestra.run.smithi076.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-09T03:44:32.201 INFO:teuthology.orchestra.run.smithi076.stdout: mon pg warn max object skew = 0 2023-11-09T03:44:32.201 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.201 INFO:teuthology.orchestra.run.smithi076.stdout: # disable pg_autoscaler by default for new pools 2023-11-09T03:44:32.201 INFO:teuthology.orchestra.run.smithi076.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-09T03:44:32.202 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.202 INFO:teuthology.orchestra.run.smithi076.stdout: osd pool default size = 2 2023-11-09T03:44:32.202 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.202 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd allow primary affinity = true 2023-11-09T03:44:32.202 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd allow pg remap = true 2023-11-09T03:44:32.203 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on legacy crush tunables = false 2023-11-09T03:44:32.203 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on crush straw calc version zero = false 2023-11-09T03:44:32.203 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on no sortbitwise = false 2023-11-09T03:44:32.203 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on osd down out interval zero = false 2023-11-09T03:44:32.203 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on too few osds = false 2023-11-09T03:44:32.204 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-09T03:44:32.204 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-09T03:44:32.204 INFO:teuthology.orchestra.run.smithi076.stdout: mon_allow_pool_size_one = true 2023-11-09T03:44:32.204 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.204 INFO:teuthology.orchestra.run.smithi076.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-09T03:44:32.204 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.205 INFO:teuthology.orchestra.run.smithi076.stdout: osd default data pool replay window = 5 2023-11-09T03:44:32.205 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.205 INFO:teuthology.orchestra.run.smithi076.stdout: mon allow pool delete = true 2023-11-09T03:44:32.205 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.205 INFO:teuthology.orchestra.run.smithi076.stdout: mon cluster log file level = debug 2023-11-09T03:44:32.206 INFO:teuthology.orchestra.run.smithi076.stdout: debug asserts on shutdown = true 2023-11-09T03:44:32.206 INFO:teuthology.orchestra.run.smithi076.stdout: mon health detail to clog = false 2023-11-09T03:44:32.206 INFO:teuthology.orchestra.run.smithi076.stdout: mon host = "172.21.15.76,172.21.15.113,[v2:172.21.15.113:3301,v1:172.21.15.113:6790]" 2023-11-09T03:44:32.206 INFO:teuthology.orchestra.run.smithi076.stdout: fsid = c213b26f-d0da-45d5-8e34-bb75c096d4f2 2023-11-09T03:44:32.206 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.206 INFO:teuthology.orchestra.run.smithi076.stdout:[osd] 2023-11-09T03:44:32.207 INFO:teuthology.orchestra.run.smithi076.stdout: osd journal size = 100 2023-11-09T03:44:32.207 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.207 INFO:teuthology.orchestra.run.smithi076.stdout: osd scrub load threshold = 5.0 2023-11-09T03:44:32.207 INFO:teuthology.orchestra.run.smithi076.stdout: osd scrub max interval = 600 2023-11-09T03:44:32.207 INFO:teuthology.orchestra.run.smithi076.stdout: osd mclock profile = high_recovery_ops 2023-11-09T03:44:32.208 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.208 INFO:teuthology.orchestra.run.smithi076.stdout: osd recover clone overlap = true 2023-11-09T03:44:32.208 INFO:teuthology.orchestra.run.smithi076.stdout: osd recovery max chunk = 1048576 2023-11-09T03:44:32.208 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.208 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug shutdown = true 2023-11-09T03:44:32.208 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug op order = true 2023-11-09T03:44:32.209 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug verify stray on activate = true 2023-11-09T03:44:32.209 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug trim objects = true 2023-11-09T03:44:32.209 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.209 INFO:teuthology.orchestra.run.smithi076.stdout: osd open classes on start = true 2023-11-09T03:44:32.209 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug pg log writeout = true 2023-11-09T03:44:32.210 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.210 INFO:teuthology.orchestra.run.smithi076.stdout: osd deep scrub update digest min age = 30 2023-11-09T03:44:32.210 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.210 INFO:teuthology.orchestra.run.smithi076.stdout: osd map max advance = 10 2023-11-09T03:44:32.210 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.211 INFO:teuthology.orchestra.run.smithi076.stdout: journal zero on create = true 2023-11-09T03:44:32.211 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.211 INFO:teuthology.orchestra.run.smithi076.stdout: filestore ondisk finisher threads = 3 2023-11-09T03:44:32.211 INFO:teuthology.orchestra.run.smithi076.stdout: filestore apply finisher threads = 3 2023-11-09T03:44:32.211 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.211 INFO:teuthology.orchestra.run.smithi076.stdout: bdev debug aio = true 2023-11-09T03:44:32.212 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug misdirected ops = true 2023-11-09T03:44:32.212 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore block size = 96636764160 2023-11-09T03:44:32.212 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore fsck on mount = True 2023-11-09T03:44:32.212 INFO:teuthology.orchestra.run.smithi076.stdout: debug bluefs = 20 2023-11-09T03:44:32.212 INFO:teuthology.orchestra.run.smithi076.stdout: debug bluestore = 20 2023-11-09T03:44:32.213 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2023-11-09T03:44:32.213 INFO:teuthology.orchestra.run.smithi076.stdout: debug osd = 20 2023-11-09T03:44:32.213 INFO:teuthology.orchestra.run.smithi076.stdout: debug rocksdb = 10 2023-11-09T03:44:32.213 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd backfillfull_ratio = 0.85 2023-11-09T03:44:32.213 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd full ratio = 0.9 2023-11-09T03:44:32.214 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd nearfull ratio = 0.8 2023-11-09T03:44:32.214 INFO:teuthology.orchestra.run.smithi076.stdout: osd failsafe full ratio = 0.95 2023-11-09T03:44:32.214 INFO:teuthology.orchestra.run.smithi076.stdout: osd objectstore = bluestore 2023-11-09T03:44:32.214 INFO:teuthology.orchestra.run.smithi076.stdout: osd op complaint time = 180 2023-11-09T03:44:32.214 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.214 INFO:teuthology.orchestra.run.smithi076.stdout:[mgr] 2023-11-09T03:44:32.215 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2023-11-09T03:44:32.215 INFO:teuthology.orchestra.run.smithi076.stdout: debug mgr = 20 2023-11-09T03:44:32.215 INFO:teuthology.orchestra.run.smithi076.stdout: debug mon = 20 2023-11-09T03:44:32.215 INFO:teuthology.orchestra.run.smithi076.stdout: debug auth = 20 2023-11-09T03:44:32.215 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min pgs per osd = 4 2023-11-09T03:44:32.216 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min bytes per osd = 10 2023-11-09T03:44:32.216 INFO:teuthology.orchestra.run.smithi076.stdout: mgr/telemetry/nag = false 2023-11-09T03:44:32.216 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.216 INFO:teuthology.orchestra.run.smithi076.stdout:[mon] 2023-11-09T03:44:32.216 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2023-11-09T03:44:32.217 INFO:teuthology.orchestra.run.smithi076.stdout: debug mon = 20 2023-11-09T03:44:32.217 INFO:teuthology.orchestra.run.smithi076.stdout: debug paxos = 20 2023-11-09T03:44:32.217 INFO:teuthology.orchestra.run.smithi076.stdout: debug auth = 20 2023-11-09T03:44:32.217 INFO:teuthology.orchestra.run.smithi076.stdout: mon data avail warn = 5 2023-11-09T03:44:32.217 INFO:teuthology.orchestra.run.smithi076.stdout: mon mgr mkfs grace = 240 2023-11-09T03:44:32.217 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min pgs per osd = 4 2023-11-09T03:44:32.218 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd reporter subtree level = osd 2023-11-09T03:44:32.218 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd prime pg temp = true 2023-11-09T03:44:32.218 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min bytes per osd = 10 2023-11-09T03:44:32.218 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.218 INFO:teuthology.orchestra.run.smithi076.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-09T03:44:32.219 INFO:teuthology.orchestra.run.smithi076.stdout: auth mon ticket ttl = 660 # 11m 2023-11-09T03:44:32.219 INFO:teuthology.orchestra.run.smithi076.stdout: auth service ticket ttl = 240 # 4m 2023-11-09T03:44:32.219 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.219 INFO:teuthology.orchestra.run.smithi076.stdout: # don't complain about insecure global_id in the test suite 2023-11-09T03:44:32.219 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-09T03:44:32.219 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-09T03:44:32.220 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.220 INFO:teuthology.orchestra.run.smithi076.stdout: # 1m isn't quite enough 2023-11-09T03:44:32.220 INFO:teuthology.orchestra.run.smithi076.stdout: mon_down_mkfs_grace = 2m 2023-11-09T03:44:32.220 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.220 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_filestore_osds = false 2023-11-09T03:44:32.221 INFO:teuthology.orchestra.run.smithi076.stdout: mon op complaint time = 120 2023-11-09T03:44:32.221 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:44:32.221 INFO:teuthology.orchestra.run.smithi076.stdout:[client] 2023-11-09T03:44:32.221 INFO:teuthology.orchestra.run.smithi076.stdout: rgw cache enabled = true 2023-11-09T03:44:32.221 INFO:teuthology.orchestra.run.smithi076.stdout: rgw enable ops log = true 2023-11-09T03:44:32.221 INFO:teuthology.orchestra.run.smithi076.stdout: rgw enable usage log = true 2023-11-09T03:44:32.222 INFO:teuthology.orchestra.run.smithi076.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-09T03:44:32.222 INFO:teuthology.orchestra.run.smithi076.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-09T03:44:32.222 INFO:teuthology.orchestra.run.smithi076.stdout: client mount timeout = 600 2023-11-09T03:44:32.222 INFO:teuthology.orchestra.run.smithi076.stdout: debug client = 20 2023-11-09T03:44:32.222 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2023-11-09T03:44:32.223 INFO:teuthology.orchestra.run.smithi076.stdout: pid file = "" 2023-11-09T03:44:32.223 INFO:teuthology.orchestra.run.smithi076.stdout: rados mon op timeout = 900 2023-11-09T03:44:32.223 INFO:teuthology.orchestra.run.smithi076.stdout: rados osd op timeout = 900 2023-11-09T03:44:32.223 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.a] 2023-11-09T03:44:32.223 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.b] 2023-11-09T03:44:32.223 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.c] 2023-11-09T03:44:32.224 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.a] 2023-11-09T03:44:32.224 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.c] 2023-11-09T03:44:32.224 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.b] 2023-11-09T03:44:32.224 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.d] 2023-11-09T03:44:32.224 INFO:teuthology.orchestra.run.smithi076.stdout:[mds] 2023-11-09T03:44:32.224 INFO:teuthology.orchestra.run.smithi076.stdout: debug mds = 20 2023-11-09T03:44:32.225 INFO:teuthology.orchestra.run.smithi076.stdout: debug mds balancer = 20 2023-11-09T03:44:32.225 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2023-11-09T03:44:32.225 INFO:teuthology.orchestra.run.smithi076.stdout: mds debug frag = True 2023-11-09T03:44:32.225 INFO:teuthology.orchestra.run.smithi076.stdout: mds debug scatterstat = True 2023-11-09T03:44:32.225 INFO:teuthology.orchestra.run.smithi076.stdout: mds op complaint time = 180 2023-11-09T03:44:32.225 INFO:teuthology.orchestra.run.smithi076.stdout: mds verify scatter = True 2023-11-09T03:44:32.226 INFO:teuthology.orchestra.run.smithi076.stdout: osd op complaint time = 180 2023-11-09T03:44:32.226 INFO:teuthology.orchestra.run.smithi076.stdout: rados mon op timeout = 900 2023-11-09T03:44:32.226 INFO:teuthology.orchestra.run.smithi076.stdout: rados osd op timeout = 900 2023-11-09T03:44:32.228 INFO:teuthology.orchestra.run.smithi113.stdout:[global] 2023-11-09T03:44:32.228 INFO:teuthology.orchestra.run.smithi113.stdout: chdir = "" 2023-11-09T03:44:32.228 INFO:teuthology.orchestra.run.smithi113.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-09T03:44:32.228 INFO:teuthology.orchestra.run.smithi113.stdout: auth supported = cephx 2023-11-09T03:44:32.228 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.229 INFO:teuthology.orchestra.run.smithi113.stdout: filestore xattr use omap = true 2023-11-09T03:44:32.229 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.229 INFO:teuthology.orchestra.run.smithi113.stdout: mon clock drift allowed = 1.000 2023-11-09T03:44:32.229 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.229 INFO:teuthology.orchestra.run.smithi113.stdout: osd crush chooseleaf type = 0 2023-11-09T03:44:32.229 INFO:teuthology.orchestra.run.smithi113.stdout: auth debug = true 2023-11-09T03:44:32.230 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.230 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on old message = true 2023-11-09T03:44:32.230 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on bug = true 2023-11-09T03:44:32.230 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.230 INFO:teuthology.orchestra.run.smithi113.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-09T03:44:32.231 INFO:teuthology.orchestra.run.smithi113.stdout: mon pg warn max object skew = 0 2023-11-09T03:44:32.231 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.231 INFO:teuthology.orchestra.run.smithi113.stdout: # disable pg_autoscaler by default for new pools 2023-11-09T03:44:32.231 INFO:teuthology.orchestra.run.smithi113.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-09T03:44:32.231 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.231 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default size = 2 2023-11-09T03:44:32.232 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.232 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow primary affinity = true 2023-11-09T03:44:32.232 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow pg remap = true 2023-11-09T03:44:32.232 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on legacy crush tunables = false 2023-11-09T03:44:32.232 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on crush straw calc version zero = false 2023-11-09T03:44:32.233 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on no sortbitwise = false 2023-11-09T03:44:32.233 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on osd down out interval zero = false 2023-11-09T03:44:32.233 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on too few osds = false 2023-11-09T03:44:32.233 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-09T03:44:32.233 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-09T03:44:32.233 INFO:teuthology.orchestra.run.smithi113.stdout: mon_allow_pool_size_one = true 2023-11-09T03:44:32.234 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.234 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-09T03:44:32.234 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.234 INFO:teuthology.orchestra.run.smithi113.stdout: osd default data pool replay window = 5 2023-11-09T03:44:32.234 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.235 INFO:teuthology.orchestra.run.smithi113.stdout: mon allow pool delete = true 2023-11-09T03:44:32.235 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.235 INFO:teuthology.orchestra.run.smithi113.stdout: mon cluster log file level = debug 2023-11-09T03:44:32.235 INFO:teuthology.orchestra.run.smithi113.stdout: debug asserts on shutdown = true 2023-11-09T03:44:32.235 INFO:teuthology.orchestra.run.smithi113.stdout: mon health detail to clog = false 2023-11-09T03:44:32.235 INFO:teuthology.orchestra.run.smithi113.stdout: mon host = "172.21.15.76,172.21.15.113,[v2:172.21.15.113:3301,v1:172.21.15.113:6790]" 2023-11-09T03:44:32.236 INFO:teuthology.orchestra.run.smithi113.stdout: fsid = c213b26f-d0da-45d5-8e34-bb75c096d4f2 2023-11-09T03:44:32.236 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.236 INFO:teuthology.orchestra.run.smithi113.stdout:[osd] 2023-11-09T03:44:32.236 INFO:teuthology.orchestra.run.smithi113.stdout: osd journal size = 100 2023-11-09T03:44:32.236 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.236 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub load threshold = 5.0 2023-11-09T03:44:32.237 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub max interval = 600 2023-11-09T03:44:32.237 INFO:teuthology.orchestra.run.smithi113.stdout: osd mclock profile = high_recovery_ops 2023-11-09T03:44:32.237 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.237 INFO:teuthology.orchestra.run.smithi113.stdout: osd recover clone overlap = true 2023-11-09T03:44:32.237 INFO:teuthology.orchestra.run.smithi113.stdout: osd recovery max chunk = 1048576 2023-11-09T03:44:32.237 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.238 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug shutdown = true 2023-11-09T03:44:32.238 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug op order = true 2023-11-09T03:44:32.238 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug verify stray on activate = true 2023-11-09T03:44:32.238 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug trim objects = true 2023-11-09T03:44:32.238 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.238 INFO:teuthology.orchestra.run.smithi113.stdout: osd open classes on start = true 2023-11-09T03:44:32.239 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug pg log writeout = true 2023-11-09T03:44:32.239 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.239 INFO:teuthology.orchestra.run.smithi113.stdout: osd deep scrub update digest min age = 30 2023-11-09T03:44:32.239 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.239 INFO:teuthology.orchestra.run.smithi113.stdout: osd map max advance = 10 2023-11-09T03:44:32.239 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.240 INFO:teuthology.orchestra.run.smithi113.stdout: journal zero on create = true 2023-11-09T03:44:32.240 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.240 INFO:teuthology.orchestra.run.smithi113.stdout: filestore ondisk finisher threads = 3 2023-11-09T03:44:32.240 INFO:teuthology.orchestra.run.smithi113.stdout: filestore apply finisher threads = 3 2023-11-09T03:44:32.240 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.241 INFO:teuthology.orchestra.run.smithi113.stdout: bdev debug aio = true 2023-11-09T03:44:32.241 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug misdirected ops = true 2023-11-09T03:44:32.241 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore block size = 96636764160 2023-11-09T03:44:32.241 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore fsck on mount = True 2023-11-09T03:44:32.241 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluefs = 20 2023-11-09T03:44:32.241 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluestore = 20 2023-11-09T03:44:32.242 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2023-11-09T03:44:32.242 INFO:teuthology.orchestra.run.smithi113.stdout: debug osd = 20 2023-11-09T03:44:32.242 INFO:teuthology.orchestra.run.smithi113.stdout: debug rocksdb = 10 2023-11-09T03:44:32.242 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd backfillfull_ratio = 0.85 2023-11-09T03:44:32.242 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd full ratio = 0.9 2023-11-09T03:44:32.243 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd nearfull ratio = 0.8 2023-11-09T03:44:32.243 INFO:teuthology.orchestra.run.smithi113.stdout: osd failsafe full ratio = 0.95 2023-11-09T03:44:32.243 INFO:teuthology.orchestra.run.smithi113.stdout: osd objectstore = bluestore 2023-11-09T03:44:32.243 INFO:teuthology.orchestra.run.smithi113.stdout: osd op complaint time = 180 2023-11-09T03:44:32.243 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.243 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr] 2023-11-09T03:44:32.244 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2023-11-09T03:44:32.244 INFO:teuthology.orchestra.run.smithi113.stdout: debug mgr = 20 2023-11-09T03:44:32.244 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2023-11-09T03:44:32.244 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2023-11-09T03:44:32.244 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2023-11-09T03:44:32.245 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2023-11-09T03:44:32.245 INFO:teuthology.orchestra.run.smithi113.stdout: mgr/telemetry/nag = false 2023-11-09T03:44:32.245 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.245 INFO:teuthology.orchestra.run.smithi113.stdout:[mon] 2023-11-09T03:44:32.245 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2023-11-09T03:44:32.245 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2023-11-09T03:44:32.246 INFO:teuthology.orchestra.run.smithi113.stdout: debug paxos = 20 2023-11-09T03:44:32.246 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2023-11-09T03:44:32.246 INFO:teuthology.orchestra.run.smithi113.stdout: mon data avail warn = 5 2023-11-09T03:44:32.246 INFO:teuthology.orchestra.run.smithi113.stdout: mon mgr mkfs grace = 240 2023-11-09T03:44:32.246 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2023-11-09T03:44:32.246 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd reporter subtree level = osd 2023-11-09T03:44:32.247 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd prime pg temp = true 2023-11-09T03:44:32.247 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2023-11-09T03:44:32.247 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.247 INFO:teuthology.orchestra.run.smithi113.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-09T03:44:32.247 INFO:teuthology.orchestra.run.smithi113.stdout: auth mon ticket ttl = 660 # 11m 2023-11-09T03:44:32.247 INFO:teuthology.orchestra.run.smithi113.stdout: auth service ticket ttl = 240 # 4m 2023-11-09T03:44:32.248 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.248 INFO:teuthology.orchestra.run.smithi113.stdout: # don't complain about insecure global_id in the test suite 2023-11-09T03:44:32.248 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-09T03:44:32.248 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-09T03:44:32.248 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.248 INFO:teuthology.orchestra.run.smithi113.stdout: # 1m isn't quite enough 2023-11-09T03:44:32.249 INFO:teuthology.orchestra.run.smithi113.stdout: mon_down_mkfs_grace = 2m 2023-11-09T03:44:32.249 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.249 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_filestore_osds = false 2023-11-09T03:44:32.249 INFO:teuthology.orchestra.run.smithi113.stdout: mon op complaint time = 120 2023-11-09T03:44:32.249 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:44:32.250 INFO:teuthology.orchestra.run.smithi113.stdout:[client] 2023-11-09T03:44:32.250 INFO:teuthology.orchestra.run.smithi113.stdout: rgw cache enabled = true 2023-11-09T03:44:32.250 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable ops log = true 2023-11-09T03:44:32.250 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable usage log = true 2023-11-09T03:44:32.250 INFO:teuthology.orchestra.run.smithi113.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-09T03:44:32.251 INFO:teuthology.orchestra.run.smithi113.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-09T03:44:32.251 INFO:teuthology.orchestra.run.smithi113.stdout: client mount timeout = 600 2023-11-09T03:44:32.251 INFO:teuthology.orchestra.run.smithi113.stdout: debug client = 20 2023-11-09T03:44:32.251 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2023-11-09T03:44:32.251 INFO:teuthology.orchestra.run.smithi113.stdout: pid file = "" 2023-11-09T03:44:32.251 INFO:teuthology.orchestra.run.smithi113.stdout: rados mon op timeout = 900 2023-11-09T03:44:32.252 INFO:teuthology.orchestra.run.smithi113.stdout: rados osd op timeout = 900 2023-11-09T03:44:32.252 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.a] 2023-11-09T03:44:32.252 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.b] 2023-11-09T03:44:32.252 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.c] 2023-11-09T03:44:32.252 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.a] 2023-11-09T03:44:32.252 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.c] 2023-11-09T03:44:32.253 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.b] 2023-11-09T03:44:32.253 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.d] 2023-11-09T03:44:32.253 INFO:teuthology.orchestra.run.smithi113.stdout:[mds] 2023-11-09T03:44:32.253 INFO:teuthology.orchestra.run.smithi113.stdout: debug mds = 20 2023-11-09T03:44:32.253 INFO:teuthology.orchestra.run.smithi113.stdout: debug mds balancer = 20 2023-11-09T03:44:32.254 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2023-11-09T03:44:32.254 INFO:teuthology.orchestra.run.smithi113.stdout: mds debug frag = True 2023-11-09T03:44:32.254 INFO:teuthology.orchestra.run.smithi113.stdout: mds debug scatterstat = True 2023-11-09T03:44:32.254 INFO:teuthology.orchestra.run.smithi113.stdout: mds op complaint time = 180 2023-11-09T03:44:32.254 INFO:teuthology.orchestra.run.smithi113.stdout: mds verify scatter = True 2023-11-09T03:44:32.254 INFO:teuthology.orchestra.run.smithi113.stdout: osd op complaint time = 180 2023-11-09T03:44:32.255 INFO:teuthology.orchestra.run.smithi113.stdout: rados mon op timeout = 900 2023-11-09T03:44:32.255 INFO:teuthology.orchestra.run.smithi113.stdout: rados osd op timeout = 900 2023-11-09T03:44:32.255 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-09T03:44:32.256 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-09T03:44:32.311 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-09T03:44:32.311 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:44:32.311 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-09T03:44:32.326 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:44:32.327 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-09T03:44:32.384 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T03:44:32.384 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:44:32.385 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-09T03:44:32.385 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-09T03:44:32.462 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:44:32.463 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-09T03:44:32.517 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi113.front.sepia.ceph.com 2023-11-09T03:44:32.518 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:44:32.518 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-09T03:44:32.519 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-09T03:44:32.558 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:44:32.558 DEBUG:teuthology.orchestra.run.smithi113:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-09T03:44:32.613 INFO:tasks.ceph:Setting up mon nodes... 2023-11-09T03:44:32.614 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-09T03:44:32.614 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-09T03:44:32.669 INFO:teuthology.orchestra.run.smithi076.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-09T03:44:32.671 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-09T03:44:32.735 INFO:teuthology.orchestra.run.smithi113.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-09T03:44:32.737 INFO:tasks.ceph:Setting up mds nodes... 2023-11-09T03:44:32.738 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-09T03:44:32.793 INFO:teuthology.orchestra.run.smithi076.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-09T03:44:32.795 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-09T03:44:32.822 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-09T03:44:32.916 INFO:teuthology.orchestra.run.smithi076.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-09T03:44:32.919 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-09T03:44:32.945 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-09T03:44:33.000 INFO:teuthology.orchestra.run.smithi113.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-09T03:44:33.003 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-09T03:44:33.029 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-09T03:44:33.123 INFO:teuthology.orchestra.run.smithi113.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-09T03:44:33.126 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-09T03:44:33.152 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-09T03:44:33.152 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-09T03:44:33.194 INFO:teuthology.orchestra.run.smithi076.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-09T03:44:33.209 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-09T03:44:33.293 INFO:teuthology.orchestra.run.smithi076.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-09T03:44:33.308 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-09T03:44:33.350 INFO:teuthology.orchestra.run.smithi113.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-09T03:44:33.365 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-09T03:44:33.447 INFO:teuthology.orchestra.run.smithi113.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-09T03:44:33.462 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-09T03:44:33.463 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi076.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi113.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-09T03:44:33.463 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-09T03:44:33.489 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-09T03:44:33.490 INFO:tasks.ceph:role: osd.0 2023-11-09T03:44:33.490 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T03:44:33.491 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-09T03:44:33.563 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-09T03:44:33.564 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-09T03:44:33.564 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-09T03:44:33.564 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-09T03:44:33.564 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-09T03:44:33.564 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2023-11-09T03:44:33.565 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-09T03:44:33.565 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-09T03:44:33.565 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-09T03:44:33.565 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-09T03:44:34.059 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2023-11-09T03:44:34.061 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2023-11-09T03:44:34.061 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-09T03:44:34.183 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-09T03:44:34.217 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-09T03:44:34.285 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-09T03:44:34.286 INFO:tasks.ceph:role: osd.1 2023-11-09T03:44:34.286 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T03:44:34.286 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-09T03:44:34.360 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-09T03:44:34.360 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-09T03:44:34.360 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-09T03:44:34.361 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-09T03:44:34.361 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-09T03:44:34.361 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2023-11-09T03:44:34.361 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-09T03:44:34.361 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-09T03:44:34.361 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-09T03:44:34.362 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-09T03:44:34.827 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2023-11-09T03:44:34.829 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2023-11-09T03:44:34.829 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-09T03:44:34.930 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-09T03:44:34.963 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-09T03:44:35.032 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-09T03:44:35.032 INFO:tasks.ceph:role: osd.2 2023-11-09T03:44:35.033 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T03:44:35.033 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-09T03:44:35.107 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-09T03:44:35.107 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-09T03:44:35.107 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-09T03:44:35.108 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-09T03:44:35.108 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-09T03:44:35.108 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2023-11-09T03:44:35.108 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-09T03:44:35.108 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-09T03:44:35.109 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-09T03:44:35.109 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-09T03:44:35.594 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2023-11-09T03:44:35.597 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2023-11-09T03:44:35.597 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-09T03:44:35.725 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-09T03:44:35.759 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-09T03:44:35.827 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-09T03:44:35.827 INFO:tasks.ceph:role: osd.3 2023-11-09T03:44:35.827 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T03:44:35.828 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-09T03:44:35.901 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-09T03:44:35.902 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-09T03:44:35.902 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-09T03:44:35.902 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-09T03:44:35.903 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-09T03:44:35.903 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2023-11-09T03:44:35.903 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-09T03:44:35.903 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-09T03:44:35.903 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-09T03:44:35.904 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-09T03:44:36.432 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2023-11-09T03:44:36.434 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2023-11-09T03:44:36.434 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-09T03:44:36.576 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-09T03:44:36.612 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-09T03:44:36.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:44:36.708+0000 7f5744c14500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-09T03:44:36.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:44:36.708+0000 7f5744c14500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-09T03:44:36.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:44:36.708+0000 7f5744c14500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-09T03:44:36.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:44:36.708+0000 7f5744c14500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-09T03:44:36.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:44:36.708+0000 7f5744c14500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-09T03:44:36.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:44:36.708+0000 7f5744c14500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-09T03:44:39.557 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-09T03:44:39.589 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-09T03:44:39.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:44:39.684+0000 7f8d3e21a500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-09T03:44:39.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:44:39.684+0000 7f8d3e21a500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-09T03:44:39.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:44:39.684+0000 7f8d3e21a500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-09T03:44:39.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:44:39.684+0000 7f8d3e21a500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-09T03:44:39.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:44:39.684+0000 7f8d3e21a500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-09T03:44:39.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:44:39.685+0000 7f8d3e21a500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-09T03:44:42.530 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-09T03:44:42.560 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-09T03:44:42.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:44:42.650+0000 7fe147010500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-09T03:44:42.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:44:42.651+0000 7fe147010500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-09T03:44:42.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:44:42.651+0000 7fe147010500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-09T03:44:42.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:44:42.651+0000 7fe147010500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-09T03:44:42.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:44:42.651+0000 7fe147010500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-09T03:44:42.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:44:42.651+0000 7fe147010500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-09T03:44:45.497 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-09T03:44:45.524 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-09T03:44:45.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:44:45.613+0000 7f9bace1f500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-09T03:44:45.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:44:45.613+0000 7f9bace1f500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-09T03:44:45.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:44:45.613+0000 7f9bace1f500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-09T03:44:45.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:44:45.613+0000 7f9bace1f500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-09T03:44:45.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:44:45.613+0000 7f9bace1f500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-09T03:44:45.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:44:45.614+0000 7f9bace1f500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-09T03:44:48.461 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-09T03:44:48.494 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-09T03:44:48.525 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-09T03:44:48.526 INFO:tasks.ceph:role: osd.4 2023-11-09T03:44:48.526 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi113.front.sepia.ceph.com 2023-11-09T03:44:48.526 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-09T03:44:48.603 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-09T03:44:48.604 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-09T03:44:48.604 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-09T03:44:48.604 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-09T03:44:48.604 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-09T03:44:48.604 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2023-11-09T03:44:48.605 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-09T03:44:48.605 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-09T03:44:48.605 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-09T03:44:48.605 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-09T03:44:49.119 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2023-11-09T03:44:49.121 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2023-11-09T03:44:49.121 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-09T03:44:49.236 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-09T03:44:49.270 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-09T03:44:49.337 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-09T03:44:49.338 INFO:tasks.ceph:role: osd.5 2023-11-09T03:44:49.338 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi113.front.sepia.ceph.com 2023-11-09T03:44:49.338 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-09T03:44:49.414 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-09T03:44:49.414 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-09T03:44:49.414 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-09T03:44:49.414 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-09T03:44:49.415 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-09T03:44:49.415 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2023-11-09T03:44:49.415 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-09T03:44:49.415 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-09T03:44:49.416 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-09T03:44:49.416 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-09T03:44:49.904 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2023-11-09T03:44:49.906 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2023-11-09T03:44:49.906 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-09T03:44:50.001 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-09T03:44:50.032 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-09T03:44:50.099 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-09T03:44:50.100 INFO:tasks.ceph:role: osd.6 2023-11-09T03:44:50.100 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi113.front.sepia.ceph.com 2023-11-09T03:44:50.100 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-09T03:44:50.174 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-09T03:44:50.175 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-09T03:44:50.175 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-09T03:44:50.175 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-09T03:44:50.176 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-09T03:44:50.176 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2023-11-09T03:44:50.176 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-09T03:44:50.176 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-09T03:44:50.176 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-09T03:44:50.176 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-09T03:44:50.698 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2023-11-09T03:44:50.700 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2023-11-09T03:44:50.701 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-09T03:44:50.810 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-09T03:44:50.842 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-09T03:44:50.910 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-09T03:44:50.910 INFO:tasks.ceph:role: osd.7 2023-11-09T03:44:50.910 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi113.front.sepia.ceph.com 2023-11-09T03:44:50.911 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-09T03:44:50.985 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-09T03:44:50.985 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-09T03:44:50.985 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-09T03:44:50.986 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-09T03:44:50.986 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-09T03:44:50.986 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2023-11-09T03:44:50.986 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-09T03:44:50.986 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-09T03:44:50.987 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-09T03:44:50.987 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-09T03:44:51.515 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2023-11-09T03:44:51.517 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2023-11-09T03:44:51.518 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-09T03:44:51.638 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-09T03:44:51.670 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-09T03:44:51.797 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:44:51.797+0000 7f97cfd10500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-09T03:44:51.800 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:44:51.800+0000 7f97cfd10500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-09T03:44:51.800 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:44:51.800+0000 7f97cfd10500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-09T03:44:51.801 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:44:51.800+0000 7f97cfd10500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-09T03:44:51.801 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:44:51.800+0000 7f97cfd10500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-09T03:44:51.801 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:44:51.801+0000 7f97cfd10500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-09T03:44:54.650 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-09T03:44:54.681 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-09T03:44:54.772 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:44:54.772+0000 7f35d160c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-09T03:44:54.773 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:44:54.772+0000 7f35d160c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-09T03:44:54.773 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:44:54.772+0000 7f35d160c500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-09T03:44:54.773 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:44:54.772+0000 7f35d160c500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-09T03:44:54.773 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:44:54.772+0000 7f35d160c500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-09T03:44:54.774 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:44:54.773+0000 7f35d160c500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-09T03:44:57.623 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-09T03:44:57.654 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-09T03:44:57.745 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:44:57.744+0000 7f5da9d46500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-09T03:44:57.745 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:44:57.745+0000 7f5da9d46500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-09T03:44:57.745 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:44:57.745+0000 7f5da9d46500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-09T03:44:57.745 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:44:57.745+0000 7f5da9d46500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-09T03:44:57.746 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:44:57.745+0000 7f5da9d46500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-09T03:44:57.746 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:44:57.745+0000 7f5da9d46500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-09T03:45:00.216 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:45:00.219 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:45:00.607 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-09T03:45:00.636 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-09T03:45:00.723 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:00.723+0000 7fe27988c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-09T03:45:00.724 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:00.723+0000 7fe27988c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-09T03:45:00.724 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:00.723+0000 7fe27988c500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-09T03:45:00.724 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:00.723+0000 7fe27988c500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-09T03:45:00.724 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:00.723+0000 7fe27988c500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-09T03:45:00.725 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:00.724+0000 7fe27988c500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-09T03:45:03.573 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-09T03:45:03.605 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-09T03:45:03.605 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:45:03.605 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-09T03:45:03.636 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:45:03.637 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-09T03:45:03.706 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:45:03.706 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-09T03:45:03.776 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:45:03.777 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-09T03:45:03.845 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:45:03.845 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-09T03:45:03.913 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:45:03.913 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-09T03:45:03.979 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:45:03.980 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-09T03:45:04.047 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:45:04.047 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-09T03:45:04.079 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:45:04.079 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-09T03:45:04.150 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:45:04.150 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-09T03:45:04.220 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:45:04.220 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-09T03:45:04.289 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:45:04.290 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-09T03:45:04.358 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:45:04.359 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-09T03:45:04.427 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:45:04.427 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-09T03:45:04.496 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:45:04.496 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-09T03:45:04.512 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:45:04.513 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-09T03:45:04.568 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:45:04.569 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-09T03:45:04.585 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:45:04.585 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-09T03:45:04.641 INFO:tasks.ceph:Adding keys to all mons... 2023-11-09T03:45:04.642 DEBUG:teuthology.orchestra.run.smithi076:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-09T03:45:04.644 DEBUG:teuthology.orchestra.run.smithi113:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-09T03:45:04.685 INFO:teuthology.orchestra.run.smithi076.stdout:[mgr.y] 2023-11-09T03:45:04.685 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCgVUxlb9L/JxAAu3IOXTn8TGvbzxYVta3yew== 2023-11-09T03:45:04.685 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.a] 2023-11-09T03:45:04.685 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCgVUxlydtlLxAAi0bnTEOD+fpGL1cYic3caA== 2023-11-09T03:45:04.686 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.c] 2023-11-09T03:45:04.686 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCgVUxlrTu9NhAAhg4GpZWZTmVPCP3ffjPo/Q== 2023-11-09T03:45:04.686 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.0] 2023-11-09T03:45:04.686 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCkVUxlXM5LKhAAM253+XEj5YSwmQywwDieTg== 2023-11-09T03:45:04.686 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.1] 2023-11-09T03:45:04.687 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCnVUxlgoDVKBAAkmNFo4E0bAQ1UxkCfQAbjw== 2023-11-09T03:45:04.687 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.2] 2023-11-09T03:45:04.687 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCqVUxlzQjVJhAAOoU5IdN86BmcPz71mTq7ng== 2023-11-09T03:45:04.687 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.3] 2023-11-09T03:45:04.687 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCtVUxlhY6VJBAAmCO/ZS8vBa7enyuXX5Y+hw== 2023-11-09T03:45:04.688 INFO:teuthology.orchestra.run.smithi076.stdout:[mgr.x] 2023-11-09T03:45:04.688 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCgVUxl7JryKxAABFx9CRqSbbJu8SIyi3Va8w== 2023-11-09T03:45:04.688 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.b] 2023-11-09T03:45:04.688 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQChVUxl2+QnABAA7CTWWEbPPXkollRFDlLxPA== 2023-11-09T03:45:04.688 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.d] 2023-11-09T03:45:04.688 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQChVUxl4IZ8BxAA8zGFE+1tlHkT5UyzL0nWxQ== 2023-11-09T03:45:04.689 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.4] 2023-11-09T03:45:04.689 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCzVUxl79egLxAAnKBOQb/9EQyaHaQt7SawIg== 2023-11-09T03:45:04.689 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.5] 2023-11-09T03:45:04.689 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQC2VUxlIfUgLhAAVPurhJCc8cmpt11pQjzL9w== 2023-11-09T03:45:04.689 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.6] 2023-11-09T03:45:04.690 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQC5VUxlrI58LBAAWxZotpACR4e8vFj4IiaEKA== 2023-11-09T03:45:04.690 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.7] 2023-11-09T03:45:04.690 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQC8VUxl0bA3KxAAwnKNJKqYIcGRLLhQHUaU9w== 2023-11-09T03:45:04.690 INFO:teuthology.orchestra.run.smithi076.stdout:[client.0] 2023-11-09T03:45:04.690 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQChVUxlHj22CxAACNj5X24hdME+GNTLw4RahQ== 2023-11-09T03:45:04.691 INFO:teuthology.orchestra.run.smithi076.stdout:[client.2] 2023-11-09T03:45:04.691 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQChVUxlGVSSERAAQts5odubZ5paZwg8US3QrQ== 2023-11-09T03:45:04.691 INFO:teuthology.orchestra.run.smithi076.stdout:[client.1] 2023-11-09T03:45:04.691 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQChVUxljXv4FBAAiTOhfIReQWTy6fbujbBrrw== 2023-11-09T03:45:04.691 INFO:teuthology.orchestra.run.smithi076.stdout:[client.3] 2023-11-09T03:45:04.692 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQChVUxlgmDHGhAA7r98HNQEMnjbqW6RjE5aig== 2023-11-09T03:45:04.708 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr.y] 2023-11-09T03:45:04.708 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQCgVUxlb9L/JxAAu3IOXTn8TGvbzxYVta3yew== 2023-11-09T03:45:04.708 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.a] 2023-11-09T03:45:04.709 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQCgVUxlydtlLxAAi0bnTEOD+fpGL1cYic3caA== 2023-11-09T03:45:04.709 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.c] 2023-11-09T03:45:04.709 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQCgVUxlrTu9NhAAhg4GpZWZTmVPCP3ffjPo/Q== 2023-11-09T03:45:04.709 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.0] 2023-11-09T03:45:04.709 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQCkVUxlXM5LKhAAM253+XEj5YSwmQywwDieTg== 2023-11-09T03:45:04.710 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.1] 2023-11-09T03:45:04.710 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQCnVUxlgoDVKBAAkmNFo4E0bAQ1UxkCfQAbjw== 2023-11-09T03:45:04.710 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.2] 2023-11-09T03:45:04.710 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQCqVUxlzQjVJhAAOoU5IdN86BmcPz71mTq7ng== 2023-11-09T03:45:04.710 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.3] 2023-11-09T03:45:04.711 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQCtVUxlhY6VJBAAmCO/ZS8vBa7enyuXX5Y+hw== 2023-11-09T03:45:04.711 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr.x] 2023-11-09T03:45:04.711 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQCgVUxl7JryKxAABFx9CRqSbbJu8SIyi3Va8w== 2023-11-09T03:45:04.711 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.b] 2023-11-09T03:45:04.711 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQChVUxl2+QnABAA7CTWWEbPPXkollRFDlLxPA== 2023-11-09T03:45:04.711 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.d] 2023-11-09T03:45:04.712 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQChVUxl4IZ8BxAA8zGFE+1tlHkT5UyzL0nWxQ== 2023-11-09T03:45:04.712 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.4] 2023-11-09T03:45:04.712 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQCzVUxl79egLxAAnKBOQb/9EQyaHaQt7SawIg== 2023-11-09T03:45:04.712 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.5] 2023-11-09T03:45:04.712 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQC2VUxlIfUgLhAAVPurhJCc8cmpt11pQjzL9w== 2023-11-09T03:45:04.712 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.6] 2023-11-09T03:45:04.713 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQC5VUxlrI58LBAAWxZotpACR4e8vFj4IiaEKA== 2023-11-09T03:45:04.713 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.7] 2023-11-09T03:45:04.713 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQC8VUxl0bA3KxAAwnKNJKqYIcGRLLhQHUaU9w== 2023-11-09T03:45:04.713 INFO:teuthology.orchestra.run.smithi113.stdout:[client.0] 2023-11-09T03:45:04.713 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQChVUxlHj22CxAACNj5X24hdME+GNTLw4RahQ== 2023-11-09T03:45:04.714 INFO:teuthology.orchestra.run.smithi113.stdout:[client.2] 2023-11-09T03:45:04.714 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQChVUxlGVSSERAAQts5odubZ5paZwg8US3QrQ== 2023-11-09T03:45:04.714 INFO:teuthology.orchestra.run.smithi113.stdout:[client.1] 2023-11-09T03:45:04.714 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQChVUxljXv4FBAAiTOhfIReQWTy6fbujbBrrw== 2023-11-09T03:45:04.714 INFO:teuthology.orchestra.run.smithi113.stdout:[client.3] 2023-11-09T03:45:04.715 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQChVUxlgmDHGhAA7r98HNQEMnjbqW6RjE5aig== 2023-11-09T03:45:04.716 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-09T03:45:04.735 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-09T03:45:04.808 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-09T03:45:04.811 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-09T03:45:04.867 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-09T03:45:04.870 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-09T03:45:04.924 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-09T03:45:04.927 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-09T03:45:04.978 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-09T03:45:04.981 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-09T03:45:05.032 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-09T03:45:05.034 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-09T03:45:05.085 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-09T03:45:05.087 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-09T03:45:05.139 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-09T03:45:05.141 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-09T03:45:05.192 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-09T03:45:05.194 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-09T03:45:05.249 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-09T03:45:05.252 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-09T03:45:05.306 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-09T03:45:05.309 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-09T03:45:05.362 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-09T03:45:05.364 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-09T03:45:05.416 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-09T03:45:05.418 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-09T03:45:05.470 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-09T03:45:05.472 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-09T03:45:05.523 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-09T03:45:05.525 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-09T03:45:05.579 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-09T03:45:05.581 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-09T03:45:05.635 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-09T03:45:05.638 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-09T03:45:05.692 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-09T03:45:05.695 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-09T03:45:05.750 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-09T03:45:05.750 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-09T03:45:05.777 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-09T03:45:05.874 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-09T03:45:05.901 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-09T03:45:05.927 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-09T03:45:06.023 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-09T03:45:06.050 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-09T03:45:06.117 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-09T03:45:06.213 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-09T03:45:06.240 DEBUG:teuthology.orchestra.run.smithi076:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-09T03:45:06.242 DEBUG:teuthology.orchestra.run.smithi113:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-09T03:45:06.294 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-09T03:45:06.294 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-09T03:45:06.295 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-09T03:45:06.297 INFO:tasks.ceph.mon.a:Started 2023-11-09T03:45:06.297 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-09T03:45:06.298 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-09T03:45:06.336 INFO:tasks.ceph.mon.b:Started 2023-11-09T03:45:06.336 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-09T03:45:06.337 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-09T03:45:06.339 INFO:tasks.ceph.mon.c:Started 2023-11-09T03:45:06.339 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-09T03:45:06.339 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-09T03:45:06.339 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-09T03:45:06.342 INFO:tasks.ceph.mgr.y:Started 2023-11-09T03:45:06.342 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-09T03:45:06.342 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-09T03:45:06.344 INFO:tasks.ceph.mgr.x:Started 2023-11-09T03:45:06.345 INFO:tasks.ceph:Setting crush tunables to default 2023-11-09T03:45:06.345 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --cluster ceph osd crush tunables default 2023-11-09T03:45:06.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.446+0000 7fe764f2c640 1 Processor -- start 2023-11-09T03:45:06.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.446+0000 7fe764f2c640 1 -- start start 2023-11-09T03:45:06.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.447+0000 7fe764f2c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7600cf180 0x7fe76006c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:06.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.447+0000 7fe764f2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe7600cd570 0x7fe7600cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:06.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.447+0000 7fe764f2c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7600cdf20 con 0x7fe76015cfd0 2023-11-09T03:45:06.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.447+0000 7fe764f2c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe76015d3b0 con 0x7fe7600cd570 2023-11-09T03:45:06.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.447+0000 7fe764f2c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7600cea20 con 0x7fe7600cf180 2023-11-09T03:45:06.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.447+0000 7fe75ed9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe7600cd570 0x7fe7600cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:06.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.447+0000 7fe75ed9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe7600cd570 0x7fe7600cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46674/0 (socket says 172.21.15.76:46674) 2023-11-09T03:45:06.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.447+0000 7fe75ed9d640 1 -- 172.21.15.76:0/109681737 learned_addr learned my addr 172.21.15.76:0/109681737 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:06.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.447+0000 7fe75ed9d640 1 -- 172.21.15.76:0/109681737 >> v2:172.21.15.76:3300/0 conn(0x7fe7600cd570 msgr2=0x7fe7600cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-09T03:45:06.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.447+0000 7fe75ed9d640 1 -- 172.21.15.76:0/109681737 >> v2:172.21.15.76:3300/0 conn(0x7fe7600cd570 msgr2=0x7fe7600cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-09T03:45:06.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.447+0000 7fe75ed9d640 1 --2- 172.21.15.76:0/109681737 >> v2:172.21.15.76:3300/0 conn(0x7fe7600cd570 0x7fe7600cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-09T03:45:06.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.447+0000 7fe75ed9d640 1 --2- 172.21.15.76:0/109681737 >> v2:172.21.15.76:3300/0 conn(0x7fe7600cd570 0x7fe7600cd9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T03:45:06.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.447+0000 7fe75dd9b640 1 --2- 172.21.15.76:0/109681737 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7600cf180 0x7fe76006c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:06.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.448+0000 7fe75dd9b640 1 -- 172.21.15.76:0/109681737 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7600cf180 msgr2=0x7fe76006c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-09T03:45:06.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.448+0000 7fe75dd9b640 1 -- 172.21.15.76:0/109681737 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7600cf180 msgr2=0x7fe76006c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-09T03:45:06.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.448+0000 7fe75dd9b640 1 --2- 172.21.15.76:0/109681737 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7600cf180 0x7fe76006c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-09T03:45:06.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.448+0000 7fe75dd9b640 1 --2- 172.21.15.76:0/109681737 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7600cf180 0x7fe76006c0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T03:45:06.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.625+0000 7fe75d59a640 1 -- 172.21.15.76:0/109681737 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2177584729 0 0) 0x7fe750001110 con 0x7fe76015cfd0 2023-11-09T03:45:06.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.625+0000 7fe75d59a640 1 -- 172.21.15.76:0/109681737 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe748003520 con 0x7fe76015cfd0 2023-11-09T03:45:06.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.627+0000 7fe75d59a640 1 -- 172.21.15.76:0/109681737 <== mon.1 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 963633701 0 0) 0x7fe748003520 con 0x7fe76015cfd0 2023-11-09T03:45:06.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.627+0000 7fe75d59a640 1 -- 172.21.15.76:0/109681737 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7600cf180 msgr2=0x7fe76006c0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T03:45:06.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.627+0000 7fe75d59a640 1 --2- 172.21.15.76:0/109681737 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7600cf180 0x7fe76006c0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:06.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.627+0000 7fe75d59a640 1 -- 172.21.15.76:0/109681737 >> v2:172.21.15.76:3300/0 conn(0x7fe7600cd570 msgr2=0x7fe7600cd9e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T03:45:06.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.627+0000 7fe75d59a640 1 --2- 172.21.15.76:0/109681737 >> v2:172.21.15.76:3300/0 conn(0x7fe7600cd570 0x7fe7600cd9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:06.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.627+0000 7fe75d59a640 1 -- 172.21.15.76:0/109681737 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7600cf610 con 0x7fe76015cfd0 2023-11-09T03:45:06.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.627+0000 7fe75d59a640 1 -- 172.21.15.76:0/109681737 <== mon.1 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fe750001800 con 0x7fe76015cfd0 2023-11-09T03:45:06.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.627+0000 7fe75d59a640 1 -- 172.21.15.76:0/109681737 >> v1:172.21.15.113:6789/0 conn(0x7fe76015cfd0 legacy=0x7fe760161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:06.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.627+0000 7fe75d59a640 1 --2- 172.21.15.76:0/109681737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe748003970 0x7fe748003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:06.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.627+0000 7fe75d59a640 1 --2- 172.21.15.76:0/109681737 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7480047c0 0x7fe748004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:06.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.627+0000 7fe75d59a640 1 --2- 172.21.15.76:0/109681737 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe748005180 0x7fe74800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:06.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.627+0000 7fe75d59a640 1 -- 172.21.15.76:0/109681737 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76015d3b0 con 0x7fe7480047c0 2023-11-09T03:45:06.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.627+0000 7fe75d59a640 1 -- 172.21.15.76:0/109681737 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7600cea20 con 0x7fe748003970 2023-11-09T03:45:06.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.627+0000 7fe75d59a640 1 -- 172.21.15.76:0/109681737 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe74800bef0 con 0x7fe748005180 2023-11-09T03:45:06.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.627+0000 7fe75dd9b640 1 --2- 172.21.15.76:0/109681737 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7480047c0 0x7fe748004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:06.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.627+0000 7fe75e59c640 1 --2- 172.21.15.76:0/109681737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe748003970 0x7fe748003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:06.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.627+0000 7fe75ed9d640 1 --2- 172.21.15.76:0/109681737 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe748005180 0x7fe74800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:06.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.628+0000 7fe75dd9b640 1 -- 172.21.15.76:0/109681737 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe748005180 msgr2=0x7fe74800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:06.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.628+0000 7fe75dd9b640 1 --2- 172.21.15.76:0/109681737 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe748005180 0x7fe74800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:06.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.628+0000 7fe75dd9b640 1 -- 172.21.15.76:0/109681737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe748003970 msgr2=0x7fe748003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:06.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.628+0000 7fe75dd9b640 1 --2- 172.21.15.76:0/109681737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe748003970 0x7fe748003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:06.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.628+0000 7fe75dd9b640 1 -- 172.21.15.76:0/109681737 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe748003520 con 0x7fe7480047c0 2023-11-09T03:45:06.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.629+0000 7fe75dd9b640 1 --2- 172.21.15.76:0/109681737 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7480047c0 0x7fe748004c40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe75400b9e0 tx=0x7fe7480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f75b434f35aa2f server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:06.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.629+0000 7fe75d59a640 1 -- 172.21.15.76:0/109681737 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754032620 con 0x7fe7480047c0 2023-11-09T03:45:06.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.629+0000 7fe75d59a640 1 -- 172.21.15.76:0/109681737 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe754032780 con 0x7fe7480047c0 2023-11-09T03:45:06.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.629+0000 7fe75d59a640 1 -- 172.21.15.76:0/109681737 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754032b30 con 0x7fe7480047c0 2023-11-09T03:45:06.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.630+0000 7fe764f2c640 1 -- 172.21.15.76:0/109681737 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7480047c0 msgr2=0x7fe748004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:06.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.630+0000 7fe764f2c640 1 --2- 172.21.15.76:0/109681737 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7480047c0 0x7fe748004c40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe75400b9e0 tx=0x7fe7480057f0 comp rx=0 tx=0).stop 2023-11-09T03:45:06.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.630+0000 7fe75ed9d640 1 -- 172.21.15.76:0/109681737 reap_dead start 2023-11-09T03:45:06.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.630+0000 7fe764f2c640 1 -- 172.21.15.76:0/109681737 shutdown_connections 2023-11-09T03:45:06.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.630+0000 7fe764f2c640 1 -- 172.21.15.76:0/109681737 >> 172.21.15.76:0/109681737 conn(0x7fe76006ea20 msgr2=0x7fe76006ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:06.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.630+0000 7fe764f2c640 1 -- 172.21.15.76:0/109681737 shutdown_connections 2023-11-09T03:45:06.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.630+0000 7fe764f2c640 1 -- 172.21.15.76:0/109681737 wait complete. 2023-11-09T03:45:06.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.631+0000 7fe764f2c640 1 Processor -- start 2023-11-09T03:45:06.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.631+0000 7fe764f2c640 1 -- start start 2023-11-09T03:45:06.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.632+0000 7fe764f2c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7480047c0 0x7fe7600c5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:06.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.632+0000 7fe764f2c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7600c7ee0 0x7fe760144ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:06.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.632+0000 7fe764f2c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7600c8360 0x7fe760145220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:06.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.632+0000 7fe764f2c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7600c6b80 con 0x7fe7480047c0 2023-11-09T03:45:06.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.632+0000 7fe764f2c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7600c5f70 con 0x7fe7600c8360 2023-11-09T03:45:06.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.632+0000 7fe764f2c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7600c6250 con 0x7fe7600c7ee0 2023-11-09T03:45:06.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.632+0000 7fe75e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7480047c0 0x7fe7600c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:06.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.632+0000 7fe75e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7480047c0 0x7fe7600c5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46688/0 (socket says 172.21.15.76:46688) 2023-11-09T03:45:06.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.632+0000 7fe75e59c640 1 -- 172.21.15.76:0/3155062745 learned_addr learned my addr 172.21.15.76:0/3155062745 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:06.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.633+0000 7fe75ed9d640 1 --2- 172.21.15.76:0/3155062745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7600c8360 0x7fe760145220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:06.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.633+0000 7fe75dd9b640 1 --2- 172.21.15.76:0/3155062745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7600c7ee0 0x7fe760144ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:06.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.633+0000 7fe75e59c640 1 -- 172.21.15.76:0/3155062745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7600c7ee0 msgr2=0x7fe760144ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:06.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.633+0000 7fe75e59c640 1 --2- 172.21.15.76:0/3155062745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7600c7ee0 0x7fe760144ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:06.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.633+0000 7fe75e59c640 1 -- 172.21.15.76:0/3155062745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7600c8360 msgr2=0x7fe760145220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:06.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.633+0000 7fe75e59c640 1 --2- 172.21.15.76:0/3155062745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7600c8360 0x7fe760145220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:06.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.633+0000 7fe75e59c640 1 -- 172.21.15.76:0/3155062745 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7600c6530 con 0x7fe7480047c0 2023-11-09T03:45:06.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.633+0000 7fe75e59c640 1 --2- 172.21.15.76:0/3155062745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7480047c0 0x7fe7600c5720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe75000f7f0 tx=0x7fe75000fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:06.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.633+0000 7fe74effd640 1 -- 172.21.15.76:0/3155062745 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75000fea0 con 0x7fe7480047c0 2023-11-09T03:45:06.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.633+0000 7fe74effd640 1 -- 172.21.15.76:0/3155062745 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7500052e0 con 0x7fe7480047c0 2023-11-09T03:45:06.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.633+0000 7fe764f2c640 1 -- 172.21.15.76:0/3155062745 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7600c6810 con 0x7fe7480047c0 2023-11-09T03:45:06.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.633+0000 7fe74effd640 1 -- 172.21.15.76:0/3155062745 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75000d950 con 0x7fe7480047c0 2023-11-09T03:45:06.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.633+0000 7fe764f2c640 1 -- 172.21.15.76:0/3155062745 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7600c5c60 con 0x7fe7480047c0 2023-11-09T03:45:06.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.633+0000 7fe74effd640 1 -- 172.21.15.76:0/3155062745 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe7500035f0 con 0x7fe7480047c0 2023-11-09T03:45:06.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.633+0000 7fe74effd640 1 -- 172.21.15.76:0/3155062745 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe75001d460 con 0x7fe7480047c0 2023-11-09T03:45:06.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.634+0000 7fe764f2c640 1 -- 172.21.15.76:0/3155062745 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7600c69a0 con 0x7fe7480047c0 2023-11-09T03:45:06.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.636+0000 7fe74effd640 1 -- 172.21.15.76:0/3155062745 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fe7600cdf20 con 0x7fe7480047c0 2023-11-09T03:45:06.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.698+0000 7fe764f2c640 1 -- 172.21.15.76:0/3155062745 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fe76006d3c0 con 0x7fe7480047c0 2023-11-09T03:45:06.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.700+0000 7fe74effd640 1 -- 172.21.15.76:0/3155062745 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fe750003a80 con 0x7fe7480047c0 2023-11-09T03:45:06.700 INFO:teuthology.orchestra.run.smithi076.stderr:adjusted tunables profile to default 2023-11-09T03:45:06.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.700+0000 7fe764f2c640 1 -- 172.21.15.76:0/3155062745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7480047c0 msgr2=0x7fe7600c5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:06.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.700+0000 7fe764f2c640 1 --2- 172.21.15.76:0/3155062745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7480047c0 0x7fe7600c5720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe75000f7f0 tx=0x7fe75000fcb0 comp rx=0 tx=0).stop 2023-11-09T03:45:06.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.700+0000 7fe764f2c640 1 -- 172.21.15.76:0/3155062745 shutdown_connections 2023-11-09T03:45:06.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.700+0000 7fe764f2c640 1 --2- 172.21.15.76:0/3155062745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7600c8360 0x7fe760145220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:06.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.700+0000 7fe764f2c640 1 --2- 172.21.15.76:0/3155062745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7600c7ee0 0x7fe760144ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:06.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.700+0000 7fe764f2c640 1 --2- 172.21.15.76:0/3155062745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7480047c0 0x7fe7600c5720 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:06.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.700+0000 7fe764f2c640 1 -- 172.21.15.76:0/3155062745 >> 172.21.15.76:0/3155062745 conn(0x7fe76006ea20 msgr2=0x7fe760160770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:06.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.700+0000 7fe764f2c640 1 -- 172.21.15.76:0/3155062745 shutdown_connections 2023-11-09T03:45:06.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:06.700+0000 7fe764f2c640 1 -- 172.21.15.76:0/3155062745 wait complete. 2023-11-09T03:45:06.711 INFO:tasks.ceph:check_enable_crimson: False 2023-11-09T03:45:06.711 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-09T03:45:06.712 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:45:06.712 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-09T03:45:06.739 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:45:06.739 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-09T03:45:06.805 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:45:06.806 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-09T03:45:06.873 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:45:06.873 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-09T03:45:06.939 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:45:06.939 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-09T03:45:06.967 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:45:06.968 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-09T03:45:07.035 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:45:07.035 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-09T03:45:07.093 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-11-09T03:45:07.093+0000 7f50fc0b6040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-09T03:45:07.102 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:45:07.102 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-09T03:45:07.152 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-11-09T03:45:07.152+0000 7f50fc0b6040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-09T03:45:07.169 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 9cd7b709-ad8a-46c9-a589-494542849bbc 0 2023-11-09T03:45:07.308 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.308+0000 7f465009b640 1 Processor -- start 2023-11-09T03:45:07.309 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.309+0000 7f465009b640 1 -- start start 2023-11-09T03:45:07.310 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.309+0000 7f465009b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f464815af20 0x7f46480b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:07.311 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.309+0000 7f465009b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f46480b64c0 0x7f46480cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:07.311 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.309+0000 7f465009b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46480d01f0 con 0x7f46480cd3b0 2023-11-09T03:45:07.311 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.309+0000 7f465009b640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46480b6a40 con 0x7f464815af20 2023-11-09T03:45:07.311 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.309+0000 7f465009b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46480d04b0 con 0x7f46480b64c0 2023-11-09T03:45:07.311 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.309+0000 7f464d636640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f46480b64c0 0x7f46480cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:07.312 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.309+0000 7f464e638640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f46480cd3b0 0x7f46480b2620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48838/0 (socket says 172.21.15.113:48838) 2023-11-09T03:45:07.312 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.309+0000 7f464e638640 1 -- 172.21.15.113:0/3424441490 learned_addr learned my addr 172.21.15.113:0/3424441490 (peer_addr_for_me v1:172.21.15.113:0/0) 2023-11-09T03:45:07.312 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.310+0000 7f464de37640 1 --2- 172.21.15.113:0/3424441490 >> v2:172.21.15.76:3300/0 conn(0x7f464815af20 0x7f46480b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:07.312 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.310+0000 7f464d636640 1 -- 172.21.15.113:0/3424441490 >> v2:172.21.15.76:3300/0 conn(0x7f464815af20 msgr2=0x7f46480b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:07.312 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.310+0000 7f464d636640 1 --2- 172.21.15.113:0/3424441490 >> v2:172.21.15.76:3300/0 conn(0x7f464815af20 0x7f46480b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.313 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.310+0000 7f464d636640 1 -- 172.21.15.113:0/3424441490 >> v1:172.21.15.113:6789/0 conn(0x7f46480cd3b0 legacy=0x7f46480b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:07.313 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.310+0000 7f464d636640 1 -- 172.21.15.113:0/3424441490 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46480d0790 con 0x7f46480b64c0 2023-11-09T03:45:07.313 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.310+0000 7f464ce35640 1 -- 172.21.15.113:0/3424441490 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4197361588 0 0) 0x7f46480d01f0 con 0x7f46480cd3b0 2023-11-09T03:45:07.313 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.310+0000 7f464d636640 1 --2- 172.21.15.113:0/3424441490 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46480b64c0 0x7f46480cbc70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f464806b0c0 tx=0x7f464402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96ee47f58fb5d3d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:07.313 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.311+0000 7f464ce35640 1 -- 172.21.15.113:0/3424441490 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4644031830 con 0x7f46480b64c0 2023-11-09T03:45:07.314 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.311+0000 7f464ce35640 1 -- 172.21.15.113:0/3424441490 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4644031990 con 0x7f46480b64c0 2023-11-09T03:45:07.314 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.311+0000 7f464ce35640 1 -- 172.21.15.113:0/3424441490 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4644031d20 con 0x7f46480b64c0 2023-11-09T03:45:07.317 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.311+0000 7f465009b640 1 -- 172.21.15.113:0/3424441490 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46480b64c0 msgr2=0x7f46480cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:07.317 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.311+0000 7f465009b640 1 --2- 172.21.15.113:0/3424441490 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46480b64c0 0x7f46480cbc70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f464806b0c0 tx=0x7f464402b8d0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.317 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.311+0000 7f465009b640 1 -- 172.21.15.113:0/3424441490 shutdown_connections 2023-11-09T03:45:07.317 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.311+0000 7f465009b640 1 --2- 172.21.15.113:0/3424441490 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46480b64c0 0x7f46480cbc70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.318 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.311+0000 7f465009b640 1 --2- 172.21.15.113:0/3424441490 >> v2:172.21.15.76:3300/0 conn(0x7f464815af20 0x7f46480b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.318 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.311+0000 7f465009b640 1 -- 172.21.15.113:0/3424441490 >> 172.21.15.113:0/3424441490 conn(0x7f464806ea20 msgr2=0x7f464806ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:07.318 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.311+0000 7f465009b640 1 -- 172.21.15.113:0/3424441490 shutdown_connections 2023-11-09T03:45:07.318 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.311+0000 7f465009b640 1 -- 172.21.15.113:0/3424441490 wait complete. 2023-11-09T03:45:07.318 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.312+0000 7f465009b640 1 Processor -- start 2023-11-09T03:45:07.319 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.312+0000 7f465009b640 1 -- start start 2023-11-09T03:45:07.319 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.313+0000 7f465009b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46480b64c0 0x7f4648154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:07.319 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.313+0000 7f465009b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46480cd3b0 0x7f464806c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:07.319 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.313+0000 7f465009b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f464815af20 0x7f464806c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:07.319 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.313+0000 7f465009b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46480b2ab0 con 0x7f46480cd3b0 2023-11-09T03:45:07.320 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.313+0000 7f465009b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46480b2d90 con 0x7f46480b64c0 2023-11-09T03:45:07.320 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.313+0000 7f465009b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46480b3070 con 0x7f464815af20 2023-11-09T03:45:07.320 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.313+0000 7f464de37640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46480b64c0 0x7f4648154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:07.320 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.313+0000 7f464de37640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46480b64c0 0x7f4648154790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52510/0 (socket says 172.21.15.113:52510) 2023-11-09T03:45:07.320 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.313+0000 7f464de37640 1 -- 172.21.15.113:0/118991689 learned_addr learned my addr 172.21.15.113:0/118991689 (peer_addr_for_me v2:172.21.15.113:0/0) 2023-11-09T03:45:07.320 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.313+0000 7f464e638640 1 --2- 172.21.15.113:0/118991689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f464815af20 0x7f464806c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:07.321 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.313+0000 7f464d636640 1 --2- 172.21.15.113:0/118991689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46480cd3b0 0x7f464806c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:07.321 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.313+0000 7f464de37640 1 -- 172.21.15.113:0/118991689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f464815af20 msgr2=0x7f464806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:07.321 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.313+0000 7f464de37640 1 --2- 172.21.15.113:0/118991689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f464815af20 0x7f464806c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.321 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.313+0000 7f464de37640 1 -- 172.21.15.113:0/118991689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46480cd3b0 msgr2=0x7f464806c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:07.321 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.313+0000 7f464de37640 1 --2- 172.21.15.113:0/118991689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46480cd3b0 0x7f464806c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.321 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.313+0000 7f464de37640 1 -- 172.21.15.113:0/118991689 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4644009400 con 0x7f46480b64c0 2023-11-09T03:45:07.322 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.313+0000 7f464d636640 1 --2- 172.21.15.113:0/118991689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46480cd3b0 0x7f464806c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:07.322 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.314+0000 7f464de37640 1 --2- 172.21.15.113:0/118991689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46480b64c0 0x7f4648154790 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f463800d5f0 tx=0x7f463800dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:07.322 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.314+0000 7f463e7fc640 1 -- 172.21.15.113:0/118991689 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46380125f0 con 0x7f46480b64c0 2023-11-09T03:45:07.322 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.314+0000 7f463e7fc640 1 -- 172.21.15.113:0/118991689 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4638012750 con 0x7f46480b64c0 2023-11-09T03:45:07.322 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.314+0000 7f463e7fc640 1 -- 172.21.15.113:0/118991689 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4638012b00 con 0x7f46480b64c0 2023-11-09T03:45:07.323 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.314+0000 7f465009b640 1 -- 172.21.15.113:0/118991689 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46480b3350 con 0x7f46480b64c0 2023-11-09T03:45:07.323 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.314+0000 7f465009b640 1 -- 172.21.15.113:0/118991689 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4648154cd0 con 0x7f46480b64c0 2023-11-09T03:45:07.323 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.314+0000 7f463e7fc640 1 -- 172.21.15.113:0/118991689 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f463801e460 con 0x7f46480b64c0 2023-11-09T03:45:07.323 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.314+0000 7f463e7fc640 1 -- 172.21.15.113:0/118991689 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4638014560 con 0x7f46480b64c0 2023-11-09T03:45:07.323 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.315+0000 7f465009b640 1 -- 172.21.15.113:0/118991689 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46480008d0 con 0x7f46480b64c0 2023-11-09T03:45:07.324 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.317+0000 7f463e7fc640 1 -- 172.21.15.113:0/118991689 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f463801e8a0 con 0x7f46480b64c0 2023-11-09T03:45:07.386 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.386+0000 7f465009b640 1 -- 172.21.15.113:0/118991689 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9cd7b709-ad8a-46c9-a589-494542849bbc", "id": 0} v 0) v1 -- 0x7f464806da70 con 0x7f46480b64c0 2023-11-09T03:45:07.389 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.389+0000 7f463e7fc640 1 -- 172.21.15.113:0/118991689 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9cd7b709-ad8a-46c9-a589-494542849bbc", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f463801eb10 con 0x7f46480b64c0 2023-11-09T03:45:07.389 INFO:teuthology.orchestra.run.smithi113.stdout:0 2023-11-09T03:45:07.390 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.390+0000 7f465009b640 1 -- 172.21.15.113:0/118991689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46480b64c0 msgr2=0x7f4648154790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:07.390 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.390+0000 7f465009b640 1 --2- 172.21.15.113:0/118991689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46480b64c0 0x7f4648154790 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f463800d5f0 tx=0x7f463800dab0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.391 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.390+0000 7f465009b640 1 -- 172.21.15.113:0/118991689 shutdown_connections 2023-11-09T03:45:07.391 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.390+0000 7f465009b640 1 --2- 172.21.15.113:0/118991689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f464815af20 0x7f464806c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.391 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.390+0000 7f465009b640 1 --2- 172.21.15.113:0/118991689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46480cd3b0 0x7f464806c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.391 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.390+0000 7f465009b640 1 --2- 172.21.15.113:0/118991689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46480b64c0 0x7f4648154790 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.391 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.390+0000 7f465009b640 1 -- 172.21.15.113:0/118991689 >> 172.21.15.113:0/118991689 conn(0x7f464806ea20 msgr2=0x7f464806f130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:07.392 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.390+0000 7f465009b640 1 -- 172.21.15.113:0/118991689 shutdown_connections 2023-11-09T03:45:07.392 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.390+0000 7f465009b640 1 -- 172.21.15.113:0/118991689 wait complete. 2023-11-09T03:45:07.400 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new b4a945ee-af9f-4132-a93b-e81753a397f3 1 2023-11-09T03:45:07.460 INFO:tasks.ceph.mgr.y.smithi076.stderr:2023-11-09T03:45:07.460+0000 7fe560a32040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-09T03:45:07.500 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.500+0000 7f7f3d064640 1 Processor -- start 2023-11-09T03:45:07.501 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.500+0000 7f7f3d064640 1 -- start start 2023-11-09T03:45:07.502 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.501+0000 7f7f3d064640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7f3806a830 0x7f7f380ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:07.502 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.501+0000 7f7f3d064640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f3815e590 0x7f7f380722d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:07.503 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.501+0000 7f7f3d064640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f380ca620 con 0x7f7f3815e1b0 2023-11-09T03:45:07.503 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.501+0000 7f7f3d064640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f3806e7e0 con 0x7f7f3806a830 2023-11-09T03:45:07.503 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.501+0000 7f7f3d064640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f380cb060 con 0x7f7f3815e590 2023-11-09T03:45:07.503 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.501+0000 7f7f3759e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f3815e590 0x7f7f380722d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:07.504 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.501+0000 7f7f3659c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7f3806a830 0x7f7f380ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:07.504 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.501+0000 7f7f3659c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7f3806a830 0x7f7f380ae770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52526/0 (socket says 172.21.15.113:52526) 2023-11-09T03:45:07.504 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.501+0000 7f7f3759e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f3815e590 0x7f7f380722d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:59046/0 (socket says 172.21.15.113:59046) 2023-11-09T03:45:07.504 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.501+0000 7f7f3659c640 1 -- 172.21.15.113:0/819588859 learned_addr learned my addr 172.21.15.113:0/819588859 (peer_addr_for_me v2:172.21.15.113:0/0) 2023-11-09T03:45:07.504 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.501+0000 7f7f36d9d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f7f3815e1b0 0x7f7f3806a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48842/0 (socket says 172.21.15.113:48842) 2023-11-09T03:45:07.505 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.502+0000 7f7f3759e640 1 -- 172.21.15.113:0/819588859 >> v2:172.21.15.113:3300/0 conn(0x7f7f3806a830 msgr2=0x7f7f380ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:07.505 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.502+0000 7f7f3759e640 1 --2- 172.21.15.113:0/819588859 >> v2:172.21.15.113:3300/0 conn(0x7f7f3806a830 0x7f7f380ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.505 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.502+0000 7f7f3759e640 1 -- 172.21.15.113:0/819588859 >> v1:172.21.15.113:6789/0 conn(0x7f7f3815e1b0 legacy=0x7f7f3806a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:07.505 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.502+0000 7f7f35d9b640 1 -- 172.21.15.113:0/819588859 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3077359443 0 0) 0x7f7f380ca620 con 0x7f7f3815e1b0 2023-11-09T03:45:07.505 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.502+0000 7f7f3759e640 1 -- 172.21.15.113:0/819588859 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f380ca060 con 0x7f7f3815e590 2023-11-09T03:45:07.506 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.502+0000 7f7f3659c640 1 --2- 172.21.15.113:0/819588859 >> v2:172.21.15.113:3300/0 conn(0x7f7f3806a830 0x7f7f380ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:45:07.506 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.502+0000 7f7f3759e640 1 --2- 172.21.15.113:0/819588859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f3815e590 0x7f7f380722d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009d90 tx=0x7f7f3002be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae00c7fe32938918 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:07.506 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.502+0000 7f7f35d9b640 1 -- 172.21.15.113:0/819588859 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3002cbc0 con 0x7f7f3815e590 2023-11-09T03:45:07.506 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.502+0000 7f7f35d9b640 1 -- 172.21.15.113:0/819588859 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3002cd20 con 0x7f7f3815e590 2023-11-09T03:45:07.506 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.502+0000 7f7f35d9b640 1 -- 172.21.15.113:0/819588859 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f30034660 con 0x7f7f3815e590 2023-11-09T03:45:07.509 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.503+0000 7f7f3d064640 1 -- 172.21.15.113:0/819588859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f3815e590 msgr2=0x7f7f380722d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:07.509 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.503+0000 7f7f3d064640 1 --2- 172.21.15.113:0/819588859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f3815e590 0x7f7f380722d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009d90 tx=0x7f7f3002be10 comp rx=0 tx=0).stop 2023-11-09T03:45:07.510 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.503+0000 7f7f3d064640 1 -- 172.21.15.113:0/819588859 shutdown_connections 2023-11-09T03:45:07.510 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.503+0000 7f7f3d064640 1 --2- 172.21.15.113:0/819588859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f3815e590 0x7f7f380722d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.510 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.503+0000 7f7f3d064640 1 --2- 172.21.15.113:0/819588859 >> v2:172.21.15.113:3300/0 conn(0x7f7f3806a830 0x7f7f380ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.510 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.503+0000 7f7f3d064640 1 -- 172.21.15.113:0/819588859 >> 172.21.15.113:0/819588859 conn(0x7f7f3806d800 msgr2=0x7f7f3806dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:07.510 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.503+0000 7f7f3d064640 1 -- 172.21.15.113:0/819588859 shutdown_connections 2023-11-09T03:45:07.511 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.503+0000 7f7f3d064640 1 -- 172.21.15.113:0/819588859 wait complete. 2023-11-09T03:45:07.511 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.504+0000 7f7f3d064640 1 Processor -- start 2023-11-09T03:45:07.511 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.505+0000 7f7f3d064640 1 -- start start 2023-11-09T03:45:07.511 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.505+0000 7f7f3d064640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f3806a830 0x7f7f380a93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:07.511 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.505+0000 7f7f3d064640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f3815e1b0 0x7f7f381f0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:07.511 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.505+0000 7f7f3d064640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f3815e590 0x7f7f380aad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:07.512 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.505+0000 7f7f3d064640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f381f0f80 con 0x7f7f3806a830 2023-11-09T03:45:07.512 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.505+0000 7f7f3d064640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f380a8320 con 0x7f7f3815e590 2023-11-09T03:45:07.512 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.505+0000 7f7f3d064640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f380a8600 con 0x7f7f3815e1b0 2023-11-09T03:45:07.512 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.505+0000 7f7f3659c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f3815e1b0 0x7f7f381f0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:07.512 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.505+0000 7f7f3759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f3815e590 0x7f7f380aad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:07.513 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.505+0000 7f7f3759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f3815e590 0x7f7f380aad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52532/0 (socket says 172.21.15.113:52532) 2023-11-09T03:45:07.513 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.505+0000 7f7f3659c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f3815e1b0 0x7f7f381f0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:59048/0 (socket says 172.21.15.113:59048) 2023-11-09T03:45:07.513 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.505+0000 7f7f3759e640 1 -- 172.21.15.113:0/726972029 learned_addr learned my addr 172.21.15.113:0/726972029 (peer_addr_for_me v2:172.21.15.113:0/0) 2023-11-09T03:45:07.513 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.505+0000 7f7f36d9d640 1 --2- 172.21.15.113:0/726972029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f3806a830 0x7f7f380a93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:07.513 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.506+0000 7f7f3759e640 1 -- 172.21.15.113:0/726972029 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f3815e1b0 msgr2=0x7f7f381f0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:07.514 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.506+0000 7f7f3759e640 1 --2- 172.21.15.113:0/726972029 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f3815e1b0 0x7f7f381f0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.514 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.506+0000 7f7f3759e640 1 -- 172.21.15.113:0/726972029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f3806a830 msgr2=0x7f7f380a93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:07.514 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.506+0000 7f7f3759e640 1 --2- 172.21.15.113:0/726972029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f3806a830 0x7f7f380a93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.514 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.506+0000 7f7f3759e640 1 -- 172.21.15.113:0/726972029 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2c0094d0 con 0x7f7f3815e590 2023-11-09T03:45:07.514 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.506+0000 7f7f3759e640 1 --2- 172.21.15.113:0/726972029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f3815e590 0x7f7f380aad40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009d90 tx=0x7f7f30004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:07.514 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.506+0000 7f7f1f7fe640 1 -- 172.21.15.113:0/726972029 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f30040650 con 0x7f7f3815e590 2023-11-09T03:45:07.515 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.506+0000 7f7f1f7fe640 1 -- 172.21.15.113:0/726972029 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3002ce90 con 0x7f7f3815e590 2023-11-09T03:45:07.515 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.506+0000 7f7f1f7fe640 1 -- 172.21.15.113:0/726972029 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f300409f0 con 0x7f7f3815e590 2023-11-09T03:45:07.515 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.506+0000 7f7f3d064640 1 -- 172.21.15.113:0/726972029 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f30009a70 con 0x7f7f3815e590 2023-11-09T03:45:07.515 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.506+0000 7f7f3d064640 1 -- 172.21.15.113:0/726972029 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f380a8bc0 con 0x7f7f3815e590 2023-11-09T03:45:07.515 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.506+0000 7f7f1f7fe640 1 -- 172.21.15.113:0/726972029 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7f30049400 con 0x7f7f3815e590 2023-11-09T03:45:07.516 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.506+0000 7f7f1f7fe640 1 -- 172.21.15.113:0/726972029 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f7f3003e7b0 con 0x7f7f3815e590 2023-11-09T03:45:07.516 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.507+0000 7f7f3d064640 1 -- 172.21.15.113:0/726972029 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f38072e20 con 0x7f7f3815e590 2023-11-09T03:45:07.516 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.509+0000 7f7f1f7fe640 1 -- 172.21.15.113:0/726972029 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f7f30067b40 con 0x7f7f3815e590 2023-11-09T03:45:07.516 INFO:tasks.ceph.mgr.y.smithi076.stderr:2023-11-09T03:45:07.511+0000 7fe560a32040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-09T03:45:07.580 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.580+0000 7f7f3d064640 1 -- 172.21.15.113:0/726972029 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b4a945ee-af9f-4132-a93b-e81753a397f3", "id": 1} v 0) v1 -- 0x7f7f380674e0 con 0x7f7f3815e590 2023-11-09T03:45:07.583 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.583+0000 7f7f1f7fe640 1 -- 172.21.15.113:0/726972029 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b4a945ee-af9f-4132-a93b-e81753a397f3", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7f30056030 con 0x7f7f3815e590 2023-11-09T03:45:07.584 INFO:teuthology.orchestra.run.smithi113.stdout:1 2023-11-09T03:45:07.585 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.584+0000 7f7f3d064640 1 -- 172.21.15.113:0/726972029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f3815e590 msgr2=0x7f7f380aad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:07.585 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.584+0000 7f7f3d064640 1 --2- 172.21.15.113:0/726972029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f3815e590 0x7f7f380aad40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009d90 tx=0x7f7f30004a40 comp rx=0 tx=0).stop 2023-11-09T03:45:07.585 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.584+0000 7f7f3d064640 1 -- 172.21.15.113:0/726972029 shutdown_connections 2023-11-09T03:45:07.585 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.584+0000 7f7f3d064640 1 --2- 172.21.15.113:0/726972029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f3815e590 0x7f7f380aad40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.585 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.584+0000 7f7f3d064640 1 --2- 172.21.15.113:0/726972029 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f3815e1b0 0x7f7f381f0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.586 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.584+0000 7f7f3d064640 1 --2- 172.21.15.113:0/726972029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f3806a830 0x7f7f380a93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.586 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.584+0000 7f7f3d064640 1 -- 172.21.15.113:0/726972029 >> 172.21.15.113:0/726972029 conn(0x7f7f3806d800 msgr2=0x7f7f380b1630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:07.586 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.584+0000 7f7f3d064640 1 -- 172.21.15.113:0/726972029 shutdown_connections 2023-11-09T03:45:07.586 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.584+0000 7f7f3d064640 1 -- 172.21.15.113:0/726972029 wait complete. 2023-11-09T03:45:07.595 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 7b7745f1-856e-4a1c-a9ff-b13ccff6881a 2 2023-11-09T03:45:07.695 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.695+0000 7f9805ae5640 1 Processor -- start 2023-11-09T03:45:07.695 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.695+0000 7f9805ae5640 1 -- start start 2023-11-09T03:45:07.696 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.695+0000 7f9805ae5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f980015cfd0 0x7f980006c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:07.696 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.695+0000 7f9805ae5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98000cea20 con 0x7f98000cd570 2023-11-09T03:45:07.696 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.695+0000 7f9805ae5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98000cf610 con 0x7f98000cf180 2023-11-09T03:45:07.697 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.695+0000 7f9805ae5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f980015d3b0 con 0x7f980015cfd0 2023-11-09T03:45:07.697 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.696+0000 7f97ff7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f980015cfd0 0x7f980006c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:07.697 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.696+0000 7f97ff7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f980015cfd0 0x7f980006c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52536/0 (socket says 172.21.15.113:52536) 2023-11-09T03:45:07.697 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.696+0000 7f97ff7fe640 1 -- 172.21.15.113:0/69507724 learned_addr learned my addr 172.21.15.113:0/69507724 (peer_addr_for_me v2:172.21.15.113:0/0) 2023-11-09T03:45:07.697 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.696+0000 7f97feffd640 1 --1- 172.21.15.113:0/69507724 >> v1:172.21.15.113:6789/0 conn(0x7f98000cf180 0x7f9800161440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48844/0 (socket says 172.21.15.113:48844) 2023-11-09T03:45:07.697 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.696+0000 7f97ff7fe640 1 -- 172.21.15.113:0/69507724 >> v1:172.21.15.113:6789/0 conn(0x7f98000cf180 legacy=0x7f9800161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:07.698 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.696+0000 7f97ff7fe640 1 -- 172.21.15.113:0/69507724 >> v1:172.21.15.76:6789/0 conn(0x7f98000cd570 legacy=0x7f98000cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:07.698 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.696+0000 7f97ff7fe640 1 -- 172.21.15.113:0/69507724 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98000cf8f0 con 0x7f980015cfd0 2023-11-09T03:45:07.699 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.697+0000 7f97ff7fe640 1 --2- 172.21.15.113:0/69507724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f980015cfd0 0x7f980006c0a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f97f0009870 tx=0x7f97f002b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27483630ce9b271c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:07.699 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.697+0000 7f97fe7fc640 1 -- 172.21.15.113:0/69507724 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f0030860 con 0x7f980015cfd0 2023-11-09T03:45:07.699 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.697+0000 7f97fe7fc640 1 -- 172.21.15.113:0/69507724 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f00309c0 con 0x7f980015cfd0 2023-11-09T03:45:07.700 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.697+0000 7f97fe7fc640 1 -- 172.21.15.113:0/69507724 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f0030d50 con 0x7f980015cfd0 2023-11-09T03:45:07.700 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.697+0000 7f9805ae5640 1 -- 172.21.15.113:0/69507724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f980015cfd0 msgr2=0x7f980006c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:07.700 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.697+0000 7f9805ae5640 1 --2- 172.21.15.113:0/69507724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f980015cfd0 0x7f980006c0a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f97f0009870 tx=0x7f97f002b9a0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.700 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.697+0000 7f9805ae5640 1 -- 172.21.15.113:0/69507724 shutdown_connections 2023-11-09T03:45:07.700 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.697+0000 7f9805ae5640 1 --2- 172.21.15.113:0/69507724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f980015cfd0 0x7f980006c0a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.701 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.697+0000 7f9805ae5640 1 -- 172.21.15.113:0/69507724 >> 172.21.15.113:0/69507724 conn(0x7f980006ea20 msgr2=0x7f980006ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:07.701 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.697+0000 7f9805ae5640 1 -- 172.21.15.113:0/69507724 shutdown_connections 2023-11-09T03:45:07.701 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.697+0000 7f9805ae5640 1 -- 172.21.15.113:0/69507724 wait complete. 2023-11-09T03:45:07.701 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.699+0000 7f9805ae5640 1 Processor -- start 2023-11-09T03:45:07.701 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.699+0000 7f9805ae5640 1 -- start start 2023-11-09T03:45:07.703 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.701+0000 7f9805ae5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98000cd570 0x7f98000b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:07.704 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.701+0000 7f9805ae5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f980015cfd0 0x7f9800150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:07.704 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.701+0000 7f9805ae5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9800153000 0x7f98000b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:07.704 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.701+0000 7f9805ae5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98000b64d0 con 0x7f9800153000 2023-11-09T03:45:07.704 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.701+0000 7f9805ae5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9800151370 con 0x7f98000cd570 2023-11-09T03:45:07.704 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.701+0000 7f9805ae5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9800151650 con 0x7f980015cfd0 2023-11-09T03:45:07.704 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.701+0000 7f97feffd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f980015cfd0 0x7f9800150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:07.705 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.701+0000 7f97ff7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98000cd570 0x7f98000b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:07.705 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.701+0000 7f97feffd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f980015cfd0 0x7f9800150a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:59064/0 (socket says 172.21.15.113:59064) 2023-11-09T03:45:07.705 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.701+0000 7f97feffd640 1 -- 172.21.15.113:0/1247545304 learned_addr learned my addr 172.21.15.113:0/1247545304 (peer_addr_for_me v2:172.21.15.113:0/0) 2023-11-09T03:45:07.705 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.701+0000 7f97fffff640 1 --2- 172.21.15.113:0/1247545304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9800153000 0x7f98000b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:07.705 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.701+0000 7f97ff7fe640 1 -- 172.21.15.113:0/1247545304 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f980015cfd0 msgr2=0x7f9800150a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:07.705 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.701+0000 7f97ff7fe640 1 --2- 172.21.15.113:0/1247545304 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f980015cfd0 0x7f9800150a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.706 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.701+0000 7f97ff7fe640 1 -- 172.21.15.113:0/1247545304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9800153000 msgr2=0x7f98000b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:07.706 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.701+0000 7f97ff7fe640 1 --2- 172.21.15.113:0/1247545304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9800153000 0x7f98000b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.706 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.701+0000 7f97ff7fe640 1 -- 172.21.15.113:0/1247545304 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f00094d0 con 0x7f98000cd570 2023-11-09T03:45:07.706 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.701+0000 7f97ff7fe640 1 --2- 172.21.15.113:0/1247545304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98000cd570 0x7f98000b5f90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f97f0033000 tx=0x7f97f0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:07.706 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.701+0000 7f97e7fff640 1 -- 172.21.15.113:0/1247545304 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f0039d90 con 0x7f98000cd570 2023-11-09T03:45:07.707 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.701+0000 7f97e7fff640 1 -- 172.21.15.113:0/1247545304 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f0043410 con 0x7f98000cd570 2023-11-09T03:45:07.707 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.701+0000 7f97e7fff640 1 -- 172.21.15.113:0/1247545304 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f00437c0 con 0x7f98000cd570 2023-11-09T03:45:07.707 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.701+0000 7f9805ae5640 1 -- 172.21.15.113:0/1247545304 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9800151930 con 0x7f98000cd570 2023-11-09T03:45:07.707 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.701+0000 7f9805ae5640 1 -- 172.21.15.113:0/1247545304 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9800151ef0 con 0x7f98000cd570 2023-11-09T03:45:07.707 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.702+0000 7f97e7fff640 1 -- 172.21.15.113:0/1247545304 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f97f0043c20 con 0x7f98000cd570 2023-11-09T03:45:07.707 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.702+0000 7f97e7fff640 1 -- 172.21.15.113:0/1247545304 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f97f002f3b0 con 0x7f98000cd570 2023-11-09T03:45:07.708 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.702+0000 7f9805ae5640 1 -- 172.21.15.113:0/1247545304 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f980006bb30 con 0x7f98000cd570 2023-11-09T03:45:07.708 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.704+0000 7f97e7fff640 1 -- 172.21.15.113:0/1247545304 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f97f0051060 con 0x7f98000cd570 2023-11-09T03:45:07.710 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-11-09T03:45:07.710+0000 7f50fc0b6040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-09T03:45:07.764 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-11-09T03:45:07.764+0000 7f50fc0b6040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-09T03:45:07.775 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.776+0000 7f9805ae5640 1 -- 172.21.15.113:0/1247545304 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7b7745f1-856e-4a1c-a9ff-b13ccff6881a", "id": 2} v 0) v1 -- 0x7f980006d3c0 con 0x7f98000cd570 2023-11-09T03:45:07.779 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.779+0000 7f97e7fff640 1 -- 172.21.15.113:0/1247545304 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7b7745f1-856e-4a1c-a9ff-b13ccff6881a", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f97f0048070 con 0x7f98000cd570 2023-11-09T03:45:07.779 INFO:teuthology.orchestra.run.smithi113.stdout:2 2023-11-09T03:45:07.780 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.780+0000 7f9805ae5640 1 -- 172.21.15.113:0/1247545304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98000cd570 msgr2=0x7f98000b5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:07.781 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.780+0000 7f9805ae5640 1 --2- 172.21.15.113:0/1247545304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98000cd570 0x7f98000b5f90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f97f0033000 tx=0x7f97f0004730 comp rx=0 tx=0).stop 2023-11-09T03:45:07.781 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.780+0000 7f9805ae5640 1 -- 172.21.15.113:0/1247545304 shutdown_connections 2023-11-09T03:45:07.781 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.780+0000 7f9805ae5640 1 --2- 172.21.15.113:0/1247545304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9800153000 0x7f98000b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.781 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.780+0000 7f9805ae5640 1 --2- 172.21.15.113:0/1247545304 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f980015cfd0 0x7f9800150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.781 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.780+0000 7f9805ae5640 1 --2- 172.21.15.113:0/1247545304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98000cd570 0x7f98000b5f90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.781 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.780+0000 7f9805ae5640 1 -- 172.21.15.113:0/1247545304 >> 172.21.15.113:0/1247545304 conn(0x7f980006ea20 msgr2=0x7f980015b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:07.782 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.780+0000 7f9805ae5640 1 -- 172.21.15.113:0/1247545304 shutdown_connections 2023-11-09T03:45:07.782 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.780+0000 7f9805ae5640 1 -- 172.21.15.113:0/1247545304 wait complete. 2023-11-09T03:45:07.791 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 0d039f94-1da7-433b-aaeb-a2db8515d851 3 2023-11-09T03:45:07.811 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-11-09T03:45:07.812+0000 7f50fc0b6040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-09T03:45:07.868 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-11-09T03:45:07.868+0000 7f50fc0b6040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-09T03:45:07.891 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.891+0000 7f0104b97640 1 Processor -- start 2023-11-09T03:45:07.891 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.891+0000 7f0104b97640 1 -- start start 2023-11-09T03:45:07.892 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.892+0000 7f0104b97640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01000cf180 0x7f010006c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:07.892 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.892+0000 7f0104b97640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f01000cd570 0x7f01000cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:07.893 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.892+0000 7f0104b97640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01000cdf20 con 0x7f010015cfd0 2023-11-09T03:45:07.893 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.892+0000 7f0104b97640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f010015d3b0 con 0x7f01000cd570 2023-11-09T03:45:07.893 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.892+0000 7f0104b97640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01000cea20 con 0x7f01000cf180 2023-11-09T03:45:07.893 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.892+0000 7f00fed9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f01000cd570 0x7f01000cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:07.893 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.892+0000 7f00fdd9b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01000cf180 0x7f010006c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:07.894 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.892+0000 7f00fed9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f01000cd570 0x7f01000cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52552/0 (socket says 172.21.15.113:52552) 2023-11-09T03:45:07.894 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.892+0000 7f00fed9d640 1 -- 172.21.15.113:0/665691776 learned_addr learned my addr 172.21.15.113:0/665691776 (peer_addr_for_me v2:172.21.15.113:0/0) 2023-11-09T03:45:07.894 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.892+0000 7f00fed9d640 1 -- 172.21.15.113:0/665691776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01000cf180 msgr2=0x7f010006c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:07.894 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.892+0000 7f00fed9d640 1 --2- 172.21.15.113:0/665691776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01000cf180 0x7f010006c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.894 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.892+0000 7f00fed9d640 1 -- 172.21.15.113:0/665691776 >> v1:172.21.15.113:6789/0 conn(0x7f010015cfd0 legacy=0x7f0100161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:07.895 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.892+0000 7f00fed9d640 1 -- 172.21.15.113:0/665691776 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01000cf610 con 0x7f01000cd570 2023-11-09T03:45:07.895 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.892+0000 7f00fd59a640 1 -- 172.21.15.113:0/665691776 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3870629810 0 0) 0x7f00f0001110 con 0x7f010015cfd0 2023-11-09T03:45:07.895 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.892+0000 7f00fdd9b640 1 --2- 172.21.15.113:0/665691776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01000cf180 0x7f010006c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:45:07.898 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.893+0000 7f00fed9d640 1 --2- 172.21.15.113:0/665691776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01000cd570 0x7f01000cd9e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f00f8009e50 tx=0x7f00f802bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74aedf80b6ed5e39 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:07.898 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.893+0000 7f00fd59a640 1 -- 172.21.15.113:0/665691776 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f802ca70 con 0x7f01000cd570 2023-11-09T03:45:07.898 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.893+0000 7f00fd59a640 1 -- 172.21.15.113:0/665691776 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f802cbd0 con 0x7f01000cd570 2023-11-09T03:45:07.899 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.893+0000 7f00fd59a640 1 -- 172.21.15.113:0/665691776 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f80343f0 con 0x7f01000cd570 2023-11-09T03:45:07.899 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.894+0000 7f0104b97640 1 -- 172.21.15.113:0/665691776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01000cd570 msgr2=0x7f01000cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:07.899 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.894+0000 7f0104b97640 1 --2- 172.21.15.113:0/665691776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01000cd570 0x7f01000cd9e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f00f8009e50 tx=0x7f00f802bed0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.899 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.894+0000 7f0104b97640 1 -- 172.21.15.113:0/665691776 shutdown_connections 2023-11-09T03:45:07.899 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.894+0000 7f0104b97640 1 --2- 172.21.15.113:0/665691776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01000cd570 0x7f01000cd9e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.900 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.894+0000 7f0104b97640 1 --2- 172.21.15.113:0/665691776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01000cf180 0x7f010006c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.900 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.894+0000 7f0104b97640 1 -- 172.21.15.113:0/665691776 >> 172.21.15.113:0/665691776 conn(0x7f010006ea20 msgr2=0x7f010006ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:07.900 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.894+0000 7f0104b97640 1 -- 172.21.15.113:0/665691776 shutdown_connections 2023-11-09T03:45:07.900 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.894+0000 7f0104b97640 1 -- 172.21.15.113:0/665691776 wait complete. 2023-11-09T03:45:07.900 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.895+0000 7f0104b97640 1 Processor -- start 2023-11-09T03:45:07.900 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.895+0000 7f0104b97640 1 -- start start 2023-11-09T03:45:07.901 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.896+0000 7f0104b97640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01000cd570 0x7f0100154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:07.901 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.896+0000 7f0104b97640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f010015cfd0 0x7f01000c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:07.901 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.896+0000 7f0104b97640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01000c5620 0x7f01000b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:07.901 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.896+0000 7f0104b97640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01000c5b10 con 0x7f01000cd570 2023-11-09T03:45:07.901 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.896+0000 7f0104b97640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01000c5df0 con 0x7f010015cfd0 2023-11-09T03:45:07.902 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.896+0000 7f0104b97640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01000c9120 con 0x7f01000c5620 2023-11-09T03:45:07.902 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.896+0000 7f00fdd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f010015cfd0 0x7f01000c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:07.902 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.896+0000 7f00fed9d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01000c5620 0x7f01000b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:07.902 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.896+0000 7f00fdd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f010015cfd0 0x7f01000c50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52554/0 (socket says 172.21.15.113:52554) 2023-11-09T03:45:07.902 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.896+0000 7f00fdd9b640 1 -- 172.21.15.113:0/2810174302 learned_addr learned my addr 172.21.15.113:0/2810174302 (peer_addr_for_me v2:172.21.15.113:0/0) 2023-11-09T03:45:07.903 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.896+0000 7f00fe59c640 1 --2- 172.21.15.113:0/2810174302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01000cd570 0x7f0100154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:07.903 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.896+0000 7f00fed9d640 1 -- 172.21.15.113:0/2810174302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f010015cfd0 msgr2=0x7f01000c50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:07.903 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.896+0000 7f00fed9d640 1 --2- 172.21.15.113:0/2810174302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f010015cfd0 0x7f01000c50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.903 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.896+0000 7f00fed9d640 1 -- 172.21.15.113:0/2810174302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01000cd570 msgr2=0x7f0100154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:07.903 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.896+0000 7f00fed9d640 1 --2- 172.21.15.113:0/2810174302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01000cd570 0x7f0100154790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.904 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.896+0000 7f00fed9d640 1 -- 172.21.15.113:0/2810174302 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f4009400 con 0x7f01000c5620 2023-11-09T03:45:07.904 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.896+0000 7f00fdd9b640 1 --2- 172.21.15.113:0/2810174302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f010015cfd0 0x7f01000c50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:45:07.904 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.896+0000 7f00fed9d640 1 --2- 172.21.15.113:0/2810174302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01000c5620 0x7f01000b19d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f00f8032000 tx=0x7f00f8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:07.904 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.897+0000 7f00eeffd640 1 -- 172.21.15.113:0/2810174302 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f8009510 con 0x7f01000c5620 2023-11-09T03:45:07.904 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.897+0000 7f00eeffd640 1 -- 172.21.15.113:0/2810174302 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f803e450 con 0x7f01000c5620 2023-11-09T03:45:07.904 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.897+0000 7f0104b97640 1 -- 172.21.15.113:0/2810174302 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00f8009b30 con 0x7f01000c5620 2023-11-09T03:45:07.905 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.897+0000 7f00eeffd640 1 -- 172.21.15.113:0/2810174302 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f803e820 con 0x7f01000c5620 2023-11-09T03:45:07.905 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.897+0000 7f0104b97640 1 -- 172.21.15.113:0/2810174302 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01000c96e0 con 0x7f01000c5620 2023-11-09T03:45:07.905 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.897+0000 7f00eeffd640 1 -- 172.21.15.113:0/2810174302 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f00f803ec80 con 0x7f01000c5620 2023-11-09T03:45:07.905 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.897+0000 7f00eeffd640 1 -- 172.21.15.113:0/2810174302 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f00f803f400 con 0x7f01000c5620 2023-11-09T03:45:07.905 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.898+0000 7f0104b97640 1 -- 172.21.15.113:0/2810174302 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01000b3480 con 0x7f01000c5620 2023-11-09T03:45:07.906 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.900+0000 7f00eeffd640 1 -- 172.21.15.113:0/2810174302 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f00f8043020 con 0x7f01000c5620 2023-11-09T03:45:07.913 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-11-09T03:45:07.913+0000 7f50fc0b6040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-09T03:45:07.971 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.971+0000 7f0104b97640 1 -- 172.21.15.113:0/2810174302 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0d039f94-1da7-433b-aaeb-a2db8515d851", "id": 3} v 0) v1 -- 0x7f010006b300 con 0x7f01000c5620 2023-11-09T03:45:07.975 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.975+0000 7f00eeffd640 1 -- 172.21.15.113:0/2810174302 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0d039f94-1da7-433b-aaeb-a2db8515d851", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f00f8048400 con 0x7f01000c5620 2023-11-09T03:45:07.975 INFO:teuthology.orchestra.run.smithi113.stdout:3 2023-11-09T03:45:07.976 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.976+0000 7f0104b97640 1 -- 172.21.15.113:0/2810174302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01000c5620 msgr2=0x7f01000b19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:07.976 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.976+0000 7f0104b97640 1 --2- 172.21.15.113:0/2810174302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01000c5620 0x7f01000b19d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f00f8032000 tx=0x7f00f8004a40 comp rx=0 tx=0).stop 2023-11-09T03:45:07.976 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.976+0000 7f0104b97640 1 -- 172.21.15.113:0/2810174302 shutdown_connections 2023-11-09T03:45:07.977 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.976+0000 7f0104b97640 1 --2- 172.21.15.113:0/2810174302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01000c5620 0x7f01000b19d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.977 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.976+0000 7f0104b97640 1 --2- 172.21.15.113:0/2810174302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f010015cfd0 0x7f01000c50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.977 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.976+0000 7f0104b97640 1 --2- 172.21.15.113:0/2810174302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01000cd570 0x7f0100154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:07.977 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.976+0000 7f0104b97640 1 -- 172.21.15.113:0/2810174302 >> 172.21.15.113:0/2810174302 conn(0x7f010006ea20 msgr2=0x7f01000cb500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:07.977 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.976+0000 7f0104b97640 1 -- 172.21.15.113:0/2810174302 shutdown_connections 2023-11-09T03:45:07.978 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:07.976+0000 7f0104b97640 1 -- 172.21.15.113:0/2810174302 wait complete. 2023-11-09T03:45:07.986 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new c12b0e9b-1482-4980-8e43-5174eab87fa5 4 2023-11-09T03:45:08.017 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-11-09T03:45:08.017+0000 7f50fc0b6040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-09T03:45:08.049 INFO:tasks.ceph.mgr.y.smithi076.stderr:2023-11-09T03:45:08.049+0000 7fe560a32040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-09T03:45:08.090 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-11-09T03:45:08.090+0000 7f50fc0b6040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-09T03:45:08.100 INFO:tasks.ceph.mgr.y.smithi076.stderr:2023-11-09T03:45:08.100+0000 7fe560a32040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-09T03:45:08.116 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.117+0000 7f36cdc34640 1 Processor -- start 2023-11-09T03:45:08.117 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.117+0000 7f36cdc34640 1 -- start start 2023-11-09T03:45:08.118 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.117+0000 7f36cdc34640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36c8153e60 0x7f36c815dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:08.119 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.117+0000 7f36cdc34640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f36c80b4880 0x7f36c80b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:08.119 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.117+0000 7f36cdc34640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36c80cde60 con 0x7f36c80cd6d0 2023-11-09T03:45:08.119 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.117+0000 7f36cdc34640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36c80b5230 con 0x7f36c80b4880 2023-11-09T03:45:08.119 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.117+0000 7f36cdc34640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c8160f90 con 0x7f36c8153e60 2023-11-09T03:45:08.119 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.117+0000 7f36c77fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36c8153e60 0x7f36c815dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:08.120 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.117+0000 7f36c7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f36c80b4880 0x7f36c80b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:08.120 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.118+0000 7f36c7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f36c80b4880 0x7f36c80b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52558/0 (socket says 172.21.15.113:52558) 2023-11-09T03:45:08.120 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.118+0000 7f36c77fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36c8153e60 0x7f36c815dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:59090/0 (socket says 172.21.15.113:59090) 2023-11-09T03:45:08.120 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.118+0000 7f36c7fff640 1 -- 172.21.15.113:0/3215020354 learned_addr learned my addr 172.21.15.113:0/3215020354 (peer_addr_for_me v2:172.21.15.113:0/0) 2023-11-09T03:45:08.120 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.118+0000 7f36c77fe640 1 -- 172.21.15.113:0/3215020354 >> v2:172.21.15.113:3300/0 conn(0x7f36c80b4880 msgr2=0x7f36c80b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:08.121 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.118+0000 7f36c77fe640 1 --2- 172.21.15.113:0/3215020354 >> v2:172.21.15.113:3300/0 conn(0x7f36c80b4880 0x7f36c80b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.121 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.118+0000 7f36c77fe640 1 -- 172.21.15.113:0/3215020354 >> v1:172.21.15.76:6789/0 conn(0x7f36c80cd6d0 legacy=0x7f36c8161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:08.121 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.118+0000 7f36c77fe640 1 -- 172.21.15.113:0/3215020354 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c815e400 con 0x7f36c8153e60 2023-11-09T03:45:08.121 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.118+0000 7f36c7fff640 1 --2- 172.21.15.113:0/3215020354 >> v2:172.21.15.113:3300/0 conn(0x7f36c80b4880 0x7f36c80b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:45:08.121 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.119+0000 7f36c77fe640 1 --2- 172.21.15.113:0/3215020354 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36c8153e60 0x7f36c815dec0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f36b80097a0 tx=0x7f36b802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb1b2a9cfcaaf24 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:08.121 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.119+0000 7f36c67fc640 1 -- 172.21.15.113:0/3215020354 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b8031830 con 0x7f36c8153e60 2023-11-09T03:45:08.122 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.119+0000 7f36c67fc640 1 -- 172.21.15.113:0/3215020354 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b8031990 con 0x7f36c8153e60 2023-11-09T03:45:08.122 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.119+0000 7f36c67fc640 1 -- 172.21.15.113:0/3215020354 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b8031d20 con 0x7f36c8153e60 2023-11-09T03:45:08.124 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.119+0000 7f36cdc34640 1 -- 172.21.15.113:0/3215020354 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36c8153e60 msgr2=0x7f36c815dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:08.124 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.119+0000 7f36cdc34640 1 --2- 172.21.15.113:0/3215020354 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36c8153e60 0x7f36c815dec0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f36b80097a0 tx=0x7f36b802b8d0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.125 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.119+0000 7f36cdc34640 1 -- 172.21.15.113:0/3215020354 shutdown_connections 2023-11-09T03:45:08.125 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.119+0000 7f36cdc34640 1 --2- 172.21.15.113:0/3215020354 >> v2:172.21.15.113:3300/0 conn(0x7f36c80b4880 0x7f36c80b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.125 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.119+0000 7f36cdc34640 1 --2- 172.21.15.113:0/3215020354 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36c8153e60 0x7f36c815dec0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.125 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.119+0000 7f36cdc34640 1 -- 172.21.15.113:0/3215020354 >> 172.21.15.113:0/3215020354 conn(0x7f36c80cf880 msgr2=0x7f36c80cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:08.125 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.119+0000 7f36cdc34640 1 -- 172.21.15.113:0/3215020354 shutdown_connections 2023-11-09T03:45:08.125 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.119+0000 7f36cdc34640 1 -- 172.21.15.113:0/3215020354 wait complete. 2023-11-09T03:45:08.126 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.120+0000 7f36cdc34640 1 Processor -- start 2023-11-09T03:45:08.126 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.121+0000 7f36cdc34640 1 -- start start 2023-11-09T03:45:08.126 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.121+0000 7f36cdc34640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36c80b4880 0x7f36c81f6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:08.126 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.121+0000 7f36cdc34640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36c80cd6d0 0x7f36c8153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:08.126 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.121+0000 7f36cdc34640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36c8153e60 0x7f36c81f2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:08.127 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.121+0000 7f36cdc34640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c80c7670 con 0x7f36c8153e60 2023-11-09T03:45:08.127 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.121+0000 7f36cdc34640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c81f7120 con 0x7f36c80cd6d0 2023-11-09T03:45:08.127 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.121+0000 7f36cdc34640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c80c6a60 con 0x7f36c80b4880 2023-11-09T03:45:08.127 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.121+0000 7f36c77fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36c80b4880 0x7f36c81f6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:08.127 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.121+0000 7f36c6ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36c80cd6d0 0x7f36c8153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:08.127 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.121+0000 7f36c77fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36c80b4880 0x7f36c81f6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:59106/0 (socket says 172.21.15.113:59106) 2023-11-09T03:45:08.128 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.121+0000 7f36c77fe640 1 -- 172.21.15.113:0/1866775922 learned_addr learned my addr 172.21.15.113:0/1866775922 (peer_addr_for_me v2:172.21.15.113:0/0) 2023-11-09T03:45:08.128 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.122+0000 7f36c7fff640 1 --2- 172.21.15.113:0/1866775922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36c8153e60 0x7f36c81f2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:08.128 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.122+0000 7f36c6ffd640 1 -- 172.21.15.113:0/1866775922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36c80b4880 msgr2=0x7f36c81f6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:08.128 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.122+0000 7f36c6ffd640 1 --2- 172.21.15.113:0/1866775922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36c80b4880 0x7f36c81f6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.128 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.122+0000 7f36c6ffd640 1 -- 172.21.15.113:0/1866775922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36c8153e60 msgr2=0x7f36c81f2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:08.128 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.122+0000 7f36c6ffd640 1 --2- 172.21.15.113:0/1866775922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36c8153e60 0x7f36c81f2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.129 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.122+0000 7f36c6ffd640 1 -- 172.21.15.113:0/1866775922 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c0009b30 con 0x7f36c80cd6d0 2023-11-09T03:45:08.129 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.122+0000 7f36c77fe640 1 --2- 172.21.15.113:0/1866775922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36c80b4880 0x7f36c81f6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:45:08.129 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.122+0000 7f36c6ffd640 1 --2- 172.21.15.113:0/1866775922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36c80cd6d0 0x7f36c8153000 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f36c80cdb10 tx=0x7f36bc00c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:08.129 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.122+0000 7f36affff640 1 -- 172.21.15.113:0/1866775922 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc0052d0 con 0x7f36c80cd6d0 2023-11-09T03:45:08.129 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.122+0000 7f36affff640 1 -- 172.21.15.113:0/1866775922 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36bc005430 con 0x7f36c80cd6d0 2023-11-09T03:45:08.129 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.122+0000 7f36cdc34640 1 -- 172.21.15.113:0/1866775922 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b8009400 con 0x7f36c80cd6d0 2023-11-09T03:45:08.130 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.122+0000 7f36affff640 1 -- 172.21.15.113:0/1866775922 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc003830 con 0x7f36c80cd6d0 2023-11-09T03:45:08.130 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.122+0000 7f36cdc34640 1 -- 172.21.15.113:0/1866775922 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c80c7020 con 0x7f36c80cd6d0 2023-11-09T03:45:08.130 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.122+0000 7f36affff640 1 -- 172.21.15.113:0/1866775922 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f36bc003c90 con 0x7f36c80cd6d0 2023-11-09T03:45:08.130 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.122+0000 7f36affff640 1 -- 172.21.15.113:0/1866775922 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f36bc012930 con 0x7f36c80cd6d0 2023-11-09T03:45:08.130 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.123+0000 7f36cdc34640 1 -- 172.21.15.113:0/1866775922 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c80c77e0 con 0x7f36c80cd6d0 2023-11-09T03:45:08.131 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.125+0000 7f36affff640 1 -- 172.21.15.113:0/1866775922 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f36bc017070 con 0x7f36c80cd6d0 2023-11-09T03:45:08.139 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-11-09T03:45:08.139+0000 7f50fc0b6040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-09T03:45:08.145 INFO:tasks.ceph.mgr.y.smithi076.stderr:2023-11-09T03:45:08.145+0000 7fe560a32040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-09T03:45:08.191 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-11-09T03:45:08.191+0000 7f50fc0b6040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-09T03:45:08.195 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.195+0000 7f36cdc34640 1 -- 172.21.15.113:0/1866775922 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c12b0e9b-1482-4980-8e43-5174eab87fa5", "id": 4} v 0) v1 -- 0x7f36c8074290 con 0x7f36c80cd6d0 2023-11-09T03:45:08.198 INFO:tasks.ceph.mgr.y.smithi076.stderr:2023-11-09T03:45:08.198+0000 7fe560a32040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-09T03:45:08.200 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.201+0000 7f36affff640 1 -- 172.21.15.113:0/1866775922 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c12b0e9b-1482-4980-8e43-5174eab87fa5", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f36bc012b40 con 0x7f36c80cd6d0 2023-11-09T03:45:08.201 INFO:teuthology.orchestra.run.smithi113.stdout:4 2023-11-09T03:45:08.202 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.202+0000 7f36cdc34640 1 -- 172.21.15.113:0/1866775922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36c80cd6d0 msgr2=0x7f36c8153000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:08.202 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.202+0000 7f36cdc34640 1 --2- 172.21.15.113:0/1866775922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36c80cd6d0 0x7f36c8153000 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f36c80cdb10 tx=0x7f36bc00c820 comp rx=0 tx=0).stop 2023-11-09T03:45:08.202 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.202+0000 7f36cdc34640 1 -- 172.21.15.113:0/1866775922 shutdown_connections 2023-11-09T03:45:08.202 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.202+0000 7f36cdc34640 1 --2- 172.21.15.113:0/1866775922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36c8153e60 0x7f36c81f2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.203 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.202+0000 7f36cdc34640 1 --2- 172.21.15.113:0/1866775922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36c80cd6d0 0x7f36c8153000 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.203 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.202+0000 7f36cdc34640 1 --2- 172.21.15.113:0/1866775922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36c80b4880 0x7f36c81f6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.203 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.202+0000 7f36cdc34640 1 -- 172.21.15.113:0/1866775922 >> 172.21.15.113:0/1866775922 conn(0x7f36c80cf880 msgr2=0x7f36c815fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:08.203 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.202+0000 7f36cdc34640 1 -- 172.21.15.113:0/1866775922 shutdown_connections 2023-11-09T03:45:08.203 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.202+0000 7f36cdc34640 1 -- 172.21.15.113:0/1866775922 wait complete. 2023-11-09T03:45:08.212 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 57bdf820-32bd-4e3a-b9e3-d9669e0987ed 5 2023-11-09T03:45:08.240 INFO:tasks.ceph.mgr.y.smithi076.stderr:2023-11-09T03:45:08.240+0000 7fe560a32040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-09T03:45:08.241 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-11-09T03:45:08.241+0000 7f50fc0b6040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-09T03:45:08.312 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.311+0000 7f39ef9a9640 1 Processor -- start 2023-11-09T03:45:08.312 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.312+0000 7f39ef9a9640 1 -- start start 2023-11-09T03:45:08.316 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.312+0000 7f39ef9a9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f39e80b1490 0x7f39e80722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:08.316 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.312+0000 7f39ef9a9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39e80cdf80 con 0x7f39e806a830 2023-11-09T03:45:08.316 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.312+0000 7f39ef9a9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39e80b1900 con 0x7f39e81603d0 2023-11-09T03:45:08.316 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.312+0000 7f39ef9a9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39e80727e0 con 0x7f39e80b1490 2023-11-09T03:45:08.316 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.313+0000 7f39ed745640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f39e81603d0 0x7f39e806a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48860/0 (socket says 172.21.15.113:48860) 2023-11-09T03:45:08.317 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.313+0000 7f39ed745640 1 -- 172.21.15.113:0/3287252654 learned_addr learned my addr 172.21.15.113:0/3287252654 (peer_addr_for_me v1:172.21.15.113:0/0) 2023-11-09T03:45:08.317 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.313+0000 7f39edf46640 1 --2- 172.21.15.113:0/3287252654 >> v2:172.21.15.76:3300/0 conn(0x7f39e80b1490 0x7f39e80722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:08.317 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.313+0000 7f39dffff640 1 -- 172.21.15.113:0/3287252654 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3049109382 0 0) 0x7f39e80b1900 con 0x7f39e81603d0 2023-11-09T03:45:08.317 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.313+0000 7f39dffff640 1 -- 172.21.15.113:0/3287252654 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39d4003520 con 0x7f39e81603d0 2023-11-09T03:45:08.317 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.313+0000 7f39dffff640 1 -- 172.21.15.113:0/3287252654 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1054562903 0 0) 0x7f39e80cdf80 con 0x7f39e806a830 2023-11-09T03:45:08.318 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.313+0000 7f39dffff640 1 -- 172.21.15.113:0/3287252654 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39e80b1900 con 0x7f39e806a830 2023-11-09T03:45:08.318 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.313+0000 7f39dffff640 1 -- 172.21.15.113:0/3287252654 <== mon.1 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2195144629 0 0) 0x7f39d4003520 con 0x7f39e81603d0 2023-11-09T03:45:08.318 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.313+0000 7f39dffff640 1 -- 172.21.15.113:0/3287252654 >> v2:172.21.15.76:3300/0 conn(0x7f39e80b1490 msgr2=0x7f39e80722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:08.318 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.313+0000 7f39dffff640 1 --2- 172.21.15.113:0/3287252654 >> v2:172.21.15.76:3300/0 conn(0x7f39e80b1490 0x7f39e80722a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.318 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.313+0000 7f39dffff640 1 -- 172.21.15.113:0/3287252654 >> v1:172.21.15.76:6789/0 conn(0x7f39e806a830 legacy=0x7f39e80ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:08.319 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.313+0000 7f39dffff640 1 -- 172.21.15.113:0/3287252654 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e80ce6f0 con 0x7f39e81603d0 2023-11-09T03:45:08.319 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.313+0000 7f39edf46640 1 --2- 172.21.15.113:0/3287252654 >> v2:172.21.15.76:3300/0 conn(0x7f39e80b1490 0x7f39e80722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:45:08.319 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.313+0000 7f39dffff640 1 -- 172.21.15.113:0/3287252654 <== mon.1 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f39e4001670 con 0x7f39e81603d0 2023-11-09T03:45:08.319 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.313+0000 7f39dffff640 1 -- 172.21.15.113:0/3287252654 >> v1:172.21.15.113:6789/0 conn(0x7f39e81603d0 legacy=0x7f39e806a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:08.319 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.313+0000 7f39dffff640 1 --2- 172.21.15.113:0/3287252654 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39d4003b00 0x7f39d4003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:08.319 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.313+0000 7f39dffff640 1 --2- 172.21.15.113:0/3287252654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39d40047c0 0x7f39d4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:08.320 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.313+0000 7f39dffff640 1 --2- 172.21.15.113:0/3287252654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39d4005180 0x7f39d400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:08.320 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.313+0000 7f39dffff640 1 -- 172.21.15.113:0/3287252654 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d400c080 con 0x7f39d40047c0 2023-11-09T03:45:08.320 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.313+0000 7f39dffff640 1 -- 172.21.15.113:0/3287252654 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d400c1f0 con 0x7f39d4005180 2023-11-09T03:45:08.320 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.313+0000 7f39dffff640 1 -- 172.21.15.113:0/3287252654 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d400c3b0 con 0x7f39d4003b00 2023-11-09T03:45:08.320 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.313+0000 7f39ed745640 1 --2- 172.21.15.113:0/3287252654 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39d4003b00 0x7f39d4003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:08.321 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.313+0000 7f39edf46640 1 --2- 172.21.15.113:0/3287252654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39d4005180 0x7f39d400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:08.321 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.313+0000 7f39ecf44640 1 --2- 172.21.15.113:0/3287252654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39d40047c0 0x7f39d4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:08.321 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.314+0000 7f39ed745640 1 -- 172.21.15.113:0/3287252654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39d4005180 msgr2=0x7f39d400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:08.321 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.314+0000 7f39ed745640 1 --2- 172.21.15.113:0/3287252654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39d4005180 0x7f39d400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.321 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.314+0000 7f39ed745640 1 -- 172.21.15.113:0/3287252654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39d40047c0 msgr2=0x7f39d4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:08.321 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.314+0000 7f39ed745640 1 --2- 172.21.15.113:0/3287252654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39d40047c0 0x7f39d4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.322 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.314+0000 7f39ed745640 1 -- 172.21.15.113:0/3287252654 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d4003520 con 0x7f39d4003b00 2023-11-09T03:45:08.322 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.314+0000 7f39edf46640 1 --2- 172.21.15.113:0/3287252654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39d4005180 0x7f39d400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:45:08.322 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.314+0000 7f39edf46640 1 -- 172.21.15.113:0/3287252654 reap_dead start 2023-11-09T03:45:08.322 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.314+0000 7f39ed745640 1 --2- 172.21.15.113:0/3287252654 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39d4003b00 0x7f39d4003f00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f39e400c160 tx=0x7f39d40057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4cddef99c9ee745 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:08.322 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.314+0000 7f39dffff640 1 -- 172.21.15.113:0/3287252654 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e4005ad0 con 0x7f39d4003b00 2023-11-09T03:45:08.323 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.315+0000 7f39dffff640 1 -- 172.21.15.113:0/3287252654 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e4005c50 con 0x7f39d4003b00 2023-11-09T03:45:08.323 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.315+0000 7f39dffff640 1 -- 172.21.15.113:0/3287252654 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e402c850 con 0x7f39d4003b00 2023-11-09T03:45:08.323 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.315+0000 7f39ef9a9640 1 -- 172.21.15.113:0/3287252654 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39d4003b00 msgr2=0x7f39d4003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:08.323 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.315+0000 7f39ef9a9640 1 --2- 172.21.15.113:0/3287252654 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39d4003b00 0x7f39d4003f00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f39e400c160 tx=0x7f39d40057f0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.323 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.315+0000 7f39ef9a9640 1 -- 172.21.15.113:0/3287252654 shutdown_connections 2023-11-09T03:45:08.324 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.315+0000 7f39ef9a9640 1 --2- 172.21.15.113:0/3287252654 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39d4003b00 0x7f39d4003f00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.324 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.315+0000 7f39ef9a9640 1 -- 172.21.15.113:0/3287252654 >> 172.21.15.113:0/3287252654 conn(0x7f39e806d800 msgr2=0x7f39e806dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:08.324 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.315+0000 7f39ef9a9640 1 -- 172.21.15.113:0/3287252654 shutdown_connections 2023-11-09T03:45:08.324 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.315+0000 7f39ef9a9640 1 -- 172.21.15.113:0/3287252654 wait complete. 2023-11-09T03:45:08.326 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.316+0000 7f39ef9a9640 1 Processor -- start 2023-11-09T03:45:08.326 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.316+0000 7f39ef9a9640 1 -- start start 2023-11-09T03:45:08.327 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.317+0000 7f39ef9a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39d4003b00 0x7f39e815aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:08.327 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.317+0000 7f39ef9a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39e80b35c0 0x7f39e815ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:08.327 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.317+0000 7f39ef9a9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39e80b3a40 0x7f39e815f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:08.327 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.317+0000 7f39ef9a9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e80ac7e0 con 0x7f39e80b3a40 2023-11-09T03:45:08.327 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.317+0000 7f39ef9a9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e80b3fa0 con 0x7f39e80b35c0 2023-11-09T03:45:08.328 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.317+0000 7f39ef9a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e80b4280 con 0x7f39d4003b00 2023-11-09T03:45:08.328 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.317+0000 7f39ed745640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39d4003b00 0x7f39e815aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:08.328 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.317+0000 7f39ecf44640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39e80b35c0 0x7f39e815ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:08.328 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.317+0000 7f39ed745640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39d4003b00 0x7f39e815aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:59118/0 (socket says 172.21.15.113:59118) 2023-11-09T03:45:08.328 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.317+0000 7f39ed745640 1 -- 172.21.15.113:0/967418916 learned_addr learned my addr 172.21.15.113:0/967418916 (peer_addr_for_me v2:172.21.15.113:0/0) 2023-11-09T03:45:08.328 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.317+0000 7f39ecf44640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39e80b35c0 0x7f39e815ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52586/0 (socket says 172.21.15.113:52586) 2023-11-09T03:45:08.329 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.317+0000 7f39edf46640 1 --2- 172.21.15.113:0/967418916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39e80b3a40 0x7f39e815f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:08.329 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.317+0000 7f39ecf44640 1 -- 172.21.15.113:0/967418916 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39d4003b00 msgr2=0x7f39e815aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:08.329 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.317+0000 7f39ecf44640 1 --2- 172.21.15.113:0/967418916 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39d4003b00 0x7f39e815aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.329 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.318+0000 7f39ecf44640 1 -- 172.21.15.113:0/967418916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39e80b3a40 msgr2=0x7f39e815f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:08.329 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.318+0000 7f39ecf44640 1 --2- 172.21.15.113:0/967418916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39e80b3a40 0x7f39e815f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.329 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.318+0000 7f39ecf44640 1 -- 172.21.15.113:0/967418916 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e80b4560 con 0x7f39e80b35c0 2023-11-09T03:45:08.330 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.318+0000 7f39ed745640 1 --2- 172.21.15.113:0/967418916 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39d4003b00 0x7f39e815aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:45:08.330 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.318+0000 7f39ecf44640 1 --2- 172.21.15.113:0/967418916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39e80b35c0 0x7f39e815ef40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f39d800f7a0 tx=0x7f39d800fba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:08.330 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.318+0000 7f39ddffb640 1 -- 172.21.15.113:0/967418916 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d800fee0 con 0x7f39e80b35c0 2023-11-09T03:45:08.330 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.318+0000 7f39ddffb640 1 -- 172.21.15.113:0/967418916 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d8005110 con 0x7f39e80b35c0 2023-11-09T03:45:08.330 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.318+0000 7f39ddffb640 1 -- 172.21.15.113:0/967418916 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d8007050 con 0x7f39e80b35c0 2023-11-09T03:45:08.331 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.318+0000 7f39ef9a9640 1 -- 172.21.15.113:0/967418916 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e80b4840 con 0x7f39e80b35c0 2023-11-09T03:45:08.331 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.318+0000 7f39ddffb640 1 -- 172.21.15.113:0/967418916 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f39d80074b0 con 0x7f39e80b35c0 2023-11-09T03:45:08.331 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.318+0000 7f39ef9a9640 1 -- 172.21.15.113:0/967418916 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e815b430 con 0x7f39e80b35c0 2023-11-09T03:45:08.331 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.318+0000 7f39ddffb640 1 -- 172.21.15.113:0/967418916 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f39d800d7d0 con 0x7f39e80b35c0 2023-11-09T03:45:08.331 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.319+0000 7f39ef9a9640 1 -- 172.21.15.113:0/967418916 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39e80b5990 con 0x7f39e80b35c0 2023-11-09T03:45:08.331 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.321+0000 7f39ddffb640 1 -- 172.21.15.113:0/967418916 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f39d801a070 con 0x7f39e80b35c0 2023-11-09T03:45:08.340 INFO:tasks.ceph.mgr.y.smithi076.stderr:2023-11-09T03:45:08.340+0000 7fe560a32040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-09T03:45:08.391 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.392+0000 7f39ef9a9640 1 -- 172.21.15.113:0/967418916 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd new", "uuid": "57bdf820-32bd-4e3a-b9e3-d9669e0987ed", "id": 5} v 0) v1 -- 0x7f39e80d0880 con 0x7f39e80b35c0 2023-11-09T03:45:08.397 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.397+0000 7f39ddffb640 1 -- 172.21.15.113:0/967418916 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "57bdf820-32bd-4e3a-b9e3-d9669e0987ed", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f39d80068a0 con 0x7f39e80b35c0 2023-11-09T03:45:08.397 INFO:teuthology.orchestra.run.smithi113.stdout:5 2023-11-09T03:45:08.398 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.398+0000 7f39ef9a9640 1 -- 172.21.15.113:0/967418916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39e80b35c0 msgr2=0x7f39e815ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:08.398 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.398+0000 7f39ef9a9640 1 --2- 172.21.15.113:0/967418916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39e80b35c0 0x7f39e815ef40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f39d800f7a0 tx=0x7f39d800fba0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.398 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.398+0000 7f39ef9a9640 1 -- 172.21.15.113:0/967418916 shutdown_connections 2023-11-09T03:45:08.398 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.398+0000 7f39ef9a9640 1 --2- 172.21.15.113:0/967418916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39e80b3a40 0x7f39e815f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.399 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.398+0000 7f39ef9a9640 1 --2- 172.21.15.113:0/967418916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39e80b35c0 0x7f39e815ef40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.399 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.398+0000 7f39ef9a9640 1 --2- 172.21.15.113:0/967418916 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39d4003b00 0x7f39e815aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.399 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.398+0000 7f39ef9a9640 1 -- 172.21.15.113:0/967418916 >> 172.21.15.113:0/967418916 conn(0x7f39e806d800 msgr2=0x7f39e80c75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:08.399 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.398+0000 7f39ef9a9640 1 -- 172.21.15.113:0/967418916 shutdown_connections 2023-11-09T03:45:08.399 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.398+0000 7f39ef9a9640 1 -- 172.21.15.113:0/967418916 wait complete. 2023-11-09T03:45:08.408 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 4211c9a9-c4f8-4519-8c8c-18a603d61651 6 2023-11-09T03:45:08.410 INFO:tasks.ceph.mgr.y.smithi076.stderr:2023-11-09T03:45:08.410+0000 7fe560a32040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-09T03:45:08.455 INFO:tasks.ceph.mgr.y.smithi076.stderr:2023-11-09T03:45:08.455+0000 7fe560a32040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-09T03:45:08.506 INFO:tasks.ceph.mgr.y.smithi076.stderr:2023-11-09T03:45:08.506+0000 7fe560a32040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-09T03:45:08.548 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.548+0000 7f97b40b4640 1 Processor -- start 2023-11-09T03:45:08.548 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.548+0000 7f97b40b4640 1 -- start start 2023-11-09T03:45:08.550 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.549+0000 7f97b40b4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f97ac15af20 0x7f97ac0b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:08.550 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.549+0000 7f97b40b4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f97ac0b64c0 0x7f97ac0cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:08.550 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.549+0000 7f97b40b4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f97ac0cd3b0 0x7f97ac0cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:08.550 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.549+0000 7f97b40b4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ac0cdd80 con 0x7f97ac0b64c0 2023-11-09T03:45:08.551 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.549+0000 7f97b40b4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ac0d01f0 con 0x7f97ac0cd3b0 2023-11-09T03:45:08.551 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.549+0000 7f97b40b4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0b6a40 con 0x7f97ac15af20 2023-11-09T03:45:08.551 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.549+0000 7f97b1e50640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f97ac15af20 0x7f97ac0b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:08.551 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.549+0000 7f97b2651640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f97ac0cd3b0 0x7f97ac0cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:08.551 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.549+0000 7f97b1e50640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f97ac15af20 0x7f97ac0b3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:59122/0 (socket says 172.21.15.113:59122) 2023-11-09T03:45:08.551 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.549+0000 7f97b1e50640 1 -- 172.21.15.113:0/3075608023 learned_addr learned my addr 172.21.15.113:0/3075608023 (peer_addr_for_me v2:172.21.15.113:0/0) 2023-11-09T03:45:08.552 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.549+0000 7f97b164f640 1 --2- 172.21.15.113:0/3075608023 >> v2:172.21.15.76:3300/0 conn(0x7f97ac0b64c0 0x7f97ac0cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:08.552 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.549+0000 7f97b1e50640 1 -- 172.21.15.113:0/3075608023 >> v2:172.21.15.113:3300/0 conn(0x7f97ac0cd3b0 msgr2=0x7f97ac0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:08.552 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.549+0000 7f97b1e50640 1 --2- 172.21.15.113:0/3075608023 >> v2:172.21.15.113:3300/0 conn(0x7f97ac0cd3b0 0x7f97ac0cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.552 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.549+0000 7f97b1e50640 1 -- 172.21.15.113:0/3075608023 >> v2:172.21.15.76:3300/0 conn(0x7f97ac0b64c0 msgr2=0x7f97ac0cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:08.552 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.549+0000 7f97b1e50640 1 --2- 172.21.15.113:0/3075608023 >> v2:172.21.15.76:3300/0 conn(0x7f97ac0b64c0 0x7f97ac0cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.553 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.549+0000 7f97b1e50640 1 -- 172.21.15.113:0/3075608023 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac0d04b0 con 0x7f97ac15af20 2023-11-09T03:45:08.553 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.549+0000 7f97b164f640 1 --2- 172.21.15.113:0/3075608023 >> v2:172.21.15.76:3300/0 conn(0x7f97ac0b64c0 0x7f97ac0cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:08.553 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.549+0000 7f97b2651640 1 --2- 172.21.15.113:0/3075608023 >> v2:172.21.15.113:3300/0 conn(0x7f97ac0cd3b0 0x7f97ac0cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:45:08.553 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.550+0000 7f97b1e50640 1 --2- 172.21.15.113:0/3075608023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f97ac15af20 0x7f97ac0b3cd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f97ac0c7b70 tx=0x7f97a402b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7cf3bb78d7b31c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:08.553 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.550+0000 7f97b0e4e640 1 -- 172.21.15.113:0/3075608023 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a4031870 con 0x7f97ac15af20 2023-11-09T03:45:08.553 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.550+0000 7f97b0e4e640 1 -- 172.21.15.113:0/3075608023 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a40319d0 con 0x7f97ac15af20 2023-11-09T03:45:08.554 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.550+0000 7f97b0e4e640 1 -- 172.21.15.113:0/3075608023 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a4031d60 con 0x7f97ac15af20 2023-11-09T03:45:08.554 INFO:tasks.ceph.mgr.y.smithi076.stderr:2023-11-09T03:45:08.552+0000 7fe560a32040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-09T03:45:08.557 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.550+0000 7f97b40b4640 1 -- 172.21.15.113:0/3075608023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f97ac15af20 msgr2=0x7f97ac0b3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:08.557 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.550+0000 7f97b40b4640 1 --2- 172.21.15.113:0/3075608023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f97ac15af20 0x7f97ac0b3cd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f97ac0c7b70 tx=0x7f97a402b910 comp rx=0 tx=0).stop 2023-11-09T03:45:08.557 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.550+0000 7f97b40b4640 1 -- 172.21.15.113:0/3075608023 shutdown_connections 2023-11-09T03:45:08.558 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.550+0000 7f97b40b4640 1 --2- 172.21.15.113:0/3075608023 >> v2:172.21.15.113:3300/0 conn(0x7f97ac0cd3b0 0x7f97ac0cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.558 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.550+0000 7f97b40b4640 1 --2- 172.21.15.113:0/3075608023 >> v2:172.21.15.76:3300/0 conn(0x7f97ac0b64c0 0x7f97ac0cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.558 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.551+0000 7f97b40b4640 1 --2- 172.21.15.113:0/3075608023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f97ac15af20 0x7f97ac0b3cd0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.558 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.551+0000 7f97b40b4640 1 -- 172.21.15.113:0/3075608023 >> 172.21.15.113:0/3075608023 conn(0x7f97ac06ea20 msgr2=0x7f97ac06ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:08.558 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.551+0000 7f97b40b4640 1 -- 172.21.15.113:0/3075608023 shutdown_connections 2023-11-09T03:45:08.559 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.551+0000 7f97b40b4640 1 -- 172.21.15.113:0/3075608023 wait complete. 2023-11-09T03:45:08.559 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.552+0000 7f97b40b4640 1 Processor -- start 2023-11-09T03:45:08.559 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.552+0000 7f97b40b4640 1 -- start start 2023-11-09T03:45:08.559 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.552+0000 7f97b40b4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f97ac0b64c0 0x7f97ac1f2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:08.559 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.552+0000 7f97b40b4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97ac0cd3b0 0x7f97ac0c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:08.559 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.552+0000 7f97b40b4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97ac15af20 0x7f97ac0c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:08.560 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.552+0000 7f97b40b4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0c6870 con 0x7f97ac0cd3b0 2023-11-09T03:45:08.560 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.552+0000 7f97b40b4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0c6b50 con 0x7f97ac15af20 2023-11-09T03:45:08.560 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.552+0000 7f97b40b4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0c6e30 con 0x7f97ac0b64c0 2023-11-09T03:45:08.560 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.553+0000 7f97b1e50640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f97ac0b64c0 0x7f97ac1f2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:08.560 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.553+0000 7f97b1e50640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f97ac0b64c0 0x7f97ac1f2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:59128/0 (socket says 172.21.15.113:59128) 2023-11-09T03:45:08.560 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.553+0000 7f97b1e50640 1 -- 172.21.15.113:0/1403023064 learned_addr learned my addr 172.21.15.113:0/1403023064 (peer_addr_for_me v2:172.21.15.113:0/0) 2023-11-09T03:45:08.561 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.553+0000 7f97b2651640 1 --2- 172.21.15.113:0/1403023064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97ac15af20 0x7f97ac0c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:08.561 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.553+0000 7f97b164f640 1 --2- 172.21.15.113:0/1403023064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97ac0cd3b0 0x7f97ac0c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:08.561 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.553+0000 7f97b1e50640 1 -- 172.21.15.113:0/1403023064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97ac15af20 msgr2=0x7f97ac0c6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:08.561 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.553+0000 7f97b1e50640 1 --2- 172.21.15.113:0/1403023064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97ac15af20 0x7f97ac0c6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.561 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.553+0000 7f97b1e50640 1 -- 172.21.15.113:0/1403023064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97ac0cd3b0 msgr2=0x7f97ac0c5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:08.562 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.553+0000 7f97b1e50640 1 --2- 172.21.15.113:0/1403023064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97ac0cd3b0 0x7f97ac0c5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.562 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.553+0000 7f97b1e50640 1 -- 172.21.15.113:0/1403023064 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a8009a70 con 0x7f97ac0b64c0 2023-11-09T03:45:08.562 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.553+0000 7f97b1e50640 1 --2- 172.21.15.113:0/1403023064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f97ac0b64c0 0x7f97ac1f2520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f97a402b950 tx=0x7f97a40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:08.562 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.553+0000 7f97a27fc640 1 -- 172.21.15.113:0/1403023064 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a403ab60 con 0x7f97ac0b64c0 2023-11-09T03:45:08.562 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.553+0000 7f97a27fc640 1 -- 172.21.15.113:0/1403023064 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a403acc0 con 0x7f97ac0b64c0 2023-11-09T03:45:08.563 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.553+0000 7f97b40b4640 1 -- 172.21.15.113:0/1403023064 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a40094d0 con 0x7f97ac0b64c0 2023-11-09T03:45:08.563 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.553+0000 7f97a27fc640 1 -- 172.21.15.113:0/1403023064 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a4044630 con 0x7f97ac0b64c0 2023-11-09T03:45:08.563 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.553+0000 7f97b40b4640 1 -- 172.21.15.113:0/1403023064 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ac0c73f0 con 0x7f97ac0b64c0 2023-11-09T03:45:08.563 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.553+0000 7f97a27fc640 1 -- 172.21.15.113:0/1403023064 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f97a4044a90 con 0x7f97ac0b64c0 2023-11-09T03:45:08.563 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.553+0000 7f97a27fc640 1 -- 172.21.15.113:0/1403023064 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f97a402d040 con 0x7f97ac0b64c0 2023-11-09T03:45:08.564 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.554+0000 7f97b40b4640 1 -- 172.21.15.113:0/1403023064 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ac0cc190 con 0x7f97ac0b64c0 2023-11-09T03:45:08.564 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.556+0000 7f97a27fc640 1 -- 172.21.15.113:0/1403023064 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f97ac0cc190 con 0x7f97ac0b64c0 2023-11-09T03:45:08.628 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.628+0000 7f97b40b4640 1 -- 172.21.15.113:0/1403023064 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4211c9a9-c4f8-4519-8c8c-18a603d61651", "id": 6} v 0) v1 -- 0x7f97ac0674e0 con 0x7f97ac0b64c0 2023-11-09T03:45:08.632 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.633+0000 7f97a27fc640 1 -- 172.21.15.113:0/1403023064 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4211c9a9-c4f8-4519-8c8c-18a603d61651", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f97a4035020 con 0x7f97ac0b64c0 2023-11-09T03:45:08.633 INFO:teuthology.orchestra.run.smithi113.stdout:6 2023-11-09T03:45:08.634 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.634+0000 7f97b40b4640 1 -- 172.21.15.113:0/1403023064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f97ac0b64c0 msgr2=0x7f97ac1f2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:08.634 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.634+0000 7f97b40b4640 1 --2- 172.21.15.113:0/1403023064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f97ac0b64c0 0x7f97ac1f2520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f97a402b950 tx=0x7f97a40046c0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.634 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.634+0000 7f97b40b4640 1 -- 172.21.15.113:0/1403023064 shutdown_connections 2023-11-09T03:45:08.635 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.634+0000 7f97b40b4640 1 --2- 172.21.15.113:0/1403023064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97ac15af20 0x7f97ac0c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.635 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.634+0000 7f97b40b4640 1 --2- 172.21.15.113:0/1403023064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97ac0cd3b0 0x7f97ac0c5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.635 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.634+0000 7f97b40b4640 1 --2- 172.21.15.113:0/1403023064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f97ac0b64c0 0x7f97ac1f2520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.635 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.634+0000 7f97b40b4640 1 -- 172.21.15.113:0/1403023064 >> 172.21.15.113:0/1403023064 conn(0x7f97ac06ea20 msgr2=0x7f97ac158a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:08.635 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.634+0000 7f97b40b4640 1 -- 172.21.15.113:0/1403023064 shutdown_connections 2023-11-09T03:45:08.636 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.634+0000 7f97b40b4640 1 -- 172.21.15.113:0/1403023064 wait complete. 2023-11-09T03:45:08.638 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-11-09T03:45:08.638+0000 7f50fc0b6040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-09T03:45:08.644 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 40f5a75c-ead7-401e-a2fa-03152ce75255 7 2023-11-09T03:45:08.743 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.743+0000 7f2d77894640 1 Processor -- start 2023-11-09T03:45:08.744 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.743+0000 7f2d77894640 1 -- start start 2023-11-09T03:45:08.745 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.744+0000 7f2d77894640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2d7015af20 0x7f2d700b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:08.746 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.744+0000 7f2d77894640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d700b64c0 0x7f2d700cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:08.746 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.744+0000 7f2d77894640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d700d01f0 con 0x7f2d700cd3b0 2023-11-09T03:45:08.746 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.744+0000 7f2d77894640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d700b6a40 con 0x7f2d7015af20 2023-11-09T03:45:08.746 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.744+0000 7f2d77894640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d700d04b0 con 0x7f2d700b64c0 2023-11-09T03:45:08.746 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.744+0000 7f2d74e2f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d700b64c0 0x7f2d700cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:08.747 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.744+0000 7f2d74e2f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d700b64c0 0x7f2d700cbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:59144/0 (socket says 172.21.15.113:59144) 2023-11-09T03:45:08.747 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.744+0000 7f2d74e2f640 1 -- 172.21.15.113:0/3247716975 learned_addr learned my addr 172.21.15.113:0/3247716975 (peer_addr_for_me v2:172.21.15.113:0/0) 2023-11-09T03:45:08.747 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.744+0000 7f2d75630640 1 --2- 172.21.15.113:0/3247716975 >> v2:172.21.15.76:3300/0 conn(0x7f2d7015af20 0x7f2d700b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:08.747 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.745+0000 7f2d74e2f640 1 -- 172.21.15.113:0/3247716975 >> v2:172.21.15.76:3300/0 conn(0x7f2d7015af20 msgr2=0x7f2d700b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:08.747 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.745+0000 7f2d74e2f640 1 --2- 172.21.15.113:0/3247716975 >> v2:172.21.15.76:3300/0 conn(0x7f2d7015af20 0x7f2d700b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.748 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.745+0000 7f2d74e2f640 1 -- 172.21.15.113:0/3247716975 >> v1:172.21.15.76:6789/0 conn(0x7f2d700cd3b0 legacy=0x7f2d700b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:08.748 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.745+0000 7f2d74e2f640 1 -- 172.21.15.113:0/3247716975 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d700d0790 con 0x7f2d700b64c0 2023-11-09T03:45:08.748 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.745+0000 7f2d74e2f640 1 --2- 172.21.15.113:0/3247716975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d700b64c0 0x7f2d700cbc70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009870 tx=0x7f2d6c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42812befd2d38c8b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:08.748 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.745+0000 7f2d63fff640 1 -- 172.21.15.113:0/3247716975 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c030840 con 0x7f2d700b64c0 2023-11-09T03:45:08.748 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.745+0000 7f2d63fff640 1 -- 172.21.15.113:0/3247716975 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6c0309a0 con 0x7f2d700b64c0 2023-11-09T03:45:08.748 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.745+0000 7f2d63fff640 1 -- 172.21.15.113:0/3247716975 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c030d30 con 0x7f2d700b64c0 2023-11-09T03:45:08.751 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.746+0000 7f2d77894640 1 -- 172.21.15.113:0/3247716975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d700b64c0 msgr2=0x7f2d700cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:08.751 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.746+0000 7f2d77894640 1 --2- 172.21.15.113:0/3247716975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d700b64c0 0x7f2d700cbc70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009870 tx=0x7f2d6c02b980 comp rx=0 tx=0).stop 2023-11-09T03:45:08.751 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.746+0000 7f2d77894640 1 -- 172.21.15.113:0/3247716975 shutdown_connections 2023-11-09T03:45:08.752 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.746+0000 7f2d77894640 1 --2- 172.21.15.113:0/3247716975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d700b64c0 0x7f2d700cbc70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.752 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.746+0000 7f2d77894640 1 --2- 172.21.15.113:0/3247716975 >> v2:172.21.15.76:3300/0 conn(0x7f2d7015af20 0x7f2d700b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.752 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.746+0000 7f2d77894640 1 -- 172.21.15.113:0/3247716975 >> 172.21.15.113:0/3247716975 conn(0x7f2d7006ea20 msgr2=0x7f2d7006ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:08.752 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.746+0000 7f2d77894640 1 -- 172.21.15.113:0/3247716975 shutdown_connections 2023-11-09T03:45:08.752 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.746+0000 7f2d77894640 1 -- 172.21.15.113:0/3247716975 wait complete. 2023-11-09T03:45:08.752 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.747+0000 7f2d77894640 1 Processor -- start 2023-11-09T03:45:08.753 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.747+0000 7f2d77894640 1 -- start start 2023-11-09T03:45:08.753 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.748+0000 7f2d77894640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d700b64c0 0x7f2d7015ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:08.753 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.748+0000 7f2d77894640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d700cd3b0 0x7f2d70146f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:08.753 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.748+0000 7f2d77894640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d7015af20 0x7f2d700b0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:08.753 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.748+0000 7f2d77894640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d700b2bf0 con 0x7f2d7015af20 2023-11-09T03:45:08.754 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.748+0000 7f2d77894640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d70147470 con 0x7f2d700cd3b0 2023-11-09T03:45:08.754 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.748+0000 7f2d77894640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d700b3420 con 0x7f2d700b64c0 2023-11-09T03:45:08.754 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.748+0000 7f2d75630640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d700b64c0 0x7f2d7015ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:08.754 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.748+0000 7f2d74e2f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d700cd3b0 0x7f2d70146f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:08.754 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.748+0000 7f2d74e2f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d700cd3b0 0x7f2d70146f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52618/0 (socket says 172.21.15.113:52618) 2023-11-09T03:45:08.755 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.748+0000 7f2d74e2f640 1 -- 172.21.15.113:0/2680727666 learned_addr learned my addr 172.21.15.113:0/2680727666 (peer_addr_for_me v2:172.21.15.113:0/0) 2023-11-09T03:45:08.755 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.748+0000 7f2d75e31640 1 --2- 172.21.15.113:0/2680727666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d7015af20 0x7f2d700b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:08.755 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.748+0000 7f2d74e2f640 1 -- 172.21.15.113:0/2680727666 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d700b64c0 msgr2=0x7f2d7015ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:08.755 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.748+0000 7f2d74e2f640 1 --2- 172.21.15.113:0/2680727666 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d700b64c0 0x7f2d7015ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.755 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.748+0000 7f2d74e2f640 1 -- 172.21.15.113:0/2680727666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d7015af20 msgr2=0x7f2d700b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:08.756 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.748+0000 7f2d74e2f640 1 --2- 172.21.15.113:0/2680727666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d7015af20 0x7f2d700b0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.756 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.749+0000 7f2d74e2f640 1 -- 172.21.15.113:0/2680727666 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c0094d0 con 0x7f2d700cd3b0 2023-11-09T03:45:08.756 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.749+0000 7f2d74e2f640 1 --2- 172.21.15.113:0/2680727666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d700cd3b0 0x7f2d70146f30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c000c00 tx=0x7f2d6c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:08.756 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.749+0000 7f2d61ffb640 1 -- 172.21.15.113:0/2680727666 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c039c90 con 0x7f2d700cd3b0 2023-11-09T03:45:08.756 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.749+0000 7f2d61ffb640 1 -- 172.21.15.113:0/2680727666 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6c039df0 con 0x7f2d700cd3b0 2023-11-09T03:45:08.757 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.749+0000 7f2d77894640 1 -- 172.21.15.113:0/2680727666 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d700b2e60 con 0x7f2d700cd3b0 2023-11-09T03:45:08.757 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.749+0000 7f2d61ffb640 1 -- 172.21.15.113:0/2680727666 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c043640 con 0x7f2d700cd3b0 2023-11-09T03:45:08.757 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.749+0000 7f2d77894640 1 -- 172.21.15.113:0/2680727666 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7015b570 con 0x7f2d700cd3b0 2023-11-09T03:45:08.757 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.749+0000 7f2d61ffb640 1 -- 172.21.15.113:0/2680727666 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2d6c043aa0 con 0x7f2d700cd3b0 2023-11-09T03:45:08.757 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.749+0000 7f2d61ffb640 1 -- 172.21.15.113:0/2680727666 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f2d6c03e070 con 0x7f2d700cd3b0 2023-11-09T03:45:08.757 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.750+0000 7f2d77894640 1 -- 172.21.15.113:0/2680727666 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d7015b790 con 0x7f2d700cd3b0 2023-11-09T03:45:08.758 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.754+0000 7f2d61ffb640 1 -- 172.21.15.113:0/2680727666 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f2d6c034070 con 0x7f2d700cd3b0 2023-11-09T03:45:08.781 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-11-09T03:45:08.781+0000 7f50fc0b6040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-09T03:45:08.825 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.825+0000 7f2d77894640 1 -- 172.21.15.113:0/2680727666 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd new", "uuid": "40f5a75c-ead7-401e-a2fa-03152ce75255", "id": 7} v 0) v1 -- 0x7f2d700caf10 con 0x7f2d700cd3b0 2023-11-09T03:45:08.830 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.831+0000 7f2d61ffb640 1 -- 172.21.15.113:0/2680727666 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "40f5a75c-ead7-401e-a2fa-03152ce75255", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2d6c04d460 con 0x7f2d700cd3b0 2023-11-09T03:45:08.831 INFO:teuthology.orchestra.run.smithi113.stdout:7 2023-11-09T03:45:08.832 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.832+0000 7f2d77894640 1 -- 172.21.15.113:0/2680727666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d700cd3b0 msgr2=0x7f2d70146f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:08.832 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.832+0000 7f2d77894640 1 --2- 172.21.15.113:0/2680727666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d700cd3b0 0x7f2d70146f30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c000c00 tx=0x7f2d6c004730 comp rx=0 tx=0).stop 2023-11-09T03:45:08.832 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.832+0000 7f2d77894640 1 -- 172.21.15.113:0/2680727666 shutdown_connections 2023-11-09T03:45:08.832 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.832+0000 7f2d77894640 1 --2- 172.21.15.113:0/2680727666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d7015af20 0x7f2d700b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.833 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.832+0000 7f2d77894640 1 --2- 172.21.15.113:0/2680727666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d700cd3b0 0x7f2d70146f30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.833 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.832+0000 7f2d77894640 1 --2- 172.21.15.113:0/2680727666 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d700b64c0 0x7f2d7015ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:08.833 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.832+0000 7f2d77894640 1 -- 172.21.15.113:0/2680727666 >> 172.21.15.113:0/2680727666 conn(0x7f2d7006ea20 msgr2=0x7f2d70159d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:08.833 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.832+0000 7f2d77894640 1 -- 172.21.15.113:0/2680727666 shutdown_connections 2023-11-09T03:45:08.833 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:45:08.832+0000 7f2d77894640 1 -- 172.21.15.113:0/2680727666 wait complete. 2023-11-09T03:45:08.842 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-09T03:45:08.842 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-09T03:45:08.844 INFO:tasks.ceph.osd.0:Started 2023-11-09T03:45:08.845 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-09T03:45:08.845 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-09T03:45:08.847 INFO:tasks.ceph.osd.1:Started 2023-11-09T03:45:08.848 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-09T03:45:08.848 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-09T03:45:08.850 INFO:tasks.ceph.osd.2:Started 2023-11-09T03:45:08.850 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-09T03:45:08.851 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-09T03:45:08.853 INFO:tasks.ceph.osd.3:Started 2023-11-09T03:45:08.853 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-09T03:45:08.854 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-09T03:45:08.856 INFO:tasks.ceph.osd.4:Started 2023-11-09T03:45:08.856 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-09T03:45:08.857 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-09T03:45:08.859 INFO:tasks.ceph.osd.5:Started 2023-11-09T03:45:08.859 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-09T03:45:08.860 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-09T03:45:08.862 INFO:tasks.ceph.osd.6:Started 2023-11-09T03:45:08.862 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-09T03:45:08.862 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-09T03:45:08.865 INFO:tasks.ceph.osd.7:Started 2023-11-09T03:45:08.865 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:45:08.940 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-11-09T03:45:08.940+0000 7f50fc0b6040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-09T03:45:08.958 INFO:tasks.ceph.mgr.y.smithi076.stderr:2023-11-09T03:45:08.958+0000 7fe560a32040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-09T03:45:08.989 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-11-09T03:45:08.989+0000 7f50fc0b6040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-09T03:45:08.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.995+0000 7fade23d7640 1 Processor -- start 2023-11-09T03:45:08.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.995+0000 7fade23d7640 1 -- start start 2023-11-09T03:45:08.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.995+0000 7fade23d7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faddc0cde60 0x7faddc15d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:08.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.995+0000 7fade23d7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faddc15daa0 0x7faddc157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:08.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.995+0000 7fade23d7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faddc15e000 con 0x7faddc0ce240 2023-11-09T03:45:08.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.995+0000 7fade23d7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faddc15e5c0 con 0x7faddc0cde60 2023-11-09T03:45:08.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.995+0000 7fade23d7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faddc15e2e0 con 0x7faddc15daa0 2023-11-09T03:45:08.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.996+0000 7faddbfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faddc0cde60 0x7faddc15d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:08.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.996+0000 7fade0974640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7faddc0ce240 0x7faddc06b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37950/0 (socket says 172.21.15.76:37950) 2023-11-09T03:45:08.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.996+0000 7fade0974640 1 -- 172.21.15.76:0/302110360 learned_addr learned my addr 172.21.15.76:0/302110360 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:08.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.996+0000 7faddb7fe640 1 --2- 172.21.15.76:0/302110360 >> v2:172.21.15.113:3300/0 conn(0x7faddc15daa0 0x7faddc157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:09.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.996+0000 7faddbfff640 1 -- 172.21.15.76:0/302110360 >> v2:172.21.15.113:3300/0 conn(0x7faddc15daa0 msgr2=0x7faddc157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:09.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.996+0000 7faddbfff640 1 --2- 172.21.15.76:0/302110360 >> v2:172.21.15.113:3300/0 conn(0x7faddc15daa0 0x7faddc157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:09.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.996+0000 7faddbfff640 1 -- 172.21.15.76:0/302110360 >> v1:172.21.15.76:6789/0 conn(0x7faddc0ce240 legacy=0x7faddc06b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:09.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.996+0000 7faddbfff640 1 -- 172.21.15.76:0/302110360 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faddc0d1120 con 0x7faddc0cde60 2023-11-09T03:45:09.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.996+0000 7faddaffd640 1 -- 172.21.15.76:0/302110360 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2746598546 0 0) 0x7faddc15e000 con 0x7faddc0ce240 2023-11-09T03:45:09.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.997+0000 7faddbfff640 1 --2- 172.21.15.76:0/302110360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faddc0cde60 0x7faddc15d560 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faddc0cf5f0 tx=0x7fadcc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f2b38160049614 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:09.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.997+0000 7faddaffd640 1 -- 172.21.15.76:0/302110360 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc031830 con 0x7faddc0cde60 2023-11-09T03:45:09.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.997+0000 7faddaffd640 1 -- 172.21.15.76:0/302110360 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadcc031990 con 0x7faddc0cde60 2023-11-09T03:45:09.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.997+0000 7faddaffd640 1 -- 172.21.15.76:0/302110360 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc031d20 con 0x7faddc0cde60 2023-11-09T03:45:09.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.997+0000 7fade23d7640 1 -- 172.21.15.76:0/302110360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faddc0cde60 msgr2=0x7faddc15d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:09.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.997+0000 7fade23d7640 1 --2- 172.21.15.76:0/302110360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faddc0cde60 0x7faddc15d560 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faddc0cf5f0 tx=0x7fadcc02b8d0 comp rx=0 tx=0).stop 2023-11-09T03:45:09.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.997+0000 7fade23d7640 1 -- 172.21.15.76:0/302110360 shutdown_connections 2023-11-09T03:45:09.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.997+0000 7fade23d7640 1 --2- 172.21.15.76:0/302110360 >> v2:172.21.15.113:3300/0 conn(0x7faddc15daa0 0x7faddc157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:09.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.997+0000 7fade23d7640 1 --2- 172.21.15.76:0/302110360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faddc0cde60 0x7faddc15d560 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:09.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.997+0000 7fade23d7640 1 -- 172.21.15.76:0/302110360 >> 172.21.15.76:0/302110360 conn(0x7faddc071f30 msgr2=0x7faddc0cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:09.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.998+0000 7fade23d7640 1 -- 172.21.15.76:0/302110360 shutdown_connections 2023-11-09T03:45:09.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.998+0000 7fade23d7640 1 -- 172.21.15.76:0/302110360 wait complete. 2023-11-09T03:45:09.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.999+0000 7fade23d7640 1 Processor -- start 2023-11-09T03:45:09.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.999+0000 7fade23d7640 1 -- start start 2023-11-09T03:45:09.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.999+0000 7fade23d7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faddc0cde60 0x7faddc156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:09.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:08.999+0000 7fade23d7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faddc0ce240 0x7faddc15c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:09.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.000+0000 7fade23d7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faddc15daa0 0x7faddc15ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:09.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.000+0000 7fade23d7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faddc1f2900 con 0x7faddc15daa0 2023-11-09T03:45:09.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.000+0000 7fade23d7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faddc15cf20 con 0x7faddc0ce240 2023-11-09T03:45:09.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.000+0000 7fade23d7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faddc157d70 con 0x7faddc0cde60 2023-11-09T03:45:09.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.000+0000 7fade0974640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faddc15daa0 0x7faddc15ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:09.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.000+0000 7faddbfff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faddc0cde60 0x7faddc156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:09.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.000+0000 7faddbfff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faddc0cde60 0x7faddc156dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:52802/0 (socket says 172.21.15.76:52802) 2023-11-09T03:45:09.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.000+0000 7faddbfff640 1 -- 172.21.15.76:0/2995840278 learned_addr learned my addr 172.21.15.76:0/2995840278 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:09.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.000+0000 7faddb7fe640 1 --2- 172.21.15.76:0/2995840278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faddc0ce240 0x7faddc15c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:09.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.000+0000 7faddbfff640 1 -- 172.21.15.76:0/2995840278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faddc0ce240 msgr2=0x7faddc15c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:09.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.000+0000 7faddbfff640 1 --2- 172.21.15.76:0/2995840278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faddc0ce240 0x7faddc15c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:09.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.000+0000 7faddbfff640 1 -- 172.21.15.76:0/2995840278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faddc15daa0 msgr2=0x7faddc15ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:09.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.000+0000 7faddbfff640 1 --2- 172.21.15.76:0/2995840278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faddc15daa0 0x7faddc15ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:09.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.000+0000 7faddbfff640 1 -- 172.21.15.76:0/2995840278 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadcc009400 con 0x7faddc0cde60 2023-11-09T03:45:09.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.000+0000 7faddbfff640 1 --2- 172.21.15.76:0/2995840278 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faddc0cde60 0x7faddc156dd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fadcc02bee0 tx=0x7fadcc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:09.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.000+0000 7fadd8ff9640 1 -- 172.21.15.76:0/2995840278 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc03a630 con 0x7faddc0cde60 2023-11-09T03:45:09.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.000+0000 7fadd8ff9640 1 -- 172.21.15.76:0/2995840278 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadcc03a7d0 con 0x7faddc0cde60 2023-11-09T03:45:09.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.000+0000 7fade23d7640 1 -- 172.21.15.76:0/2995840278 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faddc1574d0 con 0x7faddc0cde60 2023-11-09T03:45:09.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.000+0000 7fadd8ff9640 1 -- 172.21.15.76:0/2995840278 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc03ab80 con 0x7faddc0cde60 2023-11-09T03:45:09.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.000+0000 7fade23d7640 1 -- 172.21.15.76:0/2995840278 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faddc157a90 con 0x7faddc0cde60 2023-11-09T03:45:09.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.001+0000 7fadd8ff9640 1 -- 172.21.15.76:0/2995840278 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fadcc04d440 con 0x7faddc0cde60 2023-11-09T03:45:09.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.001+0000 7fadd8ff9640 1 -- 172.21.15.76:0/2995840278 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fadcc043bd0 con 0x7faddc0cde60 2023-11-09T03:45:09.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.001+0000 7fade23d7640 1 -- 172.21.15.76:0/2995840278 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faddc074290 con 0x7faddc0cde60 2023-11-09T03:45:09.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.003+0000 7fadd8ff9640 1 -- 172.21.15.76:0/2995840278 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fadcc052060 con 0x7faddc0cde60 2023-11-09T03:45:09.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.063+0000 7fade23d7640 1 -- 172.21.15.76:0/2995840278 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faddc0b5720 con 0x7faddc0cde60 2023-11-09T03:45:09.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.064+0000 7fadd8ff9640 1 -- 172.21.15.76:0/2995840278 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fadcc048070 con 0x7faddc0cde60 2023-11-09T03:45:09.065 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:09.065 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":10,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:45:08.827625+0000","last_up_change":"0.000000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:45:09.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.065+0000 7fade23d7640 1 -- 172.21.15.76:0/2995840278 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faddc0cde60 msgr2=0x7faddc156dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:09.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.065+0000 7fade23d7640 1 --2- 172.21.15.76:0/2995840278 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faddc0cde60 0x7faddc156dd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fadcc02bee0 tx=0x7fadcc0046c0 comp rx=0 tx=0).stop 2023-11-09T03:45:09.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.065+0000 7fade23d7640 1 -- 172.21.15.76:0/2995840278 shutdown_connections 2023-11-09T03:45:09.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.065+0000 7fade23d7640 1 --2- 172.21.15.76:0/2995840278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faddc15daa0 0x7faddc15ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:09.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.065+0000 7fade23d7640 1 --2- 172.21.15.76:0/2995840278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faddc0ce240 0x7faddc15c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:09.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.065+0000 7fade23d7640 1 --2- 172.21.15.76:0/2995840278 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faddc0cde60 0x7faddc156dd0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:09.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.065+0000 7fade23d7640 1 -- 172.21.15.76:0/2995840278 >> 172.21.15.76:0/2995840278 conn(0x7faddc071f30 msgr2=0x7faddc0d2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:09.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.065+0000 7fade23d7640 1 -- 172.21.15.76:0/2995840278 shutdown_connections 2023-11-09T03:45:09.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:09.065+0000 7fade23d7640 1 -- 172.21.15.76:0/2995840278 wait complete. 2023-11-09T03:45:09.075 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-09T03:45:09.076 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-09T03:45:09.086 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-11-09T03:45:09.086+0000 7f50fc0b6040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-09T03:45:09.098 INFO:tasks.ceph.mgr.y.smithi076.stderr:2023-11-09T03:45:09.098+0000 7fe560a32040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-09T03:45:09.181 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-11-09T03:45:09.181+0000 7f50fc0b6040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-09T03:45:09.213 INFO:tasks.ceph.mgr.y.smithi076.stderr:2023-11-09T03:45:09.213+0000 7fe560a32040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-09T03:45:09.259 INFO:tasks.ceph.mgr.y.smithi076.stderr:2023-11-09T03:45:09.259+0000 7fe560a32040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-09T03:45:09.320 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-11-09T03:45:09.320+0000 7f50fc0b6040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-09T03:45:09.354 INFO:tasks.ceph.mgr.y.smithi076.stderr:2023-11-09T03:45:09.354+0000 7fe560a32040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-09T03:45:09.361 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-11-09T03:45:09.361+0000 7f50fc0b6040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-09T03:45:09.448 INFO:tasks.ceph.mgr.y.smithi076.stderr:2023-11-09T03:45:09.448+0000 7fe560a32040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-09T03:45:09.454 INFO:tasks.ceph.osd.5.smithi113.stderr:2023-11-09T03:45:09.454+0000 7f039cc0d500 -1 Falling back to public interface 2023-11-09T03:45:09.458 INFO:tasks.ceph.osd.6.smithi113.stderr:2023-11-09T03:45:09.458+0000 7f2847412500 -1 Falling back to public interface 2023-11-09T03:45:09.463 INFO:tasks.ceph.osd.4.smithi113.stderr:2023-11-09T03:45:09.463+0000 7f8dad626500 -1 Falling back to public interface 2023-11-09T03:45:09.466 INFO:tasks.ceph.osd.0.smithi076.stderr:2023-11-09T03:45:09.466+0000 7f12c6212500 -1 Falling back to public interface 2023-11-09T03:45:09.467 INFO:tasks.ceph.osd.3.smithi076.stderr:2023-11-09T03:45:09.466+0000 7fd71d924500 -1 Falling back to public interface 2023-11-09T03:45:09.467 INFO:tasks.ceph.osd.0.smithi076.stderr:2023-11-09T03:45:09.466+0000 7f12c4c17640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6801/0: (98) Address already in use 2023-11-09T03:45:09.467 INFO:tasks.ceph.osd.0.smithi076.stderr:2023-11-09T03:45:09.467+0000 7f12c4c17640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6805/0: (98) Address already in use 2023-11-09T03:45:09.468 INFO:tasks.ceph.osd.0.smithi076.stderr:2023-11-09T03:45:09.467+0000 7f12c4c17640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6808/0: (98) Address already in use 2023-11-09T03:45:09.469 INFO:tasks.ceph.osd.1.smithi076.stderr:2023-11-09T03:45:09.469+0000 7fcb2c51a500 -1 Falling back to public interface 2023-11-09T03:45:09.472 INFO:tasks.ceph.osd.2.smithi076.stderr:2023-11-09T03:45:09.471+0000 7f8f0bccf500 -1 Falling back to public interface 2023-11-09T03:45:09.480 INFO:tasks.ceph.osd.7.smithi113.stderr:2023-11-09T03:45:09.480+0000 7f8a10b1a500 -1 Falling back to public interface 2023-11-09T03:45:09.505 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-11-09T03:45:09.504+0000 7f50fc0b6040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-09T03:45:09.589 INFO:tasks.ceph.mgr.y.smithi076.stderr:2023-11-09T03:45:09.589+0000 7fe560a32040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-09T03:45:09.631 INFO:tasks.ceph.mgr.y.smithi076.stderr:2023-11-09T03:45:09.631+0000 7fe560a32040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-09T03:45:09.770 INFO:tasks.ceph.mgr.y.smithi076.stderr:2023-11-09T03:45:09.770+0000 7fe560a32040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-09T03:45:09.882 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-09T03:45:09.973 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.972+0000 7f3361875640 1 Processor -- start 2023-11-09T03:45:09.973 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.973+0000 7f3361875640 1 -- start start 2023-11-09T03:45:09.976 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.973+0000 7f3361875640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f335c15e160 0x7f335c15e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:09.976 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.973+0000 7f3361875640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f335c0b64f0 0x7f335c0cb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:09.976 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.973+0000 7f3361875640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f335c15e540 con 0x7f335c0cbc90 2023-11-09T03:45:09.976 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.973+0000 7f3361875640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f335c15eda0 con 0x7f335c0b64f0 2023-11-09T03:45:09.977 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.973+0000 7f3361875640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f335c15f080 con 0x7f335c15e160 2023-11-09T03:45:09.977 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.973+0000 7f335b7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f335c0cbc90 0x7f335c070340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37958/0 (socket says 172.21.15.76:37958) 2023-11-09T03:45:09.977 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.973+0000 7f335b7fe640 1 -- 172.21.15.76:0/2423465165 learned_addr learned my addr 172.21.15.76:0/2423465165 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:09.977 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.974+0000 7f335affd640 1 --2- 172.21.15.76:0/2423465165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f335c15e160 0x7f335c15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:09.977 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.974+0000 7f3353fff640 1 --2- 172.21.15.76:0/2423465165 >> v2:172.21.15.113:3300/0 conn(0x7f335c0b64f0 0x7f335c0cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:09.978 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.974+0000 7f335a7fc640 1 -- 172.21.15.76:0/2423465165 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 510053674 0 0) 0x7f3354001490 con 0x7f335c0cbc90 2023-11-09T03:45:09.978 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.974+0000 7f335a7fc640 1 -- 172.21.15.76:0/2423465165 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3348003520 con 0x7f335c0cbc90 2023-11-09T03:45:09.978 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.974+0000 7f335affd640 1 -- 172.21.15.76:0/2423465165 >> v2:172.21.15.113:3300/0 conn(0x7f335c0b64f0 msgr2=0x7f335c0cb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:09.978 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.974+0000 7f335affd640 1 --2- 172.21.15.76:0/2423465165 >> v2:172.21.15.113:3300/0 conn(0x7f335c0b64f0 0x7f335c0cb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:09.979 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.974+0000 7f335affd640 1 -- 172.21.15.76:0/2423465165 >> v1:172.21.15.76:6789/0 conn(0x7f335c0cbc90 legacy=0x7f335c070340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:09.979 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.974+0000 7f335affd640 1 -- 172.21.15.76:0/2423465165 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335c15f360 con 0x7f335c15e160 2023-11-09T03:45:09.979 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.974+0000 7f3353fff640 1 --2- 172.21.15.76:0/2423465165 >> v2:172.21.15.113:3300/0 conn(0x7f335c0b64f0 0x7f335c0cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:45:09.979 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.975+0000 7f335affd640 1 --2- 172.21.15.76:0/2423465165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f335c15e160 0x7f335c15e750 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f334c0097d0 tx=0x7f334c02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a73320717e8b6a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:09.979 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.975+0000 7f335a7fc640 1 -- 172.21.15.76:0/2423465165 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c031800 con 0x7f335c15e160 2023-11-09T03:45:09.979 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.975+0000 7f335a7fc640 1 -- 172.21.15.76:0/2423465165 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f334c031960 con 0x7f335c15e160 2023-11-09T03:45:09.980 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.975+0000 7f335a7fc640 1 -- 172.21.15.76:0/2423465165 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c031cf0 con 0x7f335c15e160 2023-11-09T03:45:09.980 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.975+0000 7f3361875640 1 -- 172.21.15.76:0/2423465165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f335c15e160 msgr2=0x7f335c15e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:09.980 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.975+0000 7f3361875640 1 --2- 172.21.15.76:0/2423465165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f335c15e160 0x7f335c15e750 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f334c0097d0 tx=0x7f334c02b8a0 comp rx=0 tx=0).stop 2023-11-09T03:45:09.980 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.976+0000 7f3361875640 1 -- 172.21.15.76:0/2423465165 shutdown_connections 2023-11-09T03:45:09.980 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.976+0000 7f3361875640 1 --2- 172.21.15.76:0/2423465165 >> v2:172.21.15.113:3300/0 conn(0x7f335c0b64f0 0x7f335c0cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:09.981 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.976+0000 7f3361875640 1 --2- 172.21.15.76:0/2423465165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f335c15e160 0x7f335c15e750 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:09.981 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.976+0000 7f3361875640 1 -- 172.21.15.76:0/2423465165 >> 172.21.15.76:0/2423465165 conn(0x7f335c0cd500 msgr2=0x7f335c0cd910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:09.981 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.976+0000 7f3361875640 1 -- 172.21.15.76:0/2423465165 shutdown_connections 2023-11-09T03:45:09.981 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.976+0000 7f3361875640 1 -- 172.21.15.76:0/2423465165 wait complete. 2023-11-09T03:45:09.984 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.977+0000 7f3361875640 1 Processor -- start 2023-11-09T03:45:09.984 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.977+0000 7f3361875640 1 -- start start 2023-11-09T03:45:09.984 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.977+0000 7f3361875640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f335c0b64f0 0x7f335c0c9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:09.984 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.977+0000 7f3361875640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f335c0cbc90 0x7f335c144530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:09.985 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.977+0000 7f3361875640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f335c15e160 0x7f335c1423c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:09.985 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.977+0000 7f3361875640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335c0c9540 con 0x7f335c15e160 2023-11-09T03:45:09.985 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.977+0000 7f3361875640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335c141040 con 0x7f335c0cbc90 2023-11-09T03:45:09.985 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.977+0000 7f3361875640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f335c141320 con 0x7f335c0b64f0 2023-11-09T03:45:09.986 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.978+0000 7f335affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f335c0b64f0 0x7f335c0c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:09.986 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.978+0000 7f335b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f335c15e160 0x7f335c1423c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:09.986 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.978+0000 7f335affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f335c0b64f0 0x7f335c0c9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:52812/0 (socket says 172.21.15.76:52812) 2023-11-09T03:45:09.986 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.978+0000 7f335affd640 1 -- 172.21.15.76:0/3208736848 learned_addr learned my addr 172.21.15.76:0/3208736848 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:09.986 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.978+0000 7f3353fff640 1 --2- 172.21.15.76:0/3208736848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f335c0cbc90 0x7f335c144530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:09.987 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.978+0000 7f335affd640 1 -- 172.21.15.76:0/3208736848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f335c0cbc90 msgr2=0x7f335c144530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:09.987 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.978+0000 7f335affd640 1 --2- 172.21.15.76:0/3208736848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f335c0cbc90 0x7f335c144530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:09.987 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.978+0000 7f335affd640 1 -- 172.21.15.76:0/3208736848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f335c15e160 msgr2=0x7f335c1423c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:09.987 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.978+0000 7f335affd640 1 --2- 172.21.15.76:0/3208736848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f335c15e160 0x7f335c1423c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:09.987 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.978+0000 7f335affd640 1 -- 172.21.15.76:0/3208736848 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3344009400 con 0x7f335c0b64f0 2023-11-09T03:45:09.987 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.978+0000 7f335affd640 1 --2- 172.21.15.76:0/3208736848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f335c0b64f0 0x7f335c0c9000 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f334c034040 tx=0x7f334c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:09.988 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.978+0000 7f3358ff9640 1 -- 172.21.15.76:0/3208736848 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c03ac60 con 0x7f335c0b64f0 2023-11-09T03:45:09.988 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.978+0000 7f3358ff9640 1 -- 172.21.15.76:0/3208736848 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f334c03ade0 con 0x7f335c0b64f0 2023-11-09T03:45:09.988 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.978+0000 7f3358ff9640 1 -- 172.21.15.76:0/3208736848 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c044640 con 0x7f335c0b64f0 2023-11-09T03:45:09.988 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.978+0000 7f3361875640 1 -- 172.21.15.76:0/3208736848 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f334c009400 con 0x7f335c0b64f0 2023-11-09T03:45:09.988 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.978+0000 7f3361875640 1 -- 172.21.15.76:0/3208736848 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f335c1418e0 con 0x7f335c0b64f0 2023-11-09T03:45:09.989 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.978+0000 7f3358ff9640 1 -- 172.21.15.76:0/3208736848 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f334c044aa0 con 0x7f335c0b64f0 2023-11-09T03:45:09.989 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.978+0000 7f3358ff9640 1 -- 172.21.15.76:0/3208736848 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f334c02e070 con 0x7f335c0b64f0 2023-11-09T03:45:09.989 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.979+0000 7f3361875640 1 -- 172.21.15.76:0/3208736848 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f335c074290 con 0x7f335c0b64f0 2023-11-09T03:45:09.989 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:09.982+0000 7f3358ff9640 1 -- 172.21.15.76:0/3208736848 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f334c035020 con 0x7f335c0b64f0 2023-11-09T03:45:10.055 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:10.055+0000 7f3361875640 1 -- 172.21.15.76:0/3208736848 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f335c0ce650 con 0x7f335c0b64f0 2023-11-09T03:45:10.056 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:10.056+0000 7f3358ff9640 1 -- 172.21.15.76:0/3208736848 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f334c04d440 con 0x7f335c0b64f0 2023-11-09T03:45:10.057 INFO:teuthology.misc.health.smithi076.stdout: 2023-11-09T03:45:10.058 INFO:teuthology.misc.health.smithi076.stdout:{"epoch":10,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:45:08.827625+0000","last_up_change":"0.000000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:45:10.059 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:10.057+0000 7f3361875640 1 -- 172.21.15.76:0/3208736848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f335c0b64f0 msgr2=0x7f335c0c9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:10.059 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:10.057+0000 7f3361875640 1 --2- 172.21.15.76:0/3208736848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f335c0b64f0 0x7f335c0c9000 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f334c034040 tx=0x7f334c0046c0 comp rx=0 tx=0).stop 2023-11-09T03:45:10.059 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:10.057+0000 7f3361875640 1 -- 172.21.15.76:0/3208736848 shutdown_connections 2023-11-09T03:45:10.059 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:10.057+0000 7f3361875640 1 --2- 172.21.15.76:0/3208736848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f335c15e160 0x7f335c1423c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:10.060 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:10.057+0000 7f3361875640 1 --2- 172.21.15.76:0/3208736848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f335c0cbc90 0x7f335c144530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:10.060 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:10.057+0000 7f3361875640 1 --2- 172.21.15.76:0/3208736848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f335c0b64f0 0x7f335c0c9000 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:10.060 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:10.057+0000 7f3361875640 1 -- 172.21.15.76:0/3208736848 >> 172.21.15.76:0/3208736848 conn(0x7f335c0cd500 msgr2=0x7f335c0c5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:10.060 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:10.057+0000 7f3361875640 1 -- 172.21.15.76:0/3208736848 shutdown_connections 2023-11-09T03:45:10.060 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:10.057+0000 7f3361875640 1 -- 172.21.15.76:0/3208736848 wait complete. 2023-11-09T03:45:10.066 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-09T03:45:10.300 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-11-09T03:45:10.300+0000 7f50fc0b6040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-09T03:45:10.570 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-11-09T03:45:10.570+0000 7f50fc0b6040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-09T03:45:10.602 INFO:tasks.ceph.mgr.y.smithi076.stderr:2023-11-09T03:45:10.603+0000 7fe560a32040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-09T03:45:10.622 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-11-09T03:45:10.623+0000 7f50fc0b6040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-09T03:45:10.723 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-11-09T03:45:10.723+0000 7f50fc0b6040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-09T03:45:10.876 INFO:tasks.ceph.mgr.y.smithi076.stderr:2023-11-09T03:45:10.877+0000 7fe560a32040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-09T03:45:10.926 INFO:tasks.ceph.mgr.y.smithi076.stderr:2023-11-09T03:45:10.927+0000 7fe560a32040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-09T03:45:11.025 INFO:tasks.ceph.mgr.y.smithi076.stderr:2023-11-09T03:45:11.026+0000 7fe560a32040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-09T03:45:11.748 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-11-09T03:45:11.748+0000 7f509dbda640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-09T03:45:11.749 INFO:tasks.ceph.mgr.x.smithi113.stderr:2023-11-09T03:45:11.749+0000 7f509dbda640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-09T03:45:12.924 INFO:tasks.ceph.osd.5.smithi113.stderr:2023-11-09T03:45:12.924+0000 7f039cc0d500 -1 osd.5 0 log_to_monitors true 2023-11-09T03:45:12.952 INFO:tasks.ceph.osd.4.smithi113.stderr:2023-11-09T03:45:12.952+0000 7f8dad626500 -1 osd.4 0 log_to_monitors true 2023-11-09T03:45:12.953 INFO:tasks.ceph.osd.3.smithi076.stderr:2023-11-09T03:45:12.954+0000 7fd71d924500 -1 osd.3 0 log_to_monitors true 2023-11-09T03:45:12.973 INFO:tasks.ceph.osd.6.smithi113.stderr:2023-11-09T03:45:12.973+0000 7f2847412500 -1 osd.6 0 log_to_monitors true 2023-11-09T03:45:12.979 INFO:tasks.ceph.osd.0.smithi076.stderr:2023-11-09T03:45:12.980+0000 7f12c6212500 -1 osd.0 0 log_to_monitors true 2023-11-09T03:45:13.000 INFO:tasks.ceph.osd.1.smithi076.stderr:2023-11-09T03:45:13.001+0000 7fcb2c51a500 -1 osd.1 0 log_to_monitors true 2023-11-09T03:45:13.003 INFO:tasks.ceph.osd.7.smithi113.stderr:2023-11-09T03:45:13.004+0000 7f8a10b1a500 -1 osd.7 0 log_to_monitors true 2023-11-09T03:45:13.007 INFO:tasks.ceph.osd.2.smithi076.stderr:2023-11-09T03:45:13.008+0000 7f8f0bccf500 -1 osd.2 0 log_to_monitors true 2023-11-09T03:45:16.242 INFO:tasks.ceph.osd.6.smithi113.stderr:2023-11-09T03:45:16.242+0000 7f2842da9640 -1 osd.6 0 waiting for initial osdmap 2023-11-09T03:45:16.242 INFO:tasks.ceph.osd.7.smithi113.stderr:2023-11-09T03:45:16.242+0000 7f8a0caa8640 -1 osd.7 0 waiting for initial osdmap 2023-11-09T03:45:16.254 INFO:tasks.ceph.osd.6.smithi113.stderr:2023-11-09T03:45:16.254+0000 7f283abd0640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-09T03:45:16.254 INFO:tasks.ceph.osd.7.smithi113.stderr:2023-11-09T03:45:16.254+0000 7f8a048cf640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-09T03:45:16.271 INFO:tasks.ceph.osd.4.smithi113.stderr:2023-11-09T03:45:16.271+0000 7f8da8f68640 -1 osd.4 0 waiting for initial osdmap 2023-11-09T03:45:16.287 INFO:tasks.ceph.osd.4.smithi113.stderr:2023-11-09T03:45:16.287+0000 7f8da0d8f640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-09T03:45:16.526 INFO:tasks.ceph.osd.5.smithi113.stderr:2023-11-09T03:45:16.526+0000 7f03985b8640 -1 osd.5 0 waiting for initial osdmap 2023-11-09T03:45:16.540 INFO:tasks.ceph.osd.5.smithi113.stderr:2023-11-09T03:45:16.541+0000 7f03903df640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-09T03:45:16.617 INFO:tasks.ceph.osd.2.smithi076.stderr:2023-11-09T03:45:16.617+0000 7f8f07c5d640 -1 osd.2 0 waiting for initial osdmap 2023-11-09T03:45:16.637 INFO:tasks.ceph.osd.2.smithi076.stderr:2023-11-09T03:45:16.638+0000 7f8effa84640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-09T03:45:16.845 INFO:tasks.ceph.osd.1.smithi076.stderr:2023-11-09T03:45:16.847+0000 7fcb284a8640 -1 osd.1 0 waiting for initial osdmap 2023-11-09T03:45:16.857 INFO:tasks.ceph.osd.1.smithi076.stderr:2023-11-09T03:45:16.858+0000 7fcb202cf640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-09T03:45:16.873 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-09T03:45:16.885 INFO:tasks.ceph.osd.0.smithi076.stderr:2023-11-09T03:45:16.886+0000 7f12c1bfe640 -1 osd.0 0 waiting for initial osdmap 2023-11-09T03:45:16.890 INFO:tasks.ceph.osd.3.smithi076.stderr:2023-11-09T03:45:16.891+0000 7fd7198b2640 -1 osd.3 0 waiting for initial osdmap 2023-11-09T03:45:16.904 INFO:tasks.ceph.osd.0.smithi076.stderr:2023-11-09T03:45:16.905+0000 7f12b9a25640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-09T03:45:16.907 INFO:tasks.ceph.osd.3.smithi076.stderr:2023-11-09T03:45:16.909+0000 7fd7116d9640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-09T03:45:16.965 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.966+0000 7f0b8a444640 1 Processor -- start 2023-11-09T03:45:16.966 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.966+0000 7f0b8a444640 1 -- start start 2023-11-09T03:45:16.968 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.967+0000 7f0b8a444640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0b8406d910 0x7f0b8406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:16.968 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.967+0000 7f0b8a444640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b840ade60 0x7f0b840b1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:16.969 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.967+0000 7f0b8a444640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b840ca540 con 0x7f0b840b42b0 2023-11-09T03:45:16.969 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.967+0000 7f0b8a444640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b840cb2f0 con 0x7f0b8406d910 2023-11-09T03:45:16.969 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.967+0000 7f0b8a444640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8406e230 con 0x7f0b840ade60 2023-11-09T03:45:16.969 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.967+0000 7f0b83fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0b8406d910 0x7f0b8406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:16.969 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.967+0000 7f0b83fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0b8406d910 0x7f0b8406dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60362/0 (socket says 172.21.15.76:60362) 2023-11-09T03:45:16.970 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.967+0000 7f0b83fff640 1 -- 172.21.15.76:0/3812293487 learned_addr learned my addr 172.21.15.76:0/3812293487 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:16.970 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.967+0000 7f0b837fe640 1 --2- 172.21.15.76:0/3812293487 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b840ade60 0x7f0b840b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:16.970 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.967+0000 7f0b83fff640 1 -- 172.21.15.76:0/3812293487 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b840ade60 msgr2=0x7f0b840b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:16.970 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.967+0000 7f0b83fff640 1 --2- 172.21.15.76:0/3812293487 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b840ade60 0x7f0b840b1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:16.970 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.967+0000 7f0b83fff640 1 -- 172.21.15.76:0/3812293487 >> v1:172.21.15.113:6789/0 conn(0x7f0b840b42b0 legacy=0x7f0b84070b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:16.971 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.967+0000 7f0b83fff640 1 -- 172.21.15.76:0/3812293487 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b840ca770 con 0x7f0b8406d910 2023-11-09T03:45:16.971 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.968+0000 7f0b83fff640 1 --2- 172.21.15.76:0/3812293487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b8406d910 0x7f0b8406dcf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0b74009870 tx=0x7f0b7402db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6a736e4ddac192 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:16.971 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.968+0000 7f0b82ffd640 1 -- 172.21.15.76:0/3812293487 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7402f8c0 con 0x7f0b8406d910 2023-11-09T03:45:16.971 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.968+0000 7f0b82ffd640 1 -- 172.21.15.76:0/3812293487 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b7402fa20 con 0x7f0b8406d910 2023-11-09T03:45:16.971 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.968+0000 7f0b82ffd640 1 -- 172.21.15.76:0/3812293487 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7402fdb0 con 0x7f0b8406d910 2023-11-09T03:45:16.971 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.969+0000 7f0b8a444640 1 -- 172.21.15.76:0/3812293487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b8406d910 msgr2=0x7f0b8406dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:16.972 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.969+0000 7f0b8a444640 1 --2- 172.21.15.76:0/3812293487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b8406d910 0x7f0b8406dcf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0b74009870 tx=0x7f0b7402db00 comp rx=0 tx=0).stop 2023-11-09T03:45:16.972 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.969+0000 7f0b8a444640 1 -- 172.21.15.76:0/3812293487 shutdown_connections 2023-11-09T03:45:16.972 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.969+0000 7f0b8a444640 1 --2- 172.21.15.76:0/3812293487 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b840ade60 0x7f0b840b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:16.972 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.969+0000 7f0b8a444640 1 --2- 172.21.15.76:0/3812293487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b8406d910 0x7f0b8406dcf0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:16.972 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.969+0000 7f0b8a444640 1 -- 172.21.15.76:0/3812293487 >> 172.21.15.76:0/3812293487 conn(0x7f0b840ce9b0 msgr2=0x7f0b840cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:16.973 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.969+0000 7f0b8a444640 1 -- 172.21.15.76:0/3812293487 shutdown_connections 2023-11-09T03:45:16.973 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.969+0000 7f0b8a444640 1 -- 172.21.15.76:0/3812293487 wait complete. 2023-11-09T03:45:16.976 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.970+0000 7f0b8a444640 1 Processor -- start 2023-11-09T03:45:16.976 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.970+0000 7f0b8a444640 1 -- start start 2023-11-09T03:45:16.977 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.971+0000 7f0b8a444640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b8406d910 0x7f0b840b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:16.977 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.971+0000 7f0b8a444640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b840ade60 0x7f0b841ef100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:16.977 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.971+0000 7f0b8a444640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b840b42b0 0x7f0b841ebab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:16.977 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.971+0000 7f0b8a444640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b841ebff0 con 0x7f0b840ade60 2023-11-09T03:45:16.977 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.971+0000 7f0b8a444640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b840b1b30 con 0x7f0b840b42b0 2023-11-09T03:45:16.978 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.971+0000 7f0b8a444640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b840b1e10 con 0x7f0b8406d910 2023-11-09T03:45:16.978 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.971+0000 7f0b837fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b840ade60 0x7f0b841ef100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:16.978 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.971+0000 7f0b837fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b840ade60 0x7f0b841ef100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60376/0 (socket says 172.21.15.76:60376) 2023-11-09T03:45:16.978 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.971+0000 7f0b837fe640 1 -- 172.21.15.76:0/1125797754 learned_addr learned my addr 172.21.15.76:0/1125797754 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:16.978 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.971+0000 7f0b83fff640 1 --2- 172.21.15.76:0/1125797754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b8406d910 0x7f0b840b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:16.979 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.971+0000 7f0b889e1640 1 --2- 172.21.15.76:0/1125797754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b840b42b0 0x7f0b841ebab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:16.979 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.971+0000 7f0b837fe640 1 -- 172.21.15.76:0/1125797754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b8406d910 msgr2=0x7f0b840b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:16.979 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.971+0000 7f0b837fe640 1 --2- 172.21.15.76:0/1125797754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b8406d910 0x7f0b840b2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:16.979 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.971+0000 7f0b837fe640 1 -- 172.21.15.76:0/1125797754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b840b42b0 msgr2=0x7f0b841ebab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:16.980 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.971+0000 7f0b837fe640 1 --2- 172.21.15.76:0/1125797754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b840b42b0 0x7f0b841ebab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:16.980 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.971+0000 7f0b837fe640 1 -- 172.21.15.76:0/1125797754 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b740094d0 con 0x7f0b840ade60 2023-11-09T03:45:16.980 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.971+0000 7f0b83fff640 1 --2- 172.21.15.76:0/1125797754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b8406d910 0x7f0b840b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:16.980 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.971+0000 7f0b837fe640 1 --2- 172.21.15.76:0/1125797754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b840ade60 0x7f0b841ef100 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800c7f0 tx=0x7f0b7800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:16.980 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.971+0000 7f0b80ff9640 1 -- 172.21.15.76:0/1125797754 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b78004220 con 0x7f0b840ade60 2023-11-09T03:45:16.981 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.971+0000 7f0b80ff9640 1 -- 172.21.15.76:0/1125797754 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b780043a0 con 0x7f0b840ade60 2023-11-09T03:45:16.981 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.971+0000 7f0b8a444640 1 -- 172.21.15.76:0/1125797754 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b840b20f0 con 0x7f0b840ade60 2023-11-09T03:45:16.981 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.971+0000 7f0b80ff9640 1 -- 172.21.15.76:0/1125797754 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b78007860 con 0x7f0b840ade60 2023-11-09T03:45:16.981 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.971+0000 7f0b8a444640 1 -- 172.21.15.76:0/1125797754 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b840b26b0 con 0x7f0b840ade60 2023-11-09T03:45:16.981 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.972+0000 7f0b8a444640 1 -- 172.21.15.76:0/1125797754 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b8415adf0 con 0x7f0b840ade60 2023-11-09T03:45:16.981 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.973+0000 7f0b80ff9640 1 -- 172.21.15.76:0/1125797754 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f0b78007bd0 con 0x7f0b840ade60 2023-11-09T03:45:16.982 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.973+0000 7f0b80ff9640 1 --2- 172.21.15.76:0/1125797754 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0b7006ae10 0x7f0b7006d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:16.982 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.973+0000 7f0b80ff9640 1 -- 172.21.15.76:0/1125797754 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f0b78088730 con 0x7f0b840ade60 2023-11-09T03:45:16.982 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.973+0000 7f0b83fff640 1 --2- 172.21.15.76:0/1125797754 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0b7006ae10 0x7f0b7006d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:16.982 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.973+0000 7f0b83fff640 1 --2- 172.21.15.76:0/1125797754 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0b7006ae10 0x7f0b7006d2d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0b74038000 tx=0x7f0b74008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:16.982 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:16.977+0000 7f0b80ff9640 1 -- 172.21.15.76:0/1125797754 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0b78054650 con 0x7f0b840ade60 2023-11-09T03:45:17.146 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:17.147+0000 7f0b8a444640 1 -- 172.21.15.76:0/1125797754 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b840cad00 con 0x7f0b840ade60 2023-11-09T03:45:17.147 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:17.148+0000 7f0b80ff9640 1 -- 172.21.15.76:0/1125797754 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f0b7801e020 con 0x7f0b840ade60 2023-11-09T03:45:17.147 INFO:teuthology.misc.health.smithi076.stdout: 2023-11-09T03:45:17.148 INFO:teuthology.misc.health.smithi076.stdout:{"epoch":12,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:45:14.750060+0000","last_up_change":"0.000000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:45:17.149 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:17.150+0000 7f0b8a444640 1 -- 172.21.15.76:0/1125797754 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0b7006ae10 msgr2=0x7f0b7006d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:17.149 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:17.150+0000 7f0b8a444640 1 --2- 172.21.15.76:0/1125797754 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0b7006ae10 0x7f0b7006d2d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0b74038000 tx=0x7f0b74008030 comp rx=0 tx=0).stop 2023-11-09T03:45:17.150 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:17.150+0000 7f0b8a444640 1 -- 172.21.15.76:0/1125797754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b840ade60 msgr2=0x7f0b841ef100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:17.150 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:17.150+0000 7f0b8a444640 1 --2- 172.21.15.76:0/1125797754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b840ade60 0x7f0b841ef100 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800c7f0 tx=0x7f0b7800ccb0 comp rx=0 tx=0).stop 2023-11-09T03:45:17.150 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:17.150+0000 7f0b8a444640 1 -- 172.21.15.76:0/1125797754 shutdown_connections 2023-11-09T03:45:17.150 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:17.150+0000 7f0b8a444640 1 --2- 172.21.15.76:0/1125797754 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0b7006ae10 0x7f0b7006d2d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:17.150 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:17.150+0000 7f0b8a444640 1 --2- 172.21.15.76:0/1125797754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b840b42b0 0x7f0b841ebab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:17.151 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:17.150+0000 7f0b8a444640 1 --2- 172.21.15.76:0/1125797754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b840ade60 0x7f0b841ef100 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:17.151 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:17.150+0000 7f0b8a444640 1 --2- 172.21.15.76:0/1125797754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b8406d910 0x7f0b840b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:17.151 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:17.150+0000 7f0b8a444640 1 -- 172.21.15.76:0/1125797754 >> 172.21.15.76:0/1125797754 conn(0x7f0b840ce9b0 msgr2=0x7f0b8406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:17.151 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:17.150+0000 7f0b8a444640 1 -- 172.21.15.76:0/1125797754 shutdown_connections 2023-11-09T03:45:17.151 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:17.150+0000 7f0b8a444640 1 -- 172.21.15.76:0/1125797754 wait complete. 2023-11-09T03:45:17.159 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-09T03:45:23.966 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-09T03:45:24.067 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.068+0000 7f39af8d4640 1 Processor -- start 2023-11-09T03:45:24.067 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.068+0000 7f39af8d4640 1 -- start start 2023-11-09T03:45:24.071 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.069+0000 7f39af8d4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f39a815ade0 0x7f39a80707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.071 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.069+0000 7f39af8d4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39a815b1c0 0x7f39a8070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.071 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.069+0000 7f39af8d4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39a806e5a0 con 0x7f39a806c800 2023-11-09T03:45:24.071 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.069+0000 7f39af8d4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39a80cf750 con 0x7f39a815ade0 2023-11-09T03:45:24.072 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.069+0000 7f39af8d4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a806e2c0 con 0x7f39a815b1c0 2023-11-09T03:45:24.072 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.069+0000 7f39ade71640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f39a806c800 0x7f39a806cc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:60024/0 (socket says 172.21.15.76:60024) 2023-11-09T03:45:24.072 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.069+0000 7f39ade71640 1 -- 172.21.15.76:0/3239082485 learned_addr learned my addr 172.21.15.76:0/3239082485 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:24.072 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.069+0000 7f39ace6f640 1 --2- 172.21.15.76:0/3239082485 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39a815b1c0 0x7f39a8070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.073 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.069+0000 7f39ad670640 1 --2- 172.21.15.76:0/3239082485 >> v2:172.21.15.113:3300/0 conn(0x7f39a815ade0 0x7f39a80707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.073 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.069+0000 7f399ffff640 1 -- 172.21.15.76:0/3239082485 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 422721069 0 0) 0x7f39a806e5a0 con 0x7f39a806c800 2023-11-09T03:45:24.073 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.069+0000 7f399ffff640 1 -- 172.21.15.76:0/3239082485 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3994003520 con 0x7f39a806c800 2023-11-09T03:45:24.073 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.070+0000 7f399ffff640 1 -- 172.21.15.76:0/3239082485 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4051562497 0 0) 0x7f3994003520 con 0x7f39a806c800 2023-11-09T03:45:24.073 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.070+0000 7f399ffff640 1 -- 172.21.15.76:0/3239082485 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39a815b1c0 msgr2=0x7f39a8070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:24.074 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.070+0000 7f399ffff640 1 --2- 172.21.15.76:0/3239082485 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39a815b1c0 0x7f39a8070ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.074 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.070+0000 7f399ffff640 1 -- 172.21.15.76:0/3239082485 >> v2:172.21.15.113:3300/0 conn(0x7f39a815ade0 msgr2=0x7f39a80707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:24.074 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.070+0000 7f399ffff640 1 --2- 172.21.15.76:0/3239082485 >> v2:172.21.15.113:3300/0 conn(0x7f39a815ade0 0x7f39a80707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.074 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.070+0000 7f399ffff640 1 -- 172.21.15.76:0/3239082485 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a80c9ab0 con 0x7f39a806c800 2023-11-09T03:45:24.074 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.070+0000 7f399ffff640 1 -- 172.21.15.76:0/3239082485 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f39a00019f0 con 0x7f39a806c800 2023-11-09T03:45:24.075 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.070+0000 7f399ffff640 1 -- 172.21.15.76:0/3239082485 >> v1:172.21.15.76:6789/0 conn(0x7f39a806c800 legacy=0x7f39a806cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:24.075 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.070+0000 7f399ffff640 1 --2- 172.21.15.76:0/3239082485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3994003970 0x7f3994003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.075 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.070+0000 7f399ffff640 1 --2- 172.21.15.76:0/3239082485 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39a815b1c0 0x7f39940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.075 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.070+0000 7f399ffff640 1 --2- 172.21.15.76:0/3239082485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3994004d00 0x7f3994005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.076 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.070+0000 7f399ffff640 1 -- 172.21.15.76:0/3239082485 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a80cf750 con 0x7f3994003970 2023-11-09T03:45:24.076 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.070+0000 7f399ffff640 1 -- 172.21.15.76:0/3239082485 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a806e2c0 con 0x7f3994004d00 2023-11-09T03:45:24.076 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.070+0000 7f399ffff640 1 -- 172.21.15.76:0/3239082485 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399400b980 con 0x7f39a815b1c0 2023-11-09T03:45:24.076 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.070+0000 7f39ad670640 1 --2- 172.21.15.76:0/3239082485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3994003970 0x7f3994003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.076 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.070+0000 7f39ace6f640 1 --2- 172.21.15.76:0/3239082485 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39a815b1c0 0x7f39940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.077 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.070+0000 7f39ade71640 1 --2- 172.21.15.76:0/3239082485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3994004d00 0x7f3994005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.077 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.070+0000 7f39ad670640 1 -- 172.21.15.76:0/3239082485 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39a815b1c0 msgr2=0x7f39940047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:24.077 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.070+0000 7f39ad670640 1 --2- 172.21.15.76:0/3239082485 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39a815b1c0 0x7f39940047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.077 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.070+0000 7f39ad670640 1 -- 172.21.15.76:0/3239082485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3994004d00 msgr2=0x7f3994005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:24.077 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.070+0000 7f39ad670640 1 --2- 172.21.15.76:0/3239082485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3994004d00 0x7f3994005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.077 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.070+0000 7f39ad670640 1 -- 172.21.15.76:0/3239082485 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3994003520 con 0x7f3994003970 2023-11-09T03:45:24.078 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.071+0000 7f39ad670640 1 --2- 172.21.15.76:0/3239082485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3994003970 0x7f3994003d90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f39a4009af0 tx=0x7f39940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c0ea9a23d0f367 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:24.078 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.071+0000 7f399ffff640 1 -- 172.21.15.76:0/3239082485 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a402c710 con 0x7f3994003970 2023-11-09T03:45:24.078 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.071+0000 7f399ffff640 1 -- 172.21.15.76:0/3239082485 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a402c870 con 0x7f3994003970 2023-11-09T03:45:24.078 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.071+0000 7f399ffff640 1 -- 172.21.15.76:0/3239082485 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a402cc00 con 0x7f3994003970 2023-11-09T03:45:24.079 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.071+0000 7f39af8d4640 1 -- 172.21.15.76:0/3239082485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3994003970 msgr2=0x7f3994003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:24.079 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.071+0000 7f39af8d4640 1 --2- 172.21.15.76:0/3239082485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3994003970 0x7f3994003d90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f39a4009af0 tx=0x7f39940057f0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.079 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.072+0000 7f39ade71640 1 -- 172.21.15.76:0/3239082485 reap_dead start 2023-11-09T03:45:24.079 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.072+0000 7f39af8d4640 1 -- 172.21.15.76:0/3239082485 shutdown_connections 2023-11-09T03:45:24.079 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.072+0000 7f39af8d4640 1 -- 172.21.15.76:0/3239082485 >> 172.21.15.76:0/3239082485 conn(0x7f39a80ce9b0 msgr2=0x7f39a80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:24.080 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.072+0000 7f39af8d4640 1 -- 172.21.15.76:0/3239082485 shutdown_connections 2023-11-09T03:45:24.080 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.072+0000 7f39af8d4640 1 -- 172.21.15.76:0/3239082485 wait complete. 2023-11-09T03:45:24.083 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.073+0000 7f39af8d4640 1 Processor -- start 2023-11-09T03:45:24.083 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.073+0000 7f39af8d4640 1 -- start start 2023-11-09T03:45:24.083 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.074+0000 7f39af8d4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3994003970 0x7f39a81f3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.083 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.074+0000 7f39af8d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39a81f64d0 0x7f39a80c4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.084 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.074+0000 7f39af8d4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39a80c32e0 0x7f39a80c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.084 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.074+0000 7f39af8d4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a81f3ec0 con 0x7f39a81f64d0 2023-11-09T03:45:24.084 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.074+0000 7f39af8d4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a80c4850 con 0x7f3994003970 2023-11-09T03:45:24.084 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.074+0000 7f39af8d4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a80c8710 con 0x7f39a80c32e0 2023-11-09T03:45:24.084 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.074+0000 7f39ace6f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39a81f64d0 0x7f39a80c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.085 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.074+0000 7f39ace6f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39a81f64d0 0x7f39a80c4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54316/0 (socket says 172.21.15.76:54316) 2023-11-09T03:45:24.085 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.074+0000 7f39ace6f640 1 -- 172.21.15.76:0/468279374 learned_addr learned my addr 172.21.15.76:0/468279374 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:24.085 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.074+0000 7f39ade71640 1 --2- 172.21.15.76:0/468279374 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39a80c32e0 0x7f39a80c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.085 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.074+0000 7f39ad670640 1 --2- 172.21.15.76:0/468279374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3994003970 0x7f39a81f3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.085 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.074+0000 7f39ace6f640 1 -- 172.21.15.76:0/468279374 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39a80c32e0 msgr2=0x7f39a80c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:24.086 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.074+0000 7f39ace6f640 1 --2- 172.21.15.76:0/468279374 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39a80c32e0 0x7f39a80c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.086 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.074+0000 7f39ace6f640 1 -- 172.21.15.76:0/468279374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3994003970 msgr2=0x7f39a81f3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:24.086 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.074+0000 7f39ace6f640 1 --2- 172.21.15.76:0/468279374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3994003970 0x7f39a81f3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.086 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.074+0000 7f39ace6f640 1 -- 172.21.15.76:0/468279374 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a81f6a40 con 0x7f39a81f64d0 2023-11-09T03:45:24.087 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.074+0000 7f39ade71640 1 --2- 172.21.15.76:0/468279374 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39a80c32e0 0x7f39a80c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:24.087 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.074+0000 7f39ad670640 1 --2- 172.21.15.76:0/468279374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3994003970 0x7f39a81f3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:24.087 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.074+0000 7f39ace6f640 1 --2- 172.21.15.76:0/468279374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39a81f64d0 0x7f39a80c4310 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f39980119a0 tx=0x7f3998011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:24.087 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.074+0000 7f399dffb640 1 -- 172.21.15.76:0/468279374 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399800faf0 con 0x7f39a81f64d0 2023-11-09T03:45:24.087 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.074+0000 7f399dffb640 1 -- 172.21.15.76:0/468279374 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399800fc50 con 0x7f39a81f64d0 2023-11-09T03:45:24.087 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.074+0000 7f39af8d4640 1 -- 172.21.15.76:0/468279374 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a81f6d20 con 0x7f39a81f64d0 2023-11-09T03:45:24.088 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.074+0000 7f399dffb640 1 -- 172.21.15.76:0/468279374 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3998014440 con 0x7f39a81f64d0 2023-11-09T03:45:24.088 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.075+0000 7f39af8d4640 1 -- 172.21.15.76:0/468279374 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a80c71d0 con 0x7f39a81f64d0 2023-11-09T03:45:24.088 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.076+0000 7f39af8d4640 1 -- 172.21.15.76:0/468279374 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39a804e490 con 0x7f39a81f64d0 2023-11-09T03:45:24.088 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.076+0000 7f399dffb640 1 -- 172.21.15.76:0/468279374 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f39980145a0 con 0x7f39a81f64d0 2023-11-09T03:45:24.088 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.076+0000 7f399dffb640 1 --2- 172.21.15.76:0/468279374 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f398c06aea0 0x7f398c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.089 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.076+0000 7f399dffb640 1 -- 172.21.15.76:0/468279374 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3998089b30 con 0x7f39a81f64d0 2023-11-09T03:45:24.089 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.077+0000 7f39ad670640 1 --2- 172.21.15.76:0/468279374 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f398c06aea0 0x7f398c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.089 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.077+0000 7f39ad670640 1 --2- 172.21.15.76:0/468279374 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f398c06aea0 0x7f398c06d360 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f39a4036000 tx=0x7f39a4030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:24.089 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.081+0000 7f399dffb640 1 -- 172.21.15.76:0/468279374 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3998054f30 con 0x7f39a81f64d0 2023-11-09T03:45:24.247 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.247+0000 7f39af8d4640 1 -- 172.21.15.76:0/468279374 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39a80b44f0 con 0x7f39a81f64d0 2023-11-09T03:45:24.247 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.248+0000 7f399dffb640 1 -- 172.21.15.76:0/468279374 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12838 (secure 0 0 0) 0x7f3998022090 con 0x7f39a81f64d0 2023-11-09T03:45:24.248 INFO:teuthology.misc.health.smithi076.stdout: 2023-11-09T03:45:24.248 INFO:teuthology.misc.health.smithi076.stdout:{"epoch":17,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:45:21.864770+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:45:24.250 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.250+0000 7f39af8d4640 1 -- 172.21.15.76:0/468279374 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f398c06aea0 msgr2=0x7f398c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:24.250 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.250+0000 7f39af8d4640 1 --2- 172.21.15.76:0/468279374 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f398c06aea0 0x7f398c06d360 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f39a4036000 tx=0x7f39a4030000 comp rx=0 tx=0).stop 2023-11-09T03:45:24.250 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.250+0000 7f39af8d4640 1 -- 172.21.15.76:0/468279374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39a81f64d0 msgr2=0x7f39a80c4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:24.250 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.250+0000 7f39af8d4640 1 --2- 172.21.15.76:0/468279374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39a81f64d0 0x7f39a80c4310 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f39980119a0 tx=0x7f3998011e60 comp rx=0 tx=0).stop 2023-11-09T03:45:24.250 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.250+0000 7f39af8d4640 1 -- 172.21.15.76:0/468279374 shutdown_connections 2023-11-09T03:45:24.251 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.250+0000 7f39af8d4640 1 --2- 172.21.15.76:0/468279374 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f398c06aea0 0x7f398c06d360 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.251 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.250+0000 7f39af8d4640 1 --2- 172.21.15.76:0/468279374 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39a80c32e0 0x7f39a80c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.251 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.250+0000 7f39af8d4640 1 --2- 172.21.15.76:0/468279374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39a81f64d0 0x7f39a80c4310 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.251 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.250+0000 7f39af8d4640 1 --2- 172.21.15.76:0/468279374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3994003970 0x7f39a81f3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.251 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.250+0000 7f39af8d4640 1 -- 172.21.15.76:0/468279374 >> 172.21.15.76:0/468279374 conn(0x7f39a80ce9b0 msgr2=0x7f39a8071520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:24.252 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.250+0000 7f39af8d4640 1 -- 172.21.15.76:0/468279374 shutdown_connections 2023-11-09T03:45:24.252 INFO:teuthology.misc.health.smithi076.stderr:2023-11-09T03:45:24.250+0000 7f39af8d4640 1 -- 172.21.15.76:0/468279374 wait complete. 2023-11-09T03:45:24.259 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-09T03:45:24.259 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-09T03:45:24.260 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-09T03:45:24.260 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-09T03:45:24.262 INFO:tasks.ceph.mds.a:Started 2023-11-09T03:45:24.263 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-09T03:45:24.263 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-09T03:45:24.265 INFO:tasks.ceph.mds.c:Started 2023-11-09T03:45:24.265 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-09T03:45:24.266 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-09T03:45:24.268 INFO:tasks.ceph.mds.b:Started 2023-11-09T03:45:24.269 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-09T03:45:24.269 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-09T03:45:24.271 INFO:tasks.ceph.mds.d:Started 2023-11-09T03:45:24.272 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-09T03:45:24.272 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:45:24.338 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2023-11-09T03:45:24.339 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2023-11-09T03:45:24.352 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2023-11-09T03:45:24.353 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2023-11-09T03:45:24.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.379+0000 7fe47cd25640 1 Processor -- start 2023-11-09T03:45:24.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.379+0000 7fe47cd25640 1 -- start start 2023-11-09T03:45:24.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.380+0000 7fe47cd25640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe47815ac70 0x7fe4780d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.380+0000 7fe47cd25640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe4780cec10 0x7fe4780b2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.380+0000 7fe47cd25640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4780b3c10 con 0x7fe4780b2b40 2023-11-09T03:45:24.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.380+0000 7fe47cd25640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4780b2fb0 con 0x7fe4780cec10 2023-11-09T03:45:24.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.380+0000 7fe47cd25640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4780cf180 con 0x7fe47815ac70 2023-11-09T03:45:24.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.380+0000 7fe476d9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe4780b2b40 0x7fe4780701a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:60050/0 (socket says 172.21.15.76:60050) 2023-11-09T03:45:24.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.380+0000 7fe476d9d640 1 -- 172.21.15.76:0/1206193951 learned_addr learned my addr 172.21.15.76:0/1206193951 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:24.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.380+0000 7fe475d9b640 1 -- 172.21.15.76:0/1206193951 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1587396652 0 0) 0x7fe4780b3c10 con 0x7fe4780b2b40 2023-11-09T03:45:24.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.380+0000 7fe47659c640 1 --2- 172.21.15.76:0/1206193951 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe47815ac70 0x7fe4780d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.380+0000 7fe46dd9b640 1 --2- 172.21.15.76:0/1206193951 >> v2:172.21.15.113:3300/0 conn(0x7fe4780cec10 0x7fe4780b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.380+0000 7fe475d9b640 1 -- 172.21.15.76:0/1206193951 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe464003520 con 0x7fe4780b2b40 2023-11-09T03:45:24.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.381+0000 7fe475d9b640 1 -- 172.21.15.76:0/1206193951 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1367515603 0 0) 0x7fe464003520 con 0x7fe4780b2b40 2023-11-09T03:45:24.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.381+0000 7fe475d9b640 1 -- 172.21.15.76:0/1206193951 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe47815ac70 msgr2=0x7fe4780d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:24.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.381+0000 7fe475d9b640 1 --2- 172.21.15.76:0/1206193951 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe47815ac70 0x7fe4780d0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.381+0000 7fe475d9b640 1 -- 172.21.15.76:0/1206193951 >> v2:172.21.15.113:3300/0 conn(0x7fe4780cec10 msgr2=0x7fe4780b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:24.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.381+0000 7fe475d9b640 1 --2- 172.21.15.76:0/1206193951 >> v2:172.21.15.113:3300/0 conn(0x7fe4780cec10 0x7fe4780b2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.381+0000 7fe475d9b640 1 -- 172.21.15.76:0/1206193951 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4780cf460 con 0x7fe4780b2b40 2023-11-09T03:45:24.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.381+0000 7fe475d9b640 1 -- 172.21.15.76:0/1206193951 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fe470001a10 con 0x7fe4780b2b40 2023-11-09T03:45:24.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.381+0000 7fe47659c640 1 --2- 172.21.15.76:0/1206193951 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe47815ac70 0x7fe4780d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:24.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.381+0000 7fe475d9b640 1 -- 172.21.15.76:0/1206193951 >> v1:172.21.15.76:6789/0 conn(0x7fe4780b2b40 legacy=0x7fe4780701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:24.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.381+0000 7fe475d9b640 1 --2- 172.21.15.76:0/1206193951 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe47815ac70 0x7fe464003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.381+0000 7fe475d9b640 1 --2- 172.21.15.76:0/1206193951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe464003eb0 0x7fe4640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.381+0000 7fe475d9b640 1 --2- 172.21.15.76:0/1206193951 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe464004d00 0x7fe464005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.381+0000 7fe475d9b640 1 -- 172.21.15.76:0/1206193951 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4780b2fb0 con 0x7fe464004d00 2023-11-09T03:45:24.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.381+0000 7fe475d9b640 1 -- 172.21.15.76:0/1206193951 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4780cf180 con 0x7fe464003eb0 2023-11-09T03:45:24.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.381+0000 7fe475d9b640 1 -- 172.21.15.76:0/1206193951 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe46400b9b0 con 0x7fe47815ac70 2023-11-09T03:45:24.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.381+0000 7fe476d9d640 1 --2- 172.21.15.76:0/1206193951 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe464004d00 0x7fe464005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.381+0000 7fe47659c640 1 --2- 172.21.15.76:0/1206193951 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe47815ac70 0x7fe464003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.381+0000 7fe46dd9b640 1 --2- 172.21.15.76:0/1206193951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe464003eb0 0x7fe4640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.381+0000 7fe476d9d640 1 -- 172.21.15.76:0/1206193951 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe47815ac70 msgr2=0x7fe464003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:24.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.381+0000 7fe476d9d640 1 --2- 172.21.15.76:0/1206193951 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe47815ac70 0x7fe464003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.381+0000 7fe476d9d640 1 -- 172.21.15.76:0/1206193951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe464003eb0 msgr2=0x7fe4640047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:24.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.381+0000 7fe476d9d640 1 --2- 172.21.15.76:0/1206193951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe464003eb0 0x7fe4640047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.381+0000 7fe476d9d640 1 -- 172.21.15.76:0/1206193951 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe464003520 con 0x7fe464004d00 2023-11-09T03:45:24.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.382+0000 7fe476d9d640 1 --2- 172.21.15.76:0/1206193951 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe464004d00 0x7fe464005180 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe47000a850 tx=0x7fe4640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07da14dcde1a24 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:24.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.382+0000 7fe475d9b640 1 -- 172.21.15.76:0/1206193951 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe470034630 con 0x7fe464004d00 2023-11-09T03:45:24.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.382+0000 7fe475d9b640 1 -- 172.21.15.76:0/1206193951 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47002f070 con 0x7fe464004d00 2023-11-09T03:45:24.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.382+0000 7fe475d9b640 1 -- 172.21.15.76:0/1206193951 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4700349d0 con 0x7fe464004d00 2023-11-09T03:45:24.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.382+0000 7fe47cd25640 1 -- 172.21.15.76:0/1206193951 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe464004d00 msgr2=0x7fe464005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:24.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.382+0000 7fe47cd25640 1 --2- 172.21.15.76:0/1206193951 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe464004d00 0x7fe464005180 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe47000a850 tx=0x7fe4640057f0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.382+0000 7fe476d9d640 1 -- 172.21.15.76:0/1206193951 reap_dead start 2023-11-09T03:45:24.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.383+0000 7fe47cd25640 1 -- 172.21.15.76:0/1206193951 shutdown_connections 2023-11-09T03:45:24.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.383+0000 7fe47cd25640 1 -- 172.21.15.76:0/1206193951 >> 172.21.15.76:0/1206193951 conn(0x7fe4780cd600 msgr2=0x7fe4780cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:24.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.383+0000 7fe47cd25640 1 -- 172.21.15.76:0/1206193951 shutdown_connections 2023-11-09T03:45:24.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.383+0000 7fe47cd25640 1 -- 172.21.15.76:0/1206193951 wait complete. 2023-11-09T03:45:24.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.384+0000 7fe47cd25640 1 Processor -- start 2023-11-09T03:45:24.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.384+0000 7fe47cd25640 1 -- start start 2023-11-09T03:45:24.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.385+0000 7fe47cd25640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe464004d00 0x7fe4781f25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.385+0000 7fe47cd25640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4781f5bb0 0x7fe478152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.385+0000 7fe47cd25640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe47815b2c0 0x7fe4780c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.385+0000 7fe47cd25640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4781f6030 con 0x7fe47815b2c0 2023-11-09T03:45:24.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.385+0000 7fe47cd25640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4781f6ae0 con 0x7fe4781f5bb0 2023-11-09T03:45:24.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.385+0000 7fe47cd25640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe47815d1c0 con 0x7fe464004d00 2023-11-09T03:45:24.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.385+0000 7fe476d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe47815b2c0 0x7fe4780c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.385+0000 7fe476d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe47815b2c0 0x7fe4780c7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54362/0 (socket says 172.21.15.76:54362) 2023-11-09T03:45:24.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.385+0000 7fe476d9d640 1 -- 172.21.15.76:0/2570752242 learned_addr learned my addr 172.21.15.76:0/2570752242 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:24.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.385+0000 7fe47659c640 1 --2- 172.21.15.76:0/2570752242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe464004d00 0x7fe4781f25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.385+0000 7fe46dd9b640 1 --2- 172.21.15.76:0/2570752242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4781f5bb0 0x7fe478152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.385+0000 7fe476d9d640 1 -- 172.21.15.76:0/2570752242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe464004d00 msgr2=0x7fe4781f25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:24.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.385+0000 7fe476d9d640 1 --2- 172.21.15.76:0/2570752242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe464004d00 0x7fe4781f25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.385+0000 7fe476d9d640 1 -- 172.21.15.76:0/2570752242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4781f5bb0 msgr2=0x7fe478152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:24.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.385+0000 7fe476d9d640 1 --2- 172.21.15.76:0/2570752242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4781f5bb0 0x7fe478152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.385+0000 7fe476d9d640 1 -- 172.21.15.76:0/2570752242 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe47806d240 con 0x7fe47815b2c0 2023-11-09T03:45:24.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.385+0000 7fe476d9d640 1 --2- 172.21.15.76:0/2570752242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe47815b2c0 0x7fe4780c7d50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe47002cc40 tx=0x7fe470001da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:24.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.386+0000 7fe46f7fe640 1 -- 172.21.15.76:0/2570752242 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe470001ff0 con 0x7fe47815b2c0 2023-11-09T03:45:24.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.386+0000 7fe46f7fe640 1 -- 172.21.15.76:0/2570752242 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe470034640 con 0x7fe47815b2c0 2023-11-09T03:45:24.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.386+0000 7fe46f7fe640 1 -- 172.21.15.76:0/2570752242 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4700349d0 con 0x7fe47815b2c0 2023-11-09T03:45:24.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.386+0000 7fe47cd25640 1 -- 172.21.15.76:0/2570752242 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4781f6240 con 0x7fe47815b2c0 2023-11-09T03:45:24.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.386+0000 7fe47cd25640 1 -- 172.21.15.76:0/2570752242 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4781f6800 con 0x7fe47815b2c0 2023-11-09T03:45:24.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.387+0000 7fe47cd25640 1 -- 172.21.15.76:0/2570752242 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe47815d300 con 0x7fe47815b2c0 2023-11-09T03:45:24.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.387+0000 7fe46f7fe640 1 -- 172.21.15.76:0/2570752242 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe470031030 con 0x7fe47815b2c0 2023-11-09T03:45:24.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.387+0000 7fe46f7fe640 1 --2- 172.21.15.76:0/2570752242 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe45806aea0 0x7fe45806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.388+0000 7fe46f7fe640 1 -- 172.21.15.76:0/2570752242 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe4700ab590 con 0x7fe47815b2c0 2023-11-09T03:45:24.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.388+0000 7fe47659c640 1 --2- 172.21.15.76:0/2570752242 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe45806aea0 0x7fe45806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.388+0000 7fe47659c640 1 --2- 172.21.15.76:0/2570752242 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe45806aea0 0x7fe45806d360 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe4780c88f0 tx=0x7fe468008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:24.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.392+0000 7fe46f7fe640 1 -- 172.21.15.76:0/2570752242 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe470076b40 con 0x7fe47815b2c0 2023-11-09T03:45:24.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.558+0000 7fe47cd25640 1 -- 172.21.15.76:0/2570752242 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe47815d300 con 0x7fe47815b2c0 2023-11-09T03:45:24.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.559+0000 7fe46f7fe640 1 -- 172.21.15.76:0/2570752242 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12838 (secure 0 0 0) 0x7fe47007a160 con 0x7fe47815b2c0 2023-11-09T03:45:24.558 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:24.559 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":17,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:45:21.864770+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:45:24.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.561+0000 7fe47cd25640 1 -- 172.21.15.76:0/2570752242 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe45806aea0 msgr2=0x7fe45806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:24.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.561+0000 7fe47cd25640 1 --2- 172.21.15.76:0/2570752242 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe45806aea0 0x7fe45806d360 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe4780c88f0 tx=0x7fe468008040 comp rx=0 tx=0).stop 2023-11-09T03:45:24.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.561+0000 7fe47cd25640 1 -- 172.21.15.76:0/2570752242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe47815b2c0 msgr2=0x7fe4780c7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:24.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.561+0000 7fe47cd25640 1 --2- 172.21.15.76:0/2570752242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe47815b2c0 0x7fe4780c7d50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe47002cc40 tx=0x7fe470001da0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.561+0000 7fe47cd25640 1 -- 172.21.15.76:0/2570752242 shutdown_connections 2023-11-09T03:45:24.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.561+0000 7fe47cd25640 1 --2- 172.21.15.76:0/2570752242 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe45806aea0 0x7fe45806d360 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.561+0000 7fe47cd25640 1 --2- 172.21.15.76:0/2570752242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe47815b2c0 0x7fe4780c7d50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.561+0000 7fe47cd25640 1 --2- 172.21.15.76:0/2570752242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4781f5bb0 0x7fe478152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.561+0000 7fe47cd25640 1 --2- 172.21.15.76:0/2570752242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe464004d00 0x7fe4781f25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.561+0000 7fe47cd25640 1 -- 172.21.15.76:0/2570752242 >> 172.21.15.76:0/2570752242 conn(0x7fe4780cd600 msgr2=0x7fe4780b3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:24.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.561+0000 7fe47cd25640 1 -- 172.21.15.76:0/2570752242 shutdown_connections 2023-11-09T03:45:24.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.561+0000 7fe47cd25640 1 -- 172.21.15.76:0/2570752242 wait complete. 2023-11-09T03:45:24.571 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-09T03:45:24.572 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:45:24.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.675+0000 7f1504dfb640 1 Processor -- start 2023-11-09T03:45:24.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.675+0000 7f1504dfb640 1 -- start start 2023-11-09T03:45:24.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.676+0000 7f1504dfb640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f150006a830 0x7f15000ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.676+0000 7f1504dfb640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1500160990 con 0x7f15000b1a40 2023-11-09T03:45:24.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.676+0000 7f1504dfb640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15000cc7c0 con 0x7f15001605b0 2023-11-09T03:45:24.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.676+0000 7f1504dfb640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15000cd750 con 0x7f150006a830 2023-11-09T03:45:24.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.676+0000 7f14f7fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f15000b1a40 0x7f15000cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:60066/0 (socket says 172.21.15.76:60066) 2023-11-09T03:45:24.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.676+0000 7f14f7fff640 1 -- 172.21.15.76:0/219474545 learned_addr learned my addr 172.21.15.76:0/219474545 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:24.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.676+0000 7f14fed9d640 1 --2- 172.21.15.76:0/219474545 >> v2:172.21.15.113:3300/0 conn(0x7f150006a830 0x7f15000ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.676+0000 7f14fdd9b640 1 -- 172.21.15.76:0/219474545 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3142601394 0 0) 0x7f1500160990 con 0x7f15000b1a40 2023-11-09T03:45:24.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.676+0000 7f14fdd9b640 1 -- 172.21.15.76:0/219474545 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14ec003520 con 0x7f15000b1a40 2023-11-09T03:45:24.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.676+0000 7f14fdd9b640 1 -- 172.21.15.76:0/219474545 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4091133016 0 0) 0x7f14ec003520 con 0x7f15000b1a40 2023-11-09T03:45:24.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.676+0000 7f14fdd9b640 1 -- 172.21.15.76:0/219474545 >> v2:172.21.15.113:3300/0 conn(0x7f150006a830 msgr2=0x7f15000ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:24.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.676+0000 7f14fdd9b640 1 --2- 172.21.15.76:0/219474545 >> v2:172.21.15.113:3300/0 conn(0x7f150006a830 0x7f15000ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.676+0000 7f14fdd9b640 1 -- 172.21.15.76:0/219474545 >> v1:172.21.15.113:6789/0 conn(0x7f15001605b0 legacy=0x7f150006a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:24.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.676+0000 7f14fdd9b640 1 -- 172.21.15.76:0/219474545 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15000cc4e0 con 0x7f15000b1a40 2023-11-09T03:45:24.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.676+0000 7f14fdd9b640 1 -- 172.21.15.76:0/219474545 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f14e8001670 con 0x7f15000b1a40 2023-11-09T03:45:24.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.676+0000 7f14fed9d640 1 --2- 172.21.15.76:0/219474545 >> v2:172.21.15.113:3300/0 conn(0x7f150006a830 0x7f15000ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:45:24.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.676+0000 7f14fdd9b640 1 -- 172.21.15.76:0/219474545 >> v1:172.21.15.76:6789/0 conn(0x7f15000b1a40 legacy=0x7f15000cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:24.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.676+0000 7f14fdd9b640 1 --2- 172.21.15.76:0/219474545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14ec003970 0x7f14ec003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.676+0000 7f14fdd9b640 1 --2- 172.21.15.76:0/219474545 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14ec0047c0 0x7f14ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.676+0000 7f14fdd9b640 1 --2- 172.21.15.76:0/219474545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14ec005180 0x7f14ec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.676+0000 7f14fdd9b640 1 -- 172.21.15.76:0/219474545 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15000cd750 con 0x7f14ec005180 2023-11-09T03:45:24.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.676+0000 7f14fdd9b640 1 -- 172.21.15.76:0/219474545 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec00c0d0 con 0x7f14ec003970 2023-11-09T03:45:24.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.676+0000 7f14fdd9b640 1 -- 172.21.15.76:0/219474545 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec00c290 con 0x7f14ec0047c0 2023-11-09T03:45:24.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.676+0000 7f14fed9d640 1 --2- 172.21.15.76:0/219474545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14ec005180 0x7f14ec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.676+0000 7f14fe59c640 1 --2- 172.21.15.76:0/219474545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14ec003970 0x7f14ec003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.676+0000 7f14f7fff640 1 --2- 172.21.15.76:0/219474545 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14ec0047c0 0x7f14ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.677+0000 7f14fed9d640 1 -- 172.21.15.76:0/219474545 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14ec0047c0 msgr2=0x7f14ec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:24.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.677+0000 7f14fed9d640 1 --2- 172.21.15.76:0/219474545 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14ec0047c0 0x7f14ec004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.677+0000 7f14fed9d640 1 -- 172.21.15.76:0/219474545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14ec003970 msgr2=0x7f14ec003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:24.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.677+0000 7f14fed9d640 1 --2- 172.21.15.76:0/219474545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14ec003970 0x7f14ec003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.677+0000 7f14fed9d640 1 -- 172.21.15.76:0/219474545 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ec003520 con 0x7f14ec005180 2023-11-09T03:45:24.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.677+0000 7f14fed9d640 1 -- 172.21.15.76:0/219474545 reap_dead start 2023-11-09T03:45:24.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.677+0000 7f14fed9d640 1 --2- 172.21.15.76:0/219474545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14ec005180 0x7f14ec00b980 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f14f800e350 tx=0x7f14ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15bb78ce3702ef23 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:24.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.677+0000 7f14fdd9b640 1 -- 172.21.15.76:0/219474545 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f802ce00 con 0x7f14ec005180 2023-11-09T03:45:24.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.677+0000 7f14fdd9b640 1 -- 172.21.15.76:0/219474545 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f800f040 con 0x7f14ec005180 2023-11-09T03:45:24.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.677+0000 7f14fdd9b640 1 -- 172.21.15.76:0/219474545 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f8003bd0 con 0x7f14ec005180 2023-11-09T03:45:24.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.678+0000 7f1504dfb640 1 -- 172.21.15.76:0/219474545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14ec005180 msgr2=0x7f14ec00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:24.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.678+0000 7f1504dfb640 1 --2- 172.21.15.76:0/219474545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14ec005180 0x7f14ec00b980 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f14f800e350 tx=0x7f14ec0057f0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.678+0000 7f1504dfb640 1 -- 172.21.15.76:0/219474545 shutdown_connections 2023-11-09T03:45:24.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.678+0000 7f1504dfb640 1 --2- 172.21.15.76:0/219474545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14ec005180 0x7f14ec00b980 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.678+0000 7f1504dfb640 1 -- 172.21.15.76:0/219474545 >> 172.21.15.76:0/219474545 conn(0x7f1500071820 msgr2=0x7f1500071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:24.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.678+0000 7f1504dfb640 1 -- 172.21.15.76:0/219474545 shutdown_connections 2023-11-09T03:45:24.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.678+0000 7f1504dfb640 1 -- 172.21.15.76:0/219474545 wait complete. 2023-11-09T03:45:24.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.679+0000 7f1504dfb640 1 Processor -- start 2023-11-09T03:45:24.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.679+0000 7f1504dfb640 1 -- start start 2023-11-09T03:45:24.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.680+0000 7f1504dfb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14ec005180 0x7f15000ac500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.680+0000 7f1504dfb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15001f2160 0x7f15000b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.680+0000 7f1504dfb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15000ca560 0x7f15000ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.680+0000 7f1504dfb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15000caf20 con 0x7f14ec005180 2023-11-09T03:45:24.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.680+0000 7f1504dfb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15000b36e0 con 0x7f15000ca560 2023-11-09T03:45:24.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.680+0000 7f1504dfb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15000b3b90 con 0x7f15001f2160 2023-11-09T03:45:24.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.680+0000 7f14fe59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14ec005180 0x7f15000ac500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.680+0000 7f14fe59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14ec005180 0x7f15000ac500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54368/0 (socket says 172.21.15.76:54368) 2023-11-09T03:45:24.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.680+0000 7f14fe59c640 1 -- 172.21.15.76:0/999576613 learned_addr learned my addr 172.21.15.76:0/999576613 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:24.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.680+0000 7f14fed9d640 1 --2- 172.21.15.76:0/999576613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15000ca560 0x7f15000ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.680+0000 7f14f7fff640 1 --2- 172.21.15.76:0/999576613 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15001f2160 0x7f15000b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.680+0000 7f14fe59c640 1 -- 172.21.15.76:0/999576613 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15001f2160 msgr2=0x7f15000b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:24.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.680+0000 7f14fe59c640 1 --2- 172.21.15.76:0/999576613 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15001f2160 0x7f15000b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.680+0000 7f14fe59c640 1 -- 172.21.15.76:0/999576613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15000ca560 msgr2=0x7f15000ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:24.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.680+0000 7f14fe59c640 1 --2- 172.21.15.76:0/999576613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15000ca560 0x7f15000ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.680+0000 7f14fe59c640 1 -- 172.21.15.76:0/999576613 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15000acc00 con 0x7f14ec005180 2023-11-09T03:45:24.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.681+0000 7f14fe59c640 1 --2- 172.21.15.76:0/999576613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14ec005180 0x7f15000ac500 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f14f000f820 tx=0x7f14f000fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:24.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.681+0000 7f14f6ffd640 1 -- 172.21.15.76:0/999576613 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f00050d0 con 0x7f14ec005180 2023-11-09T03:45:24.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.681+0000 7f14f6ffd640 1 -- 172.21.15.76:0/999576613 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f0005250 con 0x7f14ec005180 2023-11-09T03:45:24.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.681+0000 7f1504dfb640 1 -- 172.21.15.76:0/999576613 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15000acee0 con 0x7f14ec005180 2023-11-09T03:45:24.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.681+0000 7f14f6ffd640 1 -- 172.21.15.76:0/999576613 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f000d6d0 con 0x7f14ec005180 2023-11-09T03:45:24.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.681+0000 7f1504dfb640 1 -- 172.21.15.76:0/999576613 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15000b40c0 con 0x7f14ec005180 2023-11-09T03:45:24.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.682+0000 7f1504dfb640 1 -- 172.21.15.76:0/999576613 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f150015ba80 con 0x7f14ec005180 2023-11-09T03:45:24.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.682+0000 7f14f6ffd640 1 -- 172.21.15.76:0/999576613 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f14f000d850 con 0x7f14ec005180 2023-11-09T03:45:24.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.683+0000 7f14f6ffd640 1 --2- 172.21.15.76:0/999576613 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f14e006aea0 0x7f14e006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.683+0000 7f14f6ffd640 1 -- 172.21.15.76:0/999576613 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f14f00890d0 con 0x7f14ec005180 2023-11-09T03:45:24.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.683+0000 7f14f7fff640 1 --2- 172.21.15.76:0/999576613 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f14e006aea0 0x7f14e006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.683+0000 7f14f7fff640 1 --2- 172.21.15.76:0/999576613 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f14e006aea0 0x7f14e006d360 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f15000b3b20 tx=0x7f14e8000ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:24.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.687+0000 7f14f6ffd640 1 -- 172.21.15.76:0/999576613 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f14f00544d0 con 0x7f14ec005180 2023-11-09T03:45:24.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.853+0000 7f1504dfb640 1 -- 172.21.15.76:0/999576613 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f15000672a0 con 0x7f14ec005180 2023-11-09T03:45:24.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.854+0000 7f14f6ffd640 1 -- 172.21.15.76:0/999576613 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f14f0057af0 con 0x7f14ec005180 2023-11-09T03:45:24.854 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:45:24.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.856+0000 7f1504dfb640 1 -- 172.21.15.76:0/999576613 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f14e006aea0 msgr2=0x7f14e006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:24.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.856+0000 7f1504dfb640 1 --2- 172.21.15.76:0/999576613 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f14e006aea0 0x7f14e006d360 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f15000b3b20 tx=0x7f14e8000ed0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.856+0000 7f1504dfb640 1 -- 172.21.15.76:0/999576613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14ec005180 msgr2=0x7f15000ac500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:24.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.856+0000 7f1504dfb640 1 --2- 172.21.15.76:0/999576613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14ec005180 0x7f15000ac500 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f14f000f820 tx=0x7f14f000fc20 comp rx=0 tx=0).stop 2023-11-09T03:45:24.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.856+0000 7f1504dfb640 1 -- 172.21.15.76:0/999576613 shutdown_connections 2023-11-09T03:45:24.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.856+0000 7f1504dfb640 1 --2- 172.21.15.76:0/999576613 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f14e006aea0 0x7f14e006d360 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.856+0000 7f1504dfb640 1 --2- 172.21.15.76:0/999576613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15000ca560 0x7f15000ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.856+0000 7f1504dfb640 1 --2- 172.21.15.76:0/999576613 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15001f2160 0x7f15000b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.856+0000 7f1504dfb640 1 --2- 172.21.15.76:0/999576613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14ec005180 0x7f15000ac500 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.856+0000 7f1504dfb640 1 -- 172.21.15.76:0/999576613 >> 172.21.15.76:0/999576613 conn(0x7f1500071820 msgr2=0x7f15000ccc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:24.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.856+0000 7f1504dfb640 1 -- 172.21.15.76:0/999576613 shutdown_connections 2023-11-09T03:45:24.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.856+0000 7f1504dfb640 1 -- 172.21.15.76:0/999576613 wait complete. 2023-11-09T03:45:24.865 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:45:24.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.967+0000 7fe85967c640 1 Processor -- start 2023-11-09T03:45:24.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.967+0000 7fe85967c640 1 -- start start 2023-11-09T03:45:24.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.968+0000 7fe85967c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe85415ae40 0x7fe85415bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.968+0000 7fe85967c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8540cc450 con 0x7fe8540cbc00 2023-11-09T03:45:24.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.968+0000 7fe85967c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe85415f400 con 0x7fe8540cda30 2023-11-09T03:45:24.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.968+0000 7fe85967c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85415b220 con 0x7fe85415ae40 2023-11-09T03:45:24.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.968+0000 7fe8527fc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe8540cbc00 0x7fe8540cd320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:60070/0 (socket says 172.21.15.76:60070) 2023-11-09T03:45:24.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.968+0000 7fe8527fc640 1 -- 172.21.15.76:0/2409396087 learned_addr learned my addr 172.21.15.76:0/2409396087 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:24.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.968+0000 7fe852ffd640 1 --2- 172.21.15.76:0/2409396087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe85415ae40 0x7fe85415bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.968+0000 7fe851ffb640 1 -- 172.21.15.76:0/2409396087 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1263808502 0 0) 0x7fe848001110 con 0x7fe8540cbc00 2023-11-09T03:45:24.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.968+0000 7fe851ffb640 1 -- 172.21.15.76:0/2409396087 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe83c003520 con 0x7fe8540cbc00 2023-11-09T03:45:24.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.969+0000 7fe851ffb640 1 -- 172.21.15.76:0/2409396087 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1768744478 0 0) 0x7fe85415f400 con 0x7fe8540cda30 2023-11-09T03:45:24.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.969+0000 7fe851ffb640 1 -- 172.21.15.76:0/2409396087 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe848001110 con 0x7fe8540cda30 2023-11-09T03:45:24.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.969+0000 7fe851ffb640 1 -- 172.21.15.76:0/2409396087 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3869090686 0 0) 0x7fe83c003520 con 0x7fe8540cbc00 2023-11-09T03:45:24.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.969+0000 7fe851ffb640 1 -- 172.21.15.76:0/2409396087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe85415ae40 msgr2=0x7fe85415bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:24.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.969+0000 7fe851ffb640 1 --2- 172.21.15.76:0/2409396087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe85415ae40 0x7fe85415bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.969+0000 7fe851ffb640 1 -- 172.21.15.76:0/2409396087 >> v1:172.21.15.113:6789/0 conn(0x7fe8540cda30 legacy=0x7fe8540ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:24.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.969+0000 7fe851ffb640 1 -- 172.21.15.76:0/2409396087 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85415f890 con 0x7fe8540cbc00 2023-11-09T03:45:24.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.969+0000 7fe851ffb640 1 -- 172.21.15.76:0/2409396087 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fe848001800 con 0x7fe8540cbc00 2023-11-09T03:45:24.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.969+0000 7fe851ffb640 1 -- 172.21.15.76:0/2409396087 >> v1:172.21.15.76:6789/0 conn(0x7fe8540cbc00 legacy=0x7fe8540cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:24.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.969+0000 7fe851ffb640 1 --2- 172.21.15.76:0/2409396087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe85415ae40 0x7fe83c003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.969+0000 7fe851ffb640 1 --2- 172.21.15.76:0/2409396087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe83c004090 0x7fe83c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.969+0000 7fe851ffb640 1 --2- 172.21.15.76:0/2409396087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe83c004d00 0x7fe83c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.969+0000 7fe851ffb640 1 -- 172.21.15.76:0/2409396087 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85415b220 con 0x7fe83c004090 2023-11-09T03:45:24.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.969+0000 7fe851ffb640 1 -- 172.21.15.76:0/2409396087 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c00ba60 con 0x7fe83c004d00 2023-11-09T03:45:24.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.969+0000 7fe851ffb640 1 -- 172.21.15.76:0/2409396087 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c00bc20 con 0x7fe85415ae40 2023-11-09T03:45:24.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.969+0000 7fe8527fc640 1 --2- 172.21.15.76:0/2409396087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe83c004090 0x7fe83c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.969+0000 7fe852ffd640 1 --2- 172.21.15.76:0/2409396087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe85415ae40 0x7fe83c003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.969+0000 7fe8537fe640 1 --2- 172.21.15.76:0/2409396087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe83c004d00 0x7fe83c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.969+0000 7fe8527fc640 1 -- 172.21.15.76:0/2409396087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe85415ae40 msgr2=0x7fe83c003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:24.981 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.969+0000 7fe8527fc640 1 --2- 172.21.15.76:0/2409396087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe85415ae40 0x7fe83c003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.981 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.969+0000 7fe8527fc640 1 -- 172.21.15.76:0/2409396087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe83c004d00 msgr2=0x7fe83c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:24.981 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.969+0000 7fe8527fc640 1 --2- 172.21.15.76:0/2409396087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe83c004d00 0x7fe83c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.981 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.969+0000 7fe8527fc640 1 -- 172.21.15.76:0/2409396087 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83c003520 con 0x7fe83c004090 2023-11-09T03:45:24.981 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.969+0000 7fe8537fe640 1 --2- 172.21.15.76:0/2409396087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe83c004d00 0x7fe83c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:24.981 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.970+0000 7fe8527fc640 1 --2- 172.21.15.76:0/2409396087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe83c004090 0x7fe83c0047c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe84800c160 tx=0x7fe83c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e304fdf730e39d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:24.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.970+0000 7fe851ffb640 1 -- 172.21.15.76:0/2409396087 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe848005ad0 con 0x7fe83c004090 2023-11-09T03:45:24.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.970+0000 7fe851ffb640 1 -- 172.21.15.76:0/2409396087 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe848005c50 con 0x7fe83c004090 2023-11-09T03:45:24.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.970+0000 7fe851ffb640 1 -- 172.21.15.76:0/2409396087 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84802c810 con 0x7fe83c004090 2023-11-09T03:45:24.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.970+0000 7fe85967c640 1 -- 172.21.15.76:0/2409396087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe83c004090 msgr2=0x7fe83c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:24.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.970+0000 7fe85967c640 1 --2- 172.21.15.76:0/2409396087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe83c004090 0x7fe83c0047c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe84800c160 tx=0x7fe83c0057f0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.970+0000 7fe8537fe640 1 -- 172.21.15.76:0/2409396087 reap_dead start 2023-11-09T03:45:24.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.970+0000 7fe85967c640 1 -- 172.21.15.76:0/2409396087 shutdown_connections 2023-11-09T03:45:24.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.970+0000 7fe85967c640 1 -- 172.21.15.76:0/2409396087 >> 172.21.15.76:0/2409396087 conn(0x7fe85406ea90 msgr2=0x7fe85406ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:24.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.970+0000 7fe85967c640 1 -- 172.21.15.76:0/2409396087 shutdown_connections 2023-11-09T03:45:24.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.970+0000 7fe85967c640 1 -- 172.21.15.76:0/2409396087 wait complete. 2023-11-09T03:45:24.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.972+0000 7fe85967c640 1 Processor -- start 2023-11-09T03:45:24.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.972+0000 7fe85967c640 1 -- start start 2023-11-09T03:45:24.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.972+0000 7fe85967c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe85414e9a0 0x7fe8540ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.972+0000 7fe85967c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8540c7c70 0x7fe8540c62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.972+0000 7fe85967c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8540c67e0 0x7fe8540c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.972+0000 7fe85967c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8540c7550 con 0x7fe8540c67e0 2023-11-09T03:45:24.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.972+0000 7fe85967c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8540c7060 con 0x7fe85414e9a0 2023-11-09T03:45:24.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.972+0000 7fe85967c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8540c58b0 con 0x7fe8540c7c70 2023-11-09T03:45:24.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.972+0000 7fe8537fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8540c67e0 0x7fe8540c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.973+0000 7fe8537fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8540c67e0 0x7fe8540c95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54390/0 (socket says 172.21.15.76:54390) 2023-11-09T03:45:24.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.973+0000 7fe8537fe640 1 -- 172.21.15.76:0/170629125 learned_addr learned my addr 172.21.15.76:0/170629125 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:24.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.973+0000 7fe852ffd640 1 --2- 172.21.15.76:0/170629125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe85414e9a0 0x7fe8540ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.973+0000 7fe8527fc640 1 --2- 172.21.15.76:0/170629125 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8540c7c70 0x7fe8540c62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.973+0000 7fe8537fe640 1 -- 172.21.15.76:0/170629125 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8540c7c70 msgr2=0x7fe8540c62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:24.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.973+0000 7fe8537fe640 1 --2- 172.21.15.76:0/170629125 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8540c7c70 0x7fe8540c62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.973+0000 7fe8537fe640 1 -- 172.21.15.76:0/170629125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe85414e9a0 msgr2=0x7fe8540ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:24.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.973+0000 7fe8537fe640 1 --2- 172.21.15.76:0/170629125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe85414e9a0 0x7fe8540ca770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:24.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.973+0000 7fe8537fe640 1 -- 172.21.15.76:0/170629125 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8540c6c60 con 0x7fe8540c67e0 2023-11-09T03:45:24.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.973+0000 7fe8527fc640 1 --2- 172.21.15.76:0/170629125 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8540c7c70 0x7fe8540c62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:24.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.973+0000 7fe8537fe640 1 --2- 172.21.15.76:0/170629125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8540c67e0 0x7fe8540c95d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe85404f530 tx=0x7fe84c00e7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:24.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.973+0000 7fe83b7fe640 1 -- 172.21.15.76:0/170629125 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c0052e0 con 0x7fe8540c67e0 2023-11-09T03:45:24.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.973+0000 7fe83b7fe640 1 -- 172.21.15.76:0/170629125 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84c005480 con 0x7fe8540c67e0 2023-11-09T03:45:24.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.973+0000 7fe83b7fe640 1 -- 172.21.15.76:0/170629125 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c007270 con 0x7fe8540c67e0 2023-11-09T03:45:24.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.973+0000 7fe85967c640 1 -- 172.21.15.76:0/170629125 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe85414ed80 con 0x7fe8540c67e0 2023-11-09T03:45:24.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.973+0000 7fe85967c640 1 -- 172.21.15.76:0/170629125 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8540c5260 con 0x7fe8540c67e0 2023-11-09T03:45:24.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.974+0000 7fe85967c640 1 -- 172.21.15.76:0/170629125 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8540c50c0 con 0x7fe8540c67e0 2023-11-09T03:45:24.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.974+0000 7fe83b7fe640 1 -- 172.21.15.76:0/170629125 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe84c0073d0 con 0x7fe8540c67e0 2023-11-09T03:45:24.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.975+0000 7fe83b7fe640 1 --2- 172.21.15.76:0/170629125 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe84006aea0 0x7fe84006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:24.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.975+0000 7fe83b7fe640 1 -- 172.21.15.76:0/170629125 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe84c088000 con 0x7fe8540c67e0 2023-11-09T03:45:24.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.975+0000 7fe852ffd640 1 --2- 172.21.15.76:0/170629125 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe84006aea0 0x7fe84006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:24.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.976+0000 7fe852ffd640 1 --2- 172.21.15.76:0/170629125 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe84006aea0 0x7fe84006d360 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe844014430 tx=0x7fe84400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:24.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:24.979+0000 7fe83b7fe640 1 -- 172.21.15.76:0/170629125 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe8540c50c0 con 0x7fe8540c67e0 2023-11-09T03:45:25.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:25.179+0000 7fe85967c640 1 -- 172.21.15.76:0/170629125 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8540c50c0 con 0x7fe8540c67e0 2023-11-09T03:45:25.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:25.179+0000 7fe83b7fe640 1 -- 172.21.15.76:0/170629125 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7fe8540c50c0 con 0x7fe8540c67e0 2023-11-09T03:45:25.179 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:25.179 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2686298435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2686298435},{"type":"v1","addr":"172.21.15.76:6833","nonce":2686298435}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-11-09T03:45:25.179 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 2 2023-11-09T03:45:25.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:25.181+0000 7fe85967c640 1 -- 172.21.15.76:0/170629125 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe84006aea0 msgr2=0x7fe84006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:25.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:25.181+0000 7fe85967c640 1 --2- 172.21.15.76:0/170629125 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe84006aea0 0x7fe84006d360 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe844014430 tx=0x7fe84400b040 comp rx=0 tx=0).stop 2023-11-09T03:45:25.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:25.181+0000 7fe85967c640 1 -- 172.21.15.76:0/170629125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8540c67e0 msgr2=0x7fe8540c95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:25.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:25.181+0000 7fe85967c640 1 --2- 172.21.15.76:0/170629125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8540c67e0 0x7fe8540c95d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe85404f530 tx=0x7fe84c00e7d0 comp rx=0 tx=0).stop 2023-11-09T03:45:25.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:25.181+0000 7fe85967c640 1 -- 172.21.15.76:0/170629125 shutdown_connections 2023-11-09T03:45:25.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:25.181+0000 7fe85967c640 1 --2- 172.21.15.76:0/170629125 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe84006aea0 0x7fe84006d360 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:25.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:25.181+0000 7fe85967c640 1 --2- 172.21.15.76:0/170629125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8540c67e0 0x7fe8540c95d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:25.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:25.181+0000 7fe85967c640 1 --2- 172.21.15.76:0/170629125 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8540c7c70 0x7fe8540c62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:25.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:25.181+0000 7fe85967c640 1 --2- 172.21.15.76:0/170629125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe85414e9a0 0x7fe8540ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:25.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:25.181+0000 7fe85967c640 1 -- 172.21.15.76:0/170629125 >> 172.21.15.76:0/170629125 conn(0x7fe85406ea90 msgr2=0x7fe8540ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:25.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:25.181+0000 7fe85967c640 1 -- 172.21.15.76:0/170629125 shutdown_connections 2023-11-09T03:45:25.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:25.181+0000 7fe85967c640 1 -- 172.21.15.76:0/170629125 wait complete. 2023-11-09T03:45:27.192 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:45:27.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.300+0000 7ff862d3c640 1 Processor -- start 2023-11-09T03:45:27.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.300+0000 7ff862d3c640 1 -- start start 2023-11-09T03:45:27.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.301+0000 7ff862d3c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff85c159c70 0x7ff85c0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:27.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.301+0000 7ff862d3c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff85c06db20 0x7ff85c0b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:27.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.301+0000 7ff862d3c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff85c0b4110 con 0x7ff85c0b0910 2023-11-09T03:45:27.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.301+0000 7ff862d3c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff85c0b49b0 con 0x7ff85c159c70 2023-11-09T03:45:27.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.301+0000 7ff862d3c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c0b0da0 con 0x7ff85c06db20 2023-11-09T03:45:27.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.301+0000 7ff860ad8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff85c159c70 0x7ff85c0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:27.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.301+0000 7ff8612d9640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff85c0b0910 0x7ff85c15d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:60080/0 (socket says 172.21.15.76:60080) 2023-11-09T03:45:27.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.301+0000 7ff8612d9640 1 -- 172.21.15.76:0/1356310065 learned_addr learned my addr 172.21.15.76:0/1356310065 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:27.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.301+0000 7ff860ad8640 1 --2- 172.21.15.76:0/1356310065 >> v2:172.21.15.76:3300/0 conn(0x7ff85c159c70 0x7ff85c0ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54398/0 (socket says 172.21.15.76:54398) 2023-11-09T03:45:27.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.301+0000 7ff85bfff640 1 --2- 172.21.15.76:0/1356310065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff85c06db20 0x7ff85c0b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:27.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.302+0000 7ff860ad8640 1 -- 172.21.15.76:0/1356310065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff85c06db20 msgr2=0x7ff85c0b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:27.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.302+0000 7ff860ad8640 1 --2- 172.21.15.76:0/1356310065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff85c06db20 0x7ff85c0b03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:27.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.302+0000 7ff860ad8640 1 -- 172.21.15.76:0/1356310065 >> v1:172.21.15.76:6789/0 conn(0x7ff85c0b0910 legacy=0x7ff85c15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:27.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.302+0000 7ff860ad8640 1 -- 172.21.15.76:0/1356310065 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff85c0b43f0 con 0x7ff85c159c70 2023-11-09T03:45:27.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.302+0000 7ff85b7fe640 1 -- 172.21.15.76:0/1356310065 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3669030783 0 0) 0x7ff85c0b4110 con 0x7ff85c0b0910 2023-11-09T03:45:27.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.302+0000 7ff860ad8640 1 --2- 172.21.15.76:0/1356310065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff85c159c70 0x7ff85c0ce370 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff84c0097a0 tx=0x7ff84c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46021f4f52a75f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:27.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.302+0000 7ff85b7fe640 1 -- 172.21.15.76:0/1356310065 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c031830 con 0x7ff85c159c70 2023-11-09T03:45:27.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.302+0000 7ff85b7fe640 1 -- 172.21.15.76:0/1356310065 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff84c031990 con 0x7ff85c159c70 2023-11-09T03:45:27.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.302+0000 7ff85b7fe640 1 -- 172.21.15.76:0/1356310065 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c031d20 con 0x7ff85c159c70 2023-11-09T03:45:27.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.303+0000 7ff862d3c640 1 -- 172.21.15.76:0/1356310065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff85c159c70 msgr2=0x7ff85c0ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:27.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.303+0000 7ff862d3c640 1 --2- 172.21.15.76:0/1356310065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff85c159c70 0x7ff85c0ce370 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff84c0097a0 tx=0x7ff84c02b8d0 comp rx=0 tx=0).stop 2023-11-09T03:45:27.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.303+0000 7ff862d3c640 1 -- 172.21.15.76:0/1356310065 shutdown_connections 2023-11-09T03:45:27.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.303+0000 7ff862d3c640 1 --2- 172.21.15.76:0/1356310065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff85c06db20 0x7ff85c0b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:27.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.303+0000 7ff862d3c640 1 --2- 172.21.15.76:0/1356310065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff85c159c70 0x7ff85c0ce370 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:27.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.303+0000 7ff862d3c640 1 -- 172.21.15.76:0/1356310065 >> 172.21.15.76:0/1356310065 conn(0x7ff85c0ce950 msgr2=0x7ff85c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:27.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.303+0000 7ff862d3c640 1 -- 172.21.15.76:0/1356310065 shutdown_connections 2023-11-09T03:45:27.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.303+0000 7ff862d3c640 1 -- 172.21.15.76:0/1356310065 wait complete. 2023-11-09T03:45:27.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.304+0000 7ff862d3c640 1 Processor -- start 2023-11-09T03:45:27.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.304+0000 7ff862d3c640 1 -- start start 2023-11-09T03:45:27.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.305+0000 7ff862d3c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff85c06db20 0x7ff85c155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:27.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.305+0000 7ff862d3c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff85c0b0910 0x7ff85c1512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:27.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.305+0000 7ff862d3c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff85c159c70 0x7ff85c1506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:27.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.305+0000 7ff862d3c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c1502c0 con 0x7ff85c0b0910 2023-11-09T03:45:27.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.305+0000 7ff862d3c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c151c40 con 0x7ff85c159c70 2023-11-09T03:45:27.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.305+0000 7ff862d3c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c14fc00 con 0x7ff85c06db20 2023-11-09T03:45:27.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.305+0000 7ff85bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff85c0b0910 0x7ff85c1512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:27.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.305+0000 7ff85bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff85c0b0910 0x7ff85c1512f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54408/0 (socket says 172.21.15.76:54408) 2023-11-09T03:45:27.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.305+0000 7ff85bfff640 1 -- 172.21.15.76:0/4091141340 learned_addr learned my addr 172.21.15.76:0/4091141340 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:27.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.305+0000 7ff860ad8640 1 --2- 172.21.15.76:0/4091141340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff85c06db20 0x7ff85c155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:27.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.305+0000 7ff8612d9640 1 --2- 172.21.15.76:0/4091141340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff85c159c70 0x7ff85c1506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:27.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.305+0000 7ff85bfff640 1 -- 172.21.15.76:0/4091141340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff85c06db20 msgr2=0x7ff85c155330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:27.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.305+0000 7ff85bfff640 1 --2- 172.21.15.76:0/4091141340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff85c06db20 0x7ff85c155330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:27.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.305+0000 7ff85bfff640 1 -- 172.21.15.76:0/4091141340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff85c159c70 msgr2=0x7ff85c1506e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:27.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.305+0000 7ff85bfff640 1 --2- 172.21.15.76:0/4091141340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff85c159c70 0x7ff85c1506e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:27.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.305+0000 7ff85bfff640 1 -- 172.21.15.76:0/4091141340 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff84c009400 con 0x7ff85c0b0910 2023-11-09T03:45:27.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.305+0000 7ff8612d9640 1 --2- 172.21.15.76:0/4091141340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff85c159c70 0x7ff85c1506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:27.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.306+0000 7ff85bfff640 1 --2- 172.21.15.76:0/4091141340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff85c0b0910 0x7ff85c1512f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff85000d5f0 tx=0x7ff85000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:27.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.306+0000 7ff8597fa640 1 -- 172.21.15.76:0/4091141340 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8500125f0 con 0x7ff85c0b0910 2023-11-09T03:45:27.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.306+0000 7ff8597fa640 1 -- 172.21.15.76:0/4091141340 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff850012750 con 0x7ff85c0b0910 2023-11-09T03:45:27.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.306+0000 7ff8597fa640 1 -- 172.21.15.76:0/4091141340 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff850012b00 con 0x7ff85c0b0910 2023-11-09T03:45:27.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.306+0000 7ff862d3c640 1 -- 172.21.15.76:0/4091141340 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff85c14fee0 con 0x7ff85c0b0910 2023-11-09T03:45:27.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.306+0000 7ff862d3c640 1 -- 172.21.15.76:0/4091141340 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff85c151830 con 0x7ff85c0b0910 2023-11-09T03:45:27.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.307+0000 7ff862d3c640 1 -- 172.21.15.76:0/4091141340 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff85c150530 con 0x7ff85c0b0910 2023-11-09T03:45:27.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.307+0000 7ff8597fa640 1 -- 172.21.15.76:0/4091141340 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff850012c60 con 0x7ff85c0b0910 2023-11-09T03:45:27.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.307+0000 7ff8597fa640 1 --2- 172.21.15.76:0/4091141340 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff84806ae70 0x7ff84806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:27.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.308+0000 7ff8597fa640 1 -- 172.21.15.76:0/4091141340 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff850089700 con 0x7ff85c0b0910 2023-11-09T03:45:27.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.308+0000 7ff860ad8640 1 --2- 172.21.15.76:0/4091141340 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff84806ae70 0x7ff84806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:27.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.308+0000 7ff860ad8640 1 --2- 172.21.15.76:0/4091141340 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff84806ae70 0x7ff84806d330 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff84c036000 tx=0x7ff84c03c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:27.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.312+0000 7ff8597fa640 1 -- 172.21.15.76:0/4091141340 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff850054b90 con 0x7ff85c0b0910 2023-11-09T03:45:27.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.513+0000 7ff862d3c640 1 -- 172.21.15.76:0/4091141340 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff85c14fa60 con 0x7ff85c0b0910 2023-11-09T03:45:27.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.513+0000 7ff8597fa640 1 -- 172.21.15.76:0/4091141340 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3700 (secure 0 0 0) 0x7ff85c14fa60 con 0x7ff85c0b0910 2023-11-09T03:45:27.514 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:27.514 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2686298435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2686298435},{"type":"v1","addr":"172.21.15.76:6833","nonce":2686298435}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2132717123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2132717123},{"type":"v1","addr":"172.21.15.76:6835","nonce":2132717123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4281,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2369499936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2369499936},{"type":"v1","addr":"172.21.15.113:6837","nonce":2369499936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/3293223710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3293223710},{"type":"v1","addr":"172.21.15.113:6836","nonce":3293223710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-11-09T03:45:27.515 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 3 2023-11-09T03:45:27.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.515+0000 7ff862d3c640 1 -- 172.21.15.76:0/4091141340 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff84806ae70 msgr2=0x7ff84806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:27.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.515+0000 7ff862d3c640 1 --2- 172.21.15.76:0/4091141340 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff84806ae70 0x7ff84806d330 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff84c036000 tx=0x7ff84c03c040 comp rx=0 tx=0).stop 2023-11-09T03:45:27.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.515+0000 7ff862d3c640 1 -- 172.21.15.76:0/4091141340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff85c0b0910 msgr2=0x7ff85c1512f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:27.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.515+0000 7ff862d3c640 1 --2- 172.21.15.76:0/4091141340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff85c0b0910 0x7ff85c1512f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff85000d5f0 tx=0x7ff85000dab0 comp rx=0 tx=0).stop 2023-11-09T03:45:27.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.515+0000 7ff862d3c640 1 -- 172.21.15.76:0/4091141340 shutdown_connections 2023-11-09T03:45:27.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.515+0000 7ff862d3c640 1 --2- 172.21.15.76:0/4091141340 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff84806ae70 0x7ff84806d330 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:27.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.515+0000 7ff862d3c640 1 --2- 172.21.15.76:0/4091141340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff85c159c70 0x7ff85c1506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:27.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.515+0000 7ff862d3c640 1 --2- 172.21.15.76:0/4091141340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff85c0b0910 0x7ff85c1512f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:27.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.515+0000 7ff862d3c640 1 --2- 172.21.15.76:0/4091141340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff85c06db20 0x7ff85c155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:27.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.515+0000 7ff862d3c640 1 -- 172.21.15.76:0/4091141340 >> 172.21.15.76:0/4091141340 conn(0x7ff85c0ce950 msgr2=0x7ff85c15c880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:27.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.515+0000 7ff862d3c640 1 -- 172.21.15.76:0/4091141340 shutdown_connections 2023-11-09T03:45:27.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.515+0000 7ff862d3c640 1 -- 172.21.15.76:0/4091141340 wait complete. 2023-11-09T03:45:27.526 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:45:27.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.625+0000 7f1584039640 1 Processor -- start 2023-11-09T03:45:27.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.626+0000 7f1584039640 1 -- start start 2023-11-09T03:45:27.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.626+0000 7f1584039640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f157c15cdf0 0x7f157c0b85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:27.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.626+0000 7f1584039640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f157c0b8b10 0x7f157c0b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:27.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.626+0000 7f1584039640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f157c0b62c0 con 0x7f157c0d0db0 2023-11-09T03:45:27.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.626+0000 7f1584039640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f157c0b65a0 con 0x7f157c15cdf0 2023-11-09T03:45:27.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.626+0000 7f1584039640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f157c0b6880 con 0x7f157c0b8b10 2023-11-09T03:45:27.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.627+0000 7f1581dd5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f157c15cdf0 0x7f157c0b85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:27.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.627+0000 7f1581dd5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f157c15cdf0 0x7f157c0b85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54418/0 (socket says 172.21.15.76:54418) 2023-11-09T03:45:27.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.627+0000 7f1581dd5640 1 -- 172.21.15.76:0/4223321308 learned_addr learned my addr 172.21.15.76:0/4223321308 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:27.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.627+0000 7f1581dd5640 1 -- 172.21.15.76:0/4223321308 >> v2:172.21.15.113:3300/0 conn(0x7f157c0b8b10 msgr2=0x7f157c0b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:27.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.627+0000 7f1581dd5640 1 --2- 172.21.15.76:0/4223321308 >> v2:172.21.15.113:3300/0 conn(0x7f157c0b8b10 0x7f157c0b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:27.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.627+0000 7f1581dd5640 1 -- 172.21.15.76:0/4223321308 >> v1:172.21.15.113:6789/0 conn(0x7f157c0d0db0 legacy=0x7f157c1609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:27.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.627+0000 7f1581dd5640 1 -- 172.21.15.76:0/4223321308 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157c0b6b60 con 0x7f157c15cdf0 2023-11-09T03:45:27.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.628+0000 7f1581dd5640 1 --2- 172.21.15.76:0/4223321308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f157c15cdf0 0x7f157c0b85d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f157c0d3a70 tx=0x7f157402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48ac210d16de45c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:27.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.628+0000 7f1580dd3640 1 -- 172.21.15.76:0/4223321308 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157402cb20 con 0x7f157c15cdf0 2023-11-09T03:45:27.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.628+0000 7f1580dd3640 1 -- 172.21.15.76:0/4223321308 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157402cc80 con 0x7f157c15cdf0 2023-11-09T03:45:27.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.628+0000 7f1580dd3640 1 -- 172.21.15.76:0/4223321308 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15740343f0 con 0x7f157c15cdf0 2023-11-09T03:45:27.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.628+0000 7f1584039640 1 -- 172.21.15.76:0/4223321308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f157c15cdf0 msgr2=0x7f157c0b85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:27.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.628+0000 7f1584039640 1 --2- 172.21.15.76:0/4223321308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f157c15cdf0 0x7f157c0b85d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f157c0d3a70 tx=0x7f157402bf80 comp rx=0 tx=0).stop 2023-11-09T03:45:27.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.628+0000 7f1584039640 1 -- 172.21.15.76:0/4223321308 shutdown_connections 2023-11-09T03:45:27.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.628+0000 7f1584039640 1 --2- 172.21.15.76:0/4223321308 >> v2:172.21.15.113:3300/0 conn(0x7f157c0b8b10 0x7f157c0b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:27.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.628+0000 7f1584039640 1 --2- 172.21.15.76:0/4223321308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f157c15cdf0 0x7f157c0b85d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:27.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.628+0000 7f1584039640 1 -- 172.21.15.76:0/4223321308 >> 172.21.15.76:0/4223321308 conn(0x7f157c071c70 msgr2=0x7f157c072080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:27.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.628+0000 7f1584039640 1 -- 172.21.15.76:0/4223321308 shutdown_connections 2023-11-09T03:45:27.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.628+0000 7f1584039640 1 -- 172.21.15.76:0/4223321308 wait complete. 2023-11-09T03:45:27.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.629+0000 7f1584039640 1 Processor -- start 2023-11-09T03:45:27.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.630+0000 7f1584039640 1 -- start start 2023-11-09T03:45:27.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.630+0000 7f1584039640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f157c0b8b10 0x7f157c0c6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:27.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.630+0000 7f1584039640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f157c0d0db0 0x7f157c1f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:27.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.630+0000 7f1584039640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f157c15cdf0 0x7f157c0c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:27.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.630+0000 7f1584039640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157c0cb8f0 con 0x7f157c0b8b10 2023-11-09T03:45:27.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.630+0000 7f1584039640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157c158d60 con 0x7f157c15cdf0 2023-11-09T03:45:27.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.630+0000 7f1584039640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157c15bab0 con 0x7f157c0d0db0 2023-11-09T03:45:27.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.630+0000 7f1581dd5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f157c0b8b10 0x7f157c0c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:27.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.630+0000 7f1581dd5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f157c0b8b10 0x7f157c0c6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54424/0 (socket says 172.21.15.76:54424) 2023-11-09T03:45:27.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.630+0000 7f1581dd5640 1 -- 172.21.15.76:0/1330680905 learned_addr learned my addr 172.21.15.76:0/1330680905 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:27.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.631+0000 7f15825d6640 1 --2- 172.21.15.76:0/1330680905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f157c15cdf0 0x7f157c0c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:27.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.631+0000 7f1581dd5640 1 -- 172.21.15.76:0/1330680905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f157c0d0db0 msgr2=0x7f157c1f1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:27.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.631+0000 7f15815d4640 1 --2- 172.21.15.76:0/1330680905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f157c0d0db0 0x7f157c1f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:27.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.631+0000 7f1581dd5640 1 --2- 172.21.15.76:0/1330680905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f157c0d0db0 0x7f157c1f1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:27.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.631+0000 7f1581dd5640 1 -- 172.21.15.76:0/1330680905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f157c15cdf0 msgr2=0x7f157c0c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:27.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.631+0000 7f1581dd5640 1 --2- 172.21.15.76:0/1330680905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f157c15cdf0 0x7f157c0c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:27.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.631+0000 7f1581dd5640 1 -- 172.21.15.76:0/1330680905 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1574009b30 con 0x7f157c0b8b10 2023-11-09T03:45:27.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.631+0000 7f15825d6640 1 --2- 172.21.15.76:0/1330680905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f157c15cdf0 0x7f157c0c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:27.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.631+0000 7f15815d4640 1 --2- 172.21.15.76:0/1330680905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f157c0d0db0 0x7f157c1f1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:27.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.631+0000 7f1581dd5640 1 --2- 172.21.15.76:0/1330680905 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f157c0b8b10 0x7f157c0c6970 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1574000f80 tx=0x7f1574004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:27.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.631+0000 7f157a7fc640 1 -- 172.21.15.76:0/1330680905 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1574009510 con 0x7f157c0b8b10 2023-11-09T03:45:27.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.631+0000 7f157a7fc640 1 -- 172.21.15.76:0/1330680905 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157403e450 con 0x7f157c0b8b10 2023-11-09T03:45:27.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.631+0000 7f157a7fc640 1 -- 172.21.15.76:0/1330680905 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157403e800 con 0x7f157c0b8b10 2023-11-09T03:45:27.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.631+0000 7f1584039640 1 -- 172.21.15.76:0/1330680905 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f157c0caf50 con 0x7f157c0b8b10 2023-11-09T03:45:27.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.631+0000 7f1584039640 1 -- 172.21.15.76:0/1330680905 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f157c0cb510 con 0x7f157c0b8b10 2023-11-09T03:45:27.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.632+0000 7f1584039640 1 -- 172.21.15.76:0/1330680905 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f157c067c50 con 0x7f157c0b8b10 2023-11-09T03:45:27.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.632+0000 7f157a7fc640 1 -- 172.21.15.76:0/1330680905 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f157403e960 con 0x7f157c0b8b10 2023-11-09T03:45:27.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.633+0000 7f157a7fc640 1 --2- 172.21.15.76:0/1330680905 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f156806ae70 0x7f156806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:27.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.633+0000 7f157a7fc640 1 -- 172.21.15.76:0/1330680905 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f15740abef0 con 0x7f157c0b8b10 2023-11-09T03:45:27.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.633+0000 7f15815d4640 1 --2- 172.21.15.76:0/1330680905 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f156806ae70 0x7f156806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:27.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.633+0000 7f15815d4640 1 --2- 172.21.15.76:0/1330680905 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f156806ae70 0x7f156806d330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1570006fd0 tx=0x7f1570008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:27.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.637+0000 7f157a7fc640 1 -- 172.21.15.76:0/1330680905 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1574077380 con 0x7f157c0b8b10 2023-11-09T03:45:27.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.802+0000 7f1584039640 1 -- 172.21.15.76:0/1330680905 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f157c0cba30 con 0x7f157c0b8b10 2023-11-09T03:45:27.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.803+0000 7f157a7fc640 1 -- 172.21.15.76:0/1330680905 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12838 (secure 0 0 0) 0x7f157c0cba30 con 0x7f157c0b8b10 2023-11-09T03:45:27.803 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:27.804 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":17,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:45:21.864770+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:45:27.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.805+0000 7f1584039640 1 -- 172.21.15.76:0/1330680905 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f156806ae70 msgr2=0x7f156806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:27.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.805+0000 7f1584039640 1 --2- 172.21.15.76:0/1330680905 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f156806ae70 0x7f156806d330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1570006fd0 tx=0x7f1570008040 comp rx=0 tx=0).stop 2023-11-09T03:45:27.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.805+0000 7f1584039640 1 -- 172.21.15.76:0/1330680905 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f157c0b8b10 msgr2=0x7f157c0c6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:27.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.805+0000 7f1584039640 1 --2- 172.21.15.76:0/1330680905 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f157c0b8b10 0x7f157c0c6970 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1574000f80 tx=0x7f1574004a40 comp rx=0 tx=0).stop 2023-11-09T03:45:27.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.805+0000 7f1584039640 1 -- 172.21.15.76:0/1330680905 shutdown_connections 2023-11-09T03:45:27.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.805+0000 7f1584039640 1 --2- 172.21.15.76:0/1330680905 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f156806ae70 0x7f156806d330 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:27.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.805+0000 7f1584039640 1 --2- 172.21.15.76:0/1330680905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f157c15cdf0 0x7f157c0c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:27.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.805+0000 7f1584039640 1 --2- 172.21.15.76:0/1330680905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f157c0d0db0 0x7f157c1f1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:27.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.805+0000 7f1584039640 1 --2- 172.21.15.76:0/1330680905 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f157c0b8b10 0x7f157c0c6970 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:27.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.805+0000 7f1584039640 1 -- 172.21.15.76:0/1330680905 >> 172.21.15.76:0/1330680905 conn(0x7f157c071c70 msgr2=0x7f157c15e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:27.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.805+0000 7f1584039640 1 -- 172.21.15.76:0/1330680905 shutdown_connections 2023-11-09T03:45:27.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.805+0000 7f1584039640 1 -- 172.21.15.76:0/1330680905 wait complete. 2023-11-09T03:45:27.816 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-09T03:45:27.816 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:45:27.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.919+0000 7f67c5505640 1 Processor -- start 2023-11-09T03:45:27.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.919+0000 7f67c5505640 1 -- start start 2023-11-09T03:45:27.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.920+0000 7f67c5505640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f67c00cd6d0 0x7f67c0161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:27.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.920+0000 7f67c5505640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c00cde60 con 0x7f67c0153e60 2023-11-09T03:45:27.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.920+0000 7f67c5505640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c0160f90 con 0x7f67c00b4880 2023-11-09T03:45:27.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.920+0000 7f67c5505640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c0161960 con 0x7f67c00cd6d0 2023-11-09T03:45:27.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.920+0000 7f67beffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f67c0153e60 0x7f67c015dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:60090/0 (socket says 172.21.15.76:60090) 2023-11-09T03:45:27.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.920+0000 7f67beffd640 1 -- 172.21.15.76:0/1862681745 learned_addr learned my addr 172.21.15.76:0/1862681745 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:27.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.920+0000 7f67be7fc640 1 --2- 172.21.15.76:0/1862681745 >> v2:172.21.15.113:3300/0 conn(0x7f67c00cd6d0 0x7f67c0161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:27.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.920+0000 7f67bdffb640 1 -- 172.21.15.76:0/1862681745 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1621376006 0 0) 0x7f67c00cde60 con 0x7f67c0153e60 2023-11-09T03:45:27.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.920+0000 7f67bdffb640 1 -- 172.21.15.76:0/1862681745 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67a8003520 con 0x7f67c0153e60 2023-11-09T03:45:27.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.920+0000 7f67bdffb640 1 -- 172.21.15.76:0/1862681745 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2955738328 0 0) 0x7f67a8003520 con 0x7f67c0153e60 2023-11-09T03:45:27.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.920+0000 7f67bdffb640 1 -- 172.21.15.76:0/1862681745 >> v2:172.21.15.113:3300/0 conn(0x7f67c00cd6d0 msgr2=0x7f67c0161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:27.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.920+0000 7f67bdffb640 1 --2- 172.21.15.76:0/1862681745 >> v2:172.21.15.113:3300/0 conn(0x7f67c00cd6d0 0x7f67c0161420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:27.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.920+0000 7f67bdffb640 1 -- 172.21.15.76:0/1862681745 >> v1:172.21.15.113:6789/0 conn(0x7f67c00b4880 legacy=0x7f67c00d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:27.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.920+0000 7f67bdffb640 1 -- 172.21.15.76:0/1862681745 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c00ce090 con 0x7f67c0153e60 2023-11-09T03:45:27.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.920+0000 7f67bdffb640 1 -- 172.21.15.76:0/1862681745 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f67b00019f0 con 0x7f67c0153e60 2023-11-09T03:45:27.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.920+0000 7f67bdffb640 1 -- 172.21.15.76:0/1862681745 >> v1:172.21.15.76:6789/0 conn(0x7f67c0153e60 legacy=0x7f67c015dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:27.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.920+0000 7f67bdffb640 1 --2- 172.21.15.76:0/1862681745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67a80039c0 0x7f67a8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:27.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.920+0000 7f67bdffb640 1 --2- 172.21.15.76:0/1862681745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67a80047c0 0x7f67a8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:27.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.920+0000 7f67bdffb640 1 --2- 172.21.15.76:0/1862681745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67a8005180 0x7f67a800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:27.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.921+0000 7f67bdffb640 1 -- 172.21.15.76:0/1862681745 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c0161960 con 0x7f67a8005180 2023-11-09T03:45:27.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.921+0000 7f67bdffb640 1 -- 172.21.15.76:0/1862681745 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a800c080 con 0x7f67a80039c0 2023-11-09T03:45:27.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.921+0000 7f67bdffb640 1 -- 172.21.15.76:0/1862681745 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a800c240 con 0x7f67a80047c0 2023-11-09T03:45:27.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.921+0000 7f67bf7fe640 1 --2- 172.21.15.76:0/1862681745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67a8005180 0x7f67a800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:27.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.921+0000 7f67beffd640 1 --2- 172.21.15.76:0/1862681745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67a80039c0 0x7f67a8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:27.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.921+0000 7f67be7fc640 1 --2- 172.21.15.76:0/1862681745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67a80047c0 0x7f67a8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:27.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.921+0000 7f67bf7fe640 1 -- 172.21.15.76:0/1862681745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67a80047c0 msgr2=0x7f67a8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:27.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.921+0000 7f67bf7fe640 1 --2- 172.21.15.76:0/1862681745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67a80047c0 0x7f67a8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:27.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.921+0000 7f67bf7fe640 1 -- 172.21.15.76:0/1862681745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67a80039c0 msgr2=0x7f67a8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:27.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.921+0000 7f67bf7fe640 1 --2- 172.21.15.76:0/1862681745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67a80039c0 0x7f67a8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:27.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.921+0000 7f67bf7fe640 1 -- 172.21.15.76:0/1862681745 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a8003520 con 0x7f67a8005180 2023-11-09T03:45:27.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.921+0000 7f67bf7fe640 1 -- 172.21.15.76:0/1862681745 reap_dead start 2023-11-09T03:45:27.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.922+0000 7f67bf7fe640 1 --2- 172.21.15.76:0/1862681745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67a8005180 0x7f67a800b980 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f67b8001750 tx=0x7f67a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5142be8624cafeaf server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:27.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.922+0000 7f67bdffb640 1 -- 172.21.15.76:0/1862681745 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b80052d0 con 0x7f67a8005180 2023-11-09T03:45:27.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.922+0000 7f67bdffb640 1 -- 172.21.15.76:0/1862681745 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b802e050 con 0x7f67a8005180 2023-11-09T03:45:27.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.922+0000 7f67bdffb640 1 -- 172.21.15.76:0/1862681745 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b8003830 con 0x7f67a8005180 2023-11-09T03:45:27.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.922+0000 7f67c5505640 1 -- 172.21.15.76:0/1862681745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67a8005180 msgr2=0x7f67a800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:27.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.922+0000 7f67c5505640 1 --2- 172.21.15.76:0/1862681745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67a8005180 0x7f67a800b980 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f67b8001750 tx=0x7f67a80057f0 comp rx=0 tx=0).stop 2023-11-09T03:45:27.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.922+0000 7f67c5505640 1 -- 172.21.15.76:0/1862681745 shutdown_connections 2023-11-09T03:45:27.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.922+0000 7f67c5505640 1 --2- 172.21.15.76:0/1862681745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67a8005180 0x7f67a800b980 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:27.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.922+0000 7f67c5505640 1 -- 172.21.15.76:0/1862681745 >> 172.21.15.76:0/1862681745 conn(0x7f67c00cf880 msgr2=0x7f67c00cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:27.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.922+0000 7f67c5505640 1 -- 172.21.15.76:0/1862681745 shutdown_connections 2023-11-09T03:45:27.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.922+0000 7f67c5505640 1 -- 172.21.15.76:0/1862681745 wait complete. 2023-11-09T03:45:27.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.924+0000 7f67c5505640 1 Processor -- start 2023-11-09T03:45:27.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.924+0000 7f67c5505640 1 -- start start 2023-11-09T03:45:27.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.924+0000 7f67c5505640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67a8005180 0x7f67c0153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:27.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.924+0000 7f67c5505640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67c01f3550 0x7f67c00cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:27.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.924+0000 7f67c5505640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67c00ccc30 0x7f67c00b7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:27.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.924+0000 7f67c5505640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c01f69c0 con 0x7f67c00ccc30 2023-11-09T03:45:27.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.924+0000 7f67c5505640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c00cd0e0 con 0x7f67a8005180 2023-11-09T03:45:27.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.924+0000 7f67c5505640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c01f6de0 con 0x7f67c01f3550 2023-11-09T03:45:27.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.924+0000 7f67bf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67c00ccc30 0x7f67c00b7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:27.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.924+0000 7f67bf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67c00ccc30 0x7f67c00b7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54444/0 (socket says 172.21.15.76:54444) 2023-11-09T03:45:27.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.924+0000 7f67bf7fe640 1 -- 172.21.15.76:0/1009789961 learned_addr learned my addr 172.21.15.76:0/1009789961 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:27.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.925+0000 7f67beffd640 1 --2- 172.21.15.76:0/1009789961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67a8005180 0x7f67c0153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:27.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.925+0000 7f67be7fc640 1 --2- 172.21.15.76:0/1009789961 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67c01f3550 0x7f67c00cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:27.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.925+0000 7f67bf7fe640 1 -- 172.21.15.76:0/1009789961 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67c01f3550 msgr2=0x7f67c00cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:27.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.925+0000 7f67bf7fe640 1 --2- 172.21.15.76:0/1009789961 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67c01f3550 0x7f67c00cc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:27.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.925+0000 7f67bf7fe640 1 -- 172.21.15.76:0/1009789961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67a8005180 msgr2=0x7f67c0153210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:27.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.925+0000 7f67bf7fe640 1 --2- 172.21.15.76:0/1009789961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67a8005180 0x7f67c0153210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:27.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.925+0000 7f67bf7fe640 1 -- 172.21.15.76:0/1009789961 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c01f6020 con 0x7f67c00ccc30 2023-11-09T03:45:27.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.925+0000 7f67beffd640 1 --2- 172.21.15.76:0/1009789961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67a8005180 0x7f67c0153210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:27.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.925+0000 7f67be7fc640 1 --2- 172.21.15.76:0/1009789961 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67c01f3550 0x7f67c00cc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:27.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.925+0000 7f67bf7fe640 1 --2- 172.21.15.76:0/1009789961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67c00ccc30 0x7f67c00b7430 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f67b8001840 tx=0x7f67b8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:27.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.925+0000 7f67a77fe640 1 -- 172.21.15.76:0/1009789961 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b8039070 con 0x7f67c00ccc30 2023-11-09T03:45:27.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.925+0000 7f67a77fe640 1 -- 172.21.15.76:0/1009789961 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b8005450 con 0x7f67c00ccc30 2023-11-09T03:45:27.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.925+0000 7f67c5505640 1 -- 172.21.15.76:0/1009789961 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c01f6300 con 0x7f67c00ccc30 2023-11-09T03:45:27.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.925+0000 7f67a77fe640 1 -- 172.21.15.76:0/1009789961 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b8007230 con 0x7f67c00ccc30 2023-11-09T03:45:27.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.925+0000 7f67c5505640 1 -- 172.21.15.76:0/1009789961 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c0153750 con 0x7f67c00ccc30 2023-11-09T03:45:27.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.926+0000 7f67c5505640 1 -- 172.21.15.76:0/1009789961 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c01f6b00 con 0x7f67c00ccc30 2023-11-09T03:45:27.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.926+0000 7f67a77fe640 1 -- 172.21.15.76:0/1009789961 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f67b803c070 con 0x7f67c00ccc30 2023-11-09T03:45:27.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.926+0000 7f67a77fe640 1 --2- 172.21.15.76:0/1009789961 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f67ac06ae70 0x7f67ac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:27.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.927+0000 7f67a77fe640 1 -- 172.21.15.76:0/1009789961 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f67b80a9190 con 0x7f67c00ccc30 2023-11-09T03:45:27.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.927+0000 7f67beffd640 1 --2- 172.21.15.76:0/1009789961 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f67ac06ae70 0x7f67ac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:27.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.927+0000 7f67beffd640 1 --2- 172.21.15.76:0/1009789961 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f67ac06ae70 0x7f67ac06d330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f67b0000f80 tx=0x7f67b0006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:27.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:27.931+0000 7f67a77fe640 1 -- 172.21.15.76:0/1009789961 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f67c01f6b00 con 0x7f67c00ccc30 2023-11-09T03:45:28.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.098+0000 7f67c5505640 1 -- 172.21.15.76:0/1009789961 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f67c01f6b00 con 0x7f67c00ccc30 2023-11-09T03:45:28.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.099+0000 7f67a77fe640 1 -- 172.21.15.76:0/1009789961 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f67c01f6b00 con 0x7f67c00ccc30 2023-11-09T03:45:28.099 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:45:28.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.100+0000 7f67c5505640 1 -- 172.21.15.76:0/1009789961 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f67ac06ae70 msgr2=0x7f67ac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:28.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.100+0000 7f67c5505640 1 --2- 172.21.15.76:0/1009789961 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f67ac06ae70 0x7f67ac06d330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f67b0000f80 tx=0x7f67b0006040 comp rx=0 tx=0).stop 2023-11-09T03:45:28.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.100+0000 7f67c5505640 1 -- 172.21.15.76:0/1009789961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67c00ccc30 msgr2=0x7f67c00b7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:28.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.100+0000 7f67c5505640 1 --2- 172.21.15.76:0/1009789961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67c00ccc30 0x7f67c00b7430 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f67b8001840 tx=0x7f67b8001120 comp rx=0 tx=0).stop 2023-11-09T03:45:28.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.100+0000 7f67c5505640 1 -- 172.21.15.76:0/1009789961 shutdown_connections 2023-11-09T03:45:28.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.100+0000 7f67c5505640 1 --2- 172.21.15.76:0/1009789961 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f67ac06ae70 0x7f67ac06d330 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:28.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.100+0000 7f67c5505640 1 --2- 172.21.15.76:0/1009789961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67c00ccc30 0x7f67c00b7430 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:28.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.100+0000 7f67c5505640 1 --2- 172.21.15.76:0/1009789961 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67c01f3550 0x7f67c00cc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:28.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.100+0000 7f67c5505640 1 --2- 172.21.15.76:0/1009789961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67a8005180 0x7f67c0153210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:28.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.100+0000 7f67c5505640 1 -- 172.21.15.76:0/1009789961 >> 172.21.15.76:0/1009789961 conn(0x7f67c00cf880 msgr2=0x7f67c00d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:28.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.100+0000 7f67c5505640 1 -- 172.21.15.76:0/1009789961 shutdown_connections 2023-11-09T03:45:28.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.100+0000 7f67c5505640 1 -- 172.21.15.76:0/1009789961 wait complete. 2023-11-09T03:45:28.111 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-09T03:45:28.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.211+0000 7f5991eb4640 1 Processor -- start 2023-11-09T03:45:28.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.211+0000 7f5991eb4640 1 -- start start 2023-11-09T03:45:28.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.212+0000 7f5991eb4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f598c0aec80 0x7f598c0b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:28.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.212+0000 7f5991eb4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f598c06ef40 0x7f598c0cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:28.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.212+0000 7f5991eb4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f598c0b1590 con 0x7f598c0af060 2023-11-09T03:45:28.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.212+0000 7f5991eb4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f598c155a40 con 0x7f598c06ef40 2023-11-09T03:45:28.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.212+0000 7f5991eb4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598c06a8e0 con 0x7f598c0aec80 2023-11-09T03:45:28.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.212+0000 7f598bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f598c06ef40 0x7f598c0cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:28.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.212+0000 7f598bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f598c06ef40 0x7f598c0cb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54460/0 (socket says 172.21.15.76:54460) 2023-11-09T03:45:28.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.212+0000 7f598bfff640 1 -- 172.21.15.76:0/831444509 learned_addr learned my addr 172.21.15.76:0/831444509 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:28.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.212+0000 7f598b7fe640 1 --2- 172.21.15.76:0/831444509 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f598c0aec80 0x7f598c0b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:28.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.212+0000 7f598bfff640 1 -- 172.21.15.76:0/831444509 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f598c0aec80 msgr2=0x7f598c0b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:28.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.212+0000 7f598bfff640 1 --2- 172.21.15.76:0/831444509 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f598c0aec80 0x7f598c0b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:28.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.212+0000 7f598bfff640 1 -- 172.21.15.76:0/831444509 >> v1:172.21.15.113:6789/0 conn(0x7f598c0af060 legacy=0x7f598c06a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:28.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.212+0000 7f598bfff640 1 -- 172.21.15.76:0/831444509 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598c0b1800 con 0x7f598c06ef40 2023-11-09T03:45:28.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.213+0000 7f598bfff640 1 --2- 172.21.15.76:0/831444509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f598c06ef40 0x7f598c0cb7a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5984009e50 tx=0x7f598402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9e263f6949ead6 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:28.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.213+0000 7f598a7fc640 1 -- 172.21.15.76:0/831444509 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598402ca70 con 0x7f598c06ef40 2023-11-09T03:45:28.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.213+0000 7f598a7fc640 1 -- 172.21.15.76:0/831444509 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598402cbd0 con 0x7f598c06ef40 2023-11-09T03:45:28.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.213+0000 7f598a7fc640 1 -- 172.21.15.76:0/831444509 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59840343f0 con 0x7f598c06ef40 2023-11-09T03:45:28.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.213+0000 7f5991eb4640 1 -- 172.21.15.76:0/831444509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f598c06ef40 msgr2=0x7f598c0cb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:28.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.213+0000 7f5991eb4640 1 --2- 172.21.15.76:0/831444509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f598c06ef40 0x7f598c0cb7a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5984009e50 tx=0x7f598402bed0 comp rx=0 tx=0).stop 2023-11-09T03:45:28.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.213+0000 7f5991eb4640 1 -- 172.21.15.76:0/831444509 shutdown_connections 2023-11-09T03:45:28.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.213+0000 7f5991eb4640 1 --2- 172.21.15.76:0/831444509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f598c06ef40 0x7f598c0cb7a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:28.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.213+0000 7f5991eb4640 1 --2- 172.21.15.76:0/831444509 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f598c0aec80 0x7f598c0b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:28.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.213+0000 7f5991eb4640 1 -- 172.21.15.76:0/831444509 >> 172.21.15.76:0/831444509 conn(0x7f598c098d40 msgr2=0x7f598c0cfa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:28.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.214+0000 7f5991eb4640 1 -- 172.21.15.76:0/831444509 shutdown_connections 2023-11-09T03:45:28.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.214+0000 7f5991eb4640 1 -- 172.21.15.76:0/831444509 wait complete. 2023-11-09T03:45:28.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.215+0000 7f5991eb4640 1 Processor -- start 2023-11-09T03:45:28.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.215+0000 7f5991eb4640 1 -- start start 2023-11-09T03:45:28.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.215+0000 7f5991eb4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f598c06ef40 0x7f598c151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:28.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.215+0000 7f5991eb4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f598c0aec80 0x7f598c0c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:28.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.215+0000 7f5991eb4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f598c0af060 0x7f598c0c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:28.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.215+0000 7f5991eb4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598c15d750 con 0x7f598c0af060 2023-11-09T03:45:28.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.215+0000 7f5991eb4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598c151880 con 0x7f598c06ef40 2023-11-09T03:45:28.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.215+0000 7f5991eb4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598c15cb40 con 0x7f598c0aec80 2023-11-09T03:45:28.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.216+0000 7f598bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f598c0af060 0x7f598c0c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:28.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.216+0000 7f598bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f598c0af060 0x7f598c0c84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54474/0 (socket says 172.21.15.76:54474) 2023-11-09T03:45:28.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.216+0000 7f598bfff640 1 -- 172.21.15.76:0/1141949554 learned_addr learned my addr 172.21.15.76:0/1141949554 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:28.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.216+0000 7f598b7fe640 1 --2- 172.21.15.76:0/1141949554 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f598c06ef40 0x7f598c151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:28.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.216+0000 7f598affd640 1 --2- 172.21.15.76:0/1141949554 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f598c0aec80 0x7f598c0c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:28.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.216+0000 7f598bfff640 1 -- 172.21.15.76:0/1141949554 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f598c0aec80 msgr2=0x7f598c0c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:28.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.216+0000 7f598bfff640 1 --2- 172.21.15.76:0/1141949554 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f598c0aec80 0x7f598c0c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:28.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.216+0000 7f598bfff640 1 -- 172.21.15.76:0/1141949554 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f598c06ef40 msgr2=0x7f598c151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:28.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.216+0000 7f598bfff640 1 --2- 172.21.15.76:0/1141949554 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f598c06ef40 0x7f598c151340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:28.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.216+0000 7f598bfff640 1 -- 172.21.15.76:0/1141949554 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5984009b30 con 0x7f598c0af060 2023-11-09T03:45:28.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.216+0000 7f598bfff640 1 --2- 172.21.15.76:0/1141949554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f598c0af060 0x7f598c0c84d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5984036040 tx=0x7f5984004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:28.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.216+0000 7f5973fff640 1 -- 172.21.15.76:0/1141949554 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5984009510 con 0x7f598c0af060 2023-11-09T03:45:28.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.216+0000 7f5973fff640 1 -- 172.21.15.76:0/1141949554 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598403e450 con 0x7f598c0af060 2023-11-09T03:45:28.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.216+0000 7f5973fff640 1 -- 172.21.15.76:0/1141949554 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598403e7e0 con 0x7f598c0af060 2023-11-09T03:45:28.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.216+0000 7f5991eb4640 1 -- 172.21.15.76:0/1141949554 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f598c15ce20 con 0x7f598c0af060 2023-11-09T03:45:28.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.216+0000 7f5991eb4640 1 -- 172.21.15.76:0/1141949554 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f598c15d3e0 con 0x7f598c0af060 2023-11-09T03:45:28.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.217+0000 7f5991eb4640 1 -- 172.21.15.76:0/1141949554 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f598c15d900 con 0x7f598c0af060 2023-11-09T03:45:28.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.218+0000 7f5973fff640 1 -- 172.21.15.76:0/1141949554 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f598403e940 con 0x7f598c0af060 2023-11-09T03:45:28.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.218+0000 7f5973fff640 1 --2- 172.21.15.76:0/1141949554 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f597806aea0 0x7f597806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:28.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.218+0000 7f5973fff640 1 -- 172.21.15.76:0/1141949554 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f59840abc20 con 0x7f598c0af060 2023-11-09T03:45:28.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.218+0000 7f598b7fe640 1 --2- 172.21.15.76:0/1141949554 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f597806aea0 0x7f597806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:28.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.219+0000 7f598b7fe640 1 --2- 172.21.15.76:0/1141949554 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f597806aea0 0x7f597806d360 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f597c007a30 tx=0x7f597c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:28.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.222+0000 7f5973fff640 1 -- 172.21.15.76:0/1141949554 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f598c15d900 con 0x7f598c0af060 2023-11-09T03:45:28.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.391+0000 7f5991eb4640 1 -- 172.21.15.76:0/1141949554 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f598c074290 con 0x7f598c0af060 2023-11-09T03:45:28.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.392+0000 7f5973fff640 1 -- 172.21.15.76:0/1141949554 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f59840770d0 con 0x7f598c0af060 2023-11-09T03:45:28.392 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:28.393 INFO:teuthology.orchestra.run.smithi076.stdout:[ 2023-11-09T03:45:28.393 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:45:28.393 INFO:teuthology.orchestra.run.smithi076.stdout: "poolnum": 1, 2023-11-09T03:45:28.393 INFO:teuthology.orchestra.run.smithi076.stdout: "poolname": ".mgr" 2023-11-09T03:45:28.393 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:45:28.394 INFO:teuthology.orchestra.run.smithi076.stdout:] 2023-11-09T03:45:28.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.394+0000 7f5991eb4640 1 -- 172.21.15.76:0/1141949554 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f597806aea0 msgr2=0x7f597806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:28.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.394+0000 7f5991eb4640 1 --2- 172.21.15.76:0/1141949554 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f597806aea0 0x7f597806d360 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f597c007a30 tx=0x7f597c008040 comp rx=0 tx=0).stop 2023-11-09T03:45:28.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.394+0000 7f5991eb4640 1 -- 172.21.15.76:0/1141949554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f598c0af060 msgr2=0x7f598c0c84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:28.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.394+0000 7f5991eb4640 1 --2- 172.21.15.76:0/1141949554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f598c0af060 0x7f598c0c84d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5984036040 tx=0x7f5984004a40 comp rx=0 tx=0).stop 2023-11-09T03:45:28.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.394+0000 7f5991eb4640 1 -- 172.21.15.76:0/1141949554 shutdown_connections 2023-11-09T03:45:28.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.394+0000 7f5991eb4640 1 --2- 172.21.15.76:0/1141949554 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f597806aea0 0x7f597806d360 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:28.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.394+0000 7f5991eb4640 1 --2- 172.21.15.76:0/1141949554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f598c0af060 0x7f598c0c84d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:28.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.394+0000 7f5991eb4640 1 --2- 172.21.15.76:0/1141949554 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f598c0aec80 0x7f598c0c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:28.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.394+0000 7f5991eb4640 1 --2- 172.21.15.76:0/1141949554 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f598c06ef40 0x7f598c151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:28.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.394+0000 7f5991eb4640 1 -- 172.21.15.76:0/1141949554 >> 172.21.15.76:0/1141949554 conn(0x7f598c098d40 msgr2=0x7f598c0af4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:28.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.394+0000 7f5991eb4640 1 -- 172.21.15.76:0/1141949554 shutdown_connections 2023-11-09T03:45:28.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.394+0000 7f5991eb4640 1 -- 172.21.15.76:0/1141949554 wait complete. 2023-11-09T03:45:28.404 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-09T03:45:28.405 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-09T03:45:28.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.507+0000 7fafe4651640 1 Processor -- start 2023-11-09T03:45:28.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.507+0000 7fafe4651640 1 -- start start 2023-11-09T03:45:28.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.508+0000 7fafe4651640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fafdc15d910 0x7fafdc0d1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:28.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.508+0000 7fafe4651640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafdc0d1320 con 0x7fafdc0cd9f0 2023-11-09T03:45:28.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.508+0000 7fafe4651640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafdc15df10 con 0x7fafdc15d4b0 2023-11-09T03:45:28.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.508+0000 7fafe4651640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafdc15e4d0 con 0x7fafdc15d910 2023-11-09T03:45:28.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.509+0000 7fafe23ed640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fafdc0cd9f0 0x7fafdc0cddd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:60098/0 (socket says 172.21.15.76:60098) 2023-11-09T03:45:28.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.509+0000 7fafe23ed640 1 -- 172.21.15.76:0/99172799 learned_addr learned my addr 172.21.15.76:0/99172799 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:28.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.509+0000 7fafe2bee640 1 --2- 172.21.15.76:0/99172799 >> v2:172.21.15.76:3300/0 conn(0x7fafdc15d910 0x7fafdc0d1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:28.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.509+0000 7fafe13eb640 1 -- 172.21.15.76:0/99172799 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4252474036 0 0) 0x7fafdc0d1320 con 0x7fafdc0cd9f0 2023-11-09T03:45:28.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.509+0000 7fafe13eb640 1 -- 172.21.15.76:0/99172799 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafc8003520 con 0x7fafdc0cd9f0 2023-11-09T03:45:28.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.509+0000 7fafe2bee640 1 -- 172.21.15.76:0/99172799 >> v1:172.21.15.113:6789/0 conn(0x7fafdc15d4b0 legacy=0x7fafdc0d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:28.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.509+0000 7fafe2bee640 1 -- 172.21.15.76:0/99172799 >> v1:172.21.15.76:6789/0 conn(0x7fafdc0cd9f0 legacy=0x7fafdc0cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:28.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.509+0000 7fafe2bee640 1 -- 172.21.15.76:0/99172799 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafdc15e1f0 con 0x7fafdc15d910 2023-11-09T03:45:28.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.510+0000 7fafe2bee640 1 --2- 172.21.15.76:0/99172799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafdc15d910 0x7fafdc0d1ec0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fafd40097d0 tx=0x7fafd402b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2f615290801854 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:28.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.510+0000 7fafe13eb640 1 -- 172.21.15.76:0/99172799 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd4030890 con 0x7fafdc15d910 2023-11-09T03:45:28.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.510+0000 7fafe13eb640 1 -- 172.21.15.76:0/99172799 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd40309f0 con 0x7fafdc15d910 2023-11-09T03:45:28.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.510+0000 7fafe13eb640 1 -- 172.21.15.76:0/99172799 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd4030d80 con 0x7fafdc15d910 2023-11-09T03:45:28.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.510+0000 7fafe4651640 1 -- 172.21.15.76:0/99172799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafdc15d910 msgr2=0x7fafdc0d1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:28.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.510+0000 7fafe4651640 1 --2- 172.21.15.76:0/99172799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafdc15d910 0x7fafdc0d1ec0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fafd40097d0 tx=0x7fafd402b9d0 comp rx=0 tx=0).stop 2023-11-09T03:45:28.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.510+0000 7fafe4651640 1 -- 172.21.15.76:0/99172799 shutdown_connections 2023-11-09T03:45:28.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.510+0000 7fafe4651640 1 --2- 172.21.15.76:0/99172799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafdc15d910 0x7fafdc0d1ec0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:28.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.510+0000 7fafe4651640 1 -- 172.21.15.76:0/99172799 >> 172.21.15.76:0/99172799 conn(0x7fafdc071c40 msgr2=0x7fafdc072050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:28.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.510+0000 7fafe4651640 1 -- 172.21.15.76:0/99172799 shutdown_connections 2023-11-09T03:45:28.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.510+0000 7fafe4651640 1 -- 172.21.15.76:0/99172799 wait complete. 2023-11-09T03:45:28.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.511+0000 7fafe4651640 1 Processor -- start 2023-11-09T03:45:28.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.511+0000 7fafe4651640 1 -- start start 2023-11-09T03:45:28.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.512+0000 7fafe4651640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafdc0cd9f0 0x7fafdc1f6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:28.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.512+0000 7fafe4651640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fafdc15d4b0 0x7fafdc1e69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:28.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.512+0000 7fafe4651640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafdc15d910 0x7fafdc0c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:28.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.512+0000 7fafe4651640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc0ccfd0 con 0x7fafdc15d910 2023-11-09T03:45:28.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.512+0000 7fafe4651640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc1581f0 con 0x7fafdc0cd9f0 2023-11-09T03:45:28.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.512+0000 7fafe4651640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc1f6cd0 con 0x7fafdc15d4b0 2023-11-09T03:45:28.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.512+0000 7fafe2bee640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafdc15d910 0x7fafdc0c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:28.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.512+0000 7fafe2bee640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafdc15d910 0x7fafdc0c8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54498/0 (socket says 172.21.15.76:54498) 2023-11-09T03:45:28.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.512+0000 7fafe2bee640 1 -- 172.21.15.76:0/2116931298 learned_addr learned my addr 172.21.15.76:0/2116931298 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:28.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.512+0000 7fafe2bee640 1 -- 172.21.15.76:0/2116931298 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fafdc15d4b0 msgr2=0x7fafdc1e69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:28.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.512+0000 7fafe2bee640 1 --2- 172.21.15.76:0/2116931298 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fafdc15d4b0 0x7fafdc1e69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:28.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.512+0000 7fafe2bee640 1 -- 172.21.15.76:0/2116931298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafdc0cd9f0 msgr2=0x7fafdc1f6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:28.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.512+0000 7fafe2bee640 1 --2- 172.21.15.76:0/2116931298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafdc0cd9f0 0x7fafdc1f6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:28.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.513+0000 7fafe2bee640 1 -- 172.21.15.76:0/2116931298 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd4009400 con 0x7fafdc15d910 2023-11-09T03:45:28.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.513+0000 7fafe2bee640 1 --2- 172.21.15.76:0/2116931298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafdc15d910 0x7fafdc0c8ce0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fafd402ba10 tx=0x7fafd4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:28.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.513+0000 7fafd2ffd640 1 -- 172.21.15.76:0/2116931298 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd403e040 con 0x7fafdc15d910 2023-11-09T03:45:28.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.513+0000 7fafd2ffd640 1 -- 172.21.15.76:0/2116931298 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd4039c80 con 0x7fafdc15d910 2023-11-09T03:45:28.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.513+0000 7fafe4651640 1 -- 172.21.15.76:0/2116931298 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafdc157670 con 0x7fafdc15d910 2023-11-09T03:45:28.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.513+0000 7fafd2ffd640 1 -- 172.21.15.76:0/2116931298 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd40443f0 con 0x7fafdc15d910 2023-11-09T03:45:28.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.513+0000 7fafe4651640 1 -- 172.21.15.76:0/2116931298 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafdc157c30 con 0x7fafdc15d910 2023-11-09T03:45:28.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.514+0000 7fafe4651640 1 -- 172.21.15.76:0/2116931298 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafdc159080 con 0x7fafdc15d910 2023-11-09T03:45:28.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.514+0000 7fafd2ffd640 1 -- 172.21.15.76:0/2116931298 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fafd402f3e0 con 0x7fafdc15d910 2023-11-09T03:45:28.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.515+0000 7fafd2ffd640 1 --2- 172.21.15.76:0/2116931298 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fafc006aea0 0x7fafc006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:28.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.515+0000 7fafd2ffd640 1 -- 172.21.15.76:0/2116931298 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fafd40ab830 con 0x7fafdc15d910 2023-11-09T03:45:28.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.515+0000 7fafe23ed640 1 --2- 172.21.15.76:0/2116931298 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fafc006aea0 0x7fafc006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:28.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.515+0000 7fafe23ed640 1 --2- 172.21.15.76:0/2116931298 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fafc006aea0 0x7fafc006d360 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fafd8002000 tx=0x7fafd80015f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:28.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.519+0000 7fafd2ffd640 1 -- 172.21.15.76:0/2116931298 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fafdc159080 con 0x7fafdc15d910 2023-11-09T03:45:28.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:28.685+0000 7fafe4651640 1 -- 172.21.15.76:0/2116931298 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fafdc159080 con 0x7fafdc15d910 2023-11-09T03:45:29.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.390+0000 7fafd2ffd640 1 -- 172.21.15.76:0/2116931298 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7fafdc159080 con 0x7fafdc15d910 2023-11-09T03:45:29.391 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' created 2023-11-09T03:45:29.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.393+0000 7fafe4651640 1 -- 172.21.15.76:0/2116931298 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fafc006aea0 msgr2=0x7fafc006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:29.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.393+0000 7fafe4651640 1 --2- 172.21.15.76:0/2116931298 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fafc006aea0 0x7fafc006d360 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fafd8002000 tx=0x7fafd80015f0 comp rx=0 tx=0).stop 2023-11-09T03:45:29.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.393+0000 7fafe4651640 1 -- 172.21.15.76:0/2116931298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafdc15d910 msgr2=0x7fafdc0c8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:29.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.393+0000 7fafe4651640 1 --2- 172.21.15.76:0/2116931298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafdc15d910 0x7fafdc0c8ce0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fafd402ba10 tx=0x7fafd4004730 comp rx=0 tx=0).stop 2023-11-09T03:45:29.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.393+0000 7fafe4651640 1 -- 172.21.15.76:0/2116931298 shutdown_connections 2023-11-09T03:45:29.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.393+0000 7fafe4651640 1 --2- 172.21.15.76:0/2116931298 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fafc006aea0 0x7fafc006d360 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:29.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.393+0000 7fafe4651640 1 --2- 172.21.15.76:0/2116931298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafdc15d910 0x7fafdc0c8ce0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:29.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.393+0000 7fafe4651640 1 --2- 172.21.15.76:0/2116931298 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fafdc15d4b0 0x7fafdc1e69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:29.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.393+0000 7fafe4651640 1 --2- 172.21.15.76:0/2116931298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafdc0cd9f0 0x7fafdc1f6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:29.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.393+0000 7fafe4651640 1 -- 172.21.15.76:0/2116931298 >> 172.21.15.76:0/2116931298 conn(0x7fafdc071c40 msgr2=0x7fafdc1569c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:29.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.393+0000 7fafe4651640 1 -- 172.21.15.76:0/2116931298 shutdown_connections 2023-11-09T03:45:29.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.393+0000 7fafe4651640 1 -- 172.21.15.76:0/2116931298 wait complete. 2023-11-09T03:45:29.409 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-11-09T03:45:29.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.521+0000 7f4064b05640 1 Processor -- start 2023-11-09T03:45:29.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.521+0000 7f4064b05640 1 -- start start 2023-11-09T03:45:29.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.522+0000 7f4064b05640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f406015e1b0 0x7f40600ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:29.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.522+0000 7f4064b05640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f406006a830 0x7f40600722d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:29.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.522+0000 7f4064b05640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40600ca620 con 0x7f406015e590 2023-11-09T03:45:29.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.522+0000 7f4064b05640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f406006e7e0 con 0x7f406006a830 2023-11-09T03:45:29.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.522+0000 7f4064b05640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40600cb060 con 0x7f406015e1b0 2023-11-09T03:45:29.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.522+0000 7f405ed9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f406006a830 0x7f40600722d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:29.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.522+0000 7f405ed9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f406006a830 0x7f40600722d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54512/0 (socket says 172.21.15.76:54512) 2023-11-09T03:45:29.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.522+0000 7f405ed9d640 1 -- 172.21.15.76:0/1237145153 learned_addr learned my addr 172.21.15.76:0/1237145153 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:29.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.523+0000 7f405e59c640 1 --2- 172.21.15.76:0/1237145153 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f406015e1b0 0x7f40600ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:29.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.523+0000 7f405ed9d640 1 -- 172.21.15.76:0/1237145153 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f406015e1b0 msgr2=0x7f40600ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:29.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.523+0000 7f405ed9d640 1 --2- 172.21.15.76:0/1237145153 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f406015e1b0 0x7f40600ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:29.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.523+0000 7f405ed9d640 1 -- 172.21.15.76:0/1237145153 >> v1:172.21.15.113:6789/0 conn(0x7f406015e590 legacy=0x7f406006a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:29.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.523+0000 7f405ed9d640 1 -- 172.21.15.76:0/1237145153 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40600ca060 con 0x7f406006a830 2023-11-09T03:45:29.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.523+0000 7f405ed9d640 1 --2- 172.21.15.76:0/1237145153 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f406006a830 0x7f40600722d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4058009870 tx=0x7f405802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a81fc5da5b7078a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:29.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.523+0000 7f405d59a640 1 -- 172.21.15.76:0/1237145153 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40580317c0 con 0x7f406006a830 2023-11-09T03:45:29.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.524+0000 7f405d59a640 1 -- 172.21.15.76:0/1237145153 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4058031920 con 0x7f406006a830 2023-11-09T03:45:29.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.524+0000 7f405d59a640 1 -- 172.21.15.76:0/1237145153 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4058031cb0 con 0x7f406006a830 2023-11-09T03:45:29.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.524+0000 7f4064b05640 1 -- 172.21.15.76:0/1237145153 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f406006a830 msgr2=0x7f40600722d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:29.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.524+0000 7f4064b05640 1 --2- 172.21.15.76:0/1237145153 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f406006a830 0x7f40600722d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4058009870 tx=0x7f405802b860 comp rx=0 tx=0).stop 2023-11-09T03:45:29.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.524+0000 7f4064b05640 1 -- 172.21.15.76:0/1237145153 shutdown_connections 2023-11-09T03:45:29.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.524+0000 7f4064b05640 1 --2- 172.21.15.76:0/1237145153 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f406006a830 0x7f40600722d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:29.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.524+0000 7f4064b05640 1 --2- 172.21.15.76:0/1237145153 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f406015e1b0 0x7f40600ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:29.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.524+0000 7f4064b05640 1 -- 172.21.15.76:0/1237145153 >> 172.21.15.76:0/1237145153 conn(0x7f406006d800 msgr2=0x7f406006dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:29.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.524+0000 7f4064b05640 1 -- 172.21.15.76:0/1237145153 shutdown_connections 2023-11-09T03:45:29.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.524+0000 7f4064b05640 1 -- 172.21.15.76:0/1237145153 wait complete. 2023-11-09T03:45:29.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.525+0000 7f4064b05640 1 Processor -- start 2023-11-09T03:45:29.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.525+0000 7f4064b05640 1 -- start start 2023-11-09T03:45:29.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.526+0000 7f4064b05640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f406006a830 0x7f406014a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:29.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.526+0000 7f4064b05640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f406015e1b0 0x7f40600c9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:29.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.526+0000 7f4064b05640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f406015e590 0x7f406015c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:29.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.526+0000 7f4064b05640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406014a620 con 0x7f406015e1b0 2023-11-09T03:45:29.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.526+0000 7f4064b05640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406014e0e0 con 0x7f406015e590 2023-11-09T03:45:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.526+0000 7f4064b05640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f406014e3c0 con 0x7f406006a830 2023-11-09T03:45:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.526+0000 7f405dd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f406015e1b0 0x7f40600c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.526+0000 7f405dd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f406015e1b0 0x7f40600c9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54524/0 (socket says 172.21.15.76:54524) 2023-11-09T03:45:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.526+0000 7f405dd9b640 1 -- 172.21.15.76:0/1289582931 learned_addr learned my addr 172.21.15.76:0/1289582931 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.526+0000 7f405e59c640 1 --2- 172.21.15.76:0/1289582931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f406006a830 0x7f406014a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.526+0000 7f405ed9d640 1 --2- 172.21.15.76:0/1289582931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f406015e590 0x7f406015c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:29.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.527+0000 7f405dd9b640 1 -- 172.21.15.76:0/1289582931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f406006a830 msgr2=0x7f406014a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:29.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.527+0000 7f405dd9b640 1 --2- 172.21.15.76:0/1289582931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f406006a830 0x7f406014a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:29.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.527+0000 7f405dd9b640 1 -- 172.21.15.76:0/1289582931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f406015e590 msgr2=0x7f406015c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:29.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.527+0000 7f405dd9b640 1 --2- 172.21.15.76:0/1289582931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f406015e590 0x7f406015c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:29.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.527+0000 7f405dd9b640 1 -- 172.21.15.76:0/1289582931 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40580094d0 con 0x7f406015e1b0 2023-11-09T03:45:29.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.527+0000 7f405ed9d640 1 --2- 172.21.15.76:0/1289582931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f406015e590 0x7f406015c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:29.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.527+0000 7f405dd9b640 1 --2- 172.21.15.76:0/1289582931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f406015e1b0 0x7f40600c9160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f405400c400 tx=0x7f405400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:29.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.527+0000 7f404effd640 1 -- 172.21.15.76:0/1289582931 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4054011790 con 0x7f406015e1b0 2023-11-09T03:45:29.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.527+0000 7f404effd640 1 -- 172.21.15.76:0/1289582931 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4054011930 con 0x7f406015e1b0 2023-11-09T03:45:29.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.527+0000 7f4064b05640 1 -- 172.21.15.76:0/1289582931 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f406014e6a0 con 0x7f406015e1b0 2023-11-09T03:45:29.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.527+0000 7f404effd640 1 -- 172.21.15.76:0/1289582931 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4054011ce0 con 0x7f406015e1b0 2023-11-09T03:45:29.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.527+0000 7f4064b05640 1 -- 172.21.15.76:0/1289582931 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f406014ec60 con 0x7f406015e1b0 2023-11-09T03:45:29.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.528+0000 7f4064b05640 1 -- 172.21.15.76:0/1289582931 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4060072650 con 0x7f406015e1b0 2023-11-09T03:45:29.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.528+0000 7f404effd640 1 -- 172.21.15.76:0/1289582931 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f405401c460 con 0x7f406015e1b0 2023-11-09T03:45:29.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.529+0000 7f404effd640 1 --2- 172.21.15.76:0/1289582931 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f404006ae70 0x7f404006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:29.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.529+0000 7f404effd640 1 -- 172.21.15.76:0/1289582931 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f40540892b0 con 0x7f406015e1b0 2023-11-09T03:45:29.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.529+0000 7f405e59c640 1 --2- 172.21.15.76:0/1289582931 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f404006ae70 0x7f404006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:29.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.529+0000 7f405e59c640 1 --2- 172.21.15.76:0/1289582931 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f404006ae70 0x7f404006d330 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4050003b80 tx=0x7f40500098a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:29.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.533+0000 7f404effd640 1 -- 172.21.15.76:0/1289582931 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4054054540 con 0x7f406015e1b0 2023-11-09T03:45:29.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:29.699+0000 7f4064b05640 1 -- 172.21.15.76:0/1289582931 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f40600b4f10 con 0x7f406015e1b0 2023-11-09T03:45:30.312 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:45:30.315 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:45:30.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.395+0000 7f404effd640 1 -- 172.21.15.76:0/1289582931 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) v1 ==== 171+0+0 (secure 0 0 0) 0x7f4054057b60 con 0x7f406015e1b0 2023-11-09T03:45:30.396 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' created 2023-11-09T03:45:30.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.398+0000 7f4064b05640 1 -- 172.21.15.76:0/1289582931 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f404006ae70 msgr2=0x7f404006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:30.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.398+0000 7f4064b05640 1 --2- 172.21.15.76:0/1289582931 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f404006ae70 0x7f404006d330 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4050003b80 tx=0x7f40500098a0 comp rx=0 tx=0).stop 2023-11-09T03:45:30.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.398+0000 7f4064b05640 1 -- 172.21.15.76:0/1289582931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f406015e1b0 msgr2=0x7f40600c9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:30.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.398+0000 7f4064b05640 1 --2- 172.21.15.76:0/1289582931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f406015e1b0 0x7f40600c9160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f405400c400 tx=0x7f405400c8c0 comp rx=0 tx=0).stop 2023-11-09T03:45:30.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.398+0000 7f4064b05640 1 -- 172.21.15.76:0/1289582931 shutdown_connections 2023-11-09T03:45:30.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.398+0000 7f4064b05640 1 --2- 172.21.15.76:0/1289582931 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f404006ae70 0x7f404006d330 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:30.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.398+0000 7f4064b05640 1 --2- 172.21.15.76:0/1289582931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f406015e590 0x7f406015c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:30.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.398+0000 7f4064b05640 1 --2- 172.21.15.76:0/1289582931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f406015e1b0 0x7f40600c9160 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:30.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.398+0000 7f4064b05640 1 --2- 172.21.15.76:0/1289582931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f406006a830 0x7f406014a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:30.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.398+0000 7f4064b05640 1 -- 172.21.15.76:0/1289582931 >> 172.21.15.76:0/1289582931 conn(0x7f406006d800 msgr2=0x7f40600b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:30.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.398+0000 7f4064b05640 1 -- 172.21.15.76:0/1289582931 shutdown_connections 2023-11-09T03:45:30.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.398+0000 7f4064b05640 1 -- 172.21.15.76:0/1289582931 wait complete. 2023-11-09T03:45:30.423 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-09T03:45:30.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.567+0000 7f5e59ca9640 1 Processor -- start 2023-11-09T03:45:30.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.567+0000 7f5e59ca9640 1 -- start start 2023-11-09T03:45:30.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.568+0000 7f5e59ca9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5e5415cfd0 0x7f5e5406c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:30.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.568+0000 7f5e59ca9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5e540cd570 0x7f5e540cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:30.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.568+0000 7f5e59ca9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e540cdf20 con 0x7f5e540cf180 2023-11-09T03:45:30.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.568+0000 7f5e59ca9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e5415d3b0 con 0x7f5e5415cfd0 2023-11-09T03:45:30.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.568+0000 7f5e59ca9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e540cea20 con 0x7f5e540cd570 2023-11-09T03:45:30.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.568+0000 7f5e537fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5e5415cfd0 0x7f5e5406c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:30.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.568+0000 7f5e537fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5e5415cfd0 0x7f5e5406c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54528/0 (socket says 172.21.15.76:54528) 2023-11-09T03:45:30.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.568+0000 7f5e537fe640 1 -- 172.21.15.76:0/2286887669 learned_addr learned my addr 172.21.15.76:0/2286887669 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:30.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.569+0000 7f5e537fe640 1 -- 172.21.15.76:0/2286887669 >> v2:172.21.15.113:3300/0 conn(0x7f5e540cd570 msgr2=0x7f5e540cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:30.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.569+0000 7f5e53fff640 1 --2- 172.21.15.76:0/2286887669 >> v2:172.21.15.113:3300/0 conn(0x7f5e540cd570 0x7f5e540cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:30.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.569+0000 7f5e537fe640 1 --2- 172.21.15.76:0/2286887669 >> v2:172.21.15.113:3300/0 conn(0x7f5e540cd570 0x7f5e540cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:30.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.569+0000 7f5e537fe640 1 -- 172.21.15.76:0/2286887669 >> v1:172.21.15.113:6789/0 conn(0x7f5e540cf180 legacy=0x7f5e54161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:30.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.569+0000 7f5e537fe640 1 -- 172.21.15.76:0/2286887669 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e540cf610 con 0x7f5e5415cfd0 2023-11-09T03:45:30.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.569+0000 7f5e53fff640 1 --2- 172.21.15.76:0/2286887669 >> v2:172.21.15.113:3300/0 conn(0x7f5e540cd570 0x7f5e540cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:30.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.569+0000 7f5e537fe640 1 --2- 172.21.15.76:0/2286887669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e5415cfd0 0x7f5e5406c0a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5e540ce720 tx=0x7f5e4402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e295149fc8c16a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:30.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.569+0000 7f5e527fc640 1 -- 172.21.15.76:0/2286887669 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4402c9f0 con 0x7f5e5415cfd0 2023-11-09T03:45:30.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.570+0000 7f5e527fc640 1 -- 172.21.15.76:0/2286887669 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4402cb50 con 0x7f5e5415cfd0 2023-11-09T03:45:30.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.570+0000 7f5e527fc640 1 -- 172.21.15.76:0/2286887669 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e440343f0 con 0x7f5e5415cfd0 2023-11-09T03:45:30.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.570+0000 7f5e59ca9640 1 -- 172.21.15.76:0/2286887669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e5415cfd0 msgr2=0x7f5e5406c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:30.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.570+0000 7f5e59ca9640 1 --2- 172.21.15.76:0/2286887669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e5415cfd0 0x7f5e5406c0a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5e540ce720 tx=0x7f5e4402be50 comp rx=0 tx=0).stop 2023-11-09T03:45:30.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.570+0000 7f5e59ca9640 1 -- 172.21.15.76:0/2286887669 shutdown_connections 2023-11-09T03:45:30.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.570+0000 7f5e59ca9640 1 --2- 172.21.15.76:0/2286887669 >> v2:172.21.15.113:3300/0 conn(0x7f5e540cd570 0x7f5e540cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:30.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.570+0000 7f5e59ca9640 1 --2- 172.21.15.76:0/2286887669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e5415cfd0 0x7f5e5406c0a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:30.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.570+0000 7f5e59ca9640 1 -- 172.21.15.76:0/2286887669 >> 172.21.15.76:0/2286887669 conn(0x7f5e5406ea20 msgr2=0x7f5e5406ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:30.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.570+0000 7f5e59ca9640 1 -- 172.21.15.76:0/2286887669 shutdown_connections 2023-11-09T03:45:30.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.570+0000 7f5e59ca9640 1 -- 172.21.15.76:0/2286887669 wait complete. 2023-11-09T03:45:30.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.571+0000 7f5e59ca9640 1 Processor -- start 2023-11-09T03:45:30.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.571+0000 7f5e59ca9640 1 -- start start 2023-11-09T03:45:30.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.572+0000 7f5e59ca9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e540cd570 0x7f5e540c7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:30.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.572+0000 7f5e59ca9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e5415cfd0 0x7f5e540b6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:30.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.572+0000 7f5e59ca9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e540b4a90 0x7f5e540b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:30.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.572+0000 7f5e59ca9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e54155410 con 0x7f5e5415cfd0 2023-11-09T03:45:30.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.572+0000 7f5e59ca9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e540c7680 con 0x7f5e540cd570 2023-11-09T03:45:30.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.572+0000 7f5e59ca9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e54154e50 con 0x7f5e540b4a90 2023-11-09T03:45:30.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.572+0000 7f5e52ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e5415cfd0 0x7f5e540b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:30.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.572+0000 7f5e52ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e5415cfd0 0x7f5e540b6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54536/0 (socket says 172.21.15.76:54536) 2023-11-09T03:45:30.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.572+0000 7f5e52ffd640 1 -- 172.21.15.76:0/2052381475 learned_addr learned my addr 172.21.15.76:0/2052381475 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:30.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.572+0000 7f5e537fe640 1 --2- 172.21.15.76:0/2052381475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e540cd570 0x7f5e540c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:30.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.572+0000 7f5e53fff640 1 --2- 172.21.15.76:0/2052381475 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e540b4a90 0x7f5e540b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:30.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.572+0000 7f5e52ffd640 1 -- 172.21.15.76:0/2052381475 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e540b4a90 msgr2=0x7f5e540b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:30.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.572+0000 7f5e52ffd640 1 --2- 172.21.15.76:0/2052381475 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e540b4a90 0x7f5e540b2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:30.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.572+0000 7f5e52ffd640 1 -- 172.21.15.76:0/2052381475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e540cd570 msgr2=0x7f5e540c7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:30.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.572+0000 7f5e52ffd640 1 --2- 172.21.15.76:0/2052381475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e540cd570 0x7f5e540c7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:30.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.572+0000 7f5e52ffd640 1 -- 172.21.15.76:0/2052381475 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e44009b30 con 0x7f5e5415cfd0 2023-11-09T03:45:30.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.573+0000 7f5e52ffd640 1 --2- 172.21.15.76:0/2052381475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e5415cfd0 0x7f5e540b6030 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5e4800b3c0 tx=0x7f5e4800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:30.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.573+0000 7f5e3bfff640 1 -- 172.21.15.76:0/2052381475 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e48011830 con 0x7f5e5415cfd0 2023-11-09T03:45:30.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.573+0000 7f5e3bfff640 1 -- 172.21.15.76:0/2052381475 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e48011990 con 0x7f5e5415cfd0 2023-11-09T03:45:30.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.573+0000 7f5e3bfff640 1 -- 172.21.15.76:0/2052381475 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e48011d80 con 0x7f5e5415cfd0 2023-11-09T03:45:30.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.573+0000 7f5e59ca9640 1 -- 172.21.15.76:0/2052381475 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e540b4f90 con 0x7f5e5415cfd0 2023-11-09T03:45:30.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.573+0000 7f5e59ca9640 1 -- 172.21.15.76:0/2052381475 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e540b5550 con 0x7f5e5415cfd0 2023-11-09T03:45:30.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.574+0000 7f5e59ca9640 1 -- 172.21.15.76:0/2052381475 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e5406d020 con 0x7f5e5415cfd0 2023-11-09T03:45:30.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.574+0000 7f5e3bfff640 1 -- 172.21.15.76:0/2052381475 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5e4801b460 con 0x7f5e5415cfd0 2023-11-09T03:45:30.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.575+0000 7f5e3bfff640 1 --2- 172.21.15.76:0/2052381475 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5e4006aea0 0x7f5e4006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:30.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.575+0000 7f5e3bfff640 1 -- 172.21.15.76:0/2052381475 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f5e48089000 con 0x7f5e5415cfd0 2023-11-09T03:45:30.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.575+0000 7f5e537fe640 1 --2- 172.21.15.76:0/2052381475 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5e4006aea0 0x7f5e4006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:30.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.576+0000 7f5e537fe640 1 --2- 172.21.15.76:0/2052381475 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5e4006aea0 0x7f5e4006d360 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5e44035000 tx=0x7f5e44030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:30.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.579+0000 7f5e3bfff640 1 -- 172.21.15.76:0/2052381475 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5e48054110 con 0x7f5e5415cfd0 2023-11-09T03:45:30.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:30.782+0000 7f5e59ca9640 1 -- 172.21.15.76:0/2052381475 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5e5406b300 con 0x7f5e5415cfd0 2023-11-09T03:45:31.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.398+0000 7f5e3bfff640 1 -- 172.21.15.76:0/2052381475 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f5e48057730 con 0x7f5e5415cfd0 2023-11-09T03:45:31.399 INFO:teuthology.orchestra.run.smithi076.stderr:new fs with metadata pool 2 and data pool 3 2023-11-09T03:45:31.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.401+0000 7f5e59ca9640 1 -- 172.21.15.76:0/2052381475 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5e4006aea0 msgr2=0x7f5e4006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:31.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.401+0000 7f5e59ca9640 1 --2- 172.21.15.76:0/2052381475 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5e4006aea0 0x7f5e4006d360 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5e44035000 tx=0x7f5e44030000 comp rx=0 tx=0).stop 2023-11-09T03:45:31.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.401+0000 7f5e59ca9640 1 -- 172.21.15.76:0/2052381475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e5415cfd0 msgr2=0x7f5e540b6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:31.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.401+0000 7f5e59ca9640 1 --2- 172.21.15.76:0/2052381475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e5415cfd0 0x7f5e540b6030 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5e4800b3c0 tx=0x7f5e4800b880 comp rx=0 tx=0).stop 2023-11-09T03:45:31.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.402+0000 7f5e59ca9640 1 -- 172.21.15.76:0/2052381475 shutdown_connections 2023-11-09T03:45:31.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.402+0000 7f5e59ca9640 1 --2- 172.21.15.76:0/2052381475 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5e4006aea0 0x7f5e4006d360 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:31.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.402+0000 7f5e59ca9640 1 --2- 172.21.15.76:0/2052381475 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e540b4a90 0x7f5e540b2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:31.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.402+0000 7f5e59ca9640 1 --2- 172.21.15.76:0/2052381475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e5415cfd0 0x7f5e540b6030 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:31.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.402+0000 7f5e59ca9640 1 --2- 172.21.15.76:0/2052381475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e540cd570 0x7f5e540c7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:31.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.402+0000 7f5e59ca9640 1 -- 172.21.15.76:0/2052381475 >> 172.21.15.76:0/2052381475 conn(0x7f5e5406ea20 msgr2=0x7f5e5415f070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:31.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.402+0000 7f5e59ca9640 1 -- 172.21.15.76:0/2052381475 shutdown_connections 2023-11-09T03:45:31.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.402+0000 7f5e59ca9640 1 -- 172.21.15.76:0/2052381475 wait complete. 2023-11-09T03:45:31.420 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-11-09T03:45:31.420 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-11-09T03:45:31.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.525+0000 7f2bb8821640 1 Processor -- start 2023-11-09T03:45:31.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.525+0000 7f2bb8821640 1 -- start start 2023-11-09T03:45:31.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.526+0000 7f2bb8821640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2bb40cbfc0 0x7f2bb415e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:31.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.526+0000 7f2bb8821640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb415f7a0 con 0x7f2bb415e1a0 2023-11-09T03:45:31.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.526+0000 7f2bb8821640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb40ce0e0 con 0x7f2bb40b6570 2023-11-09T03:45:31.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.526+0000 7f2bb8821640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bb40b3d80 con 0x7f2bb40cbfc0 2023-11-09T03:45:31.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.527+0000 7f2bb1d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2bb40cbfc0 0x7f2bb415e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:31.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.527+0000 7f2bb259c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2bb415e1a0 0x7f2bb40cb8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:60114/0 (socket says 172.21.15.76:60114) 2023-11-09T03:45:31.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.527+0000 7f2bb259c640 1 -- 172.21.15.76:0/1803099033 learned_addr learned my addr 172.21.15.76:0/1803099033 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:31.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.527+0000 7f2bb1d9b640 1 -- 172.21.15.76:0/1803099033 >> v1:172.21.15.113:6789/0 conn(0x7f2bb40b6570 legacy=0x7f2bb40cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:31.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.527+0000 7f2bb1d9b640 1 -- 172.21.15.76:0/1803099033 >> v1:172.21.15.76:6789/0 conn(0x7f2bb415e1a0 legacy=0x7f2bb40cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:31.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.527+0000 7f2bb1d9b640 1 -- 172.21.15.76:0/1803099033 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb415ef00 con 0x7f2bb40cbfc0 2023-11-09T03:45:31.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.527+0000 7f2bb159a640 1 -- 172.21.15.76:0/1803099033 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 582975471 0 0) 0x7f2bb415f7a0 con 0x7f2bb415e1a0 2023-11-09T03:45:31.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.527+0000 7f2bb1d9b640 1 --2- 172.21.15.76:0/1803099033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bb40cbfc0 0x7f2bb415e8b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2ba80097a0 tx=0x7f2ba802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44a185007debd3f server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:31.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.528+0000 7f2bb159a640 1 -- 172.21.15.76:0/1803099033 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba802f050 con 0x7f2bb40cbfc0 2023-11-09T03:45:31.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.528+0000 7f2bb159a640 1 -- 172.21.15.76:0/1803099033 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba8031700 con 0x7f2bb40cbfc0 2023-11-09T03:45:31.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.528+0000 7f2bb159a640 1 -- 172.21.15.76:0/1803099033 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8031a90 con 0x7f2bb40cbfc0 2023-11-09T03:45:31.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.528+0000 7f2bb8821640 1 -- 172.21.15.76:0/1803099033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bb40cbfc0 msgr2=0x7f2bb415e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:31.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.528+0000 7f2bb8821640 1 --2- 172.21.15.76:0/1803099033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bb40cbfc0 0x7f2bb415e8b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2ba80097a0 tx=0x7f2ba802b7a0 comp rx=0 tx=0).stop 2023-11-09T03:45:31.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.528+0000 7f2bb8821640 1 -- 172.21.15.76:0/1803099033 shutdown_connections 2023-11-09T03:45:31.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.528+0000 7f2bb8821640 1 --2- 172.21.15.76:0/1803099033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bb40cbfc0 0x7f2bb415e8b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:31.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.528+0000 7f2bb8821640 1 -- 172.21.15.76:0/1803099033 >> 172.21.15.76:0/1803099033 conn(0x7f2bb406ea20 msgr2=0x7f2bb406ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:31.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.528+0000 7f2bb8821640 1 -- 172.21.15.76:0/1803099033 shutdown_connections 2023-11-09T03:45:31.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.528+0000 7f2bb8821640 1 -- 172.21.15.76:0/1803099033 wait complete. 2023-11-09T03:45:31.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.530+0000 7f2bb8821640 1 Processor -- start 2023-11-09T03:45:31.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.530+0000 7f2bb8821640 1 -- start start 2023-11-09T03:45:31.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.530+0000 7f2bb8821640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bb40b6570 0x7f2bb41f2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:31.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.530+0000 7f2bb8821640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb40cbfc0 0x7f2bb40c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:31.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.530+0000 7f2bb8821640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bb415e1a0 0x7f2bb40c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:31.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.530+0000 7f2bb8821640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb41505c0 con 0x7f2bb40b6570 2023-11-09T03:45:31.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.530+0000 7f2bb8821640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb40c4e10 con 0x7f2bb40cbfc0 2023-11-09T03:45:31.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.530+0000 7f2bb8821640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb40c50f0 con 0x7f2bb415e1a0 2023-11-09T03:45:31.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.530+0000 7f2bb259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bb40b6570 0x7f2bb41f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:31.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.530+0000 7f2bb259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bb40b6570 0x7f2bb41f2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54556/0 (socket says 172.21.15.76:54556) 2023-11-09T03:45:31.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.530+0000 7f2bb259c640 1 -- 172.21.15.76:0/1852404547 learned_addr learned my addr 172.21.15.76:0/1852404547 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:31.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.531+0000 7f2bb1d9b640 1 --2- 172.21.15.76:0/1852404547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb40cbfc0 0x7f2bb40c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:31.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.531+0000 7f2bb2d9d640 1 --2- 172.21.15.76:0/1852404547 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bb415e1a0 0x7f2bb40c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:31.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.531+0000 7f2bb259c640 1 -- 172.21.15.76:0/1852404547 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bb415e1a0 msgr2=0x7f2bb40c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:31.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.531+0000 7f2bb259c640 1 --2- 172.21.15.76:0/1852404547 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bb415e1a0 0x7f2bb40c5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:31.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.531+0000 7f2bb259c640 1 -- 172.21.15.76:0/1852404547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb40cbfc0 msgr2=0x7f2bb40c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:31.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.531+0000 7f2bb259c640 1 --2- 172.21.15.76:0/1852404547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb40cbfc0 0x7f2bb40c4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:31.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.531+0000 7f2bb259c640 1 -- 172.21.15.76:0/1852404547 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba8009400 con 0x7f2bb40b6570 2023-11-09T03:45:31.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.531+0000 7f2bb1d9b640 1 --2- 172.21.15.76:0/1852404547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb40cbfc0 0x7f2bb40c4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:31.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.531+0000 7f2bb2d9d640 1 --2- 172.21.15.76:0/1852404547 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bb415e1a0 0x7f2bb40c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:31.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.531+0000 7f2bb259c640 1 --2- 172.21.15.76:0/1852404547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bb40b6570 0x7f2bb41f2590 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2ba400c890 tx=0x7f2ba400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:31.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.531+0000 7f2ba2ffd640 1 -- 172.21.15.76:0/1852404547 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba400a640 con 0x7f2bb40b6570 2023-11-09T03:45:31.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.531+0000 7f2ba2ffd640 1 -- 172.21.15.76:0/1852404547 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba400a7c0 con 0x7f2bb40b6570 2023-11-09T03:45:31.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.531+0000 7f2bb8821640 1 -- 172.21.15.76:0/1852404547 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb40c53d0 con 0x7f2bb40b6570 2023-11-09T03:45:31.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.531+0000 7f2ba2ffd640 1 -- 172.21.15.76:0/1852404547 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba400ab70 con 0x7f2bb40b6570 2023-11-09T03:45:31.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.531+0000 7f2bb8821640 1 -- 172.21.15.76:0/1852404547 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb40c5990 con 0x7f2bb40b6570 2023-11-09T03:45:31.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.532+0000 7f2bb8821640 1 -- 172.21.15.76:0/1852404547 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb40cf130 con 0x7f2bb40b6570 2023-11-09T03:45:31.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.532+0000 7f2ba2ffd640 1 -- 172.21.15.76:0/1852404547 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2ba400acd0 con 0x7f2bb40b6570 2023-11-09T03:45:31.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.533+0000 7f2ba2ffd640 1 --2- 172.21.15.76:0/1852404547 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2b9406ae70 0x7f2b9406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:31.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.533+0000 7f2ba2ffd640 1 -- 172.21.15.76:0/1852404547 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ba4088ee0 con 0x7f2bb40b6570 2023-11-09T03:45:31.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.533+0000 7f2bb1d9b640 1 --2- 172.21.15.76:0/1852404547 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2b9406ae70 0x7f2b9406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:31.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.533+0000 7f2bb1d9b640 1 --2- 172.21.15.76:0/1852404547 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2b9406ae70 0x7f2b9406d330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8035000 tx=0x7f2ba8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:31.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.537+0000 7f2ba2ffd640 1 -- 172.21.15.76:0/1852404547 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2bb40cf130 con 0x7f2bb40b6570 2023-11-09T03:45:31.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:31.712+0000 7f2bb8821640 1 -- 172.21.15.76:0/1852404547 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f2bb40cf130 con 0x7f2bb40b6570 2023-11-09T03:45:32.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.385+0000 7f2ba2ffd640 1 -- 172.21.15.76:0/1852404547 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) v1 ==== 157+0+0 (secure 0 0 0) 0x7f2bb40cf130 con 0x7f2bb40b6570 2023-11-09T03:45:32.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.388+0000 7f2bb8821640 1 -- 172.21.15.76:0/1852404547 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2b9406ae70 msgr2=0x7f2b9406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:32.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.388+0000 7f2bb8821640 1 --2- 172.21.15.76:0/1852404547 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2b9406ae70 0x7f2b9406d330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8035000 tx=0x7f2ba8008030 comp rx=0 tx=0).stop 2023-11-09T03:45:32.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.388+0000 7f2bb8821640 1 -- 172.21.15.76:0/1852404547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bb40b6570 msgr2=0x7f2bb41f2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:32.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.388+0000 7f2bb8821640 1 --2- 172.21.15.76:0/1852404547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bb40b6570 0x7f2bb41f2590 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2ba400c890 tx=0x7f2ba400cd50 comp rx=0 tx=0).stop 2023-11-09T03:45:32.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.388+0000 7f2bb8821640 1 -- 172.21.15.76:0/1852404547 shutdown_connections 2023-11-09T03:45:32.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.388+0000 7f2bb8821640 1 --2- 172.21.15.76:0/1852404547 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2b9406ae70 0x7f2b9406d330 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:32.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.388+0000 7f2bb8821640 1 --2- 172.21.15.76:0/1852404547 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bb415e1a0 0x7f2bb40c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:32.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.388+0000 7f2bb8821640 1 --2- 172.21.15.76:0/1852404547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb40cbfc0 0x7f2bb40c4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:32.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.388+0000 7f2bb8821640 1 --2- 172.21.15.76:0/1852404547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bb40b6570 0x7f2bb41f2590 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:32.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.388+0000 7f2bb8821640 1 -- 172.21.15.76:0/1852404547 >> 172.21.15.76:0/1852404547 conn(0x7f2bb406ea20 msgr2=0x7f2bb4153400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:32.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.388+0000 7f2bb8821640 1 -- 172.21.15.76:0/1852404547 shutdown_connections 2023-11-09T03:45:32.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.388+0000 7f2bb8821640 1 -- 172.21.15.76:0/1852404547 wait complete. 2023-11-09T03:45:32.401 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-11-09T03:45:32.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.514+0000 7fa340b76640 1 Processor -- start 2023-11-09T03:45:32.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.514+0000 7fa340b76640 1 -- start start 2023-11-09T03:45:32.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.515+0000 7fa340b76640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa33c0cdbf0 0x7fa33c160a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:32.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.515+0000 7fa340b76640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa33c0d09f0 0x7fa33c0d0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:32.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.515+0000 7fa340b76640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa33c0d2ba0 con 0x7fa33c0d2000 2023-11-09T03:45:32.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.515+0000 7fa340b76640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa33c0b8db0 con 0x7fa33c0d09f0 2023-11-09T03:45:32.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.515+0000 7fa340b76640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c0d2440 con 0x7fa33c0cdbf0 2023-11-09T03:45:32.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.515+0000 7fa33ad9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa33c0d09f0 0x7fa33c0d0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:32.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.515+0000 7fa33ad9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa33c0d09f0 0x7fa33c0d0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54572/0 (socket says 172.21.15.76:54572) 2023-11-09T03:45:32.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.515+0000 7fa33ad9d640 1 -- 172.21.15.76:0/940343629 learned_addr learned my addr 172.21.15.76:0/940343629 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:32.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.516+0000 7fa33a59c640 1 --2- 172.21.15.76:0/940343629 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa33c0cdbf0 0x7fa33c160a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:32.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.516+0000 7fa33ad9d640 1 -- 172.21.15.76:0/940343629 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa33c0cdbf0 msgr2=0x7fa33c160a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:32.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.516+0000 7fa33ad9d640 1 --2- 172.21.15.76:0/940343629 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa33c0cdbf0 0x7fa33c160a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:32.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.516+0000 7fa33ad9d640 1 -- 172.21.15.76:0/940343629 >> v1:172.21.15.113:6789/0 conn(0x7fa33c0d2000 legacy=0x7fa33c0cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:32.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.516+0000 7fa33ad9d640 1 -- 172.21.15.76:0/940343629 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa33c1618d0 con 0x7fa33c0d09f0 2023-11-09T03:45:32.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.516+0000 7fa33ad9d640 1 --2- 172.21.15.76:0/940343629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa33c0d09f0 0x7fa33c0d0e80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa334009e50 tx=0x7fa33402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a64d8d39c43b87 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:32.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.516+0000 7fa339d9b640 1 -- 172.21.15.76:0/940343629 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33402ca70 con 0x7fa33c0d09f0 2023-11-09T03:45:32.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.517+0000 7fa339d9b640 1 -- 172.21.15.76:0/940343629 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33402cbd0 con 0x7fa33c0d09f0 2023-11-09T03:45:32.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.517+0000 7fa339d9b640 1 -- 172.21.15.76:0/940343629 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3340343f0 con 0x7fa33c0d09f0 2023-11-09T03:45:32.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.517+0000 7fa340b76640 1 -- 172.21.15.76:0/940343629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa33c0d09f0 msgr2=0x7fa33c0d0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:32.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.517+0000 7fa340b76640 1 --2- 172.21.15.76:0/940343629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa33c0d09f0 0x7fa33c0d0e80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa334009e50 tx=0x7fa33402bed0 comp rx=0 tx=0).stop 2023-11-09T03:45:32.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.517+0000 7fa340b76640 1 -- 172.21.15.76:0/940343629 shutdown_connections 2023-11-09T03:45:32.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.517+0000 7fa340b76640 1 --2- 172.21.15.76:0/940343629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa33c0d09f0 0x7fa33c0d0e80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:32.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.517+0000 7fa340b76640 1 --2- 172.21.15.76:0/940343629 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa33c0cdbf0 0x7fa33c160a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:32.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.517+0000 7fa340b76640 1 -- 172.21.15.76:0/940343629 >> 172.21.15.76:0/940343629 conn(0x7fa33c071f30 msgr2=0x7fa33c0cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:32.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.517+0000 7fa340b76640 1 -- 172.21.15.76:0/940343629 shutdown_connections 2023-11-09T03:45:32.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.517+0000 7fa340b76640 1 -- 172.21.15.76:0/940343629 wait complete. 2023-11-09T03:45:32.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.519+0000 7fa340b76640 1 Processor -- start 2023-11-09T03:45:32.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.519+0000 7fa340b76640 1 -- start start 2023-11-09T03:45:32.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.519+0000 7fa340b76640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa33c0cdbf0 0x7fa33c1f25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:32.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.519+0000 7fa340b76640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa33c0d09f0 0x7fa33c0cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:32.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.519+0000 7fa340b76640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa33c0d2000 0x7fa33c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:32.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.519+0000 7fa340b76640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c1f2b20 con 0x7fa33c0cdbf0 2023-11-09T03:45:32.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.519+0000 7fa340b76640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c0c9ef0 con 0x7fa33c0d2000 2023-11-09T03:45:32.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.519+0000 7fa340b76640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c1f2d70 con 0x7fa33c0d09f0 2023-11-09T03:45:32.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.519+0000 7fa33a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa33c0cdbf0 0x7fa33c1f25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:32.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.519+0000 7fa33a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa33c0cdbf0 0x7fa33c1f25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54584/0 (socket says 172.21.15.76:54584) 2023-11-09T03:45:32.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.519+0000 7fa33a59c640 1 -- 172.21.15.76:0/399683824 learned_addr learned my addr 172.21.15.76:0/399683824 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:32.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.520+0000 7fa33a59c640 1 -- 172.21.15.76:0/399683824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa33c0d09f0 msgr2=0x7fa33c0cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:32.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.520+0000 7fa331d9b640 1 --2- 172.21.15.76:0/399683824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa33c0d09f0 0x7fa33c0cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:32.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.520+0000 7fa33ad9d640 1 --2- 172.21.15.76:0/399683824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa33c0d2000 0x7fa33c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:32.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.520+0000 7fa33a59c640 1 --2- 172.21.15.76:0/399683824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa33c0d09f0 0x7fa33c0cb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:32.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.520+0000 7fa33a59c640 1 -- 172.21.15.76:0/399683824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa33c0d2000 msgr2=0x7fa33c0b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:32.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.520+0000 7fa33a59c640 1 --2- 172.21.15.76:0/399683824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa33c0d2000 0x7fa33c0b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:32.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.520+0000 7fa33a59c640 1 -- 172.21.15.76:0/399683824 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa334009b30 con 0x7fa33c0cdbf0 2023-11-09T03:45:32.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.520+0000 7fa33a59c640 1 --2- 172.21.15.76:0/399683824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa33c0cdbf0 0x7fa33c1f25e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00d5f0 tx=0x7fa32c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:32.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.520+0000 7fa3337fe640 1 -- 172.21.15.76:0/399683824 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c0125f0 con 0x7fa33c0cdbf0 2023-11-09T03:45:32.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.520+0000 7fa3337fe640 1 -- 172.21.15.76:0/399683824 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa32c012750 con 0x7fa33c0cdbf0 2023-11-09T03:45:32.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.520+0000 7fa340b76640 1 -- 172.21.15.76:0/399683824 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa33c1f3050 con 0x7fa33c0cdbf0 2023-11-09T03:45:32.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.520+0000 7fa3337fe640 1 -- 172.21.15.76:0/399683824 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c012b00 con 0x7fa33c0cdbf0 2023-11-09T03:45:32.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.520+0000 7fa340b76640 1 -- 172.21.15.76:0/399683824 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa33c1f3610 con 0x7fa33c0cdbf0 2023-11-09T03:45:32.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.521+0000 7fa340b76640 1 -- 172.21.15.76:0/399683824 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa33c074290 con 0x7fa33c0cdbf0 2023-11-09T03:45:32.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.521+0000 7fa3337fe640 1 -- 172.21.15.76:0/399683824 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa32c012c60 con 0x7fa33c0cdbf0 2023-11-09T03:45:32.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.522+0000 7fa3337fe640 1 --2- 172.21.15.76:0/399683824 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa31c06ae70 0x7fa31c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:32.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.522+0000 7fa3337fe640 1 -- 172.21.15.76:0/399683824 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5935+0+0 (secure 0 0 0) 0x7fa32c089be0 con 0x7fa33c0cdbf0 2023-11-09T03:45:32.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.522+0000 7fa331d9b640 1 --2- 172.21.15.76:0/399683824 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa31c06ae70 0x7fa31c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:32.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.522+0000 7fa331d9b640 1 --2- 172.21.15.76:0/399683824 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa31c06ae70 0x7fa31c06d330 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa33c0c7990 tx=0x7fa324009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:32.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.526+0000 7fa3337fe640 1 -- 172.21.15.76:0/399683824 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa32c054bb0 con 0x7fa33c0cdbf0 2023-11-09T03:45:32.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:32.694+0000 7fa340b76640 1 -- 172.21.15.76:0/399683824 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa33c0d2650 con 0x7fa33c0cdbf0 2023-11-09T03:45:34.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.423+0000 7fa3337fe640 1 -- 172.21.15.76:0/399683824 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) v1 ==== 228+0+0 (secure 0 0 0) 0x7fa32c0581d0 con 0x7fa33c0cdbf0 2023-11-09T03:45:34.424 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data_ec' created 2023-11-09T03:45:34.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.427+0000 7fa340b76640 1 -- 172.21.15.76:0/399683824 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa31c06ae70 msgr2=0x7fa31c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:34.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.427+0000 7fa340b76640 1 --2- 172.21.15.76:0/399683824 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa31c06ae70 0x7fa31c06d330 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa33c0c7990 tx=0x7fa324009040 comp rx=0 tx=0).stop 2023-11-09T03:45:34.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.427+0000 7fa340b76640 1 -- 172.21.15.76:0/399683824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa33c0cdbf0 msgr2=0x7fa33c1f25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:34.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.427+0000 7fa340b76640 1 --2- 172.21.15.76:0/399683824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa33c0cdbf0 0x7fa33c1f25e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00d5f0 tx=0x7fa32c00dab0 comp rx=0 tx=0).stop 2023-11-09T03:45:34.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.427+0000 7fa340b76640 1 -- 172.21.15.76:0/399683824 shutdown_connections 2023-11-09T03:45:34.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.427+0000 7fa340b76640 1 --2- 172.21.15.76:0/399683824 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa31c06ae70 0x7fa31c06d330 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:34.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.427+0000 7fa340b76640 1 --2- 172.21.15.76:0/399683824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa33c0d2000 0x7fa33c0b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:34.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.427+0000 7fa340b76640 1 --2- 172.21.15.76:0/399683824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa33c0d09f0 0x7fa33c0cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:34.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.427+0000 7fa340b76640 1 --2- 172.21.15.76:0/399683824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa33c0cdbf0 0x7fa33c1f25e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:34.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.427+0000 7fa340b76640 1 -- 172.21.15.76:0/399683824 >> 172.21.15.76:0/399683824 conn(0x7fa33c071f30 msgr2=0x7fa33c160850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:34.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.427+0000 7fa340b76640 1 -- 172.21.15.76:0/399683824 shutdown_connections 2023-11-09T03:45:34.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.427+0000 7fa340b76640 1 -- 172.21.15.76:0/399683824 wait complete. 2023-11-09T03:45:34.446 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-11-09T03:45:34.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.554+0000 7fb971fce640 1 Processor -- start 2023-11-09T03:45:34.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.554+0000 7fb971fce640 1 -- start start 2023-11-09T03:45:34.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.555+0000 7fb971fce640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb96c15e1a0 0x7fb96c15e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:34.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.555+0000 7fb971fce640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb96c15f7a0 con 0x7fb96c0cbfc0 2023-11-09T03:45:34.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.555+0000 7fb971fce640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb96c0ce0e0 con 0x7fb96c0b6570 2023-11-09T03:45:34.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.555+0000 7fb971fce640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c0b3d80 con 0x7fb96c15e1a0 2023-11-09T03:45:34.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.555+0000 7fb96bfff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb96c0cbfc0 0x7fb96c0cf670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50986/0 (socket says 172.21.15.76:50986) 2023-11-09T03:45:34.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.555+0000 7fb96bfff640 1 -- 172.21.15.76:0/1530305313 learned_addr learned my addr 172.21.15.76:0/1530305313 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:34.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.555+0000 7fb96b7fe640 1 --2- 172.21.15.76:0/1530305313 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb96c15e1a0 0x7fb96c15e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:34.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.555+0000 7fb96affd640 1 -- 172.21.15.76:0/1530305313 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1225113679 0 0) 0x7fb96c15f7a0 con 0x7fb96c0cbfc0 2023-11-09T03:45:34.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.555+0000 7fb96affd640 1 -- 172.21.15.76:0/1530305313 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb958003520 con 0x7fb96c0cbfc0 2023-11-09T03:45:34.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.556+0000 7fb96affd640 1 -- 172.21.15.76:0/1530305313 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 191767155 0 0) 0x7fb96c0ce0e0 con 0x7fb96c0b6570 2023-11-09T03:45:34.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.556+0000 7fb96affd640 1 -- 172.21.15.76:0/1530305313 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb96c15f7a0 con 0x7fb96c0b6570 2023-11-09T03:45:34.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.556+0000 7fb96affd640 1 -- 172.21.15.76:0/1530305313 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2439036801 0 0) 0x7fb958003520 con 0x7fb96c0cbfc0 2023-11-09T03:45:34.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.556+0000 7fb96affd640 1 -- 172.21.15.76:0/1530305313 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb96c15e1a0 msgr2=0x7fb96c15e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:34.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.556+0000 7fb96affd640 1 --2- 172.21.15.76:0/1530305313 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb96c15e1a0 0x7fb96c15e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:34.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.556+0000 7fb96affd640 1 -- 172.21.15.76:0/1530305313 >> v1:172.21.15.113:6789/0 conn(0x7fb96c0b6570 legacy=0x7fb96c0cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:34.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.556+0000 7fb96affd640 1 -- 172.21.15.76:0/1530305313 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96c15ef00 con 0x7fb96c0cbfc0 2023-11-09T03:45:34.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.556+0000 7fb96affd640 1 -- 172.21.15.76:0/1530305313 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fb964001970 con 0x7fb96c0cbfc0 2023-11-09T03:45:34.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.556+0000 7fb96affd640 1 -- 172.21.15.76:0/1530305313 >> v1:172.21.15.76:6789/0 conn(0x7fb96c0cbfc0 legacy=0x7fb96c0cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:34.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.556+0000 7fb96affd640 1 --2- 172.21.15.76:0/1530305313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb958003b30 0x7fb958003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:34.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.556+0000 7fb96affd640 1 --2- 172.21.15.76:0/1530305313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9580047c0 0x7fb958004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:34.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.556+0000 7fb96affd640 1 --2- 172.21.15.76:0/1530305313 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb96c15e1a0 0x7fb958005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:34.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.556+0000 7fb96affd640 1 -- 172.21.15.76:0/1530305313 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c0b3d80 con 0x7fb958003b30 2023-11-09T03:45:34.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.556+0000 7fb96affd640 1 -- 172.21.15.76:0/1530305313 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95800bb80 con 0x7fb9580047c0 2023-11-09T03:45:34.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.556+0000 7fb96affd640 1 -- 172.21.15.76:0/1530305313 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb95800bd40 con 0x7fb96c15e1a0 2023-11-09T03:45:34.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.556+0000 7fb96b7fe640 1 --2- 172.21.15.76:0/1530305313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb958003b30 0x7fb958003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:34.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.556+0000 7fb96bfff640 1 --2- 172.21.15.76:0/1530305313 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb96c15e1a0 0x7fb958005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:34.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.556+0000 7fb963fff640 1 --2- 172.21.15.76:0/1530305313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9580047c0 0x7fb958004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:34.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.556+0000 7fb96b7fe640 1 -- 172.21.15.76:0/1530305313 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb96c15e1a0 msgr2=0x7fb958005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:34.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.556+0000 7fb96b7fe640 1 --2- 172.21.15.76:0/1530305313 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb96c15e1a0 0x7fb958005280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:34.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.556+0000 7fb96b7fe640 1 -- 172.21.15.76:0/1530305313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9580047c0 msgr2=0x7fb958004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:34.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.556+0000 7fb96b7fe640 1 --2- 172.21.15.76:0/1530305313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9580047c0 0x7fb958004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:34.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.556+0000 7fb96b7fe640 1 -- 172.21.15.76:0/1530305313 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb958003520 con 0x7fb958003b30 2023-11-09T03:45:34.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.557+0000 7fb96b7fe640 1 --2- 172.21.15.76:0/1530305313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb958003b30 0x7fb958003f30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb95c009650 tx=0x7fb9580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e033cb3a4ffb24e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:34.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.557+0000 7fb96affd640 1 -- 172.21.15.76:0/1530305313 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c030ac0 con 0x7fb958003b30 2023-11-09T03:45:34.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.557+0000 7fb96affd640 1 -- 172.21.15.76:0/1530305313 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95c030c20 con 0x7fb958003b30 2023-11-09T03:45:34.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.557+0000 7fb96affd640 1 -- 172.21.15.76:0/1530305313 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c038440 con 0x7fb958003b30 2023-11-09T03:45:34.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.558+0000 7fb971fce640 1 -- 172.21.15.76:0/1530305313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb958003b30 msgr2=0x7fb958003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:34.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.558+0000 7fb971fce640 1 --2- 172.21.15.76:0/1530305313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb958003b30 0x7fb958003f30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb95c009650 tx=0x7fb9580057f0 comp rx=0 tx=0).stop 2023-11-09T03:45:34.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.558+0000 7fb96bfff640 1 -- 172.21.15.76:0/1530305313 reap_dead start 2023-11-09T03:45:34.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.558+0000 7fb971fce640 1 -- 172.21.15.76:0/1530305313 shutdown_connections 2023-11-09T03:45:34.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.558+0000 7fb971fce640 1 -- 172.21.15.76:0/1530305313 >> 172.21.15.76:0/1530305313 conn(0x7fb96c06ea20 msgr2=0x7fb96c06ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:34.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.558+0000 7fb971fce640 1 -- 172.21.15.76:0/1530305313 shutdown_connections 2023-11-09T03:45:34.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.558+0000 7fb971fce640 1 -- 172.21.15.76:0/1530305313 wait complete. 2023-11-09T03:45:34.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.559+0000 7fb971fce640 1 Processor -- start 2023-11-09T03:45:34.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.559+0000 7fb971fce640 1 -- start start 2023-11-09T03:45:34.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.560+0000 7fb971fce640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb958003b30 0x7fb96c0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:34.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.560+0000 7fb971fce640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb96c0c81f0 0x7fb96c0c5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:34.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.560+0000 7fb971fce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb96c0c5cc0 0x7fb96c0b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:34.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.560+0000 7fb971fce640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c0c7030 con 0x7fb958003b30 2023-11-09T03:45:34.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.560+0000 7fb971fce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c0c4770 con 0x7fb96c0c5cc0 2023-11-09T03:45:34.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.560+0000 7fb971fce640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c0c49c0 con 0x7fb96c0c81f0 2023-11-09T03:45:34.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.560+0000 7fb96b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb958003b30 0x7fb96c0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:34.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.560+0000 7fb96b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb958003b30 0x7fb96c0c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34092/0 (socket says 172.21.15.76:34092) 2023-11-09T03:45:34.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.560+0000 7fb96b7fe640 1 -- 172.21.15.76:0/1734246607 learned_addr learned my addr 172.21.15.76:0/1734246607 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:34.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.560+0000 7fb96bfff640 1 --2- 172.21.15.76:0/1734246607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb96c0c5cc0 0x7fb96c0b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:34.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.560+0000 7fb963fff640 1 --2- 172.21.15.76:0/1734246607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb96c0c81f0 0x7fb96c0c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:34.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.560+0000 7fb96b7fe640 1 -- 172.21.15.76:0/1734246607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb96c0c81f0 msgr2=0x7fb96c0c5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:34.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.560+0000 7fb96b7fe640 1 --2- 172.21.15.76:0/1734246607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb96c0c81f0 0x7fb96c0c5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:34.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.560+0000 7fb96b7fe640 1 -- 172.21.15.76:0/1734246607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb96c0c5cc0 msgr2=0x7fb96c0b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:34.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.560+0000 7fb96b7fe640 1 --2- 172.21.15.76:0/1734246607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb96c0c5cc0 0x7fb96c0b3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:34.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.560+0000 7fb96b7fe640 1 -- 172.21.15.76:0/1734246607 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96c0c4ca0 con 0x7fb958003b30 2023-11-09T03:45:34.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.560+0000 7fb96b7fe640 1 --2- 172.21.15.76:0/1734246607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb958003b30 0x7fb96c0c7cb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb95c030730 tx=0x7fb95c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:34.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.560+0000 7fb968ff9640 1 -- 172.21.15.76:0/1734246607 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c003a20 con 0x7fb958003b30 2023-11-09T03:45:34.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.560+0000 7fb968ff9640 1 -- 172.21.15.76:0/1734246607 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95c003ba0 con 0x7fb958003b30 2023-11-09T03:45:34.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.560+0000 7fb971fce640 1 -- 172.21.15.76:0/1734246607 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb96c0c4f80 con 0x7fb958003b30 2023-11-09T03:45:34.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.560+0000 7fb968ff9640 1 -- 172.21.15.76:0/1734246607 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c044450 con 0x7fb958003b30 2023-11-09T03:45:34.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.560+0000 7fb971fce640 1 -- 172.21.15.76:0/1734246607 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb96c0b4430 con 0x7fb958003b30 2023-11-09T03:45:34.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.561+0000 7fb971fce640 1 -- 172.21.15.76:0/1734246607 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb96c0c60a0 con 0x7fb958003b30 2023-11-09T03:45:34.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.562+0000 7fb968ff9640 1 -- 172.21.15.76:0/1734246607 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb95c0445b0 con 0x7fb958003b30 2023-11-09T03:45:34.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.562+0000 7fb968ff9640 1 --2- 172.21.15.76:0/1734246607 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb94c06aea0 0x7fb94c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:34.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.562+0000 7fb968ff9640 1 -- 172.21.15.76:0/1734246607 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6428+0+0 (secure 0 0 0) 0x7fb95c0afb40 con 0x7fb958003b30 2023-11-09T03:45:34.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.562+0000 7fb963fff640 1 --2- 172.21.15.76:0/1734246607 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb94c06aea0 0x7fb94c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:34.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.563+0000 7fb963fff640 1 --2- 172.21.15.76:0/1734246607 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb94c06aea0 0x7fb94c06d360 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb954007fd0 tx=0x7fb954009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:34.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.567+0000 7fb968ff9640 1 -- 172.21.15.76:0/1734246607 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb95c07a920 con 0x7fb958003b30 2023-11-09T03:45:34.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:34.732+0000 7fb971fce640 1 -- 172.21.15.76:0/1734246607 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fb96c04e490 con 0x7fb958003b30 2023-11-09T03:45:35.366 INFO:tasks.ceph.mds.b.smithi113.stderr:2023-11-09T03:45:35.367+0000 7f7e01729640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:45:35.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.428+0000 7fb968ff9640 1 -- 172.21.15.76:0/1734246607 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb95c07df40 con 0x7fb958003b30 2023-11-09T03:45:35.429 INFO:teuthology.orchestra.run.smithi076.stderr:set pool 4 allow_ec_overwrites to true 2023-11-09T03:45:35.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.431+0000 7fb971fce640 1 -- 172.21.15.76:0/1734246607 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb94c06aea0 msgr2=0x7fb94c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:35.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.431+0000 7fb971fce640 1 --2- 172.21.15.76:0/1734246607 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb94c06aea0 0x7fb94c06d360 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb954007fd0 tx=0x7fb954009040 comp rx=0 tx=0).stop 2023-11-09T03:45:35.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.432+0000 7fb971fce640 1 -- 172.21.15.76:0/1734246607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb958003b30 msgr2=0x7fb96c0c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:35.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.432+0000 7fb971fce640 1 --2- 172.21.15.76:0/1734246607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb958003b30 0x7fb96c0c7cb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb95c030730 tx=0x7fb95c004730 comp rx=0 tx=0).stop 2023-11-09T03:45:35.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.432+0000 7fb971fce640 1 -- 172.21.15.76:0/1734246607 shutdown_connections 2023-11-09T03:45:35.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.432+0000 7fb971fce640 1 --2- 172.21.15.76:0/1734246607 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb94c06aea0 0x7fb94c06d360 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:35.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.432+0000 7fb971fce640 1 --2- 172.21.15.76:0/1734246607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb96c0c5cc0 0x7fb96c0b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:35.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.432+0000 7fb971fce640 1 --2- 172.21.15.76:0/1734246607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb96c0c81f0 0x7fb96c0c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:35.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.432+0000 7fb971fce640 1 --2- 172.21.15.76:0/1734246607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb958003b30 0x7fb96c0c7cb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:35.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.432+0000 7fb971fce640 1 -- 172.21.15.76:0/1734246607 >> 172.21.15.76:0/1734246607 conn(0x7fb96c06ea20 msgr2=0x7fb96c0cd140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:35.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.432+0000 7fb971fce640 1 -- 172.21.15.76:0/1734246607 shutdown_connections 2023-11-09T03:45:35.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.433+0000 7fb971fce640 1 -- 172.21.15.76:0/1734246607 wait complete. 2023-11-09T03:45:35.457 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-11-09T03:45:35.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.560+0000 7f76ad84f640 1 Processor -- start 2023-11-09T03:45:35.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.560+0000 7f76ad84f640 1 -- start start 2023-11-09T03:45:35.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.561+0000 7f76ad84f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76a806a830 0x7f76a80ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:35.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.561+0000 7f76ad84f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f76a815e590 0x7f76a80722d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:35.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.561+0000 7f76ad84f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a80ca620 con 0x7f76a815e1b0 2023-11-09T03:45:35.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.561+0000 7f76ad84f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a806e7e0 con 0x7f76a815e590 2023-11-09T03:45:35.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.561+0000 7f76ad84f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a80cb060 con 0x7f76a806a830 2023-11-09T03:45:35.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.561+0000 7f76a77fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f76a815e590 0x7f76a80722d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:35.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.561+0000 7f76a6ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f76a815e1b0 0x7f76a806a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50990/0 (socket says 172.21.15.76:50990) 2023-11-09T03:45:35.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.561+0000 7f76a6ffd640 1 -- 172.21.15.76:0/681908125 learned_addr learned my addr 172.21.15.76:0/681908125 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:35.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.562+0000 7f76a67fc640 1 --2- 172.21.15.76:0/681908125 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76a806a830 0x7f76a80ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:35.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.562+0000 7f76a77fe640 1 -- 172.21.15.76:0/681908125 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76a806a830 msgr2=0x7f76a80ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:35.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.562+0000 7f76a77fe640 1 --2- 172.21.15.76:0/681908125 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76a806a830 0x7f76a80ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:35.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.562+0000 7f76a77fe640 1 -- 172.21.15.76:0/681908125 >> v1:172.21.15.76:6789/0 conn(0x7f76a815e1b0 legacy=0x7f76a806a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:35.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.562+0000 7f76a77fe640 1 -- 172.21.15.76:0/681908125 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a80ca060 con 0x7f76a815e590 2023-11-09T03:45:35.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.562+0000 7f76a5ffb640 1 -- 172.21.15.76:0/681908125 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2578370364 0 0) 0x7f76a80ca620 con 0x7f76a815e1b0 2023-11-09T03:45:35.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.562+0000 7f76a67fc640 1 --2- 172.21.15.76:0/681908125 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76a806a830 0x7f76a80ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:35.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.562+0000 7f76a77fe640 1 --2- 172.21.15.76:0/681908125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76a815e590 0x7f76a80722d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f76a00097d0 tx=0x7f76a002b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f158001a4b8160db server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:35.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.562+0000 7f76a5ffb640 1 -- 172.21.15.76:0/681908125 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a0031770 con 0x7f76a815e590 2023-11-09T03:45:35.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.562+0000 7f76a5ffb640 1 -- 172.21.15.76:0/681908125 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a00318d0 con 0x7f76a815e590 2023-11-09T03:45:35.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.562+0000 7f76a5ffb640 1 -- 172.21.15.76:0/681908125 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a0031c60 con 0x7f76a815e590 2023-11-09T03:45:35.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.563+0000 7f76ad84f640 1 -- 172.21.15.76:0/681908125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76a815e590 msgr2=0x7f76a80722d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:35.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.563+0000 7f76ad84f640 1 --2- 172.21.15.76:0/681908125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76a815e590 0x7f76a80722d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f76a00097d0 tx=0x7f76a002b810 comp rx=0 tx=0).stop 2023-11-09T03:45:35.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.563+0000 7f76ad84f640 1 -- 172.21.15.76:0/681908125 shutdown_connections 2023-11-09T03:45:35.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.563+0000 7f76ad84f640 1 --2- 172.21.15.76:0/681908125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76a815e590 0x7f76a80722d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:35.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.563+0000 7f76ad84f640 1 --2- 172.21.15.76:0/681908125 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76a806a830 0x7f76a80ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:35.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.563+0000 7f76ad84f640 1 -- 172.21.15.76:0/681908125 >> 172.21.15.76:0/681908125 conn(0x7f76a806d800 msgr2=0x7f76a806dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:35.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.563+0000 7f76ad84f640 1 -- 172.21.15.76:0/681908125 shutdown_connections 2023-11-09T03:45:35.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.563+0000 7f76ad84f640 1 -- 172.21.15.76:0/681908125 wait complete. 2023-11-09T03:45:35.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.564+0000 7f76ad84f640 1 Processor -- start 2023-11-09T03:45:35.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.564+0000 7f76ad84f640 1 -- start start 2023-11-09T03:45:35.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.565+0000 7f76ad84f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76a806a830 0x7f76a80c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:35.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.565+0000 7f76ad84f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76a815e1b0 0x7f76a814df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:35.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.565+0000 7f76ad84f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76a815e590 0x7f76a80c3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:35.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.565+0000 7f76ad84f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a814e460 con 0x7f76a815e1b0 2023-11-09T03:45:35.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.565+0000 7f76ad84f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a81f1890 con 0x7f76a815e590 2023-11-09T03:45:35.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.565+0000 7f76ad84f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a81f1b70 con 0x7f76a806a830 2023-11-09T03:45:35.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.565+0000 7f76a67fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76a815e1b0 0x7f76a814df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:35.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.565+0000 7f76a67fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76a815e1b0 0x7f76a814df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34100/0 (socket says 172.21.15.76:34100) 2023-11-09T03:45:35.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.565+0000 7f76a67fc640 1 -- 172.21.15.76:0/2145740070 learned_addr learned my addr 172.21.15.76:0/2145740070 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:35.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.565+0000 7f76a67fc640 1 -- 172.21.15.76:0/2145740070 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76a806a830 msgr2=0x7f76a80c7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:35.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.565+0000 7f76a67fc640 1 --2- 172.21.15.76:0/2145740070 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76a806a830 0x7f76a80c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:35.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.565+0000 7f76a67fc640 1 -- 172.21.15.76:0/2145740070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76a815e590 msgr2=0x7f76a80c3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:35.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.565+0000 7f76a67fc640 1 --2- 172.21.15.76:0/2145740070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76a815e590 0x7f76a80c3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:35.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.565+0000 7f76a67fc640 1 -- 172.21.15.76:0/2145740070 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a0009400 con 0x7f76a815e1b0 2023-11-09T03:45:35.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.566+0000 7f76a67fc640 1 --2- 172.21.15.76:0/2145740070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76a815e1b0 0x7f76a814df20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f769c00c7f0 tx=0x7f769c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:35.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.566+0000 7f768ffff640 1 -- 172.21.15.76:0/2145740070 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c007820 con 0x7f76a815e1b0 2023-11-09T03:45:35.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.566+0000 7f768ffff640 1 -- 172.21.15.76:0/2145740070 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769c007980 con 0x7f76a815e1b0 2023-11-09T03:45:35.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.566+0000 7f76ad84f640 1 -- 172.21.15.76:0/2145740070 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a81f1e50 con 0x7f76a815e1b0 2023-11-09T03:45:35.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.566+0000 7f768ffff640 1 -- 172.21.15.76:0/2145740070 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c007cf0 con 0x7f76a815e1b0 2023-11-09T03:45:35.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.566+0000 7f76ad84f640 1 -- 172.21.15.76:0/2145740070 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a81f2410 con 0x7f76a815e1b0 2023-11-09T03:45:35.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.567+0000 7f76ad84f640 1 -- 172.21.15.76:0/2145740070 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a80c3a80 con 0x7f76a815e1b0 2023-11-09T03:45:35.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.567+0000 7f768ffff640 1 -- 172.21.15.76:0/2145740070 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f769c0113f0 con 0x7f76a815e1b0 2023-11-09T03:45:35.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.567+0000 7f768ffff640 1 --2- 172.21.15.76:0/2145740070 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f769406ae70 0x7f769406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:35.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.567+0000 7f768ffff640 1 -- 172.21.15.76:0/2145740070 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f769c089ca0 con 0x7f76a815e1b0 2023-11-09T03:45:35.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.568+0000 7f76a6ffd640 1 --2- 172.21.15.76:0/2145740070 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f769406ae70 0x7f769406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:35.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.568+0000 7f76a6ffd640 1 --2- 172.21.15.76:0/2145740070 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f769406ae70 0x7f769406d330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7698001560 tx=0x7f7698007ca0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:35.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.572+0000 7f768ffff640 1 -- 172.21.15.76:0/2145740070 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f769c054a00 con 0x7f76a815e1b0 2023-11-09T03:45:35.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:35.779+0000 7f76ad84f640 1 -- 172.21.15.76:0/2145740070 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f76a806a2c0 con 0x7f76a815e1b0 2023-11-09T03:45:36.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.432+0000 7f768ffff640 1 -- 172.21.15.76:0/2145740070 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f769c058020 con 0x7f76a815e1b0 2023-11-09T03:45:36.433 INFO:teuthology.orchestra.run.smithi076.stderr:added data pool 4 to fsmap 2023-11-09T03:45:36.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.435+0000 7f76ad84f640 1 -- 172.21.15.76:0/2145740070 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f769406ae70 msgr2=0x7f769406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:36.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.435+0000 7f76ad84f640 1 --2- 172.21.15.76:0/2145740070 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f769406ae70 0x7f769406d330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7698001560 tx=0x7f7698007ca0 comp rx=0 tx=0).stop 2023-11-09T03:45:36.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.435+0000 7f76ad84f640 1 -- 172.21.15.76:0/2145740070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76a815e1b0 msgr2=0x7f76a814df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:36.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.435+0000 7f76ad84f640 1 --2- 172.21.15.76:0/2145740070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76a815e1b0 0x7f76a814df20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f769c00c7f0 tx=0x7f769c00ccb0 comp rx=0 tx=0).stop 2023-11-09T03:45:36.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.435+0000 7f76ad84f640 1 -- 172.21.15.76:0/2145740070 shutdown_connections 2023-11-09T03:45:36.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.435+0000 7f76ad84f640 1 --2- 172.21.15.76:0/2145740070 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f769406ae70 0x7f769406d330 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:36.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.435+0000 7f76ad84f640 1 --2- 172.21.15.76:0/2145740070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76a815e590 0x7f76a80c3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:36.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.435+0000 7f76ad84f640 1 --2- 172.21.15.76:0/2145740070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76a815e1b0 0x7f76a814df20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:36.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.435+0000 7f76ad84f640 1 --2- 172.21.15.76:0/2145740070 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76a806a830 0x7f76a80c7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:36.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.435+0000 7f76ad84f640 1 -- 172.21.15.76:0/2145740070 >> 172.21.15.76:0/2145740070 conn(0x7f76a806d800 msgr2=0x7f76a815be80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:36.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.435+0000 7f76ad84f640 1 -- 172.21.15.76:0/2145740070 shutdown_connections 2023-11-09T03:45:36.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.435+0000 7f76ad84f640 1 -- 172.21.15.76:0/2145740070 wait complete. 2023-11-09T03:45:36.447 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:45:36.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.548+0000 7fc575761640 1 Processor -- start 2023-11-09T03:45:36.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.548+0000 7fc575761640 1 -- start start 2023-11-09T03:45:36.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.549+0000 7fc575761640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc570159c70 0x7fc5700ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:36.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.549+0000 7fc575761640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5700b0ae0 0x7fc57015d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:36.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.549+0000 7fc575761640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5700b4110 con 0x7fc57006db20 2023-11-09T03:45:36.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.549+0000 7fc575761640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5700b49b0 con 0x7fc570159c70 2023-11-09T03:45:36.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.549+0000 7fc575761640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5700b43f0 con 0x7fc5700b0ae0 2023-11-09T03:45:36.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.549+0000 7fc56e7fc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc57006db20 0x7fc5700b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:51000/0 (socket says 172.21.15.76:51000) 2023-11-09T03:45:36.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.549+0000 7fc56e7fc640 1 -- 172.21.15.76:0/3557339507 learned_addr learned my addr 172.21.15.76:0/3557339507 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:36.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.549+0000 7fc56dffb640 1 -- 172.21.15.76:0/3557339507 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 119048805 0 0) 0x7fc5700b4110 con 0x7fc57006db20 2023-11-09T03:45:36.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.549+0000 7fc56effd640 1 --2- 172.21.15.76:0/3557339507 >> v2:172.21.15.113:3300/0 conn(0x7fc570159c70 0x7fc5700ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:36.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.549+0000 7fc56f7fe640 1 --2- 172.21.15.76:0/3557339507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5700b0ae0 0x7fc57015d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:36.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.549+0000 7fc56dffb640 1 -- 172.21.15.76:0/3557339507 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc558003520 con 0x7fc57006db20 2023-11-09T03:45:36.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.549+0000 7fc56dffb640 1 -- 172.21.15.76:0/3557339507 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4277395519 0 0) 0x7fc558003520 con 0x7fc57006db20 2023-11-09T03:45:36.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.549+0000 7fc56dffb640 1 -- 172.21.15.76:0/3557339507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5700b0ae0 msgr2=0x7fc57015d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:36.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.549+0000 7fc56dffb640 1 --2- 172.21.15.76:0/3557339507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5700b0ae0 0x7fc57015d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:36.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.549+0000 7fc56dffb640 1 -- 172.21.15.76:0/3557339507 >> v2:172.21.15.113:3300/0 conn(0x7fc570159c70 msgr2=0x7fc5700ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:36.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.549+0000 7fc56dffb640 1 --2- 172.21.15.76:0/3557339507 >> v2:172.21.15.113:3300/0 conn(0x7fc570159c70 0x7fc5700ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:36.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.549+0000 7fc56dffb640 1 -- 172.21.15.76:0/3557339507 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5700b46d0 con 0x7fc57006db20 2023-11-09T03:45:36.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.549+0000 7fc56dffb640 1 -- 172.21.15.76:0/3557339507 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fc564001670 con 0x7fc57006db20 2023-11-09T03:45:36.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.550+0000 7fc56dffb640 1 -- 172.21.15.76:0/3557339507 >> v1:172.21.15.76:6789/0 conn(0x7fc57006db20 legacy=0x7fc5700b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:36.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.550+0000 7fc56dffb640 1 --2- 172.21.15.76:0/3557339507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5700b0ae0 0x7fc558003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:36.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.550+0000 7fc56dffb640 1 --2- 172.21.15.76:0/3557339507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc558003ef0 0x7fc5580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:36.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.550+0000 7fc56dffb640 1 --2- 172.21.15.76:0/3557339507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc558004d00 0x7fc558005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:36.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.550+0000 7fc56dffb640 1 -- 172.21.15.76:0/3557339507 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5700b49b0 con 0x7fc558004d00 2023-11-09T03:45:36.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.550+0000 7fc56dffb640 1 -- 172.21.15.76:0/3557339507 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5700b43f0 con 0x7fc558003ef0 2023-11-09T03:45:36.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.550+0000 7fc56dffb640 1 -- 172.21.15.76:0/3557339507 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc55800b980 con 0x7fc5700b0ae0 2023-11-09T03:45:36.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.550+0000 7fc56f7fe640 1 --2- 172.21.15.76:0/3557339507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc558004d00 0x7fc558005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:36.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.550+0000 7fc56effd640 1 --2- 172.21.15.76:0/3557339507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5700b0ae0 0x7fc558003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:36.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.550+0000 7fc56e7fc640 1 --2- 172.21.15.76:0/3557339507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc558003ef0 0x7fc5580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:36.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.550+0000 7fc56f7fe640 1 -- 172.21.15.76:0/3557339507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5700b0ae0 msgr2=0x7fc558003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:36.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.550+0000 7fc56f7fe640 1 --2- 172.21.15.76:0/3557339507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5700b0ae0 0x7fc558003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:36.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.550+0000 7fc56f7fe640 1 -- 172.21.15.76:0/3557339507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc558003ef0 msgr2=0x7fc5580047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:36.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.550+0000 7fc56f7fe640 1 --2- 172.21.15.76:0/3557339507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc558003ef0 0x7fc5580047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:36.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.550+0000 7fc56f7fe640 1 -- 172.21.15.76:0/3557339507 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc558003520 con 0x7fc558004d00 2023-11-09T03:45:36.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.551+0000 7fc56f7fe640 1 --2- 172.21.15.76:0/3557339507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc558004d00 0x7fc558005180 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc568009fc0 tx=0x7fc5580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13638088058914da server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:36.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.551+0000 7fc56dffb640 1 -- 172.21.15.76:0/3557339507 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56802cc00 con 0x7fc558004d00 2023-11-09T03:45:36.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.551+0000 7fc56dffb640 1 -- 172.21.15.76:0/3557339507 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc56800f040 con 0x7fc558004d00 2023-11-09T03:45:36.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.551+0000 7fc56dffb640 1 -- 172.21.15.76:0/3557339507 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc568034440 con 0x7fc558004d00 2023-11-09T03:45:36.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.551+0000 7fc575761640 1 -- 172.21.15.76:0/3557339507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc558004d00 msgr2=0x7fc558005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:36.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.551+0000 7fc575761640 1 --2- 172.21.15.76:0/3557339507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc558004d00 0x7fc558005180 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc568009fc0 tx=0x7fc5580057f0 comp rx=0 tx=0).stop 2023-11-09T03:45:36.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.551+0000 7fc56f7fe640 1 -- 172.21.15.76:0/3557339507 reap_dead start 2023-11-09T03:45:36.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.551+0000 7fc575761640 1 -- 172.21.15.76:0/3557339507 shutdown_connections 2023-11-09T03:45:36.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.551+0000 7fc575761640 1 -- 172.21.15.76:0/3557339507 >> 172.21.15.76:0/3557339507 conn(0x7fc5700ce950 msgr2=0x7fc5700ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:36.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.551+0000 7fc575761640 1 -- 172.21.15.76:0/3557339507 shutdown_connections 2023-11-09T03:45:36.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.551+0000 7fc575761640 1 -- 172.21.15.76:0/3557339507 wait complete. 2023-11-09T03:45:36.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.553+0000 7fc575761640 1 Processor -- start 2023-11-09T03:45:36.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.553+0000 7fc575761640 1 -- start start 2023-11-09T03:45:36.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.553+0000 7fc575761640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc558004d00 0x7fc5701f0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:36.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.553+0000 7fc575761640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5701f3920 0x7fc5700a74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:36.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.553+0000 7fc575761640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc57015c420 0x7fc5701ed2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:36.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.553+0000 7fc575761640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5701f12f0 con 0x7fc57015c420 2023-11-09T03:45:36.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.553+0000 7fc575761640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5700a8990 con 0x7fc558004d00 2023-11-09T03:45:36.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.553+0000 7fc575761640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5701f3e50 con 0x7fc5701f3920 2023-11-09T03:45:36.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.553+0000 7fc56f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc57015c420 0x7fc5701ed2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:36.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.553+0000 7fc56f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc57015c420 0x7fc5701ed2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34124/0 (socket says 172.21.15.76:34124) 2023-11-09T03:45:36.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.553+0000 7fc56f7fe640 1 -- 172.21.15.76:0/165441487 learned_addr learned my addr 172.21.15.76:0/165441487 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:36.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.554+0000 7fc56effd640 1 --2- 172.21.15.76:0/165441487 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc558004d00 0x7fc5701f0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:36.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.554+0000 7fc56e7fc640 1 --2- 172.21.15.76:0/165441487 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5701f3920 0x7fc5700a74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:36.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.554+0000 7fc56f7fe640 1 -- 172.21.15.76:0/165441487 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5701f3920 msgr2=0x7fc5700a74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:36.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.554+0000 7fc56f7fe640 1 --2- 172.21.15.76:0/165441487 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5701f3920 0x7fc5700a74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:36.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.554+0000 7fc56f7fe640 1 -- 172.21.15.76:0/165441487 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc558004d00 msgr2=0x7fc5701f0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:36.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.554+0000 7fc56f7fe640 1 --2- 172.21.15.76:0/165441487 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc558004d00 0x7fc5701f0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:36.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.554+0000 7fc56f7fe640 1 -- 172.21.15.76:0/165441487 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5701f4130 con 0x7fc57015c420 2023-11-09T03:45:36.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.554+0000 7fc56f7fe640 1 --2- 172.21.15.76:0/165441487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc57015c420 0x7fc5701ed2d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc568002230 tx=0x7fc568009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:36.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.554+0000 7fc5577fe640 1 -- 172.21.15.76:0/165441487 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc568043070 con 0x7fc57015c420 2023-11-09T03:45:36.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.554+0000 7fc5577fe640 1 -- 172.21.15.76:0/165441487 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5680349c0 con 0x7fc57015c420 2023-11-09T03:45:36.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.554+0000 7fc5577fe640 1 -- 172.21.15.76:0/165441487 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc568034d70 con 0x7fc57015c420 2023-11-09T03:45:36.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.554+0000 7fc575761640 1 -- 172.21.15.76:0/165441487 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5701f4410 con 0x7fc57015c420 2023-11-09T03:45:36.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.554+0000 7fc575761640 1 -- 172.21.15.76:0/165441487 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc570147a70 con 0x7fc57015c420 2023-11-09T03:45:36.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.555+0000 7fc575761640 1 -- 172.21.15.76:0/165441487 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc570074290 con 0x7fc57015c420 2023-11-09T03:45:36.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.555+0000 7fc5577fe640 1 -- 172.21.15.76:0/165441487 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc56803e440 con 0x7fc57015c420 2023-11-09T03:45:36.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.556+0000 7fc5577fe640 1 --2- 172.21.15.76:0/165441487 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc55c06aea0 0x7fc55c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:36.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.556+0000 7fc5577fe640 1 -- 172.21.15.76:0/165441487 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc5680ab510 con 0x7fc57015c420 2023-11-09T03:45:36.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.556+0000 7fc56effd640 1 --2- 172.21.15.76:0/165441487 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc55c06aea0 0x7fc55c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:36.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.556+0000 7fc56effd640 1 --2- 172.21.15.76:0/165441487 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc55c06aea0 0x7fc55c06d360 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc560009b80 tx=0x7fc560008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:36.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.560+0000 7fc5577fe640 1 -- 172.21.15.76:0/165441487 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc568076360 con 0x7fc57015c420 2023-11-09T03:45:36.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.762+0000 7fc575761640 1 -- 172.21.15.76:0/165441487 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5700ce3a0 con 0x7fc57015c420 2023-11-09T03:45:36.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.763+0000 7fc5577fe640 1 -- 172.21.15.76:0/165441487 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4915 (secure 0 0 0) 0x7fc568079980 con 0x7fc57015c420 2023-11-09T03:45:36.764 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:36.764 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2686298435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2686298435},{"type":"v1","addr":"172.21.15.76:6833","nonce":2686298435}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2132717123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2132717123},{"type":"v1","addr":"172.21.15.76:6835","nonce":2132717123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4281,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2369499936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2369499936},{"type":"v1","addr":"172.21.15.113:6837","nonce":2369499936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:45:30.784155+0000","modified":"2023-11-09T03:45:35.781908+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/3293223710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3293223710},{"type":"v1","addr":"172.21.15.113:6836","nonce":3293223710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-09T03:45:36.765 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 7 2023-11-09T03:45:36.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.765+0000 7fc575761640 1 -- 172.21.15.76:0/165441487 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc55c06aea0 msgr2=0x7fc55c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:36.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.765+0000 7fc575761640 1 --2- 172.21.15.76:0/165441487 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc55c06aea0 0x7fc55c06d360 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc560009b80 tx=0x7fc560008040 comp rx=0 tx=0).stop 2023-11-09T03:45:36.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.765+0000 7fc575761640 1 -- 172.21.15.76:0/165441487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc57015c420 msgr2=0x7fc5701ed2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:36.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.765+0000 7fc575761640 1 --2- 172.21.15.76:0/165441487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc57015c420 0x7fc5701ed2d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc568002230 tx=0x7fc568009750 comp rx=0 tx=0).stop 2023-11-09T03:45:36.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.765+0000 7fc575761640 1 -- 172.21.15.76:0/165441487 shutdown_connections 2023-11-09T03:45:36.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.765+0000 7fc575761640 1 --2- 172.21.15.76:0/165441487 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc55c06aea0 0x7fc55c06d360 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:36.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.765+0000 7fc575761640 1 --2- 172.21.15.76:0/165441487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc57015c420 0x7fc5701ed2d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:36.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.765+0000 7fc575761640 1 --2- 172.21.15.76:0/165441487 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5701f3920 0x7fc5700a74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:36.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.765+0000 7fc575761640 1 --2- 172.21.15.76:0/165441487 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc558004d00 0x7fc5701f0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:36.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.765+0000 7fc575761640 1 -- 172.21.15.76:0/165441487 >> 172.21.15.76:0/165441487 conn(0x7fc5700ce950 msgr2=0x7fc57006cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:36.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.765+0000 7fc575761640 1 -- 172.21.15.76:0/165441487 shutdown_connections 2023-11-09T03:45:36.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.765+0000 7fc575761640 1 -- 172.21.15.76:0/165441487 wait complete. 2023-11-09T03:45:36.776 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:45:36.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.875+0000 7fd31e87c640 1 Processor -- start 2023-11-09T03:45:36.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.875+0000 7fd31e87c640 1 -- start start 2023-11-09T03:45:36.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.876+0000 7fd31e87c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd31815ce70 0x7fd31806d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:36.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.876+0000 7fd31e87c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd31806a120 con 0x7fd31815d250 2023-11-09T03:45:36.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.876+0000 7fd31e87c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd31806def0 con 0x7fd3180cb5c0 2023-11-09T03:45:36.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.876+0000 7fd31e87c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3180cdc10 con 0x7fd31815ce70 2023-11-09T03:45:36.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.876+0000 7fd317fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd31815ce70 0x7fd31806d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:36.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.876+0000 7fd3177fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd31815d250 0x7fd3180c9830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:51006/0 (socket says 172.21.15.76:51006) 2023-11-09T03:45:36.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.876+0000 7fd3177fe640 1 -- 172.21.15.76:0/757462438 learned_addr learned my addr 172.21.15.76:0/757462438 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:36.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.876+0000 7fd317fff640 1 -- 172.21.15.76:0/757462438 >> v1:172.21.15.113:6789/0 conn(0x7fd3180cb5c0 legacy=0x7fd3180ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:36.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.876+0000 7fd317fff640 1 -- 172.21.15.76:0/757462438 >> v1:172.21.15.76:6789/0 conn(0x7fd31815d250 legacy=0x7fd3180c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:36.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.876+0000 7fd316ffd640 1 -- 172.21.15.76:0/757462438 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2906738275 0 0) 0x7fd31806a120 con 0x7fd31815d250 2023-11-09T03:45:36.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.876+0000 7fd317fff640 1 -- 172.21.15.76:0/757462438 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3180c9f40 con 0x7fd31815ce70 2023-11-09T03:45:36.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.877+0000 7fd317fff640 1 --2- 172.21.15.76:0/757462438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd31815ce70 0x7fd31806d9b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd3080097a0 tx=0x7fd30802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9b888a19d674bd server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:36.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.877+0000 7fd316ffd640 1 -- 172.21.15.76:0/757462438 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd308030610 con 0x7fd31815ce70 2023-11-09T03:45:36.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.877+0000 7fd316ffd640 1 -- 172.21.15.76:0/757462438 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd308030770 con 0x7fd31815ce70 2023-11-09T03:45:36.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.877+0000 7fd316ffd640 1 -- 172.21.15.76:0/757462438 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd308030b00 con 0x7fd31815ce70 2023-11-09T03:45:36.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.878+0000 7fd31e87c640 1 -- 172.21.15.76:0/757462438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd31815ce70 msgr2=0x7fd31806d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:36.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.878+0000 7fd31e87c640 1 --2- 172.21.15.76:0/757462438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd31815ce70 0x7fd31806d9b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd3080097a0 tx=0x7fd30802b960 comp rx=0 tx=0).stop 2023-11-09T03:45:36.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.878+0000 7fd31e87c640 1 -- 172.21.15.76:0/757462438 shutdown_connections 2023-11-09T03:45:36.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.878+0000 7fd31e87c640 1 --2- 172.21.15.76:0/757462438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd31815ce70 0x7fd31806d9b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:36.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.878+0000 7fd31e87c640 1 -- 172.21.15.76:0/757462438 >> 172.21.15.76:0/757462438 conn(0x7fd3180b0cd0 msgr2=0x7fd3180b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:36.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.878+0000 7fd31e87c640 1 -- 172.21.15.76:0/757462438 shutdown_connections 2023-11-09T03:45:36.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.878+0000 7fd31e87c640 1 -- 172.21.15.76:0/757462438 wait complete. 2023-11-09T03:45:36.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.879+0000 7fd31e87c640 1 Processor -- start 2023-11-09T03:45:36.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.879+0000 7fd31e87c640 1 -- start start 2023-11-09T03:45:36.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.880+0000 7fd31e87c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3180cb5c0 0x7fd3181e8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:36.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.880+0000 7fd31e87c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd31815ce70 0x7fd318144510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:36.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.880+0000 7fd31e87c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd31815d250 0x7fd3180c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:36.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.880+0000 7fd31e87c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd318145490 con 0x7fd31815ce70 2023-11-09T03:45:36.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.880+0000 7fd31e87c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd318142970 con 0x7fd3180cb5c0 2023-11-09T03:45:36.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.880+0000 7fd31e87c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3181e9140 con 0x7fd31815d250 2023-11-09T03:45:36.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.880+0000 7fd3177fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd31815ce70 0x7fd318144510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:36.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.880+0000 7fd3177fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd31815ce70 0x7fd318144510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34154/0 (socket says 172.21.15.76:34154) 2023-11-09T03:45:36.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.880+0000 7fd3177fe640 1 -- 172.21.15.76:0/1452358008 learned_addr learned my addr 172.21.15.76:0/1452358008 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:36.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.880+0000 7fd317fff640 1 --2- 172.21.15.76:0/1452358008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3180cb5c0 0x7fd3181e8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:36.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.880+0000 7fd31ce19640 1 --2- 172.21.15.76:0/1452358008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd31815d250 0x7fd3180c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:36.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.880+0000 7fd3177fe640 1 -- 172.21.15.76:0/1452358008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd31815d250 msgr2=0x7fd3180c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:36.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.880+0000 7fd3177fe640 1 --2- 172.21.15.76:0/1452358008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd31815d250 0x7fd3180c82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:36.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.880+0000 7fd3177fe640 1 -- 172.21.15.76:0/1452358008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3180cb5c0 msgr2=0x7fd3181e8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:36.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.880+0000 7fd3177fe640 1 --2- 172.21.15.76:0/1452358008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3180cb5c0 0x7fd3181e8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:36.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.880+0000 7fd3177fe640 1 -- 172.21.15.76:0/1452358008 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd308009400 con 0x7fd31815ce70 2023-11-09T03:45:36.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.880+0000 7fd3177fe640 1 --2- 172.21.15.76:0/1452358008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd31815ce70 0x7fd318144510 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd30c00c460 tx=0x7fd30c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:36.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.880+0000 7fd314ff9640 1 -- 172.21.15.76:0/1452358008 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30c0074e0 con 0x7fd31815ce70 2023-11-09T03:45:36.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.880+0000 7fd314ff9640 1 -- 172.21.15.76:0/1452358008 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd30c007680 con 0x7fd31815ce70 2023-11-09T03:45:36.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.880+0000 7fd31e87c640 1 -- 172.21.15.76:0/1452358008 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd318144ed0 con 0x7fd31815ce70 2023-11-09T03:45:36.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.880+0000 7fd314ff9640 1 -- 172.21.15.76:0/1452358008 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30c004a00 con 0x7fd31815ce70 2023-11-09T03:45:36.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.880+0000 7fd31e87c640 1 -- 172.21.15.76:0/1452358008 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd318142600 con 0x7fd31815ce70 2023-11-09T03:45:36.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.881+0000 7fd31e87c640 1 -- 172.21.15.76:0/1452358008 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd318074290 con 0x7fd31815ce70 2023-11-09T03:45:36.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.882+0000 7fd314ff9640 1 -- 172.21.15.76:0/1452358008 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd30c004b60 con 0x7fd31815ce70 2023-11-09T03:45:36.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.882+0000 7fd314ff9640 1 --2- 172.21.15.76:0/1452358008 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd30406ae70 0x7fd30406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:36.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.882+0000 7fd314ff9640 1 -- 172.21.15.76:0/1452358008 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd30c088db0 con 0x7fd31815ce70 2023-11-09T03:45:36.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.882+0000 7fd317fff640 1 --2- 172.21.15.76:0/1452358008 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd30406ae70 0x7fd30406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:36.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.883+0000 7fd317fff640 1 --2- 172.21.15.76:0/1452358008 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd30406ae70 0x7fd30406d330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd308036000 tx=0x7fd308008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:36.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:36.886+0000 7fd314ff9640 1 -- 172.21.15.76:0/1452358008 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd30c053cb0 con 0x7fd31815ce70 2023-11-09T03:45:37.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.055+0000 7fd31e87c640 1 -- 172.21.15.76:0/1452358008 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3180b0ab0 con 0x7fd31815ce70 2023-11-09T03:45:37.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.056+0000 7fd314ff9640 1 -- 172.21.15.76:0/1452358008 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18334 (secure 0 0 0) 0x7fd30c018020 con 0x7fd31815ce70 2023-11-09T03:45:37.057 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:37.057 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":25,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:45:35.781752+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:45:37.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.058+0000 7fd31e87c640 1 -- 172.21.15.76:0/1452358008 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd30406ae70 msgr2=0x7fd30406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:37.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.058+0000 7fd31e87c640 1 --2- 172.21.15.76:0/1452358008 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd30406ae70 0x7fd30406d330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd308036000 tx=0x7fd308008030 comp rx=0 tx=0).stop 2023-11-09T03:45:37.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.058+0000 7fd31e87c640 1 -- 172.21.15.76:0/1452358008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd31815ce70 msgr2=0x7fd318144510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:37.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.058+0000 7fd31e87c640 1 --2- 172.21.15.76:0/1452358008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd31815ce70 0x7fd318144510 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd30c00c460 tx=0x7fd30c00c920 comp rx=0 tx=0).stop 2023-11-09T03:45:37.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.058+0000 7fd31e87c640 1 -- 172.21.15.76:0/1452358008 shutdown_connections 2023-11-09T03:45:37.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.058+0000 7fd31e87c640 1 --2- 172.21.15.76:0/1452358008 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd30406ae70 0x7fd30406d330 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.058+0000 7fd31e87c640 1 --2- 172.21.15.76:0/1452358008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd31815d250 0x7fd3180c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.058+0000 7fd31e87c640 1 --2- 172.21.15.76:0/1452358008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd31815ce70 0x7fd318144510 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.058+0000 7fd31e87c640 1 --2- 172.21.15.76:0/1452358008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3180cb5c0 0x7fd3181e8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.058+0000 7fd31e87c640 1 -- 172.21.15.76:0/1452358008 >> 172.21.15.76:0/1452358008 conn(0x7fd3180b0cd0 msgr2=0x7fd3180abe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:37.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.058+0000 7fd31e87c640 1 -- 172.21.15.76:0/1452358008 shutdown_connections 2023-11-09T03:45:37.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.058+0000 7fd31e87c640 1 -- 172.21.15.76:0/1452358008 wait complete. 2023-11-09T03:45:37.069 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:45:37.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.168+0000 7ff07a489640 1 Processor -- start 2023-11-09T03:45:37.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.168+0000 7ff07a489640 1 -- start start 2023-11-09T03:45:37.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.169+0000 7ff07a489640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff074159c70 0x7ff0740ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:37.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.169+0000 7ff07a489640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff07406db20 0x7ff0740b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:37.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.169+0000 7ff07a489640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0740b0910 0x7ff07415d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:37.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.169+0000 7ff07a489640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0740b49b0 con 0x7ff074159c70 2023-11-09T03:45:37.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.169+0000 7ff07a489640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0740b0da0 con 0x7ff07406db20 2023-11-09T03:45:37.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.169+0000 7ff07a489640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0740b4110 con 0x7ff0740b0910 2023-11-09T03:45:37.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.169+0000 7ff073fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff074159c70 0x7ff0740ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:37.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.169+0000 7ff073fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff074159c70 0x7ff0740ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34170/0 (socket says 172.21.15.76:34170) 2023-11-09T03:45:37.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.169+0000 7ff073fff640 1 -- 172.21.15.76:0/2962065792 learned_addr learned my addr 172.21.15.76:0/2962065792 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:37.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.169+0000 7ff073fff640 1 -- 172.21.15.76:0/2962065792 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0740b0910 msgr2=0x7ff07415d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:37.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.169+0000 7ff073fff640 1 --2- 172.21.15.76:0/2962065792 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0740b0910 0x7ff07415d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.169+0000 7ff0737fe640 1 --2- 172.21.15.76:0/2962065792 >> v2:172.21.15.113:3300/0 conn(0x7ff07406db20 0x7ff0740b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:37.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.169+0000 7ff073fff640 1 -- 172.21.15.76:0/2962065792 >> v2:172.21.15.113:3300/0 conn(0x7ff07406db20 msgr2=0x7ff0740b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:37.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.170+0000 7ff073fff640 1 --2- 172.21.15.76:0/2962065792 >> v2:172.21.15.113:3300/0 conn(0x7ff07406db20 0x7ff0740b03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.170+0000 7ff073fff640 1 -- 172.21.15.76:0/2962065792 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0740b43f0 con 0x7ff074159c70 2023-11-09T03:45:37.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.170+0000 7ff073fff640 1 --2- 172.21.15.76:0/2962065792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff074159c70 0x7ff0740ce370 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff0740cf800 tx=0x7ff06402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf62be86da1ff39 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:37.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.170+0000 7ff072ffd640 1 -- 172.21.15.76:0/2962065792 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff064031870 con 0x7ff074159c70 2023-11-09T03:45:37.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.170+0000 7ff072ffd640 1 -- 172.21.15.76:0/2962065792 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0640319d0 con 0x7ff074159c70 2023-11-09T03:45:37.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.170+0000 7ff072ffd640 1 -- 172.21.15.76:0/2962065792 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff064031d60 con 0x7ff074159c70 2023-11-09T03:45:37.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.171+0000 7ff07a489640 1 -- 172.21.15.76:0/2962065792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff074159c70 msgr2=0x7ff0740ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:37.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.171+0000 7ff07a489640 1 --2- 172.21.15.76:0/2962065792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff074159c70 0x7ff0740ce370 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff0740cf800 tx=0x7ff06402b910 comp rx=0 tx=0).stop 2023-11-09T03:45:37.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.171+0000 7ff07a489640 1 -- 172.21.15.76:0/2962065792 shutdown_connections 2023-11-09T03:45:37.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.171+0000 7ff07a489640 1 --2- 172.21.15.76:0/2962065792 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0740b0910 0x7ff07415d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.171+0000 7ff07a489640 1 --2- 172.21.15.76:0/2962065792 >> v2:172.21.15.113:3300/0 conn(0x7ff07406db20 0x7ff0740b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.171+0000 7ff07a489640 1 --2- 172.21.15.76:0/2962065792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff074159c70 0x7ff0740ce370 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.171+0000 7ff07a489640 1 -- 172.21.15.76:0/2962065792 >> 172.21.15.76:0/2962065792 conn(0x7ff0740ce950 msgr2=0x7ff0740ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:37.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.171+0000 7ff07a489640 1 -- 172.21.15.76:0/2962065792 shutdown_connections 2023-11-09T03:45:37.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.171+0000 7ff07a489640 1 -- 172.21.15.76:0/2962065792 wait complete. 2023-11-09T03:45:37.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.172+0000 7ff07a489640 1 Processor -- start 2023-11-09T03:45:37.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.172+0000 7ff07a489640 1 -- start start 2023-11-09T03:45:37.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.173+0000 7ff07a489640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff07406db20 0x7ff0740b1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:37.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.173+0000 7ff07a489640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0740b0910 0x7ff0740a9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:37.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.173+0000 7ff07a489640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff074159c70 0x7ff0740706f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:37.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.173+0000 7ff07a489640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0740b2a10 con 0x7ff074159c70 2023-11-09T03:45:37.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.173+0000 7ff07a489640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0740b1e00 con 0x7ff0740b0910 2023-11-09T03:45:37.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.173+0000 7ff07a489640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0740b20e0 con 0x7ff07406db20 2023-11-09T03:45:37.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.173+0000 7ff078a26640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff074159c70 0x7ff0740706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:37.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.173+0000 7ff078a26640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff074159c70 0x7ff0740706f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34174/0 (socket says 172.21.15.76:34174) 2023-11-09T03:45:37.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.173+0000 7ff078a26640 1 -- 172.21.15.76:0/1322496947 learned_addr learned my addr 172.21.15.76:0/1322496947 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:37.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.173+0000 7ff0737fe640 1 --2- 172.21.15.76:0/1322496947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0740b0910 0x7ff0740a9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:37.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.173+0000 7ff073fff640 1 --2- 172.21.15.76:0/1322496947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff07406db20 0x7ff0740b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:37.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.173+0000 7ff078a26640 1 -- 172.21.15.76:0/1322496947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff07406db20 msgr2=0x7ff0740b1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:37.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.173+0000 7ff078a26640 1 --2- 172.21.15.76:0/1322496947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff07406db20 0x7ff0740b1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.173+0000 7ff078a26640 1 -- 172.21.15.76:0/1322496947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0740b0910 msgr2=0x7ff0740a9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:37.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.173+0000 7ff078a26640 1 --2- 172.21.15.76:0/1322496947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0740b0910 0x7ff0740a9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.173+0000 7ff078a26640 1 -- 172.21.15.76:0/1322496947 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0640094d0 con 0x7ff074159c70 2023-11-09T03:45:37.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.173+0000 7ff0737fe640 1 --2- 172.21.15.76:0/1322496947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0740b0910 0x7ff0740a9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:37.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.174+0000 7ff078a26640 1 --2- 172.21.15.76:0/1322496947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff074159c70 0x7ff0740706f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00b970 tx=0x7ff06c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:37.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.174+0000 7ff070ff9640 1 -- 172.21.15.76:0/1322496947 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c00cb40 con 0x7ff074159c70 2023-11-09T03:45:37.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.174+0000 7ff070ff9640 1 -- 172.21.15.76:0/1322496947 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06c00ccc0 con 0x7ff074159c70 2023-11-09T03:45:37.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.174+0000 7ff07a489640 1 -- 172.21.15.76:0/1322496947 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0740b23c0 con 0x7ff074159c70 2023-11-09T03:45:37.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.174+0000 7ff070ff9640 1 -- 172.21.15.76:0/1322496947 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c014680 con 0x7ff074159c70 2023-11-09T03:45:37.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.174+0000 7ff07a489640 1 -- 172.21.15.76:0/1322496947 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0740b1ab0 con 0x7ff074159c70 2023-11-09T03:45:37.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.175+0000 7ff07a489640 1 -- 172.21.15.76:0/1322496947 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff074074290 con 0x7ff074159c70 2023-11-09T03:45:37.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.175+0000 7ff070ff9640 1 -- 172.21.15.76:0/1322496947 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff06c01d3b0 con 0x7ff074159c70 2023-11-09T03:45:37.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.175+0000 7ff070ff9640 1 --2- 172.21.15.76:0/1322496947 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff06006ae70 0x7ff06006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:37.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.176+0000 7ff070ff9640 1 -- 172.21.15.76:0/1322496947 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff06c089880 con 0x7ff074159c70 2023-11-09T03:45:37.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.176+0000 7ff073fff640 1 --2- 172.21.15.76:0/1322496947 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff06006ae70 0x7ff06006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:37.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.176+0000 7ff073fff640 1 --2- 172.21.15.76:0/1322496947 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff06006ae70 0x7ff06006d330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff064036000 tx=0x7ff06403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:37.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.180+0000 7ff070ff9640 1 -- 172.21.15.76:0/1322496947 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff06c054640 con 0x7ff074159c70 2023-11-09T03:45:37.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.350+0000 7ff07a489640 1 -- 172.21.15.76:0/1322496947 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff07406d3f0 con 0x7ff074159c70 2023-11-09T03:45:37.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.351+0000 7ff070ff9640 1 -- 172.21.15.76:0/1322496947 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18334 (secure 0 0 0) 0x7ff06c019090 con 0x7ff074159c70 2023-11-09T03:45:37.352 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:37.352 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":25,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:45:35.781752+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:45:37.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.353+0000 7ff07a489640 1 -- 172.21.15.76:0/1322496947 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff06006ae70 msgr2=0x7ff06006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:37.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.353+0000 7ff07a489640 1 --2- 172.21.15.76:0/1322496947 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff06006ae70 0x7ff06006d330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff064036000 tx=0x7ff06403c040 comp rx=0 tx=0).stop 2023-11-09T03:45:37.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.353+0000 7ff07a489640 1 -- 172.21.15.76:0/1322496947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff074159c70 msgr2=0x7ff0740706f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:37.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.353+0000 7ff07a489640 1 --2- 172.21.15.76:0/1322496947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff074159c70 0x7ff0740706f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00b970 tx=0x7ff06c00be30 comp rx=0 tx=0).stop 2023-11-09T03:45:37.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.353+0000 7ff07a489640 1 -- 172.21.15.76:0/1322496947 shutdown_connections 2023-11-09T03:45:37.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.353+0000 7ff07a489640 1 --2- 172.21.15.76:0/1322496947 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff06006ae70 0x7ff06006d330 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.353+0000 7ff07a489640 1 --2- 172.21.15.76:0/1322496947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff074159c70 0x7ff0740706f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.353+0000 7ff07a489640 1 --2- 172.21.15.76:0/1322496947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0740b0910 0x7ff0740a9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.353+0000 7ff07a489640 1 --2- 172.21.15.76:0/1322496947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff07406db20 0x7ff0740b1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.353+0000 7ff07a489640 1 -- 172.21.15.76:0/1322496947 >> 172.21.15.76:0/1322496947 conn(0x7ff0740ce950 msgr2=0x7ff0740ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:37.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.353+0000 7ff07a489640 1 -- 172.21.15.76:0/1322496947 shutdown_connections 2023-11-09T03:45:37.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.353+0000 7ff07a489640 1 -- 172.21.15.76:0/1322496947 wait complete. 2023-11-09T03:45:37.409 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:45:37.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.509+0000 7f8b1fa4b640 1 Processor -- start 2023-11-09T03:45:37.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.510+0000 7f8b1fa4b640 1 -- start start 2023-11-09T03:45:37.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.510+0000 7f8b1fa4b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8b180cde60 0x7f8b1815d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:37.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.510+0000 7f8b1fa4b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8b1815daa0 0x7f8b18157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:37.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.510+0000 7f8b1fa4b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b1815e000 con 0x7f8b180ce240 2023-11-09T03:45:37.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.510+0000 7f8b1fa4b640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b1815e5c0 con 0x7f8b1815daa0 2023-11-09T03:45:37.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.510+0000 7f8b1fa4b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b1815e2e0 con 0x7f8b180cde60 2023-11-09T03:45:37.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.511+0000 7f8b1cfe6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8b1815daa0 0x7f8b18157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:37.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.511+0000 7f8b1cfe6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8b1815daa0 0x7f8b18157000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34178/0 (socket says 172.21.15.76:34178) 2023-11-09T03:45:37.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.511+0000 7f8b1cfe6640 1 -- 172.21.15.76:0/1563502246 learned_addr learned my addr 172.21.15.76:0/1563502246 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:37.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.511+0000 7f8b1d7e7640 1 --2- 172.21.15.76:0/1563502246 >> v2:172.21.15.113:3300/0 conn(0x7f8b180cde60 0x7f8b1815d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:37.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.511+0000 7f8b1cfe6640 1 -- 172.21.15.76:0/1563502246 >> v2:172.21.15.113:3300/0 conn(0x7f8b180cde60 msgr2=0x7f8b1815d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:37.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.511+0000 7f8b1cfe6640 1 --2- 172.21.15.76:0/1563502246 >> v2:172.21.15.113:3300/0 conn(0x7f8b180cde60 0x7f8b1815d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.511+0000 7f8b1cfe6640 1 -- 172.21.15.76:0/1563502246 >> v1:172.21.15.113:6789/0 conn(0x7f8b180ce240 legacy=0x7f8b1806b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:37.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.511+0000 7f8b1cfe6640 1 -- 172.21.15.76:0/1563502246 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b180d1120 con 0x7f8b1815daa0 2023-11-09T03:45:37.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.511+0000 7f8b1d7e7640 1 --2- 172.21.15.76:0/1563502246 >> v2:172.21.15.113:3300/0 conn(0x7f8b180cde60 0x7f8b1815d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:37.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.512+0000 7f8b1cfe6640 1 --2- 172.21.15.76:0/1563502246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b1815daa0 0x7f8b18157000 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8b180b5550 tx=0x7f8b0c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4e0572c55a8dad server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:37.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.512+0000 7f8b17fff640 1 -- 172.21.15.76:0/1563502246 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c031870 con 0x7f8b1815daa0 2023-11-09T03:45:37.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.512+0000 7f8b17fff640 1 -- 172.21.15.76:0/1563502246 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0c0319d0 con 0x7f8b1815daa0 2023-11-09T03:45:37.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.512+0000 7f8b17fff640 1 -- 172.21.15.76:0/1563502246 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c031d60 con 0x7f8b1815daa0 2023-11-09T03:45:37.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.512+0000 7f8b1fa4b640 1 -- 172.21.15.76:0/1563502246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b1815daa0 msgr2=0x7f8b18157000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:37.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.512+0000 7f8b1fa4b640 1 --2- 172.21.15.76:0/1563502246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b1815daa0 0x7f8b18157000 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8b180b5550 tx=0x7f8b0c02b910 comp rx=0 tx=0).stop 2023-11-09T03:45:37.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.512+0000 7f8b1fa4b640 1 -- 172.21.15.76:0/1563502246 shutdown_connections 2023-11-09T03:45:37.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.512+0000 7f8b1fa4b640 1 --2- 172.21.15.76:0/1563502246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b1815daa0 0x7f8b18157000 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.512+0000 7f8b1fa4b640 1 --2- 172.21.15.76:0/1563502246 >> v2:172.21.15.113:3300/0 conn(0x7f8b180cde60 0x7f8b1815d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.512+0000 7f8b1fa4b640 1 -- 172.21.15.76:0/1563502246 >> 172.21.15.76:0/1563502246 conn(0x7f8b18071f30 msgr2=0x7f8b180cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:37.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.513+0000 7f8b1fa4b640 1 -- 172.21.15.76:0/1563502246 shutdown_connections 2023-11-09T03:45:37.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.513+0000 7f8b1fa4b640 1 -- 172.21.15.76:0/1563502246 wait complete. 2023-11-09T03:45:37.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.514+0000 7f8b1fa4b640 1 Processor -- start 2023-11-09T03:45:37.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.514+0000 7f8b1fa4b640 1 -- start start 2023-11-09T03:45:37.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.514+0000 7f8b1fa4b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b180cde60 0x7f8b18150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:37.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.514+0000 7f8b1fa4b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b180ce240 0x7f8b1814e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:37.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.514+0000 7f8b1fa4b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b1815daa0 0x7f8b18155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:37.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.514+0000 7f8b1fa4b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1814e890 con 0x7f8b1815daa0 2023-11-09T03:45:37.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.514+0000 7f8b1fa4b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b181559b0 con 0x7f8b180ce240 2023-11-09T03:45:37.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.514+0000 7f8b1fa4b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b18150d00 con 0x7f8b180cde60 2023-11-09T03:45:37.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.514+0000 7f8b1dfe8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b1815daa0 0x7f8b18155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:37.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.515+0000 7f8b1dfe8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b1815daa0 0x7f8b18155470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34192/0 (socket says 172.21.15.76:34192) 2023-11-09T03:45:37.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.515+0000 7f8b1dfe8640 1 -- 172.21.15.76:0/1161874985 learned_addr learned my addr 172.21.15.76:0/1161874985 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:37.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.515+0000 7f8b1d7e7640 1 --2- 172.21.15.76:0/1161874985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b180cde60 0x7f8b18150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:37.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.515+0000 7f8b1cfe6640 1 --2- 172.21.15.76:0/1161874985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b180ce240 0x7f8b1814e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:37.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.515+0000 7f8b1dfe8640 1 -- 172.21.15.76:0/1161874985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b180cde60 msgr2=0x7f8b18150670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:37.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.515+0000 7f8b1dfe8640 1 --2- 172.21.15.76:0/1161874985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b180cde60 0x7f8b18150670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.515+0000 7f8b1dfe8640 1 -- 172.21.15.76:0/1161874985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b180ce240 msgr2=0x7f8b1814e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:37.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.515+0000 7f8b1dfe8640 1 --2- 172.21.15.76:0/1161874985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b180ce240 0x7f8b1814e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.515+0000 7f8b1dfe8640 1 -- 172.21.15.76:0/1161874985 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0c0094d0 con 0x7f8b1815daa0 2023-11-09T03:45:37.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.515+0000 7f8b1cfe6640 1 --2- 172.21.15.76:0/1161874985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b180ce240 0x7f8b1814e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:37.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.515+0000 7f8b1dfe8640 1 --2- 172.21.15.76:0/1161874985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b1815daa0 0x7f8b18155470 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8b0800c700 tx=0x7f8b0800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:37.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.515+0000 7f8b15ffb640 1 -- 172.21.15.76:0/1161874985 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0800aa50 con 0x7f8b1815daa0 2023-11-09T03:45:37.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.515+0000 7f8b15ffb640 1 -- 172.21.15.76:0/1161874985 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0800abb0 con 0x7f8b1815daa0 2023-11-09T03:45:37.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.515+0000 7f8b1fa4b640 1 -- 172.21.15.76:0/1161874985 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b18150fe0 con 0x7f8b1815daa0 2023-11-09T03:45:37.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.515+0000 7f8b15ffb640 1 -- 172.21.15.76:0/1161874985 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0800ce10 con 0x7f8b1815daa0 2023-11-09T03:45:37.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.515+0000 7f8b1fa4b640 1 -- 172.21.15.76:0/1161874985 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b181515a0 con 0x7f8b1815daa0 2023-11-09T03:45:37.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.516+0000 7f8b1fa4b640 1 -- 172.21.15.76:0/1161874985 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b18074290 con 0x7f8b1815daa0 2023-11-09T03:45:37.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.517+0000 7f8b15ffb640 1 -- 172.21.15.76:0/1161874985 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8b08012420 con 0x7f8b1815daa0 2023-11-09T03:45:37.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.517+0000 7f8b15ffb640 1 --2- 172.21.15.76:0/1161874985 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8b0406ae70 0x7f8b0406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:37.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.517+0000 7f8b15ffb640 1 -- 172.21.15.76:0/1161874985 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8b08089ab0 con 0x7f8b1815daa0 2023-11-09T03:45:37.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.517+0000 7f8b1d7e7640 1 --2- 172.21.15.76:0/1161874985 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8b0406ae70 0x7f8b0406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:37.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.518+0000 7f8b1d7e7640 1 --2- 172.21.15.76:0/1161874985 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8b0406ae70 0x7f8b0406d330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000a990 tx=0x7f8b10008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:37.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.521+0000 7f8b15ffb640 1 -- 172.21.15.76:0/1161874985 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8b08054900 con 0x7f8b1815daa0 2023-11-09T03:45:37.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.723+0000 7f8b1fa4b640 1 -- 172.21.15.76:0/1161874985 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b180d0600 con 0x7f8b1815daa0 2023-11-09T03:45:37.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.723+0000 7f8b15ffb640 1 -- 172.21.15.76:0/1161874985 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4915 (secure 0 0 0) 0x7f8b08016020 con 0x7f8b1815daa0 2023-11-09T03:45:37.724 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:37.724 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2686298435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2686298435},{"type":"v1","addr":"172.21.15.76:6833","nonce":2686298435}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2132717123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2132717123},{"type":"v1","addr":"172.21.15.76:6835","nonce":2132717123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4281,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2369499936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2369499936},{"type":"v1","addr":"172.21.15.113:6837","nonce":2369499936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:45:30.784155+0000","modified":"2023-11-09T03:45:35.781908+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/3293223710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3293223710},{"type":"v1","addr":"172.21.15.113:6836","nonce":3293223710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-09T03:45:37.725 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 7 2023-11-09T03:45:37.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.725+0000 7f8b1fa4b640 1 -- 172.21.15.76:0/1161874985 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8b0406ae70 msgr2=0x7f8b0406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:37.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.725+0000 7f8b1fa4b640 1 --2- 172.21.15.76:0/1161874985 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8b0406ae70 0x7f8b0406d330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000a990 tx=0x7f8b10008040 comp rx=0 tx=0).stop 2023-11-09T03:45:37.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.725+0000 7f8b1fa4b640 1 -- 172.21.15.76:0/1161874985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b1815daa0 msgr2=0x7f8b18155470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:37.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.725+0000 7f8b1fa4b640 1 --2- 172.21.15.76:0/1161874985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b1815daa0 0x7f8b18155470 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8b0800c700 tx=0x7f8b0800cbc0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.725+0000 7f8b1fa4b640 1 -- 172.21.15.76:0/1161874985 shutdown_connections 2023-11-09T03:45:37.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.725+0000 7f8b1fa4b640 1 --2- 172.21.15.76:0/1161874985 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8b0406ae70 0x7f8b0406d330 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.725+0000 7f8b1fa4b640 1 --2- 172.21.15.76:0/1161874985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b1815daa0 0x7f8b18155470 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.725+0000 7f8b1fa4b640 1 --2- 172.21.15.76:0/1161874985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b180ce240 0x7f8b1814e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.725+0000 7f8b1fa4b640 1 --2- 172.21.15.76:0/1161874985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b180cde60 0x7f8b18150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.725+0000 7f8b1fa4b640 1 -- 172.21.15.76:0/1161874985 >> 172.21.15.76:0/1161874985 conn(0x7f8b18071f30 msgr2=0x7f8b180b8060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:37.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.725+0000 7f8b1fa4b640 1 -- 172.21.15.76:0/1161874985 shutdown_connections 2023-11-09T03:45:37.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.725+0000 7f8b1fa4b640 1 -- 172.21.15.76:0/1161874985 wait complete. 2023-11-09T03:45:37.736 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:45:37.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.836+0000 7f8c77f25640 1 Processor -- start 2023-11-09T03:45:37.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.836+0000 7f8c77f25640 1 -- start start 2023-11-09T03:45:37.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.837+0000 7f8c77f25640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8c7015ce70 0x7f8c7006d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:37.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.837+0000 7f8c77f25640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c700cb5c0 0x7f8c700ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:37.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.837+0000 7f8c77f25640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c7006a120 con 0x7f8c7015d250 2023-11-09T03:45:37.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.837+0000 7f8c77f25640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c700cdc10 con 0x7f8c7015ce70 2023-11-09T03:45:37.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.837+0000 7f8c77f25640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7006def0 con 0x7f8c700cb5c0 2023-11-09T03:45:37.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.837+0000 7f8c764c2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c700cb5c0 0x7f8c700ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:37.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.837+0000 7f8c75cc1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8c7015ce70 0x7f8c7006d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:37.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.837+0000 7f8c75cc1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8c7015ce70 0x7f8c7006d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:60564/0 (socket says 172.21.15.76:60564) 2023-11-09T03:45:37.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.837+0000 7f8c75cc1640 1 -- 172.21.15.76:0/2386831916 learned_addr learned my addr 172.21.15.76:0/2386831916 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:37.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.837+0000 7f8c764c2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c700cb5c0 0x7f8c700ccdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:42732/0 (socket says 172.21.15.76:42732) 2023-11-09T03:45:37.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.837+0000 7f8c754c0640 1 --1- 172.21.15.76:0/2386831916 >> v1:172.21.15.113:6789/0 conn(0x7f8c7015d250 0x7f8c700c9830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:47478/0 (socket says 172.21.15.76:47478) 2023-11-09T03:45:37.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.838+0000 7f8c75cc1640 1 -- 172.21.15.76:0/2386831916 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c700cb5c0 msgr2=0x7f8c700ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:37.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.838+0000 7f8c75cc1640 1 --2- 172.21.15.76:0/2386831916 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c700cb5c0 0x7f8c700ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.838+0000 7f8c75cc1640 1 -- 172.21.15.76:0/2386831916 >> v1:172.21.15.113:6789/0 conn(0x7f8c7015d250 legacy=0x7f8c700c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:37.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.838+0000 7f8c75cc1640 1 -- 172.21.15.76:0/2386831916 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c700c9f40 con 0x7f8c7015ce70 2023-11-09T03:45:37.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.838+0000 7f8c764c2640 1 --2- 172.21.15.76:0/2386831916 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c700cb5c0 0x7f8c700ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:45:37.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.838+0000 7f8c75cc1640 1 --2- 172.21.15.76:0/2386831916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c7015ce70 0x7f8c7006d9b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8c700cdd50 tx=0x7f8c6c02bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=631cc67ddfe546af server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:37.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.838+0000 7f8c74cbf640 1 -- 172.21.15.76:0/2386831916 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c02caf0 con 0x7f8c7015ce70 2023-11-09T03:45:37.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.839+0000 7f8c74cbf640 1 -- 172.21.15.76:0/2386831916 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6c02cc50 con 0x7f8c7015ce70 2023-11-09T03:45:37.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.839+0000 7f8c74cbf640 1 -- 172.21.15.76:0/2386831916 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c0343f0 con 0x7f8c7015ce70 2023-11-09T03:45:37.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.839+0000 7f8c77f25640 1 -- 172.21.15.76:0/2386831916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c7015ce70 msgr2=0x7f8c7006d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:37.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.839+0000 7f8c77f25640 1 --2- 172.21.15.76:0/2386831916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c7015ce70 0x7f8c7006d9b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8c700cdd50 tx=0x7f8c6c02bf50 comp rx=0 tx=0).stop 2023-11-09T03:45:37.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.839+0000 7f8c77f25640 1 -- 172.21.15.76:0/2386831916 shutdown_connections 2023-11-09T03:45:37.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.839+0000 7f8c77f25640 1 --2- 172.21.15.76:0/2386831916 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c700cb5c0 0x7f8c700ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.839+0000 7f8c77f25640 1 --2- 172.21.15.76:0/2386831916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c7015ce70 0x7f8c7006d9b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.839+0000 7f8c77f25640 1 -- 172.21.15.76:0/2386831916 >> 172.21.15.76:0/2386831916 conn(0x7f8c700b0cd0 msgr2=0x7f8c700b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:37.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.839+0000 7f8c77f25640 1 -- 172.21.15.76:0/2386831916 shutdown_connections 2023-11-09T03:45:37.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.839+0000 7f8c77f25640 1 -- 172.21.15.76:0/2386831916 wait complete. 2023-11-09T03:45:37.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.840+0000 7f8c77f25640 1 Processor -- start 2023-11-09T03:45:37.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.840+0000 7f8c77f25640 1 -- start start 2023-11-09T03:45:37.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.841+0000 7f8c77f25640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c700cb5c0 0x7f8c701e2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:37.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.841+0000 7f8c77f25640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c7015ce70 0x7f8c700c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:37.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.841+0000 7f8c77f25640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c7015d250 0x7f8c70159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:37.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.841+0000 7f8c77f25640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c701e28a0 con 0x7f8c700cb5c0 2023-11-09T03:45:37.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.841+0000 7f8c77f25640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c70158c90 con 0x7f8c7015ce70 2023-11-09T03:45:37.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.841+0000 7f8c77f25640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c70155f40 con 0x7f8c7015d250 2023-11-09T03:45:37.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.841+0000 7f8c75cc1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c700cb5c0 0x7f8c701e2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:37.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.841+0000 7f8c75cc1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c700cb5c0 0x7f8c701e2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34202/0 (socket says 172.21.15.76:34202) 2023-11-09T03:45:37.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.841+0000 7f8c75cc1640 1 -- 172.21.15.76:0/1294739237 learned_addr learned my addr 172.21.15.76:0/1294739237 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:37.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.841+0000 7f8c754c0640 1 --2- 172.21.15.76:0/1294739237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c7015ce70 0x7f8c700c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:37.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.841+0000 7f8c764c2640 1 --2- 172.21.15.76:0/1294739237 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c7015d250 0x7f8c70159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:37.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.841+0000 7f8c75cc1640 1 -- 172.21.15.76:0/1294739237 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c7015d250 msgr2=0x7f8c70159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:37.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.841+0000 7f8c75cc1640 1 --2- 172.21.15.76:0/1294739237 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c7015d250 0x7f8c70159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.841+0000 7f8c75cc1640 1 -- 172.21.15.76:0/1294739237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c7015ce70 msgr2=0x7f8c700c7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:37.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.841+0000 7f8c75cc1640 1 --2- 172.21.15.76:0/1294739237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c7015ce70 0x7f8c700c7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:37.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.841+0000 7f8c75cc1640 1 -- 172.21.15.76:0/1294739237 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c68009400 con 0x7f8c700cb5c0 2023-11-09T03:45:37.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.842+0000 7f8c75cc1640 1 --2- 172.21.15.76:0/1294739237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c700cb5c0 0x7f8c701e2360 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c000f80 tx=0x7f8c6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:37.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.842+0000 7f8c667fc640 1 -- 172.21.15.76:0/1294739237 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c009510 con 0x7f8c700cb5c0 2023-11-09T03:45:37.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.842+0000 7f8c667fc640 1 -- 172.21.15.76:0/1294739237 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6c03e450 con 0x7f8c700cb5c0 2023-11-09T03:45:37.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.842+0000 7f8c77f25640 1 -- 172.21.15.76:0/1294739237 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c6c009b30 con 0x7f8c700cb5c0 2023-11-09T03:45:37.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.842+0000 7f8c667fc640 1 -- 172.21.15.76:0/1294739237 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c03e800 con 0x7f8c700cb5c0 2023-11-09T03:45:37.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.842+0000 7f8c77f25640 1 -- 172.21.15.76:0/1294739237 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c70158360 con 0x7f8c700cb5c0 2023-11-09T03:45:37.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.843+0000 7f8c77f25640 1 -- 172.21.15.76:0/1294739237 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c70074290 con 0x7f8c700cb5c0 2023-11-09T03:45:37.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.843+0000 7f8c667fc640 1 -- 172.21.15.76:0/1294739237 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8c6c03e960 con 0x7f8c700cb5c0 2023-11-09T03:45:37.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.843+0000 7f8c667fc640 1 --2- 172.21.15.76:0/1294739237 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8c5406ae70 0x7f8c5406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:37.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.844+0000 7f8c667fc640 1 -- 172.21.15.76:0/1294739237 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8c6c0ac650 con 0x7f8c700cb5c0 2023-11-09T03:45:37.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.844+0000 7f8c754c0640 1 --2- 172.21.15.76:0/1294739237 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8c5406ae70 0x7f8c5406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:37.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.844+0000 7f8c754c0640 1 --2- 172.21.15.76:0/1294739237 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8c5406ae70 0x7f8c5406d330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8c701e2a60 tx=0x7f8c60007c80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:37.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:37.848+0000 7f8c667fc640 1 -- 172.21.15.76:0/1294739237 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8c6c077420 con 0x7f8c700cb5c0 2023-11-09T03:45:38.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.013+0000 7f8c77f25640 1 -- 172.21.15.76:0/1294739237 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c70066170 con 0x7f8c700cb5c0 2023-11-09T03:45:38.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.014+0000 7f8c667fc640 1 -- 172.21.15.76:0/1294739237 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18334 (secure 0 0 0) 0x7f8c6c07aa40 con 0x7f8c700cb5c0 2023-11-09T03:45:38.015 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:38.015 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":25,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:45:35.781752+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:45:38.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.016+0000 7f8c77f25640 1 -- 172.21.15.76:0/1294739237 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8c5406ae70 msgr2=0x7f8c5406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:38.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.016+0000 7f8c77f25640 1 --2- 172.21.15.76:0/1294739237 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8c5406ae70 0x7f8c5406d330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8c701e2a60 tx=0x7f8c60007c80 comp rx=0 tx=0).stop 2023-11-09T03:45:38.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.016+0000 7f8c77f25640 1 -- 172.21.15.76:0/1294739237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c700cb5c0 msgr2=0x7f8c701e2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:38.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.016+0000 7f8c77f25640 1 --2- 172.21.15.76:0/1294739237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c700cb5c0 0x7f8c701e2360 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c000f80 tx=0x7f8c6c004a40 comp rx=0 tx=0).stop 2023-11-09T03:45:38.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.016+0000 7f8c77f25640 1 -- 172.21.15.76:0/1294739237 shutdown_connections 2023-11-09T03:45:38.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.016+0000 7f8c77f25640 1 --2- 172.21.15.76:0/1294739237 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8c5406ae70 0x7f8c5406d330 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.016+0000 7f8c77f25640 1 --2- 172.21.15.76:0/1294739237 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c7015d250 0x7f8c70159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.016+0000 7f8c77f25640 1 --2- 172.21.15.76:0/1294739237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c7015ce70 0x7f8c700c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.016+0000 7f8c77f25640 1 --2- 172.21.15.76:0/1294739237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c700cb5c0 0x7f8c701e2360 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.016+0000 7f8c77f25640 1 -- 172.21.15.76:0/1294739237 >> 172.21.15.76:0/1294739237 conn(0x7f8c700b0cd0 msgr2=0x7f8c700ada90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:38.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.016+0000 7f8c77f25640 1 -- 172.21.15.76:0/1294739237 shutdown_connections 2023-11-09T03:45:38.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.016+0000 7f8c77f25640 1 -- 172.21.15.76:0/1294739237 wait complete. 2023-11-09T03:45:38.430 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-09T03:45:30.784155+0000', 'modified': '2023-11-09T03:45:35.781908+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4291}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4291': {'gid': 4291, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.113:6836/3293223710', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6834', 'nonce': 3293223710}, {'type': 'v1', 'addr': '172.21.15.113:6836', 'nonce': 3293223710}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-11-09T03:45:38.430 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-09T03:45:38.431 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-09T03:45:38.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.533+0000 7f61d5a0e640 1 Processor -- start 2023-11-09T03:45:38.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.533+0000 7f61d5a0e640 1 -- start start 2023-11-09T03:45:38.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.534+0000 7f61d5a0e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f61d00cd6d0 0x7f61d0161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:38.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.534+0000 7f61d5a0e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d00cde60 con 0x7f61d0153e60 2023-11-09T03:45:38.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.534+0000 7f61d5a0e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d0160f90 con 0x7f61d00b4880 2023-11-09T03:45:38.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.534+0000 7f61d5a0e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d0161960 con 0x7f61d00cd6d0 2023-11-09T03:45:38.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.534+0000 7f61ceffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f61d0153e60 0x7f61d015dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:51012/0 (socket says 172.21.15.76:51012) 2023-11-09T03:45:38.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.534+0000 7f61ce7fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f61d00cd6d0 0x7f61d0161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:38.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.534+0000 7f61ceffd640 1 -- 172.21.15.76:0/494679011 learned_addr learned my addr 172.21.15.76:0/494679011 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:38.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.534+0000 7f61ce7fc640 1 -- 172.21.15.76:0/494679011 >> v1:172.21.15.113:6789/0 conn(0x7f61d00b4880 legacy=0x7f61d00d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:38.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.534+0000 7f61ce7fc640 1 -- 172.21.15.76:0/494679011 >> v1:172.21.15.76:6789/0 conn(0x7f61d0153e60 legacy=0x7f61d015dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:38.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.534+0000 7f61ce7fc640 1 -- 172.21.15.76:0/494679011 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d00ce090 con 0x7f61d00cd6d0 2023-11-09T03:45:38.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.534+0000 7f61cdffb640 1 -- 172.21.15.76:0/494679011 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 848463329 0 0) 0x7f61d00cde60 con 0x7f61d0153e60 2023-11-09T03:45:38.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.535+0000 7f61ce7fc640 1 --2- 172.21.15.76:0/494679011 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61d00cd6d0 0x7f61d0161420 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f61c40097a0 tx=0x7f61c402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b402513ab83f71e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:38.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.535+0000 7f61cdffb640 1 -- 172.21.15.76:0/494679011 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c4031830 con 0x7f61d00cd6d0 2023-11-09T03:45:38.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.535+0000 7f61cdffb640 1 -- 172.21.15.76:0/494679011 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c4031990 con 0x7f61d00cd6d0 2023-11-09T03:45:38.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.535+0000 7f61cdffb640 1 -- 172.21.15.76:0/494679011 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c4031d20 con 0x7f61d00cd6d0 2023-11-09T03:45:38.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.536+0000 7f61d5a0e640 1 -- 172.21.15.76:0/494679011 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61d00cd6d0 msgr2=0x7f61d0161420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:38.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.536+0000 7f61d5a0e640 1 --2- 172.21.15.76:0/494679011 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61d00cd6d0 0x7f61d0161420 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f61c40097a0 tx=0x7f61c402b8d0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.536+0000 7f61d5a0e640 1 -- 172.21.15.76:0/494679011 shutdown_connections 2023-11-09T03:45:38.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.536+0000 7f61d5a0e640 1 --2- 172.21.15.76:0/494679011 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61d00cd6d0 0x7f61d0161420 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.536+0000 7f61d5a0e640 1 -- 172.21.15.76:0/494679011 >> 172.21.15.76:0/494679011 conn(0x7f61d00cf880 msgr2=0x7f61d00cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:38.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.536+0000 7f61d5a0e640 1 -- 172.21.15.76:0/494679011 shutdown_connections 2023-11-09T03:45:38.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.536+0000 7f61d5a0e640 1 -- 172.21.15.76:0/494679011 wait complete. 2023-11-09T03:45:38.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.537+0000 7f61d5a0e640 1 Processor -- start 2023-11-09T03:45:38.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.537+0000 7f61d5a0e640 1 -- start start 2023-11-09T03:45:38.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.538+0000 7f61d5a0e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61d00b4880 0x7f61d01547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:38.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.538+0000 7f61d5a0e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61d00cd6d0 0x7f61d0152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:38.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.538+0000 7f61d5a0e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61d0153e60 0x7f61d0152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:38.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.538+0000 7f61d5a0e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d00c8cf0 con 0x7f61d00cd6d0 2023-11-09T03:45:38.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.538+0000 7f61d5a0e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d00c6420 con 0x7f61d0153e60 2023-11-09T03:45:38.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.538+0000 7f61d5a0e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d0153520 con 0x7f61d00b4880 2023-11-09T03:45:38.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.538+0000 7f61ce7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61d00cd6d0 0x7f61d0152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:38.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.538+0000 7f61ce7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61d00cd6d0 0x7f61d0152aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34224/0 (socket says 172.21.15.76:34224) 2023-11-09T03:45:38.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.538+0000 7f61ce7fc640 1 -- 172.21.15.76:0/4200786079 learned_addr learned my addr 172.21.15.76:0/4200786079 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:38.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.538+0000 7f61cf7fe640 1 --2- 172.21.15.76:0/4200786079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61d0153e60 0x7f61d0152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:38.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.538+0000 7f61ce7fc640 1 -- 172.21.15.76:0/4200786079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61d00b4880 msgr2=0x7f61d01547e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:38.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.538+0000 7f61ce7fc640 1 --2- 172.21.15.76:0/4200786079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61d00b4880 0x7f61d01547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.538+0000 7f61ce7fc640 1 -- 172.21.15.76:0/4200786079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61d0153e60 msgr2=0x7f61d0152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:38.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.538+0000 7f61ce7fc640 1 --2- 172.21.15.76:0/4200786079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61d0153e60 0x7f61d0152fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.538+0000 7f61ce7fc640 1 -- 172.21.15.76:0/4200786079 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c4009400 con 0x7f61d00cd6d0 2023-11-09T03:45:38.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.538+0000 7f61cf7fe640 1 --2- 172.21.15.76:0/4200786079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61d0153e60 0x7f61d0152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:38.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.538+0000 7f61ce7fc640 1 --2- 172.21.15.76:0/4200786079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61d00cd6d0 0x7f61d0152aa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f61c40098c0 tx=0x7f61c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:38.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.538+0000 7f61b7fff640 1 -- 172.21.15.76:0/4200786079 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c403ba70 con 0x7f61d00cd6d0 2023-11-09T03:45:38.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.538+0000 7f61b7fff640 1 -- 172.21.15.76:0/4200786079 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c403bbd0 con 0x7f61d00cd6d0 2023-11-09T03:45:38.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.538+0000 7f61b7fff640 1 -- 172.21.15.76:0/4200786079 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c40453f0 con 0x7f61d00cd6d0 2023-11-09T03:45:38.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.539+0000 7f61d5a0e640 1 -- 172.21.15.76:0/4200786079 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d00c8070 con 0x7f61d00cd6d0 2023-11-09T03:45:38.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.539+0000 7f61d5a0e640 1 -- 172.21.15.76:0/4200786079 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d00c8630 con 0x7f61d00cd6d0 2023-11-09T03:45:38.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.540+0000 7f61b7fff640 1 -- 172.21.15.76:0/4200786079 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f61c404e450 con 0x7f61d00cd6d0 2023-11-09T03:45:38.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.540+0000 7f61b7fff640 1 --2- 172.21.15.76:0/4200786079 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f61bc06ae70 0x7f61bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:38.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.540+0000 7f61b7fff640 1 -- 172.21.15.76:0/4200786079 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f61c40abb30 con 0x7f61d00cd6d0 2023-11-09T03:45:38.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.540+0000 7f61ceffd640 1 --2- 172.21.15.76:0/4200786079 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f61bc06ae70 0x7f61bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:38.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.541+0000 7f61ceffd640 1 --2- 172.21.15.76:0/4200786079 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f61bc06ae70 0x7f61bc06d330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f61c0007fd0 tx=0x7f61c0007ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:38.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.541+0000 7f61d859e740 1 Processor -- start 2023-11-09T03:45:38.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.541+0000 7f61d859e740 1 -- start start 2023-11-09T03:45:38.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.542+0000 7f61d859e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557214aec850 0x557214abf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:38.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.542+0000 7f61d859e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557214aecc30 0x557214aef5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:38.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.542+0000 7f61d859e740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557214abfc90 0x557214aefb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:38.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.542+0000 7f61d859e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572149d0740 con 0x557214aecc30 2023-11-09T03:45:38.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.542+0000 7f61d859e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x557214a328d0 con 0x557214aec850 2023-11-09T03:45:38.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.542+0000 7f61d859e740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x557214a2ffc0 con 0x557214abfc90 2023-11-09T03:45:38.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.542+0000 7f61ce7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557214aecc30 0x557214aef5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:38.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.542+0000 7f61ce7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557214aecc30 0x557214aef5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34240/0 (socket says 172.21.15.76:34240) 2023-11-09T03:45:38.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.542+0000 7f61ce7fc640 1 -- 172.21.15.76:0/2916779671 learned_addr learned my addr 172.21.15.76:0/2916779671 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:38.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.542+0000 7f61ceffd640 1 --2- 172.21.15.76:0/2916779671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557214aec850 0x557214abf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:38.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.542+0000 7f61cf7fe640 1 --2- 172.21.15.76:0/2916779671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557214abfc90 0x557214aefb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:38.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.542+0000 7f61ce7fc640 1 -- 172.21.15.76:0/2916779671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557214abfc90 msgr2=0x557214aefb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:38.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.542+0000 7f61ce7fc640 1 --2- 172.21.15.76:0/2916779671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557214abfc90 0x557214aefb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.542+0000 7f61ce7fc640 1 -- 172.21.15.76:0/2916779671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557214aec850 msgr2=0x557214abf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:38.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.542+0000 7f61ce7fc640 1 --2- 172.21.15.76:0/2916779671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557214aec850 0x557214abf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.542+0000 7f61ce7fc640 1 -- 172.21.15.76:0/2916779671 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572149eeca0 con 0x557214aecc30 2023-11-09T03:45:38.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.542+0000 7f61cf7fe640 1 --2- 172.21.15.76:0/2916779671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557214abfc90 0x557214aefb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:38.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.542+0000 7f61ce7fc640 1 --2- 172.21.15.76:0/2916779671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557214aecc30 0x557214aef5e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f61c407a5d0 tx=0x7f61c407ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1104f7a47660fd9 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:38.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.543+0000 7f61b57fa640 1 -- 172.21.15.76:0/2916779671 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c4082940 con 0x557214aecc30 2023-11-09T03:45:38.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.543+0000 7f61b57fa640 1 -- 172.21.15.76:0/2916779671 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c40789e0 con 0x557214aecc30 2023-11-09T03:45:38.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.543+0000 7f61b57fa640 1 -- 172.21.15.76:0/2916779671 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c407a8a0 con 0x557214aecc30 2023-11-09T03:45:38.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.543+0000 7f61d859e740 1 -- 172.21.15.76:0/2916779671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557214aecc30 msgr2=0x557214aef5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:38.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.543+0000 7f61d859e740 1 --2- 172.21.15.76:0/2916779671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557214aecc30 0x557214aef5e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f61c407a5d0 tx=0x7f61c407ad40 comp rx=0 tx=0).stop 2023-11-09T03:45:38.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.543+0000 7f61d859e740 1 -- 172.21.15.76:0/2916779671 shutdown_connections 2023-11-09T03:45:38.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.543+0000 7f61d859e740 1 --2- 172.21.15.76:0/2916779671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557214abfc90 0x557214aefb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.543+0000 7f61d859e740 1 --2- 172.21.15.76:0/2916779671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557214aecc30 0x557214aef5e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.543+0000 7f61d859e740 1 --2- 172.21.15.76:0/2916779671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557214aec850 0x557214abf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.543+0000 7f61d859e740 1 -- 172.21.15.76:0/2916779671 >> 172.21.15.76:0/2916779671 conn(0x557214ac1a60 msgr2=0x557214b0a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:38.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.543+0000 7f61d859e740 1 -- 172.21.15.76:0/2916779671 shutdown_connections 2023-11-09T03:45:38.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.543+0000 7f61d859e740 1 -- 172.21.15.76:0/2916779671 wait complete. 2023-11-09T03:45:38.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.544+0000 7f61d859e740 1 -- start start 2023-11-09T03:45:38.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.544+0000 7f61d859e740 1 Processor -- start 2023-11-09T03:45:38.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.545+0000 7f61d859e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557214aec850 0x557214b04e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:38.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.545+0000 7f61d859e740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557214aecc30 0x557214b0a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:38.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.545+0000 7f61d859e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557214b05340 0x557214b0a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:38.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.545+0000 7f61d859e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557214a2b8b0 con 0x557214aec850 2023-11-09T03:45:38.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.545+0000 7f61d859e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572149e5c80 con 0x557214b05340 2023-11-09T03:45:38.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.545+0000 7f61d859e740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5572149e5920 con 0x557214aecc30 2023-11-09T03:45:38.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.545+0000 7f61d859e740 10 client.0 _ll_register_callbacks cb 0x55721473dac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:45:38.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.545+0000 7f61ceffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557214aec850 0x557214b04e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:38.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.545+0000 7f61ceffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557214aec850 0x557214b04e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34256/0 (socket says 172.21.15.76:34256) 2023-11-09T03:45:38.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.545+0000 7f61ceffd640 1 -- 172.21.15.76:0/1645443789 learned_addr learned my addr 172.21.15.76:0/1645443789 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:38.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.545+0000 7f61cf7fe640 1 --2- 172.21.15.76:0/1645443789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557214b05340 0x557214b0a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:38.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.545+0000 7f61ce7fc640 1 --2- 172.21.15.76:0/1645443789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557214aecc30 0x557214b0a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:38.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.545+0000 7f61ceffd640 1 -- 172.21.15.76:0/1645443789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557214aecc30 msgr2=0x557214b0a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:38.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.545+0000 7f61ceffd640 1 --2- 172.21.15.76:0/1645443789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557214aecc30 0x557214b0a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.545+0000 7f61ceffd640 1 -- 172.21.15.76:0/1645443789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557214b05340 msgr2=0x557214b0a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:38.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.545+0000 7f61ceffd640 1 --2- 172.21.15.76:0/1645443789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557214b05340 0x557214b0a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.545+0000 7f61ceffd640 1 -- 172.21.15.76:0/1645443789 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d00c8630 con 0x557214aec850 2023-11-09T03:45:38.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.545+0000 7f61ce7fc640 1 --2- 172.21.15.76:0/1645443789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557214aecc30 0x557214b0a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:38.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.545+0000 7f61ceffd640 1 --2- 172.21.15.76:0/1645443789 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557214aec850 0x557214b04e00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f61c0007380 tx=0x7f61c0001400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc579cb34685f4c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:38.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.545+0000 7f61ae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:45:38.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.545+0000 7f61ae7fc640 1 -- 172.21.15.76:0/1645443789 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c001b050 con 0x557214aec850 2023-11-09T03:45:38.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.545+0000 7f61ae7fc640 1 -- 172.21.15.76:0/1645443789 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f61c0015940 con 0x557214aec850 2023-11-09T03:45:38.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.545+0000 7f61d859e740 1 -- 172.21.15.76:0/1645443789 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572149d7bf0 con 0x557214aec850 2023-11-09T03:45:38.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.546+0000 7f61ae7fc640 1 -- 172.21.15.76:0/1645443789 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61c0015b40 con 0x557214aec850 2023-11-09T03:45:38.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.546+0000 7f61d859e740 10 client.4422 fetch_fsmap learned FSMap version 7 2023-11-09T03:45:38.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.546+0000 7f61d859e740 1 -- 172.21.15.76:0/1645443789 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x557214a29f30 con 0x557214aec850 2023-11-09T03:45:38.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.546+0000 7f61ae7fc640 1 -- 172.21.15.76:0/1645443789 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5572149d7bf0 con 0x557214aec850 2023-11-09T03:45:38.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.546+0000 7f61ae7fc640 1 -- 172.21.15.76:0/1645443789 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f61c0015ca0 con 0x557214aec850 2023-11-09T03:45:38.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.546+0000 7f61d859e740 10 client.4422 fetch_fsmap finished waiting for FSMap version 7 2023-11-09T03:45:38.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.546+0000 7f61d859e740 10 client.4422 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-09T03:45:38.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.546+0000 7f61d859e740 20 client.4422 populate_metadata read hostname 'smithi076' 2023-11-09T03:45:38.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.546+0000 7f61d859e740 1 --2- 172.21.15.76:0/1645443789 >> [v2:172.21.15.113:6834/3293223710,v1:172.21.15.113:6836/3293223710] conn(0x557214b03e00 0x557214b1dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:38.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.546+0000 7f61d859e740 4 client.4422 mds_command: new command op to 4291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:45:38.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.546+0000 7f61d859e740 1 -- 172.21.15.76:0/1645443789 --> [v2:172.21.15.113:6834/3293223710,v1:172.21.15.113:6836/3293223710] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557214a24890 con 0x557214b03e00 2023-11-09T03:45:38.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.546+0000 7f61ce7fc640 1 --2- 172.21.15.76:0/1645443789 >> [v2:172.21.15.113:6834/3293223710,v1:172.21.15.113:6836/3293223710] conn(0x557214b03e00 0x557214b1dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:38.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.546+0000 7f61ce7fc640 1 --2- 172.21.15.76:0/1645443789 >> [v2:172.21.15.113:6834/3293223710,v1:172.21.15.113:6836/3293223710] conn(0x557214b03e00 0x557214b1dc20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e59c854d7781a6a server_cookie=36a6aa4f9554a4d6 in_seq=0 out_seq=0 2023-11-09T03:45:38.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.546+0000 7f61ae7fc640 10 client.4422 ms_handle_connect on v2:172.21.15.113:6834/3293223710 2023-11-09T03:45:38.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.547+0000 7f61ae7fc640 1 -- 172.21.15.76:0/1645443789 <== mds.0 v2:172.21.15.113:6834/3293223710 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557214a24890 con 0x557214b03e00 2023-11-09T03:45:38.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.547+0000 7f61ae7fc640 10 client.4422 handle_command_reply: tid=0 2023-11-09T03:45:38.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.547+0000 7f61d859e740 1 client.4422 shutdown 2023-11-09T03:45:38.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.547+0000 7f61d859e740 1 -- 172.21.15.76:0/1645443789 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557214aec850 msgr2=0x557214b04e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:38.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.547+0000 7f61d859e740 1 --2- 172.21.15.76:0/1645443789 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557214aec850 0x557214b04e00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f61c0007380 tx=0x7f61c0001400 comp rx=0 tx=0).stop 2023-11-09T03:45:38.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.547+0000 7f61d859e740 1 -- 172.21.15.76:0/1645443789 shutdown_connections 2023-11-09T03:45:38.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.547+0000 7f61d859e740 1 --2- 172.21.15.76:0/1645443789 >> [v2:172.21.15.113:6834/3293223710,v1:172.21.15.113:6836/3293223710] conn(0x557214b03e00 0x557214b1dc20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.547+0000 7f61d859e740 1 --2- 172.21.15.76:0/1645443789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557214b05340 0x557214b0a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.547+0000 7f61ae7fc640 0 client.4422 ms_handle_reset on v2:172.21.15.113:6834/3293223710 2023-11-09T03:45:38.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.547+0000 7f61d859e740 1 --2- 172.21.15.76:0/1645443789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557214aecc30 0x557214b0a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.547+0000 7f61d859e740 1 --2- 172.21.15.76:0/1645443789 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557214aec850 0x557214b04e00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.547+0000 7f61d859e740 1 -- 172.21.15.76:0/1645443789 >> 172.21.15.76:0/1645443789 conn(0x557214abfc90 msgr2=0x557214abe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:38.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.547+0000 7f61d859e740 1 -- 172.21.15.76:0/1645443789 shutdown_connections 2023-11-09T03:45:38.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.547+0000 7f61d859e740 1 -- 172.21.15.76:0/1645443789 wait complete. 2023-11-09T03:45:38.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.547+0000 7f61d859e740 20 client.4422 trim_cache size 0 max 16384 2023-11-09T03:45:38.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.563+0000 7f61d859e740 1 Processor -- start 2023-11-09T03:45:38.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.563+0000 7f61d859e740 1 -- start start 2023-11-09T03:45:38.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.564+0000 7f61d859e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557214aec850 0x557214aef060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:38.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.564+0000 7f61d859e740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557214aecc30 0x557214aef5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:38.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.564+0000 7f61d859e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557214b05340 0x557214abe4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:38.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.564+0000 7f61d859e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572149ab280 con 0x557214b05340 2023-11-09T03:45:38.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.564+0000 7f61d859e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x557214aec4a0 con 0x557214aec850 2023-11-09T03:45:38.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.564+0000 7f61d859e740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x557214b017b0 con 0x557214aecc30 2023-11-09T03:45:38.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.564+0000 7f61cf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557214b05340 0x557214abe4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:38.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.564+0000 7f61cf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557214b05340 0x557214abe4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34260/0 (socket says 172.21.15.76:34260) 2023-11-09T03:45:38.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.564+0000 7f61cf7fe640 1 -- 172.21.15.76:0/111505983 learned_addr learned my addr 172.21.15.76:0/111505983 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:38.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.564+0000 7f61ceffd640 1 --2- 172.21.15.76:0/111505983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557214aec850 0x557214aef060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:38.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.564+0000 7f61ce7fc640 1 --2- 172.21.15.76:0/111505983 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557214aecc30 0x557214aef5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:38.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.564+0000 7f61cf7fe640 1 -- 172.21.15.76:0/111505983 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557214aecc30 msgr2=0x557214aef5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:38.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.564+0000 7f61cf7fe640 1 --2- 172.21.15.76:0/111505983 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557214aecc30 0x557214aef5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.564+0000 7f61cf7fe640 1 -- 172.21.15.76:0/111505983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557214aec850 msgr2=0x557214aef060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:38.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.564+0000 7f61cf7fe640 1 --2- 172.21.15.76:0/111505983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557214aec850 0x557214aef060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.564+0000 7f61cf7fe640 1 -- 172.21.15.76:0/111505983 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572149c7d10 con 0x557214b05340 2023-11-09T03:45:38.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.564+0000 7f61ce7fc640 1 --2- 172.21.15.76:0/111505983 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557214aecc30 0x557214aef5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:38.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.564+0000 7f61cf7fe640 1 --2- 172.21.15.76:0/111505983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557214b05340 0x557214abe4f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f61c8004e80 tx=0x7f61c8005340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=361ccdec61b446ab server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:38.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.564+0000 7f61b5ffb640 1 -- 172.21.15.76:0/111505983 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c80049d0 con 0x557214b05340 2023-11-09T03:45:38.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.564+0000 7f61b5ffb640 1 -- 172.21.15.76:0/111505983 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c800d070 con 0x557214b05340 2023-11-09T03:45:38.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.564+0000 7f61b5ffb640 1 -- 172.21.15.76:0/111505983 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c8012cd0 con 0x557214b05340 2023-11-09T03:45:38.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.565+0000 7f61d859e740 1 -- 172.21.15.76:0/111505983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557214b05340 msgr2=0x557214abe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:38.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.565+0000 7f61d859e740 1 --2- 172.21.15.76:0/111505983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557214b05340 0x557214abe4f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f61c8004e80 tx=0x7f61c8005340 comp rx=0 tx=0).stop 2023-11-09T03:45:38.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.565+0000 7f61d859e740 1 -- 172.21.15.76:0/111505983 shutdown_connections 2023-11-09T03:45:38.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.565+0000 7f61d859e740 1 --2- 172.21.15.76:0/111505983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557214b05340 0x557214abe4f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.565+0000 7f61d859e740 1 --2- 172.21.15.76:0/111505983 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557214aecc30 0x557214aef5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.565+0000 7f61d859e740 1 --2- 172.21.15.76:0/111505983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557214aec850 0x557214aef060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.565+0000 7f61d859e740 1 -- 172.21.15.76:0/111505983 >> 172.21.15.76:0/111505983 conn(0x557214abfc90 msgr2=0x557214abac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:38.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.565+0000 7f61d859e740 1 -- 172.21.15.76:0/111505983 shutdown_connections 2023-11-09T03:45:38.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.565+0000 7f61d859e740 1 -- 172.21.15.76:0/111505983 wait complete. 2023-11-09T03:45:38.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.565+0000 7f61d859e740 1 -- start start 2023-11-09T03:45:38.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.565+0000 7f61d859e740 1 Processor -- start 2023-11-09T03:45:38.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.566+0000 7f61d859e740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557214aec850 0x557214b75120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:38.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.566+0000 7f61d859e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557214aecc30 0x557214b2a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:38.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.566+0000 7f61d859e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557214b05340 0x557214af1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:38.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.566+0000 7f61d859e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557214b75660 con 0x557214b05340 2023-11-09T03:45:38.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.566+0000 7f61d859e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x557214b2ac20 con 0x557214aecc30 2023-11-09T03:45:38.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.566+0000 7f61d859e740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x557214b2b050 con 0x557214aec850 2023-11-09T03:45:38.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.566+0000 7f61d859e740 10 client.0 _ll_register_callbacks cb 0x7f61c40318e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:45:38.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.566+0000 7f61cf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557214b05340 0x557214af1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:38.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.566+0000 7f61cf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557214b05340 0x557214af1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34266/0 (socket says 172.21.15.76:34266) 2023-11-09T03:45:38.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.566+0000 7f61cf7fe640 1 -- 172.21.15.76:0/408378228 learned_addr learned my addr 172.21.15.76:0/408378228 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:38.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.566+0000 7f61cf7fe640 1 -- 172.21.15.76:0/408378228 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557214aec850 msgr2=0x557214b75120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:38.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.566+0000 7f61cf7fe640 1 --2- 172.21.15.76:0/408378228 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557214aec850 0x557214b75120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.566+0000 7f61ce7fc640 1 --2- 172.21.15.76:0/408378228 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557214aecc30 0x557214b2a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:38.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.566+0000 7f61cf7fe640 1 -- 172.21.15.76:0/408378228 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557214aecc30 msgr2=0x557214b2a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:38.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.566+0000 7f61cf7fe640 1 --2- 172.21.15.76:0/408378228 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557214aecc30 0x557214b2a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.566+0000 7f61cf7fe640 1 -- 172.21.15.76:0/408378228 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c8004b30 con 0x557214b05340 2023-11-09T03:45:38.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.567+0000 7f61cf7fe640 1 --2- 172.21.15.76:0/408378228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557214b05340 0x557214af1630 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f61c8012690 tx=0x7f61c80036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1bc49873a2a835 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:38.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.567+0000 7f61ae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:45:38.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.567+0000 7f61ae7fc640 1 -- 172.21.15.76:0/408378228 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c8025070 con 0x557214b05340 2023-11-09T03:45:38.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.567+0000 7f61ae7fc640 1 -- 172.21.15.76:0/408378228 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f61c80075c0 con 0x557214b05340 2023-11-09T03:45:38.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.567+0000 7f61d859e740 1 -- 172.21.15.76:0/408378228 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f61c40789e0 con 0x557214b05340 2023-11-09T03:45:38.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.567+0000 7f61ae7fc640 1 -- 172.21.15.76:0/408378228 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61c80049d0 con 0x557214b05340 2023-11-09T03:45:38.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.567+0000 7f61d859e740 10 client.4428 fetch_fsmap learned FSMap version 7 2023-11-09T03:45:38.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.567+0000 7f61d859e740 1 -- 172.21.15.76:0/408378228 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x5572149ed500 con 0x557214b05340 2023-11-09T03:45:38.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.567+0000 7f61ae7fc640 1 -- 172.21.15.76:0/408378228 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c40789e0 con 0x557214b05340 2023-11-09T03:45:38.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.567+0000 7f61ae7fc640 1 -- 172.21.15.76:0/408378228 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f61c80067d0 con 0x557214b05340 2023-11-09T03:45:38.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.567+0000 7f61d859e740 10 client.4428 fetch_fsmap finished waiting for FSMap version 7 2023-11-09T03:45:38.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.567+0000 7f61d859e740 10 client.4428 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-09T03:45:38.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.567+0000 7f61d859e740 20 client.4428 populate_metadata read hostname 'smithi076' 2023-11-09T03:45:38.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.567+0000 7f61d859e740 1 --2- 172.21.15.76:0/408378228 >> [v2:172.21.15.113:6834/3293223710,v1:172.21.15.113:6836/3293223710] conn(0x557214aef1c0 0x557214aef5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:38.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.567+0000 7f61d859e740 4 client.4428 mds_command: new command op to 4291 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-09T03:45:38.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.567+0000 7f61d859e740 1 -- 172.21.15.76:0/408378228 --> [v2:172.21.15.113:6834/3293223710,v1:172.21.15.113:6836/3293223710] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557214a245b0 con 0x557214aef1c0 2023-11-09T03:45:38.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.567+0000 7f61ceffd640 1 --2- 172.21.15.76:0/408378228 >> [v2:172.21.15.113:6834/3293223710,v1:172.21.15.113:6836/3293223710] conn(0x557214aef1c0 0x557214aef5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:38.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.568+0000 7f61ceffd640 1 --2- 172.21.15.76:0/408378228 >> [v2:172.21.15.113:6834/3293223710,v1:172.21.15.113:6836/3293223710] conn(0x557214aef1c0 0x557214aef5a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e84bccc06fb3718f server_cookie=a68b6007f6709256 in_seq=0 out_seq=0 2023-11-09T03:45:38.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.568+0000 7f61ae7fc640 10 client.4428 ms_handle_connect on v2:172.21.15.113:6834/3293223710 2023-11-09T03:45:38.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.568+0000 7f61ae7fc640 1 -- 172.21.15.76:0/408378228 <== mds.0 v2:172.21.15.113:6834/3293223710 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+317 (crc 0 0 0) 0x557214a245b0 con 0x557214aef1c0 2023-11-09T03:45:38.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.568+0000 7f61ae7fc640 10 client.4428 handle_command_reply: tid=0 2023-11-09T03:45:38.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.568+0000 7f61d859e740 1 client.4428 shutdown 2023-11-09T03:45:38.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.568+0000 7f61d859e740 1 -- 172.21.15.76:0/408378228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557214b05340 msgr2=0x557214af1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:38.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.568+0000 7f61d859e740 1 --2- 172.21.15.76:0/408378228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557214b05340 0x557214af1630 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f61c8012690 tx=0x7f61c80036f0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.568+0000 7f61d859e740 1 -- 172.21.15.76:0/408378228 shutdown_connections 2023-11-09T03:45:38.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.568+0000 7f61d859e740 1 --2- 172.21.15.76:0/408378228 >> [v2:172.21.15.113:6834/3293223710,v1:172.21.15.113:6836/3293223710] conn(0x557214aef1c0 0x557214aef5a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.568+0000 7f61d859e740 1 --2- 172.21.15.76:0/408378228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557214b05340 0x557214af1630 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.568+0000 7f61d859e740 1 --2- 172.21.15.76:0/408378228 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557214aecc30 0x557214b2a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.568+0000 7f61ae7fc640 0 client.4428 ms_handle_reset on v2:172.21.15.113:6834/3293223710 2023-11-09T03:45:38.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.568+0000 7f61d859e740 1 --2- 172.21.15.76:0/408378228 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557214aec850 0x557214b75120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.568+0000 7f61d859e740 1 -- 172.21.15.76:0/408378228 >> 172.21.15.76:0/408378228 conn(0x557214abfc90 msgr2=0x557214b0acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:38.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.568+0000 7f61d859e740 1 -- 172.21.15.76:0/408378228 shutdown_connections 2023-11-09T03:45:38.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.568+0000 7f61d859e740 1 -- 172.21.15.76:0/408378228 wait complete. 2023-11-09T03:45:38.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.568+0000 7f61d859e740 20 client.4428 trim_cache size 0 max 16384 2023-11-09T03:45:38.591 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:45:38.591 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "c213b26f-d0da-45d5-8e34-bb75c096d4f2", 2023-11-09T03:45:38.591 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2023-11-09T03:45:38.591 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 4291, 2023-11-09T03:45:38.591 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2023-11-09T03:45:38.592 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2023-11-09T03:45:38.592 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2023-11-09T03:45:38.592 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 7.166053496, 2023-11-09T03:45:38.592 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 7, 2023-11-09T03:45:38.592 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 25, 2023-11-09T03:45:38.593 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 25, 2023-11-09T03:45:38.593 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 14.213959156 2023-11-09T03:45:38.593 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:45:38.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.569+0000 7f61d5a0e640 1 -- 172.21.15.76:0/4200786079 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f61bc06ae70 msgr2=0x7f61bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:38.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.569+0000 7f61d5a0e640 1 --2- 172.21.15.76:0/4200786079 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f61bc06ae70 0x7f61bc06d330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f61c0007fd0 tx=0x7f61c0007ce0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.569+0000 7f61d5a0e640 1 -- 172.21.15.76:0/4200786079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61d00cd6d0 msgr2=0x7f61d0152aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:38.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.569+0000 7f61d5a0e640 1 --2- 172.21.15.76:0/4200786079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61d00cd6d0 0x7f61d0152aa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f61c40098c0 tx=0x7f61c40046c0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.569+0000 7f61d5a0e640 1 -- 172.21.15.76:0/4200786079 shutdown_connections 2023-11-09T03:45:38.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.569+0000 7f61d5a0e640 1 --2- 172.21.15.76:0/4200786079 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f61bc06ae70 0x7f61bc06d330 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.569+0000 7f61d5a0e640 1 --2- 172.21.15.76:0/4200786079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61d0153e60 0x7f61d0152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.569+0000 7f61d5a0e640 1 --2- 172.21.15.76:0/4200786079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61d00cd6d0 0x7f61d0152aa0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.569+0000 7f61d5a0e640 1 --2- 172.21.15.76:0/4200786079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61d00b4880 0x7f61d01547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.569+0000 7f61d5a0e640 1 -- 172.21.15.76:0/4200786079 >> 172.21.15.76:0/4200786079 conn(0x7f61d00cf880 msgr2=0x7f61d00d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:38.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.569+0000 7f61d5a0e640 1 -- 172.21.15.76:0/4200786079 shutdown_connections 2023-11-09T03:45:38.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.569+0000 7f61d5a0e640 1 -- 172.21.15.76:0/4200786079 wait complete. 2023-11-09T03:45:38.597 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-09T03:45:38.598 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-09T03:45:38.598 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-09T03:45:38.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.698+0000 7f2dd1f97640 1 Processor -- start 2023-11-09T03:45:38.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.698+0000 7f2dd1f97640 1 -- start start 2023-11-09T03:45:38.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.699+0000 7f2dd1f97640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2dcc0b0ae0 0x7f2dcc0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:38.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.699+0000 7f2dd1f97640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2dcc06db20 0x7f2dcc15d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:38.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.699+0000 7f2dd1f97640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dcc0b4110 con 0x7f2dcc159c70 2023-11-09T03:45:38.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.699+0000 7f2dd1f97640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dcc0b49b0 con 0x7f2dcc0b0ae0 2023-11-09T03:45:38.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.699+0000 7f2dd1f97640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dcc0b43f0 con 0x7f2dcc06db20 2023-11-09T03:45:38.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.699+0000 7f2dcaffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2dcc0b0ae0 0x7f2dcc0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:38.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.699+0000 7f2dcb7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2dcc159c70 0x7f2dcc0b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:51016/0 (socket says 172.21.15.76:51016) 2023-11-09T03:45:38.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.699+0000 7f2dcaffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2dcc0b0ae0 0x7f2dcc0ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34282/0 (socket says 172.21.15.76:34282) 2023-11-09T03:45:38.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.699+0000 7f2dcb7fe640 1 -- 172.21.15.76:0/2582484620 learned_addr learned my addr 172.21.15.76:0/2582484620 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:38.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.699+0000 7f2dcbfff640 1 --2- 172.21.15.76:0/2582484620 >> v2:172.21.15.113:3300/0 conn(0x7f2dcc06db20 0x7f2dcc15d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:38.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.699+0000 7f2dcaffd640 1 -- 172.21.15.76:0/2582484620 >> v2:172.21.15.113:3300/0 conn(0x7f2dcc06db20 msgr2=0x7f2dcc15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:38.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.699+0000 7f2dcaffd640 1 --2- 172.21.15.76:0/2582484620 >> v2:172.21.15.113:3300/0 conn(0x7f2dcc06db20 0x7f2dcc15d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.699+0000 7f2dcaffd640 1 -- 172.21.15.76:0/2582484620 >> v1:172.21.15.76:6789/0 conn(0x7f2dcc159c70 legacy=0x7f2dcc0b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:38.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.699+0000 7f2dca7fc640 1 -- 172.21.15.76:0/2582484620 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3687769716 0 0) 0x7f2dcc0b4110 con 0x7f2dcc159c70 2023-11-09T03:45:38.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.699+0000 7f2dcaffd640 1 -- 172.21.15.76:0/2582484620 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dcc0b46d0 con 0x7f2dcc0b0ae0 2023-11-09T03:45:38.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.699+0000 7f2dcbfff640 1 --2- 172.21.15.76:0/2582484620 >> v2:172.21.15.113:3300/0 conn(0x7f2dcc06db20 0x7f2dcc15d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:38.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.700+0000 7f2dcaffd640 1 --2- 172.21.15.76:0/2582484620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2dcc0b0ae0 0x7f2dcc0ce370 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc070580 tx=0x7f2dc002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e7cc5d3a95582a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:38.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.700+0000 7f2dca7fc640 1 -- 172.21.15.76:0/2582484620 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0031830 con 0x7f2dcc0b0ae0 2023-11-09T03:45:38.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.700+0000 7f2dca7fc640 1 -- 172.21.15.76:0/2582484620 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc0031990 con 0x7f2dcc0b0ae0 2023-11-09T03:45:38.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.700+0000 7f2dca7fc640 1 -- 172.21.15.76:0/2582484620 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0031d20 con 0x7f2dcc0b0ae0 2023-11-09T03:45:38.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.701+0000 7f2dd1f97640 1 -- 172.21.15.76:0/2582484620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2dcc0b0ae0 msgr2=0x7f2dcc0ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:38.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.701+0000 7f2dd1f97640 1 --2- 172.21.15.76:0/2582484620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2dcc0b0ae0 0x7f2dcc0ce370 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc070580 tx=0x7f2dc002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.701+0000 7f2dd1f97640 1 -- 172.21.15.76:0/2582484620 shutdown_connections 2023-11-09T03:45:38.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.701+0000 7f2dd1f97640 1 --2- 172.21.15.76:0/2582484620 >> v2:172.21.15.113:3300/0 conn(0x7f2dcc06db20 0x7f2dcc15d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.701+0000 7f2dd1f97640 1 --2- 172.21.15.76:0/2582484620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2dcc0b0ae0 0x7f2dcc0ce370 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.701+0000 7f2dd1f97640 1 -- 172.21.15.76:0/2582484620 >> 172.21.15.76:0/2582484620 conn(0x7f2dcc0ce950 msgr2=0x7f2dcc0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:38.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.701+0000 7f2dd1f97640 1 -- 172.21.15.76:0/2582484620 shutdown_connections 2023-11-09T03:45:38.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.701+0000 7f2dd1f97640 1 -- 172.21.15.76:0/2582484620 wait complete. 2023-11-09T03:45:38.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.702+0000 7f2dd1f97640 1 Processor -- start 2023-11-09T03:45:38.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.702+0000 7f2dd1f97640 1 -- start start 2023-11-09T03:45:38.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.703+0000 7f2dd1f97640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2dcc06db20 0x7f2dcc0ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:38.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.703+0000 7f2dd1f97640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dcc0b0ae0 0x7f2dcc14bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:38.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.703+0000 7f2dd1f97640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dcc159c70 0x7f2dcc156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:38.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.703+0000 7f2dd1f97640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc0a8350 con 0x7f2dcc06db20 2023-11-09T03:45:38.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.703+0000 7f2dd1f97640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc14c400 con 0x7f2dcc0b0ae0 2023-11-09T03:45:38.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.703+0000 7f2dd1f97640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc0ae7b0 con 0x7f2dcc159c70 2023-11-09T03:45:38.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.703+0000 7f2dcb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2dcc06db20 0x7f2dcc0ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:38.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.703+0000 7f2dcb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2dcc06db20 0x7f2dcc0ae1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34294/0 (socket says 172.21.15.76:34294) 2023-11-09T03:45:38.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.703+0000 7f2dcb7fe640 1 -- 172.21.15.76:0/2345056407 learned_addr learned my addr 172.21.15.76:0/2345056407 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:38.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.703+0000 7f2dcbfff640 1 --2- 172.21.15.76:0/2345056407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dcc159c70 0x7f2dcc156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:38.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.703+0000 7f2dcaffd640 1 --2- 172.21.15.76:0/2345056407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dcc0b0ae0 0x7f2dcc14bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:38.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.703+0000 7f2dcb7fe640 1 -- 172.21.15.76:0/2345056407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dcc159c70 msgr2=0x7f2dcc156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:38.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.703+0000 7f2dcb7fe640 1 --2- 172.21.15.76:0/2345056407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dcc159c70 0x7f2dcc156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.703+0000 7f2dcb7fe640 1 -- 172.21.15.76:0/2345056407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dcc0b0ae0 msgr2=0x7f2dcc14bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:38.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.703+0000 7f2dcb7fe640 1 --2- 172.21.15.76:0/2345056407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dcc0b0ae0 0x7f2dcc14bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.703+0000 7f2dcb7fe640 1 -- 172.21.15.76:0/2345056407 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc0009400 con 0x7f2dcc06db20 2023-11-09T03:45:38.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.703+0000 7f2dcb7fe640 1 --2- 172.21.15.76:0/2345056407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2dcc06db20 0x7f2dcc0ae1d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc00c4d0 tx=0x7f2dbc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:38.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.703+0000 7f2db3fff640 1 -- 172.21.15.76:0/2345056407 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc0074e0 con 0x7f2dcc06db20 2023-11-09T03:45:38.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.703+0000 7f2db3fff640 1 -- 172.21.15.76:0/2345056407 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dbc007660 con 0x7f2dcc06db20 2023-11-09T03:45:38.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.703+0000 7f2dd1f97640 1 -- 172.21.15.76:0/2345056407 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dcc0aea90 con 0x7f2dcc06db20 2023-11-09T03:45:38.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.703+0000 7f2db3fff640 1 -- 172.21.15.76:0/2345056407 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc004a20 con 0x7f2dcc06db20 2023-11-09T03:45:38.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.703+0000 7f2dd1f97640 1 -- 172.21.15.76:0/2345056407 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dcc0af050 con 0x7f2dcc06db20 2023-11-09T03:45:38.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.704+0000 7f2dd1f97640 1 -- 172.21.15.76:0/2345056407 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dcc074290 con 0x7f2dcc06db20 2023-11-09T03:45:38.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.705+0000 7f2db3fff640 1 -- 172.21.15.76:0/2345056407 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2dbc004b80 con 0x7f2dcc06db20 2023-11-09T03:45:38.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.705+0000 7f2db3fff640 1 --2- 172.21.15.76:0/2345056407 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2db806ae70 0x7f2db806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:38.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.705+0000 7f2db3fff640 1 -- 172.21.15.76:0/2345056407 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2dbc088de0 con 0x7f2dcc06db20 2023-11-09T03:45:38.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.705+0000 7f2dcaffd640 1 --2- 172.21.15.76:0/2345056407 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2db806ae70 0x7f2db806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:38.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.706+0000 7f2dcaffd640 1 --2- 172.21.15.76:0/2345056407 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2db806ae70 0x7f2db806d330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0036000 tx=0x7f2dc003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:38.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.710+0000 7f2db3fff640 1 -- 172.21.15.76:0/2345056407 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2dbc053c30 con 0x7f2dcc06db20 2023-11-09T03:45:38.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.910+0000 7f2dd1f97640 1 -- 172.21.15.76:0/2345056407 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2dcc06d3f0 con 0x7f2dcc06db20 2023-11-09T03:45:38.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.910+0000 7f2db3fff640 1 -- 172.21.15.76:0/2345056407 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f2dbc018020 con 0x7f2dcc06db20 2023-11-09T03:45:38.911 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-09T03:45:38.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.912+0000 7f2dd1f97640 1 -- 172.21.15.76:0/2345056407 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2db806ae70 msgr2=0x7f2db806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:38.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.912+0000 7f2dd1f97640 1 --2- 172.21.15.76:0/2345056407 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2db806ae70 0x7f2db806d330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0036000 tx=0x7f2dc003c040 comp rx=0 tx=0).stop 2023-11-09T03:45:38.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.912+0000 7f2dd1f97640 1 -- 172.21.15.76:0/2345056407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2dcc06db20 msgr2=0x7f2dcc0ae1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:38.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.912+0000 7f2dd1f97640 1 --2- 172.21.15.76:0/2345056407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2dcc06db20 0x7f2dcc0ae1d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc00c4d0 tx=0x7f2dbc00c990 comp rx=0 tx=0).stop 2023-11-09T03:45:38.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.912+0000 7f2dd1f97640 1 -- 172.21.15.76:0/2345056407 shutdown_connections 2023-11-09T03:45:38.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.912+0000 7f2dd1f97640 1 --2- 172.21.15.76:0/2345056407 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2db806ae70 0x7f2db806d330 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.912+0000 7f2dd1f97640 1 --2- 172.21.15.76:0/2345056407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dcc159c70 0x7f2dcc156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.912+0000 7f2dd1f97640 1 --2- 172.21.15.76:0/2345056407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dcc0b0ae0 0x7f2dcc14bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.912+0000 7f2dd1f97640 1 --2- 172.21.15.76:0/2345056407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2dcc06db20 0x7f2dcc0ae1d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:38.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.912+0000 7f2dd1f97640 1 -- 172.21.15.76:0/2345056407 >> 172.21.15.76:0/2345056407 conn(0x7f2dcc0ce950 msgr2=0x7f2dcc15bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:38.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.912+0000 7f2dd1f97640 1 -- 172.21.15.76:0/2345056407 shutdown_connections 2023-11-09T03:45:38.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:38.912+0000 7f2dd1f97640 1 -- 172.21.15.76:0/2345056407 wait complete. 2023-11-09T03:45:38.923 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-09T03:45:38.923 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-09T03:45:38.923 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2023-11-09T03:45:38.924 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-11-09T03:45:38.924 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-09T03:45:38.924 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-09T03:45:38.925 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-09T03:45:38.925 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-09T03:45:38.925 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-11-09T03:45:38.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:45:38.926 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2023-11-09T03:45:38.941 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-09T03:45:38.941 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-09T03:45:38.941 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2023-11-09T03:45:38.942 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:45:38.942 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2023-11-09T03:45:38.942 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:45:38.942 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:45:38.942 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:45:38.943 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2023-11-09T03:45:38.943 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:45:38.943 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:45:38.943 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2023-11-09T03:45:38.943 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:45:38.944 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:45:38.944 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-09T03:45:38.944 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:45:38.944 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-09T03:45:38.944 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42610sec preferred_lft 42610sec 2023-11-09T03:45:38.944 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2023-11-09T03:45:38.945 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:45:38.945 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-09T03:45:38.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:45:38.946 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-09T03:45:38.946 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-09T03:45:38.946 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link add name ceph-brx type bridge 2023-11-09T03:45:38.947 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip addr flush dev ceph-brx 2023-11-09T03:45:38.947 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set ceph-brx up 2023-11-09T03:45:38.947 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-09T03:45:38.948 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-09T03:45:39.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:45:39.091 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-09T03:45:39.118 INFO:teuthology.orchestra.run.smithi076.stdout:1 2023-11-09T03:45:39.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:45:39.120 DEBUG:teuthology.orchestra.run.smithi076:> route 2023-11-09T03:45:39.187 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2023-11-09T03:45:39.188 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-09T03:45:39.188 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-09T03:45:39.188 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-09T03:45:39.188 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-09T03:45:39.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:45:39.189 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-09T03:45:39.189 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-09T03:45:39.190 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-09T03:45:39.190 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-09T03:45:39.190 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-09T03:45:39.191 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-09T03:45:39.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:45:39.455 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2023-11-09T03:45:39.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:45:39.470 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list-id 2023-11-09T03:45:39.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:45:39.525 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-09T03:45:39.525 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-09T03:45:39.526 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-09T03:45:39.526 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-11-09T03:45:39.526 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-09T03:45:39.607 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-11-09T03:45:39.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:45:39.608 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-09T03:45:39.608 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-09T03:45:39.609 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-11-09T03:45:39.609 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-09T03:45:39.609 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-11-09T03:45:39.609 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-11-09T03:45:39.610 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-11-09T03:45:39.610 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-09T03:45:39.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:45:39.744 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-09T03:45:39.744 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-09T03:45:39.744 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2023-11-09T03:45:39.745 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set dev brx.0 master ceph-brx 2023-11-09T03:45:39.745 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-09T03:45:39.787 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-11-09T03:45:39.787 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:45:39.789 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-11-09T03:45:39.841 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-11-09T03:45:39.842 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:45:39.842 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-11-09T03:45:39.896 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:45:39.951 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:45:39.951 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:45:39.951 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:45:39.951 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:45:39.952 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:45:39.952 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:45:39.952 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:45:39.952 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:45:39.952 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:45:39.953 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:45:39.953 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:45:39.953 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:45:39.953 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:45:39.953 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:45:39.954 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:45:39.954 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:45:39.954 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:45:39.954 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:45:39.954 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:45:39.955 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:45:39.955 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:45:39.955 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:45:39.955 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:45:39.955 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:45:39.956 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:45:39.956 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:45:39.956 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:45:39.956 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:45:39.956 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:45:39.957 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:45:39.957 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-09T03:45:39.957 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-09T03:45:39.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:45:39.958 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:45:40.012 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-09T03:45:40.013 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-11-09T03:45:40.055 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:45:40.070 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:45:40.071 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:45:40.071 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:45:40.071 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:45:40.071 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:45:40.071 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:45:40.072 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:45:40.072 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:45:40.072 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:45:40.072 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:45:40.072 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:45:40.072 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:45:40.073 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:45:40.073 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:45:40.073 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:45:40.073 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:45:40.073 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:45:40.074 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:45:40.074 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:45:40.074 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:45:40.074 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:45:40.074 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:45:40.075 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:45:40.075 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:45:40.075 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:45:40.075 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:45:40.075 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:45:40.075 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:45:40.076 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:45:40.076 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:45:40.076 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-09T03:45:40.076 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-09T03:45:40.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:45:40.077 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:45:40.157 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi076.stderr:2023-11-09T03:45:40.157+0000 7f0140c0e480 -1 init, newargv = 0x55997cf30bc0 newargc=15 2023-11-09T03:45:40.157 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi076.stderr:ceph-fuse[94664]: starting ceph client 2023-11-09T03:45:40.162 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi076.stderr:ceph-fuse[94664]: starting fuse 2023-11-09T03:45:41.227 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:45:41.246 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:45:41.247 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:45:41.247 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:45:41.247 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:45:41.247 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:45:41.248 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:45:41.248 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:45:41.248 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:45:41.248 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:45:41.248 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:45:41.249 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:45:41.249 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:45:41.249 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:45:41.249 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:45:41.249 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:45:41.250 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:45:41.250 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:45:41.250 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:45:41.250 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:45:41.250 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:45:41.251 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:45:41.251 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:45:41.251 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:45:41.251 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:45:41.251 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:45:41.252 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:45:41.252 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:45:41.252 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:45:41.252 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:45:41.252 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:45:41.253 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-09T03:45:41.253 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-09T03:45:41.253 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.admin 2023-11-09T03:45:41.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:45:41.254 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:45:41.311 INFO:teuthology.orchestra.run.smithi076.stdout:43 2023-11-09T03:45:41.311 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-09T03:45:41.312 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer -- python3 -c ' 2023-11-09T03:45:41.312 DEBUG:teuthology.orchestra.run.smithi076:> import glob 2023-11-09T03:45:41.313 DEBUG:teuthology.orchestra.run.smithi076:> import re 2023-11-09T03:45:41.313 DEBUG:teuthology.orchestra.run.smithi076:> import os 2023-11-09T03:45:41.313 DEBUG:teuthology.orchestra.run.smithi076:> import subprocess 2023-11-09T03:45:41.314 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:45:41.314 DEBUG:teuthology.orchestra.run.smithi076:> def _find_admin_socket(client_name): 2023-11-09T03:45:41.314 DEBUG:teuthology.orchestra.run.smithi076:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-11-09T03:45:41.315 DEBUG:teuthology.orchestra.run.smithi076:> files = glob.glob(asok_path) 2023-11-09T03:45:41.315 DEBUG:teuthology.orchestra.run.smithi076:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-11-09T03:45:41.315 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:45:41.316 DEBUG:teuthology.orchestra.run.smithi076:> # Given a non-glob path, it better be there 2023-11-09T03:45:41.316 DEBUG:teuthology.orchestra.run.smithi076:> if "*" not in asok_path: 2023-11-09T03:45:41.316 DEBUG:teuthology.orchestra.run.smithi076:> assert(len(files) == 1) 2023-11-09T03:45:41.317 DEBUG:teuthology.orchestra.run.smithi076:> return files[0] 2023-11-09T03:45:41.317 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:45:41.317 DEBUG:teuthology.orchestra.run.smithi076:> for f in files: 2023-11-09T03:45:41.318 DEBUG:teuthology.orchestra.run.smithi076:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-09T03:45:41.318 DEBUG:teuthology.orchestra.run.smithi076:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-09T03:45:41.319 DEBUG:teuthology.orchestra.run.smithi076:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-09T03:45:41.319 DEBUG:teuthology.orchestra.run.smithi076:> contents = proc_f.read() 2023-11-09T03:45:41.319 DEBUG:teuthology.orchestra.run.smithi076:> if mountpoint in contents: 2023-11-09T03:45:41.320 DEBUG:teuthology.orchestra.run.smithi076:> return f 2023-11-09T03:45:41.320 DEBUG:teuthology.orchestra.run.smithi076:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-09T03:45:41.320 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:45:41.320 DEBUG:teuthology.orchestra.run.smithi076:> print(_find_admin_socket("client.admin")) 2023-11-09T03:45:41.321 DEBUG:teuthology.orchestra.run.smithi076:> ' 2023-11-09T03:45:41.442 INFO:teuthology.orchestra.run.smithi076.stdout:/var/run/ceph/ceph-client.admin.94664.asok 2023-11-09T03:45:41.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:45:41.453 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.94664.asok 2023-11-09T03:45:41.453 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:45:41.454 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.admin.94664.asok status 2023-11-09T03:45:41.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:45:41.624 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:45:41.624 INFO:teuthology.orchestra.run.smithi076.stdout: "metadata": { 2023-11-09T03:45:41.625 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_sha1": "7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4", 2023-11-09T03:45:41.625 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev)", 2023-11-09T03:45:41.625 INFO:teuthology.orchestra.run.smithi076.stdout: "entity_id": "admin", 2023-11-09T03:45:41.625 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": "smithi076", 2023-11-09T03:45:41.625 INFO:teuthology.orchestra.run.smithi076.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-11-09T03:45:41.626 INFO:teuthology.orchestra.run.smithi076.stdout: "pid": "94664", 2023-11-09T03:45:41.626 INFO:teuthology.orchestra.run.smithi076.stdout: "root": "/" 2023-11-09T03:45:41.626 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:45:41.626 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_count": 0, 2023-11-09T03:45:41.626 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_pinned_count": 0, 2023-11-09T03:45:41.627 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 4446, 2023-11-09T03:45:41.627 INFO:teuthology.orchestra.run.smithi076.stdout: "inst": { 2023-11-09T03:45:41.627 INFO:teuthology.orchestra.run.smithi076.stdout: "name": { 2023-11-09T03:45:41.627 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "client", 2023-11-09T03:45:41.627 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 4446 2023-11-09T03:45:41.628 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:45:41.628 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2023-11-09T03:45:41.628 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2023-11-09T03:45:41.628 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2023-11-09T03:45:41.628 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 4013414584 2023-11-09T03:45:41.629 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:45:41.629 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:45:41.629 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2023-11-09T03:45:41.629 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2023-11-09T03:45:41.629 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2023-11-09T03:45:41.629 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 4013414584 2023-11-09T03:45:41.630 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:45:41.630 INFO:teuthology.orchestra.run.smithi076.stdout: "inst_str": "client.4446 192.168.0.1:0/4013414584", 2023-11-09T03:45:41.630 INFO:teuthology.orchestra.run.smithi076.stdout: "addr_str": "192.168.0.1:0/4013414584", 2023-11-09T03:45:41.630 INFO:teuthology.orchestra.run.smithi076.stdout: "inode_count": 1, 2023-11-09T03:45:41.630 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_epoch": 7, 2023-11-09T03:45:41.631 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch": 25, 2023-11-09T03:45:41.631 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch_barrier": 0, 2023-11-09T03:45:41.631 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklisted": false, 2023-11-09T03:45:41.631 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs" 2023-11-09T03:45:41.631 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:45:41.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:45:41.638 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:45:41.638 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-11-09T03:45:41.639 DEBUG:teuthology.orchestra.run.smithi076:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-11-09T03:45:41.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:45:41.697 INFO:teuthology.orchestra.run.smithi076.stdout:fuseblk 2023-11-09T03:45:41.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:45:41.704 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-11-09T03:45:41.704 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:45:41.704 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-11-09T03:45:41.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:45:41.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:45:41.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:45:41.771 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-11-09T03:45:41.807 INFO:teuthology.orchestra.run.smithi076.stdout:# file: . 2023-11-09T03:45:41.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-11-09T03:45:41.807 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:41.808 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:45:41.808 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:45:41.862 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:45:42.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.002+0000 7fa2f9383640 1 Processor -- start 2023-11-09T03:45:42.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.002+0000 7fa2f9383640 1 -- start start 2023-11-09T03:45:42.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.003+0000 7fa2f9383640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa2f40cde60 0x7fa2f415d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:42.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.003+0000 7fa2f9383640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa2f40ce240 0x7fa2f40d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:42.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.003+0000 7fa2f9383640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2f415e000 con 0x7fa2f415daa0 2023-11-09T03:45:42.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.003+0000 7fa2f9383640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2f415e5c0 con 0x7fa2f40ce240 2023-11-09T03:45:42.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.003+0000 7fa2f9383640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2f415e2e0 con 0x7fa2f40cde60 2023-11-09T03:45:42.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.003+0000 7fa2f37fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa2f40ce240 0x7fa2f40d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:42.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.003+0000 7fa2f37fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa2f40ce240 0x7fa2f40d1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34304/0 (socket says 172.21.15.76:34304) 2023-11-09T03:45:42.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.003+0000 7fa2f37fe640 1 -- 172.21.15.76:0/1190871791 learned_addr learned my addr 172.21.15.76:0/1190871791 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:42.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.003+0000 7fa2f2ffd640 1 --2- 172.21.15.76:0/1190871791 >> v2:172.21.15.113:3300/0 conn(0x7fa2f40cde60 0x7fa2f415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:42.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.003+0000 7fa2f37fe640 1 -- 172.21.15.76:0/1190871791 >> v2:172.21.15.113:3300/0 conn(0x7fa2f40cde60 msgr2=0x7fa2f415d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:42.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.003+0000 7fa2f37fe640 1 --2- 172.21.15.76:0/1190871791 >> v2:172.21.15.113:3300/0 conn(0x7fa2f40cde60 0x7fa2f415d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.003+0000 7fa2f37fe640 1 -- 172.21.15.76:0/1190871791 >> v1:172.21.15.113:6789/0 conn(0x7fa2f415daa0 legacy=0x7fa2f4157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:42.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.003+0000 7fa2f37fe640 1 -- 172.21.15.76:0/1190871791 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f40d1120 con 0x7fa2f40ce240 2023-11-09T03:45:42.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.004+0000 7fa2f37fe640 1 --2- 172.21.15.76:0/1190871791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2f40ce240 0x7fa2f40d1920 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa2f40b5550 tx=0x7fa2ec02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966a0ba73ea07ac0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:42.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.004+0000 7fa2f27fc640 1 -- 172.21.15.76:0/1190871791 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec02cb20 con 0x7fa2f40ce240 2023-11-09T03:45:42.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.004+0000 7fa2f27fc640 1 -- 172.21.15.76:0/1190871791 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2ec02cc80 con 0x7fa2f40ce240 2023-11-09T03:45:42.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.004+0000 7fa2f27fc640 1 -- 172.21.15.76:0/1190871791 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec0343f0 con 0x7fa2f40ce240 2023-11-09T03:45:42.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.004+0000 7fa2f9383640 1 -- 172.21.15.76:0/1190871791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2f40ce240 msgr2=0x7fa2f40d1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:42.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.004+0000 7fa2f9383640 1 --2- 172.21.15.76:0/1190871791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2f40ce240 0x7fa2f40d1920 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa2f40b5550 tx=0x7fa2ec02bf80 comp rx=0 tx=0).stop 2023-11-09T03:45:42.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.005+0000 7fa2f9383640 1 -- 172.21.15.76:0/1190871791 shutdown_connections 2023-11-09T03:45:42.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.005+0000 7fa2f9383640 1 --2- 172.21.15.76:0/1190871791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2f40ce240 0x7fa2f40d1920 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.005+0000 7fa2f9383640 1 --2- 172.21.15.76:0/1190871791 >> v2:172.21.15.113:3300/0 conn(0x7fa2f40cde60 0x7fa2f415d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.005+0000 7fa2f9383640 1 -- 172.21.15.76:0/1190871791 >> 172.21.15.76:0/1190871791 conn(0x7fa2f4071f30 msgr2=0x7fa2f40cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:42.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.005+0000 7fa2f9383640 1 -- 172.21.15.76:0/1190871791 shutdown_connections 2023-11-09T03:45:42.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.005+0000 7fa2f9383640 1 -- 172.21.15.76:0/1190871791 wait complete. 2023-11-09T03:45:42.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.006+0000 7fa2f9383640 1 Processor -- start 2023-11-09T03:45:42.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.006+0000 7fa2f9383640 1 -- start start 2023-11-09T03:45:42.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.006+0000 7fa2f9383640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2f40cde60 0x7fa2f4154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:42.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.006+0000 7fa2f9383640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2f40ce240 0x7fa2f4154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:42.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.006+0000 7fa2f9383640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2f415daa0 0x7fa2f40cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:42.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.006+0000 7fa2f9383640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f41f61e0 con 0x7fa2f40cde60 2023-11-09T03:45:42.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.006+0000 7fa2f9383640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f41f64c0 con 0x7fa2f40ce240 2023-11-09T03:45:42.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.006+0000 7fa2f9383640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f41f67a0 con 0x7fa2f415daa0 2023-11-09T03:45:42.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.007+0000 7fa2f2ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2f40cde60 0x7fa2f4154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:42.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.007+0000 7fa2f2ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2f40cde60 0x7fa2f4154d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34314/0 (socket says 172.21.15.76:34314) 2023-11-09T03:45:42.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.007+0000 7fa2f2ffd640 1 -- 172.21.15.76:0/3305655902 learned_addr learned my addr 172.21.15.76:0/3305655902 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:42.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.007+0000 7fa2ebfff640 1 --2- 172.21.15.76:0/3305655902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2f40ce240 0x7fa2f4154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:42.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.007+0000 7fa2f37fe640 1 --2- 172.21.15.76:0/3305655902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2f415daa0 0x7fa2f40cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:42.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.007+0000 7fa2f2ffd640 1 -- 172.21.15.76:0/3305655902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2f415daa0 msgr2=0x7fa2f40cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:42.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.007+0000 7fa2f2ffd640 1 --2- 172.21.15.76:0/3305655902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2f415daa0 0x7fa2f40cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.007+0000 7fa2f2ffd640 1 -- 172.21.15.76:0/3305655902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2f40ce240 msgr2=0x7fa2f4154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:42.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.007+0000 7fa2f2ffd640 1 --2- 172.21.15.76:0/3305655902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2f40ce240 0x7fa2f4154790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.007+0000 7fa2f2ffd640 1 -- 172.21.15.76:0/3305655902 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ec009b30 con 0x7fa2f40cde60 2023-11-09T03:45:42.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.007+0000 7fa2f2ffd640 1 --2- 172.21.15.76:0/3305655902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2f40cde60 0x7fa2f4154d80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400e9b0 tx=0x7fa2e400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:42.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.007+0000 7fa2eb7fe640 1 -- 172.21.15.76:0/3305655902 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4009670 con 0x7fa2f40cde60 2023-11-09T03:45:42.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.007+0000 7fa2eb7fe640 1 -- 172.21.15.76:0/3305655902 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e400c9a0 con 0x7fa2f40cde60 2023-11-09T03:45:42.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.007+0000 7fa2f9383640 1 -- 172.21.15.76:0/3305655902 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f41f6a80 con 0x7fa2f40cde60 2023-11-09T03:45:42.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.007+0000 7fa2eb7fe640 1 -- 172.21.15.76:0/3305655902 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e400cd50 con 0x7fa2f40cde60 2023-11-09T03:45:42.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.007+0000 7fa2f9383640 1 -- 172.21.15.76:0/3305655902 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f41554f0 con 0x7fa2f40cde60 2023-11-09T03:45:42.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.008+0000 7fa2f9383640 1 -- 172.21.15.76:0/3305655902 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2f40c8da0 con 0x7fa2f40cde60 2023-11-09T03:45:42.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.009+0000 7fa2eb7fe640 1 -- 172.21.15.76:0/3305655902 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa2e4013470 con 0x7fa2f40cde60 2023-11-09T03:45:42.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.009+0000 7fa2eb7fe640 1 --2- 172.21.15.76:0/3305655902 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa2d406ae70 0x7fa2d406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:42.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.009+0000 7fa2eb7fe640 1 -- 172.21.15.76:0/3305655902 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa2e4089ac0 con 0x7fa2f40cde60 2023-11-09T03:45:42.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.009+0000 7fa2ebfff640 1 --2- 172.21.15.76:0/3305655902 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa2d406ae70 0x7fa2d406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:42.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.010+0000 7fa2ebfff640 1 --2- 172.21.15.76:0/3305655902 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa2d406ae70 0x7fa2d406d330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa2f41e4950 tx=0x7fa2dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:42.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.013+0000 7fa2eb7fe640 1 -- 172.21.15.76:0/3305655902 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa2e4054910 con 0x7fa2f40cde60 2023-11-09T03:45:42.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.181+0000 7fa2f9383640 1 -- 172.21.15.76:0/3305655902 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2f4074290 con 0x7fa2f40cde60 2023-11-09T03:45:42.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.182+0000 7fa2eb7fe640 1 -- 172.21.15.76:0/3305655902 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18334 (secure 0 0 0) 0x7fa2e4017090 con 0x7fa2f40cde60 2023-11-09T03:45:42.183 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:42.183 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":25,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:45:35.781752+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:45:42.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.184+0000 7fa2f9383640 1 -- 172.21.15.76:0/3305655902 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa2d406ae70 msgr2=0x7fa2d406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:42.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.184+0000 7fa2f9383640 1 --2- 172.21.15.76:0/3305655902 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa2d406ae70 0x7fa2d406d330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa2f41e4950 tx=0x7fa2dc009040 comp rx=0 tx=0).stop 2023-11-09T03:45:42.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.184+0000 7fa2f9383640 1 -- 172.21.15.76:0/3305655902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2f40cde60 msgr2=0x7fa2f4154d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:42.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.184+0000 7fa2f9383640 1 --2- 172.21.15.76:0/3305655902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2f40cde60 0x7fa2f4154d80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400e9b0 tx=0x7fa2e400ee70 comp rx=0 tx=0).stop 2023-11-09T03:45:42.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.184+0000 7fa2f9383640 1 -- 172.21.15.76:0/3305655902 shutdown_connections 2023-11-09T03:45:42.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.184+0000 7fa2f9383640 1 --2- 172.21.15.76:0/3305655902 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa2d406ae70 0x7fa2d406d330 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.184+0000 7fa2f9383640 1 --2- 172.21.15.76:0/3305655902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2f415daa0 0x7fa2f40cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.184+0000 7fa2f9383640 1 --2- 172.21.15.76:0/3305655902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2f40ce240 0x7fa2f4154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.184+0000 7fa2f9383640 1 --2- 172.21.15.76:0/3305655902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2f40cde60 0x7fa2f4154d80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.184+0000 7fa2f9383640 1 -- 172.21.15.76:0/3305655902 >> 172.21.15.76:0/3305655902 conn(0x7fa2f4071f30 msgr2=0x7fa2f40cd130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:42.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.184+0000 7fa2f9383640 1 -- 172.21.15.76:0/3305655902 shutdown_connections 2023-11-09T03:45:42.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.184+0000 7fa2f9383640 1 -- 172.21.15.76:0/3305655902 wait complete. 2023-11-09T03:45:42.195 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:45:28.704715+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:45:29.717614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-09T03:45:33.440577+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-09T03:45:42.196 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:45:42.298 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.298+0000 7f49963b6640 1 Processor -- start 2023-11-09T03:45:42.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.298+0000 7f49963b6640 1 -- start start 2023-11-09T03:45:42.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.299+0000 7f49963b6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f49900d0710 0x7f49900cd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:42.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.299+0000 7f49963b6640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49900d0280 con 0x7f49900cda50 2023-11-09T03:45:42.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.299+0000 7f49963b6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49900cf670 con 0x7f499015e430 2023-11-09T03:45:42.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.299+0000 7f49963b6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49900cf950 con 0x7f49900d0710 2023-11-09T03:45:42.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.299+0000 7f4994953640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f49900cda50 0x7f49900b3d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:51026/0 (socket says 172.21.15.76:51026) 2023-11-09T03:45:42.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.299+0000 7f4994953640 1 -- 172.21.15.76:0/2470963021 learned_addr learned my addr 172.21.15.76:0/2470963021 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:42.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.299+0000 7f498f7fe640 1 --2- 172.21.15.76:0/2470963021 >> v2:172.21.15.113:3300/0 conn(0x7f49900d0710 0x7f49900cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:42.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.300+0000 7f498effd640 1 -- 172.21.15.76:0/2470963021 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1342205632 0 0) 0x7f49900d0280 con 0x7f49900cda50 2023-11-09T03:45:42.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.300+0000 7f498effd640 1 -- 172.21.15.76:0/2470963021 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4978003520 con 0x7f49900cda50 2023-11-09T03:45:42.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.300+0000 7f498f7fe640 1 -- 172.21.15.76:0/2470963021 >> v1:172.21.15.113:6789/0 conn(0x7f499015e430 legacy=0x7f49900cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:42.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.300+0000 7f498f7fe640 1 -- 172.21.15.76:0/2470963021 >> v1:172.21.15.76:6789/0 conn(0x7f49900cda50 legacy=0x7f49900b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:42.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.300+0000 7f498f7fe640 1 -- 172.21.15.76:0/2470963021 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49900cfc30 con 0x7f49900d0710 2023-11-09T03:45:42.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.301+0000 7f498f7fe640 1 --2- 172.21.15.76:0/2470963021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49900d0710 0x7f49900cd510 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f49840097a0 tx=0x7f498402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bebe335177405ae7 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:42.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.301+0000 7f498effd640 1 -- 172.21.15.76:0/2470963021 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4984030610 con 0x7f49900d0710 2023-11-09T03:45:42.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.301+0000 7f498effd640 1 -- 172.21.15.76:0/2470963021 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4984030770 con 0x7f49900d0710 2023-11-09T03:45:42.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.301+0000 7f498effd640 1 -- 172.21.15.76:0/2470963021 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4984030ae0 con 0x7f49900d0710 2023-11-09T03:45:42.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.301+0000 7f49963b6640 1 -- 172.21.15.76:0/2470963021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49900d0710 msgr2=0x7f49900cd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:42.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.301+0000 7f49963b6640 1 --2- 172.21.15.76:0/2470963021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49900d0710 0x7f49900cd510 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f49840097a0 tx=0x7f498402b960 comp rx=0 tx=0).stop 2023-11-09T03:45:42.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.301+0000 7f49963b6640 1 -- 172.21.15.76:0/2470963021 shutdown_connections 2023-11-09T03:45:42.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.301+0000 7f49963b6640 1 --2- 172.21.15.76:0/2470963021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49900d0710 0x7f49900cd510 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.301+0000 7f49963b6640 1 -- 172.21.15.76:0/2470963021 >> 172.21.15.76:0/2470963021 conn(0x7f499006ea20 msgr2=0x7f499006ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:42.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.301+0000 7f49963b6640 1 -- 172.21.15.76:0/2470963021 shutdown_connections 2023-11-09T03:45:42.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.301+0000 7f49963b6640 1 -- 172.21.15.76:0/2470963021 wait complete. 2023-11-09T03:45:42.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.302+0000 7f49963b6640 1 Processor -- start 2023-11-09T03:45:42.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.303+0000 7f49963b6640 1 -- start start 2023-11-09T03:45:42.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.303+0000 7f49963b6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49900cda50 0x7f49900c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:42.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.303+0000 7f49963b6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49900d0710 0x7f49901593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:42.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.303+0000 7f49963b6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f499015e430 0x7f499014f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:42.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.303+0000 7f49963b6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49900c89f0 con 0x7f49900d0710 2023-11-09T03:45:42.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.303+0000 7f49963b6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49900c8cd0 con 0x7f499015e430 2023-11-09T03:45:42.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.303+0000 7f49963b6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49900c8fb0 con 0x7f49900cda50 2023-11-09T03:45:42.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.303+0000 7f498f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49900d0710 0x7f49901593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:42.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.303+0000 7f498f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49900d0710 0x7f49901593f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34316/0 (socket says 172.21.15.76:34316) 2023-11-09T03:45:42.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.303+0000 7f498f7fe640 1 -- 172.21.15.76:0/222279612 learned_addr learned my addr 172.21.15.76:0/222279612 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:42.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.304+0000 7f4994953640 1 --2- 172.21.15.76:0/222279612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f499015e430 0x7f499014f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:42.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.304+0000 7f498ffff640 1 --2- 172.21.15.76:0/222279612 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49900cda50 0x7f49900c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:42.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.304+0000 7f498f7fe640 1 -- 172.21.15.76:0/222279612 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49900cda50 msgr2=0x7f49900c9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:42.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.304+0000 7f498f7fe640 1 --2- 172.21.15.76:0/222279612 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49900cda50 0x7f49900c9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.304+0000 7f498f7fe640 1 -- 172.21.15.76:0/222279612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f499015e430 msgr2=0x7f499014f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:42.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.304+0000 7f498f7fe640 1 --2- 172.21.15.76:0/222279612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f499015e430 0x7f499014f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.304+0000 7f498f7fe640 1 -- 172.21.15.76:0/222279612 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4984009400 con 0x7f49900d0710 2023-11-09T03:45:42.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.304+0000 7f498f7fe640 1 --2- 172.21.15.76:0/222279612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49900d0710 0x7f49901593f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4984000c00 tx=0x7f49840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:42.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.304+0000 7f498cff9640 1 -- 172.21.15.76:0/222279612 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498403f070 con 0x7f49900d0710 2023-11-09T03:45:42.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.304+0000 7f498cff9640 1 -- 172.21.15.76:0/222279612 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4984030e30 con 0x7f49900d0710 2023-11-09T03:45:42.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.304+0000 7f49963b6640 1 -- 172.21.15.76:0/222279612 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49900c9290 con 0x7f49900d0710 2023-11-09T03:45:42.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.304+0000 7f498cff9640 1 -- 172.21.15.76:0/222279612 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4984004230 con 0x7f49900d0710 2023-11-09T03:45:42.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.304+0000 7f49963b6640 1 -- 172.21.15.76:0/222279612 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49900c8680 con 0x7f49900d0710 2023-11-09T03:45:42.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.305+0000 7f49963b6640 1 -- 172.21.15.76:0/222279612 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49900c97b0 con 0x7f49900d0710 2023-11-09T03:45:42.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.306+0000 7f498cff9640 1 -- 172.21.15.76:0/222279612 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4984043070 con 0x7f49900d0710 2023-11-09T03:45:42.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.306+0000 7f498cff9640 1 --2- 172.21.15.76:0/222279612 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f497c06ae70 0x7f497c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:42.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.306+0000 7f498cff9640 1 -- 172.21.15.76:0/222279612 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f49840ab560 con 0x7f49900d0710 2023-11-09T03:45:42.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.306+0000 7f498ffff640 1 --2- 172.21.15.76:0/222279612 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f497c06ae70 0x7f497c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:42.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.307+0000 7f498ffff640 1 --2- 172.21.15.76:0/222279612 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f497c06ae70 0x7f497c06d330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4980001640 tx=0x7f4980009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:42.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.310+0000 7f498cff9640 1 -- 172.21.15.76:0/222279612 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f49900c97b0 con 0x7f49900d0710 2023-11-09T03:45:42.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.477+0000 7f49963b6640 1 -- 172.21.15.76:0/222279612 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f49900c97b0 con 0x7f49900d0710 2023-11-09T03:45:42.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.477+0000 7f498cff9640 1 -- 172.21.15.76:0/222279612 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f49900c97b0 con 0x7f49900d0710 2023-11-09T03:45:42.478 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:45:42.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.480+0000 7f49963b6640 1 -- 172.21.15.76:0/222279612 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f497c06ae70 msgr2=0x7f497c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:42.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.480+0000 7f49963b6640 1 --2- 172.21.15.76:0/222279612 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f497c06ae70 0x7f497c06d330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4980001640 tx=0x7f4980009040 comp rx=0 tx=0).stop 2023-11-09T03:45:42.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.480+0000 7f49963b6640 1 -- 172.21.15.76:0/222279612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49900d0710 msgr2=0x7f49901593f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:42.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.480+0000 7f49963b6640 1 --2- 172.21.15.76:0/222279612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49900d0710 0x7f49901593f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4984000c00 tx=0x7f49840046c0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.480+0000 7f49963b6640 1 -- 172.21.15.76:0/222279612 shutdown_connections 2023-11-09T03:45:42.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.480+0000 7f49963b6640 1 --2- 172.21.15.76:0/222279612 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f497c06ae70 0x7f497c06d330 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.480+0000 7f49963b6640 1 --2- 172.21.15.76:0/222279612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f499015e430 0x7f499014f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.480+0000 7f49963b6640 1 --2- 172.21.15.76:0/222279612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49900d0710 0x7f49901593f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.480+0000 7f49963b6640 1 --2- 172.21.15.76:0/222279612 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49900cda50 0x7f49900c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.480+0000 7f49963b6640 1 -- 172.21.15.76:0/222279612 >> 172.21.15.76:0/222279612 conn(0x7f499006ea20 msgr2=0x7f49900ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:42.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.480+0000 7f49963b6640 1 -- 172.21.15.76:0/222279612 shutdown_connections 2023-11-09T03:45:42.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.480+0000 7f49963b6640 1 -- 172.21.15.76:0/222279612 wait complete. 2023-11-09T03:45:42.490 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:45:42.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.593+0000 7f29bdf16640 1 Processor -- start 2023-11-09T03:45:42.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.593+0000 7f29bdf16640 1 -- start start 2023-11-09T03:45:42.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.594+0000 7f29bdf16640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29b81605b0 0x7f29b80ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:42.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.594+0000 7f29bdf16640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f29b806a830 0x7f29b80cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:42.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.594+0000 7f29bdf16640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29b8160990 con 0x7f29b80b1a40 2023-11-09T03:45:42.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.594+0000 7f29bdf16640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29b80cc7c0 con 0x7f29b806a830 2023-11-09T03:45:42.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.594+0000 7f29bdf16640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b80cd750 con 0x7f29b81605b0 2023-11-09T03:45:42.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.594+0000 7f29b6ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f29b80b1a40 0x7f29b806a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:56614/0 (socket says 172.21.15.76:56614) 2023-11-09T03:45:42.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.594+0000 7f29b6ffd640 1 -- 172.21.15.76:0/3597624525 learned_addr learned my addr 172.21.15.76:0/3597624525 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:42.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.594+0000 7f29b77fe640 1 --2- 172.21.15.76:0/3597624525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29b81605b0 0x7f29b80ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:42.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.594+0000 7f29b7fff640 1 --2- 172.21.15.76:0/3597624525 >> v2:172.21.15.113:3300/0 conn(0x7f29b806a830 0x7f29b80cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:42.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.594+0000 7f29b67fc640 1 -- 172.21.15.76:0/3597624525 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2122719228 0 0) 0x7f29b8160990 con 0x7f29b80b1a40 2023-11-09T03:45:42.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.594+0000 7f29b67fc640 1 -- 172.21.15.76:0/3597624525 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29a0003520 con 0x7f29b80b1a40 2023-11-09T03:45:42.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.594+0000 7f29b67fc640 1 -- 172.21.15.76:0/3597624525 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1793964930 0 0) 0x7f29a0003520 con 0x7f29b80b1a40 2023-11-09T03:45:42.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.594+0000 7f29b67fc640 1 -- 172.21.15.76:0/3597624525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29b81605b0 msgr2=0x7f29b80ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:42.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.594+0000 7f29b67fc640 1 --2- 172.21.15.76:0/3597624525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29b81605b0 0x7f29b80ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.594+0000 7f29b67fc640 1 -- 172.21.15.76:0/3597624525 >> v2:172.21.15.113:3300/0 conn(0x7f29b806a830 msgr2=0x7f29b80cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:42.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.594+0000 7f29b67fc640 1 --2- 172.21.15.76:0/3597624525 >> v2:172.21.15.113:3300/0 conn(0x7f29b806a830 0x7f29b80cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.594+0000 7f29b67fc640 1 -- 172.21.15.76:0/3597624525 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b80cc4e0 con 0x7f29b80b1a40 2023-11-09T03:45:42.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.595+0000 7f29b67fc640 1 -- 172.21.15.76:0/3597624525 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f29ac001670 con 0x7f29b80b1a40 2023-11-09T03:45:42.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.595+0000 7f29b67fc640 1 -- 172.21.15.76:0/3597624525 >> v1:172.21.15.76:6789/0 conn(0x7f29b80b1a40 legacy=0x7f29b806a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:42.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.595+0000 7f29b67fc640 1 --2- 172.21.15.76:0/3597624525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29b81605b0 0x7f29a0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:42.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.595+0000 7f29b67fc640 1 --2- 172.21.15.76:0/3597624525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29a0003ef0 0x7f29a00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:42.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.595+0000 7f29b67fc640 1 --2- 172.21.15.76:0/3597624525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a0004d00 0x7f29a0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:42.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.595+0000 7f29b67fc640 1 -- 172.21.15.76:0/3597624525 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b80cc7c0 con 0x7f29a0003ef0 2023-11-09T03:45:42.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.595+0000 7f29b67fc640 1 -- 172.21.15.76:0/3597624525 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b80cd750 con 0x7f29a0004d00 2023-11-09T03:45:42.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.595+0000 7f29b67fc640 1 -- 172.21.15.76:0/3597624525 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a000ba60 con 0x7f29b81605b0 2023-11-09T03:45:42.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.595+0000 7f29b6ffd640 1 --2- 172.21.15.76:0/3597624525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29a0003ef0 0x7f29a00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:42.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.595+0000 7f29b77fe640 1 --2- 172.21.15.76:0/3597624525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29b81605b0 0x7f29a0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:42.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.595+0000 7f29b7fff640 1 --2- 172.21.15.76:0/3597624525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a0004d00 0x7f29a0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:42.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.595+0000 7f29b6ffd640 1 -- 172.21.15.76:0/3597624525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29b81605b0 msgr2=0x7f29a0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:42.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.595+0000 7f29b6ffd640 1 --2- 172.21.15.76:0/3597624525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29b81605b0 0x7f29a0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.595+0000 7f29b6ffd640 1 -- 172.21.15.76:0/3597624525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a0004d00 msgr2=0x7f29a0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:42.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.595+0000 7f29b6ffd640 1 --2- 172.21.15.76:0/3597624525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a0004d00 0x7f29a0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.595+0000 7f29b6ffd640 1 -- 172.21.15.76:0/3597624525 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a0003520 con 0x7f29a0003ef0 2023-11-09T03:45:42.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.596+0000 7f29b6ffd640 1 --2- 172.21.15.76:0/3597624525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29a0003ef0 0x7f29a00047c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00c210 tx=0x7f29a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14da9a61e433ef9 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:42.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.596+0000 7f29b67fc640 1 -- 172.21.15.76:0/3597624525 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac005ad0 con 0x7f29a0003ef0 2023-11-09T03:45:42.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.596+0000 7f29b67fc640 1 -- 172.21.15.76:0/3597624525 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ac005c30 con 0x7f29a0003ef0 2023-11-09T03:45:42.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.596+0000 7f29b67fc640 1 -- 172.21.15.76:0/3597624525 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac02c8d0 con 0x7f29a0003ef0 2023-11-09T03:45:42.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.596+0000 7f29bdf16640 1 -- 172.21.15.76:0/3597624525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29a0003ef0 msgr2=0x7f29a00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:42.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.596+0000 7f29bdf16640 1 --2- 172.21.15.76:0/3597624525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29a0003ef0 0x7f29a00047c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00c210 tx=0x7f29a00057f0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.596+0000 7f29b7fff640 1 -- 172.21.15.76:0/3597624525 reap_dead start 2023-11-09T03:45:42.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.597+0000 7f29bdf16640 1 -- 172.21.15.76:0/3597624525 shutdown_connections 2023-11-09T03:45:42.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.597+0000 7f29bdf16640 1 -- 172.21.15.76:0/3597624525 >> 172.21.15.76:0/3597624525 conn(0x7f29b8071820 msgr2=0x7f29b8071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:42.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.597+0000 7f29bdf16640 1 -- 172.21.15.76:0/3597624525 shutdown_connections 2023-11-09T03:45:42.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.597+0000 7f29bdf16640 1 -- 172.21.15.76:0/3597624525 wait complete. 2023-11-09T03:45:42.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.598+0000 7f29bdf16640 1 Processor -- start 2023-11-09T03:45:42.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.598+0000 7f29bdf16640 1 -- start start 2023-11-09T03:45:42.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.599+0000 7f29bdf16640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a0003ef0 0x7f29b81f6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:42.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.599+0000 7f29bdf16640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29b80b3a80 0x7f29b81f1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:42.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.599+0000 7f29bdf16640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29b80ad3e0 0x7f29b80ad860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:42.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.599+0000 7f29bdf16640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b80adda0 con 0x7f29b80b3a80 2023-11-09T03:45:42.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.599+0000 7f29bdf16640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b80b4e70 con 0x7f29a0003ef0 2023-11-09T03:45:42.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.599+0000 7f29bdf16640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b80b4260 con 0x7f29b80ad3e0 2023-11-09T03:45:42.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.599+0000 7f29b6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29b80b3a80 0x7f29b81f1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:42.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.599+0000 7f29b6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29b80b3a80 0x7f29b81f1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57334/0 (socket says 172.21.15.76:57334) 2023-11-09T03:45:42.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.599+0000 7f29b6ffd640 1 -- 172.21.15.76:0/4093622939 learned_addr learned my addr 172.21.15.76:0/4093622939 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:42.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.599+0000 7f29b7fff640 1 --2- 172.21.15.76:0/4093622939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29b80ad3e0 0x7f29b80ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:42.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.599+0000 7f29b77fe640 1 --2- 172.21.15.76:0/4093622939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a0003ef0 0x7f29b81f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:42.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.599+0000 7f29b6ffd640 1 -- 172.21.15.76:0/4093622939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29b80ad3e0 msgr2=0x7f29b80ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:42.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.599+0000 7f29b6ffd640 1 --2- 172.21.15.76:0/4093622939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29b80ad3e0 0x7f29b80ad860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.599+0000 7f29b6ffd640 1 -- 172.21.15.76:0/4093622939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a0003ef0 msgr2=0x7f29b81f6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:42.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.599+0000 7f29b6ffd640 1 --2- 172.21.15.76:0/4093622939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a0003ef0 0x7f29b81f6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.599+0000 7f29b6ffd640 1 -- 172.21.15.76:0/4093622939 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b80b4540 con 0x7f29b80b3a80 2023-11-09T03:45:42.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.599+0000 7f29b6ffd640 1 --2- 172.21.15.76:0/4093622939 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29b80b3a80 0x7f29b81f1f40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f29ac02c470 tx=0x7f29b80b2230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:42.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.599+0000 7f299ffff640 1 -- 172.21.15.76:0/4093622939 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac02cce0 con 0x7f29b80b3a80 2023-11-09T03:45:42.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.599+0000 7f299ffff640 1 -- 172.21.15.76:0/4093622939 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ac02ce40 con 0x7f29b80b3a80 2023-11-09T03:45:42.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.599+0000 7f29bdf16640 1 -- 172.21.15.76:0/4093622939 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b80b4820 con 0x7f29b80b3a80 2023-11-09T03:45:42.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.599+0000 7f299ffff640 1 -- 172.21.15.76:0/4093622939 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac0072b0 con 0x7f29b80b3a80 2023-11-09T03:45:42.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.600+0000 7f29bdf16640 1 -- 172.21.15.76:0/4093622939 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b81f2480 con 0x7f29b80b3a80 2023-11-09T03:45:42.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.601+0000 7f29bdf16640 1 -- 172.21.15.76:0/4093622939 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b80672a0 con 0x7f29b80b3a80 2023-11-09T03:45:42.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.601+0000 7f299ffff640 1 -- 172.21.15.76:0/4093622939 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f29ac032070 con 0x7f29b80b3a80 2023-11-09T03:45:42.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.601+0000 7f299ffff640 1 --2- 172.21.15.76:0/4093622939 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f29a406ae70 0x7f29a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:42.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.601+0000 7f299ffff640 1 -- 172.21.15.76:0/4093622939 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f29ac03c040 con 0x7f29b80b3a80 2023-11-09T03:45:42.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.601+0000 7f29b77fe640 1 --2- 172.21.15.76:0/4093622939 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f29a406ae70 0x7f29a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:42.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.602+0000 7f29b77fe640 1 --2- 172.21.15.76:0/4093622939 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f29a406ae70 0x7f29a406d330 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f29a8009a20 tx=0x7f29a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:42.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.606+0000 7f299ffff640 1 -- 172.21.15.76:0/4093622939 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f29ac075a60 con 0x7f29b80b3a80 2023-11-09T03:45:42.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.772+0000 7f29bdf16640 1 -- 172.21.15.76:0/4093622939 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f29b8074290 con 0x7f29b80b3a80 2023-11-09T03:45:42.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.772+0000 7f299ffff640 1 -- 172.21.15.76:0/4093622939 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f29ac079080 con 0x7f29b80b3a80 2023-11-09T03:45:42.773 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:45:42.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.774+0000 7f29bdf16640 1 -- 172.21.15.76:0/4093622939 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f29a406ae70 msgr2=0x7f29a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:42.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.774+0000 7f29bdf16640 1 --2- 172.21.15.76:0/4093622939 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f29a406ae70 0x7f29a406d330 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f29a8009a20 tx=0x7f29a8008040 comp rx=0 tx=0).stop 2023-11-09T03:45:42.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.774+0000 7f29bdf16640 1 -- 172.21.15.76:0/4093622939 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29b80b3a80 msgr2=0x7f29b81f1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:42.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.774+0000 7f29bdf16640 1 --2- 172.21.15.76:0/4093622939 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29b80b3a80 0x7f29b81f1f40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f29ac02c470 tx=0x7f29b80b2230 comp rx=0 tx=0).stop 2023-11-09T03:45:42.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.774+0000 7f29bdf16640 1 -- 172.21.15.76:0/4093622939 shutdown_connections 2023-11-09T03:45:42.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.774+0000 7f29bdf16640 1 --2- 172.21.15.76:0/4093622939 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f29a406ae70 0x7f29a406d330 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.774+0000 7f29bdf16640 1 --2- 172.21.15.76:0/4093622939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29b80ad3e0 0x7f29b80ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.774+0000 7f29bdf16640 1 --2- 172.21.15.76:0/4093622939 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29b80b3a80 0x7f29b81f1f40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.774+0000 7f29bdf16640 1 --2- 172.21.15.76:0/4093622939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a0003ef0 0x7f29b81f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.774+0000 7f29bdf16640 1 -- 172.21.15.76:0/4093622939 >> 172.21.15.76:0/4093622939 conn(0x7f29b8071820 msgr2=0x7f29b80b0510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:42.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.774+0000 7f29bdf16640 1 -- 172.21.15.76:0/4093622939 shutdown_connections 2023-11-09T03:45:42.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.774+0000 7f29bdf16640 1 -- 172.21.15.76:0/4093622939 wait complete. 2023-11-09T03:45:42.785 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:45:42.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.889+0000 7ff1d24c2640 1 Processor -- start 2023-11-09T03:45:42.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.889+0000 7ff1d24c2640 1 -- start start 2023-11-09T03:45:42.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.890+0000 7ff1d24c2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff1cc160430 0x7ff1cc0b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:42.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.890+0000 7ff1d24c2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff1cc0b2b60 0x7ff1cc06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:42.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.890+0000 7ff1d24c2640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1cc0b2fc0 con 0x7ff1cc06c1d0 2023-11-09T03:45:42.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.890+0000 7ff1d24c2640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1cc0cb580 con 0x7ff1cc0b2b60 2023-11-09T03:45:42.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.890+0000 7ff1d24c2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1cc06ca40 con 0x7ff1cc160430 2023-11-09T03:45:42.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.890+0000 7ff1cb7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff1cc0b2b60 0x7ff1cc06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:42.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.890+0000 7ff1d0a5f640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff1cc06c1d0 0x7ff1cc160b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:56626/0 (socket says 172.21.15.76:56626) 2023-11-09T03:45:42.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.890+0000 7ff1d0a5f640 1 -- 172.21.15.76:0/804835663 learned_addr learned my addr 172.21.15.76:0/804835663 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:42.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.890+0000 7ff1cbfff640 1 --2- 172.21.15.76:0/804835663 >> v2:172.21.15.113:3300/0 conn(0x7ff1cc160430 0x7ff1cc0b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:42.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.890+0000 7ff1cb7fe640 1 -- 172.21.15.76:0/804835663 >> v2:172.21.15.113:3300/0 conn(0x7ff1cc160430 msgr2=0x7ff1cc0b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:42.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.890+0000 7ff1cb7fe640 1 --2- 172.21.15.76:0/804835663 >> v2:172.21.15.113:3300/0 conn(0x7ff1cc160430 0x7ff1cc0b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.890+0000 7ff1cb7fe640 1 -- 172.21.15.76:0/804835663 >> v1:172.21.15.76:6789/0 conn(0x7ff1cc06c1d0 legacy=0x7ff1cc160b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:42.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.890+0000 7ff1cb7fe640 1 -- 172.21.15.76:0/804835663 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1cc06cd20 con 0x7ff1cc0b2b60 2023-11-09T03:45:42.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.890+0000 7ff1caffd640 1 -- 172.21.15.76:0/804835663 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1407888668 0 0) 0x7ff1cc0b2fc0 con 0x7ff1cc06c1d0 2023-11-09T03:45:42.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.891+0000 7ff1cb7fe640 1 --2- 172.21.15.76:0/804835663 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1cc0b2b60 0x7ff1cc06bc90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc06f6d0 tx=0x7ff1c002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2aff60e52049d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:42.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.891+0000 7ff1caffd640 1 -- 172.21.15.76:0/804835663 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0031830 con 0x7ff1cc0b2b60 2023-11-09T03:45:42.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.891+0000 7ff1caffd640 1 -- 172.21.15.76:0/804835663 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c0031990 con 0x7ff1cc0b2b60 2023-11-09T03:45:42.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.891+0000 7ff1caffd640 1 -- 172.21.15.76:0/804835663 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0031d20 con 0x7ff1cc0b2b60 2023-11-09T03:45:42.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.892+0000 7ff1d24c2640 1 -- 172.21.15.76:0/804835663 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1cc0b2b60 msgr2=0x7ff1cc06bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:42.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.892+0000 7ff1d24c2640 1 --2- 172.21.15.76:0/804835663 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1cc0b2b60 0x7ff1cc06bc90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc06f6d0 tx=0x7ff1c002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.892+0000 7ff1d24c2640 1 -- 172.21.15.76:0/804835663 shutdown_connections 2023-11-09T03:45:42.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.892+0000 7ff1d24c2640 1 --2- 172.21.15.76:0/804835663 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1cc0b2b60 0x7ff1cc06bc90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.892+0000 7ff1d24c2640 1 --2- 172.21.15.76:0/804835663 >> v2:172.21.15.113:3300/0 conn(0x7ff1cc160430 0x7ff1cc0b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.892+0000 7ff1d24c2640 1 -- 172.21.15.76:0/804835663 >> 172.21.15.76:0/804835663 conn(0x7ff1cc06ea20 msgr2=0x7ff1cc06ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:42.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.892+0000 7ff1d24c2640 1 -- 172.21.15.76:0/804835663 shutdown_connections 2023-11-09T03:45:42.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.892+0000 7ff1d24c2640 1 -- 172.21.15.76:0/804835663 wait complete. 2023-11-09T03:45:42.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.893+0000 7ff1d24c2640 1 Processor -- start 2023-11-09T03:45:42.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.893+0000 7ff1d24c2640 1 -- start start 2023-11-09T03:45:42.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.894+0000 7ff1d24c2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1cc06c1d0 0x7ff1cc14f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:42.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.894+0000 7ff1d24c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1cc0b2b60 0x7ff1cc0ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:42.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.894+0000 7ff1d24c2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1cc160430 0x7ff1cc0b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:42.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.894+0000 7ff1d24c2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc0cd850 con 0x7ff1cc0b2b60 2023-11-09T03:45:42.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.894+0000 7ff1d24c2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc0c8f60 con 0x7ff1cc06c1d0 2023-11-09T03:45:42.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.894+0000 7ff1d24c2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc0cd570 con 0x7ff1cc160430 2023-11-09T03:45:42.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.894+0000 7ff1cb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1cc0b2b60 0x7ff1cc0ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:42.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.894+0000 7ff1cb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1cc0b2b60 0x7ff1cc0ccc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57358/0 (socket says 172.21.15.76:57358) 2023-11-09T03:45:42.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.894+0000 7ff1cb7fe640 1 -- 172.21.15.76:0/1464083802 learned_addr learned my addr 172.21.15.76:0/1464083802 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:42.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.894+0000 7ff1cbfff640 1 --2- 172.21.15.76:0/1464083802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1cc06c1d0 0x7ff1cc14f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:42.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.894+0000 7ff1d0a5f640 1 --2- 172.21.15.76:0/1464083802 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1cc160430 0x7ff1cc0b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:42.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.894+0000 7ff1cb7fe640 1 -- 172.21.15.76:0/1464083802 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1cc160430 msgr2=0x7ff1cc0b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:42.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.894+0000 7ff1cb7fe640 1 --2- 172.21.15.76:0/1464083802 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1cc160430 0x7ff1cc0b41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.894+0000 7ff1cb7fe640 1 -- 172.21.15.76:0/1464083802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1cc06c1d0 msgr2=0x7ff1cc14f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:42.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.894+0000 7ff1cb7fe640 1 --2- 172.21.15.76:0/1464083802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1cc06c1d0 0x7ff1cc14f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:42.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.894+0000 7ff1cb7fe640 1 -- 172.21.15.76:0/1464083802 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c0009400 con 0x7ff1cc0b2b60 2023-11-09T03:45:42.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.894+0000 7ff1cb7fe640 1 --2- 172.21.15.76:0/1464083802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1cc0b2b60 0x7ff1cc0ccc00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0009890 tx=0x7ff1c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:42.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.895+0000 7ff1c8ff9640 1 -- 172.21.15.76:0/1464083802 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c003a6f0 con 0x7ff1cc0b2b60 2023-11-09T03:45:42.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.895+0000 7ff1c8ff9640 1 -- 172.21.15.76:0/1464083802 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c003a870 con 0x7ff1cc0b2b60 2023-11-09T03:45:42.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.895+0000 7ff1d24c2640 1 -- 172.21.15.76:0/1464083802 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1cc0c9200 con 0x7ff1cc0b2b60 2023-11-09T03:45:42.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.895+0000 7ff1c8ff9640 1 -- 172.21.15.76:0/1464083802 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c003ac20 con 0x7ff1cc0b2b60 2023-11-09T03:45:42.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.895+0000 7ff1d24c2640 1 -- 172.21.15.76:0/1464083802 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1cc150230 con 0x7ff1cc0b2b60 2023-11-09T03:45:42.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.896+0000 7ff1d24c2640 1 -- 172.21.15.76:0/1464083802 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1cc0c9720 con 0x7ff1cc0b2b60 2023-11-09T03:45:42.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.896+0000 7ff1c8ff9640 1 -- 172.21.15.76:0/1464083802 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff1c0043410 con 0x7ff1cc0b2b60 2023-11-09T03:45:42.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.896+0000 7ff1c8ff9640 1 --2- 172.21.15.76:0/1464083802 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff1b806ae70 0x7ff1b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:42.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.896+0000 7ff1c8ff9640 1 -- 172.21.15.76:0/1464083802 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff1c00ab450 con 0x7ff1cc0b2b60 2023-11-09T03:45:42.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.896+0000 7ff1cbfff640 1 --2- 172.21.15.76:0/1464083802 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff1b806ae70 0x7ff1b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:42.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.897+0000 7ff1cbfff640 1 --2- 172.21.15.76:0/1464083802 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff1b806ae70 0x7ff1b806d330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009aa0 tx=0x7ff1bc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:42.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:42.900+0000 7ff1c8ff9640 1 -- 172.21.15.76:0/1464083802 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff1cc0c9720 con 0x7ff1cc0b2b60 2023-11-09T03:45:43.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.065+0000 7ff1d24c2640 1 -- 172.21.15.76:0/1464083802 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff1cc0c9720 con 0x7ff1cc0b2b60 2023-11-09T03:45:43.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.065+0000 7ff1c8ff9640 1 -- 172.21.15.76:0/1464083802 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff1cc0c9720 con 0x7ff1cc0b2b60 2023-11-09T03:45:43.065 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:45:43.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.067+0000 7ff1d24c2640 1 -- 172.21.15.76:0/1464083802 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff1b806ae70 msgr2=0x7ff1b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:43.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.067+0000 7ff1d24c2640 1 --2- 172.21.15.76:0/1464083802 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff1b806ae70 0x7ff1b806d330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009aa0 tx=0x7ff1bc009190 comp rx=0 tx=0).stop 2023-11-09T03:45:43.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.067+0000 7ff1d24c2640 1 -- 172.21.15.76:0/1464083802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1cc0b2b60 msgr2=0x7ff1cc0ccc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:43.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.067+0000 7ff1d24c2640 1 --2- 172.21.15.76:0/1464083802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1cc0b2b60 0x7ff1cc0ccc00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0009890 tx=0x7ff1c00046c0 comp rx=0 tx=0).stop 2023-11-09T03:45:43.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.067+0000 7ff1d24c2640 1 -- 172.21.15.76:0/1464083802 shutdown_connections 2023-11-09T03:45:43.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.067+0000 7ff1d24c2640 1 --2- 172.21.15.76:0/1464083802 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff1b806ae70 0x7ff1b806d330 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:43.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.067+0000 7ff1d24c2640 1 --2- 172.21.15.76:0/1464083802 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1cc160430 0x7ff1cc0b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:43.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.067+0000 7ff1d24c2640 1 --2- 172.21.15.76:0/1464083802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1cc0b2b60 0x7ff1cc0ccc00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:43.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.067+0000 7ff1d24c2640 1 --2- 172.21.15.76:0/1464083802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1cc06c1d0 0x7ff1cc14f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:43.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.067+0000 7ff1d24c2640 1 -- 172.21.15.76:0/1464083802 >> 172.21.15.76:0/1464083802 conn(0x7ff1cc06ea20 msgr2=0x7ff1cc0b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:43.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.067+0000 7ff1d24c2640 1 -- 172.21.15.76:0/1464083802 shutdown_connections 2023-11-09T03:45:43.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.067+0000 7ff1d24c2640 1 -- 172.21.15.76:0/1464083802 wait complete. 2023-11-09T03:45:43.077 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-09T03:45:43.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.181+0000 7f923604b640 1 Processor -- start 2023-11-09T03:45:43.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.181+0000 7f923604b640 1 -- start start 2023-11-09T03:45:43.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.181+0000 7f923604b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9230160430 0x7f92300b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:43.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.181+0000 7f923604b640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92300b2fc0 con 0x7f923006c3a0 2023-11-09T03:45:43.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.182+0000 7f923604b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f923006c7d0 con 0x7f92300b2b60 2023-11-09T03:45:43.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.182+0000 7f923604b640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92300cb580 con 0x7f9230160430 2023-11-09T03:45:43.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.182+0000 7f922f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9230160430 0x7f92300b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:43.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.182+0000 7f922ffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f923006c3a0 0x7f92300cbcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:56642/0 (socket says 172.21.15.76:56642) 2023-11-09T03:45:43.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.182+0000 7f922f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9230160430 0x7f92300b2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57374/0 (socket says 172.21.15.76:57374) 2023-11-09T03:45:43.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.182+0000 7f922ffff640 1 -- 172.21.15.76:0/378998417 learned_addr learned my addr 172.21.15.76:0/378998417 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:43.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.182+0000 7f922f7fe640 1 -- 172.21.15.76:0/378998417 >> v1:172.21.15.113:6789/0 conn(0x7f92300b2b60 legacy=0x7f923006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:43.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.182+0000 7f922f7fe640 1 -- 172.21.15.76:0/378998417 >> v1:172.21.15.76:6789/0 conn(0x7f923006c3a0 legacy=0x7f92300cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:43.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.182+0000 7f922f7fe640 1 -- 172.21.15.76:0/378998417 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923006ca40 con 0x7f9230160430 2023-11-09T03:45:43.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.183+0000 7f922f7fe640 1 --2- 172.21.15.76:0/378998417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9230160430 0x7f92300b2620 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f92200097a0 tx=0x7f922002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b883c16d4213bc9c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:43.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.183+0000 7f922e7fc640 1 -- 172.21.15.76:0/378998417 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9220031830 con 0x7f9230160430 2023-11-09T03:45:43.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.183+0000 7f922e7fc640 1 -- 172.21.15.76:0/378998417 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9220031990 con 0x7f9230160430 2023-11-09T03:45:43.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.183+0000 7f922e7fc640 1 -- 172.21.15.76:0/378998417 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9220031d20 con 0x7f9230160430 2023-11-09T03:45:43.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.183+0000 7f923604b640 1 -- 172.21.15.76:0/378998417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9230160430 msgr2=0x7f92300b2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:43.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.183+0000 7f923604b640 1 --2- 172.21.15.76:0/378998417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9230160430 0x7f92300b2620 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f92200097a0 tx=0x7f922002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:45:43.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.183+0000 7f923604b640 1 -- 172.21.15.76:0/378998417 shutdown_connections 2023-11-09T03:45:43.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.183+0000 7f923604b640 1 --2- 172.21.15.76:0/378998417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9230160430 0x7f92300b2620 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:43.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.183+0000 7f923604b640 1 -- 172.21.15.76:0/378998417 >> 172.21.15.76:0/378998417 conn(0x7f923006ea20 msgr2=0x7f923006ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:43.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.183+0000 7f923604b640 1 -- 172.21.15.76:0/378998417 shutdown_connections 2023-11-09T03:45:43.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.184+0000 7f923604b640 1 -- 172.21.15.76:0/378998417 wait complete. 2023-11-09T03:45:43.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.185+0000 7f923604b640 1 Processor -- start 2023-11-09T03:45:43.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.185+0000 7f923604b640 1 -- start start 2023-11-09T03:45:43.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.185+0000 7f923604b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f923006c3a0 0x7f9230153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:43.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.185+0000 7f923604b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92300b2b60 0x7f92300ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:43.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.185+0000 7f923604b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9230160430 0x7f92300c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:43.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.185+0000 7f923604b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9230153b40 con 0x7f923006c3a0 2023-11-09T03:45:43.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.185+0000 7f923604b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92300c9210 con 0x7f9230160430 2023-11-09T03:45:43.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.185+0000 7f923604b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92300c94f0 con 0x7f92300b2b60 2023-11-09T03:45:43.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.186+0000 7f922f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f923006c3a0 0x7f9230153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:43.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.186+0000 7f922f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f923006c3a0 0x7f9230153d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57388/0 (socket says 172.21.15.76:57388) 2023-11-09T03:45:43.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.186+0000 7f922f7fe640 1 -- 172.21.15.76:0/1025384608 learned_addr learned my addr 172.21.15.76:0/1025384608 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:43.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.186+0000 7f922effd640 1 --2- 172.21.15.76:0/1025384608 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92300b2b60 0x7f92300ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:43.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.186+0000 7f922ffff640 1 --2- 172.21.15.76:0/1025384608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9230160430 0x7f92300c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:43.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.186+0000 7f922f7fe640 1 -- 172.21.15.76:0/1025384608 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92300b2b60 msgr2=0x7f92300ca2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:43.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.186+0000 7f922f7fe640 1 --2- 172.21.15.76:0/1025384608 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92300b2b60 0x7f92300ca2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:43.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.186+0000 7f922f7fe640 1 -- 172.21.15.76:0/1025384608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9230160430 msgr2=0x7f92300c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:43.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.186+0000 7f922f7fe640 1 --2- 172.21.15.76:0/1025384608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9230160430 0x7f92300c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:43.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.186+0000 7f922f7fe640 1 -- 172.21.15.76:0/1025384608 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9220009400 con 0x7f923006c3a0 2023-11-09T03:45:43.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.186+0000 7f922ffff640 1 --2- 172.21.15.76:0/1025384608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9230160430 0x7f92300c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:43.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.186+0000 7f922f7fe640 1 --2- 172.21.15.76:0/1025384608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f923006c3a0 0x7f9230153d20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f92200098c0 tx=0x7f92200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:43.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.186+0000 7f9217fff640 1 -- 172.21.15.76:0/1025384608 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922003ba70 con 0x7f923006c3a0 2023-11-09T03:45:43.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.186+0000 7f9217fff640 1 -- 172.21.15.76:0/1025384608 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f922003bbf0 con 0x7f923006c3a0 2023-11-09T03:45:43.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.186+0000 7f923604b640 1 -- 172.21.15.76:0/1025384608 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92300c97d0 con 0x7f923006c3a0 2023-11-09T03:45:43.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.186+0000 7f9217fff640 1 -- 172.21.15.76:0/1025384608 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92200453f0 con 0x7f923006c3a0 2023-11-09T03:45:43.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.186+0000 7f923604b640 1 -- 172.21.15.76:0/1025384608 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92300c9d90 con 0x7f923006c3a0 2023-11-09T03:45:43.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.187+0000 7f923604b640 1 -- 172.21.15.76:0/1025384608 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f923006d400 con 0x7f923006c3a0 2023-11-09T03:45:43.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.188+0000 7f9217fff640 1 -- 172.21.15.76:0/1025384608 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f922004e450 con 0x7f923006c3a0 2023-11-09T03:45:43.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.188+0000 7f9217fff640 1 --2- 172.21.15.76:0/1025384608 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f921c06ae70 0x7f921c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:43.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.188+0000 7f9217fff640 1 -- 172.21.15.76:0/1025384608 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f92200abba0 con 0x7f923006c3a0 2023-11-09T03:45:43.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.188+0000 7f922effd640 1 --2- 172.21.15.76:0/1025384608 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f921c06ae70 0x7f921c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:43.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.189+0000 7f922effd640 1 --2- 172.21.15.76:0/1025384608 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f921c06ae70 0x7f921c06d330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9230159080 tx=0x7f9224009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:43.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.192+0000 7f9217fff640 1 -- 172.21.15.76:0/1025384608 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9220076960 con 0x7f923006c3a0 2023-11-09T03:45:43.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.357+0000 7f923604b640 1 -- 172.21.15.76:0/1025384608 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9230074290 con 0x7f923006c3a0 2023-11-09T03:45:43.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.358+0000 7f9217fff640 1 -- 172.21.15.76:0/1025384608 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9220035020 con 0x7f923006c3a0 2023-11-09T03:45:43.357 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:45:43.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.359+0000 7f923604b640 1 -- 172.21.15.76:0/1025384608 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f921c06ae70 msgr2=0x7f921c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:43.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.359+0000 7f923604b640 1 --2- 172.21.15.76:0/1025384608 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f921c06ae70 0x7f921c06d330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9230159080 tx=0x7f9224009040 comp rx=0 tx=0).stop 2023-11-09T03:45:43.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.359+0000 7f923604b640 1 -- 172.21.15.76:0/1025384608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f923006c3a0 msgr2=0x7f9230153d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:43.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.359+0000 7f923604b640 1 --2- 172.21.15.76:0/1025384608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f923006c3a0 0x7f9230153d20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f92200098c0 tx=0x7f92200046c0 comp rx=0 tx=0).stop 2023-11-09T03:45:43.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.359+0000 7f923604b640 1 -- 172.21.15.76:0/1025384608 shutdown_connections 2023-11-09T03:45:43.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.359+0000 7f923604b640 1 --2- 172.21.15.76:0/1025384608 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f921c06ae70 0x7f921c06d330 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:43.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.359+0000 7f923604b640 1 --2- 172.21.15.76:0/1025384608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9230160430 0x7f92300c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:43.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.359+0000 7f923604b640 1 --2- 172.21.15.76:0/1025384608 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92300b2b60 0x7f92300ca2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:43.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.359+0000 7f923604b640 1 --2- 172.21.15.76:0/1025384608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f923006c3a0 0x7f9230153d20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:43.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.359+0000 7f923604b640 1 -- 172.21.15.76:0/1025384608 >> 172.21.15.76:0/1025384608 conn(0x7f923006ea20 msgr2=0x7f92300ce950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:43.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.360+0000 7f923604b640 1 -- 172.21.15.76:0/1025384608 shutdown_connections 2023-11-09T03:45:43.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.360+0000 7f923604b640 1 -- 172.21.15.76:0/1025384608 wait complete. 2023-11-09T03:45:43.370 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-09T03:45:43.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.474+0000 7f767eff5640 1 Processor -- start 2023-11-09T03:45:43.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.474+0000 7f767eff5640 1 -- start start 2023-11-09T03:45:43.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.475+0000 7f767eff5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7678160430 0x7f76780b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:43.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.475+0000 7f767eff5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f76780b2b60 0x7f767806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:43.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.475+0000 7f767eff5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76780b2fc0 con 0x7f767806c1d0 2023-11-09T03:45:43.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.475+0000 7f767eff5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76780cb580 con 0x7f76780b2b60 2023-11-09T03:45:43.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.475+0000 7f767eff5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f767806ca40 con 0x7f7678160430 2023-11-09T03:45:43.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.475+0000 7f767d592640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f767806c1d0 0x7f7678160b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:56644/0 (socket says 172.21.15.76:56644) 2023-11-09T03:45:43.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.475+0000 7f767d592640 1 -- 172.21.15.76:0/531922035 learned_addr learned my addr 172.21.15.76:0/531922035 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:43.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.475+0000 7f7677fff640 1 --2- 172.21.15.76:0/531922035 >> v2:172.21.15.113:3300/0 conn(0x7f76780b2b60 0x7f767806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:43.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.475+0000 7f767cd91640 1 --2- 172.21.15.76:0/531922035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7678160430 0x7f76780b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:43.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.475+0000 7f76777fe640 1 -- 172.21.15.76:0/531922035 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3416447970 0 0) 0x7f76780b2fc0 con 0x7f767806c1d0 2023-11-09T03:45:43.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.475+0000 7f76777fe640 1 -- 172.21.15.76:0/531922035 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7668003520 con 0x7f767806c1d0 2023-11-09T03:45:43.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.476+0000 7f7677fff640 1 -- 172.21.15.76:0/531922035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7678160430 msgr2=0x7f76780b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:43.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.476+0000 7f7677fff640 1 --2- 172.21.15.76:0/531922035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7678160430 0x7f76780b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:43.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.476+0000 7f7677fff640 1 -- 172.21.15.76:0/531922035 >> v1:172.21.15.76:6789/0 conn(0x7f767806c1d0 legacy=0x7f7678160b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:43.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.476+0000 7f76777fe640 1 -- 172.21.15.76:0/531922035 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1433140699 0 0) 0x7f7668003520 con 0x7f767806c1d0 2023-11-09T03:45:43.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.476+0000 7f7677fff640 1 -- 172.21.15.76:0/531922035 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767806cd20 con 0x7f76780b2b60 2023-11-09T03:45:43.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.476+0000 7f767cd91640 1 --2- 172.21.15.76:0/531922035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7678160430 0x7f76780b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:45:43.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.476+0000 7f7677fff640 1 --2- 172.21.15.76:0/531922035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76780b2b60 0x7f767806bc90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f76640097a0 tx=0x7f766402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72484947066640fe server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:43.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.477+0000 7f76777fe640 1 -- 172.21.15.76:0/531922035 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76700019f0 con 0x7f76780b2b60 2023-11-09T03:45:43.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.477+0000 7f76777fe640 1 -- 172.21.15.76:0/531922035 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766402e050 con 0x7f76780b2b60 2023-11-09T03:45:43.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.477+0000 7f76777fe640 1 -- 172.21.15.76:0/531922035 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7664030730 con 0x7f76780b2b60 2023-11-09T03:45:43.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.477+0000 7f767eff5640 1 -- 172.21.15.76:0/531922035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76780b2b60 msgr2=0x7f767806bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:43.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.477+0000 7f767eff5640 1 --2- 172.21.15.76:0/531922035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76780b2b60 0x7f767806bc90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f76640097a0 tx=0x7f766402b790 comp rx=0 tx=0).stop 2023-11-09T03:45:43.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.477+0000 7f767eff5640 1 -- 172.21.15.76:0/531922035 shutdown_connections 2023-11-09T03:45:43.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.477+0000 7f767eff5640 1 --2- 172.21.15.76:0/531922035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76780b2b60 0x7f767806bc90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:43.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.477+0000 7f767eff5640 1 --2- 172.21.15.76:0/531922035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7678160430 0x7f76780b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:43.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.477+0000 7f767eff5640 1 -- 172.21.15.76:0/531922035 >> 172.21.15.76:0/531922035 conn(0x7f767806ea20 msgr2=0x7f767806ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:43.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.477+0000 7f767eff5640 1 -- 172.21.15.76:0/531922035 shutdown_connections 2023-11-09T03:45:43.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.477+0000 7f767eff5640 1 -- 172.21.15.76:0/531922035 wait complete. 2023-11-09T03:45:43.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.478+0000 7f767eff5640 1 Processor -- start 2023-11-09T03:45:43.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.478+0000 7f767eff5640 1 -- start start 2023-11-09T03:45:43.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.479+0000 7f767eff5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f767806c1d0 0x7f76780c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:43.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.479+0000 7f767eff5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76780b2b60 0x7f76780ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:43.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.479+0000 7f767eff5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7678160430 0x7f76780cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:43.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.479+0000 7f767eff5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76780c9710 con 0x7f767806c1d0 2023-11-09T03:45:43.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.479+0000 7f767eff5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7678151820 con 0x7f7678160430 2023-11-09T03:45:43.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.479+0000 7f767eff5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7678152200 con 0x7f76780b2b60 2023-11-09T03:45:43.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.479+0000 7f767cd91640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f767806c1d0 0x7f76780c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:43.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.479+0000 7f767cd91640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f767806c1d0 0x7f76780c60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57390/0 (socket says 172.21.15.76:57390) 2023-11-09T03:45:43.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.479+0000 7f767cd91640 1 -- 172.21.15.76:0/225874071 learned_addr learned my addr 172.21.15.76:0/225874071 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:43.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.479+0000 7f767d592640 1 --2- 172.21.15.76:0/225874071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7678160430 0x7f76780cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:43.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.479+0000 7f7677fff640 1 --2- 172.21.15.76:0/225874071 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76780b2b60 0x7f76780ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:43.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.480+0000 7f767cd91640 1 -- 172.21.15.76:0/225874071 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76780b2b60 msgr2=0x7f76780ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:43.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.480+0000 7f767cd91640 1 --2- 172.21.15.76:0/225874071 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76780b2b60 0x7f76780ca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:43.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.480+0000 7f767cd91640 1 -- 172.21.15.76:0/225874071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7678160430 msgr2=0x7f76780cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:43.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.480+0000 7f767cd91640 1 --2- 172.21.15.76:0/225874071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7678160430 0x7f76780cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:43.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.480+0000 7f767cd91640 1 -- 172.21.15.76:0/225874071 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766c009400 con 0x7f767806c1d0 2023-11-09T03:45:43.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.480+0000 7f7677fff640 1 --2- 172.21.15.76:0/225874071 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76780b2b60 0x7f76780ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:43.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.480+0000 7f767d592640 1 --2- 172.21.15.76:0/225874071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7678160430 0x7f76780cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:43.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.480+0000 7f767cd91640 1 --2- 172.21.15.76:0/225874071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f767806c1d0 0x7f76780c60a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f766c00e960 tx=0x7f766c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:43.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.480+0000 7f76757fa640 1 -- 172.21.15.76:0/225874071 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c00ca00 con 0x7f767806c1d0 2023-11-09T03:45:43.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.480+0000 7f76757fa640 1 -- 172.21.15.76:0/225874071 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766c00cb60 con 0x7f767806c1d0 2023-11-09T03:45:43.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.480+0000 7f767eff5640 1 -- 172.21.15.76:0/225874071 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7664009400 con 0x7f767806c1d0 2023-11-09T03:45:43.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.480+0000 7f76757fa640 1 -- 172.21.15.76:0/225874071 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c013470 con 0x7f767806c1d0 2023-11-09T03:45:43.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.480+0000 7f767eff5640 1 -- 172.21.15.76:0/225874071 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7678151cb0 con 0x7f767806c1d0 2023-11-09T03:45:43.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.481+0000 7f767eff5640 1 -- 172.21.15.76:0/225874071 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76780c9980 con 0x7f767806c1d0 2023-11-09T03:45:43.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.481+0000 7f76757fa640 1 -- 172.21.15.76:0/225874071 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f766c0135d0 con 0x7f767806c1d0 2023-11-09T03:45:43.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.482+0000 7f76757fa640 1 --2- 172.21.15.76:0/225874071 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f765c06ae70 0x7f765c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:43.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.482+0000 7f76757fa640 1 -- 172.21.15.76:0/225874071 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f766c089b10 con 0x7f767806c1d0 2023-11-09T03:45:43.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.482+0000 7f7677fff640 1 --2- 172.21.15.76:0/225874071 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f765c06ae70 0x7f765c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:43.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.482+0000 7f7677fff640 1 --2- 172.21.15.76:0/225874071 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f765c06ae70 0x7f765c06d330 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7664034000 tx=0x7f766402f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:43.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.486+0000 7f76757fa640 1 -- 172.21.15.76:0/225874071 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f766c054960 con 0x7f767806c1d0 2023-11-09T03:45:43.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.665+0000 7f767eff5640 1 -- 172.21.15.76:0/225874071 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7678074290 con 0x7f767806c1d0 2023-11-09T03:45:43.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.666+0000 7f76757fa640 1 -- 172.21.15.76:0/225874071 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f766c017080 con 0x7f767806c1d0 2023-11-09T03:45:43.665 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2023-11-09T03:45:43.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.668+0000 7f767eff5640 1 -- 172.21.15.76:0/225874071 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f765c06ae70 msgr2=0x7f765c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:43.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.668+0000 7f767eff5640 1 --2- 172.21.15.76:0/225874071 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f765c06ae70 0x7f765c06d330 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7664034000 tx=0x7f766402f000 comp rx=0 tx=0).stop 2023-11-09T03:45:43.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.668+0000 7f767eff5640 1 -- 172.21.15.76:0/225874071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f767806c1d0 msgr2=0x7f76780c60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:43.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.668+0000 7f767eff5640 1 --2- 172.21.15.76:0/225874071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f767806c1d0 0x7f76780c60a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f766c00e960 tx=0x7f766c00ee20 comp rx=0 tx=0).stop 2023-11-09T03:45:43.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.668+0000 7f767eff5640 1 -- 172.21.15.76:0/225874071 shutdown_connections 2023-11-09T03:45:43.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.668+0000 7f767eff5640 1 --2- 172.21.15.76:0/225874071 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f765c06ae70 0x7f765c06d330 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:43.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.668+0000 7f767eff5640 1 --2- 172.21.15.76:0/225874071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7678160430 0x7f76780cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:43.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.668+0000 7f767eff5640 1 --2- 172.21.15.76:0/225874071 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76780b2b60 0x7f76780ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:43.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.668+0000 7f767eff5640 1 --2- 172.21.15.76:0/225874071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f767806c1d0 0x7f76780c60a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:43.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.668+0000 7f767eff5640 1 -- 172.21.15.76:0/225874071 >> 172.21.15.76:0/225874071 conn(0x7f767806ea20 msgr2=0x7f76780b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:43.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.668+0000 7f767eff5640 1 -- 172.21.15.76:0/225874071 shutdown_connections 2023-11-09T03:45:43.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.668+0000 7f767eff5640 1 -- 172.21.15.76:0/225874071 wait complete. 2023-11-09T03:45:43.677 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:45:43.678 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:45:43.693 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:45:43.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.832+0000 7f2d63d9e640 1 Processor -- start 2023-11-09T03:45:43.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.833+0000 7f2d63d9e640 1 -- start start 2023-11-09T03:45:43.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.833+0000 7f2d63d9e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2d5c159c70 0x7f2d5c0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:43.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.833+0000 7f2d63d9e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d5c0b4110 con 0x7f2d5c06db20 2023-11-09T03:45:43.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.833+0000 7f2d63d9e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d5c0b43f0 con 0x7f2d5c0b0ae0 2023-11-09T03:45:43.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.833+0000 7f2d63d9e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d5c0b49b0 con 0x7f2d5c159c70 2023-11-09T03:45:43.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.834+0000 7f2d61b3a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2d5c159c70 0x7f2d5c0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:43.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.834+0000 7f2d61339640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2d5c06db20 0x7f2d5c0b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:56654/0 (socket says 172.21.15.76:56654) 2023-11-09T03:45:43.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.834+0000 7f2d61339640 1 -- 172.21.15.76:0/4175108871 learned_addr learned my addr 172.21.15.76:0/4175108871 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:43.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.834+0000 7f2d61b3a640 1 -- 172.21.15.76:0/4175108871 >> v1:172.21.15.113:6789/0 conn(0x7f2d5c0b0ae0 legacy=0x7f2d5c15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:43.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.834+0000 7f2d61b3a640 1 -- 172.21.15.76:0/4175108871 >> v1:172.21.15.76:6789/0 conn(0x7f2d5c06db20 legacy=0x7f2d5c0b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:43.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.834+0000 7f2d60b38640 1 -- 172.21.15.76:0/4175108871 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 741445826 0 0) 0x7f2d5c0b4110 con 0x7f2d5c06db20 2023-11-09T03:45:43.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.834+0000 7f2d61b3a640 1 -- 172.21.15.76:0/4175108871 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d5c0b46d0 con 0x7f2d5c159c70 2023-11-09T03:45:43.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.835+0000 7f2d61b3a640 1 --2- 172.21.15.76:0/4175108871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d5c159c70 0x7f2d5c0ce370 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0097a0 tx=0x7f2d4c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59603dd615a900b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:43.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.835+0000 7f2d60b38640 1 -- 172.21.15.76:0/4175108871 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c031830 con 0x7f2d5c159c70 2023-11-09T03:45:43.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.835+0000 7f2d60b38640 1 -- 172.21.15.76:0/4175108871 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4c031990 con 0x7f2d5c159c70 2023-11-09T03:45:43.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.835+0000 7f2d60b38640 1 -- 172.21.15.76:0/4175108871 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c031d20 con 0x7f2d5c159c70 2023-11-09T03:45:43.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.835+0000 7f2d63d9e640 1 -- 172.21.15.76:0/4175108871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d5c159c70 msgr2=0x7f2d5c0ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:43.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.835+0000 7f2d63d9e640 1 --2- 172.21.15.76:0/4175108871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d5c159c70 0x7f2d5c0ce370 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0097a0 tx=0x7f2d4c02b8d0 comp rx=0 tx=0).stop 2023-11-09T03:45:43.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.835+0000 7f2d63d9e640 1 -- 172.21.15.76:0/4175108871 shutdown_connections 2023-11-09T03:45:43.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.835+0000 7f2d63d9e640 1 --2- 172.21.15.76:0/4175108871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d5c159c70 0x7f2d5c0ce370 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:43.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.835+0000 7f2d63d9e640 1 -- 172.21.15.76:0/4175108871 >> 172.21.15.76:0/4175108871 conn(0x7f2d5c0ce950 msgr2=0x7f2d5c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:43.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.835+0000 7f2d63d9e640 1 -- 172.21.15.76:0/4175108871 shutdown_connections 2023-11-09T03:45:43.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.835+0000 7f2d63d9e640 1 -- 172.21.15.76:0/4175108871 wait complete. 2023-11-09T03:45:43.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.837+0000 7f2d63d9e640 1 Processor -- start 2023-11-09T03:45:43.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.837+0000 7f2d63d9e640 1 -- start start 2023-11-09T03:45:43.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.837+0000 7f2d63d9e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d5c06db20 0x7f2d5c1f0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:43.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.837+0000 7f2d63d9e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d5c0b0ae0 0x7f2d5c1ed890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:43.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.837+0000 7f2d63d9e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d5c159c70 0x7f2d5c156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:43.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.837+0000 7f2d63d9e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c0c8250 con 0x7f2d5c0b0ae0 2023-11-09T03:45:43.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.837+0000 7f2d63d9e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c1f1420 con 0x7f2d5c159c70 2023-11-09T03:45:43.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.837+0000 7f2d63d9e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c1eddd0 con 0x7f2d5c06db20 2023-11-09T03:45:43.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.837+0000 7f2d61339640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d5c0b0ae0 0x7f2d5c1ed890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:43.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.837+0000 7f2d61339640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d5c0b0ae0 0x7f2d5c1ed890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57420/0 (socket says 172.21.15.76:57420) 2023-11-09T03:45:43.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.837+0000 7f2d61339640 1 -- 172.21.15.76:0/724167106 learned_addr learned my addr 172.21.15.76:0/724167106 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:43.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.838+0000 7f2d6233b640 1 --2- 172.21.15.76:0/724167106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d5c159c70 0x7f2d5c156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:43.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.838+0000 7f2d61339640 1 -- 172.21.15.76:0/724167106 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d5c06db20 msgr2=0x7f2d5c1f0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:43.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.838+0000 7f2d61339640 1 --2- 172.21.15.76:0/724167106 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d5c06db20 0x7f2d5c1f0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:43.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.838+0000 7f2d61339640 1 -- 172.21.15.76:0/724167106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d5c159c70 msgr2=0x7f2d5c156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:43.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.838+0000 7f2d61339640 1 --2- 172.21.15.76:0/724167106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d5c159c70 0x7f2d5c156470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:43.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.838+0000 7f2d61339640 1 -- 172.21.15.76:0/724167106 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4c009400 con 0x7f2d5c0b0ae0 2023-11-09T03:45:43.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.838+0000 7f2d61339640 1 --2- 172.21.15.76:0/724167106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d5c0b0ae0 0x7f2d5c1ed890 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800c4d0 tx=0x7f2d5800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:43.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.838+0000 7f2d527fc640 1 -- 172.21.15.76:0/724167106 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d580074e0 con 0x7f2d5c0b0ae0 2023-11-09T03:45:43.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.838+0000 7f2d527fc640 1 -- 172.21.15.76:0/724167106 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d58007660 con 0x7f2d5c0b0ae0 2023-11-09T03:45:43.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.838+0000 7f2d63d9e640 1 -- 172.21.15.76:0/724167106 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d5c0c5d30 con 0x7f2d5c0b0ae0 2023-11-09T03:45:43.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.838+0000 7f2d527fc640 1 -- 172.21.15.76:0/724167106 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d58004a20 con 0x7f2d5c0b0ae0 2023-11-09T03:45:43.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.838+0000 7f2d63d9e640 1 -- 172.21.15.76:0/724167106 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d5c0c62f0 con 0x7f2d5c0b0ae0 2023-11-09T03:45:43.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.839+0000 7f2d63d9e640 1 -- 172.21.15.76:0/724167106 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d5c074290 con 0x7f2d5c0b0ae0 2023-11-09T03:45:43.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.839+0000 7f2d527fc640 1 -- 172.21.15.76:0/724167106 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2d58004b80 con 0x7f2d5c0b0ae0 2023-11-09T03:45:43.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.840+0000 7f2d527fc640 1 --2- 172.21.15.76:0/724167106 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2d4006aea0 0x7f2d4006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:43.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.840+0000 7f2d527fc640 1 -- 172.21.15.76:0/724167106 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2d58088a70 con 0x7f2d5c0b0ae0 2023-11-09T03:45:43.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.840+0000 7f2d61b3a640 1 --2- 172.21.15.76:0/724167106 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2d4006aea0 0x7f2d4006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:43.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.840+0000 7f2d61b3a640 1 --2- 172.21.15.76:0/724167106 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2d4006aea0 0x7f2d4006d360 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c036000 tx=0x7f2d4c030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:43.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:43.844+0000 7f2d527fc640 1 -- 172.21.15.76:0/724167106 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2d58053830 con 0x7f2d5c0b0ae0 2023-11-09T03:45:44.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.012+0000 7f2d63d9e640 1 -- 172.21.15.76:0/724167106 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d5c06d3f0 con 0x7f2d5c0b0ae0 2023-11-09T03:45:44.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.013+0000 7f2d527fc640 1 -- 172.21.15.76:0/724167106 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18334 (secure 0 0 0) 0x7f2d58018020 con 0x7f2d5c0b0ae0 2023-11-09T03:45:44.013 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:44.013 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":25,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:45:35.781752+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:45:44.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.015+0000 7f2d63d9e640 1 -- 172.21.15.76:0/724167106 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2d4006aea0 msgr2=0x7f2d4006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:44.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.015+0000 7f2d63d9e640 1 --2- 172.21.15.76:0/724167106 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2d4006aea0 0x7f2d4006d360 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c036000 tx=0x7f2d4c030000 comp rx=0 tx=0).stop 2023-11-09T03:45:44.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.015+0000 7f2d63d9e640 1 -- 172.21.15.76:0/724167106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d5c0b0ae0 msgr2=0x7f2d5c1ed890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:44.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.015+0000 7f2d63d9e640 1 --2- 172.21.15.76:0/724167106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d5c0b0ae0 0x7f2d5c1ed890 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800c4d0 tx=0x7f2d5800c990 comp rx=0 tx=0).stop 2023-11-09T03:45:44.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.015+0000 7f2d63d9e640 1 -- 172.21.15.76:0/724167106 shutdown_connections 2023-11-09T03:45:44.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.015+0000 7f2d63d9e640 1 --2- 172.21.15.76:0/724167106 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2d4006aea0 0x7f2d4006d360 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.015+0000 7f2d63d9e640 1 --2- 172.21.15.76:0/724167106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d5c159c70 0x7f2d5c156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.015+0000 7f2d63d9e640 1 --2- 172.21.15.76:0/724167106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d5c0b0ae0 0x7f2d5c1ed890 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.015+0000 7f2d63d9e640 1 --2- 172.21.15.76:0/724167106 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d5c06db20 0x7f2d5c1f0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.015+0000 7f2d63d9e640 1 -- 172.21.15.76:0/724167106 >> 172.21.15.76:0/724167106 conn(0x7f2d5c0ce950 msgr2=0x7f2d5c15b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:44.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.015+0000 7f2d63d9e640 1 -- 172.21.15.76:0/724167106 shutdown_connections 2023-11-09T03:45:44.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.015+0000 7f2d63d9e640 1 -- 172.21.15.76:0/724167106 wait complete. 2023-11-09T03:45:44.025 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:45:28.704715+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:45:29.717614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-09T03:45:33.440577+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-09T03:45:44.026 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:45:44.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.129+0000 7f6208e26640 1 Processor -- start 2023-11-09T03:45:44.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.129+0000 7f6208e26640 1 -- start start 2023-11-09T03:45:44.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.129+0000 7f6208e26640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f62040c4ff0 0x7f62040cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:44.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.129+0000 7f6208e26640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f62040adac0 0x7f62040adf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:44.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.129+0000 7f6208e26640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f620415aff0 con 0x7f6204159c00 2023-11-09T03:45:44.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.129+0000 7f6208e26640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62040ae460 con 0x7f62040c4ff0 2023-11-09T03:45:44.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.129+0000 7f6208e26640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f620415a3e0 con 0x7f62040adac0 2023-11-09T03:45:44.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.130+0000 7f61fbfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f62040c4ff0 0x7f62040cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:44.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.130+0000 7f61fbfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f62040c4ff0 0x7f62040cc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57424/0 (socket says 172.21.15.76:57424) 2023-11-09T03:45:44.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.130+0000 7f61fbfff640 1 -- 172.21.15.76:0/254459165 learned_addr learned my addr 172.21.15.76:0/254459165 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:44.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.130+0000 7f6202d9d640 1 --2- 172.21.15.76:0/254459165 >> v2:172.21.15.113:3300/0 conn(0x7f62040adac0 0x7f62040adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:44.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.130+0000 7f61fbfff640 1 -- 172.21.15.76:0/254459165 >> v2:172.21.15.113:3300/0 conn(0x7f62040adac0 msgr2=0x7f62040adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:44.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.130+0000 7f61fbfff640 1 --2- 172.21.15.76:0/254459165 >> v2:172.21.15.113:3300/0 conn(0x7f62040adac0 0x7f62040adf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.130+0000 7f61fbfff640 1 -- 172.21.15.76:0/254459165 >> v1:172.21.15.113:6789/0 conn(0x7f6204159c00 legacy=0x7f62040c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:44.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.130+0000 7f61fbfff640 1 -- 172.21.15.76:0/254459165 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620415a6c0 con 0x7f62040c4ff0 2023-11-09T03:45:44.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.131+0000 7f61fbfff640 1 --2- 172.21.15.76:0/254459165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62040c4ff0 0x7f62040cc890 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f62040c59a0 tx=0x7f61ec02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18c6f7c2fa877bf server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:44.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.131+0000 7f6201d9b640 1 -- 172.21.15.76:0/254459165 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec030860 con 0x7f62040c4ff0 2023-11-09T03:45:44.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.131+0000 7f6201d9b640 1 -- 172.21.15.76:0/254459165 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ec0309c0 con 0x7f62040c4ff0 2023-11-09T03:45:44.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.131+0000 7f6201d9b640 1 -- 172.21.15.76:0/254459165 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec030d50 con 0x7f62040c4ff0 2023-11-09T03:45:44.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.131+0000 7f6208e26640 1 -- 172.21.15.76:0/254459165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62040c4ff0 msgr2=0x7f62040cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:44.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.131+0000 7f6208e26640 1 --2- 172.21.15.76:0/254459165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62040c4ff0 0x7f62040cc890 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f62040c59a0 tx=0x7f61ec02b9a0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.131+0000 7f6208e26640 1 -- 172.21.15.76:0/254459165 shutdown_connections 2023-11-09T03:45:44.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.131+0000 7f6208e26640 1 --2- 172.21.15.76:0/254459165 >> v2:172.21.15.113:3300/0 conn(0x7f62040adac0 0x7f62040adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.131+0000 7f6208e26640 1 --2- 172.21.15.76:0/254459165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62040c4ff0 0x7f62040cc890 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.131+0000 7f6208e26640 1 -- 172.21.15.76:0/254459165 >> 172.21.15.76:0/254459165 conn(0x7f62040ce950 msgr2=0x7f62040ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:44.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.131+0000 7f6208e26640 1 -- 172.21.15.76:0/254459165 shutdown_connections 2023-11-09T03:45:44.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.131+0000 7f6208e26640 1 -- 172.21.15.76:0/254459165 wait complete. 2023-11-09T03:45:44.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.133+0000 7f6208e26640 1 Processor -- start 2023-11-09T03:45:44.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.133+0000 7f6208e26640 1 -- start start 2023-11-09T03:45:44.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.133+0000 7f6208e26640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62040adac0 0x7f6204144560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:44.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.133+0000 7f6208e26640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62040c4ff0 0x7f62040c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:44.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.133+0000 7f6208e26640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6204159c00 0x7f62040c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:44.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.133+0000 7f6208e26640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62040c6cf0 con 0x7f62040adac0 2023-11-09T03:45:44.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.133+0000 7f6208e26640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62040c60e0 con 0x7f62040c4ff0 2023-11-09T03:45:44.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.133+0000 7f6208e26640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62040c63c0 con 0x7f6204159c00 2023-11-09T03:45:44.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.133+0000 7f620259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62040adac0 0x7f6204144560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:44.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.133+0000 7f620259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62040adac0 0x7f6204144560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57430/0 (socket says 172.21.15.76:57430) 2023-11-09T03:45:44.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.134+0000 7f620259c640 1 -- 172.21.15.76:0/3815712541 learned_addr learned my addr 172.21.15.76:0/3815712541 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:44.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.134+0000 7f61fbfff640 1 --2- 172.21.15.76:0/3815712541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62040c4ff0 0x7f62040c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:44.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.134+0000 7f6202d9d640 1 --2- 172.21.15.76:0/3815712541 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6204159c00 0x7f62040c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:44.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.134+0000 7f620259c640 1 -- 172.21.15.76:0/3815712541 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6204159c00 msgr2=0x7f62040c5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:44.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.134+0000 7f620259c640 1 --2- 172.21.15.76:0/3815712541 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6204159c00 0x7f62040c5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.134+0000 7f620259c640 1 -- 172.21.15.76:0/3815712541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62040c4ff0 msgr2=0x7f62040c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:44.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.134+0000 7f620259c640 1 --2- 172.21.15.76:0/3815712541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62040c4ff0 0x7f62040c7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.134+0000 7f620259c640 1 -- 172.21.15.76:0/3815712541 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ec0094d0 con 0x7f62040adac0 2023-11-09T03:45:44.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.134+0000 7f620259c640 1 --2- 172.21.15.76:0/3815712541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62040adac0 0x7f6204144560 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f61f400e6e0 tx=0x7f61f400eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:44.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.134+0000 7f61faffd640 1 -- 172.21.15.76:0/3815712541 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f40052d0 con 0x7f62040adac0 2023-11-09T03:45:44.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.134+0000 7f61faffd640 1 -- 172.21.15.76:0/3815712541 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f4005430 con 0x7f62040adac0 2023-11-09T03:45:44.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.134+0000 7f61faffd640 1 -- 172.21.15.76:0/3815712541 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f40018e0 con 0x7f62040adac0 2023-11-09T03:45:44.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.134+0000 7f6208e26640 1 -- 172.21.15.76:0/3815712541 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62040c66a0 con 0x7f62040adac0 2023-11-09T03:45:44.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.134+0000 7f6208e26640 1 -- 172.21.15.76:0/3815712541 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62041572f0 con 0x7f62040adac0 2023-11-09T03:45:44.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.135+0000 7f6208e26640 1 -- 172.21.15.76:0/3815712541 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6204074290 con 0x7f62040adac0 2023-11-09T03:45:44.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.135+0000 7f61faffd640 1 -- 172.21.15.76:0/3815712541 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f61f4001a40 con 0x7f62040adac0 2023-11-09T03:45:44.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.136+0000 7f61faffd640 1 --2- 172.21.15.76:0/3815712541 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f61e406ae70 0x7f61e406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:44.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.136+0000 7f61faffd640 1 -- 172.21.15.76:0/3815712541 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f61f408bab0 con 0x7f62040adac0 2023-11-09T03:45:44.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.136+0000 7f61fbfff640 1 --2- 172.21.15.76:0/3815712541 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f61e406ae70 0x7f61e406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:44.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.137+0000 7f61fbfff640 1 --2- 172.21.15.76:0/3815712541 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f61e406ae70 0x7f61e406d330 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f61ec036000 tx=0x7f61ec008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:44.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.140+0000 7f61faffd640 1 -- 172.21.15.76:0/3815712541 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f61f4056890 con 0x7f62040adac0 2023-11-09T03:45:44.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.305+0000 7f6208e26640 1 -- 172.21.15.76:0/3815712541 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6204152b40 con 0x7f62040adac0 2023-11-09T03:45:44.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.306+0000 7f61faffd640 1 -- 172.21.15.76:0/3815712541 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f61f401f070 con 0x7f62040adac0 2023-11-09T03:45:44.306 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:45:44.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.307+0000 7f6208e26640 1 -- 172.21.15.76:0/3815712541 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f61e406ae70 msgr2=0x7f61e406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:44.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.307+0000 7f6208e26640 1 --2- 172.21.15.76:0/3815712541 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f61e406ae70 0x7f61e406d330 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f61ec036000 tx=0x7f61ec008030 comp rx=0 tx=0).stop 2023-11-09T03:45:44.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.307+0000 7f6208e26640 1 -- 172.21.15.76:0/3815712541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62040adac0 msgr2=0x7f6204144560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:44.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.307+0000 7f6208e26640 1 --2- 172.21.15.76:0/3815712541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62040adac0 0x7f6204144560 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f61f400e6e0 tx=0x7f61f400eba0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.307+0000 7f6208e26640 1 -- 172.21.15.76:0/3815712541 shutdown_connections 2023-11-09T03:45:44.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.307+0000 7f6208e26640 1 --2- 172.21.15.76:0/3815712541 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f61e406ae70 0x7f61e406d330 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.307+0000 7f6208e26640 1 --2- 172.21.15.76:0/3815712541 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6204159c00 0x7f62040c5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.307+0000 7f6208e26640 1 --2- 172.21.15.76:0/3815712541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62040c4ff0 0x7f62040c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.307+0000 7f6208e26640 1 --2- 172.21.15.76:0/3815712541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62040adac0 0x7f6204144560 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.307+0000 7f6208e26640 1 -- 172.21.15.76:0/3815712541 >> 172.21.15.76:0/3815712541 conn(0x7f62040ce950 msgr2=0x7f62040709f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:44.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.307+0000 7f6208e26640 1 -- 172.21.15.76:0/3815712541 shutdown_connections 2023-11-09T03:45:44.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.307+0000 7f6208e26640 1 -- 172.21.15.76:0/3815712541 wait complete. 2023-11-09T03:45:44.317 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:45:44.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.421+0000 7f0a0b7e7640 1 Processor -- start 2023-11-09T03:45:44.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.421+0000 7f0a0b7e7640 1 -- start start 2023-11-09T03:45:44.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.422+0000 7f0a0b7e7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0a04160430 0x7f0a040b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:44.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.422+0000 7f0a0b7e7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a040b2fc0 con 0x7f0a0406c3a0 2023-11-09T03:45:44.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.422+0000 7f0a0b7e7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a0406c7d0 con 0x7f0a040b2b60 2023-11-09T03:45:44.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.422+0000 7f0a0b7e7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a040cb580 con 0x7f0a04160430 2023-11-09T03:45:44.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.422+0000 7f0a09583640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0a04160430 0x7f0a040b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:44.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.422+0000 7f0a09d84640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0a0406c3a0 0x7f0a040cbcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:56658/0 (socket says 172.21.15.76:56658) 2023-11-09T03:45:44.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.422+0000 7f0a09d84640 1 -- 172.21.15.76:0/515587505 learned_addr learned my addr 172.21.15.76:0/515587505 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:44.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.423+0000 7f0a09583640 1 -- 172.21.15.76:0/515587505 >> v1:172.21.15.113:6789/0 conn(0x7f0a040b2b60 legacy=0x7f0a0406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:44.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.423+0000 7f0a09583640 1 -- 172.21.15.76:0/515587505 >> v1:172.21.15.76:6789/0 conn(0x7f0a0406c3a0 legacy=0x7f0a040cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:44.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.423+0000 7f09fbfff640 1 -- 172.21.15.76:0/515587505 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2160097495 0 0) 0x7f0a040b2fc0 con 0x7f0a0406c3a0 2023-11-09T03:45:44.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.423+0000 7f0a09583640 1 -- 172.21.15.76:0/515587505 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0406ca40 con 0x7f0a04160430 2023-11-09T03:45:44.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.423+0000 7f0a09583640 1 --2- 172.21.15.76:0/515587505 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a04160430 0x7f0a040b2620 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0a00009d90 tx=0x7f0a0002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f840700d3bfb1d2f server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:44.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.424+0000 7f09fbfff640 1 -- 172.21.15.76:0/515587505 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0002cd00 con 0x7f0a04160430 2023-11-09T03:45:44.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.424+0000 7f09fbfff640 1 -- 172.21.15.76:0/515587505 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0002ce60 con 0x7f0a04160430 2023-11-09T03:45:44.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.424+0000 7f09fbfff640 1 -- 172.21.15.76:0/515587505 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a00034660 con 0x7f0a04160430 2023-11-09T03:45:44.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.424+0000 7f0a0b7e7640 1 -- 172.21.15.76:0/515587505 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a04160430 msgr2=0x7f0a040b2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:44.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.424+0000 7f0a0b7e7640 1 --2- 172.21.15.76:0/515587505 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a04160430 0x7f0a040b2620 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0a00009d90 tx=0x7f0a0002bf50 comp rx=0 tx=0).stop 2023-11-09T03:45:44.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.424+0000 7f0a0b7e7640 1 -- 172.21.15.76:0/515587505 shutdown_connections 2023-11-09T03:45:44.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.424+0000 7f0a0b7e7640 1 --2- 172.21.15.76:0/515587505 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a04160430 0x7f0a040b2620 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.424+0000 7f0a0b7e7640 1 -- 172.21.15.76:0/515587505 >> 172.21.15.76:0/515587505 conn(0x7f0a0406ea20 msgr2=0x7f0a0406ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:44.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.424+0000 7f0a0b7e7640 1 -- 172.21.15.76:0/515587505 shutdown_connections 2023-11-09T03:45:44.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.424+0000 7f0a0b7e7640 1 -- 172.21.15.76:0/515587505 wait complete. 2023-11-09T03:45:44.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.425+0000 7f0a0b7e7640 1 Processor -- start 2023-11-09T03:45:44.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.425+0000 7f0a0b7e7640 1 -- start start 2023-11-09T03:45:44.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.426+0000 7f0a0b7e7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a0406c3a0 0x7f0a04155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:44.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.426+0000 7f0a0b7e7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a040b2b60 0x7f0a040b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:44.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.426+0000 7f0a0b7e7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a04160430 0x7f0a040b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:44.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.426+0000 7f0a0b7e7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0415c360 con 0x7f0a04160430 2023-11-09T03:45:44.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.426+0000 7f0a0b7e7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0415c010 con 0x7f0a0406c3a0 2023-11-09T03:45:44.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.426+0000 7f0a0b7e7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a040b5fd0 con 0x7f0a040b2b60 2023-11-09T03:45:44.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.426+0000 7f0a09d84640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a04160430 0x7f0a040b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:44.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.426+0000 7f0a09d84640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a04160430 0x7f0a040b0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57442/0 (socket says 172.21.15.76:57442) 2023-11-09T03:45:44.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.426+0000 7f0a09d84640 1 -- 172.21.15.76:0/2174089087 learned_addr learned my addr 172.21.15.76:0/2174089087 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:44.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.426+0000 7f0a09583640 1 --2- 172.21.15.76:0/2174089087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a0406c3a0 0x7f0a04155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:44.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.426+0000 7f0a08d82640 1 --2- 172.21.15.76:0/2174089087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a040b2b60 0x7f0a040b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:44.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.426+0000 7f0a09d84640 1 -- 172.21.15.76:0/2174089087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a040b2b60 msgr2=0x7f0a040b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:44.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.427+0000 7f0a09d84640 1 --2- 172.21.15.76:0/2174089087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a040b2b60 0x7f0a040b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.427+0000 7f0a09d84640 1 -- 172.21.15.76:0/2174089087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a0406c3a0 msgr2=0x7f0a04155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:44.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.427+0000 7f0a09d84640 1 --2- 172.21.15.76:0/2174089087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a0406c3a0 0x7f0a04155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.427+0000 7f0a09d84640 1 -- 172.21.15.76:0/2174089087 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a00009a70 con 0x7f0a04160430 2023-11-09T03:45:44.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.427+0000 7f0a08d82640 1 --2- 172.21.15.76:0/2174089087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a040b2b60 0x7f0a040b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:44.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.427+0000 7f0a09d84640 1 --2- 172.21.15.76:0/2174089087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a04160430 0x7f0a040b0b60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f09fc001eb0 tx=0x7f09fc00c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:44.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.427+0000 7f09f9ffb640 1 -- 172.21.15.76:0/2174089087 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc0074e0 con 0x7f0a04160430 2023-11-09T03:45:44.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.427+0000 7f09f9ffb640 1 -- 172.21.15.76:0/2174089087 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09fc007640 con 0x7f0a04160430 2023-11-09T03:45:44.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.427+0000 7f0a0b7e7640 1 -- 172.21.15.76:0/2174089087 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a040b62b0 con 0x7f0a04160430 2023-11-09T03:45:44.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.427+0000 7f09f9ffb640 1 -- 172.21.15.76:0/2174089087 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc004a00 con 0x7f0a04160430 2023-11-09T03:45:44.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.427+0000 7f0a0b7e7640 1 -- 172.21.15.76:0/2174089087 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a041558c0 con 0x7f0a04160430 2023-11-09T03:45:44.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.428+0000 7f0a0b7e7640 1 -- 172.21.15.76:0/2174089087 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a0406d400 con 0x7f0a04160430 2023-11-09T03:45:44.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.428+0000 7f09f9ffb640 1 -- 172.21.15.76:0/2174089087 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f09fc004b60 con 0x7f0a04160430 2023-11-09T03:45:44.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.429+0000 7f09f9ffb640 1 --2- 172.21.15.76:0/2174089087 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f09e806aea0 0x7f09e806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:44.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.429+0000 7f09f9ffb640 1 -- 172.21.15.76:0/2174089087 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f09fc0895f0 con 0x7f0a04160430 2023-11-09T03:45:44.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.429+0000 7f0a09583640 1 --2- 172.21.15.76:0/2174089087 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f09e806aea0 0x7f09e806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:44.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.429+0000 7f0a09583640 1 --2- 172.21.15.76:0/2174089087 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f09e806aea0 0x7f09e806d360 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0a00036000 tx=0x7f0a00030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:44.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.433+0000 7f09f9ffb640 1 -- 172.21.15.76:0/2174089087 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f09fc057e40 con 0x7f0a04160430 2023-11-09T03:45:44.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.600+0000 7f0a0b7e7640 1 -- 172.21.15.76:0/2174089087 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0a0415c4a0 con 0x7f0a04160430 2023-11-09T03:45:44.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.600+0000 7f09f9ffb640 1 -- 172.21.15.76:0/2174089087 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0a0415c4a0 con 0x7f0a04160430 2023-11-09T03:45:44.600 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:45:44.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.603+0000 7f0a0b7e7640 1 -- 172.21.15.76:0/2174089087 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f09e806aea0 msgr2=0x7f09e806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:44.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.603+0000 7f0a0b7e7640 1 --2- 172.21.15.76:0/2174089087 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f09e806aea0 0x7f09e806d360 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0a00036000 tx=0x7f0a00030000 comp rx=0 tx=0).stop 2023-11-09T03:45:44.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.603+0000 7f0a0b7e7640 1 -- 172.21.15.76:0/2174089087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a04160430 msgr2=0x7f0a040b0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:44.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.603+0000 7f0a0b7e7640 1 --2- 172.21.15.76:0/2174089087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a04160430 0x7f0a040b0b60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f09fc001eb0 tx=0x7f09fc00c870 comp rx=0 tx=0).stop 2023-11-09T03:45:44.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.603+0000 7f0a0b7e7640 1 -- 172.21.15.76:0/2174089087 shutdown_connections 2023-11-09T03:45:44.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.603+0000 7f0a0b7e7640 1 --2- 172.21.15.76:0/2174089087 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f09e806aea0 0x7f09e806d360 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.603+0000 7f0a0b7e7640 1 --2- 172.21.15.76:0/2174089087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a04160430 0x7f0a040b0b60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.603+0000 7f0a0b7e7640 1 --2- 172.21.15.76:0/2174089087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a040b2b60 0x7f0a040b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.603+0000 7f0a0b7e7640 1 --2- 172.21.15.76:0/2174089087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a0406c3a0 0x7f0a04155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.603+0000 7f0a0b7e7640 1 -- 172.21.15.76:0/2174089087 >> 172.21.15.76:0/2174089087 conn(0x7f0a0406ea20 msgr2=0x7f0a040b1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:44.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.603+0000 7f0a0b7e7640 1 -- 172.21.15.76:0/2174089087 shutdown_connections 2023-11-09T03:45:44.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.603+0000 7f0a0b7e7640 1 -- 172.21.15.76:0/2174089087 wait complete. 2023-11-09T03:45:44.613 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:45:44.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.715+0000 7f7d3af5e640 1 Processor -- start 2023-11-09T03:45:44.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.715+0000 7f7d3af5e640 1 -- start start 2023-11-09T03:45:44.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.716+0000 7f7d3af5e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7d340cd6d0 0x7f7d34161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:44.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.716+0000 7f7d3af5e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7d340b4880 0x7f7d340b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:44.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.716+0000 7f7d3af5e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d340cde60 con 0x7f7d34153e60 2023-11-09T03:45:44.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.716+0000 7f7d3af5e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d34160f90 con 0x7f7d340cd6d0 2023-11-09T03:45:44.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.716+0000 7f7d3af5e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d34161960 con 0x7f7d340b4880 2023-11-09T03:45:44.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.716+0000 7f7d33fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7d340cd6d0 0x7f7d34161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:44.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.716+0000 7f7d33fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7d340cd6d0 0x7f7d34161420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57444/0 (socket says 172.21.15.76:57444) 2023-11-09T03:45:44.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.716+0000 7f7d33fff640 1 -- 172.21.15.76:0/295387041 learned_addr learned my addr 172.21.15.76:0/295387041 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:44.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.717+0000 7f7d394fb640 1 --2- 172.21.15.76:0/295387041 >> v2:172.21.15.113:3300/0 conn(0x7f7d340b4880 0x7f7d340b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:44.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.717+0000 7f7d33fff640 1 -- 172.21.15.76:0/295387041 >> v2:172.21.15.113:3300/0 conn(0x7f7d340b4880 msgr2=0x7f7d340b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:44.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.717+0000 7f7d33fff640 1 --2- 172.21.15.76:0/295387041 >> v2:172.21.15.113:3300/0 conn(0x7f7d340b4880 0x7f7d340b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.717+0000 7f7d33fff640 1 -- 172.21.15.76:0/295387041 >> v1:172.21.15.113:6789/0 conn(0x7f7d34153e60 legacy=0x7f7d3415dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:44.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.717+0000 7f7d33fff640 1 -- 172.21.15.76:0/295387041 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d340b5220 con 0x7f7d340cd6d0 2023-11-09T03:45:44.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.717+0000 7f7d33fff640 1 --2- 172.21.15.76:0/295387041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d340cd6d0 0x7f7d34161420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7d340b8f40 tx=0x7f7d2802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae61c7de7190341 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:44.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.717+0000 7f7d337fe640 1 -- 172.21.15.76:0/295387041 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d28031870 con 0x7f7d340cd6d0 2023-11-09T03:45:44.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.718+0000 7f7d337fe640 1 -- 172.21.15.76:0/295387041 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d280319d0 con 0x7f7d340cd6d0 2023-11-09T03:45:44.720 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.718+0000 7f7d337fe640 1 -- 172.21.15.76:0/295387041 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d28031d60 con 0x7f7d340cd6d0 2023-11-09T03:45:44.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.718+0000 7f7d3af5e640 1 -- 172.21.15.76:0/295387041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d340cd6d0 msgr2=0x7f7d34161420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:44.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.718+0000 7f7d3af5e640 1 --2- 172.21.15.76:0/295387041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d340cd6d0 0x7f7d34161420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7d340b8f40 tx=0x7f7d2802b910 comp rx=0 tx=0).stop 2023-11-09T03:45:44.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.718+0000 7f7d3af5e640 1 -- 172.21.15.76:0/295387041 shutdown_connections 2023-11-09T03:45:44.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.718+0000 7f7d3af5e640 1 --2- 172.21.15.76:0/295387041 >> v2:172.21.15.113:3300/0 conn(0x7f7d340b4880 0x7f7d340b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.718+0000 7f7d3af5e640 1 --2- 172.21.15.76:0/295387041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d340cd6d0 0x7f7d34161420 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.718+0000 7f7d3af5e640 1 -- 172.21.15.76:0/295387041 >> 172.21.15.76:0/295387041 conn(0x7f7d340cf880 msgr2=0x7f7d340cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:44.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.718+0000 7f7d3af5e640 1 -- 172.21.15.76:0/295387041 shutdown_connections 2023-11-09T03:45:44.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.718+0000 7f7d3af5e640 1 -- 172.21.15.76:0/295387041 wait complete. 2023-11-09T03:45:44.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.719+0000 7f7d3af5e640 1 Processor -- start 2023-11-09T03:45:44.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.719+0000 7f7d3af5e640 1 -- start start 2023-11-09T03:45:44.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.720+0000 7f7d3af5e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d340b4880 0x7f7d34152e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:44.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.720+0000 7f7d3af5e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d340cd6d0 0x7f7d34156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:44.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.720+0000 7f7d3af5e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d34153e60 0x7f7d340c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:44.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.720+0000 7f7d3af5e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d34157e80 con 0x7f7d34153e60 2023-11-09T03:45:44.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.720+0000 7f7d3af5e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d34157120 con 0x7f7d340b4880 2023-11-09T03:45:44.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.720+0000 7f7d3af5e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d34158120 con 0x7f7d340cd6d0 2023-11-09T03:45:44.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.720+0000 7f7d394fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d34153e60 0x7f7d340c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:44.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.720+0000 7f7d394fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d34153e60 0x7f7d340c7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57458/0 (socket says 172.21.15.76:57458) 2023-11-09T03:45:44.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.720+0000 7f7d394fb640 1 -- 172.21.15.76:0/3288736289 learned_addr learned my addr 172.21.15.76:0/3288736289 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:44.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.720+0000 7f7d33fff640 1 --2- 172.21.15.76:0/3288736289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d340cd6d0 0x7f7d34156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:44.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.720+0000 7f7d38cfa640 1 --2- 172.21.15.76:0/3288736289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d340b4880 0x7f7d34152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:44.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.720+0000 7f7d394fb640 1 -- 172.21.15.76:0/3288736289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d340cd6d0 msgr2=0x7f7d34156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:44.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.720+0000 7f7d394fb640 1 --2- 172.21.15.76:0/3288736289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d340cd6d0 0x7f7d34156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.720+0000 7f7d394fb640 1 -- 172.21.15.76:0/3288736289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d340b4880 msgr2=0x7f7d34152e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:44.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.720+0000 7f7d394fb640 1 --2- 172.21.15.76:0/3288736289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d340b4880 0x7f7d34152e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.720+0000 7f7d394fb640 1 -- 172.21.15.76:0/3288736289 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d280094d0 con 0x7f7d34153e60 2023-11-09T03:45:44.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.721+0000 7f7d394fb640 1 --2- 172.21.15.76:0/3288736289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d34153e60 0x7f7d340c7950 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c007e40 tx=0x7f7d2c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:44.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.721+0000 7f7d317fa640 1 -- 172.21.15.76:0/3288736289 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c014620 con 0x7f7d34153e60 2023-11-09T03:45:44.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.721+0000 7f7d317fa640 1 -- 172.21.15.76:0/3288736289 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2c0147a0 con 0x7f7d34153e60 2023-11-09T03:45:44.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.721+0000 7f7d3af5e640 1 -- 172.21.15.76:0/3288736289 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d34158400 con 0x7f7d34153e60 2023-11-09T03:45:44.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.721+0000 7f7d317fa640 1 -- 172.21.15.76:0/3288736289 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c014b50 con 0x7f7d34153e60 2023-11-09T03:45:44.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.721+0000 7f7d3af5e640 1 -- 172.21.15.76:0/3288736289 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d34156d80 con 0x7f7d34153e60 2023-11-09T03:45:44.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.722+0000 7f7d3af5e640 1 -- 172.21.15.76:0/3288736289 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d34158920 con 0x7f7d34153e60 2023-11-09T03:45:44.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.722+0000 7f7d317fa640 1 -- 172.21.15.76:0/3288736289 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7d2c014cb0 con 0x7f7d34153e60 2023-11-09T03:45:44.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.723+0000 7f7d317fa640 1 --2- 172.21.15.76:0/3288736289 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7d2006aea0 0x7f7d2006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:44.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.723+0000 7f7d317fa640 1 -- 172.21.15.76:0/3288736289 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7d2c08a1c0 con 0x7f7d34153e60 2023-11-09T03:45:44.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.723+0000 7f7d38cfa640 1 --2- 172.21.15.76:0/3288736289 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7d2006aea0 0x7f7d2006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:44.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.723+0000 7f7d38cfa640 1 --2- 172.21.15.76:0/3288736289 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7d2006aea0 0x7f7d2006d360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7d24006fd0 tx=0x7f7d24008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:44.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.727+0000 7f7d317fa640 1 -- 172.21.15.76:0/3288736289 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7d34158920 con 0x7f7d34153e60 2023-11-09T03:45:44.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.892+0000 7f7d3af5e640 1 -- 172.21.15.76:0/3288736289 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7d34158920 con 0x7f7d34153e60 2023-11-09T03:45:44.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.892+0000 7f7d317fa640 1 -- 172.21.15.76:0/3288736289 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7d34158920 con 0x7f7d34153e60 2023-11-09T03:45:44.892 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:45:44.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.894+0000 7f7d3af5e640 1 -- 172.21.15.76:0/3288736289 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7d2006aea0 msgr2=0x7f7d2006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:44.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.894+0000 7f7d3af5e640 1 --2- 172.21.15.76:0/3288736289 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7d2006aea0 0x7f7d2006d360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7d24006fd0 tx=0x7f7d24008040 comp rx=0 tx=0).stop 2023-11-09T03:45:44.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.894+0000 7f7d3af5e640 1 -- 172.21.15.76:0/3288736289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d34153e60 msgr2=0x7f7d340c7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:44.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.894+0000 7f7d3af5e640 1 --2- 172.21.15.76:0/3288736289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d34153e60 0x7f7d340c7950 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c007e40 tx=0x7f7d2c00e4b0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.894+0000 7f7d3af5e640 1 -- 172.21.15.76:0/3288736289 shutdown_connections 2023-11-09T03:45:44.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.894+0000 7f7d3af5e640 1 --2- 172.21.15.76:0/3288736289 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7d2006aea0 0x7f7d2006d360 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.894+0000 7f7d3af5e640 1 --2- 172.21.15.76:0/3288736289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d34153e60 0x7f7d340c7950 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.894+0000 7f7d3af5e640 1 --2- 172.21.15.76:0/3288736289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d340cd6d0 0x7f7d34156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.894+0000 7f7d3af5e640 1 --2- 172.21.15.76:0/3288736289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d340b4880 0x7f7d34152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:44.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.894+0000 7f7d3af5e640 1 -- 172.21.15.76:0/3288736289 >> 172.21.15.76:0/3288736289 conn(0x7f7d340cf880 msgr2=0x7f7d340d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:44.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.894+0000 7f7d3af5e640 1 -- 172.21.15.76:0/3288736289 shutdown_connections 2023-11-09T03:45:44.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:44.894+0000 7f7d3af5e640 1 -- 172.21.15.76:0/3288736289 wait complete. 2023-11-09T03:45:44.904 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-09T03:45:45.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.008+0000 7f9c2bfbb640 1 Processor -- start 2023-11-09T03:45:45.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.008+0000 7f9c2bfbb640 1 -- start start 2023-11-09T03:45:45.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.009+0000 7f9c2bfbb640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9c240cc300 0x7f9c240b1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:45.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.009+0000 7f9c2bfbb640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c240caa20 con 0x7f9c240ca0a0 2023-11-09T03:45:45.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.009+0000 7f9c2bfbb640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c240cb230 con 0x7f9c2415e3f0 2023-11-09T03:45:45.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.009+0000 7f9c2bfbb640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c240b3300 con 0x7f9c240cc300 2023-11-09T03:45:45.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.009+0000 7f9c2a558640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9c240cc300 0x7f9c240b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:45.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.009+0000 7f9c29556640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f9c240ca0a0 0x7f9c240cc920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:56670/0 (socket says 172.21.15.76:56670) 2023-11-09T03:45:45.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.009+0000 7f9c2a558640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9c240cc300 0x7f9c240b1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57472/0 (socket says 172.21.15.76:57472) 2023-11-09T03:45:45.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.009+0000 7f9c29556640 1 -- 172.21.15.76:0/2745419566 learned_addr learned my addr 172.21.15.76:0/2745419566 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:45.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.009+0000 7f9c2a558640 1 -- 172.21.15.76:0/2745419566 >> v1:172.21.15.113:6789/0 conn(0x7f9c2415e3f0 legacy=0x7f9c240b2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:45.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.009+0000 7f9c2a558640 1 -- 172.21.15.76:0/2745419566 >> v1:172.21.15.76:6789/0 conn(0x7f9c240ca0a0 legacy=0x7f9c240cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:45.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.009+0000 7f9c2a558640 1 -- 172.21.15.76:0/2745419566 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c240ca530 con 0x7f9c240cc300 2023-11-09T03:45:45.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.009+0000 7f9c28d55640 1 -- 172.21.15.76:0/2745419566 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199933845 0 0) 0x7f9c240caa20 con 0x7f9c240ca0a0 2023-11-09T03:45:45.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.010+0000 7f9c2a558640 1 --2- 172.21.15.76:0/2745419566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c240cc300 0x7f9c240b1890 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009d90 tx=0x7f9c1c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c75f985b8cb2896 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:45.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.010+0000 7f9c28d55640 1 -- 172.21.15.76:0/2745419566 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c02cc60 con 0x7f9c240cc300 2023-11-09T03:45:45.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.010+0000 7f9c28d55640 1 -- 172.21.15.76:0/2745419566 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1c02cdc0 con 0x7f9c240cc300 2023-11-09T03:45:45.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.010+0000 7f9c28d55640 1 -- 172.21.15.76:0/2745419566 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c034660 con 0x7f9c240cc300 2023-11-09T03:45:45.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.011+0000 7f9c2bfbb640 1 -- 172.21.15.76:0/2745419566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c240cc300 msgr2=0x7f9c240b1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:45.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.011+0000 7f9c2bfbb640 1 --2- 172.21.15.76:0/2745419566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c240cc300 0x7f9c240b1890 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009d90 tx=0x7f9c1c02beb0 comp rx=0 tx=0).stop 2023-11-09T03:45:45.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.011+0000 7f9c2bfbb640 1 -- 172.21.15.76:0/2745419566 shutdown_connections 2023-11-09T03:45:45.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.011+0000 7f9c2bfbb640 1 --2- 172.21.15.76:0/2745419566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c240cc300 0x7f9c240b1890 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:45.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.011+0000 7f9c2bfbb640 1 -- 172.21.15.76:0/2745419566 >> 172.21.15.76:0/2745419566 conn(0x7f9c24071820 msgr2=0x7f9c24071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:45.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.011+0000 7f9c2bfbb640 1 -- 172.21.15.76:0/2745419566 shutdown_connections 2023-11-09T03:45:45.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.011+0000 7f9c2bfbb640 1 -- 172.21.15.76:0/2745419566 wait complete. 2023-11-09T03:45:45.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.012+0000 7f9c2bfbb640 1 Processor -- start 2023-11-09T03:45:45.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.012+0000 7f9c2bfbb640 1 -- start start 2023-11-09T03:45:45.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.013+0000 7f9c2bfbb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c240cc300 0x7f9c2414ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:45.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.013+0000 7f9c2bfbb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c2415e3f0 0x7f9c240af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:45.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.013+0000 7f9c2bfbb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c2415d100 0x7f9c2415a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:45.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.013+0000 7f9c2bfbb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2415a110 con 0x7f9c2415d100 2023-11-09T03:45:45.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.013+0000 7f9c2bfbb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2415d590 con 0x7f9c240cc300 2023-11-09T03:45:45.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.013+0000 7f9c2bfbb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c24159b50 con 0x7f9c2415e3f0 2023-11-09T03:45:45.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.013+0000 7f9c2a558640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c2415d100 0x7f9c2415a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:45.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.013+0000 7f9c2a558640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c2415d100 0x7f9c2415a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57484/0 (socket says 172.21.15.76:57484) 2023-11-09T03:45:45.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.013+0000 7f9c2a558640 1 -- 172.21.15.76:0/2468540578 learned_addr learned my addr 172.21.15.76:0/2468540578 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:45.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.013+0000 7f9c29d57640 1 --2- 172.21.15.76:0/2468540578 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c240cc300 0x7f9c2414ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:45.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.013+0000 7f9c29556640 1 --2- 172.21.15.76:0/2468540578 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c2415e3f0 0x7f9c240af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:45.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.013+0000 7f9c2a558640 1 -- 172.21.15.76:0/2468540578 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c2415e3f0 msgr2=0x7f9c240af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:45.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.013+0000 7f9c2a558640 1 --2- 172.21.15.76:0/2468540578 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c2415e3f0 0x7f9c240af4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:45.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.013+0000 7f9c2a558640 1 -- 172.21.15.76:0/2468540578 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c240cc300 msgr2=0x7f9c2414ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:45.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.013+0000 7f9c2a558640 1 --2- 172.21.15.76:0/2468540578 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c240cc300 0x7f9c2414ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:45.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.013+0000 7f9c2a558640 1 -- 172.21.15.76:0/2468540578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c1c009a70 con 0x7f9c2415d100 2023-11-09T03:45:45.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.013+0000 7f9c29556640 1 --2- 172.21.15.76:0/2468540578 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c2415e3f0 0x7f9c240af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:45.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.013+0000 7f9c29d57640 1 --2- 172.21.15.76:0/2468540578 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c240cc300 0x7f9c2414ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:45.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.013+0000 7f9c2a558640 1 --2- 172.21.15.76:0/2468540578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c2415d100 0x7f9c2415a720 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009d90 tx=0x7f9c1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:45.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.013+0000 7f9c1a7fc640 1 -- 172.21.15.76:0/2468540578 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c03e610 con 0x7f9c2415d100 2023-11-09T03:45:45.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.013+0000 7f9c1a7fc640 1 -- 172.21.15.76:0/2468540578 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1c03e790 con 0x7f9c2415d100 2023-11-09T03:45:45.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.013+0000 7f9c1a7fc640 1 -- 172.21.15.76:0/2468540578 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c03eb40 con 0x7f9c2415d100 2023-11-09T03:45:45.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.013+0000 7f9c2bfbb640 1 -- 172.21.15.76:0/2468540578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c24159e30 con 0x7f9c2415d100 2023-11-09T03:45:45.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.014+0000 7f9c2bfbb640 1 -- 172.21.15.76:0/2468540578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c240af9f0 con 0x7f9c2415d100 2023-11-09T03:45:45.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.015+0000 7f9c2bfbb640 1 -- 172.21.15.76:0/2468540578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c241599b0 con 0x7f9c2415d100 2023-11-09T03:45:45.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.015+0000 7f9c1a7fc640 1 -- 172.21.15.76:0/2468540578 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9c1c03b050 con 0x7f9c2415d100 2023-11-09T03:45:45.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.015+0000 7f9c1a7fc640 1 --2- 172.21.15.76:0/2468540578 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c0806aea0 0x7f9c0806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:45.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.015+0000 7f9c1a7fc640 1 -- 172.21.15.76:0/2468540578 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9c1c0abfa0 con 0x7f9c2415d100 2023-11-09T03:45:45.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.015+0000 7f9c29d57640 1 --2- 172.21.15.76:0/2468540578 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c0806aea0 0x7f9c0806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:45.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.016+0000 7f9c29d57640 1 --2- 172.21.15.76:0/2468540578 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c0806aea0 0x7f9c0806d360 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9c20006fd0 tx=0x7f9c20008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:45.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.019+0000 7f9c1a7fc640 1 -- 172.21.15.76:0/2468540578 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9c1c0deca0 con 0x7f9c2415d100 2023-11-09T03:45:45.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.186+0000 7f9c2bfbb640 1 -- 172.21.15.76:0/2468540578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9c240c75c0 con 0x7f9c2415d100 2023-11-09T03:45:45.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.187+0000 7f9c1a7fc640 1 -- 172.21.15.76:0/2468540578 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9c240c75c0 con 0x7f9c2415d100 2023-11-09T03:45:45.186 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:45:45.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.188+0000 7f9c2bfbb640 1 -- 172.21.15.76:0/2468540578 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c0806aea0 msgr2=0x7f9c0806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:45.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.188+0000 7f9c2bfbb640 1 --2- 172.21.15.76:0/2468540578 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c0806aea0 0x7f9c0806d360 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9c20006fd0 tx=0x7f9c20008040 comp rx=0 tx=0).stop 2023-11-09T03:45:45.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.188+0000 7f9c2bfbb640 1 -- 172.21.15.76:0/2468540578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c2415d100 msgr2=0x7f9c2415a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:45.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.188+0000 7f9c2bfbb640 1 --2- 172.21.15.76:0/2468540578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c2415d100 0x7f9c2415a720 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009d90 tx=0x7f9c1c004a40 comp rx=0 tx=0).stop 2023-11-09T03:45:45.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.188+0000 7f9c2bfbb640 1 -- 172.21.15.76:0/2468540578 shutdown_connections 2023-11-09T03:45:45.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.188+0000 7f9c2bfbb640 1 --2- 172.21.15.76:0/2468540578 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c0806aea0 0x7f9c0806d360 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:45.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.188+0000 7f9c2bfbb640 1 --2- 172.21.15.76:0/2468540578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c2415d100 0x7f9c2415a720 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:45.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.188+0000 7f9c2bfbb640 1 --2- 172.21.15.76:0/2468540578 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c2415e3f0 0x7f9c240af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:45.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.188+0000 7f9c2bfbb640 1 --2- 172.21.15.76:0/2468540578 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c240cc300 0x7f9c2414ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:45.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.188+0000 7f9c2bfbb640 1 -- 172.21.15.76:0/2468540578 >> 172.21.15.76:0/2468540578 conn(0x7f9c24071820 msgr2=0x7f9c24150d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:45.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.189+0000 7f9c2bfbb640 1 -- 172.21.15.76:0/2468540578 shutdown_connections 2023-11-09T03:45:45.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.189+0000 7f9c2bfbb640 1 -- 172.21.15.76:0/2468540578 wait complete. 2023-11-09T03:45:45.198 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-09T03:45:45.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.300+0000 7fd05fce5640 1 Processor -- start 2023-11-09T03:45:45.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.301+0000 7fd05fce5640 1 -- start start 2023-11-09T03:45:45.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.301+0000 7fd05fce5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd058153e60 0x7fd05815dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:45.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.301+0000 7fd05fce5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd0580cd6d0 0x7fd058161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:45.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.301+0000 7fd05fce5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0580cde60 con 0x7fd0580b4880 2023-11-09T03:45:45.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.301+0000 7fd05fce5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd058160f90 con 0x7fd0580cd6d0 2023-11-09T03:45:45.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.301+0000 7fd05fce5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd05815e400 con 0x7fd058153e60 2023-11-09T03:45:45.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.302+0000 7fd05d280640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd0580cd6d0 0x7fd058161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:45.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.302+0000 7fd05d280640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd0580cd6d0 0x7fd058161420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57500/0 (socket says 172.21.15.76:57500) 2023-11-09T03:45:45.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.302+0000 7fd05d280640 1 -- 172.21.15.76:0/2597434710 learned_addr learned my addr 172.21.15.76:0/2597434710 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:45.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.302+0000 7fd05da81640 1 --2- 172.21.15.76:0/2597434710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd058153e60 0x7fd05815dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:45.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.302+0000 7fd05d280640 1 -- 172.21.15.76:0/2597434710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd058153e60 msgr2=0x7fd05815dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:45.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.302+0000 7fd05d280640 1 --2- 172.21.15.76:0/2597434710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd058153e60 0x7fd05815dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:45.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.302+0000 7fd05d280640 1 -- 172.21.15.76:0/2597434710 >> v1:172.21.15.113:6789/0 conn(0x7fd0580b4880 legacy=0x7fd05806b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:45.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.302+0000 7fd05d280640 1 -- 172.21.15.76:0/2597434710 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd058161960 con 0x7fd0580cd6d0 2023-11-09T03:45:45.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.303+0000 7fd05d280640 1 --2- 172.21.15.76:0/2597434710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0580cd6d0 0x7fd058161420 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd054009870 tx=0x7fd05402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8e7e7149e87995 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:45.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.303+0000 7fd05ca7f640 1 -- 172.21.15.76:0/2597434710 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd054031640 con 0x7fd0580cd6d0 2023-11-09T03:45:45.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.303+0000 7fd05ca7f640 1 -- 172.21.15.76:0/2597434710 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0540317a0 con 0x7fd0580cd6d0 2023-11-09T03:45:45.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.303+0000 7fd05ca7f640 1 -- 172.21.15.76:0/2597434710 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd054031b30 con 0x7fd0580cd6d0 2023-11-09T03:45:45.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.303+0000 7fd05fce5640 1 -- 172.21.15.76:0/2597434710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0580cd6d0 msgr2=0x7fd058161420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:45.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.303+0000 7fd05fce5640 1 --2- 172.21.15.76:0/2597434710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0580cd6d0 0x7fd058161420 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd054009870 tx=0x7fd05402b8f0 comp rx=0 tx=0).stop 2023-11-09T03:45:45.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.303+0000 7fd05fce5640 1 -- 172.21.15.76:0/2597434710 shutdown_connections 2023-11-09T03:45:45.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.303+0000 7fd05fce5640 1 --2- 172.21.15.76:0/2597434710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0580cd6d0 0x7fd058161420 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:45.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.303+0000 7fd05fce5640 1 --2- 172.21.15.76:0/2597434710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd058153e60 0x7fd05815dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:45.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.303+0000 7fd05fce5640 1 -- 172.21.15.76:0/2597434710 >> 172.21.15.76:0/2597434710 conn(0x7fd0580cf880 msgr2=0x7fd0580cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:45.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.303+0000 7fd05fce5640 1 -- 172.21.15.76:0/2597434710 shutdown_connections 2023-11-09T03:45:45.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.303+0000 7fd05fce5640 1 -- 172.21.15.76:0/2597434710 wait complete. 2023-11-09T03:45:45.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.304+0000 7fd05fce5640 1 Processor -- start 2023-11-09T03:45:45.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.304+0000 7fd05fce5640 1 -- start start 2023-11-09T03:45:45.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.305+0000 7fd05fce5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0580b4880 0x7fd058156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:45.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.305+0000 7fd05fce5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0580cd6d0 0x7fd0581533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:45.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.305+0000 7fd05fce5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd058153e60 0x7fd058155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:45.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.305+0000 7fd05fce5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0581e7230 con 0x7fd058153e60 2023-11-09T03:45:45.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.305+0000 7fd05fce5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05815a420 con 0x7fd0580b4880 2023-11-09T03:45:45.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.305+0000 7fd05fce5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0581e6900 con 0x7fd0580cd6d0 2023-11-09T03:45:45.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.305+0000 7fd05e282640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd058153e60 0x7fd058155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:45.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.305+0000 7fd05e282640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd058153e60 0x7fd058155fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57506/0 (socket says 172.21.15.76:57506) 2023-11-09T03:45:45.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.305+0000 7fd05e282640 1 -- 172.21.15.76:0/3523753182 learned_addr learned my addr 172.21.15.76:0/3523753182 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:45.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.306+0000 7fd05e282640 1 -- 172.21.15.76:0/3523753182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0580cd6d0 msgr2=0x7fd0581533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:45.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.306+0000 7fd05e282640 1 --2- 172.21.15.76:0/3523753182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0580cd6d0 0x7fd0581533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:45.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.306+0000 7fd05e282640 1 -- 172.21.15.76:0/3523753182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0580b4880 msgr2=0x7fd058156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:45.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.306+0000 7fd05da81640 1 --2- 172.21.15.76:0/3523753182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0580b4880 0x7fd058156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:45.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.306+0000 7fd05e282640 1 --2- 172.21.15.76:0/3523753182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0580b4880 0x7fd058156d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:45.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.306+0000 7fd05e282640 1 -- 172.21.15.76:0/3523753182 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0540094d0 con 0x7fd058153e60 2023-11-09T03:45:45.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.306+0000 7fd05e282640 1 --2- 172.21.15.76:0/3523753182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd058153e60 0x7fd058155fb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd05000c780 tx=0x7fd05000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:45.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.306+0000 7fd04e7fc640 1 -- 172.21.15.76:0/3523753182 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd050004b60 con 0x7fd058153e60 2023-11-09T03:45:45.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.306+0000 7fd04e7fc640 1 -- 172.21.15.76:0/3523753182 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05000ce90 con 0x7fd058153e60 2023-11-09T03:45:45.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.306+0000 7fd05fce5640 1 -- 172.21.15.76:0/3523753182 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0581e6be0 con 0x7fd058153e60 2023-11-09T03:45:45.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.306+0000 7fd04e7fc640 1 -- 172.21.15.76:0/3523753182 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05000a850 con 0x7fd058153e60 2023-11-09T03:45:45.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.306+0000 7fd05fce5640 1 -- 172.21.15.76:0/3523753182 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0581564f0 con 0x7fd058153e60 2023-11-09T03:45:45.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.307+0000 7fd05fce5640 1 -- 172.21.15.76:0/3523753182 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0581e73a0 con 0x7fd058153e60 2023-11-09T03:45:45.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.307+0000 7fd04e7fc640 1 -- 172.21.15.76:0/3523753182 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd05000a9b0 con 0x7fd058153e60 2023-11-09T03:45:45.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.308+0000 7fd04e7fc640 1 --2- 172.21.15.76:0/3523753182 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd03c06aea0 0x7fd03c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:45.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.308+0000 7fd04e7fc640 1 -- 172.21.15.76:0/3523753182 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd0500899e0 con 0x7fd058153e60 2023-11-09T03:45:45.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.308+0000 7fd05da81640 1 --2- 172.21.15.76:0/3523753182 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd03c06aea0 0x7fd03c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:45.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.308+0000 7fd05da81640 1 --2- 172.21.15.76:0/3523753182 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd03c06aea0 0x7fd03c06d360 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd04800b7c0 tx=0x7fd048008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:45.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.312+0000 7fd04e7fc640 1 -- 172.21.15.76:0/3523753182 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd050054830 con 0x7fd058153e60 2023-11-09T03:45:45.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.490+0000 7fd05fce5640 1 -- 172.21.15.76:0/3523753182 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd058074290 con 0x7fd058153e60 2023-11-09T03:45:45.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.491+0000 7fd04e7fc640 1 -- 172.21.15.76:0/3523753182 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd050057e50 con 0x7fd058153e60 2023-11-09T03:45:45.490 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2023-11-09T03:45:45.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.493+0000 7fd05fce5640 1 -- 172.21.15.76:0/3523753182 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd03c06aea0 msgr2=0x7fd03c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:45.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.493+0000 7fd05fce5640 1 --2- 172.21.15.76:0/3523753182 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd03c06aea0 0x7fd03c06d360 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd04800b7c0 tx=0x7fd048008040 comp rx=0 tx=0).stop 2023-11-09T03:45:45.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.493+0000 7fd05fce5640 1 -- 172.21.15.76:0/3523753182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd058153e60 msgr2=0x7fd058155fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:45.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.493+0000 7fd05fce5640 1 --2- 172.21.15.76:0/3523753182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd058153e60 0x7fd058155fb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd05000c780 tx=0x7fd05000cc40 comp rx=0 tx=0).stop 2023-11-09T03:45:45.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.493+0000 7fd05fce5640 1 -- 172.21.15.76:0/3523753182 shutdown_connections 2023-11-09T03:45:45.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.493+0000 7fd05fce5640 1 --2- 172.21.15.76:0/3523753182 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd03c06aea0 0x7fd03c06d360 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:45.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.493+0000 7fd05fce5640 1 --2- 172.21.15.76:0/3523753182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd058153e60 0x7fd058155fb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:45.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.493+0000 7fd05fce5640 1 --2- 172.21.15.76:0/3523753182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0580cd6d0 0x7fd0581533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:45.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.493+0000 7fd05fce5640 1 --2- 172.21.15.76:0/3523753182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0580b4880 0x7fd058156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:45.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.493+0000 7fd05fce5640 1 -- 172.21.15.76:0/3523753182 >> 172.21.15.76:0/3523753182 conn(0x7fd0580cf880 msgr2=0x7fd0580d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:45.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.493+0000 7fd05fce5640 1 -- 172.21.15.76:0/3523753182 shutdown_connections 2023-11-09T03:45:45.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.493+0000 7fd05fce5640 1 -- 172.21.15.76:0/3523753182 wait complete. 2023-11-09T03:45:45.502 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi076.front.sepia.ceph.com... 2023-11-09T03:45:45.503 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:45:45.503 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-11-09T03:45:45.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:45:45.563 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi076.stderr:ceph-fuse[94664]: fuse finished with error 0 and tester_r 0 2023-11-09T03:45:45.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:45:45.567 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi076.stderr:2023-11-09T03:45:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:45:45.569 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:45:45.575 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T03:45:45.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:45:45.576 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-11-09T03:45:45.643 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:45:45.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.783+0000 7f3232f55640 1 Processor -- start 2023-11-09T03:45:45.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.783+0000 7f3232f55640 1 -- start start 2023-11-09T03:45:45.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.783+0000 7f3232f55640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f322c15cf30 0x7f322c06cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:45.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.783+0000 7f3232f55640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f322c0cce90 0x7f322c070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:45.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.783+0000 7f3232f55640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322c071500 con 0x7f322c0b07f0 2023-11-09T03:45:45.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.783+0000 7f3232f55640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f322c0c9e00 con 0x7f322c0cce90 2023-11-09T03:45:45.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.783+0000 7f3232f55640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0717e0 con 0x7f322c15cf30 2023-11-09T03:45:45.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.784+0000 7f3230cf1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f322c15cf30 0x7f322c06cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:45.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.784+0000 7f322bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f322c0cce90 0x7f322c070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:45.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.784+0000 7f32314f2640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f322c0b07f0 0x7f322c0ca220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:45980/0 (socket says 172.21.15.76:45980) 2023-11-09T03:45:45.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.784+0000 7f32314f2640 1 -- 172.21.15.76:0/4043413459 learned_addr learned my addr 172.21.15.76:0/4043413459 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:45.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.784+0000 7f322b7fe640 1 -- 172.21.15.76:0/4043413459 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2715011559 0 0) 0x7f322c071500 con 0x7f322c0b07f0 2023-11-09T03:45:45.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.784+0000 7f322b7fe640 1 -- 172.21.15.76:0/4043413459 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f321c003520 con 0x7f322c0b07f0 2023-11-09T03:45:45.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.784+0000 7f322bfff640 1 -- 172.21.15.76:0/4043413459 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f322c15cf30 msgr2=0x7f322c06cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:45.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.784+0000 7f322bfff640 1 --2- 172.21.15.76:0/4043413459 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f322c15cf30 0x7f322c06cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:45.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.784+0000 7f322bfff640 1 -- 172.21.15.76:0/4043413459 >> v1:172.21.15.113:6789/0 conn(0x7f322c0b07f0 legacy=0x7f322c0ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:45.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.784+0000 7f322bfff640 1 -- 172.21.15.76:0/4043413459 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c071ac0 con 0x7f322c0cce90 2023-11-09T03:45:45.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.784+0000 7f3230cf1640 1 --2- 172.21.15.76:0/4043413459 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f322c15cf30 0x7f322c06cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:45:45.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.785+0000 7f322bfff640 1 --2- 172.21.15.76:0/4043413459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f322c0cce90 0x7f322c070bb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f32180097a0 tx=0x7f321802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37a6f0c10375172c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:45.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.785+0000 7f322b7fe640 1 -- 172.21.15.76:0/4043413459 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321802e070 con 0x7f322c0cce90 2023-11-09T03:45:45.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.785+0000 7f322b7fe640 1 -- 172.21.15.76:0/4043413459 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32180315a0 con 0x7f322c0cce90 2023-11-09T03:45:45.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.785+0000 7f322b7fe640 1 -- 172.21.15.76:0/4043413459 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218031910 con 0x7f322c0cce90 2023-11-09T03:45:45.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.786+0000 7f3232f55640 1 -- 172.21.15.76:0/4043413459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f322c0cce90 msgr2=0x7f322c070bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:45.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.786+0000 7f3232f55640 1 --2- 172.21.15.76:0/4043413459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f322c0cce90 0x7f322c070bb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f32180097a0 tx=0x7f321802b840 comp rx=0 tx=0).stop 2023-11-09T03:45:45.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.786+0000 7f3232f55640 1 -- 172.21.15.76:0/4043413459 shutdown_connections 2023-11-09T03:45:45.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.786+0000 7f3232f55640 1 --2- 172.21.15.76:0/4043413459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f322c0cce90 0x7f322c070bb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:45.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.786+0000 7f3232f55640 1 --2- 172.21.15.76:0/4043413459 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f322c15cf30 0x7f322c06cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:45.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.786+0000 7f3232f55640 1 -- 172.21.15.76:0/4043413459 >> 172.21.15.76:0/4043413459 conn(0x7f322c0cb7f0 msgr2=0x7f322c0cbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:45.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.786+0000 7f3232f55640 1 -- 172.21.15.76:0/4043413459 shutdown_connections 2023-11-09T03:45:45.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.786+0000 7f3232f55640 1 -- 172.21.15.76:0/4043413459 wait complete. 2023-11-09T03:45:45.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.787+0000 7f3232f55640 1 Processor -- start 2023-11-09T03:45:45.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.787+0000 7f3232f55640 1 -- start start 2023-11-09T03:45:45.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.788+0000 7f3232f55640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f322c0b07f0 0x7f322c0b1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:45.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.788+0000 7f3232f55640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f322c0cce90 0x7f322c153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:45.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.788+0000 7f3232f55640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f322c15cf30 0x7f322c0c6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:45.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.788+0000 7f3232f55640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0b2ef0 con 0x7f322c0b07f0 2023-11-09T03:45:45.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.788+0000 7f3232f55640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0b2270 con 0x7f322c0cce90 2023-11-09T03:45:45.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.788+0000 7f3232f55640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0b2550 con 0x7f322c15cf30 2023-11-09T03:45:45.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.788+0000 7f3230cf1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f322c0b07f0 0x7f322c0b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:45.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.788+0000 7f3230cf1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f322c0b07f0 0x7f322c0b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57522/0 (socket says 172.21.15.76:57522) 2023-11-09T03:45:45.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.788+0000 7f3230cf1640 1 -- 172.21.15.76:0/2552526577 learned_addr learned my addr 172.21.15.76:0/2552526577 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:45.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.788+0000 7f322bfff640 1 --2- 172.21.15.76:0/2552526577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f322c0cce90 0x7f322c153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:45.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.788+0000 7f3230cf1640 1 -- 172.21.15.76:0/2552526577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f322c15cf30 msgr2=0x7f322c0c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:45.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.788+0000 7f32314f2640 1 --2- 172.21.15.76:0/2552526577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f322c15cf30 0x7f322c0c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:45.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.788+0000 7f3230cf1640 1 --2- 172.21.15.76:0/2552526577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f322c15cf30 0x7f322c0c6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:45.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.788+0000 7f3230cf1640 1 -- 172.21.15.76:0/2552526577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f322c0cce90 msgr2=0x7f322c153d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:45.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.788+0000 7f3230cf1640 1 --2- 172.21.15.76:0/2552526577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f322c0cce90 0x7f322c153d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:45.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.788+0000 7f3230cf1640 1 -- 172.21.15.76:0/2552526577 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3220009400 con 0x7f322c0b07f0 2023-11-09T03:45:45.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.788+0000 7f322bfff640 1 --2- 172.21.15.76:0/2552526577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f322c0cce90 0x7f322c153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:45.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.788+0000 7f3230cf1640 1 --2- 172.21.15.76:0/2552526577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f322c0b07f0 0x7f322c0b1990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f322000ebe0 tx=0x7f322000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:45.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.788+0000 7f32297fa640 1 -- 172.21.15.76:0/2552526577 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3220009760 con 0x7f322c0b07f0 2023-11-09T03:45:45.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.788+0000 7f32297fa640 1 -- 172.21.15.76:0/2552526577 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322000cc30 con 0x7f322c0b07f0 2023-11-09T03:45:45.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.788+0000 7f3232f55640 1 -- 172.21.15.76:0/2552526577 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3218009400 con 0x7f322c0b07f0 2023-11-09T03:45:45.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.788+0000 7f32297fa640 1 -- 172.21.15.76:0/2552526577 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322000eee0 con 0x7f322c0b07f0 2023-11-09T03:45:45.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.788+0000 7f3232f55640 1 -- 172.21.15.76:0/2552526577 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322c0b2b10 con 0x7f322c0b07f0 2023-11-09T03:45:45.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.789+0000 7f3232f55640 1 -- 172.21.15.76:0/2552526577 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f322c06b0b0 con 0x7f322c0b07f0 2023-11-09T03:45:45.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.790+0000 7f32297fa640 1 -- 172.21.15.76:0/2552526577 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3220013470 con 0x7f322c0b07f0 2023-11-09T03:45:45.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.790+0000 7f32297fa640 1 --2- 172.21.15.76:0/2552526577 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f321006ae70 0x7f321006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:45.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.790+0000 7f32297fa640 1 -- 172.21.15.76:0/2552526577 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3220089d20 con 0x7f322c0b07f0 2023-11-09T03:45:45.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.790+0000 7f322bfff640 1 --2- 172.21.15.76:0/2552526577 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f321006ae70 0x7f321006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:45.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.791+0000 7f322bfff640 1 --2- 172.21.15.76:0/2552526577 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f321006ae70 0x7f321006d330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3218036000 tx=0x7f321803d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:45.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.794+0000 7f32297fa640 1 -- 172.21.15.76:0/2552526577 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f32200549b0 con 0x7f322c0b07f0 2023-11-09T03:45:45.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.962+0000 7f3232f55640 1 -- 172.21.15.76:0/2552526577 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f322c0b3160 con 0x7f322c0b07f0 2023-11-09T03:45:45.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.963+0000 7f32297fa640 1 -- 172.21.15.76:0/2552526577 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18334 (secure 0 0 0) 0x7f3220021090 con 0x7f322c0b07f0 2023-11-09T03:45:45.963 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:45.963 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":25,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:45:35.781752+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:45:45.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.965+0000 7f3232f55640 1 -- 172.21.15.76:0/2552526577 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f321006ae70 msgr2=0x7f321006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:45.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.965+0000 7f3232f55640 1 --2- 172.21.15.76:0/2552526577 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f321006ae70 0x7f321006d330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3218036000 tx=0x7f321803d040 comp rx=0 tx=0).stop 2023-11-09T03:45:45.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.965+0000 7f3232f55640 1 -- 172.21.15.76:0/2552526577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f322c0b07f0 msgr2=0x7f322c0b1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:45.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.965+0000 7f3232f55640 1 --2- 172.21.15.76:0/2552526577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f322c0b07f0 0x7f322c0b1990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f322000ebe0 tx=0x7f322000c520 comp rx=0 tx=0).stop 2023-11-09T03:45:45.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.965+0000 7f3232f55640 1 -- 172.21.15.76:0/2552526577 shutdown_connections 2023-11-09T03:45:45.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.965+0000 7f3232f55640 1 --2- 172.21.15.76:0/2552526577 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f321006ae70 0x7f321006d330 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:45.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.965+0000 7f3232f55640 1 --2- 172.21.15.76:0/2552526577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f322c15cf30 0x7f322c0c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:45.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.965+0000 7f3232f55640 1 --2- 172.21.15.76:0/2552526577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f322c0cce90 0x7f322c153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:45.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.965+0000 7f3232f55640 1 --2- 172.21.15.76:0/2552526577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f322c0b07f0 0x7f322c0b1990 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:45.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.965+0000 7f3232f55640 1 -- 172.21.15.76:0/2552526577 >> 172.21.15.76:0/2552526577 conn(0x7f322c0cb7f0 msgr2=0x7f322c0ab8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:45.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.965+0000 7f3232f55640 1 -- 172.21.15.76:0/2552526577 shutdown_connections 2023-11-09T03:45:45.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:45.965+0000 7f3232f55640 1 -- 172.21.15.76:0/2552526577 wait complete. 2023-11-09T03:45:45.975 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:45:46.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.076+0000 7fbea69c0640 1 Processor -- start 2023-11-09T03:45:46.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.076+0000 7fbea69c0640 1 -- start start 2023-11-09T03:45:46.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.076+0000 7fbea69c0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbea00ce240 0x7fbea0157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:46.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.076+0000 7fbea69c0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbea015e000 con 0x7fbea0157540 2023-11-09T03:45:46.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.076+0000 7fbea69c0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbea015e2e0 con 0x7fbea00cde60 2023-11-09T03:45:46.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.076+0000 7fbea69c0640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbea015e5c0 con 0x7fbea00ce240 2023-11-09T03:45:46.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.077+0000 7fbea4f5d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fbea0157540 0x7fbea00d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:56686/0 (socket says 172.21.15.76:56686) 2023-11-09T03:45:46.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.077+0000 7fbea4f5d640 1 -- 172.21.15.76:0/708335264 learned_addr learned my addr 172.21.15.76:0/708335264 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:46.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.077+0000 7fbe9f7fe640 1 --2- 172.21.15.76:0/708335264 >> v2:172.21.15.113:3300/0 conn(0x7fbea00ce240 0x7fbea0157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:46.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.077+0000 7fbe9effd640 1 -- 172.21.15.76:0/708335264 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 438570245 0 0) 0x7fbea015e000 con 0x7fbea0157540 2023-11-09T03:45:46.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.077+0000 7fbe9effd640 1 -- 172.21.15.76:0/708335264 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe88003520 con 0x7fbea0157540 2023-11-09T03:45:46.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.077+0000 7fbe9effd640 1 -- 172.21.15.76:0/708335264 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1034963071 0 0) 0x7fbea015e2e0 con 0x7fbea00cde60 2023-11-09T03:45:46.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.077+0000 7fbe9effd640 1 -- 172.21.15.76:0/708335264 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbea015e000 con 0x7fbea00cde60 2023-11-09T03:45:46.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.077+0000 7fbe9effd640 1 -- 172.21.15.76:0/708335264 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2485915066 0 0) 0x7fbe88003520 con 0x7fbea0157540 2023-11-09T03:45:46.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.077+0000 7fbe9f7fe640 1 -- 172.21.15.76:0/708335264 >> v1:172.21.15.113:6789/0 conn(0x7fbea00cde60 legacy=0x7fbea015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:46.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.077+0000 7fbe9f7fe640 1 -- 172.21.15.76:0/708335264 >> v1:172.21.15.76:6789/0 conn(0x7fbea0157540 legacy=0x7fbea00d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:46.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.077+0000 7fbe9f7fe640 1 -- 172.21.15.76:0/708335264 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea00d1120 con 0x7fbea00ce240 2023-11-09T03:45:46.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.078+0000 7fbe9f7fe640 1 --2- 172.21.15.76:0/708335264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbea00ce240 0x7fbea0157000 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbe940097a0 tx=0x7fbe9402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ef4a81144bb439e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:46.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.078+0000 7fbe9effd640 1 -- 172.21.15.76:0/708335264 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9402f050 con 0x7fbea00ce240 2023-11-09T03:45:46.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.078+0000 7fbe9effd640 1 -- 172.21.15.76:0/708335264 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe94031500 con 0x7fbea00ce240 2023-11-09T03:45:46.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.078+0000 7fbe9effd640 1 -- 172.21.15.76:0/708335264 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe980019f0 con 0x7fbea00ce240 2023-11-09T03:45:46.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.079+0000 7fbea69c0640 1 -- 172.21.15.76:0/708335264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbea00ce240 msgr2=0x7fbea0157000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:46.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.079+0000 7fbea69c0640 1 --2- 172.21.15.76:0/708335264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbea00ce240 0x7fbea0157000 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbe940097a0 tx=0x7fbe9402b820 comp rx=0 tx=0).stop 2023-11-09T03:45:46.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.079+0000 7fbea69c0640 1 -- 172.21.15.76:0/708335264 shutdown_connections 2023-11-09T03:45:46.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.079+0000 7fbea69c0640 1 --2- 172.21.15.76:0/708335264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbea00ce240 0x7fbea0157000 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:46.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.079+0000 7fbea69c0640 1 -- 172.21.15.76:0/708335264 >> 172.21.15.76:0/708335264 conn(0x7fbea0071f30 msgr2=0x7fbea00cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:46.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.079+0000 7fbea69c0640 1 -- 172.21.15.76:0/708335264 shutdown_connections 2023-11-09T03:45:46.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.079+0000 7fbea69c0640 1 -- 172.21.15.76:0/708335264 wait complete. 2023-11-09T03:45:46.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.080+0000 7fbea69c0640 1 Processor -- start 2023-11-09T03:45:46.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.080+0000 7fbea69c0640 1 -- start start 2023-11-09T03:45:46.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.081+0000 7fbea69c0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbea00cde60 0x7fbea0153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:46.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.081+0000 7fbea69c0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbea00ce240 0x7fbea0158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:46.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.081+0000 7fbea69c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbea0157540 0x7fbea01550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:46.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.081+0000 7fbea69c0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea015af00 con 0x7fbea0157540 2023-11-09T03:45:46.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.081+0000 7fbea69c0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea015abd0 con 0x7fbea00ce240 2023-11-09T03:45:46.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.081+0000 7fbea69c0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea0159750 con 0x7fbea00cde60 2023-11-09T03:45:46.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.081+0000 7fbea4f5d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbea0157540 0x7fbea01550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:46.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.081+0000 7fbea4f5d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbea0157540 0x7fbea01550f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57526/0 (socket says 172.21.15.76:57526) 2023-11-09T03:45:46.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.081+0000 7fbea4f5d640 1 -- 172.21.15.76:0/1810741018 learned_addr learned my addr 172.21.15.76:0/1810741018 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:46.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.081+0000 7fbe9f7fe640 1 --2- 172.21.15.76:0/1810741018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbea00ce240 0x7fbea0158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:46.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.081+0000 7fbe9ffff640 1 --2- 172.21.15.76:0/1810741018 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbea00cde60 0x7fbea0153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:46.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.081+0000 7fbea4f5d640 1 -- 172.21.15.76:0/1810741018 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbea00cde60 msgr2=0x7fbea0153550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:46.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.081+0000 7fbea4f5d640 1 --2- 172.21.15.76:0/1810741018 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbea00cde60 0x7fbea0153550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:46.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.081+0000 7fbea4f5d640 1 -- 172.21.15.76:0/1810741018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbea00ce240 msgr2=0x7fbea0158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:46.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.081+0000 7fbea4f5d640 1 --2- 172.21.15.76:0/1810741018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbea00ce240 0x7fbea0158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:46.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.081+0000 7fbea4f5d640 1 -- 172.21.15.76:0/1810741018 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe94009400 con 0x7fbea0157540 2023-11-09T03:45:46.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.081+0000 7fbea4f5d640 1 --2- 172.21.15.76:0/1810741018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbea0157540 0x7fbea01550f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbe98001f00 tx=0x7fbe9800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:46.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.081+0000 7fbe9cff9640 1 -- 172.21.15.76:0/1810741018 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe98012400 con 0x7fbea0157540 2023-11-09T03:45:46.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.081+0000 7fbe9cff9640 1 -- 172.21.15.76:0/1810741018 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe98012560 con 0x7fbea0157540 2023-11-09T03:45:46.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.082+0000 7fbea69c0640 1 -- 172.21.15.76:0/1810741018 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbea0158eb0 con 0x7fbea0157540 2023-11-09T03:45:46.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.082+0000 7fbe9cff9640 1 -- 172.21.15.76:0/1810741018 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe98012910 con 0x7fbea0157540 2023-11-09T03:45:46.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.082+0000 7fbea69c0640 1 -- 172.21.15.76:0/1810741018 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea0159470 con 0x7fbea0157540 2023-11-09T03:45:46.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.083+0000 7fbea69c0640 1 -- 172.21.15.76:0/1810741018 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbea015b070 con 0x7fbea0157540 2023-11-09T03:45:46.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.083+0000 7fbe9cff9640 1 -- 172.21.15.76:0/1810741018 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbe98012a70 con 0x7fbea0157540 2023-11-09T03:45:46.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.083+0000 7fbe9cff9640 1 --2- 172.21.15.76:0/1810741018 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbe8c06ae70 0x7fbe8c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:46.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.084+0000 7fbe9cff9640 1 -- 172.21.15.76:0/1810741018 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbe98089f40 con 0x7fbea0157540 2023-11-09T03:45:46.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.084+0000 7fbe9ffff640 1 --2- 172.21.15.76:0/1810741018 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbe8c06ae70 0x7fbe8c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:46.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.084+0000 7fbe9ffff640 1 --2- 172.21.15.76:0/1810741018 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbe8c06ae70 0x7fbe8c06d330 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbe900011e0 tx=0x7fbe90009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:46.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.088+0000 7fbe9cff9640 1 -- 172.21.15.76:0/1810741018 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbe98054e40 con 0x7fbea0157540 2023-11-09T03:45:46.255 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.256+0000 7fbea69c0640 1 -- 172.21.15.76:0/1810741018 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbea0074290 con 0x7fbea0157540 2023-11-09T03:45:46.256 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.257+0000 7fbe9cff9640 1 -- 172.21.15.76:0/1810741018 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18334 (secure 0 0 0) 0x7fbe98058460 con 0x7fbea0157540 2023-11-09T03:45:46.257 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:46.257 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":25,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:45:35.781752+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:45:46.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.259+0000 7fbea69c0640 1 -- 172.21.15.76:0/1810741018 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbe8c06ae70 msgr2=0x7fbe8c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:46.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.259+0000 7fbea69c0640 1 --2- 172.21.15.76:0/1810741018 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbe8c06ae70 0x7fbe8c06d330 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbe900011e0 tx=0x7fbe90009040 comp rx=0 tx=0).stop 2023-11-09T03:45:46.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.259+0000 7fbea69c0640 1 -- 172.21.15.76:0/1810741018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbea0157540 msgr2=0x7fbea01550f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:46.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.259+0000 7fbea69c0640 1 --2- 172.21.15.76:0/1810741018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbea0157540 0x7fbea01550f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbe98001f00 tx=0x7fbe9800a430 comp rx=0 tx=0).stop 2023-11-09T03:45:46.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.259+0000 7fbea69c0640 1 -- 172.21.15.76:0/1810741018 shutdown_connections 2023-11-09T03:45:46.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.259+0000 7fbea69c0640 1 --2- 172.21.15.76:0/1810741018 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbe8c06ae70 0x7fbe8c06d330 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:46.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.259+0000 7fbea69c0640 1 --2- 172.21.15.76:0/1810741018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbea0157540 0x7fbea01550f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:46.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.259+0000 7fbea69c0640 1 --2- 172.21.15.76:0/1810741018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbea00ce240 0x7fbea0158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:46.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.259+0000 7fbea69c0640 1 --2- 172.21.15.76:0/1810741018 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbea00cde60 0x7fbea0153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:46.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.259+0000 7fbea69c0640 1 -- 172.21.15.76:0/1810741018 >> 172.21.15.76:0/1810741018 conn(0x7fbea0071f30 msgr2=0x7fbea00b65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:46.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.260+0000 7fbea69c0640 1 -- 172.21.15.76:0/1810741018 shutdown_connections 2023-11-09T03:45:46.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.260+0000 7fbea69c0640 1 -- 172.21.15.76:0/1810741018 wait complete. 2023-11-09T03:45:46.270 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-09T03:45:46.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.373+0000 7f4b040c2640 1 Processor -- start 2023-11-09T03:45:46.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.373+0000 7f4b040c2640 1 -- start start 2023-11-09T03:45:46.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.374+0000 7f4b040c2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4afc160430 0x7f4afc0b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:46.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.374+0000 7f4b040c2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4afc06c3a0 0x7f4afc0cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:46.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.374+0000 7f4b040c2640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4afc0b2fc0 con 0x7f4afc0b2b60 2023-11-09T03:45:46.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.374+0000 7f4b040c2640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4afc0cb580 con 0x7f4afc160430 2023-11-09T03:45:46.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.374+0000 7f4b040c2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4afc06c7d0 con 0x7f4afc06c3a0 2023-11-09T03:45:46.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.374+0000 7f4b01e5e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4afc160430 0x7f4afc0b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:46.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.374+0000 7f4b0165d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f4afc0b2b60 0x7f4afc06bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:56694/0 (socket says 172.21.15.76:56694) 2023-11-09T03:45:46.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.374+0000 7f4b0165d640 1 -- 172.21.15.76:0/2022799841 learned_addr learned my addr 172.21.15.76:0/2022799841 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:46.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.374+0000 7f4b0265f640 1 --2- 172.21.15.76:0/2022799841 >> v2:172.21.15.113:3300/0 conn(0x7f4afc06c3a0 0x7f4afc0cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:46.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.374+0000 7f4b01e5e640 1 -- 172.21.15.76:0/2022799841 >> v2:172.21.15.113:3300/0 conn(0x7f4afc06c3a0 msgr2=0x7f4afc0cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:46.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.374+0000 7f4b01e5e640 1 --2- 172.21.15.76:0/2022799841 >> v2:172.21.15.113:3300/0 conn(0x7f4afc06c3a0 0x7f4afc0cbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:46.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.374+0000 7f4b01e5e640 1 -- 172.21.15.76:0/2022799841 >> v1:172.21.15.76:6789/0 conn(0x7f4afc0b2b60 legacy=0x7f4afc06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:46.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.374+0000 7f4b00e5c640 1 -- 172.21.15.76:0/2022799841 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3320417819 0 0) 0x7f4afc0b2fc0 con 0x7f4afc0b2b60 2023-11-09T03:45:46.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.374+0000 7f4b01e5e640 1 -- 172.21.15.76:0/2022799841 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4afc06ca40 con 0x7f4afc160430 2023-11-09T03:45:46.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.375+0000 7f4b01e5e640 1 --2- 172.21.15.76:0/2022799841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4afc160430 0x7f4afc0b2620 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0c4600 tx=0x7f4af802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc025af70d3d7e70 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:46.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.375+0000 7f4b00e5c640 1 -- 172.21.15.76:0/2022799841 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af8031830 con 0x7f4afc160430 2023-11-09T03:45:46.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.375+0000 7f4b00e5c640 1 -- 172.21.15.76:0/2022799841 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af8031990 con 0x7f4afc160430 2023-11-09T03:45:46.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.375+0000 7f4b00e5c640 1 -- 172.21.15.76:0/2022799841 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af8031d20 con 0x7f4afc160430 2023-11-09T03:45:46.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.376+0000 7f4b040c2640 1 -- 172.21.15.76:0/2022799841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4afc160430 msgr2=0x7f4afc0b2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:46.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.376+0000 7f4b040c2640 1 --2- 172.21.15.76:0/2022799841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4afc160430 0x7f4afc0b2620 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0c4600 tx=0x7f4af802b8d0 comp rx=0 tx=0).stop 2023-11-09T03:45:46.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.376+0000 7f4b040c2640 1 -- 172.21.15.76:0/2022799841 shutdown_connections 2023-11-09T03:45:46.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.376+0000 7f4b040c2640 1 --2- 172.21.15.76:0/2022799841 >> v2:172.21.15.113:3300/0 conn(0x7f4afc06c3a0 0x7f4afc0cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:46.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.376+0000 7f4b040c2640 1 --2- 172.21.15.76:0/2022799841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4afc160430 0x7f4afc0b2620 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:46.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.376+0000 7f4b040c2640 1 -- 172.21.15.76:0/2022799841 >> 172.21.15.76:0/2022799841 conn(0x7f4afc06ea20 msgr2=0x7f4afc06ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:46.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.376+0000 7f4b040c2640 1 -- 172.21.15.76:0/2022799841 shutdown_connections 2023-11-09T03:45:46.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.376+0000 7f4b040c2640 1 -- 172.21.15.76:0/2022799841 wait complete. 2023-11-09T03:45:46.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.377+0000 7f4b040c2640 1 Processor -- start 2023-11-09T03:45:46.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.377+0000 7f4b040c2640 1 -- start start 2023-11-09T03:45:46.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.378+0000 7f4b040c2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4afc06c3a0 0x7f4afc152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:46.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.378+0000 7f4b040c2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4afc0b2b60 0x7f4afc0b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:46.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.378+0000 7f4b040c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4afc160430 0x7f4afc0c6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:46.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.378+0000 7f4b040c2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc152690 con 0x7f4afc160430 2023-11-09T03:45:46.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.378+0000 7f4b040c2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc0b40c0 con 0x7f4afc0b2b60 2023-11-09T03:45:46.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.378+0000 7f4b040c2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc0b43a0 con 0x7f4afc06c3a0 2023-11-09T03:45:46.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.378+0000 7f4b0265f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4afc160430 0x7f4afc0c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:46.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.378+0000 7f4b0265f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4afc160430 0x7f4afc0c6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57544/0 (socket says 172.21.15.76:57544) 2023-11-09T03:45:46.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.378+0000 7f4b0265f640 1 -- 172.21.15.76:0/2803818504 learned_addr learned my addr 172.21.15.76:0/2803818504 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:46.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.378+0000 7f4b0165d640 1 --2- 172.21.15.76:0/2803818504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4afc0b2b60 0x7f4afc0b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:46.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.378+0000 7f4b01e5e640 1 --2- 172.21.15.76:0/2803818504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4afc06c3a0 0x7f4afc152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:46.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.378+0000 7f4b0265f640 1 -- 172.21.15.76:0/2803818504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4afc06c3a0 msgr2=0x7f4afc152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:46.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.378+0000 7f4b0265f640 1 --2- 172.21.15.76:0/2803818504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4afc06c3a0 0x7f4afc152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:46.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.378+0000 7f4b0265f640 1 -- 172.21.15.76:0/2803818504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4afc0b2b60 msgr2=0x7f4afc0b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:46.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.378+0000 7f4b0265f640 1 --2- 172.21.15.76:0/2803818504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4afc0b2b60 0x7f4afc0b37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:46.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.378+0000 7f4b0265f640 1 -- 172.21.15.76:0/2803818504 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af8009400 con 0x7f4afc160430 2023-11-09T03:45:46.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.378+0000 7f4b01e5e640 1 --2- 172.21.15.76:0/2803818504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4afc06c3a0 0x7f4afc152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:46.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.378+0000 7f4b0265f640 1 --2- 172.21.15.76:0/2803818504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4afc160430 0x7f4afc0c6910 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4af400cd70 tx=0x7f4af400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:46.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.378+0000 7f4af27fc640 1 -- 172.21.15.76:0/2803818504 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af4003b90 con 0x7f4afc160430 2023-11-09T03:45:46.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.378+0000 7f4af27fc640 1 -- 172.21.15.76:0/2803818504 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af4003cf0 con 0x7f4afc160430 2023-11-09T03:45:46.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.378+0000 7f4b040c2640 1 -- 172.21.15.76:0/2803818504 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4afc0b4680 con 0x7f4afc160430 2023-11-09T03:45:46.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.378+0000 7f4af27fc640 1 -- 172.21.15.76:0/2803818504 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af4004240 con 0x7f4afc160430 2023-11-09T03:45:46.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.378+0000 7f4b040c2640 1 -- 172.21.15.76:0/2803818504 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4afc0b4c40 con 0x7f4afc160430 2023-11-09T03:45:46.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.379+0000 7f4b040c2640 1 -- 172.21.15.76:0/2803818504 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4afc0c4670 con 0x7f4afc160430 2023-11-09T03:45:46.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.380+0000 7f4af27fc640 1 -- 172.21.15.76:0/2803818504 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4af40043a0 con 0x7f4afc160430 2023-11-09T03:45:46.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.380+0000 7f4af27fc640 1 --2- 172.21.15.76:0/2803818504 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4ae006ae70 0x7f4ae006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:46.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.380+0000 7f4af27fc640 1 -- 172.21.15.76:0/2803818504 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4af4012070 con 0x7f4afc160430 2023-11-09T03:45:46.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.380+0000 7f4b01e5e640 1 --2- 172.21.15.76:0/2803818504 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4ae006ae70 0x7f4ae006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:46.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.381+0000 7f4b01e5e640 1 --2- 172.21.15.76:0/2803818504 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4ae006ae70 0x7f4ae006d330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4af8036000 tx=0x7f4af8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:46.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.384+0000 7f4af27fc640 1 -- 172.21.15.76:0/2803818504 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4afc0c4670 con 0x7f4afc160430 2023-11-09T03:45:46.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:46.590+0000 7f4b040c2640 1 -- 172.21.15.76:0/2803818504 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4afc0c4670 con 0x7f4afc160430 2023-11-09T03:45:47.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.465+0000 7f4af27fc640 1 -- 172.21.15.76:0/2803818504 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4afc0c4670 con 0x7f4afc160430 2023-11-09T03:45:47.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.468+0000 7f4b040c2640 1 -- 172.21.15.76:0/2803818504 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4ae006ae70 msgr2=0x7f4ae006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:47.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.468+0000 7f4b040c2640 1 --2- 172.21.15.76:0/2803818504 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4ae006ae70 0x7f4ae006d330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4af8036000 tx=0x7f4af8008030 comp rx=0 tx=0).stop 2023-11-09T03:45:47.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.468+0000 7f4b040c2640 1 -- 172.21.15.76:0/2803818504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4afc160430 msgr2=0x7f4afc0c6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:47.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.468+0000 7f4b040c2640 1 --2- 172.21.15.76:0/2803818504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4afc160430 0x7f4afc0c6910 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4af400cd70 tx=0x7f4af400a660 comp rx=0 tx=0).stop 2023-11-09T03:45:47.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.468+0000 7f4b040c2640 1 -- 172.21.15.76:0/2803818504 shutdown_connections 2023-11-09T03:45:47.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.468+0000 7f4b040c2640 1 --2- 172.21.15.76:0/2803818504 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4ae006ae70 0x7f4ae006d330 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:47.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.468+0000 7f4b040c2640 1 --2- 172.21.15.76:0/2803818504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4afc160430 0x7f4afc0c6910 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:47.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.468+0000 7f4b040c2640 1 --2- 172.21.15.76:0/2803818504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4afc0b2b60 0x7f4afc0b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:47.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.468+0000 7f4b040c2640 1 --2- 172.21.15.76:0/2803818504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4afc06c3a0 0x7f4afc152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:47.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.468+0000 7f4b040c2640 1 -- 172.21.15.76:0/2803818504 >> 172.21.15.76:0/2803818504 conn(0x7f4afc06ea20 msgr2=0x7f4afc0cbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:47.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.468+0000 7f4b040c2640 1 -- 172.21.15.76:0/2803818504 shutdown_connections 2023-11-09T03:45:47.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.468+0000 7f4b040c2640 1 -- 172.21.15.76:0/2803818504 wait complete. 2023-11-09T03:45:47.480 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-11-09T03:45:47.481 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-09T03:45:47.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.589+0000 7fe112965640 1 Processor -- start 2023-11-09T03:45:47.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.589+0000 7fe112965640 1 -- start start 2023-11-09T03:45:47.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.590+0000 7fe112965640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe10c15e430 0x7fe10c0cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:47.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.590+0000 7fe112965640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe10c0d0710 0x7fe10c0b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:47.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.590+0000 7fe112965640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10c0d0280 con 0x7fe10c0cf160 2023-11-09T03:45:47.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.590+0000 7fe112965640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe10c0cf670 con 0x7fe10c15e430 2023-11-09T03:45:47.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.590+0000 7fe112965640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe10c0cf950 con 0x7fe10c0d0710 2023-11-09T03:45:47.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.590+0000 7fe10bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe10c15e430 0x7fe10c0cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:47.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.590+0000 7fe10b7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe10c0cf160 0x7fe10c0cd510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:56704/0 (socket says 172.21.15.76:56704) 2023-11-09T03:45:47.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.590+0000 7fe10b7fe640 1 -- 172.21.15.76:0/1894463817 learned_addr learned my addr 172.21.15.76:0/1894463817 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:47.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.590+0000 7fe110f02640 1 --2- 172.21.15.76:0/1894463817 >> v2:172.21.15.113:3300/0 conn(0x7fe10c0d0710 0x7fe10c0b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:47.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.590+0000 7fe10bfff640 1 -- 172.21.15.76:0/1894463817 >> v2:172.21.15.113:3300/0 conn(0x7fe10c0d0710 msgr2=0x7fe10c0b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:47.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.590+0000 7fe10bfff640 1 --2- 172.21.15.76:0/1894463817 >> v2:172.21.15.113:3300/0 conn(0x7fe10c0d0710 0x7fe10c0b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:47.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.590+0000 7fe10bfff640 1 -- 172.21.15.76:0/1894463817 >> v1:172.21.15.76:6789/0 conn(0x7fe10c0cf160 legacy=0x7fe10c0cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:47.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.590+0000 7fe10bfff640 1 -- 172.21.15.76:0/1894463817 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10c0cfc30 con 0x7fe10c15e430 2023-11-09T03:45:47.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.590+0000 7fe10affd640 1 -- 172.21.15.76:0/1894463817 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3307816186 0 0) 0x7fe10c0d0280 con 0x7fe10c0cf160 2023-11-09T03:45:47.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.590+0000 7fe110f02640 1 --2- 172.21.15.76:0/1894463817 >> v2:172.21.15.113:3300/0 conn(0x7fe10c0d0710 0x7fe10c0b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:47.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.591+0000 7fe10bfff640 1 --2- 172.21.15.76:0/1894463817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe10c15e430 0x7fe10c0cec20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe10c0b6250 tx=0x7fe0fc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651bcbdaf35bae43 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:47.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.591+0000 7fe10affd640 1 -- 172.21.15.76:0/1894463817 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc031830 con 0x7fe10c15e430 2023-11-09T03:45:47.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.591+0000 7fe10affd640 1 -- 172.21.15.76:0/1894463817 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0fc031990 con 0x7fe10c15e430 2023-11-09T03:45:47.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.591+0000 7fe10affd640 1 -- 172.21.15.76:0/1894463817 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc031d20 con 0x7fe10c15e430 2023-11-09T03:45:47.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.592+0000 7fe112965640 1 -- 172.21.15.76:0/1894463817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe10c15e430 msgr2=0x7fe10c0cec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:47.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.592+0000 7fe112965640 1 --2- 172.21.15.76:0/1894463817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe10c15e430 0x7fe10c0cec20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe10c0b6250 tx=0x7fe0fc02b8d0 comp rx=0 tx=0).stop 2023-11-09T03:45:47.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.592+0000 7fe112965640 1 -- 172.21.15.76:0/1894463817 shutdown_connections 2023-11-09T03:45:47.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.592+0000 7fe112965640 1 --2- 172.21.15.76:0/1894463817 >> v2:172.21.15.113:3300/0 conn(0x7fe10c0d0710 0x7fe10c0b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:47.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.592+0000 7fe112965640 1 --2- 172.21.15.76:0/1894463817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe10c15e430 0x7fe10c0cec20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:47.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.592+0000 7fe112965640 1 -- 172.21.15.76:0/1894463817 >> 172.21.15.76:0/1894463817 conn(0x7fe10c06ea20 msgr2=0x7fe10c06ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:47.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.592+0000 7fe112965640 1 -- 172.21.15.76:0/1894463817 shutdown_connections 2023-11-09T03:45:47.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.592+0000 7fe112965640 1 -- 172.21.15.76:0/1894463817 wait complete. 2023-11-09T03:45:47.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.593+0000 7fe112965640 1 Processor -- start 2023-11-09T03:45:47.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.593+0000 7fe112965640 1 -- start start 2023-11-09T03:45:47.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.594+0000 7fe112965640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe10c0cf160 0x7fe10c0b1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:47.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.594+0000 7fe112965640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe10c0d0710 0x7fe10c0cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:47.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.594+0000 7fe112965640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe10c15e430 0x7fe10c0cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:47.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.594+0000 7fe112965640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10c0c4a30 con 0x7fe10c0d0710 2023-11-09T03:45:47.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.594+0000 7fe112965640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10c0c4d10 con 0x7fe10c15e430 2023-11-09T03:45:47.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.594+0000 7fe112965640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe10c0c4ff0 con 0x7fe10c0cf160 2023-11-09T03:45:47.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.594+0000 7fe10b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe10c0d0710 0x7fe10c0cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:47.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.594+0000 7fe10b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe10c0d0710 0x7fe10c0cb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57554/0 (socket says 172.21.15.76:57554) 2023-11-09T03:45:47.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.594+0000 7fe10b7fe640 1 -- 172.21.15.76:0/542546629 learned_addr learned my addr 172.21.15.76:0/542546629 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:47.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.594+0000 7fe10b7fe640 1 -- 172.21.15.76:0/542546629 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe10c0cf160 msgr2=0x7fe10c0b1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:47.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.594+0000 7fe10b7fe640 1 --2- 172.21.15.76:0/542546629 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe10c0cf160 0x7fe10c0b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:47.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.594+0000 7fe10b7fe640 1 -- 172.21.15.76:0/542546629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe10c15e430 msgr2=0x7fe10c0cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:47.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.594+0000 7fe10b7fe640 1 --2- 172.21.15.76:0/542546629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe10c15e430 0x7fe10c0cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:47.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.594+0000 7fe10b7fe640 1 -- 172.21.15.76:0/542546629 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0fc009400 con 0x7fe10c0d0710 2023-11-09T03:45:47.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.594+0000 7fe10b7fe640 1 --2- 172.21.15.76:0/542546629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe10c0d0710 0x7fe10c0cb3f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe10000c430 tx=0x7fe10000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:47.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.594+0000 7fe108ff9640 1 -- 172.21.15.76:0/542546629 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1000074e0 con 0x7fe10c0d0710 2023-11-09T03:45:47.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.594+0000 7fe108ff9640 1 -- 172.21.15.76:0/542546629 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe100007660 con 0x7fe10c0d0710 2023-11-09T03:45:47.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.594+0000 7fe112965640 1 -- 172.21.15.76:0/542546629 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe10c0c52d0 con 0x7fe10c0d0710 2023-11-09T03:45:47.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.594+0000 7fe108ff9640 1 -- 172.21.15.76:0/542546629 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe100004a20 con 0x7fe10c0d0710 2023-11-09T03:45:47.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.594+0000 7fe112965640 1 -- 172.21.15.76:0/542546629 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe10c0c4650 con 0x7fe10c0d0710 2023-11-09T03:45:47.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.595+0000 7fe112965640 1 -- 172.21.15.76:0/542546629 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe10c0c5c70 con 0x7fe10c0d0710 2023-11-09T03:45:47.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.596+0000 7fe108ff9640 1 -- 172.21.15.76:0/542546629 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe100004b80 con 0x7fe10c0d0710 2023-11-09T03:45:47.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.596+0000 7fe108ff9640 1 --2- 172.21.15.76:0/542546629 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe0f806ae70 0x7fe0f806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:47.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.596+0000 7fe108ff9640 1 -- 172.21.15.76:0/542546629 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe100089790 con 0x7fe10c0d0710 2023-11-09T03:45:47.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.596+0000 7fe10bfff640 1 --2- 172.21.15.76:0/542546629 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe0f806ae70 0x7fe0f806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:47.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.597+0000 7fe10bfff640 1 --2- 172.21.15.76:0/542546629 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe0f806ae70 0x7fe0f806d330 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc036000 tx=0x7fe0fc030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:47.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.601+0000 7fe108ff9640 1 -- 172.21.15.76:0/542546629 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe10c0c5c70 con 0x7fe10c0d0710 2023-11-09T03:45:47.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:47.803+0000 7fe112965640 1 -- 172.21.15.76:0/542546629 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe10c0c5c70 con 0x7fe10c0d0710 2023-11-09T03:45:48.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.473+0000 7fe108ff9640 1 -- 172.21.15.76:0/542546629 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe10c0c5c70 con 0x7fe10c0d0710 2023-11-09T03:45:48.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.475+0000 7fe112965640 1 -- 172.21.15.76:0/542546629 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe0f806ae70 msgr2=0x7fe0f806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:48.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.475+0000 7fe112965640 1 --2- 172.21.15.76:0/542546629 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe0f806ae70 0x7fe0f806d330 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc036000 tx=0x7fe0fc030000 comp rx=0 tx=0).stop 2023-11-09T03:45:48.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.475+0000 7fe112965640 1 -- 172.21.15.76:0/542546629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe10c0d0710 msgr2=0x7fe10c0cb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:48.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.475+0000 7fe112965640 1 --2- 172.21.15.76:0/542546629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe10c0d0710 0x7fe10c0cb3f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe10000c430 tx=0x7fe10000c8f0 comp rx=0 tx=0).stop 2023-11-09T03:45:48.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.475+0000 7fe112965640 1 -- 172.21.15.76:0/542546629 shutdown_connections 2023-11-09T03:45:48.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.475+0000 7fe112965640 1 --2- 172.21.15.76:0/542546629 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe0f806ae70 0x7fe0f806d330 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:48.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.475+0000 7fe112965640 1 --2- 172.21.15.76:0/542546629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe10c15e430 0x7fe10c0cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:48.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.475+0000 7fe112965640 1 --2- 172.21.15.76:0/542546629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe10c0d0710 0x7fe10c0cb3f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:48.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.475+0000 7fe112965640 1 --2- 172.21.15.76:0/542546629 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe10c0cf160 0x7fe10c0b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:48.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.475+0000 7fe112965640 1 -- 172.21.15.76:0/542546629 >> 172.21.15.76:0/542546629 conn(0x7fe10c06ea20 msgr2=0x7fe10c15cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:48.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.475+0000 7fe112965640 1 -- 172.21.15.76:0/542546629 shutdown_connections 2023-11-09T03:45:48.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.475+0000 7fe112965640 1 -- 172.21.15.76:0/542546629 wait complete. 2023-11-09T03:45:48.487 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-09T03:45:48.488 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:45:48.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.594+0000 7f1eb1b7c640 1 Processor -- start 2023-11-09T03:45:48.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.594+0000 7f1eb1b7c640 1 -- start start 2023-11-09T03:45:48.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.595+0000 7f1eb1b7c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1eac15ce70 0x7f1eac06d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:48.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.595+0000 7f1eb1b7c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1eac15d250 0x7f1eac0c9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:48.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.595+0000 7f1eb1b7c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eac06a120 con 0x7f1eac0c9d70 2023-11-09T03:45:48.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.595+0000 7f1eb1b7c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eac0cdc10 con 0x7f1eac15ce70 2023-11-09T03:45:48.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.595+0000 7f1eb1b7c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eac06def0 con 0x7f1eac15d250 2023-11-09T03:45:48.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.595+0000 7f1eab7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1eac15ce70 0x7f1eac06d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:48.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.595+0000 7f1eab7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1eac15ce70 0x7f1eac06d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57556/0 (socket says 172.21.15.76:57556) 2023-11-09T03:45:48.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.595+0000 7f1eab7fe640 1 -- 172.21.15.76:0/2129265237 learned_addr learned my addr 172.21.15.76:0/2129265237 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:48.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.595+0000 7f1eaaffd640 1 --2- 172.21.15.76:0/2129265237 >> v2:172.21.15.113:3300/0 conn(0x7f1eac15d250 0x7f1eac0c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:48.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.595+0000 7f1eab7fe640 1 -- 172.21.15.76:0/2129265237 >> v2:172.21.15.113:3300/0 conn(0x7f1eac15d250 msgr2=0x7f1eac0c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:48.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.595+0000 7f1eab7fe640 1 --2- 172.21.15.76:0/2129265237 >> v2:172.21.15.113:3300/0 conn(0x7f1eac15d250 0x7f1eac0c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:48.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.595+0000 7f1eab7fe640 1 -- 172.21.15.76:0/2129265237 >> v1:172.21.15.113:6789/0 conn(0x7f1eac0c9d70 legacy=0x7f1eac0ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:48.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.595+0000 7f1eab7fe640 1 -- 172.21.15.76:0/2129265237 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eac06a370 con 0x7f1eac15ce70 2023-11-09T03:45:48.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.595+0000 7f1eaaffd640 1 --2- 172.21.15.76:0/2129265237 >> v2:172.21.15.113:3300/0 conn(0x7f1eac15d250 0x7f1eac0c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:48.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.596+0000 7f1eab7fe640 1 --2- 172.21.15.76:0/2129265237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1eac15ce70 0x7f1eac06d9b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1eac0d02c0 tx=0x7f1e9c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683df874580ee0b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:48.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.596+0000 7f1eaa7fc640 1 -- 172.21.15.76:0/2129265237 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c030860 con 0x7f1eac15ce70 2023-11-09T03:45:48.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.596+0000 7f1eaa7fc640 1 -- 172.21.15.76:0/2129265237 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9c0309c0 con 0x7f1eac15ce70 2023-11-09T03:45:48.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.596+0000 7f1eaa7fc640 1 -- 172.21.15.76:0/2129265237 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c030d50 con 0x7f1eac15ce70 2023-11-09T03:45:48.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.597+0000 7f1eb1b7c640 1 -- 172.21.15.76:0/2129265237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1eac15ce70 msgr2=0x7f1eac06d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:48.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.597+0000 7f1eb1b7c640 1 --2- 172.21.15.76:0/2129265237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1eac15ce70 0x7f1eac06d9b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1eac0d02c0 tx=0x7f1e9c02b9a0 comp rx=0 tx=0).stop 2023-11-09T03:45:48.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.597+0000 7f1eb1b7c640 1 -- 172.21.15.76:0/2129265237 shutdown_connections 2023-11-09T03:45:48.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.597+0000 7f1eb1b7c640 1 --2- 172.21.15.76:0/2129265237 >> v2:172.21.15.113:3300/0 conn(0x7f1eac15d250 0x7f1eac0c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:48.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.597+0000 7f1eb1b7c640 1 --2- 172.21.15.76:0/2129265237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1eac15ce70 0x7f1eac06d9b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:48.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.597+0000 7f1eb1b7c640 1 -- 172.21.15.76:0/2129265237 >> 172.21.15.76:0/2129265237 conn(0x7f1eac0b0cd0 msgr2=0x7f1eac0b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:48.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.597+0000 7f1eb1b7c640 1 -- 172.21.15.76:0/2129265237 shutdown_connections 2023-11-09T03:45:48.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.597+0000 7f1eb1b7c640 1 -- 172.21.15.76:0/2129265237 wait complete. 2023-11-09T03:45:48.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.598+0000 7f1eb1b7c640 1 Processor -- start 2023-11-09T03:45:48.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.598+0000 7f1eb1b7c640 1 -- start start 2023-11-09T03:45:48.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.599+0000 7f1eb1b7c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1eac15ce70 0x7f1eac155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:48.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.599+0000 7f1eb1b7c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eac15d250 0x7f1eac1f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:48.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.599+0000 7f1eb1b7c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1eac1f2cf0 0x7f1eac1e2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:48.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.599+0000 7f1eb1b7c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eac1e3150 con 0x7f1eac1f2cf0 2023-11-09T03:45:48.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.599+0000 7f1eb1b7c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eac1573c0 con 0x7f1eac15d250 2023-11-09T03:45:48.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.599+0000 7f1eb1b7c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eac1576a0 con 0x7f1eac15ce70 2023-11-09T03:45:48.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.599+0000 7f1eabfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1eac1f2cf0 0x7f1eac1e2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:48.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.599+0000 7f1eabfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1eac1f2cf0 0x7f1eac1e2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57572/0 (socket says 172.21.15.76:57572) 2023-11-09T03:45:48.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.599+0000 7f1eabfff640 1 -- 172.21.15.76:0/280053865 learned_addr learned my addr 172.21.15.76:0/280053865 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:48.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.599+0000 7f1eab7fe640 1 --2- 172.21.15.76:0/280053865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1eac15ce70 0x7f1eac155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:48.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.599+0000 7f1eaaffd640 1 --2- 172.21.15.76:0/280053865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eac15d250 0x7f1eac1f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:48.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.599+0000 7f1eabfff640 1 -- 172.21.15.76:0/280053865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1eac15ce70 msgr2=0x7f1eac155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:48.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.599+0000 7f1eabfff640 1 --2- 172.21.15.76:0/280053865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1eac15ce70 0x7f1eac155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:48.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.599+0000 7f1eabfff640 1 -- 172.21.15.76:0/280053865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eac15d250 msgr2=0x7f1eac1f27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:48.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.599+0000 7f1eabfff640 1 --2- 172.21.15.76:0/280053865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eac15d250 0x7f1eac1f27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:48.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.599+0000 7f1eabfff640 1 -- 172.21.15.76:0/280053865 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e9c0094d0 con 0x7f1eac1f2cf0 2023-11-09T03:45:48.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.599+0000 7f1eabfff640 1 --2- 172.21.15.76:0/280053865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1eac1f2cf0 0x7f1eac1e2360 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400c730 tx=0x7f1ea400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:48.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.600+0000 7f1e93fff640 1 -- 172.21.15.76:0/280053865 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4003b70 con 0x7f1eac1f2cf0 2023-11-09T03:45:48.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.600+0000 7f1e93fff640 1 -- 172.21.15.76:0/280053865 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea4003cd0 con 0x7f1eac1f2cf0 2023-11-09T03:45:48.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.600+0000 7f1eb1b7c640 1 -- 172.21.15.76:0/280053865 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eac157980 con 0x7f1eac1f2cf0 2023-11-09T03:45:48.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.600+0000 7f1e93fff640 1 -- 172.21.15.76:0/280053865 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea400aa90 con 0x7f1eac1f2cf0 2023-11-09T03:45:48.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.600+0000 7f1eb1b7c640 1 -- 172.21.15.76:0/280053865 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eac157f40 con 0x7f1eac1f2cf0 2023-11-09T03:45:48.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.601+0000 7f1eb1b7c640 1 -- 172.21.15.76:0/280053865 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eac074290 con 0x7f1eac1f2cf0 2023-11-09T03:45:48.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.601+0000 7f1e93fff640 1 -- 172.21.15.76:0/280053865 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1ea400abf0 con 0x7f1eac1f2cf0 2023-11-09T03:45:48.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.601+0000 7f1e93fff640 1 --2- 172.21.15.76:0/280053865 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1e9806ae70 0x7f1e9806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:48.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.601+0000 7f1e93fff640 1 -- 172.21.15.76:0/280053865 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1ea40895c0 con 0x7f1eac1f2cf0 2023-11-09T03:45:48.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.602+0000 7f1eab7fe640 1 --2- 172.21.15.76:0/280053865 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1e9806ae70 0x7f1e9806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:48.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.602+0000 7f1eab7fe640 1 --2- 172.21.15.76:0/280053865 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1e9806ae70 0x7f1e9806d330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c036000 tx=0x7f1e9c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:48.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.606+0000 7f1e93fff640 1 -- 172.21.15.76:0/280053865 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1ea4054430 con 0x7f1eac1f2cf0 2023-11-09T03:45:48.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.808+0000 7f1eb1b7c640 1 -- 172.21.15.76:0/280053865 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1eac155ef0 con 0x7f1eac1f2cf0 2023-11-09T03:45:48.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.809+0000 7f1e93fff640 1 -- 172.21.15.76:0/280053865 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4915 (secure 0 0 0) 0x7f1ea4057a50 con 0x7f1eac1f2cf0 2023-11-09T03:45:48.809 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:48.810 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2686298435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2686298435},{"type":"v1","addr":"172.21.15.76:6833","nonce":2686298435}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2132717123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2132717123},{"type":"v1","addr":"172.21.15.76:6835","nonce":2132717123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4281,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2369499936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2369499936},{"type":"v1","addr":"172.21.15.113:6837","nonce":2369499936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:45:30.784155+0000","modified":"2023-11-09T03:45:47.804242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/3293223710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3293223710},{"type":"v1","addr":"172.21.15.113:6836","nonce":3293223710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-09T03:45:48.812 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 9 2023-11-09T03:45:48.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.811+0000 7f1eb1b7c640 1 -- 172.21.15.76:0/280053865 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1e9806ae70 msgr2=0x7f1e9806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:48.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.811+0000 7f1eb1b7c640 1 --2- 172.21.15.76:0/280053865 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1e9806ae70 0x7f1e9806d330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c036000 tx=0x7f1e9c008030 comp rx=0 tx=0).stop 2023-11-09T03:45:48.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.811+0000 7f1eb1b7c640 1 -- 172.21.15.76:0/280053865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1eac1f2cf0 msgr2=0x7f1eac1e2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:48.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.811+0000 7f1eb1b7c640 1 --2- 172.21.15.76:0/280053865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1eac1f2cf0 0x7f1eac1e2360 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400c730 tx=0x7f1ea400cbf0 comp rx=0 tx=0).stop 2023-11-09T03:45:48.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.811+0000 7f1eb1b7c640 1 -- 172.21.15.76:0/280053865 shutdown_connections 2023-11-09T03:45:48.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.811+0000 7f1eb1b7c640 1 --2- 172.21.15.76:0/280053865 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1e9806ae70 0x7f1e9806d330 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:48.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.811+0000 7f1eb1b7c640 1 --2- 172.21.15.76:0/280053865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1eac1f2cf0 0x7f1eac1e2360 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:48.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.811+0000 7f1eb1b7c640 1 --2- 172.21.15.76:0/280053865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eac15d250 0x7f1eac1f27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:48.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.811+0000 7f1eb1b7c640 1 --2- 172.21.15.76:0/280053865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1eac15ce70 0x7f1eac155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:48.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.811+0000 7f1eb1b7c640 1 -- 172.21.15.76:0/280053865 >> 172.21.15.76:0/280053865 conn(0x7f1eac0b0cd0 msgr2=0x7f1eac0ccd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:48.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.811+0000 7f1eb1b7c640 1 -- 172.21.15.76:0/280053865 shutdown_connections 2023-11-09T03:45:48.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.811+0000 7f1eb1b7c640 1 -- 172.21.15.76:0/280053865 wait complete. 2023-11-09T03:45:48.821 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:45:48.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.922+0000 7f69a5bc9640 1 Processor -- start 2023-11-09T03:45:48.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.922+0000 7f69a5bc9640 1 -- start start 2023-11-09T03:45:48.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.923+0000 7f69a5bc9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69a015ce70 0x7f69a006d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:48.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.923+0000 7f69a5bc9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69a006a120 con 0x7f69a00cb5c0 2023-11-09T03:45:48.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.923+0000 7f69a5bc9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69a006def0 con 0x7f69a015d250 2023-11-09T03:45:48.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.923+0000 7f69a5bc9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69a00cdc10 con 0x7f69a015ce70 2023-11-09T03:45:48.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.923+0000 7f699ffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f69a00cb5c0 0x7f69a00ccdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:56720/0 (socket says 172.21.15.76:56720) 2023-11-09T03:45:48.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.923+0000 7f699ffff640 1 -- 172.21.15.76:0/1072512937 learned_addr learned my addr 172.21.15.76:0/1072512937 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:48.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.923+0000 7f699e7fc640 1 -- 172.21.15.76:0/1072512937 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3266603909 0 0) 0x7f69a006a120 con 0x7f69a00cb5c0 2023-11-09T03:45:48.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.924+0000 7f699e7fc640 1 -- 172.21.15.76:0/1072512937 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6988003520 con 0x7f69a00cb5c0 2023-11-09T03:45:48.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.924+0000 7f699f7fe640 1 --2- 172.21.15.76:0/1072512937 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69a015ce70 0x7f69a006d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:48.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.924+0000 7f699e7fc640 1 -- 172.21.15.76:0/1072512937 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1816397234 0 0) 0x7f6988003520 con 0x7f69a00cb5c0 2023-11-09T03:45:48.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.924+0000 7f699e7fc640 1 -- 172.21.15.76:0/1072512937 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69a015ce70 msgr2=0x7f69a006d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:48.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.924+0000 7f699e7fc640 1 --2- 172.21.15.76:0/1072512937 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69a015ce70 0x7f69a006d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:48.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.924+0000 7f699e7fc640 1 -- 172.21.15.76:0/1072512937 >> v1:172.21.15.113:6789/0 conn(0x7f69a015d250 legacy=0x7f69a00c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:48.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.924+0000 7f699e7fc640 1 -- 172.21.15.76:0/1072512937 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a00c9f40 con 0x7f69a00cb5c0 2023-11-09T03:45:48.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.924+0000 7f699e7fc640 1 -- 172.21.15.76:0/1072512937 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f69980019f0 con 0x7f69a00cb5c0 2023-11-09T03:45:48.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.924+0000 7f699e7fc640 1 -- 172.21.15.76:0/1072512937 >> v1:172.21.15.76:6789/0 conn(0x7f69a00cb5c0 legacy=0x7f69a00ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:48.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.924+0000 7f699e7fc640 1 --2- 172.21.15.76:0/1072512937 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69a015ce70 0x7f69880039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:48.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.924+0000 7f699e7fc640 1 --2- 172.21.15.76:0/1072512937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6988003fd0 0x7f69880047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:48.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.924+0000 7f699e7fc640 1 --2- 172.21.15.76:0/1072512937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6988004d00 0x7f6988005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:48.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.924+0000 7f699e7fc640 1 -- 172.21.15.76:0/1072512937 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a00cdc10 con 0x7f6988003fd0 2023-11-09T03:45:48.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.924+0000 7f699e7fc640 1 -- 172.21.15.76:0/1072512937 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698800bb40 con 0x7f6988004d00 2023-11-09T03:45:48.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.924+0000 7f699e7fc640 1 -- 172.21.15.76:0/1072512937 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f698800bd00 con 0x7f69a015ce70 2023-11-09T03:45:48.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.924+0000 7f699effd640 1 --2- 172.21.15.76:0/1072512937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6988003fd0 0x7f69880047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:48.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.924+0000 7f699f7fe640 1 --2- 172.21.15.76:0/1072512937 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69a015ce70 0x7f69880039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:48.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.924+0000 7f699ffff640 1 --2- 172.21.15.76:0/1072512937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6988004d00 0x7f6988005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:48.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.924+0000 7f699effd640 1 -- 172.21.15.76:0/1072512937 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69a015ce70 msgr2=0x7f69880039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:48.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.924+0000 7f699effd640 1 --2- 172.21.15.76:0/1072512937 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69a015ce70 0x7f69880039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:48.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.924+0000 7f699effd640 1 -- 172.21.15.76:0/1072512937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6988004d00 msgr2=0x7f6988005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:48.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.924+0000 7f699effd640 1 --2- 172.21.15.76:0/1072512937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6988004d00 0x7f6988005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:48.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.924+0000 7f699effd640 1 -- 172.21.15.76:0/1072512937 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6988003520 con 0x7f6988003fd0 2023-11-09T03:45:48.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.925+0000 7f699effd640 1 --2- 172.21.15.76:0/1072512937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6988003fd0 0x7f69880047c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6994001720 tx=0x7f69880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596600cd89e6c633 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:48.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.925+0000 7f699e7fc640 1 -- 172.21.15.76:0/1072512937 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69940052d0 con 0x7f6988003fd0 2023-11-09T03:45:48.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.925+0000 7f699e7fc640 1 -- 172.21.15.76:0/1072512937 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f699402e050 con 0x7f6988003fd0 2023-11-09T03:45:48.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.925+0000 7f699e7fc640 1 -- 172.21.15.76:0/1072512937 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6994003830 con 0x7f6988003fd0 2023-11-09T03:45:48.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.925+0000 7f69a5bc9640 1 -- 172.21.15.76:0/1072512937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6988003fd0 msgr2=0x7f69880047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:48.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.925+0000 7f69a5bc9640 1 --2- 172.21.15.76:0/1072512937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6988003fd0 0x7f69880047c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6994001720 tx=0x7f69880057f0 comp rx=0 tx=0).stop 2023-11-09T03:45:48.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.925+0000 7f699ffff640 1 -- 172.21.15.76:0/1072512937 reap_dead start 2023-11-09T03:45:48.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.926+0000 7f69a5bc9640 1 -- 172.21.15.76:0/1072512937 shutdown_connections 2023-11-09T03:45:48.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.926+0000 7f69a5bc9640 1 -- 172.21.15.76:0/1072512937 >> 172.21.15.76:0/1072512937 conn(0x7f69a00b0cd0 msgr2=0x7f69a00b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:48.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.926+0000 7f69a5bc9640 1 -- 172.21.15.76:0/1072512937 shutdown_connections 2023-11-09T03:45:48.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.926+0000 7f69a5bc9640 1 -- 172.21.15.76:0/1072512937 wait complete. 2023-11-09T03:45:48.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.927+0000 7f69a5bc9640 1 Processor -- start 2023-11-09T03:45:48.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.927+0000 7f69a5bc9640 1 -- start start 2023-11-09T03:45:48.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.928+0000 7f69a5bc9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6988003fd0 0x7f69a014af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:48.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.928+0000 7f69a5bc9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69a014c330 0x7f69a0149620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:48.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.928+0000 7f69a5bc9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69a014b920 0x7f69a00c5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:48.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.928+0000 7f69a5bc9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a014ab60 con 0x7f69a014c330 2023-11-09T03:45:48.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.928+0000 7f69a5bc9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a014b4c0 con 0x7f69a014b920 2023-11-09T03:45:48.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.928+0000 7f69a5bc9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69a0149ee0 con 0x7f6988003fd0 2023-11-09T03:45:48.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.928+0000 7f699effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69a014c330 0x7f69a0149620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:48.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.928+0000 7f699effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69a014c330 0x7f69a0149620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57590/0 (socket says 172.21.15.76:57590) 2023-11-09T03:45:48.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.928+0000 7f699effd640 1 -- 172.21.15.76:0/1689198700 learned_addr learned my addr 172.21.15.76:0/1689198700 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:48.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.928+0000 7f699f7fe640 1 --2- 172.21.15.76:0/1689198700 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6988003fd0 0x7f69a014af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:48.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.928+0000 7f699ffff640 1 --2- 172.21.15.76:0/1689198700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69a014b920 0x7f69a00c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:48.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.928+0000 7f699effd640 1 -- 172.21.15.76:0/1689198700 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6988003fd0 msgr2=0x7f69a014af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:48.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.928+0000 7f699effd640 1 --2- 172.21.15.76:0/1689198700 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6988003fd0 0x7f69a014af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:48.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.928+0000 7f699effd640 1 -- 172.21.15.76:0/1689198700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69a014b920 msgr2=0x7f69a00c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:48.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.928+0000 7f699effd640 1 --2- 172.21.15.76:0/1689198700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69a014b920 0x7f69a00c5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:48.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.928+0000 7f699effd640 1 -- 172.21.15.76:0/1689198700 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a014a1c0 con 0x7f69a014c330 2023-11-09T03:45:48.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.928+0000 7f699effd640 1 --2- 172.21.15.76:0/1689198700 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69a014c330 0x7f69a0149620 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6994005040 tx=0x7f6994001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:48.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.928+0000 7f6987fff640 1 -- 172.21.15.76:0/1689198700 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6994007640 con 0x7f69a014c330 2023-11-09T03:45:48.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.928+0000 7f6987fff640 1 -- 172.21.15.76:0/1689198700 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69940069e0 con 0x7f69a014c330 2023-11-09T03:45:48.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.928+0000 7f69a5bc9640 1 -- 172.21.15.76:0/1689198700 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69a014a4a0 con 0x7f69a014c330 2023-11-09T03:45:48.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.928+0000 7f6987fff640 1 -- 172.21.15.76:0/1689198700 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6994006d90 con 0x7f69a014c330 2023-11-09T03:45:48.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.928+0000 7f69a5bc9640 1 -- 172.21.15.76:0/1689198700 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69a0149b60 con 0x7f69a014c330 2023-11-09T03:45:48.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.929+0000 7f69a5bc9640 1 -- 172.21.15.76:0/1689198700 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69a0074290 con 0x7f69a014c330 2023-11-09T03:45:48.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.930+0000 7f6987fff640 1 -- 172.21.15.76:0/1689198700 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f69940035f0 con 0x7f69a014c330 2023-11-09T03:45:48.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.930+0000 7f6987fff640 1 --2- 172.21.15.76:0/1689198700 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f698c06ae70 0x7f698c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:48.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.930+0000 7f6987fff640 1 -- 172.21.15.76:0/1689198700 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f69940ab910 con 0x7f69a014c330 2023-11-09T03:45:48.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.930+0000 7f699f7fe640 1 --2- 172.21.15.76:0/1689198700 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f698c06ae70 0x7f698c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:48.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.931+0000 7f699f7fe640 1 --2- 172.21.15.76:0/1689198700 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f698c06ae70 0x7f698c06d330 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6990006fd0 tx=0x7f6990008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:48.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:48.935+0000 7f6987fff640 1 -- 172.21.15.76:0/1689198700 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f69940767f0 con 0x7f69a014c330 2023-11-09T03:45:49.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.102+0000 7f69a5bc9640 1 -- 172.21.15.76:0/1689198700 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69a0066170 con 0x7f69a014c330 2023-11-09T03:45:49.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.103+0000 7f6987fff640 1 -- 172.21.15.76:0/1689198700 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18334 (secure 0 0 0) 0x7f6994079e10 con 0x7f69a014c330 2023-11-09T03:45:49.102 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:49.103 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":25,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:45:35.781752+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:45:49.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.105+0000 7f69a5bc9640 1 -- 172.21.15.76:0/1689198700 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f698c06ae70 msgr2=0x7f698c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:49.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.105+0000 7f69a5bc9640 1 --2- 172.21.15.76:0/1689198700 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f698c06ae70 0x7f698c06d330 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6990006fd0 tx=0x7f6990008040 comp rx=0 tx=0).stop 2023-11-09T03:45:49.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.105+0000 7f69a5bc9640 1 -- 172.21.15.76:0/1689198700 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69a014c330 msgr2=0x7f69a0149620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:49.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.105+0000 7f69a5bc9640 1 --2- 172.21.15.76:0/1689198700 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69a014c330 0x7f69a0149620 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6994005040 tx=0x7f6994001120 comp rx=0 tx=0).stop 2023-11-09T03:45:49.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.105+0000 7f69a5bc9640 1 -- 172.21.15.76:0/1689198700 shutdown_connections 2023-11-09T03:45:49.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.105+0000 7f69a5bc9640 1 --2- 172.21.15.76:0/1689198700 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f698c06ae70 0x7f698c06d330 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.105+0000 7f69a5bc9640 1 --2- 172.21.15.76:0/1689198700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69a014b920 0x7f69a00c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.105+0000 7f69a5bc9640 1 --2- 172.21.15.76:0/1689198700 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69a014c330 0x7f69a0149620 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.105+0000 7f69a5bc9640 1 --2- 172.21.15.76:0/1689198700 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6988003fd0 0x7f69a014af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.105+0000 7f69a5bc9640 1 -- 172.21.15.76:0/1689198700 >> 172.21.15.76:0/1689198700 conn(0x7f69a00b0cd0 msgr2=0x7f69a00abe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:49.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.105+0000 7f69a5bc9640 1 -- 172.21.15.76:0/1689198700 shutdown_connections 2023-11-09T03:45:49.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.105+0000 7f69a5bc9640 1 -- 172.21.15.76:0/1689198700 wait complete. 2023-11-09T03:45:49.115 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-09T03:45:49.115 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-09T03:45:49.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.217+0000 7fc541c32640 1 Processor -- start 2023-11-09T03:45:49.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.217+0000 7fc541c32640 1 -- start start 2023-11-09T03:45:49.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.218+0000 7fc541c32640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc53c15ab70 0x7fc53c0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:49.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.218+0000 7fc541c32640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc53c06eba0 0x7fc53c06a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:49.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.218+0000 7fc541c32640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc53c06aa70 con 0x7fc53c06a660 2023-11-09T03:45:49.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.218+0000 7fc541c32640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc53c0cba70 con 0x7fc53c15ab70 2023-11-09T03:45:49.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.218+0000 7fc541c32640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc53c161c50 con 0x7fc53c06eba0 2023-11-09T03:45:49.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.218+0000 7fc53b7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc53c15ab70 0x7fc53c0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:49.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.218+0000 7fc53b7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc53c15ab70 0x7fc53c0cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57606/0 (socket says 172.21.15.76:57606) 2023-11-09T03:45:49.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.218+0000 7fc53b7fe640 1 -- 172.21.15.76:0/1411552284 learned_addr learned my addr 172.21.15.76:0/1411552284 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:49.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.218+0000 7fc53affd640 1 --2- 172.21.15.76:0/1411552284 >> v2:172.21.15.113:3300/0 conn(0x7fc53c06eba0 0x7fc53c06a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:49.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.218+0000 7fc53b7fe640 1 -- 172.21.15.76:0/1411552284 >> v2:172.21.15.113:3300/0 conn(0x7fc53c06eba0 msgr2=0x7fc53c06a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:49.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.218+0000 7fc53b7fe640 1 --2- 172.21.15.76:0/1411552284 >> v2:172.21.15.113:3300/0 conn(0x7fc53c06eba0 0x7fc53c06a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.218+0000 7fc53b7fe640 1 -- 172.21.15.76:0/1411552284 >> v1:172.21.15.113:6789/0 conn(0x7fc53c06a660 legacy=0x7fc53c0ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:49.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.218+0000 7fc53b7fe640 1 -- 172.21.15.76:0/1411552284 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc53c06d310 con 0x7fc53c15ab70 2023-11-09T03:45:49.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.218+0000 7fc53affd640 1 --2- 172.21.15.76:0/1411552284 >> v2:172.21.15.113:3300/0 conn(0x7fc53c06eba0 0x7fc53c06a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:49.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.219+0000 7fc53b7fe640 1 --2- 172.21.15.76:0/1411552284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc53c15ab70 0x7fc53c0cbea0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc53c0726b0 tx=0x7fc52c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c6fa54bfa03dac server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:49.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.219+0000 7fc53a7fc640 1 -- 172.21.15.76:0/1411552284 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c004230 con 0x7fc53c15ab70 2023-11-09T03:45:49.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.219+0000 7fc53a7fc640 1 -- 172.21.15.76:0/1411552284 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52c004390 con 0x7fc53c15ab70 2023-11-09T03:45:49.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.219+0000 7fc53a7fc640 1 -- 172.21.15.76:0/1411552284 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c003810 con 0x7fc53c15ab70 2023-11-09T03:45:49.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.219+0000 7fc541c32640 1 -- 172.21.15.76:0/1411552284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc53c15ab70 msgr2=0x7fc53c0cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:49.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.219+0000 7fc541c32640 1 --2- 172.21.15.76:0/1411552284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc53c15ab70 0x7fc53c0cbea0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc53c0726b0 tx=0x7fc52c02b8f0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.219+0000 7fc541c32640 1 -- 172.21.15.76:0/1411552284 shutdown_connections 2023-11-09T03:45:49.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.219+0000 7fc541c32640 1 --2- 172.21.15.76:0/1411552284 >> v2:172.21.15.113:3300/0 conn(0x7fc53c06eba0 0x7fc53c06a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.219+0000 7fc541c32640 1 --2- 172.21.15.76:0/1411552284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc53c15ab70 0x7fc53c0cbea0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.219+0000 7fc541c32640 1 -- 172.21.15.76:0/1411552284 >> 172.21.15.76:0/1411552284 conn(0x7fc53c098d40 msgr2=0x7fc53c0cfa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:49.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.219+0000 7fc541c32640 1 -- 172.21.15.76:0/1411552284 shutdown_connections 2023-11-09T03:45:49.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.219+0000 7fc541c32640 1 -- 172.21.15.76:0/1411552284 wait complete. 2023-11-09T03:45:49.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.221+0000 7fc541c32640 1 Processor -- start 2023-11-09T03:45:49.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.221+0000 7fc541c32640 1 -- start start 2023-11-09T03:45:49.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.221+0000 7fc541c32640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc53c06a660 0x7fc53c0c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:49.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.222+0000 7fc541c32640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc53c06eba0 0x7fc53c09ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:49.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.222+0000 7fc541c32640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc53c15ab70 0x7fc53c09cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:49.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.222+0000 7fc541c32640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53c09d5f0 con 0x7fc53c06a660 2023-11-09T03:45:49.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.222+0000 7fc541c32640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53c0b2730 con 0x7fc53c06eba0 2023-11-09T03:45:49.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.222+0000 7fc541c32640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc53c0b2a10 con 0x7fc53c15ab70 2023-11-09T03:45:49.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.222+0000 7fc53b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc53c06a660 0x7fc53c0c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:49.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.222+0000 7fc53b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc53c06a660 0x7fc53c0c5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57616/0 (socket says 172.21.15.76:57616) 2023-11-09T03:45:49.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.222+0000 7fc53b7fe640 1 -- 172.21.15.76:0/2319598143 learned_addr learned my addr 172.21.15.76:0/2319598143 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:49.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.222+0000 7fc53affd640 1 --2- 172.21.15.76:0/2319598143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc53c06eba0 0x7fc53c09ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:49.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.222+0000 7fc53bfff640 1 --2- 172.21.15.76:0/2319598143 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc53c15ab70 0x7fc53c09cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:49.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.222+0000 7fc53b7fe640 1 -- 172.21.15.76:0/2319598143 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc53c15ab70 msgr2=0x7fc53c09cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:49.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.222+0000 7fc53b7fe640 1 --2- 172.21.15.76:0/2319598143 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc53c15ab70 0x7fc53c09cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.222+0000 7fc53b7fe640 1 -- 172.21.15.76:0/2319598143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc53c06eba0 msgr2=0x7fc53c09ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:49.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.222+0000 7fc53b7fe640 1 --2- 172.21.15.76:0/2319598143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc53c06eba0 0x7fc53c09ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.222+0000 7fc53b7fe640 1 -- 172.21.15.76:0/2319598143 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52c009460 con 0x7fc53c06a660 2023-11-09T03:45:49.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.222+0000 7fc53b7fe640 1 --2- 172.21.15.76:0/2319598143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc53c06a660 0x7fc53c0c5ec0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc52c000c00 tx=0x7fc52c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:49.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.222+0000 7fc523fff640 1 -- 172.21.15.76:0/2319598143 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c003c40 con 0x7fc53c06a660 2023-11-09T03:45:49.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.222+0000 7fc523fff640 1 -- 172.21.15.76:0/2319598143 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52c03f070 con 0x7fc53c06a660 2023-11-09T03:45:49.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.222+0000 7fc541c32640 1 -- 172.21.15.76:0/2319598143 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc53c0b2cf0 con 0x7fc53c06a660 2023-11-09T03:45:49.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.222+0000 7fc523fff640 1 -- 172.21.15.76:0/2319598143 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c004500 con 0x7fc53c06a660 2023-11-09T03:45:49.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.222+0000 7fc541c32640 1 -- 172.21.15.76:0/2319598143 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc53c14ed00 con 0x7fc53c06a660 2023-11-09T03:45:49.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.223+0000 7fc541c32640 1 -- 172.21.15.76:0/2319598143 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc53c0cd590 con 0x7fc53c06a660 2023-11-09T03:45:49.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.224+0000 7fc523fff640 1 -- 172.21.15.76:0/2319598143 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc52c041030 con 0x7fc53c06a660 2023-11-09T03:45:49.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.224+0000 7fc523fff640 1 --2- 172.21.15.76:0/2319598143 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc52806ae70 0x7fc52806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:49.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.224+0000 7fc523fff640 1 -- 172.21.15.76:0/2319598143 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc52c0abb20 con 0x7fc53c06a660 2023-11-09T03:45:49.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.224+0000 7fc53affd640 1 --2- 172.21.15.76:0/2319598143 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc52806ae70 0x7fc52806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:49.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.225+0000 7fc53affd640 1 --2- 172.21.15.76:0/2319598143 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc52806ae70 0x7fc52806d330 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc53c15fb10 tx=0x7fc530008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:49.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.228+0000 7fc523fff640 1 -- 172.21.15.76:0/2319598143 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc52c076970 con 0x7fc53c06a660 2023-11-09T03:45:49.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.393+0000 7fc541c32640 1 -- 172.21.15.76:0/2319598143 --> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc53c15d860 con 0x7fc52806ae70 2023-11-09T03:45:49.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.402+0000 7fc523fff640 1 -- 172.21.15.76:0/2319598143 <== mgr.4105 v2:172.21.15.113:6832/1264193042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345303 (secure 0 0 0) 0x7fc53000c570 con 0x7fc52806ae70 2023-11-09T03:45:49.405 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:49.408 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2023-11-09T03:45:49.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.406+0000 7fc541c32640 1 -- 172.21.15.76:0/2319598143 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc52806ae70 msgr2=0x7fc52806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:49.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.406+0000 7fc541c32640 1 --2- 172.21.15.76:0/2319598143 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc52806ae70 0x7fc52806d330 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc53c15fb10 tx=0x7fc530008040 comp rx=0 tx=0).stop 2023-11-09T03:45:49.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.406+0000 7fc541c32640 1 -- 172.21.15.76:0/2319598143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc53c06a660 msgr2=0x7fc53c0c5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:49.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.406+0000 7fc541c32640 1 --2- 172.21.15.76:0/2319598143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc53c06a660 0x7fc53c0c5ec0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc52c000c00 tx=0x7fc52c0046c0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.406+0000 7fc541c32640 1 -- 172.21.15.76:0/2319598143 shutdown_connections 2023-11-09T03:45:49.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.406+0000 7fc541c32640 1 --2- 172.21.15.76:0/2319598143 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc52806ae70 0x7fc52806d330 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.406+0000 7fc541c32640 1 --2- 172.21.15.76:0/2319598143 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc53c15ab70 0x7fc53c09cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.406+0000 7fc541c32640 1 --2- 172.21.15.76:0/2319598143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc53c06eba0 0x7fc53c09ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.406+0000 7fc541c32640 1 --2- 172.21.15.76:0/2319598143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc53c06a660 0x7fc53c0c5ec0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.406+0000 7fc541c32640 1 -- 172.21.15.76:0/2319598143 >> 172.21.15.76:0/2319598143 conn(0x7fc53c098d40 msgr2=0x7fc53c06e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:49.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.406+0000 7fc541c32640 1 -- 172.21.15.76:0/2319598143 shutdown_connections 2023-11-09T03:45:49.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.406+0000 7fc541c32640 1 -- 172.21.15.76:0/2319598143 wait complete. 2023-11-09T03:45:49.416 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2023-11-09T03:45:48.752233+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219468,"kb_used_data":3300,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755252,"statfs":{"total":773094113280,"available":772869378048,"internally_reserved":0,"allocated":3379200,"data_stored":2025099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2229,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003169"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.444993+0000","last_change":"2023-11-09T03:45:30.408658+0000","last_active":"2023-11-09T03:45:35.444993+0000","last_peered":"2023-11-09T03:45:35.444993+0000","last_clean":"2023-11-09T03:45:35.444993+0000","last_became_active":"2023-11-09T03:45:30.408345+0000","last_became_peered":"2023-11-09T03:45:30.408345+0000","last_unstale":"2023-11-09T03:45:35.444993+0000","last_undegraded":"2023-11-09T03:45:35.444993+0000","last_fullsized":"2023-11-09T03:45:35.444993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:12:07.086030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440545+0000","last_change":"2023-11-09T03:45:31.403012+0000","last_active":"2023-11-09T03:45:35.440545+0000","last_peered":"2023-11-09T03:45:35.440545+0000","last_clean":"2023-11-09T03:45:35.440545+0000","last_became_active":"2023-11-09T03:45:31.402623+0000","last_became_peered":"2023-11-09T03:45:31.402623+0000","last_unstale":"2023-11-09T03:45:35.440545+0000","last_undegraded":"2023-11-09T03:45:35.440545+0000","last_fullsized":"2023-11-09T03:45:35.440545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:36:23.593484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388073+0000","last_change":"2023-11-09T03:45:35.620447+0000","last_active":"2023-11-09T03:45:36.388073+0000","last_peered":"2023-11-09T03:45:36.388073+0000","last_clean":"2023-11-09T03:45:36.388073+0000","last_became_active":"2023-11-09T03:45:35.619949+0000","last_became_peered":"2023-11-09T03:45:35.619949+0000","last_unstale":"2023-11-09T03:45:36.388073+0000","last_undegraded":"2023-11-09T03:45:36.388073+0000","last_fullsized":"2023-11-09T03:45:36.388073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:13:00.499438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440637+0000","last_change":"2023-11-09T03:45:30.411660+0000","last_active":"2023-11-09T03:45:35.440637+0000","last_peered":"2023-11-09T03:45:35.440637+0000","last_clean":"2023-11-09T03:45:35.440637+0000","last_became_active":"2023-11-09T03:45:30.411274+0000","last_became_peered":"2023-11-09T03:45:30.411274+0000","last_unstale":"2023-11-09T03:45:35.440637+0000","last_undegraded":"2023-11-09T03:45:35.440637+0000","last_fullsized":"2023-11-09T03:45:35.440637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:39:29.323066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.444755+0000","last_change":"2023-11-09T03:45:31.415835+0000","last_active":"2023-11-09T03:45:35.444755+0000","last_peered":"2023-11-09T03:45:35.444755+0000","last_clean":"2023-11-09T03:45:35.444755+0000","last_became_active":"2023-11-09T03:45:31.415571+0000","last_became_peered":"2023-11-09T03:45:31.415571+0000","last_unstale":"2023-11-09T03:45:35.444755+0000","last_undegraded":"2023-11-09T03:45:35.444755+0000","last_fullsized":"2023-11-09T03:45:35.444755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:13:29.896024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388822+0000","last_change":"2023-11-09T03:45:35.445871+0000","last_active":"2023-11-09T03:45:36.388822+0000","last_peered":"2023-11-09T03:45:36.388822+0000","last_clean":"2023-11-09T03:45:36.388822+0000","last_became_active":"2023-11-09T03:45:35.445390+0000","last_became_peered":"2023-11-09T03:45:35.445390+0000","last_unstale":"2023-11-09T03:45:36.388822+0000","last_undegraded":"2023-11-09T03:45:36.388822+0000","last_fullsized":"2023-11-09T03:45:36.388822+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:26.681331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617937+0000","last_change":"2023-11-09T03:45:30.416124+0000","last_active":"2023-11-09T03:45:35.617937+0000","last_peered":"2023-11-09T03:45:35.617937+0000","last_clean":"2023-11-09T03:45:35.617937+0000","last_became_active":"2023-11-09T03:45:30.415850+0000","last_became_peered":"2023-11-09T03:45:30.415850+0000","last_unstale":"2023-11-09T03:45:35.617937+0000","last_undegraded":"2023-11-09T03:45:35.617937+0000","last_fullsized":"2023-11-09T03:45:35.617937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:43.468677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441010+0000","last_change":"2023-11-09T03:45:31.413149+0000","last_active":"2023-11-09T03:45:35.441010+0000","last_peered":"2023-11-09T03:45:35.441010+0000","last_clean":"2023-11-09T03:45:35.441010+0000","last_became_active":"2023-11-09T03:45:31.412875+0000","last_became_peered":"2023-11-09T03:45:31.412875+0000","last_unstale":"2023-11-09T03:45:35.441010+0000","last_undegraded":"2023-11-09T03:45:35.441010+0000","last_fullsized":"2023-11-09T03:45:35.441010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:04:02.978744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.389160+0000","last_change":"2023-11-09T03:45:35.444039+0000","last_active":"2023-11-09T03:45:36.389160+0000","last_peered":"2023-11-09T03:45:36.389160+0000","last_clean":"2023-11-09T03:45:36.389160+0000","last_became_active":"2023-11-09T03:45:35.443646+0000","last_became_peered":"2023-11-09T03:45:35.443646+0000","last_unstale":"2023-11-09T03:45:36.389160+0000","last_undegraded":"2023-11-09T03:45:36.389160+0000","last_fullsized":"2023-11-09T03:45:36.389160+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:02:31.124051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617462+0000","last_change":"2023-11-09T03:45:30.418418+0000","last_active":"2023-11-09T03:45:35.617462+0000","last_peered":"2023-11-09T03:45:35.617462+0000","last_clean":"2023-11-09T03:45:35.617462+0000","last_became_active":"2023-11-09T03:45:30.417963+0000","last_became_peered":"2023-11-09T03:45:30.417963+0000","last_unstale":"2023-11-09T03:45:35.617462+0000","last_undegraded":"2023-11-09T03:45:35.617462+0000","last_fullsized":"2023-11-09T03:45:35.617462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:07.347819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439849+0000","last_change":"2023-11-09T03:45:31.416138+0000","last_active":"2023-11-09T03:45:35.439849+0000","last_peered":"2023-11-09T03:45:35.439849+0000","last_clean":"2023-11-09T03:45:35.439849+0000","last_became_active":"2023-11-09T03:45:31.415636+0000","last_became_peered":"2023-11-09T03:45:31.415636+0000","last_unstale":"2023-11-09T03:45:35.439849+0000","last_undegraded":"2023-11-09T03:45:35.439849+0000","last_fullsized":"2023-11-09T03:45:35.439849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:02:27.095864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.389010+0000","last_change":"2023-11-09T03:45:35.448265+0000","last_active":"2023-11-09T03:45:36.389010+0000","last_peered":"2023-11-09T03:45:36.389010+0000","last_clean":"2023-11-09T03:45:36.389010+0000","last_became_active":"2023-11-09T03:45:35.447747+0000","last_became_peered":"2023-11-09T03:45:35.447747+0000","last_unstale":"2023-11-09T03:45:36.389010+0000","last_undegraded":"2023-11-09T03:45:36.389010+0000","last_fullsized":"2023-11-09T03:45:36.389010+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:39:46.409150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.437921+0000","last_change":"2023-11-09T03:45:31.400950+0000","last_active":"2023-11-09T03:45:35.437921+0000","last_peered":"2023-11-09T03:45:35.437921+0000","last_clean":"2023-11-09T03:45:35.437921+0000","last_became_active":"2023-11-09T03:45:31.400667+0000","last_became_peered":"2023-11-09T03:45:31.400667+0000","last_unstale":"2023-11-09T03:45:35.437921+0000","last_undegraded":"2023-11-09T03:45:35.437921+0000","last_fullsized":"2023-11-09T03:45:35.437921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:44.473424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617461+0000","last_change":"2023-11-09T03:45:30.418728+0000","last_active":"2023-11-09T03:45:35.617461+0000","last_peered":"2023-11-09T03:45:35.617461+0000","last_clean":"2023-11-09T03:45:35.617461+0000","last_became_active":"2023-11-09T03:45:30.417939+0000","last_became_peered":"2023-11-09T03:45:30.417939+0000","last_unstale":"2023-11-09T03:45:35.617461+0000","last_undegraded":"2023-11-09T03:45:35.617461+0000","last_fullsized":"2023-11-09T03:45:35.617461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:35:45.408209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.389355+0000","last_change":"2023-11-09T03:45:35.448352+0000","last_active":"2023-11-09T03:45:36.389355+0000","last_peered":"2023-11-09T03:45:36.389355+0000","last_clean":"2023-11-09T03:45:36.389355+0000","last_became_active":"2023-11-09T03:45:35.447889+0000","last_became_peered":"2023-11-09T03:45:35.447889+0000","last_unstale":"2023-11-09T03:45:36.389355+0000","last_undegraded":"2023-11-09T03:45:36.389355+0000","last_fullsized":"2023-11-09T03:45:36.389355+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:39:40.769191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439502+0000","last_change":"2023-11-09T03:45:31.398106+0000","last_active":"2023-11-09T03:45:35.439502+0000","last_peered":"2023-11-09T03:45:35.439502+0000","last_clean":"2023-11-09T03:45:35.439502+0000","last_became_active":"2023-11-09T03:45:31.397712+0000","last_became_peered":"2023-11-09T03:45:31.397712+0000","last_unstale":"2023-11-09T03:45:35.439502+0000","last_undegraded":"2023-11-09T03:45:35.439502+0000","last_fullsized":"2023-11-09T03:45:35.439502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:04:02.978744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441816+0000","last_change":"2023-11-09T03:45:30.408462+0000","last_active":"2023-11-09T03:45:35.441816+0000","last_peered":"2023-11-09T03:45:35.441816+0000","last_clean":"2023-11-09T03:45:35.441816+0000","last_became_active":"2023-11-09T03:45:30.408190+0000","last_became_peered":"2023-11-09T03:45:30.408190+0000","last_unstale":"2023-11-09T03:45:35.441816+0000","last_undegraded":"2023-11-09T03:45:35.441816+0000","last_fullsized":"2023-11-09T03:45:35.441816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:11:40.438915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.389721+0000","last_change":"2023-11-09T03:45:35.445519+0000","last_active":"2023-11-09T03:45:36.389721+0000","last_peered":"2023-11-09T03:45:36.389721+0000","last_clean":"2023-11-09T03:45:36.389721+0000","last_became_active":"2023-11-09T03:45:35.445125+0000","last_became_peered":"2023-11-09T03:45:35.445125+0000","last_unstale":"2023-11-09T03:45:36.389721+0000","last_undegraded":"2023-11-09T03:45:36.389721+0000","last_fullsized":"2023-11-09T03:45:36.389721+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:45.477938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439030+0000","last_change":"2023-11-09T03:45:31.402204+0000","last_active":"2023-11-09T03:45:35.439030+0000","last_peered":"2023-11-09T03:45:35.439030+0000","last_clean":"2023-11-09T03:45:35.439030+0000","last_became_active":"2023-11-09T03:45:31.401312+0000","last_became_peered":"2023-11-09T03:45:31.401312+0000","last_unstale":"2023-11-09T03:45:35.439030+0000","last_undegraded":"2023-11-09T03:45:35.439030+0000","last_fullsized":"2023-11-09T03:45:35.439030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:48:08.925647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441918+0000","last_change":"2023-11-09T03:45:30.414258+0000","last_active":"2023-11-09T03:45:35.441918+0000","last_peered":"2023-11-09T03:45:35.441918+0000","last_clean":"2023-11-09T03:45:35.441918+0000","last_became_active":"2023-11-09T03:45:30.413994+0000","last_became_peered":"2023-11-09T03:45:30.413994+0000","last_unstale":"2023-11-09T03:45:35.441918+0000","last_undegraded":"2023-11-09T03:45:35.441918+0000","last_fullsized":"2023-11-09T03:45:35.441918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:12:07.086030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.263008+0000","last_change":"2023-11-09T03:45:35.620757+0000","last_active":"2023-11-09T03:45:37.263008+0000","last_peered":"2023-11-09T03:45:37.263008+0000","last_clean":"2023-11-09T03:45:37.263008+0000","last_became_active":"2023-11-09T03:45:35.620219+0000","last_became_peered":"2023-11-09T03:45:35.620219+0000","last_unstale":"2023-11-09T03:45:37.263008+0000","last_undegraded":"2023-11-09T03:45:37.263008+0000","last_fullsized":"2023-11-09T03:45:37.263008+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:41:12.579051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.436441+0000","last_change":"2023-11-09T03:45:31.396755+0000","last_active":"2023-11-09T03:45:35.436441+0000","last_peered":"2023-11-09T03:45:35.436441+0000","last_clean":"2023-11-09T03:45:35.436441+0000","last_became_active":"2023-11-09T03:45:31.396475+0000","last_became_peered":"2023-11-09T03:45:31.396475+0000","last_unstale":"2023-11-09T03:45:35.436441+0000","last_undegraded":"2023-11-09T03:45:35.436441+0000","last_fullsized":"2023-11-09T03:45:35.436441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:07:02.262322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617133+0000","last_change":"2023-11-09T03:45:30.419151+0000","last_active":"2023-11-09T03:45:35.617133+0000","last_peered":"2023-11-09T03:45:35.617133+0000","last_clean":"2023-11-09T03:45:35.617133+0000","last_became_active":"2023-11-09T03:45:30.418874+0000","last_became_peered":"2023-11-09T03:45:30.418874+0000","last_unstale":"2023-11-09T03:45:35.617133+0000","last_undegraded":"2023-11-09T03:45:35.617133+0000","last_fullsized":"2023-11-09T03:45:35.617133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:34:48.400703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.389351+0000","last_change":"2023-11-09T03:45:35.445655+0000","last_active":"2023-11-09T03:45:36.389351+0000","last_peered":"2023-11-09T03:45:36.389351+0000","last_clean":"2023-11-09T03:45:36.389351+0000","last_became_active":"2023-11-09T03:45:35.445246+0000","last_became_peered":"2023-11-09T03:45:35.445246+0000","last_unstale":"2023-11-09T03:45:36.389351+0000","last_undegraded":"2023-11-09T03:45:36.389351+0000","last_fullsized":"2023-11-09T03:45:36.389351+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:04:07.006932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.443777+0000","last_change":"2023-11-09T03:45:31.397453+0000","last_active":"2023-11-09T03:45:35.443777+0000","last_peered":"2023-11-09T03:45:35.443777+0000","last_clean":"2023-11-09T03:45:35.443777+0000","last_became_active":"2023-11-09T03:45:31.396892+0000","last_became_peered":"2023-11-09T03:45:31.396892+0000","last_unstale":"2023-11-09T03:45:35.443777+0000","last_undegraded":"2023-11-09T03:45:35.443777+0000","last_fullsized":"2023-11-09T03:45:35.443777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:44.473424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440439+0000","last_change":"2023-11-09T03:45:30.416814+0000","last_active":"2023-11-09T03:45:35.440439+0000","last_peered":"2023-11-09T03:45:35.440439+0000","last_clean":"2023-11-09T03:45:35.440439+0000","last_became_active":"2023-11-09T03:45:30.416446+0000","last_became_peered":"2023-11-09T03:45:30.416446+0000","last_unstale":"2023-11-09T03:45:35.440439+0000","last_undegraded":"2023-11-09T03:45:35.440439+0000","last_fullsized":"2023-11-09T03:45:35.440439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:37:54.581788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388988+0000","last_change":"2023-11-09T03:45:35.618893+0000","last_active":"2023-11-09T03:45:36.388988+0000","last_peered":"2023-11-09T03:45:36.388988+0000","last_clean":"2023-11-09T03:45:36.388988+0000","last_became_active":"2023-11-09T03:45:35.618506+0000","last_became_peered":"2023-11-09T03:45:35.618506+0000","last_unstale":"2023-11-09T03:45:36.388988+0000","last_undegraded":"2023-11-09T03:45:36.388988+0000","last_fullsized":"2023-11-09T03:45:36.388988+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:36:27.621672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438491+0000","last_change":"2023-11-09T03:45:31.406885+0000","last_active":"2023-11-09T03:45:35.438491+0000","last_peered":"2023-11-09T03:45:35.438491+0000","last_clean":"2023-11-09T03:45:35.438491+0000","last_became_active":"2023-11-09T03:45:31.406620+0000","last_became_peered":"2023-11-09T03:45:31.406620+0000","last_unstale":"2023-11-09T03:45:35.438491+0000","last_undegraded":"2023-11-09T03:45:35.438491+0000","last_fullsized":"2023-11-09T03:45:35.438491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:29:17.503411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442020+0000","last_change":"2023-11-09T03:45:30.412534+0000","last_active":"2023-11-09T03:45:35.442020+0000","last_peered":"2023-11-09T03:45:35.442020+0000","last_clean":"2023-11-09T03:45:35.442020+0000","last_became_active":"2023-11-09T03:45:30.412015+0000","last_became_peered":"2023-11-09T03:45:30.412015+0000","last_unstale":"2023-11-09T03:45:35.442020+0000","last_undegraded":"2023-11-09T03:45:35.442020+0000","last_fullsized":"2023-11-09T03:45:35.442020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:34:48.400703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.387965+0000","last_change":"2023-11-09T03:45:35.620972+0000","last_active":"2023-11-09T03:45:36.387965+0000","last_peered":"2023-11-09T03:45:36.387965+0000","last_clean":"2023-11-09T03:45:36.387965+0000","last_became_active":"2023-11-09T03:45:35.620626+0000","last_became_peered":"2023-11-09T03:45:35.620626+0000","last_unstale":"2023-11-09T03:45:36.387965+0000","last_undegraded":"2023-11-09T03:45:36.387965+0000","last_fullsized":"2023-11-09T03:45:36.387965+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:03:57.174574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441966+0000","last_change":"2023-11-09T03:45:31.393932+0000","last_active":"2023-11-09T03:45:35.441966+0000","last_peered":"2023-11-09T03:45:35.441966+0000","last_clean":"2023-11-09T03:45:35.441966+0000","last_became_active":"2023-11-09T03:45:31.393669+0000","last_became_peered":"2023-11-09T03:45:31.393669+0000","last_unstale":"2023-11-09T03:45:35.441966+0000","last_undegraded":"2023-11-09T03:45:35.441966+0000","last_fullsized":"2023-11-09T03:45:35.441966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:27.400068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.618527+0000","last_change":"2023-11-09T03:45:30.419637+0000","last_active":"2023-11-09T03:45:35.618527+0000","last_peered":"2023-11-09T03:45:35.618527+0000","last_clean":"2023-11-09T03:45:35.618527+0000","last_became_active":"2023-11-09T03:45:30.419348+0000","last_became_peered":"2023-11-09T03:45:30.419348+0000","last_unstale":"2023-11-09T03:45:35.618527+0000","last_undegraded":"2023-11-09T03:45:35.618527+0000","last_fullsized":"2023-11-09T03:45:35.618527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:48:07.920901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.261912+0000","last_change":"2023-11-09T03:45:35.619361+0000","last_active":"2023-11-09T03:45:37.261912+0000","last_peered":"2023-11-09T03:45:37.261912+0000","last_clean":"2023-11-09T03:45:37.261912+0000","last_became_active":"2023-11-09T03:45:35.618998+0000","last_became_peered":"2023-11-09T03:45:35.618998+0000","last_unstale":"2023-11-09T03:45:37.261912+0000","last_undegraded":"2023-11-09T03:45:37.261912+0000","last_fullsized":"2023-11-09T03:45:37.261912+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:29:21.531598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439868+0000","last_change":"2023-11-09T03:45:31.408700+0000","last_active":"2023-11-09T03:45:35.439868+0000","last_peered":"2023-11-09T03:45:35.439868+0000","last_clean":"2023-11-09T03:45:35.439868+0000","last_became_active":"2023-11-09T03:45:31.408153+0000","last_became_peered":"2023-11-09T03:45:31.408153+0000","last_unstale":"2023-11-09T03:45:35.439868+0000","last_undegraded":"2023-11-09T03:45:35.439868+0000","last_fullsized":"2023-11-09T03:45:35.439868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:29:17.503411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442042+0000","last_change":"2023-11-09T03:45:30.409443+0000","last_active":"2023-11-09T03:45:35.442042+0000","last_peered":"2023-11-09T03:45:35.442042+0000","last_clean":"2023-11-09T03:45:35.442042+0000","last_became_active":"2023-11-09T03:45:30.409175+0000","last_became_peered":"2023-11-09T03:45:30.409175+0000","last_unstale":"2023-11-09T03:45:35.442042+0000","last_undegraded":"2023-11-09T03:45:35.442042+0000","last_fullsized":"2023-11-09T03:45:35.442042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:34:27.335423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262894+0000","last_change":"2023-11-09T03:45:35.452625+0000","last_active":"2023-11-09T03:45:37.262894+0000","last_peered":"2023-11-09T03:45:37.262894+0000","last_clean":"2023-11-09T03:45:37.262894+0000","last_became_active":"2023-11-09T03:45:35.451772+0000","last_became_peered":"2023-11-09T03:45:35.451772+0000","last_unstale":"2023-11-09T03:45:37.262894+0000","last_undegraded":"2023-11-09T03:45:37.262894+0000","last_fullsized":"2023-11-09T03:45:37.262894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:45.477938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.435623+0000","last_change":"2023-11-09T03:45:31.399046+0000","last_active":"2023-11-09T03:45:35.435623+0000","last_peered":"2023-11-09T03:45:35.435623+0000","last_clean":"2023-11-09T03:45:35.435623+0000","last_became_active":"2023-11-09T03:45:31.398698+0000","last_became_peered":"2023-11-09T03:45:31.398698+0000","last_unstale":"2023-11-09T03:45:35.435623+0000","last_undegraded":"2023-11-09T03:45:35.435623+0000","last_fullsized":"2023-11-09T03:45:35.435623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:35:46.412956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440345+0000","last_change":"2023-11-09T03:45:30.414519+0000","last_active":"2023-11-09T03:45:35.440345+0000","last_peered":"2023-11-09T03:45:35.440345+0000","last_clean":"2023-11-09T03:45:35.440345+0000","last_became_active":"2023-11-09T03:45:30.414025+0000","last_became_peered":"2023-11-09T03:45:30.414025+0000","last_unstale":"2023-11-09T03:45:35.440345+0000","last_undegraded":"2023-11-09T03:45:35.440345+0000","last_fullsized":"2023-11-09T03:45:35.440345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:43.468677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262508+0000","last_change":"2023-11-09T03:45:35.451076+0000","last_active":"2023-11-09T03:45:37.262508+0000","last_peered":"2023-11-09T03:45:37.262508+0000","last_clean":"2023-11-09T03:45:37.262508+0000","last_became_active":"2023-11-09T03:45:35.450721+0000","last_became_peered":"2023-11-09T03:45:35.450721+0000","last_unstale":"2023-11-09T03:45:37.262508+0000","last_undegraded":"2023-11-09T03:45:37.262508+0000","last_fullsized":"2023-11-09T03:45:37.262508+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:29:21.531598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.443831+0000","last_change":"2023-11-09T03:45:31.397098+0000","last_active":"2023-11-09T03:45:35.443831+0000","last_peered":"2023-11-09T03:45:35.443831+0000","last_clean":"2023-11-09T03:45:35.443831+0000","last_became_active":"2023-11-09T03:45:31.396835+0000","last_became_peered":"2023-11-09T03:45:31.396835+0000","last_unstale":"2023-11-09T03:45:35.443831+0000","last_undegraded":"2023-11-09T03:45:35.443831+0000","last_fullsized":"2023-11-09T03:45:35.443831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:54:30.809126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.432187+0000","last_change":"2023-11-09T03:45:30.411878+0000","last_active":"2023-11-09T03:45:35.432187+0000","last_peered":"2023-11-09T03:45:35.432187+0000","last_clean":"2023-11-09T03:45:35.432187+0000","last_became_active":"2023-11-09T03:45:30.411615+0000","last_became_peered":"2023-11-09T03:45:30.411615+0000","last_unstale":"2023-11-09T03:45:35.432187+0000","last_undegraded":"2023-11-09T03:45:35.432187+0000","last_fullsized":"2023-11-09T03:45:35.432187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:11:40.438915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.263008+0000","last_change":"2023-11-09T03:45:35.451994+0000","last_active":"2023-11-09T03:45:37.263008+0000","last_peered":"2023-11-09T03:45:37.263008+0000","last_clean":"2023-11-09T03:45:37.263008+0000","last_became_active":"2023-11-09T03:45:35.451623+0000","last_became_peered":"2023-11-09T03:45:35.451623+0000","last_unstale":"2023-11-09T03:45:37.263008+0000","last_undegraded":"2023-11-09T03:45:37.263008+0000","last_fullsized":"2023-11-09T03:45:37.263008+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:13:08.339968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441086+0000","last_change":"2023-11-09T03:45:31.400475+0000","last_active":"2023-11-09T03:45:35.441086+0000","last_peered":"2023-11-09T03:45:35.441086+0000","last_clean":"2023-11-09T03:45:35.441086+0000","last_became_active":"2023-11-09T03:45:31.400021+0000","last_became_peered":"2023-11-09T03:45:31.400021+0000","last_unstale":"2023-11-09T03:45:35.441086+0000","last_undegraded":"2023-11-09T03:45:35.441086+0000","last_fullsized":"2023-11-09T03:45:35.441086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:11:10.514126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617133+0000","last_change":"2023-11-09T03:45:30.417099+0000","last_active":"2023-11-09T03:45:35.617133+0000","last_peered":"2023-11-09T03:45:35.617133+0000","last_clean":"2023-11-09T03:45:35.617133+0000","last_became_active":"2023-11-09T03:45:30.416779+0000","last_became_peered":"2023-11-09T03:45:30.416779+0000","last_unstale":"2023-11-09T03:45:35.617133+0000","last_undegraded":"2023-11-09T03:45:35.617133+0000","last_fullsized":"2023-11-09T03:45:35.617133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:06:25.266825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262776+0000","last_change":"2023-11-09T03:45:35.618887+0000","last_active":"2023-11-09T03:45:37.262776+0000","last_peered":"2023-11-09T03:45:37.262776+0000","last_clean":"2023-11-09T03:45:37.262776+0000","last_became_active":"2023-11-09T03:45:35.618406+0000","last_became_peered":"2023-11-09T03:45:35.618406+0000","last_unstale":"2023-11-09T03:45:37.262776+0000","last_undegraded":"2023-11-09T03:45:37.262776+0000","last_fullsized":"2023-11-09T03:45:37.262776+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:41:12.579051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438270+0000","last_change":"2023-11-09T03:45:31.416419+0000","last_active":"2023-11-09T03:45:35.438270+0000","last_peered":"2023-11-09T03:45:35.438270+0000","last_clean":"2023-11-09T03:45:35.438270+0000","last_became_active":"2023-11-09T03:45:31.416137+0000","last_became_peered":"2023-11-09T03:45:31.416137+0000","last_unstale":"2023-11-09T03:45:35.438270+0000","last_undegraded":"2023-11-09T03:45:35.438270+0000","last_fullsized":"2023-11-09T03:45:35.438270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:08.352566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439728+0000","last_change":"2023-11-09T03:45:30.417920+0000","last_active":"2023-11-09T03:45:35.439728+0000","last_peered":"2023-11-09T03:45:35.439728+0000","last_clean":"2023-11-09T03:45:35.439728+0000","last_became_active":"2023-11-09T03:45:30.417578+0000","last_became_peered":"2023-11-09T03:45:30.417578+0000","last_unstale":"2023-11-09T03:45:35.439728+0000","last_undegraded":"2023-11-09T03:45:35.439728+0000","last_fullsized":"2023-11-09T03:45:35.439728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:07:01.257576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617231+0000","last_change":"2023-11-09T03:45:21.030926+0000","last_active":"2023-11-09T03:45:35.617231+0000","last_peered":"2023-11-09T03:45:35.617231+0000","last_clean":"2023-11-09T03:45:35.617231+0000","last_became_active":"2023-11-09T03:45:21.029801+0000","last_became_peered":"2023-11-09T03:45:21.029801+0000","last_unstale":"2023-11-09T03:45:35.617231+0000","last_undegraded":"2023-11-09T03:45:35.617231+0000","last_fullsized":"2023-11-09T03:45:35.617231+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_clean_scrub_stamp":"2023-11-09T03:45:19.768433+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:12:59.153713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.264013+0000","last_change":"2023-11-09T03:45:35.622614+0000","last_active":"2023-11-09T03:45:37.264013+0000","last_peered":"2023-11-09T03:45:37.264013+0000","last_clean":"2023-11-09T03:45:37.264013+0000","last_became_active":"2023-11-09T03:45:35.622138+0000","last_became_peered":"2023-11-09T03:45:35.622138+0000","last_unstale":"2023-11-09T03:45:37.264013+0000","last_undegraded":"2023-11-09T03:45:37.264013+0000","last_fullsized":"2023-11-09T03:45:37.264013+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:29:21.531598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617667+0000","last_change":"2023-11-09T03:45:31.417640+0000","last_active":"2023-11-09T03:45:35.617667+0000","last_peered":"2023-11-09T03:45:35.617667+0000","last_clean":"2023-11-09T03:45:35.617667+0000","last_became_active":"2023-11-09T03:45:31.417210+0000","last_became_peered":"2023-11-09T03:45:31.417210+0000","last_unstale":"2023-11-09T03:45:35.617667+0000","last_undegraded":"2023-11-09T03:45:35.617667+0000","last_fullsized":"2023-11-09T03:45:35.617667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:41.449750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.432324+0000","last_change":"2023-11-09T03:45:30.418193+0000","last_active":"2023-11-09T03:45:35.432324+0000","last_peered":"2023-11-09T03:45:35.432324+0000","last_clean":"2023-11-09T03:45:35.432324+0000","last_became_active":"2023-11-09T03:45:30.417874+0000","last_became_peered":"2023-11-09T03:45:30.417874+0000","last_unstale":"2023-11-09T03:45:35.432324+0000","last_undegraded":"2023-11-09T03:45:35.432324+0000","last_fullsized":"2023-11-09T03:45:35.432324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:34:27.335423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.263620+0000","last_change":"2023-11-09T03:45:35.618898+0000","last_active":"2023-11-09T03:45:37.263620+0000","last_peered":"2023-11-09T03:45:37.263620+0000","last_clean":"2023-11-09T03:45:37.263620+0000","last_became_active":"2023-11-09T03:45:35.618502+0000","last_became_peered":"2023-11-09T03:45:35.618502+0000","last_unstale":"2023-11-09T03:45:37.263620+0000","last_undegraded":"2023-11-09T03:45:37.263620+0000","last_fullsized":"2023-11-09T03:45:37.263620+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:39:40.769191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441100+0000","last_change":"2023-11-09T03:45:31.409910+0000","last_active":"2023-11-09T03:45:35.441100+0000","last_peered":"2023-11-09T03:45:35.441100+0000","last_clean":"2023-11-09T03:45:35.441100+0000","last_became_active":"2023-11-09T03:45:31.408863+0000","last_became_peered":"2023-11-09T03:45:31.408863+0000","last_unstale":"2023-11-09T03:45:35.441100+0000","last_undegraded":"2023-11-09T03:45:35.441100+0000","last_fullsized":"2023-11-09T03:45:35.441100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:41.449750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442413+0000","last_change":"2023-11-09T03:45:30.412199+0000","last_active":"2023-11-09T03:45:35.442413+0000","last_peered":"2023-11-09T03:45:35.442413+0000","last_clean":"2023-11-09T03:45:35.442413+0000","last_became_active":"2023-11-09T03:45:30.411864+0000","last_became_peered":"2023-11-09T03:45:30.411864+0000","last_unstale":"2023-11-09T03:45:35.442413+0000","last_undegraded":"2023-11-09T03:45:35.442413+0000","last_fullsized":"2023-11-09T03:45:35.442413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:13:28.891277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262128+0000","last_change":"2023-11-09T03:45:35.456250+0000","last_active":"2023-11-09T03:45:37.262128+0000","last_peered":"2023-11-09T03:45:37.262128+0000","last_clean":"2023-11-09T03:45:37.262128+0000","last_became_active":"2023-11-09T03:45:35.455866+0000","last_became_peered":"2023-11-09T03:45:35.455866+0000","last_unstale":"2023-11-09T03:45:37.262128+0000","last_undegraded":"2023-11-09T03:45:37.262128+0000","last_fullsized":"2023-11-09T03:45:37.262128+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:45.477938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.618920+0000","last_change":"2023-11-09T03:45:31.420110+0000","last_active":"2023-11-09T03:45:35.618920+0000","last_peered":"2023-11-09T03:45:35.618920+0000","last_clean":"2023-11-09T03:45:35.618920+0000","last_became_active":"2023-11-09T03:45:31.419556+0000","last_became_peered":"2023-11-09T03:45:31.419556+0000","last_unstale":"2023-11-09T03:45:35.618920+0000","last_undegraded":"2023-11-09T03:45:35.618920+0000","last_fullsized":"2023-11-09T03:45:35.618920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:51:24.601367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.432312+0000","last_change":"2023-11-09T03:45:30.415461+0000","last_active":"2023-11-09T03:45:35.432312+0000","last_peered":"2023-11-09T03:45:35.432312+0000","last_clean":"2023-11-09T03:45:35.432312+0000","last_became_active":"2023-11-09T03:45:30.415185+0000","last_became_peered":"2023-11-09T03:45:30.415185+0000","last_unstale":"2023-11-09T03:45:35.432312+0000","last_undegraded":"2023-11-09T03:45:35.432312+0000","last_fullsized":"2023-11-09T03:45:35.432312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:07:01.257576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262941+0000","last_change":"2023-11-09T03:45:35.620742+0000","last_active":"2023-11-09T03:45:37.262941+0000","last_peered":"2023-11-09T03:45:37.262941+0000","last_clean":"2023-11-09T03:45:37.262941+0000","last_became_active":"2023-11-09T03:45:35.620350+0000","last_became_peered":"2023-11-09T03:45:35.620350+0000","last_unstale":"2023-11-09T03:45:37.262941+0000","last_undegraded":"2023-11-09T03:45:37.262941+0000","last_fullsized":"2023-11-09T03:45:37.262941+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:03:57.174574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.618702+0000","last_change":"2023-11-09T03:45:31.417119+0000","last_active":"2023-11-09T03:45:35.618702+0000","last_peered":"2023-11-09T03:45:35.618702+0000","last_clean":"2023-11-09T03:45:35.618702+0000","last_became_active":"2023-11-09T03:45:31.416750+0000","last_became_peered":"2023-11-09T03:45:31.416750+0000","last_unstale":"2023-11-09T03:45:35.618702+0000","last_undegraded":"2023-11-09T03:45:35.618702+0000","last_fullsized":"2023-11-09T03:45:35.618702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:55:29.945168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.436205+0000","last_change":"2023-11-09T03:45:30.410204+0000","last_active":"2023-11-09T03:45:35.436205+0000","last_peered":"2023-11-09T03:45:35.436205+0000","last_clean":"2023-11-09T03:45:35.436205+0000","last_became_active":"2023-11-09T03:45:30.409876+0000","last_became_peered":"2023-11-09T03:45:30.409876+0000","last_unstale":"2023-11-09T03:45:35.436205+0000","last_undegraded":"2023-11-09T03:45:35.436205+0000","last_fullsized":"2023-11-09T03:45:35.436205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:23:49.677277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388781+0000","last_change":"2023-11-09T03:45:35.446608+0000","last_active":"2023-11-09T03:45:36.388781+0000","last_peered":"2023-11-09T03:45:36.388781+0000","last_clean":"2023-11-09T03:45:36.388781+0000","last_became_active":"2023-11-09T03:45:35.446121+0000","last_became_peered":"2023-11-09T03:45:35.446121+0000","last_unstale":"2023-11-09T03:45:36.388781+0000","last_undegraded":"2023-11-09T03:45:36.388781+0000","last_fullsized":"2023-11-09T03:45:36.388781+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:12.380753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617394+0000","last_change":"2023-11-09T03:45:31.418010+0000","last_active":"2023-11-09T03:45:35.617394+0000","last_peered":"2023-11-09T03:45:35.617394+0000","last_clean":"2023-11-09T03:45:35.617394+0000","last_became_active":"2023-11-09T03:45:31.417758+0000","last_became_peered":"2023-11-09T03:45:31.417758+0000","last_unstale":"2023-11-09T03:45:35.617394+0000","last_undegraded":"2023-11-09T03:45:35.617394+0000","last_fullsized":"2023-11-09T03:45:35.617394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:02:27.095864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.432121+0000","last_change":"2023-11-09T03:45:30.412289+0000","last_active":"2023-11-09T03:45:35.432121+0000","last_peered":"2023-11-09T03:45:35.432121+0000","last_clean":"2023-11-09T03:45:35.432121+0000","last_became_active":"2023-11-09T03:45:30.412035+0000","last_became_peered":"2023-11-09T03:45:30.412035+0000","last_unstale":"2023-11-09T03:45:35.432121+0000","last_undegraded":"2023-11-09T03:45:35.432121+0000","last_fullsized":"2023-11-09T03:45:35.432121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:35:45.408209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262660+0000","last_change":"2023-11-09T03:45:35.450569+0000","last_active":"2023-11-09T03:45:37.262660+0000","last_peered":"2023-11-09T03:45:37.262660+0000","last_clean":"2023-11-09T03:45:37.262660+0000","last_became_active":"2023-11-09T03:45:35.450133+0000","last_became_peered":"2023-11-09T03:45:35.450133+0000","last_unstale":"2023-11-09T03:45:37.262660+0000","last_undegraded":"2023-11-09T03:45:37.262660+0000","last_fullsized":"2023-11-09T03:45:37.262660+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:13:08.339968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440213+0000","last_change":"2023-11-09T03:45:31.400690+0000","last_active":"2023-11-09T03:45:35.440213+0000","last_peered":"2023-11-09T03:45:35.440213+0000","last_clean":"2023-11-09T03:45:35.440213+0000","last_became_active":"2023-11-09T03:45:31.400345+0000","last_became_peered":"2023-11-09T03:45:31.400345+0000","last_unstale":"2023-11-09T03:45:35.440213+0000","last_undegraded":"2023-11-09T03:45:35.440213+0000","last_fullsized":"2023-11-09T03:45:35.440213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:54:30.809126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438759+0000","last_change":"2023-11-09T03:45:30.414344+0000","last_active":"2023-11-09T03:45:35.438759+0000","last_peered":"2023-11-09T03:45:35.438759+0000","last_clean":"2023-11-09T03:45:35.438759+0000","last_became_active":"2023-11-09T03:45:30.414070+0000","last_became_peered":"2023-11-09T03:45:30.414070+0000","last_unstale":"2023-11-09T03:45:35.438759+0000","last_undegraded":"2023-11-09T03:45:35.438759+0000","last_fullsized":"2023-11-09T03:45:35.438759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:06:25.266825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388898+0000","last_change":"2023-11-09T03:45:35.618460+0000","last_active":"2023-11-09T03:45:36.388898+0000","last_peered":"2023-11-09T03:45:36.388898+0000","last_clean":"2023-11-09T03:45:36.388898+0000","last_became_active":"2023-11-09T03:45:35.618089+0000","last_became_peered":"2023-11-09T03:45:35.618089+0000","last_unstale":"2023-11-09T03:45:36.388898+0000","last_undegraded":"2023-11-09T03:45:36.388898+0000","last_fullsized":"2023-11-09T03:45:36.388898+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:24:34.808267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617389+0000","last_change":"2023-11-09T03:45:31.418108+0000","last_active":"2023-11-09T03:45:35.617389+0000","last_peered":"2023-11-09T03:45:35.617389+0000","last_clean":"2023-11-09T03:45:35.617389+0000","last_became_active":"2023-11-09T03:45:31.417869+0000","last_became_peered":"2023-11-09T03:45:31.417869+0000","last_unstale":"2023-11-09T03:45:35.617389+0000","last_undegraded":"2023-11-09T03:45:35.617389+0000","last_fullsized":"2023-11-09T03:45:35.617389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:04:02.978744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440273+0000","last_change":"2023-11-09T03:45:30.414815+0000","last_active":"2023-11-09T03:45:35.440273+0000","last_peered":"2023-11-09T03:45:35.440273+0000","last_clean":"2023-11-09T03:45:35.440273+0000","last_became_active":"2023-11-09T03:45:30.414177+0000","last_became_peered":"2023-11-09T03:45:30.414177+0000","last_unstale":"2023-11-09T03:45:35.440273+0000","last_undegraded":"2023-11-09T03:45:35.440273+0000","last_fullsized":"2023-11-09T03:45:35.440273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:54:29.804379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388529+0000","last_change":"2023-11-09T03:45:35.618416+0000","last_active":"2023-11-09T03:45:36.388529+0000","last_peered":"2023-11-09T03:45:36.388529+0000","last_clean":"2023-11-09T03:45:36.388529+0000","last_became_active":"2023-11-09T03:45:35.618055+0000","last_became_peered":"2023-11-09T03:45:35.618055+0000","last_unstale":"2023-11-09T03:45:36.388529+0000","last_undegraded":"2023-11-09T03:45:36.388529+0000","last_fullsized":"2023-11-09T03:45:36.388529+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:16:21.023356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.443358+0000","last_change":"2023-11-09T03:45:31.404777+0000","last_active":"2023-11-09T03:45:35.443358+0000","last_peered":"2023-11-09T03:45:35.443358+0000","last_clean":"2023-11-09T03:45:35.443358+0000","last_became_active":"2023-11-09T03:45:31.404472+0000","last_became_peered":"2023-11-09T03:45:31.404472+0000","last_unstale":"2023-11-09T03:45:35.443358+0000","last_undegraded":"2023-11-09T03:45:35.443358+0000","last_fullsized":"2023-11-09T03:45:35.443358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:06:26.271572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.443117+0000","last_change":"2023-11-09T03:45:30.411464+0000","last_active":"2023-11-09T03:45:35.443117+0000","last_peered":"2023-11-09T03:45:35.443117+0000","last_clean":"2023-11-09T03:45:35.443117+0000","last_became_active":"2023-11-09T03:45:30.410430+0000","last_became_peered":"2023-11-09T03:45:30.410430+0000","last_unstale":"2023-11-09T03:45:35.443117+0000","last_undegraded":"2023-11-09T03:45:35.443117+0000","last_fullsized":"2023-11-09T03:45:35.443117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:48:49.911609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262935+0000","last_change":"2023-11-09T03:45:35.618861+0000","last_active":"2023-11-09T03:45:37.262935+0000","last_peered":"2023-11-09T03:45:37.262935+0000","last_clean":"2023-11-09T03:45:37.262935+0000","last_became_active":"2023-11-09T03:45:35.618394+0000","last_became_peered":"2023-11-09T03:45:35.618394+0000","last_unstale":"2023-11-09T03:45:37.262935+0000","last_undegraded":"2023-11-09T03:45:37.262935+0000","last_fullsized":"2023-11-09T03:45:37.262935+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:07:23.630255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617562+0000","last_change":"2023-11-09T03:45:31.407151+0000","last_active":"2023-11-09T03:45:35.617562+0000","last_peered":"2023-11-09T03:45:35.617562+0000","last_clean":"2023-11-09T03:45:35.617562+0000","last_became_active":"2023-11-09T03:45:31.406652+0000","last_became_peered":"2023-11-09T03:45:31.406652+0000","last_unstale":"2023-11-09T03:45:35.617562+0000","last_undegraded":"2023-11-09T03:45:35.617562+0000","last_fullsized":"2023-11-09T03:45:35.617562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:29:17.503411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:41.806449+0000","last_change":"2023-11-09T03:45:30.411676+0000","last_active":"2023-11-09T03:45:41.806449+0000","last_peered":"2023-11-09T03:45:41.806449+0000","last_clean":"2023-11-09T03:45:41.806449+0000","last_became_active":"2023-11-09T03:45:30.411379+0000","last_became_peered":"2023-11-09T03:45:30.411379+0000","last_unstale":"2023-11-09T03:45:41.806449+0000","last_undegraded":"2023-11-09T03:45:41.806449+0000","last_fullsized":"2023-11-09T03:45:41.806449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:34:48.400703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262972+0000","last_change":"2023-11-09T03:45:35.452073+0000","last_active":"2023-11-09T03:45:37.262972+0000","last_peered":"2023-11-09T03:45:37.262972+0000","last_clean":"2023-11-09T03:45:37.262972+0000","last_became_active":"2023-11-09T03:45:35.451714+0000","last_became_peered":"2023-11-09T03:45:35.451714+0000","last_unstale":"2023-11-09T03:45:37.262972+0000","last_undegraded":"2023-11-09T03:45:37.262972+0000","last_fullsized":"2023-11-09T03:45:37.262972+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:55:33.973355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.435954+0000","last_change":"2023-11-09T03:45:31.406706+0000","last_active":"2023-11-09T03:45:35.435954+0000","last_peered":"2023-11-09T03:45:35.435954+0000","last_clean":"2023-11-09T03:45:35.435954+0000","last_became_active":"2023-11-09T03:45:31.406437+0000","last_became_peered":"2023-11-09T03:45:31.406437+0000","last_unstale":"2023-11-09T03:45:35.435954+0000","last_undegraded":"2023-11-09T03:45:35.435954+0000","last_fullsized":"2023-11-09T03:45:35.435954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:48:50.916355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442326+0000","last_change":"2023-11-09T03:45:30.418063+0000","last_active":"2023-11-09T03:45:35.442326+0000","last_peered":"2023-11-09T03:45:35.442326+0000","last_clean":"2023-11-09T03:45:35.442326+0000","last_became_active":"2023-11-09T03:45:30.417792+0000","last_became_peered":"2023-11-09T03:45:30.417792+0000","last_unstale":"2023-11-09T03:45:35.442326+0000","last_undegraded":"2023-11-09T03:45:35.442326+0000","last_fullsized":"2023-11-09T03:45:35.442326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:06:25.266825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.389474+0000","last_change":"2023-11-09T03:45:35.446788+0000","last_active":"2023-11-09T03:45:36.389474+0000","last_peered":"2023-11-09T03:45:36.389474+0000","last_clean":"2023-11-09T03:45:36.389474+0000","last_became_active":"2023-11-09T03:45:35.446280+0000","last_became_peered":"2023-11-09T03:45:35.446280+0000","last_unstale":"2023-11-09T03:45:36.389474+0000","last_undegraded":"2023-11-09T03:45:36.389474+0000","last_fullsized":"2023-11-09T03:45:36.389474+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:29:21.531598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.444838+0000","last_change":"2023-11-09T03:45:31.415653+0000","last_active":"2023-11-09T03:45:35.444838+0000","last_peered":"2023-11-09T03:45:35.444838+0000","last_clean":"2023-11-09T03:45:35.444838+0000","last_became_active":"2023-11-09T03:45:31.415219+0000","last_became_peered":"2023-11-09T03:45:31.415219+0000","last_unstale":"2023-11-09T03:45:35.444838+0000","last_undegraded":"2023-11-09T03:45:35.444838+0000","last_fullsized":"2023-11-09T03:45:35.444838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:37:55.586534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440583+0000","last_change":"2023-11-09T03:45:30.414817+0000","last_active":"2023-11-09T03:45:35.440583+0000","last_peered":"2023-11-09T03:45:35.440583+0000","last_clean":"2023-11-09T03:45:35.440583+0000","last_became_active":"2023-11-09T03:45:30.414124+0000","last_became_peered":"2023-11-09T03:45:30.414124+0000","last_unstale":"2023-11-09T03:45:35.440583+0000","last_undegraded":"2023-11-09T03:45:35.440583+0000","last_fullsized":"2023-11-09T03:45:35.440583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:11:40.438915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388126+0000","last_change":"2023-11-09T03:45:35.448965+0000","last_active":"2023-11-09T03:45:36.388126+0000","last_peered":"2023-11-09T03:45:36.388126+0000","last_clean":"2023-11-09T03:45:36.388126+0000","last_became_active":"2023-11-09T03:45:35.448513+0000","last_became_peered":"2023-11-09T03:45:35.448513+0000","last_unstale":"2023-11-09T03:45:36.388126+0000","last_undegraded":"2023-11-09T03:45:36.388126+0000","last_fullsized":"2023-11-09T03:45:36.388126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:57:39.650773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440495+0000","last_change":"2023-11-09T03:45:31.407514+0000","last_active":"2023-11-09T03:45:35.440495+0000","last_peered":"2023-11-09T03:45:35.440495+0000","last_clean":"2023-11-09T03:45:35.440495+0000","last_became_active":"2023-11-09T03:45:31.407223+0000","last_became_peered":"2023-11-09T03:45:31.407223+0000","last_unstale":"2023-11-09T03:45:35.440495+0000","last_undegraded":"2023-11-09T03:45:35.440495+0000","last_fullsized":"2023-11-09T03:45:35.440495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:55:29.945168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440341+0000","last_change":"2023-11-09T03:45:30.418184+0000","last_active":"2023-11-09T03:45:35.440341+0000","last_peered":"2023-11-09T03:45:35.440341+0000","last_clean":"2023-11-09T03:45:35.440341+0000","last_became_active":"2023-11-09T03:45:30.417910+0000","last_became_peered":"2023-11-09T03:45:30.417910+0000","last_unstale":"2023-11-09T03:45:35.440341+0000","last_undegraded":"2023-11-09T03:45:35.440341+0000","last_fullsized":"2023-11-09T03:45:35.440341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:13:28.891277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388139+0000","last_change":"2023-11-09T03:45:35.621180+0000","last_active":"2023-11-09T03:45:36.388139+0000","last_peered":"2023-11-09T03:45:36.388139+0000","last_clean":"2023-11-09T03:45:36.388139+0000","last_became_active":"2023-11-09T03:45:35.620827+0000","last_became_peered":"2023-11-09T03:45:35.620827+0000","last_unstale":"2023-11-09T03:45:36.388139+0000","last_undegraded":"2023-11-09T03:45:36.388139+0000","last_fullsized":"2023-11-09T03:45:36.388139+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:57:57.109949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262540+0000","last_change":"2023-11-09T03:45:35.620240+0000","last_active":"2023-11-09T03:45:37.262540+0000","last_peered":"2023-11-09T03:45:37.262540+0000","last_clean":"2023-11-09T03:45:37.262540+0000","last_became_active":"2023-11-09T03:45:35.619863+0000","last_became_peered":"2023-11-09T03:45:35.619863+0000","last_unstale":"2023-11-09T03:45:37.262540+0000","last_undegraded":"2023-11-09T03:45:37.262540+0000","last_fullsized":"2023-11-09T03:45:37.262540+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:55:33.973355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441745+0000","last_change":"2023-11-09T03:45:31.399005+0000","last_active":"2023-11-09T03:45:35.441745+0000","last_peered":"2023-11-09T03:45:35.441745+0000","last_clean":"2023-11-09T03:45:35.441745+0000","last_became_active":"2023-11-09T03:45:31.398753+0000","last_became_peered":"2023-11-09T03:45:31.398753+0000","last_unstale":"2023-11-09T03:45:35.441745+0000","last_undegraded":"2023-11-09T03:45:35.441745+0000","last_fullsized":"2023-11-09T03:45:35.441745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:44.473424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439574+0000","last_change":"2023-11-09T03:45:30.414502+0000","last_active":"2023-11-09T03:45:35.439574+0000","last_peered":"2023-11-09T03:45:35.439574+0000","last_clean":"2023-11-09T03:45:35.439574+0000","last_became_active":"2023-11-09T03:45:30.414208+0000","last_became_peered":"2023-11-09T03:45:30.414208+0000","last_unstale":"2023-11-09T03:45:35.439574+0000","last_undegraded":"2023-11-09T03:45:35.439574+0000","last_fullsized":"2023-11-09T03:45:35.439574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:34:48.400703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.264189+0000","last_change":"2023-11-09T03:45:35.456456+0000","last_active":"2023-11-09T03:45:37.264189+0000","last_peered":"2023-11-09T03:45:37.264189+0000","last_clean":"2023-11-09T03:45:37.264189+0000","last_became_active":"2023-11-09T03:45:35.455918+0000","last_became_peered":"2023-11-09T03:45:35.455918+0000","last_unstale":"2023-11-09T03:45:37.264189+0000","last_undegraded":"2023-11-09T03:45:37.264189+0000","last_fullsized":"2023-11-09T03:45:37.264189+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:45.477938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438961+0000","last_change":"2023-11-09T03:45:31.402076+0000","last_active":"2023-11-09T03:45:35.438961+0000","last_peered":"2023-11-09T03:45:35.438961+0000","last_clean":"2023-11-09T03:45:35.438961+0000","last_became_active":"2023-11-09T03:45:31.401819+0000","last_became_peered":"2023-11-09T03:45:31.401819+0000","last_unstale":"2023-11-09T03:45:35.438961+0000","last_undegraded":"2023-11-09T03:45:35.438961+0000","last_fullsized":"2023-11-09T03:45:35.438961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:27.400068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439935+0000","last_change":"2023-11-09T03:45:30.410155+0000","last_active":"2023-11-09T03:45:35.439935+0000","last_peered":"2023-11-09T03:45:35.439935+0000","last_clean":"2023-11-09T03:45:35.439935+0000","last_became_active":"2023-11-09T03:45:30.409732+0000","last_became_peered":"2023-11-09T03:45:30.409732+0000","last_unstale":"2023-11-09T03:45:35.439935+0000","last_undegraded":"2023-11-09T03:45:35.439935+0000","last_fullsized":"2023-11-09T03:45:35.439935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:34:48.400703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438019+0000","last_change":"2023-11-09T03:45:30.414923+0000","last_active":"2023-11-09T03:45:35.438019+0000","last_peered":"2023-11-09T03:45:35.438019+0000","last_clean":"2023-11-09T03:45:35.438019+0000","last_became_active":"2023-11-09T03:45:30.414641+0000","last_became_peered":"2023-11-09T03:45:30.414641+0000","last_unstale":"2023-11-09T03:45:35.438019+0000","last_undegraded":"2023-11-09T03:45:35.438019+0000","last_fullsized":"2023-11-09T03:45:35.438019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:12:07.086030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.443025+0000","last_change":"2023-11-09T03:45:31.400895+0000","last_active":"2023-11-09T03:45:35.443025+0000","last_peered":"2023-11-09T03:45:35.443025+0000","last_clean":"2023-11-09T03:45:35.443025+0000","last_became_active":"2023-11-09T03:45:31.400383+0000","last_became_peered":"2023-11-09T03:45:31.400383+0000","last_unstale":"2023-11-09T03:45:35.443025+0000","last_undegraded":"2023-11-09T03:45:35.443025+0000","last_fullsized":"2023-11-09T03:45:35.443025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:08.352566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.261594+0000","last_change":"2023-11-09T03:45:35.445171+0000","last_active":"2023-11-09T03:45:37.261594+0000","last_peered":"2023-11-09T03:45:37.261594+0000","last_clean":"2023-11-09T03:45:37.261594+0000","last_became_active":"2023-11-09T03:45:35.444835+0000","last_became_peered":"2023-11-09T03:45:35.444835+0000","last_unstale":"2023-11-09T03:45:37.261594+0000","last_undegraded":"2023-11-09T03:45:37.261594+0000","last_fullsized":"2023-11-09T03:45:37.261594+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:13:33.924212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439602+0000","last_change":"2023-11-09T03:45:30.416307+0000","last_active":"2023-11-09T03:45:35.439602+0000","last_peered":"2023-11-09T03:45:35.439602+0000","last_clean":"2023-11-09T03:45:35.439602+0000","last_became_active":"2023-11-09T03:45:30.416012+0000","last_became_peered":"2023-11-09T03:45:30.416012+0000","last_unstale":"2023-11-09T03:45:35.439602+0000","last_undegraded":"2023-11-09T03:45:35.439602+0000","last_fullsized":"2023-11-09T03:45:35.439602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:13:28.891277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.435281+0000","last_change":"2023-11-09T03:45:31.395421+0000","last_active":"2023-11-09T03:45:35.435281+0000","last_peered":"2023-11-09T03:45:35.435281+0000","last_clean":"2023-11-09T03:45:35.435281+0000","last_became_active":"2023-11-09T03:45:31.395148+0000","last_became_peered":"2023-11-09T03:45:31.395148+0000","last_unstale":"2023-11-09T03:45:35.435281+0000","last_undegraded":"2023-11-09T03:45:35.435281+0000","last_fullsized":"2023-11-09T03:45:35.435281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:34:28.340170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388718+0000","last_change":"2023-11-09T03:45:35.454065+0000","last_active":"2023-11-09T03:45:36.388718+0000","last_peered":"2023-11-09T03:45:36.388718+0000","last_clean":"2023-11-09T03:45:36.388718+0000","last_became_active":"2023-11-09T03:45:35.453677+0000","last_became_peered":"2023-11-09T03:45:35.453677+0000","last_unstale":"2023-11-09T03:45:36.388718+0000","last_undegraded":"2023-11-09T03:45:36.388718+0000","last_fullsized":"2023-11-09T03:45:36.388718+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:34:18.265468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440121+0000","last_change":"2023-11-09T03:45:31.401829+0000","last_active":"2023-11-09T03:45:35.440121+0000","last_peered":"2023-11-09T03:45:35.440121+0000","last_clean":"2023-11-09T03:45:35.440121+0000","last_became_active":"2023-11-09T03:45:31.401545+0000","last_became_peered":"2023-11-09T03:45:31.401545+0000","last_unstale":"2023-11-09T03:45:35.440121+0000","last_undegraded":"2023-11-09T03:45:35.440121+0000","last_fullsized":"2023-11-09T03:45:35.440121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:27.400068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.432141+0000","last_change":"2023-11-09T03:45:31.410286+0000","last_active":"2023-11-09T03:45:35.432141+0000","last_peered":"2023-11-09T03:45:35.432141+0000","last_clean":"2023-11-09T03:45:35.432141+0000","last_became_active":"2023-11-09T03:45:31.410031+0000","last_became_peered":"2023-11-09T03:45:31.410031+0000","last_unstale":"2023-11-09T03:45:35.432141+0000","last_undegraded":"2023-11-09T03:45:35.432141+0000","last_fullsized":"2023-11-09T03:45:35.432141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:48:50.916355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440196+0000","last_change":"2023-11-09T03:45:31.412282+0000","last_active":"2023-11-09T03:45:35.440196+0000","last_peered":"2023-11-09T03:45:35.440196+0000","last_clean":"2023-11-09T03:45:35.440196+0000","last_became_active":"2023-11-09T03:45:31.408879+0000","last_became_peered":"2023-11-09T03:45:31.408879+0000","last_unstale":"2023-11-09T03:45:35.440196+0000","last_undegraded":"2023-11-09T03:45:35.440196+0000","last_fullsized":"2023-11-09T03:45:35.440196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:39:42.380963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442948+0000","last_change":"2023-11-09T03:45:31.400563+0000","last_active":"2023-11-09T03:45:35.442948+0000","last_peered":"2023-11-09T03:45:35.442948+0000","last_clean":"2023-11-09T03:45:35.442948+0000","last_became_active":"2023-11-09T03:45:31.399929+0000","last_became_peered":"2023-11-09T03:45:31.399929+0000","last_unstale":"2023-11-09T03:45:35.442948+0000","last_undegraded":"2023-11-09T03:45:35.442948+0000","last_fullsized":"2023-11-09T03:45:35.442948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:48:08.925647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439486+0000","last_change":"2023-11-09T03:45:31.397966+0000","last_active":"2023-11-09T03:45:35.439486+0000","last_peered":"2023-11-09T03:45:35.439486+0000","last_clean":"2023-11-09T03:45:35.439486+0000","last_became_active":"2023-11-09T03:45:31.397681+0000","last_became_peered":"2023-11-09T03:45:31.397681+0000","last_unstale":"2023-11-09T03:45:35.439486+0000","last_undegraded":"2023-11-09T03:45:35.439486+0000","last_fullsized":"2023-11-09T03:45:35.439486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:22.653144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442230+0000","last_change":"2023-11-09T03:45:31.398359+0000","last_active":"2023-11-09T03:45:35.442230+0000","last_peered":"2023-11-09T03:45:35.442230+0000","last_clean":"2023-11-09T03:45:35.442230+0000","last_became_active":"2023-11-09T03:45:31.398104+0000","last_became_peered":"2023-11-09T03:45:31.398104+0000","last_unstale":"2023-11-09T03:45:35.442230+0000","last_undegraded":"2023-11-09T03:45:35.442230+0000","last_fullsized":"2023-11-09T03:45:35.442230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:08.352566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440856+0000","last_change":"2023-11-09T03:45:31.408644+0000","last_active":"2023-11-09T03:45:35.440856+0000","last_peered":"2023-11-09T03:45:35.440856+0000","last_clean":"2023-11-09T03:45:35.440856+0000","last_became_active":"2023-11-09T03:45:31.408353+0000","last_became_peered":"2023-11-09T03:45:31.408353+0000","last_unstale":"2023-11-09T03:45:35.440856+0000","last_undegraded":"2023-11-09T03:45:35.440856+0000","last_fullsized":"2023-11-09T03:45:35.440856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:36:23.593484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438417+0000","last_change":"2023-11-09T03:45:31.395367+0000","last_active":"2023-11-09T03:45:35.438417+0000","last_peered":"2023-11-09T03:45:35.438417+0000","last_clean":"2023-11-09T03:45:35.438417+0000","last_became_active":"2023-11-09T03:45:31.395059+0000","last_became_peered":"2023-11-09T03:45:31.395059+0000","last_unstale":"2023-11-09T03:45:35.438417+0000","last_undegraded":"2023-11-09T03:45:35.438417+0000","last_fullsized":"2023-11-09T03:45:35.438417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:39:36.741003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439489+0000","last_change":"2023-11-09T03:45:31.408626+0000","last_active":"2023-11-09T03:45:35.439489+0000","last_peered":"2023-11-09T03:45:35.439489+0000","last_clean":"2023-11-09T03:45:35.439489+0000","last_became_active":"2023-11-09T03:45:31.408085+0000","last_became_peered":"2023-11-09T03:45:31.408085+0000","last_unstale":"2023-11-09T03:45:35.439489+0000","last_undegraded":"2023-11-09T03:45:35.439489+0000","last_fullsized":"2023-11-09T03:45:35.439489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:08.352566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439241+0000","last_change":"2023-11-09T03:45:31.402017+0000","last_active":"2023-11-09T03:45:35.439241+0000","last_peered":"2023-11-09T03:45:35.439241+0000","last_clean":"2023-11-09T03:45:35.439241+0000","last_became_active":"2023-11-09T03:45:31.401773+0000","last_became_peered":"2023-11-09T03:45:31.401773+0000","last_unstale":"2023-11-09T03:45:35.439241+0000","last_undegraded":"2023-11-09T03:45:35.439241+0000","last_fullsized":"2023-11-09T03:45:35.439241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:39:42.380963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440106+0000","last_change":"2023-11-09T03:45:31.418163+0000","last_active":"2023-11-09T03:45:35.440106+0000","last_peered":"2023-11-09T03:45:35.440106+0000","last_clean":"2023-11-09T03:45:35.440106+0000","last_became_active":"2023-11-09T03:45:31.417880+0000","last_became_peered":"2023-11-09T03:45:31.417880+0000","last_unstale":"2023-11-09T03:45:35.440106+0000","last_undegraded":"2023-11-09T03:45:35.440106+0000","last_fullsized":"2023-11-09T03:45:35.440106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:39:36.741003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442845+0000","last_change":"2023-11-09T03:45:31.404842+0000","last_active":"2023-11-09T03:45:35.442845+0000","last_peered":"2023-11-09T03:45:35.442845+0000","last_clean":"2023-11-09T03:45:35.442845+0000","last_became_active":"2023-11-09T03:45:31.404484+0000","last_became_peered":"2023-11-09T03:45:31.404484+0000","last_unstale":"2023-11-09T03:45:35.442845+0000","last_undegraded":"2023-11-09T03:45:35.442845+0000","last_fullsized":"2023-11-09T03:45:35.442845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:27.400068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440223+0000","last_change":"2023-11-09T03:45:31.404211+0000","last_active":"2023-11-09T03:45:35.440223+0000","last_peered":"2023-11-09T03:45:35.440223+0000","last_clean":"2023-11-09T03:45:35.440223+0000","last_became_active":"2023-11-09T03:45:31.403962+0000","last_became_peered":"2023-11-09T03:45:31.403962+0000","last_unstale":"2023-11-09T03:45:35.440223+0000","last_undegraded":"2023-11-09T03:45:35.440223+0000","last_fullsized":"2023-11-09T03:45:35.440223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:48:08.925647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.619170+0000","last_change":"2023-11-09T03:45:31.407273+0000","last_active":"2023-11-09T03:45:35.619170+0000","last_peered":"2023-11-09T03:45:35.619170+0000","last_clean":"2023-11-09T03:45:35.619170+0000","last_became_active":"2023-11-09T03:45:31.406956+0000","last_became_peered":"2023-11-09T03:45:31.406956+0000","last_unstale":"2023-11-09T03:45:35.619170+0000","last_undegraded":"2023-11-09T03:45:35.619170+0000","last_fullsized":"2023-11-09T03:45:35.619170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:13:04.311780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.432241+0000","last_change":"2023-11-09T03:45:31.402525+0000","last_active":"2023-11-09T03:45:35.432241+0000","last_peered":"2023-11-09T03:45:35.432241+0000","last_clean":"2023-11-09T03:45:35.432241+0000","last_became_active":"2023-11-09T03:45:31.402262+0000","last_became_peered":"2023-11-09T03:45:31.402262+0000","last_unstale":"2023-11-09T03:45:35.432241+0000","last_undegraded":"2023-11-09T03:45:35.432241+0000","last_fullsized":"2023-11-09T03:45:35.432241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:09.007153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441402+0000","last_change":"2023-11-09T03:45:31.411781+0000","last_active":"2023-11-09T03:45:35.441402+0000","last_peered":"2023-11-09T03:45:35.441402+0000","last_clean":"2023-11-09T03:45:35.441402+0000","last_became_active":"2023-11-09T03:45:31.411178+0000","last_became_peered":"2023-11-09T03:45:31.411178+0000","last_unstale":"2023-11-09T03:45:35.441402+0000","last_undegraded":"2023-11-09T03:45:35.441402+0000","last_fullsized":"2023-11-09T03:45:35.441402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:13:04.311780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439393+0000","last_change":"2023-11-09T03:45:31.409602+0000","last_active":"2023-11-09T03:45:35.439393+0000","last_peered":"2023-11-09T03:45:35.439393+0000","last_clean":"2023-11-09T03:45:35.439393+0000","last_became_active":"2023-11-09T03:45:31.409313+0000","last_became_peered":"2023-11-09T03:45:31.409313+0000","last_unstale":"2023-11-09T03:45:35.439393+0000","last_undegraded":"2023-11-09T03:45:35.439393+0000","last_fullsized":"2023-11-09T03:45:35.439393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:48:08.925647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441829+0000","last_change":"2023-11-09T03:45:31.394559+0000","last_active":"2023-11-09T03:45:35.441829+0000","last_peered":"2023-11-09T03:45:35.441829+0000","last_clean":"2023-11-09T03:45:35.441829+0000","last_became_active":"2023-11-09T03:45:31.394284+0000","last_became_peered":"2023-11-09T03:45:31.394284+0000","last_unstale":"2023-11-09T03:45:35.441829+0000","last_undegraded":"2023-11-09T03:45:35.441829+0000","last_fullsized":"2023-11-09T03:45:35.441829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:11:10.514126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.443281+0000","last_change":"2023-11-09T03:45:31.397341+0000","last_active":"2023-11-09T03:45:35.443281+0000","last_peered":"2023-11-09T03:45:35.443281+0000","last_clean":"2023-11-09T03:45:35.443281+0000","last_became_active":"2023-11-09T03:45:31.396986+0000","last_became_peered":"2023-11-09T03:45:31.396986+0000","last_unstale":"2023-11-09T03:45:35.443281+0000","last_undegraded":"2023-11-09T03:45:35.443281+0000","last_fullsized":"2023-11-09T03:45:35.443281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:34:49.405450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439412+0000","last_change":"2023-11-09T03:45:31.402346+0000","last_active":"2023-11-09T03:45:35.439412+0000","last_peered":"2023-11-09T03:45:35.439412+0000","last_clean":"2023-11-09T03:45:35.439412+0000","last_became_active":"2023-11-09T03:45:31.401630+0000","last_became_peered":"2023-11-09T03:45:31.401630+0000","last_unstale":"2023-11-09T03:45:35.439412+0000","last_undegraded":"2023-11-09T03:45:35.439412+0000","last_fullsized":"2023-11-09T03:45:35.439412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:02:27.095864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.435980+0000","last_change":"2023-11-09T03:45:31.403882+0000","last_active":"2023-11-09T03:45:35.435980+0000","last_peered":"2023-11-09T03:45:35.435980+0000","last_clean":"2023-11-09T03:45:35.435980+0000","last_became_active":"2023-11-09T03:45:31.403637+0000","last_became_peered":"2023-11-09T03:45:31.403637+0000","last_unstale":"2023-11-09T03:45:35.435980+0000","last_undegraded":"2023-11-09T03:45:35.435980+0000","last_fullsized":"2023-11-09T03:45:35.435980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:39:30.327813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438776+0000","last_change":"2023-11-09T03:45:31.415358+0000","last_active":"2023-11-09T03:45:35.438776+0000","last_peered":"2023-11-09T03:45:35.438776+0000","last_clean":"2023-11-09T03:45:35.438776+0000","last_became_active":"2023-11-09T03:45:31.415018+0000","last_became_peered":"2023-11-09T03:45:31.415018+0000","last_unstale":"2023-11-09T03:45:35.438776+0000","last_undegraded":"2023-11-09T03:45:35.438776+0000","last_fullsized":"2023-11-09T03:45:35.438776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:08.352566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440770+0000","last_change":"2023-11-09T03:45:31.414693+0000","last_active":"2023-11-09T03:45:35.440770+0000","last_peered":"2023-11-09T03:45:35.440770+0000","last_clean":"2023-11-09T03:45:35.440770+0000","last_became_active":"2023-11-09T03:45:31.414400+0000","last_became_peered":"2023-11-09T03:45:31.414400+0000","last_unstale":"2023-11-09T03:45:35.440770+0000","last_undegraded":"2023-11-09T03:45:35.440770+0000","last_fullsized":"2023-11-09T03:45:35.440770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:22.653144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.618910+0000","last_change":"2023-11-09T03:45:31.419560+0000","last_active":"2023-11-09T03:45:35.618910+0000","last_peered":"2023-11-09T03:45:35.618910+0000","last_clean":"2023-11-09T03:45:35.618910+0000","last_became_active":"2023-11-09T03:45:31.418455+0000","last_became_peered":"2023-11-09T03:45:31.418455+0000","last_unstale":"2023-11-09T03:45:35.618910+0000","last_undegraded":"2023-11-09T03:45:35.618910+0000","last_fullsized":"2023-11-09T03:45:35.618910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:14:55.198635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438871+0000","last_change":"2023-11-09T03:45:31.415460+0000","last_active":"2023-11-09T03:45:35.438871+0000","last_peered":"2023-11-09T03:45:35.438871+0000","last_clean":"2023-11-09T03:45:35.438871+0000","last_became_active":"2023-11-09T03:45:31.415210+0000","last_became_peered":"2023-11-09T03:45:31.415210+0000","last_unstale":"2023-11-09T03:45:35.438871+0000","last_undegraded":"2023-11-09T03:45:35.438871+0000","last_fullsized":"2023-11-09T03:45:35.438871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:44.473424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617339+0000","last_change":"2023-11-09T03:45:31.411588+0000","last_active":"2023-11-09T03:45:35.617339+0000","last_peered":"2023-11-09T03:45:35.617339+0000","last_clean":"2023-11-09T03:45:35.617339+0000","last_became_active":"2023-11-09T03:45:31.411168+0000","last_became_peered":"2023-11-09T03:45:31.411168+0000","last_unstale":"2023-11-09T03:45:35.617339+0000","last_undegraded":"2023-11-09T03:45:35.617339+0000","last_fullsized":"2023-11-09T03:45:35.617339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:39:42.380963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440027+0000","last_change":"2023-11-09T03:45:31.412481+0000","last_active":"2023-11-09T03:45:35.440027+0000","last_peered":"2023-11-09T03:45:35.440027+0000","last_clean":"2023-11-09T03:45:35.440027+0000","last_became_active":"2023-11-09T03:45:31.412167+0000","last_became_peered":"2023-11-09T03:45:31.412167+0000","last_unstale":"2023-11-09T03:45:35.440027+0000","last_undegraded":"2023-11-09T03:45:35.440027+0000","last_fullsized":"2023-11-09T03:45:35.440027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:22.653144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617339+0000","last_change":"2023-11-09T03:45:31.405459+0000","last_active":"2023-11-09T03:45:35.617339+0000","last_peered":"2023-11-09T03:45:35.617339+0000","last_clean":"2023-11-09T03:45:35.617339+0000","last_became_active":"2023-11-09T03:45:31.405188+0000","last_became_peered":"2023-11-09T03:45:31.405188+0000","last_unstale":"2023-11-09T03:45:35.617339+0000","last_undegraded":"2023-11-09T03:45:35.617339+0000","last_fullsized":"2023-11-09T03:45:35.617339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:39:36.741003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440927+0000","last_change":"2023-11-09T03:45:31.417756+0000","last_active":"2023-11-09T03:45:35.440927+0000","last_peered":"2023-11-09T03:45:35.440927+0000","last_clean":"2023-11-09T03:45:35.440927+0000","last_became_active":"2023-11-09T03:45:31.417488+0000","last_became_peered":"2023-11-09T03:45:31.417488+0000","last_unstale":"2023-11-09T03:45:35.440927+0000","last_undegraded":"2023-11-09T03:45:35.440927+0000","last_fullsized":"2023-11-09T03:45:35.440927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:36:23.593484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442615+0000","last_change":"2023-11-09T03:45:31.397411+0000","last_active":"2023-11-09T03:45:35.442615+0000","last_peered":"2023-11-09T03:45:35.442615+0000","last_clean":"2023-11-09T03:45:35.442615+0000","last_became_active":"2023-11-09T03:45:31.396779+0000","last_became_peered":"2023-11-09T03:45:31.396779+0000","last_unstale":"2023-11-09T03:45:35.442615+0000","last_undegraded":"2023-11-09T03:45:35.442615+0000","last_fullsized":"2023-11-09T03:45:35.442615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:54:30.809126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441937+0000","last_change":"2023-11-09T03:45:31.396447+0000","last_active":"2023-11-09T03:45:35.441937+0000","last_peered":"2023-11-09T03:45:35.441937+0000","last_clean":"2023-11-09T03:45:35.441937+0000","last_became_active":"2023-11-09T03:45:31.396184+0000","last_became_peered":"2023-11-09T03:45:31.396184+0000","last_unstale":"2023-11-09T03:45:35.441937+0000","last_undegraded":"2023-11-09T03:45:35.441937+0000","last_fullsized":"2023-11-09T03:45:35.441937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:48:08.925647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.618840+0000","last_change":"2023-11-09T03:45:31.405664+0000","last_active":"2023-11-09T03:45:35.618840+0000","last_peered":"2023-11-09T03:45:35.618840+0000","last_clean":"2023-11-09T03:45:35.618840+0000","last_became_active":"2023-11-09T03:45:31.404726+0000","last_became_peered":"2023-11-09T03:45:31.404726+0000","last_unstale":"2023-11-09T03:45:35.618840+0000","last_undegraded":"2023-11-09T03:45:35.618840+0000","last_fullsized":"2023-11-09T03:45:35.618840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:41:08.550864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":695413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":104553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":104589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":104623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":868352,"data_stored":694850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-09T03:45:49.427 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-09T03:45:49.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.557+0000 7fbb8c6b4640 1 Processor -- start 2023-11-09T03:45:49.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.557+0000 7fbb8c6b4640 1 -- start start 2023-11-09T03:45:49.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.558+0000 7fbb8c6b4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbb8415d250 0x7fbb840c9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:49.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.558+0000 7fbb8c6b4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbb840c9d70 0x7fbb840ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:49.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.558+0000 7fbb8c6b4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb8406a120 con 0x7fbb8415ce70 2023-11-09T03:45:49.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.558+0000 7fbb8c6b4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb840cdc10 con 0x7fbb8415d250 2023-11-09T03:45:49.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.558+0000 7fbb8c6b4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb8406a370 con 0x7fbb840c9d70 2023-11-09T03:45:49.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.558+0000 7fbb89c4f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbb8415d250 0x7fbb840c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:49.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.558+0000 7fbb8a450640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fbb8415ce70 0x7fbb8406d9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:56728/0 (socket says 172.21.15.76:56728) 2023-11-09T03:45:49.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.558+0000 7fbb8a450640 1 -- 172.21.15.76:0/4046292405 learned_addr learned my addr 172.21.15.76:0/4046292405 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:49.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.558+0000 7fbb8ac51640 1 --2- 172.21.15.76:0/4046292405 >> v2:172.21.15.113:3300/0 conn(0x7fbb840c9d70 0x7fbb840ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:49.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.558+0000 7fbb89c4f640 1 -- 172.21.15.76:0/4046292405 >> v2:172.21.15.113:3300/0 conn(0x7fbb840c9d70 msgr2=0x7fbb840ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:49.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.558+0000 7fbb89c4f640 1 --2- 172.21.15.76:0/4046292405 >> v2:172.21.15.113:3300/0 conn(0x7fbb840c9d70 0x7fbb840ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.558+0000 7fbb89c4f640 1 -- 172.21.15.76:0/4046292405 >> v1:172.21.15.76:6789/0 conn(0x7fbb8415ce70 legacy=0x7fbb8406d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:49.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.558+0000 7fbb89c4f640 1 -- 172.21.15.76:0/4046292405 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8406a650 con 0x7fbb8415d250 2023-11-09T03:45:49.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.559+0000 7fbb89c4f640 1 --2- 172.21.15.76:0/4046292405 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb8415d250 0x7fbb840c9830 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbb8409cee0 tx=0x7fbb7402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56809eb0d263d17e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:49.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.559+0000 7fbb8944e640 1 -- 172.21.15.76:0/4046292405 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb74031830 con 0x7fbb8415d250 2023-11-09T03:45:49.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.559+0000 7fbb8944e640 1 -- 172.21.15.76:0/4046292405 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb74031990 con 0x7fbb8415d250 2023-11-09T03:45:49.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.559+0000 7fbb8944e640 1 -- 172.21.15.76:0/4046292405 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb74031d20 con 0x7fbb8415d250 2023-11-09T03:45:49.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.560+0000 7fbb8c6b4640 1 -- 172.21.15.76:0/4046292405 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb8415d250 msgr2=0x7fbb840c9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:49.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.560+0000 7fbb8c6b4640 1 --2- 172.21.15.76:0/4046292405 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb8415d250 0x7fbb840c9830 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbb8409cee0 tx=0x7fbb7402b8d0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.560+0000 7fbb8c6b4640 1 -- 172.21.15.76:0/4046292405 shutdown_connections 2023-11-09T03:45:49.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.560+0000 7fbb8c6b4640 1 --2- 172.21.15.76:0/4046292405 >> v2:172.21.15.113:3300/0 conn(0x7fbb840c9d70 0x7fbb840ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.560+0000 7fbb8c6b4640 1 --2- 172.21.15.76:0/4046292405 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb8415d250 0x7fbb840c9830 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.560+0000 7fbb8c6b4640 1 -- 172.21.15.76:0/4046292405 >> 172.21.15.76:0/4046292405 conn(0x7fbb840b0cd0 msgr2=0x7fbb840b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:49.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.560+0000 7fbb8c6b4640 1 -- 172.21.15.76:0/4046292405 shutdown_connections 2023-11-09T03:45:49.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.560+0000 7fbb8c6b4640 1 -- 172.21.15.76:0/4046292405 wait complete. 2023-11-09T03:45:49.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.561+0000 7fbb8c6b4640 1 Processor -- start 2023-11-09T03:45:49.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.561+0000 7fbb8c6b4640 1 -- start start 2023-11-09T03:45:49.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.562+0000 7fbb8c6b4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb8415ce70 0x7fbb841e2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:49.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.562+0000 7fbb8c6b4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb8415d250 0x7fbb8414d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:49.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.562+0000 7fbb8c6b4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb8415a4d0 0x7fbb841f6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:49.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.562+0000 7fbb8c6b4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb840b0190 con 0x7fbb8415a4d0 2023-11-09T03:45:49.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.562+0000 7fbb8c6b4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb841e2400 con 0x7fbb8415d250 2023-11-09T03:45:49.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.562+0000 7fbb8c6b4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb841e3250 con 0x7fbb8415ce70 2023-11-09T03:45:49.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.562+0000 7fbb8ac51640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb8415a4d0 0x7fbb841f6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:49.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.562+0000 7fbb8ac51640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb8415a4d0 0x7fbb841f6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57622/0 (socket says 172.21.15.76:57622) 2023-11-09T03:45:49.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.562+0000 7fbb8ac51640 1 -- 172.21.15.76:0/2760924638 learned_addr learned my addr 172.21.15.76:0/2760924638 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:49.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.562+0000 7fbb89c4f640 1 --2- 172.21.15.76:0/2760924638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb8415d250 0x7fbb8414d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:49.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.562+0000 7fbb8a450640 1 --2- 172.21.15.76:0/2760924638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb8415ce70 0x7fbb841e2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:49.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.562+0000 7fbb8ac51640 1 -- 172.21.15.76:0/2760924638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb8415ce70 msgr2=0x7fbb841e2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:49.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.562+0000 7fbb8ac51640 1 --2- 172.21.15.76:0/2760924638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb8415ce70 0x7fbb841e2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.562+0000 7fbb8ac51640 1 -- 172.21.15.76:0/2760924638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb8415d250 msgr2=0x7fbb8414d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:49.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.562+0000 7fbb8ac51640 1 --2- 172.21.15.76:0/2760924638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb8415d250 0x7fbb8414d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.562+0000 7fbb8ac51640 1 -- 172.21.15.76:0/2760924638 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb74009400 con 0x7fbb8415a4d0 2023-11-09T03:45:49.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.562+0000 7fbb8ac51640 1 --2- 172.21.15.76:0/2760924638 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb8415a4d0 0x7fbb841f6710 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c007e30 tx=0x7fbb7c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:49.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.562+0000 7fbb7affd640 1 -- 172.21.15.76:0/2760924638 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c014620 con 0x7fbb8415a4d0 2023-11-09T03:45:49.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.562+0000 7fbb7affd640 1 -- 172.21.15.76:0/2760924638 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7c00ee90 con 0x7fbb8415a4d0 2023-11-09T03:45:49.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.562+0000 7fbb8c6b4640 1 -- 172.21.15.76:0/2760924638 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb841e26a0 con 0x7fbb8415a4d0 2023-11-09T03:45:49.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.562+0000 7fbb7affd640 1 -- 172.21.15.76:0/2760924638 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c0149c0 con 0x7fbb8415a4d0 2023-11-09T03:45:49.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.562+0000 7fbb8c6b4640 1 -- 172.21.15.76:0/2760924638 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb841f61f0 con 0x7fbb8415a4d0 2023-11-09T03:45:49.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.563+0000 7fbb8c6b4640 1 -- 172.21.15.76:0/2760924638 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb84074290 con 0x7fbb8415a4d0 2023-11-09T03:45:49.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.564+0000 7fbb7affd640 1 -- 172.21.15.76:0/2760924638 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbb7c014b20 con 0x7fbb8415a4d0 2023-11-09T03:45:49.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.564+0000 7fbb7affd640 1 --2- 172.21.15.76:0/2760924638 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbb6806ae70 0x7fbb6806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:49.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.564+0000 7fbb7affd640 1 -- 172.21.15.76:0/2760924638 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbb7c08a0a0 con 0x7fbb8415a4d0 2023-11-09T03:45:49.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.564+0000 7fbb8a450640 1 --2- 172.21.15.76:0/2760924638 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbb6806ae70 0x7fbb6806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:49.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.565+0000 7fbb8a450640 1 --2- 172.21.15.76:0/2760924638 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbb6806ae70 0x7fbb6806d330 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbb80005880 tx=0x7fbb80009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:49.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.568+0000 7fbb7affd640 1 -- 172.21.15.76:0/2760924638 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbb7c054fa0 con 0x7fbb8415a4d0 2023-11-09T03:45:49.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.731+0000 7fbb8c6b4640 1 -- 172.21.15.76:0/2760924638 --> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbb8406ac10 con 0x7fbb6806ae70 2023-11-09T03:45:49.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.740+0000 7fbb7affd640 1 -- 172.21.15.76:0/2760924638 <== mgr.4105 v2:172.21.15.113:6832/1264193042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345303 (secure 0 0 0) 0x7fbb8000f820 con 0x7fbb6806ae70 2023-11-09T03:45:49.742 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:49.744 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2023-11-09T03:45:49.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.743+0000 7fbb8c6b4640 1 -- 172.21.15.76:0/2760924638 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbb6806ae70 msgr2=0x7fbb6806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:49.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.743+0000 7fbb8c6b4640 1 --2- 172.21.15.76:0/2760924638 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbb6806ae70 0x7fbb6806d330 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbb80005880 tx=0x7fbb80009040 comp rx=0 tx=0).stop 2023-11-09T03:45:49.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.743+0000 7fbb8c6b4640 1 -- 172.21.15.76:0/2760924638 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb8415a4d0 msgr2=0x7fbb841f6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:49.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.743+0000 7fbb8c6b4640 1 --2- 172.21.15.76:0/2760924638 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb8415a4d0 0x7fbb841f6710 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c007e30 tx=0x7fbb7c00e4b0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.743+0000 7fbb8c6b4640 1 -- 172.21.15.76:0/2760924638 shutdown_connections 2023-11-09T03:45:49.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.743+0000 7fbb8c6b4640 1 --2- 172.21.15.76:0/2760924638 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbb6806ae70 0x7fbb6806d330 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.743+0000 7fbb8c6b4640 1 --2- 172.21.15.76:0/2760924638 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb8415a4d0 0x7fbb841f6710 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.743+0000 7fbb8c6b4640 1 --2- 172.21.15.76:0/2760924638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb8415d250 0x7fbb8414d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.743+0000 7fbb8c6b4640 1 --2- 172.21.15.76:0/2760924638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb8415ce70 0x7fbb841e2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.743+0000 7fbb8c6b4640 1 -- 172.21.15.76:0/2760924638 >> 172.21.15.76:0/2760924638 conn(0x7fbb840b0cd0 msgr2=0x7fbb840ad470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:49.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.743+0000 7fbb8c6b4640 1 -- 172.21.15.76:0/2760924638 shutdown_connections 2023-11-09T03:45:49.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.743+0000 7fbb8c6b4640 1 -- 172.21.15.76:0/2760924638 wait complete. 2023-11-09T03:45:49.753 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2023-11-09T03:45:48.752233+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219468,"kb_used_data":3300,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755252,"statfs":{"total":773094113280,"available":772869378048,"internally_reserved":0,"allocated":3379200,"data_stored":2025099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2229,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003169"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.444993+0000","last_change":"2023-11-09T03:45:30.408658+0000","last_active":"2023-11-09T03:45:35.444993+0000","last_peered":"2023-11-09T03:45:35.444993+0000","last_clean":"2023-11-09T03:45:35.444993+0000","last_became_active":"2023-11-09T03:45:30.408345+0000","last_became_peered":"2023-11-09T03:45:30.408345+0000","last_unstale":"2023-11-09T03:45:35.444993+0000","last_undegraded":"2023-11-09T03:45:35.444993+0000","last_fullsized":"2023-11-09T03:45:35.444993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:12:07.086030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440545+0000","last_change":"2023-11-09T03:45:31.403012+0000","last_active":"2023-11-09T03:45:35.440545+0000","last_peered":"2023-11-09T03:45:35.440545+0000","last_clean":"2023-11-09T03:45:35.440545+0000","last_became_active":"2023-11-09T03:45:31.402623+0000","last_became_peered":"2023-11-09T03:45:31.402623+0000","last_unstale":"2023-11-09T03:45:35.440545+0000","last_undegraded":"2023-11-09T03:45:35.440545+0000","last_fullsized":"2023-11-09T03:45:35.440545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:36:23.593484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388073+0000","last_change":"2023-11-09T03:45:35.620447+0000","last_active":"2023-11-09T03:45:36.388073+0000","last_peered":"2023-11-09T03:45:36.388073+0000","last_clean":"2023-11-09T03:45:36.388073+0000","last_became_active":"2023-11-09T03:45:35.619949+0000","last_became_peered":"2023-11-09T03:45:35.619949+0000","last_unstale":"2023-11-09T03:45:36.388073+0000","last_undegraded":"2023-11-09T03:45:36.388073+0000","last_fullsized":"2023-11-09T03:45:36.388073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:13:00.499438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440637+0000","last_change":"2023-11-09T03:45:30.411660+0000","last_active":"2023-11-09T03:45:35.440637+0000","last_peered":"2023-11-09T03:45:35.440637+0000","last_clean":"2023-11-09T03:45:35.440637+0000","last_became_active":"2023-11-09T03:45:30.411274+0000","last_became_peered":"2023-11-09T03:45:30.411274+0000","last_unstale":"2023-11-09T03:45:35.440637+0000","last_undegraded":"2023-11-09T03:45:35.440637+0000","last_fullsized":"2023-11-09T03:45:35.440637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:39:29.323066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.444755+0000","last_change":"2023-11-09T03:45:31.415835+0000","last_active":"2023-11-09T03:45:35.444755+0000","last_peered":"2023-11-09T03:45:35.444755+0000","last_clean":"2023-11-09T03:45:35.444755+0000","last_became_active":"2023-11-09T03:45:31.415571+0000","last_became_peered":"2023-11-09T03:45:31.415571+0000","last_unstale":"2023-11-09T03:45:35.444755+0000","last_undegraded":"2023-11-09T03:45:35.444755+0000","last_fullsized":"2023-11-09T03:45:35.444755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:13:29.896024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388822+0000","last_change":"2023-11-09T03:45:35.445871+0000","last_active":"2023-11-09T03:45:36.388822+0000","last_peered":"2023-11-09T03:45:36.388822+0000","last_clean":"2023-11-09T03:45:36.388822+0000","last_became_active":"2023-11-09T03:45:35.445390+0000","last_became_peered":"2023-11-09T03:45:35.445390+0000","last_unstale":"2023-11-09T03:45:36.388822+0000","last_undegraded":"2023-11-09T03:45:36.388822+0000","last_fullsized":"2023-11-09T03:45:36.388822+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:26.681331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617937+0000","last_change":"2023-11-09T03:45:30.416124+0000","last_active":"2023-11-09T03:45:35.617937+0000","last_peered":"2023-11-09T03:45:35.617937+0000","last_clean":"2023-11-09T03:45:35.617937+0000","last_became_active":"2023-11-09T03:45:30.415850+0000","last_became_peered":"2023-11-09T03:45:30.415850+0000","last_unstale":"2023-11-09T03:45:35.617937+0000","last_undegraded":"2023-11-09T03:45:35.617937+0000","last_fullsized":"2023-11-09T03:45:35.617937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:43.468677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441010+0000","last_change":"2023-11-09T03:45:31.413149+0000","last_active":"2023-11-09T03:45:35.441010+0000","last_peered":"2023-11-09T03:45:35.441010+0000","last_clean":"2023-11-09T03:45:35.441010+0000","last_became_active":"2023-11-09T03:45:31.412875+0000","last_became_peered":"2023-11-09T03:45:31.412875+0000","last_unstale":"2023-11-09T03:45:35.441010+0000","last_undegraded":"2023-11-09T03:45:35.441010+0000","last_fullsized":"2023-11-09T03:45:35.441010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:04:02.978744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.389160+0000","last_change":"2023-11-09T03:45:35.444039+0000","last_active":"2023-11-09T03:45:36.389160+0000","last_peered":"2023-11-09T03:45:36.389160+0000","last_clean":"2023-11-09T03:45:36.389160+0000","last_became_active":"2023-11-09T03:45:35.443646+0000","last_became_peered":"2023-11-09T03:45:35.443646+0000","last_unstale":"2023-11-09T03:45:36.389160+0000","last_undegraded":"2023-11-09T03:45:36.389160+0000","last_fullsized":"2023-11-09T03:45:36.389160+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:02:31.124051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617462+0000","last_change":"2023-11-09T03:45:30.418418+0000","last_active":"2023-11-09T03:45:35.617462+0000","last_peered":"2023-11-09T03:45:35.617462+0000","last_clean":"2023-11-09T03:45:35.617462+0000","last_became_active":"2023-11-09T03:45:30.417963+0000","last_became_peered":"2023-11-09T03:45:30.417963+0000","last_unstale":"2023-11-09T03:45:35.617462+0000","last_undegraded":"2023-11-09T03:45:35.617462+0000","last_fullsized":"2023-11-09T03:45:35.617462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:07.347819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439849+0000","last_change":"2023-11-09T03:45:31.416138+0000","last_active":"2023-11-09T03:45:35.439849+0000","last_peered":"2023-11-09T03:45:35.439849+0000","last_clean":"2023-11-09T03:45:35.439849+0000","last_became_active":"2023-11-09T03:45:31.415636+0000","last_became_peered":"2023-11-09T03:45:31.415636+0000","last_unstale":"2023-11-09T03:45:35.439849+0000","last_undegraded":"2023-11-09T03:45:35.439849+0000","last_fullsized":"2023-11-09T03:45:35.439849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:02:27.095864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.389010+0000","last_change":"2023-11-09T03:45:35.448265+0000","last_active":"2023-11-09T03:45:36.389010+0000","last_peered":"2023-11-09T03:45:36.389010+0000","last_clean":"2023-11-09T03:45:36.389010+0000","last_became_active":"2023-11-09T03:45:35.447747+0000","last_became_peered":"2023-11-09T03:45:35.447747+0000","last_unstale":"2023-11-09T03:45:36.389010+0000","last_undegraded":"2023-11-09T03:45:36.389010+0000","last_fullsized":"2023-11-09T03:45:36.389010+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:39:46.409150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.437921+0000","last_change":"2023-11-09T03:45:31.400950+0000","last_active":"2023-11-09T03:45:35.437921+0000","last_peered":"2023-11-09T03:45:35.437921+0000","last_clean":"2023-11-09T03:45:35.437921+0000","last_became_active":"2023-11-09T03:45:31.400667+0000","last_became_peered":"2023-11-09T03:45:31.400667+0000","last_unstale":"2023-11-09T03:45:35.437921+0000","last_undegraded":"2023-11-09T03:45:35.437921+0000","last_fullsized":"2023-11-09T03:45:35.437921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:44.473424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617461+0000","last_change":"2023-11-09T03:45:30.418728+0000","last_active":"2023-11-09T03:45:35.617461+0000","last_peered":"2023-11-09T03:45:35.617461+0000","last_clean":"2023-11-09T03:45:35.617461+0000","last_became_active":"2023-11-09T03:45:30.417939+0000","last_became_peered":"2023-11-09T03:45:30.417939+0000","last_unstale":"2023-11-09T03:45:35.617461+0000","last_undegraded":"2023-11-09T03:45:35.617461+0000","last_fullsized":"2023-11-09T03:45:35.617461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:35:45.408209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.389355+0000","last_change":"2023-11-09T03:45:35.448352+0000","last_active":"2023-11-09T03:45:36.389355+0000","last_peered":"2023-11-09T03:45:36.389355+0000","last_clean":"2023-11-09T03:45:36.389355+0000","last_became_active":"2023-11-09T03:45:35.447889+0000","last_became_peered":"2023-11-09T03:45:35.447889+0000","last_unstale":"2023-11-09T03:45:36.389355+0000","last_undegraded":"2023-11-09T03:45:36.389355+0000","last_fullsized":"2023-11-09T03:45:36.389355+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:39:40.769191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439502+0000","last_change":"2023-11-09T03:45:31.398106+0000","last_active":"2023-11-09T03:45:35.439502+0000","last_peered":"2023-11-09T03:45:35.439502+0000","last_clean":"2023-11-09T03:45:35.439502+0000","last_became_active":"2023-11-09T03:45:31.397712+0000","last_became_peered":"2023-11-09T03:45:31.397712+0000","last_unstale":"2023-11-09T03:45:35.439502+0000","last_undegraded":"2023-11-09T03:45:35.439502+0000","last_fullsized":"2023-11-09T03:45:35.439502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:04:02.978744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441816+0000","last_change":"2023-11-09T03:45:30.408462+0000","last_active":"2023-11-09T03:45:35.441816+0000","last_peered":"2023-11-09T03:45:35.441816+0000","last_clean":"2023-11-09T03:45:35.441816+0000","last_became_active":"2023-11-09T03:45:30.408190+0000","last_became_peered":"2023-11-09T03:45:30.408190+0000","last_unstale":"2023-11-09T03:45:35.441816+0000","last_undegraded":"2023-11-09T03:45:35.441816+0000","last_fullsized":"2023-11-09T03:45:35.441816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:11:40.438915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.389721+0000","last_change":"2023-11-09T03:45:35.445519+0000","last_active":"2023-11-09T03:45:36.389721+0000","last_peered":"2023-11-09T03:45:36.389721+0000","last_clean":"2023-11-09T03:45:36.389721+0000","last_became_active":"2023-11-09T03:45:35.445125+0000","last_became_peered":"2023-11-09T03:45:35.445125+0000","last_unstale":"2023-11-09T03:45:36.389721+0000","last_undegraded":"2023-11-09T03:45:36.389721+0000","last_fullsized":"2023-11-09T03:45:36.389721+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:45.477938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439030+0000","last_change":"2023-11-09T03:45:31.402204+0000","last_active":"2023-11-09T03:45:35.439030+0000","last_peered":"2023-11-09T03:45:35.439030+0000","last_clean":"2023-11-09T03:45:35.439030+0000","last_became_active":"2023-11-09T03:45:31.401312+0000","last_became_peered":"2023-11-09T03:45:31.401312+0000","last_unstale":"2023-11-09T03:45:35.439030+0000","last_undegraded":"2023-11-09T03:45:35.439030+0000","last_fullsized":"2023-11-09T03:45:35.439030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:48:08.925647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441918+0000","last_change":"2023-11-09T03:45:30.414258+0000","last_active":"2023-11-09T03:45:35.441918+0000","last_peered":"2023-11-09T03:45:35.441918+0000","last_clean":"2023-11-09T03:45:35.441918+0000","last_became_active":"2023-11-09T03:45:30.413994+0000","last_became_peered":"2023-11-09T03:45:30.413994+0000","last_unstale":"2023-11-09T03:45:35.441918+0000","last_undegraded":"2023-11-09T03:45:35.441918+0000","last_fullsized":"2023-11-09T03:45:35.441918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:12:07.086030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.263008+0000","last_change":"2023-11-09T03:45:35.620757+0000","last_active":"2023-11-09T03:45:37.263008+0000","last_peered":"2023-11-09T03:45:37.263008+0000","last_clean":"2023-11-09T03:45:37.263008+0000","last_became_active":"2023-11-09T03:45:35.620219+0000","last_became_peered":"2023-11-09T03:45:35.620219+0000","last_unstale":"2023-11-09T03:45:37.263008+0000","last_undegraded":"2023-11-09T03:45:37.263008+0000","last_fullsized":"2023-11-09T03:45:37.263008+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:41:12.579051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.436441+0000","last_change":"2023-11-09T03:45:31.396755+0000","last_active":"2023-11-09T03:45:35.436441+0000","last_peered":"2023-11-09T03:45:35.436441+0000","last_clean":"2023-11-09T03:45:35.436441+0000","last_became_active":"2023-11-09T03:45:31.396475+0000","last_became_peered":"2023-11-09T03:45:31.396475+0000","last_unstale":"2023-11-09T03:45:35.436441+0000","last_undegraded":"2023-11-09T03:45:35.436441+0000","last_fullsized":"2023-11-09T03:45:35.436441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:07:02.262322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617133+0000","last_change":"2023-11-09T03:45:30.419151+0000","last_active":"2023-11-09T03:45:35.617133+0000","last_peered":"2023-11-09T03:45:35.617133+0000","last_clean":"2023-11-09T03:45:35.617133+0000","last_became_active":"2023-11-09T03:45:30.418874+0000","last_became_peered":"2023-11-09T03:45:30.418874+0000","last_unstale":"2023-11-09T03:45:35.617133+0000","last_undegraded":"2023-11-09T03:45:35.617133+0000","last_fullsized":"2023-11-09T03:45:35.617133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:34:48.400703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.389351+0000","last_change":"2023-11-09T03:45:35.445655+0000","last_active":"2023-11-09T03:45:36.389351+0000","last_peered":"2023-11-09T03:45:36.389351+0000","last_clean":"2023-11-09T03:45:36.389351+0000","last_became_active":"2023-11-09T03:45:35.445246+0000","last_became_peered":"2023-11-09T03:45:35.445246+0000","last_unstale":"2023-11-09T03:45:36.389351+0000","last_undegraded":"2023-11-09T03:45:36.389351+0000","last_fullsized":"2023-11-09T03:45:36.389351+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:04:07.006932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.443777+0000","last_change":"2023-11-09T03:45:31.397453+0000","last_active":"2023-11-09T03:45:35.443777+0000","last_peered":"2023-11-09T03:45:35.443777+0000","last_clean":"2023-11-09T03:45:35.443777+0000","last_became_active":"2023-11-09T03:45:31.396892+0000","last_became_peered":"2023-11-09T03:45:31.396892+0000","last_unstale":"2023-11-09T03:45:35.443777+0000","last_undegraded":"2023-11-09T03:45:35.443777+0000","last_fullsized":"2023-11-09T03:45:35.443777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:44.473424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440439+0000","last_change":"2023-11-09T03:45:30.416814+0000","last_active":"2023-11-09T03:45:35.440439+0000","last_peered":"2023-11-09T03:45:35.440439+0000","last_clean":"2023-11-09T03:45:35.440439+0000","last_became_active":"2023-11-09T03:45:30.416446+0000","last_became_peered":"2023-11-09T03:45:30.416446+0000","last_unstale":"2023-11-09T03:45:35.440439+0000","last_undegraded":"2023-11-09T03:45:35.440439+0000","last_fullsized":"2023-11-09T03:45:35.440439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:37:54.581788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388988+0000","last_change":"2023-11-09T03:45:35.618893+0000","last_active":"2023-11-09T03:45:36.388988+0000","last_peered":"2023-11-09T03:45:36.388988+0000","last_clean":"2023-11-09T03:45:36.388988+0000","last_became_active":"2023-11-09T03:45:35.618506+0000","last_became_peered":"2023-11-09T03:45:35.618506+0000","last_unstale":"2023-11-09T03:45:36.388988+0000","last_undegraded":"2023-11-09T03:45:36.388988+0000","last_fullsized":"2023-11-09T03:45:36.388988+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:36:27.621672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438491+0000","last_change":"2023-11-09T03:45:31.406885+0000","last_active":"2023-11-09T03:45:35.438491+0000","last_peered":"2023-11-09T03:45:35.438491+0000","last_clean":"2023-11-09T03:45:35.438491+0000","last_became_active":"2023-11-09T03:45:31.406620+0000","last_became_peered":"2023-11-09T03:45:31.406620+0000","last_unstale":"2023-11-09T03:45:35.438491+0000","last_undegraded":"2023-11-09T03:45:35.438491+0000","last_fullsized":"2023-11-09T03:45:35.438491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:29:17.503411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442020+0000","last_change":"2023-11-09T03:45:30.412534+0000","last_active":"2023-11-09T03:45:35.442020+0000","last_peered":"2023-11-09T03:45:35.442020+0000","last_clean":"2023-11-09T03:45:35.442020+0000","last_became_active":"2023-11-09T03:45:30.412015+0000","last_became_peered":"2023-11-09T03:45:30.412015+0000","last_unstale":"2023-11-09T03:45:35.442020+0000","last_undegraded":"2023-11-09T03:45:35.442020+0000","last_fullsized":"2023-11-09T03:45:35.442020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:34:48.400703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.387965+0000","last_change":"2023-11-09T03:45:35.620972+0000","last_active":"2023-11-09T03:45:36.387965+0000","last_peered":"2023-11-09T03:45:36.387965+0000","last_clean":"2023-11-09T03:45:36.387965+0000","last_became_active":"2023-11-09T03:45:35.620626+0000","last_became_peered":"2023-11-09T03:45:35.620626+0000","last_unstale":"2023-11-09T03:45:36.387965+0000","last_undegraded":"2023-11-09T03:45:36.387965+0000","last_fullsized":"2023-11-09T03:45:36.387965+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:03:57.174574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441966+0000","last_change":"2023-11-09T03:45:31.393932+0000","last_active":"2023-11-09T03:45:35.441966+0000","last_peered":"2023-11-09T03:45:35.441966+0000","last_clean":"2023-11-09T03:45:35.441966+0000","last_became_active":"2023-11-09T03:45:31.393669+0000","last_became_peered":"2023-11-09T03:45:31.393669+0000","last_unstale":"2023-11-09T03:45:35.441966+0000","last_undegraded":"2023-11-09T03:45:35.441966+0000","last_fullsized":"2023-11-09T03:45:35.441966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:27.400068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.618527+0000","last_change":"2023-11-09T03:45:30.419637+0000","last_active":"2023-11-09T03:45:35.618527+0000","last_peered":"2023-11-09T03:45:35.618527+0000","last_clean":"2023-11-09T03:45:35.618527+0000","last_became_active":"2023-11-09T03:45:30.419348+0000","last_became_peered":"2023-11-09T03:45:30.419348+0000","last_unstale":"2023-11-09T03:45:35.618527+0000","last_undegraded":"2023-11-09T03:45:35.618527+0000","last_fullsized":"2023-11-09T03:45:35.618527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:48:07.920901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.261912+0000","last_change":"2023-11-09T03:45:35.619361+0000","last_active":"2023-11-09T03:45:37.261912+0000","last_peered":"2023-11-09T03:45:37.261912+0000","last_clean":"2023-11-09T03:45:37.261912+0000","last_became_active":"2023-11-09T03:45:35.618998+0000","last_became_peered":"2023-11-09T03:45:35.618998+0000","last_unstale":"2023-11-09T03:45:37.261912+0000","last_undegraded":"2023-11-09T03:45:37.261912+0000","last_fullsized":"2023-11-09T03:45:37.261912+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:29:21.531598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439868+0000","last_change":"2023-11-09T03:45:31.408700+0000","last_active":"2023-11-09T03:45:35.439868+0000","last_peered":"2023-11-09T03:45:35.439868+0000","last_clean":"2023-11-09T03:45:35.439868+0000","last_became_active":"2023-11-09T03:45:31.408153+0000","last_became_peered":"2023-11-09T03:45:31.408153+0000","last_unstale":"2023-11-09T03:45:35.439868+0000","last_undegraded":"2023-11-09T03:45:35.439868+0000","last_fullsized":"2023-11-09T03:45:35.439868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:29:17.503411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442042+0000","last_change":"2023-11-09T03:45:30.409443+0000","last_active":"2023-11-09T03:45:35.442042+0000","last_peered":"2023-11-09T03:45:35.442042+0000","last_clean":"2023-11-09T03:45:35.442042+0000","last_became_active":"2023-11-09T03:45:30.409175+0000","last_became_peered":"2023-11-09T03:45:30.409175+0000","last_unstale":"2023-11-09T03:45:35.442042+0000","last_undegraded":"2023-11-09T03:45:35.442042+0000","last_fullsized":"2023-11-09T03:45:35.442042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:34:27.335423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262894+0000","last_change":"2023-11-09T03:45:35.452625+0000","last_active":"2023-11-09T03:45:37.262894+0000","last_peered":"2023-11-09T03:45:37.262894+0000","last_clean":"2023-11-09T03:45:37.262894+0000","last_became_active":"2023-11-09T03:45:35.451772+0000","last_became_peered":"2023-11-09T03:45:35.451772+0000","last_unstale":"2023-11-09T03:45:37.262894+0000","last_undegraded":"2023-11-09T03:45:37.262894+0000","last_fullsized":"2023-11-09T03:45:37.262894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:45.477938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.435623+0000","last_change":"2023-11-09T03:45:31.399046+0000","last_active":"2023-11-09T03:45:35.435623+0000","last_peered":"2023-11-09T03:45:35.435623+0000","last_clean":"2023-11-09T03:45:35.435623+0000","last_became_active":"2023-11-09T03:45:31.398698+0000","last_became_peered":"2023-11-09T03:45:31.398698+0000","last_unstale":"2023-11-09T03:45:35.435623+0000","last_undegraded":"2023-11-09T03:45:35.435623+0000","last_fullsized":"2023-11-09T03:45:35.435623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:35:46.412956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440345+0000","last_change":"2023-11-09T03:45:30.414519+0000","last_active":"2023-11-09T03:45:35.440345+0000","last_peered":"2023-11-09T03:45:35.440345+0000","last_clean":"2023-11-09T03:45:35.440345+0000","last_became_active":"2023-11-09T03:45:30.414025+0000","last_became_peered":"2023-11-09T03:45:30.414025+0000","last_unstale":"2023-11-09T03:45:35.440345+0000","last_undegraded":"2023-11-09T03:45:35.440345+0000","last_fullsized":"2023-11-09T03:45:35.440345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:43.468677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262508+0000","last_change":"2023-11-09T03:45:35.451076+0000","last_active":"2023-11-09T03:45:37.262508+0000","last_peered":"2023-11-09T03:45:37.262508+0000","last_clean":"2023-11-09T03:45:37.262508+0000","last_became_active":"2023-11-09T03:45:35.450721+0000","last_became_peered":"2023-11-09T03:45:35.450721+0000","last_unstale":"2023-11-09T03:45:37.262508+0000","last_undegraded":"2023-11-09T03:45:37.262508+0000","last_fullsized":"2023-11-09T03:45:37.262508+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:29:21.531598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.443831+0000","last_change":"2023-11-09T03:45:31.397098+0000","last_active":"2023-11-09T03:45:35.443831+0000","last_peered":"2023-11-09T03:45:35.443831+0000","last_clean":"2023-11-09T03:45:35.443831+0000","last_became_active":"2023-11-09T03:45:31.396835+0000","last_became_peered":"2023-11-09T03:45:31.396835+0000","last_unstale":"2023-11-09T03:45:35.443831+0000","last_undegraded":"2023-11-09T03:45:35.443831+0000","last_fullsized":"2023-11-09T03:45:35.443831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:54:30.809126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.432187+0000","last_change":"2023-11-09T03:45:30.411878+0000","last_active":"2023-11-09T03:45:35.432187+0000","last_peered":"2023-11-09T03:45:35.432187+0000","last_clean":"2023-11-09T03:45:35.432187+0000","last_became_active":"2023-11-09T03:45:30.411615+0000","last_became_peered":"2023-11-09T03:45:30.411615+0000","last_unstale":"2023-11-09T03:45:35.432187+0000","last_undegraded":"2023-11-09T03:45:35.432187+0000","last_fullsized":"2023-11-09T03:45:35.432187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:11:40.438915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.263008+0000","last_change":"2023-11-09T03:45:35.451994+0000","last_active":"2023-11-09T03:45:37.263008+0000","last_peered":"2023-11-09T03:45:37.263008+0000","last_clean":"2023-11-09T03:45:37.263008+0000","last_became_active":"2023-11-09T03:45:35.451623+0000","last_became_peered":"2023-11-09T03:45:35.451623+0000","last_unstale":"2023-11-09T03:45:37.263008+0000","last_undegraded":"2023-11-09T03:45:37.263008+0000","last_fullsized":"2023-11-09T03:45:37.263008+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:13:08.339968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441086+0000","last_change":"2023-11-09T03:45:31.400475+0000","last_active":"2023-11-09T03:45:35.441086+0000","last_peered":"2023-11-09T03:45:35.441086+0000","last_clean":"2023-11-09T03:45:35.441086+0000","last_became_active":"2023-11-09T03:45:31.400021+0000","last_became_peered":"2023-11-09T03:45:31.400021+0000","last_unstale":"2023-11-09T03:45:35.441086+0000","last_undegraded":"2023-11-09T03:45:35.441086+0000","last_fullsized":"2023-11-09T03:45:35.441086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:11:10.514126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617133+0000","last_change":"2023-11-09T03:45:30.417099+0000","last_active":"2023-11-09T03:45:35.617133+0000","last_peered":"2023-11-09T03:45:35.617133+0000","last_clean":"2023-11-09T03:45:35.617133+0000","last_became_active":"2023-11-09T03:45:30.416779+0000","last_became_peered":"2023-11-09T03:45:30.416779+0000","last_unstale":"2023-11-09T03:45:35.617133+0000","last_undegraded":"2023-11-09T03:45:35.617133+0000","last_fullsized":"2023-11-09T03:45:35.617133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:06:25.266825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262776+0000","last_change":"2023-11-09T03:45:35.618887+0000","last_active":"2023-11-09T03:45:37.262776+0000","last_peered":"2023-11-09T03:45:37.262776+0000","last_clean":"2023-11-09T03:45:37.262776+0000","last_became_active":"2023-11-09T03:45:35.618406+0000","last_became_peered":"2023-11-09T03:45:35.618406+0000","last_unstale":"2023-11-09T03:45:37.262776+0000","last_undegraded":"2023-11-09T03:45:37.262776+0000","last_fullsized":"2023-11-09T03:45:37.262776+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:41:12.579051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438270+0000","last_change":"2023-11-09T03:45:31.416419+0000","last_active":"2023-11-09T03:45:35.438270+0000","last_peered":"2023-11-09T03:45:35.438270+0000","last_clean":"2023-11-09T03:45:35.438270+0000","last_became_active":"2023-11-09T03:45:31.416137+0000","last_became_peered":"2023-11-09T03:45:31.416137+0000","last_unstale":"2023-11-09T03:45:35.438270+0000","last_undegraded":"2023-11-09T03:45:35.438270+0000","last_fullsized":"2023-11-09T03:45:35.438270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:08.352566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439728+0000","last_change":"2023-11-09T03:45:30.417920+0000","last_active":"2023-11-09T03:45:35.439728+0000","last_peered":"2023-11-09T03:45:35.439728+0000","last_clean":"2023-11-09T03:45:35.439728+0000","last_became_active":"2023-11-09T03:45:30.417578+0000","last_became_peered":"2023-11-09T03:45:30.417578+0000","last_unstale":"2023-11-09T03:45:35.439728+0000","last_undegraded":"2023-11-09T03:45:35.439728+0000","last_fullsized":"2023-11-09T03:45:35.439728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:07:01.257576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617231+0000","last_change":"2023-11-09T03:45:21.030926+0000","last_active":"2023-11-09T03:45:35.617231+0000","last_peered":"2023-11-09T03:45:35.617231+0000","last_clean":"2023-11-09T03:45:35.617231+0000","last_became_active":"2023-11-09T03:45:21.029801+0000","last_became_peered":"2023-11-09T03:45:21.029801+0000","last_unstale":"2023-11-09T03:45:35.617231+0000","last_undegraded":"2023-11-09T03:45:35.617231+0000","last_fullsized":"2023-11-09T03:45:35.617231+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_clean_scrub_stamp":"2023-11-09T03:45:19.768433+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:12:59.153713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.264013+0000","last_change":"2023-11-09T03:45:35.622614+0000","last_active":"2023-11-09T03:45:37.264013+0000","last_peered":"2023-11-09T03:45:37.264013+0000","last_clean":"2023-11-09T03:45:37.264013+0000","last_became_active":"2023-11-09T03:45:35.622138+0000","last_became_peered":"2023-11-09T03:45:35.622138+0000","last_unstale":"2023-11-09T03:45:37.264013+0000","last_undegraded":"2023-11-09T03:45:37.264013+0000","last_fullsized":"2023-11-09T03:45:37.264013+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:29:21.531598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617667+0000","last_change":"2023-11-09T03:45:31.417640+0000","last_active":"2023-11-09T03:45:35.617667+0000","last_peered":"2023-11-09T03:45:35.617667+0000","last_clean":"2023-11-09T03:45:35.617667+0000","last_became_active":"2023-11-09T03:45:31.417210+0000","last_became_peered":"2023-11-09T03:45:31.417210+0000","last_unstale":"2023-11-09T03:45:35.617667+0000","last_undegraded":"2023-11-09T03:45:35.617667+0000","last_fullsized":"2023-11-09T03:45:35.617667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:41.449750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.432324+0000","last_change":"2023-11-09T03:45:30.418193+0000","last_active":"2023-11-09T03:45:35.432324+0000","last_peered":"2023-11-09T03:45:35.432324+0000","last_clean":"2023-11-09T03:45:35.432324+0000","last_became_active":"2023-11-09T03:45:30.417874+0000","last_became_peered":"2023-11-09T03:45:30.417874+0000","last_unstale":"2023-11-09T03:45:35.432324+0000","last_undegraded":"2023-11-09T03:45:35.432324+0000","last_fullsized":"2023-11-09T03:45:35.432324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:34:27.335423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.263620+0000","last_change":"2023-11-09T03:45:35.618898+0000","last_active":"2023-11-09T03:45:37.263620+0000","last_peered":"2023-11-09T03:45:37.263620+0000","last_clean":"2023-11-09T03:45:37.263620+0000","last_became_active":"2023-11-09T03:45:35.618502+0000","last_became_peered":"2023-11-09T03:45:35.618502+0000","last_unstale":"2023-11-09T03:45:37.263620+0000","last_undegraded":"2023-11-09T03:45:37.263620+0000","last_fullsized":"2023-11-09T03:45:37.263620+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:39:40.769191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441100+0000","last_change":"2023-11-09T03:45:31.409910+0000","last_active":"2023-11-09T03:45:35.441100+0000","last_peered":"2023-11-09T03:45:35.441100+0000","last_clean":"2023-11-09T03:45:35.441100+0000","last_became_active":"2023-11-09T03:45:31.408863+0000","last_became_peered":"2023-11-09T03:45:31.408863+0000","last_unstale":"2023-11-09T03:45:35.441100+0000","last_undegraded":"2023-11-09T03:45:35.441100+0000","last_fullsized":"2023-11-09T03:45:35.441100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:41.449750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442413+0000","last_change":"2023-11-09T03:45:30.412199+0000","last_active":"2023-11-09T03:45:35.442413+0000","last_peered":"2023-11-09T03:45:35.442413+0000","last_clean":"2023-11-09T03:45:35.442413+0000","last_became_active":"2023-11-09T03:45:30.411864+0000","last_became_peered":"2023-11-09T03:45:30.411864+0000","last_unstale":"2023-11-09T03:45:35.442413+0000","last_undegraded":"2023-11-09T03:45:35.442413+0000","last_fullsized":"2023-11-09T03:45:35.442413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:13:28.891277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262128+0000","last_change":"2023-11-09T03:45:35.456250+0000","last_active":"2023-11-09T03:45:37.262128+0000","last_peered":"2023-11-09T03:45:37.262128+0000","last_clean":"2023-11-09T03:45:37.262128+0000","last_became_active":"2023-11-09T03:45:35.455866+0000","last_became_peered":"2023-11-09T03:45:35.455866+0000","last_unstale":"2023-11-09T03:45:37.262128+0000","last_undegraded":"2023-11-09T03:45:37.262128+0000","last_fullsized":"2023-11-09T03:45:37.262128+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:45.477938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.618920+0000","last_change":"2023-11-09T03:45:31.420110+0000","last_active":"2023-11-09T03:45:35.618920+0000","last_peered":"2023-11-09T03:45:35.618920+0000","last_clean":"2023-11-09T03:45:35.618920+0000","last_became_active":"2023-11-09T03:45:31.419556+0000","last_became_peered":"2023-11-09T03:45:31.419556+0000","last_unstale":"2023-11-09T03:45:35.618920+0000","last_undegraded":"2023-11-09T03:45:35.618920+0000","last_fullsized":"2023-11-09T03:45:35.618920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:51:24.601367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.432312+0000","last_change":"2023-11-09T03:45:30.415461+0000","last_active":"2023-11-09T03:45:35.432312+0000","last_peered":"2023-11-09T03:45:35.432312+0000","last_clean":"2023-11-09T03:45:35.432312+0000","last_became_active":"2023-11-09T03:45:30.415185+0000","last_became_peered":"2023-11-09T03:45:30.415185+0000","last_unstale":"2023-11-09T03:45:35.432312+0000","last_undegraded":"2023-11-09T03:45:35.432312+0000","last_fullsized":"2023-11-09T03:45:35.432312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:07:01.257576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262941+0000","last_change":"2023-11-09T03:45:35.620742+0000","last_active":"2023-11-09T03:45:37.262941+0000","last_peered":"2023-11-09T03:45:37.262941+0000","last_clean":"2023-11-09T03:45:37.262941+0000","last_became_active":"2023-11-09T03:45:35.620350+0000","last_became_peered":"2023-11-09T03:45:35.620350+0000","last_unstale":"2023-11-09T03:45:37.262941+0000","last_undegraded":"2023-11-09T03:45:37.262941+0000","last_fullsized":"2023-11-09T03:45:37.262941+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:03:57.174574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.618702+0000","last_change":"2023-11-09T03:45:31.417119+0000","last_active":"2023-11-09T03:45:35.618702+0000","last_peered":"2023-11-09T03:45:35.618702+0000","last_clean":"2023-11-09T03:45:35.618702+0000","last_became_active":"2023-11-09T03:45:31.416750+0000","last_became_peered":"2023-11-09T03:45:31.416750+0000","last_unstale":"2023-11-09T03:45:35.618702+0000","last_undegraded":"2023-11-09T03:45:35.618702+0000","last_fullsized":"2023-11-09T03:45:35.618702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:55:29.945168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.436205+0000","last_change":"2023-11-09T03:45:30.410204+0000","last_active":"2023-11-09T03:45:35.436205+0000","last_peered":"2023-11-09T03:45:35.436205+0000","last_clean":"2023-11-09T03:45:35.436205+0000","last_became_active":"2023-11-09T03:45:30.409876+0000","last_became_peered":"2023-11-09T03:45:30.409876+0000","last_unstale":"2023-11-09T03:45:35.436205+0000","last_undegraded":"2023-11-09T03:45:35.436205+0000","last_fullsized":"2023-11-09T03:45:35.436205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:23:49.677277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388781+0000","last_change":"2023-11-09T03:45:35.446608+0000","last_active":"2023-11-09T03:45:36.388781+0000","last_peered":"2023-11-09T03:45:36.388781+0000","last_clean":"2023-11-09T03:45:36.388781+0000","last_became_active":"2023-11-09T03:45:35.446121+0000","last_became_peered":"2023-11-09T03:45:35.446121+0000","last_unstale":"2023-11-09T03:45:36.388781+0000","last_undegraded":"2023-11-09T03:45:36.388781+0000","last_fullsized":"2023-11-09T03:45:36.388781+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:12.380753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617394+0000","last_change":"2023-11-09T03:45:31.418010+0000","last_active":"2023-11-09T03:45:35.617394+0000","last_peered":"2023-11-09T03:45:35.617394+0000","last_clean":"2023-11-09T03:45:35.617394+0000","last_became_active":"2023-11-09T03:45:31.417758+0000","last_became_peered":"2023-11-09T03:45:31.417758+0000","last_unstale":"2023-11-09T03:45:35.617394+0000","last_undegraded":"2023-11-09T03:45:35.617394+0000","last_fullsized":"2023-11-09T03:45:35.617394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:02:27.095864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.432121+0000","last_change":"2023-11-09T03:45:30.412289+0000","last_active":"2023-11-09T03:45:35.432121+0000","last_peered":"2023-11-09T03:45:35.432121+0000","last_clean":"2023-11-09T03:45:35.432121+0000","last_became_active":"2023-11-09T03:45:30.412035+0000","last_became_peered":"2023-11-09T03:45:30.412035+0000","last_unstale":"2023-11-09T03:45:35.432121+0000","last_undegraded":"2023-11-09T03:45:35.432121+0000","last_fullsized":"2023-11-09T03:45:35.432121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:35:45.408209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262660+0000","last_change":"2023-11-09T03:45:35.450569+0000","last_active":"2023-11-09T03:45:37.262660+0000","last_peered":"2023-11-09T03:45:37.262660+0000","last_clean":"2023-11-09T03:45:37.262660+0000","last_became_active":"2023-11-09T03:45:35.450133+0000","last_became_peered":"2023-11-09T03:45:35.450133+0000","last_unstale":"2023-11-09T03:45:37.262660+0000","last_undegraded":"2023-11-09T03:45:37.262660+0000","last_fullsized":"2023-11-09T03:45:37.262660+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:13:08.339968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440213+0000","last_change":"2023-11-09T03:45:31.400690+0000","last_active":"2023-11-09T03:45:35.440213+0000","last_peered":"2023-11-09T03:45:35.440213+0000","last_clean":"2023-11-09T03:45:35.440213+0000","last_became_active":"2023-11-09T03:45:31.400345+0000","last_became_peered":"2023-11-09T03:45:31.400345+0000","last_unstale":"2023-11-09T03:45:35.440213+0000","last_undegraded":"2023-11-09T03:45:35.440213+0000","last_fullsized":"2023-11-09T03:45:35.440213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:54:30.809126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438759+0000","last_change":"2023-11-09T03:45:30.414344+0000","last_active":"2023-11-09T03:45:35.438759+0000","last_peered":"2023-11-09T03:45:35.438759+0000","last_clean":"2023-11-09T03:45:35.438759+0000","last_became_active":"2023-11-09T03:45:30.414070+0000","last_became_peered":"2023-11-09T03:45:30.414070+0000","last_unstale":"2023-11-09T03:45:35.438759+0000","last_undegraded":"2023-11-09T03:45:35.438759+0000","last_fullsized":"2023-11-09T03:45:35.438759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:06:25.266825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388898+0000","last_change":"2023-11-09T03:45:35.618460+0000","last_active":"2023-11-09T03:45:36.388898+0000","last_peered":"2023-11-09T03:45:36.388898+0000","last_clean":"2023-11-09T03:45:36.388898+0000","last_became_active":"2023-11-09T03:45:35.618089+0000","last_became_peered":"2023-11-09T03:45:35.618089+0000","last_unstale":"2023-11-09T03:45:36.388898+0000","last_undegraded":"2023-11-09T03:45:36.388898+0000","last_fullsized":"2023-11-09T03:45:36.388898+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:24:34.808267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617389+0000","last_change":"2023-11-09T03:45:31.418108+0000","last_active":"2023-11-09T03:45:35.617389+0000","last_peered":"2023-11-09T03:45:35.617389+0000","last_clean":"2023-11-09T03:45:35.617389+0000","last_became_active":"2023-11-09T03:45:31.417869+0000","last_became_peered":"2023-11-09T03:45:31.417869+0000","last_unstale":"2023-11-09T03:45:35.617389+0000","last_undegraded":"2023-11-09T03:45:35.617389+0000","last_fullsized":"2023-11-09T03:45:35.617389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:04:02.978744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440273+0000","last_change":"2023-11-09T03:45:30.414815+0000","last_active":"2023-11-09T03:45:35.440273+0000","last_peered":"2023-11-09T03:45:35.440273+0000","last_clean":"2023-11-09T03:45:35.440273+0000","last_became_active":"2023-11-09T03:45:30.414177+0000","last_became_peered":"2023-11-09T03:45:30.414177+0000","last_unstale":"2023-11-09T03:45:35.440273+0000","last_undegraded":"2023-11-09T03:45:35.440273+0000","last_fullsized":"2023-11-09T03:45:35.440273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:54:29.804379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388529+0000","last_change":"2023-11-09T03:45:35.618416+0000","last_active":"2023-11-09T03:45:36.388529+0000","last_peered":"2023-11-09T03:45:36.388529+0000","last_clean":"2023-11-09T03:45:36.388529+0000","last_became_active":"2023-11-09T03:45:35.618055+0000","last_became_peered":"2023-11-09T03:45:35.618055+0000","last_unstale":"2023-11-09T03:45:36.388529+0000","last_undegraded":"2023-11-09T03:45:36.388529+0000","last_fullsized":"2023-11-09T03:45:36.388529+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:16:21.023356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.443358+0000","last_change":"2023-11-09T03:45:31.404777+0000","last_active":"2023-11-09T03:45:35.443358+0000","last_peered":"2023-11-09T03:45:35.443358+0000","last_clean":"2023-11-09T03:45:35.443358+0000","last_became_active":"2023-11-09T03:45:31.404472+0000","last_became_peered":"2023-11-09T03:45:31.404472+0000","last_unstale":"2023-11-09T03:45:35.443358+0000","last_undegraded":"2023-11-09T03:45:35.443358+0000","last_fullsized":"2023-11-09T03:45:35.443358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:06:26.271572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.443117+0000","last_change":"2023-11-09T03:45:30.411464+0000","last_active":"2023-11-09T03:45:35.443117+0000","last_peered":"2023-11-09T03:45:35.443117+0000","last_clean":"2023-11-09T03:45:35.443117+0000","last_became_active":"2023-11-09T03:45:30.410430+0000","last_became_peered":"2023-11-09T03:45:30.410430+0000","last_unstale":"2023-11-09T03:45:35.443117+0000","last_undegraded":"2023-11-09T03:45:35.443117+0000","last_fullsized":"2023-11-09T03:45:35.443117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:48:49.911609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262935+0000","last_change":"2023-11-09T03:45:35.618861+0000","last_active":"2023-11-09T03:45:37.262935+0000","last_peered":"2023-11-09T03:45:37.262935+0000","last_clean":"2023-11-09T03:45:37.262935+0000","last_became_active":"2023-11-09T03:45:35.618394+0000","last_became_peered":"2023-11-09T03:45:35.618394+0000","last_unstale":"2023-11-09T03:45:37.262935+0000","last_undegraded":"2023-11-09T03:45:37.262935+0000","last_fullsized":"2023-11-09T03:45:37.262935+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:07:23.630255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617562+0000","last_change":"2023-11-09T03:45:31.407151+0000","last_active":"2023-11-09T03:45:35.617562+0000","last_peered":"2023-11-09T03:45:35.617562+0000","last_clean":"2023-11-09T03:45:35.617562+0000","last_became_active":"2023-11-09T03:45:31.406652+0000","last_became_peered":"2023-11-09T03:45:31.406652+0000","last_unstale":"2023-11-09T03:45:35.617562+0000","last_undegraded":"2023-11-09T03:45:35.617562+0000","last_fullsized":"2023-11-09T03:45:35.617562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:29:17.503411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:41.806449+0000","last_change":"2023-11-09T03:45:30.411676+0000","last_active":"2023-11-09T03:45:41.806449+0000","last_peered":"2023-11-09T03:45:41.806449+0000","last_clean":"2023-11-09T03:45:41.806449+0000","last_became_active":"2023-11-09T03:45:30.411379+0000","last_became_peered":"2023-11-09T03:45:30.411379+0000","last_unstale":"2023-11-09T03:45:41.806449+0000","last_undegraded":"2023-11-09T03:45:41.806449+0000","last_fullsized":"2023-11-09T03:45:41.806449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:34:48.400703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262972+0000","last_change":"2023-11-09T03:45:35.452073+0000","last_active":"2023-11-09T03:45:37.262972+0000","last_peered":"2023-11-09T03:45:37.262972+0000","last_clean":"2023-11-09T03:45:37.262972+0000","last_became_active":"2023-11-09T03:45:35.451714+0000","last_became_peered":"2023-11-09T03:45:35.451714+0000","last_unstale":"2023-11-09T03:45:37.262972+0000","last_undegraded":"2023-11-09T03:45:37.262972+0000","last_fullsized":"2023-11-09T03:45:37.262972+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:55:33.973355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.435954+0000","last_change":"2023-11-09T03:45:31.406706+0000","last_active":"2023-11-09T03:45:35.435954+0000","last_peered":"2023-11-09T03:45:35.435954+0000","last_clean":"2023-11-09T03:45:35.435954+0000","last_became_active":"2023-11-09T03:45:31.406437+0000","last_became_peered":"2023-11-09T03:45:31.406437+0000","last_unstale":"2023-11-09T03:45:35.435954+0000","last_undegraded":"2023-11-09T03:45:35.435954+0000","last_fullsized":"2023-11-09T03:45:35.435954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:48:50.916355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442326+0000","last_change":"2023-11-09T03:45:30.418063+0000","last_active":"2023-11-09T03:45:35.442326+0000","last_peered":"2023-11-09T03:45:35.442326+0000","last_clean":"2023-11-09T03:45:35.442326+0000","last_became_active":"2023-11-09T03:45:30.417792+0000","last_became_peered":"2023-11-09T03:45:30.417792+0000","last_unstale":"2023-11-09T03:45:35.442326+0000","last_undegraded":"2023-11-09T03:45:35.442326+0000","last_fullsized":"2023-11-09T03:45:35.442326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:06:25.266825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.389474+0000","last_change":"2023-11-09T03:45:35.446788+0000","last_active":"2023-11-09T03:45:36.389474+0000","last_peered":"2023-11-09T03:45:36.389474+0000","last_clean":"2023-11-09T03:45:36.389474+0000","last_became_active":"2023-11-09T03:45:35.446280+0000","last_became_peered":"2023-11-09T03:45:35.446280+0000","last_unstale":"2023-11-09T03:45:36.389474+0000","last_undegraded":"2023-11-09T03:45:36.389474+0000","last_fullsized":"2023-11-09T03:45:36.389474+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:29:21.531598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.444838+0000","last_change":"2023-11-09T03:45:31.415653+0000","last_active":"2023-11-09T03:45:35.444838+0000","last_peered":"2023-11-09T03:45:35.444838+0000","last_clean":"2023-11-09T03:45:35.444838+0000","last_became_active":"2023-11-09T03:45:31.415219+0000","last_became_peered":"2023-11-09T03:45:31.415219+0000","last_unstale":"2023-11-09T03:45:35.444838+0000","last_undegraded":"2023-11-09T03:45:35.444838+0000","last_fullsized":"2023-11-09T03:45:35.444838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:37:55.586534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440583+0000","last_change":"2023-11-09T03:45:30.414817+0000","last_active":"2023-11-09T03:45:35.440583+0000","last_peered":"2023-11-09T03:45:35.440583+0000","last_clean":"2023-11-09T03:45:35.440583+0000","last_became_active":"2023-11-09T03:45:30.414124+0000","last_became_peered":"2023-11-09T03:45:30.414124+0000","last_unstale":"2023-11-09T03:45:35.440583+0000","last_undegraded":"2023-11-09T03:45:35.440583+0000","last_fullsized":"2023-11-09T03:45:35.440583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:11:40.438915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388126+0000","last_change":"2023-11-09T03:45:35.448965+0000","last_active":"2023-11-09T03:45:36.388126+0000","last_peered":"2023-11-09T03:45:36.388126+0000","last_clean":"2023-11-09T03:45:36.388126+0000","last_became_active":"2023-11-09T03:45:35.448513+0000","last_became_peered":"2023-11-09T03:45:35.448513+0000","last_unstale":"2023-11-09T03:45:36.388126+0000","last_undegraded":"2023-11-09T03:45:36.388126+0000","last_fullsized":"2023-11-09T03:45:36.388126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:57:39.650773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440495+0000","last_change":"2023-11-09T03:45:31.407514+0000","last_active":"2023-11-09T03:45:35.440495+0000","last_peered":"2023-11-09T03:45:35.440495+0000","last_clean":"2023-11-09T03:45:35.440495+0000","last_became_active":"2023-11-09T03:45:31.407223+0000","last_became_peered":"2023-11-09T03:45:31.407223+0000","last_unstale":"2023-11-09T03:45:35.440495+0000","last_undegraded":"2023-11-09T03:45:35.440495+0000","last_fullsized":"2023-11-09T03:45:35.440495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:55:29.945168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440341+0000","last_change":"2023-11-09T03:45:30.418184+0000","last_active":"2023-11-09T03:45:35.440341+0000","last_peered":"2023-11-09T03:45:35.440341+0000","last_clean":"2023-11-09T03:45:35.440341+0000","last_became_active":"2023-11-09T03:45:30.417910+0000","last_became_peered":"2023-11-09T03:45:30.417910+0000","last_unstale":"2023-11-09T03:45:35.440341+0000","last_undegraded":"2023-11-09T03:45:35.440341+0000","last_fullsized":"2023-11-09T03:45:35.440341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:13:28.891277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388139+0000","last_change":"2023-11-09T03:45:35.621180+0000","last_active":"2023-11-09T03:45:36.388139+0000","last_peered":"2023-11-09T03:45:36.388139+0000","last_clean":"2023-11-09T03:45:36.388139+0000","last_became_active":"2023-11-09T03:45:35.620827+0000","last_became_peered":"2023-11-09T03:45:35.620827+0000","last_unstale":"2023-11-09T03:45:36.388139+0000","last_undegraded":"2023-11-09T03:45:36.388139+0000","last_fullsized":"2023-11-09T03:45:36.388139+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:57:57.109949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262540+0000","last_change":"2023-11-09T03:45:35.620240+0000","last_active":"2023-11-09T03:45:37.262540+0000","last_peered":"2023-11-09T03:45:37.262540+0000","last_clean":"2023-11-09T03:45:37.262540+0000","last_became_active":"2023-11-09T03:45:35.619863+0000","last_became_peered":"2023-11-09T03:45:35.619863+0000","last_unstale":"2023-11-09T03:45:37.262540+0000","last_undegraded":"2023-11-09T03:45:37.262540+0000","last_fullsized":"2023-11-09T03:45:37.262540+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:55:33.973355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441745+0000","last_change":"2023-11-09T03:45:31.399005+0000","last_active":"2023-11-09T03:45:35.441745+0000","last_peered":"2023-11-09T03:45:35.441745+0000","last_clean":"2023-11-09T03:45:35.441745+0000","last_became_active":"2023-11-09T03:45:31.398753+0000","last_became_peered":"2023-11-09T03:45:31.398753+0000","last_unstale":"2023-11-09T03:45:35.441745+0000","last_undegraded":"2023-11-09T03:45:35.441745+0000","last_fullsized":"2023-11-09T03:45:35.441745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:44.473424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439574+0000","last_change":"2023-11-09T03:45:30.414502+0000","last_active":"2023-11-09T03:45:35.439574+0000","last_peered":"2023-11-09T03:45:35.439574+0000","last_clean":"2023-11-09T03:45:35.439574+0000","last_became_active":"2023-11-09T03:45:30.414208+0000","last_became_peered":"2023-11-09T03:45:30.414208+0000","last_unstale":"2023-11-09T03:45:35.439574+0000","last_undegraded":"2023-11-09T03:45:35.439574+0000","last_fullsized":"2023-11-09T03:45:35.439574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:34:48.400703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.264189+0000","last_change":"2023-11-09T03:45:35.456456+0000","last_active":"2023-11-09T03:45:37.264189+0000","last_peered":"2023-11-09T03:45:37.264189+0000","last_clean":"2023-11-09T03:45:37.264189+0000","last_became_active":"2023-11-09T03:45:35.455918+0000","last_became_peered":"2023-11-09T03:45:35.455918+0000","last_unstale":"2023-11-09T03:45:37.264189+0000","last_undegraded":"2023-11-09T03:45:37.264189+0000","last_fullsized":"2023-11-09T03:45:37.264189+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:45.477938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438961+0000","last_change":"2023-11-09T03:45:31.402076+0000","last_active":"2023-11-09T03:45:35.438961+0000","last_peered":"2023-11-09T03:45:35.438961+0000","last_clean":"2023-11-09T03:45:35.438961+0000","last_became_active":"2023-11-09T03:45:31.401819+0000","last_became_peered":"2023-11-09T03:45:31.401819+0000","last_unstale":"2023-11-09T03:45:35.438961+0000","last_undegraded":"2023-11-09T03:45:35.438961+0000","last_fullsized":"2023-11-09T03:45:35.438961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:27.400068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439935+0000","last_change":"2023-11-09T03:45:30.410155+0000","last_active":"2023-11-09T03:45:35.439935+0000","last_peered":"2023-11-09T03:45:35.439935+0000","last_clean":"2023-11-09T03:45:35.439935+0000","last_became_active":"2023-11-09T03:45:30.409732+0000","last_became_peered":"2023-11-09T03:45:30.409732+0000","last_unstale":"2023-11-09T03:45:35.439935+0000","last_undegraded":"2023-11-09T03:45:35.439935+0000","last_fullsized":"2023-11-09T03:45:35.439935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:34:48.400703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438019+0000","last_change":"2023-11-09T03:45:30.414923+0000","last_active":"2023-11-09T03:45:35.438019+0000","last_peered":"2023-11-09T03:45:35.438019+0000","last_clean":"2023-11-09T03:45:35.438019+0000","last_became_active":"2023-11-09T03:45:30.414641+0000","last_became_peered":"2023-11-09T03:45:30.414641+0000","last_unstale":"2023-11-09T03:45:35.438019+0000","last_undegraded":"2023-11-09T03:45:35.438019+0000","last_fullsized":"2023-11-09T03:45:35.438019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:12:07.086030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.443025+0000","last_change":"2023-11-09T03:45:31.400895+0000","last_active":"2023-11-09T03:45:35.443025+0000","last_peered":"2023-11-09T03:45:35.443025+0000","last_clean":"2023-11-09T03:45:35.443025+0000","last_became_active":"2023-11-09T03:45:31.400383+0000","last_became_peered":"2023-11-09T03:45:31.400383+0000","last_unstale":"2023-11-09T03:45:35.443025+0000","last_undegraded":"2023-11-09T03:45:35.443025+0000","last_fullsized":"2023-11-09T03:45:35.443025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:08.352566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.261594+0000","last_change":"2023-11-09T03:45:35.445171+0000","last_active":"2023-11-09T03:45:37.261594+0000","last_peered":"2023-11-09T03:45:37.261594+0000","last_clean":"2023-11-09T03:45:37.261594+0000","last_became_active":"2023-11-09T03:45:35.444835+0000","last_became_peered":"2023-11-09T03:45:35.444835+0000","last_unstale":"2023-11-09T03:45:37.261594+0000","last_undegraded":"2023-11-09T03:45:37.261594+0000","last_fullsized":"2023-11-09T03:45:37.261594+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:13:33.924212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439602+0000","last_change":"2023-11-09T03:45:30.416307+0000","last_active":"2023-11-09T03:45:35.439602+0000","last_peered":"2023-11-09T03:45:35.439602+0000","last_clean":"2023-11-09T03:45:35.439602+0000","last_became_active":"2023-11-09T03:45:30.416012+0000","last_became_peered":"2023-11-09T03:45:30.416012+0000","last_unstale":"2023-11-09T03:45:35.439602+0000","last_undegraded":"2023-11-09T03:45:35.439602+0000","last_fullsized":"2023-11-09T03:45:35.439602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:13:28.891277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.435281+0000","last_change":"2023-11-09T03:45:31.395421+0000","last_active":"2023-11-09T03:45:35.435281+0000","last_peered":"2023-11-09T03:45:35.435281+0000","last_clean":"2023-11-09T03:45:35.435281+0000","last_became_active":"2023-11-09T03:45:31.395148+0000","last_became_peered":"2023-11-09T03:45:31.395148+0000","last_unstale":"2023-11-09T03:45:35.435281+0000","last_undegraded":"2023-11-09T03:45:35.435281+0000","last_fullsized":"2023-11-09T03:45:35.435281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:34:28.340170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388718+0000","last_change":"2023-11-09T03:45:35.454065+0000","last_active":"2023-11-09T03:45:36.388718+0000","last_peered":"2023-11-09T03:45:36.388718+0000","last_clean":"2023-11-09T03:45:36.388718+0000","last_became_active":"2023-11-09T03:45:35.453677+0000","last_became_peered":"2023-11-09T03:45:35.453677+0000","last_unstale":"2023-11-09T03:45:36.388718+0000","last_undegraded":"2023-11-09T03:45:36.388718+0000","last_fullsized":"2023-11-09T03:45:36.388718+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:34:18.265468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440121+0000","last_change":"2023-11-09T03:45:31.401829+0000","last_active":"2023-11-09T03:45:35.440121+0000","last_peered":"2023-11-09T03:45:35.440121+0000","last_clean":"2023-11-09T03:45:35.440121+0000","last_became_active":"2023-11-09T03:45:31.401545+0000","last_became_peered":"2023-11-09T03:45:31.401545+0000","last_unstale":"2023-11-09T03:45:35.440121+0000","last_undegraded":"2023-11-09T03:45:35.440121+0000","last_fullsized":"2023-11-09T03:45:35.440121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:27.400068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.432141+0000","last_change":"2023-11-09T03:45:31.410286+0000","last_active":"2023-11-09T03:45:35.432141+0000","last_peered":"2023-11-09T03:45:35.432141+0000","last_clean":"2023-11-09T03:45:35.432141+0000","last_became_active":"2023-11-09T03:45:31.410031+0000","last_became_peered":"2023-11-09T03:45:31.410031+0000","last_unstale":"2023-11-09T03:45:35.432141+0000","last_undegraded":"2023-11-09T03:45:35.432141+0000","last_fullsized":"2023-11-09T03:45:35.432141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:48:50.916355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440196+0000","last_change":"2023-11-09T03:45:31.412282+0000","last_active":"2023-11-09T03:45:35.440196+0000","last_peered":"2023-11-09T03:45:35.440196+0000","last_clean":"2023-11-09T03:45:35.440196+0000","last_became_active":"2023-11-09T03:45:31.408879+0000","last_became_peered":"2023-11-09T03:45:31.408879+0000","last_unstale":"2023-11-09T03:45:35.440196+0000","last_undegraded":"2023-11-09T03:45:35.440196+0000","last_fullsized":"2023-11-09T03:45:35.440196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:39:42.380963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442948+0000","last_change":"2023-11-09T03:45:31.400563+0000","last_active":"2023-11-09T03:45:35.442948+0000","last_peered":"2023-11-09T03:45:35.442948+0000","last_clean":"2023-11-09T03:45:35.442948+0000","last_became_active":"2023-11-09T03:45:31.399929+0000","last_became_peered":"2023-11-09T03:45:31.399929+0000","last_unstale":"2023-11-09T03:45:35.442948+0000","last_undegraded":"2023-11-09T03:45:35.442948+0000","last_fullsized":"2023-11-09T03:45:35.442948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:48:08.925647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439486+0000","last_change":"2023-11-09T03:45:31.397966+0000","last_active":"2023-11-09T03:45:35.439486+0000","last_peered":"2023-11-09T03:45:35.439486+0000","last_clean":"2023-11-09T03:45:35.439486+0000","last_became_active":"2023-11-09T03:45:31.397681+0000","last_became_peered":"2023-11-09T03:45:31.397681+0000","last_unstale":"2023-11-09T03:45:35.439486+0000","last_undegraded":"2023-11-09T03:45:35.439486+0000","last_fullsized":"2023-11-09T03:45:35.439486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:22.653144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442230+0000","last_change":"2023-11-09T03:45:31.398359+0000","last_active":"2023-11-09T03:45:35.442230+0000","last_peered":"2023-11-09T03:45:35.442230+0000","last_clean":"2023-11-09T03:45:35.442230+0000","last_became_active":"2023-11-09T03:45:31.398104+0000","last_became_peered":"2023-11-09T03:45:31.398104+0000","last_unstale":"2023-11-09T03:45:35.442230+0000","last_undegraded":"2023-11-09T03:45:35.442230+0000","last_fullsized":"2023-11-09T03:45:35.442230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:08.352566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440856+0000","last_change":"2023-11-09T03:45:31.408644+0000","last_active":"2023-11-09T03:45:35.440856+0000","last_peered":"2023-11-09T03:45:35.440856+0000","last_clean":"2023-11-09T03:45:35.440856+0000","last_became_active":"2023-11-09T03:45:31.408353+0000","last_became_peered":"2023-11-09T03:45:31.408353+0000","last_unstale":"2023-11-09T03:45:35.440856+0000","last_undegraded":"2023-11-09T03:45:35.440856+0000","last_fullsized":"2023-11-09T03:45:35.440856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:36:23.593484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438417+0000","last_change":"2023-11-09T03:45:31.395367+0000","last_active":"2023-11-09T03:45:35.438417+0000","last_peered":"2023-11-09T03:45:35.438417+0000","last_clean":"2023-11-09T03:45:35.438417+0000","last_became_active":"2023-11-09T03:45:31.395059+0000","last_became_peered":"2023-11-09T03:45:31.395059+0000","last_unstale":"2023-11-09T03:45:35.438417+0000","last_undegraded":"2023-11-09T03:45:35.438417+0000","last_fullsized":"2023-11-09T03:45:35.438417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:39:36.741003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439489+0000","last_change":"2023-11-09T03:45:31.408626+0000","last_active":"2023-11-09T03:45:35.439489+0000","last_peered":"2023-11-09T03:45:35.439489+0000","last_clean":"2023-11-09T03:45:35.439489+0000","last_became_active":"2023-11-09T03:45:31.408085+0000","last_became_peered":"2023-11-09T03:45:31.408085+0000","last_unstale":"2023-11-09T03:45:35.439489+0000","last_undegraded":"2023-11-09T03:45:35.439489+0000","last_fullsized":"2023-11-09T03:45:35.439489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:08.352566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439241+0000","last_change":"2023-11-09T03:45:31.402017+0000","last_active":"2023-11-09T03:45:35.439241+0000","last_peered":"2023-11-09T03:45:35.439241+0000","last_clean":"2023-11-09T03:45:35.439241+0000","last_became_active":"2023-11-09T03:45:31.401773+0000","last_became_peered":"2023-11-09T03:45:31.401773+0000","last_unstale":"2023-11-09T03:45:35.439241+0000","last_undegraded":"2023-11-09T03:45:35.439241+0000","last_fullsized":"2023-11-09T03:45:35.439241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:39:42.380963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440106+0000","last_change":"2023-11-09T03:45:31.418163+0000","last_active":"2023-11-09T03:45:35.440106+0000","last_peered":"2023-11-09T03:45:35.440106+0000","last_clean":"2023-11-09T03:45:35.440106+0000","last_became_active":"2023-11-09T03:45:31.417880+0000","last_became_peered":"2023-11-09T03:45:31.417880+0000","last_unstale":"2023-11-09T03:45:35.440106+0000","last_undegraded":"2023-11-09T03:45:35.440106+0000","last_fullsized":"2023-11-09T03:45:35.440106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:39:36.741003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442845+0000","last_change":"2023-11-09T03:45:31.404842+0000","last_active":"2023-11-09T03:45:35.442845+0000","last_peered":"2023-11-09T03:45:35.442845+0000","last_clean":"2023-11-09T03:45:35.442845+0000","last_became_active":"2023-11-09T03:45:31.404484+0000","last_became_peered":"2023-11-09T03:45:31.404484+0000","last_unstale":"2023-11-09T03:45:35.442845+0000","last_undegraded":"2023-11-09T03:45:35.442845+0000","last_fullsized":"2023-11-09T03:45:35.442845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:27.400068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440223+0000","last_change":"2023-11-09T03:45:31.404211+0000","last_active":"2023-11-09T03:45:35.440223+0000","last_peered":"2023-11-09T03:45:35.440223+0000","last_clean":"2023-11-09T03:45:35.440223+0000","last_became_active":"2023-11-09T03:45:31.403962+0000","last_became_peered":"2023-11-09T03:45:31.403962+0000","last_unstale":"2023-11-09T03:45:35.440223+0000","last_undegraded":"2023-11-09T03:45:35.440223+0000","last_fullsized":"2023-11-09T03:45:35.440223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:48:08.925647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.619170+0000","last_change":"2023-11-09T03:45:31.407273+0000","last_active":"2023-11-09T03:45:35.619170+0000","last_peered":"2023-11-09T03:45:35.619170+0000","last_clean":"2023-11-09T03:45:35.619170+0000","last_became_active":"2023-11-09T03:45:31.406956+0000","last_became_peered":"2023-11-09T03:45:31.406956+0000","last_unstale":"2023-11-09T03:45:35.619170+0000","last_undegraded":"2023-11-09T03:45:35.619170+0000","last_fullsized":"2023-11-09T03:45:35.619170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:13:04.311780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.432241+0000","last_change":"2023-11-09T03:45:31.402525+0000","last_active":"2023-11-09T03:45:35.432241+0000","last_peered":"2023-11-09T03:45:35.432241+0000","last_clean":"2023-11-09T03:45:35.432241+0000","last_became_active":"2023-11-09T03:45:31.402262+0000","last_became_peered":"2023-11-09T03:45:31.402262+0000","last_unstale":"2023-11-09T03:45:35.432241+0000","last_undegraded":"2023-11-09T03:45:35.432241+0000","last_fullsized":"2023-11-09T03:45:35.432241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:09.007153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441402+0000","last_change":"2023-11-09T03:45:31.411781+0000","last_active":"2023-11-09T03:45:35.441402+0000","last_peered":"2023-11-09T03:45:35.441402+0000","last_clean":"2023-11-09T03:45:35.441402+0000","last_became_active":"2023-11-09T03:45:31.411178+0000","last_became_peered":"2023-11-09T03:45:31.411178+0000","last_unstale":"2023-11-09T03:45:35.441402+0000","last_undegraded":"2023-11-09T03:45:35.441402+0000","last_fullsized":"2023-11-09T03:45:35.441402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:13:04.311780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439393+0000","last_change":"2023-11-09T03:45:31.409602+0000","last_active":"2023-11-09T03:45:35.439393+0000","last_peered":"2023-11-09T03:45:35.439393+0000","last_clean":"2023-11-09T03:45:35.439393+0000","last_became_active":"2023-11-09T03:45:31.409313+0000","last_became_peered":"2023-11-09T03:45:31.409313+0000","last_unstale":"2023-11-09T03:45:35.439393+0000","last_undegraded":"2023-11-09T03:45:35.439393+0000","last_fullsized":"2023-11-09T03:45:35.439393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:48:08.925647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441829+0000","last_change":"2023-11-09T03:45:31.394559+0000","last_active":"2023-11-09T03:45:35.441829+0000","last_peered":"2023-11-09T03:45:35.441829+0000","last_clean":"2023-11-09T03:45:35.441829+0000","last_became_active":"2023-11-09T03:45:31.394284+0000","last_became_peered":"2023-11-09T03:45:31.394284+0000","last_unstale":"2023-11-09T03:45:35.441829+0000","last_undegraded":"2023-11-09T03:45:35.441829+0000","last_fullsized":"2023-11-09T03:45:35.441829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:11:10.514126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.443281+0000","last_change":"2023-11-09T03:45:31.397341+0000","last_active":"2023-11-09T03:45:35.443281+0000","last_peered":"2023-11-09T03:45:35.443281+0000","last_clean":"2023-11-09T03:45:35.443281+0000","last_became_active":"2023-11-09T03:45:31.396986+0000","last_became_peered":"2023-11-09T03:45:31.396986+0000","last_unstale":"2023-11-09T03:45:35.443281+0000","last_undegraded":"2023-11-09T03:45:35.443281+0000","last_fullsized":"2023-11-09T03:45:35.443281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:34:49.405450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439412+0000","last_change":"2023-11-09T03:45:31.402346+0000","last_active":"2023-11-09T03:45:35.439412+0000","last_peered":"2023-11-09T03:45:35.439412+0000","last_clean":"2023-11-09T03:45:35.439412+0000","last_became_active":"2023-11-09T03:45:31.401630+0000","last_became_peered":"2023-11-09T03:45:31.401630+0000","last_unstale":"2023-11-09T03:45:35.439412+0000","last_undegraded":"2023-11-09T03:45:35.439412+0000","last_fullsized":"2023-11-09T03:45:35.439412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:02:27.095864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.435980+0000","last_change":"2023-11-09T03:45:31.403882+0000","last_active":"2023-11-09T03:45:35.435980+0000","last_peered":"2023-11-09T03:45:35.435980+0000","last_clean":"2023-11-09T03:45:35.435980+0000","last_became_active":"2023-11-09T03:45:31.403637+0000","last_became_peered":"2023-11-09T03:45:31.403637+0000","last_unstale":"2023-11-09T03:45:35.435980+0000","last_undegraded":"2023-11-09T03:45:35.435980+0000","last_fullsized":"2023-11-09T03:45:35.435980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:39:30.327813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438776+0000","last_change":"2023-11-09T03:45:31.415358+0000","last_active":"2023-11-09T03:45:35.438776+0000","last_peered":"2023-11-09T03:45:35.438776+0000","last_clean":"2023-11-09T03:45:35.438776+0000","last_became_active":"2023-11-09T03:45:31.415018+0000","last_became_peered":"2023-11-09T03:45:31.415018+0000","last_unstale":"2023-11-09T03:45:35.438776+0000","last_undegraded":"2023-11-09T03:45:35.438776+0000","last_fullsized":"2023-11-09T03:45:35.438776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:08.352566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440770+0000","last_change":"2023-11-09T03:45:31.414693+0000","last_active":"2023-11-09T03:45:35.440770+0000","last_peered":"2023-11-09T03:45:35.440770+0000","last_clean":"2023-11-09T03:45:35.440770+0000","last_became_active":"2023-11-09T03:45:31.414400+0000","last_became_peered":"2023-11-09T03:45:31.414400+0000","last_unstale":"2023-11-09T03:45:35.440770+0000","last_undegraded":"2023-11-09T03:45:35.440770+0000","last_fullsized":"2023-11-09T03:45:35.440770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:22.653144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.618910+0000","last_change":"2023-11-09T03:45:31.419560+0000","last_active":"2023-11-09T03:45:35.618910+0000","last_peered":"2023-11-09T03:45:35.618910+0000","last_clean":"2023-11-09T03:45:35.618910+0000","last_became_active":"2023-11-09T03:45:31.418455+0000","last_became_peered":"2023-11-09T03:45:31.418455+0000","last_unstale":"2023-11-09T03:45:35.618910+0000","last_undegraded":"2023-11-09T03:45:35.618910+0000","last_fullsized":"2023-11-09T03:45:35.618910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:14:55.198635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438871+0000","last_change":"2023-11-09T03:45:31.415460+0000","last_active":"2023-11-09T03:45:35.438871+0000","last_peered":"2023-11-09T03:45:35.438871+0000","last_clean":"2023-11-09T03:45:35.438871+0000","last_became_active":"2023-11-09T03:45:31.415210+0000","last_became_peered":"2023-11-09T03:45:31.415210+0000","last_unstale":"2023-11-09T03:45:35.438871+0000","last_undegraded":"2023-11-09T03:45:35.438871+0000","last_fullsized":"2023-11-09T03:45:35.438871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:44.473424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617339+0000","last_change":"2023-11-09T03:45:31.411588+0000","last_active":"2023-11-09T03:45:35.617339+0000","last_peered":"2023-11-09T03:45:35.617339+0000","last_clean":"2023-11-09T03:45:35.617339+0000","last_became_active":"2023-11-09T03:45:31.411168+0000","last_became_peered":"2023-11-09T03:45:31.411168+0000","last_unstale":"2023-11-09T03:45:35.617339+0000","last_undegraded":"2023-11-09T03:45:35.617339+0000","last_fullsized":"2023-11-09T03:45:35.617339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:39:42.380963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440027+0000","last_change":"2023-11-09T03:45:31.412481+0000","last_active":"2023-11-09T03:45:35.440027+0000","last_peered":"2023-11-09T03:45:35.440027+0000","last_clean":"2023-11-09T03:45:35.440027+0000","last_became_active":"2023-11-09T03:45:31.412167+0000","last_became_peered":"2023-11-09T03:45:31.412167+0000","last_unstale":"2023-11-09T03:45:35.440027+0000","last_undegraded":"2023-11-09T03:45:35.440027+0000","last_fullsized":"2023-11-09T03:45:35.440027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:22.653144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617339+0000","last_change":"2023-11-09T03:45:31.405459+0000","last_active":"2023-11-09T03:45:35.617339+0000","last_peered":"2023-11-09T03:45:35.617339+0000","last_clean":"2023-11-09T03:45:35.617339+0000","last_became_active":"2023-11-09T03:45:31.405188+0000","last_became_peered":"2023-11-09T03:45:31.405188+0000","last_unstale":"2023-11-09T03:45:35.617339+0000","last_undegraded":"2023-11-09T03:45:35.617339+0000","last_fullsized":"2023-11-09T03:45:35.617339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:39:36.741003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440927+0000","last_change":"2023-11-09T03:45:31.417756+0000","last_active":"2023-11-09T03:45:35.440927+0000","last_peered":"2023-11-09T03:45:35.440927+0000","last_clean":"2023-11-09T03:45:35.440927+0000","last_became_active":"2023-11-09T03:45:31.417488+0000","last_became_peered":"2023-11-09T03:45:31.417488+0000","last_unstale":"2023-11-09T03:45:35.440927+0000","last_undegraded":"2023-11-09T03:45:35.440927+0000","last_fullsized":"2023-11-09T03:45:35.440927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:36:23.593484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442615+0000","last_change":"2023-11-09T03:45:31.397411+0000","last_active":"2023-11-09T03:45:35.442615+0000","last_peered":"2023-11-09T03:45:35.442615+0000","last_clean":"2023-11-09T03:45:35.442615+0000","last_became_active":"2023-11-09T03:45:31.396779+0000","last_became_peered":"2023-11-09T03:45:31.396779+0000","last_unstale":"2023-11-09T03:45:35.442615+0000","last_undegraded":"2023-11-09T03:45:35.442615+0000","last_fullsized":"2023-11-09T03:45:35.442615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:54:30.809126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441937+0000","last_change":"2023-11-09T03:45:31.396447+0000","last_active":"2023-11-09T03:45:35.441937+0000","last_peered":"2023-11-09T03:45:35.441937+0000","last_clean":"2023-11-09T03:45:35.441937+0000","last_became_active":"2023-11-09T03:45:31.396184+0000","last_became_peered":"2023-11-09T03:45:31.396184+0000","last_unstale":"2023-11-09T03:45:35.441937+0000","last_undegraded":"2023-11-09T03:45:35.441937+0000","last_fullsized":"2023-11-09T03:45:35.441937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:48:08.925647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.618840+0000","last_change":"2023-11-09T03:45:31.405664+0000","last_active":"2023-11-09T03:45:35.618840+0000","last_peered":"2023-11-09T03:45:35.618840+0000","last_clean":"2023-11-09T03:45:35.618840+0000","last_became_active":"2023-11-09T03:45:31.404726+0000","last_became_peered":"2023-11-09T03:45:31.404726+0000","last_unstale":"2023-11-09T03:45:35.618840+0000","last_undegraded":"2023-11-09T03:45:35.618840+0000","last_fullsized":"2023-11-09T03:45:35.618840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:41:08.550864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":695413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":104553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":104589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":104623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":868352,"data_stored":694850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-09T03:45:49.760 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-09T03:45:49.760 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-09T03:45:49.761 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-09T03:45:49.761 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-09T03:45:49.762 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-09T03:45:49.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.894+0000 7f93e530f640 1 Processor -- start 2023-11-09T03:45:49.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.894+0000 7f93e530f640 1 -- start start 2023-11-09T03:45:49.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.895+0000 7f93e530f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f93e00cde60 0x7f93e015d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:49.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.895+0000 7f93e530f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93e015daa0 0x7f93e0157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:49.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.895+0000 7f93e530f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e015e000 con 0x7f93e00ce240 2023-11-09T03:45:49.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.895+0000 7f93e530f640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93e015e5c0 con 0x7f93e00cde60 2023-11-09T03:45:49.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.895+0000 7f93e530f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e015e2e0 con 0x7f93e015daa0 2023-11-09T03:45:49.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.895+0000 7f93deffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f93e00cde60 0x7f93e015d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:49.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.895+0000 7f93d7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93e015daa0 0x7f93e0157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:49.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.895+0000 7f93deffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f93e00cde60 0x7f93e015d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:33838/0 (socket says 172.21.15.76:33838) 2023-11-09T03:45:49.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.895+0000 7f93deffd640 1 -- 172.21.15.76:0/2534582242 learned_addr learned my addr 172.21.15.76:0/2534582242 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:49.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.895+0000 7f93df7fe640 1 --1- 172.21.15.76:0/2534582242 >> v1:172.21.15.113:6789/0 conn(0x7f93e00ce240 0x7f93e006b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:46032/0 (socket says 172.21.15.76:46032) 2023-11-09T03:45:49.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.895+0000 7f93deffd640 1 -- 172.21.15.76:0/2534582242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93e015daa0 msgr2=0x7f93e0157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:49.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.895+0000 7f93deffd640 1 --2- 172.21.15.76:0/2534582242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93e015daa0 0x7f93e0157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.895+0000 7f93deffd640 1 -- 172.21.15.76:0/2534582242 >> v1:172.21.15.113:6789/0 conn(0x7f93e00ce240 legacy=0x7f93e006b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:49.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.895+0000 7f93deffd640 1 -- 172.21.15.76:0/2534582242 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e00d1120 con 0x7f93e00cde60 2023-11-09T03:45:49.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.895+0000 7f93d7fff640 1 --2- 172.21.15.76:0/2534582242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93e015daa0 0x7f93e0157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:45:49.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.896+0000 7f93deffd640 1 --2- 172.21.15.76:0/2534582242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93e00cde60 0x7f93e015d560 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009870 tx=0x7f93c802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3cde7f1f7393b4a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:49.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.896+0000 7f93de7fc640 1 -- 172.21.15.76:0/2534582242 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c8030840 con 0x7f93e00cde60 2023-11-09T03:45:49.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.896+0000 7f93de7fc640 1 -- 172.21.15.76:0/2534582242 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93c80309a0 con 0x7f93e00cde60 2023-11-09T03:45:49.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.896+0000 7f93de7fc640 1 -- 172.21.15.76:0/2534582242 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c8030d30 con 0x7f93e00cde60 2023-11-09T03:45:49.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.897+0000 7f93e530f640 1 -- 172.21.15.76:0/2534582242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93e00cde60 msgr2=0x7f93e015d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:49.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.897+0000 7f93e530f640 1 --2- 172.21.15.76:0/2534582242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93e00cde60 0x7f93e015d560 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009870 tx=0x7f93c802b980 comp rx=0 tx=0).stop 2023-11-09T03:45:49.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.897+0000 7f93e530f640 1 -- 172.21.15.76:0/2534582242 shutdown_connections 2023-11-09T03:45:49.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.897+0000 7f93e530f640 1 --2- 172.21.15.76:0/2534582242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93e015daa0 0x7f93e0157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.897+0000 7f93e530f640 1 --2- 172.21.15.76:0/2534582242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93e00cde60 0x7f93e015d560 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.897+0000 7f93e530f640 1 -- 172.21.15.76:0/2534582242 >> 172.21.15.76:0/2534582242 conn(0x7f93e0071f30 msgr2=0x7f93e00cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:49.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.897+0000 7f93e530f640 1 -- 172.21.15.76:0/2534582242 shutdown_connections 2023-11-09T03:45:49.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.897+0000 7f93e530f640 1 -- 172.21.15.76:0/2534582242 wait complete. 2023-11-09T03:45:49.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.898+0000 7f93e530f640 1 Processor -- start 2023-11-09T03:45:49.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.898+0000 7f93e530f640 1 -- start start 2023-11-09T03:45:49.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.899+0000 7f93e530f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93e00cde60 0x7f93e01f2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:49.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.899+0000 7f93e530f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93e00ce240 0x7f93e015c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:49.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.899+0000 7f93e530f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93e015daa0 0x7f93e015aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:49.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.899+0000 7f93e530f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e01f3530 con 0x7f93e00cde60 2023-11-09T03:45:49.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.899+0000 7f93e530f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e01f2a60 con 0x7f93e015daa0 2023-11-09T03:45:49.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.899+0000 7f93e530f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e015cc80 con 0x7f93e00ce240 2023-11-09T03:45:49.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.899+0000 7f93deffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93e00cde60 0x7f93e01f2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:49.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.899+0000 7f93deffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93e00cde60 0x7f93e01f2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57632/0 (socket says 172.21.15.76:57632) 2023-11-09T03:45:49.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.899+0000 7f93deffd640 1 -- 172.21.15.76:0/2758574035 learned_addr learned my addr 172.21.15.76:0/2758574035 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:49.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.899+0000 7f93df7fe640 1 --2- 172.21.15.76:0/2758574035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93e015daa0 0x7f93e015aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:49.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.899+0000 7f93d7fff640 1 --2- 172.21.15.76:0/2758574035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93e00ce240 0x7f93e015c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:49.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.899+0000 7f93deffd640 1 -- 172.21.15.76:0/2758574035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93e00ce240 msgr2=0x7f93e015c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:49.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.899+0000 7f93deffd640 1 --2- 172.21.15.76:0/2758574035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93e00ce240 0x7f93e015c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.899+0000 7f93deffd640 1 -- 172.21.15.76:0/2758574035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93e015daa0 msgr2=0x7f93e015aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:49.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.899+0000 7f93deffd640 1 --2- 172.21.15.76:0/2758574035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93e015daa0 0x7f93e015aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:49.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.899+0000 7f93deffd640 1 -- 172.21.15.76:0/2758574035 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d0009400 con 0x7f93e00cde60 2023-11-09T03:45:49.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.899+0000 7f93deffd640 1 --2- 172.21.15.76:0/2758574035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93e00cde60 0x7f93e01f2520 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f93c8000c00 tx=0x7f93c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:49.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.900+0000 7f93d77fe640 1 -- 172.21.15.76:0/2758574035 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c8039c90 con 0x7f93e00cde60 2023-11-09T03:45:49.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.900+0000 7f93d77fe640 1 -- 172.21.15.76:0/2758574035 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93c8039df0 con 0x7f93e00cde60 2023-11-09T03:45:49.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.900+0000 7f93e530f640 1 -- 172.21.15.76:0/2758574035 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93c80094d0 con 0x7f93e00cde60 2023-11-09T03:45:49.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.900+0000 7f93d77fe640 1 -- 172.21.15.76:0/2758574035 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c8043640 con 0x7f93e00cde60 2023-11-09T03:45:49.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.900+0000 7f93e530f640 1 -- 172.21.15.76:0/2758574035 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e01f2f70 con 0x7f93e00cde60 2023-11-09T03:45:49.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.901+0000 7f93e530f640 1 -- 172.21.15.76:0/2758574035 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e0074290 con 0x7f93e00cde60 2023-11-09T03:45:49.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.901+0000 7f93d77fe640 1 -- 172.21.15.76:0/2758574035 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f93c80437a0 con 0x7f93e00cde60 2023-11-09T03:45:49.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.902+0000 7f93d77fe640 1 --2- 172.21.15.76:0/2758574035 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f93c006ae70 0x7f93c006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:49.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.902+0000 7f93d77fe640 1 -- 172.21.15.76:0/2758574035 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f93c803e070 con 0x7f93e00cde60 2023-11-09T03:45:49.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.902+0000 7f93d7fff640 1 --2- 172.21.15.76:0/2758574035 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f93c006ae70 0x7f93c006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:49.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.902+0000 7f93d7fff640 1 --2- 172.21.15.76:0/2758574035 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f93c006ae70 0x7f93c006d330 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f93e00c7810 tx=0x7f93d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:49.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:49.906+0000 7f93d77fe640 1 -- 172.21.15.76:0/2758574035 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f93c80776d0 con 0x7f93e00cde60 2023-11-09T03:45:50.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.116+0000 7f93e530f640 1 -- 172.21.15.76:0/2758574035 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f93e01559c0 con 0x7f93e00cde60 2023-11-09T03:45:50.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.118+0000 7f93d77fe640 1 -- 172.21.15.76:0/2758574035 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171037 (secure 0 0 0) 0x7f93c807acf0 con 0x7f93e00cde60 2023-11-09T03:45:50.120 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:50.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.121+0000 7f93e530f640 1 -- 172.21.15.76:0/2758574035 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f93c006ae70 msgr2=0x7f93c006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.121+0000 7f93e530f640 1 --2- 172.21.15.76:0/2758574035 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f93c006ae70 0x7f93c006d330 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f93e00c7810 tx=0x7f93d0008040 comp rx=0 tx=0).stop 2023-11-09T03:45:50.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.121+0000 7f93e530f640 1 -- 172.21.15.76:0/2758574035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93e00cde60 msgr2=0x7f93e01f2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.121+0000 7f93e530f640 1 --2- 172.21.15.76:0/2758574035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93e00cde60 0x7f93e01f2520 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f93c8000c00 tx=0x7f93c8004730 comp rx=0 tx=0).stop 2023-11-09T03:45:50.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.121+0000 7f93e530f640 1 -- 172.21.15.76:0/2758574035 shutdown_connections 2023-11-09T03:45:50.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.121+0000 7f93e530f640 1 --2- 172.21.15.76:0/2758574035 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f93c006ae70 0x7f93c006d330 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.121+0000 7f93e530f640 1 --2- 172.21.15.76:0/2758574035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93e015daa0 0x7f93e015aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.121+0000 7f93e530f640 1 --2- 172.21.15.76:0/2758574035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93e00ce240 0x7f93e015c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.121+0000 7f93e530f640 1 --2- 172.21.15.76:0/2758574035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93e00cde60 0x7f93e01f2520 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.121+0000 7f93e530f640 1 -- 172.21.15.76:0/2758574035 >> 172.21.15.76:0/2758574035 conn(0x7f93e0071f30 msgr2=0x7f93e00c6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:50.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.121+0000 7f93e530f640 1 -- 172.21.15.76:0/2758574035 shutdown_connections 2023-11-09T03:45:50.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.121+0000 7f93e530f640 1 -- 172.21.15.76:0/2758574035 wait complete. 2023-11-09T03:45:50.131 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":5,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1264193042},{"type":"v1","addr":"172.21.15.113:6833","nonce":1264193042}]},"active_addr":"172.21.15.113:6833/1264193042","active_change":"2023-11-09T03:45:10.738224+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:v0.0.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:v0.0.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":327552482}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":3806467894}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":423831890}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":3837099116}]}]} 2023-11-09T03:45:50.135 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-09T03:45:50.135 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-09T03:45:50.135 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:45:50.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.235+0000 7fa29994c640 1 Processor -- start 2023-11-09T03:45:50.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.235+0000 7fa29994c640 1 -- start start 2023-11-09T03:45:50.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.236+0000 7fa29994c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa2940cce10 0x7fa2940b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.236+0000 7fa29994c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2940b18b0 0x7fa29415e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.236+0000 7fa29994c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa29415e6e0 con 0x7fa29406a120 2023-11-09T03:45:50.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.236+0000 7fa29994c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa29415dd10 con 0x7fa2940cce10 2023-11-09T03:45:50.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.236+0000 7fa29994c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa29415d720 con 0x7fa2940b18b0 2023-11-09T03:45:50.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.236+0000 7fa292ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa2940cce10 0x7fa2940b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.236+0000 7fa2937fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa29406a120 0x7fa29415ea90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:56742/0 (socket says 172.21.15.76:56742) 2023-11-09T03:45:50.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.236+0000 7fa2937fe640 1 -- 172.21.15.76:0/160609544 learned_addr learned my addr 172.21.15.76:0/160609544 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:50.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.237+0000 7fa2927fc640 1 --2- 172.21.15.76:0/160609544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2940b18b0 0x7fa29415e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.237+0000 7fa292ffd640 1 -- 172.21.15.76:0/160609544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2940b18b0 msgr2=0x7fa29415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:50.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.237+0000 7fa292ffd640 1 --2- 172.21.15.76:0/160609544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2940b18b0 0x7fa29415e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.237+0000 7fa292ffd640 1 -- 172.21.15.76:0/160609544 >> v1:172.21.15.76:6789/0 conn(0x7fa29406a120 legacy=0x7fa29415ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.237+0000 7fa292ffd640 1 -- 172.21.15.76:0/160609544 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa29406fe00 con 0x7fa2940cce10 2023-11-09T03:45:50.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.237+0000 7fa291ffb640 1 -- 172.21.15.76:0/160609544 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 793593926 0 0) 0x7fa28c001110 con 0x7fa29406a120 2023-11-09T03:45:50.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.237+0000 7fa2927fc640 1 --2- 172.21.15.76:0/160609544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2940b18b0 0x7fa29415e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:50.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.237+0000 7fa292ffd640 1 --2- 172.21.15.76:0/160609544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2940cce10 0x7fa2940b1fc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa284009dc0 tx=0x7fa28402bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a585d0955271dec server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:50.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.237+0000 7fa291ffb640 1 -- 172.21.15.76:0/160609544 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28402cb20 con 0x7fa2940cce10 2023-11-09T03:45:50.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.238+0000 7fa291ffb640 1 -- 172.21.15.76:0/160609544 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa28402cc80 con 0x7fa2940cce10 2023-11-09T03:45:50.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.238+0000 7fa291ffb640 1 -- 172.21.15.76:0/160609544 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa284034440 con 0x7fa2940cce10 2023-11-09T03:45:50.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.238+0000 7fa29994c640 1 -- 172.21.15.76:0/160609544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2940cce10 msgr2=0x7fa2940b1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.238+0000 7fa29994c640 1 --2- 172.21.15.76:0/160609544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2940cce10 0x7fa2940b1fc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa284009dc0 tx=0x7fa28402bd70 comp rx=0 tx=0).stop 2023-11-09T03:45:50.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.238+0000 7fa29994c640 1 -- 172.21.15.76:0/160609544 shutdown_connections 2023-11-09T03:45:50.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.238+0000 7fa29994c640 1 --2- 172.21.15.76:0/160609544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2940b18b0 0x7fa29415e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.238+0000 7fa29994c640 1 --2- 172.21.15.76:0/160609544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2940cce10 0x7fa2940b1fc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.238+0000 7fa29994c640 1 -- 172.21.15.76:0/160609544 >> 172.21.15.76:0/160609544 conn(0x7fa29406d8b0 msgr2=0x7fa29406dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:50.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.238+0000 7fa29994c640 1 -- 172.21.15.76:0/160609544 shutdown_connections 2023-11-09T03:45:50.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.238+0000 7fa29994c640 1 -- 172.21.15.76:0/160609544 wait complete. 2023-11-09T03:45:50.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.239+0000 7fa29994c640 1 Processor -- start 2023-11-09T03:45:50.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.239+0000 7fa29994c640 1 -- start start 2023-11-09T03:45:50.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.240+0000 7fa29994c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa29406a120 0x7fa2941f20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.240+0000 7fa29994c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2940b18b0 0x7fa2940c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.240+0000 7fa29994c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2940cce10 0x7fa2940c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.240+0000 7fa29994c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2940c5150 con 0x7fa2940cce10 2023-11-09T03:45:50.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.240+0000 7fa29994c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2940c4540 con 0x7fa29406a120 2023-11-09T03:45:50.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.240+0000 7fa29994c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2940c4820 con 0x7fa2940b18b0 2023-11-09T03:45:50.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.240+0000 7fa2937fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2940cce10 0x7fa2940c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.240+0000 7fa2937fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2940cce10 0x7fa2940c5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57640/0 (socket says 172.21.15.76:57640) 2023-11-09T03:45:50.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.240+0000 7fa2937fe640 1 -- 172.21.15.76:0/995768315 learned_addr learned my addr 172.21.15.76:0/995768315 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:50.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.240+0000 7fa2927fc640 1 --2- 172.21.15.76:0/995768315 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2940b18b0 0x7fa2940c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.240+0000 7fa292ffd640 1 --2- 172.21.15.76:0/995768315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa29406a120 0x7fa2941f20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.240+0000 7fa2937fe640 1 -- 172.21.15.76:0/995768315 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2940b18b0 msgr2=0x7fa2940c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.240+0000 7fa2937fe640 1 --2- 172.21.15.76:0/995768315 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2940b18b0 0x7fa2940c59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.240+0000 7fa2937fe640 1 -- 172.21.15.76:0/995768315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa29406a120 msgr2=0x7fa2941f20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.240+0000 7fa2937fe640 1 --2- 172.21.15.76:0/995768315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa29406a120 0x7fa2941f20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.240+0000 7fa2937fe640 1 -- 172.21.15.76:0/995768315 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa284009a70 con 0x7fa2940cce10 2023-11-09T03:45:50.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.241+0000 7fa2937fe640 1 --2- 172.21.15.76:0/995768315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2940cce10 0x7fa2940c5f30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa28c001eb0 tx=0x7fa28c00c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:50.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.241+0000 7fa27bfff640 1 -- 172.21.15.76:0/995768315 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28c0074e0 con 0x7fa2940cce10 2023-11-09T03:45:50.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.241+0000 7fa27bfff640 1 -- 172.21.15.76:0/995768315 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa28c007640 con 0x7fa2940cce10 2023-11-09T03:45:50.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.241+0000 7fa29994c640 1 -- 172.21.15.76:0/995768315 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2940c4b00 con 0x7fa2940cce10 2023-11-09T03:45:50.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.241+0000 7fa27bfff640 1 -- 172.21.15.76:0/995768315 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28c004a00 con 0x7fa2940cce10 2023-11-09T03:45:50.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.241+0000 7fa29994c640 1 -- 172.21.15.76:0/995768315 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2941f2600 con 0x7fa2940cce10 2023-11-09T03:45:50.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.242+0000 7fa29994c640 1 -- 172.21.15.76:0/995768315 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2940c5300 con 0x7fa2940cce10 2023-11-09T03:45:50.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.242+0000 7fa27bfff640 1 -- 172.21.15.76:0/995768315 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa28c004b60 con 0x7fa2940cce10 2023-11-09T03:45:50.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.243+0000 7fa27bfff640 1 --2- 172.21.15.76:0/995768315 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa28006ae70 0x7fa28006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.243+0000 7fa27bfff640 1 -- 172.21.15.76:0/995768315 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa28c0895b0 con 0x7fa2940cce10 2023-11-09T03:45:50.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.243+0000 7fa292ffd640 1 --2- 172.21.15.76:0/995768315 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa28006ae70 0x7fa28006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.243+0000 7fa292ffd640 1 --2- 172.21.15.76:0/995768315 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa28006ae70 0x7fa28006d330 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa284036000 tx=0x7fa284030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:50.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.247+0000 7fa27bfff640 1 -- 172.21.15.76:0/995768315 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa2940c5300 con 0x7fa2940cce10 2023-11-09T03:45:50.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.414+0000 7fa29994c640 1 -- 172.21.15.76:0/995768315 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa29415e6e0 con 0x7fa2940cce10 2023-11-09T03:45:50.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.415+0000 7fa27bfff640 1 -- 172.21.15.76:0/995768315 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18334 (secure 0 0 0) 0x7fa29415e6e0 con 0x7fa2940cce10 2023-11-09T03:45:50.415 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:50.415 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":25,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:45:35.781752+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:45:50.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.417+0000 7fa29994c640 1 -- 172.21.15.76:0/995768315 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa28006ae70 msgr2=0x7fa28006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.417+0000 7fa29994c640 1 --2- 172.21.15.76:0/995768315 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa28006ae70 0x7fa28006d330 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa284036000 tx=0x7fa284030000 comp rx=0 tx=0).stop 2023-11-09T03:45:50.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.417+0000 7fa29994c640 1 -- 172.21.15.76:0/995768315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2940cce10 msgr2=0x7fa2940c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.417+0000 7fa29994c640 1 --2- 172.21.15.76:0/995768315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2940cce10 0x7fa2940c5f30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa28c001eb0 tx=0x7fa28c00c870 comp rx=0 tx=0).stop 2023-11-09T03:45:50.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.417+0000 7fa29994c640 1 -- 172.21.15.76:0/995768315 shutdown_connections 2023-11-09T03:45:50.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.417+0000 7fa29994c640 1 --2- 172.21.15.76:0/995768315 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa28006ae70 0x7fa28006d330 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.417+0000 7fa29994c640 1 --2- 172.21.15.76:0/995768315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2940cce10 0x7fa2940c5f30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.417+0000 7fa29994c640 1 --2- 172.21.15.76:0/995768315 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2940b18b0 0x7fa2940c59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.417+0000 7fa29994c640 1 --2- 172.21.15.76:0/995768315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa29406a120 0x7fa2941f20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.417+0000 7fa29994c640 1 -- 172.21.15.76:0/995768315 >> 172.21.15.76:0/995768315 conn(0x7fa29406d8b0 msgr2=0x7fa2940b5200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:50.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.417+0000 7fa29994c640 1 -- 172.21.15.76:0/995768315 shutdown_connections 2023-11-09T03:45:50.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.417+0000 7fa29994c640 1 -- 172.21.15.76:0/995768315 wait complete. 2023-11-09T03:45:50.427 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-09T03:45:50.428 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:45:50.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.530+0000 7f4328207640 1 Processor -- start 2023-11-09T03:45:50.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.530+0000 7f4328207640 1 -- start start 2023-11-09T03:45:50.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.531+0000 7f4328207640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f43200ce240 0x7f4320157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.531+0000 7f4328207640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4320157540 0x7f43200d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.531+0000 7f4328207640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f432015e000 con 0x7f43200cde60 2023-11-09T03:45:50.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.531+0000 7f4328207640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f432015e5c0 con 0x7f4320157540 2023-11-09T03:45:50.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.531+0000 7f4328207640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f432015e2e0 con 0x7f43200ce240 2023-11-09T03:45:50.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.531+0000 7f43267a4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4320157540 0x7f43200d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.531+0000 7f43267a4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4320157540 0x7f43200d1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57642/0 (socket says 172.21.15.76:57642) 2023-11-09T03:45:50.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.531+0000 7f43267a4640 1 -- 172.21.15.76:0/3357197797 learned_addr learned my addr 172.21.15.76:0/3357197797 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:50.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.532+0000 7f43257a2640 1 --2- 172.21.15.76:0/3357197797 >> v2:172.21.15.113:3300/0 conn(0x7f43200ce240 0x7f4320157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.532+0000 7f43267a4640 1 -- 172.21.15.76:0/3357197797 >> v2:172.21.15.113:3300/0 conn(0x7f43200ce240 msgr2=0x7f4320157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:50.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.532+0000 7f43267a4640 1 --2- 172.21.15.76:0/3357197797 >> v2:172.21.15.113:3300/0 conn(0x7f43200ce240 0x7f4320157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.532+0000 7f43267a4640 1 -- 172.21.15.76:0/3357197797 >> v1:172.21.15.113:6789/0 conn(0x7f43200cde60 legacy=0x7f432015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.532+0000 7f43267a4640 1 -- 172.21.15.76:0/3357197797 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43200d1120 con 0x7f4320157540 2023-11-09T03:45:50.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.532+0000 7f43257a2640 1 --2- 172.21.15.76:0/3357197797 >> v2:172.21.15.113:3300/0 conn(0x7f43200ce240 0x7f4320157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:50.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.532+0000 7f43267a4640 1 --2- 172.21.15.76:0/3357197797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4320157540 0x7f43200d1920 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f43200cf5f0 tx=0x7f431802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108d0682f3193d7f server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:50.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.533+0000 7f4324fa1640 1 -- 172.21.15.76:0/3357197797 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431802cb20 con 0x7f4320157540 2023-11-09T03:45:50.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.533+0000 7f4324fa1640 1 -- 172.21.15.76:0/3357197797 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431802cc80 con 0x7f4320157540 2023-11-09T03:45:50.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.533+0000 7f4324fa1640 1 -- 172.21.15.76:0/3357197797 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43180343f0 con 0x7f4320157540 2023-11-09T03:45:50.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.533+0000 7f4328207640 1 -- 172.21.15.76:0/3357197797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4320157540 msgr2=0x7f43200d1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.533+0000 7f4328207640 1 --2- 172.21.15.76:0/3357197797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4320157540 0x7f43200d1920 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f43200cf5f0 tx=0x7f431802bf80 comp rx=0 tx=0).stop 2023-11-09T03:45:50.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.533+0000 7f4328207640 1 -- 172.21.15.76:0/3357197797 shutdown_connections 2023-11-09T03:45:50.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.533+0000 7f4328207640 1 --2- 172.21.15.76:0/3357197797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4320157540 0x7f43200d1920 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.533+0000 7f4328207640 1 --2- 172.21.15.76:0/3357197797 >> v2:172.21.15.113:3300/0 conn(0x7f43200ce240 0x7f4320157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.533+0000 7f4328207640 1 -- 172.21.15.76:0/3357197797 >> 172.21.15.76:0/3357197797 conn(0x7f4320071f30 msgr2=0x7f43200cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:50.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.533+0000 7f4328207640 1 -- 172.21.15.76:0/3357197797 shutdown_connections 2023-11-09T03:45:50.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.533+0000 7f4328207640 1 -- 172.21.15.76:0/3357197797 wait complete. 2023-11-09T03:45:50.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.534+0000 7f4328207640 1 Processor -- start 2023-11-09T03:45:50.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.534+0000 7f4328207640 1 -- start start 2023-11-09T03:45:50.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.535+0000 7f4328207640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43200cde60 0x7f43201595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.535+0000 7f4328207640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43200ce240 0x7f4320147ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.535+0000 7f4328207640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4320157540 0x7f43200c7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.535+0000 7f4328207640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4320148400 con 0x7f4320157540 2023-11-09T03:45:50.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.535+0000 7f4328207640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4320148710 con 0x7f43200ce240 2023-11-09T03:45:50.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.535+0000 7f4328207640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43201489d0 con 0x7f43200cde60 2023-11-09T03:45:50.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.535+0000 7f43267a4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4320157540 0x7f43200c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.535+0000 7f43267a4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4320157540 0x7f43200c7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57652/0 (socket says 172.21.15.76:57652) 2023-11-09T03:45:50.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.535+0000 7f43267a4640 1 -- 172.21.15.76:0/3485025677 learned_addr learned my addr 172.21.15.76:0/3485025677 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:50.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.535+0000 7f43257a2640 1 --2- 172.21.15.76:0/3485025677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43200ce240 0x7f4320147ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.535+0000 7f43267a4640 1 -- 172.21.15.76:0/3485025677 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43200cde60 msgr2=0x7f43201595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.535+0000 7f43267a4640 1 --2- 172.21.15.76:0/3485025677 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43200cde60 0x7f43201595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.535+0000 7f43267a4640 1 -- 172.21.15.76:0/3485025677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43200ce240 msgr2=0x7f4320147ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.535+0000 7f43267a4640 1 --2- 172.21.15.76:0/3485025677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43200ce240 0x7f4320147ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.535+0000 7f43267a4640 1 -- 172.21.15.76:0/3485025677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4318009b30 con 0x7f4320157540 2023-11-09T03:45:50.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.536+0000 7f43267a4640 1 --2- 172.21.15.76:0/3485025677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4320157540 0x7f43200c7960 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4318036040 tx=0x7f4318004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:50.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.536+0000 7f43167fc640 1 -- 172.21.15.76:0/3485025677 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4318007e70 con 0x7f4320157540 2023-11-09T03:45:50.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.536+0000 7f43167fc640 1 -- 172.21.15.76:0/3485025677 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4318009510 con 0x7f4320157540 2023-11-09T03:45:50.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.536+0000 7f43167fc640 1 -- 172.21.15.76:0/3485025677 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431803e690 con 0x7f4320157540 2023-11-09T03:45:50.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.536+0000 7f4328207640 1 -- 172.21.15.76:0/3485025677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4320148cb0 con 0x7f4320157540 2023-11-09T03:45:50.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.536+0000 7f4328207640 1 -- 172.21.15.76:0/3485025677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4320149270 con 0x7f4320157540 2023-11-09T03:45:50.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.537+0000 7f4328207640 1 -- 172.21.15.76:0/3485025677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4320074290 con 0x7f4320157540 2023-11-09T03:45:50.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.537+0000 7f43167fc640 1 -- 172.21.15.76:0/3485025677 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f431803e7f0 con 0x7f4320157540 2023-11-09T03:45:50.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.537+0000 7f43167fc640 1 --2- 172.21.15.76:0/3485025677 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f430406aea0 0x7f430406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.538+0000 7f43167fc640 1 -- 172.21.15.76:0/3485025677 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f43180ac4a0 con 0x7f4320157540 2023-11-09T03:45:50.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.538+0000 7f4325fa3640 1 --2- 172.21.15.76:0/3485025677 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f430406aea0 0x7f430406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.538+0000 7f4325fa3640 1 --2- 172.21.15.76:0/3485025677 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f430406aea0 0x7f430406d360 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f431c000f60 tx=0x7f431c0079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:50.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.542+0000 7f43167fc640 1 -- 172.21.15.76:0/3485025677 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4318077260 con 0x7f4320157540 2023-11-09T03:45:50.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.709+0000 7f4328207640 1 -- 172.21.15.76:0/3485025677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43200c65e0 con 0x7f4320157540 2023-11-09T03:45:50.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.710+0000 7f43167fc640 1 -- 172.21.15.76:0/3485025677 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18334 (secure 0 0 0) 0x7f431807a880 con 0x7f4320157540 2023-11-09T03:45:50.709 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:50.710 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":25,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:45:35.781752+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:45:50.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.712+0000 7f4328207640 1 -- 172.21.15.76:0/3485025677 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f430406aea0 msgr2=0x7f430406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.712+0000 7f4328207640 1 --2- 172.21.15.76:0/3485025677 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f430406aea0 0x7f430406d360 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f431c000f60 tx=0x7f431c0079b0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.712+0000 7f4328207640 1 -- 172.21.15.76:0/3485025677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4320157540 msgr2=0x7f43200c7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.712+0000 7f4328207640 1 --2- 172.21.15.76:0/3485025677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4320157540 0x7f43200c7960 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4318036040 tx=0x7f4318004a40 comp rx=0 tx=0).stop 2023-11-09T03:45:50.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.712+0000 7f4328207640 1 -- 172.21.15.76:0/3485025677 shutdown_connections 2023-11-09T03:45:50.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.712+0000 7f4328207640 1 --2- 172.21.15.76:0/3485025677 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f430406aea0 0x7f430406d360 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.712+0000 7f4328207640 1 --2- 172.21.15.76:0/3485025677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4320157540 0x7f43200c7960 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.712+0000 7f4328207640 1 --2- 172.21.15.76:0/3485025677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43200ce240 0x7f4320147ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.712+0000 7f4328207640 1 --2- 172.21.15.76:0/3485025677 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43200cde60 0x7f43201595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.712+0000 7f4328207640 1 -- 172.21.15.76:0/3485025677 >> 172.21.15.76:0/3485025677 conn(0x7f4320071f30 msgr2=0x7f43200d2330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:50.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.712+0000 7f4328207640 1 -- 172.21.15.76:0/3485025677 shutdown_connections 2023-11-09T03:45:50.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.712+0000 7f4328207640 1 -- 172.21.15.76:0/3485025677 wait complete. 2023-11-09T03:45:50.722 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-09T03:45:50.723 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-09T03:45:50.723 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-09T03:45:50.724 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-09T03:45:50.724 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-09T03:45:50.725 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-09T03:45:50.726 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-09T03:45:50.726 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-09T03:45:50.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.896+0000 7fd24ffa9640 1 Processor -- start 2023-11-09T03:45:50.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.896+0000 7f2c1f9c5640 1 Processor -- start 2023-11-09T03:45:50.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.896+0000 7f2c1f9c5640 1 -- start start 2023-11-09T03:45:50.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.897+0000 7fe7f0be2640 1 Processor -- start 2023-11-09T03:45:50.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.897+0000 7f9e1ef61640 1 Processor -- start 2023-11-09T03:45:50.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.897+0000 7fd24ffa9640 1 -- start start 2023-11-09T03:45:50.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.897+0000 7fe7f0be2640 1 -- start start 2023-11-09T03:45:50.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.897+0000 7f9e1ef61640 1 -- start start 2023-11-09T03:45:50.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.897+0000 7fa8966c3640 1 Processor -- start 2023-11-09T03:45:50.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.898+0000 7f52390df640 1 Processor -- start 2023-11-09T03:45:50.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.898+0000 7f52390df640 1 -- start start 2023-11-09T03:45:50.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.898+0000 7f86a915e640 1 Processor -- start 2023-11-09T03:45:50.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.898+0000 7f86a915e640 1 -- start start 2023-11-09T03:45:50.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.898+0000 7f2c1f9c5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2c180b4f90 0x7f2c180b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.898+0000 7f2c1f9c5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c180b5c50 0x7f2c18164e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.898+0000 7f2c1f9c5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c181653c0 con 0x7f2c18161150 2023-11-09T03:45:50.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.898+0000 7f2c1f9c5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c18165e90 con 0x7f2c180b4f90 2023-11-09T03:45:50.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.898+0000 7f2c1f9c5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c180b60a0 con 0x7f2c180b5c50 2023-11-09T03:45:50.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.898+0000 7f2c1d761640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2c18161150 0x7f2c180b4880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:56752/0 (socket says 172.21.15.76:56752) 2023-11-09T03:45:50.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.898+0000 7fe7f0be2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe7ec0c4690 0x7fe7ec0cc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.898+0000 7fe7f0be2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7ec070710 0x7fe7ec070b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.898+0000 7fe7f0be2640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7ec0c4b20 con 0x7fe7ec164760 2023-11-09T03:45:50.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.898+0000 7fe7f0be2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7ec164b40 con 0x7fe7ec0c4690 2023-11-09T03:45:50.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.898+0000 7fe7f0be2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec071080 con 0x7fe7ec070710 2023-11-09T03:45:50.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.898+0000 7f2c1d761640 1 -- 172.21.15.76:0/629512508 learned_addr learned my addr 172.21.15.76:0/629512508 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:50.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.898+0000 7f9e1ef61640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9e18071fa0 0x7f9e1806a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.898+0000 7fd24ffa9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd248142480 0x7fd2480cac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.898+0000 7f2c1cf60640 1 --2- 172.21.15.76:0/629512508 >> v2:172.21.15.113:3300/0 conn(0x7f2c180b4f90 0x7f2c180b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.898+0000 7fe7e9d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe7ec0c4690 0x7fe7ec0cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7fe7ea59c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe7ec164760 0x7fe7ec0b1860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:46042/0 (socket says 172.21.15.76:46042) 2023-11-09T03:45:50.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7fe7ea59c640 1 -- 172.21.15.76:0/3740347638 learned_addr learned my addr 172.21.15.76:0/3740347638 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:50.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.898+0000 7fe7ead9d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7ec070710 0x7fe7ec070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7fd24dd45640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd248141e20 0x7fd2480ca530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:56756/0 (socket says 172.21.15.76:56756) 2023-11-09T03:45:50.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7fd24dd45640 1 -- 172.21.15.76:0/3307225535 learned_addr learned my addr 172.21.15.76:0/3307225535 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:50.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7f2c1df62640 1 --2- 172.21.15.76:0/629512508 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c180b5c50 0x7f2c18164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7fd24d544640 1 --2- 172.21.15.76:0/3307225535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd248142480 0x7fd2480cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7fd24ffa9640 1 --2- 172.21.15.76:0/3307225535 >> v2:172.21.15.76:3300/0 conn(0x7fd2481429d0 0x7fd2480cd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7fd24ffa9640 1 -- 172.21.15.76:0/3307225535 --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2480cc960 con 0x7fd248141e20 2023-11-09T03:45:50.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7fd24ffa9640 1 -- 172.21.15.76:0/3307225535 --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2480ccc40 con 0x7fd2481429d0 2023-11-09T03:45:50.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7fe7e9d9b640 1 -- 172.21.15.76:0/3740347638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7ec070710 msgr2=0x7fe7ec070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:50.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7fe7e9d9b640 1 --2- 172.21.15.76:0/3740347638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7ec070710 0x7fe7ec070b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.898+0000 7fa8966c3640 1 -- start start 2023-11-09T03:45:50.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7fd24ffa9640 1 -- 172.21.15.76:0/3307225535 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2480ccf20 con 0x7fd248142480 2023-11-09T03:45:50.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7fe7e9d9b640 1 -- 172.21.15.76:0/3740347638 >> v1:172.21.15.113:6789/0 conn(0x7fe7ec164760 legacy=0x7fe7ec0b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7fe7e9d9b640 1 -- 172.21.15.76:0/3740347638 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ec0cca60 con 0x7fe7ec0c4690 2023-11-09T03:45:50.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7fa8966c3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8900cea70 0x7fa890164db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7f52390df640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f522c006370 0x7f522c03ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7f52390df640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f522c050350 0x7f522c0eb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7f52390df640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f522c008490 con 0x7f522c006750 2023-11-09T03:45:50.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7f52390df640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f522c0dfb00 con 0x7f522c006370 2023-11-09T03:45:50.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7f52390df640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f522c006ca0 con 0x7f522c050350 2023-11-09T03:45:50.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7fa8966c3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa8900cee50 0x7fa8900d21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7f52388de640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f522c050350 0x7f522c0eb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7f52388de640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f522c050350 0x7f522c0eb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:50910/0 (socket says 172.21.15.76:50910) 2023-11-09T03:45:50.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7f52388de640 1 -- 172.21.15.76:0/2054894763 learned_addr learned my addr 172.21.15.76:0/2054894763 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:50.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7f9e1ccfd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9e18071fa0 0x7f9e1806a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7f9e1ccfd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9e18071fa0 0x7f9e1806a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57660/0 (socket says 172.21.15.76:57660) 2023-11-09T03:45:50.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7f9e17fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f9e180b2010 0x7f9e180cc3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:56764/0 (socket says 172.21.15.76:56764) 2023-11-09T03:45:50.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7fab49490640 1 Processor -- start 2023-11-09T03:45:50.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7fab49490640 1 -- start start 2023-11-09T03:45:50.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7fa8956c1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8900cea70 0x7fa890164db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7fa894ec0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa8900cee50 0x7fa8900d21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7fa895ec2640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa8900b1840 0x7fa8900cf9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:46048/0 (socket says 172.21.15.76:46048) 2023-11-09T03:45:50.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7fa895ec2640 1 -- 172.21.15.76:0/968403825 learned_addr learned my addr 172.21.15.76:0/968403825 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:50.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7fa8966c3640 1 -- 172.21.15.76:0/968403825 --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa890165600 con 0x7fa8900b1840 2023-11-09T03:45:50.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7fa8966c3640 1 -- 172.21.15.76:0/968403825 --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8901658e0 con 0x7fa8900cee50 2023-11-09T03:45:50.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7fa8966c3640 1 -- 172.21.15.76:0/968403825 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa890165bc0 con 0x7fa8900cea70 2023-11-09T03:45:50.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7f5233fff640 1 --2- 172.21.15.76:0/2054894763 >> v2:172.21.15.113:3300/0 conn(0x7f522c006370 0x7f522c03ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7f86a915e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f86a40b4f90 0x7f86a40b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7f86a915e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f86a40b5c50 0x7f86a4164e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7f86a915e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86a41653c0 con 0x7f86a4161150 2023-11-09T03:45:50.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7f86a915e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86a4165e90 con 0x7f86a40b4f90 2023-11-09T03:45:50.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7f86a915e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a40b60a0 con 0x7f86a40b5c50 2023-11-09T03:45:50.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7f86a259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f86a40b4f90 0x7f86a40b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7f9e1ef61640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9e1806a660 0x7f9e180ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7f86a359e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f86a40b5c50 0x7f86a4164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7f86a359e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f86a40b5c50 0x7f86a4164e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:50926/0 (socket says 172.21.15.76:50926) 2023-11-09T03:45:50.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7f86a359e640 1 -- 172.21.15.76:0/1739514065 learned_addr learned my addr 172.21.15.76:0/1739514065 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:50.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7f2c1cf60640 1 -- 172.21.15.76:0/629512508 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c180b5c50 msgr2=0x7f2c18164e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:50.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7f9e17fff640 1 -- 172.21.15.76:0/3434098291 learned_addr learned my addr 172.21.15.76:0/3434098291 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:50.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7fd24e546640 1 --2- 172.21.15.76:0/3307225535 >> v2:172.21.15.76:3300/0 conn(0x7fd2481429d0 0x7fd2480cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7fd24cd43640 1 -- 172.21.15.76:0/3307225535 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2422108487 0 0) 0x7fd2480cc960 con 0x7fd248141e20 2023-11-09T03:45:50.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7fa887fff640 1 -- 172.21.15.76:0/968403825 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2006506431 0 0) 0x7fa890165600 con 0x7fa8900b1840 2023-11-09T03:45:50.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7fa887fff640 1 -- 172.21.15.76:0/968403825 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa87c003520 con 0x7fa8900b1840 2023-11-09T03:45:50.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7fe7e9d9b640 1 --2- 172.21.15.76:0/3740347638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7ec0c4690 0x7fe7ec0cc520 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009d90 tx=0x7fe7e402be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd77c367ada1a14 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:50.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7f2c1cf60640 1 --2- 172.21.15.76:0/629512508 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c180b5c50 0x7f2c18164e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7f2c1cf60640 1 -- 172.21.15.76:0/629512508 >> v1:172.21.15.76:6789/0 conn(0x7f2c18161150 legacy=0x7f2c180b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.899+0000 7f2c1cf60640 1 -- 172.21.15.76:0/629512508 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c181655f0 con 0x7f2c180b4f90 2023-11-09T03:45:50.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7f86a259c640 1 -- 172.21.15.76:0/1739514065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f86a40b5c50 msgr2=0x7f86a4164e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:50.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7f86a259c640 1 --2- 172.21.15.76:0/1739514065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f86a40b5c50 0x7f86a4164e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7fd24cd43640 1 -- 172.21.15.76:0/3307225535 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd234003520 con 0x7fd248141e20 2023-11-09T03:45:50.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7f86a259c640 1 -- 172.21.15.76:0/1739514065 >> v1:172.21.15.113:6789/0 conn(0x7f86a4161150 legacy=0x7f86a40b4880 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-09T03:45:50.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7f86a259c640 1 -- 172.21.15.76:0/1739514065 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a41655f0 con 0x7f86a40b4f90 2023-11-09T03:45:50.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7f86a359e640 1 --2- 172.21.15.76:0/1739514065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f86a40b5c50 0x7f86a4164e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:45:50.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7f2c1cf60640 1 --2- 172.21.15.76:0/629512508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c180b4f90 0x7f2c180b2270 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2c140097a0 tx=0x7f2c1402b660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aed9a10ce854f54d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:50.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7f9e1ef61640 1 -- 172.21.15.76:0/3434098291 --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e180cdfb0 con 0x7f9e180b2010 2023-11-09T03:45:50.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7f9e1ef61640 1 -- 172.21.15.76:0/3434098291 --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e1806ab00 con 0x7f9e18071fa0 2023-11-09T03:45:50.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7f9e1ef61640 1 -- 172.21.15.76:0/3434098291 --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e180cd680 con 0x7f9e1806a660 2023-11-09T03:45:50.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7f9e1d4fe640 1 --2- 172.21.15.76:0/3434098291 >> v2:172.21.15.113:3300/0 conn(0x7f9e1806a660 0x7f9e180ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7fa887fff640 1 -- 172.21.15.76:0/968403825 <== mon.1 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3024318207 0 0) 0x7fa87c003520 con 0x7fa8900b1840 2023-11-09T03:45:50.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7fa887fff640 1 -- 172.21.15.76:0/968403825 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8900cea70 msgr2=0x7fa890164db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:50.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7fa887fff640 1 --2- 172.21.15.76:0/968403825 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8900cea70 0x7fa890164db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7fa887fff640 1 -- 172.21.15.76:0/968403825 >> v2:172.21.15.113:3300/0 conn(0x7fa8900cee50 msgr2=0x7fa8900d21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:50.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7fa887fff640 1 --2- 172.21.15.76:0/968403825 >> v2:172.21.15.113:3300/0 conn(0x7fa8900cee50 0x7fa8900d21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fa887fff640 1 -- 172.21.15.76:0/968403825 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8900d19c0 con 0x7fa8900b1840 2023-11-09T03:45:50.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fa887fff640 1 -- 172.21.15.76:0/968403825 <== mon.1 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fa88c001670 con 0x7fa8900b1840 2023-11-09T03:45:50.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7f52388de640 1 -- 172.21.15.76:0/2054894763 >> v2:172.21.15.113:3300/0 conn(0x7f522c006370 msgr2=0x7f522c03ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:50.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fa887fff640 1 -- 172.21.15.76:0/968403825 >> v1:172.21.15.113:6789/0 conn(0x7fa8900b1840 legacy=0x7fa8900cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fa887fff640 1 --2- 172.21.15.76:0/968403825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa87c003970 0x7fa87c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fa887fff640 1 --2- 172.21.15.76:0/968403825 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa87c0047c0 0x7fa87c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fa887fff640 1 --2- 172.21.15.76:0/968403825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa87c0052a0 0x7fa87c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fa887fff640 1 -- 172.21.15.76:0/968403825 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8901658e0 con 0x7fa87c0052a0 2023-11-09T03:45:50.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fa887fff640 1 -- 172.21.15.76:0/968403825 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa890165bc0 con 0x7fa87c003970 2023-11-09T03:45:50.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fa887fff640 1 -- 172.21.15.76:0/968403825 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c00c1d0 con 0x7fa87c0047c0 2023-11-09T03:45:50.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7f52388de640 1 --2- 172.21.15.76:0/2054894763 >> v2:172.21.15.113:3300/0 conn(0x7f522c006370 0x7f522c03ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7f52388de640 1 -- 172.21.15.76:0/2054894763 >> v1:172.21.15.76:6789/0 conn(0x7f522c006750 legacy=0x7f522c04fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.900+0000 7f52388de640 1 -- 172.21.15.76:0/2054894763 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f522c053dd0 con 0x7f522c050350 2023-11-09T03:45:50.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fd24cd43640 1 -- 172.21.15.76:0/3307225535 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2219341400 0 0) 0x7fd234003520 con 0x7fd248141e20 2023-11-09T03:45:50.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fd24d544640 1 -- 172.21.15.76:0/3307225535 >> v2:172.21.15.76:3300/0 conn(0x7fd2481429d0 msgr2=0x7fd2480cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:50.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fd24d544640 1 --2- 172.21.15.76:0/3307225535 >> v2:172.21.15.76:3300/0 conn(0x7fd2481429d0 0x7fd2480cd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7f9e1d4fe640 1 -- 172.21.15.76:0/3434098291 >> v2:172.21.15.76:3300/0 conn(0x7f9e18071fa0 msgr2=0x7f9e1806a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:50.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7f9e1d4fe640 1 --2- 172.21.15.76:0/3434098291 >> v2:172.21.15.76:3300/0 conn(0x7f9e18071fa0 0x7f9e1806a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fd24d544640 1 -- 172.21.15.76:0/3307225535 >> v1:172.21.15.76:6789/0 conn(0x7fd248141e20 legacy=0x7fd2480ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7f9e1d4fe640 1 -- 172.21.15.76:0/3434098291 >> v1:172.21.15.76:6789/0 conn(0x7f9e180b2010 legacy=0x7f9e180cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7f9e1d4fe640 1 -- 172.21.15.76:0/3434098291 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e180cd960 con 0x7f9e1806a660 2023-11-09T03:45:50.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fab49490640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fab44158c80 0x7fab440ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fab49490640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab4406ac10 0x7fab440cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fab49490640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fab44159060 0x7fab440ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fd24d544640 1 -- 172.21.15.76:0/3307225535 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2480cd200 con 0x7fd248142480 2023-11-09T03:45:50.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7f52388de640 1 --2- 172.21.15.76:0/2054894763 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f522c050350 0x7f522c0eb450 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f523402cce0 tx=0x7f52340a61d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df177e6e02804b1f server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:50.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fa8956c1640 1 --2- 172.21.15.76:0/968403825 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8900cea70 0x7fa890164db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:50.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fe7e959a640 1 -- 172.21.15.76:0/3740347638 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e402cbd0 con 0x7fe7ec0c4690 2023-11-09T03:45:50.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fab49490640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab440cb1e0 con 0x7fab44158c80 2023-11-09T03:45:50.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fe7e959a640 1 -- 172.21.15.76:0/3740347638 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e402cd30 con 0x7fe7ec0c4690 2023-11-09T03:45:50.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fe7e959a640 1 -- 172.21.15.76:0/3740347638 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e402cea0 con 0x7fe7ec0c4690 2023-11-09T03:45:50.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fab49490640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab441595d0 con 0x7fab44159060 2023-11-09T03:45:50.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fa8956c1640 1 --2- 172.21.15.76:0/968403825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa87c003970 0x7fa87c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fab427fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab4406ac10 0x7fab440cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fa894ec0640 1 --2- 172.21.15.76:0/968403825 >> v2:172.21.15.113:3300/0 conn(0x7fa8900cee50 0x7fa8900d21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:50.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fa895ec2640 1 --2- 172.21.15.76:0/968403825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa87c0052a0 0x7fa87c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fab437fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fab44159060 0x7fab440ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fab42ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fab44158c80 0x7fab440ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fab437fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fab44159060 0x7fab440ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:33946/0 (socket says 172.21.15.76:33946) 2023-11-09T03:45:50.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fab437fe640 1 -- 172.21.15.76:0/3022623767 learned_addr learned my addr 172.21.15.76:0/3022623767 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:50.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7fab49490640 1 -- 172.21.15.76:0/3022623767 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab440caf00 con 0x7fab4406ac10 2023-11-09T03:45:50.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.902+0000 7f2c0ffff640 1 -- 172.21.15.76:0/629512508 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c14030710 con 0x7f2c180b4f90 2023-11-09T03:45:50.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.902+0000 7fa894ec0640 1 --2- 172.21.15.76:0/968403825 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa87c0047c0 0x7fa87c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.901+0000 7f5232ffd640 1 -- 172.21.15.76:0/2054894763 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52340a6e30 con 0x7f522c050350 2023-11-09T03:45:50.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.902+0000 7f5232ffd640 1 -- 172.21.15.76:0/2054894763 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52340ae440 con 0x7f522c050350 2023-11-09T03:45:50.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.902+0000 7fa8956c1640 1 -- 172.21.15.76:0/968403825 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa87c0047c0 msgr2=0x7fa87c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:50.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.902+0000 7f2c0ffff640 1 -- 172.21.15.76:0/629512508 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c1402e050 con 0x7f2c180b4f90 2023-11-09T03:45:50.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.902+0000 7f2c0ffff640 1 -- 172.21.15.76:0/629512508 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c14030a90 con 0x7f2c180b4f90 2023-11-09T03:45:50.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.902+0000 7fa8956c1640 1 --2- 172.21.15.76:0/968403825 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa87c0047c0 0x7fa87c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.902+0000 7fa8956c1640 1 -- 172.21.15.76:0/968403825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa87c0052a0 msgr2=0x7fa87c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:50.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.902+0000 7fa8956c1640 1 --2- 172.21.15.76:0/968403825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa87c0052a0 0x7fa87c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.902+0000 7fa8956c1640 1 -- 172.21.15.76:0/968403825 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c003520 con 0x7fa87c003970 2023-11-09T03:45:50.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.902+0000 7f86a259c640 1 --2- 172.21.15.76:0/1739514065 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86a40b4f90 0x7f86a40b2270 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f86980097a0 tx=0x7f869802b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=889ea01aa9ca2ed7 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:50.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.902+0000 7f86a1d9b640 1 -- 172.21.15.76:0/1739514065 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869802f050 con 0x7f86a40b4f90 2023-11-09T03:45:50.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.902+0000 7fe7f0be2640 1 -- 172.21.15.76:0/3740347638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7ec0c4690 msgr2=0x7fe7ec0cc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.902+0000 7fe7f0be2640 1 --2- 172.21.15.76:0/3740347638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7ec0c4690 0x7fe7ec0cc520 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009d90 tx=0x7fe7e402be20 comp rx=0 tx=0).stop 2023-11-09T03:45:50.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.902+0000 7f9e1d4fe640 1 --2- 172.21.15.76:0/3434098291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e1806a660 0x7f9e180ccae0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9e18072900 tx=0x7f9e0802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38dcd457707ee56b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:50.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.902+0000 7fab437fe640 1 -- 172.21.15.76:0/3022623767 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab4406ac10 msgr2=0x7fab440cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:50.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.902+0000 7fab437fe640 1 --2- 172.21.15.76:0/3022623767 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab4406ac10 0x7fab440cddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.902+0000 7fab437fe640 1 -- 172.21.15.76:0/3022623767 >> v2:172.21.15.76:3300/0 conn(0x7fab44158c80 msgr2=0x7fab440ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:50.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.902+0000 7fab437fe640 1 --2- 172.21.15.76:0/3022623767 >> v2:172.21.15.76:3300/0 conn(0x7fab44158c80 0x7fab440ca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.902+0000 7fab437fe640 1 -- 172.21.15.76:0/3022623767 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab44159870 con 0x7fab44159060 2023-11-09T03:45:50.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.902+0000 7f86a1d9b640 1 -- 172.21.15.76:0/1739514065 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8698031700 con 0x7f86a40b4f90 2023-11-09T03:45:50.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.902+0000 7f5232ffd640 1 -- 172.21.15.76:0/2054894763 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52340ae870 con 0x7f522c050350 2023-11-09T03:45:50.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.902+0000 7f86a1d9b640 1 -- 172.21.15.76:0/1739514065 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8698031a90 con 0x7f86a40b4f90 2023-11-09T03:45:50.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.902+0000 7f52390df640 1 -- 172.21.15.76:0/2054894763 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f522c050350 msgr2=0x7f522c0eb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.902+0000 7f9e177fe640 1 -- 172.21.15.76:0/3434098291 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e08031830 con 0x7f9e1806a660 2023-11-09T03:45:50.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.902+0000 7f52390df640 1 --2- 172.21.15.76:0/2054894763 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f522c050350 0x7f522c0eb450 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f523402cce0 tx=0x7f52340a61d0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7f52390df640 1 -- 172.21.15.76:0/2054894763 shutdown_connections 2023-11-09T03:45:50.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7f9e177fe640 1 -- 172.21.15.76:0/3434098291 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e08031990 con 0x7f9e1806a660 2023-11-09T03:45:50.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7f9e177fe640 1 -- 172.21.15.76:0/3434098291 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e08031d20 con 0x7f9e1806a660 2023-11-09T03:45:50.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7f52390df640 1 --2- 172.21.15.76:0/2054894763 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f522c050350 0x7f522c0eb450 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7f52390df640 1 --2- 172.21.15.76:0/2054894763 >> v2:172.21.15.113:3300/0 conn(0x7f522c006370 0x7f522c03ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7f52390df640 1 -- 172.21.15.76:0/2054894763 >> 172.21.15.76:0/2054894763 conn(0x7f522c00a990 msgr2=0x7f522c00ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:50.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7fab42ffd640 1 --2- 172.21.15.76:0/3022623767 >> v2:172.21.15.76:3300/0 conn(0x7fab44158c80 0x7fab440ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:50.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7f2c1f9c5640 1 -- 172.21.15.76:0/629512508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c180b4f90 msgr2=0x7f2c180b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7f2c1f9c5640 1 --2- 172.21.15.76:0/629512508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c180b4f90 0x7f2c180b2270 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2c140097a0 tx=0x7f2c1402b660 comp rx=0 tx=0).stop 2023-11-09T03:45:50.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7fe7f0be2640 1 -- 172.21.15.76:0/3740347638 shutdown_connections 2023-11-09T03:45:50.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7fe7f0be2640 1 --2- 172.21.15.76:0/3740347638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7ec070710 0x7fe7ec070b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7fe7f0be2640 1 --2- 172.21.15.76:0/3740347638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7ec0c4690 0x7fe7ec0cc520 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7fe7f0be2640 1 -- 172.21.15.76:0/3740347638 >> 172.21.15.76:0/3740347638 conn(0x7fe7ec0ce950 msgr2=0x7fe7ec0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:50.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.902+0000 7fd24d544640 1 --2- 172.21.15.76:0/3307225535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd248142480 0x7fd2480cac40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd2440071d0 tx=0x7fd24402c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b10cf1b4cd80266 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:50.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7fd24cd43640 1 -- 172.21.15.76:0/3307225535 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd244030050 con 0x7fd248142480 2023-11-09T03:45:50.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7fd24cd43640 1 -- 172.21.15.76:0/3307225535 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd244008bf0 con 0x7fd248142480 2023-11-09T03:45:50.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7fd24cd43640 1 -- 172.21.15.76:0/3307225535 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2400019f0 con 0x7fd248142480 2023-11-09T03:45:50.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7fa8956c1640 1 --2- 172.21.15.76:0/968403825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa87c003970 0x7fa87c003d90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa8900cf970 tx=0x7fa87c0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=159aec05858f61e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:50.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7f2c1f9c5640 1 -- 172.21.15.76:0/629512508 shutdown_connections 2023-11-09T03:45:50.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7f2c1f9c5640 1 --2- 172.21.15.76:0/629512508 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c180b5c50 0x7f2c18164e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7f2c1f9c5640 1 --2- 172.21.15.76:0/629512508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c180b4f90 0x7f2c180b2270 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7f2c1f9c5640 1 -- 172.21.15.76:0/629512508 >> 172.21.15.76:0/629512508 conn(0x7f2c180cf880 msgr2=0x7f2c180cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:50.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7f52390df640 1 -- 172.21.15.76:0/2054894763 shutdown_connections 2023-11-09T03:45:50.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7f52390df640 1 -- 172.21.15.76:0/2054894763 wait complete. 2023-11-09T03:45:50.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7fab437fe640 1 --2- 172.21.15.76:0/3022623767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab44159060 0x7fab440ce2f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fab440cc380 tx=0x7fab3c02bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfade31d637e90d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:50.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7f86a915e640 1 -- 172.21.15.76:0/1739514065 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86a40b4f90 msgr2=0x7f86a40b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7f86a915e640 1 --2- 172.21.15.76:0/1739514065 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86a40b4f90 0x7f86a40b2270 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f86980097a0 tx=0x7f869802b7a0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7fa887fff640 1 -- 172.21.15.76:0/968403825 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa880032640 con 0x7fa87c003970 2023-11-09T03:45:50.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7f2c1f9c5640 1 -- 172.21.15.76:0/629512508 shutdown_connections 2023-11-09T03:45:50.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7fa887fff640 1 -- 172.21.15.76:0/968403825 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88002e070 con 0x7fa87c003970 2023-11-09T03:45:50.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7fa887fff640 1 -- 172.21.15.76:0/968403825 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa880032a80 con 0x7fa87c003970 2023-11-09T03:45:50.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7fe7f0be2640 1 -- 172.21.15.76:0/3740347638 shutdown_connections 2023-11-09T03:45:50.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7f86a915e640 1 -- 172.21.15.76:0/1739514065 shutdown_connections 2023-11-09T03:45:50.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7f86a915e640 1 --2- 172.21.15.76:0/1739514065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f86a40b5c50 0x7f86a4164e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.963 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7f86a915e640 1 --2- 172.21.15.76:0/1739514065 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86a40b4f90 0x7f86a40b2270 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.963 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7f86a915e640 1 -- 172.21.15.76:0/1739514065 >> 172.21.15.76:0/1739514065 conn(0x7f86a40cf880 msgr2=0x7f86a40cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:50.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.904+0000 7f9e1ef61640 1 -- 172.21.15.76:0/3434098291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e1806a660 msgr2=0x7f9e180ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.904+0000 7f9e1ef61640 1 --2- 172.21.15.76:0/3434098291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e1806a660 0x7f9e180ccae0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9e18072900 tx=0x7f9e0802b8d0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.904+0000 7f9e1ef61640 1 -- 172.21.15.76:0/3434098291 shutdown_connections 2023-11-09T03:45:50.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.904+0000 7f9e1ef61640 1 --2- 172.21.15.76:0/3434098291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e1806a660 0x7f9e180ccae0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.904+0000 7f9e1ef61640 1 --2- 172.21.15.76:0/3434098291 >> v2:172.21.15.76:3300/0 conn(0x7f9e18071fa0 0x7f9e1806a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.904+0000 7f9e1ef61640 1 -- 172.21.15.76:0/3434098291 >> 172.21.15.76:0/3434098291 conn(0x7f9e1806d800 msgr2=0x7f9e1806dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:50.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.904+0000 7f9e1ef61640 1 -- 172.21.15.76:0/3434098291 shutdown_connections 2023-11-09T03:45:50.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.904+0000 7f9e1ef61640 1 -- 172.21.15.76:0/3434098291 wait complete. 2023-11-09T03:45:50.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.904+0000 7fab41ffb640 1 -- 172.21.15.76:0/3022623767 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c02ca80 con 0x7fab44159060 2023-11-09T03:45:50.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.904+0000 7fab41ffb640 1 -- 172.21.15.76:0/3022623767 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3c02cbe0 con 0x7fab44159060 2023-11-09T03:45:50.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.904+0000 7fab41ffb640 1 -- 172.21.15.76:0/3022623767 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c0343f0 con 0x7fab44159060 2023-11-09T03:45:50.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.904+0000 7fd24ffa9640 1 -- 172.21.15.76:0/3307225535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd248142480 msgr2=0x7fd2480cac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.904+0000 7fd24ffa9640 1 --2- 172.21.15.76:0/3307225535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd248142480 0x7fd2480cac40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd2440071d0 tx=0x7fd24402c820 comp rx=0 tx=0).stop 2023-11-09T03:45:50.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.904+0000 7fe7f0be2640 1 -- 172.21.15.76:0/3740347638 wait complete. 2023-11-09T03:45:50.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.904+0000 7fd24ffa9640 1 -- 172.21.15.76:0/3307225535 shutdown_connections 2023-11-09T03:45:50.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.904+0000 7fd24ffa9640 1 --2- 172.21.15.76:0/3307225535 >> v2:172.21.15.76:3300/0 conn(0x7fd2481429d0 0x7fd2480cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.903+0000 7f2c1f9c5640 1 -- 172.21.15.76:0/629512508 wait complete. 2023-11-09T03:45:50.970 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.904+0000 7fd24ffa9640 1 --2- 172.21.15.76:0/3307225535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd248142480 0x7fd2480cac40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.970 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.904+0000 7f52390df640 1 Processor -- start 2023-11-09T03:45:50.970 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.904+0000 7fd24ffa9640 1 -- 172.21.15.76:0/3307225535 >> 172.21.15.76:0/3307225535 conn(0x7fd24806a120 msgr2=0x7fd24806a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:50.971 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.904+0000 7f86a915e640 1 -- 172.21.15.76:0/1739514065 shutdown_connections 2023-11-09T03:45:50.971 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.904+0000 7fa8966c3640 1 -- 172.21.15.76:0/968403825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa87c003970 msgr2=0x7fa87c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.971 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.904+0000 7fa8966c3640 1 --2- 172.21.15.76:0/968403825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa87c003970 0x7fa87c003d90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa8900cf970 tx=0x7fa87c0057f0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.904+0000 7fa895ec2640 1 -- 172.21.15.76:0/968403825 reap_dead start 2023-11-09T03:45:50.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.904+0000 7f52390df640 1 -- start start 2023-11-09T03:45:50.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.904+0000 7fd24ffa9640 1 -- 172.21.15.76:0/3307225535 shutdown_connections 2023-11-09T03:45:50.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.905+0000 7fd24ffa9640 1 -- 172.21.15.76:0/3307225535 wait complete. 2023-11-09T03:45:50.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.904+0000 7f86a915e640 1 -- 172.21.15.76:0/1739514065 wait complete. 2023-11-09T03:45:50.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.905+0000 7f2c1f9c5640 1 Processor -- start 2023-11-09T03:45:50.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.905+0000 7f52390df640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f522c006370 0x7f522c0dff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.905+0000 7f2c1f9c5640 1 -- start start 2023-11-09T03:45:50.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.905+0000 7f52390df640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f522c006750 0x7f522c0d7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.905+0000 7f52390df640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f522c050350 0x7f522c0c2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.905+0000 7fab49490640 1 -- 172.21.15.76:0/3022623767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab44159060 msgr2=0x7fab440ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.905+0000 7fab49490640 1 --2- 172.21.15.76:0/3022623767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab44159060 0x7fab440ce2f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fab440cc380 tx=0x7fab3c02bee0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.905+0000 7f52390df640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522c0d7c50 con 0x7f522c006370 2023-11-09T03:45:50.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.905+0000 7f52390df640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522c0d7f30 con 0x7f522c050350 2023-11-09T03:45:50.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.905+0000 7fa8966c3640 1 -- 172.21.15.76:0/968403825 shutdown_connections 2023-11-09T03:45:50.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.905+0000 7fa8966c3640 1 -- 172.21.15.76:0/968403825 >> 172.21.15.76:0/968403825 conn(0x7fa89006b680 msgr2=0x7fa89006ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:50.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.905+0000 7f52390df640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f522c0d8210 con 0x7f522c006750 2023-11-09T03:45:50.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.905+0000 7f52388de640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f522c050350 0x7f522c0c2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.906+0000 7f52388de640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f522c050350 0x7f522c0c2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:33960/0 (socket says 172.21.15.76:33960) 2023-11-09T03:45:50.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.906+0000 7f5233fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f522c006370 0x7f522c0dff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.906+0000 7f52388de640 1 -- 172.21.15.76:0/4213987408 learned_addr learned my addr 172.21.15.76:0/4213987408 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:50.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.906+0000 7fe7f0be2640 1 Processor -- start 2023-11-09T03:45:50.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.906+0000 7fe7f0be2640 1 -- start start 2023-11-09T03:45:50.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.906+0000 7f9e1ef61640 1 Processor -- start 2023-11-09T03:45:50.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.906+0000 7f52337fe640 1 --2- 172.21.15.76:0/4213987408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f522c006750 0x7f522c0d7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.906+0000 7fa8966c3640 1 -- 172.21.15.76:0/968403825 shutdown_connections 2023-11-09T03:45:50.981 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.906+0000 7f9e1ef61640 1 -- start start 2023-11-09T03:45:50.981 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.906+0000 7f52388de640 1 -- 172.21.15.76:0/4213987408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f522c006750 msgr2=0x7f522c0d7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.906+0000 7f52388de640 1 --2- 172.21.15.76:0/4213987408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f522c006750 0x7f522c0d7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.906+0000 7f52388de640 1 -- 172.21.15.76:0/4213987408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f522c006370 msgr2=0x7f522c0dff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.906+0000 7f52388de640 1 --2- 172.21.15.76:0/4213987408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f522c006370 0x7f522c0dff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.906+0000 7f52388de640 1 -- 172.21.15.76:0/4213987408 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f522c046010 con 0x7f522c050350 2023-11-09T03:45:50.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.906+0000 7f5233fff640 1 --2- 172.21.15.76:0/4213987408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f522c006370 0x7f522c0dff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:45:50.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.906+0000 7f52337fe640 1 --2- 172.21.15.76:0/4213987408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f522c006750 0x7f522c0d7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:50.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.906+0000 7f86a915e640 1 Processor -- start 2023-11-09T03:45:50.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.906+0000 7f52388de640 1 --2- 172.21.15.76:0/4213987408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f522c050350 0x7f522c0c2480 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5234074540 tx=0x7f5234069f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:50.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.906+0000 7f86a915e640 1 -- start start 2023-11-09T03:45:50.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.907+0000 7fa8966c3640 1 -- 172.21.15.76:0/968403825 wait complete. 2023-11-09T03:45:50.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.906+0000 7fab49490640 1 -- 172.21.15.76:0/3022623767 shutdown_connections 2023-11-09T03:45:50.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.906+0000 7fab49490640 1 --2- 172.21.15.76:0/3022623767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab44159060 0x7fab440ce2f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.906+0000 7fab49490640 1 --2- 172.21.15.76:0/3022623767 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab4406ac10 0x7fab440cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.906+0000 7fab49490640 1 --2- 172.21.15.76:0/3022623767 >> v2:172.21.15.76:3300/0 conn(0x7fab44158c80 0x7fab440ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.906+0000 7fab49490640 1 -- 172.21.15.76:0/3022623767 >> 172.21.15.76:0/3022623767 conn(0x7fab4406a120 msgr2=0x7fab4406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:50.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.907+0000 7fab49490640 1 -- 172.21.15.76:0/3022623767 shutdown_connections 2023-11-09T03:45:50.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.907+0000 7fab49490640 1 -- 172.21.15.76:0/3022623767 wait complete. 2023-11-09T03:45:50.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.907+0000 7fd24ffa9640 1 Processor -- start 2023-11-09T03:45:50.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.907+0000 7f2c1f9c5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c180b4f90 0x7f2c18155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.907+0000 7f2c1f9c5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c180b5c50 0x7f2c181557a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.907+0000 7f2c1f9c5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c18161150 0x7f2c180caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.907+0000 7f2c1f9c5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c18156f90 con 0x7f2c18161150 2023-11-09T03:45:50.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.907+0000 7f2c1f9c5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c18156380 con 0x7f2c180b5c50 2023-11-09T03:45:50.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.907+0000 7f2c1f9c5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c18156660 con 0x7f2c180b4f90 2023-11-09T03:45:50.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.907+0000 7fd24ffa9640 1 -- start start 2023-11-09T03:45:50.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.907+0000 7f2c1df62640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c18161150 0x7f2c180caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.907+0000 7f2c1df62640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c18161150 0x7f2c180caac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57706/0 (socket says 172.21.15.76:57706) 2023-11-09T03:45:50.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.907+0000 7f2c1df62640 1 -- 172.21.15.76:0/1455002017 learned_addr learned my addr 172.21.15.76:0/1455002017 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:50.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.907+0000 7fe7f0be2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7ec070710 0x7fe7ec161db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.907+0000 7f2c1df62640 1 -- 172.21.15.76:0/1455002017 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c180b4f90 msgr2=0x7f2c18155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:50.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.907+0000 7fe7f0be2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7ec0c4690 0x7fe7ec163430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.907+0000 7f2c1df62640 1 --2- 172.21.15.76:0/1455002017 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c180b4f90 0x7f2c18155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.907+0000 7f2c1df62640 1 -- 172.21.15.76:0/1455002017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c180b5c50 msgr2=0x7f2c181557a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-09T03:45:50.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7fe7ea59c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7ec070710 0x7fe7ec161db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7fe7ea59c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7ec070710 0x7fe7ec161db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:50978/0 (socket says 172.21.15.76:50978) 2023-11-09T03:45:50.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.907+0000 7f2c1df62640 1 --2- 172.21.15.76:0/1455002017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c180b5c50 0x7f2c181557a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:50.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7f9e1ef61640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e1806a660 0x7f9e180c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7fe7ea59c640 1 -- 172.21.15.76:0/2291115551 learned_addr learned my addr 172.21.15.76:0/2291115551 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:50.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7fe7e9d9b640 1 --2- 172.21.15.76:0/2291115551 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7ec0c4690 0x7fe7ec163430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7fe7f0be2640 1 --2- 172.21.15.76:0/2291115551 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7ec164760 0x7fe7ec0b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7fe7f0be2640 1 -- 172.21.15.76:0/2291115551 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec162390 con 0x7fe7ec164760 2023-11-09T03:45:50.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7fe7f0be2640 1 -- 172.21.15.76:0/2291115551 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec162670 con 0x7fe7ec0c4690 2023-11-09T03:45:50.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7fe7f0be2640 1 -- 172.21.15.76:0/2291115551 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec162950 con 0x7fe7ec070710 2023-11-09T03:45:50.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7f9e1ef61640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e18071fa0 0x7f9e180c20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7f9e1ef61640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9e180b2010 0x7f9e181eafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7f9e1ef61640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e180c2620 con 0x7f9e18071fa0 2023-11-09T03:45:50.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7f9e1ef61640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e181eb510 con 0x7f9e1806a660 2023-11-09T03:45:50.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7f9e1ef61640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e180c9a30 con 0x7f9e180b2010 2023-11-09T03:45:50.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.907+0000 7f5230ff9640 1 -- 172.21.15.76:0/4213987408 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52340ad9e0 con 0x7f522c050350 2023-11-09T03:45:50.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.907+0000 7f5230ff9640 1 -- 172.21.15.76:0/4213987408 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52340adb40 con 0x7f522c050350 2023-11-09T03:45:50.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.907+0000 7f5230ff9640 1 -- 172.21.15.76:0/4213987408 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52340b8410 con 0x7f522c050350 2023-11-09T03:45:50.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7f52390df640 1 -- 172.21.15.76:0/4213987408 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f522c0e0480 con 0x7f522c050350 2023-11-09T03:45:50.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7f52390df640 1 -- 172.21.15.76:0/4213987408 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f522c0e0920 con 0x7f522c050350 2023-11-09T03:45:50.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7f86a915e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f86a40b4f90 0x7f86a40ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7f86a915e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86a40b5c50 0x7f86a415b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7f86a915e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86a4161150 0x7f86a414e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7f86a259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86a40b5c50 0x7f86a415b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7f86a915e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a40cc080 con 0x7f86a4161150 2023-11-09T03:45:50.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7f86a915e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a414eb40 con 0x7f86a40b5c50 2023-11-09T03:45:50.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7f86a915e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a40cafe0 con 0x7f86a40b4f90 2023-11-09T03:45:50.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7f86a259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86a40b5c50 0x7f86a415b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:33978/0 (socket says 172.21.15.76:33978) 2023-11-09T03:45:50.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7f86a259c640 1 -- 172.21.15.76:0/2653126780 learned_addr learned my addr 172.21.15.76:0/2653126780 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:50.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7f86a2d9d640 1 --2- 172.21.15.76:0/2653126780 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f86a40b4f90 0x7f86a40ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7fd24ffa9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd248141e20 0x7fd2481e4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:50.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7f86a359e640 1 --2- 172.21.15.76:0/2653126780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86a4161150 0x7f86a414e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:50.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.907+0000 7f2c1df62640 1 -- 172.21.15.76:0/1455002017 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c14009400 con 0x7f2c18161150 2023-11-09T03:45:50.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7f2c1df62640 1 --2- 172.21.15.76:0/1455002017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c18161150 0x7f2c180caac0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2c0800d800 tx=0x7f2c0800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7fe7ead9d640 1 --2- 172.21.15.76:0/2291115551 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7ec164760 0x7fe7ec0b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7fab49490640 1 Processor -- start 2023-11-09T03:45:51.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7fa8966c3640 1 Processor -- start 2023-11-09T03:45:51.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7fa8966c3640 1 -- start start 2023-11-09T03:45:51.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7fd24ffa9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd248142480 0x7fd24813be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7f86a259c640 1 -- 172.21.15.76:0/2653126780 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f86a40b4f90 msgr2=0x7f86a40ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7f86a259c640 1 --2- 172.21.15.76:0/2653126780 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f86a40b4f90 0x7f86a40ca8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7f86a259c640 1 -- 172.21.15.76:0/2653126780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86a4161150 msgr2=0x7f86a414e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.908+0000 7f86a259c640 1 --2- 172.21.15.76:0/2653126780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86a4161150 0x7f86a414e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7f86a259c640 1 -- 172.21.15.76:0/2653126780 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869c009b30 con 0x7f86a40b5c50 2023-11-09T03:45:51.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7f9e17fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e18071fa0 0x7f9e180c20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7f9e1ccfd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e1806a660 0x7f9e180c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7f9e1ccfd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e1806a660 0x7f9e180c8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:33990/0 (socket says 172.21.15.76:33990) 2023-11-09T03:45:51.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7f9e1ccfd640 1 -- 172.21.15.76:0/663163210 learned_addr learned my addr 172.21.15.76:0/663163210 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:51.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7f9e1d4fe640 1 --2- 172.21.15.76:0/663163210 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9e180b2010 0x7f9e181eafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7fd24ffa9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2481429d0 0x7fd2481e36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7fd24ffa9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2480b28c0 con 0x7fd248142480 2023-11-09T03:45:51.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7fd24ffa9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2480b4000 con 0x7fd2481429d0 2023-11-09T03:45:51.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7fd24ffa9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2480c7c30 con 0x7fd248141e20 2023-11-09T03:45:51.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7fe7ea59c640 1 -- 172.21.15.76:0/2291115551 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7ec0c4690 msgr2=0x7fe7ec163430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7fe7ea59c640 1 --2- 172.21.15.76:0/2291115551 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7ec0c4690 0x7fe7ec163430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7fe7ea59c640 1 -- 172.21.15.76:0/2291115551 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7ec164760 msgr2=0x7fe7ec0b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7fe7ea59c640 1 --2- 172.21.15.76:0/2291115551 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7ec164760 0x7fe7ec0b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7fe7ea59c640 1 -- 172.21.15.76:0/2291115551 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e4009a70 con 0x7fe7ec070710 2023-11-09T03:45:51.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7fd24dd45640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd248141e20 0x7fd2481e4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7f2c0dffb640 1 -- 172.21.15.76:0/1455002017 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0800b6c0 con 0x7f2c18161150 2023-11-09T03:45:51.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7f2c0dffb640 1 -- 172.21.15.76:0/1455002017 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c0800b840 con 0x7f2c18161150 2023-11-09T03:45:51.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7fd24dd45640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd248141e20 0x7fd2481e4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:51000/0 (socket says 172.21.15.76:51000) 2023-11-09T03:45:51.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7fd24dd45640 1 -- 172.21.15.76:0/3543056862 learned_addr learned my addr 172.21.15.76:0/3543056862 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:51.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7fd24d544640 1 --2- 172.21.15.76:0/3543056862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd248142480 0x7fd24813be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7fe7ea59c640 1 --2- 172.21.15.76:0/2291115551 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7ec070710 0x7fe7ec161db0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00c480 tx=0x7fe7dc00c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7fab49490640 1 -- start start 2023-11-09T03:45:51.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7fd24e546640 1 --2- 172.21.15.76:0/3543056862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2481429d0 0x7fd2481e36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7f9e17fff640 1 -- 172.21.15.76:0/663163210 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9e180b2010 msgr2=0x7f9e181eafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7f9e17fff640 1 --2- 172.21.15.76:0/663163210 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9e180b2010 0x7f9e181eafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7f9e17fff640 1 -- 172.21.15.76:0/663163210 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e1806a660 msgr2=0x7f9e180c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7f9e17fff640 1 --2- 172.21.15.76:0/663163210 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e1806a660 0x7f9e180c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7f9e17fff640 1 -- 172.21.15.76:0/663163210 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e08009400 con 0x7f9e18071fa0 2023-11-09T03:45:51.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7fa8966c3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa87c003970 0x7fa8900cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7f521e7fc640 1 -- 172.21.15.76:0/4213987408 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f520c000fc0 con 0x7f522c050350 2023-11-09T03:45:51.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7fd24dd45640 1 -- 172.21.15.76:0/3543056862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2481429d0 msgr2=0x7fd2481e36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7fd24dd45640 1 --2- 172.21.15.76:0/3543056862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2481429d0 0x7fd2481e36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7fd24dd45640 1 -- 172.21.15.76:0/3543056862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd248142480 msgr2=0x7fd24813be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7fd24dd45640 1 --2- 172.21.15.76:0/3543056862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd248142480 0x7fd24813be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7fd24dd45640 1 -- 172.21.15.76:0/3543056862 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24400ae30 con 0x7fd248141e20 2023-11-09T03:45:51.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7fa8966c3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8900c7940 0x7fa8900c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.909+0000 7f2c1f9c5640 1 -- 172.21.15.76:0/1455002017 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c18156940 con 0x7f2c18161150 2023-11-09T03:45:51.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7fab49490640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab4406ac10 0x7fab44147fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7fa8966c3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8900ca120 0x7fa8900ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7f86a259c640 1 --2- 172.21.15.76:0/2653126780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86a40b5c50 0x7f86a415b4d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f86980098c0 tx=0x7f86980046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7f9e17fff640 1 --2- 172.21.15.76:0/663163210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e18071fa0 0x7f9e180c20e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000caa0 tx=0x7f9e1000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7fab49490640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab44158c80 0x7fab4414efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7f868b7fe640 1 -- 172.21.15.76:0/2653126780 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8698033ab0 con 0x7f86a40b5c50 2023-11-09T03:45:51.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7f2c1f9c5640 1 -- 172.21.15.76:0/1455002017 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c180cb000 con 0x7f2c18161150 2023-11-09T03:45:51.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7f868b7fe640 1 -- 172.21.15.76:0/2653126780 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8698033c10 con 0x7f86a40b5c50 2023-11-09T03:45:51.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7f868b7fe640 1 -- 172.21.15.76:0/2653126780 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869803d410 con 0x7f86a40b5c50 2023-11-09T03:45:51.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7fab49490640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab44159060 0x7fab4414afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7fa894ec0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8900c7940 0x7fa8900c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7fa894ec0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8900c7940 0x7fa8900c7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:34004/0 (socket says 172.21.15.76:34004) 2023-11-09T03:45:51.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7fa894ec0640 1 -- 172.21.15.76:0/3587147324 learned_addr learned my addr 172.21.15.76:0/3587147324 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:51.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7fd24dd45640 1 --2- 172.21.15.76:0/3543056862 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd248141e20 0x7fd2481e4600 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd24000cca0 tx=0x7fd24000a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7fe7daffd640 1 -- 172.21.15.76:0/2291115551 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc005ce0 con 0x7fe7ec070710 2023-11-09T03:45:51.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7fa8966c3640 1 -- 172.21.15.76:0/3587147324 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8900cbeb0 con 0x7fa87c003970 2023-11-09T03:45:51.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7fab49490640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab440c88f0 con 0x7fab44158c80 2023-11-09T03:45:51.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7f2c0dffb640 1 -- 172.21.15.76:0/1455002017 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0800bbf0 con 0x7f2c18161150 2023-11-09T03:45:51.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7fab49490640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab440b2db0 con 0x7fab44159060 2023-11-09T03:45:51.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7fab49490640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4414f520 con 0x7fab4406ac10 2023-11-09T03:45:51.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7fa8966c3640 1 -- 172.21.15.76:0/3587147324 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8900c82e0 con 0x7fa8900c7940 2023-11-09T03:45:51.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7fe7f0be2640 1 -- 172.21.15.76:0/2291115551 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7ec162c30 con 0x7fe7ec070710 2023-11-09T03:45:51.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7fa8966c3640 1 -- 172.21.15.76:0/3587147324 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8900cae10 con 0x7fa8900ca120 2023-11-09T03:45:51.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7fa895ec2640 1 --2- 172.21.15.76:0/3587147324 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8900ca120 0x7fa8900ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7fe7f0be2640 1 -- 172.21.15.76:0/2291115551 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7ec0b0970 con 0x7fe7ec070710 2023-11-09T03:45:51.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.911+0000 7f5230ff9640 1 -- 172.21.15.76:0/4213987408 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f52340b5050 con 0x7f522c050350 2023-11-09T03:45:51.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7fd23e7fc640 1 -- 172.21.15.76:0/3543056862 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd240012400 con 0x7fd248141e20 2023-11-09T03:45:51.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7fd23e7fc640 1 -- 172.21.15.76:0/3543056862 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd240012580 con 0x7fd248141e20 2023-11-09T03:45:51.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.910+0000 7fd23e7fc640 1 -- 172.21.15.76:0/3543056862 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd240012930 con 0x7fd248141e20 2023-11-09T03:45:51.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.911+0000 7fd24ffa9640 1 -- 172.21.15.76:0/3543056862 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2480c7f10 con 0x7fd248141e20 2023-11-09T03:45:51.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.911+0000 7fd24ffa9640 1 -- 172.21.15.76:0/3543056862 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2480b2550 con 0x7fd248141e20 2023-11-09T03:45:51.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.911+0000 7fab427fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab44158c80 0x7fab4414efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.911+0000 7fab427fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab44158c80 0x7fab4414efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57770/0 (socket says 172.21.15.76:57770) 2023-11-09T03:45:51.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.911+0000 7fab427fc640 1 -- 172.21.15.76:0/2253059933 learned_addr learned my addr 172.21.15.76:0/2253059933 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:51.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.911+0000 7fe7daffd640 1 -- 172.21.15.76:0/2291115551 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7dc005e40 con 0x7fe7ec070710 2023-11-09T03:45:51.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.911+0000 7fab427fc640 1 -- 172.21.15.76:0/2253059933 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab4406ac10 msgr2=0x7fab44147fc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-09T03:45:51.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.911+0000 7fab427fc640 1 --2- 172.21.15.76:0/2253059933 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab4406ac10 0x7fab44147fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.911+0000 7fab427fc640 1 -- 172.21.15.76:0/2253059933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab44159060 msgr2=0x7fab4414afc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-09T03:45:51.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.911+0000 7fab427fc640 1 --2- 172.21.15.76:0/2253059933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab44159060 0x7fab4414afc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.911+0000 7f86a915e640 1 -- 172.21.15.76:0/2653126780 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8698009400 con 0x7f86a40b5c50 2023-11-09T03:45:51.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.911+0000 7f86a915e640 1 -- 172.21.15.76:0/2653126780 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a40cb5a0 con 0x7f86a40b5c50 2023-11-09T03:45:51.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.911+0000 7fab427fc640 1 -- 172.21.15.76:0/2253059933 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab3c009b30 con 0x7fab44158c80 2023-11-09T03:45:51.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.911+0000 7f5230ff9640 1 --2- 172.21.15.76:0/4213987408 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f521806af40 0x7f521806d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.911+0000 7f5230ff9640 1 -- 172.21.15.76:0/4213987408 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5234125f30 con 0x7f522c050350 2023-11-09T03:45:51.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.911+0000 7f5230ff9640 1 --2- 172.21.15.76:0/4213987408 >> [v2:172.21.15.113:6800/3754040883,v1:172.21.15.113:6801/3754040883] conn(0x7f5218074500 0x7f5218076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.911+0000 7f5230ff9640 1 -- 172.21.15.76:0/4213987408 --> [v2:172.21.15.113:6800/3754040883,v1:172.21.15.113:6801/3754040883] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5218077010 con 0x7f5218074500 2023-11-09T03:45:51.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.912+0000 7fab427fc640 1 --2- 172.21.15.76:0/2253059933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab44158c80 0x7fab4414efe0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fab3800e9b0 tx=0x7fab3800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.912+0000 7fa895ec2640 1 -- 172.21.15.76:0/3587147324 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8900c7940 msgr2=0x7fa8900c7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.912+0000 7fa895ec2640 1 --2- 172.21.15.76:0/3587147324 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8900c7940 0x7fa8900c7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.912+0000 7fa895ec2640 1 -- 172.21.15.76:0/3587147324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa87c003970 msgr2=0x7fa8900cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.912+0000 7fa895ec2640 1 --2- 172.21.15.76:0/3587147324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa87c003970 0x7fa8900cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.912+0000 7fa895ec2640 1 -- 172.21.15.76:0/3587147324 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8900cb0f0 con 0x7fa8900ca120 2023-11-09T03:45:51.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.912+0000 7f5233fff640 1 --2- 172.21.15.76:0/4213987408 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f521806af40 0x7f521806d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.912+0000 7f5230ff9640 1 -- 172.21.15.76:0/4213987408 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52340f0d60 con 0x7f522c050350 2023-11-09T03:45:51.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.912+0000 7f2c1f9c5640 1 -- 172.21.15.76:0/1455002017 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2c18165bb0 con 0x7f2c18161150 2023-11-09T03:45:51.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.912+0000 7fe7daffd640 1 -- 172.21.15.76:0/2291115551 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc0072f0 con 0x7fe7ec070710 2023-11-09T03:45:51.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.912+0000 7fe7f0be2640 1 -- 172.21.15.76:0/2291115551 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe7d0000fc0 con 0x7fe7ec070710 2023-11-09T03:45:51.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.912+0000 7fa895ec2640 1 --2- 172.21.15.76:0/3587147324 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8900ca120 0x7fa8900ca5a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa89006c4e0 tx=0x7fa88c00e960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.912+0000 7f86a915e640 1 -- 172.21.15.76:0/2653126780 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f86a4165bb0 con 0x7f86a40b5c50 2023-11-09T03:45:51.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.912+0000 7f52337fe640 1 --2- 172.21.15.76:0/4213987408 >> [v2:172.21.15.113:6800/3754040883,v1:172.21.15.113:6801/3754040883] conn(0x7f5218074500 0x7f5218076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.913+0000 7fd23e7fc640 1 -- 172.21.15.76:0/3543056862 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd240012a90 con 0x7fd248141e20 2023-11-09T03:45:51.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.913+0000 7fab2b7fe640 1 -- 172.21.15.76:0/2253059933 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3800c9a0 con 0x7fab44158c80 2023-11-09T03:45:51.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.913+0000 7fa885ffb640 1 -- 172.21.15.76:0/3587147324 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c014480 con 0x7fa8900ca120 2023-11-09T03:45:51.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.913+0000 7fab2b7fe640 1 -- 172.21.15.76:0/2253059933 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3800cb00 con 0x7fab44158c80 2023-11-09T03:45:51.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.913+0000 7fab49490640 1 -- 172.21.15.76:0/2253059933 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab440c8330 con 0x7fab44158c80 2023-11-09T03:45:51.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.913+0000 7fa885ffb640 1 -- 172.21.15.76:0/3587147324 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88c014620 con 0x7fa8900ca120 2023-11-09T03:45:51.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.913+0000 7fa885ffb640 1 -- 172.21.15.76:0/3587147324 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c0149d0 con 0x7fa8900ca120 2023-11-09T03:45:51.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.913+0000 7fab2b7fe640 1 -- 172.21.15.76:0/2253059933 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38013470 con 0x7fab44158c80 2023-11-09T03:45:51.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.913+0000 7fab49490640 1 -- 172.21.15.76:0/2253059933 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab440b3010 con 0x7fab44158c80 2023-11-09T03:45:51.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.913+0000 7fd24ffa9640 1 -- 172.21.15.76:0/3543056862 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd248140210 con 0x7fd248141e20 2023-11-09T03:45:51.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.913+0000 7fd23e7fc640 1 --2- 172.21.15.76:0/3543056862 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd22c06aec0 0x7fd22c06d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.913+0000 7f9e157fa640 1 -- 172.21.15.76:0/663163210 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1000aba0 con 0x7f9e18071fa0 2023-11-09T03:45:51.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.913+0000 7f9e157fa640 1 -- 172.21.15.76:0/663163210 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1000ad00 con 0x7f9e18071fa0 2023-11-09T03:45:51.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.913+0000 7f9e1ef61640 1 -- 172.21.15.76:0/663163210 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e180c9190 con 0x7f9e18071fa0 2023-11-09T03:45:51.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.913+0000 7f9e1ef61640 1 -- 172.21.15.76:0/663163210 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e180c9750 con 0x7f9e18071fa0 2023-11-09T03:45:51.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.913+0000 7fd23e7fc640 1 -- 172.21.15.76:0/3543056862 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd240089e00 con 0x7fd248141e20 2023-11-09T03:45:51.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.913+0000 7f868b7fe640 1 -- 172.21.15.76:0/2653126780 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f869803a050 con 0x7f86a40b5c50 2023-11-09T03:45:51.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.914+0000 7fd23e7fc640 1 --2- 172.21.15.76:0/3543056862 >> [v2:172.21.15.113:6808/570146941,v1:172.21.15.113:6809/570146941] conn(0x7fd22c074480 0x7fd22c0768c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.914+0000 7fd23e7fc640 1 -- 172.21.15.76:0/3543056862 --> [v2:172.21.15.113:6808/570146941,v1:172.21.15.113:6809/570146941] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd22c076f90 con 0x7fd22c074480 2023-11-09T03:45:51.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.913+0000 7fa8966c3640 1 -- 172.21.15.76:0/3587147324 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8900cb3d0 con 0x7fa8900ca120 2023-11-09T03:45:51.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.914+0000 7f2c0dffb640 1 -- 172.21.15.76:0/1455002017 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2c0800bd50 con 0x7f2c18161150 2023-11-09T03:45:51.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.914+0000 7f2c0dffb640 1 --2- 172.21.15.76:0/1455002017 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2bfc06af40 0x7f2bfc06d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.915+0000 7f2c0dffb640 1 -- 172.21.15.76:0/1455002017 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2c08089db0 con 0x7f2c18161150 2023-11-09T03:45:51.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.915+0000 7f2c0dffb640 1 --2- 172.21.15.76:0/1455002017 >> [v2:172.21.15.76:6803/1864895740,v1:172.21.15.76:6806/1864895740] conn(0x7f2bfc074390 0x7f2bfc0767d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.915+0000 7f2c0dffb640 1 -- 172.21.15.76:0/1455002017 --> [v2:172.21.15.76:6803/1864895740,v1:172.21.15.76:6806/1864895740] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2bfc076ea0 con 0x7f2bfc074390 2023-11-09T03:45:51.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.914+0000 7fe7daffd640 1 -- 172.21.15.76:0/2291115551 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe7dc007510 con 0x7fe7ec070710 2023-11-09T03:45:51.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.914+0000 7fe7daffd640 1 --2- 172.21.15.76:0/2291115551 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe7cc06af40 0x7fe7cc06d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.915+0000 7fe7daffd640 1 -- 172.21.15.76:0/2291115551 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe7dc0890b0 con 0x7fe7ec070710 2023-11-09T03:45:51.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.915+0000 7f9e157fa640 1 -- 172.21.15.76:0/663163210 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e10012600 con 0x7f9e18071fa0 2023-11-09T03:45:51.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.914+0000 7fa8966c3640 1 -- 172.21.15.76:0/3587147324 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8900cb990 con 0x7fa8900ca120 2023-11-09T03:45:51.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.915+0000 7fa8966c3640 1 -- 172.21.15.76:0/3587147324 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa8900d1ca0 con 0x7fa8900ca120 2023-11-09T03:45:51.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.914+0000 7fd24e546640 1 --2- 172.21.15.76:0/3543056862 >> [v2:172.21.15.113:6808/570146941,v1:172.21.15.113:6809/570146941] conn(0x7fd22c074480 0x7fd22c0768c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.914+0000 7fd24d544640 1 --2- 172.21.15.76:0/3543056862 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd22c06aec0 0x7fd22c06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.914+0000 7fd23e7fc640 1 -- 172.21.15.76:0/3543056862 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd240054b80 con 0x7fd248141e20 2023-11-09T03:45:51.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.915+0000 7fd24e546640 1 --2- 172.21.15.76:0/3543056862 >> [v2:172.21.15.113:6808/570146941,v1:172.21.15.113:6809/570146941] conn(0x7fd22c074480 0x7fd22c0768c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.915+0000 7fd24d544640 1 --2- 172.21.15.76:0/3543056862 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd22c06aec0 0x7fd22c06d380 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd244036000 tx=0x7fd24400a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.914+0000 7f868b7fe640 1 --2- 172.21.15.76:0/2653126780 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f869006af70 0x7f869006d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.914+0000 7f868b7fe640 1 -- 172.21.15.76:0/2653126780 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f86980ab400 con 0x7f86a40b5c50 2023-11-09T03:45:51.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.914+0000 7f868b7fe640 1 --2- 172.21.15.76:0/2653126780 >> [v2:172.21.15.113:6824/2785664875,v1:172.21.15.113:6825/2785664875] conn(0x7f86900745b0 0x7f86900769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.914+0000 7f868b7fe640 1 -- 172.21.15.76:0/2653126780 --> [v2:172.21.15.113:6824/2785664875,v1:172.21.15.113:6825/2785664875] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f86900770c0 con 0x7f86900745b0 2023-11-09T03:45:51.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.914+0000 7f868b7fe640 1 -- 172.21.15.76:0/2653126780 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f869802f020 con 0x7f86a40b5c50 2023-11-09T03:45:51.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.914+0000 7f86a359e640 1 --2- 172.21.15.76:0/2653126780 >> [v2:172.21.15.113:6824/2785664875,v1:172.21.15.113:6825/2785664875] conn(0x7f86900745b0 0x7f86900769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.914+0000 7f86a2d9d640 1 --2- 172.21.15.76:0/2653126780 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f869006af70 0x7f869006d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.915+0000 7f86a359e640 1 --2- 172.21.15.76:0/2653126780 >> [v2:172.21.15.113:6824/2785664875,v1:172.21.15.113:6825/2785664875] conn(0x7f86900745b0 0x7f86900769f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.914+0000 7fab49490640 1 -- 172.21.15.76:0/2253059933 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fab440cda80 con 0x7fab44158c80 2023-11-09T03:45:51.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.915+0000 7f5233fff640 1 --2- 172.21.15.76:0/4213987408 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f521806af40 0x7f521806d400 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5228007d90 tx=0x7f5228008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.915+0000 7f52337fe640 1 --2- 172.21.15.76:0/4213987408 >> [v2:172.21.15.113:6800/3754040883,v1:172.21.15.113:6801/3754040883] conn(0x7f5218074500 0x7f5218076940 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.915+0000 7f2c0dffb640 1 -- 172.21.15.76:0/1455002017 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c08054be0 con 0x7f2c18161150 2023-11-09T03:45:51.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.915+0000 7fe7daffd640 1 --2- 172.21.15.76:0/2291115551 >> [v2:172.21.15.76:6816/1166289696,v1:172.21.15.76:6817/1166289696] conn(0x7fe7cc074580 0x7fe7cc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.915+0000 7fe7e9d9b640 1 --2- 172.21.15.76:0/2291115551 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe7cc06af40 0x7fe7cc06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.915+0000 7f86a2d9d640 1 --2- 172.21.15.76:0/2653126780 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f869006af70 0x7f869006d430 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f869401a7f0 tx=0x7f8694006e70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.915+0000 7f2c1d761640 1 --2- 172.21.15.76:0/1455002017 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2bfc06af40 0x7f2bfc06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.915+0000 7fa885ffb640 1 -- 172.21.15.76:0/3587147324 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa88c014b30 con 0x7fa8900ca120 2023-11-09T03:45:51.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.915+0000 7fe7ead9d640 1 --2- 172.21.15.76:0/2291115551 >> [v2:172.21.15.76:6816/1166289696,v1:172.21.15.76:6817/1166289696] conn(0x7fe7cc074580 0x7fe7cc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.916+0000 7fe7e9d9b640 1 --2- 172.21.15.76:0/2291115551 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe7cc06af40 0x7fe7cc06d400 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4036000 tx=0x7fe7e4030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.916+0000 7fe7daffd640 1 -- 172.21.15.76:0/2291115551 --> [v2:172.21.15.76:6816/1166289696,v1:172.21.15.76:6817/1166289696] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe7cc077090 con 0x7fe7cc074580 2023-11-09T03:45:51.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.916+0000 7f9e1ef61640 1 -- 172.21.15.76:0/663163210 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9df8000fc0 con 0x7f9e18071fa0 2023-11-09T03:45:51.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.916+0000 7fa885ffb640 1 --2- 172.21.15.76:0/3587147324 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa87406af70 0x7fa87406d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.916+0000 7fe7daffd640 1 -- 172.21.15.76:0/2291115551 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7dc036ea0 con 0x7fe7ec070710 2023-11-09T03:45:51.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.916+0000 7f2c1d761640 1 --2- 172.21.15.76:0/1455002017 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2bfc06af40 0x7f2bfc06d400 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2c10007fd0 tx=0x7f2c10007d00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.916+0000 7f868b7fe640 1 -- 172.21.15.76:0/2653126780 <== osd.7 v2:172.21.15.113:6824/2785664875 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f86900770c0 con 0x7f86900745b0 2023-11-09T03:45:51.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.916+0000 7fd23e7fc640 1 -- 172.21.15.76:0/3543056862 <== osd.6 v2:172.21.15.113:6808/570146941 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fd22c076f90 con 0x7fd22c074480 2023-11-09T03:45:51.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.916+0000 7fab2b7fe640 1 -- 172.21.15.76:0/2253059933 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fab380135d0 con 0x7fab44158c80 2023-11-09T03:45:51.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.916+0000 7fa885ffb640 1 -- 172.21.15.76:0/3587147324 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa88c088e70 con 0x7fa8900ca120 2023-11-09T03:45:51.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.916+0000 7fa8956c1640 1 --2- 172.21.15.76:0/3587147324 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa87406af70 0x7fa87406d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.916+0000 7f5230ff9640 1 -- 172.21.15.76:0/4213987408 <== osd.5 v2:172.21.15.113:6800/3754040883 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f5218077010 con 0x7f5218074500 2023-11-09T03:45:51.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.916+0000 7fa885ffb640 1 --2- 172.21.15.76:0/3587147324 >> [v2:172.21.15.113:6816/1824552967,v1:172.21.15.113:6817/1824552967] conn(0x7fa8740745b0 0x7fa8740769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.916+0000 7fa885ffb640 1 -- 172.21.15.76:0/3587147324 --> [v2:172.21.15.113:6816/1824552967,v1:172.21.15.113:6817/1824552967] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa8740770c0 con 0x7fa8740745b0 2023-11-09T03:45:51.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.916+0000 7fa885ffb640 1 -- 172.21.15.76:0/3587147324 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa88c08b490 con 0x7fa8900ca120 2023-11-09T03:45:51.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.917+0000 7fa894ec0640 1 --2- 172.21.15.76:0/3587147324 >> [v2:172.21.15.113:6816/1824552967,v1:172.21.15.113:6817/1824552967] conn(0x7fa8740745b0 0x7fa8740769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.917+0000 7f9e157fa640 1 -- 172.21.15.76:0/663163210 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9e10007070 con 0x7f9e18071fa0 2023-11-09T03:45:51.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.917+0000 7fab2b7fe640 1 --2- 172.21.15.76:0/2253059933 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fab3006af40 0x7fab3006d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.916+0000 7fe7ead9d640 1 --2- 172.21.15.76:0/2291115551 >> [v2:172.21.15.76:6816/1166289696,v1:172.21.15.76:6817/1166289696] conn(0x7fe7cc074580 0x7fe7cc0769c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.917+0000 7fa8956c1640 1 --2- 172.21.15.76:0/3587147324 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa87406af70 0x7fa87406d430 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa8900d0c10 tx=0x7fa880030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.917+0000 7f2c1cf60640 1 --2- 172.21.15.76:0/1455002017 >> [v2:172.21.15.76:6803/1864895740,v1:172.21.15.76:6806/1864895740] conn(0x7f2bfc074390 0x7f2bfc0767d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.917+0000 7fab2b7fe640 1 -- 172.21.15.76:0/2253059933 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fab38089aa0 con 0x7fab44158c80 2023-11-09T03:45:51.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.917+0000 7fab2b7fe640 1 --2- 172.21.15.76:0/2253059933 >> [v2:172.21.15.76:6800/276379883,v1:172.21.15.76:6801/276379883] conn(0x7fab30074580 0x7fab300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.917+0000 7fab2b7fe640 1 -- 172.21.15.76:0/2253059933 --> [v2:172.21.15.76:6800/276379883,v1:172.21.15.76:6801/276379883] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab30077090 con 0x7fab30074580 2023-11-09T03:45:51.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.917+0000 7f9e157fa640 1 --2- 172.21.15.76:0/663163210 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9e0406af40 0x7f9e0406d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.917+0000 7f9e157fa640 1 -- 172.21.15.76:0/663163210 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9e10089640 con 0x7f9e18071fa0 2023-11-09T03:45:51.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.917+0000 7fab42ffd640 1 --2- 172.21.15.76:0/2253059933 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fab3006af40 0x7fab3006d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.917+0000 7fab2b7fe640 1 -- 172.21.15.76:0/2253059933 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab3808e2a0 con 0x7fab44158c80 2023-11-09T03:45:51.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.917+0000 7fa894ec0640 1 --2- 172.21.15.76:0/3587147324 >> [v2:172.21.15.113:6816/1824552967,v1:172.21.15.113:6817/1824552967] conn(0x7fa8740745b0 0x7fa8740769f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.917+0000 7fe7daffd640 1 -- 172.21.15.76:0/2291115551 <== osd.1 v2:172.21.15.76:6816/1166289696 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fe7cc077090 con 0x7fe7cc074580 2023-11-09T03:45:51.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.918+0000 7f2c1cf60640 1 --2- 172.21.15.76:0/1455002017 >> [v2:172.21.15.76:6803/1864895740,v1:172.21.15.76:6806/1864895740] conn(0x7f2bfc074390 0x7f2bfc0767d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.918+0000 7f9e157fa640 1 --2- 172.21.15.76:0/663163210 >> [v2:172.21.15.76:6822/2582758863,v1:172.21.15.76:6824/2582758863] conn(0x7f9e04074410 0x7f9e04076850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.918+0000 7f9e1ccfd640 1 --2- 172.21.15.76:0/663163210 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9e0406af40 0x7f9e0406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.917+0000 7fab437fe640 1 --2- 172.21.15.76:0/2253059933 >> [v2:172.21.15.76:6800/276379883,v1:172.21.15.76:6801/276379883] conn(0x7fab30074580 0x7fab300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.918+0000 7fab42ffd640 1 --2- 172.21.15.76:0/2253059933 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fab3006af40 0x7fab3006d400 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fab34006b10 tx=0x7fab34008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.918+0000 7fab437fe640 1 --2- 172.21.15.76:0/2253059933 >> [v2:172.21.15.76:6800/276379883,v1:172.21.15.76:6801/276379883] conn(0x7fab30074580 0x7fab300769c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.918+0000 7fa885ffb640 1 -- 172.21.15.76:0/3587147324 <== osd.4 v2:172.21.15.113:6816/1824552967 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fa8740770c0 con 0x7fa8740745b0 2023-11-09T03:45:51.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.918+0000 7f9e157fa640 1 -- 172.21.15.76:0/663163210 --> [v2:172.21.15.76:6822/2582758863,v1:172.21.15.76:6824/2582758863] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9e04076f20 con 0x7f9e04074410 2023-11-09T03:45:51.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.918+0000 7f9e157fa640 1 -- 172.21.15.76:0/663163210 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e1008bc60 con 0x7f9e18071fa0 2023-11-09T03:45:51.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.918+0000 7f9e1d4fe640 1 --2- 172.21.15.76:0/663163210 >> [v2:172.21.15.76:6822/2582758863,v1:172.21.15.76:6824/2582758863] conn(0x7f9e04074410 0x7f9e04076850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.919+0000 7f2c0dffb640 1 -- 172.21.15.76:0/1455002017 <== osd.0 v2:172.21.15.76:6803/1864895740 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f2bfc076ea0 con 0x7f2bfc074390 2023-11-09T03:45:51.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.920+0000 7fab2b7fe640 1 -- 172.21.15.76:0/2253059933 <== osd.3 v2:172.21.15.76:6800/276379883 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fab30077090 con 0x7fab30074580 2023-11-09T03:45:51.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.920+0000 7f9e1d4fe640 1 --2- 172.21.15.76:0/663163210 >> [v2:172.21.15.76:6822/2582758863,v1:172.21.15.76:6824/2582758863] conn(0x7f9e04074410 0x7f9e04076850 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.921+0000 7f9e157fa640 1 -- 172.21.15.76:0/663163210 <== osd.2 v2:172.21.15.76:6822/2582758863 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f9e04076f20 con 0x7f9e04074410 2023-11-09T03:45:51.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.921+0000 7f9e1ccfd640 1 --2- 172.21.15.76:0/663163210 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9e0406af40 0x7f9e0406d400 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c009c00 tx=0x7f9e0c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.938+0000 7f521e7fc640 1 -- 172.21.15.76:0/4213987408 --> [v2:172.21.15.113:6800/3754040883,v1:172.21.15.113:6801/3754040883] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f520c002cb0 con 0x7f5218074500 2023-11-09T03:45:51.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.939+0000 7f5230ff9640 1 -- 172.21.15.76:0/4213987408 <== osd.5 v2:172.21.15.113:6800/3754040883 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f520c002cb0 con 0x7f5218074500 2023-11-09T03:45:51.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.940+0000 7f52390df640 1 -- 172.21.15.76:0/4213987408 >> [v2:172.21.15.113:6800/3754040883,v1:172.21.15.113:6801/3754040883] conn(0x7f5218074500 msgr2=0x7f5218076940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.940+0000 7f52390df640 1 --2- 172.21.15.76:0/4213987408 >> [v2:172.21.15.113:6800/3754040883,v1:172.21.15.113:6801/3754040883] conn(0x7f5218074500 0x7f5218076940 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.940+0000 7f52390df640 1 -- 172.21.15.76:0/4213987408 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f521806af40 msgr2=0x7f521806d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.940+0000 7f52390df640 1 --2- 172.21.15.76:0/4213987408 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f521806af40 0x7f521806d400 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5228007d90 tx=0x7f5228008040 comp rx=0 tx=0).stop 2023-11-09T03:45:51.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.940+0000 7f52390df640 1 -- 172.21.15.76:0/4213987408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f522c050350 msgr2=0x7f522c0c2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.940+0000 7f52390df640 1 --2- 172.21.15.76:0/4213987408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f522c050350 0x7f522c0c2480 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5234074540 tx=0x7f5234069f20 comp rx=0 tx=0).stop 2023-11-09T03:45:51.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.940+0000 7f52388de640 1 -- 172.21.15.76:0/4213987408 reap_dead start 2023-11-09T03:45:51.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.941+0000 7f52390df640 1 -- 172.21.15.76:0/4213987408 shutdown_connections 2023-11-09T03:45:51.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.941+0000 7f52390df640 1 -- 172.21.15.76:0/4213987408 >> 172.21.15.76:0/4213987408 conn(0x7f522c00a990 msgr2=0x7f522c052fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:51.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.941+0000 7f52390df640 1 -- 172.21.15.76:0/4213987408 shutdown_connections 2023-11-09T03:45:51.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.941+0000 7f52390df640 1 -- 172.21.15.76:0/4213987408 wait complete. 2023-11-09T03:45:51.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.947+0000 7f86a915e640 1 -- 172.21.15.76:0/2653126780 --> [v2:172.21.15.113:6824/2785664875,v1:172.21.15.113:6825/2785664875] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f86a41658d0 con 0x7f86900745b0 2023-11-09T03:45:51.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.948+0000 7f868b7fe640 1 -- 172.21.15.76:0/2653126780 <== osd.7 v2:172.21.15.113:6824/2785664875 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f869c00f040 con 0x7f86900745b0 2023-11-09T03:45:51.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.949+0000 7f86897fa640 1 -- 172.21.15.76:0/2653126780 >> [v2:172.21.15.113:6824/2785664875,v1:172.21.15.113:6825/2785664875] conn(0x7f86900745b0 msgr2=0x7f86900769f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.949+0000 7f86897fa640 1 --2- 172.21.15.76:0/2653126780 >> [v2:172.21.15.113:6824/2785664875,v1:172.21.15.113:6825/2785664875] conn(0x7f86900745b0 0x7f86900769f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.949+0000 7f86897fa640 1 -- 172.21.15.76:0/2653126780 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f869006af70 msgr2=0x7f869006d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.949+0000 7f86897fa640 1 --2- 172.21.15.76:0/2653126780 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f869006af70 0x7f869006d430 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f869401a7f0 tx=0x7f8694006e70 comp rx=0 tx=0).stop 2023-11-09T03:45:51.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.949+0000 7f86897fa640 1 -- 172.21.15.76:0/2653126780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86a40b5c50 msgr2=0x7f86a415b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.949+0000 7f86897fa640 1 --2- 172.21.15.76:0/2653126780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86a40b5c50 0x7f86a415b4d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f86980098c0 tx=0x7f86980046c0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.949+0000 7f86a359e640 1 -- 172.21.15.76:0/2653126780 reap_dead start 2023-11-09T03:45:51.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.949+0000 7f86897fa640 1 -- 172.21.15.76:0/2653126780 shutdown_connections 2023-11-09T03:45:51.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.950+0000 7f86897fa640 1 -- 172.21.15.76:0/2653126780 >> 172.21.15.76:0/2653126780 conn(0x7f86a40cf880 msgr2=0x7f86a4164b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:51.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.950+0000 7fe7f0be2640 1 -- 172.21.15.76:0/2291115551 --> [v2:172.21.15.76:6816/1166289696,v1:172.21.15.76:6817/1166289696] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe7d0002ca0 con 0x7fe7cc074580 2023-11-09T03:45:51.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.950+0000 7f86897fa640 1 -- 172.21.15.76:0/2653126780 shutdown_connections 2023-11-09T03:45:51.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.950+0000 7f86897fa640 1 -- 172.21.15.76:0/2653126780 wait complete. 2023-11-09T03:45:51.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.950+0000 7fe7daffd640 1 -- 172.21.15.76:0/2291115551 <== osd.1 v2:172.21.15.76:6816/1166289696 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe7d0002ca0 con 0x7fe7cc074580 2023-11-09T03:45:51.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.951+0000 7fa8966c3640 1 -- 172.21.15.76:0/3587147324 --> [v2:172.21.15.113:6816/1824552967,v1:172.21.15.113:6817/1824552967] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa89006ba60 con 0x7fa8740745b0 2023-11-09T03:45:51.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.951+0000 7fa885ffb640 1 -- 172.21.15.76:0/3587147324 <== osd.4 v2:172.21.15.113:6816/1824552967 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa89006ba60 con 0x7fa8740745b0 2023-11-09T03:45:51.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.951+0000 7fe7f0be2640 1 -- 172.21.15.76:0/2291115551 >> [v2:172.21.15.76:6816/1166289696,v1:172.21.15.76:6817/1166289696] conn(0x7fe7cc074580 msgr2=0x7fe7cc0769c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.951+0000 7fe7f0be2640 1 --2- 172.21.15.76:0/2291115551 >> [v2:172.21.15.76:6816/1166289696,v1:172.21.15.76:6817/1166289696] conn(0x7fe7cc074580 0x7fe7cc0769c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.951+0000 7f2c1f9c5640 1 -- 172.21.15.76:0/1455002017 --> [v2:172.21.15.76:6803/1864895740,v1:172.21.15.76:6806/1864895740] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2c181658d0 con 0x7f2bfc074390 2023-11-09T03:45:51.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.951+0000 7fe7f0be2640 1 -- 172.21.15.76:0/2291115551 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe7cc06af40 msgr2=0x7fe7cc06d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.951+0000 7fe7f0be2640 1 --2- 172.21.15.76:0/2291115551 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe7cc06af40 0x7fe7cc06d400 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4036000 tx=0x7fe7e4030000 comp rx=0 tx=0).stop 2023-11-09T03:45:51.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.952+0000 7fe7f0be2640 1 -- 172.21.15.76:0/2291115551 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7ec070710 msgr2=0x7fe7ec161db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.952+0000 7fe7f0be2640 1 --2- 172.21.15.76:0/2291115551 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7ec070710 0x7fe7ec161db0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00c480 tx=0x7fe7dc00c940 comp rx=0 tx=0).stop 2023-11-09T03:45:51.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.952+0000 7fe7ead9d640 1 -- 172.21.15.76:0/2291115551 reap_dead start 2023-11-09T03:45:51.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.952+0000 7fa8966c3640 1 -- 172.21.15.76:0/3587147324 >> [v2:172.21.15.113:6816/1824552967,v1:172.21.15.113:6817/1824552967] conn(0x7fa8740745b0 msgr2=0x7fa8740769f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.952+0000 7fa8966c3640 1 --2- 172.21.15.76:0/3587147324 >> [v2:172.21.15.113:6816/1824552967,v1:172.21.15.113:6817/1824552967] conn(0x7fa8740745b0 0x7fa8740769f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.952+0000 7fa8966c3640 1 -- 172.21.15.76:0/3587147324 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa87406af70 msgr2=0x7fa87406d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.952+0000 7fe7f0be2640 1 -- 172.21.15.76:0/2291115551 shutdown_connections 2023-11-09T03:45:51.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.952+0000 7fe7f0be2640 1 -- 172.21.15.76:0/2291115551 >> 172.21.15.76:0/2291115551 conn(0x7fe7ec0ce950 msgr2=0x7fe7ec0cdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:51.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.952+0000 7fa8966c3640 1 --2- 172.21.15.76:0/3587147324 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa87406af70 0x7fa87406d430 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa8900d0c10 tx=0x7fa880030000 comp rx=0 tx=0).stop 2023-11-09T03:45:51.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.952+0000 7fa8966c3640 1 -- 172.21.15.76:0/3587147324 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8900ca120 msgr2=0x7fa8900ca5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.952+0000 7fa8966c3640 1 --2- 172.21.15.76:0/3587147324 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8900ca120 0x7fa8900ca5a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa89006c4e0 tx=0x7fa88c00e960 comp rx=0 tx=0).stop 2023-11-09T03:45:51.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.952+0000 7f2c0dffb640 1 -- 172.21.15.76:0/1455002017 <== osd.0 v2:172.21.15.76:6803/1864895740 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2c14004040 con 0x7f2bfc074390 2023-11-09T03:45:51.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.952+0000 7fe7f0be2640 1 -- 172.21.15.76:0/2291115551 shutdown_connections 2023-11-09T03:45:51.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.952+0000 7fe7f0be2640 1 -- 172.21.15.76:0/2291115551 wait complete. 2023-11-09T03:45:51.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.952+0000 7fa895ec2640 1 -- 172.21.15.76:0/3587147324 reap_dead start 2023-11-09T03:45:51.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.952+0000 7fa8966c3640 1 -- 172.21.15.76:0/3587147324 shutdown_connections 2023-11-09T03:45:51.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.952+0000 7fa8966c3640 1 -- 172.21.15.76:0/3587147324 >> 172.21.15.76:0/3587147324 conn(0x7fa89006b680 msgr2=0x7fa8900cf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:51.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.953+0000 7fd24ffa9640 1 -- 172.21.15.76:0/3543056862 --> [v2:172.21.15.113:6808/570146941,v1:172.21.15.113:6809/570146941] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd2480cb180 con 0x7fd22c074480 2023-11-09T03:45:51.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.953+0000 7f9e1ef61640 1 -- 172.21.15.76:0/663163210 --> [v2:172.21.15.76:6822/2582758863,v1:172.21.15.76:6824/2582758863] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9df8002cb0 con 0x7f9e04074410 2023-11-09T03:45:51.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.953+0000 7fa8966c3640 1 -- 172.21.15.76:0/3587147324 shutdown_connections 2023-11-09T03:45:51.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.953+0000 7fa8966c3640 1 -- 172.21.15.76:0/3587147324 wait complete. 2023-11-09T03:45:51.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.953+0000 7f2c1f9c5640 1 -- 172.21.15.76:0/1455002017 >> [v2:172.21.15.76:6803/1864895740,v1:172.21.15.76:6806/1864895740] conn(0x7f2bfc074390 msgr2=0x7f2bfc0767d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.953+0000 7f2c1f9c5640 1 --2- 172.21.15.76:0/1455002017 >> [v2:172.21.15.76:6803/1864895740,v1:172.21.15.76:6806/1864895740] conn(0x7f2bfc074390 0x7f2bfc0767d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.953+0000 7f2c1f9c5640 1 -- 172.21.15.76:0/1455002017 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2bfc06af40 msgr2=0x7f2bfc06d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.953+0000 7fab49490640 1 -- 172.21.15.76:0/2253059933 --> [v2:172.21.15.76:6800/276379883,v1:172.21.15.76:6801/276379883] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fab44159b50 con 0x7fab30074580 2023-11-09T03:45:51.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.953+0000 7f2c1f9c5640 1 --2- 172.21.15.76:0/1455002017 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2bfc06af40 0x7f2bfc06d400 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2c10007fd0 tx=0x7f2c10007d00 comp rx=0 tx=0).stop 2023-11-09T03:45:51.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.953+0000 7f2c1f9c5640 1 -- 172.21.15.76:0/1455002017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c18161150 msgr2=0x7f2c180caac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.953+0000 7f2c1f9c5640 1 --2- 172.21.15.76:0/1455002017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c18161150 0x7f2c180caac0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2c0800d800 tx=0x7f2c0800dcc0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.953+0000 7f2c1df62640 1 -- 172.21.15.76:0/1455002017 reap_dead start 2023-11-09T03:45:51.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.953+0000 7f2c1f9c5640 1 -- 172.21.15.76:0/1455002017 shutdown_connections 2023-11-09T03:45:51.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.953+0000 7f2c1f9c5640 1 -- 172.21.15.76:0/1455002017 >> 172.21.15.76:0/1455002017 conn(0x7f2c180cf880 msgr2=0x7f2c18160370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:51.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.954+0000 7f2c1f9c5640 1 -- 172.21.15.76:0/1455002017 shutdown_connections 2023-11-09T03:45:51.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.954+0000 7f2c1f9c5640 1 -- 172.21.15.76:0/1455002017 wait complete. 2023-11-09T03:45:51.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.953+0000 7f9e157fa640 1 -- 172.21.15.76:0/663163210 <== osd.2 v2:172.21.15.76:6822/2582758863 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9df8002cb0 con 0x7f9e04074410 2023-11-09T03:45:51.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.953+0000 7fd23e7fc640 1 -- 172.21.15.76:0/3543056862 <== osd.6 v2:172.21.15.113:6808/570146941 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd238009890 con 0x7fd22c074480 2023-11-09T03:45:51.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.954+0000 7fab2b7fe640 1 -- 172.21.15.76:0/2253059933 <== osd.3 v2:172.21.15.76:6800/276379883 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fab3c02c920 con 0x7fab30074580 2023-11-09T03:45:51.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.954+0000 7f9e1ef61640 1 -- 172.21.15.76:0/663163210 >> [v2:172.21.15.76:6822/2582758863,v1:172.21.15.76:6824/2582758863] conn(0x7f9e04074410 msgr2=0x7f9e04076850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.954+0000 7f9e1ef61640 1 --2- 172.21.15.76:0/663163210 >> [v2:172.21.15.76:6822/2582758863,v1:172.21.15.76:6824/2582758863] conn(0x7f9e04074410 0x7f9e04076850 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.954+0000 7f9e1ef61640 1 -- 172.21.15.76:0/663163210 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9e0406af40 msgr2=0x7f9e0406d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.954+0000 7f9e1ef61640 1 --2- 172.21.15.76:0/663163210 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9e0406af40 0x7f9e0406d400 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c009c00 tx=0x7f9e0c009190 comp rx=0 tx=0).stop 2023-11-09T03:45:51.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.954+0000 7f9e1ef61640 1 -- 172.21.15.76:0/663163210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e18071fa0 msgr2=0x7f9e180c20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.954+0000 7f9e1ef61640 1 --2- 172.21.15.76:0/663163210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e18071fa0 0x7f9e180c20e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000caa0 tx=0x7f9e1000cf60 comp rx=0 tx=0).stop 2023-11-09T03:45:51.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.954+0000 7f9e1d4fe640 1 -- 172.21.15.76:0/663163210 reap_dead start 2023-11-09T03:45:51.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.954+0000 7f9e1ef61640 1 -- 172.21.15.76:0/663163210 shutdown_connections 2023-11-09T03:45:51.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.954+0000 7f9e1ef61640 1 -- 172.21.15.76:0/663163210 >> 172.21.15.76:0/663163210 conn(0x7f9e1806d800 msgr2=0x7f9e181ebab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:51.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.954+0000 7fd24ffa9640 1 -- 172.21.15.76:0/3543056862 >> [v2:172.21.15.113:6808/570146941,v1:172.21.15.113:6809/570146941] conn(0x7fd22c074480 msgr2=0x7fd22c0768c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.954+0000 7fd24ffa9640 1 --2- 172.21.15.76:0/3543056862 >> [v2:172.21.15.113:6808/570146941,v1:172.21.15.113:6809/570146941] conn(0x7fd22c074480 0x7fd22c0768c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.955+0000 7fd24ffa9640 1 -- 172.21.15.76:0/3543056862 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd22c06aec0 msgr2=0x7fd22c06d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.955+0000 7fd24ffa9640 1 --2- 172.21.15.76:0/3543056862 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd22c06aec0 0x7fd22c06d380 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd244036000 tx=0x7fd24400a000 comp rx=0 tx=0).stop 2023-11-09T03:45:51.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.955+0000 7fd24ffa9640 1 -- 172.21.15.76:0/3543056862 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd248141e20 msgr2=0x7fd2481e4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.955+0000 7fd24ffa9640 1 --2- 172.21.15.76:0/3543056862 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd248141e20 0x7fd2481e4600 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd24000cca0 tx=0x7fd24000a430 comp rx=0 tx=0).stop 2023-11-09T03:45:51.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.955+0000 7fd24e546640 1 -- 172.21.15.76:0/3543056862 reap_dead start 2023-11-09T03:45:51.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.955+0000 7fd24ffa9640 1 -- 172.21.15.76:0/3543056862 shutdown_connections 2023-11-09T03:45:51.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.955+0000 7fd24ffa9640 1 -- 172.21.15.76:0/3543056862 >> 172.21.15.76:0/3543056862 conn(0x7fd24806a120 msgr2=0x7fd24806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:51.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.955+0000 7fab49490640 1 -- 172.21.15.76:0/2253059933 >> [v2:172.21.15.76:6800/276379883,v1:172.21.15.76:6801/276379883] conn(0x7fab30074580 msgr2=0x7fab300769c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.955+0000 7fab49490640 1 --2- 172.21.15.76:0/2253059933 >> [v2:172.21.15.76:6800/276379883,v1:172.21.15.76:6801/276379883] conn(0x7fab30074580 0x7fab300769c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.955+0000 7fab49490640 1 -- 172.21.15.76:0/2253059933 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fab3006af40 msgr2=0x7fab3006d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.955+0000 7fab49490640 1 --2- 172.21.15.76:0/2253059933 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fab3006af40 0x7fab3006d400 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fab34006b10 tx=0x7fab34008040 comp rx=0 tx=0).stop 2023-11-09T03:45:51.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.955+0000 7fab49490640 1 -- 172.21.15.76:0/2253059933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab44158c80 msgr2=0x7fab4414efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.955+0000 7fab49490640 1 --2- 172.21.15.76:0/2253059933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab44158c80 0x7fab4414efe0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fab3800e9b0 tx=0x7fab3800ee70 comp rx=0 tx=0).stop 2023-11-09T03:45:51.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.955+0000 7fab437fe640 1 -- 172.21.15.76:0/2253059933 reap_dead start 2023-11-09T03:45:51.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.955+0000 7f9e1ef61640 1 -- 172.21.15.76:0/663163210 shutdown_connections 2023-11-09T03:45:51.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.955+0000 7fab49490640 1 -- 172.21.15.76:0/2253059933 shutdown_connections 2023-11-09T03:45:51.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.955+0000 7fab49490640 1 -- 172.21.15.76:0/2253059933 >> 172.21.15.76:0/2253059933 conn(0x7fab4406a120 msgr2=0x7fab441515c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:51.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.955+0000 7fd24ffa9640 1 -- 172.21.15.76:0/3543056862 shutdown_connections 2023-11-09T03:45:51.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.956+0000 7fd24ffa9640 1 -- 172.21.15.76:0/3543056862 wait complete. 2023-11-09T03:45:51.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.956+0000 7f9e1ef61640 1 -- 172.21.15.76:0/663163210 wait complete. 2023-11-09T03:45:51.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.956+0000 7fab49490640 1 -- 172.21.15.76:0/2253059933 shutdown_connections 2023-11-09T03:45:51.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:50.956+0000 7fab49490640 1 -- 172.21.15.76:0/2253059933 wait complete. 2023-11-09T03:45:51.103 INFO:teuthology.orchestra.run.smithi076.stdout:55834574857 2023-11-09T03:45:51.103 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-09T03:45:51.104 INFO:teuthology.orchestra.run.smithi076.stdout:55834574857 2023-11-09T03:45:51.105 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-09T03:45:51.105 INFO:teuthology.orchestra.run.smithi076.stdout:55834574856 2023-11-09T03:45:51.106 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-09T03:45:51.107 INFO:teuthology.orchestra.run.smithi076.stdout:55834574856 2023-11-09T03:45:51.107 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-09T03:45:51.108 INFO:teuthology.orchestra.run.smithi076.stdout:55834574857 2023-11-09T03:45:51.108 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-09T03:45:51.109 INFO:teuthology.orchestra.run.smithi076.stdout:55834574857 2023-11-09T03:45:51.109 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-09T03:45:51.110 INFO:teuthology.orchestra.run.smithi076.stdout:55834574858 2023-11-09T03:45:51.110 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-09T03:45:51.111 INFO:teuthology.orchestra.run.smithi076.stdout:55834574856 2023-11-09T03:45:51.112 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-09T03:45:51.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.312+0000 7f832b0f3640 1 Processor -- start 2023-11-09T03:45:51.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.312+0000 7f82de051640 1 Processor -- start 2023-11-09T03:45:51.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.312+0000 7f82de051640 1 -- start start 2023-11-09T03:45:51.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.312+0000 7f832b0f3640 1 -- start start 2023-11-09T03:45:51.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.314+0000 7f82de051640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f82d815aa10 0x7f82d815ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.314+0000 7f82de051640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82d80cec40 con 0x7f82d80ce840 2023-11-09T03:45:51.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.314+0000 7f82de051640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82d815f4b0 con 0x7f82d80ce3d0 2023-11-09T03:45:51.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.314+0000 7f82de051640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82d815b210 con 0x7f82d815aa10 2023-11-09T03:45:51.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.314+0000 7f82d7fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f82d80ce840 0x7f82d80b2280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:56776/0 (socket says 172.21.15.76:56776) 2023-11-09T03:45:51.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.314+0000 7f82d7fff640 1 -- 172.21.15.76:0/2504845870 learned_addr learned my addr 172.21.15.76:0/2504845870 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:51.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.314+0000 7f832b0f3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83240cf6a0 0x7f832416c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.314+0000 7f832b0f3640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83240cb5c0 con 0x7f832406b230 2023-11-09T03:45:51.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.314+0000 7f83298f0640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f832406b230 0x7f83240cef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:56778/0 (socket says 172.21.15.76:56778) 2023-11-09T03:45:51.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.314+0000 7f83298f0640 1 -- 172.21.15.76:0/288526680 learned_addr learned my addr 172.21.15.76:0/288526680 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:51.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.314+0000 7f832b0f3640 1 -- 172.21.15.76:0/288526680 --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83240c3f10 con 0x7f83240affb0 2023-11-09T03:45:51.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.314+0000 7f832b0f3640 1 -- 172.21.15.76:0/288526680 --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8324161a30 con 0x7f83240cf6a0 2023-11-09T03:45:51.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.314+0000 7fa9109d0640 1 Processor -- start 2023-11-09T03:45:51.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.314+0000 7fa9109d0640 1 -- start start 2023-11-09T03:45:51.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.315+0000 7f83290ef640 1 -- 172.21.15.76:0/288526680 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537092197 0 0) 0x7f83240cb5c0 con 0x7f832406b230 2023-11-09T03:45:51.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.315+0000 7f83290ef640 1 -- 172.21.15.76:0/288526680 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8310003520 con 0x7f832406b230 2023-11-09T03:45:51.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.315+0000 7f832a8f2640 1 --2- 172.21.15.76:0/288526680 >> v2:172.21.15.113:3300/0 conn(0x7f83240cf6a0 0x7f832416c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.315+0000 7f82d67fc640 1 -- 172.21.15.76:0/2504845870 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2298581684 0 0) 0x7f82d80cec40 con 0x7f82d80ce840 2023-11-09T03:45:51.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.315+0000 7f82d67fc640 1 -- 172.21.15.76:0/2504845870 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82c8003520 con 0x7f82d80ce840 2023-11-09T03:45:51.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.315+0000 7f82d67fc640 1 -- 172.21.15.76:0/2504845870 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 727898150 0 0) 0x7f82c8003520 con 0x7f82d80ce840 2023-11-09T03:45:51.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.315+0000 7f82d67fc640 1 -- 172.21.15.76:0/2504845870 >> v2:172.21.15.113:3300/0 conn(0x7f82d815aa10 msgr2=0x7f82d815ef70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-09T03:45:51.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.315+0000 7f82d67fc640 1 --2- 172.21.15.76:0/2504845870 >> v2:172.21.15.113:3300/0 conn(0x7f82d815aa10 0x7f82d815ef70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.315+0000 7f82d67fc640 1 -- 172.21.15.76:0/2504845870 >> v1:172.21.15.113:6789/0 conn(0x7f82d80ce3d0 legacy=0x7f82d815b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:51.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.315+0000 7f82d67fc640 1 -- 172.21.15.76:0/2504845870 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d80cb3b0 con 0x7f82d80ce840 2023-11-09T03:45:51.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.315+0000 7f82d67fc640 1 -- 172.21.15.76:0/2504845870 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f82d0001890 con 0x7f82d80ce840 2023-11-09T03:45:51.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.315+0000 7fa9109d0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa90c071fa0 0x7fa90c06a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.315+0000 7f82d67fc640 1 -- 172.21.15.76:0/2504845870 >> v1:172.21.15.76:6789/0 conn(0x7f82d80ce840 legacy=0x7f82d80b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.315+0000 7f82d67fc640 1 --2- 172.21.15.76:0/2504845870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82c80039c0 0x7f82c8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.315+0000 7f82d67fc640 1 --2- 172.21.15.76:0/2504845870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82c80047c0 0x7f82c8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.315+0000 7f82d67fc640 1 --2- 172.21.15.76:0/2504845870 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82c80053d0 0x7f82c800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.315+0000 7f82d67fc640 1 -- 172.21.15.76:0/2504845870 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d815b210 con 0x7f82c80047c0 2023-11-09T03:45:51.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.315+0000 7f82d67fc640 1 -- 172.21.15.76:0/2504845870 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c800c310 con 0x7f82c80039c0 2023-11-09T03:45:51.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.315+0000 7f82d67fc640 1 -- 172.21.15.76:0/2504845870 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c800c4d0 con 0x7f82c80053d0 2023-11-09T03:45:51.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7fa90a59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa90c071fa0 0x7fa90c06a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7fa90a59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa90c071fa0 0x7fa90c06a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57778/0 (socket says 172.21.15.76:57778) 2023-11-09T03:45:51.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7fa909d9b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa90c0b2010 0x7fa90c0cc3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:46100/0 (socket says 172.21.15.76:46100) 2023-11-09T03:45:51.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7fa9109d0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa90c06a660 0x7fa90c0ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7fa90a59c640 1 -- 172.21.15.76:0/1476618281 learned_addr learned my addr 172.21.15.76:0/1476618281 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:51.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7f6295b05640 1 Processor -- start 2023-11-09T03:45:51.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.315+0000 7f83290ef640 1 -- 172.21.15.76:0/288526680 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1763090476 0 0) 0x7f8310003520 con 0x7f832406b230 2023-11-09T03:45:51.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.315+0000 7f83290ef640 1 -- 172.21.15.76:0/288526680 >> v2:172.21.15.113:3300/0 conn(0x7f83240cf6a0 msgr2=0x7f832416c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:51.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.315+0000 7f83290ef640 1 --2- 172.21.15.76:0/288526680 >> v2:172.21.15.113:3300/0 conn(0x7f83240cf6a0 0x7f832416c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.315+0000 7f83290ef640 1 -- 172.21.15.76:0/288526680 >> v1:172.21.15.113:6789/0 conn(0x7f83240affb0 legacy=0x7f83240b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.315+0000 7f83290ef640 1 -- 172.21.15.76:0/288526680 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83240c3950 con 0x7f832406b230 2023-11-09T03:45:51.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.315+0000 7f83290ef640 1 -- 172.21.15.76:0/288526680 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f8320001670 con 0x7f832406b230 2023-11-09T03:45:51.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7f83290ef640 1 -- 172.21.15.76:0/288526680 >> v1:172.21.15.76:6789/0 conn(0x7f832406b230 legacy=0x7f83240cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7f83290ef640 1 --2- 172.21.15.76:0/288526680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8310003970 0x7f8310003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7f83290ef640 1 --2- 172.21.15.76:0/288526680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83100047c0 0x7f8310004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7f83290ef640 1 --2- 172.21.15.76:0/288526680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8310005180 0x7f831000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7f83290ef640 1 -- 172.21.15.76:0/288526680 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8324161a30 con 0x7f8310005180 2023-11-09T03:45:51.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7f83290ef640 1 -- 172.21.15.76:0/288526680 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831000c120 con 0x7f8310003970 2023-11-09T03:45:51.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7f83290ef640 1 -- 172.21.15.76:0/288526680 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f831000c2e0 con 0x7f83100047c0 2023-11-09T03:45:51.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7f832a8f2640 1 --2- 172.21.15.76:0/288526680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8310005180 0x7f831000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7f83298f0640 1 --2- 172.21.15.76:0/288526680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83100047c0 0x7f8310004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7f6295b05640 1 -- start start 2023-11-09T03:45:51.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7fa9109d0640 1 -- 172.21.15.76:0/1476618281 --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa90c0cdfb0 con 0x7fa90c0b2010 2023-11-09T03:45:51.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7fa9109d0640 1 -- 172.21.15.76:0/1476618281 --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa90c06ab00 con 0x7fa90c071fa0 2023-11-09T03:45:51.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7fa9109d0640 1 -- 172.21.15.76:0/1476618281 --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa90c0cd680 con 0x7fa90c06a660 2023-11-09T03:45:51.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7fa90ad9d640 1 --2- 172.21.15.76:0/1476618281 >> v2:172.21.15.113:3300/0 conn(0x7fa90c06a660 0x7fa90c0ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7f82d6ffd640 1 --2- 172.21.15.76:0/2504845870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82c80047c0 0x7f82c8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7f82d77fe640 1 --2- 172.21.15.76:0/2504845870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82c80039c0 0x7f82c8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7f82d7fff640 1 --2- 172.21.15.76:0/2504845870 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82c80053d0 0x7f82c800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7f82d6ffd640 1 -- 172.21.15.76:0/2504845870 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82c80053d0 msgr2=0x7f82c800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:51.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7f82d6ffd640 1 --2- 172.21.15.76:0/2504845870 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82c80053d0 0x7f82c800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7f82d6ffd640 1 -- 172.21.15.76:0/2504845870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82c80039c0 msgr2=0x7f82c8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:51.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7f82d6ffd640 1 --2- 172.21.15.76:0/2504845870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82c80039c0 0x7f82c8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7f82d6ffd640 1 -- 172.21.15.76:0/2504845870 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c8003520 con 0x7f82c80047c0 2023-11-09T03:45:51.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7f832a0f1640 1 --2- 172.21.15.76:0/288526680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8310003970 0x7f8310003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7f83298f0640 1 -- 172.21.15.76:0/288526680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8310003970 msgr2=0x7f8310003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:51.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7f83298f0640 1 --2- 172.21.15.76:0/288526680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8310003970 0x7f8310003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7f83298f0640 1 -- 172.21.15.76:0/288526680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8310005180 msgr2=0x7f831000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:51.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7f83298f0640 1 --2- 172.21.15.76:0/288526680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8310005180 0x7f831000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.316+0000 7f832a8f2640 1 -- 172.21.15.76:0/288526680 reap_dead start 2023-11-09T03:45:51.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.317+0000 7f83298f0640 1 -- 172.21.15.76:0/288526680 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83240c3950 con 0x7f83100047c0 2023-11-09T03:45:51.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.317+0000 7fa90ad9d640 1 -- 172.21.15.76:0/1476618281 >> v2:172.21.15.76:3300/0 conn(0x7fa90c071fa0 msgr2=0x7fa90c06a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:51.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.317+0000 7fa90ad9d640 1 --2- 172.21.15.76:0/1476618281 >> v2:172.21.15.76:3300/0 conn(0x7fa90c071fa0 0x7fa90c06a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.317+0000 7fa90ad9d640 1 -- 172.21.15.76:0/1476618281 >> v1:172.21.15.113:6789/0 conn(0x7fa90c0b2010 legacy=0x7fa90c0cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.317+0000 7fa90ad9d640 1 -- 172.21.15.76:0/1476618281 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90c0cd960 con 0x7fa90c06a660 2023-11-09T03:45:51.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.317+0000 7fd47d7c0640 1 Processor -- start 2023-11-09T03:45:51.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.317+0000 7fd47d7c0640 1 -- start start 2023-11-09T03:45:51.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.317+0000 7f6295b05640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f62900cb990 0x7f62900c9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.317+0000 7f6295b05640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f62900b1860 0x7f62900c9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.318+0000 7f6295b05640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62901616b0 con 0x7f62901612d0 2023-11-09T03:45:51.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.318+0000 7f628f7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f62901612d0 0x7f62900cb280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:56792/0 (socket says 172.21.15.76:56792) 2023-11-09T03:45:51.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.318+0000 7f628f7fe640 1 -- 172.21.15.76:0/3244957031 learned_addr learned my addr 172.21.15.76:0/3244957031 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:51.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.318+0000 7f628effd640 1 --2- 172.21.15.76:0/3244957031 >> v2:172.21.15.113:3300/0 conn(0x7f62900cb990 0x7f62900c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.317+0000 7f82d7fff640 1 -- 172.21.15.76:0/2504845870 reap_dead start 2023-11-09T03:45:51.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.318+0000 7f82d6ffd640 1 --2- 172.21.15.76:0/2504845870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82c80047c0 0x7f82c8004c40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f82c4003db0 tx=0x7f82c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389b5f6e5ebfdece server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.318+0000 7f628ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f62900b1860 0x7f62900c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.318+0000 7f6295b05640 1 -- 172.21.15.76:0/3244957031 --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62900b00e0 con 0x7f62900b1860 2023-11-09T03:45:51.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.318+0000 7f6295b05640 1 -- 172.21.15.76:0/3244957031 --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62900af7b0 con 0x7f62900cb990 2023-11-09T03:45:51.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.318+0000 7f83298f0640 1 --2- 172.21.15.76:0/288526680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83100047c0 0x7f8310004c40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8310005560 tx=0x7f83100057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc2afc35237fba4b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.318+0000 7fa90a59c640 1 --2- 172.21.15.76:0/1476618281 >> v2:172.21.15.76:3300/0 conn(0x7fa90c071fa0 0x7fa90c06a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:51.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.318+0000 7fa90ad9d640 1 --2- 172.21.15.76:0/1476618281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa90c06a660 0x7fa90c0ccae0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa90c072900 tx=0x7fa90402bd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8763efb6468fc307 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.318+0000 7fa90959a640 1 -- 172.21.15.76:0/1476618281 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90402ca50 con 0x7fa90c06a660 2023-11-09T03:45:51.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.318+0000 7fa90959a640 1 -- 172.21.15.76:0/1476618281 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90402cbb0 con 0x7fa90c06a660 2023-11-09T03:45:51.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.318+0000 7fa90959a640 1 -- 172.21.15.76:0/1476618281 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa904034440 con 0x7fa90c06a660 2023-11-09T03:45:51.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.318+0000 7fc7ad400640 1 Processor -- start 2023-11-09T03:45:51.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.318+0000 7fc7ad400640 1 -- start start 2023-11-09T03:45:51.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.318+0000 7f82d67fc640 1 -- 172.21.15.76:0/2504845870 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4007220 con 0x7f82c80047c0 2023-11-09T03:45:51.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.318+0000 7f82d67fc640 1 -- 172.21.15.76:0/2504845870 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c402e050 con 0x7f82c80047c0 2023-11-09T03:45:51.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.319+0000 7fd476ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd4700e3350 0x7fd470053a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:56808/0 (socket says 172.21.15.76:56808) 2023-11-09T03:45:51.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.319+0000 7fd476ffd640 1 -- 172.21.15.76:0/239281669 learned_addr learned my addr 172.21.15.76:0/239281669 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:51.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.319+0000 7fd47d7c0640 1 --2- 172.21.15.76:0/239281669 >> v2:172.21.15.76:3300/0 conn(0x7fd470050120 0x7fd4700ee1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.319+0000 7fd47d7c0640 1 -- 172.21.15.76:0/239281669 --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd470006340 con 0x7fd4700e3350 2023-11-09T03:45:51.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.319+0000 7f82d67fc640 1 -- 172.21.15.76:0/2504845870 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4007220 con 0x7f82c80047c0 2023-11-09T03:45:51.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.319+0000 7fd47d7c0640 1 -- 172.21.15.76:0/239281669 --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd470006010 con 0x7fd47003a860 2023-11-09T03:45:51.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.319+0000 7fd47d7c0640 1 -- 172.21.15.76:0/239281669 --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4700e3730 con 0x7fd470050120 2023-11-09T03:45:51.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.319+0000 7fa9109d0640 1 -- 172.21.15.76:0/1476618281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa90c06a660 msgr2=0x7fa90c0ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.319+0000 7fa9109d0640 1 --2- 172.21.15.76:0/1476618281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa90c06a660 0x7fa90c0ccae0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa90c072900 tx=0x7fa90402bd20 comp rx=0 tx=0).stop 2023-11-09T03:45:51.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.319+0000 7fd4777fe640 1 --2- 172.21.15.76:0/239281669 >> v2:172.21.15.76:3300/0 conn(0x7fd470050120 0x7fd4700ee1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.319+0000 7f83290ef640 1 -- 172.21.15.76:0/288526680 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8320005ad0 con 0x7f83100047c0 2023-11-09T03:45:51.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.319+0000 7f83290ef640 1 -- 172.21.15.76:0/288526680 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8320005c30 con 0x7f83100047c0 2023-11-09T03:45:51.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.319+0000 7f83290ef640 1 -- 172.21.15.76:0/288526680 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8320007070 con 0x7f83100047c0 2023-11-09T03:45:51.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.319+0000 7fc7ad400640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc7a00df460 0x7fc7a0007520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.319+0000 7f628ffff640 1 -- 172.21.15.76:0/3244957031 >> v2:172.21.15.113:3300/0 conn(0x7f62900cb990 msgr2=0x7f62900c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:51.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.319+0000 7f628ffff640 1 --2- 172.21.15.76:0/3244957031 >> v2:172.21.15.113:3300/0 conn(0x7f62900cb990 0x7f62900c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.319+0000 7f628ffff640 1 -- 172.21.15.76:0/3244957031 >> v1:172.21.15.76:6789/0 conn(0x7f62901612d0 legacy=0x7f62900cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.319+0000 7f628ffff640 1 -- 172.21.15.76:0/3244957031 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62900afa90 con 0x7f62900b1860 2023-11-09T03:45:51.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.320+0000 7fa9109d0640 1 -- 172.21.15.76:0/1476618281 shutdown_connections 2023-11-09T03:45:51.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.320+0000 7fa9109d0640 1 --2- 172.21.15.76:0/1476618281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa90c06a660 0x7fa90c0ccae0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.320+0000 7fa9109d0640 1 --2- 172.21.15.76:0/1476618281 >> v2:172.21.15.76:3300/0 conn(0x7fa90c071fa0 0x7fa90c06a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.320+0000 7fa9109d0640 1 -- 172.21.15.76:0/1476618281 >> 172.21.15.76:0/1476618281 conn(0x7fa90c06d800 msgr2=0x7fa90c06dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:51.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.320+0000 7fa9109d0640 1 -- 172.21.15.76:0/1476618281 shutdown_connections 2023-11-09T03:45:51.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.320+0000 7fa9109d0640 1 -- 172.21.15.76:0/1476618281 wait complete. 2023-11-09T03:45:51.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.320+0000 7f82de051640 1 -- 172.21.15.76:0/2504845870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82c80047c0 msgr2=0x7f82c8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.320+0000 7f82de051640 1 --2- 172.21.15.76:0/2504845870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82c80047c0 0x7f82c8004c40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f82c4003db0 tx=0x7f82c80057f0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.320+0000 7fc7ad400640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7a00537e0 con 0x7fc7a003b0d0 2023-11-09T03:45:51.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.320+0000 7f82de051640 1 -- 172.21.15.76:0/2504845870 shutdown_connections 2023-11-09T03:45:51.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.320+0000 7fc7ad400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7a0007e50 con 0x7fc7a0050770 2023-11-09T03:45:51.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.320+0000 7fc7ad400640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7a0008130 con 0x7fc7a00df460 2023-11-09T03:45:51.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.320+0000 7f82de051640 1 --2- 172.21.15.76:0/2504845870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82c80047c0 0x7f82c8004c40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.320+0000 7fc7a67fc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc7a003b0d0 0x7fc7a0050060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:56810/0 (socket says 172.21.15.76:56810) 2023-11-09T03:45:51.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.320+0000 7fc7a67fc640 1 -- 172.21.15.76:0/1962828474 learned_addr learned my addr 172.21.15.76:0/1962828474 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:51.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.320+0000 7f628ffff640 1 --2- 172.21.15.76:0/3244957031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62900b1860 0x7f62900c9580 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f62900b4b20 tx=0x7f628802bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5bc807bfe4fc90 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.320+0000 7f32d3b55640 1 Processor -- start 2023-11-09T03:45:51.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.320+0000 7f82de051640 1 -- 172.21.15.76:0/2504845870 >> 172.21.15.76:0/2504845870 conn(0x7f82d80cd280 msgr2=0x7f82d80cd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:51.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.320+0000 7f82de051640 1 -- 172.21.15.76:0/2504845870 shutdown_connections 2023-11-09T03:45:51.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.320+0000 7f832b0f3640 1 -- 172.21.15.76:0/288526680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83100047c0 msgr2=0x7f8310004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.320+0000 7f832b0f3640 1 --2- 172.21.15.76:0/288526680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83100047c0 0x7f8310004c40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8310005560 tx=0x7f83100057f0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.320+0000 7fc7a5ffb640 1 -- 172.21.15.76:0/1962828474 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 186466261 0 0) 0x7fc7a00537e0 con 0x7fc7a003b0d0 2023-11-09T03:45:51.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.320+0000 7fc7a5ffb640 1 -- 172.21.15.76:0/1962828474 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc790003520 con 0x7fc7a003b0d0 2023-11-09T03:45:51.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.320+0000 7f32d3b55640 1 -- start start 2023-11-09T03:45:51.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.320+0000 7fc7a5ffb640 1 -- 172.21.15.76:0/1962828474 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 298531695 0 0) 0x7fc7a0007e50 con 0x7fc7a0050770 2023-11-09T03:45:51.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.321+0000 7fc7a5ffb640 1 -- 172.21.15.76:0/1962828474 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7a00537e0 con 0x7fc7a0050770 2023-11-09T03:45:51.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.320+0000 7fd475ffb640 1 -- 172.21.15.76:0/239281669 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 438897537 0 0) 0x7fd470006340 con 0x7fd4700e3350 2023-11-09T03:45:51.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.320+0000 7fd475ffb640 1 -- 172.21.15.76:0/239281669 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd460003520 con 0x7fd4700e3350 2023-11-09T03:45:51.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.321+0000 7fd4777fe640 1 -- 172.21.15.76:0/239281669 >> v1:172.21.15.113:6789/0 conn(0x7fd47003a860 legacy=0x7fd47004fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.321+0000 7fd4777fe640 1 -- 172.21.15.76:0/239281669 >> v1:172.21.15.76:6789/0 conn(0x7fd4700e3350 legacy=0x7fd470053a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.321+0000 7fd4777fe640 1 -- 172.21.15.76:0/239281669 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd470054270 con 0x7fd470050120 2023-11-09T03:45:51.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.321+0000 7f832b0f3640 1 -- 172.21.15.76:0/288526680 shutdown_connections 2023-11-09T03:45:51.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.321+0000 7f832b0f3640 1 --2- 172.21.15.76:0/288526680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83100047c0 0x7f8310004c40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.321+0000 7f832b0f3640 1 -- 172.21.15.76:0/288526680 >> 172.21.15.76:0/288526680 conn(0x7f83240cd430 msgr2=0x7f83240cd810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:51.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.321+0000 7f82de051640 1 -- 172.21.15.76:0/2504845870 wait complete. 2023-11-09T03:45:51.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.321+0000 7f628e7fc640 1 -- 172.21.15.76:0/3244957031 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628802ccb0 con 0x7f62900b1860 2023-11-09T03:45:51.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.321+0000 7f628e7fc640 1 -- 172.21.15.76:0/3244957031 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628802ce10 con 0x7f62900b1860 2023-11-09T03:45:51.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.321+0000 7fc7a6ffd640 1 --2- 172.21.15.76:0/1962828474 >> v2:172.21.15.76:3300/0 conn(0x7fc7a00df460 0x7fc7a0007520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.321+0000 7fc7a5ffb640 1 -- 172.21.15.76:0/1962828474 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 530124821 0 0) 0x7fc790003520 con 0x7fc7a003b0d0 2023-11-09T03:45:51.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.321+0000 7f628e7fc640 1 -- 172.21.15.76:0/3244957031 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6288034640 con 0x7f62900b1860 2023-11-09T03:45:51.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.321+0000 7fc7a5ffb640 1 -- 172.21.15.76:0/1962828474 >> v2:172.21.15.76:3300/0 conn(0x7fc7a00df460 msgr2=0x7fc7a0007520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:51.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.321+0000 7fc7a5ffb640 1 --2- 172.21.15.76:0/1962828474 >> v2:172.21.15.76:3300/0 conn(0x7fc7a00df460 0x7fc7a0007520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.321+0000 7fc7a5ffb640 1 -- 172.21.15.76:0/1962828474 >> v1:172.21.15.113:6789/0 conn(0x7fc7a0050770 legacy=0x7fc7a0053c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.321+0000 7fc7a5ffb640 1 -- 172.21.15.76:0/1962828474 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a0008410 con 0x7fc7a003b0d0 2023-11-09T03:45:51.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.321+0000 7fc7a5ffb640 1 -- 172.21.15.76:0/1962828474 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fc79c0015a0 con 0x7fc7a003b0d0 2023-11-09T03:45:51.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.321+0000 7fc7a5ffb640 1 -- 172.21.15.76:0/1962828474 >> v1:172.21.15.76:6789/0 conn(0x7fc7a003b0d0 legacy=0x7fc7a0050060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.322+0000 7fc7a5ffb640 1 --2- 172.21.15.76:0/1962828474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc790003b30 0x7fc790003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.322+0000 7fc7a5ffb640 1 --2- 172.21.15.76:0/1962828474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7900047c0 0x7fc790004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.322+0000 7fc7a5ffb640 1 --2- 172.21.15.76:0/1962828474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc790005180 0x7fc79000b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.322+0000 7fc7a6ffd640 1 --2- 172.21.15.76:0/1962828474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc790003b30 0x7fc790003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.321+0000 7fa9109d0640 1 Processor -- start 2023-11-09T03:45:51.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.322+0000 7fa9109d0640 1 -- start start 2023-11-09T03:45:51.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.322+0000 7f32d3b55640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32cc15af20 0x7f32cc0b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.322+0000 7fc7a67fc640 1 --2- 172.21.15.76:0/1962828474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7900047c0 0x7fc790004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.322+0000 7fc7a77fe640 1 --2- 172.21.15.76:0/1962828474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc790005180 0x7fc79000b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.322+0000 7f32d3b55640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f32cc0b64c0 0x7f32cc0cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.322+0000 7fd4777fe640 1 --2- 172.21.15.76:0/239281669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd470050120 0x7fd4700ee1c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd478074540 tx=0x7fd4780a60e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10ed1e2c3b2976d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.322+0000 7fc7a5ffb640 1 -- 172.21.15.76:0/1962828474 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a8073a90 con 0x7fc790005180 2023-11-09T03:45:51.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.322+0000 7fc7a5ffb640 1 -- 172.21.15.76:0/1962828474 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a0008130 con 0x7fc7900047c0 2023-11-09T03:45:51.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.322+0000 7fd475ffb640 1 -- 172.21.15.76:0/239281669 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4780a6d40 con 0x7fd470050120 2023-11-09T03:45:51.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.322+0000 7f6295b05640 1 -- 172.21.15.76:0/3244957031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62900b1860 msgr2=0x7f62900c9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.322+0000 7fc7a5ffb640 1 -- 172.21.15.76:0/1962828474 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc79000bfb0 con 0x7fc790003b30 2023-11-09T03:45:51.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.322+0000 7f32d18f1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32cc15af20 0x7f32cc0b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.322+0000 7f32d18f1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32cc15af20 0x7f32cc0b3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:51056/0 (socket says 172.21.15.76:51056) 2023-11-09T03:45:51.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.322+0000 7f6295b05640 1 --2- 172.21.15.76:0/3244957031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62900b1860 0x7f62900c9580 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f62900b4b20 tx=0x7f628802bf00 comp rx=0 tx=0).stop 2023-11-09T03:45:51.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.322+0000 7f32d18f1640 1 -- 172.21.15.76:0/3952560771 learned_addr learned my addr 172.21.15.76:0/3952560771 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:51.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.322+0000 7f32d3b55640 1 -- 172.21.15.76:0/3952560771 --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32cc0d01f0 con 0x7f32cc0cd3b0 2023-11-09T03:45:51.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.322+0000 7f32d3b55640 1 -- 172.21.15.76:0/3952560771 --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32cc0b6a40 con 0x7f32cc0b64c0 2023-11-09T03:45:51.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.322+0000 7f32d3b55640 1 -- 172.21.15.76:0/3952560771 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc0d04b0 con 0x7f32cc15af20 2023-11-09T03:45:51.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.322+0000 7fd475ffb640 1 -- 172.21.15.76:0/239281669 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4780ae440 con 0x7fd470050120 2023-11-09T03:45:51.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.322+0000 7fd475ffb640 1 -- 172.21.15.76:0/239281669 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4780ae870 con 0x7fd470050120 2023-11-09T03:45:51.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.322+0000 7f32d10f0640 1 --2- 172.21.15.76:0/3952560771 >> v2:172.21.15.76:3300/0 conn(0x7f32cc0b64c0 0x7f32cc0cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7f832b0f3640 1 -- 172.21.15.76:0/288526680 shutdown_connections 2023-11-09T03:45:51.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7f832b0f3640 1 -- 172.21.15.76:0/288526680 wait complete. 2023-11-09T03:45:51.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7f82de051640 1 Processor -- start 2023-11-09T03:45:51.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7f32d18f1640 1 -- 172.21.15.76:0/3952560771 >> v2:172.21.15.76:3300/0 conn(0x7f32cc0b64c0 msgr2=0x7f32cc0cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:51.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7f32d08ef640 1 -- 172.21.15.76:0/3952560771 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3052602468 0 0) 0x7f32cc0d01f0 con 0x7f32cc0cd3b0 2023-11-09T03:45:51.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7f32d18f1640 1 --2- 172.21.15.76:0/3952560771 >> v2:172.21.15.76:3300/0 conn(0x7f32cc0b64c0 0x7f32cc0cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7f32d18f1640 1 -- 172.21.15.76:0/3952560771 >> v1:172.21.15.113:6789/0 conn(0x7f32cc0cd3b0 legacy=0x7f32cc0b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7fa9109d0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa90c06a660 0x7fa90c1eb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7f32d18f1640 1 -- 172.21.15.76:0/3952560771 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32cc0d0790 con 0x7f32cc15af20 2023-11-09T03:45:51.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7f32d10f0640 1 --2- 172.21.15.76:0/3952560771 >> v2:172.21.15.76:3300/0 conn(0x7f32cc0b64c0 0x7f32cc0cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:51.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7fa9109d0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa90c071fa0 0x7fa90c0b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7f6295b05640 1 -- 172.21.15.76:0/3244957031 shutdown_connections 2023-11-09T03:45:51.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7f6295b05640 1 --2- 172.21.15.76:0/3244957031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62900b1860 0x7f62900c9580 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7fa90a59c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa90c06a660 0x7fa90c1eb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7fa90a59c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa90c06a660 0x7fa90c1eb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:51066/0 (socket says 172.21.15.76:51066) 2023-11-09T03:45:51.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7fa90a59c640 1 -- 172.21.15.76:0/1470690993 learned_addr learned my addr 172.21.15.76:0/1470690993 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:51.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7fa9109d0640 1 --2- 172.21.15.76:0/1470690993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa90c0b2010 0x7fa90c1e94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7fa909d9b640 1 --2- 172.21.15.76:0/1470690993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa90c071fa0 0x7fa90c0b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7fa9109d0640 1 -- 172.21.15.76:0/1470690993 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90c0b2c40 con 0x7fa90c071fa0 2023-11-09T03:45:51.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7fa9109d0640 1 -- 172.21.15.76:0/1470690993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90c0b2f20 con 0x7fa90c0b2010 2023-11-09T03:45:51.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7fa9109d0640 1 -- 172.21.15.76:0/1470690993 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa90c0b3200 con 0x7fa90c06a660 2023-11-09T03:45:51.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7f6295b05640 1 --2- 172.21.15.76:0/3244957031 >> v2:172.21.15.113:3300/0 conn(0x7f62900cb990 0x7f62900c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7f6295b05640 1 -- 172.21.15.76:0/3244957031 >> 172.21.15.76:0/3244957031 conn(0x7f629006c7a0 msgr2=0x7f629006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:51.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.322+0000 7fc7a67fc640 1 -- 172.21.15.76:0/1962828474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc790003b30 msgr2=0x7fc790003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:51.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.322+0000 7fc7a67fc640 1 --2- 172.21.15.76:0/1962828474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc790003b30 0x7fc790003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7fc7a67fc640 1 -- 172.21.15.76:0/1962828474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc790005180 msgr2=0x7fc79000b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:51.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7fc7a67fc640 1 --2- 172.21.15.76:0/1962828474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc790005180 0x7fc79000b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7fc7a67fc640 1 -- 172.21.15.76:0/1962828474 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc790003520 con 0x7fc7900047c0 2023-11-09T03:45:51.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7fc7a77fe640 1 -- 172.21.15.76:0/1962828474 reap_dead start 2023-11-09T03:45:51.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7fd47d7c0640 1 -- 172.21.15.76:0/239281669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd470050120 msgr2=0x7fd4700ee1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.324+0000 7fd47d7c0640 1 --2- 172.21.15.76:0/239281669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd470050120 0x7fd4700ee1c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd478074540 tx=0x7fd4780a60e0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7fc7a6ffd640 1 --2- 172.21.15.76:0/1962828474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc790003b30 0x7fc790003f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:51.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.323+0000 7f82de051640 1 -- start start 2023-11-09T03:45:51.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.324+0000 7fc7a67fc640 1 --2- 172.21.15.76:0/1962828474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7900047c0 0x7fc790004c40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc79c007ef0 tx=0x7fc7900057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64d1aa6beed236e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.324+0000 7fa90ad9d640 1 --2- 172.21.15.76:0/1470690993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa90c0b2010 0x7fa90c1e94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.324+0000 7fa90a59c640 1 -- 172.21.15.76:0/1470690993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa90c0b2010 msgr2=0x7fa90c1e94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.324+0000 7fa90a59c640 1 --2- 172.21.15.76:0/1470690993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa90c0b2010 0x7fa90c1e94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.324+0000 7fa90a59c640 1 -- 172.21.15.76:0/1470690993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa90c071fa0 msgr2=0x7fa90c0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.324+0000 7fa90a59c640 1 --2- 172.21.15.76:0/1470690993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa90c071fa0 0x7fa90c0b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.324+0000 7fd47d7c0640 1 -- 172.21.15.76:0/239281669 shutdown_connections 2023-11-09T03:45:51.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.324+0000 7fd47d7c0640 1 --2- 172.21.15.76:0/239281669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd470050120 0x7fd4700ee1c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.324+0000 7fd47d7c0640 1 -- 172.21.15.76:0/239281669 >> 172.21.15.76:0/239281669 conn(0x7fd470054c40 msgr2=0x7fd470055020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:51.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.324+0000 7fc7a5ffb640 1 -- 172.21.15.76:0/1962828474 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c02dc80 con 0x7fc7900047c0 2023-11-09T03:45:51.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.324+0000 7f6295b05640 1 -- 172.21.15.76:0/3244957031 shutdown_connections 2023-11-09T03:45:51.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.324+0000 7fc7a5ffb640 1 -- 172.21.15.76:0/1962828474 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79c02dde0 con 0x7fc7900047c0 2023-11-09T03:45:51.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.324+0000 7fc7a5ffb640 1 -- 172.21.15.76:0/1962828474 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c005b10 con 0x7fc7900047c0 2023-11-09T03:45:51.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.324+0000 7fa90a59c640 1 -- 172.21.15.76:0/1470690993 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa904009a70 con 0x7fa90c06a660 2023-11-09T03:45:51.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.324+0000 7fa909d9b640 1 --2- 172.21.15.76:0/1470690993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa90c071fa0 0x7fa90c0b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:51.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.324+0000 7fd47d7c0640 1 -- 172.21.15.76:0/239281669 shutdown_connections 2023-11-09T03:45:51.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.324+0000 7f6295b05640 1 -- 172.21.15.76:0/3244957031 wait complete. 2023-11-09T03:45:51.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.324+0000 7fd47d7c0640 1 -- 172.21.15.76:0/239281669 wait complete. 2023-11-09T03:45:51.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.324+0000 7f82de051640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82c80047c0 0x7f82d80b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.324+0000 7f32d18f1640 1 --2- 172.21.15.76:0/3952560771 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32cc15af20 0x7f32cc0b3cd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009870 tx=0x7f32bc02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6518119f89fb668c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.324+0000 7fa90a59c640 1 --2- 172.21.15.76:0/1470690993 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa90c06a660 0x7fa90c1eb2f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00c7f0 tx=0x7fa8fc00cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.324+0000 7f82de051640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82d80b1150 0x7f82d80c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.324+0000 7f82de051640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82d8159d20 0x7f82d81548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.324+0000 7f82de051640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d80c60c0 con 0x7f82c80047c0 2023-11-09T03:45:51.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.324+0000 7f82de051640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d80c7a70 con 0x7f82d8159d20 2023-11-09T03:45:51.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.324+0000 7f82de051640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d80b15b0 con 0x7f82d80b1150 2023-11-09T03:45:51.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.325+0000 7f82d6ffd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82d80b1150 0x7f82d80c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.325+0000 7f82d77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82c80047c0 0x7f82d80b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.325+0000 7f82d77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82c80047c0 0x7f82d80b2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57854/0 (socket says 172.21.15.76:57854) 2023-11-09T03:45:51.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.325+0000 7f82d6ffd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82d80b1150 0x7f82d80c5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:51076/0 (socket says 172.21.15.76:51076) 2023-11-09T03:45:51.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.325+0000 7f82d77fe640 1 -- 172.21.15.76:0/727115994 learned_addr learned my addr 172.21.15.76:0/727115994 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:51.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.324+0000 7f32d08ef640 1 -- 172.21.15.76:0/3952560771 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc030840 con 0x7f32cc15af20 2023-11-09T03:45:51.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.325+0000 7f832b0f3640 1 Processor -- start 2023-11-09T03:45:51.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.325+0000 7f32d08ef640 1 -- 172.21.15.76:0/3952560771 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32bc0309a0 con 0x7f32cc15af20 2023-11-09T03:45:51.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.325+0000 7f32d08ef640 1 -- 172.21.15.76:0/3952560771 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc030d30 con 0x7f32cc15af20 2023-11-09T03:45:51.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.325+0000 7fc7ad400640 1 -- 172.21.15.76:0/1962828474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7900047c0 msgr2=0x7fc790004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.325+0000 7fc7ad400640 1 --2- 172.21.15.76:0/1962828474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7900047c0 0x7fc790004c40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc79c007ef0 tx=0x7fc7900057f0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.325+0000 7f82d6ffd640 1 -- 172.21.15.76:0/727115994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82d8159d20 msgr2=0x7f82d81548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.325+0000 7f82d6ffd640 1 --2- 172.21.15.76:0/727115994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82d8159d20 0x7f82d81548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.325+0000 7f82d6ffd640 1 -- 172.21.15.76:0/727115994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82c80047c0 msgr2=0x7f82d80b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.325+0000 7f82d6ffd640 1 --2- 172.21.15.76:0/727115994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82c80047c0 0x7f82d80b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.325+0000 7f82d6ffd640 1 -- 172.21.15.76:0/727115994 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d80c4dc0 con 0x7f82d80b1150 2023-11-09T03:45:51.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.325+0000 7fa8faffd640 1 -- 172.21.15.76:0/1470690993 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc007a30 con 0x7fa90c06a660 2023-11-09T03:45:51.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.325+0000 7fa8faffd640 1 -- 172.21.15.76:0/1470690993 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8fc007b90 con 0x7fa90c06a660 2023-11-09T03:45:51.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.325+0000 7fa8faffd640 1 -- 172.21.15.76:0/1470690993 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc00ce40 con 0x7fa90c06a660 2023-11-09T03:45:51.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.325+0000 7f82d77fe640 1 --2- 172.21.15.76:0/727115994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82c80047c0 0x7f82d80b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:45:51.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.325+0000 7f832b0f3640 1 -- start start 2023-11-09T03:45:51.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.325+0000 7fc7ad400640 1 -- 172.21.15.76:0/1962828474 shutdown_connections 2023-11-09T03:45:51.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.325+0000 7fc7ad400640 1 --2- 172.21.15.76:0/1962828474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7900047c0 0x7fc790004c40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.325+0000 7fc7ad400640 1 -- 172.21.15.76:0/1962828474 >> 172.21.15.76:0/1962828474 conn(0x7fc7a0055390 msgr2=0x7fc7a00557a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:51.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.325+0000 7f82d6ffd640 1 --2- 172.21.15.76:0/727115994 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82d80b1150 0x7f82d80c5b80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f82c800c2e0 tx=0x7f82c40036f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.325+0000 7fa9109d0640 1 -- 172.21.15.76:0/1470690993 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa90c0b34e0 con 0x7fa90c06a660 2023-11-09T03:45:51.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.325+0000 7fc7ad400640 1 -- 172.21.15.76:0/1962828474 shutdown_connections 2023-11-09T03:45:51.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.325+0000 7fa9109d0640 1 -- 172.21.15.76:0/1470690993 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa90c0b2930 con 0x7fa90c06a660 2023-11-09T03:45:51.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.326+0000 7f6295b05640 1 Processor -- start 2023-11-09T03:45:51.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.326+0000 7f6295b05640 1 -- start start 2023-11-09T03:45:51.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.325+0000 7f82c3fff640 1 -- 172.21.15.76:0/727115994 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c403c430 con 0x7f82d80b1150 2023-11-09T03:45:51.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.326+0000 7f82c3fff640 1 -- 172.21.15.76:0/727115994 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c402be90 con 0x7f82d80b1150 2023-11-09T03:45:51.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.326+0000 7f82c3fff640 1 -- 172.21.15.76:0/727115994 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c403c790 con 0x7f82d80b1150 2023-11-09T03:45:51.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.326+0000 7f82de051640 1 -- 172.21.15.76:0/727115994 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d80c50a0 con 0x7f82d80b1150 2023-11-09T03:45:51.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.327+0000 7f82de051640 1 -- 172.21.15.76:0/727115994 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d80c5660 con 0x7f82d80b1150 2023-11-09T03:45:51.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.326+0000 7fc7ad400640 1 -- 172.21.15.76:0/1962828474 wait complete. 2023-11-09T03:45:51.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.326+0000 7f32d3b55640 1 -- 172.21.15.76:0/3952560771 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32cc15af20 msgr2=0x7f32cc0b3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.326+0000 7f32d3b55640 1 --2- 172.21.15.76:0/3952560771 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32cc15af20 0x7f32cc0b3cd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009870 tx=0x7f32bc02b980 comp rx=0 tx=0).stop 2023-11-09T03:45:51.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.326+0000 7fd47d7c0640 1 Processor -- start 2023-11-09T03:45:51.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.327+0000 7fd47d7c0640 1 -- start start 2023-11-09T03:45:51.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.327+0000 7f832b0f3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83100047c0 0x7f8324152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.327+0000 7f832b0f3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83241572f0 0x7f832415bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.327+0000 7f32d3b55640 1 -- 172.21.15.76:0/3952560771 shutdown_connections 2023-11-09T03:45:51.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.327+0000 7f832a0f1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83100047c0 0x7f8324152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.327+0000 7f32d3b55640 1 --2- 172.21.15.76:0/3952560771 >> v2:172.21.15.76:3300/0 conn(0x7f32cc0b64c0 0x7f32cc0cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.327+0000 7f32d3b55640 1 --2- 172.21.15.76:0/3952560771 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32cc15af20 0x7f32cc0b3cd0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.327+0000 7f32d3b55640 1 -- 172.21.15.76:0/3952560771 >> 172.21.15.76:0/3952560771 conn(0x7f32cc06ea20 msgr2=0x7f32cc06ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:51.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.327+0000 7f32a6e4d640 1 Processor -- start 2023-11-09T03:45:51.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.327+0000 7f32a6e4d640 1 -- start start 2023-11-09T03:45:51.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.327+0000 7f83298f0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83241572f0 0x7f832415bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7f83298f0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83241572f0 0x7f832415bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:34106/0 (socket says 172.21.15.76:34106) 2023-11-09T03:45:51.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7f832a0f1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83100047c0 0x7f8324152a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:51090/0 (socket says 172.21.15.76:51090) 2023-11-09T03:45:51.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7f83298f0640 1 -- 172.21.15.76:0/3107859883 learned_addr learned my addr 172.21.15.76:0/3107859883 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:51.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7f832b0f3640 1 --2- 172.21.15.76:0/3107859883 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8324157770 0x7f832415c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7f6295b05640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62900b1860 0x7f629014f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7f6295b05640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62900cb990 0x7f62901fa110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7f6295b05640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62901612d0 0x7f629015dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7fd47d7c0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd47003a860 0x7fd470033250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7f6295b05640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62901f7b70 con 0x7f62901612d0 2023-11-09T03:45:51.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7f6295b05640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6290164690 con 0x7f62900cb990 2023-11-09T03:45:51.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7f6295b05640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62901fb040 con 0x7f62900b1860 2023-11-09T03:45:51.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7f832a8f2640 1 --2- 172.21.15.76:0/3107859883 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8324157770 0x7f832415c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7f832b0f3640 1 -- 172.21.15.76:0/3107859883 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83240c44c0 con 0x7f8324157770 2023-11-09T03:45:51.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7f628f7fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62900b1860 0x7f629014f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7f628effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62900cb990 0x7f62901fa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7f832b0f3640 1 -- 172.21.15.76:0/3107859883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83240c4760 con 0x7f83241572f0 2023-11-09T03:45:51.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7f832b0f3640 1 -- 172.21.15.76:0/3107859883 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83240c9a20 con 0x7f83100047c0 2023-11-09T03:45:51.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7f82de051640 1 -- 172.21.15.76:0/727115994 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d80c5880 con 0x7f82d80b1150 2023-11-09T03:45:51.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7fd47d7c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd470050120 0x7fd4701783c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7f628ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62901612d0 0x7f629015dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7f628effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62900cb990 0x7f62901fa110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:34112/0 (socket says 172.21.15.76:34112) 2023-11-09T03:45:51.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7f628effd640 1 -- 172.21.15.76:0/728681820 learned_addr learned my addr 172.21.15.76:0/728681820 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:51.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7f32a6e4d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f32a00b1aa0 0x7f32a0158d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7f32a6e4d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32a006e600 0x7f32a00c9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7f32a6e4d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32a006cb20 con 0x7f32a00c9560 2023-11-09T03:45:51.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7f32a6e4d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32a006e2f0 con 0x7f32a00b1aa0 2023-11-09T03:45:51.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7f32a6e4d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00c99e0 con 0x7f32a006e600 2023-11-09T03:45:51.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7fc7ad400640 1 Processor -- start 2023-11-09T03:45:51.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7f32d3b55640 1 -- 172.21.15.76:0/3952560771 shutdown_connections 2023-11-09T03:45:51.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7fa8faffd640 1 -- 172.21.15.76:0/1470690993 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa8fc0113f0 con 0x7fa90c06a660 2023-11-09T03:45:51.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7fd476ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd47003a860 0x7fd470033250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7fd476ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd47003a860 0x7fd470033250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:34114/0 (socket says 172.21.15.76:34114) 2023-11-09T03:45:51.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7fc7ad400640 1 -- start start 2023-11-09T03:45:51.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7fd476ffd640 1 -- 172.21.15.76:0/2290244843 learned_addr learned my addr 172.21.15.76:0/2290244843 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:51.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7fd47d7c0640 1 --2- 172.21.15.76:0/2290244843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4700e3350 0x7fd470031240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7f628effd640 1 -- 172.21.15.76:0/728681820 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62900b1860 msgr2=0x7f629014f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.328+0000 7fa9109d0640 1 -- 172.21.15.76:0/1470690993 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f0005250 con 0x7fa90c06a660 2023-11-09T03:45:51.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7f628effd640 1 --2- 172.21.15.76:0/728681820 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62900b1860 0x7f629014f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7f628effd640 1 -- 172.21.15.76:0/728681820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62901612d0 msgr2=0x7f629015dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7fd47d7c0640 1 -- 172.21.15.76:0/2290244843 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd470030ba0 con 0x7fd470050120 2023-11-09T03:45:51.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7f628effd640 1 --2- 172.21.15.76:0/728681820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62901612d0 0x7f629015dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7fd47d7c0640 1 -- 172.21.15.76:0/2290244843 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4700dcc60 con 0x7fd47003a860 2023-11-09T03:45:51.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7fd47d7c0640 1 -- 172.21.15.76:0/2290244843 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4700308c0 con 0x7fd4700e3350 2023-11-09T03:45:51.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7fd4777fe640 1 --2- 172.21.15.76:0/2290244843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4700e3350 0x7fd470031240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7f628effd640 1 -- 172.21.15.76:0/728681820 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6288009a70 con 0x7f62900cb990 2023-11-09T03:45:51.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7fa8faffd640 1 --2- 172.21.15.76:0/1470690993 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa8ec06af40 0x7fa8ec06d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7f32d3b55640 1 -- 172.21.15.76:0/3952560771 wait complete. 2023-11-09T03:45:51.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7fa8faffd640 1 -- 172.21.15.76:0/1470690993 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa8fc0899c0 con 0x7fa90c06a660 2023-11-09T03:45:51.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7f83298f0640 1 -- 172.21.15.76:0/3107859883 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83100047c0 msgr2=0x7f8324152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7f83298f0640 1 --2- 172.21.15.76:0/3107859883 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83100047c0 0x7f8324152a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7f83298f0640 1 -- 172.21.15.76:0/3107859883 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8324157770 msgr2=0x7f832415c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7f83298f0640 1 --2- 172.21.15.76:0/3107859883 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8324157770 0x7f832415c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7f83298f0640 1 -- 172.21.15.76:0/3107859883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83240c9d00 con 0x7f83241572f0 2023-11-09T03:45:51.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7f628effd640 1 --2- 172.21.15.76:0/728681820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62900cb990 0x7f62901fa110 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f628000e9e0 tx=0x7f628000eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7fd4767fc640 1 --2- 172.21.15.76:0/2290244843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd470050120 0x7fd4701783c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7f32a664c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f32a00c9560 0x7f32a00cb060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:46142/0 (socket says 172.21.15.76:46142) 2023-11-09T03:45:51.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7f32a664c640 1 -- 172.21.15.76:0/1416124475 learned_addr learned my addr 172.21.15.76:0/1416124475 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:51.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7f32a564a640 1 --2- 172.21.15.76:0/1416124475 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32a006e600 0x7f32a00c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7fd476ffd640 1 -- 172.21.15.76:0/2290244843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4700e3350 msgr2=0x7fd470031240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7fd476ffd640 1 --2- 172.21.15.76:0/2290244843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4700e3350 0x7fd470031240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7fd476ffd640 1 -- 172.21.15.76:0/2290244843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd470050120 msgr2=0x7fd4701783c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7fa909d9b640 1 --2- 172.21.15.76:0/1470690993 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa8ec06af40 0x7fa8ec06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7f32a5e4b640 1 --2- 172.21.15.76:0/1416124475 >> v2:172.21.15.76:3300/0 conn(0x7f32a00b1aa0 0x7f32a0158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7fd476ffd640 1 --2- 172.21.15.76:0/2290244843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd470050120 0x7fd4701783c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7fd476ffd640 1 -- 172.21.15.76:0/2290244843 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47804f380 con 0x7fd47003a860 2023-11-09T03:45:51.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.329+0000 7f6277fff640 1 -- 172.21.15.76:0/728681820 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6280009670 con 0x7f62900cb990 2023-11-09T03:45:51.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.330+0000 7f6277fff640 1 -- 172.21.15.76:0/728681820 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628000c9d0 con 0x7f62900cb990 2023-11-09T03:45:51.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.330+0000 7f6295b05640 1 -- 172.21.15.76:0/728681820 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f629015e7f0 con 0x7f62900cb990 2023-11-09T03:45:51.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.330+0000 7f6277fff640 1 -- 172.21.15.76:0/728681820 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628000cd80 con 0x7f62900cb990 2023-11-09T03:45:51.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.330+0000 7f6295b05640 1 -- 172.21.15.76:0/728681820 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62901faa80 con 0x7f62900cb990 2023-11-09T03:45:51.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.330+0000 7f32a564a640 1 -- 172.21.15.76:0/1416124475 >> v2:172.21.15.76:3300/0 conn(0x7f32a00b1aa0 msgr2=0x7f32a0158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:51.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.330+0000 7f32a564a640 1 --2- 172.21.15.76:0/1416124475 >> v2:172.21.15.76:3300/0 conn(0x7f32a00b1aa0 0x7f32a0158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.330+0000 7f32a564a640 1 -- 172.21.15.76:0/1416124475 >> v1:172.21.15.113:6789/0 conn(0x7f32a00c9560 legacy=0x7f32a00cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.330+0000 7fc7ad400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7900047c0 0x7fc7a00e2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.330+0000 7f32a564a640 1 -- 172.21.15.76:0/1416124475 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a00c9cc0 con 0x7f32a006e600 2023-11-09T03:45:51.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.330+0000 7fc7ad400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7a00dd690 0x7fc7a0174fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.330+0000 7fc7ad400640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7a0036e60 0x7fc7a0037310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.330+0000 7f82c3fff640 1 -- 172.21.15.76:0/727115994 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f82c403c8f0 con 0x7f82d80b1150 2023-11-09T03:45:51.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.330+0000 7fc7a67fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7a00dd690 0x7fc7a0174fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.330+0000 7fc7a67fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7a00dd690 0x7fc7a0174fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:34116/0 (socket says 172.21.15.76:34116) 2023-11-09T03:45:51.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.330+0000 7fc7a6ffd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7900047c0 0x7fc7a00e2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.330+0000 7f83298f0640 1 --2- 172.21.15.76:0/3107859883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83241572f0 0x7f832415bc60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f832002c2a0 tx=0x7f832002e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.330+0000 7f32d3b55640 1 Processor -- start 2023-11-09T03:45:51.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.330+0000 7fc7a6ffd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7900047c0 0x7fc7a00e2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:51124/0 (socket says 172.21.15.76:51124) 2023-11-09T03:45:51.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.330+0000 7fc7a77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7a0036e60 0x7fc7a0037310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.330+0000 7f32d3b55640 1 -- start start 2023-11-09T03:45:51.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.330+0000 7fd4767fc640 1 --2- 172.21.15.76:0/2290244843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd470050120 0x7fd4701783c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:51.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.331+0000 7fc7a77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7a0036e60 0x7fc7a0037310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57912/0 (socket says 172.21.15.76:57912) 2023-11-09T03:45:51.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.331+0000 7f82c3fff640 1 --2- 172.21.15.76:0/727115994 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f82bc06ae70 0x7f82bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.331+0000 7fd476ffd640 1 --2- 172.21.15.76:0/2290244843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd47003a860 0x7fd470033250 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd46800c410 tx=0x7fd46800c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.331+0000 7f831affd640 1 -- 172.21.15.76:0/3107859883 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832003c040 con 0x7f83241572f0 2023-11-09T03:45:51.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.331+0000 7fc7a67fc640 1 -- 172.21.15.76:0/1884531060 learned_addr learned my addr 172.21.15.76:0/1884531060 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:51.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.331+0000 7f831affd640 1 -- 172.21.15.76:0/3107859883 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8320007540 con 0x7f83241572f0 2023-11-09T03:45:51.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.331+0000 7fc7ad400640 1 -- 172.21.15.76:0/1884531060 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a0037850 con 0x7fc7a0036e60 2023-11-09T03:45:51.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.331+0000 7f82c3fff640 1 -- 172.21.15.76:0/727115994 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f82c40ab530 con 0x7f82d80b1150 2023-11-09T03:45:51.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.331+0000 7f82d77fe640 1 --2- 172.21.15.76:0/727115994 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f82bc06ae70 0x7f82bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.331+0000 7fc7ad400640 1 -- 172.21.15.76:0/1884531060 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a00dc350 con 0x7fc7a00dd690 2023-11-09T03:45:51.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.331+0000 7fc7ad400640 1 -- 172.21.15.76:0/1884531060 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a00dc5f0 con 0x7fc7900047c0 2023-11-09T03:45:51.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.331+0000 7f831affd640 1 -- 172.21.15.76:0/3107859883 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832002c680 con 0x7f83241572f0 2023-11-09T03:45:51.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.331+0000 7f832b0f3640 1 -- 172.21.15.76:0/3107859883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83240c9fe0 con 0x7f83241572f0 2023-11-09T03:45:51.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.331+0000 7f832b0f3640 1 -- 172.21.15.76:0/3107859883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83240c9660 con 0x7f83241572f0 2023-11-09T03:45:51.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.331+0000 7f6295b05640 1 -- 172.21.15.76:0/728681820 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f629015e980 con 0x7f62900cb990 2023-11-09T03:45:51.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.332+0000 7f82d77fe640 1 --2- 172.21.15.76:0/727115994 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f82bc06ae70 0x7f82bc06d330 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009e10 tx=0x7f82cc006ee0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.331+0000 7f32a564a640 1 --2- 172.21.15.76:0/1416124475 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32a006e600 0x7f32a00c9020 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f329c0097a0 tx=0x7f329c02b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31c12764b8fec888 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.331+0000 7fa909d9b640 1 --2- 172.21.15.76:0/1470690993 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa8ec06af40 0x7fa8ec06d400 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0ce8a0 tx=0x7fa900007ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.331+0000 7fd45f7fe640 1 -- 172.21.15.76:0/2290244843 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd468005ce0 con 0x7fd47003a860 2023-11-09T03:45:51.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.331+0000 7fd45f7fe640 1 -- 172.21.15.76:0/2290244843 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd468005e40 con 0x7fd47003a860 2023-11-09T03:45:51.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.331+0000 7fd45f7fe640 1 -- 172.21.15.76:0/2290244843 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4680072f0 con 0x7fd47003a860 2023-11-09T03:45:51.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.331+0000 7fd47d7c0640 1 -- 172.21.15.76:0/2290244843 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4700dd280 con 0x7fd47003a860 2023-11-09T03:45:51.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.332+0000 7fd47d7c0640 1 -- 172.21.15.76:0/2290244843 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4700dd840 con 0x7fd47003a860 2023-11-09T03:45:51.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.331+0000 7f32d3b55640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32cc0b64c0 0x7f32cc0c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.332+0000 7f32d3b55640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32cc0cd3b0 0x7f32cc0c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.332+0000 7f32d3b55640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32cc15af20 0x7f32cc15cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.332+0000 7f32d3b55640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc0c45f0 con 0x7f32cc0cd3b0 2023-11-09T03:45:51.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.332+0000 7f32d3b55640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc0c91f0 con 0x7f32cc0b64c0 2023-11-09T03:45:51.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.332+0000 7f32d3b55640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc0c94d0 con 0x7f32cc15af20 2023-11-09T03:45:51.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.332+0000 7f32d18f1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32cc0b64c0 0x7f32cc0c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.332+0000 7f32d20f2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32cc15af20 0x7f32cc15cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.332+0000 7f32d18f1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32cc0b64c0 0x7f32cc0c8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:34122/0 (socket says 172.21.15.76:34122) 2023-11-09T03:45:51.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.332+0000 7f32d18f1640 1 -- 172.21.15.76:0/3816548299 learned_addr learned my addr 172.21.15.76:0/3816548299 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:51.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.332+0000 7f32d10f0640 1 --2- 172.21.15.76:0/3816548299 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32cc0cd3b0 0x7f32cc0c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.332+0000 7fc7a6ffd640 1 -- 172.21.15.76:0/1884531060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7a00dd690 msgr2=0x7fc7a0174fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.332+0000 7fc7a6ffd640 1 --2- 172.21.15.76:0/1884531060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7a00dd690 0x7fc7a0174fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.332+0000 7fc7a6ffd640 1 -- 172.21.15.76:0/1884531060 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7a0036e60 msgr2=0x7fc7a0037310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.332+0000 7fc7a6ffd640 1 --2- 172.21.15.76:0/1884531060 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7a0036e60 0x7fc7a0037310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.332+0000 7fc7a6ffd640 1 -- 172.21.15.76:0/1884531060 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a00dc8d0 con 0x7fc7900047c0 2023-11-09T03:45:51.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.332+0000 7fc7a77fe640 1 --2- 172.21.15.76:0/1884531060 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7a0036e60 0x7fc7a0037310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:45:51.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.332+0000 7fc7a6ffd640 1 --2- 172.21.15.76:0/1884531060 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7900047c0 0x7fc7a00e2340 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0003f30 tx=0x7fc79800f7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.332+0000 7f6277fff640 1 -- 172.21.15.76:0/728681820 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6280005990 con 0x7f62900cb990 2023-11-09T03:45:51.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.332+0000 7f32d10f0640 1 -- 172.21.15.76:0/3816548299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32cc15af20 msgr2=0x7f32cc15cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.332+0000 7f32d10f0640 1 --2- 172.21.15.76:0/3816548299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32cc15af20 0x7f32cc15cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.332+0000 7f32d10f0640 1 -- 172.21.15.76:0/3816548299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32cc0b64c0 msgr2=0x7f32cc0c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.332+0000 7f32d10f0640 1 --2- 172.21.15.76:0/3816548299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32cc0b64c0 0x7f32cc0c8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.332+0000 7f32d10f0640 1 -- 172.21.15.76:0/3816548299 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32bc0094d0 con 0x7f32cc0cd3b0 2023-11-09T03:45:51.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.333+0000 7f6277fff640 1 --2- 172.21.15.76:0/728681820 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f627c06ae70 0x7f627c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.333+0000 7f32d10f0640 1 --2- 172.21.15.76:0/3816548299 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32cc0cd3b0 0x7f32cc0c8b90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f32c8006fd0 tx=0x7f32c800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.333+0000 7f6277fff640 1 -- 172.21.15.76:0/728681820 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6280089ab0 con 0x7f62900cb990 2023-11-09T03:45:51.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.333+0000 7f628f7fe640 1 --2- 172.21.15.76:0/728681820 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f627c06ae70 0x7f627c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.333+0000 7f832b0f3640 1 -- 172.21.15.76:0/3107859883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83240c4910 con 0x7f83241572f0 2023-11-09T03:45:51.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.333+0000 7f32c27fc640 1 -- 172.21.15.76:0/3816548299 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c8004030 con 0x7f32cc0cd3b0 2023-11-09T03:45:51.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.333+0000 7f32c27fc640 1 -- 172.21.15.76:0/3816548299 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c8004190 con 0x7f32cc0cd3b0 2023-11-09T03:45:51.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.333+0000 7f32c27fc640 1 -- 172.21.15.76:0/3816548299 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c8004500 con 0x7f32cc0cd3b0 2023-11-09T03:45:51.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.333+0000 7f32d3b55640 1 -- 172.21.15.76:0/3816548299 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32cc0c4840 con 0x7f32cc0cd3b0 2023-11-09T03:45:51.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.333+0000 7fd47d7c0640 1 -- 172.21.15.76:0/2290244843 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4700dd0e0 con 0x7fd47003a860 2023-11-09T03:45:51.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.333+0000 7f32d3b55640 1 -- 172.21.15.76:0/3816548299 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32cc15b5a0 con 0x7f32cc0cd3b0 2023-11-09T03:45:51.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.334+0000 7f628f7fe640 1 --2- 172.21.15.76:0/728681820 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f627c06ae70 0x7f627c06d330 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6284005880 tx=0x7f6284009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.334+0000 7f32d3b55640 1 -- 172.21.15.76:0/3816548299 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32cc0674e0 con 0x7f32cc0cd3b0 2023-11-09T03:45:51.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.334+0000 7f32c27fc640 1 -- 172.21.15.76:0/3816548299 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f32c800c6d0 con 0x7f32cc0cd3b0 2023-11-09T03:45:51.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.335+0000 7f32c27fc640 1 --2- 172.21.15.76:0/3816548299 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f32b006aec0 0x7f32b006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.335+0000 7f32c27fc640 1 -- 172.21.15.76:0/3816548299 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f32c8089290 con 0x7f32cc0cd3b0 2023-11-09T03:45:51.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.335+0000 7f32d18f1640 1 --2- 172.21.15.76:0/3816548299 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f32b006aec0 0x7f32b006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.335+0000 7fa8faffd640 1 -- 172.21.15.76:0/1470690993 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa8fc054810 con 0x7fa90c06a660 2023-11-09T03:45:51.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.337+0000 7f6277fff640 1 -- 172.21.15.76:0/728681820 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6280054a90 con 0x7f62900cb990 2023-11-09T03:45:51.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.337+0000 7f32d18f1640 1 --2- 172.21.15.76:0/3816548299 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f32b006aec0 0x7f32b006d380 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009280 tx=0x7f32bc03b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.338+0000 7f831affd640 1 -- 172.21.15.76:0/3107859883 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8320032070 con 0x7f83241572f0 2023-11-09T03:45:51.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.338+0000 7fd45f7fe640 1 -- 172.21.15.76:0/2290244843 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd468007510 con 0x7fd47003a860 2023-11-09T03:45:51.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.339+0000 7f831affd640 1 --2- 172.21.15.76:0/3107859883 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f830806aef0 0x7f830806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.339+0000 7fd45f7fe640 1 --2- 172.21.15.76:0/2290244843 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd46406af40 0x7fd46406d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.339+0000 7f831affd640 1 -- 172.21.15.76:0/3107859883 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f83200aade0 con 0x7f83241572f0 2023-11-09T03:45:51.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.339+0000 7fd45f7fe640 1 -- 172.21.15.76:0/2290244843 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd468089200 con 0x7fd47003a860 2023-11-09T03:45:51.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.339+0000 7f832a0f1640 1 --2- 172.21.15.76:0/3107859883 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f830806aef0 0x7f830806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.339+0000 7fd4767fc640 1 --2- 172.21.15.76:0/2290244843 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd46406af40 0x7fd46406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.340+0000 7fc78f7fe640 1 -- 172.21.15.76:0/1884531060 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc798016c80 con 0x7fc7900047c0 2023-11-09T03:45:51.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.340+0000 7f32a4e49640 1 -- 172.21.15.76:0/1416124475 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c031780 con 0x7f32a006e600 2023-11-09T03:45:51.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.340+0000 7f32a4e49640 1 -- 172.21.15.76:0/1416124475 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329c0318e0 con 0x7f32a006e600 2023-11-09T03:45:51.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.340+0000 7f32a4e49640 1 -- 172.21.15.76:0/1416124475 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c031c70 con 0x7f32a006e600 2023-11-09T03:45:51.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.340+0000 7f832a0f1640 1 --2- 172.21.15.76:0/3107859883 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f830806aef0 0x7f830806d3b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f83240c95f0 tx=0x7f8314009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.340+0000 7f82c3fff640 1 -- 172.21.15.76:0/727115994 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f82c4076310 con 0x7f82d80b1150 2023-11-09T03:45:51.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.340+0000 7fc78f7fe640 1 -- 172.21.15.76:0/1884531060 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc798016de0 con 0x7fc7900047c0 2023-11-09T03:45:51.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.340+0000 7fc78f7fe640 1 -- 172.21.15.76:0/1884531060 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7980205d0 con 0x7fc7900047c0 2023-11-09T03:45:51.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.340+0000 7fc7ad400640 1 -- 172.21.15.76:0/1884531060 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a00dcbb0 con 0x7fc7900047c0 2023-11-09T03:45:51.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.340+0000 7fc7ad400640 1 -- 172.21.15.76:0/1884531060 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a00dd170 con 0x7fc7900047c0 2023-11-09T03:45:51.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.340+0000 7fd4767fc640 1 --2- 172.21.15.76:0/2290244843 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd46406af40 0x7fd46406d400 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd4700cfa20 tx=0x7fd46c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.340+0000 7f32a6e4d640 1 -- 172.21.15.76:0/1416124475 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32a006e600 msgr2=0x7f32a00c9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.340+0000 7f32a6e4d640 1 --2- 172.21.15.76:0/1416124475 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32a006e600 0x7f32a00c9020 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f329c0097a0 tx=0x7f329c02b820 comp rx=0 tx=0).stop 2023-11-09T03:45:51.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.341+0000 7f32a6e4d640 1 -- 172.21.15.76:0/1416124475 shutdown_connections 2023-11-09T03:45:51.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.341+0000 7f32a6e4d640 1 --2- 172.21.15.76:0/1416124475 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32a006e600 0x7f32a00c9020 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.341+0000 7f32a6e4d640 1 --2- 172.21.15.76:0/1416124475 >> v2:172.21.15.76:3300/0 conn(0x7f32a00b1aa0 0x7f32a0158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.341+0000 7f32a6e4d640 1 -- 172.21.15.76:0/1416124475 >> 172.21.15.76:0/1416124475 conn(0x7f32a00b02c0 msgr2=0x7f32a00b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:51.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.341+0000 7f32a6e4d640 1 -- 172.21.15.76:0/1416124475 shutdown_connections 2023-11-09T03:45:51.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.341+0000 7f32a6e4d640 1 -- 172.21.15.76:0/1416124475 wait complete. 2023-11-09T03:45:51.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.341+0000 7fc7ad400640 1 -- 172.21.15.76:0/1884531060 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7a0005790 con 0x7fc7900047c0 2023-11-09T03:45:51.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.342+0000 7fc78f7fe640 1 -- 172.21.15.76:0/1884531060 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc798020730 con 0x7fc7900047c0 2023-11-09T03:45:51.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.342+0000 7f831affd640 1 -- 172.21.15.76:0/3107859883 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8320075c30 con 0x7f83241572f0 2023-11-09T03:45:51.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.343+0000 7f32a6e4d640 1 Processor -- start 2023-11-09T03:45:51.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.343+0000 7f32a6e4d640 1 -- start start 2023-11-09T03:45:51.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.343+0000 7fc78f7fe640 1 --2- 172.21.15.76:0/1884531060 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc79406af40 0x7fc79406d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.343+0000 7fc7a67fc640 1 --2- 172.21.15.76:0/1884531060 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc79406af40 0x7fc79406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.343+0000 7f32c27fc640 1 -- 172.21.15.76:0/3816548299 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f32c80541e0 con 0x7f32cc0cd3b0 2023-11-09T03:45:51.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.343+0000 7fc78f7fe640 1 -- 172.21.15.76:0/1884531060 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc79808d5c0 con 0x7fc7900047c0 2023-11-09T03:45:51.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.344+0000 7f32a6e4d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32a006e600 0x7f32a00c1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.344+0000 7f32a6e4d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32a00b1aa0 0x7f32a014fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.344+0000 7fc7a67fc640 1 --2- 172.21.15.76:0/1884531060 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc79406af40 0x7fc79406d400 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc79c007ef0 tx=0x7fc79c030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.344+0000 7f32a6e4d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32a00c9560 0x7f32a0150160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.344+0000 7f32a6e4d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00c2300 con 0x7f32a00c9560 2023-11-09T03:45:51.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.344+0000 7f32a6e4d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00c25e0 con 0x7f32a006e600 2023-11-09T03:45:51.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.344+0000 7f32a6e4d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00c28c0 con 0x7f32a00b1aa0 2023-11-09T03:45:51.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.344+0000 7f32a664c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32a00c9560 0x7f32a0150160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.344+0000 7f32a664c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32a00c9560 0x7f32a0150160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57934/0 (socket says 172.21.15.76:57934) 2023-11-09T03:45:51.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.344+0000 7f32a664c640 1 -- 172.21.15.76:0/4008423064 learned_addr learned my addr 172.21.15.76:0/4008423064 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:51.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.344+0000 7f32a5e4b640 1 --2- 172.21.15.76:0/4008423064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32a006e600 0x7f32a00c1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.344+0000 7f32a664c640 1 -- 172.21.15.76:0/4008423064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32a00b1aa0 msgr2=0x7f32a014fc20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-09T03:45:51.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.344+0000 7f32a664c640 1 --2- 172.21.15.76:0/4008423064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32a00b1aa0 0x7f32a014fc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.344+0000 7f32a664c640 1 -- 172.21.15.76:0/4008423064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32a006e600 msgr2=0x7f32a00c1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.344+0000 7f32a664c640 1 --2- 172.21.15.76:0/4008423064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32a006e600 0x7f32a00c1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.344+0000 7f32a664c640 1 -- 172.21.15.76:0/4008423064 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f329c009400 con 0x7f32a00c9560 2023-11-09T03:45:51.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.345+0000 7f32a664c640 1 --2- 172.21.15.76:0/4008423064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32a00c9560 0x7f32a0150160 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f329000c3c0 tx=0x7f329000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.345+0000 7f32967fc640 1 -- 172.21.15.76:0/4008423064 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3290005ce0 con 0x7f32a00c9560 2023-11-09T03:45:51.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.345+0000 7f32a6e4d640 1 -- 172.21.15.76:0/4008423064 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a00c2ba0 con 0x7f32a00c9560 2023-11-09T03:45:51.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.346+0000 7f32a6e4d640 1 -- 172.21.15.76:0/4008423064 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a00c16c0 con 0x7f32a00c9560 2023-11-09T03:45:51.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.346+0000 7f32967fc640 1 -- 172.21.15.76:0/4008423064 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3290005e40 con 0x7f32a00c9560 2023-11-09T03:45:51.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.347+0000 7f32967fc640 1 -- 172.21.15.76:0/4008423064 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32900049e0 con 0x7f32a00c9560 2023-11-09T03:45:51.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.347+0000 7f32a6e4d640 1 -- 172.21.15.76:0/4008423064 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3288005250 con 0x7f32a00c9560 2023-11-09T03:45:51.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.347+0000 7fc78f7fe640 1 -- 172.21.15.76:0/1884531060 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc7980584a0 con 0x7fc7900047c0 2023-11-09T03:45:51.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.347+0000 7fd45f7fe640 1 -- 172.21.15.76:0/2290244843 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd4680541a0 con 0x7fd47003a860 2023-11-09T03:45:51.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.357+0000 7f32967fc640 1 -- 172.21.15.76:0/4008423064 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3290004c00 con 0x7f32a00c9560 2023-11-09T03:45:51.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.358+0000 7f32967fc640 1 --2- 172.21.15.76:0/4008423064 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f328406ae70 0x7f328406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:51.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.358+0000 7f32967fc640 1 -- 172.21.15.76:0/4008423064 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f32900891f0 con 0x7f32a00c9560 2023-11-09T03:45:51.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.359+0000 7f32967fc640 1 -- 172.21.15.76:0/4008423064 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f329008b780 con 0x7f32a00c9560 2023-11-09T03:45:51.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.360+0000 7f32a5e4b640 1 --2- 172.21.15.76:0/4008423064 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f328406ae70 0x7f328406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:51.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.376+0000 7f32a5e4b640 1 --2- 172.21.15.76:0/4008423064 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f328406ae70 0x7f328406d330 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f32980113e0 tx=0x7f329800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:51.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.737+0000 7fa9109d0640 1 -- 172.21.15.76:0/1470690993 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa8f0005500 con 0x7fa90c06a660 2023-11-09T03:45:51.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.738+0000 7fa8faffd640 1 -- 172.21.15.76:0/1470690993 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa8fc01f020 con 0x7fa90c06a660 2023-11-09T03:45:51.737 INFO:teuthology.orchestra.run.smithi076.stdout:55834574857 2023-11-09T03:45:51.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.741+0000 7fa8f8ff9640 1 -- 172.21.15.76:0/1470690993 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa8ec06af40 msgr2=0x7fa8ec06d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.741+0000 7fa8f8ff9640 1 --2- 172.21.15.76:0/1470690993 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa8ec06af40 0x7fa8ec06d400 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0ce8a0 tx=0x7fa900007ce0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.741+0000 7fa8f8ff9640 1 -- 172.21.15.76:0/1470690993 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa90c06a660 msgr2=0x7fa90c1eb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.741+0000 7fa8f8ff9640 1 --2- 172.21.15.76:0/1470690993 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa90c06a660 0x7fa90c1eb2f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00c7f0 tx=0x7fa8fc00cbf0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.741+0000 7fa8f8ff9640 1 -- 172.21.15.76:0/1470690993 shutdown_connections 2023-11-09T03:45:51.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.741+0000 7fa8f8ff9640 1 --2- 172.21.15.76:0/1470690993 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa8ec06af40 0x7fa8ec06d400 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.741+0000 7fa8f8ff9640 1 --2- 172.21.15.76:0/1470690993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa90c0b2010 0x7fa90c1e94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.741+0000 7fa8f8ff9640 1 --2- 172.21.15.76:0/1470690993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa90c071fa0 0x7fa90c0b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.741+0000 7fa8f8ff9640 1 --2- 172.21.15.76:0/1470690993 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa90c06a660 0x7fa90c1eb2f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.741+0000 7fa8f8ff9640 1 -- 172.21.15.76:0/1470690993 >> 172.21.15.76:0/1470690993 conn(0x7fa90c06d800 msgr2=0x7fa90c0cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:51.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.742+0000 7fa8f8ff9640 1 -- 172.21.15.76:0/1470690993 shutdown_connections 2023-11-09T03:45:51.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.742+0000 7fa8f8ff9640 1 -- 172.21.15.76:0/1470690993 wait complete. 2023-11-09T03:45:51.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.743+0000 7f82de051640 1 -- 172.21.15.76:0/727115994 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f82d804ece0 con 0x7f82d80b1150 2023-11-09T03:45:51.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.743+0000 7f82c3fff640 1 -- 172.21.15.76:0/727115994 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f82c404c020 con 0x7f82d80b1150 2023-11-09T03:45:51.744 INFO:teuthology.orchestra.run.smithi076.stdout:55834574855 2023-11-09T03:45:51.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.744+0000 7fc7ad400640 1 -- 172.21.15.76:0/1884531060 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fc7a0052f80 con 0x7fc7900047c0 2023-11-09T03:45:51.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.744+0000 7fc78f7fe640 1 -- 172.21.15.76:0/1884531060 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc79805bac0 con 0x7fc7900047c0 2023-11-09T03:45:51.745 INFO:teuthology.orchestra.run.smithi076.stdout:55834574856 2023-11-09T03:45:51.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.747+0000 7f82de051640 1 -- 172.21.15.76:0/727115994 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f82bc06ae70 msgr2=0x7f82bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.747+0000 7f82de051640 1 --2- 172.21.15.76:0/727115994 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f82bc06ae70 0x7f82bc06d330 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009e10 tx=0x7f82cc006ee0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.747+0000 7f82de051640 1 -- 172.21.15.76:0/727115994 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82d80b1150 msgr2=0x7f82d80c5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.747+0000 7f82de051640 1 --2- 172.21.15.76:0/727115994 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82d80b1150 0x7f82d80c5b80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f82c800c2e0 tx=0x7f82c40036f0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.747+0000 7f82de051640 1 -- 172.21.15.76:0/727115994 shutdown_connections 2023-11-09T03:45:51.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.747+0000 7f82de051640 1 --2- 172.21.15.76:0/727115994 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f82bc06ae70 0x7f82bc06d330 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.747+0000 7f82de051640 1 --2- 172.21.15.76:0/727115994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82d8159d20 0x7f82d81548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.747+0000 7f82de051640 1 --2- 172.21.15.76:0/727115994 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82d80b1150 0x7f82d80c5b80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.747+0000 7f82de051640 1 --2- 172.21.15.76:0/727115994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82c80047c0 0x7f82d80b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.747+0000 7f82de051640 1 -- 172.21.15.76:0/727115994 >> 172.21.15.76:0/727115994 conn(0x7f82d80cd280 msgr2=0x7f82d80cec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:51.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.748+0000 7fc78d7fa640 1 -- 172.21.15.76:0/1884531060 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc79406af40 msgr2=0x7fc79406d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.748+0000 7fc78d7fa640 1 --2- 172.21.15.76:0/1884531060 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc79406af40 0x7fc79406d400 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc79c007ef0 tx=0x7fc79c030000 comp rx=0 tx=0).stop 2023-11-09T03:45:51.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.748+0000 7fc78d7fa640 1 -- 172.21.15.76:0/1884531060 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7900047c0 msgr2=0x7fc7a00e2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.748+0000 7fc78d7fa640 1 --2- 172.21.15.76:0/1884531060 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7900047c0 0x7fc7a00e2340 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0003f30 tx=0x7fc79800f7e0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.748+0000 7f82de051640 1 -- 172.21.15.76:0/727115994 shutdown_connections 2023-11-09T03:45:51.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.748+0000 7fc78d7fa640 1 -- 172.21.15.76:0/1884531060 shutdown_connections 2023-11-09T03:45:51.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.748+0000 7fc78d7fa640 1 --2- 172.21.15.76:0/1884531060 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc79406af40 0x7fc79406d400 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.748+0000 7fc78d7fa640 1 --2- 172.21.15.76:0/1884531060 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7a0036e60 0x7fc7a0037310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.748+0000 7fc78d7fa640 1 --2- 172.21.15.76:0/1884531060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7a00dd690 0x7fc7a0174fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.748+0000 7fc78d7fa640 1 --2- 172.21.15.76:0/1884531060 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7900047c0 0x7fc7a00e2340 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.748+0000 7fc78d7fa640 1 -- 172.21.15.76:0/1884531060 >> 172.21.15.76:0/1884531060 conn(0x7fc7a0055390 msgr2=0x7fc7a004b200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:51.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.748+0000 7f82de051640 1 -- 172.21.15.76:0/727115994 wait complete. 2023-11-09T03:45:51.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.748+0000 7fc78d7fa640 1 -- 172.21.15.76:0/1884531060 shutdown_connections 2023-11-09T03:45:51.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.748+0000 7fc78d7fa640 1 -- 172.21.15.76:0/1884531060 wait complete. 2023-11-09T03:45:51.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.750+0000 7f6295b05640 1 -- 172.21.15.76:0/728681820 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f629004ece0 con 0x7f62900cb990 2023-11-09T03:45:51.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.750+0000 7f832b0f3640 1 -- 172.21.15.76:0/3107859883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f83240c4910 con 0x7f83241572f0 2023-11-09T03:45:51.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.751+0000 7f831affd640 1 -- 172.21.15.76:0/3107859883 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f83240c4910 con 0x7f83241572f0 2023-11-09T03:45:51.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.751+0000 7f6277fff640 1 -- 172.21.15.76:0/728681820 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6280017090 con 0x7f62900cb990 2023-11-09T03:45:51.755 INFO:teuthology.orchestra.run.smithi076.stdout:55834574856 2023-11-09T03:45:51.755 INFO:teuthology.orchestra.run.smithi076.stdout:55834574856 2023-11-09T03:45:51.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.752+0000 7f32d3b55640 1 -- 172.21.15.76:0/3816548299 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f32cc0c9690 con 0x7f32cc0cd3b0 2023-11-09T03:45:51.756 INFO:teuthology.orchestra.run.smithi076.stdout:55834574856 2023-11-09T03:45:51.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.753+0000 7f32c27fc640 1 -- 172.21.15.76:0/3816548299 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f32c8057800 con 0x7f32cc0cd3b0 2023-11-09T03:45:51.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.754+0000 7f832b0f3640 1 -- 172.21.15.76:0/3107859883 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f830806aef0 msgr2=0x7f830806d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.754+0000 7f6295b05640 1 -- 172.21.15.76:0/728681820 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f627c06ae70 msgr2=0x7f627c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.754+0000 7f6295b05640 1 --2- 172.21.15.76:0/728681820 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f627c06ae70 0x7f627c06d330 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6284005880 tx=0x7f6284009040 comp rx=0 tx=0).stop 2023-11-09T03:45:51.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.754+0000 7fd47d7c0640 1 -- 172.21.15.76:0/2290244843 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd4700dd0e0 con 0x7fd47003a860 2023-11-09T03:45:51.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.754+0000 7f832b0f3640 1 --2- 172.21.15.76:0/3107859883 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f830806aef0 0x7f830806d3b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f83240c95f0 tx=0x7f8314009040 comp rx=0 tx=0).stop 2023-11-09T03:45:51.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.754+0000 7f6295b05640 1 -- 172.21.15.76:0/728681820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62900cb990 msgr2=0x7f62901fa110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.754+0000 7f6295b05640 1 --2- 172.21.15.76:0/728681820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62900cb990 0x7f62901fa110 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f628000e9e0 tx=0x7f628000eea0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.754+0000 7f832b0f3640 1 -- 172.21.15.76:0/3107859883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83241572f0 msgr2=0x7f832415bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.754+0000 7f6295b05640 1 -- 172.21.15.76:0/728681820 shutdown_connections 2023-11-09T03:45:51.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.754+0000 7f6295b05640 1 --2- 172.21.15.76:0/728681820 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f627c06ae70 0x7f627c06d330 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.754+0000 7f6295b05640 1 --2- 172.21.15.76:0/728681820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62901612d0 0x7f629015dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.754+0000 7f6295b05640 1 --2- 172.21.15.76:0/728681820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62900cb990 0x7f62901fa110 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.754+0000 7f6295b05640 1 --2- 172.21.15.76:0/728681820 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62900b1860 0x7f629014f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.754+0000 7f6295b05640 1 -- 172.21.15.76:0/728681820 >> 172.21.15.76:0/728681820 conn(0x7f629006c7a0 msgr2=0x7f62900b4220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:51.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.755+0000 7fd45f7fe640 1 -- 172.21.15.76:0/2290244843 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd468017020 con 0x7fd47003a860 2023-11-09T03:45:51.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.755+0000 7f6295b05640 1 -- 172.21.15.76:0/728681820 shutdown_connections 2023-11-09T03:45:51.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.754+0000 7f832b0f3640 1 --2- 172.21.15.76:0/3107859883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83241572f0 0x7f832415bc60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f832002c2a0 tx=0x7f832002e000 comp rx=0 tx=0).stop 2023-11-09T03:45:51.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.754+0000 7f832b0f3640 1 -- 172.21.15.76:0/3107859883 shutdown_connections 2023-11-09T03:45:51.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.754+0000 7f832b0f3640 1 --2- 172.21.15.76:0/3107859883 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f830806aef0 0x7f830806d3b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.754+0000 7f832b0f3640 1 --2- 172.21.15.76:0/3107859883 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8324157770 0x7f832415c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.754+0000 7f832b0f3640 1 --2- 172.21.15.76:0/3107859883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83241572f0 0x7f832415bc60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.754+0000 7f832b0f3640 1 --2- 172.21.15.76:0/3107859883 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83100047c0 0x7f8324152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.754+0000 7f832b0f3640 1 -- 172.21.15.76:0/3107859883 >> 172.21.15.76:0/3107859883 conn(0x7f83240cd430 msgr2=0x7f832406c460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:51.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.754+0000 7f832b0f3640 1 -- 172.21.15.76:0/3107859883 shutdown_connections 2023-11-09T03:45:51.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.754+0000 7f832b0f3640 1 -- 172.21.15.76:0/3107859883 wait complete. 2023-11-09T03:45:51.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.755+0000 7f6295b05640 1 -- 172.21.15.76:0/728681820 wait complete. 2023-11-09T03:45:51.765 INFO:teuthology.orchestra.run.smithi076.stdout:55834574855 2023-11-09T03:45:51.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.757+0000 7f32d3b55640 1 -- 172.21.15.76:0/3816548299 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f32b006aec0 msgr2=0x7f32b006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.757+0000 7f32d3b55640 1 --2- 172.21.15.76:0/3816548299 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f32b006aec0 0x7f32b006d380 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009280 tx=0x7f32bc03b040 comp rx=0 tx=0).stop 2023-11-09T03:45:51.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.758+0000 7f32d3b55640 1 -- 172.21.15.76:0/3816548299 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32cc0cd3b0 msgr2=0x7f32cc0c8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.758+0000 7f32d3b55640 1 --2- 172.21.15.76:0/3816548299 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32cc0cd3b0 0x7f32cc0c8b90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f32c8006fd0 tx=0x7f32c800ec70 comp rx=0 tx=0).stop 2023-11-09T03:45:51.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.758+0000 7f32d3b55640 1 -- 172.21.15.76:0/3816548299 shutdown_connections 2023-11-09T03:45:51.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.758+0000 7f32d3b55640 1 --2- 172.21.15.76:0/3816548299 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f32b006aec0 0x7f32b006d380 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.758+0000 7f32d3b55640 1 --2- 172.21.15.76:0/3816548299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32cc15af20 0x7f32cc15cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.758+0000 7f32d3b55640 1 --2- 172.21.15.76:0/3816548299 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32cc0cd3b0 0x7f32cc0c8b90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.758+0000 7f32d3b55640 1 --2- 172.21.15.76:0/3816548299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32cc0b64c0 0x7f32cc0c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.758+0000 7f32d3b55640 1 -- 172.21.15.76:0/3816548299 >> 172.21.15.76:0/3816548299 conn(0x7f32cc06ea20 msgr2=0x7f32cc158950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:51.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.758+0000 7fd47d7c0640 1 -- 172.21.15.76:0/2290244843 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd46406af40 msgr2=0x7fd46406d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.759+0000 7fd47d7c0640 1 --2- 172.21.15.76:0/2290244843 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd46406af40 0x7fd46406d400 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd4700cfa20 tx=0x7fd46c009040 comp rx=0 tx=0).stop 2023-11-09T03:45:51.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.759+0000 7f32d3b55640 1 -- 172.21.15.76:0/3816548299 shutdown_connections 2023-11-09T03:45:51.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.759+0000 7f32d3b55640 1 -- 172.21.15.76:0/3816548299 wait complete. 2023-11-09T03:45:51.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.759+0000 7fd47d7c0640 1 -- 172.21.15.76:0/2290244843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd47003a860 msgr2=0x7fd470033250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.759+0000 7fd47d7c0640 1 --2- 172.21.15.76:0/2290244843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd47003a860 0x7fd470033250 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd46800c410 tx=0x7fd46800c8d0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.759+0000 7fd47d7c0640 1 -- 172.21.15.76:0/2290244843 shutdown_connections 2023-11-09T03:45:51.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.759+0000 7fd47d7c0640 1 --2- 172.21.15.76:0/2290244843 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd46406af40 0x7fd46406d400 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.759+0000 7fd47d7c0640 1 --2- 172.21.15.76:0/2290244843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4700e3350 0x7fd470031240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.759+0000 7fd47d7c0640 1 --2- 172.21.15.76:0/2290244843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd470050120 0x7fd4701783c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.759+0000 7fd47d7c0640 1 --2- 172.21.15.76:0/2290244843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd47003a860 0x7fd470033250 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.759+0000 7fd47d7c0640 1 -- 172.21.15.76:0/2290244843 >> 172.21.15.76:0/2290244843 conn(0x7fd470054c40 msgr2=0x7fd470048770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:51.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.759+0000 7fd47d7c0640 1 -- 172.21.15.76:0/2290244843 shutdown_connections 2023-11-09T03:45:51.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.760+0000 7fd47d7c0640 1 -- 172.21.15.76:0/2290244843 wait complete. 2023-11-09T03:45:51.773 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.3 2023-11-09T03:45:51.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.767+0000 7f32a6e4d640 1 -- 172.21.15.76:0/4008423064 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3288005500 con 0x7f32a00c9560 2023-11-09T03:45:51.774 INFO:teuthology.orchestra.run.smithi076.stdout:55834574855 2023-11-09T03:45:51.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.768+0000 7f32967fc640 1 -- 172.21.15.76:0/4008423064 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3290053f10 con 0x7f32a00c9560 2023-11-09T03:45:51.775 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.4 2023-11-09T03:45:51.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.771+0000 7f3283fff640 1 -- 172.21.15.76:0/4008423064 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f328406ae70 msgr2=0x7f328406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.772+0000 7f3283fff640 1 --2- 172.21.15.76:0/4008423064 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f328406ae70 0x7f328406d330 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f32980113e0 tx=0x7f329800b040 comp rx=0 tx=0).stop 2023-11-09T03:45:51.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.772+0000 7f3283fff640 1 -- 172.21.15.76:0/4008423064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32a00c9560 msgr2=0x7f32a0150160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:51.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.772+0000 7f3283fff640 1 --2- 172.21.15.76:0/4008423064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32a00c9560 0x7f32a0150160 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f329000c3c0 tx=0x7f329000c880 comp rx=0 tx=0).stop 2023-11-09T03:45:51.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.772+0000 7f3283fff640 1 -- 172.21.15.76:0/4008423064 shutdown_connections 2023-11-09T03:45:51.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.772+0000 7f3283fff640 1 --2- 172.21.15.76:0/4008423064 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f328406ae70 0x7f328406d330 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.772+0000 7f3283fff640 1 --2- 172.21.15.76:0/4008423064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32a00c9560 0x7f32a0150160 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.772+0000 7f3283fff640 1 --2- 172.21.15.76:0/4008423064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32a00b1aa0 0x7f32a014fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.772+0000 7f3283fff640 1 --2- 172.21.15.76:0/4008423064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32a006e600 0x7f32a00c1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:51.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.772+0000 7f3283fff640 1 -- 172.21.15.76:0/4008423064 >> 172.21.15.76:0/4008423064 conn(0x7f32a00b02c0 msgr2=0x7f32a00ce600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:51.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.772+0000 7f3283fff640 1 -- 172.21.15.76:0/4008423064 shutdown_connections 2023-11-09T03:45:51.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:51.772+0000 7f3283fff640 1 -- 172.21.15.76:0/4008423064 wait complete. 2023-11-09T03:45:51.779 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.1 2023-11-09T03:45:51.780 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.5 2023-11-09T03:45:51.780 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.7 2023-11-09T03:45:51.781 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.2 2023-11-09T03:45:51.782 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.6 2023-11-09T03:45:51.785 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.0 2023-11-09T03:45:52.774 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-09T03:45:52.776 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-09T03:45:52.780 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-09T03:45:52.780 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-09T03:45:52.781 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-09T03:45:52.782 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-09T03:45:52.783 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-09T03:45:52.786 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-09T03:45:52.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.937+0000 7f1235644640 1 Processor -- start 2023-11-09T03:45:52.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.937+0000 7f1235644640 1 -- start start 2023-11-09T03:45:52.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.938+0000 7f1235644640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f12301592c0 0x7f123006b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:52.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.938+0000 7f1235644640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f123006cd70 0x7f123006d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:52.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.938+0000 7f1235644640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f123015ce30 0x7f123015d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:52.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.938+0000 7f1235644640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f123015de20 con 0x7f123006cd70 2023-11-09T03:45:52.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.938+0000 7f1235644640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f123006bb40 con 0x7f12301592c0 2023-11-09T03:45:52.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.938+0000 7f1235644640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f123006d710 con 0x7f123015ce30 2023-11-09T03:45:52.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.939+0000 7f122e7fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f123006cd70 0x7f123006d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:52.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.939+0000 7f122e7fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f123006cd70 0x7f123006d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36184/0 (socket says 172.21.15.76:36184) 2023-11-09T03:45:52.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.939+0000 7f122e7fc640 1 -- 172.21.15.76:0/500122844 learned_addr learned my addr 172.21.15.76:0/500122844 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:52.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.939+0000 7f122f7fe640 1 --2- 172.21.15.76:0/500122844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f123015ce30 0x7f123015d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:52.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.939+0000 7f122e7fc640 1 -- 172.21.15.76:0/500122844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f123015ce30 msgr2=0x7f123015d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:52.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.939+0000 7f122e7fc640 1 --2- 172.21.15.76:0/500122844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f123015ce30 0x7f123015d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:52.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.939+0000 7f122e7fc640 1 -- 172.21.15.76:0/500122844 >> v2:172.21.15.113:3300/0 conn(0x7f12301592c0 msgr2=0x7f123006b600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-09T03:45:52.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.939+0000 7f122e7fc640 1 --2- 172.21.15.76:0/500122844 >> v2:172.21.15.113:3300/0 conn(0x7f12301592c0 0x7f123006b600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:52.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.939+0000 7f122e7fc640 1 -- 172.21.15.76:0/500122844 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f123006b170 con 0x7f123006cd70 2023-11-09T03:45:52.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.941+0000 7f122e7fc640 1 --2- 172.21.15.76:0/500122844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f123006cd70 0x7f123006d1d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f12300cda70 tx=0x7f122002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeacd7cf706850d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:52.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.943+0000 7f122dffb640 1 -- 172.21.15.76:0/500122844 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1220031870 con 0x7f123006cd70 2023-11-09T03:45:52.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.943+0000 7f122dffb640 1 -- 172.21.15.76:0/500122844 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12200319d0 con 0x7f123006cd70 2023-11-09T03:45:52.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.943+0000 7f122dffb640 1 -- 172.21.15.76:0/500122844 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1220031d60 con 0x7f123006cd70 2023-11-09T03:45:52.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.944+0000 7f1235644640 1 -- 172.21.15.76:0/500122844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f123006cd70 msgr2=0x7f123006d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:52.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.941+0000 7fc9cebf4640 1 Processor -- start 2023-11-09T03:45:52.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.941+0000 7fc9cebf4640 1 -- start start 2023-11-09T03:45:52.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.942+0000 7fc9cebf4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9c00eb480 0x7fc9c00ebdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:52.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.942+0000 7fc9cebf4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9c00574d0 con 0x7fc9c00eb860 2023-11-09T03:45:52.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.942+0000 7fc9cebf4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9c00577b0 con 0x7fc9c0007350 2023-11-09T03:45:52.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.942+0000 7fc9cebf4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9c0058150 con 0x7fc9c00eb480 2023-11-09T03:45:52.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.943+0000 7fc9cd3f1640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc9c00eb860 0x7fc9c0056cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33600/0 (socket says 172.21.15.76:33600) 2023-11-09T03:45:52.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.943+0000 7fc9cd3f1640 1 -- 172.21.15.76:0/2373585564 learned_addr learned my addr 172.21.15.76:0/2373585564 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:52.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.944+0000 7f1235644640 1 --2- 172.21.15.76:0/500122844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f123006cd70 0x7f123006d1d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f12300cda70 tx=0x7f122002b910 comp rx=0 tx=0).stop 2023-11-09T03:45:52.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.945+0000 7f1235644640 1 -- 172.21.15.76:0/500122844 shutdown_connections 2023-11-09T03:45:52.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.945+0000 7f1235644640 1 --2- 172.21.15.76:0/500122844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f123015ce30 0x7f123015d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:52.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.945+0000 7f1235644640 1 --2- 172.21.15.76:0/500122844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f123006cd70 0x7f123006d1d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:52.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.945+0000 7f1235644640 1 --2- 172.21.15.76:0/500122844 >> v2:172.21.15.113:3300/0 conn(0x7f12301592c0 0x7f123006b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:52.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.945+0000 7f1235644640 1 -- 172.21.15.76:0/500122844 >> 172.21.15.76:0/500122844 conn(0x7f12300cf140 msgr2=0x7f12300cf520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:52.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.945+0000 7f1235644640 1 -- 172.21.15.76:0/500122844 shutdown_connections 2023-11-09T03:45:52.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.947+0000 7fc9ccbf0640 1 -- 172.21.15.76:0/2373585564 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516921632 0 0) 0x7fc9c00577b0 con 0x7fc9c0007350 2023-11-09T03:45:52.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.947+0000 7fc9ccbf0640 1 -- 172.21.15.76:0/2373585564 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9b4003520 con 0x7fc9c0007350 2023-11-09T03:45:52.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.947+0000 7fc9ccbf0640 1 -- 172.21.15.76:0/2373585564 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2602781750 0 0) 0x7fc9c00574d0 con 0x7fc9c00eb860 2023-11-09T03:45:52.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.947+0000 7fc9ccbf0640 1 -- 172.21.15.76:0/2373585564 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9c00577b0 con 0x7fc9c00eb860 2023-11-09T03:45:52.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.947+0000 7fc9ccbf0640 1 -- 172.21.15.76:0/2373585564 <== mon.1 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3641697485 0 0) 0x7fc9b4003520 con 0x7fc9c0007350 2023-11-09T03:45:52.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.947+0000 7fc9ccbf0640 1 -- 172.21.15.76:0/2373585564 >> v2:172.21.15.76:3300/0 conn(0x7fc9c00eb480 msgr2=0x7fc9c00ebdb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T03:45:52.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.947+0000 7fc9ccbf0640 1 --2- 172.21.15.76:0/2373585564 >> v2:172.21.15.76:3300/0 conn(0x7fc9c00eb480 0x7fc9c00ebdb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:52.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.947+0000 7fc9ccbf0640 1 -- 172.21.15.76:0/2373585564 >> v1:172.21.15.76:6789/0 conn(0x7fc9c00eb860 legacy=0x7fc9c0056cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:52.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.947+0000 7fc9ccbf0640 1 -- 172.21.15.76:0/2373585564 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c0057a90 con 0x7fc9c0007350 2023-11-09T03:45:52.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.947+0000 7fc9ccbf0640 1 -- 172.21.15.76:0/2373585564 <== mon.1 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fc9c8073cb0 con 0x7fc9c0007350 2023-11-09T03:45:52.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.947+0000 7fc9ccbf0640 1 -- 172.21.15.76:0/2373585564 >> v1:172.21.15.113:6789/0 conn(0x7fc9c0007350 legacy=0x7fc9c00e77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:52.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.947+0000 7fc9ccbf0640 1 --2- 172.21.15.76:0/2373585564 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9b4003b00 0x7fc9b4003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:52.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.947+0000 7fc9ccbf0640 1 --2- 172.21.15.76:0/2373585564 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9b40047c0 0x7fc9b4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:52.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.947+0000 7fc9ccbf0640 1 --2- 172.21.15.76:0/2373585564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9b4005180 0x7fc9b400b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:52.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.948+0000 7fc9ccbf0640 1 -- 172.21.15.76:0/2373585564 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c0058150 con 0x7fc9b4003b00 2023-11-09T03:45:52.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.948+0000 7fc9ccbf0640 1 -- 172.21.15.76:0/2373585564 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b400c120 con 0x7fc9b4005180 2023-11-09T03:45:52.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.948+0000 7fc9ccbf0640 1 -- 172.21.15.76:0/2373585564 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b400c2e0 con 0x7fc9b40047c0 2023-11-09T03:45:52.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.948+0000 7fc9cd3f1640 1 --2- 172.21.15.76:0/2373585564 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9b40047c0 0x7fc9b4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:52.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.948+0000 7fc9ce3f3640 1 --2- 172.21.15.76:0/2373585564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9b4005180 0x7fc9b400b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:52.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.945+0000 7f1235644640 1 -- 172.21.15.76:0/500122844 wait complete. 2023-11-09T03:45:52.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.947+0000 7f1235644640 1 Processor -- start 2023-11-09T03:45:52.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.947+0000 7f1235644640 1 -- start start 2023-11-09T03:45:52.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.948+0000 7f1235644640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f123006cd70 0x7f12300c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:52.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.948+0000 7f1235644640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12301592c0 0x7f12300cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:52.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.948+0000 7f1235644640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f123015ce30 0x7f12301e50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:52.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.949+0000 7f122effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f123006cd70 0x7f12300c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:52.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.949+0000 7f122effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f123006cd70 0x7f12300c6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:50676/0 (socket says 172.21.15.76:50676) 2023-11-09T03:45:52.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.949+0000 7f122effd640 1 -- 172.21.15.76:0/2360603857 learned_addr learned my addr 172.21.15.76:0/2360603857 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:52.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.949+0000 7f1235644640 1 -- 172.21.15.76:0/2360603857 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12300c72d0 con 0x7f12301592c0 2023-11-09T03:45:52.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.949+0000 7f1235644640 1 -- 172.21.15.76:0/2360603857 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1230154980 con 0x7f123015ce30 2023-11-09T03:45:52.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.949+0000 7f1235644640 1 -- 172.21.15.76:0/2360603857 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12301e5c90 con 0x7f123006cd70 2023-11-09T03:45:52.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.949+0000 7f122f7fe640 1 --2- 172.21.15.76:0/2360603857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f123015ce30 0x7f12301e50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:52.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.949+0000 7f122effd640 1 -- 172.21.15.76:0/2360603857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f123015ce30 msgr2=0x7f12301e50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:52.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.949+0000 7f122effd640 1 --2- 172.21.15.76:0/2360603857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f123015ce30 0x7f12301e50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:52.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.949+0000 7f122effd640 1 -- 172.21.15.76:0/2360603857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12301592c0 msgr2=0x7f12300cbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:52.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.949+0000 7f122effd640 1 --2- 172.21.15.76:0/2360603857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12301592c0 0x7f12300cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:52.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.949+0000 7f122effd640 1 -- 172.21.15.76:0/2360603857 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12200094d0 con 0x7f123006cd70 2023-11-09T03:45:52.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.950+0000 7f122effd640 1 --2- 172.21.15.76:0/2360603857 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f123006cd70 0x7f12300c6d90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f122400b480 tx=0x7f122400b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:52.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.948+0000 7fc9ce3f3640 1 -- 172.21.15.76:0/2373585564 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9b40047c0 msgr2=0x7fc9b4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:52.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.948+0000 7fc9ce3f3640 1 --2- 172.21.15.76:0/2373585564 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9b40047c0 0x7fc9b4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:52.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.948+0000 7fc9ce3f3640 1 -- 172.21.15.76:0/2373585564 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9b4003b00 msgr2=0x7fc9b4003f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:52.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.948+0000 7fc9cdbf2640 1 --2- 172.21.15.76:0/2373585564 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9b4003b00 0x7fc9b4003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:52.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.948+0000 7fc9ce3f3640 1 --2- 172.21.15.76:0/2373585564 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9b4003b00 0x7fc9b4003f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:52.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.948+0000 7fc9ce3f3640 1 -- 172.21.15.76:0/2373585564 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b4003520 con 0x7fc9b4005180 2023-11-09T03:45:52.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.949+0000 7fc9ce3f3640 1 -- 172.21.15.76:0/2373585564 reap_dead start 2023-11-09T03:45:52.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.949+0000 7fc9ce3f3640 1 --2- 172.21.15.76:0/2373585564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9b4005180 0x7fc9b400b9f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc9c804f700 tx=0x7fc9b4005870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb71acbb5a43b4c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:52.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.950+0000 7fc9ccbf0640 1 -- 172.21.15.76:0/2373585564 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80ae430 con 0x7fc9b4005180 2023-11-09T03:45:52.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.951+0000 7fc9ccbf0640 1 -- 172.21.15.76:0/2373585564 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c80ae590 con 0x7fc9b4005180 2023-11-09T03:45:52.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.951+0000 7fc9ccbf0640 1 -- 172.21.15.76:0/2373585564 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80ae980 con 0x7fc9b4005180 2023-11-09T03:45:52.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.951+0000 7f12177fe640 1 -- 172.21.15.76:0/2360603857 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1224010870 con 0x7f123006cd70 2023-11-09T03:45:52.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.951+0000 7f1235644640 1 -- 172.21.15.76:0/2360603857 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12300c62b0 con 0x7f123006cd70 2023-11-09T03:45:52.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.951+0000 7f1235644640 1 -- 172.21.15.76:0/2360603857 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12300c6870 con 0x7f123006cd70 2023-11-09T03:45:52.963 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.952+0000 7f12177fe640 1 -- 172.21.15.76:0/2360603857 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12240109f0 con 0x7f123006cd70 2023-11-09T03:45:52.963 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.953+0000 7f12177fe640 1 -- 172.21.15.76:0/2360603857 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1224010da0 con 0x7f123006cd70 2023-11-09T03:45:52.963 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.953+0000 7f1235644640 1 -- 172.21.15.76:0/2360603857 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12300c6a90 con 0x7f123006cd70 2023-11-09T03:45:52.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.953+0000 7fc9cebf4640 1 -- 172.21.15.76:0/2373585564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9b4005180 msgr2=0x7fc9b400b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:52.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.953+0000 7fc9cebf4640 1 --2- 172.21.15.76:0/2373585564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9b4005180 0x7fc9b400b9f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc9c804f700 tx=0x7fc9b4005870 comp rx=0 tx=0).stop 2023-11-09T03:45:52.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.954+0000 7fc9cebf4640 1 -- 172.21.15.76:0/2373585564 shutdown_connections 2023-11-09T03:45:52.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.954+0000 7fc9cebf4640 1 --2- 172.21.15.76:0/2373585564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9b4005180 0x7fc9b400b9f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:52.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.954+0000 7fc9cebf4640 1 -- 172.21.15.76:0/2373585564 >> 172.21.15.76:0/2373585564 conn(0x7fc9c0059120 msgr2=0x7fc9c0059530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:52.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.954+0000 7fc9cebf4640 1 -- 172.21.15.76:0/2373585564 shutdown_connections 2023-11-09T03:45:52.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.955+0000 7fc9cebf4640 1 -- 172.21.15.76:0/2373585564 wait complete. 2023-11-09T03:45:52.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.956+0000 7fc9cebf4640 1 Processor -- start 2023-11-09T03:45:52.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.956+0000 7fc9cebf4640 1 -- start start 2023-11-09T03:45:52.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.960+0000 7f12177fe640 1 -- 172.21.15.76:0/2360603857 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1224023480 con 0x7f123006cd70 2023-11-09T03:45:52.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.960+0000 7f12177fe640 1 --2- 172.21.15.76:0/2360603857 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f121c06af40 0x7f121c06d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:52.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.960+0000 7f12177fe640 1 -- 172.21.15.76:0/2360603857 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1224089d90 con 0x7f123006cd70 2023-11-09T03:45:52.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.960+0000 7f12177fe640 1 -- 172.21.15.76:0/2360603857 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f12240b6ca0 con 0x7f123006cd70 2023-11-09T03:45:52.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.958+0000 7fc9cebf4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9b4005180 0x7fc9c00e6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:52.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.958+0000 7fc9cebf4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9c00e5980 0x7fc9c00e2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:52.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.958+0000 7fc9cebf4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9c00e30b0 0x7fc9c00cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:52.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.958+0000 7fc9ce3f3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9c00e30b0 0x7fc9c00cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:52.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.958+0000 7fc9ce3f3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9c00e30b0 0x7fc9c00cf4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36218/0 (socket says 172.21.15.76:36218) 2023-11-09T03:45:52.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.958+0000 7fc9ce3f3640 1 -- 172.21.15.76:0/2861498379 learned_addr learned my addr 172.21.15.76:0/2861498379 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:52.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.958+0000 7fc9cebf4640 1 -- 172.21.15.76:0/2861498379 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c00e2390 con 0x7fc9c00e30b0 2023-11-09T03:45:52.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.958+0000 7fc9cebf4640 1 -- 172.21.15.76:0/2861498379 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c00e6160 con 0x7fc9c00e5980 2023-11-09T03:45:52.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.958+0000 7fc9cebf4640 1 -- 172.21.15.76:0/2861498379 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c00e6440 con 0x7fc9b4005180 2023-11-09T03:45:52.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.963+0000 7f122e7fc640 1 --2- 172.21.15.76:0/2360603857 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f121c06af40 0x7f121c06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:52.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.963+0000 7fc9cdbf2640 1 --2- 172.21.15.76:0/2861498379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9b4005180 0x7fc9c00e6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:52.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.963+0000 7fc9cd3f1640 1 --2- 172.21.15.76:0/2861498379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9c00e5980 0x7fc9c00e2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:52.970 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.966+0000 7fc9ce3f3640 1 -- 172.21.15.76:0/2861498379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9b4005180 msgr2=0x7fc9c00e6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:52.970 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.966+0000 7fc9ce3f3640 1 --2- 172.21.15.76:0/2861498379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9b4005180 0x7fc9c00e6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:52.970 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.966+0000 7fc9ce3f3640 1 -- 172.21.15.76:0/2861498379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9c00e5980 msgr2=0x7fc9c00e2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:52.970 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.966+0000 7fc9ce3f3640 1 --2- 172.21.15.76:0/2861498379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9c00e5980 0x7fc9c00e2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:52.970 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.966+0000 7fc9ce3f3640 1 -- 172.21.15.76:0/2861498379 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c00e6720 con 0x7fc9c00e30b0 2023-11-09T03:45:52.971 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.967+0000 7fc9ce3f3640 1 --2- 172.21.15.76:0/2861498379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9c00e30b0 0x7fc9c00cf4c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8000b60 tx=0x7fc9c8069f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:52.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.973+0000 7fc9be7fc640 1 -- 172.21.15.76:0/2861498379 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80ae430 con 0x7fc9c00e30b0 2023-11-09T03:45:52.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.979+0000 7f122e7fc640 1 --2- 172.21.15.76:0/2360603857 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f121c06af40 0x7f121c06d400 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1220036000 tx=0x7f1220030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:52.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.973+0000 7fc9cebf4640 1 -- 172.21.15.76:0/2861498379 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9c00e2650 con 0x7fc9c00e30b0 2023-11-09T03:45:52.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.973+0000 7fc9cebf4640 1 -- 172.21.15.76:0/2861498379 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9c00cfa00 con 0x7fc9c00e30b0 2023-11-09T03:45:52.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.976+0000 7fc9cebf4640 1 -- 172.21.15.76:0/2861498379 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9c00058e0 con 0x7fc9c00e30b0 2023-11-09T03:45:52.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.987+0000 7fc9be7fc640 1 -- 172.21.15.76:0/2861498379 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c8073cb0 con 0x7fc9c00e30b0 2023-11-09T03:45:52.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.987+0000 7fc9be7fc640 1 -- 172.21.15.76:0/2861498379 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80b9630 con 0x7fc9c00e30b0 2023-11-09T03:45:52.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.987+0000 7fc9be7fc640 1 -- 172.21.15.76:0/2861498379 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc9c80b9850 con 0x7fc9c00e30b0 2023-11-09T03:45:52.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.988+0000 7fc9be7fc640 1 --2- 172.21.15.76:0/2861498379 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc9ac06aff0 0x7fc9ac06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:52.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.988+0000 7fc9be7fc640 1 -- 172.21.15.76:0/2861498379 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc9c8126490 con 0x7fc9c00e30b0 2023-11-09T03:45:52.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.988+0000 7fc9cdbf2640 1 --2- 172.21.15.76:0/2861498379 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc9ac06aff0 0x7fc9ac06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:52.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.989+0000 7fc9cdbf2640 1 --2- 172.21.15.76:0/2861498379 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc9ac06aff0 0x7fc9ac06d4b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4006fd0 tx=0x7fc9c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:52.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.989+0000 7fc9be7fc640 1 -- 172.21.15.76:0/2861498379 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc9c81289d0 con 0x7fc9c00e30b0 2023-11-09T03:45:52.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.994+0000 7f1b63116640 1 Processor -- start 2023-11-09T03:45:52.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.992+0000 7ff7d9f16640 1 Processor -- start 2023-11-09T03:45:52.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.993+0000 7ff7d9f16640 1 -- start start 2023-11-09T03:45:52.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.994+0000 7ff7d9f16640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff7d4071f90 0x7ff7d40b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:52.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.994+0000 7ff7d9f16640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff7d4072370 0x7ff7d40cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:52.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.994+0000 7ff7d9f16640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d40b1cc0 con 0x7ff7d415add0 2023-11-09T03:45:52.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.994+0000 7ff7d9f16640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7d40b1fa0 con 0x7ff7d4071f90 2023-11-09T03:45:52.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.994+0000 7ff7d9f16640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d415a0e0 con 0x7ff7d4072370 2023-11-09T03:45:52.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.994+0000 7f1b63116640 1 -- start start 2023-11-09T03:45:52.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.995+0000 7f1b63116640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1b5c0adfd0 0x7f1b5c0afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:52.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.995+0000 7f1b63116640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b5c0b13f0 con 0x7f1b5c0b0030 2023-11-09T03:45:52.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.995+0000 7f1b63116640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b5c0b0870 con 0x7f1b5c158c50 2023-11-09T03:45:52.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.995+0000 7f1b63116640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b5c0b0b50 con 0x7f1b5c0adfd0 2023-11-09T03:45:52.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.996+0000 7f1b60eb2640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1b5c158c50 0x7f1b5c0ad8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:60592/0 (socket says 172.21.15.76:60592) 2023-11-09T03:45:52.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.996+0000 7f1b60eb2640 1 -- 172.21.15.76:0/2224350483 learned_addr learned my addr 172.21.15.76:0/2224350483 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:52.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.996+0000 7f1b5bfff640 1 --2- 172.21.15.76:0/2224350483 >> v2:172.21.15.113:3300/0 conn(0x7f1b5c0adfd0 0x7f1b5c0afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:52.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.994+0000 7ff7d2ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff7d415add0 0x7ff7d40cecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33616/0 (socket says 172.21.15.76:33616) 2023-11-09T03:45:52.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.994+0000 7ff7d2ffd640 1 -- 172.21.15.76:0/2891671978 learned_addr learned my addr 172.21.15.76:0/2891671978 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:53.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.994+0000 7ff7d37fe640 1 --2- 172.21.15.76:0/2891671978 >> v2:172.21.15.113:3300/0 conn(0x7ff7d4071f90 0x7ff7d40b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.994+0000 7ff7d3fff640 1 --2- 172.21.15.76:0/2891671978 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff7d4072370 0x7ff7d40cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.995+0000 7ff7d27fc640 1 -- 172.21.15.76:0/2891671978 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3263965206 0 0) 0x7ff7d40b1cc0 con 0x7ff7d415add0 2023-11-09T03:45:53.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.995+0000 7ff7d27fc640 1 -- 172.21.15.76:0/2891671978 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7bc00bd30 con 0x7ff7d415add0 2023-11-09T03:45:53.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.996+0000 7f1b5bfff640 1 -- 172.21.15.76:0/2224350483 >> v1:172.21.15.113:6789/0 conn(0x7f1b5c158c50 legacy=0x7f1b5c0ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.996+0000 7f1b5bfff640 1 -- 172.21.15.76:0/2224350483 >> v1:172.21.15.76:6789/0 conn(0x7f1b5c0b0030 legacy=0x7f1b5c0c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.996+0000 7f1b5bfff640 1 -- 172.21.15.76:0/2224350483 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5c0b0e30 con 0x7f1b5c0adfd0 2023-11-09T03:45:53.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.995+0000 7ff7d37fe640 1 -- 172.21.15.76:0/2891671978 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff7d4072370 msgr2=0x7ff7d40cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:53.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.995+0000 7ff7d37fe640 1 --2- 172.21.15.76:0/2891671978 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff7d4072370 0x7ff7d40cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.995+0000 7ff7d37fe640 1 -- 172.21.15.76:0/2891671978 >> v1:172.21.15.76:6789/0 conn(0x7ff7d415add0 legacy=0x7ff7d40cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.995+0000 7ff7d37fe640 1 -- 172.21.15.76:0/2891671978 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d415a2f0 con 0x7ff7d4071f90 2023-11-09T03:45:53.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.996+0000 7ff7d37fe640 1 --2- 172.21.15.76:0/2891671978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7d4071f90 0x7ff7d40b1490 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc009d90 tx=0x7ff7cc02be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dccff8b5406637f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:53.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.996+0000 7ff7d27fc640 1 -- 172.21.15.76:0/2891671978 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc02cbf0 con 0x7ff7d4071f90 2023-11-09T03:45:53.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.996+0000 7ff7d27fc640 1 -- 172.21.15.76:0/2891671978 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7cc02cd50 con 0x7ff7d4071f90 2023-11-09T03:45:53.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.997+0000 7ff7d9f16640 1 -- 172.21.15.76:0/2891671978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7d4071f90 msgr2=0x7ff7d40b1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.997+0000 7ff7d9f16640 1 --2- 172.21.15.76:0/2891671978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7d4071f90 0x7ff7d40b1490 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc009d90 tx=0x7ff7cc02be40 comp rx=0 tx=0).stop 2023-11-09T03:45:53.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.998+0000 7ff7d9f16640 1 -- 172.21.15.76:0/2891671978 shutdown_connections 2023-11-09T03:45:53.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.998+0000 7ff7d9f16640 1 --2- 172.21.15.76:0/2891671978 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff7d4072370 0x7ff7d40cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.998+0000 7ff7d9f16640 1 --2- 172.21.15.76:0/2891671978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7d4071f90 0x7ff7d40b1490 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.998+0000 7ff7d9f16640 1 -- 172.21.15.76:0/2891671978 >> 172.21.15.76:0/2891671978 conn(0x7ff7d406d800 msgr2=0x7ff7d406dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:53.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.998+0000 7f1b5bfff640 1 --2- 172.21.15.76:0/2224350483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b5c0adfd0 0x7f1b5c0afaf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1b500097a0 tx=0x7f1b5002c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0b4d9e85fcc5d47 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:53.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.998+0000 7ff7d9f16640 1 -- 172.21.15.76:0/2891671978 shutdown_connections 2023-11-09T03:45:53.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.998+0000 7ff7d9f16640 1 -- 172.21.15.76:0/2891671978 wait complete. 2023-11-09T03:45:53.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.999+0000 7fbdb86d7640 1 Processor -- start 2023-11-09T03:45:53.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:52.999+0000 7fbdb86d7640 1 -- start start 2023-11-09T03:45:53.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.001+0000 7fbdb86d7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbdb00c4690 0x7fbdb00cc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.001+0000 7fbdb86d7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdb0070710 0x7fbdb0070b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.001+0000 7fbdb86d7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb00c4b20 con 0x7fbdb0164760 2023-11-09T03:45:53.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.001+0000 7fbdb86d7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb0164b40 con 0x7fbdb00c4690 2023-11-09T03:45:53.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.001+0000 7fbdb86d7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb0071080 con 0x7fbdb0070710 2023-11-09T03:45:53.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.000+0000 7ff7d9f16640 1 Processor -- start 2023-11-09T03:45:53.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.000+0000 7ff7d9f16640 1 -- start start 2023-11-09T03:45:53.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.002+0000 7fbdb5c72640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbdb00c4690 0x7fbdb00cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.002+0000 7fbdb6c74640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdb0070710 0x7fbdb0070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.002+0000 7fbdb6c74640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdb0070710 0x7fbdb0070b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:33520/0 (socket says 172.21.15.76:33520) 2023-11-09T03:45:53.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.002+0000 7fbdb6c74640 1 -- 172.21.15.76:0/3365325838 learned_addr learned my addr 172.21.15.76:0/3365325838 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:53.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.002+0000 7ff7d9f16640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff7d4071f90 0x7ff7d414f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.002+0000 7ff7d9f16640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7d4072370 0x7ff7d40c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.002+0000 7ff7d9f16640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7d4150360 0x7ff7d40cf260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.002+0000 7ff7d9f16640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d41514a0 con 0x7ff7d4150360 2023-11-09T03:45:53.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.002+0000 7ff7d9f16640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d4150920 con 0x7ff7d4072370 2023-11-09T03:45:53.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.002+0000 7ff7d9f16640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d4150c00 con 0x7ff7d4071f90 2023-11-09T03:45:53.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.002+0000 7fbdb6c74640 1 -- 172.21.15.76:0/3365325838 >> v2:172.21.15.76:3300/0 conn(0x7fbdb00c4690 msgr2=0x7fbdb00cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:53.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.002+0000 7fbdb6c74640 1 --2- 172.21.15.76:0/3365325838 >> v2:172.21.15.76:3300/0 conn(0x7fbdb00c4690 0x7fbdb00cc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.002+0000 7fbdb6c74640 1 -- 172.21.15.76:0/3365325838 >> v1:172.21.15.113:6789/0 conn(0x7fbdb0164760 legacy=0x7fbdb00b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.002+0000 7fbdb6c74640 1 -- 172.21.15.76:0/3365325838 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb00cca60 con 0x7fbdb0070710 2023-11-09T03:45:53.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.003+0000 7ff7d2ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7d4072370 0x7ff7d40c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.003+0000 7ff7d3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7d4150360 0x7ff7d40cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.003+0000 7ff7d3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7d4150360 0x7ff7d40cf260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36244/0 (socket says 172.21.15.76:36244) 2023-11-09T03:45:53.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.003+0000 7ff7d3fff640 1 -- 172.21.15.76:0/611945235 learned_addr learned my addr 172.21.15.76:0/611945235 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:53.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.004+0000 7ff7d37fe640 1 --2- 172.21.15.76:0/611945235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff7d4071f90 0x7ff7d414f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.004+0000 7fbdb6c74640 1 --2- 172.21.15.76:0/3365325838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdb0070710 0x7fbdb0070b40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbdb00c57e0 tx=0x7fbda802bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a39ddc3c93575402 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:53.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.004+0000 7ff7d3fff640 1 -- 172.21.15.76:0/611945235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff7d4071f90 msgr2=0x7ff7d414f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.004+0000 7ff7d3fff640 1 --2- 172.21.15.76:0/611945235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff7d4071f90 0x7ff7d414f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.004+0000 7fbdb5471640 1 -- 172.21.15.76:0/3365325838 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda802c850 con 0x7fbdb0070710 2023-11-09T03:45:53.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.005+0000 7fbdb5471640 1 -- 172.21.15.76:0/3365325838 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda802c9b0 con 0x7fbdb0070710 2023-11-09T03:45:53.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.004+0000 7ff7d3fff640 1 -- 172.21.15.76:0/611945235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7d4072370 msgr2=0x7ff7d40c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.004+0000 7ff7d3fff640 1 --2- 172.21.15.76:0/611945235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7d4072370 0x7ff7d40c74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.004+0000 7ff7d3fff640 1 -- 172.21.15.76:0/611945235 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7cc009a70 con 0x7ff7d4150360 2023-11-09T03:45:53.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.006+0000 7fbdb86d7640 1 -- 172.21.15.76:0/3365325838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdb0070710 msgr2=0x7fbdb0070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.004+0000 7ff7d2ffd640 1 --2- 172.21.15.76:0/611945235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7d4072370 0x7ff7d40c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:53.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.006+0000 7f1b5b7fe640 1 -- 172.21.15.76:0/2224350483 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b50032780 con 0x7f1b5c0adfd0 2023-11-09T03:45:53.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.006+0000 7f1b5b7fe640 1 -- 172.21.15.76:0/2224350483 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b500328e0 con 0x7f1b5c0adfd0 2023-11-09T03:45:53.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.004+0000 7ff7d3fff640 1 --2- 172.21.15.76:0/611945235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7d4150360 0x7ff7d40cf260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400ba00 tx=0x7ff7c400bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:53.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.006+0000 7f1b5b7fe640 1 -- 172.21.15.76:0/2224350483 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b50032c70 con 0x7f1b5c0adfd0 2023-11-09T03:45:53.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.005+0000 7ff7bbfff640 1 -- 172.21.15.76:0/611945235 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4008810 con 0x7ff7d4150360 2023-11-09T03:45:53.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.005+0000 7ff7d9f16640 1 -- 172.21.15.76:0/611945235 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d4150ee0 con 0x7ff7d4150360 2023-11-09T03:45:53.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.005+0000 7ff7d9f16640 1 -- 172.21.15.76:0/611945235 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d414f9a0 con 0x7ff7d4150360 2023-11-09T03:45:53.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.006+0000 7ff7bbfff640 1 -- 172.21.15.76:0/611945235 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c4008970 con 0x7ff7d4150360 2023-11-09T03:45:53.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.007+0000 7f1b63116640 1 -- 172.21.15.76:0/2224350483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b5c0adfd0 msgr2=0x7f1b5c0afaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.007+0000 7f1b63116640 1 --2- 172.21.15.76:0/2224350483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b5c0adfd0 0x7f1b5c0afaf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1b500097a0 tx=0x7f1b5002c8a0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.006+0000 7ff7b9ffb640 1 -- 172.21.15.76:0/611945235 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b4005250 con 0x7ff7d4150360 2023-11-09T03:45:53.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.006+0000 7fbdb86d7640 1 --2- 172.21.15.76:0/3365325838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdb0070710 0x7fbdb0070b40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbdb00c57e0 tx=0x7fbda802bd30 comp rx=0 tx=0).stop 2023-11-09T03:45:53.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.007+0000 7fbdb86d7640 1 -- 172.21.15.76:0/3365325838 shutdown_connections 2023-11-09T03:45:53.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.007+0000 7f2d33ccf640 1 Processor -- start 2023-11-09T03:45:53.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.007+0000 7fbdb86d7640 1 --2- 172.21.15.76:0/3365325838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdb0070710 0x7fbdb0070b40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.007+0000 7fbdb86d7640 1 --2- 172.21.15.76:0/3365325838 >> v2:172.21.15.76:3300/0 conn(0x7fbdb00c4690 0x7fbdb00cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.007+0000 7fbdb86d7640 1 -- 172.21.15.76:0/3365325838 >> 172.21.15.76:0/3365325838 conn(0x7fbdb00ce950 msgr2=0x7fbdb00ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:53.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.007+0000 7ff7bbfff640 1 -- 172.21.15.76:0/611945235 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4008d20 con 0x7ff7d4150360 2023-11-09T03:45:53.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.008+0000 7fbdb86d7640 1 -- 172.21.15.76:0/3365325838 shutdown_connections 2023-11-09T03:45:53.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.007+0000 7f2d33ccf640 1 -- start start 2023-11-09T03:45:53.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.007+0000 7f1b63116640 1 -- 172.21.15.76:0/2224350483 shutdown_connections 2023-11-09T03:45:53.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.007+0000 7f1b63116640 1 --2- 172.21.15.76:0/2224350483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b5c0adfd0 0x7f1b5c0afaf0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.007+0000 7f1b63116640 1 -- 172.21.15.76:0/2224350483 >> 172.21.15.76:0/2224350483 conn(0x7f1b5c06c7a0 msgr2=0x7f1b5c06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:53.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.009+0000 7f2d33ccf640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d2c158c80 0x7f2d2c0ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.009+0000 7f2d33ccf640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2d2c159060 0x7f2d2c0cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.009+0000 7f2d33ccf640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2d2c15a070 0x7f2d2c0ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.009+0000 7f2d33ccf640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d2c0cb1e0 con 0x7f2d2c15a070 2023-11-09T03:45:53.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.009+0000 7f2d33ccf640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d2c1595d0 con 0x7f2d2c159060 2023-11-09T03:45:53.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.009+0000 7f2d33ccf640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c0caf00 con 0x7f2d2c158c80 2023-11-09T03:45:53.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.010+0000 7f2d3126a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2d2c159060 0x7f2d2c0cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.010+0000 7f2d3126a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2d2c159060 0x7f2d2c0cddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:33540/0 (socket says 172.21.15.76:33540) 2023-11-09T03:45:53.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.010+0000 7f2d3126a640 1 -- 172.21.15.76:0/258189165 learned_addr learned my addr 172.21.15.76:0/258189165 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:53.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.008+0000 7fbdb86d7640 1 -- 172.21.15.76:0/3365325838 wait complete. 2023-11-09T03:45:53.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.010+0000 7fbdb86d7640 1 Processor -- start 2023-11-09T03:45:53.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.010+0000 7f2d3126a640 1 -- 172.21.15.76:0/258189165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d2c158c80 msgr2=0x7f2d2c0ca5f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-09T03:45:53.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.010+0000 7f2d3126a640 1 --2- 172.21.15.76:0/258189165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d2c158c80 0x7f2d2c0ca5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.010+0000 7f2d3126a640 1 -- 172.21.15.76:0/258189165 >> v2:172.21.15.76:3300/0 conn(0x7f2d2c15a070 msgr2=0x7f2d2c0ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:53.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.011+0000 7fbdb86d7640 1 -- start start 2023-11-09T03:45:53.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.010+0000 7f2d3126a640 1 --2- 172.21.15.76:0/258189165 >> v2:172.21.15.76:3300/0 conn(0x7f2d2c15a070 0x7f2d2c0ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.010+0000 7f2d3126a640 1 -- 172.21.15.76:0/258189165 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2c159870 con 0x7f2d2c159060 2023-11-09T03:45:53.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.012+0000 7fbdb86d7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdb0070710 0x7fbdb014fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.012+0000 7fbdb86d7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdb00c4690 0x7fbdb01623e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.012+0000 7f1b63116640 1 -- 172.21.15.76:0/2224350483 shutdown_connections 2023-11-09T03:45:53.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.012+0000 7fbdb86d7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdb0164760 0x7fbdb013dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.012+0000 7fbdb86d7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb00c7460 con 0x7fbdb0164760 2023-11-09T03:45:53.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.012+0000 7fbdb86d7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb013eff0 con 0x7fbdb0070710 2023-11-09T03:45:53.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.012+0000 7fbdb86d7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb013f290 con 0x7fbdb00c4690 2023-11-09T03:45:53.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.012+0000 7fbdb5c72640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdb00c4690 0x7fbdb01623e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.012+0000 7fbdb5c72640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdb00c4690 0x7fbdb01623e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:50730/0 (socket says 172.21.15.76:50730) 2023-11-09T03:45:53.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.012+0000 7fbdb5c72640 1 -- 172.21.15.76:0/1769575794 learned_addr learned my addr 172.21.15.76:0/1769575794 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:53.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.013+0000 7fbdb6473640 1 --2- 172.21.15.76:0/1769575794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdb0070710 0x7fbdb014fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.013+0000 7fbdb6c74640 1 --2- 172.21.15.76:0/1769575794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdb0164760 0x7fbdb013dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.013+0000 7f2d3126a640 1 --2- 172.21.15.76:0/258189165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d2c159060 0x7f2d2c0cddb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c0cfa50 tx=0x7f2d28030150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b6a6e6543436f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:53.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.013+0000 7fbdb5c72640 1 -- 172.21.15.76:0/1769575794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdb0070710 msgr2=0x7fbdb014fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.013+0000 7fbdb5c72640 1 --2- 172.21.15.76:0/1769575794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdb0070710 0x7fbdb014fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.013+0000 7fbdb5c72640 1 -- 172.21.15.76:0/1769575794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdb0164760 msgr2=0x7fbdb013dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.013+0000 7fbdb5c72640 1 --2- 172.21.15.76:0/1769575794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdb0164760 0x7fbdb013dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.013+0000 7fbdb5c72640 1 -- 172.21.15.76:0/1769575794 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda8009b30 con 0x7fbdb00c4690 2023-11-09T03:45:53.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.013+0000 7fbdb5c72640 1 --2- 172.21.15.76:0/1769575794 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdb00c4690 0x7fbdb01623e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbdac00d800 tx=0x7fbdac00dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:53.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.013+0000 7ff7bbfff640 1 -- 172.21.15.76:0/611945235 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff7c401e400 con 0x7ff7d4150360 2023-11-09T03:45:53.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.014+0000 7ff7bbfff640 1 --2- 172.21.15.76:0/611945235 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff7c006ae70 0x7ff7c006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.014+0000 7fbda6ffd640 1 -- 172.21.15.76:0/1769575794 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdac00b6c0 con 0x7fbdb00c4690 2023-11-09T03:45:53.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.014+0000 7fbda6ffd640 1 -- 172.21.15.76:0/1769575794 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdac00b820 con 0x7fbdb00c4690 2023-11-09T03:45:53.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.014+0000 7fbda6ffd640 1 -- 172.21.15.76:0/1769575794 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdac00bb90 con 0x7fbdb00c4690 2023-11-09T03:45:53.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.014+0000 7ff7bbfff640 1 -- 172.21.15.76:0/611945235 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff7c408a070 con 0x7ff7d4150360 2023-11-09T03:45:53.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.014+0000 7f1b63116640 1 -- 172.21.15.76:0/2224350483 wait complete. 2023-11-09T03:45:53.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.015+0000 7f1b63116640 1 Processor -- start 2023-11-09T03:45:53.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.014+0000 7f9603f33640 1 Processor -- start 2023-11-09T03:45:53.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.014+0000 7f2d30a69640 1 -- 172.21.15.76:0/258189165 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2803a400 con 0x7f2d2c159060 2023-11-09T03:45:53.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.014+0000 7f2d30a69640 1 -- 172.21.15.76:0/258189165 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d28034070 con 0x7f2d2c159060 2023-11-09T03:45:53.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.014+0000 7f2d33ccf640 1 -- 172.21.15.76:0/258189165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d2c159060 msgr2=0x7f2d2c0cddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.014+0000 7f2d33ccf640 1 --2- 172.21.15.76:0/258189165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d2c159060 0x7f2d2c0cddb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c0cfa50 tx=0x7f2d28030150 comp rx=0 tx=0).stop 2023-11-09T03:45:53.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.015+0000 7f9603f33640 1 -- start start 2023-11-09T03:45:53.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.016+0000 7f9603f33640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f95fc164760 0x7f95fc0c4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.016+0000 7f9603f33640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95fc070710 0x7f95fc070b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.016+0000 7f9603f33640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95fc071080 con 0x7f95fc0cc520 2023-11-09T03:45:53.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.016+0000 7f9603f33640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95fc164b40 con 0x7f95fc164760 2023-11-09T03:45:53.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.016+0000 7f9603f33640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc0ccc90 con 0x7f95fc070710 2023-11-09T03:45:53.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.016+0000 7f2d33ccf640 1 -- 172.21.15.76:0/258189165 shutdown_connections 2023-11-09T03:45:53.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.016+0000 7f2d33ccf640 1 --2- 172.21.15.76:0/258189165 >> v2:172.21.15.76:3300/0 conn(0x7f2d2c15a070 0x7f2d2c0ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.016+0000 7f2d33ccf640 1 --2- 172.21.15.76:0/258189165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d2c159060 0x7f2d2c0cddb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.016+0000 7f2d33ccf640 1 --2- 172.21.15.76:0/258189165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d2c158c80 0x7f2d2c0ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.016+0000 7f2d33ccf640 1 -- 172.21.15.76:0/258189165 >> 172.21.15.76:0/258189165 conn(0x7f2d2c06a120 msgr2=0x7f2d2c06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:53.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.016+0000 7fbdb86d7640 1 -- 172.21.15.76:0/1769575794 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb013f570 con 0x7fbdb00c4690 2023-11-09T03:45:53.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.016+0000 7fbdb86d7640 1 -- 172.21.15.76:0/1769575794 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb013fb30 con 0x7fbdb00c4690 2023-11-09T03:45:53.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.017+0000 7f1b63116640 1 -- start start 2023-11-09T03:45:53.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.019+0000 7f1b63116640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b5c0adfd0 0x7f1b5c1ff120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.019+0000 7f1b63116640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b5c0b0030 0x7f1b5c0cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.019+0000 7f1b63116640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b5c158c50 0x7f1b5c1e6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.019+0000 7f1b63116640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c1e5250 con 0x7f1b5c158c50 2023-11-09T03:45:53.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.019+0000 7f1b63116640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c14dfb0 con 0x7f1b5c0adfd0 2023-11-09T03:45:53.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.019+0000 7f1b63116640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c1e5460 con 0x7f1b5c0b0030 2023-11-09T03:45:53.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.016+0000 7f2d33ccf640 1 -- 172.21.15.76:0/258189165 shutdown_connections 2023-11-09T03:45:53.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.016+0000 7f2d33ccf640 1 -- 172.21.15.76:0/258189165 wait complete. 2023-11-09T03:45:53.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.018+0000 7f2d33ccf640 1 Processor -- start 2023-11-09T03:45:53.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.017+0000 7ff7bbfff640 1 -- 172.21.15.76:0/611945235 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff7c4054ea0 con 0x7ff7d4150360 2023-11-09T03:45:53.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.017+0000 7fbdb86d7640 1 -- 172.21.15.76:0/1769575794 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb013fd20 con 0x7fbdb00c4690 2023-11-09T03:45:53.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.020+0000 7fbda6ffd640 1 -- 172.21.15.76:0/1769575794 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbdac00bcf0 con 0x7fbdb00c4690 2023-11-09T03:45:53.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.020+0000 7f1b60eb2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b5c0adfd0 0x7f1b5c1ff120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.020+0000 7f1b60eb2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b5c0adfd0 0x7f1b5c1ff120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:33566/0 (socket says 172.21.15.76:33566) 2023-11-09T03:45:53.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.020+0000 7f1b60eb2640 1 -- 172.21.15.76:0/150826885 learned_addr learned my addr 172.21.15.76:0/150826885 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:53.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.020+0000 7f1b5bfff640 1 --2- 172.21.15.76:0/150826885 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b5c0b0030 0x7f1b5c0cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.020+0000 7fbda6ffd640 1 --2- 172.21.15.76:0/1769575794 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbd9406af40 0x7fbd9406d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.020+0000 7fbda6ffd640 1 -- 172.21.15.76:0/1769575794 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbdac08a1a0 con 0x7fbdb00c4690 2023-11-09T03:45:53.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.021+0000 7f1b60eb2640 1 -- 172.21.15.76:0/150826885 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b5c0b0030 msgr2=0x7f1b5c0cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.021+0000 7f1b60eb2640 1 --2- 172.21.15.76:0/150826885 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b5c0b0030 0x7f1b5c0cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.021+0000 7f1b60eb2640 1 -- 172.21.15.76:0/150826885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b5c158c50 msgr2=0x7f1b5c1e6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.021+0000 7f1b60eb2640 1 --2- 172.21.15.76:0/150826885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b5c158c50 0x7f1b5c1e6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.021+0000 7fbdb6473640 1 --2- 172.21.15.76:0/1769575794 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbd9406af40 0x7fbd9406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.021+0000 7f1b60eb2640 1 -- 172.21.15.76:0/150826885 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b50009400 con 0x7f1b5c0adfd0 2023-11-09T03:45:53.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.017+0000 7f96014ce640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f95fc0cc520 0x7f95fc0b1860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:60604/0 (socket says 172.21.15.76:60604) 2023-11-09T03:45:53.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.017+0000 7f96014ce640 1 -- 172.21.15.76:0/2014306608 learned_addr learned my addr 172.21.15.76:0/2014306608 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:53.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.018+0000 7f96024d0640 1 --2- 172.21.15.76:0/2014306608 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95fc070710 0x7f95fc070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.018+0000 7f9601ccf640 1 --2- 172.21.15.76:0/2014306608 >> v2:172.21.15.76:3300/0 conn(0x7f95fc164760 0x7f95fc0c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.018+0000 7f9600ccd640 1 -- 172.21.15.76:0/2014306608 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2738192805 0 0) 0x7f95fc071080 con 0x7f95fc0cc520 2023-11-09T03:45:53.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.018+0000 7f9600ccd640 1 -- 172.21.15.76:0/2014306608 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95e8003520 con 0x7f95fc0cc520 2023-11-09T03:45:53.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.018+0000 7f9600ccd640 1 -- 172.21.15.76:0/2014306608 <== mon.1 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 173704266 0 0) 0x7f95e8003520 con 0x7f95fc0cc520 2023-11-09T03:45:53.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.018+0000 7f9600ccd640 1 -- 172.21.15.76:0/2014306608 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95fc070710 msgr2=0x7f95fc070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:53.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.018+0000 7f9600ccd640 1 --2- 172.21.15.76:0/2014306608 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95fc070710 0x7f95fc070b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.018+0000 7f9600ccd640 1 -- 172.21.15.76:0/2014306608 >> v2:172.21.15.76:3300/0 conn(0x7f95fc164760 msgr2=0x7f95fc0c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:53.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.018+0000 7f9600ccd640 1 --2- 172.21.15.76:0/2014306608 >> v2:172.21.15.76:3300/0 conn(0x7f95fc164760 0x7f95fc0c4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.018+0000 7f9600ccd640 1 -- 172.21.15.76:0/2014306608 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95fc0ccf70 con 0x7f95fc0cc520 2023-11-09T03:45:53.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.018+0000 7f9600ccd640 1 -- 172.21.15.76:0/2014306608 <== mon.1 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f95f4001a10 con 0x7f95fc0cc520 2023-11-09T03:45:53.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.018+0000 7f9600ccd640 1 -- 172.21.15.76:0/2014306608 >> v1:172.21.15.113:6789/0 conn(0x7f95fc0cc520 legacy=0x7f95fc0b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.018+0000 7f9600ccd640 1 --2- 172.21.15.76:0/2014306608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95e8003970 0x7f95e8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.019+0000 7f9600ccd640 1 --2- 172.21.15.76:0/2014306608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95e80047c0 0x7f95e8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.019+0000 7f9600ccd640 1 --2- 172.21.15.76:0/2014306608 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95fc070710 0x7f95e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.019+0000 7f9600ccd640 1 -- 172.21.15.76:0/2014306608 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc0ccc90 con 0x7f95e8003970 2023-11-09T03:45:53.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.019+0000 7f9600ccd640 1 -- 172.21.15.76:0/2014306608 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e800b9e0 con 0x7f95e80047c0 2023-11-09T03:45:53.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.019+0000 7f9600ccd640 1 -- 172.21.15.76:0/2014306608 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e800bba0 con 0x7f95fc070710 2023-11-09T03:45:53.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.019+0000 7f96014ce640 1 --2- 172.21.15.76:0/2014306608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95e80047c0 0x7f95e8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.019+0000 7f96024d0640 1 --2- 172.21.15.76:0/2014306608 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95fc070710 0x7f95e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.019+0000 7f96014ce640 1 -- 172.21.15.76:0/2014306608 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95fc070710 msgr2=0x7f95e8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:53.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.019+0000 7f96014ce640 1 --2- 172.21.15.76:0/2014306608 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95fc070710 0x7f95e8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.019+0000 7f96014ce640 1 -- 172.21.15.76:0/2014306608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95e8003970 msgr2=0x7f95e8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:53.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.019+0000 7f96014ce640 1 --2- 172.21.15.76:0/2014306608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95e8003970 0x7f95e8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.019+0000 7f96014ce640 1 -- 172.21.15.76:0/2014306608 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95fc0ccf70 con 0x7f95e80047c0 2023-11-09T03:45:53.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.021+0000 7f96024d0640 1 --2- 172.21.15.76:0/2014306608 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95fc070710 0x7f95e8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:45:53.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.021+0000 7f96014ce640 1 --2- 172.21.15.76:0/2014306608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95e80047c0 0x7f95e8004c40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f95fc071210 tx=0x7f95e80057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0ec37c5e4d5e870 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:53.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.018+0000 7f2d33ccf640 1 -- start start 2023-11-09T03:45:53.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.020+0000 7f2d33ccf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d2c158c80 0x7f2d2c1548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.020+0000 7f2d33ccf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d2c159060 0x7f2d2c153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.020+0000 7f2d33ccf640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d2c15a070 0x7f2d2c0c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.020+0000 7f2d33ccf640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c0c7cb0 con 0x7f2d2c159060 2023-11-09T03:45:53.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.021+0000 7f2d33ccf640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c0c70e0 con 0x7f2d2c158c80 2023-11-09T03:45:53.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.021+0000 7f2d33ccf640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c155110 con 0x7f2d2c15a070 2023-11-09T03:45:53.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.021+0000 7f2d3226c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d2c15a070 0x7f2d2c0c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.021+0000 7f2d3226c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d2c15a070 0x7f2d2c0c80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:50768/0 (socket says 172.21.15.76:50768) 2023-11-09T03:45:53.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.021+0000 7f2d3226c640 1 -- 172.21.15.76:0/825111346 learned_addr learned my addr 172.21.15.76:0/825111346 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:53.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.021+0000 7f9600ccd640 1 -- 172.21.15.76:0/2014306608 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f4056630 con 0x7f95e80047c0 2023-11-09T03:45:53.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.021+0000 7f9600ccd640 1 -- 172.21.15.76:0/2014306608 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f4051070 con 0x7f95e80047c0 2023-11-09T03:45:53.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.021+0000 7f9600ccd640 1 -- 172.21.15.76:0/2014306608 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f4056a30 con 0x7f95e80047c0 2023-11-09T03:45:53.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.017+0000 7ff7d37fe640 1 --2- 172.21.15.76:0/611945235 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff7c006ae70 0x7ff7c006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.019+0000 7ff7d37fe640 1 --2- 172.21.15.76:0/611945235 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff7c006ae70 0x7ff7c006d330 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc047000 tx=0x7ff7cc030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:53.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.022+0000 7fbdb6473640 1 --2- 172.21.15.76:0/1769575794 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbd9406af40 0x7fbd9406d400 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbda0000f60 tx=0x7fbda0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:53.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.022+0000 7f2d3226c640 1 -- 172.21.15.76:0/825111346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d2c158c80 msgr2=0x7f2d2c1548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.022+0000 7f2d3226c640 1 --2- 172.21.15.76:0/825111346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d2c158c80 0x7f2d2c1548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.022+0000 7f2d3226c640 1 -- 172.21.15.76:0/825111346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d2c159060 msgr2=0x7f2d2c153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.022+0000 7f2d3226c640 1 --2- 172.21.15.76:0/825111346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d2c159060 0x7f2d2c153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.022+0000 7f2d3226c640 1 -- 172.21.15.76:0/825111346 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d28008550 con 0x7f2d2c15a070 2023-11-09T03:45:53.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.022+0000 7f2d3226c640 1 --2- 172.21.15.76:0/825111346 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d2c15a070 0x7f2d2c0c80d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400b970 tx=0x7f2d2400be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:53.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.022+0000 7f1b60eb2640 1 --2- 172.21.15.76:0/150826885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b5c0adfd0 0x7f1b5c1ff120 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c00c780 tx=0x7f1b4c00cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:53.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.022+0000 7f1b597fa640 1 -- 172.21.15.76:0/150826885 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c004c00 con 0x7f1b5c0adfd0 2023-11-09T03:45:53.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.022+0000 7f1b597fa640 1 -- 172.21.15.76:0/150826885 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4c004d80 con 0x7f1b5c0adfd0 2023-11-09T03:45:53.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.022+0000 7f1b63116640 1 -- 172.21.15.76:0/150826885 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b5c1e5740 con 0x7f1b5c0adfd0 2023-11-09T03:45:53.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.022+0000 7f9603f33640 1 -- 172.21.15.76:0/2014306608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95e80047c0 msgr2=0x7f95e8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.022+0000 7f9603f33640 1 --2- 172.21.15.76:0/2014306608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95e80047c0 0x7f95e8004c40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f95fc071210 tx=0x7f95e80057f0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.022+0000 7f1b597fa640 1 -- 172.21.15.76:0/150826885 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c007310 con 0x7f1b5c0adfd0 2023-11-09T03:45:53.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.023+0000 7f96024d0640 1 -- 172.21.15.76:0/2014306608 reap_dead start 2023-11-09T03:45:53.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.023+0000 7f9603f33640 1 -- 172.21.15.76:0/2014306608 shutdown_connections 2023-11-09T03:45:53.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.023+0000 7f9603f33640 1 -- 172.21.15.76:0/2014306608 >> 172.21.15.76:0/2014306608 conn(0x7f95fc0ce950 msgr2=0x7f95fc0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:53.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.023+0000 7fbda6ffd640 1 -- 172.21.15.76:0/1769575794 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbdac054f40 con 0x7fbdb00c4690 2023-11-09T03:45:53.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.023+0000 7f9603f33640 1 -- 172.21.15.76:0/2014306608 shutdown_connections 2023-11-09T03:45:53.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.023+0000 7f1b63116640 1 -- 172.21.15.76:0/150826885 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5c1e5d00 con 0x7f1b5c0adfd0 2023-11-09T03:45:53.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.023+0000 7f9603f33640 1 -- 172.21.15.76:0/2014306608 wait complete. 2023-11-09T03:45:53.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.024+0000 7f2d227fc640 1 -- 172.21.15.76:0/825111346 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2400cb40 con 0x7f2d2c15a070 2023-11-09T03:45:53.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.024+0000 7f2d227fc640 1 -- 172.21.15.76:0/825111346 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2400cca0 con 0x7f2d2c15a070 2023-11-09T03:45:53.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.025+0000 7f2d227fc640 1 -- 172.21.15.76:0/825111346 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d24014680 con 0x7f2d2c15a070 2023-11-09T03:45:53.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.025+0000 7f2d33ccf640 1 -- 172.21.15.76:0/825111346 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d2c0c7310 con 0x7f2d2c15a070 2023-11-09T03:45:53.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.026+0000 7f9603f33640 1 Processor -- start 2023-11-09T03:45:53.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.025+0000 7f2d33ccf640 1 -- 172.21.15.76:0/825111346 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d2c0c78d0 con 0x7f2d2c15a070 2023-11-09T03:45:53.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.026+0000 7f9603f33640 1 -- start start 2023-11-09T03:45:53.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.027+0000 7f1b597fa640 1 -- 172.21.15.76:0/150826885 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1b4c007470 con 0x7f1b5c0adfd0 2023-11-09T03:45:53.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.027+0000 7f1b597fa640 1 --2- 172.21.15.76:0/150826885 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1b4806ae70 0x7f1b4806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.027+0000 7f2d33ccf640 1 -- 172.21.15.76:0/825111346 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d2c0c7df0 con 0x7f2d2c15a070 2023-11-09T03:45:53.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.027+0000 7f1b597fa640 1 -- 172.21.15.76:0/150826885 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1b4c057e50 con 0x7f1b5c0adfd0 2023-11-09T03:45:53.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.028+0000 7f9603f33640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95fc0b3ad0 0x7f95fc0ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.028+0000 7f1b63116640 1 -- 172.21.15.76:0/150826885 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b3c005250 con 0x7f1b5c0adfd0 2023-11-09T03:45:53.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.028+0000 7f1b5bfff640 1 --2- 172.21.15.76:0/150826885 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1b4806ae70 0x7f1b4806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.028+0000 7f9603f33640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95fc14d480 0x7f95fc0aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.028+0000 7f9601ccf640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95fc0b3ad0 0x7f95fc0ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.028+0000 7f9601ccf640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95fc0b3ad0 0x7f95fc0ae1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:50782/0 (socket says 172.21.15.76:50782) 2023-11-09T03:45:53.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.029+0000 7f1b5bfff640 1 --2- 172.21.15.76:0/150826885 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1b4806ae70 0x7f1b4806d330 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1b50037000 tx=0x7f1b50008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:53.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.028+0000 7f9601ccf640 1 -- 172.21.15.76:0/8144477 learned_addr learned my addr 172.21.15.76:0/8144477 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:53.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.029+0000 7f2d227fc640 1 -- 172.21.15.76:0/825111346 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2d240147e0 con 0x7f2d2c15a070 2023-11-09T03:45:53.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.029+0000 7f96014ce640 1 --2- 172.21.15.76:0/8144477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95fc14d480 0x7f95fc0aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.029+0000 7f9603f33640 1 --2- 172.21.15.76:0/8144477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95fc0b3eb0 0x7f95fc0af500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.029+0000 7f9603f33640 1 -- 172.21.15.76:0/8144477 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc0ae710 con 0x7f95fc14d480 2023-11-09T03:45:53.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.029+0000 7f9603f33640 1 -- 172.21.15.76:0/8144477 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc14d960 con 0x7f95fc0b3eb0 2023-11-09T03:45:53.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.029+0000 7f96024d0640 1 --2- 172.21.15.76:0/8144477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95fc0b3eb0 0x7f95fc0af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.030+0000 7f2d227fc640 1 --2- 172.21.15.76:0/825111346 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2d1006aec0 0x7f2d1006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.030+0000 7f9603f33640 1 -- 172.21.15.76:0/8144477 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc14dc40 con 0x7f95fc0b3ad0 2023-11-09T03:45:53.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.030+0000 7f2d227fc640 1 -- 172.21.15.76:0/825111346 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2d240897c0 con 0x7f2d2c15a070 2023-11-09T03:45:53.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.030+0000 7f2d31a6b640 1 --2- 172.21.15.76:0/825111346 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2d1006aec0 0x7f2d1006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.030+0000 7f9601ccf640 1 -- 172.21.15.76:0/8144477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95fc0b3eb0 msgr2=0x7f95fc0af500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.031+0000 7f9601ccf640 1 --2- 172.21.15.76:0/8144477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95fc0b3eb0 0x7f95fc0af500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.031+0000 7f9601ccf640 1 -- 172.21.15.76:0/8144477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95fc14d480 msgr2=0x7f95fc0aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.031+0000 7f9601ccf640 1 --2- 172.21.15.76:0/8144477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95fc14d480 0x7f95fc0aefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.031+0000 7f9601ccf640 1 -- 172.21.15.76:0/8144477 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95fc14df20 con 0x7f95fc0b3ad0 2023-11-09T03:45:53.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.031+0000 7f2d31a6b640 1 --2- 172.21.15.76:0/825111346 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2d1006aec0 0x7f2d1006d380 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0096f0 tx=0x7f2d1c006c40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:53.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.031+0000 7f96014ce640 1 --2- 172.21.15.76:0/8144477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95fc14d480 0x7f95fc0aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:45:53.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.031+0000 7f9601ccf640 1 --2- 172.21.15.76:0/8144477 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95fc0b3ad0 0x7f95fc0ae1d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f95f800d830 tx=0x7f95f800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:53.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.033+0000 7f95f27fc640 1 -- 172.21.15.76:0/8144477 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f800b6c0 con 0x7f95fc0b3ad0 2023-11-09T03:45:53.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.033+0000 7f95f27fc640 1 -- 172.21.15.76:0/8144477 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f800b820 con 0x7f95fc0b3ad0 2023-11-09T03:45:53.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.033+0000 7f9603f33640 1 -- 172.21.15.76:0/8144477 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95fc14e200 con 0x7f95fc0b3ad0 2023-11-09T03:45:53.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.033+0000 7f2d227fc640 1 -- 172.21.15.76:0/825111346 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2d240546f0 con 0x7f2d2c15a070 2023-11-09T03:45:53.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.033+0000 7f9603f33640 1 -- 172.21.15.76:0/8144477 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95fc0afa40 con 0x7f95fc0b3ad0 2023-11-09T03:45:53.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.033+0000 7f1b597fa640 1 -- 172.21.15.76:0/150826885 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1b4c054220 con 0x7f1b5c0adfd0 2023-11-09T03:45:53.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.033+0000 7f95f27fc640 1 -- 172.21.15.76:0/8144477 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f800bbb0 con 0x7f95fc0b3ad0 2023-11-09T03:45:53.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.038+0000 7f95f27fc640 1 -- 172.21.15.76:0/8144477 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f95f800bd10 con 0x7f95fc0b3ad0 2023-11-09T03:45:53.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.039+0000 7f95dffff640 1 -- 172.21.15.76:0/8144477 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95fc14e440 con 0x7f95fc0b3ad0 2023-11-09T03:45:53.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.039+0000 7f95f27fc640 1 --2- 172.21.15.76:0/8144477 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f95e006aea0 0x7f95e006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.039+0000 7f95f27fc640 1 -- 172.21.15.76:0/8144477 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f95f8089da0 con 0x7f95fc0b3ad0 2023-11-09T03:45:53.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.039+0000 7f96014ce640 1 --2- 172.21.15.76:0/8144477 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f95e006aea0 0x7f95e006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.040+0000 7f96014ce640 1 --2- 172.21.15.76:0/8144477 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f95e006aea0 0x7f95e006d360 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f95f4056000 tx=0x7f95f4051000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:53.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.045+0000 7f95f27fc640 1 -- 172.21.15.76:0/8144477 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f95f8054ac0 con 0x7f95fc0b3ad0 2023-11-09T03:45:53.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.050+0000 7f0afb8da640 1 Processor -- start 2023-11-09T03:45:53.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.050+0000 7f0afb8da640 1 -- start start 2023-11-09T03:45:53.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.051+0000 7f0afb8da640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0af4153e60 0x7f0af415dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.052+0000 7f0afb8da640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af40cde60 con 0x7f0af40cd6d0 2023-11-09T03:45:53.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.052+0000 7f0afb8da640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af4160f90 con 0x7f0af40b4880 2023-11-09T03:45:53.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.052+0000 7f0afb8da640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af415e400 con 0x7f0af4153e60 2023-11-09T03:45:53.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.052+0000 7f0af8e75640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0af40cd6d0 0x7f0af4161420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33640/0 (socket says 172.21.15.76:33640) 2023-11-09T03:45:53.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.052+0000 7f0af8e75640 1 -- 172.21.15.76:0/426393731 learned_addr learned my addr 172.21.15.76:0/426393731 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:53.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.053+0000 7f0aebfff640 1 -- 172.21.15.76:0/426393731 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4273489991 0 0) 0x7f0af4160f90 con 0x7f0af40b4880 2023-11-09T03:45:53.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.053+0000 7f0aebfff640 1 -- 172.21.15.76:0/426393731 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ae0003520 con 0x7f0af40b4880 2023-11-09T03:45:53.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.053+0000 7f0aebfff640 1 -- 172.21.15.76:0/426393731 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 558181589 0 0) 0x7f0af40cde60 con 0x7f0af40cd6d0 2023-11-09T03:45:53.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.053+0000 7f0aebfff640 1 -- 172.21.15.76:0/426393731 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0af4160f90 con 0x7f0af40cd6d0 2023-11-09T03:45:53.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.053+0000 7f0aebfff640 1 -- 172.21.15.76:0/426393731 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2339943894 0 0) 0x7f0af4160f90 con 0x7f0af40cd6d0 2023-11-09T03:45:53.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.053+0000 7f0aebfff640 1 -- 172.21.15.76:0/426393731 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0af4153e60 msgr2=0x7f0af415dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:53.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.053+0000 7f0aebfff640 1 --2- 172.21.15.76:0/426393731 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0af4153e60 0x7f0af415dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.053+0000 7f0aebfff640 1 -- 172.21.15.76:0/426393731 >> v1:172.21.15.113:6789/0 conn(0x7f0af40b4880 legacy=0x7f0af40d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.053+0000 7f0aebfff640 1 -- 172.21.15.76:0/426393731 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af40ce090 con 0x7f0af40cd6d0 2023-11-09T03:45:53.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.053+0000 7f0aebfff640 1 -- 172.21.15.76:0/426393731 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f0af0001520 con 0x7f0af40cd6d0 2023-11-09T03:45:53.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.053+0000 7f0aebfff640 1 -- 172.21.15.76:0/426393731 >> v1:172.21.15.76:6789/0 conn(0x7f0af40cd6d0 legacy=0x7f0af4161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.053+0000 7f0aebfff640 1 --2- 172.21.15.76:0/426393731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ae0003b80 0x7f0ae0003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.053+0000 7f0aebfff640 1 --2- 172.21.15.76:0/426393731 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ae00047c0 0x7f0ae0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.053+0000 7f0aebfff640 1 --2- 172.21.15.76:0/426393731 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ae00052a0 0x7f0ae000ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.053+0000 7f0aebfff640 1 -- 172.21.15.76:0/426393731 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af415e400 con 0x7f0ae0003b80 2023-11-09T03:45:53.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.053+0000 7f0aebfff640 1 -- 172.21.15.76:0/426393731 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae000c220 con 0x7f0ae00052a0 2023-11-09T03:45:53.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.053+0000 7f0aebfff640 1 -- 172.21.15.76:0/426393731 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae000c3e0 con 0x7f0ae00047c0 2023-11-09T03:45:53.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.054+0000 7f0af9676640 1 --2- 172.21.15.76:0/426393731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ae0003b80 0x7f0ae0003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.054+0000 7f0af9676640 1 -- 172.21.15.76:0/426393731 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ae00047c0 msgr2=0x7f0ae0004c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-09T03:45:53.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.054+0000 7f0af9676640 1 --2- 172.21.15.76:0/426393731 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ae00047c0 0x7f0ae0004c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.055+0000 7f0af9676640 1 -- 172.21.15.76:0/426393731 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ae00052a0 msgr2=0x7f0ae000ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-09T03:45:53.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.055+0000 7f0af9676640 1 --2- 172.21.15.76:0/426393731 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ae00052a0 0x7f0ae000ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.055+0000 7f0af9676640 1 -- 172.21.15.76:0/426393731 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af4160f90 con 0x7f0ae0003b80 2023-11-09T03:45:53.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.056+0000 7f0af9676640 1 --2- 172.21.15.76:0/426393731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ae0003b80 0x7f0ae0003f80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0af40d3e70 tx=0x7f0ae00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d46974c15e1556 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:53.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.057+0000 7f0aebfff640 1 -- 172.21.15.76:0/426393731 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4032600 con 0x7f0ae0003b80 2023-11-09T03:45:53.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.057+0000 7f0aebfff640 1 -- 172.21.15.76:0/426393731 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae402e050 con 0x7f0ae0003b80 2023-11-09T03:45:53.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.058+0000 7f0afb8da640 1 -- 172.21.15.76:0/426393731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ae0003b80 msgr2=0x7f0ae0003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.058+0000 7f0afb8da640 1 --2- 172.21.15.76:0/426393731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ae0003b80 0x7f0ae0003f80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0af40d3e70 tx=0x7f0ae00057f0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.058+0000 7f0af9e77640 1 -- 172.21.15.76:0/426393731 reap_dead start 2023-11-09T03:45:53.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.058+0000 7f0afb8da640 1 -- 172.21.15.76:0/426393731 shutdown_connections 2023-11-09T03:45:53.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.058+0000 7f0afb8da640 1 -- 172.21.15.76:0/426393731 >> 172.21.15.76:0/426393731 conn(0x7f0af40cf880 msgr2=0x7f0af40cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:53.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.059+0000 7f0afb8da640 1 -- 172.21.15.76:0/426393731 shutdown_connections 2023-11-09T03:45:53.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.059+0000 7f0afb8da640 1 -- 172.21.15.76:0/426393731 wait complete. 2023-11-09T03:45:53.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.061+0000 7f0afb8da640 1 Processor -- start 2023-11-09T03:45:53.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.061+0000 7f0afb8da640 1 -- start start 2023-11-09T03:45:53.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.062+0000 7f0afb8da640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0af41523c0 0x7f0af41527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.062+0000 7f0afb8da640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0af40cd220 0x7f0af4156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.062+0000 7f0afb8da640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0af4157130 0x7f0af40d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.062+0000 7f0afb8da640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af41578b0 con 0x7f0af4157130 2023-11-09T03:45:53.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.062+0000 7f0afb8da640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af41575d0 con 0x7f0af40cd220 2023-11-09T03:45:53.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.062+0000 7f0afb8da640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af40ca050 con 0x7f0af41523c0 2023-11-09T03:45:53.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.062+0000 7f0af9e77640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0af4157130 0x7f0af40d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.062+0000 7f0af9e77640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0af4157130 0x7f0af40d1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36348/0 (socket says 172.21.15.76:36348) 2023-11-09T03:45:53.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.062+0000 7f0af9e77640 1 -- 172.21.15.76:0/4162132787 learned_addr learned my addr 172.21.15.76:0/4162132787 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:53.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.063+0000 7f0af8e75640 1 --2- 172.21.15.76:0/4162132787 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0af40cd220 0x7f0af4156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.063+0000 7f0af9e77640 1 -- 172.21.15.76:0/4162132787 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0af41523c0 msgr2=0x7f0af41527a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-09T03:45:53.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.063+0000 7f0af9e77640 1 --2- 172.21.15.76:0/4162132787 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0af41523c0 0x7f0af41527a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.063+0000 7f0af9e77640 1 -- 172.21.15.76:0/4162132787 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0af40cd220 msgr2=0x7f0af4156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.063+0000 7f0af9e77640 1 --2- 172.21.15.76:0/4162132787 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0af40cd220 0x7f0af4156bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.063+0000 7f0af9e77640 1 -- 172.21.15.76:0/4162132787 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af40ca330 con 0x7f0af4157130 2023-11-09T03:45:53.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.063+0000 7f0af9e77640 1 --2- 172.21.15.76:0/4162132787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0af4157130 0x7f0af40d1df0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0af40c6900 tx=0x7f0aec00cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:53.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.065+0000 7f0ae9ffb640 1 -- 172.21.15.76:0/4162132787 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec005d40 con 0x7f0af4157130 2023-11-09T03:45:53.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.065+0000 7f0afb8da640 1 -- 172.21.15.76:0/4162132787 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af40ca610 con 0x7f0af4157130 2023-11-09T03:45:53.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.065+0000 7f0afb8da640 1 -- 172.21.15.76:0/4162132787 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af40d2330 con 0x7f0af4157130 2023-11-09T03:45:53.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.066+0000 7f0ae9ffb640 1 -- 172.21.15.76:0/4162132787 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aec00a610 con 0x7f0af4157130 2023-11-09T03:45:53.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.066+0000 7f0ae9ffb640 1 -- 172.21.15.76:0/4162132787 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec00a9c0 con 0x7f0af4157130 2023-11-09T03:45:53.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.068+0000 7f0ae9ffb640 1 -- 172.21.15.76:0/4162132787 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0aec00ab20 con 0x7f0af4157130 2023-11-09T03:45:53.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.068+0000 7f0ae9ffb640 1 --2- 172.21.15.76:0/4162132787 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0ad806af40 0x7f0ad806d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:53.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.069+0000 7f0ae9ffb640 1 -- 172.21.15.76:0/4162132787 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0aec088de0 con 0x7f0af4157130 2023-11-09T03:45:53.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.069+0000 7f0af9676640 1 --2- 172.21.15.76:0/4162132787 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0ad806af40 0x7f0ad806d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:53.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.069+0000 7f0adf7fe640 1 -- 172.21.15.76:0/4162132787 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af40ca7a0 con 0x7f0af4157130 2023-11-09T03:45:53.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.079+0000 7f0ae9ffb640 1 -- 172.21.15.76:0/4162132787 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0aec017080 con 0x7f0af4157130 2023-11-09T03:45:53.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.081+0000 7f0af9676640 1 --2- 172.21.15.76:0/4162132787 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0ad806af40 0x7f0ad806d400 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4034000 tx=0x7f0ae4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:53.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.351+0000 7f1235644640 1 -- 172.21.15.76:0/2360603857 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f123006a4a0 con 0x7f123006cd70 2023-11-09T03:45:53.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.352+0000 7f12177fe640 1 -- 172.21.15.76:0/2360603857 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f122401e090 con 0x7f123006cd70 2023-11-09T03:45:53.352 INFO:teuthology.orchestra.run.smithi076.stdout:55834574857 2023-11-09T03:45:53.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.355+0000 7f12157fa640 1 -- 172.21.15.76:0/2360603857 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f121c06af40 msgr2=0x7f121c06d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.355+0000 7f12157fa640 1 --2- 172.21.15.76:0/2360603857 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f121c06af40 0x7f121c06d400 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1220036000 tx=0x7f1220030000 comp rx=0 tx=0).stop 2023-11-09T03:45:53.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.355+0000 7f12157fa640 1 -- 172.21.15.76:0/2360603857 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f123006cd70 msgr2=0x7f12300c6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.355+0000 7f12157fa640 1 --2- 172.21.15.76:0/2360603857 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f123006cd70 0x7f12300c6d90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f122400b480 tx=0x7f122400b940 comp rx=0 tx=0).stop 2023-11-09T03:45:53.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.356+0000 7f12157fa640 1 -- 172.21.15.76:0/2360603857 shutdown_connections 2023-11-09T03:45:53.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.356+0000 7f12157fa640 1 --2- 172.21.15.76:0/2360603857 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f121c06af40 0x7f121c06d400 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.356+0000 7f12157fa640 1 --2- 172.21.15.76:0/2360603857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f123015ce30 0x7f12301e50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.356+0000 7f12157fa640 1 --2- 172.21.15.76:0/2360603857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12301592c0 0x7f12300cbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.356+0000 7f12157fa640 1 --2- 172.21.15.76:0/2360603857 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f123006cd70 0x7f12300c6d90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.356+0000 7f12157fa640 1 -- 172.21.15.76:0/2360603857 >> 172.21.15.76:0/2360603857 conn(0x7f12300cf140 msgr2=0x7f12300cf520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:53.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.356+0000 7f12157fa640 1 -- 172.21.15.76:0/2360603857 shutdown_connections 2023-11-09T03:45:53.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.356+0000 7f12157fa640 1 -- 172.21.15.76:0/2360603857 wait complete. 2023-11-09T03:45:53.372 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.3 2023-11-09T03:45:53.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.378+0000 7fc9cebf4640 1 -- 172.21.15.76:0/2861498379 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fc9c0038380 con 0x7fc9c00e30b0 2023-11-09T03:45:53.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.379+0000 7fc9be7fc640 1 -- 172.21.15.76:0/2861498379 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc9c80f1370 con 0x7fc9c00e30b0 2023-11-09T03:45:53.378 INFO:teuthology.orchestra.run.smithi076.stdout:55834574856 2023-11-09T03:45:53.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.382+0000 7fc9cebf4640 1 -- 172.21.15.76:0/2861498379 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc9ac06aff0 msgr2=0x7fc9ac06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.382+0000 7fc9cebf4640 1 --2- 172.21.15.76:0/2861498379 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc9ac06aff0 0x7fc9ac06d4b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4006fd0 tx=0x7fc9c4008040 comp rx=0 tx=0).stop 2023-11-09T03:45:53.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.382+0000 7fc9cebf4640 1 -- 172.21.15.76:0/2861498379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9c00e30b0 msgr2=0x7fc9c00cf4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.382+0000 7fc9cebf4640 1 --2- 172.21.15.76:0/2861498379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9c00e30b0 0x7fc9c00cf4c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8000b60 tx=0x7fc9c8069f20 comp rx=0 tx=0).stop 2023-11-09T03:45:53.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.382+0000 7fc9cebf4640 1 -- 172.21.15.76:0/2861498379 shutdown_connections 2023-11-09T03:45:53.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.382+0000 7fc9cebf4640 1 --2- 172.21.15.76:0/2861498379 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc9ac06aff0 0x7fc9ac06d4b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.382+0000 7fc9cebf4640 1 --2- 172.21.15.76:0/2861498379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9c00e30b0 0x7fc9c00cf4c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.382+0000 7fc9cebf4640 1 --2- 172.21.15.76:0/2861498379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9c00e5980 0x7fc9c00e2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.382+0000 7fc9cebf4640 1 --2- 172.21.15.76:0/2861498379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9b4005180 0x7fc9c00e6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.382+0000 7fc9cebf4640 1 -- 172.21.15.76:0/2861498379 >> 172.21.15.76:0/2861498379 conn(0x7fc9c0059120 msgr2=0x7fc9c0039770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:53.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.382+0000 7fc9cebf4640 1 -- 172.21.15.76:0/2861498379 shutdown_connections 2023-11-09T03:45:53.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.382+0000 7fc9cebf4640 1 -- 172.21.15.76:0/2861498379 wait complete. 2023-11-09T03:45:53.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.391+0000 7fbdb86d7640 1 -- 172.21.15.76:0/1769575794 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbdb013f3d0 con 0x7fbdb00c4690 2023-11-09T03:45:53.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.392+0000 7fbda6ffd640 1 -- 172.21.15.76:0/1769575794 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbdb013f3d0 con 0x7fbdb00c4690 2023-11-09T03:45:53.391 INFO:teuthology.orchestra.run.smithi076.stdout:55834574855 2023-11-09T03:45:53.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.394+0000 7fbdb86d7640 1 -- 172.21.15.76:0/1769575794 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbd9406af40 msgr2=0x7fbd9406d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.394+0000 7fbdb86d7640 1 --2- 172.21.15.76:0/1769575794 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbd9406af40 0x7fbd9406d400 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbda0000f60 tx=0x7fbda0009040 comp rx=0 tx=0).stop 2023-11-09T03:45:53.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.394+0000 7fbdb86d7640 1 -- 172.21.15.76:0/1769575794 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdb00c4690 msgr2=0x7fbdb01623e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.394+0000 7fbdb86d7640 1 --2- 172.21.15.76:0/1769575794 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdb00c4690 0x7fbdb01623e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbdac00d800 tx=0x7fbdac00dcc0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.394+0000 7fbdb86d7640 1 -- 172.21.15.76:0/1769575794 shutdown_connections 2023-11-09T03:45:53.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.394+0000 7fbdb86d7640 1 --2- 172.21.15.76:0/1769575794 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbd9406af40 0x7fbd9406d400 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.394+0000 7fbdb86d7640 1 --2- 172.21.15.76:0/1769575794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdb0164760 0x7fbdb013dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.394+0000 7fbdb86d7640 1 --2- 172.21.15.76:0/1769575794 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdb00c4690 0x7fbdb01623e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.394+0000 7fbdb86d7640 1 --2- 172.21.15.76:0/1769575794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdb0070710 0x7fbdb014fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.394+0000 7fbdb86d7640 1 -- 172.21.15.76:0/1769575794 >> 172.21.15.76:0/1769575794 conn(0x7fbdb00ce950 msgr2=0x7fbdb0150420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:53.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.394+0000 7fbdb86d7640 1 -- 172.21.15.76:0/1769575794 shutdown_connections 2023-11-09T03:45:53.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.394+0000 7fbdb86d7640 1 -- 172.21.15.76:0/1769575794 wait complete. 2023-11-09T03:45:53.397 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.4 2023-11-09T03:45:53.405 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.0 2023-11-09T03:45:53.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.406+0000 7ff7b9ffb640 1 -- 172.21.15.76:0/611945235 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff7b40057d0 con 0x7ff7d4150360 2023-11-09T03:45:53.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.406+0000 7ff7bbfff640 1 -- 172.21.15.76:0/611945235 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff7c40584c0 con 0x7ff7d4150360 2023-11-09T03:45:53.406 INFO:teuthology.orchestra.run.smithi076.stdout:55834574856 2023-11-09T03:45:53.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.409+0000 7ff7b9ffb640 1 -- 172.21.15.76:0/611945235 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff7c006ae70 msgr2=0x7ff7c006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.409+0000 7ff7b9ffb640 1 --2- 172.21.15.76:0/611945235 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff7c006ae70 0x7ff7c006d330 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc047000 tx=0x7ff7cc030000 comp rx=0 tx=0).stop 2023-11-09T03:45:53.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.409+0000 7ff7b9ffb640 1 -- 172.21.15.76:0/611945235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7d4150360 msgr2=0x7ff7d40cf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.409+0000 7ff7b9ffb640 1 --2- 172.21.15.76:0/611945235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7d4150360 0x7ff7d40cf260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400ba00 tx=0x7ff7c400bec0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.409+0000 7ff7b9ffb640 1 -- 172.21.15.76:0/611945235 shutdown_connections 2023-11-09T03:45:53.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.409+0000 7ff7b9ffb640 1 --2- 172.21.15.76:0/611945235 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff7c006ae70 0x7ff7c006d330 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.409+0000 7ff7b9ffb640 1 --2- 172.21.15.76:0/611945235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7d4150360 0x7ff7d40cf260 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.409+0000 7ff7b9ffb640 1 --2- 172.21.15.76:0/611945235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7d4072370 0x7ff7d40c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.409+0000 7ff7b9ffb640 1 --2- 172.21.15.76:0/611945235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff7d4071f90 0x7ff7d414f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.409+0000 7ff7b9ffb640 1 -- 172.21.15.76:0/611945235 >> 172.21.15.76:0/611945235 conn(0x7ff7d406d800 msgr2=0x7ff7d415d440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:53.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.410+0000 7ff7b9ffb640 1 -- 172.21.15.76:0/611945235 shutdown_connections 2023-11-09T03:45:53.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.410+0000 7ff7b9ffb640 1 -- 172.21.15.76:0/611945235 wait complete. 2023-11-09T03:45:53.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.412+0000 7f1b63116640 1 -- 172.21.15.76:0/150826885 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f1b3c0057d0 con 0x7f1b5c0adfd0 2023-11-09T03:45:53.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.412+0000 7f1b597fa640 1 -- 172.21.15.76:0/150826885 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1b4c054220 con 0x7f1b5c0adfd0 2023-11-09T03:45:53.412 INFO:teuthology.orchestra.run.smithi076.stdout:55834574856 2023-11-09T03:45:53.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.414+0000 7f1b63116640 1 -- 172.21.15.76:0/150826885 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1b4806ae70 msgr2=0x7f1b4806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.414+0000 7f1b63116640 1 --2- 172.21.15.76:0/150826885 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1b4806ae70 0x7f1b4806d330 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1b50037000 tx=0x7f1b50008030 comp rx=0 tx=0).stop 2023-11-09T03:45:53.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.414+0000 7f1b63116640 1 -- 172.21.15.76:0/150826885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b5c0adfd0 msgr2=0x7f1b5c1ff120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.414+0000 7f1b63116640 1 --2- 172.21.15.76:0/150826885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b5c0adfd0 0x7f1b5c1ff120 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c00c780 tx=0x7f1b4c00cc40 comp rx=0 tx=0).stop 2023-11-09T03:45:53.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.414+0000 7f1b63116640 1 -- 172.21.15.76:0/150826885 shutdown_connections 2023-11-09T03:45:53.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.414+0000 7f1b63116640 1 --2- 172.21.15.76:0/150826885 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1b4806ae70 0x7f1b4806d330 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.414+0000 7f1b63116640 1 --2- 172.21.15.76:0/150826885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b5c158c50 0x7f1b5c1e6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.414+0000 7f1b63116640 1 --2- 172.21.15.76:0/150826885 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b5c0b0030 0x7f1b5c0cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.414+0000 7f1b63116640 1 --2- 172.21.15.76:0/150826885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b5c0adfd0 0x7f1b5c1ff120 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.414+0000 7f1b63116640 1 -- 172.21.15.76:0/150826885 >> 172.21.15.76:0/150826885 conn(0x7f1b5c06c7a0 msgr2=0x7f1b5c156c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:53.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.414+0000 7f1b63116640 1 -- 172.21.15.76:0/150826885 shutdown_connections 2023-11-09T03:45:53.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.414+0000 7f1b63116640 1 -- 172.21.15.76:0/150826885 wait complete. 2023-11-09T03:45:53.425 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.5 2023-11-09T03:45:53.426 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.6 2023-11-09T03:45:53.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.438+0000 7f2d33ccf640 1 -- 172.21.15.76:0/825111346 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2d2c0c7df0 con 0x7f2d2c15a070 2023-11-09T03:45:53.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.439+0000 7f2d227fc640 1 -- 172.21.15.76:0/825111346 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2d2c0c7df0 con 0x7f2d2c15a070 2023-11-09T03:45:53.438 INFO:teuthology.orchestra.run.smithi076.stdout:55834574855 2023-11-09T03:45:53.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.441+0000 7f2d33ccf640 1 -- 172.21.15.76:0/825111346 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2d1006aec0 msgr2=0x7f2d1006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.441+0000 7f2d33ccf640 1 --2- 172.21.15.76:0/825111346 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2d1006aec0 0x7f2d1006d380 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0096f0 tx=0x7f2d1c006c40 comp rx=0 tx=0).stop 2023-11-09T03:45:53.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.441+0000 7f2d33ccf640 1 -- 172.21.15.76:0/825111346 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d2c15a070 msgr2=0x7f2d2c0c80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.441+0000 7f2d33ccf640 1 --2- 172.21.15.76:0/825111346 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d2c15a070 0x7f2d2c0c80d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400b970 tx=0x7f2d2400be30 comp rx=0 tx=0).stop 2023-11-09T03:45:53.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.441+0000 7f2d33ccf640 1 -- 172.21.15.76:0/825111346 shutdown_connections 2023-11-09T03:45:53.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.441+0000 7f2d33ccf640 1 --2- 172.21.15.76:0/825111346 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2d1006aec0 0x7f2d1006d380 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.441+0000 7f2d33ccf640 1 --2- 172.21.15.76:0/825111346 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d2c15a070 0x7f2d2c0c80d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.441+0000 7f2d33ccf640 1 --2- 172.21.15.76:0/825111346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d2c159060 0x7f2d2c153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.441+0000 7f2d33ccf640 1 --2- 172.21.15.76:0/825111346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d2c158c80 0x7f2d2c1548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.441+0000 7f2d33ccf640 1 -- 172.21.15.76:0/825111346 >> 172.21.15.76:0/825111346 conn(0x7f2d2c06a120 msgr2=0x7f2d2c15ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:53.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.441+0000 7f2d33ccf640 1 -- 172.21.15.76:0/825111346 shutdown_connections 2023-11-09T03:45:53.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.441+0000 7f2d33ccf640 1 -- 172.21.15.76:0/825111346 wait complete. 2023-11-09T03:45:53.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.443+0000 7f95dffff640 1 -- 172.21.15.76:0/8144477 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f95fc14dd80 con 0x7f95fc0b3ad0 2023-11-09T03:45:53.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.443+0000 7f95f27fc640 1 -- 172.21.15.76:0/8144477 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f95fc14dd80 con 0x7f95fc0b3ad0 2023-11-09T03:45:53.443 INFO:teuthology.orchestra.run.smithi076.stdout:55834574856 2023-11-09T03:45:53.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.445+0000 7f95dffff640 1 -- 172.21.15.76:0/8144477 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f95e006aea0 msgr2=0x7f95e006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.445+0000 7f95dffff640 1 --2- 172.21.15.76:0/8144477 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f95e006aea0 0x7f95e006d360 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f95f4056000 tx=0x7f95f4051000 comp rx=0 tx=0).stop 2023-11-09T03:45:53.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.445+0000 7f95dffff640 1 -- 172.21.15.76:0/8144477 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95fc0b3ad0 msgr2=0x7f95fc0ae1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.445+0000 7f95dffff640 1 --2- 172.21.15.76:0/8144477 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95fc0b3ad0 0x7f95fc0ae1d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f95f800d830 tx=0x7f95f800dc30 comp rx=0 tx=0).stop 2023-11-09T03:45:53.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.445+0000 7f95dffff640 1 -- 172.21.15.76:0/8144477 shutdown_connections 2023-11-09T03:45:53.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.445+0000 7f95dffff640 1 --2- 172.21.15.76:0/8144477 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f95e006aea0 0x7f95e006d360 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.445+0000 7f95dffff640 1 --2- 172.21.15.76:0/8144477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95fc0b3eb0 0x7f95fc0af500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.445+0000 7f95dffff640 1 --2- 172.21.15.76:0/8144477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95fc14d480 0x7f95fc0aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.445+0000 7f95dffff640 1 --2- 172.21.15.76:0/8144477 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95fc0b3ad0 0x7f95fc0ae1d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.445+0000 7f95dffff640 1 -- 172.21.15.76:0/8144477 >> 172.21.15.76:0/8144477 conn(0x7f95fc0ce950 msgr2=0x7f95fc163a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:53.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.445+0000 7f95dffff640 1 -- 172.21.15.76:0/8144477 shutdown_connections 2023-11-09T03:45:53.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.445+0000 7f95dffff640 1 -- 172.21.15.76:0/8144477 wait complete. 2023-11-09T03:45:53.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.446+0000 7f0adf7fe640 1 -- 172.21.15.76:0/4162132787 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0af4074290 con 0x7f0af4157130 2023-11-09T03:45:53.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.446+0000 7f0ae9ffb640 1 -- 172.21.15.76:0/4162132787 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0aec053b80 con 0x7f0af4157130 2023-11-09T03:45:53.448 INFO:teuthology.orchestra.run.smithi076.stdout:55834574855 2023-11-09T03:45:53.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.448+0000 7f0adf7fe640 1 -- 172.21.15.76:0/4162132787 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0ad806af40 msgr2=0x7f0ad806d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.448+0000 7f0adf7fe640 1 --2- 172.21.15.76:0/4162132787 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0ad806af40 0x7f0ad806d400 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4034000 tx=0x7f0ae4008030 comp rx=0 tx=0).stop 2023-11-09T03:45:53.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.448+0000 7f0adf7fe640 1 -- 172.21.15.76:0/4162132787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0af4157130 msgr2=0x7f0af40d1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:53.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.448+0000 7f0adf7fe640 1 --2- 172.21.15.76:0/4162132787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0af4157130 0x7f0af40d1df0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0af40c6900 tx=0x7f0aec00cdc0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.448+0000 7f0adf7fe640 1 -- 172.21.15.76:0/4162132787 shutdown_connections 2023-11-09T03:45:53.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.448+0000 7f0adf7fe640 1 --2- 172.21.15.76:0/4162132787 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0ad806af40 0x7f0ad806d400 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.448+0000 7f0adf7fe640 1 --2- 172.21.15.76:0/4162132787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0af4157130 0x7f0af40d1df0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.448+0000 7f0adf7fe640 1 --2- 172.21.15.76:0/4162132787 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0af40cd220 0x7f0af4156bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.448+0000 7f0adf7fe640 1 --2- 172.21.15.76:0/4162132787 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0af41523c0 0x7f0af41527a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:53.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.448+0000 7f0adf7fe640 1 -- 172.21.15.76:0/4162132787 >> 172.21.15.76:0/4162132787 conn(0x7f0af40cf880 msgr2=0x7f0af40b71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:53.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.448+0000 7f0adf7fe640 1 -- 172.21.15.76:0/4162132787 shutdown_connections 2023-11-09T03:45:53.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:53.448+0000 7f0adf7fe640 1 -- 172.21.15.76:0/4162132787 wait complete. 2023-11-09T03:45:53.452 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.2 2023-11-09T03:45:53.457 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.7 2023-11-09T03:45:53.459 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.1 2023-11-09T03:45:54.373 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-09T03:45:54.397 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-09T03:45:54.406 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-09T03:45:54.426 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-09T03:45:54.426 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-09T03:45:54.453 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-09T03:45:54.458 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-09T03:45:54.460 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-09T03:45:54.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.507+0000 7f64d3f4c640 1 Processor -- start 2023-11-09T03:45:54.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.508+0000 7f64d3f4c640 1 -- start start 2023-11-09T03:45:54.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.509+0000 7f64d3f4c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f64cc14b060 0x7f64cc0ca060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.509+0000 7f64d3f4c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64cc06ac40 0x7f64cc0ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.509+0000 7f64d3f4c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64cc0cde10 con 0x7f64cc0caae0 2023-11-09T03:45:54.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.509+0000 7f64d3f4c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64cc0cd2d0 con 0x7f64cc14b060 2023-11-09T03:45:54.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.509+0000 7f64d3f4c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc0caf40 con 0x7f64cc06ac40 2023-11-09T03:45:54.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.510+0000 7f64d14e7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64cc06ac40 0x7f64cc0ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.510+0000 7f64d1ce8640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f64cc14b060 0x7f64cc0ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.510+0000 7f64d1ce8640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f64cc14b060 0x7f64cc0ca060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:33604/0 (socket says 172.21.15.76:33604) 2023-11-09T03:45:54.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.510+0000 7f64d1ce8640 1 -- 172.21.15.76:0/2100052029 learned_addr learned my addr 172.21.15.76:0/2100052029 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:54.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.510+0000 7f64d14e7640 1 -- 172.21.15.76:0/2100052029 >> v2:172.21.15.113:3300/0 conn(0x7f64cc14b060 msgr2=0x7f64cc0ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:54.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.510+0000 7f64d14e7640 1 --2- 172.21.15.76:0/2100052029 >> v2:172.21.15.113:3300/0 conn(0x7f64cc14b060 0x7f64cc0ca060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.510+0000 7f64d14e7640 1 -- 172.21.15.76:0/2100052029 >> v1:172.21.15.113:6789/0 conn(0x7f64cc0caae0 legacy=0x7f64cc164f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:54.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.510+0000 7f64d14e7640 1 -- 172.21.15.76:0/2100052029 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64cc14b440 con 0x7f64cc06ac40 2023-11-09T03:45:54.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.512+0000 7f64d14e7640 1 --2- 172.21.15.76:0/2100052029 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64cc06ac40 0x7f64cc0ca5a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f64c80097a0 tx=0x7f64c802b7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0c065638b81fbc1 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:54.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.512+0000 7f64d0ce6640 1 -- 172.21.15.76:0/2100052029 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8031710 con 0x7f64cc06ac40 2023-11-09T03:45:54.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.513+0000 7f64d0ce6640 1 -- 172.21.15.76:0/2100052029 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c802f050 con 0x7f64cc06ac40 2023-11-09T03:45:54.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.514+0000 7f64d3f4c640 1 -- 172.21.15.76:0/2100052029 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64cc06ac40 msgr2=0x7f64cc0ca5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.514+0000 7f64d3f4c640 1 --2- 172.21.15.76:0/2100052029 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64cc06ac40 0x7f64cc0ca5a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f64c80097a0 tx=0x7f64c802b7b0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.514+0000 7f64d3f4c640 1 -- 172.21.15.76:0/2100052029 shutdown_connections 2023-11-09T03:45:54.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.514+0000 7f64d3f4c640 1 --2- 172.21.15.76:0/2100052029 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64cc06ac40 0x7f64cc0ca5a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.514+0000 7f64d3f4c640 1 --2- 172.21.15.76:0/2100052029 >> v2:172.21.15.113:3300/0 conn(0x7f64cc14b060 0x7f64cc0ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.514+0000 7f64d3f4c640 1 -- 172.21.15.76:0/2100052029 >> 172.21.15.76:0/2100052029 conn(0x7f64cc06a120 msgr2=0x7f64cc06a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:54.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.514+0000 7f64d3f4c640 1 -- 172.21.15.76:0/2100052029 shutdown_connections 2023-11-09T03:45:54.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.514+0000 7f64d3f4c640 1 -- 172.21.15.76:0/2100052029 wait complete. 2023-11-09T03:45:54.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.516+0000 7f64d3f4c640 1 Processor -- start 2023-11-09T03:45:54.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.516+0000 7f64d3f4c640 1 -- start start 2023-11-09T03:45:54.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.517+0000 7f64d3f4c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64cc06ac40 0x7f64cc1f5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.517+0000 7f64d3f4c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64cc0caae0 0x7f64cc148d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.517+0000 7f64d3f4c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64cc14b060 0x7f64cc160d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.517+0000 7f64d3f4c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc1402c0 con 0x7f64cc14b060 2023-11-09T03:45:54.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.517+0000 7f64d3f4c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc148030 con 0x7f64cc0caae0 2023-11-09T03:45:54.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.517+0000 7f64d3f4c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc148310 con 0x7f64cc06ac40 2023-11-09T03:45:54.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.518+0000 7f64d1ce8640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64cc06ac40 0x7f64cc1f5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.518+0000 7f64d1ce8640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64cc06ac40 0x7f64cc1f5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:50824/0 (socket says 172.21.15.76:50824) 2023-11-09T03:45:54.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.518+0000 7f64d1ce8640 1 -- 172.21.15.76:0/3623276770 learned_addr learned my addr 172.21.15.76:0/3623276770 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:54.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.518+0000 7f64d14e7640 1 --2- 172.21.15.76:0/3623276770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64cc0caae0 0x7f64cc148d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.518+0000 7f64d1ce8640 1 -- 172.21.15.76:0/3623276770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64cc0caae0 msgr2=0x7f64cc148d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.518+0000 7f64d1ce8640 1 --2- 172.21.15.76:0/3623276770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64cc0caae0 0x7f64cc148d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.518+0000 7f64d1ce8640 1 -- 172.21.15.76:0/3623276770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64cc14b060 msgr2=0x7f64cc160d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.518+0000 7f64d24e9640 1 --2- 172.21.15.76:0/3623276770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64cc14b060 0x7f64cc160d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.518+0000 7f64d1ce8640 1 --2- 172.21.15.76:0/3623276770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64cc14b060 0x7f64cc160d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.519+0000 7f64d1ce8640 1 -- 172.21.15.76:0/3623276770 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c8009400 con 0x7f64cc06ac40 2023-11-09T03:45:54.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.519+0000 7f64d24e9640 1 --2- 172.21.15.76:0/3623276770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64cc14b060 0x7f64cc160d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:54.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.519+0000 7f64d1ce8640 1 --2- 172.21.15.76:0/3623276770 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64cc06ac40 0x7f64cc1f5720 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f64c400eed0 tx=0x7f64c400b480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:54.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.520+0000 7f64c27fc640 1 -- 172.21.15.76:0/3623276770 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4010040 con 0x7f64cc06ac40 2023-11-09T03:45:54.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.521+0000 7f64d3f4c640 1 -- 172.21.15.76:0/3623276770 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64cc1485f0 con 0x7f64cc06ac40 2023-11-09T03:45:54.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.521+0000 7f64d3f4c640 1 -- 172.21.15.76:0/3623276770 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64cc1f5c60 con 0x7f64cc06ac40 2023-11-09T03:45:54.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.523+0000 7f64c27fc640 1 -- 172.21.15.76:0/3623276770 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c40092d0 con 0x7f64cc06ac40 2023-11-09T03:45:54.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.524+0000 7f64c27fc640 1 -- 172.21.15.76:0/3623276770 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c400bdb0 con 0x7f64cc06ac40 2023-11-09T03:45:54.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.524+0000 7f64c27fc640 1 -- 172.21.15.76:0/3623276770 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f64c401d410 con 0x7f64cc06ac40 2023-11-09T03:45:54.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.524+0000 7f64c27fc640 1 --2- 172.21.15.76:0/3623276770 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f64b006afc0 0x7f64b006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.525+0000 7f64d14e7640 1 --2- 172.21.15.76:0/3623276770 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f64b006afc0 0x7f64b006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.525+0000 7f64c27fc640 1 -- 172.21.15.76:0/3623276770 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f64c4089c10 con 0x7f64cc06ac40 2023-11-09T03:45:54.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.526+0000 7f64d14e7640 1 --2- 172.21.15.76:0/3623276770 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f64b006afc0 0x7f64b006d480 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f64c8035000 tx=0x7f64c8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:54.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.526+0000 7f64d3f4c640 1 -- 172.21.15.76:0/3623276770 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64b4005250 con 0x7f64cc06ac40 2023-11-09T03:45:54.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.531+0000 7f64c27fc640 1 -- 172.21.15.76:0/3623276770 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f64c40549b0 con 0x7f64cc06ac40 2023-11-09T03:45:54.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.563+0000 7f5adb59e640 1 Processor -- start 2023-11-09T03:45:54.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.563+0000 7f5adb59e640 1 -- start start 2023-11-09T03:45:54.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.565+0000 7f5adb59e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5adc158ad0 0x7f5adc0c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.565+0000 7f5adb59e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5adc158eb0 0x7f5adc15c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.565+0000 7f5adb59e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5adc15ce40 0x7f5adc0ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.565+0000 7f5adb59e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5adc0c90b0 con 0x7f5adc158eb0 2023-11-09T03:45:54.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.565+0000 7f5adb59e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5adc0ca000 con 0x7f5adc15ce40 2023-11-09T03:45:54.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.565+0000 7f5adb59e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc15d3c0 con 0x7f5adc158ad0 2023-11-09T03:45:54.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.565+0000 7f5ad9d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5adc158eb0 0x7f5adc15c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.565+0000 7f5ad9d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5adc158eb0 0x7f5adc15c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36354/0 (socket says 172.21.15.76:36354) 2023-11-09T03:45:54.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.565+0000 7f5ad9d9b640 1 -- 172.21.15.76:0/346272896 learned_addr learned my addr 172.21.15.76:0/346272896 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:54.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.566+0000 7f5ada59c640 1 --2- 172.21.15.76:0/346272896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5adc158ad0 0x7f5adc0c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.566+0000 7f5ad9d9b640 1 -- 172.21.15.76:0/346272896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5adc158ad0 msgr2=0x7f5adc0c9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:54.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.566+0000 7f5ad9d9b640 1 --2- 172.21.15.76:0/346272896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5adc158ad0 0x7f5adc0c9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.566+0000 7f5ad9d9b640 1 -- 172.21.15.76:0/346272896 >> v2:172.21.15.113:3300/0 conn(0x7f5adc15ce40 msgr2=0x7f5adc0ce6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-09T03:45:54.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.566+0000 7f5adad9d640 1 --2- 172.21.15.76:0/346272896 >> v2:172.21.15.113:3300/0 conn(0x7f5adc15ce40 0x7f5adc0ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.567+0000 7f5ad9d9b640 1 --2- 172.21.15.76:0/346272896 >> v2:172.21.15.113:3300/0 conn(0x7f5adc15ce40 0x7f5adc0ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.567+0000 7f5ad9d9b640 1 -- 172.21.15.76:0/346272896 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5adc15d6a0 con 0x7f5adc158eb0 2023-11-09T03:45:54.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.568+0000 7fbec757c640 1 Processor -- start 2023-11-09T03:45:54.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.569+0000 7f5ad9d9b640 1 --2- 172.21.15.76:0/346272896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5adc158eb0 0x7f5adc15c900 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0b43d0 tx=0x7f5ad402e210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc1baa8a0846b808 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:54.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.569+0000 7f5adad9d640 1 --2- 172.21.15.76:0/346272896 >> v2:172.21.15.113:3300/0 conn(0x7f5adc15ce40 0x7f5adc0ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:54.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.569+0000 7f5ad959a640 1 -- 172.21.15.76:0/346272896 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4038400 con 0x7f5adc158eb0 2023-11-09T03:45:54.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.570+0000 7f5ad959a640 1 -- 172.21.15.76:0/346272896 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad4032070 con 0x7f5adc158eb0 2023-11-09T03:45:54.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.569+0000 7fbec757c640 1 -- start start 2023-11-09T03:45:54.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.570+0000 7fbec757c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbeb80df360 0x7fbeb8053090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.570+0000 7fbec757c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbeb803a8c0 0x7fbeb8037fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.570+0000 7fbec757c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb8053990 con 0x7fbeb8054a30 2023-11-09T03:45:54.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.570+0000 7fbec757c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb80df740 con 0x7fbeb803a8c0 2023-11-09T03:45:54.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.570+0000 7fbec757c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb8053c70 con 0x7fbeb80df360 2023-11-09T03:45:54.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.570+0000 7fbec5b19640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fbeb8054a30 0x7fbeb80384f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33654/0 (socket says 172.21.15.76:33654) 2023-11-09T03:45:54.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.570+0000 7fbec5b19640 1 -- 172.21.15.76:0/3006544757 learned_addr learned my addr 172.21.15.76:0/3006544757 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:54.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.570+0000 7fbec5318640 1 --2- 172.21.15.76:0/3006544757 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbeb80df360 0x7fbeb8053090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.570+0000 7fbec4b17640 1 --2- 172.21.15.76:0/3006544757 >> v2:172.21.15.76:3300/0 conn(0x7fbeb803a8c0 0x7fbeb8037fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.571+0000 7fbec5318640 1 -- 172.21.15.76:0/3006544757 >> v2:172.21.15.76:3300/0 conn(0x7fbeb803a8c0 msgr2=0x7fbeb8037fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:54.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.571+0000 7fbec5318640 1 --2- 172.21.15.76:0/3006544757 >> v2:172.21.15.76:3300/0 conn(0x7fbeb803a8c0 0x7fbeb8037fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.572+0000 7f5adb59e640 1 -- 172.21.15.76:0/346272896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5adc158eb0 msgr2=0x7f5adc15c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.572+0000 7f5adb59e640 1 --2- 172.21.15.76:0/346272896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5adc158eb0 0x7f5adc15c900 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0b43d0 tx=0x7f5ad402e210 comp rx=0 tx=0).stop 2023-11-09T03:45:54.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.571+0000 7fbec5318640 1 -- 172.21.15.76:0/3006544757 >> v1:172.21.15.76:6789/0 conn(0x7fbeb8054a30 legacy=0x7fbeb80384f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.571+0000 7fbec5318640 1 -- 172.21.15.76:0/3006544757 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb8053f50 con 0x7fbeb80df360 2023-11-09T03:45:54.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.572+0000 7fbec5318640 1 --2- 172.21.15.76:0/3006544757 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbeb80df360 0x7fbeb8053090 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00ba40 tx=0x7fbebc02df30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=359575f778927a4e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:54.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.572+0000 7fbeb7fff640 1 -- 172.21.15.76:0/3006544757 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc02ece0 con 0x7fbeb80df360 2023-11-09T03:45:54.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.572+0000 7f5adb59e640 1 -- 172.21.15.76:0/346272896 shutdown_connections 2023-11-09T03:45:54.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.572+0000 7f5adb59e640 1 --2- 172.21.15.76:0/346272896 >> v2:172.21.15.113:3300/0 conn(0x7f5adc15ce40 0x7f5adc0ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.572+0000 7f5adb59e640 1 --2- 172.21.15.76:0/346272896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5adc158eb0 0x7f5adc15c900 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.572+0000 7f5adb59e640 1 --2- 172.21.15.76:0/346272896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5adc158ad0 0x7f5adc0c9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.572+0000 7f5adb59e640 1 -- 172.21.15.76:0/346272896 >> 172.21.15.76:0/346272896 conn(0x7f5adc06a120 msgr2=0x7f5adc06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:54.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.572+0000 7f5adb59e640 1 -- 172.21.15.76:0/346272896 shutdown_connections 2023-11-09T03:45:54.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.572+0000 7fbeb7fff640 1 -- 172.21.15.76:0/3006544757 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbebc032070 con 0x7fbeb80df360 2023-11-09T03:45:54.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.573+0000 7fbec757c640 1 -- 172.21.15.76:0/3006544757 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbeb80df360 msgr2=0x7fbeb8053090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.573+0000 7fbec757c640 1 --2- 172.21.15.76:0/3006544757 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbeb80df360 0x7fbeb8053090 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00ba40 tx=0x7fbebc02df30 comp rx=0 tx=0).stop 2023-11-09T03:45:54.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.574+0000 7fbec757c640 1 -- 172.21.15.76:0/3006544757 shutdown_connections 2023-11-09T03:45:54.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.574+0000 7fbec757c640 1 --2- 172.21.15.76:0/3006544757 >> v2:172.21.15.76:3300/0 conn(0x7fbeb803a8c0 0x7fbeb8037fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.574+0000 7fbec757c640 1 --2- 172.21.15.76:0/3006544757 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbeb80df360 0x7fbeb8053090 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.574+0000 7fbec757c640 1 -- 172.21.15.76:0/3006544757 >> 172.21.15.76:0/3006544757 conn(0x7fbeb800a440 msgr2=0x7fbeb800a850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:54.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.572+0000 7f5adb59e640 1 -- 172.21.15.76:0/346272896 wait complete. 2023-11-09T03:45:54.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.574+0000 7fbec757c640 1 -- 172.21.15.76:0/3006544757 shutdown_connections 2023-11-09T03:45:54.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.575+0000 7f5adb59e640 1 Processor -- start 2023-11-09T03:45:54.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.575+0000 7f5adb59e640 1 -- start start 2023-11-09T03:45:54.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.574+0000 7fbec757c640 1 -- 172.21.15.76:0/3006544757 wait complete. 2023-11-09T03:45:54.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.576+0000 7fbec757c640 1 Processor -- start 2023-11-09T03:45:54.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.576+0000 7f5adb59e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5adc158ad0 0x7f5adc0c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.576+0000 7f5adb59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5adc158eb0 0x7f5adc15bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.576+0000 7f5ada59c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5adc158ad0 0x7f5adc0c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.576+0000 7f5ada59c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5adc158ad0 0x7f5adc0c4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:50852/0 (socket says 172.21.15.76:50852) 2023-11-09T03:45:54.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.577+0000 7f5ad9d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5adc158eb0 0x7f5adc15bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.577+0000 7f5ad9d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5adc158eb0 0x7f5adc15bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36368/0 (socket says 172.21.15.76:36368) 2023-11-09T03:45:54.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.577+0000 7f5ada59c640 1 -- 172.21.15.76:0/3001594257 learned_addr learned my addr 172.21.15.76:0/3001594257 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:54.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.577+0000 7f5adb59e640 1 --2- 172.21.15.76:0/3001594257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5adc15ce40 0x7f5adc1e2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.577+0000 7f5adb59e640 1 -- 172.21.15.76:0/3001594257 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc15c250 con 0x7f5adc158eb0 2023-11-09T03:45:54.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.577+0000 7f5adb59e640 1 -- 172.21.15.76:0/3001594257 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc15ac70 con 0x7f5adc15ce40 2023-11-09T03:45:54.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.577+0000 7f5adb59e640 1 -- 172.21.15.76:0/3001594257 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc15af50 con 0x7f5adc158ad0 2023-11-09T03:45:54.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.576+0000 7fbec757c640 1 -- start start 2023-11-09T03:45:54.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.577+0000 7fbec757c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbeb803a8c0 0x7fbeb80d2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.577+0000 7fbec757c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbeb8054a30 0x7fbeb8168220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.577+0000 7fbec757c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbeb80df360 0x7fbeb80dbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.578+0000 7fbec757c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb8036160 con 0x7fbeb80df360 2023-11-09T03:45:54.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.578+0000 7fbec5b19640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbeb80df360 0x7fbeb80dbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.578+0000 7fbec5b19640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbeb80df360 0x7fbeb80dbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36370/0 (socket says 172.21.15.76:36370) 2023-11-09T03:45:54.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.578+0000 7fbec5b19640 1 -- 172.21.15.76:0/586565732 learned_addr learned my addr 172.21.15.76:0/586565732 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:54.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.578+0000 7fbec5318640 1 --2- 172.21.15.76:0/586565732 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbeb803a8c0 0x7fbeb80d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.578+0000 7fbec757c640 1 -- 172.21.15.76:0/586565732 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb8035550 con 0x7fbeb8054a30 2023-11-09T03:45:54.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.578+0000 7f5ad9d9b640 1 -- 172.21.15.76:0/3001594257 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5adc158ad0 msgr2=0x7f5adc0c4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.578+0000 7f5ad9d9b640 1 --2- 172.21.15.76:0/3001594257 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5adc158ad0 0x7f5adc0c4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.578+0000 7f5ad9d9b640 1 -- 172.21.15.76:0/3001594257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5adc15ce40 msgr2=0x7f5adc1e2c70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-09T03:45:54.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.578+0000 7f5ad9d9b640 1 --2- 172.21.15.76:0/3001594257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5adc15ce40 0x7f5adc1e2c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.578+0000 7f5ad9d9b640 1 -- 172.21.15.76:0/3001594257 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad400d050 con 0x7f5adc158eb0 2023-11-09T03:45:54.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.578+0000 7fbec757c640 1 -- 172.21.15.76:0/586565732 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb8035830 con 0x7fbeb803a8c0 2023-11-09T03:45:54.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.579+0000 7fbec5b19640 1 -- 172.21.15.76:0/586565732 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbeb803a8c0 msgr2=0x7fbeb80d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.579+0000 7fbec5b19640 1 --2- 172.21.15.76:0/586565732 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbeb803a8c0 0x7fbeb80d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.579+0000 7fbec5b19640 1 -- 172.21.15.76:0/586565732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbeb8054a30 msgr2=0x7fbeb8168220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-09T03:45:54.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.579+0000 7fbec5b19640 1 --2- 172.21.15.76:0/586565732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbeb8054a30 0x7fbeb8168220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.579+0000 7f5ad9d9b640 1 --2- 172.21.15.76:0/3001594257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5adc158eb0 0x7f5adc15bd10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4036000 tx=0x7f5ad4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:54.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.579+0000 7fbec5b19640 1 -- 172.21.15.76:0/586565732 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbebc00b6a0 con 0x7fbeb80df360 2023-11-09T03:45:54.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.579+0000 7f5acaffd640 1 -- 172.21.15.76:0/3001594257 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4038400 con 0x7f5adc158eb0 2023-11-09T03:45:54.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.579+0000 7fbec5b19640 1 --2- 172.21.15.76:0/586565732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbeb80df360 0x7fbeb80dbde0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbec0073660 tx=0x7fbec0072860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:54.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.580+0000 7f5acaffd640 1 -- 172.21.15.76:0/3001594257 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad4008610 con 0x7f5adc158eb0 2023-11-09T03:45:54.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.580+0000 7f5acaffd640 1 -- 172.21.15.76:0/3001594257 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad40089c0 con 0x7f5adc158eb0 2023-11-09T03:45:54.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.580+0000 7fbeb5ffb640 1 -- 172.21.15.76:0/586565732 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0073050 con 0x7fbeb80df360 2023-11-09T03:45:54.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.580+0000 7fbec757c640 1 -- 172.21.15.76:0/586565732 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb8035b10 con 0x7fbeb80df360 2023-11-09T03:45:54.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.581+0000 7fbeb5ffb640 1 -- 172.21.15.76:0/586565732 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec00731f0 con 0x7fbeb80df360 2023-11-09T03:45:54.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.581+0000 7fbec757c640 1 -- 172.21.15.76:0/586565732 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb80d2a90 con 0x7fbeb80df360 2023-11-09T03:45:54.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.581+0000 7fbeb5ffb640 1 -- 172.21.15.76:0/586565732 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0093680 con 0x7fbeb80df360 2023-11-09T03:45:54.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.582+0000 7f5adb59e640 1 -- 172.21.15.76:0/3001594257 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5adc15b230 con 0x7f5adc158eb0 2023-11-09T03:45:54.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.582+0000 7f5adb59e640 1 -- 172.21.15.76:0/3001594257 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5adc15b7f0 con 0x7f5adc158eb0 2023-11-09T03:45:54.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.584+0000 7fbeb5ffb640 1 -- 172.21.15.76:0/586565732 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbec00937e0 con 0x7fbeb80df360 2023-11-09T03:45:54.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.585+0000 7f5acaffd640 1 -- 172.21.15.76:0/3001594257 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5ad4038560 con 0x7f5adc158eb0 2023-11-09T03:45:54.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.585+0000 7fbeb5ffb640 1 --2- 172.21.15.76:0/586565732 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbea406af40 0x7fbea406d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.585+0000 7fbeb5ffb640 1 -- 172.21.15.76:0/586565732 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbec0104220 con 0x7fbeb80df360 2023-11-09T03:45:54.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.585+0000 7f5acaffd640 1 --2- 172.21.15.76:0/3001594257 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5abc06af40 0x7f5abc06d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.586+0000 7f5acaffd640 1 -- 172.21.15.76:0/3001594257 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5ad40ac6b0 con 0x7f5adc158eb0 2023-11-09T03:45:54.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.586+0000 7f5ada59c640 1 --2- 172.21.15.76:0/3001594257 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5abc06af40 0x7f5abc06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.586+0000 7fbec5318640 1 --2- 172.21.15.76:0/586565732 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbea406af40 0x7fbea406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.587+0000 7fbec757c640 1 -- 172.21.15.76:0/586565732 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbea0005250 con 0x7fbeb80df360 2023-11-09T03:45:54.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.587+0000 7f5ada59c640 1 --2- 172.21.15.76:0/3001594257 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5abc06af40 0x7f5abc06d400 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5acc006fd0 tx=0x7f5acc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:54.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.587+0000 7fbec5318640 1 --2- 172.21.15.76:0/586565732 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbea406af40 0x7fbea406d400 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbebc039000 tx=0x7fbebc034000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:54.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.596+0000 7fbeb5ffb640 1 -- 172.21.15.76:0/586565732 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbec00cf050 con 0x7fbeb80df360 2023-11-09T03:45:54.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.597+0000 7f5adb59e640 1 -- 172.21.15.76:0/3001594257 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5adc15b9e0 con 0x7f5adc158eb0 2023-11-09T03:45:54.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.606+0000 7f5acaffd640 1 -- 172.21.15.76:0/3001594257 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ad4077450 con 0x7f5adc158eb0 2023-11-09T03:45:54.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.631+0000 7f35f294c640 1 Processor -- start 2023-11-09T03:45:54.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.631+0000 7f35f294c640 1 -- start start 2023-11-09T03:45:54.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.632+0000 7fba14c7c640 1 Processor -- start 2023-11-09T03:45:54.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.632+0000 7f35f294c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f35ec16e390 0x7f35ec169b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.632+0000 7f35f294c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f35ec070710 0x7f35ec070b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.632+0000 7f35f294c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35ec071560 con 0x7f35ec06d440 2023-11-09T03:45:54.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.632+0000 7f35f294c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35ec16df70 con 0x7f35ec070710 2023-11-09T03:45:54.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.632+0000 7f35f294c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35ec16a070 con 0x7f35ec16e390 2023-11-09T03:45:54.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.632+0000 7fb91b59e640 1 Processor -- start 2023-11-09T03:45:54.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.632+0000 7fb91b59e640 1 -- start start 2023-11-09T03:45:54.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.632+0000 7fba14c7c640 1 -- start start 2023-11-09T03:45:54.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.632+0000 7f35eb7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f35ec070710 0x7f35ec070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.633+0000 7f35eb7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f35ec070710 0x7f35ec070b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36384/0 (socket says 172.21.15.76:36384) 2023-11-09T03:45:54.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.633+0000 7f35eb7fe640 1 -- 172.21.15.76:0/118337042 learned_addr learned my addr 172.21.15.76:0/118337042 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:54.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.633+0000 7f35ebfff640 1 --2- 172.21.15.76:0/118337042 >> v2:172.21.15.113:3300/0 conn(0x7f35ec16e390 0x7f35ec169b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.633+0000 7f35ebfff640 1 -- 172.21.15.76:0/118337042 >> v2:172.21.15.76:3300/0 conn(0x7f35ec070710 msgr2=0x7f35ec070b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:54.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.633+0000 7f35ebfff640 1 --2- 172.21.15.76:0/118337042 >> v2:172.21.15.76:3300/0 conn(0x7f35ec070710 0x7f35ec070b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.633+0000 7f35ebfff640 1 -- 172.21.15.76:0/118337042 >> v1:172.21.15.113:6789/0 conn(0x7f35ec06d440 legacy=0x7f35ec0890e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:54.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.633+0000 7f35ebfff640 1 -- 172.21.15.76:0/118337042 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ec071280 con 0x7f35ec16e390 2023-11-09T03:45:54.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.634+0000 7fba14c7c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba08005ed0 0x7fba08007470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.634+0000 7fba14c7c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba080063d0 con 0x7fba08050270 2023-11-09T03:45:54.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.634+0000 7fba14c7c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba080066b0 con 0x7fba080df3f0 2023-11-09T03:45:54.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.634+0000 7fba14c7c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba080df7d0 con 0x7fba08005ed0 2023-11-09T03:45:54.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.634+0000 7fba0ed9d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba08005ed0 0x7fba08007470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.634+0000 7fb91b59e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb914050270 0x7fb914007470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.634+0000 7fba0ed9d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba08005ed0 0x7fba08007470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:50860/0 (socket says 172.21.15.76:50860) 2023-11-09T03:45:54.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.634+0000 7fba0ed9d640 1 -- 172.21.15.76:0/3294644621 learned_addr learned my addr 172.21.15.76:0/3294644621 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:54.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.634+0000 7fb91a59c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb9140df3f0 0x7fb91404fb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33672/0 (socket says 172.21.15.76:33672) 2023-11-09T03:45:54.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.635+0000 7faf830d7640 1 Processor -- start 2023-11-09T03:45:54.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.635+0000 7fb919d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb914050270 0x7fb914007470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.635+0000 7fba0ed9d640 1 -- 172.21.15.76:0/3294644621 >> v1:172.21.15.113:6789/0 conn(0x7fba080df3f0 legacy=0x7fba0804fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.635+0000 7fba0ed9d640 1 -- 172.21.15.76:0/3294644621 >> v1:172.21.15.76:6789/0 conn(0x7fba08050270 legacy=0x7fba080380c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.635+0000 7f35ebfff640 1 --2- 172.21.15.76:0/118337042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35ec16e390 0x7f35ec169b30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f35ec06c830 tx=0x7f35e002b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3586ca4ff27e2554 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:54.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.635+0000 7fba0ed9d640 1 -- 172.21.15.76:0/3294644621 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba08006990 con 0x7fba08005ed0 2023-11-09T03:45:54.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.635+0000 7fb919d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb914050270 0x7fb914007470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36396/0 (socket says 172.21.15.76:36396) 2023-11-09T03:45:54.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.635+0000 7fb91a59c640 1 -- 172.21.15.76:0/1336970080 learned_addr learned my addr 172.21.15.76:0/1336970080 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:54.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.635+0000 7ff4effff640 1 Processor -- start 2023-11-09T03:45:54.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.635+0000 7ff4effff640 1 -- start start 2023-11-09T03:45:54.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.635+0000 7fb91b59e640 1 --2- 172.21.15.76:0/1336970080 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb914005ed0 0x7fb914054150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.636+0000 7fb91b59e640 1 -- 172.21.15.76:0/1336970080 --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9140063d0 con 0x7fb9140df3f0 2023-11-09T03:45:54.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.636+0000 7fb91b59e640 1 -- 172.21.15.76:0/1336970080 --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9140df7d0 con 0x7fb914050270 2023-11-09T03:45:54.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.636+0000 7fb91ad9d640 1 --2- 172.21.15.76:0/1336970080 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb914005ed0 0x7fb914054150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.636+0000 7fb91959a640 1 -- 172.21.15.76:0/1336970080 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3714263292 0 0) 0x7fb9140063d0 con 0x7fb9140df3f0 2023-11-09T03:45:54.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.636+0000 7fb91b59e640 1 -- 172.21.15.76:0/1336970080 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9140066b0 con 0x7fb914005ed0 2023-11-09T03:45:54.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.635+0000 7faf830d7640 1 -- start start 2023-11-09T03:45:54.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.635+0000 7f35eaffd640 1 -- 172.21.15.76:0/118337042 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e00317c0 con 0x7f35ec16e390 2023-11-09T03:45:54.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.635+0000 7f35eaffd640 1 -- 172.21.15.76:0/118337042 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e0031920 con 0x7f35ec16e390 2023-11-09T03:45:54.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.635+0000 7f35eaffd640 1 -- 172.21.15.76:0/118337042 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e00317c0 con 0x7f35ec16e390 2023-11-09T03:45:54.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.636+0000 7fb91959a640 1 -- 172.21.15.76:0/1336970080 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb904003520 con 0x7fb9140df3f0 2023-11-09T03:45:54.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.636+0000 7fba0ed9d640 1 --2- 172.21.15.76:0/3294644621 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba08005ed0 0x7fba08007470 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fba100738a0 tx=0x7fba100a8130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9edd16a48e816932 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:54.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7fb91959a640 1 -- 172.21.15.76:0/1336970080 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3499674468 0 0) 0x7fb904003520 con 0x7fb9140df3f0 2023-11-09T03:45:54.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7fb91959a640 1 -- 172.21.15.76:0/1336970080 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb914005ed0 msgr2=0x7fb914054150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:54.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.636+0000 7ff4effff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff4f00af7f0 0x7ff4f00892f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.636+0000 7ff4effff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff4f0169bb0 0x7ff4f00afe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.636+0000 7ff4effff640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f0070990 con 0x7ff4f0070e20 2023-11-09T03:45:54.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.636+0000 7ff4effff640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4f0089bb0 con 0x7ff4f0169bb0 2023-11-09T03:45:54.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.636+0000 7ff4effff640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4f0070620 con 0x7ff4f00af7f0 2023-11-09T03:45:54.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.636+0000 7ff4e67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff4f0169bb0 0x7ff4f00afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.636+0000 7ff4ef7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff4f0070e20 0x7ff4f006df00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:60656/0 (socket says 172.21.15.76:60656) 2023-11-09T03:45:54.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.636+0000 7ff4ef7fe640 1 -- 172.21.15.76:0/271887021 learned_addr learned my addr 172.21.15.76:0/271887021 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:54.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.636+0000 7f35f294c640 1 -- 172.21.15.76:0/118337042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35ec16e390 msgr2=0x7f35ec169b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.636+0000 7f35f294c640 1 --2- 172.21.15.76:0/118337042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35ec16e390 0x7f35ec169b30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f35ec06c830 tx=0x7f35e002b860 comp rx=0 tx=0).stop 2023-11-09T03:45:54.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7fb91959a640 1 --2- 172.21.15.76:0/1336970080 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb914005ed0 0x7fb914054150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7fb91959a640 1 -- 172.21.15.76:0/1336970080 >> v2:172.21.15.76:3300/0 conn(0x7fb914050270 msgr2=0x7fb914007470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:54.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7f35f294c640 1 -- 172.21.15.76:0/118337042 shutdown_connections 2023-11-09T03:45:54.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7f35f294c640 1 --2- 172.21.15.76:0/118337042 >> v2:172.21.15.76:3300/0 conn(0x7f35ec070710 0x7f35ec070b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7f35f294c640 1 --2- 172.21.15.76:0/118337042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35ec16e390 0x7f35ec169b30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7f35f294c640 1 -- 172.21.15.76:0/118337042 >> 172.21.15.76:0/118337042 conn(0x7f35ec06a120 msgr2=0x7f35ec06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:54.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7f35f294c640 1 -- 172.21.15.76:0/118337042 shutdown_connections 2023-11-09T03:45:54.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7ff4ee7fc640 1 -- 172.21.15.76:0/271887021 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2729645619 0 0) 0x7ff4f0070990 con 0x7ff4f0070e20 2023-11-09T03:45:54.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7ff4ee7fc640 1 -- 172.21.15.76:0/271887021 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4dc003520 con 0x7ff4f0070e20 2023-11-09T03:45:54.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7f35f294c640 1 -- 172.21.15.76:0/118337042 wait complete. 2023-11-09T03:45:54.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7fba0d59a640 1 -- 172.21.15.76:0/3294644621 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba100a8d90 con 0x7fba08005ed0 2023-11-09T03:45:54.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7fba0d59a640 1 -- 172.21.15.76:0/3294644621 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba100ac070 con 0x7fba08005ed0 2023-11-09T03:45:54.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7ff4eeffd640 1 --2- 172.21.15.76:0/271887021 >> v2:172.21.15.113:3300/0 conn(0x7ff4f00af7f0 0x7ff4f00892f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7ff4ee7fc640 1 -- 172.21.15.76:0/271887021 <== mon.1 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 893371766 0 0) 0x7ff4dc003520 con 0x7ff4f0070e20 2023-11-09T03:45:54.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7ff4ee7fc640 1 -- 172.21.15.76:0/271887021 >> v2:172.21.15.113:3300/0 conn(0x7ff4f00af7f0 msgr2=0x7ff4f00892f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:54.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7ff4ee7fc640 1 --2- 172.21.15.76:0/271887021 >> v2:172.21.15.113:3300/0 conn(0x7ff4f00af7f0 0x7ff4f00892f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7ff4ee7fc640 1 -- 172.21.15.76:0/271887021 >> v2:172.21.15.76:3300/0 conn(0x7ff4f0169bb0 msgr2=0x7ff4f00afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:54.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7ff4ee7fc640 1 --2- 172.21.15.76:0/271887021 >> v2:172.21.15.76:3300/0 conn(0x7ff4f0169bb0 0x7ff4f00afe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7ff4ee7fc640 1 -- 172.21.15.76:0/271887021 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f00b0350 con 0x7ff4f0070e20 2023-11-09T03:45:54.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7ff4ee7fc640 1 -- 172.21.15.76:0/271887021 <== mon.1 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7ff4e80019f0 con 0x7ff4f0070e20 2023-11-09T03:45:54.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7ff4ee7fc640 1 -- 172.21.15.76:0/271887021 >> v1:172.21.15.113:6789/0 conn(0x7ff4f0070e20 legacy=0x7ff4f006df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.638+0000 7ff4ee7fc640 1 --2- 172.21.15.76:0/271887021 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4dc0039c0 0x7ff4dc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.638+0000 7ff4ee7fc640 1 --2- 172.21.15.76:0/271887021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc0047c0 0x7ff4dc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.638+0000 7fb91959a640 1 --2- 172.21.15.76:0/1336970080 >> v2:172.21.15.76:3300/0 conn(0x7fb914050270 0x7fb914007470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.638+0000 7fb91959a640 1 -- 172.21.15.76:0/1336970080 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb914006990 con 0x7fb9140df3f0 2023-11-09T03:45:54.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.638+0000 7fb91959a640 1 -- 172.21.15.76:0/1336970080 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fb91c073cb0 con 0x7fb9140df3f0 2023-11-09T03:45:54.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.638+0000 7fb91959a640 1 -- 172.21.15.76:0/1336970080 >> v1:172.21.15.76:6789/0 conn(0x7fb9140df3f0 legacy=0x7fb91404fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.638+0000 7fb91959a640 1 --2- 172.21.15.76:0/1336970080 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb904003970 0x7fb904003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.638+0000 7fb91959a640 1 --2- 172.21.15.76:0/1336970080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb904004310 0x7fb9040047e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.638+0000 7fb91959a640 1 --2- 172.21.15.76:0/1336970080 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb904005020 0x7fb90400b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.638+0000 7fb91959a640 1 -- 172.21.15.76:0/1336970080 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9140df7d0 con 0x7fb904003970 2023-11-09T03:45:54.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.638+0000 7fb91959a640 1 -- 172.21.15.76:0/1336970080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9140066b0 con 0x7fb904004310 2023-11-09T03:45:54.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.638+0000 7fb91959a640 1 -- 172.21.15.76:0/1336970080 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb90400bf60 con 0x7fb904005020 2023-11-09T03:45:54.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.638+0000 7fb91ad9d640 1 --2- 172.21.15.76:0/1336970080 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb914005ed0 0x7fb914054150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:54.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.638+0000 7fb919d9b640 1 --2- 172.21.15.76:0/1336970080 >> v2:172.21.15.76:3300/0 conn(0x7fb914050270 0x7fb914007470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:45:54.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.638+0000 7ff4ee7fc640 1 --2- 172.21.15.76:0/271887021 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4dc005180 0x7ff4dc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.638+0000 7fb919d9b640 1 --2- 172.21.15.76:0/1336970080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb904004310 0x7fb9040047e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.638+0000 7ff4e67fc640 1 --2- 172.21.15.76:0/271887021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc0047c0 0x7ff4dc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.638+0000 7ff4eeffd640 1 --2- 172.21.15.76:0/271887021 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4dc0039c0 0x7ff4dc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.638+0000 7ff4ee7fc640 1 -- 172.21.15.76:0/271887021 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f0089bb0 con 0x7ff4dc005180 2023-11-09T03:45:54.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.638+0000 7ff4ee7fc640 1 -- 172.21.15.76:0/271887021 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f0070620 con 0x7ff4dc0047c0 2023-11-09T03:45:54.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.638+0000 7ff4ee7fc640 1 -- 172.21.15.76:0/271887021 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc00c000 con 0x7ff4dc0039c0 2023-11-09T03:45:54.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.638+0000 7fb91a59c640 1 --2- 172.21.15.76:0/1336970080 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb904003970 0x7fb904003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.638+0000 7ff4ef7fe640 1 --2- 172.21.15.76:0/271887021 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4dc005180 0x7ff4dc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.638+0000 7f35f294c640 1 Processor -- start 2023-11-09T03:45:54.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.636+0000 7faf830d7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf7403c2c0 0x7faf74038260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.636+0000 7faf830d7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faf74058b00 0x7faf740580f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.636+0000 7faf830d7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf74053af0 con 0x7faf740e7930 2023-11-09T03:45:54.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.636+0000 7faf830d7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf74053810 con 0x7faf74058b00 2023-11-09T03:45:54.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.636+0000 7faf830d7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf74054270 con 0x7faf7403c2c0 2023-11-09T03:45:54.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.639+0000 7fba14c7c640 1 -- 172.21.15.76:0/3294644621 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba08005ed0 msgr2=0x7fba08007470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.639+0000 7fba14c7c640 1 --2- 172.21.15.76:0/3294644621 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba08005ed0 0x7fba08007470 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fba100738a0 tx=0x7fba100a8130 comp rx=0 tx=0).stop 2023-11-09T03:45:54.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7faf818d4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf7403c2c0 0x7faf74038260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7faf820d5640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7faf740e7930 0x7faf740579e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33674/0 (socket says 172.21.15.76:33674) 2023-11-09T03:45:54.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7faf820d5640 1 -- 172.21.15.76:0/3762747345 learned_addr learned my addr 172.21.15.76:0/3762747345 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:54.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7faf818d4640 1 -- 172.21.15.76:0/3762747345 >> v2:172.21.15.76:3300/0 conn(0x7faf74058b00 msgr2=0x7faf740580f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T03:45:54.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7faf818d4640 1 --2- 172.21.15.76:0/3762747345 >> v2:172.21.15.76:3300/0 conn(0x7faf74058b00 0x7faf740580f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7faf818d4640 1 -- 172.21.15.76:0/3762747345 >> v1:172.21.15.76:6789/0 conn(0x7faf740e7930 legacy=0x7faf740579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.637+0000 7faf818d4640 1 -- 172.21.15.76:0/3762747345 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf74054550 con 0x7faf7403c2c0 2023-11-09T03:45:54.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.639+0000 7ff4eeffd640 1 -- 172.21.15.76:0/271887021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc0047c0 msgr2=0x7ff4dc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:54.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.639+0000 7ff4eeffd640 1 --2- 172.21.15.76:0/271887021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc0047c0 0x7ff4dc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.639+0000 7ff4eeffd640 1 -- 172.21.15.76:0/271887021 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4dc005180 msgr2=0x7ff4dc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:54.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.639+0000 7ff4eeffd640 1 --2- 172.21.15.76:0/271887021 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4dc005180 0x7ff4dc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.639+0000 7ff4eeffd640 1 -- 172.21.15.76:0/271887021 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4dc003520 con 0x7ff4dc0039c0 2023-11-09T03:45:54.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.639+0000 7ff4ef7fe640 1 -- 172.21.15.76:0/271887021 reap_dead start 2023-11-09T03:45:54.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.639+0000 7ff4e67fc640 1 --2- 172.21.15.76:0/271887021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc0047c0 0x7ff4dc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:45:54.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.639+0000 7f35f294c640 1 -- start start 2023-11-09T03:45:54.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.639+0000 7fba14c7c640 1 -- 172.21.15.76:0/3294644621 shutdown_connections 2023-11-09T03:45:54.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.639+0000 7fba14c7c640 1 --2- 172.21.15.76:0/3294644621 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba08005ed0 0x7fba08007470 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.639+0000 7fba14c7c640 1 -- 172.21.15.76:0/3294644621 >> 172.21.15.76:0/3294644621 conn(0x7fba08054b00 msgr2=0x7fba08054f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:54.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.639+0000 7fba14c7c640 1 -- 172.21.15.76:0/3294644621 shutdown_connections 2023-11-09T03:45:54.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.639+0000 7fba14c7c640 1 -- 172.21.15.76:0/3294644621 wait complete. 2023-11-09T03:45:54.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.639+0000 7fb91a59c640 1 -- 172.21.15.76:0/1336970080 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb904005020 msgr2=0x7fb90400b970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-09T03:45:54.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.640+0000 7faf818d4640 1 --2- 172.21.15.76:0/3762747345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf7403c2c0 0x7faf74038260 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faf6c0097a0 tx=0x7faf6c02b680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11cdfaf28cf4f583 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:54.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.639+0000 7fb91a59c640 1 --2- 172.21.15.76:0/1336970080 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb904005020 0x7fb90400b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.639+0000 7fb91a59c640 1 -- 172.21.15.76:0/1336970080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb904004310 msgr2=0x7fb9040047e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:54.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.640+0000 7ff4eeffd640 1 --2- 172.21.15.76:0/271887021 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4dc0039c0 0x7ff4dc003de0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000ba30 tx=0x7ff4dc0057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e246c2759e1e7ad server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:54.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.639+0000 7fb91a59c640 1 --2- 172.21.15.76:0/1336970080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb904004310 0x7fb9040047e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.639+0000 7fb91a59c640 1 -- 172.21.15.76:0/1336970080 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb914006990 con 0x7fb904003970 2023-11-09T03:45:54.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.640+0000 7f35f294c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35ec06d440 0x7f35ec1771f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.640+0000 7faf810d3640 1 -- 172.21.15.76:0/3762747345 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6c030710 con 0x7faf7403c2c0 2023-11-09T03:45:54.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.640+0000 7faf810d3640 1 -- 172.21.15.76:0/3762747345 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6c02e050 con 0x7faf7403c2c0 2023-11-09T03:45:54.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.640+0000 7f35f294c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35ec070710 0x7f35ec165480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.640+0000 7f35f294c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35ec16e390 0x7f35ec1659c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.640+0000 7f35f294c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec16a930 con 0x7f35ec06d440 2023-11-09T03:45:54.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.640+0000 7f35f294c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec166fa0 con 0x7f35ec16e390 2023-11-09T03:45:54.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.640+0000 7f35f0ee9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35ec16e390 0x7f35ec1659c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.640+0000 7f35f0ee9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35ec16e390 0x7f35ec1659c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:33676/0 (socket says 172.21.15.76:33676) 2023-11-09T03:45:54.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.640+0000 7f35f0ee9640 1 -- 172.21.15.76:0/2034889537 learned_addr learned my addr 172.21.15.76:0/2034889537 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:54.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.641+0000 7f35eb7fe640 1 --2- 172.21.15.76:0/2034889537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35ec070710 0x7f35ec165480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.641+0000 7f35ebfff640 1 --2- 172.21.15.76:0/2034889537 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35ec06d440 0x7f35ec1771f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.641+0000 7f35f294c640 1 -- 172.21.15.76:0/2034889537 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec16a2e0 con 0x7f35ec070710 2023-11-09T03:45:54.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.641+0000 7fba14c7c640 1 Processor -- start 2023-11-09T03:45:54.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.640+0000 7ff4ee7fc640 1 -- 172.21.15.76:0/271887021 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0033440 con 0x7ff4dc0039c0 2023-11-09T03:45:54.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.640+0000 7ff4ee7fc640 1 -- 172.21.15.76:0/271887021 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e00335a0 con 0x7ff4dc0039c0 2023-11-09T03:45:54.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.640+0000 7ff4ee7fc640 1 -- 172.21.15.76:0/271887021 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0033970 con 0x7ff4dc0039c0 2023-11-09T03:45:54.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.641+0000 7ff4effff640 1 -- 172.21.15.76:0/271887021 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4dc0039c0 msgr2=0x7ff4dc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.641+0000 7ff4effff640 1 --2- 172.21.15.76:0/271887021 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4dc0039c0 0x7ff4dc003de0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000ba30 tx=0x7ff4dc0057f0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.641+0000 7f35eb7fe640 1 -- 172.21.15.76:0/2034889537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35ec16e390 msgr2=0x7f35ec1659c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.641+0000 7f35eb7fe640 1 --2- 172.21.15.76:0/2034889537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35ec16e390 0x7f35ec1659c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.641+0000 7f35eb7fe640 1 -- 172.21.15.76:0/2034889537 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35ec06d440 msgr2=0x7f35ec1771f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.641+0000 7f35eb7fe640 1 --2- 172.21.15.76:0/2034889537 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35ec06d440 0x7f35ec1771f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.641+0000 7f35eb7fe640 1 -- 172.21.15.76:0/2034889537 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e0009400 con 0x7f35ec070710 2023-11-09T03:45:54.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.641+0000 7faf830d7640 1 -- 172.21.15.76:0/3762747345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf7403c2c0 msgr2=0x7faf74038260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.641+0000 7faf830d7640 1 --2- 172.21.15.76:0/3762747345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf7403c2c0 0x7faf74038260 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faf6c0097a0 tx=0x7faf6c02b680 comp rx=0 tx=0).stop 2023-11-09T03:45:54.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.641+0000 7ff4effff640 1 -- 172.21.15.76:0/271887021 shutdown_connections 2023-11-09T03:45:54.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.641+0000 7ff4effff640 1 --2- 172.21.15.76:0/271887021 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4dc0039c0 0x7ff4dc003de0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.641+0000 7ff4effff640 1 -- 172.21.15.76:0/271887021 >> 172.21.15.76:0/271887021 conn(0x7ff4f006c780 msgr2=0x7ff4f006cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:54.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.641+0000 7fba14c7c640 1 -- start start 2023-11-09T03:45:54.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.641+0000 7ff4effff640 1 -- 172.21.15.76:0/271887021 shutdown_connections 2023-11-09T03:45:54.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.641+0000 7f35eb7fe640 1 --2- 172.21.15.76:0/2034889537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35ec070710 0x7f35ec165480 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00e850 tx=0x7f35dc00ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:54.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.641+0000 7fb91a59c640 1 --2- 172.21.15.76:0/1336970080 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb904003970 0x7fb904003d90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb91c000b60 tx=0x7fb9040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e9684bfbd76112 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:54.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.642+0000 7fb91959a640 1 -- 172.21.15.76:0/1336970080 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91c0a8540 con 0x7fb904003970 2023-11-09T03:45:54.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.642+0000 7fb91959a640 1 -- 172.21.15.76:0/1336970080 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb91c0a8700 con 0x7fb904003970 2023-11-09T03:45:54.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.642+0000 7fb91959a640 1 -- 172.21.15.76:0/1336970080 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91c0a8b70 con 0x7fb904003970 2023-11-09T03:45:54.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.641+0000 7faf830d7640 1 -- 172.21.15.76:0/3762747345 shutdown_connections 2023-11-09T03:45:54.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.641+0000 7faf830d7640 1 --2- 172.21.15.76:0/3762747345 >> v2:172.21.15.76:3300/0 conn(0x7faf74058b00 0x7faf740580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.641+0000 7faf830d7640 1 --2- 172.21.15.76:0/3762747345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf7403c2c0 0x7faf74038260 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.641+0000 7faf830d7640 1 -- 172.21.15.76:0/3762747345 >> 172.21.15.76:0/3762747345 conn(0x7faf74059120 msgr2=0x7faf74059530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:54.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.641+0000 7faf830d7640 1 -- 172.21.15.76:0/3762747345 shutdown_connections 2023-11-09T03:45:54.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.642+0000 7faf830d7640 1 -- 172.21.15.76:0/3762747345 wait complete. 2023-11-09T03:45:54.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.642+0000 7f35e8ff9640 1 -- 172.21.15.76:0/2034889537 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc00c940 con 0x7f35ec070710 2023-11-09T03:45:54.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.642+0000 7f35e8ff9640 1 -- 172.21.15.76:0/2034889537 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35dc00caa0 con 0x7f35ec070710 2023-11-09T03:45:54.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.643+0000 7fb91b59e640 1 -- 172.21.15.76:0/1336970080 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb904003970 msgr2=0x7fb904003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.643+0000 7f35e8ff9640 1 -- 172.21.15.76:0/2034889537 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc00ce10 con 0x7f35ec070710 2023-11-09T03:45:54.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.643+0000 7fba14c7c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba08050270 0x7fba08048610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.643+0000 7f35f294c640 1 -- 172.21.15.76:0/2034889537 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35ec16a5c0 con 0x7f35ec070710 2023-11-09T03:45:54.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.643+0000 7fba14c7c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba080df3f0 0x7fba0804d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.643+0000 7fb91b59e640 1 --2- 172.21.15.76:0/1336970080 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb904003970 0x7fb904003d90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb91c000b60 tx=0x7fb9040057f0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.643+0000 7fb91ad9d640 1 -- 172.21.15.76:0/1336970080 reap_dead start 2023-11-09T03:45:54.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.643+0000 7fba14c7c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba08180510 0x7fba081809a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.643+0000 7f35f294c640 1 -- 172.21.15.76:0/2034889537 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ec177730 con 0x7f35ec070710 2023-11-09T03:45:54.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.643+0000 7fba14c7c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba08048b50 con 0x7fba08050270 2023-11-09T03:45:54.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.644+0000 7fb91b59e640 1 -- 172.21.15.76:0/1336970080 shutdown_connections 2023-11-09T03:45:54.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.644+0000 7fb91b59e640 1 -- 172.21.15.76:0/1336970080 >> 172.21.15.76:0/1336970080 conn(0x7fb914054b00 msgr2=0x7fb914054f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:54.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.644+0000 7fba0dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba080df3f0 0x7fba0804d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.644+0000 7fba0dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba080df3f0 0x7fba0804d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:33694/0 (socket says 172.21.15.76:33694) 2023-11-09T03:45:54.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.644+0000 7fba0dd9b640 1 -- 172.21.15.76:0/2318425244 learned_addr learned my addr 172.21.15.76:0/2318425244 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:54.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.644+0000 7fba0e59c640 1 --2- 172.21.15.76:0/2318425244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba08050270 0x7fba08048610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.642+0000 7ff4effff640 1 -- 172.21.15.76:0/271887021 wait complete. 2023-11-09T03:45:54.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.644+0000 7ff4effff640 1 Processor -- start 2023-11-09T03:45:54.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.644+0000 7fba14c7c640 1 -- 172.21.15.76:0/2318425244 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba08180f90 con 0x7fba080df3f0 2023-11-09T03:45:54.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.644+0000 7fba14c7c640 1 -- 172.21.15.76:0/2318425244 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba08181270 con 0x7fba08180510 2023-11-09T03:45:54.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.644+0000 7fba0ed9d640 1 --2- 172.21.15.76:0/2318425244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba08180510 0x7fba081809a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.643+0000 7faf830d7640 1 Processor -- start 2023-11-09T03:45:54.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.643+0000 7faf830d7640 1 -- start start 2023-11-09T03:45:54.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.645+0000 7fba0e59c640 1 -- 172.21.15.76:0/2318425244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba08180510 msgr2=0x7fba081809a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.645+0000 7fba0e59c640 1 --2- 172.21.15.76:0/2318425244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba08180510 0x7fba081809a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.645+0000 7fba0e59c640 1 -- 172.21.15.76:0/2318425244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba080df3f0 msgr2=0x7fba0804d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.644+0000 7ff4effff640 1 -- start start 2023-11-09T03:45:54.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.645+0000 7fba0e59c640 1 --2- 172.21.15.76:0/2318425244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba080df3f0 0x7fba0804d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.645+0000 7f35f294c640 1 -- 172.21.15.76:0/2034889537 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35ec16aaa0 con 0x7f35ec070710 2023-11-09T03:45:54.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.645+0000 7fba0e59c640 1 -- 172.21.15.76:0/2318425244 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0804bbd0 con 0x7fba08050270 2023-11-09T03:45:54.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.645+0000 7fba0ed9d640 1 --2- 172.21.15.76:0/2318425244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba08180510 0x7fba081809a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:45:54.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.646+0000 7fba0e59c640 1 --2- 172.21.15.76:0/2318425244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba08050270 0x7fba08048610 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fba0000c410 tx=0x7fba0000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:54.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.646+0000 7fb9feffd640 1 -- 172.21.15.76:0/2318425244 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba00011790 con 0x7fba08050270 2023-11-09T03:45:54.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.646+0000 7fba14c7c640 1 -- 172.21.15.76:0/2318425244 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba0804beb0 con 0x7fba08050270 2023-11-09T03:45:54.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.646+0000 7fba14c7c640 1 -- 172.21.15.76:0/2318425244 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba080db410 con 0x7fba08050270 2023-11-09T03:45:54.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.646+0000 7fb9feffd640 1 -- 172.21.15.76:0/2318425244 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba000118f0 con 0x7fba08050270 2023-11-09T03:45:54.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.645+0000 7fb91b59e640 1 -- 172.21.15.76:0/1336970080 shutdown_connections 2023-11-09T03:45:54.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.645+0000 7fb91b59e640 1 -- 172.21.15.76:0/1336970080 wait complete. 2023-11-09T03:45:54.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.646+0000 7ff4effff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc0039c0 0x7ff4f0213380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.646+0000 7ff4effff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4f00c3a80 0x7ff4f00c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.646+0000 7ff4eeffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc0039c0 0x7ff4f0213380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.646+0000 7ff4eeffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc0039c0 0x7ff4f0213380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:33698/0 (socket says 172.21.15.76:33698) 2023-11-09T03:45:54.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.646+0000 7ff4eeffd640 1 -- 172.21.15.76:0/2749817544 learned_addr learned my addr 172.21.15.76:0/2749817544 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:54.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.646+0000 7ff4effff640 1 --2- 172.21.15.76:0/2749817544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4f016c190 0x7ff4f016c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.646+0000 7ff4effff640 1 -- 172.21.15.76:0/2749817544 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f016cb50 con 0x7ff4f00c3a80 2023-11-09T03:45:54.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.646+0000 7ff4effff640 1 -- 172.21.15.76:0/2749817544 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f00c4400 con 0x7ff4dc0039c0 2023-11-09T03:45:54.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.646+0000 7ff4ef7fe640 1 --2- 172.21.15.76:0/2749817544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4f016c190 0x7ff4f016c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.646+0000 7ff4effff640 1 -- 172.21.15.76:0/2749817544 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f016f7c0 con 0x7ff4f016c190 2023-11-09T03:45:54.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.645+0000 7faf830d7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7403c2c0 0x7faf740e7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.645+0000 7faf830d7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf74058b00 0x7faf740e1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.645+0000 7faf830d7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf740e7930 0x7faf7404df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.645+0000 7faf830d7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf74181a10 con 0x7faf740e7930 2023-11-09T03:45:54.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.645+0000 7faf830d7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf740e8250 con 0x7faf7403c2c0 2023-11-09T03:45:54.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.645+0000 7faf830d7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf74180e00 con 0x7faf74058b00 2023-11-09T03:45:54.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.645+0000 7faf820d5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7403c2c0 0x7faf740e7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.645+0000 7faf820d5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7403c2c0 0x7faf740e7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:33696/0 (socket says 172.21.15.76:33696) 2023-11-09T03:45:54.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.645+0000 7faf820d5640 1 -- 172.21.15.76:0/3894432627 learned_addr learned my addr 172.21.15.76:0/3894432627 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:54.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.645+0000 7faf818d4640 1 --2- 172.21.15.76:0/3894432627 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf74058b00 0x7faf740e1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.645+0000 7faf820d5640 1 -- 172.21.15.76:0/3894432627 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf74058b00 msgr2=0x7faf740e1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.645+0000 7faf820d5640 1 --2- 172.21.15.76:0/3894432627 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf74058b00 0x7faf740e1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.645+0000 7faf820d5640 1 -- 172.21.15.76:0/3894432627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf740e7930 msgr2=0x7faf7404df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.646+0000 7faf828d6640 1 --2- 172.21.15.76:0/3894432627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf740e7930 0x7faf7404df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.645+0000 7f35e8ff9640 1 -- 172.21.15.76:0/2034889537 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f35dc013470 con 0x7f35ec070710 2023-11-09T03:45:54.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.646+0000 7f35e8ff9640 1 --2- 172.21.15.76:0/2034889537 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f35d806aea0 0x7f35d806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.646+0000 7f35e8ff9640 1 -- 172.21.15.76:0/2034889537 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f35dc089a20 con 0x7f35ec070710 2023-11-09T03:45:54.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.646+0000 7f35ebfff640 1 --2- 172.21.15.76:0/2034889537 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f35d806aea0 0x7f35d806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.647+0000 7ff4e67fc640 1 --2- 172.21.15.76:0/2749817544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4f00c3a80 0x7ff4f00c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.647+0000 7faf820d5640 1 --2- 172.21.15.76:0/3894432627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf740e7930 0x7faf7404df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.647+0000 7faf820d5640 1 -- 172.21.15.76:0/3894432627 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6c009400 con 0x7faf7403c2c0 2023-11-09T03:45:54.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.647+0000 7faf828d6640 1 --2- 172.21.15.76:0/3894432627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf740e7930 0x7faf7404df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:54.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.647+0000 7ff4eeffd640 1 -- 172.21.15.76:0/2749817544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4f016c190 msgr2=0x7ff4f016c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.647+0000 7ff4eeffd640 1 --2- 172.21.15.76:0/2749817544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4f016c190 0x7ff4f016c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.647+0000 7ff4eeffd640 1 -- 172.21.15.76:0/2749817544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4f00c3a80 msgr2=0x7ff4f00c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.647+0000 7faf820d5640 1 --2- 172.21.15.76:0/3894432627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7403c2c0 0x7faf740e7d10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7faf7800c780 tx=0x7faf7800cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:54.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.647+0000 7ff4eeffd640 1 --2- 172.21.15.76:0/2749817544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4f00c3a80 0x7ff4f00c3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.647+0000 7ff4eeffd640 1 -- 172.21.15.76:0/2749817544 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f016fa80 con 0x7ff4dc0039c0 2023-11-09T03:45:54.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.647+0000 7ff4e67fc640 1 --2- 172.21.15.76:0/2749817544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4f00c3a80 0x7ff4f00c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:54.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.647+0000 7fb9feffd640 1 -- 172.21.15.76:0/2318425244 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba00011ca0 con 0x7fba08050270 2023-11-09T03:45:54.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.647+0000 7ff4ef7fe640 1 --2- 172.21.15.76:0/2749817544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4f016c190 0x7ff4f016c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:45:54.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.647+0000 7fb91b59e640 1 Processor -- start 2023-11-09T03:45:54.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.648+0000 7fb91b59e640 1 -- start start 2023-11-09T03:45:54.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.647+0000 7ff4eeffd640 1 --2- 172.21.15.76:0/2749817544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc0039c0 0x7ff4f0213380 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0035040 tx=0x7ff4e0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:54.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.648+0000 7faf72ffd640 1 -- 172.21.15.76:0/3894432627 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf780072d0 con 0x7faf7403c2c0 2023-11-09T03:45:54.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.648+0000 7faf72ffd640 1 -- 172.21.15.76:0/3894432627 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf78007430 con 0x7faf7403c2c0 2023-11-09T03:45:54.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.648+0000 7faf72ffd640 1 -- 172.21.15.76:0/3894432627 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf780047e0 con 0x7faf7403c2c0 2023-11-09T03:45:54.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.648+0000 7f35ebfff640 1 --2- 172.21.15.76:0/2034889537 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f35d806aea0 0x7f35d806d360 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f35e0036000 tx=0x7f35e003d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:54.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.649+0000 7fb91b59e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb904003970 0x7fb9140d5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.649+0000 7fb91b59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb91404c560 0x7fb91404bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.649+0000 7fb9feffd640 1 -- 172.21.15.76:0/2318425244 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fba0001c460 con 0x7fba08050270 2023-11-09T03:45:54.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.649+0000 7fb91a59c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb904003970 0x7fb9140d5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.649+0000 7fb91a59c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb904003970 0x7fb9140d5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:50958/0 (socket says 172.21.15.76:50958) 2023-11-09T03:45:54.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.649+0000 7fb91a59c640 1 -- 172.21.15.76:0/1214585738 learned_addr learned my addr 172.21.15.76:0/1214585738 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:54.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.650+0000 7fb91b59e640 1 --2- 172.21.15.76:0/1214585738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9140d9620 0x7fb9140d9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.649+0000 7ff4e7fff640 1 -- 172.21.15.76:0/2749817544 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0033ea0 con 0x7ff4dc0039c0 2023-11-09T03:45:54.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.649+0000 7ff4e7fff640 1 -- 172.21.15.76:0/2749817544 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e0040070 con 0x7ff4dc0039c0 2023-11-09T03:45:54.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.649+0000 7ff4e7fff640 1 -- 172.21.15.76:0/2749817544 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0004480 con 0x7ff4dc0039c0 2023-11-09T03:45:54.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.649+0000 7ff4effff640 1 -- 172.21.15.76:0/2749817544 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f016fd60 con 0x7ff4dc0039c0 2023-11-09T03:45:54.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.649+0000 7ff4effff640 1 -- 172.21.15.76:0/2749817544 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f0170320 con 0x7ff4dc0039c0 2023-11-09T03:45:54.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.649+0000 7faf830d7640 1 -- 172.21.15.76:0/3894432627 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf741810e0 con 0x7faf7403c2c0 2023-11-09T03:45:54.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.649+0000 7faf830d7640 1 -- 172.21.15.76:0/3894432627 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf741816a0 con 0x7faf7403c2c0 2023-11-09T03:45:54.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.650+0000 7fb919d9b640 1 --2- 172.21.15.76:0/1214585738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb91404c560 0x7fb91404bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.650+0000 7fb91b59e640 1 -- 172.21.15.76:0/1214585738 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9141748d0 con 0x7fb91404c560 2023-11-09T03:45:54.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.650+0000 7faf830d7640 1 -- 172.21.15.76:0/3894432627 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf740058e0 con 0x7faf7403c2c0 2023-11-09T03:45:54.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.650+0000 7fb91ad9d640 1 --2- 172.21.15.76:0/1214585738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9140d9620 0x7fb9140d9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.650+0000 7fb91b59e640 1 -- 172.21.15.76:0/1214585738 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb914175450 con 0x7fb9140d9620 2023-11-09T03:45:54.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.650+0000 7ff4effff640 1 -- 172.21.15.76:0/2749817544 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4f00b04e0 con 0x7ff4dc0039c0 2023-11-09T03:45:54.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.650+0000 7fb9feffd640 1 --2- 172.21.15.76:0/2318425244 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb9f006ae70 0x7fb9f006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.650+0000 7fb91b59e640 1 -- 172.21.15.76:0/1214585738 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9140d5770 con 0x7fb904003970 2023-11-09T03:45:54.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.650+0000 7fb919d9b640 1 -- 172.21.15.76:0/1214585738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb904003970 msgr2=0x7fb9140d5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.651+0000 7fb919d9b640 1 --2- 172.21.15.76:0/1214585738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb904003970 0x7fb9140d5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.651+0000 7fba0dd9b640 1 --2- 172.21.15.76:0/2318425244 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb9f006ae70 0x7fb9f006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.651+0000 7fb919d9b640 1 -- 172.21.15.76:0/1214585738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9140d9620 msgr2=0x7fb9140d9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.651+0000 7fb919d9b640 1 --2- 172.21.15.76:0/1214585738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9140d9620 0x7fb9140d9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.651+0000 7f35e8ff9640 1 -- 172.21.15.76:0/2034889537 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f35dc0547f0 con 0x7f35ec070710 2023-11-09T03:45:54.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.651+0000 7fb919d9b640 1 -- 172.21.15.76:0/1214585738 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb91404c320 con 0x7fb91404c560 2023-11-09T03:45:54.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.651+0000 7fb91a59c640 1 --2- 172.21.15.76:0/1214585738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb904003970 0x7fb9140d5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:45:54.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.651+0000 7fb919d9b640 1 --2- 172.21.15.76:0/1214585738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb91404c560 0x7fb91404bbd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0108f0 tx=0x7fb90c010db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:54.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.651+0000 7fb90affd640 1 -- 172.21.15.76:0/1214585738 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c00eb00 con 0x7fb91404c560 2023-11-09T03:45:54.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.652+0000 7fb90affd640 1 -- 172.21.15.76:0/1214585738 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb90c00ec60 con 0x7fb91404c560 2023-11-09T03:45:54.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.652+0000 7fb90affd640 1 -- 172.21.15.76:0/1214585738 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c01c400 con 0x7fb91404c560 2023-11-09T03:45:54.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.652+0000 7faf72ffd640 1 -- 172.21.15.76:0/3894432627 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faf78004960 con 0x7faf7403c2c0 2023-11-09T03:45:54.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.652+0000 7fba0dd9b640 1 --2- 172.21.15.76:0/2318425244 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb9f006ae70 0x7fb9f006d330 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fba0804a2e0 tx=0x7fba04009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:54.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.652+0000 7fb9feffd640 1 -- 172.21.15.76:0/2318425244 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fba00089860 con 0x7fba08050270 2023-11-09T03:45:54.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.652+0000 7faf72ffd640 1 --2- 172.21.15.76:0/3894432627 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf6006ae70 0x7faf6006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.720 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.652+0000 7fb91b59e640 1 -- 172.21.15.76:0/1214585738 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb914174ab0 con 0x7fb91404c560 2023-11-09T03:45:54.720 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.652+0000 7fb91b59e640 1 -- 172.21.15.76:0/1214585738 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb914175070 con 0x7fb91404c560 2023-11-09T03:45:54.720 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.652+0000 7faf72ffd640 1 -- 172.21.15.76:0/3894432627 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7faf78088dc0 con 0x7faf7403c2c0 2023-11-09T03:45:54.721 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.653+0000 7fba14c7c640 1 -- 172.21.15.76:0/2318425244 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9f4005250 con 0x7fba08050270 2023-11-09T03:45:54.721 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.653+0000 7faf818d4640 1 --2- 172.21.15.76:0/3894432627 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf6006ae70 0x7faf6006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.721 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.654+0000 7fb91b59e640 1 -- 172.21.15.76:0/1214585738 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb900005250 con 0x7fb91404c560 2023-11-09T03:45:54.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.654+0000 7faf818d4640 1 --2- 172.21.15.76:0/3894432627 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf6006ae70 0x7faf6006d330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7faf6c034000 tx=0x7faf6c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:54.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.655+0000 7fb90affd640 1 -- 172.21.15.76:0/1214585738 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb90c01c560 con 0x7fb91404c560 2023-11-09T03:45:54.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.655+0000 7ff4e7fff640 1 -- 172.21.15.76:0/2749817544 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff4e0031050 con 0x7ff4dc0039c0 2023-11-09T03:45:54.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.655+0000 7fb90affd640 1 --2- 172.21.15.76:0/1214585738 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb8fc06aea0 0x7fb8fc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.655+0000 7fb90affd640 1 -- 172.21.15.76:0/1214585738 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb90c089870 con 0x7fb91404c560 2023-11-09T03:45:54.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.655+0000 7ff4e7fff640 1 --2- 172.21.15.76:0/2749817544 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff4d006ae70 0x7ff4d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:54.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.655+0000 7fb91a59c640 1 --2- 172.21.15.76:0/1214585738 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb8fc06aea0 0x7fb8fc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.656+0000 7ff4e7fff640 1 -- 172.21.15.76:0/2749817544 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff4e00ab8f0 con 0x7ff4dc0039c0 2023-11-09T03:45:54.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.656+0000 7ff4e7fff640 1 -- 172.21.15.76:0/2749817544 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff4e00d7d40 con 0x7ff4dc0039c0 2023-11-09T03:45:54.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.656+0000 7ff4e67fc640 1 --2- 172.21.15.76:0/2749817544 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff4d006ae70 0x7ff4d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:54.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.656+0000 7fb91a59c640 1 --2- 172.21.15.76:0/1214585738 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb8fc06aea0 0x7fb8fc06d360 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb91c04f240 tx=0x7fb91c0ab000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:54.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.656+0000 7ff4e67fc640 1 --2- 172.21.15.76:0/2749817544 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff4d006ae70 0x7ff4d006d330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00ca7a0 tx=0x7ff4d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:54.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.659+0000 7fb9feffd640 1 -- 172.21.15.76:0/2318425244 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fba00054470 con 0x7fba08050270 2023-11-09T03:45:54.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.660+0000 7faf72ffd640 1 -- 172.21.15.76:0/3894432627 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faf78053cd0 con 0x7faf7403c2c0 2023-11-09T03:45:54.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.668+0000 7fb90affd640 1 -- 172.21.15.76:0/1214585738 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb90c0546d0 con 0x7fb91404c560 2023-11-09T03:45:54.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.904+0000 7f64d3f4c640 1 -- 172.21.15.76:0/3623276770 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f64b4005500 con 0x7f64cc06ac40 2023-11-09T03:45:54.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.905+0000 7f64c27fc640 1 -- 172.21.15.76:0/3623276770 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f64c4057fd0 con 0x7f64cc06ac40 2023-11-09T03:45:54.906 INFO:teuthology.orchestra.run.smithi076.stdout:55834574858 2023-11-09T03:45:54.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.908+0000 7f64d3f4c640 1 -- 172.21.15.76:0/3623276770 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f64b006afc0 msgr2=0x7f64b006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.908+0000 7f64d3f4c640 1 --2- 172.21.15.76:0/3623276770 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f64b006afc0 0x7f64b006d480 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f64c8035000 tx=0x7f64c8008030 comp rx=0 tx=0).stop 2023-11-09T03:45:54.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.908+0000 7f64d3f4c640 1 -- 172.21.15.76:0/3623276770 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64cc06ac40 msgr2=0x7f64cc1f5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.908+0000 7f64d3f4c640 1 --2- 172.21.15.76:0/3623276770 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64cc06ac40 0x7f64cc1f5720 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f64c400eed0 tx=0x7f64c400b480 comp rx=0 tx=0).stop 2023-11-09T03:45:54.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.908+0000 7f64d3f4c640 1 -- 172.21.15.76:0/3623276770 shutdown_connections 2023-11-09T03:45:54.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.908+0000 7f64d3f4c640 1 --2- 172.21.15.76:0/3623276770 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f64b006afc0 0x7f64b006d480 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.908+0000 7f64d3f4c640 1 --2- 172.21.15.76:0/3623276770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64cc14b060 0x7f64cc160d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.908+0000 7f64d3f4c640 1 --2- 172.21.15.76:0/3623276770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64cc0caae0 0x7f64cc148d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.908+0000 7f64d3f4c640 1 --2- 172.21.15.76:0/3623276770 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64cc06ac40 0x7f64cc1f5720 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.908+0000 7f64d3f4c640 1 -- 172.21.15.76:0/3623276770 >> 172.21.15.76:0/3623276770 conn(0x7f64cc06a120 msgr2=0x7f64cc06e320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:54.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.908+0000 7f64d3f4c640 1 -- 172.21.15.76:0/3623276770 shutdown_connections 2023-11-09T03:45:54.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.908+0000 7f64d3f4c640 1 -- 172.21.15.76:0/3623276770 wait complete. 2023-11-09T03:45:54.926 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.3 2023-11-09T03:45:54.926 DEBUG:teuthology.parallel:result is None 2023-11-09T03:45:54.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.970+0000 7f35f294c640 1 -- 172.21.15.76:0/2034889537 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f35ec074290 con 0x7f35ec070710 2023-11-09T03:45:54.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.970+0000 7f35e8ff9640 1 -- 172.21.15.76:0/2034889537 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f35dc057e10 con 0x7f35ec070710 2023-11-09T03:45:54.970 INFO:teuthology.orchestra.run.smithi076.stdout:55834574856 2023-11-09T03:45:54.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.972+0000 7f35f294c640 1 -- 172.21.15.76:0/2034889537 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f35d806aea0 msgr2=0x7f35d806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.972+0000 7f35f294c640 1 --2- 172.21.15.76:0/2034889537 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f35d806aea0 0x7f35d806d360 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f35e0036000 tx=0x7f35e003d040 comp rx=0 tx=0).stop 2023-11-09T03:45:54.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.973+0000 7f35f294c640 1 -- 172.21.15.76:0/2034889537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35ec070710 msgr2=0x7f35ec165480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.973+0000 7f35f294c640 1 --2- 172.21.15.76:0/2034889537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35ec070710 0x7f35ec165480 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00e850 tx=0x7f35dc00ed10 comp rx=0 tx=0).stop 2023-11-09T03:45:54.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.973+0000 7f35f294c640 1 -- 172.21.15.76:0/2034889537 shutdown_connections 2023-11-09T03:45:54.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.973+0000 7f35f294c640 1 --2- 172.21.15.76:0/2034889537 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f35d806aea0 0x7f35d806d360 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.973+0000 7f35f294c640 1 --2- 172.21.15.76:0/2034889537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35ec16e390 0x7f35ec1659c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.973+0000 7f35f294c640 1 --2- 172.21.15.76:0/2034889537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35ec070710 0x7f35ec165480 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.973+0000 7f35f294c640 1 --2- 172.21.15.76:0/2034889537 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35ec06d440 0x7f35ec1771f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.973+0000 7f35f294c640 1 -- 172.21.15.76:0/2034889537 >> 172.21.15.76:0/2034889537 conn(0x7f35ec06a120 msgr2=0x7f35ec0ce950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:54.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.973+0000 7f35f294c640 1 -- 172.21.15.76:0/2034889537 shutdown_connections 2023-11-09T03:45:54.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.973+0000 7f35f294c640 1 -- 172.21.15.76:0/2034889537 wait complete. 2023-11-09T03:45:54.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.980+0000 7fbec757c640 1 -- 172.21.15.76:0/586565732 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fbea0005500 con 0x7fbeb80df360 2023-11-09T03:45:54.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.980+0000 7fbeb5ffb640 1 -- 172.21.15.76:0/586565732 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbec008e030 con 0x7fbeb80df360 2023-11-09T03:45:54.980 INFO:teuthology.orchestra.run.smithi076.stdout:55834574857 2023-11-09T03:45:54.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.984+0000 7fbec757c640 1 -- 172.21.15.76:0/586565732 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbea406af40 msgr2=0x7fbea406d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.984 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.1 2023-11-09T03:45:54.984 DEBUG:teuthology.parallel:result is None 2023-11-09T03:45:54.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.984+0000 7fbec757c640 1 --2- 172.21.15.76:0/586565732 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbea406af40 0x7fbea406d400 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbebc039000 tx=0x7fbebc034000 comp rx=0 tx=0).stop 2023-11-09T03:45:54.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.984+0000 7fbec757c640 1 -- 172.21.15.76:0/586565732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbeb80df360 msgr2=0x7fbeb80dbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:54.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.984+0000 7fbec757c640 1 --2- 172.21.15.76:0/586565732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbeb80df360 0x7fbeb80dbde0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbec0073660 tx=0x7fbec0072860 comp rx=0 tx=0).stop 2023-11-09T03:45:54.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.984+0000 7fbec757c640 1 -- 172.21.15.76:0/586565732 shutdown_connections 2023-11-09T03:45:54.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.984+0000 7fbec757c640 1 --2- 172.21.15.76:0/586565732 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbea406af40 0x7fbea406d400 secure :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbebc039000 tx=0x7fbebc034000 comp rx=0 tx=0).stop 2023-11-09T03:45:54.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.984+0000 7fbec757c640 1 --2- 172.21.15.76:0/586565732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbeb80df360 0x7fbeb80dbde0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.984+0000 7fbec757c640 1 --2- 172.21.15.76:0/586565732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbeb8054a30 0x7fbeb8168220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.984+0000 7fbec757c640 1 --2- 172.21.15.76:0/586565732 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbeb803a8c0 0x7fbeb80d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:54.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.984+0000 7fbec757c640 1 -- 172.21.15.76:0/586565732 >> 172.21.15.76:0/586565732 conn(0x7fbeb800a440 msgr2=0x7fbeb80d4360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:54.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.984+0000 7fbec757c640 1 -- 172.21.15.76:0/586565732 shutdown_connections 2023-11-09T03:45:54.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:54.984+0000 7fbec757c640 1 -- 172.21.15.76:0/586565732 wait complete. 2023-11-09T03:45:54.997 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.4 2023-11-09T03:45:54.998 DEBUG:teuthology.parallel:result is None 2023-11-09T03:45:55.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.002+0000 7f5adb59e640 1 -- 172.21.15.76:0/3001594257 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5adc04ece0 con 0x7f5adc158eb0 2023-11-09T03:45:55.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.002+0000 7f5acaffd640 1 -- 172.21.15.76:0/3001594257 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5ad407aa70 con 0x7f5adc158eb0 2023-11-09T03:45:55.001 INFO:teuthology.orchestra.run.smithi076.stdout:55834574856 2023-11-09T03:45:55.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.005+0000 7f5adb59e640 1 -- 172.21.15.76:0/3001594257 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5abc06af40 msgr2=0x7f5abc06d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:55.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.005+0000 7f5adb59e640 1 --2- 172.21.15.76:0/3001594257 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5abc06af40 0x7f5abc06d400 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5acc006fd0 tx=0x7f5acc008040 comp rx=0 tx=0).stop 2023-11-09T03:45:55.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.005+0000 7f5adb59e640 1 -- 172.21.15.76:0/3001594257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5adc158eb0 msgr2=0x7f5adc15bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:55.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.005+0000 7f5adb59e640 1 --2- 172.21.15.76:0/3001594257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5adc158eb0 0x7f5adc15bd10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4036000 tx=0x7f5ad4004a40 comp rx=0 tx=0).stop 2023-11-09T03:45:55.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.005+0000 7f5adb59e640 1 -- 172.21.15.76:0/3001594257 shutdown_connections 2023-11-09T03:45:55.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.005+0000 7f5adb59e640 1 --2- 172.21.15.76:0/3001594257 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5abc06af40 0x7f5abc06d400 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.005+0000 7f5adb59e640 1 --2- 172.21.15.76:0/3001594257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5adc15ce40 0x7f5adc1e2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.005+0000 7f5adb59e640 1 --2- 172.21.15.76:0/3001594257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5adc158eb0 0x7f5adc15bd10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.005+0000 7f5adb59e640 1 --2- 172.21.15.76:0/3001594257 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5adc158ad0 0x7f5adc0c4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.005+0000 7f5adb59e640 1 -- 172.21.15.76:0/3001594257 >> 172.21.15.76:0/3001594257 conn(0x7f5adc06a120 msgr2=0x7f5adc0cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:55.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.005+0000 7f5adb59e640 1 -- 172.21.15.76:0/3001594257 shutdown_connections 2023-11-09T03:45:55.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.005+0000 7f5adb59e640 1 -- 172.21.15.76:0/3001594257 wait complete. 2023-11-09T03:45:55.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.019+0000 7ff4effff640 1 -- 172.21.15.76:0/2749817544 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff4f00b04e0 con 0x7ff4dc0039c0 2023-11-09T03:45:55.019 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.0 2023-11-09T03:45:55.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.020+0000 7ff4e7fff640 1 -- 172.21.15.76:0/2749817544 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff4e0042020 con 0x7ff4dc0039c0 2023-11-09T03:45:55.020 INFO:teuthology.orchestra.run.smithi076.stdout:55834574856 2023-11-09T03:45:55.020 DEBUG:teuthology.parallel:result is None 2023-11-09T03:45:55.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.022+0000 7ff4effff640 1 -- 172.21.15.76:0/2749817544 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff4d006ae70 msgr2=0x7ff4d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:55.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.022+0000 7ff4effff640 1 --2- 172.21.15.76:0/2749817544 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff4d006ae70 0x7ff4d006d330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00ca7a0 tx=0x7ff4d8008040 comp rx=0 tx=0).stop 2023-11-09T03:45:55.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.022+0000 7ff4effff640 1 -- 172.21.15.76:0/2749817544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc0039c0 msgr2=0x7ff4f0213380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:55.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.022+0000 7ff4effff640 1 --2- 172.21.15.76:0/2749817544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc0039c0 0x7ff4f0213380 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0035040 tx=0x7ff4e0004730 comp rx=0 tx=0).stop 2023-11-09T03:45:55.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.022+0000 7ff4effff640 1 -- 172.21.15.76:0/2749817544 shutdown_connections 2023-11-09T03:45:55.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.022+0000 7ff4effff640 1 --2- 172.21.15.76:0/2749817544 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff4d006ae70 0x7ff4d006d330 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.022+0000 7ff4effff640 1 --2- 172.21.15.76:0/2749817544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4f016c190 0x7ff4f016c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.022+0000 7ff4effff640 1 --2- 172.21.15.76:0/2749817544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4f00c3a80 0x7ff4f00c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.022+0000 7ff4effff640 1 --2- 172.21.15.76:0/2749817544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc0039c0 0x7ff4f0213380 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.022+0000 7ff4effff640 1 -- 172.21.15.76:0/2749817544 >> 172.21.15.76:0/2749817544 conn(0x7ff4f006c780 msgr2=0x7ff4f00cc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:55.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.022+0000 7ff4effff640 1 -- 172.21.15.76:0/2749817544 shutdown_connections 2023-11-09T03:45:55.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.022+0000 7ff4effff640 1 -- 172.21.15.76:0/2749817544 wait complete. 2023-11-09T03:45:55.031 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.2 2023-11-09T03:45:55.032 DEBUG:teuthology.parallel:result is None 2023-11-09T03:45:55.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.035+0000 7faf830d7640 1 -- 172.21.15.76:0/3894432627 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7faf740e2390 con 0x7faf7403c2c0 2023-11-09T03:45:55.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.035+0000 7faf72ffd640 1 -- 172.21.15.76:0/3894432627 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faf78018020 con 0x7faf7403c2c0 2023-11-09T03:45:55.034 INFO:teuthology.orchestra.run.smithi076.stdout:55834574857 2023-11-09T03:45:55.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.037+0000 7faf830d7640 1 -- 172.21.15.76:0/3894432627 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf6006ae70 msgr2=0x7faf6006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:55.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.037+0000 7faf830d7640 1 --2- 172.21.15.76:0/3894432627 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf6006ae70 0x7faf6006d330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7faf6c034000 tx=0x7faf6c008030 comp rx=0 tx=0).stop 2023-11-09T03:45:55.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.037+0000 7faf830d7640 1 -- 172.21.15.76:0/3894432627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7403c2c0 msgr2=0x7faf740e7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:55.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.037+0000 7faf830d7640 1 --2- 172.21.15.76:0/3894432627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7403c2c0 0x7faf740e7d10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7faf7800c780 tx=0x7faf7800cc40 comp rx=0 tx=0).stop 2023-11-09T03:45:55.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.037+0000 7faf830d7640 1 -- 172.21.15.76:0/3894432627 shutdown_connections 2023-11-09T03:45:55.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.037+0000 7faf830d7640 1 --2- 172.21.15.76:0/3894432627 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf6006ae70 0x7faf6006d330 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.037+0000 7faf830d7640 1 --2- 172.21.15.76:0/3894432627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf740e7930 0x7faf7404df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.037+0000 7faf830d7640 1 --2- 172.21.15.76:0/3894432627 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf74058b00 0x7faf740e1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.037+0000 7faf830d7640 1 --2- 172.21.15.76:0/3894432627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7403c2c0 0x7faf740e7d10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.037+0000 7faf830d7640 1 -- 172.21.15.76:0/3894432627 >> 172.21.15.76:0/3894432627 conn(0x7faf74059120 msgr2=0x7faf740579e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:55.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.037+0000 7faf830d7640 1 -- 172.21.15.76:0/3894432627 shutdown_connections 2023-11-09T03:45:55.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.037+0000 7faf830d7640 1 -- 172.21.15.76:0/3894432627 wait complete. 2023-11-09T03:45:55.047 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.7 2023-11-09T03:45:55.047 DEBUG:teuthology.parallel:result is None 2023-11-09T03:45:55.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.059+0000 7fba14c7c640 1 -- 172.21.15.76:0/2318425244 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb9f4005500 con 0x7fba08050270 2023-11-09T03:45:55.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.059+0000 7fb9feffd640 1 -- 172.21.15.76:0/2318425244 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fba00057a90 con 0x7fba08050270 2023-11-09T03:45:55.059 INFO:teuthology.orchestra.run.smithi076.stdout:55834574857 2023-11-09T03:45:55.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.061+0000 7fba14c7c640 1 -- 172.21.15.76:0/2318425244 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb9f006ae70 msgr2=0x7fb9f006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:55.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.061+0000 7fba14c7c640 1 --2- 172.21.15.76:0/2318425244 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb9f006ae70 0x7fb9f006d330 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fba0804a2e0 tx=0x7fba04009040 comp rx=0 tx=0).stop 2023-11-09T03:45:55.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.061+0000 7fba14c7c640 1 -- 172.21.15.76:0/2318425244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba08050270 msgr2=0x7fba08048610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:55.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.061+0000 7fba14c7c640 1 --2- 172.21.15.76:0/2318425244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba08050270 0x7fba08048610 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fba0000c410 tx=0x7fba0000c8d0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.061+0000 7fba14c7c640 1 -- 172.21.15.76:0/2318425244 shutdown_connections 2023-11-09T03:45:55.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.061+0000 7fba14c7c640 1 --2- 172.21.15.76:0/2318425244 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb9f006ae70 0x7fb9f006d330 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.061+0000 7fba14c7c640 1 --2- 172.21.15.76:0/2318425244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba08180510 0x7fba081809a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.061+0000 7fba14c7c640 1 --2- 172.21.15.76:0/2318425244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba080df3f0 0x7fba0804d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.061+0000 7fba14c7c640 1 --2- 172.21.15.76:0/2318425244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba08050270 0x7fba08048610 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.061+0000 7fba14c7c640 1 -- 172.21.15.76:0/2318425244 >> 172.21.15.76:0/2318425244 conn(0x7fba08054b00 msgr2=0x7fba08039560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:55.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.061+0000 7fba14c7c640 1 -- 172.21.15.76:0/2318425244 shutdown_connections 2023-11-09T03:45:55.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.061+0000 7fba14c7c640 1 -- 172.21.15.76:0/2318425244 wait complete. 2023-11-09T03:45:55.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.065+0000 7fb91b59e640 1 -- 172.21.15.76:0/1214585738 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fb900005500 con 0x7fb91404c560 2023-11-09T03:45:55.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.065+0000 7fb90affd640 1 -- 172.21.15.76:0/1214585738 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb90c05e090 con 0x7fb91404c560 2023-11-09T03:45:55.065 INFO:teuthology.orchestra.run.smithi076.stdout:55834574857 2023-11-09T03:45:55.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.067+0000 7fb91b59e640 1 -- 172.21.15.76:0/1214585738 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb8fc06aea0 msgr2=0x7fb8fc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:55.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.067+0000 7fb91b59e640 1 --2- 172.21.15.76:0/1214585738 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb8fc06aea0 0x7fb8fc06d360 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb91c04f240 tx=0x7fb91c0ab000 comp rx=0 tx=0).stop 2023-11-09T03:45:55.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.067+0000 7fb91b59e640 1 -- 172.21.15.76:0/1214585738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb91404c560 msgr2=0x7fb91404bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:55.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.067+0000 7fb91b59e640 1 --2- 172.21.15.76:0/1214585738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb91404c560 0x7fb91404bbd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0108f0 tx=0x7fb90c010db0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.067+0000 7fb91b59e640 1 -- 172.21.15.76:0/1214585738 shutdown_connections 2023-11-09T03:45:55.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.067+0000 7fb91b59e640 1 --2- 172.21.15.76:0/1214585738 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb8fc06aea0 0x7fb8fc06d360 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.067+0000 7fb91b59e640 1 --2- 172.21.15.76:0/1214585738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9140d9620 0x7fb9140d9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.067+0000 7fb91b59e640 1 --2- 172.21.15.76:0/1214585738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb91404c560 0x7fb91404bbd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.067+0000 7fb91b59e640 1 --2- 172.21.15.76:0/1214585738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb904003970 0x7fb9140d5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.067+0000 7fb91b59e640 1 -- 172.21.15.76:0/1214585738 >> 172.21.15.76:0/1214585738 conn(0x7fb914054b00 msgr2=0x7fb9140536d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:55.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.067+0000 7fb91b59e640 1 -- 172.21.15.76:0/1214585738 shutdown_connections 2023-11-09T03:45:55.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.067+0000 7fb91b59e640 1 -- 172.21.15.76:0/1214585738 wait complete. 2023-11-09T03:45:55.072 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.5 2023-11-09T03:45:55.072 DEBUG:teuthology.parallel:result is None 2023-11-09T03:45:55.077 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.6 2023-11-09T03:45:55.078 DEBUG:teuthology.parallel:result is None 2023-11-09T03:45:55.078 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-09T03:45:55.078 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-09T03:45:55.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.218+0000 7f15900bb640 1 Processor -- start 2023-11-09T03:45:55.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.218+0000 7f15900bb640 1 -- start start 2023-11-09T03:45:55.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.219+0000 7f15900bb640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1588159c70 0x7f15880ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:55.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.219+0000 7f15900bb640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f158806db20 0x7f15880b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:55.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.219+0000 7f15900bb640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15880b4110 con 0x7f15880b0910 2023-11-09T03:45:55.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.219+0000 7f15900bb640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15880b49b0 con 0x7f158806db20 2023-11-09T03:45:55.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.219+0000 7f15900bb640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15880b0da0 con 0x7f1588159c70 2023-11-09T03:45:55.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.219+0000 7f158d656640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f158806db20 0x7f15880b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:55.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.219+0000 7f158e658640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f15880b0910 0x7f158815d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33684/0 (socket says 172.21.15.76:33684) 2023-11-09T03:45:55.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.219+0000 7f158e658640 1 -- 172.21.15.76:0/340371718 learned_addr learned my addr 172.21.15.76:0/340371718 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:55.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.219+0000 7f158de57640 1 --2- 172.21.15.76:0/340371718 >> v2:172.21.15.113:3300/0 conn(0x7f1588159c70 0x7f15880ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:55.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.219+0000 7f158d656640 1 -- 172.21.15.76:0/340371718 >> v2:172.21.15.113:3300/0 conn(0x7f1588159c70 msgr2=0x7f15880ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:55.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.219+0000 7f158d656640 1 --2- 172.21.15.76:0/340371718 >> v2:172.21.15.113:3300/0 conn(0x7f1588159c70 0x7f15880ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.219+0000 7f158d656640 1 -- 172.21.15.76:0/340371718 >> v1:172.21.15.76:6789/0 conn(0x7f15880b0910 legacy=0x7f158815d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:55.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.219+0000 7f158d656640 1 -- 172.21.15.76:0/340371718 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15880b43f0 con 0x7f158806db20 2023-11-09T03:45:55.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.219+0000 7f158ce55640 1 -- 172.21.15.76:0/340371718 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 297654797 0 0) 0x7f15880b4110 con 0x7f15880b0910 2023-11-09T03:45:55.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.220+0000 7f158d656640 1 --2- 172.21.15.76:0/340371718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f158806db20 0x7f15880b03d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f15880cc440 tx=0x7f157802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe976bff940d0f49 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:55.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.220+0000 7f158ce55640 1 -- 172.21.15.76:0/340371718 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1578031830 con 0x7f158806db20 2023-11-09T03:45:55.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.220+0000 7f158ce55640 1 -- 172.21.15.76:0/340371718 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1578031990 con 0x7f158806db20 2023-11-09T03:45:55.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.220+0000 7f158ce55640 1 -- 172.21.15.76:0/340371718 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1578031d20 con 0x7f158806db20 2023-11-09T03:45:55.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.221+0000 7f15900bb640 1 -- 172.21.15.76:0/340371718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f158806db20 msgr2=0x7f15880b03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:55.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.221+0000 7f15900bb640 1 --2- 172.21.15.76:0/340371718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f158806db20 0x7f15880b03d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f15880cc440 tx=0x7f157802b8d0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.221+0000 7f15900bb640 1 -- 172.21.15.76:0/340371718 shutdown_connections 2023-11-09T03:45:55.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.221+0000 7f15900bb640 1 --2- 172.21.15.76:0/340371718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f158806db20 0x7f15880b03d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.221+0000 7f15900bb640 1 --2- 172.21.15.76:0/340371718 >> v2:172.21.15.113:3300/0 conn(0x7f1588159c70 0x7f15880ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.221+0000 7f15900bb640 1 -- 172.21.15.76:0/340371718 >> 172.21.15.76:0/340371718 conn(0x7f15880ce950 msgr2=0x7f15880ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:55.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.221+0000 7f15900bb640 1 -- 172.21.15.76:0/340371718 shutdown_connections 2023-11-09T03:45:55.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.221+0000 7f15900bb640 1 -- 172.21.15.76:0/340371718 wait complete. 2023-11-09T03:45:55.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.222+0000 7f15900bb640 1 Processor -- start 2023-11-09T03:45:55.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.222+0000 7f15900bb640 1 -- start start 2023-11-09T03:45:55.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.223+0000 7f15900bb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f158806db20 0x7f1588153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:55.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.223+0000 7f15900bb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15880b0910 0x7f1588150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:55.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.223+0000 7f15900bb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1588159c70 0x7f15881f0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:55.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.223+0000 7f15900bb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158814fa00 con 0x7f158806db20 2023-11-09T03:45:55.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.223+0000 7f15900bb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158814fce0 con 0x7f15880b0910 2023-11-09T03:45:55.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.223+0000 7f15900bb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158814ffc0 con 0x7f1588159c70 2023-11-09T03:45:55.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.223+0000 7f158de57640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f158806db20 0x7f1588153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:55.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.223+0000 7f158de57640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f158806db20 0x7f1588153dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36462/0 (socket says 172.21.15.76:36462) 2023-11-09T03:45:55.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.223+0000 7f158de57640 1 -- 172.21.15.76:0/2435796969 learned_addr learned my addr 172.21.15.76:0/2435796969 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:55.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.223+0000 7f158d656640 1 --2- 172.21.15.76:0/2435796969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15880b0910 0x7f1588150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:55.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.223+0000 7f158e658640 1 --2- 172.21.15.76:0/2435796969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1588159c70 0x7f15881f0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:55.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.223+0000 7f158de57640 1 -- 172.21.15.76:0/2435796969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1588159c70 msgr2=0x7f15881f0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:55.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.223+0000 7f158de57640 1 --2- 172.21.15.76:0/2435796969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1588159c70 0x7f15881f0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.223+0000 7f158de57640 1 -- 172.21.15.76:0/2435796969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15880b0910 msgr2=0x7f1588150aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:55.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.223+0000 7f158de57640 1 --2- 172.21.15.76:0/2435796969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15880b0910 0x7f1588150aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.223+0000 7f158de57640 1 -- 172.21.15.76:0/2435796969 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1578009400 con 0x7f158806db20 2023-11-09T03:45:55.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.223+0000 7f158e658640 1 --2- 172.21.15.76:0/2435796969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1588159c70 0x7f15881f0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:55.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.223+0000 7f158de57640 1 --2- 172.21.15.76:0/2435796969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f158806db20 0x7f1588153dc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f158400d800 tx=0x7f158400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:55.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.223+0000 7f157e7fc640 1 -- 172.21.15.76:0/2435796969 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158400b6c0 con 0x7f158806db20 2023-11-09T03:45:55.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.223+0000 7f157e7fc640 1 -- 172.21.15.76:0/2435796969 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158400b820 con 0x7f158806db20 2023-11-09T03:45:55.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.223+0000 7f15900bb640 1 -- 172.21.15.76:0/2435796969 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15881502a0 con 0x7f158806db20 2023-11-09T03:45:55.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.223+0000 7f157e7fc640 1 -- 172.21.15.76:0/2435796969 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158400bbb0 con 0x7f158806db20 2023-11-09T03:45:55.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.223+0000 7f15900bb640 1 -- 172.21.15.76:0/2435796969 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1588154300 con 0x7f158806db20 2023-11-09T03:45:55.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.224+0000 7f15900bb640 1 -- 172.21.15.76:0/2435796969 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15881544c0 con 0x7f158806db20 2023-11-09T03:45:55.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.225+0000 7f157e7fc640 1 -- 172.21.15.76:0/2435796969 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f158400bd10 con 0x7f158806db20 2023-11-09T03:45:55.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.225+0000 7f157e7fc640 1 --2- 172.21.15.76:0/2435796969 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f156c06ae70 0x7f156c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:55.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.225+0000 7f157e7fc640 1 -- 172.21.15.76:0/2435796969 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1584089de0 con 0x7f158806db20 2023-11-09T03:45:55.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.225+0000 7f158d656640 1 --2- 172.21.15.76:0/2435796969 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f156c06ae70 0x7f156c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:55.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.226+0000 7f158d656640 1 --2- 172.21.15.76:0/2435796969 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f156c06ae70 0x7f156c06d330 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1578036000 tx=0x7f157803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:55.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.229+0000 7f157e7fc640 1 -- 172.21.15.76:0/2435796969 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1584054ba0 con 0x7f158806db20 2023-11-09T03:45:55.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.393+0000 7f15900bb640 1 -- 172.21.15.76:0/2435796969 --> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f158814fe50 con 0x7f156c06ae70 2023-11-09T03:45:55.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.403+0000 7f157e7fc640 1 -- 172.21.15.76:0/2435796969 <== mgr.4105 v2:172.21.15.113:6832/1264193042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345247 (secure 0 0 0) 0x7f158814fe50 con 0x7f156c06ae70 2023-11-09T03:45:55.405 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:55.407 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2023-11-09T03:45:55.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.406+0000 7f15900bb640 1 -- 172.21.15.76:0/2435796969 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f156c06ae70 msgr2=0x7f156c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:55.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.406+0000 7f15900bb640 1 --2- 172.21.15.76:0/2435796969 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f156c06ae70 0x7f156c06d330 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1578036000 tx=0x7f157803c040 comp rx=0 tx=0).stop 2023-11-09T03:45:55.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.406+0000 7f15900bb640 1 -- 172.21.15.76:0/2435796969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f158806db20 msgr2=0x7f1588153dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:55.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.406+0000 7f15900bb640 1 --2- 172.21.15.76:0/2435796969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f158806db20 0x7f1588153dc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f158400d800 tx=0x7f158400dcc0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.406+0000 7f15900bb640 1 -- 172.21.15.76:0/2435796969 shutdown_connections 2023-11-09T03:45:55.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.406+0000 7f15900bb640 1 --2- 172.21.15.76:0/2435796969 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f156c06ae70 0x7f156c06d330 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.406+0000 7f15900bb640 1 --2- 172.21.15.76:0/2435796969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1588159c70 0x7f15881f0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.406+0000 7f15900bb640 1 --2- 172.21.15.76:0/2435796969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15880b0910 0x7f1588150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.406+0000 7f15900bb640 1 --2- 172.21.15.76:0/2435796969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f158806db20 0x7f1588153dc0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.406+0000 7f15900bb640 1 -- 172.21.15.76:0/2435796969 >> 172.21.15.76:0/2435796969 conn(0x7f15880ce950 msgr2=0x7f15880cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:55.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.406+0000 7f15900bb640 1 -- 172.21.15.76:0/2435796969 shutdown_connections 2023-11-09T03:45:55.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.406+0000 7f15900bb640 1 -- 172.21.15.76:0/2435796969 wait complete. 2023-11-09T03:45:55.416 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2023-11-09T03:45:54.754165+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219468,"kb_used_data":3300,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755252,"statfs":{"total":773094113280,"available":772869378048,"internally_reserved":0,"allocated":3379200,"data_stored":2025099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2229,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003441"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.444993+0000","last_change":"2023-11-09T03:45:30.408658+0000","last_active":"2023-11-09T03:45:35.444993+0000","last_peered":"2023-11-09T03:45:35.444993+0000","last_clean":"2023-11-09T03:45:35.444993+0000","last_became_active":"2023-11-09T03:45:30.408345+0000","last_became_peered":"2023-11-09T03:45:30.408345+0000","last_unstale":"2023-11-09T03:45:35.444993+0000","last_undegraded":"2023-11-09T03:45:35.444993+0000","last_fullsized":"2023-11-09T03:45:35.444993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:12:07.086030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440545+0000","last_change":"2023-11-09T03:45:31.403012+0000","last_active":"2023-11-09T03:45:35.440545+0000","last_peered":"2023-11-09T03:45:35.440545+0000","last_clean":"2023-11-09T03:45:35.440545+0000","last_became_active":"2023-11-09T03:45:31.402623+0000","last_became_peered":"2023-11-09T03:45:31.402623+0000","last_unstale":"2023-11-09T03:45:35.440545+0000","last_undegraded":"2023-11-09T03:45:35.440545+0000","last_fullsized":"2023-11-09T03:45:35.440545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:36:23.593484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388073+0000","last_change":"2023-11-09T03:45:35.620447+0000","last_active":"2023-11-09T03:45:36.388073+0000","last_peered":"2023-11-09T03:45:36.388073+0000","last_clean":"2023-11-09T03:45:36.388073+0000","last_became_active":"2023-11-09T03:45:35.619949+0000","last_became_peered":"2023-11-09T03:45:35.619949+0000","last_unstale":"2023-11-09T03:45:36.388073+0000","last_undegraded":"2023-11-09T03:45:36.388073+0000","last_fullsized":"2023-11-09T03:45:36.388073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:13:00.499438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440637+0000","last_change":"2023-11-09T03:45:30.411660+0000","last_active":"2023-11-09T03:45:35.440637+0000","last_peered":"2023-11-09T03:45:35.440637+0000","last_clean":"2023-11-09T03:45:35.440637+0000","last_became_active":"2023-11-09T03:45:30.411274+0000","last_became_peered":"2023-11-09T03:45:30.411274+0000","last_unstale":"2023-11-09T03:45:35.440637+0000","last_undegraded":"2023-11-09T03:45:35.440637+0000","last_fullsized":"2023-11-09T03:45:35.440637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:39:29.323066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.444755+0000","last_change":"2023-11-09T03:45:31.415835+0000","last_active":"2023-11-09T03:45:35.444755+0000","last_peered":"2023-11-09T03:45:35.444755+0000","last_clean":"2023-11-09T03:45:35.444755+0000","last_became_active":"2023-11-09T03:45:31.415571+0000","last_became_peered":"2023-11-09T03:45:31.415571+0000","last_unstale":"2023-11-09T03:45:35.444755+0000","last_undegraded":"2023-11-09T03:45:35.444755+0000","last_fullsized":"2023-11-09T03:45:35.444755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:13:29.896024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388822+0000","last_change":"2023-11-09T03:45:35.445871+0000","last_active":"2023-11-09T03:45:36.388822+0000","last_peered":"2023-11-09T03:45:36.388822+0000","last_clean":"2023-11-09T03:45:36.388822+0000","last_became_active":"2023-11-09T03:45:35.445390+0000","last_became_peered":"2023-11-09T03:45:35.445390+0000","last_unstale":"2023-11-09T03:45:36.388822+0000","last_undegraded":"2023-11-09T03:45:36.388822+0000","last_fullsized":"2023-11-09T03:45:36.388822+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:26.681331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617937+0000","last_change":"2023-11-09T03:45:30.416124+0000","last_active":"2023-11-09T03:45:35.617937+0000","last_peered":"2023-11-09T03:45:35.617937+0000","last_clean":"2023-11-09T03:45:35.617937+0000","last_became_active":"2023-11-09T03:45:30.415850+0000","last_became_peered":"2023-11-09T03:45:30.415850+0000","last_unstale":"2023-11-09T03:45:35.617937+0000","last_undegraded":"2023-11-09T03:45:35.617937+0000","last_fullsized":"2023-11-09T03:45:35.617937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:43.468677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441010+0000","last_change":"2023-11-09T03:45:31.413149+0000","last_active":"2023-11-09T03:45:35.441010+0000","last_peered":"2023-11-09T03:45:35.441010+0000","last_clean":"2023-11-09T03:45:35.441010+0000","last_became_active":"2023-11-09T03:45:31.412875+0000","last_became_peered":"2023-11-09T03:45:31.412875+0000","last_unstale":"2023-11-09T03:45:35.441010+0000","last_undegraded":"2023-11-09T03:45:35.441010+0000","last_fullsized":"2023-11-09T03:45:35.441010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:04:02.978744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.389160+0000","last_change":"2023-11-09T03:45:35.444039+0000","last_active":"2023-11-09T03:45:36.389160+0000","last_peered":"2023-11-09T03:45:36.389160+0000","last_clean":"2023-11-09T03:45:36.389160+0000","last_became_active":"2023-11-09T03:45:35.443646+0000","last_became_peered":"2023-11-09T03:45:35.443646+0000","last_unstale":"2023-11-09T03:45:36.389160+0000","last_undegraded":"2023-11-09T03:45:36.389160+0000","last_fullsized":"2023-11-09T03:45:36.389160+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:02:31.124051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617462+0000","last_change":"2023-11-09T03:45:30.418418+0000","last_active":"2023-11-09T03:45:35.617462+0000","last_peered":"2023-11-09T03:45:35.617462+0000","last_clean":"2023-11-09T03:45:35.617462+0000","last_became_active":"2023-11-09T03:45:30.417963+0000","last_became_peered":"2023-11-09T03:45:30.417963+0000","last_unstale":"2023-11-09T03:45:35.617462+0000","last_undegraded":"2023-11-09T03:45:35.617462+0000","last_fullsized":"2023-11-09T03:45:35.617462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:07.347819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439849+0000","last_change":"2023-11-09T03:45:31.416138+0000","last_active":"2023-11-09T03:45:35.439849+0000","last_peered":"2023-11-09T03:45:35.439849+0000","last_clean":"2023-11-09T03:45:35.439849+0000","last_became_active":"2023-11-09T03:45:31.415636+0000","last_became_peered":"2023-11-09T03:45:31.415636+0000","last_unstale":"2023-11-09T03:45:35.439849+0000","last_undegraded":"2023-11-09T03:45:35.439849+0000","last_fullsized":"2023-11-09T03:45:35.439849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:02:27.095864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.389010+0000","last_change":"2023-11-09T03:45:35.448265+0000","last_active":"2023-11-09T03:45:36.389010+0000","last_peered":"2023-11-09T03:45:36.389010+0000","last_clean":"2023-11-09T03:45:36.389010+0000","last_became_active":"2023-11-09T03:45:35.447747+0000","last_became_peered":"2023-11-09T03:45:35.447747+0000","last_unstale":"2023-11-09T03:45:36.389010+0000","last_undegraded":"2023-11-09T03:45:36.389010+0000","last_fullsized":"2023-11-09T03:45:36.389010+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:39:46.409150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.437921+0000","last_change":"2023-11-09T03:45:31.400950+0000","last_active":"2023-11-09T03:45:35.437921+0000","last_peered":"2023-11-09T03:45:35.437921+0000","last_clean":"2023-11-09T03:45:35.437921+0000","last_became_active":"2023-11-09T03:45:31.400667+0000","last_became_peered":"2023-11-09T03:45:31.400667+0000","last_unstale":"2023-11-09T03:45:35.437921+0000","last_undegraded":"2023-11-09T03:45:35.437921+0000","last_fullsized":"2023-11-09T03:45:35.437921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:44.473424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617461+0000","last_change":"2023-11-09T03:45:30.418728+0000","last_active":"2023-11-09T03:45:35.617461+0000","last_peered":"2023-11-09T03:45:35.617461+0000","last_clean":"2023-11-09T03:45:35.617461+0000","last_became_active":"2023-11-09T03:45:30.417939+0000","last_became_peered":"2023-11-09T03:45:30.417939+0000","last_unstale":"2023-11-09T03:45:35.617461+0000","last_undegraded":"2023-11-09T03:45:35.617461+0000","last_fullsized":"2023-11-09T03:45:35.617461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:35:45.408209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.389355+0000","last_change":"2023-11-09T03:45:35.448352+0000","last_active":"2023-11-09T03:45:36.389355+0000","last_peered":"2023-11-09T03:45:36.389355+0000","last_clean":"2023-11-09T03:45:36.389355+0000","last_became_active":"2023-11-09T03:45:35.447889+0000","last_became_peered":"2023-11-09T03:45:35.447889+0000","last_unstale":"2023-11-09T03:45:36.389355+0000","last_undegraded":"2023-11-09T03:45:36.389355+0000","last_fullsized":"2023-11-09T03:45:36.389355+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:39:40.769191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439502+0000","last_change":"2023-11-09T03:45:31.398106+0000","last_active":"2023-11-09T03:45:35.439502+0000","last_peered":"2023-11-09T03:45:35.439502+0000","last_clean":"2023-11-09T03:45:35.439502+0000","last_became_active":"2023-11-09T03:45:31.397712+0000","last_became_peered":"2023-11-09T03:45:31.397712+0000","last_unstale":"2023-11-09T03:45:35.439502+0000","last_undegraded":"2023-11-09T03:45:35.439502+0000","last_fullsized":"2023-11-09T03:45:35.439502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:04:02.978744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441816+0000","last_change":"2023-11-09T03:45:30.408462+0000","last_active":"2023-11-09T03:45:35.441816+0000","last_peered":"2023-11-09T03:45:35.441816+0000","last_clean":"2023-11-09T03:45:35.441816+0000","last_became_active":"2023-11-09T03:45:30.408190+0000","last_became_peered":"2023-11-09T03:45:30.408190+0000","last_unstale":"2023-11-09T03:45:35.441816+0000","last_undegraded":"2023-11-09T03:45:35.441816+0000","last_fullsized":"2023-11-09T03:45:35.441816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:11:40.438915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.389721+0000","last_change":"2023-11-09T03:45:35.445519+0000","last_active":"2023-11-09T03:45:36.389721+0000","last_peered":"2023-11-09T03:45:36.389721+0000","last_clean":"2023-11-09T03:45:36.389721+0000","last_became_active":"2023-11-09T03:45:35.445125+0000","last_became_peered":"2023-11-09T03:45:35.445125+0000","last_unstale":"2023-11-09T03:45:36.389721+0000","last_undegraded":"2023-11-09T03:45:36.389721+0000","last_fullsized":"2023-11-09T03:45:36.389721+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:45.477938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439030+0000","last_change":"2023-11-09T03:45:31.402204+0000","last_active":"2023-11-09T03:45:35.439030+0000","last_peered":"2023-11-09T03:45:35.439030+0000","last_clean":"2023-11-09T03:45:35.439030+0000","last_became_active":"2023-11-09T03:45:31.401312+0000","last_became_peered":"2023-11-09T03:45:31.401312+0000","last_unstale":"2023-11-09T03:45:35.439030+0000","last_undegraded":"2023-11-09T03:45:35.439030+0000","last_fullsized":"2023-11-09T03:45:35.439030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:48:08.925647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441918+0000","last_change":"2023-11-09T03:45:30.414258+0000","last_active":"2023-11-09T03:45:35.441918+0000","last_peered":"2023-11-09T03:45:35.441918+0000","last_clean":"2023-11-09T03:45:35.441918+0000","last_became_active":"2023-11-09T03:45:30.413994+0000","last_became_peered":"2023-11-09T03:45:30.413994+0000","last_unstale":"2023-11-09T03:45:35.441918+0000","last_undegraded":"2023-11-09T03:45:35.441918+0000","last_fullsized":"2023-11-09T03:45:35.441918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:12:07.086030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.263008+0000","last_change":"2023-11-09T03:45:35.620757+0000","last_active":"2023-11-09T03:45:37.263008+0000","last_peered":"2023-11-09T03:45:37.263008+0000","last_clean":"2023-11-09T03:45:37.263008+0000","last_became_active":"2023-11-09T03:45:35.620219+0000","last_became_peered":"2023-11-09T03:45:35.620219+0000","last_unstale":"2023-11-09T03:45:37.263008+0000","last_undegraded":"2023-11-09T03:45:37.263008+0000","last_fullsized":"2023-11-09T03:45:37.263008+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:41:12.579051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.436441+0000","last_change":"2023-11-09T03:45:31.396755+0000","last_active":"2023-11-09T03:45:35.436441+0000","last_peered":"2023-11-09T03:45:35.436441+0000","last_clean":"2023-11-09T03:45:35.436441+0000","last_became_active":"2023-11-09T03:45:31.396475+0000","last_became_peered":"2023-11-09T03:45:31.396475+0000","last_unstale":"2023-11-09T03:45:35.436441+0000","last_undegraded":"2023-11-09T03:45:35.436441+0000","last_fullsized":"2023-11-09T03:45:35.436441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:07:02.262322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617133+0000","last_change":"2023-11-09T03:45:30.419151+0000","last_active":"2023-11-09T03:45:35.617133+0000","last_peered":"2023-11-09T03:45:35.617133+0000","last_clean":"2023-11-09T03:45:35.617133+0000","last_became_active":"2023-11-09T03:45:30.418874+0000","last_became_peered":"2023-11-09T03:45:30.418874+0000","last_unstale":"2023-11-09T03:45:35.617133+0000","last_undegraded":"2023-11-09T03:45:35.617133+0000","last_fullsized":"2023-11-09T03:45:35.617133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:34:48.400703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.389351+0000","last_change":"2023-11-09T03:45:35.445655+0000","last_active":"2023-11-09T03:45:36.389351+0000","last_peered":"2023-11-09T03:45:36.389351+0000","last_clean":"2023-11-09T03:45:36.389351+0000","last_became_active":"2023-11-09T03:45:35.445246+0000","last_became_peered":"2023-11-09T03:45:35.445246+0000","last_unstale":"2023-11-09T03:45:36.389351+0000","last_undegraded":"2023-11-09T03:45:36.389351+0000","last_fullsized":"2023-11-09T03:45:36.389351+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:04:07.006932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.443777+0000","last_change":"2023-11-09T03:45:31.397453+0000","last_active":"2023-11-09T03:45:35.443777+0000","last_peered":"2023-11-09T03:45:35.443777+0000","last_clean":"2023-11-09T03:45:35.443777+0000","last_became_active":"2023-11-09T03:45:31.396892+0000","last_became_peered":"2023-11-09T03:45:31.396892+0000","last_unstale":"2023-11-09T03:45:35.443777+0000","last_undegraded":"2023-11-09T03:45:35.443777+0000","last_fullsized":"2023-11-09T03:45:35.443777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:44.473424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440439+0000","last_change":"2023-11-09T03:45:30.416814+0000","last_active":"2023-11-09T03:45:35.440439+0000","last_peered":"2023-11-09T03:45:35.440439+0000","last_clean":"2023-11-09T03:45:35.440439+0000","last_became_active":"2023-11-09T03:45:30.416446+0000","last_became_peered":"2023-11-09T03:45:30.416446+0000","last_unstale":"2023-11-09T03:45:35.440439+0000","last_undegraded":"2023-11-09T03:45:35.440439+0000","last_fullsized":"2023-11-09T03:45:35.440439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:37:54.581788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388988+0000","last_change":"2023-11-09T03:45:35.618893+0000","last_active":"2023-11-09T03:45:36.388988+0000","last_peered":"2023-11-09T03:45:36.388988+0000","last_clean":"2023-11-09T03:45:36.388988+0000","last_became_active":"2023-11-09T03:45:35.618506+0000","last_became_peered":"2023-11-09T03:45:35.618506+0000","last_unstale":"2023-11-09T03:45:36.388988+0000","last_undegraded":"2023-11-09T03:45:36.388988+0000","last_fullsized":"2023-11-09T03:45:36.388988+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:36:27.621672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438491+0000","last_change":"2023-11-09T03:45:31.406885+0000","last_active":"2023-11-09T03:45:35.438491+0000","last_peered":"2023-11-09T03:45:35.438491+0000","last_clean":"2023-11-09T03:45:35.438491+0000","last_became_active":"2023-11-09T03:45:31.406620+0000","last_became_peered":"2023-11-09T03:45:31.406620+0000","last_unstale":"2023-11-09T03:45:35.438491+0000","last_undegraded":"2023-11-09T03:45:35.438491+0000","last_fullsized":"2023-11-09T03:45:35.438491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:29:17.503411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442020+0000","last_change":"2023-11-09T03:45:30.412534+0000","last_active":"2023-11-09T03:45:35.442020+0000","last_peered":"2023-11-09T03:45:35.442020+0000","last_clean":"2023-11-09T03:45:35.442020+0000","last_became_active":"2023-11-09T03:45:30.412015+0000","last_became_peered":"2023-11-09T03:45:30.412015+0000","last_unstale":"2023-11-09T03:45:35.442020+0000","last_undegraded":"2023-11-09T03:45:35.442020+0000","last_fullsized":"2023-11-09T03:45:35.442020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:34:48.400703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.387965+0000","last_change":"2023-11-09T03:45:35.620972+0000","last_active":"2023-11-09T03:45:36.387965+0000","last_peered":"2023-11-09T03:45:36.387965+0000","last_clean":"2023-11-09T03:45:36.387965+0000","last_became_active":"2023-11-09T03:45:35.620626+0000","last_became_peered":"2023-11-09T03:45:35.620626+0000","last_unstale":"2023-11-09T03:45:36.387965+0000","last_undegraded":"2023-11-09T03:45:36.387965+0000","last_fullsized":"2023-11-09T03:45:36.387965+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:03:57.174574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441966+0000","last_change":"2023-11-09T03:45:31.393932+0000","last_active":"2023-11-09T03:45:35.441966+0000","last_peered":"2023-11-09T03:45:35.441966+0000","last_clean":"2023-11-09T03:45:35.441966+0000","last_became_active":"2023-11-09T03:45:31.393669+0000","last_became_peered":"2023-11-09T03:45:31.393669+0000","last_unstale":"2023-11-09T03:45:35.441966+0000","last_undegraded":"2023-11-09T03:45:35.441966+0000","last_fullsized":"2023-11-09T03:45:35.441966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:27.400068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.618527+0000","last_change":"2023-11-09T03:45:30.419637+0000","last_active":"2023-11-09T03:45:35.618527+0000","last_peered":"2023-11-09T03:45:35.618527+0000","last_clean":"2023-11-09T03:45:35.618527+0000","last_became_active":"2023-11-09T03:45:30.419348+0000","last_became_peered":"2023-11-09T03:45:30.419348+0000","last_unstale":"2023-11-09T03:45:35.618527+0000","last_undegraded":"2023-11-09T03:45:35.618527+0000","last_fullsized":"2023-11-09T03:45:35.618527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:48:07.920901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.261912+0000","last_change":"2023-11-09T03:45:35.619361+0000","last_active":"2023-11-09T03:45:37.261912+0000","last_peered":"2023-11-09T03:45:37.261912+0000","last_clean":"2023-11-09T03:45:37.261912+0000","last_became_active":"2023-11-09T03:45:35.618998+0000","last_became_peered":"2023-11-09T03:45:35.618998+0000","last_unstale":"2023-11-09T03:45:37.261912+0000","last_undegraded":"2023-11-09T03:45:37.261912+0000","last_fullsized":"2023-11-09T03:45:37.261912+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:29:21.531598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439868+0000","last_change":"2023-11-09T03:45:31.408700+0000","last_active":"2023-11-09T03:45:35.439868+0000","last_peered":"2023-11-09T03:45:35.439868+0000","last_clean":"2023-11-09T03:45:35.439868+0000","last_became_active":"2023-11-09T03:45:31.408153+0000","last_became_peered":"2023-11-09T03:45:31.408153+0000","last_unstale":"2023-11-09T03:45:35.439868+0000","last_undegraded":"2023-11-09T03:45:35.439868+0000","last_fullsized":"2023-11-09T03:45:35.439868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:29:17.503411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442042+0000","last_change":"2023-11-09T03:45:30.409443+0000","last_active":"2023-11-09T03:45:35.442042+0000","last_peered":"2023-11-09T03:45:35.442042+0000","last_clean":"2023-11-09T03:45:35.442042+0000","last_became_active":"2023-11-09T03:45:30.409175+0000","last_became_peered":"2023-11-09T03:45:30.409175+0000","last_unstale":"2023-11-09T03:45:35.442042+0000","last_undegraded":"2023-11-09T03:45:35.442042+0000","last_fullsized":"2023-11-09T03:45:35.442042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:34:27.335423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262894+0000","last_change":"2023-11-09T03:45:35.452625+0000","last_active":"2023-11-09T03:45:37.262894+0000","last_peered":"2023-11-09T03:45:37.262894+0000","last_clean":"2023-11-09T03:45:37.262894+0000","last_became_active":"2023-11-09T03:45:35.451772+0000","last_became_peered":"2023-11-09T03:45:35.451772+0000","last_unstale":"2023-11-09T03:45:37.262894+0000","last_undegraded":"2023-11-09T03:45:37.262894+0000","last_fullsized":"2023-11-09T03:45:37.262894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:45.477938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.435623+0000","last_change":"2023-11-09T03:45:31.399046+0000","last_active":"2023-11-09T03:45:35.435623+0000","last_peered":"2023-11-09T03:45:35.435623+0000","last_clean":"2023-11-09T03:45:35.435623+0000","last_became_active":"2023-11-09T03:45:31.398698+0000","last_became_peered":"2023-11-09T03:45:31.398698+0000","last_unstale":"2023-11-09T03:45:35.435623+0000","last_undegraded":"2023-11-09T03:45:35.435623+0000","last_fullsized":"2023-11-09T03:45:35.435623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:35:46.412956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440345+0000","last_change":"2023-11-09T03:45:30.414519+0000","last_active":"2023-11-09T03:45:35.440345+0000","last_peered":"2023-11-09T03:45:35.440345+0000","last_clean":"2023-11-09T03:45:35.440345+0000","last_became_active":"2023-11-09T03:45:30.414025+0000","last_became_peered":"2023-11-09T03:45:30.414025+0000","last_unstale":"2023-11-09T03:45:35.440345+0000","last_undegraded":"2023-11-09T03:45:35.440345+0000","last_fullsized":"2023-11-09T03:45:35.440345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:43.468677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262508+0000","last_change":"2023-11-09T03:45:35.451076+0000","last_active":"2023-11-09T03:45:37.262508+0000","last_peered":"2023-11-09T03:45:37.262508+0000","last_clean":"2023-11-09T03:45:37.262508+0000","last_became_active":"2023-11-09T03:45:35.450721+0000","last_became_peered":"2023-11-09T03:45:35.450721+0000","last_unstale":"2023-11-09T03:45:37.262508+0000","last_undegraded":"2023-11-09T03:45:37.262508+0000","last_fullsized":"2023-11-09T03:45:37.262508+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:29:21.531598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.443831+0000","last_change":"2023-11-09T03:45:31.397098+0000","last_active":"2023-11-09T03:45:35.443831+0000","last_peered":"2023-11-09T03:45:35.443831+0000","last_clean":"2023-11-09T03:45:35.443831+0000","last_became_active":"2023-11-09T03:45:31.396835+0000","last_became_peered":"2023-11-09T03:45:31.396835+0000","last_unstale":"2023-11-09T03:45:35.443831+0000","last_undegraded":"2023-11-09T03:45:35.443831+0000","last_fullsized":"2023-11-09T03:45:35.443831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:54:30.809126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.432187+0000","last_change":"2023-11-09T03:45:30.411878+0000","last_active":"2023-11-09T03:45:35.432187+0000","last_peered":"2023-11-09T03:45:35.432187+0000","last_clean":"2023-11-09T03:45:35.432187+0000","last_became_active":"2023-11-09T03:45:30.411615+0000","last_became_peered":"2023-11-09T03:45:30.411615+0000","last_unstale":"2023-11-09T03:45:35.432187+0000","last_undegraded":"2023-11-09T03:45:35.432187+0000","last_fullsized":"2023-11-09T03:45:35.432187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:11:40.438915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.263008+0000","last_change":"2023-11-09T03:45:35.451994+0000","last_active":"2023-11-09T03:45:37.263008+0000","last_peered":"2023-11-09T03:45:37.263008+0000","last_clean":"2023-11-09T03:45:37.263008+0000","last_became_active":"2023-11-09T03:45:35.451623+0000","last_became_peered":"2023-11-09T03:45:35.451623+0000","last_unstale":"2023-11-09T03:45:37.263008+0000","last_undegraded":"2023-11-09T03:45:37.263008+0000","last_fullsized":"2023-11-09T03:45:37.263008+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:13:08.339968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441086+0000","last_change":"2023-11-09T03:45:31.400475+0000","last_active":"2023-11-09T03:45:35.441086+0000","last_peered":"2023-11-09T03:45:35.441086+0000","last_clean":"2023-11-09T03:45:35.441086+0000","last_became_active":"2023-11-09T03:45:31.400021+0000","last_became_peered":"2023-11-09T03:45:31.400021+0000","last_unstale":"2023-11-09T03:45:35.441086+0000","last_undegraded":"2023-11-09T03:45:35.441086+0000","last_fullsized":"2023-11-09T03:45:35.441086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:11:10.514126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617133+0000","last_change":"2023-11-09T03:45:30.417099+0000","last_active":"2023-11-09T03:45:35.617133+0000","last_peered":"2023-11-09T03:45:35.617133+0000","last_clean":"2023-11-09T03:45:35.617133+0000","last_became_active":"2023-11-09T03:45:30.416779+0000","last_became_peered":"2023-11-09T03:45:30.416779+0000","last_unstale":"2023-11-09T03:45:35.617133+0000","last_undegraded":"2023-11-09T03:45:35.617133+0000","last_fullsized":"2023-11-09T03:45:35.617133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:06:25.266825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262776+0000","last_change":"2023-11-09T03:45:35.618887+0000","last_active":"2023-11-09T03:45:37.262776+0000","last_peered":"2023-11-09T03:45:37.262776+0000","last_clean":"2023-11-09T03:45:37.262776+0000","last_became_active":"2023-11-09T03:45:35.618406+0000","last_became_peered":"2023-11-09T03:45:35.618406+0000","last_unstale":"2023-11-09T03:45:37.262776+0000","last_undegraded":"2023-11-09T03:45:37.262776+0000","last_fullsized":"2023-11-09T03:45:37.262776+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:41:12.579051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438270+0000","last_change":"2023-11-09T03:45:31.416419+0000","last_active":"2023-11-09T03:45:35.438270+0000","last_peered":"2023-11-09T03:45:35.438270+0000","last_clean":"2023-11-09T03:45:35.438270+0000","last_became_active":"2023-11-09T03:45:31.416137+0000","last_became_peered":"2023-11-09T03:45:31.416137+0000","last_unstale":"2023-11-09T03:45:35.438270+0000","last_undegraded":"2023-11-09T03:45:35.438270+0000","last_fullsized":"2023-11-09T03:45:35.438270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:08.352566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439728+0000","last_change":"2023-11-09T03:45:30.417920+0000","last_active":"2023-11-09T03:45:35.439728+0000","last_peered":"2023-11-09T03:45:35.439728+0000","last_clean":"2023-11-09T03:45:35.439728+0000","last_became_active":"2023-11-09T03:45:30.417578+0000","last_became_peered":"2023-11-09T03:45:30.417578+0000","last_unstale":"2023-11-09T03:45:35.439728+0000","last_undegraded":"2023-11-09T03:45:35.439728+0000","last_fullsized":"2023-11-09T03:45:35.439728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:07:01.257576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617231+0000","last_change":"2023-11-09T03:45:21.030926+0000","last_active":"2023-11-09T03:45:35.617231+0000","last_peered":"2023-11-09T03:45:35.617231+0000","last_clean":"2023-11-09T03:45:35.617231+0000","last_became_active":"2023-11-09T03:45:21.029801+0000","last_became_peered":"2023-11-09T03:45:21.029801+0000","last_unstale":"2023-11-09T03:45:35.617231+0000","last_undegraded":"2023-11-09T03:45:35.617231+0000","last_fullsized":"2023-11-09T03:45:35.617231+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_clean_scrub_stamp":"2023-11-09T03:45:19.768433+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:12:59.153713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.264013+0000","last_change":"2023-11-09T03:45:35.622614+0000","last_active":"2023-11-09T03:45:37.264013+0000","last_peered":"2023-11-09T03:45:37.264013+0000","last_clean":"2023-11-09T03:45:37.264013+0000","last_became_active":"2023-11-09T03:45:35.622138+0000","last_became_peered":"2023-11-09T03:45:35.622138+0000","last_unstale":"2023-11-09T03:45:37.264013+0000","last_undegraded":"2023-11-09T03:45:37.264013+0000","last_fullsized":"2023-11-09T03:45:37.264013+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:29:21.531598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617667+0000","last_change":"2023-11-09T03:45:31.417640+0000","last_active":"2023-11-09T03:45:35.617667+0000","last_peered":"2023-11-09T03:45:35.617667+0000","last_clean":"2023-11-09T03:45:35.617667+0000","last_became_active":"2023-11-09T03:45:31.417210+0000","last_became_peered":"2023-11-09T03:45:31.417210+0000","last_unstale":"2023-11-09T03:45:35.617667+0000","last_undegraded":"2023-11-09T03:45:35.617667+0000","last_fullsized":"2023-11-09T03:45:35.617667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:41.449750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.432324+0000","last_change":"2023-11-09T03:45:30.418193+0000","last_active":"2023-11-09T03:45:35.432324+0000","last_peered":"2023-11-09T03:45:35.432324+0000","last_clean":"2023-11-09T03:45:35.432324+0000","last_became_active":"2023-11-09T03:45:30.417874+0000","last_became_peered":"2023-11-09T03:45:30.417874+0000","last_unstale":"2023-11-09T03:45:35.432324+0000","last_undegraded":"2023-11-09T03:45:35.432324+0000","last_fullsized":"2023-11-09T03:45:35.432324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:34:27.335423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.263620+0000","last_change":"2023-11-09T03:45:35.618898+0000","last_active":"2023-11-09T03:45:37.263620+0000","last_peered":"2023-11-09T03:45:37.263620+0000","last_clean":"2023-11-09T03:45:37.263620+0000","last_became_active":"2023-11-09T03:45:35.618502+0000","last_became_peered":"2023-11-09T03:45:35.618502+0000","last_unstale":"2023-11-09T03:45:37.263620+0000","last_undegraded":"2023-11-09T03:45:37.263620+0000","last_fullsized":"2023-11-09T03:45:37.263620+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:39:40.769191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441100+0000","last_change":"2023-11-09T03:45:31.409910+0000","last_active":"2023-11-09T03:45:35.441100+0000","last_peered":"2023-11-09T03:45:35.441100+0000","last_clean":"2023-11-09T03:45:35.441100+0000","last_became_active":"2023-11-09T03:45:31.408863+0000","last_became_peered":"2023-11-09T03:45:31.408863+0000","last_unstale":"2023-11-09T03:45:35.441100+0000","last_undegraded":"2023-11-09T03:45:35.441100+0000","last_fullsized":"2023-11-09T03:45:35.441100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:41.449750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442413+0000","last_change":"2023-11-09T03:45:30.412199+0000","last_active":"2023-11-09T03:45:35.442413+0000","last_peered":"2023-11-09T03:45:35.442413+0000","last_clean":"2023-11-09T03:45:35.442413+0000","last_became_active":"2023-11-09T03:45:30.411864+0000","last_became_peered":"2023-11-09T03:45:30.411864+0000","last_unstale":"2023-11-09T03:45:35.442413+0000","last_undegraded":"2023-11-09T03:45:35.442413+0000","last_fullsized":"2023-11-09T03:45:35.442413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:13:28.891277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262128+0000","last_change":"2023-11-09T03:45:35.456250+0000","last_active":"2023-11-09T03:45:37.262128+0000","last_peered":"2023-11-09T03:45:37.262128+0000","last_clean":"2023-11-09T03:45:37.262128+0000","last_became_active":"2023-11-09T03:45:35.455866+0000","last_became_peered":"2023-11-09T03:45:35.455866+0000","last_unstale":"2023-11-09T03:45:37.262128+0000","last_undegraded":"2023-11-09T03:45:37.262128+0000","last_fullsized":"2023-11-09T03:45:37.262128+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:45.477938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.618920+0000","last_change":"2023-11-09T03:45:31.420110+0000","last_active":"2023-11-09T03:45:35.618920+0000","last_peered":"2023-11-09T03:45:35.618920+0000","last_clean":"2023-11-09T03:45:35.618920+0000","last_became_active":"2023-11-09T03:45:31.419556+0000","last_became_peered":"2023-11-09T03:45:31.419556+0000","last_unstale":"2023-11-09T03:45:35.618920+0000","last_undegraded":"2023-11-09T03:45:35.618920+0000","last_fullsized":"2023-11-09T03:45:35.618920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:51:24.601367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.432312+0000","last_change":"2023-11-09T03:45:30.415461+0000","last_active":"2023-11-09T03:45:35.432312+0000","last_peered":"2023-11-09T03:45:35.432312+0000","last_clean":"2023-11-09T03:45:35.432312+0000","last_became_active":"2023-11-09T03:45:30.415185+0000","last_became_peered":"2023-11-09T03:45:30.415185+0000","last_unstale":"2023-11-09T03:45:35.432312+0000","last_undegraded":"2023-11-09T03:45:35.432312+0000","last_fullsized":"2023-11-09T03:45:35.432312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:07:01.257576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262941+0000","last_change":"2023-11-09T03:45:35.620742+0000","last_active":"2023-11-09T03:45:37.262941+0000","last_peered":"2023-11-09T03:45:37.262941+0000","last_clean":"2023-11-09T03:45:37.262941+0000","last_became_active":"2023-11-09T03:45:35.620350+0000","last_became_peered":"2023-11-09T03:45:35.620350+0000","last_unstale":"2023-11-09T03:45:37.262941+0000","last_undegraded":"2023-11-09T03:45:37.262941+0000","last_fullsized":"2023-11-09T03:45:37.262941+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:03:57.174574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.618702+0000","last_change":"2023-11-09T03:45:31.417119+0000","last_active":"2023-11-09T03:45:35.618702+0000","last_peered":"2023-11-09T03:45:35.618702+0000","last_clean":"2023-11-09T03:45:35.618702+0000","last_became_active":"2023-11-09T03:45:31.416750+0000","last_became_peered":"2023-11-09T03:45:31.416750+0000","last_unstale":"2023-11-09T03:45:35.618702+0000","last_undegraded":"2023-11-09T03:45:35.618702+0000","last_fullsized":"2023-11-09T03:45:35.618702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:55:29.945168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.436205+0000","last_change":"2023-11-09T03:45:30.410204+0000","last_active":"2023-11-09T03:45:35.436205+0000","last_peered":"2023-11-09T03:45:35.436205+0000","last_clean":"2023-11-09T03:45:35.436205+0000","last_became_active":"2023-11-09T03:45:30.409876+0000","last_became_peered":"2023-11-09T03:45:30.409876+0000","last_unstale":"2023-11-09T03:45:35.436205+0000","last_undegraded":"2023-11-09T03:45:35.436205+0000","last_fullsized":"2023-11-09T03:45:35.436205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:23:49.677277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388781+0000","last_change":"2023-11-09T03:45:35.446608+0000","last_active":"2023-11-09T03:45:36.388781+0000","last_peered":"2023-11-09T03:45:36.388781+0000","last_clean":"2023-11-09T03:45:36.388781+0000","last_became_active":"2023-11-09T03:45:35.446121+0000","last_became_peered":"2023-11-09T03:45:35.446121+0000","last_unstale":"2023-11-09T03:45:36.388781+0000","last_undegraded":"2023-11-09T03:45:36.388781+0000","last_fullsized":"2023-11-09T03:45:36.388781+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:12.380753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617394+0000","last_change":"2023-11-09T03:45:31.418010+0000","last_active":"2023-11-09T03:45:35.617394+0000","last_peered":"2023-11-09T03:45:35.617394+0000","last_clean":"2023-11-09T03:45:35.617394+0000","last_became_active":"2023-11-09T03:45:31.417758+0000","last_became_peered":"2023-11-09T03:45:31.417758+0000","last_unstale":"2023-11-09T03:45:35.617394+0000","last_undegraded":"2023-11-09T03:45:35.617394+0000","last_fullsized":"2023-11-09T03:45:35.617394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:02:27.095864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.432121+0000","last_change":"2023-11-09T03:45:30.412289+0000","last_active":"2023-11-09T03:45:35.432121+0000","last_peered":"2023-11-09T03:45:35.432121+0000","last_clean":"2023-11-09T03:45:35.432121+0000","last_became_active":"2023-11-09T03:45:30.412035+0000","last_became_peered":"2023-11-09T03:45:30.412035+0000","last_unstale":"2023-11-09T03:45:35.432121+0000","last_undegraded":"2023-11-09T03:45:35.432121+0000","last_fullsized":"2023-11-09T03:45:35.432121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:35:45.408209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262660+0000","last_change":"2023-11-09T03:45:35.450569+0000","last_active":"2023-11-09T03:45:37.262660+0000","last_peered":"2023-11-09T03:45:37.262660+0000","last_clean":"2023-11-09T03:45:37.262660+0000","last_became_active":"2023-11-09T03:45:35.450133+0000","last_became_peered":"2023-11-09T03:45:35.450133+0000","last_unstale":"2023-11-09T03:45:37.262660+0000","last_undegraded":"2023-11-09T03:45:37.262660+0000","last_fullsized":"2023-11-09T03:45:37.262660+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:13:08.339968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440213+0000","last_change":"2023-11-09T03:45:31.400690+0000","last_active":"2023-11-09T03:45:35.440213+0000","last_peered":"2023-11-09T03:45:35.440213+0000","last_clean":"2023-11-09T03:45:35.440213+0000","last_became_active":"2023-11-09T03:45:31.400345+0000","last_became_peered":"2023-11-09T03:45:31.400345+0000","last_unstale":"2023-11-09T03:45:35.440213+0000","last_undegraded":"2023-11-09T03:45:35.440213+0000","last_fullsized":"2023-11-09T03:45:35.440213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:54:30.809126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438759+0000","last_change":"2023-11-09T03:45:30.414344+0000","last_active":"2023-11-09T03:45:35.438759+0000","last_peered":"2023-11-09T03:45:35.438759+0000","last_clean":"2023-11-09T03:45:35.438759+0000","last_became_active":"2023-11-09T03:45:30.414070+0000","last_became_peered":"2023-11-09T03:45:30.414070+0000","last_unstale":"2023-11-09T03:45:35.438759+0000","last_undegraded":"2023-11-09T03:45:35.438759+0000","last_fullsized":"2023-11-09T03:45:35.438759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:06:25.266825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388898+0000","last_change":"2023-11-09T03:45:35.618460+0000","last_active":"2023-11-09T03:45:36.388898+0000","last_peered":"2023-11-09T03:45:36.388898+0000","last_clean":"2023-11-09T03:45:36.388898+0000","last_became_active":"2023-11-09T03:45:35.618089+0000","last_became_peered":"2023-11-09T03:45:35.618089+0000","last_unstale":"2023-11-09T03:45:36.388898+0000","last_undegraded":"2023-11-09T03:45:36.388898+0000","last_fullsized":"2023-11-09T03:45:36.388898+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:24:34.808267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617389+0000","last_change":"2023-11-09T03:45:31.418108+0000","last_active":"2023-11-09T03:45:35.617389+0000","last_peered":"2023-11-09T03:45:35.617389+0000","last_clean":"2023-11-09T03:45:35.617389+0000","last_became_active":"2023-11-09T03:45:31.417869+0000","last_became_peered":"2023-11-09T03:45:31.417869+0000","last_unstale":"2023-11-09T03:45:35.617389+0000","last_undegraded":"2023-11-09T03:45:35.617389+0000","last_fullsized":"2023-11-09T03:45:35.617389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:04:02.978744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440273+0000","last_change":"2023-11-09T03:45:30.414815+0000","last_active":"2023-11-09T03:45:35.440273+0000","last_peered":"2023-11-09T03:45:35.440273+0000","last_clean":"2023-11-09T03:45:35.440273+0000","last_became_active":"2023-11-09T03:45:30.414177+0000","last_became_peered":"2023-11-09T03:45:30.414177+0000","last_unstale":"2023-11-09T03:45:35.440273+0000","last_undegraded":"2023-11-09T03:45:35.440273+0000","last_fullsized":"2023-11-09T03:45:35.440273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:54:29.804379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388529+0000","last_change":"2023-11-09T03:45:35.618416+0000","last_active":"2023-11-09T03:45:36.388529+0000","last_peered":"2023-11-09T03:45:36.388529+0000","last_clean":"2023-11-09T03:45:36.388529+0000","last_became_active":"2023-11-09T03:45:35.618055+0000","last_became_peered":"2023-11-09T03:45:35.618055+0000","last_unstale":"2023-11-09T03:45:36.388529+0000","last_undegraded":"2023-11-09T03:45:36.388529+0000","last_fullsized":"2023-11-09T03:45:36.388529+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:16:21.023356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.443358+0000","last_change":"2023-11-09T03:45:31.404777+0000","last_active":"2023-11-09T03:45:35.443358+0000","last_peered":"2023-11-09T03:45:35.443358+0000","last_clean":"2023-11-09T03:45:35.443358+0000","last_became_active":"2023-11-09T03:45:31.404472+0000","last_became_peered":"2023-11-09T03:45:31.404472+0000","last_unstale":"2023-11-09T03:45:35.443358+0000","last_undegraded":"2023-11-09T03:45:35.443358+0000","last_fullsized":"2023-11-09T03:45:35.443358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:06:26.271572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.443117+0000","last_change":"2023-11-09T03:45:30.411464+0000","last_active":"2023-11-09T03:45:35.443117+0000","last_peered":"2023-11-09T03:45:35.443117+0000","last_clean":"2023-11-09T03:45:35.443117+0000","last_became_active":"2023-11-09T03:45:30.410430+0000","last_became_peered":"2023-11-09T03:45:30.410430+0000","last_unstale":"2023-11-09T03:45:35.443117+0000","last_undegraded":"2023-11-09T03:45:35.443117+0000","last_fullsized":"2023-11-09T03:45:35.443117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:48:49.911609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262935+0000","last_change":"2023-11-09T03:45:35.618861+0000","last_active":"2023-11-09T03:45:37.262935+0000","last_peered":"2023-11-09T03:45:37.262935+0000","last_clean":"2023-11-09T03:45:37.262935+0000","last_became_active":"2023-11-09T03:45:35.618394+0000","last_became_peered":"2023-11-09T03:45:35.618394+0000","last_unstale":"2023-11-09T03:45:37.262935+0000","last_undegraded":"2023-11-09T03:45:37.262935+0000","last_fullsized":"2023-11-09T03:45:37.262935+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:07:23.630255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617562+0000","last_change":"2023-11-09T03:45:31.407151+0000","last_active":"2023-11-09T03:45:35.617562+0000","last_peered":"2023-11-09T03:45:35.617562+0000","last_clean":"2023-11-09T03:45:35.617562+0000","last_became_active":"2023-11-09T03:45:31.406652+0000","last_became_peered":"2023-11-09T03:45:31.406652+0000","last_unstale":"2023-11-09T03:45:35.617562+0000","last_undegraded":"2023-11-09T03:45:35.617562+0000","last_fullsized":"2023-11-09T03:45:35.617562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:29:17.503411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:41.806449+0000","last_change":"2023-11-09T03:45:30.411676+0000","last_active":"2023-11-09T03:45:41.806449+0000","last_peered":"2023-11-09T03:45:41.806449+0000","last_clean":"2023-11-09T03:45:41.806449+0000","last_became_active":"2023-11-09T03:45:30.411379+0000","last_became_peered":"2023-11-09T03:45:30.411379+0000","last_unstale":"2023-11-09T03:45:41.806449+0000","last_undegraded":"2023-11-09T03:45:41.806449+0000","last_fullsized":"2023-11-09T03:45:41.806449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:34:48.400703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262972+0000","last_change":"2023-11-09T03:45:35.452073+0000","last_active":"2023-11-09T03:45:37.262972+0000","last_peered":"2023-11-09T03:45:37.262972+0000","last_clean":"2023-11-09T03:45:37.262972+0000","last_became_active":"2023-11-09T03:45:35.451714+0000","last_became_peered":"2023-11-09T03:45:35.451714+0000","last_unstale":"2023-11-09T03:45:37.262972+0000","last_undegraded":"2023-11-09T03:45:37.262972+0000","last_fullsized":"2023-11-09T03:45:37.262972+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:55:33.973355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.435954+0000","last_change":"2023-11-09T03:45:31.406706+0000","last_active":"2023-11-09T03:45:35.435954+0000","last_peered":"2023-11-09T03:45:35.435954+0000","last_clean":"2023-11-09T03:45:35.435954+0000","last_became_active":"2023-11-09T03:45:31.406437+0000","last_became_peered":"2023-11-09T03:45:31.406437+0000","last_unstale":"2023-11-09T03:45:35.435954+0000","last_undegraded":"2023-11-09T03:45:35.435954+0000","last_fullsized":"2023-11-09T03:45:35.435954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:48:50.916355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442326+0000","last_change":"2023-11-09T03:45:30.418063+0000","last_active":"2023-11-09T03:45:35.442326+0000","last_peered":"2023-11-09T03:45:35.442326+0000","last_clean":"2023-11-09T03:45:35.442326+0000","last_became_active":"2023-11-09T03:45:30.417792+0000","last_became_peered":"2023-11-09T03:45:30.417792+0000","last_unstale":"2023-11-09T03:45:35.442326+0000","last_undegraded":"2023-11-09T03:45:35.442326+0000","last_fullsized":"2023-11-09T03:45:35.442326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:06:25.266825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.389474+0000","last_change":"2023-11-09T03:45:35.446788+0000","last_active":"2023-11-09T03:45:36.389474+0000","last_peered":"2023-11-09T03:45:36.389474+0000","last_clean":"2023-11-09T03:45:36.389474+0000","last_became_active":"2023-11-09T03:45:35.446280+0000","last_became_peered":"2023-11-09T03:45:35.446280+0000","last_unstale":"2023-11-09T03:45:36.389474+0000","last_undegraded":"2023-11-09T03:45:36.389474+0000","last_fullsized":"2023-11-09T03:45:36.389474+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:29:21.531598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.444838+0000","last_change":"2023-11-09T03:45:31.415653+0000","last_active":"2023-11-09T03:45:35.444838+0000","last_peered":"2023-11-09T03:45:35.444838+0000","last_clean":"2023-11-09T03:45:35.444838+0000","last_became_active":"2023-11-09T03:45:31.415219+0000","last_became_peered":"2023-11-09T03:45:31.415219+0000","last_unstale":"2023-11-09T03:45:35.444838+0000","last_undegraded":"2023-11-09T03:45:35.444838+0000","last_fullsized":"2023-11-09T03:45:35.444838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:37:55.586534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440583+0000","last_change":"2023-11-09T03:45:30.414817+0000","last_active":"2023-11-09T03:45:35.440583+0000","last_peered":"2023-11-09T03:45:35.440583+0000","last_clean":"2023-11-09T03:45:35.440583+0000","last_became_active":"2023-11-09T03:45:30.414124+0000","last_became_peered":"2023-11-09T03:45:30.414124+0000","last_unstale":"2023-11-09T03:45:35.440583+0000","last_undegraded":"2023-11-09T03:45:35.440583+0000","last_fullsized":"2023-11-09T03:45:35.440583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:11:40.438915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388126+0000","last_change":"2023-11-09T03:45:35.448965+0000","last_active":"2023-11-09T03:45:36.388126+0000","last_peered":"2023-11-09T03:45:36.388126+0000","last_clean":"2023-11-09T03:45:36.388126+0000","last_became_active":"2023-11-09T03:45:35.448513+0000","last_became_peered":"2023-11-09T03:45:35.448513+0000","last_unstale":"2023-11-09T03:45:36.388126+0000","last_undegraded":"2023-11-09T03:45:36.388126+0000","last_fullsized":"2023-11-09T03:45:36.388126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:57:39.650773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440495+0000","last_change":"2023-11-09T03:45:31.407514+0000","last_active":"2023-11-09T03:45:35.440495+0000","last_peered":"2023-11-09T03:45:35.440495+0000","last_clean":"2023-11-09T03:45:35.440495+0000","last_became_active":"2023-11-09T03:45:31.407223+0000","last_became_peered":"2023-11-09T03:45:31.407223+0000","last_unstale":"2023-11-09T03:45:35.440495+0000","last_undegraded":"2023-11-09T03:45:35.440495+0000","last_fullsized":"2023-11-09T03:45:35.440495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:55:29.945168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440341+0000","last_change":"2023-11-09T03:45:30.418184+0000","last_active":"2023-11-09T03:45:35.440341+0000","last_peered":"2023-11-09T03:45:35.440341+0000","last_clean":"2023-11-09T03:45:35.440341+0000","last_became_active":"2023-11-09T03:45:30.417910+0000","last_became_peered":"2023-11-09T03:45:30.417910+0000","last_unstale":"2023-11-09T03:45:35.440341+0000","last_undegraded":"2023-11-09T03:45:35.440341+0000","last_fullsized":"2023-11-09T03:45:35.440341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:13:28.891277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388139+0000","last_change":"2023-11-09T03:45:35.621180+0000","last_active":"2023-11-09T03:45:36.388139+0000","last_peered":"2023-11-09T03:45:36.388139+0000","last_clean":"2023-11-09T03:45:36.388139+0000","last_became_active":"2023-11-09T03:45:35.620827+0000","last_became_peered":"2023-11-09T03:45:35.620827+0000","last_unstale":"2023-11-09T03:45:36.388139+0000","last_undegraded":"2023-11-09T03:45:36.388139+0000","last_fullsized":"2023-11-09T03:45:36.388139+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:57:57.109949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262540+0000","last_change":"2023-11-09T03:45:35.620240+0000","last_active":"2023-11-09T03:45:37.262540+0000","last_peered":"2023-11-09T03:45:37.262540+0000","last_clean":"2023-11-09T03:45:37.262540+0000","last_became_active":"2023-11-09T03:45:35.619863+0000","last_became_peered":"2023-11-09T03:45:35.619863+0000","last_unstale":"2023-11-09T03:45:37.262540+0000","last_undegraded":"2023-11-09T03:45:37.262540+0000","last_fullsized":"2023-11-09T03:45:37.262540+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:55:33.973355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441745+0000","last_change":"2023-11-09T03:45:31.399005+0000","last_active":"2023-11-09T03:45:35.441745+0000","last_peered":"2023-11-09T03:45:35.441745+0000","last_clean":"2023-11-09T03:45:35.441745+0000","last_became_active":"2023-11-09T03:45:31.398753+0000","last_became_peered":"2023-11-09T03:45:31.398753+0000","last_unstale":"2023-11-09T03:45:35.441745+0000","last_undegraded":"2023-11-09T03:45:35.441745+0000","last_fullsized":"2023-11-09T03:45:35.441745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:44.473424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439574+0000","last_change":"2023-11-09T03:45:30.414502+0000","last_active":"2023-11-09T03:45:35.439574+0000","last_peered":"2023-11-09T03:45:35.439574+0000","last_clean":"2023-11-09T03:45:35.439574+0000","last_became_active":"2023-11-09T03:45:30.414208+0000","last_became_peered":"2023-11-09T03:45:30.414208+0000","last_unstale":"2023-11-09T03:45:35.439574+0000","last_undegraded":"2023-11-09T03:45:35.439574+0000","last_fullsized":"2023-11-09T03:45:35.439574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:34:48.400703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.264189+0000","last_change":"2023-11-09T03:45:35.456456+0000","last_active":"2023-11-09T03:45:37.264189+0000","last_peered":"2023-11-09T03:45:37.264189+0000","last_clean":"2023-11-09T03:45:37.264189+0000","last_became_active":"2023-11-09T03:45:35.455918+0000","last_became_peered":"2023-11-09T03:45:35.455918+0000","last_unstale":"2023-11-09T03:45:37.264189+0000","last_undegraded":"2023-11-09T03:45:37.264189+0000","last_fullsized":"2023-11-09T03:45:37.264189+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:45.477938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438961+0000","last_change":"2023-11-09T03:45:31.402076+0000","last_active":"2023-11-09T03:45:35.438961+0000","last_peered":"2023-11-09T03:45:35.438961+0000","last_clean":"2023-11-09T03:45:35.438961+0000","last_became_active":"2023-11-09T03:45:31.401819+0000","last_became_peered":"2023-11-09T03:45:31.401819+0000","last_unstale":"2023-11-09T03:45:35.438961+0000","last_undegraded":"2023-11-09T03:45:35.438961+0000","last_fullsized":"2023-11-09T03:45:35.438961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:27.400068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439935+0000","last_change":"2023-11-09T03:45:30.410155+0000","last_active":"2023-11-09T03:45:35.439935+0000","last_peered":"2023-11-09T03:45:35.439935+0000","last_clean":"2023-11-09T03:45:35.439935+0000","last_became_active":"2023-11-09T03:45:30.409732+0000","last_became_peered":"2023-11-09T03:45:30.409732+0000","last_unstale":"2023-11-09T03:45:35.439935+0000","last_undegraded":"2023-11-09T03:45:35.439935+0000","last_fullsized":"2023-11-09T03:45:35.439935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:34:48.400703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438019+0000","last_change":"2023-11-09T03:45:30.414923+0000","last_active":"2023-11-09T03:45:35.438019+0000","last_peered":"2023-11-09T03:45:35.438019+0000","last_clean":"2023-11-09T03:45:35.438019+0000","last_became_active":"2023-11-09T03:45:30.414641+0000","last_became_peered":"2023-11-09T03:45:30.414641+0000","last_unstale":"2023-11-09T03:45:35.438019+0000","last_undegraded":"2023-11-09T03:45:35.438019+0000","last_fullsized":"2023-11-09T03:45:35.438019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:12:07.086030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.443025+0000","last_change":"2023-11-09T03:45:31.400895+0000","last_active":"2023-11-09T03:45:35.443025+0000","last_peered":"2023-11-09T03:45:35.443025+0000","last_clean":"2023-11-09T03:45:35.443025+0000","last_became_active":"2023-11-09T03:45:31.400383+0000","last_became_peered":"2023-11-09T03:45:31.400383+0000","last_unstale":"2023-11-09T03:45:35.443025+0000","last_undegraded":"2023-11-09T03:45:35.443025+0000","last_fullsized":"2023-11-09T03:45:35.443025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:08.352566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.261594+0000","last_change":"2023-11-09T03:45:35.445171+0000","last_active":"2023-11-09T03:45:37.261594+0000","last_peered":"2023-11-09T03:45:37.261594+0000","last_clean":"2023-11-09T03:45:37.261594+0000","last_became_active":"2023-11-09T03:45:35.444835+0000","last_became_peered":"2023-11-09T03:45:35.444835+0000","last_unstale":"2023-11-09T03:45:37.261594+0000","last_undegraded":"2023-11-09T03:45:37.261594+0000","last_fullsized":"2023-11-09T03:45:37.261594+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:13:33.924212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439602+0000","last_change":"2023-11-09T03:45:30.416307+0000","last_active":"2023-11-09T03:45:35.439602+0000","last_peered":"2023-11-09T03:45:35.439602+0000","last_clean":"2023-11-09T03:45:35.439602+0000","last_became_active":"2023-11-09T03:45:30.416012+0000","last_became_peered":"2023-11-09T03:45:30.416012+0000","last_unstale":"2023-11-09T03:45:35.439602+0000","last_undegraded":"2023-11-09T03:45:35.439602+0000","last_fullsized":"2023-11-09T03:45:35.439602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:13:28.891277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.435281+0000","last_change":"2023-11-09T03:45:31.395421+0000","last_active":"2023-11-09T03:45:35.435281+0000","last_peered":"2023-11-09T03:45:35.435281+0000","last_clean":"2023-11-09T03:45:35.435281+0000","last_became_active":"2023-11-09T03:45:31.395148+0000","last_became_peered":"2023-11-09T03:45:31.395148+0000","last_unstale":"2023-11-09T03:45:35.435281+0000","last_undegraded":"2023-11-09T03:45:35.435281+0000","last_fullsized":"2023-11-09T03:45:35.435281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:34:28.340170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388718+0000","last_change":"2023-11-09T03:45:35.454065+0000","last_active":"2023-11-09T03:45:36.388718+0000","last_peered":"2023-11-09T03:45:36.388718+0000","last_clean":"2023-11-09T03:45:36.388718+0000","last_became_active":"2023-11-09T03:45:35.453677+0000","last_became_peered":"2023-11-09T03:45:35.453677+0000","last_unstale":"2023-11-09T03:45:36.388718+0000","last_undegraded":"2023-11-09T03:45:36.388718+0000","last_fullsized":"2023-11-09T03:45:36.388718+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:34:18.265468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440121+0000","last_change":"2023-11-09T03:45:31.401829+0000","last_active":"2023-11-09T03:45:35.440121+0000","last_peered":"2023-11-09T03:45:35.440121+0000","last_clean":"2023-11-09T03:45:35.440121+0000","last_became_active":"2023-11-09T03:45:31.401545+0000","last_became_peered":"2023-11-09T03:45:31.401545+0000","last_unstale":"2023-11-09T03:45:35.440121+0000","last_undegraded":"2023-11-09T03:45:35.440121+0000","last_fullsized":"2023-11-09T03:45:35.440121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:27.400068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.432141+0000","last_change":"2023-11-09T03:45:31.410286+0000","last_active":"2023-11-09T03:45:35.432141+0000","last_peered":"2023-11-09T03:45:35.432141+0000","last_clean":"2023-11-09T03:45:35.432141+0000","last_became_active":"2023-11-09T03:45:31.410031+0000","last_became_peered":"2023-11-09T03:45:31.410031+0000","last_unstale":"2023-11-09T03:45:35.432141+0000","last_undegraded":"2023-11-09T03:45:35.432141+0000","last_fullsized":"2023-11-09T03:45:35.432141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:48:50.916355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440196+0000","last_change":"2023-11-09T03:45:31.412282+0000","last_active":"2023-11-09T03:45:35.440196+0000","last_peered":"2023-11-09T03:45:35.440196+0000","last_clean":"2023-11-09T03:45:35.440196+0000","last_became_active":"2023-11-09T03:45:31.408879+0000","last_became_peered":"2023-11-09T03:45:31.408879+0000","last_unstale":"2023-11-09T03:45:35.440196+0000","last_undegraded":"2023-11-09T03:45:35.440196+0000","last_fullsized":"2023-11-09T03:45:35.440196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:39:42.380963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442948+0000","last_change":"2023-11-09T03:45:31.400563+0000","last_active":"2023-11-09T03:45:35.442948+0000","last_peered":"2023-11-09T03:45:35.442948+0000","last_clean":"2023-11-09T03:45:35.442948+0000","last_became_active":"2023-11-09T03:45:31.399929+0000","last_became_peered":"2023-11-09T03:45:31.399929+0000","last_unstale":"2023-11-09T03:45:35.442948+0000","last_undegraded":"2023-11-09T03:45:35.442948+0000","last_fullsized":"2023-11-09T03:45:35.442948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:48:08.925647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439486+0000","last_change":"2023-11-09T03:45:31.397966+0000","last_active":"2023-11-09T03:45:35.439486+0000","last_peered":"2023-11-09T03:45:35.439486+0000","last_clean":"2023-11-09T03:45:35.439486+0000","last_became_active":"2023-11-09T03:45:31.397681+0000","last_became_peered":"2023-11-09T03:45:31.397681+0000","last_unstale":"2023-11-09T03:45:35.439486+0000","last_undegraded":"2023-11-09T03:45:35.439486+0000","last_fullsized":"2023-11-09T03:45:35.439486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:22.653144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442230+0000","last_change":"2023-11-09T03:45:31.398359+0000","last_active":"2023-11-09T03:45:35.442230+0000","last_peered":"2023-11-09T03:45:35.442230+0000","last_clean":"2023-11-09T03:45:35.442230+0000","last_became_active":"2023-11-09T03:45:31.398104+0000","last_became_peered":"2023-11-09T03:45:31.398104+0000","last_unstale":"2023-11-09T03:45:35.442230+0000","last_undegraded":"2023-11-09T03:45:35.442230+0000","last_fullsized":"2023-11-09T03:45:35.442230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:08.352566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440856+0000","last_change":"2023-11-09T03:45:31.408644+0000","last_active":"2023-11-09T03:45:35.440856+0000","last_peered":"2023-11-09T03:45:35.440856+0000","last_clean":"2023-11-09T03:45:35.440856+0000","last_became_active":"2023-11-09T03:45:31.408353+0000","last_became_peered":"2023-11-09T03:45:31.408353+0000","last_unstale":"2023-11-09T03:45:35.440856+0000","last_undegraded":"2023-11-09T03:45:35.440856+0000","last_fullsized":"2023-11-09T03:45:35.440856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:36:23.593484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438417+0000","last_change":"2023-11-09T03:45:31.395367+0000","last_active":"2023-11-09T03:45:35.438417+0000","last_peered":"2023-11-09T03:45:35.438417+0000","last_clean":"2023-11-09T03:45:35.438417+0000","last_became_active":"2023-11-09T03:45:31.395059+0000","last_became_peered":"2023-11-09T03:45:31.395059+0000","last_unstale":"2023-11-09T03:45:35.438417+0000","last_undegraded":"2023-11-09T03:45:35.438417+0000","last_fullsized":"2023-11-09T03:45:35.438417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:39:36.741003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439489+0000","last_change":"2023-11-09T03:45:31.408626+0000","last_active":"2023-11-09T03:45:35.439489+0000","last_peered":"2023-11-09T03:45:35.439489+0000","last_clean":"2023-11-09T03:45:35.439489+0000","last_became_active":"2023-11-09T03:45:31.408085+0000","last_became_peered":"2023-11-09T03:45:31.408085+0000","last_unstale":"2023-11-09T03:45:35.439489+0000","last_undegraded":"2023-11-09T03:45:35.439489+0000","last_fullsized":"2023-11-09T03:45:35.439489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:08.352566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439241+0000","last_change":"2023-11-09T03:45:31.402017+0000","last_active":"2023-11-09T03:45:35.439241+0000","last_peered":"2023-11-09T03:45:35.439241+0000","last_clean":"2023-11-09T03:45:35.439241+0000","last_became_active":"2023-11-09T03:45:31.401773+0000","last_became_peered":"2023-11-09T03:45:31.401773+0000","last_unstale":"2023-11-09T03:45:35.439241+0000","last_undegraded":"2023-11-09T03:45:35.439241+0000","last_fullsized":"2023-11-09T03:45:35.439241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:39:42.380963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440106+0000","last_change":"2023-11-09T03:45:31.418163+0000","last_active":"2023-11-09T03:45:35.440106+0000","last_peered":"2023-11-09T03:45:35.440106+0000","last_clean":"2023-11-09T03:45:35.440106+0000","last_became_active":"2023-11-09T03:45:31.417880+0000","last_became_peered":"2023-11-09T03:45:31.417880+0000","last_unstale":"2023-11-09T03:45:35.440106+0000","last_undegraded":"2023-11-09T03:45:35.440106+0000","last_fullsized":"2023-11-09T03:45:35.440106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:39:36.741003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442845+0000","last_change":"2023-11-09T03:45:31.404842+0000","last_active":"2023-11-09T03:45:35.442845+0000","last_peered":"2023-11-09T03:45:35.442845+0000","last_clean":"2023-11-09T03:45:35.442845+0000","last_became_active":"2023-11-09T03:45:31.404484+0000","last_became_peered":"2023-11-09T03:45:31.404484+0000","last_unstale":"2023-11-09T03:45:35.442845+0000","last_undegraded":"2023-11-09T03:45:35.442845+0000","last_fullsized":"2023-11-09T03:45:35.442845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:27.400068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440223+0000","last_change":"2023-11-09T03:45:31.404211+0000","last_active":"2023-11-09T03:45:35.440223+0000","last_peered":"2023-11-09T03:45:35.440223+0000","last_clean":"2023-11-09T03:45:35.440223+0000","last_became_active":"2023-11-09T03:45:31.403962+0000","last_became_peered":"2023-11-09T03:45:31.403962+0000","last_unstale":"2023-11-09T03:45:35.440223+0000","last_undegraded":"2023-11-09T03:45:35.440223+0000","last_fullsized":"2023-11-09T03:45:35.440223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:48:08.925647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.619170+0000","last_change":"2023-11-09T03:45:31.407273+0000","last_active":"2023-11-09T03:45:35.619170+0000","last_peered":"2023-11-09T03:45:35.619170+0000","last_clean":"2023-11-09T03:45:35.619170+0000","last_became_active":"2023-11-09T03:45:31.406956+0000","last_became_peered":"2023-11-09T03:45:31.406956+0000","last_unstale":"2023-11-09T03:45:35.619170+0000","last_undegraded":"2023-11-09T03:45:35.619170+0000","last_fullsized":"2023-11-09T03:45:35.619170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:13:04.311780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.432241+0000","last_change":"2023-11-09T03:45:31.402525+0000","last_active":"2023-11-09T03:45:35.432241+0000","last_peered":"2023-11-09T03:45:35.432241+0000","last_clean":"2023-11-09T03:45:35.432241+0000","last_became_active":"2023-11-09T03:45:31.402262+0000","last_became_peered":"2023-11-09T03:45:31.402262+0000","last_unstale":"2023-11-09T03:45:35.432241+0000","last_undegraded":"2023-11-09T03:45:35.432241+0000","last_fullsized":"2023-11-09T03:45:35.432241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:09.007153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441402+0000","last_change":"2023-11-09T03:45:31.411781+0000","last_active":"2023-11-09T03:45:35.441402+0000","last_peered":"2023-11-09T03:45:35.441402+0000","last_clean":"2023-11-09T03:45:35.441402+0000","last_became_active":"2023-11-09T03:45:31.411178+0000","last_became_peered":"2023-11-09T03:45:31.411178+0000","last_unstale":"2023-11-09T03:45:35.441402+0000","last_undegraded":"2023-11-09T03:45:35.441402+0000","last_fullsized":"2023-11-09T03:45:35.441402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:13:04.311780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439393+0000","last_change":"2023-11-09T03:45:31.409602+0000","last_active":"2023-11-09T03:45:35.439393+0000","last_peered":"2023-11-09T03:45:35.439393+0000","last_clean":"2023-11-09T03:45:35.439393+0000","last_became_active":"2023-11-09T03:45:31.409313+0000","last_became_peered":"2023-11-09T03:45:31.409313+0000","last_unstale":"2023-11-09T03:45:35.439393+0000","last_undegraded":"2023-11-09T03:45:35.439393+0000","last_fullsized":"2023-11-09T03:45:35.439393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:48:08.925647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441829+0000","last_change":"2023-11-09T03:45:31.394559+0000","last_active":"2023-11-09T03:45:35.441829+0000","last_peered":"2023-11-09T03:45:35.441829+0000","last_clean":"2023-11-09T03:45:35.441829+0000","last_became_active":"2023-11-09T03:45:31.394284+0000","last_became_peered":"2023-11-09T03:45:31.394284+0000","last_unstale":"2023-11-09T03:45:35.441829+0000","last_undegraded":"2023-11-09T03:45:35.441829+0000","last_fullsized":"2023-11-09T03:45:35.441829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:11:10.514126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.443281+0000","last_change":"2023-11-09T03:45:31.397341+0000","last_active":"2023-11-09T03:45:35.443281+0000","last_peered":"2023-11-09T03:45:35.443281+0000","last_clean":"2023-11-09T03:45:35.443281+0000","last_became_active":"2023-11-09T03:45:31.396986+0000","last_became_peered":"2023-11-09T03:45:31.396986+0000","last_unstale":"2023-11-09T03:45:35.443281+0000","last_undegraded":"2023-11-09T03:45:35.443281+0000","last_fullsized":"2023-11-09T03:45:35.443281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:34:49.405450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439412+0000","last_change":"2023-11-09T03:45:31.402346+0000","last_active":"2023-11-09T03:45:35.439412+0000","last_peered":"2023-11-09T03:45:35.439412+0000","last_clean":"2023-11-09T03:45:35.439412+0000","last_became_active":"2023-11-09T03:45:31.401630+0000","last_became_peered":"2023-11-09T03:45:31.401630+0000","last_unstale":"2023-11-09T03:45:35.439412+0000","last_undegraded":"2023-11-09T03:45:35.439412+0000","last_fullsized":"2023-11-09T03:45:35.439412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:02:27.095864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.435980+0000","last_change":"2023-11-09T03:45:31.403882+0000","last_active":"2023-11-09T03:45:35.435980+0000","last_peered":"2023-11-09T03:45:35.435980+0000","last_clean":"2023-11-09T03:45:35.435980+0000","last_became_active":"2023-11-09T03:45:31.403637+0000","last_became_peered":"2023-11-09T03:45:31.403637+0000","last_unstale":"2023-11-09T03:45:35.435980+0000","last_undegraded":"2023-11-09T03:45:35.435980+0000","last_fullsized":"2023-11-09T03:45:35.435980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:39:30.327813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438776+0000","last_change":"2023-11-09T03:45:31.415358+0000","last_active":"2023-11-09T03:45:35.438776+0000","last_peered":"2023-11-09T03:45:35.438776+0000","last_clean":"2023-11-09T03:45:35.438776+0000","last_became_active":"2023-11-09T03:45:31.415018+0000","last_became_peered":"2023-11-09T03:45:31.415018+0000","last_unstale":"2023-11-09T03:45:35.438776+0000","last_undegraded":"2023-11-09T03:45:35.438776+0000","last_fullsized":"2023-11-09T03:45:35.438776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:08.352566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440770+0000","last_change":"2023-11-09T03:45:31.414693+0000","last_active":"2023-11-09T03:45:35.440770+0000","last_peered":"2023-11-09T03:45:35.440770+0000","last_clean":"2023-11-09T03:45:35.440770+0000","last_became_active":"2023-11-09T03:45:31.414400+0000","last_became_peered":"2023-11-09T03:45:31.414400+0000","last_unstale":"2023-11-09T03:45:35.440770+0000","last_undegraded":"2023-11-09T03:45:35.440770+0000","last_fullsized":"2023-11-09T03:45:35.440770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:22.653144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.618910+0000","last_change":"2023-11-09T03:45:31.419560+0000","last_active":"2023-11-09T03:45:35.618910+0000","last_peered":"2023-11-09T03:45:35.618910+0000","last_clean":"2023-11-09T03:45:35.618910+0000","last_became_active":"2023-11-09T03:45:31.418455+0000","last_became_peered":"2023-11-09T03:45:31.418455+0000","last_unstale":"2023-11-09T03:45:35.618910+0000","last_undegraded":"2023-11-09T03:45:35.618910+0000","last_fullsized":"2023-11-09T03:45:35.618910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:14:55.198635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438871+0000","last_change":"2023-11-09T03:45:31.415460+0000","last_active":"2023-11-09T03:45:35.438871+0000","last_peered":"2023-11-09T03:45:35.438871+0000","last_clean":"2023-11-09T03:45:35.438871+0000","last_became_active":"2023-11-09T03:45:31.415210+0000","last_became_peered":"2023-11-09T03:45:31.415210+0000","last_unstale":"2023-11-09T03:45:35.438871+0000","last_undegraded":"2023-11-09T03:45:35.438871+0000","last_fullsized":"2023-11-09T03:45:35.438871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:44.473424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617339+0000","last_change":"2023-11-09T03:45:31.411588+0000","last_active":"2023-11-09T03:45:35.617339+0000","last_peered":"2023-11-09T03:45:35.617339+0000","last_clean":"2023-11-09T03:45:35.617339+0000","last_became_active":"2023-11-09T03:45:31.411168+0000","last_became_peered":"2023-11-09T03:45:31.411168+0000","last_unstale":"2023-11-09T03:45:35.617339+0000","last_undegraded":"2023-11-09T03:45:35.617339+0000","last_fullsized":"2023-11-09T03:45:35.617339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:39:42.380963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440027+0000","last_change":"2023-11-09T03:45:31.412481+0000","last_active":"2023-11-09T03:45:35.440027+0000","last_peered":"2023-11-09T03:45:35.440027+0000","last_clean":"2023-11-09T03:45:35.440027+0000","last_became_active":"2023-11-09T03:45:31.412167+0000","last_became_peered":"2023-11-09T03:45:31.412167+0000","last_unstale":"2023-11-09T03:45:35.440027+0000","last_undegraded":"2023-11-09T03:45:35.440027+0000","last_fullsized":"2023-11-09T03:45:35.440027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:22.653144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617339+0000","last_change":"2023-11-09T03:45:31.405459+0000","last_active":"2023-11-09T03:45:35.617339+0000","last_peered":"2023-11-09T03:45:35.617339+0000","last_clean":"2023-11-09T03:45:35.617339+0000","last_became_active":"2023-11-09T03:45:31.405188+0000","last_became_peered":"2023-11-09T03:45:31.405188+0000","last_unstale":"2023-11-09T03:45:35.617339+0000","last_undegraded":"2023-11-09T03:45:35.617339+0000","last_fullsized":"2023-11-09T03:45:35.617339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:39:36.741003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440927+0000","last_change":"2023-11-09T03:45:31.417756+0000","last_active":"2023-11-09T03:45:35.440927+0000","last_peered":"2023-11-09T03:45:35.440927+0000","last_clean":"2023-11-09T03:45:35.440927+0000","last_became_active":"2023-11-09T03:45:31.417488+0000","last_became_peered":"2023-11-09T03:45:31.417488+0000","last_unstale":"2023-11-09T03:45:35.440927+0000","last_undegraded":"2023-11-09T03:45:35.440927+0000","last_fullsized":"2023-11-09T03:45:35.440927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:36:23.593484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442615+0000","last_change":"2023-11-09T03:45:31.397411+0000","last_active":"2023-11-09T03:45:35.442615+0000","last_peered":"2023-11-09T03:45:35.442615+0000","last_clean":"2023-11-09T03:45:35.442615+0000","last_became_active":"2023-11-09T03:45:31.396779+0000","last_became_peered":"2023-11-09T03:45:31.396779+0000","last_unstale":"2023-11-09T03:45:35.442615+0000","last_undegraded":"2023-11-09T03:45:35.442615+0000","last_fullsized":"2023-11-09T03:45:35.442615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:54:30.809126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441937+0000","last_change":"2023-11-09T03:45:31.396447+0000","last_active":"2023-11-09T03:45:35.441937+0000","last_peered":"2023-11-09T03:45:35.441937+0000","last_clean":"2023-11-09T03:45:35.441937+0000","last_became_active":"2023-11-09T03:45:31.396184+0000","last_became_peered":"2023-11-09T03:45:31.396184+0000","last_unstale":"2023-11-09T03:45:35.441937+0000","last_undegraded":"2023-11-09T03:45:35.441937+0000","last_fullsized":"2023-11-09T03:45:35.441937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:48:08.925647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.618840+0000","last_change":"2023-11-09T03:45:31.405664+0000","last_active":"2023-11-09T03:45:35.618840+0000","last_peered":"2023-11-09T03:45:35.618840+0000","last_clean":"2023-11-09T03:45:35.618840+0000","last_became_active":"2023-11-09T03:45:31.404726+0000","last_became_peered":"2023-11-09T03:45:31.404726+0000","last_unstale":"2023-11-09T03:45:35.618840+0000","last_undegraded":"2023-11-09T03:45:35.618840+0000","last_fullsized":"2023-11-09T03:45:35.618840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:41:08.550864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":695413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":104553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":104589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.216}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":104623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.252}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":868352,"data_stored":694850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-09T03:45:55.423 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-09T03:45:55.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.558+0000 7f208efa9640 1 Processor -- start 2023-11-09T03:45:55.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.558+0000 7f208efa9640 1 -- start start 2023-11-09T03:45:55.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.559+0000 7f208efa9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2088159c70 0x7f20880ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:55.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.559+0000 7f208efa9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f208806db20 0x7f20880b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:55.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.559+0000 7f208efa9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20880b4110 con 0x7f20880b0910 2023-11-09T03:45:55.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.559+0000 7f208efa9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20880b49b0 con 0x7f2088159c70 2023-11-09T03:45:55.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.559+0000 7f208efa9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20880b0da0 con 0x7f208806db20 2023-11-09T03:45:55.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.559+0000 7f208cd45640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2088159c70 0x7f20880ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:55.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.559+0000 7f208cd45640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2088159c70 0x7f20880ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36468/0 (socket says 172.21.15.76:36468) 2023-11-09T03:45:55.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.559+0000 7f208cd45640 1 -- 172.21.15.76:0/2736852120 learned_addr learned my addr 172.21.15.76:0/2736852120 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:55.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.559+0000 7f2087fff640 1 --2- 172.21.15.76:0/2736852120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f208806db20 0x7f20880b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:55.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.559+0000 7f208cd45640 1 -- 172.21.15.76:0/2736852120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f208806db20 msgr2=0x7f20880b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:55.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.559+0000 7f208cd45640 1 --2- 172.21.15.76:0/2736852120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f208806db20 0x7f20880b03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.559+0000 7f208cd45640 1 -- 172.21.15.76:0/2736852120 >> v1:172.21.15.113:6789/0 conn(0x7f20880b0910 legacy=0x7f208815d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:55.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.559+0000 7f208cd45640 1 -- 172.21.15.76:0/2736852120 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20880b43f0 con 0x7f2088159c70 2023-11-09T03:45:55.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.560+0000 7f208cd45640 1 --2- 172.21.15.76:0/2736852120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2088159c70 0x7f20880ce370 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2078009870 tx=0x7f207802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cc87da6ea478fb server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:55.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.560+0000 7f20877fe640 1 -- 172.21.15.76:0/2736852120 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2078030840 con 0x7f2088159c70 2023-11-09T03:45:55.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.560+0000 7f20877fe640 1 -- 172.21.15.76:0/2736852120 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20780309a0 con 0x7f2088159c70 2023-11-09T03:45:55.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.560+0000 7f20877fe640 1 -- 172.21.15.76:0/2736852120 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2078030d30 con 0x7f2088159c70 2023-11-09T03:45:55.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.561+0000 7f208efa9640 1 -- 172.21.15.76:0/2736852120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2088159c70 msgr2=0x7f20880ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:55.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.561+0000 7f208efa9640 1 --2- 172.21.15.76:0/2736852120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2088159c70 0x7f20880ce370 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2078009870 tx=0x7f207802b980 comp rx=0 tx=0).stop 2023-11-09T03:45:55.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.561+0000 7f208efa9640 1 -- 172.21.15.76:0/2736852120 shutdown_connections 2023-11-09T03:45:55.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.561+0000 7f208efa9640 1 --2- 172.21.15.76:0/2736852120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f208806db20 0x7f20880b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.561+0000 7f208efa9640 1 --2- 172.21.15.76:0/2736852120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2088159c70 0x7f20880ce370 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.561+0000 7f208efa9640 1 -- 172.21.15.76:0/2736852120 >> 172.21.15.76:0/2736852120 conn(0x7f20880ce950 msgr2=0x7f20880ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:55.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.561+0000 7f208efa9640 1 -- 172.21.15.76:0/2736852120 shutdown_connections 2023-11-09T03:45:55.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.561+0000 7f208efa9640 1 -- 172.21.15.76:0/2736852120 wait complete. 2023-11-09T03:45:55.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.562+0000 7f208efa9640 1 Processor -- start 2023-11-09T03:45:55.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.562+0000 7f208efa9640 1 -- start start 2023-11-09T03:45:55.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.563+0000 7f208efa9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f208806db20 0x7f2088153b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:55.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.563+0000 7f208efa9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20880b0910 0x7f20880cd030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:55.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.563+0000 7f208efa9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2088159c70 0x7f208814ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:55.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.563+0000 7f208efa9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20881501b0 con 0x7f2088159c70 2023-11-09T03:45:55.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.563+0000 7f208efa9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208815d140 con 0x7f208806db20 2023-11-09T03:45:55.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.563+0000 7f208efa9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2088154070 con 0x7f20880b0910 2023-11-09T03:45:55.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.563+0000 7f208d546640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2088159c70 0x7f208814ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:55.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.563+0000 7f208d546640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2088159c70 0x7f208814ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36474/0 (socket says 172.21.15.76:36474) 2023-11-09T03:45:55.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.563+0000 7f208d546640 1 -- 172.21.15.76:0/905269798 learned_addr learned my addr 172.21.15.76:0/905269798 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:55.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.563+0000 7f208cd45640 1 --2- 172.21.15.76:0/905269798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f208806db20 0x7f2088153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:55.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.563+0000 7f2087fff640 1 --2- 172.21.15.76:0/905269798 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20880b0910 0x7f20880cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:55.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.563+0000 7f208d546640 1 -- 172.21.15.76:0/905269798 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20880b0910 msgr2=0x7f20880cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:55.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.563+0000 7f208d546640 1 --2- 172.21.15.76:0/905269798 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20880b0910 0x7f20880cd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.563+0000 7f208d546640 1 -- 172.21.15.76:0/905269798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f208806db20 msgr2=0x7f2088153b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:55.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.563+0000 7f208d546640 1 --2- 172.21.15.76:0/905269798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f208806db20 0x7f2088153b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.563+0000 7f208d546640 1 -- 172.21.15.76:0/905269798 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20780094d0 con 0x7f2088159c70 2023-11-09T03:45:55.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.563+0000 7f208d546640 1 --2- 172.21.15.76:0/905269798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2088159c70 0x7f208814ad60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f208000c7a0 tx=0x7f208000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:55.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.564+0000 7f20857fa640 1 -- 172.21.15.76:0/905269798 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2080003b70 con 0x7f2088159c70 2023-11-09T03:45:55.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.564+0000 7f20857fa640 1 -- 172.21.15.76:0/905269798 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2080003cf0 con 0x7f2088159c70 2023-11-09T03:45:55.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.564+0000 7f208efa9640 1 -- 172.21.15.76:0/905269798 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20880c8260 con 0x7f2088159c70 2023-11-09T03:45:55.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.564+0000 7f20857fa640 1 -- 172.21.15.76:0/905269798 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2080003ea0 con 0x7f2088159c70 2023-11-09T03:45:55.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.564+0000 7f208efa9640 1 -- 172.21.15.76:0/905269798 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f208815cdd0 con 0x7f2088159c70 2023-11-09T03:45:55.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.565+0000 7f208efa9640 1 -- 172.21.15.76:0/905269798 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2088153110 con 0x7f2088159c70 2023-11-09T03:45:55.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.565+0000 7f20857fa640 1 -- 172.21.15.76:0/905269798 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f208000aa90 con 0x7f2088159c70 2023-11-09T03:45:55.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.565+0000 7f20857fa640 1 --2- 172.21.15.76:0/905269798 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f207406ae70 0x7f207406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:55.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.566+0000 7f20857fa640 1 -- 172.21.15.76:0/905269798 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2080089540 con 0x7f2088159c70 2023-11-09T03:45:55.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.566+0000 7f208cd45640 1 --2- 172.21.15.76:0/905269798 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f207406ae70 0x7f207406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:55.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.566+0000 7f208cd45640 1 --2- 172.21.15.76:0/905269798 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f207406ae70 0x7f207406d330 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2078036000 tx=0x7f2078008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:55.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.570+0000 7f20857fa640 1 -- 172.21.15.76:0/905269798 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2080054440 con 0x7f2088159c70 2023-11-09T03:45:55.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.735+0000 7f208efa9640 1 -- 172.21.15.76:0/905269798 --> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f208815c970 con 0x7f207406ae70 2023-11-09T03:45:55.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.744+0000 7f20857fa640 1 -- 172.21.15.76:0/905269798 <== mgr.4105 v2:172.21.15.113:6832/1264193042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345247 (secure 0 0 0) 0x7f2078030b90 con 0x7f207406ae70 2023-11-09T03:45:55.746 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:55.748 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2023-11-09T03:45:55.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.747+0000 7f208efa9640 1 -- 172.21.15.76:0/905269798 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f207406ae70 msgr2=0x7f207406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:55.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.747+0000 7f208efa9640 1 --2- 172.21.15.76:0/905269798 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f207406ae70 0x7f207406d330 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2078036000 tx=0x7f2078008030 comp rx=0 tx=0).stop 2023-11-09T03:45:55.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.747+0000 7f208efa9640 1 -- 172.21.15.76:0/905269798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2088159c70 msgr2=0x7f208814ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:55.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.747+0000 7f208efa9640 1 --2- 172.21.15.76:0/905269798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2088159c70 0x7f208814ad60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f208000c7a0 tx=0x7f208000cc60 comp rx=0 tx=0).stop 2023-11-09T03:45:55.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.747+0000 7f208efa9640 1 -- 172.21.15.76:0/905269798 shutdown_connections 2023-11-09T03:45:55.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.747+0000 7f208efa9640 1 --2- 172.21.15.76:0/905269798 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f207406ae70 0x7f207406d330 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.747+0000 7f208efa9640 1 --2- 172.21.15.76:0/905269798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2088159c70 0x7f208814ad60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.747+0000 7f208efa9640 1 --2- 172.21.15.76:0/905269798 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20880b0910 0x7f20880cd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.747+0000 7f208efa9640 1 --2- 172.21.15.76:0/905269798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f208806db20 0x7f2088153b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.747+0000 7f208efa9640 1 -- 172.21.15.76:0/905269798 >> 172.21.15.76:0/905269798 conn(0x7f20880ce950 msgr2=0x7f2088152a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:55.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.747+0000 7f208efa9640 1 -- 172.21.15.76:0/905269798 shutdown_connections 2023-11-09T03:45:55.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.747+0000 7f208efa9640 1 -- 172.21.15.76:0/905269798 wait complete. 2023-11-09T03:45:55.757 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2023-11-09T03:45:54.754165+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219468,"kb_used_data":3300,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755252,"statfs":{"total":773094113280,"available":772869378048,"internally_reserved":0,"allocated":3379200,"data_stored":2025099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2229,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003441"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.444993+0000","last_change":"2023-11-09T03:45:30.408658+0000","last_active":"2023-11-09T03:45:35.444993+0000","last_peered":"2023-11-09T03:45:35.444993+0000","last_clean":"2023-11-09T03:45:35.444993+0000","last_became_active":"2023-11-09T03:45:30.408345+0000","last_became_peered":"2023-11-09T03:45:30.408345+0000","last_unstale":"2023-11-09T03:45:35.444993+0000","last_undegraded":"2023-11-09T03:45:35.444993+0000","last_fullsized":"2023-11-09T03:45:35.444993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:12:07.086030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440545+0000","last_change":"2023-11-09T03:45:31.403012+0000","last_active":"2023-11-09T03:45:35.440545+0000","last_peered":"2023-11-09T03:45:35.440545+0000","last_clean":"2023-11-09T03:45:35.440545+0000","last_became_active":"2023-11-09T03:45:31.402623+0000","last_became_peered":"2023-11-09T03:45:31.402623+0000","last_unstale":"2023-11-09T03:45:35.440545+0000","last_undegraded":"2023-11-09T03:45:35.440545+0000","last_fullsized":"2023-11-09T03:45:35.440545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:36:23.593484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388073+0000","last_change":"2023-11-09T03:45:35.620447+0000","last_active":"2023-11-09T03:45:36.388073+0000","last_peered":"2023-11-09T03:45:36.388073+0000","last_clean":"2023-11-09T03:45:36.388073+0000","last_became_active":"2023-11-09T03:45:35.619949+0000","last_became_peered":"2023-11-09T03:45:35.619949+0000","last_unstale":"2023-11-09T03:45:36.388073+0000","last_undegraded":"2023-11-09T03:45:36.388073+0000","last_fullsized":"2023-11-09T03:45:36.388073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:13:00.499438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440637+0000","last_change":"2023-11-09T03:45:30.411660+0000","last_active":"2023-11-09T03:45:35.440637+0000","last_peered":"2023-11-09T03:45:35.440637+0000","last_clean":"2023-11-09T03:45:35.440637+0000","last_became_active":"2023-11-09T03:45:30.411274+0000","last_became_peered":"2023-11-09T03:45:30.411274+0000","last_unstale":"2023-11-09T03:45:35.440637+0000","last_undegraded":"2023-11-09T03:45:35.440637+0000","last_fullsized":"2023-11-09T03:45:35.440637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:39:29.323066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.444755+0000","last_change":"2023-11-09T03:45:31.415835+0000","last_active":"2023-11-09T03:45:35.444755+0000","last_peered":"2023-11-09T03:45:35.444755+0000","last_clean":"2023-11-09T03:45:35.444755+0000","last_became_active":"2023-11-09T03:45:31.415571+0000","last_became_peered":"2023-11-09T03:45:31.415571+0000","last_unstale":"2023-11-09T03:45:35.444755+0000","last_undegraded":"2023-11-09T03:45:35.444755+0000","last_fullsized":"2023-11-09T03:45:35.444755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:13:29.896024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388822+0000","last_change":"2023-11-09T03:45:35.445871+0000","last_active":"2023-11-09T03:45:36.388822+0000","last_peered":"2023-11-09T03:45:36.388822+0000","last_clean":"2023-11-09T03:45:36.388822+0000","last_became_active":"2023-11-09T03:45:35.445390+0000","last_became_peered":"2023-11-09T03:45:35.445390+0000","last_unstale":"2023-11-09T03:45:36.388822+0000","last_undegraded":"2023-11-09T03:45:36.388822+0000","last_fullsized":"2023-11-09T03:45:36.388822+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:26.681331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617937+0000","last_change":"2023-11-09T03:45:30.416124+0000","last_active":"2023-11-09T03:45:35.617937+0000","last_peered":"2023-11-09T03:45:35.617937+0000","last_clean":"2023-11-09T03:45:35.617937+0000","last_became_active":"2023-11-09T03:45:30.415850+0000","last_became_peered":"2023-11-09T03:45:30.415850+0000","last_unstale":"2023-11-09T03:45:35.617937+0000","last_undegraded":"2023-11-09T03:45:35.617937+0000","last_fullsized":"2023-11-09T03:45:35.617937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:43.468677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441010+0000","last_change":"2023-11-09T03:45:31.413149+0000","last_active":"2023-11-09T03:45:35.441010+0000","last_peered":"2023-11-09T03:45:35.441010+0000","last_clean":"2023-11-09T03:45:35.441010+0000","last_became_active":"2023-11-09T03:45:31.412875+0000","last_became_peered":"2023-11-09T03:45:31.412875+0000","last_unstale":"2023-11-09T03:45:35.441010+0000","last_undegraded":"2023-11-09T03:45:35.441010+0000","last_fullsized":"2023-11-09T03:45:35.441010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:04:02.978744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.389160+0000","last_change":"2023-11-09T03:45:35.444039+0000","last_active":"2023-11-09T03:45:36.389160+0000","last_peered":"2023-11-09T03:45:36.389160+0000","last_clean":"2023-11-09T03:45:36.389160+0000","last_became_active":"2023-11-09T03:45:35.443646+0000","last_became_peered":"2023-11-09T03:45:35.443646+0000","last_unstale":"2023-11-09T03:45:36.389160+0000","last_undegraded":"2023-11-09T03:45:36.389160+0000","last_fullsized":"2023-11-09T03:45:36.389160+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:02:31.124051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617462+0000","last_change":"2023-11-09T03:45:30.418418+0000","last_active":"2023-11-09T03:45:35.617462+0000","last_peered":"2023-11-09T03:45:35.617462+0000","last_clean":"2023-11-09T03:45:35.617462+0000","last_became_active":"2023-11-09T03:45:30.417963+0000","last_became_peered":"2023-11-09T03:45:30.417963+0000","last_unstale":"2023-11-09T03:45:35.617462+0000","last_undegraded":"2023-11-09T03:45:35.617462+0000","last_fullsized":"2023-11-09T03:45:35.617462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:07.347819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439849+0000","last_change":"2023-11-09T03:45:31.416138+0000","last_active":"2023-11-09T03:45:35.439849+0000","last_peered":"2023-11-09T03:45:35.439849+0000","last_clean":"2023-11-09T03:45:35.439849+0000","last_became_active":"2023-11-09T03:45:31.415636+0000","last_became_peered":"2023-11-09T03:45:31.415636+0000","last_unstale":"2023-11-09T03:45:35.439849+0000","last_undegraded":"2023-11-09T03:45:35.439849+0000","last_fullsized":"2023-11-09T03:45:35.439849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:02:27.095864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.389010+0000","last_change":"2023-11-09T03:45:35.448265+0000","last_active":"2023-11-09T03:45:36.389010+0000","last_peered":"2023-11-09T03:45:36.389010+0000","last_clean":"2023-11-09T03:45:36.389010+0000","last_became_active":"2023-11-09T03:45:35.447747+0000","last_became_peered":"2023-11-09T03:45:35.447747+0000","last_unstale":"2023-11-09T03:45:36.389010+0000","last_undegraded":"2023-11-09T03:45:36.389010+0000","last_fullsized":"2023-11-09T03:45:36.389010+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:39:46.409150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.437921+0000","last_change":"2023-11-09T03:45:31.400950+0000","last_active":"2023-11-09T03:45:35.437921+0000","last_peered":"2023-11-09T03:45:35.437921+0000","last_clean":"2023-11-09T03:45:35.437921+0000","last_became_active":"2023-11-09T03:45:31.400667+0000","last_became_peered":"2023-11-09T03:45:31.400667+0000","last_unstale":"2023-11-09T03:45:35.437921+0000","last_undegraded":"2023-11-09T03:45:35.437921+0000","last_fullsized":"2023-11-09T03:45:35.437921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:44.473424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617461+0000","last_change":"2023-11-09T03:45:30.418728+0000","last_active":"2023-11-09T03:45:35.617461+0000","last_peered":"2023-11-09T03:45:35.617461+0000","last_clean":"2023-11-09T03:45:35.617461+0000","last_became_active":"2023-11-09T03:45:30.417939+0000","last_became_peered":"2023-11-09T03:45:30.417939+0000","last_unstale":"2023-11-09T03:45:35.617461+0000","last_undegraded":"2023-11-09T03:45:35.617461+0000","last_fullsized":"2023-11-09T03:45:35.617461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:35:45.408209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.389355+0000","last_change":"2023-11-09T03:45:35.448352+0000","last_active":"2023-11-09T03:45:36.389355+0000","last_peered":"2023-11-09T03:45:36.389355+0000","last_clean":"2023-11-09T03:45:36.389355+0000","last_became_active":"2023-11-09T03:45:35.447889+0000","last_became_peered":"2023-11-09T03:45:35.447889+0000","last_unstale":"2023-11-09T03:45:36.389355+0000","last_undegraded":"2023-11-09T03:45:36.389355+0000","last_fullsized":"2023-11-09T03:45:36.389355+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:39:40.769191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439502+0000","last_change":"2023-11-09T03:45:31.398106+0000","last_active":"2023-11-09T03:45:35.439502+0000","last_peered":"2023-11-09T03:45:35.439502+0000","last_clean":"2023-11-09T03:45:35.439502+0000","last_became_active":"2023-11-09T03:45:31.397712+0000","last_became_peered":"2023-11-09T03:45:31.397712+0000","last_unstale":"2023-11-09T03:45:35.439502+0000","last_undegraded":"2023-11-09T03:45:35.439502+0000","last_fullsized":"2023-11-09T03:45:35.439502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:04:02.978744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441816+0000","last_change":"2023-11-09T03:45:30.408462+0000","last_active":"2023-11-09T03:45:35.441816+0000","last_peered":"2023-11-09T03:45:35.441816+0000","last_clean":"2023-11-09T03:45:35.441816+0000","last_became_active":"2023-11-09T03:45:30.408190+0000","last_became_peered":"2023-11-09T03:45:30.408190+0000","last_unstale":"2023-11-09T03:45:35.441816+0000","last_undegraded":"2023-11-09T03:45:35.441816+0000","last_fullsized":"2023-11-09T03:45:35.441816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:11:40.438915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.389721+0000","last_change":"2023-11-09T03:45:35.445519+0000","last_active":"2023-11-09T03:45:36.389721+0000","last_peered":"2023-11-09T03:45:36.389721+0000","last_clean":"2023-11-09T03:45:36.389721+0000","last_became_active":"2023-11-09T03:45:35.445125+0000","last_became_peered":"2023-11-09T03:45:35.445125+0000","last_unstale":"2023-11-09T03:45:36.389721+0000","last_undegraded":"2023-11-09T03:45:36.389721+0000","last_fullsized":"2023-11-09T03:45:36.389721+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:45.477938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439030+0000","last_change":"2023-11-09T03:45:31.402204+0000","last_active":"2023-11-09T03:45:35.439030+0000","last_peered":"2023-11-09T03:45:35.439030+0000","last_clean":"2023-11-09T03:45:35.439030+0000","last_became_active":"2023-11-09T03:45:31.401312+0000","last_became_peered":"2023-11-09T03:45:31.401312+0000","last_unstale":"2023-11-09T03:45:35.439030+0000","last_undegraded":"2023-11-09T03:45:35.439030+0000","last_fullsized":"2023-11-09T03:45:35.439030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:48:08.925647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441918+0000","last_change":"2023-11-09T03:45:30.414258+0000","last_active":"2023-11-09T03:45:35.441918+0000","last_peered":"2023-11-09T03:45:35.441918+0000","last_clean":"2023-11-09T03:45:35.441918+0000","last_became_active":"2023-11-09T03:45:30.413994+0000","last_became_peered":"2023-11-09T03:45:30.413994+0000","last_unstale":"2023-11-09T03:45:35.441918+0000","last_undegraded":"2023-11-09T03:45:35.441918+0000","last_fullsized":"2023-11-09T03:45:35.441918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:12:07.086030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.263008+0000","last_change":"2023-11-09T03:45:35.620757+0000","last_active":"2023-11-09T03:45:37.263008+0000","last_peered":"2023-11-09T03:45:37.263008+0000","last_clean":"2023-11-09T03:45:37.263008+0000","last_became_active":"2023-11-09T03:45:35.620219+0000","last_became_peered":"2023-11-09T03:45:35.620219+0000","last_unstale":"2023-11-09T03:45:37.263008+0000","last_undegraded":"2023-11-09T03:45:37.263008+0000","last_fullsized":"2023-11-09T03:45:37.263008+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:41:12.579051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.436441+0000","last_change":"2023-11-09T03:45:31.396755+0000","last_active":"2023-11-09T03:45:35.436441+0000","last_peered":"2023-11-09T03:45:35.436441+0000","last_clean":"2023-11-09T03:45:35.436441+0000","last_became_active":"2023-11-09T03:45:31.396475+0000","last_became_peered":"2023-11-09T03:45:31.396475+0000","last_unstale":"2023-11-09T03:45:35.436441+0000","last_undegraded":"2023-11-09T03:45:35.436441+0000","last_fullsized":"2023-11-09T03:45:35.436441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:07:02.262322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617133+0000","last_change":"2023-11-09T03:45:30.419151+0000","last_active":"2023-11-09T03:45:35.617133+0000","last_peered":"2023-11-09T03:45:35.617133+0000","last_clean":"2023-11-09T03:45:35.617133+0000","last_became_active":"2023-11-09T03:45:30.418874+0000","last_became_peered":"2023-11-09T03:45:30.418874+0000","last_unstale":"2023-11-09T03:45:35.617133+0000","last_undegraded":"2023-11-09T03:45:35.617133+0000","last_fullsized":"2023-11-09T03:45:35.617133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:34:48.400703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.389351+0000","last_change":"2023-11-09T03:45:35.445655+0000","last_active":"2023-11-09T03:45:36.389351+0000","last_peered":"2023-11-09T03:45:36.389351+0000","last_clean":"2023-11-09T03:45:36.389351+0000","last_became_active":"2023-11-09T03:45:35.445246+0000","last_became_peered":"2023-11-09T03:45:35.445246+0000","last_unstale":"2023-11-09T03:45:36.389351+0000","last_undegraded":"2023-11-09T03:45:36.389351+0000","last_fullsized":"2023-11-09T03:45:36.389351+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:04:07.006932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.443777+0000","last_change":"2023-11-09T03:45:31.397453+0000","last_active":"2023-11-09T03:45:35.443777+0000","last_peered":"2023-11-09T03:45:35.443777+0000","last_clean":"2023-11-09T03:45:35.443777+0000","last_became_active":"2023-11-09T03:45:31.396892+0000","last_became_peered":"2023-11-09T03:45:31.396892+0000","last_unstale":"2023-11-09T03:45:35.443777+0000","last_undegraded":"2023-11-09T03:45:35.443777+0000","last_fullsized":"2023-11-09T03:45:35.443777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:44.473424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440439+0000","last_change":"2023-11-09T03:45:30.416814+0000","last_active":"2023-11-09T03:45:35.440439+0000","last_peered":"2023-11-09T03:45:35.440439+0000","last_clean":"2023-11-09T03:45:35.440439+0000","last_became_active":"2023-11-09T03:45:30.416446+0000","last_became_peered":"2023-11-09T03:45:30.416446+0000","last_unstale":"2023-11-09T03:45:35.440439+0000","last_undegraded":"2023-11-09T03:45:35.440439+0000","last_fullsized":"2023-11-09T03:45:35.440439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:37:54.581788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388988+0000","last_change":"2023-11-09T03:45:35.618893+0000","last_active":"2023-11-09T03:45:36.388988+0000","last_peered":"2023-11-09T03:45:36.388988+0000","last_clean":"2023-11-09T03:45:36.388988+0000","last_became_active":"2023-11-09T03:45:35.618506+0000","last_became_peered":"2023-11-09T03:45:35.618506+0000","last_unstale":"2023-11-09T03:45:36.388988+0000","last_undegraded":"2023-11-09T03:45:36.388988+0000","last_fullsized":"2023-11-09T03:45:36.388988+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:36:27.621672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438491+0000","last_change":"2023-11-09T03:45:31.406885+0000","last_active":"2023-11-09T03:45:35.438491+0000","last_peered":"2023-11-09T03:45:35.438491+0000","last_clean":"2023-11-09T03:45:35.438491+0000","last_became_active":"2023-11-09T03:45:31.406620+0000","last_became_peered":"2023-11-09T03:45:31.406620+0000","last_unstale":"2023-11-09T03:45:35.438491+0000","last_undegraded":"2023-11-09T03:45:35.438491+0000","last_fullsized":"2023-11-09T03:45:35.438491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:29:17.503411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442020+0000","last_change":"2023-11-09T03:45:30.412534+0000","last_active":"2023-11-09T03:45:35.442020+0000","last_peered":"2023-11-09T03:45:35.442020+0000","last_clean":"2023-11-09T03:45:35.442020+0000","last_became_active":"2023-11-09T03:45:30.412015+0000","last_became_peered":"2023-11-09T03:45:30.412015+0000","last_unstale":"2023-11-09T03:45:35.442020+0000","last_undegraded":"2023-11-09T03:45:35.442020+0000","last_fullsized":"2023-11-09T03:45:35.442020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:34:48.400703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.387965+0000","last_change":"2023-11-09T03:45:35.620972+0000","last_active":"2023-11-09T03:45:36.387965+0000","last_peered":"2023-11-09T03:45:36.387965+0000","last_clean":"2023-11-09T03:45:36.387965+0000","last_became_active":"2023-11-09T03:45:35.620626+0000","last_became_peered":"2023-11-09T03:45:35.620626+0000","last_unstale":"2023-11-09T03:45:36.387965+0000","last_undegraded":"2023-11-09T03:45:36.387965+0000","last_fullsized":"2023-11-09T03:45:36.387965+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:03:57.174574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441966+0000","last_change":"2023-11-09T03:45:31.393932+0000","last_active":"2023-11-09T03:45:35.441966+0000","last_peered":"2023-11-09T03:45:35.441966+0000","last_clean":"2023-11-09T03:45:35.441966+0000","last_became_active":"2023-11-09T03:45:31.393669+0000","last_became_peered":"2023-11-09T03:45:31.393669+0000","last_unstale":"2023-11-09T03:45:35.441966+0000","last_undegraded":"2023-11-09T03:45:35.441966+0000","last_fullsized":"2023-11-09T03:45:35.441966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:27.400068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.618527+0000","last_change":"2023-11-09T03:45:30.419637+0000","last_active":"2023-11-09T03:45:35.618527+0000","last_peered":"2023-11-09T03:45:35.618527+0000","last_clean":"2023-11-09T03:45:35.618527+0000","last_became_active":"2023-11-09T03:45:30.419348+0000","last_became_peered":"2023-11-09T03:45:30.419348+0000","last_unstale":"2023-11-09T03:45:35.618527+0000","last_undegraded":"2023-11-09T03:45:35.618527+0000","last_fullsized":"2023-11-09T03:45:35.618527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:48:07.920901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.261912+0000","last_change":"2023-11-09T03:45:35.619361+0000","last_active":"2023-11-09T03:45:37.261912+0000","last_peered":"2023-11-09T03:45:37.261912+0000","last_clean":"2023-11-09T03:45:37.261912+0000","last_became_active":"2023-11-09T03:45:35.618998+0000","last_became_peered":"2023-11-09T03:45:35.618998+0000","last_unstale":"2023-11-09T03:45:37.261912+0000","last_undegraded":"2023-11-09T03:45:37.261912+0000","last_fullsized":"2023-11-09T03:45:37.261912+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:29:21.531598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439868+0000","last_change":"2023-11-09T03:45:31.408700+0000","last_active":"2023-11-09T03:45:35.439868+0000","last_peered":"2023-11-09T03:45:35.439868+0000","last_clean":"2023-11-09T03:45:35.439868+0000","last_became_active":"2023-11-09T03:45:31.408153+0000","last_became_peered":"2023-11-09T03:45:31.408153+0000","last_unstale":"2023-11-09T03:45:35.439868+0000","last_undegraded":"2023-11-09T03:45:35.439868+0000","last_fullsized":"2023-11-09T03:45:35.439868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:29:17.503411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442042+0000","last_change":"2023-11-09T03:45:30.409443+0000","last_active":"2023-11-09T03:45:35.442042+0000","last_peered":"2023-11-09T03:45:35.442042+0000","last_clean":"2023-11-09T03:45:35.442042+0000","last_became_active":"2023-11-09T03:45:30.409175+0000","last_became_peered":"2023-11-09T03:45:30.409175+0000","last_unstale":"2023-11-09T03:45:35.442042+0000","last_undegraded":"2023-11-09T03:45:35.442042+0000","last_fullsized":"2023-11-09T03:45:35.442042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:34:27.335423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262894+0000","last_change":"2023-11-09T03:45:35.452625+0000","last_active":"2023-11-09T03:45:37.262894+0000","last_peered":"2023-11-09T03:45:37.262894+0000","last_clean":"2023-11-09T03:45:37.262894+0000","last_became_active":"2023-11-09T03:45:35.451772+0000","last_became_peered":"2023-11-09T03:45:35.451772+0000","last_unstale":"2023-11-09T03:45:37.262894+0000","last_undegraded":"2023-11-09T03:45:37.262894+0000","last_fullsized":"2023-11-09T03:45:37.262894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:45.477938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.435623+0000","last_change":"2023-11-09T03:45:31.399046+0000","last_active":"2023-11-09T03:45:35.435623+0000","last_peered":"2023-11-09T03:45:35.435623+0000","last_clean":"2023-11-09T03:45:35.435623+0000","last_became_active":"2023-11-09T03:45:31.398698+0000","last_became_peered":"2023-11-09T03:45:31.398698+0000","last_unstale":"2023-11-09T03:45:35.435623+0000","last_undegraded":"2023-11-09T03:45:35.435623+0000","last_fullsized":"2023-11-09T03:45:35.435623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:35:46.412956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440345+0000","last_change":"2023-11-09T03:45:30.414519+0000","last_active":"2023-11-09T03:45:35.440345+0000","last_peered":"2023-11-09T03:45:35.440345+0000","last_clean":"2023-11-09T03:45:35.440345+0000","last_became_active":"2023-11-09T03:45:30.414025+0000","last_became_peered":"2023-11-09T03:45:30.414025+0000","last_unstale":"2023-11-09T03:45:35.440345+0000","last_undegraded":"2023-11-09T03:45:35.440345+0000","last_fullsized":"2023-11-09T03:45:35.440345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:43.468677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262508+0000","last_change":"2023-11-09T03:45:35.451076+0000","last_active":"2023-11-09T03:45:37.262508+0000","last_peered":"2023-11-09T03:45:37.262508+0000","last_clean":"2023-11-09T03:45:37.262508+0000","last_became_active":"2023-11-09T03:45:35.450721+0000","last_became_peered":"2023-11-09T03:45:35.450721+0000","last_unstale":"2023-11-09T03:45:37.262508+0000","last_undegraded":"2023-11-09T03:45:37.262508+0000","last_fullsized":"2023-11-09T03:45:37.262508+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:29:21.531598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.443831+0000","last_change":"2023-11-09T03:45:31.397098+0000","last_active":"2023-11-09T03:45:35.443831+0000","last_peered":"2023-11-09T03:45:35.443831+0000","last_clean":"2023-11-09T03:45:35.443831+0000","last_became_active":"2023-11-09T03:45:31.396835+0000","last_became_peered":"2023-11-09T03:45:31.396835+0000","last_unstale":"2023-11-09T03:45:35.443831+0000","last_undegraded":"2023-11-09T03:45:35.443831+0000","last_fullsized":"2023-11-09T03:45:35.443831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:54:30.809126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.432187+0000","last_change":"2023-11-09T03:45:30.411878+0000","last_active":"2023-11-09T03:45:35.432187+0000","last_peered":"2023-11-09T03:45:35.432187+0000","last_clean":"2023-11-09T03:45:35.432187+0000","last_became_active":"2023-11-09T03:45:30.411615+0000","last_became_peered":"2023-11-09T03:45:30.411615+0000","last_unstale":"2023-11-09T03:45:35.432187+0000","last_undegraded":"2023-11-09T03:45:35.432187+0000","last_fullsized":"2023-11-09T03:45:35.432187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:11:40.438915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.263008+0000","last_change":"2023-11-09T03:45:35.451994+0000","last_active":"2023-11-09T03:45:37.263008+0000","last_peered":"2023-11-09T03:45:37.263008+0000","last_clean":"2023-11-09T03:45:37.263008+0000","last_became_active":"2023-11-09T03:45:35.451623+0000","last_became_peered":"2023-11-09T03:45:35.451623+0000","last_unstale":"2023-11-09T03:45:37.263008+0000","last_undegraded":"2023-11-09T03:45:37.263008+0000","last_fullsized":"2023-11-09T03:45:37.263008+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:13:08.339968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441086+0000","last_change":"2023-11-09T03:45:31.400475+0000","last_active":"2023-11-09T03:45:35.441086+0000","last_peered":"2023-11-09T03:45:35.441086+0000","last_clean":"2023-11-09T03:45:35.441086+0000","last_became_active":"2023-11-09T03:45:31.400021+0000","last_became_peered":"2023-11-09T03:45:31.400021+0000","last_unstale":"2023-11-09T03:45:35.441086+0000","last_undegraded":"2023-11-09T03:45:35.441086+0000","last_fullsized":"2023-11-09T03:45:35.441086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:11:10.514126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617133+0000","last_change":"2023-11-09T03:45:30.417099+0000","last_active":"2023-11-09T03:45:35.617133+0000","last_peered":"2023-11-09T03:45:35.617133+0000","last_clean":"2023-11-09T03:45:35.617133+0000","last_became_active":"2023-11-09T03:45:30.416779+0000","last_became_peered":"2023-11-09T03:45:30.416779+0000","last_unstale":"2023-11-09T03:45:35.617133+0000","last_undegraded":"2023-11-09T03:45:35.617133+0000","last_fullsized":"2023-11-09T03:45:35.617133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:06:25.266825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262776+0000","last_change":"2023-11-09T03:45:35.618887+0000","last_active":"2023-11-09T03:45:37.262776+0000","last_peered":"2023-11-09T03:45:37.262776+0000","last_clean":"2023-11-09T03:45:37.262776+0000","last_became_active":"2023-11-09T03:45:35.618406+0000","last_became_peered":"2023-11-09T03:45:35.618406+0000","last_unstale":"2023-11-09T03:45:37.262776+0000","last_undegraded":"2023-11-09T03:45:37.262776+0000","last_fullsized":"2023-11-09T03:45:37.262776+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:41:12.579051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438270+0000","last_change":"2023-11-09T03:45:31.416419+0000","last_active":"2023-11-09T03:45:35.438270+0000","last_peered":"2023-11-09T03:45:35.438270+0000","last_clean":"2023-11-09T03:45:35.438270+0000","last_became_active":"2023-11-09T03:45:31.416137+0000","last_became_peered":"2023-11-09T03:45:31.416137+0000","last_unstale":"2023-11-09T03:45:35.438270+0000","last_undegraded":"2023-11-09T03:45:35.438270+0000","last_fullsized":"2023-11-09T03:45:35.438270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:08.352566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439728+0000","last_change":"2023-11-09T03:45:30.417920+0000","last_active":"2023-11-09T03:45:35.439728+0000","last_peered":"2023-11-09T03:45:35.439728+0000","last_clean":"2023-11-09T03:45:35.439728+0000","last_became_active":"2023-11-09T03:45:30.417578+0000","last_became_peered":"2023-11-09T03:45:30.417578+0000","last_unstale":"2023-11-09T03:45:35.439728+0000","last_undegraded":"2023-11-09T03:45:35.439728+0000","last_fullsized":"2023-11-09T03:45:35.439728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:07:01.257576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617231+0000","last_change":"2023-11-09T03:45:21.030926+0000","last_active":"2023-11-09T03:45:35.617231+0000","last_peered":"2023-11-09T03:45:35.617231+0000","last_clean":"2023-11-09T03:45:35.617231+0000","last_became_active":"2023-11-09T03:45:21.029801+0000","last_became_peered":"2023-11-09T03:45:21.029801+0000","last_unstale":"2023-11-09T03:45:35.617231+0000","last_undegraded":"2023-11-09T03:45:35.617231+0000","last_fullsized":"2023-11-09T03:45:35.617231+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_clean_scrub_stamp":"2023-11-09T03:45:19.768433+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:12:59.153713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.264013+0000","last_change":"2023-11-09T03:45:35.622614+0000","last_active":"2023-11-09T03:45:37.264013+0000","last_peered":"2023-11-09T03:45:37.264013+0000","last_clean":"2023-11-09T03:45:37.264013+0000","last_became_active":"2023-11-09T03:45:35.622138+0000","last_became_peered":"2023-11-09T03:45:35.622138+0000","last_unstale":"2023-11-09T03:45:37.264013+0000","last_undegraded":"2023-11-09T03:45:37.264013+0000","last_fullsized":"2023-11-09T03:45:37.264013+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:29:21.531598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617667+0000","last_change":"2023-11-09T03:45:31.417640+0000","last_active":"2023-11-09T03:45:35.617667+0000","last_peered":"2023-11-09T03:45:35.617667+0000","last_clean":"2023-11-09T03:45:35.617667+0000","last_became_active":"2023-11-09T03:45:31.417210+0000","last_became_peered":"2023-11-09T03:45:31.417210+0000","last_unstale":"2023-11-09T03:45:35.617667+0000","last_undegraded":"2023-11-09T03:45:35.617667+0000","last_fullsized":"2023-11-09T03:45:35.617667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:41.449750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.432324+0000","last_change":"2023-11-09T03:45:30.418193+0000","last_active":"2023-11-09T03:45:35.432324+0000","last_peered":"2023-11-09T03:45:35.432324+0000","last_clean":"2023-11-09T03:45:35.432324+0000","last_became_active":"2023-11-09T03:45:30.417874+0000","last_became_peered":"2023-11-09T03:45:30.417874+0000","last_unstale":"2023-11-09T03:45:35.432324+0000","last_undegraded":"2023-11-09T03:45:35.432324+0000","last_fullsized":"2023-11-09T03:45:35.432324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:34:27.335423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.263620+0000","last_change":"2023-11-09T03:45:35.618898+0000","last_active":"2023-11-09T03:45:37.263620+0000","last_peered":"2023-11-09T03:45:37.263620+0000","last_clean":"2023-11-09T03:45:37.263620+0000","last_became_active":"2023-11-09T03:45:35.618502+0000","last_became_peered":"2023-11-09T03:45:35.618502+0000","last_unstale":"2023-11-09T03:45:37.263620+0000","last_undegraded":"2023-11-09T03:45:37.263620+0000","last_fullsized":"2023-11-09T03:45:37.263620+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:39:40.769191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441100+0000","last_change":"2023-11-09T03:45:31.409910+0000","last_active":"2023-11-09T03:45:35.441100+0000","last_peered":"2023-11-09T03:45:35.441100+0000","last_clean":"2023-11-09T03:45:35.441100+0000","last_became_active":"2023-11-09T03:45:31.408863+0000","last_became_peered":"2023-11-09T03:45:31.408863+0000","last_unstale":"2023-11-09T03:45:35.441100+0000","last_undegraded":"2023-11-09T03:45:35.441100+0000","last_fullsized":"2023-11-09T03:45:35.441100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:41.449750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442413+0000","last_change":"2023-11-09T03:45:30.412199+0000","last_active":"2023-11-09T03:45:35.442413+0000","last_peered":"2023-11-09T03:45:35.442413+0000","last_clean":"2023-11-09T03:45:35.442413+0000","last_became_active":"2023-11-09T03:45:30.411864+0000","last_became_peered":"2023-11-09T03:45:30.411864+0000","last_unstale":"2023-11-09T03:45:35.442413+0000","last_undegraded":"2023-11-09T03:45:35.442413+0000","last_fullsized":"2023-11-09T03:45:35.442413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:13:28.891277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262128+0000","last_change":"2023-11-09T03:45:35.456250+0000","last_active":"2023-11-09T03:45:37.262128+0000","last_peered":"2023-11-09T03:45:37.262128+0000","last_clean":"2023-11-09T03:45:37.262128+0000","last_became_active":"2023-11-09T03:45:35.455866+0000","last_became_peered":"2023-11-09T03:45:35.455866+0000","last_unstale":"2023-11-09T03:45:37.262128+0000","last_undegraded":"2023-11-09T03:45:37.262128+0000","last_fullsized":"2023-11-09T03:45:37.262128+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:45.477938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.618920+0000","last_change":"2023-11-09T03:45:31.420110+0000","last_active":"2023-11-09T03:45:35.618920+0000","last_peered":"2023-11-09T03:45:35.618920+0000","last_clean":"2023-11-09T03:45:35.618920+0000","last_became_active":"2023-11-09T03:45:31.419556+0000","last_became_peered":"2023-11-09T03:45:31.419556+0000","last_unstale":"2023-11-09T03:45:35.618920+0000","last_undegraded":"2023-11-09T03:45:35.618920+0000","last_fullsized":"2023-11-09T03:45:35.618920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:51:24.601367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.432312+0000","last_change":"2023-11-09T03:45:30.415461+0000","last_active":"2023-11-09T03:45:35.432312+0000","last_peered":"2023-11-09T03:45:35.432312+0000","last_clean":"2023-11-09T03:45:35.432312+0000","last_became_active":"2023-11-09T03:45:30.415185+0000","last_became_peered":"2023-11-09T03:45:30.415185+0000","last_unstale":"2023-11-09T03:45:35.432312+0000","last_undegraded":"2023-11-09T03:45:35.432312+0000","last_fullsized":"2023-11-09T03:45:35.432312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:07:01.257576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262941+0000","last_change":"2023-11-09T03:45:35.620742+0000","last_active":"2023-11-09T03:45:37.262941+0000","last_peered":"2023-11-09T03:45:37.262941+0000","last_clean":"2023-11-09T03:45:37.262941+0000","last_became_active":"2023-11-09T03:45:35.620350+0000","last_became_peered":"2023-11-09T03:45:35.620350+0000","last_unstale":"2023-11-09T03:45:37.262941+0000","last_undegraded":"2023-11-09T03:45:37.262941+0000","last_fullsized":"2023-11-09T03:45:37.262941+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:03:57.174574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.618702+0000","last_change":"2023-11-09T03:45:31.417119+0000","last_active":"2023-11-09T03:45:35.618702+0000","last_peered":"2023-11-09T03:45:35.618702+0000","last_clean":"2023-11-09T03:45:35.618702+0000","last_became_active":"2023-11-09T03:45:31.416750+0000","last_became_peered":"2023-11-09T03:45:31.416750+0000","last_unstale":"2023-11-09T03:45:35.618702+0000","last_undegraded":"2023-11-09T03:45:35.618702+0000","last_fullsized":"2023-11-09T03:45:35.618702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:55:29.945168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.436205+0000","last_change":"2023-11-09T03:45:30.410204+0000","last_active":"2023-11-09T03:45:35.436205+0000","last_peered":"2023-11-09T03:45:35.436205+0000","last_clean":"2023-11-09T03:45:35.436205+0000","last_became_active":"2023-11-09T03:45:30.409876+0000","last_became_peered":"2023-11-09T03:45:30.409876+0000","last_unstale":"2023-11-09T03:45:35.436205+0000","last_undegraded":"2023-11-09T03:45:35.436205+0000","last_fullsized":"2023-11-09T03:45:35.436205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:23:49.677277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388781+0000","last_change":"2023-11-09T03:45:35.446608+0000","last_active":"2023-11-09T03:45:36.388781+0000","last_peered":"2023-11-09T03:45:36.388781+0000","last_clean":"2023-11-09T03:45:36.388781+0000","last_became_active":"2023-11-09T03:45:35.446121+0000","last_became_peered":"2023-11-09T03:45:35.446121+0000","last_unstale":"2023-11-09T03:45:36.388781+0000","last_undegraded":"2023-11-09T03:45:36.388781+0000","last_fullsized":"2023-11-09T03:45:36.388781+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:12.380753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617394+0000","last_change":"2023-11-09T03:45:31.418010+0000","last_active":"2023-11-09T03:45:35.617394+0000","last_peered":"2023-11-09T03:45:35.617394+0000","last_clean":"2023-11-09T03:45:35.617394+0000","last_became_active":"2023-11-09T03:45:31.417758+0000","last_became_peered":"2023-11-09T03:45:31.417758+0000","last_unstale":"2023-11-09T03:45:35.617394+0000","last_undegraded":"2023-11-09T03:45:35.617394+0000","last_fullsized":"2023-11-09T03:45:35.617394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:02:27.095864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.432121+0000","last_change":"2023-11-09T03:45:30.412289+0000","last_active":"2023-11-09T03:45:35.432121+0000","last_peered":"2023-11-09T03:45:35.432121+0000","last_clean":"2023-11-09T03:45:35.432121+0000","last_became_active":"2023-11-09T03:45:30.412035+0000","last_became_peered":"2023-11-09T03:45:30.412035+0000","last_unstale":"2023-11-09T03:45:35.432121+0000","last_undegraded":"2023-11-09T03:45:35.432121+0000","last_fullsized":"2023-11-09T03:45:35.432121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:35:45.408209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262660+0000","last_change":"2023-11-09T03:45:35.450569+0000","last_active":"2023-11-09T03:45:37.262660+0000","last_peered":"2023-11-09T03:45:37.262660+0000","last_clean":"2023-11-09T03:45:37.262660+0000","last_became_active":"2023-11-09T03:45:35.450133+0000","last_became_peered":"2023-11-09T03:45:35.450133+0000","last_unstale":"2023-11-09T03:45:37.262660+0000","last_undegraded":"2023-11-09T03:45:37.262660+0000","last_fullsized":"2023-11-09T03:45:37.262660+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:13:08.339968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440213+0000","last_change":"2023-11-09T03:45:31.400690+0000","last_active":"2023-11-09T03:45:35.440213+0000","last_peered":"2023-11-09T03:45:35.440213+0000","last_clean":"2023-11-09T03:45:35.440213+0000","last_became_active":"2023-11-09T03:45:31.400345+0000","last_became_peered":"2023-11-09T03:45:31.400345+0000","last_unstale":"2023-11-09T03:45:35.440213+0000","last_undegraded":"2023-11-09T03:45:35.440213+0000","last_fullsized":"2023-11-09T03:45:35.440213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:54:30.809126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438759+0000","last_change":"2023-11-09T03:45:30.414344+0000","last_active":"2023-11-09T03:45:35.438759+0000","last_peered":"2023-11-09T03:45:35.438759+0000","last_clean":"2023-11-09T03:45:35.438759+0000","last_became_active":"2023-11-09T03:45:30.414070+0000","last_became_peered":"2023-11-09T03:45:30.414070+0000","last_unstale":"2023-11-09T03:45:35.438759+0000","last_undegraded":"2023-11-09T03:45:35.438759+0000","last_fullsized":"2023-11-09T03:45:35.438759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:06:25.266825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388898+0000","last_change":"2023-11-09T03:45:35.618460+0000","last_active":"2023-11-09T03:45:36.388898+0000","last_peered":"2023-11-09T03:45:36.388898+0000","last_clean":"2023-11-09T03:45:36.388898+0000","last_became_active":"2023-11-09T03:45:35.618089+0000","last_became_peered":"2023-11-09T03:45:35.618089+0000","last_unstale":"2023-11-09T03:45:36.388898+0000","last_undegraded":"2023-11-09T03:45:36.388898+0000","last_fullsized":"2023-11-09T03:45:36.388898+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:24:34.808267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617389+0000","last_change":"2023-11-09T03:45:31.418108+0000","last_active":"2023-11-09T03:45:35.617389+0000","last_peered":"2023-11-09T03:45:35.617389+0000","last_clean":"2023-11-09T03:45:35.617389+0000","last_became_active":"2023-11-09T03:45:31.417869+0000","last_became_peered":"2023-11-09T03:45:31.417869+0000","last_unstale":"2023-11-09T03:45:35.617389+0000","last_undegraded":"2023-11-09T03:45:35.617389+0000","last_fullsized":"2023-11-09T03:45:35.617389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:04:02.978744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440273+0000","last_change":"2023-11-09T03:45:30.414815+0000","last_active":"2023-11-09T03:45:35.440273+0000","last_peered":"2023-11-09T03:45:35.440273+0000","last_clean":"2023-11-09T03:45:35.440273+0000","last_became_active":"2023-11-09T03:45:30.414177+0000","last_became_peered":"2023-11-09T03:45:30.414177+0000","last_unstale":"2023-11-09T03:45:35.440273+0000","last_undegraded":"2023-11-09T03:45:35.440273+0000","last_fullsized":"2023-11-09T03:45:35.440273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:54:29.804379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388529+0000","last_change":"2023-11-09T03:45:35.618416+0000","last_active":"2023-11-09T03:45:36.388529+0000","last_peered":"2023-11-09T03:45:36.388529+0000","last_clean":"2023-11-09T03:45:36.388529+0000","last_became_active":"2023-11-09T03:45:35.618055+0000","last_became_peered":"2023-11-09T03:45:35.618055+0000","last_unstale":"2023-11-09T03:45:36.388529+0000","last_undegraded":"2023-11-09T03:45:36.388529+0000","last_fullsized":"2023-11-09T03:45:36.388529+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:16:21.023356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.443358+0000","last_change":"2023-11-09T03:45:31.404777+0000","last_active":"2023-11-09T03:45:35.443358+0000","last_peered":"2023-11-09T03:45:35.443358+0000","last_clean":"2023-11-09T03:45:35.443358+0000","last_became_active":"2023-11-09T03:45:31.404472+0000","last_became_peered":"2023-11-09T03:45:31.404472+0000","last_unstale":"2023-11-09T03:45:35.443358+0000","last_undegraded":"2023-11-09T03:45:35.443358+0000","last_fullsized":"2023-11-09T03:45:35.443358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:06:26.271572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.443117+0000","last_change":"2023-11-09T03:45:30.411464+0000","last_active":"2023-11-09T03:45:35.443117+0000","last_peered":"2023-11-09T03:45:35.443117+0000","last_clean":"2023-11-09T03:45:35.443117+0000","last_became_active":"2023-11-09T03:45:30.410430+0000","last_became_peered":"2023-11-09T03:45:30.410430+0000","last_unstale":"2023-11-09T03:45:35.443117+0000","last_undegraded":"2023-11-09T03:45:35.443117+0000","last_fullsized":"2023-11-09T03:45:35.443117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:48:49.911609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262935+0000","last_change":"2023-11-09T03:45:35.618861+0000","last_active":"2023-11-09T03:45:37.262935+0000","last_peered":"2023-11-09T03:45:37.262935+0000","last_clean":"2023-11-09T03:45:37.262935+0000","last_became_active":"2023-11-09T03:45:35.618394+0000","last_became_peered":"2023-11-09T03:45:35.618394+0000","last_unstale":"2023-11-09T03:45:37.262935+0000","last_undegraded":"2023-11-09T03:45:37.262935+0000","last_fullsized":"2023-11-09T03:45:37.262935+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:07:23.630255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617562+0000","last_change":"2023-11-09T03:45:31.407151+0000","last_active":"2023-11-09T03:45:35.617562+0000","last_peered":"2023-11-09T03:45:35.617562+0000","last_clean":"2023-11-09T03:45:35.617562+0000","last_became_active":"2023-11-09T03:45:31.406652+0000","last_became_peered":"2023-11-09T03:45:31.406652+0000","last_unstale":"2023-11-09T03:45:35.617562+0000","last_undegraded":"2023-11-09T03:45:35.617562+0000","last_fullsized":"2023-11-09T03:45:35.617562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:29:17.503411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:41.806449+0000","last_change":"2023-11-09T03:45:30.411676+0000","last_active":"2023-11-09T03:45:41.806449+0000","last_peered":"2023-11-09T03:45:41.806449+0000","last_clean":"2023-11-09T03:45:41.806449+0000","last_became_active":"2023-11-09T03:45:30.411379+0000","last_became_peered":"2023-11-09T03:45:30.411379+0000","last_unstale":"2023-11-09T03:45:41.806449+0000","last_undegraded":"2023-11-09T03:45:41.806449+0000","last_fullsized":"2023-11-09T03:45:41.806449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:34:48.400703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262972+0000","last_change":"2023-11-09T03:45:35.452073+0000","last_active":"2023-11-09T03:45:37.262972+0000","last_peered":"2023-11-09T03:45:37.262972+0000","last_clean":"2023-11-09T03:45:37.262972+0000","last_became_active":"2023-11-09T03:45:35.451714+0000","last_became_peered":"2023-11-09T03:45:35.451714+0000","last_unstale":"2023-11-09T03:45:37.262972+0000","last_undegraded":"2023-11-09T03:45:37.262972+0000","last_fullsized":"2023-11-09T03:45:37.262972+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:55:33.973355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.435954+0000","last_change":"2023-11-09T03:45:31.406706+0000","last_active":"2023-11-09T03:45:35.435954+0000","last_peered":"2023-11-09T03:45:35.435954+0000","last_clean":"2023-11-09T03:45:35.435954+0000","last_became_active":"2023-11-09T03:45:31.406437+0000","last_became_peered":"2023-11-09T03:45:31.406437+0000","last_unstale":"2023-11-09T03:45:35.435954+0000","last_undegraded":"2023-11-09T03:45:35.435954+0000","last_fullsized":"2023-11-09T03:45:35.435954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:48:50.916355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442326+0000","last_change":"2023-11-09T03:45:30.418063+0000","last_active":"2023-11-09T03:45:35.442326+0000","last_peered":"2023-11-09T03:45:35.442326+0000","last_clean":"2023-11-09T03:45:35.442326+0000","last_became_active":"2023-11-09T03:45:30.417792+0000","last_became_peered":"2023-11-09T03:45:30.417792+0000","last_unstale":"2023-11-09T03:45:35.442326+0000","last_undegraded":"2023-11-09T03:45:35.442326+0000","last_fullsized":"2023-11-09T03:45:35.442326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:06:25.266825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.389474+0000","last_change":"2023-11-09T03:45:35.446788+0000","last_active":"2023-11-09T03:45:36.389474+0000","last_peered":"2023-11-09T03:45:36.389474+0000","last_clean":"2023-11-09T03:45:36.389474+0000","last_became_active":"2023-11-09T03:45:35.446280+0000","last_became_peered":"2023-11-09T03:45:35.446280+0000","last_unstale":"2023-11-09T03:45:36.389474+0000","last_undegraded":"2023-11-09T03:45:36.389474+0000","last_fullsized":"2023-11-09T03:45:36.389474+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:29:21.531598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.444838+0000","last_change":"2023-11-09T03:45:31.415653+0000","last_active":"2023-11-09T03:45:35.444838+0000","last_peered":"2023-11-09T03:45:35.444838+0000","last_clean":"2023-11-09T03:45:35.444838+0000","last_became_active":"2023-11-09T03:45:31.415219+0000","last_became_peered":"2023-11-09T03:45:31.415219+0000","last_unstale":"2023-11-09T03:45:35.444838+0000","last_undegraded":"2023-11-09T03:45:35.444838+0000","last_fullsized":"2023-11-09T03:45:35.444838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:37:55.586534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440583+0000","last_change":"2023-11-09T03:45:30.414817+0000","last_active":"2023-11-09T03:45:35.440583+0000","last_peered":"2023-11-09T03:45:35.440583+0000","last_clean":"2023-11-09T03:45:35.440583+0000","last_became_active":"2023-11-09T03:45:30.414124+0000","last_became_peered":"2023-11-09T03:45:30.414124+0000","last_unstale":"2023-11-09T03:45:35.440583+0000","last_undegraded":"2023-11-09T03:45:35.440583+0000","last_fullsized":"2023-11-09T03:45:35.440583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:11:40.438915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388126+0000","last_change":"2023-11-09T03:45:35.448965+0000","last_active":"2023-11-09T03:45:36.388126+0000","last_peered":"2023-11-09T03:45:36.388126+0000","last_clean":"2023-11-09T03:45:36.388126+0000","last_became_active":"2023-11-09T03:45:35.448513+0000","last_became_peered":"2023-11-09T03:45:35.448513+0000","last_unstale":"2023-11-09T03:45:36.388126+0000","last_undegraded":"2023-11-09T03:45:36.388126+0000","last_fullsized":"2023-11-09T03:45:36.388126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:57:39.650773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440495+0000","last_change":"2023-11-09T03:45:31.407514+0000","last_active":"2023-11-09T03:45:35.440495+0000","last_peered":"2023-11-09T03:45:35.440495+0000","last_clean":"2023-11-09T03:45:35.440495+0000","last_became_active":"2023-11-09T03:45:31.407223+0000","last_became_peered":"2023-11-09T03:45:31.407223+0000","last_unstale":"2023-11-09T03:45:35.440495+0000","last_undegraded":"2023-11-09T03:45:35.440495+0000","last_fullsized":"2023-11-09T03:45:35.440495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:55:29.945168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440341+0000","last_change":"2023-11-09T03:45:30.418184+0000","last_active":"2023-11-09T03:45:35.440341+0000","last_peered":"2023-11-09T03:45:35.440341+0000","last_clean":"2023-11-09T03:45:35.440341+0000","last_became_active":"2023-11-09T03:45:30.417910+0000","last_became_peered":"2023-11-09T03:45:30.417910+0000","last_unstale":"2023-11-09T03:45:35.440341+0000","last_undegraded":"2023-11-09T03:45:35.440341+0000","last_fullsized":"2023-11-09T03:45:35.440341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:13:28.891277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388139+0000","last_change":"2023-11-09T03:45:35.621180+0000","last_active":"2023-11-09T03:45:36.388139+0000","last_peered":"2023-11-09T03:45:36.388139+0000","last_clean":"2023-11-09T03:45:36.388139+0000","last_became_active":"2023-11-09T03:45:35.620827+0000","last_became_peered":"2023-11-09T03:45:35.620827+0000","last_unstale":"2023-11-09T03:45:36.388139+0000","last_undegraded":"2023-11-09T03:45:36.388139+0000","last_fullsized":"2023-11-09T03:45:36.388139+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:57:57.109949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.262540+0000","last_change":"2023-11-09T03:45:35.620240+0000","last_active":"2023-11-09T03:45:37.262540+0000","last_peered":"2023-11-09T03:45:37.262540+0000","last_clean":"2023-11-09T03:45:37.262540+0000","last_became_active":"2023-11-09T03:45:35.619863+0000","last_became_peered":"2023-11-09T03:45:35.619863+0000","last_unstale":"2023-11-09T03:45:37.262540+0000","last_undegraded":"2023-11-09T03:45:37.262540+0000","last_fullsized":"2023-11-09T03:45:37.262540+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:55:33.973355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441745+0000","last_change":"2023-11-09T03:45:31.399005+0000","last_active":"2023-11-09T03:45:35.441745+0000","last_peered":"2023-11-09T03:45:35.441745+0000","last_clean":"2023-11-09T03:45:35.441745+0000","last_became_active":"2023-11-09T03:45:31.398753+0000","last_became_peered":"2023-11-09T03:45:31.398753+0000","last_unstale":"2023-11-09T03:45:35.441745+0000","last_undegraded":"2023-11-09T03:45:35.441745+0000","last_fullsized":"2023-11-09T03:45:35.441745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:44.473424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439574+0000","last_change":"2023-11-09T03:45:30.414502+0000","last_active":"2023-11-09T03:45:35.439574+0000","last_peered":"2023-11-09T03:45:35.439574+0000","last_clean":"2023-11-09T03:45:35.439574+0000","last_became_active":"2023-11-09T03:45:30.414208+0000","last_became_peered":"2023-11-09T03:45:30.414208+0000","last_unstale":"2023-11-09T03:45:35.439574+0000","last_undegraded":"2023-11-09T03:45:35.439574+0000","last_fullsized":"2023-11-09T03:45:35.439574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:34:48.400703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.264189+0000","last_change":"2023-11-09T03:45:35.456456+0000","last_active":"2023-11-09T03:45:37.264189+0000","last_peered":"2023-11-09T03:45:37.264189+0000","last_clean":"2023-11-09T03:45:37.264189+0000","last_became_active":"2023-11-09T03:45:35.455918+0000","last_became_peered":"2023-11-09T03:45:35.455918+0000","last_unstale":"2023-11-09T03:45:37.264189+0000","last_undegraded":"2023-11-09T03:45:37.264189+0000","last_fullsized":"2023-11-09T03:45:37.264189+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:45.477938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438961+0000","last_change":"2023-11-09T03:45:31.402076+0000","last_active":"2023-11-09T03:45:35.438961+0000","last_peered":"2023-11-09T03:45:35.438961+0000","last_clean":"2023-11-09T03:45:35.438961+0000","last_became_active":"2023-11-09T03:45:31.401819+0000","last_became_peered":"2023-11-09T03:45:31.401819+0000","last_unstale":"2023-11-09T03:45:35.438961+0000","last_undegraded":"2023-11-09T03:45:35.438961+0000","last_fullsized":"2023-11-09T03:45:35.438961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:27.400068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439935+0000","last_change":"2023-11-09T03:45:30.410155+0000","last_active":"2023-11-09T03:45:35.439935+0000","last_peered":"2023-11-09T03:45:35.439935+0000","last_clean":"2023-11-09T03:45:35.439935+0000","last_became_active":"2023-11-09T03:45:30.409732+0000","last_became_peered":"2023-11-09T03:45:30.409732+0000","last_unstale":"2023-11-09T03:45:35.439935+0000","last_undegraded":"2023-11-09T03:45:35.439935+0000","last_fullsized":"2023-11-09T03:45:35.439935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:34:48.400703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438019+0000","last_change":"2023-11-09T03:45:30.414923+0000","last_active":"2023-11-09T03:45:35.438019+0000","last_peered":"2023-11-09T03:45:35.438019+0000","last_clean":"2023-11-09T03:45:35.438019+0000","last_became_active":"2023-11-09T03:45:30.414641+0000","last_became_peered":"2023-11-09T03:45:30.414641+0000","last_unstale":"2023-11-09T03:45:35.438019+0000","last_undegraded":"2023-11-09T03:45:35.438019+0000","last_fullsized":"2023-11-09T03:45:35.438019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:12:07.086030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.443025+0000","last_change":"2023-11-09T03:45:31.400895+0000","last_active":"2023-11-09T03:45:35.443025+0000","last_peered":"2023-11-09T03:45:35.443025+0000","last_clean":"2023-11-09T03:45:35.443025+0000","last_became_active":"2023-11-09T03:45:31.400383+0000","last_became_peered":"2023-11-09T03:45:31.400383+0000","last_unstale":"2023-11-09T03:45:35.443025+0000","last_undegraded":"2023-11-09T03:45:35.443025+0000","last_fullsized":"2023-11-09T03:45:35.443025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:08.352566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:37.261594+0000","last_change":"2023-11-09T03:45:35.445171+0000","last_active":"2023-11-09T03:45:37.261594+0000","last_peered":"2023-11-09T03:45:37.261594+0000","last_clean":"2023-11-09T03:45:37.261594+0000","last_became_active":"2023-11-09T03:45:35.444835+0000","last_became_peered":"2023-11-09T03:45:35.444835+0000","last_unstale":"2023-11-09T03:45:37.261594+0000","last_undegraded":"2023-11-09T03:45:37.261594+0000","last_fullsized":"2023-11-09T03:45:37.261594+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:13:33.924212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439602+0000","last_change":"2023-11-09T03:45:30.416307+0000","last_active":"2023-11-09T03:45:35.439602+0000","last_peered":"2023-11-09T03:45:35.439602+0000","last_clean":"2023-11-09T03:45:35.439602+0000","last_became_active":"2023-11-09T03:45:30.416012+0000","last_became_peered":"2023-11-09T03:45:30.416012+0000","last_unstale":"2023-11-09T03:45:35.439602+0000","last_undegraded":"2023-11-09T03:45:35.439602+0000","last_fullsized":"2023-11-09T03:45:35.439602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:29.387691+0000","last_clean_scrub_stamp":"2023-11-09T03:45:29.387691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:13:28.891277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.435281+0000","last_change":"2023-11-09T03:45:31.395421+0000","last_active":"2023-11-09T03:45:35.435281+0000","last_peered":"2023-11-09T03:45:35.435281+0000","last_clean":"2023-11-09T03:45:35.435281+0000","last_became_active":"2023-11-09T03:45:31.395148+0000","last_became_peered":"2023-11-09T03:45:31.395148+0000","last_unstale":"2023-11-09T03:45:35.435281+0000","last_undegraded":"2023-11-09T03:45:35.435281+0000","last_fullsized":"2023-11-09T03:45:35.435281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:34:28.340170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-09T03:45:36.388718+0000","last_change":"2023-11-09T03:45:35.454065+0000","last_active":"2023-11-09T03:45:36.388718+0000","last_peered":"2023-11-09T03:45:36.388718+0000","last_clean":"2023-11-09T03:45:36.388718+0000","last_became_active":"2023-11-09T03:45:35.453677+0000","last_became_peered":"2023-11-09T03:45:35.453677+0000","last_unstale":"2023-11-09T03:45:36.388718+0000","last_undegraded":"2023-11-09T03:45:36.388718+0000","last_fullsized":"2023-11-09T03:45:36.388718+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:34.420625+0000","last_clean_scrub_stamp":"2023-11-09T03:45:34.420625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:34:18.265468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440121+0000","last_change":"2023-11-09T03:45:31.401829+0000","last_active":"2023-11-09T03:45:35.440121+0000","last_peered":"2023-11-09T03:45:35.440121+0000","last_clean":"2023-11-09T03:45:35.440121+0000","last_became_active":"2023-11-09T03:45:31.401545+0000","last_became_peered":"2023-11-09T03:45:31.401545+0000","last_unstale":"2023-11-09T03:45:35.440121+0000","last_undegraded":"2023-11-09T03:45:35.440121+0000","last_fullsized":"2023-11-09T03:45:35.440121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:27.400068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.432141+0000","last_change":"2023-11-09T03:45:31.410286+0000","last_active":"2023-11-09T03:45:35.432141+0000","last_peered":"2023-11-09T03:45:35.432141+0000","last_clean":"2023-11-09T03:45:35.432141+0000","last_became_active":"2023-11-09T03:45:31.410031+0000","last_became_peered":"2023-11-09T03:45:31.410031+0000","last_unstale":"2023-11-09T03:45:35.432141+0000","last_undegraded":"2023-11-09T03:45:35.432141+0000","last_fullsized":"2023-11-09T03:45:35.432141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:48:50.916355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440196+0000","last_change":"2023-11-09T03:45:31.412282+0000","last_active":"2023-11-09T03:45:35.440196+0000","last_peered":"2023-11-09T03:45:35.440196+0000","last_clean":"2023-11-09T03:45:35.440196+0000","last_became_active":"2023-11-09T03:45:31.408879+0000","last_became_peered":"2023-11-09T03:45:31.408879+0000","last_unstale":"2023-11-09T03:45:35.440196+0000","last_undegraded":"2023-11-09T03:45:35.440196+0000","last_fullsized":"2023-11-09T03:45:35.440196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:39:42.380963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442948+0000","last_change":"2023-11-09T03:45:31.400563+0000","last_active":"2023-11-09T03:45:35.442948+0000","last_peered":"2023-11-09T03:45:35.442948+0000","last_clean":"2023-11-09T03:45:35.442948+0000","last_became_active":"2023-11-09T03:45:31.399929+0000","last_became_peered":"2023-11-09T03:45:31.399929+0000","last_unstale":"2023-11-09T03:45:35.442948+0000","last_undegraded":"2023-11-09T03:45:35.442948+0000","last_fullsized":"2023-11-09T03:45:35.442948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:48:08.925647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439486+0000","last_change":"2023-11-09T03:45:31.397966+0000","last_active":"2023-11-09T03:45:35.439486+0000","last_peered":"2023-11-09T03:45:35.439486+0000","last_clean":"2023-11-09T03:45:35.439486+0000","last_became_active":"2023-11-09T03:45:31.397681+0000","last_became_peered":"2023-11-09T03:45:31.397681+0000","last_unstale":"2023-11-09T03:45:35.439486+0000","last_undegraded":"2023-11-09T03:45:35.439486+0000","last_fullsized":"2023-11-09T03:45:35.439486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:22.653144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442230+0000","last_change":"2023-11-09T03:45:31.398359+0000","last_active":"2023-11-09T03:45:35.442230+0000","last_peered":"2023-11-09T03:45:35.442230+0000","last_clean":"2023-11-09T03:45:35.442230+0000","last_became_active":"2023-11-09T03:45:31.398104+0000","last_became_peered":"2023-11-09T03:45:31.398104+0000","last_unstale":"2023-11-09T03:45:35.442230+0000","last_undegraded":"2023-11-09T03:45:35.442230+0000","last_fullsized":"2023-11-09T03:45:35.442230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:08.352566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440856+0000","last_change":"2023-11-09T03:45:31.408644+0000","last_active":"2023-11-09T03:45:35.440856+0000","last_peered":"2023-11-09T03:45:35.440856+0000","last_clean":"2023-11-09T03:45:35.440856+0000","last_became_active":"2023-11-09T03:45:31.408353+0000","last_became_peered":"2023-11-09T03:45:31.408353+0000","last_unstale":"2023-11-09T03:45:35.440856+0000","last_undegraded":"2023-11-09T03:45:35.440856+0000","last_fullsized":"2023-11-09T03:45:35.440856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:36:23.593484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438417+0000","last_change":"2023-11-09T03:45:31.395367+0000","last_active":"2023-11-09T03:45:35.438417+0000","last_peered":"2023-11-09T03:45:35.438417+0000","last_clean":"2023-11-09T03:45:35.438417+0000","last_became_active":"2023-11-09T03:45:31.395059+0000","last_became_peered":"2023-11-09T03:45:31.395059+0000","last_unstale":"2023-11-09T03:45:35.438417+0000","last_undegraded":"2023-11-09T03:45:35.438417+0000","last_fullsized":"2023-11-09T03:45:35.438417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:39:36.741003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439489+0000","last_change":"2023-11-09T03:45:31.408626+0000","last_active":"2023-11-09T03:45:35.439489+0000","last_peered":"2023-11-09T03:45:35.439489+0000","last_clean":"2023-11-09T03:45:35.439489+0000","last_became_active":"2023-11-09T03:45:31.408085+0000","last_became_peered":"2023-11-09T03:45:31.408085+0000","last_unstale":"2023-11-09T03:45:35.439489+0000","last_undegraded":"2023-11-09T03:45:35.439489+0000","last_fullsized":"2023-11-09T03:45:35.439489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:08.352566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439241+0000","last_change":"2023-11-09T03:45:31.402017+0000","last_active":"2023-11-09T03:45:35.439241+0000","last_peered":"2023-11-09T03:45:35.439241+0000","last_clean":"2023-11-09T03:45:35.439241+0000","last_became_active":"2023-11-09T03:45:31.401773+0000","last_became_peered":"2023-11-09T03:45:31.401773+0000","last_unstale":"2023-11-09T03:45:35.439241+0000","last_undegraded":"2023-11-09T03:45:35.439241+0000","last_fullsized":"2023-11-09T03:45:35.439241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:39:42.380963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440106+0000","last_change":"2023-11-09T03:45:31.418163+0000","last_active":"2023-11-09T03:45:35.440106+0000","last_peered":"2023-11-09T03:45:35.440106+0000","last_clean":"2023-11-09T03:45:35.440106+0000","last_became_active":"2023-11-09T03:45:31.417880+0000","last_became_peered":"2023-11-09T03:45:31.417880+0000","last_unstale":"2023-11-09T03:45:35.440106+0000","last_undegraded":"2023-11-09T03:45:35.440106+0000","last_fullsized":"2023-11-09T03:45:35.440106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:39:36.741003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442845+0000","last_change":"2023-11-09T03:45:31.404842+0000","last_active":"2023-11-09T03:45:35.442845+0000","last_peered":"2023-11-09T03:45:35.442845+0000","last_clean":"2023-11-09T03:45:35.442845+0000","last_became_active":"2023-11-09T03:45:31.404484+0000","last_became_peered":"2023-11-09T03:45:31.404484+0000","last_unstale":"2023-11-09T03:45:35.442845+0000","last_undegraded":"2023-11-09T03:45:35.442845+0000","last_fullsized":"2023-11-09T03:45:35.442845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:27.400068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440223+0000","last_change":"2023-11-09T03:45:31.404211+0000","last_active":"2023-11-09T03:45:35.440223+0000","last_peered":"2023-11-09T03:45:35.440223+0000","last_clean":"2023-11-09T03:45:35.440223+0000","last_became_active":"2023-11-09T03:45:31.403962+0000","last_became_peered":"2023-11-09T03:45:31.403962+0000","last_unstale":"2023-11-09T03:45:35.440223+0000","last_undegraded":"2023-11-09T03:45:35.440223+0000","last_fullsized":"2023-11-09T03:45:35.440223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:48:08.925647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.619170+0000","last_change":"2023-11-09T03:45:31.407273+0000","last_active":"2023-11-09T03:45:35.619170+0000","last_peered":"2023-11-09T03:45:35.619170+0000","last_clean":"2023-11-09T03:45:35.619170+0000","last_became_active":"2023-11-09T03:45:31.406956+0000","last_became_peered":"2023-11-09T03:45:31.406956+0000","last_unstale":"2023-11-09T03:45:35.619170+0000","last_undegraded":"2023-11-09T03:45:35.619170+0000","last_fullsized":"2023-11-09T03:45:35.619170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:13:04.311780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.432241+0000","last_change":"2023-11-09T03:45:31.402525+0000","last_active":"2023-11-09T03:45:35.432241+0000","last_peered":"2023-11-09T03:45:35.432241+0000","last_clean":"2023-11-09T03:45:35.432241+0000","last_became_active":"2023-11-09T03:45:31.402262+0000","last_became_peered":"2023-11-09T03:45:31.402262+0000","last_unstale":"2023-11-09T03:45:35.432241+0000","last_undegraded":"2023-11-09T03:45:35.432241+0000","last_fullsized":"2023-11-09T03:45:35.432241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:09.007153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441402+0000","last_change":"2023-11-09T03:45:31.411781+0000","last_active":"2023-11-09T03:45:35.441402+0000","last_peered":"2023-11-09T03:45:35.441402+0000","last_clean":"2023-11-09T03:45:35.441402+0000","last_became_active":"2023-11-09T03:45:31.411178+0000","last_became_peered":"2023-11-09T03:45:31.411178+0000","last_unstale":"2023-11-09T03:45:35.441402+0000","last_undegraded":"2023-11-09T03:45:35.441402+0000","last_fullsized":"2023-11-09T03:45:35.441402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:13:04.311780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439393+0000","last_change":"2023-11-09T03:45:31.409602+0000","last_active":"2023-11-09T03:45:35.439393+0000","last_peered":"2023-11-09T03:45:35.439393+0000","last_clean":"2023-11-09T03:45:35.439393+0000","last_became_active":"2023-11-09T03:45:31.409313+0000","last_became_peered":"2023-11-09T03:45:31.409313+0000","last_unstale":"2023-11-09T03:45:35.439393+0000","last_undegraded":"2023-11-09T03:45:35.439393+0000","last_fullsized":"2023-11-09T03:45:35.439393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:48:08.925647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441829+0000","last_change":"2023-11-09T03:45:31.394559+0000","last_active":"2023-11-09T03:45:35.441829+0000","last_peered":"2023-11-09T03:45:35.441829+0000","last_clean":"2023-11-09T03:45:35.441829+0000","last_became_active":"2023-11-09T03:45:31.394284+0000","last_became_peered":"2023-11-09T03:45:31.394284+0000","last_unstale":"2023-11-09T03:45:35.441829+0000","last_undegraded":"2023-11-09T03:45:35.441829+0000","last_fullsized":"2023-11-09T03:45:35.441829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:11:10.514126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.443281+0000","last_change":"2023-11-09T03:45:31.397341+0000","last_active":"2023-11-09T03:45:35.443281+0000","last_peered":"2023-11-09T03:45:35.443281+0000","last_clean":"2023-11-09T03:45:35.443281+0000","last_became_active":"2023-11-09T03:45:31.396986+0000","last_became_peered":"2023-11-09T03:45:31.396986+0000","last_unstale":"2023-11-09T03:45:35.443281+0000","last_undegraded":"2023-11-09T03:45:35.443281+0000","last_fullsized":"2023-11-09T03:45:35.443281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:34:49.405450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.439412+0000","last_change":"2023-11-09T03:45:31.402346+0000","last_active":"2023-11-09T03:45:35.439412+0000","last_peered":"2023-11-09T03:45:35.439412+0000","last_clean":"2023-11-09T03:45:35.439412+0000","last_became_active":"2023-11-09T03:45:31.401630+0000","last_became_peered":"2023-11-09T03:45:31.401630+0000","last_unstale":"2023-11-09T03:45:35.439412+0000","last_undegraded":"2023-11-09T03:45:35.439412+0000","last_fullsized":"2023-11-09T03:45:35.439412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:02:27.095864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.435980+0000","last_change":"2023-11-09T03:45:31.403882+0000","last_active":"2023-11-09T03:45:35.435980+0000","last_peered":"2023-11-09T03:45:35.435980+0000","last_clean":"2023-11-09T03:45:35.435980+0000","last_became_active":"2023-11-09T03:45:31.403637+0000","last_became_peered":"2023-11-09T03:45:31.403637+0000","last_unstale":"2023-11-09T03:45:35.435980+0000","last_undegraded":"2023-11-09T03:45:35.435980+0000","last_fullsized":"2023-11-09T03:45:35.435980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:39:30.327813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438776+0000","last_change":"2023-11-09T03:45:31.415358+0000","last_active":"2023-11-09T03:45:35.438776+0000","last_peered":"2023-11-09T03:45:35.438776+0000","last_clean":"2023-11-09T03:45:35.438776+0000","last_became_active":"2023-11-09T03:45:31.415018+0000","last_became_peered":"2023-11-09T03:45:31.415018+0000","last_unstale":"2023-11-09T03:45:35.438776+0000","last_undegraded":"2023-11-09T03:45:35.438776+0000","last_fullsized":"2023-11-09T03:45:35.438776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:08.352566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440770+0000","last_change":"2023-11-09T03:45:31.414693+0000","last_active":"2023-11-09T03:45:35.440770+0000","last_peered":"2023-11-09T03:45:35.440770+0000","last_clean":"2023-11-09T03:45:35.440770+0000","last_became_active":"2023-11-09T03:45:31.414400+0000","last_became_peered":"2023-11-09T03:45:31.414400+0000","last_unstale":"2023-11-09T03:45:35.440770+0000","last_undegraded":"2023-11-09T03:45:35.440770+0000","last_fullsized":"2023-11-09T03:45:35.440770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:22.653144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.618910+0000","last_change":"2023-11-09T03:45:31.419560+0000","last_active":"2023-11-09T03:45:35.618910+0000","last_peered":"2023-11-09T03:45:35.618910+0000","last_clean":"2023-11-09T03:45:35.618910+0000","last_became_active":"2023-11-09T03:45:31.418455+0000","last_became_peered":"2023-11-09T03:45:31.418455+0000","last_unstale":"2023-11-09T03:45:35.618910+0000","last_undegraded":"2023-11-09T03:45:35.618910+0000","last_fullsized":"2023-11-09T03:45:35.618910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:14:55.198635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.438871+0000","last_change":"2023-11-09T03:45:31.415460+0000","last_active":"2023-11-09T03:45:35.438871+0000","last_peered":"2023-11-09T03:45:35.438871+0000","last_clean":"2023-11-09T03:45:35.438871+0000","last_became_active":"2023-11-09T03:45:31.415210+0000","last_became_peered":"2023-11-09T03:45:31.415210+0000","last_unstale":"2023-11-09T03:45:35.438871+0000","last_undegraded":"2023-11-09T03:45:35.438871+0000","last_fullsized":"2023-11-09T03:45:35.438871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:44.473424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617339+0000","last_change":"2023-11-09T03:45:31.411588+0000","last_active":"2023-11-09T03:45:35.617339+0000","last_peered":"2023-11-09T03:45:35.617339+0000","last_clean":"2023-11-09T03:45:35.617339+0000","last_became_active":"2023-11-09T03:45:31.411168+0000","last_became_peered":"2023-11-09T03:45:31.411168+0000","last_unstale":"2023-11-09T03:45:35.617339+0000","last_undegraded":"2023-11-09T03:45:35.617339+0000","last_fullsized":"2023-11-09T03:45:35.617339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:39:42.380963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440027+0000","last_change":"2023-11-09T03:45:31.412481+0000","last_active":"2023-11-09T03:45:35.440027+0000","last_peered":"2023-11-09T03:45:35.440027+0000","last_clean":"2023-11-09T03:45:35.440027+0000","last_became_active":"2023-11-09T03:45:31.412167+0000","last_became_peered":"2023-11-09T03:45:31.412167+0000","last_unstale":"2023-11-09T03:45:35.440027+0000","last_undegraded":"2023-11-09T03:45:35.440027+0000","last_fullsized":"2023-11-09T03:45:35.440027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:35:22.653144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.617339+0000","last_change":"2023-11-09T03:45:31.405459+0000","last_active":"2023-11-09T03:45:35.617339+0000","last_peered":"2023-11-09T03:45:35.617339+0000","last_clean":"2023-11-09T03:45:35.617339+0000","last_became_active":"2023-11-09T03:45:31.405188+0000","last_became_peered":"2023-11-09T03:45:31.405188+0000","last_unstale":"2023-11-09T03:45:35.617339+0000","last_undegraded":"2023-11-09T03:45:35.617339+0000","last_fullsized":"2023-11-09T03:45:35.617339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:39:36.741003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.440927+0000","last_change":"2023-11-09T03:45:31.417756+0000","last_active":"2023-11-09T03:45:35.440927+0000","last_peered":"2023-11-09T03:45:35.440927+0000","last_clean":"2023-11-09T03:45:35.440927+0000","last_became_active":"2023-11-09T03:45:31.417488+0000","last_became_peered":"2023-11-09T03:45:31.417488+0000","last_unstale":"2023-11-09T03:45:35.440927+0000","last_undegraded":"2023-11-09T03:45:35.440927+0000","last_fullsized":"2023-11-09T03:45:35.440927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:36:23.593484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.442615+0000","last_change":"2023-11-09T03:45:31.397411+0000","last_active":"2023-11-09T03:45:35.442615+0000","last_peered":"2023-11-09T03:45:35.442615+0000","last_clean":"2023-11-09T03:45:35.442615+0000","last_became_active":"2023-11-09T03:45:31.396779+0000","last_became_peered":"2023-11-09T03:45:31.396779+0000","last_unstale":"2023-11-09T03:45:35.442615+0000","last_undegraded":"2023-11-09T03:45:35.442615+0000","last_fullsized":"2023-11-09T03:45:35.442615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:54:30.809126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.441937+0000","last_change":"2023-11-09T03:45:31.396447+0000","last_active":"2023-11-09T03:45:35.441937+0000","last_peered":"2023-11-09T03:45:35.441937+0000","last_clean":"2023-11-09T03:45:35.441937+0000","last_became_active":"2023-11-09T03:45:31.396184+0000","last_became_peered":"2023-11-09T03:45:31.396184+0000","last_unstale":"2023-11-09T03:45:35.441937+0000","last_undegraded":"2023-11-09T03:45:35.441937+0000","last_fullsized":"2023-11-09T03:45:35.441937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:48:08.925647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-09T03:45:35.618840+0000","last_change":"2023-11-09T03:45:31.405664+0000","last_active":"2023-11-09T03:45:35.618840+0000","last_peered":"2023-11-09T03:45:35.618840+0000","last_clean":"2023-11-09T03:45:35.618840+0000","last_became_active":"2023-11-09T03:45:31.404726+0000","last_became_peered":"2023-11-09T03:45:31.404726+0000","last_unstale":"2023-11-09T03:45:35.618840+0000","last_undegraded":"2023-11-09T03:45:35.618840+0000","last_fullsized":"2023-11-09T03:45:35.618840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:30.392438+0000","last_clean_scrub_stamp":"2023-11-09T03:45:30.392438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:41:08.550864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":695413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":104553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":104589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.216}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":104623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.252}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":868352,"data_stored":694850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-09T03:45:55.763 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-09T03:45:55.765 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-09T03:45:55.765 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-09T03:45:55.765 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-09T03:45:55.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.897+0000 7fdcea7b6640 1 Processor -- start 2023-11-09T03:45:55.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.897+0000 7fdcea7b6640 1 -- start start 2023-11-09T03:45:55.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.897+0000 7fdcea7b6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdce4157710 0x7fdce4167b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:55.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.897+0000 7fdcea7b6640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce415e000 con 0x7fdce40ce240 2023-11-09T03:45:55.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.897+0000 7fdcea7b6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce415e2e0 con 0x7fdce40cde60 2023-11-09T03:45:55.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.897+0000 7fdcea7b6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce415e5c0 con 0x7fdce4157710 2023-11-09T03:45:55.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.898+0000 7fdce37fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fdce40ce240 0x7fdce4157000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33688/0 (socket says 172.21.15.76:33688) 2023-11-09T03:45:55.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.898+0000 7fdce37fe640 1 -- 172.21.15.76:0/2557160354 learned_addr learned my addr 172.21.15.76:0/2557160354 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:55.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.898+0000 7fdce2ffd640 1 -- 172.21.15.76:0/2557160354 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3380978440 0 0) 0x7fdce415e000 con 0x7fdce40ce240 2023-11-09T03:45:55.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.898+0000 7fdce2ffd640 1 -- 172.21.15.76:0/2557160354 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdccc003520 con 0x7fdce40ce240 2023-11-09T03:45:55.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.898+0000 7fdce8d53640 1 --2- 172.21.15.76:0/2557160354 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdce4157710 0x7fdce4167b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:55.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.898+0000 7fdce2ffd640 1 -- 172.21.15.76:0/2557160354 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3214100018 0 0) 0x7fdccc003520 con 0x7fdce40ce240 2023-11-09T03:45:55.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.898+0000 7fdce2ffd640 1 -- 172.21.15.76:0/2557160354 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdce4157710 msgr2=0x7fdce4167b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:55.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.898+0000 7fdce2ffd640 1 --2- 172.21.15.76:0/2557160354 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdce4157710 0x7fdce4167b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.898+0000 7fdce2ffd640 1 -- 172.21.15.76:0/2557160354 >> v1:172.21.15.113:6789/0 conn(0x7fdce40cde60 legacy=0x7fdce415d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:55.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.898+0000 7fdce2ffd640 1 -- 172.21.15.76:0/2557160354 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce40d1120 con 0x7fdce40ce240 2023-11-09T03:45:55.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.898+0000 7fdce2ffd640 1 -- 172.21.15.76:0/2557160354 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fdcd8001670 con 0x7fdce40ce240 2023-11-09T03:45:55.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.898+0000 7fdce2ffd640 1 -- 172.21.15.76:0/2557160354 >> v1:172.21.15.76:6789/0 conn(0x7fdce40ce240 legacy=0x7fdce4157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:55.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.898+0000 7fdce2ffd640 1 --2- 172.21.15.76:0/2557160354 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdce4157710 0x7fdccc0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:55.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.898+0000 7fdce2ffd640 1 --2- 172.21.15.76:0/2557160354 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdccc003fd0 0x7fdccc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:55.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.898+0000 7fdce2ffd640 1 --2- 172.21.15.76:0/2557160354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdccc004d00 0x7fdccc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:55.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.898+0000 7fdce2ffd640 1 -- 172.21.15.76:0/2557160354 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce415e5c0 con 0x7fdccc004d00 2023-11-09T03:45:55.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.898+0000 7fdce2ffd640 1 -- 172.21.15.76:0/2557160354 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc00bb40 con 0x7fdccc003fd0 2023-11-09T03:45:55.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.898+0000 7fdce2ffd640 1 -- 172.21.15.76:0/2557160354 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc00bd00 con 0x7fdce4157710 2023-11-09T03:45:55.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.898+0000 7fdce8d53640 1 --2- 172.21.15.76:0/2557160354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdccc004d00 0x7fdccc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:55.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.898+0000 7fdce37fe640 1 --2- 172.21.15.76:0/2557160354 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdccc003fd0 0x7fdccc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:55.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.898+0000 7fdce3fff640 1 --2- 172.21.15.76:0/2557160354 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdce4157710 0x7fdccc0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:55.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.898+0000 7fdce8d53640 1 -- 172.21.15.76:0/2557160354 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdce4157710 msgr2=0x7fdccc0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:55.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.898+0000 7fdce8d53640 1 --2- 172.21.15.76:0/2557160354 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdce4157710 0x7fdccc0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.898+0000 7fdce8d53640 1 -- 172.21.15.76:0/2557160354 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdccc003fd0 msgr2=0x7fdccc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:55.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.898+0000 7fdce8d53640 1 --2- 172.21.15.76:0/2557160354 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdccc003fd0 0x7fdccc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.899+0000 7fdce8d53640 1 -- 172.21.15.76:0/2557160354 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdccc003520 con 0x7fdccc004d00 2023-11-09T03:45:55.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.899+0000 7fdce8d53640 1 --2- 172.21.15.76:0/2557160354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdccc004d00 0x7fdccc005180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc007360 tx=0x7fdccc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88449b707f61d757 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:55.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.899+0000 7fdce2ffd640 1 -- 172.21.15.76:0/2557160354 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc02ccd0 con 0x7fdccc004d00 2023-11-09T03:45:55.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.899+0000 7fdce2ffd640 1 -- 172.21.15.76:0/2557160354 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcdc02ce30 con 0x7fdccc004d00 2023-11-09T03:45:55.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.899+0000 7fdce2ffd640 1 -- 172.21.15.76:0/2557160354 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc034660 con 0x7fdccc004d00 2023-11-09T03:45:55.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.900+0000 7fdcea7b6640 1 -- 172.21.15.76:0/2557160354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdccc004d00 msgr2=0x7fdccc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:55.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.900+0000 7fdcea7b6640 1 --2- 172.21.15.76:0/2557160354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdccc004d00 0x7fdccc005180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc007360 tx=0x7fdccc0057f0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.900+0000 7fdce8d53640 1 -- 172.21.15.76:0/2557160354 reap_dead start 2023-11-09T03:45:55.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.900+0000 7fdcea7b6640 1 -- 172.21.15.76:0/2557160354 shutdown_connections 2023-11-09T03:45:55.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.900+0000 7fdcea7b6640 1 -- 172.21.15.76:0/2557160354 >> 172.21.15.76:0/2557160354 conn(0x7fdce4071f30 msgr2=0x7fdce40cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:55.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.900+0000 7fdcea7b6640 1 -- 172.21.15.76:0/2557160354 shutdown_connections 2023-11-09T03:45:55.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.900+0000 7fdcea7b6640 1 -- 172.21.15.76:0/2557160354 wait complete. 2023-11-09T03:45:55.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.901+0000 7fdcea7b6640 1 Processor -- start 2023-11-09T03:45:55.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.901+0000 7fdcea7b6640 1 -- start start 2023-11-09T03:45:55.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.902+0000 7fdcea7b6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdce41533a0 0x7fdce40c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:55.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.902+0000 7fdcea7b6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdce40ca430 0x7fdce415c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:55.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.902+0000 7fdcea7b6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdce415cc20 0x7fdce40cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:55.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.902+0000 7fdcea7b6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce4158c90 con 0x7fdce41533a0 2023-11-09T03:45:55.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.902+0000 7fdcea7b6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce415c250 con 0x7fdce415cc20 2023-11-09T03:45:55.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.902+0000 7fdcea7b6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce415b680 con 0x7fdce40ca430 2023-11-09T03:45:55.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.902+0000 7fdce3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdce41533a0 0x7fdce40c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:55.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.902+0000 7fdce3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdce41533a0 0x7fdce40c9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36504/0 (socket says 172.21.15.76:36504) 2023-11-09T03:45:55.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.902+0000 7fdce3fff640 1 -- 172.21.15.76:0/1358244763 learned_addr learned my addr 172.21.15.76:0/1358244763 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:55.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.902+0000 7fdce8d53640 1 --2- 172.21.15.76:0/1358244763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdce415cc20 0x7fdce40cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:55.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.902+0000 7fdce3fff640 1 -- 172.21.15.76:0/1358244763 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdce40ca430 msgr2=0x7fdce415c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:55.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.902+0000 7fdce37fe640 1 --2- 172.21.15.76:0/1358244763 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdce40ca430 0x7fdce415c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:55.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.902+0000 7fdce3fff640 1 --2- 172.21.15.76:0/1358244763 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdce40ca430 0x7fdce415c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.902+0000 7fdce3fff640 1 -- 172.21.15.76:0/1358244763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdce415cc20 msgr2=0x7fdce40cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:55.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.902+0000 7fdce3fff640 1 --2- 172.21.15.76:0/1358244763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdce415cc20 0x7fdce40cc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:55.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.902+0000 7fdce3fff640 1 -- 172.21.15.76:0/1358244763 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce415b920 con 0x7fdce41533a0 2023-11-09T03:45:55.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.902+0000 7fdce8d53640 1 --2- 172.21.15.76:0/1358244763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdce415cc20 0x7fdce40cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:45:55.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.903+0000 7fdce3fff640 1 --2- 172.21.15.76:0/1358244763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdce41533a0 0x7fdce40c9ef0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400f820 tx=0x7fdcd400fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:55.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.903+0000 7fdce0ff9640 1 -- 172.21.15.76:0/1358244763 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd40050d0 con 0x7fdce41533a0 2023-11-09T03:45:55.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.903+0000 7fdce0ff9640 1 -- 172.21.15.76:0/1358244763 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd4005230 con 0x7fdce41533a0 2023-11-09T03:45:55.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.903+0000 7fdcea7b6640 1 -- 172.21.15.76:0/1358244763 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce415bc00 con 0x7fdce41533a0 2023-11-09T03:45:55.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.903+0000 7fdce0ff9640 1 -- 172.21.15.76:0/1358244763 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd400d730 con 0x7fdce41533a0 2023-11-09T03:45:55.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.903+0000 7fdcea7b6640 1 -- 172.21.15.76:0/1358244763 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce4153780 con 0x7fdce41533a0 2023-11-09T03:45:55.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.904+0000 7fdcea7b6640 1 -- 172.21.15.76:0/1358244763 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce415d000 con 0x7fdce41533a0 2023-11-09T03:45:55.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.904+0000 7fdce0ff9640 1 -- 172.21.15.76:0/1358244763 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdcd400d890 con 0x7fdce41533a0 2023-11-09T03:45:55.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.905+0000 7fdce0ff9640 1 --2- 172.21.15.76:0/1358244763 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdcd006aea0 0x7fdcd006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:55.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.905+0000 7fdce0ff9640 1 -- 172.21.15.76:0/1358244763 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdcd4089820 con 0x7fdce41533a0 2023-11-09T03:45:55.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.905+0000 7fdce37fe640 1 --2- 172.21.15.76:0/1358244763 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdcd006aea0 0x7fdcd006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:55.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.905+0000 7fdce37fe640 1 --2- 172.21.15.76:0/1358244763 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdcd006aea0 0x7fdcd006d360 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8001050 tx=0x7fdcd8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:55.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:55.909+0000 7fdce0ff9640 1 -- 172.21.15.76:0/1358244763 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdcd40545e0 con 0x7fdce41533a0 2023-11-09T03:45:56.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.107+0000 7fdcea7b6640 1 -- 172.21.15.76:0/1358244763 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fdce4153910 con 0x7fdce41533a0 2023-11-09T03:45:56.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.108+0000 7fdce0ff9640 1 -- 172.21.15.76:0/1358244763 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fdcd4057c00 con 0x7fdce41533a0 2023-11-09T03:45:56.107 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:56.108 INFO:teuthology.orchestra.run.smithi076.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-09T03:45:56.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.110+0000 7fdcea7b6640 1 -- 172.21.15.76:0/1358244763 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdcd006aea0 msgr2=0x7fdcd006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:56.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.110+0000 7fdcea7b6640 1 --2- 172.21.15.76:0/1358244763 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdcd006aea0 0x7fdcd006d360 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8001050 tx=0x7fdcd8009040 comp rx=0 tx=0).stop 2023-11-09T03:45:56.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.110+0000 7fdcea7b6640 1 -- 172.21.15.76:0/1358244763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdce41533a0 msgr2=0x7fdce40c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:56.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.110+0000 7fdcea7b6640 1 --2- 172.21.15.76:0/1358244763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdce41533a0 0x7fdce40c9ef0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400f820 tx=0x7fdcd400fce0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.110+0000 7fdcea7b6640 1 -- 172.21.15.76:0/1358244763 shutdown_connections 2023-11-09T03:45:56.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.110+0000 7fdcea7b6640 1 --2- 172.21.15.76:0/1358244763 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdcd006aea0 0x7fdcd006d360 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.110+0000 7fdcea7b6640 1 --2- 172.21.15.76:0/1358244763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdce415cc20 0x7fdce40cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.110+0000 7fdcea7b6640 1 --2- 172.21.15.76:0/1358244763 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdce40ca430 0x7fdce415c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.110+0000 7fdcea7b6640 1 --2- 172.21.15.76:0/1358244763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdce41533a0 0x7fdce40c9ef0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.110+0000 7fdcea7b6640 1 -- 172.21.15.76:0/1358244763 >> 172.21.15.76:0/1358244763 conn(0x7fdce4071f30 msgr2=0x7fdce40b7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:56.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.110+0000 7fdcea7b6640 1 -- 172.21.15.76:0/1358244763 shutdown_connections 2023-11-09T03:45:56.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.110+0000 7fdcea7b6640 1 -- 172.21.15.76:0/1358244763 wait complete. 2023-11-09T03:45:56.120 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-09T03:45:56.120 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:45:56.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.222+0000 7f2c32610640 1 Processor -- start 2023-11-09T03:45:56.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.222+0000 7f2c32610640 1 -- start start 2023-11-09T03:45:56.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.222+0000 7f2c32610640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2c2c157710 0x7f2c2c167b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:56.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.222+0000 7f2c32610640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c2c15e000 con 0x7f2c2c0cde60 2023-11-09T03:45:56.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.222+0000 7f2c32610640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c2c15e2e0 con 0x7f2c2c0ce240 2023-11-09T03:45:56.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.222+0000 7f2c32610640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c2c15e5c0 con 0x7f2c2c157710 2023-11-09T03:45:56.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.223+0000 7f2c30bad640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2c2c157710 0x7f2c2c167b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:56.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.223+0000 7f2c2bfff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2c2c0cde60 0x7f2c2c15d560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33698/0 (socket says 172.21.15.76:33698) 2023-11-09T03:45:56.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.223+0000 7f2c2bfff640 1 -- 172.21.15.76:0/4214478053 learned_addr learned my addr 172.21.15.76:0/4214478053 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:56.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.223+0000 7f2c30bad640 1 -- 172.21.15.76:0/4214478053 >> v1:172.21.15.113:6789/0 conn(0x7f2c2c0ce240 legacy=0x7f2c2c157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:56.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.223+0000 7f2c30bad640 1 -- 172.21.15.76:0/4214478053 >> v1:172.21.15.76:6789/0 conn(0x7f2c2c0cde60 legacy=0x7f2c2c15d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:56.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.223+0000 7f2c30bad640 1 -- 172.21.15.76:0/4214478053 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2c0d1120 con 0x7f2c2c157710 2023-11-09T03:45:56.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.224+0000 7f2c30bad640 1 --2- 172.21.15.76:0/4214478053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c2c157710 0x7f2c2c167b90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2c24009d90 tx=0x7f2c2402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef97def9f94f4e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:56.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.224+0000 7f2c2affd640 1 -- 172.21.15.76:0/4214478053 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2402cd00 con 0x7f2c2c157710 2023-11-09T03:45:56.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.224+0000 7f2c2affd640 1 -- 172.21.15.76:0/4214478053 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c2402ce60 con 0x7f2c2c157710 2023-11-09T03:45:56.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.224+0000 7f2c2affd640 1 -- 172.21.15.76:0/4214478053 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c24034660 con 0x7f2c2c157710 2023-11-09T03:45:56.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.224+0000 7f2c32610640 1 -- 172.21.15.76:0/4214478053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c2c157710 msgr2=0x7f2c2c167b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:56.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.224+0000 7f2c32610640 1 --2- 172.21.15.76:0/4214478053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c2c157710 0x7f2c2c167b90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2c24009d90 tx=0x7f2c2402bf50 comp rx=0 tx=0).stop 2023-11-09T03:45:56.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.224+0000 7f2c32610640 1 -- 172.21.15.76:0/4214478053 shutdown_connections 2023-11-09T03:45:56.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.224+0000 7f2c32610640 1 --2- 172.21.15.76:0/4214478053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c2c157710 0x7f2c2c167b90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.224+0000 7f2c32610640 1 -- 172.21.15.76:0/4214478053 >> 172.21.15.76:0/4214478053 conn(0x7f2c2c071f30 msgr2=0x7f2c2c0cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:56.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.224+0000 7f2c32610640 1 -- 172.21.15.76:0/4214478053 shutdown_connections 2023-11-09T03:45:56.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.224+0000 7f2c32610640 1 -- 172.21.15.76:0/4214478053 wait complete. 2023-11-09T03:45:56.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.225+0000 7f2c32610640 1 Processor -- start 2023-11-09T03:45:56.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.226+0000 7f2c32610640 1 -- start start 2023-11-09T03:45:56.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.226+0000 7f2c32610640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c2c0cde60 0x7f2c2c1f2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:56.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.226+0000 7f2c32610640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c2c0ce240 0x7f2c2c0ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:56.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.226+0000 7f2c32610640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c2c157710 0x7f2c2c159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:56.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.226+0000 7f2c32610640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c0c9ef0 con 0x7f2c2c0cde60 2023-11-09T03:45:56.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.226+0000 7f2c32610640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c15c550 con 0x7f2c2c157710 2023-11-09T03:45:56.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.226+0000 7f2c32610640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c15c830 con 0x7f2c2c0ce240 2023-11-09T03:45:56.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.226+0000 7f2c2bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c2c0cde60 0x7f2c2c1f2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:56.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.226+0000 7f2c2bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c2c0cde60 0x7f2c2c1f2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36512/0 (socket says 172.21.15.76:36512) 2023-11-09T03:45:56.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.226+0000 7f2c2bfff640 1 -- 172.21.15.76:0/115314409 learned_addr learned my addr 172.21.15.76:0/115314409 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:56.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.227+0000 7f2c2bfff640 1 -- 172.21.15.76:0/115314409 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c2c0ce240 msgr2=0x7f2c2c0ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:56.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.227+0000 7f2c2bfff640 1 --2- 172.21.15.76:0/115314409 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c2c0ce240 0x7f2c2c0ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.227+0000 7f2c2bfff640 1 -- 172.21.15.76:0/115314409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c2c157710 msgr2=0x7f2c2c159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:56.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.227+0000 7f2c30bad640 1 --2- 172.21.15.76:0/115314409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c2c157710 0x7f2c2c159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:56.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.227+0000 7f2c2bfff640 1 --2- 172.21.15.76:0/115314409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c2c157710 0x7f2c2c159a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.227+0000 7f2c2bfff640 1 -- 172.21.15.76:0/115314409 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c24009a70 con 0x7f2c2c0cde60 2023-11-09T03:45:56.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.227+0000 7f2c2bfff640 1 --2- 172.21.15.76:0/115314409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c2c0cde60 0x7f2c2c1f2820 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000c4a0 tx=0x7f2c2000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:56.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.227+0000 7f2c28ff9640 1 -- 172.21.15.76:0/115314409 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20005ce0 con 0x7f2c2c0cde60 2023-11-09T03:45:56.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.227+0000 7f2c28ff9640 1 -- 172.21.15.76:0/115314409 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c20005e40 con 0x7f2c2c0cde60 2023-11-09T03:45:56.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.227+0000 7f2c28ff9640 1 -- 172.21.15.76:0/115314409 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20007270 con 0x7f2c2c0cde60 2023-11-09T03:45:56.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.227+0000 7f2c32610640 1 -- 172.21.15.76:0/115314409 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c2c15cb10 con 0x7f2c2c0cde60 2023-11-09T03:45:56.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.227+0000 7f2c32610640 1 -- 172.21.15.76:0/115314409 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c2c0ca140 con 0x7f2c2c0cde60 2023-11-09T03:45:56.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.228+0000 7f2c32610640 1 -- 172.21.15.76:0/115314409 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c2c15d030 con 0x7f2c2c0cde60 2023-11-09T03:45:56.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.228+0000 7f2c28ff9640 1 -- 172.21.15.76:0/115314409 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2c20007490 con 0x7f2c2c0cde60 2023-11-09T03:45:56.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.229+0000 7f2c28ff9640 1 --2- 172.21.15.76:0/115314409 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2c1806ae70 0x7f2c1806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:56.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.229+0000 7f2c28ff9640 1 -- 172.21.15.76:0/115314409 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2c20088c80 con 0x7f2c2c0cde60 2023-11-09T03:45:56.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.229+0000 7f2c2b7fe640 1 --2- 172.21.15.76:0/115314409 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2c1806ae70 0x7f2c1806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:56.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.229+0000 7f2c2b7fe640 1 --2- 172.21.15.76:0/115314409 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2c1806ae70 0x7f2c1806d330 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c000f60 tx=0x7f2c1c0079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:56.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.233+0000 7f2c28ff9640 1 -- 172.21.15.76:0/115314409 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2c20053b80 con 0x7f2c2c0cde60 2023-11-09T03:45:56.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.401+0000 7f2c32610640 1 -- 172.21.15.76:0/115314409 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c2c074290 con 0x7f2c2c0cde60 2023-11-09T03:45:56.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.402+0000 7f2c28ff9640 1 -- 172.21.15.76:0/115314409 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18334 (secure 0 0 0) 0x7f2c20017020 con 0x7f2c2c0cde60 2023-11-09T03:45:56.402 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:56.402 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":25,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:45:35.781752+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:45:56.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.404+0000 7f2c32610640 1 -- 172.21.15.76:0/115314409 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2c1806ae70 msgr2=0x7f2c1806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:56.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.404+0000 7f2c32610640 1 --2- 172.21.15.76:0/115314409 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2c1806ae70 0x7f2c1806d330 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c000f60 tx=0x7f2c1c0079b0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.404+0000 7f2c32610640 1 -- 172.21.15.76:0/115314409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c2c0cde60 msgr2=0x7f2c2c1f2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:56.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.404+0000 7f2c32610640 1 --2- 172.21.15.76:0/115314409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c2c0cde60 0x7f2c2c1f2820 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000c4a0 tx=0x7f2c2000c960 comp rx=0 tx=0).stop 2023-11-09T03:45:56.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.404+0000 7f2c32610640 1 -- 172.21.15.76:0/115314409 shutdown_connections 2023-11-09T03:45:56.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.404+0000 7f2c32610640 1 --2- 172.21.15.76:0/115314409 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2c1806ae70 0x7f2c1806d330 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.404+0000 7f2c32610640 1 --2- 172.21.15.76:0/115314409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c2c157710 0x7f2c2c159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.404+0000 7f2c32610640 1 --2- 172.21.15.76:0/115314409 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c2c0ce240 0x7f2c2c0ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.404+0000 7f2c32610640 1 --2- 172.21.15.76:0/115314409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c2c0cde60 0x7f2c2c1f2820 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.404+0000 7f2c32610640 1 -- 172.21.15.76:0/115314409 >> 172.21.15.76:0/115314409 conn(0x7f2c2c071f30 msgr2=0x7f2c2c0c6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:56.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.404+0000 7f2c32610640 1 -- 172.21.15.76:0/115314409 shutdown_connections 2023-11-09T03:45:56.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.404+0000 7f2c32610640 1 -- 172.21.15.76:0/115314409 wait complete. 2023-11-09T03:45:56.414 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:45:28.704715+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:45:29.717614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-09T03:45:33.440577+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-09T03:45:56.415 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:45:56.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.519+0000 7f6ab3490640 1 Processor -- start 2023-11-09T03:45:56.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.519+0000 7f6ab3490640 1 -- start start 2023-11-09T03:45:56.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.520+0000 7f6ab3490640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6aac0b2d30 0x7f6aac06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:56.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.520+0000 7f6ab3490640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6aac06c1d0 0x7f6aac0cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:56.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.520+0000 7f6ab3490640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aac06ca40 con 0x7f6aac160430 2023-11-09T03:45:56.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.520+0000 7f6ab3490640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aac0cb580 con 0x7f6aac0b2d30 2023-11-09T03:45:56.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.520+0000 7f6ab3490640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac06cd20 con 0x7f6aac06c1d0 2023-11-09T03:45:56.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.520+0000 7f6ab0a2b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6aac0b2d30 0x7f6aac06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:56.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.520+0000 7f6ab0a2b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6aac0b2d30 0x7f6aac06bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36518/0 (socket says 172.21.15.76:36518) 2023-11-09T03:45:56.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.520+0000 7f6ab0a2b640 1 -- 172.21.15.76:0/893012292 learned_addr learned my addr 172.21.15.76:0/893012292 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:56.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.520+0000 7f6ab1a2d640 1 --2- 172.21.15.76:0/893012292 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6aac06c1d0 0x7f6aac0cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:56.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.520+0000 7f6ab0a2b640 1 -- 172.21.15.76:0/893012292 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6aac06c1d0 msgr2=0x7f6aac0cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:56.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.520+0000 7f6ab0a2b640 1 --2- 172.21.15.76:0/893012292 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6aac06c1d0 0x7f6aac0cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.520+0000 7f6ab0a2b640 1 -- 172.21.15.76:0/893012292 >> v1:172.21.15.113:6789/0 conn(0x7f6aac160430 legacy=0x7f6aac0b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:56.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.520+0000 7f6ab0a2b640 1 -- 172.21.15.76:0/893012292 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aac06d000 con 0x7f6aac0b2d30 2023-11-09T03:45:56.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.521+0000 7f6ab0a2b640 1 --2- 172.21.15.76:0/893012292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aac0b2d30 0x7f6aac06bc90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c009870 tx=0x7f6a9c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15b5d6c6539f7eb server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:56.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.521+0000 7f6aa3fff640 1 -- 172.21.15.76:0/893012292 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c030840 con 0x7f6aac0b2d30 2023-11-09T03:45:56.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.521+0000 7f6aa3fff640 1 -- 172.21.15.76:0/893012292 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9c0309a0 con 0x7f6aac0b2d30 2023-11-09T03:45:56.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.521+0000 7f6aa3fff640 1 -- 172.21.15.76:0/893012292 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c030d30 con 0x7f6aac0b2d30 2023-11-09T03:45:56.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.521+0000 7f6ab3490640 1 -- 172.21.15.76:0/893012292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aac0b2d30 msgr2=0x7f6aac06bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:56.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.521+0000 7f6ab3490640 1 --2- 172.21.15.76:0/893012292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aac0b2d30 0x7f6aac06bc90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c009870 tx=0x7f6a9c02b980 comp rx=0 tx=0).stop 2023-11-09T03:45:56.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.522+0000 7f6ab3490640 1 -- 172.21.15.76:0/893012292 shutdown_connections 2023-11-09T03:45:56.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.522+0000 7f6ab3490640 1 --2- 172.21.15.76:0/893012292 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6aac06c1d0 0x7f6aac0cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.522+0000 7f6ab3490640 1 --2- 172.21.15.76:0/893012292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aac0b2d30 0x7f6aac06bc90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.522+0000 7f6ab3490640 1 -- 172.21.15.76:0/893012292 >> 172.21.15.76:0/893012292 conn(0x7f6aac06ea20 msgr2=0x7f6aac06ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:56.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.522+0000 7f6ab3490640 1 -- 172.21.15.76:0/893012292 shutdown_connections 2023-11-09T03:45:56.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.522+0000 7f6ab3490640 1 -- 172.21.15.76:0/893012292 wait complete. 2023-11-09T03:45:56.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.523+0000 7f6ab3490640 1 Processor -- start 2023-11-09T03:45:56.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.523+0000 7f6ab3490640 1 -- start start 2023-11-09T03:45:56.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.523+0000 7f6ab3490640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6aac06c1d0 0x7f6aac15d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:56.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.523+0000 7f6ab3490640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6aac0b2d30 0x7f6aac15ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:56.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.523+0000 7f6ab3490640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aac160430 0x7f6aac0c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:56.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.523+0000 7f6ab3490640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac15e8d0 con 0x7f6aac160430 2023-11-09T03:45:56.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.523+0000 7f6ab3490640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac0c4bb0 con 0x7f6aac06c1d0 2023-11-09T03:45:56.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.523+0000 7f6ab3490640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac15dcc0 con 0x7f6aac0b2d30 2023-11-09T03:45:56.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.524+0000 7f6ab1a2d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aac160430 0x7f6aac0c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:56.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.524+0000 7f6ab1a2d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aac160430 0x7f6aac0c4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36522/0 (socket says 172.21.15.76:36522) 2023-11-09T03:45:56.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.524+0000 7f6ab1a2d640 1 -- 172.21.15.76:0/2763178516 learned_addr learned my addr 172.21.15.76:0/2763178516 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:56.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.524+0000 7f6ab0a2b640 1 --2- 172.21.15.76:0/2763178516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6aac0b2d30 0x7f6aac15ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:56.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.524+0000 7f6ab122c640 1 --2- 172.21.15.76:0/2763178516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6aac06c1d0 0x7f6aac15d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:56.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.524+0000 7f6ab1a2d640 1 -- 172.21.15.76:0/2763178516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6aac0b2d30 msgr2=0x7f6aac15ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:56.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.524+0000 7f6ab1a2d640 1 --2- 172.21.15.76:0/2763178516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6aac0b2d30 0x7f6aac15ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.524+0000 7f6ab1a2d640 1 -- 172.21.15.76:0/2763178516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6aac06c1d0 msgr2=0x7f6aac15d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:56.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.524+0000 7f6ab1a2d640 1 --2- 172.21.15.76:0/2763178516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6aac06c1d0 0x7f6aac15d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.524+0000 7f6ab1a2d640 1 -- 172.21.15.76:0/2763178516 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9c0094d0 con 0x7f6aac160430 2023-11-09T03:45:56.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.524+0000 7f6ab1a2d640 1 --2- 172.21.15.76:0/2763178516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aac160430 0x7f6aac0c4670 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4007e70 tx=0x7f6aa400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:56.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.524+0000 7f6aa1ffb640 1 -- 172.21.15.76:0/2763178516 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4014620 con 0x7f6aac160430 2023-11-09T03:45:56.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.524+0000 7f6aa1ffb640 1 -- 172.21.15.76:0/2763178516 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa40147a0 con 0x7f6aac160430 2023-11-09T03:45:56.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.524+0000 7f6ab3490640 1 -- 172.21.15.76:0/2763178516 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aac15dfa0 con 0x7f6aac160430 2023-11-09T03:45:56.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.524+0000 7f6aa1ffb640 1 -- 172.21.15.76:0/2763178516 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4014b50 con 0x7f6aac160430 2023-11-09T03:45:56.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.524+0000 7f6ab3490640 1 -- 172.21.15.76:0/2763178516 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aac15e560 con 0x7f6aac160430 2023-11-09T03:45:56.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.525+0000 7f6ab3490640 1 -- 172.21.15.76:0/2763178516 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aac15ea80 con 0x7f6aac160430 2023-11-09T03:45:56.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.526+0000 7f6aa1ffb640 1 -- 172.21.15.76:0/2763178516 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6aa4014cb0 con 0x7f6aac160430 2023-11-09T03:45:56.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.526+0000 7f6aa1ffb640 1 --2- 172.21.15.76:0/2763178516 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6a9006ae70 0x7f6a9006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:56.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.526+0000 7f6aa1ffb640 1 -- 172.21.15.76:0/2763178516 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6aa408a2f0 con 0x7f6aac160430 2023-11-09T03:45:56.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.526+0000 7f6ab122c640 1 --2- 172.21.15.76:0/2763178516 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6a9006ae70 0x7f6a9006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:56.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.527+0000 7f6ab122c640 1 --2- 172.21.15.76:0/2763178516 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6a9006ae70 0x7f6a9006d330 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8000f60 tx=0x7f6aa8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:56.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.530+0000 7f6aa1ffb640 1 -- 172.21.15.76:0/2763178516 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6aac15ea80 con 0x7f6aac160430 2023-11-09T03:45:56.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.700+0000 7f6ab3490640 1 -- 172.21.15.76:0/2763178516 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6aac15ea80 con 0x7f6aac160430 2023-11-09T03:45:56.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.701+0000 7f6aa1ffb640 1 -- 172.21.15.76:0/2763178516 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6aac15ea80 con 0x7f6aac160430 2023-11-09T03:45:56.700 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:45:56.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.702+0000 7f6ab3490640 1 -- 172.21.15.76:0/2763178516 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6a9006ae70 msgr2=0x7f6a9006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:56.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.702+0000 7f6ab3490640 1 --2- 172.21.15.76:0/2763178516 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6a9006ae70 0x7f6a9006d330 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8000f60 tx=0x7f6aa8009040 comp rx=0 tx=0).stop 2023-11-09T03:45:56.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.702+0000 7f6ab3490640 1 -- 172.21.15.76:0/2763178516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aac160430 msgr2=0x7f6aac0c4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:56.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.702+0000 7f6ab3490640 1 --2- 172.21.15.76:0/2763178516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aac160430 0x7f6aac0c4670 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4007e70 tx=0x7f6aa400e4b0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.703+0000 7f6ab3490640 1 -- 172.21.15.76:0/2763178516 shutdown_connections 2023-11-09T03:45:56.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.703+0000 7f6ab3490640 1 --2- 172.21.15.76:0/2763178516 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6a9006ae70 0x7f6a9006d330 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.703+0000 7f6ab3490640 1 --2- 172.21.15.76:0/2763178516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aac160430 0x7f6aac0c4670 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.703+0000 7f6ab3490640 1 --2- 172.21.15.76:0/2763178516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6aac0b2d30 0x7f6aac15ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.703+0000 7f6ab3490640 1 --2- 172.21.15.76:0/2763178516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6aac06c1d0 0x7f6aac15d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.703+0000 7f6ab3490640 1 -- 172.21.15.76:0/2763178516 >> 172.21.15.76:0/2763178516 conn(0x7f6aac06ea20 msgr2=0x7f6aac0b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:56.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.703+0000 7f6ab3490640 1 -- 172.21.15.76:0/2763178516 shutdown_connections 2023-11-09T03:45:56.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.703+0000 7f6ab3490640 1 -- 172.21.15.76:0/2763178516 wait complete. 2023-11-09T03:45:56.712 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:45:56.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.815+0000 7f28493a9640 1 Processor -- start 2023-11-09T03:45:56.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.815+0000 7f28493a9640 1 -- start start 2023-11-09T03:45:56.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.816+0000 7f28493a9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f284415ad30 0x7f28440adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:56.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.816+0000 7f28493a9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f28440ad730 0x7f2844070710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:56.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.816+0000 7f28493a9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28440c3900 con 0x7f2844070c50 2023-11-09T03:45:56.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.816+0000 7f28493a9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28440ae260 con 0x7f284415ad30 2023-11-09T03:45:56.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.816+0000 7f28493a9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f284415f750 con 0x7f28440ad730 2023-11-09T03:45:56.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.816+0000 7f2842ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f284415ad30 0x7f28440adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:56.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.816+0000 7f28437fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2844070c50 0x7f28440c8fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33706/0 (socket says 172.21.15.76:33706) 2023-11-09T03:45:56.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.816+0000 7f28437fe640 1 -- 172.21.15.76:0/3834746465 learned_addr learned my addr 172.21.15.76:0/3834746465 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:56.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.816+0000 7f28427fc640 1 --2- 172.21.15.76:0/3834746465 >> v2:172.21.15.113:3300/0 conn(0x7f28440ad730 0x7f2844070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:56.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.816+0000 7f2842ffd640 1 -- 172.21.15.76:0/3834746465 >> v2:172.21.15.113:3300/0 conn(0x7f28440ad730 msgr2=0x7f2844070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:56.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.816+0000 7f2842ffd640 1 --2- 172.21.15.76:0/3834746465 >> v2:172.21.15.113:3300/0 conn(0x7f28440ad730 0x7f2844070710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.816+0000 7f2842ffd640 1 -- 172.21.15.76:0/3834746465 >> v1:172.21.15.76:6789/0 conn(0x7f2844070c50 legacy=0x7f28440c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:56.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.816+0000 7f2842ffd640 1 -- 172.21.15.76:0/3834746465 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28440c3be0 con 0x7f284415ad30 2023-11-09T03:45:56.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.816+0000 7f2841ffb640 1 -- 172.21.15.76:0/3834746465 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3913875603 0 0) 0x7f28440c3900 con 0x7f2844070c50 2023-11-09T03:45:56.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.816+0000 7f28427fc640 1 --2- 172.21.15.76:0/3834746465 >> v2:172.21.15.113:3300/0 conn(0x7f28440ad730 0x7f2844070710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:56.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.817+0000 7f2842ffd640 1 --2- 172.21.15.76:0/3834746465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f284415ad30 0x7f28440adc80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f28440ce7f0 tx=0x7f283402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85157c2876c0f143 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:56.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.817+0000 7f2841ffb640 1 -- 172.21.15.76:0/3834746465 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283402cbf0 con 0x7f284415ad30 2023-11-09T03:45:56.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.817+0000 7f2841ffb640 1 -- 172.21.15.76:0/3834746465 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283402cd50 con 0x7f284415ad30 2023-11-09T03:45:56.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.817+0000 7f2841ffb640 1 -- 172.21.15.76:0/3834746465 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2834034660 con 0x7f284415ad30 2023-11-09T03:45:56.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.818+0000 7f28493a9640 1 -- 172.21.15.76:0/3834746465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f284415ad30 msgr2=0x7f28440adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:56.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.818+0000 7f28493a9640 1 --2- 172.21.15.76:0/3834746465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f284415ad30 0x7f28440adc80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f28440ce7f0 tx=0x7f283402be40 comp rx=0 tx=0).stop 2023-11-09T03:45:56.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.818+0000 7f28493a9640 1 -- 172.21.15.76:0/3834746465 shutdown_connections 2023-11-09T03:45:56.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.818+0000 7f28493a9640 1 --2- 172.21.15.76:0/3834746465 >> v2:172.21.15.113:3300/0 conn(0x7f28440ad730 0x7f2844070710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.818+0000 7f28493a9640 1 --2- 172.21.15.76:0/3834746465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f284415ad30 0x7f28440adc80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.818+0000 7f28493a9640 1 -- 172.21.15.76:0/3834746465 >> 172.21.15.76:0/3834746465 conn(0x7f28440ce950 msgr2=0x7f28440ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:56.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.818+0000 7f28493a9640 1 -- 172.21.15.76:0/3834746465 shutdown_connections 2023-11-09T03:45:56.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.818+0000 7f28493a9640 1 -- 172.21.15.76:0/3834746465 wait complete. 2023-11-09T03:45:56.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.819+0000 7f28493a9640 1 Processor -- start 2023-11-09T03:45:56.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.819+0000 7f28493a9640 1 -- start start 2023-11-09T03:45:56.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.820+0000 7f28493a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2844070c50 0x7f2844159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:56.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.820+0000 7f28493a9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28440ad730 0x7f28440c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:56.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.820+0000 7f28493a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f284415ad30 0x7f2844144150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:56.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.820+0000 7f28493a9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2844141e80 con 0x7f28440ad730 2023-11-09T03:45:56.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.820+0000 7f28493a9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284415c160 con 0x7f284415ad30 2023-11-09T03:45:56.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.820+0000 7f28493a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2844148520 con 0x7f2844070c50 2023-11-09T03:45:56.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.820+0000 7f28427fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28440ad730 0x7f28440c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:56.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.820+0000 7f28427fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28440ad730 0x7f28440c5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36534/0 (socket says 172.21.15.76:36534) 2023-11-09T03:45:56.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.820+0000 7f28427fc640 1 -- 172.21.15.76:0/4151842793 learned_addr learned my addr 172.21.15.76:0/4151842793 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:56.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.820+0000 7f28437fe640 1 --2- 172.21.15.76:0/4151842793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f284415ad30 0x7f2844144150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:56.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.820+0000 7f2842ffd640 1 --2- 172.21.15.76:0/4151842793 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2844070c50 0x7f2844159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:56.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.820+0000 7f28427fc640 1 -- 172.21.15.76:0/4151842793 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2844070c50 msgr2=0x7f2844159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:56.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.820+0000 7f28427fc640 1 --2- 172.21.15.76:0/4151842793 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2844070c50 0x7f2844159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.820+0000 7f28427fc640 1 -- 172.21.15.76:0/4151842793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f284415ad30 msgr2=0x7f2844144150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:56.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.820+0000 7f28427fc640 1 --2- 172.21.15.76:0/4151842793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f284415ad30 0x7f2844144150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.820+0000 7f28427fc640 1 -- 172.21.15.76:0/4151842793 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2834009a70 con 0x7f28440ad730 2023-11-09T03:45:56.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.820+0000 7f28437fe640 1 --2- 172.21.15.76:0/4151842793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f284415ad30 0x7f2844144150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:56.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.820+0000 7f28427fc640 1 --2- 172.21.15.76:0/4151842793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28440ad730 0x7f28440c5a20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f283800e810 tx=0x7f283800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:56.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.820+0000 7f282b7fe640 1 -- 172.21.15.76:0/4151842793 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2838004030 con 0x7f28440ad730 2023-11-09T03:45:56.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.820+0000 7f282b7fe640 1 -- 172.21.15.76:0/4151842793 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2838004190 con 0x7f28440ad730 2023-11-09T03:45:56.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.820+0000 7f282b7fe640 1 -- 172.21.15.76:0/4151842793 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2838004500 con 0x7f28440ad730 2023-11-09T03:45:56.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.820+0000 7f28493a9640 1 -- 172.21.15.76:0/4151842793 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28440c84d0 con 0x7f28440ad730 2023-11-09T03:45:56.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.821+0000 7f28493a9640 1 -- 172.21.15.76:0/4151842793 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28440c8a90 con 0x7f28440ad730 2023-11-09T03:45:56.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.822+0000 7f28493a9640 1 -- 172.21.15.76:0/4151842793 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2844074290 con 0x7f28440ad730 2023-11-09T03:45:56.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.822+0000 7f282b7fe640 1 -- 172.21.15.76:0/4151842793 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f283800c6d0 con 0x7f28440ad730 2023-11-09T03:45:56.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.822+0000 7f282b7fe640 1 --2- 172.21.15.76:0/4151842793 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f283006ae70 0x7f283006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:56.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.822+0000 7f282b7fe640 1 -- 172.21.15.76:0/4151842793 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2838089590 con 0x7f28440ad730 2023-11-09T03:45:56.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.823+0000 7f2842ffd640 1 --2- 172.21.15.76:0/4151842793 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f283006ae70 0x7f283006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:56.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.823+0000 7f2842ffd640 1 --2- 172.21.15.76:0/4151842793 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f283006ae70 0x7f283006d330 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2834036000 tx=0x7f2834030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:56.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.826+0000 7f282b7fe640 1 -- 172.21.15.76:0/4151842793 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2838054350 con 0x7f28440ad730 2023-11-09T03:45:56.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.992+0000 7f28493a9640 1 -- 172.21.15.76:0/4151842793 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2844071970 con 0x7f28440ad730 2023-11-09T03:45:56.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.992+0000 7f282b7fe640 1 -- 172.21.15.76:0/4151842793 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2838057970 con 0x7f28440ad730 2023-11-09T03:45:56.992 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:45:56.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.994+0000 7f28493a9640 1 -- 172.21.15.76:0/4151842793 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f283006ae70 msgr2=0x7f283006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:56.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.994+0000 7f28493a9640 1 --2- 172.21.15.76:0/4151842793 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f283006ae70 0x7f283006d330 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2834036000 tx=0x7f2834030000 comp rx=0 tx=0).stop 2023-11-09T03:45:56.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.994+0000 7f28493a9640 1 -- 172.21.15.76:0/4151842793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28440ad730 msgr2=0x7f28440c5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:56.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.994+0000 7f28493a9640 1 --2- 172.21.15.76:0/4151842793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28440ad730 0x7f28440c5a20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f283800e810 tx=0x7f283800ecd0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.994+0000 7f28493a9640 1 -- 172.21.15.76:0/4151842793 shutdown_connections 2023-11-09T03:45:56.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.994+0000 7f28493a9640 1 --2- 172.21.15.76:0/4151842793 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f283006ae70 0x7f283006d330 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.994+0000 7f28493a9640 1 --2- 172.21.15.76:0/4151842793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f284415ad30 0x7f2844144150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.994+0000 7f28493a9640 1 --2- 172.21.15.76:0/4151842793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28440ad730 0x7f28440c5a20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.994+0000 7f28493a9640 1 --2- 172.21.15.76:0/4151842793 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2844070c50 0x7f2844159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:56.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.994+0000 7f28493a9640 1 -- 172.21.15.76:0/4151842793 >> 172.21.15.76:0/4151842793 conn(0x7f28440ce950 msgr2=0x7f2844155340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:56.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.994+0000 7f28493a9640 1 -- 172.21.15.76:0/4151842793 shutdown_connections 2023-11-09T03:45:56.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:56.994+0000 7f28493a9640 1 -- 172.21.15.76:0/4151842793 wait complete. 2023-11-09T03:45:57.004 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:45:57.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.107+0000 7f5678cc9640 1 Processor -- start 2023-11-09T03:45:57.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.107+0000 7f5678cc9640 1 -- start start 2023-11-09T03:45:57.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.108+0000 7f5678cc9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f567415ad30 0x7f56740adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:57.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.108+0000 7f5678cc9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56740c3900 con 0x7f5674070e20 2023-11-09T03:45:57.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.108+0000 7f5678cc9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56740c3be0 con 0x7f56740ad730 2023-11-09T03:45:57.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.108+0000 7f5678cc9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56740ae260 con 0x7f567415ad30 2023-11-09T03:45:57.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.108+0000 7f5672d9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f5674070e20 0x7f567415e740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33710/0 (socket says 172.21.15.76:33710) 2023-11-09T03:45:57.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.108+0000 7f5672d9d640 1 -- 172.21.15.76:0/256980452 learned_addr learned my addr 172.21.15.76:0/256980452 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:57.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.108+0000 7f567259c640 1 --2- 172.21.15.76:0/256980452 >> v2:172.21.15.113:3300/0 conn(0x7f567415ad30 0x7f56740adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:57.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.108+0000 7f5671d9b640 1 -- 172.21.15.76:0/256980452 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3516905507 0 0) 0x7f56740c3900 con 0x7f5674070e20 2023-11-09T03:45:57.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.108+0000 7f5671d9b640 1 -- 172.21.15.76:0/256980452 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5660003520 con 0x7f5674070e20 2023-11-09T03:45:57.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.108+0000 7f5671d9b640 1 -- 172.21.15.76:0/256980452 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1917579720 0 0) 0x7f5660003520 con 0x7f5674070e20 2023-11-09T03:45:57.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.108+0000 7f5671d9b640 1 -- 172.21.15.76:0/256980452 >> v2:172.21.15.113:3300/0 conn(0x7f567415ad30 msgr2=0x7f56740adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:57.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.108+0000 7f5671d9b640 1 --2- 172.21.15.76:0/256980452 >> v2:172.21.15.113:3300/0 conn(0x7f567415ad30 0x7f56740adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.108+0000 7f5671d9b640 1 -- 172.21.15.76:0/256980452 >> v1:172.21.15.113:6789/0 conn(0x7f56740ad730 legacy=0x7f5674070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:57.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.108+0000 7f5671d9b640 1 -- 172.21.15.76:0/256980452 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56740c3ec0 con 0x7f5674070e20 2023-11-09T03:45:57.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.108+0000 7f5671d9b640 1 -- 172.21.15.76:0/256980452 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f566c0019f0 con 0x7f5674070e20 2023-11-09T03:45:57.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.108+0000 7f5671d9b640 1 -- 172.21.15.76:0/256980452 >> v1:172.21.15.76:6789/0 conn(0x7f5674070e20 legacy=0x7f567415e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:57.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.108+0000 7f5671d9b640 1 --2- 172.21.15.76:0/256980452 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5660003970 0x7f5660003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:57.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.108+0000 7f5671d9b640 1 --2- 172.21.15.76:0/256980452 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56600047c0 0x7f5660004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:57.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.108+0000 7f5671d9b640 1 --2- 172.21.15.76:0/256980452 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5660005180 0x7f566000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:57.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.108+0000 7f5671d9b640 1 -- 172.21.15.76:0/256980452 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56740ae260 con 0x7f56600047c0 2023-11-09T03:45:57.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.108+0000 7f5671d9b640 1 -- 172.21.15.76:0/256980452 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566000c120 con 0x7f5660003970 2023-11-09T03:45:57.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.108+0000 7f5671d9b640 1 -- 172.21.15.76:0/256980452 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f566000c2e0 con 0x7f5660005180 2023-11-09T03:45:57.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.109+0000 7f5669d9b640 1 --2- 172.21.15.76:0/256980452 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56600047c0 0x7f5660004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:57.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.109+0000 7f567259c640 1 --2- 172.21.15.76:0/256980452 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5660003970 0x7f5660003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:57.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.109+0000 7f5672d9d640 1 --2- 172.21.15.76:0/256980452 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5660005180 0x7f566000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:57.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.109+0000 7f5669d9b640 1 -- 172.21.15.76:0/256980452 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5660005180 msgr2=0x7f566000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:57.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.109+0000 7f5669d9b640 1 --2- 172.21.15.76:0/256980452 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5660005180 0x7f566000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.109+0000 7f5669d9b640 1 -- 172.21.15.76:0/256980452 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5660003970 msgr2=0x7f5660003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:57.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.109+0000 7f5669d9b640 1 --2- 172.21.15.76:0/256980452 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5660003970 0x7f5660003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.109+0000 7f5669d9b640 1 -- 172.21.15.76:0/256980452 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5660003520 con 0x7f56600047c0 2023-11-09T03:45:57.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.109+0000 7f5672d9d640 1 -- 172.21.15.76:0/256980452 reap_dead start 2023-11-09T03:45:57.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.109+0000 7f5669d9b640 1 --2- 172.21.15.76:0/256980452 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56600047c0 0x7f5660004c40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f565c001750 tx=0x7f56600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e68d36f6e80942b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:57.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.110+0000 7f5671d9b640 1 -- 172.21.15.76:0/256980452 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c0052d0 con 0x7f56600047c0 2023-11-09T03:45:57.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.110+0000 7f5671d9b640 1 -- 172.21.15.76:0/256980452 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565c02e050 con 0x7f56600047c0 2023-11-09T03:45:57.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.110+0000 7f5671d9b640 1 -- 172.21.15.76:0/256980452 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c003830 con 0x7f56600047c0 2023-11-09T03:45:57.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.110+0000 7f5678cc9640 1 -- 172.21.15.76:0/256980452 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56600047c0 msgr2=0x7f5660004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:57.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.110+0000 7f5678cc9640 1 --2- 172.21.15.76:0/256980452 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56600047c0 0x7f5660004c40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f565c001750 tx=0x7f56600057f0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.110+0000 7f5678cc9640 1 -- 172.21.15.76:0/256980452 shutdown_connections 2023-11-09T03:45:57.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.110+0000 7f5678cc9640 1 --2- 172.21.15.76:0/256980452 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56600047c0 0x7f5660004c40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.110+0000 7f5678cc9640 1 -- 172.21.15.76:0/256980452 >> 172.21.15.76:0/256980452 conn(0x7f56740ce950 msgr2=0x7f56740ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:57.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.110+0000 7f5678cc9640 1 -- 172.21.15.76:0/256980452 shutdown_connections 2023-11-09T03:45:57.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.110+0000 7f5678cc9640 1 -- 172.21.15.76:0/256980452 wait complete. 2023-11-09T03:45:57.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.112+0000 7f5678cc9640 1 Processor -- start 2023-11-09T03:45:57.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.112+0000 7f5678cc9640 1 -- start start 2023-11-09T03:45:57.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.112+0000 7f5678cc9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56600047c0 0x7f5674143800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:57.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.112+0000 7f5678cc9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5674156b90 0x7f567414f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:57.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.112+0000 7f5678cc9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5674157010 0x7f567414fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:57.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.112+0000 7f5678cc9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5674157e60 con 0x7f5674157010 2023-11-09T03:45:57.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.112+0000 7f5678cc9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5674158240 con 0x7f5674156b90 2023-11-09T03:45:57.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.112+0000 7f5678cc9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5674143d40 con 0x7f56600047c0 2023-11-09T03:45:57.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.112+0000 7f5672d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5674157010 0x7f567414fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:57.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.113+0000 7f5672d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5674157010 0x7f567414fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36544/0 (socket says 172.21.15.76:36544) 2023-11-09T03:45:57.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.113+0000 7f5672d9d640 1 -- 172.21.15.76:0/3039625842 learned_addr learned my addr 172.21.15.76:0/3039625842 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:57.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.113+0000 7f5669d9b640 1 --2- 172.21.15.76:0/3039625842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5674156b90 0x7f567414f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:57.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.113+0000 7f567259c640 1 --2- 172.21.15.76:0/3039625842 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56600047c0 0x7f5674143800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:57.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.113+0000 7f5672d9d640 1 -- 172.21.15.76:0/3039625842 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56600047c0 msgr2=0x7f5674143800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:57.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.113+0000 7f5672d9d640 1 --2- 172.21.15.76:0/3039625842 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56600047c0 0x7f5674143800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.113+0000 7f5672d9d640 1 -- 172.21.15.76:0/3039625842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5674156b90 msgr2=0x7f567414f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:57.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.113+0000 7f5672d9d640 1 --2- 172.21.15.76:0/3039625842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5674156b90 0x7f567414f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.113+0000 7f5672d9d640 1 -- 172.21.15.76:0/3039625842 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56741575c0 con 0x7f5674157010 2023-11-09T03:45:57.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.113+0000 7f567259c640 1 --2- 172.21.15.76:0/3039625842 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56600047c0 0x7f5674143800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:57.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.113+0000 7f5672d9d640 1 --2- 172.21.15.76:0/3039625842 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5674157010 0x7f567414fce0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f56740ce890 tx=0x7f566c007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:57.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.113+0000 7f566b7fe640 1 -- 172.21.15.76:0/3039625842 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c00abd0 con 0x7f5674157010 2023-11-09T03:45:57.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.113+0000 7f566b7fe640 1 -- 172.21.15.76:0/3039625842 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566c00ad30 con 0x7f5674157010 2023-11-09T03:45:57.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.113+0000 7f566b7fe640 1 -- 172.21.15.76:0/3039625842 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c00aea0 con 0x7f5674157010 2023-11-09T03:45:57.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.113+0000 7f5678cc9640 1 -- 172.21.15.76:0/3039625842 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56741578a0 con 0x7f5674157010 2023-11-09T03:45:57.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.113+0000 7f5678cc9640 1 -- 172.21.15.76:0/3039625842 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5674140cc0 con 0x7f5674157010 2023-11-09T03:45:57.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.114+0000 7f5678cc9640 1 -- 172.21.15.76:0/3039625842 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5674157ff0 con 0x7f5674157010 2023-11-09T03:45:57.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.114+0000 7f566b7fe640 1 -- 172.21.15.76:0/3039625842 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f566c013610 con 0x7f5674157010 2023-11-09T03:45:57.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.115+0000 7f566b7fe640 1 --2- 172.21.15.76:0/3039625842 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f565406ae70 0x7f565406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:57.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.115+0000 7f566b7fe640 1 -- 172.21.15.76:0/3039625842 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f566c089ee0 con 0x7f5674157010 2023-11-09T03:45:57.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.115+0000 7f567259c640 1 --2- 172.21.15.76:0/3039625842 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f565406ae70 0x7f565406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:57.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.115+0000 7f567259c640 1 --2- 172.21.15.76:0/3039625842 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f565406ae70 0x7f565406d330 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f567406d6d0 tx=0x7f5664008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:57.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.119+0000 7f566b7fe640 1 -- 172.21.15.76:0/3039625842 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f566c054ca0 con 0x7f5674157010 2023-11-09T03:45:57.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.284+0000 7f5678cc9640 1 -- 172.21.15.76:0/3039625842 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5674072620 con 0x7f5674157010 2023-11-09T03:45:57.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.285+0000 7f566b7fe640 1 -- 172.21.15.76:0/3039625842 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f566c0582c0 con 0x7f5674157010 2023-11-09T03:45:57.284 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:45:57.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.287+0000 7f5678cc9640 1 -- 172.21.15.76:0/3039625842 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f565406ae70 msgr2=0x7f565406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:57.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.287+0000 7f5678cc9640 1 --2- 172.21.15.76:0/3039625842 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f565406ae70 0x7f565406d330 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f567406d6d0 tx=0x7f5664008040 comp rx=0 tx=0).stop 2023-11-09T03:45:57.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.287+0000 7f5678cc9640 1 -- 172.21.15.76:0/3039625842 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5674157010 msgr2=0x7f567414fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:57.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.287+0000 7f5678cc9640 1 --2- 172.21.15.76:0/3039625842 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5674157010 0x7f567414fce0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f56740ce890 tx=0x7f566c007bd0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.287+0000 7f5678cc9640 1 -- 172.21.15.76:0/3039625842 shutdown_connections 2023-11-09T03:45:57.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.287+0000 7f5678cc9640 1 --2- 172.21.15.76:0/3039625842 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f565406ae70 0x7f565406d330 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.287+0000 7f5678cc9640 1 --2- 172.21.15.76:0/3039625842 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5674157010 0x7f567414fce0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.287+0000 7f5678cc9640 1 --2- 172.21.15.76:0/3039625842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5674156b90 0x7f567414f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.287+0000 7f5678cc9640 1 --2- 172.21.15.76:0/3039625842 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56600047c0 0x7f5674143800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.287+0000 7f5678cc9640 1 -- 172.21.15.76:0/3039625842 >> 172.21.15.76:0/3039625842 conn(0x7f56740ce950 msgr2=0x7f567406dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:57.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.287+0000 7f5678cc9640 1 -- 172.21.15.76:0/3039625842 shutdown_connections 2023-11-09T03:45:57.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.287+0000 7f5678cc9640 1 -- 172.21.15.76:0/3039625842 wait complete. 2023-11-09T03:45:57.297 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-09T03:45:57.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.400+0000 7f4493941640 1 Processor -- start 2023-11-09T03:45:57.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.400+0000 7f4493941640 1 -- start start 2023-11-09T03:45:57.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.401+0000 7f4493941640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f448c0c4ff0 0x7f448c0cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:57.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.401+0000 7f4493941640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f448c0adac0 0x7f448c0adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:57.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.401+0000 7f4493941640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f448c15aff0 con 0x7f448c159c00 2023-11-09T03:45:57.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.401+0000 7f4493941640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f448c0ae450 con 0x7f448c0adac0 2023-11-09T03:45:57.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.401+0000 7f4493941640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f448c15a3e0 con 0x7f448c0c4ff0 2023-11-09T03:45:57.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.401+0000 7f44916dd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f448c159c00 0x7f448c0c48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33712/0 (socket says 172.21.15.76:33712) 2023-11-09T03:45:57.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.401+0000 7f44916dd640 1 -- 172.21.15.76:0/125623823 learned_addr learned my addr 172.21.15.76:0/125623823 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:57.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.401+0000 7f4490edc640 1 --2- 172.21.15.76:0/125623823 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f448c0c4ff0 0x7f448c0cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:57.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.401+0000 7f4491ede640 1 --2- 172.21.15.76:0/125623823 >> v2:172.21.15.113:3300/0 conn(0x7f448c0adac0 0x7f448c0adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:57.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.401+0000 7f4483fff640 1 -- 172.21.15.76:0/125623823 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2959516839 0 0) 0x7f448c15aff0 con 0x7f448c159c00 2023-11-09T03:45:57.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.401+0000 7f4483fff640 1 -- 172.21.15.76:0/125623823 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4478003520 con 0x7f448c159c00 2023-11-09T03:45:57.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.401+0000 7f4483fff640 1 -- 172.21.15.76:0/125623823 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 13960206 0 0) 0x7f4478003520 con 0x7f448c159c00 2023-11-09T03:45:57.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.401+0000 7f4483fff640 1 -- 172.21.15.76:0/125623823 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f448c0c4ff0 msgr2=0x7f448c0cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:57.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.401+0000 7f4483fff640 1 --2- 172.21.15.76:0/125623823 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f448c0c4ff0 0x7f448c0cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.401+0000 7f4483fff640 1 -- 172.21.15.76:0/125623823 >> v2:172.21.15.113:3300/0 conn(0x7f448c0adac0 msgr2=0x7f448c0adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:57.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.401+0000 7f4483fff640 1 --2- 172.21.15.76:0/125623823 >> v2:172.21.15.113:3300/0 conn(0x7f448c0adac0 0x7f448c0adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.401+0000 7f4483fff640 1 -- 172.21.15.76:0/125623823 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448c15a6c0 con 0x7f448c159c00 2023-11-09T03:45:57.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.401+0000 7f4483fff640 1 -- 172.21.15.76:0/125623823 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f4488001670 con 0x7f448c159c00 2023-11-09T03:45:57.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.401+0000 7f4483fff640 1 -- 172.21.15.76:0/125623823 >> v1:172.21.15.76:6789/0 conn(0x7f448c159c00 legacy=0x7f448c0c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:57.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.401+0000 7f4483fff640 1 --2- 172.21.15.76:0/125623823 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f448c0c4ff0 0x7f4478003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:57.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.401+0000 7f4483fff640 1 --2- 172.21.15.76:0/125623823 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4478003ef0 0x7f44780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:57.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.401+0000 7f4483fff640 1 --2- 172.21.15.76:0/125623823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4478004d00 0x7f4478005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:57.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.401+0000 7f4483fff640 1 -- 172.21.15.76:0/125623823 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c0ae450 con 0x7f4478003ef0 2023-11-09T03:45:57.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.401+0000 7f4483fff640 1 -- 172.21.15.76:0/125623823 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c15a3e0 con 0x7f4478004d00 2023-11-09T03:45:57.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.401+0000 7f4483fff640 1 -- 172.21.15.76:0/125623823 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447800b980 con 0x7f448c0c4ff0 2023-11-09T03:45:57.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.401+0000 7f4490edc640 1 --2- 172.21.15.76:0/125623823 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4478003ef0 0x7f44780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:57.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.402+0000 7f44916dd640 1 --2- 172.21.15.76:0/125623823 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f448c0c4ff0 0x7f4478003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:57.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.402+0000 7f4491ede640 1 --2- 172.21.15.76:0/125623823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4478004d00 0x7f4478005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:57.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.402+0000 7f4490edc640 1 -- 172.21.15.76:0/125623823 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f448c0c4ff0 msgr2=0x7f4478003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:57.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.402+0000 7f4490edc640 1 --2- 172.21.15.76:0/125623823 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f448c0c4ff0 0x7f4478003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.402+0000 7f4490edc640 1 -- 172.21.15.76:0/125623823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4478004d00 msgr2=0x7f4478005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:57.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.402+0000 7f4490edc640 1 --2- 172.21.15.76:0/125623823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4478004d00 0x7f4478005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.402+0000 7f4490edc640 1 -- 172.21.15.76:0/125623823 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4478003520 con 0x7f4478003ef0 2023-11-09T03:45:57.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.402+0000 7f4490edc640 1 --2- 172.21.15.76:0/125623823 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4478003ef0 0x7f44780047c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f447c009650 tx=0x7f44780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0cc278b9342d51 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:57.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.403+0000 7f4483fff640 1 -- 172.21.15.76:0/125623823 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c02e8c0 con 0x7f4478003ef0 2023-11-09T03:45:57.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.403+0000 7f4483fff640 1 -- 172.21.15.76:0/125623823 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447c02ea20 con 0x7f4478003ef0 2023-11-09T03:45:57.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.403+0000 7f4483fff640 1 -- 172.21.15.76:0/125623823 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c02edd0 con 0x7f4478003ef0 2023-11-09T03:45:57.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.403+0000 7f4493941640 1 -- 172.21.15.76:0/125623823 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4478003ef0 msgr2=0x7f44780047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:57.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.403+0000 7f4493941640 1 --2- 172.21.15.76:0/125623823 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4478003ef0 0x7f44780047c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f447c009650 tx=0x7f44780057f0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.403+0000 7f4491ede640 1 -- 172.21.15.76:0/125623823 reap_dead start 2023-11-09T03:45:57.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.403+0000 7f4493941640 1 -- 172.21.15.76:0/125623823 shutdown_connections 2023-11-09T03:45:57.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.403+0000 7f4493941640 1 -- 172.21.15.76:0/125623823 >> 172.21.15.76:0/125623823 conn(0x7f448c0ce950 msgr2=0x7f448c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:57.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.403+0000 7f4493941640 1 -- 172.21.15.76:0/125623823 shutdown_connections 2023-11-09T03:45:57.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.403+0000 7f4493941640 1 -- 172.21.15.76:0/125623823 wait complete. 2023-11-09T03:45:57.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.404+0000 7f4493941640 1 Processor -- start 2023-11-09T03:45:57.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.404+0000 7f4493941640 1 -- start start 2023-11-09T03:45:57.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.405+0000 7f4493941640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4478003ef0 0x7f448c153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:57.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.405+0000 7f4493941640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f448c1464d0 0x7f448c1dfce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:57.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.405+0000 7f4493941640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f448c0cb340 0x7f448c0cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:57.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.405+0000 7f4493941640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c1e1190 con 0x7f4478003ef0 2023-11-09T03:45:57.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.405+0000 7f4493941640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c0cbd00 con 0x7f448c0cb340 2023-11-09T03:45:57.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.405+0000 7f4493941640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f448c1e15b0 con 0x7f448c1464d0 2023-11-09T03:45:57.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.405+0000 7f44916dd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4478003ef0 0x7f448c153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:57.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.405+0000 7f44916dd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4478003ef0 0x7f448c153560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36562/0 (socket says 172.21.15.76:36562) 2023-11-09T03:45:57.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.405+0000 7f44916dd640 1 -- 172.21.15.76:0/1497297769 learned_addr learned my addr 172.21.15.76:0/1497297769 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:57.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.405+0000 7f4491ede640 1 --2- 172.21.15.76:0/1497297769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f448c0cb340 0x7f448c0cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:57.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.405+0000 7f4490edc640 1 --2- 172.21.15.76:0/1497297769 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f448c1464d0 0x7f448c1dfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:57.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.405+0000 7f44916dd640 1 -- 172.21.15.76:0/1497297769 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f448c1464d0 msgr2=0x7f448c1dfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:57.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.405+0000 7f44916dd640 1 --2- 172.21.15.76:0/1497297769 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f448c1464d0 0x7f448c1dfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.405+0000 7f44916dd640 1 -- 172.21.15.76:0/1497297769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f448c0cb340 msgr2=0x7f448c0cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:57.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.405+0000 7f44916dd640 1 --2- 172.21.15.76:0/1497297769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f448c0cb340 0x7f448c0cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.405+0000 7f44916dd640 1 -- 172.21.15.76:0/1497297769 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448c1e0510 con 0x7f4478003ef0 2023-11-09T03:45:57.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.406+0000 7f44916dd640 1 --2- 172.21.15.76:0/1497297769 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4478003ef0 0x7f448c153560 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f44880072c0 tx=0x7f4488007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:57.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.406+0000 7f4481ffb640 1 -- 172.21.15.76:0/1497297769 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448800f9c0 con 0x7f4478003ef0 2023-11-09T03:45:57.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.406+0000 7f4481ffb640 1 -- 172.21.15.76:0/1497297769 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448800fb20 con 0x7f4478003ef0 2023-11-09T03:45:57.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.406+0000 7f4481ffb640 1 -- 172.21.15.76:0/1497297769 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448800fed0 con 0x7f4478003ef0 2023-11-09T03:45:57.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.406+0000 7f4493941640 1 -- 172.21.15.76:0/1497297769 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f448c1e07f0 con 0x7f4478003ef0 2023-11-09T03:45:57.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.406+0000 7f4493941640 1 -- 172.21.15.76:0/1497297769 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f448c1e0db0 con 0x7f4478003ef0 2023-11-09T03:45:57.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.407+0000 7f4493941640 1 -- 172.21.15.76:0/1497297769 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f448c1e0f70 con 0x7f4478003ef0 2023-11-09T03:45:57.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.407+0000 7f4481ffb640 1 -- 172.21.15.76:0/1497297769 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f448801d460 con 0x7f4478003ef0 2023-11-09T03:45:57.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.407+0000 7f4481ffb640 1 --2- 172.21.15.76:0/1497297769 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f447006aea0 0x7f447006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:57.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.408+0000 7f4481ffb640 1 -- 172.21.15.76:0/1497297769 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4488089b90 con 0x7f4478003ef0 2023-11-09T03:45:57.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.408+0000 7f4490edc640 1 --2- 172.21.15.76:0/1497297769 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f447006aea0 0x7f447006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:57.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.408+0000 7f4490edc640 1 --2- 172.21.15.76:0/1497297769 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f447006aea0 0x7f447006d360 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f447c038000 tx=0x7f447c032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:57.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.412+0000 7f4481ffb640 1 -- 172.21.15.76:0/1497297769 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4488054950 con 0x7f4478003ef0 2023-11-09T03:45:57.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.579+0000 7f4493941640 1 -- 172.21.15.76:0/1497297769 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f448c0adf10 con 0x7f4478003ef0 2023-11-09T03:45:57.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.579+0000 7f4481ffb640 1 -- 172.21.15.76:0/1497297769 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4488057f70 con 0x7f4478003ef0 2023-11-09T03:45:57.579 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:45:57.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.581+0000 7f4493941640 1 -- 172.21.15.76:0/1497297769 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f447006aea0 msgr2=0x7f447006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:57.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.581+0000 7f4493941640 1 --2- 172.21.15.76:0/1497297769 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f447006aea0 0x7f447006d360 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f447c038000 tx=0x7f447c032000 comp rx=0 tx=0).stop 2023-11-09T03:45:57.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.581+0000 7f4493941640 1 -- 172.21.15.76:0/1497297769 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4478003ef0 msgr2=0x7f448c153560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:57.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.581+0000 7f4493941640 1 --2- 172.21.15.76:0/1497297769 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4478003ef0 0x7f448c153560 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f44880072c0 tx=0x7f4488007780 comp rx=0 tx=0).stop 2023-11-09T03:45:57.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.581+0000 7f4493941640 1 -- 172.21.15.76:0/1497297769 shutdown_connections 2023-11-09T03:45:57.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.581+0000 7f4493941640 1 --2- 172.21.15.76:0/1497297769 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f447006aea0 0x7f447006d360 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.581+0000 7f4493941640 1 --2- 172.21.15.76:0/1497297769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f448c0cb340 0x7f448c0cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.581+0000 7f4493941640 1 --2- 172.21.15.76:0/1497297769 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f448c1464d0 0x7f448c1dfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.581+0000 7f4493941640 1 --2- 172.21.15.76:0/1497297769 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4478003ef0 0x7f448c153560 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.581+0000 7f4493941640 1 -- 172.21.15.76:0/1497297769 >> 172.21.15.76:0/1497297769 conn(0x7f448c0ce950 msgr2=0x7f448c06d250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:57.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.581+0000 7f4493941640 1 -- 172.21.15.76:0/1497297769 shutdown_connections 2023-11-09T03:45:57.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.581+0000 7f4493941640 1 -- 172.21.15.76:0/1497297769 wait complete. 2023-11-09T03:45:57.591 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:45:57.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.692+0000 7f627e1ee640 1 Processor -- start 2023-11-09T03:45:57.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.692+0000 7f627e1ee640 1 -- start start 2023-11-09T03:45:57.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.693+0000 7f627e1ee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f627815ce70 0x7f627806d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:57.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.693+0000 7f627e1ee640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f627806a120 con 0x7f627815d250 2023-11-09T03:45:57.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.693+0000 7f627e1ee640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f627806def0 con 0x7f62780cb5c0 2023-11-09T03:45:57.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.693+0000 7f627e1ee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62780cdc10 con 0x7f627815ce70 2023-11-09T03:45:57.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.693+0000 7f6276ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f627815d250 0x7f62780c9830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33714/0 (socket says 172.21.15.76:33714) 2023-11-09T03:45:57.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.693+0000 7f6276ffd640 1 -- 172.21.15.76:0/1274372710 learned_addr learned my addr 172.21.15.76:0/1274372710 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:57.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.693+0000 7f62777fe640 1 --2- 172.21.15.76:0/1274372710 >> v2:172.21.15.113:3300/0 conn(0x7f627815ce70 0x7f627806d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:57.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.694+0000 7f62767fc640 1 -- 172.21.15.76:0/1274372710 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1462268196 0 0) 0x7f627806a120 con 0x7f627815d250 2023-11-09T03:45:57.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.694+0000 7f62767fc640 1 -- 172.21.15.76:0/1274372710 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6260003520 con 0x7f627815d250 2023-11-09T03:45:57.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.694+0000 7f62767fc640 1 -- 172.21.15.76:0/1274372710 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 539964729 0 0) 0x7f6260003520 con 0x7f627815d250 2023-11-09T03:45:57.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.694+0000 7f62767fc640 1 -- 172.21.15.76:0/1274372710 >> v2:172.21.15.113:3300/0 conn(0x7f627815ce70 msgr2=0x7f627806d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:57.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.694+0000 7f62767fc640 1 --2- 172.21.15.76:0/1274372710 >> v2:172.21.15.113:3300/0 conn(0x7f627815ce70 0x7f627806d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.694+0000 7f62767fc640 1 -- 172.21.15.76:0/1274372710 >> v1:172.21.15.113:6789/0 conn(0x7f62780cb5c0 legacy=0x7f62780ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:57.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.694+0000 7f62767fc640 1 -- 172.21.15.76:0/1274372710 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62780c9f40 con 0x7f627815d250 2023-11-09T03:45:57.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.694+0000 7f62767fc640 1 -- 172.21.15.76:0/1274372710 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f626c001670 con 0x7f627815d250 2023-11-09T03:45:57.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.694+0000 7f62767fc640 1 -- 172.21.15.76:0/1274372710 >> v1:172.21.15.76:6789/0 conn(0x7f627815d250 legacy=0x7f62780c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:57.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.694+0000 7f62767fc640 1 --2- 172.21.15.76:0/1274372710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6260003970 0x7f6260003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:57.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.694+0000 7f62767fc640 1 --2- 172.21.15.76:0/1274372710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62600047c0 0x7f6260004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:57.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.694+0000 7f62767fc640 1 --2- 172.21.15.76:0/1274372710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6260005180 0x7f626000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:57.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.694+0000 7f62767fc640 1 -- 172.21.15.76:0/1274372710 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62780cdc10 con 0x7f6260003970 2023-11-09T03:45:57.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.694+0000 7f62767fc640 1 -- 172.21.15.76:0/1274372710 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626000c020 con 0x7f62600047c0 2023-11-09T03:45:57.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.694+0000 7f62767fc640 1 -- 172.21.15.76:0/1274372710 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f626000c1e0 con 0x7f6260005180 2023-11-09T03:45:57.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.694+0000 7f62777fe640 1 --2- 172.21.15.76:0/1274372710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6260003970 0x7f6260003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:57.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.694+0000 7f6276ffd640 1 --2- 172.21.15.76:0/1274372710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62600047c0 0x7f6260004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:57.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.694+0000 7f6277fff640 1 --2- 172.21.15.76:0/1274372710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6260005180 0x7f626000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:57.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.694+0000 7f62777fe640 1 -- 172.21.15.76:0/1274372710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6260005180 msgr2=0x7f626000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:57.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.694+0000 7f62777fe640 1 --2- 172.21.15.76:0/1274372710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6260005180 0x7f626000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.694+0000 7f62777fe640 1 -- 172.21.15.76:0/1274372710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62600047c0 msgr2=0x7f6260004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:57.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.694+0000 7f62777fe640 1 --2- 172.21.15.76:0/1274372710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62600047c0 0x7f6260004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.694+0000 7f62777fe640 1 -- 172.21.15.76:0/1274372710 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6260003520 con 0x7f6260003970 2023-11-09T03:45:57.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.694+0000 7f6277fff640 1 -- 172.21.15.76:0/1274372710 reap_dead start 2023-11-09T03:45:57.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.695+0000 7f62777fe640 1 --2- 172.21.15.76:0/1274372710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6260003970 0x7f6260003d90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6268009ac0 tx=0x7f62600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9935c6b6cecddf server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:57.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.695+0000 7f62767fc640 1 -- 172.21.15.76:0/1274372710 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626802c6b0 con 0x7f6260003970 2023-11-09T03:45:57.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.695+0000 7f62767fc640 1 -- 172.21.15.76:0/1274372710 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f626802c810 con 0x7f6260003970 2023-11-09T03:45:57.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.695+0000 7f62767fc640 1 -- 172.21.15.76:0/1274372710 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626802cba0 con 0x7f6260003970 2023-11-09T03:45:57.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.696+0000 7f627e1ee640 1 -- 172.21.15.76:0/1274372710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6260003970 msgr2=0x7f6260003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:57.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.696+0000 7f627e1ee640 1 --2- 172.21.15.76:0/1274372710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6260003970 0x7f6260003d90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6268009ac0 tx=0x7f62600057f0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.696+0000 7f627e1ee640 1 -- 172.21.15.76:0/1274372710 shutdown_connections 2023-11-09T03:45:57.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.696+0000 7f627e1ee640 1 --2- 172.21.15.76:0/1274372710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6260003970 0x7f6260003d90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.696+0000 7f627e1ee640 1 -- 172.21.15.76:0/1274372710 >> 172.21.15.76:0/1274372710 conn(0x7f62780b0cd0 msgr2=0x7f62780b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:57.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.696+0000 7f627e1ee640 1 -- 172.21.15.76:0/1274372710 shutdown_connections 2023-11-09T03:45:57.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.696+0000 7f627e1ee640 1 -- 172.21.15.76:0/1274372710 wait complete. 2023-11-09T03:45:57.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.697+0000 7f627e1ee640 1 Processor -- start 2023-11-09T03:45:57.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.697+0000 7f627e1ee640 1 -- start start 2023-11-09T03:45:57.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.698+0000 7f627e1ee640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6260003970 0x7f62781440e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:57.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.698+0000 7f627e1ee640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6278142780 0x7f62780ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:57.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.698+0000 7f627e1ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62781565e0 0x7f6278156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:57.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.698+0000 7f627e1ee640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6278144620 con 0x7f6260003970 2023-11-09T03:45:57.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.698+0000 7f627e1ee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6278143040 con 0x7f62781565e0 2023-11-09T03:45:57.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.698+0000 7f627e1ee640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6278143320 con 0x7f6278142780 2023-11-09T03:45:57.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.698+0000 7f62777fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6260003970 0x7f62781440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:57.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.698+0000 7f62777fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6260003970 0x7f62781440e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36580/0 (socket says 172.21.15.76:36580) 2023-11-09T03:45:57.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.698+0000 7f62777fe640 1 -- 172.21.15.76:0/4213524837 learned_addr learned my addr 172.21.15.76:0/4213524837 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:57.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.698+0000 7f6277fff640 1 --2- 172.21.15.76:0/4213524837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62781565e0 0x7f6278156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:57.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.698+0000 7f6276ffd640 1 --2- 172.21.15.76:0/4213524837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6278142780 0x7f62780ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:57.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.698+0000 7f62777fe640 1 -- 172.21.15.76:0/4213524837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6278142780 msgr2=0x7f62780ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:57.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.698+0000 7f62777fe640 1 --2- 172.21.15.76:0/4213524837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6278142780 0x7f62780ad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.698+0000 7f62777fe640 1 -- 172.21.15.76:0/4213524837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62781565e0 msgr2=0x7f6278156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:57.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.698+0000 7f62777fe640 1 --2- 172.21.15.76:0/4213524837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62781565e0 0x7f6278156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.698+0000 7f62777fe640 1 -- 172.21.15.76:0/4213524837 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6278143600 con 0x7f6260003970 2023-11-09T03:45:57.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.698+0000 7f6277fff640 1 --2- 172.21.15.76:0/4213524837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62781565e0 0x7f6278156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:57.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.698+0000 7f6276ffd640 1 --2- 172.21.15.76:0/4213524837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6278142780 0x7f62780ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:57.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.698+0000 7f62777fe640 1 --2- 172.21.15.76:0/4213524837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6260003970 0x7f62781440e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6268009ac0 tx=0x7f6268004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:57.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.698+0000 7f625ffff640 1 -- 172.21.15.76:0/4213524837 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6268003810 con 0x7f6260003970 2023-11-09T03:45:57.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.699+0000 7f625ffff640 1 -- 172.21.15.76:0/4213524837 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6268003970 con 0x7f6260003970 2023-11-09T03:45:57.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.699+0000 7f627e1ee640 1 -- 172.21.15.76:0/4213524837 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62781438e0 con 0x7f6260003970 2023-11-09T03:45:57.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.699+0000 7f625ffff640 1 -- 172.21.15.76:0/4213524837 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6268003d00 con 0x7f6260003970 2023-11-09T03:45:57.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.699+0000 7f627e1ee640 1 -- 172.21.15.76:0/4213524837 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62780ad670 con 0x7f6260003970 2023-11-09T03:45:57.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.699+0000 7f627e1ee640 1 -- 172.21.15.76:0/4213524837 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62780ad800 con 0x7f6260003970 2023-11-09T03:45:57.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.700+0000 7f625ffff640 1 -- 172.21.15.76:0/4213524837 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6268040450 con 0x7f6260003970 2023-11-09T03:45:57.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.700+0000 7f625ffff640 1 --2- 172.21.15.76:0/4213524837 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f626406aea0 0x7f626406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:57.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.700+0000 7f625ffff640 1 -- 172.21.15.76:0/4213524837 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f62680ab790 con 0x7f6260003970 2023-11-09T03:45:57.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.701+0000 7f6276ffd640 1 --2- 172.21.15.76:0/4213524837 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f626406aea0 0x7f626406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:57.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.701+0000 7f6276ffd640 1 --2- 172.21.15.76:0/4213524837 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f626406aea0 0x7f626406d360 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f626c001050 tx=0x7f626c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:57.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.705+0000 7f625ffff640 1 -- 172.21.15.76:0/4213524837 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6268076550 con 0x7f6260003970 2023-11-09T03:45:57.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.906+0000 7f627e1ee640 1 -- 172.21.15.76:0/4213524837 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6278074290 con 0x7f6260003970 2023-11-09T03:45:57.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.906+0000 7f625ffff640 1 -- 172.21.15.76:0/4213524837 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4915 (secure 0 0 0) 0x7f6268003e60 con 0x7f6260003970 2023-11-09T03:45:57.906 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:57.906 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2686298435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2686298435},{"type":"v1","addr":"172.21.15.76:6833","nonce":2686298435}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2132717123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2132717123},{"type":"v1","addr":"172.21.15.76:6835","nonce":2132717123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4281,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2369499936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2369499936},{"type":"v1","addr":"172.21.15.113:6837","nonce":2369499936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:45:30.784155+0000","modified":"2023-11-09T03:45:47.804242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/3293223710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3293223710},{"type":"v1","addr":"172.21.15.113:6836","nonce":3293223710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-09T03:45:57.907 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 9 2023-11-09T03:45:57.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.908+0000 7f627e1ee640 1 -- 172.21.15.76:0/4213524837 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f626406aea0 msgr2=0x7f626406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:57.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.908+0000 7f627e1ee640 1 --2- 172.21.15.76:0/4213524837 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f626406aea0 0x7f626406d360 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f626c001050 tx=0x7f626c009040 comp rx=0 tx=0).stop 2023-11-09T03:45:57.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.908+0000 7f627e1ee640 1 -- 172.21.15.76:0/4213524837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6260003970 msgr2=0x7f62781440e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:57.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.908+0000 7f627e1ee640 1 --2- 172.21.15.76:0/4213524837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6260003970 0x7f62781440e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6268009ac0 tx=0x7f6268004730 comp rx=0 tx=0).stop 2023-11-09T03:45:57.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.908+0000 7f627e1ee640 1 -- 172.21.15.76:0/4213524837 shutdown_connections 2023-11-09T03:45:57.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.908+0000 7f627e1ee640 1 --2- 172.21.15.76:0/4213524837 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f626406aea0 0x7f626406d360 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.908+0000 7f627e1ee640 1 --2- 172.21.15.76:0/4213524837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62781565e0 0x7f6278156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.908+0000 7f627e1ee640 1 --2- 172.21.15.76:0/4213524837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6278142780 0x7f62780ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.908+0000 7f627e1ee640 1 --2- 172.21.15.76:0/4213524837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6260003970 0x7f62781440e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:57.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.908+0000 7f627e1ee640 1 -- 172.21.15.76:0/4213524837 >> 172.21.15.76:0/4213524837 conn(0x7f62780b0cd0 msgr2=0x7f62780ca5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:57.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.908+0000 7f627e1ee640 1 -- 172.21.15.76:0/4213524837 shutdown_connections 2023-11-09T03:45:57.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:57.908+0000 7f627e1ee640 1 -- 172.21.15.76:0/4213524837 wait complete. 2023-11-09T03:45:57.919 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:45:58.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.020+0000 7f96ba73c640 1 Processor -- start 2023-11-09T03:45:58.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.020+0000 7f96ba73c640 1 -- start start 2023-11-09T03:45:58.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.020+0000 7f96ba73c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96b40cde60 0x7f96b415d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:58.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.020+0000 7f96ba73c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b415e000 con 0x7f96b415daa0 2023-11-09T03:45:58.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.020+0000 7f96ba73c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b415e2e0 con 0x7f96b40ce240 2023-11-09T03:45:58.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.020+0000 7f96ba73c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b415e5c0 con 0x7f96b40cde60 2023-11-09T03:45:58.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.021+0000 7f96b37fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f96b415daa0 0x7f96b4157000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33730/0 (socket says 172.21.15.76:33730) 2023-11-09T03:45:58.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.021+0000 7f96b37fe640 1 -- 172.21.15.76:0/2583640888 learned_addr learned my addr 172.21.15.76:0/2583640888 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:58.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.021+0000 7f96b3fff640 1 --2- 172.21.15.76:0/2583640888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96b40cde60 0x7f96b415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:58.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.021+0000 7f96b2ffd640 1 -- 172.21.15.76:0/2583640888 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3904853060 0 0) 0x7f96b415e000 con 0x7f96b415daa0 2023-11-09T03:45:58.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.021+0000 7f96b2ffd640 1 -- 172.21.15.76:0/2583640888 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96a0003520 con 0x7f96b415daa0 2023-11-09T03:45:58.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.021+0000 7f96b2ffd640 1 -- 172.21.15.76:0/2583640888 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2213260046 0 0) 0x7f96a0003520 con 0x7f96b415daa0 2023-11-09T03:45:58.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.021+0000 7f96b2ffd640 1 -- 172.21.15.76:0/2583640888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96b40cde60 msgr2=0x7f96b415d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:58.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.021+0000 7f96b2ffd640 1 --2- 172.21.15.76:0/2583640888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96b40cde60 0x7f96b415d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.021+0000 7f96b2ffd640 1 -- 172.21.15.76:0/2583640888 >> v1:172.21.15.113:6789/0 conn(0x7f96b40ce240 legacy=0x7f96b40d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:58.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.021+0000 7f96b2ffd640 1 -- 172.21.15.76:0/2583640888 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b40d1120 con 0x7f96b415daa0 2023-11-09T03:45:58.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.021+0000 7f96b2ffd640 1 -- 172.21.15.76:0/2583640888 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f96a4001670 con 0x7f96b415daa0 2023-11-09T03:45:58.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.021+0000 7f96b2ffd640 1 -- 172.21.15.76:0/2583640888 >> v1:172.21.15.76:6789/0 conn(0x7f96b415daa0 legacy=0x7f96b4157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:58.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.021+0000 7f96b2ffd640 1 --2- 172.21.15.76:0/2583640888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96a00039a0 0x7f96a0003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:58.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.021+0000 7f96b2ffd640 1 --2- 172.21.15.76:0/2583640888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96b40cde60 0x7f96a00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:58.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.021+0000 7f96b2ffd640 1 --2- 172.21.15.76:0/2583640888 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96a0004d00 0x7f96a0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:58.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.021+0000 7f96b2ffd640 1 -- 172.21.15.76:0/2583640888 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b415e5c0 con 0x7f96a0004d00 2023-11-09T03:45:58.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.021+0000 7f96b2ffd640 1 -- 172.21.15.76:0/2583640888 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a000b980 con 0x7f96a00039a0 2023-11-09T03:45:58.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.021+0000 7f96b2ffd640 1 -- 172.21.15.76:0/2583640888 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a000bb40 con 0x7f96b40cde60 2023-11-09T03:45:58.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.021+0000 7f96b8cd9640 1 --2- 172.21.15.76:0/2583640888 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96a0004d00 0x7f96a0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:58.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.021+0000 7f96b3fff640 1 --2- 172.21.15.76:0/2583640888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96a00039a0 0x7f96a0003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:58.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.021+0000 7f96b37fe640 1 --2- 172.21.15.76:0/2583640888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96b40cde60 0x7f96a00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:58.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.022+0000 7f96b8cd9640 1 -- 172.21.15.76:0/2583640888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96b40cde60 msgr2=0x7f96a00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:58.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.022+0000 7f96b8cd9640 1 --2- 172.21.15.76:0/2583640888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96b40cde60 0x7f96a00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.022+0000 7f96b8cd9640 1 -- 172.21.15.76:0/2583640888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96a00039a0 msgr2=0x7f96a0003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:58.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.022+0000 7f96b8cd9640 1 --2- 172.21.15.76:0/2583640888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96a00039a0 0x7f96a0003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.022+0000 7f96b8cd9640 1 -- 172.21.15.76:0/2583640888 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a0003520 con 0x7f96a0004d00 2023-11-09T03:45:58.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.022+0000 7f96b8cd9640 1 --2- 172.21.15.76:0/2583640888 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96a0004d00 0x7f96a0005180 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f96ac001f60 tx=0x7f96a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980749d82ce78992 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:58.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.022+0000 7f96b2ffd640 1 -- 172.21.15.76:0/2583640888 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac02d6f0 con 0x7f96a0004d00 2023-11-09T03:45:58.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.022+0000 7f96b2ffd640 1 -- 172.21.15.76:0/2583640888 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ac02d870 con 0x7f96a0004d00 2023-11-09T03:45:58.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.022+0000 7f96b2ffd640 1 -- 172.21.15.76:0/2583640888 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac02dc20 con 0x7f96a0004d00 2023-11-09T03:45:58.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.023+0000 7f96ba73c640 1 -- 172.21.15.76:0/2583640888 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96a0004d00 msgr2=0x7f96a0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:58.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.023+0000 7f96ba73c640 1 --2- 172.21.15.76:0/2583640888 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96a0004d00 0x7f96a0005180 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f96ac001f60 tx=0x7f96a00057f0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.023+0000 7f96b8cd9640 1 -- 172.21.15.76:0/2583640888 reap_dead start 2023-11-09T03:45:58.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.023+0000 7f96ba73c640 1 -- 172.21.15.76:0/2583640888 shutdown_connections 2023-11-09T03:45:58.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.023+0000 7f96ba73c640 1 -- 172.21.15.76:0/2583640888 >> 172.21.15.76:0/2583640888 conn(0x7f96b4071f30 msgr2=0x7f96b40cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:58.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.023+0000 7f96ba73c640 1 -- 172.21.15.76:0/2583640888 shutdown_connections 2023-11-09T03:45:58.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.023+0000 7f96ba73c640 1 -- 172.21.15.76:0/2583640888 wait complete. 2023-11-09T03:45:58.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.024+0000 7f96ba73c640 1 Processor -- start 2023-11-09T03:45:58.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.024+0000 7f96ba73c640 1 -- start start 2023-11-09T03:45:58.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.025+0000 7f96ba73c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96b41f21b0 0x7f96b414ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:58.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.025+0000 7f96ba73c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96b414cf80 0x7f96b4155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:58.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.025+0000 7f96ba73c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96b41562b0 0x7f96b4148f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:58.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.025+0000 7f96ba73c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b414d400 con 0x7f96b41f21b0 2023-11-09T03:45:58.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.025+0000 7f96ba73c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b41f2590 con 0x7f96b414cf80 2023-11-09T03:45:58.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.025+0000 7f96ba73c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b4146ef0 con 0x7f96b41562b0 2023-11-09T03:45:58.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.025+0000 7f96b3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96b41f21b0 0x7f96b414ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:58.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.025+0000 7f96b3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96b41f21b0 0x7f96b414ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36604/0 (socket says 172.21.15.76:36604) 2023-11-09T03:45:58.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.025+0000 7f96b3fff640 1 -- 172.21.15.76:0/648203369 learned_addr learned my addr 172.21.15.76:0/648203369 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:58.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.025+0000 7f96b3fff640 1 -- 172.21.15.76:0/648203369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96b41562b0 msgr2=0x7f96b4148f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:58.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.025+0000 7f96b3fff640 1 --2- 172.21.15.76:0/648203369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96b41562b0 0x7f96b4148f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.025+0000 7f96b3fff640 1 -- 172.21.15.76:0/648203369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96b414cf80 msgr2=0x7f96b4155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:58.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.025+0000 7f96b3fff640 1 --2- 172.21.15.76:0/648203369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96b414cf80 0x7f96b4155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.025+0000 7f96b3fff640 1 -- 172.21.15.76:0/648203369 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b4147790 con 0x7f96b41f21b0 2023-11-09T03:45:58.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.025+0000 7f96b3fff640 1 --2- 172.21.15.76:0/648203369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96b41f21b0 0x7f96b414ca40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f969c011920 tx=0x7f969c011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:58.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.026+0000 7f96b0ff9640 1 -- 172.21.15.76:0/648203369 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c00faf0 con 0x7f96b41f21b0 2023-11-09T03:45:58.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.026+0000 7f96b0ff9640 1 -- 172.21.15.76:0/648203369 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969c00fc50 con 0x7f96b41f21b0 2023-11-09T03:45:58.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.026+0000 7f96ba73c640 1 -- 172.21.15.76:0/648203369 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b414a3e0 con 0x7f96b41f21b0 2023-11-09T03:45:58.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.026+0000 7f96b0ff9640 1 -- 172.21.15.76:0/648203369 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c014440 con 0x7f96b41f21b0 2023-11-09T03:45:58.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.026+0000 7f96ba73c640 1 -- 172.21.15.76:0/648203369 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b4147140 con 0x7f96b41f21b0 2023-11-09T03:45:58.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.027+0000 7f96ba73c640 1 -- 172.21.15.76:0/648203369 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b4074290 con 0x7f96b41f21b0 2023-11-09T03:45:58.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.027+0000 7f96b0ff9640 1 -- 172.21.15.76:0/648203369 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f969c01d400 con 0x7f96b41f21b0 2023-11-09T03:45:58.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.027+0000 7f96b0ff9640 1 --2- 172.21.15.76:0/648203369 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f969406ae70 0x7f969406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:58.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.027+0000 7f96b0ff9640 1 -- 172.21.15.76:0/648203369 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f969c08a180 con 0x7f96b41f21b0 2023-11-09T03:45:58.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.028+0000 7f96b37fe640 1 --2- 172.21.15.76:0/648203369 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f969406ae70 0x7f969406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:58.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.028+0000 7f96b37fe640 1 --2- 172.21.15.76:0/648203369 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f969406ae70 0x7f969406d330 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f96a4001cb0 tx=0x7f96a4001b40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:58.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.031+0000 7f96b0ff9640 1 -- 172.21.15.76:0/648203369 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f969c054f40 con 0x7f96b41f21b0 2023-11-09T03:45:58.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.200+0000 7f96ba73c640 1 -- 172.21.15.76:0/648203369 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96b40b5720 con 0x7f96b41f21b0 2023-11-09T03:45:58.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.201+0000 7f96b0ff9640 1 -- 172.21.15.76:0/648203369 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18334 (secure 0 0 0) 0x7f969c058560 con 0x7f96b41f21b0 2023-11-09T03:45:58.200 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:58.201 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":25,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:45:35.781752+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:45:58.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.203+0000 7f96ba73c640 1 -- 172.21.15.76:0/648203369 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f969406ae70 msgr2=0x7f969406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:58.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.203+0000 7f96ba73c640 1 --2- 172.21.15.76:0/648203369 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f969406ae70 0x7f969406d330 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f96a4001cb0 tx=0x7f96a4001b40 comp rx=0 tx=0).stop 2023-11-09T03:45:58.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.203+0000 7f96ba73c640 1 -- 172.21.15.76:0/648203369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96b41f21b0 msgr2=0x7f96b414ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:58.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.203+0000 7f96ba73c640 1 --2- 172.21.15.76:0/648203369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96b41f21b0 0x7f96b414ca40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f969c011920 tx=0x7f969c011de0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.203+0000 7f96ba73c640 1 -- 172.21.15.76:0/648203369 shutdown_connections 2023-11-09T03:45:58.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.203+0000 7f96ba73c640 1 --2- 172.21.15.76:0/648203369 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f969406ae70 0x7f969406d330 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.203+0000 7f96ba73c640 1 --2- 172.21.15.76:0/648203369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96b41562b0 0x7f96b4148f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.203+0000 7f96ba73c640 1 --2- 172.21.15.76:0/648203369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96b414cf80 0x7f96b4155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.203+0000 7f96ba73c640 1 --2- 172.21.15.76:0/648203369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96b41f21b0 0x7f96b414ca40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.203+0000 7f96ba73c640 1 -- 172.21.15.76:0/648203369 >> 172.21.15.76:0/648203369 conn(0x7f96b4071f30 msgr2=0x7f96b40c6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:58.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.203+0000 7f96ba73c640 1 -- 172.21.15.76:0/648203369 shutdown_connections 2023-11-09T03:45:58.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.203+0000 7f96ba73c640 1 -- 172.21.15.76:0/648203369 wait complete. 2023-11-09T03:45:58.213 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:45:28.704715+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:45:29.717614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-09T03:45:33.440577+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-09T03:45:58.214 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:45:58.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.318+0000 7f8e8d244640 1 Processor -- start 2023-11-09T03:45:58.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.318+0000 7f8e8d244640 1 -- start start 2023-11-09T03:45:58.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.319+0000 7f8e8d244640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e8815e1c0 0x7f8e8806b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:58.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.319+0000 7f8e8d244640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8e8815e5a0 0x7f8e8815f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:58.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.319+0000 7f8e8d244640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e88066530 con 0x7f8e880c9af0 2023-11-09T03:45:58.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.319+0000 7f8e8d244640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e8806bc40 con 0x7f8e8815e5a0 2023-11-09T03:45:58.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.319+0000 7f8e8d244640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e88065f70 con 0x7f8e8815e1c0 2023-11-09T03:45:58.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.319+0000 7f8e8659c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8e8815e5a0 0x7f8e8815f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:58.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.319+0000 7f8e8659c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8e8815e5a0 0x7f8e8815f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36612/0 (socket says 172.21.15.76:36612) 2023-11-09T03:45:58.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.319+0000 7f8e8659c640 1 -- 172.21.15.76:0/2241805824 learned_addr learned my addr 172.21.15.76:0/2241805824 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:58.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.319+0000 7f8e86d9d640 1 --2- 172.21.15.76:0/2241805824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e8815e1c0 0x7f8e8806b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:58.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.319+0000 7f8e8659c640 1 -- 172.21.15.76:0/2241805824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e8815e1c0 msgr2=0x7f8e8806b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:58.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.319+0000 7f8e8659c640 1 --2- 172.21.15.76:0/2241805824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e8815e1c0 0x7f8e8806b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.319+0000 7f8e8659c640 1 -- 172.21.15.76:0/2241805824 >> v1:172.21.15.113:6789/0 conn(0x7f8e880c9af0 legacy=0x7f8e88070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:58.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.319+0000 7f8e8659c640 1 -- 172.21.15.76:0/2241805824 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e88066250 con 0x7f8e8815e5a0 2023-11-09T03:45:58.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.319+0000 7f8e86d9d640 1 --2- 172.21.15.76:0/2241805824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e8815e1c0 0x7f8e8806b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:58.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.320+0000 7f8e8659c640 1 --2- 172.21.15.76:0/2241805824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e8815e5a0 0x7f8e8815f2d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009870 tx=0x7f8e7c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1efdd62ed9fa5807 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:58.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.320+0000 7f8e85d9b640 1 -- 172.21.15.76:0/2241805824 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c031640 con 0x7f8e8815e5a0 2023-11-09T03:45:58.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.320+0000 7f8e85d9b640 1 -- 172.21.15.76:0/2241805824 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7c0317a0 con 0x7f8e8815e5a0 2023-11-09T03:45:58.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.320+0000 7f8e85d9b640 1 -- 172.21.15.76:0/2241805824 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c031b30 con 0x7f8e8815e5a0 2023-11-09T03:45:58.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.321+0000 7f8e8d244640 1 -- 172.21.15.76:0/2241805824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e8815e5a0 msgr2=0x7f8e8815f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:58.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.321+0000 7f8e8d244640 1 --2- 172.21.15.76:0/2241805824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e8815e5a0 0x7f8e8815f2d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009870 tx=0x7f8e7c02b8f0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.321+0000 7f8e8d244640 1 -- 172.21.15.76:0/2241805824 shutdown_connections 2023-11-09T03:45:58.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.321+0000 7f8e8d244640 1 --2- 172.21.15.76:0/2241805824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e8815e5a0 0x7f8e8815f2d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.321+0000 7f8e8d244640 1 --2- 172.21.15.76:0/2241805824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e8815e1c0 0x7f8e8806b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.321+0000 7f8e8d244640 1 -- 172.21.15.76:0/2241805824 >> 172.21.15.76:0/2241805824 conn(0x7f8e880cbc90 msgr2=0x7f8e880cc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:58.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.321+0000 7f8e8d244640 1 -- 172.21.15.76:0/2241805824 shutdown_connections 2023-11-09T03:45:58.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.321+0000 7f8e8d244640 1 -- 172.21.15.76:0/2241805824 wait complete. 2023-11-09T03:45:58.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.322+0000 7f8e8d244640 1 Processor -- start 2023-11-09T03:45:58.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.322+0000 7f8e8d244640 1 -- start start 2023-11-09T03:45:58.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.323+0000 7f8e8d244640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e880c9af0 0x7f8e88154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:58.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.323+0000 7f8e8d244640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e8815e1c0 0x7f8e880c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:58.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.323+0000 7f8e8d244640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e8815e5a0 0x7f8e880c38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:58.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.323+0000 7f8e8d244640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e88156650 con 0x7f8e880c9af0 2023-11-09T03:45:58.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.323+0000 7f8e8d244640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e88155a40 con 0x7f8e8815e1c0 2023-11-09T03:45:58.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.323+0000 7f8e8d244640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e88155d20 con 0x7f8e8815e5a0 2023-11-09T03:45:58.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.323+0000 7f8e86d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e880c9af0 0x7f8e88154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:58.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.323+0000 7f8e86d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e880c9af0 0x7f8e88154a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36622/0 (socket says 172.21.15.76:36622) 2023-11-09T03:45:58.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.323+0000 7f8e86d9d640 1 -- 172.21.15.76:0/3073073386 learned_addr learned my addr 172.21.15.76:0/3073073386 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:58.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.323+0000 7f8e8659c640 1 --2- 172.21.15.76:0/3073073386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e8815e1c0 0x7f8e880c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:58.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.323+0000 7f8e8759e640 1 --2- 172.21.15.76:0/3073073386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e8815e5a0 0x7f8e880c38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:58.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.323+0000 7f8e86d9d640 1 -- 172.21.15.76:0/3073073386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e8815e5a0 msgr2=0x7f8e880c38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:58.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.323+0000 7f8e86d9d640 1 --2- 172.21.15.76:0/3073073386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e8815e5a0 0x7f8e880c38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.323+0000 7f8e86d9d640 1 -- 172.21.15.76:0/3073073386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e8815e1c0 msgr2=0x7f8e880c62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:58.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.323+0000 7f8e86d9d640 1 --2- 172.21.15.76:0/3073073386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e8815e1c0 0x7f8e880c62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.323+0000 7f8e86d9d640 1 -- 172.21.15.76:0/3073073386 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e7c0094d0 con 0x7f8e880c9af0 2023-11-09T03:45:58.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.324+0000 7f8e86d9d640 1 --2- 172.21.15.76:0/3073073386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e880c9af0 0x7f8e88154a90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8e7800e7e0 tx=0x7f8e7800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:58.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.324+0000 7f8e6f7fe640 1 -- 172.21.15.76:0/3073073386 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e78004030 con 0x7f8e880c9af0 2023-11-09T03:45:58.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.324+0000 7f8e6f7fe640 1 -- 172.21.15.76:0/3073073386 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e78004190 con 0x7f8e880c9af0 2023-11-09T03:45:58.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.324+0000 7f8e8d244640 1 -- 172.21.15.76:0/3073073386 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e88156000 con 0x7f8e880c9af0 2023-11-09T03:45:58.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.324+0000 7f8e6f7fe640 1 -- 172.21.15.76:0/3073073386 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e78004520 con 0x7f8e880c9af0 2023-11-09T03:45:58.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.324+0000 7f8e8d244640 1 -- 172.21.15.76:0/3073073386 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e88154fd0 con 0x7f8e880c9af0 2023-11-09T03:45:58.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.325+0000 7f8e8d244640 1 -- 172.21.15.76:0/3073073386 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e88074290 con 0x7f8e880c9af0 2023-11-09T03:45:58.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.325+0000 7f8e6f7fe640 1 -- 172.21.15.76:0/3073073386 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8e7800c6d0 con 0x7f8e880c9af0 2023-11-09T03:45:58.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.326+0000 7f8e6f7fe640 1 --2- 172.21.15.76:0/3073073386 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8e7406ae70 0x7f8e7406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:58.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.326+0000 7f8e6f7fe640 1 -- 172.21.15.76:0/3073073386 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8e78089560 con 0x7f8e880c9af0 2023-11-09T03:45:58.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.326+0000 7f8e8659c640 1 --2- 172.21.15.76:0/3073073386 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8e7406ae70 0x7f8e7406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:58.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.326+0000 7f8e8659c640 1 --2- 172.21.15.76:0/3073073386 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8e7406ae70 0x7f8e7406d330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c036000 tx=0x7f8e7c03c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:58.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.330+0000 7f8e6f7fe640 1 -- 172.21.15.76:0/3073073386 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8e780543b0 con 0x7f8e880c9af0 2023-11-09T03:45:58.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.494+0000 7f8e8d244640 1 -- 172.21.15.76:0/3073073386 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8e88072030 con 0x7f8e880c9af0 2023-11-09T03:45:58.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.495+0000 7f8e6f7fe640 1 -- 172.21.15.76:0/3073073386 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8e780579d0 con 0x7f8e880c9af0 2023-11-09T03:45:58.494 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:45:58.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.497+0000 7f8e8d244640 1 -- 172.21.15.76:0/3073073386 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8e7406ae70 msgr2=0x7f8e7406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:58.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.497+0000 7f8e8d244640 1 --2- 172.21.15.76:0/3073073386 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8e7406ae70 0x7f8e7406d330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c036000 tx=0x7f8e7c03c040 comp rx=0 tx=0).stop 2023-11-09T03:45:58.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.497+0000 7f8e8d244640 1 -- 172.21.15.76:0/3073073386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e880c9af0 msgr2=0x7f8e88154a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:58.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.497+0000 7f8e8d244640 1 --2- 172.21.15.76:0/3073073386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e880c9af0 0x7f8e88154a90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8e7800e7e0 tx=0x7f8e7800eca0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.497+0000 7f8e8d244640 1 -- 172.21.15.76:0/3073073386 shutdown_connections 2023-11-09T03:45:58.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.497+0000 7f8e8d244640 1 --2- 172.21.15.76:0/3073073386 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8e7406ae70 0x7f8e7406d330 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.497+0000 7f8e8d244640 1 --2- 172.21.15.76:0/3073073386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e8815e5a0 0x7f8e880c38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.497+0000 7f8e8d244640 1 --2- 172.21.15.76:0/3073073386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e8815e1c0 0x7f8e880c62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.497+0000 7f8e8d244640 1 --2- 172.21.15.76:0/3073073386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e880c9af0 0x7f8e88154a90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.497+0000 7f8e8d244640 1 -- 172.21.15.76:0/3073073386 >> 172.21.15.76:0/3073073386 conn(0x7f8e880cbc90 msgr2=0x7f8e88158b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:58.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.497+0000 7f8e8d244640 1 -- 172.21.15.76:0/3073073386 shutdown_connections 2023-11-09T03:45:58.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.497+0000 7f8e8d244640 1 -- 172.21.15.76:0/3073073386 wait complete. 2023-11-09T03:45:58.507 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:45:58.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.610+0000 7f399d890640 1 Processor -- start 2023-11-09T03:45:58.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.610+0000 7f399d890640 1 -- start start 2023-11-09T03:45:58.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.611+0000 7f399d890640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f39980cd6d0 0x7f3998161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:58.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.611+0000 7f399d890640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f39980b4880 0x7f39980b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:58.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.611+0000 7f399d890640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39980cde60 con 0x7f3998153e60 2023-11-09T03:45:58.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.611+0000 7f399d890640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3998160f90 con 0x7f39980b4880 2023-11-09T03:45:58.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.611+0000 7f399d890640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3998161960 con 0x7f39980cd6d0 2023-11-09T03:45:58.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.611+0000 7f3996ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f3998153e60 0x7f399815dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33736/0 (socket says 172.21.15.76:33736) 2023-11-09T03:45:58.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.611+0000 7f3996ffd640 1 -- 172.21.15.76:0/167417907 learned_addr learned my addr 172.21.15.76:0/167417907 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:58.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.611+0000 7f39977fe640 1 --2- 172.21.15.76:0/167417907 >> v2:172.21.15.76:3300/0 conn(0x7f39980b4880 0x7f39980b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:58.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.611+0000 7f39967fc640 1 -- 172.21.15.76:0/167417907 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2101042525 0 0) 0x7f39980cde60 con 0x7f3998153e60 2023-11-09T03:45:58.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.611+0000 7f398ffff640 1 --2- 172.21.15.76:0/167417907 >> v2:172.21.15.113:3300/0 conn(0x7f39980cd6d0 0x7f3998161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:58.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.611+0000 7f39967fc640 1 -- 172.21.15.76:0/167417907 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3984003520 con 0x7f3998153e60 2023-11-09T03:45:58.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.611+0000 7f39977fe640 1 -- 172.21.15.76:0/167417907 >> v2:172.21.15.113:3300/0 conn(0x7f39980cd6d0 msgr2=0x7f3998161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:58.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.611+0000 7f39977fe640 1 --2- 172.21.15.76:0/167417907 >> v2:172.21.15.113:3300/0 conn(0x7f39980cd6d0 0x7f3998161420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.611+0000 7f39977fe640 1 -- 172.21.15.76:0/167417907 >> v1:172.21.15.76:6789/0 conn(0x7f3998153e60 legacy=0x7f399815dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:58.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.611+0000 7f39977fe640 1 -- 172.21.15.76:0/167417907 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39980b5220 con 0x7f39980b4880 2023-11-09T03:45:58.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.612+0000 7f39977fe640 1 --2- 172.21.15.76:0/167417907 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39980b4880 0x7f39980b4ce0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f39980c9e80 tx=0x7f399002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4a13434948e293 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:58.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.612+0000 7f39967fc640 1 -- 172.21.15.76:0/167417907 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399002cc80 con 0x7f39980b4880 2023-11-09T03:45:58.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.612+0000 7f39967fc640 1 -- 172.21.15.76:0/167417907 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399002cde0 con 0x7f39980b4880 2023-11-09T03:45:58.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.612+0000 7f39967fc640 1 -- 172.21.15.76:0/167417907 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3990034660 con 0x7f39980b4880 2023-11-09T03:45:58.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.613+0000 7f399d890640 1 -- 172.21.15.76:0/167417907 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39980b4880 msgr2=0x7f39980b4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:58.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.613+0000 7f399d890640 1 --2- 172.21.15.76:0/167417907 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39980b4880 0x7f39980b4ce0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f39980c9e80 tx=0x7f399002bed0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.613+0000 7f399d890640 1 -- 172.21.15.76:0/167417907 shutdown_connections 2023-11-09T03:45:58.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.613+0000 7f399d890640 1 --2- 172.21.15.76:0/167417907 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39980b4880 0x7f39980b4ce0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.613+0000 7f399d890640 1 --2- 172.21.15.76:0/167417907 >> v2:172.21.15.113:3300/0 conn(0x7f39980cd6d0 0x7f3998161420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.613+0000 7f399d890640 1 -- 172.21.15.76:0/167417907 >> 172.21.15.76:0/167417907 conn(0x7f39980cf880 msgr2=0x7f39980cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:58.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.613+0000 7f399d890640 1 -- 172.21.15.76:0/167417907 shutdown_connections 2023-11-09T03:45:58.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.613+0000 7f399d890640 1 -- 172.21.15.76:0/167417907 wait complete. 2023-11-09T03:45:58.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.614+0000 7f399d890640 1 Processor -- start 2023-11-09T03:45:58.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.614+0000 7f399d890640 1 -- start start 2023-11-09T03:45:58.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.615+0000 7f399d890640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39980b4880 0x7f39980c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:58.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.615+0000 7f399d890640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39980cd6d0 0x7f399814f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:58.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.615+0000 7f399d890640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3998153e60 0x7f39980cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:58.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.615+0000 7f399d890640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39980ca970 con 0x7f3998153e60 2023-11-09T03:45:58.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.615+0000 7f399d890640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39980c9340 con 0x7f39980cd6d0 2023-11-09T03:45:58.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.615+0000 7f399d890640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39980ca2f0 con 0x7f39980b4880 2023-11-09T03:45:58.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.615+0000 7f39977fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3998153e60 0x7f39980cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:58.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.615+0000 7f39977fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3998153e60 0x7f39980cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36646/0 (socket says 172.21.15.76:36646) 2023-11-09T03:45:58.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.615+0000 7f39977fe640 1 -- 172.21.15.76:0/4255192893 learned_addr learned my addr 172.21.15.76:0/4255192893 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:58.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.615+0000 7f398ffff640 1 --2- 172.21.15.76:0/4255192893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39980cd6d0 0x7f399814f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:58.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.615+0000 7f3996ffd640 1 --2- 172.21.15.76:0/4255192893 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39980b4880 0x7f39980c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:58.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.615+0000 7f39977fe640 1 -- 172.21.15.76:0/4255192893 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39980b4880 msgr2=0x7f39980c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:58.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.615+0000 7f39977fe640 1 --2- 172.21.15.76:0/4255192893 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39980b4880 0x7f39980c8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.615+0000 7f39977fe640 1 -- 172.21.15.76:0/4255192893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39980cd6d0 msgr2=0x7f399814f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:58.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.615+0000 7f39977fe640 1 --2- 172.21.15.76:0/4255192893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39980cd6d0 0x7f399814f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.615+0000 7f39977fe640 1 -- 172.21.15.76:0/4255192893 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3990009a70 con 0x7f3998153e60 2023-11-09T03:45:58.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.615+0000 7f398ffff640 1 --2- 172.21.15.76:0/4255192893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39980cd6d0 0x7f399814f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:45:58.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.615+0000 7f39977fe640 1 --2- 172.21.15.76:0/4255192893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3998153e60 0x7f39980cc6e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3990009e80 tx=0x7f3990004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:58.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.616+0000 7f3994ff9640 1 -- 172.21.15.76:0/4255192893 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3990040650 con 0x7f3998153e60 2023-11-09T03:45:58.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.616+0000 7f3994ff9640 1 -- 172.21.15.76:0/4255192893 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39900407b0 con 0x7f3998153e60 2023-11-09T03:45:58.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.616+0000 7f399d890640 1 -- 172.21.15.76:0/4255192893 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39980c8da0 con 0x7f3998153e60 2023-11-09T03:45:58.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.616+0000 7f3994ff9640 1 -- 172.21.15.76:0/4255192893 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3990040b60 con 0x7f3998153e60 2023-11-09T03:45:58.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.616+0000 7f399d890640 1 -- 172.21.15.76:0/4255192893 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39980ca590 con 0x7f3998153e60 2023-11-09T03:45:58.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.617+0000 7f399d890640 1 -- 172.21.15.76:0/4255192893 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39980ca150 con 0x7f3998153e60 2023-11-09T03:45:58.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.617+0000 7f3994ff9640 1 -- 172.21.15.76:0/4255192893 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3990040cc0 con 0x7f3998153e60 2023-11-09T03:45:58.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.617+0000 7f3994ff9640 1 --2- 172.21.15.76:0/4255192893 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f397806ae70 0x7f397806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:58.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.617+0000 7f3994ff9640 1 -- 172.21.15.76:0/4255192893 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f39900ac630 con 0x7f3998153e60 2023-11-09T03:45:58.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.617+0000 7f3996ffd640 1 --2- 172.21.15.76:0/4255192893 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f397806ae70 0x7f397806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:58.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.618+0000 7f3996ffd640 1 --2- 172.21.15.76:0/4255192893 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f397806ae70 0x7f397806d330 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f39880011e0 tx=0x7f3988009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:58.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.621+0000 7f3994ff9640 1 -- 172.21.15.76:0/4255192893 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f39980ca150 con 0x7f3998153e60 2023-11-09T03:45:58.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.788+0000 7f399d890640 1 -- 172.21.15.76:0/4255192893 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f39980caab0 con 0x7f3998153e60 2023-11-09T03:45:58.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.788+0000 7f3994ff9640 1 -- 172.21.15.76:0/4255192893 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f39980caab0 con 0x7f3998153e60 2023-11-09T03:45:58.788 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:45:58.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.790+0000 7f399d890640 1 -- 172.21.15.76:0/4255192893 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f397806ae70 msgr2=0x7f397806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:58.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.790+0000 7f399d890640 1 --2- 172.21.15.76:0/4255192893 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f397806ae70 0x7f397806d330 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f39880011e0 tx=0x7f3988009040 comp rx=0 tx=0).stop 2023-11-09T03:45:58.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.790+0000 7f399d890640 1 -- 172.21.15.76:0/4255192893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3998153e60 msgr2=0x7f39980cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:58.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.790+0000 7f399d890640 1 --2- 172.21.15.76:0/4255192893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3998153e60 0x7f39980cc6e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3990009e80 tx=0x7f3990004a40 comp rx=0 tx=0).stop 2023-11-09T03:45:58.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.790+0000 7f399d890640 1 -- 172.21.15.76:0/4255192893 shutdown_connections 2023-11-09T03:45:58.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.790+0000 7f399d890640 1 --2- 172.21.15.76:0/4255192893 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f397806ae70 0x7f397806d330 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.790+0000 7f399d890640 1 --2- 172.21.15.76:0/4255192893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3998153e60 0x7f39980cc6e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.790+0000 7f399d890640 1 --2- 172.21.15.76:0/4255192893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39980cd6d0 0x7f399814f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.790+0000 7f399d890640 1 --2- 172.21.15.76:0/4255192893 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39980b4880 0x7f39980c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.790+0000 7f399d890640 1 -- 172.21.15.76:0/4255192893 >> 172.21.15.76:0/4255192893 conn(0x7f39980cf880 msgr2=0x7f399815b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:58.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.790+0000 7f399d890640 1 -- 172.21.15.76:0/4255192893 shutdown_connections 2023-11-09T03:45:58.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.790+0000 7f399d890640 1 -- 172.21.15.76:0/4255192893 wait complete. 2023-11-09T03:45:58.800 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:45:58.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.904+0000 7f7eb8f65640 1 Processor -- start 2023-11-09T03:45:58.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.904+0000 7f7eb8f65640 1 -- start start 2023-11-09T03:45:58.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.904+0000 7f7eb8f65640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7eb4159c00 0x7f7eb40cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:58.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.904+0000 7f7eb8f65640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7eb40ae1d0 0x7f7eb40c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:58.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.905+0000 7f7eb8f65640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eb415aff0 con 0x7f7eb40c48e0 2023-11-09T03:45:58.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.905+0000 7f7eb8f65640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eb40c52d0 con 0x7f7eb4159c00 2023-11-09T03:45:58.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.905+0000 7f7eb8f65640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb415a3e0 con 0x7f7eb40ae1d0 2023-11-09T03:45:58.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.905+0000 7f7eb259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7eb4159c00 0x7f7eb40cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:58.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.905+0000 7f7eb259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7eb4159c00 0x7f7eb40cc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36662/0 (socket says 172.21.15.76:36662) 2023-11-09T03:45:58.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.905+0000 7f7eb259c640 1 -- 172.21.15.76:0/3945697192 learned_addr learned my addr 172.21.15.76:0/3945697192 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:58.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.905+0000 7f7eb2d9d640 1 --2- 172.21.15.76:0/3945697192 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7eb40ae1d0 0x7f7eb40c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:58.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.905+0000 7f7eb259c640 1 -- 172.21.15.76:0/3945697192 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7eb40ae1d0 msgr2=0x7f7eb40c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:58.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.905+0000 7f7eb259c640 1 --2- 172.21.15.76:0/3945697192 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7eb40ae1d0 0x7f7eb40c4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.905+0000 7f7eb259c640 1 -- 172.21.15.76:0/3945697192 >> v1:172.21.15.113:6789/0 conn(0x7f7eb40c48e0 legacy=0x7f7eb40adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:58.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.905+0000 7f7eb259c640 1 -- 172.21.15.76:0/3945697192 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb415a6c0 con 0x7f7eb4159c00 2023-11-09T03:45:58.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.906+0000 7f7eb259c640 1 --2- 172.21.15.76:0/3945697192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7eb4159c00 0x7f7eb40cc890 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4009870 tx=0x7f7ea402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d796ed888d3238e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:58.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.906+0000 7f7eb1d9b640 1 -- 172.21.15.76:0/3945697192 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea40317c0 con 0x7f7eb4159c00 2023-11-09T03:45:58.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.906+0000 7f7eb1d9b640 1 -- 172.21.15.76:0/3945697192 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea4031920 con 0x7f7eb4159c00 2023-11-09T03:45:58.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.906+0000 7f7eb1d9b640 1 -- 172.21.15.76:0/3945697192 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4031cb0 con 0x7f7eb4159c00 2023-11-09T03:45:58.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.906+0000 7f7eb8f65640 1 -- 172.21.15.76:0/3945697192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7eb4159c00 msgr2=0x7f7eb40cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:58.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.906+0000 7f7eb8f65640 1 --2- 172.21.15.76:0/3945697192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7eb4159c00 0x7f7eb40cc890 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4009870 tx=0x7f7ea402b860 comp rx=0 tx=0).stop 2023-11-09T03:45:58.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.906+0000 7f7eb8f65640 1 -- 172.21.15.76:0/3945697192 shutdown_connections 2023-11-09T03:45:58.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.906+0000 7f7eb8f65640 1 --2- 172.21.15.76:0/3945697192 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7eb40ae1d0 0x7f7eb40c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.906+0000 7f7eb8f65640 1 --2- 172.21.15.76:0/3945697192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7eb4159c00 0x7f7eb40cc890 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.906+0000 7f7eb8f65640 1 -- 172.21.15.76:0/3945697192 >> 172.21.15.76:0/3945697192 conn(0x7f7eb40ce950 msgr2=0x7f7eb40ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:58.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.906+0000 7f7eb8f65640 1 -- 172.21.15.76:0/3945697192 shutdown_connections 2023-11-09T03:45:58.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.906+0000 7f7eb8f65640 1 -- 172.21.15.76:0/3945697192 wait complete. 2023-11-09T03:45:58.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.908+0000 7f7eb8f65640 1 Processor -- start 2023-11-09T03:45:58.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.908+0000 7f7eb8f65640 1 -- start start 2023-11-09T03:45:58.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.908+0000 7f7eb8f65640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7eb40ae1d0 0x7f7eb4157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:58.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.908+0000 7f7eb8f65640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7eb40c48e0 0x7f7eb41580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:58.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.908+0000 7f7eb8f65640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7eb4159c00 0x7f7eb40a74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:58.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.908+0000 7f7eb8f65640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb40a8a40 con 0x7f7eb40c48e0 2023-11-09T03:45:58.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.908+0000 7f7eb8f65640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb40aab50 con 0x7f7eb4159c00 2023-11-09T03:45:58.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.908+0000 7f7eb8f65640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb40aae30 con 0x7f7eb40ae1d0 2023-11-09T03:45:58.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.908+0000 7f7eabfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7eb40c48e0 0x7f7eb41580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:58.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.909+0000 7f7eabfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7eb40c48e0 0x7f7eb41580a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36678/0 (socket says 172.21.15.76:36678) 2023-11-09T03:45:58.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.909+0000 7f7eabfff640 1 -- 172.21.15.76:0/1107509913 learned_addr learned my addr 172.21.15.76:0/1107509913 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:58.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.909+0000 7f7eb2d9d640 1 --2- 172.21.15.76:0/1107509913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7eb4159c00 0x7f7eb40a74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:58.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.909+0000 7f7eb259c640 1 --2- 172.21.15.76:0/1107509913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7eb40ae1d0 0x7f7eb4157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:58.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.909+0000 7f7eabfff640 1 -- 172.21.15.76:0/1107509913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7eb40ae1d0 msgr2=0x7f7eb4157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:58.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.909+0000 7f7eabfff640 1 --2- 172.21.15.76:0/1107509913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7eb40ae1d0 0x7f7eb4157b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.909+0000 7f7eabfff640 1 -- 172.21.15.76:0/1107509913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7eb4159c00 msgr2=0x7f7eb40a74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:58.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.909+0000 7f7eabfff640 1 --2- 172.21.15.76:0/1107509913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7eb4159c00 0x7f7eb40a74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:58.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.909+0000 7f7eabfff640 1 -- 172.21.15.76:0/1107509913 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea40094d0 con 0x7f7eb40c48e0 2023-11-09T03:45:58.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.909+0000 7f7eabfff640 1 --2- 172.21.15.76:0/1107509913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7eb40c48e0 0x7f7eb41580a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00c430 tx=0x7f7e9c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:58.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.909+0000 7f7eaaffd640 1 -- 172.21.15.76:0/1107509913 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c011790 con 0x7f7eb40c48e0 2023-11-09T03:45:58.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.909+0000 7f7eaaffd640 1 -- 172.21.15.76:0/1107509913 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9c0118f0 con 0x7f7eb40c48e0 2023-11-09T03:45:58.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.909+0000 7f7eb8f65640 1 -- 172.21.15.76:0/1107509913 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb40ab110 con 0x7f7eb40c48e0 2023-11-09T03:45:58.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.909+0000 7f7eaaffd640 1 -- 172.21.15.76:0/1107509913 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c011ce0 con 0x7f7eb40c48e0 2023-11-09T03:45:58.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.909+0000 7f7eb8f65640 1 -- 172.21.15.76:0/1107509913 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb41f0710 con 0x7f7eb40c48e0 2023-11-09T03:45:58.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.910+0000 7f7eb8f65640 1 -- 172.21.15.76:0/1107509913 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb40a8c40 con 0x7f7eb40c48e0 2023-11-09T03:45:58.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.911+0000 7f7eaaffd640 1 -- 172.21.15.76:0/1107509913 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7e9c01c460 con 0x7f7eb40c48e0 2023-11-09T03:45:58.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.911+0000 7f7eaaffd640 1 --2- 172.21.15.76:0/1107509913 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7e9406ae70 0x7f7e9406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:58.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.911+0000 7f7eaaffd640 1 -- 172.21.15.76:0/1107509913 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7e9c0898d0 con 0x7f7eb40c48e0 2023-11-09T03:45:58.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.911+0000 7f7eb259c640 1 --2- 172.21.15.76:0/1107509913 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7e9406ae70 0x7f7e9406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:58.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.912+0000 7f7eb259c640 1 --2- 172.21.15.76:0/1107509913 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7e9406ae70 0x7f7e9406d330 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4036000 tx=0x7f7ea403d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:58.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:58.915+0000 7f7eaaffd640 1 -- 172.21.15.76:0/1107509913 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7eb40a8c40 con 0x7f7eb40c48e0 2023-11-09T03:45:59.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.084+0000 7f7eb8f65640 1 -- 172.21.15.76:0/1107509913 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7eb40a8c40 con 0x7f7eb40c48e0 2023-11-09T03:45:59.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.084+0000 7f7eaaffd640 1 -- 172.21.15.76:0/1107509913 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7eb40a8c40 con 0x7f7eb40c48e0 2023-11-09T03:45:59.083 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:45:59.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.086+0000 7f7eb8f65640 1 -- 172.21.15.76:0/1107509913 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7e9406ae70 msgr2=0x7f7e9406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:59.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.086+0000 7f7eb8f65640 1 --2- 172.21.15.76:0/1107509913 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7e9406ae70 0x7f7e9406d330 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4036000 tx=0x7f7ea403d040 comp rx=0 tx=0).stop 2023-11-09T03:45:59.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.086+0000 7f7eb8f65640 1 -- 172.21.15.76:0/1107509913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7eb40c48e0 msgr2=0x7f7eb41580a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:59.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.086+0000 7f7eb8f65640 1 --2- 172.21.15.76:0/1107509913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7eb40c48e0 0x7f7eb41580a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00c430 tx=0x7f7e9c00c8f0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.086+0000 7f7eb8f65640 1 -- 172.21.15.76:0/1107509913 shutdown_connections 2023-11-09T03:45:59.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.086+0000 7f7eb8f65640 1 --2- 172.21.15.76:0/1107509913 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7e9406ae70 0x7f7e9406d330 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.086+0000 7f7eb8f65640 1 --2- 172.21.15.76:0/1107509913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7eb4159c00 0x7f7eb40a74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.086+0000 7f7eb8f65640 1 --2- 172.21.15.76:0/1107509913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7eb40c48e0 0x7f7eb41580a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.086+0000 7f7eb8f65640 1 --2- 172.21.15.76:0/1107509913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7eb40ae1d0 0x7f7eb4157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.086+0000 7f7eb8f65640 1 -- 172.21.15.76:0/1107509913 >> 172.21.15.76:0/1107509913 conn(0x7f7eb40ce950 msgr2=0x7f7eb40c5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:59.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.086+0000 7f7eb8f65640 1 -- 172.21.15.76:0/1107509913 shutdown_connections 2023-11-09T03:45:59.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.086+0000 7f7eb8f65640 1 -- 172.21.15.76:0/1107509913 wait complete. 2023-11-09T03:45:59.096 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-09T03:45:59.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.199+0000 7fb27bb25640 1 Processor -- start 2023-11-09T03:45:59.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.199+0000 7fb27bb25640 1 -- start start 2023-11-09T03:45:59.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.200+0000 7fb27bb25640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb274066000 0x7fb2740cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:59.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.200+0000 7fb27bb25640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2740c9bf0 con 0x7fb274070ff0 2023-11-09T03:45:59.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.200+0000 7fb27bb25640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2740cd890 con 0x7fb27415d1b0 2023-11-09T03:45:59.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.200+0000 7fb27bb25640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2740cdb30 con 0x7fb274066000 2023-11-09T03:45:59.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.200+0000 7fb27a0c2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb274066000 0x7fb2740cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:59.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.200+0000 7fb2798c1640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb274070ff0 0x7fb2740713d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33738/0 (socket says 172.21.15.76:33738) 2023-11-09T03:45:59.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.200+0000 7fb2798c1640 1 -- 172.21.15.76:0/1365917915 learned_addr learned my addr 172.21.15.76:0/1365917915 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:59.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.200+0000 7fb27a0c2640 1 -- 172.21.15.76:0/1365917915 >> v1:172.21.15.113:6789/0 conn(0x7fb27415d1b0 legacy=0x7fb27406a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:59.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.200+0000 7fb27a0c2640 1 -- 172.21.15.76:0/1365917915 >> v1:172.21.15.76:6789/0 conn(0x7fb274070ff0 legacy=0x7fb2740713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:59.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.200+0000 7fb27a0c2640 1 -- 172.21.15.76:0/1365917915 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2740cde10 con 0x7fb274066000 2023-11-09T03:45:59.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.201+0000 7fb27a0c2640 1 --2- 172.21.15.76:0/1365917915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb274066000 0x7fb2740cb4e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb26c009d90 tx=0x7fb26c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5fb1d275aa8b51 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:59.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.201+0000 7fb2788bf640 1 -- 172.21.15.76:0/1365917915 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c02cc60 con 0x7fb274066000 2023-11-09T03:45:59.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.201+0000 7fb2788bf640 1 -- 172.21.15.76:0/1365917915 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26c02cdc0 con 0x7fb274066000 2023-11-09T03:45:59.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.201+0000 7fb2788bf640 1 -- 172.21.15.76:0/1365917915 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c034660 con 0x7fb274066000 2023-11-09T03:45:59.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.201+0000 7fb27bb25640 1 -- 172.21.15.76:0/1365917915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb274066000 msgr2=0x7fb2740cb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:59.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.201+0000 7fb27bb25640 1 --2- 172.21.15.76:0/1365917915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb274066000 0x7fb2740cb4e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb26c009d90 tx=0x7fb26c02beb0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.202+0000 7fb27bb25640 1 -- 172.21.15.76:0/1365917915 shutdown_connections 2023-11-09T03:45:59.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.202+0000 7fb27bb25640 1 --2- 172.21.15.76:0/1365917915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb274066000 0x7fb2740cb4e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.202+0000 7fb27bb25640 1 -- 172.21.15.76:0/1365917915 >> 172.21.15.76:0/1365917915 conn(0x7fb2740cbc90 msgr2=0x7fb2740cc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:59.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.202+0000 7fb27bb25640 1 -- 172.21.15.76:0/1365917915 shutdown_connections 2023-11-09T03:45:59.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.202+0000 7fb27bb25640 1 -- 172.21.15.76:0/1365917915 wait complete. 2023-11-09T03:45:59.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.203+0000 7fb27bb25640 1 Processor -- start 2023-11-09T03:45:59.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.203+0000 7fb27bb25640 1 -- start start 2023-11-09T03:45:59.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.203+0000 7fb27bb25640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb274066000 0x7fb2741560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:59.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.203+0000 7fb27bb25640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb274070ff0 0x7fb274157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:59.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.203+0000 7fb27bb25640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb27415d1b0 0x7fb274157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:59.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.203+0000 7fb27bb25640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2740b40f0 con 0x7fb27415d1b0 2023-11-09T03:45:59.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.203+0000 7fb27bb25640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2740b43d0 con 0x7fb274066000 2023-11-09T03:45:59.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.203+0000 7fb27bb25640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2740b46b0 con 0x7fb274070ff0 2023-11-09T03:45:59.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.204+0000 7fb27a0c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb27415d1b0 0x7fb274157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:59.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.204+0000 7fb27a0c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb27415d1b0 0x7fb274157760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36702/0 (socket says 172.21.15.76:36702) 2023-11-09T03:45:59.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.204+0000 7fb27a0c2640 1 -- 172.21.15.76:0/4222609458 learned_addr learned my addr 172.21.15.76:0/4222609458 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:59.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.204+0000 7fb2798c1640 1 --2- 172.21.15.76:0/4222609458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb274066000 0x7fb2741560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:59.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.204+0000 7fb2790c0640 1 --2- 172.21.15.76:0/4222609458 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb274070ff0 0x7fb274157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:59.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.204+0000 7fb27a0c2640 1 -- 172.21.15.76:0/4222609458 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb274070ff0 msgr2=0x7fb274157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:59.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.204+0000 7fb27a0c2640 1 --2- 172.21.15.76:0/4222609458 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb274070ff0 0x7fb274157220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.204+0000 7fb27a0c2640 1 -- 172.21.15.76:0/4222609458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb274066000 msgr2=0x7fb2741560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:59.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.204+0000 7fb27a0c2640 1 --2- 172.21.15.76:0/4222609458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb274066000 0x7fb2741560a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.204+0000 7fb27a0c2640 1 -- 172.21.15.76:0/4222609458 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26c009a70 con 0x7fb27415d1b0 2023-11-09T03:45:59.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.204+0000 7fb27a0c2640 1 --2- 172.21.15.76:0/4222609458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb27415d1b0 0x7fb274157760 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb26c009d90 tx=0x7fb26c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:59.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.204+0000 7fb26a7fc640 1 -- 172.21.15.76:0/4222609458 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c040450 con 0x7fb27415d1b0 2023-11-09T03:45:59.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.204+0000 7fb26a7fc640 1 -- 172.21.15.76:0/4222609458 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26c0405d0 con 0x7fb27415d1b0 2023-11-09T03:45:59.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.204+0000 7fb27bb25640 1 -- 172.21.15.76:0/4222609458 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2740b4990 con 0x7fb27415d1b0 2023-11-09T03:45:59.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.204+0000 7fb26a7fc640 1 -- 172.21.15.76:0/4222609458 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c040980 con 0x7fb27415d1b0 2023-11-09T03:45:59.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.204+0000 7fb27bb25640 1 -- 172.21.15.76:0/4222609458 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2741565e0 con 0x7fb27415d1b0 2023-11-09T03:45:59.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.205+0000 7fb27bb25640 1 -- 172.21.15.76:0/4222609458 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb274074290 con 0x7fb27415d1b0 2023-11-09T03:45:59.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.206+0000 7fb26a7fc640 1 -- 172.21.15.76:0/4222609458 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb26c040ae0 con 0x7fb27415d1b0 2023-11-09T03:45:59.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.206+0000 7fb26a7fc640 1 --2- 172.21.15.76:0/4222609458 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb25806ae70 0x7fb25806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:59.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.206+0000 7fb26a7fc640 1 -- 172.21.15.76:0/4222609458 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb26c0ac460 con 0x7fb27415d1b0 2023-11-09T03:45:59.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.206+0000 7fb2798c1640 1 --2- 172.21.15.76:0/4222609458 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb25806ae70 0x7fb25806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:59.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.207+0000 7fb2798c1640 1 --2- 172.21.15.76:0/4222609458 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb25806ae70 0x7fb25806d330 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb270005880 tx=0x7fb270009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:59.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.210+0000 7fb26a7fc640 1 -- 172.21.15.76:0/4222609458 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb26c0772e0 con 0x7fb27415d1b0 2023-11-09T03:45:59.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.377+0000 7fb27bb25640 1 -- 172.21.15.76:0/4222609458 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb2740b4230 con 0x7fb27415d1b0 2023-11-09T03:45:59.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.377+0000 7fb26a7fc640 1 -- 172.21.15.76:0/4222609458 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb2740b4230 con 0x7fb27415d1b0 2023-11-09T03:45:59.377 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:45:59.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.379+0000 7fb27bb25640 1 -- 172.21.15.76:0/4222609458 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb25806ae70 msgr2=0x7fb25806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:59.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.379+0000 7fb27bb25640 1 --2- 172.21.15.76:0/4222609458 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb25806ae70 0x7fb25806d330 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb270005880 tx=0x7fb270009040 comp rx=0 tx=0).stop 2023-11-09T03:45:59.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.379+0000 7fb27bb25640 1 -- 172.21.15.76:0/4222609458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb27415d1b0 msgr2=0x7fb274157760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:59.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.379+0000 7fb27bb25640 1 --2- 172.21.15.76:0/4222609458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb27415d1b0 0x7fb274157760 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb26c009d90 tx=0x7fb26c004a40 comp rx=0 tx=0).stop 2023-11-09T03:45:59.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.379+0000 7fb27bb25640 1 -- 172.21.15.76:0/4222609458 shutdown_connections 2023-11-09T03:45:59.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.379+0000 7fb27bb25640 1 --2- 172.21.15.76:0/4222609458 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb25806ae70 0x7fb25806d330 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.379+0000 7fb27bb25640 1 --2- 172.21.15.76:0/4222609458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb27415d1b0 0x7fb274157760 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.379+0000 7fb27bb25640 1 --2- 172.21.15.76:0/4222609458 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb274070ff0 0x7fb274157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.379+0000 7fb27bb25640 1 --2- 172.21.15.76:0/4222609458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb274066000 0x7fb2741560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.379+0000 7fb27bb25640 1 -- 172.21.15.76:0/4222609458 >> 172.21.15.76:0/4222609458 conn(0x7fb2740cbc90 msgr2=0x7fb274072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:59.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.379+0000 7fb27bb25640 1 -- 172.21.15.76:0/4222609458 shutdown_connections 2023-11-09T03:45:59.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.379+0000 7fb27bb25640 1 -- 172.21.15.76:0/4222609458 wait complete. 2023-11-09T03:45:59.389 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:45:59.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.491+0000 7f593e62d640 1 Processor -- start 2023-11-09T03:45:59.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.491+0000 7f593e62d640 1 -- start start 2023-11-09T03:45:59.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.492+0000 7f593e62d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f59380cde60 0x7f593815d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:59.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.492+0000 7f593e62d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59380ce240 0x7f59380d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:59.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.492+0000 7f593e62d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f593815e000 con 0x7f593815daa0 2023-11-09T03:45:59.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.492+0000 7f593e62d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f593815e5c0 con 0x7f59380cde60 2023-11-09T03:45:59.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.492+0000 7f593e62d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f593815e2e0 con 0x7f59380ce240 2023-11-09T03:45:59.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.492+0000 7f59377fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f593815daa0 0x7f5938157000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33752/0 (socket says 172.21.15.76:33752) 2023-11-09T03:45:59.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.492+0000 7f59377fe640 1 -- 172.21.15.76:0/2625806384 learned_addr learned my addr 172.21.15.76:0/2625806384 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:45:59.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.493+0000 7f593cbca640 1 --2- 172.21.15.76:0/2625806384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59380ce240 0x7f59380d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:59.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.493+0000 7f5937fff640 1 --2- 172.21.15.76:0/2625806384 >> v2:172.21.15.113:3300/0 conn(0x7f59380cde60 0x7f593815d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:59.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.493+0000 7f5936ffd640 1 -- 172.21.15.76:0/2625806384 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 931884829 0 0) 0x7f593815e000 con 0x7f593815daa0 2023-11-09T03:45:59.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.493+0000 7f5936ffd640 1 -- 172.21.15.76:0/2625806384 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5920003520 con 0x7f593815daa0 2023-11-09T03:45:59.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.493+0000 7f5936ffd640 1 -- 172.21.15.76:0/2625806384 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 996673512 0 0) 0x7f5920003520 con 0x7f593815daa0 2023-11-09T03:45:59.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.493+0000 7f5936ffd640 1 -- 172.21.15.76:0/2625806384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59380ce240 msgr2=0x7f59380d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:59.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.493+0000 7f5936ffd640 1 --2- 172.21.15.76:0/2625806384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59380ce240 0x7f59380d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.493+0000 7f5936ffd640 1 -- 172.21.15.76:0/2625806384 >> v2:172.21.15.113:3300/0 conn(0x7f59380cde60 msgr2=0x7f593815d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:59.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.493+0000 7f5936ffd640 1 --2- 172.21.15.76:0/2625806384 >> v2:172.21.15.113:3300/0 conn(0x7f59380cde60 0x7f593815d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.493+0000 7f5936ffd640 1 -- 172.21.15.76:0/2625806384 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59380d1120 con 0x7f593815daa0 2023-11-09T03:45:59.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.493+0000 7f5936ffd640 1 -- 172.21.15.76:0/2625806384 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f592c001670 con 0x7f593815daa0 2023-11-09T03:45:59.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.493+0000 7f5937fff640 1 --2- 172.21.15.76:0/2625806384 >> v2:172.21.15.113:3300/0 conn(0x7f59380cde60 0x7f593815d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:45:59.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.493+0000 7f593cbca640 1 --2- 172.21.15.76:0/2625806384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59380ce240 0x7f59380d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:45:59.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.493+0000 7f5936ffd640 1 -- 172.21.15.76:0/2625806384 >> v1:172.21.15.76:6789/0 conn(0x7f593815daa0 legacy=0x7f5938157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:59.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.493+0000 7f5936ffd640 1 --2- 172.21.15.76:0/2625806384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5920003970 0x7f5920003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:59.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.493+0000 7f5936ffd640 1 --2- 172.21.15.76:0/2625806384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59380ce240 0x7f59200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:59.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.493+0000 7f5936ffd640 1 --2- 172.21.15.76:0/2625806384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5920004d00 0x7f5920005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:59.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.493+0000 7f5936ffd640 1 -- 172.21.15.76:0/2625806384 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593815e5c0 con 0x7f5920004d00 2023-11-09T03:45:59.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.493+0000 7f5936ffd640 1 -- 172.21.15.76:0/2625806384 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593815e2e0 con 0x7f5920003970 2023-11-09T03:45:59.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.493+0000 7f5936ffd640 1 -- 172.21.15.76:0/2625806384 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f592000b980 con 0x7f59380ce240 2023-11-09T03:45:59.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.493+0000 7f593cbca640 1 --2- 172.21.15.76:0/2625806384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5920004d00 0x7f5920005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:59.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.493+0000 7f59377fe640 1 --2- 172.21.15.76:0/2625806384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59380ce240 0x7f59200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:59.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.493+0000 7f5937fff640 1 --2- 172.21.15.76:0/2625806384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5920003970 0x7f5920003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:59.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.493+0000 7f593cbca640 1 -- 172.21.15.76:0/2625806384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59380ce240 msgr2=0x7f59200047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:59.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.493+0000 7f593cbca640 1 --2- 172.21.15.76:0/2625806384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59380ce240 0x7f59200047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.493+0000 7f593cbca640 1 -- 172.21.15.76:0/2625806384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5920003970 msgr2=0x7f5920003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:59.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.493+0000 7f593cbca640 1 --2- 172.21.15.76:0/2625806384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5920003970 0x7f5920003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.493+0000 7f593cbca640 1 -- 172.21.15.76:0/2625806384 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5930009a70 con 0x7f5920004d00 2023-11-09T03:45:59.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.494+0000 7f593cbca640 1 --2- 172.21.15.76:0/2625806384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5920004d00 0x7f5920005180 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f593000e320 tx=0x7f59200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f297f25be0f949a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:59.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.494+0000 7f5936ffd640 1 -- 172.21.15.76:0/2625806384 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593002cd60 con 0x7f5920004d00 2023-11-09T03:45:59.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.494+0000 7f5936ffd640 1 -- 172.21.15.76:0/2625806384 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593000f040 con 0x7f5920004d00 2023-11-09T03:45:59.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.494+0000 7f5936ffd640 1 -- 172.21.15.76:0/2625806384 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5930003b90 con 0x7f5920004d00 2023-11-09T03:45:59.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.495+0000 7f593e62d640 1 -- 172.21.15.76:0/2625806384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5920004d00 msgr2=0x7f5920005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:59.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.495+0000 7f593e62d640 1 --2- 172.21.15.76:0/2625806384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5920004d00 0x7f5920005180 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f593000e320 tx=0x7f59200057f0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.495+0000 7f593cbca640 1 -- 172.21.15.76:0/2625806384 reap_dead start 2023-11-09T03:45:59.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.495+0000 7f593e62d640 1 -- 172.21.15.76:0/2625806384 shutdown_connections 2023-11-09T03:45:59.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.495+0000 7f593e62d640 1 -- 172.21.15.76:0/2625806384 >> 172.21.15.76:0/2625806384 conn(0x7f5938071f30 msgr2=0x7f59380cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:59.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.495+0000 7f593e62d640 1 -- 172.21.15.76:0/2625806384 shutdown_connections 2023-11-09T03:45:59.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.495+0000 7f593e62d640 1 -- 172.21.15.76:0/2625806384 wait complete. 2023-11-09T03:45:59.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.496+0000 7f593e62d640 1 Processor -- start 2023-11-09T03:45:59.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.496+0000 7f593e62d640 1 -- start start 2023-11-09T03:45:59.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.497+0000 7f593e62d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5938155210 0x7f59381f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:59.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.497+0000 7f593e62d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59381555f0 0x7f59381f69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:59.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.497+0000 7f593e62d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5938154840 0x7f59380b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:59.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.497+0000 7f593e62d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59381f6230 con 0x7f5938154840 2023-11-09T03:45:59.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.497+0000 7f593e62d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59381f3440 con 0x7f5938155210 2023-11-09T03:45:59.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.497+0000 7f593e62d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59381f3720 con 0x7f59381555f0 2023-11-09T03:45:59.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.497+0000 7f593cbca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5938154840 0x7f59380b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:59.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.497+0000 7f593cbca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5938154840 0x7f59380b77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36714/0 (socket says 172.21.15.76:36714) 2023-11-09T03:45:59.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.497+0000 7f593cbca640 1 -- 172.21.15.76:0/628964677 learned_addr learned my addr 172.21.15.76:0/628964677 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:59.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.497+0000 7f59377fe640 1 --2- 172.21.15.76:0/628964677 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59381555f0 0x7f59381f69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:59.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.497+0000 7f5937fff640 1 --2- 172.21.15.76:0/628964677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5938155210 0x7f59381f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:59.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.497+0000 7f593cbca640 1 -- 172.21.15.76:0/628964677 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59381555f0 msgr2=0x7f59381f69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:59.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.497+0000 7f593cbca640 1 --2- 172.21.15.76:0/628964677 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59381555f0 0x7f59381f69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.497+0000 7f593cbca640 1 -- 172.21.15.76:0/628964677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5938155210 msgr2=0x7f59381f2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:59.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.497+0000 7f593cbca640 1 --2- 172.21.15.76:0/628964677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5938155210 0x7f59381f2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.497+0000 7f593cbca640 1 -- 172.21.15.76:0/628964677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59381f64a0 con 0x7f5938154840 2023-11-09T03:45:59.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.497+0000 7f593cbca640 1 --2- 172.21.15.76:0/628964677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5938154840 0x7f59380b77a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5930036040 tx=0x7f5930009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:59.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.498+0000 7f5934ff9640 1 -- 172.21.15.76:0/628964677 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5930045070 con 0x7f5938154840 2023-11-09T03:45:59.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.498+0000 7f5934ff9640 1 -- 172.21.15.76:0/628964677 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5930034af0 con 0x7f5938154840 2023-11-09T03:45:59.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.498+0000 7f593e62d640 1 -- 172.21.15.76:0/628964677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59381f30e0 con 0x7f5938154840 2023-11-09T03:45:59.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.498+0000 7f5934ff9640 1 -- 172.21.15.76:0/628964677 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5930034e80 con 0x7f5938154840 2023-11-09T03:45:59.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.498+0000 7f593e62d640 1 -- 172.21.15.76:0/628964677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59381f6ee0 con 0x7f5938154840 2023-11-09T03:45:59.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.499+0000 7f593e62d640 1 -- 172.21.15.76:0/628964677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59381f6630 con 0x7f5938154840 2023-11-09T03:45:59.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.499+0000 7f5934ff9640 1 -- 172.21.15.76:0/628964677 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5930040470 con 0x7f5938154840 2023-11-09T03:45:59.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.500+0000 7f5934ff9640 1 --2- 172.21.15.76:0/628964677 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f592406ae70 0x7f592406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:59.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.500+0000 7f5934ff9640 1 -- 172.21.15.76:0/628964677 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f59300abe20 con 0x7f5938154840 2023-11-09T03:45:59.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.500+0000 7f5937fff640 1 --2- 172.21.15.76:0/628964677 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f592406ae70 0x7f592406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:59.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.500+0000 7f5937fff640 1 --2- 172.21.15.76:0/628964677 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f592406ae70 0x7f592406d330 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5928009bf0 tx=0x7f5928008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:59.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.504+0000 7f5934ff9640 1 -- 172.21.15.76:0/628964677 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5930076c70 con 0x7f5938154840 2023-11-09T03:45:59.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.708+0000 7f593e62d640 1 -- 172.21.15.76:0/628964677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59381f6630 con 0x7f5938154840 2023-11-09T03:45:59.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.708+0000 7f5934ff9640 1 -- 172.21.15.76:0/628964677 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4915 (secure 0 0 0) 0x7f5930009de0 con 0x7f5938154840 2023-11-09T03:45:59.708 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:45:59.708 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2686298435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2686298435},{"type":"v1","addr":"172.21.15.76:6833","nonce":2686298435}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2132717123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2132717123},{"type":"v1","addr":"172.21.15.76:6835","nonce":2132717123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4281,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2369499936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2369499936},{"type":"v1","addr":"172.21.15.113:6837","nonce":2369499936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:45:30.784155+0000","modified":"2023-11-09T03:45:47.804242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/3293223710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3293223710},{"type":"v1","addr":"172.21.15.113:6836","nonce":3293223710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-09T03:45:59.708 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 9 2023-11-09T03:45:59.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.710+0000 7f593e62d640 1 -- 172.21.15.76:0/628964677 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f592406ae70 msgr2=0x7f592406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:59.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.710+0000 7f593e62d640 1 --2- 172.21.15.76:0/628964677 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f592406ae70 0x7f592406d330 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5928009bf0 tx=0x7f5928008040 comp rx=0 tx=0).stop 2023-11-09T03:45:59.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.710+0000 7f593e62d640 1 -- 172.21.15.76:0/628964677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5938154840 msgr2=0x7f59380b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:59.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.710+0000 7f593e62d640 1 --2- 172.21.15.76:0/628964677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5938154840 0x7f59380b77a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5930036040 tx=0x7f5930009750 comp rx=0 tx=0).stop 2023-11-09T03:45:59.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.710+0000 7f593e62d640 1 -- 172.21.15.76:0/628964677 shutdown_connections 2023-11-09T03:45:59.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.710+0000 7f593e62d640 1 --2- 172.21.15.76:0/628964677 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f592406ae70 0x7f592406d330 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.710+0000 7f593e62d640 1 --2- 172.21.15.76:0/628964677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5938154840 0x7f59380b77a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.710+0000 7f593e62d640 1 --2- 172.21.15.76:0/628964677 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59381555f0 0x7f59381f69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.710+0000 7f593e62d640 1 --2- 172.21.15.76:0/628964677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5938155210 0x7f59381f2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.710+0000 7f593e62d640 1 -- 172.21.15.76:0/628964677 >> 172.21.15.76:0/628964677 conn(0x7f5938071f30 msgr2=0x7f59380c9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:59.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.711+0000 7f593e62d640 1 -- 172.21.15.76:0/628964677 shutdown_connections 2023-11-09T03:45:59.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.711+0000 7f593e62d640 1 -- 172.21.15.76:0/628964677 wait complete. 2023-11-09T03:45:59.720 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:45:59.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.821+0000 7fbd9130f640 1 Processor -- start 2023-11-09T03:45:59.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.821+0000 7fbd9130f640 1 -- start start 2023-11-09T03:45:59.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.822+0000 7fbd9130f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbd8c15ad80 0x7fbd8c06c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:59.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.822+0000 7fbd9130f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbd8c0b42c0 0x7fbd8c0c9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:59.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.822+0000 7fbd9130f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd8c06cd00 con 0x7fbd8c0c9c40 2023-11-09T03:45:59.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.822+0000 7fbd9130f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd8c0ca020 con 0x7fbd8c0b42c0 2023-11-09T03:45:59.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.822+0000 7fbd9130f640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd8c06cfa0 con 0x7fbd8c15ad80 2023-11-09T03:45:59.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.822+0000 7fbd8a7fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbd8c0b42c0 0x7fbd8c0c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:59.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.822+0000 7fbd8a7fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbd8c0b42c0 0x7fbd8c0c9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36730/0 (socket says 172.21.15.76:36730) 2023-11-09T03:45:59.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.822+0000 7fbd8a7fc640 1 -- 172.21.15.76:0/1961905488 learned_addr learned my addr 172.21.15.76:0/1961905488 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:59.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.823+0000 7fbd8affd640 1 --2- 172.21.15.76:0/1961905488 >> v2:172.21.15.113:3300/0 conn(0x7fbd8c15ad80 0x7fbd8c06c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:59.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.823+0000 7fbd8a7fc640 1 -- 172.21.15.76:0/1961905488 >> v2:172.21.15.113:3300/0 conn(0x7fbd8c15ad80 msgr2=0x7fbd8c06c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:59.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.823+0000 7fbd8a7fc640 1 --2- 172.21.15.76:0/1961905488 >> v2:172.21.15.113:3300/0 conn(0x7fbd8c15ad80 0x7fbd8c06c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.823+0000 7fbd8a7fc640 1 -- 172.21.15.76:0/1961905488 >> v1:172.21.15.113:6789/0 conn(0x7fbd8c0c9c40 legacy=0x7fbd8c070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:45:59.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.823+0000 7fbd8a7fc640 1 -- 172.21.15.76:0/1961905488 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8c06d280 con 0x7fbd8c0b42c0 2023-11-09T03:45:59.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.823+0000 7fbd8a7fc640 1 --2- 172.21.15.76:0/1961905488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd8c0b42c0 0x7fbd8c0c9700 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0c8e50 tx=0x7fbd8002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98535edab1a5e419 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:59.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.823+0000 7fbd89ffb640 1 -- 172.21.15.76:0/1961905488 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd80030840 con 0x7fbd8c0b42c0 2023-11-09T03:45:59.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.824+0000 7fbd89ffb640 1 -- 172.21.15.76:0/1961905488 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd800309a0 con 0x7fbd8c0b42c0 2023-11-09T03:45:59.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.824+0000 7fbd89ffb640 1 -- 172.21.15.76:0/1961905488 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd80030d30 con 0x7fbd8c0b42c0 2023-11-09T03:45:59.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.824+0000 7fbd9130f640 1 -- 172.21.15.76:0/1961905488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd8c0b42c0 msgr2=0x7fbd8c0c9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:59.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.824+0000 7fbd9130f640 1 --2- 172.21.15.76:0/1961905488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd8c0b42c0 0x7fbd8c0c9700 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0c8e50 tx=0x7fbd8002b980 comp rx=0 tx=0).stop 2023-11-09T03:45:59.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.824+0000 7fbd9130f640 1 -- 172.21.15.76:0/1961905488 shutdown_connections 2023-11-09T03:45:59.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.824+0000 7fbd9130f640 1 --2- 172.21.15.76:0/1961905488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd8c0b42c0 0x7fbd8c0c9700 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.824+0000 7fbd9130f640 1 --2- 172.21.15.76:0/1961905488 >> v2:172.21.15.113:3300/0 conn(0x7fbd8c15ad80 0x7fbd8c06c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.824+0000 7fbd9130f640 1 -- 172.21.15.76:0/1961905488 >> 172.21.15.76:0/1961905488 conn(0x7fbd8c0ce950 msgr2=0x7fbd8c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:45:59.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.824+0000 7fbd9130f640 1 -- 172.21.15.76:0/1961905488 shutdown_connections 2023-11-09T03:45:59.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.824+0000 7fbd9130f640 1 -- 172.21.15.76:0/1961905488 wait complete. 2023-11-09T03:45:59.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.825+0000 7fbd9130f640 1 Processor -- start 2023-11-09T03:45:59.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.825+0000 7fbd9130f640 1 -- start start 2023-11-09T03:45:59.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.826+0000 7fbd9130f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd8c0b42c0 0x7fbd8c151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:59.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.826+0000 7fbd9130f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd8c0c9c40 0x7fbd8c0cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:59.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.826+0000 7fbd9130f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd8c15ad80 0x7fbd8c0aca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:59.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.826+0000 7fbd9130f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c0aba10 con 0x7fbd8c0b42c0 2023-11-09T03:45:59.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.826+0000 7fbd9130f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c0abcb0 con 0x7fbd8c0c9c40 2023-11-09T03:45:59.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.826+0000 7fbd9130f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c0abf90 con 0x7fbd8c15ad80 2023-11-09T03:45:59.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.826+0000 7fbd8affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd8c0b42c0 0x7fbd8c151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:59.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.826+0000 7fbd8affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd8c0b42c0 0x7fbd8c151a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36742/0 (socket says 172.21.15.76:36742) 2023-11-09T03:45:59.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.826+0000 7fbd8affd640 1 -- 172.21.15.76:0/2286609130 learned_addr learned my addr 172.21.15.76:0/2286609130 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:45:59.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.826+0000 7fbd8a7fc640 1 --2- 172.21.15.76:0/2286609130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd8c0c9c40 0x7fbd8c0cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:59.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.826+0000 7fbd8b7fe640 1 --2- 172.21.15.76:0/2286609130 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd8c15ad80 0x7fbd8c0aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:59.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.826+0000 7fbd8affd640 1 -- 172.21.15.76:0/2286609130 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd8c15ad80 msgr2=0x7fbd8c0aca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:59.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.826+0000 7fbd8affd640 1 --2- 172.21.15.76:0/2286609130 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd8c15ad80 0x7fbd8c0aca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.826+0000 7fbd8affd640 1 -- 172.21.15.76:0/2286609130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd8c0c9c40 msgr2=0x7fbd8c0cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:45:59.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.826+0000 7fbd8affd640 1 --2- 172.21.15.76:0/2286609130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd8c0c9c40 0x7fbd8c0cdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:45:59.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.826+0000 7fbd8affd640 1 -- 172.21.15.76:0/2286609130 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd800094d0 con 0x7fbd8c0b42c0 2023-11-09T03:45:59.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.827+0000 7fbd8affd640 1 --2- 172.21.15.76:0/2286609130 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd8c0b42c0 0x7fbd8c151a60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c007e40 tx=0x7fbd7c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:59.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.827+0000 7fbd737fe640 1 -- 172.21.15.76:0/2286609130 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c014620 con 0x7fbd8c0b42c0 2023-11-09T03:45:59.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.827+0000 7fbd737fe640 1 -- 172.21.15.76:0/2286609130 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7c00ee90 con 0x7fbd8c0b42c0 2023-11-09T03:45:59.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.827+0000 7fbd9130f640 1 -- 172.21.15.76:0/2286609130 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd8c0ac270 con 0x7fbd8c0b42c0 2023-11-09T03:45:59.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.827+0000 7fbd737fe640 1 -- 172.21.15.76:0/2286609130 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c0149c0 con 0x7fbd8c0b42c0 2023-11-09T03:45:59.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.827+0000 7fbd9130f640 1 -- 172.21.15.76:0/2286609130 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd8c152310 con 0x7fbd8c0b42c0 2023-11-09T03:45:59.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.828+0000 7fbd9130f640 1 -- 172.21.15.76:0/2286609130 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd8c074290 con 0x7fbd8c0b42c0 2023-11-09T03:45:59.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.828+0000 7fbd737fe640 1 -- 172.21.15.76:0/2286609130 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbd7c014b20 con 0x7fbd8c0b42c0 2023-11-09T03:45:59.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.828+0000 7fbd737fe640 1 --2- 172.21.15.76:0/2286609130 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbd7806ae70 0x7fbd7806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:45:59.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.829+0000 7fbd737fe640 1 -- 172.21.15.76:0/2286609130 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbd7c08a070 con 0x7fbd8c0b42c0 2023-11-09T03:45:59.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.829+0000 7fbd8a7fc640 1 --2- 172.21.15.76:0/2286609130 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbd7806ae70 0x7fbd7806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:45:59.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.829+0000 7fbd8a7fc640 1 --2- 172.21.15.76:0/2286609130 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbd7806ae70 0x7fbd7806d330 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbd80036000 tx=0x7fbd80008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:45:59.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:45:59.833+0000 7fbd737fe640 1 -- 172.21.15.76:0/2286609130 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbd7c054f70 con 0x7fbd8c0b42c0 2023-11-09T03:46:00.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.000+0000 7fbd9130f640 1 -- 172.21.15.76:0/2286609130 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd8c0b4880 con 0x7fbd8c0b42c0 2023-11-09T03:46:00.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.001+0000 7fbd737fe640 1 -- 172.21.15.76:0/2286609130 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18334 (secure 0 0 0) 0x7fbd7c058590 con 0x7fbd8c0b42c0 2023-11-09T03:46:00.002 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:00.002 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":25,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:45:35.781752+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:46:00.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.003+0000 7fbd9130f640 1 -- 172.21.15.76:0/2286609130 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbd7806ae70 msgr2=0x7fbd7806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:00.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.003+0000 7fbd9130f640 1 --2- 172.21.15.76:0/2286609130 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbd7806ae70 0x7fbd7806d330 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbd80036000 tx=0x7fbd80008030 comp rx=0 tx=0).stop 2023-11-09T03:46:00.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.003+0000 7fbd9130f640 1 -- 172.21.15.76:0/2286609130 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd8c0b42c0 msgr2=0x7fbd8c151a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:00.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.003+0000 7fbd9130f640 1 --2- 172.21.15.76:0/2286609130 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd8c0b42c0 0x7fbd8c151a60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c007e40 tx=0x7fbd7c00e4b0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.003+0000 7fbd9130f640 1 -- 172.21.15.76:0/2286609130 shutdown_connections 2023-11-09T03:46:00.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.003+0000 7fbd9130f640 1 --2- 172.21.15.76:0/2286609130 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbd7806ae70 0x7fbd7806d330 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.003+0000 7fbd9130f640 1 --2- 172.21.15.76:0/2286609130 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd8c15ad80 0x7fbd8c0aca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.003+0000 7fbd9130f640 1 --2- 172.21.15.76:0/2286609130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd8c0c9c40 0x7fbd8c0cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.003+0000 7fbd9130f640 1 --2- 172.21.15.76:0/2286609130 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd8c0b42c0 0x7fbd8c151a60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.003+0000 7fbd9130f640 1 -- 172.21.15.76:0/2286609130 >> 172.21.15.76:0/2286609130 conn(0x7fbd8c0ce950 msgr2=0x7fbd8c0b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:00.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.003+0000 7fbd9130f640 1 -- 172.21.15.76:0/2286609130 shutdown_connections 2023-11-09T03:46:00.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.003+0000 7fbd9130f640 1 -- 172.21.15.76:0/2286609130 wait complete. 2023-11-09T03:46:00.015 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:46:00.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.114+0000 7f3fc5044640 1 Processor -- start 2023-11-09T03:46:00.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.114+0000 7f3fc5044640 1 -- start start 2023-11-09T03:46:00.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.115+0000 7f3fc5044640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3fc0157710 0x7f3fc0167b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:00.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.115+0000 7f3fc5044640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc015e000 con 0x7f3fc00cde60 2023-11-09T03:46:00.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.115+0000 7f3fc5044640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc015e2e0 con 0x7f3fc00ce240 2023-11-09T03:46:00.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.115+0000 7f3fc5044640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc015e5c0 con 0x7f3fc0157710 2023-11-09T03:46:00.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.115+0000 7f3fbed9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f3fc00cde60 0x7f3fc015d560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33766/0 (socket says 172.21.15.76:33766) 2023-11-09T03:46:00.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.115+0000 7f3fbed9d640 1 -- 172.21.15.76:0/315543493 learned_addr learned my addr 172.21.15.76:0/315543493 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:00.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.115+0000 7f3fbf59e640 1 --2- 172.21.15.76:0/315543493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3fc0157710 0x7f3fc0167b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:00.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.115+0000 7f3fbdd9b640 1 -- 172.21.15.76:0/315543493 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 810405524 0 0) 0x7f3fc015e000 con 0x7f3fc00cde60 2023-11-09T03:46:00.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.116+0000 7f3fbdd9b640 1 -- 172.21.15.76:0/315543493 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fa8003520 con 0x7f3fc00cde60 2023-11-09T03:46:00.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.116+0000 7f3fbdd9b640 1 -- 172.21.15.76:0/315543493 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2687316013 0 0) 0x7f3fc015e2e0 con 0x7f3fc00ce240 2023-11-09T03:46:00.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.116+0000 7f3fbdd9b640 1 -- 172.21.15.76:0/315543493 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fc015e000 con 0x7f3fc00ce240 2023-11-09T03:46:00.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.116+0000 7f3fbdd9b640 1 -- 172.21.15.76:0/315543493 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 353554791 0 0) 0x7f3fa8003520 con 0x7f3fc00cde60 2023-11-09T03:46:00.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.116+0000 7f3fbf59e640 1 -- 172.21.15.76:0/315543493 >> v1:172.21.15.113:6789/0 conn(0x7f3fc00ce240 legacy=0x7f3fc0157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:00.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.116+0000 7f3fbf59e640 1 -- 172.21.15.76:0/315543493 >> v1:172.21.15.76:6789/0 conn(0x7f3fc00cde60 legacy=0x7f3fc015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:00.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.116+0000 7f3fbf59e640 1 -- 172.21.15.76:0/315543493 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc00d1120 con 0x7f3fc0157710 2023-11-09T03:46:00.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.117+0000 7f3fbf59e640 1 --2- 172.21.15.76:0/315543493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3fc0157710 0x7f3fc0167b90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3fb80097d0 tx=0x7f3fb802b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9761c116c84d32b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:00.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.117+0000 7f3fbdd9b640 1 -- 172.21.15.76:0/315543493 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb00019f0 con 0x7f3fc0157710 2023-11-09T03:46:00.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.117+0000 7f3fbdd9b640 1 -- 172.21.15.76:0/315543493 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb802e050 con 0x7f3fc0157710 2023-11-09T03:46:00.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.117+0000 7f3fbdd9b640 1 -- 172.21.15.76:0/315543493 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8030730 con 0x7f3fc0157710 2023-11-09T03:46:00.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.117+0000 7f3fc5044640 1 -- 172.21.15.76:0/315543493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3fc0157710 msgr2=0x7f3fc0167b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:00.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.117+0000 7f3fc5044640 1 --2- 172.21.15.76:0/315543493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3fc0157710 0x7f3fc0167b90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3fb80097d0 tx=0x7f3fb802b800 comp rx=0 tx=0).stop 2023-11-09T03:46:00.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.117+0000 7f3fc5044640 1 -- 172.21.15.76:0/315543493 shutdown_connections 2023-11-09T03:46:00.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.117+0000 7f3fc5044640 1 --2- 172.21.15.76:0/315543493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3fc0157710 0x7f3fc0167b90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.117+0000 7f3fc5044640 1 -- 172.21.15.76:0/315543493 >> 172.21.15.76:0/315543493 conn(0x7f3fc0071f30 msgr2=0x7f3fc00cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:00.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.117+0000 7f3fc5044640 1 -- 172.21.15.76:0/315543493 shutdown_connections 2023-11-09T03:46:00.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.117+0000 7f3fc5044640 1 -- 172.21.15.76:0/315543493 wait complete. 2023-11-09T03:46:00.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.119+0000 7f3fc5044640 1 Processor -- start 2023-11-09T03:46:00.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.119+0000 7f3fc5044640 1 -- start start 2023-11-09T03:46:00.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.120+0000 7f3fc5044640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fc00cde60 0x7f3fc015b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:00.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.120+0000 7f3fc5044640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fc00ce240 0x7f3fc00ca790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:00.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.120+0000 7f3fc5044640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3fc0157710 0x7f3fc00cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:00.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.120+0000 7f3fc5044640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc0157c20 con 0x7f3fc00ce240 2023-11-09T03:46:00.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.120+0000 7f3fc5044640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc01535e0 con 0x7f3fc00cde60 2023-11-09T03:46:00.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.120+0000 7f3fc5044640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc0153090 con 0x7f3fc0157710 2023-11-09T03:46:00.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.120+0000 7f3fbe59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fc00ce240 0x7f3fc00ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:00.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.120+0000 7f3fbe59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fc00ce240 0x7f3fc00ca790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36758/0 (socket says 172.21.15.76:36758) 2023-11-09T03:46:00.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.120+0000 7f3fbe59c640 1 -- 172.21.15.76:0/3066596451 learned_addr learned my addr 172.21.15.76:0/3066596451 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:00.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.120+0000 7f3fbed9d640 1 --2- 172.21.15.76:0/3066596451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fc00cde60 0x7f3fc015b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:00.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.120+0000 7f3fbf59e640 1 --2- 172.21.15.76:0/3066596451 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3fc0157710 0x7f3fc00cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:00.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.120+0000 7f3fbe59c640 1 -- 172.21.15.76:0/3066596451 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3fc0157710 msgr2=0x7f3fc00cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:00.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.120+0000 7f3fbe59c640 1 --2- 172.21.15.76:0/3066596451 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3fc0157710 0x7f3fc00cacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.120+0000 7f3fbe59c640 1 -- 172.21.15.76:0/3066596451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fc00cde60 msgr2=0x7f3fc015b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:00.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.120+0000 7f3fbe59c640 1 --2- 172.21.15.76:0/3066596451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fc00cde60 0x7f3fc015b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.120+0000 7f3fbe59c640 1 -- 172.21.15.76:0/3066596451 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb8009400 con 0x7f3fc00ce240 2023-11-09T03:46:00.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.120+0000 7f3fbe59c640 1 --2- 172.21.15.76:0/3066596451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fc00ce240 0x7f3fc00ca790 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400c4c0 tx=0x7f3fb400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:00.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.120+0000 7f3fa77fe640 1 -- 172.21.15.76:0/3066596451 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb40052d0 con 0x7f3fc00ce240 2023-11-09T03:46:00.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.120+0000 7f3fa77fe640 1 -- 172.21.15.76:0/3066596451 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb4005430 con 0x7f3fc00ce240 2023-11-09T03:46:00.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.120+0000 7f3fc5044640 1 -- 172.21.15.76:0/3066596451 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc015c4c0 con 0x7f3fc00ce240 2023-11-09T03:46:00.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.120+0000 7f3fa77fe640 1 -- 172.21.15.76:0/3066596451 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4012870 con 0x7f3fc00ce240 2023-11-09T03:46:00.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.120+0000 7f3fc5044640 1 -- 172.21.15.76:0/3066596451 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc015c1e0 con 0x7f3fc00ce240 2023-11-09T03:46:00.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.121+0000 7f3fc5044640 1 -- 172.21.15.76:0/3066596451 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc0074290 con 0x7f3fc00ce240 2023-11-09T03:46:00.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.122+0000 7f3fa77fe640 1 -- 172.21.15.76:0/3066596451 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3fb40129d0 con 0x7f3fc00ce240 2023-11-09T03:46:00.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.122+0000 7f3fa77fe640 1 --2- 172.21.15.76:0/3066596451 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3fac06ae70 0x7f3fac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:00.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.122+0000 7f3fa77fe640 1 -- 172.21.15.76:0/3066596451 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3fb4089390 con 0x7f3fc00ce240 2023-11-09T03:46:00.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.122+0000 7f3fbed9d640 1 --2- 172.21.15.76:0/3066596451 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3fac06ae70 0x7f3fac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:00.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.123+0000 7f3fbed9d640 1 --2- 172.21.15.76:0/3066596451 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3fac06ae70 0x7f3fac06d330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3fb00012e0 tx=0x7f3fb0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:00.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.126+0000 7f3fa77fe640 1 -- 172.21.15.76:0/3066596451 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3fb4054200 con 0x7f3fc00ce240 2023-11-09T03:46:00.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.325+0000 7f3fc5044640 1 -- 172.21.15.76:0/3066596451 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3fc015d6e0 con 0x7f3fc00ce240 2023-11-09T03:46:00.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.325+0000 7f3fa77fe640 1 -- 172.21.15.76:0/3066596451 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4915 (secure 0 0 0) 0x7f3fb4057820 con 0x7f3fc00ce240 2023-11-09T03:46:00.326 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:00.326 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2686298435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2686298435},{"type":"v1","addr":"172.21.15.76:6833","nonce":2686298435}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2132717123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2132717123},{"type":"v1","addr":"172.21.15.76:6835","nonce":2132717123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4281,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2369499936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2369499936},{"type":"v1","addr":"172.21.15.113:6837","nonce":2369499936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:45:30.784155+0000","modified":"2023-11-09T03:45:47.804242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/3293223710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3293223710},{"type":"v1","addr":"172.21.15.113:6836","nonce":3293223710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-09T03:46:00.327 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 9 2023-11-09T03:46:00.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.327+0000 7f3fc5044640 1 -- 172.21.15.76:0/3066596451 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3fac06ae70 msgr2=0x7f3fac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:00.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.327+0000 7f3fc5044640 1 --2- 172.21.15.76:0/3066596451 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3fac06ae70 0x7f3fac06d330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3fb00012e0 tx=0x7f3fb0009040 comp rx=0 tx=0).stop 2023-11-09T03:46:00.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.327+0000 7f3fc5044640 1 -- 172.21.15.76:0/3066596451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fc00ce240 msgr2=0x7f3fc00ca790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:00.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.327+0000 7f3fc5044640 1 --2- 172.21.15.76:0/3066596451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fc00ce240 0x7f3fc00ca790 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400c4c0 tx=0x7f3fb400c980 comp rx=0 tx=0).stop 2023-11-09T03:46:00.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.327+0000 7f3fc5044640 1 -- 172.21.15.76:0/3066596451 shutdown_connections 2023-11-09T03:46:00.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.327+0000 7f3fc5044640 1 --2- 172.21.15.76:0/3066596451 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3fac06ae70 0x7f3fac06d330 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.327+0000 7f3fc5044640 1 --2- 172.21.15.76:0/3066596451 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3fc0157710 0x7f3fc00cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.327+0000 7f3fc5044640 1 --2- 172.21.15.76:0/3066596451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fc00ce240 0x7f3fc00ca790 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.327+0000 7f3fc5044640 1 --2- 172.21.15.76:0/3066596451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fc00cde60 0x7f3fc015b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.327+0000 7f3fc5044640 1 -- 172.21.15.76:0/3066596451 >> 172.21.15.76:0/3066596451 conn(0x7f3fc0071f30 msgr2=0x7f3fc00b5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:00.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.327+0000 7f3fc5044640 1 -- 172.21.15.76:0/3066596451 shutdown_connections 2023-11-09T03:46:00.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.327+0000 7f3fc5044640 1 -- 172.21.15.76:0/3066596451 wait complete. 2023-11-09T03:46:00.378 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:46:00.381 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:46:00.740 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-09T03:45:30.784155+0000', 'modified': '2023-11-09T03:45:47.804242+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4291}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4291': {'gid': 4291, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.113:6836/3293223710', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6834', 'nonce': 3293223710}, {'type': 'v1', 'addr': '172.21.15.113:6836', 'nonce': 3293223710}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-09T03:46:00.740 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-09T03:46:00.741 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-09T03:46:00.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.842+0000 7f25bdf4c640 1 Processor -- start 2023-11-09T03:46:00.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.842+0000 7f25bdf4c640 1 -- start start 2023-11-09T03:46:00.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.843+0000 7f25bdf4c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f25b8153e60 0x7f25b815dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:00.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.843+0000 7f25bdf4c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25b80cd6d0 0x7f25b8161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:00.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.843+0000 7f25bdf4c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f25b80b4880 0x7f25b80b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:00.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.843+0000 7f25bdf4c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b80cde60 con 0x7f25b8153e60 2023-11-09T03:46:00.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.843+0000 7f25bdf4c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b8160f90 con 0x7f25b80b4880 2023-11-09T03:46:00.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.843+0000 7f25bdf4c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b80b51e0 con 0x7f25b80cd6d0 2023-11-09T03:46:00.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.843+0000 7f25b77fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f25b8153e60 0x7f25b815dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:00.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.843+0000 7f25b77fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f25b8153e60 0x7f25b815dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36766/0 (socket says 172.21.15.76:36766) 2023-11-09T03:46:00.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.843+0000 7f25b77fe640 1 -- 172.21.15.76:0/3550317410 learned_addr learned my addr 172.21.15.76:0/3550317410 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:00.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.844+0000 7f25b6ffd640 1 --2- 172.21.15.76:0/3550317410 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25b80cd6d0 0x7f25b8161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:00.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.844+0000 7f25b7fff640 1 --2- 172.21.15.76:0/3550317410 >> v2:172.21.15.113:3300/0 conn(0x7f25b80b4880 0x7f25b80b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:00.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.844+0000 7f25b77fe640 1 -- 172.21.15.76:0/3550317410 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25b80cd6d0 msgr2=0x7f25b8161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:00.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.844+0000 7f25b77fe640 1 --2- 172.21.15.76:0/3550317410 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25b80cd6d0 0x7f25b8161420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.844+0000 7f25b77fe640 1 -- 172.21.15.76:0/3550317410 >> v2:172.21.15.113:3300/0 conn(0x7f25b80b4880 msgr2=0x7f25b80b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:00.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.844+0000 7f25b77fe640 1 --2- 172.21.15.76:0/3550317410 >> v2:172.21.15.113:3300/0 conn(0x7f25b80b4880 0x7f25b80b4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.844+0000 7f25b77fe640 1 -- 172.21.15.76:0/3550317410 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b815e400 con 0x7f25b8153e60 2023-11-09T03:46:00.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.844+0000 7f25b77fe640 1 --2- 172.21.15.76:0/3550317410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25b8153e60 0x7f25b815dec0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f25b80c9e50 tx=0x7f25a802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6764971461335 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:00.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.845+0000 7f25b67fc640 1 -- 172.21.15.76:0/3550317410 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a8031870 con 0x7f25b8153e60 2023-11-09T03:46:00.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.845+0000 7f25b67fc640 1 -- 172.21.15.76:0/3550317410 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a80319d0 con 0x7f25b8153e60 2023-11-09T03:46:00.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.845+0000 7f25b67fc640 1 -- 172.21.15.76:0/3550317410 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a8031d60 con 0x7f25b8153e60 2023-11-09T03:46:00.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.845+0000 7f25bdf4c640 1 -- 172.21.15.76:0/3550317410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25b8153e60 msgr2=0x7f25b815dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:00.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.845+0000 7f25bdf4c640 1 --2- 172.21.15.76:0/3550317410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25b8153e60 0x7f25b815dec0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f25b80c9e50 tx=0x7f25a802b910 comp rx=0 tx=0).stop 2023-11-09T03:46:00.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.845+0000 7f25bdf4c640 1 -- 172.21.15.76:0/3550317410 shutdown_connections 2023-11-09T03:46:00.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.845+0000 7f25bdf4c640 1 --2- 172.21.15.76:0/3550317410 >> v2:172.21.15.113:3300/0 conn(0x7f25b80b4880 0x7f25b80b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.845+0000 7f25bdf4c640 1 --2- 172.21.15.76:0/3550317410 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25b80cd6d0 0x7f25b8161420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.845+0000 7f25bdf4c640 1 --2- 172.21.15.76:0/3550317410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25b8153e60 0x7f25b815dec0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.845+0000 7f25bdf4c640 1 -- 172.21.15.76:0/3550317410 >> 172.21.15.76:0/3550317410 conn(0x7f25b80cf880 msgr2=0x7f25b80cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:00.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.845+0000 7f25bdf4c640 1 -- 172.21.15.76:0/3550317410 shutdown_connections 2023-11-09T03:46:00.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.845+0000 7f25bdf4c640 1 -- 172.21.15.76:0/3550317410 wait complete. 2023-11-09T03:46:00.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.846+0000 7f25bdf4c640 1 Processor -- start 2023-11-09T03:46:00.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.847+0000 7f25bdf4c640 1 -- start start 2023-11-09T03:46:00.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.847+0000 7f25bdf4c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25b80b4880 0x7f25b80c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:00.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.847+0000 7f25bdf4c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25b80cd6d0 0x7f25b80c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:00.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.847+0000 7f25bdf4c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25b8153e60 0x7f25b8155890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:00.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.847+0000 7f25bdf4c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b80c8f30 con 0x7f25b8153e60 2023-11-09T03:46:00.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.847+0000 7f25bdf4c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b80ca7d0 con 0x7f25b80b4880 2023-11-09T03:46:00.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.847+0000 7f25bdf4c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b80caab0 con 0x7f25b80cd6d0 2023-11-09T03:46:00.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.847+0000 7f25b7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25b8153e60 0x7f25b8155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:00.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.847+0000 7f25b7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25b8153e60 0x7f25b8155890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36782/0 (socket says 172.21.15.76:36782) 2023-11-09T03:46:00.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.847+0000 7f25b7fff640 1 -- 172.21.15.76:0/356540994 learned_addr learned my addr 172.21.15.76:0/356540994 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:00.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.847+0000 7f25b77fe640 1 --2- 172.21.15.76:0/356540994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25b80b4880 0x7f25b80c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:00.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.848+0000 7f25b6ffd640 1 --2- 172.21.15.76:0/356540994 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25b80cd6d0 0x7f25b80c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:00.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.848+0000 7f25b7fff640 1 -- 172.21.15.76:0/356540994 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25b80cd6d0 msgr2=0x7f25b80c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:00.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.848+0000 7f25b7fff640 1 --2- 172.21.15.76:0/356540994 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25b80cd6d0 0x7f25b80c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.848+0000 7f25b7fff640 1 -- 172.21.15.76:0/356540994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25b80b4880 msgr2=0x7f25b80c89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:00.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.848+0000 7f25b7fff640 1 --2- 172.21.15.76:0/356540994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25b80b4880 0x7f25b80c89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.848+0000 7f25b7fff640 1 -- 172.21.15.76:0/356540994 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a80094d0 con 0x7f25b8153e60 2023-11-09T03:46:00.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.848+0000 7f25b7fff640 1 --2- 172.21.15.76:0/356540994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25b8153e60 0x7f25b8155890 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f25b000eed0 tx=0x7f25b000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:00.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.848+0000 7f259ffff640 1 -- 172.21.15.76:0/356540994 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b0010040 con 0x7f25b8153e60 2023-11-09T03:46:00.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.848+0000 7f259ffff640 1 -- 172.21.15.76:0/356540994 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b0014400 con 0x7f25b8153e60 2023-11-09T03:46:00.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.848+0000 7f25bdf4c640 1 -- 172.21.15.76:0/356540994 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b80cad90 con 0x7f25b8153e60 2023-11-09T03:46:00.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.848+0000 7f259ffff640 1 -- 172.21.15.76:0/356540994 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b0009ea0 con 0x7f25b8153e60 2023-11-09T03:46:00.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.848+0000 7f25bdf4c640 1 -- 172.21.15.76:0/356540994 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b80cb350 con 0x7f25b8153e60 2023-11-09T03:46:00.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.850+0000 7f259ffff640 1 -- 172.21.15.76:0/356540994 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f25b00147b0 con 0x7f25b8153e60 2023-11-09T03:46:00.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.850+0000 7f259ffff640 1 --2- 172.21.15.76:0/356540994 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f25a406ae70 0x7f25a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:00.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.850+0000 7f259ffff640 1 -- 172.21.15.76:0/356540994 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f25b008a200 con 0x7f25b8153e60 2023-11-09T03:46:00.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.850+0000 7f25b77fe640 1 --2- 172.21.15.76:0/356540994 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f25a406ae70 0x7f25a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:00.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.851+0000 7f25b77fe640 1 --2- 172.21.15.76:0/356540994 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f25a406ae70 0x7f25a406d330 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f25a8036000 tx=0x7f25a803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:00.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.851+0000 7f25c0bb4740 1 Processor -- start 2023-11-09T03:46:00.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.851+0000 7f25c0bb4740 1 -- start start 2023-11-09T03:46:00.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.852+0000 7f25c0bb4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be4809c850 0x55be4806f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:00.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.852+0000 7f25c0bb4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be4809cc30 0x55be4809f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:00.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.852+0000 7f25c0bb4740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be4806fc90 0x55be4809fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:00.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.852+0000 7f25c0bb4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be47f80740 con 0x55be4809c850 2023-11-09T03:46:00.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.852+0000 7f25c0bb4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be47fe28d0 con 0x55be4809cc30 2023-11-09T03:46:00.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.852+0000 7f25c0bb4740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be47fdffc0 con 0x55be4806fc90 2023-11-09T03:46:00.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.852+0000 7f25b77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be4809c850 0x55be4806f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:00.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.852+0000 7f25b77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be4809c850 0x55be4806f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36786/0 (socket says 172.21.15.76:36786) 2023-11-09T03:46:00.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.852+0000 7f25b77fe640 1 -- 172.21.15.76:0/2678013991 learned_addr learned my addr 172.21.15.76:0/2678013991 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:00.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.852+0000 7f25b7fff640 1 --2- 172.21.15.76:0/2678013991 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be4806fc90 0x55be4809fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:00.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.852+0000 7f25b6ffd640 1 --2- 172.21.15.76:0/2678013991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be4809cc30 0x55be4809f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:00.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.852+0000 7f25b77fe640 1 -- 172.21.15.76:0/2678013991 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be4806fc90 msgr2=0x55be4809fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:00.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.852+0000 7f25b77fe640 1 --2- 172.21.15.76:0/2678013991 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be4806fc90 0x55be4809fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.852+0000 7f25b77fe640 1 -- 172.21.15.76:0/2678013991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be4809cc30 msgr2=0x55be4809f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:00.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.852+0000 7f25b77fe640 1 --2- 172.21.15.76:0/2678013991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be4809cc30 0x55be4809f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.852+0000 7f25b77fe640 1 -- 172.21.15.76:0/2678013991 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be47f9eca0 con 0x55be4809c850 2023-11-09T03:46:00.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.852+0000 7f25b77fe640 1 --2- 172.21.15.76:0/2678013991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be4809c850 0x55be4806f750 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f25a8034040 tx=0x7f25a8030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3c309a54457dda server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:00.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.853+0000 7f259d7fa640 1 -- 172.21.15.76:0/2678013991 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a803aaf0 con 0x55be4809c850 2023-11-09T03:46:00.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.853+0000 7f259d7fa640 1 -- 172.21.15.76:0/2678013991 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a803ac50 con 0x55be4809c850 2023-11-09T03:46:00.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.853+0000 7f259d7fa640 1 -- 172.21.15.76:0/2678013991 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a804d410 con 0x55be4809c850 2023-11-09T03:46:00.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.853+0000 7f25c0bb4740 1 -- 172.21.15.76:0/2678013991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be4809c850 msgr2=0x55be4806f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:00.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.853+0000 7f25c0bb4740 1 --2- 172.21.15.76:0/2678013991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be4809c850 0x55be4806f750 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f25a8034040 tx=0x7f25a8030000 comp rx=0 tx=0).stop 2023-11-09T03:46:00.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.853+0000 7f25c0bb4740 1 -- 172.21.15.76:0/2678013991 shutdown_connections 2023-11-09T03:46:00.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.853+0000 7f25c0bb4740 1 --2- 172.21.15.76:0/2678013991 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be4806fc90 0x55be4809fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.853+0000 7f25c0bb4740 1 --2- 172.21.15.76:0/2678013991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be4809cc30 0x55be4809f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.853+0000 7f25c0bb4740 1 --2- 172.21.15.76:0/2678013991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be4809c850 0x55be4806f750 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.853+0000 7f25c0bb4740 1 -- 172.21.15.76:0/2678013991 >> 172.21.15.76:0/2678013991 conn(0x55be48071a60 msgr2=0x55be480ba5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:00.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.853+0000 7f25c0bb4740 1 -- 172.21.15.76:0/2678013991 shutdown_connections 2023-11-09T03:46:00.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.853+0000 7f25c0bb4740 1 -- 172.21.15.76:0/2678013991 wait complete. 2023-11-09T03:46:00.869 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.854+0000 7f25c0bb4740 1 -- start start 2023-11-09T03:46:00.869 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.854+0000 7f25c0bb4740 1 Processor -- start 2023-11-09T03:46:00.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.855+0000 7f25c0bb4740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be4809c850 0x55be480b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:00.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.855+0000 7f25c0bb4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be4809cc30 0x55be480b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:00.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.855+0000 7f25c0bb4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be480b4c10 0x55be480b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:00.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.855+0000 7f25c0bb4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be47fdb8b0 con 0x55be480b4c10 2023-11-09T03:46:00.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.855+0000 7f25c0bb4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be47f95c80 con 0x55be4809cc30 2023-11-09T03:46:00.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.855+0000 7f25c0bb4740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be47f95920 con 0x55be4809c850 2023-11-09T03:46:00.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.855+0000 7f25c0bb4740 10 client.0 _ll_register_callbacks cb 0x55be47cedac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:46:00.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.855+0000 7f25b7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be480b4c10 0x55be480b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:00.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.855+0000 7f25b7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be480b4c10 0x55be480b3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36788/0 (socket says 172.21.15.76:36788) 2023-11-09T03:46:00.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.855+0000 7f25b7fff640 1 -- 172.21.15.76:0/296151083 learned_addr learned my addr 172.21.15.76:0/296151083 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:00.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.855+0000 7f25b6ffd640 1 --2- 172.21.15.76:0/296151083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be4809cc30 0x55be480b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:00.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.855+0000 7f25b77fe640 1 --2- 172.21.15.76:0/296151083 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be4809c850 0x55be480b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:00.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.855+0000 7f25b7fff640 1 -- 172.21.15.76:0/296151083 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be4809c850 msgr2=0x55be480b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:00.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.855+0000 7f25b7fff640 1 --2- 172.21.15.76:0/296151083 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be4809c850 0x55be480b1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.855+0000 7f25b7fff640 1 -- 172.21.15.76:0/296151083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be4809cc30 msgr2=0x55be480b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:00.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.855+0000 7f25b7fff640 1 --2- 172.21.15.76:0/296151083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be4809cc30 0x55be480b35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.855+0000 7f25b7fff640 1 -- 172.21.15.76:0/296151083 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b815e400 con 0x55be480b4c10 2023-11-09T03:46:00.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.855+0000 7f25b7fff640 1 --2- 172.21.15.76:0/296151083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be480b4c10 0x55be480b3b30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f25b005f050 tx=0x7f25b00587b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f834b5f4385c86e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:00.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.855+0000 7f25967fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:46:00.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.855+0000 7f25967fc640 1 -- 172.21.15.76:0/296151083 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b0057190 con 0x55be480b4c10 2023-11-09T03:46:00.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.855+0000 7f25967fc640 1 -- 172.21.15.76:0/296151083 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f25b005db10 con 0x55be480b4c10 2023-11-09T03:46:00.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.855+0000 7f25c0bb4740 1 -- 172.21.15.76:0/296151083 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be47f87bf0 con 0x55be480b4c10 2023-11-09T03:46:00.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.855+0000 7f25967fc640 1 -- 172.21.15.76:0/296151083 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25b005dcd0 con 0x55be480b4c10 2023-11-09T03:46:00.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.855+0000 7f25c0bb4740 10 client.4902 fetch_fsmap learned FSMap version 9 2023-11-09T03:46:00.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.855+0000 7f25c0bb4740 1 -- 172.21.15.76:0/296151083 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55be47fd9f30 con 0x55be480b4c10 2023-11-09T03:46:00.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.856+0000 7f25967fc640 1 -- 172.21.15.76:0/296151083 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55be47f87bf0 con 0x55be480b4c10 2023-11-09T03:46:00.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.856+0000 7f25967fc640 1 -- 172.21.15.76:0/296151083 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f25b0024020 con 0x55be480b4c10 2023-11-09T03:46:00.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.856+0000 7f25c0bb4740 10 client.4902 fetch_fsmap finished waiting for FSMap version 9 2023-11-09T03:46:00.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.856+0000 7f25c0bb4740 10 client.4902 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-09T03:46:00.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.856+0000 7f25c0bb4740 20 client.4902 populate_metadata read hostname 'smithi076' 2023-11-09T03:46:00.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.856+0000 7f25c0bb4740 1 --2- 172.21.15.76:0/296151083 >> [v2:172.21.15.113:6834/3293223710,v1:172.21.15.113:6836/3293223710] conn(0x55be480a1620 0x55be480d8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:00.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.856+0000 7f25c0bb4740 4 client.4902 mds_command: new command op to 4291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:46:00.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.856+0000 7f25c0bb4740 1 -- 172.21.15.76:0/296151083 --> [v2:172.21.15.113:6834/3293223710,v1:172.21.15.113:6836/3293223710] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be47fd4890 con 0x55be480a1620 2023-11-09T03:46:00.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.856+0000 7f25b77fe640 1 --2- 172.21.15.76:0/296151083 >> [v2:172.21.15.113:6834/3293223710,v1:172.21.15.113:6836/3293223710] conn(0x55be480a1620 0x55be480d8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:00.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.856+0000 7f25967fc640 10 client.4902 ms_handle_connect on v2:172.21.15.113:6834/3293223710 2023-11-09T03:46:00.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.856+0000 7f25b77fe640 1 --2- 172.21.15.76:0/296151083 >> [v2:172.21.15.113:6834/3293223710,v1:172.21.15.113:6836/3293223710] conn(0x55be480a1620 0x55be480d8120 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1144e0df3d22ad1 server_cookie=8755eca816f505d in_seq=0 out_seq=0 2023-11-09T03:46:00.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.857+0000 7f25967fc640 1 -- 172.21.15.76:0/296151083 <== mds.0 v2:172.21.15.113:6834/3293223710 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55be47fd4890 con 0x55be480a1620 2023-11-09T03:46:00.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.857+0000 7f25967fc640 10 client.4902 handle_command_reply: tid=0 2023-11-09T03:46:00.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.857+0000 7f25c0bb4740 1 client.4902 shutdown 2023-11-09T03:46:00.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.857+0000 7f25c0bb4740 1 -- 172.21.15.76:0/296151083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be480b4c10 msgr2=0x55be480b3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:00.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.857+0000 7f25c0bb4740 1 --2- 172.21.15.76:0/296151083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be480b4c10 0x55be480b3b30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f25b005f050 tx=0x7f25b00587b0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.857+0000 7f25c0bb4740 1 -- 172.21.15.76:0/296151083 shutdown_connections 2023-11-09T03:46:00.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.857+0000 7f25c0bb4740 1 --2- 172.21.15.76:0/296151083 >> [v2:172.21.15.113:6834/3293223710,v1:172.21.15.113:6836/3293223710] conn(0x55be480a1620 0x55be480d8120 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.857+0000 7f25c0bb4740 1 --2- 172.21.15.76:0/296151083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be480b4c10 0x55be480b3b30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.857+0000 7f25c0bb4740 1 --2- 172.21.15.76:0/296151083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be4809cc30 0x55be480b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.857+0000 7f25c0bb4740 1 --2- 172.21.15.76:0/296151083 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be4809c850 0x55be480b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.857+0000 7f25967fc640 0 client.4902 ms_handle_reset on v2:172.21.15.113:6834/3293223710 2023-11-09T03:46:00.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.857+0000 7f25c0bb4740 1 -- 172.21.15.76:0/296151083 >> 172.21.15.76:0/296151083 conn(0x55be4806fc90 msgr2=0x55be480736d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:00.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.857+0000 7f25c0bb4740 1 -- 172.21.15.76:0/296151083 shutdown_connections 2023-11-09T03:46:00.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.857+0000 7f25c0bb4740 1 -- 172.21.15.76:0/296151083 wait complete. 2023-11-09T03:46:00.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.857+0000 7f25c0bb4740 20 client.4902 trim_cache size 0 max 16384 2023-11-09T03:46:00.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.873+0000 7f25c0bb4740 1 Processor -- start 2023-11-09T03:46:00.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.873+0000 7f25c0bb4740 1 -- start start 2023-11-09T03:46:00.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.873+0000 7f25c0bb4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be4809c850 0x55be480bacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:00.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.874+0000 7f25c0bb4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be4809cc30 0x55be480a2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:00.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.874+0000 7f25c0bb4740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be480b4c10 0x55be480a2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:00.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.874+0000 7f25c0bb4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be47f5b280 con 0x55be4809cc30 2023-11-09T03:46:00.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.874+0000 7f25c0bb4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be480a1e10 con 0x55be4809c850 2023-11-09T03:46:00.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.874+0000 7f25c0bb4740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be480bb240 con 0x55be480b4c10 2023-11-09T03:46:00.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.874+0000 7f25b6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be4809cc30 0x55be480a2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:00.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.874+0000 7f25b6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be4809cc30 0x55be480a2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36804/0 (socket says 172.21.15.76:36804) 2023-11-09T03:46:00.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.874+0000 7f25b6ffd640 1 -- 172.21.15.76:0/2135017099 learned_addr learned my addr 172.21.15.76:0/2135017099 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:00.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.874+0000 7f25b7fff640 1 --2- 172.21.15.76:0/2135017099 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be480b4c10 0x55be480a2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:00.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.874+0000 7f25b77fe640 1 --2- 172.21.15.76:0/2135017099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be4809c850 0x55be480bacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:00.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.874+0000 7f25b6ffd640 1 -- 172.21.15.76:0/2135017099 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be480b4c10 msgr2=0x55be480a2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:00.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.874+0000 7f25b6ffd640 1 --2- 172.21.15.76:0/2135017099 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be480b4c10 0x55be480a2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.874+0000 7f25b6ffd640 1 -- 172.21.15.76:0/2135017099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be4809c850 msgr2=0x55be480bacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:00.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.874+0000 7f25b6ffd640 1 --2- 172.21.15.76:0/2135017099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be4809c850 0x55be480bacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.874+0000 7f25b6ffd640 1 -- 172.21.15.76:0/2135017099 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be47f77d10 con 0x55be4809cc30 2023-11-09T03:46:00.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.874+0000 7f25b7fff640 1 --2- 172.21.15.76:0/2135017099 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be480b4c10 0x55be480a2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:00.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.874+0000 7f25b6ffd640 1 --2- 172.21.15.76:0/2135017099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be4809cc30 0x55be480a2550 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55be4809d590 tx=0x7f25ac009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd636faf20f09ec server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:00.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.874+0000 7f259dffb640 1 -- 172.21.15.76:0/2135017099 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac009e60 con 0x55be4809cc30 2023-11-09T03:46:00.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.874+0000 7f259dffb640 1 -- 172.21.15.76:0/2135017099 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25ac004030 con 0x55be4809cc30 2023-11-09T03:46:00.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.874+0000 7f259dffb640 1 -- 172.21.15.76:0/2135017099 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac0043e0 con 0x55be4809cc30 2023-11-09T03:46:00.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.875+0000 7f25c0bb4740 1 -- 172.21.15.76:0/2135017099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be4809cc30 msgr2=0x55be480a2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:00.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.875+0000 7f25c0bb4740 1 --2- 172.21.15.76:0/2135017099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be4809cc30 0x55be480a2550 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55be4809d590 tx=0x7f25ac009190 comp rx=0 tx=0).stop 2023-11-09T03:46:00.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.875+0000 7f25c0bb4740 1 -- 172.21.15.76:0/2135017099 shutdown_connections 2023-11-09T03:46:00.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.875+0000 7f25c0bb4740 1 --2- 172.21.15.76:0/2135017099 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be480b4c10 0x55be480a2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.875+0000 7f25c0bb4740 1 --2- 172.21.15.76:0/2135017099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be4809cc30 0x55be480a2550 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.875+0000 7f25c0bb4740 1 --2- 172.21.15.76:0/2135017099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be4809c850 0x55be480bacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.875+0000 7f25c0bb4740 1 -- 172.21.15.76:0/2135017099 >> 172.21.15.76:0/2135017099 conn(0x55be4806fc90 msgr2=0x55be480909f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:00.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.875+0000 7f25c0bb4740 1 -- 172.21.15.76:0/2135017099 shutdown_connections 2023-11-09T03:46:00.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.875+0000 7f25c0bb4740 1 -- 172.21.15.76:0/2135017099 wait complete. 2023-11-09T03:46:00.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.875+0000 7f25c0bb4740 1 -- start start 2023-11-09T03:46:00.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.875+0000 7f25c0bb4740 1 Processor -- start 2023-11-09T03:46:00.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.876+0000 7f25c0bb4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be4809c850 0x55be48129410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:00.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.876+0000 7f25c0bb4740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be4809cc30 0x55be4810e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:00.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.876+0000 7f25c0bb4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be480b4c10 0x55be48125380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:00.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.876+0000 7f25c0bb4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be48128ff0 con 0x55be480b4c10 2023-11-09T03:46:00.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.876+0000 7f25c0bb4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be48126140 con 0x55be4809c850 2023-11-09T03:46:00.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.876+0000 7f25c0bb4740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be4806e7f0 con 0x55be4809cc30 2023-11-09T03:46:00.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.876+0000 7f25c0bb4740 10 client.0 _ll_register_callbacks cb 0x7f25ac009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:46:00.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.876+0000 7f25b7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be480b4c10 0x55be48125380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:00.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.876+0000 7f25b7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be480b4c10 0x55be48125380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36806/0 (socket says 172.21.15.76:36806) 2023-11-09T03:46:00.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.876+0000 7f25b7fff640 1 -- 172.21.15.76:0/1010583284 learned_addr learned my addr 172.21.15.76:0/1010583284 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:00.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.876+0000 7f25b77fe640 1 --2- 172.21.15.76:0/1010583284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be4809c850 0x55be48129410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:00.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.876+0000 7f25b6ffd640 1 --2- 172.21.15.76:0/1010583284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be4809cc30 0x55be4810e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:00.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.877+0000 7f25b7fff640 1 -- 172.21.15.76:0/1010583284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be4809cc30 msgr2=0x55be4810e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:00.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.877+0000 7f25b7fff640 1 --2- 172.21.15.76:0/1010583284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be4809cc30 0x55be4810e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.877+0000 7f25b7fff640 1 -- 172.21.15.76:0/1010583284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be4809c850 msgr2=0x55be48129410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:00.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.877+0000 7f25b7fff640 1 --2- 172.21.15.76:0/1010583284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be4809c850 0x55be48129410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.877+0000 7f25b7fff640 1 -- 172.21.15.76:0/1010583284 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25ac009870 con 0x55be480b4c10 2023-11-09T03:46:00.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.877+0000 7f25b7fff640 1 --2- 172.21.15.76:0/1010583284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be480b4c10 0x55be48125380 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f25b000b980 tx=0x7f25b00551a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd3e3fa58eb70e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:00.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.877+0000 7f25967fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:46:00.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.877+0000 7f25967fc640 1 -- 172.21.15.76:0/1010583284 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b0057190 con 0x55be480b4c10 2023-11-09T03:46:00.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.877+0000 7f25967fc640 1 -- 172.21.15.76:0/1010583284 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f25b0058d30 con 0x55be480b4c10 2023-11-09T03:46:00.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.877+0000 7f25c0bb4740 1 -- 172.21.15.76:0/1010583284 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25a803ac50 con 0x55be480b4c10 2023-11-09T03:46:00.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.877+0000 7f25967fc640 1 -- 172.21.15.76:0/1010583284 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25b0054b20 con 0x55be480b4c10 2023-11-09T03:46:00.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.877+0000 7f25c0bb4740 10 client.4908 fetch_fsmap learned FSMap version 9 2023-11-09T03:46:00.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.877+0000 7f25c0bb4740 1 -- 172.21.15.76:0/1010583284 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55be47f9d500 con 0x55be480b4c10 2023-11-09T03:46:00.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.877+0000 7f25967fc640 1 -- 172.21.15.76:0/1010583284 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a803ac50 con 0x55be480b4c10 2023-11-09T03:46:00.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.877+0000 7f25967fc640 1 -- 172.21.15.76:0/1010583284 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f25b0024070 con 0x55be480b4c10 2023-11-09T03:46:00.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.877+0000 7f25c0bb4740 10 client.4908 fetch_fsmap finished waiting for FSMap version 9 2023-11-09T03:46:00.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.877+0000 7f25c0bb4740 10 client.4908 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-09T03:46:00.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.877+0000 7f25c0bb4740 20 client.4908 populate_metadata read hostname 'smithi076' 2023-11-09T03:46:00.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.877+0000 7f25c0bb4740 1 --2- 172.21.15.76:0/1010583284 >> [v2:172.21.15.113:6834/3293223710,v1:172.21.15.113:6836/3293223710] conn(0x55be480bda60 0x55be480bde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:00.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.877+0000 7f25c0bb4740 4 client.4908 mds_command: new command op to 4291 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-09T03:46:00.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.877+0000 7f25c0bb4740 1 -- 172.21.15.76:0/1010583284 --> [v2:172.21.15.113:6834/3293223710,v1:172.21.15.113:6836/3293223710] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55be47fd45b0 con 0x55be480bda60 2023-11-09T03:46:00.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.877+0000 7f25b77fe640 1 --2- 172.21.15.76:0/1010583284 >> [v2:172.21.15.113:6834/3293223710,v1:172.21.15.113:6836/3293223710] conn(0x55be480bda60 0x55be480bde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:00.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.878+0000 7f25b77fe640 1 --2- 172.21.15.76:0/1010583284 >> [v2:172.21.15.113:6834/3293223710,v1:172.21.15.113:6836/3293223710] conn(0x55be480bda60 0x55be480bde40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ace57232e3c5da48 server_cookie=c7b9e37a5b268c9f in_seq=0 out_seq=0 2023-11-09T03:46:00.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.878+0000 7f25967fc640 10 client.4908 ms_handle_connect on v2:172.21.15.113:6834/3293223710 2023-11-09T03:46:00.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.878+0000 7f25967fc640 1 -- 172.21.15.76:0/1010583284 <== mds.0 v2:172.21.15.113:6834/3293223710 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55be47fd45b0 con 0x55be480bda60 2023-11-09T03:46:00.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.878+0000 7f25967fc640 10 client.4908 handle_command_reply: tid=0 2023-11-09T03:46:00.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.878+0000 7f25c0bb4740 1 client.4908 shutdown 2023-11-09T03:46:00.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.878+0000 7f25c0bb4740 1 -- 172.21.15.76:0/1010583284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be480b4c10 msgr2=0x55be48125380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:00.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.878+0000 7f25c0bb4740 1 --2- 172.21.15.76:0/1010583284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be480b4c10 0x55be48125380 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f25b000b980 tx=0x7f25b00551a0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.878+0000 7f25c0bb4740 1 -- 172.21.15.76:0/1010583284 shutdown_connections 2023-11-09T03:46:00.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.878+0000 7f25c0bb4740 1 --2- 172.21.15.76:0/1010583284 >> [v2:172.21.15.113:6834/3293223710,v1:172.21.15.113:6836/3293223710] conn(0x55be480bda60 0x55be480bde40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.878+0000 7f25c0bb4740 1 --2- 172.21.15.76:0/1010583284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55be480b4c10 0x55be48125380 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.878+0000 7f25c0bb4740 1 --2- 172.21.15.76:0/1010583284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be4809cc30 0x55be4810e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.878+0000 7f25967fc640 0 client.4908 ms_handle_reset on v2:172.21.15.113:6834/3293223710 2023-11-09T03:46:00.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.878+0000 7f25c0bb4740 1 --2- 172.21.15.76:0/1010583284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be4809c850 0x55be48129410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.878+0000 7f25c0bb4740 1 -- 172.21.15.76:0/1010583284 >> 172.21.15.76:0/1010583284 conn(0x55be4806fc90 msgr2=0x55be480a2530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:00.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.878+0000 7f25c0bb4740 1 -- 172.21.15.76:0/1010583284 shutdown_connections 2023-11-09T03:46:00.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.878+0000 7f25c0bb4740 1 -- 172.21.15.76:0/1010583284 wait complete. 2023-11-09T03:46:00.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.878+0000 7f25c0bb4740 20 client.4908 trim_cache size 0 max 16384 2023-11-09T03:46:00.907 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:46:00.907 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "c213b26f-d0da-45d5-8e34-bb75c096d4f2", 2023-11-09T03:46:00.908 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2023-11-09T03:46:00.908 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 4291, 2023-11-09T03:46:00.908 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2023-11-09T03:46:00.908 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2023-11-09T03:46:00.908 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2023-11-09T03:46:00.909 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 29.476151552000001, 2023-11-09T03:46:00.909 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 9, 2023-11-09T03:46:00.909 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 25, 2023-11-09T03:46:00.909 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 25, 2023-11-09T03:46:00.909 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 36.524057517000003 2023-11-09T03:46:00.910 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:46:00.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.879+0000 7f25bdf4c640 1 -- 172.21.15.76:0/356540994 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f25a406ae70 msgr2=0x7f25a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:00.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.879+0000 7f25bdf4c640 1 --2- 172.21.15.76:0/356540994 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f25a406ae70 0x7f25a406d330 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f25a8036000 tx=0x7f25a803c040 comp rx=0 tx=0).stop 2023-11-09T03:46:00.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.879+0000 7f25bdf4c640 1 -- 172.21.15.76:0/356540994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25b8153e60 msgr2=0x7f25b8155890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:00.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.879+0000 7f25bdf4c640 1 --2- 172.21.15.76:0/356540994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25b8153e60 0x7f25b8155890 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f25b000eed0 tx=0x7f25b000b480 comp rx=0 tx=0).stop 2023-11-09T03:46:00.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.879+0000 7f25bdf4c640 1 -- 172.21.15.76:0/356540994 shutdown_connections 2023-11-09T03:46:00.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.879+0000 7f25bdf4c640 1 --2- 172.21.15.76:0/356540994 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f25a406ae70 0x7f25a406d330 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.879+0000 7f25bdf4c640 1 --2- 172.21.15.76:0/356540994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25b8153e60 0x7f25b8155890 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.879+0000 7f25bdf4c640 1 --2- 172.21.15.76:0/356540994 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25b80cd6d0 0x7f25b80c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.879+0000 7f25bdf4c640 1 --2- 172.21.15.76:0/356540994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25b80b4880 0x7f25b80c89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:00.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.879+0000 7f25bdf4c640 1 -- 172.21.15.76:0/356540994 >> 172.21.15.76:0/356540994 conn(0x7f25b80cf880 msgr2=0x7f25b80d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:00.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.879+0000 7f25bdf4c640 1 -- 172.21.15.76:0/356540994 shutdown_connections 2023-11-09T03:46:00.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:00.879+0000 7f25bdf4c640 1 -- 172.21.15.76:0/356540994 wait complete. 2023-11-09T03:46:00.914 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-11-09T03:46:00.949 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-11-09T03:46:00.949 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-09T03:46:00.950 INFO:tasks.ceph_fuse:client.0 config is {} 2023-11-09T03:46:00.950 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-09T03:46:00.950 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-09T03:46:00.951 INFO:tasks.ceph_fuse:client.2 config is {} 2023-11-09T03:46:00.951 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-09T03:46:00.951 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-09T03:46:00.951 INFO:tasks.ceph_fuse:client.1 config is {} 2023-11-09T03:46:00.952 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-09T03:46:00.952 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-09T03:46:00.952 INFO:tasks.ceph_fuse:client.3 config is {} 2023-11-09T03:46:00.952 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-09T03:46:00.953 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-09T03:46:00.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:00.953 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2023-11-09T03:46:00.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-11-09T03:46:00.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:00.970 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-09T03:46:01.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:01.036 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link delete ceph-brx 2023-11-09T03:46:01.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:01.129 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2023-11-09T03:46:01.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:01.148 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link delete ceph-brx 2023-11-09T03:46:01.222 INFO:teuthology.orchestra.run.smithi113.stderr:Cannot find device "ceph-brx" 2023-11-09T03:46:01.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:46:01.224 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-11-09T03:46:01.224 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-09T03:46:01.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.325+0000 7f5e1c5f5640 1 Processor -- start 2023-11-09T03:46:01.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.325+0000 7f5e1c5f5640 1 -- start start 2023-11-09T03:46:01.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.326+0000 7f5e1c5f5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5e1415ae40 0x7f5e1415bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:01.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.326+0000 7f5e1c5f5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e140cda30 0x7f5e140ced70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:01.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.326+0000 7f5e1c5f5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e140cc450 con 0x7f5e140cbc00 2023-11-09T03:46:01.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.326+0000 7f5e1c5f5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e1415f400 con 0x7f5e1415ae40 2023-11-09T03:46:01.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.326+0000 7f5e1c5f5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1415b220 con 0x7f5e140cda30 2023-11-09T03:46:01.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.326+0000 7f5e1a391640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5e1415ae40 0x7f5e1415bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:01.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.326+0000 7f5e1a391640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5e1415ae40 0x7f5e1415bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36814/0 (socket says 172.21.15.76:36814) 2023-11-09T03:46:01.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.326+0000 7f5e1a391640 1 -- 172.21.15.76:0/3749281501 learned_addr learned my addr 172.21.15.76:0/3749281501 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:01.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.326+0000 7f5e19b90640 1 --1- 172.21.15.76:0/3749281501 >> v1:172.21.15.76:6789/0 conn(0x7f5e140cbc00 0x7f5e140cd320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33768/0 (socket says 172.21.15.76:33768) 2023-11-09T03:46:01.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.326+0000 7f5e1a391640 1 -- 172.21.15.76:0/3749281501 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e140cda30 msgr2=0x7f5e140ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:01.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.326+0000 7f5e1a391640 1 --2- 172.21.15.76:0/3749281501 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e140cda30 0x7f5e140ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:01.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.326+0000 7f5e1a391640 1 -- 172.21.15.76:0/3749281501 >> v1:172.21.15.76:6789/0 conn(0x7f5e140cbc00 legacy=0x7f5e140cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:01.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.326+0000 7f5e1a391640 1 -- 172.21.15.76:0/3749281501 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1415f890 con 0x7f5e1415ae40 2023-11-09T03:46:01.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.326+0000 7f5e1938f640 1 -- 172.21.15.76:0/3749281501 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3288725004 0 0) 0x7f5e04001110 con 0x7f5e140cbc00 2023-11-09T03:46:01.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.327+0000 7f5e1a391640 1 --2- 172.21.15.76:0/3749281501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e1415ae40 0x7f5e1415bfb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009870 tx=0x7f5e1002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaee0c11b8ff2e14 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:01.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.327+0000 7f5e1938f640 1 -- 172.21.15.76:0/3749281501 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10030840 con 0x7f5e1415ae40 2023-11-09T03:46:01.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.327+0000 7f5e1938f640 1 -- 172.21.15.76:0/3749281501 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e100309a0 con 0x7f5e1415ae40 2023-11-09T03:46:01.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.327+0000 7f5e1938f640 1 -- 172.21.15.76:0/3749281501 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10030d30 con 0x7f5e1415ae40 2023-11-09T03:46:01.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.328+0000 7f5e1c5f5640 1 -- 172.21.15.76:0/3749281501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e1415ae40 msgr2=0x7f5e1415bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:01.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.328+0000 7f5e1c5f5640 1 --2- 172.21.15.76:0/3749281501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e1415ae40 0x7f5e1415bfb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009870 tx=0x7f5e1002b980 comp rx=0 tx=0).stop 2023-11-09T03:46:01.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.328+0000 7f5e1c5f5640 1 -- 172.21.15.76:0/3749281501 shutdown_connections 2023-11-09T03:46:01.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.328+0000 7f5e1c5f5640 1 --2- 172.21.15.76:0/3749281501 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e140cda30 0x7f5e140ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:01.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.328+0000 7f5e1c5f5640 1 --2- 172.21.15.76:0/3749281501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e1415ae40 0x7f5e1415bfb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:01.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.328+0000 7f5e1c5f5640 1 -- 172.21.15.76:0/3749281501 >> 172.21.15.76:0/3749281501 conn(0x7f5e1406ea90 msgr2=0x7f5e1406ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:01.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.328+0000 7f5e1c5f5640 1 -- 172.21.15.76:0/3749281501 shutdown_connections 2023-11-09T03:46:01.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.328+0000 7f5e1c5f5640 1 -- 172.21.15.76:0/3749281501 wait complete. 2023-11-09T03:46:01.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.329+0000 7f5e1c5f5640 1 Processor -- start 2023-11-09T03:46:01.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.329+0000 7f5e1c5f5640 1 -- start start 2023-11-09T03:46:01.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.330+0000 7f5e1c5f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e140cbc00 0x7f5e1414f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:01.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.330+0000 7f5e1c5f5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e140cda30 0x7f5e140c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:01.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.330+0000 7f5e1c5f5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e1415ae40 0x7f5e141f1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:01.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.330+0000 7f5e1c5f5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e141f0a00 con 0x7f5e1415ae40 2023-11-09T03:46:01.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.330+0000 7f5e1c5f5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e140c69a0 con 0x7f5e140cbc00 2023-11-09T03:46:01.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.330+0000 7f5e1c5f5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e141f0c80 con 0x7f5e140cda30 2023-11-09T03:46:01.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.330+0000 7f5e1ab92640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e1415ae40 0x7f5e141f1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:01.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.330+0000 7f5e1ab92640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e1415ae40 0x7f5e141f1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36828/0 (socket says 172.21.15.76:36828) 2023-11-09T03:46:01.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.330+0000 7f5e1ab92640 1 -- 172.21.15.76:0/2210514140 learned_addr learned my addr 172.21.15.76:0/2210514140 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:01.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.330+0000 7f5e19b90640 1 --2- 172.21.15.76:0/2210514140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e140cda30 0x7f5e140c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:01.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.330+0000 7f5e1ab92640 1 -- 172.21.15.76:0/2210514140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e140cda30 msgr2=0x7f5e140c6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:01.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.330+0000 7f5e1ab92640 1 --2- 172.21.15.76:0/2210514140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e140cda30 0x7f5e140c6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:01.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.330+0000 7f5e1ab92640 1 -- 172.21.15.76:0/2210514140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e140cbc00 msgr2=0x7f5e1414f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:01.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.330+0000 7f5e1ab92640 1 --2- 172.21.15.76:0/2210514140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e140cbc00 0x7f5e1414f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:01.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.330+0000 7f5e1ab92640 1 -- 172.21.15.76:0/2210514140 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e100094d0 con 0x7f5e1415ae40 2023-11-09T03:46:01.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.330+0000 7f5e19b90640 1 --2- 172.21.15.76:0/2210514140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e140cda30 0x7f5e140c6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:01.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.330+0000 7f5e1ab92640 1 --2- 172.21.15.76:0/2210514140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e1415ae40 0x7f5e141f1110 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c00b9a0 tx=0x7f5e0c00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:01.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.331+0000 7f5e0affd640 1 -- 172.21.15.76:0/2210514140 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c00cba0 con 0x7f5e1415ae40 2023-11-09T03:46:01.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.331+0000 7f5e0affd640 1 -- 172.21.15.76:0/2210514140 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e0c00cd20 con 0x7f5e1415ae40 2023-11-09T03:46:01.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.331+0000 7f5e1c5f5640 1 -- 172.21.15.76:0/2210514140 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e140c71f0 con 0x7f5e1415ae40 2023-11-09T03:46:01.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.331+0000 7f5e0affd640 1 -- 172.21.15.76:0/2210514140 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c014680 con 0x7f5e1415ae40 2023-11-09T03:46:01.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.331+0000 7f5e1c5f5640 1 -- 172.21.15.76:0/2210514140 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e140c4c20 con 0x7f5e1415ae40 2023-11-09T03:46:01.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.332+0000 7f5e1c5f5640 1 -- 172.21.15.76:0/2210514140 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e140ce2d0 con 0x7f5e1415ae40 2023-11-09T03:46:01.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.332+0000 7f5e0affd640 1 -- 172.21.15.76:0/2210514140 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5e0c01d3b0 con 0x7f5e1415ae40 2023-11-09T03:46:01.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.332+0000 7f5e0affd640 1 --2- 172.21.15.76:0/2210514140 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5df806aea0 0x7f5df806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:01.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.333+0000 7f5e0affd640 1 -- 172.21.15.76:0/2210514140 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5e0c089850 con 0x7f5e1415ae40 2023-11-09T03:46:01.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.333+0000 7f5e1a391640 1 --2- 172.21.15.76:0/2210514140 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5df806aea0 0x7f5df806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:01.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.333+0000 7f5e1a391640 1 --2- 172.21.15.76:0/2210514140 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5df806aea0 0x7f5df806d360 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5e10036000 tx=0x7f5e1003b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:01.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.337+0000 7f5e0affd640 1 -- 172.21.15.76:0/2210514140 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5e0c054610 con 0x7f5e1415ae40 2023-11-09T03:46:01.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.540+0000 7f5e1c5f5640 1 -- 172.21.15.76:0/2210514140 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5e14074290 con 0x7f5e1415ae40 2023-11-09T03:46:01.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.540+0000 7f5e0affd640 1 -- 172.21.15.76:0/2210514140 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f5e0c019070 con 0x7f5e1415ae40 2023-11-09T03:46:01.541 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-09T03:46:01.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.542+0000 7f5e1c5f5640 1 -- 172.21.15.76:0/2210514140 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5df806aea0 msgr2=0x7f5df806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:01.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.542+0000 7f5e1c5f5640 1 --2- 172.21.15.76:0/2210514140 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5df806aea0 0x7f5df806d360 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5e10036000 tx=0x7f5e1003b040 comp rx=0 tx=0).stop 2023-11-09T03:46:01.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.542+0000 7f5e1c5f5640 1 -- 172.21.15.76:0/2210514140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e1415ae40 msgr2=0x7f5e141f1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:01.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.542+0000 7f5e1c5f5640 1 --2- 172.21.15.76:0/2210514140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e1415ae40 0x7f5e141f1110 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c00b9a0 tx=0x7f5e0c00be60 comp rx=0 tx=0).stop 2023-11-09T03:46:01.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.542+0000 7f5e1c5f5640 1 -- 172.21.15.76:0/2210514140 shutdown_connections 2023-11-09T03:46:01.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.542+0000 7f5e1c5f5640 1 --2- 172.21.15.76:0/2210514140 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5df806aea0 0x7f5df806d360 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:01.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.542+0000 7f5e1c5f5640 1 --2- 172.21.15.76:0/2210514140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e1415ae40 0x7f5e141f1110 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:01.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.542+0000 7f5e1c5f5640 1 --2- 172.21.15.76:0/2210514140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e140cda30 0x7f5e140c6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:01.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.542+0000 7f5e1c5f5640 1 --2- 172.21.15.76:0/2210514140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e140cbc00 0x7f5e1414f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:01.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.542+0000 7f5e1c5f5640 1 -- 172.21.15.76:0/2210514140 >> 172.21.15.76:0/2210514140 conn(0x7f5e1406ea90 msgr2=0x7f5e1415dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:01.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.542+0000 7f5e1c5f5640 1 -- 172.21.15.76:0/2210514140 shutdown_connections 2023-11-09T03:46:01.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:01.542+0000 7f5e1c5f5640 1 -- 172.21.15.76:0/2210514140 wait complete. 2023-11-09T03:46:01.553 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-09T03:46:01.553 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-09T03:46:01.553 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2023-11-09T03:46:01.554 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-09T03:46:01.554 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-09T03:46:01.554 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-09T03:46:01.554 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-09T03:46:01.555 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-09T03:46:01.555 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:46:01.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:01.556 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2023-11-09T03:46:01.571 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-09T03:46:01.571 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-09T03:46:01.571 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2023-11-09T03:46:01.571 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:01.572 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2023-11-09T03:46:01.572 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:01.572 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:46:01.572 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:46:01.572 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2023-11-09T03:46:01.573 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:46:01.573 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:46:01.573 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2023-11-09T03:46:01.573 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:46:01.573 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:46:01.574 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-09T03:46:01.574 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:46:01.574 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-09T03:46:01.574 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42587sec preferred_lft 42587sec 2023-11-09T03:46:01.574 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2023-11-09T03:46:01.575 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:01.575 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-09T03:46:01.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:01.576 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-09T03:46:01.576 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-09T03:46:01.576 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link add name ceph-brx type bridge 2023-11-09T03:46:01.576 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip addr flush dev ceph-brx 2023-11-09T03:46:01.577 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set ceph-brx up 2023-11-09T03:46:01.577 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-09T03:46:01.577 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-09T03:46:01.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:01.680 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-09T03:46:01.705 INFO:teuthology.orchestra.run.smithi076.stdout:1 2023-11-09T03:46:01.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:01.707 DEBUG:teuthology.orchestra.run.smithi076:> route 2023-11-09T03:46:01.763 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2023-11-09T03:46:01.763 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-09T03:46:01.764 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-09T03:46:01.764 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-09T03:46:01.764 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-09T03:46:01.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:01.765 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-09T03:46:01.766 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-09T03:46:01.767 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-09T03:46:01.767 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-09T03:46:01.767 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-09T03:46:01.768 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-09T03:46:01.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:01.858 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2023-11-09T03:46:01.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:01.873 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list-id 2023-11-09T03:46:01.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:01.928 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-09T03:46:01.928 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-09T03:46:01.928 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:46:01.928 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-09T03:46:01.929 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-09T03:46:02.010 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-09T03:46:02.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:02.011 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-09T03:46:02.011 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-09T03:46:02.011 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-09T03:46:02.011 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-09T03:46:02.012 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-09T03:46:02.012 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-09T03:46:02.012 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-09T03:46:02.012 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-09T03:46:02.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:02.135 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-09T03:46:02.135 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-09T03:46:02.135 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2023-11-09T03:46:02.136 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set dev brx.0 master ceph-brx 2023-11-09T03:46:02.136 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-09T03:46:02.177 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-09T03:46:02.177 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:46:02.178 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-09T03:46:02.231 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:46:02.231 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:46:02.232 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-09T03:46:02.285 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:46:02.341 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:46:02.341 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:46:02.341 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:46:02.341 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:46:02.342 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:46:02.342 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:46:02.342 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:46:02.342 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:46:02.342 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:46:02.343 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:46:02.343 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:46:02.343 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:46:02.343 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:46:02.344 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:46:02.344 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:46:02.344 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:46:02.344 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:46:02.344 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:46:02.345 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:46:02.345 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:46:02.345 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:46:02.345 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:46:02.345 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:46:02.345 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:46:02.346 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:46:02.346 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:46:02.346 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:46:02.346 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:46:02.347 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:46:02.347 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:46:02.347 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:46:02.347 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:46:02.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:02.348 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:46:02.402 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-09T03:46:02.403 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-09T03:46:02.445 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:46:02.461 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:46:02.461 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:46:02.461 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:46:02.461 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:46:02.461 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:46:02.462 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:46:02.462 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:46:02.462 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:46:02.462 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:46:02.462 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:46:02.463 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:46:02.463 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:46:02.463 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:46:02.463 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:46:02.463 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:46:02.464 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:46:02.464 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:46:02.464 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:46:02.464 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:46:02.464 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:46:02.465 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:46:02.465 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:46:02.465 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:46:02.465 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:46:02.465 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:46:02.466 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:46:02.466 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:46:02.466 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:46:02.466 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:46:02.466 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:46:02.466 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:46:02.467 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:46:02.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:02.467 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:46:02.536 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:46:02.536+0000 7f5017638480 -1 init, newargv = 0x5643dd3288e0 newargc=15 2023-11-09T03:46:02.536 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[98478]: starting ceph client 2023-11-09T03:46:02.541 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[98478]: starting fuse 2023-11-09T03:46:03.617 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:46:03.635 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:46:03.635 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:46:03.635 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:46:03.635 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:46:03.636 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:46:03.636 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:46:03.636 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:46:03.636 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:46:03.636 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:46:03.636 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:46:03.637 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:46:03.637 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:46:03.637 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:46:03.637 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:46:03.637 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:46:03.638 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:46:03.638 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:46:03.638 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:46:03.638 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:46:03.638 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:46:03.639 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:46:03.639 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:46:03.639 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:46:03.639 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:46:03.639 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:46:03.640 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:46:03.640 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:46:03.640 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:46:03.640 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:46:03.640 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:46:03.640 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:46:03.641 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:46:03.641 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-09T03:46:03.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:03.641 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:46:03.691 INFO:teuthology.orchestra.run.smithi076.stdout:43 2023-11-09T03:46:03.691 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-09T03:46:03.691 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer -- python3 -c ' 2023-11-09T03:46:03.692 DEBUG:teuthology.orchestra.run.smithi076:> import glob 2023-11-09T03:46:03.692 DEBUG:teuthology.orchestra.run.smithi076:> import re 2023-11-09T03:46:03.692 DEBUG:teuthology.orchestra.run.smithi076:> import os 2023-11-09T03:46:03.693 DEBUG:teuthology.orchestra.run.smithi076:> import subprocess 2023-11-09T03:46:03.693 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:46:03.693 DEBUG:teuthology.orchestra.run.smithi076:> def _find_admin_socket(client_name): 2023-11-09T03:46:03.693 DEBUG:teuthology.orchestra.run.smithi076:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-09T03:46:03.694 DEBUG:teuthology.orchestra.run.smithi076:> files = glob.glob(asok_path) 2023-11-09T03:46:03.694 DEBUG:teuthology.orchestra.run.smithi076:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-09T03:46:03.694 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:46:03.694 DEBUG:teuthology.orchestra.run.smithi076:> # Given a non-glob path, it better be there 2023-11-09T03:46:03.695 DEBUG:teuthology.orchestra.run.smithi076:> if "*" not in asok_path: 2023-11-09T03:46:03.695 DEBUG:teuthology.orchestra.run.smithi076:> assert(len(files) == 1) 2023-11-09T03:46:03.695 DEBUG:teuthology.orchestra.run.smithi076:> return files[0] 2023-11-09T03:46:03.696 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:46:03.696 DEBUG:teuthology.orchestra.run.smithi076:> for f in files: 2023-11-09T03:46:03.696 DEBUG:teuthology.orchestra.run.smithi076:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-09T03:46:03.696 DEBUG:teuthology.orchestra.run.smithi076:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-09T03:46:03.697 DEBUG:teuthology.orchestra.run.smithi076:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-09T03:46:03.697 DEBUG:teuthology.orchestra.run.smithi076:> contents = proc_f.read() 2023-11-09T03:46:03.697 DEBUG:teuthology.orchestra.run.smithi076:> if mountpoint in contents: 2023-11-09T03:46:03.697 DEBUG:teuthology.orchestra.run.smithi076:> return f 2023-11-09T03:46:03.698 DEBUG:teuthology.orchestra.run.smithi076:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-09T03:46:03.698 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:46:03.698 DEBUG:teuthology.orchestra.run.smithi076:> print(_find_admin_socket("client.0")) 2023-11-09T03:46:03.699 DEBUG:teuthology.orchestra.run.smithi076:> ' 2023-11-09T03:46:03.814 INFO:teuthology.orchestra.run.smithi076.stdout:/var/run/ceph/ceph-client.0.98478.asok 2023-11-09T03:46:03.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:46:03.825 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.98478.asok 2023-11-09T03:46:03.825 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:46:03.825 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.98478.asok status 2023-11-09T03:46:03.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:46:03.992 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:46:03.993 INFO:teuthology.orchestra.run.smithi076.stdout: "metadata": { 2023-11-09T03:46:03.993 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_sha1": "7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4", 2023-11-09T03:46:03.993 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev)", 2023-11-09T03:46:03.993 INFO:teuthology.orchestra.run.smithi076.stdout: "entity_id": "0", 2023-11-09T03:46:03.994 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": "smithi076", 2023-11-09T03:46:03.994 INFO:teuthology.orchestra.run.smithi076.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-09T03:46:03.994 INFO:teuthology.orchestra.run.smithi076.stdout: "pid": "98478", 2023-11-09T03:46:03.994 INFO:teuthology.orchestra.run.smithi076.stdout: "root": "/" 2023-11-09T03:46:03.994 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:03.995 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_count": 0, 2023-11-09T03:46:03.995 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_pinned_count": 0, 2023-11-09T03:46:03.995 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 4926, 2023-11-09T03:46:03.995 INFO:teuthology.orchestra.run.smithi076.stdout: "inst": { 2023-11-09T03:46:03.995 INFO:teuthology.orchestra.run.smithi076.stdout: "name": { 2023-11-09T03:46:03.996 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "client", 2023-11-09T03:46:03.996 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 4926 2023-11-09T03:46:03.996 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:03.996 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2023-11-09T03:46:03.996 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2023-11-09T03:46:03.996 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2023-11-09T03:46:03.997 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 1480010360 2023-11-09T03:46:03.997 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:03.997 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:03.997 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2023-11-09T03:46:03.997 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2023-11-09T03:46:03.997 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2023-11-09T03:46:03.998 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 1480010360 2023-11-09T03:46:03.998 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:03.998 INFO:teuthology.orchestra.run.smithi076.stdout: "inst_str": "client.4926 192.168.0.1:0/1480010360", 2023-11-09T03:46:03.998 INFO:teuthology.orchestra.run.smithi076.stdout: "addr_str": "192.168.0.1:0/1480010360", 2023-11-09T03:46:03.998 INFO:teuthology.orchestra.run.smithi076.stdout: "inode_count": 1, 2023-11-09T03:46:03.999 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_epoch": 9, 2023-11-09T03:46:03.999 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch": 25, 2023-11-09T03:46:03.999 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch_barrier": 0, 2023-11-09T03:46:03.999 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklisted": false, 2023-11-09T03:46:03.999 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs" 2023-11-09T03:46:03.999 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:46:04.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:46:04.007 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-09T03:46:04.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.106+0000 7f93e32c2640 1 Processor -- start 2023-11-09T03:46:04.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.107+0000 7f93e32c2640 1 -- start start 2023-11-09T03:46:04.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.107+0000 7f93e32c2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f93dc0b42c0 0x7f93dc0c9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:04.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.107+0000 7f93e32c2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f93dc0c9c40 0x7f93dc0b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:04.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.108+0000 7f93e32c2640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93dc0ca050 con 0x7f93dc15ad80 2023-11-09T03:46:04.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.108+0000 7f93e32c2640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93dc06cfa0 con 0x7f93dc0c9c40 2023-11-09T03:46:04.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.108+0000 7f93e32c2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93dc06d280 con 0x7f93dc0b42c0 2023-11-09T03:46:04.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.108+0000 7f93e185f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f93dc0c9c40 0x7f93dc0b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:04.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.108+0000 7f93e105e640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f93dc15ad80 0x7f93dc06c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46784/0 (socket says 172.21.15.76:46784) 2023-11-09T03:46:04.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.108+0000 7f93e105e640 1 -- 172.21.15.76:0/1608911305 learned_addr learned my addr 172.21.15.76:0/1608911305 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:04.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.108+0000 7f93e085d640 1 --2- 172.21.15.76:0/1608911305 >> v2:172.21.15.113:3300/0 conn(0x7f93dc0b42c0 0x7f93dc0c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:04.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.108+0000 7f93e185f640 1 -- 172.21.15.76:0/1608911305 >> v2:172.21.15.113:3300/0 conn(0x7f93dc0b42c0 msgr2=0x7f93dc0c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:04.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.108+0000 7f93e185f640 1 --2- 172.21.15.76:0/1608911305 >> v2:172.21.15.113:3300/0 conn(0x7f93dc0b42c0 0x7f93dc0c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:04.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.108+0000 7f93e185f640 1 -- 172.21.15.76:0/1608911305 >> v1:172.21.15.76:6789/0 conn(0x7f93dc15ad80 legacy=0x7f93dc06c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:04.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.108+0000 7f93d3fff640 1 -- 172.21.15.76:0/1608911305 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1937106983 0 0) 0x7f93dc0ca050 con 0x7f93dc15ad80 2023-11-09T03:46:04.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.108+0000 7f93e185f640 1 -- 172.21.15.76:0/1608911305 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93dc06d560 con 0x7f93dc0c9c40 2023-11-09T03:46:04.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.109+0000 7f93e185f640 1 --2- 172.21.15.76:0/1608911305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93dc0c9c40 0x7f93dc0b1a40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0c23c0 tx=0x7f93d402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9953d1b83829c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:04.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.109+0000 7f93d3fff640 1 -- 172.21.15.76:0/1608911305 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d402cd00 con 0x7f93dc0c9c40 2023-11-09T03:46:04.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.109+0000 7f93d3fff640 1 -- 172.21.15.76:0/1608911305 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d402ce60 con 0x7f93dc0c9c40 2023-11-09T03:46:04.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.109+0000 7f93d3fff640 1 -- 172.21.15.76:0/1608911305 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d4034660 con 0x7f93dc0c9c40 2023-11-09T03:46:04.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.109+0000 7f93e32c2640 1 -- 172.21.15.76:0/1608911305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93dc0c9c40 msgr2=0x7f93dc0b1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:04.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.109+0000 7f93e32c2640 1 --2- 172.21.15.76:0/1608911305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93dc0c9c40 0x7f93dc0b1a40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0c23c0 tx=0x7f93d402bf50 comp rx=0 tx=0).stop 2023-11-09T03:46:04.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.109+0000 7f93e32c2640 1 -- 172.21.15.76:0/1608911305 shutdown_connections 2023-11-09T03:46:04.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.109+0000 7f93e32c2640 1 --2- 172.21.15.76:0/1608911305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93dc0c9c40 0x7f93dc0b1a40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:04.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.109+0000 7f93e32c2640 1 --2- 172.21.15.76:0/1608911305 >> v2:172.21.15.113:3300/0 conn(0x7f93dc0b42c0 0x7f93dc0c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:04.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.109+0000 7f93e32c2640 1 -- 172.21.15.76:0/1608911305 >> 172.21.15.76:0/1608911305 conn(0x7f93dc0ce950 msgr2=0x7f93dc0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:04.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.109+0000 7f93e32c2640 1 -- 172.21.15.76:0/1608911305 shutdown_connections 2023-11-09T03:46:04.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.110+0000 7f93e32c2640 1 -- 172.21.15.76:0/1608911305 wait complete. 2023-11-09T03:46:04.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.111+0000 7f93e32c2640 1 Processor -- start 2023-11-09T03:46:04.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.111+0000 7f93e32c2640 1 -- start start 2023-11-09T03:46:04.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.111+0000 7f93e32c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93dc0b42c0 0x7f93dc14ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:04.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.111+0000 7f93e32c2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93dc0c9c40 0x7f93dc157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:04.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.111+0000 7f93e32c2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93dc15ad80 0x7f93dc1f0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:04.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.111+0000 7f93e32c2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93dc1f15b0 con 0x7f93dc0b42c0 2023-11-09T03:46:04.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.111+0000 7f93e32c2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93dc0c7610 con 0x7f93dc15ad80 2023-11-09T03:46:04.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.111+0000 7f93e32c2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93dc157d90 con 0x7f93dc0c9c40 2023-11-09T03:46:04.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.111+0000 7f93e105e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93dc0b42c0 0x7f93dc14ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:04.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.111+0000 7f93e105e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93dc0b42c0 0x7f93dc14ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54284/0 (socket says 172.21.15.76:54284) 2023-11-09T03:46:04.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.111+0000 7f93e105e640 1 -- 172.21.15.76:0/1569682578 learned_addr learned my addr 172.21.15.76:0/1569682578 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:04.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.112+0000 7f93e085d640 1 --2- 172.21.15.76:0/1569682578 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93dc0c9c40 0x7f93dc157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:04.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.112+0000 7f93e185f640 1 --2- 172.21.15.76:0/1569682578 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93dc15ad80 0x7f93dc1f0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:04.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.112+0000 7f93e105e640 1 -- 172.21.15.76:0/1569682578 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93dc0c9c40 msgr2=0x7f93dc157850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:04.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.112+0000 7f93e105e640 1 --2- 172.21.15.76:0/1569682578 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93dc0c9c40 0x7f93dc157850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:04.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.112+0000 7f93e105e640 1 -- 172.21.15.76:0/1569682578 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93dc15ad80 msgr2=0x7f93dc1f0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:04.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.112+0000 7f93e105e640 1 --2- 172.21.15.76:0/1569682578 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93dc15ad80 0x7f93dc1f0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:04.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.112+0000 7f93e105e640 1 -- 172.21.15.76:0/1569682578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d4009a70 con 0x7f93dc0b42c0 2023-11-09T03:46:04.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.112+0000 7f93e185f640 1 --2- 172.21.15.76:0/1569682578 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93dc15ad80 0x7f93dc1f0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:04.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.112+0000 7f93e105e640 1 --2- 172.21.15.76:0/1569682578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93dc0b42c0 0x7f93dc14ae80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f93cc00c530 tx=0x7f93cc00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:04.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.112+0000 7f93d1ffb640 1 -- 172.21.15.76:0/1569682578 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc0074e0 con 0x7f93dc0b42c0 2023-11-09T03:46:04.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.112+0000 7f93d1ffb640 1 -- 172.21.15.76:0/1569682578 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93cc007660 con 0x7f93dc0b42c0 2023-11-09T03:46:04.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.112+0000 7f93e32c2640 1 -- 172.21.15.76:0/1569682578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93dc1f0a90 con 0x7f93dc0b42c0 2023-11-09T03:46:04.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.112+0000 7f93d1ffb640 1 -- 172.21.15.76:0/1569682578 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc004a20 con 0x7f93dc0b42c0 2023-11-09T03:46:04.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.112+0000 7f93e32c2640 1 -- 172.21.15.76:0/1569682578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93dc1f0ff0 con 0x7f93dc0b42c0 2023-11-09T03:46:04.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.113+0000 7f93e32c2640 1 -- 172.21.15.76:0/1569682578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93dc0c7370 con 0x7f93dc0b42c0 2023-11-09T03:46:04.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.114+0000 7f93d1ffb640 1 -- 172.21.15.76:0/1569682578 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f93cc004b80 con 0x7f93dc0b42c0 2023-11-09T03:46:04.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.114+0000 7f93d1ffb640 1 --2- 172.21.15.76:0/1569682578 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f93c006ae70 0x7f93c006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:04.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.114+0000 7f93d1ffb640 1 -- 172.21.15.76:0/1569682578 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f93cc088e00 con 0x7f93dc0b42c0 2023-11-09T03:46:04.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.114+0000 7f93e085d640 1 --2- 172.21.15.76:0/1569682578 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f93c006ae70 0x7f93c006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:04.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.115+0000 7f93e085d640 1 --2- 172.21.15.76:0/1569682578 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f93c006ae70 0x7f93c006d330 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f93d8006fd0 tx=0x7f93d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:04.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.118+0000 7f93d1ffb640 1 -- 172.21.15.76:0/1569682578 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f93dc0c7370 con 0x7f93dc0b42c0 2023-11-09T03:46:04.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.321+0000 7f93e32c2640 1 -- 172.21.15.76:0/1569682578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f93dc0c7370 con 0x7f93dc0b42c0 2023-11-09T03:46:04.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.321+0000 7f93d1ffb640 1 -- 172.21.15.76:0/1569682578 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f93dc0c7370 con 0x7f93dc0b42c0 2023-11-09T03:46:04.322 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-09T03:46:04.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.323+0000 7f93e32c2640 1 -- 172.21.15.76:0/1569682578 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f93c006ae70 msgr2=0x7f93c006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:04.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.323+0000 7f93e32c2640 1 --2- 172.21.15.76:0/1569682578 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f93c006ae70 0x7f93c006d330 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f93d8006fd0 tx=0x7f93d8008040 comp rx=0 tx=0).stop 2023-11-09T03:46:04.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.323+0000 7f93e32c2640 1 -- 172.21.15.76:0/1569682578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93dc0b42c0 msgr2=0x7f93dc14ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:04.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.323+0000 7f93e32c2640 1 --2- 172.21.15.76:0/1569682578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93dc0b42c0 0x7f93dc14ae80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f93cc00c530 tx=0x7f93cc00c9f0 comp rx=0 tx=0).stop 2023-11-09T03:46:04.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.323+0000 7f93e32c2640 1 -- 172.21.15.76:0/1569682578 shutdown_connections 2023-11-09T03:46:04.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.323+0000 7f93e32c2640 1 --2- 172.21.15.76:0/1569682578 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f93c006ae70 0x7f93c006d330 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:04.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.323+0000 7f93e32c2640 1 --2- 172.21.15.76:0/1569682578 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93dc15ad80 0x7f93dc1f0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:04.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.323+0000 7f93e32c2640 1 --2- 172.21.15.76:0/1569682578 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93dc0c9c40 0x7f93dc157850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:04.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.323+0000 7f93e32c2640 1 --2- 172.21.15.76:0/1569682578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93dc0b42c0 0x7f93dc14ae80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:04.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.323+0000 7f93e32c2640 1 -- 172.21.15.76:0/1569682578 >> 172.21.15.76:0/1569682578 conn(0x7f93dc0ce950 msgr2=0x7f93dc070ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:04.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.323+0000 7f93e32c2640 1 -- 172.21.15.76:0/1569682578 shutdown_connections 2023-11-09T03:46:04.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:04.323+0000 7f93e32c2640 1 -- 172.21.15.76:0/1569682578 wait complete. 2023-11-09T03:46:04.334 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-09T03:46:04.334 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-09T03:46:04.335 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2023-11-09T03:46:04.335 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-09T03:46:04.335 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-09T03:46:04.335 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-09T03:46:04.336 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-09T03:46:04.336 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-09T03:46:04.336 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-09T03:46:04.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:04.337 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2023-11-09T03:46:04.352 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-09T03:46:04.352 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-09T03:46:04.352 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2023-11-09T03:46:04.352 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:04.352 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2023-11-09T03:46:04.353 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:04.353 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:46:04.353 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:46:04.353 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2023-11-09T03:46:04.353 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:46:04.354 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:46:04.354 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2023-11-09T03:46:04.354 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:46:04.354 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:46:04.354 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-09T03:46:04.354 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:46:04.355 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-09T03:46:04.355 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42584sec preferred_lft 42584sec 2023-11-09T03:46:04.355 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2023-11-09T03:46:04.355 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:04.355 INFO:teuthology.orchestra.run.smithi076.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-09T03:46:04.355 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 6a:8d:94:d7:e5:00 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:46:04.356 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-09T03:46:04.356 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:04.356 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::bcda:38ff:fed4:fdb0/64 scope link 2023-11-09T03:46:04.356 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:04.356 INFO:teuthology.orchestra.run.smithi076.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-09T03:46:04.356 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 6a:8d:94:d7:e5:00 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:46:04.357 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::688d:94ff:fed7:e500/64 scope link 2023-11-09T03:46:04.357 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:04.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:04.357 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-09T03:46:04.417 INFO:teuthology.orchestra.run.smithi076.stdout:1 2023-11-09T03:46:04.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:04.419 DEBUG:teuthology.orchestra.run.smithi076:> route 2023-11-09T03:46:04.474 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2023-11-09T03:46:04.475 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-09T03:46:04.475 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-09T03:46:04.475 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-09T03:46:04.475 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-09T03:46:04.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:04.476 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-09T03:46:04.476 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-09T03:46:04.476 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-09T03:46:04.477 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-09T03:46:04.477 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-09T03:46:04.477 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-09T03:46:04.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:04.569 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2023-11-09T03:46:04.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-09T03:46:04.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:04.584 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list-id 2023-11-09T03:46:04.638 INFO:teuthology.orchestra.run.smithi076.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-09T03:46:04.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:04.639 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-09T03:46:04.639 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-09T03:46:04.639 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:46:04.640 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-09T03:46:04.640 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-09T03:46:04.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:04.720 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-09T03:46:04.786 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-09T03:46:04.786 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-09T03:46:04.786 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2023-11-09T03:46:04.787 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:04.787 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2023-11-09T03:46:04.787 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:04.787 INFO:teuthology.orchestra.run.smithi076.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-09T03:46:04.787 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether fa:ac:d4:0d:cf:d4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-09T03:46:04.787 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-09T03:46:04.788 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:04.788 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::f8ac:d4ff:fe0d:cfd4/64 scope link 2023-11-09T03:46:04.788 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:04.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:04.789 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-09T03:46:04.855 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-09T03:46:04.855 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-09T03:46:04.855 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2023-11-09T03:46:04.855 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:04.856 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2023-11-09T03:46:04.856 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:04.856 INFO:teuthology.orchestra.run.smithi076.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-09T03:46:04.856 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether fa:ac:d4:0d:cf:d4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-09T03:46:04.856 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-09T03:46:04.857 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:04.857 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::f8ac:d4ff:fe0d:cfd4/64 scope link 2023-11-09T03:46:04.857 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:04.858 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-09T03:46:04.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:04.859 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-09T03:46:04.859 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-09T03:46:04.859 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-09T03:46:04.860 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-09T03:46:04.860 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-09T03:46:04.860 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-09T03:46:04.860 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-09T03:46:04.861 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-09T03:46:04.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:04.982 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-09T03:46:04.982 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-09T03:46:04.982 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.1 up 2023-11-09T03:46:04.983 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set dev brx.1 master ceph-brx 2023-11-09T03:46:04.983 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-09T03:46:05.024 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-11-09T03:46:05.024 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:46:05.025 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-09T03:46:05.080 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-09T03:46:05.080 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:46:05.080 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-09T03:46:05.134 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:46:05.189 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:46:05.189 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:46:05.190 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:46:05.190 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:46:05.190 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:46:05.190 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:46:05.191 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:46:05.191 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:46:05.191 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:46:05.191 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:46:05.191 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:46:05.192 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:46:05.192 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:46:05.192 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:46:05.192 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:46:05.192 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:46:05.193 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:46:05.193 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:46:05.193 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:46:05.193 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:46:05.193 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:46:05.193 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:46:05.194 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:46:05.194 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:46:05.194 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:46:05.194 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:46:05.194 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:46:05.195 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:46:05.195 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:46:05.195 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:46:05.195 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:46:05.195 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:46:05.196 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-09T03:46:05.196 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:46:05.196 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:46:05.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:05.197 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:46:05.244 INFO:teuthology.orchestra.run.smithi076.stdout:43 2023-11-09T03:46:05.245 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2023-11-09T03:46:05.245 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-11-09T03:46:05.287 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:46:05.302 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:46:05.302 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:46:05.302 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:46:05.303 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:46:05.303 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:46:05.303 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:46:05.303 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:46:05.303 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:46:05.303 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:46:05.304 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:46:05.304 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:46:05.304 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:46:05.304 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:46:05.304 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:46:05.304 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:46:05.305 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:46:05.305 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:46:05.305 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:46:05.305 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:46:05.305 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:46:05.306 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:46:05.306 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:46:05.306 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:46:05.306 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:46:05.306 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:46:05.306 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:46:05.307 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:46:05.307 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:46:05.307 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:46:05.307 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:46:05.307 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:46:05.307 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:46:05.308 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-09T03:46:05.308 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:46:05.308 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:46:05.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:05.309 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:46:05.357 INFO:teuthology.orchestra.run.smithi076.stdout:43 2023-11-09T03:46:05.377 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi076.stderr:2023-11-09T03:46:05.376+0000 7f364d159480 -1 init, newargv = 0x5557b0e81a90 newargc=15 2023-11-09T03:46:05.377 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi076.stderr:ceph-fuse[98971]: starting ceph client 2023-11-09T03:46:05.381 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi076.stderr:ceph-fuse[98971]: starting fuse 2023-11-09T03:46:06.458 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:46:06.477 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:46:06.478 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:46:06.478 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:46:06.478 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:46:06.479 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:46:06.479 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:46:06.479 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:46:06.479 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:46:06.479 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:46:06.479 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:46:06.480 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:46:06.480 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:46:06.480 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:46:06.480 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:46:06.480 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:46:06.481 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:46:06.481 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:46:06.481 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:46:06.481 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:46:06.481 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:46:06.482 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:46:06.482 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:46:06.482 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:46:06.482 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:46:06.482 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:46:06.483 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:46:06.483 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:46:06.483 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:46:06.483 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:46:06.483 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:46:06.484 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:46:06.484 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:46:06.484 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-09T03:46:06.484 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:46:06.484 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:46:06.485 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.2 2023-11-09T03:46:06.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:06.485 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:46:06.535 INFO:teuthology.orchestra.run.smithi076.stdout:43 2023-11-09T03:46:06.536 INFO:teuthology.orchestra.run.smithi076.stdout:44 2023-11-09T03:46:06.536 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2023-11-09T03:46:06.536 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer -- python3 -c ' 2023-11-09T03:46:06.537 DEBUG:teuthology.orchestra.run.smithi076:> import glob 2023-11-09T03:46:06.537 DEBUG:teuthology.orchestra.run.smithi076:> import re 2023-11-09T03:46:06.537 DEBUG:teuthology.orchestra.run.smithi076:> import os 2023-11-09T03:46:06.538 DEBUG:teuthology.orchestra.run.smithi076:> import subprocess 2023-11-09T03:46:06.538 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:46:06.538 DEBUG:teuthology.orchestra.run.smithi076:> def _find_admin_socket(client_name): 2023-11-09T03:46:06.538 DEBUG:teuthology.orchestra.run.smithi076:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-11-09T03:46:06.539 DEBUG:teuthology.orchestra.run.smithi076:> files = glob.glob(asok_path) 2023-11-09T03:46:06.539 DEBUG:teuthology.orchestra.run.smithi076:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-11-09T03:46:06.539 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:46:06.540 DEBUG:teuthology.orchestra.run.smithi076:> # Given a non-glob path, it better be there 2023-11-09T03:46:06.540 DEBUG:teuthology.orchestra.run.smithi076:> if "*" not in asok_path: 2023-11-09T03:46:06.540 DEBUG:teuthology.orchestra.run.smithi076:> assert(len(files) == 1) 2023-11-09T03:46:06.540 DEBUG:teuthology.orchestra.run.smithi076:> return files[0] 2023-11-09T03:46:06.541 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:46:06.541 DEBUG:teuthology.orchestra.run.smithi076:> for f in files: 2023-11-09T03:46:06.541 DEBUG:teuthology.orchestra.run.smithi076:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-09T03:46:06.541 DEBUG:teuthology.orchestra.run.smithi076:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-09T03:46:06.542 DEBUG:teuthology.orchestra.run.smithi076:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-09T03:46:06.542 DEBUG:teuthology.orchestra.run.smithi076:> contents = proc_f.read() 2023-11-09T03:46:06.542 DEBUG:teuthology.orchestra.run.smithi076:> if mountpoint in contents: 2023-11-09T03:46:06.543 DEBUG:teuthology.orchestra.run.smithi076:> return f 2023-11-09T03:46:06.543 DEBUG:teuthology.orchestra.run.smithi076:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-09T03:46:06.543 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:46:06.544 DEBUG:teuthology.orchestra.run.smithi076:> print(_find_admin_socket("client.2")) 2023-11-09T03:46:06.544 DEBUG:teuthology.orchestra.run.smithi076:> ' 2023-11-09T03:46:06.667 INFO:teuthology.orchestra.run.smithi076.stdout:/var/run/ceph/ceph-client.2.98971.asok 2023-11-09T03:46:06.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:46:06.679 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.98971.asok 2023-11-09T03:46:06.679 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:46:06.680 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.2.98971.asok status 2023-11-09T03:46:06.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:46:06.849 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:46:06.849 INFO:teuthology.orchestra.run.smithi076.stdout: "metadata": { 2023-11-09T03:46:06.850 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_sha1": "7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4", 2023-11-09T03:46:06.850 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev)", 2023-11-09T03:46:06.850 INFO:teuthology.orchestra.run.smithi076.stdout: "entity_id": "2", 2023-11-09T03:46:06.850 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": "smithi076", 2023-11-09T03:46:06.850 INFO:teuthology.orchestra.run.smithi076.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-11-09T03:46:06.851 INFO:teuthology.orchestra.run.smithi076.stdout: "pid": "98971", 2023-11-09T03:46:06.851 INFO:teuthology.orchestra.run.smithi076.stdout: "root": "/" 2023-11-09T03:46:06.851 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:06.851 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_count": 0, 2023-11-09T03:46:06.851 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_pinned_count": 0, 2023-11-09T03:46:06.851 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 4941, 2023-11-09T03:46:06.852 INFO:teuthology.orchestra.run.smithi076.stdout: "inst": { 2023-11-09T03:46:06.852 INFO:teuthology.orchestra.run.smithi076.stdout: "name": { 2023-11-09T03:46:06.852 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "client", 2023-11-09T03:46:06.852 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 4941 2023-11-09T03:46:06.852 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:06.853 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2023-11-09T03:46:06.853 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2023-11-09T03:46:06.853 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.2:0", 2023-11-09T03:46:06.853 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 4164760658 2023-11-09T03:46:06.853 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:06.853 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:06.854 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2023-11-09T03:46:06.854 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2023-11-09T03:46:06.854 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.2:0", 2023-11-09T03:46:06.854 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 4164760658 2023-11-09T03:46:06.854 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:06.855 INFO:teuthology.orchestra.run.smithi076.stdout: "inst_str": "client.4941 192.168.0.2:0/4164760658", 2023-11-09T03:46:06.855 INFO:teuthology.orchestra.run.smithi076.stdout: "addr_str": "192.168.0.2:0/4164760658", 2023-11-09T03:46:06.855 INFO:teuthology.orchestra.run.smithi076.stdout: "inode_count": 1, 2023-11-09T03:46:06.855 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_epoch": 9, 2023-11-09T03:46:06.855 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch": 25, 2023-11-09T03:46:06.856 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch_barrier": 0, 2023-11-09T03:46:06.856 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklisted": false, 2023-11-09T03:46:06.856 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs" 2023-11-09T03:46:06.856 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:46:06.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:46:06.863 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-09T03:46:06.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.964+0000 7f7021675640 1 Processor -- start 2023-11-09T03:46:06.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.964+0000 7f7021675640 1 -- start start 2023-11-09T03:46:06.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.964+0000 7f7021675640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f701c0b42c0 0x7f701c0c9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:06.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.964+0000 7f7021675640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f701c0c9c40 0x7f701c0b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:06.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.964+0000 7f7021675640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f701c0ca090 con 0x7f701c15ad80 2023-11-09T03:46:06.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.964+0000 7f7021675640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f701c06cfa0 con 0x7f701c0c9c40 2023-11-09T03:46:06.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.964+0000 7f7021675640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f701c06d280 con 0x7f701c0b42c0 2023-11-09T03:46:06.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.965+0000 7f701a7fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f701c0b42c0 0x7f701c0c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:06.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.965+0000 7f701b7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f701c0c9c40 0x7f701c0b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:06.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.965+0000 7f701b7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f701c0c9c40 0x7f701c0b1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:56808/0 (socket says 172.21.15.76:56808) 2023-11-09T03:46:06.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.965+0000 7f701a7fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f701c0b42c0 0x7f701c0c9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:43520/0 (socket says 172.21.15.76:43520) 2023-11-09T03:46:06.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.965+0000 7f701b7fe640 1 -- 172.21.15.76:0/3415063462 learned_addr learned my addr 172.21.15.76:0/3415063462 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:06.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.965+0000 7f701affd640 1 --1- 172.21.15.76:0/3415063462 >> v1:172.21.15.113:6789/0 conn(0x7f701c15ad80 0x7f701c06c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:43158/0 (socket says 172.21.15.76:43158) 2023-11-09T03:46:06.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.965+0000 7f701b7fe640 1 -- 172.21.15.76:0/3415063462 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f701c0b42c0 msgr2=0x7f701c0c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:06.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.965+0000 7f701b7fe640 1 --2- 172.21.15.76:0/3415063462 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f701c0b42c0 0x7f701c0c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:06.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.965+0000 7f701b7fe640 1 -- 172.21.15.76:0/3415063462 >> v1:172.21.15.113:6789/0 conn(0x7f701c15ad80 legacy=0x7f701c06c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:06.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.965+0000 7f7019ffb640 1 -- 172.21.15.76:0/3415063462 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4174033791 0 0) 0x7f7014001110 con 0x7f701c15ad80 2023-11-09T03:46:06.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.965+0000 7f701b7fe640 1 -- 172.21.15.76:0/3415063462 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701c06d560 con 0x7f701c0c9c40 2023-11-09T03:46:06.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.965+0000 7f701a7fc640 1 --2- 172.21.15.76:0/3415063462 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f701c0b42c0 0x7f701c0c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:46:06.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.966+0000 7f701b7fe640 1 --2- 172.21.15.76:0/3415063462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f701c0c9c40 0x7f701c0b1a40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f700c009870 tx=0x7f700c02ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17e4a4d66871c3aa server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:06.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.966+0000 7f7019ffb640 1 -- 172.21.15.76:0/3415063462 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c02f840 con 0x7f701c0c9c40 2023-11-09T03:46:06.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.966+0000 7f7019ffb640 1 -- 172.21.15.76:0/3415063462 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700c02f9a0 con 0x7f701c0c9c40 2023-11-09T03:46:06.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.966+0000 7f7019ffb640 1 -- 172.21.15.76:0/3415063462 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c02fd30 con 0x7f701c0c9c40 2023-11-09T03:46:06.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.966+0000 7f7021675640 1 -- 172.21.15.76:0/3415063462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f701c0c9c40 msgr2=0x7f701c0b1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:06.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.966+0000 7f7021675640 1 --2- 172.21.15.76:0/3415063462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f701c0c9c40 0x7f701c0b1a40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f700c009870 tx=0x7f700c02ba20 comp rx=0 tx=0).stop 2023-11-09T03:46:06.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.967+0000 7f7021675640 1 -- 172.21.15.76:0/3415063462 shutdown_connections 2023-11-09T03:46:06.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.967+0000 7f7021675640 1 --2- 172.21.15.76:0/3415063462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f701c0c9c40 0x7f701c0b1a40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:06.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.967+0000 7f7021675640 1 --2- 172.21.15.76:0/3415063462 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f701c0b42c0 0x7f701c0c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:06.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.967+0000 7f7021675640 1 -- 172.21.15.76:0/3415063462 >> 172.21.15.76:0/3415063462 conn(0x7f701c0ce950 msgr2=0x7f701c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:06.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.967+0000 7f7021675640 1 -- 172.21.15.76:0/3415063462 shutdown_connections 2023-11-09T03:46:06.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.967+0000 7f7021675640 1 -- 172.21.15.76:0/3415063462 wait complete. 2023-11-09T03:46:06.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.968+0000 7f7021675640 1 Processor -- start 2023-11-09T03:46:06.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.968+0000 7f7021675640 1 -- start start 2023-11-09T03:46:06.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.969+0000 7f7021675640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f701c0b42c0 0x7f701c1efc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:06.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.969+0000 7f7021675640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f701c0c9c40 0x7f701c0a74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:06.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.969+0000 7f7021675640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f701c15ad80 0x7f701c0a7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:06.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.969+0000 7f7021675640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c15d230 con 0x7f701c0c9c40 2023-11-09T03:46:06.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.969+0000 7f7021675640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c15d510 con 0x7f701c0b42c0 2023-11-09T03:46:06.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.969+0000 7f7021675640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f701c15d7f0 con 0x7f701c15ad80 2023-11-09T03:46:06.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.969+0000 7f701a7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f701c0c9c40 0x7f701c0a74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:06.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.969+0000 7f701a7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f701c0c9c40 0x7f701c0a74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54294/0 (socket says 172.21.15.76:54294) 2023-11-09T03:46:06.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.969+0000 7f701a7fc640 1 -- 172.21.15.76:0/926232155 learned_addr learned my addr 172.21.15.76:0/926232155 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:06.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.969+0000 7f701affd640 1 --2- 172.21.15.76:0/926232155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f701c0b42c0 0x7f701c1efc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:06.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.969+0000 7f701b7fe640 1 --2- 172.21.15.76:0/926232155 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f701c15ad80 0x7f701c0a7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:06.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.969+0000 7f701a7fc640 1 -- 172.21.15.76:0/926232155 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f701c15ad80 msgr2=0x7f701c0a7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:06.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.969+0000 7f701a7fc640 1 --2- 172.21.15.76:0/926232155 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f701c15ad80 0x7f701c0a7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:06.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.969+0000 7f701a7fc640 1 -- 172.21.15.76:0/926232155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f701c0b42c0 msgr2=0x7f701c1efc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:06.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.969+0000 7f701a7fc640 1 --2- 172.21.15.76:0/926232155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f701c0b42c0 0x7f701c1efc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:06.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.969+0000 7f701a7fc640 1 -- 172.21.15.76:0/926232155 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7010009a70 con 0x7f701c0c9c40 2023-11-09T03:46:06.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.969+0000 7f701a7fc640 1 --2- 172.21.15.76:0/926232155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f701c0c9c40 0x7f701c0a74d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f701000efc0 tx=0x7f701000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:06.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.969+0000 7f70037fe640 1 -- 172.21.15.76:0/926232155 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7010014680 con 0x7f701c0c9c40 2023-11-09T03:46:06.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.969+0000 7f70037fe640 1 -- 172.21.15.76:0/926232155 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f701000f040 con 0x7f701c0c9c40 2023-11-09T03:46:06.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.969+0000 7f7021675640 1 -- 172.21.15.76:0/926232155 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700c0094d0 con 0x7f701c0c9c40 2023-11-09T03:46:06.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.969+0000 7f70037fe640 1 -- 172.21.15.76:0/926232155 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7010014a00 con 0x7f701c0c9c40 2023-11-09T03:46:06.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.969+0000 7f7021675640 1 -- 172.21.15.76:0/926232155 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f701c1f1630 con 0x7f701c0c9c40 2023-11-09T03:46:06.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.970+0000 7f7021675640 1 -- 172.21.15.76:0/926232155 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f701c074290 con 0x7f701c0c9c40 2023-11-09T03:46:06.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.971+0000 7f70037fe640 1 -- 172.21.15.76:0/926232155 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7010014b60 con 0x7f701c0c9c40 2023-11-09T03:46:06.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.971+0000 7f70037fe640 1 --2- 172.21.15.76:0/926232155 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f700806ae70 0x7f700806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:06.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.971+0000 7f70037fe640 1 -- 172.21.15.76:0/926232155 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f701008a6d0 con 0x7f701c0c9c40 2023-11-09T03:46:06.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.971+0000 7f701affd640 1 --2- 172.21.15.76:0/926232155 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f700806ae70 0x7f700806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:06.981 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.972+0000 7f701affd640 1 --2- 172.21.15.76:0/926232155 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f700806ae70 0x7f700806d330 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7014000f60 tx=0x7f7014007b90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:06.981 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:06.975+0000 7f70037fe640 1 -- 172.21.15.76:0/926232155 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f70100555d0 con 0x7f701c0c9c40 2023-11-09T03:46:07.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:07.177+0000 7f7021675640 1 -- 172.21.15.76:0/926232155 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f701c0b4880 con 0x7f701c0c9c40 2023-11-09T03:46:07.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:07.177+0000 7f70037fe640 1 -- 172.21.15.76:0/926232155 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f7010058bf0 con 0x7f701c0c9c40 2023-11-09T03:46:07.178 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-09T03:46:07.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:07.179+0000 7f7021675640 1 -- 172.21.15.76:0/926232155 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f700806ae70 msgr2=0x7f700806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:07.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:07.179+0000 7f7021675640 1 --2- 172.21.15.76:0/926232155 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f700806ae70 0x7f700806d330 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7014000f60 tx=0x7f7014007b90 comp rx=0 tx=0).stop 2023-11-09T03:46:07.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:07.179+0000 7f7021675640 1 -- 172.21.15.76:0/926232155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f701c0c9c40 msgr2=0x7f701c0a74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:07.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:07.179+0000 7f7021675640 1 --2- 172.21.15.76:0/926232155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f701c0c9c40 0x7f701c0a74d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f701000efc0 tx=0x7f701000c490 comp rx=0 tx=0).stop 2023-11-09T03:46:07.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:07.179+0000 7f7021675640 1 -- 172.21.15.76:0/926232155 shutdown_connections 2023-11-09T03:46:07.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:07.179+0000 7f7021675640 1 --2- 172.21.15.76:0/926232155 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f700806ae70 0x7f700806d330 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:07.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:07.179+0000 7f7021675640 1 --2- 172.21.15.76:0/926232155 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f701c15ad80 0x7f701c0a7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:07.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:07.179+0000 7f7021675640 1 --2- 172.21.15.76:0/926232155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f701c0c9c40 0x7f701c0a74d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:07.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:07.179+0000 7f7021675640 1 --2- 172.21.15.76:0/926232155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f701c0b42c0 0x7f701c1efc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:07.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:07.179+0000 7f7021675640 1 -- 172.21.15.76:0/926232155 >> 172.21.15.76:0/926232155 conn(0x7f701c0ce950 msgr2=0x7f701c0ae500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:07.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:07.179+0000 7f7021675640 1 -- 172.21.15.76:0/926232155 shutdown_connections 2023-11-09T03:46:07.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:07.179+0000 7f7021675640 1 -- 172.21.15.76:0/926232155 wait complete. 2023-11-09T03:46:07.190 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-09T03:46:07.190 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-09T03:46:07.191 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2023-11-09T03:46:07.191 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-09T03:46:07.191 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-09T03:46:07.191 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-09T03:46:07.191 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-09T03:46:07.192 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-09T03:46:07.192 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-09T03:46:07.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:07.193 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2023-11-09T03:46:07.211 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-09T03:46:07.211 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-09T03:46:07.212 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2023-11-09T03:46:07.212 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:07.212 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2023-11-09T03:46:07.212 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:07.212 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:46:07.213 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:46:07.213 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2023-11-09T03:46:07.213 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:46:07.213 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:46:07.213 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2023-11-09T03:46:07.214 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:46:07.214 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:46:07.214 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-09T03:46:07.214 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:46:07.214 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-09T03:46:07.215 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42609sec preferred_lft 42609sec 2023-11-09T03:46:07.215 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::ec4:7aff:fe8f:d7c1/64 scope link 2023-11-09T03:46:07.215 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:07.215 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-09T03:46:07.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:07.216 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2023-11-09T03:46:07.216 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2023-11-09T03:46:07.216 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link add name ceph-brx type bridge 2023-11-09T03:46:07.217 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip addr flush dev ceph-brx 2023-11-09T03:46:07.217 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set ceph-brx up 2023-11-09T03:46:07.217 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-09T03:46:07.217 DEBUG:teuthology.orchestra.run.smithi113:> ') 2023-11-09T03:46:07.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:07.400 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-09T03:46:07.427 INFO:teuthology.orchestra.run.smithi113.stdout:1 2023-11-09T03:46:07.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:07.429 DEBUG:teuthology.orchestra.run.smithi113:> route 2023-11-09T03:46:07.498 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2023-11-09T03:46:07.498 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-09T03:46:07.499 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-09T03:46:07.499 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-09T03:46:07.499 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-09T03:46:07.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:07.500 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2023-11-09T03:46:07.500 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2023-11-09T03:46:07.500 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-09T03:46:07.500 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-09T03:46:07.501 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-09T03:46:07.501 DEBUG:teuthology.orchestra.run.smithi113:> ') 2023-11-09T03:46:07.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:07.791 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2023-11-09T03:46:07.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:07.806 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list-id 2023-11-09T03:46:07.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:07.861 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2023-11-09T03:46:07.861 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2023-11-09T03:46:07.861 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-09T03:46:07.861 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-09T03:46:07.862 DEBUG:teuthology.orchestra.run.smithi113:> ') 2023-11-09T03:46:07.946 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-09T03:46:07.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:07.946 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2023-11-09T03:46:07.947 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2023-11-09T03:46:07.947 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-09T03:46:07.947 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-09T03:46:07.948 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-09T03:46:07.948 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-09T03:46:07.948 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-09T03:46:07.948 DEBUG:teuthology.orchestra.run.smithi113:> ') 2023-11-09T03:46:08.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:08.049 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2023-11-09T03:46:08.049 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2023-11-09T03:46:08.050 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2023-11-09T03:46:08.050 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set dev brx.0 master ceph-brx 2023-11-09T03:46:08.050 DEBUG:teuthology.orchestra.run.smithi113:> ') 2023-11-09T03:46:08.092 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-09T03:46:08.093 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:46:08.093 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-09T03:46:08.147 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-09T03:46:08.147 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:46:08.148 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-09T03:46:08.202 DEBUG:teuthology.orchestra.run.smithi113:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:46:08.257 INFO:teuthology.orchestra.run.smithi113.stdout:/proc 2023-11-09T03:46:08.257 INFO:teuthology.orchestra.run.smithi113.stdout:/sys 2023-11-09T03:46:08.257 INFO:teuthology.orchestra.run.smithi113.stdout:/dev 2023-11-09T03:46:08.258 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/security 2023-11-09T03:46:08.258 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/shm 2023-11-09T03:46:08.258 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/pts 2023-11-09T03:46:08.258 INFO:teuthology.orchestra.run.smithi113.stdout:/run 2023-11-09T03:46:08.258 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/cgroup 2023-11-09T03:46:08.259 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/pstore 2023-11-09T03:46:08.259 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/bpf 2023-11-09T03:46:08.259 INFO:teuthology.orchestra.run.smithi113.stdout:/ 2023-11-09T03:46:08.259 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/selinux 2023-11-09T03:46:08.259 INFO:teuthology.orchestra.run.smithi113.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:46:08.259 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/hugepages 2023-11-09T03:46:08.260 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/mqueue 2023-11-09T03:46:08.260 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/debug 2023-11-09T03:46:08.260 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/tracing 2023-11-09T03:46:08.260 INFO:teuthology.orchestra.run.smithi113.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:46:08.260 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/fuse/connections 2023-11-09T03:46:08.260 INFO:teuthology.orchestra.run.smithi113.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:46:08.261 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/config 2023-11-09T03:46:08.261 INFO:teuthology.orchestra.run.smithi113.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:46:08.261 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:46:08.261 INFO:teuthology.orchestra.run.smithi113.stdout:/run/user/1000 2023-11-09T03:46:08.261 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph 2023-11-09T03:46:08.262 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-4 2023-11-09T03:46:08.262 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-5 2023-11-09T03:46:08.262 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-6 2023-11-09T03:46:08.262 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-7 2023-11-09T03:46:08.262 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns 2023-11-09T03:46:08.262 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-09T03:46:08.263 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-09T03:46:08.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:08.263 DEBUG:teuthology.orchestra.run.smithi113:> ls /sys/fs/fuse/connections 2023-11-09T03:46:08.311 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-09T03:46:08.311 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-09T03:46:08.353 DEBUG:teuthology.orchestra.run.smithi113:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:46:08.369 INFO:teuthology.orchestra.run.smithi113.stdout:/proc 2023-11-09T03:46:08.369 INFO:teuthology.orchestra.run.smithi113.stdout:/sys 2023-11-09T03:46:08.369 INFO:teuthology.orchestra.run.smithi113.stdout:/dev 2023-11-09T03:46:08.369 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/security 2023-11-09T03:46:08.369 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/shm 2023-11-09T03:46:08.370 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/pts 2023-11-09T03:46:08.370 INFO:teuthology.orchestra.run.smithi113.stdout:/run 2023-11-09T03:46:08.370 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/cgroup 2023-11-09T03:46:08.370 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/pstore 2023-11-09T03:46:08.370 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/bpf 2023-11-09T03:46:08.371 INFO:teuthology.orchestra.run.smithi113.stdout:/ 2023-11-09T03:46:08.371 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/selinux 2023-11-09T03:46:08.371 INFO:teuthology.orchestra.run.smithi113.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:46:08.371 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/hugepages 2023-11-09T03:46:08.371 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/mqueue 2023-11-09T03:46:08.372 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/debug 2023-11-09T03:46:08.372 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/tracing 2023-11-09T03:46:08.372 INFO:teuthology.orchestra.run.smithi113.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:46:08.372 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/fuse/connections 2023-11-09T03:46:08.372 INFO:teuthology.orchestra.run.smithi113.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:46:08.372 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/config 2023-11-09T03:46:08.373 INFO:teuthology.orchestra.run.smithi113.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:46:08.373 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:46:08.373 INFO:teuthology.orchestra.run.smithi113.stdout:/run/user/1000 2023-11-09T03:46:08.373 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph 2023-11-09T03:46:08.373 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-4 2023-11-09T03:46:08.374 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-5 2023-11-09T03:46:08.374 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-6 2023-11-09T03:46:08.374 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-7 2023-11-09T03:46:08.374 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns 2023-11-09T03:46:08.374 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-09T03:46:08.375 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-09T03:46:08.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:08.375 DEBUG:teuthology.orchestra.run.smithi113:> ls /sys/fs/fuse/connections 2023-11-09T03:46:08.456 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi113.stderr:2023-11-09T03:46:08.457+0000 7f351df33480 -1 init, newargv = 0x55f828c34a00 newargc=15 2023-11-09T03:46:08.457 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi113.stderr:ceph-fuse[93700]: starting ceph client 2023-11-09T03:46:08.461 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi113.stderr:ceph-fuse[93700]: starting fuse 2023-11-09T03:46:09.525 DEBUG:teuthology.orchestra.run.smithi113:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:46:09.547 INFO:teuthology.orchestra.run.smithi113.stdout:/proc 2023-11-09T03:46:09.547 INFO:teuthology.orchestra.run.smithi113.stdout:/sys 2023-11-09T03:46:09.547 INFO:teuthology.orchestra.run.smithi113.stdout:/dev 2023-11-09T03:46:09.547 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/security 2023-11-09T03:46:09.547 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/shm 2023-11-09T03:46:09.548 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/pts 2023-11-09T03:46:09.548 INFO:teuthology.orchestra.run.smithi113.stdout:/run 2023-11-09T03:46:09.548 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/cgroup 2023-11-09T03:46:09.548 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/pstore 2023-11-09T03:46:09.548 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/bpf 2023-11-09T03:46:09.548 INFO:teuthology.orchestra.run.smithi113.stdout:/ 2023-11-09T03:46:09.549 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/selinux 2023-11-09T03:46:09.549 INFO:teuthology.orchestra.run.smithi113.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:46:09.549 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/hugepages 2023-11-09T03:46:09.549 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/mqueue 2023-11-09T03:46:09.549 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/debug 2023-11-09T03:46:09.549 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/tracing 2023-11-09T03:46:09.550 INFO:teuthology.orchestra.run.smithi113.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:46:09.550 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/fuse/connections 2023-11-09T03:46:09.550 INFO:teuthology.orchestra.run.smithi113.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:46:09.550 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/config 2023-11-09T03:46:09.550 INFO:teuthology.orchestra.run.smithi113.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:46:09.551 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:46:09.551 INFO:teuthology.orchestra.run.smithi113.stdout:/run/user/1000 2023-11-09T03:46:09.551 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph 2023-11-09T03:46:09.551 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-4 2023-11-09T03:46:09.551 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-5 2023-11-09T03:46:09.551 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-6 2023-11-09T03:46:09.552 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-7 2023-11-09T03:46:09.552 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns 2023-11-09T03:46:09.552 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-09T03:46:09.552 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-09T03:46:09.552 INFO:teuthology.orchestra.run.smithi113.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-09T03:46:09.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:09.553 DEBUG:teuthology.orchestra.run.smithi113:> ls /sys/fs/fuse/connections 2023-11-09T03:46:09.604 INFO:teuthology.orchestra.run.smithi113.stdout:41 2023-11-09T03:46:09.605 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-09T03:46:09.605 DEBUG:teuthology.orchestra.run.smithi113:> sudo stdin-killer -- python3 -c ' 2023-11-09T03:46:09.605 DEBUG:teuthology.orchestra.run.smithi113:> import glob 2023-11-09T03:46:09.606 DEBUG:teuthology.orchestra.run.smithi113:> import re 2023-11-09T03:46:09.606 DEBUG:teuthology.orchestra.run.smithi113:> import os 2023-11-09T03:46:09.606 DEBUG:teuthology.orchestra.run.smithi113:> import subprocess 2023-11-09T03:46:09.606 DEBUG:teuthology.orchestra.run.smithi113:> 2023-11-09T03:46:09.607 DEBUG:teuthology.orchestra.run.smithi113:> def _find_admin_socket(client_name): 2023-11-09T03:46:09.607 DEBUG:teuthology.orchestra.run.smithi113:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-09T03:46:09.607 DEBUG:teuthology.orchestra.run.smithi113:> files = glob.glob(asok_path) 2023-11-09T03:46:09.608 DEBUG:teuthology.orchestra.run.smithi113:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-09T03:46:09.608 DEBUG:teuthology.orchestra.run.smithi113:> 2023-11-09T03:46:09.608 DEBUG:teuthology.orchestra.run.smithi113:> # Given a non-glob path, it better be there 2023-11-09T03:46:09.608 DEBUG:teuthology.orchestra.run.smithi113:> if "*" not in asok_path: 2023-11-09T03:46:09.609 DEBUG:teuthology.orchestra.run.smithi113:> assert(len(files) == 1) 2023-11-09T03:46:09.609 DEBUG:teuthology.orchestra.run.smithi113:> return files[0] 2023-11-09T03:46:09.609 DEBUG:teuthology.orchestra.run.smithi113:> 2023-11-09T03:46:09.610 DEBUG:teuthology.orchestra.run.smithi113:> for f in files: 2023-11-09T03:46:09.610 DEBUG:teuthology.orchestra.run.smithi113:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-09T03:46:09.610 DEBUG:teuthology.orchestra.run.smithi113:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-09T03:46:09.611 DEBUG:teuthology.orchestra.run.smithi113:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-09T03:46:09.611 DEBUG:teuthology.orchestra.run.smithi113:> contents = proc_f.read() 2023-11-09T03:46:09.611 DEBUG:teuthology.orchestra.run.smithi113:> if mountpoint in contents: 2023-11-09T03:46:09.611 DEBUG:teuthology.orchestra.run.smithi113:> return f 2023-11-09T03:46:09.612 DEBUG:teuthology.orchestra.run.smithi113:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-09T03:46:09.612 DEBUG:teuthology.orchestra.run.smithi113:> 2023-11-09T03:46:09.612 DEBUG:teuthology.orchestra.run.smithi113:> print(_find_admin_socket("client.1")) 2023-11-09T03:46:09.613 DEBUG:teuthology.orchestra.run.smithi113:> ' 2023-11-09T03:46:09.732 INFO:teuthology.orchestra.run.smithi113.stdout:/var/run/ceph/ceph-client.1.93700.asok 2023-11-09T03:46:09.736 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:46:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:46:09.744 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.93700.asok 2023-11-09T03:46:09.744 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:46:09.745 DEBUG:teuthology.orchestra.run.smithi113:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.1.93700.asok status 2023-11-09T03:46:09.843 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:46:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:46:09.913 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2023-11-09T03:46:09.913 INFO:teuthology.orchestra.run.smithi113.stdout: "metadata": { 2023-11-09T03:46:09.914 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_sha1": "7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4", 2023-11-09T03:46:09.914 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_version": "ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev)", 2023-11-09T03:46:09.914 INFO:teuthology.orchestra.run.smithi113.stdout: "entity_id": "1", 2023-11-09T03:46:09.914 INFO:teuthology.orchestra.run.smithi113.stdout: "hostname": "smithi113", 2023-11-09T03:46:09.914 INFO:teuthology.orchestra.run.smithi113.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-09T03:46:09.915 INFO:teuthology.orchestra.run.smithi113.stdout: "pid": "93700", 2023-11-09T03:46:09.915 INFO:teuthology.orchestra.run.smithi113.stdout: "root": "/" 2023-11-09T03:46:09.915 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2023-11-09T03:46:09.915 INFO:teuthology.orchestra.run.smithi113.stdout: "dentry_count": 0, 2023-11-09T03:46:09.915 INFO:teuthology.orchestra.run.smithi113.stdout: "dentry_pinned_count": 0, 2023-11-09T03:46:09.916 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 4780, 2023-11-09T03:46:09.916 INFO:teuthology.orchestra.run.smithi113.stdout: "inst": { 2023-11-09T03:46:09.916 INFO:teuthology.orchestra.run.smithi113.stdout: "name": { 2023-11-09T03:46:09.916 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "client", 2023-11-09T03:46:09.916 INFO:teuthology.orchestra.run.smithi113.stdout: "num": 4780 2023-11-09T03:46:09.917 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2023-11-09T03:46:09.917 INFO:teuthology.orchestra.run.smithi113.stdout: "addr": { 2023-11-09T03:46:09.917 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "v1", 2023-11-09T03:46:09.917 INFO:teuthology.orchestra.run.smithi113.stdout: "addr": "192.168.0.1:0", 2023-11-09T03:46:09.917 INFO:teuthology.orchestra.run.smithi113.stdout: "nonce": 2633118316 2023-11-09T03:46:09.917 INFO:teuthology.orchestra.run.smithi113.stdout: } 2023-11-09T03:46:09.918 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2023-11-09T03:46:09.918 INFO:teuthology.orchestra.run.smithi113.stdout: "addr": { 2023-11-09T03:46:09.918 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "v1", 2023-11-09T03:46:09.918 INFO:teuthology.orchestra.run.smithi113.stdout: "addr": "192.168.0.1:0", 2023-11-09T03:46:09.919 INFO:teuthology.orchestra.run.smithi113.stdout: "nonce": 2633118316 2023-11-09T03:46:09.919 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2023-11-09T03:46:09.919 INFO:teuthology.orchestra.run.smithi113.stdout: "inst_str": "client.4780 192.168.0.1:0/2633118316", 2023-11-09T03:46:09.919 INFO:teuthology.orchestra.run.smithi113.stdout: "addr_str": "192.168.0.1:0/2633118316", 2023-11-09T03:46:09.919 INFO:teuthology.orchestra.run.smithi113.stdout: "inode_count": 1, 2023-11-09T03:46:09.920 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_epoch": 9, 2023-11-09T03:46:09.920 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_epoch": 25, 2023-11-09T03:46:09.920 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_epoch_barrier": 0, 2023-11-09T03:46:09.920 INFO:teuthology.orchestra.run.smithi113.stdout: "blocklisted": false, 2023-11-09T03:46:09.920 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs" 2023-11-09T03:46:09.920 INFO:teuthology.orchestra.run.smithi113.stdout:} 2023-11-09T03:46:09.921 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:46:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:46:09.927 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-09T03:46:10.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.037+0000 7f22584ec640 1 Processor -- start 2023-11-09T03:46:10.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.037+0000 7f22584ec640 1 -- start start 2023-11-09T03:46:10.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.038+0000 7f22584ec640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f22500cda60 0x7f22500ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:10.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.038+0000 7f22584ec640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22500cc990 con 0x7f22500b0cd0 2023-11-09T03:46:10.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.038+0000 7f22584ec640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22500ccc70 con 0x7f2250070d40 2023-11-09T03:46:10.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.038+0000 7f22584ec640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22500c9ce0 con 0x7f22500cda60 2023-11-09T03:46:10.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.038+0000 7f2255a87640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f22500b0cd0 0x7f2250070630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46800/0 (socket says 172.21.15.76:46800) 2023-11-09T03:46:10.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.038+0000 7f2255a87640 1 -- 172.21.15.76:0/2852150514 learned_addr learned my addr 172.21.15.76:0/2852150514 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:10.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.038+0000 7f2255286640 1 -- 172.21.15.76:0/2852150514 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2668715049 0 0) 0x7f22500cc990 con 0x7f22500b0cd0 2023-11-09T03:46:10.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.038+0000 7f2256288640 1 --2- 172.21.15.76:0/2852150514 >> v2:172.21.15.113:3300/0 conn(0x7f22500cda60 0x7f22500ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:10.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.038+0000 7f2255286640 1 -- 172.21.15.76:0/2852150514 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f223c004630 con 0x7f22500b0cd0 2023-11-09T03:46:10.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.038+0000 7f2255286640 1 -- 172.21.15.76:0/2852150514 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 537113859 0 0) 0x7f223c004630 con 0x7f22500b0cd0 2023-11-09T03:46:10.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.038+0000 7f2255286640 1 -- 172.21.15.76:0/2852150514 >> v2:172.21.15.113:3300/0 conn(0x7f22500cda60 msgr2=0x7f22500ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:10.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.038+0000 7f2255286640 1 --2- 172.21.15.76:0/2852150514 >> v2:172.21.15.113:3300/0 conn(0x7f22500cda60 0x7f22500ca7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:10.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.038+0000 7f2255286640 1 -- 172.21.15.76:0/2852150514 >> v1:172.21.15.113:6789/0 conn(0x7f2250070d40 legacy=0x7f225006c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:10.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.038+0000 7f2255286640 1 -- 172.21.15.76:0/2852150514 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22500c9fc0 con 0x7f22500b0cd0 2023-11-09T03:46:10.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.038+0000 7f2255286640 1 -- 172.21.15.76:0/2852150514 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f2240001670 con 0x7f22500b0cd0 2023-11-09T03:46:10.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.038+0000 7f2255286640 1 -- 172.21.15.76:0/2852150514 >> v1:172.21.15.76:6789/0 conn(0x7f22500b0cd0 legacy=0x7f2250070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:10.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.038+0000 7f2255286640 1 --2- 172.21.15.76:0/2852150514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f223c004a80 0x7f223c004e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:10.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.039+0000 7f2255286640 1 --2- 172.21.15.76:0/2852150514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f223c005480 0x7f223c005950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:10.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.039+0000 7f2255286640 1 --2- 172.21.15.76:0/2852150514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f223c006150 0x7f223c003520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:10.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.039+0000 7f2255286640 1 -- 172.21.15.76:0/2852150514 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22500c9ce0 con 0x7f223c006150 2023-11-09T03:46:10.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.039+0000 7f2255286640 1 -- 172.21.15.76:0/2852150514 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223c003b90 con 0x7f223c004a80 2023-11-09T03:46:10.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.039+0000 7f2255286640 1 -- 172.21.15.76:0/2852150514 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f223c003d50 con 0x7f223c005480 2023-11-09T03:46:10.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.039+0000 7f2256a89640 1 --2- 172.21.15.76:0/2852150514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f223c006150 0x7f223c003520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:10.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.039+0000 7f2256288640 1 --2- 172.21.15.76:0/2852150514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f223c004a80 0x7f223c004e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:10.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.039+0000 7f2255a87640 1 --2- 172.21.15.76:0/2852150514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f223c005480 0x7f223c005950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:10.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.039+0000 7f2256a89640 1 -- 172.21.15.76:0/2852150514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f223c005480 msgr2=0x7f223c005950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:10.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.039+0000 7f2256a89640 1 --2- 172.21.15.76:0/2852150514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f223c005480 0x7f223c005950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:10.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.039+0000 7f2256a89640 1 -- 172.21.15.76:0/2852150514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f223c004a80 msgr2=0x7f223c004e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:10.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.039+0000 7f2256a89640 1 --2- 172.21.15.76:0/2852150514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f223c004a80 0x7f223c004e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:10.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.039+0000 7f2256a89640 1 -- 172.21.15.76:0/2852150514 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223c004630 con 0x7f223c006150 2023-11-09T03:46:10.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.040+0000 7f2256a89640 1 -- 172.21.15.76:0/2852150514 reap_dead start 2023-11-09T03:46:10.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.040+0000 7f2256a89640 1 --2- 172.21.15.76:0/2852150514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f223c006150 0x7f223c003520 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2248001f60 tx=0x7f223c006900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26ee08650f58531 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:10.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.040+0000 7f2255286640 1 -- 172.21.15.76:0/2852150514 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2248003b70 con 0x7f223c006150 2023-11-09T03:46:10.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.040+0000 7f2255286640 1 -- 172.21.15.76:0/2852150514 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2248003cf0 con 0x7f223c006150 2023-11-09T03:46:10.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.040+0000 7f2255286640 1 -- 172.21.15.76:0/2852150514 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22480052b0 con 0x7f223c006150 2023-11-09T03:46:10.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.040+0000 7f22584ec640 1 -- 172.21.15.76:0/2852150514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f223c006150 msgr2=0x7f223c003520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:10.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.040+0000 7f22584ec640 1 --2- 172.21.15.76:0/2852150514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f223c006150 0x7f223c003520 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2248001f60 tx=0x7f223c006900 comp rx=0 tx=0).stop 2023-11-09T03:46:10.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.040+0000 7f22584ec640 1 -- 172.21.15.76:0/2852150514 shutdown_connections 2023-11-09T03:46:10.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.040+0000 7f22584ec640 1 --2- 172.21.15.76:0/2852150514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f223c006150 0x7f223c003520 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:10.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.040+0000 7f22584ec640 1 -- 172.21.15.76:0/2852150514 >> 172.21.15.76:0/2852150514 conn(0x7f22500ce950 msgr2=0x7f22500ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:10.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.040+0000 7f22584ec640 1 -- 172.21.15.76:0/2852150514 shutdown_connections 2023-11-09T03:46:10.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.040+0000 7f22584ec640 1 -- 172.21.15.76:0/2852150514 wait complete. 2023-11-09T03:46:10.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.042+0000 7f22584ec640 1 Processor -- start 2023-11-09T03:46:10.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.042+0000 7f22584ec640 1 -- start start 2023-11-09T03:46:10.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.042+0000 7f22584ec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f223c006150 0x7f22500b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:10.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.042+0000 7f22584ec640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2250157b30 0x7f22501ee0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:10.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.042+0000 7f22584ec640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2250157350 0x7f225015a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:10.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.042+0000 7f22584ec640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22500c6e40 con 0x7f223c006150 2023-11-09T03:46:10.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.042+0000 7f22584ec640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22501ee600 con 0x7f2250157b30 2023-11-09T03:46:10.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.042+0000 7f22584ec640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22500c7050 con 0x7f2250157350 2023-11-09T03:46:10.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.042+0000 7f2256288640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f223c006150 0x7f22500b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:10.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.042+0000 7f2256288640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f223c006150 0x7f22500b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54318/0 (socket says 172.21.15.76:54318) 2023-11-09T03:46:10.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.042+0000 7f2256288640 1 -- 172.21.15.76:0/1172671450 learned_addr learned my addr 172.21.15.76:0/1172671450 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:10.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.043+0000 7f2255a87640 1 --2- 172.21.15.76:0/1172671450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2250157b30 0x7f22501ee0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:10.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.043+0000 7f2256a89640 1 --2- 172.21.15.76:0/1172671450 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2250157350 0x7f225015a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:10.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.043+0000 7f2256288640 1 -- 172.21.15.76:0/1172671450 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2250157350 msgr2=0x7f225015a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:10.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.043+0000 7f2256288640 1 --2- 172.21.15.76:0/1172671450 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2250157350 0x7f225015a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:10.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.043+0000 7f2256288640 1 -- 172.21.15.76:0/1172671450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2250157b30 msgr2=0x7f22501ee0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:10.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.043+0000 7f2256288640 1 --2- 172.21.15.76:0/1172671450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2250157b30 0x7f22501ee0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:10.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.043+0000 7f2256288640 1 -- 172.21.15.76:0/1172671450 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22500c7330 con 0x7f223c006150 2023-11-09T03:46:10.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.043+0000 7f2256288640 1 --2- 172.21.15.76:0/1172671450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f223c006150 0x7f22500b3fd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f224c011910 tx=0x7f224c011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:10.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.043+0000 7f2246ffd640 1 -- 172.21.15.76:0/1172671450 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c00fa30 con 0x7f223c006150 2023-11-09T03:46:10.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.043+0000 7f2246ffd640 1 -- 172.21.15.76:0/1172671450 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f224c00fb90 con 0x7f223c006150 2023-11-09T03:46:10.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.043+0000 7f22584ec640 1 -- 172.21.15.76:0/1172671450 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22500c7610 con 0x7f223c006150 2023-11-09T03:46:10.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.043+0000 7f2246ffd640 1 -- 172.21.15.76:0/1172671450 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c014440 con 0x7f223c006150 2023-11-09T03:46:10.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.043+0000 7f22584ec640 1 -- 172.21.15.76:0/1172671450 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22501582b0 con 0x7f223c006150 2023-11-09T03:46:10.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.044+0000 7f22584ec640 1 -- 172.21.15.76:0/1172671450 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2250074290 con 0x7f223c006150 2023-11-09T03:46:10.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.045+0000 7f2246ffd640 1 -- 172.21.15.76:0/1172671450 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f224c0145a0 con 0x7f223c006150 2023-11-09T03:46:10.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.045+0000 7f2246ffd640 1 --2- 172.21.15.76:0/1172671450 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f223406aea0 0x7f223406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:10.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.045+0000 7f2246ffd640 1 -- 172.21.15.76:0/1172671450 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f224c08a0a0 con 0x7f223c006150 2023-11-09T03:46:10.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.045+0000 7f2255a87640 1 --2- 172.21.15.76:0/1172671450 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f223406aea0 0x7f223406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:10.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.045+0000 7f2255a87640 1 --2- 172.21.15.76:0/1172671450 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f223406aea0 0x7f223406d360 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2250070d10 tx=0x7f2240000ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:10.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.049+0000 7f2246ffd640 1 -- 172.21.15.76:0/1172671450 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f224c054dd0 con 0x7f223c006150 2023-11-09T03:46:10.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.250+0000 7f22584ec640 1 -- 172.21.15.76:0/1172671450 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f22500b48d0 con 0x7f223c006150 2023-11-09T03:46:10.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.250+0000 7f2246ffd640 1 -- 172.21.15.76:0/1172671450 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f224c0583f0 con 0x7f223c006150 2023-11-09T03:46:10.251 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-09T03:46:10.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.252+0000 7f22584ec640 1 -- 172.21.15.76:0/1172671450 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f223406aea0 msgr2=0x7f223406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:10.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.252+0000 7f22584ec640 1 --2- 172.21.15.76:0/1172671450 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f223406aea0 0x7f223406d360 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2250070d10 tx=0x7f2240000ed0 comp rx=0 tx=0).stop 2023-11-09T03:46:10.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.252+0000 7f22584ec640 1 -- 172.21.15.76:0/1172671450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f223c006150 msgr2=0x7f22500b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:10.253 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.252+0000 7f22584ec640 1 --2- 172.21.15.76:0/1172671450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f223c006150 0x7f22500b3fd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f224c011910 tx=0x7f224c011d10 comp rx=0 tx=0).stop 2023-11-09T03:46:10.253 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.252+0000 7f22584ec640 1 -- 172.21.15.76:0/1172671450 shutdown_connections 2023-11-09T03:46:10.253 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.252+0000 7f22584ec640 1 --2- 172.21.15.76:0/1172671450 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f223406aea0 0x7f223406d360 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:10.253 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.252+0000 7f22584ec640 1 --2- 172.21.15.76:0/1172671450 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2250157350 0x7f225015a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:10.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.252+0000 7f22584ec640 1 --2- 172.21.15.76:0/1172671450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2250157b30 0x7f22501ee0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:10.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.252+0000 7f22584ec640 1 --2- 172.21.15.76:0/1172671450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f223c006150 0x7f22500b3fd0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:10.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.252+0000 7f22584ec640 1 -- 172.21.15.76:0/1172671450 >> 172.21.15.76:0/1172671450 conn(0x7f22500ce950 msgr2=0x7f2250159a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:10.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.252+0000 7f22584ec640 1 -- 172.21.15.76:0/1172671450 shutdown_connections 2023-11-09T03:46:10.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:10.252+0000 7f22584ec640 1 -- 172.21.15.76:0/1172671450 wait complete. 2023-11-09T03:46:10.263 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-09T03:46:10.263 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-09T03:46:10.263 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2023-11-09T03:46:10.263 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-09T03:46:10.264 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-09T03:46:10.264 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-09T03:46:10.264 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-09T03:46:10.264 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-09T03:46:10.265 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-09T03:46:10.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:10.265 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2023-11-09T03:46:10.281 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-09T03:46:10.281 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-09T03:46:10.281 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2023-11-09T03:46:10.282 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:10.282 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2023-11-09T03:46:10.282 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:10.282 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:46:10.283 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:46:10.283 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2023-11-09T03:46:10.283 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:46:10.283 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:46:10.283 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2023-11-09T03:46:10.284 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:46:10.284 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:46:10.284 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-09T03:46:10.284 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:46:10.284 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-09T03:46:10.285 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42606sec preferred_lft 42606sec 2023-11-09T03:46:10.285 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::ec4:7aff:fe8f:d7c1/64 scope link 2023-11-09T03:46:10.285 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:10.285 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-09T03:46:10.285 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5e:34:15:bb:dc:aa brd ff:ff:ff:ff:ff:ff 2023-11-09T03:46:10.286 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-09T03:46:10.286 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:10.286 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::64b3:2bff:fed8:4277/64 scope link 2023-11-09T03:46:10.286 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:10.286 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-09T03:46:10.287 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5e:34:15:bb:dc:aa brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-09T03:46:10.287 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::5c34:15ff:febb:dcaa/64 scope link 2023-11-09T03:46:10.287 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:10.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:10.288 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-09T03:46:10.347 INFO:teuthology.orchestra.run.smithi113.stdout:1 2023-11-09T03:46:10.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:10.349 DEBUG:teuthology.orchestra.run.smithi113:> route 2023-11-09T03:46:10.404 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2023-11-09T03:46:10.404 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-09T03:46:10.404 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-09T03:46:10.404 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-09T03:46:10.404 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-09T03:46:10.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:10.405 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2023-11-09T03:46:10.405 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2023-11-09T03:46:10.406 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-09T03:46:10.406 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-09T03:46:10.406 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-09T03:46:10.406 DEBUG:teuthology.orchestra.run.smithi113:> ') 2023-11-09T03:46:10.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:10.499 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2023-11-09T03:46:10.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-09T03:46:10.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:10.515 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list-id 2023-11-09T03:46:10.569 INFO:teuthology.orchestra.run.smithi113.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-09T03:46:10.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:10.570 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2023-11-09T03:46:10.571 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2023-11-09T03:46:10.571 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-09T03:46:10.571 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-09T03:46:10.571 DEBUG:teuthology.orchestra.run.smithi113:> ') 2023-11-09T03:46:10.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:10.652 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-09T03:46:10.718 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-09T03:46:10.719 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-09T03:46:10.719 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2023-11-09T03:46:10.719 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:10.719 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2023-11-09T03:46:10.720 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:10.720 INFO:teuthology.orchestra.run.smithi113.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-09T03:46:10.720 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5a:bd:de:ec:4f:5b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-09T03:46:10.720 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-09T03:46:10.720 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:10.721 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::58bd:deff:feec:4f5b/64 scope link 2023-11-09T03:46:10.721 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:10.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:10.722 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-09T03:46:10.788 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-09T03:46:10.788 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-09T03:46:10.789 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2023-11-09T03:46:10.789 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:10.789 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2023-11-09T03:46:10.789 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:10.789 INFO:teuthology.orchestra.run.smithi113.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-09T03:46:10.790 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5a:bd:de:ec:4f:5b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-09T03:46:10.790 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-09T03:46:10.790 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:10.790 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::58bd:deff:feec:4f5b/64 scope link 2023-11-09T03:46:10.790 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:10.791 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-09T03:46:10.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:10.792 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2023-11-09T03:46:10.793 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2023-11-09T03:46:10.793 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-09T03:46:10.793 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-09T03:46:10.793 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-09T03:46:10.794 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-09T03:46:10.794 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-09T03:46:10.794 DEBUG:teuthology.orchestra.run.smithi113:> ') 2023-11-09T03:46:10.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:10.915 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2023-11-09T03:46:10.916 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2023-11-09T03:46:10.916 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.1 up 2023-11-09T03:46:10.916 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set dev brx.1 master ceph-brx 2023-11-09T03:46:10.916 DEBUG:teuthology.orchestra.run.smithi113:> ') 2023-11-09T03:46:10.957 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-11-09T03:46:10.958 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:46:10.958 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-09T03:46:11.012 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-09T03:46:11.013 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:46:11.013 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-09T03:46:11.067 DEBUG:teuthology.orchestra.run.smithi113:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:46:11.121 INFO:teuthology.orchestra.run.smithi113.stdout:/proc 2023-11-09T03:46:11.122 INFO:teuthology.orchestra.run.smithi113.stdout:/sys 2023-11-09T03:46:11.122 INFO:teuthology.orchestra.run.smithi113.stdout:/dev 2023-11-09T03:46:11.122 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/security 2023-11-09T03:46:11.122 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/shm 2023-11-09T03:46:11.123 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/pts 2023-11-09T03:46:11.123 INFO:teuthology.orchestra.run.smithi113.stdout:/run 2023-11-09T03:46:11.123 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/cgroup 2023-11-09T03:46:11.123 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/pstore 2023-11-09T03:46:11.123 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/bpf 2023-11-09T03:46:11.124 INFO:teuthology.orchestra.run.smithi113.stdout:/ 2023-11-09T03:46:11.124 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/selinux 2023-11-09T03:46:11.124 INFO:teuthology.orchestra.run.smithi113.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:46:11.124 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/hugepages 2023-11-09T03:46:11.124 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/mqueue 2023-11-09T03:46:11.125 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/debug 2023-11-09T03:46:11.125 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/tracing 2023-11-09T03:46:11.125 INFO:teuthology.orchestra.run.smithi113.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:46:11.125 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/fuse/connections 2023-11-09T03:46:11.125 INFO:teuthology.orchestra.run.smithi113.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:46:11.126 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/config 2023-11-09T03:46:11.126 INFO:teuthology.orchestra.run.smithi113.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:46:11.126 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:46:11.126 INFO:teuthology.orchestra.run.smithi113.stdout:/run/user/1000 2023-11-09T03:46:11.126 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph 2023-11-09T03:46:11.126 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-4 2023-11-09T03:46:11.127 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-5 2023-11-09T03:46:11.127 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-6 2023-11-09T03:46:11.127 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-7 2023-11-09T03:46:11.127 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns 2023-11-09T03:46:11.127 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-09T03:46:11.127 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-09T03:46:11.128 INFO:teuthology.orchestra.run.smithi113.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-09T03:46:11.128 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-09T03:46:11.128 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-09T03:46:11.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:11.129 DEBUG:teuthology.orchestra.run.smithi113:> ls /sys/fs/fuse/connections 2023-11-09T03:46:11.176 INFO:teuthology.orchestra.run.smithi113.stdout:41 2023-11-09T03:46:11.176 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2023-11-09T03:46:11.177 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-11-09T03:46:11.218 DEBUG:teuthology.orchestra.run.smithi113:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:46:11.232 INFO:teuthology.orchestra.run.smithi113.stdout:/proc 2023-11-09T03:46:11.233 INFO:teuthology.orchestra.run.smithi113.stdout:/sys 2023-11-09T03:46:11.233 INFO:teuthology.orchestra.run.smithi113.stdout:/dev 2023-11-09T03:46:11.233 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/security 2023-11-09T03:46:11.233 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/shm 2023-11-09T03:46:11.233 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/pts 2023-11-09T03:46:11.234 INFO:teuthology.orchestra.run.smithi113.stdout:/run 2023-11-09T03:46:11.234 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/cgroup 2023-11-09T03:46:11.234 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/pstore 2023-11-09T03:46:11.234 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/bpf 2023-11-09T03:46:11.234 INFO:teuthology.orchestra.run.smithi113.stdout:/ 2023-11-09T03:46:11.235 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/selinux 2023-11-09T03:46:11.235 INFO:teuthology.orchestra.run.smithi113.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:46:11.235 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/hugepages 2023-11-09T03:46:11.235 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/mqueue 2023-11-09T03:46:11.235 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/debug 2023-11-09T03:46:11.235 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/tracing 2023-11-09T03:46:11.236 INFO:teuthology.orchestra.run.smithi113.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:46:11.236 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/fuse/connections 2023-11-09T03:46:11.236 INFO:teuthology.orchestra.run.smithi113.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:46:11.236 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/config 2023-11-09T03:46:11.236 INFO:teuthology.orchestra.run.smithi113.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:46:11.237 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:46:11.237 INFO:teuthology.orchestra.run.smithi113.stdout:/run/user/1000 2023-11-09T03:46:11.237 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph 2023-11-09T03:46:11.237 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-4 2023-11-09T03:46:11.237 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-5 2023-11-09T03:46:11.237 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-6 2023-11-09T03:46:11.238 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-7 2023-11-09T03:46:11.238 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns 2023-11-09T03:46:11.238 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-09T03:46:11.238 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-09T03:46:11.238 INFO:teuthology.orchestra.run.smithi113.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-09T03:46:11.239 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-09T03:46:11.239 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-09T03:46:11.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:11.239 DEBUG:teuthology.orchestra.run.smithi113:> ls /sys/fs/fuse/connections 2023-11-09T03:46:11.288 INFO:teuthology.orchestra.run.smithi113.stdout:41 2023-11-09T03:46:11.309 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi113.stderr:2023-11-09T03:46:11.310+0000 7f77aa20c480 -1 init, newargv = 0x55af09eca010 newargc=15 2023-11-09T03:46:11.309 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi113.stderr:ceph-fuse[94152]: starting ceph client 2023-11-09T03:46:11.313 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi113.stderr:ceph-fuse[94152]: starting fuse 2023-11-09T03:46:12.390 DEBUG:teuthology.orchestra.run.smithi113:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:46:12.410 INFO:teuthology.orchestra.run.smithi113.stdout:/proc 2023-11-09T03:46:12.411 INFO:teuthology.orchestra.run.smithi113.stdout:/sys 2023-11-09T03:46:12.411 INFO:teuthology.orchestra.run.smithi113.stdout:/dev 2023-11-09T03:46:12.411 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/security 2023-11-09T03:46:12.411 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/shm 2023-11-09T03:46:12.411 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/pts 2023-11-09T03:46:12.412 INFO:teuthology.orchestra.run.smithi113.stdout:/run 2023-11-09T03:46:12.412 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/cgroup 2023-11-09T03:46:12.412 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/pstore 2023-11-09T03:46:12.412 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/bpf 2023-11-09T03:46:12.412 INFO:teuthology.orchestra.run.smithi113.stdout:/ 2023-11-09T03:46:12.413 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/selinux 2023-11-09T03:46:12.413 INFO:teuthology.orchestra.run.smithi113.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:46:12.413 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/hugepages 2023-11-09T03:46:12.413 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/mqueue 2023-11-09T03:46:12.413 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/debug 2023-11-09T03:46:12.414 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/tracing 2023-11-09T03:46:12.414 INFO:teuthology.orchestra.run.smithi113.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:46:12.414 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/fuse/connections 2023-11-09T03:46:12.414 INFO:teuthology.orchestra.run.smithi113.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:46:12.414 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/config 2023-11-09T03:46:12.415 INFO:teuthology.orchestra.run.smithi113.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:46:12.415 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:46:12.415 INFO:teuthology.orchestra.run.smithi113.stdout:/run/user/1000 2023-11-09T03:46:12.415 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph 2023-11-09T03:46:12.415 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-4 2023-11-09T03:46:12.415 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-5 2023-11-09T03:46:12.416 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-6 2023-11-09T03:46:12.416 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-7 2023-11-09T03:46:12.416 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns 2023-11-09T03:46:12.416 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-09T03:46:12.416 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-09T03:46:12.417 INFO:teuthology.orchestra.run.smithi113.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-09T03:46:12.417 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-09T03:46:12.417 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-09T03:46:12.417 INFO:teuthology.orchestra.run.smithi113.stdout:/home/ubuntu/cephtest/mnt.3 2023-11-09T03:46:12.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:12.418 DEBUG:teuthology.orchestra.run.smithi113:> ls /sys/fs/fuse/connections 2023-11-09T03:46:12.467 INFO:teuthology.orchestra.run.smithi113.stdout:41 2023-11-09T03:46:12.468 INFO:teuthology.orchestra.run.smithi113.stdout:44 2023-11-09T03:46:12.468 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 44] 2023-11-09T03:46:12.468 DEBUG:teuthology.orchestra.run.smithi113:> sudo stdin-killer -- python3 -c ' 2023-11-09T03:46:12.469 DEBUG:teuthology.orchestra.run.smithi113:> import glob 2023-11-09T03:46:12.469 DEBUG:teuthology.orchestra.run.smithi113:> import re 2023-11-09T03:46:12.469 DEBUG:teuthology.orchestra.run.smithi113:> import os 2023-11-09T03:46:12.469 DEBUG:teuthology.orchestra.run.smithi113:> import subprocess 2023-11-09T03:46:12.470 DEBUG:teuthology.orchestra.run.smithi113:> 2023-11-09T03:46:12.470 DEBUG:teuthology.orchestra.run.smithi113:> def _find_admin_socket(client_name): 2023-11-09T03:46:12.470 DEBUG:teuthology.orchestra.run.smithi113:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-11-09T03:46:12.471 DEBUG:teuthology.orchestra.run.smithi113:> files = glob.glob(asok_path) 2023-11-09T03:46:12.471 DEBUG:teuthology.orchestra.run.smithi113:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-11-09T03:46:12.471 DEBUG:teuthology.orchestra.run.smithi113:> 2023-11-09T03:46:12.471 DEBUG:teuthology.orchestra.run.smithi113:> # Given a non-glob path, it better be there 2023-11-09T03:46:12.472 DEBUG:teuthology.orchestra.run.smithi113:> if "*" not in asok_path: 2023-11-09T03:46:12.472 DEBUG:teuthology.orchestra.run.smithi113:> assert(len(files) == 1) 2023-11-09T03:46:12.472 DEBUG:teuthology.orchestra.run.smithi113:> return files[0] 2023-11-09T03:46:12.472 DEBUG:teuthology.orchestra.run.smithi113:> 2023-11-09T03:46:12.473 DEBUG:teuthology.orchestra.run.smithi113:> for f in files: 2023-11-09T03:46:12.473 DEBUG:teuthology.orchestra.run.smithi113:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-09T03:46:12.473 DEBUG:teuthology.orchestra.run.smithi113:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-09T03:46:12.474 DEBUG:teuthology.orchestra.run.smithi113:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-09T03:46:12.474 DEBUG:teuthology.orchestra.run.smithi113:> contents = proc_f.read() 2023-11-09T03:46:12.474 DEBUG:teuthology.orchestra.run.smithi113:> if mountpoint in contents: 2023-11-09T03:46:12.474 DEBUG:teuthology.orchestra.run.smithi113:> return f 2023-11-09T03:46:12.475 DEBUG:teuthology.orchestra.run.smithi113:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-09T03:46:12.475 DEBUG:teuthology.orchestra.run.smithi113:> 2023-11-09T03:46:12.475 DEBUG:teuthology.orchestra.run.smithi113:> print(_find_admin_socket("client.3")) 2023-11-09T03:46:12.475 DEBUG:teuthology.orchestra.run.smithi113:> ' 2023-11-09T03:46:12.602 INFO:teuthology.orchestra.run.smithi113.stdout:/var/run/ceph/ceph-client.3.94152.asok 2023-11-09T03:46:12.605 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:46:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:46:12.612 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.94152.asok 2023-11-09T03:46:12.613 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:46:12.613 DEBUG:teuthology.orchestra.run.smithi113:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.3.94152.asok status 2023-11-09T03:46:12.712 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:46:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:46:12.783 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2023-11-09T03:46:12.783 INFO:teuthology.orchestra.run.smithi113.stdout: "metadata": { 2023-11-09T03:46:12.783 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_sha1": "7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4", 2023-11-09T03:46:12.784 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_version": "ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev)", 2023-11-09T03:46:12.784 INFO:teuthology.orchestra.run.smithi113.stdout: "entity_id": "3", 2023-11-09T03:46:12.784 INFO:teuthology.orchestra.run.smithi113.stdout: "hostname": "smithi113", 2023-11-09T03:46:12.784 INFO:teuthology.orchestra.run.smithi113.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-11-09T03:46:12.785 INFO:teuthology.orchestra.run.smithi113.stdout: "pid": "94152", 2023-11-09T03:46:12.785 INFO:teuthology.orchestra.run.smithi113.stdout: "root": "/" 2023-11-09T03:46:12.785 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2023-11-09T03:46:12.785 INFO:teuthology.orchestra.run.smithi113.stdout: "dentry_count": 0, 2023-11-09T03:46:12.785 INFO:teuthology.orchestra.run.smithi113.stdout: "dentry_pinned_count": 0, 2023-11-09T03:46:12.785 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 4604, 2023-11-09T03:46:12.786 INFO:teuthology.orchestra.run.smithi113.stdout: "inst": { 2023-11-09T03:46:12.786 INFO:teuthology.orchestra.run.smithi113.stdout: "name": { 2023-11-09T03:46:12.786 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "client", 2023-11-09T03:46:12.786 INFO:teuthology.orchestra.run.smithi113.stdout: "num": 4604 2023-11-09T03:46:12.786 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2023-11-09T03:46:12.787 INFO:teuthology.orchestra.run.smithi113.stdout: "addr": { 2023-11-09T03:46:12.787 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "v1", 2023-11-09T03:46:12.787 INFO:teuthology.orchestra.run.smithi113.stdout: "addr": "192.168.0.2:0", 2023-11-09T03:46:12.787 INFO:teuthology.orchestra.run.smithi113.stdout: "nonce": 1027454171 2023-11-09T03:46:12.787 INFO:teuthology.orchestra.run.smithi113.stdout: } 2023-11-09T03:46:12.788 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2023-11-09T03:46:12.788 INFO:teuthology.orchestra.run.smithi113.stdout: "addr": { 2023-11-09T03:46:12.788 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "v1", 2023-11-09T03:46:12.788 INFO:teuthology.orchestra.run.smithi113.stdout: "addr": "192.168.0.2:0", 2023-11-09T03:46:12.788 INFO:teuthology.orchestra.run.smithi113.stdout: "nonce": 1027454171 2023-11-09T03:46:12.789 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2023-11-09T03:46:12.789 INFO:teuthology.orchestra.run.smithi113.stdout: "inst_str": "client.4604 192.168.0.2:0/1027454171", 2023-11-09T03:46:12.789 INFO:teuthology.orchestra.run.smithi113.stdout: "addr_str": "192.168.0.2:0/1027454171", 2023-11-09T03:46:12.789 INFO:teuthology.orchestra.run.smithi113.stdout: "inode_count": 1, 2023-11-09T03:46:12.789 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_epoch": 9, 2023-11-09T03:46:12.790 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_epoch": 25, 2023-11-09T03:46:12.790 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_epoch_barrier": 0, 2023-11-09T03:46:12.790 INFO:teuthology.orchestra.run.smithi113.stdout: "blocklisted": false, 2023-11-09T03:46:12.790 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs" 2023-11-09T03:46:12.790 INFO:teuthology.orchestra.run.smithi113.stdout:} 2023-11-09T03:46:12.791 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:46:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:46:12.798 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:46:12.798 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-11-09T03:46:12.798 DEBUG:teuthology.orchestra.run.smithi076:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-09T03:46:12.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:46:12.869 INFO:teuthology.orchestra.run.smithi076.stdout:fuseblk 2023-11-09T03:46:12.869 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:46:12.877 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-09T03:46:12.877 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:46:12.878 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-09T03:46:12.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:46:12.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:46:12.951 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:46:12.952 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-11-09T03:46:12.952 DEBUG:teuthology.orchestra.run.smithi076:> ' -- /home/ubuntu/cephtest/mnt.2 2023-11-09T03:46:13.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:46:13.013 INFO:teuthology.orchestra.run.smithi076.stdout:fuseblk 2023-11-09T03:46:13.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:46:13.020 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-11-09T03:46:13.020 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:46:13.020 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-09T03:46:13.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:46:13.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:46:13.086 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:46:13.087 DEBUG:teuthology.orchestra.run.smithi113:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-11-09T03:46:13.087 DEBUG:teuthology.orchestra.run.smithi113:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-09T03:46:13.146 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:46:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:46:13.147 INFO:teuthology.orchestra.run.smithi113.stdout:fuseblk 2023-11-09T03:46:13.147 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:46:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:46:13.154 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-09T03:46:13.154 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:46:13.154 DEBUG:teuthology.orchestra.run.smithi113:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-09T03:46:13.212 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:46:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:46:13.213 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:46:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:46:13.219 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:46:13.219 DEBUG:teuthology.orchestra.run.smithi113:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-11-09T03:46:13.219 DEBUG:teuthology.orchestra.run.smithi113:> ' -- /home/ubuntu/cephtest/mnt.3 2023-11-09T03:46:13.277 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:46:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:46:13.278 INFO:teuthology.orchestra.run.smithi113.stdout:fuseblk 2023-11-09T03:46:13.278 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:46:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:46:13.284 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-11-09T03:46:13.285 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:46:13.285 DEBUG:teuthology.orchestra.run.smithi113:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-09T03:46:13.343 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:46:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:46:13.344 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:46:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:46:13.350 INFO:teuthology.run_tasks:Running task exec... 2023-11-09T03:46:13.360 INFO:teuthology.task.exec:Executing custom commands... 2023-11-09T03:46:13.361 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T03:46:13.361 DEBUG:teuthology.orchestra.run.smithi076:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/crash/warn_recent_interval 0' 2023-11-09T03:46:13.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.463+0000 7f8492539640 1 Processor -- start 2023-11-09T03:46:13.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.463+0000 7f8492539640 1 -- start start 2023-11-09T03:46:13.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.463+0000 7f8492539640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f848c15d0d0 0x7f848c0cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:13.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.463+0000 7f8492539640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f848c15d4b0 0x7f848c0c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:13.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.463+0000 7f8492539640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f848c06b450 con 0x7f848c0c55d0 2023-11-09T03:46:13.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.463+0000 7f8492539640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f848c0ce480 con 0x7f848c15d0d0 2023-11-09T03:46:13.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.463+0000 7f8492539640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f848c06b6f0 con 0x7f848c15d4b0 2023-11-09T03:46:13.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.464+0000 7f848bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f848c15d0d0 0x7f848c0cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:13.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.464+0000 7f8490ad6640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f848c0c55d0 0x7f848c15d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43564/0 (socket says 172.21.15.76:43564) 2023-11-09T03:46:13.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.464+0000 7f8490ad6640 1 -- 172.21.15.76:0/2471649447 learned_addr learned my addr 172.21.15.76:0/2471649447 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:13.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.464+0000 7f848b7fe640 1 --2- 172.21.15.76:0/2471649447 >> v2:172.21.15.113:3300/0 conn(0x7f848c15d4b0 0x7f848c0c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:13.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.464+0000 7f848bfff640 1 -- 172.21.15.76:0/2471649447 >> v2:172.21.15.113:3300/0 conn(0x7f848c15d4b0 msgr2=0x7f848c0c9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:13.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.464+0000 7f848bfff640 1 --2- 172.21.15.76:0/2471649447 >> v2:172.21.15.113:3300/0 conn(0x7f848c15d4b0 0x7f848c0c9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:13.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.464+0000 7f848bfff640 1 -- 172.21.15.76:0/2471649447 >> v1:172.21.15.76:6789/0 conn(0x7f848c0c55d0 legacy=0x7f848c15d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:13.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.464+0000 7f848bfff640 1 -- 172.21.15.76:0/2471649447 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848c0ce760 con 0x7f848c15d0d0 2023-11-09T03:46:13.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.464+0000 7f848affd640 1 -- 172.21.15.76:0/2471649447 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2632951672 0 0) 0x7f848c06b450 con 0x7f848c0c55d0 2023-11-09T03:46:13.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.465+0000 7f848bfff640 1 --2- 172.21.15.76:0/2471649447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f848c15d0d0 0x7f848c0cc590 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f848c0cfac0 tx=0x7f847c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd904bcee2ae58f server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:13.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.465+0000 7f848affd640 1 -- 172.21.15.76:0/2471649447 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c031830 con 0x7f848c15d0d0 2023-11-09T03:46:13.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.465+0000 7f848affd640 1 -- 172.21.15.76:0/2471649447 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847c031990 con 0x7f848c15d0d0 2023-11-09T03:46:13.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.465+0000 7f848affd640 1 -- 172.21.15.76:0/2471649447 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c031d20 con 0x7f848c15d0d0 2023-11-09T03:46:13.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.465+0000 7f8492539640 1 -- 172.21.15.76:0/2471649447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f848c15d0d0 msgr2=0x7f848c0cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:13.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.465+0000 7f8492539640 1 --2- 172.21.15.76:0/2471649447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f848c15d0d0 0x7f848c0cc590 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f848c0cfac0 tx=0x7f847c02b8d0 comp rx=0 tx=0).stop 2023-11-09T03:46:13.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.465+0000 7f8492539640 1 -- 172.21.15.76:0/2471649447 shutdown_connections 2023-11-09T03:46:13.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.465+0000 7f8492539640 1 --2- 172.21.15.76:0/2471649447 >> v2:172.21.15.113:3300/0 conn(0x7f848c15d4b0 0x7f848c0c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:13.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.465+0000 7f8492539640 1 --2- 172.21.15.76:0/2471649447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f848c15d0d0 0x7f848c0cc590 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:13.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.465+0000 7f8492539640 1 -- 172.21.15.76:0/2471649447 >> 172.21.15.76:0/2471649447 conn(0x7f848c0ce9c0 msgr2=0x7f848c0cedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:13.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.465+0000 7f8492539640 1 -- 172.21.15.76:0/2471649447 shutdown_connections 2023-11-09T03:46:13.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.465+0000 7f8492539640 1 -- 172.21.15.76:0/2471649447 wait complete. 2023-11-09T03:46:13.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.466+0000 7f8492539640 1 Processor -- start 2023-11-09T03:46:13.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.467+0000 7f8492539640 1 -- start start 2023-11-09T03:46:13.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.467+0000 7f8492539640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f848c0c55d0 0x7f848c148e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:13.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.467+0000 7f8492539640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f848c15d0d0 0x7f848c0ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:13.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.467+0000 7f8492539640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f848c15d4b0 0x7f848c0c7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:13.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.467+0000 7f8492539640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c0af7d0 con 0x7f848c15d4b0 2023-11-09T03:46:13.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.467+0000 7f8492539640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c149390 con 0x7f848c0c55d0 2023-11-09T03:46:13.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.467+0000 7f8492539640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f848c0aeb50 con 0x7f848c15d0d0 2023-11-09T03:46:13.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.468+0000 7f8490ad6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f848c15d4b0 0x7f848c0c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:13.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.468+0000 7f8490ad6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f848c15d4b0 0x7f848c0c7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36614/0 (socket says 172.21.15.76:36614) 2023-11-09T03:46:13.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.468+0000 7f8490ad6640 1 -- 172.21.15.76:0/3192867388 learned_addr learned my addr 172.21.15.76:0/3192867388 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:13.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.468+0000 7f848bfff640 1 --2- 172.21.15.76:0/3192867388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f848c0c55d0 0x7f848c148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:13.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.468+0000 7f848b7fe640 1 --2- 172.21.15.76:0/3192867388 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f848c15d0d0 0x7f848c0ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:13.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.468+0000 7f8490ad6640 1 -- 172.21.15.76:0/3192867388 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f848c15d0d0 msgr2=0x7f848c0ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:13.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.468+0000 7f8490ad6640 1 --2- 172.21.15.76:0/3192867388 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f848c15d0d0 0x7f848c0ae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:13.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.468+0000 7f8490ad6640 1 -- 172.21.15.76:0/3192867388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f848c0c55d0 msgr2=0x7f848c148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:13.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.468+0000 7f8490ad6640 1 --2- 172.21.15.76:0/3192867388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f848c0c55d0 0x7f848c148e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:13.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.468+0000 7f8490ad6640 1 -- 172.21.15.76:0/3192867388 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847c009400 con 0x7f848c15d4b0 2023-11-09T03:46:13.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.468+0000 7f8490ad6640 1 --2- 172.21.15.76:0/3192867388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f848c15d4b0 0x7f848c0c7480 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8484001560 tx=0x7f848400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:13.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.468+0000 7f8488ff9640 1 -- 172.21.15.76:0/3192867388 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848400a640 con 0x7f848c15d4b0 2023-11-09T03:46:13.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.468+0000 7f8488ff9640 1 -- 172.21.15.76:0/3192867388 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848400a7a0 con 0x7f848c15d4b0 2023-11-09T03:46:13.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.468+0000 7f8492539640 1 -- 172.21.15.76:0/3192867388 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f848c0aee30 con 0x7f848c15d4b0 2023-11-09T03:46:13.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.468+0000 7f8488ff9640 1 -- 172.21.15.76:0/3192867388 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848400ab30 con 0x7f848c15d4b0 2023-11-09T03:46:13.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.468+0000 7f8492539640 1 -- 172.21.15.76:0/3192867388 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f848c0af3f0 con 0x7f848c15d4b0 2023-11-09T03:46:13.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.469+0000 7f8492539640 1 -- 172.21.15.76:0/3192867388 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f848c04ece0 con 0x7f848c15d4b0 2023-11-09T03:46:13.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.470+0000 7f8488ff9640 1 -- 172.21.15.76:0/3192867388 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f848400ac90 con 0x7f848c15d4b0 2023-11-09T03:46:13.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.470+0000 7f8488ff9640 1 --2- 172.21.15.76:0/3192867388 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f847806ae70 0x7f847806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:13.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.470+0000 7f8488ff9640 1 -- 172.21.15.76:0/3192867388 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f84840890c0 con 0x7f848c15d4b0 2023-11-09T03:46:13.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.470+0000 7f848bfff640 1 --2- 172.21.15.76:0/3192867388 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f847806ae70 0x7f847806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:13.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.471+0000 7f848bfff640 1 --2- 172.21.15.76:0/3192867388 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f847806ae70 0x7f847806d330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f847c036000 tx=0x7f847c03c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:13.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.475+0000 7f8488ff9640 1 -- 172.21.15.76:0/3192867388 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8484053ed0 con 0x7f848c15d4b0 2023-11-09T03:46:13.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.641+0000 7f8492539640 1 -- 172.21.15.76:0/3192867388 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command([{prefix=config set, name=mgr/crash/warn_recent_interval}] v 0) v1 -- 0x7f848c06d620 con 0x7f848c15d4b0 2023-11-09T03:46:13.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.646+0000 7f8488ff9640 1 -- 172.21.15.76:0/3192867388 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/crash/warn_recent_interval}]=0 v10)=0 v10) v1 ==== 128+0+0 (secure 0 0 0) 0x7f84840180c0 con 0x7f848c15d4b0 2023-11-09T03:46:13.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.648+0000 7f8492539640 1 -- 172.21.15.76:0/3192867388 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f847806ae70 msgr2=0x7f847806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:13.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.648+0000 7f8492539640 1 --2- 172.21.15.76:0/3192867388 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f847806ae70 0x7f847806d330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f847c036000 tx=0x7f847c03c040 comp rx=0 tx=0).stop 2023-11-09T03:46:13.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.648+0000 7f8492539640 1 -- 172.21.15.76:0/3192867388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f848c15d4b0 msgr2=0x7f848c0c7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:13.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.648+0000 7f8492539640 1 --2- 172.21.15.76:0/3192867388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f848c15d4b0 0x7f848c0c7480 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8484001560 tx=0x7f848400cc70 comp rx=0 tx=0).stop 2023-11-09T03:46:13.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.648+0000 7f8492539640 1 -- 172.21.15.76:0/3192867388 shutdown_connections 2023-11-09T03:46:13.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.648+0000 7f8492539640 1 --2- 172.21.15.76:0/3192867388 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f847806ae70 0x7f847806d330 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:13.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.648+0000 7f8492539640 1 --2- 172.21.15.76:0/3192867388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f848c15d4b0 0x7f848c0c7480 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:13.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.648+0000 7f8492539640 1 --2- 172.21.15.76:0/3192867388 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f848c15d0d0 0x7f848c0ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:13.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.648+0000 7f8492539640 1 --2- 172.21.15.76:0/3192867388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f848c0c55d0 0x7f848c148e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:13.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.648+0000 7f8492539640 1 -- 172.21.15.76:0/3192867388 >> 172.21.15.76:0/3192867388 conn(0x7f848c0ce9c0 msgr2=0x7f848c066090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:13.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.648+0000 7f8492539640 1 -- 172.21.15.76:0/3192867388 shutdown_connections 2023-11-09T03:46:13.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.648+0000 7f8492539640 1 -- 172.21.15.76:0/3192867388 wait complete. 2023-11-09T03:46:13.659 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-09T03:46:13.706 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:46:13.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.806+0000 7f52d15a9640 1 Processor -- start 2023-11-09T03:46:13.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.806+0000 7f52d15a9640 1 -- start start 2023-11-09T03:46:13.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.807+0000 7f52d15a9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f52cc15ce60 0x7f52cc06b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:13.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.807+0000 7f52d15a9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f52cc06bd30 0x7f52cc160b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:13.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.807+0000 7f52d15a9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52cc1615d0 con 0x7f52cc161050 2023-11-09T03:46:13.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.807+0000 7f52d15a9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52cc0cbde0 con 0x7f52cc15ce60 2023-11-09T03:46:13.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.807+0000 7f52d15a9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52cc1618b0 con 0x7f52cc06bd30 2023-11-09T03:46:13.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.807+0000 7f52caffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f52cc15ce60 0x7f52cc06b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:13.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.807+0000 7f52cb7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f52cc161050 0x7f52cc0cf160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43570/0 (socket says 172.21.15.76:43570) 2023-11-09T03:46:13.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.807+0000 7f52caffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f52cc15ce60 0x7f52cc06b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36618/0 (socket says 172.21.15.76:36618) 2023-11-09T03:46:13.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.807+0000 7f52cb7fe640 1 -- 172.21.15.76:0/4224679870 learned_addr learned my addr 172.21.15.76:0/4224679870 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:13.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.807+0000 7f52ca7fc640 1 --2- 172.21.15.76:0/4224679870 >> v2:172.21.15.113:3300/0 conn(0x7f52cc06bd30 0x7f52cc160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:13.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.807+0000 7f52caffd640 1 -- 172.21.15.76:0/4224679870 >> v2:172.21.15.113:3300/0 conn(0x7f52cc06bd30 msgr2=0x7f52cc160b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:13.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.807+0000 7f52caffd640 1 --2- 172.21.15.76:0/4224679870 >> v2:172.21.15.113:3300/0 conn(0x7f52cc06bd30 0x7f52cc160b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:13.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.807+0000 7f52caffd640 1 -- 172.21.15.76:0/4224679870 >> v1:172.21.15.76:6789/0 conn(0x7f52cc161050 legacy=0x7f52cc0cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:13.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.807+0000 7f52caffd640 1 -- 172.21.15.76:0/4224679870 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52cc0cba00 con 0x7f52cc15ce60 2023-11-09T03:46:13.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.807+0000 7f52c9ffb640 1 -- 172.21.15.76:0/4224679870 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 838306737 0 0) 0x7f52cc1615d0 con 0x7f52cc161050 2023-11-09T03:46:13.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.808+0000 7f52caffd640 1 --2- 172.21.15.76:0/4224679870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52cc15ce60 0x7f52cc06b7f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f52cc072590 tx=0x7f52bc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d87d8ac2d2557b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:13.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.808+0000 7f52c9ffb640 1 -- 172.21.15.76:0/4224679870 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc031830 con 0x7f52cc15ce60 2023-11-09T03:46:13.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.808+0000 7f52c9ffb640 1 -- 172.21.15.76:0/4224679870 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc031990 con 0x7f52cc15ce60 2023-11-09T03:46:13.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.808+0000 7f52c9ffb640 1 -- 172.21.15.76:0/4224679870 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc031d20 con 0x7f52cc15ce60 2023-11-09T03:46:13.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.809+0000 7f52d15a9640 1 -- 172.21.15.76:0/4224679870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52cc15ce60 msgr2=0x7f52cc06b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:13.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.809+0000 7f52d15a9640 1 --2- 172.21.15.76:0/4224679870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52cc15ce60 0x7f52cc06b7f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f52cc072590 tx=0x7f52bc02b8d0 comp rx=0 tx=0).stop 2023-11-09T03:46:13.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.809+0000 7f52d15a9640 1 -- 172.21.15.76:0/4224679870 shutdown_connections 2023-11-09T03:46:13.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.809+0000 7f52d15a9640 1 --2- 172.21.15.76:0/4224679870 >> v2:172.21.15.113:3300/0 conn(0x7f52cc06bd30 0x7f52cc160b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:13.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.809+0000 7f52d15a9640 1 --2- 172.21.15.76:0/4224679870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52cc15ce60 0x7f52cc06b7f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:13.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.809+0000 7f52d15a9640 1 -- 172.21.15.76:0/4224679870 >> 172.21.15.76:0/4224679870 conn(0x7f52cc06ea90 msgr2=0x7f52cc06ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:13.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.809+0000 7f52d15a9640 1 -- 172.21.15.76:0/4224679870 shutdown_connections 2023-11-09T03:46:13.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.809+0000 7f52d15a9640 1 -- 172.21.15.76:0/4224679870 wait complete. 2023-11-09T03:46:13.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.810+0000 7f52d15a9640 1 Processor -- start 2023-11-09T03:46:13.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.810+0000 7f52d15a9640 1 -- start start 2023-11-09T03:46:13.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.811+0000 7f52d15a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52cc15ce60 0x7f52cc15a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:13.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.811+0000 7f52d15a9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52cc161050 0x7f52cc1f66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:13.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.811+0000 7f52d15a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52cc0cf160 0x7f52cc0c6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:13.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.811+0000 7f52d15a9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc1f6230 con 0x7f52cc161050 2023-11-09T03:46:13.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.811+0000 7f52d15a9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc0ca560 con 0x7f52cc0cf160 2023-11-09T03:46:13.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.811+0000 7f52d15a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc0ca840 con 0x7f52cc15ce60 2023-11-09T03:46:13.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.811+0000 7f52ca7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52cc161050 0x7f52cc1f66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:13.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.811+0000 7f52ca7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52cc161050 0x7f52cc1f66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36626/0 (socket says 172.21.15.76:36626) 2023-11-09T03:46:13.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.811+0000 7f52ca7fc640 1 -- 172.21.15.76:0/2164966872 learned_addr learned my addr 172.21.15.76:0/2164966872 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:13.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.811+0000 7f52caffd640 1 --2- 172.21.15.76:0/2164966872 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52cc15ce60 0x7f52cc15a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:13.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.811+0000 7f52cb7fe640 1 --2- 172.21.15.76:0/2164966872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52cc0cf160 0x7f52cc0c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:13.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.811+0000 7f52ca7fc640 1 -- 172.21.15.76:0/2164966872 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52cc15ce60 msgr2=0x7f52cc15a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:13.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.811+0000 7f52ca7fc640 1 --2- 172.21.15.76:0/2164966872 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52cc15ce60 0x7f52cc15a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:13.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.811+0000 7f52ca7fc640 1 -- 172.21.15.76:0/2164966872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52cc0cf160 msgr2=0x7f52cc0c6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:13.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.811+0000 7f52ca7fc640 1 --2- 172.21.15.76:0/2164966872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52cc0cf160 0x7f52cc0c6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:13.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.811+0000 7f52ca7fc640 1 -- 172.21.15.76:0/2164966872 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52bc009400 con 0x7f52cc161050 2023-11-09T03:46:13.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.811+0000 7f52cb7fe640 1 --2- 172.21.15.76:0/2164966872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52cc0cf160 0x7f52cc0c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:13.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.811+0000 7f52ca7fc640 1 --2- 172.21.15.76:0/2164966872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52cc161050 0x7f52cc1f66c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f52c000d5f0 tx=0x7f52c000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:13.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.811+0000 7f52b37fe640 1 -- 172.21.15.76:0/2164966872 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c00125f0 con 0x7f52cc161050 2023-11-09T03:46:13.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.812+0000 7f52b37fe640 1 -- 172.21.15.76:0/2164966872 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c0012750 con 0x7f52cc161050 2023-11-09T03:46:13.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.812+0000 7f52d15a9640 1 -- 172.21.15.76:0/2164966872 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52cc1f5be0 con 0x7f52cc161050 2023-11-09T03:46:13.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.812+0000 7f52b37fe640 1 -- 172.21.15.76:0/2164966872 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c0012b00 con 0x7f52cc161050 2023-11-09T03:46:13.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.812+0000 7f52d15a9640 1 -- 172.21.15.76:0/2164966872 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52cc0ca1a0 con 0x7f52cc161050 2023-11-09T03:46:13.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.813+0000 7f52d15a9640 1 -- 172.21.15.76:0/2164966872 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52cc1f63a0 con 0x7f52cc161050 2023-11-09T03:46:13.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.813+0000 7f52b37fe640 1 -- 172.21.15.76:0/2164966872 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f52c0012c60 con 0x7f52cc161050 2023-11-09T03:46:13.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.813+0000 7f52b37fe640 1 --2- 172.21.15.76:0/2164966872 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f52b806aea0 0x7f52b806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:13.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.813+0000 7f52b37fe640 1 -- 172.21.15.76:0/2164966872 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f52c0089ec0 con 0x7f52cc161050 2023-11-09T03:46:13.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.813+0000 7f52caffd640 1 --2- 172.21.15.76:0/2164966872 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f52b806aea0 0x7f52b806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:13.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.814+0000 7f52caffd640 1 --2- 172.21.15.76:0/2164966872 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f52b806aea0 0x7f52b806d360 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f52bc036000 tx=0x7f52bc03c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:13.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.817+0000 7f52b37fe640 1 -- 172.21.15.76:0/2164966872 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f52c0054b90 con 0x7f52cc161050 2023-11-09T03:46:13.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.984+0000 7f52d15a9640 1 -- 172.21.15.76:0/2164966872 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52cc074290 con 0x7f52cc161050 2023-11-09T03:46:13.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.985+0000 7f52b37fe640 1 -- 172.21.15.76:0/2164966872 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18710 (secure 0 0 0) 0x7f52c00581b0 con 0x7f52cc161050 2023-11-09T03:46:13.986 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:13.986 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":27,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:46:12.537265+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:46:13.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.987+0000 7f52d15a9640 1 -- 172.21.15.76:0/2164966872 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f52b806aea0 msgr2=0x7f52b806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:13.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.987+0000 7f52d15a9640 1 --2- 172.21.15.76:0/2164966872 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f52b806aea0 0x7f52b806d360 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f52bc036000 tx=0x7f52bc03c040 comp rx=0 tx=0).stop 2023-11-09T03:46:13.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.987+0000 7f52d15a9640 1 -- 172.21.15.76:0/2164966872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52cc161050 msgr2=0x7f52cc1f66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:13.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.987+0000 7f52d15a9640 1 --2- 172.21.15.76:0/2164966872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52cc161050 0x7f52cc1f66c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f52c000d5f0 tx=0x7f52c000dab0 comp rx=0 tx=0).stop 2023-11-09T03:46:13.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.987+0000 7f52d15a9640 1 -- 172.21.15.76:0/2164966872 shutdown_connections 2023-11-09T03:46:13.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.987+0000 7f52d15a9640 1 --2- 172.21.15.76:0/2164966872 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f52b806aea0 0x7f52b806d360 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:13.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.987+0000 7f52d15a9640 1 --2- 172.21.15.76:0/2164966872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52cc0cf160 0x7f52cc0c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:13.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.987+0000 7f52d15a9640 1 --2- 172.21.15.76:0/2164966872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52cc161050 0x7f52cc1f66c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:13.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.987+0000 7f52d15a9640 1 --2- 172.21.15.76:0/2164966872 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52cc15ce60 0x7f52cc15a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:13.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.987+0000 7f52d15a9640 1 -- 172.21.15.76:0/2164966872 >> 172.21.15.76:0/2164966872 conn(0x7f52cc06ea90 msgr2=0x7f52cc15be50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:13.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.987+0000 7f52d15a9640 1 -- 172.21.15.76:0/2164966872 shutdown_connections 2023-11-09T03:46:13.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:13.988+0000 7f52d15a9640 1 -- 172.21.15.76:0/2164966872 wait complete. 2023-11-09T03:46:13.999 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:45:28.704715+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:45:29.717614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-09T03:45:33.440577+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-09T03:46:14.000 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:46:14.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.102+0000 7fc83bb7c640 1 Processor -- start 2023-11-09T03:46:14.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.102+0000 7fc83bb7c640 1 -- start start 2023-11-09T03:46:14.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.103+0000 7fc83bb7c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc8340b4880 0x7fc8340b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:14.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.103+0000 7fc83bb7c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8340cde60 con 0x7fc8340cd6d0 2023-11-09T03:46:14.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.103+0000 7fc83bb7c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc834160f90 con 0x7fc834153e60 2023-11-09T03:46:14.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.103+0000 7fc83bb7c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8340b5200 con 0x7fc8340b4880 2023-11-09T03:46:14.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.103+0000 7fc83a119640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc8340b4880 0x7fc8340b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:14.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.103+0000 7fc839117640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc8340cd6d0 0x7fc834161420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43578/0 (socket says 172.21.15.76:43578) 2023-11-09T03:46:14.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.103+0000 7fc839117640 1 -- 172.21.15.76:0/72409349 learned_addr learned my addr 172.21.15.76:0/72409349 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:14.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.103+0000 7fc83a119640 1 -- 172.21.15.76:0/72409349 >> v1:172.21.15.113:6789/0 conn(0x7fc834153e60 legacy=0x7fc83415dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:14.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.103+0000 7fc83a119640 1 -- 172.21.15.76:0/72409349 >> v1:172.21.15.76:6789/0 conn(0x7fc8340cd6d0 legacy=0x7fc834161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:14.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.103+0000 7fc838916640 1 -- 172.21.15.76:0/72409349 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1214213608 0 0) 0x7fc8340cde60 con 0x7fc8340cd6d0 2023-11-09T03:46:14.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.103+0000 7fc83a119640 1 -- 172.21.15.76:0/72409349 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8340ce090 con 0x7fc8340b4880 2023-11-09T03:46:14.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.104+0000 7fc83a119640 1 --2- 172.21.15.76:0/72409349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8340b4880 0x7fc8340b4cc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009d90 tx=0x7fc82c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784d875872ad2cba server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:14.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.104+0000 7fc838916640 1 -- 172.21.15.76:0/72409349 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82c02cd00 con 0x7fc8340b4880 2023-11-09T03:46:14.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.104+0000 7fc838916640 1 -- 172.21.15.76:0/72409349 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82c02ce60 con 0x7fc8340b4880 2023-11-09T03:46:14.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.104+0000 7fc838916640 1 -- 172.21.15.76:0/72409349 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82c034660 con 0x7fc8340b4880 2023-11-09T03:46:14.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.105+0000 7fc83bb7c640 1 -- 172.21.15.76:0/72409349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8340b4880 msgr2=0x7fc8340b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:14.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.105+0000 7fc83bb7c640 1 --2- 172.21.15.76:0/72409349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8340b4880 0x7fc8340b4cc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009d90 tx=0x7fc82c02bf50 comp rx=0 tx=0).stop 2023-11-09T03:46:14.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.105+0000 7fc83bb7c640 1 -- 172.21.15.76:0/72409349 shutdown_connections 2023-11-09T03:46:14.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.105+0000 7fc83bb7c640 1 --2- 172.21.15.76:0/72409349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8340b4880 0x7fc8340b4cc0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:14.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.105+0000 7fc83bb7c640 1 -- 172.21.15.76:0/72409349 >> 172.21.15.76:0/72409349 conn(0x7fc8340cf880 msgr2=0x7fc8340cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:14.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.105+0000 7fc83bb7c640 1 -- 172.21.15.76:0/72409349 shutdown_connections 2023-11-09T03:46:14.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.105+0000 7fc83bb7c640 1 -- 172.21.15.76:0/72409349 wait complete. 2023-11-09T03:46:14.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.106+0000 7fc83bb7c640 1 Processor -- start 2023-11-09T03:46:14.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.106+0000 7fc83bb7c640 1 -- start start 2023-11-09T03:46:14.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.106+0000 7fc83bb7c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8340b4880 0x7fc834156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:14.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.106+0000 7fc83bb7c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8340cd6d0 0x7fc83415a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:14.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.107+0000 7fc83bb7c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc834153e60 0x7fc83415b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:14.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.107+0000 7fc83bb7c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8340b7e70 con 0x7fc834153e60 2023-11-09T03:46:14.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.107+0000 7fc83bb7c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc834159c30 con 0x7fc8340b4880 2023-11-09T03:46:14.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.107+0000 7fc83bb7c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc834159f10 con 0x7fc8340cd6d0 2023-11-09T03:46:14.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.107+0000 7fc83a119640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc834153e60 0x7fc83415b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:14.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.107+0000 7fc83a119640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc834153e60 0x7fc83415b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36642/0 (socket says 172.21.15.76:36642) 2023-11-09T03:46:14.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.107+0000 7fc83a119640 1 -- 172.21.15.76:0/1509790528 learned_addr learned my addr 172.21.15.76:0/1509790528 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:14.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.107+0000 7fc839918640 1 --2- 172.21.15.76:0/1509790528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8340b4880 0x7fc834156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:14.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.107+0000 7fc83a119640 1 -- 172.21.15.76:0/1509790528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8340cd6d0 msgr2=0x7fc83415a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:14.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.107+0000 7fc83a119640 1 --2- 172.21.15.76:0/1509790528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8340cd6d0 0x7fc83415a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:14.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.107+0000 7fc83a119640 1 -- 172.21.15.76:0/1509790528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8340b4880 msgr2=0x7fc834156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:14.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.107+0000 7fc83a119640 1 --2- 172.21.15.76:0/1509790528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8340b4880 0x7fc834156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:14.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.107+0000 7fc83a119640 1 -- 172.21.15.76:0/1509790528 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82c009a70 con 0x7fc834153e60 2023-11-09T03:46:14.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.107+0000 7fc839918640 1 --2- 172.21.15.76:0/1509790528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8340b4880 0x7fc834156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:14.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.107+0000 7fc83a119640 1 --2- 172.21.15.76:0/1509790528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc834153e60 0x7fc83415b8c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009d90 tx=0x7fc82c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:14.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.107+0000 7fc82a7fc640 1 -- 172.21.15.76:0/1509790528 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82c03d600 con 0x7fc834153e60 2023-11-09T03:46:14.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.107+0000 7fc82a7fc640 1 -- 172.21.15.76:0/1509790528 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82c03d780 con 0x7fc834153e60 2023-11-09T03:46:14.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.107+0000 7fc83bb7c640 1 -- 172.21.15.76:0/1509790528 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc83415a1f0 con 0x7fc834153e60 2023-11-09T03:46:14.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.107+0000 7fc82a7fc640 1 -- 172.21.15.76:0/1509790528 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82c03db30 con 0x7fc834153e60 2023-11-09T03:46:14.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.107+0000 7fc83bb7c640 1 -- 172.21.15.76:0/1509790528 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc83415ac50 con 0x7fc834153e60 2023-11-09T03:46:14.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.108+0000 7fc83bb7c640 1 -- 172.21.15.76:0/1509790528 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc83415a3e0 con 0x7fc834153e60 2023-11-09T03:46:14.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.109+0000 7fc82a7fc640 1 -- 172.21.15.76:0/1509790528 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc82c03b040 con 0x7fc834153e60 2023-11-09T03:46:14.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.109+0000 7fc82a7fc640 1 --2- 172.21.15.76:0/1509790528 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc81806ae70 0x7fc81806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:14.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.109+0000 7fc82a7fc640 1 -- 172.21.15.76:0/1509790528 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fc82c0aba40 con 0x7fc834153e60 2023-11-09T03:46:14.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.109+0000 7fc839918640 1 --2- 172.21.15.76:0/1509790528 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc81806ae70 0x7fc81806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:14.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.110+0000 7fc839918640 1 --2- 172.21.15.76:0/1509790528 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc81806ae70 0x7fc81806d330 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc830000f60 tx=0x7fc830009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:14.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.113+0000 7fc82a7fc640 1 -- 172.21.15.76:0/1509790528 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc82c0ade60 con 0x7fc834153e60 2023-11-09T03:46:14.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.286+0000 7fc83bb7c640 1 -- 172.21.15.76:0/1509790528 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc834074290 con 0x7fc834153e60 2023-11-09T03:46:14.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.286+0000 7fc82a7fc640 1 -- 172.21.15.76:0/1509790528 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc82c076710 con 0x7fc834153e60 2023-11-09T03:46:14.287 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:46:14.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.288+0000 7fc83bb7c640 1 -- 172.21.15.76:0/1509790528 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc81806ae70 msgr2=0x7fc81806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:14.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.288+0000 7fc83bb7c640 1 --2- 172.21.15.76:0/1509790528 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc81806ae70 0x7fc81806d330 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc830000f60 tx=0x7fc830009040 comp rx=0 tx=0).stop 2023-11-09T03:46:14.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.288+0000 7fc83bb7c640 1 -- 172.21.15.76:0/1509790528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc834153e60 msgr2=0x7fc83415b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:14.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.288+0000 7fc83bb7c640 1 --2- 172.21.15.76:0/1509790528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc834153e60 0x7fc83415b8c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009d90 tx=0x7fc82c004a40 comp rx=0 tx=0).stop 2023-11-09T03:46:14.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.288+0000 7fc83bb7c640 1 -- 172.21.15.76:0/1509790528 shutdown_connections 2023-11-09T03:46:14.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.288+0000 7fc83bb7c640 1 --2- 172.21.15.76:0/1509790528 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc81806ae70 0x7fc81806d330 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:14.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.288+0000 7fc83bb7c640 1 --2- 172.21.15.76:0/1509790528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc834153e60 0x7fc83415b8c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:14.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.288+0000 7fc83bb7c640 1 --2- 172.21.15.76:0/1509790528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8340cd6d0 0x7fc83415a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:14.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.288+0000 7fc83bb7c640 1 --2- 172.21.15.76:0/1509790528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8340b4880 0x7fc834156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:14.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.288+0000 7fc83bb7c640 1 -- 172.21.15.76:0/1509790528 >> 172.21.15.76:0/1509790528 conn(0x7fc8340cf880 msgr2=0x7fc83415afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:14.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.288+0000 7fc83bb7c640 1 -- 172.21.15.76:0/1509790528 shutdown_connections 2023-11-09T03:46:14.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.288+0000 7fc83bb7c640 1 -- 172.21.15.76:0/1509790528 wait complete. 2023-11-09T03:46:14.299 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:46:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.401+0000 7fe94d8da640 1 Processor -- start 2023-11-09T03:46:14.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.401+0000 7fe94d8da640 1 -- start start 2023-11-09T03:46:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.402+0000 7fe94d8da640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe948153e60 0x7fe94815dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.402+0000 7fe94d8da640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe9480cd6d0 0x7fe948161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.402+0000 7fe94d8da640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9480cde60 con 0x7fe9480b4880 2023-11-09T03:46:14.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.402+0000 7fe94d8da640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe948160f90 con 0x7fe9480cd6d0 2023-11-09T03:46:14.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.402+0000 7fe94d8da640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe94815e400 con 0x7fe948153e60 2023-11-09T03:46:14.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.402+0000 7fe9477fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe9480b4880 0x7fe94806b810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43592/0 (socket says 172.21.15.76:43592) 2023-11-09T03:46:14.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.402+0000 7fe9477fe640 1 -- 172.21.15.76:0/2780386543 learned_addr learned my addr 172.21.15.76:0/2780386543 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:14.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.402+0000 7fe946ffd640 1 --2- 172.21.15.76:0/2780386543 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe948153e60 0x7fe94815dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:14.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.402+0000 7fe9467fc640 1 --2- 172.21.15.76:0/2780386543 >> v2:172.21.15.113:3300/0 conn(0x7fe9480cd6d0 0x7fe948161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:14.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.402+0000 7fe945ffb640 1 -- 172.21.15.76:0/2780386543 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 296687383 0 0) 0x7fe9480cde60 con 0x7fe9480b4880 2023-11-09T03:46:14.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.403+0000 7fe945ffb640 1 -- 172.21.15.76:0/2780386543 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe930003520 con 0x7fe9480b4880 2023-11-09T03:46:14.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.403+0000 7fe945ffb640 1 -- 172.21.15.76:0/2780386543 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1961835720 0 0) 0x7fe930003520 con 0x7fe9480b4880 2023-11-09T03:46:14.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.403+0000 7fe9467fc640 1 -- 172.21.15.76:0/2780386543 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe948153e60 msgr2=0x7fe94815dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:14.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.403+0000 7fe9467fc640 1 --2- 172.21.15.76:0/2780386543 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe948153e60 0x7fe94815dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:14.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.403+0000 7fe9467fc640 1 -- 172.21.15.76:0/2780386543 >> v1:172.21.15.76:6789/0 conn(0x7fe9480b4880 legacy=0x7fe94806b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:14.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.403+0000 7fe9467fc640 1 -- 172.21.15.76:0/2780386543 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe948161960 con 0x7fe9480cd6d0 2023-11-09T03:46:14.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.403+0000 7fe946ffd640 1 --2- 172.21.15.76:0/2780386543 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe948153e60 0x7fe94815dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:46:14.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.404+0000 7fe9467fc640 1 --2- 172.21.15.76:0/2780386543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9480cd6d0 0x7fe948161420 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe93c0097a0 tx=0x7fe93c02b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=217c3002ba77cdde server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:14.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.404+0000 7fe945ffb640 1 -- 172.21.15.76:0/2780386543 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940001670 con 0x7fe9480cd6d0 2023-11-09T03:46:14.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.404+0000 7fe945ffb640 1 -- 172.21.15.76:0/2780386543 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93c02e050 con 0x7fe9480cd6d0 2023-11-09T03:46:14.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.404+0000 7fe945ffb640 1 -- 172.21.15.76:0/2780386543 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c030710 con 0x7fe9480cd6d0 2023-11-09T03:46:14.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.404+0000 7fe94d8da640 1 -- 172.21.15.76:0/2780386543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9480cd6d0 msgr2=0x7fe948161420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:14.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.404+0000 7fe94d8da640 1 --2- 172.21.15.76:0/2780386543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9480cd6d0 0x7fe948161420 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe93c0097a0 tx=0x7fe93c02b790 comp rx=0 tx=0).stop 2023-11-09T03:46:14.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.404+0000 7fe94d8da640 1 -- 172.21.15.76:0/2780386543 shutdown_connections 2023-11-09T03:46:14.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.404+0000 7fe94d8da640 1 --2- 172.21.15.76:0/2780386543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9480cd6d0 0x7fe948161420 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:14.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.404+0000 7fe94d8da640 1 --2- 172.21.15.76:0/2780386543 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe948153e60 0x7fe94815dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:14.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.404+0000 7fe94d8da640 1 -- 172.21.15.76:0/2780386543 >> 172.21.15.76:0/2780386543 conn(0x7fe9480cf880 msgr2=0x7fe9480cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:14.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.404+0000 7fe94d8da640 1 -- 172.21.15.76:0/2780386543 shutdown_connections 2023-11-09T03:46:14.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.404+0000 7fe94d8da640 1 -- 172.21.15.76:0/2780386543 wait complete. 2023-11-09T03:46:14.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.405+0000 7fe94d8da640 1 Processor -- start 2023-11-09T03:46:14.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.406+0000 7fe94d8da640 1 -- start start 2023-11-09T03:46:14.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.406+0000 7fe94d8da640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9480b4880 0x7fe948145b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:14.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.406+0000 7fe94d8da640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9480cd6d0 0x7fe9481f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:14.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.406+0000 7fe94d8da640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe948153e60 0x7fe9481f6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:14.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.406+0000 7fe94d8da640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9481511d0 con 0x7fe948153e60 2023-11-09T03:46:14.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.406+0000 7fe94d8da640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9480c97b0 con 0x7fe9480cd6d0 2023-11-09T03:46:14.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.406+0000 7fe94d8da640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9481f36c0 con 0x7fe9480b4880 2023-11-09T03:46:14.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.406+0000 7fe9477fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe948153e60 0x7fe9481f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:14.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.406+0000 7fe9477fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe948153e60 0x7fe9481f6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36648/0 (socket says 172.21.15.76:36648) 2023-11-09T03:46:14.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.406+0000 7fe9477fe640 1 -- 172.21.15.76:0/823417484 learned_addr learned my addr 172.21.15.76:0/823417484 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:14.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.407+0000 7fe946ffd640 1 --2- 172.21.15.76:0/823417484 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9480b4880 0x7fe948145b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:14.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.407+0000 7fe9467fc640 1 --2- 172.21.15.76:0/823417484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9480cd6d0 0x7fe9481f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:14.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.407+0000 7fe9477fe640 1 -- 172.21.15.76:0/823417484 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9480b4880 msgr2=0x7fe948145b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:14.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.407+0000 7fe9477fe640 1 --2- 172.21.15.76:0/823417484 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9480b4880 0x7fe948145b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:14.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.407+0000 7fe9477fe640 1 -- 172.21.15.76:0/823417484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9480cd6d0 msgr2=0x7fe9481f5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:14.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.407+0000 7fe9477fe640 1 --2- 172.21.15.76:0/823417484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9480cd6d0 0x7fe9481f5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:14.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.407+0000 7fe9477fe640 1 -- 172.21.15.76:0/823417484 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe938009a70 con 0x7fe948153e60 2023-11-09T03:46:14.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.407+0000 7fe9477fe640 1 --2- 172.21.15.76:0/823417484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe948153e60 0x7fe9481f6030 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe9400017a0 tx=0x7fe94000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:14.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.407+0000 7fe92ffff640 1 -- 172.21.15.76:0/823417484 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940005ad0 con 0x7fe948153e60 2023-11-09T03:46:14.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.407+0000 7fe92ffff640 1 -- 172.21.15.76:0/823417484 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe940005c30 con 0x7fe948153e60 2023-11-09T03:46:14.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.407+0000 7fe94d8da640 1 -- 172.21.15.76:0/823417484 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe93c009400 con 0x7fe948153e60 2023-11-09T03:46:14.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.407+0000 7fe92ffff640 1 -- 172.21.15.76:0/823417484 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940007070 con 0x7fe948153e60 2023-11-09T03:46:14.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.407+0000 7fe94d8da640 1 -- 172.21.15.76:0/823417484 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9480c94d0 con 0x7fe948153e60 2023-11-09T03:46:14.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.408+0000 7fe94d8da640 1 -- 172.21.15.76:0/823417484 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe948151340 con 0x7fe948153e60 2023-11-09T03:46:14.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.408+0000 7fe92ffff640 1 -- 172.21.15.76:0/823417484 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe9400071d0 con 0x7fe948153e60 2023-11-09T03:46:14.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.409+0000 7fe92ffff640 1 --2- 172.21.15.76:0/823417484 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe93406aea0 0x7fe93406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:14.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.409+0000 7fe92ffff640 1 -- 172.21.15.76:0/823417484 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fe9400896c0 con 0x7fe948153e60 2023-11-09T03:46:14.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.409+0000 7fe946ffd640 1 --2- 172.21.15.76:0/823417484 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe93406aea0 0x7fe93406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:14.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.410+0000 7fe946ffd640 1 --2- 172.21.15.76:0/823417484 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe93406aea0 0x7fe93406d360 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe948160220 tx=0x7fe93800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:14.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.413+0000 7fe92ffff640 1 -- 172.21.15.76:0/823417484 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe940054390 con 0x7fe948153e60 2023-11-09T03:46:14.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.577+0000 7fe94d8da640 1 -- 172.21.15.76:0/823417484 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe9480cdc70 con 0x7fe948153e60 2023-11-09T03:46:14.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.578+0000 7fe92ffff640 1 -- 172.21.15.76:0/823417484 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe9400579b0 con 0x7fe948153e60 2023-11-09T03:46:14.578 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:46:14.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.580+0000 7fe94d8da640 1 -- 172.21.15.76:0/823417484 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe93406aea0 msgr2=0x7fe93406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:14.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.580+0000 7fe94d8da640 1 --2- 172.21.15.76:0/823417484 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe93406aea0 0x7fe93406d360 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe948160220 tx=0x7fe93800b040 comp rx=0 tx=0).stop 2023-11-09T03:46:14.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.580+0000 7fe94d8da640 1 -- 172.21.15.76:0/823417484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe948153e60 msgr2=0x7fe9481f6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:14.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.580+0000 7fe94d8da640 1 --2- 172.21.15.76:0/823417484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe948153e60 0x7fe9481f6030 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe9400017a0 tx=0x7fe94000cbe0 comp rx=0 tx=0).stop 2023-11-09T03:46:14.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.580+0000 7fe94d8da640 1 -- 172.21.15.76:0/823417484 shutdown_connections 2023-11-09T03:46:14.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.580+0000 7fe94d8da640 1 --2- 172.21.15.76:0/823417484 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe93406aea0 0x7fe93406d360 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:14.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.580+0000 7fe94d8da640 1 --2- 172.21.15.76:0/823417484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe948153e60 0x7fe9481f6030 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:14.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.580+0000 7fe94d8da640 1 --2- 172.21.15.76:0/823417484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9480cd6d0 0x7fe9481f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:14.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.580+0000 7fe94d8da640 1 --2- 172.21.15.76:0/823417484 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9480b4880 0x7fe948145b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:14.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.580+0000 7fe94d8da640 1 -- 172.21.15.76:0/823417484 >> 172.21.15.76:0/823417484 conn(0x7fe9480cf880 msgr2=0x7fe94815fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:14.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.580+0000 7fe94d8da640 1 -- 172.21.15.76:0/823417484 shutdown_connections 2023-11-09T03:46:14.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.580+0000 7fe94d8da640 1 -- 172.21.15.76:0/823417484 wait complete. 2023-11-09T03:46:14.591 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:46:14.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.695+0000 7f63a75c0640 1 Processor -- start 2023-11-09T03:46:14.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.695+0000 7f63a75c0640 1 -- start start 2023-11-09T03:46:14.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.696+0000 7f63a75c0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f63a00c4ff0 0x7f63a00cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:14.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.696+0000 7f63a75c0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f63a00adac0 0x7f63a00adf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:14.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.696+0000 7f63a75c0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63a015aff0 con 0x7f63a0159c00 2023-11-09T03:46:14.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.696+0000 7f63a75c0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63a00ae460 con 0x7f63a00c4ff0 2023-11-09T03:46:14.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.696+0000 7f63a75c0640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63a015a3e0 con 0x7f63a00adac0 2023-11-09T03:46:14.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.696+0000 7f63a4b5b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f63a00c4ff0 0x7f63a00cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:14.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.696+0000 7f63a4b5b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f63a00c4ff0 0x7f63a00cc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36664/0 (socket says 172.21.15.76:36664) 2023-11-09T03:46:14.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.696+0000 7f63a4b5b640 1 -- 172.21.15.76:0/945625558 learned_addr learned my addr 172.21.15.76:0/945625558 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:14.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.696+0000 7f63a4b5b640 1 -- 172.21.15.76:0/945625558 >> v2:172.21.15.113:3300/0 conn(0x7f63a00adac0 msgr2=0x7f63a00adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:14.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.696+0000 7f63a5b5d640 1 --2- 172.21.15.76:0/945625558 >> v2:172.21.15.113:3300/0 conn(0x7f63a00adac0 0x7f63a00adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:14.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.696+0000 7f63a4b5b640 1 --2- 172.21.15.76:0/945625558 >> v2:172.21.15.113:3300/0 conn(0x7f63a00adac0 0x7f63a00adf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:14.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.696+0000 7f63a4b5b640 1 -- 172.21.15.76:0/945625558 >> v1:172.21.15.113:6789/0 conn(0x7f63a0159c00 legacy=0x7f63a00c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:14.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.696+0000 7f63a4b5b640 1 -- 172.21.15.76:0/945625558 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a015a6c0 con 0x7f63a00c4ff0 2023-11-09T03:46:14.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.697+0000 7f63a4b5b640 1 --2- 172.21.15.76:0/945625558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63a00c4ff0 0x7f63a00cc890 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f63a00c59a0 tx=0x7f639002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbdf3b7629fa844 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:14.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.697+0000 7f6397fff640 1 -- 172.21.15.76:0/945625558 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6390031870 con 0x7f63a00c4ff0 2023-11-09T03:46:14.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.697+0000 7f6397fff640 1 -- 172.21.15.76:0/945625558 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63900319d0 con 0x7f63a00c4ff0 2023-11-09T03:46:14.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.697+0000 7f6397fff640 1 -- 172.21.15.76:0/945625558 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6390031d60 con 0x7f63a00c4ff0 2023-11-09T03:46:14.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.697+0000 7f63a75c0640 1 -- 172.21.15.76:0/945625558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63a00c4ff0 msgr2=0x7f63a00cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:14.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.697+0000 7f63a75c0640 1 --2- 172.21.15.76:0/945625558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63a00c4ff0 0x7f63a00cc890 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f63a00c59a0 tx=0x7f639002b910 comp rx=0 tx=0).stop 2023-11-09T03:46:14.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.698+0000 7f63a75c0640 1 -- 172.21.15.76:0/945625558 shutdown_connections 2023-11-09T03:46:14.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.698+0000 7f63a75c0640 1 --2- 172.21.15.76:0/945625558 >> v2:172.21.15.113:3300/0 conn(0x7f63a00adac0 0x7f63a00adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:14.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.698+0000 7f63a75c0640 1 --2- 172.21.15.76:0/945625558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63a00c4ff0 0x7f63a00cc890 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:14.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.698+0000 7f63a75c0640 1 -- 172.21.15.76:0/945625558 >> 172.21.15.76:0/945625558 conn(0x7f63a00ce950 msgr2=0x7f63a00ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:14.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.698+0000 7f63a75c0640 1 -- 172.21.15.76:0/945625558 shutdown_connections 2023-11-09T03:46:14.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.698+0000 7f63a75c0640 1 -- 172.21.15.76:0/945625558 wait complete. 2023-11-09T03:46:14.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.699+0000 7f63a75c0640 1 Processor -- start 2023-11-09T03:46:14.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.699+0000 7f63a75c0640 1 -- start start 2023-11-09T03:46:14.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.699+0000 7f63a75c0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63a00adac0 0x7f63a01f1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:14.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.699+0000 7f63a75c0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63a00c4ff0 0x7f63a0153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:14.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.700+0000 7f63a75c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63a0159c00 0x7f63a00b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:14.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.700+0000 7f63a75c0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a0158950 con 0x7f63a0159c00 2023-11-09T03:46:14.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.700+0000 7f63a75c0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a01f1840 con 0x7f63a00adac0 2023-11-09T03:46:14.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.700+0000 7f63a75c0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a0158f80 con 0x7f63a00c4ff0 2023-11-09T03:46:14.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.700+0000 7f63a5b5d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63a0159c00 0x7f63a00b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:14.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.700+0000 7f63a5b5d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63a0159c00 0x7f63a00b06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36668/0 (socket says 172.21.15.76:36668) 2023-11-09T03:46:14.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.700+0000 7f63a5b5d640 1 -- 172.21.15.76:0/202654282 learned_addr learned my addr 172.21.15.76:0/202654282 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:14.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.700+0000 7f63a535c640 1 --2- 172.21.15.76:0/202654282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63a00adac0 0x7f63a01f1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:14.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.700+0000 7f63a5b5d640 1 -- 172.21.15.76:0/202654282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63a00c4ff0 msgr2=0x7f63a0153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:14.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.700+0000 7f63a4b5b640 1 --2- 172.21.15.76:0/202654282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63a00c4ff0 0x7f63a0153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:14.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.700+0000 7f63a5b5d640 1 --2- 172.21.15.76:0/202654282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63a00c4ff0 0x7f63a0153070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:14.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.700+0000 7f63a5b5d640 1 -- 172.21.15.76:0/202654282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63a00adac0 msgr2=0x7f63a01f1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:14.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.700+0000 7f63a5b5d640 1 --2- 172.21.15.76:0/202654282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63a00adac0 0x7f63a01f1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:14.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.700+0000 7f63a5b5d640 1 -- 172.21.15.76:0/202654282 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63900094d0 con 0x7f63a0159c00 2023-11-09T03:46:14.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.700+0000 7f63a5b5d640 1 --2- 172.21.15.76:0/202654282 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63a0159c00 0x7f63a00b06b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f639800cda0 tx=0x7f639800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:14.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.700+0000 7f6395ffb640 1 -- 172.21.15.76:0/202654282 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6398003b90 con 0x7f63a0159c00 2023-11-09T03:46:14.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.700+0000 7f6395ffb640 1 -- 172.21.15.76:0/202654282 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6398003cf0 con 0x7f63a0159c00 2023-11-09T03:46:14.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.700+0000 7f6395ffb640 1 -- 172.21.15.76:0/202654282 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6398004240 con 0x7f63a0159c00 2023-11-09T03:46:14.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.700+0000 7f63a75c0640 1 -- 172.21.15.76:0/202654282 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a00aeb20 con 0x7f63a0159c00 2023-11-09T03:46:14.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.701+0000 7f63a75c0640 1 -- 172.21.15.76:0/202654282 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a00af0e0 con 0x7f63a0159c00 2023-11-09T03:46:14.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.702+0000 7f63a75c0640 1 -- 172.21.15.76:0/202654282 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a0074290 con 0x7f63a0159c00 2023-11-09T03:46:14.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.702+0000 7f6395ffb640 1 -- 172.21.15.76:0/202654282 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f63980043a0 con 0x7f63a0159c00 2023-11-09T03:46:14.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.702+0000 7f6395ffb640 1 --2- 172.21.15.76:0/202654282 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f638406ae70 0x7f638406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:14.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.702+0000 7f6395ffb640 1 -- 172.21.15.76:0/202654282 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f6398012070 con 0x7f63a0159c00 2023-11-09T03:46:14.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.702+0000 7f63a535c640 1 --2- 172.21.15.76:0/202654282 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f638406ae70 0x7f638406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:14.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.703+0000 7f63a535c640 1 --2- 172.21.15.76:0/202654282 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f638406ae70 0x7f638406d330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f639c006fd0 tx=0x7f639c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:14.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.706+0000 7f6395ffb640 1 -- 172.21.15.76:0/202654282 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6398055080 con 0x7f63a0159c00 2023-11-09T03:46:14.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.875+0000 7f63a75c0640 1 -- 172.21.15.76:0/202654282 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f63a00cd9f0 con 0x7f63a0159c00 2023-11-09T03:46:14.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.875+0000 7f6395ffb640 1 -- 172.21.15.76:0/202654282 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f63980586a0 con 0x7f63a0159c00 2023-11-09T03:46:14.876 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:46:14.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.878+0000 7f63a75c0640 1 -- 172.21.15.76:0/202654282 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f638406ae70 msgr2=0x7f638406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:14.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.878+0000 7f63a75c0640 1 --2- 172.21.15.76:0/202654282 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f638406ae70 0x7f638406d330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f639c006fd0 tx=0x7f639c008040 comp rx=0 tx=0).stop 2023-11-09T03:46:14.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.878+0000 7f63a75c0640 1 -- 172.21.15.76:0/202654282 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63a0159c00 msgr2=0x7f63a00b06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:14.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.878+0000 7f63a75c0640 1 --2- 172.21.15.76:0/202654282 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63a0159c00 0x7f63a00b06b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f639800cda0 tx=0x7f639800a660 comp rx=0 tx=0).stop 2023-11-09T03:46:14.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.878+0000 7f63a75c0640 1 -- 172.21.15.76:0/202654282 shutdown_connections 2023-11-09T03:46:14.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.878+0000 7f63a75c0640 1 --2- 172.21.15.76:0/202654282 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f638406ae70 0x7f638406d330 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:14.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.878+0000 7f63a75c0640 1 --2- 172.21.15.76:0/202654282 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63a0159c00 0x7f63a00b06b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:14.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.878+0000 7f63a75c0640 1 --2- 172.21.15.76:0/202654282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63a00c4ff0 0x7f63a0153070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:14.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.878+0000 7f63a75c0640 1 --2- 172.21.15.76:0/202654282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63a00adac0 0x7f63a01f1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:14.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.878+0000 7f63a75c0640 1 -- 172.21.15.76:0/202654282 >> 172.21.15.76:0/202654282 conn(0x7f63a00ce950 msgr2=0x7f63a00709f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:14.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.878+0000 7f63a75c0640 1 -- 172.21.15.76:0/202654282 shutdown_connections 2023-11-09T03:46:14.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.878+0000 7f63a75c0640 1 -- 172.21.15.76:0/202654282 wait complete. 2023-11-09T03:46:14.889 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-09T03:46:14.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.991+0000 7fa7d7f90640 1 Processor -- start 2023-11-09T03:46:14.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.991+0000 7fa7d7f90640 1 -- start start 2023-11-09T03:46:14.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.992+0000 7fa7d7f90640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa7d0153e60 0x7fa7d015dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:14.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.992+0000 7fa7d7f90640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa7d00cd6d0 0x7fa7d0161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:14.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.992+0000 7fa7d7f90640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa7d00b4880 0x7fa7d00b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:14.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.992+0000 7fa7d7f90640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d00cde60 con 0x7fa7d0153e60 2023-11-09T03:46:14.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.992+0000 7fa7d7f90640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d00b5220 con 0x7fa7d00cd6d0 2023-11-09T03:46:14.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.992+0000 7fa7d7f90640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d0160f90 con 0x7fa7d00b4880 2023-11-09T03:46:14.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.992+0000 7fa7d5d2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa7d0153e60 0x7fa7d015dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:14.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.992+0000 7fa7d5d2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa7d0153e60 0x7fa7d015dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36682/0 (socket says 172.21.15.76:36682) 2023-11-09T03:46:14.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.992+0000 7fa7d5d2c640 1 -- 172.21.15.76:0/3075348185 learned_addr learned my addr 172.21.15.76:0/3075348185 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:14.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.993+0000 7fa7d5d2c640 1 -- 172.21.15.76:0/3075348185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa7d00b4880 msgr2=0x7fa7d00b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:14.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.993+0000 7fa7d5d2c640 1 --2- 172.21.15.76:0/3075348185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa7d00b4880 0x7fa7d00b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:14.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.993+0000 7fa7d5d2c640 1 -- 172.21.15.76:0/3075348185 >> v2:172.21.15.113:3300/0 conn(0x7fa7d00cd6d0 msgr2=0x7fa7d0161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:14.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.993+0000 7fa7d5d2c640 1 --2- 172.21.15.76:0/3075348185 >> v2:172.21.15.113:3300/0 conn(0x7fa7d00cd6d0 0x7fa7d0161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:14.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.993+0000 7fa7d5d2c640 1 -- 172.21.15.76:0/3075348185 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d015e400 con 0x7fa7d0153e60 2023-11-09T03:46:14.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.993+0000 7fa7d5d2c640 1 --2- 172.21.15.76:0/3075348185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7d0153e60 0x7fa7d015dec0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00c9e80 tx=0x7fa7cc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2e4ed555bdd1b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:14.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.993+0000 7fa7d4d2a640 1 -- 172.21.15.76:0/3075348185 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc02ca70 con 0x7fa7d0153e60 2023-11-09T03:46:14.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.994+0000 7fa7d4d2a640 1 -- 172.21.15.76:0/3075348185 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7cc02cbd0 con 0x7fa7d0153e60 2023-11-09T03:46:15.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.994+0000 7fa7d4d2a640 1 -- 172.21.15.76:0/3075348185 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc0343f0 con 0x7fa7d0153e60 2023-11-09T03:46:15.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.994+0000 7fa7d7f90640 1 -- 172.21.15.76:0/3075348185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7d0153e60 msgr2=0x7fa7d015dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:15.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.994+0000 7fa7d7f90640 1 --2- 172.21.15.76:0/3075348185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7d0153e60 0x7fa7d015dec0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00c9e80 tx=0x7fa7cc02bed0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.994+0000 7fa7d7f90640 1 -- 172.21.15.76:0/3075348185 shutdown_connections 2023-11-09T03:46:15.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.994+0000 7fa7d7f90640 1 --2- 172.21.15.76:0/3075348185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa7d00b4880 0x7fa7d00b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.994+0000 7fa7d7f90640 1 --2- 172.21.15.76:0/3075348185 >> v2:172.21.15.113:3300/0 conn(0x7fa7d00cd6d0 0x7fa7d0161420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.994+0000 7fa7d7f90640 1 --2- 172.21.15.76:0/3075348185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7d0153e60 0x7fa7d015dec0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.994+0000 7fa7d7f90640 1 -- 172.21.15.76:0/3075348185 >> 172.21.15.76:0/3075348185 conn(0x7fa7d00cf880 msgr2=0x7fa7d00cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:15.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.994+0000 7fa7d7f90640 1 -- 172.21.15.76:0/3075348185 shutdown_connections 2023-11-09T03:46:15.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.994+0000 7fa7d7f90640 1 -- 172.21.15.76:0/3075348185 wait complete. 2023-11-09T03:46:15.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.995+0000 7fa7d7f90640 1 Processor -- start 2023-11-09T03:46:15.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.995+0000 7fa7d7f90640 1 -- start start 2023-11-09T03:46:15.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.996+0000 7fa7d7f90640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7d00b4880 0x7fa7d0154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:15.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.996+0000 7fa7d7f90640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7d00cd6d0 0x7fa7d0155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:15.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.996+0000 7fa7d7f90640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa7d0153e60 0x7fa7d015b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:15.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.996+0000 7fa7d7f90640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d01f2610 con 0x7fa7d00b4880 2023-11-09T03:46:15.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.996+0000 7fa7d7f90640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d0154890 con 0x7fa7d00cd6d0 2023-11-09T03:46:15.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.996+0000 7fa7d7f90640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d0154b70 con 0x7fa7d0153e60 2023-11-09T03:46:15.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.996+0000 7fa7d5d2c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7d00b4880 0x7fa7d0154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:15.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.996+0000 7fa7d5d2c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7d00b4880 0x7fa7d0154240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36698/0 (socket says 172.21.15.76:36698) 2023-11-09T03:46:15.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.996+0000 7fa7d5d2c640 1 -- 172.21.15.76:0/3554522928 learned_addr learned my addr 172.21.15.76:0/3554522928 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:15.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.996+0000 7fa7d5d2c640 1 -- 172.21.15.76:0/3554522928 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa7d0153e60 msgr2=0x7fa7d015b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:15.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.996+0000 7fa7d552b640 1 --2- 172.21.15.76:0/3554522928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7d00cd6d0 0x7fa7d0155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:15.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.996+0000 7fa7d5d2c640 1 --2- 172.21.15.76:0/3554522928 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa7d0153e60 0x7fa7d015b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.996+0000 7fa7d5d2c640 1 -- 172.21.15.76:0/3554522928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7d00cd6d0 msgr2=0x7fa7d0155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:15.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.996+0000 7fa7d5d2c640 1 --2- 172.21.15.76:0/3554522928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7d00cd6d0 0x7fa7d0155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.996+0000 7fa7d5d2c640 1 -- 172.21.15.76:0/3554522928 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7cc009b30 con 0x7fa7d00b4880 2023-11-09T03:46:15.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.996+0000 7fa7d552b640 1 --2- 172.21.15.76:0/3554522928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7d00cd6d0 0x7fa7d0155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:15.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.997+0000 7fa7d5d2c640 1 --2- 172.21.15.76:0/3554522928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7d00b4880 0x7fa7d0154240 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009e50 tx=0x7fa7cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:15.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.997+0000 7fa7c67fc640 1 -- 172.21.15.76:0/3554522928 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc03e450 con 0x7fa7d00b4880 2023-11-09T03:46:15.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.997+0000 7fa7c67fc640 1 -- 172.21.15.76:0/3554522928 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7cc03e5b0 con 0x7fa7d00b4880 2023-11-09T03:46:15.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.997+0000 7fa7d7f90640 1 -- 172.21.15.76:0/3554522928 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d0154e50 con 0x7fa7d00b4880 2023-11-09T03:46:15.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.997+0000 7fa7c67fc640 1 -- 172.21.15.76:0/3554522928 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc03e960 con 0x7fa7d00b4880 2023-11-09T03:46:15.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.997+0000 7fa7d7f90640 1 -- 172.21.15.76:0/3554522928 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d0155410 con 0x7fa7d00b4880 2023-11-09T03:46:15.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.998+0000 7fa7d7f90640 1 -- 172.21.15.76:0/3554522928 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d0155600 con 0x7fa7d00b4880 2023-11-09T03:46:15.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.998+0000 7fa7c67fc640 1 -- 172.21.15.76:0/3554522928 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa7cc03eac0 con 0x7fa7d00b4880 2023-11-09T03:46:15.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.999+0000 7fa7c67fc640 1 --2- 172.21.15.76:0/3554522928 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa7b406aea0 0x7fa7b406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:15.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.999+0000 7fa7c67fc640 1 -- 172.21.15.76:0/3554522928 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fa7cc0ac5b0 con 0x7fa7d00b4880 2023-11-09T03:46:15.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.999+0000 7fa7d552b640 1 --2- 172.21.15.76:0/3554522928 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa7b406aea0 0x7fa7b406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:15.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:14.999+0000 7fa7d552b640 1 --2- 172.21.15.76:0/3554522928 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa7b406aea0 0x7fa7b406d360 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0006fd0 tx=0x7fa7c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:15.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.003+0000 7fa7c67fc640 1 -- 172.21.15.76:0/3554522928 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa7cc077310 con 0x7fa7d00b4880 2023-11-09T03:46:15.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.168+0000 7fa7d7f90640 1 -- 172.21.15.76:0/3554522928 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa7d0074290 con 0x7fa7d00b4880 2023-11-09T03:46:15.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.169+0000 7fa7c67fc640 1 -- 172.21.15.76:0/3554522928 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa7cc07fe60 con 0x7fa7d00b4880 2023-11-09T03:46:15.169 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:46:15.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.172+0000 7fa7d7f90640 1 -- 172.21.15.76:0/3554522928 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa7b406aea0 msgr2=0x7fa7b406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:15.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.172+0000 7fa7d7f90640 1 --2- 172.21.15.76:0/3554522928 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa7b406aea0 0x7fa7b406d360 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0006fd0 tx=0x7fa7c0008040 comp rx=0 tx=0).stop 2023-11-09T03:46:15.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.172+0000 7fa7d7f90640 1 -- 172.21.15.76:0/3554522928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7d00b4880 msgr2=0x7fa7d0154240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:15.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.172+0000 7fa7d7f90640 1 --2- 172.21.15.76:0/3554522928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7d00b4880 0x7fa7d0154240 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009e50 tx=0x7fa7cc004a40 comp rx=0 tx=0).stop 2023-11-09T03:46:15.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.172+0000 7fa7d7f90640 1 -- 172.21.15.76:0/3554522928 shutdown_connections 2023-11-09T03:46:15.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.172+0000 7fa7d7f90640 1 --2- 172.21.15.76:0/3554522928 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa7b406aea0 0x7fa7b406d360 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.172+0000 7fa7d7f90640 1 --2- 172.21.15.76:0/3554522928 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa7d0153e60 0x7fa7d015b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.172+0000 7fa7d7f90640 1 --2- 172.21.15.76:0/3554522928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7d00cd6d0 0x7fa7d0155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.172+0000 7fa7d7f90640 1 --2- 172.21.15.76:0/3554522928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7d00b4880 0x7fa7d0154240 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.172+0000 7fa7d7f90640 1 -- 172.21.15.76:0/3554522928 >> 172.21.15.76:0/3554522928 conn(0x7fa7d00cf880 msgr2=0x7fa7d00b7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:15.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.172+0000 7fa7d7f90640 1 -- 172.21.15.76:0/3554522928 shutdown_connections 2023-11-09T03:46:15.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.172+0000 7fa7d7f90640 1 -- 172.21.15.76:0/3554522928 wait complete. 2023-11-09T03:46:15.182 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:46:15.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.281+0000 7facf2039640 1 Processor -- start 2023-11-09T03:46:15.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.281+0000 7facf2039640 1 -- start start 2023-11-09T03:46:15.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.282+0000 7facf2039640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7facec15ad80 0x7facec06c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:15.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.282+0000 7facf2039640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facec06cd00 con 0x7facec0b42c0 2023-11-09T03:46:15.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.282+0000 7facf2039640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facec06cfa0 con 0x7facec0c9e10 2023-11-09T03:46:15.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.282+0000 7facf2039640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facec06d280 con 0x7facec15ad80 2023-11-09T03:46:15.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.282+0000 7faceb7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7facec15ad80 0x7facec06c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:15.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.282+0000 7faceaffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7facec0b42c0 0x7facec0c9700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43596/0 (socket says 172.21.15.76:43596) 2023-11-09T03:46:15.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.282+0000 7faceaffd640 1 -- 172.21.15.76:0/2584164368 learned_addr learned my addr 172.21.15.76:0/2584164368 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:15.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.282+0000 7faceb7fe640 1 -- 172.21.15.76:0/2584164368 >> v1:172.21.15.113:6789/0 conn(0x7facec0c9e10 legacy=0x7facec0b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:15.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.282+0000 7faceb7fe640 1 -- 172.21.15.76:0/2584164368 >> v1:172.21.15.76:6789/0 conn(0x7facec0b42c0 legacy=0x7facec0c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:15.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.283+0000 7facea7fc640 1 -- 172.21.15.76:0/2584164368 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2568162160 0 0) 0x7facec06cd00 con 0x7facec0b42c0 2023-11-09T03:46:15.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.283+0000 7faceb7fe640 1 -- 172.21.15.76:0/2584164368 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facec06d560 con 0x7facec15ad80 2023-11-09T03:46:15.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.283+0000 7faceb7fe640 1 --2- 172.21.15.76:0/2584164368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facec15ad80 0x7facec06c7c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7facdc0097a0 tx=0x7facdc02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d607eba5ebe7132 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:15.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.283+0000 7facea7fc640 1 -- 172.21.15.76:0/2584164368 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc030610 con 0x7facec15ad80 2023-11-09T03:46:15.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.283+0000 7facea7fc640 1 -- 172.21.15.76:0/2584164368 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facdc030770 con 0x7facec15ad80 2023-11-09T03:46:15.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.283+0000 7facea7fc640 1 -- 172.21.15.76:0/2584164368 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc030b00 con 0x7facec15ad80 2023-11-09T03:46:15.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.284+0000 7facf2039640 1 -- 172.21.15.76:0/2584164368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facec15ad80 msgr2=0x7facec06c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:15.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.284+0000 7facf2039640 1 --2- 172.21.15.76:0/2584164368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facec15ad80 0x7facec06c7c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7facdc0097a0 tx=0x7facdc02b960 comp rx=0 tx=0).stop 2023-11-09T03:46:15.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.284+0000 7facf2039640 1 -- 172.21.15.76:0/2584164368 shutdown_connections 2023-11-09T03:46:15.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.284+0000 7facf2039640 1 --2- 172.21.15.76:0/2584164368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facec15ad80 0x7facec06c7c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.284+0000 7facf2039640 1 -- 172.21.15.76:0/2584164368 >> 172.21.15.76:0/2584164368 conn(0x7facec0ce950 msgr2=0x7facec0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:15.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.284+0000 7facf2039640 1 -- 172.21.15.76:0/2584164368 shutdown_connections 2023-11-09T03:46:15.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.284+0000 7facf2039640 1 -- 172.21.15.76:0/2584164368 wait complete. 2023-11-09T03:46:15.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.285+0000 7facf2039640 1 Processor -- start 2023-11-09T03:46:15.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.285+0000 7facf2039640 1 -- start start 2023-11-09T03:46:15.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.286+0000 7facf2039640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facec0b42c0 0x7facec1541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:15.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.286+0000 7facf2039640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facec0c9e10 0x7facec1f2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:15.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.286+0000 7facf2039640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facec15ad80 0x7facec15cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:15.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.286+0000 7facf2039640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec15e2a0 con 0x7facec15ad80 2023-11-09T03:46:15.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.286+0000 7facf2039640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec15da00 con 0x7facec0c9e10 2023-11-09T03:46:15.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.286+0000 7facf2039640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facec15dce0 con 0x7facec0b42c0 2023-11-09T03:46:15.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.286+0000 7facebfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facec15ad80 0x7facec15cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:15.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.286+0000 7facebfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facec15ad80 0x7facec15cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36712/0 (socket says 172.21.15.76:36712) 2023-11-09T03:46:15.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.286+0000 7facebfff640 1 -- 172.21.15.76:0/3480530785 learned_addr learned my addr 172.21.15.76:0/3480530785 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:15.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.286+0000 7faceb7fe640 1 --2- 172.21.15.76:0/3480530785 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facec0b42c0 0x7facec1541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:15.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.286+0000 7facebfff640 1 -- 172.21.15.76:0/3480530785 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facec0b42c0 msgr2=0x7facec1541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:15.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.286+0000 7faceaffd640 1 --2- 172.21.15.76:0/3480530785 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facec0c9e10 0x7facec1f2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:15.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.287+0000 7facebfff640 1 --2- 172.21.15.76:0/3480530785 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facec0b42c0 0x7facec1541f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.287+0000 7facebfff640 1 -- 172.21.15.76:0/3480530785 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facec0c9e10 msgr2=0x7facec1f2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:15.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.287+0000 7facebfff640 1 --2- 172.21.15.76:0/3480530785 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facec0c9e10 0x7facec1f2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.287+0000 7facebfff640 1 -- 172.21.15.76:0/3480530785 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facdc009400 con 0x7facec15ad80 2023-11-09T03:46:15.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.287+0000 7facebfff640 1 --2- 172.21.15.76:0/3480530785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facec15ad80 0x7facec15cfb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7face400c780 tx=0x7face400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:15.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.287+0000 7facd3fff640 1 -- 172.21.15.76:0/3480530785 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4004b60 con 0x7facec15ad80 2023-11-09T03:46:15.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.287+0000 7facd3fff640 1 -- 172.21.15.76:0/3480530785 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face400ce90 con 0x7facec15ad80 2023-11-09T03:46:15.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.287+0000 7facd3fff640 1 -- 172.21.15.76:0/3480530785 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face400a850 con 0x7facec15ad80 2023-11-09T03:46:15.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.287+0000 7facf2039640 1 -- 172.21.15.76:0/3480530785 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facec15dfc0 con 0x7facec15ad80 2023-11-09T03:46:15.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.287+0000 7facf2039640 1 -- 172.21.15.76:0/3480530785 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facec1f2ab0 con 0x7facec15ad80 2023-11-09T03:46:15.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.288+0000 7facf2039640 1 -- 172.21.15.76:0/3480530785 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facec0ccb60 con 0x7facec15ad80 2023-11-09T03:46:15.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.288+0000 7facd3fff640 1 -- 172.21.15.76:0/3480530785 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7face400a9b0 con 0x7facec15ad80 2023-11-09T03:46:15.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.289+0000 7facd3fff640 1 --2- 172.21.15.76:0/3480530785 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7facd806ae70 0x7facd806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:15.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.289+0000 7facd3fff640 1 -- 172.21.15.76:0/3480530785 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7face4089b40 con 0x7facec15ad80 2023-11-09T03:46:15.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.289+0000 7faceb7fe640 1 --2- 172.21.15.76:0/3480530785 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7facd806ae70 0x7facd806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:15.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.289+0000 7faceb7fe640 1 --2- 172.21.15.76:0/3480530785 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7facd806ae70 0x7facd806d330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7facdc036000 tx=0x7facdc03b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:15.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.294+0000 7facd3fff640 1 -- 172.21.15.76:0/3480530785 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7facec0ccb60 con 0x7facec15ad80 2023-11-09T03:46:15.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.464+0000 7facf2039640 1 -- 172.21.15.76:0/3480530785 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7facec0ccb60 con 0x7facec15ad80 2023-11-09T03:46:15.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.465+0000 7facd3fff640 1 -- 172.21.15.76:0/3480530785 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18710 (secure 0 0 0) 0x7facec0ccb60 con 0x7facec15ad80 2023-11-09T03:46:15.465 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:15.465 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":27,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:46:12.537265+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:46:15.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.467+0000 7facf2039640 1 -- 172.21.15.76:0/3480530785 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7facd806ae70 msgr2=0x7facd806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:15.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.467+0000 7facf2039640 1 --2- 172.21.15.76:0/3480530785 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7facd806ae70 0x7facd806d330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7facdc036000 tx=0x7facdc03b040 comp rx=0 tx=0).stop 2023-11-09T03:46:15.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.467+0000 7facf2039640 1 -- 172.21.15.76:0/3480530785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facec15ad80 msgr2=0x7facec15cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:15.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.467+0000 7facf2039640 1 --2- 172.21.15.76:0/3480530785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facec15ad80 0x7facec15cfb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7face400c780 tx=0x7face400cc40 comp rx=0 tx=0).stop 2023-11-09T03:46:15.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.467+0000 7facf2039640 1 -- 172.21.15.76:0/3480530785 shutdown_connections 2023-11-09T03:46:15.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.467+0000 7facf2039640 1 --2- 172.21.15.76:0/3480530785 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7facd806ae70 0x7facd806d330 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.467+0000 7facf2039640 1 --2- 172.21.15.76:0/3480530785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facec15ad80 0x7facec15cfb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.467+0000 7facf2039640 1 --2- 172.21.15.76:0/3480530785 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facec0c9e10 0x7facec1f2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.467+0000 7facf2039640 1 --2- 172.21.15.76:0/3480530785 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facec0b42c0 0x7facec1541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.467+0000 7facf2039640 1 -- 172.21.15.76:0/3480530785 >> 172.21.15.76:0/3480530785 conn(0x7facec0ce950 msgr2=0x7facec0b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:15.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.467+0000 7facf2039640 1 -- 172.21.15.76:0/3480530785 shutdown_connections 2023-11-09T03:46:15.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.467+0000 7facf2039640 1 -- 172.21.15.76:0/3480530785 wait complete. 2023-11-09T03:46:15.478 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:45:28.704715+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:45:29.717614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-09T03:45:33.440577+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-09T03:46:15.479 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:46:15.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.581+0000 7fad86f78640 1 Processor -- start 2023-11-09T03:46:15.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.581+0000 7fad86f78640 1 -- start start 2023-11-09T03:46:15.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.582+0000 7fad86f78640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fad800d0710 0x7fad800cd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:15.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.582+0000 7fad86f78640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fad800cda50 0x7fad800b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:15.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.582+0000 7fad86f78640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad800d0280 con 0x7fad8015e430 2023-11-09T03:46:15.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.582+0000 7fad86f78640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad800cf670 con 0x7fad800cda50 2023-11-09T03:46:15.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.582+0000 7fad86f78640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad800cf950 con 0x7fad800d0710 2023-11-09T03:46:15.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.582+0000 7fad85515640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fad800cda50 0x7fad800b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:15.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.582+0000 7fad84d14640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fad8015e430 0x7fad800cec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43606/0 (socket says 172.21.15.76:43606) 2023-11-09T03:46:15.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.582+0000 7fad84d14640 1 -- 172.21.15.76:0/2099615138 learned_addr learned my addr 172.21.15.76:0/2099615138 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:15.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.582+0000 7fad7ffff640 1 --2- 172.21.15.76:0/2099615138 >> v2:172.21.15.113:3300/0 conn(0x7fad800d0710 0x7fad800cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:15.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.582+0000 7fad85515640 1 -- 172.21.15.76:0/2099615138 >> v2:172.21.15.113:3300/0 conn(0x7fad800d0710 msgr2=0x7fad800cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:15.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.582+0000 7fad85515640 1 --2- 172.21.15.76:0/2099615138 >> v2:172.21.15.113:3300/0 conn(0x7fad800d0710 0x7fad800cd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.582+0000 7fad85515640 1 -- 172.21.15.76:0/2099615138 >> v1:172.21.15.76:6789/0 conn(0x7fad8015e430 legacy=0x7fad800cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:15.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.582+0000 7fad85515640 1 -- 172.21.15.76:0/2099615138 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad800cfc30 con 0x7fad800cda50 2023-11-09T03:46:15.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.582+0000 7fad7f7fe640 1 -- 172.21.15.76:0/2099615138 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2345356654 0 0) 0x7fad800d0280 con 0x7fad8015e430 2023-11-09T03:46:15.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.582+0000 7fad7ffff640 1 --2- 172.21.15.76:0/2099615138 >> v2:172.21.15.113:3300/0 conn(0x7fad800d0710 0x7fad800cd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:15.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.582+0000 7fad85515640 1 --2- 172.21.15.76:0/2099615138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad800cda50 0x7fad800b3d40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fad800cb3c0 tx=0x7fad7802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59dfe27e0b9f5535 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:15.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.583+0000 7fad7f7fe640 1 -- 172.21.15.76:0/2099615138 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7802cc80 con 0x7fad800cda50 2023-11-09T03:46:15.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.583+0000 7fad7f7fe640 1 -- 172.21.15.76:0/2099615138 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7802cde0 con 0x7fad800cda50 2023-11-09T03:46:15.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.583+0000 7fad7f7fe640 1 -- 172.21.15.76:0/2099615138 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad78034660 con 0x7fad800cda50 2023-11-09T03:46:15.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.583+0000 7fad86f78640 1 -- 172.21.15.76:0/2099615138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad800cda50 msgr2=0x7fad800b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:15.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.583+0000 7fad86f78640 1 --2- 172.21.15.76:0/2099615138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad800cda50 0x7fad800b3d40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fad800cb3c0 tx=0x7fad7802bed0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.583+0000 7fad86f78640 1 -- 172.21.15.76:0/2099615138 shutdown_connections 2023-11-09T03:46:15.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.583+0000 7fad86f78640 1 --2- 172.21.15.76:0/2099615138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad800cda50 0x7fad800b3d40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.583+0000 7fad86f78640 1 --2- 172.21.15.76:0/2099615138 >> v2:172.21.15.113:3300/0 conn(0x7fad800d0710 0x7fad800cd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.583+0000 7fad86f78640 1 -- 172.21.15.76:0/2099615138 >> 172.21.15.76:0/2099615138 conn(0x7fad8006ea20 msgr2=0x7fad8006ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:15.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.583+0000 7fad86f78640 1 -- 172.21.15.76:0/2099615138 shutdown_connections 2023-11-09T03:46:15.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.583+0000 7fad86f78640 1 -- 172.21.15.76:0/2099615138 wait complete. 2023-11-09T03:46:15.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.584+0000 7fad86f78640 1 Processor -- start 2023-11-09T03:46:15.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.584+0000 7fad86f78640 1 -- start start 2023-11-09T03:46:15.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.586+0000 7fad86f78640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad800cda50 0x7fad800af050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:15.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.586+0000 7fad86f78640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad800d0710 0x7fad801f5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:15.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.586+0000 7fad86f78640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad8015e430 0x7fad800b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:15.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.586+0000 7fad86f78640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad801f66b0 con 0x7fad800d0710 2023-11-09T03:46:15.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.586+0000 7fad86f78640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad801f20c0 con 0x7fad800cda50 2023-11-09T03:46:15.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.586+0000 7fad86f78640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad801f5e10 con 0x7fad8015e430 2023-11-09T03:46:15.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.586+0000 7fad7ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad800d0710 0x7fad801f5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:15.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.586+0000 7fad7ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad800d0710 0x7fad801f5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36728/0 (socket says 172.21.15.76:36728) 2023-11-09T03:46:15.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.586+0000 7fad7ffff640 1 -- 172.21.15.76:0/1734660078 learned_addr learned my addr 172.21.15.76:0/1734660078 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:15.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.586+0000 7fad7ffff640 1 -- 172.21.15.76:0/1734660078 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad8015e430 msgr2=0x7fad800b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:15.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.586+0000 7fad7ffff640 1 --2- 172.21.15.76:0/1734660078 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad8015e430 0x7fad800b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.586+0000 7fad7ffff640 1 -- 172.21.15.76:0/1734660078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad800cda50 msgr2=0x7fad800af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:15.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.586+0000 7fad7ffff640 1 --2- 172.21.15.76:0/1734660078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad800cda50 0x7fad800af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.586+0000 7fad7ffff640 1 -- 172.21.15.76:0/1734660078 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad78009a70 con 0x7fad800d0710 2023-11-09T03:46:15.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.586+0000 7fad7ffff640 1 --2- 172.21.15.76:0/1734660078 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad800d0710 0x7fad801f5710 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fad7400c820 tx=0x7fad7400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:15.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.586+0000 7fad7d7fa640 1 -- 172.21.15.76:0/1734660078 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad74007a30 con 0x7fad800d0710 2023-11-09T03:46:15.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.586+0000 7fad7d7fa640 1 -- 172.21.15.76:0/1734660078 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad74007b90 con 0x7fad800d0710 2023-11-09T03:46:15.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.586+0000 7fad86f78640 1 -- 172.21.15.76:0/1734660078 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad801f60f0 con 0x7fad800d0710 2023-11-09T03:46:15.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.586+0000 7fad7d7fa640 1 -- 172.21.15.76:0/1734660078 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7400ce70 con 0x7fad800d0710 2023-11-09T03:46:15.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.586+0000 7fad86f78640 1 -- 172.21.15.76:0/1734660078 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad801f2330 con 0x7fad800d0710 2023-11-09T03:46:15.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.587+0000 7fad86f78640 1 -- 172.21.15.76:0/1734660078 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad801f2550 con 0x7fad800d0710 2023-11-09T03:46:15.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.587+0000 7fad7d7fa640 1 -- 172.21.15.76:0/1734660078 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fad740113f0 con 0x7fad800d0710 2023-11-09T03:46:15.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.588+0000 7fad7d7fa640 1 --2- 172.21.15.76:0/1734660078 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fad6c06ae70 0x7fad6c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:15.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.588+0000 7fad7d7fa640 1 -- 172.21.15.76:0/1734660078 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fad74089b90 con 0x7fad800d0710 2023-11-09T03:46:15.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.588+0000 7fad84d14640 1 --2- 172.21.15.76:0/1734660078 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fad6c06ae70 0x7fad6c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:15.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.588+0000 7fad84d14640 1 --2- 172.21.15.76:0/1734660078 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fad6c06ae70 0x7fad6c06d330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fad70007fd0 tx=0x7fad70007920 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:15.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.592+0000 7fad7d7fa640 1 -- 172.21.15.76:0/1734660078 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fad74054860 con 0x7fad800d0710 2023-11-09T03:46:15.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.762+0000 7fad86f78640 1 -- 172.21.15.76:0/1734660078 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fad80074290 con 0x7fad800d0710 2023-11-09T03:46:15.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.763+0000 7fad7d7fa640 1 -- 172.21.15.76:0/1734660078 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fad7401f0a0 con 0x7fad800d0710 2023-11-09T03:46:15.762 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:46:15.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.764+0000 7fad86f78640 1 -- 172.21.15.76:0/1734660078 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fad6c06ae70 msgr2=0x7fad6c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:15.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.764+0000 7fad86f78640 1 --2- 172.21.15.76:0/1734660078 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fad6c06ae70 0x7fad6c06d330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fad70007fd0 tx=0x7fad70007920 comp rx=0 tx=0).stop 2023-11-09T03:46:15.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.764+0000 7fad86f78640 1 -- 172.21.15.76:0/1734660078 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad800d0710 msgr2=0x7fad801f5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:15.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.764+0000 7fad86f78640 1 --2- 172.21.15.76:0/1734660078 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad800d0710 0x7fad801f5710 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fad7400c820 tx=0x7fad7400cc20 comp rx=0 tx=0).stop 2023-11-09T03:46:15.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.764+0000 7fad86f78640 1 -- 172.21.15.76:0/1734660078 shutdown_connections 2023-11-09T03:46:15.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.764+0000 7fad86f78640 1 --2- 172.21.15.76:0/1734660078 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fad6c06ae70 0x7fad6c06d330 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.764+0000 7fad86f78640 1 --2- 172.21.15.76:0/1734660078 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad8015e430 0x7fad800b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.764+0000 7fad86f78640 1 --2- 172.21.15.76:0/1734660078 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad800d0710 0x7fad801f5710 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.764+0000 7fad86f78640 1 --2- 172.21.15.76:0/1734660078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad800cda50 0x7fad800af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.764+0000 7fad86f78640 1 -- 172.21.15.76:0/1734660078 >> 172.21.15.76:0/1734660078 conn(0x7fad8006ea20 msgr2=0x7fad800b57d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:15.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.764+0000 7fad86f78640 1 -- 172.21.15.76:0/1734660078 shutdown_connections 2023-11-09T03:46:15.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.765+0000 7fad86f78640 1 -- 172.21.15.76:0/1734660078 wait complete. 2023-11-09T03:46:15.775 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:46:15.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.878+0000 7f72bb9c5640 1 Processor -- start 2023-11-09T03:46:15.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.878+0000 7f72bb9c5640 1 -- start start 2023-11-09T03:46:15.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.879+0000 7f72bb9c5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f72b40ae1d0 0x7f72b40cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:15.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.879+0000 7f72bb9c5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72b415aff0 con 0x7f72b4159c00 2023-11-09T03:46:15.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.879+0000 7f72bb9c5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72b415a3e0 con 0x7f72b40c4ff0 2023-11-09T03:46:15.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.879+0000 7f72bb9c5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72b415a6c0 con 0x7f72b40ae1d0 2023-11-09T03:46:15.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.879+0000 7f72b9761640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f72b4159c00 0x7f72b40c48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43620/0 (socket says 172.21.15.76:43620) 2023-11-09T03:46:15.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.879+0000 7f72b9761640 1 -- 172.21.15.76:0/2674309058 learned_addr learned my addr 172.21.15.76:0/2674309058 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:15.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.879+0000 7f72abfff640 1 -- 172.21.15.76:0/2674309058 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3424094610 0 0) 0x7f72b415aff0 con 0x7f72b4159c00 2023-11-09T03:46:15.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.879+0000 7f72abfff640 1 -- 172.21.15.76:0/2674309058 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72a0003520 con 0x7f72b4159c00 2023-11-09T03:46:15.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.880+0000 7f72b9f62640 1 --2- 172.21.15.76:0/2674309058 >> v2:172.21.15.113:3300/0 conn(0x7f72b40ae1d0 0x7f72b40cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:15.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.880+0000 7f72abfff640 1 -- 172.21.15.76:0/2674309058 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4014498354 0 0) 0x7f72a0003520 con 0x7f72b4159c00 2023-11-09T03:46:15.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.880+0000 7f72abfff640 1 -- 172.21.15.76:0/2674309058 >> v2:172.21.15.113:3300/0 conn(0x7f72b40ae1d0 msgr2=0x7f72b40cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:15.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.880+0000 7f72abfff640 1 --2- 172.21.15.76:0/2674309058 >> v2:172.21.15.113:3300/0 conn(0x7f72b40ae1d0 0x7f72b40cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.880+0000 7f72abfff640 1 -- 172.21.15.76:0/2674309058 >> v1:172.21.15.113:6789/0 conn(0x7f72b40c4ff0 legacy=0x7f72b40adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:15.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.880+0000 7f72abfff640 1 -- 172.21.15.76:0/2674309058 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b415a9a0 con 0x7f72b4159c00 2023-11-09T03:46:15.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.880+0000 7f72abfff640 1 -- 172.21.15.76:0/2674309058 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f72b00019f0 con 0x7f72b4159c00 2023-11-09T03:46:15.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.880+0000 7f72abfff640 1 -- 172.21.15.76:0/2674309058 >> v1:172.21.15.76:6789/0 conn(0x7f72b4159c00 legacy=0x7f72b40c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:15.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.880+0000 7f72abfff640 1 --2- 172.21.15.76:0/2674309058 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72a0003970 0x7f72a0003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:15.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.880+0000 7f72abfff640 1 --2- 172.21.15.76:0/2674309058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a00047c0 0x7f72a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:15.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.880+0000 7f72abfff640 1 --2- 172.21.15.76:0/2674309058 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72a0005180 0x7f72a000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:15.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.880+0000 7f72abfff640 1 -- 172.21.15.76:0/2674309058 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b415a6c0 con 0x7f72a00047c0 2023-11-09T03:46:15.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.880+0000 7f72abfff640 1 -- 172.21.15.76:0/2674309058 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a000c0b0 con 0x7f72a0005180 2023-11-09T03:46:15.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.880+0000 7f72abfff640 1 -- 172.21.15.76:0/2674309058 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a000c270 con 0x7f72a0003970 2023-11-09T03:46:15.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.880+0000 7f72b8f60640 1 --2- 172.21.15.76:0/2674309058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a00047c0 0x7f72a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:15.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.880+0000 7f72b9761640 1 --2- 172.21.15.76:0/2674309058 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72a0003970 0x7f72a0003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:15.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.880+0000 7f72b9f62640 1 --2- 172.21.15.76:0/2674309058 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72a0005180 0x7f72a000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:15.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.880+0000 7f72b8f60640 1 -- 172.21.15.76:0/2674309058 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72a0003970 msgr2=0x7f72a0003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:15.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.880+0000 7f72b8f60640 1 --2- 172.21.15.76:0/2674309058 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72a0003970 0x7f72a0003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.880+0000 7f72b8f60640 1 -- 172.21.15.76:0/2674309058 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72a0005180 msgr2=0x7f72a000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:15.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.880+0000 7f72b8f60640 1 --2- 172.21.15.76:0/2674309058 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72a0005180 0x7f72a000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.880+0000 7f72b9f62640 1 -- 172.21.15.76:0/2674309058 reap_dead start 2023-11-09T03:46:15.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.880+0000 7f72b8f60640 1 -- 172.21.15.76:0/2674309058 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a0003520 con 0x7f72a00047c0 2023-11-09T03:46:15.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.881+0000 7f72b8f60640 1 --2- 172.21.15.76:0/2674309058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a00047c0 0x7f72a0004c40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f72b40b4a60 tx=0x7f72a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6d333344c5ed36 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:15.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.881+0000 7f72abfff640 1 -- 172.21.15.76:0/2674309058 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a4032600 con 0x7f72a00047c0 2023-11-09T03:46:15.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.881+0000 7f72abfff640 1 -- 172.21.15.76:0/2674309058 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a402e050 con 0x7f72a00047c0 2023-11-09T03:46:15.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.881+0000 7f72abfff640 1 -- 172.21.15.76:0/2674309058 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a40329a0 con 0x7f72a00047c0 2023-11-09T03:46:15.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.881+0000 7f72bb9c5640 1 -- 172.21.15.76:0/2674309058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a00047c0 msgr2=0x7f72a0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:15.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.881+0000 7f72bb9c5640 1 --2- 172.21.15.76:0/2674309058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a00047c0 0x7f72a0004c40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f72b40b4a60 tx=0x7f72a00057f0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.882+0000 7f72bb9c5640 1 -- 172.21.15.76:0/2674309058 shutdown_connections 2023-11-09T03:46:15.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.882+0000 7f72bb9c5640 1 --2- 172.21.15.76:0/2674309058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a00047c0 0x7f72a0004c40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.882+0000 7f72bb9c5640 1 -- 172.21.15.76:0/2674309058 >> 172.21.15.76:0/2674309058 conn(0x7f72b40ce950 msgr2=0x7f72b40ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:15.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.882+0000 7f72bb9c5640 1 -- 172.21.15.76:0/2674309058 shutdown_connections 2023-11-09T03:46:15.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.882+0000 7f72bb9c5640 1 -- 172.21.15.76:0/2674309058 wait complete. 2023-11-09T03:46:15.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.883+0000 7f72bb9c5640 1 Processor -- start 2023-11-09T03:46:15.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.883+0000 7f72bb9c5640 1 -- start start 2023-11-09T03:46:15.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.883+0000 7f72bb9c5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72a00047c0 0x7f72b4150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:15.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.883+0000 7f72bb9c5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72b4152830 0x7f72b40ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:15.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.883+0000 7f72bb9c5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72b40adc90 0x7f72b41f2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:15.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.883+0000 7f72bb9c5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b41539d0 con 0x7f72b4152830 2023-11-09T03:46:15.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.883+0000 7f72bb9c5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b4152dc0 con 0x7f72a00047c0 2023-11-09T03:46:15.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.883+0000 7f72bb9c5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b41530a0 con 0x7f72b40adc90 2023-11-09T03:46:15.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.884+0000 7f72b8f60640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72b4152830 0x7f72b40ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:15.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.884+0000 7f72b8f60640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72b4152830 0x7f72b40ad750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36744/0 (socket says 172.21.15.76:36744) 2023-11-09T03:46:15.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.884+0000 7f72b8f60640 1 -- 172.21.15.76:0/2999528963 learned_addr learned my addr 172.21.15.76:0/2999528963 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:15.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.884+0000 7f72b9761640 1 --2- 172.21.15.76:0/2999528963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72a00047c0 0x7f72b4150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:15.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.884+0000 7f72b9f62640 1 --2- 172.21.15.76:0/2999528963 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72b40adc90 0x7f72b41f2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:15.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.884+0000 7f72b8f60640 1 -- 172.21.15.76:0/2999528963 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72b40adc90 msgr2=0x7f72b41f2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:15.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.884+0000 7f72b8f60640 1 --2- 172.21.15.76:0/2999528963 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72b40adc90 0x7f72b41f2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.884+0000 7f72b8f60640 1 -- 172.21.15.76:0/2999528963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72a00047c0 msgr2=0x7f72b4150cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:15.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.884+0000 7f72b8f60640 1 --2- 172.21.15.76:0/2999528963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72a00047c0 0x7f72b4150cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:15.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.884+0000 7f72b8f60640 1 -- 172.21.15.76:0/2999528963 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b4153380 con 0x7f72b4152830 2023-11-09T03:46:15.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.884+0000 7f72b9f62640 1 --2- 172.21.15.76:0/2999528963 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72b40adc90 0x7f72b41f2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:15.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.884+0000 7f72b8f60640 1 --2- 172.21.15.76:0/2999528963 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72b4152830 0x7f72b40ad750 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f72a4035040 tx=0x7f72a40036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:15.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.884+0000 7f72a9ffb640 1 -- 172.21.15.76:0/2999528963 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a4007680 con 0x7f72b4152830 2023-11-09T03:46:15.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.884+0000 7f72a9ffb640 1 -- 172.21.15.76:0/2999528963 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a403e450 con 0x7f72b4152830 2023-11-09T03:46:15.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.884+0000 7f72bb9c5640 1 -- 172.21.15.76:0/2999528963 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b4153660 con 0x7f72b4152830 2023-11-09T03:46:15.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.884+0000 7f72a9ffb640 1 -- 172.21.15.76:0/2999528963 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a403e7e0 con 0x7f72b4152830 2023-11-09T03:46:15.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.884+0000 7f72bb9c5640 1 -- 172.21.15.76:0/2999528963 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b41513f0 con 0x7f72b4152830 2023-11-09T03:46:15.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.885+0000 7f72bb9c5640 1 -- 172.21.15.76:0/2999528963 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72b41537f0 con 0x7f72b4152830 2023-11-09T03:46:15.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.886+0000 7f72a9ffb640 1 -- 172.21.15.76:0/2999528963 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f72a403e940 con 0x7f72b4152830 2023-11-09T03:46:15.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.886+0000 7f72a9ffb640 1 --2- 172.21.15.76:0/2999528963 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f729806ae70 0x7f729806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:15.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.886+0000 7f72a9ffb640 1 -- 172.21.15.76:0/2999528963 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f72a40ab630 con 0x7f72b4152830 2023-11-09T03:46:15.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.886+0000 7f72b9761640 1 --2- 172.21.15.76:0/2999528963 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f729806ae70 0x7f729806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:15.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.887+0000 7f72b9761640 1 --2- 172.21.15.76:0/2999528963 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f729806ae70 0x7f729806d330 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f72b0002030 tx=0x7f72b00021f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:15.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:15.890+0000 7f72a9ffb640 1 -- 172.21.15.76:0/2999528963 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f72a4076420 con 0x7f72b4152830 2023-11-09T03:46:16.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.054+0000 7f72bb9c5640 1 -- 172.21.15.76:0/2999528963 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f72b4074290 con 0x7f72b4152830 2023-11-09T03:46:16.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.055+0000 7f72a9ffb640 1 -- 172.21.15.76:0/2999528963 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f72a4079a40 con 0x7f72b4152830 2023-11-09T03:46:16.055 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:46:16.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.057+0000 7f72bb9c5640 1 -- 172.21.15.76:0/2999528963 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f729806ae70 msgr2=0x7f729806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:16.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.057+0000 7f72bb9c5640 1 --2- 172.21.15.76:0/2999528963 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f729806ae70 0x7f729806d330 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f72b0002030 tx=0x7f72b00021f0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.057+0000 7f72bb9c5640 1 -- 172.21.15.76:0/2999528963 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72b4152830 msgr2=0x7f72b40ad750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:16.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.057+0000 7f72bb9c5640 1 --2- 172.21.15.76:0/2999528963 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72b4152830 0x7f72b40ad750 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f72a4035040 tx=0x7f72a40036f0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.057+0000 7f72bb9c5640 1 -- 172.21.15.76:0/2999528963 shutdown_connections 2023-11-09T03:46:16.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.057+0000 7f72bb9c5640 1 --2- 172.21.15.76:0/2999528963 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f729806ae70 0x7f729806d330 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.057+0000 7f72bb9c5640 1 --2- 172.21.15.76:0/2999528963 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72b40adc90 0x7f72b41f2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.057+0000 7f72bb9c5640 1 --2- 172.21.15.76:0/2999528963 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72b4152830 0x7f72b40ad750 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.057+0000 7f72bb9c5640 1 --2- 172.21.15.76:0/2999528963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72a00047c0 0x7f72b4150cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.057+0000 7f72bb9c5640 1 -- 172.21.15.76:0/2999528963 >> 172.21.15.76:0/2999528963 conn(0x7f72b40ce950 msgr2=0x7f72b415b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:16.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.057+0000 7f72bb9c5640 1 -- 172.21.15.76:0/2999528963 shutdown_connections 2023-11-09T03:46:16.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.057+0000 7f72bb9c5640 1 -- 172.21.15.76:0/2999528963 wait complete. 2023-11-09T03:46:16.067 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:46:16.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.170+0000 7f06d359e640 1 Processor -- start 2023-11-09T03:46:16.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.170+0000 7f06d359e640 1 -- start start 2023-11-09T03:46:16.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.171+0000 7f06d359e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f06d415ce20 0x7f06d40cba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:16.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.171+0000 7f06d359e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06d40ce410 0x7f06d40ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:16.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.171+0000 7f06d359e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06d4160f60 con 0x7f06d415d200 2023-11-09T03:46:16.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.171+0000 7f06d359e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06d4161240 con 0x7f06d415ce20 2023-11-09T03:46:16.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.171+0000 7f06d359e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d4161520 con 0x7f06d40ce410 2023-11-09T03:46:16.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.171+0000 7f06d259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f06d415ce20 0x7f06d40cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:16.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.171+0000 7f06d1d9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f06d415d200 0x7f06d40cdd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43632/0 (socket says 172.21.15.76:43632) 2023-11-09T03:46:16.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.171+0000 7f06d1d9b640 1 -- 172.21.15.76:0/2455234441 learned_addr learned my addr 172.21.15.76:0/2455234441 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:16.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.171+0000 7f06d2d9d640 1 --2- 172.21.15.76:0/2455234441 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06d40ce410 0x7f06d40ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:16.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.171+0000 7f06d259c640 1 -- 172.21.15.76:0/2455234441 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06d40ce410 msgr2=0x7f06d40ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:16.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.171+0000 7f06d259c640 1 --2- 172.21.15.76:0/2455234441 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06d40ce410 0x7f06d40ca640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.171+0000 7f06d259c640 1 -- 172.21.15.76:0/2455234441 >> v1:172.21.15.76:6789/0 conn(0x7f06d415d200 legacy=0x7f06d40cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:16.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.171+0000 7f06d259c640 1 -- 172.21.15.76:0/2455234441 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d4161800 con 0x7f06d415ce20 2023-11-09T03:46:16.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.171+0000 7f06d159a640 1 -- 172.21.15.76:0/2455234441 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3979550214 0 0) 0x7f06d4160f60 con 0x7f06d415d200 2023-11-09T03:46:16.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.172+0000 7f06d259c640 1 --2- 172.21.15.76:0/2455234441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06d415ce20 0x7f06d40cba80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f06c4009d90 tx=0x7f06c402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36c352cfc80cf5d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:16.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.172+0000 7f06d159a640 1 -- 172.21.15.76:0/2455234441 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c402cc50 con 0x7f06d415ce20 2023-11-09T03:46:16.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.172+0000 7f06d159a640 1 -- 172.21.15.76:0/2455234441 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c402cdb0 con 0x7f06d415ce20 2023-11-09T03:46:16.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.172+0000 7f06d159a640 1 -- 172.21.15.76:0/2455234441 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c4034660 con 0x7f06d415ce20 2023-11-09T03:46:16.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.173+0000 7f06d359e640 1 -- 172.21.15.76:0/2455234441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06d415ce20 msgr2=0x7f06d40cba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:16.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.173+0000 7f06d359e640 1 --2- 172.21.15.76:0/2455234441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06d415ce20 0x7f06d40cba80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f06c4009d90 tx=0x7f06c402bea0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.173+0000 7f06d359e640 1 -- 172.21.15.76:0/2455234441 shutdown_connections 2023-11-09T03:46:16.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.173+0000 7f06d359e640 1 --2- 172.21.15.76:0/2455234441 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06d40ce410 0x7f06d40ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.173+0000 7f06d359e640 1 --2- 172.21.15.76:0/2455234441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06d415ce20 0x7f06d40cba80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.173+0000 7f06d359e640 1 -- 172.21.15.76:0/2455234441 >> 172.21.15.76:0/2455234441 conn(0x7f06d40cf820 msgr2=0x7f06d40cfc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:16.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.173+0000 7f06d359e640 1 -- 172.21.15.76:0/2455234441 shutdown_connections 2023-11-09T03:46:16.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.173+0000 7f06d359e640 1 -- 172.21.15.76:0/2455234441 wait complete. 2023-11-09T03:46:16.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.174+0000 7f06d359e640 1 Processor -- start 2023-11-09T03:46:16.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.174+0000 7f06d359e640 1 -- start start 2023-11-09T03:46:16.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.175+0000 7f06d359e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06d40ce410 0x7f06d415f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:16.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.175+0000 7f06d359e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06d415ce20 0x7f06d406aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:16.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.175+0000 7f06d359e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06d415d200 0x7f06d40c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:16.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.175+0000 7f06d359e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d415ee70 con 0x7f06d415d200 2023-11-09T03:46:16.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.175+0000 7f06d359e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d415e840 con 0x7f06d40ce410 2023-11-09T03:46:16.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.175+0000 7f06d359e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d41f33f0 con 0x7f06d415ce20 2023-11-09T03:46:16.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.175+0000 7f06d2d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06d415d200 0x7f06d40c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:16.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.175+0000 7f06d2d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06d415d200 0x7f06d40c7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36768/0 (socket says 172.21.15.76:36768) 2023-11-09T03:46:16.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.175+0000 7f06d2d9d640 1 -- 172.21.15.76:0/3316764351 learned_addr learned my addr 172.21.15.76:0/3316764351 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:16.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.175+0000 7f06d259c640 1 --2- 172.21.15.76:0/3316764351 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06d40ce410 0x7f06d415f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:16.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.175+0000 7f06d1d9b640 1 --2- 172.21.15.76:0/3316764351 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06d415ce20 0x7f06d406aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:16.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.175+0000 7f06d2d9d640 1 -- 172.21.15.76:0/3316764351 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06d415ce20 msgr2=0x7f06d406aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:16.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.175+0000 7f06d2d9d640 1 --2- 172.21.15.76:0/3316764351 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06d415ce20 0x7f06d406aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.175+0000 7f06d2d9d640 1 -- 172.21.15.76:0/3316764351 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06d40ce410 msgr2=0x7f06d415f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:16.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.175+0000 7f06d2d9d640 1 --2- 172.21.15.76:0/3316764351 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06d40ce410 0x7f06d415f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.175+0000 7f06d2d9d640 1 -- 172.21.15.76:0/3316764351 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c4009a70 con 0x7f06d415d200 2023-11-09T03:46:16.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.175+0000 7f06d1d9b640 1 --2- 172.21.15.76:0/3316764351 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06d415ce20 0x7f06d406aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:16.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.175+0000 7f06d2d9d640 1 --2- 172.21.15.76:0/3316764351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06d415d200 0x7f06d40c7ad0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f06cc00d5d0 tx=0x7f06cc00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:16.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.176+0000 7f06c2ffd640 1 -- 172.21.15.76:0/3316764351 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06cc0125f0 con 0x7f06d415d200 2023-11-09T03:46:16.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.176+0000 7f06c2ffd640 1 -- 172.21.15.76:0/3316764351 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06cc012750 con 0x7f06d415d200 2023-11-09T03:46:16.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.176+0000 7f06d359e640 1 -- 172.21.15.76:0/3316764351 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d415eb00 con 0x7f06d415d200 2023-11-09T03:46:16.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.176+0000 7f06c2ffd640 1 -- 172.21.15.76:0/3316764351 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06cc012b00 con 0x7f06d415d200 2023-11-09T03:46:16.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.176+0000 7f06d359e640 1 -- 172.21.15.76:0/3316764351 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d406b1d0 con 0x7f06d415d200 2023-11-09T03:46:16.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.177+0000 7f06d359e640 1 -- 172.21.15.76:0/3316764351 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06d415f020 con 0x7f06d415d200 2023-11-09T03:46:16.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.177+0000 7f06c2ffd640 1 -- 172.21.15.76:0/3316764351 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f06cc012c60 con 0x7f06d415d200 2023-11-09T03:46:16.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.177+0000 7f06c2ffd640 1 --2- 172.21.15.76:0/3316764351 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f06b406ae70 0x7f06b406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:16.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.177+0000 7f06c2ffd640 1 -- 172.21.15.76:0/3316764351 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f06cc089cd0 con 0x7f06d415d200 2023-11-09T03:46:16.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.178+0000 7f06d259c640 1 --2- 172.21.15.76:0/3316764351 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f06b406ae70 0x7f06b406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:16.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.178+0000 7f06d259c640 1 --2- 172.21.15.76:0/3316764351 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f06b406ae70 0x7f06b406d330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f06c4036000 tx=0x7f06c4030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:16.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.182+0000 7f06c2ffd640 1 -- 172.21.15.76:0/3316764351 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f06d415f020 con 0x7f06d415d200 2023-11-09T03:46:16.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.348+0000 7f06d359e640 1 -- 172.21.15.76:0/3316764351 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f06d406b1d0 con 0x7f06d415d200 2023-11-09T03:46:16.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.348+0000 7f06c2ffd640 1 -- 172.21.15.76:0/3316764351 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f06d406b1d0 con 0x7f06d415d200 2023-11-09T03:46:16.348 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:46:16.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.350+0000 7f06d359e640 1 -- 172.21.15.76:0/3316764351 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f06b406ae70 msgr2=0x7f06b406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:16.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.350+0000 7f06d359e640 1 --2- 172.21.15.76:0/3316764351 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f06b406ae70 0x7f06b406d330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f06c4036000 tx=0x7f06c4030000 comp rx=0 tx=0).stop 2023-11-09T03:46:16.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.350+0000 7f06d359e640 1 -- 172.21.15.76:0/3316764351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06d415d200 msgr2=0x7f06d40c7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:16.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.350+0000 7f06d359e640 1 --2- 172.21.15.76:0/3316764351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06d415d200 0x7f06d40c7ad0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f06cc00d5d0 tx=0x7f06cc00da90 comp rx=0 tx=0).stop 2023-11-09T03:46:16.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.350+0000 7f06d359e640 1 -- 172.21.15.76:0/3316764351 shutdown_connections 2023-11-09T03:46:16.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.350+0000 7f06d359e640 1 --2- 172.21.15.76:0/3316764351 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f06b406ae70 0x7f06b406d330 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.350+0000 7f06d359e640 1 --2- 172.21.15.76:0/3316764351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06d415d200 0x7f06d40c7ad0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.350+0000 7f06d359e640 1 --2- 172.21.15.76:0/3316764351 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06d415ce20 0x7f06d406aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.350+0000 7f06d359e640 1 --2- 172.21.15.76:0/3316764351 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06d40ce410 0x7f06d415f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.350+0000 7f06d359e640 1 -- 172.21.15.76:0/3316764351 >> 172.21.15.76:0/3316764351 conn(0x7f06d40cf820 msgr2=0x7f06d4072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:16.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.350+0000 7f06d359e640 1 -- 172.21.15.76:0/3316764351 shutdown_connections 2023-11-09T03:46:16.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.350+0000 7f06d359e640 1 -- 172.21.15.76:0/3316764351 wait complete. 2023-11-09T03:46:16.360 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-09T03:46:16.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.464+0000 7fd1653d7640 1 Processor -- start 2023-11-09T03:46:16.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.464+0000 7fd1653d7640 1 -- start start 2023-11-09T03:46:16.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.465+0000 7fd1653d7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1600cf180 0x7fd16006c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:16.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.465+0000 7fd1653d7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd1600cd570 0x7fd1600cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:16.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.465+0000 7fd1653d7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1600cdf20 con 0x7fd16015cfd0 2023-11-09T03:46:16.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.465+0000 7fd1653d7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd16015d3b0 con 0x7fd1600cd570 2023-11-09T03:46:16.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.465+0000 7fd1653d7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1600cea20 con 0x7fd1600cf180 2023-11-09T03:46:16.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.465+0000 7fd15f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd1600cd570 0x7fd1600cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:16.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.465+0000 7fd15effd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd16015cfd0 0x7fd160161440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43636/0 (socket says 172.21.15.76:43636) 2023-11-09T03:46:16.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.465+0000 7fd15effd640 1 -- 172.21.15.76:0/1882115426 learned_addr learned my addr 172.21.15.76:0/1882115426 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:16.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.466+0000 7fd15e7fc640 1 --2- 172.21.15.76:0/1882115426 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1600cf180 0x7fd16006c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:16.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.466+0000 7fd15f7fe640 1 -- 172.21.15.76:0/1882115426 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1600cf180 msgr2=0x7fd16006c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:16.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.466+0000 7fd15f7fe640 1 --2- 172.21.15.76:0/1882115426 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1600cf180 0x7fd16006c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.466+0000 7fd15f7fe640 1 -- 172.21.15.76:0/1882115426 >> v1:172.21.15.76:6789/0 conn(0x7fd16015cfd0 legacy=0x7fd160161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:16.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.466+0000 7fd15dffb640 1 -- 172.21.15.76:0/1882115426 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4220828103 0 0) 0x7fd150001490 con 0x7fd16015cfd0 2023-11-09T03:46:16.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.466+0000 7fd15f7fe640 1 -- 172.21.15.76:0/1882115426 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1600cf610 con 0x7fd1600cd570 2023-11-09T03:46:16.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.466+0000 7fd15f7fe640 1 --2- 172.21.15.76:0/1882115426 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1600cd570 0x7fd1600cd9e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd1580097a0 tx=0x7fd15802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3e0fc53804b74a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:16.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.467+0000 7fd15dffb640 1 -- 172.21.15.76:0/1882115426 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15802e070 con 0x7fd1600cd570 2023-11-09T03:46:16.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.467+0000 7fd15dffb640 1 -- 172.21.15.76:0/1882115426 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1580315a0 con 0x7fd1600cd570 2023-11-09T03:46:16.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.467+0000 7fd15dffb640 1 -- 172.21.15.76:0/1882115426 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd158031930 con 0x7fd1600cd570 2023-11-09T03:46:16.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.467+0000 7fd1653d7640 1 -- 172.21.15.76:0/1882115426 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1600cd570 msgr2=0x7fd1600cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:16.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.467+0000 7fd1653d7640 1 --2- 172.21.15.76:0/1882115426 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1600cd570 0x7fd1600cd9e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd1580097a0 tx=0x7fd15802b840 comp rx=0 tx=0).stop 2023-11-09T03:46:16.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.467+0000 7fd1653d7640 1 -- 172.21.15.76:0/1882115426 shutdown_connections 2023-11-09T03:46:16.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.467+0000 7fd1653d7640 1 --2- 172.21.15.76:0/1882115426 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1600cd570 0x7fd1600cd9e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.467+0000 7fd1653d7640 1 --2- 172.21.15.76:0/1882115426 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1600cf180 0x7fd16006c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.467+0000 7fd1653d7640 1 -- 172.21.15.76:0/1882115426 >> 172.21.15.76:0/1882115426 conn(0x7fd16006ea20 msgr2=0x7fd16006ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:16.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.467+0000 7fd1653d7640 1 -- 172.21.15.76:0/1882115426 shutdown_connections 2023-11-09T03:46:16.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.467+0000 7fd1653d7640 1 -- 172.21.15.76:0/1882115426 wait complete. 2023-11-09T03:46:16.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.468+0000 7fd1653d7640 1 Processor -- start 2023-11-09T03:46:16.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.468+0000 7fd1653d7640 1 -- start start 2023-11-09T03:46:16.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.469+0000 7fd1653d7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1600cd570 0x7fd1600ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:16.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.469+0000 7fd1653d7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd16015cfd0 0x7fd1600c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:16.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.469+0000 7fd1653d7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1600b5e20 0x7fd1600b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:16.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.469+0000 7fd1653d7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1600ca550 con 0x7fd1600cd570 2023-11-09T03:46:16.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.469+0000 7fd1653d7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1601589f0 con 0x7fd16015cfd0 2023-11-09T03:46:16.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.469+0000 7fd1653d7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1600c6fa0 con 0x7fd1600b5e20 2023-11-09T03:46:16.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.469+0000 7fd15effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1600cd570 0x7fd1600ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:16.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.469+0000 7fd15effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1600cd570 0x7fd1600ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36776/0 (socket says 172.21.15.76:36776) 2023-11-09T03:46:16.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.469+0000 7fd15effd640 1 -- 172.21.15.76:0/2871916407 learned_addr learned my addr 172.21.15.76:0/2871916407 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:16.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.469+0000 7fd15f7fe640 1 --2- 172.21.15.76:0/2871916407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1600b5e20 0x7fd1600b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:16.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.470+0000 7fd15effd640 1 -- 172.21.15.76:0/2871916407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1600b5e20 msgr2=0x7fd1600b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:16.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.470+0000 7fd15e7fc640 1 --2- 172.21.15.76:0/2871916407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd16015cfd0 0x7fd1600c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:16.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.470+0000 7fd15effd640 1 --2- 172.21.15.76:0/2871916407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1600b5e20 0x7fd1600b2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.470+0000 7fd15effd640 1 -- 172.21.15.76:0/2871916407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd16015cfd0 msgr2=0x7fd1600c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:16.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.470+0000 7fd15effd640 1 --2- 172.21.15.76:0/2871916407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd16015cfd0 0x7fd1600c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.470+0000 7fd15effd640 1 -- 172.21.15.76:0/2871916407 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd158009400 con 0x7fd1600cd570 2023-11-09T03:46:16.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.470+0000 7fd15f7fe640 1 --2- 172.21.15.76:0/2871916407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1600b5e20 0x7fd1600b2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:16.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.470+0000 7fd15e7fc640 1 --2- 172.21.15.76:0/2871916407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd16015cfd0 0x7fd1600c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:16.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.470+0000 7fd15effd640 1 --2- 172.21.15.76:0/2871916407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1600cd570 0x7fd1600ca970 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd15000c7f0 tx=0x7fd15000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:16.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.470+0000 7fd1477fe640 1 -- 172.21.15.76:0/2871916407 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15000a640 con 0x7fd1600cd570 2023-11-09T03:46:16.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.470+0000 7fd1477fe640 1 -- 172.21.15.76:0/2871916407 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd15000a7c0 con 0x7fd1600cd570 2023-11-09T03:46:16.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.470+0000 7fd1653d7640 1 -- 172.21.15.76:0/2871916407 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1600b3250 con 0x7fd1600cd570 2023-11-09T03:46:16.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.470+0000 7fd1477fe640 1 -- 172.21.15.76:0/2871916407 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15000ab70 con 0x7fd1600cd570 2023-11-09T03:46:16.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.470+0000 7fd1653d7640 1 -- 172.21.15.76:0/2871916407 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1600c74d0 con 0x7fd1600cd570 2023-11-09T03:46:16.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.471+0000 7fd1653d7640 1 -- 172.21.15.76:0/2871916407 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd16006b300 con 0x7fd1600cd570 2023-11-09T03:46:16.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.471+0000 7fd1477fe640 1 -- 172.21.15.76:0/2871916407 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd15000acd0 con 0x7fd1600cd570 2023-11-09T03:46:16.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.472+0000 7fd1477fe640 1 --2- 172.21.15.76:0/2871916407 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd14c06aea0 0x7fd14c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:16.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.472+0000 7fd1477fe640 1 -- 172.21.15.76:0/2871916407 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fd150088fb0 con 0x7fd1600cd570 2023-11-09T03:46:16.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.472+0000 7fd15e7fc640 1 --2- 172.21.15.76:0/2871916407 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd14c06aea0 0x7fd14c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:16.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.472+0000 7fd15e7fc640 1 --2- 172.21.15.76:0/2871916407 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd14c06aea0 0x7fd14c06d360 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd154006fd0 tx=0x7fd154008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:16.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.476+0000 7fd1477fe640 1 -- 172.21.15.76:0/2871916407 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd1500bbca0 con 0x7fd1600cd570 2023-11-09T03:46:16.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.642+0000 7fd1653d7640 1 -- 172.21.15.76:0/2871916407 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd1600cdf20 con 0x7fd1600cd570 2023-11-09T03:46:16.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.642+0000 7fd1477fe640 1 -- 172.21.15.76:0/2871916407 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd1600cdf20 con 0x7fd1600cd570 2023-11-09T03:46:16.641 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:46:16.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.644+0000 7fd1653d7640 1 -- 172.21.15.76:0/2871916407 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd14c06aea0 msgr2=0x7fd14c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:16.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.644+0000 7fd1653d7640 1 --2- 172.21.15.76:0/2871916407 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd14c06aea0 0x7fd14c06d360 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd154006fd0 tx=0x7fd154008040 comp rx=0 tx=0).stop 2023-11-09T03:46:16.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.644+0000 7fd1653d7640 1 -- 172.21.15.76:0/2871916407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1600cd570 msgr2=0x7fd1600ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:16.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.644+0000 7fd1653d7640 1 --2- 172.21.15.76:0/2871916407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1600cd570 0x7fd1600ca970 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd15000c7f0 tx=0x7fd15000ccb0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.644+0000 7fd1653d7640 1 -- 172.21.15.76:0/2871916407 shutdown_connections 2023-11-09T03:46:16.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.644+0000 7fd1653d7640 1 --2- 172.21.15.76:0/2871916407 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd14c06aea0 0x7fd14c06d360 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.644+0000 7fd1653d7640 1 --2- 172.21.15.76:0/2871916407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1600b5e20 0x7fd1600b2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.644+0000 7fd1653d7640 1 --2- 172.21.15.76:0/2871916407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd16015cfd0 0x7fd1600c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.644+0000 7fd1653d7640 1 --2- 172.21.15.76:0/2871916407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1600cd570 0x7fd1600ca970 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.644+0000 7fd1653d7640 1 -- 172.21.15.76:0/2871916407 >> 172.21.15.76:0/2871916407 conn(0x7fd16006ea20 msgr2=0x7fd16015c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:16.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.644+0000 7fd1653d7640 1 -- 172.21.15.76:0/2871916407 shutdown_connections 2023-11-09T03:46:16.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.644+0000 7fd1653d7640 1 -- 172.21.15.76:0/2871916407 wait complete. 2023-11-09T03:46:16.654 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:46:16.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.755+0000 7fb587183640 1 Processor -- start 2023-11-09T03:46:16.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.755+0000 7fb587183640 1 -- start start 2023-11-09T03:46:16.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.755+0000 7fb587183640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5800b0ae0 0x7fb5800ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:16.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.755+0000 7fb587183640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb58006db20 0x7fb58015d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:16.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.755+0000 7fb587183640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5800b4110 con 0x7fb580159c70 2023-11-09T03:46:16.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.755+0000 7fb587183640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5800b49b0 con 0x7fb58006db20 2023-11-09T03:46:16.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.755+0000 7fb587183640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5800b43f0 con 0x7fb5800b0ae0 2023-11-09T03:46:16.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.755+0000 7fb585720640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb58006db20 0x7fb58015d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:16.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.755+0000 7fb584f1f640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb580159c70 0x7fb5800b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43650/0 (socket says 172.21.15.76:43650) 2023-11-09T03:46:16.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.755+0000 7fb584f1f640 1 -- 172.21.15.76:0/3891940746 learned_addr learned my addr 172.21.15.76:0/3891940746 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:16.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.756+0000 7fb585720640 1 -- 172.21.15.76:0/3891940746 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5800b0ae0 msgr2=0x7fb5800ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:16.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.756+0000 7fb585720640 1 --2- 172.21.15.76:0/3891940746 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5800b0ae0 0x7fb5800ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.756+0000 7fb585720640 1 -- 172.21.15.76:0/3891940746 >> v1:172.21.15.76:6789/0 conn(0x7fb580159c70 legacy=0x7fb5800b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:16.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.756+0000 7fb585720640 1 -- 172.21.15.76:0/3891940746 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5800b46d0 con 0x7fb58006db20 2023-11-09T03:46:16.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.756+0000 7fb57f7fe640 1 -- 172.21.15.76:0/3891940746 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1782455186 0 0) 0x7fb5800b4110 con 0x7fb580159c70 2023-11-09T03:46:16.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.756+0000 7fb585720640 1 --2- 172.21.15.76:0/3891940746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb58006db20 0x7fb58015d7d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb5680097a0 tx=0x7fb56802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a2460a8cc0455c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:16.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.757+0000 7fb57f7fe640 1 -- 172.21.15.76:0/3891940746 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb568031830 con 0x7fb58006db20 2023-11-09T03:46:16.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.757+0000 7fb57f7fe640 1 -- 172.21.15.76:0/3891940746 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb568031990 con 0x7fb58006db20 2023-11-09T03:46:16.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.757+0000 7fb57f7fe640 1 -- 172.21.15.76:0/3891940746 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb568031d20 con 0x7fb58006db20 2023-11-09T03:46:16.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.757+0000 7fb587183640 1 -- 172.21.15.76:0/3891940746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb58006db20 msgr2=0x7fb58015d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:16.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.757+0000 7fb587183640 1 --2- 172.21.15.76:0/3891940746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb58006db20 0x7fb58015d7d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb5680097a0 tx=0x7fb56802b8d0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.757+0000 7fb587183640 1 -- 172.21.15.76:0/3891940746 shutdown_connections 2023-11-09T03:46:16.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.757+0000 7fb587183640 1 --2- 172.21.15.76:0/3891940746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb58006db20 0x7fb58015d7d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.757+0000 7fb587183640 1 --2- 172.21.15.76:0/3891940746 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5800b0ae0 0x7fb5800ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.757+0000 7fb587183640 1 -- 172.21.15.76:0/3891940746 >> 172.21.15.76:0/3891940746 conn(0x7fb5800ce950 msgr2=0x7fb5800ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:16.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.757+0000 7fb587183640 1 -- 172.21.15.76:0/3891940746 shutdown_connections 2023-11-09T03:46:16.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.757+0000 7fb587183640 1 -- 172.21.15.76:0/3891940746 wait complete. 2023-11-09T03:46:16.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.758+0000 7fb587183640 1 Processor -- start 2023-11-09T03:46:16.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.758+0000 7fb587183640 1 -- start start 2023-11-09T03:46:16.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.759+0000 7fb587183640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb58006db20 0x7fb580153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:16.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.759+0000 7fb587183640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5800b0ae0 0x7fb58015c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:16.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.759+0000 7fb587183640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb580159c70 0x7fb58015cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:16.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.759+0000 7fb587183640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58015aa90 con 0x7fb580159c70 2023-11-09T03:46:16.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.759+0000 7fb587183640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5801f3eb0 con 0x7fb58006db20 2023-11-09T03:46:16.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.759+0000 7fb587183640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb58015a160 con 0x7fb5800b0ae0 2023-11-09T03:46:16.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.759+0000 7fb585720640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb580159c70 0x7fb58015cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:16.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.759+0000 7fb585720640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb580159c70 0x7fb58015cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36792/0 (socket says 172.21.15.76:36792) 2023-11-09T03:46:16.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.759+0000 7fb585720640 1 -- 172.21.15.76:0/2171595915 learned_addr learned my addr 172.21.15.76:0/2171595915 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:16.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.759+0000 7fb584f1f640 1 --2- 172.21.15.76:0/2171595915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb58006db20 0x7fb580153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:16.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.759+0000 7fb57ffff640 1 --2- 172.21.15.76:0/2171595915 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5800b0ae0 0x7fb58015c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:16.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.760+0000 7fb585720640 1 -- 172.21.15.76:0/2171595915 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5800b0ae0 msgr2=0x7fb58015c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:16.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.760+0000 7fb585720640 1 --2- 172.21.15.76:0/2171595915 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5800b0ae0 0x7fb58015c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.760+0000 7fb585720640 1 -- 172.21.15.76:0/2171595915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb58006db20 msgr2=0x7fb580153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:16.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.760+0000 7fb585720640 1 --2- 172.21.15.76:0/2171595915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb58006db20 0x7fb580153030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.760+0000 7fb585720640 1 -- 172.21.15.76:0/2171595915 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb568009400 con 0x7fb580159c70 2023-11-09T03:46:16.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.760+0000 7fb585720640 1 --2- 172.21.15.76:0/2171595915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb580159c70 0x7fb58015cbd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb5680098c0 tx=0x7fb568004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:16.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.760+0000 7fb57d7fa640 1 -- 172.21.15.76:0/2171595915 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb568031e00 con 0x7fb580159c70 2023-11-09T03:46:16.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.760+0000 7fb57d7fa640 1 -- 172.21.15.76:0/2171595915 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb56803bb80 con 0x7fb580159c70 2023-11-09T03:46:16.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.760+0000 7fb587183640 1 -- 172.21.15.76:0/2171595915 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb58015a440 con 0x7fb580159c70 2023-11-09T03:46:16.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.760+0000 7fb57d7fa640 1 -- 172.21.15.76:0/2171595915 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb568045410 con 0x7fb580159c70 2023-11-09T03:46:16.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.760+0000 7fb587183640 1 -- 172.21.15.76:0/2171595915 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb580153570 con 0x7fb580159c70 2023-11-09T03:46:16.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.761+0000 7fb587183640 1 -- 172.21.15.76:0/2171595915 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb58015ac40 con 0x7fb580159c70 2023-11-09T03:46:16.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.762+0000 7fb57d7fa640 1 -- 172.21.15.76:0/2171595915 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb56804e3b0 con 0x7fb580159c70 2023-11-09T03:46:16.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.762+0000 7fb57d7fa640 1 --2- 172.21.15.76:0/2171595915 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb56006aea0 0x7fb56006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:16.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.762+0000 7fb57d7fa640 1 -- 172.21.15.76:0/2171595915 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fb5680adc20 con 0x7fb580159c70 2023-11-09T03:46:16.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.763+0000 7fb584f1f640 1 --2- 172.21.15.76:0/2171595915 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb56006aea0 0x7fb56006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:16.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.763+0000 7fb584f1f640 1 --2- 172.21.15.76:0/2171595915 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb56006aea0 0x7fb56006d360 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb5780012e0 tx=0x7fb578007ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:16.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.766+0000 7fb57d7fa640 1 -- 172.21.15.76:0/2171595915 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb5680789d0 con 0x7fb580159c70 2023-11-09T03:46:16.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.933+0000 7fb587183640 1 -- 172.21.15.76:0/2171595915 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb58015ac40 con 0x7fb580159c70 2023-11-09T03:46:16.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.935+0000 7fb57d7fa640 1 -- 172.21.15.76:0/2171595915 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18710 (secure 0 0 0) 0x7fb58015ac40 con 0x7fb580159c70 2023-11-09T03:46:16.934 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:16.935 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":27,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:46:12.537265+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:46:16.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.937+0000 7fb587183640 1 -- 172.21.15.76:0/2171595915 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb56006aea0 msgr2=0x7fb56006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:16.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.937+0000 7fb587183640 1 --2- 172.21.15.76:0/2171595915 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb56006aea0 0x7fb56006d360 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb5780012e0 tx=0x7fb578007ed0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.937+0000 7fb587183640 1 -- 172.21.15.76:0/2171595915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb580159c70 msgr2=0x7fb58015cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:16.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.937+0000 7fb587183640 1 --2- 172.21.15.76:0/2171595915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb580159c70 0x7fb58015cbd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb5680098c0 tx=0x7fb568004730 comp rx=0 tx=0).stop 2023-11-09T03:46:16.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.937+0000 7fb587183640 1 -- 172.21.15.76:0/2171595915 shutdown_connections 2023-11-09T03:46:16.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.937+0000 7fb587183640 1 --2- 172.21.15.76:0/2171595915 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb56006aea0 0x7fb56006d360 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.937+0000 7fb587183640 1 --2- 172.21.15.76:0/2171595915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb580159c70 0x7fb58015cbd0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.937+0000 7fb587183640 1 --2- 172.21.15.76:0/2171595915 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5800b0ae0 0x7fb58015c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.937+0000 7fb587183640 1 --2- 172.21.15.76:0/2171595915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb58006db20 0x7fb580153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:16.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.937+0000 7fb587183640 1 -- 172.21.15.76:0/2171595915 >> 172.21.15.76:0/2171595915 conn(0x7fb5800ce950 msgr2=0x7fb5800b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:16.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.937+0000 7fb587183640 1 -- 172.21.15.76:0/2171595915 shutdown_connections 2023-11-09T03:46:16.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:16.937+0000 7fb587183640 1 -- 172.21.15.76:0/2171595915 wait complete. 2023-11-09T03:46:16.947 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:45:28.704715+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:45:29.717614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-09T03:45:33.440577+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-09T03:46:16.948 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:46:17.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.051+0000 7fe6bb5fc640 1 Processor -- start 2023-11-09T03:46:17.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.051+0000 7fe6bb5fc640 1 -- start start 2023-11-09T03:46:17.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.052+0000 7fe6bb5fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe6b41609e0 0x7fe6b40cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:17.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.052+0000 7fe6bb5fc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b406e1a0 con 0x7fe6b40ccde0 2023-11-09T03:46:17.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.052+0000 7fe6bb5fc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b40cd330 con 0x7fe6b4160600 2023-11-09T03:46:17.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.052+0000 7fe6bb5fc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b406db50 con 0x7fe6b41609e0 2023-11-09T03:46:17.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.052+0000 7fe6b8b97640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe6b41609e0 0x7fe6b40cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:17.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.052+0000 7fe6b9b99640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe6b40ccde0 0x7fe6b406e630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43656/0 (socket says 172.21.15.76:43656) 2023-11-09T03:46:17.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.052+0000 7fe6b9b99640 1 -- 172.21.15.76:0/4003676608 learned_addr learned my addr 172.21.15.76:0/4003676608 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:17.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.053+0000 7fe6b8b97640 1 -- 172.21.15.76:0/4003676608 >> v1:172.21.15.113:6789/0 conn(0x7fe6b4160600 legacy=0x7fe6b40cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:17.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.053+0000 7fe6b8b97640 1 -- 172.21.15.76:0/4003676608 >> v1:172.21.15.76:6789/0 conn(0x7fe6b40ccde0 legacy=0x7fe6b406e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:17.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.053+0000 7fe6abfff640 1 -- 172.21.15.76:0/4003676608 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3418753116 0 0) 0x7fe6b406e1a0 con 0x7fe6b40ccde0 2023-11-09T03:46:17.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.053+0000 7fe6b8b97640 1 -- 172.21.15.76:0/4003676608 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b406de30 con 0x7fe6b41609e0 2023-11-09T03:46:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.053+0000 7fe6b8b97640 1 --2- 172.21.15.76:0/4003676608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6b41609e0 0x7fe6b40cdd20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe6a40097a0 tx=0x7fe6a402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6fc57a7e5cf070a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:17.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.053+0000 7fe6abfff640 1 -- 172.21.15.76:0/4003676608 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4030610 con 0x7fe6b41609e0 2023-11-09T03:46:17.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.053+0000 7fe6abfff640 1 -- 172.21.15.76:0/4003676608 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a4030770 con 0x7fe6b41609e0 2023-11-09T03:46:17.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.053+0000 7fe6abfff640 1 -- 172.21.15.76:0/4003676608 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4030b00 con 0x7fe6b41609e0 2023-11-09T03:46:17.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.054+0000 7fe6bb5fc640 1 -- 172.21.15.76:0/4003676608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6b41609e0 msgr2=0x7fe6b40cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:17.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.054+0000 7fe6bb5fc640 1 --2- 172.21.15.76:0/4003676608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6b41609e0 0x7fe6b40cdd20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe6a40097a0 tx=0x7fe6a402b960 comp rx=0 tx=0).stop 2023-11-09T03:46:17.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.054+0000 7fe6bb5fc640 1 -- 172.21.15.76:0/4003676608 shutdown_connections 2023-11-09T03:46:17.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.054+0000 7fe6bb5fc640 1 --2- 172.21.15.76:0/4003676608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6b41609e0 0x7fe6b40cdd20 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.054+0000 7fe6bb5fc640 1 -- 172.21.15.76:0/4003676608 >> 172.21.15.76:0/4003676608 conn(0x7fe6b406a120 msgr2=0x7fe6b406a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:17.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.054+0000 7fe6bb5fc640 1 -- 172.21.15.76:0/4003676608 shutdown_connections 2023-11-09T03:46:17.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.054+0000 7fe6bb5fc640 1 -- 172.21.15.76:0/4003676608 wait complete. 2023-11-09T03:46:17.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.055+0000 7fe6bb5fc640 1 Processor -- start 2023-11-09T03:46:17.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.055+0000 7fe6bb5fc640 1 -- start start 2023-11-09T03:46:17.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.056+0000 7fe6bb5fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6b40ccde0 0x7fe6b40c5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:17.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.056+0000 7fe6bb5fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6b4160600 0x7fe6b414e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:17.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.056+0000 7fe6bb5fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6b41609e0 0x7fe6b40c2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:17.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.056+0000 7fe6bb5fc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b40abd30 con 0x7fe6b40ccde0 2023-11-09T03:46:17.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.056+0000 7fe6bb5fc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b40ac010 con 0x7fe6b41609e0 2023-11-09T03:46:17.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.056+0000 7fe6bb5fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b40ac2f0 con 0x7fe6b4160600 2023-11-09T03:46:17.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.056+0000 7fe6b9398640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6b40ccde0 0x7fe6b40c5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:17.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.056+0000 7fe6b9398640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6b40ccde0 0x7fe6b40c5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36802/0 (socket says 172.21.15.76:36802) 2023-11-09T03:46:17.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.056+0000 7fe6b9398640 1 -- 172.21.15.76:0/883922675 learned_addr learned my addr 172.21.15.76:0/883922675 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:17.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.056+0000 7fe6b9398640 1 -- 172.21.15.76:0/883922675 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6b4160600 msgr2=0x7fe6b414e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:17.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.056+0000 7fe6b9398640 1 --2- 172.21.15.76:0/883922675 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6b4160600 0x7fe6b414e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.056+0000 7fe6b9398640 1 -- 172.21.15.76:0/883922675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6b41609e0 msgr2=0x7fe6b40c2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:17.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.056+0000 7fe6b9398640 1 --2- 172.21.15.76:0/883922675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6b41609e0 0x7fe6b40c2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.056+0000 7fe6b9398640 1 -- 172.21.15.76:0/883922675 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a4009400 con 0x7fe6b40ccde0 2023-11-09T03:46:17.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.057+0000 7fe6b9398640 1 --2- 172.21.15.76:0/883922675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6b40ccde0 0x7fe6b40c5e00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe6b000c7a0 tx=0x7fe6b000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:17.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.057+0000 7fe6a9ffb640 1 -- 172.21.15.76:0/883922675 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0003d80 con 0x7fe6b40ccde0 2023-11-09T03:46:17.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.057+0000 7fe6a9ffb640 1 -- 172.21.15.76:0/883922675 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b0005070 con 0x7fe6b40ccde0 2023-11-09T03:46:17.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.057+0000 7fe6bb5fc640 1 -- 172.21.15.76:0/883922675 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b40ac5d0 con 0x7fe6b40ccde0 2023-11-09T03:46:17.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.057+0000 7fe6a9ffb640 1 -- 172.21.15.76:0/883922675 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0005460 con 0x7fe6b40ccde0 2023-11-09T03:46:17.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.057+0000 7fe6bb5fc640 1 -- 172.21.15.76:0/883922675 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b414e800 con 0x7fe6b40ccde0 2023-11-09T03:46:17.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.058+0000 7fe6bb5fc640 1 -- 172.21.15.76:0/883922675 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b40acaf0 con 0x7fe6b40ccde0 2023-11-09T03:46:17.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.058+0000 7fe6a9ffb640 1 -- 172.21.15.76:0/883922675 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe6b00055c0 con 0x7fe6b40ccde0 2023-11-09T03:46:17.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.059+0000 7fe6a9ffb640 1 --2- 172.21.15.76:0/883922675 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe69806ae70 0x7fe69806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:17.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.059+0000 7fe6a9ffb640 1 -- 172.21.15.76:0/883922675 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fe6b00893f0 con 0x7fe6b40ccde0 2023-11-09T03:46:17.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.059+0000 7fe6b8b97640 1 --2- 172.21.15.76:0/883922675 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe69806ae70 0x7fe69806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:17.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.059+0000 7fe6b8b97640 1 --2- 172.21.15.76:0/883922675 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe69806ae70 0x7fe69806d330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4036000 tx=0x7fe6a4030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:17.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.063+0000 7fe6a9ffb640 1 -- 172.21.15.76:0/883922675 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe6b40acaf0 con 0x7fe6b40ccde0 2023-11-09T03:46:17.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.228+0000 7fe6bb5fc640 1 -- 172.21.15.76:0/883922675 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe6b40acaf0 con 0x7fe6b40ccde0 2023-11-09T03:46:17.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.228+0000 7fe6a9ffb640 1 -- 172.21.15.76:0/883922675 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe6b40acaf0 con 0x7fe6b40ccde0 2023-11-09T03:46:17.228 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:46:17.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.230+0000 7fe6bb5fc640 1 -- 172.21.15.76:0/883922675 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe69806ae70 msgr2=0x7fe69806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:17.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.230+0000 7fe6bb5fc640 1 --2- 172.21.15.76:0/883922675 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe69806ae70 0x7fe69806d330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4036000 tx=0x7fe6a4030000 comp rx=0 tx=0).stop 2023-11-09T03:46:17.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.230+0000 7fe6bb5fc640 1 -- 172.21.15.76:0/883922675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6b40ccde0 msgr2=0x7fe6b40c5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:17.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.230+0000 7fe6bb5fc640 1 --2- 172.21.15.76:0/883922675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6b40ccde0 0x7fe6b40c5e00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe6b000c7a0 tx=0x7fe6b000cc60 comp rx=0 tx=0).stop 2023-11-09T03:46:17.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.230+0000 7fe6bb5fc640 1 -- 172.21.15.76:0/883922675 shutdown_connections 2023-11-09T03:46:17.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.230+0000 7fe6bb5fc640 1 --2- 172.21.15.76:0/883922675 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe69806ae70 0x7fe69806d330 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.230+0000 7fe6bb5fc640 1 --2- 172.21.15.76:0/883922675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6b41609e0 0x7fe6b40c2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.230+0000 7fe6bb5fc640 1 --2- 172.21.15.76:0/883922675 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6b4160600 0x7fe6b414e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.230+0000 7fe6bb5fc640 1 --2- 172.21.15.76:0/883922675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6b40ccde0 0x7fe6b40c5e00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.230+0000 7fe6bb5fc640 1 -- 172.21.15.76:0/883922675 >> 172.21.15.76:0/883922675 conn(0x7fe6b406a120 msgr2=0x7fe6b40afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:17.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.230+0000 7fe6bb5fc640 1 -- 172.21.15.76:0/883922675 shutdown_connections 2023-11-09T03:46:17.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.230+0000 7fe6bb5fc640 1 -- 172.21.15.76:0/883922675 wait complete. 2023-11-09T03:46:17.240 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:46:17.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.343+0000 7fdf0942d640 1 Processor -- start 2023-11-09T03:46:17.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.343+0000 7fdf0942d640 1 -- start start 2023-11-09T03:46:17.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.344+0000 7fdf0942d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdf040b4fb0 0x7fdf040cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:17.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.344+0000 7fdf0942d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf04160df0 con 0x7fdf0415ccc0 2023-11-09T03:46:17.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.344+0000 7fdf0942d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf040ce260 con 0x7fdf0415d0a0 2023-11-09T03:46:17.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.344+0000 7fdf0942d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf040ce540 con 0x7fdf040b4fb0 2023-11-09T03:46:17.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.344+0000 7fdf027fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdf040b4fb0 0x7fdf040cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:17.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.344+0000 7fdf02ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fdf0415ccc0 0x7fdf040b48a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43660/0 (socket says 172.21.15.76:43660) 2023-11-09T03:46:17.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.344+0000 7fdf02ffd640 1 -- 172.21.15.76:0/1789700316 learned_addr learned my addr 172.21.15.76:0/1789700316 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:17.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.345+0000 7fdf027fc640 1 -- 172.21.15.76:0/1789700316 >> v1:172.21.15.113:6789/0 conn(0x7fdf0415d0a0 legacy=0x7fdf040d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:17.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.345+0000 7fdf027fc640 1 -- 172.21.15.76:0/1789700316 >> v1:172.21.15.76:6789/0 conn(0x7fdf0415ccc0 legacy=0x7fdf040b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:17.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.345+0000 7fdf027fc640 1 -- 172.21.15.76:0/1789700316 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf040d1050 con 0x7fdf040b4fb0 2023-11-09T03:46:17.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.345+0000 7fdf01ffb640 1 -- 172.21.15.76:0/1789700316 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3641615196 0 0) 0x7fdef4001490 con 0x7fdf0415ccc0 2023-11-09T03:46:17.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.345+0000 7fdf027fc640 1 --2- 172.21.15.76:0/1789700316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf040b4fb0 0x7fdf040cd680 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdef80097a0 tx=0x7fdef802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7719a4ae0cb066b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:17.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.345+0000 7fdf01ffb640 1 -- 172.21.15.76:0/1789700316 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef8031830 con 0x7fdf040b4fb0 2023-11-09T03:46:17.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.346+0000 7fdf01ffb640 1 -- 172.21.15.76:0/1789700316 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef8031990 con 0x7fdf040b4fb0 2023-11-09T03:46:17.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.346+0000 7fdf01ffb640 1 -- 172.21.15.76:0/1789700316 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef8031d20 con 0x7fdf040b4fb0 2023-11-09T03:46:17.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.346+0000 7fdf0942d640 1 -- 172.21.15.76:0/1789700316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf040b4fb0 msgr2=0x7fdf040cd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:17.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.346+0000 7fdf0942d640 1 --2- 172.21.15.76:0/1789700316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf040b4fb0 0x7fdf040cd680 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdef80097a0 tx=0x7fdef802b8d0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.346+0000 7fdf0942d640 1 -- 172.21.15.76:0/1789700316 shutdown_connections 2023-11-09T03:46:17.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.346+0000 7fdf0942d640 1 --2- 172.21.15.76:0/1789700316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf040b4fb0 0x7fdf040cd680 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.346+0000 7fdf0942d640 1 -- 172.21.15.76:0/1789700316 >> 172.21.15.76:0/1789700316 conn(0x7fdf040cf8a0 msgr2=0x7fdf040cfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:17.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.346+0000 7fdf0942d640 1 -- 172.21.15.76:0/1789700316 shutdown_connections 2023-11-09T03:46:17.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.346+0000 7fdf0942d640 1 -- 172.21.15.76:0/1789700316 wait complete. 2023-11-09T03:46:17.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.347+0000 7fdf0942d640 1 Processor -- start 2023-11-09T03:46:17.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.347+0000 7fdf0942d640 1 -- start start 2023-11-09T03:46:17.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.348+0000 7fdf0942d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf040b4fb0 0x7fdf04147190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:17.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.348+0000 7fdf0942d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf0415ccc0 0x7fdf04156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:17.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.348+0000 7fdf0942d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf0415d0a0 0x7fdf04156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:17.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.348+0000 7fdf0942d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf041476d0 con 0x7fdf0415d0a0 2023-11-09T03:46:17.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.348+0000 7fdf0942d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf040b85f0 con 0x7fdf040b4fb0 2023-11-09T03:46:17.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.348+0000 7fdf0942d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf040c6950 con 0x7fdf0415ccc0 2023-11-09T03:46:17.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.348+0000 7fdf037fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf0415d0a0 0x7fdf04156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:17.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.348+0000 7fdf037fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf0415d0a0 0x7fdf04156780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36816/0 (socket says 172.21.15.76:36816) 2023-11-09T03:46:17.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.348+0000 7fdf037fe640 1 -- 172.21.15.76:0/2942269179 learned_addr learned my addr 172.21.15.76:0/2942269179 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:17.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.348+0000 7fdf027fc640 1 --2- 172.21.15.76:0/2942269179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf0415ccc0 0x7fdf04156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:17.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.348+0000 7fdf02ffd640 1 --2- 172.21.15.76:0/2942269179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf040b4fb0 0x7fdf04147190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:17.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.349+0000 7fdf037fe640 1 -- 172.21.15.76:0/2942269179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf0415ccc0 msgr2=0x7fdf04156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:17.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.349+0000 7fdf037fe640 1 --2- 172.21.15.76:0/2942269179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf0415ccc0 0x7fdf04156240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.349+0000 7fdf037fe640 1 -- 172.21.15.76:0/2942269179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf040b4fb0 msgr2=0x7fdf04147190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:17.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.349+0000 7fdf037fe640 1 --2- 172.21.15.76:0/2942269179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf040b4fb0 0x7fdf04147190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.349+0000 7fdf037fe640 1 -- 172.21.15.76:0/2942269179 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef8009400 con 0x7fdf0415d0a0 2023-11-09T03:46:17.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.349+0000 7fdf02ffd640 1 --2- 172.21.15.76:0/2942269179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf040b4fb0 0x7fdf04147190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:17.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.349+0000 7fdf027fc640 1 --2- 172.21.15.76:0/2942269179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf0415ccc0 0x7fdf04156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:17.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.349+0000 7fdf037fe640 1 --2- 172.21.15.76:0/2942269179 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf0415d0a0 0x7fdf04156780 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00b410 tx=0x7fdefc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:17.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.349+0000 7fdeeb7fe640 1 -- 172.21.15.76:0/2942269179 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc004020 con 0x7fdf0415d0a0 2023-11-09T03:46:17.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.349+0000 7fdeeb7fe640 1 -- 172.21.15.76:0/2942269179 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdefc004180 con 0x7fdf0415d0a0 2023-11-09T03:46:17.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.349+0000 7fdeeb7fe640 1 -- 172.21.15.76:0/2942269179 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc011630 con 0x7fdf0415d0a0 2023-11-09T03:46:17.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.349+0000 7fdf0942d640 1 -- 172.21.15.76:0/2942269179 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf040c6c30 con 0x7fdf0415d0a0 2023-11-09T03:46:17.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.349+0000 7fdf0942d640 1 -- 172.21.15.76:0/2942269179 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf040c71f0 con 0x7fdf0415d0a0 2023-11-09T03:46:17.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.350+0000 7fdf0942d640 1 -- 172.21.15.76:0/2942269179 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf040c73e0 con 0x7fdf0415d0a0 2023-11-09T03:46:17.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.350+0000 7fdeeb7fe640 1 -- 172.21.15.76:0/2942269179 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdefc011790 con 0x7fdf0415d0a0 2023-11-09T03:46:17.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.351+0000 7fdeeb7fe640 1 --2- 172.21.15.76:0/2942269179 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdef006ae70 0x7fdef006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:17.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.351+0000 7fdeeb7fe640 1 -- 172.21.15.76:0/2942269179 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fdefc089670 con 0x7fdf0415d0a0 2023-11-09T03:46:17.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.351+0000 7fdf02ffd640 1 --2- 172.21.15.76:0/2942269179 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdef006ae70 0x7fdef006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:17.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.351+0000 7fdf02ffd640 1 --2- 172.21.15.76:0/2942269179 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdef006ae70 0x7fdef006d330 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdef40012e0 tx=0x7fdef4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:17.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.355+0000 7fdeeb7fe640 1 -- 172.21.15.76:0/2942269179 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdefc0543d0 con 0x7fdf0415d0a0 2023-11-09T03:46:17.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.521+0000 7fdf0942d640 1 -- 172.21.15.76:0/2942269179 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdf040d0b60 con 0x7fdf0415d0a0 2023-11-09T03:46:17.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.521+0000 7fdeeb7fe640 1 -- 172.21.15.76:0/2942269179 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdefc0579f0 con 0x7fdf0415d0a0 2023-11-09T03:46:17.521 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:46:17.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.523+0000 7fdf0942d640 1 -- 172.21.15.76:0/2942269179 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdef006ae70 msgr2=0x7fdef006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:17.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.523+0000 7fdf0942d640 1 --2- 172.21.15.76:0/2942269179 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdef006ae70 0x7fdef006d330 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdef40012e0 tx=0x7fdef4009040 comp rx=0 tx=0).stop 2023-11-09T03:46:17.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.523+0000 7fdf0942d640 1 -- 172.21.15.76:0/2942269179 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf0415d0a0 msgr2=0x7fdf04156780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:17.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.523+0000 7fdf0942d640 1 --2- 172.21.15.76:0/2942269179 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf0415d0a0 0x7fdf04156780 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00b410 tx=0x7fdefc00b8d0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.523+0000 7fdf0942d640 1 -- 172.21.15.76:0/2942269179 shutdown_connections 2023-11-09T03:46:17.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.523+0000 7fdf0942d640 1 --2- 172.21.15.76:0/2942269179 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdef006ae70 0x7fdef006d330 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.523+0000 7fdf0942d640 1 --2- 172.21.15.76:0/2942269179 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf0415d0a0 0x7fdf04156780 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.523+0000 7fdf0942d640 1 --2- 172.21.15.76:0/2942269179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf0415ccc0 0x7fdf04156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.523+0000 7fdf0942d640 1 --2- 172.21.15.76:0/2942269179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf040b4fb0 0x7fdf04147190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.523+0000 7fdf0942d640 1 -- 172.21.15.76:0/2942269179 >> 172.21.15.76:0/2942269179 conn(0x7fdf040cf8a0 msgr2=0x7fdf040d23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:17.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.523+0000 7fdf0942d640 1 -- 172.21.15.76:0/2942269179 shutdown_connections 2023-11-09T03:46:17.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.523+0000 7fdf0942d640 1 -- 172.21.15.76:0/2942269179 wait complete. 2023-11-09T03:46:17.533 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:46:17.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.638+0000 7fb75ec90640 1 Processor -- start 2023-11-09T03:46:17.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.638+0000 7fb75ec90640 1 -- start start 2023-11-09T03:46:17.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.639+0000 7fb75ec90640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb758070e20 0x7fb7580adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:17.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.639+0000 7fb75ec90640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7580c3900 con 0x7fb7580ad730 2023-11-09T03:46:17.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.639+0000 7fb75ec90640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7580c3be0 con 0x7fb75815ad30 2023-11-09T03:46:17.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.639+0000 7fb75ec90640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7580ae260 con 0x7fb758070e20 2023-11-09T03:46:17.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.639+0000 7fb75d22d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb7580ad730 0x7fb75815e740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43664/0 (socket says 172.21.15.76:43664) 2023-11-09T03:46:17.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.639+0000 7fb75d22d640 1 -- 172.21.15.76:0/1467864665 learned_addr learned my addr 172.21.15.76:0/1467864665 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:17.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.639+0000 7fb757fff640 1 --2- 172.21.15.76:0/1467864665 >> v2:172.21.15.113:3300/0 conn(0x7fb758070e20 0x7fb7580adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:17.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.639+0000 7fb7577fe640 1 -- 172.21.15.76:0/1467864665 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1422616450 0 0) 0x7fb7580c3900 con 0x7fb7580ad730 2023-11-09T03:46:17.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.639+0000 7fb7577fe640 1 -- 172.21.15.76:0/1467864665 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb744003520 con 0x7fb7580ad730 2023-11-09T03:46:17.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.640+0000 7fb7577fe640 1 -- 172.21.15.76:0/1467864665 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3335209824 0 0) 0x7fb744003520 con 0x7fb7580ad730 2023-11-09T03:46:17.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.640+0000 7fb7577fe640 1 -- 172.21.15.76:0/1467864665 >> v2:172.21.15.113:3300/0 conn(0x7fb758070e20 msgr2=0x7fb7580adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:17.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.640+0000 7fb7577fe640 1 --2- 172.21.15.76:0/1467864665 >> v2:172.21.15.113:3300/0 conn(0x7fb758070e20 0x7fb7580adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.640+0000 7fb7577fe640 1 -- 172.21.15.76:0/1467864665 >> v1:172.21.15.113:6789/0 conn(0x7fb75815ad30 legacy=0x7fb758070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:17.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.640+0000 7fb7577fe640 1 -- 172.21.15.76:0/1467864665 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7580c3ec0 con 0x7fb7580ad730 2023-11-09T03:46:17.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.640+0000 7fb7577fe640 1 -- 172.21.15.76:0/1467864665 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fb748001670 con 0x7fb7580ad730 2023-11-09T03:46:17.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.640+0000 7fb7577fe640 1 -- 172.21.15.76:0/1467864665 >> v1:172.21.15.76:6789/0 conn(0x7fb7580ad730 legacy=0x7fb75815e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:17.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.640+0000 7fb7577fe640 1 --2- 172.21.15.76:0/1467864665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb744003970 0x7fb744003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:17.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.640+0000 7fb7577fe640 1 --2- 172.21.15.76:0/1467864665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb744004370 0x7fb744004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:17.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.640+0000 7fb7577fe640 1 --2- 172.21.15.76:0/1467864665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7440050e0 0x7fb74400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:17.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.640+0000 7fb7577fe640 1 -- 172.21.15.76:0/1467864665 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7580ae260 con 0x7fb7440050e0 2023-11-09T03:46:17.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.640+0000 7fb7577fe640 1 -- 172.21.15.76:0/1467864665 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74400c020 con 0x7fb744003970 2023-11-09T03:46:17.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.640+0000 7fb7577fe640 1 -- 172.21.15.76:0/1467864665 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb74400c1e0 con 0x7fb744004370 2023-11-09T03:46:17.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.640+0000 7fb75d22d640 1 --2- 172.21.15.76:0/1467864665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7440050e0 0x7fb74400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:17.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.640+0000 7fb75ca2c640 1 --2- 172.21.15.76:0/1467864665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb744003970 0x7fb744003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:17.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.640+0000 7fb757fff640 1 --2- 172.21.15.76:0/1467864665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb744004370 0x7fb744004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:17.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.640+0000 7fb75d22d640 1 -- 172.21.15.76:0/1467864665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb744004370 msgr2=0x7fb744004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:17.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.640+0000 7fb75d22d640 1 --2- 172.21.15.76:0/1467864665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb744004370 0x7fb744004840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.640+0000 7fb75d22d640 1 -- 172.21.15.76:0/1467864665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb744003970 msgr2=0x7fb744003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:17.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.640+0000 7fb75d22d640 1 --2- 172.21.15.76:0/1467864665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb744003970 0x7fb744003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.640+0000 7fb75d22d640 1 -- 172.21.15.76:0/1467864665 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb744003520 con 0x7fb7440050e0 2023-11-09T03:46:17.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.640+0000 7fb757fff640 1 --2- 172.21.15.76:0/1467864665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb744004370 0x7fb744004840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:17.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.641+0000 7fb75d22d640 1 -- 172.21.15.76:0/1467864665 reap_dead start 2023-11-09T03:46:17.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.641+0000 7fb75d22d640 1 --2- 172.21.15.76:0/1467864665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7440050e0 0x7fb74400b980 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb74800c240 tx=0x7fb7440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d119f65abd1855 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:17.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.641+0000 7fb7577fe640 1 -- 172.21.15.76:0/1467864665 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb748005ad0 con 0x7fb7440050e0 2023-11-09T03:46:17.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.641+0000 7fb7577fe640 1 -- 172.21.15.76:0/1467864665 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb748005c70 con 0x7fb7440050e0 2023-11-09T03:46:17.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.641+0000 7fb7577fe640 1 -- 172.21.15.76:0/1467864665 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74802c8a0 con 0x7fb7440050e0 2023-11-09T03:46:17.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.641+0000 7fb75ec90640 1 -- 172.21.15.76:0/1467864665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7440050e0 msgr2=0x7fb74400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:17.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.641+0000 7fb75ec90640 1 --2- 172.21.15.76:0/1467864665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7440050e0 0x7fb74400b980 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb74800c240 tx=0x7fb7440057f0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.641+0000 7fb75ec90640 1 -- 172.21.15.76:0/1467864665 shutdown_connections 2023-11-09T03:46:17.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.641+0000 7fb75ec90640 1 --2- 172.21.15.76:0/1467864665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7440050e0 0x7fb74400b980 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.641+0000 7fb75ec90640 1 -- 172.21.15.76:0/1467864665 >> 172.21.15.76:0/1467864665 conn(0x7fb7580ce950 msgr2=0x7fb7580ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:17.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.641+0000 7fb75ec90640 1 -- 172.21.15.76:0/1467864665 shutdown_connections 2023-11-09T03:46:17.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.642+0000 7fb75ec90640 1 -- 172.21.15.76:0/1467864665 wait complete. 2023-11-09T03:46:17.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.643+0000 7fb75ec90640 1 Processor -- start 2023-11-09T03:46:17.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.643+0000 7fb75ec90640 1 -- start start 2023-11-09T03:46:17.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.643+0000 7fb75ec90640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7440050e0 0x7fb758159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:17.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.643+0000 7fb75ec90640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7581517e0 0x7fb75814e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:17.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.643+0000 7fb75ec90640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb758151ec0 0x7fb75815b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:17.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.643+0000 7fb75ec90640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb758159c70 con 0x7fb7440050e0 2023-11-09T03:46:17.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.643+0000 7fb75ec90640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb758158690 con 0x7fb758151ec0 2023-11-09T03:46:17.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.643+0000 7fb75ec90640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb758158970 con 0x7fb7581517e0 2023-11-09T03:46:17.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.644+0000 7fb75ca2c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7440050e0 0x7fb758159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:17.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.644+0000 7fb75ca2c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7440050e0 0x7fb758159730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36826/0 (socket says 172.21.15.76:36826) 2023-11-09T03:46:17.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.644+0000 7fb75ca2c640 1 -- 172.21.15.76:0/1245489614 learned_addr learned my addr 172.21.15.76:0/1245489614 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:17.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.644+0000 7fb75d22d640 1 --2- 172.21.15.76:0/1245489614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb758151ec0 0x7fb75815b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:17.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.644+0000 7fb757fff640 1 --2- 172.21.15.76:0/1245489614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7581517e0 0x7fb75814e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:17.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.644+0000 7fb75ca2c640 1 -- 172.21.15.76:0/1245489614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7581517e0 msgr2=0x7fb75814e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:17.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.644+0000 7fb75ca2c640 1 --2- 172.21.15.76:0/1245489614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7581517e0 0x7fb75814e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.644+0000 7fb75ca2c640 1 -- 172.21.15.76:0/1245489614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb758151ec0 msgr2=0x7fb75815b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:17.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.644+0000 7fb75ca2c640 1 --2- 172.21.15.76:0/1245489614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb758151ec0 0x7fb75815b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.644+0000 7fb75ca2c640 1 -- 172.21.15.76:0/1245489614 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb758158c50 con 0x7fb7440050e0 2023-11-09T03:46:17.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.644+0000 7fb75d22d640 1 --2- 172.21.15.76:0/1245489614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb758151ec0 0x7fb75815b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:17.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.644+0000 7fb75ca2c640 1 --2- 172.21.15.76:0/1245489614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7440050e0 0x7fb758159730 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb75000fb60 tx=0x7fb75000ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:17.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.644+0000 7fb7557fa640 1 -- 172.21.15.76:0/1245489614 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb750003b80 con 0x7fb7440050e0 2023-11-09T03:46:17.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.644+0000 7fb7557fa640 1 -- 172.21.15.76:0/1245489614 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb750003d00 con 0x7fb7440050e0 2023-11-09T03:46:17.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.644+0000 7fb75ec90640 1 -- 172.21.15.76:0/1245489614 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb758158f30 con 0x7fb7440050e0 2023-11-09T03:46:17.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.644+0000 7fb7557fa640 1 -- 172.21.15.76:0/1245489614 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75000db20 con 0x7fb7440050e0 2023-11-09T03:46:17.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.644+0000 7fb75ec90640 1 -- 172.21.15.76:0/1245489614 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb758158320 con 0x7fb7440050e0 2023-11-09T03:46:17.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.645+0000 7fb75ec90640 1 -- 172.21.15.76:0/1245489614 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb758159450 con 0x7fb7440050e0 2023-11-09T03:46:17.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.646+0000 7fb7557fa640 1 -- 172.21.15.76:0/1245489614 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb75000dc80 con 0x7fb7440050e0 2023-11-09T03:46:17.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.646+0000 7fb7557fa640 1 --2- 172.21.15.76:0/1245489614 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb73806aea0 0x7fb73806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:17.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.646+0000 7fb7557fa640 1 -- 172.21.15.76:0/1245489614 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fb750089da0 con 0x7fb7440050e0 2023-11-09T03:46:17.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.646+0000 7fb757fff640 1 --2- 172.21.15.76:0/1245489614 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb73806aea0 0x7fb73806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:17.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.647+0000 7fb757fff640 1 --2- 172.21.15.76:0/1245489614 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb73806aea0 0x7fb73806d360 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb7580b4a30 tx=0x7fb740008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:17.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.650+0000 7fb7557fa640 1 -- 172.21.15.76:0/1245489614 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb758159450 con 0x7fb7440050e0 2023-11-09T03:46:17.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.817+0000 7fb75ec90640 1 -- 172.21.15.76:0/1245489614 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb758159450 con 0x7fb7440050e0 2023-11-09T03:46:17.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.817+0000 7fb7557fa640 1 -- 172.21.15.76:0/1245489614 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb758159450 con 0x7fb7440050e0 2023-11-09T03:46:17.817 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:46:17.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.819+0000 7fb75ec90640 1 -- 172.21.15.76:0/1245489614 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb73806aea0 msgr2=0x7fb73806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:17.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.819+0000 7fb75ec90640 1 --2- 172.21.15.76:0/1245489614 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb73806aea0 0x7fb73806d360 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb7580b4a30 tx=0x7fb740008040 comp rx=0 tx=0).stop 2023-11-09T03:46:17.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.819+0000 7fb75ec90640 1 -- 172.21.15.76:0/1245489614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7440050e0 msgr2=0x7fb758159730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:17.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.819+0000 7fb75ec90640 1 --2- 172.21.15.76:0/1245489614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7440050e0 0x7fb758159730 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb75000fb60 tx=0x7fb75000ff60 comp rx=0 tx=0).stop 2023-11-09T03:46:17.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.819+0000 7fb75ec90640 1 -- 172.21.15.76:0/1245489614 shutdown_connections 2023-11-09T03:46:17.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.819+0000 7fb75ec90640 1 --2- 172.21.15.76:0/1245489614 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb73806aea0 0x7fb73806d360 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.819+0000 7fb75ec90640 1 --2- 172.21.15.76:0/1245489614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb758151ec0 0x7fb75815b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.819+0000 7fb75ec90640 1 --2- 172.21.15.76:0/1245489614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7581517e0 0x7fb75814e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.819+0000 7fb75ec90640 1 --2- 172.21.15.76:0/1245489614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7440050e0 0x7fb758159730 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.819+0000 7fb75ec90640 1 -- 172.21.15.76:0/1245489614 >> 172.21.15.76:0/1245489614 conn(0x7fb7580ce950 msgr2=0x7fb75806cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:17.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.819+0000 7fb75ec90640 1 -- 172.21.15.76:0/1245489614 shutdown_connections 2023-11-09T03:46:17.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.819+0000 7fb75ec90640 1 -- 172.21.15.76:0/1245489614 wait complete. 2023-11-09T03:46:17.829 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-09T03:46:17.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.933+0000 7fb94032c640 1 Processor -- start 2023-11-09T03:46:17.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.933+0000 7fb94032c640 1 -- start start 2023-11-09T03:46:17.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.934+0000 7fb94032c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb93815ad40 0x7fb93806b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:17.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.934+0000 7fb94032c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb938065df0 con 0x7fb93815b120 2023-11-09T03:46:17.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.934+0000 7fb94032c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb938066040 con 0x7fb9380c9b60 2023-11-09T03:46:17.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.934+0000 7fb94032c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb93806bc40 con 0x7fb93815ad40 2023-11-09T03:46:17.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.934+0000 7fb93d8c7640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb93815b120 0x7fb93815f3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43676/0 (socket says 172.21.15.76:43676) 2023-11-09T03:46:17.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.934+0000 7fb93d8c7640 1 -- 172.21.15.76:0/2186731075 learned_addr learned my addr 172.21.15.76:0/2186731075 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:17.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.934+0000 7fb93e0c8640 1 --2- 172.21.15.76:0/2186731075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb93815ad40 0x7fb93806b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:17.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.934+0000 7fb93d0c6640 1 -- 172.21.15.76:0/2186731075 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 104892475 0 0) 0x7fb938065df0 con 0x7fb93815b120 2023-11-09T03:46:17.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.934+0000 7fb93d0c6640 1 -- 172.21.15.76:0/2186731075 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb924003520 con 0x7fb93815b120 2023-11-09T03:46:17.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.934+0000 7fb93d0c6640 1 -- 172.21.15.76:0/2186731075 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1779801644 0 0) 0x7fb924003520 con 0x7fb93815b120 2023-11-09T03:46:17.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.934+0000 7fb93d0c6640 1 -- 172.21.15.76:0/2186731075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb93815ad40 msgr2=0x7fb93806b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:17.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.934+0000 7fb93d0c6640 1 --2- 172.21.15.76:0/2186731075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb93815ad40 0x7fb93806b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.934+0000 7fb93d0c6640 1 -- 172.21.15.76:0/2186731075 >> v1:172.21.15.113:6789/0 conn(0x7fb9380c9b60 legacy=0x7fb938070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:17.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.934+0000 7fb93d0c6640 1 -- 172.21.15.76:0/2186731075 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb938066320 con 0x7fb93815b120 2023-11-09T03:46:17.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.934+0000 7fb93d0c6640 1 -- 172.21.15.76:0/2186731075 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fb928001670 con 0x7fb93815b120 2023-11-09T03:46:17.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.934+0000 7fb93d0c6640 1 -- 172.21.15.76:0/2186731075 >> v1:172.21.15.76:6789/0 conn(0x7fb93815b120 legacy=0x7fb93815f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:17.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.934+0000 7fb93d0c6640 1 --2- 172.21.15.76:0/2186731075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb93815ad40 0x7fb9240039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:17.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.934+0000 7fb93d0c6640 1 --2- 172.21.15.76:0/2186731075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb924003ee0 0x7fb9240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:17.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.935+0000 7fb93d0c6640 1 --2- 172.21.15.76:0/2186731075 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb924004d00 0x7fb924005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:17.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.935+0000 7fb93d0c6640 1 -- 172.21.15.76:0/2186731075 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93806bc40 con 0x7fb924004d00 2023-11-09T03:46:17.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.935+0000 7fb93d0c6640 1 -- 172.21.15.76:0/2186731075 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92400b9e0 con 0x7fb924003ee0 2023-11-09T03:46:17.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.935+0000 7fb93d0c6640 1 -- 172.21.15.76:0/2186731075 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92400bba0 con 0x7fb93815ad40 2023-11-09T03:46:17.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.935+0000 7fb93e8c9640 1 --2- 172.21.15.76:0/2186731075 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb924004d00 0x7fb924005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:17.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.935+0000 7fb93e0c8640 1 --2- 172.21.15.76:0/2186731075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb93815ad40 0x7fb9240039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:17.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.935+0000 7fb93d8c7640 1 --2- 172.21.15.76:0/2186731075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb924003ee0 0x7fb9240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:17.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.935+0000 7fb93e8c9640 1 -- 172.21.15.76:0/2186731075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb93815ad40 msgr2=0x7fb9240039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:17.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.935+0000 7fb93e8c9640 1 --2- 172.21.15.76:0/2186731075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb93815ad40 0x7fb9240039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.935+0000 7fb93e8c9640 1 -- 172.21.15.76:0/2186731075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb924003ee0 msgr2=0x7fb9240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:17.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.935+0000 7fb93e8c9640 1 --2- 172.21.15.76:0/2186731075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb924003ee0 0x7fb9240047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.935+0000 7fb93e8c9640 1 -- 172.21.15.76:0/2186731075 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb924003520 con 0x7fb924004d00 2023-11-09T03:46:17.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.935+0000 7fb93d8c7640 1 --2- 172.21.15.76:0/2186731075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb924003ee0 0x7fb9240047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:17.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.936+0000 7fb93e8c9640 1 --2- 172.21.15.76:0/2186731075 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb924004d00 0x7fb924005180 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb930001f30 tx=0x7fb9240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac56466df7d7f server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:17.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.936+0000 7fb93d0c6640 1 -- 172.21.15.76:0/2186731075 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93002d6c0 con 0x7fb924004d00 2023-11-09T03:46:17.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.936+0000 7fb93d0c6640 1 -- 172.21.15.76:0/2186731075 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93002d840 con 0x7fb924004d00 2023-11-09T03:46:17.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.936+0000 7fb93d0c6640 1 -- 172.21.15.76:0/2186731075 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93002dbf0 con 0x7fb924004d00 2023-11-09T03:46:17.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.936+0000 7fb94032c640 1 -- 172.21.15.76:0/2186731075 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb924004d00 msgr2=0x7fb924005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:17.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.936+0000 7fb94032c640 1 --2- 172.21.15.76:0/2186731075 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb924004d00 0x7fb924005180 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb930001f30 tx=0x7fb9240057f0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.936+0000 7fb93e8c9640 1 -- 172.21.15.76:0/2186731075 reap_dead start 2023-11-09T03:46:17.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.936+0000 7fb94032c640 1 -- 172.21.15.76:0/2186731075 shutdown_connections 2023-11-09T03:46:17.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.936+0000 7fb94032c640 1 -- 172.21.15.76:0/2186731075 >> 172.21.15.76:0/2186731075 conn(0x7fb9380cbc90 msgr2=0x7fb9380cc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:17.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.936+0000 7fb94032c640 1 -- 172.21.15.76:0/2186731075 shutdown_connections 2023-11-09T03:46:17.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.936+0000 7fb94032c640 1 -- 172.21.15.76:0/2186731075 wait complete. 2023-11-09T03:46:17.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.938+0000 7fb94032c640 1 Processor -- start 2023-11-09T03:46:17.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.938+0000 7fb94032c640 1 -- start start 2023-11-09T03:46:17.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.938+0000 7fb94032c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb924004d00 0x7fb93814b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:17.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.938+0000 7fb94032c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9380c3e60 0x7fb9380c42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:17.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.938+0000 7fb94032c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb93814e080 0x7fb93814e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:17.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.938+0000 7fb94032c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93814ea70 con 0x7fb93814e080 2023-11-09T03:46:17.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.938+0000 7fb94032c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93814f430 con 0x7fb9380c3e60 2023-11-09T03:46:17.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.938+0000 7fb94032c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9380c8740 con 0x7fb924004d00 2023-11-09T03:46:17.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.938+0000 7fb93e8c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb93814e080 0x7fb93814e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:17.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.938+0000 7fb93e8c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb93814e080 0x7fb93814e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36842/0 (socket says 172.21.15.76:36842) 2023-11-09T03:46:17.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.938+0000 7fb93e8c9640 1 -- 172.21.15.76:0/4081919443 learned_addr learned my addr 172.21.15.76:0/4081919443 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:17.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.939+0000 7fb93e0c8640 1 --2- 172.21.15.76:0/4081919443 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb924004d00 0x7fb93814b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:17.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.939+0000 7fb93d8c7640 1 --2- 172.21.15.76:0/4081919443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9380c3e60 0x7fb9380c42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:17.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.939+0000 7fb93e8c9640 1 -- 172.21.15.76:0/4081919443 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb924004d00 msgr2=0x7fb93814b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:17.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.939+0000 7fb93e8c9640 1 --2- 172.21.15.76:0/4081919443 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb924004d00 0x7fb93814b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.939+0000 7fb93e8c9640 1 -- 172.21.15.76:0/4081919443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9380c3e60 msgr2=0x7fb9380c42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:17.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.939+0000 7fb93e8c9640 1 --2- 172.21.15.76:0/4081919443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9380c3e60 0x7fb9380c42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:17.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.939+0000 7fb93e8c9640 1 -- 172.21.15.76:0/4081919443 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9380c8a20 con 0x7fb93814e080 2023-11-09T03:46:17.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.939+0000 7fb93e8c9640 1 --2- 172.21.15.76:0/4081919443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb93814e080 0x7fb93814e530 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb930002050 tx=0x7fb9300015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:17.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.939+0000 7fb92effd640 1 -- 172.21.15.76:0/4081919443 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb930003b70 con 0x7fb93814e080 2023-11-09T03:46:17.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.939+0000 7fb92effd640 1 -- 172.21.15.76:0/4081919443 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb930003cd0 con 0x7fb93814e080 2023-11-09T03:46:17.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.939+0000 7fb94032c640 1 -- 172.21.15.76:0/4081919443 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9380c8d00 con 0x7fb93814e080 2023-11-09T03:46:17.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.939+0000 7fb92effd640 1 -- 172.21.15.76:0/4081919443 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9300052b0 con 0x7fb93814e080 2023-11-09T03:46:17.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.939+0000 7fb94032c640 1 -- 172.21.15.76:0/4081919443 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9380c82d0 con 0x7fb93814e080 2023-11-09T03:46:17.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.940+0000 7fb94032c640 1 -- 172.21.15.76:0/4081919443 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9380c5ad0 con 0x7fb93814e080 2023-11-09T03:46:17.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.941+0000 7fb92effd640 1 -- 172.21.15.76:0/4081919443 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb930005410 con 0x7fb93814e080 2023-11-09T03:46:17.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.941+0000 7fb92effd640 1 --2- 172.21.15.76:0/4081919443 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb91c06aea0 0x7fb91c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:17.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.941+0000 7fb92effd640 1 -- 172.21.15.76:0/4081919443 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fb930031070 con 0x7fb93814e080 2023-11-09T03:46:17.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.941+0000 7fb93e0c8640 1 --2- 172.21.15.76:0/4081919443 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb91c06aea0 0x7fb91c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:17.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.942+0000 7fb93e0c8640 1 --2- 172.21.15.76:0/4081919443 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb91c06aea0 0x7fb91c06d360 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb934009bc0 tx=0x7fb934009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:17.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:17.945+0000 7fb92effd640 1 -- 172.21.15.76:0/4081919443 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb930076490 con 0x7fb93814e080 2023-11-09T03:46:18.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.110+0000 7fb94032c640 1 -- 172.21.15.76:0/4081919443 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb93815ebd0 con 0x7fb93814e080 2023-11-09T03:46:18.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.111+0000 7fb92effd640 1 -- 172.21.15.76:0/4081919443 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb930079ab0 con 0x7fb93814e080 2023-11-09T03:46:18.110 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:46:18.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.112+0000 7fb94032c640 1 -- 172.21.15.76:0/4081919443 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb91c06aea0 msgr2=0x7fb91c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:18.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.112+0000 7fb94032c640 1 --2- 172.21.15.76:0/4081919443 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb91c06aea0 0x7fb91c06d360 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb934009bc0 tx=0x7fb934009190 comp rx=0 tx=0).stop 2023-11-09T03:46:18.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.112+0000 7fb94032c640 1 -- 172.21.15.76:0/4081919443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb93814e080 msgr2=0x7fb93814e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:18.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.112+0000 7fb94032c640 1 --2- 172.21.15.76:0/4081919443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb93814e080 0x7fb93814e530 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb930002050 tx=0x7fb9300015b0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.113+0000 7fb94032c640 1 -- 172.21.15.76:0/4081919443 shutdown_connections 2023-11-09T03:46:18.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.113+0000 7fb94032c640 1 --2- 172.21.15.76:0/4081919443 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb91c06aea0 0x7fb91c06d360 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.113+0000 7fb94032c640 1 --2- 172.21.15.76:0/4081919443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb93814e080 0x7fb93814e530 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.113+0000 7fb94032c640 1 --2- 172.21.15.76:0/4081919443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9380c3e60 0x7fb9380c42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.113+0000 7fb94032c640 1 --2- 172.21.15.76:0/4081919443 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb924004d00 0x7fb93814b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.113+0000 7fb94032c640 1 -- 172.21.15.76:0/4081919443 >> 172.21.15.76:0/4081919443 conn(0x7fb9380cbc90 msgr2=0x7fb93815c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:18.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.113+0000 7fb94032c640 1 -- 172.21.15.76:0/4081919443 shutdown_connections 2023-11-09T03:46:18.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.113+0000 7fb94032c640 1 -- 172.21.15.76:0/4081919443 wait complete. 2023-11-09T03:46:18.123 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:46:18.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.223+0000 7f905042d640 1 Processor -- start 2023-11-09T03:46:18.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.223+0000 7f905042d640 1 -- start start 2023-11-09T03:46:18.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.224+0000 7f905042d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f90480cde60 0x7f904815d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:18.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.224+0000 7f905042d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f904815daa0 0x7f9048157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:18.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.224+0000 7f905042d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f904815e000 con 0x7f90480ce240 2023-11-09T03:46:18.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.224+0000 7f905042d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f904815e5c0 con 0x7f904815daa0 2023-11-09T03:46:18.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.224+0000 7f905042d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f904815e2e0 con 0x7f90480cde60 2023-11-09T03:46:18.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.224+0000 7f904d9c8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f904815daa0 0x7f9048157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:18.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.224+0000 7f904e9ca640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f90480ce240 0x7f904806b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43678/0 (socket says 172.21.15.76:43678) 2023-11-09T03:46:18.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.224+0000 7f904e9ca640 1 -- 172.21.15.76:0/4042676112 learned_addr learned my addr 172.21.15.76:0/4042676112 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:18.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.225+0000 7f904e1c9640 1 --2- 172.21.15.76:0/4042676112 >> v2:172.21.15.113:3300/0 conn(0x7f90480cde60 0x7f904815d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:18.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.225+0000 7f904d9c8640 1 -- 172.21.15.76:0/4042676112 >> v2:172.21.15.113:3300/0 conn(0x7f90480cde60 msgr2=0x7f904815d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:18.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.225+0000 7f904d9c8640 1 --2- 172.21.15.76:0/4042676112 >> v2:172.21.15.113:3300/0 conn(0x7f90480cde60 0x7f904815d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.225+0000 7f904d9c8640 1 -- 172.21.15.76:0/4042676112 >> v1:172.21.15.76:6789/0 conn(0x7f90480ce240 legacy=0x7f904806b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:18.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.225+0000 7f904d9c8640 1 -- 172.21.15.76:0/4042676112 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90480d1120 con 0x7f904815daa0 2023-11-09T03:46:18.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.225+0000 7f904e1c9640 1 --2- 172.21.15.76:0/4042676112 >> v2:172.21.15.113:3300/0 conn(0x7f90480cde60 0x7f904815d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:18.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.225+0000 7f904d9c8640 1 --2- 172.21.15.76:0/4042676112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f904815daa0 0x7f9048157000 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f90480b5550 tx=0x7f903802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d75e97b9a98a75 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:18.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.226+0000 7f904d1c7640 1 -- 172.21.15.76:0/4042676112 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9038031830 con 0x7f904815daa0 2023-11-09T03:46:18.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.226+0000 7f904d1c7640 1 -- 172.21.15.76:0/4042676112 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9038031990 con 0x7f904815daa0 2023-11-09T03:46:18.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.226+0000 7f904d1c7640 1 -- 172.21.15.76:0/4042676112 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9038031d20 con 0x7f904815daa0 2023-11-09T03:46:18.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.226+0000 7f905042d640 1 -- 172.21.15.76:0/4042676112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f904815daa0 msgr2=0x7f9048157000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:18.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.226+0000 7f905042d640 1 --2- 172.21.15.76:0/4042676112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f904815daa0 0x7f9048157000 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f90480b5550 tx=0x7f903802b8d0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.226+0000 7f905042d640 1 -- 172.21.15.76:0/4042676112 shutdown_connections 2023-11-09T03:46:18.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.226+0000 7f905042d640 1 --2- 172.21.15.76:0/4042676112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f904815daa0 0x7f9048157000 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.226+0000 7f905042d640 1 --2- 172.21.15.76:0/4042676112 >> v2:172.21.15.113:3300/0 conn(0x7f90480cde60 0x7f904815d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.226+0000 7f905042d640 1 -- 172.21.15.76:0/4042676112 >> 172.21.15.76:0/4042676112 conn(0x7f9048071f30 msgr2=0x7f90480cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:18.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.226+0000 7f905042d640 1 -- 172.21.15.76:0/4042676112 shutdown_connections 2023-11-09T03:46:18.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.226+0000 7f905042d640 1 -- 172.21.15.76:0/4042676112 wait complete. 2023-11-09T03:46:18.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.227+0000 7f905042d640 1 Processor -- start 2023-11-09T03:46:18.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.227+0000 7f905042d640 1 -- start start 2023-11-09T03:46:18.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.228+0000 7f905042d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90480cde60 0x7f90481477a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:18.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.228+0000 7f905042d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90480ce240 0x7f90481f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:18.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.228+0000 7f905042d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f904815daa0 0x7f90481f5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:18.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.228+0000 7f905042d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90480c9ef0 con 0x7f90480cde60 2023-11-09T03:46:18.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.228+0000 7f905042d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9048147ce0 con 0x7f904815daa0 2023-11-09T03:46:18.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.228+0000 7f905042d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9048144cb0 con 0x7f90480ce240 2023-11-09T03:46:18.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.228+0000 7f904e1c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90480cde60 0x7f90481477a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:18.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.228+0000 7f904e1c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90480cde60 0x7f90481477a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36864/0 (socket says 172.21.15.76:36864) 2023-11-09T03:46:18.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.228+0000 7f904e1c9640 1 -- 172.21.15.76:0/3921875707 learned_addr learned my addr 172.21.15.76:0/3921875707 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:18.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.228+0000 7f904e9ca640 1 --2- 172.21.15.76:0/3921875707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f904815daa0 0x7f90481f5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:18.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.228+0000 7f904d9c8640 1 --2- 172.21.15.76:0/3921875707 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90480ce240 0x7f90481f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:18.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.228+0000 7f904e1c9640 1 -- 172.21.15.76:0/3921875707 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90480ce240 msgr2=0x7f90481f5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:18.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.228+0000 7f904e1c9640 1 --2- 172.21.15.76:0/3921875707 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90480ce240 0x7f90481f5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.228+0000 7f904e1c9640 1 -- 172.21.15.76:0/3921875707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f904815daa0 msgr2=0x7f90481f5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:18.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.228+0000 7f904e1c9640 1 --2- 172.21.15.76:0/3921875707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f904815daa0 0x7f90481f5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.228+0000 7f904e1c9640 1 -- 172.21.15.76:0/3921875707 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9038009400 con 0x7f90480cde60 2023-11-09T03:46:18.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.229+0000 7f904e1c9640 1 --2- 172.21.15.76:0/3921875707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90480cde60 0x7f90481477a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f904400cf60 tx=0x7f904400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:18.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.229+0000 7f903effd640 1 -- 172.21.15.76:0/3921875707 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904400d840 con 0x7f90480cde60 2023-11-09T03:46:18.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.229+0000 7f903effd640 1 -- 172.21.15.76:0/3921875707 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904400d9a0 con 0x7f90480cde60 2023-11-09T03:46:18.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.229+0000 7f905042d640 1 -- 172.21.15.76:0/3921875707 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90480cd210 con 0x7f90480cde60 2023-11-09T03:46:18.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.229+0000 7f903effd640 1 -- 172.21.15.76:0/3921875707 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904400dd30 con 0x7f90480cde60 2023-11-09T03:46:18.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.229+0000 7f905042d640 1 -- 172.21.15.76:0/3921875707 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90480ca100 con 0x7f90480cde60 2023-11-09T03:46:18.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.230+0000 7f905042d640 1 -- 172.21.15.76:0/3921875707 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9048074290 con 0x7f90480cde60 2023-11-09T03:46:18.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.230+0000 7f903effd640 1 -- 172.21.15.76:0/3921875707 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9044015440 con 0x7f90480cde60 2023-11-09T03:46:18.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.231+0000 7f903effd640 1 --2- 172.21.15.76:0/3921875707 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f902c06ae70 0x7f902c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:18.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.231+0000 7f903effd640 1 -- 172.21.15.76:0/3921875707 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f904408a380 con 0x7f90480cde60 2023-11-09T03:46:18.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.231+0000 7f904d9c8640 1 --2- 172.21.15.76:0/3921875707 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f902c06ae70 0x7f902c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:18.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.231+0000 7f904d9c8640 1 --2- 172.21.15.76:0/3921875707 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f902c06ae70 0x7f902c06d330 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9038036000 tx=0x7f903803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:18.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.235+0000 7f903effd640 1 -- 172.21.15.76:0/3921875707 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9044055100 con 0x7f90480cde60 2023-11-09T03:46:18.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.401+0000 7f905042d640 1 -- 172.21.15.76:0/3921875707 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90480d3a10 con 0x7f90480cde60 2023-11-09T03:46:18.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.402+0000 7f903effd640 1 -- 172.21.15.76:0/3921875707 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18710 (secure 0 0 0) 0x7f9044058720 con 0x7f90480cde60 2023-11-09T03:46:18.402 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:18.402 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":27,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:46:12.537265+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:46:18.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.404+0000 7f905042d640 1 -- 172.21.15.76:0/3921875707 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f902c06ae70 msgr2=0x7f902c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:18.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.404+0000 7f905042d640 1 --2- 172.21.15.76:0/3921875707 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f902c06ae70 0x7f902c06d330 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9038036000 tx=0x7f903803c040 comp rx=0 tx=0).stop 2023-11-09T03:46:18.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.404+0000 7f905042d640 1 -- 172.21.15.76:0/3921875707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90480cde60 msgr2=0x7f90481477a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:18.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.404+0000 7f905042d640 1 --2- 172.21.15.76:0/3921875707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90480cde60 0x7f90481477a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f904400cf60 tx=0x7f904400cf90 comp rx=0 tx=0).stop 2023-11-09T03:46:18.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.404+0000 7f905042d640 1 -- 172.21.15.76:0/3921875707 shutdown_connections 2023-11-09T03:46:18.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.404+0000 7f905042d640 1 --2- 172.21.15.76:0/3921875707 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f902c06ae70 0x7f902c06d330 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.404+0000 7f905042d640 1 --2- 172.21.15.76:0/3921875707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f904815daa0 0x7f90481f5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.404+0000 7f905042d640 1 --2- 172.21.15.76:0/3921875707 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90480ce240 0x7f90481f5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.404+0000 7f905042d640 1 --2- 172.21.15.76:0/3921875707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90480cde60 0x7f90481477a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.404+0000 7f905042d640 1 -- 172.21.15.76:0/3921875707 >> 172.21.15.76:0/3921875707 conn(0x7f9048071f30 msgr2=0x7f90480b6050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:18.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.404+0000 7f905042d640 1 -- 172.21.15.76:0/3921875707 shutdown_connections 2023-11-09T03:46:18.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.404+0000 7f905042d640 1 -- 172.21.15.76:0/3921875707 wait complete. 2023-11-09T03:46:18.415 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:45:28.704715+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:45:29.717614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-09T03:45:33.440577+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-09T03:46:18.416 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:46:18.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.519+0000 7f8686c90640 1 Processor -- start 2023-11-09T03:46:18.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.519+0000 7f8686c90640 1 -- start start 2023-11-09T03:46:18.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.520+0000 7f8686c90640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f868015ad30 0x7f86800adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:18.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.520+0000 7f8686c90640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f86800ad730 0x7f8680070710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:18.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.520+0000 7f8686c90640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8680070c50 0x7f86800c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:18.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.520+0000 7f8686c90640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86800ae260 con 0x7f86800ad730 2023-11-09T03:46:18.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.520+0000 7f8686c90640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86800710e0 con 0x7f868015ad30 2023-11-09T03:46:18.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.520+0000 7f8686c90640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86800c3900 con 0x7f8680070c50 2023-11-09T03:46:18.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.520+0000 7f867ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f86800ad730 0x7f8680070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:18.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.520+0000 7f867ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f86800ad730 0x7f8680070710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36866/0 (socket says 172.21.15.76:36866) 2023-11-09T03:46:18.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.520+0000 7f867ffff640 1 -- 172.21.15.76:0/3046872023 learned_addr learned my addr 172.21.15.76:0/3046872023 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:18.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.520+0000 7f867ffff640 1 -- 172.21.15.76:0/3046872023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8680070c50 msgr2=0x7f86800c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:18.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.520+0000 7f867ffff640 1 --2- 172.21.15.76:0/3046872023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8680070c50 0x7f86800c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.520+0000 7f867ffff640 1 -- 172.21.15.76:0/3046872023 >> v2:172.21.15.113:3300/0 conn(0x7f868015ad30 msgr2=0x7f86800adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:18.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.520+0000 7f867ffff640 1 --2- 172.21.15.76:0/3046872023 >> v2:172.21.15.113:3300/0 conn(0x7f868015ad30 0x7f86800adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.520+0000 7f867ffff640 1 -- 172.21.15.76:0/3046872023 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86800c3be0 con 0x7f86800ad730 2023-11-09T03:46:18.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.521+0000 7f867ffff640 1 --2- 172.21.15.76:0/3046872023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86800ad730 0x7f8680070710 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f868006bb00 tx=0x7f867402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba624cd4adbe00f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:18.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.521+0000 7f867f7fe640 1 -- 172.21.15.76:0/3046872023 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86740317c0 con 0x7f86800ad730 2023-11-09T03:46:18.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.521+0000 7f867f7fe640 1 -- 172.21.15.76:0/3046872023 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8674031920 con 0x7f86800ad730 2023-11-09T03:46:18.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.521+0000 7f867f7fe640 1 -- 172.21.15.76:0/3046872023 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674031cb0 con 0x7f86800ad730 2023-11-09T03:46:18.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.522+0000 7f8686c90640 1 -- 172.21.15.76:0/3046872023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86800ad730 msgr2=0x7f8680070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:18.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.522+0000 7f8686c90640 1 --2- 172.21.15.76:0/3046872023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86800ad730 0x7f8680070710 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f868006bb00 tx=0x7f867402b860 comp rx=0 tx=0).stop 2023-11-09T03:46:18.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.522+0000 7f8686c90640 1 -- 172.21.15.76:0/3046872023 shutdown_connections 2023-11-09T03:46:18.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.522+0000 7f8686c90640 1 --2- 172.21.15.76:0/3046872023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8680070c50 0x7f86800c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.522+0000 7f8686c90640 1 --2- 172.21.15.76:0/3046872023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86800ad730 0x7f8680070710 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.522+0000 7f8686c90640 1 --2- 172.21.15.76:0/3046872023 >> v2:172.21.15.113:3300/0 conn(0x7f868015ad30 0x7f86800adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.522+0000 7f8686c90640 1 -- 172.21.15.76:0/3046872023 >> 172.21.15.76:0/3046872023 conn(0x7f86800ce950 msgr2=0x7f86800ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:18.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.522+0000 7f8686c90640 1 -- 172.21.15.76:0/3046872023 shutdown_connections 2023-11-09T03:46:18.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.522+0000 7f8686c90640 1 -- 172.21.15.76:0/3046872023 wait complete. 2023-11-09T03:46:18.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.523+0000 7f8686c90640 1 Processor -- start 2023-11-09T03:46:18.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.523+0000 7f8686c90640 1 -- start start 2023-11-09T03:46:18.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.524+0000 7f8686c90640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8680070c50 0x7f868013dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:18.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.524+0000 7f8686c90640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86800ad730 0x7f86800c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:18.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.524+0000 7f8686c90640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f868015ad30 0x7f86800c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:18.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.524+0000 7f8686c90640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8680159b80 con 0x7f86800ad730 2023-11-09T03:46:18.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.524+0000 7f8686c90640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868015e0a0 con 0x7f868015ad30 2023-11-09T03:46:18.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.524+0000 7f8686c90640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8680151850 con 0x7f8680070c50 2023-11-09T03:46:18.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.524+0000 7f867ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86800ad730 0x7f86800c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:18.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.524+0000 7f867ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86800ad730 0x7f86800c5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36868/0 (socket says 172.21.15.76:36868) 2023-11-09T03:46:18.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.524+0000 7f867ffff640 1 -- 172.21.15.76:0/1151615033 learned_addr learned my addr 172.21.15.76:0/1151615033 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:18.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.524+0000 7f867ffff640 1 -- 172.21.15.76:0/1151615033 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8680070c50 msgr2=0x7f868013dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:18.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.524+0000 7f867ffff640 1 --2- 172.21.15.76:0/1151615033 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8680070c50 0x7f868013dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.524+0000 7f867ffff640 1 -- 172.21.15.76:0/1151615033 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f868015ad30 msgr2=0x7f86800c5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:18.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.524+0000 7f867ffff640 1 --2- 172.21.15.76:0/1151615033 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f868015ad30 0x7f86800c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.524+0000 7f867ffff640 1 -- 172.21.15.76:0/1151615033 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86740094d0 con 0x7f86800ad730 2023-11-09T03:46:18.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.525+0000 7f867ffff640 1 --2- 172.21.15.76:0/1151615033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86800ad730 0x7f86800c5a20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8674000c00 tx=0x7f86740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:18.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.525+0000 7f867d7fa640 1 -- 172.21.15.76:0/1151615033 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674032860 con 0x7f86800ad730 2023-11-09T03:46:18.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.525+0000 7f867d7fa640 1 -- 172.21.15.76:0/1151615033 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86740329e0 con 0x7f86800ad730 2023-11-09T03:46:18.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.525+0000 7f8686c90640 1 -- 172.21.15.76:0/1151615033 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8680158f00 con 0x7f86800ad730 2023-11-09T03:46:18.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.525+0000 7f867d7fa640 1 -- 172.21.15.76:0/1151615033 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674032d90 con 0x7f86800ad730 2023-11-09T03:46:18.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.525+0000 7f8686c90640 1 -- 172.21.15.76:0/1151615033 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86801594c0 con 0x7f86800ad730 2023-11-09T03:46:18.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.526+0000 7f8686c90640 1 -- 172.21.15.76:0/1151615033 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8680074290 con 0x7f86800ad730 2023-11-09T03:46:18.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.526+0000 7f867d7fa640 1 -- 172.21.15.76:0/1151615033 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f867404d3f0 con 0x7f86800ad730 2023-11-09T03:46:18.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.526+0000 7f867d7fa640 1 --2- 172.21.15.76:0/1151615033 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f866c06ae70 0x7f866c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:18.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.527+0000 7f867d7fa640 1 -- 172.21.15.76:0/1151615033 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f86740ab810 con 0x7f86800ad730 2023-11-09T03:46:18.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.527+0000 7f8684a2c640 1 --2- 172.21.15.76:0/1151615033 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f866c06ae70 0x7f866c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:18.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.527+0000 7f8684a2c640 1 --2- 172.21.15.76:0/1151615033 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f866c06ae70 0x7f866c06d330 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8670009670 tx=0x7f8670006bf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:18.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.531+0000 7f867d7fa640 1 -- 172.21.15.76:0/1151615033 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8674076570 con 0x7f86800ad730 2023-11-09T03:46:18.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.697+0000 7f8686c90640 1 -- 172.21.15.76:0/1151615033 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f86800b1120 con 0x7f86800ad730 2023-11-09T03:46:18.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.697+0000 7f867d7fa640 1 -- 172.21.15.76:0/1151615033 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8674079b90 con 0x7f86800ad730 2023-11-09T03:46:18.697 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:46:18.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.699+0000 7f8686c90640 1 -- 172.21.15.76:0/1151615033 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f866c06ae70 msgr2=0x7f866c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:18.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.699+0000 7f8686c90640 1 --2- 172.21.15.76:0/1151615033 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f866c06ae70 0x7f866c06d330 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8670009670 tx=0x7f8670006bf0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.699+0000 7f8686c90640 1 -- 172.21.15.76:0/1151615033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86800ad730 msgr2=0x7f86800c5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:18.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.699+0000 7f8686c90640 1 --2- 172.21.15.76:0/1151615033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86800ad730 0x7f86800c5a20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8674000c00 tx=0x7f86740046c0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.699+0000 7f8686c90640 1 -- 172.21.15.76:0/1151615033 shutdown_connections 2023-11-09T03:46:18.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.699+0000 7f8686c90640 1 --2- 172.21.15.76:0/1151615033 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f866c06ae70 0x7f866c06d330 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.699+0000 7f8686c90640 1 --2- 172.21.15.76:0/1151615033 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f868015ad30 0x7f86800c5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.699+0000 7f8686c90640 1 --2- 172.21.15.76:0/1151615033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86800ad730 0x7f86800c5a20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.699+0000 7f8686c90640 1 --2- 172.21.15.76:0/1151615033 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8680070c50 0x7f868013dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.699+0000 7f8686c90640 1 -- 172.21.15.76:0/1151615033 >> 172.21.15.76:0/1151615033 conn(0x7f86800ce950 msgr2=0x7f868006c710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:18.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.700+0000 7f8686c90640 1 -- 172.21.15.76:0/1151615033 shutdown_connections 2023-11-09T03:46:18.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.700+0000 7f8686c90640 1 -- 172.21.15.76:0/1151615033 wait complete. 2023-11-09T03:46:18.709 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:46:18.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.812+0000 7f8036978640 1 Processor -- start 2023-11-09T03:46:18.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.812+0000 7f8036978640 1 -- start start 2023-11-09T03:46:18.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.813+0000 7f8036978640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8030153e60 0x7f803015dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:18.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.813+0000 7f8036978640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80300b4880 0x7f80300b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:18.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.813+0000 7f8036978640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80300cde60 con 0x7f80300cd6d0 2023-11-09T03:46:18.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.813+0000 7f8036978640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8030160f90 con 0x7f8030153e60 2023-11-09T03:46:18.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.813+0000 7f8036978640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80300b5200 con 0x7f80300b4880 2023-11-09T03:46:18.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.814+0000 7f8034f15640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80300b4880 0x7f80300b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:18.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.814+0000 7f802ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8030153e60 0x7f803015dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:18.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.814+0000 7f802f7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f80300cd6d0 0x7f8030161420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:60098/0 (socket says 172.21.15.76:60098) 2023-11-09T03:46:18.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.814+0000 7f802f7fe640 1 -- 172.21.15.76:0/2974003571 learned_addr learned my addr 172.21.15.76:0/2974003571 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:18.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.814+0000 7f802ffff640 1 -- 172.21.15.76:0/2974003571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80300b4880 msgr2=0x7f80300b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:18.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.814+0000 7f802ffff640 1 --2- 172.21.15.76:0/2974003571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80300b4880 0x7f80300b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.814+0000 7f802ffff640 1 -- 172.21.15.76:0/2974003571 >> v1:172.21.15.113:6789/0 conn(0x7f80300cd6d0 legacy=0x7f8030161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:18.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.814+0000 7f802ffff640 1 -- 172.21.15.76:0/2974003571 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803015e400 con 0x7f8030153e60 2023-11-09T03:46:18.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.814+0000 7f8034f15640 1 --2- 172.21.15.76:0/2974003571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80300b4880 0x7f80300b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:46:18.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.815+0000 7f802ffff640 1 --2- 172.21.15.76:0/2974003571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8030153e60 0x7f803015dec0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f80200097a0 tx=0x7f802002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d409cc6a4e79698 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:18.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.815+0000 7f802effd640 1 -- 172.21.15.76:0/2974003571 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020031830 con 0x7f8030153e60 2023-11-09T03:46:18.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.815+0000 7f802effd640 1 -- 172.21.15.76:0/2974003571 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8020031990 con 0x7f8030153e60 2023-11-09T03:46:18.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.815+0000 7f802effd640 1 -- 172.21.15.76:0/2974003571 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020031d20 con 0x7f8030153e60 2023-11-09T03:46:18.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.815+0000 7f8036978640 1 -- 172.21.15.76:0/2974003571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8030153e60 msgr2=0x7f803015dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:18.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.815+0000 7f8036978640 1 --2- 172.21.15.76:0/2974003571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8030153e60 0x7f803015dec0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f80200097a0 tx=0x7f802002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.816+0000 7f8036978640 1 -- 172.21.15.76:0/2974003571 shutdown_connections 2023-11-09T03:46:18.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.816+0000 7f8036978640 1 --2- 172.21.15.76:0/2974003571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80300b4880 0x7f80300b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.816+0000 7f8036978640 1 --2- 172.21.15.76:0/2974003571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8030153e60 0x7f803015dec0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.816+0000 7f8036978640 1 -- 172.21.15.76:0/2974003571 >> 172.21.15.76:0/2974003571 conn(0x7f80300cf880 msgr2=0x7f80300cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:18.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.816+0000 7f8036978640 1 -- 172.21.15.76:0/2974003571 shutdown_connections 2023-11-09T03:46:18.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.816+0000 7f8036978640 1 -- 172.21.15.76:0/2974003571 wait complete. 2023-11-09T03:46:18.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.817+0000 7f8036978640 1 Processor -- start 2023-11-09T03:46:18.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.817+0000 7f8036978640 1 -- start start 2023-11-09T03:46:18.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.817+0000 7f8036978640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80300b4880 0x7f8030153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:18.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.817+0000 7f8036978640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80300cd6d0 0x7f80300cc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:18.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.817+0000 7f8036978640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8030153e60 0x7f80300cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:18.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.817+0000 7f8036978640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803015cac0 con 0x7f8030153e60 2023-11-09T03:46:18.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.817+0000 7f8036978640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803015beb0 con 0x7f80300b4880 2023-11-09T03:46:18.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.817+0000 7f8036978640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803015c190 con 0x7f80300cd6d0 2023-11-09T03:46:18.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.818+0000 7f8034f15640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8030153e60 0x7f80300cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:18.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.818+0000 7f8034f15640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8030153e60 0x7f80300cc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36872/0 (socket says 172.21.15.76:36872) 2023-11-09T03:46:18.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.818+0000 7f8034f15640 1 -- 172.21.15.76:0/2545620939 learned_addr learned my addr 172.21.15.76:0/2545620939 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:18.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.818+0000 7f802f7fe640 1 --2- 172.21.15.76:0/2545620939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80300cd6d0 0x7f80300cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:18.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.818+0000 7f802ffff640 1 --2- 172.21.15.76:0/2545620939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80300b4880 0x7f8030153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:18.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.818+0000 7f8034f15640 1 -- 172.21.15.76:0/2545620939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80300cd6d0 msgr2=0x7f80300cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:18.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.818+0000 7f8034f15640 1 --2- 172.21.15.76:0/2545620939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80300cd6d0 0x7f80300cc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.818+0000 7f8034f15640 1 -- 172.21.15.76:0/2545620939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80300b4880 msgr2=0x7f8030153410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:18.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.818+0000 7f8034f15640 1 --2- 172.21.15.76:0/2545620939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80300b4880 0x7f8030153410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.818+0000 7f8034f15640 1 -- 172.21.15.76:0/2545620939 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8028009a70 con 0x7f8030153e60 2023-11-09T03:46:18.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.818+0000 7f8034f15640 1 --2- 172.21.15.76:0/2545620939 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8030153e60 0x7f80300cc7c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f802800efc0 tx=0x7f8028009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:18.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.818+0000 7f802cff9640 1 -- 172.21.15.76:0/2545620939 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8028009d70 con 0x7f8030153e60 2023-11-09T03:46:18.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.818+0000 7f802cff9640 1 -- 172.21.15.76:0/2545620939 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802800f040 con 0x7f8030153e60 2023-11-09T03:46:18.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.818+0000 7f802cff9640 1 -- 172.21.15.76:0/2545620939 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8028003990 con 0x7f8030153e60 2023-11-09T03:46:18.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.818+0000 7f8036978640 1 -- 172.21.15.76:0/2545620939 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8020009400 con 0x7f8030153e60 2023-11-09T03:46:18.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.818+0000 7f8036978640 1 -- 172.21.15.76:0/2545620939 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f803015c750 con 0x7f8030153e60 2023-11-09T03:46:18.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.819+0000 7f8036978640 1 -- 172.21.15.76:0/2545620939 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8030074290 con 0x7f8030153e60 2023-11-09T03:46:18.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.820+0000 7f802cff9640 1 -- 172.21.15.76:0/2545620939 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f80280144a0 con 0x7f8030153e60 2023-11-09T03:46:18.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.820+0000 7f802cff9640 1 --2- 172.21.15.76:0/2545620939 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f801c06ae70 0x7f801c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:18.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.820+0000 7f802cff9640 1 -- 172.21.15.76:0/2545620939 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f802808a570 con 0x7f8030153e60 2023-11-09T03:46:18.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.821+0000 7f802ffff640 1 --2- 172.21.15.76:0/2545620939 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f801c06ae70 0x7f801c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:18.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.821+0000 7f802ffff640 1 --2- 172.21.15.76:0/2545620939 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f801c06ae70 0x7f801c06d330 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8020036000 tx=0x7f802003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:18.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.825+0000 7f802cff9640 1 -- 172.21.15.76:0/2545620939 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f80280552f0 con 0x7f8030153e60 2023-11-09T03:46:18.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.990+0000 7f8036978640 1 -- 172.21.15.76:0/2545620939 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f80300d1d00 con 0x7f8030153e60 2023-11-09T03:46:18.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.991+0000 7f802cff9640 1 -- 172.21.15.76:0/2545620939 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8028058910 con 0x7f8030153e60 2023-11-09T03:46:18.991 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:46:18.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.993+0000 7f8036978640 1 -- 172.21.15.76:0/2545620939 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f801c06ae70 msgr2=0x7f801c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:18.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.993+0000 7f8036978640 1 --2- 172.21.15.76:0/2545620939 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f801c06ae70 0x7f801c06d330 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8020036000 tx=0x7f802003c040 comp rx=0 tx=0).stop 2023-11-09T03:46:18.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.993+0000 7f8036978640 1 -- 172.21.15.76:0/2545620939 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8030153e60 msgr2=0x7f80300cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:18.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.993+0000 7f8036978640 1 --2- 172.21.15.76:0/2545620939 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8030153e60 0x7f80300cc7c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f802800efc0 tx=0x7f8028009f90 comp rx=0 tx=0).stop 2023-11-09T03:46:18.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.993+0000 7f8036978640 1 -- 172.21.15.76:0/2545620939 shutdown_connections 2023-11-09T03:46:18.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.993+0000 7f8036978640 1 --2- 172.21.15.76:0/2545620939 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f801c06ae70 0x7f801c06d330 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.993+0000 7f8036978640 1 --2- 172.21.15.76:0/2545620939 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8030153e60 0x7f80300cc7c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.993+0000 7f8036978640 1 --2- 172.21.15.76:0/2545620939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80300cd6d0 0x7f80300cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.993+0000 7f8036978640 1 --2- 172.21.15.76:0/2545620939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80300b4880 0x7f8030153410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:18.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.993+0000 7f8036978640 1 -- 172.21.15.76:0/2545620939 >> 172.21.15.76:0/2545620939 conn(0x7f80300cf880 msgr2=0x7f80300d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:18.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.993+0000 7f8036978640 1 -- 172.21.15.76:0/2545620939 shutdown_connections 2023-11-09T03:46:18.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:18.993+0000 7f8036978640 1 -- 172.21.15.76:0/2545620939 wait complete. 2023-11-09T03:46:19.003 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:46:19.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.107+0000 7fb13a47c640 1 Processor -- start 2023-11-09T03:46:19.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.107+0000 7fb13a47c640 1 -- start start 2023-11-09T03:46:19.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.108+0000 7fb13a47c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb1341460c0 0x7fb1340b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:19.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.108+0000 7fb13a47c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1340c37d0 0x7fb1340c3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:19.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.108+0000 7fb13a47c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb134146950 con 0x7fb134070710 2023-11-09T03:46:19.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.108+0000 7fb13a47c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1341464a0 con 0x7fb1341460c0 2023-11-09T03:46:19.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.108+0000 7fb13a47c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1340c4170 con 0x7fb1340c37d0 2023-11-09T03:46:19.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.108+0000 7fb133fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb1341460c0 0x7fb1340b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:19.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.108+0000 7fb133fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb1341460c0 0x7fb1340b1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36878/0 (socket says 172.21.15.76:36878) 2023-11-09T03:46:19.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.108+0000 7fb133fff640 1 -- 172.21.15.76:0/206662671 learned_addr learned my addr 172.21.15.76:0/206662671 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:19.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.108+0000 7fb1337fe640 1 --2- 172.21.15.76:0/206662671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1340c37d0 0x7fb1340c3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:19.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.108+0000 7fb133fff640 1 -- 172.21.15.76:0/206662671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1340c37d0 msgr2=0x7fb1340c3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:19.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.108+0000 7fb133fff640 1 --2- 172.21.15.76:0/206662671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1340c37d0 0x7fb1340c3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.108+0000 7fb133fff640 1 -- 172.21.15.76:0/206662671 >> v1:172.21.15.113:6789/0 conn(0x7fb134070710 legacy=0x7fb1340ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:19.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.108+0000 7fb133fff640 1 -- 172.21.15.76:0/206662671 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1340cc5c0 con 0x7fb1341460c0 2023-11-09T03:46:19.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.108+0000 7fb1337fe640 1 --2- 172.21.15.76:0/206662671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1340c37d0 0x7fb1340c3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:19.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.109+0000 7fb133fff640 1 --2- 172.21.15.76:0/206662671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1341460c0 0x7fb1340b1860 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb124009870 tx=0x7fb12402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa2cb62d325d917 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:19.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.109+0000 7fb132ffd640 1 -- 172.21.15.76:0/206662671 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb124031640 con 0x7fb1341460c0 2023-11-09T03:46:19.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.109+0000 7fb132ffd640 1 -- 172.21.15.76:0/206662671 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1240317a0 con 0x7fb1341460c0 2023-11-09T03:46:19.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.109+0000 7fb132ffd640 1 -- 172.21.15.76:0/206662671 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb124031b30 con 0x7fb1341460c0 2023-11-09T03:46:19.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.110+0000 7fb13a47c640 1 -- 172.21.15.76:0/206662671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1341460c0 msgr2=0x7fb1340b1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:19.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.110+0000 7fb13a47c640 1 --2- 172.21.15.76:0/206662671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1341460c0 0x7fb1340b1860 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb124009870 tx=0x7fb12402b8f0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.110+0000 7fb13a47c640 1 -- 172.21.15.76:0/206662671 shutdown_connections 2023-11-09T03:46:19.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.110+0000 7fb13a47c640 1 --2- 172.21.15.76:0/206662671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1340c37d0 0x7fb1340c3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.110+0000 7fb13a47c640 1 --2- 172.21.15.76:0/206662671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1341460c0 0x7fb1340b1860 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.110+0000 7fb13a47c640 1 -- 172.21.15.76:0/206662671 >> 172.21.15.76:0/206662671 conn(0x7fb1340ce950 msgr2=0x7fb1340ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:19.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.110+0000 7fb13a47c640 1 -- 172.21.15.76:0/206662671 shutdown_connections 2023-11-09T03:46:19.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.110+0000 7fb13a47c640 1 -- 172.21.15.76:0/206662671 wait complete. 2023-11-09T03:46:19.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.111+0000 7fb13a47c640 1 Processor -- start 2023-11-09T03:46:19.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.111+0000 7fb13a47c640 1 -- start start 2023-11-09T03:46:19.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.112+0000 7fb13a47c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb134070710 0x7fb1341e6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:19.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.112+0000 7fb13a47c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1340c37d0 0x7fb1341e6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:19.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.112+0000 7fb13a47c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1341460c0 0x7fb13415e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:19.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.112+0000 7fb13a47c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1340b14e0 con 0x7fb1341460c0 2023-11-09T03:46:19.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.112+0000 7fb13a47c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1341e7100 con 0x7fb1340c37d0 2023-11-09T03:46:19.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.112+0000 7fb13a47c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1340b1730 con 0x7fb134070710 2023-11-09T03:46:19.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.112+0000 7fb138a19640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1341460c0 0x7fb13415e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:19.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.112+0000 7fb138a19640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1341460c0 0x7fb13415e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36888/0 (socket says 172.21.15.76:36888) 2023-11-09T03:46:19.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.112+0000 7fb138a19640 1 -- 172.21.15.76:0/2805028660 learned_addr learned my addr 172.21.15.76:0/2805028660 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:19.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.112+0000 7fb1337fe640 1 --2- 172.21.15.76:0/2805028660 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1340c37d0 0x7fb1341e6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:19.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.112+0000 7fb133fff640 1 --2- 172.21.15.76:0/2805028660 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb134070710 0x7fb1341e6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:19.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.112+0000 7fb138a19640 1 -- 172.21.15.76:0/2805028660 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb134070710 msgr2=0x7fb1341e6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:19.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.112+0000 7fb138a19640 1 --2- 172.21.15.76:0/2805028660 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb134070710 0x7fb1341e6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.112+0000 7fb138a19640 1 -- 172.21.15.76:0/2805028660 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1340c37d0 msgr2=0x7fb1341e6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:19.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.112+0000 7fb138a19640 1 --2- 172.21.15.76:0/2805028660 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1340c37d0 0x7fb1341e6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.112+0000 7fb138a19640 1 -- 172.21.15.76:0/2805028660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1240094d0 con 0x7fb1341460c0 2023-11-09T03:46:19.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.112+0000 7fb133fff640 1 --2- 172.21.15.76:0/2805028660 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb134070710 0x7fb1341e6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:19.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.113+0000 7fb138a19640 1 --2- 172.21.15.76:0/2805028660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1341460c0 0x7fb13415e800 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00c780 tx=0x7fb12c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:19.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.113+0000 7fb130ff9640 1 -- 172.21.15.76:0/2805028660 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c004b60 con 0x7fb1341460c0 2023-11-09T03:46:19.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.113+0000 7fb130ff9640 1 -- 172.21.15.76:0/2805028660 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12c00ce90 con 0x7fb1341460c0 2023-11-09T03:46:19.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.113+0000 7fb13a47c640 1 -- 172.21.15.76:0/2805028660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1340b1a10 con 0x7fb1341460c0 2023-11-09T03:46:19.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.113+0000 7fb130ff9640 1 -- 172.21.15.76:0/2805028660 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c00a850 con 0x7fb1341460c0 2023-11-09T03:46:19.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.113+0000 7fb13a47c640 1 -- 172.21.15.76:0/2805028660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1340c5db0 con 0x7fb1341460c0 2023-11-09T03:46:19.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.114+0000 7fb13a47c640 1 -- 172.21.15.76:0/2805028660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1340b1f30 con 0x7fb1341460c0 2023-11-09T03:46:19.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.114+0000 7fb130ff9640 1 -- 172.21.15.76:0/2805028660 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb12c00a9b0 con 0x7fb1341460c0 2023-11-09T03:46:19.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.115+0000 7fb130ff9640 1 --2- 172.21.15.76:0/2805028660 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb12006aea0 0x7fb12006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:19.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.115+0000 7fb130ff9640 1 -- 172.21.15.76:0/2805028660 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fb12c089e20 con 0x7fb1341460c0 2023-11-09T03:46:19.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.115+0000 7fb133fff640 1 --2- 172.21.15.76:0/2805028660 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb12006aea0 0x7fb12006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:19.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.115+0000 7fb133fff640 1 --2- 172.21.15.76:0/2805028660 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb12006aea0 0x7fb12006d360 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb124036000 tx=0x7fb12403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:19.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.119+0000 7fb130ff9640 1 -- 172.21.15.76:0/2805028660 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb1340b1f30 con 0x7fb1341460c0 2023-11-09T03:46:19.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.285+0000 7fb13a47c640 1 -- 172.21.15.76:0/2805028660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb1340b1f30 con 0x7fb1341460c0 2023-11-09T03:46:19.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.286+0000 7fb130ff9640 1 -- 172.21.15.76:0/2805028660 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb1340b1f30 con 0x7fb1341460c0 2023-11-09T03:46:19.285 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:46:19.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.287+0000 7fb13a47c640 1 -- 172.21.15.76:0/2805028660 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb12006aea0 msgr2=0x7fb12006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:19.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.287+0000 7fb13a47c640 1 --2- 172.21.15.76:0/2805028660 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb12006aea0 0x7fb12006d360 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb124036000 tx=0x7fb12403c040 comp rx=0 tx=0).stop 2023-11-09T03:46:19.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.287+0000 7fb13a47c640 1 -- 172.21.15.76:0/2805028660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1341460c0 msgr2=0x7fb13415e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:19.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.287+0000 7fb13a47c640 1 --2- 172.21.15.76:0/2805028660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1341460c0 0x7fb13415e800 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00c780 tx=0x7fb12c00cc40 comp rx=0 tx=0).stop 2023-11-09T03:46:19.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.287+0000 7fb13a47c640 1 -- 172.21.15.76:0/2805028660 shutdown_connections 2023-11-09T03:46:19.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.287+0000 7fb13a47c640 1 --2- 172.21.15.76:0/2805028660 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb12006aea0 0x7fb12006d360 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.287+0000 7fb13a47c640 1 --2- 172.21.15.76:0/2805028660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1341460c0 0x7fb13415e800 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.287+0000 7fb13a47c640 1 --2- 172.21.15.76:0/2805028660 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1340c37d0 0x7fb1341e6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.287+0000 7fb13a47c640 1 --2- 172.21.15.76:0/2805028660 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb134070710 0x7fb1341e6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.287+0000 7fb13a47c640 1 -- 172.21.15.76:0/2805028660 >> 172.21.15.76:0/2805028660 conn(0x7fb1340ce950 msgr2=0x7fb1340b0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:19.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.288+0000 7fb13a47c640 1 -- 172.21.15.76:0/2805028660 shutdown_connections 2023-11-09T03:46:19.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.288+0000 7fb13a47c640 1 -- 172.21.15.76:0/2805028660 wait complete. 2023-11-09T03:46:19.298 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-09T03:46:19.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.403+0000 7fd23b739640 1 Processor -- start 2023-11-09T03:46:19.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.403+0000 7fd23b739640 1 -- start start 2023-11-09T03:46:19.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.404+0000 7fd23b739640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd234153e60 0x7fd23415dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:19.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.404+0000 7fd23b739640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2340cde60 con 0x7fd2340cd6d0 2023-11-09T03:46:19.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.404+0000 7fd23b739640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd234160f90 con 0x7fd2340b4880 2023-11-09T03:46:19.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.404+0000 7fd23b739640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd23415e400 con 0x7fd234153e60 2023-11-09T03:46:19.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.404+0000 7fd2394d5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd234153e60 0x7fd23415dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:19.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.404+0000 7fd238cd4640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd2340cd6d0 0x7fd234161420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43690/0 (socket says 172.21.15.76:43690) 2023-11-09T03:46:19.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.404+0000 7fd238cd4640 1 -- 172.21.15.76:0/2215611991 learned_addr learned my addr 172.21.15.76:0/2215611991 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:19.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.404+0000 7fd2394d5640 1 -- 172.21.15.76:0/2215611991 >> v1:172.21.15.113:6789/0 conn(0x7fd2340b4880 legacy=0x7fd2340d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:19.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.404+0000 7fd2394d5640 1 -- 172.21.15.76:0/2215611991 >> v1:172.21.15.76:6789/0 conn(0x7fd2340cd6d0 legacy=0x7fd234161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:19.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.404+0000 7fd22bfff640 1 -- 172.21.15.76:0/2215611991 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1482793776 0 0) 0x7fd2340cde60 con 0x7fd2340cd6d0 2023-11-09T03:46:19.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.404+0000 7fd2394d5640 1 -- 172.21.15.76:0/2215611991 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2340ce090 con 0x7fd234153e60 2023-11-09T03:46:19.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.405+0000 7fd2394d5640 1 --2- 172.21.15.76:0/2215611991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd234153e60 0x7fd23415dec0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd2300097a0 tx=0x7fd23002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e134e1b728dfd22 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:19.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.405+0000 7fd22bfff640 1 -- 172.21.15.76:0/2215611991 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd230030610 con 0x7fd234153e60 2023-11-09T03:46:19.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.405+0000 7fd22bfff640 1 -- 172.21.15.76:0/2215611991 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd230030770 con 0x7fd234153e60 2023-11-09T03:46:19.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.405+0000 7fd22bfff640 1 -- 172.21.15.76:0/2215611991 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd230030b00 con 0x7fd234153e60 2023-11-09T03:46:19.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.406+0000 7fd23b739640 1 -- 172.21.15.76:0/2215611991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd234153e60 msgr2=0x7fd23415dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:19.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.406+0000 7fd23b739640 1 --2- 172.21.15.76:0/2215611991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd234153e60 0x7fd23415dec0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd2300097a0 tx=0x7fd23002b960 comp rx=0 tx=0).stop 2023-11-09T03:46:19.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.406+0000 7fd23b739640 1 -- 172.21.15.76:0/2215611991 shutdown_connections 2023-11-09T03:46:19.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.406+0000 7fd23b739640 1 --2- 172.21.15.76:0/2215611991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd234153e60 0x7fd23415dec0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.406+0000 7fd23b739640 1 -- 172.21.15.76:0/2215611991 >> 172.21.15.76:0/2215611991 conn(0x7fd2340cf880 msgr2=0x7fd2340cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:19.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.406+0000 7fd23b739640 1 -- 172.21.15.76:0/2215611991 shutdown_connections 2023-11-09T03:46:19.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.406+0000 7fd23b739640 1 -- 172.21.15.76:0/2215611991 wait complete. 2023-11-09T03:46:19.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.407+0000 7fd23b739640 1 Processor -- start 2023-11-09T03:46:19.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.407+0000 7fd23b739640 1 -- start start 2023-11-09T03:46:19.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.408+0000 7fd23b739640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2340b4880 0x7fd2341507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:19.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.408+0000 7fd23b739640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2340cd6d0 0x7fd2340d2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:19.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.408+0000 7fd23b739640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd234153e60 0x7fd2340d3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:19.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.408+0000 7fd23b739640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd234155e50 con 0x7fd2340cd6d0 2023-11-09T03:46:19.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.408+0000 7fd23b739640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd234156130 con 0x7fd234153e60 2023-11-09T03:46:19.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.408+0000 7fd23b739640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd234156410 con 0x7fd2340b4880 2023-11-09T03:46:19.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.408+0000 7fd238cd4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2340cd6d0 0x7fd2340d2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:19.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.408+0000 7fd238cd4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2340cd6d0 0x7fd2340d2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36910/0 (socket says 172.21.15.76:36910) 2023-11-09T03:46:19.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.408+0000 7fd2394d5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2340b4880 0x7fd2341507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:19.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.408+0000 7fd238cd4640 1 -- 172.21.15.76:0/2114528862 learned_addr learned my addr 172.21.15.76:0/2114528862 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:19.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.408+0000 7fd239cd6640 1 --2- 172.21.15.76:0/2114528862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd234153e60 0x7fd2340d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:19.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.408+0000 7fd238cd4640 1 -- 172.21.15.76:0/2114528862 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2340b4880 msgr2=0x7fd2341507c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:19.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.408+0000 7fd238cd4640 1 --2- 172.21.15.76:0/2114528862 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2340b4880 0x7fd2341507c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.408+0000 7fd238cd4640 1 -- 172.21.15.76:0/2114528862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd234153e60 msgr2=0x7fd2340d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:19.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.408+0000 7fd238cd4640 1 --2- 172.21.15.76:0/2114528862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd234153e60 0x7fd2340d3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.408+0000 7fd238cd4640 1 -- 172.21.15.76:0/2114528862 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd230009400 con 0x7fd2340cd6d0 2023-11-09T03:46:19.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.408+0000 7fd238cd4640 1 --2- 172.21.15.76:0/2114528862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2340cd6d0 0x7fd2340d2de0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd22400c460 tx=0x7fd22400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:19.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.409+0000 7fd229ffb640 1 -- 172.21.15.76:0/2114528862 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2240074e0 con 0x7fd2340cd6d0 2023-11-09T03:46:19.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.409+0000 7fd229ffb640 1 -- 172.21.15.76:0/2114528862 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd224007680 con 0x7fd2340cd6d0 2023-11-09T03:46:19.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.409+0000 7fd23b739640 1 -- 172.21.15.76:0/2114528862 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2340ca5c0 con 0x7fd2340cd6d0 2023-11-09T03:46:19.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.409+0000 7fd229ffb640 1 -- 172.21.15.76:0/2114528862 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd224004a00 con 0x7fd2340cd6d0 2023-11-09T03:46:19.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.409+0000 7fd23b739640 1 -- 172.21.15.76:0/2114528862 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2341550b0 con 0x7fd2340cd6d0 2023-11-09T03:46:19.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.410+0000 7fd23b739640 1 -- 172.21.15.76:0/2114528862 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2340ca420 con 0x7fd2340cd6d0 2023-11-09T03:46:19.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.410+0000 7fd229ffb640 1 -- 172.21.15.76:0/2114528862 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd224004b60 con 0x7fd2340cd6d0 2023-11-09T03:46:19.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.411+0000 7fd229ffb640 1 --2- 172.21.15.76:0/2114528862 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd21806aea0 0x7fd21806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:19.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.411+0000 7fd229ffb640 1 -- 172.21.15.76:0/2114528862 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fd224088b30 con 0x7fd2340cd6d0 2023-11-09T03:46:19.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.411+0000 7fd2394d5640 1 --2- 172.21.15.76:0/2114528862 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd21806aea0 0x7fd21806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:19.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.411+0000 7fd2394d5640 1 --2- 172.21.15.76:0/2114528862 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd21806aea0 0x7fd21806d360 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd230036000 tx=0x7fd23003b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:19.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.415+0000 7fd229ffb640 1 -- 172.21.15.76:0/2114528862 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd2340ca420 con 0x7fd2340cd6d0 2023-11-09T03:46:19.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.580+0000 7fd23b739640 1 -- 172.21.15.76:0/2114528862 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd2340ca420 con 0x7fd2340cd6d0 2023-11-09T03:46:19.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.581+0000 7fd229ffb640 1 -- 172.21.15.76:0/2114528862 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd2340ca420 con 0x7fd2340cd6d0 2023-11-09T03:46:19.580 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:46:19.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.582+0000 7fd23b739640 1 -- 172.21.15.76:0/2114528862 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd21806aea0 msgr2=0x7fd21806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:19.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.582+0000 7fd23b739640 1 --2- 172.21.15.76:0/2114528862 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd21806aea0 0x7fd21806d360 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd230036000 tx=0x7fd23003b040 comp rx=0 tx=0).stop 2023-11-09T03:46:19.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.582+0000 7fd23b739640 1 -- 172.21.15.76:0/2114528862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2340cd6d0 msgr2=0x7fd2340d2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:19.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.582+0000 7fd23b739640 1 --2- 172.21.15.76:0/2114528862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2340cd6d0 0x7fd2340d2de0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd22400c460 tx=0x7fd22400c920 comp rx=0 tx=0).stop 2023-11-09T03:46:19.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.582+0000 7fd23b739640 1 -- 172.21.15.76:0/2114528862 shutdown_connections 2023-11-09T03:46:19.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.582+0000 7fd23b739640 1 --2- 172.21.15.76:0/2114528862 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd21806aea0 0x7fd21806d360 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.582+0000 7fd23b739640 1 --2- 172.21.15.76:0/2114528862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd234153e60 0x7fd2340d3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.582+0000 7fd23b739640 1 --2- 172.21.15.76:0/2114528862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2340cd6d0 0x7fd2340d2de0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.582+0000 7fd23b739640 1 --2- 172.21.15.76:0/2114528862 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2340b4880 0x7fd2341507c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.583+0000 7fd23b739640 1 -- 172.21.15.76:0/2114528862 >> 172.21.15.76:0/2114528862 conn(0x7fd2340cf880 msgr2=0x7fd23415fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:19.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.583+0000 7fd23b739640 1 -- 172.21.15.76:0/2114528862 shutdown_connections 2023-11-09T03:46:19.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.583+0000 7fd23b739640 1 -- 172.21.15.76:0/2114528862 wait complete. 2023-11-09T03:46:19.651 INFO:tasks.cephfs_test_runner:Starting test: test_attaching_pools_with_snaps_to_fs_fails (tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools) Test that attempt to attach pool with snapshots to an fs fails 2023-11-09T03:46:19.652 INFO:tasks.cephfs_test_runner:test_attaching_pools_with_snaps_to_fs_fails (tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools) 2023-11-09T03:46:19.652 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:46:19.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.754+0000 7feb638cf640 1 Processor -- start 2023-11-09T03:46:19.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.754+0000 7feb638cf640 1 -- start start 2023-11-09T03:46:19.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.755+0000 7feb638cf640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7feb5c159c70 0x7feb5c0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:19.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.755+0000 7feb638cf640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feb5c06db20 0x7feb5c0b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:19.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.755+0000 7feb638cf640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb5c0b0910 0x7feb5c15d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:19.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.755+0000 7feb638cf640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb5c0b49b0 con 0x7feb5c159c70 2023-11-09T03:46:19.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.755+0000 7feb638cf640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb5c0b0da0 con 0x7feb5c06db20 2023-11-09T03:46:19.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.755+0000 7feb638cf640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c0b4110 con 0x7feb5c0b0910 2023-11-09T03:46:19.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.755+0000 7feb6166b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7feb5c159c70 0x7feb5c0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:19.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.755+0000 7feb6166b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7feb5c159c70 0x7feb5c0ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36918/0 (socket says 172.21.15.76:36918) 2023-11-09T03:46:19.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.755+0000 7feb6166b640 1 -- 172.21.15.76:0/1378224752 learned_addr learned my addr 172.21.15.76:0/1378224752 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:19.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.755+0000 7feb6166b640 1 -- 172.21.15.76:0/1378224752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb5c0b0910 msgr2=0x7feb5c15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:19.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.755+0000 7feb6166b640 1 --2- 172.21.15.76:0/1378224752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb5c0b0910 0x7feb5c15d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.755+0000 7feb6166b640 1 -- 172.21.15.76:0/1378224752 >> v2:172.21.15.113:3300/0 conn(0x7feb5c06db20 msgr2=0x7feb5c0b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:19.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.755+0000 7feb6166b640 1 --2- 172.21.15.76:0/1378224752 >> v2:172.21.15.113:3300/0 conn(0x7feb5c06db20 0x7feb5c0b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.755+0000 7feb6166b640 1 -- 172.21.15.76:0/1378224752 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb5c0b43f0 con 0x7feb5c159c70 2023-11-09T03:46:19.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.756+0000 7feb6166b640 1 --2- 172.21.15.76:0/1378224752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb5c159c70 0x7feb5c0ce370 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7feb5c0cf800 tx=0x7feb5802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38f6c6d2bf4da10 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:19.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.756+0000 7feb53fff640 1 -- 172.21.15.76:0/1378224752 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb58031870 con 0x7feb5c159c70 2023-11-09T03:46:19.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.756+0000 7feb53fff640 1 -- 172.21.15.76:0/1378224752 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb580319d0 con 0x7feb5c159c70 2023-11-09T03:46:19.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.756+0000 7feb53fff640 1 -- 172.21.15.76:0/1378224752 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb58031d60 con 0x7feb5c159c70 2023-11-09T03:46:19.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.756+0000 7feb638cf640 1 -- 172.21.15.76:0/1378224752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb5c159c70 msgr2=0x7feb5c0ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:19.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.756+0000 7feb638cf640 1 --2- 172.21.15.76:0/1378224752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb5c159c70 0x7feb5c0ce370 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7feb5c0cf800 tx=0x7feb5802b910 comp rx=0 tx=0).stop 2023-11-09T03:46:19.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.756+0000 7feb638cf640 1 -- 172.21.15.76:0/1378224752 shutdown_connections 2023-11-09T03:46:19.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.756+0000 7feb638cf640 1 --2- 172.21.15.76:0/1378224752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb5c0b0910 0x7feb5c15d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.756+0000 7feb638cf640 1 --2- 172.21.15.76:0/1378224752 >> v2:172.21.15.113:3300/0 conn(0x7feb5c06db20 0x7feb5c0b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.756+0000 7feb638cf640 1 --2- 172.21.15.76:0/1378224752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb5c159c70 0x7feb5c0ce370 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.756+0000 7feb638cf640 1 -- 172.21.15.76:0/1378224752 >> 172.21.15.76:0/1378224752 conn(0x7feb5c0ce950 msgr2=0x7feb5c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:19.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.757+0000 7feb638cf640 1 -- 172.21.15.76:0/1378224752 shutdown_connections 2023-11-09T03:46:19.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.757+0000 7feb638cf640 1 -- 172.21.15.76:0/1378224752 wait complete. 2023-11-09T03:46:19.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.758+0000 7feb638cf640 1 Processor -- start 2023-11-09T03:46:19.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.758+0000 7feb638cf640 1 -- start start 2023-11-09T03:46:19.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.758+0000 7feb638cf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb5c06db20 0x7feb5c0c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:19.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.758+0000 7feb638cf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb5c0b0910 0x7feb5c1554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:19.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.758+0000 7feb638cf640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb5c159c70 0x7feb5c1559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:19.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.758+0000 7feb638cf640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c1f0a80 con 0x7feb5c06db20 2023-11-09T03:46:19.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.758+0000 7feb638cf640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c156030 con 0x7feb5c0b0910 2023-11-09T03:46:19.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.758+0000 7feb638cf640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c156310 con 0x7feb5c159c70 2023-11-09T03:46:19.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.758+0000 7feb6166b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb5c06db20 0x7feb5c0c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:19.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.758+0000 7feb6166b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb5c06db20 0x7feb5c0c7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36934/0 (socket says 172.21.15.76:36934) 2023-11-09T03:46:19.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.758+0000 7feb6166b640 1 -- 172.21.15.76:0/2129620592 learned_addr learned my addr 172.21.15.76:0/2129620592 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:19.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.759+0000 7feb61e6c640 1 --2- 172.21.15.76:0/2129620592 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb5c159c70 0x7feb5c1559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:19.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.759+0000 7feb60e6a640 1 --2- 172.21.15.76:0/2129620592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb5c0b0910 0x7feb5c1554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:19.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.759+0000 7feb6166b640 1 -- 172.21.15.76:0/2129620592 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb5c159c70 msgr2=0x7feb5c1559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:19.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.759+0000 7feb6166b640 1 --2- 172.21.15.76:0/2129620592 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb5c159c70 0x7feb5c1559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.759+0000 7feb6166b640 1 -- 172.21.15.76:0/2129620592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb5c0b0910 msgr2=0x7feb5c1554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:19.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.759+0000 7feb6166b640 1 --2- 172.21.15.76:0/2129620592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb5c0b0910 0x7feb5c1554b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.759+0000 7feb6166b640 1 -- 172.21.15.76:0/2129620592 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb580094d0 con 0x7feb5c06db20 2023-11-09T03:46:19.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.759+0000 7feb6166b640 1 --2- 172.21.15.76:0/2129620592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb5c06db20 0x7feb5c0c7e10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7feb58000c00 tx=0x7feb580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:19.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.759+0000 7feb51ffb640 1 -- 172.21.15.76:0/2129620592 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb58031840 con 0x7feb5c06db20 2023-11-09T03:46:19.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.759+0000 7feb51ffb640 1 -- 172.21.15.76:0/2129620592 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb580443f0 con 0x7feb5c06db20 2023-11-09T03:46:19.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.759+0000 7feb638cf640 1 -- 172.21.15.76:0/2129620592 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb5c1565f0 con 0x7feb5c06db20 2023-11-09T03:46:19.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.759+0000 7feb51ffb640 1 -- 172.21.15.76:0/2129620592 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb580447a0 con 0x7feb5c06db20 2023-11-09T03:46:19.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.759+0000 7feb638cf640 1 -- 172.21.15.76:0/2129620592 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb5c150940 con 0x7feb5c06db20 2023-11-09T03:46:19.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.760+0000 7feb638cf640 1 -- 172.21.15.76:0/2129620592 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb5c074290 con 0x7feb5c06db20 2023-11-09T03:46:19.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.761+0000 7feb51ffb640 1 -- 172.21.15.76:0/2129620592 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feb5804d450 con 0x7feb5c06db20 2023-11-09T03:46:19.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.761+0000 7feb51ffb640 1 --2- 172.21.15.76:0/2129620592 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7feb4006ae70 0x7feb4006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:19.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.761+0000 7feb51ffb640 1 -- 172.21.15.76:0/2129620592 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7feb5802d040 con 0x7feb5c06db20 2023-11-09T03:46:19.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.761+0000 7feb60e6a640 1 --2- 172.21.15.76:0/2129620592 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7feb4006ae70 0x7feb4006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:19.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.762+0000 7feb60e6a640 1 --2- 172.21.15.76:0/2129620592 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7feb4006ae70 0x7feb4006d330 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feb5c1550b0 tx=0x7feb4c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:19.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.765+0000 7feb51ffb640 1 -- 172.21.15.76:0/2129620592 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feb580deca0 con 0x7feb5c06db20 2023-11-09T03:46:19.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.933+0000 7feb638cf640 1 -- 172.21.15.76:0/2129620592 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb5c06d3f0 con 0x7feb5c06db20 2023-11-09T03:46:19.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.934+0000 7feb51ffb640 1 -- 172.21.15.76:0/2129620592 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18710 (secure 0 0 0) 0x7feb58076e20 con 0x7feb5c06db20 2023-11-09T03:46:19.934 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:19.934 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":27,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:46:12.537265+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:46:19.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.936+0000 7feb638cf640 1 -- 172.21.15.76:0/2129620592 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7feb4006ae70 msgr2=0x7feb4006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:19.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.936+0000 7feb638cf640 1 --2- 172.21.15.76:0/2129620592 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7feb4006ae70 0x7feb4006d330 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feb5c1550b0 tx=0x7feb4c008040 comp rx=0 tx=0).stop 2023-11-09T03:46:19.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.936+0000 7feb638cf640 1 -- 172.21.15.76:0/2129620592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb5c06db20 msgr2=0x7feb5c0c7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:19.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.936+0000 7feb638cf640 1 --2- 172.21.15.76:0/2129620592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb5c06db20 0x7feb5c0c7e10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7feb58000c00 tx=0x7feb580046c0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.936+0000 7feb638cf640 1 -- 172.21.15.76:0/2129620592 shutdown_connections 2023-11-09T03:46:19.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.936+0000 7feb638cf640 1 --2- 172.21.15.76:0/2129620592 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7feb4006ae70 0x7feb4006d330 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.936+0000 7feb638cf640 1 --2- 172.21.15.76:0/2129620592 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb5c159c70 0x7feb5c1559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.936+0000 7feb638cf640 1 --2- 172.21.15.76:0/2129620592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb5c0b0910 0x7feb5c1554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.936+0000 7feb638cf640 1 --2- 172.21.15.76:0/2129620592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb5c06db20 0x7feb5c0c7e10 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:19.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.936+0000 7feb638cf640 1 -- 172.21.15.76:0/2129620592 >> 172.21.15.76:0/2129620592 conn(0x7feb5c0ce950 msgr2=0x7feb5c15a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:19.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.936+0000 7feb638cf640 1 -- 172.21.15.76:0/2129620592 shutdown_connections 2023-11-09T03:46:19.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:19.936+0000 7feb638cf640 1 -- 172.21.15.76:0/2129620592 wait complete. 2023-11-09T03:46:19.946 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:45:28.704715+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:45:29.717614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-09T03:45:33.440577+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-09T03:46:19.947 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:46:20.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.050+0000 7f74d0f6c640 1 Processor -- start 2023-11-09T03:46:20.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.050+0000 7f74d0f6c640 1 -- start start 2023-11-09T03:46:20.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.051+0000 7f74d0f6c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f74cc15ad30 0x7f74cc0adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:20.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.051+0000 7f74d0f6c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74cc0c3900 con 0x7f74cc0ad730 2023-11-09T03:46:20.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.051+0000 7f74d0f6c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74cc0c3be0 con 0x7f74cc070e20 2023-11-09T03:46:20.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.051+0000 7f74d0f6c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74cc0ae260 con 0x7f74cc15ad30 2023-11-09T03:46:20.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.051+0000 7f74ca59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f74cc15ad30 0x7f74cc0adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:20.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.051+0000 7f74c9d9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f74cc0ad730 0x7f74cc070710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43696/0 (socket says 172.21.15.76:43696) 2023-11-09T03:46:20.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.051+0000 7f74c9d9b640 1 -- 172.21.15.76:0/3291222563 learned_addr learned my addr 172.21.15.76:0/3291222563 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:20.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.052+0000 7f74ca59c640 1 -- 172.21.15.76:0/3291222563 >> v1:172.21.15.113:6789/0 conn(0x7f74cc070e20 legacy=0x7f74cc15e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:20.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.052+0000 7f74ca59c640 1 -- 172.21.15.76:0/3291222563 >> v1:172.21.15.76:6789/0 conn(0x7f74cc0ad730 legacy=0x7f74cc070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:20.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.052+0000 7f74c959a640 1 -- 172.21.15.76:0/3291222563 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3813601842 0 0) 0x7f74cc0c3900 con 0x7f74cc0ad730 2023-11-09T03:46:20.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.052+0000 7f74ca59c640 1 -- 172.21.15.76:0/3291222563 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74cc0c3ec0 con 0x7f74cc15ad30 2023-11-09T03:46:20.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.052+0000 7f74ca59c640 1 --2- 172.21.15.76:0/3291222563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74cc15ad30 0x7f74cc0adc80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f74bc009d90 tx=0x7f74bc02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a765f7b593c7c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:20.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.052+0000 7f74c959a640 1 -- 172.21.15.76:0/3291222563 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc02cc60 con 0x7f74cc15ad30 2023-11-09T03:46:20.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.053+0000 7f74c959a640 1 -- 172.21.15.76:0/3291222563 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74bc02cdc0 con 0x7f74cc15ad30 2023-11-09T03:46:20.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.053+0000 7f74c959a640 1 -- 172.21.15.76:0/3291222563 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc034660 con 0x7f74cc15ad30 2023-11-09T03:46:20.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.053+0000 7f74d0f6c640 1 -- 172.21.15.76:0/3291222563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74cc15ad30 msgr2=0x7f74cc0adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:20.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.053+0000 7f74d0f6c640 1 --2- 172.21.15.76:0/3291222563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74cc15ad30 0x7f74cc0adc80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f74bc009d90 tx=0x7f74bc02beb0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.053+0000 7f74d0f6c640 1 -- 172.21.15.76:0/3291222563 shutdown_connections 2023-11-09T03:46:20.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.053+0000 7f74d0f6c640 1 --2- 172.21.15.76:0/3291222563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74cc15ad30 0x7f74cc0adc80 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.053+0000 7f74d0f6c640 1 -- 172.21.15.76:0/3291222563 >> 172.21.15.76:0/3291222563 conn(0x7f74cc0ce950 msgr2=0x7f74cc0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:20.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.053+0000 7f74d0f6c640 1 -- 172.21.15.76:0/3291222563 shutdown_connections 2023-11-09T03:46:20.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.053+0000 7f74d0f6c640 1 -- 172.21.15.76:0/3291222563 wait complete. 2023-11-09T03:46:20.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.054+0000 7f74d0f6c640 1 Processor -- start 2023-11-09T03:46:20.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.055+0000 7f74d0f6c640 1 -- start start 2023-11-09T03:46:20.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.055+0000 7f74d0f6c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74cc070e20 0x7f74cc14aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:20.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.055+0000 7f74d0f6c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74cc0ad730 0x7f74cc156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:20.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.055+0000 7f74d0f6c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74cc15ad30 0x7f74cc1571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:20.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.055+0000 7f74d0f6c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc0b12b0 con 0x7f74cc0ad730 2023-11-09T03:46:20.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.055+0000 7f74d0f6c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc14b530 con 0x7f74cc15ad30 2023-11-09T03:46:20.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.055+0000 7f74d0f6c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc0c8220 con 0x7f74cc070e20 2023-11-09T03:46:20.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.055+0000 7f74c9d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74cc0ad730 0x7f74cc156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:20.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.055+0000 7f74c9d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74cc0ad730 0x7f74cc156c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36946/0 (socket says 172.21.15.76:36946) 2023-11-09T03:46:20.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.055+0000 7f74c9d9b640 1 -- 172.21.15.76:0/1186921813 learned_addr learned my addr 172.21.15.76:0/1186921813 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:20.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.056+0000 7f74cad9d640 1 --2- 172.21.15.76:0/1186921813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74cc15ad30 0x7f74cc1571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:20.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.056+0000 7f74ca59c640 1 --2- 172.21.15.76:0/1186921813 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74cc070e20 0x7f74cc14aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:20.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.056+0000 7f74c9d9b640 1 -- 172.21.15.76:0/1186921813 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74cc070e20 msgr2=0x7f74cc14aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:20.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.056+0000 7f74c9d9b640 1 --2- 172.21.15.76:0/1186921813 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74cc070e20 0x7f74cc14aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.056+0000 7f74c9d9b640 1 -- 172.21.15.76:0/1186921813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74cc15ad30 msgr2=0x7f74cc1571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:20.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.056+0000 7f74c9d9b640 1 --2- 172.21.15.76:0/1186921813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74cc15ad30 0x7f74cc1571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.056+0000 7f74c9d9b640 1 -- 172.21.15.76:0/1186921813 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74bc009a70 con 0x7f74cc0ad730 2023-11-09T03:46:20.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.056+0000 7f74c9d9b640 1 --2- 172.21.15.76:0/1186921813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74cc0ad730 0x7f74cc156c60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f74c000c4a0 tx=0x7f74c000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:20.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.056+0000 7f74baffd640 1 -- 172.21.15.76:0/1186921813 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c00074e0 con 0x7f74cc0ad730 2023-11-09T03:46:20.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.056+0000 7f74baffd640 1 -- 172.21.15.76:0/1186921813 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c0007640 con 0x7f74cc0ad730 2023-11-09T03:46:20.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.056+0000 7f74baffd640 1 -- 172.21.15.76:0/1186921813 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c0004a00 con 0x7f74cc0ad730 2023-11-09T03:46:20.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.056+0000 7f74d0f6c640 1 -- 172.21.15.76:0/1186921813 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74cc0b14c0 con 0x7f74cc0ad730 2023-11-09T03:46:20.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.056+0000 7f74d0f6c640 1 -- 172.21.15.76:0/1186921813 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74cc0b1a80 con 0x7f74cc0ad730 2023-11-09T03:46:20.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.057+0000 7f74d0f6c640 1 -- 172.21.15.76:0/1186921813 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74cc0b1c70 con 0x7f74cc0ad730 2023-11-09T03:46:20.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.057+0000 7f74baffd640 1 -- 172.21.15.76:0/1186921813 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f74c0004b60 con 0x7f74cc0ad730 2023-11-09T03:46:20.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.058+0000 7f74baffd640 1 --2- 172.21.15.76:0/1186921813 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f74ac06ae70 0x7f74ac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:20.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.058+0000 7f74baffd640 1 -- 172.21.15.76:0/1186921813 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f74c0088bf0 con 0x7f74cc0ad730 2023-11-09T03:46:20.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.058+0000 7f74ca59c640 1 --2- 172.21.15.76:0/1186921813 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f74ac06ae70 0x7f74ac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:20.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.058+0000 7f74ca59c640 1 --2- 172.21.15.76:0/1186921813 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f74ac06ae70 0x7f74ac06d330 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f74bc036000 tx=0x7f74bc030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:20.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.062+0000 7f74baffd640 1 -- 172.21.15.76:0/1186921813 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f74c00538c0 con 0x7f74cc0ad730 2023-11-09T03:46:20.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.228+0000 7f74d0f6c640 1 -- 172.21.15.76:0/1186921813 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f74cc074290 con 0x7f74cc0ad730 2023-11-09T03:46:20.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.228+0000 7f74baffd640 1 -- 172.21.15.76:0/1186921813 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f74c0018020 con 0x7f74cc0ad730 2023-11-09T03:46:20.227 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:46:20.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.230+0000 7f74d0f6c640 1 -- 172.21.15.76:0/1186921813 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f74ac06ae70 msgr2=0x7f74ac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:20.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.230+0000 7f74d0f6c640 1 --2- 172.21.15.76:0/1186921813 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f74ac06ae70 0x7f74ac06d330 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f74bc036000 tx=0x7f74bc030000 comp rx=0 tx=0).stop 2023-11-09T03:46:20.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.230+0000 7f74d0f6c640 1 -- 172.21.15.76:0/1186921813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74cc0ad730 msgr2=0x7f74cc156c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:20.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.230+0000 7f74d0f6c640 1 --2- 172.21.15.76:0/1186921813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74cc0ad730 0x7f74cc156c60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f74c000c4a0 tx=0x7f74c000c960 comp rx=0 tx=0).stop 2023-11-09T03:46:20.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.230+0000 7f74d0f6c640 1 -- 172.21.15.76:0/1186921813 shutdown_connections 2023-11-09T03:46:20.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.230+0000 7f74d0f6c640 1 --2- 172.21.15.76:0/1186921813 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f74ac06ae70 0x7f74ac06d330 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.230+0000 7f74d0f6c640 1 --2- 172.21.15.76:0/1186921813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74cc15ad30 0x7f74cc1571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.230+0000 7f74d0f6c640 1 --2- 172.21.15.76:0/1186921813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74cc0ad730 0x7f74cc156c60 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.230+0000 7f74d0f6c640 1 --2- 172.21.15.76:0/1186921813 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74cc070e20 0x7f74cc14aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.230+0000 7f74d0f6c640 1 -- 172.21.15.76:0/1186921813 >> 172.21.15.76:0/1186921813 conn(0x7f74cc0ce950 msgr2=0x7f74cc159f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:20.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.230+0000 7f74d0f6c640 1 -- 172.21.15.76:0/1186921813 shutdown_connections 2023-11-09T03:46:20.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.230+0000 7f74d0f6c640 1 -- 172.21.15.76:0/1186921813 wait complete. 2023-11-09T03:46:20.240 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:46:20.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.343+0000 7ffb3bad4640 1 Processor -- start 2023-11-09T03:46:20.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.343+0000 7ffb3bad4640 1 -- start start 2023-11-09T03:46:20.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.344+0000 7ffb3bad4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffb340b2d30 0x7ffb3406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:20.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.344+0000 7ffb3bad4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb3406c1d0 0x7ffb340cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:20.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.344+0000 7ffb3bad4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb3406ca40 con 0x7ffb34160430 2023-11-09T03:46:20.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.344+0000 7ffb3bad4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb340cb580 con 0x7ffb340b2d30 2023-11-09T03:46:20.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.344+0000 7ffb3bad4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3406cd20 con 0x7ffb3406c1d0 2023-11-09T03:46:20.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.345+0000 7ffb3906f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffb340b2d30 0x7ffb3406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:20.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.345+0000 7ffb3a071640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb3406c1d0 0x7ffb340cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:20.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.345+0000 7ffb39870640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ffb34160430 0x7ffb340b2620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:60126/0 (socket says 172.21.15.76:60126) 2023-11-09T03:46:20.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.345+0000 7ffb39870640 1 -- 172.21.15.76:0/456385980 learned_addr learned my addr 172.21.15.76:0/456385980 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:20.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.345+0000 7ffb3906f640 1 -- 172.21.15.76:0/456385980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb3406c1d0 msgr2=0x7ffb340cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:20.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.345+0000 7ffb3906f640 1 --2- 172.21.15.76:0/456385980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb3406c1d0 0x7ffb340cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.345+0000 7ffb3906f640 1 -- 172.21.15.76:0/456385980 >> v1:172.21.15.113:6789/0 conn(0x7ffb34160430 legacy=0x7ffb340b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:20.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.345+0000 7ffb3886e640 1 -- 172.21.15.76:0/456385980 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 132997343 0 0) 0x7ffb3406ca40 con 0x7ffb34160430 2023-11-09T03:46:20.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.345+0000 7ffb3906f640 1 -- 172.21.15.76:0/456385980 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3406d000 con 0x7ffb340b2d30 2023-11-09T03:46:20.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.345+0000 7ffb3a071640 1 --2- 172.21.15.76:0/456385980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb3406c1d0 0x7ffb340cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:46:20.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.346+0000 7ffb3906f640 1 --2- 172.21.15.76:0/456385980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb340b2d30 0x7ffb3406bc90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ffb280097a0 tx=0x7ffb2802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b18925987fe4aae1 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:20.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.346+0000 7ffb3886e640 1 -- 172.21.15.76:0/456385980 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb28031830 con 0x7ffb340b2d30 2023-11-09T03:46:20.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.346+0000 7ffb3886e640 1 -- 172.21.15.76:0/456385980 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb28031990 con 0x7ffb340b2d30 2023-11-09T03:46:20.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.346+0000 7ffb3886e640 1 -- 172.21.15.76:0/456385980 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb28031d20 con 0x7ffb340b2d30 2023-11-09T03:46:20.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.346+0000 7ffb3bad4640 1 -- 172.21.15.76:0/456385980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb340b2d30 msgr2=0x7ffb3406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:20.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.346+0000 7ffb3bad4640 1 --2- 172.21.15.76:0/456385980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb340b2d30 0x7ffb3406bc90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ffb280097a0 tx=0x7ffb2802b8d0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.347+0000 7ffb3bad4640 1 -- 172.21.15.76:0/456385980 shutdown_connections 2023-11-09T03:46:20.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.347+0000 7ffb3bad4640 1 --2- 172.21.15.76:0/456385980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb3406c1d0 0x7ffb340cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.347+0000 7ffb3bad4640 1 --2- 172.21.15.76:0/456385980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb340b2d30 0x7ffb3406bc90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.347+0000 7ffb3bad4640 1 -- 172.21.15.76:0/456385980 >> 172.21.15.76:0/456385980 conn(0x7ffb3406ea20 msgr2=0x7ffb3406ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:20.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.347+0000 7ffb3bad4640 1 -- 172.21.15.76:0/456385980 shutdown_connections 2023-11-09T03:46:20.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.347+0000 7ffb3bad4640 1 -- 172.21.15.76:0/456385980 wait complete. 2023-11-09T03:46:20.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.348+0000 7ffb3bad4640 1 Processor -- start 2023-11-09T03:46:20.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.348+0000 7ffb3bad4640 1 -- start start 2023-11-09T03:46:20.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.348+0000 7ffb3bad4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb3406c1d0 0x7ffb34151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:20.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.349+0000 7ffb3bad4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb340b2d30 0x7ffb340b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:20.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.349+0000 7ffb3bad4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb34160430 0x7ffb340b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:20.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.349+0000 7ffb3bad4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb340b5590 con 0x7ffb340b2d30 2023-11-09T03:46:20.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.349+0000 7ffb3bad4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb340b4840 con 0x7ffb34160430 2023-11-09T03:46:20.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.349+0000 7ffb3bad4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb340b4280 con 0x7ffb3406c1d0 2023-11-09T03:46:20.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.349+0000 7ffb3906f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb340b2d30 0x7ffb340b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:20.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.349+0000 7ffb3906f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb340b2d30 0x7ffb340b5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36956/0 (socket says 172.21.15.76:36956) 2023-11-09T03:46:20.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.349+0000 7ffb3906f640 1 -- 172.21.15.76:0/1423240696 learned_addr learned my addr 172.21.15.76:0/1423240696 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:20.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.349+0000 7ffb39870640 1 --2- 172.21.15.76:0/1423240696 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb3406c1d0 0x7ffb34151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:20.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.349+0000 7ffb3a071640 1 --2- 172.21.15.76:0/1423240696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb34160430 0x7ffb340b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:20.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.349+0000 7ffb3906f640 1 -- 172.21.15.76:0/1423240696 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb3406c1d0 msgr2=0x7ffb34151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:20.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.349+0000 7ffb3906f640 1 --2- 172.21.15.76:0/1423240696 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb3406c1d0 0x7ffb34151fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.349+0000 7ffb3906f640 1 -- 172.21.15.76:0/1423240696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb34160430 msgr2=0x7ffb340b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:20.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.349+0000 7ffb3906f640 1 --2- 172.21.15.76:0/1423240696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb34160430 0x7ffb340b37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.349+0000 7ffb3906f640 1 -- 172.21.15.76:0/1423240696 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2c009400 con 0x7ffb340b2d30 2023-11-09T03:46:20.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.349+0000 7ffb3906f640 1 --2- 172.21.15.76:0/1423240696 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb340b2d30 0x7ffb340b5a20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ffb28000c00 tx=0x7ffb280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:20.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.349+0000 7ffb327fc640 1 -- 172.21.15.76:0/1423240696 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb28004230 con 0x7ffb340b2d30 2023-11-09T03:46:20.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.349+0000 7ffb327fc640 1 -- 172.21.15.76:0/1423240696 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb28004390 con 0x7ffb340b2d30 2023-11-09T03:46:20.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.349+0000 7ffb3bad4640 1 -- 172.21.15.76:0/1423240696 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb28009400 con 0x7ffb340b2d30 2023-11-09T03:46:20.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.349+0000 7ffb327fc640 1 -- 172.21.15.76:0/1423240696 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2803a880 con 0x7ffb340b2d30 2023-11-09T03:46:20.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.349+0000 7ffb3bad4640 1 -- 172.21.15.76:0/1423240696 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb340b5220 con 0x7ffb340b2d30 2023-11-09T03:46:20.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.350+0000 7ffb3bad4640 1 -- 172.21.15.76:0/1423240696 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb340b5740 con 0x7ffb340b2d30 2023-11-09T03:46:20.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.351+0000 7ffb327fc640 1 -- 172.21.15.76:0/1423240696 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffb2803a9e0 con 0x7ffb340b2d30 2023-11-09T03:46:20.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.351+0000 7ffb327fc640 1 --2- 172.21.15.76:0/1423240696 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ffb2006ae70 0x7ffb2006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:20.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.351+0000 7ffb327fc640 1 -- 172.21.15.76:0/1423240696 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7ffb280ac310 con 0x7ffb340b2d30 2023-11-09T03:46:20.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.351+0000 7ffb39870640 1 --2- 172.21.15.76:0/1423240696 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ffb2006ae70 0x7ffb2006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:20.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.352+0000 7ffb39870640 1 --2- 172.21.15.76:0/1423240696 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ffb2006ae70 0x7ffb2006d330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffb240012e0 tx=0x7ffb24005eb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:20.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.355+0000 7ffb327fc640 1 -- 172.21.15.76:0/1423240696 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffb340b5740 con 0x7ffb340b2d30 2023-11-09T03:46:20.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.522+0000 7ffb3bad4640 1 -- 172.21.15.76:0/1423240696 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffb34074290 con 0x7ffb340b2d30 2023-11-09T03:46:20.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.522+0000 7ffb327fc640 1 -- 172.21.15.76:0/1423240696 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffb28077190 con 0x7ffb340b2d30 2023-11-09T03:46:20.522 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:46:20.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.524+0000 7ffb3bad4640 1 -- 172.21.15.76:0/1423240696 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ffb2006ae70 msgr2=0x7ffb2006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:20.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.524+0000 7ffb3bad4640 1 --2- 172.21.15.76:0/1423240696 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ffb2006ae70 0x7ffb2006d330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffb240012e0 tx=0x7ffb24005eb0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.524+0000 7ffb3bad4640 1 -- 172.21.15.76:0/1423240696 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb340b2d30 msgr2=0x7ffb340b5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:20.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.524+0000 7ffb3bad4640 1 --2- 172.21.15.76:0/1423240696 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb340b2d30 0x7ffb340b5a20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ffb28000c00 tx=0x7ffb280046c0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.524+0000 7ffb3bad4640 1 -- 172.21.15.76:0/1423240696 shutdown_connections 2023-11-09T03:46:20.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.524+0000 7ffb3bad4640 1 --2- 172.21.15.76:0/1423240696 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ffb2006ae70 0x7ffb2006d330 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.524+0000 7ffb3bad4640 1 --2- 172.21.15.76:0/1423240696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb34160430 0x7ffb340b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.524+0000 7ffb3bad4640 1 --2- 172.21.15.76:0/1423240696 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb340b2d30 0x7ffb340b5a20 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.524+0000 7ffb3bad4640 1 --2- 172.21.15.76:0/1423240696 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb3406c1d0 0x7ffb34151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.524+0000 7ffb3bad4640 1 -- 172.21.15.76:0/1423240696 >> 172.21.15.76:0/1423240696 conn(0x7ffb3406ea20 msgr2=0x7ffb340cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:20.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.524+0000 7ffb3bad4640 1 -- 172.21.15.76:0/1423240696 shutdown_connections 2023-11-09T03:46:20.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.524+0000 7ffb3bad4640 1 -- 172.21.15.76:0/1423240696 wait complete. 2023-11-09T03:46:20.534 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:46:20.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.639+0000 7f8b956a9640 1 Processor -- start 2023-11-09T03:46:20.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.639+0000 7f8b956a9640 1 -- start start 2023-11-09T03:46:20.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.640+0000 7f8b956a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b900cd6d0 0x7f8b90161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:20.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.640+0000 7f8b956a9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b900cde60 con 0x7f8b900b4880 2023-11-09T03:46:20.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.640+0000 7f8b956a9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b90160f90 con 0x7f8b90153e60 2023-11-09T03:46:20.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.640+0000 7f8b956a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b90161960 con 0x7f8b900cd6d0 2023-11-09T03:46:20.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.640+0000 7f8b8f7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8b900b4880 0x7f8b900b4ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43700/0 (socket says 172.21.15.76:43700) 2023-11-09T03:46:20.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.640+0000 7f8b8f7fe640 1 -- 172.21.15.76:0/2679171614 learned_addr learned my addr 172.21.15.76:0/2679171614 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:20.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.640+0000 7f8b8e7fc640 1 --2- 172.21.15.76:0/2679171614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b900cd6d0 0x7f8b90161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:20.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.640+0000 7f8b8dffb640 1 -- 172.21.15.76:0/2679171614 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 754085562 0 0) 0x7f8b900cde60 con 0x7f8b900b4880 2023-11-09T03:46:20.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.640+0000 7f8b8dffb640 1 -- 172.21.15.76:0/2679171614 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b78003520 con 0x7f8b900b4880 2023-11-09T03:46:20.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.640+0000 7f8b8dffb640 1 -- 172.21.15.76:0/2679171614 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2862139845 0 0) 0x7f8b78003520 con 0x7f8b900b4880 2023-11-09T03:46:20.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.641+0000 7f8b8dffb640 1 -- 172.21.15.76:0/2679171614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b900cd6d0 msgr2=0x7f8b90161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:20.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.641+0000 7f8b8dffb640 1 --2- 172.21.15.76:0/2679171614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b900cd6d0 0x7f8b90161420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.641+0000 7f8b8dffb640 1 -- 172.21.15.76:0/2679171614 >> v1:172.21.15.113:6789/0 conn(0x7f8b90153e60 legacy=0x7f8b9015dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:20.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.641+0000 7f8b8dffb640 1 -- 172.21.15.76:0/2679171614 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b900ce090 con 0x7f8b900b4880 2023-11-09T03:46:20.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.641+0000 7f8b8dffb640 1 -- 172.21.15.76:0/2679171614 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f8b880019f0 con 0x7f8b900b4880 2023-11-09T03:46:20.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.641+0000 7f8b8dffb640 1 -- 172.21.15.76:0/2679171614 >> v1:172.21.15.76:6789/0 conn(0x7f8b900b4880 legacy=0x7f8b900b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:20.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.641+0000 7f8b8dffb640 1 --2- 172.21.15.76:0/2679171614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b900cd6d0 0x7f8b780039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:20.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.641+0000 7f8b8dffb640 1 --2- 172.21.15.76:0/2679171614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b78003ee0 0x7f8b780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:20.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.641+0000 7f8b8dffb640 1 --2- 172.21.15.76:0/2679171614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b78004d00 0x7f8b78005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:20.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.641+0000 7f8b8dffb640 1 -- 172.21.15.76:0/2679171614 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b90161960 con 0x7f8b78004d00 2023-11-09T03:46:20.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.641+0000 7f8b8dffb640 1 -- 172.21.15.76:0/2679171614 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7800b9e0 con 0x7f8b78003ee0 2023-11-09T03:46:20.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.641+0000 7f8b8dffb640 1 -- 172.21.15.76:0/2679171614 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7800bba0 con 0x7f8b900cd6d0 2023-11-09T03:46:20.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.641+0000 7f8b8f7fe640 1 --2- 172.21.15.76:0/2679171614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b78004d00 0x7f8b78005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:20.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.641+0000 7f8b8effd640 1 --2- 172.21.15.76:0/2679171614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b900cd6d0 0x7f8b780039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:20.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.641+0000 7f8b8e7fc640 1 --2- 172.21.15.76:0/2679171614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b78003ee0 0x7f8b780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:20.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.641+0000 7f8b8f7fe640 1 -- 172.21.15.76:0/2679171614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b900cd6d0 msgr2=0x7f8b780039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:20.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.641+0000 7f8b8f7fe640 1 --2- 172.21.15.76:0/2679171614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b900cd6d0 0x7f8b780039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.641+0000 7f8b8f7fe640 1 -- 172.21.15.76:0/2679171614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b78003ee0 msgr2=0x7f8b780047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:20.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.641+0000 7f8b8f7fe640 1 --2- 172.21.15.76:0/2679171614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b78003ee0 0x7f8b780047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.641+0000 7f8b8f7fe640 1 -- 172.21.15.76:0/2679171614 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b78003520 con 0x7f8b78004d00 2023-11-09T03:46:20.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.642+0000 7f8b8f7fe640 1 --2- 172.21.15.76:0/2679171614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b78004d00 0x7f8b78005180 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800a840 tx=0x7f8b780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e52d952e92a14d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:20.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.642+0000 7f8b8dffb640 1 -- 172.21.15.76:0/2679171614 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8802cea0 con 0x7f8b78004d00 2023-11-09T03:46:20.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.642+0000 7f8b8dffb640 1 -- 172.21.15.76:0/2679171614 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8802f070 con 0x7f8b78004d00 2023-11-09T03:46:20.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.642+0000 7f8b8dffb640 1 -- 172.21.15.76:0/2679171614 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b880348b0 con 0x7f8b78004d00 2023-11-09T03:46:20.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.642+0000 7f8b956a9640 1 -- 172.21.15.76:0/2679171614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b78004d00 msgr2=0x7f8b78005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:20.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.642+0000 7f8b956a9640 1 --2- 172.21.15.76:0/2679171614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b78004d00 0x7f8b78005180 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800a840 tx=0x7f8b780057f0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.642+0000 7f8b8f7fe640 1 -- 172.21.15.76:0/2679171614 reap_dead start 2023-11-09T03:46:20.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.642+0000 7f8b956a9640 1 -- 172.21.15.76:0/2679171614 shutdown_connections 2023-11-09T03:46:20.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.642+0000 7f8b956a9640 1 -- 172.21.15.76:0/2679171614 >> 172.21.15.76:0/2679171614 conn(0x7f8b900cf880 msgr2=0x7f8b900cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:20.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.642+0000 7f8b956a9640 1 -- 172.21.15.76:0/2679171614 shutdown_connections 2023-11-09T03:46:20.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.642+0000 7f8b956a9640 1 -- 172.21.15.76:0/2679171614 wait complete. 2023-11-09T03:46:20.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.644+0000 7f8b956a9640 1 Processor -- start 2023-11-09T03:46:20.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.644+0000 7f8b956a9640 1 -- start start 2023-11-09T03:46:20.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.644+0000 7f8b956a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b9015a5d0 0x7f8b900cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:20.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.644+0000 7f8b956a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b90148470 0x7f8b9015bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:20.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.644+0000 7f8b956a9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b9015c0d0 0x7f8b901f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:20.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.644+0000 7f8b956a9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9015c580 con 0x7f8b9015c0d0 2023-11-09T03:46:20.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.644+0000 7f8b956a9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b90146620 con 0x7f8b90148470 2023-11-09T03:46:20.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.645+0000 7f8b956a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b900cbe70 con 0x7f8b9015a5d0 2023-11-09T03:46:20.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.645+0000 7f8b8f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b9015c0d0 0x7f8b901f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:20.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.645+0000 7f8b8f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b9015c0d0 0x7f8b901f5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36970/0 (socket says 172.21.15.76:36970) 2023-11-09T03:46:20.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.645+0000 7f8b8f7fe640 1 -- 172.21.15.76:0/597381341 learned_addr learned my addr 172.21.15.76:0/597381341 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:20.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.645+0000 7f8b8effd640 1 --2- 172.21.15.76:0/597381341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b9015a5d0 0x7f8b900cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:20.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.645+0000 7f8b8e7fc640 1 --2- 172.21.15.76:0/597381341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b90148470 0x7f8b9015bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:20.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.645+0000 7f8b8f7fe640 1 -- 172.21.15.76:0/597381341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b9015a5d0 msgr2=0x7f8b900cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:20.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.645+0000 7f8b8f7fe640 1 --2- 172.21.15.76:0/597381341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b9015a5d0 0x7f8b900cb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.645+0000 7f8b8f7fe640 1 -- 172.21.15.76:0/597381341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b90148470 msgr2=0x7f8b9015bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:20.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.645+0000 7f8b8f7fe640 1 --2- 172.21.15.76:0/597381341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b90148470 0x7f8b9015bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.645+0000 7f8b8f7fe640 1 -- 172.21.15.76:0/597381341 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b9015a9b0 con 0x7f8b9015c0d0 2023-11-09T03:46:20.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.645+0000 7f8b8f7fe640 1 --2- 172.21.15.76:0/597381341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b9015c0d0 0x7f8b901f5c70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800a840 tx=0x7f8b90160f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:20.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.645+0000 7f8b777fe640 1 -- 172.21.15.76:0/597381341 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b88034840 con 0x7f8b9015c0d0 2023-11-09T03:46:20.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.645+0000 7f8b777fe640 1 -- 172.21.15.76:0/597381341 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b880349a0 con 0x7f8b9015c0d0 2023-11-09T03:46:20.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.645+0000 7f8b956a9640 1 -- 172.21.15.76:0/597381341 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b900cc0d0 con 0x7f8b9015c0d0 2023-11-09T03:46:20.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.645+0000 7f8b777fe640 1 -- 172.21.15.76:0/597381341 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b88034d10 con 0x7f8b9015c0d0 2023-11-09T03:46:20.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.645+0000 7f8b956a9640 1 -- 172.21.15.76:0/597381341 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b901462b0 con 0x7f8b9015c0d0 2023-11-09T03:46:20.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.646+0000 7f8b956a9640 1 -- 172.21.15.76:0/597381341 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b90145e30 con 0x7f8b9015c0d0 2023-11-09T03:46:20.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.647+0000 7f8b777fe640 1 -- 172.21.15.76:0/597381341 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8b88031030 con 0x7f8b9015c0d0 2023-11-09T03:46:20.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.647+0000 7f8b777fe640 1 --2- 172.21.15.76:0/597381341 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8b7c06aea0 0x7f8b7c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:20.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.647+0000 7f8b777fe640 1 -- 172.21.15.76:0/597381341 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f8b880ab3e0 con 0x7f8b9015c0d0 2023-11-09T03:46:20.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.647+0000 7f8b8effd640 1 --2- 172.21.15.76:0/597381341 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8b7c06aea0 0x7f8b7c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:20.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.648+0000 7f8b8effd640 1 --2- 172.21.15.76:0/597381341 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8b7c06aea0 0x7f8b7c06d360 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8b80001910 tx=0x7f8b80009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:20.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.651+0000 7f8b777fe640 1 -- 172.21.15.76:0/597381341 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8b90145e30 con 0x7f8b9015c0d0 2023-11-09T03:46:20.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.819+0000 7f8b956a9640 1 -- 172.21.15.76:0/597381341 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8b901467d0 con 0x7f8b9015c0d0 2023-11-09T03:46:20.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.819+0000 7f8b777fe640 1 -- 172.21.15.76:0/597381341 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8b901467d0 con 0x7f8b9015c0d0 2023-11-09T03:46:20.819 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:46:20.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.821+0000 7f8b956a9640 1 -- 172.21.15.76:0/597381341 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8b7c06aea0 msgr2=0x7f8b7c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:20.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.821+0000 7f8b956a9640 1 --2- 172.21.15.76:0/597381341 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8b7c06aea0 0x7f8b7c06d360 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8b80001910 tx=0x7f8b80009040 comp rx=0 tx=0).stop 2023-11-09T03:46:20.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.821+0000 7f8b956a9640 1 -- 172.21.15.76:0/597381341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b9015c0d0 msgr2=0x7f8b901f5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:20.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.821+0000 7f8b956a9640 1 --2- 172.21.15.76:0/597381341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b9015c0d0 0x7f8b901f5c70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800a840 tx=0x7f8b90160f00 comp rx=0 tx=0).stop 2023-11-09T03:46:20.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.821+0000 7f8b956a9640 1 -- 172.21.15.76:0/597381341 shutdown_connections 2023-11-09T03:46:20.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.821+0000 7f8b956a9640 1 --2- 172.21.15.76:0/597381341 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8b7c06aea0 0x7f8b7c06d360 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.821+0000 7f8b956a9640 1 --2- 172.21.15.76:0/597381341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b9015c0d0 0x7f8b901f5c70 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.821+0000 7f8b956a9640 1 --2- 172.21.15.76:0/597381341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b90148470 0x7f8b9015bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.821+0000 7f8b956a9640 1 --2- 172.21.15.76:0/597381341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b9015a5d0 0x7f8b900cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.821+0000 7f8b956a9640 1 -- 172.21.15.76:0/597381341 >> 172.21.15.76:0/597381341 conn(0x7f8b900cf880 msgr2=0x7f8b900d2490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:20.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.821+0000 7f8b956a9640 1 -- 172.21.15.76:0/597381341 shutdown_connections 2023-11-09T03:46:20.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.821+0000 7f8b956a9640 1 -- 172.21.15.76:0/597381341 wait complete. 2023-11-09T03:46:20.831 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-09T03:46:20.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.934+0000 7ffb60539640 1 Processor -- start 2023-11-09T03:46:20.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.934+0000 7ffb60539640 1 -- start start 2023-11-09T03:46:20.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.935+0000 7ffb60539640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffb580c4ff0 0x7ffb580cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:20.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.935+0000 7ffb60539640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffb580adac0 0x7ffb580adf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:20.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.935+0000 7ffb60539640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb5815aff0 con 0x7ffb58159c00 2023-11-09T03:46:20.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.935+0000 7ffb60539640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb580ae460 con 0x7ffb580adac0 2023-11-09T03:46:20.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.935+0000 7ffb60539640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb5815a3e0 con 0x7ffb580c4ff0 2023-11-09T03:46:20.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.935+0000 7ffb5ead6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffb580adac0 0x7ffb580adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:20.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.935+0000 7ffb5e2d5640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ffb58159c00 0x7ffb580c48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43704/0 (socket says 172.21.15.76:43704) 2023-11-09T03:46:20.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.935+0000 7ffb5e2d5640 1 -- 172.21.15.76:0/1533238927 learned_addr learned my addr 172.21.15.76:0/1533238927 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:20.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.935+0000 7ffb5ead6640 1 -- 172.21.15.76:0/1533238927 >> v2:172.21.15.113:3300/0 conn(0x7ffb580c4ff0 msgr2=0x7ffb580cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:20.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.935+0000 7ffb5ead6640 1 --2- 172.21.15.76:0/1533238927 >> v2:172.21.15.113:3300/0 conn(0x7ffb580c4ff0 0x7ffb580cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.935+0000 7ffb5ead6640 1 -- 172.21.15.76:0/1533238927 >> v1:172.21.15.76:6789/0 conn(0x7ffb58159c00 legacy=0x7ffb580c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:20.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.935+0000 7ffb5ead6640 1 -- 172.21.15.76:0/1533238927 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5815a6c0 con 0x7ffb580adac0 2023-11-09T03:46:20.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.935+0000 7ffb5d2d3640 1 -- 172.21.15.76:0/1533238927 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3224331500 0 0) 0x7ffb5815aff0 con 0x7ffb58159c00 2023-11-09T03:46:20.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.936+0000 7ffb5ead6640 1 --2- 172.21.15.76:0/1533238927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb580adac0 0x7ffb580adf20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ffb580cde50 tx=0x7ffb5002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a302d83395b329 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:20.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.936+0000 7ffb5d2d3640 1 -- 172.21.15.76:0/1533238927 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5002cc90 con 0x7ffb580adac0 2023-11-09T03:46:20.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.936+0000 7ffb5d2d3640 1 -- 172.21.15.76:0/1533238927 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5002cdf0 con 0x7ffb580adac0 2023-11-09T03:46:20.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.936+0000 7ffb5d2d3640 1 -- 172.21.15.76:0/1533238927 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb50034660 con 0x7ffb580adac0 2023-11-09T03:46:20.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.936+0000 7ffb60539640 1 -- 172.21.15.76:0/1533238927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb580adac0 msgr2=0x7ffb580adf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:20.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.936+0000 7ffb60539640 1 --2- 172.21.15.76:0/1533238927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb580adac0 0x7ffb580adf20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ffb580cde50 tx=0x7ffb5002bee0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.937+0000 7ffb60539640 1 -- 172.21.15.76:0/1533238927 shutdown_connections 2023-11-09T03:46:20.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.937+0000 7ffb60539640 1 --2- 172.21.15.76:0/1533238927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb580adac0 0x7ffb580adf20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.937+0000 7ffb60539640 1 --2- 172.21.15.76:0/1533238927 >> v2:172.21.15.113:3300/0 conn(0x7ffb580c4ff0 0x7ffb580cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.937+0000 7ffb60539640 1 -- 172.21.15.76:0/1533238927 >> 172.21.15.76:0/1533238927 conn(0x7ffb580ce950 msgr2=0x7ffb580ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:20.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.937+0000 7ffb60539640 1 -- 172.21.15.76:0/1533238927 shutdown_connections 2023-11-09T03:46:20.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.937+0000 7ffb60539640 1 -- 172.21.15.76:0/1533238927 wait complete. 2023-11-09T03:46:20.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.938+0000 7ffb60539640 1 Processor -- start 2023-11-09T03:46:20.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.938+0000 7ffb60539640 1 -- start start 2023-11-09T03:46:20.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.938+0000 7ffb60539640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb580adac0 0x7ffb58141ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:20.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.938+0000 7ffb60539640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb580c4ff0 0x7ffb580c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:20.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.938+0000 7ffb60539640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb58159c00 0x7ffb580c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:20.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.938+0000 7ffb60539640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb580a7870 con 0x7ffb580c4ff0 2023-11-09T03:46:20.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.938+0000 7ffb60539640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb58143f50 con 0x7ffb580adac0 2023-11-09T03:46:20.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.938+0000 7ffb60539640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb58142410 con 0x7ffb58159c00 2023-11-09T03:46:20.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.939+0000 7ffb5dad4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb580c4ff0 0x7ffb580c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:20.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.939+0000 7ffb5dad4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb580c4ff0 0x7ffb580c7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36990/0 (socket says 172.21.15.76:36990) 2023-11-09T03:46:20.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.939+0000 7ffb5dad4640 1 -- 172.21.15.76:0/3934806115 learned_addr learned my addr 172.21.15.76:0/3934806115 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:20.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.939+0000 7ffb5e2d5640 1 --2- 172.21.15.76:0/3934806115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb580adac0 0x7ffb58141ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:20.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.939+0000 7ffb5ead6640 1 --2- 172.21.15.76:0/3934806115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb58159c00 0x7ffb580c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:20.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.939+0000 7ffb5dad4640 1 -- 172.21.15.76:0/3934806115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb58159c00 msgr2=0x7ffb580c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:20.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.939+0000 7ffb5dad4640 1 --2- 172.21.15.76:0/3934806115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb58159c00 0x7ffb580c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.939+0000 7ffb5dad4640 1 -- 172.21.15.76:0/3934806115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb580adac0 msgr2=0x7ffb58141ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:20.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.939+0000 7ffb5dad4640 1 --2- 172.21.15.76:0/3934806115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb580adac0 0x7ffb58141ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:20.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.939+0000 7ffb5dad4640 1 -- 172.21.15.76:0/3934806115 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb50009a70 con 0x7ffb580c4ff0 2023-11-09T03:46:20.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.939+0000 7ffb5e2d5640 1 --2- 172.21.15.76:0/3934806115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb580adac0 0x7ffb58141ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:20.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.939+0000 7ffb5ead6640 1 --2- 172.21.15.76:0/3934806115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb58159c00 0x7ffb580c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:20.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.939+0000 7ffb5dad4640 1 --2- 172.21.15.76:0/3934806115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb580c4ff0 0x7ffb580c7c30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ffb4800b430 tx=0x7ffb4800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:20.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.939+0000 7ffb4effd640 1 -- 172.21.15.76:0/3934806115 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb48011830 con 0x7ffb580c4ff0 2023-11-09T03:46:20.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.939+0000 7ffb4effd640 1 -- 172.21.15.76:0/3934806115 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb48011990 con 0x7ffb580c4ff0 2023-11-09T03:46:20.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.939+0000 7ffb60539640 1 -- 172.21.15.76:0/3934806115 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb58143340 con 0x7ffb580c4ff0 2023-11-09T03:46:20.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.939+0000 7ffb4effd640 1 -- 172.21.15.76:0/3934806115 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb48011d20 con 0x7ffb580c4ff0 2023-11-09T03:46:20.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.939+0000 7ffb60539640 1 -- 172.21.15.76:0/3934806115 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb58143900 con 0x7ffb580c4ff0 2023-11-09T03:46:20.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.940+0000 7ffb60539640 1 -- 172.21.15.76:0/3934806115 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb58144100 con 0x7ffb580c4ff0 2023-11-09T03:46:20.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.941+0000 7ffb4effd640 1 -- 172.21.15.76:0/3934806115 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffb4801a400 con 0x7ffb580c4ff0 2023-11-09T03:46:20.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.941+0000 7ffb4effd640 1 --2- 172.21.15.76:0/3934806115 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ffb3c06aea0 0x7ffb3c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:20.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.941+0000 7ffb4effd640 1 -- 172.21.15.76:0/3934806115 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7ffb48089b00 con 0x7ffb580c4ff0 2023-11-09T03:46:20.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.941+0000 7ffb5e2d5640 1 --2- 172.21.15.76:0/3934806115 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ffb3c06aea0 0x7ffb3c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:20.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.942+0000 7ffb5e2d5640 1 --2- 172.21.15.76:0/3934806115 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ffb3c06aea0 0x7ffb3c06d360 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ffb54000f60 tx=0x7ffb54009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:20.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:20.945+0000 7ffb4effd640 1 -- 172.21.15.76:0/3934806115 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffb480547d0 con 0x7ffb580c4ff0 2023-11-09T03:46:21.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.113+0000 7ffb60539640 1 -- 172.21.15.76:0/3934806115 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ffb58074290 con 0x7ffb580c4ff0 2023-11-09T03:46:21.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.113+0000 7ffb4effd640 1 -- 172.21.15.76:0/3934806115 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7ffb48057df0 con 0x7ffb580c4ff0 2023-11-09T03:46:21.113 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:46:21.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.115+0000 7ffb60539640 1 -- 172.21.15.76:0/3934806115 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ffb3c06aea0 msgr2=0x7ffb3c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:21.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.115+0000 7ffb60539640 1 --2- 172.21.15.76:0/3934806115 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ffb3c06aea0 0x7ffb3c06d360 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ffb54000f60 tx=0x7ffb54009040 comp rx=0 tx=0).stop 2023-11-09T03:46:21.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.115+0000 7ffb60539640 1 -- 172.21.15.76:0/3934806115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb580c4ff0 msgr2=0x7ffb580c7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:21.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.115+0000 7ffb60539640 1 --2- 172.21.15.76:0/3934806115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb580c4ff0 0x7ffb580c7c30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ffb4800b430 tx=0x7ffb4800b8f0 comp rx=0 tx=0).stop 2023-11-09T03:46:21.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.115+0000 7ffb60539640 1 -- 172.21.15.76:0/3934806115 shutdown_connections 2023-11-09T03:46:21.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.115+0000 7ffb60539640 1 --2- 172.21.15.76:0/3934806115 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ffb3c06aea0 0x7ffb3c06d360 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:21.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.115+0000 7ffb60539640 1 --2- 172.21.15.76:0/3934806115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb58159c00 0x7ffb580c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:21.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.115+0000 7ffb60539640 1 --2- 172.21.15.76:0/3934806115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb580c4ff0 0x7ffb580c7c30 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:21.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.115+0000 7ffb60539640 1 --2- 172.21.15.76:0/3934806115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb580adac0 0x7ffb58141ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:21.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.115+0000 7ffb60539640 1 -- 172.21.15.76:0/3934806115 >> 172.21.15.76:0/3934806115 conn(0x7ffb580ce950 msgr2=0x7ffb580709f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:21.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.115+0000 7ffb60539640 1 -- 172.21.15.76:0/3934806115 shutdown_connections 2023-11-09T03:46:21.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.115+0000 7ffb60539640 1 -- 172.21.15.76:0/3934806115 wait complete. 2023-11-09T03:46:21.125 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_attaching_pools_with_snaps_to_fs_fails' 2023-11-09T03:46:21.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.229+0000 7fe58944f640 1 Processor -- start 2023-11-09T03:46:21.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.229+0000 7fe58944f640 1 -- start start 2023-11-09T03:46:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.230+0000 7fe58944f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5840cde60 0x7fe58415d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.230+0000 7fe58944f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe58415daa0 0x7fe584157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.230+0000 7fe58944f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe58415e000 con 0x7fe5840ce240 2023-11-09T03:46:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.230+0000 7fe58944f640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe58415e5c0 con 0x7fe58415daa0 2023-11-09T03:46:21.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.230+0000 7fe58944f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe58415e2e0 con 0x7fe5840cde60 2023-11-09T03:46:21.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.230+0000 7fe57bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe58415daa0 0x7fe584157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:21.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.230+0000 7fe582ffd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5840cde60 0x7fe58415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:21.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.230+0000 7fe5837fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe5840ce240 0x7fe58406b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:60144/0 (socket says 172.21.15.76:60144) 2023-11-09T03:46:21.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.230+0000 7fe5837fe640 1 -- 172.21.15.76:0/636989797 learned_addr learned my addr 172.21.15.76:0/636989797 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:21.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.230+0000 7fe582ffd640 1 -- 172.21.15.76:0/636989797 >> v2:172.21.15.113:3300/0 conn(0x7fe58415daa0 msgr2=0x7fe584157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:21.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.230+0000 7fe582ffd640 1 --2- 172.21.15.76:0/636989797 >> v2:172.21.15.113:3300/0 conn(0x7fe58415daa0 0x7fe584157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:21.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.230+0000 7fe582ffd640 1 -- 172.21.15.76:0/636989797 >> v1:172.21.15.113:6789/0 conn(0x7fe5840ce240 legacy=0x7fe58406b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:21.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.230+0000 7fe582ffd640 1 -- 172.21.15.76:0/636989797 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5840d1120 con 0x7fe5840cde60 2023-11-09T03:46:21.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.230+0000 7fe57bfff640 1 --2- 172.21.15.76:0/636989797 >> v2:172.21.15.113:3300/0 conn(0x7fe58415daa0 0x7fe584157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:46:21.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.231+0000 7fe582ffd640 1 --2- 172.21.15.76:0/636989797 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5840cde60 0x7fe58415d560 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe5740097a0 tx=0x7fe57402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52fd1a3901961b33 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:21.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.231+0000 7fe5827fc640 1 -- 172.21.15.76:0/636989797 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe574031830 con 0x7fe5840cde60 2023-11-09T03:46:21.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.231+0000 7fe5827fc640 1 -- 172.21.15.76:0/636989797 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe574031990 con 0x7fe5840cde60 2023-11-09T03:46:21.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.231+0000 7fe5827fc640 1 -- 172.21.15.76:0/636989797 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe574031d20 con 0x7fe5840cde60 2023-11-09T03:46:21.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.232+0000 7fe58944f640 1 -- 172.21.15.76:0/636989797 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5840cde60 msgr2=0x7fe58415d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:21.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.232+0000 7fe58944f640 1 --2- 172.21.15.76:0/636989797 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5840cde60 0x7fe58415d560 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe5740097a0 tx=0x7fe57402b8d0 comp rx=0 tx=0).stop 2023-11-09T03:46:21.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.232+0000 7fe58944f640 1 -- 172.21.15.76:0/636989797 shutdown_connections 2023-11-09T03:46:21.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.232+0000 7fe58944f640 1 --2- 172.21.15.76:0/636989797 >> v2:172.21.15.113:3300/0 conn(0x7fe58415daa0 0x7fe584157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:21.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.232+0000 7fe58944f640 1 --2- 172.21.15.76:0/636989797 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5840cde60 0x7fe58415d560 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:21.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.232+0000 7fe58944f640 1 -- 172.21.15.76:0/636989797 >> 172.21.15.76:0/636989797 conn(0x7fe584071f30 msgr2=0x7fe5840cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:21.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.232+0000 7fe58944f640 1 -- 172.21.15.76:0/636989797 shutdown_connections 2023-11-09T03:46:21.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.232+0000 7fe58944f640 1 -- 172.21.15.76:0/636989797 wait complete. 2023-11-09T03:46:21.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.233+0000 7fe58944f640 1 Processor -- start 2023-11-09T03:46:21.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.233+0000 7fe58944f640 1 -- start start 2023-11-09T03:46:21.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.234+0000 7fe58944f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5840cde60 0x7fe5840b7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:21.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.234+0000 7fe58944f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5840ce240 0x7fe5841f3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:21.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.234+0000 7fe58944f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe58415daa0 0x7fe584151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:21.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.234+0000 7fe58944f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe584159550 con 0x7fe58415daa0 2023-11-09T03:46:21.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.234+0000 7fe58944f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5840cb2b0 con 0x7fe5840cde60 2023-11-09T03:46:21.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.234+0000 7fe58944f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5840cb590 con 0x7fe5840ce240 2023-11-09T03:46:21.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.234+0000 7fe5837fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe58415daa0 0x7fe584151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:21.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.234+0000 7fe5837fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe58415daa0 0x7fe584151330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37002/0 (socket says 172.21.15.76:37002) 2023-11-09T03:46:21.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.234+0000 7fe5837fe640 1 -- 172.21.15.76:0/2236665406 learned_addr learned my addr 172.21.15.76:0/2236665406 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:21.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.234+0000 7fe582ffd640 1 --2- 172.21.15.76:0/2236665406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5840cde60 0x7fe5840b7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:21.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.234+0000 7fe57bfff640 1 --2- 172.21.15.76:0/2236665406 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5840ce240 0x7fe5841f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:21.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.235+0000 7fe5837fe640 1 -- 172.21.15.76:0/2236665406 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5840ce240 msgr2=0x7fe5841f3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:21.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.235+0000 7fe5837fe640 1 --2- 172.21.15.76:0/2236665406 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5840ce240 0x7fe5841f3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:21.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.235+0000 7fe5837fe640 1 -- 172.21.15.76:0/2236665406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5840cde60 msgr2=0x7fe5840b7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:21.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.235+0000 7fe5837fe640 1 --2- 172.21.15.76:0/2236665406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5840cde60 0x7fe5840b7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:21.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.235+0000 7fe5837fe640 1 -- 172.21.15.76:0/2236665406 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56c009400 con 0x7fe58415daa0 2023-11-09T03:46:21.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.235+0000 7fe582ffd640 1 --2- 172.21.15.76:0/2236665406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5840cde60 0x7fe5840b7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:21.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.235+0000 7fe5837fe640 1 --2- 172.21.15.76:0/2236665406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe58415daa0 0x7fe584151330 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00cbc0 tx=0x7fe57c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:21.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.235+0000 7fe57b7fe640 1 -- 172.21.15.76:0/2236665406 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c00ace0 con 0x7fe58415daa0 2023-11-09T03:46:21.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.235+0000 7fe57b7fe640 1 -- 172.21.15.76:0/2236665406 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57c00ae40 con 0x7fe58415daa0 2023-11-09T03:46:21.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.235+0000 7fe58944f640 1 -- 172.21.15.76:0/2236665406 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe574009400 con 0x7fe58415daa0 2023-11-09T03:46:21.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.235+0000 7fe57b7fe640 1 -- 172.21.15.76:0/2236665406 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c012600 con 0x7fe58415daa0 2023-11-09T03:46:21.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.235+0000 7fe58944f640 1 -- 172.21.15.76:0/2236665406 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5840cbe30 con 0x7fe58415daa0 2023-11-09T03:46:21.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.236+0000 7fe58944f640 1 -- 172.21.15.76:0/2236665406 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe584074290 con 0x7fe58415daa0 2023-11-09T03:46:21.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.236+0000 7fe57b7fe640 1 -- 172.21.15.76:0/2236665406 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe57c007070 con 0x7fe58415daa0 2023-11-09T03:46:21.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.237+0000 7fe57b7fe640 1 --2- 172.21.15.76:0/2236665406 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe56406ae70 0x7fe56406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:21.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.237+0000 7fe57b7fe640 1 -- 172.21.15.76:0/2236665406 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fe57c089740 con 0x7fe58415daa0 2023-11-09T03:46:21.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.237+0000 7fe582ffd640 1 --2- 172.21.15.76:0/2236665406 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe56406ae70 0x7fe56406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:21.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.238+0000 7fe582ffd640 1 --2- 172.21.15.76:0/2236665406 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe56406ae70 0x7fe56406d330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe574036000 tx=0x7fe57403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:21.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.241+0000 7fe57b7fe640 1 -- 172.21.15.76:0/2236665406 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe57c054550 con 0x7fe58415daa0 2023-11-09T03:46:21.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.477+0000 7fe58944f640 1 -- 172.21.15.76:0/2236665406 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_attaching_pools_with_snaps_to_fs_fails"]} v 0) v1 -- 0x7fe5840cc020 con 0x7fe58415daa0 2023-11-09T03:46:21.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.678+0000 7fe57b7fe640 1 -- 172.21.15.76:0/2236665406 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_attaching_pools_with_snaps_to_fs_fails"]}]=0 v69) v1 ==== 176+0+0 (secure 0 0 0) 0x7fe57c057b70 con 0x7fe58415daa0 2023-11-09T03:46:21.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.680+0000 7fe58944f640 1 -- 172.21.15.76:0/2236665406 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe56406ae70 msgr2=0x7fe56406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:21.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.680+0000 7fe58944f640 1 --2- 172.21.15.76:0/2236665406 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe56406ae70 0x7fe56406d330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe574036000 tx=0x7fe57403c040 comp rx=0 tx=0).stop 2023-11-09T03:46:21.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.680+0000 7fe58944f640 1 -- 172.21.15.76:0/2236665406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe58415daa0 msgr2=0x7fe584151330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:21.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.680+0000 7fe58944f640 1 --2- 172.21.15.76:0/2236665406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe58415daa0 0x7fe584151330 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00cbc0 tx=0x7fe57c00a430 comp rx=0 tx=0).stop 2023-11-09T03:46:21.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.680+0000 7fe58944f640 1 -- 172.21.15.76:0/2236665406 shutdown_connections 2023-11-09T03:46:21.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.680+0000 7fe58944f640 1 --2- 172.21.15.76:0/2236665406 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe56406ae70 0x7fe56406d330 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:21.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.680+0000 7fe58944f640 1 --2- 172.21.15.76:0/2236665406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe58415daa0 0x7fe584151330 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:21.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.680+0000 7fe58944f640 1 --2- 172.21.15.76:0/2236665406 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5840ce240 0x7fe5841f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:21.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.680+0000 7fe58944f640 1 --2- 172.21.15.76:0/2236665406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5840cde60 0x7fe5840b7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:21.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.680+0000 7fe58944f640 1 -- 172.21.15.76:0/2236665406 >> 172.21.15.76:0/2236665406 conn(0x7fe584071f30 msgr2=0x7fe5840d09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:21.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.680+0000 7fe58944f640 1 -- 172.21.15.76:0/2236665406 shutdown_connections 2023-11-09T03:46:21.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.680+0000 7fe58944f640 1 -- 172.21.15.76:0/2236665406 wait complete. 2023-11-09T03:46:21.691 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-09T03:46:21.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.798+0000 7ff896178640 1 Processor -- start 2023-11-09T03:46:21.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.798+0000 7ff896178640 1 -- start start 2023-11-09T03:46:21.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.799+0000 7ff896178640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff8901603d0 0x7ff8900b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:21.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.799+0000 7ff896178640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8900cdf80 con 0x7ff89006a830 2023-11-09T03:46:21.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.799+0000 7ff896178640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8900ce6f0 con 0x7ff8900722a0 2023-11-09T03:46:21.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.799+0000 7ff896178640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8900b19d0 con 0x7ff8901603d0 2023-11-09T03:46:21.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.799+0000 7ff88ffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff89006a830 0x7ff8900ca380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43720/0 (socket says 172.21.15.76:43720) 2023-11-09T03:46:21.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.799+0000 7ff88ffff640 1 -- 172.21.15.76:0/2416368578 learned_addr learned my addr 172.21.15.76:0/2416368578 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:21.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.800+0000 7ff88e7fc640 1 -- 172.21.15.76:0/2416368578 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 851437066 0 0) 0x7ff8900cdf80 con 0x7ff89006a830 2023-11-09T03:46:21.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.800+0000 7ff88e7fc640 1 -- 172.21.15.76:0/2416368578 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff878003520 con 0x7ff89006a830 2023-11-09T03:46:21.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.800+0000 7ff88f7fe640 1 --2- 172.21.15.76:0/2416368578 >> v2:172.21.15.113:3300/0 conn(0x7ff8901603d0 0x7ff8900b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:21.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.800+0000 7ff88e7fc640 1 -- 172.21.15.76:0/2416368578 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2976979148 0 0) 0x7ff878003520 con 0x7ff89006a830 2023-11-09T03:46:21.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.800+0000 7ff88e7fc640 1 -- 172.21.15.76:0/2416368578 >> v2:172.21.15.113:3300/0 conn(0x7ff8901603d0 msgr2=0x7ff8900b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:21.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.800+0000 7ff88e7fc640 1 --2- 172.21.15.76:0/2416368578 >> v2:172.21.15.113:3300/0 conn(0x7ff8901603d0 0x7ff8900b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:21.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.800+0000 7ff88e7fc640 1 -- 172.21.15.76:0/2416368578 >> v1:172.21.15.113:6789/0 conn(0x7ff8900722a0 legacy=0x7ff89006a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:21.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.800+0000 7ff88e7fc640 1 -- 172.21.15.76:0/2416368578 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff890072700 con 0x7ff89006a830 2023-11-09T03:46:21.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.800+0000 7ff88e7fc640 1 -- 172.21.15.76:0/2416368578 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7ff8880019f0 con 0x7ff89006a830 2023-11-09T03:46:21.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.800+0000 7ff88e7fc640 1 -- 172.21.15.76:0/2416368578 >> v1:172.21.15.76:6789/0 conn(0x7ff89006a830 legacy=0x7ff8900ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:21.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.800+0000 7ff88e7fc640 1 --2- 172.21.15.76:0/2416368578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff878003970 0x7ff878003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:21.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.800+0000 7ff88e7fc640 1 --2- 172.21.15.76:0/2416368578 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8780047c0 0x7ff878004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:21.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.800+0000 7ff88e7fc640 1 --2- 172.21.15.76:0/2416368578 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff878005180 0x7ff87800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:21.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.800+0000 7ff88e7fc640 1 -- 172.21.15.76:0/2416368578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87800c0f0 con 0x7ff878003970 2023-11-09T03:46:21.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.800+0000 7ff88e7fc640 1 -- 172.21.15.76:0/2416368578 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87800c260 con 0x7ff8780047c0 2023-11-09T03:46:21.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.800+0000 7ff88e7fc640 1 -- 172.21.15.76:0/2416368578 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87800c420 con 0x7ff878005180 2023-11-09T03:46:21.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.800+0000 7ff88f7fe640 1 --2- 172.21.15.76:0/2416368578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff878003970 0x7ff878003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:21.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.800+0000 7ff88effd640 1 --2- 172.21.15.76:0/2416368578 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8780047c0 0x7ff878004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:21.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.800+0000 7ff88ffff640 1 --2- 172.21.15.76:0/2416368578 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff878005180 0x7ff87800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:21.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.800+0000 7ff88f7fe640 1 -- 172.21.15.76:0/2416368578 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff878005180 msgr2=0x7ff87800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:21.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.800+0000 7ff88f7fe640 1 --2- 172.21.15.76:0/2416368578 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff878005180 0x7ff87800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:21.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.800+0000 7ff88f7fe640 1 -- 172.21.15.76:0/2416368578 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8780047c0 msgr2=0x7ff878004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:21.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.800+0000 7ff88f7fe640 1 --2- 172.21.15.76:0/2416368578 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8780047c0 0x7ff878004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:21.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.800+0000 7ff88f7fe640 1 -- 172.21.15.76:0/2416368578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff878003520 con 0x7ff878003970 2023-11-09T03:46:21.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.800+0000 7ff88effd640 1 --2- 172.21.15.76:0/2416368578 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8780047c0 0x7ff878004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:21.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.800+0000 7ff88ffff640 1 -- 172.21.15.76:0/2416368578 reap_dead start 2023-11-09T03:46:21.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.801+0000 7ff88f7fe640 1 --2- 172.21.15.76:0/2416368578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff878003970 0x7ff878003d90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff880009ac0 tx=0x7ff8780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b3cc992fad75f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:21.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.801+0000 7ff88e7fc640 1 -- 172.21.15.76:0/2416368578 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88002c6d0 con 0x7ff878003970 2023-11-09T03:46:21.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.801+0000 7ff88e7fc640 1 -- 172.21.15.76:0/2416368578 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88002c830 con 0x7ff878003970 2023-11-09T03:46:21.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.801+0000 7ff88e7fc640 1 -- 172.21.15.76:0/2416368578 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88002cbc0 con 0x7ff878003970 2023-11-09T03:46:21.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.802+0000 7ff896178640 1 -- 172.21.15.76:0/2416368578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff878003970 msgr2=0x7ff878003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:21.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.802+0000 7ff896178640 1 --2- 172.21.15.76:0/2416368578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff878003970 0x7ff878003d90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff880009ac0 tx=0x7ff8780057f0 comp rx=0 tx=0).stop 2023-11-09T03:46:21.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.802+0000 7ff896178640 1 -- 172.21.15.76:0/2416368578 shutdown_connections 2023-11-09T03:46:21.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.802+0000 7ff896178640 1 --2- 172.21.15.76:0/2416368578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff878003970 0x7ff878003d90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:21.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.802+0000 7ff896178640 1 -- 172.21.15.76:0/2416368578 >> 172.21.15.76:0/2416368578 conn(0x7ff89006d800 msgr2=0x7ff89006dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:21.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.802+0000 7ff896178640 1 -- 172.21.15.76:0/2416368578 shutdown_connections 2023-11-09T03:46:21.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.802+0000 7ff896178640 1 -- 172.21.15.76:0/2416368578 wait complete. 2023-11-09T03:46:21.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.803+0000 7ff896178640 1 Processor -- start 2023-11-09T03:46:21.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.803+0000 7ff896178640 1 -- start start 2023-11-09T03:46:21.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.804+0000 7ff896178640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff878003970 0x7ff8900c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:21.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.804+0000 7ff896178640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8900c5390 0x7ff8900b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:21.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.804+0000 7ff896178640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8900b3f40 0x7ff8900c3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:21.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.804+0000 7ff896178640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff890151c10 con 0x7ff8900c5390 2023-11-09T03:46:21.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.804+0000 7ff896178640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8901514a0 con 0x7ff8900b3f40 2023-11-09T03:46:21.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.804+0000 7ff896178640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8900b3570 con 0x7ff878003970 2023-11-09T03:46:21.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.804+0000 7ff88effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8900c5390 0x7ff8900b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:21.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.804+0000 7ff88effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8900c5390 0x7ff8900b3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37028/0 (socket says 172.21.15.76:37028) 2023-11-09T03:46:21.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.804+0000 7ff88effd640 1 -- 172.21.15.76:0/1261785048 learned_addr learned my addr 172.21.15.76:0/1261785048 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:21.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.804+0000 7ff88f7fe640 1 --2- 172.21.15.76:0/1261785048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff878003970 0x7ff8900c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:21.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.804+0000 7ff88ffff640 1 --2- 172.21.15.76:0/1261785048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8900b3f40 0x7ff8900c3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:21.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.804+0000 7ff88effd640 1 -- 172.21.15.76:0/1261785048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff878003970 msgr2=0x7ff8900c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:21.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.804+0000 7ff88effd640 1 --2- 172.21.15.76:0/1261785048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff878003970 0x7ff8900c5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:21.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.804+0000 7ff88effd640 1 -- 172.21.15.76:0/1261785048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8900b3f40 msgr2=0x7ff8900c3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:21.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.804+0000 7ff88effd640 1 --2- 172.21.15.76:0/1261785048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8900b3f40 0x7ff8900c3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:21.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.804+0000 7ff88effd640 1 -- 172.21.15.76:0/1261785048 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8900b3220 con 0x7ff8900c5390 2023-11-09T03:46:21.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.804+0000 7ff88ffff640 1 --2- 172.21.15.76:0/1261785048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8900b3f40 0x7ff8900c3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:21.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.804+0000 7ff88effd640 1 --2- 172.21.15.76:0/1261785048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8900c5390 0x7ff8900b3a00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff88400e480 tx=0x7ff88400e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:21.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.804+0000 7ff877fff640 1 -- 172.21.15.76:0/1261785048 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff884017070 con 0x7ff8900c5390 2023-11-09T03:46:21.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.804+0000 7ff877fff640 1 -- 172.21.15.76:0/1261785048 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8840052e0 con 0x7ff8900c5390 2023-11-09T03:46:21.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.804+0000 7ff896178640 1 -- 172.21.15.76:0/1261785048 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8901543d0 con 0x7ff8900c5390 2023-11-09T03:46:21.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.804+0000 7ff877fff640 1 -- 172.21.15.76:0/1261785048 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff884005490 con 0x7ff8900c5390 2023-11-09T03:46:21.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.804+0000 7ff896178640 1 -- 172.21.15.76:0/1261785048 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8900c63f0 con 0x7ff8900c5390 2023-11-09T03:46:21.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.805+0000 7ff896178640 1 -- 172.21.15.76:0/1261785048 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8900b36e0 con 0x7ff8900c5390 2023-11-09T03:46:21.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.806+0000 7ff877fff640 1 -- 172.21.15.76:0/1261785048 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff884007020 con 0x7ff8900c5390 2023-11-09T03:46:21.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.806+0000 7ff877fff640 1 --2- 172.21.15.76:0/1261785048 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff87c06ae70 0x7ff87c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:21.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.806+0000 7ff877fff640 1 -- 172.21.15.76:0/1261785048 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7ff884088f80 con 0x7ff8900c5390 2023-11-09T03:46:21.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.806+0000 7ff88f7fe640 1 --2- 172.21.15.76:0/1261785048 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff87c06ae70 0x7ff87c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:21.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.807+0000 7ff88f7fe640 1 --2- 172.21.15.76:0/1261785048 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff87c06ae70 0x7ff87c06d330 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff880036000 tx=0x7ff880030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:21.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.810+0000 7ff877fff640 1 -- 172.21.15.76:0/1261785048 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff8840bbca0 con 0x7ff8900c5390 2023-11-09T03:46:21.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.979+0000 7ff896178640 1 -- 172.21.15.76:0/1261785048 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff8901619d0 con 0x7ff8900c5390 2023-11-09T03:46:21.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.984+0000 7ff877fff640 1 -- 172.21.15.76:0/1261785048 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v11)=0 v11) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff884053ce0 con 0x7ff8900c5390 2023-11-09T03:46:21.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.986+0000 7ff896178640 1 -- 172.21.15.76:0/1261785048 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff87c06ae70 msgr2=0x7ff87c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:21.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.986+0000 7ff896178640 1 --2- 172.21.15.76:0/1261785048 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff87c06ae70 0x7ff87c06d330 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff880036000 tx=0x7ff880030000 comp rx=0 tx=0).stop 2023-11-09T03:46:21.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.986+0000 7ff896178640 1 -- 172.21.15.76:0/1261785048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8900c5390 msgr2=0x7ff8900b3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:21.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.986+0000 7ff896178640 1 --2- 172.21.15.76:0/1261785048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8900c5390 0x7ff8900b3a00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff88400e480 tx=0x7ff88400e940 comp rx=0 tx=0).stop 2023-11-09T03:46:21.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.986+0000 7ff896178640 1 -- 172.21.15.76:0/1261785048 shutdown_connections 2023-11-09T03:46:21.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.986+0000 7ff896178640 1 --2- 172.21.15.76:0/1261785048 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff87c06ae70 0x7ff87c06d330 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:21.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.986+0000 7ff896178640 1 --2- 172.21.15.76:0/1261785048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8900b3f40 0x7ff8900c3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:21.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.986+0000 7ff896178640 1 --2- 172.21.15.76:0/1261785048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8900c5390 0x7ff8900b3a00 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:21.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.986+0000 7ff896178640 1 --2- 172.21.15.76:0/1261785048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff878003970 0x7ff8900c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:21.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.986+0000 7ff896178640 1 -- 172.21.15.76:0/1261785048 >> 172.21.15.76:0/1261785048 conn(0x7ff89006d800 msgr2=0x7ff8900cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:21.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.986+0000 7ff896178640 1 -- 172.21.15.76:0/1261785048 shutdown_connections 2023-11-09T03:46:21.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:21.986+0000 7ff896178640 1 -- 172.21.15.76:0/1261785048 wait complete. 2023-11-09T03:46:21.995 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-09T03:46:21.996 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-09T03:46:22.037 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-09T03:46:22.093 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-09T03:46:22.093 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:46:22.093 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:46:22.109 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:46:22.109 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:46:22.164 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:46:22.205 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:46:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:46:22.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.305+0000 7f3658d83640 1 Processor -- start 2023-11-09T03:46:22.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.305+0000 7f3658d83640 1 -- start start 2023-11-09T03:46:22.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.306+0000 7f3658d83640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f365415ac70 0x7f36540d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:22.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.306+0000 7f3658d83640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36540b2d10 0x7f36540b4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:22.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.306+0000 7f3658d83640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36540b3c10 con 0x7f36540cec10 2023-11-09T03:46:22.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.306+0000 7f3658d83640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36540cf180 con 0x7f365415ac70 2023-11-09T03:46:22.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.306+0000 7f3658d83640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36540cf460 con 0x7f36540b2d10 2023-11-09T03:46:22.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.306+0000 7f365259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f365415ac70 0x7f36540d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:22.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.306+0000 7f3651d9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f36540cec10 0x7f36540b2600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43722/0 (socket says 172.21.15.76:43722) 2023-11-09T03:46:22.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.306+0000 7f3651d9b640 1 -- 172.21.15.76:0/3582193628 learned_addr learned my addr 172.21.15.76:0/3582193628 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:22.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.306+0000 7f365259c640 1 -- 172.21.15.76:0/3582193628 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36540b2d10 msgr2=0x7f36540b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:22.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.306+0000 7f365259c640 1 --2- 172.21.15.76:0/3582193628 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36540b2d10 0x7f36540b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:22.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.306+0000 7f365259c640 1 -- 172.21.15.76:0/3582193628 >> v1:172.21.15.76:6789/0 conn(0x7f36540cec10 legacy=0x7f36540b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:22.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.306+0000 7f365259c640 1 -- 172.21.15.76:0/3582193628 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36540cf740 con 0x7f365415ac70 2023-11-09T03:46:22.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.306+0000 7f365159a640 1 -- 172.21.15.76:0/3582193628 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3733300870 0 0) 0x7f36540b3c10 con 0x7f36540cec10 2023-11-09T03:46:22.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.307+0000 7f365259c640 1 --2- 172.21.15.76:0/3582193628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f365415ac70 0x7f36540d0330 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3644009790 tx=0x7f364402b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e61aafaa2bcbe8 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:22.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.307+0000 7f365159a640 1 -- 172.21.15.76:0/3582193628 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3644031800 con 0x7f365415ac70 2023-11-09T03:46:22.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.307+0000 7f365159a640 1 -- 172.21.15.76:0/3582193628 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3644031960 con 0x7f365415ac70 2023-11-09T03:46:22.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.307+0000 7f365159a640 1 -- 172.21.15.76:0/3582193628 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3644031cf0 con 0x7f365415ac70 2023-11-09T03:46:22.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.308+0000 7f3658d83640 1 -- 172.21.15.76:0/3582193628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f365415ac70 msgr2=0x7f36540d0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:22.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.308+0000 7f3658d83640 1 --2- 172.21.15.76:0/3582193628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f365415ac70 0x7f36540d0330 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3644009790 tx=0x7f364402b8a0 comp rx=0 tx=0).stop 2023-11-09T03:46:22.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.308+0000 7f3658d83640 1 -- 172.21.15.76:0/3582193628 shutdown_connections 2023-11-09T03:46:22.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.308+0000 7f3658d83640 1 --2- 172.21.15.76:0/3582193628 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36540b2d10 0x7f36540b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:22.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.308+0000 7f3658d83640 1 --2- 172.21.15.76:0/3582193628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f365415ac70 0x7f36540d0330 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:22.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.308+0000 7f3658d83640 1 -- 172.21.15.76:0/3582193628 >> 172.21.15.76:0/3582193628 conn(0x7f36540cd600 msgr2=0x7f36540cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:22.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.308+0000 7f3658d83640 1 -- 172.21.15.76:0/3582193628 shutdown_connections 2023-11-09T03:46:22.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.308+0000 7f3658d83640 1 -- 172.21.15.76:0/3582193628 wait complete. 2023-11-09T03:46:22.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.309+0000 7f3658d83640 1 Processor -- start 2023-11-09T03:46:22.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.309+0000 7f3658d83640 1 -- start start 2023-11-09T03:46:22.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.310+0000 7f3658d83640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36540b2d10 0x7f365415dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:22.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.310+0000 7f3658d83640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36540cec10 0x7f365406bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:22.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.310+0000 7f3658d83640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f365415ac70 0x7f3654152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:22.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.310+0000 7f3658d83640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365406cbe0 con 0x7f36540b2d10 2023-11-09T03:46:22.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.310+0000 7f3658d83640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365415be80 con 0x7f365415ac70 2023-11-09T03:46:22.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.310+0000 7f3658d83640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f365406c620 con 0x7f36540cec10 2023-11-09T03:46:22.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.310+0000 7f365259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36540b2d10 0x7f365415dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:22.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.310+0000 7f365259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36540b2d10 0x7f365415dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37046/0 (socket says 172.21.15.76:37046) 2023-11-09T03:46:22.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.310+0000 7f365259c640 1 -- 172.21.15.76:0/248797086 learned_addr learned my addr 172.21.15.76:0/248797086 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:22.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.311+0000 7f3651d9b640 1 --2- 172.21.15.76:0/248797086 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36540cec10 0x7f365406bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:22.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.311+0000 7f3652d9d640 1 --2- 172.21.15.76:0/248797086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f365415ac70 0x7f3654152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:22.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.311+0000 7f365259c640 1 -- 172.21.15.76:0/248797086 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36540cec10 msgr2=0x7f365406bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:22.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.311+0000 7f365259c640 1 --2- 172.21.15.76:0/248797086 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36540cec10 0x7f365406bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:22.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.311+0000 7f365259c640 1 -- 172.21.15.76:0/248797086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f365415ac70 msgr2=0x7f3654152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:22.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.311+0000 7f365259c640 1 --2- 172.21.15.76:0/248797086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f365415ac70 0x7f3654152ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:22.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.311+0000 7f365259c640 1 -- 172.21.15.76:0/248797086 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36440093a0 con 0x7f36540b2d10 2023-11-09T03:46:22.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.311+0000 7f3651d9b640 1 --2- 172.21.15.76:0/248797086 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36540cec10 0x7f365406bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:22.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.311+0000 7f365259c640 1 --2- 172.21.15.76:0/248797086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36540b2d10 0x7f365415dd80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3644000c00 tx=0x7f36440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:22.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.311+0000 7f3642ffd640 1 -- 172.21.15.76:0/248797086 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3644004220 con 0x7f36540b2d10 2023-11-09T03:46:22.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.311+0000 7f3642ffd640 1 -- 172.21.15.76:0/248797086 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3644004380 con 0x7f36540b2d10 2023-11-09T03:46:22.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.311+0000 7f3642ffd640 1 -- 172.21.15.76:0/248797086 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3644005bd0 con 0x7f36540b2d10 2023-11-09T03:46:22.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.311+0000 7f3658d83640 1 -- 172.21.15.76:0/248797086 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f365406c900 con 0x7f36540b2d10 2023-11-09T03:46:22.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.311+0000 7f3658d83640 1 -- 172.21.15.76:0/248797086 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36541525b0 con 0x7f36540b2d10 2023-11-09T03:46:22.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.312+0000 7f3658d83640 1 -- 172.21.15.76:0/248797086 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f365415b970 con 0x7f36540b2d10 2023-11-09T03:46:22.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.313+0000 7f3642ffd640 1 -- 172.21.15.76:0/248797086 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3644043070 con 0x7f36540b2d10 2023-11-09T03:46:22.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.313+0000 7f3642ffd640 1 --2- 172.21.15.76:0/248797086 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f363406aea0 0x7f363406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:22.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.314+0000 7f3642ffd640 1 -- 172.21.15.76:0/248797086 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f364402e070 con 0x7f36540b2d10 2023-11-09T03:46:22.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.314+0000 7f3651d9b640 1 --2- 172.21.15.76:0/248797086 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f363406aea0 0x7f363406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:22.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.314+0000 7f3651d9b640 1 --2- 172.21.15.76:0/248797086 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f363406aea0 0x7f363406d360 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f36540b1cd0 tx=0x7f3648009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:22.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.318+0000 7f3642ffd640 1 -- 172.21.15.76:0/248797086 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f365415b970 con 0x7f36540b2d10 2023-11-09T03:46:22.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.486+0000 7f3658d83640 1 -- 172.21.15.76:0/248797086 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f365415b970 con 0x7f36540b2d10 2023-11-09T03:46:22.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.487+0000 7f3642ffd640 1 -- 172.21.15.76:0/248797086 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18710 (secure 0 0 0) 0x7f365415b970 con 0x7f36540b2d10 2023-11-09T03:46:22.487 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:22.487 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":27,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:46:12.537265+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:46:22.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.489+0000 7f3658d83640 1 -- 172.21.15.76:0/248797086 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f363406aea0 msgr2=0x7f363406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:22.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.489+0000 7f3658d83640 1 --2- 172.21.15.76:0/248797086 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f363406aea0 0x7f363406d360 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f36540b1cd0 tx=0x7f3648009040 comp rx=0 tx=0).stop 2023-11-09T03:46:22.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.489+0000 7f3658d83640 1 -- 172.21.15.76:0/248797086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36540b2d10 msgr2=0x7f365415dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:22.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.489+0000 7f3658d83640 1 --2- 172.21.15.76:0/248797086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36540b2d10 0x7f365415dd80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3644000c00 tx=0x7f36440046c0 comp rx=0 tx=0).stop 2023-11-09T03:46:22.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.489+0000 7f3658d83640 1 -- 172.21.15.76:0/248797086 shutdown_connections 2023-11-09T03:46:22.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.489+0000 7f3658d83640 1 --2- 172.21.15.76:0/248797086 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f363406aea0 0x7f363406d360 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:22.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.489+0000 7f3658d83640 1 --2- 172.21.15.76:0/248797086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f365415ac70 0x7f3654152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:22.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.489+0000 7f3658d83640 1 --2- 172.21.15.76:0/248797086 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36540cec10 0x7f365406bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:22.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.489+0000 7f3658d83640 1 --2- 172.21.15.76:0/248797086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36540b2d10 0x7f365415dd80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:22.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.489+0000 7f3658d83640 1 -- 172.21.15.76:0/248797086 >> 172.21.15.76:0/248797086 conn(0x7f36540cd600 msgr2=0x7f36540cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:22.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.489+0000 7f3658d83640 1 -- 172.21.15.76:0/248797086 shutdown_connections 2023-11-09T03:46:22.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.489+0000 7f3658d83640 1 -- 172.21.15.76:0/248797086 wait complete. 2023-11-09T03:46:22.499 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:45:28.704715+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:45:29.717614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-09T03:45:33.440577+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-09T03:46:22.500 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:46:22.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.603+0000 7fd8e4355640 1 Processor -- start 2023-11-09T03:46:22.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.603+0000 7fd8e4355640 1 -- start start 2023-11-09T03:46:22.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.604+0000 7fd8e4355640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd8dc070e20 0x7fd8dc0adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:22.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.604+0000 7fd8e4355640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8dc0c3900 con 0x7fd8dc0ad730 2023-11-09T03:46:22.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.604+0000 7fd8e4355640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8dc0c3be0 con 0x7fd8dc15ad30 2023-11-09T03:46:22.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.604+0000 7fd8e4355640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8dc0ae260 con 0x7fd8dc070e20 2023-11-09T03:46:22.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.604+0000 7fd8e28f2640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd8dc0ad730 0x7fd8dc15e740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53006/0 (socket says 172.21.15.76:53006) 2023-11-09T03:46:22.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.604+0000 7fd8e28f2640 1 -- 172.21.15.76:0/4183650360 learned_addr learned my addr 172.21.15.76:0/4183650360 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:22.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.604+0000 7fd8e18f0640 1 --2- 172.21.15.76:0/4183650360 >> v2:172.21.15.113:3300/0 conn(0x7fd8dc070e20 0x7fd8dc0adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:22.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.604+0000 7fd8e10ef640 1 -- 172.21.15.76:0/4183650360 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491222998 0 0) 0x7fd8dc0c3900 con 0x7fd8dc0ad730 2023-11-09T03:46:22.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.604+0000 7fd8e10ef640 1 -- 172.21.15.76:0/4183650360 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8c8003520 con 0x7fd8dc0ad730 2023-11-09T03:46:22.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.605+0000 7fd8e10ef640 1 -- 172.21.15.76:0/4183650360 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2321113497 0 0) 0x7fd8c8003520 con 0x7fd8dc0ad730 2023-11-09T03:46:22.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.605+0000 7fd8e10ef640 1 -- 172.21.15.76:0/4183650360 >> v2:172.21.15.113:3300/0 conn(0x7fd8dc070e20 msgr2=0x7fd8dc0adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:22.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.605+0000 7fd8e10ef640 1 --2- 172.21.15.76:0/4183650360 >> v2:172.21.15.113:3300/0 conn(0x7fd8dc070e20 0x7fd8dc0adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:22.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.605+0000 7fd8e10ef640 1 -- 172.21.15.76:0/4183650360 >> v1:172.21.15.113:6789/0 conn(0x7fd8dc15ad30 legacy=0x7fd8dc070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:22.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.605+0000 7fd8e10ef640 1 -- 172.21.15.76:0/4183650360 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8dc0c3ec0 con 0x7fd8dc0ad730 2023-11-09T03:46:22.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.605+0000 7fd8e10ef640 1 -- 172.21.15.76:0/4183650360 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fd8d4001670 con 0x7fd8dc0ad730 2023-11-09T03:46:22.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.605+0000 7fd8e10ef640 1 -- 172.21.15.76:0/4183650360 >> v1:172.21.15.76:6789/0 conn(0x7fd8dc0ad730 legacy=0x7fd8dc15e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:22.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.605+0000 7fd8e10ef640 1 --2- 172.21.15.76:0/4183650360 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8c8003970 0x7fd8c8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:22.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.605+0000 7fd8e10ef640 1 --2- 172.21.15.76:0/4183650360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8c80047c0 0x7fd8c8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:22.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.605+0000 7fd8e10ef640 1 --2- 172.21.15.76:0/4183650360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8c8005180 0x7fd8c800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:22.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.605+0000 7fd8e10ef640 1 -- 172.21.15.76:0/4183650360 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8dc0ae260 con 0x7fd8c8005180 2023-11-09T03:46:22.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.605+0000 7fd8e10ef640 1 -- 172.21.15.76:0/4183650360 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c800c120 con 0x7fd8c80047c0 2023-11-09T03:46:22.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.605+0000 7fd8e10ef640 1 -- 172.21.15.76:0/4183650360 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c800c2e0 con 0x7fd8c8003970 2023-11-09T03:46:22.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.605+0000 7fd8e28f2640 1 --2- 172.21.15.76:0/4183650360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8c8005180 0x7fd8c800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:22.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.605+0000 7fd8e18f0640 1 --2- 172.21.15.76:0/4183650360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8c80047c0 0x7fd8c8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:22.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.605+0000 7fd8e20f1640 1 --2- 172.21.15.76:0/4183650360 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8c8003970 0x7fd8c8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:22.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.605+0000 7fd8e28f2640 1 -- 172.21.15.76:0/4183650360 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8c8003970 msgr2=0x7fd8c8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:22.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.605+0000 7fd8e28f2640 1 --2- 172.21.15.76:0/4183650360 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8c8003970 0x7fd8c8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:22.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.605+0000 7fd8e28f2640 1 -- 172.21.15.76:0/4183650360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8c80047c0 msgr2=0x7fd8c8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:22.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.605+0000 7fd8e28f2640 1 --2- 172.21.15.76:0/4183650360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8c80047c0 0x7fd8c8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:22.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.605+0000 7fd8e28f2640 1 -- 172.21.15.76:0/4183650360 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c8003520 con 0x7fd8c8005180 2023-11-09T03:46:22.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.606+0000 7fd8e28f2640 1 -- 172.21.15.76:0/4183650360 reap_dead start 2023-11-09T03:46:22.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.606+0000 7fd8e28f2640 1 --2- 172.21.15.76:0/4183650360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8c8005180 0x7fd8c800b980 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd8d400c190 tx=0x7fd8c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40a14003328cf83 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:22.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.606+0000 7fd8e10ef640 1 -- 172.21.15.76:0/4183650360 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d4005ad0 con 0x7fd8c8005180 2023-11-09T03:46:22.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.606+0000 7fd8e10ef640 1 -- 172.21.15.76:0/4183650360 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d4005c70 con 0x7fd8c8005180 2023-11-09T03:46:22.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.606+0000 7fd8e10ef640 1 -- 172.21.15.76:0/4183650360 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d402c8a0 con 0x7fd8c8005180 2023-11-09T03:46:22.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.607+0000 7fd8e4355640 1 -- 172.21.15.76:0/4183650360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8c8005180 msgr2=0x7fd8c800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:22.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.607+0000 7fd8e4355640 1 --2- 172.21.15.76:0/4183650360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8c8005180 0x7fd8c800b980 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd8d400c190 tx=0x7fd8c80057f0 comp rx=0 tx=0).stop 2023-11-09T03:46:22.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.607+0000 7fd8e4355640 1 -- 172.21.15.76:0/4183650360 shutdown_connections 2023-11-09T03:46:22.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.607+0000 7fd8e4355640 1 --2- 172.21.15.76:0/4183650360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8c8005180 0x7fd8c800b980 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:22.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.607+0000 7fd8e4355640 1 -- 172.21.15.76:0/4183650360 >> 172.21.15.76:0/4183650360 conn(0x7fd8dc0ce950 msgr2=0x7fd8dc0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:22.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.607+0000 7fd8e4355640 1 -- 172.21.15.76:0/4183650360 shutdown_connections 2023-11-09T03:46:22.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.607+0000 7fd8e4355640 1 -- 172.21.15.76:0/4183650360 wait complete. 2023-11-09T03:46:22.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.608+0000 7fd8e4355640 1 Processor -- start 2023-11-09T03:46:22.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.608+0000 7fd8e4355640 1 -- start start 2023-11-09T03:46:22.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.608+0000 7fd8e4355640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8c8005180 0x7fd8dc156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:22.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.609+0000 7fd8e4355640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8dc06d1a0 0x7fd8dc06d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:22.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.609+0000 7fd8e4355640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8dc158d20 0x7fd8dc1591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:22.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.609+0000 7fd8e4355640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8dc142c10 con 0x7fd8c8005180 2023-11-09T03:46:22.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.609+0000 7fd8e4355640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8dc06db60 con 0x7fd8dc06d1a0 2023-11-09T03:46:22.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.609+0000 7fd8e4355640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8dc141f90 con 0x7fd8dc158d20 2023-11-09T03:46:22.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.609+0000 7fd8e20f1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8c8005180 0x7fd8dc156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:22.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.609+0000 7fd8e20f1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8c8005180 0x7fd8dc156ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35640/0 (socket says 172.21.15.76:35640) 2023-11-09T03:46:22.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.609+0000 7fd8e20f1640 1 -- 172.21.15.76:0/1900709689 learned_addr learned my addr 172.21.15.76:0/1900709689 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:22.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.609+0000 7fd8e28f2640 1 --2- 172.21.15.76:0/1900709689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8dc158d20 0x7fd8dc1591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:22.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.609+0000 7fd8e18f0640 1 --2- 172.21.15.76:0/1900709689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8dc06d1a0 0x7fd8dc06d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:22.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.609+0000 7fd8e20f1640 1 -- 172.21.15.76:0/1900709689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8dc158d20 msgr2=0x7fd8dc1591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:22.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.609+0000 7fd8e20f1640 1 --2- 172.21.15.76:0/1900709689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8dc158d20 0x7fd8dc1591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:22.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.609+0000 7fd8e20f1640 1 -- 172.21.15.76:0/1900709689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8dc06d1a0 msgr2=0x7fd8dc06d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:22.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.609+0000 7fd8e20f1640 1 --2- 172.21.15.76:0/1900709689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8dc06d1a0 0x7fd8dc06d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:22.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.609+0000 7fd8e20f1640 1 -- 172.21.15.76:0/1900709689 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8dc142270 con 0x7fd8c8005180 2023-11-09T03:46:22.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.609+0000 7fd8e20f1640 1 --2- 172.21.15.76:0/1900709689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8c8005180 0x7fd8dc156ef0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd8d800f960 tx=0x7fd8d800fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:22.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.609+0000 7fd8d2ffd640 1 -- 172.21.15.76:0/1900709689 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8003b80 con 0x7fd8c8005180 2023-11-09T03:46:22.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.609+0000 7fd8d2ffd640 1 -- 172.21.15.76:0/1900709689 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d8003ce0 con 0x7fd8c8005180 2023-11-09T03:46:22.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.609+0000 7fd8e4355640 1 -- 172.21.15.76:0/1900709689 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8dc142550 con 0x7fd8c8005180 2023-11-09T03:46:22.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.609+0000 7fd8d2ffd640 1 -- 172.21.15.76:0/1900709689 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d80052f0 con 0x7fd8c8005180 2023-11-09T03:46:22.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.609+0000 7fd8e4355640 1 -- 172.21.15.76:0/1900709689 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8dc157430 con 0x7fd8c8005180 2023-11-09T03:46:22.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.610+0000 7fd8e4355640 1 -- 172.21.15.76:0/1900709689 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8dc074290 con 0x7fd8c8005180 2023-11-09T03:46:22.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.611+0000 7fd8d2ffd640 1 -- 172.21.15.76:0/1900709689 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd8d8005450 con 0x7fd8c8005180 2023-11-09T03:46:22.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.611+0000 7fd8d2ffd640 1 --2- 172.21.15.76:0/1900709689 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd8c006aea0 0x7fd8c006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:22.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.611+0000 7fd8d2ffd640 1 -- 172.21.15.76:0/1900709689 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fd8d8089aa0 con 0x7fd8c8005180 2023-11-09T03:46:22.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.611+0000 7fd8e18f0640 1 --2- 172.21.15.76:0/1900709689 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd8c006aea0 0x7fd8c006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:22.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.612+0000 7fd8e18f0640 1 --2- 172.21.15.76:0/1900709689 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd8c006aea0 0x7fd8c006d360 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc06ddb0 tx=0x7fd8cc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:22.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.615+0000 7fd8d2ffd640 1 -- 172.21.15.76:0/1900709689 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd8d8054770 con 0x7fd8c8005180 2023-11-09T03:46:22.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.783+0000 7fd8e4355640 1 -- 172.21.15.76:0/1900709689 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd8dc071970 con 0x7fd8c8005180 2023-11-09T03:46:22.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.783+0000 7fd8d2ffd640 1 -- 172.21.15.76:0/1900709689 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd8d8057d90 con 0x7fd8c8005180 2023-11-09T03:46:22.783 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:46:22.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.785+0000 7fd8e4355640 1 -- 172.21.15.76:0/1900709689 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd8c006aea0 msgr2=0x7fd8c006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:22.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.785+0000 7fd8e4355640 1 --2- 172.21.15.76:0/1900709689 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd8c006aea0 0x7fd8c006d360 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc06ddb0 tx=0x7fd8cc009190 comp rx=0 tx=0).stop 2023-11-09T03:46:22.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.785+0000 7fd8e4355640 1 -- 172.21.15.76:0/1900709689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8c8005180 msgr2=0x7fd8dc156ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:22.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.785+0000 7fd8e4355640 1 --2- 172.21.15.76:0/1900709689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8c8005180 0x7fd8dc156ef0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd8d800f960 tx=0x7fd8d800fd60 comp rx=0 tx=0).stop 2023-11-09T03:46:22.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.785+0000 7fd8e4355640 1 -- 172.21.15.76:0/1900709689 shutdown_connections 2023-11-09T03:46:22.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.785+0000 7fd8e4355640 1 --2- 172.21.15.76:0/1900709689 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd8c006aea0 0x7fd8c006d360 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:22.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.785+0000 7fd8e4355640 1 --2- 172.21.15.76:0/1900709689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8dc158d20 0x7fd8dc1591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:22.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.785+0000 7fd8e4355640 1 --2- 172.21.15.76:0/1900709689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8dc06d1a0 0x7fd8dc06d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:22.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.785+0000 7fd8e4355640 1 --2- 172.21.15.76:0/1900709689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8c8005180 0x7fd8dc156ef0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:22.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.785+0000 7fd8e4355640 1 -- 172.21.15.76:0/1900709689 >> 172.21.15.76:0/1900709689 conn(0x7fd8dc0ce950 msgr2=0x7fd8dc1553f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:22.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.785+0000 7fd8e4355640 1 -- 172.21.15.76:0/1900709689 shutdown_connections 2023-11-09T03:46:22.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.785+0000 7fd8e4355640 1 -- 172.21.15.76:0/1900709689 wait complete. 2023-11-09T03:46:22.795 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:46:22.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.898+0000 7f73ddcd4640 1 Processor -- start 2023-11-09T03:46:22.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.898+0000 7f73ddcd4640 1 -- start start 2023-11-09T03:46:22.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.899+0000 7f73ddcd4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f73d8159c00 0x7f73d80cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:22.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.899+0000 7f73ddcd4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f73d80c48e0 0x7f73d80c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:22.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.899+0000 7f73ddcd4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73d815aff0 con 0x7f73d80adac0 2023-11-09T03:46:22.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.899+0000 7f73ddcd4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d80c52d0 con 0x7f73d80c48e0 2023-11-09T03:46:22.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.899+0000 7f73ddcd4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d815a3e0 con 0x7f73d8159c00 2023-11-09T03:46:22.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.899+0000 7f73d6ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f73d80c48e0 0x7f73d80c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:22.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.899+0000 7f73d7fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f73d80adac0 0x7f73d80adea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53014/0 (socket says 172.21.15.76:53014) 2023-11-09T03:46:22.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.899+0000 7f73d7fff640 1 -- 172.21.15.76:0/2081536677 learned_addr learned my addr 172.21.15.76:0/2081536677 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:22.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.899+0000 7f73d77fe640 1 --2- 172.21.15.76:0/2081536677 >> v2:172.21.15.113:3300/0 conn(0x7f73d8159c00 0x7f73d80cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:22.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.900+0000 7f73d6ffd640 1 -- 172.21.15.76:0/2081536677 >> v2:172.21.15.113:3300/0 conn(0x7f73d8159c00 msgr2=0x7f73d80cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:22.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.900+0000 7f73d6ffd640 1 --2- 172.21.15.76:0/2081536677 >> v2:172.21.15.113:3300/0 conn(0x7f73d8159c00 0x7f73d80cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:22.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.900+0000 7f73d6ffd640 1 -- 172.21.15.76:0/2081536677 >> v1:172.21.15.76:6789/0 conn(0x7f73d80adac0 legacy=0x7f73d80adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:22.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.900+0000 7f73d67fc640 1 -- 172.21.15.76:0/2081536677 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1242419376 0 0) 0x7f73d815aff0 con 0x7f73d80adac0 2023-11-09T03:46:22.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.900+0000 7f73d6ffd640 1 -- 172.21.15.76:0/2081536677 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d815a6c0 con 0x7f73d80c48e0 2023-11-09T03:46:22.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.900+0000 7f73d6ffd640 1 --2- 172.21.15.76:0/2081536677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73d80c48e0 0x7f73d80c4d90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f73d80cc4d0 tx=0x7f73cc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c34face43a889c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:22.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.900+0000 7f73d67fc640 1 -- 172.21.15.76:0/2081536677 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc02e070 con 0x7f73d80c48e0 2023-11-09T03:46:22.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.900+0000 7f73d67fc640 1 -- 172.21.15.76:0/2081536677 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73cc0315a0 con 0x7f73d80c48e0 2023-11-09T03:46:22.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.901+0000 7f73d67fc640 1 -- 172.21.15.76:0/2081536677 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc031930 con 0x7f73d80c48e0 2023-11-09T03:46:22.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.901+0000 7f73ddcd4640 1 -- 172.21.15.76:0/2081536677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73d80c48e0 msgr2=0x7f73d80c4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:22.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.901+0000 7f73ddcd4640 1 --2- 172.21.15.76:0/2081536677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73d80c48e0 0x7f73d80c4d90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f73d80cc4d0 tx=0x7f73cc02b840 comp rx=0 tx=0).stop 2023-11-09T03:46:22.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.901+0000 7f73ddcd4640 1 -- 172.21.15.76:0/2081536677 shutdown_connections 2023-11-09T03:46:22.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.901+0000 7f73ddcd4640 1 --2- 172.21.15.76:0/2081536677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73d80c48e0 0x7f73d80c4d90 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:22.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.901+0000 7f73ddcd4640 1 --2- 172.21.15.76:0/2081536677 >> v2:172.21.15.113:3300/0 conn(0x7f73d8159c00 0x7f73d80cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:22.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.901+0000 7f73ddcd4640 1 -- 172.21.15.76:0/2081536677 >> 172.21.15.76:0/2081536677 conn(0x7f73d80ce950 msgr2=0x7f73d80ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:22.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.901+0000 7f73ddcd4640 1 -- 172.21.15.76:0/2081536677 shutdown_connections 2023-11-09T03:46:22.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.901+0000 7f73ddcd4640 1 -- 172.21.15.76:0/2081536677 wait complete. 2023-11-09T03:46:22.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.902+0000 7f73ddcd4640 1 Processor -- start 2023-11-09T03:46:22.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.902+0000 7f73ddcd4640 1 -- start start 2023-11-09T03:46:22.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.903+0000 7f73ddcd4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73d80adac0 0x7f73d814b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:22.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.903+0000 7f73ddcd4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73d80c48e0 0x7f73d80c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:22.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.903+0000 7f73ddcd4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73d8159c00 0x7f73d80c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:22.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.903+0000 7f73ddcd4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d80ab930 con 0x7f73d80c48e0 2023-11-09T03:46:22.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.903+0000 7f73ddcd4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d80a8440 con 0x7f73d8159c00 2023-11-09T03:46:22.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.903+0000 7f73ddcd4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d80a8720 con 0x7f73d80adac0 2023-11-09T03:46:22.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.903+0000 7f73d6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73d80c48e0 0x7f73d80c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:22.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.903+0000 7f73d6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73d80c48e0 0x7f73d80c7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35652/0 (socket says 172.21.15.76:35652) 2023-11-09T03:46:22.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.903+0000 7f73d6ffd640 1 -- 172.21.15.76:0/324879868 learned_addr learned my addr 172.21.15.76:0/324879868 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:22.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.903+0000 7f73d6ffd640 1 -- 172.21.15.76:0/324879868 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73d80adac0 msgr2=0x7f73d814b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:22.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.903+0000 7f73d6ffd640 1 --2- 172.21.15.76:0/324879868 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73d80adac0 0x7f73d814b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:22.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.903+0000 7f73d6ffd640 1 -- 172.21.15.76:0/324879868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73d8159c00 msgr2=0x7f73d80c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:22.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.903+0000 7f73d6ffd640 1 --2- 172.21.15.76:0/324879868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73d8159c00 0x7f73d80c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:22.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.904+0000 7f73d6ffd640 1 -- 172.21.15.76:0/324879868 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73cc009400 con 0x7f73d80c48e0 2023-11-09T03:46:22.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.904+0000 7f73d6ffd640 1 --2- 172.21.15.76:0/324879868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73d80c48e0 0x7f73d80c7b30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009890 tx=0x7f73cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:22.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.904+0000 7f73bffff640 1 -- 172.21.15.76:0/324879868 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc032840 con 0x7f73d80c48e0 2023-11-09T03:46:22.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.904+0000 7f73bffff640 1 -- 172.21.15.76:0/324879868 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73cc0329a0 con 0x7f73d80c48e0 2023-11-09T03:46:22.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.904+0000 7f73ddcd4640 1 -- 172.21.15.76:0/324879868 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d80a8a00 con 0x7f73d80c48e0 2023-11-09T03:46:22.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.904+0000 7f73bffff640 1 -- 172.21.15.76:0/324879868 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc032d50 con 0x7f73d80c48e0 2023-11-09T03:46:22.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.904+0000 7f73ddcd4640 1 -- 172.21.15.76:0/324879868 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d80a7690 con 0x7f73d80c48e0 2023-11-09T03:46:22.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.905+0000 7f73ddcd4640 1 -- 172.21.15.76:0/324879868 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d80a8f20 con 0x7f73d80c48e0 2023-11-09T03:46:22.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.905+0000 7f73bffff640 1 -- 172.21.15.76:0/324879868 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f73cc03b3f0 con 0x7f73d80c48e0 2023-11-09T03:46:22.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.906+0000 7f73bffff640 1 --2- 172.21.15.76:0/324879868 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f73c406ae70 0x7f73c406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:22.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.906+0000 7f73bffff640 1 -- 172.21.15.76:0/324879868 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f73cc0aba90 con 0x7f73d80c48e0 2023-11-09T03:46:22.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.906+0000 7f73d77fe640 1 --2- 172.21.15.76:0/324879868 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f73c406ae70 0x7f73c406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:22.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.906+0000 7f73d77fe640 1 --2- 172.21.15.76:0/324879868 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f73c406ae70 0x7f73c406d330 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f73c8006fd0 tx=0x7f73c8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:22.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:22.910+0000 7f73bffff640 1 -- 172.21.15.76:0/324879868 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f73d80a8f20 con 0x7f73d80c48e0 2023-11-09T03:46:23.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.076+0000 7f73ddcd4640 1 -- 172.21.15.76:0/324879868 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f73d80a8f20 con 0x7f73d80c48e0 2023-11-09T03:46:23.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.076+0000 7f73bffff640 1 -- 172.21.15.76:0/324879868 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f73d80a8f20 con 0x7f73d80c48e0 2023-11-09T03:46:23.076 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:46:23.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.078+0000 7f73ddcd4640 1 -- 172.21.15.76:0/324879868 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f73c406ae70 msgr2=0x7f73c406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:23.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.078+0000 7f73ddcd4640 1 --2- 172.21.15.76:0/324879868 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f73c406ae70 0x7f73c406d330 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f73c8006fd0 tx=0x7f73c8008040 comp rx=0 tx=0).stop 2023-11-09T03:46:23.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.078+0000 7f73ddcd4640 1 -- 172.21.15.76:0/324879868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73d80c48e0 msgr2=0x7f73d80c7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:23.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.078+0000 7f73ddcd4640 1 --2- 172.21.15.76:0/324879868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73d80c48e0 0x7f73d80c7b30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009890 tx=0x7f73cc0046c0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.078+0000 7f73ddcd4640 1 -- 172.21.15.76:0/324879868 shutdown_connections 2023-11-09T03:46:23.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.078+0000 7f73ddcd4640 1 --2- 172.21.15.76:0/324879868 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f73c406ae70 0x7f73c406d330 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.078+0000 7f73ddcd4640 1 --2- 172.21.15.76:0/324879868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73d8159c00 0x7f73d80c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.078+0000 7f73ddcd4640 1 --2- 172.21.15.76:0/324879868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73d80c48e0 0x7f73d80c7b30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.078+0000 7f73ddcd4640 1 --2- 172.21.15.76:0/324879868 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73d80adac0 0x7f73d814b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.078+0000 7f73ddcd4640 1 -- 172.21.15.76:0/324879868 >> 172.21.15.76:0/324879868 conn(0x7f73d80ce950 msgr2=0x7f73d80cd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:23.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.078+0000 7f73ddcd4640 1 -- 172.21.15.76:0/324879868 shutdown_connections 2023-11-09T03:46:23.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.078+0000 7f73ddcd4640 1 -- 172.21.15.76:0/324879868 wait complete. 2023-11-09T03:46:23.088 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:46:23.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.192+0000 7f118e861640 1 Processor -- start 2023-11-09T03:46:23.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.192+0000 7f118e861640 1 -- start start 2023-11-09T03:46:23.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.193+0000 7f118e861640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f118815e430 0x7f11880cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:23.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.193+0000 7f118e861640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f11880cf160 0x7f11880cd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:23.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.193+0000 7f118e861640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f11880d0710 0x7f11880cda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:23.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.193+0000 7f118e861640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11880cdf90 con 0x7f11880cf160 2023-11-09T03:46:23.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.193+0000 7f118e861640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11880d0280 con 0x7f11880d0710 2023-11-09T03:46:23.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.193+0000 7f118e861640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11880cf670 con 0x7f118815e430 2023-11-09T03:46:23.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.193+0000 7f11877fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f11880cf160 0x7f11880cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:23.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.193+0000 7f11877fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f11880cf160 0x7f11880cd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35666/0 (socket says 172.21.15.76:35666) 2023-11-09T03:46:23.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.193+0000 7f11877fe640 1 -- 172.21.15.76:0/369249543 learned_addr learned my addr 172.21.15.76:0/369249543 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:23.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.193+0000 7f1187fff640 1 --2- 172.21.15.76:0/369249543 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f118815e430 0x7f11880cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:23.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.193+0000 7f118cdfe640 1 --2- 172.21.15.76:0/369249543 >> v2:172.21.15.113:3300/0 conn(0x7f11880d0710 0x7f11880cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:23.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.193+0000 7f11877fe640 1 -- 172.21.15.76:0/369249543 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f118815e430 msgr2=0x7f11880cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:23.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.193+0000 7f11877fe640 1 --2- 172.21.15.76:0/369249543 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f118815e430 0x7f11880cec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.193+0000 7f11877fe640 1 -- 172.21.15.76:0/369249543 >> v2:172.21.15.113:3300/0 conn(0x7f11880d0710 msgr2=0x7f11880cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:23.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.193+0000 7f11877fe640 1 --2- 172.21.15.76:0/369249543 >> v2:172.21.15.113:3300/0 conn(0x7f11880d0710 0x7f11880cda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.193+0000 7f11877fe640 1 -- 172.21.15.76:0/369249543 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11880cf950 con 0x7f11880cf160 2023-11-09T03:46:23.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.193+0000 7f1187fff640 1 --2- 172.21.15.76:0/369249543 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f118815e430 0x7f11880cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:23.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.193+0000 7f118cdfe640 1 --2- 172.21.15.76:0/369249543 >> v2:172.21.15.113:3300/0 conn(0x7f11880d0710 0x7f11880cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:23.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.194+0000 7f11877fe640 1 --2- 172.21.15.76:0/369249543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11880cf160 0x7f11880cd510 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f118806f860 tx=0x7f117c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccbcd406cf184186 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:23.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.194+0000 7f1186ffd640 1 -- 172.21.15.76:0/369249543 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c031870 con 0x7f11880cf160 2023-11-09T03:46:23.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.194+0000 7f1186ffd640 1 -- 172.21.15.76:0/369249543 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c0319d0 con 0x7f11880cf160 2023-11-09T03:46:23.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.194+0000 7f1186ffd640 1 -- 172.21.15.76:0/369249543 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c031d60 con 0x7f11880cf160 2023-11-09T03:46:23.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.195+0000 7f118e861640 1 -- 172.21.15.76:0/369249543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11880cf160 msgr2=0x7f11880cd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:23.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.195+0000 7f118e861640 1 --2- 172.21.15.76:0/369249543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11880cf160 0x7f11880cd510 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f118806f860 tx=0x7f117c02b910 comp rx=0 tx=0).stop 2023-11-09T03:46:23.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.195+0000 7f118e861640 1 -- 172.21.15.76:0/369249543 shutdown_connections 2023-11-09T03:46:23.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.195+0000 7f118e861640 1 --2- 172.21.15.76:0/369249543 >> v2:172.21.15.113:3300/0 conn(0x7f11880d0710 0x7f11880cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.195+0000 7f118e861640 1 --2- 172.21.15.76:0/369249543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11880cf160 0x7f11880cd510 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.195+0000 7f118e861640 1 --2- 172.21.15.76:0/369249543 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f118815e430 0x7f11880cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.195+0000 7f118e861640 1 -- 172.21.15.76:0/369249543 >> 172.21.15.76:0/369249543 conn(0x7f118806ea20 msgr2=0x7f118806ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:23.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.195+0000 7f118e861640 1 -- 172.21.15.76:0/369249543 shutdown_connections 2023-11-09T03:46:23.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.195+0000 7f118e861640 1 -- 172.21.15.76:0/369249543 wait complete. 2023-11-09T03:46:23.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.196+0000 7f118e861640 1 Processor -- start 2023-11-09T03:46:23.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.196+0000 7f118e861640 1 -- start start 2023-11-09T03:46:23.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.197+0000 7f118e861640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11880cf160 0x7f118815cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:23.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.197+0000 7f118e861640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11880d0710 0x7f11880c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:23.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.197+0000 7f118e861640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f118815e430 0x7f1188157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:23.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.197+0000 7f118e861640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11881f3e80 con 0x7f11880d0710 2023-11-09T03:46:23.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.197+0000 7f118e861640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11881f4160 con 0x7f118815e430 2023-11-09T03:46:23.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.197+0000 7f118e861640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11881f4440 con 0x7f11880cf160 2023-11-09T03:46:23.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.197+0000 7f11877fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11880d0710 0x7f11880c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:23.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.197+0000 7f11877fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11880d0710 0x7f11880c96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35678/0 (socket says 172.21.15.76:35678) 2023-11-09T03:46:23.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.197+0000 7f11877fe640 1 -- 172.21.15.76:0/2458677335 learned_addr learned my addr 172.21.15.76:0/2458677335 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:23.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.197+0000 7f118cdfe640 1 --2- 172.21.15.76:0/2458677335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f118815e430 0x7f1188157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:23.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.197+0000 7f1187fff640 1 --2- 172.21.15.76:0/2458677335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11880cf160 0x7f118815cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:23.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.197+0000 7f11877fe640 1 -- 172.21.15.76:0/2458677335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11880cf160 msgr2=0x7f118815cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:23.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.197+0000 7f11877fe640 1 --2- 172.21.15.76:0/2458677335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11880cf160 0x7f118815cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.197+0000 7f11877fe640 1 -- 172.21.15.76:0/2458677335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f118815e430 msgr2=0x7f1188157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:23.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.197+0000 7f11877fe640 1 --2- 172.21.15.76:0/2458677335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f118815e430 0x7f1188157a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.197+0000 7f11877fe640 1 -- 172.21.15.76:0/2458677335 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c0094d0 con 0x7f11880d0710 2023-11-09T03:46:23.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.197+0000 7f118cdfe640 1 --2- 172.21.15.76:0/2458677335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f118815e430 0x7f1188157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:23.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.197+0000 7f1187fff640 1 --2- 172.21.15.76:0/2458677335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11880cf160 0x7f118815cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:23.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.197+0000 7f11877fe640 1 --2- 172.21.15.76:0/2458677335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11880d0710 0x7f11880c96c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f117c02b950 tx=0x7f117c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:23.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.198+0000 7f1184ff9640 1 -- 172.21.15.76:0/2458677335 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c031860 con 0x7f11880d0710 2023-11-09T03:46:23.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.198+0000 7f1184ff9640 1 -- 172.21.15.76:0/2458677335 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c0443f0 con 0x7f11880d0710 2023-11-09T03:46:23.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.198+0000 7f118e861640 1 -- 172.21.15.76:0/2458677335 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11881f4720 con 0x7f11880d0710 2023-11-09T03:46:23.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.198+0000 7f1184ff9640 1 -- 172.21.15.76:0/2458677335 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c0447a0 con 0x7f11880d0710 2023-11-09T03:46:23.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.198+0000 7f118e861640 1 -- 172.21.15.76:0/2458677335 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f118815d0f0 con 0x7f11880d0710 2023-11-09T03:46:23.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.199+0000 7f118e861640 1 -- 172.21.15.76:0/2458677335 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11881f48b0 con 0x7f11880d0710 2023-11-09T03:46:23.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.199+0000 7f1184ff9640 1 -- 172.21.15.76:0/2458677335 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f117c04d450 con 0x7f11880d0710 2023-11-09T03:46:23.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.199+0000 7f1184ff9640 1 --2- 172.21.15.76:0/2458677335 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f117406ae70 0x7f117406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:23.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.200+0000 7f1184ff9640 1 -- 172.21.15.76:0/2458677335 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f117c02d040 con 0x7f11880d0710 2023-11-09T03:46:23.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.200+0000 7f1187fff640 1 --2- 172.21.15.76:0/2458677335 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f117406ae70 0x7f117406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:23.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.200+0000 7f1187fff640 1 --2- 172.21.15.76:0/2458677335 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f117406ae70 0x7f117406d330 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f11880b4590 tx=0x7f1178008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:23.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.204+0000 7f1184ff9640 1 -- 172.21.15.76:0/2458677335 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f117c076e20 con 0x7f11880d0710 2023-11-09T03:46:23.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.370+0000 7f118e861640 1 -- 172.21.15.76:0/2458677335 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f11881f4580 con 0x7f11880d0710 2023-11-09T03:46:23.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.370+0000 7f1184ff9640 1 -- 172.21.15.76:0/2458677335 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f11881f4580 con 0x7f11880d0710 2023-11-09T03:46:23.370 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:46:23.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.372+0000 7f118e861640 1 -- 172.21.15.76:0/2458677335 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f117406ae70 msgr2=0x7f117406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:23.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.372+0000 7f118e861640 1 --2- 172.21.15.76:0/2458677335 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f117406ae70 0x7f117406d330 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f11880b4590 tx=0x7f1178008040 comp rx=0 tx=0).stop 2023-11-09T03:46:23.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.372+0000 7f118e861640 1 -- 172.21.15.76:0/2458677335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11880d0710 msgr2=0x7f11880c96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:23.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.372+0000 7f118e861640 1 --2- 172.21.15.76:0/2458677335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11880d0710 0x7f11880c96c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f117c02b950 tx=0x7f117c0046c0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.372+0000 7f118e861640 1 -- 172.21.15.76:0/2458677335 shutdown_connections 2023-11-09T03:46:23.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.372+0000 7f118e861640 1 --2- 172.21.15.76:0/2458677335 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f117406ae70 0x7f117406d330 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.372+0000 7f118e861640 1 --2- 172.21.15.76:0/2458677335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f118815e430 0x7f1188157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.372+0000 7f118e861640 1 --2- 172.21.15.76:0/2458677335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11880d0710 0x7f11880c96c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.372+0000 7f118e861640 1 --2- 172.21.15.76:0/2458677335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f11880cf160 0x7f118815cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.372+0000 7f118e861640 1 -- 172.21.15.76:0/2458677335 >> 172.21.15.76:0/2458677335 conn(0x7f118806ea20 msgr2=0x7f11880cb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:23.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.372+0000 7f118e861640 1 -- 172.21.15.76:0/2458677335 shutdown_connections 2023-11-09T03:46:23.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.372+0000 7f118e861640 1 -- 172.21.15.76:0/2458677335 wait complete. 2023-11-09T03:46:23.382 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-09T03:46:23.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.488+0000 7fed79373640 1 Processor -- start 2023-11-09T03:46:23.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.488+0000 7fed79373640 1 -- start start 2023-11-09T03:46:23.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.488+0000 7fed79373640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fed74160430 0x7fed740b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:23.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.488+0000 7fed79373640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed7406c3a0 0x7fed740cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:23.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.488+0000 7fed79373640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed740b2fc0 con 0x7fed740b2b60 2023-11-09T03:46:23.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.488+0000 7fed79373640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed740cb580 con 0x7fed74160430 2023-11-09T03:46:23.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.488+0000 7fed79373640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7406c7f0 con 0x7fed7406c3a0 2023-11-09T03:46:23.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.489+0000 7fed737fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed7406c3a0 0x7fed740cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:23.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.489+0000 7fed72ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fed74160430 0x7fed740b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:23.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.489+0000 7fed727fc640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fed740b2b60 0x7fed7406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:48138/0 (socket says 172.21.15.76:48138) 2023-11-09T03:46:23.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.489+0000 7fed727fc640 1 -- 172.21.15.76:0/2237059266 learned_addr learned my addr 172.21.15.76:0/2237059266 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:23.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.489+0000 7fed72ffd640 1 -- 172.21.15.76:0/2237059266 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed7406c3a0 msgr2=0x7fed740cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:23.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.489+0000 7fed72ffd640 1 --2- 172.21.15.76:0/2237059266 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed7406c3a0 0x7fed740cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.489+0000 7fed72ffd640 1 -- 172.21.15.76:0/2237059266 >> v1:172.21.15.113:6789/0 conn(0x7fed740b2b60 legacy=0x7fed7406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:23.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.489+0000 7fed72ffd640 1 -- 172.21.15.76:0/2237059266 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7406ca40 con 0x7fed74160430 2023-11-09T03:46:23.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.489+0000 7fed737fe640 1 --2- 172.21.15.76:0/2237059266 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed7406c3a0 0x7fed740cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:46:23.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.490+0000 7fed72ffd640 1 --2- 172.21.15.76:0/2237059266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed74160430 0x7fed740b2620 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fed640097a0 tx=0x7fed6402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a9dffb28b85a4a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:23.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.490+0000 7fed71ffb640 1 -- 172.21.15.76:0/2237059266 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64031830 con 0x7fed74160430 2023-11-09T03:46:23.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.490+0000 7fed71ffb640 1 -- 172.21.15.76:0/2237059266 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed64031990 con 0x7fed74160430 2023-11-09T03:46:23.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.490+0000 7fed71ffb640 1 -- 172.21.15.76:0/2237059266 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64031d20 con 0x7fed74160430 2023-11-09T03:46:23.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.491+0000 7fed79373640 1 -- 172.21.15.76:0/2237059266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed74160430 msgr2=0x7fed740b2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:23.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.491+0000 7fed79373640 1 --2- 172.21.15.76:0/2237059266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed74160430 0x7fed740b2620 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fed640097a0 tx=0x7fed6402b8d0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.491+0000 7fed79373640 1 -- 172.21.15.76:0/2237059266 shutdown_connections 2023-11-09T03:46:23.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.491+0000 7fed79373640 1 --2- 172.21.15.76:0/2237059266 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed7406c3a0 0x7fed740cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.491+0000 7fed79373640 1 --2- 172.21.15.76:0/2237059266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed74160430 0x7fed740b2620 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.491+0000 7fed79373640 1 -- 172.21.15.76:0/2237059266 >> 172.21.15.76:0/2237059266 conn(0x7fed7406ea20 msgr2=0x7fed7406ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:23.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.491+0000 7fed79373640 1 -- 172.21.15.76:0/2237059266 shutdown_connections 2023-11-09T03:46:23.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.491+0000 7fed79373640 1 -- 172.21.15.76:0/2237059266 wait complete. 2023-11-09T03:46:23.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.492+0000 7fed79373640 1 Processor -- start 2023-11-09T03:46:23.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.492+0000 7fed79373640 1 -- start start 2023-11-09T03:46:23.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.493+0000 7fed79373640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed7406c3a0 0x7fed7415ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:23.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.493+0000 7fed79373640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed740b2b60 0x7fed741f3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:23.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.493+0000 7fed79373640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed74160430 0x7fed7415edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:23.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.493+0000 7fed79373640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed740c5c30 con 0x7fed740b2b60 2023-11-09T03:46:23.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.493+0000 7fed79373640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed740c5f10 con 0x7fed7406c3a0 2023-11-09T03:46:23.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.493+0000 7fed79373640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7415c320 con 0x7fed74160430 2023-11-09T03:46:23.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.493+0000 7fed727fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed740b2b60 0x7fed741f3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:23.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.493+0000 7fed727fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed740b2b60 0x7fed741f3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35688/0 (socket says 172.21.15.76:35688) 2023-11-09T03:46:23.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.493+0000 7fed727fc640 1 -- 172.21.15.76:0/2504131902 learned_addr learned my addr 172.21.15.76:0/2504131902 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:23.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.493+0000 7fed72ffd640 1 --2- 172.21.15.76:0/2504131902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed7406c3a0 0x7fed7415ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:23.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.493+0000 7fed737fe640 1 --2- 172.21.15.76:0/2504131902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed74160430 0x7fed7415edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:23.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.493+0000 7fed727fc640 1 -- 172.21.15.76:0/2504131902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed74160430 msgr2=0x7fed7415edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:23.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.493+0000 7fed727fc640 1 --2- 172.21.15.76:0/2504131902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed74160430 0x7fed7415edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.493+0000 7fed727fc640 1 -- 172.21.15.76:0/2504131902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed7406c3a0 msgr2=0x7fed7415ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:23.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.493+0000 7fed727fc640 1 --2- 172.21.15.76:0/2504131902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed7406c3a0 0x7fed7415ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.493+0000 7fed727fc640 1 -- 172.21.15.76:0/2504131902 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6c009a70 con 0x7fed740b2b60 2023-11-09T03:46:23.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.493+0000 7fed727fc640 1 --2- 172.21.15.76:0/2504131902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed740b2b60 0x7fed741f3490 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fed6800c4a0 tx=0x7fed6800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:23.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.494+0000 7fed5b7fe640 1 -- 172.21.15.76:0/2504131902 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed68005ce0 con 0x7fed740b2b60 2023-11-09T03:46:23.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.494+0000 7fed5b7fe640 1 -- 172.21.15.76:0/2504131902 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed68005e40 con 0x7fed740b2b60 2023-11-09T03:46:23.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.494+0000 7fed5b7fe640 1 -- 172.21.15.76:0/2504131902 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed680072d0 con 0x7fed740b2b60 2023-11-09T03:46:23.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.494+0000 7fed79373640 1 -- 172.21.15.76:0/2504131902 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed64009400 con 0x7fed740b2b60 2023-11-09T03:46:23.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.494+0000 7fed79373640 1 -- 172.21.15.76:0/2504131902 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7415c8e0 con 0x7fed740b2b60 2023-11-09T03:46:23.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.495+0000 7fed79373640 1 -- 172.21.15.76:0/2504131902 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed74074290 con 0x7fed740b2b60 2023-11-09T03:46:23.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.495+0000 7fed5b7fe640 1 -- 172.21.15.76:0/2504131902 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fed680074f0 con 0x7fed740b2b60 2023-11-09T03:46:23.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.495+0000 7fed5b7fe640 1 --2- 172.21.15.76:0/2504131902 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fed6006ae70 0x7fed6006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:23.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.496+0000 7fed5b7fe640 1 -- 172.21.15.76:0/2504131902 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fed68088e20 con 0x7fed740b2b60 2023-11-09T03:46:23.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.496+0000 7fed72ffd640 1 --2- 172.21.15.76:0/2504131902 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fed6006ae70 0x7fed6006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:23.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.496+0000 7fed72ffd640 1 --2- 172.21.15.76:0/2504131902 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fed6006ae70 0x7fed6006d330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fed64036000 tx=0x7fed6403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:23.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.500+0000 7fed5b7fe640 1 -- 172.21.15.76:0/2504131902 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fed68053ba0 con 0x7fed740b2b60 2023-11-09T03:46:23.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.668+0000 7fed79373640 1 -- 172.21.15.76:0/2504131902 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fed740b6b20 con 0x7fed740b2b60 2023-11-09T03:46:23.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.668+0000 7fed5b7fe640 1 -- 172.21.15.76:0/2504131902 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fed680170a0 con 0x7fed740b2b60 2023-11-09T03:46:23.668 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:46:23.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.670+0000 7fed79373640 1 -- 172.21.15.76:0/2504131902 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fed6006ae70 msgr2=0x7fed6006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:23.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.670+0000 7fed79373640 1 --2- 172.21.15.76:0/2504131902 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fed6006ae70 0x7fed6006d330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fed64036000 tx=0x7fed6403c040 comp rx=0 tx=0).stop 2023-11-09T03:46:23.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.670+0000 7fed79373640 1 -- 172.21.15.76:0/2504131902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed740b2b60 msgr2=0x7fed741f3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:23.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.670+0000 7fed79373640 1 --2- 172.21.15.76:0/2504131902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed740b2b60 0x7fed741f3490 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fed6800c4a0 tx=0x7fed6800c8a0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.670+0000 7fed79373640 1 -- 172.21.15.76:0/2504131902 shutdown_connections 2023-11-09T03:46:23.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.670+0000 7fed79373640 1 --2- 172.21.15.76:0/2504131902 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fed6006ae70 0x7fed6006d330 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.670+0000 7fed79373640 1 --2- 172.21.15.76:0/2504131902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed74160430 0x7fed7415edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.670+0000 7fed79373640 1 --2- 172.21.15.76:0/2504131902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed740b2b60 0x7fed741f3490 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.670+0000 7fed79373640 1 --2- 172.21.15.76:0/2504131902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed7406c3a0 0x7fed7415ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.670+0000 7fed79373640 1 -- 172.21.15.76:0/2504131902 >> 172.21.15.76:0/2504131902 conn(0x7fed7406ea20 msgr2=0x7fed7415b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:23.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.670+0000 7fed79373640 1 -- 172.21.15.76:0/2504131902 shutdown_connections 2023-11-09T03:46:23.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.670+0000 7fed79373640 1 -- 172.21.15.76:0/2504131902 wait complete. 2023-11-09T03:46:23.680 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-09T03:46:23.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.782+0000 7fdb7a78a640 1 Processor -- start 2023-11-09T03:46:23.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.782+0000 7fdb7a78a640 1 -- start start 2023-11-09T03:46:23.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.783+0000 7fdb7a78a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdb740d0710 0x7fdb740cd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:23.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.783+0000 7fdb7a78a640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb740d0280 con 0x7fdb740cda50 2023-11-09T03:46:23.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.783+0000 7fdb7a78a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb740cf670 con 0x7fdb7415e430 2023-11-09T03:46:23.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.783+0000 7fdb7a78a640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb740cf950 con 0x7fdb740d0710 2023-11-09T03:46:23.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.783+0000 7fdb737fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdb740d0710 0x7fdb740cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:23.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.783+0000 7fdb78d27640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fdb740cda50 0x7fdb740b3d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53020/0 (socket says 172.21.15.76:53020) 2023-11-09T03:46:23.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.783+0000 7fdb78d27640 1 -- 172.21.15.76:0/1743317753 learned_addr learned my addr 172.21.15.76:0/1743317753 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:23.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.784+0000 7fdb737fe640 1 -- 172.21.15.76:0/1743317753 >> v1:172.21.15.113:6789/0 conn(0x7fdb7415e430 legacy=0x7fdb740cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:23.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.784+0000 7fdb737fe640 1 -- 172.21.15.76:0/1743317753 >> v1:172.21.15.76:6789/0 conn(0x7fdb740cda50 legacy=0x7fdb740b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:23.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.784+0000 7fdb72ffd640 1 -- 172.21.15.76:0/1743317753 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4011739340 0 0) 0x7fdb740d0280 con 0x7fdb740cda50 2023-11-09T03:46:23.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.784+0000 7fdb737fe640 1 -- 172.21.15.76:0/1743317753 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb740cfc30 con 0x7fdb740d0710 2023-11-09T03:46:23.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.785+0000 7fdb737fe640 1 --2- 172.21.15.76:0/1743317753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb740d0710 0x7fdb740cd510 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdb680097a0 tx=0x7fdb6802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b3214b65d408de server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:23.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.785+0000 7fdb72ffd640 1 -- 172.21.15.76:0/1743317753 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb68030610 con 0x7fdb740d0710 2023-11-09T03:46:23.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.785+0000 7fdb72ffd640 1 -- 172.21.15.76:0/1743317753 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb68030770 con 0x7fdb740d0710 2023-11-09T03:46:23.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.785+0000 7fdb72ffd640 1 -- 172.21.15.76:0/1743317753 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb68030b00 con 0x7fdb740d0710 2023-11-09T03:46:23.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.785+0000 7fdb7a78a640 1 -- 172.21.15.76:0/1743317753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb740d0710 msgr2=0x7fdb740cd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:23.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.785+0000 7fdb7a78a640 1 --2- 172.21.15.76:0/1743317753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb740d0710 0x7fdb740cd510 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdb680097a0 tx=0x7fdb6802b960 comp rx=0 tx=0).stop 2023-11-09T03:46:23.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.785+0000 7fdb7a78a640 1 -- 172.21.15.76:0/1743317753 shutdown_connections 2023-11-09T03:46:23.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.785+0000 7fdb7a78a640 1 --2- 172.21.15.76:0/1743317753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb740d0710 0x7fdb740cd510 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.785+0000 7fdb7a78a640 1 -- 172.21.15.76:0/1743317753 >> 172.21.15.76:0/1743317753 conn(0x7fdb7406ea20 msgr2=0x7fdb7406ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:23.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.785+0000 7fdb7a78a640 1 -- 172.21.15.76:0/1743317753 shutdown_connections 2023-11-09T03:46:23.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.785+0000 7fdb7a78a640 1 -- 172.21.15.76:0/1743317753 wait complete. 2023-11-09T03:46:23.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.786+0000 7fdb7a78a640 1 Processor -- start 2023-11-09T03:46:23.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.787+0000 7fdb7a78a640 1 -- start start 2023-11-09T03:46:23.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.787+0000 7fdb7a78a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb740cda50 0x7fdb74157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:23.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.787+0000 7fdb7a78a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb740d0710 0x7fdb741f0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:23.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.787+0000 7fdb7a78a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb7415e430 0x7fdb741f0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:23.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.787+0000 7fdb7a78a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb740c8460 con 0x7fdb740d0710 2023-11-09T03:46:23.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.787+0000 7fdb7a78a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb740a75b0 con 0x7fdb7415e430 2023-11-09T03:46:23.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.787+0000 7fdb7a78a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb740b3490 con 0x7fdb740cda50 2023-11-09T03:46:23.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.787+0000 7fdb737fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb740d0710 0x7fdb741f0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:23.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.787+0000 7fdb737fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb740d0710 0x7fdb741f0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35716/0 (socket says 172.21.15.76:35716) 2023-11-09T03:46:23.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.787+0000 7fdb737fe640 1 -- 172.21.15.76:0/283957068 learned_addr learned my addr 172.21.15.76:0/283957068 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:23.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.788+0000 7fdb73fff640 1 --2- 172.21.15.76:0/283957068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb740cda50 0x7fdb74157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:23.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.788+0000 7fdb78d27640 1 --2- 172.21.15.76:0/283957068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb7415e430 0x7fdb741f0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:23.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.788+0000 7fdb737fe640 1 -- 172.21.15.76:0/283957068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb740cda50 msgr2=0x7fdb74157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:23.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.788+0000 7fdb737fe640 1 --2- 172.21.15.76:0/283957068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb740cda50 0x7fdb74157c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.788+0000 7fdb737fe640 1 -- 172.21.15.76:0/283957068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb7415e430 msgr2=0x7fdb741f0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:23.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.788+0000 7fdb737fe640 1 --2- 172.21.15.76:0/283957068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb7415e430 0x7fdb741f0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.788+0000 7fdb737fe640 1 -- 172.21.15.76:0/283957068 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb68009400 con 0x7fdb740d0710 2023-11-09T03:46:23.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.788+0000 7fdb73fff640 1 --2- 172.21.15.76:0/283957068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb740cda50 0x7fdb74157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:23.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.788+0000 7fdb78d27640 1 --2- 172.21.15.76:0/283957068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb7415e430 0x7fdb741f0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:23.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.788+0000 7fdb737fe640 1 --2- 172.21.15.76:0/283957068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb740d0710 0x7fdb741f0410 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fdb6802bf70 tx=0x7fdb680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:23.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.788+0000 7fdb70ff9640 1 -- 172.21.15.76:0/283957068 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6803e070 con 0x7fdb740d0710 2023-11-09T03:46:23.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.788+0000 7fdb70ff9640 1 -- 172.21.15.76:0/283957068 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb68039a10 con 0x7fdb740d0710 2023-11-09T03:46:23.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.788+0000 7fdb70ff9640 1 -- 172.21.15.76:0/283957068 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb68039da0 con 0x7fdb740d0710 2023-11-09T03:46:23.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.788+0000 7fdb7a78a640 1 -- 172.21.15.76:0/283957068 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb740af810 con 0x7fdb740d0710 2023-11-09T03:46:23.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.788+0000 7fdb7a78a640 1 -- 172.21.15.76:0/283957068 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb740c7da0 con 0x7fdb740d0710 2023-11-09T03:46:23.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.789+0000 7fdb7a78a640 1 -- 172.21.15.76:0/283957068 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb740c85a0 con 0x7fdb740d0710 2023-11-09T03:46:23.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.789+0000 7fdb70ff9640 1 -- 172.21.15.76:0/283957068 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdb6804b400 con 0x7fdb740d0710 2023-11-09T03:46:23.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.790+0000 7fdb70ff9640 1 --2- 172.21.15.76:0/283957068 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdb6006ae70 0x7fdb6006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:23.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.790+0000 7fdb70ff9640 1 -- 172.21.15.76:0/283957068 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fdb680abe80 con 0x7fdb740d0710 2023-11-09T03:46:23.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.790+0000 7fdb73fff640 1 --2- 172.21.15.76:0/283957068 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdb6006ae70 0x7fdb6006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:23.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.791+0000 7fdb73fff640 1 --2- 172.21.15.76:0/283957068 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdb6006ae70 0x7fdb6006d330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdb64007fd0 tx=0x7fdb64009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:23.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.794+0000 7fdb70ff9640 1 -- 172.21.15.76:0/283957068 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdb740c85a0 con 0x7fdb740d0710 2023-11-09T03:46:23.971 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.971+0000 7fdb7a78a640 1 -- 172.21.15.76:0/283957068 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdb740c85a0 con 0x7fdb740d0710 2023-11-09T03:46:23.971 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.971+0000 7fdb70ff9640 1 -- 172.21.15.76:0/283957068 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdb740c85a0 con 0x7fdb740d0710 2023-11-09T03:46:23.971 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2023-11-09T03:46:23.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.973+0000 7fdb7a78a640 1 -- 172.21.15.76:0/283957068 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdb6006ae70 msgr2=0x7fdb6006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:23.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.973+0000 7fdb7a78a640 1 --2- 172.21.15.76:0/283957068 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdb6006ae70 0x7fdb6006d330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdb64007fd0 tx=0x7fdb64009040 comp rx=0 tx=0).stop 2023-11-09T03:46:23.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.973+0000 7fdb7a78a640 1 -- 172.21.15.76:0/283957068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb740d0710 msgr2=0x7fdb741f0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:23.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.973+0000 7fdb7a78a640 1 --2- 172.21.15.76:0/283957068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb740d0710 0x7fdb741f0410 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fdb6802bf70 tx=0x7fdb680046c0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.973+0000 7fdb7a78a640 1 -- 172.21.15.76:0/283957068 shutdown_connections 2023-11-09T03:46:23.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.973+0000 7fdb7a78a640 1 --2- 172.21.15.76:0/283957068 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdb6006ae70 0x7fdb6006d330 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.973+0000 7fdb7a78a640 1 --2- 172.21.15.76:0/283957068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb7415e430 0x7fdb741f0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.973+0000 7fdb7a78a640 1 --2- 172.21.15.76:0/283957068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb740d0710 0x7fdb741f0410 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.973+0000 7fdb7a78a640 1 --2- 172.21.15.76:0/283957068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb740cda50 0x7fdb74157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:23.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.973+0000 7fdb7a78a640 1 -- 172.21.15.76:0/283957068 >> 172.21.15.76:0/283957068 conn(0x7fdb7406ea20 msgr2=0x7fdb740b4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:23.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.973+0000 7fdb7a78a640 1 -- 172.21.15.76:0/283957068 shutdown_connections 2023-11-09T03:46:23.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:23.973+0000 7fdb7a78a640 1 -- 172.21.15.76:0/283957068 wait complete. 2023-11-09T03:46:23.983 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:46:23.984 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:46:23.999 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:46:24.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.140+0000 7f07215bb640 1 Processor -- start 2023-11-09T03:46:24.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.140+0000 7f07215bb640 1 -- start start 2023-11-09T03:46:24.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.141+0000 7f07215bb640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f071c0cde60 0x7f071c15d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:24.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.141+0000 7f07215bb640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f071c0ce240 0x7f071c0d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:24.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.141+0000 7f07215bb640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071c15e000 con 0x7f071c15daa0 2023-11-09T03:46:24.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.141+0000 7f07215bb640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f071c15e5c0 con 0x7f071c0ce240 2023-11-09T03:46:24.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.141+0000 7f07215bb640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f071c15e2e0 con 0x7f071c0cde60 2023-11-09T03:46:24.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.141+0000 7f071b7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f071c0ce240 0x7f071c0d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:24.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.141+0000 7f071b7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f071c0ce240 0x7f071c0d1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35728/0 (socket says 172.21.15.76:35728) 2023-11-09T03:46:24.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.141+0000 7f071b7fe640 1 -- 172.21.15.76:0/2531075560 learned_addr learned my addr 172.21.15.76:0/2531075560 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:24.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.141+0000 7f071affd640 1 --2- 172.21.15.76:0/2531075560 >> v2:172.21.15.113:3300/0 conn(0x7f071c0cde60 0x7f071c15d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:24.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.141+0000 7f071b7fe640 1 -- 172.21.15.76:0/2531075560 >> v2:172.21.15.113:3300/0 conn(0x7f071c0cde60 msgr2=0x7f071c15d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:24.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.141+0000 7f071b7fe640 1 --2- 172.21.15.76:0/2531075560 >> v2:172.21.15.113:3300/0 conn(0x7f071c0cde60 0x7f071c15d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.141+0000 7f071b7fe640 1 -- 172.21.15.76:0/2531075560 >> v1:172.21.15.113:6789/0 conn(0x7f071c15daa0 legacy=0x7f071c157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:24.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.141+0000 7f071b7fe640 1 -- 172.21.15.76:0/2531075560 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071c0d1120 con 0x7f071c0ce240 2023-11-09T03:46:24.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.142+0000 7f071b7fe640 1 --2- 172.21.15.76:0/2531075560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f071c0ce240 0x7f071c0d1920 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f071c0b5550 tx=0x7f071402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60432ae7b2608867 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:24.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.142+0000 7f0719ffb640 1 -- 172.21.15.76:0/2531075560 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071402cb20 con 0x7f071c0ce240 2023-11-09T03:46:24.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.142+0000 7f0719ffb640 1 -- 172.21.15.76:0/2531075560 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071402cc80 con 0x7f071c0ce240 2023-11-09T03:46:24.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.142+0000 7f0719ffb640 1 -- 172.21.15.76:0/2531075560 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07140343f0 con 0x7f071c0ce240 2023-11-09T03:46:24.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.143+0000 7f07215bb640 1 -- 172.21.15.76:0/2531075560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f071c0ce240 msgr2=0x7f071c0d1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:24.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.143+0000 7f07215bb640 1 --2- 172.21.15.76:0/2531075560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f071c0ce240 0x7f071c0d1920 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f071c0b5550 tx=0x7f071402bf80 comp rx=0 tx=0).stop 2023-11-09T03:46:24.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.143+0000 7f07215bb640 1 -- 172.21.15.76:0/2531075560 shutdown_connections 2023-11-09T03:46:24.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.143+0000 7f07215bb640 1 --2- 172.21.15.76:0/2531075560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f071c0ce240 0x7f071c0d1920 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.143+0000 7f07215bb640 1 --2- 172.21.15.76:0/2531075560 >> v2:172.21.15.113:3300/0 conn(0x7f071c0cde60 0x7f071c15d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.143+0000 7f07215bb640 1 -- 172.21.15.76:0/2531075560 >> 172.21.15.76:0/2531075560 conn(0x7f071c071f30 msgr2=0x7f071c0cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:24.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.143+0000 7f07215bb640 1 -- 172.21.15.76:0/2531075560 shutdown_connections 2023-11-09T03:46:24.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.143+0000 7f07215bb640 1 -- 172.21.15.76:0/2531075560 wait complete. 2023-11-09T03:46:24.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.144+0000 7f07215bb640 1 Processor -- start 2023-11-09T03:46:24.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.144+0000 7f07215bb640 1 -- start start 2023-11-09T03:46:24.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.145+0000 7f07215bb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f071c0cde60 0x7f071c154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:24.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.145+0000 7f07215bb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f071c0ce240 0x7f071c154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:24.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.145+0000 7f07215bb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f071c15daa0 0x7f071c0cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:24.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.145+0000 7f07215bb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c1f61e0 con 0x7f071c0cde60 2023-11-09T03:46:24.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.145+0000 7f07215bb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c1f64c0 con 0x7f071c0ce240 2023-11-09T03:46:24.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.145+0000 7f07215bb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071c1f67a0 con 0x7f071c15daa0 2023-11-09T03:46:24.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.145+0000 7f071affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f071c0cde60 0x7f071c154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:24.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.145+0000 7f071affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f071c0cde60 0x7f071c154d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35730/0 (socket says 172.21.15.76:35730) 2023-11-09T03:46:24.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.145+0000 7f071affd640 1 -- 172.21.15.76:0/1079246935 learned_addr learned my addr 172.21.15.76:0/1079246935 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:24.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.145+0000 7f071b7fe640 1 --2- 172.21.15.76:0/1079246935 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f071c15daa0 0x7f071c0cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:24.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.145+0000 7f071a7fc640 1 --2- 172.21.15.76:0/1079246935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f071c0ce240 0x7f071c154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:24.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.145+0000 7f071affd640 1 -- 172.21.15.76:0/1079246935 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f071c15daa0 msgr2=0x7f071c0cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:24.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.145+0000 7f071affd640 1 --2- 172.21.15.76:0/1079246935 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f071c15daa0 0x7f071c0cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.145+0000 7f071affd640 1 -- 172.21.15.76:0/1079246935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f071c0ce240 msgr2=0x7f071c154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:24.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.145+0000 7f071affd640 1 --2- 172.21.15.76:0/1079246935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f071c0ce240 0x7f071c154790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.145+0000 7f071affd640 1 -- 172.21.15.76:0/1079246935 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0714009b30 con 0x7f071c0cde60 2023-11-09T03:46:24.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.145+0000 7f071b7fe640 1 --2- 172.21.15.76:0/1079246935 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f071c15daa0 0x7f071c0cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:24.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.145+0000 7f071a7fc640 1 --2- 172.21.15.76:0/1079246935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f071c0ce240 0x7f071c154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:24.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.145+0000 7f071affd640 1 --2- 172.21.15.76:0/1079246935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f071c0cde60 0x7f071c154d80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f070400d800 tx=0x7f070400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:24.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.145+0000 7f07137fe640 1 -- 172.21.15.76:0/1079246935 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070400b6c0 con 0x7f071c0cde60 2023-11-09T03:46:24.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.145+0000 7f07137fe640 1 -- 172.21.15.76:0/1079246935 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070400b820 con 0x7f071c0cde60 2023-11-09T03:46:24.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.145+0000 7f07137fe640 1 -- 172.21.15.76:0/1079246935 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070400bbb0 con 0x7f071c0cde60 2023-11-09T03:46:24.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.145+0000 7f07215bb640 1 -- 172.21.15.76:0/1079246935 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f071c1f6a80 con 0x7f071c0cde60 2023-11-09T03:46:24.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.146+0000 7f07215bb640 1 -- 172.21.15.76:0/1079246935 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f071c1554f0 con 0x7f071c0cde60 2023-11-09T03:46:24.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.146+0000 7f07215bb640 1 -- 172.21.15.76:0/1079246935 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f071c0c8da0 con 0x7f071c0cde60 2023-11-09T03:46:24.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.147+0000 7f07137fe640 1 -- 172.21.15.76:0/1079246935 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f070400bd10 con 0x7f071c0cde60 2023-11-09T03:46:24.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.147+0000 7f07137fe640 1 --2- 172.21.15.76:0/1079246935 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f06fc06ae70 0x7f06fc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:24.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.147+0000 7f07137fe640 1 -- 172.21.15.76:0/1079246935 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f0704089f00 con 0x7f071c0cde60 2023-11-09T03:46:24.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.147+0000 7f071a7fc640 1 --2- 172.21.15.76:0/1079246935 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f06fc06ae70 0x7f06fc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:24.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.148+0000 7f071a7fc640 1 --2- 172.21.15.76:0/1079246935 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f06fc06ae70 0x7f06fc06d330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f070c007fd0 tx=0x7f070c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:24.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.151+0000 7f07137fe640 1 -- 172.21.15.76:0/1079246935 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0704054c60 con 0x7f071c0cde60 2023-11-09T03:46:24.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.317+0000 7f07215bb640 1 -- 172.21.15.76:0/1079246935 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f071c074290 con 0x7f071c0cde60 2023-11-09T03:46:24.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.319+0000 7f07137fe640 1 -- 172.21.15.76:0/1079246935 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18710 (secure 0 0 0) 0x7f0704058280 con 0x7f071c0cde60 2023-11-09T03:46:24.318 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:24.319 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":27,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:46:12.537265+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:46:24.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.320+0000 7f07215bb640 1 -- 172.21.15.76:0/1079246935 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f06fc06ae70 msgr2=0x7f06fc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:24.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.320+0000 7f07215bb640 1 --2- 172.21.15.76:0/1079246935 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f06fc06ae70 0x7f06fc06d330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f070c007fd0 tx=0x7f070c009040 comp rx=0 tx=0).stop 2023-11-09T03:46:24.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.320+0000 7f07215bb640 1 -- 172.21.15.76:0/1079246935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f071c0cde60 msgr2=0x7f071c154d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:24.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.320+0000 7f07215bb640 1 --2- 172.21.15.76:0/1079246935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f071c0cde60 0x7f071c154d80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f070400d800 tx=0x7f070400dcc0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.321+0000 7f07215bb640 1 -- 172.21.15.76:0/1079246935 shutdown_connections 2023-11-09T03:46:24.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.321+0000 7f07215bb640 1 --2- 172.21.15.76:0/1079246935 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f06fc06ae70 0x7f06fc06d330 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.321+0000 7f07215bb640 1 --2- 172.21.15.76:0/1079246935 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f071c15daa0 0x7f071c0cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.321+0000 7f07215bb640 1 --2- 172.21.15.76:0/1079246935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f071c0ce240 0x7f071c154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.321+0000 7f07215bb640 1 --2- 172.21.15.76:0/1079246935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f071c0cde60 0x7f071c154d80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.321+0000 7f07215bb640 1 -- 172.21.15.76:0/1079246935 >> 172.21.15.76:0/1079246935 conn(0x7f071c071f30 msgr2=0x7f071c0cd130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:24.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.321+0000 7f07215bb640 1 -- 172.21.15.76:0/1079246935 shutdown_connections 2023-11-09T03:46:24.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.321+0000 7f07215bb640 1 -- 172.21.15.76:0/1079246935 wait complete. 2023-11-09T03:46:24.331 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:45:28.704715+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:45:29.717614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-09T03:45:33.440577+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-09T03:46:24.332 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:46:24.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.435+0000 7f3eadc2d640 1 Processor -- start 2023-11-09T03:46:24.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.435+0000 7f3eadc2d640 1 -- start start 2023-11-09T03:46:24.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.436+0000 7f3eadc2d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3ea8159c00 0x7f3ea80cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:24.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.436+0000 7f3eadc2d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ea80c48e0 0x7f3ea80c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:24.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.436+0000 7f3eadc2d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea815aff0 con 0x7f3ea80adac0 2023-11-09T03:46:24.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.436+0000 7f3eadc2d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ea80c52d0 con 0x7f3ea8159c00 2023-11-09T03:46:24.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.436+0000 7f3eadc2d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea815a3e0 con 0x7f3ea80c48e0 2023-11-09T03:46:24.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.437+0000 7f3ea77fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3ea8159c00 0x7f3ea80cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:24.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.437+0000 7f3ea6ffd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ea80c48e0 0x7f3ea80c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:24.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.437+0000 7f3ea7fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3ea80adac0 0x7f3ea80adf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:48164/0 (socket says 172.21.15.76:48164) 2023-11-09T03:46:24.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.437+0000 7f3ea7fff640 1 -- 172.21.15.76:0/2044862048 learned_addr learned my addr 172.21.15.76:0/2044862048 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:24.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.437+0000 7f3ea67fc640 1 -- 172.21.15.76:0/2044862048 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1771713059 0 0) 0x7f3ea815aff0 con 0x7f3ea80adac0 2023-11-09T03:46:24.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.437+0000 7f3ea67fc640 1 -- 172.21.15.76:0/2044862048 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e90003520 con 0x7f3ea80adac0 2023-11-09T03:46:24.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.437+0000 7f3ea77fe640 1 -- 172.21.15.76:0/2044862048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ea80c48e0 msgr2=0x7f3ea80c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:24.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.437+0000 7f3ea77fe640 1 --2- 172.21.15.76:0/2044862048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ea80c48e0 0x7f3ea80c4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.437+0000 7f3ea77fe640 1 -- 172.21.15.76:0/2044862048 >> v1:172.21.15.113:6789/0 conn(0x7f3ea80adac0 legacy=0x7f3ea80adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:24.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.437+0000 7f3ea77fe640 1 -- 172.21.15.76:0/2044862048 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea815a6c0 con 0x7f3ea8159c00 2023-11-09T03:46:24.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.437+0000 7f3ea6ffd640 1 --2- 172.21.15.76:0/2044862048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ea80c48e0 0x7f3ea80c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:46:24.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.438+0000 7f3ea77fe640 1 --2- 172.21.15.76:0/2044862048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ea8159c00 0x7f3ea80cc890 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3e980097a0 tx=0x7f3e9802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=588905b235875e83 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:24.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.438+0000 7f3ea67fc640 1 -- 172.21.15.76:0/2044862048 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e98031830 con 0x7f3ea8159c00 2023-11-09T03:46:24.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.438+0000 7f3ea67fc640 1 -- 172.21.15.76:0/2044862048 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e98031990 con 0x7f3ea8159c00 2023-11-09T03:46:24.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.438+0000 7f3ea67fc640 1 -- 172.21.15.76:0/2044862048 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e98031d00 con 0x7f3ea8159c00 2023-11-09T03:46:24.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.439+0000 7f3eadc2d640 1 -- 172.21.15.76:0/2044862048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ea8159c00 msgr2=0x7f3ea80cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:24.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.439+0000 7f3eadc2d640 1 --2- 172.21.15.76:0/2044862048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ea8159c00 0x7f3ea80cc890 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3e980097a0 tx=0x7f3e9802b8d0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.439+0000 7f3eadc2d640 1 -- 172.21.15.76:0/2044862048 shutdown_connections 2023-11-09T03:46:24.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.439+0000 7f3eadc2d640 1 --2- 172.21.15.76:0/2044862048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ea80c48e0 0x7f3ea80c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.439+0000 7f3eadc2d640 1 --2- 172.21.15.76:0/2044862048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ea8159c00 0x7f3ea80cc890 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.439+0000 7f3eadc2d640 1 -- 172.21.15.76:0/2044862048 >> 172.21.15.76:0/2044862048 conn(0x7f3ea80ce950 msgr2=0x7f3ea80ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:24.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.439+0000 7f3eadc2d640 1 -- 172.21.15.76:0/2044862048 shutdown_connections 2023-11-09T03:46:24.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.439+0000 7f3eadc2d640 1 -- 172.21.15.76:0/2044862048 wait complete. 2023-11-09T03:46:24.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.440+0000 7f3eadc2d640 1 Processor -- start 2023-11-09T03:46:24.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.440+0000 7f3eadc2d640 1 -- start start 2023-11-09T03:46:24.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.441+0000 7f3eadc2d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ea80adac0 0x7f3ea8147690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:24.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.441+0000 7f3eadc2d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ea80c48e0 0x7f3ea80c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:24.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.441+0000 7f3eadc2d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ea8159c00 0x7f3ea8148f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:24.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.441+0000 7f3eadc2d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea80c6da0 con 0x7f3ea8159c00 2023-11-09T03:46:24.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.441+0000 7f3eadc2d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea814ed50 con 0x7f3ea80adac0 2023-11-09T03:46:24.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.441+0000 7f3eadc2d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea80c5f60 con 0x7f3ea80c48e0 2023-11-09T03:46:24.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.441+0000 7f3ea7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ea8159c00 0x7f3ea8148f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:24.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.441+0000 7f3ea7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ea8159c00 0x7f3ea8148f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35732/0 (socket says 172.21.15.76:35732) 2023-11-09T03:46:24.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.441+0000 7f3ea7fff640 1 -- 172.21.15.76:0/3443809717 learned_addr learned my addr 172.21.15.76:0/3443809717 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:24.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.441+0000 7f3ea77fe640 1 --2- 172.21.15.76:0/3443809717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ea80adac0 0x7f3ea8147690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:24.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.441+0000 7f3ea6ffd640 1 --2- 172.21.15.76:0/3443809717 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ea80c48e0 0x7f3ea80c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:24.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.441+0000 7f3ea7fff640 1 -- 172.21.15.76:0/3443809717 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ea80c48e0 msgr2=0x7f3ea80c5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:24.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.441+0000 7f3ea7fff640 1 --2- 172.21.15.76:0/3443809717 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ea80c48e0 0x7f3ea80c5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.441+0000 7f3ea7fff640 1 -- 172.21.15.76:0/3443809717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ea80adac0 msgr2=0x7f3ea8147690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:24.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.441+0000 7f3ea7fff640 1 --2- 172.21.15.76:0/3443809717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ea80adac0 0x7f3ea8147690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.441+0000 7f3ea7fff640 1 -- 172.21.15.76:0/3443809717 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9c00b6a0 con 0x7f3ea8159c00 2023-11-09T03:46:24.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.441+0000 7f3ea77fe640 1 --2- 172.21.15.76:0/3443809717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ea80adac0 0x7f3ea8147690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:24.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.441+0000 7f3ea7fff640 1 --2- 172.21.15.76:0/3443809717 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ea8159c00 0x7f3ea8148f30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000cb70 tx=0x7f3ea000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:24.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.441+0000 7f3e8ffff640 1 -- 172.21.15.76:0/3443809717 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea000ad40 con 0x7f3ea8159c00 2023-11-09T03:46:24.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.441+0000 7f3e8ffff640 1 -- 172.21.15.76:0/3443809717 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea0012400 con 0x7f3ea8159c00 2023-11-09T03:46:24.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.441+0000 7f3eadc2d640 1 -- 172.21.15.76:0/3443809717 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e98009400 con 0x7f3ea8159c00 2023-11-09T03:46:24.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.441+0000 7f3e8ffff640 1 -- 172.21.15.76:0/3443809717 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea000aea0 con 0x7f3ea8159c00 2023-11-09T03:46:24.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.442+0000 7f3eadc2d640 1 -- 172.21.15.76:0/3443809717 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea80c6190 con 0x7f3ea8159c00 2023-11-09T03:46:24.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.442+0000 7f3eadc2d640 1 -- 172.21.15.76:0/3443809717 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea8074290 con 0x7f3ea8159c00 2023-11-09T03:46:24.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.443+0000 7f3e8ffff640 1 -- 172.21.15.76:0/3443809717 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3ea0007070 con 0x7f3ea8159c00 2023-11-09T03:46:24.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.443+0000 7f3e8ffff640 1 --2- 172.21.15.76:0/3443809717 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3e9406aea0 0x7f3e9406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:24.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.443+0000 7f3e8ffff640 1 -- 172.21.15.76:0/3443809717 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f3ea0089910 con 0x7f3ea8159c00 2023-11-09T03:46:24.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.443+0000 7f3ea77fe640 1 --2- 172.21.15.76:0/3443809717 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3e9406aea0 0x7f3e9406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:24.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.444+0000 7f3ea77fe640 1 --2- 172.21.15.76:0/3443809717 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3e9406aea0 0x7f3e9406d360 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3e98036000 tx=0x7f3e9803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:24.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.447+0000 7f3e8ffff640 1 -- 172.21.15.76:0/3443809717 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3ea0054690 con 0x7f3ea8159c00 2023-11-09T03:46:24.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.614+0000 7f3eadc2d640 1 -- 172.21.15.76:0/3443809717 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3ea806c890 con 0x7f3ea8159c00 2023-11-09T03:46:24.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.614+0000 7f3e8ffff640 1 -- 172.21.15.76:0/3443809717 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3ea00170a0 con 0x7f3ea8159c00 2023-11-09T03:46:24.614 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:46:24.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.616+0000 7f3eadc2d640 1 -- 172.21.15.76:0/3443809717 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3e9406aea0 msgr2=0x7f3e9406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:24.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.616+0000 7f3eadc2d640 1 --2- 172.21.15.76:0/3443809717 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3e9406aea0 0x7f3e9406d360 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3e98036000 tx=0x7f3e9803c040 comp rx=0 tx=0).stop 2023-11-09T03:46:24.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.616+0000 7f3eadc2d640 1 -- 172.21.15.76:0/3443809717 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ea8159c00 msgr2=0x7f3ea8148f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:24.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.616+0000 7f3eadc2d640 1 --2- 172.21.15.76:0/3443809717 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ea8159c00 0x7f3ea8148f30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000cb70 tx=0x7f3ea000a430 comp rx=0 tx=0).stop 2023-11-09T03:46:24.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.616+0000 7f3eadc2d640 1 -- 172.21.15.76:0/3443809717 shutdown_connections 2023-11-09T03:46:24.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.616+0000 7f3eadc2d640 1 --2- 172.21.15.76:0/3443809717 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3e9406aea0 0x7f3e9406d360 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.616+0000 7f3eadc2d640 1 --2- 172.21.15.76:0/3443809717 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ea8159c00 0x7f3ea8148f30 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.616+0000 7f3eadc2d640 1 --2- 172.21.15.76:0/3443809717 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ea80c48e0 0x7f3ea80c5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.616+0000 7f3eadc2d640 1 --2- 172.21.15.76:0/3443809717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ea80adac0 0x7f3ea8147690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.616+0000 7f3eadc2d640 1 -- 172.21.15.76:0/3443809717 >> 172.21.15.76:0/3443809717 conn(0x7f3ea80ce950 msgr2=0x7f3ea80cb340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:24.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.616+0000 7f3eadc2d640 1 -- 172.21.15.76:0/3443809717 shutdown_connections 2023-11-09T03:46:24.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.616+0000 7f3eadc2d640 1 -- 172.21.15.76:0/3443809717 wait complete. 2023-11-09T03:46:24.626 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:46:24.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.730+0000 7effedb01640 1 Processor -- start 2023-11-09T03:46:24.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.731+0000 7effedb01640 1 -- start start 2023-11-09T03:46:24.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.731+0000 7effedb01640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effe8153e60 0x7effe815dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:24.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.731+0000 7effedb01640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7effe80cd6d0 0x7effe8161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:24.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.731+0000 7effedb01640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe80cde60 con 0x7effe80b4880 2023-11-09T03:46:24.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.731+0000 7effedb01640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe8160f90 con 0x7effe80cd6d0 2023-11-09T03:46:24.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.731+0000 7effedb01640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe815e400 con 0x7effe8153e60 2023-11-09T03:46:24.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.731+0000 7effe6ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7effe80cd6d0 0x7effe8161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:24.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.731+0000 7effe7fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7effe80b4880 0x7effe806b810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53036/0 (socket says 172.21.15.76:53036) 2023-11-09T03:46:24.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.731+0000 7effe7fff640 1 -- 172.21.15.76:0/483799107 learned_addr learned my addr 172.21.15.76:0/483799107 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:24.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.732+0000 7effe77fe640 1 --2- 172.21.15.76:0/483799107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effe8153e60 0x7effe815dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:24.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.732+0000 7effe6ffd640 1 -- 172.21.15.76:0/483799107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effe8153e60 msgr2=0x7effe815dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:24.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.732+0000 7effe6ffd640 1 --2- 172.21.15.76:0/483799107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effe8153e60 0x7effe815dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.732+0000 7effe6ffd640 1 -- 172.21.15.76:0/483799107 >> v1:172.21.15.76:6789/0 conn(0x7effe80b4880 legacy=0x7effe806b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:24.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.732+0000 7effe67fc640 1 -- 172.21.15.76:0/483799107 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2201331994 0 0) 0x7effe80cde60 con 0x7effe80b4880 2023-11-09T03:46:24.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.732+0000 7effe6ffd640 1 -- 172.21.15.76:0/483799107 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe8161960 con 0x7effe80cd6d0 2023-11-09T03:46:24.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.733+0000 7effe6ffd640 1 --2- 172.21.15.76:0/483799107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effe80cd6d0 0x7effe8161420 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7effd00097a0 tx=0x7effd002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4040d3463edb2cbd server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:24.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.733+0000 7effe67fc640 1 -- 172.21.15.76:0/483799107 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd0031830 con 0x7effe80cd6d0 2023-11-09T03:46:24.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.733+0000 7effe67fc640 1 -- 172.21.15.76:0/483799107 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd0031990 con 0x7effe80cd6d0 2023-11-09T03:46:24.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.733+0000 7effe67fc640 1 -- 172.21.15.76:0/483799107 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd0031d20 con 0x7effe80cd6d0 2023-11-09T03:46:24.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.733+0000 7effedb01640 1 -- 172.21.15.76:0/483799107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effe80cd6d0 msgr2=0x7effe8161420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:24.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.733+0000 7effedb01640 1 --2- 172.21.15.76:0/483799107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effe80cd6d0 0x7effe8161420 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7effd00097a0 tx=0x7effd002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.733+0000 7effedb01640 1 -- 172.21.15.76:0/483799107 shutdown_connections 2023-11-09T03:46:24.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.733+0000 7effedb01640 1 --2- 172.21.15.76:0/483799107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effe80cd6d0 0x7effe8161420 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.733+0000 7effedb01640 1 --2- 172.21.15.76:0/483799107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effe8153e60 0x7effe815dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.733+0000 7effedb01640 1 -- 172.21.15.76:0/483799107 >> 172.21.15.76:0/483799107 conn(0x7effe80cf880 msgr2=0x7effe80cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:24.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.733+0000 7effedb01640 1 -- 172.21.15.76:0/483799107 shutdown_connections 2023-11-09T03:46:24.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.733+0000 7effedb01640 1 -- 172.21.15.76:0/483799107 wait complete. 2023-11-09T03:46:24.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.734+0000 7effedb01640 1 Processor -- start 2023-11-09T03:46:24.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.735+0000 7effedb01640 1 -- start start 2023-11-09T03:46:24.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.735+0000 7effedb01640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effe80b4880 0x7effe81f2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:24.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.735+0000 7effedb01640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effe80cd6d0 0x7effe8154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:24.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.735+0000 7effedb01640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effe8153e60 0x7effe8158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:24.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.735+0000 7effedb01640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe81f2870 con 0x7effe80b4880 2023-11-09T03:46:24.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.735+0000 7effedb01640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe80c6ca0 con 0x7effe80cd6d0 2023-11-09T03:46:24.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.735+0000 7effedb01640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe8154780 con 0x7effe8153e60 2023-11-09T03:46:24.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.735+0000 7effe77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effe80b4880 0x7effe81f2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:24.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.735+0000 7effe77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effe80b4880 0x7effe81f2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35762/0 (socket says 172.21.15.76:35762) 2023-11-09T03:46:24.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.735+0000 7effe77fe640 1 -- 172.21.15.76:0/3460012149 learned_addr learned my addr 172.21.15.76:0/3460012149 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:24.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.736+0000 7effe6ffd640 1 --2- 172.21.15.76:0/3460012149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effe80cd6d0 0x7effe8154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:24.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.736+0000 7effe7fff640 1 --2- 172.21.15.76:0/3460012149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effe8153e60 0x7effe8158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:24.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.736+0000 7effe77fe640 1 -- 172.21.15.76:0/3460012149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effe8153e60 msgr2=0x7effe8158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:24.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.736+0000 7effe77fe640 1 --2- 172.21.15.76:0/3460012149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effe8153e60 0x7effe8158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.736+0000 7effe77fe640 1 -- 172.21.15.76:0/3460012149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effe80cd6d0 msgr2=0x7effe8154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:24.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.736+0000 7effe77fe640 1 --2- 172.21.15.76:0/3460012149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effe80cd6d0 0x7effe8154240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.736+0000 7effe77fe640 1 -- 172.21.15.76:0/3460012149 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd0009400 con 0x7effe80b4880 2023-11-09T03:46:24.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.736+0000 7effe77fe640 1 --2- 172.21.15.76:0/3460012149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effe80b4880 0x7effe81f2330 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7effe0007e70 tx=0x7effe000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:24.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.736+0000 7effdffff640 1 -- 172.21.15.76:0/3460012149 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe0014620 con 0x7effe80b4880 2023-11-09T03:46:24.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.736+0000 7effdffff640 1 -- 172.21.15.76:0/3460012149 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe0014780 con 0x7effe80b4880 2023-11-09T03:46:24.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.736+0000 7effdffff640 1 -- 172.21.15.76:0/3460012149 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe0014b30 con 0x7effe80b4880 2023-11-09T03:46:24.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.736+0000 7effedb01640 1 -- 172.21.15.76:0/3460012149 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe80c7ad0 con 0x7effe80b4880 2023-11-09T03:46:24.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.736+0000 7effedb01640 1 -- 172.21.15.76:0/3460012149 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe80c8090 con 0x7effe80b4880 2023-11-09T03:46:24.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.737+0000 7effedb01640 1 -- 172.21.15.76:0/3460012149 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe8074290 con 0x7effe80b4880 2023-11-09T03:46:24.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.737+0000 7effdffff640 1 -- 172.21.15.76:0/3460012149 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7effe0014c90 con 0x7effe80b4880 2023-11-09T03:46:24.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.738+0000 7effdffff640 1 --2- 172.21.15.76:0/3460012149 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7effc806ae70 0x7effc806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:24.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.738+0000 7effdffff640 1 -- 172.21.15.76:0/3460012149 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7effe008a290 con 0x7effe80b4880 2023-11-09T03:46:24.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.738+0000 7effe6ffd640 1 --2- 172.21.15.76:0/3460012149 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7effc806ae70 0x7effc806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:24.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.739+0000 7effe6ffd640 1 --2- 172.21.15.76:0/3460012149 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7effc806ae70 0x7effc806d330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7effd0036000 tx=0x7effd003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:24.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.742+0000 7effdffff640 1 -- 172.21.15.76:0/3460012149 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7effe0054ff0 con 0x7effe80b4880 2023-11-09T03:46:24.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.911+0000 7effedb01640 1 -- 172.21.15.76:0/3460012149 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7effe8157580 con 0x7effe80b4880 2023-11-09T03:46:24.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.911+0000 7effdffff640 1 -- 172.21.15.76:0/3460012149 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7effe0058610 con 0x7effe80b4880 2023-11-09T03:46:24.911 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:46:24.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.914+0000 7effedb01640 1 -- 172.21.15.76:0/3460012149 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7effc806ae70 msgr2=0x7effc806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:24.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.914+0000 7effedb01640 1 --2- 172.21.15.76:0/3460012149 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7effc806ae70 0x7effc806d330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7effd0036000 tx=0x7effd003c040 comp rx=0 tx=0).stop 2023-11-09T03:46:24.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.914+0000 7effedb01640 1 -- 172.21.15.76:0/3460012149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effe80b4880 msgr2=0x7effe81f2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:24.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.914+0000 7effedb01640 1 --2- 172.21.15.76:0/3460012149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effe80b4880 0x7effe81f2330 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7effe0007e70 tx=0x7effe000e4b0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.914+0000 7effedb01640 1 -- 172.21.15.76:0/3460012149 shutdown_connections 2023-11-09T03:46:24.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.914+0000 7effedb01640 1 --2- 172.21.15.76:0/3460012149 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7effc806ae70 0x7effc806d330 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.914+0000 7effedb01640 1 --2- 172.21.15.76:0/3460012149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effe8153e60 0x7effe8158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.914+0000 7effedb01640 1 --2- 172.21.15.76:0/3460012149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effe80cd6d0 0x7effe8154240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.914+0000 7effedb01640 1 --2- 172.21.15.76:0/3460012149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effe80b4880 0x7effe81f2330 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:24.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.914+0000 7effedb01640 1 -- 172.21.15.76:0/3460012149 >> 172.21.15.76:0/3460012149 conn(0x7effe80cf880 msgr2=0x7effe815e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:24.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.914+0000 7effedb01640 1 -- 172.21.15.76:0/3460012149 shutdown_connections 2023-11-09T03:46:24.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:24.914+0000 7effedb01640 1 -- 172.21.15.76:0/3460012149 wait complete. 2023-11-09T03:46:24.924 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:46:25.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.027+0000 7f4feb5ee640 1 Processor -- start 2023-11-09T03:46:25.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.027+0000 7f4feb5ee640 1 -- start start 2023-11-09T03:46:25.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.028+0000 7f4feb5ee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4fe41605b0 0x7f4fe40ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:25.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.028+0000 7f4feb5ee640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe4160990 con 0x7f4fe40b1a40 2023-11-09T03:46:25.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.028+0000 7f4feb5ee640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe40cc7c0 con 0x7f4fe406a830 2023-11-09T03:46:25.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.028+0000 7f4feb5ee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe40cd750 con 0x7f4fe41605b0 2023-11-09T03:46:25.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.028+0000 7f4fe8b89640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f4fe40b1a40 0x7f4fe406a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53038/0 (socket says 172.21.15.76:53038) 2023-11-09T03:46:25.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.028+0000 7f4fe8b89640 1 -- 172.21.15.76:0/2142505811 learned_addr learned my addr 172.21.15.76:0/2142505811 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:25.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.028+0000 7f4fdbfff640 1 -- 172.21.15.76:0/2142505811 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3129085460 0 0) 0x7f4fe4160990 con 0x7f4fe40b1a40 2023-11-09T03:46:25.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.028+0000 7f4fdbfff640 1 -- 172.21.15.76:0/2142505811 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fd0003520 con 0x7f4fe40b1a40 2023-11-09T03:46:25.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.028+0000 7f4fe938a640 1 --2- 172.21.15.76:0/2142505811 >> v2:172.21.15.113:3300/0 conn(0x7f4fe41605b0 0x7f4fe40ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:25.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.029+0000 7f4fdbfff640 1 -- 172.21.15.76:0/2142505811 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1769951505 0 0) 0x7f4fd0003520 con 0x7f4fe40b1a40 2023-11-09T03:46:25.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.029+0000 7f4fdbfff640 1 -- 172.21.15.76:0/2142505811 >> v2:172.21.15.113:3300/0 conn(0x7f4fe41605b0 msgr2=0x7f4fe40ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:25.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.029+0000 7f4fdbfff640 1 --2- 172.21.15.76:0/2142505811 >> v2:172.21.15.113:3300/0 conn(0x7f4fe41605b0 0x7f4fe40ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.029+0000 7f4fdbfff640 1 -- 172.21.15.76:0/2142505811 >> v1:172.21.15.113:6789/0 conn(0x7f4fe406a830 legacy=0x7f4fe40cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:25.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.029+0000 7f4fdbfff640 1 -- 172.21.15.76:0/2142505811 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe40cc4e0 con 0x7f4fe40b1a40 2023-11-09T03:46:25.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.029+0000 7f4fdbfff640 1 -- 172.21.15.76:0/2142505811 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f4fd4001670 con 0x7f4fe40b1a40 2023-11-09T03:46:25.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.029+0000 7f4fdbfff640 1 -- 172.21.15.76:0/2142505811 >> v1:172.21.15.76:6789/0 conn(0x7f4fe40b1a40 legacy=0x7f4fe406a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:25.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.029+0000 7f4fdbfff640 1 --2- 172.21.15.76:0/2142505811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fd0003970 0x7f4fd0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:25.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.029+0000 7f4fdbfff640 1 --2- 172.21.15.76:0/2142505811 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fd00047c0 0x7f4fd0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:25.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.029+0000 7f4fdbfff640 1 --2- 172.21.15.76:0/2142505811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fd0005380 0x7f4fd000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:25.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.029+0000 7f4fdbfff640 1 -- 172.21.15.76:0/2142505811 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40cd750 con 0x7f4fd0005380 2023-11-09T03:46:25.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.029+0000 7f4fdbfff640 1 -- 172.21.15.76:0/2142505811 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd000c210 con 0x7f4fd0003970 2023-11-09T03:46:25.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.029+0000 7f4fdbfff640 1 -- 172.21.15.76:0/2142505811 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd000c3d0 con 0x7f4fd00047c0 2023-11-09T03:46:25.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.029+0000 7f4fe9b8b640 1 --2- 172.21.15.76:0/2142505811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fd0005380 0x7f4fd000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:25.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.029+0000 7f4fe938a640 1 --2- 172.21.15.76:0/2142505811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fd0003970 0x7f4fd0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:25.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.029+0000 7f4fe8b89640 1 --2- 172.21.15.76:0/2142505811 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fd00047c0 0x7f4fd0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:25.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.029+0000 7f4fe9b8b640 1 -- 172.21.15.76:0/2142505811 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fd00047c0 msgr2=0x7f4fd0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:25.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.029+0000 7f4fe9b8b640 1 --2- 172.21.15.76:0/2142505811 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fd00047c0 0x7f4fd0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.029+0000 7f4fe9b8b640 1 -- 172.21.15.76:0/2142505811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fd0003970 msgr2=0x7f4fd0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:25.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.029+0000 7f4fe9b8b640 1 --2- 172.21.15.76:0/2142505811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fd0003970 0x7f4fd0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.029+0000 7f4fe9b8b640 1 -- 172.21.15.76:0/2142505811 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd0003520 con 0x7f4fd0005380 2023-11-09T03:46:25.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.030+0000 7f4fe9b8b640 1 -- 172.21.15.76:0/2142505811 reap_dead start 2023-11-09T03:46:25.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.030+0000 7f4fe9b8b640 1 --2- 172.21.15.76:0/2142505811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fd0005380 0x7f4fd000b980 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc001e50 tx=0x7f4fd00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad9405c8ebeacad server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:25.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.030+0000 7f4fdbfff640 1 -- 172.21.15.76:0/2142505811 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc003b70 con 0x7f4fd0005380 2023-11-09T03:46:25.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.030+0000 7f4fdbfff640 1 -- 172.21.15.76:0/2142505811 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fdc003d10 con 0x7f4fd0005380 2023-11-09T03:46:25.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.030+0000 7f4fdbfff640 1 -- 172.21.15.76:0/2142505811 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc005310 con 0x7f4fd0005380 2023-11-09T03:46:25.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.030+0000 7f4feb5ee640 1 -- 172.21.15.76:0/2142505811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fd0005380 msgr2=0x7f4fd000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:25.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.030+0000 7f4feb5ee640 1 --2- 172.21.15.76:0/2142505811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fd0005380 0x7f4fd000b980 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc001e50 tx=0x7f4fd00057f0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.030+0000 7f4feb5ee640 1 -- 172.21.15.76:0/2142505811 shutdown_connections 2023-11-09T03:46:25.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.031+0000 7f4feb5ee640 1 --2- 172.21.15.76:0/2142505811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fd0005380 0x7f4fd000b980 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.031+0000 7f4feb5ee640 1 -- 172.21.15.76:0/2142505811 >> 172.21.15.76:0/2142505811 conn(0x7f4fe4071820 msgr2=0x7f4fe4071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:25.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.031+0000 7f4feb5ee640 1 -- 172.21.15.76:0/2142505811 shutdown_connections 2023-11-09T03:46:25.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.031+0000 7f4feb5ee640 1 -- 172.21.15.76:0/2142505811 wait complete. 2023-11-09T03:46:25.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.032+0000 7f4feb5ee640 1 Processor -- start 2023-11-09T03:46:25.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.032+0000 7f4feb5ee640 1 -- start start 2023-11-09T03:46:25.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.032+0000 7f4feb5ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fd0005380 0x7f4fe415ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:25.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.032+0000 7f4feb5ee640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fe40b2800 0x7f4fe40b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:25.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.032+0000 7f4feb5ee640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fe41f2a80 0x7f4fe41f2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:25.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.033+0000 7f4feb5ee640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40acf60 con 0x7f4fe40b2800 2023-11-09T03:46:25.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.033+0000 7f4feb5ee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40ac930 con 0x7f4fd0005380 2023-11-09T03:46:25.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.033+0000 7f4feb5ee640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe41f0490 con 0x7f4fe41f2a80 2023-11-09T03:46:25.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.033+0000 7f4fe8b89640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fe40b2800 0x7f4fe40b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:25.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.033+0000 7f4fe8b89640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fe40b2800 0x7f4fe40b4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35792/0 (socket says 172.21.15.76:35792) 2023-11-09T03:46:25.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.033+0000 7f4fe8b89640 1 -- 172.21.15.76:0/2917506857 learned_addr learned my addr 172.21.15.76:0/2917506857 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:25.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.033+0000 7f4fe938a640 1 --2- 172.21.15.76:0/2917506857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fd0005380 0x7f4fe415ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:25.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.033+0000 7f4fe9b8b640 1 --2- 172.21.15.76:0/2917506857 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fe41f2a80 0x7f4fe41f2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:25.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.033+0000 7f4fe8b89640 1 -- 172.21.15.76:0/2917506857 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fe41f2a80 msgr2=0x7f4fe41f2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:25.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.033+0000 7f4fe8b89640 1 --2- 172.21.15.76:0/2917506857 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fe41f2a80 0x7f4fe41f2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.033+0000 7f4fe8b89640 1 -- 172.21.15.76:0/2917506857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fd0005380 msgr2=0x7f4fe415ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:25.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.033+0000 7f4fe8b89640 1 --2- 172.21.15.76:0/2917506857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fd0005380 0x7f4fe415ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.033+0000 7f4fe8b89640 1 -- 172.21.15.76:0/2917506857 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe415f540 con 0x7f4fe40b2800 2023-11-09T03:46:25.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.033+0000 7f4fe938a640 1 --2- 172.21.15.76:0/2917506857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fd0005380 0x7f4fe415ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:25.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.033+0000 7f4fe8b89640 1 --2- 172.21.15.76:0/2917506857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fe40b2800 0x7f4fe40b4ff0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4007290 tx=0x7f4fd4007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:25.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.033+0000 7f4fd9ffb640 1 -- 172.21.15.76:0/2917506857 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4001430 con 0x7f4fe40b2800 2023-11-09T03:46:25.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.033+0000 7f4fd9ffb640 1 -- 172.21.15.76:0/2917506857 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd400f9c0 con 0x7f4fe40b2800 2023-11-09T03:46:25.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.033+0000 7f4feb5ee640 1 -- 172.21.15.76:0/2917506857 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe415f820 con 0x7f4fe40b2800 2023-11-09T03:46:25.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.033+0000 7f4fd9ffb640 1 -- 172.21.15.76:0/2917506857 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd400fd50 con 0x7f4fe40b2800 2023-11-09T03:46:25.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.033+0000 7f4feb5ee640 1 -- 172.21.15.76:0/2917506857 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe40b5530 con 0x7f4fe40b2800 2023-11-09T03:46:25.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.034+0000 7f4feb5ee640 1 -- 172.21.15.76:0/2917506857 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe40672a0 con 0x7f4fe40b2800 2023-11-09T03:46:25.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.035+0000 7f4fd9ffb640 1 -- 172.21.15.76:0/2917506857 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4fd401d460 con 0x7f4fe40b2800 2023-11-09T03:46:25.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.035+0000 7f4fd9ffb640 1 --2- 172.21.15.76:0/2917506857 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4fc806ae70 0x7f4fc806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:25.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.035+0000 7f4fd9ffb640 1 -- 172.21.15.76:0/2917506857 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f4fd4089b10 con 0x7f4fe40b2800 2023-11-09T03:46:25.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.035+0000 7f4fe938a640 1 --2- 172.21.15.76:0/2917506857 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4fc806ae70 0x7f4fc806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:25.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.036+0000 7f4fe938a640 1 --2- 172.21.15.76:0/2917506857 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4fc806ae70 0x7f4fc806d330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0006fd0 tx=0x7f4fe0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:25.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.039+0000 7f4fd9ffb640 1 -- 172.21.15.76:0/2917506857 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4fd4054870 con 0x7f4fe40b2800 2023-11-09T03:46:25.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.207+0000 7f4feb5ee640 1 -- 172.21.15.76:0/2917506857 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4fe40ad0a0 con 0x7f4fe40b2800 2023-11-09T03:46:25.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.207+0000 7f4fd9ffb640 1 -- 172.21.15.76:0/2917506857 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4fe40ad0a0 con 0x7f4fe40b2800 2023-11-09T03:46:25.206 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:46:25.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.209+0000 7f4feb5ee640 1 -- 172.21.15.76:0/2917506857 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4fc806ae70 msgr2=0x7f4fc806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:25.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.209+0000 7f4feb5ee640 1 --2- 172.21.15.76:0/2917506857 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4fc806ae70 0x7f4fc806d330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0006fd0 tx=0x7f4fe0008040 comp rx=0 tx=0).stop 2023-11-09T03:46:25.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.209+0000 7f4feb5ee640 1 -- 172.21.15.76:0/2917506857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fe40b2800 msgr2=0x7f4fe40b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:25.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.209+0000 7f4feb5ee640 1 --2- 172.21.15.76:0/2917506857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fe40b2800 0x7f4fe40b4ff0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4007290 tx=0x7f4fd4007750 comp rx=0 tx=0).stop 2023-11-09T03:46:25.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.209+0000 7f4feb5ee640 1 -- 172.21.15.76:0/2917506857 shutdown_connections 2023-11-09T03:46:25.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.209+0000 7f4feb5ee640 1 --2- 172.21.15.76:0/2917506857 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4fc806ae70 0x7f4fc806d330 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.209+0000 7f4feb5ee640 1 --2- 172.21.15.76:0/2917506857 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fe41f2a80 0x7f4fe41f2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.209+0000 7f4feb5ee640 1 --2- 172.21.15.76:0/2917506857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fe40b2800 0x7f4fe40b4ff0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.209+0000 7f4feb5ee640 1 --2- 172.21.15.76:0/2917506857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fd0005380 0x7f4fe415ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.209+0000 7f4feb5ee640 1 -- 172.21.15.76:0/2917506857 >> 172.21.15.76:0/2917506857 conn(0x7f4fe4071820 msgr2=0x7f4fe40aebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:25.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.209+0000 7f4feb5ee640 1 -- 172.21.15.76:0/2917506857 shutdown_connections 2023-11-09T03:46:25.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.209+0000 7f4feb5ee640 1 -- 172.21.15.76:0/2917506857 wait complete. 2023-11-09T03:46:25.219 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-09T03:46:25.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.322+0000 7fa83214c640 1 Processor -- start 2023-11-09T03:46:25.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.322+0000 7fa83214c640 1 -- start start 2023-11-09T03:46:25.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.323+0000 7fa83214c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa82c15ad30 0x7fa82c0adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:25.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.323+0000 7fa83214c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa82c0ad730 0x7fa82c070710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:25.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.323+0000 7fa83214c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa82c0c3900 con 0x7fa82c070c50 2023-11-09T03:46:25.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.323+0000 7fa83214c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa82c0ae260 con 0x7fa82c0ad730 2023-11-09T03:46:25.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.323+0000 7fa83214c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa82c15f750 con 0x7fa82c15ad30 2023-11-09T03:46:25.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.323+0000 7fa82affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa82c0ad730 0x7fa82c070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:25.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.323+0000 7fa82affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa82c0ad730 0x7fa82c070710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35800/0 (socket says 172.21.15.76:35800) 2023-11-09T03:46:25.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.323+0000 7fa82affd640 1 -- 172.21.15.76:0/3692254983 learned_addr learned my addr 172.21.15.76:0/3692254983 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:25.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.323+0000 7fa82b7fe640 1 --2- 172.21.15.76:0/3692254983 >> v2:172.21.15.113:3300/0 conn(0x7fa82c15ad30 0x7fa82c0adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:25.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.323+0000 7fa82affd640 1 -- 172.21.15.76:0/3692254983 >> v2:172.21.15.113:3300/0 conn(0x7fa82c15ad30 msgr2=0x7fa82c0adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:25.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.323+0000 7fa82affd640 1 --2- 172.21.15.76:0/3692254983 >> v2:172.21.15.113:3300/0 conn(0x7fa82c15ad30 0x7fa82c0adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.323+0000 7fa82affd640 1 -- 172.21.15.76:0/3692254983 >> v1:172.21.15.113:6789/0 conn(0x7fa82c070c50 legacy=0x7fa82c0c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:25.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.323+0000 7fa82affd640 1 -- 172.21.15.76:0/3692254983 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82c0c3be0 con 0x7fa82c0ad730 2023-11-09T03:46:25.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.323+0000 7fa82b7fe640 1 --2- 172.21.15.76:0/3692254983 >> v2:172.21.15.113:3300/0 conn(0x7fa82c15ad30 0x7fa82c0adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:25.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.324+0000 7fa82affd640 1 --2- 172.21.15.76:0/3692254983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa82c0ad730 0x7fa82c070710 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa82c06bb00 tx=0x7fa82002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ef2bf642a2e98b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:25.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.324+0000 7fa82a7fc640 1 -- 172.21.15.76:0/3692254983 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8200317e0 con 0x7fa82c0ad730 2023-11-09T03:46:25.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.324+0000 7fa82a7fc640 1 -- 172.21.15.76:0/3692254983 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa820031940 con 0x7fa82c0ad730 2023-11-09T03:46:25.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.324+0000 7fa82a7fc640 1 -- 172.21.15.76:0/3692254983 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa820031cd0 con 0x7fa82c0ad730 2023-11-09T03:46:25.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.325+0000 7fa83214c640 1 -- 172.21.15.76:0/3692254983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa82c0ad730 msgr2=0x7fa82c070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:25.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.325+0000 7fa83214c640 1 --2- 172.21.15.76:0/3692254983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa82c0ad730 0x7fa82c070710 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa82c06bb00 tx=0x7fa82002b880 comp rx=0 tx=0).stop 2023-11-09T03:46:25.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.325+0000 7fa83214c640 1 -- 172.21.15.76:0/3692254983 shutdown_connections 2023-11-09T03:46:25.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.325+0000 7fa83214c640 1 --2- 172.21.15.76:0/3692254983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa82c0ad730 0x7fa82c070710 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.325+0000 7fa83214c640 1 --2- 172.21.15.76:0/3692254983 >> v2:172.21.15.113:3300/0 conn(0x7fa82c15ad30 0x7fa82c0adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.325+0000 7fa83214c640 1 -- 172.21.15.76:0/3692254983 >> 172.21.15.76:0/3692254983 conn(0x7fa82c0ce950 msgr2=0x7fa82c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:25.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.325+0000 7fa83214c640 1 -- 172.21.15.76:0/3692254983 shutdown_connections 2023-11-09T03:46:25.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.325+0000 7fa83214c640 1 -- 172.21.15.76:0/3692254983 wait complete. 2023-11-09T03:46:25.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.326+0000 7fa83214c640 1 Processor -- start 2023-11-09T03:46:25.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.326+0000 7fa83214c640 1 -- start start 2023-11-09T03:46:25.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.327+0000 7fa83214c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa82c070c50 0x7fa82c157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:25.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.327+0000 7fa83214c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa82c0ad730 0x7fa82c0c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:25.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.327+0000 7fa83214c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa82c15ad30 0x7fa82c14f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:25.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.327+0000 7fa83214c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c0c5bc0 con 0x7fa82c070c50 2023-11-09T03:46:25.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.327+0000 7fa83214c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c0c5ea0 con 0x7fa82c15ad30 2023-11-09T03:46:25.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.327+0000 7fa83214c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c0c6180 con 0x7fa82c0ad730 2023-11-09T03:46:25.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.327+0000 7fa82b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa82c070c50 0x7fa82c157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:25.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.327+0000 7fa82b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa82c070c50 0x7fa82c157790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35804/0 (socket says 172.21.15.76:35804) 2023-11-09T03:46:25.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.327+0000 7fa82b7fe640 1 -- 172.21.15.76:0/1025332013 learned_addr learned my addr 172.21.15.76:0/1025332013 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:25.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.327+0000 7fa82affd640 1 --2- 172.21.15.76:0/1025332013 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa82c0ad730 0x7fa82c0c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:25.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.327+0000 7fa82bfff640 1 --2- 172.21.15.76:0/1025332013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa82c15ad30 0x7fa82c14f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:25.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.327+0000 7fa82b7fe640 1 -- 172.21.15.76:0/1025332013 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa82c0ad730 msgr2=0x7fa82c0c8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:25.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.327+0000 7fa82b7fe640 1 --2- 172.21.15.76:0/1025332013 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa82c0ad730 0x7fa82c0c8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.327+0000 7fa82b7fe640 1 -- 172.21.15.76:0/1025332013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa82c15ad30 msgr2=0x7fa82c14f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:25.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.327+0000 7fa82b7fe640 1 --2- 172.21.15.76:0/1025332013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa82c15ad30 0x7fa82c14f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.327+0000 7fa82b7fe640 1 -- 172.21.15.76:0/1025332013 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8200094d0 con 0x7fa82c070c50 2023-11-09T03:46:25.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.327+0000 7fa82b7fe640 1 --2- 172.21.15.76:0/1025332013 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa82c070c50 0x7fa82c157790 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00c7f0 tx=0x7fa81c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:25.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.328+0000 7fa813fff640 1 -- 172.21.15.76:0/1025332013 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c007a30 con 0x7fa82c070c50 2023-11-09T03:46:25.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.328+0000 7fa813fff640 1 -- 172.21.15.76:0/1025332013 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa81c007b90 con 0x7fa82c070c50 2023-11-09T03:46:25.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.328+0000 7fa813fff640 1 -- 172.21.15.76:0/1025332013 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c00ce40 con 0x7fa82c070c50 2023-11-09T03:46:25.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.328+0000 7fa83214c640 1 -- 172.21.15.76:0/1025332013 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82c0c6460 con 0x7fa82c070c50 2023-11-09T03:46:25.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.328+0000 7fa83214c640 1 -- 172.21.15.76:0/1025332013 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82c0c8650 con 0x7fa82c070c50 2023-11-09T03:46:25.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.329+0000 7fa83214c640 1 -- 172.21.15.76:0/1025332013 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa82c074290 con 0x7fa82c070c50 2023-11-09T03:46:25.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.329+0000 7fa813fff640 1 -- 172.21.15.76:0/1025332013 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa81c0113f0 con 0x7fa82c070c50 2023-11-09T03:46:25.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.329+0000 7fa813fff640 1 --2- 172.21.15.76:0/1025332013 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa81806ae70 0x7fa81806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:25.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.329+0000 7fa813fff640 1 -- 172.21.15.76:0/1025332013 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fa81c089a50 con 0x7fa82c070c50 2023-11-09T03:46:25.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.330+0000 7fa82affd640 1 --2- 172.21.15.76:0/1025332013 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa81806ae70 0x7fa81806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:25.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.330+0000 7fa82affd640 1 --2- 172.21.15.76:0/1025332013 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa81806ae70 0x7fa81806d330 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa820036000 tx=0x7fa82003d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:25.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.334+0000 7fa813fff640 1 -- 172.21.15.76:0/1025332013 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa81c054860 con 0x7fa82c070c50 2023-11-09T03:46:25.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.501+0000 7fa83214c640 1 -- 172.21.15.76:0/1025332013 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa82c0c5d00 con 0x7fa82c070c50 2023-11-09T03:46:25.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.502+0000 7fa813fff640 1 -- 172.21.15.76:0/1025332013 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa82c0c5d00 con 0x7fa82c070c50 2023-11-09T03:46:25.501 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:46:25.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.504+0000 7fa83214c640 1 -- 172.21.15.76:0/1025332013 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa81806ae70 msgr2=0x7fa81806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:25.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.504+0000 7fa83214c640 1 --2- 172.21.15.76:0/1025332013 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa81806ae70 0x7fa81806d330 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa820036000 tx=0x7fa82003d040 comp rx=0 tx=0).stop 2023-11-09T03:46:25.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.504+0000 7fa83214c640 1 -- 172.21.15.76:0/1025332013 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa82c070c50 msgr2=0x7fa82c157790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:25.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.504+0000 7fa83214c640 1 --2- 172.21.15.76:0/1025332013 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa82c070c50 0x7fa82c157790 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00c7f0 tx=0x7fa81c00cbf0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.504+0000 7fa83214c640 1 -- 172.21.15.76:0/1025332013 shutdown_connections 2023-11-09T03:46:25.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.504+0000 7fa83214c640 1 --2- 172.21.15.76:0/1025332013 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa81806ae70 0x7fa81806d330 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.504+0000 7fa83214c640 1 --2- 172.21.15.76:0/1025332013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa82c15ad30 0x7fa82c14f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.504+0000 7fa83214c640 1 --2- 172.21.15.76:0/1025332013 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa82c0ad730 0x7fa82c0c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.504+0000 7fa83214c640 1 --2- 172.21.15.76:0/1025332013 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa82c070c50 0x7fa82c157790 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.504+0000 7fa83214c640 1 -- 172.21.15.76:0/1025332013 >> 172.21.15.76:0/1025332013 conn(0x7fa82c0ce950 msgr2=0x7fa82c06db10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:25.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.504+0000 7fa83214c640 1 -- 172.21.15.76:0/1025332013 shutdown_connections 2023-11-09T03:46:25.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.504+0000 7fa83214c640 1 -- 172.21.15.76:0/1025332013 wait complete. 2023-11-09T03:46:25.514 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-09T03:46:25.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.615+0000 7f1fb1790640 1 Processor -- start 2023-11-09T03:46:25.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.615+0000 7f1fb1790640 1 -- start start 2023-11-09T03:46:25.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.616+0000 7f1fb1790640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1fac0cdc80 0x7f1fac06c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:25.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.616+0000 7f1fb1790640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fac0cea20 con 0x7f1fac0cf180 2023-11-09T03:46:25.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.616+0000 7f1fb1790640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fac0cf610 con 0x7f1fac15cfd0 2023-11-09T03:46:25.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.616+0000 7f1fb1790640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fac15d3b0 con 0x7f1fac0cdc80 2023-11-09T03:46:25.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.616+0000 7f1faa7fc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f1fac0cf180 0x7f1fac0cd570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53042/0 (socket says 172.21.15.76:53042) 2023-11-09T03:46:25.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.616+0000 7f1faa7fc640 1 -- 172.21.15.76:0/2167040227 learned_addr learned my addr 172.21.15.76:0/2167040227 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:25.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.616+0000 7f1fa9ffb640 1 -- 172.21.15.76:0/2167040227 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 438100792 0 0) 0x7f1fac0cea20 con 0x7f1fac0cf180 2023-11-09T03:46:25.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.617+0000 7f1fab7fe640 1 --2- 172.21.15.76:0/2167040227 >> v2:172.21.15.113:3300/0 conn(0x7f1fac0cdc80 0x7f1fac06c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:25.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.617+0000 7f1fa9ffb640 1 -- 172.21.15.76:0/2167040227 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f98003520 con 0x7f1fac0cf180 2023-11-09T03:46:25.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.617+0000 7f1fa9ffb640 1 -- 172.21.15.76:0/2167040227 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1230496671 0 0) 0x7f1f98003520 con 0x7f1fac0cf180 2023-11-09T03:46:25.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.617+0000 7f1fa9ffb640 1 -- 172.21.15.76:0/2167040227 >> v2:172.21.15.113:3300/0 conn(0x7f1fac0cdc80 msgr2=0x7f1fac06c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:25.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.617+0000 7f1fa9ffb640 1 --2- 172.21.15.76:0/2167040227 >> v2:172.21.15.113:3300/0 conn(0x7f1fac0cdc80 0x7f1fac06c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.617+0000 7f1fa9ffb640 1 -- 172.21.15.76:0/2167040227 >> v1:172.21.15.113:6789/0 conn(0x7f1fac15cfd0 legacy=0x7f1fac161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:25.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.617+0000 7f1fa9ffb640 1 -- 172.21.15.76:0/2167040227 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fac0cf8f0 con 0x7f1fac0cf180 2023-11-09T03:46:25.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.617+0000 7f1fa9ffb640 1 -- 172.21.15.76:0/2167040227 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f1f94001670 con 0x7f1fac0cf180 2023-11-09T03:46:25.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.617+0000 7f1fa9ffb640 1 -- 172.21.15.76:0/2167040227 >> v1:172.21.15.76:6789/0 conn(0x7f1fac0cf180 legacy=0x7f1fac0cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:25.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.617+0000 7f1fa9ffb640 1 --2- 172.21.15.76:0/2167040227 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f980039c0 0x7f1f98003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:25.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.617+0000 7f1fa9ffb640 1 --2- 172.21.15.76:0/2167040227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f980047c0 0x7f1f98004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:25.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.617+0000 7f1fa9ffb640 1 --2- 172.21.15.76:0/2167040227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f980051d0 0x7f1f9800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:25.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.617+0000 7f1fa9ffb640 1 -- 172.21.15.76:0/2167040227 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac15d3b0 con 0x7f1f980047c0 2023-11-09T03:46:25.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.617+0000 7f1fa9ffb640 1 -- 172.21.15.76:0/2167040227 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9800c120 con 0x7f1f980051d0 2023-11-09T03:46:25.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.617+0000 7f1fa9ffb640 1 -- 172.21.15.76:0/2167040227 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9800c2e0 con 0x7f1f980039c0 2023-11-09T03:46:25.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.617+0000 7f1faa7fc640 1 --2- 172.21.15.76:0/2167040227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f980047c0 0x7f1f98004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:25.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.617+0000 7f1faaffd640 1 --2- 172.21.15.76:0/2167040227 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f980039c0 0x7f1f98003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:25.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.617+0000 7f1fab7fe640 1 --2- 172.21.15.76:0/2167040227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f980051d0 0x7f1f9800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:25.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.617+0000 7f1faa7fc640 1 -- 172.21.15.76:0/2167040227 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f980039c0 msgr2=0x7f1f98003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:25.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.617+0000 7f1faa7fc640 1 --2- 172.21.15.76:0/2167040227 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f980039c0 0x7f1f98003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.617+0000 7f1faa7fc640 1 -- 172.21.15.76:0/2167040227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f980051d0 msgr2=0x7f1f9800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:25.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.617+0000 7f1faa7fc640 1 --2- 172.21.15.76:0/2167040227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f980051d0 0x7f1f9800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.617+0000 7f1faa7fc640 1 -- 172.21.15.76:0/2167040227 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f98003520 con 0x7f1f980047c0 2023-11-09T03:46:25.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.617+0000 7f1fab7fe640 1 -- 172.21.15.76:0/2167040227 reap_dead start 2023-11-09T03:46:25.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.618+0000 7f1faa7fc640 1 --2- 172.21.15.76:0/2167040227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f980047c0 0x7f1f98004c40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1f94001eb0 tx=0x7f1f980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4b504b01dbf661 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:25.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.618+0000 7f1fa9ffb640 1 -- 172.21.15.76:0/2167040227 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f94005ad0 con 0x7f1f980047c0 2023-11-09T03:46:25.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.618+0000 7f1fa9ffb640 1 -- 172.21.15.76:0/2167040227 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f94005c30 con 0x7f1f980047c0 2023-11-09T03:46:25.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.618+0000 7f1fa9ffb640 1 -- 172.21.15.76:0/2167040227 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f940072b0 con 0x7f1f980047c0 2023-11-09T03:46:25.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.619+0000 7f1fb1790640 1 -- 172.21.15.76:0/2167040227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f980047c0 msgr2=0x7f1f98004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:25.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.619+0000 7f1fb1790640 1 --2- 172.21.15.76:0/2167040227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f980047c0 0x7f1f98004c40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1f94001eb0 tx=0x7f1f980057f0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.619+0000 7f1fb1790640 1 -- 172.21.15.76:0/2167040227 shutdown_connections 2023-11-09T03:46:25.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.619+0000 7f1fb1790640 1 --2- 172.21.15.76:0/2167040227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f980047c0 0x7f1f98004c40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.619+0000 7f1fb1790640 1 -- 172.21.15.76:0/2167040227 >> 172.21.15.76:0/2167040227 conn(0x7f1fac06ea20 msgr2=0x7f1fac06ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:25.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.619+0000 7f1fb1790640 1 -- 172.21.15.76:0/2167040227 shutdown_connections 2023-11-09T03:46:25.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.619+0000 7f1fb1790640 1 -- 172.21.15.76:0/2167040227 wait complete. 2023-11-09T03:46:25.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.620+0000 7f1fb1790640 1 Processor -- start 2023-11-09T03:46:25.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.620+0000 7f1fb1790640 1 -- start start 2023-11-09T03:46:25.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.621+0000 7f1fb1790640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f980047c0 0x7f1fac0c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:25.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.621+0000 7f1fb1790640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fac150f40 0x7f1fac0ab950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:25.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.621+0000 7f1fb1790640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fac0cb500 0x7f1fac0cb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:25.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.621+0000 7f1fb1790640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac0c9a30 con 0x7f1fac150f40 2023-11-09T03:46:25.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.621+0000 7f1fb1790640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac0cbec0 con 0x7f1fac0cb500 2023-11-09T03:46:25.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.621+0000 7f1fb1790640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac0c8db0 con 0x7f1f980047c0 2023-11-09T03:46:25.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.621+0000 7f1faa7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fac150f40 0x7f1fac0ab950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:25.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.621+0000 7f1faa7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fac150f40 0x7f1fac0ab950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35830/0 (socket says 172.21.15.76:35830) 2023-11-09T03:46:25.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.621+0000 7f1faa7fc640 1 -- 172.21.15.76:0/1428327843 learned_addr learned my addr 172.21.15.76:0/1428327843 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:25.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.621+0000 7f1fab7fe640 1 --2- 172.21.15.76:0/1428327843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fac0cb500 0x7f1fac0cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:25.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.621+0000 7f1faaffd640 1 --2- 172.21.15.76:0/1428327843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f980047c0 0x7f1fac0c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:25.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.621+0000 7f1faa7fc640 1 -- 172.21.15.76:0/1428327843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f980047c0 msgr2=0x7f1fac0c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:25.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.621+0000 7f1faa7fc640 1 --2- 172.21.15.76:0/1428327843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f980047c0 0x7f1fac0c9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.621+0000 7f1faa7fc640 1 -- 172.21.15.76:0/1428327843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fac0cb500 msgr2=0x7f1fac0cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:25.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.621+0000 7f1faa7fc640 1 --2- 172.21.15.76:0/1428327843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fac0cb500 0x7f1fac0cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.621+0000 7f1faa7fc640 1 -- 172.21.15.76:0/1428327843 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fac0c9090 con 0x7f1fac150f40 2023-11-09T03:46:25.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.621+0000 7f1faa7fc640 1 --2- 172.21.15.76:0/1428327843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fac150f40 0x7f1fac0ab950 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1f94000c00 tx=0x7f1f9402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:25.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.621+0000 7f1fa37fe640 1 -- 172.21.15.76:0/1428327843 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9403c040 con 0x7f1fac150f40 2023-11-09T03:46:25.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.621+0000 7f1fa37fe640 1 -- 172.21.15.76:0/1428327843 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f940075e0 con 0x7f1fac150f40 2023-11-09T03:46:25.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.621+0000 7f1fb1790640 1 -- 172.21.15.76:0/1428327843 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fac0c9370 con 0x7f1fac150f40 2023-11-09T03:46:25.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.621+0000 7f1fa37fe640 1 -- 172.21.15.76:0/1428327843 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9402c980 con 0x7f1fac150f40 2023-11-09T03:46:25.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.622+0000 7f1fb1790640 1 -- 172.21.15.76:0/1428327843 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fac1513a0 con 0x7f1fac150f40 2023-11-09T03:46:25.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.622+0000 7f1fb1790640 1 -- 172.21.15.76:0/1428327843 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fac0c9b70 con 0x7f1fac150f40 2023-11-09T03:46:25.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.623+0000 7f1fa37fe640 1 -- 172.21.15.76:0/1428327843 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1f94032070 con 0x7f1fac150f40 2023-11-09T03:46:25.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.623+0000 7f1fa37fe640 1 --2- 172.21.15.76:0/1428327843 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1f8c06aea0 0x7f1f8c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:25.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.623+0000 7f1fa37fe640 1 -- 172.21.15.76:0/1428327843 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f1f940aaec0 con 0x7f1fac150f40 2023-11-09T03:46:25.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.624+0000 7f1faaffd640 1 --2- 172.21.15.76:0/1428327843 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1f8c06aea0 0x7f1f8c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:25.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.624+0000 7f1faaffd640 1 --2- 172.21.15.76:0/1428327843 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1f8c06aea0 0x7f1f8c06d360 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40035f0 tx=0x7f1fa4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:25.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.628+0000 7f1fa37fe640 1 -- 172.21.15.76:0/1428327843 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1fac0c9b70 con 0x7f1fac150f40 2023-11-09T03:46:25.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.807+0000 7f1fb1790640 1 -- 172.21.15.76:0/1428327843 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1fac0c9b70 con 0x7f1fac150f40 2023-11-09T03:46:25.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.808+0000 7f1fa37fe640 1 -- 172.21.15.76:0/1428327843 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1fac0c9b70 con 0x7f1fac150f40 2023-11-09T03:46:25.807 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2023-11-09T03:46:25.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.809+0000 7f1fb1790640 1 -- 172.21.15.76:0/1428327843 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1f8c06aea0 msgr2=0x7f1f8c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:25.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.809+0000 7f1fb1790640 1 --2- 172.21.15.76:0/1428327843 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1f8c06aea0 0x7f1f8c06d360 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40035f0 tx=0x7f1fa4009040 comp rx=0 tx=0).stop 2023-11-09T03:46:25.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.809+0000 7f1fb1790640 1 -- 172.21.15.76:0/1428327843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fac150f40 msgr2=0x7f1fac0ab950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:25.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.809+0000 7f1fb1790640 1 --2- 172.21.15.76:0/1428327843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fac150f40 0x7f1fac0ab950 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1f94000c00 tx=0x7f1f9402e070 comp rx=0 tx=0).stop 2023-11-09T03:46:25.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.810+0000 7f1fb1790640 1 -- 172.21.15.76:0/1428327843 shutdown_connections 2023-11-09T03:46:25.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.810+0000 7f1fb1790640 1 --2- 172.21.15.76:0/1428327843 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1f8c06aea0 0x7f1f8c06d360 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.810+0000 7f1fb1790640 1 --2- 172.21.15.76:0/1428327843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fac0cb500 0x7f1fac0cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.810+0000 7f1fb1790640 1 --2- 172.21.15.76:0/1428327843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fac150f40 0x7f1fac0ab950 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.810+0000 7f1fb1790640 1 --2- 172.21.15.76:0/1428327843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f980047c0 0x7f1fac0c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:25.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.810+0000 7f1fb1790640 1 -- 172.21.15.76:0/1428327843 >> 172.21.15.76:0/1428327843 conn(0x7f1fac06ea20 msgr2=0x7f1fac0cc730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:25.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.810+0000 7f1fb1790640 1 -- 172.21.15.76:0/1428327843 shutdown_connections 2023-11-09T03:46:25.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25.810+0000 7f1fb1790640 1 -- 172.21.15.76:0/1428327843 wait complete. 2023-11-09T03:46:25.819 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi076.front.sepia.ceph.com... 2023-11-09T03:46:25.820 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:46:25.820 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-09T03:46:25.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:46:25.880 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[98478]: fuse finished with error 0 and tester_r 0 2023-11-09T03:46:25.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:46:25.884 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:46:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:46:25.887 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:46:25.893 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T03:46:25.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:25.893 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-09T03:46:25.948 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:46:25.948 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:46:25.968 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:46:25.968 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:46:26.026 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:46:26.067 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi113.stderr:2023-11-09T03:46:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:46:26.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.127+0000 7f3d17410640 1 Processor -- start 2023-11-09T03:46:26.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.128+0000 7f3d17410640 1 -- start start 2023-11-09T03:46:26.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.128+0000 7f3d17410640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d100cde60 0x7f3d1015d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:26.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.128+0000 7f3d17410640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3d100ce240 0x7f3d100d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:26.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.128+0000 7f3d17410640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d1015e000 con 0x7f3d1015daa0 2023-11-09T03:46:26.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.128+0000 7f3d17410640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d1015e5c0 con 0x7f3d100ce240 2023-11-09T03:46:26.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.128+0000 7f3d17410640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1015e2e0 con 0x7f3d100cde60 2023-11-09T03:46:26.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.128+0000 7f3d159ad640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3d100ce240 0x7f3d100d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:26.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.129+0000 7f3d159ad640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3d100ce240 0x7f3d100d1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35836/0 (socket says 172.21.15.76:35836) 2023-11-09T03:46:26.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.129+0000 7f3d159ad640 1 -- 172.21.15.76:0/935106087 learned_addr learned my addr 172.21.15.76:0/935106087 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:26.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.129+0000 7f3d151ac640 1 --2- 172.21.15.76:0/935106087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d100cde60 0x7f3d1015d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:26.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.129+0000 7f3d159ad640 1 -- 172.21.15.76:0/935106087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d100cde60 msgr2=0x7f3d1015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:26.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.129+0000 7f3d159ad640 1 --2- 172.21.15.76:0/935106087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d100cde60 0x7f3d1015d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.129+0000 7f3d159ad640 1 -- 172.21.15.76:0/935106087 >> v1:172.21.15.113:6789/0 conn(0x7f3d1015daa0 legacy=0x7f3d10157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:26.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.129+0000 7f3d159ad640 1 -- 172.21.15.76:0/935106087 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d100d1120 con 0x7f3d100ce240 2023-11-09T03:46:26.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.130+0000 7f3d159ad640 1 --2- 172.21.15.76:0/935106087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d100ce240 0x7f3d100d1920 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c009870 tx=0x7f3d0c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ffa391a09b139a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:26.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.130+0000 7f3d07fff640 1 -- 172.21.15.76:0/935106087 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c030840 con 0x7f3d100ce240 2023-11-09T03:46:26.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.130+0000 7f3d07fff640 1 -- 172.21.15.76:0/935106087 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0c0309a0 con 0x7f3d100ce240 2023-11-09T03:46:26.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.130+0000 7f3d07fff640 1 -- 172.21.15.76:0/935106087 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c030d30 con 0x7f3d100ce240 2023-11-09T03:46:26.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.130+0000 7f3d17410640 1 -- 172.21.15.76:0/935106087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d100ce240 msgr2=0x7f3d100d1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:26.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.130+0000 7f3d17410640 1 --2- 172.21.15.76:0/935106087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d100ce240 0x7f3d100d1920 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c009870 tx=0x7f3d0c02b980 comp rx=0 tx=0).stop 2023-11-09T03:46:26.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.130+0000 7f3d17410640 1 -- 172.21.15.76:0/935106087 shutdown_connections 2023-11-09T03:46:26.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.130+0000 7f3d17410640 1 --2- 172.21.15.76:0/935106087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d100ce240 0x7f3d100d1920 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.130+0000 7f3d17410640 1 --2- 172.21.15.76:0/935106087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d100cde60 0x7f3d1015d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.130+0000 7f3d17410640 1 -- 172.21.15.76:0/935106087 >> 172.21.15.76:0/935106087 conn(0x7f3d10071f30 msgr2=0x7f3d100cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:26.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.130+0000 7f3d17410640 1 -- 172.21.15.76:0/935106087 shutdown_connections 2023-11-09T03:46:26.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.130+0000 7f3d17410640 1 -- 172.21.15.76:0/935106087 wait complete. 2023-11-09T03:46:26.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.131+0000 7f3d17410640 1 Processor -- start 2023-11-09T03:46:26.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.131+0000 7f3d17410640 1 -- start start 2023-11-09T03:46:26.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.132+0000 7f3d17410640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d100cde60 0x7f3d1014fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:26.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.132+0000 7f3d17410640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d100ce240 0x7f3d101e8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:26.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.132+0000 7f3d17410640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d1015daa0 0x7f3d10154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:26.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.132+0000 7f3d17410640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1014e660 con 0x7f3d100cde60 2023-11-09T03:46:26.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.132+0000 7f3d17410640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d100cb6d0 con 0x7f3d100ce240 2023-11-09T03:46:26.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.132+0000 7f3d17410640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1014f000 con 0x7f3d1015daa0 2023-11-09T03:46:26.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.132+0000 7f3d151ac640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d100cde60 0x7f3d1014fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:26.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.132+0000 7f3d151ac640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d100cde60 0x7f3d1014fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35852/0 (socket says 172.21.15.76:35852) 2023-11-09T03:46:26.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.132+0000 7f3d151ac640 1 -- 172.21.15.76:0/2955249245 learned_addr learned my addr 172.21.15.76:0/2955249245 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:26.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.133+0000 7f3d159ad640 1 --2- 172.21.15.76:0/2955249245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d1015daa0 0x7f3d10154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:26.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.133+0000 7f3d149ab640 1 --2- 172.21.15.76:0/2955249245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d100ce240 0x7f3d101e8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:26.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.133+0000 7f3d151ac640 1 -- 172.21.15.76:0/2955249245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d1015daa0 msgr2=0x7f3d10154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:26.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.133+0000 7f3d151ac640 1 --2- 172.21.15.76:0/2955249245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d1015daa0 0x7f3d10154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.133+0000 7f3d151ac640 1 -- 172.21.15.76:0/2955249245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d100ce240 msgr2=0x7f3d101e8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:26.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.133+0000 7f3d151ac640 1 --2- 172.21.15.76:0/2955249245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d100ce240 0x7f3d101e8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.133+0000 7f3d151ac640 1 -- 172.21.15.76:0/2955249245 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0c0094d0 con 0x7f3d100cde60 2023-11-09T03:46:26.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.133+0000 7f3d151ac640 1 --2- 172.21.15.76:0/2955249245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d100cde60 0x7f3d1014fd30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3d08007ef0 tx=0x7f3d0800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:26.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.133+0000 7f3d05ffb640 1 -- 172.21.15.76:0/2955249245 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d08014620 con 0x7f3d100cde60 2023-11-09T03:46:26.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.133+0000 7f3d05ffb640 1 -- 172.21.15.76:0/2955249245 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d08014780 con 0x7f3d100cde60 2023-11-09T03:46:26.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.133+0000 7f3d17410640 1 -- 172.21.15.76:0/2955249245 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d101e9280 con 0x7f3d100cde60 2023-11-09T03:46:26.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.133+0000 7f3d05ffb640 1 -- 172.21.15.76:0/2955249245 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d08014b30 con 0x7f3d100cde60 2023-11-09T03:46:26.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.133+0000 7f3d17410640 1 -- 172.21.15.76:0/2955249245 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d1014f250 con 0x7f3d100cde60 2023-11-09T03:46:26.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.134+0000 7f3d17410640 1 -- 172.21.15.76:0/2955249245 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d10074290 con 0x7f3d100cde60 2023-11-09T03:46:26.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.134+0000 7f3d05ffb640 1 -- 172.21.15.76:0/2955249245 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3d08014c90 con 0x7f3d100cde60 2023-11-09T03:46:26.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.135+0000 7f3d05ffb640 1 --2- 172.21.15.76:0/2955249245 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3cf406aea0 0x7f3cf406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:26.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.135+0000 7f3d05ffb640 1 -- 172.21.15.76:0/2955249245 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f3d0808a3d0 con 0x7f3d100cde60 2023-11-09T03:46:26.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.135+0000 7f3d149ab640 1 --2- 172.21.15.76:0/2955249245 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3cf406aea0 0x7f3cf406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:26.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.136+0000 7f3d149ab640 1 --2- 172.21.15.76:0/2955249245 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3cf406aea0 0x7f3cf406d360 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3d100d2490 tx=0x7f3d00009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:26.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.139+0000 7f3d05ffb640 1 -- 172.21.15.76:0/2955249245 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3d080550a0 con 0x7f3d100cde60 2023-11-09T03:46:26.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.308+0000 7f3d17410640 1 -- 172.21.15.76:0/2955249245 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d100c66f0 con 0x7f3d100cde60 2023-11-09T03:46:26.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.309+0000 7f3d05ffb640 1 -- 172.21.15.76:0/2955249245 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18710 (secure 0 0 0) 0x7f3d080586c0 con 0x7f3d100cde60 2023-11-09T03:46:26.309 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:26.309 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":27,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:46:12.537265+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:46:26.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.311+0000 7f3d17410640 1 -- 172.21.15.76:0/2955249245 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3cf406aea0 msgr2=0x7f3cf406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:26.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.311+0000 7f3d17410640 1 --2- 172.21.15.76:0/2955249245 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3cf406aea0 0x7f3cf406d360 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3d100d2490 tx=0x7f3d00009040 comp rx=0 tx=0).stop 2023-11-09T03:46:26.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.311+0000 7f3d17410640 1 -- 172.21.15.76:0/2955249245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d100cde60 msgr2=0x7f3d1014fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:26.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.311+0000 7f3d17410640 1 --2- 172.21.15.76:0/2955249245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d100cde60 0x7f3d1014fd30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3d08007ef0 tx=0x7f3d0800e4b0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.311+0000 7f3d17410640 1 -- 172.21.15.76:0/2955249245 shutdown_connections 2023-11-09T03:46:26.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.311+0000 7f3d17410640 1 --2- 172.21.15.76:0/2955249245 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3cf406aea0 0x7f3cf406d360 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.311+0000 7f3d17410640 1 --2- 172.21.15.76:0/2955249245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d1015daa0 0x7f3d10154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.311+0000 7f3d17410640 1 --2- 172.21.15.76:0/2955249245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d100ce240 0x7f3d101e8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.311+0000 7f3d17410640 1 --2- 172.21.15.76:0/2955249245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d100cde60 0x7f3d1014fd30 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.311+0000 7f3d17410640 1 -- 172.21.15.76:0/2955249245 >> 172.21.15.76:0/2955249245 conn(0x7f3d10071f30 msgr2=0x7f3d100b7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:26.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.312+0000 7f3d17410640 1 -- 172.21.15.76:0/2955249245 shutdown_connections 2023-11-09T03:46:26.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.312+0000 7f3d17410640 1 -- 172.21.15.76:0/2955249245 wait complete. 2023-11-09T03:46:26.322 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:45:28.704715+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:45:29.717614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-09T03:45:33.440577+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-09T03:46:26.322 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:46:26.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.427+0000 7f23a68e7640 1 Processor -- start 2023-11-09T03:46:26.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.427+0000 7f23a68e7640 1 -- start start 2023-11-09T03:46:26.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.428+0000 7f23a68e7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f23a0153e60 0x7f23a015dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:26.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.428+0000 7f23a68e7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23a00b4880 0x7f23a00b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:26.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.428+0000 7f23a68e7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a00cde60 con 0x7f23a00cd6d0 2023-11-09T03:46:26.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.428+0000 7f23a68e7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a0160f90 con 0x7f23a0153e60 2023-11-09T03:46:26.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.428+0000 7f23a68e7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a00b5200 con 0x7f23a00b4880 2023-11-09T03:46:26.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.428+0000 7f23a4e84640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23a00b4880 0x7f23a00b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:26.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.428+0000 7f239ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f23a0153e60 0x7f23a015dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:26.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.429+0000 7f239f7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f23a00cd6d0 0x7f23a0161420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:48208/0 (socket says 172.21.15.76:48208) 2023-11-09T03:46:26.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.429+0000 7f239ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f23a0153e60 0x7f23a015dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:41248/0 (socket says 172.21.15.76:41248) 2023-11-09T03:46:26.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.429+0000 7f23a4e84640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23a00b4880 0x7f23a00b4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:57758/0 (socket says 172.21.15.76:57758) 2023-11-09T03:46:26.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.429+0000 7f239f7fe640 1 -- 172.21.15.76:0/4274438607 learned_addr learned my addr 172.21.15.76:0/4274438607 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:26.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.429+0000 7f23a4e84640 1 -- 172.21.15.76:0/4274438607 >> v2:172.21.15.113:3300/0 conn(0x7f23a0153e60 msgr2=0x7f23a015dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:26.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.429+0000 7f23a4e84640 1 --2- 172.21.15.76:0/4274438607 >> v2:172.21.15.113:3300/0 conn(0x7f23a0153e60 0x7f23a015dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.429+0000 7f23a4e84640 1 -- 172.21.15.76:0/4274438607 >> v1:172.21.15.113:6789/0 conn(0x7f23a00cd6d0 legacy=0x7f23a0161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:26.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.429+0000 7f23a4e84640 1 -- 172.21.15.76:0/4274438607 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a015e400 con 0x7f23a00b4880 2023-11-09T03:46:26.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.429+0000 7f239ffff640 1 --2- 172.21.15.76:0/4274438607 >> v2:172.21.15.113:3300/0 conn(0x7f23a0153e60 0x7f23a015dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:46:26.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.430+0000 7f23a4e84640 1 --2- 172.21.15.76:0/4274438607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23a00b4880 0x7f23a00b4cc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2398009d90 tx=0x7f239802bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58baa7a429bf04aa server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:26.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.430+0000 7f239effd640 1 -- 172.21.15.76:0/4274438607 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239802cc50 con 0x7f23a00b4880 2023-11-09T03:46:26.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.430+0000 7f239effd640 1 -- 172.21.15.76:0/4274438607 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239802cdb0 con 0x7f23a00b4880 2023-11-09T03:46:26.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.430+0000 7f239effd640 1 -- 172.21.15.76:0/4274438607 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398034660 con 0x7f23a00b4880 2023-11-09T03:46:26.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.430+0000 7f23a68e7640 1 -- 172.21.15.76:0/4274438607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23a00b4880 msgr2=0x7f23a00b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:26.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.430+0000 7f23a68e7640 1 --2- 172.21.15.76:0/4274438607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23a00b4880 0x7f23a00b4cc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2398009d90 tx=0x7f239802bea0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.430+0000 7f23a68e7640 1 -- 172.21.15.76:0/4274438607 shutdown_connections 2023-11-09T03:46:26.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.430+0000 7f23a68e7640 1 --2- 172.21.15.76:0/4274438607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23a00b4880 0x7f23a00b4cc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.430+0000 7f23a68e7640 1 --2- 172.21.15.76:0/4274438607 >> v2:172.21.15.113:3300/0 conn(0x7f23a0153e60 0x7f23a015dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.430+0000 7f23a68e7640 1 -- 172.21.15.76:0/4274438607 >> 172.21.15.76:0/4274438607 conn(0x7f23a00cf880 msgr2=0x7f23a00cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:26.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.430+0000 7f23a68e7640 1 -- 172.21.15.76:0/4274438607 shutdown_connections 2023-11-09T03:46:26.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.430+0000 7f23a68e7640 1 -- 172.21.15.76:0/4274438607 wait complete. 2023-11-09T03:46:26.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.432+0000 7f23a68e7640 1 Processor -- start 2023-11-09T03:46:26.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.432+0000 7f23a68e7640 1 -- start start 2023-11-09T03:46:26.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.432+0000 7f23a68e7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23a00b4880 0x7f23a015a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:26.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.432+0000 7f23a68e7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23a00cd6d0 0x7f23a00c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:26.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.432+0000 7f23a68e7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23a0153e60 0x7f23a00b7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:26.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.432+0000 7f23a68e7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a00ca640 con 0x7f23a0153e60 2023-11-09T03:46:26.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.432+0000 7f23a68e7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a0156770 con 0x7f23a00b4880 2023-11-09T03:46:26.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.432+0000 7f23a68e7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a015a990 con 0x7f23a00cd6d0 2023-11-09T03:46:26.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.433+0000 7f23a4e84640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23a0153e60 0x7f23a00b7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:26.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.433+0000 7f23a4e84640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23a0153e60 0x7f23a00b7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35866/0 (socket says 172.21.15.76:35866) 2023-11-09T03:46:26.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.433+0000 7f239ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23a00b4880 0x7f23a015a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:26.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.433+0000 7f23a4e84640 1 -- 172.21.15.76:0/3769776238 learned_addr learned my addr 172.21.15.76:0/3769776238 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:26.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.433+0000 7f239f7fe640 1 --2- 172.21.15.76:0/3769776238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23a00cd6d0 0x7f23a00c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:26.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.433+0000 7f23a4e84640 1 -- 172.21.15.76:0/3769776238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23a00cd6d0 msgr2=0x7f23a00c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:26.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.433+0000 7f23a4e84640 1 --2- 172.21.15.76:0/3769776238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23a00cd6d0 0x7f23a00c8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.433+0000 7f23a4e84640 1 -- 172.21.15.76:0/3769776238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23a00b4880 msgr2=0x7f23a015a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:26.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.433+0000 7f23a4e84640 1 --2- 172.21.15.76:0/3769776238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23a00b4880 0x7f23a015a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.433+0000 7f23a4e84640 1 -- 172.21.15.76:0/3769776238 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2390009400 con 0x7f23a0153e60 2023-11-09T03:46:26.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.433+0000 7f23a4e84640 1 --2- 172.21.15.76:0/3769776238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23a0153e60 0x7f23a00b7670 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2398009eb0 tx=0x7f2398004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:26.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.433+0000 7f239cff9640 1 -- 172.21.15.76:0/3769776238 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398040650 con 0x7f23a0153e60 2023-11-09T03:46:26.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.433+0000 7f239cff9640 1 -- 172.21.15.76:0/3769776238 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23980407b0 con 0x7f23a0153e60 2023-11-09T03:46:26.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.433+0000 7f23a68e7640 1 -- 172.21.15.76:0/3769776238 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2398009a70 con 0x7f23a0153e60 2023-11-09T03:46:26.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.433+0000 7f239cff9640 1 -- 172.21.15.76:0/3769776238 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398040b60 con 0x7f23a0153e60 2023-11-09T03:46:26.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.433+0000 7f23a68e7640 1 -- 172.21.15.76:0/3769776238 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a01561b0 con 0x7f23a0153e60 2023-11-09T03:46:26.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.434+0000 7f23a68e7640 1 -- 172.21.15.76:0/3769776238 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a00ca780 con 0x7f23a0153e60 2023-11-09T03:46:26.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.435+0000 7f239cff9640 1 -- 172.21.15.76:0/3769776238 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2398040cc0 con 0x7f23a0153e60 2023-11-09T03:46:26.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.435+0000 7f239cff9640 1 --2- 172.21.15.76:0/3769776238 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f238c06ae70 0x7f238c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:26.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.435+0000 7f239cff9640 1 -- 172.21.15.76:0/3769776238 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f23980ac4f0 con 0x7f23a0153e60 2023-11-09T03:46:26.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.435+0000 7f239ffff640 1 --2- 172.21.15.76:0/3769776238 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f238c06ae70 0x7f238c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:26.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.436+0000 7f239ffff640 1 --2- 172.21.15.76:0/3769776238 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f238c06ae70 0x7f238c06d330 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f23a01f5af0 tx=0x7f2390009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:26.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.439+0000 7f239cff9640 1 -- 172.21.15.76:0/3769776238 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2398077300 con 0x7f23a0153e60 2023-11-09T03:46:26.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.604+0000 7f23a68e7640 1 -- 172.21.15.76:0/3769776238 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f23a0155bf0 con 0x7f23a0153e60 2023-11-09T03:46:26.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.605+0000 7f239cff9640 1 -- 172.21.15.76:0/3769776238 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f23a0155bf0 con 0x7f23a0153e60 2023-11-09T03:46:26.605 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:46:26.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.607+0000 7f23a68e7640 1 -- 172.21.15.76:0/3769776238 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f238c06ae70 msgr2=0x7f238c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:26.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.607+0000 7f23a68e7640 1 --2- 172.21.15.76:0/3769776238 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f238c06ae70 0x7f238c06d330 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f23a01f5af0 tx=0x7f2390009190 comp rx=0 tx=0).stop 2023-11-09T03:46:26.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.607+0000 7f23a68e7640 1 -- 172.21.15.76:0/3769776238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23a0153e60 msgr2=0x7f23a00b7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:26.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.607+0000 7f23a68e7640 1 --2- 172.21.15.76:0/3769776238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23a0153e60 0x7f23a00b7670 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2398009eb0 tx=0x7f2398004a40 comp rx=0 tx=0).stop 2023-11-09T03:46:26.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.607+0000 7f23a68e7640 1 -- 172.21.15.76:0/3769776238 shutdown_connections 2023-11-09T03:46:26.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.607+0000 7f23a68e7640 1 --2- 172.21.15.76:0/3769776238 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f238c06ae70 0x7f238c06d330 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.607+0000 7f23a68e7640 1 --2- 172.21.15.76:0/3769776238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23a0153e60 0x7f23a00b7670 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.607+0000 7f23a68e7640 1 --2- 172.21.15.76:0/3769776238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23a00cd6d0 0x7f23a00c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.607+0000 7f23a68e7640 1 --2- 172.21.15.76:0/3769776238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23a00b4880 0x7f23a015a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.607+0000 7f23a68e7640 1 -- 172.21.15.76:0/3769776238 >> 172.21.15.76:0/3769776238 conn(0x7f23a00cf880 msgr2=0x7f23a00d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:26.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.607+0000 7f23a68e7640 1 -- 172.21.15.76:0/3769776238 shutdown_connections 2023-11-09T03:46:26.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.607+0000 7f23a68e7640 1 -- 172.21.15.76:0/3769776238 wait complete. 2023-11-09T03:46:26.617 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:46:26.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.722+0000 7f3a0acb0640 1 Processor -- start 2023-11-09T03:46:26.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.722+0000 7f3a0acb0640 1 -- start start 2023-11-09T03:46:26.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.723+0000 7f3a0acb0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3a04153e60 0x7f3a0415dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:26.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.723+0000 7f3a0acb0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3a040b4880 0x7f3a040b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:26.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.723+0000 7f3a0acb0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a040cde60 con 0x7f3a040cd6d0 2023-11-09T03:46:26.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.723+0000 7f3a0acb0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a04160f90 con 0x7f3a04153e60 2023-11-09T03:46:26.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.723+0000 7f3a0acb0640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a040b5200 con 0x7f3a040b4880 2023-11-09T03:46:26.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.723+0000 7f3a08a4c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3a04153e60 0x7f3a0415dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:26.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.723+0000 7f3a08a4c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3a04153e60 0x7f3a0415dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35880/0 (socket says 172.21.15.76:35880) 2023-11-09T03:46:26.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.723+0000 7f3a08a4c640 1 -- 172.21.15.76:0/3909376984 learned_addr learned my addr 172.21.15.76:0/3909376984 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:26.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.724+0000 7f3a08a4c640 1 -- 172.21.15.76:0/3909376984 >> v2:172.21.15.113:3300/0 conn(0x7f3a040b4880 msgr2=0x7f3a040b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:26.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.724+0000 7f3a08a4c640 1 --2- 172.21.15.76:0/3909376984 >> v2:172.21.15.113:3300/0 conn(0x7f3a040b4880 0x7f3a040b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.724+0000 7f3a08a4c640 1 -- 172.21.15.76:0/3909376984 >> v1:172.21.15.113:6789/0 conn(0x7f3a040cd6d0 legacy=0x7f3a04161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:26.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.724+0000 7f3a08a4c640 1 -- 172.21.15.76:0/3909376984 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a0415e400 con 0x7f3a04153e60 2023-11-09T03:46:26.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.724+0000 7f3a08a4c640 1 --2- 172.21.15.76:0/3909376984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a04153e60 0x7f3a0415dec0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3a040b8f40 tx=0x7f39f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307dbf735d5226ed server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:26.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.724+0000 7f3a037fe640 1 -- 172.21.15.76:0/3909376984 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f4031870 con 0x7f3a04153e60 2023-11-09T03:46:26.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.724+0000 7f3a037fe640 1 -- 172.21.15.76:0/3909376984 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f40319d0 con 0x7f3a04153e60 2023-11-09T03:46:26.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.724+0000 7f3a037fe640 1 -- 172.21.15.76:0/3909376984 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f4031d60 con 0x7f3a04153e60 2023-11-09T03:46:26.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.725+0000 7f3a0acb0640 1 -- 172.21.15.76:0/3909376984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a04153e60 msgr2=0x7f3a0415dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:26.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.725+0000 7f3a0acb0640 1 --2- 172.21.15.76:0/3909376984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a04153e60 0x7f3a0415dec0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3a040b8f40 tx=0x7f39f402b910 comp rx=0 tx=0).stop 2023-11-09T03:46:26.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.725+0000 7f3a0acb0640 1 -- 172.21.15.76:0/3909376984 shutdown_connections 2023-11-09T03:46:26.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.725+0000 7f3a0acb0640 1 --2- 172.21.15.76:0/3909376984 >> v2:172.21.15.113:3300/0 conn(0x7f3a040b4880 0x7f3a040b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.725+0000 7f3a0acb0640 1 --2- 172.21.15.76:0/3909376984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a04153e60 0x7f3a0415dec0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.725+0000 7f3a0acb0640 1 -- 172.21.15.76:0/3909376984 >> 172.21.15.76:0/3909376984 conn(0x7f3a040cf880 msgr2=0x7f3a040cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:26.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.725+0000 7f3a0acb0640 1 -- 172.21.15.76:0/3909376984 shutdown_connections 2023-11-09T03:46:26.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.725+0000 7f3a0acb0640 1 -- 172.21.15.76:0/3909376984 wait complete. 2023-11-09T03:46:26.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.726+0000 7f3a0acb0640 1 Processor -- start 2023-11-09T03:46:26.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.726+0000 7f3a0acb0640 1 -- start start 2023-11-09T03:46:26.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.727+0000 7f3a0acb0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a040b4880 0x7f3a0415a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:26.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.727+0000 7f3a0acb0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a040cd6d0 0x7f3a040cca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:26.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.727+0000 7f3a0acb0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a04153e60 0x7f3a040cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:26.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.727+0000 7f3a0acb0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a041533a0 con 0x7f3a040b4880 2023-11-09T03:46:26.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.727+0000 7f3a0acb0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a040cd580 con 0x7f3a040cd6d0 2023-11-09T03:46:26.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.727+0000 7f3a0acb0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a040ca9a0 con 0x7f3a04153e60 2023-11-09T03:46:26.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.727+0000 7f3a08a4c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a040b4880 0x7f3a0415a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:26.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.727+0000 7f3a08a4c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a040b4880 0x7f3a0415a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35884/0 (socket says 172.21.15.76:35884) 2023-11-09T03:46:26.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.727+0000 7f3a08a4c640 1 -- 172.21.15.76:0/522026278 learned_addr learned my addr 172.21.15.76:0/522026278 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:26.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.727+0000 7f3a08a4c640 1 -- 172.21.15.76:0/522026278 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a04153e60 msgr2=0x7f3a040cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:26.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.727+0000 7f3a08a4c640 1 --2- 172.21.15.76:0/522026278 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a04153e60 0x7f3a040cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.727+0000 7f3a03fff640 1 --2- 172.21.15.76:0/522026278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a040cd6d0 0x7f3a040cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:26.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.727+0000 7f3a08a4c640 1 -- 172.21.15.76:0/522026278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a040cd6d0 msgr2=0x7f3a040cca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:26.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.727+0000 7f3a08a4c640 1 --2- 172.21.15.76:0/522026278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a040cd6d0 0x7f3a040cca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.728+0000 7f3a08a4c640 1 -- 172.21.15.76:0/522026278 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f40094d0 con 0x7f3a040b4880 2023-11-09T03:46:26.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.728+0000 7f3a03fff640 1 --2- 172.21.15.76:0/522026278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a040cd6d0 0x7f3a040cca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:26.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.728+0000 7f3a08a4c640 1 --2- 172.21.15.76:0/522026278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a040b4880 0x7f3a0415a9e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f39f402b950 tx=0x7f39f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:26.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.728+0000 7f3a017fa640 1 -- 172.21.15.76:0/522026278 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f403ad10 con 0x7f3a040b4880 2023-11-09T03:46:26.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.728+0000 7f3a017fa640 1 -- 172.21.15.76:0/522026278 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f403ae70 con 0x7f3a040b4880 2023-11-09T03:46:26.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.728+0000 7f3a0acb0640 1 -- 172.21.15.76:0/522026278 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a040ca070 con 0x7f3a040b4880 2023-11-09T03:46:26.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.728+0000 7f3a017fa640 1 -- 172.21.15.76:0/522026278 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f4044610 con 0x7f3a040b4880 2023-11-09T03:46:26.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.728+0000 7f3a0acb0640 1 -- 172.21.15.76:0/522026278 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a040ca630 con 0x7f3a040b4880 2023-11-09T03:46:26.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.729+0000 7f3a0acb0640 1 -- 172.21.15.76:0/522026278 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a04074290 con 0x7f3a040b4880 2023-11-09T03:46:26.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.729+0000 7f3a017fa640 1 -- 172.21.15.76:0/522026278 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f39f4044770 con 0x7f3a040b4880 2023-11-09T03:46:26.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.730+0000 7f3a017fa640 1 --2- 172.21.15.76:0/522026278 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f39f006ae70 0x7f39f006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:26.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.730+0000 7f3a017fa640 1 -- 172.21.15.76:0/522026278 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f39f402d040 con 0x7f3a040b4880 2023-11-09T03:46:26.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.730+0000 7f3a03fff640 1 --2- 172.21.15.76:0/522026278 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f39f006ae70 0x7f39f006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:26.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.730+0000 7f3a03fff640 1 --2- 172.21.15.76:0/522026278 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f39f006ae70 0x7f39f006d330 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f39f8007fd0 tx=0x7f39f8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:26.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.734+0000 7f3a017fa640 1 -- 172.21.15.76:0/522026278 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f39f4076f20 con 0x7f3a040b4880 2023-11-09T03:46:26.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.898+0000 7f3a0acb0640 1 -- 172.21.15.76:0/522026278 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3a040d1d00 con 0x7f3a040b4880 2023-11-09T03:46:26.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.898+0000 7f3a017fa640 1 -- 172.21.15.76:0/522026278 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f39f407a540 con 0x7f3a040b4880 2023-11-09T03:46:26.898 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:46:26.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.900+0000 7f3a0acb0640 1 -- 172.21.15.76:0/522026278 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f39f006ae70 msgr2=0x7f39f006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:26.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.900+0000 7f3a0acb0640 1 --2- 172.21.15.76:0/522026278 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f39f006ae70 0x7f39f006d330 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f39f8007fd0 tx=0x7f39f8009040 comp rx=0 tx=0).stop 2023-11-09T03:46:26.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.900+0000 7f3a0acb0640 1 -- 172.21.15.76:0/522026278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a040b4880 msgr2=0x7f3a0415a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:26.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.900+0000 7f3a0acb0640 1 --2- 172.21.15.76:0/522026278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a040b4880 0x7f3a0415a9e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f39f402b950 tx=0x7f39f40046c0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.900+0000 7f3a0acb0640 1 -- 172.21.15.76:0/522026278 shutdown_connections 2023-11-09T03:46:26.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.900+0000 7f3a0acb0640 1 --2- 172.21.15.76:0/522026278 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f39f006ae70 0x7f39f006d330 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.900+0000 7f3a0acb0640 1 --2- 172.21.15.76:0/522026278 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a04153e60 0x7f3a040cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.900+0000 7f3a0acb0640 1 --2- 172.21.15.76:0/522026278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a040cd6d0 0x7f3a040cca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.900+0000 7f3a0acb0640 1 --2- 172.21.15.76:0/522026278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a040b4880 0x7f3a0415a9e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:26.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.900+0000 7f3a0acb0640 1 -- 172.21.15.76:0/522026278 >> 172.21.15.76:0/522026278 conn(0x7f3a040cf880 msgr2=0x7f3a040d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:26.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.901+0000 7f3a0acb0640 1 -- 172.21.15.76:0/522026278 shutdown_connections 2023-11-09T03:46:26.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:26.901+0000 7f3a0acb0640 1 -- 172.21.15.76:0/522026278 wait complete. 2023-11-09T03:46:26.910 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:46:27.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.014+0000 7f792173e640 1 Processor -- start 2023-11-09T03:46:27.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.014+0000 7f792173e640 1 -- start start 2023-11-09T03:46:27.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.014+0000 7f792173e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f791c0cd6d0 0x7f791c161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:27.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.014+0000 7f792173e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f791c0cde60 con 0x7f791c153e60 2023-11-09T03:46:27.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.014+0000 7f792173e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f791c160f90 con 0x7f791c0b4880 2023-11-09T03:46:27.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.014+0000 7f792173e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791c161960 con 0x7f791c0cd6d0 2023-11-09T03:46:27.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.015+0000 7f791affd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f791c153e60 0x7f791c15dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53048/0 (socket says 172.21.15.76:53048) 2023-11-09T03:46:27.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.015+0000 7f791affd640 1 -- 172.21.15.76:0/1606219294 learned_addr learned my addr 172.21.15.76:0/1606219294 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:27.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.015+0000 7f791a7fc640 1 --2- 172.21.15.76:0/1606219294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f791c0cd6d0 0x7f791c161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:27.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.015+0000 7f7919ffb640 1 -- 172.21.15.76:0/1606219294 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 95139971 0 0) 0x7f791c0cde60 con 0x7f791c153e60 2023-11-09T03:46:27.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.015+0000 7f7919ffb640 1 -- 172.21.15.76:0/1606219294 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7904003520 con 0x7f791c153e60 2023-11-09T03:46:27.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.015+0000 7f7919ffb640 1 -- 172.21.15.76:0/1606219294 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2371347588 0 0) 0x7f791c160f90 con 0x7f791c0b4880 2023-11-09T03:46:27.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.015+0000 7f7919ffb640 1 -- 172.21.15.76:0/1606219294 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f791c0cde60 con 0x7f791c0b4880 2023-11-09T03:46:27.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.015+0000 7f7919ffb640 1 -- 172.21.15.76:0/1606219294 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 705504627 0 0) 0x7f7904003520 con 0x7f791c153e60 2023-11-09T03:46:27.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.015+0000 7f7919ffb640 1 -- 172.21.15.76:0/1606219294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f791c0cd6d0 msgr2=0x7f791c161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:27.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.015+0000 7f7919ffb640 1 --2- 172.21.15.76:0/1606219294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f791c0cd6d0 0x7f791c161420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.015+0000 7f7919ffb640 1 -- 172.21.15.76:0/1606219294 >> v1:172.21.15.113:6789/0 conn(0x7f791c0b4880 legacy=0x7f791c0b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:27.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.015+0000 7f7919ffb640 1 -- 172.21.15.76:0/1606219294 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791c0ce090 con 0x7f791c153e60 2023-11-09T03:46:27.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.015+0000 7f7919ffb640 1 -- 172.21.15.76:0/1606219294 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f790c001670 con 0x7f791c153e60 2023-11-09T03:46:27.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.015+0000 7f791a7fc640 1 --2- 172.21.15.76:0/1606219294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f791c0cd6d0 0x7f791c161420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:46:27.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.015+0000 7f7919ffb640 1 -- 172.21.15.76:0/1606219294 >> v1:172.21.15.76:6789/0 conn(0x7f791c153e60 legacy=0x7f791c15dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:27.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.015+0000 7f7919ffb640 1 --2- 172.21.15.76:0/1606219294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7904003b30 0x7f7904003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:27.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.015+0000 7f7919ffb640 1 --2- 172.21.15.76:0/1606219294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79040047c0 0x7f7904004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:27.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.015+0000 7f7919ffb640 1 --2- 172.21.15.76:0/1606219294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f791c0cd6d0 0x7f7904005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:27.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.015+0000 7f7919ffb640 1 -- 172.21.15.76:0/1606219294 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791c161960 con 0x7f79040047c0 2023-11-09T03:46:27.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.015+0000 7f7919ffb640 1 -- 172.21.15.76:0/1606219294 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790400b9e0 con 0x7f7904003b30 2023-11-09T03:46:27.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.015+0000 7f7919ffb640 1 -- 172.21.15.76:0/1606219294 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f790400bba0 con 0x7f791c0cd6d0 2023-11-09T03:46:27.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.015+0000 7f791a7fc640 1 --2- 172.21.15.76:0/1606219294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79040047c0 0x7f7904004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:27.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.015+0000 7f791affd640 1 --2- 172.21.15.76:0/1606219294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7904003b30 0x7f7904003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:27.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.015+0000 7f791b7fe640 1 --2- 172.21.15.76:0/1606219294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f791c0cd6d0 0x7f7904005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:27.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.016+0000 7f791a7fc640 1 -- 172.21.15.76:0/1606219294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f791c0cd6d0 msgr2=0x7f7904005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:27.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.016+0000 7f791a7fc640 1 --2- 172.21.15.76:0/1606219294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f791c0cd6d0 0x7f7904005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.016+0000 7f791a7fc640 1 -- 172.21.15.76:0/1606219294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7904003b30 msgr2=0x7f7904003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:27.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.016+0000 7f791a7fc640 1 --2- 172.21.15.76:0/1606219294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7904003b30 0x7f7904003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.016+0000 7f791a7fc640 1 -- 172.21.15.76:0/1606219294 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7910009400 con 0x7f79040047c0 2023-11-09T03:46:27.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.016+0000 7f791a7fc640 1 --2- 172.21.15.76:0/1606219294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79040047c0 0x7f7904004c40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7910009c60 tx=0x7f79040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7cdfdfaeabd7c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:27.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.016+0000 7f7919ffb640 1 -- 172.21.15.76:0/1606219294 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791002c8a0 con 0x7f79040047c0 2023-11-09T03:46:27.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.016+0000 7f7919ffb640 1 -- 172.21.15.76:0/1606219294 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f791002ca00 con 0x7f79040047c0 2023-11-09T03:46:27.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.016+0000 7f7919ffb640 1 -- 172.21.15.76:0/1606219294 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791002cd70 con 0x7f79040047c0 2023-11-09T03:46:27.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.017+0000 7f792173e640 1 -- 172.21.15.76:0/1606219294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79040047c0 msgr2=0x7f7904004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:27.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.017+0000 7f792173e640 1 --2- 172.21.15.76:0/1606219294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79040047c0 0x7f7904004c40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7910009c60 tx=0x7f79040057f0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.017+0000 7f791b7fe640 1 -- 172.21.15.76:0/1606219294 reap_dead start 2023-11-09T03:46:27.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.017+0000 7f792173e640 1 -- 172.21.15.76:0/1606219294 shutdown_connections 2023-11-09T03:46:27.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.017+0000 7f792173e640 1 -- 172.21.15.76:0/1606219294 >> 172.21.15.76:0/1606219294 conn(0x7f791c0cf880 msgr2=0x7f791c0cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:27.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.017+0000 7f792173e640 1 -- 172.21.15.76:0/1606219294 shutdown_connections 2023-11-09T03:46:27.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.017+0000 7f792173e640 1 -- 172.21.15.76:0/1606219294 wait complete. 2023-11-09T03:46:27.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.018+0000 7f792173e640 1 Processor -- start 2023-11-09T03:46:27.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.018+0000 7f792173e640 1 -- start start 2023-11-09T03:46:27.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.019+0000 7f792173e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79040047c0 0x7f791c150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:27.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.019+0000 7f792173e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f791c0cba30 0x7f791c0c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:27.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.019+0000 7f792173e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f791c0ca410 0x7f791c15baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:27.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.019+0000 7f792173e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791c0cb610 con 0x7f791c0cba30 2023-11-09T03:46:27.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.019+0000 7f792173e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791c150940 con 0x7f791c0ca410 2023-11-09T03:46:27.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.019+0000 7f792173e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791c0ca990 con 0x7f79040047c0 2023-11-09T03:46:27.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.019+0000 7f791a7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f791c0cba30 0x7f791c0c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:27.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.019+0000 7f791a7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f791c0cba30 0x7f791c0c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35900/0 (socket says 172.21.15.76:35900) 2023-11-09T03:46:27.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.019+0000 7f791a7fc640 1 -- 172.21.15.76:0/3918446777 learned_addr learned my addr 172.21.15.76:0/3918446777 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:27.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.019+0000 7f791a7fc640 1 -- 172.21.15.76:0/3918446777 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79040047c0 msgr2=0x7f791c150400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:27.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.019+0000 7f791a7fc640 1 --2- 172.21.15.76:0/3918446777 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79040047c0 0x7f791c150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.019+0000 7f791a7fc640 1 -- 172.21.15.76:0/3918446777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f791c0ca410 msgr2=0x7f791c15baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:27.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.019+0000 7f791a7fc640 1 --2- 172.21.15.76:0/3918446777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f791c0ca410 0x7f791c15baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.019+0000 7f791a7fc640 1 -- 172.21.15.76:0/3918446777 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791c0cac70 con 0x7f791c0cba30 2023-11-09T03:46:27.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.020+0000 7f791a7fc640 1 --2- 172.21.15.76:0/3918446777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f791c0cba30 0x7f791c0c9ed0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7910037040 tx=0x7f7910004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:27.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.020+0000 7f79037fe640 1 -- 172.21.15.76:0/3918446777 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7910004020 con 0x7f791c0cba30 2023-11-09T03:46:27.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.020+0000 7f79037fe640 1 -- 172.21.15.76:0/3918446777 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7910004180 con 0x7f791c0cba30 2023-11-09T03:46:27.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.020+0000 7f792173e640 1 -- 172.21.15.76:0/3918446777 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f791c0caf50 con 0x7f791c0cba30 2023-11-09T03:46:27.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.020+0000 7f79037fe640 1 -- 172.21.15.76:0/3918446777 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79100044f0 con 0x7f791c0cba30 2023-11-09T03:46:27.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.020+0000 7f792173e640 1 -- 172.21.15.76:0/3918446777 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f791c0c8850 con 0x7f791c0cba30 2023-11-09T03:46:27.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.021+0000 7f792173e640 1 -- 172.21.15.76:0/3918446777 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f791c074290 con 0x7f791c0cba30 2023-11-09T03:46:27.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.021+0000 7f79037fe640 1 -- 172.21.15.76:0/3918446777 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7910034980 con 0x7f791c0cba30 2023-11-09T03:46:27.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.022+0000 7f79037fe640 1 --2- 172.21.15.76:0/3918446777 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f790806aea0 0x7f790806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:27.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.022+0000 7f79037fe640 1 -- 172.21.15.76:0/3918446777 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f79100abbd0 con 0x7f791c0cba30 2023-11-09T03:46:27.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.022+0000 7f791affd640 1 --2- 172.21.15.76:0/3918446777 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f790806aea0 0x7f790806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:27.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.022+0000 7f791affd640 1 --2- 172.21.15.76:0/3918446777 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f790806aea0 0x7f790806d360 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f790c001af0 tx=0x7f790c001a80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:27.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.026+0000 7f79037fe640 1 -- 172.21.15.76:0/3918446777 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7910076930 con 0x7f791c0cba30 2023-11-09T03:46:27.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.192+0000 7f792173e640 1 -- 172.21.15.76:0/3918446777 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f791c0cb750 con 0x7f791c0cba30 2023-11-09T03:46:27.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.192+0000 7f79037fe640 1 -- 172.21.15.76:0/3918446777 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f791c0cb750 con 0x7f791c0cba30 2023-11-09T03:46:27.192 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:46:27.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.194+0000 7f792173e640 1 -- 172.21.15.76:0/3918446777 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f790806aea0 msgr2=0x7f790806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:27.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.194+0000 7f792173e640 1 --2- 172.21.15.76:0/3918446777 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f790806aea0 0x7f790806d360 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f790c001af0 tx=0x7f790c001a80 comp rx=0 tx=0).stop 2023-11-09T03:46:27.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.194+0000 7f792173e640 1 -- 172.21.15.76:0/3918446777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f791c0cba30 msgr2=0x7f791c0c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:27.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.194+0000 7f792173e640 1 --2- 172.21.15.76:0/3918446777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f791c0cba30 0x7f791c0c9ed0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7910037040 tx=0x7f7910004730 comp rx=0 tx=0).stop 2023-11-09T03:46:27.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.194+0000 7f792173e640 1 -- 172.21.15.76:0/3918446777 shutdown_connections 2023-11-09T03:46:27.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.194+0000 7f792173e640 1 --2- 172.21.15.76:0/3918446777 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f790806aea0 0x7f790806d360 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.194+0000 7f792173e640 1 --2- 172.21.15.76:0/3918446777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f791c0ca410 0x7f791c15baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.194+0000 7f792173e640 1 --2- 172.21.15.76:0/3918446777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f791c0cba30 0x7f791c0c9ed0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.194+0000 7f792173e640 1 --2- 172.21.15.76:0/3918446777 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79040047c0 0x7f791c150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.194+0000 7f792173e640 1 -- 172.21.15.76:0/3918446777 >> 172.21.15.76:0/3918446777 conn(0x7f791c0cf880 msgr2=0x7f791c15fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:27.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.194+0000 7f792173e640 1 -- 172.21.15.76:0/3918446777 shutdown_connections 2023-11-09T03:46:27.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.194+0000 7f792173e640 1 -- 172.21.15.76:0/3918446777 wait complete. 2023-11-09T03:46:27.204 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-09T03:46:27.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.307+0000 7fb1b3bc0640 1 Processor -- start 2023-11-09T03:46:27.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.308+0000 7fb1b3bc0640 1 -- start start 2023-11-09T03:46:27.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.308+0000 7fb1b3bc0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1ac159c00 0x7fb1ac0cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:27.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.308+0000 7fb1b3bc0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb1ac0ae1d0 0x7fb1ac0c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:27.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.308+0000 7fb1b3bc0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1ac15aff0 con 0x7fb1ac0c48e0 2023-11-09T03:46:27.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.308+0000 7fb1b3bc0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1ac0c52a0 con 0x7fb1ac0ae1d0 2023-11-09T03:46:27.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.308+0000 7fb1b3bc0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac15a3e0 con 0x7fb1ac159c00 2023-11-09T03:46:27.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.309+0000 7fb1b215d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb1ac0ae1d0 0x7fb1ac0c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:27.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.309+0000 7fb1b215d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb1ac0ae1d0 0x7fb1ac0c4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35910/0 (socket says 172.21.15.76:35910) 2023-11-09T03:46:27.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.309+0000 7fb1b215d640 1 -- 172.21.15.76:0/2648888005 learned_addr learned my addr 172.21.15.76:0/2648888005 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:27.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.309+0000 7fb1b195c640 1 --2- 172.21.15.76:0/2648888005 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1ac159c00 0x7fb1ac0cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:27.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.309+0000 7fb1b215d640 1 -- 172.21.15.76:0/2648888005 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1ac159c00 msgr2=0x7fb1ac0cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:27.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.309+0000 7fb1b215d640 1 --2- 172.21.15.76:0/2648888005 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1ac159c00 0x7fb1ac0cc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.309+0000 7fb1b215d640 1 -- 172.21.15.76:0/2648888005 >> v1:172.21.15.76:6789/0 conn(0x7fb1ac0c48e0 legacy=0x7fb1ac0adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:27.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.309+0000 7fb1b215d640 1 -- 172.21.15.76:0/2648888005 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ac15a6c0 con 0x7fb1ac0ae1d0 2023-11-09T03:46:27.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.310+0000 7fb1b215d640 1 --2- 172.21.15.76:0/2648888005 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1ac0ae1d0 0x7fb1ac0c4d60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4009e50 tx=0x7fb1a402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=536b0115de7dee86 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:27.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.310+0000 7fb1b095a640 1 -- 172.21.15.76:0/2648888005 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a402cb00 con 0x7fb1ac0ae1d0 2023-11-09T03:46:27.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.310+0000 7fb1b095a640 1 -- 172.21.15.76:0/2648888005 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a402cc60 con 0x7fb1ac0ae1d0 2023-11-09T03:46:27.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.310+0000 7fb1b095a640 1 -- 172.21.15.76:0/2648888005 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a40343f0 con 0x7fb1ac0ae1d0 2023-11-09T03:46:27.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.310+0000 7fb1b3bc0640 1 -- 172.21.15.76:0/2648888005 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1ac0ae1d0 msgr2=0x7fb1ac0c4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:27.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.310+0000 7fb1b3bc0640 1 --2- 172.21.15.76:0/2648888005 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1ac0ae1d0 0x7fb1ac0c4d60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4009e50 tx=0x7fb1a402bf60 comp rx=0 tx=0).stop 2023-11-09T03:46:27.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.310+0000 7fb1b3bc0640 1 -- 172.21.15.76:0/2648888005 shutdown_connections 2023-11-09T03:46:27.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.310+0000 7fb1b3bc0640 1 --2- 172.21.15.76:0/2648888005 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1ac0ae1d0 0x7fb1ac0c4d60 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.310+0000 7fb1b3bc0640 1 --2- 172.21.15.76:0/2648888005 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1ac159c00 0x7fb1ac0cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.310+0000 7fb1b3bc0640 1 -- 172.21.15.76:0/2648888005 >> 172.21.15.76:0/2648888005 conn(0x7fb1ac0ce950 msgr2=0x7fb1ac0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:27.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.310+0000 7fb1b3bc0640 1 -- 172.21.15.76:0/2648888005 shutdown_connections 2023-11-09T03:46:27.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.310+0000 7fb1b3bc0640 1 -- 172.21.15.76:0/2648888005 wait complete. 2023-11-09T03:46:27.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.311+0000 7fb1b3bc0640 1 Processor -- start 2023-11-09T03:46:27.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.311+0000 7fb1b3bc0640 1 -- start start 2023-11-09T03:46:27.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.312+0000 7fb1b3bc0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1ac0ae1d0 0x7fb1ac0c7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:27.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.312+0000 7fb1b3bc0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1ac0c48e0 0x7fb1ac0b2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:27.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.312+0000 7fb1b3bc0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1ac159c00 0x7fb1ac14c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:27.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.312+0000 7fb1b3bc0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac0b2ce0 con 0x7fb1ac0c48e0 2023-11-09T03:46:27.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.312+0000 7fb1b3bc0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac1f0090 con 0x7fb1ac0ae1d0 2023-11-09T03:46:27.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.312+0000 7fb1b3bc0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac0b2a00 con 0x7fb1ac159c00 2023-11-09T03:46:27.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.312+0000 7fb1b115b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1ac0c48e0 0x7fb1ac0b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:27.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.312+0000 7fb1b115b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1ac0c48e0 0x7fb1ac0b2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35922/0 (socket says 172.21.15.76:35922) 2023-11-09T03:46:27.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.312+0000 7fb1b115b640 1 -- 172.21.15.76:0/948068652 learned_addr learned my addr 172.21.15.76:0/948068652 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:27.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.313+0000 7fb1b115b640 1 -- 172.21.15.76:0/948068652 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1ac159c00 msgr2=0x7fb1ac14c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:27.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.313+0000 7fb1b115b640 1 --2- 172.21.15.76:0/948068652 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1ac159c00 0x7fb1ac14c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.313+0000 7fb1b115b640 1 -- 172.21.15.76:0/948068652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1ac0ae1d0 msgr2=0x7fb1ac0c7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:27.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.313+0000 7fb1b195c640 1 --2- 172.21.15.76:0/948068652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1ac0ae1d0 0x7fb1ac0c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:27.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.313+0000 7fb1b115b640 1 --2- 172.21.15.76:0/948068652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1ac0ae1d0 0x7fb1ac0c7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.313+0000 7fb1b115b640 1 -- 172.21.15.76:0/948068652 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a4009b30 con 0x7fb1ac0c48e0 2023-11-09T03:46:27.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.313+0000 7fb1b195c640 1 --2- 172.21.15.76:0/948068652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1ac0ae1d0 0x7fb1ac0c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:27.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.313+0000 7fb1b115b640 1 --2- 172.21.15.76:0/948068652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1ac0c48e0 0x7fb1ac0b2060 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00c400 tx=0x7fb19c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:27.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.313+0000 7fb1a27fc640 1 -- 172.21.15.76:0/948068652 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c0052d0 con 0x7fb1ac0c48e0 2023-11-09T03:46:27.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.313+0000 7fb1a27fc640 1 -- 172.21.15.76:0/948068652 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19c005430 con 0x7fb1ac0c48e0 2023-11-09T03:46:27.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.313+0000 7fb1a27fc640 1 -- 172.21.15.76:0/948068652 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c003830 con 0x7fb1ac0c48e0 2023-11-09T03:46:27.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.313+0000 7fb1b3bc0640 1 -- 172.21.15.76:0/948068652 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1ac1f0310 con 0x7fb1ac0c48e0 2023-11-09T03:46:27.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.313+0000 7fb1b3bc0640 1 -- 172.21.15.76:0/948068652 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1ac1f08d0 con 0x7fb1ac0c48e0 2023-11-09T03:46:27.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.314+0000 7fb1b3bc0640 1 -- 172.21.15.76:0/948068652 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1ac1f0ac0 con 0x7fb1ac0c48e0 2023-11-09T03:46:27.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.314+0000 7fb1a27fc640 1 -- 172.21.15.76:0/948068652 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb19c003990 con 0x7fb1ac0c48e0 2023-11-09T03:46:27.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.315+0000 7fb1a27fc640 1 --2- 172.21.15.76:0/948068652 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb19006ae70 0x7fb19006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:27.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.315+0000 7fb1a27fc640 1 -- 172.21.15.76:0/948068652 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fb19c088f80 con 0x7fb1ac0c48e0 2023-11-09T03:46:27.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.315+0000 7fb1b195c640 1 --2- 172.21.15.76:0/948068652 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb19006ae70 0x7fb19006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:27.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.315+0000 7fb1b195c640 1 --2- 172.21.15.76:0/948068652 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb19006ae70 0x7fb19006d330 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac0cf230 tx=0x7fb1a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:27.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.319+0000 7fb1a27fc640 1 -- 172.21.15.76:0/948068652 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb19c053c40 con 0x7fb1ac0c48e0 2023-11-09T03:46:27.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.491+0000 7fb1b3bc0640 1 -- 172.21.15.76:0/948068652 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb1ac074290 con 0x7fb1ac0c48e0 2023-11-09T03:46:27.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.491+0000 7fb1a27fc640 1 -- 172.21.15.76:0/948068652 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb19c0170a0 con 0x7fb1ac0c48e0 2023-11-09T03:46:27.491 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:46:27.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.493+0000 7fb1b3bc0640 1 -- 172.21.15.76:0/948068652 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb19006ae70 msgr2=0x7fb19006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:27.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.493+0000 7fb1b3bc0640 1 --2- 172.21.15.76:0/948068652 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb19006ae70 0x7fb19006d330 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac0cf230 tx=0x7fb1a8008040 comp rx=0 tx=0).stop 2023-11-09T03:46:27.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.493+0000 7fb1b3bc0640 1 -- 172.21.15.76:0/948068652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1ac0c48e0 msgr2=0x7fb1ac0b2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:27.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.493+0000 7fb1b3bc0640 1 --2- 172.21.15.76:0/948068652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1ac0c48e0 0x7fb1ac0b2060 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00c400 tx=0x7fb19c00c8c0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.493+0000 7fb1b3bc0640 1 -- 172.21.15.76:0/948068652 shutdown_connections 2023-11-09T03:46:27.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.493+0000 7fb1b3bc0640 1 --2- 172.21.15.76:0/948068652 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb19006ae70 0x7fb19006d330 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.493+0000 7fb1b3bc0640 1 --2- 172.21.15.76:0/948068652 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1ac159c00 0x7fb1ac14c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.493+0000 7fb1b3bc0640 1 --2- 172.21.15.76:0/948068652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1ac0c48e0 0x7fb1ac0b2060 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.493+0000 7fb1b3bc0640 1 --2- 172.21.15.76:0/948068652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1ac0ae1d0 0x7fb1ac0c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.493+0000 7fb1b3bc0640 1 -- 172.21.15.76:0/948068652 >> 172.21.15.76:0/948068652 conn(0x7fb1ac0ce950 msgr2=0x7fb1ac0c5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:27.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.493+0000 7fb1b3bc0640 1 -- 172.21.15.76:0/948068652 shutdown_connections 2023-11-09T03:46:27.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.494+0000 7fb1b3bc0640 1 -- 172.21.15.76:0/948068652 wait complete. 2023-11-09T03:46:27.504 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-09T03:46:27.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.605+0000 7f5af2c44640 1 Processor -- start 2023-11-09T03:46:27.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.605+0000 7f5af2c44640 1 -- start start 2023-11-09T03:46:27.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.606+0000 7f5af2c44640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5aec0cd6d0 0x7f5aec161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:27.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.606+0000 7f5af2c44640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aec0cde60 con 0x7f5aec153e60 2023-11-09T03:46:27.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.606+0000 7f5af2c44640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aec160f90 con 0x7f5aec0b4880 2023-11-09T03:46:27.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.606+0000 7f5af2c44640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aec161960 con 0x7f5aec0cd6d0 2023-11-09T03:46:27.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.606+0000 7f5af09e0640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f5aec153e60 0x7f5aec15dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53062/0 (socket says 172.21.15.76:53062) 2023-11-09T03:46:27.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.606+0000 7f5af09e0640 1 -- 172.21.15.76:0/2997312214 learned_addr learned my addr 172.21.15.76:0/2997312214 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:27.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.607+0000 7f5aeb7fe640 1 -- 172.21.15.76:0/2997312214 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 661272851 0 0) 0x7f5aec0cde60 con 0x7f5aec153e60 2023-11-09T03:46:27.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.607+0000 7f5aeb7fe640 1 -- 172.21.15.76:0/2997312214 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ad4003520 con 0x7f5aec153e60 2023-11-09T03:46:27.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.607+0000 7f5aebfff640 1 --2- 172.21.15.76:0/2997312214 >> v2:172.21.15.113:3300/0 conn(0x7f5aec0cd6d0 0x7f5aec161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:27.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.607+0000 7f5aeb7fe640 1 -- 172.21.15.76:0/2997312214 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3527440107 0 0) 0x7f5ad4003520 con 0x7f5aec153e60 2023-11-09T03:46:27.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.607+0000 7f5aeb7fe640 1 -- 172.21.15.76:0/2997312214 >> v2:172.21.15.113:3300/0 conn(0x7f5aec0cd6d0 msgr2=0x7f5aec161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:27.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.607+0000 7f5aeb7fe640 1 --2- 172.21.15.76:0/2997312214 >> v2:172.21.15.113:3300/0 conn(0x7f5aec0cd6d0 0x7f5aec161420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.607+0000 7f5aeb7fe640 1 -- 172.21.15.76:0/2997312214 >> v1:172.21.15.113:6789/0 conn(0x7f5aec0b4880 legacy=0x7f5aec0d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:27.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.607+0000 7f5aeb7fe640 1 -- 172.21.15.76:0/2997312214 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec0ce090 con 0x7f5aec153e60 2023-11-09T03:46:27.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.607+0000 7f5aeb7fe640 1 -- 172.21.15.76:0/2997312214 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f5adc001670 con 0x7f5aec153e60 2023-11-09T03:46:27.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.607+0000 7f5aeb7fe640 1 -- 172.21.15.76:0/2997312214 >> v1:172.21.15.76:6789/0 conn(0x7f5aec153e60 legacy=0x7f5aec15dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:27.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.607+0000 7f5aeb7fe640 1 --2- 172.21.15.76:0/2997312214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ad40039c0 0x7f5ad4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:27.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.607+0000 7f5aeb7fe640 1 --2- 172.21.15.76:0/2997312214 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ad40047c0 0x7f5ad4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:27.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.607+0000 7f5aeb7fe640 1 --2- 172.21.15.76:0/2997312214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ad4005180 0x7f5ad4014190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:27.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.607+0000 7f5aeb7fe640 1 -- 172.21.15.76:0/2997312214 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec161960 con 0x7f5ad4005180 2023-11-09T03:46:27.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.607+0000 7f5aeb7fe640 1 -- 172.21.15.76:0/2997312214 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad4014890 con 0x7f5ad40039c0 2023-11-09T03:46:27.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.607+0000 7f5aeb7fe640 1 -- 172.21.15.76:0/2997312214 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad4014a50 con 0x7f5ad40047c0 2023-11-09T03:46:27.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.607+0000 7f5af11e1640 1 --2- 172.21.15.76:0/2997312214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ad4005180 0x7f5ad4014190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:27.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.607+0000 7f5af09e0640 1 --2- 172.21.15.76:0/2997312214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ad40039c0 0x7f5ad4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:27.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.607+0000 7f5aebfff640 1 --2- 172.21.15.76:0/2997312214 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ad40047c0 0x7f5ad4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:27.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.607+0000 7f5af11e1640 1 -- 172.21.15.76:0/2997312214 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ad40047c0 msgr2=0x7f5ad4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:27.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.607+0000 7f5af11e1640 1 --2- 172.21.15.76:0/2997312214 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ad40047c0 0x7f5ad4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.607+0000 7f5af11e1640 1 -- 172.21.15.76:0/2997312214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ad40039c0 msgr2=0x7f5ad4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:27.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.607+0000 7f5af11e1640 1 --2- 172.21.15.76:0/2997312214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ad40039c0 0x7f5ad4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.607+0000 7f5af11e1640 1 -- 172.21.15.76:0/2997312214 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad4003520 con 0x7f5ad4005180 2023-11-09T03:46:27.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.607+0000 7f5aebfff640 1 --2- 172.21.15.76:0/2997312214 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ad40047c0 0x7f5ad4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:27.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.608+0000 7f5af11e1640 1 -- 172.21.15.76:0/2997312214 reap_dead start 2023-11-09T03:46:27.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.608+0000 7f5af11e1640 1 --2- 172.21.15.76:0/2997312214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ad4005180 0x7f5ad4014190 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4001e50 tx=0x7f5ad40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57dd652fb0c8309a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:27.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.608+0000 7f5aeb7fe640 1 -- 172.21.15.76:0/2997312214 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4003b70 con 0x7f5ad4005180 2023-11-09T03:46:27.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.608+0000 7f5aeb7fe640 1 -- 172.21.15.76:0/2997312214 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae4003d10 con 0x7f5ad4005180 2023-11-09T03:46:27.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.608+0000 7f5aeb7fe640 1 -- 172.21.15.76:0/2997312214 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4005310 con 0x7f5ad4005180 2023-11-09T03:46:27.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.609+0000 7f5af2c44640 1 -- 172.21.15.76:0/2997312214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ad4005180 msgr2=0x7f5ad4014190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:27.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.609+0000 7f5af2c44640 1 --2- 172.21.15.76:0/2997312214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ad4005180 0x7f5ad4014190 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4001e50 tx=0x7f5ad40057f0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.609+0000 7f5af2c44640 1 -- 172.21.15.76:0/2997312214 shutdown_connections 2023-11-09T03:46:27.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.609+0000 7f5af2c44640 1 --2- 172.21.15.76:0/2997312214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ad4005180 0x7f5ad4014190 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.609+0000 7f5af2c44640 1 -- 172.21.15.76:0/2997312214 >> 172.21.15.76:0/2997312214 conn(0x7f5aec0cf880 msgr2=0x7f5aec0cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:27.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.609+0000 7f5af2c44640 1 -- 172.21.15.76:0/2997312214 shutdown_connections 2023-11-09T03:46:27.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.609+0000 7f5af2c44640 1 -- 172.21.15.76:0/2997312214 wait complete. 2023-11-09T03:46:27.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.610+0000 7f5af2c44640 1 Processor -- start 2023-11-09T03:46:27.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.610+0000 7f5af2c44640 1 -- start start 2023-11-09T03:46:27.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.611+0000 7f5af2c44640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ad4005180 0x7f5aec14b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:27.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.611+0000 7f5af2c44640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aec14bbd0 0x7f5aec14c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:27.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.611+0000 7f5af2c44640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5aec14eea0 0x7f5aec14f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:27.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.611+0000 7f5af2c44640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec14ce40 con 0x7f5aec14bbd0 2023-11-09T03:46:27.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.611+0000 7f5af2c44640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec14f860 con 0x7f5ad4005180 2023-11-09T03:46:27.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.611+0000 7f5af2c44640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec150320 con 0x7f5aec14eea0 2023-11-09T03:46:27.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.611+0000 7f5aebfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aec14bbd0 0x7f5aec14c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:27.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.611+0000 7f5aebfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aec14bbd0 0x7f5aec14c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35940/0 (socket says 172.21.15.76:35940) 2023-11-09T03:46:27.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.611+0000 7f5aebfff640 1 -- 172.21.15.76:0/3580644957 learned_addr learned my addr 172.21.15.76:0/3580644957 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:27.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.611+0000 7f5aebfff640 1 -- 172.21.15.76:0/3580644957 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5aec14eea0 msgr2=0x7f5aec14f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:27.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.611+0000 7f5aebfff640 1 --2- 172.21.15.76:0/3580644957 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5aec14eea0 0x7f5aec14f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.611+0000 7f5aebfff640 1 -- 172.21.15.76:0/3580644957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ad4005180 msgr2=0x7f5aec14b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:27.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.611+0000 7f5aebfff640 1 --2- 172.21.15.76:0/3580644957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ad4005180 0x7f5aec14b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.611+0000 7f5aebfff640 1 -- 172.21.15.76:0/3580644957 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec0c6370 con 0x7f5aec14bbd0 2023-11-09T03:46:27.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.611+0000 7f5aebfff640 1 --2- 172.21.15.76:0/3580644957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aec14bbd0 0x7f5aec14c050 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0011540 tx=0x7f5ae0011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:27.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.611+0000 7f5ae97fa640 1 -- 172.21.15.76:0/3580644957 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0015870 con 0x7f5aec14bbd0 2023-11-09T03:46:27.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.611+0000 7f5ae97fa640 1 -- 172.21.15.76:0/3580644957 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae00159d0 con 0x7f5aec14bbd0 2023-11-09T03:46:27.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.611+0000 7f5af2c44640 1 -- 172.21.15.76:0/3580644957 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aec14d0e0 con 0x7f5aec14bbd0 2023-11-09T03:46:27.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.611+0000 7f5ae97fa640 1 -- 172.21.15.76:0/3580644957 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0015d40 con 0x7f5aec14bbd0 2023-11-09T03:46:27.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.611+0000 7f5af2c44640 1 -- 172.21.15.76:0/3580644957 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aec0c6650 con 0x7f5aec14bbd0 2023-11-09T03:46:27.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.613+0000 7f5af2c44640 1 -- 172.21.15.76:0/3580644957 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aec0d1fb0 con 0x7f5aec14bbd0 2023-11-09T03:46:27.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.613+0000 7f5ae97fa640 1 -- 172.21.15.76:0/3580644957 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5ae000f460 con 0x7f5aec14bbd0 2023-11-09T03:46:27.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.613+0000 7f5ae97fa640 1 --2- 172.21.15.76:0/3580644957 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5ad806aea0 0x7f5ad806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:27.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.613+0000 7f5ae97fa640 1 -- 172.21.15.76:0/3580644957 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f5ae008a140 con 0x7f5aec14bbd0 2023-11-09T03:46:27.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.614+0000 7f5af09e0640 1 --2- 172.21.15.76:0/3580644957 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5ad806aea0 0x7f5ad806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:27.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.614+0000 7f5af09e0640 1 --2- 172.21.15.76:0/3580644957 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5ad806aea0 0x7f5ad806d360 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5adc001bc0 tx=0x7f5adc001a50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:27.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.618+0000 7f5ae97fa640 1 -- 172.21.15.76:0/3580644957 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ae0054e10 con 0x7f5aec14bbd0 2023-11-09T03:46:27.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.799+0000 7f5af2c44640 1 -- 172.21.15.76:0/3580644957 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5aec074290 con 0x7f5aec14bbd0 2023-11-09T03:46:27.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.800+0000 7f5ae97fa640 1 -- 172.21.15.76:0/3580644957 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5ae0058430 con 0x7f5aec14bbd0 2023-11-09T03:46:27.799 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2023-11-09T03:46:27.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.801+0000 7f5af2c44640 1 -- 172.21.15.76:0/3580644957 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5ad806aea0 msgr2=0x7f5ad806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:27.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.801+0000 7f5af2c44640 1 --2- 172.21.15.76:0/3580644957 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5ad806aea0 0x7f5ad806d360 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5adc001bc0 tx=0x7f5adc001a50 comp rx=0 tx=0).stop 2023-11-09T03:46:27.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.801+0000 7f5af2c44640 1 -- 172.21.15.76:0/3580644957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aec14bbd0 msgr2=0x7f5aec14c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:27.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.801+0000 7f5af2c44640 1 --2- 172.21.15.76:0/3580644957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aec14bbd0 0x7f5aec14c050 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0011540 tx=0x7f5ae0011a00 comp rx=0 tx=0).stop 2023-11-09T03:46:27.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.801+0000 7f5af2c44640 1 -- 172.21.15.76:0/3580644957 shutdown_connections 2023-11-09T03:46:27.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.801+0000 7f5af2c44640 1 --2- 172.21.15.76:0/3580644957 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5ad806aea0 0x7f5ad806d360 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.801+0000 7f5af2c44640 1 --2- 172.21.15.76:0/3580644957 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5aec14eea0 0x7f5aec14f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.801+0000 7f5af2c44640 1 --2- 172.21.15.76:0/3580644957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aec14bbd0 0x7f5aec14c050 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.801+0000 7f5af2c44640 1 --2- 172.21.15.76:0/3580644957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ad4005180 0x7f5aec14b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.801+0000 7f5af2c44640 1 -- 172.21.15.76:0/3580644957 >> 172.21.15.76:0/3580644957 conn(0x7f5aec0cf880 msgr2=0x7f5aec0d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:27.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.802+0000 7f5af2c44640 1 -- 172.21.15.76:0/3580644957 shutdown_connections 2023-11-09T03:46:27.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.802+0000 7f5af2c44640 1 -- 172.21.15.76:0/3580644957 wait complete. 2023-11-09T03:46:27.812 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:46:27.812 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:46:27.831 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:46:27.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.932+0000 7fa0bf007640 1 Processor -- start 2023-11-09T03:46:27.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.932+0000 7fa0bf007640 1 -- start start 2023-11-09T03:46:27.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.933+0000 7fa0bf007640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa0b8159d30 0x7fa0b815d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:27.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.933+0000 7fa0bf007640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0b806dc70 0x7fa0b815dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:27.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.933+0000 7fa0bf007640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0b815a110 con 0x7fa0b806e630 2023-11-09T03:46:27.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.933+0000 7fa0bf007640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0b806e450 con 0x7fa0b8159d30 2023-11-09T03:46:27.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.933+0000 7fa0bf007640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b80ce630 con 0x7fa0b806dc70 2023-11-09T03:46:27.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.933+0000 7fa0bcda3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa0b8159d30 0x7fa0b815d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:27.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.933+0000 7fa0bcda3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa0b8159d30 0x7fa0b815d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35944/0 (socket says 172.21.15.76:35944) 2023-11-09T03:46:27.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.933+0000 7fa0bcda3640 1 -- 172.21.15.76:0/741253152 learned_addr learned my addr 172.21.15.76:0/741253152 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:27.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.933+0000 7fa0bcda3640 1 -- 172.21.15.76:0/741253152 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0b806dc70 msgr2=0x7fa0b815dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:27.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.933+0000 7fa0bcda3640 1 --2- 172.21.15.76:0/741253152 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0b806dc70 0x7fa0b815dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.933+0000 7fa0bcda3640 1 -- 172.21.15.76:0/741253152 >> v1:172.21.15.76:6789/0 conn(0x7fa0b806e630 legacy=0x7fa0b806c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:27.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.933+0000 7fa0bcda3640 1 -- 172.21.15.76:0/741253152 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b80adb90 con 0x7fa0b8159d30 2023-11-09T03:46:27.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.934+0000 7fa0bcda3640 1 --2- 172.21.15.76:0/741253152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0b8159d30 0x7fa0b815d7b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8009870 tx=0x7fa0a802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a98664acc795a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:27.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.934+0000 7fa0b77fe640 1 -- 172.21.15.76:0/741253152 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8030840 con 0x7fa0b8159d30 2023-11-09T03:46:27.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.934+0000 7fa0b77fe640 1 -- 172.21.15.76:0/741253152 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a80309a0 con 0x7fa0b8159d30 2023-11-09T03:46:27.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.934+0000 7fa0b77fe640 1 -- 172.21.15.76:0/741253152 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8030d30 con 0x7fa0b8159d30 2023-11-09T03:46:27.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.935+0000 7fa0bf007640 1 -- 172.21.15.76:0/741253152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0b8159d30 msgr2=0x7fa0b815d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:27.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.935+0000 7fa0bf007640 1 --2- 172.21.15.76:0/741253152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0b8159d30 0x7fa0b815d7b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8009870 tx=0x7fa0a802b980 comp rx=0 tx=0).stop 2023-11-09T03:46:27.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.935+0000 7fa0bf007640 1 -- 172.21.15.76:0/741253152 shutdown_connections 2023-11-09T03:46:27.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.935+0000 7fa0bf007640 1 --2- 172.21.15.76:0/741253152 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0b806dc70 0x7fa0b815dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.935+0000 7fa0bf007640 1 --2- 172.21.15.76:0/741253152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0b8159d30 0x7fa0b815d7b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.935+0000 7fa0bf007640 1 -- 172.21.15.76:0/741253152 >> 172.21.15.76:0/741253152 conn(0x7fa0b80b03c0 msgr2=0x7fa0b80b07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:27.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.935+0000 7fa0bf007640 1 -- 172.21.15.76:0/741253152 shutdown_connections 2023-11-09T03:46:27.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.935+0000 7fa0bf007640 1 -- 172.21.15.76:0/741253152 wait complete. 2023-11-09T03:46:27.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.936+0000 7fa0bf007640 1 Processor -- start 2023-11-09T03:46:27.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.936+0000 7fa0bf007640 1 -- start start 2023-11-09T03:46:27.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.937+0000 7fa0bf007640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0b806dc70 0x7fa0b806ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:27.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.937+0000 7fa0bf007640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0b806e630 0x7fa0b81e4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:27.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.937+0000 7fa0bf007640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0b8159d30 0x7fa0b8070610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:27.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.937+0000 7fa0bf007640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b80a8810 con 0x7fa0b806e630 2023-11-09T03:46:27.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.937+0000 7fa0bf007640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b80a84c0 con 0x7fa0b806dc70 2023-11-09T03:46:27.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.937+0000 7fa0bf007640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b81e58e0 con 0x7fa0b8159d30 2023-11-09T03:46:27.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.937+0000 7fa0b7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0b806e630 0x7fa0b81e4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:27.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.937+0000 7fa0b7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0b806e630 0x7fa0b81e4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35952/0 (socket says 172.21.15.76:35952) 2023-11-09T03:46:27.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.937+0000 7fa0b7fff640 1 -- 172.21.15.76:0/2505188784 learned_addr learned my addr 172.21.15.76:0/2505188784 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:27.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.937+0000 7fa0b7fff640 1 -- 172.21.15.76:0/2505188784 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0b8159d30 msgr2=0x7fa0b8070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:27.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.937+0000 7fa0b7fff640 1 --2- 172.21.15.76:0/2505188784 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0b8159d30 0x7fa0b8070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.937+0000 7fa0b7fff640 1 -- 172.21.15.76:0/2505188784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0b806dc70 msgr2=0x7fa0b806ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:27.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.937+0000 7fa0b7fff640 1 --2- 172.21.15.76:0/2505188784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0b806dc70 0x7fa0b806ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:27.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.937+0000 7fa0b7fff640 1 -- 172.21.15.76:0/2505188784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a80094d0 con 0x7fa0b806e630 2023-11-09T03:46:27.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.937+0000 7fa0b7fff640 1 --2- 172.21.15.76:0/2505188784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0b806e630 0x7fa0b81e4e20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00b4a0 tx=0x7fa0ac00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:27.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.938+0000 7fa0b57fa640 1 -- 172.21.15.76:0/2505188784 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac005a10 con 0x7fa0b806e630 2023-11-09T03:46:27.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.938+0000 7fa0b57fa640 1 -- 172.21.15.76:0/2505188784 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ac005b70 con 0x7fa0b806e630 2023-11-09T03:46:27.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.938+0000 7fa0b57fa640 1 -- 172.21.15.76:0/2505188784 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac010610 con 0x7fa0b806e630 2023-11-09T03:46:27.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.938+0000 7fa0bf007640 1 -- 172.21.15.76:0/2505188784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b81e5bc0 con 0x7fa0b806e630 2023-11-09T03:46:27.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.938+0000 7fa0bf007640 1 -- 172.21.15.76:0/2505188784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b815bc40 con 0x7fa0b806e630 2023-11-09T03:46:27.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.939+0000 7fa0bf007640 1 -- 172.21.15.76:0/2505188784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b8074290 con 0x7fa0b806e630 2023-11-09T03:46:27.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.939+0000 7fa0b57fa640 1 -- 172.21.15.76:0/2505188784 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa0ac010770 con 0x7fa0b806e630 2023-11-09T03:46:27.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.940+0000 7fa0b57fa640 1 --2- 172.21.15.76:0/2505188784 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa0a406ae70 0x7fa0a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:27.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.940+0000 7fa0b57fa640 1 -- 172.21.15.76:0/2505188784 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fa0ac089220 con 0x7fa0b806e630 2023-11-09T03:46:27.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.940+0000 7fa0bcda3640 1 --2- 172.21.15.76:0/2505188784 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa0a406ae70 0x7fa0a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:27.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.940+0000 7fa0bcda3640 1 --2- 172.21.15.76:0/2505188784 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa0a406ae70 0x7fa0a406d330 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8036000 tx=0x7fa0a803b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:27.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:27.944+0000 7fa0b57fa640 1 -- 172.21.15.76:0/2505188784 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa0ac053f00 con 0x7fa0b806e630 2023-11-09T03:46:28.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.111+0000 7fa0bf007640 1 -- 172.21.15.76:0/2505188784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0b80a8950 con 0x7fa0b806e630 2023-11-09T03:46:28.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.112+0000 7fa0b57fa640 1 -- 172.21.15.76:0/2505188784 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18710 (secure 0 0 0) 0x7fa0ac057520 con 0x7fa0b806e630 2023-11-09T03:46:28.112 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:28.112 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":27,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:46:12.537265+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:46:28.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.114+0000 7fa0bf007640 1 -- 172.21.15.76:0/2505188784 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa0a406ae70 msgr2=0x7fa0a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:28.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.114+0000 7fa0bf007640 1 --2- 172.21.15.76:0/2505188784 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa0a406ae70 0x7fa0a406d330 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8036000 tx=0x7fa0a803b040 comp rx=0 tx=0).stop 2023-11-09T03:46:28.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.114+0000 7fa0bf007640 1 -- 172.21.15.76:0/2505188784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0b806e630 msgr2=0x7fa0b81e4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:28.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.114+0000 7fa0bf007640 1 --2- 172.21.15.76:0/2505188784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0b806e630 0x7fa0b81e4e20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00b4a0 tx=0x7fa0ac00b960 comp rx=0 tx=0).stop 2023-11-09T03:46:28.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.114+0000 7fa0bf007640 1 -- 172.21.15.76:0/2505188784 shutdown_connections 2023-11-09T03:46:28.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.114+0000 7fa0bf007640 1 --2- 172.21.15.76:0/2505188784 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa0a406ae70 0x7fa0a406d330 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.114+0000 7fa0bf007640 1 --2- 172.21.15.76:0/2505188784 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0b8159d30 0x7fa0b8070610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.114+0000 7fa0bf007640 1 --2- 172.21.15.76:0/2505188784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0b806e630 0x7fa0b81e4e20 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.114+0000 7fa0bf007640 1 --2- 172.21.15.76:0/2505188784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0b806dc70 0x7fa0b806ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.114+0000 7fa0bf007640 1 -- 172.21.15.76:0/2505188784 >> 172.21.15.76:0/2505188784 conn(0x7fa0b80b03c0 msgr2=0x7fa0b80cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:28.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.114+0000 7fa0bf007640 1 -- 172.21.15.76:0/2505188784 shutdown_connections 2023-11-09T03:46:28.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.114+0000 7fa0bf007640 1 -- 172.21.15.76:0/2505188784 wait complete. 2023-11-09T03:46:28.125 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:45:28.704715+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:45:29.717614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-09T03:45:33.440577+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-09T03:46:28.125 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:46:28.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.228+0000 7f112332c640 1 Processor -- start 2023-11-09T03:46:28.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.228+0000 7f112332c640 1 -- start start 2023-11-09T03:46:28.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.229+0000 7f112332c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f111c153e60 0x7f111c15dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:28.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.229+0000 7f112332c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f111c0b4880 0x7f111c0b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:28.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.229+0000 7f112332c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f111c0cde60 con 0x7f111c0cd6d0 2023-11-09T03:46:28.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.229+0000 7f112332c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f111c160f90 con 0x7f111c0b4880 2023-11-09T03:46:28.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.229+0000 7f112332c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f111c0b5200 con 0x7f111c153e60 2023-11-09T03:46:28.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.229+0000 7f11218c9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f111c0b4880 0x7f111c0b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:28.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.229+0000 7f11218c9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f111c0b4880 0x7f111c0b4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35956/0 (socket says 172.21.15.76:35956) 2023-11-09T03:46:28.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.229+0000 7f11218c9640 1 -- 172.21.15.76:0/224762969 learned_addr learned my addr 172.21.15.76:0/224762969 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:28.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.230+0000 7f11210c8640 1 --2- 172.21.15.76:0/224762969 >> v2:172.21.15.113:3300/0 conn(0x7f111c153e60 0x7f111c15dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:28.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.230+0000 7f11218c9640 1 -- 172.21.15.76:0/224762969 >> v2:172.21.15.113:3300/0 conn(0x7f111c153e60 msgr2=0x7f111c15dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:28.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.230+0000 7f11218c9640 1 --2- 172.21.15.76:0/224762969 >> v2:172.21.15.113:3300/0 conn(0x7f111c153e60 0x7f111c15dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.230+0000 7f11218c9640 1 -- 172.21.15.76:0/224762969 >> v1:172.21.15.113:6789/0 conn(0x7f111c0cd6d0 legacy=0x7f111c161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:28.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.230+0000 7f11218c9640 1 -- 172.21.15.76:0/224762969 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111c15e400 con 0x7f111c0b4880 2023-11-09T03:46:28.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.230+0000 7f11218c9640 1 --2- 172.21.15.76:0/224762969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f111c0b4880 0x7f111c0b4cc0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f111c08abe0 tx=0x7f111402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8847e3c62ed3da server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:28.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.231+0000 7f1113fff640 1 -- 172.21.15.76:0/224762969 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1114031870 con 0x7f111c0b4880 2023-11-09T03:46:28.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.231+0000 7f1113fff640 1 -- 172.21.15.76:0/224762969 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11140319d0 con 0x7f111c0b4880 2023-11-09T03:46:28.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.231+0000 7f1113fff640 1 -- 172.21.15.76:0/224762969 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1114031d60 con 0x7f111c0b4880 2023-11-09T03:46:28.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.231+0000 7f112332c640 1 -- 172.21.15.76:0/224762969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f111c0b4880 msgr2=0x7f111c0b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:28.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.231+0000 7f112332c640 1 --2- 172.21.15.76:0/224762969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f111c0b4880 0x7f111c0b4cc0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f111c08abe0 tx=0x7f111402b910 comp rx=0 tx=0).stop 2023-11-09T03:46:28.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.231+0000 7f112332c640 1 -- 172.21.15.76:0/224762969 shutdown_connections 2023-11-09T03:46:28.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.231+0000 7f112332c640 1 --2- 172.21.15.76:0/224762969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f111c0b4880 0x7f111c0b4cc0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.231+0000 7f112332c640 1 --2- 172.21.15.76:0/224762969 >> v2:172.21.15.113:3300/0 conn(0x7f111c153e60 0x7f111c15dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.231+0000 7f112332c640 1 -- 172.21.15.76:0/224762969 >> 172.21.15.76:0/224762969 conn(0x7f111c0cf880 msgr2=0x7f111c0cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:28.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.231+0000 7f112332c640 1 -- 172.21.15.76:0/224762969 shutdown_connections 2023-11-09T03:46:28.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.231+0000 7f112332c640 1 -- 172.21.15.76:0/224762969 wait complete. 2023-11-09T03:46:28.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.232+0000 7f112332c640 1 Processor -- start 2023-11-09T03:46:28.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.232+0000 7f112332c640 1 -- start start 2023-11-09T03:46:28.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.233+0000 7f112332c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f111c0b4880 0x7f111c0c7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:28.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.233+0000 7f112332c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f111c0cd6d0 0x7f111c15a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:28.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.233+0000 7f112332c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f111c153e60 0x7f111c14d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:28.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.233+0000 7f112332c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c1e8a20 con 0x7f111c0b4880 2023-11-09T03:46:28.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.233+0000 7f112332c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c1e8d00 con 0x7f111c153e60 2023-11-09T03:46:28.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.233+0000 7f112332c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f111c1e8fe0 con 0x7f111c0cd6d0 2023-11-09T03:46:28.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.233+0000 7f11210c8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f111c0b4880 0x7f111c0c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:28.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.233+0000 7f11210c8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f111c0b4880 0x7f111c0c7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35960/0 (socket says 172.21.15.76:35960) 2023-11-09T03:46:28.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.233+0000 7f11210c8640 1 -- 172.21.15.76:0/161330428 learned_addr learned my addr 172.21.15.76:0/161330428 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:28.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.233+0000 7f11218c9640 1 --2- 172.21.15.76:0/161330428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f111c153e60 0x7f111c14d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:28.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.233+0000 7f11208c7640 1 --2- 172.21.15.76:0/161330428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f111c0cd6d0 0x7f111c15a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:28.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.234+0000 7f11210c8640 1 -- 172.21.15.76:0/161330428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f111c0cd6d0 msgr2=0x7f111c15a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:28.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.234+0000 7f11210c8640 1 --2- 172.21.15.76:0/161330428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f111c0cd6d0 0x7f111c15a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.234+0000 7f11210c8640 1 -- 172.21.15.76:0/161330428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f111c153e60 msgr2=0x7f111c14d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:28.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.234+0000 7f11210c8640 1 --2- 172.21.15.76:0/161330428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f111c153e60 0x7f111c14d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.234+0000 7f11210c8640 1 -- 172.21.15.76:0/161330428 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11140094d0 con 0x7f111c0b4880 2023-11-09T03:46:28.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.234+0000 7f11218c9640 1 --2- 172.21.15.76:0/161330428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f111c153e60 0x7f111c14d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:28.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.234+0000 7f11210c8640 1 --2- 172.21.15.76:0/161330428 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f111c0b4880 0x7f111c0c7540 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f111800ccd0 tx=0x7f111800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:28.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.234+0000 7f1111ffb640 1 -- 172.21.15.76:0/161330428 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1118003990 con 0x7f111c0b4880 2023-11-09T03:46:28.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.234+0000 7f1111ffb640 1 -- 172.21.15.76:0/161330428 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1118003af0 con 0x7f111c0b4880 2023-11-09T03:46:28.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.234+0000 7f112332c640 1 -- 172.21.15.76:0/161330428 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f111c1511a0 con 0x7f111c0b4880 2023-11-09T03:46:28.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.234+0000 7f1111ffb640 1 -- 172.21.15.76:0/161330428 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1118004020 con 0x7f111c0b4880 2023-11-09T03:46:28.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.234+0000 7f112332c640 1 -- 172.21.15.76:0/161330428 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f111c15ad70 con 0x7f111c0b4880 2023-11-09T03:46:28.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.235+0000 7f112332c640 1 -- 172.21.15.76:0/161330428 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f111c1516c0 con 0x7f111c0b4880 2023-11-09T03:46:28.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.235+0000 7f1111ffb640 1 -- 172.21.15.76:0/161330428 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1118004180 con 0x7f111c0b4880 2023-11-09T03:46:28.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.236+0000 7f1111ffb640 1 --2- 172.21.15.76:0/161330428 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f110006aea0 0x7f110006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:28.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.236+0000 7f1111ffb640 1 -- 172.21.15.76:0/161330428 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f1118089e30 con 0x7f111c0b4880 2023-11-09T03:46:28.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.236+0000 7f11208c7640 1 --2- 172.21.15.76:0/161330428 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f110006aea0 0x7f110006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:28.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.236+0000 7f11208c7640 1 --2- 172.21.15.76:0/161330428 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f110006aea0 0x7f110006d360 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f111c161640 tx=0x7f110c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:28.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.240+0000 7f1111ffb640 1 -- 172.21.15.76:0/161330428 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f111c1516c0 con 0x7f111c0b4880 2023-11-09T03:46:28.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.405+0000 7f112332c640 1 -- 172.21.15.76:0/161330428 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f111c1516c0 con 0x7f111c0b4880 2023-11-09T03:46:28.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.405+0000 7f1111ffb640 1 -- 172.21.15.76:0/161330428 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f111c1516c0 con 0x7f111c0b4880 2023-11-09T03:46:28.405 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:46:28.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.407+0000 7f112332c640 1 -- 172.21.15.76:0/161330428 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f110006aea0 msgr2=0x7f110006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:28.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.407+0000 7f112332c640 1 --2- 172.21.15.76:0/161330428 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f110006aea0 0x7f110006d360 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f111c161640 tx=0x7f110c009040 comp rx=0 tx=0).stop 2023-11-09T03:46:28.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.407+0000 7f112332c640 1 -- 172.21.15.76:0/161330428 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f111c0b4880 msgr2=0x7f111c0c7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:28.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.407+0000 7f112332c640 1 --2- 172.21.15.76:0/161330428 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f111c0b4880 0x7f111c0c7540 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f111800ccd0 tx=0x7f111800a5c0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.407+0000 7f112332c640 1 -- 172.21.15.76:0/161330428 shutdown_connections 2023-11-09T03:46:28.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.407+0000 7f112332c640 1 --2- 172.21.15.76:0/161330428 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f110006aea0 0x7f110006d360 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.407+0000 7f112332c640 1 --2- 172.21.15.76:0/161330428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f111c153e60 0x7f111c14d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.407+0000 7f112332c640 1 --2- 172.21.15.76:0/161330428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f111c0cd6d0 0x7f111c15a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.407+0000 7f112332c640 1 --2- 172.21.15.76:0/161330428 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f111c0b4880 0x7f111c0c7540 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.407+0000 7f112332c640 1 -- 172.21.15.76:0/161330428 >> 172.21.15.76:0/161330428 conn(0x7f111c0cf880 msgr2=0x7f111c0d2570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:28.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.407+0000 7f112332c640 1 -- 172.21.15.76:0/161330428 shutdown_connections 2023-11-09T03:46:28.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.407+0000 7f112332c640 1 -- 172.21.15.76:0/161330428 wait complete. 2023-11-09T03:46:28.417 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:46:28.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.520+0000 7f850b256640 1 Processor -- start 2023-11-09T03:46:28.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.520+0000 7f850b256640 1 -- start start 2023-11-09T03:46:28.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.520+0000 7f850b256640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8504159c00 0x7f85040cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:28.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.520+0000 7f850b256640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f85040ae1d0 0x7f85040c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:28.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.520+0000 7f850b256640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f850415aff0 con 0x7f85040c48e0 2023-11-09T03:46:28.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.520+0000 7f850b256640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85040c52d0 con 0x7f8504159c00 2023-11-09T03:46:28.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.520+0000 7f850b256640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f850415a3e0 con 0x7f85040ae1d0 2023-11-09T03:46:28.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.521+0000 7f8508ff2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8504159c00 0x7f85040cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:28.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.521+0000 7f8508ff2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8504159c00 0x7f85040cc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35968/0 (socket says 172.21.15.76:35968) 2023-11-09T03:46:28.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.521+0000 7f8508ff2640 1 -- 172.21.15.76:0/2327100881 learned_addr learned my addr 172.21.15.76:0/2327100881 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:28.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.521+0000 7f85097f3640 1 --2- 172.21.15.76:0/2327100881 >> v2:172.21.15.113:3300/0 conn(0x7f85040ae1d0 0x7f85040c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:28.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.521+0000 7f8508ff2640 1 -- 172.21.15.76:0/2327100881 >> v2:172.21.15.113:3300/0 conn(0x7f85040ae1d0 msgr2=0x7f85040c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:28.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.521+0000 7f8508ff2640 1 --2- 172.21.15.76:0/2327100881 >> v2:172.21.15.113:3300/0 conn(0x7f85040ae1d0 0x7f85040c4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.521+0000 7f8508ff2640 1 -- 172.21.15.76:0/2327100881 >> v1:172.21.15.113:6789/0 conn(0x7f85040c48e0 legacy=0x7f85040adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:28.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.521+0000 7f8508ff2640 1 -- 172.21.15.76:0/2327100881 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850415a6c0 con 0x7f8504159c00 2023-11-09T03:46:28.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.522+0000 7f8508ff2640 1 --2- 172.21.15.76:0/2327100881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8504159c00 0x7f85040cc890 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f85040ce800 tx=0x7f84f802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934bc9255bd8b7f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:28.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.522+0000 7f85037fe640 1 -- 172.21.15.76:0/2327100881 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8031870 con 0x7f8504159c00 2023-11-09T03:46:28.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.522+0000 7f85037fe640 1 -- 172.21.15.76:0/2327100881 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f80319d0 con 0x7f8504159c00 2023-11-09T03:46:28.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.522+0000 7f85037fe640 1 -- 172.21.15.76:0/2327100881 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8031d60 con 0x7f8504159c00 2023-11-09T03:46:28.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.522+0000 7f850b256640 1 -- 172.21.15.76:0/2327100881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8504159c00 msgr2=0x7f85040cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:28.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.522+0000 7f850b256640 1 --2- 172.21.15.76:0/2327100881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8504159c00 0x7f85040cc890 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f85040ce800 tx=0x7f84f802b910 comp rx=0 tx=0).stop 2023-11-09T03:46:28.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.522+0000 7f850b256640 1 -- 172.21.15.76:0/2327100881 shutdown_connections 2023-11-09T03:46:28.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.522+0000 7f850b256640 1 --2- 172.21.15.76:0/2327100881 >> v2:172.21.15.113:3300/0 conn(0x7f85040ae1d0 0x7f85040c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.522+0000 7f850b256640 1 --2- 172.21.15.76:0/2327100881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8504159c00 0x7f85040cc890 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.522+0000 7f850b256640 1 -- 172.21.15.76:0/2327100881 >> 172.21.15.76:0/2327100881 conn(0x7f85040ce950 msgr2=0x7f85040ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:28.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.522+0000 7f850b256640 1 -- 172.21.15.76:0/2327100881 shutdown_connections 2023-11-09T03:46:28.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.522+0000 7f850b256640 1 -- 172.21.15.76:0/2327100881 wait complete. 2023-11-09T03:46:28.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.524+0000 7f850b256640 1 Processor -- start 2023-11-09T03:46:28.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.524+0000 7f850b256640 1 -- start start 2023-11-09T03:46:28.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.524+0000 7f850b256640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85040ae1d0 0x7f850414d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:28.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.524+0000 7f850b256640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85040c48e0 0x7f850414e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:28.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.524+0000 7f850b256640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8504159c00 0x7f8504145d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:28.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.524+0000 7f850b256640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850414dbc0 con 0x7f85040c48e0 2023-11-09T03:46:28.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.524+0000 7f850b256640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850414dea0 con 0x7f85040ae1d0 2023-11-09T03:46:28.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.524+0000 7f850b256640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850414e180 con 0x7f8504159c00 2023-11-09T03:46:28.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.525+0000 7f8503fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85040c48e0 0x7f850414e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:28.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.525+0000 7f8503fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85040c48e0 0x7f850414e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35980/0 (socket says 172.21.15.76:35980) 2023-11-09T03:46:28.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.525+0000 7f8503fff640 1 -- 172.21.15.76:0/477005046 learned_addr learned my addr 172.21.15.76:0/477005046 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:28.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.525+0000 7f85097f3640 1 --2- 172.21.15.76:0/477005046 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8504159c00 0x7f8504145d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:28.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.525+0000 7f8508ff2640 1 --2- 172.21.15.76:0/477005046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85040ae1d0 0x7f850414d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:28.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.525+0000 7f8503fff640 1 -- 172.21.15.76:0/477005046 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8504159c00 msgr2=0x7f8504145d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:28.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.525+0000 7f8503fff640 1 --2- 172.21.15.76:0/477005046 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8504159c00 0x7f8504145d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.525+0000 7f8503fff640 1 -- 172.21.15.76:0/477005046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85040ae1d0 msgr2=0x7f850414d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:28.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.525+0000 7f8503fff640 1 --2- 172.21.15.76:0/477005046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85040ae1d0 0x7f850414d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.525+0000 7f8503fff640 1 -- 172.21.15.76:0/477005046 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f80094d0 con 0x7f85040c48e0 2023-11-09T03:46:28.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.525+0000 7f8503fff640 1 --2- 172.21.15.76:0/477005046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85040c48e0 0x7f850414e9a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f84f000c450 tx=0x7f84f000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:28.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.525+0000 7f85017fa640 1 -- 172.21.15.76:0/477005046 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f00052d0 con 0x7f85040c48e0 2023-11-09T03:46:28.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.525+0000 7f85017fa640 1 -- 172.21.15.76:0/477005046 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f0005430 con 0x7f85040c48e0 2023-11-09T03:46:28.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.525+0000 7f850b256640 1 -- 172.21.15.76:0/477005046 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f850414e460 con 0x7f85040c48e0 2023-11-09T03:46:28.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.525+0000 7f85017fa640 1 -- 172.21.15.76:0/477005046 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f0012870 con 0x7f85040c48e0 2023-11-09T03:46:28.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.525+0000 7f850b256640 1 -- 172.21.15.76:0/477005046 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f850414d820 con 0x7f85040c48e0 2023-11-09T03:46:28.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.526+0000 7f850b256640 1 -- 172.21.15.76:0/477005046 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85040b2d30 con 0x7f85040c48e0 2023-11-09T03:46:28.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.527+0000 7f85017fa640 1 -- 172.21.15.76:0/477005046 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f84f00129d0 con 0x7f85040c48e0 2023-11-09T03:46:28.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.527+0000 7f85017fa640 1 --2- 172.21.15.76:0/477005046 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f84e806ae70 0x7f84e806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:28.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.527+0000 7f85017fa640 1 -- 172.21.15.76:0/477005046 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f84f00893e0 con 0x7f85040c48e0 2023-11-09T03:46:28.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.527+0000 7f8508ff2640 1 --2- 172.21.15.76:0/477005046 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f84e806ae70 0x7f84e806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:28.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.528+0000 7f8508ff2640 1 --2- 172.21.15.76:0/477005046 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f84e806ae70 0x7f84e806d330 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f84f8036000 tx=0x7f84f803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:28.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.531+0000 7f85017fa640 1 -- 172.21.15.76:0/477005046 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f85040b2d30 con 0x7f85040c48e0 2023-11-09T03:46:28.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.698+0000 7f850b256640 1 -- 172.21.15.76:0/477005046 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f85040b2d30 con 0x7f85040c48e0 2023-11-09T03:46:28.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.698+0000 7f85017fa640 1 -- 172.21.15.76:0/477005046 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f85040b2d30 con 0x7f85040c48e0 2023-11-09T03:46:28.698 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:46:28.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.700+0000 7f850b256640 1 -- 172.21.15.76:0/477005046 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f84e806ae70 msgr2=0x7f84e806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:28.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.700+0000 7f850b256640 1 --2- 172.21.15.76:0/477005046 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f84e806ae70 0x7f84e806d330 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f84f8036000 tx=0x7f84f803c040 comp rx=0 tx=0).stop 2023-11-09T03:46:28.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.700+0000 7f850b256640 1 -- 172.21.15.76:0/477005046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85040c48e0 msgr2=0x7f850414e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:28.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.700+0000 7f850b256640 1 --2- 172.21.15.76:0/477005046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85040c48e0 0x7f850414e9a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f84f000c450 tx=0x7f84f000c910 comp rx=0 tx=0).stop 2023-11-09T03:46:28.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.700+0000 7f850b256640 1 -- 172.21.15.76:0/477005046 shutdown_connections 2023-11-09T03:46:28.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.700+0000 7f850b256640 1 --2- 172.21.15.76:0/477005046 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f84e806ae70 0x7f84e806d330 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.700+0000 7f850b256640 1 --2- 172.21.15.76:0/477005046 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8504159c00 0x7f8504145d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.700+0000 7f850b256640 1 --2- 172.21.15.76:0/477005046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85040c48e0 0x7f850414e9a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.700+0000 7f850b256640 1 --2- 172.21.15.76:0/477005046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85040ae1d0 0x7f850414d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.700+0000 7f850b256640 1 -- 172.21.15.76:0/477005046 >> 172.21.15.76:0/477005046 conn(0x7f85040ce950 msgr2=0x7f85041589e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:28.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.700+0000 7f850b256640 1 -- 172.21.15.76:0/477005046 shutdown_connections 2023-11-09T03:46:28.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.700+0000 7f850b256640 1 -- 172.21.15.76:0/477005046 wait complete. 2023-11-09T03:46:28.710 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:46:28.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.814+0000 7ff4a8d55640 1 Processor -- start 2023-11-09T03:46:28.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.814+0000 7ff4a8d55640 1 -- start start 2023-11-09T03:46:28.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.815+0000 7ff4a8d55640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff4a4153e60 0x7ff4a415dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:28.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.815+0000 7ff4a8d55640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4a40cd6d0 0x7ff4a4161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:28.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.815+0000 7ff4a8d55640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a40cde60 con 0x7ff4a40b4880 2023-11-09T03:46:28.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.815+0000 7ff4a8d55640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a4160f90 con 0x7ff4a4153e60 2023-11-09T03:46:28.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.815+0000 7ff4a8d55640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a415e400 con 0x7ff4a40cd6d0 2023-11-09T03:46:28.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.815+0000 7ff4a259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff4a4153e60 0x7ff4a415dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:28.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.815+0000 7ff4a259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff4a4153e60 0x7ff4a415dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35996/0 (socket says 172.21.15.76:35996) 2023-11-09T03:46:28.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.815+0000 7ff4a259c640 1 -- 172.21.15.76:0/1274084760 learned_addr learned my addr 172.21.15.76:0/1274084760 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:28.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.816+0000 7ff4a259c640 1 -- 172.21.15.76:0/1274084760 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4a40cd6d0 msgr2=0x7ff4a4161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:28.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.816+0000 7ff4a259c640 1 --2- 172.21.15.76:0/1274084760 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4a40cd6d0 0x7ff4a4161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.816+0000 7ff4a259c640 1 -- 172.21.15.76:0/1274084760 >> v1:172.21.15.113:6789/0 conn(0x7ff4a40b4880 legacy=0x7ff4a40b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:28.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.816+0000 7ff4a259c640 1 -- 172.21.15.76:0/1274084760 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a4161960 con 0x7ff4a4153e60 2023-11-09T03:46:28.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.816+0000 7ff4a259c640 1 --2- 172.21.15.76:0/1274084760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4a4153e60 0x7ff4a415dec0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff494009e50 tx=0x7ff49402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947649bb71ef29e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:28.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.817+0000 7ff4a159a640 1 -- 172.21.15.76:0/1274084760 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49402ca70 con 0x7ff4a4153e60 2023-11-09T03:46:28.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.817+0000 7ff4a159a640 1 -- 172.21.15.76:0/1274084760 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49402cbd0 con 0x7ff4a4153e60 2023-11-09T03:46:28.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.817+0000 7ff4a159a640 1 -- 172.21.15.76:0/1274084760 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4940343f0 con 0x7ff4a4153e60 2023-11-09T03:46:28.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.817+0000 7ff4a8d55640 1 -- 172.21.15.76:0/1274084760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4a4153e60 msgr2=0x7ff4a415dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:28.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.817+0000 7ff4a8d55640 1 --2- 172.21.15.76:0/1274084760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4a4153e60 0x7ff4a415dec0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff494009e50 tx=0x7ff49402bed0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.817+0000 7ff4a8d55640 1 -- 172.21.15.76:0/1274084760 shutdown_connections 2023-11-09T03:46:28.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.817+0000 7ff4a8d55640 1 --2- 172.21.15.76:0/1274084760 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4a40cd6d0 0x7ff4a4161420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.817+0000 7ff4a8d55640 1 --2- 172.21.15.76:0/1274084760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4a4153e60 0x7ff4a415dec0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.817+0000 7ff4a8d55640 1 -- 172.21.15.76:0/1274084760 >> 172.21.15.76:0/1274084760 conn(0x7ff4a40cf880 msgr2=0x7ff4a40cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:28.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.817+0000 7ff4a8d55640 1 -- 172.21.15.76:0/1274084760 shutdown_connections 2023-11-09T03:46:28.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.817+0000 7ff4a8d55640 1 -- 172.21.15.76:0/1274084760 wait complete. 2023-11-09T03:46:28.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.819+0000 7ff4a8d55640 1 Processor -- start 2023-11-09T03:46:28.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.819+0000 7ff4a8d55640 1 -- start start 2023-11-09T03:46:28.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.819+0000 7ff4a8d55640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4a40b4880 0x7ff4a40cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:28.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.819+0000 7ff4a8d55640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4a40cd6d0 0x7ff4a4154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:28.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.819+0000 7ff4a8d55640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4a4153e60 0x7ff4a414f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:28.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.819+0000 7ff4a8d55640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a4154780 con 0x7ff4a40cd6d0 2023-11-09T03:46:28.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.819+0000 7ff4a8d55640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a414fec0 con 0x7ff4a4153e60 2023-11-09T03:46:28.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.819+0000 7ff4a8d55640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a4154990 con 0x7ff4a40b4880 2023-11-09T03:46:28.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.819+0000 7ff4a1d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4a40cd6d0 0x7ff4a4154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:28.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.819+0000 7ff4a1d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4a40cd6d0 0x7ff4a4154240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36010/0 (socket says 172.21.15.76:36010) 2023-11-09T03:46:28.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.819+0000 7ff4a1d9b640 1 -- 172.21.15.76:0/2302899060 learned_addr learned my addr 172.21.15.76:0/2302899060 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:28.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.820+0000 7ff4a259c640 1 --2- 172.21.15.76:0/2302899060 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4a40b4880 0x7ff4a40cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:28.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.820+0000 7ff4a2d9d640 1 --2- 172.21.15.76:0/2302899060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4a4153e60 0x7ff4a414f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:28.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.820+0000 7ff4a1d9b640 1 -- 172.21.15.76:0/2302899060 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4a40b4880 msgr2=0x7ff4a40cc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:28.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.820+0000 7ff4a1d9b640 1 --2- 172.21.15.76:0/2302899060 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4a40b4880 0x7ff4a40cc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.820+0000 7ff4a1d9b640 1 -- 172.21.15.76:0/2302899060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4a4153e60 msgr2=0x7ff4a414f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:28.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.820+0000 7ff4a1d9b640 1 --2- 172.21.15.76:0/2302899060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4a4153e60 0x7ff4a414f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.820+0000 7ff4a1d9b640 1 -- 172.21.15.76:0/2302899060 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff494009b30 con 0x7ff4a40cd6d0 2023-11-09T03:46:28.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.820+0000 7ff4a1d9b640 1 --2- 172.21.15.76:0/2302899060 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4a40cd6d0 0x7ff4a4154240 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff49800b4a0 tx=0x7ff49800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:28.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.820+0000 7ff492ffd640 1 -- 172.21.15.76:0/2302899060 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff498010610 con 0x7ff4a40cd6d0 2023-11-09T03:46:28.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.820+0000 7ff492ffd640 1 -- 172.21.15.76:0/2302899060 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff498010770 con 0x7ff4a40cd6d0 2023-11-09T03:46:28.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.820+0000 7ff492ffd640 1 -- 172.21.15.76:0/2302899060 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff498010b20 con 0x7ff4a40cd6d0 2023-11-09T03:46:28.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.820+0000 7ff4a8d55640 1 -- 172.21.15.76:0/2302899060 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a4154c70 con 0x7ff4a40cd6d0 2023-11-09T03:46:28.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.820+0000 7ff4a8d55640 1 -- 172.21.15.76:0/2302899060 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a4155230 con 0x7ff4a40cd6d0 2023-11-09T03:46:28.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.821+0000 7ff4a8d55640 1 -- 172.21.15.76:0/2302899060 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a4074290 con 0x7ff4a40cd6d0 2023-11-09T03:46:28.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.821+0000 7ff492ffd640 1 -- 172.21.15.76:0/2302899060 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff498010c80 con 0x7ff4a40cd6d0 2023-11-09T03:46:28.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.822+0000 7ff492ffd640 1 --2- 172.21.15.76:0/2302899060 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff48406ae70 0x7ff48406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:28.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.822+0000 7ff492ffd640 1 -- 172.21.15.76:0/2302899060 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7ff498089570 con 0x7ff4a40cd6d0 2023-11-09T03:46:28.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.822+0000 7ff4a259c640 1 --2- 172.21.15.76:0/2302899060 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff48406ae70 0x7ff48406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:28.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.822+0000 7ff4a259c640 1 --2- 172.21.15.76:0/2302899060 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff48406ae70 0x7ff48406d330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff494035000 tx=0x7ff494030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:28.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.826+0000 7ff492ffd640 1 -- 172.21.15.76:0/2302899060 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff4980542f0 con 0x7ff4a40cd6d0 2023-11-09T03:46:28.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.994+0000 7ff4a8d55640 1 -- 172.21.15.76:0/2302899060 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff4a40d0b80 con 0x7ff4a40cd6d0 2023-11-09T03:46:28.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.995+0000 7ff492ffd640 1 -- 172.21.15.76:0/2302899060 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff498057910 con 0x7ff4a40cd6d0 2023-11-09T03:46:28.994 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:46:28.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.997+0000 7ff4a8d55640 1 -- 172.21.15.76:0/2302899060 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff48406ae70 msgr2=0x7ff48406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:28.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.997+0000 7ff4a8d55640 1 --2- 172.21.15.76:0/2302899060 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff48406ae70 0x7ff48406d330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff494035000 tx=0x7ff494030000 comp rx=0 tx=0).stop 2023-11-09T03:46:28.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.997+0000 7ff4a8d55640 1 -- 172.21.15.76:0/2302899060 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4a40cd6d0 msgr2=0x7ff4a4154240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:28.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.997+0000 7ff4a8d55640 1 --2- 172.21.15.76:0/2302899060 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4a40cd6d0 0x7ff4a4154240 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff49800b4a0 tx=0x7ff49800b960 comp rx=0 tx=0).stop 2023-11-09T03:46:28.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.997+0000 7ff4a8d55640 1 -- 172.21.15.76:0/2302899060 shutdown_connections 2023-11-09T03:46:28.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.997+0000 7ff4a8d55640 1 --2- 172.21.15.76:0/2302899060 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff48406ae70 0x7ff48406d330 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.997+0000 7ff4a8d55640 1 --2- 172.21.15.76:0/2302899060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4a4153e60 0x7ff4a414f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.997+0000 7ff4a8d55640 1 --2- 172.21.15.76:0/2302899060 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4a40cd6d0 0x7ff4a4154240 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.997+0000 7ff4a8d55640 1 --2- 172.21.15.76:0/2302899060 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4a40b4880 0x7ff4a40cc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:28.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.997+0000 7ff4a8d55640 1 -- 172.21.15.76:0/2302899060 >> 172.21.15.76:0/2302899060 conn(0x7ff4a40cf880 msgr2=0x7ff4a415fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:28.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.997+0000 7ff4a8d55640 1 -- 172.21.15.76:0/2302899060 shutdown_connections 2023-11-09T03:46:28.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:28.997+0000 7ff4a8d55640 1 -- 172.21.15.76:0/2302899060 wait complete. 2023-11-09T03:46:29.007 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-09T03:46:29.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.110+0000 7f1cb85fc640 1 Processor -- start 2023-11-09T03:46:29.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.110+0000 7f1cb85fc640 1 -- start start 2023-11-09T03:46:29.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.111+0000 7f1cb85fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cb0159c00 0x7f1cb00cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:29.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.111+0000 7f1cb85fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cb00c48e0 0x7f1cb00c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:29.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.111+0000 7f1cb85fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cb00adac0 0x7f1cb00adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:29.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.111+0000 7f1cb85fc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cb00c52d0 con 0x7f1cb00c48e0 2023-11-09T03:46:29.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.111+0000 7f1cb85fc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cb015aff0 con 0x7f1cb0159c00 2023-11-09T03:46:29.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.111+0000 7f1cb85fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb015a3e0 con 0x7f1cb00adac0 2023-11-09T03:46:29.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.111+0000 7f1cb5b97640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cb00c48e0 0x7f1cb00c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:29.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.111+0000 7f1cb5b97640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cb00c48e0 0x7f1cb00c4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36014/0 (socket says 172.21.15.76:36014) 2023-11-09T03:46:29.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.111+0000 7f1cb5b97640 1 -- 172.21.15.76:0/1624712894 learned_addr learned my addr 172.21.15.76:0/1624712894 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:29.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.112+0000 7f1cb6398640 1 --2- 172.21.15.76:0/1624712894 >> v2:172.21.15.113:3300/0 conn(0x7f1cb0159c00 0x7f1cb00cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:29.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.112+0000 7f1cb6b99640 1 --2- 172.21.15.76:0/1624712894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cb00adac0 0x7f1cb00adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:29.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.112+0000 7f1cb5b97640 1 -- 172.21.15.76:0/1624712894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cb00adac0 msgr2=0x7f1cb00adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:29.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.112+0000 7f1cb5b97640 1 --2- 172.21.15.76:0/1624712894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cb00adac0 0x7f1cb00adea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:29.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.112+0000 7f1cb5b97640 1 -- 172.21.15.76:0/1624712894 >> v2:172.21.15.113:3300/0 conn(0x7f1cb0159c00 msgr2=0x7f1cb00cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:29.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.112+0000 7f1cb5b97640 1 --2- 172.21.15.76:0/1624712894 >> v2:172.21.15.113:3300/0 conn(0x7f1cb0159c00 0x7f1cb00cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:29.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.112+0000 7f1cb5b97640 1 -- 172.21.15.76:0/1624712894 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb015a6c0 con 0x7f1cb00c48e0 2023-11-09T03:46:29.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.112+0000 7f1cb6398640 1 --2- 172.21.15.76:0/1624712894 >> v2:172.21.15.113:3300/0 conn(0x7f1cb0159c00 0x7f1cb00cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:29.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.112+0000 7f1cb5b97640 1 --2- 172.21.15.76:0/1624712894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cb00c48e0 0x7f1cb00c4d90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00cc4d0 tx=0x7f1ca002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88b4acf1f8f9f90 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:29.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.113+0000 7f1cb5396640 1 -- 172.21.15.76:0/1624712894 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0031870 con 0x7f1cb00c48e0 2023-11-09T03:46:29.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.113+0000 7f1cb5396640 1 -- 172.21.15.76:0/1624712894 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca00319d0 con 0x7f1cb00c48e0 2023-11-09T03:46:29.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.113+0000 7f1cb5396640 1 -- 172.21.15.76:0/1624712894 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0031d60 con 0x7f1cb00c48e0 2023-11-09T03:46:29.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.113+0000 7f1cb85fc640 1 -- 172.21.15.76:0/1624712894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cb00c48e0 msgr2=0x7f1cb00c4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:29.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.113+0000 7f1cb85fc640 1 --2- 172.21.15.76:0/1624712894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cb00c48e0 0x7f1cb00c4d90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00cc4d0 tx=0x7f1ca002b910 comp rx=0 tx=0).stop 2023-11-09T03:46:29.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.113+0000 7f1cb85fc640 1 -- 172.21.15.76:0/1624712894 shutdown_connections 2023-11-09T03:46:29.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.113+0000 7f1cb85fc640 1 --2- 172.21.15.76:0/1624712894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cb00adac0 0x7f1cb00adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:29.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.113+0000 7f1cb85fc640 1 --2- 172.21.15.76:0/1624712894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cb00c48e0 0x7f1cb00c4d90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:29.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.113+0000 7f1cb85fc640 1 --2- 172.21.15.76:0/1624712894 >> v2:172.21.15.113:3300/0 conn(0x7f1cb0159c00 0x7f1cb00cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:29.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.113+0000 7f1cb85fc640 1 -- 172.21.15.76:0/1624712894 >> 172.21.15.76:0/1624712894 conn(0x7f1cb00ce950 msgr2=0x7f1cb00ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:29.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.113+0000 7f1cb85fc640 1 -- 172.21.15.76:0/1624712894 shutdown_connections 2023-11-09T03:46:29.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.113+0000 7f1cb85fc640 1 -- 172.21.15.76:0/1624712894 wait complete. 2023-11-09T03:46:29.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.114+0000 7f1cb85fc640 1 Processor -- start 2023-11-09T03:46:29.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.114+0000 7f1cb85fc640 1 -- start start 2023-11-09T03:46:29.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.115+0000 7f1cb85fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cb00adac0 0x7f1cb00b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:29.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.115+0000 7f1cb85fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cb00c48e0 0x7f1cb00aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:29.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.115+0000 7f1cb85fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cb0159c00 0x7f1cb0157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:29.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.115+0000 7f1cb85fc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb01f1510 con 0x7f1cb00c48e0 2023-11-09T03:46:29.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.115+0000 7f1cb85fc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb00b4470 con 0x7f1cb00adac0 2023-11-09T03:46:29.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.115+0000 7f1cb85fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb01f0990 con 0x7f1cb0159c00 2023-11-09T03:46:29.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.115+0000 7f1cb5b97640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cb00c48e0 0x7f1cb00aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:29.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.115+0000 7f1cb5b97640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cb00c48e0 0x7f1cb00aecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36026/0 (socket says 172.21.15.76:36026) 2023-11-09T03:46:29.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.115+0000 7f1cb5b97640 1 -- 172.21.15.76:0/406393349 learned_addr learned my addr 172.21.15.76:0/406393349 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:29.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.115+0000 7f1cb6398640 1 --2- 172.21.15.76:0/406393349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cb00adac0 0x7f1cb00b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:29.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.115+0000 7f1cb6b99640 1 --2- 172.21.15.76:0/406393349 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cb0159c00 0x7f1cb0157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:29.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.115+0000 7f1cb5b97640 1 -- 172.21.15.76:0/406393349 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cb0159c00 msgr2=0x7f1cb0157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:29.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.115+0000 7f1cb5b97640 1 --2- 172.21.15.76:0/406393349 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cb0159c00 0x7f1cb0157c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:29.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.115+0000 7f1cb5b97640 1 -- 172.21.15.76:0/406393349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cb00adac0 msgr2=0x7f1cb00b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:29.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.115+0000 7f1cb5b97640 1 --2- 172.21.15.76:0/406393349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cb00adac0 0x7f1cb00b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:29.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.115+0000 7f1cb5b97640 1 -- 172.21.15.76:0/406393349 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca00094d0 con 0x7f1cb00c48e0 2023-11-09T03:46:29.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.115+0000 7f1cb6398640 1 --2- 172.21.15.76:0/406393349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cb00adac0 0x7f1cb00b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:29.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.116+0000 7f1cb5b97640 1 --2- 172.21.15.76:0/406393349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cb00c48e0 0x7f1cb00aecf0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1ca002b950 tx=0x7f1ca00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:29.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.116+0000 7f1ca6ffd640 1 -- 172.21.15.76:0/406393349 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0031860 con 0x7f1cb00c48e0 2023-11-09T03:46:29.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.116+0000 7f1ca6ffd640 1 -- 172.21.15.76:0/406393349 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca00443f0 con 0x7f1cb00c48e0 2023-11-09T03:46:29.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.116+0000 7f1ca6ffd640 1 -- 172.21.15.76:0/406393349 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca00447a0 con 0x7f1cb00c48e0 2023-11-09T03:46:29.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.116+0000 7f1cb85fc640 1 -- 172.21.15.76:0/406393349 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb01f0c70 con 0x7f1cb00c48e0 2023-11-09T03:46:29.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.116+0000 7f1cb85fc640 1 -- 172.21.15.76:0/406393349 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb01f1230 con 0x7f1cb00c48e0 2023-11-09T03:46:29.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.117+0000 7f1cb85fc640 1 -- 172.21.15.76:0/406393349 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb0074290 con 0x7f1cb00c48e0 2023-11-09T03:46:29.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.117+0000 7f1ca6ffd640 1 -- 172.21.15.76:0/406393349 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1ca004d450 con 0x7f1cb00c48e0 2023-11-09T03:46:29.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.118+0000 7f1ca6ffd640 1 --2- 172.21.15.76:0/406393349 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1c9406ae70 0x7f1c9406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:29.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.118+0000 7f1ca6ffd640 1 -- 172.21.15.76:0/406393349 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f1ca002d040 con 0x7f1cb00c48e0 2023-11-09T03:46:29.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.118+0000 7f1cb6398640 1 --2- 172.21.15.76:0/406393349 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1c9406ae70 0x7f1c9406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:29.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.118+0000 7f1cb6398640 1 --2- 172.21.15.76:0/406393349 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1c9406ae70 0x7f1c9406d330 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1cac007f40 tx=0x7f1cac007490 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:29.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.122+0000 7f1ca6ffd640 1 -- 172.21.15.76:0/406393349 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1ca0076eb0 con 0x7f1cb00c48e0 2023-11-09T03:46:29.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.287+0000 7f1cb85fc640 1 -- 172.21.15.76:0/406393349 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1cb00b12a0 con 0x7f1cb00c48e0 2023-11-09T03:46:29.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.288+0000 7f1ca6ffd640 1 -- 172.21.15.76:0/406393349 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1ca007a4d0 con 0x7f1cb00c48e0 2023-11-09T03:46:29.287 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:46:29.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.290+0000 7f1cb85fc640 1 -- 172.21.15.76:0/406393349 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1c9406ae70 msgr2=0x7f1c9406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:29.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.290+0000 7f1cb85fc640 1 --2- 172.21.15.76:0/406393349 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1c9406ae70 0x7f1c9406d330 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1cac007f40 tx=0x7f1cac007490 comp rx=0 tx=0).stop 2023-11-09T03:46:29.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.290+0000 7f1cb85fc640 1 -- 172.21.15.76:0/406393349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cb00c48e0 msgr2=0x7f1cb00aecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:29.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.290+0000 7f1cb85fc640 1 --2- 172.21.15.76:0/406393349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cb00c48e0 0x7f1cb00aecf0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1ca002b950 tx=0x7f1ca00046c0 comp rx=0 tx=0).stop 2023-11-09T03:46:29.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.290+0000 7f1cb85fc640 1 -- 172.21.15.76:0/406393349 shutdown_connections 2023-11-09T03:46:29.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.290+0000 7f1cb85fc640 1 --2- 172.21.15.76:0/406393349 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1c9406ae70 0x7f1c9406d330 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:29.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.290+0000 7f1cb85fc640 1 --2- 172.21.15.76:0/406393349 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cb0159c00 0x7f1cb0157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:29.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.290+0000 7f1cb85fc640 1 --2- 172.21.15.76:0/406393349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cb00c48e0 0x7f1cb00aecf0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:29.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.290+0000 7f1cb85fc640 1 --2- 172.21.15.76:0/406393349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cb00adac0 0x7f1cb00b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:29.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.290+0000 7f1cb85fc640 1 -- 172.21.15.76:0/406393349 >> 172.21.15.76:0/406393349 conn(0x7f1cb00ce950 msgr2=0x7f1cb00c7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:29.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.290+0000 7f1cb85fc640 1 -- 172.21.15.76:0/406393349 shutdown_connections 2023-11-09T03:46:29.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.290+0000 7f1cb85fc640 1 -- 172.21.15.76:0/406393349 wait complete. 2023-11-09T03:46:29.300 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-09T03:46:29.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.402+0000 7f0490125640 1 Processor -- start 2023-11-09T03:46:29.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.402+0000 7f0490125640 1 -- start start 2023-11-09T03:46:29.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.403+0000 7f0490125640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f048806cbd0 0x7f048806e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:29.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.403+0000 7f0490125640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0488067090 0x7f04880b2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:29.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.403+0000 7f0490125640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04880b24f0 con 0x7f048815b140 2023-11-09T03:46:29.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.403+0000 7f0490125640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04880ce220 con 0x7f0488067090 2023-11-09T03:46:29.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.403+0000 7f0490125640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04880cdc60 con 0x7f048806cbd0 2023-11-09T03:46:29.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.403+0000 7f048d6c0640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f048815b140 0x7f048806cfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53078/0 (socket says 172.21.15.76:53078) 2023-11-09T03:46:29.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.403+0000 7f048d6c0640 1 -- 172.21.15.76:0/3985106716 learned_addr learned my addr 172.21.15.76:0/3985106716 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:29.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.403+0000 7f048e6c2640 1 --2- 172.21.15.76:0/3985106716 >> v2:172.21.15.113:3300/0 conn(0x7f0488067090 0x7f04880b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:29.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.403+0000 7f048dec1640 1 --2- 172.21.15.76:0/3985106716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f048806cbd0 0x7f048806e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:29.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.403+0000 7f048cebf640 1 -- 172.21.15.76:0/3985106716 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3577583240 0 0) 0x7f04880b24f0 con 0x7f048815b140 2023-11-09T03:46:29.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.403+0000 7f048cebf640 1 -- 172.21.15.76:0/3985106716 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0474003520 con 0x7f048815b140 2023-11-09T03:46:29.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.403+0000 7f048cebf640 1 -- 172.21.15.76:0/3985106716 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3749999772 0 0) 0x7f0474003520 con 0x7f048815b140 2023-11-09T03:46:29.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.403+0000 7f048cebf640 1 -- 172.21.15.76:0/3985106716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f048806cbd0 msgr2=0x7f048806e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:29.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.403+0000 7f048cebf640 1 --2- 172.21.15.76:0/3985106716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f048806cbd0 0x7f048806e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:29.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.403+0000 7f048cebf640 1 -- 172.21.15.76:0/3985106716 >> v2:172.21.15.113:3300/0 conn(0x7f0488067090 msgr2=0x7f04880b2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:29.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.403+0000 7f048cebf640 1 --2- 172.21.15.76:0/3985106716 >> v2:172.21.15.113:3300/0 conn(0x7f0488067090 0x7f04880b2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:29.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.403+0000 7f048cebf640 1 -- 172.21.15.76:0/3985106716 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04880cdf40 con 0x7f048815b140 2023-11-09T03:46:29.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.403+0000 7f048cebf640 1 -- 172.21.15.76:0/3985106716 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f0478001670 con 0x7f048815b140 2023-11-09T03:46:29.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.403+0000 7f048e6c2640 1 --2- 172.21.15.76:0/3985106716 >> v2:172.21.15.113:3300/0 conn(0x7f0488067090 0x7f04880b2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:46:29.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.403+0000 7f048cebf640 1 -- 172.21.15.76:0/3985106716 >> v1:172.21.15.76:6789/0 conn(0x7f048815b140 legacy=0x7f048806cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:29.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.403+0000 7f048cebf640 1 --2- 172.21.15.76:0/3985106716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0474003970 0x7f0474003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:29.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.403+0000 7f048cebf640 1 --2- 172.21.15.76:0/3985106716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f048806cbd0 0x7f04740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:29.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.403+0000 7f048cebf640 1 --2- 172.21.15.76:0/3985106716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0474004d00 0x7f0474005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:29.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.403+0000 7f048cebf640 1 -- 172.21.15.76:0/3985106716 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04880ce220 con 0x7f0474003970 2023-11-09T03:46:29.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.403+0000 7f048cebf640 1 -- 172.21.15.76:0/3985106716 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04880cdc60 con 0x7f0474004d00 2023-11-09T03:46:29.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.403+0000 7f048cebf640 1 -- 172.21.15.76:0/3985106716 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047400b980 con 0x7f048806cbd0 2023-11-09T03:46:29.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.404+0000 7f048dec1640 1 --2- 172.21.15.76:0/3985106716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0474003970 0x7f0474003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:29.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.404+0000 7f048d6c0640 1 --2- 172.21.15.76:0/3985106716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f048806cbd0 0x7f04740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:29.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.404+0000 7f048e6c2640 1 --2- 172.21.15.76:0/3985106716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0474004d00 0x7f0474005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:29.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.404+0000 7f048dec1640 1 -- 172.21.15.76:0/3985106716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f048806cbd0 msgr2=0x7f04740047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:29.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.404+0000 7f048dec1640 1 --2- 172.21.15.76:0/3985106716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f048806cbd0 0x7f04740047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:29.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.404+0000 7f048dec1640 1 -- 172.21.15.76:0/3985106716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0474004d00 msgr2=0x7f0474005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:29.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.404+0000 7f048dec1640 1 --2- 172.21.15.76:0/3985106716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0474004d00 0x7f0474005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:29.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.404+0000 7f048dec1640 1 -- 172.21.15.76:0/3985106716 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0474003520 con 0x7f0474003970 2023-11-09T03:46:29.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.404+0000 7f048e6c2640 1 --2- 172.21.15.76:0/3985106716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0474004d00 0x7f0474005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:29.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.405+0000 7f048dec1640 1 --2- 172.21.15.76:0/3985106716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0474003970 0x7f0474003d90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0484009ac0 tx=0x7f04740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50ae213e6d96345 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:29.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.405+0000 7f048cebf640 1 -- 172.21.15.76:0/3985106716 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048402c6b0 con 0x7f0474003970 2023-11-09T03:46:29.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.405+0000 7f048cebf640 1 -- 172.21.15.76:0/3985106716 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048402c810 con 0x7f0474003970 2023-11-09T03:46:29.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.405+0000 7f048cebf640 1 -- 172.21.15.76:0/3985106716 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048402cba0 con 0x7f0474003970 2023-11-09T03:46:29.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.405+0000 7f0490125640 1 -- 172.21.15.76:0/3985106716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0474003970 msgr2=0x7f0474003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:29.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.405+0000 7f0490125640 1 --2- 172.21.15.76:0/3985106716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0474003970 0x7f0474003d90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0484009ac0 tx=0x7f04740057f0 comp rx=0 tx=0).stop 2023-11-09T03:46:29.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.405+0000 7f048e6c2640 1 -- 172.21.15.76:0/3985106716 reap_dead start 2023-11-09T03:46:29.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.405+0000 7f0490125640 1 -- 172.21.15.76:0/3985106716 shutdown_connections 2023-11-09T03:46:29.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.405+0000 7f0490125640 1 -- 172.21.15.76:0/3985106716 >> 172.21.15.76:0/3985106716 conn(0x7f048806a120 msgr2=0x7f048806a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:29.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.405+0000 7f0490125640 1 -- 172.21.15.76:0/3985106716 shutdown_connections 2023-11-09T03:46:29.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.405+0000 7f0490125640 1 -- 172.21.15.76:0/3985106716 wait complete. 2023-11-09T03:46:29.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.406+0000 7f0490125640 1 Processor -- start 2023-11-09T03:46:29.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.407+0000 7f0490125640 1 -- start start 2023-11-09T03:46:29.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.407+0000 7f0490125640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0474003970 0x7f04881f1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:29.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.407+0000 7f0490125640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0488148030 0x7f048815c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:29.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.407+0000 7f0490125640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f048815cab0 0x7f04880c2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:29.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.407+0000 7f0490125640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04880c49d0 con 0x7f048815cab0 2023-11-09T03:46:29.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.407+0000 7f0490125640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048815cf60 con 0x7f0474003970 2023-11-09T03:46:29.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.407+0000 7f0490125640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04881f1a80 con 0x7f0488148030 2023-11-09T03:46:29.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.407+0000 7f048e6c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f048815cab0 0x7f04880c2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:29.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.407+0000 7f048e6c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f048815cab0 0x7f04880c2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36046/0 (socket says 172.21.15.76:36046) 2023-11-09T03:46:29.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.407+0000 7f048e6c2640 1 -- 172.21.15.76:0/171644501 learned_addr learned my addr 172.21.15.76:0/171644501 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:29.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.408+0000 7f048d6c0640 1 --2- 172.21.15.76:0/171644501 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0488148030 0x7f048815c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:29.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.408+0000 7f048dec1640 1 --2- 172.21.15.76:0/171644501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0474003970 0x7f04881f1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:29.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.408+0000 7f048e6c2640 1 -- 172.21.15.76:0/171644501 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0488148030 msgr2=0x7f048815c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:29.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.408+0000 7f048e6c2640 1 --2- 172.21.15.76:0/171644501 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0488148030 0x7f048815c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:29.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.408+0000 7f048e6c2640 1 -- 172.21.15.76:0/171644501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0474003970 msgr2=0x7f04881f1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:29.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.408+0000 7f048e6c2640 1 --2- 172.21.15.76:0/171644501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0474003970 0x7f04881f1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:29.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.408+0000 7f048e6c2640 1 -- 172.21.15.76:0/171644501 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04880c4df0 con 0x7f048815cab0 2023-11-09T03:46:29.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.408+0000 7f048e6c2640 1 --2- 172.21.15.76:0/171644501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f048815cab0 0x7f04880c2840 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f048806b230 tx=0x7f04800115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:29.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.408+0000 7f047e7fc640 1 -- 172.21.15.76:0/171644501 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048000e070 con 0x7f048815cab0 2023-11-09T03:46:29.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.408+0000 7f047e7fc640 1 -- 172.21.15.76:0/171644501 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0480011ed0 con 0x7f048815cab0 2023-11-09T03:46:29.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.408+0000 7f0490125640 1 -- 172.21.15.76:0/171644501 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04880c3d50 con 0x7f048815cab0 2023-11-09T03:46:29.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.408+0000 7f047e7fc640 1 -- 172.21.15.76:0/171644501 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0480003990 con 0x7f048815cab0 2023-11-09T03:46:29.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.408+0000 7f0490125640 1 -- 172.21.15.76:0/171644501 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04880c4310 con 0x7f048815cab0 2023-11-09T03:46:29.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.409+0000 7f0490125640 1 -- 172.21.15.76:0/171644501 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04880c4b10 con 0x7f048815cab0 2023-11-09T03:46:29.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.409+0000 7f047e7fc640 1 -- 172.21.15.76:0/171644501 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0480003af0 con 0x7f048815cab0 2023-11-09T03:46:29.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.410+0000 7f047e7fc640 1 --2- 172.21.15.76:0/171644501 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f046c06ae70 0x7f046c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:29.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.410+0000 7f047e7fc640 1 -- 172.21.15.76:0/171644501 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f048008a100 con 0x7f048815cab0 2023-11-09T03:46:29.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.410+0000 7f048dec1640 1 --2- 172.21.15.76:0/171644501 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f046c06ae70 0x7f046c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:29.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.410+0000 7f048dec1640 1 --2- 172.21.15.76:0/171644501 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f046c06ae70 0x7f046c06d330 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0484036000 tx=0x7f0484030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:29.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.414+0000 7f047e7fc640 1 -- 172.21.15.76:0/171644501 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f04880c4b10 con 0x7f048815cab0 2023-11-09T03:46:29.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.593+0000 7f0490125640 1 -- 172.21.15.76:0/171644501 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f04880c4b10 con 0x7f048815cab0 2023-11-09T03:46:29.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.594+0000 7f047e7fc640 1 -- 172.21.15.76:0/171644501 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f04880c4b10 con 0x7f048815cab0 2023-11-09T03:46:29.593 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2023-11-09T03:46:29.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.596+0000 7f0490125640 1 -- 172.21.15.76:0/171644501 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f046c06ae70 msgr2=0x7f046c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:29.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.596+0000 7f0490125640 1 --2- 172.21.15.76:0/171644501 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f046c06ae70 0x7f046c06d330 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0484036000 tx=0x7f0484030000 comp rx=0 tx=0).stop 2023-11-09T03:46:29.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.596+0000 7f0490125640 1 -- 172.21.15.76:0/171644501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f048815cab0 msgr2=0x7f04880c2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:29.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.596+0000 7f0490125640 1 --2- 172.21.15.76:0/171644501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f048815cab0 0x7f04880c2840 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f048806b230 tx=0x7f04800115a0 comp rx=0 tx=0).stop 2023-11-09T03:46:29.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.596+0000 7f0490125640 1 -- 172.21.15.76:0/171644501 shutdown_connections 2023-11-09T03:46:29.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.596+0000 7f0490125640 1 --2- 172.21.15.76:0/171644501 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f046c06ae70 0x7f046c06d330 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:29.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.596+0000 7f0490125640 1 --2- 172.21.15.76:0/171644501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f048815cab0 0x7f04880c2840 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:29.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.596+0000 7f0490125640 1 --2- 172.21.15.76:0/171644501 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0488148030 0x7f048815c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:29.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.596+0000 7f0490125640 1 --2- 172.21.15.76:0/171644501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0474003970 0x7f04881f1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:29.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.596+0000 7f0490125640 1 -- 172.21.15.76:0/171644501 >> 172.21.15.76:0/171644501 conn(0x7f048806a120 msgr2=0x7f04880b4160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:29.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.596+0000 7f0490125640 1 -- 172.21.15.76:0/171644501 shutdown_connections 2023-11-09T03:46:29.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:29.596+0000 7f0490125640 1 -- 172.21.15.76:0/171644501 wait complete. 2023-11-09T03:46:29.606 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi113.front.sepia.ceph.com... 2023-11-09T03:46:29.606 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:46:29.606 DEBUG:teuthology.orchestra.run.smithi113:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-09T03:46:29.674 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:46:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:46:29.675 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi113.stderr:ceph-fuse[93700]: fuse finished with error 0 and tester_r 0 2023-11-09T03:46:29.675 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:46:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:46:29.679 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi113.stderr:2023-11-09T03:46:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:46:29.684 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:46:29.689 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2023-11-09T03:46:29.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:29.689 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-09T03:46:29.886 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:46:29.887 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:46:29.903 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:46:29.903 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:46:29.958 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:46:30.000 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi076.stderr:2023-11-09T03:46:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:46:30.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.098+0000 7fa2d35fc640 1 Processor -- start 2023-11-09T03:46:30.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.098+0000 7fa2d35fc640 1 -- start start 2023-11-09T03:46:30.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.099+0000 7fa2d35fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa2cc0b0ae0 0x7fa2cc0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:30.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.099+0000 7fa2d35fc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2cc0b4110 con 0x7fa2cc159c70 2023-11-09T03:46:30.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.099+0000 7fa2d35fc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2cc0b43f0 con 0x7fa2cc06db20 2023-11-09T03:46:30.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.099+0000 7fa2d35fc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2cc0b49b0 con 0x7fa2cc0b0ae0 2023-11-09T03:46:30.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.099+0000 7fa2d1398640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa2cc159c70 0x7fa2cc0b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53090/0 (socket says 172.21.15.76:53090) 2023-11-09T03:46:30.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.099+0000 7fa2d1398640 1 -- 172.21.15.76:0/2011519984 learned_addr learned my addr 172.21.15.76:0/2011519984 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:30.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.099+0000 7fa2d0b97640 1 --2- 172.21.15.76:0/2011519984 >> v2:172.21.15.113:3300/0 conn(0x7fa2cc0b0ae0 0x7fa2cc0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:30.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.099+0000 7fa2c3fff640 1 -- 172.21.15.76:0/2011519984 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3104628109 0 0) 0x7fa2cc0b4110 con 0x7fa2cc159c70 2023-11-09T03:46:30.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.099+0000 7fa2c3fff640 1 -- 172.21.15.76:0/2011519984 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2b8003520 con 0x7fa2cc159c70 2023-11-09T03:46:30.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.099+0000 7fa2c3fff640 1 -- 172.21.15.76:0/2011519984 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1369800961 0 0) 0x7fa2b8003520 con 0x7fa2cc159c70 2023-11-09T03:46:30.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.099+0000 7fa2c3fff640 1 -- 172.21.15.76:0/2011519984 >> v2:172.21.15.113:3300/0 conn(0x7fa2cc0b0ae0 msgr2=0x7fa2cc0ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:30.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.099+0000 7fa2c3fff640 1 --2- 172.21.15.76:0/2011519984 >> v2:172.21.15.113:3300/0 conn(0x7fa2cc0b0ae0 0x7fa2cc0ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.099+0000 7fa2c3fff640 1 -- 172.21.15.76:0/2011519984 >> v1:172.21.15.113:6789/0 conn(0x7fa2cc06db20 legacy=0x7fa2cc15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:30.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.099+0000 7fa2c3fff640 1 -- 172.21.15.76:0/2011519984 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2cc0b46d0 con 0x7fa2cc159c70 2023-11-09T03:46:30.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.099+0000 7fa2c3fff640 1 -- 172.21.15.76:0/2011519984 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fa2c8001670 con 0x7fa2cc159c70 2023-11-09T03:46:30.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.099+0000 7fa2c3fff640 1 -- 172.21.15.76:0/2011519984 >> v1:172.21.15.76:6789/0 conn(0x7fa2cc159c70 legacy=0x7fa2cc0b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:30.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.099+0000 7fa2c3fff640 1 --2- 172.21.15.76:0/2011519984 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2b8003970 0x7fa2b8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:30.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.100+0000 7fa2c3fff640 1 --2- 172.21.15.76:0/2011519984 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2b80047c0 0x7fa2b8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:30.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.100+0000 7fa2c3fff640 1 --2- 172.21.15.76:0/2011519984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2b8005180 0x7fa2b800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:30.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.100+0000 7fa2c3fff640 1 -- 172.21.15.76:0/2011519984 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc0b49b0 con 0x7fa2b8005180 2023-11-09T03:46:30.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.100+0000 7fa2c3fff640 1 -- 172.21.15.76:0/2011519984 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b800c020 con 0x7fa2b80047c0 2023-11-09T03:46:30.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.100+0000 7fa2c3fff640 1 -- 172.21.15.76:0/2011519984 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b800c1e0 con 0x7fa2b8003970 2023-11-09T03:46:30.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.100+0000 7fa2d1b99640 1 --2- 172.21.15.76:0/2011519984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2b8005180 0x7fa2b800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:30.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.100+0000 7fa2d0b97640 1 --2- 172.21.15.76:0/2011519984 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2b80047c0 0x7fa2b8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:30.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.100+0000 7fa2d1398640 1 --2- 172.21.15.76:0/2011519984 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2b8003970 0x7fa2b8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:30.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.100+0000 7fa2d1b99640 1 -- 172.21.15.76:0/2011519984 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2b8003970 msgr2=0x7fa2b8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:30.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.100+0000 7fa2d1b99640 1 --2- 172.21.15.76:0/2011519984 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2b8003970 0x7fa2b8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.100+0000 7fa2d1b99640 1 -- 172.21.15.76:0/2011519984 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2b80047c0 msgr2=0x7fa2b8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:30.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.100+0000 7fa2d1b99640 1 --2- 172.21.15.76:0/2011519984 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2b80047c0 0x7fa2b8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.100+0000 7fa2d1b99640 1 -- 172.21.15.76:0/2011519984 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b8003520 con 0x7fa2b8005180 2023-11-09T03:46:30.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.100+0000 7fa2d1398640 1 --2- 172.21.15.76:0/2011519984 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2b8003970 0x7fa2b8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:30.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.101+0000 7fa2d1b99640 1 -- 172.21.15.76:0/2011519984 reap_dead start 2023-11-09T03:46:30.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.101+0000 7fa2d1b99640 1 --2- 172.21.15.76:0/2011519984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2b8005180 0x7fa2b800b980 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4001e50 tx=0x7fa2b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51871f97b66f948 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:30.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.101+0000 7fa2c3fff640 1 -- 172.21.15.76:0/2011519984 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4003b70 con 0x7fa2b8005180 2023-11-09T03:46:30.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.101+0000 7fa2c3fff640 1 -- 172.21.15.76:0/2011519984 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c4003d10 con 0x7fa2b8005180 2023-11-09T03:46:30.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.101+0000 7fa2c3fff640 1 -- 172.21.15.76:0/2011519984 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4005310 con 0x7fa2b8005180 2023-11-09T03:46:30.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.101+0000 7fa2d35fc640 1 -- 172.21.15.76:0/2011519984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2b8005180 msgr2=0x7fa2b800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:30.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.101+0000 7fa2d35fc640 1 --2- 172.21.15.76:0/2011519984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2b8005180 0x7fa2b800b980 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4001e50 tx=0x7fa2b80057f0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.101+0000 7fa2d35fc640 1 -- 172.21.15.76:0/2011519984 shutdown_connections 2023-11-09T03:46:30.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.101+0000 7fa2d35fc640 1 --2- 172.21.15.76:0/2011519984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2b8005180 0x7fa2b800b980 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.101+0000 7fa2d35fc640 1 -- 172.21.15.76:0/2011519984 >> 172.21.15.76:0/2011519984 conn(0x7fa2cc0ce950 msgr2=0x7fa2cc0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:30.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.101+0000 7fa2d35fc640 1 -- 172.21.15.76:0/2011519984 shutdown_connections 2023-11-09T03:46:30.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.101+0000 7fa2d35fc640 1 -- 172.21.15.76:0/2011519984 wait complete. 2023-11-09T03:46:30.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.103+0000 7fa2d35fc640 1 Processor -- start 2023-11-09T03:46:30.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.103+0000 7fa2d35fc640 1 -- start start 2023-11-09T03:46:30.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.103+0000 7fa2d35fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2b8005180 0x7fa2cc1e42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:30.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.103+0000 7fa2d35fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2cc146d70 0x7fa2cc156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:30.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.103+0000 7fa2d35fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2cc1e2ac0 0x7fa2cc1e2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:30.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.103+0000 7fa2d35fc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc1e2630 con 0x7fa2b8005180 2023-11-09T03:46:30.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.103+0000 7fa2d35fc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc157730 con 0x7fa2cc1e2ac0 2023-11-09T03:46:30.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.103+0000 7fa2d35fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc157a10 con 0x7fa2cc146d70 2023-11-09T03:46:30.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.103+0000 7fa2d1398640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2b8005180 0x7fa2cc1e42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:30.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.103+0000 7fa2d1398640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2b8005180 0x7fa2cc1e42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36068/0 (socket says 172.21.15.76:36068) 2023-11-09T03:46:30.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.103+0000 7fa2d1398640 1 -- 172.21.15.76:0/3322969940 learned_addr learned my addr 172.21.15.76:0/3322969940 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:30.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.104+0000 7fa2d1b99640 1 --2- 172.21.15.76:0/3322969940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2cc1e2ac0 0x7fa2cc1e2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:30.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.104+0000 7fa2d0b97640 1 --2- 172.21.15.76:0/3322969940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2cc146d70 0x7fa2cc156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:30.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.104+0000 7fa2d1398640 1 -- 172.21.15.76:0/3322969940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2cc146d70 msgr2=0x7fa2cc156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:30.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.104+0000 7fa2d1398640 1 --2- 172.21.15.76:0/3322969940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2cc146d70 0x7fa2cc156f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.104+0000 7fa2d1398640 1 -- 172.21.15.76:0/3322969940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2cc1e2ac0 msgr2=0x7fa2cc1e2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:30.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.104+0000 7fa2d1398640 1 --2- 172.21.15.76:0/3322969940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2cc1e2ac0 0x7fa2cc1e2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.104+0000 7fa2d1398640 1 -- 172.21.15.76:0/3322969940 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2cc1e22c0 con 0x7fa2b8005180 2023-11-09T03:46:30.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.104+0000 7fa2d0b97640 1 --2- 172.21.15.76:0/3322969940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2cc146d70 0x7fa2cc156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:30.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.104+0000 7fa2d1b99640 1 --2- 172.21.15.76:0/3322969940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2cc1e2ac0 0x7fa2cc1e2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:30.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.104+0000 7fa2d1398640 1 --2- 172.21.15.76:0/3322969940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2b8005180 0x7fa2cc1e42c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800e760 tx=0x7fa2c800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:30.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.104+0000 7fa2c1ffb640 1 -- 172.21.15.76:0/3322969940 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c800ee70 con 0x7fa2b8005180 2023-11-09T03:46:30.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.104+0000 7fa2c1ffb640 1 -- 172.21.15.76:0/3322969940 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c800c8d0 con 0x7fa2b8005180 2023-11-09T03:46:30.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.104+0000 7fa2c1ffb640 1 -- 172.21.15.76:0/3322969940 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c800cc40 con 0x7fa2b8005180 2023-11-09T03:46:30.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.104+0000 7fa2d35fc640 1 -- 172.21.15.76:0/3322969940 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2cc157440 con 0x7fa2b8005180 2023-11-09T03:46:30.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.104+0000 7fa2d35fc640 1 -- 172.21.15.76:0/3322969940 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2cc1471f0 con 0x7fa2b8005180 2023-11-09T03:46:30.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.105+0000 7fa2d35fc640 1 -- 172.21.15.76:0/3322969940 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2cc074290 con 0x7fa2b8005180 2023-11-09T03:46:30.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.105+0000 7fa2c1ffb640 1 -- 172.21.15.76:0/3322969940 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa2c801d460 con 0x7fa2b8005180 2023-11-09T03:46:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.106+0000 7fa2c1ffb640 1 --2- 172.21.15.76:0/3322969940 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa2b006aea0 0x7fa2b006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.106+0000 7fa2c1ffb640 1 -- 172.21.15.76:0/3322969940 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fa2c8089960 con 0x7fa2b8005180 2023-11-09T03:46:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.106+0000 7fa2d0b97640 1 --2- 172.21.15.76:0/3322969940 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa2b006aea0 0x7fa2b006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.106+0000 7fa2d0b97640 1 --2- 172.21.15.76:0/3322969940 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa2b006aea0 0x7fa2b006d360 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009a40 tx=0x7fa2bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.110+0000 7fa2c1ffb640 1 -- 172.21.15.76:0/3322969940 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa2c80580c0 con 0x7fa2b8005180 2023-11-09T03:46:30.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.276+0000 7fa2d35fc640 1 -- 172.21.15.76:0/3322969940 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2cc0c9f10 con 0x7fa2b8005180 2023-11-09T03:46:30.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.277+0000 7fa2c1ffb640 1 -- 172.21.15.76:0/3322969940 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18710 (secure 0 0 0) 0x7fa2c8057c50 con 0x7fa2b8005180 2023-11-09T03:46:30.277 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:30.277 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":27,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:46:12.537265+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:46:30.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.279+0000 7fa2d35fc640 1 -- 172.21.15.76:0/3322969940 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa2b006aea0 msgr2=0x7fa2b006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:30.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.279+0000 7fa2d35fc640 1 --2- 172.21.15.76:0/3322969940 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa2b006aea0 0x7fa2b006d360 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009a40 tx=0x7fa2bc008040 comp rx=0 tx=0).stop 2023-11-09T03:46:30.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.279+0000 7fa2d35fc640 1 -- 172.21.15.76:0/3322969940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2b8005180 msgr2=0x7fa2cc1e42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:30.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.279+0000 7fa2d35fc640 1 --2- 172.21.15.76:0/3322969940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2b8005180 0x7fa2cc1e42c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800e760 tx=0x7fa2c800ec20 comp rx=0 tx=0).stop 2023-11-09T03:46:30.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.279+0000 7fa2d35fc640 1 -- 172.21.15.76:0/3322969940 shutdown_connections 2023-11-09T03:46:30.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.279+0000 7fa2d35fc640 1 --2- 172.21.15.76:0/3322969940 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa2b006aea0 0x7fa2b006d360 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.279+0000 7fa2d35fc640 1 --2- 172.21.15.76:0/3322969940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2cc1e2ac0 0x7fa2cc1e2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.279+0000 7fa2d35fc640 1 --2- 172.21.15.76:0/3322969940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2cc146d70 0x7fa2cc156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.279+0000 7fa2d35fc640 1 --2- 172.21.15.76:0/3322969940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2b8005180 0x7fa2cc1e42c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.279+0000 7fa2d35fc640 1 -- 172.21.15.76:0/3322969940 >> 172.21.15.76:0/3322969940 conn(0x7fa2cc0ce950 msgr2=0x7fa2cc0ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:30.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.279+0000 7fa2d35fc640 1 -- 172.21.15.76:0/3322969940 shutdown_connections 2023-11-09T03:46:30.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.279+0000 7fa2d35fc640 1 -- 172.21.15.76:0/3322969940 wait complete. 2023-11-09T03:46:30.290 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:45:28.704715+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:45:29.717614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-09T03:45:33.440577+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-09T03:46:30.290 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:46:30.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.393+0000 7f868a71d640 1 Processor -- start 2023-11-09T03:46:30.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.393+0000 7f868a71d640 1 -- start start 2023-11-09T03:46:30.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.394+0000 7f868a71d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8684070e20 0x7f86840adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:30.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.394+0000 7f868a71d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f86840ad730 0x7f868415e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:30.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.394+0000 7f868a71d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86840c3900 con 0x7f868415ad30 2023-11-09T03:46:30.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.394+0000 7f868a71d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86840ae260 con 0x7f86840ad730 2023-11-09T03:46:30.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.394+0000 7f868a71d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86840c3be0 con 0x7f8684070e20 2023-11-09T03:46:30.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.394+0000 7f8683fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f868415ad30 0x7f8684070710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53092/0 (socket says 172.21.15.76:53092) 2023-11-09T03:46:30.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.394+0000 7f8683fff640 1 -- 172.21.15.76:0/1595032294 learned_addr learned my addr 172.21.15.76:0/1595032294 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:30.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.394+0000 7f86837fe640 1 --2- 172.21.15.76:0/1595032294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8684070e20 0x7f86840adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:30.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.394+0000 7f8688cba640 1 --2- 172.21.15.76:0/1595032294 >> v2:172.21.15.113:3300/0 conn(0x7f86840ad730 0x7f868415e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:30.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.394+0000 7f8682ffd640 1 -- 172.21.15.76:0/1595032294 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1169910438 0 0) 0x7f86840c3900 con 0x7f868415ad30 2023-11-09T03:46:30.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.394+0000 7f8682ffd640 1 -- 172.21.15.76:0/1595032294 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f866c003520 con 0x7f868415ad30 2023-11-09T03:46:30.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.394+0000 7f8682ffd640 1 -- 172.21.15.76:0/1595032294 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2751776458 0 0) 0x7f866c003520 con 0x7f868415ad30 2023-11-09T03:46:30.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.394+0000 7f8682ffd640 1 -- 172.21.15.76:0/1595032294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8684070e20 msgr2=0x7f86840adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:30.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.394+0000 7f8682ffd640 1 --2- 172.21.15.76:0/1595032294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8684070e20 0x7f86840adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.395+0000 7f8682ffd640 1 -- 172.21.15.76:0/1595032294 >> v2:172.21.15.113:3300/0 conn(0x7f86840ad730 msgr2=0x7f868415e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:30.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.395+0000 7f8682ffd640 1 --2- 172.21.15.76:0/1595032294 >> v2:172.21.15.113:3300/0 conn(0x7f86840ad730 0x7f868415e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.395+0000 7f8682ffd640 1 -- 172.21.15.76:0/1595032294 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86840c3ec0 con 0x7f868415ad30 2023-11-09T03:46:30.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.395+0000 7f8682ffd640 1 -- 172.21.15.76:0/1595032294 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f8674001670 con 0x7f868415ad30 2023-11-09T03:46:30.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.395+0000 7f8682ffd640 1 -- 172.21.15.76:0/1595032294 >> v1:172.21.15.76:6789/0 conn(0x7f868415ad30 legacy=0x7f8684070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:30.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.395+0000 7f8682ffd640 1 --2- 172.21.15.76:0/1595032294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f866c003970 0x7f866c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:30.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.395+0000 7f8682ffd640 1 --2- 172.21.15.76:0/1595032294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8684070e20 0x7f866c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:30.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.395+0000 7f8682ffd640 1 --2- 172.21.15.76:0/1595032294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f866c004d00 0x7f866c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:30.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.395+0000 7f8682ffd640 1 -- 172.21.15.76:0/1595032294 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86840ae260 con 0x7f866c003970 2023-11-09T03:46:30.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.395+0000 7f8682ffd640 1 -- 172.21.15.76:0/1595032294 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86840c3be0 con 0x7f866c004d00 2023-11-09T03:46:30.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.395+0000 7f8682ffd640 1 -- 172.21.15.76:0/1595032294 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866c00b980 con 0x7f8684070e20 2023-11-09T03:46:30.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.395+0000 7f8683fff640 1 --2- 172.21.15.76:0/1595032294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f866c003970 0x7f866c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:30.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.395+0000 7f86837fe640 1 --2- 172.21.15.76:0/1595032294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8684070e20 0x7f866c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:30.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.395+0000 7f8688cba640 1 --2- 172.21.15.76:0/1595032294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f866c004d00 0x7f866c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:30.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.395+0000 7f8683fff640 1 -- 172.21.15.76:0/1595032294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8684070e20 msgr2=0x7f866c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:30.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.395+0000 7f8683fff640 1 --2- 172.21.15.76:0/1595032294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8684070e20 0x7f866c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.395+0000 7f8683fff640 1 -- 172.21.15.76:0/1595032294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f866c004d00 msgr2=0x7f866c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:30.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.395+0000 7f8683fff640 1 --2- 172.21.15.76:0/1595032294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f866c004d00 0x7f866c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.395+0000 7f8683fff640 1 -- 172.21.15.76:0/1595032294 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866c003520 con 0x7f866c003970 2023-11-09T03:46:30.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.396+0000 7f8683fff640 1 --2- 172.21.15.76:0/1595032294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f866c003970 0x7f866c003d90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f867400c290 tx=0x7f866c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff204b5ec5eb3e4b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:30.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.396+0000 7f8682ffd640 1 -- 172.21.15.76:0/1595032294 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674005ad0 con 0x7f866c003970 2023-11-09T03:46:30.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.396+0000 7f8682ffd640 1 -- 172.21.15.76:0/1595032294 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8674005c30 con 0x7f866c003970 2023-11-09T03:46:30.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.396+0000 7f8682ffd640 1 -- 172.21.15.76:0/1595032294 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867402c8d0 con 0x7f866c003970 2023-11-09T03:46:30.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.396+0000 7f868a71d640 1 -- 172.21.15.76:0/1595032294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f866c003970 msgr2=0x7f866c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:30.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.396+0000 7f868a71d640 1 --2- 172.21.15.76:0/1595032294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f866c003970 0x7f866c003d90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f867400c290 tx=0x7f866c0057f0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.396+0000 7f8688cba640 1 -- 172.21.15.76:0/1595032294 reap_dead start 2023-11-09T03:46:30.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.396+0000 7f868a71d640 1 -- 172.21.15.76:0/1595032294 shutdown_connections 2023-11-09T03:46:30.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.396+0000 7f868a71d640 1 -- 172.21.15.76:0/1595032294 >> 172.21.15.76:0/1595032294 conn(0x7f86840ce950 msgr2=0x7f86840ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:30.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.396+0000 7f868a71d640 1 -- 172.21.15.76:0/1595032294 shutdown_connections 2023-11-09T03:46:30.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.396+0000 7f868a71d640 1 -- 172.21.15.76:0/1595032294 wait complete. 2023-11-09T03:46:30.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.397+0000 7f868a71d640 1 Processor -- start 2023-11-09T03:46:30.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.398+0000 7f868a71d640 1 -- start start 2023-11-09T03:46:30.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.398+0000 7f868a71d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f866c003970 0x7f8684158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:30.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.398+0000 7f868a71d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f868414f680 0x7f8684156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:30.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.398+0000 7f868a71d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8684157470 0x7f86840cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:30.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.398+0000 7f868a71d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8684157960 con 0x7f8684157470 2023-11-09T03:46:30.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.398+0000 7f868a71d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8684157c40 con 0x7f868414f680 2023-11-09T03:46:30.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.398+0000 7f868a71d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8684157f20 con 0x7f866c003970 2023-11-09T03:46:30.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.399+0000 7f8688cba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8684157470 0x7f86840cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:30.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.399+0000 7f8688cba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8684157470 0x7f86840cd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36092/0 (socket says 172.21.15.76:36092) 2023-11-09T03:46:30.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.399+0000 7f8688cba640 1 -- 172.21.15.76:0/279421365 learned_addr learned my addr 172.21.15.76:0/279421365 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:30.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.399+0000 7f8683fff640 1 --2- 172.21.15.76:0/279421365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f866c003970 0x7f8684158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:30.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.399+0000 7f86837fe640 1 --2- 172.21.15.76:0/279421365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f868414f680 0x7f8684156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:30.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.399+0000 7f8688cba640 1 -- 172.21.15.76:0/279421365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f866c003970 msgr2=0x7f8684158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:30.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.399+0000 7f8688cba640 1 --2- 172.21.15.76:0/279421365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f866c003970 0x7f8684158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.399+0000 7f8688cba640 1 -- 172.21.15.76:0/279421365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f868414f680 msgr2=0x7f8684156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:30.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.399+0000 7f8688cba640 1 --2- 172.21.15.76:0/279421365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f868414f680 0x7f8684156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.399+0000 7f8688cba640 1 -- 172.21.15.76:0/279421365 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8684158200 con 0x7f8684157470 2023-11-09T03:46:30.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.399+0000 7f86837fe640 1 --2- 172.21.15.76:0/279421365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f868414f680 0x7f8684156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:30.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.399+0000 7f8683fff640 1 --2- 172.21.15.76:0/279421365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f866c003970 0x7f8684158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:30.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.399+0000 7f8688cba640 1 --2- 172.21.15.76:0/279421365 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8684157470 0x7f86840cd0d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f86840ce890 tx=0x7f867c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:30.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.399+0000 7f8680ff9640 1 -- 172.21.15.76:0/279421365 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c00e070 con 0x7f8684157470 2023-11-09T03:46:30.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.399+0000 7f8680ff9640 1 -- 172.21.15.76:0/279421365 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c011e10 con 0x7f8684157470 2023-11-09T03:46:30.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.399+0000 7f868a71d640 1 -- 172.21.15.76:0/279421365 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86841584e0 con 0x7f8684157470 2023-11-09T03:46:30.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.399+0000 7f8680ff9640 1 -- 172.21.15.76:0/279421365 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c003990 con 0x7f8684157470 2023-11-09T03:46:30.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.399+0000 7f868a71d640 1 -- 172.21.15.76:0/279421365 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86840cddb0 con 0x7f8684157470 2023-11-09T03:46:30.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.400+0000 7f868a71d640 1 -- 172.21.15.76:0/279421365 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86841586d0 con 0x7f8684157470 2023-11-09T03:46:30.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.401+0000 7f8680ff9640 1 -- 172.21.15.76:0/279421365 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f867c003af0 con 0x7f8684157470 2023-11-09T03:46:30.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.401+0000 7f8680ff9640 1 --2- 172.21.15.76:0/279421365 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f867006ae70 0x7f867006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:30.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.401+0000 7f8680ff9640 1 -- 172.21.15.76:0/279421365 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f867c089fd0 con 0x7f8684157470 2023-11-09T03:46:30.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.401+0000 7f8683fff640 1 --2- 172.21.15.76:0/279421365 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f867006ae70 0x7f867006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:30.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.402+0000 7f8683fff640 1 --2- 172.21.15.76:0/279421365 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f867006ae70 0x7f867006d330 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8674034000 tx=0x7f8674007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:30.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.405+0000 7f8680ff9640 1 -- 172.21.15.76:0/279421365 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f867c054d30 con 0x7f8684157470 2023-11-09T03:46:30.502 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:46:30.505 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:46:30.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.575+0000 7f868a71d640 1 -- 172.21.15.76:0/279421365 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8684074290 con 0x7f8684157470 2023-11-09T03:46:30.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.575+0000 7f8680ff9640 1 -- 172.21.15.76:0/279421365 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f867c058350 con 0x7f8684157470 2023-11-09T03:46:30.575 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:46:30.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.577+0000 7f868a71d640 1 -- 172.21.15.76:0/279421365 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f867006ae70 msgr2=0x7f867006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:30.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.577+0000 7f868a71d640 1 --2- 172.21.15.76:0/279421365 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f867006ae70 0x7f867006d330 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8674034000 tx=0x7f8674007000 comp rx=0 tx=0).stop 2023-11-09T03:46:30.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.577+0000 7f868a71d640 1 -- 172.21.15.76:0/279421365 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8684157470 msgr2=0x7f86840cd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:30.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.577+0000 7f868a71d640 1 --2- 172.21.15.76:0/279421365 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8684157470 0x7f86840cd0d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f86840ce890 tx=0x7f867c0115a0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.577+0000 7f868a71d640 1 -- 172.21.15.76:0/279421365 shutdown_connections 2023-11-09T03:46:30.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.577+0000 7f868a71d640 1 --2- 172.21.15.76:0/279421365 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f867006ae70 0x7f867006d330 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.577+0000 7f868a71d640 1 --2- 172.21.15.76:0/279421365 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8684157470 0x7f86840cd0d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.577+0000 7f868a71d640 1 --2- 172.21.15.76:0/279421365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f868414f680 0x7f8684156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.577+0000 7f868a71d640 1 --2- 172.21.15.76:0/279421365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f866c003970 0x7f8684158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.577+0000 7f868a71d640 1 -- 172.21.15.76:0/279421365 >> 172.21.15.76:0/279421365 conn(0x7f86840ce950 msgr2=0x7f8684072ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:30.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.578+0000 7f868a71d640 1 -- 172.21.15.76:0/279421365 shutdown_connections 2023-11-09T03:46:30.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.578+0000 7f868a71d640 1 -- 172.21.15.76:0/279421365 wait complete. 2023-11-09T03:46:30.587 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:46:30.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.732+0000 7f21d16e0640 1 Processor -- start 2023-11-09T03:46:30.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.732+0000 7f21d16e0640 1 -- start start 2023-11-09T03:46:30.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.733+0000 7f21d16e0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f21cc159c00 0x7f21cc0cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:30.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.733+0000 7f21d16e0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21cc0ae1d0 0x7f21cc0c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:30.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.733+0000 7f21d16e0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21cc15aff0 con 0x7f21cc0c48e0 2023-11-09T03:46:30.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.733+0000 7f21d16e0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21cc0c52d0 con 0x7f21cc159c00 2023-11-09T03:46:30.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.733+0000 7f21d16e0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc15a3e0 con 0x7f21cc0ae1d0 2023-11-09T03:46:30.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.733+0000 7f21caffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f21cc159c00 0x7f21cc0cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:30.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.733+0000 7f21caffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f21cc159c00 0x7f21cc0cc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36104/0 (socket says 172.21.15.76:36104) 2023-11-09T03:46:30.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.733+0000 7f21caffd640 1 -- 172.21.15.76:0/277448906 learned_addr learned my addr 172.21.15.76:0/277448906 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:30.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.733+0000 7f21caffd640 1 -- 172.21.15.76:0/277448906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21cc0ae1d0 msgr2=0x7f21cc0c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:30.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.733+0000 7f21cb7fe640 1 --2- 172.21.15.76:0/277448906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21cc0ae1d0 0x7f21cc0c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:30.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.733+0000 7f21caffd640 1 --2- 172.21.15.76:0/277448906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21cc0ae1d0 0x7f21cc0c4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.733+0000 7f21caffd640 1 -- 172.21.15.76:0/277448906 >> v1:172.21.15.113:6789/0 conn(0x7f21cc0c48e0 legacy=0x7f21cc0adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:30.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.733+0000 7f21caffd640 1 -- 172.21.15.76:0/277448906 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21cc15a6c0 con 0x7f21cc159c00 2023-11-09T03:46:30.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.733+0000 7f21cb7fe640 1 --2- 172.21.15.76:0/277448906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21cc0ae1d0 0x7f21cc0c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:30.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.734+0000 7f21caffd640 1 --2- 172.21.15.76:0/277448906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21cc159c00 0x7f21cc0cc890 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f21bc009e50 tx=0x7f21bc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd94e0787bf190c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:30.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.734+0000 7f21c9ffb640 1 -- 172.21.15.76:0/277448906 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc02c9e0 con 0x7f21cc159c00 2023-11-09T03:46:30.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.734+0000 7f21c9ffb640 1 -- 172.21.15.76:0/277448906 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21bc02cb40 con 0x7f21cc159c00 2023-11-09T03:46:30.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.734+0000 7f21c9ffb640 1 -- 172.21.15.76:0/277448906 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc0343f0 con 0x7f21cc159c00 2023-11-09T03:46:30.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.735+0000 7f21d16e0640 1 -- 172.21.15.76:0/277448906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21cc159c00 msgr2=0x7f21cc0cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:30.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.735+0000 7f21d16e0640 1 --2- 172.21.15.76:0/277448906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21cc159c00 0x7f21cc0cc890 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f21bc009e50 tx=0x7f21bc02be40 comp rx=0 tx=0).stop 2023-11-09T03:46:30.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.735+0000 7f21d16e0640 1 -- 172.21.15.76:0/277448906 shutdown_connections 2023-11-09T03:46:30.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.735+0000 7f21d16e0640 1 --2- 172.21.15.76:0/277448906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21cc0ae1d0 0x7f21cc0c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.735+0000 7f21d16e0640 1 --2- 172.21.15.76:0/277448906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21cc159c00 0x7f21cc0cc890 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.735+0000 7f21d16e0640 1 -- 172.21.15.76:0/277448906 >> 172.21.15.76:0/277448906 conn(0x7f21cc0ce950 msgr2=0x7f21cc0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:30.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.735+0000 7f21d16e0640 1 -- 172.21.15.76:0/277448906 shutdown_connections 2023-11-09T03:46:30.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.735+0000 7f21d16e0640 1 -- 172.21.15.76:0/277448906 wait complete. 2023-11-09T03:46:30.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.736+0000 7f21d16e0640 1 Processor -- start 2023-11-09T03:46:30.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.736+0000 7f21d16e0640 1 -- start start 2023-11-09T03:46:30.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.737+0000 7f21d16e0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21cc0ae1d0 0x7f21cc0c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:30.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.737+0000 7f21d16e0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21cc0c48e0 0x7f21cc15cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:30.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.737+0000 7f21d16e0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21cc159c00 0x7f21cc13ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:30.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.737+0000 7f21d16e0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc0c7aa0 con 0x7f21cc0c48e0 2023-11-09T03:46:30.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.737+0000 7f21d16e0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc143fe0 con 0x7f21cc0ae1d0 2023-11-09T03:46:30.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.737+0000 7f21d16e0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc15d280 con 0x7f21cc159c00 2023-11-09T03:46:30.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.737+0000 7f21ca7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21cc0c48e0 0x7f21cc15cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:30.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.737+0000 7f21ca7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21cc0c48e0 0x7f21cc15cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36116/0 (socket says 172.21.15.76:36116) 2023-11-09T03:46:30.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.737+0000 7f21ca7fc640 1 -- 172.21.15.76:0/548003831 learned_addr learned my addr 172.21.15.76:0/548003831 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:30.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.737+0000 7f21caffd640 1 --2- 172.21.15.76:0/548003831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21cc0ae1d0 0x7f21cc0c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:30.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.737+0000 7f21cb7fe640 1 --2- 172.21.15.76:0/548003831 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21cc159c00 0x7f21cc13ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:30.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.737+0000 7f21ca7fc640 1 -- 172.21.15.76:0/548003831 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21cc159c00 msgr2=0x7f21cc13ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:30.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.737+0000 7f21ca7fc640 1 --2- 172.21.15.76:0/548003831 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21cc159c00 0x7f21cc13ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.737+0000 7f21ca7fc640 1 -- 172.21.15.76:0/548003831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21cc0ae1d0 msgr2=0x7f21cc0c67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:30.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.737+0000 7f21ca7fc640 1 --2- 172.21.15.76:0/548003831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21cc0ae1d0 0x7f21cc0c67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.737+0000 7f21ca7fc640 1 -- 172.21.15.76:0/548003831 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21bc009b30 con 0x7f21cc0c48e0 2023-11-09T03:46:30.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.737+0000 7f21cb7fe640 1 --2- 172.21.15.76:0/548003831 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21cc159c00 0x7f21cc13ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:30.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.737+0000 7f21ca7fc640 1 --2- 172.21.15.76:0/548003831 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21cc0c48e0 0x7f21cc15cc10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f21c000c430 tx=0x7f21c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:30.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.737+0000 7f21b37fe640 1 -- 172.21.15.76:0/548003831 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c0011790 con 0x7f21cc0c48e0 2023-11-09T03:46:30.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.737+0000 7f21b37fe640 1 -- 172.21.15.76:0/548003831 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c00118f0 con 0x7f21cc0c48e0 2023-11-09T03:46:30.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.737+0000 7f21b37fe640 1 -- 172.21.15.76:0/548003831 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c0011ce0 con 0x7f21cc0c48e0 2023-11-09T03:46:30.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.737+0000 7f21d16e0640 1 -- 172.21.15.76:0/548003831 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21cc0c5a80 con 0x7f21cc0c48e0 2023-11-09T03:46:30.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.738+0000 7f21d16e0640 1 -- 172.21.15.76:0/548003831 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21cc0c62c0 con 0x7f21cc0c48e0 2023-11-09T03:46:30.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.739+0000 7f21d16e0640 1 -- 172.21.15.76:0/548003831 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21cc074290 con 0x7f21cc0c48e0 2023-11-09T03:46:30.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.739+0000 7f21b37fe640 1 -- 172.21.15.76:0/548003831 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f21c001c460 con 0x7f21cc0c48e0 2023-11-09T03:46:30.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.739+0000 7f21b37fe640 1 --2- 172.21.15.76:0/548003831 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f21b806ae70 0x7f21b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:30.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.739+0000 7f21b37fe640 1 -- 172.21.15.76:0/548003831 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f21c0089870 con 0x7f21cc0c48e0 2023-11-09T03:46:30.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.740+0000 7f21caffd640 1 --2- 172.21.15.76:0/548003831 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f21b806ae70 0x7f21b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:30.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.740+0000 7f21caffd640 1 --2- 172.21.15.76:0/548003831 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f21b806ae70 0x7f21b806d330 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f21bc035000 tx=0x7f21bc030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:30.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.744+0000 7f21b37fe640 1 -- 172.21.15.76:0/548003831 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f21c00545d0 con 0x7f21cc0c48e0 2023-11-09T03:46:30.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.908+0000 7f21d16e0640 1 -- 172.21.15.76:0/548003831 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f21cc0b4850 con 0x7f21cc0c48e0 2023-11-09T03:46:30.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.908+0000 7f21b37fe640 1 -- 172.21.15.76:0/548003831 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f21c0057bf0 con 0x7f21cc0c48e0 2023-11-09T03:46:30.908 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:46:30.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.910+0000 7f21d16e0640 1 -- 172.21.15.76:0/548003831 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f21b806ae70 msgr2=0x7f21b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:30.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.910+0000 7f21d16e0640 1 --2- 172.21.15.76:0/548003831 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f21b806ae70 0x7f21b806d330 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f21bc035000 tx=0x7f21bc030000 comp rx=0 tx=0).stop 2023-11-09T03:46:30.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.910+0000 7f21d16e0640 1 -- 172.21.15.76:0/548003831 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21cc0c48e0 msgr2=0x7f21cc15cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:30.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.910+0000 7f21d16e0640 1 --2- 172.21.15.76:0/548003831 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21cc0c48e0 0x7f21cc15cc10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f21c000c430 tx=0x7f21c000c8f0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.910+0000 7f21d16e0640 1 -- 172.21.15.76:0/548003831 shutdown_connections 2023-11-09T03:46:30.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.910+0000 7f21d16e0640 1 --2- 172.21.15.76:0/548003831 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f21b806ae70 0x7f21b806d330 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.910+0000 7f21d16e0640 1 --2- 172.21.15.76:0/548003831 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21cc159c00 0x7f21cc13ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.910+0000 7f21d16e0640 1 --2- 172.21.15.76:0/548003831 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21cc0c48e0 0x7f21cc15cc10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.910+0000 7f21d16e0640 1 --2- 172.21.15.76:0/548003831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21cc0ae1d0 0x7f21cc0c67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:30.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.910+0000 7f21d16e0640 1 -- 172.21.15.76:0/548003831 >> 172.21.15.76:0/548003831 conn(0x7f21cc0ce950 msgr2=0x7f21cc158ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:30.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.910+0000 7f21d16e0640 1 -- 172.21.15.76:0/548003831 shutdown_connections 2023-11-09T03:46:30.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:30.910+0000 7f21d16e0640 1 -- 172.21.15.76:0/548003831 wait complete. 2023-11-09T03:46:30.920 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:46:31.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.024+0000 7f7e2e1b6640 1 Processor -- start 2023-11-09T03:46:31.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.024+0000 7f7e2e1b6640 1 -- start start 2023-11-09T03:46:31.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.025+0000 7f7e2e1b6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e2815ad30 0x7f7e280adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.025+0000 7f7e2e1b6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7e28070e20 0x7f7e2815e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.025+0000 7f7e2e1b6640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e280c3900 con 0x7f7e280ad730 2023-11-09T03:46:31.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.025+0000 7f7e2e1b6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e280ae260 con 0x7f7e28070e20 2023-11-09T03:46:31.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.025+0000 7f7e2e1b6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e280c3be0 con 0x7f7e2815ad30 2023-11-09T03:46:31.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.025+0000 7f7e26ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f7e280ad730 0x7f7e28070710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53104/0 (socket says 172.21.15.76:53104) 2023-11-09T03:46:31.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.025+0000 7f7e26ffd640 1 -- 172.21.15.76:0/842108180 learned_addr learned my addr 172.21.15.76:0/842108180 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:31.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.025+0000 7f7e27fff640 1 --2- 172.21.15.76:0/842108180 >> v2:172.21.15.113:3300/0 conn(0x7f7e28070e20 0x7f7e2815e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.025+0000 7f7e277fe640 1 --2- 172.21.15.76:0/842108180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e2815ad30 0x7f7e280adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.025+0000 7f7e267fc640 1 -- 172.21.15.76:0/842108180 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 83513777 0 0) 0x7f7e280c3900 con 0x7f7e280ad730 2023-11-09T03:46:31.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.025+0000 7f7e267fc640 1 -- 172.21.15.76:0/842108180 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e10003520 con 0x7f7e280ad730 2023-11-09T03:46:31.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.025+0000 7f7e267fc640 1 -- 172.21.15.76:0/842108180 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 738160708 0 0) 0x7f7e10003520 con 0x7f7e280ad730 2023-11-09T03:46:31.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.025+0000 7f7e267fc640 1 -- 172.21.15.76:0/842108180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e2815ad30 msgr2=0x7f7e280adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:31.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.025+0000 7f7e267fc640 1 --2- 172.21.15.76:0/842108180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e2815ad30 0x7f7e280adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.025+0000 7f7e267fc640 1 -- 172.21.15.76:0/842108180 >> v2:172.21.15.113:3300/0 conn(0x7f7e28070e20 msgr2=0x7f7e2815e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:31.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.025+0000 7f7e267fc640 1 --2- 172.21.15.76:0/842108180 >> v2:172.21.15.113:3300/0 conn(0x7f7e28070e20 0x7f7e2815e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.025+0000 7f7e267fc640 1 -- 172.21.15.76:0/842108180 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e280c3ec0 con 0x7f7e280ad730 2023-11-09T03:46:31.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.025+0000 7f7e267fc640 1 -- 172.21.15.76:0/842108180 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f7e1c001670 con 0x7f7e280ad730 2023-11-09T03:46:31.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.026+0000 7f7e267fc640 1 -- 172.21.15.76:0/842108180 >> v1:172.21.15.76:6789/0 conn(0x7f7e280ad730 legacy=0x7f7e28070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:31.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.026+0000 7f7e267fc640 1 --2- 172.21.15.76:0/842108180 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e10003970 0x7f7e10003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.026+0000 7f7e267fc640 1 --2- 172.21.15.76:0/842108180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e100047c0 0x7f7e10004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.026+0000 7f7e267fc640 1 --2- 172.21.15.76:0/842108180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e2815ad30 0x7f7e10005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.026+0000 7f7e267fc640 1 -- 172.21.15.76:0/842108180 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e280ae260 con 0x7f7e10003970 2023-11-09T03:46:31.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.026+0000 7f7e267fc640 1 -- 172.21.15.76:0/842108180 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e280c3be0 con 0x7f7e100047c0 2023-11-09T03:46:31.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.026+0000 7f7e267fc640 1 -- 172.21.15.76:0/842108180 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1000b980 con 0x7f7e2815ad30 2023-11-09T03:46:31.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.026+0000 7f7e277fe640 1 --2- 172.21.15.76:0/842108180 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e10003970 0x7f7e10003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.026+0000 7f7e26ffd640 1 --2- 172.21.15.76:0/842108180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e100047c0 0x7f7e10004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.026+0000 7f7e27fff640 1 --2- 172.21.15.76:0/842108180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e2815ad30 0x7f7e10005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.026+0000 7f7e277fe640 1 -- 172.21.15.76:0/842108180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e2815ad30 msgr2=0x7f7e10005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:31.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.026+0000 7f7e277fe640 1 --2- 172.21.15.76:0/842108180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e2815ad30 0x7f7e10005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.026+0000 7f7e277fe640 1 -- 172.21.15.76:0/842108180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e100047c0 msgr2=0x7f7e10004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:31.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.026+0000 7f7e277fe640 1 --2- 172.21.15.76:0/842108180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e100047c0 0x7f7e10004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.026+0000 7f7e277fe640 1 -- 172.21.15.76:0/842108180 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e10003520 con 0x7f7e10003970 2023-11-09T03:46:31.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.026+0000 7f7e26ffd640 1 --2- 172.21.15.76:0/842108180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e100047c0 0x7f7e10004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:31.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.027+0000 7f7e277fe640 1 --2- 172.21.15.76:0/842108180 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e10003970 0x7f7e10003d90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009af0 tx=0x7f7e100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9074679fd2af33f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:31.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.027+0000 7f7e267fc640 1 -- 172.21.15.76:0/842108180 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1802c6b0 con 0x7f7e10003970 2023-11-09T03:46:31.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.027+0000 7f7e267fc640 1 -- 172.21.15.76:0/842108180 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e1802c810 con 0x7f7e10003970 2023-11-09T03:46:31.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.027+0000 7f7e267fc640 1 -- 172.21.15.76:0/842108180 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1802cba0 con 0x7f7e10003970 2023-11-09T03:46:31.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.027+0000 7f7e2e1b6640 1 -- 172.21.15.76:0/842108180 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e10003970 msgr2=0x7f7e10003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:31.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.027+0000 7f7e2e1b6640 1 --2- 172.21.15.76:0/842108180 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e10003970 0x7f7e10003d90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009af0 tx=0x7f7e100057f0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.027+0000 7f7e27fff640 1 -- 172.21.15.76:0/842108180 reap_dead start 2023-11-09T03:46:31.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.027+0000 7f7e2e1b6640 1 -- 172.21.15.76:0/842108180 shutdown_connections 2023-11-09T03:46:31.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.027+0000 7f7e2e1b6640 1 -- 172.21.15.76:0/842108180 >> 172.21.15.76:0/842108180 conn(0x7f7e280ce950 msgr2=0x7f7e280ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:31.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.027+0000 7f7e2e1b6640 1 -- 172.21.15.76:0/842108180 shutdown_connections 2023-11-09T03:46:31.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.027+0000 7f7e2e1b6640 1 -- 172.21.15.76:0/842108180 wait complete. 2023-11-09T03:46:31.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.028+0000 7f7e2e1b6640 1 Processor -- start 2023-11-09T03:46:31.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.029+0000 7f7e2e1b6640 1 -- start start 2023-11-09T03:46:31.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.029+0000 7f7e2e1b6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e10003970 0x7f7e281e4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.029+0000 7f7e2e1b6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e28147470 0x7f7e28159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.029+0000 7f7e2e1b6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e28159860 0x7f7e281e3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.029+0000 7f7e2e1b6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e28159ce0 con 0x7f7e28147470 2023-11-09T03:46:31.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.029+0000 7f7e2e1b6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e281e4a00 con 0x7f7e28159860 2023-11-09T03:46:31.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.029+0000 7f7e2e1b6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e281e3df0 con 0x7f7e10003970 2023-11-09T03:46:31.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.029+0000 7f7e26ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e28147470 0x7f7e28159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.029+0000 7f7e26ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e28147470 0x7f7e28159320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36140/0 (socket says 172.21.15.76:36140) 2023-11-09T03:46:31.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.030+0000 7f7e26ffd640 1 -- 172.21.15.76:0/248935911 learned_addr learned my addr 172.21.15.76:0/248935911 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:31.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.030+0000 7f7e277fe640 1 --2- 172.21.15.76:0/248935911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e10003970 0x7f7e281e4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.030+0000 7f7e27fff640 1 --2- 172.21.15.76:0/248935911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e28159860 0x7f7e281e3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.030+0000 7f7e26ffd640 1 -- 172.21.15.76:0/248935911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e10003970 msgr2=0x7f7e281e4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:31.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.030+0000 7f7e26ffd640 1 --2- 172.21.15.76:0/248935911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e10003970 0x7f7e281e4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.030+0000 7f7e26ffd640 1 -- 172.21.15.76:0/248935911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e28159860 msgr2=0x7f7e281e3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:31.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.030+0000 7f7e26ffd640 1 --2- 172.21.15.76:0/248935911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e28159860 0x7f7e281e3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.030+0000 7f7e26ffd640 1 -- 172.21.15.76:0/248935911 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e281e40d0 con 0x7f7e28147470 2023-11-09T03:46:31.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.030+0000 7f7e26ffd640 1 --2- 172.21.15.76:0/248935911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e28147470 0x7f7e28159320 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00e7b0 tx=0x7f7e1c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:31.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.030+0000 7f7e0ffff640 1 -- 172.21.15.76:0/248935911 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c00ee60 con 0x7f7e28147470 2023-11-09T03:46:31.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.030+0000 7f7e0ffff640 1 -- 172.21.15.76:0/248935911 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e1c00c870 con 0x7f7e28147470 2023-11-09T03:46:31.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.030+0000 7f7e0ffff640 1 -- 172.21.15.76:0/248935911 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c00cbe0 con 0x7f7e28147470 2023-11-09T03:46:31.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.030+0000 7f7e2e1b6640 1 -- 172.21.15.76:0/248935911 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e281e43b0 con 0x7f7e28147470 2023-11-09T03:46:31.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.030+0000 7f7e2e1b6640 1 -- 172.21.15.76:0/248935911 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e281478f0 con 0x7f7e28147470 2023-11-09T03:46:31.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.031+0000 7f7e2e1b6640 1 -- 172.21.15.76:0/248935911 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e28074290 con 0x7f7e28147470 2023-11-09T03:46:31.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.031+0000 7f7e0ffff640 1 -- 172.21.15.76:0/248935911 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7e1c00cd40 con 0x7f7e28147470 2023-11-09T03:46:31.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.032+0000 7f7e0ffff640 1 --2- 172.21.15.76:0/248935911 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7e1406ae70 0x7f7e1406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.032+0000 7f7e0ffff640 1 -- 172.21.15.76:0/248935911 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f7e1c089a10 con 0x7f7e28147470 2023-11-09T03:46:31.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.032+0000 7f7e277fe640 1 --2- 172.21.15.76:0/248935911 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7e1406ae70 0x7f7e1406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.033+0000 7f7e277fe640 1 --2- 172.21.15.76:0/248935911 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7e1406ae70 0x7f7e1406d330 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7e18036000 tx=0x7f7e18030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:31.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.036+0000 7f7e0ffff640 1 -- 172.21.15.76:0/248935911 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7e1c0580e0 con 0x7f7e28147470 2023-11-09T03:46:31.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.203+0000 7f7e2e1b6640 1 -- 172.21.15.76:0/248935911 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7e28071970 con 0x7f7e28147470 2023-11-09T03:46:31.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.204+0000 7f7e0ffff640 1 -- 172.21.15.76:0/248935911 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7e1c057c70 con 0x7f7e28147470 2023-11-09T03:46:31.203 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:46:31.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.206+0000 7f7e2e1b6640 1 -- 172.21.15.76:0/248935911 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7e1406ae70 msgr2=0x7f7e1406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:31.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.206+0000 7f7e2e1b6640 1 --2- 172.21.15.76:0/248935911 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7e1406ae70 0x7f7e1406d330 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7e18036000 tx=0x7f7e18030000 comp rx=0 tx=0).stop 2023-11-09T03:46:31.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.206+0000 7f7e2e1b6640 1 -- 172.21.15.76:0/248935911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e28147470 msgr2=0x7f7e28159320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:31.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.206+0000 7f7e2e1b6640 1 --2- 172.21.15.76:0/248935911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e28147470 0x7f7e28159320 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00e7b0 tx=0x7f7e1c00ec70 comp rx=0 tx=0).stop 2023-11-09T03:46:31.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.206+0000 7f7e2e1b6640 1 -- 172.21.15.76:0/248935911 shutdown_connections 2023-11-09T03:46:31.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.206+0000 7f7e2e1b6640 1 --2- 172.21.15.76:0/248935911 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7e1406ae70 0x7f7e1406d330 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.206+0000 7f7e2e1b6640 1 --2- 172.21.15.76:0/248935911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e28159860 0x7f7e281e3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.206+0000 7f7e2e1b6640 1 --2- 172.21.15.76:0/248935911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e28147470 0x7f7e28159320 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.206+0000 7f7e2e1b6640 1 --2- 172.21.15.76:0/248935911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e10003970 0x7f7e281e4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.206+0000 7f7e2e1b6640 1 -- 172.21.15.76:0/248935911 >> 172.21.15.76:0/248935911 conn(0x7f7e280ce950 msgr2=0x7f7e2806db20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:31.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.206+0000 7f7e2e1b6640 1 -- 172.21.15.76:0/248935911 shutdown_connections 2023-11-09T03:46:31.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.206+0000 7f7e2e1b6640 1 -- 172.21.15.76:0/248935911 wait complete. 2023-11-09T03:46:31.216 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-09T03:46:31.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.319+0000 7fc0fbe64640 1 Processor -- start 2023-11-09T03:46:31.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.319+0000 7fc0fbe64640 1 -- start start 2023-11-09T03:46:31.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.320+0000 7fc0fbe64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc0f415d1b0 0x7fc0f40cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.320+0000 7fc0fbe64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0f4066000 0x7fc0f406a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.320+0000 7fc0fbe64640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0f40c9bf0 con 0x7fc0f4070ff0 2023-11-09T03:46:31.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.320+0000 7fc0fbe64640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0f40cd890 con 0x7fc0f415d1b0 2023-11-09T03:46:31.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.320+0000 7fc0fbe64640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f40cdb30 con 0x7fc0f4066000 2023-11-09T03:46:31.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.320+0000 7fc0f93ff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc0f415d1b0 0x7fc0f40cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.320+0000 7fc0f93ff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc0f415d1b0 0x7fc0f40cb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36146/0 (socket says 172.21.15.76:36146) 2023-11-09T03:46:31.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.320+0000 7fc0f93ff640 1 -- 172.21.15.76:0/3687629053 learned_addr learned my addr 172.21.15.76:0/3687629053 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:31.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.320+0000 7fc0fa401640 1 --2- 172.21.15.76:0/3687629053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0f4066000 0x7fc0f406a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.320+0000 7fc0f93ff640 1 -- 172.21.15.76:0/3687629053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0f4066000 msgr2=0x7fc0f406a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:31.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.320+0000 7fc0f93ff640 1 --2- 172.21.15.76:0/3687629053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0f4066000 0x7fc0f406a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.320+0000 7fc0f93ff640 1 -- 172.21.15.76:0/3687629053 >> v1:172.21.15.113:6789/0 conn(0x7fc0f4070ff0 legacy=0x7fc0f40713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:31.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.320+0000 7fc0f93ff640 1 -- 172.21.15.76:0/3687629053 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f40cde10 con 0x7fc0f415d1b0 2023-11-09T03:46:31.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.321+0000 7fc0f93ff640 1 --2- 172.21.15.76:0/3687629053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0f415d1b0 0x7fc0f40cb4e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009870 tx=0x7fc0e402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9746a630e17983e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:31.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.321+0000 7fc0f8bfe640 1 -- 172.21.15.76:0/3687629053 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4031640 con 0x7fc0f415d1b0 2023-11-09T03:46:31.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.321+0000 7fc0f8bfe640 1 -- 172.21.15.76:0/3687629053 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e40317a0 con 0x7fc0f415d1b0 2023-11-09T03:46:31.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.321+0000 7fc0f8bfe640 1 -- 172.21.15.76:0/3687629053 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4031b30 con 0x7fc0f415d1b0 2023-11-09T03:46:31.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.322+0000 7fc0fbe64640 1 -- 172.21.15.76:0/3687629053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0f415d1b0 msgr2=0x7fc0f40cb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:31.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.322+0000 7fc0fbe64640 1 --2- 172.21.15.76:0/3687629053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0f415d1b0 0x7fc0f40cb4e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009870 tx=0x7fc0e402b8f0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.322+0000 7fc0fbe64640 1 -- 172.21.15.76:0/3687629053 shutdown_connections 2023-11-09T03:46:31.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.322+0000 7fc0fbe64640 1 --2- 172.21.15.76:0/3687629053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0f4066000 0x7fc0f406a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.322+0000 7fc0fbe64640 1 --2- 172.21.15.76:0/3687629053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0f415d1b0 0x7fc0f40cb4e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.322+0000 7fc0fbe64640 1 -- 172.21.15.76:0/3687629053 >> 172.21.15.76:0/3687629053 conn(0x7fc0f40cbc90 msgr2=0x7fc0f40cc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:31.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.322+0000 7fc0fbe64640 1 -- 172.21.15.76:0/3687629053 shutdown_connections 2023-11-09T03:46:31.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.322+0000 7fc0fbe64640 1 -- 172.21.15.76:0/3687629053 wait complete. 2023-11-09T03:46:31.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.323+0000 7fc0fbe64640 1 Processor -- start 2023-11-09T03:46:31.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.323+0000 7fc0fbe64640 1 -- start start 2023-11-09T03:46:31.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.324+0000 7fc0fbe64640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0f4066000 0x7fc0f415fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.324+0000 7fc0fbe64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0f4070ff0 0x7fc0f41565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.324+0000 7fc0fbe64640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0f415d1b0 0x7fc0f40c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.324+0000 7fc0fbe64640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f415f950 con 0x7fc0f415d1b0 2023-11-09T03:46:31.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.324+0000 7fc0fbe64640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f40c91f0 con 0x7fc0f4066000 2023-11-09T03:46:31.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.324+0000 7fc0fbe64640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f40c8f10 con 0x7fc0f4070ff0 2023-11-09T03:46:31.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.324+0000 7fc0fa401640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0f415d1b0 0x7fc0f40c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.324+0000 7fc0fa401640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0f415d1b0 0x7fc0f40c5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36148/0 (socket says 172.21.15.76:36148) 2023-11-09T03:46:31.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.324+0000 7fc0fa401640 1 -- 172.21.15.76:0/2516372040 learned_addr learned my addr 172.21.15.76:0/2516372040 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:31.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.324+0000 7fc0f93ff640 1 --2- 172.21.15.76:0/2516372040 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0f4070ff0 0x7fc0f41565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.324+0000 7fc0f9c00640 1 --2- 172.21.15.76:0/2516372040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0f4066000 0x7fc0f415fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.324+0000 7fc0fa401640 1 -- 172.21.15.76:0/2516372040 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0f4070ff0 msgr2=0x7fc0f41565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:31.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.324+0000 7fc0fa401640 1 --2- 172.21.15.76:0/2516372040 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0f4070ff0 0x7fc0f41565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.324+0000 7fc0fa401640 1 -- 172.21.15.76:0/2516372040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0f4066000 msgr2=0x7fc0f415fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:31.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.324+0000 7fc0fa401640 1 --2- 172.21.15.76:0/2516372040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0f4066000 0x7fc0f415fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.324+0000 7fc0fa401640 1 -- 172.21.15.76:0/2516372040 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e40094d0 con 0x7fc0f415d1b0 2023-11-09T03:46:31.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.324+0000 7fc0f93ff640 1 --2- 172.21.15.76:0/2516372040 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0f4070ff0 0x7fc0f41565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:31.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.324+0000 7fc0f9c00640 1 --2- 172.21.15.76:0/2516372040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0f4066000 0x7fc0f415fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:31.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.324+0000 7fc0fa401640 1 --2- 172.21.15.76:0/2516372040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0f415d1b0 0x7fc0f40c5690 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00cc30 tx=0x7fc0ec00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:31.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.324+0000 7fc0ea7fc640 1 -- 172.21.15.76:0/2516372040 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec00ce40 con 0x7fc0f415d1b0 2023-11-09T03:46:31.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.324+0000 7fc0ea7fc640 1 -- 172.21.15.76:0/2516372040 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0ec003990 con 0x7fc0f415d1b0 2023-11-09T03:46:31.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.324+0000 7fc0fbe64640 1 -- 172.21.15.76:0/2516372040 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f415f290 con 0x7fc0f415d1b0 2023-11-09T03:46:31.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.324+0000 7fc0ea7fc640 1 -- 172.21.15.76:0/2516372040 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec003d20 con 0x7fc0f415d1b0 2023-11-09T03:46:31.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.324+0000 7fc0fbe64640 1 -- 172.21.15.76:0/2516372040 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f40c5bd0 con 0x7fc0f415d1b0 2023-11-09T03:46:31.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.325+0000 7fc0fbe64640 1 -- 172.21.15.76:0/2516372040 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0f415f0c0 con 0x7fc0f415d1b0 2023-11-09T03:46:31.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.326+0000 7fc0ea7fc640 1 -- 172.21.15.76:0/2516372040 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc0ec003e80 con 0x7fc0f415d1b0 2023-11-09T03:46:31.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.326+0000 7fc0ea7fc640 1 --2- 172.21.15.76:0/2516372040 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc0d806ae70 0x7fc0d806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.326+0000 7fc0ea7fc640 1 -- 172.21.15.76:0/2516372040 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fc0ec089e20 con 0x7fc0f415d1b0 2023-11-09T03:46:31.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.326+0000 7fc0f9c00640 1 --2- 172.21.15.76:0/2516372040 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc0d806ae70 0x7fc0d806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.327+0000 7fc0f9c00640 1 --2- 172.21.15.76:0/2516372040 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc0d806ae70 0x7fc0d806d330 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40c7ab0 tx=0x7fc0f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:31.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.330+0000 7fc0ea7fc640 1 -- 172.21.15.76:0/2516372040 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc0ec054af0 con 0x7fc0f415d1b0 2023-11-09T03:46:31.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.501+0000 7fc0fbe64640 1 -- 172.21.15.76:0/2516372040 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc0f4074290 con 0x7fc0f415d1b0 2023-11-09T03:46:31.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.502+0000 7fc0ea7fc640 1 -- 172.21.15.76:0/2516372040 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc0ec01d070 con 0x7fc0f415d1b0 2023-11-09T03:46:31.501 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:46:31.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.504+0000 7fc0fbe64640 1 -- 172.21.15.76:0/2516372040 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc0d806ae70 msgr2=0x7fc0d806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:31.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.504+0000 7fc0fbe64640 1 --2- 172.21.15.76:0/2516372040 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc0d806ae70 0x7fc0d806d330 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40c7ab0 tx=0x7fc0f0009040 comp rx=0 tx=0).stop 2023-11-09T03:46:31.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.504+0000 7fc0fbe64640 1 -- 172.21.15.76:0/2516372040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0f415d1b0 msgr2=0x7fc0f40c5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:31.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.504+0000 7fc0fbe64640 1 --2- 172.21.15.76:0/2516372040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0f415d1b0 0x7fc0f40c5690 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00cc30 tx=0x7fc0ec00a5c0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.504+0000 7fc0fbe64640 1 -- 172.21.15.76:0/2516372040 shutdown_connections 2023-11-09T03:46:31.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.504+0000 7fc0fbe64640 1 --2- 172.21.15.76:0/2516372040 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc0d806ae70 0x7fc0d806d330 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.504+0000 7fc0fbe64640 1 --2- 172.21.15.76:0/2516372040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0f415d1b0 0x7fc0f40c5690 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.504+0000 7fc0fbe64640 1 --2- 172.21.15.76:0/2516372040 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0f4070ff0 0x7fc0f41565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.504+0000 7fc0fbe64640 1 --2- 172.21.15.76:0/2516372040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0f4066000 0x7fc0f415fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.504+0000 7fc0fbe64640 1 -- 172.21.15.76:0/2516372040 >> 172.21.15.76:0/2516372040 conn(0x7fc0f40cbc90 msgr2=0x7fc0f4160830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:31.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.504+0000 7fc0fbe64640 1 -- 172.21.15.76:0/2516372040 shutdown_connections 2023-11-09T03:46:31.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.504+0000 7fc0fbe64640 1 -- 172.21.15.76:0/2516372040 wait complete. 2023-11-09T03:46:31.514 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-09T03:46:31.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.616+0000 7fa6e7778640 1 Processor -- start 2023-11-09T03:46:31.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.616+0000 7fa6e7778640 1 -- start start 2023-11-09T03:46:31.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.617+0000 7fa6e7778640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6e015ad40 0x7fa6e006b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.617+0000 7fa6e7778640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa6e015b120 0x7fa6e015f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.617+0000 7fa6e7778640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6e0065df0 con 0x7fa6e00c9b60 2023-11-09T03:46:31.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.617+0000 7fa6e7778640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6e006bc40 con 0x7fa6e015b120 2023-11-09T03:46:31.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.617+0000 7fa6e7778640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e015f910 con 0x7fa6e015ad40 2023-11-09T03:46:31.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.617+0000 7fa6e5d15640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa6e00c9b60 0x7fa6e0070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53116/0 (socket says 172.21.15.76:53116) 2023-11-09T03:46:31.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.617+0000 7fa6e5d15640 1 -- 172.21.15.76:0/4108583418 learned_addr learned my addr 172.21.15.76:0/4108583418 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:31.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.617+0000 7fa6e4d13640 1 --2- 172.21.15.76:0/4108583418 >> v2:172.21.15.113:3300/0 conn(0x7fa6e015b120 0x7fa6e015f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.617+0000 7fa6e5514640 1 --2- 172.21.15.76:0/4108583418 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6e015ad40 0x7fa6e006b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.617+0000 7fa6d7fff640 1 -- 172.21.15.76:0/4108583418 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1165773649 0 0) 0x7fa6e0065df0 con 0x7fa6e00c9b60 2023-11-09T03:46:31.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.617+0000 7fa6d7fff640 1 -- 172.21.15.76:0/4108583418 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6cc003520 con 0x7fa6e00c9b60 2023-11-09T03:46:31.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.618+0000 7fa6d7fff640 1 -- 172.21.15.76:0/4108583418 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 860332724 0 0) 0x7fa6cc003520 con 0x7fa6e00c9b60 2023-11-09T03:46:31.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.618+0000 7fa6d7fff640 1 -- 172.21.15.76:0/4108583418 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6e015ad40 msgr2=0x7fa6e006b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:31.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.618+0000 7fa6d7fff640 1 --2- 172.21.15.76:0/4108583418 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6e015ad40 0x7fa6e006b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.618+0000 7fa6d7fff640 1 -- 172.21.15.76:0/4108583418 >> v2:172.21.15.113:3300/0 conn(0x7fa6e015b120 msgr2=0x7fa6e015f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:31.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.618+0000 7fa6d7fff640 1 --2- 172.21.15.76:0/4108583418 >> v2:172.21.15.113:3300/0 conn(0x7fa6e015b120 0x7fa6e015f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.618+0000 7fa6d7fff640 1 -- 172.21.15.76:0/4108583418 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e0066040 con 0x7fa6e00c9b60 2023-11-09T03:46:31.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.618+0000 7fa6d7fff640 1 -- 172.21.15.76:0/4108583418 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fa6d8001670 con 0x7fa6e00c9b60 2023-11-09T03:46:31.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.618+0000 7fa6d7fff640 1 -- 172.21.15.76:0/4108583418 >> v1:172.21.15.76:6789/0 conn(0x7fa6e00c9b60 legacy=0x7fa6e0070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:31.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.618+0000 7fa6d7fff640 1 --2- 172.21.15.76:0/4108583418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6cc003970 0x7fa6cc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.618+0000 7fa6d7fff640 1 --2- 172.21.15.76:0/4108583418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6cc0047c0 0x7fa6cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.618+0000 7fa6d7fff640 1 --2- 172.21.15.76:0/4108583418 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6e015ad40 0x7fa6cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.618+0000 7fa6d7fff640 1 -- 172.21.15.76:0/4108583418 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e006bc40 con 0x7fa6cc003970 2023-11-09T03:46:31.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.618+0000 7fa6d7fff640 1 -- 172.21.15.76:0/4108583418 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e015f910 con 0x7fa6cc0047c0 2023-11-09T03:46:31.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.618+0000 7fa6d7fff640 1 -- 172.21.15.76:0/4108583418 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6cc00b980 con 0x7fa6e015ad40 2023-11-09T03:46:31.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.618+0000 7fa6e5514640 1 --2- 172.21.15.76:0/4108583418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6cc003970 0x7fa6cc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.618+0000 7fa6e4d13640 1 --2- 172.21.15.76:0/4108583418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6cc0047c0 0x7fa6cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.618+0000 7fa6e5d15640 1 --2- 172.21.15.76:0/4108583418 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6e015ad40 0x7fa6cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.618+0000 7fa6e5514640 1 -- 172.21.15.76:0/4108583418 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6e015ad40 msgr2=0x7fa6cc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:31.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.618+0000 7fa6e5514640 1 --2- 172.21.15.76:0/4108583418 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6e015ad40 0x7fa6cc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.618+0000 7fa6e5514640 1 -- 172.21.15.76:0/4108583418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6cc0047c0 msgr2=0x7fa6cc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:31.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.618+0000 7fa6e5514640 1 --2- 172.21.15.76:0/4108583418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6cc0047c0 0x7fa6cc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.618+0000 7fa6e5514640 1 -- 172.21.15.76:0/4108583418 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6cc003520 con 0x7fa6cc003970 2023-11-09T03:46:31.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.618+0000 7fa6e5d15640 1 --2- 172.21.15.76:0/4108583418 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6e015ad40 0x7fa6cc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:31.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.618+0000 7fa6e4d13640 1 --2- 172.21.15.76:0/4108583418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6cc0047c0 0x7fa6cc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:31.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.619+0000 7fa6e5514640 1 --2- 172.21.15.76:0/4108583418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6cc003970 0x7fa6cc003d90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc009fc0 tx=0x7fa6cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e0c25be66d4cba2 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:31.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.619+0000 7fa6d7fff640 1 -- 172.21.15.76:0/4108583418 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc02cc20 con 0x7fa6cc003970 2023-11-09T03:46:31.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.619+0000 7fa6d7fff640 1 -- 172.21.15.76:0/4108583418 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6dc00f040 con 0x7fa6cc003970 2023-11-09T03:46:31.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.619+0000 7fa6d7fff640 1 -- 172.21.15.76:0/4108583418 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc034440 con 0x7fa6cc003970 2023-11-09T03:46:31.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.620+0000 7fa6e7778640 1 -- 172.21.15.76:0/4108583418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6cc003970 msgr2=0x7fa6cc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:31.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.620+0000 7fa6e7778640 1 --2- 172.21.15.76:0/4108583418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6cc003970 0x7fa6cc003d90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc009fc0 tx=0x7fa6cc0057f0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.620+0000 7fa6e5d15640 1 -- 172.21.15.76:0/4108583418 reap_dead start 2023-11-09T03:46:31.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.620+0000 7fa6e7778640 1 -- 172.21.15.76:0/4108583418 shutdown_connections 2023-11-09T03:46:31.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.620+0000 7fa6e7778640 1 -- 172.21.15.76:0/4108583418 >> 172.21.15.76:0/4108583418 conn(0x7fa6e00cbc90 msgr2=0x7fa6e00cc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:31.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.620+0000 7fa6e7778640 1 -- 172.21.15.76:0/4108583418 shutdown_connections 2023-11-09T03:46:31.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.620+0000 7fa6e7778640 1 -- 172.21.15.76:0/4108583418 wait complete. 2023-11-09T03:46:31.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.622+0000 7fa6e7778640 1 Processor -- start 2023-11-09T03:46:31.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.622+0000 7fa6e7778640 1 -- start start 2023-11-09T03:46:31.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.622+0000 7fa6e7778640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6cc003970 0x7fa6e014fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.622+0000 7fa6e7778640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6e0151c70 0x7fa6e014ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.622+0000 7fa6e7778640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6e013d470 0x7fa6e013d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.622+0000 7fa6e7778640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e00c2520 con 0x7fa6e0151c70 2023-11-09T03:46:31.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.622+0000 7fa6e7778640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e0152150 con 0x7fa6cc003970 2023-11-09T03:46:31.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.622+0000 7fa6e7778640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e0152430 con 0x7fa6e013d470 2023-11-09T03:46:31.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.623+0000 7fa6e4d13640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6e0151c70 0x7fa6e014ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.623+0000 7fa6e4d13640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6e0151c70 0x7fa6e014ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36162/0 (socket says 172.21.15.76:36162) 2023-11-09T03:46:31.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.623+0000 7fa6e4d13640 1 -- 172.21.15.76:0/2648654212 learned_addr learned my addr 172.21.15.76:0/2648654212 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:31.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.623+0000 7fa6e5514640 1 --2- 172.21.15.76:0/2648654212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6cc003970 0x7fa6e014fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.623+0000 7fa6e5d15640 1 --2- 172.21.15.76:0/2648654212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6e013d470 0x7fa6e013d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.623+0000 7fa6e4d13640 1 -- 172.21.15.76:0/2648654212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6e013d470 msgr2=0x7fa6e013d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:31.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.623+0000 7fa6e4d13640 1 --2- 172.21.15.76:0/2648654212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6e013d470 0x7fa6e013d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.623+0000 7fa6e4d13640 1 -- 172.21.15.76:0/2648654212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6cc003970 msgr2=0x7fa6e014fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:31.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.623+0000 7fa6e4d13640 1 --2- 172.21.15.76:0/2648654212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6cc003970 0x7fa6e014fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.623+0000 7fa6e4d13640 1 -- 172.21.15.76:0/2648654212 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e0152710 con 0x7fa6e0151c70 2023-11-09T03:46:31.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.623+0000 7fa6e5d15640 1 --2- 172.21.15.76:0/2648654212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6e013d470 0x7fa6e013d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:31.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.623+0000 7fa6e4d13640 1 --2- 172.21.15.76:0/2648654212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6e0151c70 0x7fa6e014ff50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0011660 tx=0x7fa6d0011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:31.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.623+0000 7fa6d5ffb640 1 -- 172.21.15.76:0/2648654212 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d000f9c0 con 0x7fa6e0151c70 2023-11-09T03:46:31.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.623+0000 7fa6d5ffb640 1 -- 172.21.15.76:0/2648654212 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d000fb20 con 0x7fa6e0151c70 2023-11-09T03:46:31.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.623+0000 7fa6e7778640 1 -- 172.21.15.76:0/2648654212 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e01529f0 con 0x7fa6e0151c70 2023-11-09T03:46:31.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.623+0000 7fa6d5ffb640 1 -- 172.21.15.76:0/2648654212 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d000fed0 con 0x7fa6e0151c70 2023-11-09T03:46:31.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.623+0000 7fa6e7778640 1 -- 172.21.15.76:0/2648654212 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e0141220 con 0x7fa6e0151c70 2023-11-09T03:46:31.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.624+0000 7fa6e7778640 1 -- 172.21.15.76:0/2648654212 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e0074290 con 0x7fa6e0151c70 2023-11-09T03:46:31.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.625+0000 7fa6d5ffb640 1 -- 172.21.15.76:0/2648654212 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa6d0015460 con 0x7fa6e0151c70 2023-11-09T03:46:31.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.625+0000 7fa6d5ffb640 1 --2- 172.21.15.76:0/2648654212 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa6c406ae70 0x7fa6c406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.625+0000 7fa6d5ffb640 1 -- 172.21.15.76:0/2648654212 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fa6d008a190 con 0x7fa6e0151c70 2023-11-09T03:46:31.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.626+0000 7fa6e5514640 1 --2- 172.21.15.76:0/2648654212 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa6c406ae70 0x7fa6c406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.626+0000 7fa6e5514640 1 --2- 172.21.15.76:0/2648654212 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa6c406ae70 0x7fa6c406d330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc036000 tx=0x7fa6dc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:31.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.630+0000 7fa6d5ffb640 1 -- 172.21.15.76:0/2648654212 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa6d0054e60 con 0x7fa6e0151c70 2023-11-09T03:46:31.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.811+0000 7fa6e7778640 1 -- 172.21.15.76:0/2648654212 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa6e00ccac0 con 0x7fa6e0151c70 2023-11-09T03:46:31.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.812+0000 7fa6d5ffb640 1 -- 172.21.15.76:0/2648654212 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa6d0058480 con 0x7fa6e0151c70 2023-11-09T03:46:31.811 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2023-11-09T03:46:31.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.814+0000 7fa6e7778640 1 -- 172.21.15.76:0/2648654212 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa6c406ae70 msgr2=0x7fa6c406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:31.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.814+0000 7fa6e7778640 1 --2- 172.21.15.76:0/2648654212 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa6c406ae70 0x7fa6c406d330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc036000 tx=0x7fa6dc02e000 comp rx=0 tx=0).stop 2023-11-09T03:46:31.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.814+0000 7fa6e7778640 1 -- 172.21.15.76:0/2648654212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6e0151c70 msgr2=0x7fa6e014ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:31.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.814+0000 7fa6e7778640 1 --2- 172.21.15.76:0/2648654212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6e0151c70 0x7fa6e014ff50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0011660 tx=0x7fa6d0011b20 comp rx=0 tx=0).stop 2023-11-09T03:46:31.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.814+0000 7fa6e7778640 1 -- 172.21.15.76:0/2648654212 shutdown_connections 2023-11-09T03:46:31.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.814+0000 7fa6e7778640 1 --2- 172.21.15.76:0/2648654212 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa6c406ae70 0x7fa6c406d330 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.814+0000 7fa6e7778640 1 --2- 172.21.15.76:0/2648654212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6e013d470 0x7fa6e013d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.814+0000 7fa6e7778640 1 --2- 172.21.15.76:0/2648654212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6e0151c70 0x7fa6e014ff50 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.814+0000 7fa6e7778640 1 --2- 172.21.15.76:0/2648654212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6cc003970 0x7fa6e014fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.814+0000 7fa6e7778640 1 -- 172.21.15.76:0/2648654212 >> 172.21.15.76:0/2648654212 conn(0x7fa6e00cbc90 msgr2=0x7fa6e015daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:31.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.814+0000 7fa6e7778640 1 -- 172.21.15.76:0/2648654212 shutdown_connections 2023-11-09T03:46:31.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.814+0000 7fa6e7778640 1 -- 172.21.15.76:0/2648654212 wait complete. 2023-11-09T03:46:31.825 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:46:31.826 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:46:31.842 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:46:31.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.982+0000 7f44c71ce640 1 Processor -- start 2023-11-09T03:46:31.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.982+0000 7f44c71ce640 1 -- start start 2023-11-09T03:46:31.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.983+0000 7f44c71ce640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f44c00cda60 0x7f44c00ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.983+0000 7f44c71ce640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f44c0070d40 0x7f44c006c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.983+0000 7f44c71ce640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44c00cc990 con 0x7f44c00b0cd0 2023-11-09T03:46:31.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.983+0000 7f44c71ce640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44c00ccc70 con 0x7f44c00cda60 2023-11-09T03:46:31.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.983+0000 7f44c71ce640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c00c9ce0 con 0x7f44c0070d40 2023-11-09T03:46:31.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.983+0000 7f44c4f6a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f44c00cda60 0x7f44c00ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.983+0000 7f44c4f6a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f44c00cda60 0x7f44c00ca7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36170/0 (socket says 172.21.15.76:36170) 2023-11-09T03:46:31.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.983+0000 7f44c4f6a640 1 -- 172.21.15.76:0/2343553816 learned_addr learned my addr 172.21.15.76:0/2343553816 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:31.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.983+0000 7f44bffff640 1 --1- 172.21.15.76:0/2343553816 >> v1:172.21.15.76:6789/0 conn(0x7f44c00b0cd0 0x7f44c0070630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53132/0 (socket says 172.21.15.76:53132) 2023-11-09T03:46:31.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.983+0000 7f44c4f6a640 1 -- 172.21.15.76:0/2343553816 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f44c0070d40 msgr2=0x7f44c006c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:31.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.983+0000 7f44c4f6a640 1 --2- 172.21.15.76:0/2343553816 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f44c0070d40 0x7f44c006c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.983+0000 7f44c4f6a640 1 -- 172.21.15.76:0/2343553816 >> v1:172.21.15.76:6789/0 conn(0x7f44c00b0cd0 legacy=0x7f44c0070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:31.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.983+0000 7f44c4f6a640 1 -- 172.21.15.76:0/2343553816 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c00c9fc0 con 0x7f44c00cda60 2023-11-09T03:46:31.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.984+0000 7f44c4f6a640 1 --2- 172.21.15.76:0/2343553816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44c00cda60 0x7f44c00ca7a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f44b000bb00 tx=0x7f44b002df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4afaaf597432cfe1 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:31.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.984+0000 7f44bf7fe640 1 -- 172.21.15.76:0/2343553816 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b002ece0 con 0x7f44c00cda60 2023-11-09T03:46:31.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.984+0000 7f44bf7fe640 1 -- 172.21.15.76:0/2343553816 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b0032070 con 0x7f44c00cda60 2023-11-09T03:46:31.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.984+0000 7f44bf7fe640 1 -- 172.21.15.76:0/2343553816 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b0038620 con 0x7f44c00cda60 2023-11-09T03:46:31.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.985+0000 7f44c71ce640 1 -- 172.21.15.76:0/2343553816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44c00cda60 msgr2=0x7f44c00ca7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:31.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.985+0000 7f44c71ce640 1 --2- 172.21.15.76:0/2343553816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44c00cda60 0x7f44c00ca7a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f44b000bb00 tx=0x7f44b002df30 comp rx=0 tx=0).stop 2023-11-09T03:46:31.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.985+0000 7f44c71ce640 1 -- 172.21.15.76:0/2343553816 shutdown_connections 2023-11-09T03:46:31.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.985+0000 7f44c71ce640 1 --2- 172.21.15.76:0/2343553816 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f44c0070d40 0x7f44c006c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.985+0000 7f44c71ce640 1 --2- 172.21.15.76:0/2343553816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44c00cda60 0x7f44c00ca7a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.985+0000 7f44c71ce640 1 -- 172.21.15.76:0/2343553816 >> 172.21.15.76:0/2343553816 conn(0x7f44c00ce950 msgr2=0x7f44c00ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:31.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.985+0000 7f44c71ce640 1 -- 172.21.15.76:0/2343553816 shutdown_connections 2023-11-09T03:46:31.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.985+0000 7f44c71ce640 1 -- 172.21.15.76:0/2343553816 wait complete. 2023-11-09T03:46:31.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.986+0000 7f44c71ce640 1 Processor -- start 2023-11-09T03:46:31.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.986+0000 7f44c71ce640 1 -- start start 2023-11-09T03:46:31.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.987+0000 7f44c71ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44c0070d40 0x7f44c00c5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.987+0000 7f44c71ce640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f44c00b0cd0 0x7f44c00c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.987+0000 7f44c71ce640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44c00cda60 0x7f44c0157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.987+0000 7f44c71ce640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c00c8ca0 con 0x7f44c00cda60 2023-11-09T03:46:31.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.987+0000 7f44c71ce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c00c7130 con 0x7f44c0070d40 2023-11-09T03:46:31.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.987+0000 7f44c71ce640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c00c6090 con 0x7f44c00b0cd0 2023-11-09T03:46:31.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.987+0000 7f44c576b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44c00cda60 0x7f44c0157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.987+0000 7f44c576b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44c00cda60 0x7f44c0157650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36182/0 (socket says 172.21.15.76:36182) 2023-11-09T03:46:31.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.987+0000 7f44c576b640 1 -- 172.21.15.76:0/4098303050 learned_addr learned my addr 172.21.15.76:0/4098303050 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:31.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.987+0000 7f44c4f6a640 1 --2- 172.21.15.76:0/4098303050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44c0070d40 0x7f44c00c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.987+0000 7f44bffff640 1 --2- 172.21.15.76:0/4098303050 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f44c00b0cd0 0x7f44c00c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.987+0000 7f44c576b640 1 -- 172.21.15.76:0/4098303050 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f44c00b0cd0 msgr2=0x7f44c00c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:31.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.987+0000 7f44c576b640 1 --2- 172.21.15.76:0/4098303050 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f44c00b0cd0 0x7f44c00c8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.987+0000 7f44c576b640 1 -- 172.21.15.76:0/4098303050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44c0070d40 msgr2=0x7f44c00c5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:31.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.987+0000 7f44c576b640 1 --2- 172.21.15.76:0/4098303050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44c0070d40 0x7f44c00c5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:31.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.987+0000 7f44c576b640 1 -- 172.21.15.76:0/4098303050 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b000b760 con 0x7f44c00cda60 2023-11-09T03:46:31.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.987+0000 7f44bffff640 1 --2- 172.21.15.76:0/4098303050 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f44c00b0cd0 0x7f44c00c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:31.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.987+0000 7f44c576b640 1 --2- 172.21.15.76:0/4098303050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44c00cda60 0x7f44c0157650 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f44b800b970 tx=0x7f44b800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:31.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.988+0000 7f44bd7fa640 1 -- 172.21.15.76:0/4098303050 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b800cb40 con 0x7f44c00cda60 2023-11-09T03:46:31.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.988+0000 7f44bd7fa640 1 -- 172.21.15.76:0/4098303050 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b800ccc0 con 0x7f44c00cda60 2023-11-09T03:46:31.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.988+0000 7f44c71ce640 1 -- 172.21.15.76:0/4098303050 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c00c6370 con 0x7f44c00cda60 2023-11-09T03:46:31.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.988+0000 7f44bd7fa640 1 -- 172.21.15.76:0/4098303050 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b8014680 con 0x7f44c00cda60 2023-11-09T03:46:31.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.988+0000 7f44c71ce640 1 -- 172.21.15.76:0/4098303050 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c00c6930 con 0x7f44c00cda60 2023-11-09T03:46:31.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.989+0000 7f44c71ce640 1 -- 172.21.15.76:0/4098303050 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44c0074290 con 0x7f44c00cda60 2023-11-09T03:46:31.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.989+0000 7f44bd7fa640 1 -- 172.21.15.76:0/4098303050 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f44b801d3b0 con 0x7f44c00cda60 2023-11-09T03:46:31.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.989+0000 7f44bd7fa640 1 --2- 172.21.15.76:0/4098303050 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f44ac06ae70 0x7f44ac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:31.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.990+0000 7f44bd7fa640 1 -- 172.21.15.76:0/4098303050 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f44b8089850 con 0x7f44c00cda60 2023-11-09T03:46:31.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.990+0000 7f44c4f6a640 1 --2- 172.21.15.76:0/4098303050 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f44ac06ae70 0x7f44ac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:31.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.990+0000 7f44c4f6a640 1 --2- 172.21.15.76:0/4098303050 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f44ac06ae70 0x7f44ac06d330 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f44b0039000 tx=0x7f44b0034000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:31.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:31.994+0000 7f44bd7fa640 1 -- 172.21.15.76:0/4098303050 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f44b8054660 con 0x7f44c00cda60 2023-11-09T03:46:32.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.159+0000 7f44c71ce640 1 -- 172.21.15.76:0/4098303050 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44c00b48d0 con 0x7f44c00cda60 2023-11-09T03:46:32.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.160+0000 7f44bd7fa640 1 -- 172.21.15.76:0/4098303050 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18710 (secure 0 0 0) 0x7f44b8019090 con 0x7f44c00cda60 2023-11-09T03:46:32.160 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:32.160 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":27,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:46:12.537265+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:46:32.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.162+0000 7f44c71ce640 1 -- 172.21.15.76:0/4098303050 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f44ac06ae70 msgr2=0x7f44ac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:32.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.162+0000 7f44c71ce640 1 --2- 172.21.15.76:0/4098303050 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f44ac06ae70 0x7f44ac06d330 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f44b0039000 tx=0x7f44b0034000 comp rx=0 tx=0).stop 2023-11-09T03:46:32.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.162+0000 7f44c71ce640 1 -- 172.21.15.76:0/4098303050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44c00cda60 msgr2=0x7f44c0157650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:32.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.162+0000 7f44c71ce640 1 --2- 172.21.15.76:0/4098303050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44c00cda60 0x7f44c0157650 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f44b800b970 tx=0x7f44b800be30 comp rx=0 tx=0).stop 2023-11-09T03:46:32.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.162+0000 7f44c71ce640 1 -- 172.21.15.76:0/4098303050 shutdown_connections 2023-11-09T03:46:32.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.162+0000 7f44c71ce640 1 --2- 172.21.15.76:0/4098303050 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f44ac06ae70 0x7f44ac06d330 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.162+0000 7f44c71ce640 1 --2- 172.21.15.76:0/4098303050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44c00cda60 0x7f44c0157650 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.162+0000 7f44c71ce640 1 --2- 172.21.15.76:0/4098303050 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f44c00b0cd0 0x7f44c00c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.162+0000 7f44c71ce640 1 --2- 172.21.15.76:0/4098303050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44c0070d40 0x7f44c00c5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.162+0000 7f44c71ce640 1 -- 172.21.15.76:0/4098303050 >> 172.21.15.76:0/4098303050 conn(0x7f44c00ce950 msgr2=0x7f44c0158bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:32.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.162+0000 7f44c71ce640 1 -- 172.21.15.76:0/4098303050 shutdown_connections 2023-11-09T03:46:32.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.162+0000 7f44c71ce640 1 -- 172.21.15.76:0/4098303050 wait complete. 2023-11-09T03:46:32.172 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:45:28.704715+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:45:29.717614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-09T03:45:33.440577+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-09T03:46:32.173 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:46:32.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.276+0000 7f1670561640 1 Processor -- start 2023-11-09T03:46:32.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.276+0000 7f1670561640 1 -- start start 2023-11-09T03:46:32.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.277+0000 7f1670561640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f16680cc520 0x7f1668070fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:32.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.277+0000 7f1670561640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f16680cc9d0 0x7f16680714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:32.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.277+0000 7f1670561640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16680c3860 con 0x7f1668153850 2023-11-09T03:46:32.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.277+0000 7f1670561640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16680c9250 con 0x7f16680cc9d0 2023-11-09T03:46:32.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.277+0000 7f1670561640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1668153c30 con 0x7f16680cc520 2023-11-09T03:46:32.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.278+0000 7f166eafe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f16680cc9d0 0x7f16680714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:32.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.278+0000 7f166eafe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f16680cc9d0 0x7f16680714e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36188/0 (socket says 172.21.15.76:36188) 2023-11-09T03:46:32.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.278+0000 7f166eafe640 1 -- 172.21.15.76:0/3687692291 learned_addr learned my addr 172.21.15.76:0/3687692291 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:32.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.278+0000 7f166e2fd640 1 --1- 172.21.15.76:0/3687692291 >> v1:172.21.15.76:6789/0 conn(0x7f1668153850 0x7f166815a1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53146/0 (socket says 172.21.15.76:53146) 2023-11-09T03:46:32.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.278+0000 7f166dafc640 1 --2- 172.21.15.76:0/3687692291 >> v2:172.21.15.113:3300/0 conn(0x7f16680cc520 0x7f1668070fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:32.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.278+0000 7f166eafe640 1 -- 172.21.15.76:0/3687692291 >> v2:172.21.15.113:3300/0 conn(0x7f16680cc520 msgr2=0x7f1668070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:32.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.278+0000 7f166eafe640 1 --2- 172.21.15.76:0/3687692291 >> v2:172.21.15.113:3300/0 conn(0x7f16680cc520 0x7f1668070fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.278+0000 7f166eafe640 1 -- 172.21.15.76:0/3687692291 >> v1:172.21.15.76:6789/0 conn(0x7f1668153850 legacy=0x7f166815a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:32.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.278+0000 7f166eafe640 1 -- 172.21.15.76:0/3687692291 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16680ce590 con 0x7f16680cc9d0 2023-11-09T03:46:32.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.279+0000 7f166eafe640 1 --2- 172.21.15.76:0/3687692291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16680cc9d0 0x7f16680714e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1668070560 tx=0x7f166002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c8a326d57d7ff1 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:32.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.279+0000 7f166d2fb640 1 -- 172.21.15.76:0/3687692291 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1660030860 con 0x7f16680cc9d0 2023-11-09T03:46:32.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.279+0000 7f166d2fb640 1 -- 172.21.15.76:0/3687692291 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16600309c0 con 0x7f16680cc9d0 2023-11-09T03:46:32.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.279+0000 7f166d2fb640 1 -- 172.21.15.76:0/3687692291 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1660030d50 con 0x7f16680cc9d0 2023-11-09T03:46:32.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.279+0000 7f1670561640 1 -- 172.21.15.76:0/3687692291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16680cc9d0 msgr2=0x7f16680714e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:32.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.279+0000 7f1670561640 1 --2- 172.21.15.76:0/3687692291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16680cc9d0 0x7f16680714e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1668070560 tx=0x7f166002b9a0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.279+0000 7f1670561640 1 -- 172.21.15.76:0/3687692291 shutdown_connections 2023-11-09T03:46:32.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.279+0000 7f1670561640 1 --2- 172.21.15.76:0/3687692291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16680cc9d0 0x7f16680714e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.279+0000 7f1670561640 1 --2- 172.21.15.76:0/3687692291 >> v2:172.21.15.113:3300/0 conn(0x7f16680cc520 0x7f1668070fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.279+0000 7f1670561640 1 -- 172.21.15.76:0/3687692291 >> 172.21.15.76:0/3687692291 conn(0x7f16680ce950 msgr2=0x7f16680ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:32.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.279+0000 7f1670561640 1 -- 172.21.15.76:0/3687692291 shutdown_connections 2023-11-09T03:46:32.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.280+0000 7f1670561640 1 -- 172.21.15.76:0/3687692291 wait complete. 2023-11-09T03:46:32.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.281+0000 7f1670561640 1 Processor -- start 2023-11-09T03:46:32.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.281+0000 7f1670561640 1 -- start start 2023-11-09T03:46:32.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.281+0000 7f1670561640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16680cc520 0x7f16681470d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:32.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.281+0000 7f1670561640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16680cc9d0 0x7f166814d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:32.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.281+0000 7f1670561640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1668153850 0x7f166814b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:32.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.281+0000 7f1670561640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16680b2720 con 0x7f1668153850 2023-11-09T03:46:32.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.281+0000 7f1670561640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166814e4a0 con 0x7f16680cc9d0 2023-11-09T03:46:32.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.281+0000 7f1670561640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166814e1c0 con 0x7f16680cc520 2023-11-09T03:46:32.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.281+0000 7f166eafe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1668153850 0x7f166814b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:32.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.281+0000 7f166eafe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1668153850 0x7f166814b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36194/0 (socket says 172.21.15.76:36194) 2023-11-09T03:46:32.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.281+0000 7f166eafe640 1 -- 172.21.15.76:0/254950806 learned_addr learned my addr 172.21.15.76:0/254950806 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:32.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.282+0000 7f166e2fd640 1 --2- 172.21.15.76:0/254950806 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16680cc520 0x7f16681470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:32.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.282+0000 7f166dafc640 1 --2- 172.21.15.76:0/254950806 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16680cc9d0 0x7f166814d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:32.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.282+0000 7f166eafe640 1 -- 172.21.15.76:0/254950806 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16680cc520 msgr2=0x7f16681470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:32.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.282+0000 7f166eafe640 1 --2- 172.21.15.76:0/254950806 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16680cc520 0x7f16681470d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.282+0000 7f166eafe640 1 -- 172.21.15.76:0/254950806 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16680cc9d0 msgr2=0x7f166814d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:32.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.282+0000 7f166eafe640 1 --2- 172.21.15.76:0/254950806 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16680cc9d0 0x7f166814d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.282+0000 7f166eafe640 1 -- 172.21.15.76:0/254950806 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16600094d0 con 0x7f1668153850 2023-11-09T03:46:32.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.282+0000 7f166eafe640 1 --2- 172.21.15.76:0/254950806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1668153850 0x7f166814b090 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f166002b9e0 tx=0x7f1660004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:32.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.282+0000 7f165effd640 1 -- 172.21.15.76:0/254950806 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16600307f0 con 0x7f1668153850 2023-11-09T03:46:32.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.282+0000 7f165effd640 1 -- 172.21.15.76:0/254950806 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1660043400 con 0x7f1668153850 2023-11-09T03:46:32.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.282+0000 7f165effd640 1 -- 172.21.15.76:0/254950806 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16600437b0 con 0x7f1668153850 2023-11-09T03:46:32.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.282+0000 7f1670561640 1 -- 172.21.15.76:0/254950806 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16680b2950 con 0x7f1668153850 2023-11-09T03:46:32.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.282+0000 7f1670561640 1 -- 172.21.15.76:0/254950806 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16680b2f10 con 0x7f1668153850 2023-11-09T03:46:32.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.283+0000 7f1670561640 1 -- 172.21.15.76:0/254950806 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1668074290 con 0x7f1668153850 2023-11-09T03:46:32.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.283+0000 7f165effd640 1 -- 172.21.15.76:0/254950806 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1660043910 con 0x7f1668153850 2023-11-09T03:46:32.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.284+0000 7f165effd640 1 --2- 172.21.15.76:0/254950806 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f164c06ae70 0x7f164c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:32.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.284+0000 7f165effd640 1 -- 172.21.15.76:0/254950806 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f166003e070 con 0x7f1668153850 2023-11-09T03:46:32.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.284+0000 7f166e2fd640 1 --2- 172.21.15.76:0/254950806 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f164c06ae70 0x7f164c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:32.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.284+0000 7f166e2fd640 1 --2- 172.21.15.76:0/254950806 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f164c06ae70 0x7f164c06d330 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f16640012e0 tx=0x7f1664009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:32.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.288+0000 7f165effd640 1 -- 172.21.15.76:0/254950806 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1660077c80 con 0x7f1668153850 2023-11-09T03:46:32.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.459+0000 7f1670561640 1 -- 172.21.15.76:0/254950806 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f16680c5010 con 0x7f1668153850 2023-11-09T03:46:32.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.459+0000 7f165effd640 1 -- 172.21.15.76:0/254950806 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f166007b2a0 con 0x7f1668153850 2023-11-09T03:46:32.459 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:46:32.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.461+0000 7f1670561640 1 -- 172.21.15.76:0/254950806 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f164c06ae70 msgr2=0x7f164c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:32.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.461+0000 7f1670561640 1 --2- 172.21.15.76:0/254950806 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f164c06ae70 0x7f164c06d330 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f16640012e0 tx=0x7f1664009040 comp rx=0 tx=0).stop 2023-11-09T03:46:32.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.461+0000 7f1670561640 1 -- 172.21.15.76:0/254950806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1668153850 msgr2=0x7f166814b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:32.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.461+0000 7f1670561640 1 --2- 172.21.15.76:0/254950806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1668153850 0x7f166814b090 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f166002b9e0 tx=0x7f1660004730 comp rx=0 tx=0).stop 2023-11-09T03:46:32.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.461+0000 7f1670561640 1 -- 172.21.15.76:0/254950806 shutdown_connections 2023-11-09T03:46:32.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.461+0000 7f1670561640 1 --2- 172.21.15.76:0/254950806 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f164c06ae70 0x7f164c06d330 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.461+0000 7f1670561640 1 --2- 172.21.15.76:0/254950806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1668153850 0x7f166814b090 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.461+0000 7f1670561640 1 --2- 172.21.15.76:0/254950806 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16680cc9d0 0x7f166814d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.461+0000 7f1670561640 1 --2- 172.21.15.76:0/254950806 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16680cc520 0x7f16681470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.461+0000 7f1670561640 1 -- 172.21.15.76:0/254950806 >> 172.21.15.76:0/254950806 conn(0x7f16680ce950 msgr2=0x7f16680c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:32.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.461+0000 7f1670561640 1 -- 172.21.15.76:0/254950806 shutdown_connections 2023-11-09T03:46:32.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.461+0000 7f1670561640 1 -- 172.21.15.76:0/254950806 wait complete. 2023-11-09T03:46:32.471 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:46:32.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.575+0000 7f197c483640 1 Processor -- start 2023-11-09T03:46:32.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.575+0000 7f197c483640 1 -- start start 2023-11-09T03:46:32.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.576+0000 7f197c483640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f197415d1b0 0x7f19740cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:32.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.576+0000 7f197c483640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1974066000 0x7f197406a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:32.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.576+0000 7f197c483640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19740c9bf0 con 0x7f1974070ff0 2023-11-09T03:46:32.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.576+0000 7f197c483640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19740cd890 con 0x7f1974066000 2023-11-09T03:46:32.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.576+0000 7f197c483640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19740cdb30 con 0x7f197415d1b0 2023-11-09T03:46:32.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.576+0000 7f197a21f640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f1974070ff0 0x7f19740713d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40636/0 (socket says 172.21.15.76:40636) 2023-11-09T03:46:32.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.576+0000 7f197a21f640 1 -- 172.21.15.76:0/2341991651 learned_addr learned my addr 172.21.15.76:0/2341991651 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:32.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.576+0000 7f197aa20640 1 --2- 172.21.15.76:0/2341991651 >> v2:172.21.15.76:3300/0 conn(0x7f1974066000 0x7f197406a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:32.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.576+0000 7f1979a1e640 1 --2- 172.21.15.76:0/2341991651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f197415d1b0 0x7f19740cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:32.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.576+0000 7f197aa20640 1 -- 172.21.15.76:0/2341991651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f197415d1b0 msgr2=0x7f19740cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:32.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.576+0000 7f197aa20640 1 --2- 172.21.15.76:0/2341991651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f197415d1b0 0x7f19740cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.576+0000 7f197aa20640 1 -- 172.21.15.76:0/2341991651 >> v1:172.21.15.76:6789/0 conn(0x7f1974070ff0 legacy=0x7f19740713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:32.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.576+0000 7f197aa20640 1 -- 172.21.15.76:0/2341991651 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19740cde10 con 0x7f1974066000 2023-11-09T03:46:32.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.577+0000 7f197aa20640 1 --2- 172.21.15.76:0/2341991651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1974066000 0x7f197406a500 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f196c009d90 tx=0x7f196c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ffff8927c54b8b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:32.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.577+0000 7f197921d640 1 -- 172.21.15.76:0/2341991651 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c02cc50 con 0x7f1974066000 2023-11-09T03:46:32.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.577+0000 7f197921d640 1 -- 172.21.15.76:0/2341991651 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196c02cdb0 con 0x7f1974066000 2023-11-09T03:46:32.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.577+0000 7f197921d640 1 -- 172.21.15.76:0/2341991651 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c034660 con 0x7f1974066000 2023-11-09T03:46:32.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.577+0000 7f197c483640 1 -- 172.21.15.76:0/2341991651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1974066000 msgr2=0x7f197406a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:32.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.577+0000 7f197c483640 1 --2- 172.21.15.76:0/2341991651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1974066000 0x7f197406a500 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f196c009d90 tx=0x7f196c02bea0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.578+0000 7f197c483640 1 -- 172.21.15.76:0/2341991651 shutdown_connections 2023-11-09T03:46:32.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.578+0000 7f197c483640 1 --2- 172.21.15.76:0/2341991651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1974066000 0x7f197406a500 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.578+0000 7f197c483640 1 --2- 172.21.15.76:0/2341991651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f197415d1b0 0x7f19740cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.578+0000 7f197c483640 1 -- 172.21.15.76:0/2341991651 >> 172.21.15.76:0/2341991651 conn(0x7f19740cbc90 msgr2=0x7f19740cc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:32.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.578+0000 7f197c483640 1 -- 172.21.15.76:0/2341991651 shutdown_connections 2023-11-09T03:46:32.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.578+0000 7f197c483640 1 -- 172.21.15.76:0/2341991651 wait complete. 2023-11-09T03:46:32.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.579+0000 7f197c483640 1 Processor -- start 2023-11-09T03:46:32.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.579+0000 7f197c483640 1 -- start start 2023-11-09T03:46:32.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.579+0000 7f197c483640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1974066000 0x7f19740b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:32.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.579+0000 7f197c483640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1974070ff0 0x7f19741455a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:32.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.579+0000 7f197c483640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f197415d1b0 0x7f19740c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:32.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.579+0000 7f197c483640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19741561e0 con 0x7f197415d1b0 2023-11-09T03:46:32.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.579+0000 7f197c483640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1974145ae0 con 0x7f1974066000 2023-11-09T03:46:32.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.579+0000 7f197c483640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19740b3640 con 0x7f1974070ff0 2023-11-09T03:46:32.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.580+0000 7f197aa20640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f197415d1b0 0x7f19740c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:32.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.580+0000 7f197aa20640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f197415d1b0 0x7f19740c4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34870/0 (socket says 172.21.15.76:34870) 2023-11-09T03:46:32.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.580+0000 7f197aa20640 1 -- 172.21.15.76:0/3694260873 learned_addr learned my addr 172.21.15.76:0/3694260873 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:32.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.580+0000 7f1979a1e640 1 --2- 172.21.15.76:0/3694260873 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1974070ff0 0x7f19741455a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:32.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.580+0000 7f197a21f640 1 --2- 172.21.15.76:0/3694260873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1974066000 0x7f19740b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:32.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.580+0000 7f197aa20640 1 -- 172.21.15.76:0/3694260873 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1974070ff0 msgr2=0x7f19741455a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:32.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.580+0000 7f197aa20640 1 --2- 172.21.15.76:0/3694260873 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1974070ff0 0x7f19741455a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.580+0000 7f197aa20640 1 -- 172.21.15.76:0/3694260873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1974066000 msgr2=0x7f19740b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:32.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.580+0000 7f197aa20640 1 --2- 172.21.15.76:0/3694260873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1974066000 0x7f19740b2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.580+0000 7f197aa20640 1 -- 172.21.15.76:0/3694260873 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c009a70 con 0x7f197415d1b0 2023-11-09T03:46:32.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.580+0000 7f197aa20640 1 --2- 172.21.15.76:0/3694260873 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f197415d1b0 0x7f19740c4b40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f196c009eb0 tx=0x7f196c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:32.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.580+0000 7f196affd640 1 -- 172.21.15.76:0/3694260873 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c040650 con 0x7f197415d1b0 2023-11-09T03:46:32.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.580+0000 7f196affd640 1 -- 172.21.15.76:0/3694260873 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196c0407b0 con 0x7f197415d1b0 2023-11-09T03:46:32.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.580+0000 7f197c483640 1 -- 172.21.15.76:0/3694260873 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19740b3920 con 0x7f197415d1b0 2023-11-09T03:46:32.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.580+0000 7f196affd640 1 -- 172.21.15.76:0/3694260873 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c040b60 con 0x7f197415d1b0 2023-11-09T03:46:32.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.580+0000 7f197c483640 1 -- 172.21.15.76:0/3694260873 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19740b3ee0 con 0x7f197415d1b0 2023-11-09T03:46:32.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.581+0000 7f197c483640 1 -- 172.21.15.76:0/3694260873 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1974074290 con 0x7f197415d1b0 2023-11-09T03:46:32.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.582+0000 7f196affd640 1 -- 172.21.15.76:0/3694260873 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f196c040cc0 con 0x7f197415d1b0 2023-11-09T03:46:32.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.582+0000 7f196affd640 1 --2- 172.21.15.76:0/3694260873 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f195806aea0 0x7f195806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:32.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.582+0000 7f196affd640 1 -- 172.21.15.76:0/3694260873 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f196c0ac700 con 0x7f197415d1b0 2023-11-09T03:46:32.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.582+0000 7f197a21f640 1 --2- 172.21.15.76:0/3694260873 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f195806aea0 0x7f195806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:32.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.583+0000 7f197a21f640 1 --2- 172.21.15.76:0/3694260873 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f195806aea0 0x7f195806d360 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1970005880 tx=0x7f1970009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:32.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.586+0000 7f196affd640 1 -- 172.21.15.76:0/3694260873 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f196c0773d0 con 0x7f197415d1b0 2023-11-09T03:46:32.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.751+0000 7f197c483640 1 -- 172.21.15.76:0/3694260873 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f19740b5190 con 0x7f197415d1b0 2023-11-09T03:46:32.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.751+0000 7f196affd640 1 -- 172.21.15.76:0/3694260873 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f196c009450 con 0x7f197415d1b0 2023-11-09T03:46:32.752 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:46:32.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.753+0000 7f197c483640 1 -- 172.21.15.76:0/3694260873 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f195806aea0 msgr2=0x7f195806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:32.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.753+0000 7f197c483640 1 --2- 172.21.15.76:0/3694260873 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f195806aea0 0x7f195806d360 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1970005880 tx=0x7f1970009040 comp rx=0 tx=0).stop 2023-11-09T03:46:32.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.753+0000 7f197c483640 1 -- 172.21.15.76:0/3694260873 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f197415d1b0 msgr2=0x7f19740c4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:32.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.753+0000 7f197c483640 1 --2- 172.21.15.76:0/3694260873 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f197415d1b0 0x7f19740c4b40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f196c009eb0 tx=0x7f196c004a40 comp rx=0 tx=0).stop 2023-11-09T03:46:32.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.753+0000 7f197c483640 1 -- 172.21.15.76:0/3694260873 shutdown_connections 2023-11-09T03:46:32.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.753+0000 7f197c483640 1 --2- 172.21.15.76:0/3694260873 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f195806aea0 0x7f195806d360 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.753+0000 7f197c483640 1 --2- 172.21.15.76:0/3694260873 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f197415d1b0 0x7f19740c4b40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.753+0000 7f197c483640 1 --2- 172.21.15.76:0/3694260873 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1974070ff0 0x7f19741455a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.753+0000 7f197c483640 1 --2- 172.21.15.76:0/3694260873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1974066000 0x7f19740b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.753+0000 7f197c483640 1 -- 172.21.15.76:0/3694260873 >> 172.21.15.76:0/3694260873 conn(0x7f19740cbc90 msgr2=0x7f197406b120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:32.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.753+0000 7f197c483640 1 -- 172.21.15.76:0/3694260873 shutdown_connections 2023-11-09T03:46:32.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.753+0000 7f197c483640 1 -- 172.21.15.76:0/3694260873 wait complete. 2023-11-09T03:46:32.764 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:46:32.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.866+0000 7f2247325640 1 Processor -- start 2023-11-09T03:46:32.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.867+0000 7f2247325640 1 -- start start 2023-11-09T03:46:32.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.867+0000 7f2247325640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f22400d0710 0x7f22400cd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:32.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.867+0000 7f2247325640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22400cda50 0x7f22400b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:32.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.867+0000 7f2247325640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22400d0280 con 0x7f224015e430 2023-11-09T03:46:32.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.867+0000 7f2247325640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22400cf670 con 0x7f22400d0710 2023-11-09T03:46:32.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.867+0000 7f2247325640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22400cf950 con 0x7f22400cda50 2023-11-09T03:46:32.869 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.868+0000 7f22458c2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22400cda50 0x7f22400b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:32.869 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.868+0000 7f22448c0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f22400d0710 0x7f22400cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:32.869 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.868+0000 7f22450c1640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f224015e430 0x7f22400cec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:56028/0 (socket says 172.21.15.76:56028) 2023-11-09T03:46:32.869 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.868+0000 7f22450c1640 1 -- 172.21.15.76:0/3357655438 learned_addr learned my addr 172.21.15.76:0/3357655438 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:32.871 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.868+0000 7f22448c0640 1 -- 172.21.15.76:0/3357655438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22400cda50 msgr2=0x7f22400b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:32.871 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.868+0000 7f22448c0640 1 --2- 172.21.15.76:0/3357655438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22400cda50 0x7f22400b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.871 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.868+0000 7f22448c0640 1 -- 172.21.15.76:0/3357655438 >> v1:172.21.15.113:6789/0 conn(0x7f224015e430 legacy=0x7f22400cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:32.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.868+0000 7f22448c0640 1 -- 172.21.15.76:0/3357655438 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22400cfc30 con 0x7f22400d0710 2023-11-09T03:46:32.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.868+0000 7f2237fff640 1 -- 172.21.15.76:0/3357655438 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 965193747 0 0) 0x7f22400d0280 con 0x7f224015e430 2023-11-09T03:46:32.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.868+0000 7f22458c2640 1 --2- 172.21.15.76:0/3357655438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22400cda50 0x7f22400b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:46:32.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.869+0000 7f22448c0640 1 --2- 172.21.15.76:0/3357655438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22400d0710 0x7f22400cd510 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f22300097a0 tx=0x7f223002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb92724ba8d5c228 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:32.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.869+0000 7f2237fff640 1 -- 172.21.15.76:0/3357655438 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2230031830 con 0x7f22400d0710 2023-11-09T03:46:32.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.869+0000 7f2237fff640 1 -- 172.21.15.76:0/3357655438 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2230031990 con 0x7f22400d0710 2023-11-09T03:46:32.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.869+0000 7f2237fff640 1 -- 172.21.15.76:0/3357655438 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2230031d20 con 0x7f22400d0710 2023-11-09T03:46:32.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.869+0000 7f2247325640 1 -- 172.21.15.76:0/3357655438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22400d0710 msgr2=0x7f22400cd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:32.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.869+0000 7f2247325640 1 --2- 172.21.15.76:0/3357655438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22400d0710 0x7f22400cd510 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f22300097a0 tx=0x7f223002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.869+0000 7f2247325640 1 -- 172.21.15.76:0/3357655438 shutdown_connections 2023-11-09T03:46:32.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.869+0000 7f2247325640 1 --2- 172.21.15.76:0/3357655438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22400cda50 0x7f22400b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.869+0000 7f2247325640 1 --2- 172.21.15.76:0/3357655438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22400d0710 0x7f22400cd510 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.869+0000 7f2247325640 1 -- 172.21.15.76:0/3357655438 >> 172.21.15.76:0/3357655438 conn(0x7f224006ea20 msgr2=0x7f224006ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:32.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.870+0000 7f2247325640 1 -- 172.21.15.76:0/3357655438 shutdown_connections 2023-11-09T03:46:32.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.870+0000 7f2247325640 1 -- 172.21.15.76:0/3357655438 wait complete. 2023-11-09T03:46:32.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.871+0000 7f2247325640 1 Processor -- start 2023-11-09T03:46:32.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.871+0000 7f2247325640 1 -- start start 2023-11-09T03:46:32.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.871+0000 7f2247325640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22400cda50 0x7f2240157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:32.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.871+0000 7f2247325640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22400d0710 0x7f22400c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:32.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.871+0000 7f2247325640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f224015e430 0x7f2240156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:32.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.871+0000 7f2247325640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22401579b0 con 0x7f22400d0710 2023-11-09T03:46:32.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.871+0000 7f2247325640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22400c69f0 con 0x7f22400cda50 2023-11-09T03:46:32.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.871+0000 7f2247325640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2240158f80 con 0x7f224015e430 2023-11-09T03:46:32.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.872+0000 7f22448c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22400d0710 0x7f22400c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:32.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.872+0000 7f22448c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22400d0710 0x7f22400c61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34886/0 (socket says 172.21.15.76:34886) 2023-11-09T03:46:32.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.872+0000 7f22448c0640 1 -- 172.21.15.76:0/1681650305 learned_addr learned my addr 172.21.15.76:0/1681650305 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:32.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.872+0000 7f22458c2640 1 --2- 172.21.15.76:0/1681650305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f224015e430 0x7f2240156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:32.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.872+0000 7f22450c1640 1 --2- 172.21.15.76:0/1681650305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22400cda50 0x7f2240157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:32.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.872+0000 7f22448c0640 1 -- 172.21.15.76:0/1681650305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f224015e430 msgr2=0x7f2240156070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:32.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.872+0000 7f22448c0640 1 --2- 172.21.15.76:0/1681650305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f224015e430 0x7f2240156070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.872+0000 7f22448c0640 1 -- 172.21.15.76:0/1681650305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22400cda50 msgr2=0x7f2240157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:32.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.872+0000 7f22448c0640 1 --2- 172.21.15.76:0/1681650305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22400cda50 0x7f2240157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:32.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.872+0000 7f22448c0640 1 -- 172.21.15.76:0/1681650305 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2238009400 con 0x7f22400d0710 2023-11-09T03:46:32.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.872+0000 7f22448c0640 1 --2- 172.21.15.76:0/1681650305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22400d0710 0x7f22400c61f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f22300097a0 tx=0x7f22300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:32.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.872+0000 7f2235ffb640 1 -- 172.21.15.76:0/1681650305 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2230004230 con 0x7f22400d0710 2023-11-09T03:46:32.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.872+0000 7f2235ffb640 1 -- 172.21.15.76:0/1681650305 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2230004390 con 0x7f22400d0710 2023-11-09T03:46:32.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.872+0000 7f2235ffb640 1 -- 172.21.15.76:0/1681650305 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223003a870 con 0x7f22400d0710 2023-11-09T03:46:32.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.872+0000 7f2247325640 1 -- 172.21.15.76:0/1681650305 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2230009400 con 0x7f22400d0710 2023-11-09T03:46:32.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.872+0000 7f2247325640 1 -- 172.21.15.76:0/1681650305 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2240158650 con 0x7f22400d0710 2023-11-09T03:46:32.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.873+0000 7f2247325640 1 -- 172.21.15.76:0/1681650305 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2240074290 con 0x7f22400d0710 2023-11-09T03:46:32.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.874+0000 7f2235ffb640 1 -- 172.21.15.76:0/1681650305 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f223003a9d0 con 0x7f22400d0710 2023-11-09T03:46:32.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.874+0000 7f2235ffb640 1 --2- 172.21.15.76:0/1681650305 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f222406ae70 0x7f222406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:32.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.874+0000 7f2235ffb640 1 -- 172.21.15.76:0/1681650305 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f22300ab900 con 0x7f22400d0710 2023-11-09T03:46:32.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.874+0000 7f22450c1640 1 --2- 172.21.15.76:0/1681650305 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f222406ae70 0x7f222406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:32.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.875+0000 7f22450c1640 1 --2- 172.21.15.76:0/1681650305 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f222406ae70 0x7f222406d330 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f223c0012e0 tx=0x7f223c001700 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:32.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:32.878+0000 7f2235ffb640 1 -- 172.21.15.76:0/1681650305 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2230076780 con 0x7f22400d0710 2023-11-09T03:46:33.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.045+0000 7f2247325640 1 -- 172.21.15.76:0/1681650305 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f22401590c0 con 0x7f22400d0710 2023-11-09T03:46:33.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.045+0000 7f2235ffb640 1 -- 172.21.15.76:0/1681650305 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f22300350a0 con 0x7f22400d0710 2023-11-09T03:46:33.046 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:46:33.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.047+0000 7f2247325640 1 -- 172.21.15.76:0/1681650305 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f222406ae70 msgr2=0x7f222406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:33.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.047+0000 7f2247325640 1 --2- 172.21.15.76:0/1681650305 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f222406ae70 0x7f222406d330 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f223c0012e0 tx=0x7f223c001700 comp rx=0 tx=0).stop 2023-11-09T03:46:33.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.047+0000 7f2247325640 1 -- 172.21.15.76:0/1681650305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22400d0710 msgr2=0x7f22400c61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:33.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.047+0000 7f2247325640 1 --2- 172.21.15.76:0/1681650305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22400d0710 0x7f22400c61f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f22300097a0 tx=0x7f22300046c0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.047+0000 7f2247325640 1 -- 172.21.15.76:0/1681650305 shutdown_connections 2023-11-09T03:46:33.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.047+0000 7f2247325640 1 --2- 172.21.15.76:0/1681650305 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f222406ae70 0x7f222406d330 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.047+0000 7f2247325640 1 --2- 172.21.15.76:0/1681650305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f224015e430 0x7f2240156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.047+0000 7f2247325640 1 --2- 172.21.15.76:0/1681650305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22400d0710 0x7f22400c61f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.047+0000 7f2247325640 1 --2- 172.21.15.76:0/1681650305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22400cda50 0x7f2240157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.047+0000 7f2247325640 1 -- 172.21.15.76:0/1681650305 >> 172.21.15.76:0/1681650305 conn(0x7f224006ea20 msgr2=0x7f22400cb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:33.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.048+0000 7f2247325640 1 -- 172.21.15.76:0/1681650305 shutdown_connections 2023-11-09T03:46:33.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.048+0000 7f2247325640 1 -- 172.21.15.76:0/1681650305 wait complete. 2023-11-09T03:46:33.059 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-09T03:46:33.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.161+0000 7fc817ee1640 1 Processor -- start 2023-11-09T03:46:33.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.161+0000 7fc817ee1640 1 -- start start 2023-11-09T03:46:33.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.162+0000 7fc817ee1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8100b4880 0x7fc8100b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:33.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.162+0000 7fc817ee1640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8100cde60 con 0x7fc8100cd6d0 2023-11-09T03:46:33.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.162+0000 7fc817ee1640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc810160f90 con 0x7fc810153e60 2023-11-09T03:46:33.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.162+0000 7fc817ee1640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8100b5200 con 0x7fc8100b4880 2023-11-09T03:46:33.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.162+0000 7fc81547c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc8100cd6d0 0x7fc810161420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40638/0 (socket says 172.21.15.76:40638) 2023-11-09T03:46:33.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.162+0000 7fc81547c640 1 -- 172.21.15.76:0/2347801268 learned_addr learned my addr 172.21.15.76:0/2347801268 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:33.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.163+0000 7fc81647e640 1 --2- 172.21.15.76:0/2347801268 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8100b4880 0x7fc8100b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:33.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.163+0000 7fc814c7b640 1 -- 172.21.15.76:0/2347801268 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 929190130 0 0) 0x7fc8100cde60 con 0x7fc8100cd6d0 2023-11-09T03:46:33.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.163+0000 7fc814c7b640 1 -- 172.21.15.76:0/2347801268 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7fc003520 con 0x7fc8100cd6d0 2023-11-09T03:46:33.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.163+0000 7fc814c7b640 1 -- 172.21.15.76:0/2347801268 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3153044554 0 0) 0x7fc7fc003520 con 0x7fc8100cd6d0 2023-11-09T03:46:33.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.163+0000 7fc814c7b640 1 -- 172.21.15.76:0/2347801268 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8100b4880 msgr2=0x7fc8100b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:33.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.163+0000 7fc814c7b640 1 --2- 172.21.15.76:0/2347801268 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8100b4880 0x7fc8100b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.163+0000 7fc814c7b640 1 -- 172.21.15.76:0/2347801268 >> v1:172.21.15.113:6789/0 conn(0x7fc810153e60 legacy=0x7fc81015dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:33.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.163+0000 7fc814c7b640 1 -- 172.21.15.76:0/2347801268 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8100ce090 con 0x7fc8100cd6d0 2023-11-09T03:46:33.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.163+0000 7fc814c7b640 1 -- 172.21.15.76:0/2347801268 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fc800001670 con 0x7fc8100cd6d0 2023-11-09T03:46:33.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.163+0000 7fc814c7b640 1 -- 172.21.15.76:0/2347801268 >> v1:172.21.15.76:6789/0 conn(0x7fc8100cd6d0 legacy=0x7fc810161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:33.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.163+0000 7fc814c7b640 1 --2- 172.21.15.76:0/2347801268 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7fc0039a0 0x7fc7fc003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:33.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.163+0000 7fc814c7b640 1 --2- 172.21.15.76:0/2347801268 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8100b4880 0x7fc7fc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:33.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.163+0000 7fc814c7b640 1 --2- 172.21.15.76:0/2347801268 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7fc004d00 0x7fc7fc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:33.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.163+0000 7fc814c7b640 1 -- 172.21.15.76:0/2347801268 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8100b5200 con 0x7fc7fc0039a0 2023-11-09T03:46:33.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.163+0000 7fc814c7b640 1 -- 172.21.15.76:0/2347801268 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7fc00b9b0 con 0x7fc7fc004d00 2023-11-09T03:46:33.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.163+0000 7fc814c7b640 1 -- 172.21.15.76:0/2347801268 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7fc00bb70 con 0x7fc8100b4880 2023-11-09T03:46:33.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.163+0000 7fc815c7d640 1 --2- 172.21.15.76:0/2347801268 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7fc0039a0 0x7fc7fc003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:33.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.163+0000 7fc81647e640 1 --2- 172.21.15.76:0/2347801268 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7fc004d00 0x7fc7fc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:33.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.163+0000 7fc81547c640 1 --2- 172.21.15.76:0/2347801268 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8100b4880 0x7fc7fc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:33.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.163+0000 7fc815c7d640 1 -- 172.21.15.76:0/2347801268 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8100b4880 msgr2=0x7fc7fc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:33.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.163+0000 7fc815c7d640 1 --2- 172.21.15.76:0/2347801268 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8100b4880 0x7fc7fc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.163+0000 7fc815c7d640 1 -- 172.21.15.76:0/2347801268 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7fc004d00 msgr2=0x7fc7fc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:33.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.163+0000 7fc815c7d640 1 --2- 172.21.15.76:0/2347801268 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7fc004d00 0x7fc7fc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.163+0000 7fc815c7d640 1 -- 172.21.15.76:0/2347801268 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7fc003520 con 0x7fc7fc0039a0 2023-11-09T03:46:33.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.164+0000 7fc815c7d640 1 --2- 172.21.15.76:0/2347801268 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7fc0039a0 0x7fc7fc003d80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc80c001f60 tx=0x7fc7fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db2fbcf2c9a65c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:33.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.164+0000 7fc814c7b640 1 -- 172.21.15.76:0/2347801268 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c02d670 con 0x7fc7fc0039a0 2023-11-09T03:46:33.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.164+0000 7fc814c7b640 1 -- 172.21.15.76:0/2347801268 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80c02d7f0 con 0x7fc7fc0039a0 2023-11-09T03:46:33.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.164+0000 7fc814c7b640 1 -- 172.21.15.76:0/2347801268 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c02dba0 con 0x7fc7fc0039a0 2023-11-09T03:46:33.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.165+0000 7fc817ee1640 1 -- 172.21.15.76:0/2347801268 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7fc0039a0 msgr2=0x7fc7fc003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:33.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.165+0000 7fc817ee1640 1 --2- 172.21.15.76:0/2347801268 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7fc0039a0 0x7fc7fc003d80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc80c001f60 tx=0x7fc7fc0057f0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.165+0000 7fc81647e640 1 -- 172.21.15.76:0/2347801268 reap_dead start 2023-11-09T03:46:33.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.165+0000 7fc817ee1640 1 -- 172.21.15.76:0/2347801268 shutdown_connections 2023-11-09T03:46:33.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.165+0000 7fc817ee1640 1 -- 172.21.15.76:0/2347801268 >> 172.21.15.76:0/2347801268 conn(0x7fc8100cf880 msgr2=0x7fc8100cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:33.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.165+0000 7fc817ee1640 1 -- 172.21.15.76:0/2347801268 shutdown_connections 2023-11-09T03:46:33.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.165+0000 7fc817ee1640 1 -- 172.21.15.76:0/2347801268 wait complete. 2023-11-09T03:46:33.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.166+0000 7fc817ee1640 1 Processor -- start 2023-11-09T03:46:33.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.166+0000 7fc817ee1640 1 -- start start 2023-11-09T03:46:33.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.167+0000 7fc817ee1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7fc0039a0 0x7fc810155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:33.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.167+0000 7fc817ee1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8100cc100 0x7fc810150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:33.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.167+0000 7fc817ee1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8100d0da0 0x7fc8100d1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:33.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.167+0000 7fc817ee1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8100d1760 con 0x7fc8100d0da0 2023-11-09T03:46:33.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.167+0000 7fc817ee1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8100cbc70 con 0x7fc8100cc100 2023-11-09T03:46:33.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.167+0000 7fc817ee1640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8100cb060 con 0x7fc7fc0039a0 2023-11-09T03:46:33.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.167+0000 7fc81647e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8100d0da0 0x7fc8100d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:33.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.167+0000 7fc81647e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8100d0da0 0x7fc8100d1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34900/0 (socket says 172.21.15.76:34900) 2023-11-09T03:46:33.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.167+0000 7fc81647e640 1 -- 172.21.15.76:0/2722601886 learned_addr learned my addr 172.21.15.76:0/2722601886 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:33.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.167+0000 7fc815c7d640 1 --2- 172.21.15.76:0/2722601886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7fc0039a0 0x7fc810155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:33.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.167+0000 7fc81547c640 1 --2- 172.21.15.76:0/2722601886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8100cc100 0x7fc810150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:33.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.167+0000 7fc81647e640 1 -- 172.21.15.76:0/2722601886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7fc0039a0 msgr2=0x7fc810155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:33.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.167+0000 7fc81647e640 1 --2- 172.21.15.76:0/2722601886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7fc0039a0 0x7fc810155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.167+0000 7fc81647e640 1 -- 172.21.15.76:0/2722601886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8100cc100 msgr2=0x7fc810150400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:33.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.167+0000 7fc81647e640 1 --2- 172.21.15.76:0/2722601886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8100cc100 0x7fc810150400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.167+0000 7fc81647e640 1 -- 172.21.15.76:0/2722601886 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8100cb340 con 0x7fc8100d0da0 2023-11-09T03:46:33.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.167+0000 7fc815c7d640 1 --2- 172.21.15.76:0/2722601886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7fc0039a0 0x7fc810155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:33.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.167+0000 7fc81547c640 1 --2- 172.21.15.76:0/2722601886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8100cc100 0x7fc810150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:33.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.167+0000 7fc81647e640 1 --2- 172.21.15.76:0/2722601886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8100d0da0 0x7fc8100d1220 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc81006c900 tx=0x7fc808011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:33.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.167+0000 7fc8067fc640 1 -- 172.21.15.76:0/2722601886 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808004440 con 0x7fc8100d0da0 2023-11-09T03:46:33.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.167+0000 7fc8067fc640 1 -- 172.21.15.76:0/2722601886 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc808003610 con 0x7fc8100d0da0 2023-11-09T03:46:33.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.167+0000 7fc817ee1640 1 -- 172.21.15.76:0/2722601886 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8100cb620 con 0x7fc8100d0da0 2023-11-09T03:46:33.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.167+0000 7fc8067fc640 1 -- 172.21.15.76:0/2722601886 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808003a00 con 0x7fc8100d0da0 2023-11-09T03:46:33.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.167+0000 7fc817ee1640 1 -- 172.21.15.76:0/2722601886 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc810156500 con 0x7fc8100d0da0 2023-11-09T03:46:33.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.169+0000 7fc817ee1640 1 -- 172.21.15.76:0/2722601886 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc810160c00 con 0x7fc8100d0da0 2023-11-09T03:46:33.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.169+0000 7fc8067fc640 1 -- 172.21.15.76:0/2722601886 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc808003b60 con 0x7fc8100d0da0 2023-11-09T03:46:33.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.169+0000 7fc8067fc640 1 --2- 172.21.15.76:0/2722601886 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc7f406ae70 0x7fc7f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:33.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.169+0000 7fc8067fc640 1 -- 172.21.15.76:0/2722601886 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fc808089af0 con 0x7fc8100d0da0 2023-11-09T03:46:33.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.170+0000 7fc815c7d640 1 --2- 172.21.15.76:0/2722601886 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc7f406ae70 0x7fc7f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:33.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.170+0000 7fc815c7d640 1 --2- 172.21.15.76:0/2722601886 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc7f406ae70 0x7fc7f406d330 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc80c036000 tx=0x7fc80c031000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:33.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.174+0000 7fc8067fc640 1 -- 172.21.15.76:0/2722601886 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc810160c00 con 0x7fc8100d0da0 2023-11-09T03:46:33.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.339+0000 7fc817ee1640 1 -- 172.21.15.76:0/2722601886 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc810160c00 con 0x7fc8100d0da0 2023-11-09T03:46:33.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.340+0000 7fc8067fc640 1 -- 172.21.15.76:0/2722601886 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc810160c00 con 0x7fc8100d0da0 2023-11-09T03:46:33.340 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:46:33.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.342+0000 7fc817ee1640 1 -- 172.21.15.76:0/2722601886 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc7f406ae70 msgr2=0x7fc7f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:33.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.342+0000 7fc817ee1640 1 --2- 172.21.15.76:0/2722601886 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc7f406ae70 0x7fc7f406d330 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc80c036000 tx=0x7fc80c031000 comp rx=0 tx=0).stop 2023-11-09T03:46:33.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.342+0000 7fc817ee1640 1 -- 172.21.15.76:0/2722601886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8100d0da0 msgr2=0x7fc8100d1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:33.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.342+0000 7fc817ee1640 1 --2- 172.21.15.76:0/2722601886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8100d0da0 0x7fc8100d1220 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc81006c900 tx=0x7fc808011920 comp rx=0 tx=0).stop 2023-11-09T03:46:33.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.342+0000 7fc817ee1640 1 -- 172.21.15.76:0/2722601886 shutdown_connections 2023-11-09T03:46:33.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.342+0000 7fc817ee1640 1 --2- 172.21.15.76:0/2722601886 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc7f406ae70 0x7fc7f406d330 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.342+0000 7fc817ee1640 1 --2- 172.21.15.76:0/2722601886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8100d0da0 0x7fc8100d1220 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.342+0000 7fc817ee1640 1 --2- 172.21.15.76:0/2722601886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8100cc100 0x7fc810150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.342+0000 7fc817ee1640 1 --2- 172.21.15.76:0/2722601886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7fc0039a0 0x7fc810155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.342+0000 7fc817ee1640 1 -- 172.21.15.76:0/2722601886 >> 172.21.15.76:0/2722601886 conn(0x7fc8100cf880 msgr2=0x7fc81015e100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:33.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.342+0000 7fc817ee1640 1 -- 172.21.15.76:0/2722601886 shutdown_connections 2023-11-09T03:46:33.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.342+0000 7fc817ee1640 1 -- 172.21.15.76:0/2722601886 wait complete. 2023-11-09T03:46:33.353 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-09T03:46:33.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.454+0000 7f7ada7e2640 1 Processor -- start 2023-11-09T03:46:33.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.454+0000 7f7ada7e2640 1 -- start start 2023-11-09T03:46:33.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.455+0000 7f7ada7e2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ad4160430 0x7f7ad40b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:33.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.455+0000 7f7ada7e2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7ad40b2b60 0x7f7ad406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:33.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.455+0000 7f7ada7e2640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad40b2fc0 con 0x7f7ad406c1d0 2023-11-09T03:46:33.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.455+0000 7f7ada7e2640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ad40cb580 con 0x7f7ad40b2b60 2023-11-09T03:46:33.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.455+0000 7f7ada7e2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad406ca40 con 0x7f7ad4160430 2023-11-09T03:46:33.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.455+0000 7f7ad37fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7ad40b2b60 0x7f7ad406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:33.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.455+0000 7f7ad37fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7ad40b2b60 0x7f7ad406bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34914/0 (socket says 172.21.15.76:34914) 2023-11-09T03:46:33.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.455+0000 7f7ad37fe640 1 -- 172.21.15.76:0/3734858529 learned_addr learned my addr 172.21.15.76:0/3734858529 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:33.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.455+0000 7f7ad3fff640 1 --2- 172.21.15.76:0/3734858529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ad4160430 0x7f7ad40b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:33.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.455+0000 7f7ad37fe640 1 -- 172.21.15.76:0/3734858529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ad4160430 msgr2=0x7f7ad40b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:33.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.455+0000 7f7ad37fe640 1 --2- 172.21.15.76:0/3734858529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ad4160430 0x7f7ad40b2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.455+0000 7f7ad37fe640 1 -- 172.21.15.76:0/3734858529 >> v1:172.21.15.113:6789/0 conn(0x7f7ad406c1d0 legacy=0x7f7ad4160b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:33.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.455+0000 7f7ad37fe640 1 -- 172.21.15.76:0/3734858529 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad406cd20 con 0x7f7ad40b2b60 2023-11-09T03:46:33.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.456+0000 7f7ad37fe640 1 --2- 172.21.15.76:0/3734858529 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ad40b2b60 0x7f7ad406bc90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009870 tx=0x7f7ac802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0eb87924a783c94 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:33.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.456+0000 7f7ad2ffd640 1 -- 172.21.15.76:0/3734858529 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8031640 con 0x7f7ad40b2b60 2023-11-09T03:46:33.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.456+0000 7f7ad2ffd640 1 -- 172.21.15.76:0/3734858529 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac80317a0 con 0x7f7ad40b2b60 2023-11-09T03:46:33.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.456+0000 7f7ad2ffd640 1 -- 172.21.15.76:0/3734858529 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8031b30 con 0x7f7ad40b2b60 2023-11-09T03:46:33.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.457+0000 7f7ada7e2640 1 -- 172.21.15.76:0/3734858529 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ad40b2b60 msgr2=0x7f7ad406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:33.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.457+0000 7f7ada7e2640 1 --2- 172.21.15.76:0/3734858529 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ad40b2b60 0x7f7ad406bc90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009870 tx=0x7f7ac802b8f0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.457+0000 7f7ada7e2640 1 -- 172.21.15.76:0/3734858529 shutdown_connections 2023-11-09T03:46:33.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.457+0000 7f7ada7e2640 1 --2- 172.21.15.76:0/3734858529 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ad40b2b60 0x7f7ad406bc90 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.457+0000 7f7ada7e2640 1 --2- 172.21.15.76:0/3734858529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ad4160430 0x7f7ad40b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.457+0000 7f7ada7e2640 1 -- 172.21.15.76:0/3734858529 >> 172.21.15.76:0/3734858529 conn(0x7f7ad406ea20 msgr2=0x7f7ad406ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:33.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.457+0000 7f7ada7e2640 1 -- 172.21.15.76:0/3734858529 shutdown_connections 2023-11-09T03:46:33.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.457+0000 7f7ada7e2640 1 -- 172.21.15.76:0/3734858529 wait complete. 2023-11-09T03:46:33.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.458+0000 7f7ada7e2640 1 Processor -- start 2023-11-09T03:46:33.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.458+0000 7f7ada7e2640 1 -- start start 2023-11-09T03:46:33.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.459+0000 7f7ada7e2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ad406c1d0 0x7f7ad4152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:33.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.459+0000 7f7ada7e2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ad40b2b60 0x7f7ad40c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:33.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.459+0000 7f7ada7e2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ad4160430 0x7f7ad40cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:33.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.459+0000 7f7ada7e2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad40b3bd0 con 0x7f7ad40b2b60 2023-11-09T03:46:33.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.459+0000 7f7ada7e2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad4152690 con 0x7f7ad406c1d0 2023-11-09T03:46:33.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.459+0000 7f7ada7e2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad40c8c10 con 0x7f7ad4160430 2023-11-09T03:46:33.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.459+0000 7f7ad37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ad40b2b60 0x7f7ad40c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:33.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.459+0000 7f7ad37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ad40b2b60 0x7f7ad40c7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34916/0 (socket says 172.21.15.76:34916) 2023-11-09T03:46:33.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.459+0000 7f7ad37fe640 1 -- 172.21.15.76:0/1913917830 learned_addr learned my addr 172.21.15.76:0/1913917830 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:33.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.459+0000 7f7ad37fe640 1 -- 172.21.15.76:0/1913917830 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ad4160430 msgr2=0x7f7ad40cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:33.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.459+0000 7f7ad3fff640 1 --2- 172.21.15.76:0/1913917830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ad406c1d0 0x7f7ad4152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:33.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.459+0000 7f7ad8d7f640 1 --2- 172.21.15.76:0/1913917830 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ad4160430 0x7f7ad40cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:33.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.459+0000 7f7ad37fe640 1 --2- 172.21.15.76:0/1913917830 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ad4160430 0x7f7ad40cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.459+0000 7f7ad37fe640 1 -- 172.21.15.76:0/1913917830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ad406c1d0 msgr2=0x7f7ad4152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:33.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.459+0000 7f7ad37fe640 1 --2- 172.21.15.76:0/1913917830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ad406c1d0 0x7f7ad4152150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.459+0000 7f7ad37fe640 1 -- 172.21.15.76:0/1913917830 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac80094d0 con 0x7f7ad40b2b60 2023-11-09T03:46:33.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.459+0000 7f7ad37fe640 1 --2- 172.21.15.76:0/1913917830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ad40b2b60 0x7f7ad40c7c70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009870 tx=0x7f7ac80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:33.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.459+0000 7f7ad0ff9640 1 -- 172.21.15.76:0/1913917830 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac803ac60 con 0x7f7ad40b2b60 2023-11-09T03:46:33.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.459+0000 7f7ad0ff9640 1 -- 172.21.15.76:0/1913917830 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac803ade0 con 0x7f7ad40b2b60 2023-11-09T03:46:33.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.459+0000 7f7ada7e2640 1 -- 172.21.15.76:0/1913917830 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad40c8650 con 0x7f7ad40b2b60 2023-11-09T03:46:33.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.459+0000 7f7ad0ff9640 1 -- 172.21.15.76:0/1913917830 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8044630 con 0x7f7ad40b2b60 2023-11-09T03:46:33.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.460+0000 7f7ada7e2640 1 -- 172.21.15.76:0/1913917830 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad40b3860 con 0x7f7ad40b2b60 2023-11-09T03:46:33.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.460+0000 7f7ada7e2640 1 -- 172.21.15.76:0/1913917830 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ad40b3d40 con 0x7f7ad40b2b60 2023-11-09T03:46:33.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.461+0000 7f7ad0ff9640 1 -- 172.21.15.76:0/1913917830 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7ac802f050 con 0x7f7ad40b2b60 2023-11-09T03:46:33.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.461+0000 7f7ad0ff9640 1 --2- 172.21.15.76:0/1913917830 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7ac006ae70 0x7f7ac006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:33.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.461+0000 7f7ad0ff9640 1 -- 172.21.15.76:0/1913917830 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f7ac80abae0 con 0x7f7ad40b2b60 2023-11-09T03:46:33.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.461+0000 7f7ad3fff640 1 --2- 172.21.15.76:0/1913917830 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7ac006ae70 0x7f7ac006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:33.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.462+0000 7f7ad3fff640 1 --2- 172.21.15.76:0/1913917830 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7ac006ae70 0x7f7ac006d330 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400b7c0 tx=0x7f7ac4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:33.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.465+0000 7f7ad0ff9640 1 -- 172.21.15.76:0/1913917830 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7ac8076840 con 0x7f7ad40b2b60 2023-11-09T03:46:33.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.643+0000 7f7ada7e2640 1 -- 172.21.15.76:0/1913917830 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7ad4074290 con 0x7f7ad40b2b60 2023-11-09T03:46:33.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.643+0000 7f7ad0ff9640 1 -- 172.21.15.76:0/1913917830 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7ac8035080 con 0x7f7ad40b2b60 2023-11-09T03:46:33.644 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2023-11-09T03:46:33.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.645+0000 7f7ada7e2640 1 -- 172.21.15.76:0/1913917830 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7ac006ae70 msgr2=0x7f7ac006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:33.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.645+0000 7f7ada7e2640 1 --2- 172.21.15.76:0/1913917830 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7ac006ae70 0x7f7ac006d330 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400b7c0 tx=0x7f7ac4008040 comp rx=0 tx=0).stop 2023-11-09T03:46:33.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.645+0000 7f7ada7e2640 1 -- 172.21.15.76:0/1913917830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ad40b2b60 msgr2=0x7f7ad40c7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:33.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.645+0000 7f7ada7e2640 1 --2- 172.21.15.76:0/1913917830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ad40b2b60 0x7f7ad40c7c70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009870 tx=0x7f7ac80046c0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.645+0000 7f7ada7e2640 1 -- 172.21.15.76:0/1913917830 shutdown_connections 2023-11-09T03:46:33.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.645+0000 7f7ada7e2640 1 --2- 172.21.15.76:0/1913917830 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7ac006ae70 0x7f7ac006d330 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.645+0000 7f7ada7e2640 1 --2- 172.21.15.76:0/1913917830 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ad4160430 0x7f7ad40cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.645+0000 7f7ada7e2640 1 --2- 172.21.15.76:0/1913917830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ad40b2b60 0x7f7ad40c7c70 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.645+0000 7f7ada7e2640 1 --2- 172.21.15.76:0/1913917830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ad406c1d0 0x7f7ad4152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.645+0000 7f7ada7e2640 1 -- 172.21.15.76:0/1913917830 >> 172.21.15.76:0/1913917830 conn(0x7f7ad406ea20 msgr2=0x7f7ad415d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:33.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.645+0000 7f7ada7e2640 1 -- 172.21.15.76:0/1913917830 shutdown_connections 2023-11-09T03:46:33.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.645+0000 7f7ada7e2640 1 -- 172.21.15.76:0/1913917830 wait complete. 2023-11-09T03:46:33.656 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi076.front.sepia.ceph.com... 2023-11-09T03:46:33.657 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:46:33.657 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.2 2023-11-09T03:46:33.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:46:33.719 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi076.stderr:ceph-fuse[98971]: fuse finished with error 0 and tester_r 0 2023-11-09T03:46:33.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:46:33.723 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi076.stderr:2023-11-09T03:46:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:46:33.726 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:46:33.731 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T03:46:33.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:33.732 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-09T03:46:33.787 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:46:33.787 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:46:33.806 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:46:33.807 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:46:33.865 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:46:33.906 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi113.stderr:2023-11-09T03:46:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:46:33.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.964+0000 7fe20a07c640 1 Processor -- start 2023-11-09T03:46:33.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.964+0000 7fe20a07c640 1 -- start start 2023-11-09T03:46:33.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.965+0000 7fe20a07c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe204159c70 0x7fe2040ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:33.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.965+0000 7fe20a07c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe2040b0ae0 0x7fe20415d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:33.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.965+0000 7fe20a07c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2040b4110 con 0x7fe20406db20 2023-11-09T03:46:33.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.965+0000 7fe20a07c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2040b49b0 con 0x7fe2040b0ae0 2023-11-09T03:46:33.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.965+0000 7fe20a07c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2040b43f0 con 0x7fe204159c70 2023-11-09T03:46:33.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.965+0000 7fe203fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe2040b0ae0 0x7fe20415d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:33.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.965+0000 7fe203fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe2040b0ae0 0x7fe20415d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34922/0 (socket says 172.21.15.76:34922) 2023-11-09T03:46:33.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.965+0000 7fe203fff640 1 -- 172.21.15.76:0/3509457124 learned_addr learned my addr 172.21.15.76:0/3509457124 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:33.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.966+0000 7fe2037fe640 1 --2- 172.21.15.76:0/3509457124 >> v2:172.21.15.113:3300/0 conn(0x7fe204159c70 0x7fe2040ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:33.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.966+0000 7fe203fff640 1 -- 172.21.15.76:0/3509457124 >> v2:172.21.15.113:3300/0 conn(0x7fe204159c70 msgr2=0x7fe2040ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:33.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.966+0000 7fe203fff640 1 --2- 172.21.15.76:0/3509457124 >> v2:172.21.15.113:3300/0 conn(0x7fe204159c70 0x7fe2040ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.966+0000 7fe203fff640 1 -- 172.21.15.76:0/3509457124 >> v1:172.21.15.76:6789/0 conn(0x7fe20406db20 legacy=0x7fe2040b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:33.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.966+0000 7fe203fff640 1 -- 172.21.15.76:0/3509457124 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2040b46d0 con 0x7fe2040b0ae0 2023-11-09T03:46:33.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.966+0000 7fe2027fc640 1 -- 172.21.15.76:0/3509457124 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3678155772 0 0) 0x7fe2040b4110 con 0x7fe20406db20 2023-11-09T03:46:33.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.966+0000 7fe2037fe640 1 --2- 172.21.15.76:0/3509457124 >> v2:172.21.15.113:3300/0 conn(0x7fe204159c70 0x7fe2040ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:33.971 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.966+0000 7fe203fff640 1 --2- 172.21.15.76:0/3509457124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2040b0ae0 0x7fe20415d7d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe2040cc440 tx=0x7fe1fc02e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a631d5558bb8b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:33.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.966+0000 7fe2027fc640 1 -- 172.21.15.76:0/3509457124 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc00b0d0 con 0x7fe2040b0ae0 2023-11-09T03:46:33.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.967+0000 7fe2027fc640 1 -- 172.21.15.76:0/3509457124 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1fc00b230 con 0x7fe2040b0ae0 2023-11-09T03:46:33.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.967+0000 7fe2027fc640 1 -- 172.21.15.76:0/3509457124 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc009600 con 0x7fe2040b0ae0 2023-11-09T03:46:33.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.967+0000 7fe20a07c640 1 -- 172.21.15.76:0/3509457124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2040b0ae0 msgr2=0x7fe20415d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:33.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.967+0000 7fe20a07c640 1 --2- 172.21.15.76:0/3509457124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2040b0ae0 0x7fe20415d7d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe2040cc440 tx=0x7fe1fc02e180 comp rx=0 tx=0).stop 2023-11-09T03:46:33.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.967+0000 7fe20a07c640 1 -- 172.21.15.76:0/3509457124 shutdown_connections 2023-11-09T03:46:33.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.967+0000 7fe20a07c640 1 --2- 172.21.15.76:0/3509457124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2040b0ae0 0x7fe20415d7d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.967+0000 7fe20a07c640 1 --2- 172.21.15.76:0/3509457124 >> v2:172.21.15.113:3300/0 conn(0x7fe204159c70 0x7fe2040ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.967+0000 7fe20a07c640 1 -- 172.21.15.76:0/3509457124 >> 172.21.15.76:0/3509457124 conn(0x7fe2040ce950 msgr2=0x7fe2040ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:33.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.967+0000 7fe20a07c640 1 -- 172.21.15.76:0/3509457124 shutdown_connections 2023-11-09T03:46:33.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.967+0000 7fe20a07c640 1 -- 172.21.15.76:0/3509457124 wait complete. 2023-11-09T03:46:33.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.968+0000 7fe20a07c640 1 Processor -- start 2023-11-09T03:46:33.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.968+0000 7fe20a07c640 1 -- start start 2023-11-09T03:46:33.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.969+0000 7fe20a07c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe20406db20 0x7fe204154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:33.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.969+0000 7fe20a07c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2040b0ae0 0x7fe20415ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:33.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.969+0000 7fe20a07c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe204159c70 0x7fe2040c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:33.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.969+0000 7fe20a07c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2040aea40 con 0x7fe2040b0ae0 2023-11-09T03:46:33.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.969+0000 7fe20a07c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20415b320 con 0x7fe20406db20 2023-11-09T03:46:33.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.969+0000 7fe20a07c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2040c62b0 con 0x7fe204159c70 2023-11-09T03:46:33.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.969+0000 7fe202ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2040b0ae0 0x7fe20415ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:33.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.969+0000 7fe202ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2040b0ae0 0x7fe20415ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34930/0 (socket says 172.21.15.76:34930) 2023-11-09T03:46:33.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.969+0000 7fe202ffd640 1 -- 172.21.15.76:0/1760914094 learned_addr learned my addr 172.21.15.76:0/1760914094 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:33.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.970+0000 7fe202ffd640 1 -- 172.21.15.76:0/1760914094 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe204159c70 msgr2=0x7fe2040c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:33.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.970+0000 7fe202ffd640 1 --2- 172.21.15.76:0/1760914094 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe204159c70 0x7fe2040c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.970+0000 7fe202ffd640 1 -- 172.21.15.76:0/1760914094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe20406db20 msgr2=0x7fe204154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:33.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.970+0000 7fe202ffd640 1 --2- 172.21.15.76:0/1760914094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe20406db20 0x7fe204154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:33.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.970+0000 7fe202ffd640 1 -- 172.21.15.76:0/1760914094 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc00bd40 con 0x7fe2040b0ae0 2023-11-09T03:46:33.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.970+0000 7fe202ffd640 1 --2- 172.21.15.76:0/1760914094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2040b0ae0 0x7fe20415ade0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800c410 tx=0x7fe1f800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:33.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.970+0000 7fe1ebfff640 1 -- 172.21.15.76:0/1760914094 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f80035f0 con 0x7fe2040b0ae0 2023-11-09T03:46:33.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.970+0000 7fe1ebfff640 1 -- 172.21.15.76:0/1760914094 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f8003770 con 0x7fe2040b0ae0 2023-11-09T03:46:33.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.970+0000 7fe20a07c640 1 -- 172.21.15.76:0/1760914094 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2040c6590 con 0x7fe2040b0ae0 2023-11-09T03:46:33.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.970+0000 7fe1ebfff640 1 -- 172.21.15.76:0/1760914094 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8003b20 con 0x7fe2040b0ae0 2023-11-09T03:46:33.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.970+0000 7fe20a07c640 1 -- 172.21.15.76:0/1760914094 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2041514f0 con 0x7fe2040b0ae0 2023-11-09T03:46:33.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.971+0000 7fe20a07c640 1 -- 172.21.15.76:0/1760914094 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2040c6ab0 con 0x7fe2040b0ae0 2023-11-09T03:46:33.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.971+0000 7fe1ebfff640 1 -- 172.21.15.76:0/1760914094 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe1f8003c80 con 0x7fe2040b0ae0 2023-11-09T03:46:33.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.972+0000 7fe1ebfff640 1 --2- 172.21.15.76:0/1760914094 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe1f006ae70 0x7fe1f006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:33.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.972+0000 7fe1ebfff640 1 -- 172.21.15.76:0/1760914094 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fe1f8089000 con 0x7fe2040b0ae0 2023-11-09T03:46:33.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.972+0000 7fe2037fe640 1 --2- 172.21.15.76:0/1760914094 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe1f006ae70 0x7fe1f006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:33.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.972+0000 7fe2037fe640 1 --2- 172.21.15.76:0/1760914094 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe1f006ae70 0x7fe1f006d330 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4006fd0 tx=0x7fe1f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:33.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:33.976+0000 7fe1ebfff640 1 -- 172.21.15.76:0/1760914094 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe2040c6ab0 con 0x7fe2040b0ae0 2023-11-09T03:46:34.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.141+0000 7fe20a07c640 1 -- 172.21.15.76:0/1760914094 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2040c6ab0 con 0x7fe2040b0ae0 2023-11-09T03:46:34.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.142+0000 7fe1ebfff640 1 -- 172.21.15.76:0/1760914094 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18710 (secure 0 0 0) 0x7fe2040c6ab0 con 0x7fe2040b0ae0 2023-11-09T03:46:34.143 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:34.143 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":27,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:46:12.537265+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:46:34.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.144+0000 7fe20a07c640 1 -- 172.21.15.76:0/1760914094 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe1f006ae70 msgr2=0x7fe1f006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:34.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.144+0000 7fe20a07c640 1 --2- 172.21.15.76:0/1760914094 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe1f006ae70 0x7fe1f006d330 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4006fd0 tx=0x7fe1f4008040 comp rx=0 tx=0).stop 2023-11-09T03:46:34.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.144+0000 7fe20a07c640 1 -- 172.21.15.76:0/1760914094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2040b0ae0 msgr2=0x7fe20415ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:34.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.144+0000 7fe20a07c640 1 --2- 172.21.15.76:0/1760914094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2040b0ae0 0x7fe20415ade0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800c410 tx=0x7fe1f800c8d0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.144+0000 7fe20a07c640 1 -- 172.21.15.76:0/1760914094 shutdown_connections 2023-11-09T03:46:34.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.144+0000 7fe20a07c640 1 --2- 172.21.15.76:0/1760914094 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe1f006ae70 0x7fe1f006d330 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.144+0000 7fe20a07c640 1 --2- 172.21.15.76:0/1760914094 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe204159c70 0x7fe2040c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.144+0000 7fe20a07c640 1 --2- 172.21.15.76:0/1760914094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2040b0ae0 0x7fe20415ade0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.144+0000 7fe20a07c640 1 --2- 172.21.15.76:0/1760914094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe20406db20 0x7fe204154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.144+0000 7fe20a07c640 1 -- 172.21.15.76:0/1760914094 >> 172.21.15.76:0/1760914094 conn(0x7fe2040ce950 msgr2=0x7fe2040b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:34.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.144+0000 7fe20a07c640 1 -- 172.21.15.76:0/1760914094 shutdown_connections 2023-11-09T03:46:34.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.144+0000 7fe20a07c640 1 -- 172.21.15.76:0/1760914094 wait complete. 2023-11-09T03:46:34.155 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:45:28.704715+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:45:29.717614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-09T03:45:33.440577+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-09T03:46:34.156 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:46:34.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.259+0000 7f374a778640 1 Processor -- start 2023-11-09T03:46:34.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.259+0000 7f374a778640 1 -- start start 2023-11-09T03:46:34.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.259+0000 7f374a778640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f37440cd6d0 0x7f3744161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:34.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.259+0000 7f374a778640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37440b4880 0x7f37440b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:34.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.259+0000 7f374a778640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37440cde60 con 0x7f3744153e60 2023-11-09T03:46:34.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.259+0000 7f374a778640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3744160f90 con 0x7f37440cd6d0 2023-11-09T03:46:34.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.259+0000 7f374a778640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3744161960 con 0x7f37440b4880 2023-11-09T03:46:34.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.260+0000 7f3743fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f3744153e60 0x7f374415dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40650/0 (socket says 172.21.15.76:40650) 2023-11-09T03:46:34.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.260+0000 7f3743fff640 1 -- 172.21.15.76:0/2820438972 learned_addr learned my addr 172.21.15.76:0/2820438972 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:34.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.260+0000 7f37437fe640 1 --2- 172.21.15.76:0/2820438972 >> v2:172.21.15.76:3300/0 conn(0x7f37440cd6d0 0x7f3744161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:34.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.260+0000 7f3748d15640 1 --2- 172.21.15.76:0/2820438972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37440b4880 0x7f37440b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:34.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.260+0000 7f37437fe640 1 -- 172.21.15.76:0/2820438972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37440b4880 msgr2=0x7f37440b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:34.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.260+0000 7f37437fe640 1 --2- 172.21.15.76:0/2820438972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37440b4880 0x7f37440b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.260+0000 7f37437fe640 1 -- 172.21.15.76:0/2820438972 >> v1:172.21.15.76:6789/0 conn(0x7f3744153e60 legacy=0x7f374415dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:34.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.260+0000 7f37437fe640 1 -- 172.21.15.76:0/2820438972 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37440b5220 con 0x7f37440cd6d0 2023-11-09T03:46:34.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.260+0000 7f3742ffd640 1 -- 172.21.15.76:0/2820438972 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2808031114 0 0) 0x7f37440cde60 con 0x7f3744153e60 2023-11-09T03:46:34.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.261+0000 7f37437fe640 1 --2- 172.21.15.76:0/2820438972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37440cd6d0 0x7f3744161420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f372c0097a0 tx=0x7f372c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5d20edd0c60efd server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:34.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.261+0000 7f3742ffd640 1 -- 172.21.15.76:0/2820438972 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c031830 con 0x7f37440cd6d0 2023-11-09T03:46:34.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.261+0000 7f3742ffd640 1 -- 172.21.15.76:0/2820438972 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372c031990 con 0x7f37440cd6d0 2023-11-09T03:46:34.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.261+0000 7f3742ffd640 1 -- 172.21.15.76:0/2820438972 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c031d20 con 0x7f37440cd6d0 2023-11-09T03:46:34.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.261+0000 7f374a778640 1 -- 172.21.15.76:0/2820438972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37440cd6d0 msgr2=0x7f3744161420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:34.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.261+0000 7f374a778640 1 --2- 172.21.15.76:0/2820438972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37440cd6d0 0x7f3744161420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f372c0097a0 tx=0x7f372c02b8d0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.261+0000 7f374a778640 1 -- 172.21.15.76:0/2820438972 shutdown_connections 2023-11-09T03:46:34.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.261+0000 7f374a778640 1 --2- 172.21.15.76:0/2820438972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37440b4880 0x7f37440b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.261+0000 7f374a778640 1 --2- 172.21.15.76:0/2820438972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37440cd6d0 0x7f3744161420 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.261+0000 7f374a778640 1 -- 172.21.15.76:0/2820438972 >> 172.21.15.76:0/2820438972 conn(0x7f37440cf880 msgr2=0x7f37440cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:34.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.261+0000 7f374a778640 1 -- 172.21.15.76:0/2820438972 shutdown_connections 2023-11-09T03:46:34.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.261+0000 7f374a778640 1 -- 172.21.15.76:0/2820438972 wait complete. 2023-11-09T03:46:34.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.262+0000 7f374a778640 1 Processor -- start 2023-11-09T03:46:34.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.263+0000 7f374a778640 1 -- start start 2023-11-09T03:46:34.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.263+0000 7f374a778640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37440b4880 0x7f37441f2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:34.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.263+0000 7f374a778640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37440cd6d0 0x7f374415c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:34.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.263+0000 7f374a778640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3744153e60 0x7f374415f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:34.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.263+0000 7f374a778640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37441572d0 con 0x7f37440b4880 2023-11-09T03:46:34.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.263+0000 7f374a778640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37440c9070 con 0x7f37440cd6d0 2023-11-09T03:46:34.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.263+0000 7f374a778640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37441f3230 con 0x7f3744153e60 2023-11-09T03:46:34.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.263+0000 7f3743fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37440b4880 0x7f37441f2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:34.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.263+0000 7f3743fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37440b4880 0x7f37441f2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34954/0 (socket says 172.21.15.76:34954) 2023-11-09T03:46:34.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.264+0000 7f3743fff640 1 -- 172.21.15.76:0/1794026026 learned_addr learned my addr 172.21.15.76:0/1794026026 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:34.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.264+0000 7f37437fe640 1 --2- 172.21.15.76:0/1794026026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37440cd6d0 0x7f374415c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:34.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.264+0000 7f3748d15640 1 --2- 172.21.15.76:0/1794026026 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3744153e60 0x7f374415f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:34.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.264+0000 7f3743fff640 1 -- 172.21.15.76:0/1794026026 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3744153e60 msgr2=0x7f374415f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:34.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.264+0000 7f3743fff640 1 --2- 172.21.15.76:0/1794026026 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3744153e60 0x7f374415f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.264+0000 7f3743fff640 1 -- 172.21.15.76:0/1794026026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37440cd6d0 msgr2=0x7f374415c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:34.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.264+0000 7f3743fff640 1 --2- 172.21.15.76:0/1794026026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37440cd6d0 0x7f374415c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.264+0000 7f3743fff640 1 -- 172.21.15.76:0/1794026026 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f372c009400 con 0x7f37440b4880 2023-11-09T03:46:34.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.264+0000 7f37437fe640 1 --2- 172.21.15.76:0/1794026026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37440cd6d0 0x7f374415c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:34.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.264+0000 7f3748d15640 1 --2- 172.21.15.76:0/1794026026 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3744153e60 0x7f374415f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:34.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.264+0000 7f3743fff640 1 --2- 172.21.15.76:0/1794026026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37440b4880 0x7f37441f2990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f373400c430 tx=0x7f373400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:34.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.264+0000 7f3740ff9640 1 -- 172.21.15.76:0/1794026026 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37340074e0 con 0x7f37440b4880 2023-11-09T03:46:34.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.264+0000 7f3740ff9640 1 -- 172.21.15.76:0/1794026026 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3734007660 con 0x7f37440b4880 2023-11-09T03:46:34.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.264+0000 7f374a778640 1 -- 172.21.15.76:0/1794026026 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37441f3510 con 0x7f37440b4880 2023-11-09T03:46:34.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.264+0000 7f3740ff9640 1 -- 172.21.15.76:0/1794026026 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3734004a20 con 0x7f37440b4880 2023-11-09T03:46:34.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.264+0000 7f374a778640 1 -- 172.21.15.76:0/1794026026 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37441f38a0 con 0x7f37440b4880 2023-11-09T03:46:34.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.265+0000 7f374a778640 1 -- 172.21.15.76:0/1794026026 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3744074290 con 0x7f37440b4880 2023-11-09T03:46:34.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.266+0000 7f3740ff9640 1 -- 172.21.15.76:0/1794026026 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3734004b80 con 0x7f37440b4880 2023-11-09T03:46:34.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.266+0000 7f3740ff9640 1 --2- 172.21.15.76:0/1794026026 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f372406ae70 0x7f372406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:34.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.266+0000 7f3740ff9640 1 -- 172.21.15.76:0/1794026026 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f3734089880 con 0x7f37440b4880 2023-11-09T03:46:34.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.266+0000 7f37437fe640 1 --2- 172.21.15.76:0/1794026026 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f372406ae70 0x7f372406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:34.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.267+0000 7f37437fe640 1 --2- 172.21.15.76:0/1794026026 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f372406ae70 0x7f372406d330 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f372c036000 tx=0x7f372c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:34.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.270+0000 7f3740ff9640 1 -- 172.21.15.76:0/1794026026 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3734054550 con 0x7f37440b4880 2023-11-09T03:46:34.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.438+0000 7f374a778640 1 -- 172.21.15.76:0/1794026026 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f37440d0b80 con 0x7f37440b4880 2023-11-09T03:46:34.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.438+0000 7f3740ff9640 1 -- 172.21.15.76:0/1794026026 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3734057b70 con 0x7f37440b4880 2023-11-09T03:46:34.439 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:46:34.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.440+0000 7f374a778640 1 -- 172.21.15.76:0/1794026026 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f372406ae70 msgr2=0x7f372406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:34.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.440+0000 7f374a778640 1 --2- 172.21.15.76:0/1794026026 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f372406ae70 0x7f372406d330 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f372c036000 tx=0x7f372c008030 comp rx=0 tx=0).stop 2023-11-09T03:46:34.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.440+0000 7f374a778640 1 -- 172.21.15.76:0/1794026026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37440b4880 msgr2=0x7f37441f2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:34.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.440+0000 7f374a778640 1 --2- 172.21.15.76:0/1794026026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37440b4880 0x7f37441f2990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f373400c430 tx=0x7f373400c8f0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.440+0000 7f374a778640 1 -- 172.21.15.76:0/1794026026 shutdown_connections 2023-11-09T03:46:34.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.440+0000 7f374a778640 1 --2- 172.21.15.76:0/1794026026 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f372406ae70 0x7f372406d330 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.440+0000 7f374a778640 1 --2- 172.21.15.76:0/1794026026 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3744153e60 0x7f374415f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.440+0000 7f374a778640 1 --2- 172.21.15.76:0/1794026026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37440cd6d0 0x7f374415c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.440+0000 7f374a778640 1 --2- 172.21.15.76:0/1794026026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37440b4880 0x7f37441f2990 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.440+0000 7f374a778640 1 -- 172.21.15.76:0/1794026026 >> 172.21.15.76:0/1794026026 conn(0x7f37440cf880 msgr2=0x7f37440d24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:34.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.440+0000 7f374a778640 1 -- 172.21.15.76:0/1794026026 shutdown_connections 2023-11-09T03:46:34.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.440+0000 7f374a778640 1 -- 172.21.15.76:0/1794026026 wait complete. 2023-11-09T03:46:34.451 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:46:34.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.554+0000 7fc5195a9640 1 Processor -- start 2023-11-09T03:46:34.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.554+0000 7fc5195a9640 1 -- start start 2023-11-09T03:46:34.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.555+0000 7fc5195a9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc514153e60 0x7fc51415dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:34.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.555+0000 7fc5195a9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc5140cd6d0 0x7fc514161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:34.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.555+0000 7fc5195a9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5140cde60 con 0x7fc5140b4880 2023-11-09T03:46:34.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.555+0000 7fc5195a9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc514160f90 con 0x7fc514153e60 2023-11-09T03:46:34.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.555+0000 7fc5195a9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc51415e400 con 0x7fc5140cd6d0 2023-11-09T03:46:34.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.555+0000 7fc512ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc514153e60 0x7fc51415dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:34.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.555+0000 7fc5137fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc5140b4880 0x7fc51406b810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40656/0 (socket says 172.21.15.76:40656) 2023-11-09T03:46:34.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.555+0000 7fc5137fe640 1 -- 172.21.15.76:0/3623390862 learned_addr learned my addr 172.21.15.76:0/3623390862 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:34.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.555+0000 7fc50bfff640 1 --2- 172.21.15.76:0/3623390862 >> v2:172.21.15.113:3300/0 conn(0x7fc5140cd6d0 0x7fc514161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:34.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.555+0000 7fc512ffd640 1 -- 172.21.15.76:0/3623390862 >> v2:172.21.15.113:3300/0 conn(0x7fc5140cd6d0 msgr2=0x7fc514161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:34.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.555+0000 7fc512ffd640 1 --2- 172.21.15.76:0/3623390862 >> v2:172.21.15.113:3300/0 conn(0x7fc5140cd6d0 0x7fc514161420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.555+0000 7fc512ffd640 1 -- 172.21.15.76:0/3623390862 >> v1:172.21.15.76:6789/0 conn(0x7fc5140b4880 legacy=0x7fc51406b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:34.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.555+0000 7fc512ffd640 1 -- 172.21.15.76:0/3623390862 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc514161960 con 0x7fc514153e60 2023-11-09T03:46:34.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.555+0000 7fc5127fc640 1 -- 172.21.15.76:0/3623390862 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4201403499 0 0) 0x7fc5140cde60 con 0x7fc5140b4880 2023-11-09T03:46:34.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.555+0000 7fc50bfff640 1 --2- 172.21.15.76:0/3623390862 >> v2:172.21.15.113:3300/0 conn(0x7fc5140cd6d0 0x7fc514161420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:34.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.556+0000 7fc512ffd640 1 --2- 172.21.15.76:0/3623390862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc514153e60 0x7fc51415dec0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc5140c9e80 tx=0x7fc50402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5bba56aede6491f server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:34.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.556+0000 7fc5127fc640 1 -- 172.21.15.76:0/3623390862 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc504031830 con 0x7fc514153e60 2023-11-09T03:46:34.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.556+0000 7fc5127fc640 1 -- 172.21.15.76:0/3623390862 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc504031990 con 0x7fc514153e60 2023-11-09T03:46:34.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.556+0000 7fc5127fc640 1 -- 172.21.15.76:0/3623390862 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc504031d20 con 0x7fc514153e60 2023-11-09T03:46:34.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.557+0000 7fc5195a9640 1 -- 172.21.15.76:0/3623390862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc514153e60 msgr2=0x7fc51415dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:34.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.557+0000 7fc5195a9640 1 --2- 172.21.15.76:0/3623390862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc514153e60 0x7fc51415dec0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc5140c9e80 tx=0x7fc50402b8d0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.557+0000 7fc5195a9640 1 -- 172.21.15.76:0/3623390862 shutdown_connections 2023-11-09T03:46:34.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.557+0000 7fc5195a9640 1 --2- 172.21.15.76:0/3623390862 >> v2:172.21.15.113:3300/0 conn(0x7fc5140cd6d0 0x7fc514161420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.557+0000 7fc5195a9640 1 --2- 172.21.15.76:0/3623390862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc514153e60 0x7fc51415dec0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.557+0000 7fc5195a9640 1 -- 172.21.15.76:0/3623390862 >> 172.21.15.76:0/3623390862 conn(0x7fc5140cf880 msgr2=0x7fc5140cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:34.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.557+0000 7fc5195a9640 1 -- 172.21.15.76:0/3623390862 shutdown_connections 2023-11-09T03:46:34.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.557+0000 7fc5195a9640 1 -- 172.21.15.76:0/3623390862 wait complete. 2023-11-09T03:46:34.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.558+0000 7fc5195a9640 1 Processor -- start 2023-11-09T03:46:34.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.558+0000 7fc5195a9640 1 -- start start 2023-11-09T03:46:34.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.558+0000 7fc5195a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5140b4880 0x7fc5140ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:34.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.558+0000 7fc5195a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5140cd6d0 0x7fc5140cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:34.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.559+0000 7fc5195a9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc514153e60 0x7fc5140b7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:34.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.559+0000 7fc5195a9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5140c9290 con 0x7fc514153e60 2023-11-09T03:46:34.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.559+0000 7fc5195a9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5140cbd00 con 0x7fc5140cd6d0 2023-11-09T03:46:34.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.559+0000 7fc5195a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5140cbfe0 con 0x7fc5140b4880 2023-11-09T03:46:34.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.559+0000 7fc5137fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc514153e60 0x7fc5140b7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:34.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.559+0000 7fc5137fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc514153e60 0x7fc5140b7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34968/0 (socket says 172.21.15.76:34968) 2023-11-09T03:46:34.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.559+0000 7fc5137fe640 1 -- 172.21.15.76:0/973310917 learned_addr learned my addr 172.21.15.76:0/973310917 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:34.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.559+0000 7fc50bfff640 1 --2- 172.21.15.76:0/973310917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5140cd6d0 0x7fc5140cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:34.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.559+0000 7fc512ffd640 1 --2- 172.21.15.76:0/973310917 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5140b4880 0x7fc5140ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:34.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.559+0000 7fc5137fe640 1 -- 172.21.15.76:0/973310917 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5140b4880 msgr2=0x7fc5140ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:34.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.559+0000 7fc5137fe640 1 --2- 172.21.15.76:0/973310917 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5140b4880 0x7fc5140ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.559+0000 7fc5137fe640 1 -- 172.21.15.76:0/973310917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5140cd6d0 msgr2=0x7fc5140cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:34.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.559+0000 7fc5137fe640 1 --2- 172.21.15.76:0/973310917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5140cd6d0 0x7fc5140cb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.559+0000 7fc5137fe640 1 -- 172.21.15.76:0/973310917 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc504009400 con 0x7fc514153e60 2023-11-09T03:46:34.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.559+0000 7fc5137fe640 1 --2- 172.21.15.76:0/973310917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc514153e60 0x7fc5140b7020 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc50c00c7a0 tx=0x7fc50c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:34.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.559+0000 7fc50b7fe640 1 -- 172.21.15.76:0/973310917 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c00a640 con 0x7fc514153e60 2023-11-09T03:46:34.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.559+0000 7fc50b7fe640 1 -- 172.21.15.76:0/973310917 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50c00a7a0 con 0x7fc514153e60 2023-11-09T03:46:34.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.559+0000 7fc50b7fe640 1 -- 172.21.15.76:0/973310917 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c00abd0 con 0x7fc514153e60 2023-11-09T03:46:34.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.559+0000 7fc5195a9640 1 -- 172.21.15.76:0/973310917 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5140cc2c0 con 0x7fc514153e60 2023-11-09T03:46:34.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.559+0000 7fc5195a9640 1 -- 172.21.15.76:0/973310917 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5140cc880 con 0x7fc514153e60 2023-11-09T03:46:34.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.560+0000 7fc5195a9640 1 -- 172.21.15.76:0/973310917 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5140ccaa0 con 0x7fc514153e60 2023-11-09T03:46:34.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.561+0000 7fc50b7fe640 1 -- 172.21.15.76:0/973310917 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc50c00ad30 con 0x7fc514153e60 2023-11-09T03:46:34.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.561+0000 7fc50b7fe640 1 --2- 172.21.15.76:0/973310917 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc4f406ae70 0x7fc4f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:34.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.561+0000 7fc50b7fe640 1 -- 172.21.15.76:0/973310917 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fc50c088f80 con 0x7fc514153e60 2023-11-09T03:46:34.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.561+0000 7fc512ffd640 1 --2- 172.21.15.76:0/973310917 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc4f406ae70 0x7fc4f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:34.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.562+0000 7fc512ffd640 1 --2- 172.21.15.76:0/973310917 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc4f406ae70 0x7fc4f406d330 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc504036000 tx=0x7fc504008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:34.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.565+0000 7fc50b7fe640 1 -- 172.21.15.76:0/973310917 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc50c0bbca0 con 0x7fc514153e60 2023-11-09T03:46:34.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.731+0000 7fc5195a9640 1 -- 172.21.15.76:0/973310917 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc514074290 con 0x7fc514153e60 2023-11-09T03:46:34.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.731+0000 7fc50b7fe640 1 -- 172.21.15.76:0/973310917 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc50c018020 con 0x7fc514153e60 2023-11-09T03:46:34.732 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:46:34.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.733+0000 7fc5195a9640 1 -- 172.21.15.76:0/973310917 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc4f406ae70 msgr2=0x7fc4f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:34.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.733+0000 7fc5195a9640 1 --2- 172.21.15.76:0/973310917 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc4f406ae70 0x7fc4f406d330 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc504036000 tx=0x7fc504008030 comp rx=0 tx=0).stop 2023-11-09T03:46:34.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.733+0000 7fc5195a9640 1 -- 172.21.15.76:0/973310917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc514153e60 msgr2=0x7fc5140b7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:34.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.733+0000 7fc5195a9640 1 --2- 172.21.15.76:0/973310917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc514153e60 0x7fc5140b7020 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc50c00c7a0 tx=0x7fc50c00cc60 comp rx=0 tx=0).stop 2023-11-09T03:46:34.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.733+0000 7fc5195a9640 1 -- 172.21.15.76:0/973310917 shutdown_connections 2023-11-09T03:46:34.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.733+0000 7fc5195a9640 1 --2- 172.21.15.76:0/973310917 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc4f406ae70 0x7fc4f406d330 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.733+0000 7fc5195a9640 1 --2- 172.21.15.76:0/973310917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc514153e60 0x7fc5140b7020 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.733+0000 7fc5195a9640 1 --2- 172.21.15.76:0/973310917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5140cd6d0 0x7fc5140cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.733+0000 7fc5195a9640 1 --2- 172.21.15.76:0/973310917 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5140b4880 0x7fc5140ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.733+0000 7fc5195a9640 1 -- 172.21.15.76:0/973310917 >> 172.21.15.76:0/973310917 conn(0x7fc5140cf880 msgr2=0x7fc5140d2300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:34.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.733+0000 7fc5195a9640 1 -- 172.21.15.76:0/973310917 shutdown_connections 2023-11-09T03:46:34.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.733+0000 7fc5195a9640 1 -- 172.21.15.76:0/973310917 wait complete. 2023-11-09T03:46:34.744 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:46:34.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.847+0000 7fc8e292c640 1 Processor -- start 2023-11-09T03:46:34.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.847+0000 7fc8e292c640 1 -- start start 2023-11-09T03:46:34.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.848+0000 7fc8e292c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc8dc15ad30 0x7fc8dc0adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:34.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.848+0000 7fc8e292c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8dc070e20 0x7fc8dc15e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:34.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.848+0000 7fc8e292c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8dc0c3900 con 0x7fc8dc0ad730 2023-11-09T03:46:34.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.848+0000 7fc8e292c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8dc0ae260 con 0x7fc8dc15ad30 2023-11-09T03:46:34.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.848+0000 7fc8e292c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc0c3be0 con 0x7fc8dc070e20 2023-11-09T03:46:34.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.848+0000 7fc8dbfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc8dc15ad30 0x7fc8dc0adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:34.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.848+0000 7fc8db7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc8dc0ad730 0x7fc8dc070710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40672/0 (socket says 172.21.15.76:40672) 2023-11-09T03:46:34.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.848+0000 7fc8dbfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc8dc15ad30 0x7fc8dc0adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34970/0 (socket says 172.21.15.76:34970) 2023-11-09T03:46:34.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.848+0000 7fc8db7fe640 1 -- 172.21.15.76:0/1539720474 learned_addr learned my addr 172.21.15.76:0/1539720474 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:34.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.848+0000 7fc8dbfff640 1 -- 172.21.15.76:0/1539720474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8dc070e20 msgr2=0x7fc8dc15e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:34.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.848+0000 7fc8dbfff640 1 --2- 172.21.15.76:0/1539720474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8dc070e20 0x7fc8dc15e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.848+0000 7fc8dbfff640 1 -- 172.21.15.76:0/1539720474 >> v1:172.21.15.76:6789/0 conn(0x7fc8dc0ad730 legacy=0x7fc8dc070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:34.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.848+0000 7fc8daffd640 1 -- 172.21.15.76:0/1539720474 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1327434 0 0) 0x7fc8dc0c3900 con 0x7fc8dc0ad730 2023-11-09T03:46:34.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.849+0000 7fc8dbfff640 1 -- 172.21.15.76:0/1539720474 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8dc0c3ec0 con 0x7fc8dc15ad30 2023-11-09T03:46:34.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.849+0000 7fc8dbfff640 1 --2- 172.21.15.76:0/1539720474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8dc15ad30 0x7fc8dc0adc80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc0097a0 tx=0x7fc8cc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0f7b6bdea4ade0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:34.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.849+0000 7fc8daffd640 1 -- 172.21.15.76:0/1539720474 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc02e070 con 0x7fc8dc15ad30 2023-11-09T03:46:34.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.849+0000 7fc8daffd640 1 -- 172.21.15.76:0/1539720474 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8cc0315a0 con 0x7fc8dc15ad30 2023-11-09T03:46:34.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.849+0000 7fc8daffd640 1 -- 172.21.15.76:0/1539720474 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc031930 con 0x7fc8dc15ad30 2023-11-09T03:46:34.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.850+0000 7fc8e292c640 1 -- 172.21.15.76:0/1539720474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8dc15ad30 msgr2=0x7fc8dc0adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:34.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.850+0000 7fc8e292c640 1 --2- 172.21.15.76:0/1539720474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8dc15ad30 0x7fc8dc0adc80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc0097a0 tx=0x7fc8cc02b840 comp rx=0 tx=0).stop 2023-11-09T03:46:34.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.850+0000 7fc8e292c640 1 -- 172.21.15.76:0/1539720474 shutdown_connections 2023-11-09T03:46:34.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.850+0000 7fc8e292c640 1 --2- 172.21.15.76:0/1539720474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8dc070e20 0x7fc8dc15e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.850+0000 7fc8e292c640 1 --2- 172.21.15.76:0/1539720474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8dc15ad30 0x7fc8dc0adc80 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.850+0000 7fc8e292c640 1 -- 172.21.15.76:0/1539720474 >> 172.21.15.76:0/1539720474 conn(0x7fc8dc0ce950 msgr2=0x7fc8dc0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:34.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.850+0000 7fc8e292c640 1 -- 172.21.15.76:0/1539720474 shutdown_connections 2023-11-09T03:46:34.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.850+0000 7fc8e292c640 1 -- 172.21.15.76:0/1539720474 wait complete. 2023-11-09T03:46:34.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.851+0000 7fc8e292c640 1 Processor -- start 2023-11-09T03:46:34.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.851+0000 7fc8e292c640 1 -- start start 2023-11-09T03:46:34.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.852+0000 7fc8e292c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8dc070e20 0x7fc8dc156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:34.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.852+0000 7fc8e292c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8dc0ad730 0x7fc8dc0cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:34.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.852+0000 7fc8e292c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8dc15ad30 0x7fc8dc1453e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:34.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.852+0000 7fc8e292c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc0ccbf0 con 0x7fc8dc0ad730 2023-11-09T03:46:34.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.852+0000 7fc8e292c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc0cda60 con 0x7fc8dc15ad30 2023-11-09T03:46:34.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.852+0000 7fc8e292c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc0cce20 con 0x7fc8dc070e20 2023-11-09T03:46:34.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.852+0000 7fc8db7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8dc0ad730 0x7fc8dc0cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:34.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.852+0000 7fc8db7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8dc0ad730 0x7fc8dc0cc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34986/0 (socket says 172.21.15.76:34986) 2023-11-09T03:46:34.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.852+0000 7fc8db7fe640 1 -- 172.21.15.76:0/3249098637 learned_addr learned my addr 172.21.15.76:0/3249098637 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:34.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.852+0000 7fc8dbfff640 1 --2- 172.21.15.76:0/3249098637 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8dc070e20 0x7fc8dc156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:34.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.852+0000 7fc8e0ec9640 1 --2- 172.21.15.76:0/3249098637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8dc15ad30 0x7fc8dc1453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:34.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.852+0000 7fc8db7fe640 1 -- 172.21.15.76:0/3249098637 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8dc070e20 msgr2=0x7fc8dc156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:34.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.852+0000 7fc8db7fe640 1 --2- 172.21.15.76:0/3249098637 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8dc070e20 0x7fc8dc156d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.852+0000 7fc8db7fe640 1 -- 172.21.15.76:0/3249098637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8dc15ad30 msgr2=0x7fc8dc1453e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:34.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.852+0000 7fc8db7fe640 1 --2- 172.21.15.76:0/3249098637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8dc15ad30 0x7fc8dc1453e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:34.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.852+0000 7fc8db7fe640 1 -- 172.21.15.76:0/3249098637 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8cc009400 con 0x7fc8dc0ad730 2023-11-09T03:46:34.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.852+0000 7fc8e0ec9640 1 --2- 172.21.15.76:0/3249098637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8dc15ad30 0x7fc8dc1453e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:34.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.853+0000 7fc8db7fe640 1 --2- 172.21.15.76:0/3249098637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8dc0ad730 0x7fc8dc0cc6b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc8d000c4d0 tx=0x7fc8d000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:34.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.853+0000 7fc8d8ff9640 1 -- 172.21.15.76:0/3249098637 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d00074e0 con 0x7fc8dc0ad730 2023-11-09T03:46:34.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.853+0000 7fc8d8ff9640 1 -- 172.21.15.76:0/3249098637 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d0007660 con 0x7fc8dc0ad730 2023-11-09T03:46:34.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.853+0000 7fc8e292c640 1 -- 172.21.15.76:0/3249098637 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8dc145920 con 0x7fc8dc0ad730 2023-11-09T03:46:34.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.853+0000 7fc8d8ff9640 1 -- 172.21.15.76:0/3249098637 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0004a20 con 0x7fc8dc0ad730 2023-11-09T03:46:34.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.853+0000 7fc8e292c640 1 -- 172.21.15.76:0/3249098637 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8dc0cd410 con 0x7fc8dc0ad730 2023-11-09T03:46:34.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.854+0000 7fc8e292c640 1 -- 172.21.15.76:0/3249098637 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8dc0cdbd0 con 0x7fc8dc0ad730 2023-11-09T03:46:34.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.854+0000 7fc8d8ff9640 1 -- 172.21.15.76:0/3249098637 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc8d0004b80 con 0x7fc8dc0ad730 2023-11-09T03:46:34.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.855+0000 7fc8d8ff9640 1 --2- 172.21.15.76:0/3249098637 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc8c806ae70 0x7fc8c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:34.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.855+0000 7fc8d8ff9640 1 -- 172.21.15.76:0/3249098637 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fc8d0088ec0 con 0x7fc8dc0ad730 2023-11-09T03:46:34.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.855+0000 7fc8dbfff640 1 --2- 172.21.15.76:0/3249098637 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc8c806ae70 0x7fc8c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:34.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.855+0000 7fc8dbfff640 1 --2- 172.21.15.76:0/3249098637 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc8c806ae70 0x7fc8c806d330 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc036000 tx=0x7fc8cc03d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:34.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:34.859+0000 7fc8d8ff9640 1 -- 172.21.15.76:0/3249098637 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc8d0053c20 con 0x7fc8dc0ad730 2023-11-09T03:46:35.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.027+0000 7fc8e292c640 1 -- 172.21.15.76:0/3249098637 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc8dc074290 con 0x7fc8dc0ad730 2023-11-09T03:46:35.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.027+0000 7fc8d8ff9640 1 -- 172.21.15.76:0/3249098637 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc8d0018020 con 0x7fc8dc0ad730 2023-11-09T03:46:35.028 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:46:35.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.029+0000 7fc8e292c640 1 -- 172.21.15.76:0/3249098637 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc8c806ae70 msgr2=0x7fc8c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:35.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.029+0000 7fc8e292c640 1 --2- 172.21.15.76:0/3249098637 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc8c806ae70 0x7fc8c806d330 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc036000 tx=0x7fc8cc03d040 comp rx=0 tx=0).stop 2023-11-09T03:46:35.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.029+0000 7fc8e292c640 1 -- 172.21.15.76:0/3249098637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8dc0ad730 msgr2=0x7fc8dc0cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:35.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.029+0000 7fc8e292c640 1 --2- 172.21.15.76:0/3249098637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8dc0ad730 0x7fc8dc0cc6b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc8d000c4d0 tx=0x7fc8d000c990 comp rx=0 tx=0).stop 2023-11-09T03:46:35.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.029+0000 7fc8e292c640 1 -- 172.21.15.76:0/3249098637 shutdown_connections 2023-11-09T03:46:35.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.029+0000 7fc8e292c640 1 --2- 172.21.15.76:0/3249098637 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc8c806ae70 0x7fc8c806d330 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.029+0000 7fc8e292c640 1 --2- 172.21.15.76:0/3249098637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8dc15ad30 0x7fc8dc1453e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.029+0000 7fc8e292c640 1 --2- 172.21.15.76:0/3249098637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8dc0ad730 0x7fc8dc0cc6b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.029+0000 7fc8e292c640 1 --2- 172.21.15.76:0/3249098637 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8dc070e20 0x7fc8dc156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.029+0000 7fc8e292c640 1 -- 172.21.15.76:0/3249098637 >> 172.21.15.76:0/3249098637 conn(0x7fc8dc0ce950 msgr2=0x7fc8dc06cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:35.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.029+0000 7fc8e292c640 1 -- 172.21.15.76:0/3249098637 shutdown_connections 2023-11-09T03:46:35.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.029+0000 7fc8e292c640 1 -- 172.21.15.76:0/3249098637 wait complete. 2023-11-09T03:46:35.041 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-09T03:46:35.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.143+0000 7f5118dc0640 1 Processor -- start 2023-11-09T03:46:35.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.143+0000 7f5118dc0640 1 -- start start 2023-11-09T03:46:35.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.144+0000 7f5118dc0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5114153850 0x7f511415a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:35.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.144+0000 7f5118dc0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f51140cc9a0 0x7f51140c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:35.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.144+0000 7f5118dc0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51140c3860 con 0x7f51140cc520 2023-11-09T03:46:35.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.144+0000 7f5118dc0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51140c9250 con 0x7f51140cc9a0 2023-11-09T03:46:35.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.144+0000 7f5118dc0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5114153c30 con 0x7f5114153850 2023-11-09T03:46:35.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.144+0000 7f5112d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f51140cc9a0 0x7f51140c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:35.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.144+0000 7f5111d9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f51140cc520 0x7f5114070fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40684/0 (socket says 172.21.15.76:40684) 2023-11-09T03:46:35.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.144+0000 7f5111d9b640 1 -- 172.21.15.76:0/2883503379 learned_addr learned my addr 172.21.15.76:0/2883503379 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:35.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.145+0000 7f511259c640 1 --2- 172.21.15.76:0/2883503379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5114153850 0x7f511415a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:35.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.145+0000 7f5112d9d640 1 -- 172.21.15.76:0/2883503379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5114153850 msgr2=0x7f511415a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:35.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.145+0000 7f5112d9d640 1 --2- 172.21.15.76:0/2883503379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5114153850 0x7f511415a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.145+0000 7f5112d9d640 1 -- 172.21.15.76:0/2883503379 >> v1:172.21.15.76:6789/0 conn(0x7f51140cc520 legacy=0x7f5114070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:35.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.145+0000 7f5112d9d640 1 -- 172.21.15.76:0/2883503379 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511415a700 con 0x7f51140cc9a0 2023-11-09T03:46:35.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.145+0000 7f511159a640 1 -- 172.21.15.76:0/2883503379 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1385348636 0 0) 0x7f51140c3860 con 0x7f51140cc520 2023-11-09T03:46:35.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.145+0000 7f5112d9d640 1 --2- 172.21.15.76:0/2883503379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51140cc9a0 0x7f51140c3cf0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f510c00ba40 tx=0x7f510c02de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd8cdf1e81015d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:35.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.145+0000 7f511159a640 1 -- 172.21.15.76:0/2883503379 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c02ec10 con 0x7f51140cc9a0 2023-11-09T03:46:35.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.146+0000 7f511159a640 1 -- 172.21.15.76:0/2883503379 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510c032070 con 0x7f51140cc9a0 2023-11-09T03:46:35.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.146+0000 7f511159a640 1 -- 172.21.15.76:0/2883503379 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c038400 con 0x7f51140cc9a0 2023-11-09T03:46:35.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.146+0000 7f5118dc0640 1 -- 172.21.15.76:0/2883503379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51140cc9a0 msgr2=0x7f51140c3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:35.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.146+0000 7f5118dc0640 1 --2- 172.21.15.76:0/2883503379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51140cc9a0 0x7f51140c3cf0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f510c00ba40 tx=0x7f510c02de60 comp rx=0 tx=0).stop 2023-11-09T03:46:35.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.146+0000 7f5118dc0640 1 -- 172.21.15.76:0/2883503379 shutdown_connections 2023-11-09T03:46:35.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.146+0000 7f5118dc0640 1 --2- 172.21.15.76:0/2883503379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51140cc9a0 0x7f51140c3cf0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.146+0000 7f5118dc0640 1 --2- 172.21.15.76:0/2883503379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5114153850 0x7f511415a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.146+0000 7f5118dc0640 1 -- 172.21.15.76:0/2883503379 >> 172.21.15.76:0/2883503379 conn(0x7f51140ce950 msgr2=0x7f51140ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:35.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.146+0000 7f5118dc0640 1 -- 172.21.15.76:0/2883503379 shutdown_connections 2023-11-09T03:46:35.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.146+0000 7f5118dc0640 1 -- 172.21.15.76:0/2883503379 wait complete. 2023-11-09T03:46:35.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.147+0000 7f5118dc0640 1 Processor -- start 2023-11-09T03:46:35.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.147+0000 7f5118dc0640 1 -- start start 2023-11-09T03:46:35.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.148+0000 7f5118dc0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51140cc520 0x7f51140b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:35.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.148+0000 7f5118dc0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51140cc9a0 0x7f51140acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:35.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.148+0000 7f5118dc0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5114153850 0x7f5114153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:35.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.148+0000 7f5118dc0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51140b1a70 con 0x7f51140cc520 2023-11-09T03:46:35.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.148+0000 7f5118dc0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51140b4360 con 0x7f5114153850 2023-11-09T03:46:35.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.148+0000 7f5118dc0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51140b4640 con 0x7f51140cc9a0 2023-11-09T03:46:35.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.148+0000 7f511259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51140cc520 0x7f51140b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:35.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.148+0000 7f511259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51140cc520 0x7f51140b1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35000/0 (socket says 172.21.15.76:35000) 2023-11-09T03:46:35.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.148+0000 7f511259c640 1 -- 172.21.15.76:0/3950819817 learned_addr learned my addr 172.21.15.76:0/3950819817 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:35.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.149+0000 7f511259c640 1 -- 172.21.15.76:0/3950819817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51140cc9a0 msgr2=0x7f51140acae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:35.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.149+0000 7f511259c640 1 --2- 172.21.15.76:0/3950819817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51140cc9a0 0x7f51140acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.149+0000 7f511259c640 1 -- 172.21.15.76:0/3950819817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5114153850 msgr2=0x7f5114153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:35.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.149+0000 7f511259c640 1 --2- 172.21.15.76:0/3950819817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5114153850 0x7f5114153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.149+0000 7f511259c640 1 -- 172.21.15.76:0/3950819817 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510c00b6a0 con 0x7f51140cc520 2023-11-09T03:46:35.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.149+0000 7f511259c640 1 --2- 172.21.15.76:0/3950819817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51140cc520 0x7f51140b1530 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f510400ce10 tx=0x7f510400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:35.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.149+0000 7f5102ffd640 1 -- 172.21.15.76:0/3950819817 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5104003b90 con 0x7f51140cc520 2023-11-09T03:46:35.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.149+0000 7f5102ffd640 1 -- 172.21.15.76:0/3950819817 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5104003cf0 con 0x7f51140cc520 2023-11-09T03:46:35.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.149+0000 7f5102ffd640 1 -- 172.21.15.76:0/3950819817 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5104003ea0 con 0x7f51140cc520 2023-11-09T03:46:35.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.149+0000 7f5118dc0640 1 -- 172.21.15.76:0/3950819817 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51140ac5c0 con 0x7f51140cc520 2023-11-09T03:46:35.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.149+0000 7f5118dc0640 1 -- 172.21.15.76:0/3950819817 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51140ad020 con 0x7f51140cc520 2023-11-09T03:46:35.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.150+0000 7f5118dc0640 1 -- 172.21.15.76:0/3950819817 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51140ac800 con 0x7f51140cc520 2023-11-09T03:46:35.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.150+0000 7f5102ffd640 1 -- 172.21.15.76:0/3950819817 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5104004240 con 0x7f51140cc520 2023-11-09T03:46:35.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.151+0000 7f5102ffd640 1 --2- 172.21.15.76:0/3950819817 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f50f406ae70 0x7f50f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:35.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.151+0000 7f5102ffd640 1 -- 172.21.15.76:0/3950819817 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f5104012070 con 0x7f51140cc520 2023-11-09T03:46:35.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.151+0000 7f5111d9b640 1 --2- 172.21.15.76:0/3950819817 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f50f406ae70 0x7f50f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:35.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.151+0000 7f5111d9b640 1 --2- 172.21.15.76:0/3950819817 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f50f406ae70 0x7f50f406d330 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5108007fd0 tx=0x7f5108007960 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:35.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.155+0000 7f5102ffd640 1 -- 172.21.15.76:0/3950819817 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f51140ac800 con 0x7f51140cc520 2023-11-09T03:46:35.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.321+0000 7f5118dc0640 1 -- 172.21.15.76:0/3950819817 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f51140ac800 con 0x7f51140cc520 2023-11-09T03:46:35.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.321+0000 7f5102ffd640 1 -- 172.21.15.76:0/3950819817 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f51140ac800 con 0x7f51140cc520 2023-11-09T03:46:35.322 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:46:35.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.323+0000 7f5118dc0640 1 -- 172.21.15.76:0/3950819817 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f50f406ae70 msgr2=0x7f50f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:35.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.323+0000 7f5118dc0640 1 --2- 172.21.15.76:0/3950819817 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f50f406ae70 0x7f50f406d330 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5108007fd0 tx=0x7f5108007960 comp rx=0 tx=0).stop 2023-11-09T03:46:35.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.323+0000 7f5118dc0640 1 -- 172.21.15.76:0/3950819817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51140cc520 msgr2=0x7f51140b1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:35.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.323+0000 7f5118dc0640 1 --2- 172.21.15.76:0/3950819817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51140cc520 0x7f51140b1530 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f510400ce10 tx=0x7f510400a660 comp rx=0 tx=0).stop 2023-11-09T03:46:35.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.323+0000 7f5118dc0640 1 -- 172.21.15.76:0/3950819817 shutdown_connections 2023-11-09T03:46:35.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.323+0000 7f5118dc0640 1 --2- 172.21.15.76:0/3950819817 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f50f406ae70 0x7f50f406d330 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.323+0000 7f5118dc0640 1 --2- 172.21.15.76:0/3950819817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5114153850 0x7f5114153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.323+0000 7f5118dc0640 1 --2- 172.21.15.76:0/3950819817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51140cc9a0 0x7f51140acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.323+0000 7f5118dc0640 1 --2- 172.21.15.76:0/3950819817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51140cc520 0x7f51140b1530 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.323+0000 7f5118dc0640 1 -- 172.21.15.76:0/3950819817 >> 172.21.15.76:0/3950819817 conn(0x7f51140ce950 msgr2=0x7f51141526f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:35.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.323+0000 7f5118dc0640 1 -- 172.21.15.76:0/3950819817 shutdown_connections 2023-11-09T03:46:35.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.323+0000 7f5118dc0640 1 -- 172.21.15.76:0/3950819817 wait complete. 2023-11-09T03:46:35.335 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-09T03:46:35.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.437+0000 7f190cd55640 1 Processor -- start 2023-11-09T03:46:35.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.437+0000 7f190cd55640 1 -- start start 2023-11-09T03:46:35.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.438+0000 7f190cd55640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1908153e60 0x7f190815dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:35.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.438+0000 7f190cd55640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f19080b4880 0x7f19080b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:35.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.438+0000 7f190cd55640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19080cde60 con 0x7f19080cd6d0 2023-11-09T03:46:35.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.438+0000 7f190cd55640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19080b5230 con 0x7f19080b4880 2023-11-09T03:46:35.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.438+0000 7f190cd55640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1908160f90 con 0x7f1908153e60 2023-11-09T03:46:35.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.438+0000 7f190659c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1908153e60 0x7f190815dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:35.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.438+0000 7f1906d9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f19080b4880 0x7f19080b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:35.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.438+0000 7f190659c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1908153e60 0x7f190815dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:45036/0 (socket says 172.21.15.76:45036) 2023-11-09T03:46:35.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.438+0000 7f1905d9b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f19080cd6d0 0x7f1908161420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:56050/0 (socket says 172.21.15.76:56050) 2023-11-09T03:46:35.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.438+0000 7f1906d9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f19080b4880 0x7f19080b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:44456/0 (socket says 172.21.15.76:44456) 2023-11-09T03:46:35.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.438+0000 7f190659c640 1 -- 172.21.15.76:0/1177532524 learned_addr learned my addr 172.21.15.76:0/1177532524 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:35.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.439+0000 7f1906d9d640 1 -- 172.21.15.76:0/1177532524 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1908153e60 msgr2=0x7f190815dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:35.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.439+0000 7f1906d9d640 1 --2- 172.21.15.76:0/1177532524 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1908153e60 0x7f190815dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.439+0000 7f1906d9d640 1 -- 172.21.15.76:0/1177532524 >> v1:172.21.15.113:6789/0 conn(0x7f19080cd6d0 legacy=0x7f1908161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:35.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.439+0000 7f190559a640 1 -- 172.21.15.76:0/1177532524 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 656818117 0 0) 0x7f19080cde60 con 0x7f19080cd6d0 2023-11-09T03:46:35.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.439+0000 7f1906d9d640 1 -- 172.21.15.76:0/1177532524 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190815e400 con 0x7f19080b4880 2023-11-09T03:46:35.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.439+0000 7f190659c640 1 --2- 172.21.15.76:0/1177532524 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1908153e60 0x7f190815dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:46:35.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.439+0000 7f1906d9d640 1 --2- 172.21.15.76:0/1177532524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19080b4880 0x7f19080b4cf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1900009d90 tx=0x7f190002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6707ef32d7b3ae06 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:35.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.440+0000 7f190559a640 1 -- 172.21.15.76:0/1177532524 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190002cc50 con 0x7f19080b4880 2023-11-09T03:46:35.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.440+0000 7f190559a640 1 -- 172.21.15.76:0/1177532524 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190002cdb0 con 0x7f19080b4880 2023-11-09T03:46:35.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.440+0000 7f190559a640 1 -- 172.21.15.76:0/1177532524 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1900034660 con 0x7f19080b4880 2023-11-09T03:46:35.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.440+0000 7f190cd55640 1 -- 172.21.15.76:0/1177532524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19080b4880 msgr2=0x7f19080b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:35.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.440+0000 7f190cd55640 1 --2- 172.21.15.76:0/1177532524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19080b4880 0x7f19080b4cf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1900009d90 tx=0x7f190002bea0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.440+0000 7f190cd55640 1 -- 172.21.15.76:0/1177532524 shutdown_connections 2023-11-09T03:46:35.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.440+0000 7f190cd55640 1 --2- 172.21.15.76:0/1177532524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19080b4880 0x7f19080b4cf0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.440+0000 7f190cd55640 1 --2- 172.21.15.76:0/1177532524 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1908153e60 0x7f190815dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.440+0000 7f190cd55640 1 -- 172.21.15.76:0/1177532524 >> 172.21.15.76:0/1177532524 conn(0x7f19080cf880 msgr2=0x7f19080cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:35.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.440+0000 7f190cd55640 1 -- 172.21.15.76:0/1177532524 shutdown_connections 2023-11-09T03:46:35.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.440+0000 7f190cd55640 1 -- 172.21.15.76:0/1177532524 wait complete. 2023-11-09T03:46:35.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.441+0000 7f190cd55640 1 Processor -- start 2023-11-09T03:46:35.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.442+0000 7f190cd55640 1 -- start start 2023-11-09T03:46:35.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.442+0000 7f190cd55640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19080b4880 0x7f1908157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:35.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.442+0000 7f190cd55640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19080cd6d0 0x7f190815e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:35.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.442+0000 7f190cd55640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1908153e60 0x7f190815eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:35.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.442+0000 7f190cd55640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190815a9f0 con 0x7f19080cd6d0 2023-11-09T03:46:35.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.442+0000 7f190cd55640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19081596b0 con 0x7f19080b4880 2023-11-09T03:46:35.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.442+0000 7f190cd55640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1908159950 con 0x7f1908153e60 2023-11-09T03:46:35.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.442+0000 7f1905d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19080cd6d0 0x7f190815e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:35.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.442+0000 7f1905d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19080cd6d0 0x7f190815e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35014/0 (socket says 172.21.15.76:35014) 2023-11-09T03:46:35.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.442+0000 7f1905d9b640 1 -- 172.21.15.76:0/3669887162 learned_addr learned my addr 172.21.15.76:0/3669887162 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:35.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.442+0000 7f190659c640 1 --2- 172.21.15.76:0/3669887162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19080b4880 0x7f1908157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:35.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.442+0000 7f1906d9d640 1 --2- 172.21.15.76:0/3669887162 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1908153e60 0x7f190815eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:35.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.443+0000 7f1905d9b640 1 -- 172.21.15.76:0/3669887162 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1908153e60 msgr2=0x7f190815eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:35.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.443+0000 7f1905d9b640 1 --2- 172.21.15.76:0/3669887162 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1908153e60 0x7f190815eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.443+0000 7f1905d9b640 1 -- 172.21.15.76:0/3669887162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19080b4880 msgr2=0x7f1908157b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:35.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.443+0000 7f1905d9b640 1 --2- 172.21.15.76:0/3669887162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19080b4880 0x7f1908157b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.443+0000 7f1905d9b640 1 -- 172.21.15.76:0/3669887162 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f80094d0 con 0x7f19080cd6d0 2023-11-09T03:46:35.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.443+0000 7f1906d9d640 1 --2- 172.21.15.76:0/3669887162 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1908153e60 0x7f190815eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:35.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.443+0000 7f1905d9b640 1 --2- 172.21.15.76:0/3669887162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19080cd6d0 0x7f190815e5c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00c4b0 tx=0x7f18fc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:35.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.443+0000 7f18f6ffd640 1 -- 172.21.15.76:0/3669887162 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc0074e0 con 0x7f19080cd6d0 2023-11-09T03:46:35.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.443+0000 7f18f6ffd640 1 -- 172.21.15.76:0/3669887162 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18fc007660 con 0x7f19080cd6d0 2023-11-09T03:46:35.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.443+0000 7f190cd55640 1 -- 172.21.15.76:0/3669887162 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1900009a70 con 0x7f19080cd6d0 2023-11-09T03:46:35.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.443+0000 7f18f6ffd640 1 -- 172.21.15.76:0/3669887162 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc004a20 con 0x7f19080cd6d0 2023-11-09T03:46:35.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.443+0000 7f190cd55640 1 -- 172.21.15.76:0/3669887162 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1908159f10 con 0x7f19080cd6d0 2023-11-09T03:46:35.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.444+0000 7f190cd55640 1 -- 172.21.15.76:0/3669887162 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1908074290 con 0x7f19080cd6d0 2023-11-09T03:46:35.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.444+0000 7f18f6ffd640 1 -- 172.21.15.76:0/3669887162 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f18fc004b80 con 0x7f19080cd6d0 2023-11-09T03:46:35.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.445+0000 7f18f6ffd640 1 --2- 172.21.15.76:0/3669887162 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f18e806ae70 0x7f18e806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:35.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.445+0000 7f18f6ffd640 1 -- 172.21.15.76:0/3669887162 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f18fc088d70 con 0x7f19080cd6d0 2023-11-09T03:46:35.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.445+0000 7f190659c640 1 --2- 172.21.15.76:0/3669887162 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f18e806ae70 0x7f18e806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:35.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.446+0000 7f190659c640 1 --2- 172.21.15.76:0/3669887162 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f18e806ae70 0x7f18e806d330 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f19080d08d0 tx=0x7f18f8009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:35.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.449+0000 7f18f6ffd640 1 -- 172.21.15.76:0/3669887162 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f18fc053c40 con 0x7f19080cd6d0 2023-11-09T03:46:35.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.629+0000 7f190cd55640 1 -- 172.21.15.76:0/3669887162 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f19080d0b80 con 0x7f19080cd6d0 2023-11-09T03:46:35.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.629+0000 7f18f6ffd640 1 -- 172.21.15.76:0/3669887162 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f18fc018020 con 0x7f19080cd6d0 2023-11-09T03:46:35.630 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2023-11-09T03:46:35.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.631+0000 7f190cd55640 1 -- 172.21.15.76:0/3669887162 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f18e806ae70 msgr2=0x7f18e806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:35.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.631+0000 7f190cd55640 1 --2- 172.21.15.76:0/3669887162 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f18e806ae70 0x7f18e806d330 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f19080d08d0 tx=0x7f18f8009240 comp rx=0 tx=0).stop 2023-11-09T03:46:35.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.631+0000 7f190cd55640 1 -- 172.21.15.76:0/3669887162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19080cd6d0 msgr2=0x7f190815e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:35.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.631+0000 7f190cd55640 1 --2- 172.21.15.76:0/3669887162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19080cd6d0 0x7f190815e5c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00c4b0 tx=0x7f18fc00c970 comp rx=0 tx=0).stop 2023-11-09T03:46:35.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.631+0000 7f190cd55640 1 -- 172.21.15.76:0/3669887162 shutdown_connections 2023-11-09T03:46:35.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.631+0000 7f190cd55640 1 --2- 172.21.15.76:0/3669887162 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f18e806ae70 0x7f18e806d330 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.631+0000 7f190cd55640 1 --2- 172.21.15.76:0/3669887162 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1908153e60 0x7f190815eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.631+0000 7f190cd55640 1 --2- 172.21.15.76:0/3669887162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19080cd6d0 0x7f190815e5c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.631+0000 7f190cd55640 1 --2- 172.21.15.76:0/3669887162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19080b4880 0x7f1908157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.631+0000 7f190cd55640 1 -- 172.21.15.76:0/3669887162 >> 172.21.15.76:0/3669887162 conn(0x7f19080cf880 msgr2=0x7f19080d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:35.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.631+0000 7f190cd55640 1 -- 172.21.15.76:0/3669887162 shutdown_connections 2023-11-09T03:46:35.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.631+0000 7f190cd55640 1 -- 172.21.15.76:0/3669887162 wait complete. 2023-11-09T03:46:35.642 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:46:35.643 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:46:35.662 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:46:35.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.763+0000 7f21b1125640 1 Processor -- start 2023-11-09T03:46:35.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.763+0000 7f21b1125640 1 -- start start 2023-11-09T03:46:35.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.763+0000 7f21b1125640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f21ac0cbeb0 0x7f21ac0b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:35.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.763+0000 7f21b1125640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21ac0b1590 con 0x7f21ac0aec80 2023-11-09T03:46:35.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.763+0000 7f21b1125640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21ac06a8e0 con 0x7f21ac0af060 2023-11-09T03:46:35.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.763+0000 7f21b1125640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21ac155a40 con 0x7f21ac0cbeb0 2023-11-09T03:46:35.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.763+0000 7f21aad9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f21ac0aec80 0x7f21ac06a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40686/0 (socket says 172.21.15.76:40686) 2023-11-09T03:46:35.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.764+0000 7f21aad9d640 1 -- 172.21.15.76:0/486059773 learned_addr learned my addr 172.21.15.76:0/486059773 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:35.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.764+0000 7f21a9d9b640 1 -- 172.21.15.76:0/486059773 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 294090573 0 0) 0x7f21ac0b1590 con 0x7f21ac0aec80 2023-11-09T03:46:35.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.764+0000 7f21ab59e640 1 --2- 172.21.15.76:0/486059773 >> v2:172.21.15.113:3300/0 conn(0x7f21ac0cbeb0 0x7f21ac0b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:35.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.764+0000 7f21a9d9b640 1 -- 172.21.15.76:0/486059773 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2194003520 con 0x7f21ac0aec80 2023-11-09T03:46:35.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.764+0000 7f21a9d9b640 1 -- 172.21.15.76:0/486059773 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2031461533 0 0) 0x7f2194003520 con 0x7f21ac0aec80 2023-11-09T03:46:35.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.764+0000 7f21a9d9b640 1 -- 172.21.15.76:0/486059773 >> v2:172.21.15.113:3300/0 conn(0x7f21ac0cbeb0 msgr2=0x7f21ac0b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:35.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.764+0000 7f21a9d9b640 1 --2- 172.21.15.76:0/486059773 >> v2:172.21.15.113:3300/0 conn(0x7f21ac0cbeb0 0x7f21ac0b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.764+0000 7f21a9d9b640 1 -- 172.21.15.76:0/486059773 >> v1:172.21.15.113:6789/0 conn(0x7f21ac0af060 legacy=0x7f21ac0cb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:35.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.764+0000 7f21a9d9b640 1 -- 172.21.15.76:0/486059773 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac0b1800 con 0x7f21ac0aec80 2023-11-09T03:46:35.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.764+0000 7f21a9d9b640 1 -- 172.21.15.76:0/486059773 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f219c0017f0 con 0x7f21ac0aec80 2023-11-09T03:46:35.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.764+0000 7f21a9d9b640 1 -- 172.21.15.76:0/486059773 >> v1:172.21.15.76:6789/0 conn(0x7f21ac0aec80 legacy=0x7f21ac06a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:35.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.764+0000 7f21a9d9b640 1 --2- 172.21.15.76:0/486059773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2194003970 0x7f2194003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:35.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.764+0000 7f21a9d9b640 1 --2- 172.21.15.76:0/486059773 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21940047c0 0x7f2194004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:35.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.764+0000 7f21a9d9b640 1 --2- 172.21.15.76:0/486059773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2194005180 0x7f219400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:35.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.764+0000 7f21a9d9b640 1 -- 172.21.15.76:0/486059773 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac155a40 con 0x7f2194003970 2023-11-09T03:46:35.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.764+0000 7f21a9d9b640 1 -- 172.21.15.76:0/486059773 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219400c020 con 0x7f2194005180 2023-11-09T03:46:35.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.764+0000 7f21a9d9b640 1 -- 172.21.15.76:0/486059773 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f219400c1e0 con 0x7f21940047c0 2023-11-09T03:46:35.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.764+0000 7f21aad9d640 1 --2- 172.21.15.76:0/486059773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2194003970 0x7f2194003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:35.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.764+0000 7f21ab59e640 1 --2- 172.21.15.76:0/486059773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2194005180 0x7f219400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:35.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.764+0000 7f21aa59c640 1 --2- 172.21.15.76:0/486059773 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21940047c0 0x7f2194004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:35.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.764+0000 7f21aad9d640 1 -- 172.21.15.76:0/486059773 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21940047c0 msgr2=0x7f2194004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:35.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.764+0000 7f21aad9d640 1 --2- 172.21.15.76:0/486059773 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21940047c0 0x7f2194004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.764+0000 7f21aad9d640 1 -- 172.21.15.76:0/486059773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2194005180 msgr2=0x7f219400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:35.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.764+0000 7f21aad9d640 1 --2- 172.21.15.76:0/486059773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2194005180 0x7f219400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.764+0000 7f21ab59e640 1 -- 172.21.15.76:0/486059773 reap_dead start 2023-11-09T03:46:35.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.764+0000 7f21aad9d640 1 -- 172.21.15.76:0/486059773 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2194003520 con 0x7f2194003970 2023-11-09T03:46:35.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.765+0000 7f21aad9d640 1 --2- 172.21.15.76:0/486059773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2194003970 0x7f2194003d90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f219c00c190 tx=0x7f21940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77e9c36548bd4aa server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:35.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.765+0000 7f21a9d9b640 1 -- 172.21.15.76:0/486059773 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c007270 con 0x7f2194003970 2023-11-09T03:46:35.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.765+0000 7f21a9d9b640 1 -- 172.21.15.76:0/486059773 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219c0073f0 con 0x7f2194003970 2023-11-09T03:46:35.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.765+0000 7f21a9d9b640 1 -- 172.21.15.76:0/486059773 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c02c870 con 0x7f2194003970 2023-11-09T03:46:35.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.766+0000 7f21b1125640 1 -- 172.21.15.76:0/486059773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2194003970 msgr2=0x7f2194003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:35.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.766+0000 7f21b1125640 1 --2- 172.21.15.76:0/486059773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2194003970 0x7f2194003d90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f219c00c190 tx=0x7f21940057f0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.766+0000 7f21b1125640 1 -- 172.21.15.76:0/486059773 shutdown_connections 2023-11-09T03:46:35.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.766+0000 7f21b1125640 1 --2- 172.21.15.76:0/486059773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2194003970 0x7f2194003d90 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.766+0000 7f21b1125640 1 -- 172.21.15.76:0/486059773 >> 172.21.15.76:0/486059773 conn(0x7f21ac098d40 msgr2=0x7f21ac0cfa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:35.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.766+0000 7f21b1125640 1 -- 172.21.15.76:0/486059773 shutdown_connections 2023-11-09T03:46:35.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.766+0000 7f21b1125640 1 -- 172.21.15.76:0/486059773 wait complete. 2023-11-09T03:46:35.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.767+0000 7f21b1125640 1 Processor -- start 2023-11-09T03:46:35.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.767+0000 7f21b1125640 1 -- start start 2023-11-09T03:46:35.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.768+0000 7f21b1125640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2194003970 0x7f21ac151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:35.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.768+0000 7f21b1125640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21ac0c8510 0x7f21ac15c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:35.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.768+0000 7f21b1125640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21ac15cdf0 0x7f21ac0cf960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:35.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.768+0000 7f21b1125640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac0b2690 con 0x7f21ac15cdf0 2023-11-09T03:46:35.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.768+0000 7f21b1125640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac0b34f0 con 0x7f2194003970 2023-11-09T03:46:35.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.768+0000 7f21b1125640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac151560 con 0x7f21ac0c8510 2023-11-09T03:46:35.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.768+0000 7f21ab59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21ac15cdf0 0x7f21ac0cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:35.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.768+0000 7f21ab59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21ac15cdf0 0x7f21ac0cf960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35032/0 (socket says 172.21.15.76:35032) 2023-11-09T03:46:35.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.768+0000 7f21ab59e640 1 -- 172.21.15.76:0/1873677495 learned_addr learned my addr 172.21.15.76:0/1873677495 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:35.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.768+0000 7f21aa59c640 1 --2- 172.21.15.76:0/1873677495 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21ac0c8510 0x7f21ac15c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:35.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.768+0000 7f21aad9d640 1 --2- 172.21.15.76:0/1873677495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2194003970 0x7f21ac151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:35.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.768+0000 7f21ab59e640 1 -- 172.21.15.76:0/1873677495 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21ac0c8510 msgr2=0x7f21ac15c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:35.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.768+0000 7f21ab59e640 1 --2- 172.21.15.76:0/1873677495 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21ac0c8510 0x7f21ac15c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.768+0000 7f21ab59e640 1 -- 172.21.15.76:0/1873677495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2194003970 msgr2=0x7f21ac151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:35.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.768+0000 7f21ab59e640 1 --2- 172.21.15.76:0/1873677495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2194003970 0x7f21ac151020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.768+0000 7f21ab59e640 1 -- 172.21.15.76:0/1873677495 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac0b28e0 con 0x7f21ac15cdf0 2023-11-09T03:46:35.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.769+0000 7f21ab59e640 1 --2- 172.21.15.76:0/1873677495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21ac15cdf0 0x7f21ac0cf960 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f21ac15fb20 tx=0x7f21a40115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:35.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.769+0000 7f21937fe640 1 -- 172.21.15.76:0/1873677495 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a400e070 con 0x7f21ac15cdf0 2023-11-09T03:46:35.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.769+0000 7f21937fe640 1 -- 172.21.15.76:0/1873677495 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a4011e10 con 0x7f21ac15cdf0 2023-11-09T03:46:35.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.769+0000 7f21b1125640 1 -- 172.21.15.76:0/1873677495 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21ac0b2bc0 con 0x7f21ac15cdf0 2023-11-09T03:46:35.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.769+0000 7f21937fe640 1 -- 172.21.15.76:0/1873677495 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a4003990 con 0x7f21ac15cdf0 2023-11-09T03:46:35.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.769+0000 7f21b1125640 1 -- 172.21.15.76:0/1873677495 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21ac0b3180 con 0x7f21ac15cdf0 2023-11-09T03:46:35.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.770+0000 7f21b1125640 1 -- 172.21.15.76:0/1873677495 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21ac06eec0 con 0x7f21ac15cdf0 2023-11-09T03:46:35.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.770+0000 7f21937fe640 1 -- 172.21.15.76:0/1873677495 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f21a4003af0 con 0x7f21ac15cdf0 2023-11-09T03:46:35.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.770+0000 7f21937fe640 1 --2- 172.21.15.76:0/1873677495 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f219806ae70 0x7f219806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:35.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.771+0000 7f21937fe640 1 -- 172.21.15.76:0/1873677495 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f21a408a120 con 0x7f21ac15cdf0 2023-11-09T03:46:35.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.771+0000 7f21aad9d640 1 --2- 172.21.15.76:0/1873677495 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f219806ae70 0x7f219806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:35.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.771+0000 7f21aad9d640 1 --2- 172.21.15.76:0/1873677495 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f219806ae70 0x7f219806d330 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f219c034000 tx=0x7f219c009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:35.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.775+0000 7f21937fe640 1 -- 172.21.15.76:0/1873677495 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f21a4054e80 con 0x7f21ac15cdf0 2023-11-09T03:46:35.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.942+0000 7f21b1125640 1 -- 172.21.15.76:0/1873677495 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21ac074290 con 0x7f21ac15cdf0 2023-11-09T03:46:35.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.943+0000 7f21937fe640 1 -- 172.21.15.76:0/1873677495 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18710 (secure 0 0 0) 0x7f21a40584a0 con 0x7f21ac15cdf0 2023-11-09T03:46:35.944 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:35.944 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":27,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:46:12.537265+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:46:35.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.945+0000 7f21b1125640 1 -- 172.21.15.76:0/1873677495 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f219806ae70 msgr2=0x7f219806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:35.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.945+0000 7f21b1125640 1 --2- 172.21.15.76:0/1873677495 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f219806ae70 0x7f219806d330 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f219c034000 tx=0x7f219c009030 comp rx=0 tx=0).stop 2023-11-09T03:46:35.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.945+0000 7f21b1125640 1 -- 172.21.15.76:0/1873677495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21ac15cdf0 msgr2=0x7f21ac0cf960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:35.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.945+0000 7f21b1125640 1 --2- 172.21.15.76:0/1873677495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21ac15cdf0 0x7f21ac0cf960 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f21ac15fb20 tx=0x7f21a40115a0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.945+0000 7f21b1125640 1 -- 172.21.15.76:0/1873677495 shutdown_connections 2023-11-09T03:46:35.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.945+0000 7f21b1125640 1 --2- 172.21.15.76:0/1873677495 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f219806ae70 0x7f219806d330 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.945+0000 7f21b1125640 1 --2- 172.21.15.76:0/1873677495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21ac15cdf0 0x7f21ac0cf960 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.945+0000 7f21b1125640 1 --2- 172.21.15.76:0/1873677495 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21ac0c8510 0x7f21ac15c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.945+0000 7f21b1125640 1 --2- 172.21.15.76:0/1873677495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2194003970 0x7f21ac151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:35.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.945+0000 7f21b1125640 1 -- 172.21.15.76:0/1873677495 >> 172.21.15.76:0/1873677495 conn(0x7f21ac098d40 msgr2=0x7f21ac06a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:35.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.945+0000 7f21b1125640 1 -- 172.21.15.76:0/1873677495 shutdown_connections 2023-11-09T03:46:35.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:35.945+0000 7f21b1125640 1 -- 172.21.15.76:0/1873677495 wait complete. 2023-11-09T03:46:35.957 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:45:28.704715+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:45:29.717614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-09T03:45:33.440577+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-09T03:46:35.958 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:46:36.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.059+0000 7efe47ae1640 1 Processor -- start 2023-11-09T03:46:36.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.060+0000 7efe47ae1640 1 -- start start 2023-11-09T03:46:36.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.060+0000 7efe47ae1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe40159c00 0x7efe400cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:36.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.060+0000 7efe47ae1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7efe400c48e0 0x7efe400c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:36.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.060+0000 7efe47ae1640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4015aff0 con 0x7efe400adac0 2023-11-09T03:46:36.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.060+0000 7efe47ae1640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe400c52a0 con 0x7efe400c48e0 2023-11-09T03:46:36.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.060+0000 7efe47ae1640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4015a3e0 con 0x7efe40159c00 2023-11-09T03:46:36.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.061+0000 7efe4507c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7efe400c48e0 0x7efe400c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:36.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.061+0000 7efe4607e640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7efe400adac0 0x7efe400adef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40688/0 (socket says 172.21.15.76:40688) 2023-11-09T03:46:36.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.061+0000 7efe4607e640 1 -- 172.21.15.76:0/97445580 learned_addr learned my addr 172.21.15.76:0/97445580 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:36.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.061+0000 7efe4507c640 1 -- 172.21.15.76:0/97445580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe40159c00 msgr2=0x7efe400cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:36.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.061+0000 7efe4587d640 1 --2- 172.21.15.76:0/97445580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe40159c00 0x7efe400cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:36.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.061+0000 7efe4487b640 1 -- 172.21.15.76:0/97445580 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3251773271 0 0) 0x7efe4015aff0 con 0x7efe400adac0 2023-11-09T03:46:36.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.061+0000 7efe4507c640 1 --2- 172.21.15.76:0/97445580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe40159c00 0x7efe400cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.061+0000 7efe4507c640 1 -- 172.21.15.76:0/97445580 >> v1:172.21.15.76:6789/0 conn(0x7efe400adac0 legacy=0x7efe400adef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:36.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.061+0000 7efe4507c640 1 -- 172.21.15.76:0/97445580 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4015a6c0 con 0x7efe400c48e0 2023-11-09T03:46:36.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.061+0000 7efe4587d640 1 --2- 172.21.15.76:0/97445580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe40159c00 0x7efe400cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:36.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.062+0000 7efe4507c640 1 --2- 172.21.15.76:0/97445580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe400c48e0 0x7efe400c4d60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7efe300097a0 tx=0x7efe3002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b2f04e98b34fad server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:36.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.062+0000 7efe4487b640 1 -- 172.21.15.76:0/97445580 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe30031830 con 0x7efe400c48e0 2023-11-09T03:46:36.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.062+0000 7efe4487b640 1 -- 172.21.15.76:0/97445580 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe30031990 con 0x7efe400c48e0 2023-11-09T03:46:36.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.062+0000 7efe4487b640 1 -- 172.21.15.76:0/97445580 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe30031d20 con 0x7efe400c48e0 2023-11-09T03:46:36.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.062+0000 7efe47ae1640 1 -- 172.21.15.76:0/97445580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe400c48e0 msgr2=0x7efe400c4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:36.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.062+0000 7efe47ae1640 1 --2- 172.21.15.76:0/97445580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe400c48e0 0x7efe400c4d60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7efe300097a0 tx=0x7efe3002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.062+0000 7efe47ae1640 1 -- 172.21.15.76:0/97445580 shutdown_connections 2023-11-09T03:46:36.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.062+0000 7efe47ae1640 1 --2- 172.21.15.76:0/97445580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe400c48e0 0x7efe400c4d60 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.062+0000 7efe47ae1640 1 --2- 172.21.15.76:0/97445580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe40159c00 0x7efe400cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.062+0000 7efe47ae1640 1 -- 172.21.15.76:0/97445580 >> 172.21.15.76:0/97445580 conn(0x7efe400ce950 msgr2=0x7efe400ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:36.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.062+0000 7efe47ae1640 1 -- 172.21.15.76:0/97445580 shutdown_connections 2023-11-09T03:46:36.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.062+0000 7efe47ae1640 1 -- 172.21.15.76:0/97445580 wait complete. 2023-11-09T03:46:36.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.063+0000 7efe47ae1640 1 Processor -- start 2023-11-09T03:46:36.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.064+0000 7efe47ae1640 1 -- start start 2023-11-09T03:46:36.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.064+0000 7efe47ae1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe400adac0 0x7efe40153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:36.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.064+0000 7efe47ae1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe400c48e0 0x7efe401f2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:36.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.064+0000 7efe47ae1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe40159c00 0x7efe4015b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:36.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.064+0000 7efe47ae1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe401538b0 con 0x7efe40159c00 2023-11-09T03:46:36.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.064+0000 7efe47ae1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe400b2a50 con 0x7efe400adac0 2023-11-09T03:46:36.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.064+0000 7efe47ae1640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe400b2ca0 con 0x7efe400c48e0 2023-11-09T03:46:36.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.064+0000 7efe4607e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe40159c00 0x7efe4015b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:36.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.064+0000 7efe4607e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe40159c00 0x7efe4015b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35060/0 (socket says 172.21.15.76:35060) 2023-11-09T03:46:36.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.064+0000 7efe4607e640 1 -- 172.21.15.76:0/679310634 learned_addr learned my addr 172.21.15.76:0/679310634 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:36.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.065+0000 7efe4507c640 1 --2- 172.21.15.76:0/679310634 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe400c48e0 0x7efe401f2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:36.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.065+0000 7efe4587d640 1 --2- 172.21.15.76:0/679310634 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe400adac0 0x7efe40153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:36.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.065+0000 7efe4607e640 1 -- 172.21.15.76:0/679310634 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe400c48e0 msgr2=0x7efe401f2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:36.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.065+0000 7efe4607e640 1 --2- 172.21.15.76:0/679310634 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe400c48e0 0x7efe401f2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.065+0000 7efe4607e640 1 -- 172.21.15.76:0/679310634 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe400adac0 msgr2=0x7efe40153370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:36.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.065+0000 7efe4607e640 1 --2- 172.21.15.76:0/679310634 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe400adac0 0x7efe40153370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.065+0000 7efe4607e640 1 -- 172.21.15.76:0/679310634 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe30009400 con 0x7efe40159c00 2023-11-09T03:46:36.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.065+0000 7efe4607e640 1 --2- 172.21.15.76:0/679310634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe40159c00 0x7efe4015b1a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efe38001eb0 tx=0x7efe3800c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:36.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.065+0000 7efe367fc640 1 -- 172.21.15.76:0/679310634 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe380074e0 con 0x7efe40159c00 2023-11-09T03:46:36.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.065+0000 7efe367fc640 1 -- 172.21.15.76:0/679310634 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe38007640 con 0x7efe40159c00 2023-11-09T03:46:36.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.065+0000 7efe47ae1640 1 -- 172.21.15.76:0/679310634 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe400b2f80 con 0x7efe40159c00 2023-11-09T03:46:36.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.065+0000 7efe367fc640 1 -- 172.21.15.76:0/679310634 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe38004a00 con 0x7efe40159c00 2023-11-09T03:46:36.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.065+0000 7efe47ae1640 1 -- 172.21.15.76:0/679310634 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe400b3540 con 0x7efe40159c00 2023-11-09T03:46:36.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.066+0000 7efe47ae1640 1 -- 172.21.15.76:0/679310634 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe400b3730 con 0x7efe40159c00 2023-11-09T03:46:36.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.067+0000 7efe367fc640 1 -- 172.21.15.76:0/679310634 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efe38004b60 con 0x7efe40159c00 2023-11-09T03:46:36.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.067+0000 7efe367fc640 1 --2- 172.21.15.76:0/679310634 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efe2406ae70 0x7efe2406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:36.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.067+0000 7efe367fc640 1 -- 172.21.15.76:0/679310634 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7efe380897f0 con 0x7efe40159c00 2023-11-09T03:46:36.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.067+0000 7efe4587d640 1 --2- 172.21.15.76:0/679310634 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efe2406ae70 0x7efe2406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:36.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.068+0000 7efe4587d640 1 --2- 172.21.15.76:0/679310634 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efe2406ae70 0x7efe2406d330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0075b0 tx=0x7efe3c007540 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:36.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.071+0000 7efe367fc640 1 -- 172.21.15.76:0/679310634 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efe38057f50 con 0x7efe40159c00 2023-11-09T03:46:36.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.235+0000 7efe47ae1640 1 -- 172.21.15.76:0/679310634 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efe40074290 con 0x7efe40159c00 2023-11-09T03:46:36.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.236+0000 7efe367fc640 1 -- 172.21.15.76:0/679310634 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7efe38057ae0 con 0x7efe40159c00 2023-11-09T03:46:36.236 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:46:36.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.238+0000 7efe47ae1640 1 -- 172.21.15.76:0/679310634 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efe2406ae70 msgr2=0x7efe2406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:36.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.238+0000 7efe47ae1640 1 --2- 172.21.15.76:0/679310634 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efe2406ae70 0x7efe2406d330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0075b0 tx=0x7efe3c007540 comp rx=0 tx=0).stop 2023-11-09T03:46:36.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.238+0000 7efe47ae1640 1 -- 172.21.15.76:0/679310634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe40159c00 msgr2=0x7efe4015b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:36.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.238+0000 7efe47ae1640 1 --2- 172.21.15.76:0/679310634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe40159c00 0x7efe4015b1a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efe38001eb0 tx=0x7efe3800c870 comp rx=0 tx=0).stop 2023-11-09T03:46:36.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.238+0000 7efe47ae1640 1 -- 172.21.15.76:0/679310634 shutdown_connections 2023-11-09T03:46:36.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.238+0000 7efe47ae1640 1 --2- 172.21.15.76:0/679310634 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efe2406ae70 0x7efe2406d330 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.238+0000 7efe47ae1640 1 --2- 172.21.15.76:0/679310634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe40159c00 0x7efe4015b1a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.238+0000 7efe47ae1640 1 --2- 172.21.15.76:0/679310634 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe400c48e0 0x7efe401f2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.238+0000 7efe47ae1640 1 --2- 172.21.15.76:0/679310634 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe400adac0 0x7efe40153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.238+0000 7efe47ae1640 1 -- 172.21.15.76:0/679310634 >> 172.21.15.76:0/679310634 conn(0x7efe400ce950 msgr2=0x7efe400c4070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:36.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.238+0000 7efe47ae1640 1 -- 172.21.15.76:0/679310634 shutdown_connections 2023-11-09T03:46:36.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.238+0000 7efe47ae1640 1 -- 172.21.15.76:0/679310634 wait complete. 2023-11-09T03:46:36.249 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:46:36.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.351+0000 7f09c14c9640 1 Processor -- start 2023-11-09T03:46:36.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.351+0000 7f09c14c9640 1 -- start start 2023-11-09T03:46:36.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.352+0000 7f09c14c9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09bc0ae1d0 0x7f09bc0cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:36.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.352+0000 7f09c14c9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09bc15aff0 con 0x7f09bc159c00 2023-11-09T03:46:36.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.352+0000 7f09c14c9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09bc15a3e0 con 0x7f09bc0c4ff0 2023-11-09T03:46:36.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.352+0000 7f09c14c9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc15a6c0 con 0x7f09bc0ae1d0 2023-11-09T03:46:36.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.352+0000 7f09baffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f09bc159c00 0x7f09bc0c48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40696/0 (socket says 172.21.15.76:40696) 2023-11-09T03:46:36.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.352+0000 7f09baffd640 1 -- 172.21.15.76:0/1154531866 learned_addr learned my addr 172.21.15.76:0/1154531866 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:36.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.352+0000 7f09bb7fe640 1 --2- 172.21.15.76:0/1154531866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09bc0ae1d0 0x7f09bc0cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:36.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.352+0000 7f09b9ffb640 1 -- 172.21.15.76:0/1154531866 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 8174489 0 0) 0x7f09bc15aff0 con 0x7f09bc159c00 2023-11-09T03:46:36.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.352+0000 7f09b9ffb640 1 -- 172.21.15.76:0/1154531866 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09a4003520 con 0x7f09bc159c00 2023-11-09T03:46:36.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.353+0000 7f09b9ffb640 1 -- 172.21.15.76:0/1154531866 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 597042718 0 0) 0x7f09a4003520 con 0x7f09bc159c00 2023-11-09T03:46:36.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.353+0000 7f09b9ffb640 1 -- 172.21.15.76:0/1154531866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09bc0ae1d0 msgr2=0x7f09bc0cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:36.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.353+0000 7f09b9ffb640 1 --2- 172.21.15.76:0/1154531866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09bc0ae1d0 0x7f09bc0cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.353+0000 7f09b9ffb640 1 -- 172.21.15.76:0/1154531866 >> v1:172.21.15.113:6789/0 conn(0x7f09bc0c4ff0 legacy=0x7f09bc0adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:36.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.353+0000 7f09b9ffb640 1 -- 172.21.15.76:0/1154531866 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09bc15a9a0 con 0x7f09bc159c00 2023-11-09T03:46:36.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.353+0000 7f09b9ffb640 1 -- 172.21.15.76:0/1154531866 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f09ac0019f0 con 0x7f09bc159c00 2023-11-09T03:46:36.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.353+0000 7f09b9ffb640 1 -- 172.21.15.76:0/1154531866 >> v1:172.21.15.76:6789/0 conn(0x7f09bc159c00 legacy=0x7f09bc0c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:36.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.353+0000 7f09b9ffb640 1 --2- 172.21.15.76:0/1154531866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09a40039a0 0x7f09a4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:36.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.353+0000 7f09b9ffb640 1 --2- 172.21.15.76:0/1154531866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09bc0ae1d0 0x7f09a40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:36.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.353+0000 7f09b9ffb640 1 --2- 172.21.15.76:0/1154531866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09a4004d00 0x7f09a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:36.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.353+0000 7f09b9ffb640 1 -- 172.21.15.76:0/1154531866 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc15a6c0 con 0x7f09a4004d00 2023-11-09T03:46:36.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.353+0000 7f09b9ffb640 1 -- 172.21.15.76:0/1154531866 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a400ba60 con 0x7f09a40039a0 2023-11-09T03:46:36.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.353+0000 7f09b9ffb640 1 -- 172.21.15.76:0/1154531866 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a400bc20 con 0x7f09bc0ae1d0 2023-11-09T03:46:36.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.353+0000 7f09bb7fe640 1 --2- 172.21.15.76:0/1154531866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09a4004d00 0x7f09a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:36.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.353+0000 7f09ba7fc640 1 --2- 172.21.15.76:0/1154531866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09bc0ae1d0 0x7f09a40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:36.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.353+0000 7f09baffd640 1 --2- 172.21.15.76:0/1154531866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09a40039a0 0x7f09a4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:36.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.353+0000 7f09bb7fe640 1 -- 172.21.15.76:0/1154531866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09bc0ae1d0 msgr2=0x7f09a40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:36.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.353+0000 7f09bb7fe640 1 --2- 172.21.15.76:0/1154531866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09bc0ae1d0 0x7f09a40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.353+0000 7f09bb7fe640 1 -- 172.21.15.76:0/1154531866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09a40039a0 msgr2=0x7f09a4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:36.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.353+0000 7f09bb7fe640 1 --2- 172.21.15.76:0/1154531866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09a40039a0 0x7f09a4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.353+0000 7f09bb7fe640 1 -- 172.21.15.76:0/1154531866 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a4003520 con 0x7f09a4004d00 2023-11-09T03:46:36.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.354+0000 7f09bb7fe640 1 --2- 172.21.15.76:0/1154531866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09a4004d00 0x7f09a4005180 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f09b4009a90 tx=0x7f09a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7bd5bb5a2d893a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:36.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.354+0000 7f09b9ffb640 1 -- 172.21.15.76:0/1154531866 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b402c680 con 0x7f09a4004d00 2023-11-09T03:46:36.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.354+0000 7f09b9ffb640 1 -- 172.21.15.76:0/1154531866 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b402c7e0 con 0x7f09a4004d00 2023-11-09T03:46:36.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.354+0000 7f09b9ffb640 1 -- 172.21.15.76:0/1154531866 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b402cb70 con 0x7f09a4004d00 2023-11-09T03:46:36.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.354+0000 7f09c14c9640 1 -- 172.21.15.76:0/1154531866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09a4004d00 msgr2=0x7f09a4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:36.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.354+0000 7f09c14c9640 1 --2- 172.21.15.76:0/1154531866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09a4004d00 0x7f09a4005180 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f09b4009a90 tx=0x7f09a40057f0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.354+0000 7f09bb7fe640 1 -- 172.21.15.76:0/1154531866 reap_dead start 2023-11-09T03:46:36.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.355+0000 7f09c14c9640 1 -- 172.21.15.76:0/1154531866 shutdown_connections 2023-11-09T03:46:36.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.355+0000 7f09c14c9640 1 -- 172.21.15.76:0/1154531866 >> 172.21.15.76:0/1154531866 conn(0x7f09bc0ce950 msgr2=0x7f09bc0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:36.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.355+0000 7f09c14c9640 1 -- 172.21.15.76:0/1154531866 shutdown_connections 2023-11-09T03:46:36.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.355+0000 7f09c14c9640 1 -- 172.21.15.76:0/1154531866 wait complete. 2023-11-09T03:46:36.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.356+0000 7f09c14c9640 1 Processor -- start 2023-11-09T03:46:36.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.356+0000 7f09c14c9640 1 -- start start 2023-11-09T03:46:36.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.357+0000 7f09c14c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09bc151380 0x7f09bc156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:36.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.357+0000 7f09c14c9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09bc151760 0x7f09bc152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:36.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.357+0000 7f09c14c9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09bc0c7d30 0x7f09bc153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:36.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.357+0000 7f09c14c9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc1eea60 con 0x7f09bc151380 2023-11-09T03:46:36.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.357+0000 7f09c14c9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc156a80 con 0x7f09bc0c7d30 2023-11-09T03:46:36.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.357+0000 7f09c14c9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc156d60 con 0x7f09bc151760 2023-11-09T03:46:36.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.357+0000 7f09baffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09bc151380 0x7f09bc156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:36.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.357+0000 7f09baffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09bc151380 0x7f09bc156190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35072/0 (socket says 172.21.15.76:35072) 2023-11-09T03:46:36.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.357+0000 7f09baffd640 1 -- 172.21.15.76:0/556806217 learned_addr learned my addr 172.21.15.76:0/556806217 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:36.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.357+0000 7f09ba7fc640 1 --2- 172.21.15.76:0/556806217 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09bc151760 0x7f09bc152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:36.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.357+0000 7f09bb7fe640 1 --2- 172.21.15.76:0/556806217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09bc0c7d30 0x7f09bc153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:36.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.357+0000 7f09baffd640 1 -- 172.21.15.76:0/556806217 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09bc151760 msgr2=0x7f09bc152d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:36.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.357+0000 7f09baffd640 1 --2- 172.21.15.76:0/556806217 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09bc151760 0x7f09bc152d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.357+0000 7f09baffd640 1 -- 172.21.15.76:0/556806217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09bc0c7d30 msgr2=0x7f09bc153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:36.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.357+0000 7f09baffd640 1 --2- 172.21.15.76:0/556806217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09bc0c7d30 0x7f09bc153250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.357+0000 7f09baffd640 1 -- 172.21.15.76:0/556806217 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09bc157040 con 0x7f09bc151380 2023-11-09T03:46:36.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.357+0000 7f09baffd640 1 --2- 172.21.15.76:0/556806217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09bc151380 0x7f09bc156190 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f09ac001560 tx=0x7f09ac00a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:36.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.357+0000 7f09a37fe640 1 -- 172.21.15.76:0/556806217 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ac013600 con 0x7f09bc151380 2023-11-09T03:46:36.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.357+0000 7f09a37fe640 1 -- 172.21.15.76:0/556806217 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ac013760 con 0x7f09bc151380 2023-11-09T03:46:36.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.357+0000 7f09c14c9640 1 -- 172.21.15.76:0/556806217 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09bc157320 con 0x7f09bc151380 2023-11-09T03:46:36.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.357+0000 7f09a37fe640 1 -- 172.21.15.76:0/556806217 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ac013b10 con 0x7f09bc151380 2023-11-09T03:46:36.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.357+0000 7f09c14c9640 1 -- 172.21.15.76:0/556806217 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09bc0c5a20 con 0x7f09bc151380 2023-11-09T03:46:36.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.358+0000 7f09c14c9640 1 -- 172.21.15.76:0/556806217 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09bc1eeba0 con 0x7f09bc151380 2023-11-09T03:46:36.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.359+0000 7f09a37fe640 1 -- 172.21.15.76:0/556806217 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f09ac013c70 con 0x7f09bc151380 2023-11-09T03:46:36.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.359+0000 7f09a37fe640 1 --2- 172.21.15.76:0/556806217 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f09a806ae00 0x7f09a806d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:36.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.359+0000 7f09a37fe640 1 -- 172.21.15.76:0/556806217 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f09ac08abc0 con 0x7f09bc151380 2023-11-09T03:46:36.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.359+0000 7f09ba7fc640 1 --2- 172.21.15.76:0/556806217 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f09a806ae00 0x7f09a806d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:36.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.360+0000 7f09ba7fc640 1 --2- 172.21.15.76:0/556806217 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f09a806ae00 0x7f09a806d2c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f09bc151b40 tx=0x7f09b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:36.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.363+0000 7f09a37fe640 1 -- 172.21.15.76:0/556806217 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f09ac055890 con 0x7f09bc151380 2023-11-09T03:46:36.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.530+0000 7f09c14c9640 1 -- 172.21.15.76:0/556806217 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f09bc074290 con 0x7f09bc151380 2023-11-09T03:46:36.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.530+0000 7f09a37fe640 1 -- 172.21.15.76:0/556806217 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f09ac0170a0 con 0x7f09bc151380 2023-11-09T03:46:36.531 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:46:36.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.532+0000 7f09c14c9640 1 -- 172.21.15.76:0/556806217 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f09a806ae00 msgr2=0x7f09a806d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:36.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.532+0000 7f09c14c9640 1 --2- 172.21.15.76:0/556806217 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f09a806ae00 0x7f09a806d2c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f09bc151b40 tx=0x7f09b0009040 comp rx=0 tx=0).stop 2023-11-09T03:46:36.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.532+0000 7f09c14c9640 1 -- 172.21.15.76:0/556806217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09bc151380 msgr2=0x7f09bc156190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:36.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.532+0000 7f09c14c9640 1 --2- 172.21.15.76:0/556806217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09bc151380 0x7f09bc156190 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f09ac001560 tx=0x7f09ac00a7a0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.532+0000 7f09c14c9640 1 -- 172.21.15.76:0/556806217 shutdown_connections 2023-11-09T03:46:36.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.532+0000 7f09c14c9640 1 --2- 172.21.15.76:0/556806217 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f09a806ae00 0x7f09a806d2c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.532+0000 7f09c14c9640 1 --2- 172.21.15.76:0/556806217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09bc0c7d30 0x7f09bc153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.532+0000 7f09c14c9640 1 --2- 172.21.15.76:0/556806217 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09bc151760 0x7f09bc152d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.532+0000 7f09c14c9640 1 --2- 172.21.15.76:0/556806217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09bc151380 0x7f09bc156190 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.532+0000 7f09c14c9640 1 -- 172.21.15.76:0/556806217 >> 172.21.15.76:0/556806217 conn(0x7f09bc0ce950 msgr2=0x7f09bc0c4350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:36.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.532+0000 7f09c14c9640 1 -- 172.21.15.76:0/556806217 shutdown_connections 2023-11-09T03:46:36.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.532+0000 7f09c14c9640 1 -- 172.21.15.76:0/556806217 wait complete. 2023-11-09T03:46:36.544 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:46:36.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.686+0000 7fec87ce5640 1 Processor -- start 2023-11-09T03:46:36.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.686+0000 7fec87ce5640 1 -- start start 2023-11-09T03:46:36.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.687+0000 7fec87ce5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fec80153850 0x7fec8015a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:36.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.687+0000 7fec87ce5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec800c3860 con 0x7fec800cc520 2023-11-09T03:46:36.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.687+0000 7fec87ce5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec800c9250 con 0x7fec800cc9d0 2023-11-09T03:46:36.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.687+0000 7fec87ce5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec80153c30 con 0x7fec80153850 2023-11-09T03:46:36.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.687+0000 7fec85a81640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fec80153850 0x7fec8015a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:36.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.687+0000 7fec85280640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fec800cc520 0x7fec80070fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40700/0 (socket says 172.21.15.76:40700) 2023-11-09T03:46:36.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.687+0000 7fec85280640 1 -- 172.21.15.76:0/4224955574 learned_addr learned my addr 172.21.15.76:0/4224955574 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:36.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.687+0000 7fec85a81640 1 -- 172.21.15.76:0/4224955574 >> v1:172.21.15.113:6789/0 conn(0x7fec800cc9d0 legacy=0x7fec800c3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:36.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.687+0000 7fec85a81640 1 -- 172.21.15.76:0/4224955574 >> v1:172.21.15.76:6789/0 conn(0x7fec800cc520 legacy=0x7fec80070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:36.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.687+0000 7fec85a81640 1 -- 172.21.15.76:0/4224955574 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec8015a700 con 0x7fec80153850 2023-11-09T03:46:36.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.687+0000 7fec84a7f640 1 -- 172.21.15.76:0/4224955574 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3244155170 0 0) 0x7fec800c3860 con 0x7fec800cc520 2023-11-09T03:46:36.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.688+0000 7fec85a81640 1 --2- 172.21.15.76:0/4224955574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec80153850 0x7fec8015a1c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0097a0 tx=0x7fec7c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e935f08383ae91dd server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:36.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.688+0000 7fec84a7f640 1 -- 172.21.15.76:0/4224955574 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c030610 con 0x7fec80153850 2023-11-09T03:46:36.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.688+0000 7fec84a7f640 1 -- 172.21.15.76:0/4224955574 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7c030770 con 0x7fec80153850 2023-11-09T03:46:36.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.688+0000 7fec84a7f640 1 -- 172.21.15.76:0/4224955574 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c030b00 con 0x7fec80153850 2023-11-09T03:46:36.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.688+0000 7fec87ce5640 1 -- 172.21.15.76:0/4224955574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec80153850 msgr2=0x7fec8015a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:36.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.688+0000 7fec87ce5640 1 --2- 172.21.15.76:0/4224955574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec80153850 0x7fec8015a1c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0097a0 tx=0x7fec7c02b960 comp rx=0 tx=0).stop 2023-11-09T03:46:36.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.689+0000 7fec87ce5640 1 -- 172.21.15.76:0/4224955574 shutdown_connections 2023-11-09T03:46:36.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.689+0000 7fec87ce5640 1 --2- 172.21.15.76:0/4224955574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec80153850 0x7fec8015a1c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.689+0000 7fec87ce5640 1 -- 172.21.15.76:0/4224955574 >> 172.21.15.76:0/4224955574 conn(0x7fec800ce950 msgr2=0x7fec800ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:36.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.689+0000 7fec87ce5640 1 -- 172.21.15.76:0/4224955574 shutdown_connections 2023-11-09T03:46:36.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.689+0000 7fec87ce5640 1 -- 172.21.15.76:0/4224955574 wait complete. 2023-11-09T03:46:36.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.690+0000 7fec87ce5640 1 Processor -- start 2023-11-09T03:46:36.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.690+0000 7fec87ce5640 1 -- start start 2023-11-09T03:46:36.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.690+0000 7fec87ce5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec800cc520 0x7fec80146780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:36.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.690+0000 7fec87ce5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec800cc9d0 0x7fec80167140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:36.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.690+0000 7fec87ce5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec80153850 0x7fec80167680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:36.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.690+0000 7fec87ce5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec801482c0 con 0x7fec800cc520 2023-11-09T03:46:36.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.690+0000 7fec87ce5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec800c89d0 con 0x7fec80153850 2023-11-09T03:46:36.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.690+0000 7fec87ce5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec80147220 con 0x7fec800cc9d0 2023-11-09T03:46:36.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.691+0000 7fec85a81640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec800cc520 0x7fec80146780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:36.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.691+0000 7fec85a81640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec800cc520 0x7fec80146780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35086/0 (socket says 172.21.15.76:35086) 2023-11-09T03:46:36.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.691+0000 7fec85a81640 1 -- 172.21.15.76:0/1470398560 learned_addr learned my addr 172.21.15.76:0/1470398560 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:36.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.691+0000 7fec85280640 1 --2- 172.21.15.76:0/1470398560 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec800cc9d0 0x7fec80167140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:36.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.691+0000 7fec86282640 1 --2- 172.21.15.76:0/1470398560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec80153850 0x7fec80167680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:36.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.691+0000 7fec85a81640 1 -- 172.21.15.76:0/1470398560 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec800cc9d0 msgr2=0x7fec80167140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:36.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.691+0000 7fec85a81640 1 --2- 172.21.15.76:0/1470398560 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec800cc9d0 0x7fec80167140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.691+0000 7fec85a81640 1 -- 172.21.15.76:0/1470398560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec80153850 msgr2=0x7fec80167680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:36.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.691+0000 7fec85a81640 1 --2- 172.21.15.76:0/1470398560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec80153850 0x7fec80167680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.691+0000 7fec85a81640 1 -- 172.21.15.76:0/1470398560 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7c009400 con 0x7fec800cc520 2023-11-09T03:46:36.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.691+0000 7fec85a81640 1 --2- 172.21.15.76:0/1470398560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec800cc520 0x7fec80146780 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0098c0 tx=0x7fec7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:36.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.691+0000 7fec767fc640 1 -- 172.21.15.76:0/1470398560 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c03e070 con 0x7fec800cc520 2023-11-09T03:46:36.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.691+0000 7fec767fc640 1 -- 172.21.15.76:0/1470398560 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7c039a10 con 0x7fec800cc520 2023-11-09T03:46:36.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.691+0000 7fec87ce5640 1 -- 172.21.15.76:0/1470398560 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec80147500 con 0x7fec800cc520 2023-11-09T03:46:36.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.691+0000 7fec767fc640 1 -- 172.21.15.76:0/1470398560 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c039da0 con 0x7fec800cc520 2023-11-09T03:46:36.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.691+0000 7fec87ce5640 1 -- 172.21.15.76:0/1470398560 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec80147ac0 con 0x7fec800cc520 2023-11-09T03:46:36.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.692+0000 7fec87ce5640 1 -- 172.21.15.76:0/1470398560 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec80074290 con 0x7fec800cc520 2023-11-09T03:46:36.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.693+0000 7fec767fc640 1 -- 172.21.15.76:0/1470398560 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fec7c04b400 con 0x7fec800cc520 2023-11-09T03:46:36.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.693+0000 7fec767fc640 1 --2- 172.21.15.76:0/1470398560 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fec6406aea0 0x7fec6406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:36.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.693+0000 7fec767fc640 1 -- 172.21.15.76:0/1470398560 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fec7c0abeb0 con 0x7fec800cc520 2023-11-09T03:46:36.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.693+0000 7fec85280640 1 --2- 172.21.15.76:0/1470398560 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fec6406aea0 0x7fec6406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:36.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.694+0000 7fec85280640 1 --2- 172.21.15.76:0/1470398560 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fec6406aea0 0x7fec6406d360 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fec8014c070 tx=0x7fec70007b90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:36.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.697+0000 7fec767fc640 1 -- 172.21.15.76:0/1470398560 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fec7c076cc0 con 0x7fec800cc520 2023-11-09T03:46:36.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.864+0000 7fec87ce5640 1 -- 172.21.15.76:0/1470398560 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fec800c5010 con 0x7fec800cc520 2023-11-09T03:46:36.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.864+0000 7fec767fc640 1 -- 172.21.15.76:0/1470398560 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fec7c034020 con 0x7fec800cc520 2023-11-09T03:46:36.865 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:46:36.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.866+0000 7fec87ce5640 1 -- 172.21.15.76:0/1470398560 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fec6406aea0 msgr2=0x7fec6406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:36.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.866+0000 7fec87ce5640 1 --2- 172.21.15.76:0/1470398560 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fec6406aea0 0x7fec6406d360 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fec8014c070 tx=0x7fec70007b90 comp rx=0 tx=0).stop 2023-11-09T03:46:36.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.866+0000 7fec87ce5640 1 -- 172.21.15.76:0/1470398560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec800cc520 msgr2=0x7fec80146780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:36.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.866+0000 7fec87ce5640 1 --2- 172.21.15.76:0/1470398560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec800cc520 0x7fec80146780 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0098c0 tx=0x7fec7c0046c0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.866+0000 7fec87ce5640 1 -- 172.21.15.76:0/1470398560 shutdown_connections 2023-11-09T03:46:36.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.866+0000 7fec87ce5640 1 --2- 172.21.15.76:0/1470398560 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fec6406aea0 0x7fec6406d360 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.866+0000 7fec87ce5640 1 --2- 172.21.15.76:0/1470398560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec80153850 0x7fec80167680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.866+0000 7fec87ce5640 1 --2- 172.21.15.76:0/1470398560 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec800cc9d0 0x7fec80167140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.866+0000 7fec87ce5640 1 --2- 172.21.15.76:0/1470398560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec800cc520 0x7fec80146780 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.866+0000 7fec87ce5640 1 -- 172.21.15.76:0/1470398560 >> 172.21.15.76:0/1470398560 conn(0x7fec800ce950 msgr2=0x7fec8015b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:36.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.866+0000 7fec87ce5640 1 -- 172.21.15.76:0/1470398560 shutdown_connections 2023-11-09T03:46:36.869 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.866+0000 7fec87ce5640 1 -- 172.21.15.76:0/1470398560 wait complete. 2023-11-09T03:46:36.877 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-09T03:46:36.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.979+0000 7fdf3790f640 1 Processor -- start 2023-11-09T03:46:36.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.980+0000 7fdf3790f640 1 -- start start 2023-11-09T03:46:36.981 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.980+0000 7fdf3790f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdf300b4880 0x7fdf300b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:36.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.980+0000 7fdf3790f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf300cde60 con 0x7fdf300cd6d0 2023-11-09T03:46:36.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.980+0000 7fdf3790f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf30160f90 con 0x7fdf30153e60 2023-11-09T03:46:36.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.980+0000 7fdf3790f640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf300b5200 con 0x7fdf300b4880 2023-11-09T03:46:36.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.981+0000 7fdf34eaa640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fdf300cd6d0 0x7fdf30161420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40708/0 (socket says 172.21.15.76:40708) 2023-11-09T03:46:36.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.981+0000 7fdf34eaa640 1 -- 172.21.15.76:0/2981080083 learned_addr learned my addr 172.21.15.76:0/2981080083 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:36.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.981+0000 7fdf35eac640 1 --2- 172.21.15.76:0/2981080083 >> v2:172.21.15.113:3300/0 conn(0x7fdf300b4880 0x7fdf300b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:36.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.981+0000 7fdf27fff640 1 -- 172.21.15.76:0/2981080083 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1351042982 0 0) 0x7fdf300cde60 con 0x7fdf300cd6d0 2023-11-09T03:46:36.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.981+0000 7fdf27fff640 1 -- 172.21.15.76:0/2981080083 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf1c003520 con 0x7fdf300cd6d0 2023-11-09T03:46:36.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.981+0000 7fdf27fff640 1 -- 172.21.15.76:0/2981080083 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1878881311 0 0) 0x7fdf1c003520 con 0x7fdf300cd6d0 2023-11-09T03:46:36.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.981+0000 7fdf35eac640 1 -- 172.21.15.76:0/2981080083 >> v1:172.21.15.113:6789/0 conn(0x7fdf30153e60 legacy=0x7fdf3015dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:36.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.981+0000 7fdf35eac640 1 -- 172.21.15.76:0/2981080083 >> v1:172.21.15.76:6789/0 conn(0x7fdf300cd6d0 legacy=0x7fdf30161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:36.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.981+0000 7fdf35eac640 1 -- 172.21.15.76:0/2981080083 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf300ce090 con 0x7fdf300b4880 2023-11-09T03:46:36.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.982+0000 7fdf35eac640 1 --2- 172.21.15.76:0/2981080083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf300b4880 0x7fdf300b4cc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdf28009d90 tx=0x7fdf2802be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d55e06b8caa9360 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:36.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.982+0000 7fdf27fff640 1 -- 172.21.15.76:0/2981080083 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf20001670 con 0x7fdf300b4880 2023-11-09T03:46:36.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.982+0000 7fdf27fff640 1 -- 172.21.15.76:0/2981080083 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2802c940 con 0x7fdf300b4880 2023-11-09T03:46:36.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.982+0000 7fdf27fff640 1 -- 172.21.15.76:0/2981080083 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2802ccb0 con 0x7fdf300b4880 2023-11-09T03:46:36.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.983+0000 7fdf3790f640 1 -- 172.21.15.76:0/2981080083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf300b4880 msgr2=0x7fdf300b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:36.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.983+0000 7fdf3790f640 1 --2- 172.21.15.76:0/2981080083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf300b4880 0x7fdf300b4cc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdf28009d90 tx=0x7fdf2802be10 comp rx=0 tx=0).stop 2023-11-09T03:46:36.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.983+0000 7fdf3790f640 1 -- 172.21.15.76:0/2981080083 shutdown_connections 2023-11-09T03:46:36.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.983+0000 7fdf3790f640 1 --2- 172.21.15.76:0/2981080083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf300b4880 0x7fdf300b4cc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.983+0000 7fdf3790f640 1 -- 172.21.15.76:0/2981080083 >> 172.21.15.76:0/2981080083 conn(0x7fdf300cf880 msgr2=0x7fdf300cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:36.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.983+0000 7fdf3790f640 1 -- 172.21.15.76:0/2981080083 shutdown_connections 2023-11-09T03:46:36.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.983+0000 7fdf3790f640 1 -- 172.21.15.76:0/2981080083 wait complete. 2023-11-09T03:46:36.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.984+0000 7fdf3790f640 1 Processor -- start 2023-11-09T03:46:36.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.984+0000 7fdf3790f640 1 -- start start 2023-11-09T03:46:36.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.985+0000 7fdf3790f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf300b4880 0x7fdf301505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:36.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.985+0000 7fdf3790f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf300cd6d0 0x7fdf301555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:36.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.985+0000 7fdf3790f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf30153e60 0x7fdf30156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:36.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.985+0000 7fdf3790f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf30150b20 con 0x7fdf300cd6d0 2023-11-09T03:46:36.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.985+0000 7fdf3790f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf30154810 con 0x7fdf30153e60 2023-11-09T03:46:36.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.985+0000 7fdf3790f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf30154af0 con 0x7fdf300b4880 2023-11-09T03:46:36.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.985+0000 7fdf34eaa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf300cd6d0 0x7fdf301555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:36.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.985+0000 7fdf34eaa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf300cd6d0 0x7fdf301555d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35088/0 (socket says 172.21.15.76:35088) 2023-11-09T03:46:36.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.985+0000 7fdf34eaa640 1 -- 172.21.15.76:0/2728016271 learned_addr learned my addr 172.21.15.76:0/2728016271 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:36.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.985+0000 7fdf356ab640 1 --2- 172.21.15.76:0/2728016271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf300b4880 0x7fdf301505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:36.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.985+0000 7fdf35eac640 1 --2- 172.21.15.76:0/2728016271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf30153e60 0x7fdf30156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:36.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.985+0000 7fdf34eaa640 1 -- 172.21.15.76:0/2728016271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf300b4880 msgr2=0x7fdf301505e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:36.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.985+0000 7fdf34eaa640 1 --2- 172.21.15.76:0/2728016271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf300b4880 0x7fdf301505e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.985+0000 7fdf34eaa640 1 -- 172.21.15.76:0/2728016271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf30153e60 msgr2=0x7fdf30156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:36.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.985+0000 7fdf34eaa640 1 --2- 172.21.15.76:0/2728016271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf30153e60 0x7fdf30156740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:36.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.985+0000 7fdf34eaa640 1 -- 172.21.15.76:0/2728016271 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf28009a70 con 0x7fdf300cd6d0 2023-11-09T03:46:36.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.985+0000 7fdf34eaa640 1 --2- 172.21.15.76:0/2728016271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf300cd6d0 0x7fdf301555d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdf200017a0 tx=0x7fdf2000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:36.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.985+0000 7fdf25ffb640 1 -- 172.21.15.76:0/2728016271 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf20005ad0 con 0x7fdf300cd6d0 2023-11-09T03:46:36.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.985+0000 7fdf25ffb640 1 -- 172.21.15.76:0/2728016271 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf20005c30 con 0x7fdf300cd6d0 2023-11-09T03:46:36.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.985+0000 7fdf3790f640 1 -- 172.21.15.76:0/2728016271 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf30154dd0 con 0x7fdf300cd6d0 2023-11-09T03:46:36.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.985+0000 7fdf25ffb640 1 -- 172.21.15.76:0/2728016271 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf200072b0 con 0x7fdf300cd6d0 2023-11-09T03:46:36.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.986+0000 7fdf3790f640 1 -- 172.21.15.76:0/2728016271 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3015a1a0 con 0x7fdf300cd6d0 2023-11-09T03:46:36.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.987+0000 7fdf3790f640 1 -- 172.21.15.76:0/2728016271 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf30074290 con 0x7fdf300cd6d0 2023-11-09T03:46:36.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.987+0000 7fdf25ffb640 1 -- 172.21.15.76:0/2728016271 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdf20007410 con 0x7fdf300cd6d0 2023-11-09T03:46:36.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.987+0000 7fdf25ffb640 1 --2- 172.21.15.76:0/2728016271 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdf1406ae70 0x7fdf1406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:36.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.987+0000 7fdf25ffb640 1 -- 172.21.15.76:0/2728016271 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fdf20089650 con 0x7fdf300cd6d0 2023-11-09T03:46:36.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.988+0000 7fdf356ab640 1 --2- 172.21.15.76:0/2728016271 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdf1406ae70 0x7fdf1406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:36.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.988+0000 7fdf356ab640 1 --2- 172.21.15.76:0/2728016271 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdf1406ae70 0x7fdf1406d330 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdf300b85a0 tx=0x7fdf2c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:36.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:36.992+0000 7fdf25ffb640 1 -- 172.21.15.76:0/2728016271 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdf200543b0 con 0x7fdf300cd6d0 2023-11-09T03:46:37.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.157+0000 7fdf3790f640 1 -- 172.21.15.76:0/2728016271 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdf300d0b80 con 0x7fdf300cd6d0 2023-11-09T03:46:37.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.158+0000 7fdf25ffb640 1 -- 172.21.15.76:0/2728016271 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdf200579d0 con 0x7fdf300cd6d0 2023-11-09T03:46:37.159 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:46:37.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.160+0000 7fdf3790f640 1 -- 172.21.15.76:0/2728016271 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdf1406ae70 msgr2=0x7fdf1406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:37.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.160+0000 7fdf3790f640 1 --2- 172.21.15.76:0/2728016271 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdf1406ae70 0x7fdf1406d330 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdf300b85a0 tx=0x7fdf2c009040 comp rx=0 tx=0).stop 2023-11-09T03:46:37.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.160+0000 7fdf3790f640 1 -- 172.21.15.76:0/2728016271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf300cd6d0 msgr2=0x7fdf301555d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:37.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.160+0000 7fdf3790f640 1 --2- 172.21.15.76:0/2728016271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf300cd6d0 0x7fdf301555d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdf200017a0 tx=0x7fdf2000cbe0 comp rx=0 tx=0).stop 2023-11-09T03:46:37.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.160+0000 7fdf3790f640 1 -- 172.21.15.76:0/2728016271 shutdown_connections 2023-11-09T03:46:37.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.160+0000 7fdf3790f640 1 --2- 172.21.15.76:0/2728016271 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdf1406ae70 0x7fdf1406d330 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:37.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.160+0000 7fdf3790f640 1 --2- 172.21.15.76:0/2728016271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf30153e60 0x7fdf30156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:37.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.160+0000 7fdf3790f640 1 --2- 172.21.15.76:0/2728016271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf300cd6d0 0x7fdf301555d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:37.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.160+0000 7fdf3790f640 1 --2- 172.21.15.76:0/2728016271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf300b4880 0x7fdf301505e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:37.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.160+0000 7fdf3790f640 1 -- 172.21.15.76:0/2728016271 >> 172.21.15.76:0/2728016271 conn(0x7fdf300cf880 msgr2=0x7fdf3015c970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:37.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.160+0000 7fdf3790f640 1 -- 172.21.15.76:0/2728016271 shutdown_connections 2023-11-09T03:46:37.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.160+0000 7fdf3790f640 1 -- 172.21.15.76:0/2728016271 wait complete. 2023-11-09T03:46:37.171 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-09T03:46:37.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.271+0000 7fb1de1b0640 1 Processor -- start 2023-11-09T03:46:37.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.271+0000 7fb1de1b0640 1 -- start start 2023-11-09T03:46:37.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.272+0000 7fb1de1b0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb1d806a830 0x7fb1d80ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:37.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.272+0000 7fb1de1b0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1d8160990 con 0x7fb1d80b1a40 2023-11-09T03:46:37.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.272+0000 7fb1de1b0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1d80cc7c0 con 0x7fb1d81605b0 2023-11-09T03:46:37.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.272+0000 7fb1de1b0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1d80cd750 con 0x7fb1d806a830 2023-11-09T03:46:37.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.272+0000 7fb1d7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb1d806a830 0x7fb1d80ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:37.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.272+0000 7fb1d7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb1d806a830 0x7fb1d80ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35100/0 (socket says 172.21.15.76:35100) 2023-11-09T03:46:37.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.272+0000 7fb1d6ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb1d80b1a40 0x7fb1d80cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40714/0 (socket says 172.21.15.76:40714) 2023-11-09T03:46:37.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.272+0000 7fb1d7fff640 1 -- 172.21.15.76:0/632334629 learned_addr learned my addr 172.21.15.76:0/632334629 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:37.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.273+0000 7fb1d7fff640 1 -- 172.21.15.76:0/632334629 >> v1:172.21.15.113:6789/0 conn(0x7fb1d81605b0 legacy=0x7fb1d806a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:37.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.273+0000 7fb1d7fff640 1 -- 172.21.15.76:0/632334629 >> v1:172.21.15.76:6789/0 conn(0x7fb1d80b1a40 legacy=0x7fb1d80cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:37.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.273+0000 7fb1d67fc640 1 -- 172.21.15.76:0/632334629 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 708011292 0 0) 0x7fb1d8160990 con 0x7fb1d80b1a40 2023-11-09T03:46:37.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.273+0000 7fb1d7fff640 1 -- 172.21.15.76:0/632334629 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d80cc4e0 con 0x7fb1d806a830 2023-11-09T03:46:37.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.273+0000 7fb1d7fff640 1 --2- 172.21.15.76:0/632334629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1d806a830 0x7fb1d80ccbc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0009e50 tx=0x7fb1d002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=610ec4d99d01df2c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:37.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.274+0000 7fb1d67fc640 1 -- 172.21.15.76:0/632334629 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d002ca80 con 0x7fb1d806a830 2023-11-09T03:46:37.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.274+0000 7fb1d67fc640 1 -- 172.21.15.76:0/632334629 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d002cbe0 con 0x7fb1d806a830 2023-11-09T03:46:37.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.274+0000 7fb1d67fc640 1 -- 172.21.15.76:0/632334629 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d00343f0 con 0x7fb1d806a830 2023-11-09T03:46:37.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.274+0000 7fb1de1b0640 1 -- 172.21.15.76:0/632334629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1d806a830 msgr2=0x7fb1d80ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:37.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.274+0000 7fb1de1b0640 1 --2- 172.21.15.76:0/632334629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1d806a830 0x7fb1d80ccbc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0009e50 tx=0x7fb1d002bee0 comp rx=0 tx=0).stop 2023-11-09T03:46:37.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.274+0000 7fb1de1b0640 1 -- 172.21.15.76:0/632334629 shutdown_connections 2023-11-09T03:46:37.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.274+0000 7fb1de1b0640 1 --2- 172.21.15.76:0/632334629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1d806a830 0x7fb1d80ccbc0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:37.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.274+0000 7fb1de1b0640 1 -- 172.21.15.76:0/632334629 >> 172.21.15.76:0/632334629 conn(0x7fb1d8071820 msgr2=0x7fb1d8071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:37.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.274+0000 7fb1de1b0640 1 -- 172.21.15.76:0/632334629 shutdown_connections 2023-11-09T03:46:37.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.274+0000 7fb1de1b0640 1 -- 172.21.15.76:0/632334629 wait complete. 2023-11-09T03:46:37.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.275+0000 7fb1de1b0640 1 Processor -- start 2023-11-09T03:46:37.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.276+0000 7fb1de1b0640 1 -- start start 2023-11-09T03:46:37.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.276+0000 7fb1de1b0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1d806a830 0x7fb1d8150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:37.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.276+0000 7fb1de1b0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1d81605b0 0x7fb1d80b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:37.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.276+0000 7fb1de1b0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1d80c8b30 0x7fb1d80c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:37.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.276+0000 7fb1de1b0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d80b4f60 con 0x7fb1d806a830 2023-11-09T03:46:37.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.276+0000 7fb1de1b0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d81514d0 con 0x7fb1d80c8b30 2023-11-09T03:46:37.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.276+0000 7fb1de1b0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d80c9500 con 0x7fb1d81605b0 2023-11-09T03:46:37.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.276+0000 7fb1d77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1d806a830 0x7fb1d8150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:37.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.276+0000 7fb1d77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1d806a830 0x7fb1d8150f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35102/0 (socket says 172.21.15.76:35102) 2023-11-09T03:46:37.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.276+0000 7fb1d77fe640 1 -- 172.21.15.76:0/601668882 learned_addr learned my addr 172.21.15.76:0/601668882 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:37.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.277+0000 7fb1d6ffd640 1 --2- 172.21.15.76:0/601668882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1d81605b0 0x7fb1d80b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:37.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.277+0000 7fb1d7fff640 1 --2- 172.21.15.76:0/601668882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1d80c8b30 0x7fb1d80c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:37.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.277+0000 7fb1d77fe640 1 -- 172.21.15.76:0/601668882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1d81605b0 msgr2=0x7fb1d80b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:37.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.277+0000 7fb1d77fe640 1 --2- 172.21.15.76:0/601668882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1d81605b0 0x7fb1d80b0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:37.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.277+0000 7fb1d77fe640 1 -- 172.21.15.76:0/601668882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1d80c8b30 msgr2=0x7fb1d80c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:37.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.277+0000 7fb1d77fe640 1 --2- 172.21.15.76:0/601668882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1d80c8b30 0x7fb1d80c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:37.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.277+0000 7fb1d77fe640 1 -- 172.21.15.76:0/601668882 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d0009b30 con 0x7fb1d806a830 2023-11-09T03:46:37.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.277+0000 7fb1d77fe640 1 --2- 172.21.15.76:0/601668882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1d806a830 0x7fb1d8150f90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb1c800c480 tx=0x7fb1c800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:37.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.277+0000 7fb1bffff640 1 -- 172.21.15.76:0/601668882 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c80052d0 con 0x7fb1d806a830 2023-11-09T03:46:37.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.277+0000 7fb1bffff640 1 -- 172.21.15.76:0/601668882 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c8005430 con 0x7fb1d806a830 2023-11-09T03:46:37.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.277+0000 7fb1bffff640 1 -- 172.21.15.76:0/601668882 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8012870 con 0x7fb1d806a830 2023-11-09T03:46:37.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.277+0000 7fb1de1b0640 1 -- 172.21.15.76:0/601668882 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d80b5380 con 0x7fb1d806a830 2023-11-09T03:46:37.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.277+0000 7fb1de1b0640 1 -- 172.21.15.76:0/601668882 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d80b45c0 con 0x7fb1d806a830 2023-11-09T03:46:37.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.278+0000 7fb1de1b0640 1 -- 172.21.15.76:0/601668882 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d80672a0 con 0x7fb1d806a830 2023-11-09T03:46:37.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.278+0000 7fb1bffff640 1 -- 172.21.15.76:0/601668882 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb1c80129d0 con 0x7fb1d806a830 2023-11-09T03:46:37.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.279+0000 7fb1bffff640 1 --2- 172.21.15.76:0/601668882 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb1c406aea0 0x7fb1c406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:37.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.279+0000 7fb1bffff640 1 -- 172.21.15.76:0/601668882 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fb1c8089450 con 0x7fb1d806a830 2023-11-09T03:46:37.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.279+0000 7fb1d6ffd640 1 --2- 172.21.15.76:0/601668882 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb1c406aea0 0x7fb1c406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:37.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.279+0000 7fb1d6ffd640 1 --2- 172.21.15.76:0/601668882 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb1c406aea0 0x7fb1c406d360 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb1d80b01a0 tx=0x7fb1cc007d00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:37.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.283+0000 7fb1bffff640 1 -- 172.21.15.76:0/601668882 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb1c80541b0 con 0x7fb1d806a830 2023-11-09T03:46:37.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.462+0000 7fb1de1b0640 1 -- 172.21.15.76:0/601668882 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb1d8074290 con 0x7fb1d806a830 2023-11-09T03:46:37.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.462+0000 7fb1bffff640 1 -- 172.21.15.76:0/601668882 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb1c80577d0 con 0x7fb1d806a830 2023-11-09T03:46:37.463 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2023-11-09T03:46:37.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.464+0000 7fb1de1b0640 1 -- 172.21.15.76:0/601668882 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb1c406aea0 msgr2=0x7fb1c406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:37.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.464+0000 7fb1de1b0640 1 --2- 172.21.15.76:0/601668882 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb1c406aea0 0x7fb1c406d360 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb1d80b01a0 tx=0x7fb1cc007d00 comp rx=0 tx=0).stop 2023-11-09T03:46:37.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.464+0000 7fb1de1b0640 1 -- 172.21.15.76:0/601668882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1d806a830 msgr2=0x7fb1d8150f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:37.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.464+0000 7fb1de1b0640 1 --2- 172.21.15.76:0/601668882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1d806a830 0x7fb1d8150f90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb1c800c480 tx=0x7fb1c800c940 comp rx=0 tx=0).stop 2023-11-09T03:46:37.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.464+0000 7fb1de1b0640 1 -- 172.21.15.76:0/601668882 shutdown_connections 2023-11-09T03:46:37.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.464+0000 7fb1de1b0640 1 --2- 172.21.15.76:0/601668882 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb1c406aea0 0x7fb1c406d360 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:37.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.464+0000 7fb1de1b0640 1 --2- 172.21.15.76:0/601668882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1d80c8b30 0x7fb1d80c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:37.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.464+0000 7fb1de1b0640 1 --2- 172.21.15.76:0/601668882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1d81605b0 0x7fb1d80b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:37.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.464+0000 7fb1de1b0640 1 --2- 172.21.15.76:0/601668882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1d806a830 0x7fb1d8150f90 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:37.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.464+0000 7fb1de1b0640 1 -- 172.21.15.76:0/601668882 >> 172.21.15.76:0/601668882 conn(0x7fb1d8071820 msgr2=0x7fb1d8071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:37.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.464+0000 7fb1de1b0640 1 -- 172.21.15.76:0/601668882 shutdown_connections 2023-11-09T03:46:37.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.464+0000 7fb1de1b0640 1 -- 172.21.15.76:0/601668882 wait complete. 2023-11-09T03:46:37.475 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi113.front.sepia.ceph.com... 2023-11-09T03:46:37.475 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:46:37.476 DEBUG:teuthology.orchestra.run.smithi113:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.3 2023-11-09T03:46:37.545 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:46:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:46:37.545 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi113.stderr:ceph-fuse[94152]: fuse finished with error 0 and tester_r 0 2023-11-09T03:46:37.546 INFO:teuthology.orchestra.run.smithi113.stderr:2023-11-09T03:46:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:46:37.550 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi113.stderr:2023-11-09T03:46:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:46:37.554 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:46:37.559 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2023-11-09T03:46:37.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:37.559 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-09T03:46:37.615 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-09T03:46:37.616 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:46:37.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.716+0000 7f90fb290640 1 Processor -- start 2023-11-09T03:46:37.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.716+0000 7f90fb290640 1 -- start start 2023-11-09T03:46:37.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.717+0000 7f90fb290640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90f415ce60 0x7f90f406b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:37.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.717+0000 7f90fb290640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f90f40cf160 0x7f90f40cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:37.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.717+0000 7f90fb290640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90f41615d0 con 0x7f90f406bd30 2023-11-09T03:46:37.720 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.717+0000 7f90fb290640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90f40cbde0 con 0x7f90f40cf160 2023-11-09T03:46:37.720 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.717+0000 7f90fb290640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f40cfb50 con 0x7f90f415ce60 2023-11-09T03:46:37.720 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.717+0000 7f90f982d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f90f40cf160 0x7f90f40cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:37.720 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.717+0000 7f90f882b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f90f406bd30 0x7f90f4160b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40716/0 (socket says 172.21.15.76:40716) 2023-11-09T03:46:37.720 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.717+0000 7f90f882b640 1 -- 172.21.15.76:0/3251274369 learned_addr learned my addr 172.21.15.76:0/3251274369 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:37.721 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.717+0000 7f90f902c640 1 --2- 172.21.15.76:0/3251274369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90f415ce60 0x7f90f406b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:37.721 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.717+0000 7f90f982d640 1 -- 172.21.15.76:0/3251274369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90f415ce60 msgr2=0x7f90f406b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:37.721 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.717+0000 7f90f982d640 1 --2- 172.21.15.76:0/3251274369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90f415ce60 0x7f90f406b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:37.721 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.717+0000 7f90f982d640 1 -- 172.21.15.76:0/3251274369 >> v1:172.21.15.76:6789/0 conn(0x7f90f406bd30 legacy=0x7f90f4160b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:37.721 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.717+0000 7f90ebfff640 1 -- 172.21.15.76:0/3251274369 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3718835035 0 0) 0x7f90f41615d0 con 0x7f90f406bd30 2023-11-09T03:46:37.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.717+0000 7f90f982d640 1 -- 172.21.15.76:0/3251274369 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f41618b0 con 0x7f90f40cf160 2023-11-09T03:46:37.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.718+0000 7f90f982d640 1 --2- 172.21.15.76:0/3251274369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90f40cf160 0x7f90f40cf610 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f90ec009d90 tx=0x7f90ec02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e30a880405ee85 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:37.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.718+0000 7f90ebfff640 1 -- 172.21.15.76:0/3251274369 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec02cc50 con 0x7f90f40cf160 2023-11-09T03:46:37.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.718+0000 7f90ebfff640 1 -- 172.21.15.76:0/3251274369 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90ec02cdb0 con 0x7f90f40cf160 2023-11-09T03:46:37.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.718+0000 7f90ebfff640 1 -- 172.21.15.76:0/3251274369 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec034660 con 0x7f90f40cf160 2023-11-09T03:46:37.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.719+0000 7f90fb290640 1 -- 172.21.15.76:0/3251274369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90f40cf160 msgr2=0x7f90f40cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:37.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.719+0000 7f90fb290640 1 --2- 172.21.15.76:0/3251274369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90f40cf160 0x7f90f40cf610 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f90ec009d90 tx=0x7f90ec02bea0 comp rx=0 tx=0).stop 2023-11-09T03:46:37.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.719+0000 7f90fb290640 1 -- 172.21.15.76:0/3251274369 shutdown_connections 2023-11-09T03:46:37.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.719+0000 7f90fb290640 1 --2- 172.21.15.76:0/3251274369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90f40cf160 0x7f90f40cf610 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:37.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.719+0000 7f90fb290640 1 --2- 172.21.15.76:0/3251274369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90f415ce60 0x7f90f406b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:37.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.719+0000 7f90fb290640 1 -- 172.21.15.76:0/3251274369 >> 172.21.15.76:0/3251274369 conn(0x7f90f406ea90 msgr2=0x7f90f406ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:37.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.719+0000 7f90fb290640 1 -- 172.21.15.76:0/3251274369 shutdown_connections 2023-11-09T03:46:37.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.719+0000 7f90fb290640 1 -- 172.21.15.76:0/3251274369 wait complete. 2023-11-09T03:46:37.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.720+0000 7f90fb290640 1 Processor -- start 2023-11-09T03:46:37.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.720+0000 7f90fb290640 1 -- start start 2023-11-09T03:46:37.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.721+0000 7f90fb290640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90f40cf160 0x7f90f40b4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:37.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.721+0000 7f90fb290640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90f415ce60 0x7f90f415f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:37.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.721+0000 7f90fb290640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90f415dc50 0x7f90f4157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:37.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.721+0000 7f90fb290640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f40c9100 con 0x7f90f415dc50 2023-11-09T03:46:37.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.721+0000 7f90fb290640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f415e610 con 0x7f90f415ce60 2023-11-09T03:46:37.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.721+0000 7f90fb290640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f40c9370 con 0x7f90f40cf160 2023-11-09T03:46:37.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.721+0000 7f90f982d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90f415dc50 0x7f90f4157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:37.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.721+0000 7f90f982d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90f415dc50 0x7f90f4157fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35120/0 (socket says 172.21.15.76:35120) 2023-11-09T03:46:37.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.721+0000 7f90f982d640 1 -- 172.21.15.76:0/2961349415 learned_addr learned my addr 172.21.15.76:0/2961349415 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:37.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.721+0000 7f90f902c640 1 --2- 172.21.15.76:0/2961349415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90f40cf160 0x7f90f40b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:37.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.721+0000 7f90f882b640 1 --2- 172.21.15.76:0/2961349415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90f415ce60 0x7f90f415f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:37.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.721+0000 7f90f982d640 1 -- 172.21.15.76:0/2961349415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90f40cf160 msgr2=0x7f90f40b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:37.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.721+0000 7f90f982d640 1 --2- 172.21.15.76:0/2961349415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90f40cf160 0x7f90f40b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:37.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.721+0000 7f90f982d640 1 -- 172.21.15.76:0/2961349415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90f415ce60 msgr2=0x7f90f415f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:37.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.721+0000 7f90f982d640 1 --2- 172.21.15.76:0/2961349415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90f415ce60 0x7f90f415f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:37.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.721+0000 7f90f982d640 1 -- 172.21.15.76:0/2961349415 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ec009a70 con 0x7f90f415dc50 2023-11-09T03:46:37.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.721+0000 7f90f982d640 1 --2- 172.21.15.76:0/2961349415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90f415dc50 0x7f90f4157fb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f90ec009d90 tx=0x7f90ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:37.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.723+0000 7f90e9ffb640 1 -- 172.21.15.76:0/2961349415 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec040650 con 0x7f90f415dc50 2023-11-09T03:46:37.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.723+0000 7f90e9ffb640 1 -- 172.21.15.76:0/2961349415 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90ec0407d0 con 0x7f90f415dc50 2023-11-09T03:46:37.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.723+0000 7f90fb290640 1 -- 172.21.15.76:0/2961349415 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f40c9650 con 0x7f90f415dc50 2023-11-09T03:46:37.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.723+0000 7f90e9ffb640 1 -- 172.21.15.76:0/2961349415 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec040b80 con 0x7f90f415dc50 2023-11-09T03:46:37.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.723+0000 7f90fb290640 1 -- 172.21.15.76:0/2961349415 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f415fb30 con 0x7f90f415dc50 2023-11-09T03:46:37.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.724+0000 7f90fb290640 1 -- 172.21.15.76:0/2961349415 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f40c9b70 con 0x7f90f415dc50 2023-11-09T03:46:37.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.724+0000 7f90e9ffb640 1 -- 172.21.15.76:0/2961349415 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f90ec040ce0 con 0x7f90f415dc50 2023-11-09T03:46:37.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.725+0000 7f90e9ffb640 1 --2- 172.21.15.76:0/2961349415 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f90d806aea0 0x7f90d806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:37.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.725+0000 7f90e9ffb640 1 -- 172.21.15.76:0/2961349415 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f90ec0ac5f0 con 0x7f90f415dc50 2023-11-09T03:46:37.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.725+0000 7f90f902c640 1 --2- 172.21.15.76:0/2961349415 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f90d806aea0 0x7f90d806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:37.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.725+0000 7f90f902c640 1 --2- 172.21.15.76:0/2961349415 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f90d806aea0 0x7f90d806d360 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f90f406b760 tx=0x7f90f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:37.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.729+0000 7f90e9ffb640 1 -- 172.21.15.76:0/2961349415 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f90f40c9b70 con 0x7f90f415dc50 2023-11-09T03:46:37.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.930+0000 7f90fb290640 1 -- 172.21.15.76:0/2961349415 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90f40c9b70 con 0x7f90f415dc50 2023-11-09T03:46:37.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.930+0000 7f90e9ffb640 1 -- 172.21.15.76:0/2961349415 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4915 (secure 0 0 0) 0x7f90f40c9b70 con 0x7f90f415dc50 2023-11-09T03:46:37.931 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:37.931 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2686298435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2686298435},{"type":"v1","addr":"172.21.15.76:6833","nonce":2686298435}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2132717123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2132717123},{"type":"v1","addr":"172.21.15.76:6835","nonce":2132717123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4281,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2369499936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2369499936},{"type":"v1","addr":"172.21.15.113:6837","nonce":2369499936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:45:30.784155+0000","modified":"2023-11-09T03:45:47.804242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/3293223710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3293223710},{"type":"v1","addr":"172.21.15.113:6836","nonce":3293223710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-09T03:46:37.933 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 9 2023-11-09T03:46:37.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.932+0000 7f90fb290640 1 -- 172.21.15.76:0/2961349415 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f90d806aea0 msgr2=0x7f90d806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:37.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.932+0000 7f90fb290640 1 --2- 172.21.15.76:0/2961349415 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f90d806aea0 0x7f90d806d360 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f90f406b760 tx=0x7f90f0008040 comp rx=0 tx=0).stop 2023-11-09T03:46:37.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.932+0000 7f90fb290640 1 -- 172.21.15.76:0/2961349415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90f415dc50 msgr2=0x7f90f4157fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:37.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.932+0000 7f90fb290640 1 --2- 172.21.15.76:0/2961349415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90f415dc50 0x7f90f4157fb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f90ec009d90 tx=0x7f90ec004a40 comp rx=0 tx=0).stop 2023-11-09T03:46:37.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.932+0000 7f90fb290640 1 -- 172.21.15.76:0/2961349415 shutdown_connections 2023-11-09T03:46:37.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.932+0000 7f90fb290640 1 --2- 172.21.15.76:0/2961349415 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f90d806aea0 0x7f90d806d360 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:37.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.932+0000 7f90fb290640 1 --2- 172.21.15.76:0/2961349415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90f415dc50 0x7f90f4157fb0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:37.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.932+0000 7f90fb290640 1 --2- 172.21.15.76:0/2961349415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90f415ce60 0x7f90f415f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:37.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.932+0000 7f90fb290640 1 --2- 172.21.15.76:0/2961349415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90f40cf160 0x7f90f40b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:37.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.932+0000 7f90fb290640 1 -- 172.21.15.76:0/2961349415 >> 172.21.15.76:0/2961349415 conn(0x7f90f406ea90 msgr2=0x7f90f415be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:37.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.932+0000 7f90fb290640 1 -- 172.21.15.76:0/2961349415 shutdown_connections 2023-11-09T03:46:37.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:37.932+0000 7f90fb290640 1 -- 172.21.15.76:0/2961349415 wait complete. 2023-11-09T03:46:37.943 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:46:38.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.044+0000 7efcc52d4640 1 Processor -- start 2023-11-09T03:46:38.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.044+0000 7efcc52d4640 1 -- start start 2023-11-09T03:46:38.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.044+0000 7efcc52d4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efcc00ce240 0x7efcc0157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:38.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.044+0000 7efcc52d4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efcc0157540 0x7efcc00d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:38.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.044+0000 7efcc52d4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcc015e000 con 0x7efcc00cde60 2023-11-09T03:46:38.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.044+0000 7efcc52d4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcc015e5c0 con 0x7efcc00ce240 2023-11-09T03:46:38.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.044+0000 7efcc52d4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc015e2e0 con 0x7efcc0157540 2023-11-09T03:46:38.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.044+0000 7efcbeffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7efcc00cde60 0x7efcc015d560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40722/0 (socket says 172.21.15.76:40722) 2023-11-09T03:46:38.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.045+0000 7efcbeffd640 1 -- 172.21.15.76:0/3221363142 learned_addr learned my addr 172.21.15.76:0/3221363142 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:38.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.045+0000 7efcbf7fe640 1 --2- 172.21.15.76:0/3221363142 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efcc0157540 0x7efcc00d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:38.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.045+0000 7efcbe7fc640 1 --2- 172.21.15.76:0/3221363142 >> v2:172.21.15.113:3300/0 conn(0x7efcc00ce240 0x7efcc0157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:38.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.045+0000 7efcbdffb640 1 -- 172.21.15.76:0/3221363142 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3864477458 0 0) 0x7efcc015e000 con 0x7efcc00cde60 2023-11-09T03:46:38.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.045+0000 7efcbdffb640 1 -- 172.21.15.76:0/3221363142 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efca8003520 con 0x7efcc00cde60 2023-11-09T03:46:38.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.045+0000 7efcbdffb640 1 -- 172.21.15.76:0/3221363142 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2218267740 0 0) 0x7efca8003520 con 0x7efcc00cde60 2023-11-09T03:46:38.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.045+0000 7efcbdffb640 1 -- 172.21.15.76:0/3221363142 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efcc0157540 msgr2=0x7efcc00d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:38.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.045+0000 7efcbdffb640 1 --2- 172.21.15.76:0/3221363142 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efcc0157540 0x7efcc00d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.045+0000 7efcbdffb640 1 -- 172.21.15.76:0/3221363142 >> v2:172.21.15.113:3300/0 conn(0x7efcc00ce240 msgr2=0x7efcc0157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:38.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.045+0000 7efcbdffb640 1 --2- 172.21.15.76:0/3221363142 >> v2:172.21.15.113:3300/0 conn(0x7efcc00ce240 0x7efcc0157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.045+0000 7efcbdffb640 1 -- 172.21.15.76:0/3221363142 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc00d1120 con 0x7efcc00cde60 2023-11-09T03:46:38.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.045+0000 7efcbdffb640 1 -- 172.21.15.76:0/3221363142 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7efcb0001670 con 0x7efcc00cde60 2023-11-09T03:46:38.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.045+0000 7efcbf7fe640 1 --2- 172.21.15.76:0/3221363142 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efcc0157540 0x7efcc00d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:38.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.045+0000 7efcbdffb640 1 -- 172.21.15.76:0/3221363142 >> v1:172.21.15.76:6789/0 conn(0x7efcc00cde60 legacy=0x7efcc015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:38.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.045+0000 7efcbdffb640 1 --2- 172.21.15.76:0/3221363142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efca8003970 0x7efca8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:38.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.045+0000 7efcbdffb640 1 --2- 172.21.15.76:0/3221363142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efca80047c0 0x7efca8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:38.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.045+0000 7efcbdffb640 1 --2- 172.21.15.76:0/3221363142 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efcc0157540 0x7efca8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:38.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.045+0000 7efcbdffb640 1 -- 172.21.15.76:0/3221363142 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc015e5c0 con 0x7efca8003970 2023-11-09T03:46:38.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.045+0000 7efcbdffb640 1 -- 172.21.15.76:0/3221363142 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc015e2e0 con 0x7efca80047c0 2023-11-09T03:46:38.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.045+0000 7efcbdffb640 1 -- 172.21.15.76:0/3221363142 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca800b980 con 0x7efcc0157540 2023-11-09T03:46:38.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.045+0000 7efcbeffd640 1 --2- 172.21.15.76:0/3221363142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efca8003970 0x7efca8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:38.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.045+0000 7efcbe7fc640 1 --2- 172.21.15.76:0/3221363142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efca80047c0 0x7efca8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:38.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.045+0000 7efcbf7fe640 1 --2- 172.21.15.76:0/3221363142 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efcc0157540 0x7efca8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:38.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.045+0000 7efcbeffd640 1 -- 172.21.15.76:0/3221363142 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efcc0157540 msgr2=0x7efca8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:38.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.045+0000 7efcbeffd640 1 --2- 172.21.15.76:0/3221363142 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efcc0157540 0x7efca8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.045+0000 7efcbeffd640 1 -- 172.21.15.76:0/3221363142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efca80047c0 msgr2=0x7efca8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:38.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.045+0000 7efcbeffd640 1 --2- 172.21.15.76:0/3221363142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efca80047c0 0x7efca8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.046+0000 7efcbeffd640 1 -- 172.21.15.76:0/3221363142 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca8003520 con 0x7efca8003970 2023-11-09T03:46:38.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.046+0000 7efcbf7fe640 1 --2- 172.21.15.76:0/3221363142 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efcc0157540 0x7efca8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:38.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.046+0000 7efcbeffd640 1 --2- 172.21.15.76:0/3221363142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efca8003970 0x7efca8003d90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7efcb000c270 tx=0x7efca80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10220fef18003a49 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:38.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.046+0000 7efcbdffb640 1 -- 172.21.15.76:0/3221363142 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb002cae0 con 0x7efca8003970 2023-11-09T03:46:38.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.046+0000 7efcbdffb640 1 -- 172.21.15.76:0/3221363142 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb002cc40 con 0x7efca8003970 2023-11-09T03:46:38.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.046+0000 7efcbdffb640 1 -- 172.21.15.76:0/3221363142 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb00058d0 con 0x7efca8003970 2023-11-09T03:46:38.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.047+0000 7efcc52d4640 1 -- 172.21.15.76:0/3221363142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efca8003970 msgr2=0x7efca8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:38.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.047+0000 7efcc52d4640 1 --2- 172.21.15.76:0/3221363142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efca8003970 0x7efca8003d90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7efcb000c270 tx=0x7efca80057f0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.047+0000 7efcbf7fe640 1 -- 172.21.15.76:0/3221363142 reap_dead start 2023-11-09T03:46:38.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.047+0000 7efcc52d4640 1 -- 172.21.15.76:0/3221363142 shutdown_connections 2023-11-09T03:46:38.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.047+0000 7efcc52d4640 1 -- 172.21.15.76:0/3221363142 >> 172.21.15.76:0/3221363142 conn(0x7efcc0071f30 msgr2=0x7efcc00cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:38.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.047+0000 7efcc52d4640 1 -- 172.21.15.76:0/3221363142 shutdown_connections 2023-11-09T03:46:38.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.047+0000 7efcc52d4640 1 -- 172.21.15.76:0/3221363142 wait complete. 2023-11-09T03:46:38.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.048+0000 7efcc52d4640 1 Processor -- start 2023-11-09T03:46:38.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.048+0000 7efcc52d4640 1 -- start start 2023-11-09T03:46:38.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.049+0000 7efcc52d4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efca8003970 0x7efcc00cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:38.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.049+0000 7efcc52d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcc0159080 0x7efcc01594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:38.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.049+0000 7efcc52d4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efcc0157b30 0x7efcc0157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:38.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.049+0000 7efcc52d4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc015aa30 con 0x7efcc0159080 2023-11-09T03:46:38.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.049+0000 7efcc52d4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc0159e20 con 0x7efca8003970 2023-11-09T03:46:38.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.049+0000 7efcc52d4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc015a100 con 0x7efcc0157b30 2023-11-09T03:46:38.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.049+0000 7efcbe7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcc0159080 0x7efcc01594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:38.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.049+0000 7efcbe7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcc0159080 0x7efcc01594e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35130/0 (socket says 172.21.15.76:35130) 2023-11-09T03:46:38.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.049+0000 7efcbe7fc640 1 -- 172.21.15.76:0/1875218866 learned_addr learned my addr 172.21.15.76:0/1875218866 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:38.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.049+0000 7efcbf7fe640 1 --2- 172.21.15.76:0/1875218866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efcc0157b30 0x7efcc0157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:38.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.049+0000 7efcbe7fc640 1 -- 172.21.15.76:0/1875218866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efcc0157b30 msgr2=0x7efcc0157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:38.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.049+0000 7efcbe7fc640 1 --2- 172.21.15.76:0/1875218866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efcc0157b30 0x7efcc0157fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.049+0000 7efcbe7fc640 1 -- 172.21.15.76:0/1875218866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efca8003970 msgr2=0x7efcc00cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:38.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.049+0000 7efcbeffd640 1 --2- 172.21.15.76:0/1875218866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efca8003970 0x7efcc00cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:38.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.049+0000 7efcbe7fc640 1 --2- 172.21.15.76:0/1875218866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efca8003970 0x7efcc00cbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.049+0000 7efcbe7fc640 1 -- 172.21.15.76:0/1875218866 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc015a3e0 con 0x7efcc0159080 2023-11-09T03:46:38.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.050+0000 7efcbe7fc640 1 --2- 172.21.15.76:0/1875218866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcc0159080 0x7efcc01594e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7efcb4011a70 tx=0x7efcb4011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:38.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.050+0000 7efca77fe640 1 -- 172.21.15.76:0/1875218866 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb4009680 con 0x7efcc0159080 2023-11-09T03:46:38.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.050+0000 7efca77fe640 1 -- 172.21.15.76:0/1875218866 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb400fb50 con 0x7efcc0159080 2023-11-09T03:46:38.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.050+0000 7efca77fe640 1 -- 172.21.15.76:0/1875218866 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb400fee0 con 0x7efcc0159080 2023-11-09T03:46:38.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.050+0000 7efcc52d4640 1 -- 172.21.15.76:0/1875218866 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc015a6c0 con 0x7efcc0159080 2023-11-09T03:46:38.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.050+0000 7efcc52d4640 1 -- 172.21.15.76:0/1875218866 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc00cc230 con 0x7efcc0159080 2023-11-09T03:46:38.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.051+0000 7efcc52d4640 1 -- 172.21.15.76:0/1875218866 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcc015aba0 con 0x7efcc0159080 2023-11-09T03:46:38.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.051+0000 7efca77fe640 1 -- 172.21.15.76:0/1875218866 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efcb4014440 con 0x7efcc0159080 2023-11-09T03:46:38.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.051+0000 7efca77fe640 1 --2- 172.21.15.76:0/1875218866 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efcac06ae70 0x7efcac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:38.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.052+0000 7efca77fe640 1 -- 172.21.15.76:0/1875218866 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7efcb408a250 con 0x7efcc0159080 2023-11-09T03:46:38.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.052+0000 7efcbeffd640 1 --2- 172.21.15.76:0/1875218866 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efcac06ae70 0x7efcac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:38.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.052+0000 7efcbeffd640 1 --2- 172.21.15.76:0/1875218866 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efcac06ae70 0x7efcac06d330 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efcb0034000 tx=0x7efcb0007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:38.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.056+0000 7efca77fe640 1 -- 172.21.15.76:0/1875218866 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efcb4054f20 con 0x7efcc0159080 2023-11-09T03:46:38.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.225+0000 7efcc52d4640 1 -- 172.21.15.76:0/1875218866 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efcc0074290 con 0x7efcc0159080 2023-11-09T03:46:38.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.226+0000 7efca77fe640 1 -- 172.21.15.76:0/1875218866 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18710 (secure 0 0 0) 0x7efcb4058540 con 0x7efcc0159080 2023-11-09T03:46:38.227 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:38.227 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":27,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:46:12.537265+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:46:38.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.228+0000 7efcc52d4640 1 -- 172.21.15.76:0/1875218866 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efcac06ae70 msgr2=0x7efcac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:38.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.228+0000 7efcc52d4640 1 --2- 172.21.15.76:0/1875218866 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efcac06ae70 0x7efcac06d330 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efcb0034000 tx=0x7efcb0007000 comp rx=0 tx=0).stop 2023-11-09T03:46:38.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.228+0000 7efcc52d4640 1 -- 172.21.15.76:0/1875218866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcc0159080 msgr2=0x7efcc01594e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:38.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.228+0000 7efcc52d4640 1 --2- 172.21.15.76:0/1875218866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcc0159080 0x7efcc01594e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7efcb4011a70 tx=0x7efcb4011f30 comp rx=0 tx=0).stop 2023-11-09T03:46:38.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.228+0000 7efcc52d4640 1 -- 172.21.15.76:0/1875218866 shutdown_connections 2023-11-09T03:46:38.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.228+0000 7efcc52d4640 1 --2- 172.21.15.76:0/1875218866 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efcac06ae70 0x7efcac06d330 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.228+0000 7efcc52d4640 1 --2- 172.21.15.76:0/1875218866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efcc0157b30 0x7efcc0157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.228+0000 7efcc52d4640 1 --2- 172.21.15.76:0/1875218866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcc0159080 0x7efcc01594e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.228+0000 7efcc52d4640 1 --2- 172.21.15.76:0/1875218866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efca8003970 0x7efcc00cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.228+0000 7efcc52d4640 1 -- 172.21.15.76:0/1875218866 >> 172.21.15.76:0/1875218866 conn(0x7efcc0071f30 msgr2=0x7efcc00b5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:38.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.228+0000 7efcc52d4640 1 -- 172.21.15.76:0/1875218866 shutdown_connections 2023-11-09T03:46:38.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.228+0000 7efcc52d4640 1 -- 172.21.15.76:0/1875218866 wait complete. 2023-11-09T03:46:38.240 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:45:28.704715+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:45:29.717614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-09T03:45:33.440577+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-09T03:46:38.240 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:46:38.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.343+0000 7fc3fc755640 1 Processor -- start 2023-11-09T03:46:38.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.343+0000 7fc3fc755640 1 -- start start 2023-11-09T03:46:38.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.344+0000 7fc3fc755640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc3f4160430 0x7fc3f40b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:38.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.344+0000 7fc3fc755640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc3f406c3a0 0x7fc3f40cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:38.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.344+0000 7fc3fc755640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3f40b2fc0 con 0x7fc3f40b2b60 2023-11-09T03:46:38.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.344+0000 7fc3fc755640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3f40cb580 con 0x7fc3f4160430 2023-11-09T03:46:38.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.344+0000 7fc3fc755640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3f406c7d0 con 0x7fc3f406c3a0 2023-11-09T03:46:38.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.344+0000 7fc3fa4f1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc3f4160430 0x7fc3f40b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:38.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.344+0000 7fc3fa4f1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc3f4160430 0x7fc3f40b2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35132/0 (socket says 172.21.15.76:35132) 2023-11-09T03:46:38.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.344+0000 7fc3fa4f1640 1 -- 172.21.15.76:0/495151313 learned_addr learned my addr 172.21.15.76:0/495151313 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:38.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.344+0000 7fc3facf2640 1 --2- 172.21.15.76:0/495151313 >> v2:172.21.15.113:3300/0 conn(0x7fc3f406c3a0 0x7fc3f40cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:38.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.345+0000 7fc3fa4f1640 1 -- 172.21.15.76:0/495151313 >> v2:172.21.15.113:3300/0 conn(0x7fc3f406c3a0 msgr2=0x7fc3f40cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:38.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.345+0000 7fc3fa4f1640 1 --2- 172.21.15.76:0/495151313 >> v2:172.21.15.113:3300/0 conn(0x7fc3f406c3a0 0x7fc3f40cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.345+0000 7fc3fa4f1640 1 -- 172.21.15.76:0/495151313 >> v1:172.21.15.113:6789/0 conn(0x7fc3f40b2b60 legacy=0x7fc3f406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:38.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.345+0000 7fc3fa4f1640 1 -- 172.21.15.76:0/495151313 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f406ca40 con 0x7fc3f4160430 2023-11-09T03:46:38.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.345+0000 7fc3fa4f1640 1 --2- 172.21.15.76:0/495151313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3f4160430 0x7fc3f40b2620 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc3f40c4600 tx=0x7fc3f002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38301e34dd86987 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:38.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.345+0000 7fc3f94ef640 1 -- 172.21.15.76:0/495151313 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0030860 con 0x7fc3f4160430 2023-11-09T03:46:38.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.345+0000 7fc3f94ef640 1 -- 172.21.15.76:0/495151313 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f00309c0 con 0x7fc3f4160430 2023-11-09T03:46:38.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.345+0000 7fc3f94ef640 1 -- 172.21.15.76:0/495151313 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0030d50 con 0x7fc3f4160430 2023-11-09T03:46:38.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.346+0000 7fc3fc755640 1 -- 172.21.15.76:0/495151313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3f4160430 msgr2=0x7fc3f40b2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:38.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.346+0000 7fc3fc755640 1 --2- 172.21.15.76:0/495151313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3f4160430 0x7fc3f40b2620 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc3f40c4600 tx=0x7fc3f002b9a0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.346+0000 7fc3fc755640 1 -- 172.21.15.76:0/495151313 shutdown_connections 2023-11-09T03:46:38.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.346+0000 7fc3fc755640 1 --2- 172.21.15.76:0/495151313 >> v2:172.21.15.113:3300/0 conn(0x7fc3f406c3a0 0x7fc3f40cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.346+0000 7fc3fc755640 1 --2- 172.21.15.76:0/495151313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3f4160430 0x7fc3f40b2620 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.346+0000 7fc3fc755640 1 -- 172.21.15.76:0/495151313 >> 172.21.15.76:0/495151313 conn(0x7fc3f406ea20 msgr2=0x7fc3f406ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:38.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.346+0000 7fc3fc755640 1 -- 172.21.15.76:0/495151313 shutdown_connections 2023-11-09T03:46:38.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.346+0000 7fc3fc755640 1 -- 172.21.15.76:0/495151313 wait complete. 2023-11-09T03:46:38.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.347+0000 7fc3fc755640 1 Processor -- start 2023-11-09T03:46:38.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.347+0000 7fc3fc755640 1 -- start start 2023-11-09T03:46:38.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.348+0000 7fc3fc755640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3f406c3a0 0x7fc3f40b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:38.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.348+0000 7fc3fc755640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3f40b2b60 0x7fc3f4152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:38.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.348+0000 7fc3fc755640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3f4160430 0x7fc3f40c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:38.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.348+0000 7fc3fc755640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f40b37c0 con 0x7fc3f406c3a0 2023-11-09T03:46:38.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.348+0000 7fc3fc755640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f4155e60 con 0x7fc3f4160430 2023-11-09T03:46:38.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.348+0000 7fc3fc755640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f4154cf0 con 0x7fc3f40b2b60 2023-11-09T03:46:38.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.348+0000 7fc3fa4f1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3f406c3a0 0x7fc3f40b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:38.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.348+0000 7fc3fa4f1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3f406c3a0 0x7fc3f40b61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35142/0 (socket says 172.21.15.76:35142) 2023-11-09T03:46:38.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.348+0000 7fc3fa4f1640 1 -- 172.21.15.76:0/3900225954 learned_addr learned my addr 172.21.15.76:0/3900225954 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:38.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.348+0000 7fc3facf2640 1 --2- 172.21.15.76:0/3900225954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3f4160430 0x7fc3f40c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:38.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.348+0000 7fc3f9cf0640 1 --2- 172.21.15.76:0/3900225954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3f40b2b60 0x7fc3f4152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:38.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.348+0000 7fc3fa4f1640 1 -- 172.21.15.76:0/3900225954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3f40b2b60 msgr2=0x7fc3f4152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:38.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.348+0000 7fc3fa4f1640 1 --2- 172.21.15.76:0/3900225954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3f40b2b60 0x7fc3f4152370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.348+0000 7fc3fa4f1640 1 -- 172.21.15.76:0/3900225954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3f4160430 msgr2=0x7fc3f40c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:38.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.348+0000 7fc3fa4f1640 1 --2- 172.21.15.76:0/3900225954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3f4160430 0x7fc3f40c69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.348+0000 7fc3fa4f1640 1 -- 172.21.15.76:0/3900225954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f00094d0 con 0x7fc3f406c3a0 2023-11-09T03:46:38.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.348+0000 7fc3f9cf0640 1 --2- 172.21.15.76:0/3900225954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3f40b2b60 0x7fc3f4152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:38.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.349+0000 7fc3fa4f1640 1 --2- 172.21.15.76:0/3900225954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3f406c3a0 0x7fc3f40b61b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc3f002b9e0 tx=0x7fc3f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:38.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.349+0000 7fc3eaffd640 1 -- 172.21.15.76:0/3900225954 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0039a90 con 0x7fc3f406c3a0 2023-11-09T03:46:38.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.349+0000 7fc3eaffd640 1 -- 172.21.15.76:0/3900225954 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f0039c10 con 0x7fc3f406c3a0 2023-11-09T03:46:38.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.349+0000 7fc3eaffd640 1 -- 172.21.15.76:0/3900225954 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0043410 con 0x7fc3f406c3a0 2023-11-09T03:46:38.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.349+0000 7fc3fc755640 1 -- 172.21.15.76:0/3900225954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f4154fd0 con 0x7fc3f406c3a0 2023-11-09T03:46:38.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.349+0000 7fc3fc755640 1 -- 172.21.15.76:0/3900225954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f40b39d0 con 0x7fc3f406c3a0 2023-11-09T03:46:38.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.350+0000 7fc3fc755640 1 -- 172.21.15.76:0/3900225954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f4074290 con 0x7fc3f406c3a0 2023-11-09T03:46:38.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.350+0000 7fc3eaffd640 1 -- 172.21.15.76:0/3900225954 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc3f002f3b0 con 0x7fc3f406c3a0 2023-11-09T03:46:38.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.350+0000 7fc3eaffd640 1 --2- 172.21.15.76:0/3900225954 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc3d806ae70 0x7fc3d806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:38.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.351+0000 7fc3eaffd640 1 -- 172.21.15.76:0/3900225954 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fc3f00abc00 con 0x7fc3f406c3a0 2023-11-09T03:46:38.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.351+0000 7fc3f9cf0640 1 --2- 172.21.15.76:0/3900225954 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc3d806ae70 0x7fc3d806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:38.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.351+0000 7fc3f9cf0640 1 --2- 172.21.15.76:0/3900225954 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc3d806ae70 0x7fc3d806d330 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4007fd0 tx=0x7fc3e4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:38.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.355+0000 7fc3eaffd640 1 -- 172.21.15.76:0/3900225954 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc3f00b2080 con 0x7fc3f406c3a0 2023-11-09T03:46:38.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.520+0000 7fc3fc755640 1 -- 172.21.15.76:0/3900225954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc3f415fac0 con 0x7fc3f406c3a0 2023-11-09T03:46:38.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.520+0000 7fc3eaffd640 1 -- 172.21.15.76:0/3900225954 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc3f0048020 con 0x7fc3f406c3a0 2023-11-09T03:46:38.521 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:46:38.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.522+0000 7fc3fc755640 1 -- 172.21.15.76:0/3900225954 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc3d806ae70 msgr2=0x7fc3d806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:38.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.522+0000 7fc3fc755640 1 --2- 172.21.15.76:0/3900225954 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc3d806ae70 0x7fc3d806d330 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4007fd0 tx=0x7fc3e4009040 comp rx=0 tx=0).stop 2023-11-09T03:46:38.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.522+0000 7fc3fc755640 1 -- 172.21.15.76:0/3900225954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3f406c3a0 msgr2=0x7fc3f40b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:38.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.522+0000 7fc3fc755640 1 --2- 172.21.15.76:0/3900225954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3f406c3a0 0x7fc3f40b61b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc3f002b9e0 tx=0x7fc3f0004730 comp rx=0 tx=0).stop 2023-11-09T03:46:38.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.522+0000 7fc3fc755640 1 -- 172.21.15.76:0/3900225954 shutdown_connections 2023-11-09T03:46:38.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.522+0000 7fc3fc755640 1 --2- 172.21.15.76:0/3900225954 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc3d806ae70 0x7fc3d806d330 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.522+0000 7fc3fc755640 1 --2- 172.21.15.76:0/3900225954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3f4160430 0x7fc3f40c69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.522+0000 7fc3fc755640 1 --2- 172.21.15.76:0/3900225954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3f40b2b60 0x7fc3f4152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.522+0000 7fc3fc755640 1 --2- 172.21.15.76:0/3900225954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3f406c3a0 0x7fc3f40b61b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.522+0000 7fc3fc755640 1 -- 172.21.15.76:0/3900225954 >> 172.21.15.76:0/3900225954 conn(0x7fc3f406ea20 msgr2=0x7fc3f40b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:38.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.522+0000 7fc3fc755640 1 -- 172.21.15.76:0/3900225954 shutdown_connections 2023-11-09T03:46:38.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.522+0000 7fc3fc755640 1 -- 172.21.15.76:0/3900225954 wait complete. 2023-11-09T03:46:38.533 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:46:38.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.636+0000 7f8d7b058640 1 Processor -- start 2023-11-09T03:46:38.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.636+0000 7f8d7b058640 1 -- start start 2023-11-09T03:46:38.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.637+0000 7f8d7b058640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8d740cd6d0 0x7f8d74161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:38.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.637+0000 7f8d7b058640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d740cde60 con 0x7f8d74153e60 2023-11-09T03:46:38.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.637+0000 7f8d7b058640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d74160f90 con 0x7f8d740b4880 2023-11-09T03:46:38.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.637+0000 7f8d7b058640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d74161960 con 0x7f8d740cd6d0 2023-11-09T03:46:38.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.637+0000 7f8d73fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8d740cd6d0 0x7f8d74161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:38.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.637+0000 7f8d78df4640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8d74153e60 0x7f8d7415dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40730/0 (socket says 172.21.15.76:40730) 2023-11-09T03:46:38.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.637+0000 7f8d73fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8d740cd6d0 0x7f8d74161420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35148/0 (socket says 172.21.15.76:35148) 2023-11-09T03:46:38.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.637+0000 7f8d78df4640 1 -- 172.21.15.76:0/4104192319 learned_addr learned my addr 172.21.15.76:0/4104192319 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:38.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.638+0000 7f8d73fff640 1 -- 172.21.15.76:0/4104192319 >> v1:172.21.15.113:6789/0 conn(0x7f8d740b4880 legacy=0x7f8d740d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:38.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.638+0000 7f8d73fff640 1 -- 172.21.15.76:0/4104192319 >> v1:172.21.15.76:6789/0 conn(0x7f8d74153e60 legacy=0x7f8d7415dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:38.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.638+0000 7f8d737fe640 1 -- 172.21.15.76:0/4104192319 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2094910187 0 0) 0x7f8d740cde60 con 0x7f8d74153e60 2023-11-09T03:46:38.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.638+0000 7f8d73fff640 1 -- 172.21.15.76:0/4104192319 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d740ce090 con 0x7f8d740cd6d0 2023-11-09T03:46:38.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.638+0000 7f8d73fff640 1 --2- 172.21.15.76:0/4104192319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d740cd6d0 0x7f8d74161420 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8d680097a0 tx=0x7f8d6802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e793ba2408f7fa15 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:38.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.639+0000 7f8d737fe640 1 -- 172.21.15.76:0/4104192319 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68030610 con 0x7f8d740cd6d0 2023-11-09T03:46:38.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.639+0000 7f8d737fe640 1 -- 172.21.15.76:0/4104192319 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d68030770 con 0x7f8d740cd6d0 2023-11-09T03:46:38.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.639+0000 7f8d737fe640 1 -- 172.21.15.76:0/4104192319 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68030b00 con 0x7f8d740cd6d0 2023-11-09T03:46:38.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.639+0000 7f8d7b058640 1 -- 172.21.15.76:0/4104192319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d740cd6d0 msgr2=0x7f8d74161420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:38.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.639+0000 7f8d7b058640 1 --2- 172.21.15.76:0/4104192319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d740cd6d0 0x7f8d74161420 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8d680097a0 tx=0x7f8d6802b960 comp rx=0 tx=0).stop 2023-11-09T03:46:38.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.639+0000 7f8d7b058640 1 -- 172.21.15.76:0/4104192319 shutdown_connections 2023-11-09T03:46:38.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.639+0000 7f8d7b058640 1 --2- 172.21.15.76:0/4104192319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d740cd6d0 0x7f8d74161420 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.639+0000 7f8d7b058640 1 -- 172.21.15.76:0/4104192319 >> 172.21.15.76:0/4104192319 conn(0x7f8d740cf880 msgr2=0x7f8d740cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:38.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.639+0000 7f8d7b058640 1 -- 172.21.15.76:0/4104192319 shutdown_connections 2023-11-09T03:46:38.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.639+0000 7f8d7b058640 1 -- 172.21.15.76:0/4104192319 wait complete. 2023-11-09T03:46:38.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.640+0000 7f8d7b058640 1 Processor -- start 2023-11-09T03:46:38.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.640+0000 7f8d7b058640 1 -- start start 2023-11-09T03:46:38.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.641+0000 7f8d7b058640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d740b4880 0x7f8d74159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:38.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.641+0000 7f8d7b058640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d740cd6d0 0x7f8d74156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:38.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.641+0000 7f8d7b058640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d74153e60 0x7f8d741f2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:38.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.641+0000 7f8d7b058640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d74157b70 con 0x7f8d740cd6d0 2023-11-09T03:46:38.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.641+0000 7f8d7b058640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d741565a0 con 0x7f8d740b4880 2023-11-09T03:46:38.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.641+0000 7f8d7b058640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d74159fa0 con 0x7f8d74153e60 2023-11-09T03:46:38.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.641+0000 7f8d73fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d740cd6d0 0x7f8d74156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:38.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.641+0000 7f8d73fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d740cd6d0 0x7f8d74156060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35152/0 (socket says 172.21.15.76:35152) 2023-11-09T03:46:38.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.641+0000 7f8d73fff640 1 -- 172.21.15.76:0/1081328922 learned_addr learned my addr 172.21.15.76:0/1081328922 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:38.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.642+0000 7f8d73fff640 1 -- 172.21.15.76:0/1081328922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d74153e60 msgr2=0x7f8d741f2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:38.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.642+0000 7f8d73fff640 1 --2- 172.21.15.76:0/1081328922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d74153e60 0x7f8d741f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.642+0000 7f8d73fff640 1 -- 172.21.15.76:0/1081328922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d740b4880 msgr2=0x7f8d74159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:38.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.642+0000 7f8d73fff640 1 --2- 172.21.15.76:0/1081328922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d740b4880 0x7f8d74159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.642+0000 7f8d73fff640 1 -- 172.21.15.76:0/1081328922 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d68009400 con 0x7f8d740cd6d0 2023-11-09T03:46:38.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.642+0000 7f8d73fff640 1 --2- 172.21.15.76:0/1081328922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d740cd6d0 0x7f8d74156060 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8d680098c0 tx=0x7f8d68004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:38.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.642+0000 7f8d717fa640 1 -- 172.21.15.76:0/1081328922 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6803e040 con 0x7f8d740cd6d0 2023-11-09T03:46:38.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.642+0000 7f8d717fa640 1 -- 172.21.15.76:0/1081328922 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d68039cb0 con 0x7f8d740cd6d0 2023-11-09T03:46:38.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.642+0000 7f8d717fa640 1 -- 172.21.15.76:0/1081328922 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68044630 con 0x7f8d740cd6d0 2023-11-09T03:46:38.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.642+0000 7f8d7b058640 1 -- 172.21.15.76:0/1081328922 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d7415a280 con 0x7f8d740cd6d0 2023-11-09T03:46:38.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.642+0000 7f8d7b058640 1 -- 172.21.15.76:0/1081328922 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d7415a840 con 0x7f8d740cd6d0 2023-11-09T03:46:38.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.643+0000 7f8d7b058640 1 -- 172.21.15.76:0/1081328922 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d74157ce0 con 0x7f8d740cd6d0 2023-11-09T03:46:38.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.643+0000 7f8d717fa640 1 -- 172.21.15.76:0/1081328922 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8d68043390 con 0x7f8d740cd6d0 2023-11-09T03:46:38.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.644+0000 7f8d717fa640 1 --2- 172.21.15.76:0/1081328922 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8d6006ae70 0x7f8d6006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:38.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.644+0000 7f8d717fa640 1 -- 172.21.15.76:0/1081328922 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f8d680abcf0 con 0x7f8d740cd6d0 2023-11-09T03:46:38.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.644+0000 7f8d78df4640 1 --2- 172.21.15.76:0/1081328922 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8d6006ae70 0x7f8d6006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:38.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.644+0000 7f8d78df4640 1 --2- 172.21.15.76:0/1081328922 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8d6006ae70 0x7f8d6006d330 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8d64000c00 tx=0x7f8d640079c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:38.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.648+0000 7f8d717fa640 1 -- 172.21.15.76:0/1081328922 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8d680b0070 con 0x7f8d740cd6d0 2023-11-09T03:46:38.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.815+0000 7f8d7b058640 1 -- 172.21.15.76:0/1081328922 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8d740d0b80 con 0x7f8d740cd6d0 2023-11-09T03:46:38.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.816+0000 7f8d717fa640 1 -- 172.21.15.76:0/1081328922 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8d68034020 con 0x7f8d740cd6d0 2023-11-09T03:46:38.816 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:46:38.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.817+0000 7f8d7b058640 1 -- 172.21.15.76:0/1081328922 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8d6006ae70 msgr2=0x7f8d6006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:38.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.818+0000 7f8d7b058640 1 --2- 172.21.15.76:0/1081328922 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8d6006ae70 0x7f8d6006d330 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8d64000c00 tx=0x7f8d640079c0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.818+0000 7f8d7b058640 1 -- 172.21.15.76:0/1081328922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d740cd6d0 msgr2=0x7f8d74156060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:38.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.818+0000 7f8d7b058640 1 --2- 172.21.15.76:0/1081328922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d740cd6d0 0x7f8d74156060 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8d680098c0 tx=0x7f8d68004730 comp rx=0 tx=0).stop 2023-11-09T03:46:38.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.818+0000 7f8d7b058640 1 -- 172.21.15.76:0/1081328922 shutdown_connections 2023-11-09T03:46:38.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.818+0000 7f8d7b058640 1 --2- 172.21.15.76:0/1081328922 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8d6006ae70 0x7f8d6006d330 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.818+0000 7f8d7b058640 1 --2- 172.21.15.76:0/1081328922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d74153e60 0x7f8d741f2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.818+0000 7f8d7b058640 1 --2- 172.21.15.76:0/1081328922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d740cd6d0 0x7f8d74156060 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.818+0000 7f8d7b058640 1 --2- 172.21.15.76:0/1081328922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d740b4880 0x7f8d74159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.818+0000 7f8d7b058640 1 -- 172.21.15.76:0/1081328922 >> 172.21.15.76:0/1081328922 conn(0x7f8d740cf880 msgr2=0x7f8d7415fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:38.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.818+0000 7f8d7b058640 1 -- 172.21.15.76:0/1081328922 shutdown_connections 2023-11-09T03:46:38.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.818+0000 7f8d7b058640 1 -- 172.21.15.76:0/1081328922 wait complete. 2023-11-09T03:46:38.829 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:46:38.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.931+0000 7f7a9ca44640 1 Processor -- start 2023-11-09T03:46:38.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.931+0000 7f7a9ca44640 1 -- start start 2023-11-09T03:46:38.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.932+0000 7f7a9ca44640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7a98070e20 0x7f7a980adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:38.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.932+0000 7f7a9ca44640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7a980ad730 0x7f7a9815e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:38.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.932+0000 7f7a9ca44640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a980c3900 con 0x7f7a9815ad30 2023-11-09T03:46:38.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.932+0000 7f7a9ca44640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a980ae260 con 0x7f7a980ad730 2023-11-09T03:46:38.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.932+0000 7f7a9ca44640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a980c3be0 con 0x7f7a98070e20 2023-11-09T03:46:38.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.932+0000 7f7a96d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7a980ad730 0x7f7a9815e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:38.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.932+0000 7f7a96d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7a980ad730 0x7f7a9815e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35158/0 (socket says 172.21.15.76:35158) 2023-11-09T03:46:38.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.932+0000 7f7a96d9d640 1 -- 172.21.15.76:0/1002214701 learned_addr learned my addr 172.21.15.76:0/1002214701 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:38.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.932+0000 7f7a8ffff640 1 --2- 172.21.15.76:0/1002214701 >> v2:172.21.15.113:3300/0 conn(0x7f7a98070e20 0x7f7a980adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:38.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.932+0000 7f7a96d9d640 1 -- 172.21.15.76:0/1002214701 >> v2:172.21.15.113:3300/0 conn(0x7f7a98070e20 msgr2=0x7f7a980adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:38.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.932+0000 7f7a96d9d640 1 --2- 172.21.15.76:0/1002214701 >> v2:172.21.15.113:3300/0 conn(0x7f7a98070e20 0x7f7a980adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.932+0000 7f7a96d9d640 1 -- 172.21.15.76:0/1002214701 >> v1:172.21.15.113:6789/0 conn(0x7f7a9815ad30 legacy=0x7f7a98070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:38.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.932+0000 7f7a96d9d640 1 -- 172.21.15.76:0/1002214701 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a980c3ec0 con 0x7f7a980ad730 2023-11-09T03:46:38.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.933+0000 7f7a96d9d640 1 --2- 172.21.15.76:0/1002214701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a980ad730 0x7f7a9815e740 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7a980ce7f0 tx=0x7f7a9002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad1f197af53084a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:38.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.933+0000 7f7a95d9b640 1 -- 172.21.15.76:0/1002214701 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9002c9f0 con 0x7f7a980ad730 2023-11-09T03:46:38.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.933+0000 7f7a95d9b640 1 -- 172.21.15.76:0/1002214701 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9002cb50 con 0x7f7a980ad730 2023-11-09T03:46:38.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.933+0000 7f7a95d9b640 1 -- 172.21.15.76:0/1002214701 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a900343f0 con 0x7f7a980ad730 2023-11-09T03:46:38.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.934+0000 7f7a9ca44640 1 -- 172.21.15.76:0/1002214701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a980ad730 msgr2=0x7f7a9815e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:38.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.934+0000 7f7a9ca44640 1 --2- 172.21.15.76:0/1002214701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a980ad730 0x7f7a9815e740 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7a980ce7f0 tx=0x7f7a9002be50 comp rx=0 tx=0).stop 2023-11-09T03:46:38.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.934+0000 7f7a9ca44640 1 -- 172.21.15.76:0/1002214701 shutdown_connections 2023-11-09T03:46:38.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.934+0000 7f7a9ca44640 1 --2- 172.21.15.76:0/1002214701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a980ad730 0x7f7a9815e740 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.934+0000 7f7a9ca44640 1 --2- 172.21.15.76:0/1002214701 >> v2:172.21.15.113:3300/0 conn(0x7f7a98070e20 0x7f7a980adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.934+0000 7f7a9ca44640 1 -- 172.21.15.76:0/1002214701 >> 172.21.15.76:0/1002214701 conn(0x7f7a980ce950 msgr2=0x7f7a980ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:38.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.934+0000 7f7a9ca44640 1 -- 172.21.15.76:0/1002214701 shutdown_connections 2023-11-09T03:46:38.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.934+0000 7f7a9ca44640 1 -- 172.21.15.76:0/1002214701 wait complete. 2023-11-09T03:46:38.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.935+0000 7f7a9ca44640 1 Processor -- start 2023-11-09T03:46:38.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.935+0000 7f7a9ca44640 1 -- start start 2023-11-09T03:46:38.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.936+0000 7f7a9ca44640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a98070e20 0x7f7a981f4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:38.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.936+0000 7f7a9ca44640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a980ad730 0x7f7a9815a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:38.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.936+0000 7f7a9ca44640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a9815ad30 0x7f7a980b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:38.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.936+0000 7f7a9ca44640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a981f5090 con 0x7f7a980ad730 2023-11-09T03:46:38.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.936+0000 7f7a9ca44640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a981585c0 con 0x7f7a9815ad30 2023-11-09T03:46:38.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.936+0000 7f7a9ca44640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a980b17f0 con 0x7f7a98070e20 2023-11-09T03:46:38.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.936+0000 7f7a8ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a980ad730 0x7f7a9815a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:38.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.936+0000 7f7a8ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a980ad730 0x7f7a9815a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35170/0 (socket says 172.21.15.76:35170) 2023-11-09T03:46:38.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.936+0000 7f7a8ffff640 1 -- 172.21.15.76:0/1077286616 learned_addr learned my addr 172.21.15.76:0/1077286616 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:38.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.936+0000 7f7a8ffff640 1 -- 172.21.15.76:0/1077286616 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a98070e20 msgr2=0x7f7a981f4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:38.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.936+0000 7f7a8ffff640 1 --2- 172.21.15.76:0/1077286616 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a98070e20 0x7f7a981f4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.936+0000 7f7a8ffff640 1 -- 172.21.15.76:0/1077286616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a9815ad30 msgr2=0x7f7a980b12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:38.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.936+0000 7f7a96d9d640 1 --2- 172.21.15.76:0/1077286616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a9815ad30 0x7f7a980b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:38.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.936+0000 7f7a8ffff640 1 --2- 172.21.15.76:0/1077286616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a9815ad30 0x7f7a980b12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:38.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.936+0000 7f7a8ffff640 1 -- 172.21.15.76:0/1077286616 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a90009b30 con 0x7f7a980ad730 2023-11-09T03:46:38.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.936+0000 7f7a8ffff640 1 --2- 172.21.15.76:0/1077286616 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a980ad730 0x7f7a9815a140 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7a8000c820 tx=0x7f7a8000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:38.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.936+0000 7f7a8effd640 1 -- 172.21.15.76:0/1077286616 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a80004220 con 0x7f7a980ad730 2023-11-09T03:46:38.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.936+0000 7f7a8effd640 1 -- 172.21.15.76:0/1077286616 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a800043a0 con 0x7f7a980ad730 2023-11-09T03:46:38.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.936+0000 7f7a8effd640 1 -- 172.21.15.76:0/1077286616 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a80007a70 con 0x7f7a980ad730 2023-11-09T03:46:38.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.936+0000 7f7a9ca44640 1 -- 172.21.15.76:0/1077286616 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a98151350 con 0x7f7a980ad730 2023-11-09T03:46:38.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.936+0000 7f7a9ca44640 1 -- 172.21.15.76:0/1077286616 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a981f4ad0 con 0x7f7a980ad730 2023-11-09T03:46:38.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.937+0000 7f7a9ca44640 1 -- 172.21.15.76:0/1077286616 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a98158320 con 0x7f7a980ad730 2023-11-09T03:46:38.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.938+0000 7f7a8effd640 1 -- 172.21.15.76:0/1077286616 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7a80007bd0 con 0x7f7a980ad730 2023-11-09T03:46:38.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.938+0000 7f7a8effd640 1 --2- 172.21.15.76:0/1077286616 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7a7806ae70 0x7f7a7806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:38.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.938+0000 7f7a8effd640 1 -- 172.21.15.76:0/1077286616 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f7a800896e0 con 0x7f7a980ad730 2023-11-09T03:46:38.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.939+0000 7f7a9659c640 1 --2- 172.21.15.76:0/1077286616 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7a7806ae70 0x7f7a7806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:38.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.939+0000 7f7a9659c640 1 --2- 172.21.15.76:0/1077286616 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7a7806ae70 0x7f7a7806d330 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009650 tx=0x7f7a88006ba0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:38.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:38.943+0000 7f7a8effd640 1 -- 172.21.15.76:0/1077286616 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7a98158320 con 0x7f7a980ad730 2023-11-09T03:46:39.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.107+0000 7f7a9ca44640 1 -- 172.21.15.76:0/1077286616 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7a98158320 con 0x7f7a980ad730 2023-11-09T03:46:39.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.108+0000 7f7a8effd640 1 -- 172.21.15.76:0/1077286616 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7a98158320 con 0x7f7a980ad730 2023-11-09T03:46:39.108 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:46:39.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.109+0000 7f7a9ca44640 1 -- 172.21.15.76:0/1077286616 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7a7806ae70 msgr2=0x7f7a7806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:39.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.109+0000 7f7a9ca44640 1 --2- 172.21.15.76:0/1077286616 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7a7806ae70 0x7f7a7806d330 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009650 tx=0x7f7a88006ba0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.110+0000 7f7a9ca44640 1 -- 172.21.15.76:0/1077286616 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a980ad730 msgr2=0x7f7a9815a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:39.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.110+0000 7f7a9ca44640 1 --2- 172.21.15.76:0/1077286616 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a980ad730 0x7f7a9815a140 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7a8000c820 tx=0x7f7a8000cce0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.110+0000 7f7a9ca44640 1 -- 172.21.15.76:0/1077286616 shutdown_connections 2023-11-09T03:46:39.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.110+0000 7f7a9ca44640 1 --2- 172.21.15.76:0/1077286616 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7a7806ae70 0x7f7a7806d330 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.110+0000 7f7a9ca44640 1 --2- 172.21.15.76:0/1077286616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a9815ad30 0x7f7a980b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.110+0000 7f7a9ca44640 1 --2- 172.21.15.76:0/1077286616 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a980ad730 0x7f7a9815a140 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.110+0000 7f7a9ca44640 1 --2- 172.21.15.76:0/1077286616 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a98070e20 0x7f7a981f4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.110+0000 7f7a9ca44640 1 -- 172.21.15.76:0/1077286616 >> 172.21.15.76:0/1077286616 conn(0x7f7a980ce950 msgr2=0x7f7a980ca480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:39.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.110+0000 7f7a9ca44640 1 -- 172.21.15.76:0/1077286616 shutdown_connections 2023-11-09T03:46:39.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.110+0000 7f7a9ca44640 1 -- 172.21.15.76:0/1077286616 wait complete. 2023-11-09T03:46:39.121 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-09T03:46:39.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.224+0000 7fa4c4bd7640 1 Processor -- start 2023-11-09T03:46:39.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.224+0000 7fa4c4bd7640 1 -- start start 2023-11-09T03:46:39.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.224+0000 7fa4c4bd7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4c0153850 0x7fa4c015a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:39.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.224+0000 7fa4c4bd7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa4c00cc520 0x7fa4c0070fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:39.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.224+0000 7fa4c4bd7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4c00c3860 con 0x7fa4c00cc9a0 2023-11-09T03:46:39.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.224+0000 7fa4c4bd7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4c00c9250 con 0x7fa4c00cc520 2023-11-09T03:46:39.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.225+0000 7fa4c4bd7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c0153c30 con 0x7fa4c0153850 2023-11-09T03:46:39.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.225+0000 7fa4be59c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4c0153850 0x7fa4c015a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:39.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.225+0000 7fa4b7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa4c00cc520 0x7fa4c0070fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:39.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.225+0000 7fa4bed9d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa4c00cc9a0 0x7fa4c00c3cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:56122/0 (socket says 172.21.15.76:56122) 2023-11-09T03:46:39.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.225+0000 7fa4bed9d640 1 -- 172.21.15.76:0/2683115203 learned_addr learned my addr 172.21.15.76:0/2683115203 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:39.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.225+0000 7fa4be59c640 1 -- 172.21.15.76:0/2683115203 >> v2:172.21.15.113:3300/0 conn(0x7fa4c00cc520 msgr2=0x7fa4c0070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:39.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.225+0000 7fa4be59c640 1 --2- 172.21.15.76:0/2683115203 >> v2:172.21.15.113:3300/0 conn(0x7fa4c00cc520 0x7fa4c0070fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.225+0000 7fa4be59c640 1 -- 172.21.15.76:0/2683115203 >> v1:172.21.15.113:6789/0 conn(0x7fa4c00cc9a0 legacy=0x7fa4c00c3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:39.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.225+0000 7fa4be59c640 1 -- 172.21.15.76:0/2683115203 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c015a700 con 0x7fa4c0153850 2023-11-09T03:46:39.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.225+0000 7fa4b7fff640 1 --2- 172.21.15.76:0/2683115203 >> v2:172.21.15.113:3300/0 conn(0x7fa4c00cc520 0x7fa4c0070fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:46:39.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.226+0000 7fa4be59c640 1 --2- 172.21.15.76:0/2683115203 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4c0153850 0x7fa4c015a1c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa4b00097a0 tx=0x7fa4b002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=163d476fc9454ed7 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:39.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.226+0000 7fa4bdd9b640 1 -- 172.21.15.76:0/2683115203 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0031830 con 0x7fa4c0153850 2023-11-09T03:46:39.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.226+0000 7fa4bdd9b640 1 -- 172.21.15.76:0/2683115203 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b0031990 con 0x7fa4c0153850 2023-11-09T03:46:39.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.226+0000 7fa4bdd9b640 1 -- 172.21.15.76:0/2683115203 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0031d20 con 0x7fa4c0153850 2023-11-09T03:46:39.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.227+0000 7fa4c4bd7640 1 -- 172.21.15.76:0/2683115203 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4c0153850 msgr2=0x7fa4c015a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:39.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.227+0000 7fa4c4bd7640 1 --2- 172.21.15.76:0/2683115203 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4c0153850 0x7fa4c015a1c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa4b00097a0 tx=0x7fa4b002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.227+0000 7fa4c4bd7640 1 -- 172.21.15.76:0/2683115203 shutdown_connections 2023-11-09T03:46:39.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.227+0000 7fa4c4bd7640 1 --2- 172.21.15.76:0/2683115203 >> v2:172.21.15.113:3300/0 conn(0x7fa4c00cc520 0x7fa4c0070fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.227+0000 7fa4c4bd7640 1 --2- 172.21.15.76:0/2683115203 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4c0153850 0x7fa4c015a1c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.227+0000 7fa4c4bd7640 1 -- 172.21.15.76:0/2683115203 >> 172.21.15.76:0/2683115203 conn(0x7fa4c00ce950 msgr2=0x7fa4c00ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:39.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.227+0000 7fa4c4bd7640 1 -- 172.21.15.76:0/2683115203 shutdown_connections 2023-11-09T03:46:39.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.227+0000 7fa4c4bd7640 1 -- 172.21.15.76:0/2683115203 wait complete. 2023-11-09T03:46:39.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.228+0000 7fa4c4bd7640 1 Processor -- start 2023-11-09T03:46:39.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.228+0000 7fa4c4bd7640 1 -- start start 2023-11-09T03:46:39.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.229+0000 7fa4c4bd7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4c00cc520 0x7fa4c01f44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:39.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.229+0000 7fa4c4bd7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4c00cc9a0 0x7fa4c00c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:39.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.229+0000 7fa4c4bd7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c0153850 0x7fa4c01f05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:39.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.229+0000 7fa4c4bd7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c01f14b0 con 0x7fa4c00cc9a0 2023-11-09T03:46:39.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.229+0000 7fa4c4bd7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c01f0ef0 con 0x7fa4c0153850 2023-11-09T03:46:39.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.229+0000 7fa4c4bd7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c01f11d0 con 0x7fa4c00cc520 2023-11-09T03:46:39.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.229+0000 7fa4b7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4c00cc9a0 0x7fa4c00c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:39.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.229+0000 7fa4b7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4c00cc9a0 0x7fa4c00c6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35180/0 (socket says 172.21.15.76:35180) 2023-11-09T03:46:39.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.229+0000 7fa4b7fff640 1 -- 172.21.15.76:0/1637670831 learned_addr learned my addr 172.21.15.76:0/1637670831 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:39.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.229+0000 7fa4be59c640 1 --2- 172.21.15.76:0/1637670831 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4c00cc520 0x7fa4c01f44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:39.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.229+0000 7fa4bed9d640 1 --2- 172.21.15.76:0/1637670831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c0153850 0x7fa4c01f05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:39.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.229+0000 7fa4b7fff640 1 -- 172.21.15.76:0/1637670831 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4c00cc520 msgr2=0x7fa4c01f44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:39.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.229+0000 7fa4b7fff640 1 --2- 172.21.15.76:0/1637670831 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4c00cc520 0x7fa4c01f44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.229+0000 7fa4b7fff640 1 -- 172.21.15.76:0/1637670831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c0153850 msgr2=0x7fa4c01f05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:39.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.229+0000 7fa4b7fff640 1 --2- 172.21.15.76:0/1637670831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c0153850 0x7fa4c01f05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.229+0000 7fa4b7fff640 1 -- 172.21.15.76:0/1637670831 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a8009400 con 0x7fa4c00cc9a0 2023-11-09T03:46:39.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.229+0000 7fa4b7fff640 1 --2- 172.21.15.76:0/1637670831 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4c00cc9a0 0x7fa4c00c6dc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800ebe0 tx=0x7fa4a800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:39.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.229+0000 7fa4b6ffd640 1 -- 172.21.15.76:0/1637670831 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8009760 con 0x7fa4c00cc9a0 2023-11-09T03:46:39.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.229+0000 7fa4b6ffd640 1 -- 172.21.15.76:0/1637670831 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a800cc30 con 0x7fa4c00cc9a0 2023-11-09T03:46:39.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.229+0000 7fa4b6ffd640 1 -- 172.21.15.76:0/1637670831 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a800eee0 con 0x7fa4c00cc9a0 2023-11-09T03:46:39.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.230+0000 7fa4c4bd7640 1 -- 172.21.15.76:0/1637670831 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4b0009400 con 0x7fa4c00cc9a0 2023-11-09T03:46:39.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.230+0000 7fa4c4bd7640 1 -- 172.21.15.76:0/1637670831 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c01f3f80 con 0x7fa4c00cc9a0 2023-11-09T03:46:39.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.230+0000 7fa4c4bd7640 1 -- 172.21.15.76:0/1637670831 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4c0074290 con 0x7fa4c00cc9a0 2023-11-09T03:46:39.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.231+0000 7fa4b6ffd640 1 -- 172.21.15.76:0/1637670831 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa4a8013470 con 0x7fa4c00cc9a0 2023-11-09T03:46:39.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.231+0000 7fa4b6ffd640 1 --2- 172.21.15.76:0/1637670831 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa4a006ae70 0x7fa4a006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:39.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.231+0000 7fa4b6ffd640 1 -- 172.21.15.76:0/1637670831 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fa4a8089cf0 con 0x7fa4c00cc9a0 2023-11-09T03:46:39.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.232+0000 7fa4be59c640 1 --2- 172.21.15.76:0/1637670831 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa4a006ae70 0x7fa4a006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:39.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.232+0000 7fa4be59c640 1 --2- 172.21.15.76:0/1637670831 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa4a006ae70 0x7fa4a006d330 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0036000 tx=0x7fa4b003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:39.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.236+0000 7fa4b6ffd640 1 -- 172.21.15.76:0/1637670831 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa4a808e070 con 0x7fa4c00cc9a0 2023-11-09T03:46:39.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.400+0000 7fa4c4bd7640 1 -- 172.21.15.76:0/1637670831 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa4c00c5010 con 0x7fa4c00cc9a0 2023-11-09T03:46:39.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.401+0000 7fa4b6ffd640 1 -- 172.21.15.76:0/1637670831 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa4a80170a0 con 0x7fa4c00cc9a0 2023-11-09T03:46:39.401 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:46:39.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.402+0000 7fa4c4bd7640 1 -- 172.21.15.76:0/1637670831 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa4a006ae70 msgr2=0x7fa4a006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:39.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.402+0000 7fa4c4bd7640 1 --2- 172.21.15.76:0/1637670831 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa4a006ae70 0x7fa4a006d330 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0036000 tx=0x7fa4b003c040 comp rx=0 tx=0).stop 2023-11-09T03:46:39.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.402+0000 7fa4c4bd7640 1 -- 172.21.15.76:0/1637670831 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4c00cc9a0 msgr2=0x7fa4c00c6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:39.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.402+0000 7fa4c4bd7640 1 --2- 172.21.15.76:0/1637670831 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4c00cc9a0 0x7fa4c00c6dc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800ebe0 tx=0x7fa4a800c520 comp rx=0 tx=0).stop 2023-11-09T03:46:39.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.402+0000 7fa4c4bd7640 1 -- 172.21.15.76:0/1637670831 shutdown_connections 2023-11-09T03:46:39.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.402+0000 7fa4c4bd7640 1 --2- 172.21.15.76:0/1637670831 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa4a006ae70 0x7fa4a006d330 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.402+0000 7fa4c4bd7640 1 --2- 172.21.15.76:0/1637670831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c0153850 0x7fa4c01f05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.402+0000 7fa4c4bd7640 1 --2- 172.21.15.76:0/1637670831 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4c00cc9a0 0x7fa4c00c6dc0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.402+0000 7fa4c4bd7640 1 --2- 172.21.15.76:0/1637670831 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4c00cc520 0x7fa4c01f44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.402+0000 7fa4c4bd7640 1 -- 172.21.15.76:0/1637670831 >> 172.21.15.76:0/1637670831 conn(0x7fa4c00ce950 msgr2=0x7fa4c014f370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:39.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.403+0000 7fa4c4bd7640 1 -- 172.21.15.76:0/1637670831 shutdown_connections 2023-11-09T03:46:39.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.403+0000 7fa4c4bd7640 1 -- 172.21.15.76:0/1637670831 wait complete. 2023-11-09T03:46:39.414 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:46:39.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.514+0000 7f6047490640 1 Processor -- start 2023-11-09T03:46:39.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.515+0000 7f6047490640 1 -- start start 2023-11-09T03:46:39.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.515+0000 7f6047490640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6040159c70 0x7f60400ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:39.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.515+0000 7f6047490640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f604006db20 0x7f60400b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:39.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.515+0000 7f6047490640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f60400b0910 0x7f604015d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:39.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.515+0000 7f6047490640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60400b49b0 con 0x7f6040159c70 2023-11-09T03:46:39.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.515+0000 7f6047490640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60400b0da0 con 0x7f60400b0910 2023-11-09T03:46:39.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.515+0000 7f6047490640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60400b4110 con 0x7f604006db20 2023-11-09T03:46:39.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.516+0000 7f604522c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6040159c70 0x7f60400ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:39.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.516+0000 7f604522c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6040159c70 0x7f60400ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35196/0 (socket says 172.21.15.76:35196) 2023-11-09T03:46:39.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.516+0000 7f604522c640 1 -- 172.21.15.76:0/3539882708 learned_addr learned my addr 172.21.15.76:0/3539882708 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:39.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.516+0000 7f6045a2d640 1 --2- 172.21.15.76:0/3539882708 >> v2:172.21.15.113:3300/0 conn(0x7f60400b0910 0x7f604015d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:39.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.516+0000 7f6044a2b640 1 --2- 172.21.15.76:0/3539882708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f604006db20 0x7f60400b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:39.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.516+0000 7f604522c640 1 -- 172.21.15.76:0/3539882708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f604006db20 msgr2=0x7f60400b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:39.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.516+0000 7f604522c640 1 --2- 172.21.15.76:0/3539882708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f604006db20 0x7f60400b03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.516+0000 7f604522c640 1 -- 172.21.15.76:0/3539882708 >> v2:172.21.15.113:3300/0 conn(0x7f60400b0910 msgr2=0x7f604015d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:39.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.516+0000 7f604522c640 1 --2- 172.21.15.76:0/3539882708 >> v2:172.21.15.113:3300/0 conn(0x7f60400b0910 0x7f604015d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.516+0000 7f604522c640 1 -- 172.21.15.76:0/3539882708 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60400b43f0 con 0x7f6040159c70 2023-11-09T03:46:39.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.516+0000 7f6044a2b640 1 --2- 172.21.15.76:0/3539882708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f604006db20 0x7f60400b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:39.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.517+0000 7f604522c640 1 --2- 172.21.15.76:0/3539882708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6040159c70 0x7f60400ce370 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f60400ca130 tx=0x7f603002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2853171a2cd7a82d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:39.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.517+0000 7f6037fff640 1 -- 172.21.15.76:0/3539882708 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6030031870 con 0x7f6040159c70 2023-11-09T03:46:39.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.517+0000 7f6037fff640 1 -- 172.21.15.76:0/3539882708 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60300319d0 con 0x7f6040159c70 2023-11-09T03:46:39.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.517+0000 7f6037fff640 1 -- 172.21.15.76:0/3539882708 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6030031d60 con 0x7f6040159c70 2023-11-09T03:46:39.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.517+0000 7f6047490640 1 -- 172.21.15.76:0/3539882708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6040159c70 msgr2=0x7f60400ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:39.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.517+0000 7f6047490640 1 --2- 172.21.15.76:0/3539882708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6040159c70 0x7f60400ce370 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f60400ca130 tx=0x7f603002b910 comp rx=0 tx=0).stop 2023-11-09T03:46:39.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.517+0000 7f6047490640 1 -- 172.21.15.76:0/3539882708 shutdown_connections 2023-11-09T03:46:39.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.517+0000 7f6047490640 1 --2- 172.21.15.76:0/3539882708 >> v2:172.21.15.113:3300/0 conn(0x7f60400b0910 0x7f604015d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.517+0000 7f6047490640 1 --2- 172.21.15.76:0/3539882708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f604006db20 0x7f60400b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.517+0000 7f6047490640 1 --2- 172.21.15.76:0/3539882708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6040159c70 0x7f60400ce370 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.517+0000 7f6047490640 1 -- 172.21.15.76:0/3539882708 >> 172.21.15.76:0/3539882708 conn(0x7f60400ce950 msgr2=0x7f60400ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:39.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.517+0000 7f6047490640 1 -- 172.21.15.76:0/3539882708 shutdown_connections 2023-11-09T03:46:39.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.517+0000 7f6047490640 1 -- 172.21.15.76:0/3539882708 wait complete. 2023-11-09T03:46:39.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.518+0000 7f6047490640 1 Processor -- start 2023-11-09T03:46:39.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.519+0000 7f6047490640 1 -- start start 2023-11-09T03:46:39.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.519+0000 7f6047490640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f604006db20 0x7f60400c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:39.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.519+0000 7f6047490640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60400b0910 0x7f60400b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:39.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.519+0000 7f6047490640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6040159c70 0x7f60400b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:39.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.519+0000 7f6047490640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60400c88e0 con 0x7f60400b0910 2023-11-09T03:46:39.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.519+0000 7f6047490640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604015a1f0 con 0x7f604006db20 2023-11-09T03:46:39.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.519+0000 7f6047490640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f604015a4d0 con 0x7f6040159c70 2023-11-09T03:46:39.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.519+0000 7f6044a2b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60400b0910 0x7f60400b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:39.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.519+0000 7f6044a2b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60400b0910 0x7f60400b3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35212/0 (socket says 172.21.15.76:35212) 2023-11-09T03:46:39.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.519+0000 7f6044a2b640 1 -- 172.21.15.76:0/805086974 learned_addr learned my addr 172.21.15.76:0/805086974 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:39.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.520+0000 7f6045a2d640 1 --2- 172.21.15.76:0/805086974 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6040159c70 0x7f60400b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:39.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.520+0000 7f604522c640 1 --2- 172.21.15.76:0/805086974 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f604006db20 0x7f60400c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:39.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.520+0000 7f6044a2b640 1 -- 172.21.15.76:0/805086974 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6040159c70 msgr2=0x7f60400b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:39.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.520+0000 7f6044a2b640 1 --2- 172.21.15.76:0/805086974 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6040159c70 0x7f60400b38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.520+0000 7f6044a2b640 1 -- 172.21.15.76:0/805086974 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f604006db20 msgr2=0x7f60400c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:39.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.520+0000 7f6044a2b640 1 --2- 172.21.15.76:0/805086974 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f604006db20 0x7f60400c83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.520+0000 7f6044a2b640 1 -- 172.21.15.76:0/805086974 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60300094d0 con 0x7f60400b0910 2023-11-09T03:46:39.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.520+0000 7f6044a2b640 1 --2- 172.21.15.76:0/805086974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60400b0910 0x7f60400b3370 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f603c00c7f0 tx=0x7f603c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:39.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.520+0000 7f6035ffb640 1 -- 172.21.15.76:0/805086974 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c007820 con 0x7f60400b0910 2023-11-09T03:46:39.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.520+0000 7f6035ffb640 1 -- 172.21.15.76:0/805086974 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603c007980 con 0x7f60400b0910 2023-11-09T03:46:39.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.520+0000 7f6035ffb640 1 -- 172.21.15.76:0/805086974 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c007d30 con 0x7f60400b0910 2023-11-09T03:46:39.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.520+0000 7f6047490640 1 -- 172.21.15.76:0/805086974 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f604015a7b0 con 0x7f60400b0910 2023-11-09T03:46:39.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.520+0000 7f6047490640 1 -- 172.21.15.76:0/805086974 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f604015ad70 con 0x7f60400b0910 2023-11-09T03:46:39.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.521+0000 7f6047490640 1 -- 172.21.15.76:0/805086974 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f604015af90 con 0x7f60400b0910 2023-11-09T03:46:39.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.521+0000 7f6035ffb640 1 -- 172.21.15.76:0/805086974 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f603c0113f0 con 0x7f60400b0910 2023-11-09T03:46:39.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.522+0000 7f6035ffb640 1 --2- 172.21.15.76:0/805086974 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f602406ae70 0x7f602406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:39.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.522+0000 7f6035ffb640 1 -- 172.21.15.76:0/805086974 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f603c089d80 con 0x7f60400b0910 2023-11-09T03:46:39.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.522+0000 7f604522c640 1 --2- 172.21.15.76:0/805086974 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f602406ae70 0x7f602406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:39.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.523+0000 7f604522c640 1 --2- 172.21.15.76:0/805086974 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f602406ae70 0x7f602406d330 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6030036000 tx=0x7f603003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:39.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.526+0000 7f6035ffb640 1 -- 172.21.15.76:0/805086974 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f603c0549d0 con 0x7f60400b0910 2023-11-09T03:46:39.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.731+0000 7f6047490640 1 -- 172.21.15.76:0/805086974 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60400b1d30 con 0x7f60400b0910 2023-11-09T03:46:39.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.731+0000 7f6035ffb640 1 -- 172.21.15.76:0/805086974 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4915 (secure 0 0 0) 0x7f603c057ff0 con 0x7f60400b0910 2023-11-09T03:46:39.732 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:39.732 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2686298435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2686298435},{"type":"v1","addr":"172.21.15.76:6833","nonce":2686298435}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2132717123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2132717123},{"type":"v1","addr":"172.21.15.76:6835","nonce":2132717123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4281,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2369499936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2369499936},{"type":"v1","addr":"172.21.15.113:6837","nonce":2369499936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:45:30.784155+0000","modified":"2023-11-09T03:45:47.804242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/3293223710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3293223710},{"type":"v1","addr":"172.21.15.113:6836","nonce":3293223710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-09T03:46:39.733 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 9 2023-11-09T03:46:39.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.733+0000 7f6047490640 1 -- 172.21.15.76:0/805086974 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f602406ae70 msgr2=0x7f602406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:39.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.733+0000 7f6047490640 1 --2- 172.21.15.76:0/805086974 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f602406ae70 0x7f602406d330 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6030036000 tx=0x7f603003c040 comp rx=0 tx=0).stop 2023-11-09T03:46:39.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.733+0000 7f6047490640 1 -- 172.21.15.76:0/805086974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60400b0910 msgr2=0x7f60400b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:39.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.733+0000 7f6047490640 1 --2- 172.21.15.76:0/805086974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60400b0910 0x7f60400b3370 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f603c00c7f0 tx=0x7f603c00ccb0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.734+0000 7f6047490640 1 -- 172.21.15.76:0/805086974 shutdown_connections 2023-11-09T03:46:39.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.734+0000 7f6047490640 1 --2- 172.21.15.76:0/805086974 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f602406ae70 0x7f602406d330 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.734+0000 7f6047490640 1 --2- 172.21.15.76:0/805086974 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6040159c70 0x7f60400b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.734+0000 7f6047490640 1 --2- 172.21.15.76:0/805086974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60400b0910 0x7f60400b3370 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.734+0000 7f6047490640 1 --2- 172.21.15.76:0/805086974 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f604006db20 0x7f60400c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.734+0000 7f6047490640 1 -- 172.21.15.76:0/805086974 >> 172.21.15.76:0/805086974 conn(0x7f60400ce950 msgr2=0x7f604015d510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:39.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.734+0000 7f6047490640 1 -- 172.21.15.76:0/805086974 shutdown_connections 2023-11-09T03:46:39.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.734+0000 7f6047490640 1 -- 172.21.15.76:0/805086974 wait complete. 2023-11-09T03:46:39.745 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:46:39.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.845+0000 7f0530275640 1 Processor -- start 2023-11-09T03:46:39.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.845+0000 7f0530275640 1 -- start start 2023-11-09T03:46:39.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.846+0000 7f0530275640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0528159c70 0x7f05280ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:39.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.846+0000 7f0530275640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f052806db20 0x7f05280b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:39.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.846+0000 7f0530275640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05280b4110 con 0x7f05280b0910 2023-11-09T03:46:39.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.846+0000 7f0530275640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05280b49b0 con 0x7f0528159c70 2023-11-09T03:46:39.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.846+0000 7f0530275640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05280b0da0 con 0x7f052806db20 2023-11-09T03:46:39.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.846+0000 7f052e011640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0528159c70 0x7f05280ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:39.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.846+0000 7f052d810640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f052806db20 0x7f05280b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:39.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.846+0000 7f052e011640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0528159c70 0x7f05280ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:44712/0 (socket says 172.21.15.76:44712) 2023-11-09T03:46:39.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.846+0000 7f052e011640 1 -- 172.21.15.76:0/2895644254 learned_addr learned my addr 172.21.15.76:0/2895644254 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:39.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.846+0000 7f052e812640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f05280b0910 0x7f052815d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:56138/0 (socket says 172.21.15.76:56138) 2023-11-09T03:46:39.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.847+0000 7f052e011640 1 -- 172.21.15.76:0/2895644254 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f052806db20 msgr2=0x7f05280b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:39.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.847+0000 7f052e011640 1 --2- 172.21.15.76:0/2895644254 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f052806db20 0x7f05280b03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.847+0000 7f052e011640 1 -- 172.21.15.76:0/2895644254 >> v1:172.21.15.113:6789/0 conn(0x7f05280b0910 legacy=0x7f052815d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:39.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.847+0000 7f052e011640 1 -- 172.21.15.76:0/2895644254 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05280b43f0 con 0x7f0528159c70 2023-11-09T03:46:39.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.847+0000 7f052d00f640 1 -- 172.21.15.76:0/2895644254 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1430459151 0 0) 0x7f05280b4110 con 0x7f05280b0910 2023-11-09T03:46:39.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.847+0000 7f052d810640 1 --2- 172.21.15.76:0/2895644254 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f052806db20 0x7f05280b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:46:39.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.847+0000 7f052e011640 1 --2- 172.21.15.76:0/2895644254 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0528159c70 0x7f05280ce370 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0524009e50 tx=0x7f052402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8868f5cba42086ca server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:39.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.848+0000 7f052d00f640 1 -- 172.21.15.76:0/2895644254 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052402cb00 con 0x7f0528159c70 2023-11-09T03:46:39.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.848+0000 7f052d00f640 1 -- 172.21.15.76:0/2895644254 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052402cc60 con 0x7f0528159c70 2023-11-09T03:46:39.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.848+0000 7f052d00f640 1 -- 172.21.15.76:0/2895644254 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05240343f0 con 0x7f0528159c70 2023-11-09T03:46:39.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.848+0000 7f0530275640 1 -- 172.21.15.76:0/2895644254 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0528159c70 msgr2=0x7f05280ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:39.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.848+0000 7f0530275640 1 --2- 172.21.15.76:0/2895644254 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0528159c70 0x7f05280ce370 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0524009e50 tx=0x7f052402bf60 comp rx=0 tx=0).stop 2023-11-09T03:46:39.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.848+0000 7f0530275640 1 -- 172.21.15.76:0/2895644254 shutdown_connections 2023-11-09T03:46:39.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.848+0000 7f0530275640 1 --2- 172.21.15.76:0/2895644254 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f052806db20 0x7f05280b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.848+0000 7f0530275640 1 --2- 172.21.15.76:0/2895644254 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0528159c70 0x7f05280ce370 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.848+0000 7f0530275640 1 -- 172.21.15.76:0/2895644254 >> 172.21.15.76:0/2895644254 conn(0x7f05280ce950 msgr2=0x7f05280ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:39.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.848+0000 7f0530275640 1 -- 172.21.15.76:0/2895644254 shutdown_connections 2023-11-09T03:46:39.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.848+0000 7f0530275640 1 -- 172.21.15.76:0/2895644254 wait complete. 2023-11-09T03:46:39.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.849+0000 7f0530275640 1 Processor -- start 2023-11-09T03:46:39.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.849+0000 7f0530275640 1 -- start start 2023-11-09T03:46:39.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.850+0000 7f0530275640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f052806db20 0x7f052813ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:39.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.850+0000 7f0530275640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05280b0910 0x7f0528155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:39.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.850+0000 7f0530275640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0528159c70 0x7f05280c45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:39.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.850+0000 7f0530275640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0528156f60 con 0x7f05280b0910 2023-11-09T03:46:39.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.850+0000 7f0530275640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052813d3c0 con 0x7f052806db20 2023-11-09T03:46:39.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.850+0000 7f0530275640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05281560d0 con 0x7f0528159c70 2023-11-09T03:46:39.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.850+0000 7f052d810640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05280b0910 0x7f0528155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:39.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.850+0000 7f052d810640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05280b0910 0x7f0528155b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35222/0 (socket says 172.21.15.76:35222) 2023-11-09T03:46:39.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.850+0000 7f052d810640 1 -- 172.21.15.76:0/4019148844 learned_addr learned my addr 172.21.15.76:0/4019148844 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:39.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.850+0000 7f052e812640 1 --2- 172.21.15.76:0/4019148844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0528159c70 0x7f05280c45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:39.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.850+0000 7f052e011640 1 --2- 172.21.15.76:0/4019148844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f052806db20 0x7f052813ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:39.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.850+0000 7f052d810640 1 -- 172.21.15.76:0/4019148844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0528159c70 msgr2=0x7f05280c45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:39.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.850+0000 7f052d810640 1 --2- 172.21.15.76:0/4019148844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0528159c70 0x7f05280c45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.850+0000 7f052d810640 1 -- 172.21.15.76:0/4019148844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f052806db20 msgr2=0x7f052813ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:39.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.850+0000 7f052d810640 1 --2- 172.21.15.76:0/4019148844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f052806db20 0x7f052813ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:39.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.850+0000 7f052d810640 1 -- 172.21.15.76:0/4019148844 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0518009400 con 0x7f05280b0910 2023-11-09T03:46:39.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.851+0000 7f052d810640 1 --2- 172.21.15.76:0/4019148844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05280b0910 0x7f0528155b90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f051800ec10 tx=0x7f051800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:39.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.851+0000 7f051effd640 1 -- 172.21.15.76:0/4019148844 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0518009760 con 0x7f05280b0910 2023-11-09T03:46:39.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.851+0000 7f051effd640 1 -- 172.21.15.76:0/4019148844 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051800cc30 con 0x7f05280b0910 2023-11-09T03:46:39.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.851+0000 7f0530275640 1 -- 172.21.15.76:0/4019148844 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0524009b30 con 0x7f05280b0910 2023-11-09T03:46:39.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.851+0000 7f051effd640 1 -- 172.21.15.76:0/4019148844 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0518013470 con 0x7f05280b0910 2023-11-09T03:46:39.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.851+0000 7f0530275640 1 -- 172.21.15.76:0/4019148844 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0528156630 con 0x7f05280b0910 2023-11-09T03:46:39.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.852+0000 7f0530275640 1 -- 172.21.15.76:0/4019148844 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05281570d0 con 0x7f05280b0910 2023-11-09T03:46:39.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.852+0000 7f051effd640 1 -- 172.21.15.76:0/4019148844 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f05180135d0 con 0x7f05280b0910 2023-11-09T03:46:39.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.853+0000 7f051effd640 1 --2- 172.21.15.76:0/4019148844 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f050c06ae70 0x7f050c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:39.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.853+0000 7f051effd640 1 -- 172.21.15.76:0/4019148844 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f051808a020 con 0x7f05280b0910 2023-11-09T03:46:39.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.853+0000 7f052e011640 1 --2- 172.21.15.76:0/4019148844 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f050c06ae70 0x7f050c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:39.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.853+0000 7f052e011640 1 --2- 172.21.15.76:0/4019148844 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f050c06ae70 0x7f050c06d330 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0524035000 tx=0x7f0524030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:39.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:39.857+0000 7f051effd640 1 -- 172.21.15.76:0/4019148844 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0518054d80 con 0x7f05280b0910 2023-11-09T03:46:40.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.023+0000 7f0530275640 1 -- 172.21.15.76:0/4019148844 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0528074290 con 0x7f05280b0910 2023-11-09T03:46:40.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.025+0000 7f051effd640 1 -- 172.21.15.76:0/4019148844 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18710 (secure 0 0 0) 0x7f05180583a0 con 0x7f05280b0910 2023-11-09T03:46:40.025 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:40.025 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":27,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:46:12.537265+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:46:40.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.027+0000 7f0530275640 1 -- 172.21.15.76:0/4019148844 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f050c06ae70 msgr2=0x7f050c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:40.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.027+0000 7f0530275640 1 --2- 172.21.15.76:0/4019148844 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f050c06ae70 0x7f050c06d330 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0524035000 tx=0x7f0524030000 comp rx=0 tx=0).stop 2023-11-09T03:46:40.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.027+0000 7f0530275640 1 -- 172.21.15.76:0/4019148844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05280b0910 msgr2=0x7f0528155b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:40.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.027+0000 7f0530275640 1 --2- 172.21.15.76:0/4019148844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05280b0910 0x7f0528155b90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f051800ec10 tx=0x7f051800c520 comp rx=0 tx=0).stop 2023-11-09T03:46:40.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.027+0000 7f0530275640 1 -- 172.21.15.76:0/4019148844 shutdown_connections 2023-11-09T03:46:40.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.027+0000 7f0530275640 1 --2- 172.21.15.76:0/4019148844 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f050c06ae70 0x7f050c06d330 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.027+0000 7f0530275640 1 --2- 172.21.15.76:0/4019148844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0528159c70 0x7f05280c45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.027+0000 7f0530275640 1 --2- 172.21.15.76:0/4019148844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05280b0910 0x7f0528155b90 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.027+0000 7f0530275640 1 --2- 172.21.15.76:0/4019148844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f052806db20 0x7f052813ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.027+0000 7f0530275640 1 -- 172.21.15.76:0/4019148844 >> 172.21.15.76:0/4019148844 conn(0x7f05280ce950 msgr2=0x7f052815c060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:40.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.027+0000 7f0530275640 1 -- 172.21.15.76:0/4019148844 shutdown_connections 2023-11-09T03:46:40.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.027+0000 7f0530275640 1 -- 172.21.15.76:0/4019148844 wait complete. 2023-11-09T03:46:40.038 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-09T03:46:40.038 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:46:40.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.137+0000 7f05c8bbb640 1 Processor -- start 2023-11-09T03:46:40.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.137+0000 7f05c8bbb640 1 -- start start 2023-11-09T03:46:40.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.138+0000 7f05c8bbb640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f05c40cde60 0x7f05c415d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:40.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.138+0000 7f05c8bbb640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f05c415daa0 0x7f05c4157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:40.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.138+0000 7f05c8bbb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f05c40ce240 0x7f05c4157540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:40.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.138+0000 7f05c8bbb640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05c415e5c0 con 0x7f05c40cde60 2023-11-09T03:46:40.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.138+0000 7f05c8bbb640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05c4157a80 con 0x7f05c415daa0 2023-11-09T03:46:40.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.138+0000 7f05c8bbb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05c415e000 con 0x7f05c40ce240 2023-11-09T03:46:40.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.138+0000 7f05c259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f05c40cde60 0x7f05c415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:40.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.138+0000 7f05c259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f05c40cde60 0x7f05c415d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35224/0 (socket says 172.21.15.76:35224) 2023-11-09T03:46:40.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.138+0000 7f05c259c640 1 -- 172.21.15.76:0/3601136835 learned_addr learned my addr 172.21.15.76:0/3601136835 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:40.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.138+0000 7f05b9d9b640 1 --2- 172.21.15.76:0/3601136835 >> v2:172.21.15.113:3300/0 conn(0x7f05c415daa0 0x7f05c4157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:40.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.138+0000 7f05c2d9d640 1 --2- 172.21.15.76:0/3601136835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f05c40ce240 0x7f05c4157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:40.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.139+0000 7f05c259c640 1 -- 172.21.15.76:0/3601136835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f05c40ce240 msgr2=0x7f05c4157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:40.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.139+0000 7f05c259c640 1 --2- 172.21.15.76:0/3601136835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f05c40ce240 0x7f05c4157540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.139+0000 7f05c259c640 1 -- 172.21.15.76:0/3601136835 >> v2:172.21.15.113:3300/0 conn(0x7f05c415daa0 msgr2=0x7f05c4157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:40.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.139+0000 7f05c259c640 1 --2- 172.21.15.76:0/3601136835 >> v2:172.21.15.113:3300/0 conn(0x7f05c415daa0 0x7f05c4157000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.139+0000 7f05c259c640 1 -- 172.21.15.76:0/3601136835 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c415e2e0 con 0x7f05c40cde60 2023-11-09T03:46:40.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.139+0000 7f05c259c640 1 --2- 172.21.15.76:0/3601136835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05c40cde60 0x7f05c415d560 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f05c40cf5f0 tx=0x7f05b402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4fc1f5e952b8ab server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:40.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.139+0000 7f05c1d9b640 1 -- 172.21.15.76:0/3601136835 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b4031870 con 0x7f05c40cde60 2023-11-09T03:46:40.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.139+0000 7f05c1d9b640 1 -- 172.21.15.76:0/3601136835 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b40319d0 con 0x7f05c40cde60 2023-11-09T03:46:40.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.139+0000 7f05c1d9b640 1 -- 172.21.15.76:0/3601136835 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b4031d60 con 0x7f05c40cde60 2023-11-09T03:46:40.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.140+0000 7f05c8bbb640 1 -- 172.21.15.76:0/3601136835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05c40cde60 msgr2=0x7f05c415d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:40.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.140+0000 7f05c8bbb640 1 --2- 172.21.15.76:0/3601136835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05c40cde60 0x7f05c415d560 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f05c40cf5f0 tx=0x7f05b402b910 comp rx=0 tx=0).stop 2023-11-09T03:46:40.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.140+0000 7f05c8bbb640 1 -- 172.21.15.76:0/3601136835 shutdown_connections 2023-11-09T03:46:40.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.140+0000 7f05c8bbb640 1 --2- 172.21.15.76:0/3601136835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f05c40ce240 0x7f05c4157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.140+0000 7f05c8bbb640 1 --2- 172.21.15.76:0/3601136835 >> v2:172.21.15.113:3300/0 conn(0x7f05c415daa0 0x7f05c4157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.140+0000 7f05c8bbb640 1 --2- 172.21.15.76:0/3601136835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05c40cde60 0x7f05c415d560 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.140+0000 7f05c8bbb640 1 -- 172.21.15.76:0/3601136835 >> 172.21.15.76:0/3601136835 conn(0x7f05c4071f30 msgr2=0x7f05c40cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:40.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.140+0000 7f05c8bbb640 1 -- 172.21.15.76:0/3601136835 shutdown_connections 2023-11-09T03:46:40.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.140+0000 7f05c8bbb640 1 -- 172.21.15.76:0/3601136835 wait complete. 2023-11-09T03:46:40.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.141+0000 7f05c8bbb640 1 Processor -- start 2023-11-09T03:46:40.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.142+0000 7f05c8bbb640 1 -- start start 2023-11-09T03:46:40.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.142+0000 7f05c8bbb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05c40cde60 0x7f05c414b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:40.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.142+0000 7f05c8bbb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f05c40ce240 0x7f05c41f2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:40.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.142+0000 7f05c8bbb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05c415daa0 0x7f05c4148e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:40.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.142+0000 7f05c8bbb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c415c4a0 con 0x7f05c415daa0 2023-11-09T03:46:40.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.142+0000 7f05c8bbb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c41f5da0 con 0x7f05c40cde60 2023-11-09T03:46:40.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.142+0000 7f05c8bbb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05c414be60 con 0x7f05c40ce240 2023-11-09T03:46:40.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.142+0000 7f05c2d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05c415daa0 0x7f05c4148e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:40.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.142+0000 7f05c2d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05c415daa0 0x7f05c4148e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35228/0 (socket says 172.21.15.76:35228) 2023-11-09T03:46:40.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.142+0000 7f05c2d9d640 1 -- 172.21.15.76:0/2602237457 learned_addr learned my addr 172.21.15.76:0/2602237457 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:40.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.143+0000 7f05c259c640 1 --2- 172.21.15.76:0/2602237457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05c40cde60 0x7f05c414b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:40.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.143+0000 7f05b9d9b640 1 --2- 172.21.15.76:0/2602237457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f05c40ce240 0x7f05c41f2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:40.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.143+0000 7f05c2d9d640 1 -- 172.21.15.76:0/2602237457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f05c40ce240 msgr2=0x7f05c41f2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:40.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.143+0000 7f05c2d9d640 1 --2- 172.21.15.76:0/2602237457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f05c40ce240 0x7f05c41f2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.143+0000 7f05c2d9d640 1 -- 172.21.15.76:0/2602237457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05c40cde60 msgr2=0x7f05c414b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:40.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.143+0000 7f05c2d9d640 1 --2- 172.21.15.76:0/2602237457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05c40cde60 0x7f05c414b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.143+0000 7f05c2d9d640 1 -- 172.21.15.76:0/2602237457 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b40094d0 con 0x7f05c415daa0 2023-11-09T03:46:40.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.143+0000 7f05b9d9b640 1 --2- 172.21.15.76:0/2602237457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f05c40ce240 0x7f05c41f2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:40.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.143+0000 7f05c259c640 1 --2- 172.21.15.76:0/2602237457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05c40cde60 0x7f05c414b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:40.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.143+0000 7f05c2d9d640 1 --2- 172.21.15.76:0/2602237457 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05c415daa0 0x7f05c4148e80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f05bc00cd20 tx=0x7f05bc00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:40.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.143+0000 7f05bb7fe640 1 -- 172.21.15.76:0/2602237457 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc003b90 con 0x7f05c415daa0 2023-11-09T03:46:40.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.143+0000 7f05bb7fe640 1 -- 172.21.15.76:0/2602237457 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05bc003cf0 con 0x7f05c415daa0 2023-11-09T03:46:40.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.143+0000 7f05bb7fe640 1 -- 172.21.15.76:0/2602237457 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc004260 con 0x7f05c415daa0 2023-11-09T03:46:40.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.143+0000 7f05c8bbb640 1 -- 172.21.15.76:0/2602237457 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c4159e20 con 0x7f05c415daa0 2023-11-09T03:46:40.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.143+0000 7f05c8bbb640 1 -- 172.21.15.76:0/2602237457 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c415c9d0 con 0x7f05c415daa0 2023-11-09T03:46:40.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.144+0000 7f05c8bbb640 1 -- 172.21.15.76:0/2602237457 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05c4074290 con 0x7f05c415daa0 2023-11-09T03:46:40.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.144+0000 7f05bb7fe640 1 -- 172.21.15.76:0/2602237457 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f05bc0043c0 con 0x7f05c415daa0 2023-11-09T03:46:40.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.145+0000 7f05bb7fe640 1 --2- 172.21.15.76:0/2602237457 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f05a406ae70 0x7f05a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:40.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.145+0000 7f05bb7fe640 1 -- 172.21.15.76:0/2602237457 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f05bc012070 con 0x7f05c415daa0 2023-11-09T03:46:40.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.145+0000 7f05c259c640 1 --2- 172.21.15.76:0/2602237457 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f05a406ae70 0x7f05a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:40.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.146+0000 7f05c259c640 1 --2- 172.21.15.76:0/2602237457 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f05a406ae70 0x7f05a406d330 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f05b4036000 tx=0x7f05b403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:40.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.149+0000 7f05bb7fe640 1 -- 172.21.15.76:0/2602237457 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f05bc055010 con 0x7f05c415daa0 2023-11-09T03:46:40.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.353+0000 7f05c8bbb640 1 -- 172.21.15.76:0/2602237457 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05c415a1f0 con 0x7f05c415daa0 2023-11-09T03:46:40.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.353+0000 7f05bb7fe640 1 -- 172.21.15.76:0/2602237457 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4915 (secure 0 0 0) 0x7f05bc058630 con 0x7f05c415daa0 2023-11-09T03:46:40.354 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:40.354 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2686298435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2686298435},{"type":"v1","addr":"172.21.15.76:6833","nonce":2686298435}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2132717123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2132717123},{"type":"v1","addr":"172.21.15.76:6835","nonce":2132717123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4281,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2369499936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2369499936},{"type":"v1","addr":"172.21.15.113:6837","nonce":2369499936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:45:30.784155+0000","modified":"2023-11-09T03:45:47.804242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/3293223710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3293223710},{"type":"v1","addr":"172.21.15.113:6836","nonce":3293223710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-09T03:46:40.354 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 9 2023-11-09T03:46:40.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.355+0000 7f05c8bbb640 1 -- 172.21.15.76:0/2602237457 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f05a406ae70 msgr2=0x7f05a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:40.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.355+0000 7f05c8bbb640 1 --2- 172.21.15.76:0/2602237457 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f05a406ae70 0x7f05a406d330 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f05b4036000 tx=0x7f05b403c040 comp rx=0 tx=0).stop 2023-11-09T03:46:40.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.355+0000 7f05c8bbb640 1 -- 172.21.15.76:0/2602237457 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05c415daa0 msgr2=0x7f05c4148e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:40.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.355+0000 7f05c8bbb640 1 --2- 172.21.15.76:0/2602237457 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05c415daa0 0x7f05c4148e80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f05bc00cd20 tx=0x7f05bc00a660 comp rx=0 tx=0).stop 2023-11-09T03:46:40.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.355+0000 7f05c8bbb640 1 -- 172.21.15.76:0/2602237457 shutdown_connections 2023-11-09T03:46:40.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.355+0000 7f05c8bbb640 1 --2- 172.21.15.76:0/2602237457 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f05a406ae70 0x7f05a406d330 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.355+0000 7f05c8bbb640 1 --2- 172.21.15.76:0/2602237457 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05c415daa0 0x7f05c4148e80 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.355+0000 7f05c8bbb640 1 --2- 172.21.15.76:0/2602237457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f05c40ce240 0x7f05c41f2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.355+0000 7f05c8bbb640 1 --2- 172.21.15.76:0/2602237457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05c40cde60 0x7f05c414b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.355+0000 7f05c8bbb640 1 -- 172.21.15.76:0/2602237457 >> 172.21.15.76:0/2602237457 conn(0x7f05c4071f30 msgr2=0x7f05c40b7880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:40.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.355+0000 7f05c8bbb640 1 -- 172.21.15.76:0/2602237457 shutdown_connections 2023-11-09T03:46:40.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.355+0000 7f05c8bbb640 1 -- 172.21.15.76:0/2602237457 wait complete. 2023-11-09T03:46:40.366 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:46:40.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.465+0000 7f01affa9640 1 Processor -- start 2023-11-09T03:46:40.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.466+0000 7f01affa9640 1 -- start start 2023-11-09T03:46:40.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.466+0000 7f01affa9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f01a8157710 0x7f01a8167b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:40.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.466+0000 7f01affa9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01a815e000 con 0x7f01a80cde60 2023-11-09T03:46:40.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.466+0000 7f01affa9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01a815e2e0 con 0x7f01a80ce240 2023-11-09T03:46:40.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.466+0000 7f01affa9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a815e5c0 con 0x7f01a8157710 2023-11-09T03:46:40.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.467+0000 7f01add45640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f01a80cde60 0x7f01a815d560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40736/0 (socket says 172.21.15.76:40736) 2023-11-09T03:46:40.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.467+0000 7f01add45640 1 -- 172.21.15.76:0/1762047879 learned_addr learned my addr 172.21.15.76:0/1762047879 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:40.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.467+0000 7f01ae546640 1 --2- 172.21.15.76:0/1762047879 >> v2:172.21.15.113:3300/0 conn(0x7f01a8157710 0x7f01a8167b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:40.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.467+0000 7f01acd43640 1 -- 172.21.15.76:0/1762047879 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1360137402 0 0) 0x7f01a815e000 con 0x7f01a80cde60 2023-11-09T03:46:40.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.467+0000 7f01acd43640 1 -- 172.21.15.76:0/1762047879 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0194003520 con 0x7f01a80cde60 2023-11-09T03:46:40.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.467+0000 7f01acd43640 1 -- 172.21.15.76:0/1762047879 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3930534318 0 0) 0x7f0194003520 con 0x7f01a80cde60 2023-11-09T03:46:40.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.467+0000 7f01acd43640 1 -- 172.21.15.76:0/1762047879 >> v2:172.21.15.113:3300/0 conn(0x7f01a8157710 msgr2=0x7f01a8167b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:40.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.467+0000 7f01acd43640 1 --2- 172.21.15.76:0/1762047879 >> v2:172.21.15.113:3300/0 conn(0x7f01a8157710 0x7f01a8167b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.467+0000 7f01acd43640 1 -- 172.21.15.76:0/1762047879 >> v1:172.21.15.113:6789/0 conn(0x7f01a80ce240 legacy=0x7f01a8157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:40.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.467+0000 7f01acd43640 1 -- 172.21.15.76:0/1762047879 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a80d1120 con 0x7f01a80cde60 2023-11-09T03:46:40.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.467+0000 7f01acd43640 1 -- 172.21.15.76:0/1762047879 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f01a4001670 con 0x7f01a80cde60 2023-11-09T03:46:40.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.467+0000 7f01acd43640 1 -- 172.21.15.76:0/1762047879 >> v1:172.21.15.76:6789/0 conn(0x7f01a80cde60 legacy=0x7f01a815d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:40.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.467+0000 7f01acd43640 1 --2- 172.21.15.76:0/1762047879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0194003970 0x7f0194003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:40.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.467+0000 7f01acd43640 1 --2- 172.21.15.76:0/1762047879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01940047c0 0x7f0194004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:40.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.467+0000 7f01acd43640 1 --2- 172.21.15.76:0/1762047879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0194005180 0x7f019400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:40.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.467+0000 7f01acd43640 1 -- 172.21.15.76:0/1762047879 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a815e5c0 con 0x7f01940047c0 2023-11-09T03:46:40.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.467+0000 7f01acd43640 1 -- 172.21.15.76:0/1762047879 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019400c020 con 0x7f0194005180 2023-11-09T03:46:40.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.467+0000 7f01acd43640 1 -- 172.21.15.76:0/1762047879 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f019400c1e0 con 0x7f0194003970 2023-11-09T03:46:40.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.467+0000 7f01ad544640 1 --2- 172.21.15.76:0/1762047879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01940047c0 0x7f0194004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:40.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.467+0000 7f01ae546640 1 --2- 172.21.15.76:0/1762047879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0194005180 0x7f019400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:40.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.467+0000 7f01add45640 1 --2- 172.21.15.76:0/1762047879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0194003970 0x7f0194003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:40.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.468+0000 7f01ad544640 1 -- 172.21.15.76:0/1762047879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0194003970 msgr2=0x7f0194003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:40.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.468+0000 7f01ad544640 1 --2- 172.21.15.76:0/1762047879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0194003970 0x7f0194003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.468+0000 7f01ad544640 1 -- 172.21.15.76:0/1762047879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0194005180 msgr2=0x7f019400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:40.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.468+0000 7f01ad544640 1 --2- 172.21.15.76:0/1762047879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0194005180 0x7f019400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.468+0000 7f01ad544640 1 -- 172.21.15.76:0/1762047879 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0194003520 con 0x7f01940047c0 2023-11-09T03:46:40.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.468+0000 7f01ae546640 1 -- 172.21.15.76:0/1762047879 reap_dead start 2023-11-09T03:46:40.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.468+0000 7f01ad544640 1 --2- 172.21.15.76:0/1762047879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01940047c0 0x7f0194004c40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0198001750 tx=0x7f01940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70ec1fc578bc5fc server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:40.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.468+0000 7f01acd43640 1 -- 172.21.15.76:0/1762047879 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01980052d0 con 0x7f01940047c0 2023-11-09T03:46:40.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.468+0000 7f01acd43640 1 -- 172.21.15.76:0/1762047879 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019802e050 con 0x7f01940047c0 2023-11-09T03:46:40.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.468+0000 7f01acd43640 1 -- 172.21.15.76:0/1762047879 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0198003830 con 0x7f01940047c0 2023-11-09T03:46:40.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.469+0000 7f01affa9640 1 -- 172.21.15.76:0/1762047879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01940047c0 msgr2=0x7f0194004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:40.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.469+0000 7f01affa9640 1 --2- 172.21.15.76:0/1762047879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01940047c0 0x7f0194004c40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0198001750 tx=0x7f01940057f0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.469+0000 7f01affa9640 1 -- 172.21.15.76:0/1762047879 shutdown_connections 2023-11-09T03:46:40.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.469+0000 7f01affa9640 1 --2- 172.21.15.76:0/1762047879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01940047c0 0x7f0194004c40 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.469+0000 7f01affa9640 1 -- 172.21.15.76:0/1762047879 >> 172.21.15.76:0/1762047879 conn(0x7f01a8071f30 msgr2=0x7f01a80cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:40.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.469+0000 7f01affa9640 1 -- 172.21.15.76:0/1762047879 shutdown_connections 2023-11-09T03:46:40.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.469+0000 7f01affa9640 1 -- 172.21.15.76:0/1762047879 wait complete. 2023-11-09T03:46:40.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.470+0000 7f01affa9640 1 Processor -- start 2023-11-09T03:46:40.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.470+0000 7f01affa9640 1 -- start start 2023-11-09T03:46:40.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.471+0000 7f01affa9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01940047c0 0x7f01a81f1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:40.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.471+0000 7f01affa9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01a80ca900 0x7f01a80cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:40.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.471+0000 7f01affa9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01a80d2090 0x7f01a80d2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:40.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.471+0000 7f01affa9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a80d2a80 con 0x7f01a80d2090 2023-11-09T03:46:40.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.471+0000 7f01affa9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a80cad80 con 0x7f01940047c0 2023-11-09T03:46:40.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.471+0000 7f01affa9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a80cca40 con 0x7f01a80ca900 2023-11-09T03:46:40.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.471+0000 7f01ae546640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01a80d2090 0x7f01a80d2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:40.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.471+0000 7f01ae546640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01a80d2090 0x7f01a80d2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35254/0 (socket says 172.21.15.76:35254) 2023-11-09T03:46:40.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.471+0000 7f01ae546640 1 -- 172.21.15.76:0/2149180366 learned_addr learned my addr 172.21.15.76:0/2149180366 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:40.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.471+0000 7f01ad544640 1 --2- 172.21.15.76:0/2149180366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01a80ca900 0x7f01a80cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:40.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.471+0000 7f01add45640 1 --2- 172.21.15.76:0/2149180366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01940047c0 0x7f01a81f1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:40.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.471+0000 7f01ae546640 1 -- 172.21.15.76:0/2149180366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01a80ca900 msgr2=0x7f01a80cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:40.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.471+0000 7f01ae546640 1 --2- 172.21.15.76:0/2149180366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01a80ca900 0x7f01a80cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.471+0000 7f01ae546640 1 -- 172.21.15.76:0/2149180366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01940047c0 msgr2=0x7f01a81f1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:40.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.471+0000 7f01ae546640 1 --2- 172.21.15.76:0/2149180366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01940047c0 0x7f01a81f1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.471+0000 7f01ae546640 1 -- 172.21.15.76:0/2149180366 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a80ccd20 con 0x7f01a80d2090 2023-11-09T03:46:40.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.472+0000 7f01ae546640 1 --2- 172.21.15.76:0/2149180366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01a80d2090 0x7f01a80d2540 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f01a80cf5c0 tx=0x7f01a000f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:40.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.472+0000 7f019e7fc640 1 -- 172.21.15.76:0/2149180366 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a000e040 con 0x7f01a80d2090 2023-11-09T03:46:40.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.472+0000 7f019e7fc640 1 -- 172.21.15.76:0/2149180366 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a000f040 con 0x7f01a80d2090 2023-11-09T03:46:40.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.472+0000 7f019e7fc640 1 -- 172.21.15.76:0/2149180366 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a0015430 con 0x7f01a80d2090 2023-11-09T03:46:40.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.472+0000 7f01affa9640 1 -- 172.21.15.76:0/2149180366 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a80cd000 con 0x7f01a80d2090 2023-11-09T03:46:40.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.472+0000 7f01affa9640 1 -- 172.21.15.76:0/2149180366 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a80cc730 con 0x7f01a80d2090 2023-11-09T03:46:40.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.473+0000 7f01affa9640 1 -- 172.21.15.76:0/2149180366 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a80cd520 con 0x7f01a80d2090 2023-11-09T03:46:40.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.473+0000 7f019e7fc640 1 -- 172.21.15.76:0/2149180366 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f01a0015590 con 0x7f01a80d2090 2023-11-09T03:46:40.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.474+0000 7f019e7fc640 1 --2- 172.21.15.76:0/2149180366 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f018c06aea0 0x7f018c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:40.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.474+0000 7f019e7fc640 1 -- 172.21.15.76:0/2149180366 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f01a008a0b0 con 0x7f01a80d2090 2023-11-09T03:46:40.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.474+0000 7f01add45640 1 --2- 172.21.15.76:0/2149180366 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f018c06aea0 0x7f018c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:40.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.475+0000 7f01add45640 1 --2- 172.21.15.76:0/2149180366 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f018c06aea0 0x7f018c06d360 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f01a4007c40 tx=0x7f01a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:40.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.478+0000 7f019e7fc640 1 -- 172.21.15.76:0/2149180366 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f01a80cd520 con 0x7f01a80d2090 2023-11-09T03:46:40.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.677+0000 7f01affa9640 1 -- 172.21.15.76:0/2149180366 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01a80cd520 con 0x7f01a80d2090 2023-11-09T03:46:40.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.678+0000 7f019e7fc640 1 -- 172.21.15.76:0/2149180366 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4915 (secure 0 0 0) 0x7f01a80cd520 con 0x7f01a80d2090 2023-11-09T03:46:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:40.679 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2686298435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2686298435},{"type":"v1","addr":"172.21.15.76:6833","nonce":2686298435}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2132717123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2132717123},{"type":"v1","addr":"172.21.15.76:6835","nonce":2132717123}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4281,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2369499936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2369499936},{"type":"v1","addr":"172.21.15.113:6837","nonce":2369499936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:45:30.784155+0000","modified":"2023-11-09T03:45:47.804242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/3293223710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3293223710},{"type":"v1","addr":"172.21.15.113:6836","nonce":3293223710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-09T03:46:40.680 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 9 2023-11-09T03:46:40.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.680+0000 7f01affa9640 1 -- 172.21.15.76:0/2149180366 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f018c06aea0 msgr2=0x7f018c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:40.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.680+0000 7f01affa9640 1 --2- 172.21.15.76:0/2149180366 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f018c06aea0 0x7f018c06d360 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f01a4007c40 tx=0x7f01a4009040 comp rx=0 tx=0).stop 2023-11-09T03:46:40.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.680+0000 7f01affa9640 1 -- 172.21.15.76:0/2149180366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01a80d2090 msgr2=0x7f01a80d2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:40.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.680+0000 7f01affa9640 1 --2- 172.21.15.76:0/2149180366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01a80d2090 0x7f01a80d2540 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f01a80cf5c0 tx=0x7f01a000f1b0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.680+0000 7f01affa9640 1 -- 172.21.15.76:0/2149180366 shutdown_connections 2023-11-09T03:46:40.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.680+0000 7f01affa9640 1 --2- 172.21.15.76:0/2149180366 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f018c06aea0 0x7f018c06d360 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.680+0000 7f01affa9640 1 --2- 172.21.15.76:0/2149180366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01a80d2090 0x7f01a80d2540 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.680+0000 7f01affa9640 1 --2- 172.21.15.76:0/2149180366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01a80ca900 0x7f01a80cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.680+0000 7f01affa9640 1 --2- 172.21.15.76:0/2149180366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01940047c0 0x7f01a81f1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.680+0000 7f01affa9640 1 -- 172.21.15.76:0/2149180366 >> 172.21.15.76:0/2149180366 conn(0x7f01a8071f30 msgr2=0x7f01a80b7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:40.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.680+0000 7f01affa9640 1 -- 172.21.15.76:0/2149180366 shutdown_connections 2023-11-09T03:46:40.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.680+0000 7f01affa9640 1 -- 172.21.15.76:0/2149180366 wait complete. 2023-11-09T03:46:40.691 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:46:40.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.791+0000 7f3dc7ff5640 1 Processor -- start 2023-11-09T03:46:40.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.792+0000 7f3dc7ff5640 1 -- start start 2023-11-09T03:46:40.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.792+0000 7f3dc7ff5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3dc00ce240 0x7f3dc0157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:40.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.792+0000 7f3dc7ff5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dc015e000 con 0x7f3dc00cde60 2023-11-09T03:46:40.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.792+0000 7f3dc7ff5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dc015e2e0 con 0x7f3dc0157540 2023-11-09T03:46:40.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.792+0000 7f3dc7ff5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc015e5c0 con 0x7f3dc00ce240 2023-11-09T03:46:40.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.793+0000 7f3dc5590640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3dc00ce240 0x7f3dc0157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:40.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.793+0000 7f3dc5d91640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f3dc00cde60 0x7f3dc015d560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40740/0 (socket says 172.21.15.76:40740) 2023-11-09T03:46:40.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.793+0000 7f3dc5d91640 1 -- 172.21.15.76:0/2243147354 learned_addr learned my addr 172.21.15.76:0/2243147354 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:40.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.793+0000 7f3dc5590640 1 -- 172.21.15.76:0/2243147354 >> v1:172.21.15.113:6789/0 conn(0x7f3dc0157540 legacy=0x7f3dc00d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:40.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.793+0000 7f3dc5590640 1 -- 172.21.15.76:0/2243147354 >> v1:172.21.15.76:6789/0 conn(0x7f3dc00cde60 legacy=0x7f3dc015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:40.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.793+0000 7f3dc4d8f640 1 -- 172.21.15.76:0/2243147354 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4019705307 0 0) 0x7f3dc015e000 con 0x7f3dc00cde60 2023-11-09T03:46:40.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.793+0000 7f3dc5590640 1 -- 172.21.15.76:0/2243147354 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc00d1120 con 0x7f3dc00ce240 2023-11-09T03:46:40.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.794+0000 7f3dc5590640 1 --2- 172.21.15.76:0/2243147354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3dc00ce240 0x7f3dc0157000 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3db00097a0 tx=0x7f3db002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3252cd9aecd12e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:40.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.794+0000 7f3dc4d8f640 1 -- 172.21.15.76:0/2243147354 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db0030610 con 0x7f3dc00ce240 2023-11-09T03:46:40.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.794+0000 7f3dc4d8f640 1 -- 172.21.15.76:0/2243147354 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db0030770 con 0x7f3dc00ce240 2023-11-09T03:46:40.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.794+0000 7f3dc4d8f640 1 -- 172.21.15.76:0/2243147354 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db0030b00 con 0x7f3dc00ce240 2023-11-09T03:46:40.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.794+0000 7f3dc7ff5640 1 -- 172.21.15.76:0/2243147354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3dc00ce240 msgr2=0x7f3dc0157000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:40.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.794+0000 7f3dc7ff5640 1 --2- 172.21.15.76:0/2243147354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3dc00ce240 0x7f3dc0157000 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3db00097a0 tx=0x7f3db002b960 comp rx=0 tx=0).stop 2023-11-09T03:46:40.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.794+0000 7f3dc7ff5640 1 -- 172.21.15.76:0/2243147354 shutdown_connections 2023-11-09T03:46:40.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.794+0000 7f3dc7ff5640 1 --2- 172.21.15.76:0/2243147354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3dc00ce240 0x7f3dc0157000 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.794+0000 7f3dc7ff5640 1 -- 172.21.15.76:0/2243147354 >> 172.21.15.76:0/2243147354 conn(0x7f3dc0071f30 msgr2=0x7f3dc00cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:40.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.794+0000 7f3dc7ff5640 1 -- 172.21.15.76:0/2243147354 shutdown_connections 2023-11-09T03:46:40.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.794+0000 7f3dc7ff5640 1 -- 172.21.15.76:0/2243147354 wait complete. 2023-11-09T03:46:40.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.796+0000 7f3dc7ff5640 1 Processor -- start 2023-11-09T03:46:40.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.796+0000 7f3dc7ff5640 1 -- start start 2023-11-09T03:46:40.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.796+0000 7f3dc7ff5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dc00cde60 0x7f3dc00cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:40.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.796+0000 7f3dc7ff5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3dc00ce240 0x7f3dc014ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:40.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.796+0000 7f3dc7ff5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3dc0157540 0x7f3dc014f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:40.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.796+0000 7f3dc7ff5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc01f71a0 con 0x7f3dc00ce240 2023-11-09T03:46:40.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.796+0000 7f3dc7ff5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc0159970 con 0x7f3dc00cde60 2023-11-09T03:46:40.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.796+0000 7f3dc7ff5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc0159c50 con 0x7f3dc0157540 2023-11-09T03:46:40.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.796+0000 7f3dc5590640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3dc00ce240 0x7f3dc014ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:40.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.796+0000 7f3dc5590640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3dc00ce240 0x7f3dc014ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35272/0 (socket says 172.21.15.76:35272) 2023-11-09T03:46:40.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.796+0000 7f3dc5590640 1 -- 172.21.15.76:0/3655862785 learned_addr learned my addr 172.21.15.76:0/3655862785 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:40.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.797+0000 7f3dc6592640 1 --2- 172.21.15.76:0/3655862785 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3dc0157540 0x7f3dc014f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:40.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.797+0000 7f3dc5d91640 1 --2- 172.21.15.76:0/3655862785 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dc00cde60 0x7f3dc00cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:40.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.797+0000 7f3dc5590640 1 -- 172.21.15.76:0/3655862785 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3dc0157540 msgr2=0x7f3dc014f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:40.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.797+0000 7f3dc5590640 1 --2- 172.21.15.76:0/3655862785 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3dc0157540 0x7f3dc014f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.797+0000 7f3dc5590640 1 -- 172.21.15.76:0/3655862785 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dc00cde60 msgr2=0x7f3dc00cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:40.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.797+0000 7f3dc5590640 1 --2- 172.21.15.76:0/3655862785 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dc00cde60 0x7f3dc00cbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.797+0000 7f3dc5590640 1 -- 172.21.15.76:0/3655862785 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db0009400 con 0x7f3dc00ce240 2023-11-09T03:46:40.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.797+0000 7f3dc5d91640 1 --2- 172.21.15.76:0/3655862785 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dc00cde60 0x7f3dc00cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:40.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.797+0000 7f3dc5590640 1 --2- 172.21.15.76:0/3655862785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3dc00ce240 0x7f3dc014ee50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3db00098c0 tx=0x7f3db0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:40.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.797+0000 7f3db67fc640 1 -- 172.21.15.76:0/3655862785 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db003e040 con 0x7f3dc00ce240 2023-11-09T03:46:40.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.797+0000 7f3db67fc640 1 -- 172.21.15.76:0/3655862785 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db0039cb0 con 0x7f3dc00ce240 2023-11-09T03:46:40.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.797+0000 7f3dc7ff5640 1 -- 172.21.15.76:0/3655862785 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc0159f30 con 0x7f3dc00ce240 2023-11-09T03:46:40.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.797+0000 7f3db67fc640 1 -- 172.21.15.76:0/3655862785 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db0044630 con 0x7f3dc00ce240 2023-11-09T03:46:40.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.797+0000 7f3dc7ff5640 1 -- 172.21.15.76:0/3655862785 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc01f2d90 con 0x7f3dc00ce240 2023-11-09T03:46:40.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.798+0000 7f3dc7ff5640 1 -- 172.21.15.76:0/3655862785 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc00d0950 con 0x7f3dc00ce240 2023-11-09T03:46:40.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.798+0000 7f3db67fc640 1 -- 172.21.15.76:0/3655862785 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3db004d3b0 con 0x7f3dc00ce240 2023-11-09T03:46:40.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.799+0000 7f3db67fc640 1 --2- 172.21.15.76:0/3655862785 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3da406ae70 0x7f3da406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:40.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.799+0000 7f3db67fc640 1 -- 172.21.15.76:0/3655862785 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f3db00abf40 con 0x7f3dc00ce240 2023-11-09T03:46:40.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.799+0000 7f3dc5d91640 1 --2- 172.21.15.76:0/3655862785 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3da406ae70 0x7f3da406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:40.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.800+0000 7f3dc5d91640 1 --2- 172.21.15.76:0/3655862785 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3da406ae70 0x7f3da406d330 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3db8007fd0 tx=0x7f3db8005b70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:40.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.803+0000 7f3db67fc640 1 -- 172.21.15.76:0/3655862785 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3db0076cc0 con 0x7f3dc00ce240 2023-11-09T03:46:40.971 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.971+0000 7f3dc7ff5640 1 -- 172.21.15.76:0/3655862785 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3dc0074290 con 0x7f3dc00ce240 2023-11-09T03:46:40.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.972+0000 7f3db67fc640 1 -- 172.21.15.76:0/3655862785 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18710 (secure 0 0 0) 0x7f3db007a2e0 con 0x7f3dc00ce240 2023-11-09T03:46:40.973 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:40.973 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":27,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:46:12.537265+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:45:28.704715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-09T03:45:29.717614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-09T03:45:33.440577+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:46:40.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.974+0000 7f3dc7ff5640 1 -- 172.21.15.76:0/3655862785 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3da406ae70 msgr2=0x7f3da406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:40.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.974+0000 7f3dc7ff5640 1 --2- 172.21.15.76:0/3655862785 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3da406ae70 0x7f3da406d330 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3db8007fd0 tx=0x7f3db8005b70 comp rx=0 tx=0).stop 2023-11-09T03:46:40.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.974+0000 7f3dc7ff5640 1 -- 172.21.15.76:0/3655862785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3dc00ce240 msgr2=0x7f3dc014ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:40.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.974+0000 7f3dc7ff5640 1 --2- 172.21.15.76:0/3655862785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3dc00ce240 0x7f3dc014ee50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3db00098c0 tx=0x7f3db0004730 comp rx=0 tx=0).stop 2023-11-09T03:46:40.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.974+0000 7f3dc7ff5640 1 -- 172.21.15.76:0/3655862785 shutdown_connections 2023-11-09T03:46:40.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.974+0000 7f3dc7ff5640 1 --2- 172.21.15.76:0/3655862785 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3da406ae70 0x7f3da406d330 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.974+0000 7f3dc7ff5640 1 --2- 172.21.15.76:0/3655862785 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3dc0157540 0x7f3dc014f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.974+0000 7f3dc7ff5640 1 --2- 172.21.15.76:0/3655862785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3dc00ce240 0x7f3dc014ee50 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.974+0000 7f3dc7ff5640 1 --2- 172.21.15.76:0/3655862785 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dc00cde60 0x7f3dc00cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:40.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.974+0000 7f3dc7ff5640 1 -- 172.21.15.76:0/3655862785 >> 172.21.15.76:0/3655862785 conn(0x7f3dc0071f30 msgr2=0x7f3dc00d22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:40.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.975+0000 7f3dc7ff5640 1 -- 172.21.15.76:0/3655862785 shutdown_connections 2023-11-09T03:46:40.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:40.975+0000 7f3dc7ff5640 1 -- 172.21.15.76:0/3655862785 wait complete. 2023-11-09T03:46:40.986 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-09T03:46:41.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.087+0000 7efc585a5640 1 Processor -- start 2023-11-09T03:46:41.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.087+0000 7efc585a5640 1 -- start start 2023-11-09T03:46:41.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.088+0000 7efc585a5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efc500c4ff0 0x7efc500cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:41.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.088+0000 7efc585a5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc5015aff0 con 0x7efc500adac0 2023-11-09T03:46:41.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.088+0000 7efc585a5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc5015a3e0 con 0x7efc50159c00 2023-11-09T03:46:41.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.088+0000 7efc585a5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc5015a6c0 con 0x7efc500c4ff0 2023-11-09T03:46:41.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.088+0000 7efc56b42640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7efc500adac0 0x7efc500adf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40748/0 (socket says 172.21.15.76:40748) 2023-11-09T03:46:41.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.088+0000 7efc56b42640 1 -- 172.21.15.76:0/3836619894 learned_addr learned my addr 172.21.15.76:0/3836619894 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:41.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.088+0000 7efc55b40640 1 --2- 172.21.15.76:0/3836619894 >> v2:172.21.15.113:3300/0 conn(0x7efc500c4ff0 0x7efc500cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:41.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.088+0000 7efc5533f640 1 -- 172.21.15.76:0/3836619894 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1375455982 0 0) 0x7efc5015aff0 con 0x7efc500adac0 2023-11-09T03:46:41.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.088+0000 7efc5533f640 1 -- 172.21.15.76:0/3836619894 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc3c003520 con 0x7efc500adac0 2023-11-09T03:46:41.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.088+0000 7efc5533f640 1 -- 172.21.15.76:0/3836619894 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 827275072 0 0) 0x7efc3c003520 con 0x7efc500adac0 2023-11-09T03:46:41.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.088+0000 7efc5533f640 1 -- 172.21.15.76:0/3836619894 >> v2:172.21.15.113:3300/0 conn(0x7efc500c4ff0 msgr2=0x7efc500cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:41.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.088+0000 7efc5533f640 1 --2- 172.21.15.76:0/3836619894 >> v2:172.21.15.113:3300/0 conn(0x7efc500c4ff0 0x7efc500cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:41.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.088+0000 7efc5533f640 1 -- 172.21.15.76:0/3836619894 >> v1:172.21.15.113:6789/0 conn(0x7efc50159c00 legacy=0x7efc500c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:41.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.088+0000 7efc5533f640 1 -- 172.21.15.76:0/3836619894 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5015a9a0 con 0x7efc500adac0 2023-11-09T03:46:41.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.088+0000 7efc5533f640 1 -- 172.21.15.76:0/3836619894 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7efc4c0019f0 con 0x7efc500adac0 2023-11-09T03:46:41.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.088+0000 7efc5533f640 1 -- 172.21.15.76:0/3836619894 >> v1:172.21.15.76:6789/0 conn(0x7efc500adac0 legacy=0x7efc500adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:41.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.088+0000 7efc5533f640 1 --2- 172.21.15.76:0/3836619894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc3c003970 0x7efc3c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:41.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.088+0000 7efc5533f640 1 --2- 172.21.15.76:0/3836619894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc3c0047c0 0x7efc3c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:41.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.088+0000 7efc5533f640 1 --2- 172.21.15.76:0/3836619894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc3c005180 0x7efc3c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:41.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.088+0000 7efc5533f640 1 -- 172.21.15.76:0/3836619894 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5015a6c0 con 0x7efc3c003970 2023-11-09T03:46:41.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.088+0000 7efc5533f640 1 -- 172.21.15.76:0/3836619894 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c00c020 con 0x7efc3c0047c0 2023-11-09T03:46:41.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.088+0000 7efc5533f640 1 -- 172.21.15.76:0/3836619894 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c00c1e0 con 0x7efc3c005180 2023-11-09T03:46:41.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.089+0000 7efc56341640 1 --2- 172.21.15.76:0/3836619894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc3c003970 0x7efc3c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:41.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.089+0000 7efc56b42640 1 --2- 172.21.15.76:0/3836619894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc3c005180 0x7efc3c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:41.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.089+0000 7efc55b40640 1 --2- 172.21.15.76:0/3836619894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc3c0047c0 0x7efc3c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:41.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.089+0000 7efc56341640 1 -- 172.21.15.76:0/3836619894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc3c005180 msgr2=0x7efc3c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:41.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.089+0000 7efc56341640 1 --2- 172.21.15.76:0/3836619894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc3c005180 0x7efc3c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:41.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.089+0000 7efc56341640 1 -- 172.21.15.76:0/3836619894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc3c0047c0 msgr2=0x7efc3c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:41.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.089+0000 7efc56341640 1 --2- 172.21.15.76:0/3836619894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc3c0047c0 0x7efc3c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:41.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.089+0000 7efc56341640 1 -- 172.21.15.76:0/3836619894 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3c003520 con 0x7efc3c003970 2023-11-09T03:46:41.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.089+0000 7efc56b42640 1 -- 172.21.15.76:0/3836619894 reap_dead start 2023-11-09T03:46:41.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.089+0000 7efc56341640 1 --2- 172.21.15.76:0/3836619894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc3c003970 0x7efc3c003d90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7efc48001750 tx=0x7efc3c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45139514fcf42486 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:41.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.090+0000 7efc5533f640 1 -- 172.21.15.76:0/3836619894 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc480052d0 con 0x7efc3c003970 2023-11-09T03:46:41.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.090+0000 7efc5533f640 1 -- 172.21.15.76:0/3836619894 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4802e050 con 0x7efc3c003970 2023-11-09T03:46:41.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.090+0000 7efc5533f640 1 -- 172.21.15.76:0/3836619894 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc48003830 con 0x7efc3c003970 2023-11-09T03:46:41.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.090+0000 7efc585a5640 1 -- 172.21.15.76:0/3836619894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc3c003970 msgr2=0x7efc3c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:41.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.090+0000 7efc585a5640 1 --2- 172.21.15.76:0/3836619894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc3c003970 0x7efc3c003d90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7efc48001750 tx=0x7efc3c0057f0 comp rx=0 tx=0).stop 2023-11-09T03:46:41.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.090+0000 7efc585a5640 1 -- 172.21.15.76:0/3836619894 shutdown_connections 2023-11-09T03:46:41.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.090+0000 7efc585a5640 1 --2- 172.21.15.76:0/3836619894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc3c003970 0x7efc3c003d90 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:41.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.090+0000 7efc585a5640 1 -- 172.21.15.76:0/3836619894 >> 172.21.15.76:0/3836619894 conn(0x7efc500ce950 msgr2=0x7efc500ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:41.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.090+0000 7efc585a5640 1 -- 172.21.15.76:0/3836619894 shutdown_connections 2023-11-09T03:46:41.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.090+0000 7efc585a5640 1 -- 172.21.15.76:0/3836619894 wait complete. 2023-11-09T03:46:41.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.091+0000 7efc585a5640 1 Processor -- start 2023-11-09T03:46:41.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.091+0000 7efc585a5640 1 -- start start 2023-11-09T03:46:41.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.092+0000 7efc585a5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc3c003970 0x7efc50152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:41.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.092+0000 7efc585a5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc50151cc0 0x7efc50153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:41.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.092+0000 7efc585a5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc50153ea0 0x7efc50154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:41.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.092+0000 7efc585a5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5013cda0 con 0x7efc50151cc0 2023-11-09T03:46:41.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.092+0000 7efc585a5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc50153710 con 0x7efc50153ea0 2023-11-09T03:46:41.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.092+0000 7efc585a5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc50153430 con 0x7efc3c003970 2023-11-09T03:46:41.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.092+0000 7efc55b40640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc50151cc0 0x7efc50153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:41.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.092+0000 7efc55b40640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc50151cc0 0x7efc50153960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35282/0 (socket says 172.21.15.76:35282) 2023-11-09T03:46:41.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.092+0000 7efc55b40640 1 -- 172.21.15.76:0/1743036020 learned_addr learned my addr 172.21.15.76:0/1743036020 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:41.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.092+0000 7efc56b42640 1 --2- 172.21.15.76:0/1743036020 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc50153ea0 0x7efc50154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:41.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.092+0000 7efc56341640 1 --2- 172.21.15.76:0/1743036020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc3c003970 0x7efc50152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:41.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.092+0000 7efc55b40640 1 -- 172.21.15.76:0/1743036020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc3c003970 msgr2=0x7efc50152bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:41.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.092+0000 7efc55b40640 1 --2- 172.21.15.76:0/1743036020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc3c003970 0x7efc50152bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:41.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.092+0000 7efc55b40640 1 -- 172.21.15.76:0/1743036020 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc50153ea0 msgr2=0x7efc50154320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:41.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.092+0000 7efc55b40640 1 --2- 172.21.15.76:0/1743036020 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc50153ea0 0x7efc50154320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:41.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.093+0000 7efc55b40640 1 -- 172.21.15.76:0/1743036020 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5014b260 con 0x7efc50151cc0 2023-11-09T03:46:41.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.093+0000 7efc56341640 1 --2- 172.21.15.76:0/1743036020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc3c003970 0x7efc50152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:41.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.093+0000 7efc55b40640 1 --2- 172.21.15.76:0/1743036020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc50151cc0 0x7efc50153960 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7efc40011590 tx=0x7efc40011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:41.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.093+0000 7efc46ffd640 1 -- 172.21.15.76:0/1743036020 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc40004440 con 0x7efc50151cc0 2023-11-09T03:46:41.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.093+0000 7efc46ffd640 1 -- 172.21.15.76:0/1743036020 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc40003610 con 0x7efc50151cc0 2023-11-09T03:46:41.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.093+0000 7efc46ffd640 1 -- 172.21.15.76:0/1743036020 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc400039a0 con 0x7efc50151cc0 2023-11-09T03:46:41.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.093+0000 7efc585a5640 1 -- 172.21.15.76:0/1743036020 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc5014b540 con 0x7efc50151cc0 2023-11-09T03:46:41.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.093+0000 7efc585a5640 1 -- 172.21.15.76:0/1743036020 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc501530f0 con 0x7efc50151cc0 2023-11-09T03:46:41.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.094+0000 7efc585a5640 1 -- 172.21.15.76:0/1743036020 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc50074290 con 0x7efc50151cc0 2023-11-09T03:46:41.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.094+0000 7efc46ffd640 1 -- 172.21.15.76:0/1743036020 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efc40003b00 con 0x7efc50151cc0 2023-11-09T03:46:41.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.095+0000 7efc46ffd640 1 --2- 172.21.15.76:0/1743036020 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efc3406aea0 0x7efc3406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:41.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.095+0000 7efc46ffd640 1 -- 172.21.15.76:0/1743036020 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7efc40089ab0 con 0x7efc50151cc0 2023-11-09T03:46:41.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.095+0000 7efc56341640 1 --2- 172.21.15.76:0/1743036020 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efc3406aea0 0x7efc3406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:41.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.095+0000 7efc56341640 1 --2- 172.21.15.76:0/1743036020 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efc3406aea0 0x7efc3406d360 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efc48033000 tx=0x7efc48009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:41.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.099+0000 7efc46ffd640 1 -- 172.21.15.76:0/1743036020 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efc400548a0 con 0x7efc50151cc0 2023-11-09T03:46:41.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.300+0000 7efc585a5640 1 -- 172.21.15.76:0/1743036020 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7efc5006b760 con 0x7efc50151cc0 2023-11-09T03:46:41.301 INFO:tasks.ceph.mon.a.smithi076.stderr:2023-11-09T03:46:41.301+0000 7fe4a393d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-09T03:46:41.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.302+0000 7efc46ffd640 1 -- 172.21.15.76:0/1743036020 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7efc40057ec0 con 0x7efc50151cc0 2023-11-09T03:46:41.303 INFO:teuthology.orchestra.run.smithi076.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-09T03:46:41.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.304+0000 7efc585a5640 1 -- 172.21.15.76:0/1743036020 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efc3406aea0 msgr2=0x7efc3406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:41.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.304+0000 7efc585a5640 1 --2- 172.21.15.76:0/1743036020 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efc3406aea0 0x7efc3406d360 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efc48033000 tx=0x7efc48009030 comp rx=0 tx=0).stop 2023-11-09T03:46:41.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.304+0000 7efc585a5640 1 -- 172.21.15.76:0/1743036020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc50151cc0 msgr2=0x7efc50153960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:41.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.304+0000 7efc585a5640 1 --2- 172.21.15.76:0/1743036020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc50151cc0 0x7efc50153960 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7efc40011590 tx=0x7efc40011a50 comp rx=0 tx=0).stop 2023-11-09T03:46:41.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.304+0000 7efc585a5640 1 -- 172.21.15.76:0/1743036020 shutdown_connections 2023-11-09T03:46:41.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.304+0000 7efc585a5640 1 --2- 172.21.15.76:0/1743036020 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efc3406aea0 0x7efc3406d360 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:41.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.304+0000 7efc585a5640 1 --2- 172.21.15.76:0/1743036020 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc50153ea0 0x7efc50154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:41.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.304+0000 7efc585a5640 1 --2- 172.21.15.76:0/1743036020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc50151cc0 0x7efc50153960 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:41.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.304+0000 7efc585a5640 1 --2- 172.21.15.76:0/1743036020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc3c003970 0x7efc50152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:41.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.304+0000 7efc585a5640 1 -- 172.21.15.76:0/1743036020 >> 172.21.15.76:0/1743036020 conn(0x7efc500ce950 msgr2=0x7efc5006d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:41.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.304+0000 7efc585a5640 1 -- 172.21.15.76:0/1743036020 shutdown_connections 2023-11-09T03:46:41.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.304+0000 7efc585a5640 1 -- 172.21.15.76:0/1743036020 wait complete. 2023-11-09T03:46:41.308 INFO:tasks.ceph.mds.b.smithi113.stderr: -4578> 2023-11-09T03:45:35.367+0000 7f7e01729640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:46:41.316 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-09T03:46:41.339 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2023-11-09T03:46:41.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.419+0000 7fea764da640 1 Processor -- start 2023-11-09T03:46:41.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.419+0000 7fea764da640 1 -- start start 2023-11-09T03:46:41.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.420+0000 7fea764da640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fea70153e60 0x7fea7015dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:41.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.420+0000 7fea764da640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea700b4880 0x7fea700b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:41.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.420+0000 7fea764da640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea700cde60 con 0x7fea700cd6d0 2023-11-09T03:46:41.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.420+0000 7fea764da640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea70160f90 con 0x7fea70153e60 2023-11-09T03:46:41.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.420+0000 7fea764da640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea700b5200 con 0x7fea700b4880 2023-11-09T03:46:41.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.420+0000 7fea6f7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fea700cd6d0 0x7fea70161420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40754/0 (socket says 172.21.15.76:40754) 2023-11-09T03:46:41.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.420+0000 7fea6f7fe640 1 -- 172.21.15.76:0/2300926132 learned_addr learned my addr 172.21.15.76:0/2300926132 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:41.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.420+0000 7fea6ffff640 1 --2- 172.21.15.76:0/2300926132 >> v2:172.21.15.113:3300/0 conn(0x7fea70153e60 0x7fea7015dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:41.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.420+0000 7fea74a77640 1 --2- 172.21.15.76:0/2300926132 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea700b4880 0x7fea700b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:41.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.420+0000 7fea6effd640 1 -- 172.21.15.76:0/2300926132 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3796985181 0 0) 0x7fea700cde60 con 0x7fea700cd6d0 2023-11-09T03:46:41.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.420+0000 7fea6effd640 1 -- 172.21.15.76:0/2300926132 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea5c003520 con 0x7fea700cd6d0 2023-11-09T03:46:41.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.420+0000 7fea74a77640 1 -- 172.21.15.76:0/2300926132 >> v2:172.21.15.113:3300/0 conn(0x7fea70153e60 msgr2=0x7fea7015dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:41.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.420+0000 7fea74a77640 1 --2- 172.21.15.76:0/2300926132 >> v2:172.21.15.113:3300/0 conn(0x7fea70153e60 0x7fea7015dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:41.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.420+0000 7fea74a77640 1 -- 172.21.15.76:0/2300926132 >> v1:172.21.15.76:6789/0 conn(0x7fea700cd6d0 legacy=0x7fea70161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:41.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.420+0000 7fea74a77640 1 -- 172.21.15.76:0/2300926132 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7015e400 con 0x7fea700b4880 2023-11-09T03:46:41.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.420+0000 7fea6ffff640 1 --2- 172.21.15.76:0/2300926132 >> v2:172.21.15.113:3300/0 conn(0x7fea70153e60 0x7fea7015dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:46:41.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.421+0000 7fea74a77640 1 --2- 172.21.15.76:0/2300926132 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea700b4880 0x7fea700b4cc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fea68009d90 tx=0x7fea6802bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdba8f4e4f466e97 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:41.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.421+0000 7fea6effd640 1 -- 172.21.15.76:0/2300926132 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6802cc50 con 0x7fea700b4880 2023-11-09T03:46:41.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.421+0000 7fea6effd640 1 -- 172.21.15.76:0/2300926132 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6802cdb0 con 0x7fea700b4880 2023-11-09T03:46:41.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.421+0000 7fea6effd640 1 -- 172.21.15.76:0/2300926132 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea68034660 con 0x7fea700b4880 2023-11-09T03:46:41.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.422+0000 7fea764da640 1 -- 172.21.15.76:0/2300926132 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea700b4880 msgr2=0x7fea700b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:41.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.422+0000 7fea764da640 1 --2- 172.21.15.76:0/2300926132 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea700b4880 0x7fea700b4cc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fea68009d90 tx=0x7fea6802bea0 comp rx=0 tx=0).stop 2023-11-09T03:46:41.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.422+0000 7fea764da640 1 -- 172.21.15.76:0/2300926132 shutdown_connections 2023-11-09T03:46:41.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.422+0000 7fea764da640 1 --2- 172.21.15.76:0/2300926132 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea700b4880 0x7fea700b4cc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:41.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.422+0000 7fea764da640 1 --2- 172.21.15.76:0/2300926132 >> v2:172.21.15.113:3300/0 conn(0x7fea70153e60 0x7fea7015dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:41.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.422+0000 7fea764da640 1 -- 172.21.15.76:0/2300926132 >> 172.21.15.76:0/2300926132 conn(0x7fea700cf880 msgr2=0x7fea700cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:41.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.422+0000 7fea764da640 1 -- 172.21.15.76:0/2300926132 shutdown_connections 2023-11-09T03:46:41.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.422+0000 7fea764da640 1 -- 172.21.15.76:0/2300926132 wait complete. 2023-11-09T03:46:41.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.423+0000 7fea764da640 1 Processor -- start 2023-11-09T03:46:41.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.423+0000 7fea764da640 1 -- start start 2023-11-09T03:46:41.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.424+0000 7fea764da640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea700b4880 0x7fea7015a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:41.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.424+0000 7fea764da640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea700cd6d0 0x7fea700c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:41.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.424+0000 7fea764da640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea70153e60 0x7fea700cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:41.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.424+0000 7fea764da640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea700c8f60 con 0x7fea700b4880 2023-11-09T03:46:41.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.424+0000 7fea764da640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea70154b50 con 0x7fea700cd6d0 2023-11-09T03:46:41.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.424+0000 7fea764da640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea70152db0 con 0x7fea70153e60 2023-11-09T03:46:41.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.424+0000 7fea6ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea700b4880 0x7fea7015a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:41.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.424+0000 7fea6ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea700b4880 0x7fea7015a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35290/0 (socket says 172.21.15.76:35290) 2023-11-09T03:46:41.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.424+0000 7fea6ffff640 1 -- 172.21.15.76:0/2110443255 learned_addr learned my addr 172.21.15.76:0/2110443255 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:41.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.424+0000 7fea74a77640 1 --2- 172.21.15.76:0/2110443255 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea70153e60 0x7fea700cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:41.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.424+0000 7fea6f7fe640 1 --2- 172.21.15.76:0/2110443255 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea700cd6d0 0x7fea700c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:41.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.424+0000 7fea6ffff640 1 -- 172.21.15.76:0/2110443255 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea70153e60 msgr2=0x7fea700cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:41.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.424+0000 7fea6ffff640 1 --2- 172.21.15.76:0/2110443255 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea70153e60 0x7fea700cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:41.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.424+0000 7fea6ffff640 1 -- 172.21.15.76:0/2110443255 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea700cd6d0 msgr2=0x7fea700c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:41.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.424+0000 7fea6ffff640 1 --2- 172.21.15.76:0/2110443255 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea700cd6d0 0x7fea700c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:41.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.424+0000 7fea6ffff640 1 -- 172.21.15.76:0/2110443255 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea58009400 con 0x7fea700b4880 2023-11-09T03:46:41.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.424+0000 7fea74a77640 1 --2- 172.21.15.76:0/2110443255 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea70153e60 0x7fea700cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:41.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.424+0000 7fea6ffff640 1 --2- 172.21.15.76:0/2110443255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea700b4880 0x7fea7015a1a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fea5800e990 tx=0x7fea5800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:41.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.425+0000 7fea6cff9640 1 -- 172.21.15.76:0/2110443255 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5800c9e0 con 0x7fea700b4880 2023-11-09T03:46:41.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.425+0000 7fea6cff9640 1 -- 172.21.15.76:0/2110443255 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5800cb40 con 0x7fea700b4880 2023-11-09T03:46:41.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.425+0000 7fea764da640 1 -- 172.21.15.76:0/2110443255 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea68009a70 con 0x7fea700b4880 2023-11-09T03:46:41.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.425+0000 7fea6cff9640 1 -- 172.21.15.76:0/2110443255 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5800ced0 con 0x7fea700b4880 2023-11-09T03:46:41.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.425+0000 7fea764da640 1 -- 172.21.15.76:0/2110443255 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea7015a8a0 con 0x7fea700b4880 2023-11-09T03:46:41.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.425+0000 7fea764da640 1 -- 172.21.15.76:0/2110443255 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea70074290 con 0x7fea700b4880 2023-11-09T03:46:41.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.426+0000 7fea6cff9640 1 -- 172.21.15.76:0/2110443255 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fea58013470 con 0x7fea700b4880 2023-11-09T03:46:41.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.426+0000 7fea6cff9640 1 --2- 172.21.15.76:0/2110443255 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fea5006ae70 0x7fea5006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:41.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.426+0000 7fea6cff9640 1 -- 172.21.15.76:0/2110443255 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6778+0+0 (secure 0 0 0) 0x7fea58089e40 con 0x7fea700b4880 2023-11-09T03:46:41.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.426+0000 7fea6f7fe640 1 --2- 172.21.15.76:0/2110443255 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fea5006ae70 0x7fea5006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:41.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.427+0000 7fea6f7fe640 1 --2- 172.21.15.76:0/2110443255 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fea5006ae70 0x7fea5006d330 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fea701f6110 tx=0x7fea60009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:41.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.431+0000 7fea6cff9640 1 -- 172.21.15.76:0/2110443255 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fea580549b0 con 0x7fea700b4880 2023-11-09T03:46:41.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:41.633+0000 7fea764da640 1 -- 172.21.15.76:0/2110443255 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fea7015e1e0 con 0x7fea700b4880 2023-11-09T03:46:42.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.304+0000 7fea6cff9640 1 -- 172.21.15.76:0/2110443255 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fea58057fd0 con 0x7fea700b4880 2023-11-09T03:46:42.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.306+0000 7fea764da640 1 -- 172.21.15.76:0/2110443255 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fea5006ae70 msgr2=0x7fea5006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:42.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.306+0000 7fea764da640 1 --2- 172.21.15.76:0/2110443255 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fea5006ae70 0x7fea5006d330 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fea701f6110 tx=0x7fea60009040 comp rx=0 tx=0).stop 2023-11-09T03:46:42.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.306+0000 7fea764da640 1 -- 172.21.15.76:0/2110443255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea700b4880 msgr2=0x7fea7015a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:42.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.306+0000 7fea764da640 1 --2- 172.21.15.76:0/2110443255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea700b4880 0x7fea7015a1a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fea5800e990 tx=0x7fea5800ee50 comp rx=0 tx=0).stop 2023-11-09T03:46:42.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.306+0000 7fea764da640 1 -- 172.21.15.76:0/2110443255 shutdown_connections 2023-11-09T03:46:42.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.306+0000 7fea764da640 1 --2- 172.21.15.76:0/2110443255 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fea5006ae70 0x7fea5006d330 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:42.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.306+0000 7fea764da640 1 --2- 172.21.15.76:0/2110443255 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea70153e60 0x7fea700cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:42.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.306+0000 7fea764da640 1 --2- 172.21.15.76:0/2110443255 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea700cd6d0 0x7fea700c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:42.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.306+0000 7fea764da640 1 --2- 172.21.15.76:0/2110443255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea700b4880 0x7fea7015a1a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:42.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.306+0000 7fea764da640 1 -- 172.21.15.76:0/2110443255 >> 172.21.15.76:0/2110443255 conn(0x7fea700cf880 msgr2=0x7fea7015ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:42.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.306+0000 7fea764da640 1 -- 172.21.15.76:0/2110443255 shutdown_connections 2023-11-09T03:46:42.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.306+0000 7fea764da640 1 -- 172.21.15.76:0/2110443255 wait complete. 2023-11-09T03:46:42.319 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-09T03:46:42.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.426+0000 7f9688458640 1 Processor -- start 2023-11-09T03:46:42.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.426+0000 7f9688458640 1 -- start start 2023-11-09T03:46:42.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.427+0000 7f9688458640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f968006f6b0 0x7f9680065df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:42.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.427+0000 7f9688458640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f968006e320 0x7f968006e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:42.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.427+0000 7f9688458640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96800cbad0 con 0x7f968015d0b0 2023-11-09T03:46:42.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.427+0000 7f9688458640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f968006ecf0 con 0x7f968006f6b0 2023-11-09T03:46:42.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.427+0000 7f9688458640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96800cb5a0 con 0x7f968006e320 2023-11-09T03:46:42.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.427+0000 7f96869f5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f968006e320 0x7f968006e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:42.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.427+0000 7f96859f3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f968006f6b0 0x7f9680065df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:42.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.427+0000 7f96861f4640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f968015d0b0 0x7f96800c9dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:56174/0 (socket says 172.21.15.76:56174) 2023-11-09T03:46:42.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.427+0000 7f96861f4640 1 -- 172.21.15.76:0/745271149 learned_addr learned my addr 172.21.15.76:0/745271149 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:42.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.428+0000 7f96869f5640 1 -- 172.21.15.76:0/745271149 >> v2:172.21.15.113:3300/0 conn(0x7f968006f6b0 msgr2=0x7f9680065df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:42.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.428+0000 7f96869f5640 1 --2- 172.21.15.76:0/745271149 >> v2:172.21.15.113:3300/0 conn(0x7f968006f6b0 0x7f9680065df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:42.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.428+0000 7f96869f5640 1 -- 172.21.15.76:0/745271149 >> v1:172.21.15.113:6789/0 conn(0x7f968015d0b0 legacy=0x7f96800c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:42.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.428+0000 7f96869f5640 1 -- 172.21.15.76:0/745271149 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96800cb7f0 con 0x7f968006e320 2023-11-09T03:46:42.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.428+0000 7f96859f3640 1 --2- 172.21.15.76:0/745271149 >> v2:172.21.15.113:3300/0 conn(0x7f968006f6b0 0x7f9680065df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:46:42.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.429+0000 7f96869f5640 1 --2- 172.21.15.76:0/745271149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f968006e320 0x7f968006e7b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9678009d90 tx=0x7f967802bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49a876657f497bfe server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:42.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.429+0000 7f96851f2640 1 -- 172.21.15.76:0/745271149 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967802ca10 con 0x7f968006e320 2023-11-09T03:46:42.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.429+0000 7f96851f2640 1 -- 172.21.15.76:0/745271149 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967802cb70 con 0x7f968006e320 2023-11-09T03:46:42.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.429+0000 7f96851f2640 1 -- 172.21.15.76:0/745271149 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678034440 con 0x7f968006e320 2023-11-09T03:46:42.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.429+0000 7f9688458640 1 -- 172.21.15.76:0/745271149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f968006e320 msgr2=0x7f968006e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:42.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.429+0000 7f9688458640 1 --2- 172.21.15.76:0/745271149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f968006e320 0x7f968006e7b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9678009d90 tx=0x7f967802bce0 comp rx=0 tx=0).stop 2023-11-09T03:46:42.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.429+0000 7f9688458640 1 -- 172.21.15.76:0/745271149 shutdown_connections 2023-11-09T03:46:42.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.429+0000 7f9688458640 1 --2- 172.21.15.76:0/745271149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f968006e320 0x7f968006e7b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:42.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.429+0000 7f9688458640 1 --2- 172.21.15.76:0/745271149 >> v2:172.21.15.113:3300/0 conn(0x7f968006f6b0 0x7f9680065df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:42.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.429+0000 7f9688458640 1 -- 172.21.15.76:0/745271149 >> 172.21.15.76:0/745271149 conn(0x7f96800b0ce0 msgr2=0x7f96800b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:42.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.429+0000 7f9688458640 1 -- 172.21.15.76:0/745271149 shutdown_connections 2023-11-09T03:46:42.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.429+0000 7f9688458640 1 -- 172.21.15.76:0/745271149 wait complete. 2023-11-09T03:46:42.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.431+0000 7f9688458640 1 Processor -- start 2023-11-09T03:46:42.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.431+0000 7f9688458640 1 -- start start 2023-11-09T03:46:42.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.431+0000 7f9688458640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f968006e320 0x7f96801576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:42.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.431+0000 7f9688458640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f968006f6b0 0x7f96800b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:42.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.431+0000 7f9688458640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f968015d0b0 0x7f96800b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:42.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.431+0000 7f9688458640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96800b3310 con 0x7f968015d0b0 2023-11-09T03:46:42.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.431+0000 7f9688458640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96800b4da0 con 0x7f968006e320 2023-11-09T03:46:42.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.431+0000 7f9688458640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96800b2690 con 0x7f968006f6b0 2023-11-09T03:46:42.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.431+0000 7f96869f5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f968015d0b0 0x7f96800b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:42.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.431+0000 7f96869f5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f968015d0b0 0x7f96800b3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35292/0 (socket says 172.21.15.76:35292) 2023-11-09T03:46:42.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.431+0000 7f96869f5640 1 -- 172.21.15.76:0/325552030 learned_addr learned my addr 172.21.15.76:0/325552030 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:42.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.431+0000 7f96861f4640 1 --2- 172.21.15.76:0/325552030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f968006e320 0x7f96801576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:42.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.432+0000 7f96859f3640 1 --2- 172.21.15.76:0/325552030 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f968006f6b0 0x7f96800b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:42.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.432+0000 7f96869f5640 1 -- 172.21.15.76:0/325552030 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f968006f6b0 msgr2=0x7f96800b1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:42.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.432+0000 7f96869f5640 1 --2- 172.21.15.76:0/325552030 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f968006f6b0 0x7f96800b1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:42.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.432+0000 7f96869f5640 1 -- 172.21.15.76:0/325552030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f968006e320 msgr2=0x7f96801576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:42.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.432+0000 7f96869f5640 1 --2- 172.21.15.76:0/325552030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f968006e320 0x7f96801576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:42.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.432+0000 7f96869f5640 1 -- 172.21.15.76:0/325552030 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9670009400 con 0x7f968015d0b0 2023-11-09T03:46:42.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.432+0000 7f96869f5640 1 --2- 172.21.15.76:0/325552030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f968015d0b0 0x7f96800b3f70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9678009eb0 tx=0x7f9678004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:42.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.432+0000 7f9676ffd640 1 -- 172.21.15.76:0/325552030 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678040650 con 0x7f968015d0b0 2023-11-09T03:46:42.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.432+0000 7f9676ffd640 1 -- 172.21.15.76:0/325552030 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96780407b0 con 0x7f968015d0b0 2023-11-09T03:46:42.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.432+0000 7f9688458640 1 -- 172.21.15.76:0/325552030 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9678009a70 con 0x7f968015d0b0 2023-11-09T03:46:42.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.432+0000 7f9676ffd640 1 -- 172.21.15.76:0/325552030 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678040b60 con 0x7f968015d0b0 2023-11-09T03:46:42.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.432+0000 7f9688458640 1 -- 172.21.15.76:0/325552030 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96800b2c50 con 0x7f968015d0b0 2023-11-09T03:46:42.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.433+0000 7f9688458640 1 -- 172.21.15.76:0/325552030 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f968004e490 con 0x7f968015d0b0 2023-11-09T03:46:42.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.433+0000 7f9676ffd640 1 -- 172.21.15.76:0/325552030 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9678040cc0 con 0x7f968015d0b0 2023-11-09T03:46:42.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.434+0000 7f9676ffd640 1 --2- 172.21.15.76:0/325552030 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f966406ae70 0x7f966406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:42.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.434+0000 7f9676ffd640 1 -- 172.21.15.76:0/325552030 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6778+0+0 (secure 0 0 0) 0x7f96780ac5e0 con 0x7f968015d0b0 2023-11-09T03:46:42.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.434+0000 7f96861f4640 1 --2- 172.21.15.76:0/325552030 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f966406ae70 0x7f966406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:42.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.434+0000 7f96861f4640 1 --2- 172.21.15.76:0/325552030 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f966406ae70 0x7f966406d330 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f967c000f60 tx=0x7f967c007e00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:42.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.438+0000 7f9676ffd640 1 -- 172.21.15.76:0/325552030 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9678077430 con 0x7f968015d0b0 2023-11-09T03:46:42.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:42.605+0000 7f9688458640 1 -- 172.21.15.76:0/325552030 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f96800b5170 con 0x7f968015d0b0 2023-11-09T03:46:43.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.305+0000 7f9676ffd640 1 -- 172.21.15.76:0/325552030 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9678009450 con 0x7f968015d0b0 2023-11-09T03:46:43.306 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' removed 2023-11-09T03:46:43.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.308+0000 7f9688458640 1 -- 172.21.15.76:0/325552030 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f966406ae70 msgr2=0x7f966406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:43.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.308+0000 7f9688458640 1 --2- 172.21.15.76:0/325552030 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f966406ae70 0x7f966406d330 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f967c000f60 tx=0x7f967c007e00 comp rx=0 tx=0).stop 2023-11-09T03:46:43.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.308+0000 7f9688458640 1 -- 172.21.15.76:0/325552030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f968015d0b0 msgr2=0x7f96800b3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:43.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.308+0000 7f9688458640 1 --2- 172.21.15.76:0/325552030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f968015d0b0 0x7f96800b3f70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9678009eb0 tx=0x7f9678004a40 comp rx=0 tx=0).stop 2023-11-09T03:46:43.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.308+0000 7f9688458640 1 -- 172.21.15.76:0/325552030 shutdown_connections 2023-11-09T03:46:43.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.308+0000 7f9688458640 1 --2- 172.21.15.76:0/325552030 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f966406ae70 0x7f966406d330 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:43.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.308+0000 7f9688458640 1 --2- 172.21.15.76:0/325552030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f968015d0b0 0x7f96800b3f70 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:43.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.308+0000 7f9688458640 1 --2- 172.21.15.76:0/325552030 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f968006f6b0 0x7f96800b1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:43.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.308+0000 7f9688458640 1 --2- 172.21.15.76:0/325552030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f968006e320 0x7f96801576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:43.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.308+0000 7f9688458640 1 -- 172.21.15.76:0/325552030 >> 172.21.15.76:0/325552030 conn(0x7f96800b0ce0 msgr2=0x7f968015fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:43.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.308+0000 7f9688458640 1 -- 172.21.15.76:0/325552030 shutdown_connections 2023-11-09T03:46:43.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.308+0000 7f9688458640 1 -- 172.21.15.76:0/325552030 wait complete. 2023-11-09T03:46:43.321 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-09T03:46:43.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.430+0000 7f1289651640 1 Processor -- start 2023-11-09T03:46:43.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.430+0000 7f1289651640 1 -- start start 2023-11-09T03:46:43.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.431+0000 7f1289651640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f12840cc8e0 0x7f12840ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:43.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.431+0000 7f1289651640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12840cc420 con 0x7f128415d7b0 2023-11-09T03:46:43.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.431+0000 7f1289651640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12840cf820 con 0x7f1284159cc0 2023-11-09T03:46:43.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.431+0000 7f1289651640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f128415a0a0 con 0x7f12840cc8e0 2023-11-09T03:46:43.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.431+0000 7f12827fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f12840cc8e0 0x7f12840ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:43.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.431+0000 7f12827fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f12840cc8e0 0x7f12840ccd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43980/0 (socket says 172.21.15.76:43980) 2023-11-09T03:46:43.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.431+0000 7f12837fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f128415d7b0 0x7f128406f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50578/0 (socket says 172.21.15.76:50578) 2023-11-09T03:46:43.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.431+0000 7f12827fc640 1 -- 172.21.15.76:0/3985235280 learned_addr learned my addr 172.21.15.76:0/3985235280 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:43.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.431+0000 7f12827fc640 1 -- 172.21.15.76:0/3985235280 >> v1:172.21.15.113:6789/0 conn(0x7f1284159cc0 legacy=0x7f12840ae9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:43.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.431+0000 7f12827fc640 1 -- 172.21.15.76:0/3985235280 >> v1:172.21.15.76:6789/0 conn(0x7f128415d7b0 legacy=0x7f128406f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:43.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.431+0000 7f12827fc640 1 -- 172.21.15.76:0/3985235280 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12840cb800 con 0x7f12840cc8e0 2023-11-09T03:46:43.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.432+0000 7f12827fc640 1 --2- 172.21.15.76:0/3985235280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12840cc8e0 0x7f12840ccd60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f12840caf40 tx=0x7f127802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dcf11ed14081762 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:43.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.432+0000 7f1281ffb640 1 -- 172.21.15.76:0/3985235280 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1278031640 con 0x7f12840cc8e0 2023-11-09T03:46:43.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.432+0000 7f1281ffb640 1 -- 172.21.15.76:0/3985235280 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12780317a0 con 0x7f12840cc8e0 2023-11-09T03:46:43.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.432+0000 7f1281ffb640 1 -- 172.21.15.76:0/3985235280 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1278031b30 con 0x7f12840cc8e0 2023-11-09T03:46:43.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.433+0000 7f1289651640 1 -- 172.21.15.76:0/3985235280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12840cc8e0 msgr2=0x7f12840ccd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:43.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.433+0000 7f1289651640 1 --2- 172.21.15.76:0/3985235280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12840cc8e0 0x7f12840ccd60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f12840caf40 tx=0x7f127802b8f0 comp rx=0 tx=0).stop 2023-11-09T03:46:43.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.433+0000 7f1289651640 1 -- 172.21.15.76:0/3985235280 shutdown_connections 2023-11-09T03:46:43.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.433+0000 7f1289651640 1 --2- 172.21.15.76:0/3985235280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12840cc8e0 0x7f12840ccd60 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:43.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.433+0000 7f1289651640 1 -- 172.21.15.76:0/3985235280 >> 172.21.15.76:0/3985235280 conn(0x7f128406d890 msgr2=0x7f128406dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:43.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.433+0000 7f1289651640 1 -- 172.21.15.76:0/3985235280 shutdown_connections 2023-11-09T03:46:43.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.433+0000 7f1289651640 1 -- 172.21.15.76:0/3985235280 wait complete. 2023-11-09T03:46:43.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.434+0000 7f1289651640 1 Processor -- start 2023-11-09T03:46:43.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.434+0000 7f1289651640 1 -- start start 2023-11-09T03:46:43.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.435+0000 7f1289651640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12840cc8e0 0x7f12840aec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:43.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.435+0000 7f1289651640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1284159cc0 0x7f12840c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:43.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.435+0000 7f1289651640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f128415d7b0 0x7f12840c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:43.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.435+0000 7f1289651640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12840afe20 con 0x7f12840cc8e0 2023-11-09T03:46:43.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.435+0000 7f1289651640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12840af580 con 0x7f1284159cc0 2023-11-09T03:46:43.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.435+0000 7f1289651640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12840af860 con 0x7f128415d7b0 2023-11-09T03:46:43.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.435+0000 7f1282ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12840cc8e0 0x7f12840aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:43.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.435+0000 7f1282ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12840cc8e0 0x7f12840aec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43984/0 (socket says 172.21.15.76:43984) 2023-11-09T03:46:43.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.435+0000 7f1282ffd640 1 -- 172.21.15.76:0/1599858503 learned_addr learned my addr 172.21.15.76:0/1599858503 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:43.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.435+0000 7f1282ffd640 1 -- 172.21.15.76:0/1599858503 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f128415d7b0 msgr2=0x7f12840c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:43.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.435+0000 7f1282ffd640 1 --2- 172.21.15.76:0/1599858503 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f128415d7b0 0x7f12840c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:43.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.435+0000 7f1282ffd640 1 -- 172.21.15.76:0/1599858503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1284159cc0 msgr2=0x7f12840c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:43.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.435+0000 7f1282ffd640 1 --2- 172.21.15.76:0/1599858503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1284159cc0 0x7f12840c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:43.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.435+0000 7f1282ffd640 1 -- 172.21.15.76:0/1599858503 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12780094d0 con 0x7f12840cc8e0 2023-11-09T03:46:43.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.436+0000 7f1282ffd640 1 --2- 172.21.15.76:0/1599858503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12840cc8e0 0x7f12840aec80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f127400c7a0 tx=0x7f127400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:43.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.436+0000 7f126b7fe640 1 -- 172.21.15.76:0/1599858503 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274003d80 con 0x7f12840cc8e0 2023-11-09T03:46:43.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.436+0000 7f126b7fe640 1 -- 172.21.15.76:0/1599858503 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1274005070 con 0x7f12840cc8e0 2023-11-09T03:46:43.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.436+0000 7f1289651640 1 -- 172.21.15.76:0/1599858503 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12840afb40 con 0x7f12840cc8e0 2023-11-09T03:46:43.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.436+0000 7f126b7fe640 1 -- 172.21.15.76:0/1599858503 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274005460 con 0x7f12840cc8e0 2023-11-09T03:46:43.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.436+0000 7f1289651640 1 -- 172.21.15.76:0/1599858503 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12840af1c0 con 0x7f12840cc8e0 2023-11-09T03:46:43.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.437+0000 7f1289651640 1 -- 172.21.15.76:0/1599858503 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1284067c50 con 0x7f12840cc8e0 2023-11-09T03:46:43.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.437+0000 7f126b7fe640 1 -- 172.21.15.76:0/1599858503 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f12740055c0 con 0x7f12840cc8e0 2023-11-09T03:46:43.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.437+0000 7f126b7fe640 1 --2- 172.21.15.76:0/1599858503 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f127006aea0 0x7f127006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:43.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.437+0000 7f126b7fe640 1 -- 172.21.15.76:0/1599858503 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6340+0+0 (secure 0 0 0) 0x7f1274088ca0 con 0x7f12840cc8e0 2023-11-09T03:46:43.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.438+0000 7f12827fc640 1 --2- 172.21.15.76:0/1599858503 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f127006aea0 0x7f127006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:43.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.438+0000 7f12827fc640 1 --2- 172.21.15.76:0/1599858503 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f127006aea0 0x7f127006d360 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1278036000 tx=0x7f1278030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:43.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.442+0000 7f126b7fe640 1 -- 172.21.15.76:0/1599858503 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1274053b60 con 0x7f12840cc8e0 2023-11-09T03:46:43.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:43.612+0000 7f1289651640 1 -- 172.21.15.76:0/1599858503 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f12840aea60 con 0x7f12840cc8e0 2023-11-09T03:46:44.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.311+0000 7f126b7fe640 1 -- 172.21.15.76:0/1599858503 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1274057180 con 0x7f12840cc8e0 2023-11-09T03:46:44.312 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' removed 2023-11-09T03:46:44.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.315+0000 7f1289651640 1 -- 172.21.15.76:0/1599858503 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f127006aea0 msgr2=0x7f127006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:44.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.315+0000 7f1289651640 1 --2- 172.21.15.76:0/1599858503 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f127006aea0 0x7f127006d360 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1278036000 tx=0x7f1278030000 comp rx=0 tx=0).stop 2023-11-09T03:46:44.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.315+0000 7f1289651640 1 -- 172.21.15.76:0/1599858503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12840cc8e0 msgr2=0x7f12840aec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:44.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.315+0000 7f1289651640 1 --2- 172.21.15.76:0/1599858503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12840cc8e0 0x7f12840aec80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f127400c7a0 tx=0x7f127400cc60 comp rx=0 tx=0).stop 2023-11-09T03:46:44.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.315+0000 7f1289651640 1 -- 172.21.15.76:0/1599858503 shutdown_connections 2023-11-09T03:46:44.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.315+0000 7f1289651640 1 --2- 172.21.15.76:0/1599858503 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f127006aea0 0x7f127006d360 secure :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1278036000 tx=0x7f1278030000 comp rx=0 tx=0).stop 2023-11-09T03:46:44.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.315+0000 7f1289651640 1 --2- 172.21.15.76:0/1599858503 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f128415d7b0 0x7f12840c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:44.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.315+0000 7f1289651640 1 --2- 172.21.15.76:0/1599858503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1284159cc0 0x7f12840c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:44.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.315+0000 7f1289651640 1 --2- 172.21.15.76:0/1599858503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12840cc8e0 0x7f12840aec80 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:44.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.315+0000 7f1289651640 1 -- 172.21.15.76:0/1599858503 >> 172.21.15.76:0/1599858503 conn(0x7f128406d890 msgr2=0x7f128415c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:44.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.316+0000 7f1289651640 1 -- 172.21.15.76:0/1599858503 shutdown_connections 2023-11-09T03:46:44.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.316+0000 7f1289651640 1 -- 172.21.15.76:0/1599858503 wait complete. 2023-11-09T03:46:44.331 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-11-09T03:46:44.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.441+0000 7f68e90c9640 1 Processor -- start 2023-11-09T03:46:44.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.441+0000 7f68e90c9640 1 -- start start 2023-11-09T03:46:44.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.442+0000 7f68e90c9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68e406dd50 0x7f68e4165bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:44.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.442+0000 7f68e90c9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e415c0d0 con 0x7f68e415f510 2023-11-09T03:46:44.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.442+0000 7f68e90c9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e406cf90 con 0x7f68e4151ea0 2023-11-09T03:46:44.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.442+0000 7f68e90c9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e406d270 con 0x7f68e406dd50 2023-11-09T03:46:44.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.442+0000 7f68e259c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f68e415f510 0x7f68e40cb150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50580/0 (socket says 172.21.15.76:50580) 2023-11-09T03:46:44.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.442+0000 7f68e259c640 1 -- 172.21.15.76:0/4228901428 learned_addr learned my addr 172.21.15.76:0/4228901428 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:44.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.442+0000 7f68e1d9b640 1 -- 172.21.15.76:0/4228901428 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2195696907 0 0) 0x7f68e415c0d0 con 0x7f68e415f510 2023-11-09T03:46:44.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.442+0000 7f68e359e640 1 --2- 172.21.15.76:0/4228901428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68e406dd50 0x7f68e4165bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:44.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.442+0000 7f68e1d9b640 1 -- 172.21.15.76:0/4228901428 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68cc003520 con 0x7f68e415f510 2023-11-09T03:46:44.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.442+0000 7f68e1d9b640 1 -- 172.21.15.76:0/4228901428 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2441075548 0 0) 0x7f68cc003520 con 0x7f68e415f510 2023-11-09T03:46:44.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.442+0000 7f68e1d9b640 1 -- 172.21.15.76:0/4228901428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68e406dd50 msgr2=0x7f68e4165bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:44.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.442+0000 7f68e1d9b640 1 --2- 172.21.15.76:0/4228901428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68e406dd50 0x7f68e4165bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:44.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.442+0000 7f68e1d9b640 1 -- 172.21.15.76:0/4228901428 >> v1:172.21.15.113:6789/0 conn(0x7f68e4151ea0 legacy=0x7f68e415b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:44.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.442+0000 7f68e1d9b640 1 -- 172.21.15.76:0/4228901428 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e406d550 con 0x7f68e415f510 2023-11-09T03:46:44.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.442+0000 7f68e1d9b640 1 -- 172.21.15.76:0/4228901428 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f68d8001670 con 0x7f68e415f510 2023-11-09T03:46:44.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.442+0000 7f68e1d9b640 1 -- 172.21.15.76:0/4228901428 >> v1:172.21.15.76:6789/0 conn(0x7f68e415f510 legacy=0x7f68e40cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:44.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.442+0000 7f68e1d9b640 1 --2- 172.21.15.76:0/4228901428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68cc0039a0 0x7f68cc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:44.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.442+0000 7f68e1d9b640 1 --2- 172.21.15.76:0/4228901428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68e406dd50 0x7f68cc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:44.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.442+0000 7f68e1d9b640 1 --2- 172.21.15.76:0/4228901428 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68cc004d00 0x7f68cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:44.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.442+0000 7f68e1d9b640 1 -- 172.21.15.76:0/4228901428 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e406d270 con 0x7f68cc004d00 2023-11-09T03:46:44.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.442+0000 7f68e1d9b640 1 -- 172.21.15.76:0/4228901428 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc00ba60 con 0x7f68cc0039a0 2023-11-09T03:46:44.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.442+0000 7f68e1d9b640 1 -- 172.21.15.76:0/4228901428 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc00bc20 con 0x7f68e406dd50 2023-11-09T03:46:44.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.443+0000 7f68e359e640 1 --2- 172.21.15.76:0/4228901428 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68cc004d00 0x7f68cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:44.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.443+0000 7f68e2d9d640 1 --2- 172.21.15.76:0/4228901428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68cc0039a0 0x7f68cc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:44.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.443+0000 7f68e259c640 1 --2- 172.21.15.76:0/4228901428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68e406dd50 0x7f68cc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:44.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.443+0000 7f68e359e640 1 -- 172.21.15.76:0/4228901428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68e406dd50 msgr2=0x7f68cc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:44.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.443+0000 7f68e359e640 1 --2- 172.21.15.76:0/4228901428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68e406dd50 0x7f68cc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:44.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.443+0000 7f68e359e640 1 -- 172.21.15.76:0/4228901428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68cc0039a0 msgr2=0x7f68cc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:44.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.443+0000 7f68e359e640 1 --2- 172.21.15.76:0/4228901428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68cc0039a0 0x7f68cc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:44.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.443+0000 7f68e359e640 1 -- 172.21.15.76:0/4228901428 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68cc003520 con 0x7f68cc004d00 2023-11-09T03:46:44.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.444+0000 7f68e359e640 1 --2- 172.21.15.76:0/4228901428 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68cc004d00 0x7f68cc005180 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009a90 tx=0x7f68cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ed9d4ca390c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:44.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.444+0000 7f68e1d9b640 1 -- 172.21.15.76:0/4228901428 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc02c680 con 0x7f68cc004d00 2023-11-09T03:46:44.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.444+0000 7f68e1d9b640 1 -- 172.21.15.76:0/4228901428 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68dc02c7e0 con 0x7f68cc004d00 2023-11-09T03:46:44.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.444+0000 7f68e1d9b640 1 -- 172.21.15.76:0/4228901428 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc02cb70 con 0x7f68cc004d00 2023-11-09T03:46:44.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.444+0000 7f68e90c9640 1 -- 172.21.15.76:0/4228901428 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68cc004d00 msgr2=0x7f68cc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:44.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.444+0000 7f68e90c9640 1 --2- 172.21.15.76:0/4228901428 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68cc004d00 0x7f68cc005180 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009a90 tx=0x7f68cc0057f0 comp rx=0 tx=0).stop 2023-11-09T03:46:44.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.444+0000 7f68e359e640 1 -- 172.21.15.76:0/4228901428 reap_dead start 2023-11-09T03:46:44.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.444+0000 7f68e90c9640 1 -- 172.21.15.76:0/4228901428 shutdown_connections 2023-11-09T03:46:44.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.444+0000 7f68e90c9640 1 -- 172.21.15.76:0/4228901428 >> 172.21.15.76:0/4228901428 conn(0x7f68e40b05d0 msgr2=0x7f68e40b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:44.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.444+0000 7f68e90c9640 1 -- 172.21.15.76:0/4228901428 shutdown_connections 2023-11-09T03:46:44.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.444+0000 7f68e90c9640 1 -- 172.21.15.76:0/4228901428 wait complete. 2023-11-09T03:46:44.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.446+0000 7f68e90c9640 1 Processor -- start 2023-11-09T03:46:44.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.446+0000 7f68e90c9640 1 -- start start 2023-11-09T03:46:44.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.446+0000 7f68e90c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68cc004d00 0x7f68e4146010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:44.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.446+0000 7f68e90c9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68e4147540 0x7f68e40cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:44.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.446+0000 7f68e90c9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68e40cdf20 0x7f68e40c7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:44.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.447+0000 7f68e90c9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e40c67b0 con 0x7f68cc004d00 2023-11-09T03:46:44.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.447+0000 7f68e90c9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e40ce3a0 con 0x7f68e40cdf20 2023-11-09T03:46:44.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.447+0000 7f68e90c9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e4146550 con 0x7f68e4147540 2023-11-09T03:46:44.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.447+0000 7f68e2d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68cc004d00 0x7f68e4146010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:44.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.447+0000 7f68e2d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68cc004d00 0x7f68e4146010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44008/0 (socket says 172.21.15.76:44008) 2023-11-09T03:46:44.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.447+0000 7f68e2d9d640 1 -- 172.21.15.76:0/1553312655 learned_addr learned my addr 172.21.15.76:0/1553312655 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:44.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.447+0000 7f68e259c640 1 --2- 172.21.15.76:0/1553312655 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68e4147540 0x7f68e40cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:44.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.447+0000 7f68e359e640 1 --2- 172.21.15.76:0/1553312655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68e40cdf20 0x7f68e40c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:44.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.447+0000 7f68e2d9d640 1 -- 172.21.15.76:0/1553312655 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68e4147540 msgr2=0x7f68e40cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:44.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.447+0000 7f68e2d9d640 1 --2- 172.21.15.76:0/1553312655 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68e4147540 0x7f68e40cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:44.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.447+0000 7f68e2d9d640 1 -- 172.21.15.76:0/1553312655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68e40cdf20 msgr2=0x7f68e40c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:44.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.447+0000 7f68e2d9d640 1 --2- 172.21.15.76:0/1553312655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68e40cdf20 0x7f68e40c7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:44.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.447+0000 7f68e2d9d640 1 -- 172.21.15.76:0/1553312655 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e40c5b30 con 0x7f68cc004d00 2023-11-09T03:46:44.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.447+0000 7f68e359e640 1 --2- 172.21.15.76:0/1553312655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68e40cdf20 0x7f68e40c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:44.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.447+0000 7f68e259c640 1 --2- 172.21.15.76:0/1553312655 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68e4147540 0x7f68e40cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:44.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.447+0000 7f68e2d9d640 1 --2- 172.21.15.76:0/1553312655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68cc004d00 0x7f68e4146010 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f68d400fb90 tx=0x7f68d400d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:44.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.447+0000 7f68cb7fe640 1 -- 172.21.15.76:0/1553312655 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d400fe90 con 0x7f68cc004d00 2023-11-09T03:46:44.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.447+0000 7f68cb7fe640 1 -- 172.21.15.76:0/1553312655 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d400dc30 con 0x7f68cc004d00 2023-11-09T03:46:44.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.447+0000 7f68e90c9640 1 -- 172.21.15.76:0/1553312655 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e40c5e10 con 0x7f68cc004d00 2023-11-09T03:46:44.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.447+0000 7f68cb7fe640 1 -- 172.21.15.76:0/1553312655 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d4003970 con 0x7f68cc004d00 2023-11-09T03:46:44.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.447+0000 7f68e90c9640 1 -- 172.21.15.76:0/1553312655 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e40c63d0 con 0x7f68cc004d00 2023-11-09T03:46:44.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.448+0000 7f68e90c9640 1 -- 172.21.15.76:0/1553312655 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e404e490 con 0x7f68cc004d00 2023-11-09T03:46:44.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.449+0000 7f68cb7fe640 1 -- 172.21.15.76:0/1553312655 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f68d4003ad0 con 0x7f68cc004d00 2023-11-09T03:46:44.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.449+0000 7f68cb7fe640 1 --2- 172.21.15.76:0/1553312655 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f68d006ae70 0x7f68d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:44.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.449+0000 7f68cb7fe640 1 -- 172.21.15.76:0/1553312655 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5781+0+0 (secure 0 0 0) 0x7f68d4089900 con 0x7f68cc004d00 2023-11-09T03:46:44.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.449+0000 7f68e259c640 1 --2- 172.21.15.76:0/1553312655 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f68d006ae70 0x7f68d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:44.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.450+0000 7f68e259c640 1 --2- 172.21.15.76:0/1553312655 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f68d006ae70 0x7f68d006d330 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f68e414a220 tx=0x7f68d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:44.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.453+0000 7f68cb7fe640 1 -- 172.21.15.76:0/1553312655 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f68d40549f0 con 0x7f68cc004d00 2023-11-09T03:46:44.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:44.626+0000 7f68e90c9640 1 -- 172.21.15.76:0/1553312655 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f68e40c6590 con 0x7f68cc004d00 2023-11-09T03:46:45.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.318+0000 7f68cb7fe640 1 -- 172.21.15.76:0/1553312655 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) v1 ==== 178+0+0 (secure 0 0 0) 0x7f68d4058010 con 0x7f68cc004d00 2023-11-09T03:46:45.319 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data_ec' removed 2023-11-09T03:46:45.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.321+0000 7f68e90c9640 1 -- 172.21.15.76:0/1553312655 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f68d006ae70 msgr2=0x7f68d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:45.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.321+0000 7f68e90c9640 1 --2- 172.21.15.76:0/1553312655 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f68d006ae70 0x7f68d006d330 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f68e414a220 tx=0x7f68d8009040 comp rx=0 tx=0).stop 2023-11-09T03:46:45.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.321+0000 7f68e90c9640 1 -- 172.21.15.76:0/1553312655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68cc004d00 msgr2=0x7f68e4146010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:45.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.321+0000 7f68e90c9640 1 --2- 172.21.15.76:0/1553312655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68cc004d00 0x7f68e4146010 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f68d400fb90 tx=0x7f68d400d520 comp rx=0 tx=0).stop 2023-11-09T03:46:45.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.321+0000 7f68e90c9640 1 -- 172.21.15.76:0/1553312655 shutdown_connections 2023-11-09T03:46:45.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.321+0000 7f68e90c9640 1 --2- 172.21.15.76:0/1553312655 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f68d006ae70 0x7f68d006d330 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:45.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.321+0000 7f68e90c9640 1 --2- 172.21.15.76:0/1553312655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68e40cdf20 0x7f68e40c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:45.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.321+0000 7f68e90c9640 1 --2- 172.21.15.76:0/1553312655 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68e4147540 0x7f68e40cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:45.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.321+0000 7f68e90c9640 1 --2- 172.21.15.76:0/1553312655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68cc004d00 0x7f68e4146010 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:45.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.322+0000 7f68e90c9640 1 -- 172.21.15.76:0/1553312655 >> 172.21.15.76:0/1553312655 conn(0x7f68e40b05d0 msgr2=0x7f68e40ceaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:45.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.322+0000 7f68e90c9640 1 -- 172.21.15.76:0/1553312655 shutdown_connections 2023-11-09T03:46:45.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.322+0000 7f68e90c9640 1 -- 172.21.15.76:0/1553312655 wait complete. 2023-11-09T03:46:45.335 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-09T03:46:45.335 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-09T03:46:45.336 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-09T03:46:45.336 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:46:45.336 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-09T03:46:45.336 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-09T03:46:45.337 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-09T03:46:45.337 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:46:45.337 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-09T03:46:45.337 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-09T03:46:45.338 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-09T03:46:45.338 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:46:45.338 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-09T03:46:45.338 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-09T03:46:45.339 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-09T03:46:45.339 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:46:45.345 INFO:tasks.ceph.mds.a:Stopped 2023-11-09T03:46:45.346 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-09T03:46:45.347 INFO:tasks.ceph.mds.c:Stopped 2023-11-09T03:46:45.347 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-09T03:46:45.350 INFO:tasks.ceph.mds.b:Stopped 2023-11-09T03:46:45.350 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-09T03:46:45.351 INFO:tasks.ceph.mds.d:Stopped 2023-11-09T03:46:45.351 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-09T03:46:45.389 INFO:tasks.ceph.mds.a:Started 2023-11-09T03:46:45.389 DEBUG:teuthology.parallel:result is None 2023-11-09T03:46:45.390 INFO:tasks.ceph.mds.c:Started 2023-11-09T03:46:45.391 DEBUG:teuthology.parallel:result is None 2023-11-09T03:46:45.393 INFO:tasks.ceph.mds.b:Started 2023-11-09T03:46:45.393 DEBUG:teuthology.parallel:result is None 2023-11-09T03:46:45.394 INFO:tasks.ceph.mds.d:Started 2023-11-09T03:46:45.394 DEBUG:teuthology.parallel:result is None 2023-11-09T03:46:45.395 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-09T03:46:45.467 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2023-11-09T03:46:45.467 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2023-11-09T03:46:45.477 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2023-11-09T03:46:45.477 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2023-11-09T03:46:45.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.502+0000 7fcbc0333640 1 Processor -- start 2023-11-09T03:46:45.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.502+0000 7fcbc0333640 1 -- start start 2023-11-09T03:46:45.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.503+0000 7fcbc0333640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcbb80cd6d0 0x7fcbb8161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:45.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.503+0000 7fcbc0333640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbb80b4880 0x7fcbb80b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:45.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.503+0000 7fcbc0333640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbb80cde60 con 0x7fcbb8153e60 2023-11-09T03:46:45.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.503+0000 7fcbc0333640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbb8160f90 con 0x7fcbb80cd6d0 2023-11-09T03:46:45.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.503+0000 7fcbc0333640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb8161960 con 0x7fcbb80b4880 2023-11-09T03:46:45.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.503+0000 7fcbbd8ce640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcbb80cd6d0 0x7fcbb8161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:45.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.503+0000 7fcbbd8ce640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcbb80cd6d0 0x7fcbb8161420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44042/0 (socket says 172.21.15.76:44042) 2023-11-09T03:46:45.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.503+0000 7fcbbd8ce640 1 -- 172.21.15.76:0/4022873853 learned_addr learned my addr 172.21.15.76:0/4022873853 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:45.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.503+0000 7fcbbd8ce640 1 -- 172.21.15.76:0/4022873853 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbb80b4880 msgr2=0x7fcbb80b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:45.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.503+0000 7fcbbe8d0640 1 --2- 172.21.15.76:0/4022873853 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbb80b4880 0x7fcbb80b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:45.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.503+0000 7fcbbd8ce640 1 --2- 172.21.15.76:0/4022873853 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbb80b4880 0x7fcbb80b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:45.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.503+0000 7fcbbd8ce640 1 -- 172.21.15.76:0/4022873853 >> v1:172.21.15.76:6789/0 conn(0x7fcbb8153e60 legacy=0x7fcbb815dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:45.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.503+0000 7fcbbd8ce640 1 -- 172.21.15.76:0/4022873853 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb80b5220 con 0x7fcbb80cd6d0 2023-11-09T03:46:45.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.503+0000 7fcbbe8d0640 1 --2- 172.21.15.76:0/4022873853 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbb80b4880 0x7fcbb80b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:45.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.504+0000 7fcbbd8ce640 1 --2- 172.21.15.76:0/4022873853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbb80cd6d0 0x7fcbb8161420 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009870 tx=0x7fcba802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e579fe4bc44145 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:45.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.504+0000 7fcbbd0cd640 1 -- 172.21.15.76:0/4022873853 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8030840 con 0x7fcbb80cd6d0 2023-11-09T03:46:45.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.504+0000 7fcbbd0cd640 1 -- 172.21.15.76:0/4022873853 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba80309a0 con 0x7fcbb80cd6d0 2023-11-09T03:46:45.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.504+0000 7fcbbd0cd640 1 -- 172.21.15.76:0/4022873853 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8030d30 con 0x7fcbb80cd6d0 2023-11-09T03:46:45.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.505+0000 7fcbc0333640 1 -- 172.21.15.76:0/4022873853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbb80cd6d0 msgr2=0x7fcbb8161420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:45.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.505+0000 7fcbc0333640 1 --2- 172.21.15.76:0/4022873853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbb80cd6d0 0x7fcbb8161420 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009870 tx=0x7fcba802b980 comp rx=0 tx=0).stop 2023-11-09T03:46:45.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.505+0000 7fcbc0333640 1 -- 172.21.15.76:0/4022873853 shutdown_connections 2023-11-09T03:46:45.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.505+0000 7fcbc0333640 1 --2- 172.21.15.76:0/4022873853 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbb80b4880 0x7fcbb80b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:45.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.505+0000 7fcbc0333640 1 --2- 172.21.15.76:0/4022873853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbb80cd6d0 0x7fcbb8161420 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:45.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.505+0000 7fcbc0333640 1 -- 172.21.15.76:0/4022873853 >> 172.21.15.76:0/4022873853 conn(0x7fcbb80cf880 msgr2=0x7fcbb80cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:45.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.505+0000 7fcbc0333640 1 -- 172.21.15.76:0/4022873853 shutdown_connections 2023-11-09T03:46:45.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.505+0000 7fcbc0333640 1 -- 172.21.15.76:0/4022873853 wait complete. 2023-11-09T03:46:45.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.506+0000 7fcbc0333640 1 Processor -- start 2023-11-09T03:46:45.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.506+0000 7fcbc0333640 1 -- start start 2023-11-09T03:46:45.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.507+0000 7fcbc0333640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbb80b4880 0x7fcbb8158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:45.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.507+0000 7fcbc0333640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbb80cd6d0 0x7fcbb8153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:45.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.507+0000 7fcbc0333640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbb8153e60 0x7fcbb815c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:45.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.507+0000 7fcbc0333640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb8156fe0 con 0x7fcbb8153e60 2023-11-09T03:46:45.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.507+0000 7fcbc0333640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb8158ac0 con 0x7fcbb80cd6d0 2023-11-09T03:46:45.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.507+0000 7fcbc0333640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb815bb70 con 0x7fcbb80b4880 2023-11-09T03:46:45.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.507+0000 7fcbbe8d0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbb8153e60 0x7fcbb815c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:45.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.507+0000 7fcbbe8d0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbb8153e60 0x7fcbb815c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44044/0 (socket says 172.21.15.76:44044) 2023-11-09T03:46:45.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.507+0000 7fcbbe8d0640 1 -- 172.21.15.76:0/1195724235 learned_addr learned my addr 172.21.15.76:0/1195724235 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:45.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.507+0000 7fcbbd8ce640 1 --2- 172.21.15.76:0/1195724235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbb80cd6d0 0x7fcbb8153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:45.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.507+0000 7fcbbe0cf640 1 --2- 172.21.15.76:0/1195724235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbb80b4880 0x7fcbb8158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:45.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.507+0000 7fcbbe8d0640 1 -- 172.21.15.76:0/1195724235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbb80b4880 msgr2=0x7fcbb8158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:45.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.507+0000 7fcbbe8d0640 1 --2- 172.21.15.76:0/1195724235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbb80b4880 0x7fcbb8158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:45.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.507+0000 7fcbbe8d0640 1 -- 172.21.15.76:0/1195724235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbb80cd6d0 msgr2=0x7fcbb8153210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:45.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.507+0000 7fcbbe8d0640 1 --2- 172.21.15.76:0/1195724235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbb80cd6d0 0x7fcbb8153210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:45.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.507+0000 7fcbbe8d0640 1 -- 172.21.15.76:0/1195724235 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba80094d0 con 0x7fcbb8153e60 2023-11-09T03:46:45.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.507+0000 7fcbbe8d0640 1 --2- 172.21.15.76:0/1195724235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbb8153e60 0x7fcbb815c000 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fcbb000cef0 tx=0x7fcbb000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:45.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.507+0000 7fcbaeffd640 1 -- 172.21.15.76:0/1195724235 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb000d840 con 0x7fcbb8153e60 2023-11-09T03:46:45.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.507+0000 7fcbaeffd640 1 -- 172.21.15.76:0/1195724235 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb000d9c0 con 0x7fcbb8153e60 2023-11-09T03:46:45.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.507+0000 7fcbaeffd640 1 -- 172.21.15.76:0/1195724235 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb000dd70 con 0x7fcbb8153e60 2023-11-09T03:46:45.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.507+0000 7fcbc0333640 1 -- 172.21.15.76:0/1195724235 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb815b240 con 0x7fcbb8153e60 2023-11-09T03:46:45.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.507+0000 7fcbc0333640 1 -- 172.21.15.76:0/1195724235 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb815b800 con 0x7fcbb8153e60 2023-11-09T03:46:45.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.508+0000 7fcbc0333640 1 -- 172.21.15.76:0/1195724235 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb8074290 con 0x7fcbb8153e60 2023-11-09T03:46:45.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.509+0000 7fcbaeffd640 1 -- 172.21.15.76:0/1195724235 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcbb001e3b0 con 0x7fcbb8153e60 2023-11-09T03:46:45.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.509+0000 7fcbaeffd640 1 --2- 172.21.15.76:0/1195724235 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcb9c06ae70 0x7fcb9c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:45.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.509+0000 7fcbaeffd640 1 -- 172.21.15.76:0/1195724235 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fcbb0089df0 con 0x7fcbb8153e60 2023-11-09T03:46:45.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.509+0000 7fcbbe0cf640 1 --2- 172.21.15.76:0/1195724235 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcb9c06ae70 0x7fcb9c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:45.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.510+0000 7fcbbe0cf640 1 --2- 172.21.15.76:0/1195724235 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcb9c06ae70 0x7fcb9c06d330 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcbb81f5f70 tx=0x7fcbb4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:45.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.513+0000 7fcbaeffd640 1 -- 172.21.15.76:0/1195724235 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcbb0055260 con 0x7fcbb8153e60 2023-11-09T03:46:45.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:45.692+0000 7fcbc0333640 1 -- 172.21.15.76:0/1195724235 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fcbb80d0b80 con 0x7fcbb8153e60 2023-11-09T03:46:46.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.332+0000 7fcbaeffd640 1 -- 172.21.15.76:0/1195724235 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcbb001a080 con 0x7fcbb8153e60 2023-11-09T03:46:46.332 INFO:teuthology.orchestra.run.smithi076.stderr: removed all blocklist entries 2023-11-09T03:46:46.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.334+0000 7fcbc0333640 1 -- 172.21.15.76:0/1195724235 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcb9c06ae70 msgr2=0x7fcb9c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:46.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.334+0000 7fcbc0333640 1 --2- 172.21.15.76:0/1195724235 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcb9c06ae70 0x7fcb9c06d330 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcbb81f5f70 tx=0x7fcbb4009040 comp rx=0 tx=0).stop 2023-11-09T03:46:46.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.334+0000 7fcbc0333640 1 -- 172.21.15.76:0/1195724235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbb8153e60 msgr2=0x7fcbb815c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:46.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.334+0000 7fcbc0333640 1 --2- 172.21.15.76:0/1195724235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbb8153e60 0x7fcbb815c000 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fcbb000cef0 tx=0x7fcbb000a710 comp rx=0 tx=0).stop 2023-11-09T03:46:46.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.334+0000 7fcbc0333640 1 -- 172.21.15.76:0/1195724235 shutdown_connections 2023-11-09T03:46:46.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.334+0000 7fcbc0333640 1 --2- 172.21.15.76:0/1195724235 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcb9c06ae70 0x7fcb9c06d330 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:46.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.334+0000 7fcbc0333640 1 --2- 172.21.15.76:0/1195724235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbb8153e60 0x7fcbb815c000 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:46.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.334+0000 7fcbc0333640 1 --2- 172.21.15.76:0/1195724235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbb80cd6d0 0x7fcbb8153210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:46.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.334+0000 7fcbc0333640 1 --2- 172.21.15.76:0/1195724235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbb80b4880 0x7fcbb8158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:46.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.334+0000 7fcbc0333640 1 -- 172.21.15.76:0/1195724235 >> 172.21.15.76:0/1195724235 conn(0x7fcbb80cf880 msgr2=0x7fcbb80d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:46.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.334+0000 7fcbc0333640 1 -- 172.21.15.76:0/1195724235 shutdown_connections 2023-11-09T03:46:46.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.334+0000 7fcbc0333640 1 -- 172.21.15.76:0/1195724235 wait complete. 2023-11-09T03:46:46.348 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-09T03:46:46.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.449+0000 7f41be1c9640 1 Processor -- start 2023-11-09T03:46:46.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.449+0000 7f41be1c9640 1 -- start start 2023-11-09T03:46:46.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.450+0000 7f41be1c9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41b8157710 0x7f41b8167b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:46.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.450+0000 7f41be1c9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41b815e000 con 0x7f41b80cde60 2023-11-09T03:46:46.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.450+0000 7f41be1c9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41b815e2e0 con 0x7f41b80ce240 2023-11-09T03:46:46.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.450+0000 7f41be1c9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b815e5c0 con 0x7f41b8157710 2023-11-09T03:46:46.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.450+0000 7f41b77fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f41b80cde60 0x7f41b815d560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50630/0 (socket says 172.21.15.76:50630) 2023-11-09T03:46:46.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.450+0000 7f41b77fe640 1 -- 172.21.15.76:0/4235995943 learned_addr learned my addr 172.21.15.76:0/4235995943 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:46.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.451+0000 7f41b6ffd640 1 -- 172.21.15.76:0/4235995943 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1489349568 0 0) 0x7f41b815e000 con 0x7f41b80cde60 2023-11-09T03:46:46.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.451+0000 7f41b7fff640 1 --2- 172.21.15.76:0/4235995943 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41b8157710 0x7f41b8167b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:46.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.451+0000 7f41b6ffd640 1 -- 172.21.15.76:0/4235995943 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41a4003520 con 0x7f41b80cde60 2023-11-09T03:46:46.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.451+0000 7f41b6ffd640 1 -- 172.21.15.76:0/4235995943 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2250716704 0 0) 0x7f41a4003520 con 0x7f41b80cde60 2023-11-09T03:46:46.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.451+0000 7f41b6ffd640 1 -- 172.21.15.76:0/4235995943 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41b8157710 msgr2=0x7f41b8167b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:46.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.451+0000 7f41b6ffd640 1 --2- 172.21.15.76:0/4235995943 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41b8157710 0x7f41b8167b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:46.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.451+0000 7f41b6ffd640 1 -- 172.21.15.76:0/4235995943 >> v1:172.21.15.113:6789/0 conn(0x7f41b80ce240 legacy=0x7f41b8157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:46.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.451+0000 7f41b6ffd640 1 -- 172.21.15.76:0/4235995943 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b80d1120 con 0x7f41b80cde60 2023-11-09T03:46:46.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.451+0000 7f41b6ffd640 1 -- 172.21.15.76:0/4235995943 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f41a8001670 con 0x7f41b80cde60 2023-11-09T03:46:46.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.451+0000 7f41b6ffd640 1 -- 172.21.15.76:0/4235995943 >> v1:172.21.15.76:6789/0 conn(0x7f41b80cde60 legacy=0x7f41b815d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:46.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.451+0000 7f41b6ffd640 1 --2- 172.21.15.76:0/4235995943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41a40039a0 0x7f41a4003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:46.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.451+0000 7f41b6ffd640 1 --2- 172.21.15.76:0/4235995943 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41b8157710 0x7f41a40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:46.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.451+0000 7f41b6ffd640 1 --2- 172.21.15.76:0/4235995943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41a4004d00 0x7f41a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:46.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.451+0000 7f41b6ffd640 1 -- 172.21.15.76:0/4235995943 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b815e5c0 con 0x7f41a4004d00 2023-11-09T03:46:46.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.451+0000 7f41b6ffd640 1 -- 172.21.15.76:0/4235995943 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a400b980 con 0x7f41a40039a0 2023-11-09T03:46:46.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.451+0000 7f41b6ffd640 1 -- 172.21.15.76:0/4235995943 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a400bb40 con 0x7f41b8157710 2023-11-09T03:46:46.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.451+0000 7f41b7fff640 1 --2- 172.21.15.76:0/4235995943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41a4004d00 0x7f41a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:46.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.451+0000 7f41b77fe640 1 --2- 172.21.15.76:0/4235995943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41a40039a0 0x7f41a4003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:46.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.451+0000 7f41aeffd640 1 --2- 172.21.15.76:0/4235995943 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41b8157710 0x7f41a40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:46.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.451+0000 7f41b7fff640 1 -- 172.21.15.76:0/4235995943 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41b8157710 msgr2=0x7f41a40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:46.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.451+0000 7f41b7fff640 1 --2- 172.21.15.76:0/4235995943 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41b8157710 0x7f41a40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:46.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.451+0000 7f41b7fff640 1 -- 172.21.15.76:0/4235995943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41a40039a0 msgr2=0x7f41a4003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:46.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.451+0000 7f41b7fff640 1 --2- 172.21.15.76:0/4235995943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41a40039a0 0x7f41a4003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:46.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.451+0000 7f41b7fff640 1 -- 172.21.15.76:0/4235995943 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a4003520 con 0x7f41a4004d00 2023-11-09T03:46:46.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.452+0000 7f41b7fff640 1 --2- 172.21.15.76:0/4235995943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41a4004d00 0x7f41a4005180 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009fc0 tx=0x7f41a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ad068ea4608034 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:46.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.452+0000 7f41b6ffd640 1 -- 172.21.15.76:0/4235995943 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b002cc20 con 0x7f41a4004d00 2023-11-09T03:46:46.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.452+0000 7f41b6ffd640 1 -- 172.21.15.76:0/4235995943 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b000f040 con 0x7f41a4004d00 2023-11-09T03:46:46.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.452+0000 7f41b6ffd640 1 -- 172.21.15.76:0/4235995943 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b0034440 con 0x7f41a4004d00 2023-11-09T03:46:46.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.453+0000 7f41be1c9640 1 -- 172.21.15.76:0/4235995943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41a4004d00 msgr2=0x7f41a4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:46.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.453+0000 7f41be1c9640 1 --2- 172.21.15.76:0/4235995943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41a4004d00 0x7f41a4005180 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009fc0 tx=0x7f41a40057f0 comp rx=0 tx=0).stop 2023-11-09T03:46:46.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.453+0000 7f41b7fff640 1 -- 172.21.15.76:0/4235995943 reap_dead start 2023-11-09T03:46:46.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.453+0000 7f41be1c9640 1 -- 172.21.15.76:0/4235995943 shutdown_connections 2023-11-09T03:46:46.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.453+0000 7f41be1c9640 1 -- 172.21.15.76:0/4235995943 >> 172.21.15.76:0/4235995943 conn(0x7f41b8071f30 msgr2=0x7f41b80cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:46.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.453+0000 7f41be1c9640 1 -- 172.21.15.76:0/4235995943 shutdown_connections 2023-11-09T03:46:46.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.453+0000 7f41be1c9640 1 -- 172.21.15.76:0/4235995943 wait complete. 2023-11-09T03:46:46.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.454+0000 7f41be1c9640 1 Processor -- start 2023-11-09T03:46:46.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.454+0000 7f41be1c9640 1 -- start start 2023-11-09T03:46:46.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.455+0000 7f41be1c9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41a4004d00 0x7f41b8149700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:46.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.455+0000 7f41be1c9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41b8158a40 0x7f41b81f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:46.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.455+0000 7f41be1c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41b80b78d0 0x7f41b80b7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:46.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.455+0000 7f41be1c9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b80b8290 con 0x7f41b80b78d0 2023-11-09T03:46:46.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.455+0000 7f41be1c9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b814ab20 con 0x7f41a4004d00 2023-11-09T03:46:46.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.455+0000 7f41be1c9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b8149c40 con 0x7f41b8158a40 2023-11-09T03:46:46.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.455+0000 7f41b7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41b80b78d0 0x7f41b80b7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:46.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.455+0000 7f41b7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41b80b78d0 0x7f41b80b7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44066/0 (socket says 172.21.15.76:44066) 2023-11-09T03:46:46.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.455+0000 7f41b7fff640 1 -- 172.21.15.76:0/1752685884 learned_addr learned my addr 172.21.15.76:0/1752685884 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:46.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.455+0000 7f41aeffd640 1 --2- 172.21.15.76:0/1752685884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41b8158a40 0x7f41b81f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:46.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.455+0000 7f41b77fe640 1 --2- 172.21.15.76:0/1752685884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41a4004d00 0x7f41b8149700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:46.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.455+0000 7f41b7fff640 1 -- 172.21.15.76:0/1752685884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41b8158a40 msgr2=0x7f41b81f1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:46.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.455+0000 7f41b7fff640 1 --2- 172.21.15.76:0/1752685884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41b8158a40 0x7f41b81f1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:46.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.455+0000 7f41b7fff640 1 -- 172.21.15.76:0/1752685884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41a4004d00 msgr2=0x7f41b8149700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:46.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.455+0000 7f41b7fff640 1 --2- 172.21.15.76:0/1752685884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41a4004d00 0x7f41b8149700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:46.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.455+0000 7f41b7fff640 1 -- 172.21.15.76:0/1752685884 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b8149f10 con 0x7f41b80b78d0 2023-11-09T03:46:46.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.455+0000 7f41b77fe640 1 --2- 172.21.15.76:0/1752685884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41a4004d00 0x7f41b8149700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:46.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.455+0000 7f41b7fff640 1 --2- 172.21.15.76:0/1752685884 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41b80b78d0 0x7f41b80b7d50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f41b0037040 tx=0x7f41b0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:46.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.456+0000 7f41b4ff9640 1 -- 172.21.15.76:0/1752685884 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b0043070 con 0x7f41b80b78d0 2023-11-09T03:46:46.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.456+0000 7f41b4ff9640 1 -- 172.21.15.76:0/1752685884 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b00349c0 con 0x7f41b80b78d0 2023-11-09T03:46:46.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.456+0000 7f41be1c9640 1 -- 172.21.15.76:0/1752685884 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41b814a1f0 con 0x7f41b80b78d0 2023-11-09T03:46:46.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.456+0000 7f41b4ff9640 1 -- 172.21.15.76:0/1752685884 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b0034d70 con 0x7f41b80b78d0 2023-11-09T03:46:46.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.456+0000 7f41be1c9640 1 -- 172.21.15.76:0/1752685884 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41b814a7b0 con 0x7f41b80b78d0 2023-11-09T03:46:46.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.457+0000 7f41be1c9640 1 -- 172.21.15.76:0/1752685884 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41b80b5230 con 0x7f41b80b78d0 2023-11-09T03:46:46.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.457+0000 7f41b4ff9640 1 -- 172.21.15.76:0/1752685884 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f41b003e440 con 0x7f41b80b78d0 2023-11-09T03:46:46.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.457+0000 7f41b4ff9640 1 --2- 172.21.15.76:0/1752685884 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f419806ae70 0x7f419806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:46.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.458+0000 7f41b4ff9640 1 -- 172.21.15.76:0/1752685884 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f41b00aae90 con 0x7f41b80b78d0 2023-11-09T03:46:46.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.458+0000 7f41b77fe640 1 --2- 172.21.15.76:0/1752685884 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f419806ae70 0x7f419806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:46.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.458+0000 7f41b77fe640 1 --2- 172.21.15.76:0/1752685884 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f419806ae70 0x7f419806d330 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f41a8000c00 tx=0x7f41a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:46.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.462+0000 7f41b4ff9640 1 -- 172.21.15.76:0/1752685884 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f41b0076200 con 0x7f41b80b78d0 2023-11-09T03:46:46.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.692+0000 7f41be1c9640 1 -- 172.21.15.76:0/1752685884 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f41b8074290 con 0x7f41b80b78d0 2023-11-09T03:46:46.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.692+0000 7f41b4ff9640 1 -- 172.21.15.76:0/1752685884 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f41b0039090 con 0x7f41b80b78d0 2023-11-09T03:46:46.693 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:46.693 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:46:46.693 INFO:teuthology.orchestra.run.smithi076.stdout: "auth_dump": [ 2023-11-09T03:46:46.693 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:46:46.693 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.a", 2023-11-09T03:46:46.694 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCgVUxlydtlLxAAi0bnTEOD+fpGL1cYic3caA==", 2023-11-09T03:46:46.694 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:46:46.694 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:46:46.694 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2023-11-09T03:46:46.695 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2023-11-09T03:46:46.695 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:46:46.695 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:46.695 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:46.695 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:46:46.695 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.b", 2023-11-09T03:46:46.696 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxl2+QnABAA7CTWWEbPPXkollRFDlLxPA==", 2023-11-09T03:46:46.696 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:46:46.696 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:46:46.696 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2023-11-09T03:46:46.696 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2023-11-09T03:46:46.697 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:46:46.697 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:46.697 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:46.697 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:46:46.697 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.c", 2023-11-09T03:46:46.697 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCgVUxlrTu9NhAAhg4GpZWZTmVPCP3ffjPo/Q==", 2023-11-09T03:46:46.698 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:46:46.698 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:46:46.698 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2023-11-09T03:46:46.698 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2023-11-09T03:46:46.699 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:46:46.699 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:46.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:46.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:46:46.699 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.d", 2023-11-09T03:46:46.700 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxl4IZ8BxAA8zGFE+1tlHkT5UyzL0nWxQ==", 2023-11-09T03:46:46.700 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:46:46.700 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:46:46.700 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2023-11-09T03:46:46.700 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2023-11-09T03:46:46.700 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:46:46.701 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:46.701 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:46.701 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:46:46.701 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.0", 2023-11-09T03:46:46.701 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCkVUxlXM5LKhAAM253+XEj5YSwmQywwDieTg==", 2023-11-09T03:46:46.701 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:46:46.702 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:46:46.702 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:46:46.702 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:46:46.702 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:46.702 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:46.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:46:46.703 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.1", 2023-11-09T03:46:46.703 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCnVUxlgoDVKBAAkmNFo4E0bAQ1UxkCfQAbjw==", 2023-11-09T03:46:46.703 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:46:46.703 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:46:46.704 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:46:46.704 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:46:46.704 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:46.704 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:46.704 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:46:46.704 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.2", 2023-11-09T03:46:46.705 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCqVUxlzQjVJhAAOoU5IdN86BmcPz71mTq7ng==", 2023-11-09T03:46:46.705 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:46:46.705 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:46:46.705 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:46:46.705 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:46:46.706 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:46.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:46.706 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:46:46.706 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.3", 2023-11-09T03:46:46.706 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCtVUxlhY6VJBAAmCO/ZS8vBa7enyuXX5Y+hw==", 2023-11-09T03:46:46.707 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:46:46.707 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:46:46.707 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:46:46.707 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:46:46.707 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:46.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:46.708 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:46:46.708 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.4", 2023-11-09T03:46:46.708 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCzVUxl79egLxAAnKBOQb/9EQyaHaQt7SawIg==", 2023-11-09T03:46:46.708 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:46:46.709 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:46:46.709 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:46:46.709 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:46:46.709 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:46.709 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:46.709 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:46:46.710 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.5", 2023-11-09T03:46:46.710 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQC2VUxlIfUgLhAAVPurhJCc8cmpt11pQjzL9w==", 2023-11-09T03:46:46.710 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:46:46.710 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:46:46.710 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:46:46.711 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:46:46.711 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:46.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:46.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:46:46.711 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.6", 2023-11-09T03:46:46.711 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQC5VUxlrI58LBAAWxZotpACR4e8vFj4IiaEKA==", 2023-11-09T03:46:46.712 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:46:46.712 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:46:46.712 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:46:46.712 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:46:46.712 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:46.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:46.713 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:46:46.713 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.7", 2023-11-09T03:46:46.713 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQC8VUxl0bA3KxAAwnKNJKqYIcGRLLhQHUaU9w==", 2023-11-09T03:46:46.713 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:46:46.713 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:46:46.714 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:46:46.714 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:46:46.714 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:46.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:46.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:46:46.715 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.0", 2023-11-09T03:46:46.715 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxlHj22CxAACNj5X24hdME+GNTLw4RahQ==", 2023-11-09T03:46:46.715 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:46:46.715 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:46:46.715 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2023-11-09T03:46:46.715 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2023-11-09T03:46:46.716 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2023-11-09T03:46:46.716 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:46.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:46.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:46:46.716 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.1", 2023-11-09T03:46:46.716 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxljXv4FBAAiTOhfIReQWTy6fbujbBrrw==", 2023-11-09T03:46:46.717 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:46:46.717 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:46:46.717 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2023-11-09T03:46:46.717 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2023-11-09T03:46:46.717 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2023-11-09T03:46:46.718 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:46.718 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:46.718 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:46:46.718 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.2", 2023-11-09T03:46:46.718 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxlGVSSERAAQts5odubZ5paZwg8US3QrQ==", 2023-11-09T03:46:46.718 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:46:46.719 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:46:46.719 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2023-11-09T03:46:46.719 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2023-11-09T03:46:46.719 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2023-11-09T03:46:46.719 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:46.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:46.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:46:46.720 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.3", 2023-11-09T03:46:46.720 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxlgmDHGhAA7r98HNQEMnjbqW6RjE5aig==", 2023-11-09T03:46:46.720 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:46:46.720 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:46:46.721 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2023-11-09T03:46:46.721 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2023-11-09T03:46:46.721 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2023-11-09T03:46:46.721 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:46.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:46.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:46:46.722 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.admin", 2023-11-09T03:46:46.722 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCgVUxlV4l8EhAAnrgeLaTaR99vNYGCJjI1TA==", 2023-11-09T03:46:46.722 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:46:46.722 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2023-11-09T03:46:46.722 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2023-11-09T03:46:46.723 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2023-11-09T03:46:46.723 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:46:46.723 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:46.723 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:46.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:46:46.723 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.bootstrap-mds", 2023-11-09T03:46:46.724 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDCVUxlaE/BJBAAGFsUjm4i3WukEJ9dbvaUlg==", 2023-11-09T03:46:46.724 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:46:46.724 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile bootstrap-mds" 2023-11-09T03:46:46.724 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:46.724 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:46.725 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:46:46.725 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.bootstrap-mgr", 2023-11-09T03:46:46.725 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDCVUxlS1fBJBAA0dl74DSbgBjOg3jjydkvPA==", 2023-11-09T03:46:46.725 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:46:46.725 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-09T03:46:46.725 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:46.726 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:46.726 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:46:46.726 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.bootstrap-osd", 2023-11-09T03:46:46.726 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDCVUxlRF7BJBAAE2Q9+8ani5r3r3nx5tQcxQ==", 2023-11-09T03:46:46.726 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:46:46.727 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile bootstrap-osd" 2023-11-09T03:46:46.727 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:46.727 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:46.727 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:46:46.727 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.bootstrap-rbd", 2023-11-09T03:46:46.727 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDCVUxl/mTBJBAA6XhUXJkNKpCJooPzbuENUg==", 2023-11-09T03:46:46.728 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:46:46.728 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-09T03:46:46.728 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:46.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:46.728 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:46:46.729 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-09T03:46:46.729 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDCVUxlGGzBJBAAx01nozHX34XHPSnEQMERMA==", 2023-11-09T03:46:46.729 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:46:46.729 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-09T03:46:46.729 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:46.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:46.730 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:46:46.730 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.bootstrap-rgw", 2023-11-09T03:46:46.730 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDCVUxla3TBJBAAjfqYH8b2bV5lYOxAIQqr0Q==", 2023-11-09T03:46:46.730 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:46:46.730 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-09T03:46:46.731 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:46.731 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:46.731 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:46:46.731 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mgr.x", 2023-11-09T03:46:46.731 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCgVUxl7JryKxAABFx9CRqSbbJu8SIyi3Va8w==", 2023-11-09T03:46:46.731 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:46:46.732 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2023-11-09T03:46:46.732 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile mgr", 2023-11-09T03:46:46.732 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:46:46.732 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:46.732 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:46.732 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:46:46.733 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mgr.y", 2023-11-09T03:46:46.733 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCgVUxlb9L/JxAAu3IOXTn8TGvbzxYVta3yew==", 2023-11-09T03:46:46.733 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:46:46.733 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2023-11-09T03:46:46.733 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile mgr", 2023-11-09T03:46:46.734 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:46:46.734 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:46.734 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:46.734 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2023-11-09T03:46:46.734 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:46:46.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.694+0000 7f41be1c9640 1 -- 172.21.15.76:0/1752685884 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f419806ae70 msgr2=0x7f419806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:46.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.694+0000 7f41be1c9640 1 --2- 172.21.15.76:0/1752685884 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f419806ae70 0x7f419806d330 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f41a8000c00 tx=0x7f41a8009040 comp rx=0 tx=0).stop 2023-11-09T03:46:46.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.694+0000 7f41be1c9640 1 -- 172.21.15.76:0/1752685884 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41b80b78d0 msgr2=0x7f41b80b7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:46.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.694+0000 7f41be1c9640 1 --2- 172.21.15.76:0/1752685884 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41b80b78d0 0x7f41b80b7d50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f41b0037040 tx=0x7f41b0009750 comp rx=0 tx=0).stop 2023-11-09T03:46:46.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.694+0000 7f41be1c9640 1 -- 172.21.15.76:0/1752685884 shutdown_connections 2023-11-09T03:46:46.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.694+0000 7f41be1c9640 1 --2- 172.21.15.76:0/1752685884 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f419806ae70 0x7f419806d330 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:46.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.694+0000 7f41be1c9640 1 --2- 172.21.15.76:0/1752685884 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41b80b78d0 0x7f41b80b7d50 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:46.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.694+0000 7f41be1c9640 1 --2- 172.21.15.76:0/1752685884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41b8158a40 0x7f41b81f1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:46.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.694+0000 7f41be1c9640 1 --2- 172.21.15.76:0/1752685884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41a4004d00 0x7f41b8149700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:46.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.694+0000 7f41be1c9640 1 -- 172.21.15.76:0/1752685884 >> 172.21.15.76:0/1752685884 conn(0x7f41b8071f30 msgr2=0x7f41b80cd130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:46.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.694+0000 7f41be1c9640 1 -- 172.21.15.76:0/1752685884 shutdown_connections 2023-11-09T03:46:46.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.694+0000 7f41be1c9640 1 -- 172.21.15.76:0/1752685884 wait complete. 2023-11-09T03:46:46.738 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-09T03:46:46.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.839+0000 7fcc3355a640 1 Processor -- start 2023-11-09T03:46:46.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.839+0000 7fcc3355a640 1 -- start start 2023-11-09T03:46:46.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.840+0000 7fcc3355a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc2c15d1b0 0x7fcc2c0cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:46.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.840+0000 7fcc3355a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcc2c066000 0x7fcc2c06a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:46.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.840+0000 7fcc3355a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc2c0c9bf0 con 0x7fcc2c070ff0 2023-11-09T03:46:46.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.840+0000 7fcc3355a640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc2c0cd890 con 0x7fcc2c066000 2023-11-09T03:46:46.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.840+0000 7fcc3355a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c0cdb30 con 0x7fcc2c15d1b0 2023-11-09T03:46:46.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.840+0000 7fcc31af7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcc2c066000 0x7fcc2c06a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:46.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.840+0000 7fcc31af7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcc2c066000 0x7fcc2c06a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44070/0 (socket says 172.21.15.76:44070) 2023-11-09T03:46:46.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.840+0000 7fcc31af7640 1 -- 172.21.15.76:0/4203722039 learned_addr learned my addr 172.21.15.76:0/4203722039 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:46.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.840+0000 7fcc30af5640 1 --2- 172.21.15.76:0/4203722039 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc2c15d1b0 0x7fcc2c0cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:46.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.841+0000 7fcc31af7640 1 -- 172.21.15.76:0/4203722039 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc2c15d1b0 msgr2=0x7fcc2c0cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:46.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.841+0000 7fcc31af7640 1 --2- 172.21.15.76:0/4203722039 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc2c15d1b0 0x7fcc2c0cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:46.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.841+0000 7fcc31af7640 1 -- 172.21.15.76:0/4203722039 >> v1:172.21.15.113:6789/0 conn(0x7fcc2c070ff0 legacy=0x7fcc2c0713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:46.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.841+0000 7fcc31af7640 1 -- 172.21.15.76:0/4203722039 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2c0cde10 con 0x7fcc2c066000 2023-11-09T03:46:46.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.841+0000 7fcc31af7640 1 --2- 172.21.15.76:0/4203722039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc2c066000 0x7fcc2c06a500 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009870 tx=0x7fcc2402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00ab3061cb7379f server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:46.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.841+0000 7fcc23fff640 1 -- 172.21.15.76:0/4203722039 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc24030840 con 0x7fcc2c066000 2023-11-09T03:46:46.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.841+0000 7fcc23fff640 1 -- 172.21.15.76:0/4203722039 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc240309a0 con 0x7fcc2c066000 2023-11-09T03:46:46.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.842+0000 7fcc23fff640 1 -- 172.21.15.76:0/4203722039 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc24030d30 con 0x7fcc2c066000 2023-11-09T03:46:46.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.842+0000 7fcc3355a640 1 -- 172.21.15.76:0/4203722039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc2c066000 msgr2=0x7fcc2c06a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:46.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.842+0000 7fcc3355a640 1 --2- 172.21.15.76:0/4203722039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc2c066000 0x7fcc2c06a500 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009870 tx=0x7fcc2402b980 comp rx=0 tx=0).stop 2023-11-09T03:46:46.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.842+0000 7fcc3355a640 1 -- 172.21.15.76:0/4203722039 shutdown_connections 2023-11-09T03:46:46.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.842+0000 7fcc3355a640 1 --2- 172.21.15.76:0/4203722039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc2c066000 0x7fcc2c06a500 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:46.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.842+0000 7fcc3355a640 1 --2- 172.21.15.76:0/4203722039 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc2c15d1b0 0x7fcc2c0cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:46.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.842+0000 7fcc3355a640 1 -- 172.21.15.76:0/4203722039 >> 172.21.15.76:0/4203722039 conn(0x7fcc2c0cbc90 msgr2=0x7fcc2c0cc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:46.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.842+0000 7fcc3355a640 1 -- 172.21.15.76:0/4203722039 shutdown_connections 2023-11-09T03:46:46.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.842+0000 7fcc3355a640 1 -- 172.21.15.76:0/4203722039 wait complete. 2023-11-09T03:46:46.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.843+0000 7fcc3355a640 1 Processor -- start 2023-11-09T03:46:46.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.843+0000 7fcc3355a640 1 -- start start 2023-11-09T03:46:46.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.844+0000 7fcc3355a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc2c066000 0x7fcc2c0c3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:46.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.844+0000 7fcc3355a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc2c070ff0 0x7fcc2c0c4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:46.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.844+0000 7fcc3355a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc2c15d1b0 0x7fcc2c15f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:46.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.844+0000 7fcc3355a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c0c3d30 con 0x7fcc2c070ff0 2023-11-09T03:46:46.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.844+0000 7fcc3355a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c0c4010 con 0x7fcc2c066000 2023-11-09T03:46:46.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.844+0000 7fcc3355a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c0c42f0 con 0x7fcc2c15d1b0 2023-11-09T03:46:46.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.844+0000 7fcc30af5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc2c070ff0 0x7fcc2c0c4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:46.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.844+0000 7fcc30af5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc2c070ff0 0x7fcc2c0c4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44080/0 (socket says 172.21.15.76:44080) 2023-11-09T03:46:46.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.844+0000 7fcc30af5640 1 -- 172.21.15.76:0/3596038134 learned_addr learned my addr 172.21.15.76:0/3596038134 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:46.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.844+0000 7fcc31af7640 1 --2- 172.21.15.76:0/3596038134 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc2c15d1b0 0x7fcc2c15f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:46.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.844+0000 7fcc312f6640 1 --2- 172.21.15.76:0/3596038134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc2c066000 0x7fcc2c0c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:46.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.844+0000 7fcc30af5640 1 -- 172.21.15.76:0/3596038134 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc2c15d1b0 msgr2=0x7fcc2c15f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:46.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.844+0000 7fcc30af5640 1 --2- 172.21.15.76:0/3596038134 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc2c15d1b0 0x7fcc2c15f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:46.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.844+0000 7fcc30af5640 1 -- 172.21.15.76:0/3596038134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc2c066000 msgr2=0x7fcc2c0c3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:46.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.844+0000 7fcc30af5640 1 --2- 172.21.15.76:0/3596038134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc2c066000 0x7fcc2c0c3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:46.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.844+0000 7fcc30af5640 1 -- 172.21.15.76:0/3596038134 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc240094d0 con 0x7fcc2c070ff0 2023-11-09T03:46:46.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.845+0000 7fcc30af5640 1 --2- 172.21.15.76:0/3596038134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc2c070ff0 0x7fcc2c0c4b30 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00d800 tx=0x7fcc1c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:46.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.845+0000 7fcc21ffb640 1 -- 172.21.15.76:0/3596038134 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c00b6c0 con 0x7fcc2c070ff0 2023-11-09T03:46:46.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.845+0000 7fcc21ffb640 1 -- 172.21.15.76:0/3596038134 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c00b820 con 0x7fcc2c070ff0 2023-11-09T03:46:46.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.845+0000 7fcc21ffb640 1 -- 172.21.15.76:0/3596038134 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c00bbb0 con 0x7fcc2c070ff0 2023-11-09T03:46:46.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.845+0000 7fcc3355a640 1 -- 172.21.15.76:0/3596038134 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc2c0ac930 con 0x7fcc2c070ff0 2023-11-09T03:46:46.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.845+0000 7fcc3355a640 1 -- 172.21.15.76:0/3596038134 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc2c0c5070 con 0x7fcc2c070ff0 2023-11-09T03:46:46.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.846+0000 7fcc3355a640 1 -- 172.21.15.76:0/3596038134 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc2c074290 con 0x7fcc2c070ff0 2023-11-09T03:46:46.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.846+0000 7fcc21ffb640 1 -- 172.21.15.76:0/3596038134 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcc1c00bd10 con 0x7fcc2c070ff0 2023-11-09T03:46:46.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.846+0000 7fcc21ffb640 1 --2- 172.21.15.76:0/3596038134 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcc1006ae70 0x7fcc1006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:46.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.847+0000 7fcc21ffb640 1 -- 172.21.15.76:0/3596038134 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fcc1c089790 con 0x7fcc2c070ff0 2023-11-09T03:46:46.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.847+0000 7fcc312f6640 1 --2- 172.21.15.76:0/3596038134 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcc1006ae70 0x7fcc1006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:46.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.847+0000 7fcc312f6640 1 --2- 172.21.15.76:0/3596038134 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcc1006ae70 0x7fcc1006d330 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcc280012e0 tx=0x7fcc28009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:46.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:46.851+0000 7fcc21ffb640 1 -- 172.21.15.76:0/3596038134 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcc1c054b90 con 0x7fcc2c070ff0 2023-11-09T03:46:47.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.080+0000 7fcc3355a640 1 -- 172.21.15.76:0/3596038134 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fcc2c0b5190 con 0x7fcc2c070ff0 2023-11-09T03:46:47.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.082+0000 7fcc21ffb640 1 -- 172.21.15.76:0/3596038134 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fcc1c0581b0 con 0x7fcc2c070ff0 2023-11-09T03:46:47.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.084+0000 7fcc3355a640 1 -- 172.21.15.76:0/3596038134 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcc1006ae70 msgr2=0x7fcc1006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:47.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.084+0000 7fcc3355a640 1 --2- 172.21.15.76:0/3596038134 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcc1006ae70 0x7fcc1006d330 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcc280012e0 tx=0x7fcc28009040 comp rx=0 tx=0).stop 2023-11-09T03:46:47.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.084+0000 7fcc3355a640 1 -- 172.21.15.76:0/3596038134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc2c070ff0 msgr2=0x7fcc2c0c4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:47.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.084+0000 7fcc3355a640 1 --2- 172.21.15.76:0/3596038134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc2c070ff0 0x7fcc2c0c4b30 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00d800 tx=0x7fcc1c00dcc0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.084+0000 7fcc3355a640 1 -- 172.21.15.76:0/3596038134 shutdown_connections 2023-11-09T03:46:47.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.084+0000 7fcc3355a640 1 --2- 172.21.15.76:0/3596038134 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcc1006ae70 0x7fcc1006d330 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.084+0000 7fcc3355a640 1 --2- 172.21.15.76:0/3596038134 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc2c15d1b0 0x7fcc2c15f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.084+0000 7fcc3355a640 1 --2- 172.21.15.76:0/3596038134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc2c070ff0 0x7fcc2c0c4b30 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.084+0000 7fcc3355a640 1 --2- 172.21.15.76:0/3596038134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc2c066000 0x7fcc2c0c3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.084+0000 7fcc3355a640 1 -- 172.21.15.76:0/3596038134 >> 172.21.15.76:0/3596038134 conn(0x7fcc2c0cbc90 msgr2=0x7fcc2c160830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:47.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.084+0000 7fcc3355a640 1 -- 172.21.15.76:0/3596038134 shutdown_connections 2023-11-09T03:46:47.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.084+0000 7fcc3355a640 1 -- 172.21.15.76:0/3596038134 wait complete. 2023-11-09T03:46:47.095 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-09T03:46:47.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.195+0000 7ffb3556c640 1 Processor -- start 2023-11-09T03:46:47.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.195+0000 7ffb3556c640 1 -- start start 2023-11-09T03:46:47.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.196+0000 7ffb3556c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffb30153e60 0x7ffb3015dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:47.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.196+0000 7ffb3556c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffb300b4880 0x7ffb300b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:47.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.196+0000 7ffb3556c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb300cde60 con 0x7ffb300cd6d0 2023-11-09T03:46:47.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.196+0000 7ffb3556c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb30160f90 con 0x7ffb30153e60 2023-11-09T03:46:47.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.196+0000 7ffb3556c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb300b5200 con 0x7ffb300b4880 2023-11-09T03:46:47.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.196+0000 7ffb2effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffb30153e60 0x7ffb3015dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:47.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.196+0000 7ffb2e7fc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ffb300cd6d0 0x7ffb30161420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50640/0 (socket says 172.21.15.76:50640) 2023-11-09T03:46:47.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.196+0000 7ffb2e7fc640 1 -- 172.21.15.76:0/3055191059 learned_addr learned my addr 172.21.15.76:0/3055191059 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:47.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.196+0000 7ffb2f7fe640 1 --2- 172.21.15.76:0/3055191059 >> v2:172.21.15.113:3300/0 conn(0x7ffb300b4880 0x7ffb300b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:47.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.196+0000 7ffb2effd640 1 -- 172.21.15.76:0/3055191059 >> v2:172.21.15.113:3300/0 conn(0x7ffb300b4880 msgr2=0x7ffb300b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:47.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.196+0000 7ffb2effd640 1 --2- 172.21.15.76:0/3055191059 >> v2:172.21.15.113:3300/0 conn(0x7ffb300b4880 0x7ffb300b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.196+0000 7ffb2effd640 1 -- 172.21.15.76:0/3055191059 >> v1:172.21.15.76:6789/0 conn(0x7ffb300cd6d0 legacy=0x7ffb30161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:47.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.196+0000 7ffb2effd640 1 -- 172.21.15.76:0/3055191059 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3015e400 con 0x7ffb30153e60 2023-11-09T03:46:47.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.196+0000 7ffb2dffb640 1 -- 172.21.15.76:0/3055191059 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 387503389 0 0) 0x7ffb300cde60 con 0x7ffb300cd6d0 2023-11-09T03:46:47.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.197+0000 7ffb2effd640 1 --2- 172.21.15.76:0/3055191059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb30153e60 0x7ffb3015dec0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ffb300b8f40 tx=0x7ffb2002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85aaae6dfa9b0a73 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:47.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.197+0000 7ffb2dffb640 1 -- 172.21.15.76:0/3055191059 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2002cc80 con 0x7ffb30153e60 2023-11-09T03:46:47.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.197+0000 7ffb2dffb640 1 -- 172.21.15.76:0/3055191059 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2002cde0 con 0x7ffb30153e60 2023-11-09T03:46:47.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.197+0000 7ffb2dffb640 1 -- 172.21.15.76:0/3055191059 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb20034660 con 0x7ffb30153e60 2023-11-09T03:46:47.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.198+0000 7ffb3556c640 1 -- 172.21.15.76:0/3055191059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb30153e60 msgr2=0x7ffb3015dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:47.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.198+0000 7ffb3556c640 1 --2- 172.21.15.76:0/3055191059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb30153e60 0x7ffb3015dec0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ffb300b8f40 tx=0x7ffb2002bed0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.198+0000 7ffb3556c640 1 -- 172.21.15.76:0/3055191059 shutdown_connections 2023-11-09T03:46:47.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.198+0000 7ffb3556c640 1 --2- 172.21.15.76:0/3055191059 >> v2:172.21.15.113:3300/0 conn(0x7ffb300b4880 0x7ffb300b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.198+0000 7ffb3556c640 1 --2- 172.21.15.76:0/3055191059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb30153e60 0x7ffb3015dec0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.198+0000 7ffb3556c640 1 -- 172.21.15.76:0/3055191059 >> 172.21.15.76:0/3055191059 conn(0x7ffb300cf880 msgr2=0x7ffb300cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:47.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.198+0000 7ffb3556c640 1 -- 172.21.15.76:0/3055191059 shutdown_connections 2023-11-09T03:46:47.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.198+0000 7ffb3556c640 1 -- 172.21.15.76:0/3055191059 wait complete. 2023-11-09T03:46:47.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.199+0000 7ffb3556c640 1 Processor -- start 2023-11-09T03:46:47.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.199+0000 7ffb3556c640 1 -- start start 2023-11-09T03:46:47.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.200+0000 7ffb3556c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb300b4880 0x7ffb3015c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:47.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.200+0000 7ffb3556c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb300cd6d0 0x7ffb300c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:47.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.200+0000 7ffb3556c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb30153e60 0x7ffb300cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:47.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.200+0000 7ffb3556c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb300cabf0 con 0x7ffb300b4880 2023-11-09T03:46:47.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.200+0000 7ffb3556c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb300cc4a0 con 0x7ffb30153e60 2023-11-09T03:46:47.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.200+0000 7ffb3556c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb30157c80 con 0x7ffb300cd6d0 2023-11-09T03:46:47.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.200+0000 7ffb2effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb300b4880 0x7ffb3015c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:47.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.200+0000 7ffb2effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb300b4880 0x7ffb3015c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44108/0 (socket says 172.21.15.76:44108) 2023-11-09T03:46:47.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.200+0000 7ffb2effd640 1 -- 172.21.15.76:0/2081873390 learned_addr learned my addr 172.21.15.76:0/2081873390 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:47.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.200+0000 7ffb2effd640 1 -- 172.21.15.76:0/2081873390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb300cd6d0 msgr2=0x7ffb300c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:47.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.200+0000 7ffb2effd640 1 --2- 172.21.15.76:0/2081873390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb300cd6d0 0x7ffb300c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.200+0000 7ffb2f7fe640 1 --2- 172.21.15.76:0/2081873390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb30153e60 0x7ffb300cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:47.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.200+0000 7ffb2effd640 1 -- 172.21.15.76:0/2081873390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb30153e60 msgr2=0x7ffb300cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:47.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.200+0000 7ffb2effd640 1 --2- 172.21.15.76:0/2081873390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb30153e60 0x7ffb300cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.200+0000 7ffb2effd640 1 -- 172.21.15.76:0/2081873390 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb20009a70 con 0x7ffb300b4880 2023-11-09T03:46:47.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.200+0000 7ffb2effd640 1 --2- 172.21.15.76:0/2081873390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb300b4880 0x7ffb3015c380 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ffb20009e80 tx=0x7ffb20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:47.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.200+0000 7ffb177fe640 1 -- 172.21.15.76:0/2081873390 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb20040650 con 0x7ffb300b4880 2023-11-09T03:46:47.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.200+0000 7ffb177fe640 1 -- 172.21.15.76:0/2081873390 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb200407f0 con 0x7ffb300b4880 2023-11-09T03:46:47.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.200+0000 7ffb3556c640 1 -- 172.21.15.76:0/2081873390 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb3015c8c0 con 0x7ffb300b4880 2023-11-09T03:46:47.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.200+0000 7ffb177fe640 1 -- 172.21.15.76:0/2081873390 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb20040ba0 con 0x7ffb300b4880 2023-11-09T03:46:47.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.200+0000 7ffb3556c640 1 -- 172.21.15.76:0/2081873390 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb300ca250 con 0x7ffb300b4880 2023-11-09T03:46:47.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.201+0000 7ffb3556c640 1 -- 172.21.15.76:0/2081873390 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb30074290 con 0x7ffb300b4880 2023-11-09T03:46:47.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.202+0000 7ffb177fe640 1 -- 172.21.15.76:0/2081873390 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffb20040d00 con 0x7ffb300b4880 2023-11-09T03:46:47.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.202+0000 7ffb177fe640 1 --2- 172.21.15.76:0/2081873390 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ffb1c06ae70 0x7ffb1c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:47.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.202+0000 7ffb177fe640 1 -- 172.21.15.76:0/2081873390 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7ffb200ac310 con 0x7ffb300b4880 2023-11-09T03:46:47.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.203+0000 7ffb2e7fc640 1 --2- 172.21.15.76:0/2081873390 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ffb1c06ae70 0x7ffb1c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:47.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.203+0000 7ffb2e7fc640 1 --2- 172.21.15.76:0/2081873390 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ffb1c06ae70 0x7ffb1c06d330 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ffb24007fd0 tx=0x7ffb24007920 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:47.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.206+0000 7ffb177fe640 1 -- 172.21.15.76:0/2081873390 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffb200774c0 con 0x7ffb300b4880 2023-11-09T03:46:47.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.438+0000 7ffb3556c640 1 -- 172.21.15.76:0/2081873390 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7ffb300d0b80 con 0x7ffb300b4880 2023-11-09T03:46:47.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.440+0000 7ffb177fe640 1 -- 172.21.15.76:0/2081873390 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7ffb2007aae0 con 0x7ffb300b4880 2023-11-09T03:46:47.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.442+0000 7ffb3556c640 1 -- 172.21.15.76:0/2081873390 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ffb1c06ae70 msgr2=0x7ffb1c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:47.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.442+0000 7ffb3556c640 1 --2- 172.21.15.76:0/2081873390 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ffb1c06ae70 0x7ffb1c06d330 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ffb24007fd0 tx=0x7ffb24007920 comp rx=0 tx=0).stop 2023-11-09T03:46:47.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.442+0000 7ffb3556c640 1 -- 172.21.15.76:0/2081873390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb300b4880 msgr2=0x7ffb3015c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:47.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.442+0000 7ffb3556c640 1 --2- 172.21.15.76:0/2081873390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb300b4880 0x7ffb3015c380 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ffb20009e80 tx=0x7ffb20004a40 comp rx=0 tx=0).stop 2023-11-09T03:46:47.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.442+0000 7ffb3556c640 1 -- 172.21.15.76:0/2081873390 shutdown_connections 2023-11-09T03:46:47.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.442+0000 7ffb3556c640 1 --2- 172.21.15.76:0/2081873390 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ffb1c06ae70 0x7ffb1c06d330 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.442+0000 7ffb3556c640 1 --2- 172.21.15.76:0/2081873390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb30153e60 0x7ffb300cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.442+0000 7ffb3556c640 1 --2- 172.21.15.76:0/2081873390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb300cd6d0 0x7ffb300c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.442+0000 7ffb3556c640 1 --2- 172.21.15.76:0/2081873390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb300b4880 0x7ffb3015c380 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.442+0000 7ffb3556c640 1 -- 172.21.15.76:0/2081873390 >> 172.21.15.76:0/2081873390 conn(0x7ffb300cf880 msgr2=0x7ffb300d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:47.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.442+0000 7ffb3556c640 1 -- 172.21.15.76:0/2081873390 shutdown_connections 2023-11-09T03:46:47.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.442+0000 7ffb3556c640 1 -- 172.21.15.76:0/2081873390 wait complete. 2023-11-09T03:46:47.453 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-09T03:46:47.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.553+0000 7efc67955640 1 Processor -- start 2023-11-09T03:46:47.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.553+0000 7efc67955640 1 -- start start 2023-11-09T03:46:47.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.554+0000 7efc67955640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc60159c00 0x7efc600cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:47.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.554+0000 7efc67955640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7efc600c48e0 0x7efc600c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:47.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.554+0000 7efc67955640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc6015aff0 con 0x7efc600adac0 2023-11-09T03:46:47.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.554+0000 7efc67955640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc600c52a0 con 0x7efc600c48e0 2023-11-09T03:46:47.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.554+0000 7efc67955640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc6015a3e0 con 0x7efc60159c00 2023-11-09T03:46:47.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.554+0000 7efc64ef0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7efc600c48e0 0x7efc600c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:47.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.554+0000 7efc65ef2640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7efc600adac0 0x7efc600adef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50642/0 (socket says 172.21.15.76:50642) 2023-11-09T03:46:47.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.554+0000 7efc65ef2640 1 -- 172.21.15.76:0/3647022215 learned_addr learned my addr 172.21.15.76:0/3647022215 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:47.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.554+0000 7efc64ef0640 1 --2- 172.21.15.76:0/3647022215 >> v2:172.21.15.76:3300/0 conn(0x7efc600c48e0 0x7efc600c4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44118/0 (socket says 172.21.15.76:44118) 2023-11-09T03:46:47.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.555+0000 7efc64ef0640 1 -- 172.21.15.76:0/3647022215 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc60159c00 msgr2=0x7efc600cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:47.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.555+0000 7efc656f1640 1 --2- 172.21.15.76:0/3647022215 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc60159c00 0x7efc600cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:47.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.555+0000 7efc64ef0640 1 --2- 172.21.15.76:0/3647022215 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc60159c00 0x7efc600cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.555+0000 7efc64ef0640 1 -- 172.21.15.76:0/3647022215 >> v1:172.21.15.76:6789/0 conn(0x7efc600adac0 legacy=0x7efc600adef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:47.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.555+0000 7efc57fff640 1 -- 172.21.15.76:0/3647022215 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 966470959 0 0) 0x7efc6015aff0 con 0x7efc600adac0 2023-11-09T03:46:47.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.555+0000 7efc64ef0640 1 -- 172.21.15.76:0/3647022215 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc6015a6c0 con 0x7efc600c48e0 2023-11-09T03:46:47.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.555+0000 7efc656f1640 1 --2- 172.21.15.76:0/3647022215 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc60159c00 0x7efc600cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:47.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.555+0000 7efc64ef0640 1 --2- 172.21.15.76:0/3647022215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc600c48e0 0x7efc600c4d60 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7efc5000ba40 tx=0x7efc5002e080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580999d65c805ad server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:47.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.555+0000 7efc57fff640 1 -- 172.21.15.76:0/3647022215 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5002ee30 con 0x7efc600c48e0 2023-11-09T03:46:47.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.556+0000 7efc57fff640 1 -- 172.21.15.76:0/3647022215 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc50036440 con 0x7efc600c48e0 2023-11-09T03:46:47.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.556+0000 7efc57fff640 1 -- 172.21.15.76:0/3647022215 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc500367d0 con 0x7efc600c48e0 2023-11-09T03:46:47.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.556+0000 7efc67955640 1 -- 172.21.15.76:0/3647022215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc600c48e0 msgr2=0x7efc600c4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:47.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.556+0000 7efc67955640 1 --2- 172.21.15.76:0/3647022215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc600c48e0 0x7efc600c4d60 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7efc5000ba40 tx=0x7efc5002e080 comp rx=0 tx=0).stop 2023-11-09T03:46:47.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.556+0000 7efc67955640 1 -- 172.21.15.76:0/3647022215 shutdown_connections 2023-11-09T03:46:47.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.556+0000 7efc67955640 1 --2- 172.21.15.76:0/3647022215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc600c48e0 0x7efc600c4d60 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.556+0000 7efc67955640 1 --2- 172.21.15.76:0/3647022215 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc60159c00 0x7efc600cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.556+0000 7efc67955640 1 -- 172.21.15.76:0/3647022215 >> 172.21.15.76:0/3647022215 conn(0x7efc600ce950 msgr2=0x7efc600ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:47.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.556+0000 7efc67955640 1 -- 172.21.15.76:0/3647022215 shutdown_connections 2023-11-09T03:46:47.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.556+0000 7efc67955640 1 -- 172.21.15.76:0/3647022215 wait complete. 2023-11-09T03:46:47.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.557+0000 7efc67955640 1 Processor -- start 2023-11-09T03:46:47.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.557+0000 7efc67955640 1 -- start start 2023-11-09T03:46:47.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.558+0000 7efc67955640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc600adac0 0x7efc600c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:47.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.558+0000 7efc67955640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc600c48e0 0x7efc601f5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:47.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.558+0000 7efc67955640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc60159c00 0x7efc601f60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:47.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.558+0000 7efc67955640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc600b30a0 con 0x7efc600c48e0 2023-11-09T03:46:47.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.558+0000 7efc67955640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc601f3180 con 0x7efc600adac0 2023-11-09T03:46:47.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.558+0000 7efc67955640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc601f3460 con 0x7efc60159c00 2023-11-09T03:46:47.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.558+0000 7efc64ef0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc600c48e0 0x7efc601f5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:47.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.558+0000 7efc64ef0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc600c48e0 0x7efc601f5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44130/0 (socket says 172.21.15.76:44130) 2023-11-09T03:46:47.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.558+0000 7efc64ef0640 1 -- 172.21.15.76:0/1086326512 learned_addr learned my addr 172.21.15.76:0/1086326512 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:47.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.558+0000 7efc65ef2640 1 --2- 172.21.15.76:0/1086326512 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc60159c00 0x7efc601f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:47.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.558+0000 7efc656f1640 1 --2- 172.21.15.76:0/1086326512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc600adac0 0x7efc600c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:47.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.558+0000 7efc64ef0640 1 -- 172.21.15.76:0/1086326512 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc60159c00 msgr2=0x7efc601f60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:47.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.558+0000 7efc64ef0640 1 --2- 172.21.15.76:0/1086326512 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc60159c00 0x7efc601f60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.558+0000 7efc64ef0640 1 -- 172.21.15.76:0/1086326512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc600adac0 msgr2=0x7efc600c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:47.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.558+0000 7efc64ef0640 1 --2- 172.21.15.76:0/1086326512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc600adac0 0x7efc600c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.558+0000 7efc64ef0640 1 -- 172.21.15.76:0/1086326512 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5000b6a0 con 0x7efc600c48e0 2023-11-09T03:46:47.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.559+0000 7efc64ef0640 1 --2- 172.21.15.76:0/1086326512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc600c48e0 0x7efc601f5b90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7efc50000c00 tx=0x7efc50004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:47.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.559+0000 7efc55ffb640 1 -- 172.21.15.76:0/1086326512 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50004230 con 0x7efc600c48e0 2023-11-09T03:46:47.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.559+0000 7efc55ffb640 1 -- 172.21.15.76:0/1086326512 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc50004390 con 0x7efc600c48e0 2023-11-09T03:46:47.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.559+0000 7efc67955640 1 -- 172.21.15.76:0/1086326512 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc600b2a50 con 0x7efc600c48e0 2023-11-09T03:46:47.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.559+0000 7efc55ffb640 1 -- 172.21.15.76:0/1086326512 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50034ba0 con 0x7efc600c48e0 2023-11-09T03:46:47.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.559+0000 7efc67955640 1 -- 172.21.15.76:0/1086326512 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc601f6970 con 0x7efc600c48e0 2023-11-09T03:46:47.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.560+0000 7efc67955640 1 -- 172.21.15.76:0/1086326512 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc600b3250 con 0x7efc600c48e0 2023-11-09T03:46:47.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.560+0000 7efc55ffb640 1 -- 172.21.15.76:0/1086326512 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efc5003d040 con 0x7efc600c48e0 2023-11-09T03:46:47.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.561+0000 7efc55ffb640 1 --2- 172.21.15.76:0/1086326512 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efc4406aea0 0x7efc4406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:47.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.561+0000 7efc55ffb640 1 -- 172.21.15.76:0/1086326512 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7efc500ac3a0 con 0x7efc600c48e0 2023-11-09T03:46:47.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.561+0000 7efc656f1640 1 --2- 172.21.15.76:0/1086326512 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efc4406aea0 0x7efc4406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:47.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.561+0000 7efc656f1640 1 --2- 172.21.15.76:0/1086326512 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efc4406aea0 0x7efc4406d360 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efc5c007510 tx=0x7efc5c0074a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:47.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.565+0000 7efc55ffb640 1 -- 172.21.15.76:0/1086326512 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efc600b3250 con 0x7efc600c48e0 2023-11-09T03:46:47.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.796+0000 7efc67955640 1 -- 172.21.15.76:0/1086326512 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7efc60074290 con 0x7efc600c48e0 2023-11-09T03:46:47.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.798+0000 7efc55ffb640 1 -- 172.21.15.76:0/1086326512 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7efc50077660 con 0x7efc600c48e0 2023-11-09T03:46:47.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.800+0000 7efc67955640 1 -- 172.21.15.76:0/1086326512 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efc4406aea0 msgr2=0x7efc4406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:47.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.800+0000 7efc67955640 1 --2- 172.21.15.76:0/1086326512 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efc4406aea0 0x7efc4406d360 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efc5c007510 tx=0x7efc5c0074a0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.800+0000 7efc67955640 1 -- 172.21.15.76:0/1086326512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc600c48e0 msgr2=0x7efc601f5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:47.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.800+0000 7efc67955640 1 --2- 172.21.15.76:0/1086326512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc600c48e0 0x7efc601f5b90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7efc50000c00 tx=0x7efc50004730 comp rx=0 tx=0).stop 2023-11-09T03:46:47.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.800+0000 7efc67955640 1 -- 172.21.15.76:0/1086326512 shutdown_connections 2023-11-09T03:46:47.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.800+0000 7efc67955640 1 --2- 172.21.15.76:0/1086326512 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efc4406aea0 0x7efc4406d360 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.800+0000 7efc67955640 1 --2- 172.21.15.76:0/1086326512 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc60159c00 0x7efc601f60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.800+0000 7efc67955640 1 --2- 172.21.15.76:0/1086326512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc600c48e0 0x7efc601f5b90 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.800+0000 7efc67955640 1 --2- 172.21.15.76:0/1086326512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc600adac0 0x7efc600c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.800+0000 7efc67955640 1 -- 172.21.15.76:0/1086326512 >> 172.21.15.76:0/1086326512 conn(0x7efc600ce950 msgr2=0x7efc600cd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:47.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.801+0000 7efc67955640 1 -- 172.21.15.76:0/1086326512 shutdown_connections 2023-11-09T03:46:47.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.801+0000 7efc67955640 1 -- 172.21.15.76:0/1086326512 wait complete. 2023-11-09T03:46:47.811 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-09T03:46:47.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.912+0000 7f8f8dada640 1 Processor -- start 2023-11-09T03:46:47.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.912+0000 7f8f8dada640 1 -- start start 2023-11-09T03:46:47.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.913+0000 7f8f8dada640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f880c4ff0 0x7f8f880cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:47.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.913+0000 7f8f8dada640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8f880adac0 0x7f8f880adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:47.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.913+0000 7f8f8dada640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f8815aff0 con 0x7f8f88159c00 2023-11-09T03:46:47.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.913+0000 7f8f8dada640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f880ae450 con 0x7f8f880adac0 2023-11-09T03:46:47.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.913+0000 7f8f8dada640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8815a3e0 con 0x7f8f880c4ff0 2023-11-09T03:46:47.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.913+0000 7f8f87fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8f880adac0 0x7f8f880adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:47.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.913+0000 7f8f877fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8f88159c00 0x7f8f880c48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50654/0 (socket says 172.21.15.76:50654) 2023-11-09T03:46:47.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.913+0000 7f8f877fe640 1 -- 172.21.15.76:0/840958531 learned_addr learned my addr 172.21.15.76:0/840958531 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:47.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.913+0000 7f8f87fff640 1 -- 172.21.15.76:0/840958531 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f880c4ff0 msgr2=0x7f8f880cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:47.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.913+0000 7f8f87fff640 1 --2- 172.21.15.76:0/840958531 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f880c4ff0 0x7f8f880cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.913+0000 7f8f87fff640 1 -- 172.21.15.76:0/840958531 >> v1:172.21.15.76:6789/0 conn(0x7f8f88159c00 legacy=0x7f8f880c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:47.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.913+0000 7f8f87fff640 1 -- 172.21.15.76:0/840958531 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f8815a6c0 con 0x7f8f880adac0 2023-11-09T03:46:47.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.913+0000 7f8f867fc640 1 -- 172.21.15.76:0/840958531 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2850617074 0 0) 0x7f8f8815aff0 con 0x7f8f88159c00 2023-11-09T03:46:47.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.914+0000 7f8f87fff640 1 --2- 172.21.15.76:0/840958531 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f880adac0 0x7f8f880adf10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000bfa0 tx=0x7f8f8002e0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79d9005944231ce server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:47.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.914+0000 7f8f867fc640 1 -- 172.21.15.76:0/840958531 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8002ea40 con 0x7f8f880adac0 2023-11-09T03:46:47.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.914+0000 7f8f867fc640 1 -- 172.21.15.76:0/840958531 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8002eba0 con 0x7f8f880adac0 2023-11-09T03:46:47.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.914+0000 7f8f867fc640 1 -- 172.21.15.76:0/840958531 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f80009540 con 0x7f8f880adac0 2023-11-09T03:46:47.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.914+0000 7f8f8dada640 1 -- 172.21.15.76:0/840958531 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f880adac0 msgr2=0x7f8f880adf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:47.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.914+0000 7f8f8dada640 1 --2- 172.21.15.76:0/840958531 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f880adac0 0x7f8f880adf10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000bfa0 tx=0x7f8f8002e0a0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.915+0000 7f8f8dada640 1 -- 172.21.15.76:0/840958531 shutdown_connections 2023-11-09T03:46:47.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.915+0000 7f8f8dada640 1 --2- 172.21.15.76:0/840958531 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f880adac0 0x7f8f880adf10 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.915+0000 7f8f8dada640 1 --2- 172.21.15.76:0/840958531 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f880c4ff0 0x7f8f880cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.915+0000 7f8f8dada640 1 -- 172.21.15.76:0/840958531 >> 172.21.15.76:0/840958531 conn(0x7f8f880ce950 msgr2=0x7f8f880ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:47.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.915+0000 7f8f8dada640 1 -- 172.21.15.76:0/840958531 shutdown_connections 2023-11-09T03:46:47.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.915+0000 7f8f8dada640 1 -- 172.21.15.76:0/840958531 wait complete. 2023-11-09T03:46:47.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.916+0000 7f8f8dada640 1 Processor -- start 2023-11-09T03:46:47.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.916+0000 7f8f8dada640 1 -- start start 2023-11-09T03:46:47.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.917+0000 7f8f8dada640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f880adac0 0x7f8f881eea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:47.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.917+0000 7f8f8dada640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f880c4ff0 0x7f8f880acd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:47.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.917+0000 7f8f8dada640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f88159c00 0x7f8f88157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:47.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.917+0000 7f8f8dada640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f881eefa0 con 0x7f8f880adac0 2023-11-09T03:46:47.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.917+0000 7f8f8dada640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f88152f60 con 0x7f8f88159c00 2023-11-09T03:46:47.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.917+0000 7f8f8dada640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f88155f00 con 0x7f8f880c4ff0 2023-11-09T03:46:47.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.917+0000 7f8f877fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f880adac0 0x7f8f881eea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:47.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.917+0000 7f8f877fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f880adac0 0x7f8f881eea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44142/0 (socket says 172.21.15.76:44142) 2023-11-09T03:46:47.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.917+0000 7f8f877fe640 1 -- 172.21.15.76:0/3081289012 learned_addr learned my addr 172.21.15.76:0/3081289012 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:47.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.917+0000 7f8f86ffd640 1 --2- 172.21.15.76:0/3081289012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f880c4ff0 0x7f8f880acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:47.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.917+0000 7f8f87fff640 1 --2- 172.21.15.76:0/3081289012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f88159c00 0x7f8f88157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:47.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.917+0000 7f8f877fe640 1 -- 172.21.15.76:0/3081289012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f880c4ff0 msgr2=0x7f8f880acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:47.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.917+0000 7f8f877fe640 1 --2- 172.21.15.76:0/3081289012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f880c4ff0 0x7f8f880acd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.917+0000 7f8f877fe640 1 -- 172.21.15.76:0/3081289012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f88159c00 msgr2=0x7f8f88157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:47.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.917+0000 7f8f877fe640 1 --2- 172.21.15.76:0/3081289012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f88159c00 0x7f8f88157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:47.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.917+0000 7f8f877fe640 1 -- 172.21.15.76:0/3081289012 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f8000bc80 con 0x7f8f880adac0 2023-11-09T03:46:47.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.917+0000 7f8f87fff640 1 --2- 172.21.15.76:0/3081289012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f88159c00 0x7f8f88157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:47.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.917+0000 7f8f877fe640 1 --2- 172.21.15.76:0/3081289012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f880adac0 0x7f8f881eea60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8f7800c430 tx=0x7f8f7800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:47.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.917+0000 7f8f6ffff640 1 -- 172.21.15.76:0/3081289012 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f780074e0 con 0x7f8f880adac0 2023-11-09T03:46:47.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.917+0000 7f8f6ffff640 1 -- 172.21.15.76:0/3081289012 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f78007660 con 0x7f8f880adac0 2023-11-09T03:46:47.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.917+0000 7f8f8dada640 1 -- 172.21.15.76:0/3081289012 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f881561e0 con 0x7f8f880adac0 2023-11-09T03:46:47.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.917+0000 7f8f6ffff640 1 -- 172.21.15.76:0/3081289012 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f78004a20 con 0x7f8f880adac0 2023-11-09T03:46:47.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.917+0000 7f8f8dada640 1 -- 172.21.15.76:0/3081289012 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f881567a0 con 0x7f8f880adac0 2023-11-09T03:46:47.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.918+0000 7f8f8dada640 1 -- 172.21.15.76:0/3081289012 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f88074290 con 0x7f8f880adac0 2023-11-09T03:46:47.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.919+0000 7f8f6ffff640 1 -- 172.21.15.76:0/3081289012 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8f78004b80 con 0x7f8f880adac0 2023-11-09T03:46:47.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.919+0000 7f8f6ffff640 1 --2- 172.21.15.76:0/3081289012 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8f7406ae70 0x7f8f7406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:47.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.919+0000 7f8f6ffff640 1 -- 172.21.15.76:0/3081289012 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f8f78089200 con 0x7f8f880adac0 2023-11-09T03:46:47.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.919+0000 7f8f86ffd640 1 --2- 172.21.15.76:0/3081289012 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8f7406ae70 0x7f8f7406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:47.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.920+0000 7f8f86ffd640 1 --2- 172.21.15.76:0/3081289012 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8f7406ae70 0x7f8f7406d330 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8f880c37f0 tx=0x7f8f7c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:47.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:47.923+0000 7f8f6ffff640 1 -- 172.21.15.76:0/3081289012 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8f78054550 con 0x7f8f880adac0 2023-11-09T03:46:48.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.152+0000 7f8f8dada640 1 -- 172.21.15.76:0/3081289012 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f8f88070f70 con 0x7f8f880adac0 2023-11-09T03:46:48.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.154+0000 7f8f6ffff640 1 -- 172.21.15.76:0/3081289012 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8f78057b70 con 0x7f8f880adac0 2023-11-09T03:46:48.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.156+0000 7f8f8dada640 1 -- 172.21.15.76:0/3081289012 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8f7406ae70 msgr2=0x7f8f7406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:48.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.156+0000 7f8f8dada640 1 --2- 172.21.15.76:0/3081289012 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8f7406ae70 0x7f8f7406d330 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8f880c37f0 tx=0x7f8f7c008040 comp rx=0 tx=0).stop 2023-11-09T03:46:48.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.156+0000 7f8f8dada640 1 -- 172.21.15.76:0/3081289012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f880adac0 msgr2=0x7f8f881eea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:48.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.156+0000 7f8f8dada640 1 --2- 172.21.15.76:0/3081289012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f880adac0 0x7f8f881eea60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8f7800c430 tx=0x7f8f7800c8f0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.156+0000 7f8f8dada640 1 -- 172.21.15.76:0/3081289012 shutdown_connections 2023-11-09T03:46:48.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.156+0000 7f8f8dada640 1 --2- 172.21.15.76:0/3081289012 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8f7406ae70 0x7f8f7406d330 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.156+0000 7f8f8dada640 1 --2- 172.21.15.76:0/3081289012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f88159c00 0x7f8f88157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.156+0000 7f8f8dada640 1 --2- 172.21.15.76:0/3081289012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f880c4ff0 0x7f8f880acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.156+0000 7f8f8dada640 1 --2- 172.21.15.76:0/3081289012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f880adac0 0x7f8f881eea60 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.156+0000 7f8f8dada640 1 -- 172.21.15.76:0/3081289012 >> 172.21.15.76:0/3081289012 conn(0x7f8f880ce950 msgr2=0x7f8f8806d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:48.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.156+0000 7f8f8dada640 1 -- 172.21.15.76:0/3081289012 shutdown_connections 2023-11-09T03:46:48.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.156+0000 7f8f8dada640 1 -- 172.21.15.76:0/3081289012 wait complete. 2023-11-09T03:46:48.167 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-09T03:46:48.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.267+0000 7f90db4c2640 1 Processor -- start 2023-11-09T03:46:48.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.267+0000 7f90db4c2640 1 -- start start 2023-11-09T03:46:48.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.268+0000 7f90db4c2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f90d4153e60 0x7f90d415dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:48.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.268+0000 7f90db4c2640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90d40cde60 con 0x7f90d40cd6d0 2023-11-09T03:46:48.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.268+0000 7f90db4c2640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90d4160f90 con 0x7f90d40b4880 2023-11-09T03:46:48.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.268+0000 7f90db4c2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90d415e400 con 0x7f90d4153e60 2023-11-09T03:46:48.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.268+0000 7f90d8a5d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f90d40cd6d0 0x7f90d4161420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50656/0 (socket says 172.21.15.76:50656) 2023-11-09T03:46:48.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.268+0000 7f90d8a5d640 1 -- 172.21.15.76:0/2213292807 learned_addr learned my addr 172.21.15.76:0/2213292807 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:48.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.269+0000 7f90cbfff640 1 -- 172.21.15.76:0/2213292807 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1128737256 0 0) 0x7f90d40cde60 con 0x7f90d40cd6d0 2023-11-09T03:46:48.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.269+0000 7f90cbfff640 1 -- 172.21.15.76:0/2213292807 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90c0003520 con 0x7f90d40cd6d0 2023-11-09T03:46:48.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.269+0000 7f90d925e640 1 --2- 172.21.15.76:0/2213292807 >> v2:172.21.15.113:3300/0 conn(0x7f90d4153e60 0x7f90d415dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:48.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.269+0000 7f90cbfff640 1 -- 172.21.15.76:0/2213292807 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 170262728 0 0) 0x7f90c0003520 con 0x7f90d40cd6d0 2023-11-09T03:46:48.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.269+0000 7f90cbfff640 1 -- 172.21.15.76:0/2213292807 >> v2:172.21.15.113:3300/0 conn(0x7f90d4153e60 msgr2=0x7f90d415dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:48.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.269+0000 7f90cbfff640 1 --2- 172.21.15.76:0/2213292807 >> v2:172.21.15.113:3300/0 conn(0x7f90d4153e60 0x7f90d415dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.269+0000 7f90cbfff640 1 -- 172.21.15.76:0/2213292807 >> v1:172.21.15.113:6789/0 conn(0x7f90d40b4880 legacy=0x7f90d40d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:48.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.269+0000 7f90cbfff640 1 -- 172.21.15.76:0/2213292807 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d40ce090 con 0x7f90d40cd6d0 2023-11-09T03:46:48.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.269+0000 7f90cbfff640 1 -- 172.21.15.76:0/2213292807 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f90c4001670 con 0x7f90d40cd6d0 2023-11-09T03:46:48.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.269+0000 7f90d925e640 1 --2- 172.21.15.76:0/2213292807 >> v2:172.21.15.113:3300/0 conn(0x7f90d4153e60 0x7f90d415dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:48.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.269+0000 7f90cbfff640 1 -- 172.21.15.76:0/2213292807 >> v1:172.21.15.76:6789/0 conn(0x7f90d40cd6d0 legacy=0x7f90d4161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:48.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.269+0000 7f90cbfff640 1 --2- 172.21.15.76:0/2213292807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90c0003970 0x7f90c0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:48.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.269+0000 7f90cbfff640 1 --2- 172.21.15.76:0/2213292807 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90c00047c0 0x7f90c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:48.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.269+0000 7f90cbfff640 1 --2- 172.21.15.76:0/2213292807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90c0005180 0x7f90c000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:48.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.269+0000 7f90cbfff640 1 -- 172.21.15.76:0/2213292807 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d415e400 con 0x7f90c0005180 2023-11-09T03:46:48.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.269+0000 7f90cbfff640 1 -- 172.21.15.76:0/2213292807 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c000c0d0 con 0x7f90c0003970 2023-11-09T03:46:48.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.269+0000 7f90cbfff640 1 -- 172.21.15.76:0/2213292807 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c000c290 con 0x7f90c00047c0 2023-11-09T03:46:48.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.269+0000 7f90d9a5f640 1 --2- 172.21.15.76:0/2213292807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90c0005180 0x7f90c000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:48.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.269+0000 7f90d925e640 1 --2- 172.21.15.76:0/2213292807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90c0003970 0x7f90c0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:48.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.269+0000 7f90d8a5d640 1 --2- 172.21.15.76:0/2213292807 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90c00047c0 0x7f90c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:48.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.269+0000 7f90d9a5f640 1 -- 172.21.15.76:0/2213292807 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90c00047c0 msgr2=0x7f90c0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:48.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.269+0000 7f90d9a5f640 1 --2- 172.21.15.76:0/2213292807 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90c00047c0 0x7f90c0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.269+0000 7f90d9a5f640 1 -- 172.21.15.76:0/2213292807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90c0003970 msgr2=0x7f90c0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:48.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.269+0000 7f90d9a5f640 1 --2- 172.21.15.76:0/2213292807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90c0003970 0x7f90c0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.269+0000 7f90d9a5f640 1 -- 172.21.15.76:0/2213292807 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c0003520 con 0x7f90c0005180 2023-11-09T03:46:48.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.270+0000 7f90d9a5f640 1 -- 172.21.15.76:0/2213292807 reap_dead start 2023-11-09T03:46:48.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.270+0000 7f90d9a5f640 1 --2- 172.21.15.76:0/2213292807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90c0005180 0x7f90c000b980 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f90cc001750 tx=0x7f90c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534578211e8ef88d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:48.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.270+0000 7f90cbfff640 1 -- 172.21.15.76:0/2213292807 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc0052d0 con 0x7f90c0005180 2023-11-09T03:46:48.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.270+0000 7f90cbfff640 1 -- 172.21.15.76:0/2213292807 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90cc02e050 con 0x7f90c0005180 2023-11-09T03:46:48.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.270+0000 7f90cbfff640 1 -- 172.21.15.76:0/2213292807 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc003830 con 0x7f90c0005180 2023-11-09T03:46:48.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.271+0000 7f90db4c2640 1 -- 172.21.15.76:0/2213292807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90c0005180 msgr2=0x7f90c000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:48.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.271+0000 7f90db4c2640 1 --2- 172.21.15.76:0/2213292807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90c0005180 0x7f90c000b980 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f90cc001750 tx=0x7f90c00057f0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.271+0000 7f90db4c2640 1 -- 172.21.15.76:0/2213292807 shutdown_connections 2023-11-09T03:46:48.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.271+0000 7f90db4c2640 1 --2- 172.21.15.76:0/2213292807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90c0005180 0x7f90c000b980 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.271+0000 7f90db4c2640 1 -- 172.21.15.76:0/2213292807 >> 172.21.15.76:0/2213292807 conn(0x7f90d40cf880 msgr2=0x7f90d40cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:48.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.271+0000 7f90db4c2640 1 -- 172.21.15.76:0/2213292807 shutdown_connections 2023-11-09T03:46:48.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.271+0000 7f90db4c2640 1 -- 172.21.15.76:0/2213292807 wait complete. 2023-11-09T03:46:48.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.272+0000 7f90db4c2640 1 Processor -- start 2023-11-09T03:46:48.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.272+0000 7f90db4c2640 1 -- start start 2023-11-09T03:46:48.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.273+0000 7f90db4c2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90c0005180 0x7f90d40cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:48.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.273+0000 7f90db4c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90d41545d0 0x7f90d40c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:48.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.273+0000 7f90db4c2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90d40b7180 0x7f90d40b7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:48.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.273+0000 7f90db4c2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d41f3cd0 con 0x7f90d41545d0 2023-11-09T03:46:48.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.273+0000 7f90db4c2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d40b7b40 con 0x7f90d40b7180 2023-11-09T03:46:48.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.273+0000 7f90db4c2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d4154a50 con 0x7f90c0005180 2023-11-09T03:46:48.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.273+0000 7f90d8a5d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90d41545d0 0x7f90d40c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:48.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.273+0000 7f90d8a5d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90d41545d0 0x7f90d40c88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44164/0 (socket says 172.21.15.76:44164) 2023-11-09T03:46:48.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.273+0000 7f90d8a5d640 1 -- 172.21.15.76:0/2800492536 learned_addr learned my addr 172.21.15.76:0/2800492536 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:48.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.273+0000 7f90d8a5d640 1 -- 172.21.15.76:0/2800492536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90c0005180 msgr2=0x7f90d40cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:48.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.273+0000 7f90d925e640 1 --2- 172.21.15.76:0/2800492536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90c0005180 0x7f90d40cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:48.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.273+0000 7f90d8a5d640 1 --2- 172.21.15.76:0/2800492536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90c0005180 0x7f90d40cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.273+0000 7f90d9a5f640 1 --2- 172.21.15.76:0/2800492536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90d40b7180 0x7f90d40b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:48.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.273+0000 7f90d8a5d640 1 -- 172.21.15.76:0/2800492536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90d40b7180 msgr2=0x7f90d40b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:48.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.273+0000 7f90d8a5d640 1 --2- 172.21.15.76:0/2800492536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90d40b7180 0x7f90d40b7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.273+0000 7f90d8a5d640 1 -- 172.21.15.76:0/2800492536 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d4154d00 con 0x7f90d41545d0 2023-11-09T03:46:48.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.273+0000 7f90d8a5d640 1 --2- 172.21.15.76:0/2800492536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90d41545d0 0x7f90d40c88b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f90c4007290 tx=0x7f90c4007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:48.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.273+0000 7f90c9ffb640 1 -- 172.21.15.76:0/2800492536 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c4001630 con 0x7f90d41545d0 2023-11-09T03:46:48.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.273+0000 7f90c9ffb640 1 -- 172.21.15.76:0/2800492536 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c400f9c0 con 0x7f90d41545d0 2023-11-09T03:46:48.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.273+0000 7f90c9ffb640 1 -- 172.21.15.76:0/2800492536 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c400fd50 con 0x7f90d41545d0 2023-11-09T03:46:48.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.273+0000 7f90db4c2640 1 -- 172.21.15.76:0/2800492536 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d4154fe0 con 0x7f90d41545d0 2023-11-09T03:46:48.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.274+0000 7f90db4c2640 1 -- 172.21.15.76:0/2800492536 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d41555a0 con 0x7f90d41545d0 2023-11-09T03:46:48.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.275+0000 7f90db4c2640 1 -- 172.21.15.76:0/2800492536 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90d41f3e40 con 0x7f90d41545d0 2023-11-09T03:46:48.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.275+0000 7f90c9ffb640 1 -- 172.21.15.76:0/2800492536 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f90c401d460 con 0x7f90d41545d0 2023-11-09T03:46:48.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.275+0000 7f90c9ffb640 1 --2- 172.21.15.76:0/2800492536 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f90b806aea0 0x7f90b806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:48.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.275+0000 7f90c9ffb640 1 -- 172.21.15.76:0/2800492536 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f90c4089520 con 0x7f90d41545d0 2023-11-09T03:46:48.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.276+0000 7f90d925e640 1 --2- 172.21.15.76:0/2800492536 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f90b806aea0 0x7f90b806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:48.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.276+0000 7f90d925e640 1 --2- 172.21.15.76:0/2800492536 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f90b806aea0 0x7f90b806d360 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f90d0009f50 tx=0x7f90d000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:48.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.279+0000 7f90c9ffb640 1 -- 172.21.15.76:0/2800492536 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f90d41f3e40 con 0x7f90d41545d0 2023-11-09T03:46:48.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.511+0000 7f90db4c2640 1 -- 172.21.15.76:0/2800492536 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f90d4155760 con 0x7f90d41545d0 2023-11-09T03:46:48.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.513+0000 7f90c9ffb640 1 -- 172.21.15.76:0/2800492536 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f90c40547e0 con 0x7f90d41545d0 2023-11-09T03:46:48.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.515+0000 7f90db4c2640 1 -- 172.21.15.76:0/2800492536 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f90b806aea0 msgr2=0x7f90b806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:48.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.515+0000 7f90db4c2640 1 --2- 172.21.15.76:0/2800492536 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f90b806aea0 0x7f90b806d360 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f90d0009f50 tx=0x7f90d000b040 comp rx=0 tx=0).stop 2023-11-09T03:46:48.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.515+0000 7f90db4c2640 1 -- 172.21.15.76:0/2800492536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90d41545d0 msgr2=0x7f90d40c88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:48.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.515+0000 7f90db4c2640 1 --2- 172.21.15.76:0/2800492536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90d41545d0 0x7f90d40c88b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f90c4007290 tx=0x7f90c4007750 comp rx=0 tx=0).stop 2023-11-09T03:46:48.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.515+0000 7f90db4c2640 1 -- 172.21.15.76:0/2800492536 shutdown_connections 2023-11-09T03:46:48.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.515+0000 7f90db4c2640 1 --2- 172.21.15.76:0/2800492536 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f90b806aea0 0x7f90b806d360 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.515+0000 7f90db4c2640 1 --2- 172.21.15.76:0/2800492536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90d40b7180 0x7f90d40b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.515+0000 7f90db4c2640 1 --2- 172.21.15.76:0/2800492536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90d41545d0 0x7f90d40c88b0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.515+0000 7f90db4c2640 1 --2- 172.21.15.76:0/2800492536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90c0005180 0x7f90d40cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.515+0000 7f90db4c2640 1 -- 172.21.15.76:0/2800492536 >> 172.21.15.76:0/2800492536 conn(0x7f90d40cf880 msgr2=0x7f90d406baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:48.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.515+0000 7f90db4c2640 1 -- 172.21.15.76:0/2800492536 shutdown_connections 2023-11-09T03:46:48.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.515+0000 7f90db4c2640 1 -- 172.21.15.76:0/2800492536 wait complete. 2023-11-09T03:46:48.525 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-09T03:46:48.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.626+0000 7effcef65640 1 Processor -- start 2023-11-09T03:46:48.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.626+0000 7effcef65640 1 -- start start 2023-11-09T03:46:48.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.627+0000 7effcef65640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7effc8070ff0 0x7effc80cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:48.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.627+0000 7effcef65640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc80c9bf0 con 0x7effc8066000 2023-11-09T03:46:48.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.627+0000 7effcef65640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc80cd890 con 0x7effc815d1b0 2023-11-09T03:46:48.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.627+0000 7effcef65640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc80cdb30 con 0x7effc8070ff0 2023-11-09T03:46:48.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.627+0000 7effcd502640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7effc8066000 0x7effc806a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50668/0 (socket says 172.21.15.76:50668) 2023-11-09T03:46:48.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.627+0000 7effcd502640 1 -- 172.21.15.76:0/902295604 learned_addr learned my addr 172.21.15.76:0/902295604 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:48.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.627+0000 7effccd01640 1 --2- 172.21.15.76:0/902295604 >> v2:172.21.15.113:3300/0 conn(0x7effc8070ff0 0x7effc80cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:48.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.627+0000 7effc77fe640 1 -- 172.21.15.76:0/902295604 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4157286632 0 0) 0x7effc80c9bf0 con 0x7effc8066000 2023-11-09T03:46:48.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.628+0000 7effc77fe640 1 -- 172.21.15.76:0/902295604 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effb0003520 con 0x7effc8066000 2023-11-09T03:46:48.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.628+0000 7effc77fe640 1 -- 172.21.15.76:0/902295604 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 725834071 0 0) 0x7effb0003520 con 0x7effc8066000 2023-11-09T03:46:48.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.628+0000 7effccd01640 1 -- 172.21.15.76:0/902295604 >> v1:172.21.15.113:6789/0 conn(0x7effc815d1b0 legacy=0x7effc80713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:48.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.628+0000 7effccd01640 1 -- 172.21.15.76:0/902295604 >> v1:172.21.15.76:6789/0 conn(0x7effc8066000 legacy=0x7effc806a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:48.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.628+0000 7effccd01640 1 -- 172.21.15.76:0/902295604 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc80cde10 con 0x7effc8070ff0 2023-11-09T03:46:48.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.629+0000 7effccd01640 1 --2- 172.21.15.76:0/902295604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effc8070ff0 0x7effc80cb4e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7effb80097a0 tx=0x7effb802b680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=173e66eca6b56101 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:48.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.629+0000 7effc77fe640 1 -- 172.21.15.76:0/902295604 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc00019f0 con 0x7effc8070ff0 2023-11-09T03:46:48.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.629+0000 7effc77fe640 1 -- 172.21.15.76:0/902295604 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb802e050 con 0x7effc8070ff0 2023-11-09T03:46:48.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.629+0000 7effc77fe640 1 -- 172.21.15.76:0/902295604 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb8030730 con 0x7effc8070ff0 2023-11-09T03:46:48.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.629+0000 7effcef65640 1 -- 172.21.15.76:0/902295604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effc8070ff0 msgr2=0x7effc80cb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:48.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.629+0000 7effcef65640 1 --2- 172.21.15.76:0/902295604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effc8070ff0 0x7effc80cb4e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7effb80097a0 tx=0x7effb802b680 comp rx=0 tx=0).stop 2023-11-09T03:46:48.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.629+0000 7effcef65640 1 -- 172.21.15.76:0/902295604 shutdown_connections 2023-11-09T03:46:48.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.629+0000 7effcef65640 1 --2- 172.21.15.76:0/902295604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effc8070ff0 0x7effc80cb4e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.629+0000 7effcef65640 1 -- 172.21.15.76:0/902295604 >> 172.21.15.76:0/902295604 conn(0x7effc80cbc90 msgr2=0x7effc80cc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:48.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.629+0000 7effcef65640 1 -- 172.21.15.76:0/902295604 shutdown_connections 2023-11-09T03:46:48.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.629+0000 7effcef65640 1 -- 172.21.15.76:0/902295604 wait complete. 2023-11-09T03:46:48.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.630+0000 7effcef65640 1 Processor -- start 2023-11-09T03:46:48.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.630+0000 7effcef65640 1 -- start start 2023-11-09T03:46:48.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.631+0000 7effcef65640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effc8066000 0x7effc81f2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:48.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.631+0000 7effcef65640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effc8070ff0 0x7effc8158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:48.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.631+0000 7effcef65640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effc815d1b0 0x7effc815e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:48.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.631+0000 7effcef65640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc80c7e30 con 0x7effc815d1b0 2023-11-09T03:46:48.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.631+0000 7effcef65640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc80c7800 con 0x7effc8070ff0 2023-11-09T03:46:48.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.631+0000 7effcef65640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc815a170 con 0x7effc8066000 2023-11-09T03:46:48.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.631+0000 7effcd502640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effc815d1b0 0x7effc815e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:48.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.631+0000 7effcd502640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effc815d1b0 0x7effc815e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44172/0 (socket says 172.21.15.76:44172) 2023-11-09T03:46:48.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.631+0000 7effcd502640 1 -- 172.21.15.76:0/2101949284 learned_addr learned my addr 172.21.15.76:0/2101949284 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:48.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.632+0000 7effccd01640 1 --2- 172.21.15.76:0/2101949284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effc8066000 0x7effc81f2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:48.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.632+0000 7effc7fff640 1 --2- 172.21.15.76:0/2101949284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effc8070ff0 0x7effc8158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:48.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.632+0000 7effcd502640 1 -- 172.21.15.76:0/2101949284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effc8066000 msgr2=0x7effc81f2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:48.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.632+0000 7effcd502640 1 --2- 172.21.15.76:0/2101949284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effc8066000 0x7effc81f2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.632+0000 7effcd502640 1 -- 172.21.15.76:0/2101949284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effc8070ff0 msgr2=0x7effc8158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:48.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.632+0000 7effcd502640 1 --2- 172.21.15.76:0/2101949284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effc8070ff0 0x7effc8158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.632+0000 7effcd502640 1 -- 172.21.15.76:0/2101949284 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb8009400 con 0x7effc815d1b0 2023-11-09T03:46:48.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.632+0000 7effccd01640 1 --2- 172.21.15.76:0/2101949284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effc8066000 0x7effc81f2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:48.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.632+0000 7effcd502640 1 --2- 172.21.15.76:0/2101949284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effc815d1b0 0x7effc815e4e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7effc000cac0 tx=0x7effc000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:48.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.632+0000 7effc57fa640 1 -- 172.21.15.76:0/2101949284 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc000a640 con 0x7effc815d1b0 2023-11-09T03:46:48.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.632+0000 7effc57fa640 1 -- 172.21.15.76:0/2101949284 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc000a7a0 con 0x7effc815d1b0 2023-11-09T03:46:48.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.632+0000 7effcef65640 1 -- 172.21.15.76:0/2101949284 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc815a450 con 0x7effc815d1b0 2023-11-09T03:46:48.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.632+0000 7effc57fa640 1 -- 172.21.15.76:0/2101949284 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc000abd0 con 0x7effc815d1b0 2023-11-09T03:46:48.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.632+0000 7effcef65640 1 -- 172.21.15.76:0/2101949284 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc8158a50 con 0x7effc815d1b0 2023-11-09T03:46:48.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.633+0000 7effcef65640 1 -- 172.21.15.76:0/2101949284 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc815a690 con 0x7effc815d1b0 2023-11-09T03:46:48.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.634+0000 7effc57fa640 1 -- 172.21.15.76:0/2101949284 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7effc000ad30 con 0x7effc815d1b0 2023-11-09T03:46:48.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.634+0000 7effc57fa640 1 --2- 172.21.15.76:0/2101949284 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7effb406ae70 0x7effb406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:48.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.634+0000 7effc57fa640 1 -- 172.21.15.76:0/2101949284 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7effc0088bf0 con 0x7effc815d1b0 2023-11-09T03:46:48.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.634+0000 7effccd01640 1 --2- 172.21.15.76:0/2101949284 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7effb406ae70 0x7effb406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:48.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.635+0000 7effccd01640 1 --2- 172.21.15.76:0/2101949284 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7effb406ae70 0x7effb406d330 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7effb8034000 tx=0x7effb8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:48.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.638+0000 7effc57fa640 1 -- 172.21.15.76:0/2101949284 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7effc0053ff0 con 0x7effc815d1b0 2023-11-09T03:46:48.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.867+0000 7effcef65640 1 -- 172.21.15.76:0/2101949284 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7effc8074290 con 0x7effc815d1b0 2023-11-09T03:46:48.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.869+0000 7effc57fa640 1 -- 172.21.15.76:0/2101949284 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7effc0018020 con 0x7effc815d1b0 2023-11-09T03:46:48.871 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.871+0000 7effcef65640 1 -- 172.21.15.76:0/2101949284 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7effb406ae70 msgr2=0x7effb406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:48.871 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.871+0000 7effcef65640 1 --2- 172.21.15.76:0/2101949284 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7effb406ae70 0x7effb406d330 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7effb8034000 tx=0x7effb8008030 comp rx=0 tx=0).stop 2023-11-09T03:46:48.871 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.871+0000 7effcef65640 1 -- 172.21.15.76:0/2101949284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effc815d1b0 msgr2=0x7effc815e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:48.871 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.871+0000 7effcef65640 1 --2- 172.21.15.76:0/2101949284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effc815d1b0 0x7effc815e4e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7effc000cac0 tx=0x7effc000cf80 comp rx=0 tx=0).stop 2023-11-09T03:46:48.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.871+0000 7effcef65640 1 -- 172.21.15.76:0/2101949284 shutdown_connections 2023-11-09T03:46:48.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.871+0000 7effcef65640 1 --2- 172.21.15.76:0/2101949284 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7effb406ae70 0x7effb406d330 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.871+0000 7effcef65640 1 --2- 172.21.15.76:0/2101949284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effc815d1b0 0x7effc815e4e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.871+0000 7effcef65640 1 --2- 172.21.15.76:0/2101949284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effc8070ff0 0x7effc8158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.871+0000 7effcef65640 1 --2- 172.21.15.76:0/2101949284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effc8066000 0x7effc81f2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.871+0000 7effcef65640 1 -- 172.21.15.76:0/2101949284 >> 172.21.15.76:0/2101949284 conn(0x7effc80cbc90 msgr2=0x7effc806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:48.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.871+0000 7effcef65640 1 -- 172.21.15.76:0/2101949284 shutdown_connections 2023-11-09T03:46:48.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.871+0000 7effcef65640 1 -- 172.21.15.76:0/2101949284 wait complete. 2023-11-09T03:46:48.881 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:46:48.981 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.982+0000 7fc23e6cf640 1 Processor -- start 2023-11-09T03:46:48.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.982+0000 7fc23e6cf640 1 -- start start 2023-11-09T03:46:48.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.983+0000 7fc23e6cf640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2380b42c0 0x7fc2380c9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:48.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.983+0000 7fc23e6cf640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc2380c9c40 0x7fc2380b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:48.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.983+0000 7fc23e6cf640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2380ca090 con 0x7fc23815ad80 2023-11-09T03:46:48.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.983+0000 7fc23e6cf640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc23806cfa0 con 0x7fc2380c9c40 2023-11-09T03:46:48.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.983+0000 7fc23e6cf640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc23806d280 con 0x7fc2380b42c0 2023-11-09T03:46:48.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.983+0000 7fc23cc6c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc2380c9c40 0x7fc2380b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:48.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.983+0000 7fc23cc6c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc2380c9c40 0x7fc2380b1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44176/0 (socket says 172.21.15.76:44176) 2023-11-09T03:46:48.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.983+0000 7fc23cc6c640 1 -- 172.21.15.76:0/1542168856 learned_addr learned my addr 172.21.15.76:0/1542168856 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:48.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.983+0000 7fc2377fe640 1 --2- 172.21.15.76:0/1542168856 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2380b42c0 0x7fc2380c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:48.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.983+0000 7fc23cc6c640 1 -- 172.21.15.76:0/1542168856 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2380b42c0 msgr2=0x7fc2380c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:48.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.983+0000 7fc23cc6c640 1 --2- 172.21.15.76:0/1542168856 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2380b42c0 0x7fc2380c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.983+0000 7fc23cc6c640 1 -- 172.21.15.76:0/1542168856 >> v1:172.21.15.113:6789/0 conn(0x7fc23815ad80 legacy=0x7fc23806c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:48.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.983+0000 7fc23cc6c640 1 -- 172.21.15.76:0/1542168856 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc23806d560 con 0x7fc2380c9c40 2023-11-09T03:46:48.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.984+0000 7fc23cc6c640 1 --2- 172.21.15.76:0/1542168856 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2380c9c40 0x7fc2380b1a40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc230009e50 tx=0x7fc23002bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1eb0d2855b09cf6 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:48.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.984+0000 7fc236ffd640 1 -- 172.21.15.76:0/1542168856 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23002cb90 con 0x7fc2380c9c40 2023-11-09T03:46:48.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.984+0000 7fc236ffd640 1 -- 172.21.15.76:0/1542168856 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23002ccf0 con 0x7fc2380c9c40 2023-11-09T03:46:48.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.984+0000 7fc236ffd640 1 -- 172.21.15.76:0/1542168856 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc230034610 con 0x7fc2380c9c40 2023-11-09T03:46:48.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.985+0000 7fc23e6cf640 1 -- 172.21.15.76:0/1542168856 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2380c9c40 msgr2=0x7fc2380b1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:48.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.985+0000 7fc23e6cf640 1 --2- 172.21.15.76:0/1542168856 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2380c9c40 0x7fc2380b1a40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc230009e50 tx=0x7fc23002bff0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.985+0000 7fc23e6cf640 1 -- 172.21.15.76:0/1542168856 shutdown_connections 2023-11-09T03:46:48.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.985+0000 7fc23e6cf640 1 --2- 172.21.15.76:0/1542168856 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2380c9c40 0x7fc2380b1a40 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.985+0000 7fc23e6cf640 1 --2- 172.21.15.76:0/1542168856 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2380b42c0 0x7fc2380c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.985+0000 7fc23e6cf640 1 -- 172.21.15.76:0/1542168856 >> 172.21.15.76:0/1542168856 conn(0x7fc2380ce950 msgr2=0x7fc2380ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:48.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.985+0000 7fc23e6cf640 1 -- 172.21.15.76:0/1542168856 shutdown_connections 2023-11-09T03:46:48.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.985+0000 7fc23e6cf640 1 -- 172.21.15.76:0/1542168856 wait complete. 2023-11-09T03:46:48.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.986+0000 7fc23e6cf640 1 Processor -- start 2023-11-09T03:46:48.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.986+0000 7fc23e6cf640 1 -- start start 2023-11-09T03:46:48.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.987+0000 7fc23e6cf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2380b42c0 0x7fc2381efb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:48.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.987+0000 7fc23e6cf640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2380c9c40 0x7fc2380c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:48.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.987+0000 7fc23e6cf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc23815ad80 0x7fc23815ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:48.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.987+0000 7fc23e6cf640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23815de50 con 0x7fc23815ad80 2023-11-09T03:46:48.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.987+0000 7fc23e6cf640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2380c6880 con 0x7fc2380b42c0 2023-11-09T03:46:48.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.987+0000 7fc23e6cf640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2380c5990 con 0x7fc2380c9c40 2023-11-09T03:46:48.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.987+0000 7fc23cc6c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc23815ad80 0x7fc23815ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:48.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.987+0000 7fc23cc6c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc23815ad80 0x7fc23815ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44186/0 (socket says 172.21.15.76:44186) 2023-11-09T03:46:48.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.987+0000 7fc23cc6c640 1 -- 172.21.15.76:0/2110860402 learned_addr learned my addr 172.21.15.76:0/2110860402 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:48.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.987+0000 7fc23cc6c640 1 -- 172.21.15.76:0/2110860402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2380c9c40 msgr2=0x7fc2380c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:48.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.987+0000 7fc2377fe640 1 --2- 172.21.15.76:0/2110860402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2380c9c40 0x7fc2380c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:48.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.987+0000 7fc23cc6c640 1 --2- 172.21.15.76:0/2110860402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2380c9c40 0x7fc2380c8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.987+0000 7fc23cc6c640 1 -- 172.21.15.76:0/2110860402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2380b42c0 msgr2=0x7fc2381efb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:48.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.987+0000 7fc237fff640 1 --2- 172.21.15.76:0/2110860402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2380b42c0 0x7fc2381efb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:48.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.987+0000 7fc23cc6c640 1 --2- 172.21.15.76:0/2110860402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2380b42c0 0x7fc2381efb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:48.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.987+0000 7fc23cc6c640 1 -- 172.21.15.76:0/2110860402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc230009b30 con 0x7fc23815ad80 2023-11-09T03:46:48.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.987+0000 7fc23cc6c640 1 --2- 172.21.15.76:0/2110860402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc23815ad80 0x7fc23815ccf0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc230009e50 tx=0x7fc230004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:48.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.988+0000 7fc234ff9640 1 -- 172.21.15.76:0/2110860402 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc230009510 con 0x7fc23815ad80 2023-11-09T03:46:48.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.988+0000 7fc234ff9640 1 -- 172.21.15.76:0/2110860402 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23003e450 con 0x7fc23815ad80 2023-11-09T03:46:48.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.988+0000 7fc23e6cf640 1 -- 172.21.15.76:0/2110860402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2380c5c70 con 0x7fc23815ad80 2023-11-09T03:46:48.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.988+0000 7fc234ff9640 1 -- 172.21.15.76:0/2110860402 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23003e800 con 0x7fc23815ad80 2023-11-09T03:46:48.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.988+0000 7fc23e6cf640 1 -- 172.21.15.76:0/2110860402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2380c6230 con 0x7fc23815ad80 2023-11-09T03:46:48.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.989+0000 7fc23e6cf640 1 -- 172.21.15.76:0/2110860402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2380c6a10 con 0x7fc23815ad80 2023-11-09T03:46:48.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.989+0000 7fc234ff9640 1 -- 172.21.15.76:0/2110860402 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc23003e960 con 0x7fc23815ad80 2023-11-09T03:46:48.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.989+0000 7fc234ff9640 1 --2- 172.21.15.76:0/2110860402 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc22406ae70 0x7fc22406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:48.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.989+0000 7fc234ff9640 1 -- 172.21.15.76:0/2110860402 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fc2300ac050 con 0x7fc23815ad80 2023-11-09T03:46:48.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.990+0000 7fc237fff640 1 --2- 172.21.15.76:0/2110860402 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc22406ae70 0x7fc22406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:48.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.990+0000 7fc237fff640 1 --2- 172.21.15.76:0/2110860402 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc22406ae70 0x7fc22406d330 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc228000f60 tx=0x7fc228009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:48.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:48.994+0000 7fc234ff9640 1 -- 172.21.15.76:0/2110860402 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc230077310 con 0x7fc23815ad80 2023-11-09T03:46:49.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.159+0000 7fc23e6cf640 1 -- 172.21.15.76:0/2110860402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc238074290 con 0x7fc23815ad80 2023-11-09T03:46:49.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.160+0000 7fc234ff9640 1 -- 172.21.15.76:0/2110860402 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+13050 (secure 0 0 0) 0x7fc23007fe60 con 0x7fc23815ad80 2023-11-09T03:46:49.159 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:49.159 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":35,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:46:46.343971+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:46:49.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.161+0000 7fc23e6cf640 1 -- 172.21.15.76:0/2110860402 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc22406ae70 msgr2=0x7fc22406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:49.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.161+0000 7fc23e6cf640 1 --2- 172.21.15.76:0/2110860402 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc22406ae70 0x7fc22406d330 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc228000f60 tx=0x7fc228009040 comp rx=0 tx=0).stop 2023-11-09T03:46:49.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.161+0000 7fc23e6cf640 1 -- 172.21.15.76:0/2110860402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc23815ad80 msgr2=0x7fc23815ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:49.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.161+0000 7fc23e6cf640 1 --2- 172.21.15.76:0/2110860402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc23815ad80 0x7fc23815ccf0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc230009e50 tx=0x7fc230004a40 comp rx=0 tx=0).stop 2023-11-09T03:46:49.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.161+0000 7fc23e6cf640 1 -- 172.21.15.76:0/2110860402 shutdown_connections 2023-11-09T03:46:49.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.161+0000 7fc23e6cf640 1 --2- 172.21.15.76:0/2110860402 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc22406ae70 0x7fc22406d330 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:49.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.161+0000 7fc23e6cf640 1 --2- 172.21.15.76:0/2110860402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc23815ad80 0x7fc23815ccf0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:49.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.161+0000 7fc23e6cf640 1 --2- 172.21.15.76:0/2110860402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2380c9c40 0x7fc2380c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:49.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.161+0000 7fc23e6cf640 1 --2- 172.21.15.76:0/2110860402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2380b42c0 0x7fc2381efb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:49.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.161+0000 7fc23e6cf640 1 -- 172.21.15.76:0/2110860402 >> 172.21.15.76:0/2110860402 conn(0x7fc2380ce950 msgr2=0x7fc2380b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:49.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.162+0000 7fc23e6cf640 1 -- 172.21.15.76:0/2110860402 shutdown_connections 2023-11-09T03:46:49.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.162+0000 7fc23e6cf640 1 -- 172.21.15.76:0/2110860402 wait complete. 2023-11-09T03:46:49.171 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-09T03:46:49.172 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:46:49.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.275+0000 7fdbaa7b6640 1 Processor -- start 2023-11-09T03:46:49.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.275+0000 7fdbaa7b6640 1 -- start start 2023-11-09T03:46:49.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.276+0000 7fdbaa7b6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdba40b4880 0x7fdba40b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:49.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.276+0000 7fdbaa7b6640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdba40cde60 con 0x7fdba4153e60 2023-11-09T03:46:49.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.276+0000 7fdbaa7b6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdba4160f90 con 0x7fdba40cd6d0 2023-11-09T03:46:49.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.276+0000 7fdbaa7b6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba40b5200 con 0x7fdba40b4880 2023-11-09T03:46:49.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.276+0000 7fdba3fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fdba4153e60 0x7fdba415dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50672/0 (socket says 172.21.15.76:50672) 2023-11-09T03:46:49.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.276+0000 7fdba3fff640 1 -- 172.21.15.76:0/3874834478 learned_addr learned my addr 172.21.15.76:0/3874834478 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:49.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.277+0000 7fdba2ffd640 1 -- 172.21.15.76:0/3874834478 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3853825540 0 0) 0x7fdba40cde60 con 0x7fdba4153e60 2023-11-09T03:46:49.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.277+0000 7fdba2ffd640 1 -- 172.21.15.76:0/3874834478 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb8c003520 con 0x7fdba4153e60 2023-11-09T03:46:49.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.277+0000 7fdba8d53640 1 --2- 172.21.15.76:0/3874834478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdba40b4880 0x7fdba40b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:49.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.277+0000 7fdba2ffd640 1 -- 172.21.15.76:0/3874834478 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 325972581 0 0) 0x7fdb8c003520 con 0x7fdba4153e60 2023-11-09T03:46:49.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.277+0000 7fdba2ffd640 1 -- 172.21.15.76:0/3874834478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdba40b4880 msgr2=0x7fdba40b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:49.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.277+0000 7fdba2ffd640 1 --2- 172.21.15.76:0/3874834478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdba40b4880 0x7fdba40b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:49.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.277+0000 7fdba2ffd640 1 -- 172.21.15.76:0/3874834478 >> v1:172.21.15.113:6789/0 conn(0x7fdba40cd6d0 legacy=0x7fdba4161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:49.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.277+0000 7fdba2ffd640 1 -- 172.21.15.76:0/3874834478 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba40ce090 con 0x7fdba4153e60 2023-11-09T03:46:49.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.277+0000 7fdba2ffd640 1 -- 172.21.15.76:0/3874834478 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fdb94001670 con 0x7fdba4153e60 2023-11-09T03:46:49.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.277+0000 7fdba2ffd640 1 -- 172.21.15.76:0/3874834478 >> v1:172.21.15.76:6789/0 conn(0x7fdba4153e60 legacy=0x7fdba415dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:49.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.277+0000 7fdba2ffd640 1 --2- 172.21.15.76:0/3874834478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb8c0039a0 0x7fdb8c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:49.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.277+0000 7fdba2ffd640 1 --2- 172.21.15.76:0/3874834478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb8c0047c0 0x7fdb8c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:49.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.277+0000 7fdba2ffd640 1 --2- 172.21.15.76:0/3874834478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdba40b4880 0x7fdb8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:49.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.277+0000 7fdba2ffd640 1 -- 172.21.15.76:0/3874834478 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba40b5200 con 0x7fdb8c0047c0 2023-11-09T03:46:49.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.277+0000 7fdba2ffd640 1 -- 172.21.15.76:0/3874834478 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c00ba60 con 0x7fdb8c0039a0 2023-11-09T03:46:49.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.277+0000 7fdba2ffd640 1 -- 172.21.15.76:0/3874834478 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c00bc20 con 0x7fdba40b4880 2023-11-09T03:46:49.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.277+0000 7fdba37fe640 1 --2- 172.21.15.76:0/3874834478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb8c0047c0 0x7fdb8c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:49.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.277+0000 7fdba3fff640 1 --2- 172.21.15.76:0/3874834478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb8c0039a0 0x7fdb8c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:49.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.277+0000 7fdba8d53640 1 --2- 172.21.15.76:0/3874834478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdba40b4880 0x7fdb8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:49.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.277+0000 7fdba37fe640 1 -- 172.21.15.76:0/3874834478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdba40b4880 msgr2=0x7fdb8c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:49.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.277+0000 7fdba37fe640 1 --2- 172.21.15.76:0/3874834478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdba40b4880 0x7fdb8c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:49.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.277+0000 7fdba37fe640 1 -- 172.21.15.76:0/3874834478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb8c0039a0 msgr2=0x7fdb8c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:49.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.277+0000 7fdba37fe640 1 --2- 172.21.15.76:0/3874834478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb8c0039a0 0x7fdb8c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:49.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.277+0000 7fdba37fe640 1 -- 172.21.15.76:0/3874834478 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8c003520 con 0x7fdb8c0047c0 2023-11-09T03:46:49.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.278+0000 7fdba37fe640 1 --2- 172.21.15.76:0/3874834478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb8c0047c0 0x7fdb8c004c40 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdba4161b30 tx=0x7fdb8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cbd4e630dd6a6ee server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:49.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.278+0000 7fdba2ffd640 1 -- 172.21.15.76:0/3874834478 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb980052d0 con 0x7fdb8c0047c0 2023-11-09T03:46:49.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.278+0000 7fdba2ffd640 1 -- 172.21.15.76:0/3874834478 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9802e050 con 0x7fdb8c0047c0 2023-11-09T03:46:49.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.278+0000 7fdba2ffd640 1 -- 172.21.15.76:0/3874834478 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb980035f0 con 0x7fdb8c0047c0 2023-11-09T03:46:49.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.279+0000 7fdbaa7b6640 1 -- 172.21.15.76:0/3874834478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb8c0047c0 msgr2=0x7fdb8c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:49.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.279+0000 7fdbaa7b6640 1 --2- 172.21.15.76:0/3874834478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb8c0047c0 0x7fdb8c004c40 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdba4161b30 tx=0x7fdb8c0057f0 comp rx=0 tx=0).stop 2023-11-09T03:46:49.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.279+0000 7fdba8d53640 1 -- 172.21.15.76:0/3874834478 reap_dead start 2023-11-09T03:46:49.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.279+0000 7fdbaa7b6640 1 -- 172.21.15.76:0/3874834478 shutdown_connections 2023-11-09T03:46:49.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.279+0000 7fdbaa7b6640 1 -- 172.21.15.76:0/3874834478 >> 172.21.15.76:0/3874834478 conn(0x7fdba40cf880 msgr2=0x7fdba40cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:49.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.279+0000 7fdbaa7b6640 1 -- 172.21.15.76:0/3874834478 shutdown_connections 2023-11-09T03:46:49.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.279+0000 7fdbaa7b6640 1 -- 172.21.15.76:0/3874834478 wait complete. 2023-11-09T03:46:49.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.280+0000 7fdbaa7b6640 1 Processor -- start 2023-11-09T03:46:49.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.280+0000 7fdbaa7b6640 1 -- start start 2023-11-09T03:46:49.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.281+0000 7fdbaa7b6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb8c0047c0 0x7fdba4145f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:49.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.281+0000 7fdbaa7b6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdba41f69e0 0x7fdba4147b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:49.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.281+0000 7fdbaa7b6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdba41480a0 0x7fdba41f5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:49.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.281+0000 7fdbaa7b6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba4156100 con 0x7fdb8c0047c0 2023-11-09T03:46:49.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.281+0000 7fdbaa7b6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba415a490 con 0x7fdba41480a0 2023-11-09T03:46:49.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.281+0000 7fdbaa7b6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba40c6900 con 0x7fdba41f69e0 2023-11-09T03:46:49.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.281+0000 7fdba3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb8c0047c0 0x7fdba4145f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:49.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.281+0000 7fdba3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb8c0047c0 0x7fdba4145f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44200/0 (socket says 172.21.15.76:44200) 2023-11-09T03:46:49.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.281+0000 7fdba3fff640 1 -- 172.21.15.76:0/1168476212 learned_addr learned my addr 172.21.15.76:0/1168476212 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:49.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.281+0000 7fdba8d53640 1 --2- 172.21.15.76:0/1168476212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdba41480a0 0x7fdba41f5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:49.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.281+0000 7fdba37fe640 1 --2- 172.21.15.76:0/1168476212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdba41f69e0 0x7fdba4147b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:49.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.281+0000 7fdba3fff640 1 -- 172.21.15.76:0/1168476212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdba41f69e0 msgr2=0x7fdba4147b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:49.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.281+0000 7fdba3fff640 1 --2- 172.21.15.76:0/1168476212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdba41f69e0 0x7fdba4147b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:49.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.281+0000 7fdba3fff640 1 -- 172.21.15.76:0/1168476212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdba41480a0 msgr2=0x7fdba41f5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:49.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.281+0000 7fdba3fff640 1 --2- 172.21.15.76:0/1168476212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdba41480a0 0x7fdba41f5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:49.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.281+0000 7fdba3fff640 1 -- 172.21.15.76:0/1168476212 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba40c6b50 con 0x7fdb8c0047c0 2023-11-09T03:46:49.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.281+0000 7fdba3fff640 1 --2- 172.21.15.76:0/1168476212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb8c0047c0 0x7fdba4145f60 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdb940072f0 tx=0x7fdb940077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:49.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.281+0000 7fdba0ff9640 1 -- 172.21.15.76:0/1168476212 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9400f890 con 0x7fdb8c0047c0 2023-11-09T03:46:49.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.281+0000 7fdba0ff9640 1 -- 172.21.15.76:0/1168476212 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9400fa30 con 0x7fdb8c0047c0 2023-11-09T03:46:49.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.281+0000 7fdbaa7b6640 1 -- 172.21.15.76:0/1168476212 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba415a6b0 con 0x7fdb8c0047c0 2023-11-09T03:46:49.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.281+0000 7fdba0ff9640 1 -- 172.21.15.76:0/1168476212 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9400fde0 con 0x7fdb8c0047c0 2023-11-09T03:46:49.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.281+0000 7fdbaa7b6640 1 -- 172.21.15.76:0/1168476212 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba41464a0 con 0x7fdb8c0047c0 2023-11-09T03:46:49.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.282+0000 7fdbaa7b6640 1 -- 172.21.15.76:0/1168476212 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba4156240 con 0x7fdb8c0047c0 2023-11-09T03:46:49.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.283+0000 7fdba0ff9640 1 -- 172.21.15.76:0/1168476212 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdb9401e3b0 con 0x7fdb8c0047c0 2023-11-09T03:46:49.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.283+0000 7fdba0ff9640 1 --2- 172.21.15.76:0/1168476212 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdb9006ae70 0x7fdb9006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:49.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.283+0000 7fdba0ff9640 1 -- 172.21.15.76:0/1168476212 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fdb94089070 con 0x7fdb8c0047c0 2023-11-09T03:46:49.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.284+0000 7fdba37fe640 1 --2- 172.21.15.76:0/1168476212 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdb9006ae70 0x7fdb9006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:49.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.284+0000 7fdba37fe640 1 --2- 172.21.15.76:0/1168476212 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdb9006ae70 0x7fdb9006d330 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdb98033000 tx=0x7fdb98009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:49.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.288+0000 7fdba0ff9640 1 -- 172.21.15.76:0/1168476212 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdba4156240 con 0x7fdb8c0047c0 2023-11-09T03:46:49.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.453+0000 7fdbaa7b6640 1 -- 172.21.15.76:0/1168476212 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdba4156240 con 0x7fdb8c0047c0 2023-11-09T03:46:49.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.454+0000 7fdba0ff9640 1 -- 172.21.15.76:0/1168476212 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdba4156240 con 0x7fdb8c0047c0 2023-11-09T03:46:49.453 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:46:49.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.456+0000 7fdbaa7b6640 1 -- 172.21.15.76:0/1168476212 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdb9006ae70 msgr2=0x7fdb9006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:49.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.456+0000 7fdbaa7b6640 1 --2- 172.21.15.76:0/1168476212 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdb9006ae70 0x7fdb9006d330 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdb98033000 tx=0x7fdb98009030 comp rx=0 tx=0).stop 2023-11-09T03:46:49.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.456+0000 7fdbaa7b6640 1 -- 172.21.15.76:0/1168476212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb8c0047c0 msgr2=0x7fdba4145f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:49.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.456+0000 7fdbaa7b6640 1 --2- 172.21.15.76:0/1168476212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb8c0047c0 0x7fdba4145f60 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdb940072f0 tx=0x7fdb940077b0 comp rx=0 tx=0).stop 2023-11-09T03:46:49.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.456+0000 7fdbaa7b6640 1 -- 172.21.15.76:0/1168476212 shutdown_connections 2023-11-09T03:46:49.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.456+0000 7fdbaa7b6640 1 --2- 172.21.15.76:0/1168476212 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdb9006ae70 0x7fdb9006d330 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:49.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.456+0000 7fdbaa7b6640 1 --2- 172.21.15.76:0/1168476212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdba41480a0 0x7fdba41f5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:49.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.456+0000 7fdbaa7b6640 1 --2- 172.21.15.76:0/1168476212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdba41f69e0 0x7fdba4147b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:49.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.456+0000 7fdbaa7b6640 1 --2- 172.21.15.76:0/1168476212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb8c0047c0 0x7fdba4145f60 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:49.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.456+0000 7fdbaa7b6640 1 -- 172.21.15.76:0/1168476212 >> 172.21.15.76:0/1168476212 conn(0x7fdba40cf880 msgr2=0x7fdba40d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:49.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.456+0000 7fdbaa7b6640 1 -- 172.21.15.76:0/1168476212 shutdown_connections 2023-11-09T03:46:49.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.456+0000 7fdbaa7b6640 1 -- 172.21.15.76:0/1168476212 wait complete. 2023-11-09T03:46:49.466 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-09T03:46:49.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.568+0000 7f1186a07640 1 Processor -- start 2023-11-09T03:46:49.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.568+0000 7f1186a07640 1 -- start start 2023-11-09T03:46:49.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.568+0000 7f1186a07640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1180159c70 0x7f11800ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:49.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.568+0000 7f1186a07640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11800b4110 con 0x7f11800b0ae0 2023-11-09T03:46:49.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.569+0000 7f1186a07640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11800b43f0 con 0x7f118006db20 2023-11-09T03:46:49.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.569+0000 7f1186a07640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11800b49b0 con 0x7f1180159c70 2023-11-09T03:46:49.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.569+0000 7f1184fa4640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f11800b0ae0 0x7f118015d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50678/0 (socket says 172.21.15.76:50678) 2023-11-09T03:46:49.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.569+0000 7f1184fa4640 1 -- 172.21.15.76:0/2221964826 learned_addr learned my addr 172.21.15.76:0/2221964826 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:49.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.569+0000 7f117ffff640 1 --2- 172.21.15.76:0/2221964826 >> v2:172.21.15.113:3300/0 conn(0x7f1180159c70 0x7f11800ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:49.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.569+0000 7f117effd640 1 -- 172.21.15.76:0/2221964826 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3942721735 0 0) 0x7f11800b4110 con 0x7f11800b0ae0 2023-11-09T03:46:49.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.569+0000 7f117effd640 1 -- 172.21.15.76:0/2221964826 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1168003520 con 0x7f11800b0ae0 2023-11-09T03:46:49.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.569+0000 7f117ffff640 1 -- 172.21.15.76:0/2221964826 >> v1:172.21.15.113:6789/0 conn(0x7f118006db20 legacy=0x7f11800b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:49.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.569+0000 7f117effd640 1 -- 172.21.15.76:0/2221964826 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3383269505 0 0) 0x7f1168003520 con 0x7f11800b0ae0 2023-11-09T03:46:49.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.569+0000 7f117ffff640 1 -- 172.21.15.76:0/2221964826 >> v1:172.21.15.76:6789/0 conn(0x7f11800b0ae0 legacy=0x7f118015d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:49.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.569+0000 7f117ffff640 1 -- 172.21.15.76:0/2221964826 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11800b46d0 con 0x7f1180159c70 2023-11-09T03:46:49.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.570+0000 7f117ffff640 1 --2- 172.21.15.76:0/2221964826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1180159c70 0x7f11800ce370 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f11740097a0 tx=0x7f117402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b36f08b72f90ef24 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:49.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.570+0000 7f117effd640 1 -- 172.21.15.76:0/2221964826 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117402f050 con 0x7f1180159c70 2023-11-09T03:46:49.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.570+0000 7f117effd640 1 -- 172.21.15.76:0/2221964826 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1174031500 con 0x7f1180159c70 2023-11-09T03:46:49.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.570+0000 7f117effd640 1 -- 172.21.15.76:0/2221964826 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11780019f0 con 0x7f1180159c70 2023-11-09T03:46:49.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.571+0000 7f1186a07640 1 -- 172.21.15.76:0/2221964826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1180159c70 msgr2=0x7f11800ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:49.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.571+0000 7f1186a07640 1 --2- 172.21.15.76:0/2221964826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1180159c70 0x7f11800ce370 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f11740097a0 tx=0x7f117402b820 comp rx=0 tx=0).stop 2023-11-09T03:46:49.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.571+0000 7f1186a07640 1 -- 172.21.15.76:0/2221964826 shutdown_connections 2023-11-09T03:46:49.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.571+0000 7f1186a07640 1 --2- 172.21.15.76:0/2221964826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1180159c70 0x7f11800ce370 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:49.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.571+0000 7f1186a07640 1 -- 172.21.15.76:0/2221964826 >> 172.21.15.76:0/2221964826 conn(0x7f11800ce950 msgr2=0x7f11800ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:49.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.571+0000 7f1186a07640 1 -- 172.21.15.76:0/2221964826 shutdown_connections 2023-11-09T03:46:49.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.571+0000 7f1186a07640 1 -- 172.21.15.76:0/2221964826 wait complete. 2023-11-09T03:46:49.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.572+0000 7f1186a07640 1 Processor -- start 2023-11-09T03:46:49.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.572+0000 7f1186a07640 1 -- start start 2023-11-09T03:46:49.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.573+0000 7f1186a07640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f118006db20 0x7f1180154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:49.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.573+0000 7f1186a07640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11800b0ae0 0x7f118014e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:49.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.573+0000 7f1186a07640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1180159c70 0x7f118014ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:49.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.573+0000 7f1186a07640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118014f440 con 0x7f11800b0ae0 2023-11-09T03:46:49.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.573+0000 7f1186a07640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118014fb90 con 0x7f118006db20 2023-11-09T03:46:49.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.573+0000 7f1186a07640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1180154700 con 0x7f1180159c70 2023-11-09T03:46:49.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.573+0000 7f117f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11800b0ae0 0x7f118014e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:49.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.573+0000 7f117f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11800b0ae0 0x7f118014e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44206/0 (socket says 172.21.15.76:44206) 2023-11-09T03:46:49.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.573+0000 7f117f7fe640 1 -- 172.21.15.76:0/1454461552 learned_addr learned my addr 172.21.15.76:0/1454461552 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:49.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.573+0000 7f1184fa4640 1 --2- 172.21.15.76:0/1454461552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1180159c70 0x7f118014ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:49.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.573+0000 7f117ffff640 1 --2- 172.21.15.76:0/1454461552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f118006db20 0x7f1180154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:49.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.573+0000 7f117f7fe640 1 -- 172.21.15.76:0/1454461552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1180159c70 msgr2=0x7f118014ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:49.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.573+0000 7f117f7fe640 1 --2- 172.21.15.76:0/1454461552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1180159c70 0x7f118014ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:49.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.573+0000 7f117f7fe640 1 -- 172.21.15.76:0/1454461552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f118006db20 msgr2=0x7f1180154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:49.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.573+0000 7f117f7fe640 1 --2- 172.21.15.76:0/1454461552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f118006db20 0x7f1180154d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:49.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.573+0000 7f117f7fe640 1 -- 172.21.15.76:0/1454461552 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1174009400 con 0x7f11800b0ae0 2023-11-09T03:46:49.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.573+0000 7f117ffff640 1 --2- 172.21.15.76:0/1454461552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f118006db20 0x7f1180154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:49.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.573+0000 7f117f7fe640 1 --2- 172.21.15.76:0/1454461552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11800b0ae0 0x7f118014e9c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f117000e700 tx=0x7f117000ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:49.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.574+0000 7f117cff9640 1 -- 172.21.15.76:0/1454461552 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11700052d0 con 0x7f11800b0ae0 2023-11-09T03:46:49.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.574+0000 7f117cff9640 1 -- 172.21.15.76:0/1454461552 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1170005430 con 0x7f11800b0ae0 2023-11-09T03:46:49.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.574+0000 7f1186a07640 1 -- 172.21.15.76:0/1454461552 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1180153e60 con 0x7f11800b0ae0 2023-11-09T03:46:49.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.574+0000 7f117cff9640 1 -- 172.21.15.76:0/1454461552 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117000cca0 con 0x7f11800b0ae0 2023-11-09T03:46:49.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.574+0000 7f1186a07640 1 -- 172.21.15.76:0/1454461552 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1180154420 con 0x7f11800b0ae0 2023-11-09T03:46:49.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.574+0000 7f1186a07640 1 -- 172.21.15.76:0/1454461552 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1180074290 con 0x7f11800b0ae0 2023-11-09T03:46:49.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.575+0000 7f117cff9640 1 -- 172.21.15.76:0/1454461552 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f11700035f0 con 0x7f11800b0ae0 2023-11-09T03:46:49.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.575+0000 7f117cff9640 1 --2- 172.21.15.76:0/1454461552 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f116c06aea0 0x7f116c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:49.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.575+0000 7f117cff9640 1 -- 172.21.15.76:0/1454461552 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f117008b650 con 0x7f11800b0ae0 2023-11-09T03:46:49.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.576+0000 7f117ffff640 1 --2- 172.21.15.76:0/1454461552 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f116c06aea0 0x7f116c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:49.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.576+0000 7f117ffff640 1 --2- 172.21.15.76:0/1454461552 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f116c06aea0 0x7f116c06d360 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1174035000 tx=0x7f1174008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:49.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.580+0000 7f117cff9640 1 -- 172.21.15.76:0/1454461552 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1170056890 con 0x7f11800b0ae0 2023-11-09T03:46:49.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.751+0000 7f1186a07640 1 -- 172.21.15.76:0/1454461552 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f118006d3f0 con 0x7f11800b0ae0 2023-11-09T03:46:49.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.752+0000 7f117cff9640 1 -- 172.21.15.76:0/1454461552 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1170059eb0 con 0x7f11800b0ae0 2023-11-09T03:46:49.752 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:49.752 INFO:teuthology.orchestra.run.smithi076.stdout:[ 2023-11-09T03:46:49.752 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:46:49.752 INFO:teuthology.orchestra.run.smithi076.stdout: "poolnum": 1, 2023-11-09T03:46:49.752 INFO:teuthology.orchestra.run.smithi076.stdout: "poolname": ".mgr" 2023-11-09T03:46:49.753 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:49.753 INFO:teuthology.orchestra.run.smithi076.stdout:] 2023-11-09T03:46:49.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.754+0000 7f1186a07640 1 -- 172.21.15.76:0/1454461552 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f116c06aea0 msgr2=0x7f116c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:49.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.754+0000 7f1186a07640 1 --2- 172.21.15.76:0/1454461552 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f116c06aea0 0x7f116c06d360 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1174035000 tx=0x7f1174008030 comp rx=0 tx=0).stop 2023-11-09T03:46:49.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.754+0000 7f1186a07640 1 -- 172.21.15.76:0/1454461552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11800b0ae0 msgr2=0x7f118014e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:49.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.754+0000 7f1186a07640 1 --2- 172.21.15.76:0/1454461552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11800b0ae0 0x7f118014e9c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f117000e700 tx=0x7f117000ebc0 comp rx=0 tx=0).stop 2023-11-09T03:46:49.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.754+0000 7f1186a07640 1 -- 172.21.15.76:0/1454461552 shutdown_connections 2023-11-09T03:46:49.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.754+0000 7f1186a07640 1 --2- 172.21.15.76:0/1454461552 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f116c06aea0 0x7f116c06d360 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:49.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.754+0000 7f1186a07640 1 --2- 172.21.15.76:0/1454461552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1180159c70 0x7f118014ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:49.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.754+0000 7f1186a07640 1 --2- 172.21.15.76:0/1454461552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11800b0ae0 0x7f118014e9c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:49.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.754+0000 7f1186a07640 1 --2- 172.21.15.76:0/1454461552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f118006db20 0x7f1180154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:49.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.754+0000 7f1186a07640 1 -- 172.21.15.76:0/1454461552 >> 172.21.15.76:0/1454461552 conn(0x7f11800ce950 msgr2=0x7f11800b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:49.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.754+0000 7f1186a07640 1 -- 172.21.15.76:0/1454461552 shutdown_connections 2023-11-09T03:46:49.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.754+0000 7f1186a07640 1 -- 172.21.15.76:0/1454461552 wait complete. 2023-11-09T03:46:49.763 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-09T03:46:49.764 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-09T03:46:49.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.867+0000 7fe9e9b05640 1 Processor -- start 2023-11-09T03:46:49.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.868+0000 7fe9e9b05640 1 -- start start 2023-11-09T03:46:49.869 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.868+0000 7fe9e9b05640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe9e415e420 0x7fe9e40ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:49.869 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.868+0000 7fe9e9b05640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9e40c5890 con 0x7fe9e4159be0 2023-11-09T03:46:49.869 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.868+0000 7fe9e9b05640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9e40cd8e0 con 0x7fe9e40adc70 2023-11-09T03:46:49.870 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.868+0000 7fe9e9b05640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9e40c5670 con 0x7fe9e415e420 2023-11-09T03:46:49.870 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.868+0000 7fe9e37fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe9e4159be0 0x7fe9e40c4f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50680/0 (socket says 172.21.15.76:50680) 2023-11-09T03:46:49.870 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.868+0000 7fe9e37fe640 1 -- 172.21.15.76:0/3332486056 learned_addr learned my addr 172.21.15.76:0/3332486056 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:49.870 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.869+0000 7fe9e2ffd640 1 --2- 172.21.15.76:0/3332486056 >> v2:172.21.15.113:3300/0 conn(0x7fe9e415e420 0x7fe9e40ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:49.870 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.869+0000 7fe9e27fc640 1 -- 172.21.15.76:0/3332486056 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1575142367 0 0) 0x7fe9e40c5890 con 0x7fe9e4159be0 2023-11-09T03:46:49.871 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.869+0000 7fe9e27fc640 1 -- 172.21.15.76:0/3332486056 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9cc003520 con 0x7fe9e4159be0 2023-11-09T03:46:49.871 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.869+0000 7fe9e27fc640 1 -- 172.21.15.76:0/3332486056 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3463114807 0 0) 0x7fe9e40cd8e0 con 0x7fe9e40adc70 2023-11-09T03:46:49.871 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.869+0000 7fe9e27fc640 1 -- 172.21.15.76:0/3332486056 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9e40c5890 con 0x7fe9e40adc70 2023-11-09T03:46:49.871 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.869+0000 7fe9e27fc640 1 -- 172.21.15.76:0/3332486056 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 276649094 0 0) 0x7fe9cc003520 con 0x7fe9e4159be0 2023-11-09T03:46:49.871 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.869+0000 7fe9e2ffd640 1 -- 172.21.15.76:0/3332486056 >> v1:172.21.15.113:6789/0 conn(0x7fe9e40adc70 legacy=0x7fe9e40c3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:49.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.869+0000 7fe9e2ffd640 1 -- 172.21.15.76:0/3332486056 >> v1:172.21.15.76:6789/0 conn(0x7fe9e4159be0 legacy=0x7fe9e40c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:49.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.869+0000 7fe9e2ffd640 1 -- 172.21.15.76:0/3332486056 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e40cdbc0 con 0x7fe9e415e420 2023-11-09T03:46:49.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.870+0000 7fe9e2ffd640 1 --2- 172.21.15.76:0/3332486056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9e415e420 0x7fe9e40ad730 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe9d80097a0 tx=0x7fe9d802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2575d0f31dfb68f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:49.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.870+0000 7fe9e27fc640 1 -- 172.21.15.76:0/3332486056 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4001670 con 0x7fe9e415e420 2023-11-09T03:46:49.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.870+0000 7fe9e27fc640 1 -- 172.21.15.76:0/3332486056 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d802e050 con 0x7fe9e415e420 2023-11-09T03:46:49.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.870+0000 7fe9e27fc640 1 -- 172.21.15.76:0/3332486056 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8030710 con 0x7fe9e415e420 2023-11-09T03:46:49.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.870+0000 7fe9e9b05640 1 -- 172.21.15.76:0/3332486056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9e415e420 msgr2=0x7fe9e40ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:49.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.870+0000 7fe9e9b05640 1 --2- 172.21.15.76:0/3332486056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9e415e420 0x7fe9e40ad730 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe9d80097a0 tx=0x7fe9d802b790 comp rx=0 tx=0).stop 2023-11-09T03:46:49.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.870+0000 7fe9e9b05640 1 -- 172.21.15.76:0/3332486056 shutdown_connections 2023-11-09T03:46:49.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.870+0000 7fe9e9b05640 1 --2- 172.21.15.76:0/3332486056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9e415e420 0x7fe9e40ad730 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:49.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.870+0000 7fe9e9b05640 1 -- 172.21.15.76:0/3332486056 >> 172.21.15.76:0/3332486056 conn(0x7fe9e40b05d0 msgr2=0x7fe9e40b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:49.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.870+0000 7fe9e9b05640 1 -- 172.21.15.76:0/3332486056 shutdown_connections 2023-11-09T03:46:49.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.871+0000 7fe9e9b05640 1 -- 172.21.15.76:0/3332486056 wait complete. 2023-11-09T03:46:49.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.872+0000 7fe9e9b05640 1 Processor -- start 2023-11-09T03:46:49.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.872+0000 7fe9e9b05640 1 -- start start 2023-11-09T03:46:49.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.872+0000 7fe9e9b05640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9e40adc70 0x7fe9e4150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:49.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.873+0000 7fe9e9b05640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9e4159be0 0x7fe9e406c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:49.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.873+0000 7fe9e9b05640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9e415e420 0x7fe9e406cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:49.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.873+0000 7fe9e9b05640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e41579b0 con 0x7fe9e4159be0 2023-11-09T03:46:49.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.873+0000 7fe9e9b05640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e40ac8a0 con 0x7fe9e415e420 2023-11-09T03:46:49.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.873+0000 7fe9e9b05640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e40acb80 con 0x7fe9e40adc70 2023-11-09T03:46:49.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.873+0000 7fe9e2ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9e4159be0 0x7fe9e406c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:49.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.873+0000 7fe9e2ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9e4159be0 0x7fe9e406c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44222/0 (socket says 172.21.15.76:44222) 2023-11-09T03:46:49.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.873+0000 7fe9e2ffd640 1 -- 172.21.15.76:0/4219054585 learned_addr learned my addr 172.21.15.76:0/4219054585 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:49.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.873+0000 7fe9e37fe640 1 --2- 172.21.15.76:0/4219054585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9e40adc70 0x7fe9e4150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:49.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.873+0000 7fe9e3fff640 1 --2- 172.21.15.76:0/4219054585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9e415e420 0x7fe9e406cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:49.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.873+0000 7fe9e2ffd640 1 -- 172.21.15.76:0/4219054585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9e40adc70 msgr2=0x7fe9e4150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:49.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.873+0000 7fe9e2ffd640 1 --2- 172.21.15.76:0/4219054585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9e40adc70 0x7fe9e4150890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:49.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.873+0000 7fe9e2ffd640 1 -- 172.21.15.76:0/4219054585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9e415e420 msgr2=0x7fe9e406cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:49.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.873+0000 7fe9e2ffd640 1 --2- 172.21.15.76:0/4219054585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9e415e420 0x7fe9e406cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:49.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.873+0000 7fe9e2ffd640 1 -- 172.21.15.76:0/4219054585 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d8009400 con 0x7fe9e4159be0 2023-11-09T03:46:49.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.873+0000 7fe9e2ffd640 1 --2- 172.21.15.76:0/4219054585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9e4159be0 0x7fe9e406c710 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8000c00 tx=0x7fe9d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:49.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.873+0000 7fe9cbfff640 1 -- 172.21.15.76:0/4219054585 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d803c620 con 0x7fe9e4159be0 2023-11-09T03:46:49.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.873+0000 7fe9cbfff640 1 -- 172.21.15.76:0/4219054585 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d803c7a0 con 0x7fe9e4159be0 2023-11-09T03:46:49.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.873+0000 7fe9cbfff640 1 -- 172.21.15.76:0/4219054585 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d803cb50 con 0x7fe9e4159be0 2023-11-09T03:46:49.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.873+0000 7fe9e9b05640 1 -- 172.21.15.76:0/4219054585 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e41572f0 con 0x7fe9e4159be0 2023-11-09T03:46:49.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.873+0000 7fe9e9b05640 1 -- 172.21.15.76:0/4219054585 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e40abb70 con 0x7fe9e4159be0 2023-11-09T03:46:49.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.874+0000 7fe9e9b05640 1 -- 172.21.15.76:0/4219054585 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9e404e490 con 0x7fe9e4159be0 2023-11-09T03:46:49.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.875+0000 7fe9cbfff640 1 -- 172.21.15.76:0/4219054585 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe9d8039070 con 0x7fe9e4159be0 2023-11-09T03:46:49.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.875+0000 7fe9cbfff640 1 --2- 172.21.15.76:0/4219054585 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe9d006ae70 0x7fe9d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:49.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.875+0000 7fe9cbfff640 1 -- 172.21.15.76:0/4219054585 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fe9d80aaaf0 con 0x7fe9e4159be0 2023-11-09T03:46:49.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.875+0000 7fe9e37fe640 1 --2- 172.21.15.76:0/4219054585 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe9d006ae70 0x7fe9d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:49.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.876+0000 7fe9e37fe640 1 --2- 172.21.15.76:0/4219054585 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe9d006ae70 0x7fe9d006d330 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4000fd0 tx=0x7fe9d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:49.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:49.879+0000 7fe9cbfff640 1 -- 172.21.15.76:0/4219054585 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe9d8075e40 con 0x7fe9e4159be0 2023-11-09T03:46:50.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.046+0000 7fe9e9b05640 1 -- 172.21.15.76:0/4219054585 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe9e40b0ab0 con 0x7fe9e4159be0 2023-11-09T03:46:50.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.516+0000 7fe9cbfff640 1 -- 172.21.15.76:0/4219054585 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe9d80370b0 con 0x7fe9e4159be0 2023-11-09T03:46:50.515 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' created 2023-11-09T03:46:50.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.518+0000 7fe9e9b05640 1 -- 172.21.15.76:0/4219054585 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe9d006ae70 msgr2=0x7fe9d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:50.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.518+0000 7fe9e9b05640 1 --2- 172.21.15.76:0/4219054585 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe9d006ae70 0x7fe9d006d330 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4000fd0 tx=0x7fe9d4009040 comp rx=0 tx=0).stop 2023-11-09T03:46:50.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.518+0000 7fe9e9b05640 1 -- 172.21.15.76:0/4219054585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9e4159be0 msgr2=0x7fe9e406c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:50.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.518+0000 7fe9e9b05640 1 --2- 172.21.15.76:0/4219054585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9e4159be0 0x7fe9e406c710 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8000c00 tx=0x7fe9d80046c0 comp rx=0 tx=0).stop 2023-11-09T03:46:50.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.518+0000 7fe9e9b05640 1 -- 172.21.15.76:0/4219054585 shutdown_connections 2023-11-09T03:46:50.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.518+0000 7fe9e9b05640 1 --2- 172.21.15.76:0/4219054585 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe9d006ae70 0x7fe9d006d330 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:50.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.518+0000 7fe9e9b05640 1 --2- 172.21.15.76:0/4219054585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9e415e420 0x7fe9e406cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:50.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.518+0000 7fe9e9b05640 1 --2- 172.21.15.76:0/4219054585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9e4159be0 0x7fe9e406c710 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:50.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.518+0000 7fe9e9b05640 1 --2- 172.21.15.76:0/4219054585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9e40adc70 0x7fe9e4150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:50.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.518+0000 7fe9e9b05640 1 -- 172.21.15.76:0/4219054585 >> 172.21.15.76:0/4219054585 conn(0x7fe9e40b05d0 msgr2=0x7fe9e40c3840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:50.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.518+0000 7fe9e9b05640 1 -- 172.21.15.76:0/4219054585 shutdown_connections 2023-11-09T03:46:50.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.518+0000 7fe9e9b05640 1 -- 172.21.15.76:0/4219054585 wait complete. 2023-11-09T03:46:50.531 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-09T03:46:50.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.639+0000 7fe0a4f9e640 1 Processor -- start 2023-11-09T03:46:50.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.639+0000 7fe0a4f9e640 1 -- start start 2023-11-09T03:46:50.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.640+0000 7fe0a4f9e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe0a00cee40 0x7fe0a00ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:50.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.640+0000 7fe0a4f9e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0a015e4e0 0x7fe0a00b0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:50.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.640+0000 7fe0a4f9e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0a00cf930 con 0x7fe0a00d1460 2023-11-09T03:46:50.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.640+0000 7fe0a4f9e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0a00cea50 con 0x7fe0a00cee40 2023-11-09T03:46:50.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.640+0000 7fe0a4f9e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a00d0290 con 0x7fe0a015e4e0 2023-11-09T03:46:50.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.640+0000 7fe09f7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe0a00cee40 0x7fe0a00ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:50.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.640+0000 7fe09ffff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0a015e4e0 0x7fe0a00b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:50.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.641+0000 7fe09f7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe0a00cee40 0x7fe0a00ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:49800/0 (socket says 172.21.15.76:49800) 2023-11-09T03:46:50.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.641+0000 7fe09ffff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0a015e4e0 0x7fe0a00b0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:44660/0 (socket says 172.21.15.76:44660) 2023-11-09T03:46:50.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.641+0000 7fe09effd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe0a00d1460 0x7fe0a015ddd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:45880/0 (socket says 172.21.15.76:45880) 2023-11-09T03:46:50.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.641+0000 7fe09f7fe640 1 -- 172.21.15.76:0/333080916 learned_addr learned my addr 172.21.15.76:0/333080916 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:50.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.641+0000 7fe09e7fc640 1 -- 172.21.15.76:0/333080916 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 169046359 0 0) 0x7fe0a00cf930 con 0x7fe0a00d1460 2023-11-09T03:46:50.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.641+0000 7fe09e7fc640 1 -- 172.21.15.76:0/333080916 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe088003520 con 0x7fe0a00d1460 2023-11-09T03:46:50.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.641+0000 7fe09ffff640 1 -- 172.21.15.76:0/333080916 >> v2:172.21.15.113:3300/0 conn(0x7fe0a00cee40 msgr2=0x7fe0a00ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:50.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.641+0000 7fe09ffff640 1 --2- 172.21.15.76:0/333080916 >> v2:172.21.15.113:3300/0 conn(0x7fe0a00cee40 0x7fe0a00ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:50.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.641+0000 7fe09ffff640 1 -- 172.21.15.76:0/333080916 >> v1:172.21.15.113:6789/0 conn(0x7fe0a00d1460 legacy=0x7fe0a015ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:50.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.641+0000 7fe09ffff640 1 -- 172.21.15.76:0/333080916 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a00b0440 con 0x7fe0a015e4e0 2023-11-09T03:46:50.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.641+0000 7fe09f7fe640 1 --2- 172.21.15.76:0/333080916 >> v2:172.21.15.113:3300/0 conn(0x7fe0a00cee40 0x7fe0a00ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:46:50.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.642+0000 7fe09ffff640 1 --2- 172.21.15.76:0/333080916 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0a015e4e0 0x7fe0a00b0920 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe098009d90 tx=0x7fe09802bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fa24f42b37b9b10 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:50.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.642+0000 7fe09e7fc640 1 -- 172.21.15.76:0/333080916 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09802cc50 con 0x7fe0a015e4e0 2023-11-09T03:46:50.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.642+0000 7fe09e7fc640 1 -- 172.21.15.76:0/333080916 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09802cdb0 con 0x7fe0a015e4e0 2023-11-09T03:46:50.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.642+0000 7fe09e7fc640 1 -- 172.21.15.76:0/333080916 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe098034640 con 0x7fe0a015e4e0 2023-11-09T03:46:50.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.642+0000 7fe0a4f9e640 1 -- 172.21.15.76:0/333080916 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0a015e4e0 msgr2=0x7fe0a00b0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:50.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.642+0000 7fe0a4f9e640 1 --2- 172.21.15.76:0/333080916 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0a015e4e0 0x7fe0a00b0920 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe098009d90 tx=0x7fe09802bea0 comp rx=0 tx=0).stop 2023-11-09T03:46:50.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.642+0000 7fe0a4f9e640 1 -- 172.21.15.76:0/333080916 shutdown_connections 2023-11-09T03:46:50.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.642+0000 7fe0a4f9e640 1 --2- 172.21.15.76:0/333080916 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0a015e4e0 0x7fe0a00b0920 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:50.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.642+0000 7fe0a4f9e640 1 --2- 172.21.15.76:0/333080916 >> v2:172.21.15.113:3300/0 conn(0x7fe0a00cee40 0x7fe0a00ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:50.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.642+0000 7fe0a4f9e640 1 -- 172.21.15.76:0/333080916 >> 172.21.15.76:0/333080916 conn(0x7fe0a00d0940 msgr2=0x7fe0a00d0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:50.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.642+0000 7fe0a4f9e640 1 -- 172.21.15.76:0/333080916 shutdown_connections 2023-11-09T03:46:50.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.642+0000 7fe0a4f9e640 1 -- 172.21.15.76:0/333080916 wait complete. 2023-11-09T03:46:50.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.644+0000 7fe0a4f9e640 1 Processor -- start 2023-11-09T03:46:50.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.644+0000 7fe0a4f9e640 1 -- start start 2023-11-09T03:46:50.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.644+0000 7fe0a4f9e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0a00cee40 0x7fe0a006f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:50.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.644+0000 7fe0a4f9e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0a00d1460 0x7fe0a006bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:50.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.644+0000 7fe0a4f9e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0a015e4e0 0x7fe0a015a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:50.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.644+0000 7fe0a4f9e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a006c180 con 0x7fe0a00cee40 2023-11-09T03:46:50.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.645+0000 7fe0a4f9e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a006f890 con 0x7fe0a015e4e0 2023-11-09T03:46:50.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.645+0000 7fe0a4f9e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a006fb70 con 0x7fe0a00d1460 2023-11-09T03:46:50.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.645+0000 7fe09f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0a00cee40 0x7fe0a006f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:50.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.645+0000 7fe09f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0a00cee40 0x7fe0a006f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44228/0 (socket says 172.21.15.76:44228) 2023-11-09T03:46:50.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.645+0000 7fe09f7fe640 1 -- 172.21.15.76:0/2133081493 learned_addr learned my addr 172.21.15.76:0/2133081493 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:50.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.645+0000 7fe09effd640 1 --2- 172.21.15.76:0/2133081493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0a00d1460 0x7fe0a006bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:50.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.645+0000 7fe09ffff640 1 --2- 172.21.15.76:0/2133081493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0a015e4e0 0x7fe0a015a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:50.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.645+0000 7fe09f7fe640 1 -- 172.21.15.76:0/2133081493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0a00d1460 msgr2=0x7fe0a006bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:50.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.645+0000 7fe09f7fe640 1 --2- 172.21.15.76:0/2133081493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0a00d1460 0x7fe0a006bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:50.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.645+0000 7fe09f7fe640 1 -- 172.21.15.76:0/2133081493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0a015e4e0 msgr2=0x7fe0a015a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:50.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.645+0000 7fe09f7fe640 1 --2- 172.21.15.76:0/2133081493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0a015e4e0 0x7fe0a015a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:50.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.645+0000 7fe09f7fe640 1 -- 172.21.15.76:0/2133081493 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe090009460 con 0x7fe0a00cee40 2023-11-09T03:46:50.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.645+0000 7fe09ffff640 1 --2- 172.21.15.76:0/2133081493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0a015e4e0 0x7fe0a015a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:50.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.645+0000 7fe09f7fe640 1 --2- 172.21.15.76:0/2133081493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0a00cee40 0x7fe0a006f260 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe09000e9b0 tx=0x7fe09000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:50.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.645+0000 7fe087fff640 1 -- 172.21.15.76:0/2133081493 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09000ca00 con 0x7fe0a00cee40 2023-11-09T03:46:50.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.645+0000 7fe087fff640 1 -- 172.21.15.76:0/2133081493 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09000cb60 con 0x7fe0a00cee40 2023-11-09T03:46:50.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.645+0000 7fe0a4f9e640 1 -- 172.21.15.76:0/2133081493 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe098009a70 con 0x7fe0a00cee40 2023-11-09T03:46:50.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.645+0000 7fe087fff640 1 -- 172.21.15.76:0/2133081493 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe090005990 con 0x7fe0a00cee40 2023-11-09T03:46:50.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.645+0000 7fe0a4f9e640 1 -- 172.21.15.76:0/2133081493 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a0070130 con 0x7fe0a00cee40 2023-11-09T03:46:50.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.646+0000 7fe0a4f9e640 1 -- 172.21.15.76:0/2133081493 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0a004ece0 con 0x7fe0a00cee40 2023-11-09T03:46:50.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.647+0000 7fe087fff640 1 -- 172.21.15.76:0/2133081493 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe090005af0 con 0x7fe0a00cee40 2023-11-09T03:46:50.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.647+0000 7fe087fff640 1 --2- 172.21.15.76:0/2133081493 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe08c06ae70 0x7fe08c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:50.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.647+0000 7fe087fff640 1 -- 172.21.15.76:0/2133081493 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5547+0+0 (secure 0 0 0) 0x7fe090089990 con 0x7fe0a00cee40 2023-11-09T03:46:50.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.647+0000 7fe09effd640 1 --2- 172.21.15.76:0/2133081493 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe08c06ae70 0x7fe08c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:50.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.648+0000 7fe09effd640 1 --2- 172.21.15.76:0/2133081493 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe08c06ae70 0x7fe08c06d330 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe0a01f5840 tx=0x7fe094007cf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:50.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.652+0000 7fe087fff640 1 -- 172.21.15.76:0/2133081493 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe090054b90 con 0x7fe0a00cee40 2023-11-09T03:46:50.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:50.819+0000 7fe0a4f9e640 1 -- 172.21.15.76:0/2133081493 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe0a015bc10 con 0x7fe0a00cee40 2023-11-09T03:46:51.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.523+0000 7fe087fff640 1 -- 172.21.15.76:0/2133081493 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe090020070 con 0x7fe0a00cee40 2023-11-09T03:46:51.522 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' created 2023-11-09T03:46:51.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.527+0000 7fe0a4f9e640 1 -- 172.21.15.76:0/2133081493 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe08c06ae70 msgr2=0x7fe08c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:51.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.527+0000 7fe0a4f9e640 1 --2- 172.21.15.76:0/2133081493 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe08c06ae70 0x7fe08c06d330 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe0a01f5840 tx=0x7fe094007cf0 comp rx=0 tx=0).stop 2023-11-09T03:46:51.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.527+0000 7fe0a4f9e640 1 -- 172.21.15.76:0/2133081493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0a00cee40 msgr2=0x7fe0a006f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:51.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.527+0000 7fe0a4f9e640 1 --2- 172.21.15.76:0/2133081493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0a00cee40 0x7fe0a006f260 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe09000e9b0 tx=0x7fe09000ee70 comp rx=0 tx=0).stop 2023-11-09T03:46:51.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.527+0000 7fe0a4f9e640 1 -- 172.21.15.76:0/2133081493 shutdown_connections 2023-11-09T03:46:51.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.527+0000 7fe0a4f9e640 1 --2- 172.21.15.76:0/2133081493 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe08c06ae70 0x7fe08c06d330 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:51.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.527+0000 7fe0a4f9e640 1 --2- 172.21.15.76:0/2133081493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0a015e4e0 0x7fe0a015a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:51.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.527+0000 7fe0a4f9e640 1 --2- 172.21.15.76:0/2133081493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0a00d1460 0x7fe0a006bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:51.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.527+0000 7fe0a4f9e640 1 --2- 172.21.15.76:0/2133081493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0a00cee40 0x7fe0a006f260 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:51.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.527+0000 7fe0a4f9e640 1 -- 172.21.15.76:0/2133081493 >> 172.21.15.76:0/2133081493 conn(0x7fe0a00d0940 msgr2=0x7fe0a0154f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:51.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.527+0000 7fe0a4f9e640 1 -- 172.21.15.76:0/2133081493 shutdown_connections 2023-11-09T03:46:51.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.527+0000 7fe0a4f9e640 1 -- 172.21.15.76:0/2133081493 wait complete. 2023-11-09T03:46:51.552 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-09T03:46:51.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.660+0000 7fc559733640 1 Processor -- start 2023-11-09T03:46:51.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.660+0000 7fc559733640 1 -- start start 2023-11-09T03:46:51.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.661+0000 7fc559733640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc55415ccc0 0x7fc5540b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:51.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.661+0000 7fc559733640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc5540cdd90 0x7fc5540b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:51.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.661+0000 7fc559733640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc554160df0 con 0x7fc55415d0a0 2023-11-09T03:46:51.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.661+0000 7fc559733640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5540ce260 con 0x7fc55415ccc0 2023-11-09T03:46:51.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.661+0000 7fc559733640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5540ce540 con 0x7fc5540cdd90 2023-11-09T03:46:51.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.661+0000 7fc552ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc55415ccc0 0x7fc5540b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:51.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.661+0000 7fc552ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc55415ccc0 0x7fc5540b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44234/0 (socket says 172.21.15.76:44234) 2023-11-09T03:46:51.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.661+0000 7fc552ffd640 1 -- 172.21.15.76:0/3413551539 learned_addr learned my addr 172.21.15.76:0/3413551539 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:51.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.662+0000 7fc5537fe640 1 --2- 172.21.15.76:0/3413551539 >> v2:172.21.15.113:3300/0 conn(0x7fc5540cdd90 0x7fc5540b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:51.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.662+0000 7fc552ffd640 1 -- 172.21.15.76:0/3413551539 >> v2:172.21.15.113:3300/0 conn(0x7fc5540cdd90 msgr2=0x7fc5540b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:51.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.662+0000 7fc552ffd640 1 --2- 172.21.15.76:0/3413551539 >> v2:172.21.15.113:3300/0 conn(0x7fc5540cdd90 0x7fc5540b4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:51.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.662+0000 7fc552ffd640 1 -- 172.21.15.76:0/3413551539 >> v1:172.21.15.113:6789/0 conn(0x7fc55415d0a0 legacy=0x7fc5540cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:51.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.662+0000 7fc552ffd640 1 -- 172.21.15.76:0/3413551539 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5540d1050 con 0x7fc55415ccc0 2023-11-09T03:46:51.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.662+0000 7fc552ffd640 1 --2- 172.21.15.76:0/3413551539 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc55415ccc0 0x7fc5540b48a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc5540ce800 tx=0x7fc54c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90807dcd0586a406 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:51.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.662+0000 7fc551ffb640 1 -- 172.21.15.76:0/3413551539 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c02ca80 con 0x7fc55415ccc0 2023-11-09T03:46:51.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.663+0000 7fc551ffb640 1 -- 172.21.15.76:0/3413551539 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54c02cbe0 con 0x7fc55415ccc0 2023-11-09T03:46:51.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.663+0000 7fc551ffb640 1 -- 172.21.15.76:0/3413551539 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c0343f0 con 0x7fc55415ccc0 2023-11-09T03:46:51.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.663+0000 7fc559733640 1 -- 172.21.15.76:0/3413551539 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc55415ccc0 msgr2=0x7fc5540b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:51.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.663+0000 7fc559733640 1 --2- 172.21.15.76:0/3413551539 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc55415ccc0 0x7fc5540b48a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc5540ce800 tx=0x7fc54c02bee0 comp rx=0 tx=0).stop 2023-11-09T03:46:51.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.663+0000 7fc559733640 1 -- 172.21.15.76:0/3413551539 shutdown_connections 2023-11-09T03:46:51.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.663+0000 7fc559733640 1 --2- 172.21.15.76:0/3413551539 >> v2:172.21.15.113:3300/0 conn(0x7fc5540cdd90 0x7fc5540b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:51.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.663+0000 7fc559733640 1 --2- 172.21.15.76:0/3413551539 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc55415ccc0 0x7fc5540b48a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:51.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.663+0000 7fc559733640 1 -- 172.21.15.76:0/3413551539 >> 172.21.15.76:0/3413551539 conn(0x7fc5540cf8a0 msgr2=0x7fc5540cfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:51.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.663+0000 7fc559733640 1 -- 172.21.15.76:0/3413551539 shutdown_connections 2023-11-09T03:46:51.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.663+0000 7fc559733640 1 -- 172.21.15.76:0/3413551539 wait complete. 2023-11-09T03:46:51.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.664+0000 7fc559733640 1 Processor -- start 2023-11-09T03:46:51.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.664+0000 7fc559733640 1 -- start start 2023-11-09T03:46:51.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.665+0000 7fc559733640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5540cdd90 0x7fc5540c6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:51.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.665+0000 7fc559733640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc55415ccc0 0x7fc5540b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:51.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.665+0000 7fc559733640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc55415d0a0 0x7fc5540b4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:51.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.665+0000 7fc559733640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5540c97c0 con 0x7fc55415ccc0 2023-11-09T03:46:51.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.665+0000 7fc559733640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5540c94e0 con 0x7fc55415d0a0 2023-11-09T03:46:51.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.665+0000 7fc559733640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc55415f8d0 con 0x7fc5540cdd90 2023-11-09T03:46:51.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.665+0000 7fc5527fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc55415ccc0 0x7fc5540b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:51.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.665+0000 7fc5527fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc55415ccc0 0x7fc5540b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44240/0 (socket says 172.21.15.76:44240) 2023-11-09T03:46:51.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.665+0000 7fc5527fc640 1 -- 172.21.15.76:0/1979581781 learned_addr learned my addr 172.21.15.76:0/1979581781 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:51.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.665+0000 7fc5537fe640 1 --2- 172.21.15.76:0/1979581781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc55415d0a0 0x7fc5540b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:51.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.665+0000 7fc552ffd640 1 --2- 172.21.15.76:0/1979581781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5540cdd90 0x7fc5540c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:51.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.666+0000 7fc5527fc640 1 -- 172.21.15.76:0/1979581781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5540cdd90 msgr2=0x7fc5540c6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:51.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.666+0000 7fc5527fc640 1 --2- 172.21.15.76:0/1979581781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5540cdd90 0x7fc5540c6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:51.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.666+0000 7fc5527fc640 1 -- 172.21.15.76:0/1979581781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc55415d0a0 msgr2=0x7fc5540b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:51.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.666+0000 7fc5527fc640 1 --2- 172.21.15.76:0/1979581781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc55415d0a0 0x7fc5540b4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:51.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.666+0000 7fc5527fc640 1 -- 172.21.15.76:0/1979581781 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c009b30 con 0x7fc55415ccc0 2023-11-09T03:46:51.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.666+0000 7fc5527fc640 1 --2- 172.21.15.76:0/1979581781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc55415ccc0 0x7fc5540b4100 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00b3f0 tx=0x7fc53c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:51.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.666+0000 7fc54b7fe640 1 -- 172.21.15.76:0/1979581781 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c011830 con 0x7fc55415ccc0 2023-11-09T03:46:51.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.666+0000 7fc54b7fe640 1 -- 172.21.15.76:0/1979581781 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc53c011990 con 0x7fc55415ccc0 2023-11-09T03:46:51.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.666+0000 7fc559733640 1 -- 172.21.15.76:0/1979581781 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc55415fbb0 con 0x7fc55415ccc0 2023-11-09T03:46:51.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.666+0000 7fc54b7fe640 1 -- 172.21.15.76:0/1979581781 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c011d40 con 0x7fc55415ccc0 2023-11-09T03:46:51.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.666+0000 7fc559733640 1 -- 172.21.15.76:0/1979581781 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5541558a0 con 0x7fc55415ccc0 2023-11-09T03:46:51.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.667+0000 7fc559733640 1 -- 172.21.15.76:0/1979581781 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5541600d0 con 0x7fc55415ccc0 2023-11-09T03:46:51.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.667+0000 7fc54b7fe640 1 -- 172.21.15.76:0/1979581781 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc53c01a400 con 0x7fc55415ccc0 2023-11-09T03:46:51.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.668+0000 7fc54b7fe640 1 --2- 172.21.15.76:0/1979581781 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc53406ae70 0x7fc53406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:51.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.668+0000 7fc54b7fe640 1 -- 172.21.15.76:0/1979581781 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5929+0+0 (secure 0 0 0) 0x7fc53c089320 con 0x7fc55415ccc0 2023-11-09T03:46:51.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.668+0000 7fc552ffd640 1 --2- 172.21.15.76:0/1979581781 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc53406ae70 0x7fc53406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:51.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.668+0000 7fc552ffd640 1 --2- 172.21.15.76:0/1979581781 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc53406ae70 0x7fc53406d330 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc54c035000 tx=0x7fc54c030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:51.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.672+0000 7fc54b7fe640 1 -- 172.21.15.76:0/1979581781 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc5541600d0 con 0x7fc55415ccc0 2023-11-09T03:46:51.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:51.872+0000 7fc559733640 1 -- 172.21.15.76:0/1979581781 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc5541600d0 con 0x7fc55415ccc0 2023-11-09T03:46:52.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.525+0000 7fc54b7fe640 1 -- 172.21.15.76:0/1979581781 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fc5541600d0 con 0x7fc55415ccc0 2023-11-09T03:46:52.524 INFO:teuthology.orchestra.run.smithi076.stderr:new fs with metadata pool 5 and data pool 6 2023-11-09T03:46:52.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.529+0000 7fc559733640 1 -- 172.21.15.76:0/1979581781 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc53406ae70 msgr2=0x7fc53406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:52.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.529+0000 7fc559733640 1 --2- 172.21.15.76:0/1979581781 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc53406ae70 0x7fc53406d330 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc54c035000 tx=0x7fc54c030000 comp rx=0 tx=0).stop 2023-11-09T03:46:52.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.529+0000 7fc559733640 1 -- 172.21.15.76:0/1979581781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc55415ccc0 msgr2=0x7fc5540b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:52.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.529+0000 7fc559733640 1 --2- 172.21.15.76:0/1979581781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc55415ccc0 0x7fc5540b4100 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00b3f0 tx=0x7fc53c00b8b0 comp rx=0 tx=0).stop 2023-11-09T03:46:52.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.529+0000 7fc559733640 1 -- 172.21.15.76:0/1979581781 shutdown_connections 2023-11-09T03:46:52.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.529+0000 7fc559733640 1 --2- 172.21.15.76:0/1979581781 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc53406ae70 0x7fc53406d330 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:52.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.529+0000 7fc559733640 1 --2- 172.21.15.76:0/1979581781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc55415d0a0 0x7fc5540b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:52.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.529+0000 7fc559733640 1 --2- 172.21.15.76:0/1979581781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc55415ccc0 0x7fc5540b4100 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:52.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.529+0000 7fc559733640 1 --2- 172.21.15.76:0/1979581781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc5540cdd90 0x7fc5540c6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:52.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.529+0000 7fc559733640 1 -- 172.21.15.76:0/1979581781 >> 172.21.15.76:0/1979581781 conn(0x7fc5540cf8a0 msgr2=0x7fc5540cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:52.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.530+0000 7fc559733640 1 -- 172.21.15.76:0/1979581781 shutdown_connections 2023-11-09T03:46:52.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.534+0000 7fc559733640 1 -- 172.21.15.76:0/1979581781 wait complete. 2023-11-09T03:46:52.557 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:46:52.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.659+0000 7faf417ee640 1 Processor -- start 2023-11-09T03:46:52.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.659+0000 7faf417ee640 1 -- start start 2023-11-09T03:46:52.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.660+0000 7faf417ee640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faf3c157710 0x7faf3c167b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:52.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.660+0000 7faf417ee640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf3c15e000 con 0x7faf3c0ce240 2023-11-09T03:46:52.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.660+0000 7faf417ee640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf3c15e2e0 con 0x7faf3c0cde60 2023-11-09T03:46:52.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.660+0000 7faf417ee640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf3c15e5c0 con 0x7faf3c157710 2023-11-09T03:46:52.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.660+0000 7faf3b7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faf3c157710 0x7faf3c167b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:52.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.660+0000 7faf33fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7faf3c0ce240 0x7faf3c157000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59856/0 (socket says 172.21.15.76:59856) 2023-11-09T03:46:52.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.660+0000 7faf33fff640 1 -- 172.21.15.76:0/1247190374 learned_addr learned my addr 172.21.15.76:0/1247190374 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:52.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.660+0000 7faf3b7fe640 1 -- 172.21.15.76:0/1247190374 >> v1:172.21.15.113:6789/0 conn(0x7faf3c0cde60 legacy=0x7faf3c15d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:52.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.660+0000 7faf3b7fe640 1 -- 172.21.15.76:0/1247190374 >> v1:172.21.15.76:6789/0 conn(0x7faf3c0ce240 legacy=0x7faf3c157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:52.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.660+0000 7faf3a7fc640 1 -- 172.21.15.76:0/1247190374 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3882093130 0 0) 0x7faf3c15e000 con 0x7faf3c0ce240 2023-11-09T03:46:52.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.660+0000 7faf3b7fe640 1 -- 172.21.15.76:0/1247190374 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3c0d1120 con 0x7faf3c157710 2023-11-09T03:46:52.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.661+0000 7faf3b7fe640 1 --2- 172.21.15.76:0/1247190374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf3c157710 0x7faf3c167b90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7faf34009d90 tx=0x7faf3402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65c3427255c81e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:52.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.661+0000 7faf3a7fc640 1 -- 172.21.15.76:0/1247190374 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3402cd00 con 0x7faf3c157710 2023-11-09T03:46:52.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.661+0000 7faf3a7fc640 1 -- 172.21.15.76:0/1247190374 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3402ce60 con 0x7faf3c157710 2023-11-09T03:46:52.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.661+0000 7faf3a7fc640 1 -- 172.21.15.76:0/1247190374 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf34034660 con 0x7faf3c157710 2023-11-09T03:46:52.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.662+0000 7faf417ee640 1 -- 172.21.15.76:0/1247190374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf3c157710 msgr2=0x7faf3c167b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:52.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.662+0000 7faf417ee640 1 --2- 172.21.15.76:0/1247190374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf3c157710 0x7faf3c167b90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7faf34009d90 tx=0x7faf3402bf50 comp rx=0 tx=0).stop 2023-11-09T03:46:52.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.662+0000 7faf417ee640 1 -- 172.21.15.76:0/1247190374 shutdown_connections 2023-11-09T03:46:52.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.662+0000 7faf417ee640 1 --2- 172.21.15.76:0/1247190374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf3c157710 0x7faf3c167b90 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:52.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.662+0000 7faf417ee640 1 -- 172.21.15.76:0/1247190374 >> 172.21.15.76:0/1247190374 conn(0x7faf3c071f30 msgr2=0x7faf3c0cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:52.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.662+0000 7faf417ee640 1 -- 172.21.15.76:0/1247190374 shutdown_connections 2023-11-09T03:46:52.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.662+0000 7faf417ee640 1 -- 172.21.15.76:0/1247190374 wait complete. 2023-11-09T03:46:52.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.663+0000 7faf417ee640 1 Processor -- start 2023-11-09T03:46:52.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.663+0000 7faf417ee640 1 -- start start 2023-11-09T03:46:52.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.664+0000 7faf417ee640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf3c0cde60 0x7faf3c152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:52.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.664+0000 7faf417ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf3c0ce240 0x7faf3c0c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:52.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.664+0000 7faf417ee640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf3c157710 0x7faf3c0ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:52.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.664+0000 7faf417ee640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c157d80 con 0x7faf3c0cde60 2023-11-09T03:46:52.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.664+0000 7faf417ee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c0cc0e0 con 0x7faf3c0ce240 2023-11-09T03:46:52.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.664+0000 7faf417ee640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c0cc3c0 con 0x7faf3c157710 2023-11-09T03:46:52.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.664+0000 7faf3affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf3c0cde60 0x7faf3c152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:52.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.664+0000 7faf3affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf3c0cde60 0x7faf3c152800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41566/0 (socket says 172.21.15.76:41566) 2023-11-09T03:46:52.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.664+0000 7faf3affd640 1 -- 172.21.15.76:0/4023532559 learned_addr learned my addr 172.21.15.76:0/4023532559 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:52.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.664+0000 7faf3affd640 1 -- 172.21.15.76:0/4023532559 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf3c157710 msgr2=0x7faf3c0ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:52.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.664+0000 7faf3b7fe640 1 --2- 172.21.15.76:0/4023532559 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf3c157710 0x7faf3c0ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:52.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.664+0000 7faf33fff640 1 --2- 172.21.15.76:0/4023532559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf3c0ce240 0x7faf3c0c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:52.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.664+0000 7faf3affd640 1 --2- 172.21.15.76:0/4023532559 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf3c157710 0x7faf3c0ca430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:52.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.664+0000 7faf3affd640 1 -- 172.21.15.76:0/4023532559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf3c0ce240 msgr2=0x7faf3c0c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:52.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.664+0000 7faf3affd640 1 --2- 172.21.15.76:0/4023532559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf3c0ce240 0x7faf3c0c9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:52.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.664+0000 7faf3affd640 1 -- 172.21.15.76:0/4023532559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf34009a70 con 0x7faf3c0cde60 2023-11-09T03:46:52.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.664+0000 7faf3affd640 1 --2- 172.21.15.76:0/4023532559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf3c0cde60 0x7faf3c152800 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7faf2c00c450 tx=0x7faf2c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:52.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.664+0000 7faf337fe640 1 -- 172.21.15.76:0/4023532559 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c0052d0 con 0x7faf3c0cde60 2023-11-09T03:46:52.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.664+0000 7faf337fe640 1 -- 172.21.15.76:0/4023532559 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2c005430 con 0x7faf3c0cde60 2023-11-09T03:46:52.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.664+0000 7faf337fe640 1 -- 172.21.15.76:0/4023532559 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c012870 con 0x7faf3c0cde60 2023-11-09T03:46:52.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.664+0000 7faf417ee640 1 -- 172.21.15.76:0/4023532559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf3c0cc6a0 con 0x7faf3c0cde60 2023-11-09T03:46:52.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.664+0000 7faf417ee640 1 -- 172.21.15.76:0/4023532559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf3c1537a0 con 0x7faf3c0cde60 2023-11-09T03:46:52.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.665+0000 7faf417ee640 1 -- 172.21.15.76:0/4023532559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3c074290 con 0x7faf3c0cde60 2023-11-09T03:46:52.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.666+0000 7faf337fe640 1 -- 172.21.15.76:0/4023532559 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faf2c0129d0 con 0x7faf3c0cde60 2023-11-09T03:46:52.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.666+0000 7faf337fe640 1 --2- 172.21.15.76:0/4023532559 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf1c06ae70 0x7faf1c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:52.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.666+0000 7faf337fe640 1 -- 172.21.15.76:0/4023532559 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7faf2c0891c0 con 0x7faf3c0cde60 2023-11-09T03:46:52.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.666+0000 7faf33fff640 1 --2- 172.21.15.76:0/4023532559 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf1c06ae70 0x7faf1c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:52.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.667+0000 7faf33fff640 1 --2- 172.21.15.76:0/4023532559 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf1c06ae70 0x7faf1c06d330 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faf24007fd0 tx=0x7faf24009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:52.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.670+0000 7faf337fe640 1 -- 172.21.15.76:0/4023532559 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faf2c0541c0 con 0x7faf3c0cde60 2023-11-09T03:46:52.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.837+0000 7faf417ee640 1 -- 172.21.15.76:0/4023532559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf3c073060 con 0x7faf3c0cde60 2023-11-09T03:46:52.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.838+0000 7faf337fe640 1 -- 172.21.15.76:0/4023532559 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16775 (secure 0 0 0) 0x7faf2c0577e0 con 0x7faf3c0cde60 2023-11-09T03:46:52.838 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:52.838 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":38,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:46:51.873814+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:46:50.064765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-09T03:46:50.837125+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:46:52.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.840+0000 7faf417ee640 1 -- 172.21.15.76:0/4023532559 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf1c06ae70 msgr2=0x7faf1c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:52.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.840+0000 7faf417ee640 1 --2- 172.21.15.76:0/4023532559 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf1c06ae70 0x7faf1c06d330 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faf24007fd0 tx=0x7faf24009040 comp rx=0 tx=0).stop 2023-11-09T03:46:52.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.840+0000 7faf417ee640 1 -- 172.21.15.76:0/4023532559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf3c0cde60 msgr2=0x7faf3c152800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:52.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.840+0000 7faf417ee640 1 --2- 172.21.15.76:0/4023532559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf3c0cde60 0x7faf3c152800 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7faf2c00c450 tx=0x7faf2c00c910 comp rx=0 tx=0).stop 2023-11-09T03:46:52.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.840+0000 7faf417ee640 1 -- 172.21.15.76:0/4023532559 shutdown_connections 2023-11-09T03:46:52.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.840+0000 7faf417ee640 1 --2- 172.21.15.76:0/4023532559 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf1c06ae70 0x7faf1c06d330 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:52.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.840+0000 7faf417ee640 1 --2- 172.21.15.76:0/4023532559 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf3c157710 0x7faf3c0ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:52.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.840+0000 7faf417ee640 1 --2- 172.21.15.76:0/4023532559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf3c0ce240 0x7faf3c0c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:52.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.840+0000 7faf417ee640 1 --2- 172.21.15.76:0/4023532559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf3c0cde60 0x7faf3c152800 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:52.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.840+0000 7faf417ee640 1 -- 172.21.15.76:0/4023532559 >> 172.21.15.76:0/4023532559 conn(0x7faf3c071f30 msgr2=0x7faf3c0d09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:52.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.840+0000 7faf417ee640 1 -- 172.21.15.76:0/4023532559 shutdown_connections 2023-11-09T03:46:52.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.840+0000 7faf417ee640 1 -- 172.21.15.76:0/4023532559 wait complete. 2023-11-09T03:46:52.851 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:46:52.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.951+0000 7f8884133640 1 Processor -- start 2023-11-09T03:46:52.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.951+0000 7f8884133640 1 -- start start 2023-11-09T03:46:52.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.952+0000 7f8884133640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f887c159c70 0x7f887c0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:52.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.952+0000 7f8884133640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f887c0b0ae0 0x7f887c15d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:52.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.952+0000 7f8884133640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f887c0b4110 con 0x7f887c06db20 2023-11-09T03:46:52.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.952+0000 7f8884133640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f887c0b49b0 con 0x7f887c159c70 2023-11-09T03:46:52.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.952+0000 7f8884133640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887c0b43f0 con 0x7f887c0b0ae0 2023-11-09T03:46:52.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.952+0000 7f8881ecf640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f887c159c70 0x7f887c0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:52.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.952+0000 7f8881ecf640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f887c159c70 0x7f887c0ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41570/0 (socket says 172.21.15.76:41570) 2023-11-09T03:46:52.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.952+0000 7f8881ecf640 1 -- 172.21.15.76:0/3952261436 learned_addr learned my addr 172.21.15.76:0/3952261436 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:52.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.952+0000 7f8881ecf640 1 -- 172.21.15.76:0/3952261436 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f887c0b0ae0 msgr2=0x7f887c15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:52.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.952+0000 7f88826d0640 1 --2- 172.21.15.76:0/3952261436 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f887c0b0ae0 0x7f887c15d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:52.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.952+0000 7f8881ecf640 1 --2- 172.21.15.76:0/3952261436 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f887c0b0ae0 0x7f887c15d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:52.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.952+0000 7f8881ecf640 1 -- 172.21.15.76:0/3952261436 >> v1:172.21.15.113:6789/0 conn(0x7f887c06db20 legacy=0x7f887c0b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:52.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.952+0000 7f8881ecf640 1 -- 172.21.15.76:0/3952261436 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887c0b46d0 con 0x7f887c159c70 2023-11-09T03:46:52.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.953+0000 7f8881ecf640 1 --2- 172.21.15.76:0/3952261436 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f887c159c70 0x7f887c0ce370 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8878009870 tx=0x7f887802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20781875573b956c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:52.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.953+0000 7f8880ecd640 1 -- 172.21.15.76:0/3952261436 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8878031640 con 0x7f887c159c70 2023-11-09T03:46:52.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.953+0000 7f8880ecd640 1 -- 172.21.15.76:0/3952261436 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88780317a0 con 0x7f887c159c70 2023-11-09T03:46:52.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.953+0000 7f8880ecd640 1 -- 172.21.15.76:0/3952261436 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8878031b30 con 0x7f887c159c70 2023-11-09T03:46:52.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.953+0000 7f8884133640 1 -- 172.21.15.76:0/3952261436 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f887c159c70 msgr2=0x7f887c0ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:52.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.953+0000 7f8884133640 1 --2- 172.21.15.76:0/3952261436 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f887c159c70 0x7f887c0ce370 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8878009870 tx=0x7f887802b8f0 comp rx=0 tx=0).stop 2023-11-09T03:46:52.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.953+0000 7f8884133640 1 -- 172.21.15.76:0/3952261436 shutdown_connections 2023-11-09T03:46:52.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.953+0000 7f8884133640 1 --2- 172.21.15.76:0/3952261436 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f887c0b0ae0 0x7f887c15d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:52.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.953+0000 7f8884133640 1 --2- 172.21.15.76:0/3952261436 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f887c159c70 0x7f887c0ce370 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:52.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.953+0000 7f8884133640 1 -- 172.21.15.76:0/3952261436 >> 172.21.15.76:0/3952261436 conn(0x7f887c0ce950 msgr2=0x7f887c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:52.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.954+0000 7f8884133640 1 -- 172.21.15.76:0/3952261436 shutdown_connections 2023-11-09T03:46:52.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.954+0000 7f8884133640 1 -- 172.21.15.76:0/3952261436 wait complete. 2023-11-09T03:46:52.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.955+0000 7f8884133640 1 Processor -- start 2023-11-09T03:46:52.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.955+0000 7f8884133640 1 -- start start 2023-11-09T03:46:52.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.955+0000 7f8884133640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f887c06db20 0x7f887c15b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:52.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.955+0000 7f8884133640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f887c0b0ae0 0x7f887c1525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:52.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.955+0000 7f8884133640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f887c15c850 0x7f887c1505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:52.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.955+0000 7f8884133640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c152ae0 con 0x7f887c15c850 2023-11-09T03:46:52.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.955+0000 7f8884133640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c15b7b0 con 0x7f887c06db20 2023-11-09T03:46:52.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.955+0000 7f8884133640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887c15ba90 con 0x7f887c0b0ae0 2023-11-09T03:46:52.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.956+0000 7f88826d0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f887c15c850 0x7f887c1505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:52.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.956+0000 7f88826d0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f887c15c850 0x7f887c1505c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41572/0 (socket says 172.21.15.76:41572) 2023-11-09T03:46:52.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.956+0000 7f88826d0640 1 -- 172.21.15.76:0/3608891009 learned_addr learned my addr 172.21.15.76:0/3608891009 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:52.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.956+0000 7f8881ecf640 1 --2- 172.21.15.76:0/3608891009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f887c06db20 0x7f887c15b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:52.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.956+0000 7f88816ce640 1 --2- 172.21.15.76:0/3608891009 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f887c0b0ae0 0x7f887c1525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:52.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.956+0000 7f88826d0640 1 -- 172.21.15.76:0/3608891009 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f887c0b0ae0 msgr2=0x7f887c1525a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:52.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.956+0000 7f88826d0640 1 --2- 172.21.15.76:0/3608891009 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f887c0b0ae0 0x7f887c1525a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:52.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.956+0000 7f88826d0640 1 -- 172.21.15.76:0/3608891009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f887c06db20 msgr2=0x7f887c15b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:52.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.956+0000 7f88826d0640 1 --2- 172.21.15.76:0/3608891009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f887c06db20 0x7f887c15b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:52.963 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.956+0000 7f88826d0640 1 -- 172.21.15.76:0/3608891009 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88780094d0 con 0x7f887c15c850 2023-11-09T03:46:52.963 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.956+0000 7f88826d0640 1 --2- 172.21.15.76:0/3608891009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f887c15c850 0x7f887c1505c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f887400cc30 tx=0x7f887400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:52.963 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.956+0000 7f88727fc640 1 -- 172.21.15.76:0/3608891009 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887400ce40 con 0x7f887c15c850 2023-11-09T03:46:52.963 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.956+0000 7f88727fc640 1 -- 172.21.15.76:0/3608891009 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8874003990 con 0x7f887c15c850 2023-11-09T03:46:52.963 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.956+0000 7f88727fc640 1 -- 172.21.15.76:0/3608891009 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8874003d20 con 0x7f887c15c850 2023-11-09T03:46:52.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.956+0000 7f8884133640 1 -- 172.21.15.76:0/3608891009 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f887c15bd70 con 0x7f887c15c850 2023-11-09T03:46:52.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.956+0000 7f8884133640 1 -- 172.21.15.76:0/3608891009 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f887c15c330 con 0x7f887c15c850 2023-11-09T03:46:52.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.957+0000 7f8884133640 1 -- 172.21.15.76:0/3608891009 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f887c15c550 con 0x7f887c15c850 2023-11-09T03:46:52.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.958+0000 7f88727fc640 1 -- 172.21.15.76:0/3608891009 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8874003e80 con 0x7f887c15c850 2023-11-09T03:46:52.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.958+0000 7f88727fc640 1 --2- 172.21.15.76:0/3608891009 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f886006ae70 0x7f886006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:52.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.958+0000 7f88727fc640 1 -- 172.21.15.76:0/3608891009 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8874089a90 con 0x7f887c15c850 2023-11-09T03:46:52.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.958+0000 7f8881ecf640 1 --2- 172.21.15.76:0/3608891009 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f886006ae70 0x7f886006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:52.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.959+0000 7f8881ecf640 1 --2- 172.21.15.76:0/3608891009 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f886006ae70 0x7f886006d330 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8878036000 tx=0x7f887803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:52.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:52.962+0000 7f88727fc640 1 -- 172.21.15.76:0/3608891009 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8874054ab0 con 0x7f887c15c850 2023-11-09T03:46:53.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.128+0000 7f8884133640 1 -- 172.21.15.76:0/3608891009 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f887c074290 con 0x7f887c15c850 2023-11-09T03:46:53.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.129+0000 7f88727fc640 1 -- 172.21.15.76:0/3608891009 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16775 (secure 0 0 0) 0x7f887401d070 con 0x7f887c15c850 2023-11-09T03:46:53.129 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:53.129 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":38,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:46:51.873814+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:46:50.064765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-09T03:46:50.837125+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:46:53.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.131+0000 7f8884133640 1 -- 172.21.15.76:0/3608891009 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f886006ae70 msgr2=0x7f886006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:53.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.131+0000 7f8884133640 1 --2- 172.21.15.76:0/3608891009 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f886006ae70 0x7f886006d330 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8878036000 tx=0x7f887803c040 comp rx=0 tx=0).stop 2023-11-09T03:46:53.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.131+0000 7f8884133640 1 -- 172.21.15.76:0/3608891009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f887c15c850 msgr2=0x7f887c1505c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:53.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.131+0000 7f8884133640 1 --2- 172.21.15.76:0/3608891009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f887c15c850 0x7f887c1505c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f887400cc30 tx=0x7f887400a5c0 comp rx=0 tx=0).stop 2023-11-09T03:46:53.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.131+0000 7f8884133640 1 -- 172.21.15.76:0/3608891009 shutdown_connections 2023-11-09T03:46:53.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.131+0000 7f8884133640 1 --2- 172.21.15.76:0/3608891009 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f886006ae70 0x7f886006d330 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:53.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.131+0000 7f8884133640 1 --2- 172.21.15.76:0/3608891009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f887c15c850 0x7f887c1505c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:53.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.131+0000 7f8884133640 1 --2- 172.21.15.76:0/3608891009 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f887c0b0ae0 0x7f887c1525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:53.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.131+0000 7f8884133640 1 --2- 172.21.15.76:0/3608891009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f887c06db20 0x7f887c15b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:53.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.131+0000 7f8884133640 1 -- 172.21.15.76:0/3608891009 >> 172.21.15.76:0/3608891009 conn(0x7f887c0ce950 msgr2=0x7f887c15a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:53.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.131+0000 7f8884133640 1 -- 172.21.15.76:0/3608891009 shutdown_connections 2023-11-09T03:46:53.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.131+0000 7f8884133640 1 -- 172.21.15.76:0/3608891009 wait complete. 2023-11-09T03:46:53.141 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-09T03:46:53.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.245+0000 7f9daf397640 1 Processor -- start 2023-11-09T03:46:53.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.246+0000 7f9daf397640 1 -- start start 2023-11-09T03:46:53.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.246+0000 7f9daf397640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9da8153e60 0x7f9da815dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:53.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.246+0000 7f9daf397640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9da80cd6d0 0x7f9da8161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:53.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.246+0000 7f9daf397640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9da80b4880 0x7f9da80b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:53.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.246+0000 7f9daf397640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9da80cde60 con 0x7f9da80cd6d0 2023-11-09T03:46:53.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.246+0000 7f9daf397640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9da80b5220 con 0x7f9da8153e60 2023-11-09T03:46:53.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.246+0000 7f9daf397640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da8160f90 con 0x7f9da80b4880 2023-11-09T03:46:53.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.246+0000 7f9dac932640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9da80cd6d0 0x7f9da8161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:53.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.246+0000 7f9dac932640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9da80cd6d0 0x7f9da8161420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41574/0 (socket says 172.21.15.76:41574) 2023-11-09T03:46:53.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.246+0000 7f9dac932640 1 -- 172.21.15.76:0/323670897 learned_addr learned my addr 172.21.15.76:0/323670897 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:53.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.247+0000 7f9dad934640 1 --2- 172.21.15.76:0/323670897 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9da80b4880 0x7f9da80b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:53.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.247+0000 7f9dad133640 1 --2- 172.21.15.76:0/323670897 >> v2:172.21.15.113:3300/0 conn(0x7f9da8153e60 0x7f9da815dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:53.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.247+0000 7f9dac932640 1 -- 172.21.15.76:0/323670897 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9da80b4880 msgr2=0x7f9da80b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:53.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.247+0000 7f9dac932640 1 --2- 172.21.15.76:0/323670897 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9da80b4880 0x7f9da80b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:53.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.247+0000 7f9dac932640 1 -- 172.21.15.76:0/323670897 >> v2:172.21.15.113:3300/0 conn(0x7f9da8153e60 msgr2=0x7f9da815dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:53.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.247+0000 7f9dac932640 1 --2- 172.21.15.76:0/323670897 >> v2:172.21.15.113:3300/0 conn(0x7f9da8153e60 0x7f9da815dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:53.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.247+0000 7f9dac932640 1 -- 172.21.15.76:0/323670897 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da815e400 con 0x7f9da80cd6d0 2023-11-09T03:46:53.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.247+0000 7f9dac932640 1 --2- 172.21.15.76:0/323670897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9da80cd6d0 0x7f9da8161420 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9da808abe0 tx=0x7f9d9802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7089e07415b7f936 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:53.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.248+0000 7f9d9ffff640 1 -- 172.21.15.76:0/323670897 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d98031640 con 0x7f9da80cd6d0 2023-11-09T03:46:53.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.248+0000 7f9d9ffff640 1 -- 172.21.15.76:0/323670897 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d980317a0 con 0x7f9da80cd6d0 2023-11-09T03:46:53.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.248+0000 7f9d9ffff640 1 -- 172.21.15.76:0/323670897 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d98031b30 con 0x7f9da80cd6d0 2023-11-09T03:46:53.253 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.248+0000 7f9daf397640 1 -- 172.21.15.76:0/323670897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9da80cd6d0 msgr2=0x7f9da8161420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:53.253 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.248+0000 7f9daf397640 1 --2- 172.21.15.76:0/323670897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9da80cd6d0 0x7f9da8161420 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9da808abe0 tx=0x7f9d9802b8f0 comp rx=0 tx=0).stop 2023-11-09T03:46:53.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.248+0000 7f9daf397640 1 -- 172.21.15.76:0/323670897 shutdown_connections 2023-11-09T03:46:53.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.248+0000 7f9daf397640 1 --2- 172.21.15.76:0/323670897 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9da80b4880 0x7f9da80b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:53.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.248+0000 7f9daf397640 1 --2- 172.21.15.76:0/323670897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9da80cd6d0 0x7f9da8161420 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:53.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.248+0000 7f9daf397640 1 --2- 172.21.15.76:0/323670897 >> v2:172.21.15.113:3300/0 conn(0x7f9da8153e60 0x7f9da815dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:53.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.248+0000 7f9daf397640 1 -- 172.21.15.76:0/323670897 >> 172.21.15.76:0/323670897 conn(0x7f9da80cf880 msgr2=0x7f9da80cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:53.255 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.248+0000 7f9daf397640 1 -- 172.21.15.76:0/323670897 shutdown_connections 2023-11-09T03:46:53.255 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.248+0000 7f9daf397640 1 -- 172.21.15.76:0/323670897 wait complete. 2023-11-09T03:46:53.255 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.249+0000 7f9daf397640 1 Processor -- start 2023-11-09T03:46:53.255 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.249+0000 7f9daf397640 1 -- start start 2023-11-09T03:46:53.255 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.250+0000 7f9daf397640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9da80b4880 0x7f9da814d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:53.256 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.250+0000 7f9daf397640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9da80cd6d0 0x7f9da80c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:53.256 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.250+0000 7f9daf397640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9da8153e60 0x7f9da814f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:53.256 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.250+0000 7f9daf397640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da814d910 con 0x7f9da80b4880 2023-11-09T03:46:53.256 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.250+0000 7f9daf397640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da80cafe0 con 0x7f9da8153e60 2023-11-09T03:46:53.256 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.250+0000 7f9daf397640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da80ca3d0 con 0x7f9da80cd6d0 2023-11-09T03:46:53.256 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.250+0000 7f9dad133640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9da80b4880 0x7f9da814d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:53.257 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.250+0000 7f9dad133640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9da80b4880 0x7f9da814d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41576/0 (socket says 172.21.15.76:41576) 2023-11-09T03:46:53.257 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.250+0000 7f9dad133640 1 -- 172.21.15.76:0/3414437207 learned_addr learned my addr 172.21.15.76:0/3414437207 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:53.257 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.250+0000 7f9dac932640 1 --2- 172.21.15.76:0/3414437207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9da80cd6d0 0x7f9da80c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:53.257 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.250+0000 7f9dad934640 1 --2- 172.21.15.76:0/3414437207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9da8153e60 0x7f9da814f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:53.257 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.251+0000 7f9dad133640 1 -- 172.21.15.76:0/3414437207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9da80cd6d0 msgr2=0x7f9da80c7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:53.258 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.251+0000 7f9dad133640 1 --2- 172.21.15.76:0/3414437207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9da80cd6d0 0x7f9da80c7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:53.258 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.251+0000 7f9dad133640 1 -- 172.21.15.76:0/3414437207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9da8153e60 msgr2=0x7f9da814f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:53.258 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.251+0000 7f9dad133640 1 --2- 172.21.15.76:0/3414437207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9da8153e60 0x7f9da814f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:53.258 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.251+0000 7f9dad133640 1 -- 172.21.15.76:0/3414437207 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d980094d0 con 0x7f9da80b4880 2023-11-09T03:46:53.258 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.251+0000 7f9dad934640 1 --2- 172.21.15.76:0/3414437207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9da8153e60 0x7f9da814f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:53.258 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.251+0000 7f9dad133640 1 --2- 172.21.15.76:0/3414437207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9da80b4880 0x7f9da814d3d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9da4007e40 tx=0x7f9da400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:53.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.251+0000 7f9d9dffb640 1 -- 172.21.15.76:0/3414437207 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da4014620 con 0x7f9da80b4880 2023-11-09T03:46:53.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.251+0000 7f9d9dffb640 1 -- 172.21.15.76:0/3414437207 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da400ee90 con 0x7f9da80b4880 2023-11-09T03:46:53.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.251+0000 7f9daf397640 1 -- 172.21.15.76:0/3414437207 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da80ca6b0 con 0x7f9da80b4880 2023-11-09T03:46:53.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.251+0000 7f9d9dffb640 1 -- 172.21.15.76:0/3414437207 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da40149c0 con 0x7f9da80b4880 2023-11-09T03:46:53.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.251+0000 7f9daf397640 1 -- 172.21.15.76:0/3414437207 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da80cac70 con 0x7f9da80b4880 2023-11-09T03:46:53.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.252+0000 7f9daf397640 1 -- 172.21.15.76:0/3414437207 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da80cae00 con 0x7f9da80b4880 2023-11-09T03:46:53.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.252+0000 7f9d9dffb640 1 -- 172.21.15.76:0/3414437207 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9da4014b20 con 0x7f9da80b4880 2023-11-09T03:46:53.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.253+0000 7f9d9dffb640 1 --2- 172.21.15.76:0/3414437207 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9d8c06ae70 0x7f9d8c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:53.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.253+0000 7f9d9dffb640 1 -- 172.21.15.76:0/3414437207 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9da408a090 con 0x7f9da80b4880 2023-11-09T03:46:53.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.253+0000 7f9dac932640 1 --2- 172.21.15.76:0/3414437207 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9d8c06ae70 0x7f9d8c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:53.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.254+0000 7f9dac932640 1 --2- 172.21.15.76:0/3414437207 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9d8c06ae70 0x7f9d8c06d330 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9d98036000 tx=0x7f9d9803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:53.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.257+0000 7f9d9dffb640 1 -- 172.21.15.76:0/3414437207 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9da4054f70 con 0x7f9da80b4880 2023-11-09T03:46:53.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.458+0000 7f9daf397640 1 -- 172.21.15.76:0/3414437207 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9da815e220 con 0x7f9da80b4880 2023-11-09T03:46:53.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.536+0000 7f9d9dffb640 1 -- 172.21.15.76:0/3414437207 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9da4058590 con 0x7f9da80b4880 2023-11-09T03:46:53.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.538+0000 7f9daf397640 1 -- 172.21.15.76:0/3414437207 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9d8c06ae70 msgr2=0x7f9d8c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:53.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.538+0000 7f9daf397640 1 --2- 172.21.15.76:0/3414437207 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9d8c06ae70 0x7f9d8c06d330 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9d98036000 tx=0x7f9d9803c040 comp rx=0 tx=0).stop 2023-11-09T03:46:53.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.538+0000 7f9daf397640 1 -- 172.21.15.76:0/3414437207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9da80b4880 msgr2=0x7f9da814d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:53.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.538+0000 7f9daf397640 1 --2- 172.21.15.76:0/3414437207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9da80b4880 0x7f9da814d3d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9da4007e40 tx=0x7f9da400e4b0 comp rx=0 tx=0).stop 2023-11-09T03:46:53.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.538+0000 7f9daf397640 1 -- 172.21.15.76:0/3414437207 shutdown_connections 2023-11-09T03:46:53.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.538+0000 7f9daf397640 1 --2- 172.21.15.76:0/3414437207 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9d8c06ae70 0x7f9d8c06d330 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:53.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.538+0000 7f9daf397640 1 --2- 172.21.15.76:0/3414437207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9da8153e60 0x7f9da814f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:53.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.538+0000 7f9daf397640 1 --2- 172.21.15.76:0/3414437207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9da80cd6d0 0x7f9da80c7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:53.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.538+0000 7f9daf397640 1 --2- 172.21.15.76:0/3414437207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9da80b4880 0x7f9da814d3d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:53.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.538+0000 7f9daf397640 1 -- 172.21.15.76:0/3414437207 >> 172.21.15.76:0/3414437207 conn(0x7f9da80cf880 msgr2=0x7f9da815fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:53.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.538+0000 7f9daf397640 1 -- 172.21.15.76:0/3414437207 shutdown_connections 2023-11-09T03:46:53.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.538+0000 7f9daf397640 1 -- 172.21.15.76:0/3414437207 wait complete. 2023-11-09T03:46:53.549 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-09T03:46:53.550 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-09T03:46:53.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.654+0000 7fae783b6640 1 Processor -- start 2023-11-09T03:46:53.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.654+0000 7fae783b6640 1 -- start start 2023-11-09T03:46:53.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.655+0000 7fae783b6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae7006cbd0 0x7fae7006e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:53.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.655+0000 7fae783b6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fae7015b140 0x7fae7006cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:53.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.655+0000 7fae783b6640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae700b24f0 con 0x7fae70067090 2023-11-09T03:46:53.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.655+0000 7fae783b6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae7006d4f0 con 0x7fae7015b140 2023-11-09T03:46:53.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.655+0000 7fae783b6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae700ce220 con 0x7fae7006cbd0 2023-11-09T03:46:53.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.655+0000 7fae76953640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fae70067090 0x7fae700cc900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59860/0 (socket says 172.21.15.76:59860) 2023-11-09T03:46:53.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.655+0000 7fae76953640 1 -- 172.21.15.76:0/4028419129 learned_addr learned my addr 172.21.15.76:0/4028419129 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:53.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.655+0000 7fae75150640 1 -- 172.21.15.76:0/4028419129 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1419363776 0 0) 0x7fae700b24f0 con 0x7fae70067090 2023-11-09T03:46:53.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.655+0000 7fae75150640 1 -- 172.21.15.76:0/4028419129 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae5c003520 con 0x7fae70067090 2023-11-09T03:46:53.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.655+0000 7fae76152640 1 --2- 172.21.15.76:0/4028419129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae7006cbd0 0x7fae7006e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:53.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.655+0000 7fae75951640 1 --2- 172.21.15.76:0/4028419129 >> v2:172.21.15.113:3300/0 conn(0x7fae7015b140 0x7fae7006cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:53.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.655+0000 7fae75150640 1 -- 172.21.15.76:0/4028419129 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 578222959 0 0) 0x7fae5c003520 con 0x7fae70067090 2023-11-09T03:46:53.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.655+0000 7fae75150640 1 -- 172.21.15.76:0/4028419129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae7006cbd0 msgr2=0x7fae7006e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:53.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.655+0000 7fae75150640 1 --2- 172.21.15.76:0/4028419129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae7006cbd0 0x7fae7006e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:53.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.655+0000 7fae75150640 1 -- 172.21.15.76:0/4028419129 >> v2:172.21.15.113:3300/0 conn(0x7fae7015b140 msgr2=0x7fae7006cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:53.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.655+0000 7fae75150640 1 --2- 172.21.15.76:0/4028419129 >> v2:172.21.15.113:3300/0 conn(0x7fae7015b140 0x7fae7006cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:53.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.655+0000 7fae75150640 1 -- 172.21.15.76:0/4028419129 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae700cdc60 con 0x7fae70067090 2023-11-09T03:46:53.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.655+0000 7fae75150640 1 -- 172.21.15.76:0/4028419129 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fae680019f0 con 0x7fae70067090 2023-11-09T03:46:53.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.656+0000 7fae75150640 1 -- 172.21.15.76:0/4028419129 >> v1:172.21.15.76:6789/0 conn(0x7fae70067090 legacy=0x7fae700cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:53.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.656+0000 7fae75150640 1 --2- 172.21.15.76:0/4028419129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae5c003970 0x7fae5c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:53.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.656+0000 7fae75150640 1 --2- 172.21.15.76:0/4028419129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae7006cbd0 0x7fae5c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:53.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.656+0000 7fae75150640 1 --2- 172.21.15.76:0/4028419129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae5c004d00 0x7fae5c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:53.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.656+0000 7fae75150640 1 -- 172.21.15.76:0/4028419129 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7006d4f0 con 0x7fae5c004d00 2023-11-09T03:46:53.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.656+0000 7fae75150640 1 -- 172.21.15.76:0/4028419129 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae700ce220 con 0x7fae5c003970 2023-11-09T03:46:53.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.656+0000 7fae75150640 1 -- 172.21.15.76:0/4028419129 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c00bae0 con 0x7fae7006cbd0 2023-11-09T03:46:53.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.656+0000 7fae76953640 1 --2- 172.21.15.76:0/4028419129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae5c004d00 0x7fae5c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:53.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.656+0000 7fae76152640 1 --2- 172.21.15.76:0/4028419129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae5c003970 0x7fae5c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:53.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.656+0000 7fae75951640 1 --2- 172.21.15.76:0/4028419129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae7006cbd0 0x7fae5c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:53.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.656+0000 7fae76953640 1 -- 172.21.15.76:0/4028419129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae7006cbd0 msgr2=0x7fae5c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:53.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.656+0000 7fae76953640 1 --2- 172.21.15.76:0/4028419129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae7006cbd0 0x7fae5c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:53.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.656+0000 7fae76953640 1 -- 172.21.15.76:0/4028419129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae5c003970 msgr2=0x7fae5c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:53.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.656+0000 7fae76953640 1 --2- 172.21.15.76:0/4028419129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae5c003970 0x7fae5c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:53.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.656+0000 7fae76953640 1 -- 172.21.15.76:0/4028419129 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5c003520 con 0x7fae5c004d00 2023-11-09T03:46:53.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.657+0000 7fae76953640 1 --2- 172.21.15.76:0/4028419129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae5c004d00 0x7fae5c005180 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fae70070af0 tx=0x7fae5c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10733def3a01869c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:53.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.657+0000 7fae75150640 1 -- 172.21.15.76:0/4028419129 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae68034630 con 0x7fae5c004d00 2023-11-09T03:46:53.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.657+0000 7fae75150640 1 -- 172.21.15.76:0/4028419129 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae6802f070 con 0x7fae5c004d00 2023-11-09T03:46:53.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.657+0000 7fae75150640 1 -- 172.21.15.76:0/4028419129 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae680349d0 con 0x7fae5c004d00 2023-11-09T03:46:53.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.657+0000 7fae783b6640 1 -- 172.21.15.76:0/4028419129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae5c004d00 msgr2=0x7fae5c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:53.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.657+0000 7fae783b6640 1 --2- 172.21.15.76:0/4028419129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae5c004d00 0x7fae5c005180 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fae70070af0 tx=0x7fae5c0057f0 comp rx=0 tx=0).stop 2023-11-09T03:46:53.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.657+0000 7fae76953640 1 -- 172.21.15.76:0/4028419129 reap_dead start 2023-11-09T03:46:53.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.657+0000 7fae783b6640 1 -- 172.21.15.76:0/4028419129 shutdown_connections 2023-11-09T03:46:53.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.657+0000 7fae783b6640 1 -- 172.21.15.76:0/4028419129 >> 172.21.15.76:0/4028419129 conn(0x7fae7006a120 msgr2=0x7fae7006a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:53.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.657+0000 7fae783b6640 1 -- 172.21.15.76:0/4028419129 shutdown_connections 2023-11-09T03:46:53.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.657+0000 7fae783b6640 1 -- 172.21.15.76:0/4028419129 wait complete. 2023-11-09T03:46:53.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.659+0000 7fae783b6640 1 Processor -- start 2023-11-09T03:46:53.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.659+0000 7fae783b6640 1 -- start start 2023-11-09T03:46:53.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.659+0000 7fae783b6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae5c004d00 0x7fae701544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:53.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.659+0000 7fae783b6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae701f3a40 0x7fae70151030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:53.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.659+0000 7fae783b6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae700c25b0 0x7fae70158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:53.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.659+0000 7fae783b6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae70154a30 con 0x7fae701f3a40 2023-11-09T03:46:53.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.659+0000 7fae783b6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae700cd900 con 0x7fae5c004d00 2023-11-09T03:46:53.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.659+0000 7fae783b6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae701f3f40 con 0x7fae700c25b0 2023-11-09T03:46:53.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.659+0000 7fae75951640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae701f3a40 0x7fae70151030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:53.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.659+0000 7fae75951640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae701f3a40 0x7fae70151030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41608/0 (socket says 172.21.15.76:41608) 2023-11-09T03:46:53.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.659+0000 7fae75951640 1 -- 172.21.15.76:0/1645608740 learned_addr learned my addr 172.21.15.76:0/1645608740 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:53.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.660+0000 7fae75951640 1 -- 172.21.15.76:0/1645608740 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae700c25b0 msgr2=0x7fae70158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:53.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.660+0000 7fae75951640 1 --2- 172.21.15.76:0/1645608740 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae700c25b0 0x7fae70158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:53.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.660+0000 7fae75951640 1 -- 172.21.15.76:0/1645608740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae5c004d00 msgr2=0x7fae701544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:53.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.660+0000 7fae75951640 1 --2- 172.21.15.76:0/1645608740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae5c004d00 0x7fae701544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:53.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.660+0000 7fae75951640 1 -- 172.21.15.76:0/1645608740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae701f4220 con 0x7fae701f3a40 2023-11-09T03:46:53.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.660+0000 7fae75951640 1 --2- 172.21.15.76:0/1645608740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae701f3a40 0x7fae70151030 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fae6c00b7f0 tx=0x7fae6c00bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:53.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.660+0000 7fae66ffd640 1 -- 172.21.15.76:0/1645608740 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c0107b0 con 0x7fae701f3a40 2023-11-09T03:46:53.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.660+0000 7fae66ffd640 1 -- 172.21.15.76:0/1645608740 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae6c010910 con 0x7fae701f3a40 2023-11-09T03:46:53.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.660+0000 7fae783b6640 1 -- 172.21.15.76:0/1645608740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae701f4500 con 0x7fae701f3a40 2023-11-09T03:46:53.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.660+0000 7fae66ffd640 1 -- 172.21.15.76:0/1645608740 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c010ca0 con 0x7fae701f3a40 2023-11-09T03:46:53.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.660+0000 7fae783b6640 1 -- 172.21.15.76:0/1645608740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae701f0450 con 0x7fae701f3a40 2023-11-09T03:46:53.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.661+0000 7fae783b6640 1 -- 172.21.15.76:0/1645608740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae70074290 con 0x7fae701f3a40 2023-11-09T03:46:53.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.661+0000 7fae66ffd640 1 -- 172.21.15.76:0/1645608740 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fae6c014430 con 0x7fae701f3a40 2023-11-09T03:46:53.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.662+0000 7fae66ffd640 1 --2- 172.21.15.76:0/1645608740 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae5406ae70 0x7fae5406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:53.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.662+0000 7fae66ffd640 1 -- 172.21.15.76:0/1645608740 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fae6c08a040 con 0x7fae701f3a40 2023-11-09T03:46:53.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.662+0000 7fae76152640 1 --2- 172.21.15.76:0/1645608740 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae5406ae70 0x7fae5406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:53.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.662+0000 7fae76152640 1 --2- 172.21.15.76:0/1645608740 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae5406ae70 0x7fae5406d330 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fae60006fd0 tx=0x7fae600040a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:53.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.666+0000 7fae66ffd640 1 -- 172.21.15.76:0/1645608740 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fae6c054f20 con 0x7fae701f3a40 2023-11-09T03:46:53.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:53.867+0000 7fae783b6640 1 -- 172.21.15.76:0/1645608740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fae70158970 con 0x7fae701f3a40 2023-11-09T03:46:54.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.548+0000 7fae66ffd640 1 -- 172.21.15.76:0/1645608740 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fae6c0220c0 con 0x7fae701f3a40 2023-11-09T03:46:54.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.551+0000 7fae783b6640 1 -- 172.21.15.76:0/1645608740 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae5406ae70 msgr2=0x7fae5406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:54.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.551+0000 7fae783b6640 1 --2- 172.21.15.76:0/1645608740 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae5406ae70 0x7fae5406d330 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fae60006fd0 tx=0x7fae600040a0 comp rx=0 tx=0).stop 2023-11-09T03:46:54.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.551+0000 7fae783b6640 1 -- 172.21.15.76:0/1645608740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae701f3a40 msgr2=0x7fae70151030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:54.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.551+0000 7fae783b6640 1 --2- 172.21.15.76:0/1645608740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae701f3a40 0x7fae70151030 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fae6c00b7f0 tx=0x7fae6c00bcb0 comp rx=0 tx=0).stop 2023-11-09T03:46:54.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.551+0000 7fae783b6640 1 -- 172.21.15.76:0/1645608740 shutdown_connections 2023-11-09T03:46:54.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.551+0000 7fae783b6640 1 --2- 172.21.15.76:0/1645608740 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae5406ae70 0x7fae5406d330 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:54.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.551+0000 7fae783b6640 1 --2- 172.21.15.76:0/1645608740 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae700c25b0 0x7fae70158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:54.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.551+0000 7fae783b6640 1 --2- 172.21.15.76:0/1645608740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae701f3a40 0x7fae70151030 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:54.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.551+0000 7fae783b6640 1 --2- 172.21.15.76:0/1645608740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae5c004d00 0x7fae701544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:54.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.551+0000 7fae783b6640 1 -- 172.21.15.76:0/1645608740 >> 172.21.15.76:0/1645608740 conn(0x7fae7006a120 msgr2=0x7fae70157be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:54.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.551+0000 7fae783b6640 1 -- 172.21.15.76:0/1645608740 shutdown_connections 2023-11-09T03:46:54.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.551+0000 7fae783b6640 1 -- 172.21.15.76:0/1645608740 wait complete. 2023-11-09T03:46:54.563 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-09T03:46:54.563 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:46:54.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.667+0000 7f94b1555640 1 Processor -- start 2023-11-09T03:46:54.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.667+0000 7f94b1555640 1 -- start start 2023-11-09T03:46:54.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.668+0000 7f94b1555640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f94ac15cf30 0x7f94ac06cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:54.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.668+0000 7f94b1555640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f94ac0cce90 0x7f94ac070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:54.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.668+0000 7f94b1555640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94ac0b07f0 0x7f94ac0b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:54.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.668+0000 7f94b1555640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94ac0c9e00 con 0x7f94ac15cf30 2023-11-09T03:46:54.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.668+0000 7f94b1555640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94ac0b11a0 con 0x7f94ac0cce90 2023-11-09T03:46:54.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.668+0000 7f94b1555640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac071500 con 0x7f94ac0b07f0 2023-11-09T03:46:54.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.668+0000 7f94aaffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f94ac15cf30 0x7f94ac06cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:54.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.668+0000 7f94aaffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f94ac15cf30 0x7f94ac06cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41624/0 (socket says 172.21.15.76:41624) 2023-11-09T03:46:54.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.668+0000 7f94aaffd640 1 -- 172.21.15.76:0/2241545098 learned_addr learned my addr 172.21.15.76:0/2241545098 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:54.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.668+0000 7f94a3fff640 1 --2- 172.21.15.76:0/2241545098 >> v2:172.21.15.113:3300/0 conn(0x7f94ac0cce90 0x7f94ac070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:54.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.668+0000 7f94ab7fe640 1 --2- 172.21.15.76:0/2241545098 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94ac0b07f0 0x7f94ac0b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:54.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.668+0000 7f94aaffd640 1 -- 172.21.15.76:0/2241545098 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94ac0b07f0 msgr2=0x7f94ac0b0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:54.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.668+0000 7f94aaffd640 1 --2- 172.21.15.76:0/2241545098 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94ac0b07f0 0x7f94ac0b0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:54.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.668+0000 7f94aaffd640 1 -- 172.21.15.76:0/2241545098 >> v2:172.21.15.113:3300/0 conn(0x7f94ac0cce90 msgr2=0x7f94ac070bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:54.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.668+0000 7f94aaffd640 1 --2- 172.21.15.76:0/2241545098 >> v2:172.21.15.113:3300/0 conn(0x7f94ac0cce90 0x7f94ac070bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:54.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.668+0000 7f94aaffd640 1 -- 172.21.15.76:0/2241545098 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ac0717e0 con 0x7f94ac15cf30 2023-11-09T03:46:54.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.669+0000 7f94aaffd640 1 --2- 172.21.15.76:0/2241545098 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94ac15cf30 0x7f94ac06cbb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0cb760 tx=0x7f949c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b648e2bffd1c93ab server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:54.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.669+0000 7f94aa7fc640 1 -- 172.21.15.76:0/2241545098 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c0317e0 con 0x7f94ac15cf30 2023-11-09T03:46:54.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.669+0000 7f94aa7fc640 1 -- 172.21.15.76:0/2241545098 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949c031940 con 0x7f94ac15cf30 2023-11-09T03:46:54.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.669+0000 7f94aa7fc640 1 -- 172.21.15.76:0/2241545098 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c031cd0 con 0x7f94ac15cf30 2023-11-09T03:46:54.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.670+0000 7f94b1555640 1 -- 172.21.15.76:0/2241545098 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94ac15cf30 msgr2=0x7f94ac06cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:54.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.670+0000 7f94b1555640 1 --2- 172.21.15.76:0/2241545098 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94ac15cf30 0x7f94ac06cbb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0cb760 tx=0x7f949c02b880 comp rx=0 tx=0).stop 2023-11-09T03:46:54.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.670+0000 7f94b1555640 1 -- 172.21.15.76:0/2241545098 shutdown_connections 2023-11-09T03:46:54.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.670+0000 7f94b1555640 1 --2- 172.21.15.76:0/2241545098 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94ac0b07f0 0x7f94ac0b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:54.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.670+0000 7f94b1555640 1 --2- 172.21.15.76:0/2241545098 >> v2:172.21.15.113:3300/0 conn(0x7f94ac0cce90 0x7f94ac070bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:54.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.670+0000 7f94b1555640 1 --2- 172.21.15.76:0/2241545098 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94ac15cf30 0x7f94ac06cbb0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:54.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.670+0000 7f94b1555640 1 -- 172.21.15.76:0/2241545098 >> 172.21.15.76:0/2241545098 conn(0x7f94ac0cb7f0 msgr2=0x7f94ac0cbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:54.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.670+0000 7f94b1555640 1 -- 172.21.15.76:0/2241545098 shutdown_connections 2023-11-09T03:46:54.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.670+0000 7f94b1555640 1 -- 172.21.15.76:0/2241545098 wait complete. 2023-11-09T03:46:54.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.671+0000 7f94b1555640 1 Processor -- start 2023-11-09T03:46:54.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.671+0000 7f94b1555640 1 -- start start 2023-11-09T03:46:54.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.672+0000 7f94b1555640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94ac0b07f0 0x7f94ac0c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:54.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.672+0000 7f94b1555640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94ac0cce90 0x7f94ac0c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:54.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.672+0000 7f94b1555640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94ac15cf30 0x7f94ac0c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:54.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.672+0000 7f94b1555640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac15faf0 con 0x7f94ac0b07f0 2023-11-09T03:46:54.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.672+0000 7f94b1555640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac0c6380 con 0x7f94ac15cf30 2023-11-09T03:46:54.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.672+0000 7f94b1555640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac0c65b0 con 0x7f94ac0cce90 2023-11-09T03:46:54.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.672+0000 7f94aaffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94ac0b07f0 0x7f94ac0c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:54.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.672+0000 7f94aaffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94ac0b07f0 0x7f94ac0c5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41634/0 (socket says 172.21.15.76:41634) 2023-11-09T03:46:54.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.672+0000 7f94aaffd640 1 -- 172.21.15.76:0/3077202575 learned_addr learned my addr 172.21.15.76:0/3077202575 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:54.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.672+0000 7f94a3fff640 1 --2- 172.21.15.76:0/3077202575 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94ac0cce90 0x7f94ac0c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:54.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.672+0000 7f94ab7fe640 1 --2- 172.21.15.76:0/3077202575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94ac15cf30 0x7f94ac0c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:54.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.672+0000 7f94aaffd640 1 -- 172.21.15.76:0/3077202575 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94ac0cce90 msgr2=0x7f94ac0c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:54.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.672+0000 7f94aaffd640 1 --2- 172.21.15.76:0/3077202575 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94ac0cce90 0x7f94ac0c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:54.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.672+0000 7f94aaffd640 1 -- 172.21.15.76:0/3077202575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94ac15cf30 msgr2=0x7f94ac0c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:54.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.672+0000 7f94aaffd640 1 --2- 172.21.15.76:0/3077202575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94ac15cf30 0x7f94ac0c8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:54.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.672+0000 7f94aaffd640 1 -- 172.21.15.76:0/3077202575 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949c0094d0 con 0x7f94ac0b07f0 2023-11-09T03:46:54.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.672+0000 7f94a3fff640 1 --2- 172.21.15.76:0/3077202575 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94ac0cce90 0x7f94ac0c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:54.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.672+0000 7f94ab7fe640 1 --2- 172.21.15.76:0/3077202575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94ac15cf30 0x7f94ac0c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:54.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.672+0000 7f94aaffd640 1 --2- 172.21.15.76:0/3077202575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94ac0b07f0 0x7f94ac0c5e40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f949c000c00 tx=0x7f949c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:54.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.672+0000 7f94a37fe640 1 -- 172.21.15.76:0/3077202575 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c032880 con 0x7f94ac0b07f0 2023-11-09T03:46:54.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.672+0000 7f94a37fe640 1 -- 172.21.15.76:0/3077202575 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949c0329e0 con 0x7f94ac0b07f0 2023-11-09T03:46:54.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.672+0000 7f94b1555640 1 -- 172.21.15.76:0/3077202575 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94ac0c6890 con 0x7f94ac0b07f0 2023-11-09T03:46:54.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.672+0000 7f94a37fe640 1 -- 172.21.15.76:0/3077202575 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c032d90 con 0x7f94ac0b07f0 2023-11-09T03:46:54.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.672+0000 7f94b1555640 1 -- 172.21.15.76:0/3077202575 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ac0c6e50 con 0x7f94ac0b07f0 2023-11-09T03:46:54.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.673+0000 7f94b1555640 1 -- 172.21.15.76:0/3077202575 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ac06b0b0 con 0x7f94ac0b07f0 2023-11-09T03:46:54.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.674+0000 7f94a37fe640 1 -- 172.21.15.76:0/3077202575 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f949c0453f0 con 0x7f94ac0b07f0 2023-11-09T03:46:54.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.674+0000 7f94a37fe640 1 --2- 172.21.15.76:0/3077202575 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f948c06ae70 0x7f948c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:54.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.674+0000 7f94a37fe640 1 -- 172.21.15.76:0/3077202575 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f949c0ab7a0 con 0x7f94ac0b07f0 2023-11-09T03:46:54.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.674+0000 7f94a3fff640 1 --2- 172.21.15.76:0/3077202575 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f948c06ae70 0x7f948c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:54.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.675+0000 7f94a3fff640 1 --2- 172.21.15.76:0/3077202575 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f948c06ae70 0x7f948c06d330 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0ce960 tx=0x7f9494008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:54.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.678+0000 7f94a37fe640 1 -- 172.21.15.76:0/3077202575 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f949c0767d0 con 0x7f94ac0b07f0 2023-11-09T03:46:54.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.880+0000 7f94b1555640 1 -- 172.21.15.76:0/3077202575 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94ac06d0a0 con 0x7f94ac0b07f0 2023-11-09T03:46:54.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.881+0000 7f94a37fe640 1 -- 172.21.15.76:0/3077202575 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f949c079df0 con 0x7f94ac0b07f0 2023-11-09T03:46:54.880 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:54.880 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4880,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/711686619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":711686619},{"type":"v1","addr":"172.21.15.76:6835","nonce":711686619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/521006441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":521006441},{"type":"v1","addr":"172.21.15.113:6837","nonce":521006441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5212,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2652975857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2652975857},{"type":"v1","addr":"172.21.15.113:6836","nonce":2652975857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:46:51.874121+0000","modified":"2023-11-09T03:46:53.868634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5601":{"gid":5601,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6834/981490030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":981490030},{"type":"v1","addr":"172.21.15.76:6834","nonce":981490030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-09T03:46:54.881 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 16 2023-11-09T03:46:54.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.882+0000 7f94b1555640 1 -- 172.21.15.76:0/3077202575 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f948c06ae70 msgr2=0x7f948c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:54.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.882+0000 7f94b1555640 1 --2- 172.21.15.76:0/3077202575 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f948c06ae70 0x7f948c06d330 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0ce960 tx=0x7f9494008040 comp rx=0 tx=0).stop 2023-11-09T03:46:54.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.882+0000 7f94b1555640 1 -- 172.21.15.76:0/3077202575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94ac0b07f0 msgr2=0x7f94ac0c5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:54.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.882+0000 7f94b1555640 1 --2- 172.21.15.76:0/3077202575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94ac0b07f0 0x7f94ac0c5e40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f949c000c00 tx=0x7f949c0046c0 comp rx=0 tx=0).stop 2023-11-09T03:46:54.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.883+0000 7f94b1555640 1 -- 172.21.15.76:0/3077202575 shutdown_connections 2023-11-09T03:46:54.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.883+0000 7f94b1555640 1 --2- 172.21.15.76:0/3077202575 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f948c06ae70 0x7f948c06d330 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:54.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.883+0000 7f94b1555640 1 --2- 172.21.15.76:0/3077202575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94ac15cf30 0x7f94ac0c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:54.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.883+0000 7f94b1555640 1 --2- 172.21.15.76:0/3077202575 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f94ac0cce90 0x7f94ac0c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:54.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.883+0000 7f94b1555640 1 --2- 172.21.15.76:0/3077202575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94ac0b07f0 0x7f94ac0c5e40 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:54.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.883+0000 7f94b1555640 1 -- 172.21.15.76:0/3077202575 >> 172.21.15.76:0/3077202575 conn(0x7f94ac0cb7f0 msgr2=0x7f94ac06d330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:54.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.883+0000 7f94b1555640 1 -- 172.21.15.76:0/3077202575 shutdown_connections 2023-11-09T03:46:54.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.883+0000 7f94b1555640 1 -- 172.21.15.76:0/3077202575 wait complete. 2023-11-09T03:46:54.893 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:46:54.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.993+0000 7fb2bba7c640 1 Processor -- start 2023-11-09T03:46:54.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.993+0000 7fb2bba7c640 1 -- start start 2023-11-09T03:46:54.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.994+0000 7fb2bba7c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2b40b42c0 0x7fb2b40c9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:54.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.994+0000 7fb2bba7c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb2b40c9c40 0x7fb2b40b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:54.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.994+0000 7fb2bba7c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2b40ca090 con 0x7fb2b415ad80 2023-11-09T03:46:54.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.994+0000 7fb2bba7c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2b406cfa0 con 0x7fb2b40c9c40 2023-11-09T03:46:54.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.994+0000 7fb2bba7c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b406d280 con 0x7fb2b40b42c0 2023-11-09T03:46:54.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.994+0000 7fb2b9017640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2b40b42c0 0x7fb2b40c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:54.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.994+0000 7fb2ba019640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb2b40c9c40 0x7fb2b40b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:54.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.994+0000 7fb2ba019640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb2b40c9c40 0x7fb2b40b1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:46096/0 (socket says 172.21.15.76:46096) 2023-11-09T03:46:54.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.994+0000 7fb2b9017640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2b40b42c0 0x7fb2b40c9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:37484/0 (socket says 172.21.15.76:37484) 2023-11-09T03:46:54.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.994+0000 7fb2ba019640 1 -- 172.21.15.76:0/2598770169 learned_addr learned my addr 172.21.15.76:0/2598770169 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:54.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.994+0000 7fb2b9017640 1 -- 172.21.15.76:0/2598770169 >> v2:172.21.15.113:3300/0 conn(0x7fb2b40c9c40 msgr2=0x7fb2b40b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:54.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.994+0000 7fb2b9017640 1 --2- 172.21.15.76:0/2598770169 >> v2:172.21.15.113:3300/0 conn(0x7fb2b40c9c40 0x7fb2b40b1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:54.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.994+0000 7fb2b9017640 1 -- 172.21.15.76:0/2598770169 >> v1:172.21.15.113:6789/0 conn(0x7fb2b415ad80 legacy=0x7fb2b406c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:54.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.994+0000 7fb2b9017640 1 -- 172.21.15.76:0/2598770169 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b406d560 con 0x7fb2b40b42c0 2023-11-09T03:46:54.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.995+0000 7fb2b9017640 1 --2- 172.21.15.76:0/2598770169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2b40b42c0 0x7fb2b40c9700 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb2a40097a0 tx=0x7fb2a402b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39e1de9350308fcf server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:54.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.995+0000 7fb2b8816640 1 -- 172.21.15.76:0/2598770169 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4030610 con 0x7fb2b40b42c0 2023-11-09T03:46:54.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.995+0000 7fb2b8816640 1 -- 172.21.15.76:0/2598770169 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a4030770 con 0x7fb2b40b42c0 2023-11-09T03:46:54.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.995+0000 7fb2b8816640 1 -- 172.21.15.76:0/2598770169 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4030b00 con 0x7fb2b40b42c0 2023-11-09T03:46:54.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.996+0000 7fb2bba7c640 1 -- 172.21.15.76:0/2598770169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2b40b42c0 msgr2=0x7fb2b40c9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:54.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.996+0000 7fb2bba7c640 1 --2- 172.21.15.76:0/2598770169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2b40b42c0 0x7fb2b40c9700 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb2a40097a0 tx=0x7fb2a402b960 comp rx=0 tx=0).stop 2023-11-09T03:46:54.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.996+0000 7fb2bba7c640 1 -- 172.21.15.76:0/2598770169 shutdown_connections 2023-11-09T03:46:54.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.996+0000 7fb2bba7c640 1 --2- 172.21.15.76:0/2598770169 >> v2:172.21.15.113:3300/0 conn(0x7fb2b40c9c40 0x7fb2b40b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:54.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.996+0000 7fb2bba7c640 1 --2- 172.21.15.76:0/2598770169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2b40b42c0 0x7fb2b40c9700 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:54.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.996+0000 7fb2bba7c640 1 -- 172.21.15.76:0/2598770169 >> 172.21.15.76:0/2598770169 conn(0x7fb2b40ce950 msgr2=0x7fb2b40ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:54.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.996+0000 7fb2bba7c640 1 -- 172.21.15.76:0/2598770169 shutdown_connections 2023-11-09T03:46:55.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.996+0000 7fb2bba7c640 1 -- 172.21.15.76:0/2598770169 wait complete. 2023-11-09T03:46:55.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.997+0000 7fb2bba7c640 1 Processor -- start 2023-11-09T03:46:55.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.997+0000 7fb2bba7c640 1 -- start start 2023-11-09T03:46:55.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.998+0000 7fb2bba7c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2b40b42c0 0x7fb2b4156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:55.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.998+0000 7fb2bba7c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2b40c9c40 0x7fb2b40ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:55.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.998+0000 7fb2bba7c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2b415ad80 0x7fb2b40cc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:55.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.998+0000 7fb2bba7c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b40c70a0 con 0x7fb2b40c9c40 2023-11-09T03:46:55.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.998+0000 7fb2bba7c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b40c6d70 con 0x7fb2b40b42c0 2023-11-09T03:46:55.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.998+0000 7fb2bba7c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b40cd780 con 0x7fb2b415ad80 2023-11-09T03:46:55.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.998+0000 7fb2b9017640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2b40c9c40 0x7fb2b40ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:55.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.998+0000 7fb2b9017640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2b40c9c40 0x7fb2b40ab8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41642/0 (socket says 172.21.15.76:41642) 2023-11-09T03:46:55.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.998+0000 7fb2b9017640 1 -- 172.21.15.76:0/2747949613 learned_addr learned my addr 172.21.15.76:0/2747949613 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:55.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.998+0000 7fb2ba019640 1 --2- 172.21.15.76:0/2747949613 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2b415ad80 0x7fb2b40cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:55.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.998+0000 7fb2b9818640 1 --2- 172.21.15.76:0/2747949613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2b40b42c0 0x7fb2b4156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:55.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.998+0000 7fb2b9017640 1 -- 172.21.15.76:0/2747949613 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2b415ad80 msgr2=0x7fb2b40cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:55.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.998+0000 7fb2b9017640 1 --2- 172.21.15.76:0/2747949613 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2b415ad80 0x7fb2b40cc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.998+0000 7fb2b9017640 1 -- 172.21.15.76:0/2747949613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2b40b42c0 msgr2=0x7fb2b4156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:55.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.998+0000 7fb2b9017640 1 --2- 172.21.15.76:0/2747949613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2b40b42c0 0x7fb2b4156c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.998+0000 7fb2b9017640 1 -- 172.21.15.76:0/2747949613 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a4009400 con 0x7fb2b40c9c40 2023-11-09T03:46:55.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.999+0000 7fb2b9017640 1 --2- 172.21.15.76:0/2747949613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2b40c9c40 0x7fb2b40ab8e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009890 tx=0x7fb2a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:55.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.999+0000 7fb2aa7fc640 1 -- 172.21.15.76:0/2747949613 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a403e040 con 0x7fb2b40c9c40 2023-11-09T03:46:55.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.999+0000 7fb2aa7fc640 1 -- 172.21.15.76:0/2747949613 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a4039ce0 con 0x7fb2b40c9c40 2023-11-09T03:46:55.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.999+0000 7fb2bba7c640 1 -- 172.21.15.76:0/2747949613 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b40ccee0 con 0x7fb2b40c9c40 2023-11-09T03:46:55.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.999+0000 7fb2aa7fc640 1 -- 172.21.15.76:0/2747949613 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4043680 con 0x7fb2b40c9c40 2023-11-09T03:46:55.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:54.999+0000 7fb2bba7c640 1 -- 172.21.15.76:0/2747949613 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b40cd4a0 con 0x7fb2b40c9c40 2023-11-09T03:46:55.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.000+0000 7fb2bba7c640 1 -- 172.21.15.76:0/2747949613 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2b4074290 con 0x7fb2b40c9c40 2023-11-09T03:46:55.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.000+0000 7fb2aa7fc640 1 -- 172.21.15.76:0/2747949613 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb2a40423e0 con 0x7fb2b40c9c40 2023-11-09T03:46:55.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.000+0000 7fb2aa7fc640 1 --2- 172.21.15.76:0/2747949613 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb29806aea0 0x7fb29806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:55.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.001+0000 7fb2aa7fc640 1 -- 172.21.15.76:0/2747949613 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb2a40ab7b0 con 0x7fb2b40c9c40 2023-11-09T03:46:55.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.001+0000 7fb2b9818640 1 --2- 172.21.15.76:0/2747949613 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb29806aea0 0x7fb29806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:55.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.001+0000 7fb2b9818640 1 --2- 172.21.15.76:0/2747949613 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb29806aea0 0x7fb29806d360 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0000f60 tx=0x7fb2b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:55.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.005+0000 7fb2aa7fc640 1 -- 172.21.15.76:0/2747949613 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb2a40766a0 con 0x7fb2b40c9c40 2023-11-09T03:46:55.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.171+0000 7fb2bba7c640 1 -- 172.21.15.76:0/2747949613 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2b40b4880 con 0x7fb2b40c9c40 2023-11-09T03:46:55.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.172+0000 7fb2aa7fc640 1 -- 172.21.15.76:0/2747949613 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16762 (secure 0 0 0) 0x7fb2a4079cc0 con 0x7fb2b40c9c40 2023-11-09T03:46:55.171 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:55.172 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":39,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:46:53.535949+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:46:50.064765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-09T03:46:50.837125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:46:55.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.174+0000 7fb2bba7c640 1 -- 172.21.15.76:0/2747949613 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb29806aea0 msgr2=0x7fb29806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:55.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.174+0000 7fb2bba7c640 1 --2- 172.21.15.76:0/2747949613 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb29806aea0 0x7fb29806d360 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0000f60 tx=0x7fb2b0009040 comp rx=0 tx=0).stop 2023-11-09T03:46:55.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.174+0000 7fb2bba7c640 1 -- 172.21.15.76:0/2747949613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2b40c9c40 msgr2=0x7fb2b40ab8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:55.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.174+0000 7fb2bba7c640 1 --2- 172.21.15.76:0/2747949613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2b40c9c40 0x7fb2b40ab8e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009890 tx=0x7fb2a4004730 comp rx=0 tx=0).stop 2023-11-09T03:46:55.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.174+0000 7fb2bba7c640 1 -- 172.21.15.76:0/2747949613 shutdown_connections 2023-11-09T03:46:55.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.174+0000 7fb2bba7c640 1 --2- 172.21.15.76:0/2747949613 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb29806aea0 0x7fb29806d360 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.174+0000 7fb2bba7c640 1 --2- 172.21.15.76:0/2747949613 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2b415ad80 0x7fb2b40cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.174+0000 7fb2bba7c640 1 --2- 172.21.15.76:0/2747949613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2b40c9c40 0x7fb2b40ab8e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.174+0000 7fb2bba7c640 1 --2- 172.21.15.76:0/2747949613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2b40b42c0 0x7fb2b4156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.174+0000 7fb2bba7c640 1 -- 172.21.15.76:0/2747949613 >> 172.21.15.76:0/2747949613 conn(0x7fb2b40ce950 msgr2=0x7fb2b4070630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:55.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.174+0000 7fb2bba7c640 1 -- 172.21.15.76:0/2747949613 shutdown_connections 2023-11-09T03:46:55.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.174+0000 7fb2bba7c640 1 -- 172.21.15.76:0/2747949613 wait complete. 2023-11-09T03:46:55.184 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-09T03:46:55.184 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-09T03:46:55.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.286+0000 7fcba7690640 1 Processor -- start 2023-11-09T03:46:55.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.286+0000 7fcba7690640 1 -- start start 2023-11-09T03:46:55.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.287+0000 7fcba7690640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcba00cde60 0x7fcba015d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:55.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.287+0000 7fcba7690640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcba00ce240 0x7fcba00d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:55.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.287+0000 7fcba7690640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba015e000 con 0x7fcba015daa0 2023-11-09T03:46:55.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.287+0000 7fcba7690640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcba015e5c0 con 0x7fcba00cde60 2023-11-09T03:46:55.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.287+0000 7fcba7690640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcba015e2e0 con 0x7fcba00ce240 2023-11-09T03:46:55.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.287+0000 7fcba542c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcba00cde60 0x7fcba015d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:55.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.287+0000 7fcba4c2b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fcba015daa0 0x7fcba0157000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59872/0 (socket says 172.21.15.76:59872) 2023-11-09T03:46:55.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.287+0000 7fcba4c2b640 1 -- 172.21.15.76:0/1196848520 learned_addr learned my addr 172.21.15.76:0/1196848520 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:55.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.288+0000 7fcba5c2d640 1 --2- 172.21.15.76:0/1196848520 >> v2:172.21.15.113:3300/0 conn(0x7fcba00ce240 0x7fcba00d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:55.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.288+0000 7fcba542c640 1 -- 172.21.15.76:0/1196848520 >> v2:172.21.15.113:3300/0 conn(0x7fcba00ce240 msgr2=0x7fcba00d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:55.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.288+0000 7fcba542c640 1 --2- 172.21.15.76:0/1196848520 >> v2:172.21.15.113:3300/0 conn(0x7fcba00ce240 0x7fcba00d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.288+0000 7fcba542c640 1 -- 172.21.15.76:0/1196848520 >> v1:172.21.15.76:6789/0 conn(0x7fcba015daa0 legacy=0x7fcba0157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:55.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.288+0000 7fcba542c640 1 -- 172.21.15.76:0/1196848520 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba00d1120 con 0x7fcba00cde60 2023-11-09T03:46:55.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.288+0000 7fcb97fff640 1 -- 172.21.15.76:0/1196848520 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 881881572 0 0) 0x7fcba015e000 con 0x7fcba015daa0 2023-11-09T03:46:55.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.288+0000 7fcba542c640 1 --2- 172.21.15.76:0/1196848520 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcba00cde60 0x7fcba015d560 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fcba00cf2e0 tx=0x7fcb9c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3c16c80909abac server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:55.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.289+0000 7fcb97fff640 1 -- 172.21.15.76:0/1196848520 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c02cc80 con 0x7fcba00cde60 2023-11-09T03:46:55.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.289+0000 7fcb97fff640 1 -- 172.21.15.76:0/1196848520 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9c02cde0 con 0x7fcba00cde60 2023-11-09T03:46:55.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.289+0000 7fcb97fff640 1 -- 172.21.15.76:0/1196848520 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c034660 con 0x7fcba00cde60 2023-11-09T03:46:55.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.289+0000 7fcba7690640 1 -- 172.21.15.76:0/1196848520 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcba00cde60 msgr2=0x7fcba015d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:55.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.289+0000 7fcba7690640 1 --2- 172.21.15.76:0/1196848520 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcba00cde60 0x7fcba015d560 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fcba00cf2e0 tx=0x7fcb9c02bed0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.289+0000 7fcba7690640 1 -- 172.21.15.76:0/1196848520 shutdown_connections 2023-11-09T03:46:55.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.289+0000 7fcba7690640 1 --2- 172.21.15.76:0/1196848520 >> v2:172.21.15.113:3300/0 conn(0x7fcba00ce240 0x7fcba00d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.289+0000 7fcba7690640 1 --2- 172.21.15.76:0/1196848520 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcba00cde60 0x7fcba015d560 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.289+0000 7fcba7690640 1 -- 172.21.15.76:0/1196848520 >> 172.21.15.76:0/1196848520 conn(0x7fcba0071f30 msgr2=0x7fcba00cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:55.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.289+0000 7fcba7690640 1 -- 172.21.15.76:0/1196848520 shutdown_connections 2023-11-09T03:46:55.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.289+0000 7fcba7690640 1 -- 172.21.15.76:0/1196848520 wait complete. 2023-11-09T03:46:55.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.290+0000 7fcba7690640 1 Processor -- start 2023-11-09T03:46:55.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.290+0000 7fcba7690640 1 -- start start 2023-11-09T03:46:55.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.291+0000 7fcba7690640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcba00cde60 0x7fcba0159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:55.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.291+0000 7fcba7690640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcba00ce240 0x7fcba0157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:55.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.291+0000 7fcba7690640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcba015daa0 0x7fcba015c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:55.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.291+0000 7fcba7690640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba015a0d0 con 0x7fcba00ce240 2023-11-09T03:46:55.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.291+0000 7fcba7690640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba0158450 con 0x7fcba00cde60 2023-11-09T03:46:55.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.291+0000 7fcba7690640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba0156e70 con 0x7fcba015daa0 2023-11-09T03:46:55.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.291+0000 7fcba4c2b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcba00ce240 0x7fcba0157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:55.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.291+0000 7fcba4c2b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcba00ce240 0x7fcba0157f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41658/0 (socket says 172.21.15.76:41658) 2023-11-09T03:46:55.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.291+0000 7fcba4c2b640 1 -- 172.21.15.76:0/2232559955 learned_addr learned my addr 172.21.15.76:0/2232559955 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:55.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.291+0000 7fcba4c2b640 1 -- 172.21.15.76:0/2232559955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcba015daa0 msgr2=0x7fcba015c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:55.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.291+0000 7fcba4c2b640 1 --2- 172.21.15.76:0/2232559955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcba015daa0 0x7fcba015c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.291+0000 7fcba4c2b640 1 -- 172.21.15.76:0/2232559955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcba00cde60 msgr2=0x7fcba0159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:55.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.291+0000 7fcba542c640 1 --2- 172.21.15.76:0/2232559955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcba00cde60 0x7fcba0159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:55.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.291+0000 7fcba4c2b640 1 --2- 172.21.15.76:0/2232559955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcba00cde60 0x7fcba0159b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.298 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.291+0000 7fcba4c2b640 1 -- 172.21.15.76:0/2232559955 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb9c009a70 con 0x7fcba00ce240 2023-11-09T03:46:55.298 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.292+0000 7fcba4c2b640 1 --2- 172.21.15.76:0/2232559955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcba00ce240 0x7fcba0157f10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000c410 tx=0x7fcb9000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:55.298 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.292+0000 7fcb95ffb640 1 -- 172.21.15.76:0/2232559955 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb900074e0 con 0x7fcba00ce240 2023-11-09T03:46:55.298 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.292+0000 7fcb95ffb640 1 -- 172.21.15.76:0/2232559955 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb90007660 con 0x7fcba00ce240 2023-11-09T03:46:55.298 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.292+0000 7fcba7690640 1 -- 172.21.15.76:0/2232559955 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba0157150 con 0x7fcba00ce240 2023-11-09T03:46:55.298 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.292+0000 7fcb95ffb640 1 -- 172.21.15.76:0/2232559955 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90004a20 con 0x7fcba00ce240 2023-11-09T03:46:55.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.292+0000 7fcba7690640 1 -- 172.21.15.76:0/2232559955 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba0157710 con 0x7fcba00ce240 2023-11-09T03:46:55.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.293+0000 7fcba7690640 1 -- 172.21.15.76:0/2232559955 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba0074290 con 0x7fcba00ce240 2023-11-09T03:46:55.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.293+0000 7fcb95ffb640 1 -- 172.21.15.76:0/2232559955 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcb90004b80 con 0x7fcba00ce240 2023-11-09T03:46:55.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.293+0000 7fcb95ffb640 1 --2- 172.21.15.76:0/2232559955 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcb8406ae70 0x7fcb8406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:55.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.294+0000 7fcb95ffb640 1 -- 172.21.15.76:0/2232559955 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcb90089530 con 0x7fcba00ce240 2023-11-09T03:46:55.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.294+0000 7fcba542c640 1 --2- 172.21.15.76:0/2232559955 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcb8406ae70 0x7fcb8406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:55.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.294+0000 7fcba542c640 1 --2- 172.21.15.76:0/2232559955 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcb8406ae70 0x7fcb8406d330 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c036000 tx=0x7fcb9c030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:55.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.298+0000 7fcb95ffb640 1 -- 172.21.15.76:0/2232559955 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcb90057f30 con 0x7fcba00ce240 2023-11-09T03:46:55.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.462+0000 7fcba7690640 1 -- 172.21.15.76:0/2232559955 --> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcba00d1400 con 0x7fcb8406ae70 2023-11-09T03:46:55.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.470+0000 7fcb95ffb640 1 -- 172.21.15.76:0/2232559955 <== mgr.4105 v2:172.21.15.113:6832/1264193042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288910 (secure 0 0 0) 0x7fcb9c0344c0 con 0x7fcb8406ae70 2023-11-09T03:46:55.472 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:55.473 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2023-11-09T03:46:55.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.473+0000 7fcba7690640 1 -- 172.21.15.76:0/2232559955 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcb8406ae70 msgr2=0x7fcb8406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:55.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.473+0000 7fcba7690640 1 --2- 172.21.15.76:0/2232559955 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcb8406ae70 0x7fcb8406d330 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c036000 tx=0x7fcb9c030000 comp rx=0 tx=0).stop 2023-11-09T03:46:55.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.473+0000 7fcba7690640 1 -- 172.21.15.76:0/2232559955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcba00ce240 msgr2=0x7fcba0157f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:55.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.473+0000 7fcba7690640 1 --2- 172.21.15.76:0/2232559955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcba00ce240 0x7fcba0157f10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000c410 tx=0x7fcb9000c8d0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.473+0000 7fcba7690640 1 -- 172.21.15.76:0/2232559955 shutdown_connections 2023-11-09T03:46:55.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.473+0000 7fcba7690640 1 --2- 172.21.15.76:0/2232559955 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcb8406ae70 0x7fcb8406d330 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.473+0000 7fcba7690640 1 --2- 172.21.15.76:0/2232559955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcba015daa0 0x7fcba015c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.473+0000 7fcba7690640 1 --2- 172.21.15.76:0/2232559955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcba00ce240 0x7fcba0157f10 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.473+0000 7fcba7690640 1 --2- 172.21.15.76:0/2232559955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcba00cde60 0x7fcba0159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.473+0000 7fcba7690640 1 -- 172.21.15.76:0/2232559955 >> 172.21.15.76:0/2232559955 conn(0x7fcba0071f30 msgr2=0x7fcba00c6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:55.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.473+0000 7fcba7690640 1 -- 172.21.15.76:0/2232559955 shutdown_connections 2023-11-09T03:46:55.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.473+0000 7fcba7690640 1 -- 172.21.15.76:0/2232559955 wait complete. 2023-11-09T03:46:55.483 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2023-11-09T03:46:54.770073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220688,"kb_used_data":4500,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754032,"statfs":{"total":773094113280,"available":772868128768,"internally_reserved":0,"allocated":4608000,"data_stored":2624610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.252635"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.555771+0000","last_change":"2023-11-09T03:46:52.549534+0000","last_active":"2023-11-09T03:46:52.555771+0000","last_peered":"2023-11-09T03:46:52.555771+0000","last_clean":"2023-11-09T03:46:52.555771+0000","last_became_active":"2023-11-09T03:46:52.549269+0000","last_became_peered":"2023-11-09T03:46:52.549269+0000","last_unstale":"2023-11-09T03:46:52.555771+0000","last_undegraded":"2023-11-09T03:46:52.555771+0000","last_fullsized":"2023-11-09T03:46:52.555771+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:44.261724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.549872+0000","last_change":"2023-11-09T03:46:52.544197+0000","last_active":"2023-11-09T03:46:52.549872+0000","last_peered":"2023-11-09T03:46:52.549872+0000","last_clean":"2023-11-09T03:46:52.549872+0000","last_became_active":"2023-11-09T03:46:52.543955+0000","last_became_peered":"2023-11-09T03:46:52.543955+0000","last_unstale":"2023-11-09T03:46:52.549872+0000","last_undegraded":"2023-11-09T03:46:52.549872+0000","last_fullsized":"2023-11-09T03:46:52.549872+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:18:49.182588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556110+0000","last_change":"2023-11-09T03:46:52.548802+0000","last_active":"2023-11-09T03:46:52.556110+0000","last_peered":"2023-11-09T03:46:52.556110+0000","last_clean":"2023-11-09T03:46:52.556110+0000","last_became_active":"2023-11-09T03:46:52.548527+0000","last_became_peered":"2023-11-09T03:46:52.548527+0000","last_unstale":"2023-11-09T03:46:52.556110+0000","last_undegraded":"2023-11-09T03:46:52.556110+0000","last_fullsized":"2023-11-09T03:46:52.556110+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:30:39.800810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.555891+0000","last_change":"2023-11-09T03:46:52.547613+0000","last_active":"2023-11-09T03:46:52.555891+0000","last_peered":"2023-11-09T03:46:52.555891+0000","last_clean":"2023-11-09T03:46:52.555891+0000","last_became_active":"2023-11-09T03:46:52.547371+0000","last_became_peered":"2023-11-09T03:46:52.547371+0000","last_unstale":"2023-11-09T03:46:52.555891+0000","last_undegraded":"2023-11-09T03:46:52.555891+0000","last_fullsized":"2023-11-09T03:46:52.555891+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:05:57.334303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.555825+0000","last_change":"2023-11-09T03:46:52.547070+0000","last_active":"2023-11-09T03:46:52.555825+0000","last_peered":"2023-11-09T03:46:52.555825+0000","last_clean":"2023-11-09T03:46:52.555825+0000","last_became_active":"2023-11-09T03:46:52.546795+0000","last_became_peered":"2023-11-09T03:46:52.546795+0000","last_unstale":"2023-11-09T03:46:52.555825+0000","last_undegraded":"2023-11-09T03:46:52.555825+0000","last_fullsized":"2023-11-09T03:46:52.555825+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:14:56.145315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.550611+0000","last_change":"2023-11-09T03:46:52.547806+0000","last_active":"2023-11-09T03:46:52.550611+0000","last_peered":"2023-11-09T03:46:52.550611+0000","last_clean":"2023-11-09T03:46:52.550611+0000","last_became_active":"2023-11-09T03:46:52.547536+0000","last_became_peered":"2023-11-09T03:46:52.547536+0000","last_unstale":"2023-11-09T03:46:52.550611+0000","last_undegraded":"2023-11-09T03:46:52.550611+0000","last_fullsized":"2023-11-09T03:46:52.550611+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:33:19.817774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.542641+0000","last_change":"2023-11-09T03:46:52.541151+0000","last_active":"2023-11-09T03:46:52.542641+0000","last_peered":"2023-11-09T03:46:52.542641+0000","last_clean":"2023-11-09T03:46:52.542641+0000","last_became_active":"2023-11-09T03:46:52.540868+0000","last_became_peered":"2023-11-09T03:46:52.540868+0000","last_unstale":"2023-11-09T03:46:52.542641+0000","last_undegraded":"2023-11-09T03:46:52.542641+0000","last_fullsized":"2023-11-09T03:46:52.542641+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:24:42.894740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556038+0000","last_change":"2023-11-09T03:46:52.550418+0000","last_active":"2023-11-09T03:46:52.556038+0000","last_peered":"2023-11-09T03:46:52.556038+0000","last_clean":"2023-11-09T03:46:52.556038+0000","last_became_active":"2023-11-09T03:46:52.550180+0000","last_became_peered":"2023-11-09T03:46:52.550180+0000","last_unstale":"2023-11-09T03:46:52.556038+0000","last_undegraded":"2023-11-09T03:46:52.556038+0000","last_fullsized":"2023-11-09T03:46:52.556038+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:29:03.369650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556078+0000","last_change":"2023-11-09T03:46:52.548094+0000","last_active":"2023-11-09T03:46:52.556078+0000","last_peered":"2023-11-09T03:46:52.556078+0000","last_clean":"2023-11-09T03:46:52.556078+0000","last_became_active":"2023-11-09T03:46:52.547849+0000","last_became_peered":"2023-11-09T03:46:52.547849+0000","last_unstale":"2023-11-09T03:46:52.556078+0000","last_undegraded":"2023-11-09T03:46:52.556078+0000","last_fullsized":"2023-11-09T03:46:52.556078+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:14:57.063826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556405+0000","last_change":"2023-11-09T03:46:52.549352+0000","last_active":"2023-11-09T03:46:52.556405+0000","last_peered":"2023-11-09T03:46:52.556405+0000","last_clean":"2023-11-09T03:46:52.556405+0000","last_became_active":"2023-11-09T03:46:52.549113+0000","last_became_peered":"2023-11-09T03:46:52.549113+0000","last_unstale":"2023-11-09T03:46:52.556405+0000","last_undegraded":"2023-11-09T03:46:52.556405+0000","last_fullsized":"2023-11-09T03:46:52.556405+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:57:09.422287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.559055+0000","last_change":"2023-11-09T03:46:52.553667+0000","last_active":"2023-11-09T03:46:52.559055+0000","last_peered":"2023-11-09T03:46:52.559055+0000","last_clean":"2023-11-09T03:46:52.559055+0000","last_became_active":"2023-11-09T03:46:52.551543+0000","last_became_peered":"2023-11-09T03:46:52.551543+0000","last_unstale":"2023-11-09T03:46:52.559055+0000","last_undegraded":"2023-11-09T03:46:52.559055+0000","last_fullsized":"2023-11-09T03:46:52.559055+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:29:03.369650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556133+0000","last_change":"2023-11-09T03:46:52.548952+0000","last_active":"2023-11-09T03:46:52.556133+0000","last_peered":"2023-11-09T03:46:52.556133+0000","last_clean":"2023-11-09T03:46:52.556133+0000","last_became_active":"2023-11-09T03:46:52.548676+0000","last_became_peered":"2023-11-09T03:46:52.548676+0000","last_unstale":"2023-11-09T03:46:52.556133+0000","last_undegraded":"2023-11-09T03:46:52.556133+0000","last_fullsized":"2023-11-09T03:46:52.556133+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:17:18.364598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556287+0000","last_change":"2023-11-09T03:46:52.548868+0000","last_active":"2023-11-09T03:46:52.556287+0000","last_peered":"2023-11-09T03:46:52.556287+0000","last_clean":"2023-11-09T03:46:52.556287+0000","last_became_active":"2023-11-09T03:46:52.548619+0000","last_became_peered":"2023-11-09T03:46:52.548619+0000","last_unstale":"2023-11-09T03:46:52.556287+0000","last_undegraded":"2023-11-09T03:46:52.556287+0000","last_fullsized":"2023-11-09T03:46:52.556287+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:41:30.870772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.559003+0000","last_change":"2023-11-09T03:46:52.554119+0000","last_active":"2023-11-09T03:46:52.559003+0000","last_peered":"2023-11-09T03:46:52.559003+0000","last_clean":"2023-11-09T03:46:52.559003+0000","last_became_active":"2023-11-09T03:46:52.553810+0000","last_became_peered":"2023-11-09T03:46:52.553810+0000","last_unstale":"2023-11-09T03:46:52.559003+0000","last_undegraded":"2023-11-09T03:46:52.559003+0000","last_fullsized":"2023-11-09T03:46:52.559003+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:33:19.817774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556479+0000","last_change":"2023-11-09T03:46:52.549554+0000","last_active":"2023-11-09T03:46:52.556479+0000","last_peered":"2023-11-09T03:46:52.556479+0000","last_clean":"2023-11-09T03:46:52.556479+0000","last_became_active":"2023-11-09T03:46:52.549296+0000","last_became_peered":"2023-11-09T03:46:52.549296+0000","last_unstale":"2023-11-09T03:46:52.556479+0000","last_undegraded":"2023-11-09T03:46:52.556479+0000","last_fullsized":"2023-11-09T03:46:52.556479+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:15:53.977101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.549050+0000","last_change":"2023-11-09T03:46:52.544776+0000","last_active":"2023-11-09T03:46:52.549050+0000","last_peered":"2023-11-09T03:46:52.549050+0000","last_clean":"2023-11-09T03:46:52.549050+0000","last_became_active":"2023-11-09T03:46:52.544530+0000","last_became_peered":"2023-11-09T03:46:52.544530+0000","last_unstale":"2023-11-09T03:46:52.549050+0000","last_undegraded":"2023-11-09T03:46:52.549050+0000","last_fullsized":"2023-11-09T03:46:52.549050+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:33:19.817774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.554750+0000","last_change":"2023-11-09T03:46:52.547461+0000","last_active":"2023-11-09T03:46:52.554750+0000","last_peered":"2023-11-09T03:46:52.554750+0000","last_clean":"2023-11-09T03:46:52.554750+0000","last_became_active":"2023-11-09T03:46:52.547189+0000","last_became_peered":"2023-11-09T03:46:52.547189+0000","last_unstale":"2023-11-09T03:46:52.554750+0000","last_undegraded":"2023-11-09T03:46:52.554750+0000","last_fullsized":"2023-11-09T03:46:52.554750+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:54:01.680591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556422+0000","last_change":"2023-11-09T03:46:52.550701+0000","last_active":"2023-11-09T03:46:52.556422+0000","last_peered":"2023-11-09T03:46:52.556422+0000","last_clean":"2023-11-09T03:46:52.556422+0000","last_became_active":"2023-11-09T03:46:52.550425+0000","last_became_peered":"2023-11-09T03:46:52.550425+0000","last_unstale":"2023-11-09T03:46:52.556422+0000","last_undegraded":"2023-11-09T03:46:52.556422+0000","last_fullsized":"2023-11-09T03:46:52.556422+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:45:32.257609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.553224+0000","last_change":"2023-11-09T03:46:52.542025+0000","last_active":"2023-11-09T03:46:52.553224+0000","last_peered":"2023-11-09T03:46:52.553224+0000","last_clean":"2023-11-09T03:46:52.553224+0000","last_became_active":"2023-11-09T03:46:52.541728+0000","last_became_peered":"2023-11-09T03:46:52.541728+0000","last_unstale":"2023-11-09T03:46:52.553224+0000","last_undegraded":"2023-11-09T03:46:52.553224+0000","last_fullsized":"2023-11-09T03:46:52.553224+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:49:00.630877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556363+0000","last_change":"2023-11-09T03:46:52.551316+0000","last_active":"2023-11-09T03:46:52.556363+0000","last_peered":"2023-11-09T03:46:52.556363+0000","last_clean":"2023-11-09T03:46:52.556363+0000","last_became_active":"2023-11-09T03:46:52.551061+0000","last_became_peered":"2023-11-09T03:46:52.551061+0000","last_unstale":"2023-11-09T03:46:52.556363+0000","last_undegraded":"2023-11-09T03:46:52.556363+0000","last_fullsized":"2023-11-09T03:46:52.556363+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:33:19.817774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.550330+0000","last_change":"2023-11-09T03:46:52.545583+0000","last_active":"2023-11-09T03:46:52.550330+0000","last_peered":"2023-11-09T03:46:52.550330+0000","last_clean":"2023-11-09T03:46:52.550330+0000","last_became_active":"2023-11-09T03:46:52.545323+0000","last_became_peered":"2023-11-09T03:46:52.545323+0000","last_unstale":"2023-11-09T03:46:52.550330+0000","last_undegraded":"2023-11-09T03:46:52.550330+0000","last_fullsized":"2023-11-09T03:46:52.550330+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:21:25.255050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.555937+0000","last_change":"2023-11-09T03:46:52.549453+0000","last_active":"2023-11-09T03:46:52.555937+0000","last_peered":"2023-11-09T03:46:52.555937+0000","last_clean":"2023-11-09T03:46:52.555937+0000","last_became_active":"2023-11-09T03:46:52.549167+0000","last_became_peered":"2023-11-09T03:46:52.549167+0000","last_unstale":"2023-11-09T03:46:52.555937+0000","last_undegraded":"2023-11-09T03:46:52.555937+0000","last_fullsized":"2023-11-09T03:46:52.555937+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:45:32.257609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.553413+0000","last_change":"2023-11-09T03:46:52.549056+0000","last_active":"2023-11-09T03:46:52.553413+0000","last_peered":"2023-11-09T03:46:52.553413+0000","last_clean":"2023-11-09T03:46:52.553413+0000","last_became_active":"2023-11-09T03:46:52.548267+0000","last_became_peered":"2023-11-09T03:46:52.548267+0000","last_unstale":"2023-11-09T03:46:52.553413+0000","last_undegraded":"2023-11-09T03:46:52.553413+0000","last_fullsized":"2023-11-09T03:46:52.553413+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:45:42.507078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.545633+0000","last_change":"2023-11-09T03:46:52.538756+0000","last_active":"2023-11-09T03:46:52.545633+0000","last_peered":"2023-11-09T03:46:52.545633+0000","last_clean":"2023-11-09T03:46:52.545633+0000","last_became_active":"2023-11-09T03:46:52.538242+0000","last_became_peered":"2023-11-09T03:46:52.538242+0000","last_unstale":"2023-11-09T03:46:52.545633+0000","last_undegraded":"2023-11-09T03:46:52.545633+0000","last_fullsized":"2023-11-09T03:46:52.545633+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:59:08.370172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.547544+0000","last_change":"2023-11-09T03:46:52.538850+0000","last_active":"2023-11-09T03:46:52.547544+0000","last_peered":"2023-11-09T03:46:52.547544+0000","last_clean":"2023-11-09T03:46:52.547544+0000","last_became_active":"2023-11-09T03:46:52.538555+0000","last_became_peered":"2023-11-09T03:46:52.538555+0000","last_unstale":"2023-11-09T03:46:52.547544+0000","last_undegraded":"2023-11-09T03:46:52.547544+0000","last_fullsized":"2023-11-09T03:46:52.547544+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:01:07.202627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.544225+0000","last_change":"2023-11-09T03:46:52.542931+0000","last_active":"2023-11-09T03:46:52.544225+0000","last_peered":"2023-11-09T03:46:52.544225+0000","last_clean":"2023-11-09T03:46:52.544225+0000","last_became_active":"2023-11-09T03:46:52.542659+0000","last_became_peered":"2023-11-09T03:46:52.542659+0000","last_unstale":"2023-11-09T03:46:52.544225+0000","last_undegraded":"2023-11-09T03:46:52.544225+0000","last_fullsized":"2023-11-09T03:46:52.544225+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:14:57.063826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.537457+0000","last_change":"2023-11-09T03:46:52.534044+0000","last_active":"2023-11-09T03:46:52.537457+0000","last_peered":"2023-11-09T03:46:52.537457+0000","last_clean":"2023-11-09T03:46:52.537457+0000","last_became_active":"2023-11-09T03:46:52.533706+0000","last_became_peered":"2023-11-09T03:46:52.533706+0000","last_unstale":"2023-11-09T03:46:52.537457+0000","last_undegraded":"2023-11-09T03:46:52.537457+0000","last_fullsized":"2023-11-09T03:46:52.537457+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:14:57.063826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.550286+0000","last_change":"2023-11-09T03:46:52.544936+0000","last_active":"2023-11-09T03:46:52.550286+0000","last_peered":"2023-11-09T03:46:52.550286+0000","last_clean":"2023-11-09T03:46:52.550286+0000","last_became_active":"2023-11-09T03:46:52.544663+0000","last_became_peered":"2023-11-09T03:46:52.544663+0000","last_unstale":"2023-11-09T03:46:52.550286+0000","last_undegraded":"2023-11-09T03:46:52.550286+0000","last_fullsized":"2023-11-09T03:46:52.550286+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:06:21.934969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.552118+0000","last_change":"2023-11-09T03:46:52.544101+0000","last_active":"2023-11-09T03:46:52.552118+0000","last_peered":"2023-11-09T03:46:52.552118+0000","last_clean":"2023-11-09T03:46:52.552118+0000","last_became_active":"2023-11-09T03:46:52.543822+0000","last_became_peered":"2023-11-09T03:46:52.543822+0000","last_unstale":"2023-11-09T03:46:52.552118+0000","last_undegraded":"2023-11-09T03:46:52.552118+0000","last_fullsized":"2023-11-09T03:46:52.552118+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:49:44.445256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.549941+0000","last_change":"2023-11-09T03:46:52.539639+0000","last_active":"2023-11-09T03:46:52.549941+0000","last_peered":"2023-11-09T03:46:52.549941+0000","last_clean":"2023-11-09T03:46:52.549941+0000","last_became_active":"2023-11-09T03:46:52.537981+0000","last_became_peered":"2023-11-09T03:46:52.537981+0000","last_unstale":"2023-11-09T03:46:52.549941+0000","last_undegraded":"2023-11-09T03:46:52.549941+0000","last_fullsized":"2023-11-09T03:46:52.549941+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:17:18.364598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.551373+0000","last_change":"2023-11-09T03:46:52.548900+0000","last_active":"2023-11-09T03:46:52.551373+0000","last_peered":"2023-11-09T03:46:52.551373+0000","last_clean":"2023-11-09T03:46:52.551373+0000","last_became_active":"2023-11-09T03:46:52.548645+0000","last_became_peered":"2023-11-09T03:46:52.548645+0000","last_unstale":"2023-11-09T03:46:52.551373+0000","last_undegraded":"2023-11-09T03:46:52.551373+0000","last_fullsized":"2023-11-09T03:46:52.551373+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:41:30.870772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.550665+0000","last_change":"2023-11-09T03:46:52.548409+0000","last_active":"2023-11-09T03:46:52.550665+0000","last_peered":"2023-11-09T03:46:52.550665+0000","last_clean":"2023-11-09T03:46:52.550665+0000","last_became_active":"2023-11-09T03:46:52.548078+0000","last_became_peered":"2023-11-09T03:46:52.548078+0000","last_unstale":"2023-11-09T03:46:52.550665+0000","last_undegraded":"2023-11-09T03:46:52.550665+0000","last_fullsized":"2023-11-09T03:46:52.550665+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:17:18.364598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.551120+0000","last_change":"2023-11-09T03:46:52.546459+0000","last_active":"2023-11-09T03:46:52.551120+0000","last_peered":"2023-11-09T03:46:52.551120+0000","last_clean":"2023-11-09T03:46:52.551120+0000","last_became_active":"2023-11-09T03:46:52.546201+0000","last_became_peered":"2023-11-09T03:46:52.546201+0000","last_unstale":"2023-11-09T03:46:52.551120+0000","last_undegraded":"2023-11-09T03:46:52.551120+0000","last_fullsized":"2023-11-09T03:46:52.551120+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:24:42.894740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.559620+0000","last_change":"2023-11-09T03:46:51.546351+0000","last_active":"2023-11-09T03:46:52.559620+0000","last_peered":"2023-11-09T03:46:52.559620+0000","last_clean":"2023-11-09T03:46:52.559620+0000","last_became_active":"2023-11-09T03:46:51.544196+0000","last_became_peered":"2023-11-09T03:46:51.544196+0000","last_unstale":"2023-11-09T03:46:52.559620+0000","last_undegraded":"2023-11-09T03:46:52.559620+0000","last_fullsized":"2023-11-09T03:46:52.559620+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:56:44.359014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.550062+0000","last_change":"2023-11-09T03:46:52.546830+0000","last_active":"2023-11-09T03:46:52.550062+0000","last_peered":"2023-11-09T03:46:52.550062+0000","last_clean":"2023-11-09T03:46:52.550062+0000","last_became_active":"2023-11-09T03:46:52.546532+0000","last_became_peered":"2023-11-09T03:46:52.546532+0000","last_unstale":"2023-11-09T03:46:52.550062+0000","last_undegraded":"2023-11-09T03:46:52.550062+0000","last_fullsized":"2023-11-09T03:46:52.550062+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:18:34.570683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.550988+0000","last_change":"2023-11-09T03:46:51.543725+0000","last_active":"2023-11-09T03:46:52.550988+0000","last_peered":"2023-11-09T03:46:52.550988+0000","last_clean":"2023-11-09T03:46:52.550988+0000","last_became_active":"2023-11-09T03:46:51.542473+0000","last_became_peered":"2023-11-09T03:46:51.542473+0000","last_unstale":"2023-11-09T03:46:52.550988+0000","last_undegraded":"2023-11-09T03:46:52.550988+0000","last_fullsized":"2023-11-09T03:46:52.550988+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:50:37.650301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.558963+0000","last_change":"2023-11-09T03:46:52.553157+0000","last_active":"2023-11-09T03:46:52.558963+0000","last_peered":"2023-11-09T03:46:52.558963+0000","last_clean":"2023-11-09T03:46:52.558963+0000","last_became_active":"2023-11-09T03:46:52.552695+0000","last_became_peered":"2023-11-09T03:46:52.552695+0000","last_unstale":"2023-11-09T03:46:52.558963+0000","last_undegraded":"2023-11-09T03:46:52.558963+0000","last_fullsized":"2023-11-09T03:46:52.558963+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:01:07.202627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.551888+0000","last_change":"2023-11-09T03:46:51.543977+0000","last_active":"2023-11-09T03:46:52.551888+0000","last_peered":"2023-11-09T03:46:52.551888+0000","last_clean":"2023-11-09T03:46:52.551888+0000","last_became_active":"2023-11-09T03:46:51.543662+0000","last_became_peered":"2023-11-09T03:46:51.543662+0000","last_unstale":"2023-11-09T03:46:52.551888+0000","last_undegraded":"2023-11-09T03:46:52.551888+0000","last_fullsized":"2023-11-09T03:46:52.551888+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:30:38.798792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.553457+0000","last_change":"2023-11-09T03:46:52.548238+0000","last_active":"2023-11-09T03:46:52.553457+0000","last_peered":"2023-11-09T03:46:52.553457+0000","last_clean":"2023-11-09T03:46:52.553457+0000","last_became_active":"2023-11-09T03:46:52.547989+0000","last_became_peered":"2023-11-09T03:46:52.547989+0000","last_unstale":"2023-11-09T03:46:52.553457+0000","last_undegraded":"2023-11-09T03:46:52.553457+0000","last_fullsized":"2023-11-09T03:46:52.553457+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:15:53.977101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.558949+0000","last_change":"2023-11-09T03:46:51.541786+0000","last_active":"2023-11-09T03:46:52.558949+0000","last_peered":"2023-11-09T03:46:52.558949+0000","last_clean":"2023-11-09T03:46:52.558949+0000","last_became_active":"2023-11-09T03:46:51.541457+0000","last_became_peered":"2023-11-09T03:46:51.541457+0000","last_unstale":"2023-11-09T03:46:52.558949+0000","last_undegraded":"2023-11-09T03:46:52.558949+0000","last_fullsized":"2023-11-09T03:46:52.558949+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:50:37.650301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.549095+0000","last_change":"2023-11-09T03:46:52.543573+0000","last_active":"2023-11-09T03:46:52.549095+0000","last_peered":"2023-11-09T03:46:52.549095+0000","last_clean":"2023-11-09T03:46:52.549095+0000","last_became_active":"2023-11-09T03:46:52.543301+0000","last_became_peered":"2023-11-09T03:46:52.543301+0000","last_unstale":"2023-11-09T03:46:52.549095+0000","last_undegraded":"2023-11-09T03:46:52.549095+0000","last_fullsized":"2023-11-09T03:46:52.549095+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:23:12.573527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556695+0000","last_change":"2023-11-09T03:46:51.543897+0000","last_active":"2023-11-09T03:46:52.556695+0000","last_peered":"2023-11-09T03:46:52.556695+0000","last_clean":"2023-11-09T03:46:52.556695+0000","last_became_active":"2023-11-09T03:46:51.543536+0000","last_became_peered":"2023-11-09T03:46:51.543536+0000","last_unstale":"2023-11-09T03:46:52.556695+0000","last_undegraded":"2023-11-09T03:46:52.556695+0000","last_fullsized":"2023-11-09T03:46:52.556695+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:41.783929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.550312+0000","last_change":"2023-11-09T03:46:52.545027+0000","last_active":"2023-11-09T03:46:52.550312+0000","last_peered":"2023-11-09T03:46:52.550312+0000","last_clean":"2023-11-09T03:46:52.550312+0000","last_became_active":"2023-11-09T03:46:52.544727+0000","last_became_peered":"2023-11-09T03:46:52.544727+0000","last_unstale":"2023-11-09T03:46:52.550312+0000","last_undegraded":"2023-11-09T03:46:52.550312+0000","last_fullsized":"2023-11-09T03:46:52.550312+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:45:42.507078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.550636+0000","last_change":"2023-11-09T03:46:51.550073+0000","last_active":"2023-11-09T03:46:52.550636+0000","last_peered":"2023-11-09T03:46:52.550636+0000","last_clean":"2023-11-09T03:46:52.550636+0000","last_became_active":"2023-11-09T03:46:51.549783+0000","last_became_peered":"2023-11-09T03:46:51.549783+0000","last_unstale":"2023-11-09T03:46:52.550636+0000","last_undegraded":"2023-11-09T03:46:52.550636+0000","last_fullsized":"2023-11-09T03:46:52.550636+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:30:38.798792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.555802+0000","last_change":"2023-11-09T03:46:52.546609+0000","last_active":"2023-11-09T03:46:52.555802+0000","last_peered":"2023-11-09T03:46:52.555802+0000","last_clean":"2023-11-09T03:46:52.555802+0000","last_became_active":"2023-11-09T03:46:52.546232+0000","last_became_peered":"2023-11-09T03:46:52.546232+0000","last_unstale":"2023-11-09T03:46:52.555802+0000","last_undegraded":"2023-11-09T03:46:52.555802+0000","last_fullsized":"2023-11-09T03:46:52.555802+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:56:45.361033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.533032+0000","last_change":"2023-11-09T03:46:51.545360+0000","last_active":"2023-11-09T03:46:52.533032+0000","last_peered":"2023-11-09T03:46:52.533032+0000","last_clean":"2023-11-09T03:46:52.533032+0000","last_became_active":"2023-11-09T03:46:51.545025+0000","last_became_peered":"2023-11-09T03:46:51.545025+0000","last_unstale":"2023-11-09T03:46:52.533032+0000","last_undegraded":"2023-11-09T03:46:52.533032+0000","last_fullsized":"2023-11-09T03:46:52.533032+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:23:11.571508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.552927+0000","last_change":"2023-11-09T03:46:52.545844+0000","last_active":"2023-11-09T03:46:52.552927+0000","last_peered":"2023-11-09T03:46:52.552927+0000","last_clean":"2023-11-09T03:46:52.552927+0000","last_became_active":"2023-11-09T03:46:52.545553+0000","last_became_peered":"2023-11-09T03:46:52.545553+0000","last_unstale":"2023-11-09T03:46:52.552927+0000","last_undegraded":"2023-11-09T03:46:52.552927+0000","last_fullsized":"2023-11-09T03:46:52.552927+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:47:43.093864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.531928+0000","last_change":"2023-11-09T03:46:51.545228+0000","last_active":"2023-11-09T03:46:52.531928+0000","last_peered":"2023-11-09T03:46:52.531928+0000","last_clean":"2023-11-09T03:46:52.531928+0000","last_became_active":"2023-11-09T03:46:51.544953+0000","last_became_peered":"2023-11-09T03:46:51.544953+0000","last_unstale":"2023-11-09T03:46:52.531928+0000","last_undegraded":"2023-11-09T03:46:52.531928+0000","last_fullsized":"2023-11-09T03:46:52.531928+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:17:17.362580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.555874+0000","last_change":"2023-11-09T03:46:52.546443+0000","last_active":"2023-11-09T03:46:52.555874+0000","last_peered":"2023-11-09T03:46:52.555874+0000","last_clean":"2023-11-09T03:46:52.555874+0000","last_became_active":"2023-11-09T03:46:52.546202+0000","last_became_peered":"2023-11-09T03:46:52.546202+0000","last_unstale":"2023-11-09T03:46:52.555874+0000","last_undegraded":"2023-11-09T03:46:52.555874+0000","last_fullsized":"2023-11-09T03:46:52.555874+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:49:44.445256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.535319+0000","last_change":"2023-11-09T03:46:51.539567+0000","last_active":"2023-11-09T03:46:52.535319+0000","last_peered":"2023-11-09T03:46:52.535319+0000","last_clean":"2023-11-09T03:46:52.535319+0000","last_became_active":"2023-11-09T03:46:51.539267+0000","last_became_peered":"2023-11-09T03:46:51.539267+0000","last_unstale":"2023-11-09T03:46:52.535319+0000","last_undegraded":"2023-11-09T03:46:52.535319+0000","last_fullsized":"2023-11-09T03:46:52.535319+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:37:53.997975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.551146+0000","last_change":"2023-11-09T03:46:52.548194+0000","last_active":"2023-11-09T03:46:52.551146+0000","last_peered":"2023-11-09T03:46:52.551146+0000","last_clean":"2023-11-09T03:46:52.551146+0000","last_became_active":"2023-11-09T03:46:52.547621+0000","last_became_peered":"2023-11-09T03:46:52.547621+0000","last_unstale":"2023-11-09T03:46:52.551146+0000","last_undegraded":"2023-11-09T03:46:52.551146+0000","last_fullsized":"2023-11-09T03:46:52.551146+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:19:23.645673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.534907+0000","last_change":"2023-11-09T03:46:51.544088+0000","last_active":"2023-11-09T03:46:52.534907+0000","last_peered":"2023-11-09T03:46:52.534907+0000","last_clean":"2023-11-09T03:46:52.534907+0000","last_became_active":"2023-11-09T03:46:51.543757+0000","last_became_peered":"2023-11-09T03:46:51.543757+0000","last_unstale":"2023-11-09T03:46:52.534907+0000","last_undegraded":"2023-11-09T03:46:52.534907+0000","last_fullsized":"2023-11-09T03:46:52.534907+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:18:48.180569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.544717+0000","last_change":"2023-11-09T03:46:52.539685+0000","last_active":"2023-11-09T03:46:52.544717+0000","last_peered":"2023-11-09T03:46:52.544717+0000","last_clean":"2023-11-09T03:46:52.544717+0000","last_became_active":"2023-11-09T03:46:52.539423+0000","last_became_peered":"2023-11-09T03:46:52.539423+0000","last_unstale":"2023-11-09T03:46:52.544717+0000","last_undegraded":"2023-11-09T03:46:52.544717+0000","last_fullsized":"2023-11-09T03:46:52.544717+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:33:19.817774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.559118+0000","last_change":"2023-11-09T03:46:51.544185+0000","last_active":"2023-11-09T03:46:52.559118+0000","last_peered":"2023-11-09T03:46:52.559118+0000","last_clean":"2023-11-09T03:46:52.559118+0000","last_became_active":"2023-11-09T03:46:51.543867+0000","last_became_peered":"2023-11-09T03:46:51.543867+0000","last_unstale":"2023-11-09T03:46:52.559118+0000","last_undegraded":"2023-11-09T03:46:52.559118+0000","last_fullsized":"2023-11-09T03:46:52.559118+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:54:00.678572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556060+0000","last_change":"2023-11-09T03:46:52.549350+0000","last_active":"2023-11-09T03:46:52.556060+0000","last_peered":"2023-11-09T03:46:52.556060+0000","last_clean":"2023-11-09T03:46:52.556060+0000","last_became_active":"2023-11-09T03:46:52.549121+0000","last_became_peered":"2023-11-09T03:46:52.549121+0000","last_unstale":"2023-11-09T03:46:52.556060+0000","last_undegraded":"2023-11-09T03:46:52.556060+0000","last_fullsized":"2023-11-09T03:46:52.556060+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:14:57.063826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.559200+0000","last_change":"2023-11-09T03:46:51.542543+0000","last_active":"2023-11-09T03:46:52.559200+0000","last_peered":"2023-11-09T03:46:52.559200+0000","last_clean":"2023-11-09T03:46:52.559200+0000","last_became_active":"2023-11-09T03:46:51.542258+0000","last_became_peered":"2023-11-09T03:46:51.542258+0000","last_unstale":"2023-11-09T03:46:52.559200+0000","last_undegraded":"2023-11-09T03:46:52.559200+0000","last_fullsized":"2023-11-09T03:46:52.559200+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:06:20.932951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556166+0000","last_change":"2023-11-09T03:46:52.549204+0000","last_active":"2023-11-09T03:46:52.556166+0000","last_peered":"2023-11-09T03:46:52.556166+0000","last_clean":"2023-11-09T03:46:52.556166+0000","last_became_active":"2023-11-09T03:46:52.548902+0000","last_became_peered":"2023-11-09T03:46:52.548902+0000","last_unstale":"2023-11-09T03:46:52.556166+0000","last_undegraded":"2023-11-09T03:46:52.556166+0000","last_fullsized":"2023-11-09T03:46:52.556166+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:45:32.257609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.549178+0000","last_change":"2023-11-09T03:46:51.543845+0000","last_active":"2023-11-09T03:46:52.549178+0000","last_peered":"2023-11-09T03:46:52.549178+0000","last_clean":"2023-11-09T03:46:52.549178+0000","last_became_active":"2023-11-09T03:46:51.543592+0000","last_became_peered":"2023-11-09T03:46:51.543592+0000","last_unstale":"2023-11-09T03:46:52.549178+0000","last_undegraded":"2023-11-09T03:46:52.549178+0000","last_fullsized":"2023-11-09T03:46:52.549178+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:56:45.569356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.552942+0000","last_change":"2023-11-09T03:46:52.545527+0000","last_active":"2023-11-09T03:46:52.552942+0000","last_peered":"2023-11-09T03:46:52.552942+0000","last_clean":"2023-11-09T03:46:52.552942+0000","last_became_active":"2023-11-09T03:46:52.545262+0000","last_became_peered":"2023-11-09T03:46:52.545262+0000","last_unstale":"2023-11-09T03:46:52.552942+0000","last_undegraded":"2023-11-09T03:46:52.552942+0000","last_fullsized":"2023-11-09T03:46:52.552942+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:41:30.870772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.534481+0000","last_change":"2023-11-09T03:46:51.547646+0000","last_active":"2023-11-09T03:46:52.534481+0000","last_peered":"2023-11-09T03:46:52.534481+0000","last_clean":"2023-11-09T03:46:52.534481+0000","last_became_active":"2023-11-09T03:46:51.547347+0000","last_became_peered":"2023-11-09T03:46:51.547347+0000","last_unstale":"2023-11-09T03:46:52.534481+0000","last_undegraded":"2023-11-09T03:46:52.534481+0000","last_fullsized":"2023-11-09T03:46:52.534481+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:50:37.650301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.551054+0000","last_change":"2023-11-09T03:46:52.547960+0000","last_active":"2023-11-09T03:46:52.551054+0000","last_peered":"2023-11-09T03:46:52.551054+0000","last_clean":"2023-11-09T03:46:52.551054+0000","last_became_active":"2023-11-09T03:46:52.547708+0000","last_became_peered":"2023-11-09T03:46:52.547708+0000","last_unstale":"2023-11-09T03:46:52.551054+0000","last_undegraded":"2023-11-09T03:46:52.551054+0000","last_fullsized":"2023-11-09T03:46:52.551054+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:50:38.652320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.529206+0000","last_change":"2023-11-09T03:46:51.545011+0000","last_active":"2023-11-09T03:46:52.529206+0000","last_peered":"2023-11-09T03:46:52.529206+0000","last_clean":"2023-11-09T03:46:52.529206+0000","last_became_active":"2023-11-09T03:46:51.544757+0000","last_became_peered":"2023-11-09T03:46:51.544757+0000","last_unstale":"2023-11-09T03:46:52.529206+0000","last_undegraded":"2023-11-09T03:46:52.529206+0000","last_fullsized":"2023-11-09T03:46:52.529206+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:23:11.571508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.555844+0000","last_change":"2023-11-09T03:46:52.549727+0000","last_active":"2023-11-09T03:46:52.555844+0000","last_peered":"2023-11-09T03:46:52.555844+0000","last_clean":"2023-11-09T03:46:52.555844+0000","last_became_active":"2023-11-09T03:46:52.549460+0000","last_became_peered":"2023-11-09T03:46:52.549460+0000","last_unstale":"2023-11-09T03:46:52.555844+0000","last_undegraded":"2023-11-09T03:46:52.555844+0000","last_fullsized":"2023-11-09T03:46:52.555844+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:54:01.680591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":19,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556823+0000","last_change":"2023-11-09T03:46:51.550027+0000","last_active":"2023-11-09T03:46:52.556823+0000","last_peered":"2023-11-09T03:46:52.556823+0000","last_clean":"2023-11-09T03:46:52.556823+0000","last_became_active":"2023-11-09T03:46:51.549729+0000","last_became_peered":"2023-11-09T03:46:51.549729+0000","last_unstale":"2023-11-09T03:46:52.556823+0000","last_undegraded":"2023-11-09T03:46:52.556823+0000","last_fullsized":"2023-11-09T03:46:52.556823+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:21:24.253031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.546260+0000","last_change":"2023-11-09T03:46:52.540718+0000","last_active":"2023-11-09T03:46:52.546260+0000","last_peered":"2023-11-09T03:46:52.546260+0000","last_clean":"2023-11-09T03:46:52.546260+0000","last_became_active":"2023-11-09T03:46:52.540445+0000","last_became_peered":"2023-11-09T03:46:52.540445+0000","last_unstale":"2023-11-09T03:46:52.546260+0000","last_undegraded":"2023-11-09T03:46:52.546260+0000","last_fullsized":"2023-11-09T03:46:52.546260+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:16:40.068588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.559242+0000","last_change":"2023-11-09T03:46:51.545514+0000","last_active":"2023-11-09T03:46:52.559242+0000","last_peered":"2023-11-09T03:46:52.559242+0000","last_clean":"2023-11-09T03:46:52.559242+0000","last_became_active":"2023-11-09T03:46:51.545188+0000","last_became_peered":"2023-11-09T03:46:51.545188+0000","last_unstale":"2023-11-09T03:46:52.559242+0000","last_undegraded":"2023-11-09T03:46:52.559242+0000","last_fullsized":"2023-11-09T03:46:52.559242+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:47:42.091845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556299+0000","last_change":"2023-11-09T03:46:52.546462+0000","last_active":"2023-11-09T03:46:52.556299+0000","last_peered":"2023-11-09T03:46:52.556299+0000","last_clean":"2023-11-09T03:46:52.556299+0000","last_became_active":"2023-11-09T03:46:52.546199+0000","last_became_peered":"2023-11-09T03:46:52.546199+0000","last_unstale":"2023-11-09T03:46:52.556299+0000","last_undegraded":"2023-11-09T03:46:52.556299+0000","last_fullsized":"2023-11-09T03:46:52.556299+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:19:23.645673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.559423+0000","last_change":"2023-11-09T03:46:51.544802+0000","last_active":"2023-11-09T03:46:52.559423+0000","last_peered":"2023-11-09T03:46:52.559423+0000","last_clean":"2023-11-09T03:46:52.559423+0000","last_became_active":"2023-11-09T03:46:51.544516+0000","last_became_peered":"2023-11-09T03:46:51.544516+0000","last_unstale":"2023-11-09T03:46:52.559423+0000","last_undegraded":"2023-11-09T03:46:52.559423+0000","last_fullsized":"2023-11-09T03:46:52.559423+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:18:33.568664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.551166+0000","last_change":"2023-11-09T03:46:52.545832+0000","last_active":"2023-11-09T03:46:52.551166+0000","last_peered":"2023-11-09T03:46:52.551166+0000","last_clean":"2023-11-09T03:46:52.551166+0000","last_became_active":"2023-11-09T03:46:52.545495+0000","last_became_peered":"2023-11-09T03:46:52.545495+0000","last_unstale":"2023-11-09T03:46:52.551166+0000","last_undegraded":"2023-11-09T03:46:52.551166+0000","last_fullsized":"2023-11-09T03:46:52.551166+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:23:12.573527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.555506+0000","last_change":"2023-11-09T03:46:51.543340+0000","last_active":"2023-11-09T03:46:52.555506+0000","last_peered":"2023-11-09T03:46:52.555506+0000","last_clean":"2023-11-09T03:46:52.555506+0000","last_became_active":"2023-11-09T03:46:51.543057+0000","last_became_peered":"2023-11-09T03:46:51.543057+0000","last_unstale":"2023-11-09T03:46:52.555506+0000","last_undegraded":"2023-11-09T03:46:52.555506+0000","last_fullsized":"2023-11-09T03:46:52.555506+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:47:13.365771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.553189+0000","last_change":"2023-11-09T03:46:52.541800+0000","last_active":"2023-11-09T03:46:52.553189+0000","last_peered":"2023-11-09T03:46:52.553189+0000","last_clean":"2023-11-09T03:46:52.553189+0000","last_became_active":"2023-11-09T03:46:52.541545+0000","last_became_peered":"2023-11-09T03:46:52.541545+0000","last_unstale":"2023-11-09T03:46:52.553189+0000","last_undegraded":"2023-11-09T03:46:52.553189+0000","last_fullsized":"2023-11-09T03:46:52.553189+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:05:30.357983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.535418+0000","last_change":"2023-11-09T03:46:51.544307+0000","last_active":"2023-11-09T03:46:52.535418+0000","last_peered":"2023-11-09T03:46:52.535418+0000","last_clean":"2023-11-09T03:46:52.535418+0000","last_became_active":"2023-11-09T03:46:51.543903+0000","last_became_peered":"2023-11-09T03:46:51.543903+0000","last_unstale":"2023-11-09T03:46:52.535418+0000","last_undegraded":"2023-11-09T03:46:52.535418+0000","last_fullsized":"2023-11-09T03:46:52.535418+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:21:24.253031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.551170+0000","last_change":"2023-11-09T03:46:52.548578+0000","last_active":"2023-11-09T03:46:52.551170+0000","last_peered":"2023-11-09T03:46:52.551170+0000","last_clean":"2023-11-09T03:46:52.551170+0000","last_became_active":"2023-11-09T03:46:52.548288+0000","last_became_peered":"2023-11-09T03:46:52.548288+0000","last_unstale":"2023-11-09T03:46:52.551170+0000","last_undegraded":"2023-11-09T03:46:52.551170+0000","last_fullsized":"2023-11-09T03:46:52.551170+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:04:23.215294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556491+0000","last_change":"2023-11-09T03:46:51.538030+0000","last_active":"2023-11-09T03:46:52.556491+0000","last_peered":"2023-11-09T03:46:52.556491+0000","last_clean":"2023-11-09T03:46:52.556491+0000","last_became_active":"2023-11-09T03:46:51.537773+0000","last_became_peered":"2023-11-09T03:46:51.537773+0000","last_unstale":"2023-11-09T03:46:52.556491+0000","last_undegraded":"2023-11-09T03:46:52.556491+0000","last_fullsized":"2023-11-09T03:46:52.556491+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:35:31.702881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.542596+0000","last_change":"2023-11-09T03:46:52.538634+0000","last_active":"2023-11-09T03:46:52.542596+0000","last_peered":"2023-11-09T03:46:52.542596+0000","last_clean":"2023-11-09T03:46:52.542596+0000","last_became_active":"2023-11-09T03:46:52.538289+0000","last_became_peered":"2023-11-09T03:46:52.538289+0000","last_unstale":"2023-11-09T03:46:52.542596+0000","last_undegraded":"2023-11-09T03:46:52.542596+0000","last_fullsized":"2023-11-09T03:46:52.542596+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:56:45.361033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.527789+0000","last_change":"2023-11-09T03:46:51.536546+0000","last_active":"2023-11-09T03:46:52.527789+0000","last_peered":"2023-11-09T03:46:52.527789+0000","last_clean":"2023-11-09T03:46:52.527789+0000","last_became_active":"2023-11-09T03:46:51.536234+0000","last_became_peered":"2023-11-09T03:46:51.536234+0000","last_unstale":"2023-11-09T03:46:52.527789+0000","last_undegraded":"2023-11-09T03:46:52.527789+0000","last_fullsized":"2023-11-09T03:46:52.527789+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:49:35.405663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.550449+0000","last_change":"2023-11-09T03:46:52.545767+0000","last_active":"2023-11-09T03:46:52.550449+0000","last_peered":"2023-11-09T03:46:52.550449+0000","last_clean":"2023-11-09T03:46:52.550449+0000","last_became_active":"2023-11-09T03:46:52.545496+0000","last_became_peered":"2023-11-09T03:46:52.545496+0000","last_unstale":"2023-11-09T03:46:52.550449+0000","last_undegraded":"2023-11-09T03:46:52.550449+0000","last_fullsized":"2023-11-09T03:46:52.550449+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:16:40.068588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.533384+0000","last_change":"2023-11-09T03:46:51.546183+0000","last_active":"2023-11-09T03:46:52.533384+0000","last_peered":"2023-11-09T03:46:52.533384+0000","last_clean":"2023-11-09T03:46:52.533384+0000","last_became_active":"2023-11-09T03:46:51.545928+0000","last_became_peered":"2023-11-09T03:46:51.545928+0000","last_unstale":"2023-11-09T03:46:52.533384+0000","last_undegraded":"2023-11-09T03:46:52.533384+0000","last_fullsized":"2023-11-09T03:46:52.533384+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:41.783929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.550325+0000","last_change":"2023-11-09T03:46:52.544861+0000","last_active":"2023-11-09T03:46:52.550325+0000","last_peered":"2023-11-09T03:46:52.550325+0000","last_clean":"2023-11-09T03:46:52.550325+0000","last_became_active":"2023-11-09T03:46:52.544615+0000","last_became_peered":"2023-11-09T03:46:52.544615+0000","last_unstale":"2023-11-09T03:46:52.550325+0000","last_undegraded":"2023-11-09T03:46:52.550325+0000","last_fullsized":"2023-11-09T03:46:52.550325+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:04:23.215294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.528218+0000","last_change":"2023-11-09T03:46:51.534485+0000","last_active":"2023-11-09T03:46:52.528218+0000","last_peered":"2023-11-09T03:46:52.528218+0000","last_clean":"2023-11-09T03:46:52.528218+0000","last_became_active":"2023-11-09T03:46:51.534177+0000","last_became_peered":"2023-11-09T03:46:51.534177+0000","last_unstale":"2023-11-09T03:46:52.528218+0000","last_undegraded":"2023-11-09T03:46:52.528218+0000","last_fullsized":"2023-11-09T03:46:52.528218+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:41:25.354316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.550639+0000","last_change":"2023-11-09T03:46:52.547161+0000","last_active":"2023-11-09T03:46:52.550639+0000","last_peered":"2023-11-09T03:46:52.550639+0000","last_clean":"2023-11-09T03:46:52.550639+0000","last_became_active":"2023-11-09T03:46:52.546513+0000","last_became_peered":"2023-11-09T03:46:52.546513+0000","last_unstale":"2023-11-09T03:46:52.550639+0000","last_undegraded":"2023-11-09T03:46:52.550639+0000","last_fullsized":"2023-11-09T03:46:52.550639+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:45:42.507078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556522+0000","last_change":"2023-11-09T03:46:51.544691+0000","last_active":"2023-11-09T03:46:52.556522+0000","last_peered":"2023-11-09T03:46:52.556522+0000","last_clean":"2023-11-09T03:46:52.556522+0000","last_became_active":"2023-11-09T03:46:51.544360+0000","last_became_peered":"2023-11-09T03:46:51.544360+0000","last_unstale":"2023-11-09T03:46:52.556522+0000","last_undegraded":"2023-11-09T03:46:52.556522+0000","last_fullsized":"2023-11-09T03:46:52.556522+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:56:45.569356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.551128+0000","last_change":"2023-11-09T03:46:52.548177+0000","last_active":"2023-11-09T03:46:52.551128+0000","last_peered":"2023-11-09T03:46:52.551128+0000","last_clean":"2023-11-09T03:46:52.551128+0000","last_became_active":"2023-11-09T03:46:52.546935+0000","last_became_peered":"2023-11-09T03:46:52.546935+0000","last_unstale":"2023-11-09T03:46:52.551128+0000","last_undegraded":"2023-11-09T03:46:52.551128+0000","last_fullsized":"2023-11-09T03:46:52.551128+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:47:43.093864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.528156+0000","last_change":"2023-11-09T03:46:51.545122+0000","last_active":"2023-11-09T03:46:52.528156+0000","last_peered":"2023-11-09T03:46:52.528156+0000","last_clean":"2023-11-09T03:46:52.528156+0000","last_became_active":"2023-11-09T03:46:51.544846+0000","last_became_peered":"2023-11-09T03:46:51.544846+0000","last_unstale":"2023-11-09T03:46:52.528156+0000","last_undegraded":"2023-11-09T03:46:52.528156+0000","last_fullsized":"2023-11-09T03:46:52.528156+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:56:22.557010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.558290+0000","last_change":"2023-11-09T03:46:52.553307+0000","last_active":"2023-11-09T03:46:52.558290+0000","last_peered":"2023-11-09T03:46:52.558290+0000","last_clean":"2023-11-09T03:46:52.558290+0000","last_became_active":"2023-11-09T03:46:52.553028+0000","last_became_peered":"2023-11-09T03:46:52.553028+0000","last_unstale":"2023-11-09T03:46:52.558290+0000","last_undegraded":"2023-11-09T03:46:52.558290+0000","last_fullsized":"2023-11-09T03:46:52.558290+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:57:09.422287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.532800+0000","last_change":"2023-11-09T03:46:51.540889+0000","last_active":"2023-11-09T03:46:52.532800+0000","last_peered":"2023-11-09T03:46:52.532800+0000","last_clean":"2023-11-09T03:46:52.532800+0000","last_became_active":"2023-11-09T03:46:51.540634+0000","last_became_peered":"2023-11-09T03:46:51.540634+0000","last_unstale":"2023-11-09T03:46:52.532800+0000","last_undegraded":"2023-11-09T03:46:52.532800+0000","last_fullsized":"2023-11-09T03:46:52.532800+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:45:41.505059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.553124+0000","last_change":"2023-11-09T03:46:52.545181+0000","last_active":"2023-11-09T03:46:52.553124+0000","last_peered":"2023-11-09T03:46:52.553124+0000","last_clean":"2023-11-09T03:46:52.553124+0000","last_became_active":"2023-11-09T03:46:52.544385+0000","last_became_peered":"2023-11-09T03:46:52.544385+0000","last_unstale":"2023-11-09T03:46:52.553124+0000","last_undegraded":"2023-11-09T03:46:52.553124+0000","last_fullsized":"2023-11-09T03:46:52.553124+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:18:34.570683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.533587+0000","last_change":"2023-11-09T03:46:51.545342+0000","last_active":"2023-11-09T03:46:52.533587+0000","last_peered":"2023-11-09T03:46:52.533587+0000","last_clean":"2023-11-09T03:46:52.533587+0000","last_became_active":"2023-11-09T03:46:51.544936+0000","last_became_peered":"2023-11-09T03:46:51.544936+0000","last_unstale":"2023-11-09T03:46:52.533587+0000","last_undegraded":"2023-11-09T03:46:52.533587+0000","last_fullsized":"2023-11-09T03:46:52.533587+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:21:24.253031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.552984+0000","last_change":"2023-11-09T03:46:52.549117+0000","last_active":"2023-11-09T03:46:52.552984+0000","last_peered":"2023-11-09T03:46:52.552984+0000","last_clean":"2023-11-09T03:46:52.552984+0000","last_became_active":"2023-11-09T03:46:52.548879+0000","last_became_peered":"2023-11-09T03:46:52.548879+0000","last_unstale":"2023-11-09T03:46:52.552984+0000","last_undegraded":"2023-11-09T03:46:52.552984+0000","last_fullsized":"2023-11-09T03:46:52.552984+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:56:45.361033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.529283+0000","last_change":"2023-11-09T03:46:51.544622+0000","last_active":"2023-11-09T03:46:52.529283+0000","last_peered":"2023-11-09T03:46:52.529283+0000","last_clean":"2023-11-09T03:46:52.529283+0000","last_became_active":"2023-11-09T03:46:51.544307+0000","last_became_peered":"2023-11-09T03:46:51.544307+0000","last_unstale":"2023-11-09T03:46:52.529283+0000","last_undegraded":"2023-11-09T03:46:52.529283+0000","last_fullsized":"2023-11-09T03:46:52.529283+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:18:33.568664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.558765+0000","last_change":"2023-11-09T03:46:52.551774+0000","last_active":"2023-11-09T03:46:52.558765+0000","last_peered":"2023-11-09T03:46:52.558765+0000","last_clean":"2023-11-09T03:46:52.558765+0000","last_became_active":"2023-11-09T03:46:52.551524+0000","last_became_peered":"2023-11-09T03:46:52.551524+0000","last_unstale":"2023-11-09T03:46:52.558765+0000","last_undegraded":"2023-11-09T03:46:52.558765+0000","last_fullsized":"2023-11-09T03:46:52.558765+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:45:42.507078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.536303+0000","last_change":"2023-11-09T03:46:51.542254+0000","last_active":"2023-11-09T03:46:52.536303+0000","last_peered":"2023-11-09T03:46:52.536303+0000","last_clean":"2023-11-09T03:46:52.536303+0000","last_became_active":"2023-11-09T03:46:51.541994+0000","last_became_peered":"2023-11-09T03:46:51.541994+0000","last_unstale":"2023-11-09T03:46:52.536303+0000","last_undegraded":"2023-11-09T03:46:52.536303+0000","last_fullsized":"2023-11-09T03:46:52.536303+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:30:38.798792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.551674+0000","last_change":"2023-11-09T03:46:52.547974+0000","last_active":"2023-11-09T03:46:52.551674+0000","last_peered":"2023-11-09T03:46:52.551674+0000","last_clean":"2023-11-09T03:46:52.551674+0000","last_became_active":"2023-11-09T03:46:52.547420+0000","last_became_peered":"2023-11-09T03:46:52.547420+0000","last_unstale":"2023-11-09T03:46:52.551674+0000","last_undegraded":"2023-11-09T03:46:52.551674+0000","last_fullsized":"2023-11-09T03:46:52.551674+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:56:46.571375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.531990+0000","last_change":"2023-11-09T03:46:51.540571+0000","last_active":"2023-11-09T03:46:52.531990+0000","last_peered":"2023-11-09T03:46:52.531990+0000","last_clean":"2023-11-09T03:46:52.531990+0000","last_became_active":"2023-11-09T03:46:51.540228+0000","last_became_peered":"2023-11-09T03:46:51.540228+0000","last_unstale":"2023-11-09T03:46:52.531990+0000","last_undegraded":"2023-11-09T03:46:52.531990+0000","last_fullsized":"2023-11-09T03:46:52.531990+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:41.783929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.548749+0000","last_change":"2023-11-09T03:46:52.542914+0000","last_active":"2023-11-09T03:46:52.548749+0000","last_peered":"2023-11-09T03:46:52.548749+0000","last_clean":"2023-11-09T03:46:52.548749+0000","last_became_active":"2023-11-09T03:46:52.542615+0000","last_became_peered":"2023-11-09T03:46:52.542615+0000","last_unstale":"2023-11-09T03:46:52.548749+0000","last_undegraded":"2023-11-09T03:46:52.548749+0000","last_fullsized":"2023-11-09T03:46:52.548749+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:04:23.215294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.533327+0000","last_change":"2023-11-09T03:46:51.545962+0000","last_active":"2023-11-09T03:46:52.533327+0000","last_peered":"2023-11-09T03:46:52.533327+0000","last_clean":"2023-11-09T03:46:52.533327+0000","last_became_active":"2023-11-09T03:46:51.545711+0000","last_became_peered":"2023-11-09T03:46:51.545711+0000","last_unstale":"2023-11-09T03:46:52.533327+0000","last_undegraded":"2023-11-09T03:46:52.533327+0000","last_fullsized":"2023-11-09T03:46:52.533327+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:29:02.367631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":179,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-09T03:46:51.540568+0000","last_change":"2023-11-09T03:45:21.030926+0000","last_active":"2023-11-09T03:46:51.540568+0000","last_peered":"2023-11-09T03:46:51.540568+0000","last_clean":"2023-11-09T03:46:51.540568+0000","last_became_active":"2023-11-09T03:45:21.029801+0000","last_became_peered":"2023-11-09T03:45:21.029801+0000","last_unstale":"2023-11-09T03:46:51.540568+0000","last_undegraded":"2023-11-09T03:46:51.540568+0000","last_fullsized":"2023-11-09T03:46:51.540568+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_clean_scrub_stamp":"2023-11-09T03:45:19.768433+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:12:59.153713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":769970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.60199999999999998}]},{"osd":1,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":4.9139999999999997,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.433}]},{"osd":3,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.503},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.52200000000000002}]},{"osd":4,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.56499999999999995}]},{"osd":5,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.54800000000000004}]},{"osd":6,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.61899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":187279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.73199999999999998}]},{"osd":1,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.753},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.60199999999999998}]},{"osd":2,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.77200000000000002}]},{"osd":3,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.78800000000000003}]},{"osd":4,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.68100000000000005}]},{"osd":7,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.65700000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":179602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.498}]},{"osd":1,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.60199999999999998}]},{"osd":2,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.51000000000000001}]},{"osd":3,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.70699999999999996}]},{"osd":4,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34000000000000002}]},{"osd":6,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.41999999999999998}]},{"osd":7,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.435}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":179602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.57499999999999996}]},{"osd":1,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.56399999999999995}]},{"osd":2,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.59399999999999997}]},{"osd":3,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.55100000000000005}]},{"osd":5,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.38100000000000001}]},{"osd":6,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.50700000000000001}]},{"osd":7,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.48599999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":179602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.224},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.36099999999999999}]},{"osd":1,"last update":"Thu Nov 9 03:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.34599999999999997}]},{"osd":2,"last update":"Thu Nov 9 03:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.29899999999999999}]},{"osd":4,"last update":"Thu Nov 9 03:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.48299999999999998}]},{"osd":5,"last update":"Thu Nov 9 03:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.504},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.42599999999999999}]},{"osd":6,"last update":"Thu Nov 9 03:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.44900000000000001}]},{"osd":7,"last update":"Thu Nov 9 03:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.55900000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":179602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.32300000000000001}]},{"osd":1,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.29199999999999998}]},{"osd":3,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.33600000000000002}]},{"osd":4,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.749}]},{"osd":5,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.58999999999999997}]},{"osd":7,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.73699999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":179602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.316,"15min":0.316},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"last":0.20799999999999999},{"interface":"front","average":{"1min":0.31,"5min":0.31,"15min":0.31},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.432,"5min":0.432,"15min":0.432},"last":0.26800000000000002}]},{"osd":2,"last update":"Thu Nov 9 03:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.308,"15min":0.308},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"last":0.252}]},{"osd":3,"last update":"Thu Nov 9 03:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.34300000000000003}]},{"osd":4,"last update":"Thu Nov 9 03:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Nov 9 03:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47599999999999998}]},{"osd":6,"last update":"Thu Nov 9 03:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.433}]},{"osd":7,"last update":"Thu Nov 9 03:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.503}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1015808,"data_stored":769351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.219}]},{"osd":2,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.254}]},{"osd":3,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.32000000000000001}]},{"osd":4,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.48499999999999999}]},{"osd":5,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.441},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.55400000000000005}]},{"osd":6,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.53300000000000003}]},{"osd":7,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.46500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-09T03:46:55.489 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-09T03:46:55.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.623+0000 7f4b0d1a9640 1 Processor -- start 2023-11-09T03:46:55.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.623+0000 7f4b0d1a9640 1 -- start start 2023-11-09T03:46:55.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.624+0000 7f4b0d1a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b080ce240 0x7f4b08157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:55.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.624+0000 7f4b0d1a9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4b08157540 0x7f4b080d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:55.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.624+0000 7f4b0d1a9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b0815e000 con 0x7f4b080cde60 2023-11-09T03:46:55.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.624+0000 7f4b0d1a9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b0815e5c0 con 0x7f4b08157540 2023-11-09T03:46:55.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.624+0000 7f4b0d1a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0815e2e0 con 0x7f4b080ce240 2023-11-09T03:46:55.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.624+0000 7f4b0759e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4b08157540 0x7f4b080d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:55.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.625+0000 7f4b0759e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4b08157540 0x7f4b080d1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41664/0 (socket says 172.21.15.76:41664) 2023-11-09T03:46:55.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.625+0000 7f4b0759e640 1 -- 172.21.15.76:0/1676595821 learned_addr learned my addr 172.21.15.76:0/1676595821 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:55.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.625+0000 7f4b0659c640 1 --2- 172.21.15.76:0/1676595821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b080ce240 0x7f4b08157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:55.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.625+0000 7f4b0759e640 1 -- 172.21.15.76:0/1676595821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b080ce240 msgr2=0x7f4b08157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:55.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.625+0000 7f4b0759e640 1 --2- 172.21.15.76:0/1676595821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b080ce240 0x7f4b08157000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.625+0000 7f4b0759e640 1 -- 172.21.15.76:0/1676595821 >> v1:172.21.15.113:6789/0 conn(0x7f4b080cde60 legacy=0x7f4b0815d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:55.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.625+0000 7f4b0759e640 1 -- 172.21.15.76:0/1676595821 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b080d1120 con 0x7f4b08157540 2023-11-09T03:46:55.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.626+0000 7f4b0759e640 1 --2- 172.21.15.76:0/1676595821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b08157540 0x7f4b080d1920 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009870 tx=0x7f4b0002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a22deb00612dc1 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:55.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.626+0000 7f4b05d9b640 1 -- 172.21.15.76:0/1676595821 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b00031640 con 0x7f4b08157540 2023-11-09T03:46:55.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.626+0000 7f4b05d9b640 1 -- 172.21.15.76:0/1676595821 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b000317a0 con 0x7f4b08157540 2023-11-09T03:46:55.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.626+0000 7f4b05d9b640 1 -- 172.21.15.76:0/1676595821 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b00031b30 con 0x7f4b08157540 2023-11-09T03:46:55.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.626+0000 7f4b0d1a9640 1 -- 172.21.15.76:0/1676595821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b08157540 msgr2=0x7f4b080d1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:55.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.626+0000 7f4b0d1a9640 1 --2- 172.21.15.76:0/1676595821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b08157540 0x7f4b080d1920 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009870 tx=0x7f4b0002b8f0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.626+0000 7f4b0d1a9640 1 -- 172.21.15.76:0/1676595821 shutdown_connections 2023-11-09T03:46:55.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.626+0000 7f4b0d1a9640 1 --2- 172.21.15.76:0/1676595821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b08157540 0x7f4b080d1920 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.626+0000 7f4b0d1a9640 1 --2- 172.21.15.76:0/1676595821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b080ce240 0x7f4b08157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.626+0000 7f4b0d1a9640 1 -- 172.21.15.76:0/1676595821 >> 172.21.15.76:0/1676595821 conn(0x7f4b08071f30 msgr2=0x7f4b080cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:55.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.626+0000 7f4b0d1a9640 1 -- 172.21.15.76:0/1676595821 shutdown_connections 2023-11-09T03:46:55.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.626+0000 7f4b0d1a9640 1 -- 172.21.15.76:0/1676595821 wait complete. 2023-11-09T03:46:55.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.628+0000 7f4b0d1a9640 1 Processor -- start 2023-11-09T03:46:55.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.628+0000 7f4b0d1a9640 1 -- start start 2023-11-09T03:46:55.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.628+0000 7f4b0d1a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b080cde60 0x7f4b08153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:55.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.628+0000 7f4b0d1a9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b080ce240 0x7f4b081f3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:55.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.628+0000 7f4b0d1a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b08157540 0x7f4b081591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:55.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.628+0000 7f4b0d1a9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b081f38b0 con 0x7f4b080ce240 2023-11-09T03:46:55.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.629+0000 7f4b0d1a9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b08158010 con 0x7f4b080cde60 2023-11-09T03:46:55.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.629+0000 7f4b0d1a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b081582f0 con 0x7f4b08157540 2023-11-09T03:46:55.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.629+0000 7f4b0659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b080ce240 0x7f4b081f3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:55.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.629+0000 7f4b0659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b080ce240 0x7f4b081f3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41666/0 (socket says 172.21.15.76:41666) 2023-11-09T03:46:55.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.629+0000 7f4b0659c640 1 -- 172.21.15.76:0/1539967741 learned_addr learned my addr 172.21.15.76:0/1539967741 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:55.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.629+0000 7f4b0759e640 1 --2- 172.21.15.76:0/1539967741 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b08157540 0x7f4b081591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:55.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.629+0000 7f4b06d9d640 1 --2- 172.21.15.76:0/1539967741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b080cde60 0x7f4b08153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:55.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.629+0000 7f4b0659c640 1 -- 172.21.15.76:0/1539967741 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b08157540 msgr2=0x7f4b081591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:55.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.629+0000 7f4b0659c640 1 --2- 172.21.15.76:0/1539967741 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b08157540 0x7f4b081591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.629+0000 7f4b0659c640 1 -- 172.21.15.76:0/1539967741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b080cde60 msgr2=0x7f4b08153560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:55.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.629+0000 7f4b0659c640 1 --2- 172.21.15.76:0/1539967741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b080cde60 0x7f4b08153560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.629+0000 7f4b0659c640 1 -- 172.21.15.76:0/1539967741 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b000094d0 con 0x7f4b080ce240 2023-11-09T03:46:55.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.629+0000 7f4b0659c640 1 --2- 172.21.15.76:0/1539967741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b080ce240 0x7f4b081f3370 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4afc00c7f0 tx=0x7f4afc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:55.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.629+0000 7f4aef7fe640 1 -- 172.21.15.76:0/1539967741 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc004220 con 0x7f4b080ce240 2023-11-09T03:46:55.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.629+0000 7f4aef7fe640 1 -- 172.21.15.76:0/1539967741 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4afc0043a0 con 0x7f4b080ce240 2023-11-09T03:46:55.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.629+0000 7f4b0d1a9640 1 -- 172.21.15.76:0/1539967741 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b081585d0 con 0x7f4b080ce240 2023-11-09T03:46:55.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.629+0000 7f4aef7fe640 1 -- 172.21.15.76:0/1539967741 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc007860 con 0x7f4b080ce240 2023-11-09T03:46:55.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.629+0000 7f4b0d1a9640 1 -- 172.21.15.76:0/1539967741 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b08158b90 con 0x7f4b080ce240 2023-11-09T03:46:55.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.630+0000 7f4b0d1a9640 1 -- 172.21.15.76:0/1539967741 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b08066a40 con 0x7f4b080ce240 2023-11-09T03:46:55.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.631+0000 7f4aef7fe640 1 -- 172.21.15.76:0/1539967741 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4afc007bd0 con 0x7f4b080ce240 2023-11-09T03:46:55.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.631+0000 7f4aef7fe640 1 --2- 172.21.15.76:0/1539967741 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4af406ae70 0x7f4af406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:55.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.631+0000 7f4aef7fe640 1 -- 172.21.15.76:0/1539967741 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4afc089590 con 0x7f4b080ce240 2023-11-09T03:46:55.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.631+0000 7f4b06d9d640 1 --2- 172.21.15.76:0/1539967741 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4af406ae70 0x7f4af406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:55.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.632+0000 7f4b06d9d640 1 --2- 172.21.15.76:0/1539967741 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4af406ae70 0x7f4af406d330 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4b080d1000 tx=0x7f4af8003d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:55.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.635+0000 7f4aef7fe640 1 -- 172.21.15.76:0/1539967741 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4afc054640 con 0x7f4b080ce240 2023-11-09T03:46:55.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.799+0000 7f4b0d1a9640 1 -- 172.21.15.76:0/1539967741 --> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4b080b89e0 con 0x7f4af406ae70 2023-11-09T03:46:55.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.806+0000 7f4aef7fe640 1 -- 172.21.15.76:0/1539967741 <== mgr.4105 v2:172.21.15.113:6832/1264193042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288910 (secure 0 0 0) 0x7f4b080b89e0 con 0x7f4af406ae70 2023-11-09T03:46:55.808 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:55.809 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2023-11-09T03:46:55.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.809+0000 7f4b0d1a9640 1 -- 172.21.15.76:0/1539967741 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4af406ae70 msgr2=0x7f4af406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:55.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.809+0000 7f4b0d1a9640 1 --2- 172.21.15.76:0/1539967741 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4af406ae70 0x7f4af406d330 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4b080d1000 tx=0x7f4af8003d90 comp rx=0 tx=0).stop 2023-11-09T03:46:55.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.809+0000 7f4b0d1a9640 1 -- 172.21.15.76:0/1539967741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b080ce240 msgr2=0x7f4b081f3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:55.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.809+0000 7f4b0d1a9640 1 --2- 172.21.15.76:0/1539967741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b080ce240 0x7f4b081f3370 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4afc00c7f0 tx=0x7f4afc00ccb0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.809+0000 7f4b0d1a9640 1 -- 172.21.15.76:0/1539967741 shutdown_connections 2023-11-09T03:46:55.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.809+0000 7f4b0d1a9640 1 --2- 172.21.15.76:0/1539967741 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4af406ae70 0x7f4af406d330 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.809+0000 7f4b0d1a9640 1 --2- 172.21.15.76:0/1539967741 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b08157540 0x7f4b081591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.809+0000 7f4b0d1a9640 1 --2- 172.21.15.76:0/1539967741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b080ce240 0x7f4b081f3370 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.809+0000 7f4b0d1a9640 1 --2- 172.21.15.76:0/1539967741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b080cde60 0x7f4b08153560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.809+0000 7f4b0d1a9640 1 -- 172.21.15.76:0/1539967741 >> 172.21.15.76:0/1539967741 conn(0x7f4b08071f30 msgr2=0x7f4b080b6630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:55.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.809+0000 7f4b0d1a9640 1 -- 172.21.15.76:0/1539967741 shutdown_connections 2023-11-09T03:46:55.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.809+0000 7f4b0d1a9640 1 -- 172.21.15.76:0/1539967741 wait complete. 2023-11-09T03:46:55.819 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2023-11-09T03:46:54.770073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220688,"kb_used_data":4500,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754032,"statfs":{"total":773094113280,"available":772868128768,"internally_reserved":0,"allocated":4608000,"data_stored":2624610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.252635"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.555771+0000","last_change":"2023-11-09T03:46:52.549534+0000","last_active":"2023-11-09T03:46:52.555771+0000","last_peered":"2023-11-09T03:46:52.555771+0000","last_clean":"2023-11-09T03:46:52.555771+0000","last_became_active":"2023-11-09T03:46:52.549269+0000","last_became_peered":"2023-11-09T03:46:52.549269+0000","last_unstale":"2023-11-09T03:46:52.555771+0000","last_undegraded":"2023-11-09T03:46:52.555771+0000","last_fullsized":"2023-11-09T03:46:52.555771+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:44.261724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.549872+0000","last_change":"2023-11-09T03:46:52.544197+0000","last_active":"2023-11-09T03:46:52.549872+0000","last_peered":"2023-11-09T03:46:52.549872+0000","last_clean":"2023-11-09T03:46:52.549872+0000","last_became_active":"2023-11-09T03:46:52.543955+0000","last_became_peered":"2023-11-09T03:46:52.543955+0000","last_unstale":"2023-11-09T03:46:52.549872+0000","last_undegraded":"2023-11-09T03:46:52.549872+0000","last_fullsized":"2023-11-09T03:46:52.549872+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:18:49.182588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556110+0000","last_change":"2023-11-09T03:46:52.548802+0000","last_active":"2023-11-09T03:46:52.556110+0000","last_peered":"2023-11-09T03:46:52.556110+0000","last_clean":"2023-11-09T03:46:52.556110+0000","last_became_active":"2023-11-09T03:46:52.548527+0000","last_became_peered":"2023-11-09T03:46:52.548527+0000","last_unstale":"2023-11-09T03:46:52.556110+0000","last_undegraded":"2023-11-09T03:46:52.556110+0000","last_fullsized":"2023-11-09T03:46:52.556110+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:30:39.800810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.555891+0000","last_change":"2023-11-09T03:46:52.547613+0000","last_active":"2023-11-09T03:46:52.555891+0000","last_peered":"2023-11-09T03:46:52.555891+0000","last_clean":"2023-11-09T03:46:52.555891+0000","last_became_active":"2023-11-09T03:46:52.547371+0000","last_became_peered":"2023-11-09T03:46:52.547371+0000","last_unstale":"2023-11-09T03:46:52.555891+0000","last_undegraded":"2023-11-09T03:46:52.555891+0000","last_fullsized":"2023-11-09T03:46:52.555891+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:05:57.334303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.555825+0000","last_change":"2023-11-09T03:46:52.547070+0000","last_active":"2023-11-09T03:46:52.555825+0000","last_peered":"2023-11-09T03:46:52.555825+0000","last_clean":"2023-11-09T03:46:52.555825+0000","last_became_active":"2023-11-09T03:46:52.546795+0000","last_became_peered":"2023-11-09T03:46:52.546795+0000","last_unstale":"2023-11-09T03:46:52.555825+0000","last_undegraded":"2023-11-09T03:46:52.555825+0000","last_fullsized":"2023-11-09T03:46:52.555825+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:14:56.145315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.550611+0000","last_change":"2023-11-09T03:46:52.547806+0000","last_active":"2023-11-09T03:46:52.550611+0000","last_peered":"2023-11-09T03:46:52.550611+0000","last_clean":"2023-11-09T03:46:52.550611+0000","last_became_active":"2023-11-09T03:46:52.547536+0000","last_became_peered":"2023-11-09T03:46:52.547536+0000","last_unstale":"2023-11-09T03:46:52.550611+0000","last_undegraded":"2023-11-09T03:46:52.550611+0000","last_fullsized":"2023-11-09T03:46:52.550611+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:33:19.817774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.542641+0000","last_change":"2023-11-09T03:46:52.541151+0000","last_active":"2023-11-09T03:46:52.542641+0000","last_peered":"2023-11-09T03:46:52.542641+0000","last_clean":"2023-11-09T03:46:52.542641+0000","last_became_active":"2023-11-09T03:46:52.540868+0000","last_became_peered":"2023-11-09T03:46:52.540868+0000","last_unstale":"2023-11-09T03:46:52.542641+0000","last_undegraded":"2023-11-09T03:46:52.542641+0000","last_fullsized":"2023-11-09T03:46:52.542641+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:24:42.894740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556038+0000","last_change":"2023-11-09T03:46:52.550418+0000","last_active":"2023-11-09T03:46:52.556038+0000","last_peered":"2023-11-09T03:46:52.556038+0000","last_clean":"2023-11-09T03:46:52.556038+0000","last_became_active":"2023-11-09T03:46:52.550180+0000","last_became_peered":"2023-11-09T03:46:52.550180+0000","last_unstale":"2023-11-09T03:46:52.556038+0000","last_undegraded":"2023-11-09T03:46:52.556038+0000","last_fullsized":"2023-11-09T03:46:52.556038+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:29:03.369650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556078+0000","last_change":"2023-11-09T03:46:52.548094+0000","last_active":"2023-11-09T03:46:52.556078+0000","last_peered":"2023-11-09T03:46:52.556078+0000","last_clean":"2023-11-09T03:46:52.556078+0000","last_became_active":"2023-11-09T03:46:52.547849+0000","last_became_peered":"2023-11-09T03:46:52.547849+0000","last_unstale":"2023-11-09T03:46:52.556078+0000","last_undegraded":"2023-11-09T03:46:52.556078+0000","last_fullsized":"2023-11-09T03:46:52.556078+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:14:57.063826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556405+0000","last_change":"2023-11-09T03:46:52.549352+0000","last_active":"2023-11-09T03:46:52.556405+0000","last_peered":"2023-11-09T03:46:52.556405+0000","last_clean":"2023-11-09T03:46:52.556405+0000","last_became_active":"2023-11-09T03:46:52.549113+0000","last_became_peered":"2023-11-09T03:46:52.549113+0000","last_unstale":"2023-11-09T03:46:52.556405+0000","last_undegraded":"2023-11-09T03:46:52.556405+0000","last_fullsized":"2023-11-09T03:46:52.556405+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:57:09.422287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.559055+0000","last_change":"2023-11-09T03:46:52.553667+0000","last_active":"2023-11-09T03:46:52.559055+0000","last_peered":"2023-11-09T03:46:52.559055+0000","last_clean":"2023-11-09T03:46:52.559055+0000","last_became_active":"2023-11-09T03:46:52.551543+0000","last_became_peered":"2023-11-09T03:46:52.551543+0000","last_unstale":"2023-11-09T03:46:52.559055+0000","last_undegraded":"2023-11-09T03:46:52.559055+0000","last_fullsized":"2023-11-09T03:46:52.559055+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:29:03.369650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556133+0000","last_change":"2023-11-09T03:46:52.548952+0000","last_active":"2023-11-09T03:46:52.556133+0000","last_peered":"2023-11-09T03:46:52.556133+0000","last_clean":"2023-11-09T03:46:52.556133+0000","last_became_active":"2023-11-09T03:46:52.548676+0000","last_became_peered":"2023-11-09T03:46:52.548676+0000","last_unstale":"2023-11-09T03:46:52.556133+0000","last_undegraded":"2023-11-09T03:46:52.556133+0000","last_fullsized":"2023-11-09T03:46:52.556133+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:17:18.364598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556287+0000","last_change":"2023-11-09T03:46:52.548868+0000","last_active":"2023-11-09T03:46:52.556287+0000","last_peered":"2023-11-09T03:46:52.556287+0000","last_clean":"2023-11-09T03:46:52.556287+0000","last_became_active":"2023-11-09T03:46:52.548619+0000","last_became_peered":"2023-11-09T03:46:52.548619+0000","last_unstale":"2023-11-09T03:46:52.556287+0000","last_undegraded":"2023-11-09T03:46:52.556287+0000","last_fullsized":"2023-11-09T03:46:52.556287+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:41:30.870772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.559003+0000","last_change":"2023-11-09T03:46:52.554119+0000","last_active":"2023-11-09T03:46:52.559003+0000","last_peered":"2023-11-09T03:46:52.559003+0000","last_clean":"2023-11-09T03:46:52.559003+0000","last_became_active":"2023-11-09T03:46:52.553810+0000","last_became_peered":"2023-11-09T03:46:52.553810+0000","last_unstale":"2023-11-09T03:46:52.559003+0000","last_undegraded":"2023-11-09T03:46:52.559003+0000","last_fullsized":"2023-11-09T03:46:52.559003+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:33:19.817774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556479+0000","last_change":"2023-11-09T03:46:52.549554+0000","last_active":"2023-11-09T03:46:52.556479+0000","last_peered":"2023-11-09T03:46:52.556479+0000","last_clean":"2023-11-09T03:46:52.556479+0000","last_became_active":"2023-11-09T03:46:52.549296+0000","last_became_peered":"2023-11-09T03:46:52.549296+0000","last_unstale":"2023-11-09T03:46:52.556479+0000","last_undegraded":"2023-11-09T03:46:52.556479+0000","last_fullsized":"2023-11-09T03:46:52.556479+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:15:53.977101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.549050+0000","last_change":"2023-11-09T03:46:52.544776+0000","last_active":"2023-11-09T03:46:52.549050+0000","last_peered":"2023-11-09T03:46:52.549050+0000","last_clean":"2023-11-09T03:46:52.549050+0000","last_became_active":"2023-11-09T03:46:52.544530+0000","last_became_peered":"2023-11-09T03:46:52.544530+0000","last_unstale":"2023-11-09T03:46:52.549050+0000","last_undegraded":"2023-11-09T03:46:52.549050+0000","last_fullsized":"2023-11-09T03:46:52.549050+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:33:19.817774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.554750+0000","last_change":"2023-11-09T03:46:52.547461+0000","last_active":"2023-11-09T03:46:52.554750+0000","last_peered":"2023-11-09T03:46:52.554750+0000","last_clean":"2023-11-09T03:46:52.554750+0000","last_became_active":"2023-11-09T03:46:52.547189+0000","last_became_peered":"2023-11-09T03:46:52.547189+0000","last_unstale":"2023-11-09T03:46:52.554750+0000","last_undegraded":"2023-11-09T03:46:52.554750+0000","last_fullsized":"2023-11-09T03:46:52.554750+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:54:01.680591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556422+0000","last_change":"2023-11-09T03:46:52.550701+0000","last_active":"2023-11-09T03:46:52.556422+0000","last_peered":"2023-11-09T03:46:52.556422+0000","last_clean":"2023-11-09T03:46:52.556422+0000","last_became_active":"2023-11-09T03:46:52.550425+0000","last_became_peered":"2023-11-09T03:46:52.550425+0000","last_unstale":"2023-11-09T03:46:52.556422+0000","last_undegraded":"2023-11-09T03:46:52.556422+0000","last_fullsized":"2023-11-09T03:46:52.556422+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:45:32.257609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.553224+0000","last_change":"2023-11-09T03:46:52.542025+0000","last_active":"2023-11-09T03:46:52.553224+0000","last_peered":"2023-11-09T03:46:52.553224+0000","last_clean":"2023-11-09T03:46:52.553224+0000","last_became_active":"2023-11-09T03:46:52.541728+0000","last_became_peered":"2023-11-09T03:46:52.541728+0000","last_unstale":"2023-11-09T03:46:52.553224+0000","last_undegraded":"2023-11-09T03:46:52.553224+0000","last_fullsized":"2023-11-09T03:46:52.553224+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:49:00.630877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556363+0000","last_change":"2023-11-09T03:46:52.551316+0000","last_active":"2023-11-09T03:46:52.556363+0000","last_peered":"2023-11-09T03:46:52.556363+0000","last_clean":"2023-11-09T03:46:52.556363+0000","last_became_active":"2023-11-09T03:46:52.551061+0000","last_became_peered":"2023-11-09T03:46:52.551061+0000","last_unstale":"2023-11-09T03:46:52.556363+0000","last_undegraded":"2023-11-09T03:46:52.556363+0000","last_fullsized":"2023-11-09T03:46:52.556363+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:33:19.817774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.550330+0000","last_change":"2023-11-09T03:46:52.545583+0000","last_active":"2023-11-09T03:46:52.550330+0000","last_peered":"2023-11-09T03:46:52.550330+0000","last_clean":"2023-11-09T03:46:52.550330+0000","last_became_active":"2023-11-09T03:46:52.545323+0000","last_became_peered":"2023-11-09T03:46:52.545323+0000","last_unstale":"2023-11-09T03:46:52.550330+0000","last_undegraded":"2023-11-09T03:46:52.550330+0000","last_fullsized":"2023-11-09T03:46:52.550330+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:21:25.255050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.555937+0000","last_change":"2023-11-09T03:46:52.549453+0000","last_active":"2023-11-09T03:46:52.555937+0000","last_peered":"2023-11-09T03:46:52.555937+0000","last_clean":"2023-11-09T03:46:52.555937+0000","last_became_active":"2023-11-09T03:46:52.549167+0000","last_became_peered":"2023-11-09T03:46:52.549167+0000","last_unstale":"2023-11-09T03:46:52.555937+0000","last_undegraded":"2023-11-09T03:46:52.555937+0000","last_fullsized":"2023-11-09T03:46:52.555937+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:45:32.257609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.553413+0000","last_change":"2023-11-09T03:46:52.549056+0000","last_active":"2023-11-09T03:46:52.553413+0000","last_peered":"2023-11-09T03:46:52.553413+0000","last_clean":"2023-11-09T03:46:52.553413+0000","last_became_active":"2023-11-09T03:46:52.548267+0000","last_became_peered":"2023-11-09T03:46:52.548267+0000","last_unstale":"2023-11-09T03:46:52.553413+0000","last_undegraded":"2023-11-09T03:46:52.553413+0000","last_fullsized":"2023-11-09T03:46:52.553413+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:45:42.507078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.545633+0000","last_change":"2023-11-09T03:46:52.538756+0000","last_active":"2023-11-09T03:46:52.545633+0000","last_peered":"2023-11-09T03:46:52.545633+0000","last_clean":"2023-11-09T03:46:52.545633+0000","last_became_active":"2023-11-09T03:46:52.538242+0000","last_became_peered":"2023-11-09T03:46:52.538242+0000","last_unstale":"2023-11-09T03:46:52.545633+0000","last_undegraded":"2023-11-09T03:46:52.545633+0000","last_fullsized":"2023-11-09T03:46:52.545633+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:59:08.370172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.547544+0000","last_change":"2023-11-09T03:46:52.538850+0000","last_active":"2023-11-09T03:46:52.547544+0000","last_peered":"2023-11-09T03:46:52.547544+0000","last_clean":"2023-11-09T03:46:52.547544+0000","last_became_active":"2023-11-09T03:46:52.538555+0000","last_became_peered":"2023-11-09T03:46:52.538555+0000","last_unstale":"2023-11-09T03:46:52.547544+0000","last_undegraded":"2023-11-09T03:46:52.547544+0000","last_fullsized":"2023-11-09T03:46:52.547544+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:01:07.202627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.544225+0000","last_change":"2023-11-09T03:46:52.542931+0000","last_active":"2023-11-09T03:46:52.544225+0000","last_peered":"2023-11-09T03:46:52.544225+0000","last_clean":"2023-11-09T03:46:52.544225+0000","last_became_active":"2023-11-09T03:46:52.542659+0000","last_became_peered":"2023-11-09T03:46:52.542659+0000","last_unstale":"2023-11-09T03:46:52.544225+0000","last_undegraded":"2023-11-09T03:46:52.544225+0000","last_fullsized":"2023-11-09T03:46:52.544225+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:14:57.063826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.537457+0000","last_change":"2023-11-09T03:46:52.534044+0000","last_active":"2023-11-09T03:46:52.537457+0000","last_peered":"2023-11-09T03:46:52.537457+0000","last_clean":"2023-11-09T03:46:52.537457+0000","last_became_active":"2023-11-09T03:46:52.533706+0000","last_became_peered":"2023-11-09T03:46:52.533706+0000","last_unstale":"2023-11-09T03:46:52.537457+0000","last_undegraded":"2023-11-09T03:46:52.537457+0000","last_fullsized":"2023-11-09T03:46:52.537457+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:14:57.063826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.550286+0000","last_change":"2023-11-09T03:46:52.544936+0000","last_active":"2023-11-09T03:46:52.550286+0000","last_peered":"2023-11-09T03:46:52.550286+0000","last_clean":"2023-11-09T03:46:52.550286+0000","last_became_active":"2023-11-09T03:46:52.544663+0000","last_became_peered":"2023-11-09T03:46:52.544663+0000","last_unstale":"2023-11-09T03:46:52.550286+0000","last_undegraded":"2023-11-09T03:46:52.550286+0000","last_fullsized":"2023-11-09T03:46:52.550286+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:06:21.934969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.552118+0000","last_change":"2023-11-09T03:46:52.544101+0000","last_active":"2023-11-09T03:46:52.552118+0000","last_peered":"2023-11-09T03:46:52.552118+0000","last_clean":"2023-11-09T03:46:52.552118+0000","last_became_active":"2023-11-09T03:46:52.543822+0000","last_became_peered":"2023-11-09T03:46:52.543822+0000","last_unstale":"2023-11-09T03:46:52.552118+0000","last_undegraded":"2023-11-09T03:46:52.552118+0000","last_fullsized":"2023-11-09T03:46:52.552118+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:49:44.445256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.549941+0000","last_change":"2023-11-09T03:46:52.539639+0000","last_active":"2023-11-09T03:46:52.549941+0000","last_peered":"2023-11-09T03:46:52.549941+0000","last_clean":"2023-11-09T03:46:52.549941+0000","last_became_active":"2023-11-09T03:46:52.537981+0000","last_became_peered":"2023-11-09T03:46:52.537981+0000","last_unstale":"2023-11-09T03:46:52.549941+0000","last_undegraded":"2023-11-09T03:46:52.549941+0000","last_fullsized":"2023-11-09T03:46:52.549941+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:17:18.364598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.551373+0000","last_change":"2023-11-09T03:46:52.548900+0000","last_active":"2023-11-09T03:46:52.551373+0000","last_peered":"2023-11-09T03:46:52.551373+0000","last_clean":"2023-11-09T03:46:52.551373+0000","last_became_active":"2023-11-09T03:46:52.548645+0000","last_became_peered":"2023-11-09T03:46:52.548645+0000","last_unstale":"2023-11-09T03:46:52.551373+0000","last_undegraded":"2023-11-09T03:46:52.551373+0000","last_fullsized":"2023-11-09T03:46:52.551373+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:41:30.870772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.550665+0000","last_change":"2023-11-09T03:46:52.548409+0000","last_active":"2023-11-09T03:46:52.550665+0000","last_peered":"2023-11-09T03:46:52.550665+0000","last_clean":"2023-11-09T03:46:52.550665+0000","last_became_active":"2023-11-09T03:46:52.548078+0000","last_became_peered":"2023-11-09T03:46:52.548078+0000","last_unstale":"2023-11-09T03:46:52.550665+0000","last_undegraded":"2023-11-09T03:46:52.550665+0000","last_fullsized":"2023-11-09T03:46:52.550665+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:17:18.364598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.551120+0000","last_change":"2023-11-09T03:46:52.546459+0000","last_active":"2023-11-09T03:46:52.551120+0000","last_peered":"2023-11-09T03:46:52.551120+0000","last_clean":"2023-11-09T03:46:52.551120+0000","last_became_active":"2023-11-09T03:46:52.546201+0000","last_became_peered":"2023-11-09T03:46:52.546201+0000","last_unstale":"2023-11-09T03:46:52.551120+0000","last_undegraded":"2023-11-09T03:46:52.551120+0000","last_fullsized":"2023-11-09T03:46:52.551120+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:24:42.894740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.559620+0000","last_change":"2023-11-09T03:46:51.546351+0000","last_active":"2023-11-09T03:46:52.559620+0000","last_peered":"2023-11-09T03:46:52.559620+0000","last_clean":"2023-11-09T03:46:52.559620+0000","last_became_active":"2023-11-09T03:46:51.544196+0000","last_became_peered":"2023-11-09T03:46:51.544196+0000","last_unstale":"2023-11-09T03:46:52.559620+0000","last_undegraded":"2023-11-09T03:46:52.559620+0000","last_fullsized":"2023-11-09T03:46:52.559620+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:56:44.359014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.550062+0000","last_change":"2023-11-09T03:46:52.546830+0000","last_active":"2023-11-09T03:46:52.550062+0000","last_peered":"2023-11-09T03:46:52.550062+0000","last_clean":"2023-11-09T03:46:52.550062+0000","last_became_active":"2023-11-09T03:46:52.546532+0000","last_became_peered":"2023-11-09T03:46:52.546532+0000","last_unstale":"2023-11-09T03:46:52.550062+0000","last_undegraded":"2023-11-09T03:46:52.550062+0000","last_fullsized":"2023-11-09T03:46:52.550062+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:18:34.570683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.550988+0000","last_change":"2023-11-09T03:46:51.543725+0000","last_active":"2023-11-09T03:46:52.550988+0000","last_peered":"2023-11-09T03:46:52.550988+0000","last_clean":"2023-11-09T03:46:52.550988+0000","last_became_active":"2023-11-09T03:46:51.542473+0000","last_became_peered":"2023-11-09T03:46:51.542473+0000","last_unstale":"2023-11-09T03:46:52.550988+0000","last_undegraded":"2023-11-09T03:46:52.550988+0000","last_fullsized":"2023-11-09T03:46:52.550988+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:50:37.650301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.558963+0000","last_change":"2023-11-09T03:46:52.553157+0000","last_active":"2023-11-09T03:46:52.558963+0000","last_peered":"2023-11-09T03:46:52.558963+0000","last_clean":"2023-11-09T03:46:52.558963+0000","last_became_active":"2023-11-09T03:46:52.552695+0000","last_became_peered":"2023-11-09T03:46:52.552695+0000","last_unstale":"2023-11-09T03:46:52.558963+0000","last_undegraded":"2023-11-09T03:46:52.558963+0000","last_fullsized":"2023-11-09T03:46:52.558963+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:01:07.202627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.551888+0000","last_change":"2023-11-09T03:46:51.543977+0000","last_active":"2023-11-09T03:46:52.551888+0000","last_peered":"2023-11-09T03:46:52.551888+0000","last_clean":"2023-11-09T03:46:52.551888+0000","last_became_active":"2023-11-09T03:46:51.543662+0000","last_became_peered":"2023-11-09T03:46:51.543662+0000","last_unstale":"2023-11-09T03:46:52.551888+0000","last_undegraded":"2023-11-09T03:46:52.551888+0000","last_fullsized":"2023-11-09T03:46:52.551888+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:30:38.798792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.553457+0000","last_change":"2023-11-09T03:46:52.548238+0000","last_active":"2023-11-09T03:46:52.553457+0000","last_peered":"2023-11-09T03:46:52.553457+0000","last_clean":"2023-11-09T03:46:52.553457+0000","last_became_active":"2023-11-09T03:46:52.547989+0000","last_became_peered":"2023-11-09T03:46:52.547989+0000","last_unstale":"2023-11-09T03:46:52.553457+0000","last_undegraded":"2023-11-09T03:46:52.553457+0000","last_fullsized":"2023-11-09T03:46:52.553457+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:15:53.977101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.558949+0000","last_change":"2023-11-09T03:46:51.541786+0000","last_active":"2023-11-09T03:46:52.558949+0000","last_peered":"2023-11-09T03:46:52.558949+0000","last_clean":"2023-11-09T03:46:52.558949+0000","last_became_active":"2023-11-09T03:46:51.541457+0000","last_became_peered":"2023-11-09T03:46:51.541457+0000","last_unstale":"2023-11-09T03:46:52.558949+0000","last_undegraded":"2023-11-09T03:46:52.558949+0000","last_fullsized":"2023-11-09T03:46:52.558949+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:50:37.650301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.549095+0000","last_change":"2023-11-09T03:46:52.543573+0000","last_active":"2023-11-09T03:46:52.549095+0000","last_peered":"2023-11-09T03:46:52.549095+0000","last_clean":"2023-11-09T03:46:52.549095+0000","last_became_active":"2023-11-09T03:46:52.543301+0000","last_became_peered":"2023-11-09T03:46:52.543301+0000","last_unstale":"2023-11-09T03:46:52.549095+0000","last_undegraded":"2023-11-09T03:46:52.549095+0000","last_fullsized":"2023-11-09T03:46:52.549095+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:23:12.573527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556695+0000","last_change":"2023-11-09T03:46:51.543897+0000","last_active":"2023-11-09T03:46:52.556695+0000","last_peered":"2023-11-09T03:46:52.556695+0000","last_clean":"2023-11-09T03:46:52.556695+0000","last_became_active":"2023-11-09T03:46:51.543536+0000","last_became_peered":"2023-11-09T03:46:51.543536+0000","last_unstale":"2023-11-09T03:46:52.556695+0000","last_undegraded":"2023-11-09T03:46:52.556695+0000","last_fullsized":"2023-11-09T03:46:52.556695+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:41.783929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.550312+0000","last_change":"2023-11-09T03:46:52.545027+0000","last_active":"2023-11-09T03:46:52.550312+0000","last_peered":"2023-11-09T03:46:52.550312+0000","last_clean":"2023-11-09T03:46:52.550312+0000","last_became_active":"2023-11-09T03:46:52.544727+0000","last_became_peered":"2023-11-09T03:46:52.544727+0000","last_unstale":"2023-11-09T03:46:52.550312+0000","last_undegraded":"2023-11-09T03:46:52.550312+0000","last_fullsized":"2023-11-09T03:46:52.550312+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:45:42.507078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.550636+0000","last_change":"2023-11-09T03:46:51.550073+0000","last_active":"2023-11-09T03:46:52.550636+0000","last_peered":"2023-11-09T03:46:52.550636+0000","last_clean":"2023-11-09T03:46:52.550636+0000","last_became_active":"2023-11-09T03:46:51.549783+0000","last_became_peered":"2023-11-09T03:46:51.549783+0000","last_unstale":"2023-11-09T03:46:52.550636+0000","last_undegraded":"2023-11-09T03:46:52.550636+0000","last_fullsized":"2023-11-09T03:46:52.550636+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:30:38.798792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.555802+0000","last_change":"2023-11-09T03:46:52.546609+0000","last_active":"2023-11-09T03:46:52.555802+0000","last_peered":"2023-11-09T03:46:52.555802+0000","last_clean":"2023-11-09T03:46:52.555802+0000","last_became_active":"2023-11-09T03:46:52.546232+0000","last_became_peered":"2023-11-09T03:46:52.546232+0000","last_unstale":"2023-11-09T03:46:52.555802+0000","last_undegraded":"2023-11-09T03:46:52.555802+0000","last_fullsized":"2023-11-09T03:46:52.555802+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:56:45.361033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.533032+0000","last_change":"2023-11-09T03:46:51.545360+0000","last_active":"2023-11-09T03:46:52.533032+0000","last_peered":"2023-11-09T03:46:52.533032+0000","last_clean":"2023-11-09T03:46:52.533032+0000","last_became_active":"2023-11-09T03:46:51.545025+0000","last_became_peered":"2023-11-09T03:46:51.545025+0000","last_unstale":"2023-11-09T03:46:52.533032+0000","last_undegraded":"2023-11-09T03:46:52.533032+0000","last_fullsized":"2023-11-09T03:46:52.533032+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:23:11.571508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.552927+0000","last_change":"2023-11-09T03:46:52.545844+0000","last_active":"2023-11-09T03:46:52.552927+0000","last_peered":"2023-11-09T03:46:52.552927+0000","last_clean":"2023-11-09T03:46:52.552927+0000","last_became_active":"2023-11-09T03:46:52.545553+0000","last_became_peered":"2023-11-09T03:46:52.545553+0000","last_unstale":"2023-11-09T03:46:52.552927+0000","last_undegraded":"2023-11-09T03:46:52.552927+0000","last_fullsized":"2023-11-09T03:46:52.552927+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:47:43.093864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.531928+0000","last_change":"2023-11-09T03:46:51.545228+0000","last_active":"2023-11-09T03:46:52.531928+0000","last_peered":"2023-11-09T03:46:52.531928+0000","last_clean":"2023-11-09T03:46:52.531928+0000","last_became_active":"2023-11-09T03:46:51.544953+0000","last_became_peered":"2023-11-09T03:46:51.544953+0000","last_unstale":"2023-11-09T03:46:52.531928+0000","last_undegraded":"2023-11-09T03:46:52.531928+0000","last_fullsized":"2023-11-09T03:46:52.531928+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:17:17.362580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.555874+0000","last_change":"2023-11-09T03:46:52.546443+0000","last_active":"2023-11-09T03:46:52.555874+0000","last_peered":"2023-11-09T03:46:52.555874+0000","last_clean":"2023-11-09T03:46:52.555874+0000","last_became_active":"2023-11-09T03:46:52.546202+0000","last_became_peered":"2023-11-09T03:46:52.546202+0000","last_unstale":"2023-11-09T03:46:52.555874+0000","last_undegraded":"2023-11-09T03:46:52.555874+0000","last_fullsized":"2023-11-09T03:46:52.555874+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:49:44.445256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.535319+0000","last_change":"2023-11-09T03:46:51.539567+0000","last_active":"2023-11-09T03:46:52.535319+0000","last_peered":"2023-11-09T03:46:52.535319+0000","last_clean":"2023-11-09T03:46:52.535319+0000","last_became_active":"2023-11-09T03:46:51.539267+0000","last_became_peered":"2023-11-09T03:46:51.539267+0000","last_unstale":"2023-11-09T03:46:52.535319+0000","last_undegraded":"2023-11-09T03:46:52.535319+0000","last_fullsized":"2023-11-09T03:46:52.535319+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:37:53.997975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.551146+0000","last_change":"2023-11-09T03:46:52.548194+0000","last_active":"2023-11-09T03:46:52.551146+0000","last_peered":"2023-11-09T03:46:52.551146+0000","last_clean":"2023-11-09T03:46:52.551146+0000","last_became_active":"2023-11-09T03:46:52.547621+0000","last_became_peered":"2023-11-09T03:46:52.547621+0000","last_unstale":"2023-11-09T03:46:52.551146+0000","last_undegraded":"2023-11-09T03:46:52.551146+0000","last_fullsized":"2023-11-09T03:46:52.551146+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:19:23.645673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.534907+0000","last_change":"2023-11-09T03:46:51.544088+0000","last_active":"2023-11-09T03:46:52.534907+0000","last_peered":"2023-11-09T03:46:52.534907+0000","last_clean":"2023-11-09T03:46:52.534907+0000","last_became_active":"2023-11-09T03:46:51.543757+0000","last_became_peered":"2023-11-09T03:46:51.543757+0000","last_unstale":"2023-11-09T03:46:52.534907+0000","last_undegraded":"2023-11-09T03:46:52.534907+0000","last_fullsized":"2023-11-09T03:46:52.534907+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:18:48.180569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.544717+0000","last_change":"2023-11-09T03:46:52.539685+0000","last_active":"2023-11-09T03:46:52.544717+0000","last_peered":"2023-11-09T03:46:52.544717+0000","last_clean":"2023-11-09T03:46:52.544717+0000","last_became_active":"2023-11-09T03:46:52.539423+0000","last_became_peered":"2023-11-09T03:46:52.539423+0000","last_unstale":"2023-11-09T03:46:52.544717+0000","last_undegraded":"2023-11-09T03:46:52.544717+0000","last_fullsized":"2023-11-09T03:46:52.544717+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:33:19.817774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.559118+0000","last_change":"2023-11-09T03:46:51.544185+0000","last_active":"2023-11-09T03:46:52.559118+0000","last_peered":"2023-11-09T03:46:52.559118+0000","last_clean":"2023-11-09T03:46:52.559118+0000","last_became_active":"2023-11-09T03:46:51.543867+0000","last_became_peered":"2023-11-09T03:46:51.543867+0000","last_unstale":"2023-11-09T03:46:52.559118+0000","last_undegraded":"2023-11-09T03:46:52.559118+0000","last_fullsized":"2023-11-09T03:46:52.559118+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:54:00.678572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556060+0000","last_change":"2023-11-09T03:46:52.549350+0000","last_active":"2023-11-09T03:46:52.556060+0000","last_peered":"2023-11-09T03:46:52.556060+0000","last_clean":"2023-11-09T03:46:52.556060+0000","last_became_active":"2023-11-09T03:46:52.549121+0000","last_became_peered":"2023-11-09T03:46:52.549121+0000","last_unstale":"2023-11-09T03:46:52.556060+0000","last_undegraded":"2023-11-09T03:46:52.556060+0000","last_fullsized":"2023-11-09T03:46:52.556060+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:14:57.063826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.559200+0000","last_change":"2023-11-09T03:46:51.542543+0000","last_active":"2023-11-09T03:46:52.559200+0000","last_peered":"2023-11-09T03:46:52.559200+0000","last_clean":"2023-11-09T03:46:52.559200+0000","last_became_active":"2023-11-09T03:46:51.542258+0000","last_became_peered":"2023-11-09T03:46:51.542258+0000","last_unstale":"2023-11-09T03:46:52.559200+0000","last_undegraded":"2023-11-09T03:46:52.559200+0000","last_fullsized":"2023-11-09T03:46:52.559200+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:06:20.932951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556166+0000","last_change":"2023-11-09T03:46:52.549204+0000","last_active":"2023-11-09T03:46:52.556166+0000","last_peered":"2023-11-09T03:46:52.556166+0000","last_clean":"2023-11-09T03:46:52.556166+0000","last_became_active":"2023-11-09T03:46:52.548902+0000","last_became_peered":"2023-11-09T03:46:52.548902+0000","last_unstale":"2023-11-09T03:46:52.556166+0000","last_undegraded":"2023-11-09T03:46:52.556166+0000","last_fullsized":"2023-11-09T03:46:52.556166+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:45:32.257609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.549178+0000","last_change":"2023-11-09T03:46:51.543845+0000","last_active":"2023-11-09T03:46:52.549178+0000","last_peered":"2023-11-09T03:46:52.549178+0000","last_clean":"2023-11-09T03:46:52.549178+0000","last_became_active":"2023-11-09T03:46:51.543592+0000","last_became_peered":"2023-11-09T03:46:51.543592+0000","last_unstale":"2023-11-09T03:46:52.549178+0000","last_undegraded":"2023-11-09T03:46:52.549178+0000","last_fullsized":"2023-11-09T03:46:52.549178+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:56:45.569356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.552942+0000","last_change":"2023-11-09T03:46:52.545527+0000","last_active":"2023-11-09T03:46:52.552942+0000","last_peered":"2023-11-09T03:46:52.552942+0000","last_clean":"2023-11-09T03:46:52.552942+0000","last_became_active":"2023-11-09T03:46:52.545262+0000","last_became_peered":"2023-11-09T03:46:52.545262+0000","last_unstale":"2023-11-09T03:46:52.552942+0000","last_undegraded":"2023-11-09T03:46:52.552942+0000","last_fullsized":"2023-11-09T03:46:52.552942+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:41:30.870772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.534481+0000","last_change":"2023-11-09T03:46:51.547646+0000","last_active":"2023-11-09T03:46:52.534481+0000","last_peered":"2023-11-09T03:46:52.534481+0000","last_clean":"2023-11-09T03:46:52.534481+0000","last_became_active":"2023-11-09T03:46:51.547347+0000","last_became_peered":"2023-11-09T03:46:51.547347+0000","last_unstale":"2023-11-09T03:46:52.534481+0000","last_undegraded":"2023-11-09T03:46:52.534481+0000","last_fullsized":"2023-11-09T03:46:52.534481+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:50:37.650301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.551054+0000","last_change":"2023-11-09T03:46:52.547960+0000","last_active":"2023-11-09T03:46:52.551054+0000","last_peered":"2023-11-09T03:46:52.551054+0000","last_clean":"2023-11-09T03:46:52.551054+0000","last_became_active":"2023-11-09T03:46:52.547708+0000","last_became_peered":"2023-11-09T03:46:52.547708+0000","last_unstale":"2023-11-09T03:46:52.551054+0000","last_undegraded":"2023-11-09T03:46:52.551054+0000","last_fullsized":"2023-11-09T03:46:52.551054+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:50:38.652320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.529206+0000","last_change":"2023-11-09T03:46:51.545011+0000","last_active":"2023-11-09T03:46:52.529206+0000","last_peered":"2023-11-09T03:46:52.529206+0000","last_clean":"2023-11-09T03:46:52.529206+0000","last_became_active":"2023-11-09T03:46:51.544757+0000","last_became_peered":"2023-11-09T03:46:51.544757+0000","last_unstale":"2023-11-09T03:46:52.529206+0000","last_undegraded":"2023-11-09T03:46:52.529206+0000","last_fullsized":"2023-11-09T03:46:52.529206+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:23:11.571508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.555844+0000","last_change":"2023-11-09T03:46:52.549727+0000","last_active":"2023-11-09T03:46:52.555844+0000","last_peered":"2023-11-09T03:46:52.555844+0000","last_clean":"2023-11-09T03:46:52.555844+0000","last_became_active":"2023-11-09T03:46:52.549460+0000","last_became_peered":"2023-11-09T03:46:52.549460+0000","last_unstale":"2023-11-09T03:46:52.555844+0000","last_undegraded":"2023-11-09T03:46:52.555844+0000","last_fullsized":"2023-11-09T03:46:52.555844+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:54:01.680591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":19,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556823+0000","last_change":"2023-11-09T03:46:51.550027+0000","last_active":"2023-11-09T03:46:52.556823+0000","last_peered":"2023-11-09T03:46:52.556823+0000","last_clean":"2023-11-09T03:46:52.556823+0000","last_became_active":"2023-11-09T03:46:51.549729+0000","last_became_peered":"2023-11-09T03:46:51.549729+0000","last_unstale":"2023-11-09T03:46:52.556823+0000","last_undegraded":"2023-11-09T03:46:52.556823+0000","last_fullsized":"2023-11-09T03:46:52.556823+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:21:24.253031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.546260+0000","last_change":"2023-11-09T03:46:52.540718+0000","last_active":"2023-11-09T03:46:52.546260+0000","last_peered":"2023-11-09T03:46:52.546260+0000","last_clean":"2023-11-09T03:46:52.546260+0000","last_became_active":"2023-11-09T03:46:52.540445+0000","last_became_peered":"2023-11-09T03:46:52.540445+0000","last_unstale":"2023-11-09T03:46:52.546260+0000","last_undegraded":"2023-11-09T03:46:52.546260+0000","last_fullsized":"2023-11-09T03:46:52.546260+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:16:40.068588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.559242+0000","last_change":"2023-11-09T03:46:51.545514+0000","last_active":"2023-11-09T03:46:52.559242+0000","last_peered":"2023-11-09T03:46:52.559242+0000","last_clean":"2023-11-09T03:46:52.559242+0000","last_became_active":"2023-11-09T03:46:51.545188+0000","last_became_peered":"2023-11-09T03:46:51.545188+0000","last_unstale":"2023-11-09T03:46:52.559242+0000","last_undegraded":"2023-11-09T03:46:52.559242+0000","last_fullsized":"2023-11-09T03:46:52.559242+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:47:42.091845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556299+0000","last_change":"2023-11-09T03:46:52.546462+0000","last_active":"2023-11-09T03:46:52.556299+0000","last_peered":"2023-11-09T03:46:52.556299+0000","last_clean":"2023-11-09T03:46:52.556299+0000","last_became_active":"2023-11-09T03:46:52.546199+0000","last_became_peered":"2023-11-09T03:46:52.546199+0000","last_unstale":"2023-11-09T03:46:52.556299+0000","last_undegraded":"2023-11-09T03:46:52.556299+0000","last_fullsized":"2023-11-09T03:46:52.556299+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:19:23.645673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.559423+0000","last_change":"2023-11-09T03:46:51.544802+0000","last_active":"2023-11-09T03:46:52.559423+0000","last_peered":"2023-11-09T03:46:52.559423+0000","last_clean":"2023-11-09T03:46:52.559423+0000","last_became_active":"2023-11-09T03:46:51.544516+0000","last_became_peered":"2023-11-09T03:46:51.544516+0000","last_unstale":"2023-11-09T03:46:52.559423+0000","last_undegraded":"2023-11-09T03:46:52.559423+0000","last_fullsized":"2023-11-09T03:46:52.559423+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:18:33.568664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.551166+0000","last_change":"2023-11-09T03:46:52.545832+0000","last_active":"2023-11-09T03:46:52.551166+0000","last_peered":"2023-11-09T03:46:52.551166+0000","last_clean":"2023-11-09T03:46:52.551166+0000","last_became_active":"2023-11-09T03:46:52.545495+0000","last_became_peered":"2023-11-09T03:46:52.545495+0000","last_unstale":"2023-11-09T03:46:52.551166+0000","last_undegraded":"2023-11-09T03:46:52.551166+0000","last_fullsized":"2023-11-09T03:46:52.551166+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:23:12.573527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.555506+0000","last_change":"2023-11-09T03:46:51.543340+0000","last_active":"2023-11-09T03:46:52.555506+0000","last_peered":"2023-11-09T03:46:52.555506+0000","last_clean":"2023-11-09T03:46:52.555506+0000","last_became_active":"2023-11-09T03:46:51.543057+0000","last_became_peered":"2023-11-09T03:46:51.543057+0000","last_unstale":"2023-11-09T03:46:52.555506+0000","last_undegraded":"2023-11-09T03:46:52.555506+0000","last_fullsized":"2023-11-09T03:46:52.555506+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:47:13.365771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.553189+0000","last_change":"2023-11-09T03:46:52.541800+0000","last_active":"2023-11-09T03:46:52.553189+0000","last_peered":"2023-11-09T03:46:52.553189+0000","last_clean":"2023-11-09T03:46:52.553189+0000","last_became_active":"2023-11-09T03:46:52.541545+0000","last_became_peered":"2023-11-09T03:46:52.541545+0000","last_unstale":"2023-11-09T03:46:52.553189+0000","last_undegraded":"2023-11-09T03:46:52.553189+0000","last_fullsized":"2023-11-09T03:46:52.553189+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:05:30.357983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.535418+0000","last_change":"2023-11-09T03:46:51.544307+0000","last_active":"2023-11-09T03:46:52.535418+0000","last_peered":"2023-11-09T03:46:52.535418+0000","last_clean":"2023-11-09T03:46:52.535418+0000","last_became_active":"2023-11-09T03:46:51.543903+0000","last_became_peered":"2023-11-09T03:46:51.543903+0000","last_unstale":"2023-11-09T03:46:52.535418+0000","last_undegraded":"2023-11-09T03:46:52.535418+0000","last_fullsized":"2023-11-09T03:46:52.535418+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:21:24.253031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.551170+0000","last_change":"2023-11-09T03:46:52.548578+0000","last_active":"2023-11-09T03:46:52.551170+0000","last_peered":"2023-11-09T03:46:52.551170+0000","last_clean":"2023-11-09T03:46:52.551170+0000","last_became_active":"2023-11-09T03:46:52.548288+0000","last_became_peered":"2023-11-09T03:46:52.548288+0000","last_unstale":"2023-11-09T03:46:52.551170+0000","last_undegraded":"2023-11-09T03:46:52.551170+0000","last_fullsized":"2023-11-09T03:46:52.551170+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:04:23.215294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556491+0000","last_change":"2023-11-09T03:46:51.538030+0000","last_active":"2023-11-09T03:46:52.556491+0000","last_peered":"2023-11-09T03:46:52.556491+0000","last_clean":"2023-11-09T03:46:52.556491+0000","last_became_active":"2023-11-09T03:46:51.537773+0000","last_became_peered":"2023-11-09T03:46:51.537773+0000","last_unstale":"2023-11-09T03:46:52.556491+0000","last_undegraded":"2023-11-09T03:46:52.556491+0000","last_fullsized":"2023-11-09T03:46:52.556491+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:35:31.702881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.542596+0000","last_change":"2023-11-09T03:46:52.538634+0000","last_active":"2023-11-09T03:46:52.542596+0000","last_peered":"2023-11-09T03:46:52.542596+0000","last_clean":"2023-11-09T03:46:52.542596+0000","last_became_active":"2023-11-09T03:46:52.538289+0000","last_became_peered":"2023-11-09T03:46:52.538289+0000","last_unstale":"2023-11-09T03:46:52.542596+0000","last_undegraded":"2023-11-09T03:46:52.542596+0000","last_fullsized":"2023-11-09T03:46:52.542596+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:56:45.361033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.527789+0000","last_change":"2023-11-09T03:46:51.536546+0000","last_active":"2023-11-09T03:46:52.527789+0000","last_peered":"2023-11-09T03:46:52.527789+0000","last_clean":"2023-11-09T03:46:52.527789+0000","last_became_active":"2023-11-09T03:46:51.536234+0000","last_became_peered":"2023-11-09T03:46:51.536234+0000","last_unstale":"2023-11-09T03:46:52.527789+0000","last_undegraded":"2023-11-09T03:46:52.527789+0000","last_fullsized":"2023-11-09T03:46:52.527789+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:49:35.405663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.550449+0000","last_change":"2023-11-09T03:46:52.545767+0000","last_active":"2023-11-09T03:46:52.550449+0000","last_peered":"2023-11-09T03:46:52.550449+0000","last_clean":"2023-11-09T03:46:52.550449+0000","last_became_active":"2023-11-09T03:46:52.545496+0000","last_became_peered":"2023-11-09T03:46:52.545496+0000","last_unstale":"2023-11-09T03:46:52.550449+0000","last_undegraded":"2023-11-09T03:46:52.550449+0000","last_fullsized":"2023-11-09T03:46:52.550449+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:16:40.068588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.533384+0000","last_change":"2023-11-09T03:46:51.546183+0000","last_active":"2023-11-09T03:46:52.533384+0000","last_peered":"2023-11-09T03:46:52.533384+0000","last_clean":"2023-11-09T03:46:52.533384+0000","last_became_active":"2023-11-09T03:46:51.545928+0000","last_became_peered":"2023-11-09T03:46:51.545928+0000","last_unstale":"2023-11-09T03:46:52.533384+0000","last_undegraded":"2023-11-09T03:46:52.533384+0000","last_fullsized":"2023-11-09T03:46:52.533384+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:41.783929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.550325+0000","last_change":"2023-11-09T03:46:52.544861+0000","last_active":"2023-11-09T03:46:52.550325+0000","last_peered":"2023-11-09T03:46:52.550325+0000","last_clean":"2023-11-09T03:46:52.550325+0000","last_became_active":"2023-11-09T03:46:52.544615+0000","last_became_peered":"2023-11-09T03:46:52.544615+0000","last_unstale":"2023-11-09T03:46:52.550325+0000","last_undegraded":"2023-11-09T03:46:52.550325+0000","last_fullsized":"2023-11-09T03:46:52.550325+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:04:23.215294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.528218+0000","last_change":"2023-11-09T03:46:51.534485+0000","last_active":"2023-11-09T03:46:52.528218+0000","last_peered":"2023-11-09T03:46:52.528218+0000","last_clean":"2023-11-09T03:46:52.528218+0000","last_became_active":"2023-11-09T03:46:51.534177+0000","last_became_peered":"2023-11-09T03:46:51.534177+0000","last_unstale":"2023-11-09T03:46:52.528218+0000","last_undegraded":"2023-11-09T03:46:52.528218+0000","last_fullsized":"2023-11-09T03:46:52.528218+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:41:25.354316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.550639+0000","last_change":"2023-11-09T03:46:52.547161+0000","last_active":"2023-11-09T03:46:52.550639+0000","last_peered":"2023-11-09T03:46:52.550639+0000","last_clean":"2023-11-09T03:46:52.550639+0000","last_became_active":"2023-11-09T03:46:52.546513+0000","last_became_peered":"2023-11-09T03:46:52.546513+0000","last_unstale":"2023-11-09T03:46:52.550639+0000","last_undegraded":"2023-11-09T03:46:52.550639+0000","last_fullsized":"2023-11-09T03:46:52.550639+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:45:42.507078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.556522+0000","last_change":"2023-11-09T03:46:51.544691+0000","last_active":"2023-11-09T03:46:52.556522+0000","last_peered":"2023-11-09T03:46:52.556522+0000","last_clean":"2023-11-09T03:46:52.556522+0000","last_became_active":"2023-11-09T03:46:51.544360+0000","last_became_peered":"2023-11-09T03:46:51.544360+0000","last_unstale":"2023-11-09T03:46:52.556522+0000","last_undegraded":"2023-11-09T03:46:52.556522+0000","last_fullsized":"2023-11-09T03:46:52.556522+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:56:45.569356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.551128+0000","last_change":"2023-11-09T03:46:52.548177+0000","last_active":"2023-11-09T03:46:52.551128+0000","last_peered":"2023-11-09T03:46:52.551128+0000","last_clean":"2023-11-09T03:46:52.551128+0000","last_became_active":"2023-11-09T03:46:52.546935+0000","last_became_peered":"2023-11-09T03:46:52.546935+0000","last_unstale":"2023-11-09T03:46:52.551128+0000","last_undegraded":"2023-11-09T03:46:52.551128+0000","last_fullsized":"2023-11-09T03:46:52.551128+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:47:43.093864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.528156+0000","last_change":"2023-11-09T03:46:51.545122+0000","last_active":"2023-11-09T03:46:52.528156+0000","last_peered":"2023-11-09T03:46:52.528156+0000","last_clean":"2023-11-09T03:46:52.528156+0000","last_became_active":"2023-11-09T03:46:51.544846+0000","last_became_peered":"2023-11-09T03:46:51.544846+0000","last_unstale":"2023-11-09T03:46:52.528156+0000","last_undegraded":"2023-11-09T03:46:52.528156+0000","last_fullsized":"2023-11-09T03:46:52.528156+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:56:22.557010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.558290+0000","last_change":"2023-11-09T03:46:52.553307+0000","last_active":"2023-11-09T03:46:52.558290+0000","last_peered":"2023-11-09T03:46:52.558290+0000","last_clean":"2023-11-09T03:46:52.558290+0000","last_became_active":"2023-11-09T03:46:52.553028+0000","last_became_peered":"2023-11-09T03:46:52.553028+0000","last_unstale":"2023-11-09T03:46:52.558290+0000","last_undegraded":"2023-11-09T03:46:52.558290+0000","last_fullsized":"2023-11-09T03:46:52.558290+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:57:09.422287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.532800+0000","last_change":"2023-11-09T03:46:51.540889+0000","last_active":"2023-11-09T03:46:52.532800+0000","last_peered":"2023-11-09T03:46:52.532800+0000","last_clean":"2023-11-09T03:46:52.532800+0000","last_became_active":"2023-11-09T03:46:51.540634+0000","last_became_peered":"2023-11-09T03:46:51.540634+0000","last_unstale":"2023-11-09T03:46:52.532800+0000","last_undegraded":"2023-11-09T03:46:52.532800+0000","last_fullsized":"2023-11-09T03:46:52.532800+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:45:41.505059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.553124+0000","last_change":"2023-11-09T03:46:52.545181+0000","last_active":"2023-11-09T03:46:52.553124+0000","last_peered":"2023-11-09T03:46:52.553124+0000","last_clean":"2023-11-09T03:46:52.553124+0000","last_became_active":"2023-11-09T03:46:52.544385+0000","last_became_peered":"2023-11-09T03:46:52.544385+0000","last_unstale":"2023-11-09T03:46:52.553124+0000","last_undegraded":"2023-11-09T03:46:52.553124+0000","last_fullsized":"2023-11-09T03:46:52.553124+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:18:34.570683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.533587+0000","last_change":"2023-11-09T03:46:51.545342+0000","last_active":"2023-11-09T03:46:52.533587+0000","last_peered":"2023-11-09T03:46:52.533587+0000","last_clean":"2023-11-09T03:46:52.533587+0000","last_became_active":"2023-11-09T03:46:51.544936+0000","last_became_peered":"2023-11-09T03:46:51.544936+0000","last_unstale":"2023-11-09T03:46:52.533587+0000","last_undegraded":"2023-11-09T03:46:52.533587+0000","last_fullsized":"2023-11-09T03:46:52.533587+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:21:24.253031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.552984+0000","last_change":"2023-11-09T03:46:52.549117+0000","last_active":"2023-11-09T03:46:52.552984+0000","last_peered":"2023-11-09T03:46:52.552984+0000","last_clean":"2023-11-09T03:46:52.552984+0000","last_became_active":"2023-11-09T03:46:52.548879+0000","last_became_peered":"2023-11-09T03:46:52.548879+0000","last_unstale":"2023-11-09T03:46:52.552984+0000","last_undegraded":"2023-11-09T03:46:52.552984+0000","last_fullsized":"2023-11-09T03:46:52.552984+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:56:45.361033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.529283+0000","last_change":"2023-11-09T03:46:51.544622+0000","last_active":"2023-11-09T03:46:52.529283+0000","last_peered":"2023-11-09T03:46:52.529283+0000","last_clean":"2023-11-09T03:46:52.529283+0000","last_became_active":"2023-11-09T03:46:51.544307+0000","last_became_peered":"2023-11-09T03:46:51.544307+0000","last_unstale":"2023-11-09T03:46:52.529283+0000","last_undegraded":"2023-11-09T03:46:52.529283+0000","last_fullsized":"2023-11-09T03:46:52.529283+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:18:33.568664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.558765+0000","last_change":"2023-11-09T03:46:52.551774+0000","last_active":"2023-11-09T03:46:52.558765+0000","last_peered":"2023-11-09T03:46:52.558765+0000","last_clean":"2023-11-09T03:46:52.558765+0000","last_became_active":"2023-11-09T03:46:52.551524+0000","last_became_peered":"2023-11-09T03:46:52.551524+0000","last_unstale":"2023-11-09T03:46:52.558765+0000","last_undegraded":"2023-11-09T03:46:52.558765+0000","last_fullsized":"2023-11-09T03:46:52.558765+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:45:42.507078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.536303+0000","last_change":"2023-11-09T03:46:51.542254+0000","last_active":"2023-11-09T03:46:52.536303+0000","last_peered":"2023-11-09T03:46:52.536303+0000","last_clean":"2023-11-09T03:46:52.536303+0000","last_became_active":"2023-11-09T03:46:51.541994+0000","last_became_peered":"2023-11-09T03:46:51.541994+0000","last_unstale":"2023-11-09T03:46:52.536303+0000","last_undegraded":"2023-11-09T03:46:52.536303+0000","last_fullsized":"2023-11-09T03:46:52.536303+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:30:38.798792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.551674+0000","last_change":"2023-11-09T03:46:52.547974+0000","last_active":"2023-11-09T03:46:52.551674+0000","last_peered":"2023-11-09T03:46:52.551674+0000","last_clean":"2023-11-09T03:46:52.551674+0000","last_became_active":"2023-11-09T03:46:52.547420+0000","last_became_peered":"2023-11-09T03:46:52.547420+0000","last_unstale":"2023-11-09T03:46:52.551674+0000","last_undegraded":"2023-11-09T03:46:52.551674+0000","last_fullsized":"2023-11-09T03:46:52.551674+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:56:46.571375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.531990+0000","last_change":"2023-11-09T03:46:51.540571+0000","last_active":"2023-11-09T03:46:52.531990+0000","last_peered":"2023-11-09T03:46:52.531990+0000","last_clean":"2023-11-09T03:46:52.531990+0000","last_became_active":"2023-11-09T03:46:51.540228+0000","last_became_peered":"2023-11-09T03:46:51.540228+0000","last_unstale":"2023-11-09T03:46:52.531990+0000","last_undegraded":"2023-11-09T03:46:52.531990+0000","last_fullsized":"2023-11-09T03:46:52.531990+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:31:41.783929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.548749+0000","last_change":"2023-11-09T03:46:52.542914+0000","last_active":"2023-11-09T03:46:52.548749+0000","last_peered":"2023-11-09T03:46:52.548749+0000","last_clean":"2023-11-09T03:46:52.548749+0000","last_became_active":"2023-11-09T03:46:52.542615+0000","last_became_peered":"2023-11-09T03:46:52.542615+0000","last_unstale":"2023-11-09T03:46:52.548749+0000","last_undegraded":"2023-11-09T03:46:52.548749+0000","last_fullsized":"2023-11-09T03:46:52.548749+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:51.516877+0000","last_clean_scrub_stamp":"2023-11-09T03:46:51.516877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:04:23.215294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-09T03:46:52.533327+0000","last_change":"2023-11-09T03:46:51.545962+0000","last_active":"2023-11-09T03:46:52.533327+0000","last_peered":"2023-11-09T03:46:52.533327+0000","last_clean":"2023-11-09T03:46:52.533327+0000","last_became_active":"2023-11-09T03:46:51.545711+0000","last_became_peered":"2023-11-09T03:46:51.545711+0000","last_unstale":"2023-11-09T03:46:52.533327+0000","last_undegraded":"2023-11-09T03:46:52.533327+0000","last_fullsized":"2023-11-09T03:46:52.533327+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:46:50.514859+0000","last_clean_scrub_stamp":"2023-11-09T03:46:50.514859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:29:02.367631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":179,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-09T03:46:51.540568+0000","last_change":"2023-11-09T03:45:21.030926+0000","last_active":"2023-11-09T03:46:51.540568+0000","last_peered":"2023-11-09T03:46:51.540568+0000","last_clean":"2023-11-09T03:46:51.540568+0000","last_became_active":"2023-11-09T03:45:21.029801+0000","last_became_peered":"2023-11-09T03:45:21.029801+0000","last_unstale":"2023-11-09T03:46:51.540568+0000","last_undegraded":"2023-11-09T03:46:51.540568+0000","last_fullsized":"2023-11-09T03:46:51.540568+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_clean_scrub_stamp":"2023-11-09T03:45:19.768433+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:12:59.153713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":769970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.60199999999999998}]},{"osd":1,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":4.9139999999999997,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.433}]},{"osd":3,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.503},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.52200000000000002}]},{"osd":4,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.56499999999999995}]},{"osd":5,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.54800000000000004}]},{"osd":6,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.61899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":187279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.73199999999999998}]},{"osd":1,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.753},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.60199999999999998}]},{"osd":2,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.77200000000000002}]},{"osd":3,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.78800000000000003}]},{"osd":4,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.68100000000000005}]},{"osd":7,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.65700000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":179602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.498}]},{"osd":1,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.60199999999999998}]},{"osd":2,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.51000000000000001}]},{"osd":3,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.70699999999999996}]},{"osd":4,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34000000000000002}]},{"osd":6,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.41999999999999998}]},{"osd":7,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.435}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":179602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.57499999999999996}]},{"osd":1,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.56399999999999995}]},{"osd":2,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.59399999999999997}]},{"osd":3,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.55100000000000005}]},{"osd":5,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.38100000000000001}]},{"osd":6,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.50700000000000001}]},{"osd":7,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.48599999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":179602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.224},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.36099999999999999}]},{"osd":1,"last update":"Thu Nov 9 03:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.34599999999999997}]},{"osd":2,"last update":"Thu Nov 9 03:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.29899999999999999}]},{"osd":4,"last update":"Thu Nov 9 03:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.48299999999999998}]},{"osd":5,"last update":"Thu Nov 9 03:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.504},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.42599999999999999}]},{"osd":6,"last update":"Thu Nov 9 03:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.44900000000000001}]},{"osd":7,"last update":"Thu Nov 9 03:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.55900000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":179602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.32300000000000001}]},{"osd":1,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.29199999999999998}]},{"osd":3,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.33600000000000002}]},{"osd":4,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.749}]},{"osd":5,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.58999999999999997}]},{"osd":7,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.73699999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":179602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.316,"15min":0.316},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"last":0.20799999999999999},{"interface":"front","average":{"1min":0.31,"5min":0.31,"15min":0.31},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.432,"5min":0.432,"15min":0.432},"last":0.26800000000000002}]},{"osd":2,"last update":"Thu Nov 9 03:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.308,"15min":0.308},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"last":0.252}]},{"osd":3,"last update":"Thu Nov 9 03:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.34300000000000003}]},{"osd":4,"last update":"Thu Nov 9 03:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Nov 9 03:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47599999999999998}]},{"osd":6,"last update":"Thu Nov 9 03:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.433}]},{"osd":7,"last update":"Thu Nov 9 03:46:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.503}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1015808,"data_stored":769351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.219}]},{"osd":2,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.254}]},{"osd":3,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.32000000000000001}]},{"osd":4,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.48499999999999999}]},{"osd":5,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.441},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.55400000000000005}]},{"osd":6,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.53300000000000003}]},{"osd":7,"last update":"Thu Nov 9 03:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.46500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-09T03:46:55.825 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-09T03:46:55.825 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-09T03:46:55.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.966+0000 7fd8aa46f640 1 Processor -- start 2023-11-09T03:46:55.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.966+0000 7fd8aa46f640 1 -- start start 2023-11-09T03:46:55.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.967+0000 7fd8aa46f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd8a40d0710 0x7fd8a40cd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:55.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.967+0000 7fd8aa46f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8a40cda50 0x7fd8a40b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:55.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.967+0000 7fd8aa46f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8a40d0280 con 0x7fd8a415e430 2023-11-09T03:46:55.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.967+0000 7fd8aa46f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8a40cf670 con 0x7fd8a40d0710 2023-11-09T03:46:55.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.967+0000 7fd8aa46f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a40cf950 con 0x7fd8a40cda50 2023-11-09T03:46:55.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.967+0000 7fd8a37fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd8a40d0710 0x7fd8a40cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:55.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.967+0000 7fd8a3fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd8a415e430 0x7fd8a40cec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59874/0 (socket says 172.21.15.76:59874) 2023-11-09T03:46:55.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.967+0000 7fd8a3fff640 1 -- 172.21.15.76:0/1273776562 learned_addr learned my addr 172.21.15.76:0/1273776562 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:55.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.968+0000 7fd8a37fe640 1 -- 172.21.15.76:0/1273776562 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8a40cda50 msgr2=0x7fd8a40b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:55.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.968+0000 7fd8a37fe640 1 --2- 172.21.15.76:0/1273776562 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8a40cda50 0x7fd8a40b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.968+0000 7fd8a37fe640 1 -- 172.21.15.76:0/1273776562 >> v1:172.21.15.76:6789/0 conn(0x7fd8a415e430 legacy=0x7fd8a40cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:55.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.968+0000 7fd8a2ffd640 1 -- 172.21.15.76:0/1273776562 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 107056283 0 0) 0x7fd8a40d0280 con 0x7fd8a415e430 2023-11-09T03:46:55.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.968+0000 7fd8a37fe640 1 -- 172.21.15.76:0/1273776562 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a40cfc30 con 0x7fd8a40d0710 2023-11-09T03:46:55.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.968+0000 7fd8a37fe640 1 --2- 172.21.15.76:0/1273776562 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8a40d0710 0x7fd8a40cd510 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd8980097a0 tx=0x7fd89802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d20b901ea4d15e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:55.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.969+0000 7fd8a2ffd640 1 -- 172.21.15.76:0/1273776562 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd898031830 con 0x7fd8a40d0710 2023-11-09T03:46:55.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.969+0000 7fd8a2ffd640 1 -- 172.21.15.76:0/1273776562 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd898031990 con 0x7fd8a40d0710 2023-11-09T03:46:55.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.969+0000 7fd8a2ffd640 1 -- 172.21.15.76:0/1273776562 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd898031d20 con 0x7fd8a40d0710 2023-11-09T03:46:55.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.969+0000 7fd8aa46f640 1 -- 172.21.15.76:0/1273776562 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8a40d0710 msgr2=0x7fd8a40cd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:55.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.969+0000 7fd8aa46f640 1 --2- 172.21.15.76:0/1273776562 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8a40d0710 0x7fd8a40cd510 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd8980097a0 tx=0x7fd89802b8d0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.969+0000 7fd8aa46f640 1 -- 172.21.15.76:0/1273776562 shutdown_connections 2023-11-09T03:46:55.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.969+0000 7fd8aa46f640 1 --2- 172.21.15.76:0/1273776562 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8a40cda50 0x7fd8a40b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.969+0000 7fd8aa46f640 1 --2- 172.21.15.76:0/1273776562 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8a40d0710 0x7fd8a40cd510 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.969+0000 7fd8aa46f640 1 -- 172.21.15.76:0/1273776562 >> 172.21.15.76:0/1273776562 conn(0x7fd8a406ea20 msgr2=0x7fd8a406ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:55.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.969+0000 7fd8aa46f640 1 -- 172.21.15.76:0/1273776562 shutdown_connections 2023-11-09T03:46:55.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.969+0000 7fd8aa46f640 1 -- 172.21.15.76:0/1273776562 wait complete. 2023-11-09T03:46:55.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.970+0000 7fd8aa46f640 1 Processor -- start 2023-11-09T03:46:55.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.970+0000 7fd8aa46f640 1 -- start start 2023-11-09T03:46:55.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.971+0000 7fd8aa46f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8a40cda50 0x7fd8a40b2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:55.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.971+0000 7fd8aa46f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8a40d0710 0x7fd8a40ca790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:55.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.971+0000 7fd8aa46f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8a415e430 0x7fd8a41e4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:55.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.971+0000 7fd8aa46f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a40b1e20 con 0x7fd8a415e430 2023-11-09T03:46:55.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.971+0000 7fd8aa46f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a41f1610 con 0x7fd8a40d0710 2023-11-09T03:46:55.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.971+0000 7fd8aa46f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a41f1330 con 0x7fd8a40cda50 2023-11-09T03:46:55.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.971+0000 7fd8a8a0c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8a415e430 0x7fd8a41e4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:55.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.971+0000 7fd8a8a0c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8a415e430 0x7fd8a41e4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41680/0 (socket says 172.21.15.76:41680) 2023-11-09T03:46:55.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.971+0000 7fd8a8a0c640 1 -- 172.21.15.76:0/526679960 learned_addr learned my addr 172.21.15.76:0/526679960 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:55.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.971+0000 7fd8a3fff640 1 --2- 172.21.15.76:0/526679960 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8a40cda50 0x7fd8a40b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:55.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.971+0000 7fd8a37fe640 1 --2- 172.21.15.76:0/526679960 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8a40d0710 0x7fd8a40ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:55.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.972+0000 7fd8a8a0c640 1 -- 172.21.15.76:0/526679960 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8a40cda50 msgr2=0x7fd8a40b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:55.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.972+0000 7fd8a8a0c640 1 --2- 172.21.15.76:0/526679960 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8a40cda50 0x7fd8a40b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.972+0000 7fd8a8a0c640 1 -- 172.21.15.76:0/526679960 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8a40d0710 msgr2=0x7fd8a40ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:55.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.972+0000 7fd8a8a0c640 1 --2- 172.21.15.76:0/526679960 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8a40d0710 0x7fd8a40ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:55.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.972+0000 7fd8a8a0c640 1 -- 172.21.15.76:0/526679960 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd898009400 con 0x7fd8a415e430 2023-11-09T03:46:55.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.972+0000 7fd8a8a0c640 1 --2- 172.21.15.76:0/526679960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8a415e430 0x7fd8a41e4b10 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00b970 tx=0x7fd89c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:55.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.972+0000 7fd8a0ff9640 1 -- 172.21.15.76:0/526679960 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c00cb40 con 0x7fd8a415e430 2023-11-09T03:46:55.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.972+0000 7fd8a0ff9640 1 -- 172.21.15.76:0/526679960 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89c00ccc0 con 0x7fd8a415e430 2023-11-09T03:46:55.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.972+0000 7fd8aa46f640 1 -- 172.21.15.76:0/526679960 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a40b1760 con 0x7fd8a415e430 2023-11-09T03:46:55.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.972+0000 7fd8a0ff9640 1 -- 172.21.15.76:0/526679960 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c014680 con 0x7fd8a415e430 2023-11-09T03:46:55.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.972+0000 7fd8aa46f640 1 -- 172.21.15.76:0/526679960 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a40cacd0 con 0x7fd8a415e430 2023-11-09T03:46:55.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.973+0000 7fd8aa46f640 1 -- 172.21.15.76:0/526679960 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8a40b1f60 con 0x7fd8a415e430 2023-11-09T03:46:55.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.973+0000 7fd8a0ff9640 1 -- 172.21.15.76:0/526679960 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd89c01d3b0 con 0x7fd8a415e430 2023-11-09T03:46:55.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.974+0000 7fd8a0ff9640 1 --2- 172.21.15.76:0/526679960 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd89006ae70 0x7fd89006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:55.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.974+0000 7fd8a0ff9640 1 -- 172.21.15.76:0/526679960 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd89c0895c0 con 0x7fd8a415e430 2023-11-09T03:46:55.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.974+0000 7fd8a3fff640 1 --2- 172.21.15.76:0/526679960 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd89006ae70 0x7fd89006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:55.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.975+0000 7fd8a3fff640 1 --2- 172.21.15.76:0/526679960 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd89006ae70 0x7fd89006d330 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd894000f60 tx=0x7fd894007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:55.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:55.978+0000 7fd8a0ff9640 1 -- 172.21.15.76:0/526679960 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd8a40b1f60 con 0x7fd8a415e430 2023-11-09T03:46:56.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.207+0000 7fd8aa46f640 1 -- 172.21.15.76:0/526679960 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd8a40b1f60 con 0x7fd8a415e430 2023-11-09T03:46:56.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.209+0000 7fd8a0ff9640 1 -- 172.21.15.76:0/526679960 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7fd8a40b1f60 con 0x7fd8a415e430 2023-11-09T03:46:56.209 INFO:teuthology.orchestra.run.smithi076.stdout:[client.0] 2023-11-09T03:46:56.209 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQChVUxlHj22CxAACNj5X24hdME+GNTLw4RahQ== 2023-11-09T03:46:56.209 INFO:teuthology.orchestra.run.smithi076.stdout: caps mds = "allow" 2023-11-09T03:46:56.210 INFO:teuthology.orchestra.run.smithi076.stdout: caps mon = "allow r" 2023-11-09T03:46:56.210 INFO:teuthology.orchestra.run.smithi076.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-09T03:46:56.210 INFO:teuthology.orchestra.run.smithi076.stderr:updated caps for client.0 2023-11-09T03:46:56.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.212+0000 7fd8aa46f640 1 -- 172.21.15.76:0/526679960 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd89006ae70 msgr2=0x7fd89006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:56.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.212+0000 7fd8aa46f640 1 --2- 172.21.15.76:0/526679960 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd89006ae70 0x7fd89006d330 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd894000f60 tx=0x7fd894007d50 comp rx=0 tx=0).stop 2023-11-09T03:46:56.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.212+0000 7fd8aa46f640 1 -- 172.21.15.76:0/526679960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8a415e430 msgr2=0x7fd8a41e4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:56.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.212+0000 7fd8aa46f640 1 --2- 172.21.15.76:0/526679960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8a415e430 0x7fd8a41e4b10 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00b970 tx=0x7fd89c00be30 comp rx=0 tx=0).stop 2023-11-09T03:46:56.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.212+0000 7fd8aa46f640 1 -- 172.21.15.76:0/526679960 shutdown_connections 2023-11-09T03:46:56.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.212+0000 7fd8aa46f640 1 --2- 172.21.15.76:0/526679960 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd89006ae70 0x7fd89006d330 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:56.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.212+0000 7fd8aa46f640 1 --2- 172.21.15.76:0/526679960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8a415e430 0x7fd8a41e4b10 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:56.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.212+0000 7fd8aa46f640 1 --2- 172.21.15.76:0/526679960 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8a40d0710 0x7fd8a40ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:56.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.212+0000 7fd8aa46f640 1 --2- 172.21.15.76:0/526679960 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8a40cda50 0x7fd8a40b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:56.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.212+0000 7fd8aa46f640 1 -- 172.21.15.76:0/526679960 >> 172.21.15.76:0/526679960 conn(0x7fd8a406ea20 msgr2=0x7fd8a415b990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:56.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.212+0000 7fd8aa46f640 1 -- 172.21.15.76:0/526679960 shutdown_connections 2023-11-09T03:46:56.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.212+0000 7fd8aa46f640 1 -- 172.21.15.76:0/526679960 wait complete. 2023-11-09T03:46:56.222 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:46:56.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.323+0000 7fa117fb6640 1 Processor -- start 2023-11-09T03:46:56.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.323+0000 7fa117fb6640 1 -- start start 2023-11-09T03:46:56.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.324+0000 7fa117fb6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa1100b0ae0 0x7fa1100ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:56.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.324+0000 7fa117fb6640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1100b4110 con 0x7fa11006db20 2023-11-09T03:46:56.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.324+0000 7fa117fb6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1100b43f0 con 0x7fa110159c70 2023-11-09T03:46:56.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.324+0000 7fa117fb6640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1100b49b0 con 0x7fa1100b0ae0 2023-11-09T03:46:56.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.324+0000 7fa115551640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa1100b0ae0 0x7fa1100ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:56.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.324+0000 7fa116553640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa11006db20 0x7fa11015d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59880/0 (socket says 172.21.15.76:59880) 2023-11-09T03:46:56.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.324+0000 7fa116553640 1 -- 172.21.15.76:0/1988979889 learned_addr learned my addr 172.21.15.76:0/1988979889 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:56.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.325+0000 7fa115551640 1 -- 172.21.15.76:0/1988979889 >> v1:172.21.15.113:6789/0 conn(0x7fa110159c70 legacy=0x7fa1100b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:56.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.325+0000 7fa115551640 1 -- 172.21.15.76:0/1988979889 >> v1:172.21.15.76:6789/0 conn(0x7fa11006db20 legacy=0x7fa11015d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:56.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.325+0000 7fa115551640 1 -- 172.21.15.76:0/1988979889 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1100b46d0 con 0x7fa1100b0ae0 2023-11-09T03:46:56.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.325+0000 7fa115551640 1 --2- 172.21.15.76:0/1988979889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1100b0ae0 0x7fa1100ce370 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa1000097a0 tx=0x7fa10002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783dadd651cac716 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:56.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.325+0000 7fa114d50640 1 -- 172.21.15.76:0/1988979889 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa100031830 con 0x7fa1100b0ae0 2023-11-09T03:46:56.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.326+0000 7fa114d50640 1 -- 172.21.15.76:0/1988979889 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa100031990 con 0x7fa1100b0ae0 2023-11-09T03:46:56.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.326+0000 7fa114d50640 1 -- 172.21.15.76:0/1988979889 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa100031d20 con 0x7fa1100b0ae0 2023-11-09T03:46:56.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.326+0000 7fa117fb6640 1 -- 172.21.15.76:0/1988979889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1100b0ae0 msgr2=0x7fa1100ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:56.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.326+0000 7fa117fb6640 1 --2- 172.21.15.76:0/1988979889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1100b0ae0 0x7fa1100ce370 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa1000097a0 tx=0x7fa10002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:46:56.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.326+0000 7fa117fb6640 1 -- 172.21.15.76:0/1988979889 shutdown_connections 2023-11-09T03:46:56.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.326+0000 7fa117fb6640 1 --2- 172.21.15.76:0/1988979889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1100b0ae0 0x7fa1100ce370 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:56.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.326+0000 7fa117fb6640 1 -- 172.21.15.76:0/1988979889 >> 172.21.15.76:0/1988979889 conn(0x7fa1100ce950 msgr2=0x7fa1100ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:56.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.326+0000 7fa117fb6640 1 -- 172.21.15.76:0/1988979889 shutdown_connections 2023-11-09T03:46:56.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.326+0000 7fa117fb6640 1 -- 172.21.15.76:0/1988979889 wait complete. 2023-11-09T03:46:56.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.327+0000 7fa117fb6640 1 Processor -- start 2023-11-09T03:46:56.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.327+0000 7fa117fb6640 1 -- start start 2023-11-09T03:46:56.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.328+0000 7fa117fb6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa11006db20 0x7fa1100afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:56.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.328+0000 7fa117fb6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1100b0ae0 0x7fa110156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:56.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.328+0000 7fa117fb6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa110159c70 0x7fa110157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:56.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.328+0000 7fa117fb6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1101e33d0 con 0x7fa11006db20 2023-11-09T03:46:56.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.328+0000 7fa117fb6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1101e36b0 con 0x7fa110159c70 2023-11-09T03:46:56.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.328+0000 7fa117fb6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1100af2c0 con 0x7fa1100b0ae0 2023-11-09T03:46:56.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.328+0000 7fa115d52640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa11006db20 0x7fa1100afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:56.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.328+0000 7fa115d52640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa11006db20 0x7fa1100afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41692/0 (socket says 172.21.15.76:41692) 2023-11-09T03:46:56.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.328+0000 7fa115d52640 1 -- 172.21.15.76:0/1193201790 learned_addr learned my addr 172.21.15.76:0/1193201790 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:56.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.328+0000 7fa115d52640 1 -- 172.21.15.76:0/1193201790 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1100b0ae0 msgr2=0x7fa110156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:56.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.328+0000 7fa115d52640 1 --2- 172.21.15.76:0/1193201790 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1100b0ae0 0x7fa110156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:56.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.328+0000 7fa115d52640 1 -- 172.21.15.76:0/1193201790 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa110159c70 msgr2=0x7fa110157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:56.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.328+0000 7fa116553640 1 --2- 172.21.15.76:0/1193201790 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa110159c70 0x7fa110157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:56.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.328+0000 7fa115d52640 1 --2- 172.21.15.76:0/1193201790 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa110159c70 0x7fa110157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:56.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.328+0000 7fa115d52640 1 -- 172.21.15.76:0/1193201790 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa100009400 con 0x7fa11006db20 2023-11-09T03:46:56.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.328+0000 7fa116553640 1 --2- 172.21.15.76:0/1193201790 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa110159c70 0x7fa110157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:56.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.329+0000 7fa115d52640 1 --2- 172.21.15.76:0/1193201790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa11006db20 0x7fa1100afac0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00ba50 tx=0x7fa10c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:56.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.329+0000 7fa1067fc640 1 -- 172.21.15.76:0/1193201790 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c00cbe0 con 0x7fa11006db20 2023-11-09T03:46:56.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.329+0000 7fa1067fc640 1 -- 172.21.15.76:0/1193201790 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10c00cd80 con 0x7fa11006db20 2023-11-09T03:46:56.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.329+0000 7fa1067fc640 1 -- 172.21.15.76:0/1193201790 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c014680 con 0x7fa11006db20 2023-11-09T03:46:56.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.329+0000 7fa117fb6640 1 -- 172.21.15.76:0/1193201790 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1100af5a0 con 0x7fa11006db20 2023-11-09T03:46:56.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.329+0000 7fa117fb6640 1 -- 172.21.15.76:0/1193201790 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1100b0000 con 0x7fa11006db20 2023-11-09T03:46:56.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.330+0000 7fa117fb6640 1 -- 172.21.15.76:0/1193201790 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1100af7e0 con 0x7fa11006db20 2023-11-09T03:46:56.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.330+0000 7fa1067fc640 1 -- 172.21.15.76:0/1193201790 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa10c0147e0 con 0x7fa11006db20 2023-11-09T03:46:56.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.330+0000 7fa1067fc640 1 --2- 172.21.15.76:0/1193201790 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa0f406ae70 0x7fa0f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:56.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.331+0000 7fa1067fc640 1 -- 172.21.15.76:0/1193201790 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa10c089560 con 0x7fa11006db20 2023-11-09T03:46:56.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.331+0000 7fa115551640 1 --2- 172.21.15.76:0/1193201790 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa0f406ae70 0x7fa0f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:56.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.331+0000 7fa115551640 1 --2- 172.21.15.76:0/1193201790 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa0f406ae70 0x7fa0f406d330 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa100036000 tx=0x7fa100030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:56.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.335+0000 7fa1067fc640 1 -- 172.21.15.76:0/1193201790 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa1100af7e0 con 0x7fa11006db20 2023-11-09T03:46:56.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.536+0000 7fa117fb6640 1 -- 172.21.15.76:0/1193201790 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1100af7e0 con 0x7fa11006db20 2023-11-09T03:46:56.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.537+0000 7fa1067fc640 1 -- 172.21.15.76:0/1193201790 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4910 (secure 0 0 0) 0x7fa1100af7e0 con 0x7fa11006db20 2023-11-09T03:46:56.536 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:46:56.536 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4880,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/711686619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":711686619},{"type":"v1","addr":"172.21.15.76:6835","nonce":711686619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/521006441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":521006441},{"type":"v1","addr":"172.21.15.113:6837","nonce":521006441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5212,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2652975857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2652975857},{"type":"v1","addr":"172.21.15.113:6836","nonce":2652975857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:46:51.874121+0000","modified":"2023-11-09T03:46:53.868634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5601":{"gid":5601,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6834/981490030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":981490030},{"type":"v1","addr":"172.21.15.76:6834","nonce":981490030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-09T03:46:56.537 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 16 2023-11-09T03:46:56.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.538+0000 7fa117fb6640 1 -- 172.21.15.76:0/1193201790 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa0f406ae70 msgr2=0x7fa0f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:56.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.538+0000 7fa117fb6640 1 --2- 172.21.15.76:0/1193201790 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa0f406ae70 0x7fa0f406d330 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa100036000 tx=0x7fa100030000 comp rx=0 tx=0).stop 2023-11-09T03:46:56.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.538+0000 7fa117fb6640 1 -- 172.21.15.76:0/1193201790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa11006db20 msgr2=0x7fa1100afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:56.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.538+0000 7fa117fb6640 1 --2- 172.21.15.76:0/1193201790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa11006db20 0x7fa1100afac0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00ba50 tx=0x7fa10c00bf10 comp rx=0 tx=0).stop 2023-11-09T03:46:56.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.539+0000 7fa117fb6640 1 -- 172.21.15.76:0/1193201790 shutdown_connections 2023-11-09T03:46:56.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.539+0000 7fa117fb6640 1 --2- 172.21.15.76:0/1193201790 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa0f406ae70 0x7fa0f406d330 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:56.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.539+0000 7fa117fb6640 1 --2- 172.21.15.76:0/1193201790 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa110159c70 0x7fa110157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:56.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.539+0000 7fa117fb6640 1 --2- 172.21.15.76:0/1193201790 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1100b0ae0 0x7fa110156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:56.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.539+0000 7fa117fb6640 1 --2- 172.21.15.76:0/1193201790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa11006db20 0x7fa1100afac0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:56.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.539+0000 7fa117fb6640 1 -- 172.21.15.76:0/1193201790 >> 172.21.15.76:0/1193201790 conn(0x7fa1100ce950 msgr2=0x7fa1100c79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:56.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.539+0000 7fa117fb6640 1 -- 172.21.15.76:0/1193201790 shutdown_connections 2023-11-09T03:46:56.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:56.539+0000 7fa117fb6640 1 -- 172.21.15.76:0/1193201790 wait complete. 2023-11-09T03:46:56.951 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-09T03:46:51.874121+0000', 'modified': '2023-11-09T03:46:53.868634+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5601}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5601': {'gid': 5601, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6834/981490030', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6832', 'nonce': 981490030}, {'type': 'v1', 'addr': '172.21.15.76:6834', 'nonce': 981490030}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-09T03:46:56.952 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-09T03:46:56.952 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-09T03:46:57.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.055+0000 7f89ba889640 1 Processor -- start 2023-11-09T03:46:57.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.056+0000 7f89ba889640 1 -- start start 2023-11-09T03:46:57.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.056+0000 7f89ba889640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f89b40b4880 0x7f89b40b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:57.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.056+0000 7f89ba889640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89b40cde60 con 0x7f89b40cd6d0 2023-11-09T03:46:57.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.056+0000 7f89ba889640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89b4160f90 con 0x7f89b4153e60 2023-11-09T03:46:57.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.056+0000 7f89ba889640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89b40b5200 con 0x7f89b40b4880 2023-11-09T03:46:57.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.056+0000 7f89b8e26640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f89b40b4880 0x7f89b40b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:57.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.056+0000 7f89b37fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f89b40cd6d0 0x7f89b4161420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59894/0 (socket says 172.21.15.76:59894) 2023-11-09T03:46:57.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.056+0000 7f89b37fe640 1 -- 172.21.15.76:0/1689875326 learned_addr learned my addr 172.21.15.76:0/1689875326 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:57.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.057+0000 7f89b8e26640 1 -- 172.21.15.76:0/1689875326 >> v1:172.21.15.113:6789/0 conn(0x7f89b4153e60 legacy=0x7f89b415dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:57.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.057+0000 7f89b8e26640 1 -- 172.21.15.76:0/1689875326 >> v1:172.21.15.76:6789/0 conn(0x7f89b40cd6d0 legacy=0x7f89b4161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:57.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.057+0000 7f89b8e26640 1 -- 172.21.15.76:0/1689875326 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b40ce090 con 0x7f89b40b4880 2023-11-09T03:46:57.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.057+0000 7f89b8e26640 1 --2- 172.21.15.76:0/1689875326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89b40b4880 0x7f89b40b4cc0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009d90 tx=0x7f89ac02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7117a223cb76d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:57.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.058+0000 7f89b2ffd640 1 -- 172.21.15.76:0/1689875326 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac02cc60 con 0x7f89b40b4880 2023-11-09T03:46:57.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.058+0000 7f89b2ffd640 1 -- 172.21.15.76:0/1689875326 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ac02cdc0 con 0x7f89b40b4880 2023-11-09T03:46:57.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.058+0000 7f89b2ffd640 1 -- 172.21.15.76:0/1689875326 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac034660 con 0x7f89b40b4880 2023-11-09T03:46:57.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.058+0000 7f89ba889640 1 -- 172.21.15.76:0/1689875326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89b40b4880 msgr2=0x7f89b40b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:57.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.058+0000 7f89ba889640 1 --2- 172.21.15.76:0/1689875326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89b40b4880 0x7f89b40b4cc0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009d90 tx=0x7f89ac02beb0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.058+0000 7f89ba889640 1 -- 172.21.15.76:0/1689875326 shutdown_connections 2023-11-09T03:46:57.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.058+0000 7f89ba889640 1 --2- 172.21.15.76:0/1689875326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89b40b4880 0x7f89b40b4cc0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.058+0000 7f89ba889640 1 -- 172.21.15.76:0/1689875326 >> 172.21.15.76:0/1689875326 conn(0x7f89b40cf880 msgr2=0x7f89b40cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:57.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.058+0000 7f89ba889640 1 -- 172.21.15.76:0/1689875326 shutdown_connections 2023-11-09T03:46:57.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.058+0000 7f89ba889640 1 -- 172.21.15.76:0/1689875326 wait complete. 2023-11-09T03:46:57.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.059+0000 7f89ba889640 1 Processor -- start 2023-11-09T03:46:57.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.059+0000 7f89ba889640 1 -- start start 2023-11-09T03:46:57.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.060+0000 7f89ba889640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89b40b4880 0x7f89b40c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:57.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.060+0000 7f89ba889640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89b40cd6d0 0x7f89b4156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:57.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.060+0000 7f89ba889640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f89b4153e60 0x7f89b4156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:57.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.060+0000 7f89ba889640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b41560e0 con 0x7f89b40cd6d0 2023-11-09T03:46:57.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.060+0000 7f89ba889640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b40c8e80 con 0x7f89b40b4880 2023-11-09T03:46:57.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.060+0000 7f89ba889640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b4155460 con 0x7f89b4153e60 2023-11-09T03:46:57.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.060+0000 7f89b37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89b40cd6d0 0x7f89b4156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:57.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.060+0000 7f89b37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89b40cd6d0 0x7f89b4156790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41702/0 (socket says 172.21.15.76:41702) 2023-11-09T03:46:57.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.060+0000 7f89b37fe640 1 -- 172.21.15.76:0/3127445696 learned_addr learned my addr 172.21.15.76:0/3127445696 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:57.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.060+0000 7f89b3fff640 1 --2- 172.21.15.76:0/3127445696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89b40b4880 0x7f89b40c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:57.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.060+0000 7f89b8e26640 1 --2- 172.21.15.76:0/3127445696 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f89b4153e60 0x7f89b4156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:57.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.061+0000 7f89b37fe640 1 -- 172.21.15.76:0/3127445696 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f89b4153e60 msgr2=0x7f89b4156cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:57.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.061+0000 7f89b37fe640 1 --2- 172.21.15.76:0/3127445696 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f89b4153e60 0x7f89b4156cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.061+0000 7f89b37fe640 1 -- 172.21.15.76:0/3127445696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89b40b4880 msgr2=0x7f89b40c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:57.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.061+0000 7f89b37fe640 1 --2- 172.21.15.76:0/3127445696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89b40b4880 0x7f89b40c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.061+0000 7f89b37fe640 1 -- 172.21.15.76:0/3127445696 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ac009a70 con 0x7f89b40cd6d0 2023-11-09T03:46:57.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.061+0000 7f89b8e26640 1 --2- 172.21.15.76:0/3127445696 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f89b4153e60 0x7f89b4156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:57.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.061+0000 7f89b37fe640 1 --2- 172.21.15.76:0/3127445696 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89b40cd6d0 0x7f89b4156790 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f89a800c500 tx=0x7f89a800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:57.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.061+0000 7f89b0ff9640 1 -- 172.21.15.76:0/3127445696 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a8005ce0 con 0x7f89b40cd6d0 2023-11-09T03:46:57.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.061+0000 7f89b0ff9640 1 -- 172.21.15.76:0/3127445696 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a8005e40 con 0x7f89b40cd6d0 2023-11-09T03:46:57.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.061+0000 7f89b0ff9640 1 -- 172.21.15.76:0/3127445696 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a80072f0 con 0x7f89b40cd6d0 2023-11-09T03:46:57.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.061+0000 7f89ba889640 1 -- 172.21.15.76:0/3127445696 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b4155740 con 0x7f89b40cd6d0 2023-11-09T03:46:57.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.061+0000 7f89ba889640 1 -- 172.21.15.76:0/3127445696 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b4155d00 con 0x7f89b40cd6d0 2023-11-09T03:46:57.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.062+0000 7f89b0ff9640 1 -- 172.21.15.76:0/3127445696 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f89a8007510 con 0x7f89b40cd6d0 2023-11-09T03:46:57.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.063+0000 7f89b0ff9640 1 --2- 172.21.15.76:0/3127445696 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f89a006ae70 0x7f89a006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:57.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.063+0000 7f89b0ff9640 1 -- 172.21.15.76:0/3127445696 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f89a8088b90 con 0x7f89b40cd6d0 2023-11-09T03:46:57.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.063+0000 7f89b3fff640 1 --2- 172.21.15.76:0/3127445696 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f89a006ae70 0x7f89a006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:57.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.064+0000 7f89b3fff640 1 --2- 172.21.15.76:0/3127445696 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f89a006ae70 0x7f89a006d330 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f89a4006fd0 tx=0x7f89a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:57.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.064+0000 7f89bd41d740 1 Processor -- start 2023-11-09T03:46:57.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.064+0000 7f89bd41d740 1 -- start start 2023-11-09T03:46:57.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.064+0000 7f89bd41d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a90522850 0x563a904f5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:57.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.064+0000 7f89bd41d740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a90522c30 0x563a905255e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:57.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.065+0000 7f89bd41d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a904f5c90 0x563a90525b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:57.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.065+0000 7f89bd41d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a90406740 con 0x563a90522850 2023-11-09T03:46:57.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.065+0000 7f89bd41d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a904688d0 con 0x563a904f5c90 2023-11-09T03:46:57.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.065+0000 7f89bd41d740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a90465fc0 con 0x563a90522c30 2023-11-09T03:46:57.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.065+0000 7f89b3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a90522850 0x563a904f5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:57.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.065+0000 7f89b3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a90522850 0x563a904f5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41714/0 (socket says 172.21.15.76:41714) 2023-11-09T03:46:57.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.065+0000 7f89b3fff640 1 -- 172.21.15.76:0/161918898 learned_addr learned my addr 172.21.15.76:0/161918898 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:57.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.065+0000 7f89b8e26640 1 --2- 172.21.15.76:0/161918898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a904f5c90 0x563a90525b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:57.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.065+0000 7f89b37fe640 1 --2- 172.21.15.76:0/161918898 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a90522c30 0x563a905255e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:57.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.065+0000 7f89b3fff640 1 -- 172.21.15.76:0/161918898 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a90522c30 msgr2=0x563a905255e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:57.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.065+0000 7f89b3fff640 1 --2- 172.21.15.76:0/161918898 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a90522c30 0x563a905255e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.065+0000 7f89b3fff640 1 -- 172.21.15.76:0/161918898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a904f5c90 msgr2=0x563a90525b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:57.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.065+0000 7f89b3fff640 1 --2- 172.21.15.76:0/161918898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a904f5c90 0x563a90525b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.065+0000 7f89b3fff640 1 -- 172.21.15.76:0/161918898 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a90424ca0 con 0x563a90522850 2023-11-09T03:46:57.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.065+0000 7f89b8e26640 1 --2- 172.21.15.76:0/161918898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a904f5c90 0x563a90525b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:57.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.065+0000 7f89b37fe640 1 --2- 172.21.15.76:0/161918898 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a90522c30 0x563a905255e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:46:57.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.065+0000 7f89b3fff640 1 --2- 172.21.15.76:0/161918898 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a90522850 0x563a904f5750 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f89a400d860 tx=0x7f89a4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e3cdddce35eea2 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:57.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.065+0000 7f8999ffb640 1 -- 172.21.15.76:0/161918898 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4004020 con 0x563a90522850 2023-11-09T03:46:57.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.065+0000 7f8999ffb640 1 -- 172.21.15.76:0/161918898 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a4004180 con 0x563a90522850 2023-11-09T03:46:57.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.065+0000 7f8999ffb640 1 -- 172.21.15.76:0/161918898 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a40149b0 con 0x563a90522850 2023-11-09T03:46:57.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.066+0000 7f89bd41d740 1 -- 172.21.15.76:0/161918898 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a90522850 msgr2=0x563a904f5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:57.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.066+0000 7f89bd41d740 1 --2- 172.21.15.76:0/161918898 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a90522850 0x563a904f5750 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f89a400d860 tx=0x7f89a4010000 comp rx=0 tx=0).stop 2023-11-09T03:46:57.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.066+0000 7f89bd41d740 1 -- 172.21.15.76:0/161918898 shutdown_connections 2023-11-09T03:46:57.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.066+0000 7f89bd41d740 1 --2- 172.21.15.76:0/161918898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a904f5c90 0x563a90525b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.066+0000 7f89bd41d740 1 --2- 172.21.15.76:0/161918898 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a90522c30 0x563a905255e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.066+0000 7f89bd41d740 1 --2- 172.21.15.76:0/161918898 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a90522850 0x563a904f5750 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.066+0000 7f89bd41d740 1 -- 172.21.15.76:0/161918898 >> 172.21.15.76:0/161918898 conn(0x563a904f7a60 msgr2=0x563a905405e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:57.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.066+0000 7f89bd41d740 1 -- 172.21.15.76:0/161918898 shutdown_connections 2023-11-09T03:46:57.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.066+0000 7f89bd41d740 1 -- 172.21.15.76:0/161918898 wait complete. 2023-11-09T03:46:57.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.066+0000 7f89bd41d740 1 -- start start 2023-11-09T03:46:57.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.067+0000 7f89bd41d740 1 Processor -- start 2023-11-09T03:46:57.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.067+0000 7f89bd41d740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a90522850 0x563a905235c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:57.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.067+0000 7f89bd41d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a90522c30 0x563a90537100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:57.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.067+0000 7f89bd41d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a90537640 0x563a9050cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:57.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.067+0000 7f89bd41d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a904618b0 con 0x563a90537640 2023-11-09T03:46:57.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.067+0000 7f89bd41d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a9041bc80 con 0x563a90522c30 2023-11-09T03:46:57.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.067+0000 7f89bd41d740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a9041b920 con 0x563a90522850 2023-11-09T03:46:57.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.067+0000 7f89bd41d740 10 client.0 _ll_register_callbacks cb 0x563a90173ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:46:57.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.067+0000 7f89b8e26640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a90537640 0x563a9050cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:57.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.067+0000 7f89b8e26640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a90537640 0x563a9050cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41716/0 (socket says 172.21.15.76:41716) 2023-11-09T03:46:57.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.067+0000 7f89b8e26640 1 -- 172.21.15.76:0/1384290377 learned_addr learned my addr 172.21.15.76:0/1384290377 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:57.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.068+0000 7f89b37fe640 1 --2- 172.21.15.76:0/1384290377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a90522c30 0x563a90537100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:57.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.068+0000 7f89b3fff640 1 --2- 172.21.15.76:0/1384290377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a90522850 0x563a905235c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:57.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.068+0000 7f89b8e26640 1 -- 172.21.15.76:0/1384290377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a90522850 msgr2=0x563a905235c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:57.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.068+0000 7f89b8e26640 1 --2- 172.21.15.76:0/1384290377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a90522850 0x563a905235c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.068+0000 7f89b8e26640 1 -- 172.21.15.76:0/1384290377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a90522c30 msgr2=0x563a90537100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:57.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.068+0000 7f89b8e26640 1 --2- 172.21.15.76:0/1384290377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a90522c30 0x563a90537100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.068+0000 7f89b8e26640 1 -- 172.21.15.76:0/1384290377 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a400e040 con 0x563a90537640 2023-11-09T03:46:57.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.068+0000 7f89b8e26640 1 --2- 172.21.15.76:0/1384290377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a90537640 0x563a9050cf30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f89ac037040 tx=0x7f89ac004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b3b37812caa9af server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:57.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.068+0000 7f8992ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:46:57.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.068+0000 7f8992ffd640 1 -- 172.21.15.76:0/1384290377 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac034ea0 con 0x563a90537640 2023-11-09T03:46:57.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.068+0000 7f8992ffd640 1 -- 172.21.15.76:0/1384290377 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f89ac045070 con 0x563a90537640 2023-11-09T03:46:57.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.068+0000 7f89bd41d740 1 -- 172.21.15.76:0/1384290377 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a9040dbf0 con 0x563a90537640 2023-11-09T03:46:57.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.068+0000 7f8992ffd640 1 -- 172.21.15.76:0/1384290377 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89ac04a480 con 0x563a90537640 2023-11-09T03:46:57.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.068+0000 7f89bd41d740 10 client.5796 fetch_fsmap learned FSMap version 16 2023-11-09T03:46:57.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.068+0000 7f89bd41d740 1 -- 172.21.15.76:0/1384290377 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x563a9045ff30 con 0x563a90537640 2023-11-09T03:46:57.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.068+0000 7f8992ffd640 1 -- 172.21.15.76:0/1384290377 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563a9040dbf0 con 0x563a90537640 2023-11-09T03:46:57.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.068+0000 7f8992ffd640 1 -- 172.21.15.76:0/1384290377 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f89ac03f450 con 0x563a90537640 2023-11-09T03:46:57.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.068+0000 7f89bd41d740 10 client.5796 fetch_fsmap finished waiting for FSMap version 16 2023-11-09T03:46:57.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.068+0000 7f89bd41d740 10 client.5796 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-09T03:46:57.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.068+0000 7f89bd41d740 20 client.5796 populate_metadata read hostname 'smithi076' 2023-11-09T03:46:57.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.068+0000 7f89bd41d740 1 --2- 172.21.15.76:0/1384290377 >> [v2:172.21.15.76:6832/981490030,v1:172.21.15.76:6834/981490030] conn(0x563a905504c0 0x563a905508a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:57.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.068+0000 7f89bd41d740 4 client.5796 mds_command: new command op to 5601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:46:57.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.068+0000 7f89bd41d740 1 -- 172.21.15.76:0/1384290377 --> [v2:172.21.15.76:6832/981490030,v1:172.21.15.76:6834/981490030] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a9045a890 con 0x563a905504c0 2023-11-09T03:46:57.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.069+0000 7f89b3fff640 1 --2- 172.21.15.76:0/1384290377 >> [v2:172.21.15.76:6832/981490030,v1:172.21.15.76:6834/981490030] conn(0x563a905504c0 0x563a905508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:57.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.069+0000 7f8992ffd640 10 client.5796 ms_handle_connect on v2:172.21.15.76:6832/981490030 2023-11-09T03:46:57.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.069+0000 7f89b3fff640 1 --2- 172.21.15.76:0/1384290377 >> [v2:172.21.15.76:6832/981490030,v1:172.21.15.76:6834/981490030] conn(0x563a905504c0 0x563a905508a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ae04f630d77ff6e server_cookie=c9ae214e67642b8e in_seq=0 out_seq=0 2023-11-09T03:46:57.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.069+0000 7f8992ffd640 1 -- 172.21.15.76:0/1384290377 <== mds.0 v2:172.21.15.76:6832/981490030 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563a9045a890 con 0x563a905504c0 2023-11-09T03:46:57.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.069+0000 7f8992ffd640 10 client.5796 handle_command_reply: tid=0 2023-11-09T03:46:57.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.069+0000 7f89bd41d740 1 client.5796 shutdown 2023-11-09T03:46:57.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.070+0000 7f89bd41d740 1 -- 172.21.15.76:0/1384290377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a90537640 msgr2=0x563a9050cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:57.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.070+0000 7f89bd41d740 1 --2- 172.21.15.76:0/1384290377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a90537640 0x563a9050cf30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f89ac037040 tx=0x7f89ac004ab0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.070+0000 7f89bd41d740 1 -- 172.21.15.76:0/1384290377 shutdown_connections 2023-11-09T03:46:57.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.070+0000 7f89bd41d740 1 --2- 172.21.15.76:0/1384290377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a90537640 0x563a9050cf30 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.070+0000 7f89bd41d740 1 --2- 172.21.15.76:0/1384290377 >> [v2:172.21.15.76:6832/981490030,v1:172.21.15.76:6834/981490030] conn(0x563a905504c0 0x563a905508a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.070+0000 7f89bd41d740 1 --2- 172.21.15.76:0/1384290377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a90522c30 0x563a90537100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.070+0000 7f89bd41d740 1 --2- 172.21.15.76:0/1384290377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a90522850 0x563a905235c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.070+0000 7f8992ffd640 0 client.5796 ms_handle_reset on v2:172.21.15.76:6832/981490030 2023-11-09T03:46:57.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.070+0000 7f89bd41d740 1 -- 172.21.15.76:0/1384290377 >> 172.21.15.76:0/1384290377 conn(0x563a904f5c90 msgr2=0x563a904f9600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:57.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.070+0000 7f89bd41d740 1 -- 172.21.15.76:0/1384290377 shutdown_connections 2023-11-09T03:46:57.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.070+0000 7f89bd41d740 1 -- 172.21.15.76:0/1384290377 wait complete. 2023-11-09T03:46:57.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.070+0000 7f89bd41d740 20 client.5796 trim_cache size 0 max 16384 2023-11-09T03:46:57.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.085+0000 7f89bd41d740 1 Processor -- start 2023-11-09T03:46:57.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.085+0000 7f89bd41d740 1 -- start start 2023-11-09T03:46:57.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.086+0000 7f89bd41d740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a90522850 0x563a90543480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:57.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.086+0000 7f89bd41d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a90522c30 0x563a90537d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:57.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.086+0000 7f89bd41d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a90537640 0x563a905382a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:57.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.086+0000 7f89bd41d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a903e1280 con 0x563a90522c30 2023-11-09T03:46:57.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.086+0000 7f89bd41d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a904f4fe0 con 0x563a90537640 2023-11-09T03:46:57.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.086+0000 7f89bd41d740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a904fbec0 con 0x563a90522850 2023-11-09T03:46:57.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.086+0000 7f89b37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a90522c30 0x563a90537d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:57.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.086+0000 7f89b37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a90522c30 0x563a90537d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41724/0 (socket says 172.21.15.76:41724) 2023-11-09T03:46:57.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.086+0000 7f89b37fe640 1 -- 172.21.15.76:0/2271507373 learned_addr learned my addr 172.21.15.76:0/2271507373 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:57.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.086+0000 7f89b37fe640 1 -- 172.21.15.76:0/2271507373 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a90522850 msgr2=0x563a90543480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:57.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.086+0000 7f89b37fe640 1 --2- 172.21.15.76:0/2271507373 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a90522850 0x563a90543480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.086+0000 7f89b37fe640 1 -- 172.21.15.76:0/2271507373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a90537640 msgr2=0x563a905382a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:57.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.086+0000 7f89b37fe640 1 --2- 172.21.15.76:0/2271507373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a90537640 0x563a905382a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.086+0000 7f89b37fe640 1 -- 172.21.15.76:0/2271507373 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a903fdd10 con 0x563a90522c30 2023-11-09T03:46:57.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.086+0000 7f89b37fe640 1 --2- 172.21.15.76:0/2271507373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a90522c30 0x563a90537d60 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f89a8087010 tx=0x7f89a8057550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb5f4a126c9787e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:57.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.086+0000 7f899a7fc640 1 -- 172.21.15.76:0/2271507373 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a8055ca0 con 0x563a90522c30 2023-11-09T03:46:57.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.086+0000 7f899a7fc640 1 -- 172.21.15.76:0/2271507373 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a8054010 con 0x563a90522c30 2023-11-09T03:46:57.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.087+0000 7f899a7fc640 1 -- 172.21.15.76:0/2271507373 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a8057d60 con 0x563a90522c30 2023-11-09T03:46:57.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.087+0000 7f89bd41d740 1 -- 172.21.15.76:0/2271507373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a90522c30 msgr2=0x563a90537d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:57.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.087+0000 7f89bd41d740 1 --2- 172.21.15.76:0/2271507373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a90522c30 0x563a90537d60 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f89a8087010 tx=0x7f89a8057550 comp rx=0 tx=0).stop 2023-11-09T03:46:57.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.087+0000 7f89bd41d740 1 -- 172.21.15.76:0/2271507373 shutdown_connections 2023-11-09T03:46:57.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.087+0000 7f89bd41d740 1 --2- 172.21.15.76:0/2271507373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a90537640 0x563a905382a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.087+0000 7f89bd41d740 1 --2- 172.21.15.76:0/2271507373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a90522c30 0x563a90537d60 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.087+0000 7f89bd41d740 1 --2- 172.21.15.76:0/2271507373 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a90522850 0x563a90543480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.087+0000 7f89bd41d740 1 -- 172.21.15.76:0/2271507373 >> 172.21.15.76:0/2271507373 conn(0x563a904f5c90 msgr2=0x563a904f0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:57.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.087+0000 7f89bd41d740 1 -- 172.21.15.76:0/2271507373 shutdown_connections 2023-11-09T03:46:57.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.087+0000 7f89bd41d740 1 -- 172.21.15.76:0/2271507373 wait complete. 2023-11-09T03:46:57.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.088+0000 7f89bd41d740 1 -- start start 2023-11-09T03:46:57.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.088+0000 7f89bd41d740 1 Processor -- start 2023-11-09T03:46:57.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.088+0000 7f89bd41d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a90522850 0x563a905aab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:57.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.088+0000 7f89bd41d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a90537640 0x563a9054f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:57.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.088+0000 7f89bd41d740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a90537d60 0x563a905381e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:57.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.088+0000 7f89bd41d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a90529660 con 0x563a90522850 2023-11-09T03:46:57.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.088+0000 7f89bd41d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a90538720 con 0x563a90537640 2023-11-09T03:46:57.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.088+0000 7f89bd41d740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a904f63c0 con 0x563a90537d60 2023-11-09T03:46:57.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.088+0000 7f89bd41d740 10 client.0 _ll_register_callbacks cb 0x7f89a400bd80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:46:57.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.089+0000 7f89b3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a90522850 0x563a905aab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:57.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.089+0000 7f89b3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a90522850 0x563a905aab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41730/0 (socket says 172.21.15.76:41730) 2023-11-09T03:46:57.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.089+0000 7f89b3fff640 1 -- 172.21.15.76:0/4150759215 learned_addr learned my addr 172.21.15.76:0/4150759215 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:57.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.089+0000 7f89b8e26640 1 --2- 172.21.15.76:0/4150759215 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a90537d60 0x563a905381e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:57.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.089+0000 7f89b37fe640 1 --2- 172.21.15.76:0/4150759215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a90537640 0x563a9054f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:57.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.089+0000 7f89b3fff640 1 -- 172.21.15.76:0/4150759215 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a90537d60 msgr2=0x563a905381e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:57.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.089+0000 7f89b3fff640 1 --2- 172.21.15.76:0/4150759215 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a90537d60 0x563a905381e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.089+0000 7f89b3fff640 1 -- 172.21.15.76:0/4150759215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a90537640 msgr2=0x563a9054f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:57.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.089+0000 7f89b3fff640 1 --2- 172.21.15.76:0/4150759215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a90537640 0x563a9054f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.089+0000 7f89b3fff640 1 -- 172.21.15.76:0/4150759215 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b4155d00 con 0x563a90522850 2023-11-09T03:46:57.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.089+0000 7f89b3fff640 1 --2- 172.21.15.76:0/4150759215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a90522850 0x563a905aab60 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f89a400d530 tx=0x7f89a40046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e63a4f30636ca89 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:57.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.089+0000 7f8992ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:46:57.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.089+0000 7f8992ffd640 1 -- 172.21.15.76:0/4150759215 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a400d6c0 con 0x563a90522850 2023-11-09T03:46:57.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.089+0000 7f8992ffd640 1 -- 172.21.15.76:0/4150759215 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f89a4014930 con 0x563a90522850 2023-11-09T03:46:57.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.089+0000 7f89bd41d740 1 -- 172.21.15.76:0/4150759215 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89a4004180 con 0x563a90522850 2023-11-09T03:46:57.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.089+0000 7f8992ffd640 1 -- 172.21.15.76:0/4150759215 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89a4014b10 con 0x563a90522850 2023-11-09T03:46:57.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.089+0000 7f89bd41d740 10 client.5802 fetch_fsmap learned FSMap version 16 2023-11-09T03:46:57.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.089+0000 7f89bd41d740 1 -- 172.21.15.76:0/4150759215 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x563a90423500 con 0x563a90522850 2023-11-09T03:46:57.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.089+0000 7f8992ffd640 1 -- 172.21.15.76:0/4150759215 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a4004180 con 0x563a90522850 2023-11-09T03:46:57.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.090+0000 7f8992ffd640 1 -- 172.21.15.76:0/4150759215 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f89a4014c70 con 0x563a90522850 2023-11-09T03:46:57.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.090+0000 7f89bd41d740 10 client.5802 fetch_fsmap finished waiting for FSMap version 16 2023-11-09T03:46:57.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.090+0000 7f89bd41d740 10 client.5802 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-09T03:46:57.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.090+0000 7f89bd41d740 20 client.5802 populate_metadata read hostname 'smithi076' 2023-11-09T03:46:57.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.090+0000 7f89bd41d740 1 --2- 172.21.15.76:0/4150759215 >> [v2:172.21.15.76:6832/981490030,v1:172.21.15.76:6834/981490030] conn(0x563a90553300 0x563a904fa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:57.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.090+0000 7f89bd41d740 4 client.5802 mds_command: new command op to 5601 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-09T03:46:57.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.090+0000 7f89bd41d740 1 -- 172.21.15.76:0/4150759215 --> [v2:172.21.15.76:6832/981490030,v1:172.21.15.76:6834/981490030] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563a9045a5b0 con 0x563a90553300 2023-11-09T03:46:57.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.090+0000 7f89b37fe640 1 --2- 172.21.15.76:0/4150759215 >> [v2:172.21.15.76:6832/981490030,v1:172.21.15.76:6834/981490030] conn(0x563a90553300 0x563a904fa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:57.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.090+0000 7f8992ffd640 10 client.5802 ms_handle_connect on v2:172.21.15.76:6832/981490030 2023-11-09T03:46:57.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.090+0000 7f89b37fe640 1 --2- 172.21.15.76:0/4150759215 >> [v2:172.21.15.76:6832/981490030,v1:172.21.15.76:6834/981490030] conn(0x563a90553300 0x563a904fa4f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91cdb52d692f08c1 server_cookie=5eadf465e25a3f33 in_seq=0 out_seq=0 2023-11-09T03:46:57.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.090+0000 7f8992ffd640 1 -- 172.21.15.76:0/4150759215 <== mds.0 v2:172.21.15.76:6832/981490030 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x563a9045a5b0 con 0x563a90553300 2023-11-09T03:46:57.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.090+0000 7f8992ffd640 10 client.5802 handle_command_reply: tid=0 2023-11-09T03:46:57.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.090+0000 7f89bd41d740 1 client.5802 shutdown 2023-11-09T03:46:57.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.090+0000 7f89bd41d740 1 -- 172.21.15.76:0/4150759215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a90522850 msgr2=0x563a905aab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:57.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.090+0000 7f89bd41d740 1 --2- 172.21.15.76:0/4150759215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a90522850 0x563a905aab60 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f89a400d530 tx=0x7f89a40046f0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.090+0000 7f89bd41d740 1 -- 172.21.15.76:0/4150759215 shutdown_connections 2023-11-09T03:46:57.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.090+0000 7f89bd41d740 1 --2- 172.21.15.76:0/4150759215 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a90537d60 0x563a905381e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.090+0000 7f89bd41d740 1 --2- 172.21.15.76:0/4150759215 >> [v2:172.21.15.76:6832/981490030,v1:172.21.15.76:6834/981490030] conn(0x563a90553300 0x563a904fa4f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.090+0000 7f89bd41d740 1 --2- 172.21.15.76:0/4150759215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a90537640 0x563a9054f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.090+0000 7f8992ffd640 0 client.5802 ms_handle_reset on v2:172.21.15.76:6832/981490030 2023-11-09T03:46:57.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.090+0000 7f89bd41d740 1 --2- 172.21.15.76:0/4150759215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563a90522850 0x563a905aab60 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.090+0000 7f89bd41d740 1 -- 172.21.15.76:0/4150759215 >> 172.21.15.76:0/4150759215 conn(0x563a904f5c90 msgr2=0x563a90525220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:57.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.090+0000 7f89bd41d740 1 -- 172.21.15.76:0/4150759215 shutdown_connections 2023-11-09T03:46:57.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.090+0000 7f89bd41d740 1 -- 172.21.15.76:0/4150759215 wait complete. 2023-11-09T03:46:57.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.091+0000 7f89bd41d740 20 client.5802 trim_cache size 0 max 16384 2023-11-09T03:46:57.111 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:46:57.111 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "c213b26f-d0da-45d5-8e34-bb75c096d4f2", 2023-11-09T03:46:57.112 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2023-11-09T03:46:57.112 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 5601, 2023-11-09T03:46:57.112 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2023-11-09T03:46:57.112 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2023-11-09T03:46:57.112 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2023-11-09T03:46:57.113 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 4.5651092459999996, 2023-11-09T03:46:57.113 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 16, 2023-11-09T03:46:57.113 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 39, 2023-11-09T03:46:57.113 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 39, 2023-11-09T03:46:57.113 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 11.620972841 2023-11-09T03:46:57.113 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:46:57.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.091+0000 7f89ba889640 1 -- 172.21.15.76:0/3127445696 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f89a006ae70 msgr2=0x7f89a006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:57.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.091+0000 7f89ba889640 1 --2- 172.21.15.76:0/3127445696 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f89a006ae70 0x7f89a006d330 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f89a4006fd0 tx=0x7f89a4008040 comp rx=0 tx=0).stop 2023-11-09T03:46:57.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.091+0000 7f89ba889640 1 -- 172.21.15.76:0/3127445696 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89b40cd6d0 msgr2=0x7f89b4156790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:57.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.091+0000 7f89ba889640 1 --2- 172.21.15.76:0/3127445696 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89b40cd6d0 0x7f89b4156790 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f89a800c500 tx=0x7f89a800c9c0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.091+0000 7f89ba889640 1 -- 172.21.15.76:0/3127445696 shutdown_connections 2023-11-09T03:46:57.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.091+0000 7f89ba889640 1 --2- 172.21.15.76:0/3127445696 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f89a006ae70 0x7f89a006d330 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.091+0000 7f89ba889640 1 --2- 172.21.15.76:0/3127445696 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f89b4153e60 0x7f89b4156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.091+0000 7f89ba889640 1 --2- 172.21.15.76:0/3127445696 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89b40cd6d0 0x7f89b4156790 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.091+0000 7f89ba889640 1 --2- 172.21.15.76:0/3127445696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f89b40b4880 0x7f89b40c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.091+0000 7f89ba889640 1 -- 172.21.15.76:0/3127445696 >> 172.21.15.76:0/3127445696 conn(0x7f89b40cf880 msgr2=0x7f89b415afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:57.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.091+0000 7f89ba889640 1 -- 172.21.15.76:0/3127445696 shutdown_connections 2023-11-09T03:46:57.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.091+0000 7f89ba889640 1 -- 172.21.15.76:0/3127445696 wait complete. 2023-11-09T03:46:57.117 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-09T03:46:57.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.218+0000 7f2c1e49b640 1 Processor -- start 2023-11-09T03:46:57.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.218+0000 7f2c1e49b640 1 -- start start 2023-11-09T03:46:57.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.219+0000 7f2c1e49b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2c18159c70 0x7f2c180ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:57.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.219+0000 7f2c1e49b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2c180b0ae0 0x7f2c1815d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:57.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.219+0000 7f2c1e49b640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c180b4110 con 0x7f2c1806db20 2023-11-09T03:46:57.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.219+0000 7f2c1e49b640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c180b49b0 con 0x7f2c180b0ae0 2023-11-09T03:46:57.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.219+0000 7f2c1e49b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c180b43f0 con 0x7f2c18159c70 2023-11-09T03:46:57.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.219+0000 7f2c1ca38640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2c180b0ae0 0x7f2c1815d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:57.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.219+0000 7f2c177fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2c1806db20 0x7f2c180b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59906/0 (socket says 172.21.15.76:59906) 2023-11-09T03:46:57.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.219+0000 7f2c177fe640 1 -- 172.21.15.76:0/4003776104 learned_addr learned my addr 172.21.15.76:0/4003776104 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:57.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.219+0000 7f2c17fff640 1 --2- 172.21.15.76:0/4003776104 >> v2:172.21.15.113:3300/0 conn(0x7f2c18159c70 0x7f2c180ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:57.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.219+0000 7f2c1ca38640 1 -- 172.21.15.76:0/4003776104 >> v2:172.21.15.113:3300/0 conn(0x7f2c18159c70 msgr2=0x7f2c180ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:57.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.219+0000 7f2c1ca38640 1 --2- 172.21.15.76:0/4003776104 >> v2:172.21.15.113:3300/0 conn(0x7f2c18159c70 0x7f2c180ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.219+0000 7f2c1ca38640 1 -- 172.21.15.76:0/4003776104 >> v1:172.21.15.76:6789/0 conn(0x7f2c1806db20 legacy=0x7f2c180b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:57.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.219+0000 7f2c1ca38640 1 -- 172.21.15.76:0/4003776104 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c180b46d0 con 0x7f2c180b0ae0 2023-11-09T03:46:57.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.219+0000 7f2c16ffd640 1 -- 172.21.15.76:0/4003776104 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3659757635 0 0) 0x7f2c180b4110 con 0x7f2c1806db20 2023-11-09T03:46:57.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.220+0000 7f2c1ca38640 1 --2- 172.21.15.76:0/4003776104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c180b0ae0 0x7f2c1815d7d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2c180cc440 tx=0x7f2c1002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6197c0f15d9ac503 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:57.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.220+0000 7f2c16ffd640 1 -- 172.21.15.76:0/4003776104 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1002cc80 con 0x7f2c180b0ae0 2023-11-09T03:46:57.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.220+0000 7f2c16ffd640 1 -- 172.21.15.76:0/4003776104 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c1002cde0 con 0x7f2c180b0ae0 2023-11-09T03:46:57.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.220+0000 7f2c16ffd640 1 -- 172.21.15.76:0/4003776104 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c10034660 con 0x7f2c180b0ae0 2023-11-09T03:46:57.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.221+0000 7f2c1e49b640 1 -- 172.21.15.76:0/4003776104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c180b0ae0 msgr2=0x7f2c1815d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:57.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.221+0000 7f2c1e49b640 1 --2- 172.21.15.76:0/4003776104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c180b0ae0 0x7f2c1815d7d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2c180cc440 tx=0x7f2c1002bed0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.221+0000 7f2c1e49b640 1 -- 172.21.15.76:0/4003776104 shutdown_connections 2023-11-09T03:46:57.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.221+0000 7f2c1e49b640 1 --2- 172.21.15.76:0/4003776104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c180b0ae0 0x7f2c1815d7d0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.221+0000 7f2c1e49b640 1 --2- 172.21.15.76:0/4003776104 >> v2:172.21.15.113:3300/0 conn(0x7f2c18159c70 0x7f2c180ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.221+0000 7f2c1e49b640 1 -- 172.21.15.76:0/4003776104 >> 172.21.15.76:0/4003776104 conn(0x7f2c180ce950 msgr2=0x7f2c180ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:57.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.221+0000 7f2c1e49b640 1 -- 172.21.15.76:0/4003776104 shutdown_connections 2023-11-09T03:46:57.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.221+0000 7f2c1e49b640 1 -- 172.21.15.76:0/4003776104 wait complete. 2023-11-09T03:46:57.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.222+0000 7f2c1e49b640 1 Processor -- start 2023-11-09T03:46:57.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.222+0000 7f2c1e49b640 1 -- start start 2023-11-09T03:46:57.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.223+0000 7f2c1e49b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c1806db20 0x7f2c18158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:57.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.223+0000 7f2c1e49b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c180b0ae0 0x7f2c18150b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:57.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.223+0000 7f2c1e49b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c18159c70 0x7f2c181510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:57.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.223+0000 7f2c1e49b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c180c5d10 con 0x7f2c18159c70 2023-11-09T03:46:57.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.223+0000 7f2c1e49b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c180c5f60 con 0x7f2c1806db20 2023-11-09T03:46:57.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.223+0000 7f2c1e49b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c180c6240 con 0x7f2c180b0ae0 2023-11-09T03:46:57.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.223+0000 7f2c1ca38640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c18159c70 0x7f2c181510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:57.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.223+0000 7f2c1ca38640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c18159c70 0x7f2c181510b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41742/0 (socket says 172.21.15.76:41742) 2023-11-09T03:46:57.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.223+0000 7f2c1ca38640 1 -- 172.21.15.76:0/1755064410 learned_addr learned my addr 172.21.15.76:0/1755064410 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:57.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.223+0000 7f2c17fff640 1 --2- 172.21.15.76:0/1755064410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c1806db20 0x7f2c18158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:57.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.223+0000 7f2c177fe640 1 --2- 172.21.15.76:0/1755064410 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c180b0ae0 0x7f2c18150b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:57.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.223+0000 7f2c1ca38640 1 -- 172.21.15.76:0/1755064410 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c180b0ae0 msgr2=0x7f2c18150b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:57.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.223+0000 7f2c1ca38640 1 --2- 172.21.15.76:0/1755064410 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c180b0ae0 0x7f2c18150b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.223+0000 7f2c1ca38640 1 -- 172.21.15.76:0/1755064410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c1806db20 msgr2=0x7f2c18158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:57.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.223+0000 7f2c1ca38640 1 --2- 172.21.15.76:0/1755064410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c1806db20 0x7f2c18158940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.223+0000 7f2c1ca38640 1 -- 172.21.15.76:0/1755064410 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c10009a70 con 0x7f2c18159c70 2023-11-09T03:46:57.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.223+0000 7f2c17fff640 1 --2- 172.21.15.76:0/1755064410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c1806db20 0x7f2c18158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:57.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.223+0000 7f2c177fe640 1 --2- 172.21.15.76:0/1755064410 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c180b0ae0 0x7f2c18150b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:46:57.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.223+0000 7f2c1ca38640 1 --2- 172.21.15.76:0/1755064410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c18159c70 0x7f2c181510b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009e80 tx=0x7f2c10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:57.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.224+0000 7f2c14ff9640 1 -- 172.21.15.76:0/1755064410 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c10040650 con 0x7f2c18159c70 2023-11-09T03:46:57.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.224+0000 7f2c14ff9640 1 -- 172.21.15.76:0/1755064410 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c100407b0 con 0x7f2c18159c70 2023-11-09T03:46:57.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.224+0000 7f2c14ff9640 1 -- 172.21.15.76:0/1755064410 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c10040b60 con 0x7f2c18159c70 2023-11-09T03:46:57.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.224+0000 7f2c1e49b640 1 -- 172.21.15.76:0/1755064410 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c180c6520 con 0x7f2c18159c70 2023-11-09T03:46:57.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.224+0000 7f2c1e49b640 1 -- 172.21.15.76:0/1755064410 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c18153e00 con 0x7f2c18159c70 2023-11-09T03:46:57.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.225+0000 7f2c1e49b640 1 -- 172.21.15.76:0/1755064410 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c18074290 con 0x7f2c18159c70 2023-11-09T03:46:57.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.225+0000 7f2c14ff9640 1 -- 172.21.15.76:0/1755064410 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2c10040cc0 con 0x7f2c18159c70 2023-11-09T03:46:57.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.225+0000 7f2c14ff9640 1 --2- 172.21.15.76:0/1755064410 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2c0406ae70 0x7f2c0406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:57.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.225+0000 7f2c14ff9640 1 -- 172.21.15.76:0/1755064410 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2c100ac4d0 con 0x7f2c18159c70 2023-11-09T03:46:57.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.226+0000 7f2c17fff640 1 --2- 172.21.15.76:0/1755064410 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2c0406ae70 0x7f2c0406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:57.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.226+0000 7f2c17fff640 1 --2- 172.21.15.76:0/1755064410 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2c0406ae70 0x7f2c0406d330 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2c08006fd0 tx=0x7f2c08008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:57.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.230+0000 7f2c14ff9640 1 -- 172.21.15.76:0/1755064410 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2c10077440 con 0x7f2c18159c70 2023-11-09T03:46:57.334 INFO:tasks.ceph.mds.a.smithi076.stderr:2023-11-09T03:46:57.335+0000 7f0aa61a1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:46:57.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.433+0000 7f2c1e49b640 1 -- 172.21.15.76:0/1755064410 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2c180c8ef0 con 0x7f2c18159c70 2023-11-09T03:46:57.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.434+0000 7f2c14ff9640 1 -- 172.21.15.76:0/1755064410 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2c10009450 con 0x7f2c18159c70 2023-11-09T03:46:57.433 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-09T03:46:57.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.436+0000 7f2c1e49b640 1 -- 172.21.15.76:0/1755064410 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2c0406ae70 msgr2=0x7f2c0406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:57.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.436+0000 7f2c1e49b640 1 --2- 172.21.15.76:0/1755064410 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2c0406ae70 0x7f2c0406d330 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2c08006fd0 tx=0x7f2c08008040 comp rx=0 tx=0).stop 2023-11-09T03:46:57.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.436+0000 7f2c1e49b640 1 -- 172.21.15.76:0/1755064410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c18159c70 msgr2=0x7f2c181510b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:57.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.436+0000 7f2c1e49b640 1 --2- 172.21.15.76:0/1755064410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c18159c70 0x7f2c181510b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009e80 tx=0x7f2c10004a40 comp rx=0 tx=0).stop 2023-11-09T03:46:57.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.436+0000 7f2c1e49b640 1 -- 172.21.15.76:0/1755064410 shutdown_connections 2023-11-09T03:46:57.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.436+0000 7f2c1e49b640 1 --2- 172.21.15.76:0/1755064410 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2c0406ae70 0x7f2c0406d330 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.436+0000 7f2c1e49b640 1 --2- 172.21.15.76:0/1755064410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c18159c70 0x7f2c181510b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.436+0000 7f2c1e49b640 1 --2- 172.21.15.76:0/1755064410 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c180b0ae0 0x7f2c18150b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.436+0000 7f2c1e49b640 1 --2- 172.21.15.76:0/1755064410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c1806db20 0x7f2c18158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:57.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.436+0000 7f2c1e49b640 1 -- 172.21.15.76:0/1755064410 >> 172.21.15.76:0/1755064410 conn(0x7f2c180ce950 msgr2=0x7f2c180cb060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:57.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.436+0000 7f2c1e49b640 1 -- 172.21.15.76:0/1755064410 shutdown_connections 2023-11-09T03:46:57.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:57.436+0000 7f2c1e49b640 1 -- 172.21.15.76:0/1755064410 wait complete. 2023-11-09T03:46:57.445 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-09T03:46:57.446 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-09T03:46:57.446 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2023-11-09T03:46:57.446 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-09T03:46:57.447 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-09T03:46:57.447 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-09T03:46:57.447 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-09T03:46:57.448 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-09T03:46:57.448 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:46:57.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:57.449 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2023-11-09T03:46:57.464 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-09T03:46:57.464 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-09T03:46:57.464 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2023-11-09T03:46:57.465 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:57.465 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2023-11-09T03:46:57.465 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:57.465 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:46:57.465 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:46:57.465 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2023-11-09T03:46:57.466 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:46:57.466 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:46:57.466 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2023-11-09T03:46:57.466 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:46:57.466 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:46:57.467 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-09T03:46:57.467 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:46:57.467 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-09T03:46:57.467 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42531sec preferred_lft 42531sec 2023-11-09T03:46:57.467 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2023-11-09T03:46:57.467 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:57.468 INFO:teuthology.orchestra.run.smithi076.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-09T03:46:57.468 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2e:15:bd:03:a8:2e brd ff:ff:ff:ff:ff:ff 2023-11-09T03:46:57.468 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-09T03:46:57.468 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:57.468 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::bcda:38ff:fed4:fdb0/64 scope link 2023-11-09T03:46:57.469 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:57.469 INFO:teuthology.orchestra.run.smithi076.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-09T03:46:57.469 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 6a:8d:94:d7:e5:00 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:46:57.469 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::688d:94ff:fed7:e500/64 scope link 2023-11-09T03:46:57.469 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:57.469 INFO:teuthology.orchestra.run.smithi076.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-09T03:46:57.470 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2e:15:bd:03:a8:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:46:57.470 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::2c15:bdff:fe03:a82e/64 scope link 2023-11-09T03:46:57.470 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:46:57.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:57.471 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-09T03:46:57.529 INFO:teuthology.orchestra.run.smithi076.stdout:1 2023-11-09T03:46:57.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:57.531 DEBUG:teuthology.orchestra.run.smithi076:> route 2023-11-09T03:46:57.587 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2023-11-09T03:46:57.587 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-09T03:46:57.587 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-09T03:46:57.588 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-09T03:46:57.588 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-09T03:46:57.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:57.589 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-09T03:46:57.589 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-09T03:46:57.589 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-09T03:46:57.590 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-09T03:46:57.590 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-09T03:46:57.590 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-09T03:46:57.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:57.683 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2023-11-09T03:46:57.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-09T03:46:57.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-09T03:46:57.698 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:46:57.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:57.699 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2023-11-09T03:46:57.765 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-09T03:46:57.765 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:46:57.765 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-09T03:46:57.820 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:46:57.820 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:46:57.820 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-09T03:46:57.874 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:46:57.929 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:46:57.929 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:46:57.929 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:46:57.930 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:46:57.930 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:46:57.930 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:46:57.930 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:46:57.930 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:46:57.930 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:46:57.931 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:46:57.931 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:46:57.931 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:46:57.931 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:46:57.931 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:46:57.932 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:46:57.932 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:46:57.932 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:46:57.932 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:46:57.932 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:46:57.932 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:46:57.933 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:46:57.933 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:46:57.933 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:46:57.933 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:46:57.933 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:46:57.934 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:46:57.934 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:46:57.934 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:46:57.934 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:46:57.934 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:46:57.935 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:46:57.935 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:46:57.935 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:46:57.935 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:46:57.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:57.936 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:46:57.983 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-09T03:46:57.984 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-09T03:46:58.027 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:46:58.042 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:46:58.042 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:46:58.042 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:46:58.042 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:46:58.042 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:46:58.043 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:46:58.043 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:46:58.043 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:46:58.043 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:46:58.043 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:46:58.044 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:46:58.044 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:46:58.044 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:46:58.044 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:46:58.044 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:46:58.045 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:46:58.045 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:46:58.045 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:46:58.045 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:46:58.045 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:46:58.046 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:46:58.046 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:46:58.046 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:46:58.046 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:46:58.046 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:46:58.047 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:46:58.047 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:46:58.047 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:46:58.047 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:46:58.047 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:46:58.047 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:46:58.048 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:46:58.048 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:46:58.048 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:46:58.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:58.049 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:46:58.118 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:46:58.119+0000 7fd01621d480 -1 init, newargv = 0x5600535d3990 newargc=15 2023-11-09T03:46:58.118 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[104760]: starting ceph client 2023-11-09T03:46:58.122 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[104760]: starting fuse 2023-11-09T03:46:59.197 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:46:59.216 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:46:59.217 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:46:59.217 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:46:59.217 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:46:59.217 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:46:59.217 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:46:59.218 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:46:59.218 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:46:59.218 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:46:59.218 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:46:59.218 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:46:59.218 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:46:59.219 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:46:59.219 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:46:59.219 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:46:59.219 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:46:59.219 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:46:59.220 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:46:59.220 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:46:59.220 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:46:59.220 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:46:59.220 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:46:59.220 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:46:59.221 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:46:59.221 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:46:59.221 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:46:59.221 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:46:59.221 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:46:59.221 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:46:59.222 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:46:59.222 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:46:59.222 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:46:59.222 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:46:59.222 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:46:59.223 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-09T03:46:59.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:46:59.224 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:46:59.280 INFO:teuthology.orchestra.run.smithi076.stdout:43 2023-11-09T03:46:59.281 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-09T03:46:59.281 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer -- python3 -c ' 2023-11-09T03:46:59.282 DEBUG:teuthology.orchestra.run.smithi076:> import glob 2023-11-09T03:46:59.282 DEBUG:teuthology.orchestra.run.smithi076:> import re 2023-11-09T03:46:59.282 DEBUG:teuthology.orchestra.run.smithi076:> import os 2023-11-09T03:46:59.283 DEBUG:teuthology.orchestra.run.smithi076:> import subprocess 2023-11-09T03:46:59.283 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:46:59.283 DEBUG:teuthology.orchestra.run.smithi076:> def _find_admin_socket(client_name): 2023-11-09T03:46:59.284 DEBUG:teuthology.orchestra.run.smithi076:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-09T03:46:59.284 DEBUG:teuthology.orchestra.run.smithi076:> files = glob.glob(asok_path) 2023-11-09T03:46:59.284 DEBUG:teuthology.orchestra.run.smithi076:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-09T03:46:59.285 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:46:59.285 DEBUG:teuthology.orchestra.run.smithi076:> # Given a non-glob path, it better be there 2023-11-09T03:46:59.285 DEBUG:teuthology.orchestra.run.smithi076:> if "*" not in asok_path: 2023-11-09T03:46:59.286 DEBUG:teuthology.orchestra.run.smithi076:> assert(len(files) == 1) 2023-11-09T03:46:59.286 DEBUG:teuthology.orchestra.run.smithi076:> return files[0] 2023-11-09T03:46:59.286 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:46:59.287 DEBUG:teuthology.orchestra.run.smithi076:> for f in files: 2023-11-09T03:46:59.287 DEBUG:teuthology.orchestra.run.smithi076:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-09T03:46:59.287 DEBUG:teuthology.orchestra.run.smithi076:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-09T03:46:59.288 DEBUG:teuthology.orchestra.run.smithi076:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-09T03:46:59.288 DEBUG:teuthology.orchestra.run.smithi076:> contents = proc_f.read() 2023-11-09T03:46:59.288 DEBUG:teuthology.orchestra.run.smithi076:> if mountpoint in contents: 2023-11-09T03:46:59.289 DEBUG:teuthology.orchestra.run.smithi076:> return f 2023-11-09T03:46:59.289 DEBUG:teuthology.orchestra.run.smithi076:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-09T03:46:59.289 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:46:59.290 DEBUG:teuthology.orchestra.run.smithi076:> print(_find_admin_socket("client.0")) 2023-11-09T03:46:59.290 DEBUG:teuthology.orchestra.run.smithi076:> ' 2023-11-09T03:46:59.408 INFO:teuthology.orchestra.run.smithi076.stdout:/var/run/ceph/ceph-client.0.104760.asok 2023-11-09T03:46:59.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:46:59.419 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.104760.asok 2023-11-09T03:46:59.419 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:46:59.420 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.104760.asok status 2023-11-09T03:46:59.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:46:59.589 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:46:59.589 INFO:teuthology.orchestra.run.smithi076.stdout: "metadata": { 2023-11-09T03:46:59.589 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_sha1": "7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4", 2023-11-09T03:46:59.590 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev)", 2023-11-09T03:46:59.590 INFO:teuthology.orchestra.run.smithi076.stdout: "entity_id": "0", 2023-11-09T03:46:59.590 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": "smithi076", 2023-11-09T03:46:59.590 INFO:teuthology.orchestra.run.smithi076.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-09T03:46:59.590 INFO:teuthology.orchestra.run.smithi076.stdout: "pid": "104760", 2023-11-09T03:46:59.591 INFO:teuthology.orchestra.run.smithi076.stdout: "root": "/" 2023-11-09T03:46:59.591 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:59.591 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_count": 0, 2023-11-09T03:46:59.591 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_pinned_count": 0, 2023-11-09T03:46:59.591 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 5817, 2023-11-09T03:46:59.591 INFO:teuthology.orchestra.run.smithi076.stdout: "inst": { 2023-11-09T03:46:59.592 INFO:teuthology.orchestra.run.smithi076.stdout: "name": { 2023-11-09T03:46:59.592 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "client", 2023-11-09T03:46:59.592 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 5817 2023-11-09T03:46:59.592 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:59.592 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2023-11-09T03:46:59.592 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2023-11-09T03:46:59.593 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2023-11-09T03:46:59.593 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 370183147 2023-11-09T03:46:59.593 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:46:59.593 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:59.593 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2023-11-09T03:46:59.594 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2023-11-09T03:46:59.594 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2023-11-09T03:46:59.594 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 370183147 2023-11-09T03:46:59.594 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:46:59.594 INFO:teuthology.orchestra.run.smithi076.stdout: "inst_str": "client.5817 192.168.0.1:0/370183147", 2023-11-09T03:46:59.595 INFO:teuthology.orchestra.run.smithi076.stdout: "addr_str": "192.168.0.1:0/370183147", 2023-11-09T03:46:59.595 INFO:teuthology.orchestra.run.smithi076.stdout: "inode_count": 1, 2023-11-09T03:46:59.595 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_epoch": 16, 2023-11-09T03:46:59.595 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch": 39, 2023-11-09T03:46:59.595 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch_barrier": 0, 2023-11-09T03:46:59.595 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklisted": false, 2023-11-09T03:46:59.596 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs" 2023-11-09T03:46:59.596 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:46:59.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:46:59.603 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:46:59.603 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-11-09T03:46:59.604 DEBUG:teuthology.orchestra.run.smithi076:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-09T03:46:59.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:46:59.663 INFO:teuthology.orchestra.run.smithi076.stdout:fuseblk 2023-11-09T03:46:59.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:46:59.670 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-09T03:46:59.670 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:46:59.671 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-09T03:46:59.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:46:59.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:46:59.737 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create snap-test-pool 2023-11-09T03:46:59.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.839+0000 7efd43739640 1 Processor -- start 2023-11-09T03:46:59.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.839+0000 7efd43739640 1 -- start start 2023-11-09T03:46:59.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.840+0000 7efd43739640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efd3c0c4ff0 0x7efd3c0cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:59.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.840+0000 7efd43739640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7efd3c0adac0 0x7efd3c0adf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:59.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.840+0000 7efd43739640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd3c15aff0 con 0x7efd3c159c00 2023-11-09T03:46:59.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.840+0000 7efd43739640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd3c0ae460 con 0x7efd3c0adac0 2023-11-09T03:46:59.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.840+0000 7efd43739640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd3c15a3e0 con 0x7efd3c0c4ff0 2023-11-09T03:46:59.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.840+0000 7efd414d5640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7efd3c159c00 0x7efd3c0c48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59918/0 (socket says 172.21.15.76:59918) 2023-11-09T03:46:59.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.840+0000 7efd414d5640 1 -- 172.21.15.76:0/3528487892 learned_addr learned my addr 172.21.15.76:0/3528487892 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:46:59.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.840+0000 7efd41cd6640 1 --2- 172.21.15.76:0/3528487892 >> v2:172.21.15.76:3300/0 conn(0x7efd3c0adac0 0x7efd3c0adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:59.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.840+0000 7efd33fff640 1 -- 172.21.15.76:0/3528487892 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 47297138 0 0) 0x7efd3c15aff0 con 0x7efd3c159c00 2023-11-09T03:46:59.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.840+0000 7efd33fff640 1 -- 172.21.15.76:0/3528487892 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd28003520 con 0x7efd3c159c00 2023-11-09T03:46:59.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.840+0000 7efd41cd6640 1 -- 172.21.15.76:0/3528487892 >> v2:172.21.15.113:3300/0 conn(0x7efd3c0c4ff0 msgr2=0x7efd3c0cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:46:59.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.840+0000 7efd40cd4640 1 --2- 172.21.15.76:0/3528487892 >> v2:172.21.15.113:3300/0 conn(0x7efd3c0c4ff0 0x7efd3c0cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:59.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.840+0000 7efd41cd6640 1 --2- 172.21.15.76:0/3528487892 >> v2:172.21.15.113:3300/0 conn(0x7efd3c0c4ff0 0x7efd3c0cc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:59.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.840+0000 7efd41cd6640 1 -- 172.21.15.76:0/3528487892 >> v1:172.21.15.76:6789/0 conn(0x7efd3c159c00 legacy=0x7efd3c0c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:59.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.840+0000 7efd41cd6640 1 -- 172.21.15.76:0/3528487892 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3c15a6c0 con 0x7efd3c0adac0 2023-11-09T03:46:59.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.841+0000 7efd41cd6640 1 --2- 172.21.15.76:0/3528487892 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd3c0adac0 0x7efd3c0adf20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0cde50 tx=0x7efd3402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b62046312969a31 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:59.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.841+0000 7efd33fff640 1 -- 172.21.15.76:0/3528487892 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3402cc80 con 0x7efd3c0adac0 2023-11-09T03:46:59.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.841+0000 7efd33fff640 1 -- 172.21.15.76:0/3528487892 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3402cde0 con 0x7efd3c0adac0 2023-11-09T03:46:59.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.841+0000 7efd33fff640 1 -- 172.21.15.76:0/3528487892 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34034660 con 0x7efd3c0adac0 2023-11-09T03:46:59.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.842+0000 7efd43739640 1 -- 172.21.15.76:0/3528487892 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd3c0adac0 msgr2=0x7efd3c0adf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:59.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.842+0000 7efd43739640 1 --2- 172.21.15.76:0/3528487892 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd3c0adac0 0x7efd3c0adf20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0cde50 tx=0x7efd3402bed0 comp rx=0 tx=0).stop 2023-11-09T03:46:59.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.842+0000 7efd43739640 1 -- 172.21.15.76:0/3528487892 shutdown_connections 2023-11-09T03:46:59.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.842+0000 7efd43739640 1 --2- 172.21.15.76:0/3528487892 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd3c0adac0 0x7efd3c0adf20 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:59.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.842+0000 7efd43739640 1 --2- 172.21.15.76:0/3528487892 >> v2:172.21.15.113:3300/0 conn(0x7efd3c0c4ff0 0x7efd3c0cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:59.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.842+0000 7efd43739640 1 -- 172.21.15.76:0/3528487892 >> 172.21.15.76:0/3528487892 conn(0x7efd3c0ce950 msgr2=0x7efd3c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:46:59.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.842+0000 7efd43739640 1 -- 172.21.15.76:0/3528487892 shutdown_connections 2023-11-09T03:46:59.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.842+0000 7efd43739640 1 -- 172.21.15.76:0/3528487892 wait complete. 2023-11-09T03:46:59.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.843+0000 7efd43739640 1 Processor -- start 2023-11-09T03:46:59.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.843+0000 7efd43739640 1 -- start start 2023-11-09T03:46:59.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.844+0000 7efd43739640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd3c0adac0 0x7efd3c15c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:59.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.844+0000 7efd43739640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd3c0c4ff0 0x7efd3c1f6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:59.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.844+0000 7efd43739640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd3c159c00 0x7efd3c1f6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:59.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.844+0000 7efd43739640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c1f36f0 con 0x7efd3c159c00 2023-11-09T03:46:59.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.844+0000 7efd43739640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c1f2a70 con 0x7efd3c0adac0 2023-11-09T03:46:59.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.844+0000 7efd43739640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c1f2d50 con 0x7efd3c0c4ff0 2023-11-09T03:46:59.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.844+0000 7efd41cd6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd3c159c00 0x7efd3c1f6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:59.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.844+0000 7efd41cd6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd3c159c00 0x7efd3c1f6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41762/0 (socket says 172.21.15.76:41762) 2023-11-09T03:46:59.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.844+0000 7efd41cd6640 1 -- 172.21.15.76:0/395727688 learned_addr learned my addr 172.21.15.76:0/395727688 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:46:59.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.844+0000 7efd414d5640 1 --2- 172.21.15.76:0/395727688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd3c0adac0 0x7efd3c15c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:59.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.844+0000 7efd40cd4640 1 --2- 172.21.15.76:0/395727688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd3c0c4ff0 0x7efd3c1f6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:59.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.844+0000 7efd41cd6640 1 -- 172.21.15.76:0/395727688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd3c0c4ff0 msgr2=0x7efd3c1f6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:59.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.844+0000 7efd41cd6640 1 --2- 172.21.15.76:0/395727688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd3c0c4ff0 0x7efd3c1f6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:59.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.844+0000 7efd41cd6640 1 -- 172.21.15.76:0/395727688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd3c0adac0 msgr2=0x7efd3c15c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:46:59.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.844+0000 7efd41cd6640 1 --2- 172.21.15.76:0/395727688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd3c0adac0 0x7efd3c15c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:46:59.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.844+0000 7efd41cd6640 1 -- 172.21.15.76:0/395727688 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd34009a70 con 0x7efd3c159c00 2023-11-09T03:46:59.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.844+0000 7efd41cd6640 1 --2- 172.21.15.76:0/395727688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd3c159c00 0x7efd3c1f6aa0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7efd34009e80 tx=0x7efd34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:59.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.844+0000 7efd31ffb640 1 -- 172.21.15.76:0/395727688 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34040650 con 0x7efd3c159c00 2023-11-09T03:46:59.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.845+0000 7efd31ffb640 1 -- 172.21.15.76:0/395727688 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd340407b0 con 0x7efd3c159c00 2023-11-09T03:46:59.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.845+0000 7efd43739640 1 -- 172.21.15.76:0/395727688 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd3c1f3030 con 0x7efd3c159c00 2023-11-09T03:46:59.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.845+0000 7efd31ffb640 1 -- 172.21.15.76:0/395727688 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34040b60 con 0x7efd3c159c00 2023-11-09T03:46:59.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.845+0000 7efd43739640 1 -- 172.21.15.76:0/395727688 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd3c15cee0 con 0x7efd3c159c00 2023-11-09T03:46:59.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.846+0000 7efd43739640 1 -- 172.21.15.76:0/395727688 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd3c1f3830 con 0x7efd3c159c00 2023-11-09T03:46:59.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.846+0000 7efd31ffb640 1 -- 172.21.15.76:0/395727688 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efd34040cc0 con 0x7efd3c159c00 2023-11-09T03:46:59.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.846+0000 7efd31ffb640 1 --2- 172.21.15.76:0/395727688 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efd2006ae70 0x7efd2006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:46:59.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.847+0000 7efd31ffb640 1 -- 172.21.15.76:0/395727688 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7efd340ac420 con 0x7efd3c159c00 2023-11-09T03:46:59.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.847+0000 7efd414d5640 1 --2- 172.21.15.76:0/395727688 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efd2006ae70 0x7efd2006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:46:59.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.847+0000 7efd414d5640 1 --2- 172.21.15.76:0/395727688 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efd2006ae70 0x7efd2006d330 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efd380011e0 tx=0x7efd38009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:46:59.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:46:59.851+0000 7efd31ffb640 1 -- 172.21.15.76:0/395727688 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efd3c1f3830 con 0x7efd3c159c00 2023-11-09T03:47:00.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.018+0000 7efd43739640 1 -- 172.21.15.76:0/395727688 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "snap-test-pool"} v 0) v1 -- 0x7efd3c1f3830 con 0x7efd3c159c00 2023-11-09T03:47:00.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.568+0000 7efd31ffb640 1 -- 172.21.15.76:0/395727688 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "snap-test-pool"}]=0 pool 'snap-test-pool' created v40) v1 ==== 118+0+0 (secure 0 0 0) 0x7efd3c1f3830 con 0x7efd3c159c00 2023-11-09T03:47:00.568 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'snap-test-pool' created 2023-11-09T03:47:00.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.571+0000 7efd43739640 1 -- 172.21.15.76:0/395727688 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efd2006ae70 msgr2=0x7efd2006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:00.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.571+0000 7efd43739640 1 --2- 172.21.15.76:0/395727688 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efd2006ae70 0x7efd2006d330 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efd380011e0 tx=0x7efd38009040 comp rx=0 tx=0).stop 2023-11-09T03:47:00.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.571+0000 7efd43739640 1 -- 172.21.15.76:0/395727688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd3c159c00 msgr2=0x7efd3c1f6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:00.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.571+0000 7efd43739640 1 --2- 172.21.15.76:0/395727688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd3c159c00 0x7efd3c1f6aa0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7efd34009e80 tx=0x7efd34004a40 comp rx=0 tx=0).stop 2023-11-09T03:47:00.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.572+0000 7efd43739640 1 -- 172.21.15.76:0/395727688 shutdown_connections 2023-11-09T03:47:00.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.572+0000 7efd43739640 1 --2- 172.21.15.76:0/395727688 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efd2006ae70 0x7efd2006d330 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:00.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.572+0000 7efd43739640 1 --2- 172.21.15.76:0/395727688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd3c159c00 0x7efd3c1f6aa0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:00.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.572+0000 7efd43739640 1 --2- 172.21.15.76:0/395727688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd3c0c4ff0 0x7efd3c1f6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:00.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.572+0000 7efd43739640 1 --2- 172.21.15.76:0/395727688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd3c0adac0 0x7efd3c15c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:00.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.572+0000 7efd43739640 1 -- 172.21.15.76:0/395727688 >> 172.21.15.76:0/395727688 conn(0x7efd3c0ce950 msgr2=0x7efd3c06c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:00.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.572+0000 7efd43739640 1 -- 172.21.15.76:0/395727688 shutdown_connections 2023-11-09T03:47:00.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.573+0000 7efd43739640 1 -- 172.21.15.76:0/395727688 wait complete. 2023-11-09T03:47:00.575 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:47:00.578 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:47:00.587 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool snap-test-pool mksnap snap3 2023-11-09T03:47:00.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.669+0000 7fb80bc40f40 1 Processor -- start 2023-11-09T03:47:00.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.669+0000 7fb80bc40f40 1 -- start start 2023-11-09T03:47:00.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.670+0000 7fb80bc40f40 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7f40030b0 0x55eb6a837370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:00.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.670+0000 7fb80bc40f40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb6a792a30 0x55eb6a8c3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:00.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.670+0000 7fb80bc40f40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb6a764710 0x55eb6a838a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:00.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.670+0000 7fb80bc40f40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb6a7163b0 con 0x55eb6a764710 2023-11-09T03:47:00.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.670+0000 7fb80bc40f40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb6a6b0650 con 0x55eb6a792a30 2023-11-09T03:47:00.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.670+0000 7fb80bc40f40 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb6a6b02f0 con 0x7fb7f40030b0 2023-11-09T03:47:00.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.670+0000 7fb809831640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb6a764710 0x55eb6a838a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:00.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.670+0000 7fb809831640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb6a764710 0x55eb6a838a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41782/0 (socket says 172.21.15.76:41782) 2023-11-09T03:47:00.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.670+0000 7fb809831640 1 -- 172.21.15.76:0/3097555185 learned_addr learned my addr 172.21.15.76:0/3097555185 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:00.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.670+0000 7fb809831640 1 -- 172.21.15.76:0/3097555185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7f40030b0 msgr2=0x55eb6a837370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:00.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.670+0000 7fb809831640 1 --2- 172.21.15.76:0/3097555185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7f40030b0 0x55eb6a837370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:00.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.670+0000 7fb809831640 1 -- 172.21.15.76:0/3097555185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb6a792a30 msgr2=0x55eb6a8c3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:00.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.670+0000 7fb809831640 1 --2- 172.21.15.76:0/3097555185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb6a792a30 0x55eb6a8c3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:00.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.670+0000 7fb809831640 1 -- 172.21.15.76:0/3097555185 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb6a6bc780 con 0x55eb6a764710 2023-11-09T03:47:00.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.670+0000 7fb809831640 1 --2- 172.21.15.76:0/3097555185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb6a764710 0x55eb6a838a30 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x55eb6a7b0af0 tx=0x7fb804005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a855b5fa88564f server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:00.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.670+0000 7fb7fb7fe640 1 -- 172.21.15.76:0/3097555185 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80400f830 con 0x55eb6a764710 2023-11-09T03:47:00.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.670+0000 7fb7fb7fe640 1 -- 172.21.15.76:0/3097555185 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80400f990 con 0x55eb6a764710 2023-11-09T03:47:00.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.670+0000 7fb7fb7fe640 1 -- 172.21.15.76:0/3097555185 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80400fd40 con 0x55eb6a764710 2023-11-09T03:47:00.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.671+0000 7fb80bc40f40 1 -- 172.21.15.76:0/3097555185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb6a764710 msgr2=0x55eb6a838a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:00.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.671+0000 7fb80bc40f40 1 --2- 172.21.15.76:0/3097555185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb6a764710 0x55eb6a838a30 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x55eb6a7b0af0 tx=0x7fb804005d80 comp rx=0 tx=0).stop 2023-11-09T03:47:00.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.671+0000 7fb80bc40f40 1 -- 172.21.15.76:0/3097555185 shutdown_connections 2023-11-09T03:47:00.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.671+0000 7fb80bc40f40 1 --2- 172.21.15.76:0/3097555185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb6a764710 0x55eb6a838a30 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:00.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.671+0000 7fb80bc40f40 1 --2- 172.21.15.76:0/3097555185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb6a792a30 0x55eb6a8c3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:00.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.671+0000 7fb80bc40f40 1 --2- 172.21.15.76:0/3097555185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7f40030b0 0x55eb6a837370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:00.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.671+0000 7fb80bc40f40 1 -- 172.21.15.76:0/3097555185 >> 172.21.15.76:0/3097555185 conn(0x55eb6a792f40 msgr2=0x55eb6a837fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:00.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.671+0000 7fb80bc40f40 1 -- 172.21.15.76:0/3097555185 shutdown_connections 2023-11-09T03:47:00.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.671+0000 7fb80bc40f40 1 -- 172.21.15.76:0/3097555185 wait complete. 2023-11-09T03:47:00.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.672+0000 7fb80bc40f40 1 Processor -- start 2023-11-09T03:47:00.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.672+0000 7fb80bc40f40 1 -- start start 2023-11-09T03:47:00.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.672+0000 7fb80bc40f40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb6a764710 0x55eb6a7ab4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:00.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.672+0000 7fb80bc40f40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb6a792a30 0x55eb6a82bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:00.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.672+0000 7fb80bc40f40 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7f40030b0 0x55eb6a838a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:00.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.672+0000 7fb80bc40f40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb6a6904f0 con 0x55eb6a792a30 2023-11-09T03:47:00.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.672+0000 7fb80bc40f40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb6a8c22c0 con 0x55eb6a764710 2023-11-09T03:47:00.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.672+0000 7fb80bc40f40 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb6a8c25a0 con 0x7fb7f40030b0 2023-11-09T03:47:00.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.672+0000 7fb80882f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb6a792a30 0x55eb6a82bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:00.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.672+0000 7fb80882f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb6a792a30 0x55eb6a82bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41784/0 (socket says 172.21.15.76:41784) 2023-11-09T03:47:00.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.672+0000 7fb80882f640 1 -- 172.21.15.76:0/1101920693 learned_addr learned my addr 172.21.15.76:0/1101920693 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:00.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.673+0000 7fb809030640 1 --2- 172.21.15.76:0/1101920693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb6a764710 0x55eb6a7ab4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:00.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.673+0000 7fb809831640 1 --2- 172.21.15.76:0/1101920693 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7f40030b0 0x55eb6a838a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:00.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.673+0000 7fb80882f640 1 -- 172.21.15.76:0/1101920693 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7f40030b0 msgr2=0x55eb6a838a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:00.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.673+0000 7fb80882f640 1 --2- 172.21.15.76:0/1101920693 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7f40030b0 0x55eb6a838a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:00.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.673+0000 7fb80882f640 1 -- 172.21.15.76:0/1101920693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb6a764710 msgr2=0x55eb6a7ab4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:00.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.673+0000 7fb80882f640 1 --2- 172.21.15.76:0/1101920693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb6a764710 0x55eb6a7ab4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:00.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.673+0000 7fb80882f640 1 -- 172.21.15.76:0/1101920693 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80400b440 con 0x55eb6a792a30 2023-11-09T03:47:00.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.673+0000 7fb809831640 1 --2- 172.21.15.76:0/1101920693 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7f40030b0 0x55eb6a838a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:00.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.673+0000 7fb80882f640 1 --2- 172.21.15.76:0/1101920693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb6a792a30 0x55eb6a82bf30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb800009ad0 tx=0x7fb800001c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:00.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.673+0000 7fb7fa7fc640 1 -- 172.21.15.76:0/1101920693 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8000558e0 con 0x55eb6a792a30 2023-11-09T03:47:00.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.673+0000 7fb7fa7fc640 1 -- 172.21.15.76:0/1101920693 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb800055a40 con 0x55eb6a792a30 2023-11-09T03:47:00.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.673+0000 7fb80bc40f40 1 -- 172.21.15.76:0/1101920693 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55eb6a6af930 con 0x55eb6a792a30 2023-11-09T03:47:00.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.673+0000 7fb7fa7fc640 1 -- 172.21.15.76:0/1101920693 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb800055db0 con 0x55eb6a792a30 2023-11-09T03:47:00.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.673+0000 7fb80bc40f40 1 -- 172.21.15.76:0/1101920693 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb6a6af390 con 0x55eb6a792a30 2023-11-09T03:47:00.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.675+0000 7fb7fa7fc640 1 -- 172.21.15.76:0/1101920693 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb800061480 con 0x55eb6a792a30 2023-11-09T03:47:00.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.675+0000 7fb7fa7fc640 1 --2- 172.21.15.76:0/1101920693 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb7ec069820 0x7fb7ec06bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:00.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.675+0000 7fb7fa7fc640 1 -- 172.21.15.76:0/1101920693 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb8000cd3e0 con 0x55eb6a792a30 2023-11-09T03:47:00.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.675+0000 7fb80bc40f40 1 -- 172.21.15.76:0/1101920693 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- pool_op(create snap pool 7 tid 1 name snap3 v0) v4 -- 0x55eb6a720330 con 0x55eb6a792a30 2023-11-09T03:47:00.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.675+0000 7fb809030640 1 --2- 172.21.15.76:0/1101920693 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb7ec069820 0x7fb7ec06bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:00.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:00.676+0000 7fb809030640 1 --2- 172.21.15.76:0/1101920693 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb7ec069820 0x7fb7ec06bce0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc002ff0 tx=0x7fb7fc005a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:01.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.576+0000 7fb7fa7fc640 1 -- 172.21.15.76:0/1101920693 <== mon.0 v2:172.21.15.76:3300/0 6 ==== pool_op_reply(tid 1 (0) Success v41) v1 ==== 43+0+0 (secure 0 0 0) 0x7fb8000983a0 con 0x55eb6a792a30 2023-11-09T03:47:01.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.576+0000 7fb7fa7fc640 1 -- 172.21.15.76:0/1101920693 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=41}) v3 -- 0x7fb7ec072510 con 0x55eb6a792a30 2023-11-09T03:47:01.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.577+0000 7fb7fa7fc640 1 -- 172.21.15.76:0/1101920693 <== mon.0 v2:172.21.15.76:3300/0 7 ==== osd_map(41..41 src has 1..41) v4 ==== 718+0+0 (secure 0 0 0) 0x7fb80009c760 con 0x55eb6a792a30 2023-11-09T03:47:01.576 INFO:teuthology.orchestra.run.smithi076.stdout:created pool snap-test-pool snap snap3 2023-11-09T03:47:01.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.577+0000 7fb80bc40f40 1 -- 172.21.15.76:0/1101920693 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb7ec069820 msgr2=0x7fb7ec06bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:01.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.577+0000 7fb80bc40f40 1 --2- 172.21.15.76:0/1101920693 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb7ec069820 0x7fb7ec06bce0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc002ff0 tx=0x7fb7fc005a00 comp rx=0 tx=0).stop 2023-11-09T03:47:01.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.577+0000 7fb80bc40f40 1 -- 172.21.15.76:0/1101920693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb6a792a30 msgr2=0x55eb6a82bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:01.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.577+0000 7fb80bc40f40 1 --2- 172.21.15.76:0/1101920693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb6a792a30 0x55eb6a82bf30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb800009ad0 tx=0x7fb800001c40 comp rx=0 tx=0).stop 2023-11-09T03:47:01.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.577+0000 7fb80bc40f40 1 -- 172.21.15.76:0/1101920693 shutdown_connections 2023-11-09T03:47:01.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.577+0000 7fb80bc40f40 1 --2- 172.21.15.76:0/1101920693 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb7ec069820 0x7fb7ec06bce0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:01.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.577+0000 7fb80bc40f40 1 --2- 172.21.15.76:0/1101920693 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7f40030b0 0x55eb6a838a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:01.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.577+0000 7fb80bc40f40 1 --2- 172.21.15.76:0/1101920693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55eb6a792a30 0x55eb6a82bf30 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:01.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.577+0000 7fb80bc40f40 1 --2- 172.21.15.76:0/1101920693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eb6a764710 0x55eb6a7ab4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:01.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.577+0000 7fb80bc40f40 1 -- 172.21.15.76:0/1101920693 >> 172.21.15.76:0/1101920693 conn(0x55eb6a792f40 msgr2=0x55eb6a837fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:01.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.578+0000 7fb80bc40f40 1 -- 172.21.15.76:0/1101920693 shutdown_connections 2023-11-09T03:47:01.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.578+0000 7fb80bc40f40 1 -- 172.21.15.76:0/1101920693 wait complete. 2023-11-09T03:47:01.582 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs snap-test-pool 2023-11-09T03:47:01.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.730+0000 7f2c6d40e640 1 Processor -- start 2023-11-09T03:47:01.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.730+0000 7f2c6d40e640 1 -- start start 2023-11-09T03:47:01.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.731+0000 7f2c6d40e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c68160430 0x7f2c680b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:01.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.731+0000 7f2c6d40e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2c680b2b60 0x7f2c6806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:01.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.731+0000 7f2c6d40e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c680b2fc0 con 0x7f2c6806c1d0 2023-11-09T03:47:01.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.731+0000 7f2c6d40e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c680cb580 con 0x7f2c680b2b60 2023-11-09T03:47:01.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.731+0000 7f2c6d40e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6806ca40 con 0x7f2c68160430 2023-11-09T03:47:01.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.731+0000 7f2c66ffd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c68160430 0x7f2c680b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:01.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.731+0000 7f2c5ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2c680b2b60 0x7f2c6806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:01.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.731+0000 7f2c5ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2c680b2b60 0x7f2c6806bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:46308/0 (socket says 172.21.15.76:46308) 2023-11-09T03:47:01.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.731+0000 7f2c66ffd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c68160430 0x7f2c680b2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:37652/0 (socket says 172.21.15.76:37652) 2023-11-09T03:47:01.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.731+0000 7f2c5ffff640 1 -- 172.21.15.76:0/3803629738 learned_addr learned my addr 172.21.15.76:0/3803629738 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:01.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.731+0000 7f2c677fe640 1 --1- 172.21.15.76:0/3803629738 >> v1:172.21.15.113:6789/0 conn(0x7f2c6806c1d0 0x7f2c68160b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:57358/0 (socket says 172.21.15.76:57358) 2023-11-09T03:47:01.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.732+0000 7f2c5ffff640 1 -- 172.21.15.76:0/3803629738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c68160430 msgr2=0x7f2c680b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:01.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.732+0000 7f2c5ffff640 1 --2- 172.21.15.76:0/3803629738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c68160430 0x7f2c680b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:01.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.732+0000 7f2c5ffff640 1 -- 172.21.15.76:0/3803629738 >> v1:172.21.15.113:6789/0 conn(0x7f2c6806c1d0 legacy=0x7f2c68160b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:01.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.732+0000 7f2c5ffff640 1 -- 172.21.15.76:0/3803629738 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c6806cd20 con 0x7f2c680b2b60 2023-11-09T03:47:01.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.732+0000 7f2c66ffd640 1 --2- 172.21.15.76:0/3803629738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c68160430 0x7f2c680b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:47:01.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.733+0000 7f2c5ffff640 1 --2- 172.21.15.76:0/3803629738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c680b2b60 0x7f2c6806bc90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2c50009870 tx=0x7f2c5002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f408d8371136715a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:01.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.733+0000 7f2c667fc640 1 -- 172.21.15.76:0/3803629738 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c50030840 con 0x7f2c680b2b60 2023-11-09T03:47:01.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.733+0000 7f2c667fc640 1 -- 172.21.15.76:0/3803629738 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c500309a0 con 0x7f2c680b2b60 2023-11-09T03:47:01.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.733+0000 7f2c667fc640 1 -- 172.21.15.76:0/3803629738 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c50030d30 con 0x7f2c680b2b60 2023-11-09T03:47:01.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.733+0000 7f2c6d40e640 1 -- 172.21.15.76:0/3803629738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c680b2b60 msgr2=0x7f2c6806bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:01.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.733+0000 7f2c6d40e640 1 --2- 172.21.15.76:0/3803629738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c680b2b60 0x7f2c6806bc90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2c50009870 tx=0x7f2c5002b980 comp rx=0 tx=0).stop 2023-11-09T03:47:01.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.733+0000 7f2c6d40e640 1 -- 172.21.15.76:0/3803629738 shutdown_connections 2023-11-09T03:47:01.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.733+0000 7f2c6d40e640 1 --2- 172.21.15.76:0/3803629738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c680b2b60 0x7f2c6806bc90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:01.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.733+0000 7f2c6d40e640 1 --2- 172.21.15.76:0/3803629738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c68160430 0x7f2c680b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:01.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.733+0000 7f2c6d40e640 1 -- 172.21.15.76:0/3803629738 >> 172.21.15.76:0/3803629738 conn(0x7f2c6806ea20 msgr2=0x7f2c6806ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:01.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.733+0000 7f2c6d40e640 1 -- 172.21.15.76:0/3803629738 shutdown_connections 2023-11-09T03:47:01.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.733+0000 7f2c6d40e640 1 -- 172.21.15.76:0/3803629738 wait complete. 2023-11-09T03:47:01.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.735+0000 7f2c6d40e640 1 Processor -- start 2023-11-09T03:47:01.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.735+0000 7f2c6d40e640 1 -- start start 2023-11-09T03:47:01.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.735+0000 7f2c6d40e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c6806c1d0 0x7f2c680ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:01.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.735+0000 7f2c6d40e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c680b2b60 0x7f2c680c4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:01.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.735+0000 7f2c6d40e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c68160430 0x7f2c68142dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:01.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.735+0000 7f2c6d40e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c680cd090 con 0x7f2c6806c1d0 2023-11-09T03:47:01.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.735+0000 7f2c6d40e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c680c4eb0 con 0x7f2c68160430 2023-11-09T03:47:01.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.735+0000 7f2c6d40e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c680cc760 con 0x7f2c680b2b60 2023-11-09T03:47:01.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.735+0000 7f2c66ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c6806c1d0 0x7f2c680ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:01.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.735+0000 7f2c66ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c6806c1d0 0x7f2c680ca720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41792/0 (socket says 172.21.15.76:41792) 2023-11-09T03:47:01.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.735+0000 7f2c66ffd640 1 -- 172.21.15.76:0/768842078 learned_addr learned my addr 172.21.15.76:0/768842078 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:01.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.736+0000 7f2c677fe640 1 --2- 172.21.15.76:0/768842078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c68160430 0x7f2c68142dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:01.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.736+0000 7f2c66ffd640 1 -- 172.21.15.76:0/768842078 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c680b2b60 msgr2=0x7f2c680c4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:01.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.736+0000 7f2c66ffd640 1 --2- 172.21.15.76:0/768842078 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c680b2b60 0x7f2c680c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:01.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.736+0000 7f2c66ffd640 1 -- 172.21.15.76:0/768842078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c68160430 msgr2=0x7f2c68142dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:01.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.736+0000 7f2c66ffd640 1 --2- 172.21.15.76:0/768842078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c68160430 0x7f2c68142dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:01.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.736+0000 7f2c66ffd640 1 -- 172.21.15.76:0/768842078 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c58009400 con 0x7f2c6806c1d0 2023-11-09T03:47:01.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.736+0000 7f2c66ffd640 1 --2- 172.21.15.76:0/768842078 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c6806c1d0 0x7f2c680ca720 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800e960 tx=0x7f2c5800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:01.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.736+0000 7f2c5f7fe640 1 -- 172.21.15.76:0/768842078 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5800ca00 con 0x7f2c6806c1d0 2023-11-09T03:47:01.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.736+0000 7f2c5f7fe640 1 -- 172.21.15.76:0/768842078 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5800cb60 con 0x7f2c6806c1d0 2023-11-09T03:47:01.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.736+0000 7f2c6d40e640 1 -- 172.21.15.76:0/768842078 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c500094d0 con 0x7f2c6806c1d0 2023-11-09T03:47:01.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.736+0000 7f2c5f7fe640 1 -- 172.21.15.76:0/768842078 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c58013470 con 0x7f2c6806c1d0 2023-11-09T03:47:01.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.736+0000 7f2c6d40e640 1 -- 172.21.15.76:0/768842078 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c680ccd20 con 0x7f2c6806c1d0 2023-11-09T03:47:01.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.737+0000 7f2c6d40e640 1 -- 172.21.15.76:0/768842078 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c680b2980 con 0x7f2c6806c1d0 2023-11-09T03:47:01.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.737+0000 7f2c5f7fe640 1 -- 172.21.15.76:0/768842078 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2c580135d0 con 0x7f2c6806c1d0 2023-11-09T03:47:01.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.738+0000 7f2c5f7fe640 1 --2- 172.21.15.76:0/768842078 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2c4806ae70 0x7f2c4806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:01.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.738+0000 7f2c5f7fe640 1 -- 172.21.15.76:0/768842078 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6421+0+0 (secure 0 0 0) 0x7f2c58089f90 con 0x7f2c6806c1d0 2023-11-09T03:47:01.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.738+0000 7f2c5ffff640 1 --2- 172.21.15.76:0/768842078 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2c4806ae70 0x7f2c4806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:01.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.738+0000 7f2c5ffff640 1 --2- 172.21.15.76:0/768842078 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2c4806ae70 0x7f2c4806d330 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2c50036000 tx=0x7f2c5003b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:01.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.742+0000 7f2c5f7fe640 1 -- 172.21.15.76:0/768842078 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2c580bcca0 con 0x7f2c6806c1d0 2023-11-09T03:47:01.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.944+0000 7f2c6d40e640 1 -- 172.21.15.76:0/768842078 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "snap-test-pool"} v 0) v1 -- 0x7f2c68074290 con 0x7f2c6806c1d0 2023-11-09T03:47:01.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.945+0000 7f2c5f7fe640 1 -- 172.21.15.76:0/768842078 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "snap-test-pool"}]=-95 pool(7) already has mon-managed snaps; can't attach pool to fs v16) v1 ==== 173+0+0 (secure 0 0 0) 0x7f2c58054cf0 con 0x7f2c6806c1d0 2023-11-09T03:47:01.944 INFO:teuthology.orchestra.run.smithi076.stderr:Error ENOTSUP: pool(7) already has mon-managed snaps; can't attach pool to fs 2023-11-09T03:47:01.949 DEBUG:teuthology.orchestra.run:got remote process result: 95 2023-11-09T03:47:01.950 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool snap-test-pool rmsnap snap3 2023-11-09T03:47:01.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.992+0000 7f4f9cfdaf40 1 Processor -- start 2023-11-09T03:47:01.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.992+0000 7f4f9cfdaf40 1 -- start start 2023-11-09T03:47:01.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.993+0000 7f4f9cfdaf40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5576b53de190 0x5576b54625a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:01.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.993+0000 7f4f9cfdaf40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5576b53df340 0x5576b546b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:01.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.993+0000 7f4f9cfdaf40 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5576b546c8c0 0x5576b5468660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:01.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.993+0000 7f4f9cfdaf40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576b53443b0 con 0x5576b53de190 2023-11-09T03:47:01.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.993+0000 7f4f9cfdaf40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576b52de650 con 0x5576b53df340 2023-11-09T03:47:01.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.993+0000 7f4f9cfdaf40 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576b52de2f0 con 0x5576b546c8c0 2023-11-09T03:47:01.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.993+0000 7f4f9aade640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5576b53de190 0x5576b54625a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:01.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.993+0000 7f4f9aade640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5576b53de190 0x5576b54625a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41810/0 (socket says 172.21.15.76:41810) 2023-11-09T03:47:01.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.993+0000 7f4f9aade640 1 -- 172.21.15.76:0/3591146493 learned_addr learned my addr 172.21.15.76:0/3591146493 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:01.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.993+0000 7f4f9b2df640 1 --2- 172.21.15.76:0/3591146493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5576b546c8c0 0x5576b5468660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:01.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.993+0000 7f4f9a2dd640 1 --2- 172.21.15.76:0/3591146493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5576b53df340 0x5576b546b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:01.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.993+0000 7f4f9aade640 1 -- 172.21.15.76:0/3591146493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5576b546c8c0 msgr2=0x5576b5468660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:01.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.993+0000 7f4f9aade640 1 --2- 172.21.15.76:0/3591146493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5576b546c8c0 0x5576b5468660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:01.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.993+0000 7f4f9aade640 1 -- 172.21.15.76:0/3591146493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5576b53df340 msgr2=0x5576b546b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:01.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.993+0000 7f4f9aade640 1 --2- 172.21.15.76:0/3591146493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5576b53df340 0x5576b546b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:01.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.993+0000 7f4f9aade640 1 -- 172.21.15.76:0/3591146493 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f94005d20 con 0x5576b53de190 2023-11-09T03:47:01.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.993+0000 7f4f9aade640 1 --2- 172.21.15.76:0/3591146493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5576b53de190 0x5576b54625a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c0016b0 tx=0x7f4f8c001b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5200dee72651fe7c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:01.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.993+0000 7f4f992db640 1 -- 172.21.15.76:0/3591146493 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c003650 con 0x5576b53de190 2023-11-09T03:47:01.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.993+0000 7f4f992db640 1 -- 172.21.15.76:0/3591146493 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8c0037b0 con 0x5576b53de190 2023-11-09T03:47:01.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.993+0000 7f4f992db640 1 -- 172.21.15.76:0/3591146493 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c003b40 con 0x5576b53de190 2023-11-09T03:47:01.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.994+0000 7f4f9cfdaf40 1 -- 172.21.15.76:0/3591146493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5576b53de190 msgr2=0x5576b54625a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:02.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.994+0000 7f4f9cfdaf40 1 --2- 172.21.15.76:0/3591146493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5576b53de190 0x5576b54625a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c0016b0 tx=0x7f4f8c001b70 comp rx=0 tx=0).stop 2023-11-09T03:47:02.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.994+0000 7f4f9cfdaf40 1 -- 172.21.15.76:0/3591146493 shutdown_connections 2023-11-09T03:47:02.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.994+0000 7f4f9cfdaf40 1 --2- 172.21.15.76:0/3591146493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5576b546c8c0 0x5576b5468660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:02.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.994+0000 7f4f9cfdaf40 1 --2- 172.21.15.76:0/3591146493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5576b53df340 0x5576b546b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:02.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.994+0000 7f4f9cfdaf40 1 --2- 172.21.15.76:0/3591146493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5576b53de190 0x5576b54625a0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:02.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.994+0000 7f4f9cfdaf40 1 -- 172.21.15.76:0/3591146493 >> 172.21.15.76:0/3591146493 conn(0x5576b53dd670 msgr2=0x5576b5393e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:02.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.994+0000 7f4f9cfdaf40 1 -- 172.21.15.76:0/3591146493 shutdown_connections 2023-11-09T03:47:02.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.994+0000 7f4f9cfdaf40 1 -- 172.21.15.76:0/3591146493 wait complete. 2023-11-09T03:47:02.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.994+0000 7f4f9cfdaf40 1 Processor -- start 2023-11-09T03:47:02.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.994+0000 7f4f9cfdaf40 1 -- start start 2023-11-09T03:47:02.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.995+0000 7f4f9cfdaf40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5576b53de190 0x5576b53c3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:02.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.995+0000 7f4f9cfdaf40 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5576b53df340 0x5576b5467bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:02.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.995+0000 7f4f9cfdaf40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5576b546c8c0 0x5576b53c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:02.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.995+0000 7f4f9cfdaf40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576b52be4f0 con 0x5576b546c8c0 2023-11-09T03:47:02.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.995+0000 7f4f9cfdaf40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576b53da650 con 0x5576b53de190 2023-11-09T03:47:02.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.995+0000 7f4f9cfdaf40 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576b53da8a0 con 0x5576b53df340 2023-11-09T03:47:02.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.995+0000 7f4f9b2df640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5576b546c8c0 0x5576b53c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:02.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.995+0000 7f4f9b2df640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5576b546c8c0 0x5576b53c57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41826/0 (socket says 172.21.15.76:41826) 2023-11-09T03:47:02.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.995+0000 7f4f9b2df640 1 -- 172.21.15.76:0/4016669194 learned_addr learned my addr 172.21.15.76:0/4016669194 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:02.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.995+0000 7f4f9a2dd640 1 --2- 172.21.15.76:0/4016669194 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5576b53df340 0x5576b5467bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:02.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.995+0000 7f4f9aade640 1 --2- 172.21.15.76:0/4016669194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5576b53de190 0x5576b53c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:02.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.995+0000 7f4f9b2df640 1 -- 172.21.15.76:0/4016669194 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5576b53df340 msgr2=0x5576b5467bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:02.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.995+0000 7f4f9b2df640 1 --2- 172.21.15.76:0/4016669194 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5576b53df340 0x5576b5467bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:02.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.995+0000 7f4f9b2df640 1 -- 172.21.15.76:0/4016669194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5576b53de190 msgr2=0x5576b53c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:02.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.995+0000 7f4f9b2df640 1 --2- 172.21.15.76:0/4016669194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5576b53de190 0x5576b53c3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:02.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.995+0000 7f4f9b2df640 1 -- 172.21.15.76:0/4016669194 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f8c006de0 con 0x5576b546c8c0 2023-11-09T03:47:02.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.995+0000 7f4f9aade640 1 --2- 172.21.15.76:0/4016669194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5576b53de190 0x5576b53c3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:02.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.996+0000 7f4f9b2df640 1 --2- 172.21.15.76:0/4016669194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5576b546c8c0 0x5576b53c57f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4f94055040 tx=0x7f4f94002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:02.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.996+0000 7f4f93fff640 1 -- 172.21.15.76:0/4016669194 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f94052a00 con 0x5576b546c8c0 2023-11-09T03:47:02.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.996+0000 7f4f93fff640 1 -- 172.21.15.76:0/4016669194 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f94052b60 con 0x5576b546c8c0 2023-11-09T03:47:02.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.996+0000 7f4f93fff640 1 -- 172.21.15.76:0/4016669194 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9405d420 con 0x5576b546c8c0 2023-11-09T03:47:02.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.996+0000 7f4f9cfdaf40 1 -- 172.21.15.76:0/4016669194 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5576b52ea780 con 0x5576b546c8c0 2023-11-09T03:47:02.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.996+0000 7f4f9cfdaf40 1 -- 172.21.15.76:0/4016669194 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5576b52dd930 con 0x5576b546c8c0 2023-11-09T03:47:02.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.997+0000 7f4f93fff640 1 -- 172.21.15.76:0/4016669194 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4f9405d580 con 0x5576b546c8c0 2023-11-09T03:47:02.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.997+0000 7f4f93fff640 1 --2- 172.21.15.76:0/4016669194 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4f7c069820 0x7f4f7c06bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:02.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.998+0000 7f4f93fff640 1 -- 172.21.15.76:0/4016669194 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6421+0+0 (secure 0 0 0) 0x7f4f940c9f00 con 0x5576b546c8c0 2023-11-09T03:47:02.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.998+0000 7f4f9aade640 1 --2- 172.21.15.76:0/4016669194 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4f7c069820 0x7f4f7c06bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:02.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.998+0000 7f4f9cfdaf40 1 -- 172.21.15.76:0/4016669194 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- pool_op(delete snap pool 7 tid 1 name snap3 v0) v4 -- 0x5576b534e330 con 0x5576b546c8c0 2023-11-09T03:47:02.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:01.998+0000 7f4f9aade640 1 --2- 172.21.15.76:0/4016669194 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4f7c069820 0x7f4f7c06bce0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c0016b0 tx=0x7f4f8c008070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:02.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.580+0000 7f4f93fff640 1 -- 172.21.15.76:0/4016669194 <== mon.0 v2:172.21.15.76:3300/0 6 ==== pool_op_reply(tid 1 (0) Success v42) v1 ==== 43+0+0 (secure 0 0 0) 0x7f4f94061080 con 0x5576b546c8c0 2023-11-09T03:47:02.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.580+0000 7f4f93fff640 1 -- 172.21.15.76:0/4016669194 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=42}) v3 -- 0x7f4f7c072570 con 0x5576b546c8c0 2023-11-09T03:47:02.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.581+0000 7f4f93fff640 1 -- 172.21.15.76:0/4016669194 <== mon.0 v2:172.21.15.76:3300/0 7 ==== osd_map(42..42 src has 1..42) v4 ==== 643+0+0 (secure 0 0 0) 0x7f4f94098ae0 con 0x5576b546c8c0 2023-11-09T03:47:02.581 INFO:teuthology.orchestra.run.smithi076.stdout:removed pool snap-test-pool snap snap3 2023-11-09T03:47:02.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.581+0000 7f4f9cfdaf40 1 -- 172.21.15.76:0/4016669194 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4f7c069820 msgr2=0x7f4f7c06bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:02.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.581+0000 7f4f9cfdaf40 1 --2- 172.21.15.76:0/4016669194 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4f7c069820 0x7f4f7c06bce0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c0016b0 tx=0x7f4f8c008070 comp rx=0 tx=0).stop 2023-11-09T03:47:02.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.581+0000 7f4f9cfdaf40 1 -- 172.21.15.76:0/4016669194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5576b546c8c0 msgr2=0x5576b53c57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:02.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.581+0000 7f4f9cfdaf40 1 --2- 172.21.15.76:0/4016669194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5576b546c8c0 0x5576b53c57f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4f94055040 tx=0x7f4f94002fb0 comp rx=0 tx=0).stop 2023-11-09T03:47:02.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.581+0000 7f4f9cfdaf40 1 -- 172.21.15.76:0/4016669194 shutdown_connections 2023-11-09T03:47:02.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.581+0000 7f4f9cfdaf40 1 --2- 172.21.15.76:0/4016669194 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4f7c069820 0x7f4f7c06bce0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:02.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.581+0000 7f4f9cfdaf40 1 --2- 172.21.15.76:0/4016669194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5576b546c8c0 0x5576b53c57f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:02.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.581+0000 7f4f9cfdaf40 1 --2- 172.21.15.76:0/4016669194 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5576b53df340 0x5576b5467bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:02.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.581+0000 7f4f9cfdaf40 1 --2- 172.21.15.76:0/4016669194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5576b53de190 0x5576b53c3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:02.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.581+0000 7f4f9cfdaf40 1 -- 172.21.15.76:0/4016669194 >> 172.21.15.76:0/4016669194 conn(0x5576b53dd670 msgr2=0x5576b5393e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:02.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.581+0000 7f4f9cfdaf40 1 -- 172.21.15.76:0/4016669194 shutdown_connections 2023-11-09T03:47:02.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.581+0000 7f4f9cfdaf40 1 -- 172.21.15.76:0/4016669194 wait complete. 2023-11-09T03:47:02.586 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete snap-test-pool 2023-11-09T03:47:02.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.733+0000 7fec0e86f640 1 Processor -- start 2023-11-09T03:47:02.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.733+0000 7fec0e86f640 1 -- start start 2023-11-09T03:47:02.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.734+0000 7fec0e86f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec08159c00 0x7fec080cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:02.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.734+0000 7fec0e86f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fec080ae1d0 0x7fec080c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:02.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.734+0000 7fec0e86f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec0815aff0 con 0x7fec080c48e0 2023-11-09T03:47:02.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.734+0000 7fec0e86f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec080c52a0 con 0x7fec080ae1d0 2023-11-09T03:47:02.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.734+0000 7fec0e86f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec0815a3e0 con 0x7fec08159c00 2023-11-09T03:47:02.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.734+0000 7fec0ce0c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fec080ae1d0 0x7fec080c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:02.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.734+0000 7fec077fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fec080c48e0 0x7fec080adac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37338/0 (socket says 172.21.15.76:37338) 2023-11-09T03:47:02.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.734+0000 7fec077fe640 1 -- 172.21.15.76:0/502720269 learned_addr learned my addr 172.21.15.76:0/502720269 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:02.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.734+0000 7fec07fff640 1 --2- 172.21.15.76:0/502720269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec08159c00 0x7fec080cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:02.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.734+0000 7fec0ce0c640 1 -- 172.21.15.76:0/502720269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec08159c00 msgr2=0x7fec080cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:02.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.734+0000 7fec0ce0c640 1 --2- 172.21.15.76:0/502720269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec08159c00 0x7fec080cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:02.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.734+0000 7fec0ce0c640 1 -- 172.21.15.76:0/502720269 >> v1:172.21.15.76:6789/0 conn(0x7fec080c48e0 legacy=0x7fec080adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:02.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.734+0000 7fec06ffd640 1 -- 172.21.15.76:0/502720269 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1227853172 0 0) 0x7fec0815aff0 con 0x7fec080c48e0 2023-11-09T03:47:02.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.734+0000 7fec0ce0c640 1 -- 172.21.15.76:0/502720269 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec0815a6c0 con 0x7fec080ae1d0 2023-11-09T03:47:02.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.735+0000 7fec0ce0c640 1 --2- 172.21.15.76:0/502720269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec080ae1d0 0x7fec080c4d60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fec00009d90 tx=0x7fec0002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca03d6abdbf9716d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:02.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.735+0000 7fec06ffd640 1 -- 172.21.15.76:0/502720269 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0002cc50 con 0x7fec080ae1d0 2023-11-09T03:47:02.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.735+0000 7fec06ffd640 1 -- 172.21.15.76:0/502720269 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec0002cdb0 con 0x7fec080ae1d0 2023-11-09T03:47:02.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.735+0000 7fec06ffd640 1 -- 172.21.15.76:0/502720269 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec00034660 con 0x7fec080ae1d0 2023-11-09T03:47:02.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.736+0000 7fec0e86f640 1 -- 172.21.15.76:0/502720269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec080ae1d0 msgr2=0x7fec080c4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:02.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.736+0000 7fec0e86f640 1 --2- 172.21.15.76:0/502720269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec080ae1d0 0x7fec080c4d60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fec00009d90 tx=0x7fec0002bea0 comp rx=0 tx=0).stop 2023-11-09T03:47:02.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.736+0000 7fec0e86f640 1 -- 172.21.15.76:0/502720269 shutdown_connections 2023-11-09T03:47:02.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.736+0000 7fec0e86f640 1 --2- 172.21.15.76:0/502720269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec080ae1d0 0x7fec080c4d60 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:02.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.736+0000 7fec0e86f640 1 --2- 172.21.15.76:0/502720269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec08159c00 0x7fec080cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:02.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.736+0000 7fec0e86f640 1 -- 172.21.15.76:0/502720269 >> 172.21.15.76:0/502720269 conn(0x7fec080ce950 msgr2=0x7fec080ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:02.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.736+0000 7fec0e86f640 1 -- 172.21.15.76:0/502720269 shutdown_connections 2023-11-09T03:47:02.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.736+0000 7fec0e86f640 1 -- 172.21.15.76:0/502720269 wait complete. 2023-11-09T03:47:02.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.737+0000 7fec0e86f640 1 Processor -- start 2023-11-09T03:47:02.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.737+0000 7fec0e86f640 1 -- start start 2023-11-09T03:47:02.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.738+0000 7fec0e86f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec080ae1d0 0x7fec08154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:02.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.738+0000 7fec0e86f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec080c48e0 0x7fec080b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:02.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.738+0000 7fec0e86f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec08159c00 0x7fec081e75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:02.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.738+0000 7fec0e86f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec080b26b0 con 0x7fec08159c00 2023-11-09T03:47:02.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.738+0000 7fec0e86f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec080b1aa0 con 0x7fec080c48e0 2023-11-09T03:47:02.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.738+0000 7fec0e86f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec080b1d80 con 0x7fec080ae1d0 2023-11-09T03:47:02.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.738+0000 7fec0ce0c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec08159c00 0x7fec081e75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:02.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.738+0000 7fec0ce0c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec08159c00 0x7fec081e75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40238/0 (socket says 172.21.15.76:40238) 2023-11-09T03:47:02.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.738+0000 7fec0ce0c640 1 -- 172.21.15.76:0/2691321565 learned_addr learned my addr 172.21.15.76:0/2691321565 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:02.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.738+0000 7fec077fe640 1 --2- 172.21.15.76:0/2691321565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec080c48e0 0x7fec080b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:02.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.738+0000 7fec07fff640 1 --2- 172.21.15.76:0/2691321565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec080ae1d0 0x7fec08154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:02.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.738+0000 7fec0ce0c640 1 -- 172.21.15.76:0/2691321565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec080ae1d0 msgr2=0x7fec08154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:02.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.738+0000 7fec0ce0c640 1 --2- 172.21.15.76:0/2691321565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec080ae1d0 0x7fec08154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:02.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.738+0000 7fec0ce0c640 1 -- 172.21.15.76:0/2691321565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec080c48e0 msgr2=0x7fec080b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:02.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.738+0000 7fec0ce0c640 1 --2- 172.21.15.76:0/2691321565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec080c48e0 0x7fec080b1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:02.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.738+0000 7fec0ce0c640 1 -- 172.21.15.76:0/2691321565 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec00009a70 con 0x7fec08159c00 2023-11-09T03:47:02.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.738+0000 7fec0ce0c640 1 --2- 172.21.15.76:0/2691321565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec08159c00 0x7fec081e75a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fec00009d90 tx=0x7fec00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:02.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.738+0000 7fec04ff9640 1 -- 172.21.15.76:0/2691321565 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec00040650 con 0x7fec08159c00 2023-11-09T03:47:02.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.738+0000 7fec04ff9640 1 -- 172.21.15.76:0/2691321565 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec000407d0 con 0x7fec08159c00 2023-11-09T03:47:02.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.738+0000 7fec0e86f640 1 -- 172.21.15.76:0/2691321565 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec080b2060 con 0x7fec08159c00 2023-11-09T03:47:02.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.738+0000 7fec04ff9640 1 -- 172.21.15.76:0/2691321565 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec00040b80 con 0x7fec08159c00 2023-11-09T03:47:02.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.738+0000 7fec0e86f640 1 -- 172.21.15.76:0/2691321565 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec08154ff0 con 0x7fec08159c00 2023-11-09T03:47:02.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.740+0000 7fec0e86f640 1 -- 172.21.15.76:0/2691321565 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec080b2860 con 0x7fec08159c00 2023-11-09T03:47:02.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.740+0000 7fec04ff9640 1 -- 172.21.15.76:0/2691321565 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fec00040ce0 con 0x7fec08159c00 2023-11-09T03:47:02.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.740+0000 7fec04ff9640 1 --2- 172.21.15.76:0/2691321565 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7febe806ae70 0x7febe806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:02.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.740+0000 7fec04ff9640 1 -- 172.21.15.76:0/2691321565 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6438+0+0 (secure 0 0 0) 0x7fec000ac7a0 con 0x7fec08159c00 2023-11-09T03:47:02.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.740+0000 7fec07fff640 1 --2- 172.21.15.76:0/2691321565 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7febe806ae70 0x7febe806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:02.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.741+0000 7fec07fff640 1 --2- 172.21.15.76:0/2691321565 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7febe806ae70 0x7febe806d330 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7febf8006fd0 tx=0x7febf8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:02.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.744+0000 7fec04ff9640 1 -- 172.21.15.76:0/2691321565 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fec080b2860 con 0x7fec08159c00 2023-11-09T03:47:02.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.914+0000 7fec0e86f640 1 -- 172.21.15.76:0/2691321565 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "snap-test-pool"} v 0) v1 -- 0x7fec08074290 con 0x7fec08159c00 2023-11-09T03:47:02.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:02.915+0000 7fec04ff9640 1 -- 172.21.15.76:0/2691321565 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "snap-test-pool"}]=-1 WARNING: this will *PERMANENTLY DESTROY* all data stored in pool snap-test-pool. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. v42) v1 ==== 296+0+0 (secure 0 0 0) 0x7fec00077600 con 0x7fec08159c00 2023-11-09T03:47:02.914 INFO:teuthology.orchestra.run.smithi076.stderr:Error EPERM: WARNING: this will *PERMANENTLY DESTROY* all data stored in pool snap-test-pool. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. 2023-11-09T03:47:02.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:47:02.920 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-09T03:47:02.920 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-09T03:47:02.962 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-09T03:47:03.005 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-09T03:47:03.005 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:47:03.005 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:47:03.021 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:47:03.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.164+0000 7fb1270e7640 1 Processor -- start 2023-11-09T03:47:03.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.165+0000 7fb1270e7640 1 -- start start 2023-11-09T03:47:03.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.165+0000 7fb1270e7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb1200cde60 0x7fb12015d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:03.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.165+0000 7fb1270e7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1200ce240 0x7fb1200d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:03.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.165+0000 7fb1270e7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb12015e000 con 0x7fb12015daa0 2023-11-09T03:47:03.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.165+0000 7fb1270e7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb12015e5c0 con 0x7fb1200cde60 2023-11-09T03:47:03.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.165+0000 7fb1270e7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb12015e2e0 con 0x7fb1200ce240 2023-11-09T03:47:03.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.165+0000 7fb124e83640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb1200cde60 0x7fb12015d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:03.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.165+0000 7fb124e83640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb1200cde60 0x7fb12015d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40250/0 (socket says 172.21.15.76:40250) 2023-11-09T03:47:03.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.166+0000 7fb124e83640 1 -- 172.21.15.76:0/2586281138 learned_addr learned my addr 172.21.15.76:0/2586281138 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:03.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.166+0000 7fb125684640 1 --2- 172.21.15.76:0/2586281138 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1200ce240 0x7fb1200d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:03.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.166+0000 7fb124e83640 1 -- 172.21.15.76:0/2586281138 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1200ce240 msgr2=0x7fb1200d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:03.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.166+0000 7fb124e83640 1 --2- 172.21.15.76:0/2586281138 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1200ce240 0x7fb1200d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.166+0000 7fb124e83640 1 -- 172.21.15.76:0/2586281138 >> v1:172.21.15.113:6789/0 conn(0x7fb12015daa0 legacy=0x7fb120157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:03.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.166+0000 7fb124e83640 1 -- 172.21.15.76:0/2586281138 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1200d1120 con 0x7fb1200cde60 2023-11-09T03:47:03.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.167+0000 7fb124e83640 1 --2- 172.21.15.76:0/2586281138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1200cde60 0x7fb12015d560 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb110009870 tx=0x7fb11002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5905cb033290db6d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:03.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.167+0000 7fb11f7fe640 1 -- 172.21.15.76:0/2586281138 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb110030840 con 0x7fb1200cde60 2023-11-09T03:47:03.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.167+0000 7fb11f7fe640 1 -- 172.21.15.76:0/2586281138 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1100309a0 con 0x7fb1200cde60 2023-11-09T03:47:03.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.167+0000 7fb11f7fe640 1 -- 172.21.15.76:0/2586281138 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb110030d30 con 0x7fb1200cde60 2023-11-09T03:47:03.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.167+0000 7fb1270e7640 1 -- 172.21.15.76:0/2586281138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1200cde60 msgr2=0x7fb12015d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:03.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.167+0000 7fb1270e7640 1 --2- 172.21.15.76:0/2586281138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1200cde60 0x7fb12015d560 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb110009870 tx=0x7fb11002b980 comp rx=0 tx=0).stop 2023-11-09T03:47:03.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.167+0000 7fb1270e7640 1 -- 172.21.15.76:0/2586281138 shutdown_connections 2023-11-09T03:47:03.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.167+0000 7fb1270e7640 1 --2- 172.21.15.76:0/2586281138 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1200ce240 0x7fb1200d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.167+0000 7fb1270e7640 1 --2- 172.21.15.76:0/2586281138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1200cde60 0x7fb12015d560 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.167+0000 7fb1270e7640 1 -- 172.21.15.76:0/2586281138 >> 172.21.15.76:0/2586281138 conn(0x7fb120071f30 msgr2=0x7fb1200cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:03.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.167+0000 7fb1270e7640 1 -- 172.21.15.76:0/2586281138 shutdown_connections 2023-11-09T03:47:03.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.167+0000 7fb1270e7640 1 -- 172.21.15.76:0/2586281138 wait complete. 2023-11-09T03:47:03.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.168+0000 7fb1270e7640 1 Processor -- start 2023-11-09T03:47:03.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.168+0000 7fb1270e7640 1 -- start start 2023-11-09T03:47:03.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.169+0000 7fb1270e7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1200cde60 0x7fb120152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:03.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.169+0000 7fb1270e7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1200ce240 0x7fb1200cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:03.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.169+0000 7fb1270e7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb12015daa0 0x7fb120154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:03.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.169+0000 7fb1270e7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1201f3830 con 0x7fb12015daa0 2023-11-09T03:47:03.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.169+0000 7fb1270e7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1201f3550 con 0x7fb1200cde60 2023-11-09T03:47:03.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.169+0000 7fb1270e7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1200cb0e0 con 0x7fb1200ce240 2023-11-09T03:47:03.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.169+0000 7fb125684640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb12015daa0 0x7fb120154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:03.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.169+0000 7fb125684640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb12015daa0 0x7fb120154640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40260/0 (socket says 172.21.15.76:40260) 2023-11-09T03:47:03.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.169+0000 7fb125684640 1 -- 172.21.15.76:0/3313579958 learned_addr learned my addr 172.21.15.76:0/3313579958 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:03.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.170+0000 7fb124e83640 1 --2- 172.21.15.76:0/3313579958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1200cde60 0x7fb120152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:03.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.170+0000 7fb11ffff640 1 --2- 172.21.15.76:0/3313579958 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1200ce240 0x7fb1200cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:03.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.170+0000 7fb125684640 1 -- 172.21.15.76:0/3313579958 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1200ce240 msgr2=0x7fb1200cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:03.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.170+0000 7fb125684640 1 --2- 172.21.15.76:0/3313579958 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1200ce240 0x7fb1200cbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.170+0000 7fb125684640 1 -- 172.21.15.76:0/3313579958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1200cde60 msgr2=0x7fb120152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:03.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.170+0000 7fb125684640 1 --2- 172.21.15.76:0/3313579958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1200cde60 0x7fb120152250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.170+0000 7fb125684640 1 -- 172.21.15.76:0/3313579958 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1100094d0 con 0x7fb12015daa0 2023-11-09T03:47:03.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.170+0000 7fb11ffff640 1 --2- 172.21.15.76:0/3313579958 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1200ce240 0x7fb1200cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:03.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.170+0000 7fb125684640 1 --2- 172.21.15.76:0/3313579958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb12015daa0 0x7fb120154640 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb118007e30 tx=0x7fb11800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:03.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.170+0000 7fb11d7fa640 1 -- 172.21.15.76:0/3313579958 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb118014620 con 0x7fb12015daa0 2023-11-09T03:47:03.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.170+0000 7fb11d7fa640 1 -- 172.21.15.76:0/3313579958 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11800ee90 con 0x7fb12015daa0 2023-11-09T03:47:03.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.170+0000 7fb1270e7640 1 -- 172.21.15.76:0/3313579958 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1200cb3c0 con 0x7fb12015daa0 2023-11-09T03:47:03.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.170+0000 7fb11d7fa640 1 -- 172.21.15.76:0/3313579958 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1180149c0 con 0x7fb12015daa0 2023-11-09T03:47:03.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.170+0000 7fb1270e7640 1 -- 172.21.15.76:0/3313579958 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb120154b80 con 0x7fb12015daa0 2023-11-09T03:47:03.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.171+0000 7fb1270e7640 1 -- 172.21.15.76:0/3313579958 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1200caf40 con 0x7fb12015daa0 2023-11-09T03:47:03.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.172+0000 7fb11d7fa640 1 -- 172.21.15.76:0/3313579958 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb118014b20 con 0x7fb12015daa0 2023-11-09T03:47:03.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.172+0000 7fb11d7fa640 1 --2- 172.21.15.76:0/3313579958 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb10c06ae70 0x7fb10c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:03.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.172+0000 7fb11d7fa640 1 -- 172.21.15.76:0/3313579958 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6438+0+0 (secure 0 0 0) 0x7fb11808a240 con 0x7fb12015daa0 2023-11-09T03:47:03.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.172+0000 7fb124e83640 1 --2- 172.21.15.76:0/3313579958 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb10c06ae70 0x7fb10c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:03.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.173+0000 7fb124e83640 1 --2- 172.21.15.76:0/3313579958 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb10c06ae70 0x7fb10c06d330 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb110036000 tx=0x7fb11003b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:03.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.176+0000 7fb11d7fa640 1 -- 172.21.15.76:0/3313579958 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb1200caf40 con 0x7fb12015daa0 2023-11-09T03:47:03.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.345+0000 7fb1270e7640 1 -- 172.21.15.76:0/3313579958 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1200cb8e0 con 0x7fb12015daa0 2023-11-09T03:47:03.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.346+0000 7fb11d7fa640 1 -- 172.21.15.76:0/3313579958 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18601 (secure 0 0 0) 0x7fb1200cb8e0 con 0x7fb12015daa0 2023-11-09T03:47:03.346 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:03.346 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":42,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:47:02.574636+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:46:50.064765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-09T03:46:50.837125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":1,"length":1}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":1,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:47:03.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.348+0000 7fb1270e7640 1 -- 172.21.15.76:0/3313579958 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb10c06ae70 msgr2=0x7fb10c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:03.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.348+0000 7fb1270e7640 1 --2- 172.21.15.76:0/3313579958 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb10c06ae70 0x7fb10c06d330 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb110036000 tx=0x7fb11003b040 comp rx=0 tx=0).stop 2023-11-09T03:47:03.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.348+0000 7fb1270e7640 1 -- 172.21.15.76:0/3313579958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb12015daa0 msgr2=0x7fb120154640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:03.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.348+0000 7fb1270e7640 1 --2- 172.21.15.76:0/3313579958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb12015daa0 0x7fb120154640 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb118007e30 tx=0x7fb11800e4b0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.348+0000 7fb1270e7640 1 -- 172.21.15.76:0/3313579958 shutdown_connections 2023-11-09T03:47:03.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.348+0000 7fb1270e7640 1 --2- 172.21.15.76:0/3313579958 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb10c06ae70 0x7fb10c06d330 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.348+0000 7fb1270e7640 1 --2- 172.21.15.76:0/3313579958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb12015daa0 0x7fb120154640 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.348+0000 7fb1270e7640 1 --2- 172.21.15.76:0/3313579958 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1200ce240 0x7fb1200cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.348+0000 7fb1270e7640 1 --2- 172.21.15.76:0/3313579958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1200cde60 0x7fb120152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.348+0000 7fb1270e7640 1 -- 172.21.15.76:0/3313579958 >> 172.21.15.76:0/3313579958 conn(0x7fb120071f30 msgr2=0x7fb1200c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:03.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.348+0000 7fb1270e7640 1 -- 172.21.15.76:0/3313579958 shutdown_connections 2023-11-09T03:47:03.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.348+0000 7fb1270e7640 1 -- 172.21.15.76:0/3313579958 wait complete. 2023-11-09T03:47:03.358 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:46:50.064765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:46:50.837125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-09T03:47:00.036540+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-09T03:47:03.359 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:47:03.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.462+0000 7f401b67c640 1 Processor -- start 2023-11-09T03:47:03.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.463+0000 7f401b67c640 1 -- start start 2023-11-09T03:47:03.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.463+0000 7f401b67c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f401415e430 0x7f40140cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:03.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.463+0000 7f401b67c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40140d0710 0x7f40140b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:03.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.463+0000 7f401b67c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40140d0280 con 0x7f40140cf160 2023-11-09T03:47:03.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.463+0000 7f401b67c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40140cf670 con 0x7f401415e430 2023-11-09T03:47:03.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.464+0000 7f401b67c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40140cf950 con 0x7f40140d0710 2023-11-09T03:47:03.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.464+0000 7f4019418640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f401415e430 0x7f40140cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:03.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.464+0000 7f4018c17640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f40140cf160 0x7f40140cd510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37346/0 (socket says 172.21.15.76:37346) 2023-11-09T03:47:03.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.464+0000 7f4018c17640 1 -- 172.21.15.76:0/3040061080 learned_addr learned my addr 172.21.15.76:0/3040061080 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:03.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.464+0000 7f4019418640 1 -- 172.21.15.76:0/3040061080 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40140d0710 msgr2=0x7f40140b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:03.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.464+0000 7f4019418640 1 --2- 172.21.15.76:0/3040061080 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40140d0710 0x7f40140b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.464+0000 7f4019418640 1 -- 172.21.15.76:0/3040061080 >> v1:172.21.15.76:6789/0 conn(0x7f40140cf160 legacy=0x7f40140cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:03.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.464+0000 7f4019418640 1 -- 172.21.15.76:0/3040061080 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40140cfc30 con 0x7f401415e430 2023-11-09T03:47:03.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.464+0000 7f400bfff640 1 -- 172.21.15.76:0/3040061080 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 203182129 0 0) 0x7f40140d0280 con 0x7f40140cf160 2023-11-09T03:47:03.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.465+0000 7f4019418640 1 --2- 172.21.15.76:0/3040061080 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f401415e430 0x7f40140cec20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f40040097a0 tx=0x7f400402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0fde3fa150f9701 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:03.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.465+0000 7f400bfff640 1 -- 172.21.15.76:0/3040061080 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4004031830 con 0x7f401415e430 2023-11-09T03:47:03.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.465+0000 7f400bfff640 1 -- 172.21.15.76:0/3040061080 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4004031990 con 0x7f401415e430 2023-11-09T03:47:03.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.465+0000 7f400bfff640 1 -- 172.21.15.76:0/3040061080 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4004031d20 con 0x7f401415e430 2023-11-09T03:47:03.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.465+0000 7f401b67c640 1 -- 172.21.15.76:0/3040061080 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f401415e430 msgr2=0x7f40140cec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:03.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.465+0000 7f401b67c640 1 --2- 172.21.15.76:0/3040061080 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f401415e430 0x7f40140cec20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f40040097a0 tx=0x7f400402b8d0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.465+0000 7f401b67c640 1 -- 172.21.15.76:0/3040061080 shutdown_connections 2023-11-09T03:47:03.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.465+0000 7f401b67c640 1 --2- 172.21.15.76:0/3040061080 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40140d0710 0x7f40140b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.465+0000 7f401b67c640 1 --2- 172.21.15.76:0/3040061080 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f401415e430 0x7f40140cec20 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.465+0000 7f401b67c640 1 -- 172.21.15.76:0/3040061080 >> 172.21.15.76:0/3040061080 conn(0x7f401406ea20 msgr2=0x7f401406ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:03.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.465+0000 7f401b67c640 1 -- 172.21.15.76:0/3040061080 shutdown_connections 2023-11-09T03:47:03.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.465+0000 7f401b67c640 1 -- 172.21.15.76:0/3040061080 wait complete. 2023-11-09T03:47:03.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.467+0000 7f401b67c640 1 Processor -- start 2023-11-09T03:47:03.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.467+0000 7f401b67c640 1 -- start start 2023-11-09T03:47:03.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.467+0000 7f401b67c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40140cf160 0x7f40140aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:03.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.467+0000 7f401b67c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40140d0710 0x7f40141f39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:03.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.467+0000 7f401b67c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f401415e430 0x7f40140b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:03.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.467+0000 7f401b67c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40140c43c0 con 0x7f40140cf160 2023-11-09T03:47:03.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.467+0000 7f401b67c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40140af500 con 0x7f40140d0710 2023-11-09T03:47:03.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.467+0000 7f401b67c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40141f4280 con 0x7f401415e430 2023-11-09T03:47:03.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.467+0000 7f4019418640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40140cf160 0x7f40140aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:03.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.467+0000 7f4019418640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40140cf160 0x7f40140aefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40282/0 (socket says 172.21.15.76:40282) 2023-11-09T03:47:03.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.467+0000 7f4019418640 1 -- 172.21.15.76:0/574465136 learned_addr learned my addr 172.21.15.76:0/574465136 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:03.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.468+0000 7f4019c19640 1 --2- 172.21.15.76:0/574465136 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f401415e430 0x7f40140b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:03.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.468+0000 7f4018c17640 1 --2- 172.21.15.76:0/574465136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40140d0710 0x7f40141f39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:03.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.468+0000 7f4019418640 1 -- 172.21.15.76:0/574465136 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f401415e430 msgr2=0x7f40140b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:03.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.468+0000 7f4019418640 1 --2- 172.21.15.76:0/574465136 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f401415e430 0x7f40140b3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.468+0000 7f4019418640 1 -- 172.21.15.76:0/574465136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40140d0710 msgr2=0x7f40141f39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:03.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.468+0000 7f4019418640 1 --2- 172.21.15.76:0/574465136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40140d0710 0x7f40141f39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.468+0000 7f4019418640 1 -- 172.21.15.76:0/574465136 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4004009400 con 0x7f40140cf160 2023-11-09T03:47:03.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.468+0000 7f4019418640 1 --2- 172.21.15.76:0/574465136 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40140cf160 0x7f40140aefc0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f40040098c0 tx=0x7f40040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:03.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.468+0000 7f4009ffb640 1 -- 172.21.15.76:0/574465136 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400403ba70 con 0x7f40140cf160 2023-11-09T03:47:03.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.468+0000 7f4009ffb640 1 -- 172.21.15.76:0/574465136 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400403bbd0 con 0x7f40140cf160 2023-11-09T03:47:03.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.468+0000 7f401b67c640 1 -- 172.21.15.76:0/574465136 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40141f4560 con 0x7f40140cf160 2023-11-09T03:47:03.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.468+0000 7f4009ffb640 1 -- 172.21.15.76:0/574465136 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40040453f0 con 0x7f40140cf160 2023-11-09T03:47:03.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.468+0000 7f401b67c640 1 -- 172.21.15.76:0/574465136 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40140c4050 con 0x7f40140cf160 2023-11-09T03:47:03.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.469+0000 7f401b67c640 1 -- 172.21.15.76:0/574465136 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4014074290 con 0x7f40140cf160 2023-11-09T03:47:03.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.470+0000 7f4009ffb640 1 -- 172.21.15.76:0/574465136 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f400404e450 con 0x7f40140cf160 2023-11-09T03:47:03.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.470+0000 7f4009ffb640 1 --2- 172.21.15.76:0/574465136 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3ff806ae70 0x7f3ff806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:03.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.470+0000 7f4009ffb640 1 -- 172.21.15.76:0/574465136 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6438+0+0 (secure 0 0 0) 0x7f40040abb20 con 0x7f40140cf160 2023-11-09T03:47:03.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.470+0000 7f4018c17640 1 --2- 172.21.15.76:0/574465136 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3ff806ae70 0x7f3ff806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:03.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.471+0000 7f4018c17640 1 --2- 172.21.15.76:0/574465136 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3ff806ae70 0x7f3ff806d330 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4014159f00 tx=0x7f4010007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:03.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.474+0000 7f4009ffb640 1 -- 172.21.15.76:0/574465136 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4004076980 con 0x7f40140cf160 2023-11-09T03:47:03.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.642+0000 7f401b67c640 1 -- 172.21.15.76:0/574465136 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f40140b57e0 con 0x7f40140cf160 2023-11-09T03:47:03.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.642+0000 7f4009ffb640 1 -- 172.21.15.76:0/574465136 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4004079fa0 con 0x7f40140cf160 2023-11-09T03:47:03.641 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:47:03.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.644+0000 7f401b67c640 1 -- 172.21.15.76:0/574465136 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3ff806ae70 msgr2=0x7f3ff806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:03.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.644+0000 7f401b67c640 1 --2- 172.21.15.76:0/574465136 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3ff806ae70 0x7f3ff806d330 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4014159f00 tx=0x7f4010007c00 comp rx=0 tx=0).stop 2023-11-09T03:47:03.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.644+0000 7f401b67c640 1 -- 172.21.15.76:0/574465136 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40140cf160 msgr2=0x7f40140aefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:03.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.644+0000 7f401b67c640 1 --2- 172.21.15.76:0/574465136 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40140cf160 0x7f40140aefc0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f40040098c0 tx=0x7f40040046c0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.644+0000 7f401b67c640 1 -- 172.21.15.76:0/574465136 shutdown_connections 2023-11-09T03:47:03.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.644+0000 7f401b67c640 1 --2- 172.21.15.76:0/574465136 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3ff806ae70 0x7f3ff806d330 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.644+0000 7f401b67c640 1 --2- 172.21.15.76:0/574465136 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f401415e430 0x7f40140b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.644+0000 7f401b67c640 1 --2- 172.21.15.76:0/574465136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40140d0710 0x7f40141f39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.644+0000 7f401b67c640 1 --2- 172.21.15.76:0/574465136 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40140cf160 0x7f40140aefc0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.644+0000 7f401b67c640 1 -- 172.21.15.76:0/574465136 >> 172.21.15.76:0/574465136 conn(0x7f401406ea20 msgr2=0x7f40140cb3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:03.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.644+0000 7f401b67c640 1 -- 172.21.15.76:0/574465136 shutdown_connections 2023-11-09T03:47:03.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.644+0000 7f401b67c640 1 -- 172.21.15.76:0/574465136 wait complete. 2023-11-09T03:47:03.654 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:47:03.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.757+0000 7f0a39b8a640 1 Processor -- start 2023-11-09T03:47:03.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.757+0000 7f0a39b8a640 1 -- start start 2023-11-09T03:47:03.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.758+0000 7f0a39b8a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0a34070e20 0x7f0a340adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:03.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.758+0000 7f0a39b8a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a340ad730 0x7f0a3415e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:03.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.758+0000 7f0a39b8a640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a340c3900 con 0x7f0a3415ad30 2023-11-09T03:47:03.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.758+0000 7f0a39b8a640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a340ae260 con 0x7f0a34070e20 2023-11-09T03:47:03.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.758+0000 7f0a39b8a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a340c3be0 con 0x7f0a340ad730 2023-11-09T03:47:03.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.758+0000 7f0a337fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0a3415ad30 0x7f0a34070710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37352/0 (socket says 172.21.15.76:37352) 2023-11-09T03:47:03.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.758+0000 7f0a337fe640 1 -- 172.21.15.76:0/2038129695 learned_addr learned my addr 172.21.15.76:0/2038129695 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:03.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.758+0000 7f0a32ffd640 1 --2- 172.21.15.76:0/2038129695 >> v2:172.21.15.76:3300/0 conn(0x7f0a34070e20 0x7f0a340adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:03.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.758+0000 7f0a327fc640 1 -- 172.21.15.76:0/2038129695 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3491022150 0 0) 0x7f0a340c3900 con 0x7f0a3415ad30 2023-11-09T03:47:03.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.758+0000 7f0a327fc640 1 -- 172.21.15.76:0/2038129695 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a1c003520 con 0x7f0a3415ad30 2023-11-09T03:47:03.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.759+0000 7f0a33fff640 1 --2- 172.21.15.76:0/2038129695 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a340ad730 0x7f0a3415e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:03.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.759+0000 7f0a32ffd640 1 -- 172.21.15.76:0/2038129695 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a340ad730 msgr2=0x7f0a3415e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:03.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.759+0000 7f0a32ffd640 1 --2- 172.21.15.76:0/2038129695 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a340ad730 0x7f0a3415e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.759+0000 7f0a32ffd640 1 -- 172.21.15.76:0/2038129695 >> v1:172.21.15.76:6789/0 conn(0x7f0a3415ad30 legacy=0x7f0a34070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:03.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.759+0000 7f0a32ffd640 1 -- 172.21.15.76:0/2038129695 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a340c3ec0 con 0x7f0a34070e20 2023-11-09T03:47:03.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.759+0000 7f0a32ffd640 1 --2- 172.21.15.76:0/2038129695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a34070e20 0x7f0a340adc80 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0a280097a0 tx=0x7f0a2802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e594fa879b90b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:03.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.759+0000 7f0a327fc640 1 -- 172.21.15.76:0/2038129695 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2802e070 con 0x7f0a34070e20 2023-11-09T03:47:03.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.759+0000 7f0a327fc640 1 -- 172.21.15.76:0/2038129695 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a280315a0 con 0x7f0a34070e20 2023-11-09T03:47:03.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.759+0000 7f0a327fc640 1 -- 172.21.15.76:0/2038129695 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a28031930 con 0x7f0a34070e20 2023-11-09T03:47:03.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.760+0000 7f0a39b8a640 1 -- 172.21.15.76:0/2038129695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a34070e20 msgr2=0x7f0a340adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:03.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.760+0000 7f0a39b8a640 1 --2- 172.21.15.76:0/2038129695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a34070e20 0x7f0a340adc80 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0a280097a0 tx=0x7f0a2802b840 comp rx=0 tx=0).stop 2023-11-09T03:47:03.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.760+0000 7f0a39b8a640 1 -- 172.21.15.76:0/2038129695 shutdown_connections 2023-11-09T03:47:03.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.760+0000 7f0a39b8a640 1 --2- 172.21.15.76:0/2038129695 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a340ad730 0x7f0a3415e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.760+0000 7f0a39b8a640 1 --2- 172.21.15.76:0/2038129695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a34070e20 0x7f0a340adc80 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.760+0000 7f0a39b8a640 1 -- 172.21.15.76:0/2038129695 >> 172.21.15.76:0/2038129695 conn(0x7f0a340ce950 msgr2=0x7f0a340ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:03.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.760+0000 7f0a39b8a640 1 -- 172.21.15.76:0/2038129695 shutdown_connections 2023-11-09T03:47:03.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.760+0000 7f0a39b8a640 1 -- 172.21.15.76:0/2038129695 wait complete. 2023-11-09T03:47:03.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.761+0000 7f0a39b8a640 1 Processor -- start 2023-11-09T03:47:03.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.761+0000 7f0a39b8a640 1 -- start start 2023-11-09T03:47:03.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.762+0000 7f0a39b8a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a34070e20 0x7f0a340c8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:03.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.762+0000 7f0a39b8a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a340ad730 0x7f0a34147370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:03.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.762+0000 7f0a39b8a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a3415ad30 0x7f0a34159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:03.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.762+0000 7f0a39b8a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a340c8860 con 0x7f0a3415ad30 2023-11-09T03:47:03.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.762+0000 7f0a39b8a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3415b320 con 0x7f0a340ad730 2023-11-09T03:47:03.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.762+0000 7f0a39b8a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3415b600 con 0x7f0a34070e20 2023-11-09T03:47:03.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.762+0000 7f0a33fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a3415ad30 0x7f0a34159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:03.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.762+0000 7f0a33fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a3415ad30 0x7f0a34159370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40312/0 (socket says 172.21.15.76:40312) 2023-11-09T03:47:03.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.762+0000 7f0a33fff640 1 -- 172.21.15.76:0/3790705214 learned_addr learned my addr 172.21.15.76:0/3790705214 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:03.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.762+0000 7f0a337fe640 1 --2- 172.21.15.76:0/3790705214 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a34070e20 0x7f0a340c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:03.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.762+0000 7f0a33fff640 1 -- 172.21.15.76:0/3790705214 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a34070e20 msgr2=0x7f0a340c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:03.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.762+0000 7f0a33fff640 1 --2- 172.21.15.76:0/3790705214 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a34070e20 0x7f0a340c8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.762+0000 7f0a32ffd640 1 --2- 172.21.15.76:0/3790705214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a340ad730 0x7f0a34147370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:03.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.762+0000 7f0a33fff640 1 -- 172.21.15.76:0/3790705214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a340ad730 msgr2=0x7f0a34147370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:03.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.762+0000 7f0a33fff640 1 --2- 172.21.15.76:0/3790705214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a340ad730 0x7f0a34147370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.762+0000 7f0a33fff640 1 -- 172.21.15.76:0/3790705214 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a28009400 con 0x7f0a3415ad30 2023-11-09T03:47:03.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.763+0000 7f0a32ffd640 1 --2- 172.21.15.76:0/3790705214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a340ad730 0x7f0a34147370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:03.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.763+0000 7f0a33fff640 1 --2- 172.21.15.76:0/3790705214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a3415ad30 0x7f0a34159370 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c007e40 tx=0x7f0a2c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:03.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.763+0000 7f0a1bfff640 1 -- 172.21.15.76:0/3790705214 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c014620 con 0x7f0a3415ad30 2023-11-09T03:47:03.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.763+0000 7f0a1bfff640 1 -- 172.21.15.76:0/3790705214 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2c00ee90 con 0x7f0a3415ad30 2023-11-09T03:47:03.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.763+0000 7f0a39b8a640 1 -- 172.21.15.76:0/3790705214 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3415b8e0 con 0x7f0a3415ad30 2023-11-09T03:47:03.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.763+0000 7f0a1bfff640 1 -- 172.21.15.76:0/3790705214 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c0149e0 con 0x7f0a3415ad30 2023-11-09T03:47:03.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.763+0000 7f0a39b8a640 1 -- 172.21.15.76:0/3790705214 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3415bea0 con 0x7f0a3415ad30 2023-11-09T03:47:03.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.764+0000 7f0a39b8a640 1 -- 172.21.15.76:0/3790705214 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a34074290 con 0x7f0a3415ad30 2023-11-09T03:47:03.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.764+0000 7f0a1bfff640 1 -- 172.21.15.76:0/3790705214 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0a2c014b40 con 0x7f0a3415ad30 2023-11-09T03:47:03.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.764+0000 7f0a1bfff640 1 --2- 172.21.15.76:0/3790705214 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a2006aea0 0x7f0a2006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:03.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.765+0000 7f0a1bfff640 1 -- 172.21.15.76:0/3790705214 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6438+0+0 (secure 0 0 0) 0x7f0a2c08a2a0 con 0x7f0a3415ad30 2023-11-09T03:47:03.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.765+0000 7f0a337fe640 1 --2- 172.21.15.76:0/3790705214 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a2006aea0 0x7f0a2006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:03.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.765+0000 7f0a337fe640 1 --2- 172.21.15.76:0/3790705214 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a2006aea0 0x7f0a2006d360 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0a240011e0 tx=0x7f0a24009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:03.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.769+0000 7f0a1bfff640 1 -- 172.21.15.76:0/3790705214 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0a2c055070 con 0x7f0a3415ad30 2023-11-09T03:47:03.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.937+0000 7f0a39b8a640 1 -- 172.21.15.76:0/3790705214 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0a34071970 con 0x7f0a3415ad30 2023-11-09T03:47:03.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.937+0000 7f0a1bfff640 1 -- 172.21.15.76:0/3790705214 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0a2c058690 con 0x7f0a3415ad30 2023-11-09T03:47:03.937 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:47:03.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.939+0000 7f0a39b8a640 1 -- 172.21.15.76:0/3790705214 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a2006aea0 msgr2=0x7f0a2006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:03.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.939+0000 7f0a39b8a640 1 --2- 172.21.15.76:0/3790705214 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a2006aea0 0x7f0a2006d360 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0a240011e0 tx=0x7f0a24009040 comp rx=0 tx=0).stop 2023-11-09T03:47:03.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.939+0000 7f0a39b8a640 1 -- 172.21.15.76:0/3790705214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a3415ad30 msgr2=0x7f0a34159370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:03.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.939+0000 7f0a39b8a640 1 --2- 172.21.15.76:0/3790705214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a3415ad30 0x7f0a34159370 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c007e40 tx=0x7f0a2c00e4b0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.939+0000 7f0a39b8a640 1 -- 172.21.15.76:0/3790705214 shutdown_connections 2023-11-09T03:47:03.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.939+0000 7f0a39b8a640 1 --2- 172.21.15.76:0/3790705214 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a2006aea0 0x7f0a2006d360 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.939+0000 7f0a39b8a640 1 --2- 172.21.15.76:0/3790705214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a3415ad30 0x7f0a34159370 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.939+0000 7f0a39b8a640 1 --2- 172.21.15.76:0/3790705214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a340ad730 0x7f0a34147370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.939+0000 7f0a39b8a640 1 --2- 172.21.15.76:0/3790705214 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a34070e20 0x7f0a340c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:03.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.939+0000 7f0a39b8a640 1 -- 172.21.15.76:0/3790705214 >> 172.21.15.76:0/3790705214 conn(0x7f0a340ce950 msgr2=0x7f0a340ca2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:03.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.939+0000 7f0a39b8a640 1 -- 172.21.15.76:0/3790705214 shutdown_connections 2023-11-09T03:47:03.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:03.939+0000 7f0a39b8a640 1 -- 172.21.15.76:0/3790705214 wait complete. 2023-11-09T03:47:03.949 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:47:04.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.052+0000 7fb03f1fb640 1 Processor -- start 2023-11-09T03:47:04.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.052+0000 7fb03f1fb640 1 -- start start 2023-11-09T03:47:04.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.052+0000 7fb03f1fb640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb03815db40 0x7fb038071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:04.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.052+0000 7fb03f1fb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0380cc700 0x7fb0380ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:04.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.052+0000 7fb03f1fb640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0380cddf0 con 0x7fb038159390 2023-11-09T03:47:04.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.052+0000 7fb03f1fb640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0380cd0f0 con 0x7fb03815db40 2023-11-09T03:47:04.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.052+0000 7fb03f1fb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0380cdc00 con 0x7fb0380cc700 2023-11-09T03:47:04.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.053+0000 7fb037fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb03815db40 0x7fb038071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:04.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.053+0000 7fb03cf97640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb038159390 0x7fb0380cd4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37356/0 (socket says 172.21.15.76:37356) 2023-11-09T03:47:04.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.053+0000 7fb03cf97640 1 -- 172.21.15.76:0/1915621445 learned_addr learned my addr 172.21.15.76:0/1915621445 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:04.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.053+0000 7fb03d798640 1 --2- 172.21.15.76:0/1915621445 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0380cc700 0x7fb0380ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:04.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.053+0000 7fb037fff640 1 -- 172.21.15.76:0/1915621445 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0380cc700 msgr2=0x7fb0380ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:04.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.053+0000 7fb037fff640 1 --2- 172.21.15.76:0/1915621445 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0380cc700 0x7fb0380ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.053+0000 7fb037fff640 1 -- 172.21.15.76:0/1915621445 >> v1:172.21.15.76:6789/0 conn(0x7fb038159390 legacy=0x7fb0380cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:04.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.053+0000 7fb037fff640 1 -- 172.21.15.76:0/1915621445 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb038070740 con 0x7fb03815db40 2023-11-09T03:47:04.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.053+0000 7fb0377fe640 1 -- 172.21.15.76:0/1915621445 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3101336614 0 0) 0x7fb0380cddf0 con 0x7fb038159390 2023-11-09T03:47:04.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.053+0000 7fb03d798640 1 --2- 172.21.15.76:0/1915621445 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0380cc700 0x7fb0380ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:04.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.054+0000 7fb037fff640 1 --2- 172.21.15.76:0/1915621445 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb03815db40 0x7fb038071220 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb0200097d0 tx=0x7fb02002b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19d0d7ad54eebff server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:04.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.054+0000 7fb0377fe640 1 -- 172.21.15.76:0/1915621445 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb020031800 con 0x7fb03815db40 2023-11-09T03:47:04.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.054+0000 7fb0377fe640 1 -- 172.21.15.76:0/1915621445 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb020031960 con 0x7fb03815db40 2023-11-09T03:47:04.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.054+0000 7fb0377fe640 1 -- 172.21.15.76:0/1915621445 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb020031cf0 con 0x7fb03815db40 2023-11-09T03:47:04.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.054+0000 7fb03f1fb640 1 -- 172.21.15.76:0/1915621445 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb03815db40 msgr2=0x7fb038071220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:04.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.054+0000 7fb03f1fb640 1 --2- 172.21.15.76:0/1915621445 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb03815db40 0x7fb038071220 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb0200097d0 tx=0x7fb02002b8a0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.054+0000 7fb03f1fb640 1 -- 172.21.15.76:0/1915621445 shutdown_connections 2023-11-09T03:47:04.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.054+0000 7fb03f1fb640 1 --2- 172.21.15.76:0/1915621445 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0380cc700 0x7fb0380ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.054+0000 7fb03f1fb640 1 --2- 172.21.15.76:0/1915621445 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb03815db40 0x7fb038071220 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.054+0000 7fb03f1fb640 1 -- 172.21.15.76:0/1915621445 >> 172.21.15.76:0/1915621445 conn(0x7fb0380b1c50 msgr2=0x7fb0380b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:04.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.054+0000 7fb03f1fb640 1 -- 172.21.15.76:0/1915621445 shutdown_connections 2023-11-09T03:47:04.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.054+0000 7fb03f1fb640 1 -- 172.21.15.76:0/1915621445 wait complete. 2023-11-09T03:47:04.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.055+0000 7fb03f1fb640 1 Processor -- start 2023-11-09T03:47:04.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.055+0000 7fb03f1fb640 1 -- start start 2023-11-09T03:47:04.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.056+0000 7fb03f1fb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0380cc700 0x7fb0380c1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:04.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.056+0000 7fb03f1fb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb038159390 0x7fb0381440b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:04.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.056+0000 7fb03f1fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb03815db40 0x7fb0381445f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:04.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.056+0000 7fb03f1fb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03813a760 con 0x7fb03815db40 2023-11-09T03:47:04.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.056+0000 7fb03f1fb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03813aa40 con 0x7fb038159390 2023-11-09T03:47:04.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.056+0000 7fb03f1fb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03814bf60 con 0x7fb0380cc700 2023-11-09T03:47:04.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.056+0000 7fb03d798640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb03815db40 0x7fb0381445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:04.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.056+0000 7fb03d798640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb03815db40 0x7fb0381445f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40338/0 (socket says 172.21.15.76:40338) 2023-11-09T03:47:04.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.056+0000 7fb03d798640 1 -- 172.21.15.76:0/1233655521 learned_addr learned my addr 172.21.15.76:0/1233655521 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:04.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.057+0000 7fb03cf97640 1 --2- 172.21.15.76:0/1233655521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0380cc700 0x7fb0380c1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:04.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.057+0000 7fb037fff640 1 --2- 172.21.15.76:0/1233655521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb038159390 0x7fb0381440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:04.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.057+0000 7fb03d798640 1 -- 172.21.15.76:0/1233655521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0380cc700 msgr2=0x7fb0380c1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:04.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.057+0000 7fb03d798640 1 --2- 172.21.15.76:0/1233655521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0380cc700 0x7fb0380c1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.057+0000 7fb03d798640 1 -- 172.21.15.76:0/1233655521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb038159390 msgr2=0x7fb0381440b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:04.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.057+0000 7fb03d798640 1 --2- 172.21.15.76:0/1233655521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb038159390 0x7fb0381440b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.057+0000 7fb03d798640 1 -- 172.21.15.76:0/1233655521 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb020009400 con 0x7fb03815db40 2023-11-09T03:47:04.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.057+0000 7fb03cf97640 1 --2- 172.21.15.76:0/1233655521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0380cc700 0x7fb0380c1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:04.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.057+0000 7fb037fff640 1 --2- 172.21.15.76:0/1233655521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb038159390 0x7fb0381440b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:04.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.057+0000 7fb03d798640 1 --2- 172.21.15.76:0/1233655521 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb03815db40 0x7fb0381445f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb03000cef0 tx=0x7fb03000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:04.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.057+0000 7fb0357fa640 1 -- 172.21.15.76:0/1233655521 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03000d840 con 0x7fb03815db40 2023-11-09T03:47:04.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.057+0000 7fb0357fa640 1 -- 172.21.15.76:0/1233655521 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03000d9c0 con 0x7fb03815db40 2023-11-09T03:47:04.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.057+0000 7fb03f1fb640 1 -- 172.21.15.76:0/1233655521 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb03814c240 con 0x7fb03815db40 2023-11-09T03:47:04.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.057+0000 7fb0357fa640 1 -- 172.21.15.76:0/1233655521 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03000dd70 con 0x7fb03815db40 2023-11-09T03:47:04.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.057+0000 7fb03f1fb640 1 -- 172.21.15.76:0/1233655521 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0380c18c0 con 0x7fb03815db40 2023-11-09T03:47:04.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.058+0000 7fb03f1fb640 1 -- 172.21.15.76:0/1233655521 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb03813abb0 con 0x7fb03815db40 2023-11-09T03:47:04.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.058+0000 7fb0357fa640 1 -- 172.21.15.76:0/1233655521 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb03001e3b0 con 0x7fb03815db40 2023-11-09T03:47:04.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.059+0000 7fb0357fa640 1 --2- 172.21.15.76:0/1233655521 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb01806ae70 0x7fb01806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:04.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.059+0000 7fb0357fa640 1 -- 172.21.15.76:0/1233655521 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6438+0+0 (secure 0 0 0) 0x7fb03008a340 con 0x7fb03815db40 2023-11-09T03:47:04.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.059+0000 7fb03cf97640 1 --2- 172.21.15.76:0/1233655521 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb01806ae70 0x7fb01806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:04.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.059+0000 7fb03cf97640 1 --2- 172.21.15.76:0/1233655521 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb01806ae70 0x7fb01806d330 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb028007fd0 tx=0x7fb028009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:04.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.063+0000 7fb0357fa640 1 -- 172.21.15.76:0/1233655521 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb030055260 con 0x7fb03815db40 2023-11-09T03:47:04.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.230+0000 7fb03f1fb640 1 -- 172.21.15.76:0/1233655521 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb038074290 con 0x7fb03815db40 2023-11-09T03:47:04.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.230+0000 7fb0357fa640 1 -- 172.21.15.76:0/1233655521 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb030058880 con 0x7fb03815db40 2023-11-09T03:47:04.230 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:47:04.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.232+0000 7fb03f1fb640 1 -- 172.21.15.76:0/1233655521 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb01806ae70 msgr2=0x7fb01806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:04.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.232+0000 7fb03f1fb640 1 --2- 172.21.15.76:0/1233655521 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb01806ae70 0x7fb01806d330 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb028007fd0 tx=0x7fb028009040 comp rx=0 tx=0).stop 2023-11-09T03:47:04.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.232+0000 7fb03f1fb640 1 -- 172.21.15.76:0/1233655521 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb03815db40 msgr2=0x7fb0381445f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:04.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.232+0000 7fb03f1fb640 1 --2- 172.21.15.76:0/1233655521 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb03815db40 0x7fb0381445f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb03000cef0 tx=0x7fb03000a710 comp rx=0 tx=0).stop 2023-11-09T03:47:04.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.232+0000 7fb03f1fb640 1 -- 172.21.15.76:0/1233655521 shutdown_connections 2023-11-09T03:47:04.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.232+0000 7fb03f1fb640 1 --2- 172.21.15.76:0/1233655521 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb01806ae70 0x7fb01806d330 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.232+0000 7fb03f1fb640 1 --2- 172.21.15.76:0/1233655521 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb03815db40 0x7fb0381445f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.232+0000 7fb03f1fb640 1 --2- 172.21.15.76:0/1233655521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb038159390 0x7fb0381440b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.232+0000 7fb03f1fb640 1 --2- 172.21.15.76:0/1233655521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0380cc700 0x7fb0380c1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.232+0000 7fb03f1fb640 1 -- 172.21.15.76:0/1233655521 >> 172.21.15.76:0/1233655521 conn(0x7fb0380b1c50 msgr2=0x7fb0380b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:04.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.232+0000 7fb03f1fb640 1 -- 172.21.15.76:0/1233655521 shutdown_connections 2023-11-09T03:47:04.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.232+0000 7fb03f1fb640 1 -- 172.21.15.76:0/1233655521 wait complete. 2023-11-09T03:47:04.242 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-09T03:47:04.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.346+0000 7f8aeebf5640 1 Processor -- start 2023-11-09T03:47:04.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.347+0000 7f8aeebf5640 1 -- start start 2023-11-09T03:47:04.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.347+0000 7f8aeebf5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ae8153e60 0x7f8ae815dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:04.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.347+0000 7f8aeebf5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8ae80b4880 0x7f8ae80b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:04.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.347+0000 7f8aeebf5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae80cde60 con 0x7f8ae80cd6d0 2023-11-09T03:47:04.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.347+0000 7f8aeebf5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae80b5230 con 0x7f8ae80b4880 2023-11-09T03:47:04.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.347+0000 7f8aeebf5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae8160f90 con 0x7f8ae8153e60 2023-11-09T03:47:04.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.347+0000 7f8ae7fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8ae80cd6d0 0x7f8ae8161420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37360/0 (socket says 172.21.15.76:37360) 2023-11-09T03:47:04.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.347+0000 7f8ae7fff640 1 -- 172.21.15.76:0/671846062 learned_addr learned my addr 172.21.15.76:0/671846062 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:04.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.348+0000 7f8aed192640 1 --2- 172.21.15.76:0/671846062 >> v2:172.21.15.113:3300/0 conn(0x7f8ae80b4880 0x7f8ae80b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:04.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.348+0000 7f8aec991640 1 --2- 172.21.15.76:0/671846062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ae8153e60 0x7f8ae815dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:04.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.348+0000 7f8ae77fe640 1 -- 172.21.15.76:0/671846062 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3283219879 0 0) 0x7f8ae80cde60 con 0x7f8ae80cd6d0 2023-11-09T03:47:04.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.348+0000 7f8ae77fe640 1 -- 172.21.15.76:0/671846062 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ad8003520 con 0x7f8ae80cd6d0 2023-11-09T03:47:04.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.348+0000 7f8ae77fe640 1 -- 172.21.15.76:0/671846062 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 602724050 0 0) 0x7f8ad8003520 con 0x7f8ae80cd6d0 2023-11-09T03:47:04.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.348+0000 7f8ae77fe640 1 -- 172.21.15.76:0/671846062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ae8153e60 msgr2=0x7f8ae815dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:04.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.348+0000 7f8ae77fe640 1 --2- 172.21.15.76:0/671846062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ae8153e60 0x7f8ae815dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.348+0000 7f8ae77fe640 1 -- 172.21.15.76:0/671846062 >> v2:172.21.15.113:3300/0 conn(0x7f8ae80b4880 msgr2=0x7f8ae80b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:04.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.348+0000 7f8ae77fe640 1 --2- 172.21.15.76:0/671846062 >> v2:172.21.15.113:3300/0 conn(0x7f8ae80b4880 0x7f8ae80b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.348+0000 7f8ae77fe640 1 -- 172.21.15.76:0/671846062 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae815e400 con 0x7f8ae80cd6d0 2023-11-09T03:47:04.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.348+0000 7f8ae77fe640 1 -- 172.21.15.76:0/671846062 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f8ad4001670 con 0x7f8ae80cd6d0 2023-11-09T03:47:04.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.348+0000 7f8ae77fe640 1 -- 172.21.15.76:0/671846062 >> v1:172.21.15.76:6789/0 conn(0x7f8ae80cd6d0 legacy=0x7f8ae8161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:04.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.348+0000 7f8ae77fe640 1 --2- 172.21.15.76:0/671846062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ad8003970 0x7f8ad8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:04.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.348+0000 7f8ae77fe640 1 --2- 172.21.15.76:0/671846062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ae8153e60 0x7f8ad80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:04.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.348+0000 7f8ae77fe640 1 --2- 172.21.15.76:0/671846062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ad8004d00 0x7f8ad8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:04.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.348+0000 7f8ae77fe640 1 -- 172.21.15.76:0/671846062 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae80b5230 con 0x7f8ad8003970 2023-11-09T03:47:04.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.348+0000 7f8ae77fe640 1 -- 172.21.15.76:0/671846062 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae8160f90 con 0x7f8ad8004d00 2023-11-09T03:47:04.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.348+0000 7f8ae77fe640 1 -- 172.21.15.76:0/671846062 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad800b980 con 0x7f8ae8153e60 2023-11-09T03:47:04.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.348+0000 7f8aec991640 1 --2- 172.21.15.76:0/671846062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ad8003970 0x7f8ad8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:04.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.348+0000 7f8ae7fff640 1 --2- 172.21.15.76:0/671846062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ae8153e60 0x7f8ad80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:04.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.348+0000 7f8aed192640 1 --2- 172.21.15.76:0/671846062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ad8004d00 0x7f8ad8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:04.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.348+0000 7f8aec991640 1 -- 172.21.15.76:0/671846062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ae8153e60 msgr2=0x7f8ad80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:04.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.348+0000 7f8aec991640 1 --2- 172.21.15.76:0/671846062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ae8153e60 0x7f8ad80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.348+0000 7f8aec991640 1 -- 172.21.15.76:0/671846062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ad8004d00 msgr2=0x7f8ad8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:04.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.348+0000 7f8aec991640 1 --2- 172.21.15.76:0/671846062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ad8004d00 0x7f8ad8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.349+0000 7f8aec991640 1 -- 172.21.15.76:0/671846062 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad8003520 con 0x7f8ad8003970 2023-11-09T03:47:04.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.349+0000 7f8ae7fff640 1 --2- 172.21.15.76:0/671846062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ae8153e60 0x7f8ad80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:04.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.349+0000 7f8aec991640 1 --2- 172.21.15.76:0/671846062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ad8003970 0x7f8ad8003d90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8adc009ac0 tx=0x7f8ad80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c8a3272d7fcaac server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:04.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.349+0000 7f8ae77fe640 1 -- 172.21.15.76:0/671846062 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc02c6b0 con 0x7f8ad8003970 2023-11-09T03:47:04.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.349+0000 7f8ae77fe640 1 -- 172.21.15.76:0/671846062 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8adc02c810 con 0x7f8ad8003970 2023-11-09T03:47:04.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.349+0000 7f8ae77fe640 1 -- 172.21.15.76:0/671846062 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc02cba0 con 0x7f8ad8003970 2023-11-09T03:47:04.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.350+0000 7f8aeebf5640 1 -- 172.21.15.76:0/671846062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ad8003970 msgr2=0x7f8ad8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:04.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.350+0000 7f8aeebf5640 1 --2- 172.21.15.76:0/671846062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ad8003970 0x7f8ad8003d90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8adc009ac0 tx=0x7f8ad80057f0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.350+0000 7f8aed192640 1 -- 172.21.15.76:0/671846062 reap_dead start 2023-11-09T03:47:04.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.350+0000 7f8aeebf5640 1 -- 172.21.15.76:0/671846062 shutdown_connections 2023-11-09T03:47:04.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.350+0000 7f8aeebf5640 1 -- 172.21.15.76:0/671846062 >> 172.21.15.76:0/671846062 conn(0x7f8ae80cf880 msgr2=0x7f8ae80cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:04.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.350+0000 7f8aeebf5640 1 -- 172.21.15.76:0/671846062 shutdown_connections 2023-11-09T03:47:04.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.350+0000 7f8aeebf5640 1 -- 172.21.15.76:0/671846062 wait complete. 2023-11-09T03:47:04.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.351+0000 7f8aeebf5640 1 Processor -- start 2023-11-09T03:47:04.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.351+0000 7f8aeebf5640 1 -- start start 2023-11-09T03:47:04.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.352+0000 7f8aeebf5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ad8003970 0x7f8ae8152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:04.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.352+0000 7f8aeebf5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ae80c8b00 0x7f8ae80c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:04.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.352+0000 7f8aeebf5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ae81f2d60 0x7f8ae81f31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:04.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.352+0000 7f8aeebf5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae8154d10 con 0x7f8ae81f2d60 2023-11-09T03:47:04.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.352+0000 7f8aeebf5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae80c94c0 con 0x7f8ae80c8b00 2023-11-09T03:47:04.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.352+0000 7f8aeebf5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae81f3a90 con 0x7f8ad8003970 2023-11-09T03:47:04.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.352+0000 7f8aed192640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ae81f2d60 0x7f8ae81f31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:04.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.352+0000 7f8aed192640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ae81f2d60 0x7f8ae81f31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40342/0 (socket says 172.21.15.76:40342) 2023-11-09T03:47:04.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.352+0000 7f8aed192640 1 -- 172.21.15.76:0/3504185708 learned_addr learned my addr 172.21.15.76:0/3504185708 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:04.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.352+0000 7f8ae7fff640 1 --2- 172.21.15.76:0/3504185708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ae80c8b00 0x7f8ae80c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:04.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.352+0000 7f8aec991640 1 --2- 172.21.15.76:0/3504185708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ad8003970 0x7f8ae8152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:04.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.352+0000 7f8aed192640 1 -- 172.21.15.76:0/3504185708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ad8003970 msgr2=0x7f8ae8152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:04.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.352+0000 7f8aed192640 1 --2- 172.21.15.76:0/3504185708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ad8003970 0x7f8ae8152d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.352+0000 7f8aed192640 1 -- 172.21.15.76:0/3504185708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ae80c8b00 msgr2=0x7f8ae80c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:04.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.352+0000 7f8aed192640 1 --2- 172.21.15.76:0/3504185708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ae80c8b00 0x7f8ae80c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.352+0000 7f8aed192640 1 -- 172.21.15.76:0/3504185708 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae8154370 con 0x7f8ae81f2d60 2023-11-09T03:47:04.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.353+0000 7f8aed192640 1 --2- 172.21.15.76:0/3504185708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ae81f2d60 0x7f8ae81f31e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f8ae80c6900 tx=0x7f8ae000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:04.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.353+0000 7f8ae57fa640 1 -- 172.21.15.76:0/3504185708 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae000e040 con 0x7f8ae81f2d60 2023-11-09T03:47:04.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.353+0000 7f8ae57fa640 1 -- 172.21.15.76:0/3504185708 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae000f040 con 0x7f8ae81f2d60 2023-11-09T03:47:04.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.353+0000 7f8aeebf5640 1 -- 172.21.15.76:0/3504185708 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae8154650 con 0x7f8ae81f2d60 2023-11-09T03:47:04.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.353+0000 7f8ae57fa640 1 -- 172.21.15.76:0/3504185708 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0015430 con 0x7f8ae81f2d60 2023-11-09T03:47:04.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.353+0000 7f8aeebf5640 1 -- 172.21.15.76:0/3504185708 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae81f3720 con 0x7f8ae81f2d60 2023-11-09T03:47:04.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.354+0000 7f8aeebf5640 1 -- 172.21.15.76:0/3504185708 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae8154e50 con 0x7f8ae81f2d60 2023-11-09T03:47:04.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.354+0000 7f8ae57fa640 1 -- 172.21.15.76:0/3504185708 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8ae0015590 con 0x7f8ae81f2d60 2023-11-09T03:47:04.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.355+0000 7f8ae57fa640 1 --2- 172.21.15.76:0/3504185708 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8acc06aea0 0x7f8acc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:04.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.355+0000 7f8ae57fa640 1 -- 172.21.15.76:0/3504185708 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6438+0+0 (secure 0 0 0) 0x7f8ae008a1b0 con 0x7f8ae81f2d60 2023-11-09T03:47:04.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.355+0000 7f8aec991640 1 --2- 172.21.15.76:0/3504185708 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8acc06aea0 0x7f8acc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:04.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.355+0000 7f8aec991640 1 --2- 172.21.15.76:0/3504185708 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8acc06aea0 0x7f8acc06d360 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8adc036000 tx=0x7f8adc030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:04.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.359+0000 7f8ae57fa640 1 -- 172.21.15.76:0/3504185708 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8ae8154e50 con 0x7f8ae81f2d60 2023-11-09T03:47:04.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.527+0000 7f8aeebf5640 1 -- 172.21.15.76:0/3504185708 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f8ae8154e50 con 0x7f8ae81f2d60 2023-11-09T03:47:04.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.527+0000 7f8ae57fa640 1 -- 172.21.15.76:0/3504185708 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v42) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8ae8154e50 con 0x7f8ae81f2d60 2023-11-09T03:47:04.527 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:47:04.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.529+0000 7f8aeebf5640 1 -- 172.21.15.76:0/3504185708 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8acc06aea0 msgr2=0x7f8acc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:04.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.529+0000 7f8aeebf5640 1 --2- 172.21.15.76:0/3504185708 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8acc06aea0 0x7f8acc06d360 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8adc036000 tx=0x7f8adc030000 comp rx=0 tx=0).stop 2023-11-09T03:47:04.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.529+0000 7f8aeebf5640 1 -- 172.21.15.76:0/3504185708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ae81f2d60 msgr2=0x7f8ae81f31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:04.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.529+0000 7f8aeebf5640 1 --2- 172.21.15.76:0/3504185708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ae81f2d60 0x7f8ae81f31e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f8ae80c6900 tx=0x7f8ae000f580 comp rx=0 tx=0).stop 2023-11-09T03:47:04.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.529+0000 7f8aeebf5640 1 -- 172.21.15.76:0/3504185708 shutdown_connections 2023-11-09T03:47:04.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.529+0000 7f8aeebf5640 1 --2- 172.21.15.76:0/3504185708 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8acc06aea0 0x7f8acc06d360 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.529+0000 7f8aeebf5640 1 --2- 172.21.15.76:0/3504185708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ae81f2d60 0x7f8ae81f31e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.529+0000 7f8aeebf5640 1 --2- 172.21.15.76:0/3504185708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ae80c8b00 0x7f8ae80c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.529+0000 7f8aeebf5640 1 --2- 172.21.15.76:0/3504185708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ad8003970 0x7f8ae8152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.529+0000 7f8aeebf5640 1 -- 172.21.15.76:0/3504185708 >> 172.21.15.76:0/3504185708 conn(0x7f8ae80cf880 msgr2=0x7f8ae80b7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:04.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.529+0000 7f8aeebf5640 1 -- 172.21.15.76:0/3504185708 shutdown_connections 2023-11-09T03:47:04.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.529+0000 7f8aeebf5640 1 -- 172.21.15.76:0/3504185708 wait complete. 2023-11-09T03:47:04.540 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-09T03:47:04.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.641+0000 7fab1c9a9640 1 Processor -- start 2023-11-09T03:47:04.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.641+0000 7fab1c9a9640 1 -- start start 2023-11-09T03:47:04.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.642+0000 7fab1c9a9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fab1815ad30 0x7fab180adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:04.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.642+0000 7fab1c9a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab18070e20 0x7fab1815e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:04.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.642+0000 7fab1c9a9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab180c3900 con 0x7fab180ad730 2023-11-09T03:47:04.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.642+0000 7fab1c9a9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab180ae260 con 0x7fab1815ad30 2023-11-09T03:47:04.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.642+0000 7fab1c9a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab180c3be0 con 0x7fab18070e20 2023-11-09T03:47:04.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.642+0000 7fab1659c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fab1815ad30 0x7fab180adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:04.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.642+0000 7fab1659c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fab1815ad30 0x7fab180adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40348/0 (socket says 172.21.15.76:40348) 2023-11-09T03:47:04.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.642+0000 7fab1659c640 1 -- 172.21.15.76:0/1964170367 learned_addr learned my addr 172.21.15.76:0/1964170367 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:04.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.643+0000 7fab16d9d640 1 --2- 172.21.15.76:0/1964170367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab18070e20 0x7fab1815e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:04.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.643+0000 7fab1659c640 1 -- 172.21.15.76:0/1964170367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab18070e20 msgr2=0x7fab1815e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:04.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.643+0000 7fab1659c640 1 --2- 172.21.15.76:0/1964170367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab18070e20 0x7fab1815e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.643+0000 7fab1659c640 1 -- 172.21.15.76:0/1964170367 >> v1:172.21.15.113:6789/0 conn(0x7fab180ad730 legacy=0x7fab18070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:04.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.643+0000 7fab1659c640 1 -- 172.21.15.76:0/1964170367 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab180c3ec0 con 0x7fab1815ad30 2023-11-09T03:47:04.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.643+0000 7fab1659c640 1 --2- 172.21.15.76:0/1964170367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab1815ad30 0x7fab180adc80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fab08009e50 tx=0x7fab0802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278c482737797c23 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:04.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.644+0000 7fab1559a640 1 -- 172.21.15.76:0/1964170367 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0802c9e0 con 0x7fab1815ad30 2023-11-09T03:47:04.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.644+0000 7fab1559a640 1 -- 172.21.15.76:0/1964170367 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0802cb40 con 0x7fab1815ad30 2023-11-09T03:47:04.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.644+0000 7fab1559a640 1 -- 172.21.15.76:0/1964170367 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab080343f0 con 0x7fab1815ad30 2023-11-09T03:47:04.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.644+0000 7fab1c9a9640 1 -- 172.21.15.76:0/1964170367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab1815ad30 msgr2=0x7fab180adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:04.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.644+0000 7fab1c9a9640 1 --2- 172.21.15.76:0/1964170367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab1815ad30 0x7fab180adc80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fab08009e50 tx=0x7fab0802be40 comp rx=0 tx=0).stop 2023-11-09T03:47:04.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.644+0000 7fab1c9a9640 1 -- 172.21.15.76:0/1964170367 shutdown_connections 2023-11-09T03:47:04.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.644+0000 7fab1c9a9640 1 --2- 172.21.15.76:0/1964170367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab18070e20 0x7fab1815e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.644+0000 7fab1c9a9640 1 --2- 172.21.15.76:0/1964170367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab1815ad30 0x7fab180adc80 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.644+0000 7fab1c9a9640 1 -- 172.21.15.76:0/1964170367 >> 172.21.15.76:0/1964170367 conn(0x7fab180ce950 msgr2=0x7fab180ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:04.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.644+0000 7fab1c9a9640 1 -- 172.21.15.76:0/1964170367 shutdown_connections 2023-11-09T03:47:04.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.644+0000 7fab1c9a9640 1 -- 172.21.15.76:0/1964170367 wait complete. 2023-11-09T03:47:04.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.645+0000 7fab1c9a9640 1 Processor -- start 2023-11-09T03:47:04.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.645+0000 7fab1c9a9640 1 -- start start 2023-11-09T03:47:04.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.646+0000 7fab1c9a9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab18070e20 0x7fab18150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:04.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.646+0000 7fab1c9a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab180ad730 0x7fab180b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:04.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.646+0000 7fab1c9a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab1815ad30 0x7fab180cd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:04.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.646+0000 7fab1c9a9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab180ac510 con 0x7fab18070e20 2023-11-09T03:47:04.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.646+0000 7fab1c9a9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab18150eb0 con 0x7fab1815ad30 2023-11-09T03:47:04.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.646+0000 7fab1c9a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab180ab900 con 0x7fab180ad730 2023-11-09T03:47:04.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.646+0000 7fab1659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab18070e20 0x7fab18150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:04.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.646+0000 7fab1659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab18070e20 0x7fab18150970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40356/0 (socket says 172.21.15.76:40356) 2023-11-09T03:47:04.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.646+0000 7fab1659c640 1 -- 172.21.15.76:0/3728365962 learned_addr learned my addr 172.21.15.76:0/3728365962 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:04.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.646+0000 7fab1659c640 1 -- 172.21.15.76:0/3728365962 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab180ad730 msgr2=0x7fab180b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:04.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.646+0000 7fab1659c640 1 --2- 172.21.15.76:0/3728365962 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab180ad730 0x7fab180b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.646+0000 7fab1659c640 1 -- 172.21.15.76:0/3728365962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab1815ad30 msgr2=0x7fab180cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:04.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.646+0000 7fab1659c640 1 --2- 172.21.15.76:0/3728365962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab1815ad30 0x7fab180cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.647+0000 7fab1659c640 1 -- 172.21.15.76:0/3728365962 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab08009b30 con 0x7fab18070e20 2023-11-09T03:47:04.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.647+0000 7fab1659c640 1 --2- 172.21.15.76:0/3728365962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab18070e20 0x7fab18150970 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fab08009e50 tx=0x7fab08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:04.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.647+0000 7fab06ffd640 1 -- 172.21.15.76:0/3728365962 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab08009510 con 0x7fab18070e20 2023-11-09T03:47:04.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.647+0000 7fab06ffd640 1 -- 172.21.15.76:0/3728365962 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0803e450 con 0x7fab18070e20 2023-11-09T03:47:04.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.647+0000 7fab1c9a9640 1 -- 172.21.15.76:0/3728365962 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab180abbe0 con 0x7fab18070e20 2023-11-09T03:47:04.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.647+0000 7fab06ffd640 1 -- 172.21.15.76:0/3728365962 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0803e800 con 0x7fab18070e20 2023-11-09T03:47:04.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.647+0000 7fab1c9a9640 1 -- 172.21.15.76:0/3728365962 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab180ac1a0 con 0x7fab18070e20 2023-11-09T03:47:04.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.648+0000 7fab1c9a9640 1 -- 172.21.15.76:0/3728365962 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab1815b8b0 con 0x7fab18070e20 2023-11-09T03:47:04.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.648+0000 7fab06ffd640 1 -- 172.21.15.76:0/3728365962 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fab0803e960 con 0x7fab18070e20 2023-11-09T03:47:04.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.649+0000 7fab06ffd640 1 --2- 172.21.15.76:0/3728365962 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faaf806ae70 0x7faaf806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:04.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.649+0000 7fab06ffd640 1 -- 172.21.15.76:0/3728365962 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6438+0+0 (secure 0 0 0) 0x7fab080ac780 con 0x7fab18070e20 2023-11-09T03:47:04.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.649+0000 7fab15d9b640 1 --2- 172.21.15.76:0/3728365962 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faaf806ae70 0x7faaf806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:04.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.649+0000 7fab15d9b640 1 --2- 172.21.15.76:0/3728365962 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faaf806ae70 0x7faaf806d330 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fab180cd6d0 tx=0x7fab0c0079d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:04.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.653+0000 7fab06ffd640 1 -- 172.21.15.76:0/3728365962 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fab08077550 con 0x7fab18070e20 2023-11-09T03:47:04.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.831+0000 7fab1c9a9640 1 -- 172.21.15.76:0/3728365962 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fab18074290 con 0x7fab18070e20 2023-11-09T03:47:04.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.831+0000 7fab06ffd640 1 -- 172.21.15.76:0/3728365962 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) v1 ==== 80+0+0 (secure 0 0 0) 0x7fab0807ab70 con 0x7fab18070e20 2023-11-09T03:47:04.831 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2023-11-09T03:47:04.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.833+0000 7fab1c9a9640 1 -- 172.21.15.76:0/3728365962 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faaf806ae70 msgr2=0x7faaf806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:04.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.833+0000 7fab1c9a9640 1 --2- 172.21.15.76:0/3728365962 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faaf806ae70 0x7faaf806d330 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fab180cd6d0 tx=0x7fab0c0079d0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.833+0000 7fab1c9a9640 1 -- 172.21.15.76:0/3728365962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab18070e20 msgr2=0x7fab18150970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:04.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.833+0000 7fab1c9a9640 1 --2- 172.21.15.76:0/3728365962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab18070e20 0x7fab18150970 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fab08009e50 tx=0x7fab08004a40 comp rx=0 tx=0).stop 2023-11-09T03:47:04.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.833+0000 7fab1c9a9640 1 -- 172.21.15.76:0/3728365962 shutdown_connections 2023-11-09T03:47:04.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.833+0000 7fab1c9a9640 1 --2- 172.21.15.76:0/3728365962 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faaf806ae70 0x7faaf806d330 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.833+0000 7fab1c9a9640 1 --2- 172.21.15.76:0/3728365962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab1815ad30 0x7fab180cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.833+0000 7fab1c9a9640 1 --2- 172.21.15.76:0/3728365962 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab180ad730 0x7fab180b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.833+0000 7fab1c9a9640 1 --2- 172.21.15.76:0/3728365962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab18070e20 0x7fab18150970 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:04.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.833+0000 7fab1c9a9640 1 -- 172.21.15.76:0/3728365962 >> 172.21.15.76:0/3728365962 conn(0x7fab180ce950 msgr2=0x7fab1815b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:04.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.833+0000 7fab1c9a9640 1 -- 172.21.15.76:0/3728365962 shutdown_connections 2023-11-09T03:47:04.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04.833+0000 7fab1c9a9640 1 -- 172.21.15.76:0/3728365962 wait complete. 2023-11-09T03:47:04.843 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi076.front.sepia.ceph.com... 2023-11-09T03:47:04.843 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:47:04.844 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-09T03:47:04.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:47:04.902 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[104760]: fuse finished with error 0 and tester_r 0 2023-11-09T03:47:04.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:47:04.908 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:47:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:47:04.910 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T03:47:04.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:47:04.911 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-09T03:47:04.925 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:47:04.926 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:47:04.944 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2440 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-09T03:47:04.945 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2023-11-09T03:47:04.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:47:04.946 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-09T03:47:05.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:47:05.002 INFO:teuthology.orchestra.run.smithi113.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-09T03:47:05.002 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:47:05.002 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:47:05.018 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056381,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14761 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248868k,nr_inodes=812217,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-09T03:47:05.018 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T03:47:05.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:47:05.021 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-09T03:47:05.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:47:05.074 INFO:teuthology.orchestra.run.smithi076.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-09T03:47:05.074 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:47:05.075 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:47:05.094 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2440 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-09T03:47:05.094 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2023-11-09T03:47:05.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:47:05.095 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-09T03:47:05.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:47:05.152 INFO:teuthology.orchestra.run.smithi113.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-09T03:47:05.152 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:47:05.253 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.254+0000 7f47b787c640 1 Processor -- start 2023-11-09T03:47:05.253 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.254+0000 7f47b787c640 1 -- start start 2023-11-09T03:47:05.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.255+0000 7f47b787c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f47b00b0ae0 0x7f47b00ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:05.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.255+0000 7f47b787c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47b00b4110 con 0x7f47b006db20 2023-11-09T03:47:05.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.255+0000 7f47b787c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47b00b43f0 con 0x7f47b0159c70 2023-11-09T03:47:05.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.255+0000 7f47b787c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47b00b49b0 con 0x7f47b00b0ae0 2023-11-09T03:47:05.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.255+0000 7f47b4e17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f47b00b0ae0 0x7f47b00ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:05.264 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.255+0000 7f47b5e19640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f47b006db20 0x7f47b015d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37374/0 (socket says 172.21.15.76:37374) 2023-11-09T03:47:05.264 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.255+0000 7f47b5e19640 1 -- 172.21.15.76:0/2078505622 learned_addr learned my addr 172.21.15.76:0/2078505622 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:05.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.255+0000 7f47b4e17640 1 -- 172.21.15.76:0/2078505622 >> v1:172.21.15.113:6789/0 conn(0x7f47b0159c70 legacy=0x7f47b00b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:05.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.255+0000 7f47b4e17640 1 -- 172.21.15.76:0/2078505622 >> v1:172.21.15.76:6789/0 conn(0x7f47b006db20 legacy=0x7f47b015d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:05.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.255+0000 7f47b4e17640 1 -- 172.21.15.76:0/2078505622 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b00b46d0 con 0x7f47b00b0ae0 2023-11-09T03:47:05.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.256+0000 7f47b4e17640 1 --2- 172.21.15.76:0/2078505622 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47b00b0ae0 0x7f47b00ce370 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f47a00097a0 tx=0x7f47a002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca060b9d7dd8ff8 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:05.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.256+0000 7f47a7fff640 1 -- 172.21.15.76:0/2078505622 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a0031830 con 0x7f47b00b0ae0 2023-11-09T03:47:05.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.256+0000 7f47a7fff640 1 -- 172.21.15.76:0/2078505622 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a0031990 con 0x7f47b00b0ae0 2023-11-09T03:47:05.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.256+0000 7f47a7fff640 1 -- 172.21.15.76:0/2078505622 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a0031d20 con 0x7f47b00b0ae0 2023-11-09T03:47:05.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.257+0000 7f47b787c640 1 -- 172.21.15.76:0/2078505622 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47b00b0ae0 msgr2=0x7f47b00ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:05.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.257+0000 7f47b787c640 1 --2- 172.21.15.76:0/2078505622 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47b00b0ae0 0x7f47b00ce370 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f47a00097a0 tx=0x7f47a002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:47:05.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.257+0000 7f47b787c640 1 -- 172.21.15.76:0/2078505622 shutdown_connections 2023-11-09T03:47:05.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.257+0000 7f47b787c640 1 --2- 172.21.15.76:0/2078505622 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47b00b0ae0 0x7f47b00ce370 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:05.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.257+0000 7f47b787c640 1 -- 172.21.15.76:0/2078505622 >> 172.21.15.76:0/2078505622 conn(0x7f47b00ce950 msgr2=0x7f47b00ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:05.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.257+0000 7f47b787c640 1 -- 172.21.15.76:0/2078505622 shutdown_connections 2023-11-09T03:47:05.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.257+0000 7f47b787c640 1 -- 172.21.15.76:0/2078505622 wait complete. 2023-11-09T03:47:05.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.258+0000 7f47b787c640 1 Processor -- start 2023-11-09T03:47:05.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.258+0000 7f47b787c640 1 -- start start 2023-11-09T03:47:05.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.258+0000 7f47b787c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47b006db20 0x7f47b0155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:05.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.258+0000 7f47b787c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f47b00b0ae0 0x7f47b00c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:05.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.258+0000 7f47b787c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47b0159c70 0x7f47b0155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:05.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.259+0000 7f47b787c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b00c74b0 con 0x7f47b0159c70 2023-11-09T03:47:05.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.259+0000 7f47b787c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b00c7790 con 0x7f47b006db20 2023-11-09T03:47:05.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.259+0000 7f47b787c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b00c7a70 con 0x7f47b00b0ae0 2023-11-09T03:47:05.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.259+0000 7f47b5e19640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47b0159c70 0x7f47b0155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:05.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.259+0000 7f47b5e19640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47b0159c70 0x7f47b0155270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40380/0 (socket says 172.21.15.76:40380) 2023-11-09T03:47:05.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.259+0000 7f47b5e19640 1 -- 172.21.15.76:0/3616979446 learned_addr learned my addr 172.21.15.76:0/3616979446 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:05.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.259+0000 7f47b4e17640 1 --2- 172.21.15.76:0/3616979446 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f47b00b0ae0 0x7f47b00c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:05.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.259+0000 7f47b5e19640 1 -- 172.21.15.76:0/3616979446 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f47b00b0ae0 msgr2=0x7f47b00c8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:05.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.259+0000 7f47b5618640 1 --2- 172.21.15.76:0/3616979446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47b006db20 0x7f47b0155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:05.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.259+0000 7f47b5e19640 1 --2- 172.21.15.76:0/3616979446 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f47b00b0ae0 0x7f47b00c8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:05.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.259+0000 7f47b5e19640 1 -- 172.21.15.76:0/3616979446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47b006db20 msgr2=0x7f47b0155f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:05.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.259+0000 7f47b5e19640 1 --2- 172.21.15.76:0/3616979446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47b006db20 0x7f47b0155f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:05.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.259+0000 7f47b5e19640 1 -- 172.21.15.76:0/3616979446 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a0009400 con 0x7f47b0159c70 2023-11-09T03:47:05.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.259+0000 7f47b5e19640 1 --2- 172.21.15.76:0/3616979446 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47b0159c70 0x7f47b0155270 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f47a800cbc0 tx=0x7f47a800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:05.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.259+0000 7f47a5ffb640 1 -- 172.21.15.76:0/3616979446 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a800ad40 con 0x7f47b0159c70 2023-11-09T03:47:05.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.259+0000 7f47a5ffb640 1 -- 172.21.15.76:0/3616979446 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a8012400 con 0x7f47b0159c70 2023-11-09T03:47:05.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.259+0000 7f47b787c640 1 -- 172.21.15.76:0/3616979446 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b00c7d50 con 0x7f47b0159c70 2023-11-09T03:47:05.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.259+0000 7f47a5ffb640 1 -- 172.21.15.76:0/3616979446 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a800aea0 con 0x7f47b0159c70 2023-11-09T03:47:05.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.259+0000 7f47b787c640 1 -- 172.21.15.76:0/3616979446 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b01557b0 con 0x7f47b0159c70 2023-11-09T03:47:05.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.260+0000 7f47b787c640 1 -- 172.21.15.76:0/3616979446 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b0074290 con 0x7f47b0159c70 2023-11-09T03:47:05.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.261+0000 7f47a5ffb640 1 -- 172.21.15.76:0/3616979446 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f47a8007070 con 0x7f47b0159c70 2023-11-09T03:47:05.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.261+0000 7f47a5ffb640 1 --2- 172.21.15.76:0/3616979446 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f479406ae70 0x7f479406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:05.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.261+0000 7f47a5ffb640 1 -- 172.21.15.76:0/3616979446 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6438+0+0 (secure 0 0 0) 0x7f47a8089750 con 0x7f47b0159c70 2023-11-09T03:47:05.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.261+0000 7f47b5618640 1 --2- 172.21.15.76:0/3616979446 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f479406ae70 0x7f479406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:05.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.262+0000 7f47b5618640 1 --2- 172.21.15.76:0/3616979446 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f479406ae70 0x7f479406d330 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f47b00b2ba0 tx=0x7f47ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:05.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.266+0000 7f47a5ffb640 1 -- 172.21.15.76:0/3616979446 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f47a80545d0 con 0x7f47b0159c70 2023-11-09T03:47:05.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.463+0000 7f47b787c640 1 -- 172.21.15.76:0/3616979446 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47b006d3f0 con 0x7f47b0159c70 2023-11-09T03:47:05.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.464+0000 7f47a5ffb640 1 -- 172.21.15.76:0/3616979446 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f47a8057bf0 con 0x7f47b0159c70 2023-11-09T03:47:05.465 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:05.465 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4880,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/711686619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":711686619},{"type":"v1","addr":"172.21.15.76:6835","nonce":711686619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/521006441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":521006441},{"type":"v1","addr":"172.21.15.113:6837","nonce":521006441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5212,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2652975857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2652975857},{"type":"v1","addr":"172.21.15.113:6836","nonce":2652975857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:46:51.874121+0000","modified":"2023-11-09T03:46:53.868634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5601":{"gid":5601,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6834/981490030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":981490030},{"type":"v1","addr":"172.21.15.76:6834","nonce":981490030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-09T03:47:05.465 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 16 2023-11-09T03:47:05.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.466+0000 7f47b787c640 1 -- 172.21.15.76:0/3616979446 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f479406ae70 msgr2=0x7f479406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:05.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.466+0000 7f47b787c640 1 --2- 172.21.15.76:0/3616979446 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f479406ae70 0x7f479406d330 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f47b00b2ba0 tx=0x7f47ac008040 comp rx=0 tx=0).stop 2023-11-09T03:47:05.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.466+0000 7f47b787c640 1 -- 172.21.15.76:0/3616979446 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47b0159c70 msgr2=0x7f47b0155270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:05.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.466+0000 7f47b787c640 1 --2- 172.21.15.76:0/3616979446 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47b0159c70 0x7f47b0155270 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f47a800cbc0 tx=0x7f47a800a430 comp rx=0 tx=0).stop 2023-11-09T03:47:05.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.466+0000 7f47b787c640 1 -- 172.21.15.76:0/3616979446 shutdown_connections 2023-11-09T03:47:05.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.466+0000 7f47b787c640 1 --2- 172.21.15.76:0/3616979446 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f479406ae70 0x7f479406d330 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:05.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.466+0000 7f47b787c640 1 --2- 172.21.15.76:0/3616979446 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47b0159c70 0x7f47b0155270 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:05.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.466+0000 7f47b787c640 1 --2- 172.21.15.76:0/3616979446 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f47b00b0ae0 0x7f47b00c8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:05.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.466+0000 7f47b787c640 1 --2- 172.21.15.76:0/3616979446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47b006db20 0x7f47b0155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:05.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.466+0000 7f47b787c640 1 -- 172.21.15.76:0/3616979446 >> 172.21.15.76:0/3616979446 conn(0x7f47b00ce950 msgr2=0x7f47b015a260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:05.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.466+0000 7f47b787c640 1 -- 172.21.15.76:0/3616979446 shutdown_connections 2023-11-09T03:47:05.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.466+0000 7f47b787c640 1 -- 172.21.15.76:0/3616979446 wait complete. 2023-11-09T03:47:05.478 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:47:05.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.577+0000 7f2c645e2640 1 Processor -- start 2023-11-09T03:47:05.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.577+0000 7f2c645e2640 1 -- start start 2023-11-09T03:47:05.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.578+0000 7f2c645e2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2c5c070d40 0x7f2c5c06c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:05.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.578+0000 7f2c645e2640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c5c0cc990 con 0x7f2c5c0cda60 2023-11-09T03:47:05.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.578+0000 7f2c645e2640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c5c0ccc70 con 0x7f2c5c0b0cd0 2023-11-09T03:47:05.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.578+0000 7f2c645e2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c5c0c9ce0 con 0x7f2c5c070d40 2023-11-09T03:47:05.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.578+0000 7f2c6237e640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2c5c0cda60 0x7f2c5c0ca7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37382/0 (socket says 172.21.15.76:37382) 2023-11-09T03:47:05.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.578+0000 7f2c6237e640 1 -- 172.21.15.76:0/456205416 learned_addr learned my addr 172.21.15.76:0/456205416 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:05.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.578+0000 7f2c62b7f640 1 --2- 172.21.15.76:0/456205416 >> v2:172.21.15.113:3300/0 conn(0x7f2c5c070d40 0x7f2c5c06c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:05.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.578+0000 7f2c6137c640 1 -- 172.21.15.76:0/456205416 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3496284501 0 0) 0x7f2c5c0cc990 con 0x7f2c5c0cda60 2023-11-09T03:47:05.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.578+0000 7f2c6137c640 1 -- 172.21.15.76:0/456205416 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c48003520 con 0x7f2c5c0cda60 2023-11-09T03:47:05.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.579+0000 7f2c6137c640 1 -- 172.21.15.76:0/456205416 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3386125759 0 0) 0x7f2c48003520 con 0x7f2c5c0cda60 2023-11-09T03:47:05.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.579+0000 7f2c6137c640 1 -- 172.21.15.76:0/456205416 >> v2:172.21.15.113:3300/0 conn(0x7f2c5c070d40 msgr2=0x7f2c5c06c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:05.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.579+0000 7f2c6137c640 1 --2- 172.21.15.76:0/456205416 >> v2:172.21.15.113:3300/0 conn(0x7f2c5c070d40 0x7f2c5c06c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:05.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.579+0000 7f2c6137c640 1 -- 172.21.15.76:0/456205416 >> v1:172.21.15.113:6789/0 conn(0x7f2c5c0b0cd0 legacy=0x7f2c5c070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:05.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.579+0000 7f2c6137c640 1 -- 172.21.15.76:0/456205416 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5c0c9fc0 con 0x7f2c5c0cda60 2023-11-09T03:47:05.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.579+0000 7f2c6137c640 1 -- 172.21.15.76:0/456205416 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f2c58001670 con 0x7f2c5c0cda60 2023-11-09T03:47:05.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.579+0000 7f2c6137c640 1 -- 172.21.15.76:0/456205416 >> v1:172.21.15.76:6789/0 conn(0x7f2c5c0cda60 legacy=0x7f2c5c0ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:05.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.579+0000 7f2c6137c640 1 --2- 172.21.15.76:0/456205416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c48003970 0x7f2c48003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:05.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.579+0000 7f2c6137c640 1 --2- 172.21.15.76:0/456205416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c480047c0 0x7f2c48004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:05.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.579+0000 7f2c6137c640 1 --2- 172.21.15.76:0/456205416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c48005180 0x7f2c4800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:05.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.579+0000 7f2c6137c640 1 -- 172.21.15.76:0/456205416 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c0c9ce0 con 0x7f2c480047c0 2023-11-09T03:47:05.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.579+0000 7f2c6137c640 1 -- 172.21.15.76:0/456205416 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4800c020 con 0x7f2c48003970 2023-11-09T03:47:05.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.579+0000 7f2c6137c640 1 -- 172.21.15.76:0/456205416 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4800c1e0 con 0x7f2c48005180 2023-11-09T03:47:05.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.579+0000 7f2c61b7d640 1 --2- 172.21.15.76:0/456205416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c480047c0 0x7f2c48004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:05.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.579+0000 7f2c6237e640 1 --2- 172.21.15.76:0/456205416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c48003970 0x7f2c48003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:05.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.579+0000 7f2c62b7f640 1 --2- 172.21.15.76:0/456205416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c48005180 0x7f2c4800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:05.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.579+0000 7f2c61b7d640 1 -- 172.21.15.76:0/456205416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c48005180 msgr2=0x7f2c4800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:05.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.579+0000 7f2c61b7d640 1 --2- 172.21.15.76:0/456205416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c48005180 0x7f2c4800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:05.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.579+0000 7f2c61b7d640 1 -- 172.21.15.76:0/456205416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c48003970 msgr2=0x7f2c48003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:05.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.579+0000 7f2c61b7d640 1 --2- 172.21.15.76:0/456205416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c48003970 0x7f2c48003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:05.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.579+0000 7f2c61b7d640 1 -- 172.21.15.76:0/456205416 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c48003520 con 0x7f2c480047c0 2023-11-09T03:47:05.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.579+0000 7f2c62b7f640 1 -- 172.21.15.76:0/456205416 reap_dead start 2023-11-09T03:47:05.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.580+0000 7f2c61b7d640 1 --2- 172.21.15.76:0/456205416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c480047c0 0x7f2c48004c40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c001720 tx=0x7f2c480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba5400dad8f3040 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:05.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.580+0000 7f2c6137c640 1 -- 172.21.15.76:0/456205416 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c0052d0 con 0x7f2c480047c0 2023-11-09T03:47:05.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.580+0000 7f2c6137c640 1 -- 172.21.15.76:0/456205416 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4c02e050 con 0x7f2c480047c0 2023-11-09T03:47:05.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.580+0000 7f2c6137c640 1 -- 172.21.15.76:0/456205416 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c003830 con 0x7f2c480047c0 2023-11-09T03:47:05.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.580+0000 7f2c645e2640 1 -- 172.21.15.76:0/456205416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c480047c0 msgr2=0x7f2c48004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:05.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.580+0000 7f2c645e2640 1 --2- 172.21.15.76:0/456205416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c480047c0 0x7f2c48004c40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c001720 tx=0x7f2c480057f0 comp rx=0 tx=0).stop 2023-11-09T03:47:05.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.581+0000 7f2c645e2640 1 -- 172.21.15.76:0/456205416 shutdown_connections 2023-11-09T03:47:05.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.581+0000 7f2c645e2640 1 --2- 172.21.15.76:0/456205416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c480047c0 0x7f2c48004c40 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:05.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.581+0000 7f2c645e2640 1 -- 172.21.15.76:0/456205416 >> 172.21.15.76:0/456205416 conn(0x7f2c5c0ce950 msgr2=0x7f2c5c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:05.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.581+0000 7f2c645e2640 1 -- 172.21.15.76:0/456205416 shutdown_connections 2023-11-09T03:47:05.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.581+0000 7f2c645e2640 1 -- 172.21.15.76:0/456205416 wait complete. 2023-11-09T03:47:05.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.582+0000 7f2c645e2640 1 Processor -- start 2023-11-09T03:47:05.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.582+0000 7f2c645e2640 1 -- start start 2023-11-09T03:47:05.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.583+0000 7f2c645e2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c480047c0 0x7f2c5c14e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:05.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.583+0000 7f2c645e2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c5c0c7100 0x7f2c5c0c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:05.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.583+0000 7f2c645e2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c5c0b2fc0 0x7f2c5c0b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:05.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.583+0000 7f2c645e2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c0b19f0 con 0x7f2c480047c0 2023-11-09T03:47:05.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.583+0000 7f2c645e2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c0b1cd0 con 0x7f2c5c0c7100 2023-11-09T03:47:05.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.583+0000 7f2c645e2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c0b1fb0 con 0x7f2c5c0b2fc0 2023-11-09T03:47:05.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.583+0000 7f2c6237e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c480047c0 0x7f2c5c14e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:05.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.583+0000 7f2c6237e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c480047c0 0x7f2c5c14e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40396/0 (socket says 172.21.15.76:40396) 2023-11-09T03:47:05.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.583+0000 7f2c6237e640 1 -- 172.21.15.76:0/1691671536 learned_addr learned my addr 172.21.15.76:0/1691671536 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:05.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.583+0000 7f2c61b7d640 1 --2- 172.21.15.76:0/1691671536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c5c0c7100 0x7f2c5c0c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:05.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.583+0000 7f2c62b7f640 1 --2- 172.21.15.76:0/1691671536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c5c0b2fc0 0x7f2c5c0b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:05.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.583+0000 7f2c6237e640 1 -- 172.21.15.76:0/1691671536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c5c0b2fc0 msgr2=0x7f2c5c0b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:05.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.583+0000 7f2c6237e640 1 --2- 172.21.15.76:0/1691671536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c5c0b2fc0 0x7f2c5c0b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:05.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.583+0000 7f2c6237e640 1 -- 172.21.15.76:0/1691671536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c5c0c7100 msgr2=0x7f2c5c0c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:05.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.583+0000 7f2c6237e640 1 --2- 172.21.15.76:0/1691671536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c5c0c7100 0x7f2c5c0c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:05.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.583+0000 7f2c6237e640 1 -- 172.21.15.76:0/1691671536 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5c0b2290 con 0x7f2c480047c0 2023-11-09T03:47:05.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.583+0000 7f2c6237e640 1 --2- 172.21.15.76:0/1691671536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c480047c0 0x7f2c5c14e140 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2c58007290 tx=0x7f2c58007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:05.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.583+0000 7f2c52ffd640 1 -- 172.21.15.76:0/1691671536 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c58001650 con 0x7f2c480047c0 2023-11-09T03:47:05.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.583+0000 7f2c52ffd640 1 -- 172.21.15.76:0/1691671536 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5800f9c0 con 0x7f2c480047c0 2023-11-09T03:47:05.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.583+0000 7f2c645e2640 1 -- 172.21.15.76:0/1691671536 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c5c0c7c70 con 0x7f2c480047c0 2023-11-09T03:47:05.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.583+0000 7f2c52ffd640 1 -- 172.21.15.76:0/1691671536 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5800fd50 con 0x7f2c480047c0 2023-11-09T03:47:05.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.583+0000 7f2c645e2640 1 -- 172.21.15.76:0/1691671536 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c5c14e680 con 0x7f2c480047c0 2023-11-09T03:47:05.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.584+0000 7f2c645e2640 1 -- 172.21.15.76:0/1691671536 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c5c14e870 con 0x7f2c480047c0 2023-11-09T03:47:05.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.585+0000 7f2c52ffd640 1 -- 172.21.15.76:0/1691671536 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2c5801d460 con 0x7f2c480047c0 2023-11-09T03:47:05.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.585+0000 7f2c52ffd640 1 --2- 172.21.15.76:0/1691671536 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2c4006aea0 0x7f2c4006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:05.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.585+0000 7f2c52ffd640 1 -- 172.21.15.76:0/1691671536 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6438+0+0 (secure 0 0 0) 0x7f2c580899c0 con 0x7f2c480047c0 2023-11-09T03:47:05.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.585+0000 7f2c61b7d640 1 --2- 172.21.15.76:0/1691671536 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2c4006aea0 0x7f2c4006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:05.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.586+0000 7f2c61b7d640 1 --2- 172.21.15.76:0/1691671536 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2c4006aea0 0x7f2c4006d360 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c033000 tx=0x7f2c4c009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:05.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.591+0000 7f2c52ffd640 1 -- 172.21.15.76:0/1691671536 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2c58054790 con 0x7f2c480047c0 2023-11-09T03:47:05.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.758+0000 7f2c645e2640 1 -- 172.21.15.76:0/1691671536 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c5c0b1b30 con 0x7f2c480047c0 2023-11-09T03:47:05.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.759+0000 7f2c52ffd640 1 -- 172.21.15.76:0/1691671536 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18601 (secure 0 0 0) 0x7f2c5c0b1b30 con 0x7f2c480047c0 2023-11-09T03:47:05.759 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:05.760 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":42,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:47:02.574636+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:46:50.064765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-09T03:46:50.837125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":1,"length":1}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":1,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:47:05.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.761+0000 7f2c645e2640 1 -- 172.21.15.76:0/1691671536 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2c4006aea0 msgr2=0x7f2c4006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:05.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.761+0000 7f2c645e2640 1 --2- 172.21.15.76:0/1691671536 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2c4006aea0 0x7f2c4006d360 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c033000 tx=0x7f2c4c009030 comp rx=0 tx=0).stop 2023-11-09T03:47:05.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.761+0000 7f2c645e2640 1 -- 172.21.15.76:0/1691671536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c480047c0 msgr2=0x7f2c5c14e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:05.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.761+0000 7f2c645e2640 1 --2- 172.21.15.76:0/1691671536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c480047c0 0x7f2c5c14e140 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2c58007290 tx=0x7f2c58007750 comp rx=0 tx=0).stop 2023-11-09T03:47:05.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.761+0000 7f2c645e2640 1 -- 172.21.15.76:0/1691671536 shutdown_connections 2023-11-09T03:47:05.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.761+0000 7f2c645e2640 1 --2- 172.21.15.76:0/1691671536 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2c4006aea0 0x7f2c4006d360 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:05.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.761+0000 7f2c645e2640 1 --2- 172.21.15.76:0/1691671536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c5c0b2fc0 0x7f2c5c0b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:05.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.761+0000 7f2c645e2640 1 --2- 172.21.15.76:0/1691671536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c5c0c7100 0x7f2c5c0c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:05.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.761+0000 7f2c645e2640 1 --2- 172.21.15.76:0/1691671536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c480047c0 0x7f2c5c14e140 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:05.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.761+0000 7f2c645e2640 1 -- 172.21.15.76:0/1691671536 >> 172.21.15.76:0/1691671536 conn(0x7f2c5c0ce950 msgr2=0x7f2c5c152850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:05.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.761+0000 7f2c645e2640 1 -- 172.21.15.76:0/1691671536 shutdown_connections 2023-11-09T03:47:05.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.761+0000 7f2c645e2640 1 -- 172.21.15.76:0/1691671536 wait complete. 2023-11-09T03:47:05.772 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:46:50.064765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:46:50.837125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-09T03:47:00.036540+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-09T03:47:05.773 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:47:05.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.876+0000 7f2176576640 1 Processor -- start 2023-11-09T03:47:05.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.876+0000 7f2176576640 1 -- start start 2023-11-09T03:47:05.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.876+0000 7f2176576640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2170153e60 0x7f217015dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:05.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.876+0000 7f2176576640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21700cd6d0 0x7f2170161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:05.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.876+0000 7f2176576640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f21700b4880 0x7f21700b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:05.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.876+0000 7f2176576640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21700cde60 con 0x7f2170153e60 2023-11-09T03:47:05.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.876+0000 7f2176576640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2170160f90 con 0x7f21700b4880 2023-11-09T03:47:05.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.876+0000 7f2176576640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21700b51e0 con 0x7f21700cd6d0 2023-11-09T03:47:05.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.877+0000 7f216ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2170153e60 0x7f217015dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:05.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.877+0000 7f216ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2170153e60 0x7f217015dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40400/0 (socket says 172.21.15.76:40400) 2023-11-09T03:47:05.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.877+0000 7f216ffff640 1 -- 172.21.15.76:0/4029349498 learned_addr learned my addr 172.21.15.76:0/4029349498 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:05.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.877+0000 7f216ffff640 1 -- 172.21.15.76:0/4029349498 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21700cd6d0 msgr2=0x7f2170161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:05.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.877+0000 7f216ffff640 1 --2- 172.21.15.76:0/4029349498 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21700cd6d0 0x7f2170161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:05.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.877+0000 7f216ffff640 1 -- 172.21.15.76:0/4029349498 >> v2:172.21.15.113:3300/0 conn(0x7f21700b4880 msgr2=0x7f21700b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:05.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.877+0000 7f2174b13640 1 --2- 172.21.15.76:0/4029349498 >> v2:172.21.15.113:3300/0 conn(0x7f21700b4880 0x7f21700b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:05.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.877+0000 7f216ffff640 1 --2- 172.21.15.76:0/4029349498 >> v2:172.21.15.113:3300/0 conn(0x7f21700b4880 0x7f21700b4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:05.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.877+0000 7f216ffff640 1 -- 172.21.15.76:0/4029349498 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217015e400 con 0x7f2170153e60 2023-11-09T03:47:05.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.878+0000 7f216ffff640 1 --2- 172.21.15.76:0/4029349498 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2170153e60 0x7f217015dec0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f21700c9e50 tx=0x7f216002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b4d57ea587219c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:05.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.878+0000 7f216effd640 1 -- 172.21.15.76:0/4029349498 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2160031870 con 0x7f2170153e60 2023-11-09T03:47:05.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.878+0000 7f216effd640 1 -- 172.21.15.76:0/4029349498 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21600319d0 con 0x7f2170153e60 2023-11-09T03:47:05.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.878+0000 7f216effd640 1 -- 172.21.15.76:0/4029349498 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2160031d60 con 0x7f2170153e60 2023-11-09T03:47:05.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.878+0000 7f2176576640 1 -- 172.21.15.76:0/4029349498 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2170153e60 msgr2=0x7f217015dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:05.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.878+0000 7f2176576640 1 --2- 172.21.15.76:0/4029349498 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2170153e60 0x7f217015dec0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f21700c9e50 tx=0x7f216002b910 comp rx=0 tx=0).stop 2023-11-09T03:47:05.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.878+0000 7f2176576640 1 -- 172.21.15.76:0/4029349498 shutdown_connections 2023-11-09T03:47:05.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.878+0000 7f2176576640 1 --2- 172.21.15.76:0/4029349498 >> v2:172.21.15.113:3300/0 conn(0x7f21700b4880 0x7f21700b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:05.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.878+0000 7f2176576640 1 --2- 172.21.15.76:0/4029349498 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21700cd6d0 0x7f2170161420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:05.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.878+0000 7f2176576640 1 --2- 172.21.15.76:0/4029349498 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2170153e60 0x7f217015dec0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:05.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.878+0000 7f2176576640 1 -- 172.21.15.76:0/4029349498 >> 172.21.15.76:0/4029349498 conn(0x7f21700cf880 msgr2=0x7f21700cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:05.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.878+0000 7f2176576640 1 -- 172.21.15.76:0/4029349498 shutdown_connections 2023-11-09T03:47:05.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.878+0000 7f2176576640 1 -- 172.21.15.76:0/4029349498 wait complete. 2023-11-09T03:47:05.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.879+0000 7f2176576640 1 Processor -- start 2023-11-09T03:47:05.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.880+0000 7f2176576640 1 -- start start 2023-11-09T03:47:05.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.880+0000 7f2176576640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21700b4880 0x7f21700c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:05.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.880+0000 7f2176576640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21700cd6d0 0x7f21700c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:05.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.880+0000 7f2176576640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2170153e60 0x7f21700cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:05.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.880+0000 7f2176576640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2170154ff0 con 0x7f21700cd6d0 2023-11-09T03:47:05.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.880+0000 7f2176576640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21701f28a0 con 0x7f2170153e60 2023-11-09T03:47:05.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.880+0000 7f2176576640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2170155410 con 0x7f21700b4880 2023-11-09T03:47:05.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.880+0000 7f216f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21700cd6d0 0x7f21700c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:05.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.880+0000 7f216f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21700cd6d0 0x7f21700c8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40402/0 (socket says 172.21.15.76:40402) 2023-11-09T03:47:05.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.880+0000 7f216f7fe640 1 -- 172.21.15.76:0/3019081608 learned_addr learned my addr 172.21.15.76:0/3019081608 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:05.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.881+0000 7f216ffff640 1 --2- 172.21.15.76:0/3019081608 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21700b4880 0x7f21700c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:05.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.881+0000 7f2174b13640 1 --2- 172.21.15.76:0/3019081608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2170153e60 0x7f21700cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:05.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.881+0000 7f216f7fe640 1 -- 172.21.15.76:0/3019081608 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21700b4880 msgr2=0x7f21700c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:05.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.881+0000 7f216f7fe640 1 --2- 172.21.15.76:0/3019081608 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21700b4880 0x7f21700c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:05.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.881+0000 7f216f7fe640 1 -- 172.21.15.76:0/3019081608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2170153e60 msgr2=0x7f21700cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:05.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.881+0000 7f216f7fe640 1 --2- 172.21.15.76:0/3019081608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2170153e60 0x7f21700cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:05.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.881+0000 7f216f7fe640 1 -- 172.21.15.76:0/3019081608 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21600094d0 con 0x7f21700cd6d0 2023-11-09T03:47:05.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.881+0000 7f216f7fe640 1 --2- 172.21.15.76:0/3019081608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21700cd6d0 0x7f21700c8e40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f216400b4d0 tx=0x7f216400b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:05.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.881+0000 7f216cff9640 1 -- 172.21.15.76:0/3019081608 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2164010870 con 0x7f21700cd6d0 2023-11-09T03:47:05.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.881+0000 7f216cff9640 1 -- 172.21.15.76:0/3019081608 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21640109d0 con 0x7f21700cd6d0 2023-11-09T03:47:05.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.881+0000 7f2176576640 1 -- 172.21.15.76:0/3019081608 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2170154370 con 0x7f21700cd6d0 2023-11-09T03:47:05.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.881+0000 7f216cff9640 1 -- 172.21.15.76:0/3019081608 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2164010d80 con 0x7f21700cd6d0 2023-11-09T03:47:05.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.881+0000 7f2176576640 1 -- 172.21.15.76:0/3019081608 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2170154930 con 0x7f21700cd6d0 2023-11-09T03:47:05.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.882+0000 7f2176576640 1 -- 172.21.15.76:0/3019081608 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2170074290 con 0x7f21700cd6d0 2023-11-09T03:47:05.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.883+0000 7f216cff9640 1 -- 172.21.15.76:0/3019081608 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f216400f3c0 con 0x7f21700cd6d0 2023-11-09T03:47:05.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.883+0000 7f216cff9640 1 --2- 172.21.15.76:0/3019081608 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f215c06aea0 0x7f215c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:05.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.883+0000 7f216cff9640 1 -- 172.21.15.76:0/3019081608 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6438+0+0 (secure 0 0 0) 0x7f21640897d0 con 0x7f21700cd6d0 2023-11-09T03:47:05.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.883+0000 7f216ffff640 1 --2- 172.21.15.76:0/3019081608 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f215c06aea0 0x7f215c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:05.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.884+0000 7f216ffff640 1 --2- 172.21.15.76:0/3019081608 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f215c06aea0 0x7f215c06d360 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2160036000 tx=0x7f216003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:05.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:05.887+0000 7f216cff9640 1 -- 172.21.15.76:0/3019081608 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2164054630 con 0x7f21700cd6d0 2023-11-09T03:47:06.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.052+0000 7f2176576640 1 -- 172.21.15.76:0/3019081608 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f21700d0b80 con 0x7f21700cd6d0 2023-11-09T03:47:06.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.053+0000 7f216cff9640 1 -- 172.21.15.76:0/3019081608 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2164014020 con 0x7f21700cd6d0 2023-11-09T03:47:06.053 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:47:06.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.054+0000 7f2176576640 1 -- 172.21.15.76:0/3019081608 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f215c06aea0 msgr2=0x7f215c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:06.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.054+0000 7f2176576640 1 --2- 172.21.15.76:0/3019081608 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f215c06aea0 0x7f215c06d360 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2160036000 tx=0x7f216003c040 comp rx=0 tx=0).stop 2023-11-09T03:47:06.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.055+0000 7f2176576640 1 -- 172.21.15.76:0/3019081608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21700cd6d0 msgr2=0x7f21700c8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:06.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.055+0000 7f2176576640 1 --2- 172.21.15.76:0/3019081608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21700cd6d0 0x7f21700c8e40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f216400b4d0 tx=0x7f216400b990 comp rx=0 tx=0).stop 2023-11-09T03:47:06.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.055+0000 7f2176576640 1 -- 172.21.15.76:0/3019081608 shutdown_connections 2023-11-09T03:47:06.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.055+0000 7f2176576640 1 --2- 172.21.15.76:0/3019081608 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f215c06aea0 0x7f215c06d360 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.055+0000 7f2176576640 1 --2- 172.21.15.76:0/3019081608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2170153e60 0x7f21700cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.055+0000 7f2176576640 1 --2- 172.21.15.76:0/3019081608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21700cd6d0 0x7f21700c8e40 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.055+0000 7f2176576640 1 --2- 172.21.15.76:0/3019081608 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21700b4880 0x7f21700c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.055+0000 7f2176576640 1 -- 172.21.15.76:0/3019081608 >> 172.21.15.76:0/3019081608 conn(0x7f21700cf880 msgr2=0x7f21700b8120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:06.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.055+0000 7f2176576640 1 -- 172.21.15.76:0/3019081608 shutdown_connections 2023-11-09T03:47:06.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.055+0000 7f2176576640 1 -- 172.21.15.76:0/3019081608 wait complete. 2023-11-09T03:47:06.066 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:47:06.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.168+0000 7fd20d0c2640 1 Processor -- start 2023-11-09T03:47:06.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.168+0000 7fd20d0c2640 1 -- start start 2023-11-09T03:47:06.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.169+0000 7fd20d0c2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd20815ad30 0x7fd2080adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:06.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.169+0000 7fd20d0c2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd208070e20 0x7fd20815e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:06.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.169+0000 7fd20d0c2640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2080c3900 con 0x7fd2080ad730 2023-11-09T03:47:06.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.169+0000 7fd20d0c2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2080ae260 con 0x7fd20815ad30 2023-11-09T03:47:06.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.169+0000 7fd20d0c2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2080c3be0 con 0x7fd208070e20 2023-11-09T03:47:06.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.169+0000 7fd20659c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd2080ad730 0x7fd208070710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37388/0 (socket says 172.21.15.76:37388) 2023-11-09T03:47:06.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.169+0000 7fd20659c640 1 -- 172.21.15.76:0/4114985821 learned_addr learned my addr 172.21.15.76:0/4114985821 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:06.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.169+0000 7fd206d9d640 1 --2- 172.21.15.76:0/4114985821 >> v2:172.21.15.113:3300/0 conn(0x7fd20815ad30 0x7fd2080adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:06.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.169+0000 7fd20759e640 1 --2- 172.21.15.76:0/4114985821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd208070e20 0x7fd20815e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:06.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.169+0000 7fd205d9b640 1 -- 172.21.15.76:0/4114985821 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3541875375 0 0) 0x7fd2080c3900 con 0x7fd2080ad730 2023-11-09T03:47:06.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.169+0000 7fd205d9b640 1 -- 172.21.15.76:0/4114985821 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1f0003520 con 0x7fd2080ad730 2023-11-09T03:47:06.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.170+0000 7fd206d9d640 1 -- 172.21.15.76:0/4114985821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd208070e20 msgr2=0x7fd20815e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:06.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.170+0000 7fd206d9d640 1 --2- 172.21.15.76:0/4114985821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd208070e20 0x7fd20815e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.170+0000 7fd206d9d640 1 -- 172.21.15.76:0/4114985821 >> v1:172.21.15.76:6789/0 conn(0x7fd2080ad730 legacy=0x7fd208070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:06.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.170+0000 7fd205d9b640 1 -- 172.21.15.76:0/4114985821 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2163075001 0 0) 0x7fd1f0003520 con 0x7fd2080ad730 2023-11-09T03:47:06.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.170+0000 7fd206d9d640 1 -- 172.21.15.76:0/4114985821 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2080c3ec0 con 0x7fd20815ad30 2023-11-09T03:47:06.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.170+0000 7fd206d9d640 1 --2- 172.21.15.76:0/4114985821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd20815ad30 0x7fd2080adc80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8009d90 tx=0x7fd1f802bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e559904f20e27fd server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:06.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.171+0000 7fd205d9b640 1 -- 172.21.15.76:0/4114985821 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc001670 con 0x7fd20815ad30 2023-11-09T03:47:06.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.171+0000 7fd205d9b640 1 -- 172.21.15.76:0/4114985821 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f802c810 con 0x7fd20815ad30 2023-11-09T03:47:06.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.171+0000 7fd205d9b640 1 -- 172.21.15.76:0/4114985821 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f802cb80 con 0x7fd20815ad30 2023-11-09T03:47:06.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.171+0000 7fd20d0c2640 1 -- 172.21.15.76:0/4114985821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd20815ad30 msgr2=0x7fd2080adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:06.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.171+0000 7fd20d0c2640 1 --2- 172.21.15.76:0/4114985821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd20815ad30 0x7fd2080adc80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8009d90 tx=0x7fd1f802bce0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.171+0000 7fd20d0c2640 1 -- 172.21.15.76:0/4114985821 shutdown_connections 2023-11-09T03:47:06.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.171+0000 7fd20d0c2640 1 --2- 172.21.15.76:0/4114985821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd208070e20 0x7fd20815e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.171+0000 7fd20d0c2640 1 --2- 172.21.15.76:0/4114985821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd20815ad30 0x7fd2080adc80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.171+0000 7fd20d0c2640 1 -- 172.21.15.76:0/4114985821 >> 172.21.15.76:0/4114985821 conn(0x7fd2080ce950 msgr2=0x7fd2080ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:06.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.171+0000 7fd20d0c2640 1 -- 172.21.15.76:0/4114985821 shutdown_connections 2023-11-09T03:47:06.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.171+0000 7fd20d0c2640 1 -- 172.21.15.76:0/4114985821 wait complete. 2023-11-09T03:47:06.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.173+0000 7fd20d0c2640 1 Processor -- start 2023-11-09T03:47:06.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.173+0000 7fd20d0c2640 1 -- start start 2023-11-09T03:47:06.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.173+0000 7fd20d0c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd208070e20 0x7fd20814e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:06.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.173+0000 7fd20d0c2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2080ad730 0x7fd208143200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:06.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.173+0000 7fd20d0c2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd20815ad30 0x7fd2081f0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:06.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.173+0000 7fd20d0c2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20814f940 con 0x7fd208070e20 2023-11-09T03:47:06.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.173+0000 7fd20d0c2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20814ea90 con 0x7fd2080ad730 2023-11-09T03:47:06.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.173+0000 7fd20d0c2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd20814ed30 con 0x7fd20815ad30 2023-11-09T03:47:06.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.174+0000 7fd206d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd208070e20 0x7fd20814e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:06.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.174+0000 7fd206d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd208070e20 0x7fd20814e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40416/0 (socket says 172.21.15.76:40416) 2023-11-09T03:47:06.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.174+0000 7fd206d9d640 1 -- 172.21.15.76:0/3946822320 learned_addr learned my addr 172.21.15.76:0/3946822320 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:06.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.174+0000 7fd20759e640 1 --2- 172.21.15.76:0/3946822320 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd20815ad30 0x7fd2081f0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:06.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.174+0000 7fd20659c640 1 --2- 172.21.15.76:0/3946822320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2080ad730 0x7fd208143200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:06.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.174+0000 7fd206d9d640 1 -- 172.21.15.76:0/3946822320 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd20815ad30 msgr2=0x7fd2081f0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:06.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.174+0000 7fd206d9d640 1 --2- 172.21.15.76:0/3946822320 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd20815ad30 0x7fd2081f0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.174+0000 7fd206d9d640 1 -- 172.21.15.76:0/3946822320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2080ad730 msgr2=0x7fd208143200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:06.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.174+0000 7fd206d9d640 1 --2- 172.21.15.76:0/3946822320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2080ad730 0x7fd208143200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.174+0000 7fd206d9d640 1 -- 172.21.15.76:0/3946822320 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f8009a70 con 0x7fd208070e20 2023-11-09T03:47:06.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.174+0000 7fd20759e640 1 --2- 172.21.15.76:0/3946822320 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd20815ad30 0x7fd2081f0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:06.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.174+0000 7fd20659c640 1 --2- 172.21.15.76:0/3946822320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2080ad730 0x7fd208143200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:06.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.174+0000 7fd206d9d640 1 --2- 172.21.15.76:0/3946822320 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd208070e20 0x7fd20814e550 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8009d90 tx=0x7fd1f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:06.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.174+0000 7fd1ef7fe640 1 -- 172.21.15.76:0/3946822320 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8009490 con 0x7fd208070e20 2023-11-09T03:47:06.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.174+0000 7fd1ef7fe640 1 -- 172.21.15.76:0/3946822320 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f8035c90 con 0x7fd208070e20 2023-11-09T03:47:06.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.174+0000 7fd20d0c2640 1 -- 172.21.15.76:0/3946822320 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd20814f010 con 0x7fd208070e20 2023-11-09T03:47:06.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.174+0000 7fd1ef7fe640 1 -- 172.21.15.76:0/3946822320 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f803e410 con 0x7fd208070e20 2023-11-09T03:47:06.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.174+0000 7fd20d0c2640 1 -- 172.21.15.76:0/3946822320 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd20814f5d0 con 0x7fd208070e20 2023-11-09T03:47:06.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.175+0000 7fd20d0c2640 1 -- 172.21.15.76:0/3946822320 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd20814f760 con 0x7fd208070e20 2023-11-09T03:47:06.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.176+0000 7fd1ef7fe640 1 -- 172.21.15.76:0/3946822320 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd1f803b050 con 0x7fd208070e20 2023-11-09T03:47:06.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.176+0000 7fd1ef7fe640 1 --2- 172.21.15.76:0/3946822320 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd1f406aea0 0x7fd1f406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:06.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.176+0000 7fd1ef7fe640 1 -- 172.21.15.76:0/3946822320 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6438+0+0 (secure 0 0 0) 0x7fd1f80ab470 con 0x7fd208070e20 2023-11-09T03:47:06.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.176+0000 7fd20659c640 1 --2- 172.21.15.76:0/3946822320 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd1f406aea0 0x7fd1f406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:06.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.177+0000 7fd20659c640 1 --2- 172.21.15.76:0/3946822320 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd1f406aea0 0x7fd1f406d360 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd208150f70 tx=0x7fd1fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:06.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.180+0000 7fd1ef7fe640 1 -- 172.21.15.76:0/3946822320 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd1f80762d0 con 0x7fd208070e20 2023-11-09T03:47:06.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.345+0000 7fd20d0c2640 1 -- 172.21.15.76:0/3946822320 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd208074290 con 0x7fd208070e20 2023-11-09T03:47:06.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.345+0000 7fd1ef7fe640 1 -- 172.21.15.76:0/3946822320 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd1f80798f0 con 0x7fd208070e20 2023-11-09T03:47:06.346 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:47:06.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.347+0000 7fd20d0c2640 1 -- 172.21.15.76:0/3946822320 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd1f406aea0 msgr2=0x7fd1f406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:06.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.347+0000 7fd20d0c2640 1 --2- 172.21.15.76:0/3946822320 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd1f406aea0 0x7fd1f406d360 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd208150f70 tx=0x7fd1fc009040 comp rx=0 tx=0).stop 2023-11-09T03:47:06.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.347+0000 7fd20d0c2640 1 -- 172.21.15.76:0/3946822320 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd208070e20 msgr2=0x7fd20814e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:06.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.347+0000 7fd20d0c2640 1 --2- 172.21.15.76:0/3946822320 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd208070e20 0x7fd20814e550 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8009d90 tx=0x7fd1f8004a40 comp rx=0 tx=0).stop 2023-11-09T03:47:06.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.347+0000 7fd20d0c2640 1 -- 172.21.15.76:0/3946822320 shutdown_connections 2023-11-09T03:47:06.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.347+0000 7fd20d0c2640 1 --2- 172.21.15.76:0/3946822320 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd1f406aea0 0x7fd1f406d360 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.347+0000 7fd20d0c2640 1 --2- 172.21.15.76:0/3946822320 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd20815ad30 0x7fd2081f0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.347+0000 7fd20d0c2640 1 --2- 172.21.15.76:0/3946822320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2080ad730 0x7fd208143200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.347+0000 7fd20d0c2640 1 --2- 172.21.15.76:0/3946822320 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd208070e20 0x7fd20814e550 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.347+0000 7fd20d0c2640 1 -- 172.21.15.76:0/3946822320 >> 172.21.15.76:0/3946822320 conn(0x7fd2080ce950 msgr2=0x7fd2080ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:06.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.347+0000 7fd20d0c2640 1 -- 172.21.15.76:0/3946822320 shutdown_connections 2023-11-09T03:47:06.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.347+0000 7fd20d0c2640 1 -- 172.21.15.76:0/3946822320 wait complete. 2023-11-09T03:47:06.358 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:47:06.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.460+0000 7fb240cd4640 1 Processor -- start 2023-11-09T03:47:06.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.460+0000 7fb240cd4640 1 -- start start 2023-11-09T03:47:06.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.461+0000 7fb240cd4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb23c0c4ff0 0x7fb23c0cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:06.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.461+0000 7fb240cd4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb23c0adac0 0x7fb23c0adf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:06.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.461+0000 7fb240cd4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb23c15aff0 con 0x7fb23c159c00 2023-11-09T03:47:06.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.461+0000 7fb240cd4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb23c0ae460 con 0x7fb23c0adac0 2023-11-09T03:47:06.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.461+0000 7fb240cd4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb23c15a3e0 con 0x7fb23c0c4ff0 2023-11-09T03:47:06.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.461+0000 7fb23ad9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb23c0adac0 0x7fb23c0adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:06.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.461+0000 7fb23a59c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb23c159c00 0x7fb23c0c48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37396/0 (socket says 172.21.15.76:37396) 2023-11-09T03:47:06.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.461+0000 7fb23ad9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb23c0adac0 0x7fb23c0adf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40420/0 (socket says 172.21.15.76:40420) 2023-11-09T03:47:06.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.461+0000 7fb23a59c640 1 -- 172.21.15.76:0/2116052948 learned_addr learned my addr 172.21.15.76:0/2116052948 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:06.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.461+0000 7fb23ad9d640 1 -- 172.21.15.76:0/2116052948 >> v2:172.21.15.113:3300/0 conn(0x7fb23c0c4ff0 msgr2=0x7fb23c0cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:06.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.461+0000 7fb23ad9d640 1 --2- 172.21.15.76:0/2116052948 >> v2:172.21.15.113:3300/0 conn(0x7fb23c0c4ff0 0x7fb23c0cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.461+0000 7fb23ad9d640 1 -- 172.21.15.76:0/2116052948 >> v1:172.21.15.76:6789/0 conn(0x7fb23c159c00 legacy=0x7fb23c0c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:06.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.461+0000 7fb23ad9d640 1 -- 172.21.15.76:0/2116052948 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23c15a6c0 con 0x7fb23c0adac0 2023-11-09T03:47:06.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.461+0000 7fb239d9b640 1 -- 172.21.15.76:0/2116052948 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3945106405 0 0) 0x7fb23c15aff0 con 0x7fb23c159c00 2023-11-09T03:47:06.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.462+0000 7fb23ad9d640 1 --2- 172.21.15.76:0/2116052948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb23c0adac0 0x7fb23c0adf20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0cde50 tx=0x7fb23402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3539b90dedb045 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:06.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.462+0000 7fb239d9b640 1 -- 172.21.15.76:0/2116052948 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23402cc90 con 0x7fb23c0adac0 2023-11-09T03:47:06.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.462+0000 7fb239d9b640 1 -- 172.21.15.76:0/2116052948 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb23402cdf0 con 0x7fb23c0adac0 2023-11-09T03:47:06.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.462+0000 7fb239d9b640 1 -- 172.21.15.76:0/2116052948 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb234034660 con 0x7fb23c0adac0 2023-11-09T03:47:06.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.463+0000 7fb240cd4640 1 -- 172.21.15.76:0/2116052948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb23c0adac0 msgr2=0x7fb23c0adf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:06.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.463+0000 7fb240cd4640 1 --2- 172.21.15.76:0/2116052948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb23c0adac0 0x7fb23c0adf20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0cde50 tx=0x7fb23402bee0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.463+0000 7fb240cd4640 1 -- 172.21.15.76:0/2116052948 shutdown_connections 2023-11-09T03:47:06.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.463+0000 7fb240cd4640 1 --2- 172.21.15.76:0/2116052948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb23c0adac0 0x7fb23c0adf20 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.463+0000 7fb240cd4640 1 --2- 172.21.15.76:0/2116052948 >> v2:172.21.15.113:3300/0 conn(0x7fb23c0c4ff0 0x7fb23c0cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.463+0000 7fb240cd4640 1 -- 172.21.15.76:0/2116052948 >> 172.21.15.76:0/2116052948 conn(0x7fb23c0ce950 msgr2=0x7fb23c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:06.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.463+0000 7fb240cd4640 1 -- 172.21.15.76:0/2116052948 shutdown_connections 2023-11-09T03:47:06.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.463+0000 7fb240cd4640 1 -- 172.21.15.76:0/2116052948 wait complete. 2023-11-09T03:47:06.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.464+0000 7fb240cd4640 1 Processor -- start 2023-11-09T03:47:06.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.464+0000 7fb240cd4640 1 -- start start 2023-11-09T03:47:06.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.465+0000 7fb240cd4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb23c0adac0 0x7fb23c158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:06.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.465+0000 7fb240cd4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb23c0c4ff0 0x7fb23c1f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:06.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.465+0000 7fb240cd4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb23c159c00 0x7fb23c145370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:06.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.465+0000 7fb240cd4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c155d10 con 0x7fb23c0adac0 2023-11-09T03:47:06.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.465+0000 7fb240cd4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c158c60 con 0x7fb23c0c4ff0 2023-11-09T03:47:06.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.465+0000 7fb240cd4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c155f60 con 0x7fb23c159c00 2023-11-09T03:47:06.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.465+0000 7fb23a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb23c0adac0 0x7fb23c158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:06.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.465+0000 7fb23a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb23c0adac0 0x7fb23c158640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40424/0 (socket says 172.21.15.76:40424) 2023-11-09T03:47:06.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.465+0000 7fb23a59c640 1 -- 172.21.15.76:0/2193993761 learned_addr learned my addr 172.21.15.76:0/2193993761 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:06.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.465+0000 7fb23ad9d640 1 --2- 172.21.15.76:0/2193993761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb23c159c00 0x7fb23c145370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:06.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.465+0000 7fb233fff640 1 --2- 172.21.15.76:0/2193993761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb23c0c4ff0 0x7fb23c1f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:06.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.465+0000 7fb23a59c640 1 -- 172.21.15.76:0/2193993761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb23c159c00 msgr2=0x7fb23c145370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:06.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.465+0000 7fb23a59c640 1 --2- 172.21.15.76:0/2193993761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb23c159c00 0x7fb23c145370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.465+0000 7fb23a59c640 1 -- 172.21.15.76:0/2193993761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb23c0c4ff0 msgr2=0x7fb23c1f49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:06.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.465+0000 7fb23a59c640 1 --2- 172.21.15.76:0/2193993761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb23c0c4ff0 0x7fb23c1f49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.465+0000 7fb23a59c640 1 -- 172.21.15.76:0/2193993761 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb234009a70 con 0x7fb23c0adac0 2023-11-09T03:47:06.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.465+0000 7fb23a59c640 1 --2- 172.21.15.76:0/2193993761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb23c0adac0 0x7fb23c158640 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00c430 tx=0x7fb22c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:06.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.465+0000 7fb232ffd640 1 -- 172.21.15.76:0/2193993761 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c0074e0 con 0x7fb23c0adac0 2023-11-09T03:47:06.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.465+0000 7fb232ffd640 1 -- 172.21.15.76:0/2193993761 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22c007660 con 0x7fb23c0adac0 2023-11-09T03:47:06.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.465+0000 7fb232ffd640 1 -- 172.21.15.76:0/2193993761 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c004a20 con 0x7fb23c0adac0 2023-11-09T03:47:06.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.465+0000 7fb240cd4640 1 -- 172.21.15.76:0/2193993761 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb23c156240 con 0x7fb23c0adac0 2023-11-09T03:47:06.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.465+0000 7fb240cd4640 1 -- 172.21.15.76:0/2193993761 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb23c1458b0 con 0x7fb23c0adac0 2023-11-09T03:47:06.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.466+0000 7fb240cd4640 1 -- 172.21.15.76:0/2193993761 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb23c156760 con 0x7fb23c0adac0 2023-11-09T03:47:06.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.467+0000 7fb232ffd640 1 -- 172.21.15.76:0/2193993761 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb22c004b80 con 0x7fb23c0adac0 2023-11-09T03:47:06.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.467+0000 7fb232ffd640 1 --2- 172.21.15.76:0/2193993761 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb21c06ae70 0x7fb21c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:06.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.467+0000 7fb232ffd640 1 -- 172.21.15.76:0/2193993761 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6438+0+0 (secure 0 0 0) 0x7fb22c089700 con 0x7fb23c0adac0 2023-11-09T03:47:06.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.467+0000 7fb233fff640 1 --2- 172.21.15.76:0/2193993761 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb21c06ae70 0x7fb21c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:06.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.468+0000 7fb233fff640 1 --2- 172.21.15.76:0/2193993761 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb21c06ae70 0x7fb21c06d330 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0ad5f0 tx=0x7fb224008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:06.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.471+0000 7fb232ffd640 1 -- 172.21.15.76:0/2193993761 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb23c156760 con 0x7fb23c0adac0 2023-11-09T03:47:06.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.640+0000 7fb240cd4640 1 -- 172.21.15.76:0/2193993761 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb23c074290 con 0x7fb23c0adac0 2023-11-09T03:47:06.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.640+0000 7fb232ffd640 1 -- 172.21.15.76:0/2193993761 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb22c0544d0 con 0x7fb23c0adac0 2023-11-09T03:47:06.640 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:47:06.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.642+0000 7fb240cd4640 1 -- 172.21.15.76:0/2193993761 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb21c06ae70 msgr2=0x7fb21c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:06.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.642+0000 7fb240cd4640 1 --2- 172.21.15.76:0/2193993761 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb21c06ae70 0x7fb21c06d330 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0ad5f0 tx=0x7fb224008040 comp rx=0 tx=0).stop 2023-11-09T03:47:06.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.642+0000 7fb240cd4640 1 -- 172.21.15.76:0/2193993761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb23c0adac0 msgr2=0x7fb23c158640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:06.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.642+0000 7fb240cd4640 1 --2- 172.21.15.76:0/2193993761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb23c0adac0 0x7fb23c158640 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00c430 tx=0x7fb22c00c8f0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.642+0000 7fb240cd4640 1 -- 172.21.15.76:0/2193993761 shutdown_connections 2023-11-09T03:47:06.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.642+0000 7fb240cd4640 1 --2- 172.21.15.76:0/2193993761 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb21c06ae70 0x7fb21c06d330 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.642+0000 7fb240cd4640 1 --2- 172.21.15.76:0/2193993761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb23c159c00 0x7fb23c145370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.642+0000 7fb240cd4640 1 --2- 172.21.15.76:0/2193993761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb23c0c4ff0 0x7fb23c1f49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.642+0000 7fb240cd4640 1 --2- 172.21.15.76:0/2193993761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb23c0adac0 0x7fb23c158640 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.642+0000 7fb240cd4640 1 -- 172.21.15.76:0/2193993761 >> 172.21.15.76:0/2193993761 conn(0x7fb23c0ce950 msgr2=0x7fb23c0709f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:06.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.642+0000 7fb240cd4640 1 -- 172.21.15.76:0/2193993761 shutdown_connections 2023-11-09T03:47:06.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.642+0000 7fb240cd4640 1 -- 172.21.15.76:0/2193993761 wait complete. 2023-11-09T03:47:06.653 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-09T03:47:06.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.756+0000 7fee7dbd0640 1 Processor -- start 2023-11-09T03:47:06.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.756+0000 7fee7dbd0640 1 -- start start 2023-11-09T03:47:06.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.757+0000 7fee7dbd0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fee780ad730 0x7fee780adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:06.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.757+0000 7fee7dbd0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee780c3900 con 0x7fee78070e20 2023-11-09T03:47:06.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.757+0000 7fee7dbd0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee780c3be0 con 0x7fee7815ad30 2023-11-09T03:47:06.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.757+0000 7fee7dbd0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee780ae260 con 0x7fee780ad730 2023-11-09T03:47:06.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.757+0000 7fee77fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fee780ad730 0x7fee780adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:06.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.757+0000 7fee76ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fee78070e20 0x7fee7815e740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37412/0 (socket says 172.21.15.76:37412) 2023-11-09T03:47:06.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.757+0000 7fee76ffd640 1 -- 172.21.15.76:0/3505710388 learned_addr learned my addr 172.21.15.76:0/3505710388 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:06.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.757+0000 7fee77fff640 1 -- 172.21.15.76:0/3505710388 >> v1:172.21.15.113:6789/0 conn(0x7fee7815ad30 legacy=0x7fee78070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:06.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.757+0000 7fee77fff640 1 -- 172.21.15.76:0/3505710388 >> v1:172.21.15.76:6789/0 conn(0x7fee78070e20 legacy=0x7fee7815e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:06.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.757+0000 7fee77fff640 1 -- 172.21.15.76:0/3505710388 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee780c3ec0 con 0x7fee780ad730 2023-11-09T03:47:06.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.757+0000 7fee767fc640 1 -- 172.21.15.76:0/3505710388 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1695462437 0 0) 0x7fee780c3900 con 0x7fee78070e20 2023-11-09T03:47:06.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.758+0000 7fee77fff640 1 --2- 172.21.15.76:0/3505710388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee780ad730 0x7fee780adc80 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fee70009d90 tx=0x7fee7002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b787f94021dc95f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:06.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.758+0000 7fee767fc640 1 -- 172.21.15.76:0/3505710388 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7002cc60 con 0x7fee780ad730 2023-11-09T03:47:06.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.758+0000 7fee767fc640 1 -- 172.21.15.76:0/3505710388 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7002cdc0 con 0x7fee780ad730 2023-11-09T03:47:06.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.758+0000 7fee767fc640 1 -- 172.21.15.76:0/3505710388 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee70034660 con 0x7fee780ad730 2023-11-09T03:47:06.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.758+0000 7fee7dbd0640 1 -- 172.21.15.76:0/3505710388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee780ad730 msgr2=0x7fee780adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:06.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.758+0000 7fee7dbd0640 1 --2- 172.21.15.76:0/3505710388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee780ad730 0x7fee780adc80 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fee70009d90 tx=0x7fee7002beb0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.759+0000 7fee7dbd0640 1 -- 172.21.15.76:0/3505710388 shutdown_connections 2023-11-09T03:47:06.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.759+0000 7fee7dbd0640 1 --2- 172.21.15.76:0/3505710388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee780ad730 0x7fee780adc80 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.759+0000 7fee7dbd0640 1 -- 172.21.15.76:0/3505710388 >> 172.21.15.76:0/3505710388 conn(0x7fee780ce950 msgr2=0x7fee780ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:06.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.759+0000 7fee7dbd0640 1 -- 172.21.15.76:0/3505710388 shutdown_connections 2023-11-09T03:47:06.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.759+0000 7fee7dbd0640 1 -- 172.21.15.76:0/3505710388 wait complete. 2023-11-09T03:47:06.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.760+0000 7fee7dbd0640 1 Processor -- start 2023-11-09T03:47:06.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.760+0000 7fee7dbd0640 1 -- start start 2023-11-09T03:47:06.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.761+0000 7fee7dbd0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee78070e20 0x7fee780c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:06.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.761+0000 7fee7dbd0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee780ad730 0x7fee7815da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:06.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.761+0000 7fee7dbd0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee7815ad30 0x7fee78149720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:06.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.761+0000 7fee7dbd0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7815ca90 con 0x7fee780ad730 2023-11-09T03:47:06.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.761+0000 7fee7dbd0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee78149f30 con 0x7fee7815ad30 2023-11-09T03:47:06.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.761+0000 7fee7dbd0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7815dfc0 con 0x7fee78070e20 2023-11-09T03:47:06.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.761+0000 7fee76ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee780ad730 0x7fee7815da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:06.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.761+0000 7fee76ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee780ad730 0x7fee7815da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40442/0 (socket says 172.21.15.76:40442) 2023-11-09T03:47:06.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.761+0000 7fee76ffd640 1 -- 172.21.15.76:0/1960942731 learned_addr learned my addr 172.21.15.76:0/1960942731 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:06.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.761+0000 7fee777fe640 1 --2- 172.21.15.76:0/1960942731 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee78070e20 0x7fee780c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:06.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.761+0000 7fee77fff640 1 --2- 172.21.15.76:0/1960942731 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee7815ad30 0x7fee78149720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:06.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.761+0000 7fee76ffd640 1 -- 172.21.15.76:0/1960942731 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee78070e20 msgr2=0x7fee780c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:06.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.761+0000 7fee76ffd640 1 --2- 172.21.15.76:0/1960942731 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee78070e20 0x7fee780c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.761+0000 7fee76ffd640 1 -- 172.21.15.76:0/1960942731 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee7815ad30 msgr2=0x7fee78149720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:06.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.761+0000 7fee76ffd640 1 --2- 172.21.15.76:0/1960942731 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee7815ad30 0x7fee78149720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.761+0000 7fee76ffd640 1 -- 172.21.15.76:0/1960942731 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee70009a70 con 0x7fee780ad730 2023-11-09T03:47:06.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.761+0000 7fee76ffd640 1 --2- 172.21.15.76:0/1960942731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee780ad730 0x7fee7815da80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00c4a0 tx=0x7fee6c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:06.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.761+0000 7fee5ffff640 1 -- 172.21.15.76:0/1960942731 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c0074e0 con 0x7fee780ad730 2023-11-09T03:47:06.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.761+0000 7fee5ffff640 1 -- 172.21.15.76:0/1960942731 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6c007640 con 0x7fee780ad730 2023-11-09T03:47:06.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.761+0000 7fee7dbd0640 1 -- 172.21.15.76:0/1960942731 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee78149c60 con 0x7fee780ad730 2023-11-09T03:47:06.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.761+0000 7fee5ffff640 1 -- 172.21.15.76:0/1960942731 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c004a00 con 0x7fee780ad730 2023-11-09T03:47:06.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.761+0000 7fee7dbd0640 1 -- 172.21.15.76:0/1960942731 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee7815cfa0 con 0x7fee780ad730 2023-11-09T03:47:06.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.762+0000 7fee7dbd0640 1 -- 172.21.15.76:0/1960942731 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee78074290 con 0x7fee780ad730 2023-11-09T03:47:06.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.763+0000 7fee5ffff640 1 -- 172.21.15.76:0/1960942731 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fee6c004b60 con 0x7fee780ad730 2023-11-09T03:47:06.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.763+0000 7fee5ffff640 1 --2- 172.21.15.76:0/1960942731 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fee6406ae70 0x7fee6406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:06.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.763+0000 7fee5ffff640 1 -- 172.21.15.76:0/1960942731 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6438+0+0 (secure 0 0 0) 0x7fee6c088b40 con 0x7fee780ad730 2023-11-09T03:47:06.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.763+0000 7fee777fe640 1 --2- 172.21.15.76:0/1960942731 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fee6406ae70 0x7fee6406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:06.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.764+0000 7fee777fe640 1 --2- 172.21.15.76:0/1960942731 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fee6406ae70 0x7fee6406d330 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fee78153fc0 tx=0x7fee68009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:06.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.768+0000 7fee5ffff640 1 -- 172.21.15.76:0/1960942731 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fee6c053910 con 0x7fee780ad730 2023-11-09T03:47:06.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.932+0000 7fee7dbd0640 1 -- 172.21.15.76:0/1960942731 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7fee78071970 con 0x7fee780ad730 2023-11-09T03:47:06.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.932+0000 7fee5ffff640 1 -- 172.21.15.76:0/1960942731 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v42) v1 ==== 103+0+11 (secure 0 0 0) 0x7fee6c018070 con 0x7fee780ad730 2023-11-09T03:47:06.933 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:47:06.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.934+0000 7fee7dbd0640 1 -- 172.21.15.76:0/1960942731 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fee6406ae70 msgr2=0x7fee6406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:06.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.934+0000 7fee7dbd0640 1 --2- 172.21.15.76:0/1960942731 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fee6406ae70 0x7fee6406d330 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fee78153fc0 tx=0x7fee68009040 comp rx=0 tx=0).stop 2023-11-09T03:47:06.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.934+0000 7fee7dbd0640 1 -- 172.21.15.76:0/1960942731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee780ad730 msgr2=0x7fee7815da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:06.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.934+0000 7fee7dbd0640 1 --2- 172.21.15.76:0/1960942731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee780ad730 0x7fee7815da80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00c4a0 tx=0x7fee6c00c960 comp rx=0 tx=0).stop 2023-11-09T03:47:06.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.934+0000 7fee7dbd0640 1 -- 172.21.15.76:0/1960942731 shutdown_connections 2023-11-09T03:47:06.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.934+0000 7fee7dbd0640 1 --2- 172.21.15.76:0/1960942731 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fee6406ae70 0x7fee6406d330 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.934+0000 7fee7dbd0640 1 --2- 172.21.15.76:0/1960942731 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee7815ad30 0x7fee78149720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.934+0000 7fee7dbd0640 1 --2- 172.21.15.76:0/1960942731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee780ad730 0x7fee7815da80 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.934+0000 7fee7dbd0640 1 --2- 172.21.15.76:0/1960942731 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee78070e20 0x7fee780c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:06.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.934+0000 7fee7dbd0640 1 -- 172.21.15.76:0/1960942731 >> 172.21.15.76:0/1960942731 conn(0x7fee780ce950 msgr2=0x7fee78151ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:06.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.934+0000 7fee7dbd0640 1 -- 172.21.15.76:0/1960942731 shutdown_connections 2023-11-09T03:47:06.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:06.934+0000 7fee7dbd0640 1 -- 172.21.15.76:0/1960942731 wait complete. 2023-11-09T03:47:06.945 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:47:07.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.045+0000 7f8d036c2640 1 Processor -- start 2023-11-09T03:47:07.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.045+0000 7f8d036c2640 1 -- start start 2023-11-09T03:47:07.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.046+0000 7f8d036c2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8cfc0ce370 0x7f8cfc06db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:07.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.046+0000 7f8d036c2640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cfc06e060 con 0x7f8cfc159c70 2023-11-09T03:47:07.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.046+0000 7f8d036c2640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cfc0b4110 con 0x7f8cfc0b0ae0 2023-11-09T03:47:07.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.046+0000 7f8d036c2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc0b49b0 con 0x7f8cfc0ce370 2023-11-09T03:47:07.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.046+0000 7f8d0145e640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8cfc159c70 0x7f8cfc0b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37418/0 (socket says 172.21.15.76:37418) 2023-11-09T03:47:07.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.046+0000 7f8d0145e640 1 -- 172.21.15.76:0/2675295699 learned_addr learned my addr 172.21.15.76:0/2675295699 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:07.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.046+0000 7f8cf3fff640 1 -- 172.21.15.76:0/2675295699 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3308798392 0 0) 0x7f8cfc06e060 con 0x7f8cfc159c70 2023-11-09T03:47:07.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.046+0000 7f8cf3fff640 1 -- 172.21.15.76:0/2675295699 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ce8003520 con 0x7f8cfc159c70 2023-11-09T03:47:07.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.046+0000 7f8d01c5f640 1 --2- 172.21.15.76:0/2675295699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8cfc0ce370 0x7f8cfc06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:07.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.046+0000 7f8cf3fff640 1 -- 172.21.15.76:0/2675295699 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3468653325 0 0) 0x7f8ce8003520 con 0x7f8cfc159c70 2023-11-09T03:47:07.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.046+0000 7f8cf3fff640 1 -- 172.21.15.76:0/2675295699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8cfc0ce370 msgr2=0x7f8cfc06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:07.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.046+0000 7f8cf3fff640 1 --2- 172.21.15.76:0/2675295699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8cfc0ce370 0x7f8cfc06db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.046+0000 7f8cf3fff640 1 -- 172.21.15.76:0/2675295699 >> v1:172.21.15.113:6789/0 conn(0x7f8cfc0b0ae0 legacy=0x7f8cfc15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:07.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.046+0000 7f8cf3fff640 1 -- 172.21.15.76:0/2675295699 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cfc0b43f0 con 0x7f8cfc159c70 2023-11-09T03:47:07.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.046+0000 7f8cf3fff640 1 -- 172.21.15.76:0/2675295699 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f8cf8001670 con 0x7f8cfc159c70 2023-11-09T03:47:07.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.046+0000 7f8cf3fff640 1 -- 172.21.15.76:0/2675295699 >> v1:172.21.15.76:6789/0 conn(0x7f8cfc159c70 legacy=0x7f8cfc0b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:07.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.046+0000 7f8cf3fff640 1 --2- 172.21.15.76:0/2675295699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8cfc0ce370 0x7f8ce80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:07.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.046+0000 7f8cf3fff640 1 --2- 172.21.15.76:0/2675295699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ce8003f20 0x7f8ce80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:07.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.046+0000 7f8cf3fff640 1 --2- 172.21.15.76:0/2675295699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ce8004d00 0x7f8ce8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:07.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.046+0000 7f8cf3fff640 1 -- 172.21.15.76:0/2675295699 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc0b49b0 con 0x7f8ce8003f20 2023-11-09T03:47:07.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.046+0000 7f8cf3fff640 1 -- 172.21.15.76:0/2675295699 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce800ba60 con 0x7f8ce8004d00 2023-11-09T03:47:07.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.046+0000 7f8cf3fff640 1 -- 172.21.15.76:0/2675295699 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce800bc20 con 0x7f8cfc0ce370 2023-11-09T03:47:07.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.047+0000 7f8d00c5d640 1 --2- 172.21.15.76:0/2675295699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ce8003f20 0x7f8ce80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:07.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.047+0000 7f8d0145e640 1 --2- 172.21.15.76:0/2675295699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8cfc0ce370 0x7f8ce80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:07.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.047+0000 7f8d01c5f640 1 --2- 172.21.15.76:0/2675295699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ce8004d00 0x7f8ce8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:07.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.047+0000 7f8d00c5d640 1 -- 172.21.15.76:0/2675295699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8cfc0ce370 msgr2=0x7f8ce80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:07.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.047+0000 7f8d00c5d640 1 --2- 172.21.15.76:0/2675295699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8cfc0ce370 0x7f8ce80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.047+0000 7f8d00c5d640 1 -- 172.21.15.76:0/2675295699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ce8004d00 msgr2=0x7f8ce8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:07.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.047+0000 7f8d00c5d640 1 --2- 172.21.15.76:0/2675295699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ce8004d00 0x7f8ce8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.047+0000 7f8d00c5d640 1 -- 172.21.15.76:0/2675295699 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce8003520 con 0x7f8ce8003f20 2023-11-09T03:47:07.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.047+0000 7f8d01c5f640 1 --2- 172.21.15.76:0/2675295699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ce8004d00 0x7f8ce8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:07.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.048+0000 7f8d00c5d640 1 --2- 172.21.15.76:0/2675295699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ce8003f20 0x7f8ce80047c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8cec001750 tx=0x7f8ce80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9152583f29a0e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:07.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.048+0000 7f8cf3fff640 1 -- 172.21.15.76:0/2675295699 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec0052d0 con 0x7f8ce8003f20 2023-11-09T03:47:07.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.048+0000 7f8cf3fff640 1 -- 172.21.15.76:0/2675295699 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cec02e050 con 0x7f8ce8003f20 2023-11-09T03:47:07.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.048+0000 7f8cf3fff640 1 -- 172.21.15.76:0/2675295699 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec003830 con 0x7f8ce8003f20 2023-11-09T03:47:07.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.048+0000 7f8d036c2640 1 -- 172.21.15.76:0/2675295699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ce8003f20 msgr2=0x7f8ce80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:07.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.048+0000 7f8d036c2640 1 --2- 172.21.15.76:0/2675295699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ce8003f20 0x7f8ce80047c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8cec001750 tx=0x7f8ce80057f0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.048+0000 7f8d01c5f640 1 -- 172.21.15.76:0/2675295699 reap_dead start 2023-11-09T03:47:07.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.048+0000 7f8d036c2640 1 -- 172.21.15.76:0/2675295699 shutdown_connections 2023-11-09T03:47:07.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.048+0000 7f8d036c2640 1 -- 172.21.15.76:0/2675295699 >> 172.21.15.76:0/2675295699 conn(0x7f8cfc0ce950 msgr2=0x7f8cfc0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:07.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.048+0000 7f8d036c2640 1 -- 172.21.15.76:0/2675295699 shutdown_connections 2023-11-09T03:47:07.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.048+0000 7f8d036c2640 1 -- 172.21.15.76:0/2675295699 wait complete. 2023-11-09T03:47:07.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.049+0000 7f8d036c2640 1 Processor -- start 2023-11-09T03:47:07.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.049+0000 7f8d036c2640 1 -- start start 2023-11-09T03:47:07.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.050+0000 7f8d036c2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ce8003f20 0x7f8cfc157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:07.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.050+0000 7f8d036c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cfc14ef60 0x7f8cfc14e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:07.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.050+0000 7f8d036c2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8cfc1e6630 0x7f8cfc1e6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:07.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.050+0000 7f8d036c2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc150bd0 con 0x7f8cfc14ef60 2023-11-09T03:47:07.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.050+0000 7f8d036c2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc1e6ff0 con 0x7f8ce8003f20 2023-11-09T03:47:07.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.050+0000 7f8d036c2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc150de0 con 0x7f8cfc1e6630 2023-11-09T03:47:07.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.050+0000 7f8d00c5d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cfc14ef60 0x7f8cfc14e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:07.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.050+0000 7f8d00c5d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cfc14ef60 0x7f8cfc14e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40458/0 (socket says 172.21.15.76:40458) 2023-11-09T03:47:07.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.050+0000 7f8d00c5d640 1 -- 172.21.15.76:0/469410934 learned_addr learned my addr 172.21.15.76:0/469410934 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:07.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.050+0000 7f8d00c5d640 1 -- 172.21.15.76:0/469410934 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8cfc1e6630 msgr2=0x7f8cfc1e6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:07.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.050+0000 7f8d00c5d640 1 --2- 172.21.15.76:0/469410934 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8cfc1e6630 0x7f8cfc1e6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.050+0000 7f8d00c5d640 1 -- 172.21.15.76:0/469410934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ce8003f20 msgr2=0x7f8cfc157f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:07.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.050+0000 7f8d00c5d640 1 --2- 172.21.15.76:0/469410934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ce8003f20 0x7f8cfc157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.051+0000 7f8d00c5d640 1 -- 172.21.15.76:0/469410934 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cfc1510c0 con 0x7f8cfc14ef60 2023-11-09T03:47:07.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.051+0000 7f8d00c5d640 1 --2- 172.21.15.76:0/469410934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cfc14ef60 0x7f8cfc14e750 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8cec008fd0 tx=0x7f8cec001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:07.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.051+0000 7f8cf1ffb640 1 -- 172.21.15.76:0/469410934 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec007a30 con 0x7f8cfc14ef60 2023-11-09T03:47:07.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.051+0000 7f8cf1ffb640 1 -- 172.21.15.76:0/469410934 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cec007640 con 0x7f8cfc14ef60 2023-11-09T03:47:07.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.051+0000 7f8d036c2640 1 -- 172.21.15.76:0/469410934 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cfc1513a0 con 0x7f8cfc14ef60 2023-11-09T03:47:07.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.051+0000 7f8cf1ffb640 1 -- 172.21.15.76:0/469410934 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec006c00 con 0x7f8cfc14ef60 2023-11-09T03:47:07.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.051+0000 7f8d036c2640 1 -- 172.21.15.76:0/469410934 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cfc1584c0 con 0x7f8cfc14ef60 2023-11-09T03:47:07.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.052+0000 7f8d036c2640 1 -- 172.21.15.76:0/469410934 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cfc074290 con 0x7f8cfc14ef60 2023-11-09T03:47:07.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.052+0000 7f8cf1ffb640 1 -- 172.21.15.76:0/469410934 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8cec0035f0 con 0x7f8cfc14ef60 2023-11-09T03:47:07.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.053+0000 7f8cf1ffb640 1 --2- 172.21.15.76:0/469410934 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8ce006ae70 0x7f8ce006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:07.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.053+0000 7f8cf1ffb640 1 -- 172.21.15.76:0/469410934 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6438+0+0 (secure 0 0 0) 0x7f8cec0ab590 con 0x7f8cfc14ef60 2023-11-09T03:47:07.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.053+0000 7f8d0145e640 1 --2- 172.21.15.76:0/469410934 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8ce006ae70 0x7f8ce006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:07.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.054+0000 7f8d0145e640 1 --2- 172.21.15.76:0/469410934 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8ce006ae70 0x7f8ce006d330 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8001050 tx=0x7f8cf8001b20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:07.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.057+0000 7f8cf1ffb640 1 -- 172.21.15.76:0/469410934 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8cec076360 con 0x7f8cfc14ef60 2023-11-09T03:47:07.256 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.256+0000 7f8d036c2640 1 -- 172.21.15.76:0/469410934 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cfc06d3f0 con 0x7f8cfc14ef60 2023-11-09T03:47:07.257 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.257+0000 7f8cf1ffb640 1 -- 172.21.15.76:0/469410934 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f8cec079980 con 0x7f8cfc14ef60 2023-11-09T03:47:07.257 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:07.258 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4880,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/711686619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":711686619},{"type":"v1","addr":"172.21.15.76:6835","nonce":711686619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/521006441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":521006441},{"type":"v1","addr":"172.21.15.113:6837","nonce":521006441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5212,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2652975857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2652975857},{"type":"v1","addr":"172.21.15.113:6836","nonce":2652975857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:46:51.874121+0000","modified":"2023-11-09T03:46:53.868634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5601":{"gid":5601,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6834/981490030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":981490030},{"type":"v1","addr":"172.21.15.76:6834","nonce":981490030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-09T03:47:07.258 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 16 2023-11-09T03:47:07.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.259+0000 7f8d036c2640 1 -- 172.21.15.76:0/469410934 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8ce006ae70 msgr2=0x7f8ce006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:07.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.259+0000 7f8d036c2640 1 --2- 172.21.15.76:0/469410934 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8ce006ae70 0x7f8ce006d330 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8001050 tx=0x7f8cf8001b20 comp rx=0 tx=0).stop 2023-11-09T03:47:07.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.259+0000 7f8d036c2640 1 -- 172.21.15.76:0/469410934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cfc14ef60 msgr2=0x7f8cfc14e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:07.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.259+0000 7f8d036c2640 1 --2- 172.21.15.76:0/469410934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cfc14ef60 0x7f8cfc14e750 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8cec008fd0 tx=0x7f8cec001120 comp rx=0 tx=0).stop 2023-11-09T03:47:07.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.259+0000 7f8d036c2640 1 -- 172.21.15.76:0/469410934 shutdown_connections 2023-11-09T03:47:07.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.259+0000 7f8d036c2640 1 --2- 172.21.15.76:0/469410934 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8ce006ae70 0x7f8ce006d330 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.259+0000 7f8d036c2640 1 --2- 172.21.15.76:0/469410934 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8cfc1e6630 0x7f8cfc1e6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.259+0000 7f8d036c2640 1 --2- 172.21.15.76:0/469410934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cfc14ef60 0x7f8cfc14e750 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.259+0000 7f8d036c2640 1 --2- 172.21.15.76:0/469410934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ce8003f20 0x7f8cfc157f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.259+0000 7f8d036c2640 1 -- 172.21.15.76:0/469410934 >> 172.21.15.76:0/469410934 conn(0x7f8cfc0ce950 msgr2=0x7f8cfc070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:07.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.259+0000 7f8d036c2640 1 -- 172.21.15.76:0/469410934 shutdown_connections 2023-11-09T03:47:07.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.259+0000 7f8d036c2640 1 -- 172.21.15.76:0/469410934 wait complete. 2023-11-09T03:47:07.308 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:47:07.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.407+0000 7fdf5fbee640 1 Processor -- start 2023-11-09T03:47:07.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.408+0000 7fdf5fbee640 1 -- start start 2023-11-09T03:47:07.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.408+0000 7fdf5fbee640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf580cde60 0x7fdf5815d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:07.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.408+0000 7fdf5fbee640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdf580ce240 0x7fdf580d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:07.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.408+0000 7fdf5fbee640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf5815e000 con 0x7fdf5815daa0 2023-11-09T03:47:07.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.408+0000 7fdf5fbee640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf5815e5c0 con 0x7fdf580ce240 2023-11-09T03:47:07.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.408+0000 7fdf5fbee640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5815e2e0 con 0x7fdf580cde60 2023-11-09T03:47:07.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.409+0000 7fdf5e18b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdf580ce240 0x7fdf580d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:07.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.409+0000 7fdf5e18b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdf580ce240 0x7fdf580d1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40474/0 (socket says 172.21.15.76:40474) 2023-11-09T03:47:07.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.409+0000 7fdf5d189640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fdf5815daa0 0x7fdf58157000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37424/0 (socket says 172.21.15.76:37424) 2023-11-09T03:47:07.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.409+0000 7fdf5e18b640 1 -- 172.21.15.76:0/4040782607 learned_addr learned my addr 172.21.15.76:0/4040782607 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:07.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.409+0000 7fdf5d98a640 1 --2- 172.21.15.76:0/4040782607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf580cde60 0x7fdf5815d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:07.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.409+0000 7fdf5e18b640 1 -- 172.21.15.76:0/4040782607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf580cde60 msgr2=0x7fdf5815d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:07.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.409+0000 7fdf5e18b640 1 --2- 172.21.15.76:0/4040782607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf580cde60 0x7fdf5815d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.409+0000 7fdf5e18b640 1 -- 172.21.15.76:0/4040782607 >> v1:172.21.15.76:6789/0 conn(0x7fdf5815daa0 legacy=0x7fdf58157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:07.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.409+0000 7fdf5e18b640 1 -- 172.21.15.76:0/4040782607 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf580d1120 con 0x7fdf580ce240 2023-11-09T03:47:07.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.410+0000 7fdf5e18b640 1 --2- 172.21.15.76:0/4040782607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf580ce240 0x7fdf580d1920 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009870 tx=0x7fdf5402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785179d478e73bf server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:07.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.410+0000 7fdf5c988640 1 -- 172.21.15.76:0/4040782607 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf54030840 con 0x7fdf580ce240 2023-11-09T03:47:07.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.410+0000 7fdf5c988640 1 -- 172.21.15.76:0/4040782607 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf540309a0 con 0x7fdf580ce240 2023-11-09T03:47:07.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.410+0000 7fdf5c988640 1 -- 172.21.15.76:0/4040782607 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf54030d30 con 0x7fdf580ce240 2023-11-09T03:47:07.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.410+0000 7fdf5fbee640 1 -- 172.21.15.76:0/4040782607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf580ce240 msgr2=0x7fdf580d1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:07.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.410+0000 7fdf5fbee640 1 --2- 172.21.15.76:0/4040782607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf580ce240 0x7fdf580d1920 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009870 tx=0x7fdf5402b980 comp rx=0 tx=0).stop 2023-11-09T03:47:07.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.410+0000 7fdf5fbee640 1 -- 172.21.15.76:0/4040782607 shutdown_connections 2023-11-09T03:47:07.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.410+0000 7fdf5fbee640 1 --2- 172.21.15.76:0/4040782607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf580ce240 0x7fdf580d1920 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.410+0000 7fdf5fbee640 1 --2- 172.21.15.76:0/4040782607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf580cde60 0x7fdf5815d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.410+0000 7fdf5fbee640 1 -- 172.21.15.76:0/4040782607 >> 172.21.15.76:0/4040782607 conn(0x7fdf58071f30 msgr2=0x7fdf580cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:07.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.410+0000 7fdf5fbee640 1 -- 172.21.15.76:0/4040782607 shutdown_connections 2023-11-09T03:47:07.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.410+0000 7fdf5fbee640 1 -- 172.21.15.76:0/4040782607 wait complete. 2023-11-09T03:47:07.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.412+0000 7fdf5fbee640 1 Processor -- start 2023-11-09T03:47:07.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.412+0000 7fdf5fbee640 1 -- start start 2023-11-09T03:47:07.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.412+0000 7fdf5fbee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf580cde60 0x7fdf5814fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:07.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.412+0000 7fdf5fbee640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf580ce240 0x7fdf581e8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:07.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.412+0000 7fdf5fbee640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf5815daa0 0x7fdf58154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:07.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.412+0000 7fdf5fbee640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5814e660 con 0x7fdf580ce240 2023-11-09T03:47:07.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.412+0000 7fdf5fbee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf580cb6d0 con 0x7fdf580cde60 2023-11-09T03:47:07.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.412+0000 7fdf5fbee640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5814f000 con 0x7fdf5815daa0 2023-11-09T03:47:07.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.412+0000 7fdf5d189640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf580ce240 0x7fdf581e8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:07.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.412+0000 7fdf5d189640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf580ce240 0x7fdf581e8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40478/0 (socket says 172.21.15.76:40478) 2023-11-09T03:47:07.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.412+0000 7fdf5d189640 1 -- 172.21.15.76:0/1123186595 learned_addr learned my addr 172.21.15.76:0/1123186595 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:07.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.413+0000 7fdf5d98a640 1 --2- 172.21.15.76:0/1123186595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf580cde60 0x7fdf5814fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:07.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.413+0000 7fdf5e18b640 1 --2- 172.21.15.76:0/1123186595 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf5815daa0 0x7fdf58154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:07.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.413+0000 7fdf5d189640 1 -- 172.21.15.76:0/1123186595 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf5815daa0 msgr2=0x7fdf58154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:07.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.413+0000 7fdf5d189640 1 --2- 172.21.15.76:0/1123186595 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf5815daa0 0x7fdf58154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.413+0000 7fdf5d189640 1 -- 172.21.15.76:0/1123186595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf580cde60 msgr2=0x7fdf5814fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:07.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.413+0000 7fdf5d189640 1 --2- 172.21.15.76:0/1123186595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf580cde60 0x7fdf5814fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.413+0000 7fdf5d189640 1 -- 172.21.15.76:0/1123186595 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf540094d0 con 0x7fdf580ce240 2023-11-09T03:47:07.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.413+0000 7fdf5d189640 1 --2- 172.21.15.76:0/1123186595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf580ce240 0x7fdf581e8d40 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800c3e0 tx=0x7fdf4800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:07.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.413+0000 7fdf4e7fc640 1 -- 172.21.15.76:0/1123186595 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf480074e0 con 0x7fdf580ce240 2023-11-09T03:47:07.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.413+0000 7fdf4e7fc640 1 -- 172.21.15.76:0/1123186595 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf48007640 con 0x7fdf580ce240 2023-11-09T03:47:07.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.413+0000 7fdf5fbee640 1 -- 172.21.15.76:0/1123186595 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf581e9280 con 0x7fdf580ce240 2023-11-09T03:47:07.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.413+0000 7fdf4e7fc640 1 -- 172.21.15.76:0/1123186595 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf480049e0 con 0x7fdf580ce240 2023-11-09T03:47:07.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.413+0000 7fdf5fbee640 1 -- 172.21.15.76:0/1123186595 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5814f250 con 0x7fdf580ce240 2023-11-09T03:47:07.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.414+0000 7fdf5fbee640 1 -- 172.21.15.76:0/1123186595 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf58074290 con 0x7fdf580ce240 2023-11-09T03:47:07.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.415+0000 7fdf4e7fc640 1 -- 172.21.15.76:0/1123186595 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdf48004b40 con 0x7fdf580ce240 2023-11-09T03:47:07.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.415+0000 7fdf4e7fc640 1 --2- 172.21.15.76:0/1123186595 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdf3c06aea0 0x7fdf3c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:07.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.415+0000 7fdf4e7fc640 1 -- 172.21.15.76:0/1123186595 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6438+0+0 (secure 0 0 0) 0x7fdf480892b0 con 0x7fdf580ce240 2023-11-09T03:47:07.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.415+0000 7fdf5d98a640 1 --2- 172.21.15.76:0/1123186595 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdf3c06aea0 0x7fdf3c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:07.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.416+0000 7fdf5d98a640 1 --2- 172.21.15.76:0/1123186595 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdf3c06aea0 0x7fdf3c06d360 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdf58150680 tx=0x7fdf500039b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:07.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.419+0000 7fdf4e7fc640 1 -- 172.21.15.76:0/1123186595 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdf48054080 con 0x7fdf580ce240 2023-11-09T03:47:07.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.589+0000 7fdf5fbee640 1 -- 172.21.15.76:0/1123186595 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf580c66f0 con 0x7fdf580ce240 2023-11-09T03:47:07.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.590+0000 7fdf4e7fc640 1 -- 172.21.15.76:0/1123186595 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18601 (secure 0 0 0) 0x7fdf480576a0 con 0x7fdf580ce240 2023-11-09T03:47:07.591 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:07.591 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":42,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:47:02.574636+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:46:50.064765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-09T03:46:50.837125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":1,"length":1}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":1,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:47:07.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.592+0000 7fdf5fbee640 1 -- 172.21.15.76:0/1123186595 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdf3c06aea0 msgr2=0x7fdf3c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:07.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.592+0000 7fdf5fbee640 1 --2- 172.21.15.76:0/1123186595 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdf3c06aea0 0x7fdf3c06d360 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdf58150680 tx=0x7fdf500039b0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.592+0000 7fdf5fbee640 1 -- 172.21.15.76:0/1123186595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf580ce240 msgr2=0x7fdf581e8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:07.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.592+0000 7fdf5fbee640 1 --2- 172.21.15.76:0/1123186595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf580ce240 0x7fdf581e8d40 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800c3e0 tx=0x7fdf4800c8a0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.592+0000 7fdf5fbee640 1 -- 172.21.15.76:0/1123186595 shutdown_connections 2023-11-09T03:47:07.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.592+0000 7fdf5fbee640 1 --2- 172.21.15.76:0/1123186595 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdf3c06aea0 0x7fdf3c06d360 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.592+0000 7fdf5fbee640 1 --2- 172.21.15.76:0/1123186595 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf5815daa0 0x7fdf58154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.592+0000 7fdf5fbee640 1 --2- 172.21.15.76:0/1123186595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf580ce240 0x7fdf581e8d40 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.592+0000 7fdf5fbee640 1 --2- 172.21.15.76:0/1123186595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf580cde60 0x7fdf5814fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.592+0000 7fdf5fbee640 1 -- 172.21.15.76:0/1123186595 >> 172.21.15.76:0/1123186595 conn(0x7fdf58071f30 msgr2=0x7fdf580b7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:07.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.592+0000 7fdf5fbee640 1 -- 172.21.15.76:0/1123186595 shutdown_connections 2023-11-09T03:47:07.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.592+0000 7fdf5fbee640 1 -- 172.21.15.76:0/1123186595 wait complete. 2023-11-09T03:47:07.604 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-09T03:47:07.605 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:47:07.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.704+0000 7efc559ee640 1 Processor -- start 2023-11-09T03:47:07.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.704+0000 7efc559ee640 1 -- start start 2023-11-09T03:47:07.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.705+0000 7efc559ee640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7efc500ce240 0x7efc50157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:07.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.705+0000 7efc559ee640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc50157540 0x7efc500d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:07.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.705+0000 7efc559ee640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc5015e000 con 0x7efc500cde60 2023-11-09T03:47:07.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.705+0000 7efc559ee640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc5015e5c0 con 0x7efc500ce240 2023-11-09T03:47:07.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.705+0000 7efc559ee640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc5015e2e0 con 0x7efc50157540 2023-11-09T03:47:07.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.705+0000 7efc4e7fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7efc500ce240 0x7efc50157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:07.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.705+0000 7efc4effd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7efc500cde60 0x7efc5015d560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37434/0 (socket says 172.21.15.76:37434) 2023-11-09T03:47:07.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.705+0000 7efc4effd640 1 -- 172.21.15.76:0/2439082891 learned_addr learned my addr 172.21.15.76:0/2439082891 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:07.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.705+0000 7efc4f7fe640 1 --2- 172.21.15.76:0/2439082891 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc50157540 0x7efc500d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:07.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.705+0000 7efc4e7fc640 1 -- 172.21.15.76:0/2439082891 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc50157540 msgr2=0x7efc500d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:07.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.705+0000 7efc4e7fc640 1 --2- 172.21.15.76:0/2439082891 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc50157540 0x7efc500d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.705+0000 7efc4e7fc640 1 -- 172.21.15.76:0/2439082891 >> v1:172.21.15.76:6789/0 conn(0x7efc500cde60 legacy=0x7efc5015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:07.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.705+0000 7efc4e7fc640 1 -- 172.21.15.76:0/2439082891 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc500d1120 con 0x7efc500ce240 2023-11-09T03:47:07.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.705+0000 7efc4dffb640 1 -- 172.21.15.76:0/2439082891 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3915033979 0 0) 0x7efc5015e000 con 0x7efc500cde60 2023-11-09T03:47:07.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.705+0000 7efc4f7fe640 1 --2- 172.21.15.76:0/2439082891 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc50157540 0x7efc500d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:07.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.706+0000 7efc4e7fc640 1 --2- 172.21.15.76:0/2439082891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc500ce240 0x7efc50157000 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7efc440097a0 tx=0x7efc4402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1358fbce2093890b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:07.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.706+0000 7efc4dffb640 1 -- 172.21.15.76:0/2439082891 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4402d040 con 0x7efc500ce240 2023-11-09T03:47:07.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.706+0000 7efc4dffb640 1 -- 172.21.15.76:0/2439082891 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc44031810 con 0x7efc500ce240 2023-11-09T03:47:07.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.706+0000 7efc4dffb640 1 -- 172.21.15.76:0/2439082891 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc44031ba0 con 0x7efc500ce240 2023-11-09T03:47:07.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.707+0000 7efc559ee640 1 -- 172.21.15.76:0/2439082891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc500ce240 msgr2=0x7efc50157000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:07.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.707+0000 7efc559ee640 1 --2- 172.21.15.76:0/2439082891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc500ce240 0x7efc50157000 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7efc440097a0 tx=0x7efc4402b8b0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.707+0000 7efc559ee640 1 -- 172.21.15.76:0/2439082891 shutdown_connections 2023-11-09T03:47:07.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.707+0000 7efc559ee640 1 --2- 172.21.15.76:0/2439082891 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc50157540 0x7efc500d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.707+0000 7efc559ee640 1 --2- 172.21.15.76:0/2439082891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc500ce240 0x7efc50157000 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.707+0000 7efc559ee640 1 -- 172.21.15.76:0/2439082891 >> 172.21.15.76:0/2439082891 conn(0x7efc50071f30 msgr2=0x7efc500cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:07.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.707+0000 7efc559ee640 1 -- 172.21.15.76:0/2439082891 shutdown_connections 2023-11-09T03:47:07.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.707+0000 7efc559ee640 1 -- 172.21.15.76:0/2439082891 wait complete. 2023-11-09T03:47:07.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.708+0000 7efc559ee640 1 Processor -- start 2023-11-09T03:47:07.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.708+0000 7efc559ee640 1 -- start start 2023-11-09T03:47:07.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.709+0000 7efc559ee640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc500cde60 0x7efc50142880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:07.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.709+0000 7efc559ee640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc500ce240 0x7efc50154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:07.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.709+0000 7efc559ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc50157540 0x7efc50140e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:07.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.709+0000 7efc559ee640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc501423f0 con 0x7efc500ce240 2023-11-09T03:47:07.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.709+0000 7efc559ee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc501417e0 con 0x7efc50157540 2023-11-09T03:47:07.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.709+0000 7efc559ee640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc50141ac0 con 0x7efc500cde60 2023-11-09T03:47:07.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.709+0000 7efc4e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc500ce240 0x7efc50154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:07.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.709+0000 7efc4e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc500ce240 0x7efc50154d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40496/0 (socket says 172.21.15.76:40496) 2023-11-09T03:47:07.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.709+0000 7efc4e7fc640 1 -- 172.21.15.76:0/1341668234 learned_addr learned my addr 172.21.15.76:0/1341668234 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:07.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.709+0000 7efc4effd640 1 --2- 172.21.15.76:0/1341668234 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc500cde60 0x7efc50142880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:07.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.709+0000 7efc4e7fc640 1 -- 172.21.15.76:0/1341668234 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc500cde60 msgr2=0x7efc50142880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:07.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.709+0000 7efc4f7fe640 1 --2- 172.21.15.76:0/1341668234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc50157540 0x7efc50140e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:07.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.709+0000 7efc4e7fc640 1 --2- 172.21.15.76:0/1341668234 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc500cde60 0x7efc50142880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.709+0000 7efc4e7fc640 1 -- 172.21.15.76:0/1341668234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc50157540 msgr2=0x7efc50140e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:07.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.709+0000 7efc4e7fc640 1 --2- 172.21.15.76:0/1341668234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc50157540 0x7efc50140e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.709+0000 7efc4e7fc640 1 -- 172.21.15.76:0/1341668234 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc44009400 con 0x7efc500ce240 2023-11-09T03:47:07.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.709+0000 7efc4effd640 1 --2- 172.21.15.76:0/1341668234 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc500cde60 0x7efc50142880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:07.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.709+0000 7efc4f7fe640 1 --2- 172.21.15.76:0/1341668234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc50157540 0x7efc50140e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:07.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.709+0000 7efc4e7fc640 1 --2- 172.21.15.76:0/1341668234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc500ce240 0x7efc50154d50 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7efc440098c0 tx=0x7efc440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:07.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.709+0000 7efc37fff640 1 -- 172.21.15.76:0/1341668234 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4403ad60 con 0x7efc500ce240 2023-11-09T03:47:07.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.709+0000 7efc37fff640 1 -- 172.21.15.76:0/1341668234 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc44044400 con 0x7efc500ce240 2023-11-09T03:47:07.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.709+0000 7efc559ee640 1 -- 172.21.15.76:0/1341668234 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc50141da0 con 0x7efc500ce240 2023-11-09T03:47:07.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.709+0000 7efc37fff640 1 -- 172.21.15.76:0/1341668234 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc440447b0 con 0x7efc500ce240 2023-11-09T03:47:07.720 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.709+0000 7efc559ee640 1 -- 172.21.15.76:0/1341668234 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc50155290 con 0x7efc500ce240 2023-11-09T03:47:07.720 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.710+0000 7efc559ee640 1 -- 172.21.15.76:0/1341668234 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc50074290 con 0x7efc500ce240 2023-11-09T03:47:07.720 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.711+0000 7efc37fff640 1 -- 172.21.15.76:0/1341668234 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efc4404d440 con 0x7efc500ce240 2023-11-09T03:47:07.720 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.711+0000 7efc37fff640 1 --2- 172.21.15.76:0/1341668234 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efc3c06ae70 0x7efc3c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:07.720 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.711+0000 7efc37fff640 1 -- 172.21.15.76:0/1341668234 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6438+0+0 (secure 0 0 0) 0x7efc440abda0 con 0x7efc500ce240 2023-11-09T03:47:07.720 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.711+0000 7efc4effd640 1 --2- 172.21.15.76:0/1341668234 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efc3c06ae70 0x7efc3c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:07.721 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.712+0000 7efc4effd640 1 --2- 172.21.15.76:0/1341668234 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efc3c06ae70 0x7efc3c06d330 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efc400012e0 tx=0x7efc40009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:07.721 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.715+0000 7efc37fff640 1 -- 172.21.15.76:0/1341668234 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efc44076b70 con 0x7efc500ce240 2023-11-09T03:47:07.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.915+0000 7efc559ee640 1 -- 172.21.15.76:0/1341668234 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc500d2a90 con 0x7efc500ce240 2023-11-09T03:47:07.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.916+0000 7efc37fff640 1 -- 172.21.15.76:0/1341668234 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4910 (secure 0 0 0) 0x7efc44035090 con 0x7efc500ce240 2023-11-09T03:47:07.917 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:07.917 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4880,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/711686619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":711686619},{"type":"v1","addr":"172.21.15.76:6835","nonce":711686619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/521006441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":521006441},{"type":"v1","addr":"172.21.15.113:6837","nonce":521006441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5212,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2652975857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2652975857},{"type":"v1","addr":"172.21.15.113:6836","nonce":2652975857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:46:51.874121+0000","modified":"2023-11-09T03:46:53.868634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5601":{"gid":5601,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6834/981490030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":981490030},{"type":"v1","addr":"172.21.15.76:6834","nonce":981490030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-09T03:47:07.917 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 16 2023-11-09T03:47:07.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.918+0000 7efc559ee640 1 -- 172.21.15.76:0/1341668234 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efc3c06ae70 msgr2=0x7efc3c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:07.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.918+0000 7efc559ee640 1 --2- 172.21.15.76:0/1341668234 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efc3c06ae70 0x7efc3c06d330 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efc400012e0 tx=0x7efc40009040 comp rx=0 tx=0).stop 2023-11-09T03:47:07.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.918+0000 7efc559ee640 1 -- 172.21.15.76:0/1341668234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc500ce240 msgr2=0x7efc50154d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:07.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.918+0000 7efc559ee640 1 --2- 172.21.15.76:0/1341668234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc500ce240 0x7efc50154d50 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7efc440098c0 tx=0x7efc440046c0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.918+0000 7efc559ee640 1 -- 172.21.15.76:0/1341668234 shutdown_connections 2023-11-09T03:47:07.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.918+0000 7efc559ee640 1 --2- 172.21.15.76:0/1341668234 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efc3c06ae70 0x7efc3c06d330 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.918+0000 7efc559ee640 1 --2- 172.21.15.76:0/1341668234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc50157540 0x7efc50140e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.918+0000 7efc559ee640 1 --2- 172.21.15.76:0/1341668234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc500ce240 0x7efc50154d50 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.918+0000 7efc559ee640 1 --2- 172.21.15.76:0/1341668234 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc500cde60 0x7efc50142880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:07.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.918+0000 7efc559ee640 1 -- 172.21.15.76:0/1341668234 >> 172.21.15.76:0/1341668234 conn(0x7efc50071f30 msgr2=0x7efc500c8890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:07.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.918+0000 7efc559ee640 1 -- 172.21.15.76:0/1341668234 shutdown_connections 2023-11-09T03:47:07.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:07.918+0000 7efc559ee640 1 -- 172.21.15.76:0/1341668234 wait complete. 2023-11-09T03:47:07.929 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:47:08.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.028+0000 7f46467b6640 1 Processor -- start 2023-11-09T03:47:08.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.028+0000 7f46467b6640 1 -- start start 2023-11-09T03:47:08.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.029+0000 7f46467b6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46400cda60 0x7f46400ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:08.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.029+0000 7f46467b6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4640070d40 0x7f464006c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:08.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.029+0000 7f46467b6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46400cc990 con 0x7f46400b0cd0 2023-11-09T03:47:08.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.029+0000 7f46467b6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46400ccc70 con 0x7f4640070d40 2023-11-09T03:47:08.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.029+0000 7f46467b6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46400c9ce0 con 0x7f46400cda60 2023-11-09T03:47:08.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.029+0000 7f4644d53640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4640070d40 0x7f464006c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:08.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.029+0000 7f463ffff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46400cda60 0x7f46400ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:08.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.029+0000 7f463ffff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46400cda60 0x7f46400ca7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:34524/0 (socket says 172.21.15.76:34524) 2023-11-09T03:47:08.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.029+0000 7f4644d53640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4640070d40 0x7f464006c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:43480/0 (socket says 172.21.15.76:43480) 2023-11-09T03:47:08.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.029+0000 7f463f7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f46400b0cd0 0x7f4640070630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:47454/0 (socket says 172.21.15.76:47454) 2023-11-09T03:47:08.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.029+0000 7f463ffff640 1 -- 172.21.15.76:0/578342448 learned_addr learned my addr 172.21.15.76:0/578342448 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:08.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.030+0000 7f463ffff640 1 -- 172.21.15.76:0/578342448 >> v2:172.21.15.113:3300/0 conn(0x7f4640070d40 msgr2=0x7f464006c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:08.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.030+0000 7f463ffff640 1 --2- 172.21.15.76:0/578342448 >> v2:172.21.15.113:3300/0 conn(0x7f4640070d40 0x7f464006c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.030+0000 7f463ffff640 1 -- 172.21.15.76:0/578342448 >> v1:172.21.15.113:6789/0 conn(0x7f46400b0cd0 legacy=0x7f4640070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:08.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.030+0000 7f463ffff640 1 -- 172.21.15.76:0/578342448 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46400c9fc0 con 0x7f46400cda60 2023-11-09T03:47:08.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.030+0000 7f4644d53640 1 --2- 172.21.15.76:0/578342448 >> v2:172.21.15.113:3300/0 conn(0x7f4640070d40 0x7f464006c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:47:08.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.030+0000 7f463ffff640 1 --2- 172.21.15.76:0/578342448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46400cda60 0x7f46400ca7a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f463000aa10 tx=0x7f463002ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33c8131b599d50dd server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:08.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.030+0000 7f463effd640 1 -- 172.21.15.76:0/578342448 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463002dc10 con 0x7f46400cda60 2023-11-09T03:47:08.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.031+0000 7f463effd640 1 -- 172.21.15.76:0/578342448 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463002dd70 con 0x7f46400cda60 2023-11-09T03:47:08.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.031+0000 7f463effd640 1 -- 172.21.15.76:0/578342448 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4630035660 con 0x7f46400cda60 2023-11-09T03:47:08.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.031+0000 7f46467b6640 1 -- 172.21.15.76:0/578342448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46400cda60 msgr2=0x7f46400ca7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:08.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.031+0000 7f46467b6640 1 --2- 172.21.15.76:0/578342448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46400cda60 0x7f46400ca7a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f463000aa10 tx=0x7f463002ce60 comp rx=0 tx=0).stop 2023-11-09T03:47:08.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.031+0000 7f46467b6640 1 -- 172.21.15.76:0/578342448 shutdown_connections 2023-11-09T03:47:08.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.031+0000 7f46467b6640 1 --2- 172.21.15.76:0/578342448 >> v2:172.21.15.113:3300/0 conn(0x7f4640070d40 0x7f464006c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.031+0000 7f46467b6640 1 --2- 172.21.15.76:0/578342448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46400cda60 0x7f46400ca7a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.031+0000 7f46467b6640 1 -- 172.21.15.76:0/578342448 >> 172.21.15.76:0/578342448 conn(0x7f46400ce950 msgr2=0x7f46400ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:08.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.031+0000 7f46467b6640 1 -- 172.21.15.76:0/578342448 shutdown_connections 2023-11-09T03:47:08.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.031+0000 7f46467b6640 1 -- 172.21.15.76:0/578342448 wait complete. 2023-11-09T03:47:08.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.032+0000 7f46467b6640 1 Processor -- start 2023-11-09T03:47:08.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.032+0000 7f46467b6640 1 -- start start 2023-11-09T03:47:08.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.033+0000 7f46467b6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4640070d40 0x7f4640145910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:08.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.033+0000 7f46467b6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46400b0cd0 0x7f46400b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:08.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.033+0000 7f46467b6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46400cda60 0x7f46400b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:08.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.033+0000 7f46467b6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46401471b0 con 0x7f4640070d40 2023-11-09T03:47:08.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.033+0000 7f46467b6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4640147490 con 0x7f46400b0cd0 2023-11-09T03:47:08.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.033+0000 7f46467b6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4640147770 con 0x7f46400cda60 2023-11-09T03:47:08.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.033+0000 7f463ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4640070d40 0x7f4640145910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:08.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.033+0000 7f463ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4640070d40 0x7f4640145910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40508/0 (socket says 172.21.15.76:40508) 2023-11-09T03:47:08.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.033+0000 7f463ffff640 1 -- 172.21.15.76:0/914880479 learned_addr learned my addr 172.21.15.76:0/914880479 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:08.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.034+0000 7f463f7fe640 1 --2- 172.21.15.76:0/914880479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46400b0cd0 0x7f46400b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:08.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.034+0000 7f4644d53640 1 --2- 172.21.15.76:0/914880479 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46400cda60 0x7f46400b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:08.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.034+0000 7f463ffff640 1 -- 172.21.15.76:0/914880479 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46400cda60 msgr2=0x7f46400b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:08.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.034+0000 7f463ffff640 1 --2- 172.21.15.76:0/914880479 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46400cda60 0x7f46400b3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.034+0000 7f463ffff640 1 -- 172.21.15.76:0/914880479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46400b0cd0 msgr2=0x7f46400b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:08.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.034+0000 7f463ffff640 1 --2- 172.21.15.76:0/914880479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46400b0cd0 0x7f46400b35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.034+0000 7f463ffff640 1 -- 172.21.15.76:0/914880479 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4638009a70 con 0x7f4640070d40 2023-11-09T03:47:08.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.034+0000 7f463f7fe640 1 --2- 172.21.15.76:0/914880479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46400b0cd0 0x7f46400b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:08.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.034+0000 7f463ffff640 1 --2- 172.21.15.76:0/914880479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4640070d40 0x7f4640145910 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f463002cea0 tx=0x7f46300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:08.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.034+0000 7f463cff9640 1 -- 172.21.15.76:0/914880479 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463003f600 con 0x7f4640070d40 2023-11-09T03:47:08.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.034+0000 7f463cff9640 1 -- 172.21.15.76:0/914880479 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463003f780 con 0x7f4640070d40 2023-11-09T03:47:08.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.034+0000 7f46467b6640 1 -- 172.21.15.76:0/914880479 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f463000a670 con 0x7f4640070d40 2023-11-09T03:47:08.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.034+0000 7f463cff9640 1 -- 172.21.15.76:0/914880479 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463003fb30 con 0x7f4640070d40 2023-11-09T03:47:08.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.034+0000 7f46467b6640 1 -- 172.21.15.76:0/914880479 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4640147d30 con 0x7f4640070d40 2023-11-09T03:47:08.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.035+0000 7f46467b6640 1 -- 172.21.15.76:0/914880479 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4640074290 con 0x7f4640070d40 2023-11-09T03:47:08.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.035+0000 7f463cff9640 1 -- 172.21.15.76:0/914880479 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f463003fc90 con 0x7f4640070d40 2023-11-09T03:47:08.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.036+0000 7f463cff9640 1 --2- 172.21.15.76:0/914880479 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f462006ae70 0x7f462006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:08.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.036+0000 7f463cff9640 1 -- 172.21.15.76:0/914880479 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6438+0+0 (secure 0 0 0) 0x7f46300abd60 con 0x7f4640070d40 2023-11-09T03:47:08.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.036+0000 7f463f7fe640 1 --2- 172.21.15.76:0/914880479 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f462006ae70 0x7f462006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:08.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.037+0000 7f463f7fe640 1 --2- 172.21.15.76:0/914880479 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f462006ae70 0x7f462006d330 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f46401f3f60 tx=0x7f4628009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:08.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.040+0000 7f463cff9640 1 -- 172.21.15.76:0/914880479 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4630076ab0 con 0x7f4640070d40 2023-11-09T03:47:08.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.240+0000 7f46467b6640 1 -- 172.21.15.76:0/914880479 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46400b48d0 con 0x7f4640070d40 2023-11-09T03:47:08.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.241+0000 7f463cff9640 1 -- 172.21.15.76:0/914880479 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f463007a0d0 con 0x7f4640070d40 2023-11-09T03:47:08.241 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:08.242 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4880,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/711686619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":711686619},{"type":"v1","addr":"172.21.15.76:6835","nonce":711686619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/521006441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":521006441},{"type":"v1","addr":"172.21.15.113:6837","nonce":521006441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5212,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2652975857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2652975857},{"type":"v1","addr":"172.21.15.113:6836","nonce":2652975857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:46:51.874121+0000","modified":"2023-11-09T03:46:53.868634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5601":{"gid":5601,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6834/981490030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":981490030},{"type":"v1","addr":"172.21.15.76:6834","nonce":981490030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-09T03:47:08.242 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 16 2023-11-09T03:47:08.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.243+0000 7f46467b6640 1 -- 172.21.15.76:0/914880479 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f462006ae70 msgr2=0x7f462006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:08.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.243+0000 7f46467b6640 1 --2- 172.21.15.76:0/914880479 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f462006ae70 0x7f462006d330 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f46401f3f60 tx=0x7f4628009040 comp rx=0 tx=0).stop 2023-11-09T03:47:08.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.243+0000 7f46467b6640 1 -- 172.21.15.76:0/914880479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4640070d40 msgr2=0x7f4640145910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:08.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.243+0000 7f46467b6640 1 --2- 172.21.15.76:0/914880479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4640070d40 0x7f4640145910 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f463002cea0 tx=0x7f46300046c0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.243+0000 7f46467b6640 1 -- 172.21.15.76:0/914880479 shutdown_connections 2023-11-09T03:47:08.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.243+0000 7f46467b6640 1 --2- 172.21.15.76:0/914880479 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f462006ae70 0x7f462006d330 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.243+0000 7f46467b6640 1 --2- 172.21.15.76:0/914880479 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46400cda60 0x7f46400b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.243+0000 7f46467b6640 1 --2- 172.21.15.76:0/914880479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46400b0cd0 0x7f46400b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.243+0000 7f46467b6640 1 --2- 172.21.15.76:0/914880479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4640070d40 0x7f4640145910 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.243+0000 7f46467b6640 1 -- 172.21.15.76:0/914880479 >> 172.21.15.76:0/914880479 conn(0x7f46400ce950 msgr2=0x7f46400af8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:08.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.243+0000 7f46467b6640 1 -- 172.21.15.76:0/914880479 shutdown_connections 2023-11-09T03:47:08.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.243+0000 7f46467b6640 1 -- 172.21.15.76:0/914880479 wait complete. 2023-11-09T03:47:08.254 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:47:08.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.354+0000 7faef3b33640 1 Processor -- start 2023-11-09T03:47:08.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.354+0000 7faef3b33640 1 -- start start 2023-11-09T03:47:08.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.355+0000 7faef3b33640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faeec0ce240 0x7faeec157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:08.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.355+0000 7faef3b33640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeec15e000 con 0x7faeec0cde60 2023-11-09T03:47:08.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.355+0000 7faef3b33640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeec15e2e0 con 0x7faeec157540 2023-11-09T03:47:08.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.355+0000 7faef3b33640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeec15e5c0 con 0x7faeec0ce240 2023-11-09T03:47:08.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.355+0000 7faef18cf640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7faeec0cde60 0x7faeec15d560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37448/0 (socket says 172.21.15.76:37448) 2023-11-09T03:47:08.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.355+0000 7faef18cf640 1 -- 172.21.15.76:0/2484252824 learned_addr learned my addr 172.21.15.76:0/2484252824 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:08.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.356+0000 7faef08cd640 1 -- 172.21.15.76:0/2484252824 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 650094496 0 0) 0x7faeec15e000 con 0x7faeec0cde60 2023-11-09T03:47:08.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.356+0000 7faef08cd640 1 -- 172.21.15.76:0/2484252824 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faed8003520 con 0x7faeec0cde60 2023-11-09T03:47:08.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.356+0000 7faef10ce640 1 --2- 172.21.15.76:0/2484252824 >> v2:172.21.15.113:3300/0 conn(0x7faeec0ce240 0x7faeec157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:08.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.356+0000 7faef08cd640 1 -- 172.21.15.76:0/2484252824 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1713622848 0 0) 0x7faed8003520 con 0x7faeec0cde60 2023-11-09T03:47:08.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.356+0000 7faef08cd640 1 -- 172.21.15.76:0/2484252824 >> v2:172.21.15.113:3300/0 conn(0x7faeec0ce240 msgr2=0x7faeec157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:08.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.356+0000 7faef08cd640 1 --2- 172.21.15.76:0/2484252824 >> v2:172.21.15.113:3300/0 conn(0x7faeec0ce240 0x7faeec157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.356+0000 7faef08cd640 1 -- 172.21.15.76:0/2484252824 >> v1:172.21.15.113:6789/0 conn(0x7faeec157540 legacy=0x7faeec0d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:08.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.356+0000 7faef08cd640 1 -- 172.21.15.76:0/2484252824 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeec0d1120 con 0x7faeec0cde60 2023-11-09T03:47:08.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.356+0000 7faef08cd640 1 -- 172.21.15.76:0/2484252824 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7faee80019f0 con 0x7faeec0cde60 2023-11-09T03:47:08.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.356+0000 7faef08cd640 1 -- 172.21.15.76:0/2484252824 >> v1:172.21.15.76:6789/0 conn(0x7faeec0cde60 legacy=0x7faeec15d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:08.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.356+0000 7faef08cd640 1 --2- 172.21.15.76:0/2484252824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faed8003970 0x7faed8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:08.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.356+0000 7faef08cd640 1 --2- 172.21.15.76:0/2484252824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faed80047c0 0x7faed8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:08.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.356+0000 7faef08cd640 1 --2- 172.21.15.76:0/2484252824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faed8005180 0x7faed800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:08.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.356+0000 7faef08cd640 1 -- 172.21.15.76:0/2484252824 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeec15e5c0 con 0x7faed8005180 2023-11-09T03:47:08.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.356+0000 7faef08cd640 1 -- 172.21.15.76:0/2484252824 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed800c020 con 0x7faed80047c0 2023-11-09T03:47:08.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.356+0000 7faef08cd640 1 -- 172.21.15.76:0/2484252824 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed800c1e0 con 0x7faed8003970 2023-11-09T03:47:08.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.356+0000 7faef20d0640 1 --2- 172.21.15.76:0/2484252824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faed8005180 0x7faed800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:08.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.356+0000 7faef18cf640 1 --2- 172.21.15.76:0/2484252824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faed8003970 0x7faed8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:08.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.356+0000 7faef10ce640 1 --2- 172.21.15.76:0/2484252824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faed80047c0 0x7faed8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:08.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.356+0000 7faef20d0640 1 -- 172.21.15.76:0/2484252824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faed8003970 msgr2=0x7faed8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:08.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.356+0000 7faef20d0640 1 --2- 172.21.15.76:0/2484252824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faed8003970 0x7faed8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.356+0000 7faef20d0640 1 -- 172.21.15.76:0/2484252824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faed80047c0 msgr2=0x7faed8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:08.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.356+0000 7faef20d0640 1 --2- 172.21.15.76:0/2484252824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faed80047c0 0x7faed8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.356+0000 7faef20d0640 1 -- 172.21.15.76:0/2484252824 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed8003520 con 0x7faed8005180 2023-11-09T03:47:08.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.357+0000 7faef20d0640 1 -- 172.21.15.76:0/2484252824 reap_dead start 2023-11-09T03:47:08.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.357+0000 7faef20d0640 1 --2- 172.21.15.76:0/2484252824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faed8005180 0x7faed800b980 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7faee4001750 tx=0x7faed80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d8d2f80ca291e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:08.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.357+0000 7faef08cd640 1 -- 172.21.15.76:0/2484252824 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee40052d0 con 0x7faed8005180 2023-11-09T03:47:08.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.357+0000 7faef08cd640 1 -- 172.21.15.76:0/2484252824 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee402e050 con 0x7faed8005180 2023-11-09T03:47:08.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.357+0000 7faef08cd640 1 -- 172.21.15.76:0/2484252824 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee40035f0 con 0x7faed8005180 2023-11-09T03:47:08.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.358+0000 7faef3b33640 1 -- 172.21.15.76:0/2484252824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faed8005180 msgr2=0x7faed800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:08.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.358+0000 7faef3b33640 1 --2- 172.21.15.76:0/2484252824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faed8005180 0x7faed800b980 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7faee4001750 tx=0x7faed80057f0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.358+0000 7faef3b33640 1 -- 172.21.15.76:0/2484252824 shutdown_connections 2023-11-09T03:47:08.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.358+0000 7faef3b33640 1 --2- 172.21.15.76:0/2484252824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faed8005180 0x7faed800b980 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.358+0000 7faef3b33640 1 -- 172.21.15.76:0/2484252824 >> 172.21.15.76:0/2484252824 conn(0x7faeec071f30 msgr2=0x7faeec0cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:08.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.358+0000 7faef3b33640 1 -- 172.21.15.76:0/2484252824 shutdown_connections 2023-11-09T03:47:08.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.358+0000 7faef3b33640 1 -- 172.21.15.76:0/2484252824 wait complete. 2023-11-09T03:47:08.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.359+0000 7faef3b33640 1 Processor -- start 2023-11-09T03:47:08.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.359+0000 7faef3b33640 1 -- start start 2023-11-09T03:47:08.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.360+0000 7faef3b33640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faed8005180 0x7faeec0b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:08.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.360+0000 7faef3b33640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faeec149b50 0x7faeec149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:08.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.360+0000 7faef3b33640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faeec15bd20 0x7faeec15c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:08.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.360+0000 7faef3b33640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeec14af60 con 0x7faeec149b50 2023-11-09T03:47:08.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.360+0000 7faef3b33640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeec15c6e0 con 0x7faeec15bd20 2023-11-09T03:47:08.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.360+0000 7faef3b33640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeec1564e0 con 0x7faed8005180 2023-11-09T03:47:08.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.360+0000 7faef10ce640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faeec149b50 0x7faeec149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:08.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.360+0000 7faef10ce640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faeec149b50 0x7faeec149fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40526/0 (socket says 172.21.15.76:40526) 2023-11-09T03:47:08.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.360+0000 7faef10ce640 1 -- 172.21.15.76:0/4110212121 learned_addr learned my addr 172.21.15.76:0/4110212121 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:08.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.360+0000 7faef18cf640 1 --2- 172.21.15.76:0/4110212121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faed8005180 0x7faeec0b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:08.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.360+0000 7faef20d0640 1 --2- 172.21.15.76:0/4110212121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faeec15bd20 0x7faeec15c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:08.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.360+0000 7faef10ce640 1 -- 172.21.15.76:0/4110212121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faed8005180 msgr2=0x7faeec0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:08.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.360+0000 7faef10ce640 1 --2- 172.21.15.76:0/4110212121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faed8005180 0x7faeec0b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.360+0000 7faef10ce640 1 -- 172.21.15.76:0/4110212121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faeec15bd20 msgr2=0x7faeec15c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:08.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.360+0000 7faef10ce640 1 --2- 172.21.15.76:0/4110212121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faeec15bd20 0x7faeec15c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.360+0000 7faef10ce640 1 -- 172.21.15.76:0/4110212121 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeec14a6c0 con 0x7faeec149b50 2023-11-09T03:47:08.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.360+0000 7faef20d0640 1 --2- 172.21.15.76:0/4110212121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faeec15bd20 0x7faeec15c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:08.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.360+0000 7faef18cf640 1 --2- 172.21.15.76:0/4110212121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faed8005180 0x7faeec0b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:08.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.360+0000 7faef10ce640 1 --2- 172.21.15.76:0/4110212121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faeec149b50 0x7faeec149fd0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7faedc0118e0 tx=0x7faedc011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:08.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.360+0000 7faee27fc640 1 -- 172.21.15.76:0/4110212121 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faedc00faf0 con 0x7faeec149b50 2023-11-09T03:47:08.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.360+0000 7faee27fc640 1 -- 172.21.15.76:0/4110212121 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faedc00fc50 con 0x7faeec149b50 2023-11-09T03:47:08.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.360+0000 7faef3b33640 1 -- 172.21.15.76:0/4110212121 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeec14a9a0 con 0x7faeec149b50 2023-11-09T03:47:08.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.360+0000 7faee27fc640 1 -- 172.21.15.76:0/4110212121 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faedc014440 con 0x7faeec149b50 2023-11-09T03:47:08.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.360+0000 7faef3b33640 1 -- 172.21.15.76:0/4110212121 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeec14c760 con 0x7faeec149b50 2023-11-09T03:47:08.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.361+0000 7faef3b33640 1 -- 172.21.15.76:0/4110212121 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeec0d07e0 con 0x7faeec149b50 2023-11-09T03:47:08.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.362+0000 7faee27fc640 1 -- 172.21.15.76:0/4110212121 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faedc0145a0 con 0x7faeec149b50 2023-11-09T03:47:08.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.362+0000 7faee27fc640 1 --2- 172.21.15.76:0/4110212121 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faed006ae70 0x7faed006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:08.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.362+0000 7faee27fc640 1 -- 172.21.15.76:0/4110212121 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6438+0+0 (secure 0 0 0) 0x7faedc08a1a0 con 0x7faeec149b50 2023-11-09T03:47:08.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.363+0000 7faef18cf640 1 --2- 172.21.15.76:0/4110212121 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faed006ae70 0x7faed006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:08.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.363+0000 7faef18cf640 1 --2- 172.21.15.76:0/4110212121 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faed006ae70 0x7faed006d330 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7faee80013d0 tx=0x7faee8006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:08.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.367+0000 7faee27fc640 1 -- 172.21.15.76:0/4110212121 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faedc055000 con 0x7faeec149b50 2023-11-09T03:47:08.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.533+0000 7faef3b33640 1 -- 172.21.15.76:0/4110212121 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faeec074290 con 0x7faeec149b50 2023-11-09T03:47:08.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.534+0000 7faee27fc640 1 -- 172.21.15.76:0/4110212121 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18601 (secure 0 0 0) 0x7faedc058620 con 0x7faeec149b50 2023-11-09T03:47:08.535 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:08.535 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":42,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:47:02.574636+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:46:50.064765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-09T03:46:50.837125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":1,"length":1}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":1,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:47:08.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.536+0000 7faef3b33640 1 -- 172.21.15.76:0/4110212121 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faed006ae70 msgr2=0x7faed006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:08.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.536+0000 7faef3b33640 1 --2- 172.21.15.76:0/4110212121 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faed006ae70 0x7faed006d330 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7faee80013d0 tx=0x7faee8006040 comp rx=0 tx=0).stop 2023-11-09T03:47:08.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.536+0000 7faef3b33640 1 -- 172.21.15.76:0/4110212121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faeec149b50 msgr2=0x7faeec149fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:08.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.536+0000 7faef3b33640 1 --2- 172.21.15.76:0/4110212121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faeec149b50 0x7faeec149fd0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7faedc0118e0 tx=0x7faedc011da0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.536+0000 7faef3b33640 1 -- 172.21.15.76:0/4110212121 shutdown_connections 2023-11-09T03:47:08.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.536+0000 7faef3b33640 1 --2- 172.21.15.76:0/4110212121 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faed006ae70 0x7faed006d330 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.536+0000 7faef3b33640 1 --2- 172.21.15.76:0/4110212121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faeec15bd20 0x7faeec15c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.536+0000 7faef3b33640 1 --2- 172.21.15.76:0/4110212121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faeec149b50 0x7faeec149fd0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.536+0000 7faef3b33640 1 --2- 172.21.15.76:0/4110212121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faed8005180 0x7faeec0b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.536+0000 7faef3b33640 1 -- 172.21.15.76:0/4110212121 >> 172.21.15.76:0/4110212121 conn(0x7faeec071f30 msgr2=0x7faeec0c6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:08.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.537+0000 7faef3b33640 1 -- 172.21.15.76:0/4110212121 shutdown_connections 2023-11-09T03:47:08.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.537+0000 7faef3b33640 1 -- 172.21.15.76:0/4110212121 wait complete. 2023-11-09T03:47:08.549 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-09T03:47:08.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.651+0000 7f8c2e2e0640 1 Processor -- start 2023-11-09T03:47:08.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.651+0000 7f8c2e2e0640 1 -- start start 2023-11-09T03:47:08.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.652+0000 7f8c2e2e0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c28070e20 0x7f8c280adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:08.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.652+0000 7f8c2e2e0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c280c3900 con 0x7f8c2815ad30 2023-11-09T03:47:08.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.652+0000 7f8c2e2e0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c280c3be0 con 0x7f8c280ad730 2023-11-09T03:47:08.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.652+0000 7f8c2e2e0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c280ae260 con 0x7f8c28070e20 2023-11-09T03:47:08.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.652+0000 7f8c27fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8c2815ad30 0x7f8c28070710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37454/0 (socket says 172.21.15.76:37454) 2023-11-09T03:47:08.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.652+0000 7f8c27fff640 1 -- 172.21.15.76:0/2002566380 learned_addr learned my addr 172.21.15.76:0/2002566380 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:08.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.652+0000 7f8c277fe640 1 --2- 172.21.15.76:0/2002566380 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c28070e20 0x7f8c280adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:08.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.652+0000 7f8c26ffd640 1 -- 172.21.15.76:0/2002566380 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 512168533 0 0) 0x7f8c280c3900 con 0x7f8c2815ad30 2023-11-09T03:47:08.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.652+0000 7f8c26ffd640 1 -- 172.21.15.76:0/2002566380 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c10003520 con 0x7f8c2815ad30 2023-11-09T03:47:08.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.652+0000 7f8c26ffd640 1 -- 172.21.15.76:0/2002566380 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2609813719 0 0) 0x7f8c280c3be0 con 0x7f8c280ad730 2023-11-09T03:47:08.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.652+0000 7f8c26ffd640 1 -- 172.21.15.76:0/2002566380 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c280c3900 con 0x7f8c280ad730 2023-11-09T03:47:08.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.652+0000 7f8c26ffd640 1 -- 172.21.15.76:0/2002566380 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 617021637 0 0) 0x7f8c10003520 con 0x7f8c2815ad30 2023-11-09T03:47:08.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.652+0000 7f8c26ffd640 1 -- 172.21.15.76:0/2002566380 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c28070e20 msgr2=0x7f8c280adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:08.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.652+0000 7f8c26ffd640 1 --2- 172.21.15.76:0/2002566380 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c28070e20 0x7f8c280adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.652+0000 7f8c26ffd640 1 -- 172.21.15.76:0/2002566380 >> v1:172.21.15.113:6789/0 conn(0x7f8c280ad730 legacy=0x7f8c2815e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:08.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.652+0000 7f8c26ffd640 1 -- 172.21.15.76:0/2002566380 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c280c3ec0 con 0x7f8c2815ad30 2023-11-09T03:47:08.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.652+0000 7f8c277fe640 1 --2- 172.21.15.76:0/2002566380 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c28070e20 0x7f8c280adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:47:08.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.652+0000 7f8c26ffd640 1 -- 172.21.15.76:0/2002566380 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f8c18001670 con 0x7f8c2815ad30 2023-11-09T03:47:08.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.652+0000 7f8c26ffd640 1 -- 172.21.15.76:0/2002566380 >> v1:172.21.15.76:6789/0 conn(0x7f8c2815ad30 legacy=0x7f8c28070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:08.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.653+0000 7f8c26ffd640 1 --2- 172.21.15.76:0/2002566380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c10003b30 0x7f8c10003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:08.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.653+0000 7f8c26ffd640 1 --2- 172.21.15.76:0/2002566380 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c28070e20 0x7f8c100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:08.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.653+0000 7f8c26ffd640 1 --2- 172.21.15.76:0/2002566380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c10004d00 0x7f8c10005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:08.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.653+0000 7f8c26ffd640 1 -- 172.21.15.76:0/2002566380 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c280ae260 con 0x7f8c10004d00 2023-11-09T03:47:08.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.653+0000 7f8c26ffd640 1 -- 172.21.15.76:0/2002566380 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1000ba90 con 0x7f8c10003b30 2023-11-09T03:47:08.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.653+0000 7f8c26ffd640 1 -- 172.21.15.76:0/2002566380 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1000bc50 con 0x7f8c28070e20 2023-11-09T03:47:08.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.653+0000 7f8c2c87d640 1 --2- 172.21.15.76:0/2002566380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c10004d00 0x7f8c10005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:08.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.653+0000 7f8c27fff640 1 --2- 172.21.15.76:0/2002566380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c10003b30 0x7f8c10003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:08.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.653+0000 7f8c277fe640 1 --2- 172.21.15.76:0/2002566380 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c28070e20 0x7f8c100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:08.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.653+0000 7f8c2c87d640 1 -- 172.21.15.76:0/2002566380 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c28070e20 msgr2=0x7f8c100047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:08.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.653+0000 7f8c2c87d640 1 --2- 172.21.15.76:0/2002566380 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c28070e20 0x7f8c100047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.653+0000 7f8c2c87d640 1 -- 172.21.15.76:0/2002566380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c10003b30 msgr2=0x7f8c10003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:08.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.653+0000 7f8c2c87d640 1 --2- 172.21.15.76:0/2002566380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c10003b30 0x7f8c10003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.653+0000 7f8c2c87d640 1 -- 172.21.15.76:0/2002566380 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1c009400 con 0x7f8c10004d00 2023-11-09T03:47:08.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.653+0000 7f8c277fe640 1 --2- 172.21.15.76:0/2002566380 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c28070e20 0x7f8c100047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:08.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.654+0000 7f8c2c87d640 1 --2- 172.21.15.76:0/2002566380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c10004d00 0x7f8c10005180 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8c20001fd0 tx=0x7f8c100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8411805021ba5a23 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:08.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.654+0000 7f8c26ffd640 1 -- 172.21.15.76:0/2002566380 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c20003b70 con 0x7f8c10004d00 2023-11-09T03:47:08.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.654+0000 7f8c26ffd640 1 -- 172.21.15.76:0/2002566380 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c20003cd0 con 0x7f8c10004d00 2023-11-09T03:47:08.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.654+0000 7f8c26ffd640 1 -- 172.21.15.76:0/2002566380 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c200052f0 con 0x7f8c10004d00 2023-11-09T03:47:08.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.654+0000 7f8c2e2e0640 1 -- 172.21.15.76:0/2002566380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c10004d00 msgr2=0x7f8c10005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:08.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.654+0000 7f8c2e2e0640 1 --2- 172.21.15.76:0/2002566380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c10004d00 0x7f8c10005180 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8c20001fd0 tx=0x7f8c100057f0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.654+0000 7f8c2c87d640 1 -- 172.21.15.76:0/2002566380 reap_dead start 2023-11-09T03:47:08.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.654+0000 7f8c2e2e0640 1 -- 172.21.15.76:0/2002566380 shutdown_connections 2023-11-09T03:47:08.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.654+0000 7f8c2e2e0640 1 -- 172.21.15.76:0/2002566380 >> 172.21.15.76:0/2002566380 conn(0x7f8c280ce950 msgr2=0x7f8c280ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:08.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.654+0000 7f8c2e2e0640 1 -- 172.21.15.76:0/2002566380 shutdown_connections 2023-11-09T03:47:08.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.654+0000 7f8c2e2e0640 1 -- 172.21.15.76:0/2002566380 wait complete. 2023-11-09T03:47:08.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.655+0000 7f8c2e2e0640 1 Processor -- start 2023-11-09T03:47:08.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.656+0000 7f8c2e2e0640 1 -- start start 2023-11-09T03:47:08.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.656+0000 7f8c2e2e0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c2815c270 0x7f8c2815b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:08.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.656+0000 7f8c2e2e0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c2815dbc0 0x7f8c280c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:08.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.656+0000 7f8c2e2e0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c280c81a0 0x7f8c280ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:08.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.656+0000 7f8c2e2e0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c280c8620 con 0x7f8c280c81a0 2023-11-09T03:47:08.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.656+0000 7f8c2e2e0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2815e040 con 0x7f8c2815dbc0 2023-11-09T03:47:08.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.656+0000 7f8c2e2e0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c281f6720 con 0x7f8c2815c270 2023-11-09T03:47:08.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.656+0000 7f8c2c87d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c280c81a0 0x7f8c280ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:08.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.656+0000 7f8c2c87d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c280c81a0 0x7f8c280ab8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40536/0 (socket says 172.21.15.76:40536) 2023-11-09T03:47:08.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.656+0000 7f8c2c87d640 1 -- 172.21.15.76:0/950823118 learned_addr learned my addr 172.21.15.76:0/950823118 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:08.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.657+0000 7f8c277fe640 1 --2- 172.21.15.76:0/950823118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c2815dbc0 0x7f8c280c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:08.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.657+0000 7f8c2c87d640 1 -- 172.21.15.76:0/950823118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c2815c270 msgr2=0x7f8c2815b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:08.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.657+0000 7f8c27fff640 1 --2- 172.21.15.76:0/950823118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c2815c270 0x7f8c2815b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:08.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.657+0000 7f8c2c87d640 1 --2- 172.21.15.76:0/950823118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c2815c270 0x7f8c2815b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.657+0000 7f8c2c87d640 1 -- 172.21.15.76:0/950823118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c2815dbc0 msgr2=0x7f8c280c7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:08.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.657+0000 7f8c2c87d640 1 --2- 172.21.15.76:0/950823118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c2815dbc0 0x7f8c280c7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.657+0000 7f8c2c87d640 1 -- 172.21.15.76:0/950823118 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c281f6a00 con 0x7f8c280c81a0 2023-11-09T03:47:08.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.657+0000 7f8c277fe640 1 --2- 172.21.15.76:0/950823118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c2815dbc0 0x7f8c280c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:08.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.657+0000 7f8c2c87d640 1 --2- 172.21.15.76:0/950823118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c280c81a0 0x7f8c280ab8d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8c20001fd0 tx=0x7f8c20001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:08.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.657+0000 7f8c24ff9640 1 -- 172.21.15.76:0/950823118 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c20041070 con 0x7f8c280c81a0 2023-11-09T03:47:08.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.657+0000 7f8c24ff9640 1 -- 172.21.15.76:0/950823118 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c20003e40 con 0x7f8c280c81a0 2023-11-09T03:47:08.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.657+0000 7f8c24ff9640 1 -- 172.21.15.76:0/950823118 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c200052e0 con 0x7f8c280c81a0 2023-11-09T03:47:08.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.657+0000 7f8c2e2e0640 1 -- 172.21.15.76:0/950823118 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c281f6ce0 con 0x7f8c280c81a0 2023-11-09T03:47:08.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.657+0000 7f8c2e2e0640 1 -- 172.21.15.76:0/950823118 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c281f72a0 con 0x7f8c280c81a0 2023-11-09T03:47:08.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.658+0000 7f8c2e2e0640 1 -- 172.21.15.76:0/950823118 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c28074290 con 0x7f8c280c81a0 2023-11-09T03:47:08.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.658+0000 7f8c24ff9640 1 -- 172.21.15.76:0/950823118 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8c20005440 con 0x7f8c280c81a0 2023-11-09T03:47:08.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.659+0000 7f8c24ff9640 1 --2- 172.21.15.76:0/950823118 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8c1406aea0 0x7f8c1406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:08.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.659+0000 7f8c24ff9640 1 -- 172.21.15.76:0/950823118 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6438+0+0 (secure 0 0 0) 0x7f8c200aa9b0 con 0x7f8c280c81a0 2023-11-09T03:47:08.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.659+0000 7f8c27fff640 1 --2- 172.21.15.76:0/950823118 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8c1406aea0 0x7f8c1406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:08.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.660+0000 7f8c27fff640 1 --2- 172.21.15.76:0/950823118 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8c1406aea0 0x7f8c1406d360 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8c18004840 tx=0x7f8c18009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:08.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.663+0000 7f8c24ff9640 1 -- 172.21.15.76:0/950823118 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8c20075780 con 0x7f8c280c81a0 2023-11-09T03:47:08.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.864+0000 7f8c2e2e0640 1 -- 172.21.15.76:0/950823118 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8c28074290 con 0x7f8c280c81a0 2023-11-09T03:47:08.865 INFO:tasks.ceph.mon.a.smithi076.stderr:2023-11-09T03:47:08.865+0000 7fe4a393d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-09T03:47:08.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.868+0000 7f8c24ff9640 1 -- 172.21.15.76:0/950823118 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8c20078da0 con 0x7f8c280c81a0 2023-11-09T03:47:08.868 INFO:teuthology.orchestra.run.smithi076.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-09T03:47:08.871 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.871+0000 7f8c2e2e0640 1 -- 172.21.15.76:0/950823118 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8c1406aea0 msgr2=0x7f8c1406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:08.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.871+0000 7f8c2e2e0640 1 --2- 172.21.15.76:0/950823118 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8c1406aea0 0x7f8c1406d360 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8c18004840 tx=0x7f8c18009040 comp rx=0 tx=0).stop 2023-11-09T03:47:08.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.871+0000 7f8c2e2e0640 1 -- 172.21.15.76:0/950823118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c280c81a0 msgr2=0x7f8c280ab8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:08.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.871+0000 7f8c2e2e0640 1 --2- 172.21.15.76:0/950823118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c280c81a0 0x7f8c280ab8d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8c20001fd0 tx=0x7f8c20001650 comp rx=0 tx=0).stop 2023-11-09T03:47:08.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.871+0000 7f8c2e2e0640 1 -- 172.21.15.76:0/950823118 shutdown_connections 2023-11-09T03:47:08.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.871+0000 7f8c2e2e0640 1 --2- 172.21.15.76:0/950823118 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8c1406aea0 0x7f8c1406d360 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.871+0000 7f8c2e2e0640 1 --2- 172.21.15.76:0/950823118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c280c81a0 0x7f8c280ab8d0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.871+0000 7f8c2e2e0640 1 --2- 172.21.15.76:0/950823118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c2815dbc0 0x7f8c280c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.871+0000 7f8c2e2e0640 1 --2- 172.21.15.76:0/950823118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c2815c270 0x7f8c2815b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.871+0000 7f8c2e2e0640 1 -- 172.21.15.76:0/950823118 >> 172.21.15.76:0/950823118 conn(0x7f8c280ce950 msgr2=0x7f8c280b0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:08.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.871+0000 7f8c2e2e0640 1 -- 172.21.15.76:0/950823118 shutdown_connections 2023-11-09T03:47:08.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.871+0000 7f8c2e2e0640 1 -- 172.21.15.76:0/950823118 wait complete. 2023-11-09T03:47:08.874 INFO:tasks.ceph.mds.a.smithi076.stderr: -620> 2023-11-09T03:46:57.335+0000 7f0aa61a1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:47:08.883 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-09T03:47:08.897 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2023-11-09T03:47:08.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.985+0000 7f4f1fd9e640 1 Processor -- start 2023-11-09T03:47:08.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.985+0000 7f4f1fd9e640 1 -- start start 2023-11-09T03:47:08.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.986+0000 7f4f1fd9e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f18159c00 0x7f4f180cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:08.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.986+0000 7f4f1fd9e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4f180c48e0 0x7f4f180c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:08.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.986+0000 7f4f1fd9e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f1815aff0 con 0x7f4f180adac0 2023-11-09T03:47:08.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.986+0000 7f4f1fd9e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f180c52a0 con 0x7f4f180c48e0 2023-11-09T03:47:08.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.986+0000 7f4f1fd9e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1815a3e0 con 0x7f4f18159c00 2023-11-09T03:47:08.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.986+0000 7f4f1d339640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4f180c48e0 0x7f4f180c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:08.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.986+0000 7f4f1e33b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f4f180adac0 0x7f4f180adef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37476/0 (socket says 172.21.15.76:37476) 2023-11-09T03:47:08.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.986+0000 7f4f1e33b640 1 -- 172.21.15.76:0/1949562743 learned_addr learned my addr 172.21.15.76:0/1949562743 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:08.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.986+0000 7f4f1d339640 1 -- 172.21.15.76:0/1949562743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f18159c00 msgr2=0x7f4f180cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:08.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.986+0000 7f4f1db3a640 1 --2- 172.21.15.76:0/1949562743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f18159c00 0x7f4f180cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:08.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.986+0000 7f4f1d339640 1 --2- 172.21.15.76:0/1949562743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f18159c00 0x7f4f180cc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.986+0000 7f4f1cb38640 1 -- 172.21.15.76:0/1949562743 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1623963298 0 0) 0x7f4f1815aff0 con 0x7f4f180adac0 2023-11-09T03:47:08.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.986+0000 7f4f1d339640 1 -- 172.21.15.76:0/1949562743 >> v1:172.21.15.76:6789/0 conn(0x7f4f180adac0 legacy=0x7f4f180adef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:08.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.986+0000 7f4f1d339640 1 -- 172.21.15.76:0/1949562743 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1815a6c0 con 0x7f4f180c48e0 2023-11-09T03:47:08.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.987+0000 7f4f1d339640 1 --2- 172.21.15.76:0/1949562743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f180c48e0 0x7f4f180c4d60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4f080097a0 tx=0x7f4f0802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5bfbb21463155de server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:08.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.987+0000 7f4f1cb38640 1 -- 172.21.15.76:0/1949562743 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f08031830 con 0x7f4f180c48e0 2023-11-09T03:47:08.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.987+0000 7f4f1cb38640 1 -- 172.21.15.76:0/1949562743 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f08031990 con 0x7f4f180c48e0 2023-11-09T03:47:08.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.987+0000 7f4f1cb38640 1 -- 172.21.15.76:0/1949562743 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f08031d20 con 0x7f4f180c48e0 2023-11-09T03:47:08.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.988+0000 7f4f1fd9e640 1 -- 172.21.15.76:0/1949562743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f180c48e0 msgr2=0x7f4f180c4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:08.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.988+0000 7f4f1fd9e640 1 --2- 172.21.15.76:0/1949562743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f180c48e0 0x7f4f180c4d60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4f080097a0 tx=0x7f4f0802b8d0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.988+0000 7f4f1fd9e640 1 -- 172.21.15.76:0/1949562743 shutdown_connections 2023-11-09T03:47:08.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.988+0000 7f4f1fd9e640 1 --2- 172.21.15.76:0/1949562743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f180c48e0 0x7f4f180c4d60 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.988+0000 7f4f1fd9e640 1 --2- 172.21.15.76:0/1949562743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f18159c00 0x7f4f180cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.988+0000 7f4f1fd9e640 1 -- 172.21.15.76:0/1949562743 >> 172.21.15.76:0/1949562743 conn(0x7f4f180ce950 msgr2=0x7f4f180ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:08.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.988+0000 7f4f1fd9e640 1 -- 172.21.15.76:0/1949562743 shutdown_connections 2023-11-09T03:47:08.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.988+0000 7f4f1fd9e640 1 -- 172.21.15.76:0/1949562743 wait complete. 2023-11-09T03:47:08.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.989+0000 7f4f1fd9e640 1 Processor -- start 2023-11-09T03:47:08.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.989+0000 7f4f1fd9e640 1 -- start start 2023-11-09T03:47:08.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.990+0000 7f4f1fd9e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f180adac0 0x7f4f181f0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:08.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.990+0000 7f4f1fd9e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f180c48e0 0x7f4f1814c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:08.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.990+0000 7f4f1fd9e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f18159c00 0x7f4f18158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:08.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.990+0000 7f4f1fd9e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f181f13d0 con 0x7f4f18159c00 2023-11-09T03:47:08.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.990+0000 7f4f1fd9e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f181f3c70 con 0x7f4f180adac0 2023-11-09T03:47:08.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.990+0000 7f4f1fd9e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1814cb20 con 0x7f4f180c48e0 2023-11-09T03:47:08.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.990+0000 7f4f1e33b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f18159c00 0x7f4f18158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:08.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.990+0000 7f4f1e33b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f18159c00 0x7f4f18158770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40590/0 (socket says 172.21.15.76:40590) 2023-11-09T03:47:08.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.990+0000 7f4f1e33b640 1 -- 172.21.15.76:0/4263771341 learned_addr learned my addr 172.21.15.76:0/4263771341 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:08.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.990+0000 7f4f1d339640 1 --2- 172.21.15.76:0/4263771341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f180c48e0 0x7f4f1814c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:08.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.990+0000 7f4f1db3a640 1 --2- 172.21.15.76:0/4263771341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f180adac0 0x7f4f181f0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:08.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.990+0000 7f4f1e33b640 1 -- 172.21.15.76:0/4263771341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f180c48e0 msgr2=0x7f4f1814c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:08.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.990+0000 7f4f1e33b640 1 --2- 172.21.15.76:0/4263771341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f180c48e0 0x7f4f1814c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.990+0000 7f4f1e33b640 1 -- 172.21.15.76:0/4263771341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f180adac0 msgr2=0x7f4f181f0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:08.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.990+0000 7f4f1e33b640 1 --2- 172.21.15.76:0/4263771341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f180adac0 0x7f4f181f0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:08.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.990+0000 7f4f1e33b640 1 -- 172.21.15.76:0/4263771341 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f08009400 con 0x7f4f18159c00 2023-11-09T03:47:08.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.990+0000 7f4f1e33b640 1 --2- 172.21.15.76:0/4263771341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f18159c00 0x7f4f18158770 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4f10001eb0 tx=0x7f4f1000c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:08.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.990+0000 7f4f0e7fc640 1 -- 172.21.15.76:0/4263771341 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f100074e0 con 0x7f4f18159c00 2023-11-09T03:47:08.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.990+0000 7f4f0e7fc640 1 -- 172.21.15.76:0/4263771341 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f10007640 con 0x7f4f18159c00 2023-11-09T03:47:08.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.990+0000 7f4f1fd9e640 1 -- 172.21.15.76:0/4263771341 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f18158cb0 con 0x7f4f18159c00 2023-11-09T03:47:08.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.991+0000 7f4f0e7fc640 1 -- 172.21.15.76:0/4263771341 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f10004a00 con 0x7f4f18159c00 2023-11-09T03:47:08.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.991+0000 7f4f1fd9e640 1 -- 172.21.15.76:0/4263771341 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f181f0e10 con 0x7f4f18159c00 2023-11-09T03:47:09.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.991+0000 7f4f1fd9e640 1 -- 172.21.15.76:0/4263771341 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f181f3a40 con 0x7f4f18159c00 2023-11-09T03:47:09.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.992+0000 7f4f0e7fc640 1 -- 172.21.15.76:0/4263771341 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4f10004b60 con 0x7f4f18159c00 2023-11-09T03:47:09.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.992+0000 7f4f0e7fc640 1 --2- 172.21.15.76:0/4263771341 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4efc06ae70 0x7f4efc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:09.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.992+0000 7f4f0e7fc640 1 -- 172.21.15.76:0/4263771341 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4f10089700 con 0x7f4f18159c00 2023-11-09T03:47:09.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.992+0000 7f4f1db3a640 1 --2- 172.21.15.76:0/4263771341 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4efc06ae70 0x7f4efc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:09.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.993+0000 7f4f1db3a640 1 --2- 172.21.15.76:0/4263771341 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4efc06ae70 0x7f4efc06d330 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4f180c43e0 tx=0x7f4f14008460 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:09.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:08.996+0000 7f4f0e7fc640 1 -- 172.21.15.76:0/4263771341 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4f181f3a40 con 0x7f4f18159c00 2023-11-09T03:47:09.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:09.199+0000 7f4f1fd9e640 1 -- 172.21.15.76:0/4263771341 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4f181f3a40 con 0x7f4f18159c00 2023-11-09T03:47:09.870 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:09.870+0000 7f4f0e7fc640 1 -- 172.21.15.76:0/4263771341 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4f181f3a40 con 0x7f4f18159c00 2023-11-09T03:47:09.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:09.872+0000 7f4f1fd9e640 1 -- 172.21.15.76:0/4263771341 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4efc06ae70 msgr2=0x7f4efc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:09.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:09.872+0000 7f4f1fd9e640 1 --2- 172.21.15.76:0/4263771341 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4efc06ae70 0x7f4efc06d330 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4f180c43e0 tx=0x7f4f14008460 comp rx=0 tx=0).stop 2023-11-09T03:47:09.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:09.872+0000 7f4f1fd9e640 1 -- 172.21.15.76:0/4263771341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f18159c00 msgr2=0x7f4f18158770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:09.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:09.872+0000 7f4f1fd9e640 1 --2- 172.21.15.76:0/4263771341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f18159c00 0x7f4f18158770 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4f10001eb0 tx=0x7f4f1000c870 comp rx=0 tx=0).stop 2023-11-09T03:47:09.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:09.872+0000 7f4f1fd9e640 1 -- 172.21.15.76:0/4263771341 shutdown_connections 2023-11-09T03:47:09.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:09.872+0000 7f4f1fd9e640 1 --2- 172.21.15.76:0/4263771341 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4efc06ae70 0x7f4efc06d330 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:09.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:09.872+0000 7f4f1fd9e640 1 --2- 172.21.15.76:0/4263771341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f18159c00 0x7f4f18158770 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:09.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:09.872+0000 7f4f1fd9e640 1 --2- 172.21.15.76:0/4263771341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f180c48e0 0x7f4f1814c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:09.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:09.872+0000 7f4f1fd9e640 1 --2- 172.21.15.76:0/4263771341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f180adac0 0x7f4f181f0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:09.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:09.872+0000 7f4f1fd9e640 1 -- 172.21.15.76:0/4263771341 >> 172.21.15.76:0/4263771341 conn(0x7f4f180ce950 msgr2=0x7f4f180c7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:09.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:09.872+0000 7f4f1fd9e640 1 -- 172.21.15.76:0/4263771341 shutdown_connections 2023-11-09T03:47:09.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:09.872+0000 7f4f1fd9e640 1 -- 172.21.15.76:0/4263771341 wait complete. 2023-11-09T03:47:09.885 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-09T03:47:10.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.032+0000 7f46b14af640 1 Processor -- start 2023-11-09T03:47:10.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.032+0000 7f46b14af640 1 -- start start 2023-11-09T03:47:10.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.033+0000 7f46b14af640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46ac0cd9f0 0x7f46ac0cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:10.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.033+0000 7f46b14af640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f46ac15d4b0 0x7f46ac15d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:10.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.033+0000 7f46b14af640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46ac0d1320 con 0x7f46ac0b47e0 2023-11-09T03:47:10.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.033+0000 7f46b14af640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46ac15e4d0 con 0x7f46ac15d4b0 2023-11-09T03:47:10.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.033+0000 7f46b14af640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac0ce310 con 0x7f46ac0cd9f0 2023-11-09T03:47:10.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.033+0000 7f46aaffd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46ac0cd9f0 0x7f46ac0cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:10.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.034+0000 7f46aaffd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46ac0cd9f0 0x7f46ac0cddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:34620/0 (socket says 172.21.15.76:34620) 2023-11-09T03:47:10.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.034+0000 7f46aaffd640 1 -- 172.21.15.76:0/802215706 learned_addr learned my addr 172.21.15.76:0/802215706 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:10.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.034+0000 7f46aa7fc640 1 --2- 172.21.15.76:0/802215706 >> v2:172.21.15.113:3300/0 conn(0x7f46ac15d4b0 0x7f46ac15d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:10.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.034+0000 7f46aaffd640 1 -- 172.21.15.76:0/802215706 >> v2:172.21.15.113:3300/0 conn(0x7f46ac15d4b0 msgr2=0x7f46ac15d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:10.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.034+0000 7f46aaffd640 1 --2- 172.21.15.76:0/802215706 >> v2:172.21.15.113:3300/0 conn(0x7f46ac15d4b0 0x7f46ac15d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:10.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.034+0000 7f46aaffd640 1 -- 172.21.15.76:0/802215706 >> v1:172.21.15.113:6789/0 conn(0x7f46ac0b47e0 legacy=0x7f46ac0b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:10.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.034+0000 7f46aaffd640 1 -- 172.21.15.76:0/802215706 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ac15df10 con 0x7f46ac0cd9f0 2023-11-09T03:47:10.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.034+0000 7f46aa7fc640 1 --2- 172.21.15.76:0/802215706 >> v2:172.21.15.113:3300/0 conn(0x7f46ac15d4b0 0x7f46ac15d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:47:10.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.035+0000 7f46aaffd640 1 --2- 172.21.15.76:0/802215706 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46ac0cd9f0 0x7f46ac0cddd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f469c009e50 tx=0x7f469c02bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f8f9af53bcedde7 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:10.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.035+0000 7f46a9ffb640 1 -- 172.21.15.76:0/802215706 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c02cb90 con 0x7f46ac0cd9f0 2023-11-09T03:47:10.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.035+0000 7f46a9ffb640 1 -- 172.21.15.76:0/802215706 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f469c02ccf0 con 0x7f46ac0cd9f0 2023-11-09T03:47:10.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.035+0000 7f46a9ffb640 1 -- 172.21.15.76:0/802215706 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c034610 con 0x7f46ac0cd9f0 2023-11-09T03:47:10.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.035+0000 7f46b14af640 1 -- 172.21.15.76:0/802215706 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46ac0cd9f0 msgr2=0x7f46ac0cddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:10.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.035+0000 7f46b14af640 1 --2- 172.21.15.76:0/802215706 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46ac0cd9f0 0x7f46ac0cddd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f469c009e50 tx=0x7f469c02bff0 comp rx=0 tx=0).stop 2023-11-09T03:47:10.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.035+0000 7f46b14af640 1 -- 172.21.15.76:0/802215706 shutdown_connections 2023-11-09T03:47:10.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.035+0000 7f46b14af640 1 --2- 172.21.15.76:0/802215706 >> v2:172.21.15.113:3300/0 conn(0x7f46ac15d4b0 0x7f46ac15d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:10.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.035+0000 7f46b14af640 1 --2- 172.21.15.76:0/802215706 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46ac0cd9f0 0x7f46ac0cddd0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:10.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.035+0000 7f46b14af640 1 -- 172.21.15.76:0/802215706 >> 172.21.15.76:0/802215706 conn(0x7f46ac071c40 msgr2=0x7f46ac072050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:10.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.035+0000 7f46b14af640 1 -- 172.21.15.76:0/802215706 shutdown_connections 2023-11-09T03:47:10.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.035+0000 7f46b14af640 1 -- 172.21.15.76:0/802215706 wait complete. 2023-11-09T03:47:10.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.037+0000 7f46b14af640 1 Processor -- start 2023-11-09T03:47:10.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.037+0000 7f46b14af640 1 -- start start 2023-11-09T03:47:10.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.037+0000 7f46b14af640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46ac0b47e0 0x7f46ac0c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:10.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.037+0000 7f46b14af640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46ac0cd9f0 0x7f46ac157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:10.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.037+0000 7f46b14af640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46ac15d4b0 0x7f46ac1580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:10.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.037+0000 7f46b14af640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac0ca1d0 con 0x7f46ac0cd9f0 2023-11-09T03:47:10.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.037+0000 7f46b14af640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac0ca4b0 con 0x7f46ac0b47e0 2023-11-09T03:47:10.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.037+0000 7f46b14af640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac0ca790 con 0x7f46ac15d4b0 2023-11-09T03:47:10.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.038+0000 7f46aa7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46ac0cd9f0 0x7f46ac157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:10.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.038+0000 7f46aa7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46ac0cd9f0 0x7f46ac157b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40602/0 (socket says 172.21.15.76:40602) 2023-11-09T03:47:10.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.038+0000 7f46aa7fc640 1 -- 172.21.15.76:0/957762307 learned_addr learned my addr 172.21.15.76:0/957762307 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:10.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.038+0000 7f46ab7fe640 1 --2- 172.21.15.76:0/957762307 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46ac15d4b0 0x7f46ac1580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:10.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.038+0000 7f46aaffd640 1 --2- 172.21.15.76:0/957762307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46ac0b47e0 0x7f46ac0c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:10.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.038+0000 7f46aa7fc640 1 -- 172.21.15.76:0/957762307 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46ac15d4b0 msgr2=0x7f46ac1580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:10.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.038+0000 7f46aa7fc640 1 --2- 172.21.15.76:0/957762307 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46ac15d4b0 0x7f46ac1580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:10.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.038+0000 7f46aa7fc640 1 -- 172.21.15.76:0/957762307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46ac0b47e0 msgr2=0x7f46ac0c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:10.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.038+0000 7f46aa7fc640 1 --2- 172.21.15.76:0/957762307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46ac0b47e0 0x7f46ac0c8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:10.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.038+0000 7f46aa7fc640 1 -- 172.21.15.76:0/957762307 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4694009400 con 0x7f46ac0cd9f0 2023-11-09T03:47:10.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.038+0000 7f46aa7fc640 1 --2- 172.21.15.76:0/957762307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46ac0cd9f0 0x7f46ac157b80 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f469400e960 tx=0x7f469400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:10.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.038+0000 7f46a37fe640 1 -- 172.21.15.76:0/957762307 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469400ca00 con 0x7f46ac0cd9f0 2023-11-09T03:47:10.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.038+0000 7f46a37fe640 1 -- 172.21.15.76:0/957762307 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f469400cb60 con 0x7f46ac0cd9f0 2023-11-09T03:47:10.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.038+0000 7f46a37fe640 1 -- 172.21.15.76:0/957762307 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4694013470 con 0x7f46ac0cd9f0 2023-11-09T03:47:10.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.038+0000 7f46b14af640 1 -- 172.21.15.76:0/957762307 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f469c009b30 con 0x7f46ac0cd9f0 2023-11-09T03:47:10.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.038+0000 7f46b14af640 1 -- 172.21.15.76:0/957762307 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46ac0cad50 con 0x7f46ac0cd9f0 2023-11-09T03:47:10.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.039+0000 7f46b14af640 1 -- 172.21.15.76:0/957762307 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46ac0b73b0 con 0x7f46ac0cd9f0 2023-11-09T03:47:10.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.040+0000 7f46a37fe640 1 -- 172.21.15.76:0/957762307 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f46940135d0 con 0x7f46ac0cd9f0 2023-11-09T03:47:10.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.040+0000 7f46a37fe640 1 --2- 172.21.15.76:0/957762307 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f468c06ae70 0x7f468c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:10.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.040+0000 7f46a37fe640 1 -- 172.21.15.76:0/957762307 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4694089b10 con 0x7f46ac0cd9f0 2023-11-09T03:47:10.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.040+0000 7f46aaffd640 1 --2- 172.21.15.76:0/957762307 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f468c06ae70 0x7f468c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:10.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.041+0000 7f46aaffd640 1 --2- 172.21.15.76:0/957762307 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f468c06ae70 0x7f468c06d330 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f469c035000 tx=0x7f469c030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:10.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.044+0000 7f46a37fe640 1 -- 172.21.15.76:0/957762307 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4694054960 con 0x7f46ac0cd9f0 2023-11-09T03:47:10.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.212+0000 7f46b14af640 1 -- 172.21.15.76:0/957762307 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f46ac04e490 con 0x7f46ac0cd9f0 2023-11-09T03:47:10.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.871+0000 7f46a37fe640 1 -- 172.21.15.76:0/957762307 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v44) v1 ==== 181+0+0 (secure 0 0 0) 0x7f46940170d0 con 0x7f46ac0cd9f0 2023-11-09T03:47:10.872 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' removed 2023-11-09T03:47:10.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.875+0000 7f46b14af640 1 -- 172.21.15.76:0/957762307 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f468c06ae70 msgr2=0x7f468c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:10.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.875+0000 7f46b14af640 1 --2- 172.21.15.76:0/957762307 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f468c06ae70 0x7f468c06d330 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f469c035000 tx=0x7f469c030000 comp rx=0 tx=0).stop 2023-11-09T03:47:10.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.875+0000 7f46b14af640 1 -- 172.21.15.76:0/957762307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46ac0cd9f0 msgr2=0x7f46ac157b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:10.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.875+0000 7f46b14af640 1 --2- 172.21.15.76:0/957762307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46ac0cd9f0 0x7f46ac157b80 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f469400e960 tx=0x7f469400ee20 comp rx=0 tx=0).stop 2023-11-09T03:47:10.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.875+0000 7f46b14af640 1 -- 172.21.15.76:0/957762307 shutdown_connections 2023-11-09T03:47:10.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.875+0000 7f46b14af640 1 --2- 172.21.15.76:0/957762307 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f468c06ae70 0x7f468c06d330 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:10.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.875+0000 7f46b14af640 1 --2- 172.21.15.76:0/957762307 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46ac15d4b0 0x7f46ac1580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:10.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.875+0000 7f46b14af640 1 --2- 172.21.15.76:0/957762307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46ac0cd9f0 0x7f46ac157b80 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:10.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.875+0000 7f46b14af640 1 --2- 172.21.15.76:0/957762307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46ac0b47e0 0x7f46ac0c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:10.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.875+0000 7f46b14af640 1 -- 172.21.15.76:0/957762307 >> 172.21.15.76:0/957762307 conn(0x7f46ac071c40 msgr2=0x7f46ac0cf770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:10.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.875+0000 7f46b14af640 1 -- 172.21.15.76:0/957762307 shutdown_connections 2023-11-09T03:47:10.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.875+0000 7f46b14af640 1 -- 172.21.15.76:0/957762307 wait complete. 2023-11-09T03:47:10.889 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-09T03:47:10.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.999+0000 7fed8cc90640 1 Processor -- start 2023-11-09T03:47:10.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:10.999+0000 7fed8cc90640 1 -- start start 2023-11-09T03:47:11.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.000+0000 7fed8cc90640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fed880cb270 0x7fed880b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:11.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.000+0000 7fed8cc90640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fed8806a830 0x7fed880ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:11.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.000+0000 7fed8cc90640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed880cdc30 con 0x7fed880cb650 2023-11-09T03:47:11.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.000+0000 7fed8cc90640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed880caa00 con 0x7fed880cb270 2023-11-09T03:47:11.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.000+0000 7fed8cc90640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed880cd620 con 0x7fed8806a830 2023-11-09T03:47:11.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.000+0000 7fed8659c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fed880cb270 0x7fed880b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:11.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.000+0000 7fed85d9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fed880cb650 0x7fed8806a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37484/0 (socket says 172.21.15.76:37484) 2023-11-09T03:47:11.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.000+0000 7fed85d9b640 1 -- 172.21.15.76:0/1212723878 learned_addr learned my addr 172.21.15.76:0/1212723878 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:11.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.000+0000 7fed8659c640 1 -- 172.21.15.76:0/1212723878 >> v2:172.21.15.113:3300/0 conn(0x7fed8806a830 msgr2=0x7fed880ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:11.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.000+0000 7fed8659c640 1 --2- 172.21.15.76:0/1212723878 >> v2:172.21.15.113:3300/0 conn(0x7fed8806a830 0x7fed880ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:11.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.000+0000 7fed8659c640 1 -- 172.21.15.76:0/1212723878 >> v1:172.21.15.76:6789/0 conn(0x7fed880cb650 legacy=0x7fed8806a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:11.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.000+0000 7fed8659c640 1 -- 172.21.15.76:0/1212723878 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8806fe00 con 0x7fed880cb270 2023-11-09T03:47:11.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.000+0000 7fed8559a640 1 -- 172.21.15.76:0/1212723878 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 521170304 0 0) 0x7fed880cdc30 con 0x7fed880cb650 2023-11-09T03:47:11.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.001+0000 7fed8659c640 1 --2- 172.21.15.76:0/1212723878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed880cb270 0x7fed880b1c20 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fed88067520 tx=0x7fed8002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65009bf581fd6550 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:11.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.001+0000 7fed8559a640 1 -- 172.21.15.76:0/1212723878 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8002cc80 con 0x7fed880cb270 2023-11-09T03:47:11.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.001+0000 7fed8559a640 1 -- 172.21.15.76:0/1212723878 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8002cde0 con 0x7fed880cb270 2023-11-09T03:47:11.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.001+0000 7fed8559a640 1 -- 172.21.15.76:0/1212723878 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed80034660 con 0x7fed880cb270 2023-11-09T03:47:11.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.001+0000 7fed8cc90640 1 -- 172.21.15.76:0/1212723878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed880cb270 msgr2=0x7fed880b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:11.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.001+0000 7fed8cc90640 1 --2- 172.21.15.76:0/1212723878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed880cb270 0x7fed880b1c20 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fed88067520 tx=0x7fed8002bed0 comp rx=0 tx=0).stop 2023-11-09T03:47:11.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.001+0000 7fed8cc90640 1 -- 172.21.15.76:0/1212723878 shutdown_connections 2023-11-09T03:47:11.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.001+0000 7fed8cc90640 1 --2- 172.21.15.76:0/1212723878 >> v2:172.21.15.113:3300/0 conn(0x7fed8806a830 0x7fed880ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:11.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.001+0000 7fed8cc90640 1 --2- 172.21.15.76:0/1212723878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed880cb270 0x7fed880b1c20 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:11.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.001+0000 7fed8cc90640 1 -- 172.21.15.76:0/1212723878 >> 172.21.15.76:0/1212723878 conn(0x7fed8806d820 msgr2=0x7fed8806dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:11.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.002+0000 7fed8cc90640 1 -- 172.21.15.76:0/1212723878 shutdown_connections 2023-11-09T03:47:11.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.002+0000 7fed8cc90640 1 -- 172.21.15.76:0/1212723878 wait complete. 2023-11-09T03:47:11.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.003+0000 7fed8cc90640 1 Processor -- start 2023-11-09T03:47:11.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.003+0000 7fed8cc90640 1 -- start start 2023-11-09T03:47:11.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.004+0000 7fed8cc90640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed8806a830 0x7fed880c6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:11.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.004+0000 7fed8cc90640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed880cb270 0x7fed881f2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:11.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.004+0000 7fed8cc90640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed880cb650 0x7fed881f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:11.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.004+0000 7fed8cc90640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed880c8490 con 0x7fed8806a830 2023-11-09T03:47:11.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.004+0000 7fed8cc90640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed880c7810 con 0x7fed880cb270 2023-11-09T03:47:11.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.004+0000 7fed8cc90640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed880c7af0 con 0x7fed880cb650 2023-11-09T03:47:11.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.004+0000 7fed8659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed8806a830 0x7fed880c6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:11.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.004+0000 7fed8659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed8806a830 0x7fed880c6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40622/0 (socket says 172.21.15.76:40622) 2023-11-09T03:47:11.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.004+0000 7fed8659c640 1 -- 172.21.15.76:0/3799655068 learned_addr learned my addr 172.21.15.76:0/3799655068 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:11.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.004+0000 7fed85d9b640 1 --2- 172.21.15.76:0/3799655068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed880cb270 0x7fed881f2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:11.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.004+0000 7fed86d9d640 1 --2- 172.21.15.76:0/3799655068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed880cb650 0x7fed881f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:11.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.004+0000 7fed8659c640 1 -- 172.21.15.76:0/3799655068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed880cb650 msgr2=0x7fed881f2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:11.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.004+0000 7fed8659c640 1 --2- 172.21.15.76:0/3799655068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed880cb650 0x7fed881f2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:11.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.004+0000 7fed8659c640 1 -- 172.21.15.76:0/3799655068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed880cb270 msgr2=0x7fed881f2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:11.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.004+0000 7fed8659c640 1 --2- 172.21.15.76:0/3799655068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed880cb270 0x7fed881f2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:11.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.004+0000 7fed8659c640 1 -- 172.21.15.76:0/3799655068 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed80009a70 con 0x7fed8806a830 2023-11-09T03:47:11.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.004+0000 7fed8659c640 1 --2- 172.21.15.76:0/3799655068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed8806a830 0x7fed880c6d10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fed8002c4e0 tx=0x7fed80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:11.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.004+0000 7fed76ffd640 1 -- 172.21.15.76:0/3799655068 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed80003b90 con 0x7fed8806a830 2023-11-09T03:47:11.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.004+0000 7fed76ffd640 1 -- 172.21.15.76:0/3799655068 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed80003cf0 con 0x7fed8806a830 2023-11-09T03:47:11.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.004+0000 7fed8cc90640 1 -- 172.21.15.76:0/3799655068 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed880c7dd0 con 0x7fed8806a830 2023-11-09T03:47:11.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.004+0000 7fed76ffd640 1 -- 172.21.15.76:0/3799655068 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed80004270 con 0x7fed8806a830 2023-11-09T03:47:11.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.004+0000 7fed8cc90640 1 -- 172.21.15.76:0/3799655068 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed880c9270 con 0x7fed8806a830 2023-11-09T03:47:11.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.005+0000 7fed8cc90640 1 -- 172.21.15.76:0/3799655068 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed8804e490 con 0x7fed8806a830 2023-11-09T03:47:11.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.006+0000 7fed76ffd640 1 -- 172.21.15.76:0/3799655068 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fed800043d0 con 0x7fed8806a830 2023-11-09T03:47:11.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.006+0000 7fed76ffd640 1 --2- 172.21.15.76:0/3799655068 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fed6806ae70 0x7fed6806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:11.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.006+0000 7fed76ffd640 1 -- 172.21.15.76:0/3799655068 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fed800ac690 con 0x7fed8806a830 2023-11-09T03:47:11.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.006+0000 7fed85d9b640 1 --2- 172.21.15.76:0/3799655068 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fed6806ae70 0x7fed6806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:11.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.007+0000 7fed85d9b640 1 --2- 172.21.15.76:0/3799655068 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fed6806ae70 0x7fed6806d330 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fed881f0420 tx=0x7fed7c007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:11.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.010+0000 7fed76ffd640 1 -- 172.21.15.76:0/3799655068 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fed80077670 con 0x7fed8806a830 2023-11-09T03:47:11.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.175+0000 7fed8cc90640 1 -- 172.21.15.76:0/3799655068 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fed880b59d0 con 0x7fed8806a830 2023-11-09T03:47:11.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.400+0000 7fed76ffd640 1 -- 172.21.15.76:0/3799655068 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v45) v1 ==== 169+0+0 (secure 0 0 0) 0x7fed8007ac90 con 0x7fed8806a830 2023-11-09T03:47:11.401 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' removed 2023-11-09T03:47:11.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.403+0000 7fed8cc90640 1 -- 172.21.15.76:0/3799655068 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fed6806ae70 msgr2=0x7fed6806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:11.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.403+0000 7fed8cc90640 1 --2- 172.21.15.76:0/3799655068 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fed6806ae70 0x7fed6806d330 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fed881f0420 tx=0x7fed7c007c00 comp rx=0 tx=0).stop 2023-11-09T03:47:11.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.403+0000 7fed8cc90640 1 -- 172.21.15.76:0/3799655068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed8806a830 msgr2=0x7fed880c6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:11.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.403+0000 7fed8cc90640 1 --2- 172.21.15.76:0/3799655068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed8806a830 0x7fed880c6d10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fed8002c4e0 tx=0x7fed80004a40 comp rx=0 tx=0).stop 2023-11-09T03:47:11.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.403+0000 7fed8cc90640 1 -- 172.21.15.76:0/3799655068 shutdown_connections 2023-11-09T03:47:11.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.403+0000 7fed8cc90640 1 --2- 172.21.15.76:0/3799655068 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fed6806ae70 0x7fed6806d330 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:11.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.403+0000 7fed8cc90640 1 --2- 172.21.15.76:0/3799655068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed880cb650 0x7fed881f2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:11.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.403+0000 7fed8cc90640 1 --2- 172.21.15.76:0/3799655068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed880cb270 0x7fed881f2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:11.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.403+0000 7fed8cc90640 1 --2- 172.21.15.76:0/3799655068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed8806a830 0x7fed880c6d10 secure :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fed8002c4e0 tx=0x7fed80004a40 comp rx=0 tx=0).stop 2023-11-09T03:47:11.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.403+0000 7fed8cc90640 1 -- 172.21.15.76:0/3799655068 >> 172.21.15.76:0/3799655068 conn(0x7fed8806d820 msgr2=0x7fed880cf980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:11.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.404+0000 7fed8cc90640 1 -- 172.21.15.76:0/3799655068 shutdown_connections 2023-11-09T03:47:11.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.404+0000 7fed8cc90640 1 -- 172.21.15.76:0/3799655068 wait complete. 2023-11-09T03:47:11.417 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-09T03:47:11.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.521+0000 7fe2abba5640 1 Processor -- start 2023-11-09T03:47:11.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.521+0000 7fe2abba5640 1 -- start start 2023-11-09T03:47:11.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.522+0000 7fe2abba5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe2a4153850 0x7fe2a415a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:11.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.522+0000 7fe2abba5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe2a40cc520 0x7fe2a4070fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:11.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.522+0000 7fe2abba5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2a40c3860 con 0x7fe2a40cc9d0 2023-11-09T03:47:11.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.522+0000 7fe2abba5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2a40c9250 con 0x7fe2a40cc520 2023-11-09T03:47:11.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.522+0000 7fe2abba5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2a4153c30 con 0x7fe2a4153850 2023-11-09T03:47:11.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.522+0000 7fe2a9140640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe2a40cc520 0x7fe2a4070fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:11.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.522+0000 7fe2a9140640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe2a40cc520 0x7fe2a4070fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40638/0 (socket says 172.21.15.76:40638) 2023-11-09T03:47:11.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.522+0000 7fe2a9140640 1 -- 172.21.15.76:0/2010887320 learned_addr learned my addr 172.21.15.76:0/2010887320 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:11.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.522+0000 7fe2a9941640 1 --2- 172.21.15.76:0/2010887320 >> v2:172.21.15.113:3300/0 conn(0x7fe2a4153850 0x7fe2a415a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:11.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.522+0000 7fe2a9140640 1 -- 172.21.15.76:0/2010887320 >> v2:172.21.15.113:3300/0 conn(0x7fe2a4153850 msgr2=0x7fe2a415a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:11.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.522+0000 7fe2a9140640 1 --2- 172.21.15.76:0/2010887320 >> v2:172.21.15.113:3300/0 conn(0x7fe2a4153850 0x7fe2a415a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:11.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.522+0000 7fe2a9140640 1 -- 172.21.15.76:0/2010887320 >> v1:172.21.15.113:6789/0 conn(0x7fe2a40cc9d0 legacy=0x7fe2a40c3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:11.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.522+0000 7fe2a9140640 1 -- 172.21.15.76:0/2010887320 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a415a700 con 0x7fe2a40cc520 2023-11-09T03:47:11.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.523+0000 7fe2a9140640 1 --2- 172.21.15.76:0/2010887320 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2a40cc520 0x7fe2a4070fa0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe2a40c5710 tx=0x7fe29402cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42e5505aaecb0d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:11.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.523+0000 7fe2a893f640 1 -- 172.21.15.76:0/2010887320 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29402dba0 con 0x7fe2a40cc520 2023-11-09T03:47:11.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.523+0000 7fe2a893f640 1 -- 172.21.15.76:0/2010887320 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29402dd00 con 0x7fe2a40cc520 2023-11-09T03:47:11.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.523+0000 7fe2a893f640 1 -- 172.21.15.76:0/2010887320 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe294035660 con 0x7fe2a40cc520 2023-11-09T03:47:11.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.523+0000 7fe2abba5640 1 -- 172.21.15.76:0/2010887320 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2a40cc520 msgr2=0x7fe2a4070fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:11.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.523+0000 7fe2abba5640 1 --2- 172.21.15.76:0/2010887320 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2a40cc520 0x7fe2a4070fa0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe2a40c5710 tx=0x7fe29402cdf0 comp rx=0 tx=0).stop 2023-11-09T03:47:11.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.524+0000 7fe2abba5640 1 -- 172.21.15.76:0/2010887320 shutdown_connections 2023-11-09T03:47:11.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.524+0000 7fe2abba5640 1 --2- 172.21.15.76:0/2010887320 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2a40cc520 0x7fe2a4070fa0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:11.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.524+0000 7fe2abba5640 1 --2- 172.21.15.76:0/2010887320 >> v2:172.21.15.113:3300/0 conn(0x7fe2a4153850 0x7fe2a415a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:11.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.524+0000 7fe2abba5640 1 -- 172.21.15.76:0/2010887320 >> 172.21.15.76:0/2010887320 conn(0x7fe2a40ce950 msgr2=0x7fe2a40ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:11.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.524+0000 7fe2abba5640 1 -- 172.21.15.76:0/2010887320 shutdown_connections 2023-11-09T03:47:11.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.524+0000 7fe2abba5640 1 -- 172.21.15.76:0/2010887320 wait complete. 2023-11-09T03:47:11.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.525+0000 7fe2abba5640 1 Processor -- start 2023-11-09T03:47:11.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.525+0000 7fe2abba5640 1 -- start start 2023-11-09T03:47:11.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.525+0000 7fe2abba5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2a40cc520 0x7fe2a40c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:11.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.525+0000 7fe2abba5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2a40cc9d0 0x7fe2a414bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:11.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.525+0000 7fe2abba5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2a4153850 0x7fe2a4158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:11.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.525+0000 7fe2abba5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a40c83f0 con 0x7fe2a40cc9d0 2023-11-09T03:47:11.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.525+0000 7fe2abba5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a40c77e0 con 0x7fe2a40cc520 2023-11-09T03:47:11.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.525+0000 7fe2abba5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a40c8810 con 0x7fe2a4153850 2023-11-09T03:47:11.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.526+0000 7fe2a9140640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2a40cc9d0 0x7fe2a414bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:11.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.526+0000 7fe2a9140640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2a40cc9d0 0x7fe2a414bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40650/0 (socket says 172.21.15.76:40650) 2023-11-09T03:47:11.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.526+0000 7fe2a9140640 1 -- 172.21.15.76:0/2645811821 learned_addr learned my addr 172.21.15.76:0/2645811821 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:11.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.526+0000 7fe2a9941640 1 --2- 172.21.15.76:0/2645811821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2a40cc520 0x7fe2a40c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:11.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.526+0000 7fe2aa142640 1 --2- 172.21.15.76:0/2645811821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2a4153850 0x7fe2a4158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:11.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.526+0000 7fe2a9140640 1 -- 172.21.15.76:0/2645811821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2a4153850 msgr2=0x7fe2a4158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:11.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.526+0000 7fe2a9140640 1 --2- 172.21.15.76:0/2645811821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2a4153850 0x7fe2a4158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:11.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.526+0000 7fe2a9140640 1 -- 172.21.15.76:0/2645811821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2a40cc520 msgr2=0x7fe2a40c6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:11.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.526+0000 7fe2a9140640 1 --2- 172.21.15.76:0/2645811821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2a40cc520 0x7fe2a40c6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:11.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.526+0000 7fe2a9140640 1 -- 172.21.15.76:0/2645811821 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29400a670 con 0x7fe2a40cc9d0 2023-11-09T03:47:11.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.526+0000 7fe2a9140640 1 --2- 172.21.15.76:0/2645811821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2a40cc9d0 0x7fe2a414bd90 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fe29400a1c0 tx=0x7fe2940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:11.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.526+0000 7fe29a7fc640 1 -- 172.21.15.76:0/2645811821 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe294044070 con 0x7fe2a40cc9d0 2023-11-09T03:47:11.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.526+0000 7fe29a7fc640 1 -- 172.21.15.76:0/2645811821 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe294040480 con 0x7fe2a40cc9d0 2023-11-09T03:47:11.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.526+0000 7fe2abba5640 1 -- 172.21.15.76:0/2645811821 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2a40c7a50 con 0x7fe2a40cc9d0 2023-11-09T03:47:11.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.526+0000 7fe29a7fc640 1 -- 172.21.15.76:0/2645811821 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe294040830 con 0x7fe2a40cc9d0 2023-11-09T03:47:11.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.526+0000 7fe2abba5640 1 -- 172.21.15.76:0/2645811821 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2a40c8010 con 0x7fe2a40cc9d0 2023-11-09T03:47:11.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.527+0000 7fe2abba5640 1 -- 172.21.15.76:0/2645811821 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2a4074290 con 0x7fe2a40cc9d0 2023-11-09T03:47:11.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.528+0000 7fe29a7fc640 1 -- 172.21.15.76:0/2645811821 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe294040990 con 0x7fe2a40cc9d0 2023-11-09T03:47:11.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.528+0000 7fe29a7fc640 1 --2- 172.21.15.76:0/2645811821 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe28806ae70 0x7fe28806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:11.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.528+0000 7fe29a7fc640 1 -- 172.21.15.76:0/2645811821 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5644+0+0 (secure 0 0 0) 0x7fe2940ab670 con 0x7fe2a40cc9d0 2023-11-09T03:47:11.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.528+0000 7fe2a9941640 1 --2- 172.21.15.76:0/2645811821 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe28806ae70 0x7fe28806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:11.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.529+0000 7fe2a9941640 1 --2- 172.21.15.76:0/2645811821 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe28806ae70 0x7fe28806d330 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe2a00079a0 tx=0x7fe2a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:11.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.532+0000 7fe29a7fc640 1 -- 172.21.15.76:0/2645811821 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe294076760 con 0x7fe2a40cc9d0 2023-11-09T03:47:11.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.706+0000 7fe2abba5640 1 -- 172.21.15.76:0/2645811821 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe2a4074290 con 0x7fe2a40cc9d0 2023-11-09T03:47:11.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.709+0000 7fe29a7fc640 1 -- 172.21.15.76:0/2645811821 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v12) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe294079d80 con 0x7fe2a40cc9d0 2023-11-09T03:47:11.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.711+0000 7fe2abba5640 1 -- 172.21.15.76:0/2645811821 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe28806ae70 msgr2=0x7fe28806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:11.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.711+0000 7fe2abba5640 1 --2- 172.21.15.76:0/2645811821 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe28806ae70 0x7fe28806d330 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe2a00079a0 tx=0x7fe2a0008040 comp rx=0 tx=0).stop 2023-11-09T03:47:11.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.711+0000 7fe2abba5640 1 -- 172.21.15.76:0/2645811821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2a40cc9d0 msgr2=0x7fe2a414bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:11.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.711+0000 7fe2abba5640 1 --2- 172.21.15.76:0/2645811821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2a40cc9d0 0x7fe2a414bd90 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fe29400a1c0 tx=0x7fe2940046c0 comp rx=0 tx=0).stop 2023-11-09T03:47:11.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.711+0000 7fe2abba5640 1 -- 172.21.15.76:0/2645811821 shutdown_connections 2023-11-09T03:47:11.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.711+0000 7fe2abba5640 1 --2- 172.21.15.76:0/2645811821 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe28806ae70 0x7fe28806d330 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:11.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.711+0000 7fe2abba5640 1 --2- 172.21.15.76:0/2645811821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2a4153850 0x7fe2a4158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:11.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.711+0000 7fe2abba5640 1 --2- 172.21.15.76:0/2645811821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2a40cc9d0 0x7fe2a414bd90 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:11.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.711+0000 7fe2abba5640 1 --2- 172.21.15.76:0/2645811821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2a40cc520 0x7fe2a40c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:11.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.711+0000 7fe2abba5640 1 -- 172.21.15.76:0/2645811821 >> 172.21.15.76:0/2645811821 conn(0x7fe2a40ce950 msgr2=0x7fe2a4152830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:11.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.711+0000 7fe2abba5640 1 -- 172.21.15.76:0/2645811821 shutdown_connections 2023-11-09T03:47:11.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.711+0000 7fe2abba5640 1 -- 172.21.15.76:0/2645811821 wait complete. 2023-11-09T03:47:11.722 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_attaching_pools_with_snaps_to_fs_fails' 2023-11-09T03:47:11.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.823+0000 7fafbf0a9640 1 Processor -- start 2023-11-09T03:47:11.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.823+0000 7fafbf0a9640 1 -- start start 2023-11-09T03:47:11.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.824+0000 7fafbf0a9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fafb8159c70 0x7fafb80ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:11.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.824+0000 7fafbf0a9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fafb806db20 0x7fafb80b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:11.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.824+0000 7fafbf0a9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafb80b4110 con 0x7fafb80b0910 2023-11-09T03:47:11.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.824+0000 7fafbf0a9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafb80b49b0 con 0x7fafb806db20 2023-11-09T03:47:11.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.824+0000 7fafbf0a9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafb80b0da0 con 0x7fafb8159c70 2023-11-09T03:47:11.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.824+0000 7fafb7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fafb806db20 0x7fafb80b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:11.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.824+0000 7fafb7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fafb806db20 0x7fafb80b03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40654/0 (socket says 172.21.15.76:40654) 2023-11-09T03:47:11.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.824+0000 7fafb7fff640 1 -- 172.21.15.76:0/3938231095 learned_addr learned my addr 172.21.15.76:0/3938231095 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:11.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.824+0000 7fafbce45640 1 --2- 172.21.15.76:0/3938231095 >> v2:172.21.15.113:3300/0 conn(0x7fafb8159c70 0x7fafb80ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:11.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.824+0000 7fafb7fff640 1 -- 172.21.15.76:0/3938231095 >> v2:172.21.15.113:3300/0 conn(0x7fafb8159c70 msgr2=0x7fafb80ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:11.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.824+0000 7fafb7fff640 1 --2- 172.21.15.76:0/3938231095 >> v2:172.21.15.113:3300/0 conn(0x7fafb8159c70 0x7fafb80ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:11.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.824+0000 7fafb7fff640 1 -- 172.21.15.76:0/3938231095 >> v1:172.21.15.113:6789/0 conn(0x7fafb80b0910 legacy=0x7fafb815d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:11.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.824+0000 7fafb7fff640 1 -- 172.21.15.76:0/3938231095 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb80b43f0 con 0x7fafb806db20 2023-11-09T03:47:11.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.825+0000 7fafb7fff640 1 --2- 172.21.15.76:0/3938231095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafb806db20 0x7fafb80b03d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fafb80cc440 tx=0x7fafa402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2052a4f5a47dd server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:11.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.825+0000 7fafb77fe640 1 -- 172.21.15.76:0/3938231095 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa4030860 con 0x7fafb806db20 2023-11-09T03:47:11.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.825+0000 7fafb77fe640 1 -- 172.21.15.76:0/3938231095 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa40309c0 con 0x7fafb806db20 2023-11-09T03:47:11.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.825+0000 7fafb77fe640 1 -- 172.21.15.76:0/3938231095 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa4030d50 con 0x7fafb806db20 2023-11-09T03:47:11.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.826+0000 7fafbf0a9640 1 -- 172.21.15.76:0/3938231095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafb806db20 msgr2=0x7fafb80b03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:11.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.826+0000 7fafbf0a9640 1 --2- 172.21.15.76:0/3938231095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafb806db20 0x7fafb80b03d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fafb80cc440 tx=0x7fafa402b9a0 comp rx=0 tx=0).stop 2023-11-09T03:47:11.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.826+0000 7fafbf0a9640 1 -- 172.21.15.76:0/3938231095 shutdown_connections 2023-11-09T03:47:11.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.826+0000 7fafbf0a9640 1 --2- 172.21.15.76:0/3938231095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafb806db20 0x7fafb80b03d0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:11.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.826+0000 7fafbf0a9640 1 --2- 172.21.15.76:0/3938231095 >> v2:172.21.15.113:3300/0 conn(0x7fafb8159c70 0x7fafb80ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:11.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.826+0000 7fafbf0a9640 1 -- 172.21.15.76:0/3938231095 >> 172.21.15.76:0/3938231095 conn(0x7fafb80ce950 msgr2=0x7fafb80ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:11.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.826+0000 7fafbf0a9640 1 -- 172.21.15.76:0/3938231095 shutdown_connections 2023-11-09T03:47:11.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.826+0000 7fafbf0a9640 1 -- 172.21.15.76:0/3938231095 wait complete. 2023-11-09T03:47:11.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.827+0000 7fafbf0a9640 1 Processor -- start 2023-11-09T03:47:11.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.827+0000 7fafbf0a9640 1 -- start start 2023-11-09T03:47:11.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.828+0000 7fafbf0a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fafb806db20 0x7fafb8152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:11.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.828+0000 7fafbf0a9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafb80b0910 0x7fafb8146840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:11.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.828+0000 7fafbf0a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafb8159c70 0x7fafb8146d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:11.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.828+0000 7fafbf0a9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb80c67c0 con 0x7fafb80b0910 2023-11-09T03:47:11.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.828+0000 7fafbf0a9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb814e230 con 0x7fafb8159c70 2023-11-09T03:47:11.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.828+0000 7fafbf0a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb81472c0 con 0x7fafb806db20 2023-11-09T03:47:11.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.828+0000 7fafb7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafb80b0910 0x7fafb8146840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:11.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.828+0000 7fafb7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafb80b0910 0x7fafb8146840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40670/0 (socket says 172.21.15.76:40670) 2023-11-09T03:47:11.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.828+0000 7fafb7fff640 1 -- 172.21.15.76:0/272164036 learned_addr learned my addr 172.21.15.76:0/272164036 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:11.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.828+0000 7fafbd646640 1 --2- 172.21.15.76:0/272164036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafb8159c70 0x7fafb8146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:11.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.828+0000 7fafbce45640 1 --2- 172.21.15.76:0/272164036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fafb806db20 0x7fafb8152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:11.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.828+0000 7fafb7fff640 1 -- 172.21.15.76:0/272164036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fafb806db20 msgr2=0x7fafb8152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:11.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.828+0000 7fafb7fff640 1 --2- 172.21.15.76:0/272164036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fafb806db20 0x7fafb8152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:11.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.828+0000 7fafb7fff640 1 -- 172.21.15.76:0/272164036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafb8159c70 msgr2=0x7fafb8146d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:11.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.828+0000 7fafb7fff640 1 --2- 172.21.15.76:0/272164036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafb8159c70 0x7fafb8146d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:11.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.828+0000 7fafb7fff640 1 -- 172.21.15.76:0/272164036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa40094d0 con 0x7fafb80b0910 2023-11-09T03:47:11.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.828+0000 7fafbce45640 1 --2- 172.21.15.76:0/272164036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fafb806db20 0x7fafb8152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:11.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.828+0000 7fafb7fff640 1 --2- 172.21.15.76:0/272164036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafb80b0910 0x7fafb8146840 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fafa402b9e0 tx=0x7fafa4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:11.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.828+0000 7fafb57fa640 1 -- 172.21.15.76:0/272164036 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa4039b60 con 0x7fafb80b0910 2023-11-09T03:47:11.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.828+0000 7fafb57fa640 1 -- 172.21.15.76:0/272164036 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa4039cc0 con 0x7fafb80b0910 2023-11-09T03:47:11.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.828+0000 7fafbf0a9640 1 -- 172.21.15.76:0/272164036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb814e440 con 0x7fafb80b0910 2023-11-09T03:47:11.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.828+0000 7fafb57fa640 1 -- 172.21.15.76:0/272164036 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa4043650 con 0x7fafb80b0910 2023-11-09T03:47:11.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.829+0000 7fafbf0a9640 1 -- 172.21.15.76:0/272164036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb814ea00 con 0x7fafb80b0910 2023-11-09T03:47:11.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.830+0000 7fafbf0a9640 1 -- 172.21.15.76:0/272164036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb8074290 con 0x7fafb80b0910 2023-11-09T03:47:11.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.830+0000 7fafb57fa640 1 -- 172.21.15.76:0/272164036 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fafa404c3b0 con 0x7fafb80b0910 2023-11-09T03:47:11.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.830+0000 7fafb57fa640 1 --2- 172.21.15.76:0/272164036 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf9c06ae70 0x7faf9c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:11.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.830+0000 7fafb57fa640 1 -- 172.21.15.76:0/272164036 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5644+0+0 (secure 0 0 0) 0x7fafa40abb30 con 0x7fafb80b0910 2023-11-09T03:47:11.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.831+0000 7fafbce45640 1 --2- 172.21.15.76:0/272164036 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf9c06ae70 0x7faf9c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:11.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.831+0000 7fafbce45640 1 --2- 172.21.15.76:0/272164036 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf9c06ae70 0x7faf9c06d330 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fafac006fd0 tx=0x7fafac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:11.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:11.835+0000 7fafb57fa640 1 -- 172.21.15.76:0/272164036 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fafa4076e80 con 0x7fafb80b0910 2023-11-09T03:47:12.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:12.063+0000 7fafbf0a9640 1 -- 172.21.15.76:0/272164036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_attaching_pools_with_snaps_to_fs_fails"]} v 0) v1 -- 0x7fafb806d3f0 con 0x7fafb80b0910 2023-11-09T03:47:12.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:12.878+0000 7fafb57fa640 1 -- 172.21.15.76:0/272164036 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_attaching_pools_with_snaps_to_fs_fails"]}]=0 v118) v1 ==== 173+0+0 (secure 0 0 0) 0x7fafa40340f0 con 0x7fafb80b0910 2023-11-09T03:47:12.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:12.880+0000 7fafbf0a9640 1 -- 172.21.15.76:0/272164036 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf9c06ae70 msgr2=0x7faf9c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:12.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:12.880+0000 7fafbf0a9640 1 --2- 172.21.15.76:0/272164036 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf9c06ae70 0x7faf9c06d330 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fafac006fd0 tx=0x7fafac008040 comp rx=0 tx=0).stop 2023-11-09T03:47:12.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:12.880+0000 7fafbf0a9640 1 -- 172.21.15.76:0/272164036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafb80b0910 msgr2=0x7fafb8146840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:12.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:12.880+0000 7fafbf0a9640 1 --2- 172.21.15.76:0/272164036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafb80b0910 0x7fafb8146840 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fafa402b9e0 tx=0x7fafa4004730 comp rx=0 tx=0).stop 2023-11-09T03:47:12.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:12.880+0000 7fafbf0a9640 1 -- 172.21.15.76:0/272164036 shutdown_connections 2023-11-09T03:47:12.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:12.881+0000 7fafbf0a9640 1 --2- 172.21.15.76:0/272164036 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf9c06ae70 0x7faf9c06d330 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:12.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:12.881+0000 7fafbf0a9640 1 --2- 172.21.15.76:0/272164036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafb8159c70 0x7fafb8146d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:12.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:12.881+0000 7fafbf0a9640 1 --2- 172.21.15.76:0/272164036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafb80b0910 0x7fafb8146840 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:12.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:12.881+0000 7fafbf0a9640 1 --2- 172.21.15.76:0/272164036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fafb806db20 0x7fafb8152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:12.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:12.881+0000 7fafbf0a9640 1 -- 172.21.15.76:0/272164036 >> 172.21.15.76:0/272164036 conn(0x7fafb80ce950 msgr2=0x7fafb80cc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:12.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:12.881+0000 7fafbf0a9640 1 -- 172.21.15.76:0/272164036 shutdown_connections 2023-11-09T03:47:12.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:12.881+0000 7fafbf0a9640 1 -- 172.21.15.76:0/272164036 wait complete. 2023-11-09T03:47:12.894 INFO:tasks.cephfs_test_runner:Test that attempt to attach pool with snapshots to an fs fails ... ok 2023-11-09T03:47:12.894 INFO:tasks.cephfs_test_runner:Starting test: test_disallow_monitor_managed_snaps_for_fs_pools (tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools) Test that creation of monitor managed snaps fails for pools attached 2023-11-09T03:47:12.895 INFO:tasks.cephfs_test_runner:test_disallow_monitor_managed_snaps_for_fs_pools (tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools) 2023-11-09T03:47:12.895 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:47:13.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.000+0000 7f20a61d7640 1 Processor -- start 2023-11-09T03:47:13.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.000+0000 7f20a61d7640 1 -- start start 2023-11-09T03:47:13.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.001+0000 7f20a61d7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f20a00cde60 0x7f20a015d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:13.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.001+0000 7f20a61d7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20a015daa0 0x7f20a0157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:13.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.001+0000 7f20a61d7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f20a00ce240 0x7f20a0157540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:13.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.001+0000 7f20a61d7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20a015e5c0 con 0x7f20a00ce240 2023-11-09T03:47:13.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.001+0000 7f20a61d7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20a0157a80 con 0x7f20a00cde60 2023-11-09T03:47:13.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.001+0000 7f20a61d7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a015e000 con 0x7f20a015daa0 2023-11-09T03:47:13.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.001+0000 7f209ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f20a00ce240 0x7f20a0157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:13.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.001+0000 7f209ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f20a00ce240 0x7f20a0157540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60238/0 (socket says 172.21.15.76:60238) 2023-11-09T03:47:13.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.001+0000 7f209ffff640 1 -- 172.21.15.76:0/3414788817 learned_addr learned my addr 172.21.15.76:0/3414788817 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:13.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.002+0000 7f209f7fe640 1 --2- 172.21.15.76:0/3414788817 >> v2:172.21.15.113:3300/0 conn(0x7f20a00cde60 0x7f20a015d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:13.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.002+0000 7f209effd640 1 --2- 172.21.15.76:0/3414788817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20a015daa0 0x7f20a0157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:13.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.002+0000 7f209ffff640 1 -- 172.21.15.76:0/3414788817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20a015daa0 msgr2=0x7f20a0157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:13.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.002+0000 7f209ffff640 1 --2- 172.21.15.76:0/3414788817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20a015daa0 0x7f20a0157000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.002+0000 7f209ffff640 1 -- 172.21.15.76:0/3414788817 >> v2:172.21.15.113:3300/0 conn(0x7f20a00cde60 msgr2=0x7f20a015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:13.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.002+0000 7f209ffff640 1 --2- 172.21.15.76:0/3414788817 >> v2:172.21.15.113:3300/0 conn(0x7f20a00cde60 0x7f20a015d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.002+0000 7f209ffff640 1 -- 172.21.15.76:0/3414788817 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a015e2e0 con 0x7f20a00ce240 2023-11-09T03:47:13.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.002+0000 7f209f7fe640 1 --2- 172.21.15.76:0/3414788817 >> v2:172.21.15.113:3300/0 conn(0x7f20a00cde60 0x7f20a015d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:13.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.002+0000 7f209ffff640 1 --2- 172.21.15.76:0/3414788817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20a00ce240 0x7f20a0157540 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f20a00b5550 tx=0x7f209802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37aea768f827dbc4 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:13.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.003+0000 7f209e7fc640 1 -- 172.21.15.76:0/3414788817 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209802ca70 con 0x7f20a00ce240 2023-11-09T03:47:13.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.003+0000 7f209e7fc640 1 -- 172.21.15.76:0/3414788817 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209802cbd0 con 0x7f20a00ce240 2023-11-09T03:47:13.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.003+0000 7f209e7fc640 1 -- 172.21.15.76:0/3414788817 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20980343f0 con 0x7f20a00ce240 2023-11-09T03:47:13.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.003+0000 7f20a61d7640 1 -- 172.21.15.76:0/3414788817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20a00ce240 msgr2=0x7f20a0157540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:13.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.003+0000 7f20a61d7640 1 --2- 172.21.15.76:0/3414788817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20a00ce240 0x7f20a0157540 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f20a00b5550 tx=0x7f209802bed0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.003+0000 7f20a61d7640 1 -- 172.21.15.76:0/3414788817 shutdown_connections 2023-11-09T03:47:13.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.003+0000 7f20a61d7640 1 --2- 172.21.15.76:0/3414788817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20a00ce240 0x7f20a0157540 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.003+0000 7f20a61d7640 1 --2- 172.21.15.76:0/3414788817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20a015daa0 0x7f20a0157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.003+0000 7f20a61d7640 1 --2- 172.21.15.76:0/3414788817 >> v2:172.21.15.113:3300/0 conn(0x7f20a00cde60 0x7f20a015d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.003+0000 7f20a61d7640 1 -- 172.21.15.76:0/3414788817 >> 172.21.15.76:0/3414788817 conn(0x7f20a0071f30 msgr2=0x7f20a00cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:13.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.003+0000 7f20a61d7640 1 -- 172.21.15.76:0/3414788817 shutdown_connections 2023-11-09T03:47:13.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.003+0000 7f20a61d7640 1 -- 172.21.15.76:0/3414788817 wait complete. 2023-11-09T03:47:13.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.004+0000 7f20a61d7640 1 Processor -- start 2023-11-09T03:47:13.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.005+0000 7f20a61d7640 1 -- start start 2023-11-09T03:47:13.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.005+0000 7f20a61d7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20a00cde60 0x7f20a01541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:13.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.005+0000 7f20a61d7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20a00ce240 0x7f20a00cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:13.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.005+0000 7f20a61d7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20a015daa0 0x7f20a0157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:13.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.005+0000 7f20a61d7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a01f5880 con 0x7f20a00ce240 2023-11-09T03:47:13.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.005+0000 7f20a61d7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a015cb60 con 0x7f20a015daa0 2023-11-09T03:47:13.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.005+0000 7f20a61d7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a01f5b20 con 0x7f20a00cde60 2023-11-09T03:47:13.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.005+0000 7f209effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20a00ce240 0x7f20a00cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:13.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.005+0000 7f209effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20a00ce240 0x7f20a00cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60250/0 (socket says 172.21.15.76:60250) 2023-11-09T03:47:13.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.005+0000 7f209effd640 1 -- 172.21.15.76:0/445809450 learned_addr learned my addr 172.21.15.76:0/445809450 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:13.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.005+0000 7f209f7fe640 1 --2- 172.21.15.76:0/445809450 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20a00cde60 0x7f20a01541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:13.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.006+0000 7f209ffff640 1 --2- 172.21.15.76:0/445809450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20a015daa0 0x7f20a0157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:13.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.006+0000 7f209effd640 1 -- 172.21.15.76:0/445809450 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20a00cde60 msgr2=0x7f20a01541a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:13.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.006+0000 7f209effd640 1 --2- 172.21.15.76:0/445809450 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20a00cde60 0x7f20a01541a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.006+0000 7f209effd640 1 -- 172.21.15.76:0/445809450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20a015daa0 msgr2=0x7f20a0157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:13.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.006+0000 7f209effd640 1 --2- 172.21.15.76:0/445809450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20a015daa0 0x7f20a0157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.006+0000 7f209effd640 1 -- 172.21.15.76:0/445809450 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2098009b30 con 0x7f20a00ce240 2023-11-09T03:47:13.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.006+0000 7f209f7fe640 1 --2- 172.21.15.76:0/445809450 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20a00cde60 0x7f20a01541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:13.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.006+0000 7f209ffff640 1 --2- 172.21.15.76:0/445809450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20a015daa0 0x7f20a0157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:13.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.006+0000 7f209effd640 1 --2- 172.21.15.76:0/445809450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20a00ce240 0x7f20a00cce70 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f209400c7f0 tx=0x7f209400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:13.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.006+0000 7f2087fff640 1 -- 172.21.15.76:0/445809450 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2094004220 con 0x7f20a00ce240 2023-11-09T03:47:13.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.006+0000 7f2087fff640 1 -- 172.21.15.76:0/445809450 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2094004380 con 0x7f20a00ce240 2023-11-09T03:47:13.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.006+0000 7f20a61d7640 1 -- 172.21.15.76:0/445809450 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a01f5e00 con 0x7f20a00ce240 2023-11-09T03:47:13.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.006+0000 7f2087fff640 1 -- 172.21.15.76:0/445809450 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2094007840 con 0x7f20a00ce240 2023-11-09T03:47:13.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.006+0000 7f20a61d7640 1 -- 172.21.15.76:0/445809450 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a01f63c0 con 0x7f20a00ce240 2023-11-09T03:47:13.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.007+0000 7f20a61d7640 1 -- 172.21.15.76:0/445809450 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a0074290 con 0x7f20a00ce240 2023-11-09T03:47:13.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.008+0000 7f2087fff640 1 -- 172.21.15.76:0/445809450 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f20940079a0 con 0x7f20a00ce240 2023-11-09T03:47:13.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.008+0000 7f2087fff640 1 --2- 172.21.15.76:0/445809450 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f208c06ae70 0x7f208c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:13.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.008+0000 7f2087fff640 1 -- 172.21.15.76:0/445809450 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5644+0+0 (secure 0 0 0) 0x7f2094089230 con 0x7f20a00ce240 2023-11-09T03:47:13.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.008+0000 7f209f7fe640 1 --2- 172.21.15.76:0/445809450 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f208c06ae70 0x7f208c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:13.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.009+0000 7f209f7fe640 1 --2- 172.21.15.76:0/445809450 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f208c06ae70 0x7f208c06d330 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2090006b10 tx=0x7f2090008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:13.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.012+0000 7f2087fff640 1 -- 172.21.15.76:0/445809450 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f20940543b0 con 0x7f20a00ce240 2023-11-09T03:47:13.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.179+0000 7f20a61d7640 1 -- 172.21.15.76:0/445809450 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20a00d0c20 con 0x7f20a00ce240 2023-11-09T03:47:13.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.180+0000 7f2087fff640 1 -- 172.21.15.76:0/445809450 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+14973 (secure 0 0 0) 0x7f20940579d0 con 0x7f20a00ce240 2023-11-09T03:47:13.181 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:13.181 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":45,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:47:11.400643+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:6834/981490030":"2023-11-10T03:47:08.866583+0000","172.21.15.76:6832/981490030":"2023-11-10T03:47:08.866583+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":7,"snaps":[{"begin":1,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:47:13.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.182+0000 7f20a61d7640 1 -- 172.21.15.76:0/445809450 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f208c06ae70 msgr2=0x7f208c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:13.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.182+0000 7f20a61d7640 1 --2- 172.21.15.76:0/445809450 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f208c06ae70 0x7f208c06d330 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2090006b10 tx=0x7f2090008040 comp rx=0 tx=0).stop 2023-11-09T03:47:13.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.182+0000 7f20a61d7640 1 -- 172.21.15.76:0/445809450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20a00ce240 msgr2=0x7f20a00cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:13.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.182+0000 7f20a61d7640 1 --2- 172.21.15.76:0/445809450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20a00ce240 0x7f20a00cce70 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f209400c7f0 tx=0x7f209400ccb0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.182+0000 7f20a61d7640 1 -- 172.21.15.76:0/445809450 shutdown_connections 2023-11-09T03:47:13.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.182+0000 7f20a61d7640 1 --2- 172.21.15.76:0/445809450 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f208c06ae70 0x7f208c06d330 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.182+0000 7f20a61d7640 1 --2- 172.21.15.76:0/445809450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20a015daa0 0x7f20a0157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.182+0000 7f20a61d7640 1 --2- 172.21.15.76:0/445809450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20a00ce240 0x7f20a00cce70 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.182+0000 7f20a61d7640 1 --2- 172.21.15.76:0/445809450 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20a00cde60 0x7f20a01541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.182+0000 7f20a61d7640 1 -- 172.21.15.76:0/445809450 >> 172.21.15.76:0/445809450 conn(0x7f20a0071f30 msgr2=0x7f20a00d2510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:13.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.182+0000 7f20a61d7640 1 -- 172.21.15.76:0/445809450 shutdown_connections 2023-11-09T03:47:13.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.182+0000 7f20a61d7640 1 -- 172.21.15.76:0/445809450 wait complete. 2023-11-09T03:47:13.193 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-09T03:47:00.036540+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-09T03:47:13.194 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:47:13.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.297+0000 7fdba6a5d640 1 Processor -- start 2023-11-09T03:47:13.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.297+0000 7fdba6a5d640 1 -- start start 2023-11-09T03:47:13.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.298+0000 7fdba6a5d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdba0159c00 0x7fdba00cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:13.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.298+0000 7fdba6a5d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdba00c48e0 0x7fdba00c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:13.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.298+0000 7fdba6a5d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdba015aff0 con 0x7fdba00adac0 2023-11-09T03:47:13.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.298+0000 7fdba6a5d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdba00c52a0 con 0x7fdba00c48e0 2023-11-09T03:47:13.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.298+0000 7fdba6a5d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba015a3e0 con 0x7fdba0159c00 2023-11-09T03:47:13.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.298+0000 7fdb9f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdba00c48e0 0x7fdba00c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:13.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.298+0000 7fdba4ffa640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fdba00adac0 0x7fdba00adef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57152/0 (socket says 172.21.15.76:57152) 2023-11-09T03:47:13.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.298+0000 7fdba4ffa640 1 -- 172.21.15.76:0/1780769137 learned_addr learned my addr 172.21.15.76:0/1780769137 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:13.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.298+0000 7fdb9ffff640 1 --2- 172.21.15.76:0/1780769137 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdba0159c00 0x7fdba00cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:13.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.298+0000 7fdb9f7fe640 1 -- 172.21.15.76:0/1780769137 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdba0159c00 msgr2=0x7fdba00cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:13.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.298+0000 7fdb9f7fe640 1 --2- 172.21.15.76:0/1780769137 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdba0159c00 0x7fdba00cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.298+0000 7fdb9f7fe640 1 -- 172.21.15.76:0/1780769137 >> v1:172.21.15.76:6789/0 conn(0x7fdba00adac0 legacy=0x7fdba00adef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:13.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.298+0000 7fdb9effd640 1 -- 172.21.15.76:0/1780769137 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3311831216 0 0) 0x7fdba015aff0 con 0x7fdba00adac0 2023-11-09T03:47:13.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.298+0000 7fdb9f7fe640 1 -- 172.21.15.76:0/1780769137 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba015a6c0 con 0x7fdba00c48e0 2023-11-09T03:47:13.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.299+0000 7fdb9f7fe640 1 --2- 172.21.15.76:0/1780769137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdba00c48e0 0x7fdba00c4d60 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fdb940097a0 tx=0x7fdb9402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f6da2fe1050cd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:13.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.299+0000 7fdb9effd640 1 -- 172.21.15.76:0/1780769137 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94031830 con 0x7fdba00c48e0 2023-11-09T03:47:13.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.299+0000 7fdb9effd640 1 -- 172.21.15.76:0/1780769137 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb94031990 con 0x7fdba00c48e0 2023-11-09T03:47:13.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.299+0000 7fdb9effd640 1 -- 172.21.15.76:0/1780769137 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94031d20 con 0x7fdba00c48e0 2023-11-09T03:47:13.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.300+0000 7fdba6a5d640 1 -- 172.21.15.76:0/1780769137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdba00c48e0 msgr2=0x7fdba00c4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:13.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.300+0000 7fdba6a5d640 1 --2- 172.21.15.76:0/1780769137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdba00c48e0 0x7fdba00c4d60 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fdb940097a0 tx=0x7fdb9402b8d0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.300+0000 7fdba6a5d640 1 -- 172.21.15.76:0/1780769137 shutdown_connections 2023-11-09T03:47:13.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.300+0000 7fdba6a5d640 1 --2- 172.21.15.76:0/1780769137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdba00c48e0 0x7fdba00c4d60 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.300+0000 7fdba6a5d640 1 --2- 172.21.15.76:0/1780769137 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdba0159c00 0x7fdba00cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.300+0000 7fdba6a5d640 1 -- 172.21.15.76:0/1780769137 >> 172.21.15.76:0/1780769137 conn(0x7fdba00ce950 msgr2=0x7fdba00ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:13.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.300+0000 7fdba6a5d640 1 -- 172.21.15.76:0/1780769137 shutdown_connections 2023-11-09T03:47:13.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.300+0000 7fdba6a5d640 1 -- 172.21.15.76:0/1780769137 wait complete. 2023-11-09T03:47:13.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.301+0000 7fdba6a5d640 1 Processor -- start 2023-11-09T03:47:13.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.301+0000 7fdba6a5d640 1 -- start start 2023-11-09T03:47:13.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.302+0000 7fdba6a5d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdba00adac0 0x7fdba014dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:13.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.302+0000 7fdba6a5d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdba00c48e0 0x7fdba00cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:13.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.302+0000 7fdba6a5d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdba0159c00 0x7fdba00ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:13.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.302+0000 7fdba6a5d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba00c5f70 con 0x7fdba00c48e0 2023-11-09T03:47:13.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.302+0000 7fdba6a5d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba00c6250 con 0x7fdba00adac0 2023-11-09T03:47:13.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.302+0000 7fdba6a5d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba00c6530 con 0x7fdba0159c00 2023-11-09T03:47:13.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.302+0000 7fdb9f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdba00c48e0 0x7fdba00cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:13.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.302+0000 7fdb9f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdba00c48e0 0x7fdba00cc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60276/0 (socket says 172.21.15.76:60276) 2023-11-09T03:47:13.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.302+0000 7fdb9f7fe640 1 -- 172.21.15.76:0/4164790611 learned_addr learned my addr 172.21.15.76:0/4164790611 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:13.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.302+0000 7fdba4ffa640 1 --2- 172.21.15.76:0/4164790611 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdba0159c00 0x7fdba00ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:13.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.302+0000 7fdb9ffff640 1 --2- 172.21.15.76:0/4164790611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdba00adac0 0x7fdba014dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:13.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.302+0000 7fdb9f7fe640 1 -- 172.21.15.76:0/4164790611 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdba0159c00 msgr2=0x7fdba00ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:13.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.302+0000 7fdb9f7fe640 1 --2- 172.21.15.76:0/4164790611 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdba0159c00 0x7fdba00ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.302+0000 7fdb9f7fe640 1 -- 172.21.15.76:0/4164790611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdba00adac0 msgr2=0x7fdba014dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:13.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.302+0000 7fdb9f7fe640 1 --2- 172.21.15.76:0/4164790611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdba00adac0 0x7fdba014dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.302+0000 7fdb9f7fe640 1 -- 172.21.15.76:0/4164790611 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb94009400 con 0x7fdba00c48e0 2023-11-09T03:47:13.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.302+0000 7fdb9f7fe640 1 --2- 172.21.15.76:0/4164790611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdba00c48e0 0x7fdba00cc890 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fdb9402bee0 tx=0x7fdb940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:13.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.303+0000 7fdb9cff9640 1 -- 172.21.15.76:0/4164790611 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9403a6f0 con 0x7fdba00c48e0 2023-11-09T03:47:13.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.303+0000 7fdb9cff9640 1 -- 172.21.15.76:0/4164790611 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9403a870 con 0x7fdba00c48e0 2023-11-09T03:47:13.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.303+0000 7fdba6a5d640 1 -- 172.21.15.76:0/4164790611 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba00c6810 con 0x7fdba00c48e0 2023-11-09T03:47:13.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.303+0000 7fdb9cff9640 1 -- 172.21.15.76:0/4164790611 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9403ac20 con 0x7fdba00c48e0 2023-11-09T03:47:13.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.303+0000 7fdba6a5d640 1 -- 172.21.15.76:0/4164790611 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba014dff0 con 0x7fdba00c48e0 2023-11-09T03:47:13.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.304+0000 7fdba6a5d640 1 -- 172.21.15.76:0/4164790611 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba01416d0 con 0x7fdba00c48e0 2023-11-09T03:47:13.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.304+0000 7fdb9cff9640 1 -- 172.21.15.76:0/4164790611 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdb9404c3b0 con 0x7fdba00c48e0 2023-11-09T03:47:13.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.304+0000 7fdb9cff9640 1 --2- 172.21.15.76:0/4164790611 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdb8c06ae70 0x7fdb8c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:13.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.304+0000 7fdb9cff9640 1 -- 172.21.15.76:0/4164790611 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5644+0+0 (secure 0 0 0) 0x7fdb940ab410 con 0x7fdba00c48e0 2023-11-09T03:47:13.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.305+0000 7fdb9ffff640 1 --2- 172.21.15.76:0/4164790611 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdb8c06ae70 0x7fdb8c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:13.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.305+0000 7fdb9ffff640 1 --2- 172.21.15.76:0/4164790611 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdb8c06ae70 0x7fdb8c06d330 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdb90006fd0 tx=0x7fdb90008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:13.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.309+0000 7fdb9cff9640 1 -- 172.21.15.76:0/4164790611 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdba01416d0 con 0x7fdba00c48e0 2023-11-09T03:47:13.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.474+0000 7fdba6a5d640 1 -- 172.21.15.76:0/4164790611 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdba0141690 con 0x7fdba00c48e0 2023-11-09T03:47:13.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.474+0000 7fdb9cff9640 1 -- 172.21.15.76:0/4164790611 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdb94080030 con 0x7fdba00c48e0 2023-11-09T03:47:13.475 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:47:13.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.476+0000 7fdba6a5d640 1 -- 172.21.15.76:0/4164790611 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdb8c06ae70 msgr2=0x7fdb8c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:13.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.476+0000 7fdba6a5d640 1 --2- 172.21.15.76:0/4164790611 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdb8c06ae70 0x7fdb8c06d330 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdb90006fd0 tx=0x7fdb90008040 comp rx=0 tx=0).stop 2023-11-09T03:47:13.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.476+0000 7fdba6a5d640 1 -- 172.21.15.76:0/4164790611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdba00c48e0 msgr2=0x7fdba00cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:13.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.476+0000 7fdba6a5d640 1 --2- 172.21.15.76:0/4164790611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdba00c48e0 0x7fdba00cc890 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fdb9402bee0 tx=0x7fdb940046c0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.476+0000 7fdba6a5d640 1 -- 172.21.15.76:0/4164790611 shutdown_connections 2023-11-09T03:47:13.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.476+0000 7fdba6a5d640 1 --2- 172.21.15.76:0/4164790611 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdb8c06ae70 0x7fdb8c06d330 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.476+0000 7fdba6a5d640 1 --2- 172.21.15.76:0/4164790611 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdba0159c00 0x7fdba00ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.476+0000 7fdba6a5d640 1 --2- 172.21.15.76:0/4164790611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdba00c48e0 0x7fdba00cc890 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.476+0000 7fdba6a5d640 1 --2- 172.21.15.76:0/4164790611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdba00adac0 0x7fdba014dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.476+0000 7fdba6a5d640 1 -- 172.21.15.76:0/4164790611 >> 172.21.15.76:0/4164790611 conn(0x7fdba00ce950 msgr2=0x7fdba00cb340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:13.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.476+0000 7fdba6a5d640 1 -- 172.21.15.76:0/4164790611 shutdown_connections 2023-11-09T03:47:13.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.476+0000 7fdba6a5d640 1 -- 172.21.15.76:0/4164790611 wait complete. 2023-11-09T03:47:13.487 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-09T03:47:13.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.591+0000 7f5bcdac9640 1 Processor -- start 2023-11-09T03:47:13.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.591+0000 7f5bcdac9640 1 -- start start 2023-11-09T03:47:13.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.592+0000 7f5bcdac9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5bc815ad30 0x7f5bc80adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:13.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.592+0000 7f5bcdac9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5bc80ad730 0x7f5bc8070710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:13.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.592+0000 7f5bcdac9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5bc8070c50 0x7f5bc80c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:13.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.592+0000 7f5bcdac9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc80ae260 con 0x7f5bc815ad30 2023-11-09T03:47:13.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.592+0000 7f5bcdac9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc80710e0 con 0x7f5bc80ad730 2023-11-09T03:47:13.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.592+0000 7f5bcdac9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc80c3900 con 0x7f5bc8070c50 2023-11-09T03:47:13.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.592+0000 7f5bc77fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5bc815ad30 0x7f5bc80adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:13.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.592+0000 7f5bc77fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5bc815ad30 0x7f5bc80adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60292/0 (socket says 172.21.15.76:60292) 2023-11-09T03:47:13.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.592+0000 7f5bc77fe640 1 -- 172.21.15.76:0/2853457902 learned_addr learned my addr 172.21.15.76:0/2853457902 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:13.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.592+0000 7f5bc6ffd640 1 --2- 172.21.15.76:0/2853457902 >> v2:172.21.15.113:3300/0 conn(0x7f5bc80ad730 0x7f5bc8070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:13.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.592+0000 7f5bc7fff640 1 --2- 172.21.15.76:0/2853457902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5bc8070c50 0x7f5bc80c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:13.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.592+0000 7f5bc77fe640 1 -- 172.21.15.76:0/2853457902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5bc8070c50 msgr2=0x7f5bc80c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:13.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.592+0000 7f5bc77fe640 1 --2- 172.21.15.76:0/2853457902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5bc8070c50 0x7f5bc80c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.592+0000 7f5bc77fe640 1 -- 172.21.15.76:0/2853457902 >> v2:172.21.15.113:3300/0 conn(0x7f5bc80ad730 msgr2=0x7f5bc8070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:13.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.592+0000 7f5bc77fe640 1 --2- 172.21.15.76:0/2853457902 >> v2:172.21.15.113:3300/0 conn(0x7f5bc80ad730 0x7f5bc8070710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.593+0000 7f5bc77fe640 1 -- 172.21.15.76:0/2853457902 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc80c3be0 con 0x7f5bc815ad30 2023-11-09T03:47:13.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.593+0000 7f5bc77fe640 1 --2- 172.21.15.76:0/2853457902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bc815ad30 0x7f5bc80adc80 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5bc80ce7f0 tx=0x7f5bb802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1db36b2767a92f server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:13.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.593+0000 7f5bc67fc640 1 -- 172.21.15.76:0/2853457902 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb80317c0 con 0x7f5bc815ad30 2023-11-09T03:47:13.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.593+0000 7f5bc67fc640 1 -- 172.21.15.76:0/2853457902 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb8031920 con 0x7f5bc815ad30 2023-11-09T03:47:13.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.593+0000 7f5bc67fc640 1 -- 172.21.15.76:0/2853457902 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8031cb0 con 0x7f5bc815ad30 2023-11-09T03:47:13.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.594+0000 7f5bcdac9640 1 -- 172.21.15.76:0/2853457902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bc815ad30 msgr2=0x7f5bc80adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:13.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.594+0000 7f5bcdac9640 1 --2- 172.21.15.76:0/2853457902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bc815ad30 0x7f5bc80adc80 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5bc80ce7f0 tx=0x7f5bb802b860 comp rx=0 tx=0).stop 2023-11-09T03:47:13.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.594+0000 7f5bcdac9640 1 -- 172.21.15.76:0/2853457902 shutdown_connections 2023-11-09T03:47:13.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.594+0000 7f5bcdac9640 1 --2- 172.21.15.76:0/2853457902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5bc8070c50 0x7f5bc80c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.594+0000 7f5bcdac9640 1 --2- 172.21.15.76:0/2853457902 >> v2:172.21.15.113:3300/0 conn(0x7f5bc80ad730 0x7f5bc8070710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.594+0000 7f5bcdac9640 1 --2- 172.21.15.76:0/2853457902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bc815ad30 0x7f5bc80adc80 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.594+0000 7f5bcdac9640 1 -- 172.21.15.76:0/2853457902 >> 172.21.15.76:0/2853457902 conn(0x7f5bc80ce950 msgr2=0x7f5bc80ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:13.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.594+0000 7f5bcdac9640 1 -- 172.21.15.76:0/2853457902 shutdown_connections 2023-11-09T03:47:13.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.594+0000 7f5bcdac9640 1 -- 172.21.15.76:0/2853457902 wait complete. 2023-11-09T03:47:13.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.595+0000 7f5bcdac9640 1 Processor -- start 2023-11-09T03:47:13.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.595+0000 7f5bcdac9640 1 -- start start 2023-11-09T03:47:13.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.596+0000 7f5bcdac9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bc8070c50 0x7f5bc8153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:13.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.596+0000 7f5bcdac9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5bc80ad730 0x7f5bc81463f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:13.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.596+0000 7f5bcdac9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5bc815ad30 0x7f5bc815d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:13.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.596+0000 7f5bcdac9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc815c730 con 0x7f5bc8070c50 2023-11-09T03:47:13.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.596+0000 7f5bcdac9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc8146930 con 0x7f5bc80ad730 2023-11-09T03:47:13.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.596+0000 7f5bcdac9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc815df20 con 0x7f5bc815ad30 2023-11-09T03:47:13.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.596+0000 7f5bc77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bc8070c50 0x7f5bc8153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:13.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.596+0000 7f5bc77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bc8070c50 0x7f5bc8153460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60302/0 (socket says 172.21.15.76:60302) 2023-11-09T03:47:13.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.596+0000 7f5bc77fe640 1 -- 172.21.15.76:0/2981249034 learned_addr learned my addr 172.21.15.76:0/2981249034 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:13.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.596+0000 7f5bc6ffd640 1 --2- 172.21.15.76:0/2981249034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5bc80ad730 0x7f5bc81463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:13.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.596+0000 7f5bc7fff640 1 --2- 172.21.15.76:0/2981249034 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5bc815ad30 0x7f5bc815d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:13.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.596+0000 7f5bc77fe640 1 -- 172.21.15.76:0/2981249034 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5bc815ad30 msgr2=0x7f5bc815d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:13.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.596+0000 7f5bc77fe640 1 --2- 172.21.15.76:0/2981249034 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5bc815ad30 0x7f5bc815d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.596+0000 7f5bc77fe640 1 -- 172.21.15.76:0/2981249034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5bc80ad730 msgr2=0x7f5bc81463f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:13.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.596+0000 7f5bc77fe640 1 --2- 172.21.15.76:0/2981249034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5bc80ad730 0x7f5bc81463f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.596+0000 7f5bc77fe640 1 -- 172.21.15.76:0/2981249034 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb80094d0 con 0x7f5bc8070c50 2023-11-09T03:47:13.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.596+0000 7f5bc6ffd640 1 --2- 172.21.15.76:0/2981249034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5bc80ad730 0x7f5bc81463f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:13.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.596+0000 7f5bc7fff640 1 --2- 172.21.15.76:0/2981249034 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5bc815ad30 0x7f5bc815d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:13.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.597+0000 7f5bc77fe640 1 --2- 172.21.15.76:0/2981249034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bc8070c50 0x7f5bc8153460 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5bb802b8a0 tx=0x7f5bb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:13.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.597+0000 7f5baffff640 1 -- 172.21.15.76:0/2981249034 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8032860 con 0x7f5bc8070c50 2023-11-09T03:47:13.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.597+0000 7f5baffff640 1 -- 172.21.15.76:0/2981249034 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb80329e0 con 0x7f5bc8070c50 2023-11-09T03:47:13.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.597+0000 7f5bcdac9640 1 -- 172.21.15.76:0/2981249034 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc815bb20 con 0x7f5bc8070c50 2023-11-09T03:47:13.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.597+0000 7f5baffff640 1 -- 172.21.15.76:0/2981249034 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8032d90 con 0x7f5bc8070c50 2023-11-09T03:47:13.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.597+0000 7f5bcdac9640 1 -- 172.21.15.76:0/2981249034 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc815c0e0 con 0x7f5bc8070c50 2023-11-09T03:47:13.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.598+0000 7f5bcdac9640 1 -- 172.21.15.76:0/2981249034 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bc8074290 con 0x7f5bc8070c50 2023-11-09T03:47:13.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.598+0000 7f5baffff640 1 -- 172.21.15.76:0/2981249034 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5bb804e450 con 0x7f5bc8070c50 2023-11-09T03:47:13.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.598+0000 7f5baffff640 1 --2- 172.21.15.76:0/2981249034 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5bb406ae70 0x7f5bb406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:13.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.599+0000 7f5baffff640 1 -- 172.21.15.76:0/2981249034 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5644+0+0 (secure 0 0 0) 0x7f5bb80ab8a0 con 0x7f5bc8070c50 2023-11-09T03:47:13.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.599+0000 7f5bc6ffd640 1 --2- 172.21.15.76:0/2981249034 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5bb406ae70 0x7f5bb406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:13.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.599+0000 7f5bc6ffd640 1 --2- 172.21.15.76:0/2981249034 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5bb406ae70 0x7f5bb406d330 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5bc815a5e0 tx=0x7f5bbc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:13.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.603+0000 7f5baffff640 1 -- 172.21.15.76:0/2981249034 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5bb8076ab0 con 0x7f5bc8070c50 2023-11-09T03:47:13.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.768+0000 7f5bcdac9640 1 -- 172.21.15.76:0/2981249034 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f5bc8071970 con 0x7f5bc8070c50 2023-11-09T03:47:13.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.769+0000 7f5baffff640 1 -- 172.21.15.76:0/2981249034 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v45) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5bb807a0d0 con 0x7f5bc8070c50 2023-11-09T03:47:13.769 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:47:13.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.771+0000 7f5bcdac9640 1 -- 172.21.15.76:0/2981249034 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5bb406ae70 msgr2=0x7f5bb406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:13.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.771+0000 7f5bcdac9640 1 --2- 172.21.15.76:0/2981249034 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5bb406ae70 0x7f5bb406d330 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5bc815a5e0 tx=0x7f5bbc008040 comp rx=0 tx=0).stop 2023-11-09T03:47:13.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.771+0000 7f5bcdac9640 1 -- 172.21.15.76:0/2981249034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bc8070c50 msgr2=0x7f5bc8153460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:13.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.771+0000 7f5bcdac9640 1 --2- 172.21.15.76:0/2981249034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bc8070c50 0x7f5bc8153460 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5bb802b8a0 tx=0x7f5bb80046c0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.771+0000 7f5bcdac9640 1 -- 172.21.15.76:0/2981249034 shutdown_connections 2023-11-09T03:47:13.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.771+0000 7f5bcdac9640 1 --2- 172.21.15.76:0/2981249034 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5bb406ae70 0x7f5bb406d330 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.771+0000 7f5bcdac9640 1 --2- 172.21.15.76:0/2981249034 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5bc815ad30 0x7f5bc815d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.771+0000 7f5bcdac9640 1 --2- 172.21.15.76:0/2981249034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5bc80ad730 0x7f5bc81463f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.771+0000 7f5bcdac9640 1 --2- 172.21.15.76:0/2981249034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bc8070c50 0x7f5bc8153460 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.771+0000 7f5bcdac9640 1 -- 172.21.15.76:0/2981249034 >> 172.21.15.76:0/2981249034 conn(0x7f5bc80ce950 msgr2=0x7f5bc806d780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:13.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.771+0000 7f5bcdac9640 1 -- 172.21.15.76:0/2981249034 shutdown_connections 2023-11-09T03:47:13.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.771+0000 7f5bcdac9640 1 -- 172.21.15.76:0/2981249034 wait complete. 2023-11-09T03:47:13.782 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_disallow_monitor_managed_snaps_for_fs_pools' 2023-11-09T03:47:13.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.881+0000 7f42e4a90640 1 Processor -- start 2023-11-09T03:47:13.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.881+0000 7f42e4a90640 1 -- start start 2023-11-09T03:47:13.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.882+0000 7f42e4a90640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f42e00cdd50 0x7f42e00cca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:13.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.882+0000 7f42e4a90640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42e00ccfa0 con 0x7f42e015b300 2023-11-09T03:47:13.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.882+0000 7f42e4a90640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42e00cb9c0 con 0x7f42e015af20 2023-11-09T03:47:13.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.882+0000 7f42e4a90640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e00cbca0 con 0x7f42e00cdd50 2023-11-09T03:47:13.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.882+0000 7f42d7fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f42e015b300 0x7f42e00cd640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57166/0 (socket says 172.21.15.76:57166) 2023-11-09T03:47:13.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.882+0000 7f42d7fff640 1 -- 172.21.15.76:0/2542130506 learned_addr learned my addr 172.21.15.76:0/2542130506 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:13.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.882+0000 7f42ded9d640 1 --2- 172.21.15.76:0/2542130506 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f42e00cdd50 0x7f42e00cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:13.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.882+0000 7f42ddd9b640 1 -- 172.21.15.76:0/2542130506 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1110084104 0 0) 0x7f42c8001110 con 0x7f42e015b300 2023-11-09T03:47:13.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.883+0000 7f42ddd9b640 1 -- 172.21.15.76:0/2542130506 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42cc003520 con 0x7f42e015b300 2023-11-09T03:47:13.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.883+0000 7f42ddd9b640 1 -- 172.21.15.76:0/2542130506 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 424022341 0 0) 0x7f42cc003520 con 0x7f42e015b300 2023-11-09T03:47:13.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.883+0000 7f42ddd9b640 1 -- 172.21.15.76:0/2542130506 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f42e00cdd50 msgr2=0x7f42e00cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:13.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.883+0000 7f42ddd9b640 1 --2- 172.21.15.76:0/2542130506 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f42e00cdd50 0x7f42e00cca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.883+0000 7f42ddd9b640 1 -- 172.21.15.76:0/2542130506 >> v1:172.21.15.113:6789/0 conn(0x7f42e015af20 legacy=0x7f42e00cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:13.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.883+0000 7f42ddd9b640 1 -- 172.21.15.76:0/2542130506 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e00cbf80 con 0x7f42e015b300 2023-11-09T03:47:13.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.883+0000 7f42ddd9b640 1 -- 172.21.15.76:0/2542130506 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f42c8001800 con 0x7f42e015b300 2023-11-09T03:47:13.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.883+0000 7f42ddd9b640 1 -- 172.21.15.76:0/2542130506 >> v1:172.21.15.76:6789/0 conn(0x7f42e015b300 legacy=0x7f42e00cd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:13.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.883+0000 7f42ddd9b640 1 --2- 172.21.15.76:0/2542130506 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f42e00cdd50 0x7f42cc0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:13.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.883+0000 7f42ddd9b640 1 --2- 172.21.15.76:0/2542130506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42cc003f20 0x7f42cc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:13.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.883+0000 7f42ddd9b640 1 --2- 172.21.15.76:0/2542130506 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42cc004d00 0x7f42cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:13.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.883+0000 7f42ddd9b640 1 -- 172.21.15.76:0/2542130506 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e00cbca0 con 0x7f42cc004d00 2023-11-09T03:47:13.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.883+0000 7f42ddd9b640 1 -- 172.21.15.76:0/2542130506 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc00ba60 con 0x7f42cc003f20 2023-11-09T03:47:13.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.883+0000 7f42ddd9b640 1 -- 172.21.15.76:0/2542130506 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc00bc20 con 0x7f42e00cdd50 2023-11-09T03:47:13.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.883+0000 7f42ded9d640 1 --2- 172.21.15.76:0/2542130506 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42cc004d00 0x7f42cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:13.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.883+0000 7f42de59c640 1 --2- 172.21.15.76:0/2542130506 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f42e00cdd50 0x7f42cc0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:13.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.883+0000 7f42d7fff640 1 --2- 172.21.15.76:0/2542130506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42cc003f20 0x7f42cc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:13.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.883+0000 7f42ded9d640 1 -- 172.21.15.76:0/2542130506 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f42e00cdd50 msgr2=0x7f42cc0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:13.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.883+0000 7f42ded9d640 1 --2- 172.21.15.76:0/2542130506 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f42e00cdd50 0x7f42cc0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.883+0000 7f42ded9d640 1 -- 172.21.15.76:0/2542130506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42cc003f20 msgr2=0x7f42cc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:13.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.883+0000 7f42ded9d640 1 --2- 172.21.15.76:0/2542130506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42cc003f20 0x7f42cc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.883+0000 7f42ded9d640 1 -- 172.21.15.76:0/2542130506 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42cc003520 con 0x7f42cc004d00 2023-11-09T03:47:13.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.884+0000 7f42ded9d640 1 --2- 172.21.15.76:0/2542130506 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42cc004d00 0x7f42cc005180 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f42d8009f90 tx=0x7f42cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893d26857665cb68 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:13.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.884+0000 7f42ddd9b640 1 -- 172.21.15.76:0/2542130506 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d802cbd0 con 0x7f42cc004d00 2023-11-09T03:47:13.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.884+0000 7f42ddd9b640 1 -- 172.21.15.76:0/2542130506 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d800f040 con 0x7f42cc004d00 2023-11-09T03:47:13.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.884+0000 7f42ddd9b640 1 -- 172.21.15.76:0/2542130506 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d8034440 con 0x7f42cc004d00 2023-11-09T03:47:13.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.885+0000 7f42e4a90640 1 -- 172.21.15.76:0/2542130506 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42cc004d00 msgr2=0x7f42cc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:13.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.885+0000 7f42e4a90640 1 --2- 172.21.15.76:0/2542130506 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42cc004d00 0x7f42cc005180 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f42d8009f90 tx=0x7f42cc0057f0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.885+0000 7f42ded9d640 1 -- 172.21.15.76:0/2542130506 reap_dead start 2023-11-09T03:47:13.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.885+0000 7f42e4a90640 1 -- 172.21.15.76:0/2542130506 shutdown_connections 2023-11-09T03:47:13.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.885+0000 7f42e4a90640 1 -- 172.21.15.76:0/2542130506 >> 172.21.15.76:0/2542130506 conn(0x7f42e00d0ca0 msgr2=0x7f42e00d10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:13.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.885+0000 7f42e4a90640 1 -- 172.21.15.76:0/2542130506 shutdown_connections 2023-11-09T03:47:13.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.885+0000 7f42e4a90640 1 -- 172.21.15.76:0/2542130506 wait complete. 2023-11-09T03:47:13.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.886+0000 7f42e4a90640 1 Processor -- start 2023-11-09T03:47:13.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.886+0000 7f42e4a90640 1 -- start start 2023-11-09T03:47:13.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.887+0000 7f42e4a90640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f42cc004d00 0x7f42e0144c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:13.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.887+0000 7f42e4a90640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42e01f1880 0x7f42e00b6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:13.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.887+0000 7f42e4a90640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42e00b4510 0x7f42e00b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:13.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.887+0000 7f42e4a90640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e00b37e0 con 0x7f42e01f1880 2023-11-09T03:47:13.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.887+0000 7f42e4a90640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e00b4080 con 0x7f42e00b4510 2023-11-09T03:47:13.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.887+0000 7f42e4a90640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e00b6930 con 0x7f42cc004d00 2023-11-09T03:47:13.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.887+0000 7f42d7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42e01f1880 0x7f42e00b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:13.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.887+0000 7f42d7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42e01f1880 0x7f42e00b6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60324/0 (socket says 172.21.15.76:60324) 2023-11-09T03:47:13.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.887+0000 7f42d7fff640 1 -- 172.21.15.76:0/1008197055 learned_addr learned my addr 172.21.15.76:0/1008197055 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:13.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.887+0000 7f42d7fff640 1 -- 172.21.15.76:0/1008197055 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f42cc004d00 msgr2=0x7f42e0144c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:13.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.887+0000 7f42d7fff640 1 --2- 172.21.15.76:0/1008197055 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f42cc004d00 0x7f42e0144c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.887+0000 7f42d7fff640 1 -- 172.21.15.76:0/1008197055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42e00b4510 msgr2=0x7f42e00b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:13.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.887+0000 7f42ded9d640 1 --2- 172.21.15.76:0/1008197055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42e00b4510 0x7f42e00b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:13.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.887+0000 7f42d7fff640 1 --2- 172.21.15.76:0/1008197055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42e00b4510 0x7f42e00b49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:13.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.887+0000 7f42d7fff640 1 -- 172.21.15.76:0/1008197055 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e00b3a30 con 0x7f42e01f1880 2023-11-09T03:47:13.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.887+0000 7f42d7fff640 1 --2- 172.21.15.76:0/1008197055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42e01f1880 0x7f42e00b6100 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f42c80072c0 tx=0x7f42c8007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:13.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.887+0000 7f42d6ffd640 1 -- 172.21.15.76:0/1008197055 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c800fa20 con 0x7f42e01f1880 2023-11-09T03:47:13.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.887+0000 7f42d6ffd640 1 -- 172.21.15.76:0/1008197055 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c800fb80 con 0x7f42e01f1880 2023-11-09T03:47:13.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.887+0000 7f42e4a90640 1 -- 172.21.15.76:0/1008197055 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e00b3d10 con 0x7f42e01f1880 2023-11-09T03:47:13.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.887+0000 7f42d6ffd640 1 -- 172.21.15.76:0/1008197055 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c8012350 con 0x7f42e01f1880 2023-11-09T03:47:13.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.887+0000 7f42e4a90640 1 -- 172.21.15.76:0/1008197055 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e00b4f00 con 0x7f42e01f1880 2023-11-09T03:47:13.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.888+0000 7f42e4a90640 1 -- 172.21.15.76:0/1008197055 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42e00674e0 con 0x7f42e01f1880 2023-11-09T03:47:13.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.889+0000 7f42d6ffd640 1 -- 172.21.15.76:0/1008197055 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f42c80124b0 con 0x7f42e01f1880 2023-11-09T03:47:13.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.889+0000 7f42d6ffd640 1 --2- 172.21.15.76:0/1008197055 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f42c006aea0 0x7f42c006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:13.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.889+0000 7f42d6ffd640 1 -- 172.21.15.76:0/1008197055 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5644+0+0 (secure 0 0 0) 0x7f42c8089110 con 0x7f42e01f1880 2023-11-09T03:47:13.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.890+0000 7f42de59c640 1 --2- 172.21.15.76:0/1008197055 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f42c006aea0 0x7f42c006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:13.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.890+0000 7f42de59c640 1 --2- 172.21.15.76:0/1008197055 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f42c006aea0 0x7f42c006d360 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f42d00016d0 tx=0x7f42d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:13.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:13.894+0000 7f42d6ffd640 1 -- 172.21.15.76:0/1008197055 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f42c8054290 con 0x7f42e01f1880 2023-11-09T03:47:14.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:14.126+0000 7f42e4a90640 1 -- 172.21.15.76:0/1008197055 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_disallow_monitor_managed_snaps_for_fs_pools"]} v 0) v1 -- 0x7f42e00ccfa0 con 0x7f42e01f1880 2023-11-09T03:47:14.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:14.885+0000 7f42d6ffd640 1 -- 172.21.15.76:0/1008197055 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_disallow_monitor_managed_snaps_for_fs_pools"]}]=0 v120) v1 ==== 181+0+0 (secure 0 0 0) 0x7f42e00ccfa0 con 0x7f42e01f1880 2023-11-09T03:47:14.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:14.887+0000 7f42e4a90640 1 -- 172.21.15.76:0/1008197055 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f42c006aea0 msgr2=0x7f42c006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:14.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:14.887+0000 7f42e4a90640 1 --2- 172.21.15.76:0/1008197055 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f42c006aea0 0x7f42c006d360 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f42d00016d0 tx=0x7f42d0009040 comp rx=0 tx=0).stop 2023-11-09T03:47:14.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:14.887+0000 7f42e4a90640 1 -- 172.21.15.76:0/1008197055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42e01f1880 msgr2=0x7f42e00b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:14.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:14.887+0000 7f42e4a90640 1 --2- 172.21.15.76:0/1008197055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42e01f1880 0x7f42e00b6100 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f42c80072c0 tx=0x7f42c8007780 comp rx=0 tx=0).stop 2023-11-09T03:47:14.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:14.887+0000 7f42e4a90640 1 -- 172.21.15.76:0/1008197055 shutdown_connections 2023-11-09T03:47:14.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:14.887+0000 7f42e4a90640 1 --2- 172.21.15.76:0/1008197055 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f42c006aea0 0x7f42c006d360 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:14.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:14.887+0000 7f42e4a90640 1 --2- 172.21.15.76:0/1008197055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42e00b4510 0x7f42e00b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:14.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:14.887+0000 7f42e4a90640 1 --2- 172.21.15.76:0/1008197055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42e01f1880 0x7f42e00b6100 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:14.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:14.887+0000 7f42e4a90640 1 --2- 172.21.15.76:0/1008197055 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f42cc004d00 0x7f42e0144c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:14.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:14.887+0000 7f42e4a90640 1 -- 172.21.15.76:0/1008197055 >> 172.21.15.76:0/1008197055 conn(0x7f42e00d0ca0 msgr2=0x7f42e0159fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:14.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:14.887+0000 7f42e4a90640 1 -- 172.21.15.76:0/1008197055 shutdown_connections 2023-11-09T03:47:14.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:14.887+0000 7f42e4a90640 1 -- 172.21.15.76:0/1008197055 wait complete. 2023-11-09T03:47:14.901 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-09T03:47:15.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.008+0000 7f090fd3e640 1 Processor -- start 2023-11-09T03:47:15.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.009+0000 7f090fd3e640 1 -- start start 2023-11-09T03:47:15.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.009+0000 7f090fd3e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0908153850 0x7f090815a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:15.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.009+0000 7f090fd3e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f09080cc9d0 0x7f09080c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:15.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.009+0000 7f090fd3e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09080c3860 con 0x7f09080cc520 2023-11-09T03:47:15.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.009+0000 7f090fd3e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09080c9250 con 0x7f09080cc9d0 2023-11-09T03:47:15.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.009+0000 7f090fd3e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0908153c30 con 0x7f0908153850 2023-11-09T03:47:15.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.009+0000 7f090e2db640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f09080cc9d0 0x7f09080c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:15.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.010+0000 7f090e2db640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f09080cc9d0 0x7f09080c3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60334/0 (socket says 172.21.15.76:60334) 2023-11-09T03:47:15.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.010+0000 7f090e2db640 1 -- 172.21.15.76:0/589026594 learned_addr learned my addr 172.21.15.76:0/589026594 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:15.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.010+0000 7f090dada640 1 --2- 172.21.15.76:0/589026594 >> v2:172.21.15.113:3300/0 conn(0x7f0908153850 0x7f090815a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:15.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.010+0000 7f090e2db640 1 -- 172.21.15.76:0/589026594 >> v2:172.21.15.113:3300/0 conn(0x7f0908153850 msgr2=0x7f090815a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:15.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.010+0000 7f090e2db640 1 --2- 172.21.15.76:0/589026594 >> v2:172.21.15.113:3300/0 conn(0x7f0908153850 0x7f090815a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:15.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.010+0000 7f090e2db640 1 -- 172.21.15.76:0/589026594 >> v1:172.21.15.113:6789/0 conn(0x7f09080cc520 legacy=0x7f0908070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:15.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.010+0000 7f090e2db640 1 -- 172.21.15.76:0/589026594 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090815a700 con 0x7f09080cc9d0 2023-11-09T03:47:15.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.011+0000 7f090e2db640 1 --2- 172.21.15.76:0/589026594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09080cc9d0 0x7f09080c3cf0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f09080c5710 tx=0x7f090002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ea66cf685e90ac server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:15.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.011+0000 7f090cad8640 1 -- 172.21.15.76:0/589026594 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0900031870 con 0x7f09080cc9d0 2023-11-09T03:47:15.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.011+0000 7f090cad8640 1 -- 172.21.15.76:0/589026594 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09000319d0 con 0x7f09080cc9d0 2023-11-09T03:47:15.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.011+0000 7f090cad8640 1 -- 172.21.15.76:0/589026594 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0900031d60 con 0x7f09080cc9d0 2023-11-09T03:47:15.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.011+0000 7f090fd3e640 1 -- 172.21.15.76:0/589026594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09080cc9d0 msgr2=0x7f09080c3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:15.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.011+0000 7f090fd3e640 1 --2- 172.21.15.76:0/589026594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09080cc9d0 0x7f09080c3cf0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f09080c5710 tx=0x7f090002b910 comp rx=0 tx=0).stop 2023-11-09T03:47:15.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.011+0000 7f090fd3e640 1 -- 172.21.15.76:0/589026594 shutdown_connections 2023-11-09T03:47:15.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.011+0000 7f090fd3e640 1 --2- 172.21.15.76:0/589026594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09080cc9d0 0x7f09080c3cf0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:15.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.011+0000 7f090fd3e640 1 --2- 172.21.15.76:0/589026594 >> v2:172.21.15.113:3300/0 conn(0x7f0908153850 0x7f090815a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:15.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.011+0000 7f090fd3e640 1 -- 172.21.15.76:0/589026594 >> 172.21.15.76:0/589026594 conn(0x7f09080ce950 msgr2=0x7f09080ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:15.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.011+0000 7f090fd3e640 1 -- 172.21.15.76:0/589026594 shutdown_connections 2023-11-09T03:47:15.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.011+0000 7f090fd3e640 1 -- 172.21.15.76:0/589026594 wait complete. 2023-11-09T03:47:15.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.012+0000 7f090fd3e640 1 Processor -- start 2023-11-09T03:47:15.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.012+0000 7f090fd3e640 1 -- start start 2023-11-09T03:47:15.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.013+0000 7f090fd3e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09080cc520 0x7f09081e19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:15.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.013+0000 7f090fd3e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09080cc9d0 0x7f090814f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:15.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.013+0000 7f090fd3e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0908153850 0x7f09080c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:15.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.013+0000 7f090fd3e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09081f2db0 con 0x7f09080cc520 2023-11-09T03:47:15.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.013+0000 7f090fd3e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09080af2a0 con 0x7f09080cc9d0 2023-11-09T03:47:15.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.013+0000 7f090fd3e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09081e1ef0 con 0x7f0908153850 2023-11-09T03:47:15.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.013+0000 7f090dada640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09080cc520 0x7f09081e19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:15.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.013+0000 7f090dada640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09080cc520 0x7f09081e19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60344/0 (socket says 172.21.15.76:60344) 2023-11-09T03:47:15.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.013+0000 7f090dada640 1 -- 172.21.15.76:0/3404207631 learned_addr learned my addr 172.21.15.76:0/3404207631 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:15.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.013+0000 7f090e2db640 1 --2- 172.21.15.76:0/3404207631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0908153850 0x7f09080c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:15.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.013+0000 7f090d2d9640 1 --2- 172.21.15.76:0/3404207631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09080cc9d0 0x7f090814f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:15.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.014+0000 7f090dada640 1 -- 172.21.15.76:0/3404207631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0908153850 msgr2=0x7f09080c7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:15.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.014+0000 7f090dada640 1 --2- 172.21.15.76:0/3404207631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0908153850 0x7f09080c7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:15.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.014+0000 7f090dada640 1 -- 172.21.15.76:0/3404207631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09080cc9d0 msgr2=0x7f090814f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:15.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.014+0000 7f090dada640 1 --2- 172.21.15.76:0/3404207631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09080cc9d0 0x7f090814f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:15.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.014+0000 7f090dada640 1 -- 172.21.15.76:0/3404207631 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09000094d0 con 0x7f09080cc520 2023-11-09T03:47:15.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.014+0000 7f090d2d9640 1 --2- 172.21.15.76:0/3404207631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09080cc9d0 0x7f090814f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:15.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.014+0000 7f090dada640 1 --2- 172.21.15.76:0/3404207631 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09080cc520 0x7f09081e19b0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f090400cd00 tx=0x7f090400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:15.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.014+0000 7f08fe7fc640 1 -- 172.21.15.76:0/3404207631 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0904003990 con 0x7f09080cc520 2023-11-09T03:47:15.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.014+0000 7f08fe7fc640 1 -- 172.21.15.76:0/3404207631 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0904003af0 con 0x7f09080cc520 2023-11-09T03:47:15.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.014+0000 7f090fd3e640 1 -- 172.21.15.76:0/3404207631 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09081e2190 con 0x7f09080cc520 2023-11-09T03:47:15.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.014+0000 7f08fe7fc640 1 -- 172.21.15.76:0/3404207631 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0904003e80 con 0x7f09080cc520 2023-11-09T03:47:15.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.014+0000 7f090fd3e640 1 -- 172.21.15.76:0/3404207631 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09081e2520 con 0x7f09080cc520 2023-11-09T03:47:15.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.015+0000 7f090fd3e640 1 -- 172.21.15.76:0/3404207631 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0908074290 con 0x7f09080cc520 2023-11-09T03:47:15.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.015+0000 7f08fe7fc640 1 -- 172.21.15.76:0/3404207631 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0904004020 con 0x7f09080cc520 2023-11-09T03:47:15.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.016+0000 7f08fe7fc640 1 --2- 172.21.15.76:0/3404207631 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f08ec06ae70 0x7f08ec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:15.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.016+0000 7f08fe7fc640 1 -- 172.21.15.76:0/3404207631 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5644+0+0 (secure 0 0 0) 0x7f09040898c0 con 0x7f09080cc520 2023-11-09T03:47:15.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.016+0000 7f090d2d9640 1 --2- 172.21.15.76:0/3404207631 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f08ec06ae70 0x7f08ec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:15.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.016+0000 7f090d2d9640 1 --2- 172.21.15.76:0/3404207631 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f08ec06ae70 0x7f08ec06d330 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f09080c8ed0 tx=0x7f08f8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:15.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.020+0000 7f08fe7fc640 1 -- 172.21.15.76:0/3404207631 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0904054a40 con 0x7f09080cc520 2023-11-09T03:47:15.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.185+0000 7f090fd3e640 1 -- 172.21.15.76:0/3404207631 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f09081e2af0 con 0x7f09080cc520 2023-11-09T03:47:15.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.189+0000 7f08fe7fc640 1 -- 172.21.15.76:0/3404207631 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v13)=0 v13) v1 ==== 122+0+0 (secure 0 0 0) 0x7f090401d070 con 0x7f09080cc520 2023-11-09T03:47:15.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.191+0000 7f090fd3e640 1 -- 172.21.15.76:0/3404207631 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f08ec06ae70 msgr2=0x7f08ec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:15.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.191+0000 7f090fd3e640 1 --2- 172.21.15.76:0/3404207631 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f08ec06ae70 0x7f08ec06d330 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f09080c8ed0 tx=0x7f08f8009040 comp rx=0 tx=0).stop 2023-11-09T03:47:15.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.191+0000 7f090fd3e640 1 -- 172.21.15.76:0/3404207631 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09080cc520 msgr2=0x7f09081e19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:15.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.191+0000 7f090fd3e640 1 --2- 172.21.15.76:0/3404207631 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09080cc520 0x7f09081e19b0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f090400cd00 tx=0x7f090400a5c0 comp rx=0 tx=0).stop 2023-11-09T03:47:15.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.191+0000 7f090fd3e640 1 -- 172.21.15.76:0/3404207631 shutdown_connections 2023-11-09T03:47:15.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.191+0000 7f090fd3e640 1 --2- 172.21.15.76:0/3404207631 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f08ec06ae70 0x7f08ec06d330 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:15.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.191+0000 7f090fd3e640 1 --2- 172.21.15.76:0/3404207631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0908153850 0x7f09080c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:15.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.191+0000 7f090fd3e640 1 --2- 172.21.15.76:0/3404207631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09080cc9d0 0x7f090814f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:15.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.191+0000 7f090fd3e640 1 --2- 172.21.15.76:0/3404207631 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09080cc520 0x7f09081e19b0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:15.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.191+0000 7f090fd3e640 1 -- 172.21.15.76:0/3404207631 >> 172.21.15.76:0/3404207631 conn(0x7f09080ce950 msgr2=0x7f09080c4720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:15.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.191+0000 7f090fd3e640 1 -- 172.21.15.76:0/3404207631 shutdown_connections 2023-11-09T03:47:15.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.191+0000 7f090fd3e640 1 -- 172.21.15.76:0/3404207631 wait complete. 2023-11-09T03:47:15.202 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-09T03:47:15.203 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-09T03:47:15.248 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-09T03:47:15.294 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-09T03:47:15.295 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:47:15.295 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:47:15.310 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056381,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14761 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248868k,nr_inodes=812217,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-09T03:47:15.310 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:47:15.311 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:47:15.328 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2440 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-09T03:47:15.328 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:47:15.328 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:47:15.365 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056381,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14761 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248868k,nr_inodes=812217,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-09T03:47:15.365 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:47:15.366 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:47:15.384 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2440 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-09T03:47:15.385 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-09T03:47:15.385 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:47:15.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.503+0000 7fda8401a640 1 Processor -- start 2023-11-09T03:47:15.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.503+0000 7fda8401a640 1 -- start start 2023-11-09T03:47:15.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.504+0000 7fda8401a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fda7c15ad80 0x7fda7c06c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:15.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.504+0000 7fda8401a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fda7c0b42c0 0x7fda7c0c9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:15.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.504+0000 7fda8401a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda7c0ca070 con 0x7fda7c0c9c40 2023-11-09T03:47:15.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.504+0000 7fda8401a640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda7c06cd00 con 0x7fda7c0b42c0 2023-11-09T03:47:15.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.504+0000 7fda8401a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c06cfa0 con 0x7fda7c15ad80 2023-11-09T03:47:15.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.504+0000 7fda815b5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fda7c0b42c0 0x7fda7c0c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:15.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.504+0000 7fda815b5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fda7c0b42c0 0x7fda7c0c9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60346/0 (socket says 172.21.15.76:60346) 2023-11-09T03:47:15.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.504+0000 7fda815b5640 1 -- 172.21.15.76:0/212112617 learned_addr learned my addr 172.21.15.76:0/212112617 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:15.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.505+0000 7fda81db6640 1 --2- 172.21.15.76:0/212112617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fda7c15ad80 0x7fda7c06c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:15.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.505+0000 7fda815b5640 1 -- 172.21.15.76:0/212112617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fda7c15ad80 msgr2=0x7fda7c06c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:15.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.505+0000 7fda815b5640 1 --2- 172.21.15.76:0/212112617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fda7c15ad80 0x7fda7c06c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:15.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.505+0000 7fda815b5640 1 -- 172.21.15.76:0/212112617 >> v1:172.21.15.113:6789/0 conn(0x7fda7c0c9c40 legacy=0x7fda7c070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:15.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.505+0000 7fda815b5640 1 -- 172.21.15.76:0/212112617 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda7c06d280 con 0x7fda7c0b42c0 2023-11-09T03:47:15.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.505+0000 7fda81db6640 1 --2- 172.21.15.76:0/212112617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fda7c15ad80 0x7fda7c06c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:15.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.505+0000 7fda815b5640 1 --2- 172.21.15.76:0/212112617 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda7c0b42c0 0x7fda7c0c9700 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fda7c06ec00 tx=0x7fda6c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44629ee9bbd4b8bd server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:15.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.506+0000 7fda80db4640 1 -- 172.21.15.76:0/212112617 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c030610 con 0x7fda7c0b42c0 2023-11-09T03:47:15.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.506+0000 7fda80db4640 1 -- 172.21.15.76:0/212112617 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6c030770 con 0x7fda7c0b42c0 2023-11-09T03:47:15.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.506+0000 7fda80db4640 1 -- 172.21.15.76:0/212112617 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c030b00 con 0x7fda7c0b42c0 2023-11-09T03:47:15.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.506+0000 7fda8401a640 1 -- 172.21.15.76:0/212112617 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda7c0b42c0 msgr2=0x7fda7c0c9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:15.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.506+0000 7fda8401a640 1 --2- 172.21.15.76:0/212112617 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda7c0b42c0 0x7fda7c0c9700 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fda7c06ec00 tx=0x7fda6c02b960 comp rx=0 tx=0).stop 2023-11-09T03:47:15.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.506+0000 7fda8401a640 1 -- 172.21.15.76:0/212112617 shutdown_connections 2023-11-09T03:47:15.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.506+0000 7fda8401a640 1 --2- 172.21.15.76:0/212112617 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda7c0b42c0 0x7fda7c0c9700 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:15.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.506+0000 7fda8401a640 1 --2- 172.21.15.76:0/212112617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fda7c15ad80 0x7fda7c06c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:15.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.506+0000 7fda8401a640 1 -- 172.21.15.76:0/212112617 >> 172.21.15.76:0/212112617 conn(0x7fda7c0ce950 msgr2=0x7fda7c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:15.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.506+0000 7fda8401a640 1 -- 172.21.15.76:0/212112617 shutdown_connections 2023-11-09T03:47:15.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.506+0000 7fda8401a640 1 -- 172.21.15.76:0/212112617 wait complete. 2023-11-09T03:47:15.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.507+0000 7fda8401a640 1 Processor -- start 2023-11-09T03:47:15.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.507+0000 7fda8401a640 1 -- start start 2023-11-09T03:47:15.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.508+0000 7fda8401a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fda7c0b42c0 0x7fda7c156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:15.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.508+0000 7fda8401a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda7c0c9c40 0x7fda7c148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:15.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.508+0000 7fda8401a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda7c15ad80 0x7fda7c0c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:15.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.508+0000 7fda8401a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c0c74c0 con 0x7fda7c0c9c40 2023-11-09T03:47:15.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.508+0000 7fda8401a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c1570c0 con 0x7fda7c15ad80 2023-11-09T03:47:15.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.508+0000 7fda8401a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c149360 con 0x7fda7c0b42c0 2023-11-09T03:47:15.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.508+0000 7fda815b5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda7c0c9c40 0x7fda7c148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:15.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.508+0000 7fda815b5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda7c0c9c40 0x7fda7c148e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60356/0 (socket says 172.21.15.76:60356) 2023-11-09T03:47:15.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.508+0000 7fda815b5640 1 -- 172.21.15.76:0/1142225236 learned_addr learned my addr 172.21.15.76:0/1142225236 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:15.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.508+0000 7fda81db6640 1 --2- 172.21.15.76:0/1142225236 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fda7c0b42c0 0x7fda7c156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:15.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.508+0000 7fda825b7640 1 --2- 172.21.15.76:0/1142225236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda7c15ad80 0x7fda7c0c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:15.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.508+0000 7fda815b5640 1 -- 172.21.15.76:0/1142225236 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fda7c0b42c0 msgr2=0x7fda7c156b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:15.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.508+0000 7fda815b5640 1 --2- 172.21.15.76:0/1142225236 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fda7c0b42c0 0x7fda7c156b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:15.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.508+0000 7fda815b5640 1 -- 172.21.15.76:0/1142225236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda7c15ad80 msgr2=0x7fda7c0c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:15.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.508+0000 7fda815b5640 1 --2- 172.21.15.76:0/1142225236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda7c15ad80 0x7fda7c0c7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:15.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.508+0000 7fda815b5640 1 -- 172.21.15.76:0/1142225236 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6c0094d0 con 0x7fda7c0c9c40 2023-11-09T03:47:15.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.509+0000 7fda81db6640 1 --2- 172.21.15.76:0/1142225236 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fda7c0b42c0 0x7fda7c156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:15.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.509+0000 7fda825b7640 1 --2- 172.21.15.76:0/1142225236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda7c15ad80 0x7fda7c0c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:15.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.509+0000 7fda815b5640 1 --2- 172.21.15.76:0/1142225236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda7c0c9c40 0x7fda7c148e20 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fda6c000c00 tx=0x7fda6c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:15.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.509+0000 7fda727fc640 1 -- 172.21.15.76:0/1142225236 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c039a90 con 0x7fda7c0c9c40 2023-11-09T03:47:15.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.509+0000 7fda727fc640 1 -- 172.21.15.76:0/1142225236 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6c039bf0 con 0x7fda7c0c9c40 2023-11-09T03:47:15.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.509+0000 7fda727fc640 1 -- 172.21.15.76:0/1142225236 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c0433f0 con 0x7fda7c0c9c40 2023-11-09T03:47:15.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.509+0000 7fda8401a640 1 -- 172.21.15.76:0/1142225236 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda7c15cd60 con 0x7fda7c0c9c40 2023-11-09T03:47:15.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.509+0000 7fda8401a640 1 -- 172.21.15.76:0/1142225236 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda7c15d320 con 0x7fda7c0c9c40 2023-11-09T03:47:15.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.510+0000 7fda8401a640 1 -- 172.21.15.76:0/1142225236 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda7c074290 con 0x7fda7c0c9c40 2023-11-09T03:47:15.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.510+0000 7fda727fc640 1 -- 172.21.15.76:0/1142225236 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fda6c042390 con 0x7fda7c0c9c40 2023-11-09T03:47:15.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.510+0000 7fda727fc640 1 --2- 172.21.15.76:0/1142225236 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fda6006ae70 0x7fda6006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:15.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.511+0000 7fda727fc640 1 -- 172.21.15.76:0/1142225236 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5644+0+0 (secure 0 0 0) 0x7fda6c0ab6e0 con 0x7fda7c0c9c40 2023-11-09T03:47:15.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.511+0000 7fda81db6640 1 --2- 172.21.15.76:0/1142225236 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fda6006ae70 0x7fda6006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:15.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.511+0000 7fda81db6640 1 --2- 172.21.15.76:0/1142225236 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fda6006ae70 0x7fda6006d330 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fda7800d430 tx=0x7fda78007530 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:15.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.515+0000 7fda727fc640 1 -- 172.21.15.76:0/1142225236 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fda6c0768f0 con 0x7fda7c0c9c40 2023-11-09T03:47:15.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.718+0000 7fda8401a640 1 -- 172.21.15.76:0/1142225236 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda7c0b4880 con 0x7fda7c0c9c40 2023-11-09T03:47:15.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.718+0000 7fda727fc640 1 -- 172.21.15.76:0/1142225236 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3699 (secure 0 0 0) 0x7fda6c048e90 con 0x7fda7c0c9c40 2023-11-09T03:47:15.719 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:15.719 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4880,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/711686619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":711686619},{"type":"v1","addr":"172.21.15.76:6835","nonce":711686619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/521006441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":521006441},{"type":"v1","addr":"172.21.15.113:6837","nonce":521006441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5212,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2652975857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2652975857},{"type":"v1","addr":"172.21.15.113:6836","nonce":2652975857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/2390698159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2390698159},{"type":"v1","addr":"172.21.15.76:6834","nonce":2390698159}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[]} 2023-11-09T03:47:15.719 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 18 2023-11-09T03:47:15.721 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.720+0000 7fda8401a640 1 -- 172.21.15.76:0/1142225236 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fda6006ae70 msgr2=0x7fda6006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:15.721 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.720+0000 7fda8401a640 1 --2- 172.21.15.76:0/1142225236 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fda6006ae70 0x7fda6006d330 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fda7800d430 tx=0x7fda78007530 comp rx=0 tx=0).stop 2023-11-09T03:47:15.721 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.720+0000 7fda8401a640 1 -- 172.21.15.76:0/1142225236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda7c0c9c40 msgr2=0x7fda7c148e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:15.721 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.720+0000 7fda8401a640 1 --2- 172.21.15.76:0/1142225236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda7c0c9c40 0x7fda7c148e20 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fda6c000c00 tx=0x7fda6c004730 comp rx=0 tx=0).stop 2023-11-09T03:47:15.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.720+0000 7fda8401a640 1 -- 172.21.15.76:0/1142225236 shutdown_connections 2023-11-09T03:47:15.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.720+0000 7fda8401a640 1 --2- 172.21.15.76:0/1142225236 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fda6006ae70 0x7fda6006d330 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:15.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.720+0000 7fda8401a640 1 --2- 172.21.15.76:0/1142225236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda7c15ad80 0x7fda7c0c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:15.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.720+0000 7fda8401a640 1 --2- 172.21.15.76:0/1142225236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda7c0c9c40 0x7fda7c148e20 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:15.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.720+0000 7fda8401a640 1 --2- 172.21.15.76:0/1142225236 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fda7c0b42c0 0x7fda7c156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:15.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.720+0000 7fda8401a640 1 -- 172.21.15.76:0/1142225236 >> 172.21.15.76:0/1142225236 conn(0x7fda7c0ce950 msgr2=0x7fda7c0af790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:15.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.720+0000 7fda8401a640 1 -- 172.21.15.76:0/1142225236 shutdown_connections 2023-11-09T03:47:15.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.720+0000 7fda8401a640 1 -- 172.21.15.76:0/1142225236 wait complete. 2023-11-09T03:47:15.731 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-09T03:47:15.732 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-09T03:47:15.732 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-09T03:47:15.732 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:47:15.733 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-09T03:47:15.733 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-09T03:47:15.733 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-09T03:47:15.733 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:47:15.734 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-09T03:47:15.734 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-09T03:47:15.734 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-09T03:47:15.735 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:47:15.735 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-09T03:47:15.735 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-09T03:47:15.736 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-09T03:47:15.736 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:47:15.741 INFO:tasks.ceph.mds.a:Stopped 2023-11-09T03:47:15.742 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-09T03:47:15.743 INFO:tasks.ceph.mds.c:Stopped 2023-11-09T03:47:15.743 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-09T03:47:15.746 INFO:tasks.ceph.mds.b:Stopped 2023-11-09T03:47:15.747 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-09T03:47:15.748 INFO:tasks.ceph.mds.d:Stopped 2023-11-09T03:47:15.748 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-09T03:47:15.785 INFO:tasks.ceph.mds.a:Started 2023-11-09T03:47:15.785 DEBUG:teuthology.parallel:result is None 2023-11-09T03:47:15.786 INFO:tasks.ceph.mds.c:Started 2023-11-09T03:47:15.786 DEBUG:teuthology.parallel:result is None 2023-11-09T03:47:15.790 INFO:tasks.ceph.mds.b:Started 2023-11-09T03:47:15.790 DEBUG:teuthology.parallel:result is None 2023-11-09T03:47:15.791 INFO:tasks.ceph.mds.d:Started 2023-11-09T03:47:15.792 DEBUG:teuthology.parallel:result is None 2023-11-09T03:47:15.792 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-09T03:47:15.860 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2023-11-09T03:47:15.860 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2023-11-09T03:47:15.873 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2023-11-09T03:47:15.873 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2023-11-09T03:47:15.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.898+0000 7f4b58a97640 1 Processor -- start 2023-11-09T03:47:15.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.898+0000 7f4b58a97640 1 -- start start 2023-11-09T03:47:15.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.899+0000 7f4b58a97640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4b5415af20 0x7f4b540b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:15.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.899+0000 7f4b58a97640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b540cd3b0 0x7f4b540cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:15.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.899+0000 7f4b58a97640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b540d01f0 con 0x7f4b540b64c0 2023-11-09T03:47:15.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.899+0000 7f4b58a97640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b540cc380 con 0x7f4b5415af20 2023-11-09T03:47:15.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.899+0000 7f4b58a97640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b540cdd80 con 0x7f4b540cd3b0 2023-11-09T03:47:15.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.899+0000 7f4b5259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4b5415af20 0x7f4b540b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:15.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.899+0000 7f4b5259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4b5415af20 0x7f4b540b3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60372/0 (socket says 172.21.15.76:60372) 2023-11-09T03:47:15.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.899+0000 7f4b5259c640 1 -- 172.21.15.76:0/1419242860 learned_addr learned my addr 172.21.15.76:0/1419242860 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:15.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.899+0000 7f4b52d9d640 1 --2- 172.21.15.76:0/1419242860 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b540cd3b0 0x7f4b540cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:15.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.899+0000 7f4b5259c640 1 -- 172.21.15.76:0/1419242860 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b540cd3b0 msgr2=0x7f4b540cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:15.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.899+0000 7f4b5259c640 1 --2- 172.21.15.76:0/1419242860 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b540cd3b0 0x7f4b540cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:15.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.899+0000 7f4b5259c640 1 -- 172.21.15.76:0/1419242860 >> v1:172.21.15.113:6789/0 conn(0x7f4b540b64c0 legacy=0x7f4b540cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:15.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.899+0000 7f4b5259c640 1 -- 172.21.15.76:0/1419242860 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b540b6a40 con 0x7f4b5415af20 2023-11-09T03:47:15.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.900+0000 7f4b5259c640 1 --2- 172.21.15.76:0/1419242860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b5415af20 0x7f4b540b3cd0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4b440098a0 tx=0x7f4b4402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d6210facd117b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:15.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.900+0000 7f4b51d9b640 1 -- 172.21.15.76:0/1419242860 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b44030840 con 0x7f4b5415af20 2023-11-09T03:47:15.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.901+0000 7f4b51d9b640 1 -- 172.21.15.76:0/1419242860 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b440309a0 con 0x7f4b5415af20 2023-11-09T03:47:15.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.901+0000 7f4b51d9b640 1 -- 172.21.15.76:0/1419242860 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b44030d30 con 0x7f4b5415af20 2023-11-09T03:47:15.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.901+0000 7f4b58a97640 1 -- 172.21.15.76:0/1419242860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b5415af20 msgr2=0x7f4b540b3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:15.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.901+0000 7f4b58a97640 1 --2- 172.21.15.76:0/1419242860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b5415af20 0x7f4b540b3cd0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4b440098a0 tx=0x7f4b4402b980 comp rx=0 tx=0).stop 2023-11-09T03:47:15.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.901+0000 7f4b58a97640 1 -- 172.21.15.76:0/1419242860 shutdown_connections 2023-11-09T03:47:15.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.901+0000 7f4b58a97640 1 --2- 172.21.15.76:0/1419242860 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b540cd3b0 0x7f4b540cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:15.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.901+0000 7f4b58a97640 1 --2- 172.21.15.76:0/1419242860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b5415af20 0x7f4b540b3cd0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:15.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.901+0000 7f4b58a97640 1 -- 172.21.15.76:0/1419242860 >> 172.21.15.76:0/1419242860 conn(0x7f4b5406ea20 msgr2=0x7f4b5406ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:15.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.901+0000 7f4b58a97640 1 -- 172.21.15.76:0/1419242860 shutdown_connections 2023-11-09T03:47:15.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.901+0000 7f4b58a97640 1 -- 172.21.15.76:0/1419242860 wait complete. 2023-11-09T03:47:15.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.903+0000 7f4b58a97640 1 Processor -- start 2023-11-09T03:47:15.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.903+0000 7f4b58a97640 1 -- start start 2023-11-09T03:47:15.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.903+0000 7f4b58a97640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b540b64c0 0x7f4b540c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:15.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.903+0000 7f4b58a97640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b540cd3b0 0x7f4b5415baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:15.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.903+0000 7f4b58a97640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b5415af20 0x7f4b5415d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:15.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.903+0000 7f4b58a97640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b540c7960 con 0x7f4b5415af20 2023-11-09T03:47:15.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.903+0000 7f4b58a97640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b540ca600 con 0x7f4b540cd3b0 2023-11-09T03:47:15.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.903+0000 7f4b58a97640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b540ca040 con 0x7f4b540b64c0 2023-11-09T03:47:15.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.904+0000 7f4b52d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b5415af20 0x7f4b5415d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:15.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.904+0000 7f4b5259c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b540b64c0 0x7f4b540c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:15.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.904+0000 7f4b49d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b540cd3b0 0x7f4b5415baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:15.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.904+0000 7f4b52d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b5415af20 0x7f4b5415d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60386/0 (socket says 172.21.15.76:60386) 2023-11-09T03:47:15.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.904+0000 7f4b5259c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b540b64c0 0x7f4b540c7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:57588/0 (socket says 172.21.15.76:57588) 2023-11-09T03:47:15.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.904+0000 7f4b52d9d640 1 -- 172.21.15.76:0/2034197910 learned_addr learned my addr 172.21.15.76:0/2034197910 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:15.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.904+0000 7f4b5259c640 1 -- 172.21.15.76:0/2034197910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b540cd3b0 msgr2=0x7f4b5415baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:15.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.904+0000 7f4b5259c640 1 --2- 172.21.15.76:0/2034197910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b540cd3b0 0x7f4b5415baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:15.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.904+0000 7f4b5259c640 1 -- 172.21.15.76:0/2034197910 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b5415af20 msgr2=0x7f4b5415d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:15.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.904+0000 7f4b5259c640 1 --2- 172.21.15.76:0/2034197910 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b5415af20 0x7f4b5415d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:15.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.904+0000 7f4b5259c640 1 -- 172.21.15.76:0/2034197910 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b440094d0 con 0x7f4b540b64c0 2023-11-09T03:47:15.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.904+0000 7f4b49d9b640 1 --2- 172.21.15.76:0/2034197910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b540cd3b0 0x7f4b5415baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:47:15.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.904+0000 7f4b52d9d640 1 --2- 172.21.15.76:0/2034197910 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b5415af20 0x7f4b5415d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:47:15.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.904+0000 7f4b5259c640 1 --2- 172.21.15.76:0/2034197910 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b540b64c0 0x7f4b540c7420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4b44033000 tx=0x7f4b44004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:15.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.904+0000 7f4b4b7fe640 1 -- 172.21.15.76:0/2034197910 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b44039a90 con 0x7f4b540b64c0 2023-11-09T03:47:15.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.904+0000 7f4b4b7fe640 1 -- 172.21.15.76:0/2034197910 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b44039bf0 con 0x7f4b540b64c0 2023-11-09T03:47:15.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.904+0000 7f4b58a97640 1 -- 172.21.15.76:0/2034197910 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b540ca320 con 0x7f4b540b64c0 2023-11-09T03:47:15.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.904+0000 7f4b4b7fe640 1 -- 172.21.15.76:0/2034197910 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b44043410 con 0x7f4b540b64c0 2023-11-09T03:47:15.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.905+0000 7f4b58a97640 1 -- 172.21.15.76:0/2034197910 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b540b14d0 con 0x7f4b540b64c0 2023-11-09T03:47:15.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.905+0000 7f4b58a97640 1 -- 172.21.15.76:0/2034197910 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b54151c20 con 0x7f4b540b64c0 2023-11-09T03:47:15.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.906+0000 7f4b4b7fe640 1 -- 172.21.15.76:0/2034197910 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4b4402f390 con 0x7f4b540b64c0 2023-11-09T03:47:15.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.906+0000 7f4b4b7fe640 1 --2- 172.21.15.76:0/2034197910 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4b3406ae70 0x7f4b3406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:15.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.906+0000 7f4b4b7fe640 1 -- 172.21.15.76:0/2034197910 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f4b440ab600 con 0x7f4b540b64c0 2023-11-09T03:47:15.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.906+0000 7f4b49d9b640 1 --2- 172.21.15.76:0/2034197910 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4b3406ae70 0x7f4b3406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:15.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.907+0000 7f4b49d9b640 1 --2- 172.21.15.76:0/2034197910 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4b3406ae70 0x7f4b3406d330 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4b540afd10 tx=0x7f4b3c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:15.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:15.911+0000 7f4b4b7fe640 1 -- 172.21.15.76:0/2034197910 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4b44076740 con 0x7f4b540b64c0 2023-11-09T03:47:16.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:16.091+0000 7f4b58a97640 1 -- 172.21.15.76:0/2034197910 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f4b541579e0 con 0x7f4b540b64c0 2023-11-09T03:47:16.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:16.910+0000 7f4b4b7fe640 1 -- 172.21.15.76:0/2034197910 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v50) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4b44079d60 con 0x7f4b540b64c0 2023-11-09T03:47:16.911 INFO:teuthology.orchestra.run.smithi076.stderr: removed all blocklist entries 2023-11-09T03:47:16.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:16.912+0000 7f4b58a97640 1 -- 172.21.15.76:0/2034197910 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4b3406ae70 msgr2=0x7f4b3406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:16.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:16.912+0000 7f4b58a97640 1 --2- 172.21.15.76:0/2034197910 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4b3406ae70 0x7f4b3406d330 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4b540afd10 tx=0x7f4b3c009040 comp rx=0 tx=0).stop 2023-11-09T03:47:16.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:16.912+0000 7f4b58a97640 1 -- 172.21.15.76:0/2034197910 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b540b64c0 msgr2=0x7f4b540c7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:16.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:16.912+0000 7f4b58a97640 1 --2- 172.21.15.76:0/2034197910 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b540b64c0 0x7f4b540c7420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4b44033000 tx=0x7f4b44004730 comp rx=0 tx=0).stop 2023-11-09T03:47:16.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:16.913+0000 7f4b58a97640 1 -- 172.21.15.76:0/2034197910 shutdown_connections 2023-11-09T03:47:16.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:16.913+0000 7f4b58a97640 1 --2- 172.21.15.76:0/2034197910 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4b3406ae70 0x7f4b3406d330 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:16.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:16.913+0000 7f4b58a97640 1 --2- 172.21.15.76:0/2034197910 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b5415af20 0x7f4b5415d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:16.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:16.913+0000 7f4b58a97640 1 --2- 172.21.15.76:0/2034197910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b540cd3b0 0x7f4b5415baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:16.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:16.913+0000 7f4b58a97640 1 --2- 172.21.15.76:0/2034197910 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b540b64c0 0x7f4b540c7420 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:16.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:16.913+0000 7f4b58a97640 1 -- 172.21.15.76:0/2034197910 >> 172.21.15.76:0/2034197910 conn(0x7f4b5406ea20 msgr2=0x7f4b54156e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:16.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:16.913+0000 7f4b58a97640 1 -- 172.21.15.76:0/2034197910 shutdown_connections 2023-11-09T03:47:16.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:16.913+0000 7f4b58a97640 1 -- 172.21.15.76:0/2034197910 wait complete. 2023-11-09T03:47:16.925 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-09T03:47:17.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.027+0000 7f9354b55640 1 Processor -- start 2023-11-09T03:47:17.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.027+0000 7f9354b55640 1 -- start start 2023-11-09T03:47:17.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.027+0000 7f9354b55640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9350159c70 0x7f93500ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:17.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.027+0000 7f9354b55640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f935006db20 0x7f93500b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:17.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.027+0000 7f9354b55640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f93500b0910 0x7f935015d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:17.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.027+0000 7f9354b55640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93500b49b0 con 0x7f935006db20 2023-11-09T03:47:17.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.027+0000 7f9354b55640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93500b0da0 con 0x7f93500b0910 2023-11-09T03:47:17.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.027+0000 7f9354b55640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93500b4110 con 0x7f9350159c70 2023-11-09T03:47:17.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.028+0000 7f9345d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f935006db20 0x7f93500b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:17.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.028+0000 7f9345d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f935006db20 0x7f93500b03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60388/0 (socket says 172.21.15.76:60388) 2023-11-09T03:47:17.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.028+0000 7f9345d9b640 1 -- 172.21.15.76:0/13136212 learned_addr learned my addr 172.21.15.76:0/13136212 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:17.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.028+0000 7f934e59c640 1 --2- 172.21.15.76:0/13136212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9350159c70 0x7f93500ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:17.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.028+0000 7f934ed9d640 1 --2- 172.21.15.76:0/13136212 >> v2:172.21.15.113:3300/0 conn(0x7f93500b0910 0x7f935015d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:17.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.028+0000 7f9345d9b640 1 -- 172.21.15.76:0/13136212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9350159c70 msgr2=0x7f93500ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:17.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.028+0000 7f9345d9b640 1 --2- 172.21.15.76:0/13136212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9350159c70 0x7f93500ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.028+0000 7f9345d9b640 1 -- 172.21.15.76:0/13136212 >> v2:172.21.15.113:3300/0 conn(0x7f93500b0910 msgr2=0x7f935015d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:17.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.028+0000 7f9345d9b640 1 --2- 172.21.15.76:0/13136212 >> v2:172.21.15.113:3300/0 conn(0x7f93500b0910 0x7f935015d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.028+0000 7f9345d9b640 1 -- 172.21.15.76:0/13136212 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93500b43f0 con 0x7f935006db20 2023-11-09T03:47:17.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.028+0000 7f934e59c640 1 --2- 172.21.15.76:0/13136212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9350159c70 0x7f93500ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:17.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.029+0000 7f9345d9b640 1 --2- 172.21.15.76:0/13136212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f935006db20 0x7f93500b03d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f93500ca130 tx=0x7f933802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18fdbb44102c48f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:17.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.029+0000 7f934dd9b640 1 -- 172.21.15.76:0/13136212 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9338031870 con 0x7f935006db20 2023-11-09T03:47:17.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.029+0000 7f934dd9b640 1 -- 172.21.15.76:0/13136212 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93380319d0 con 0x7f935006db20 2023-11-09T03:47:17.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.029+0000 7f934dd9b640 1 -- 172.21.15.76:0/13136212 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9338031d60 con 0x7f935006db20 2023-11-09T03:47:17.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.029+0000 7f9354b55640 1 -- 172.21.15.76:0/13136212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f935006db20 msgr2=0x7f93500b03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:17.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.029+0000 7f9354b55640 1 --2- 172.21.15.76:0/13136212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f935006db20 0x7f93500b03d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f93500ca130 tx=0x7f933802b910 comp rx=0 tx=0).stop 2023-11-09T03:47:17.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.029+0000 7f9354b55640 1 -- 172.21.15.76:0/13136212 shutdown_connections 2023-11-09T03:47:17.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.029+0000 7f9354b55640 1 --2- 172.21.15.76:0/13136212 >> v2:172.21.15.113:3300/0 conn(0x7f93500b0910 0x7f935015d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.029+0000 7f9354b55640 1 --2- 172.21.15.76:0/13136212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f935006db20 0x7f93500b03d0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.029+0000 7f9354b55640 1 --2- 172.21.15.76:0/13136212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9350159c70 0x7f93500ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.029+0000 7f9354b55640 1 -- 172.21.15.76:0/13136212 >> 172.21.15.76:0/13136212 conn(0x7f93500ce950 msgr2=0x7f93500ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:17.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.029+0000 7f9354b55640 1 -- 172.21.15.76:0/13136212 shutdown_connections 2023-11-09T03:47:17.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.029+0000 7f9354b55640 1 -- 172.21.15.76:0/13136212 wait complete. 2023-11-09T03:47:17.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.031+0000 7f9354b55640 1 Processor -- start 2023-11-09T03:47:17.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.031+0000 7f9354b55640 1 -- start start 2023-11-09T03:47:17.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.032+0000 7f9354b55640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f935006db20 0x7f93501565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:17.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.032+0000 7f9354b55640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93500b0910 0x7f93500afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:17.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.032+0000 7f9354b55640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9350159c70 0x7f9350153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:17.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.032+0000 7f9354b55640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9350154c70 con 0x7f935006db20 2023-11-09T03:47:17.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.032+0000 7f9354b55640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93500b0300 con 0x7f9350159c70 2023-11-09T03:47:17.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.032+0000 7f9354b55640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93501546b0 con 0x7f93500b0910 2023-11-09T03:47:17.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.032+0000 7f934e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f935006db20 0x7f93501565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:17.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.032+0000 7f934e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f935006db20 0x7f93501565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60394/0 (socket says 172.21.15.76:60394) 2023-11-09T03:47:17.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.032+0000 7f934e59c640 1 -- 172.21.15.76:0/2029537287 learned_addr learned my addr 172.21.15.76:0/2029537287 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:17.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.032+0000 7f9345d9b640 1 --2- 172.21.15.76:0/2029537287 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93500b0910 0x7f93500afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:17.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.032+0000 7f934ed9d640 1 --2- 172.21.15.76:0/2029537287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9350159c70 0x7f9350153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:17.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.032+0000 7f934e59c640 1 -- 172.21.15.76:0/2029537287 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93500b0910 msgr2=0x7f93500afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:17.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.032+0000 7f934e59c640 1 --2- 172.21.15.76:0/2029537287 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93500b0910 0x7f93500afdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.032+0000 7f934e59c640 1 -- 172.21.15.76:0/2029537287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9350159c70 msgr2=0x7f9350153e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:17.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.032+0000 7f934e59c640 1 --2- 172.21.15.76:0/2029537287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9350159c70 0x7f9350153e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.032+0000 7f934e59c640 1 -- 172.21.15.76:0/2029537287 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93380094d0 con 0x7f935006db20 2023-11-09T03:47:17.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.032+0000 7f934ed9d640 1 --2- 172.21.15.76:0/2029537287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9350159c70 0x7f9350153e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:17.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.032+0000 7f934e59c640 1 --2- 172.21.15.76:0/2029537287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f935006db20 0x7f93501565b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f934000c7f0 tx=0x7f934000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:17.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.032+0000 7f93477fe640 1 -- 172.21.15.76:0/2029537287 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340007a30 con 0x7f935006db20 2023-11-09T03:47:17.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.032+0000 7f93477fe640 1 -- 172.21.15.76:0/2029537287 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9340007b90 con 0x7f935006db20 2023-11-09T03:47:17.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.032+0000 7f93477fe640 1 -- 172.21.15.76:0/2029537287 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934000ce40 con 0x7f935006db20 2023-11-09T03:47:17.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.032+0000 7f9354b55640 1 -- 172.21.15.76:0/2029537287 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9350154990 con 0x7f935006db20 2023-11-09T03:47:17.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.032+0000 7f9354b55640 1 -- 172.21.15.76:0/2029537287 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93500b2a00 con 0x7f935006db20 2023-11-09T03:47:17.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.033+0000 7f9354b55640 1 -- 172.21.15.76:0/2029537287 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93500b2c20 con 0x7f935006db20 2023-11-09T03:47:17.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.034+0000 7f93477fe640 1 -- 172.21.15.76:0/2029537287 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f93400113f0 con 0x7f935006db20 2023-11-09T03:47:17.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.034+0000 7f93477fe640 1 --2- 172.21.15.76:0/2029537287 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f933006ae70 0x7f933006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:17.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.034+0000 7f93477fe640 1 -- 172.21.15.76:0/2029537287 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f9340089700 con 0x7f935006db20 2023-11-09T03:47:17.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.034+0000 7f9345d9b640 1 --2- 172.21.15.76:0/2029537287 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f933006ae70 0x7f933006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:17.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.035+0000 7f9345d9b640 1 --2- 172.21.15.76:0/2029537287 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f933006ae70 0x7f933006d330 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9338036000 tx=0x7f933803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:17.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.039+0000 7f93477fe640 1 -- 172.21.15.76:0/2029537287 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9340054860 con 0x7f935006db20 2023-11-09T03:47:17.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.267+0000 7f9354b55640 1 -- 172.21.15.76:0/2029537287 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9350074290 con 0x7f935006db20 2023-11-09T03:47:17.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.268+0000 7f93477fe640 1 -- 172.21.15.76:0/2029537287 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9340057e80 con 0x7f935006db20 2023-11-09T03:47:17.269 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:17.270 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:47:17.270 INFO:teuthology.orchestra.run.smithi076.stdout: "auth_dump": [ 2023-11-09T03:47:17.270 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:17.271 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.a", 2023-11-09T03:47:17.271 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCgVUxlydtlLxAAi0bnTEOD+fpGL1cYic3caA==", 2023-11-09T03:47:17.271 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:17.271 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:47:17.272 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2023-11-09T03:47:17.272 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2023-11-09T03:47:17.272 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:17.273 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:17.273 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:17.273 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:17.274 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.b", 2023-11-09T03:47:17.274 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxl2+QnABAA7CTWWEbPPXkollRFDlLxPA==", 2023-11-09T03:47:17.274 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:17.274 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:47:17.275 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2023-11-09T03:47:17.275 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2023-11-09T03:47:17.275 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:17.276 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:17.276 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:17.276 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:17.276 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.c", 2023-11-09T03:47:17.277 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCgVUxlrTu9NhAAhg4GpZWZTmVPCP3ffjPo/Q==", 2023-11-09T03:47:17.277 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:17.277 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:47:17.277 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2023-11-09T03:47:17.278 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2023-11-09T03:47:17.278 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:17.278 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:17.278 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:17.281 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:17.281 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.d", 2023-11-09T03:47:17.281 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxl4IZ8BxAA8zGFE+1tlHkT5UyzL0nWxQ==", 2023-11-09T03:47:17.282 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:17.282 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:47:17.282 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2023-11-09T03:47:17.282 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2023-11-09T03:47:17.283 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:17.283 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:17.283 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:17.284 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:17.284 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.0", 2023-11-09T03:47:17.284 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCkVUxlXM5LKhAAM253+XEj5YSwmQywwDieTg==", 2023-11-09T03:47:17.284 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:17.285 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:47:17.285 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:47:17.285 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:17.285 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:17.286 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:17.286 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:17.286 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.1", 2023-11-09T03:47:17.286 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCnVUxlgoDVKBAAkmNFo4E0bAQ1UxkCfQAbjw==", 2023-11-09T03:47:17.287 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:17.287 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:47:17.287 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:47:17.288 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:17.288 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:17.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:17.288 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:17.289 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.2", 2023-11-09T03:47:17.289 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCqVUxlzQjVJhAAOoU5IdN86BmcPz71mTq7ng==", 2023-11-09T03:47:17.289 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:17.289 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:47:17.289 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:47:17.290 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:17.290 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:17.290 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:17.290 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:17.291 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.3", 2023-11-09T03:47:17.291 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCtVUxlhY6VJBAAmCO/ZS8vBa7enyuXX5Y+hw==", 2023-11-09T03:47:17.291 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:17.291 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:47:17.292 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:47:17.292 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:17.292 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:17.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:17.293 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:17.293 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.4", 2023-11-09T03:47:17.293 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCzVUxl79egLxAAnKBOQb/9EQyaHaQt7SawIg==", 2023-11-09T03:47:17.293 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:17.294 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:47:17.294 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:47:17.294 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:17.294 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:17.295 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:17.295 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:17.295 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.5", 2023-11-09T03:47:17.295 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQC2VUxlIfUgLhAAVPurhJCc8cmpt11pQjzL9w==", 2023-11-09T03:47:17.296 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:17.296 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:47:17.296 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:47:17.296 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:17.297 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:17.297 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:17.297 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:17.297 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.6", 2023-11-09T03:47:17.298 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQC5VUxlrI58LBAAWxZotpACR4e8vFj4IiaEKA==", 2023-11-09T03:47:17.298 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:17.298 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:47:17.298 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:47:17.299 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:17.299 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:17.299 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:17.299 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:17.300 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.7", 2023-11-09T03:47:17.300 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQC8VUxl0bA3KxAAwnKNJKqYIcGRLLhQHUaU9w==", 2023-11-09T03:47:17.300 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:17.300 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:47:17.301 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:47:17.301 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:17.301 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:17.301 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:17.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:17.302 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.0", 2023-11-09T03:47:17.302 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxlHj22CxAACNj5X24hdME+GNTLw4RahQ==", 2023-11-09T03:47:17.302 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:17.302 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:47:17.303 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow r", 2023-11-09T03:47:17.303 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-09T03:47:17.303 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:17.303 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:17.304 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:17.304 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.1", 2023-11-09T03:47:17.304 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxljXv4FBAAiTOhfIReQWTy6fbujbBrrw==", 2023-11-09T03:47:17.304 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:17.305 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:47:17.305 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2023-11-09T03:47:17.305 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2023-11-09T03:47:17.305 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2023-11-09T03:47:17.305 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:17.306 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:17.306 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:17.306 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.2", 2023-11-09T03:47:17.306 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxlGVSSERAAQts5odubZ5paZwg8US3QrQ==", 2023-11-09T03:47:17.307 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:17.307 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:47:17.307 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2023-11-09T03:47:17.307 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2023-11-09T03:47:17.308 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2023-11-09T03:47:17.308 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:17.308 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:17.308 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:17.309 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.3", 2023-11-09T03:47:17.309 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxlgmDHGhAA7r98HNQEMnjbqW6RjE5aig==", 2023-11-09T03:47:17.309 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:17.309 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:47:17.310 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2023-11-09T03:47:17.310 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2023-11-09T03:47:17.310 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2023-11-09T03:47:17.310 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:17.311 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:17.311 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:17.311 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.admin", 2023-11-09T03:47:17.311 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCgVUxlV4l8EhAAnrgeLaTaR99vNYGCJjI1TA==", 2023-11-09T03:47:17.312 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:17.312 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2023-11-09T03:47:17.312 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2023-11-09T03:47:17.312 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2023-11-09T03:47:17.313 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:17.313 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:17.313 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:17.313 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:17.314 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mgr.x", 2023-11-09T03:47:17.314 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCgVUxl7JryKxAABFx9CRqSbbJu8SIyi3Va8w==", 2023-11-09T03:47:17.314 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:17.314 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2023-11-09T03:47:17.315 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile mgr", 2023-11-09T03:47:17.315 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:17.315 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:17.315 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:17.316 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:17.316 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mgr.y", 2023-11-09T03:47:17.316 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCgVUxlb9L/JxAAu3IOXTn8TGvbzxYVta3yew==", 2023-11-09T03:47:17.316 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:17.317 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2023-11-09T03:47:17.317 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile mgr", 2023-11-09T03:47:17.317 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:17.317 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:17.317 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:17.318 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2023-11-09T03:47:17.318 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:47:17.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.270+0000 7f9354b55640 1 -- 172.21.15.76:0/2029537287 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f933006ae70 msgr2=0x7f933006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:17.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.270+0000 7f9354b55640 1 --2- 172.21.15.76:0/2029537287 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f933006ae70 0x7f933006d330 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9338036000 tx=0x7f933803c040 comp rx=0 tx=0).stop 2023-11-09T03:47:17.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.270+0000 7f9354b55640 1 -- 172.21.15.76:0/2029537287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f935006db20 msgr2=0x7f93501565b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:17.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.270+0000 7f9354b55640 1 --2- 172.21.15.76:0/2029537287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f935006db20 0x7f93501565b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f934000c7f0 tx=0x7f934000cbf0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.270+0000 7f9354b55640 1 -- 172.21.15.76:0/2029537287 shutdown_connections 2023-11-09T03:47:17.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.270+0000 7f9354b55640 1 --2- 172.21.15.76:0/2029537287 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f933006ae70 0x7f933006d330 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.270+0000 7f9354b55640 1 --2- 172.21.15.76:0/2029537287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9350159c70 0x7f9350153e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.270+0000 7f9354b55640 1 --2- 172.21.15.76:0/2029537287 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93500b0910 0x7f93500afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.270+0000 7f9354b55640 1 --2- 172.21.15.76:0/2029537287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f935006db20 0x7f93501565b0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.270+0000 7f9354b55640 1 -- 172.21.15.76:0/2029537287 >> 172.21.15.76:0/2029537287 conn(0x7f93500ce950 msgr2=0x7f93500b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:17.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.270+0000 7f9354b55640 1 -- 172.21.15.76:0/2029537287 shutdown_connections 2023-11-09T03:47:17.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.270+0000 7f9354b55640 1 -- 172.21.15.76:0/2029537287 wait complete. 2023-11-09T03:47:17.323 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:47:17.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.423+0000 7f17c50d4640 1 Processor -- start 2023-11-09T03:47:17.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.423+0000 7f17c50d4640 1 -- start start 2023-11-09T03:47:17.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.424+0000 7f17c50d4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17c015ce60 0x7f17c006b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:17.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.424+0000 7f17c50d4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f17c00cf160 0x7f17c00cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:17.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.424+0000 7f17c50d4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c01615d0 con 0x7f17c006bd30 2023-11-09T03:47:17.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.424+0000 7f17c50d4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c00cbde0 con 0x7f17c00cf160 2023-11-09T03:47:17.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.424+0000 7f17c50d4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c00cfb50 con 0x7f17c015ce60 2023-11-09T03:47:17.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.424+0000 7f17b7fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f17c006bd30 0x7f17c0160b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57188/0 (socket says 172.21.15.76:57188) 2023-11-09T03:47:17.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.424+0000 7f17b7fff640 1 -- 172.21.15.76:0/44887398 learned_addr learned my addr 172.21.15.76:0/44887398 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:17.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.425+0000 7f17be59c640 1 -- 172.21.15.76:0/44887398 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1589839252 0 0) 0x7f17c01615d0 con 0x7f17c006bd30 2023-11-09T03:47:17.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.425+0000 7f17bed9d640 1 --2- 172.21.15.76:0/44887398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17c015ce60 0x7f17c006b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:17.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.425+0000 7f17be59c640 1 -- 172.21.15.76:0/44887398 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17ac003520 con 0x7f17c006bd30 2023-11-09T03:47:17.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.425+0000 7f17bf59e640 1 --2- 172.21.15.76:0/44887398 >> v2:172.21.15.113:3300/0 conn(0x7f17c00cf160 0x7f17c00cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:17.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.425+0000 7f17be59c640 1 -- 172.21.15.76:0/44887398 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2905041982 0 0) 0x7f17ac003520 con 0x7f17c006bd30 2023-11-09T03:47:17.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.425+0000 7f17be59c640 1 -- 172.21.15.76:0/44887398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17c015ce60 msgr2=0x7f17c006b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:17.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.425+0000 7f17be59c640 1 --2- 172.21.15.76:0/44887398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17c015ce60 0x7f17c006b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.425+0000 7f17be59c640 1 -- 172.21.15.76:0/44887398 >> v2:172.21.15.113:3300/0 conn(0x7f17c00cf160 msgr2=0x7f17c00cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:17.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.425+0000 7f17be59c640 1 --2- 172.21.15.76:0/44887398 >> v2:172.21.15.113:3300/0 conn(0x7f17c00cf160 0x7f17c00cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.425+0000 7f17be59c640 1 -- 172.21.15.76:0/44887398 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c01618b0 con 0x7f17c006bd30 2023-11-09T03:47:17.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.425+0000 7f17be59c640 1 -- 172.21.15.76:0/44887398 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f17a8001670 con 0x7f17c006bd30 2023-11-09T03:47:17.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.425+0000 7f17bf59e640 1 --2- 172.21.15.76:0/44887398 >> v2:172.21.15.113:3300/0 conn(0x7f17c00cf160 0x7f17c00cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:17.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.425+0000 7f17be59c640 1 -- 172.21.15.76:0/44887398 >> v1:172.21.15.76:6789/0 conn(0x7f17c006bd30 legacy=0x7f17c0160b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:17.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.425+0000 7f17be59c640 1 --2- 172.21.15.76:0/44887398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17ac003970 0x7f17ac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:17.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.425+0000 7f17be59c640 1 --2- 172.21.15.76:0/44887398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17c015ce60 0x7f17ac0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:17.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.425+0000 7f17be59c640 1 --2- 172.21.15.76:0/44887398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17ac004d00 0x7f17ac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:17.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.425+0000 7f17be59c640 1 -- 172.21.15.76:0/44887398 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c00cbde0 con 0x7f17ac004d00 2023-11-09T03:47:17.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.425+0000 7f17be59c640 1 -- 172.21.15.76:0/44887398 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c00cfb50 con 0x7f17ac003970 2023-11-09T03:47:17.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.425+0000 7f17be59c640 1 -- 172.21.15.76:0/44887398 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac00b980 con 0x7f17c015ce60 2023-11-09T03:47:17.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.425+0000 7f17bf59e640 1 --2- 172.21.15.76:0/44887398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17ac004d00 0x7f17ac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:17.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.425+0000 7f17bed9d640 1 --2- 172.21.15.76:0/44887398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17ac003970 0x7f17ac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:17.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.425+0000 7f17b7fff640 1 --2- 172.21.15.76:0/44887398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17c015ce60 0x7f17ac0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:17.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.425+0000 7f17bf59e640 1 -- 172.21.15.76:0/44887398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17c015ce60 msgr2=0x7f17ac0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:17.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.425+0000 7f17bf59e640 1 --2- 172.21.15.76:0/44887398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17c015ce60 0x7f17ac0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.425+0000 7f17bf59e640 1 -- 172.21.15.76:0/44887398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17ac003970 msgr2=0x7f17ac003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:17.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.425+0000 7f17bf59e640 1 --2- 172.21.15.76:0/44887398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17ac003970 0x7f17ac003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.425+0000 7f17bf59e640 1 -- 172.21.15.76:0/44887398 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ac003520 con 0x7f17ac004d00 2023-11-09T03:47:17.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.425+0000 7f17bed9d640 1 --2- 172.21.15.76:0/44887398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17ac003970 0x7f17ac003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:17.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.426+0000 7f17bf59e640 1 --2- 172.21.15.76:0/44887398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17ac004d00 0x7f17ac005180 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f17b8007410 tx=0x7f17ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4024df116967ca server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:17.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.426+0000 7f17be59c640 1 -- 172.21.15.76:0/44887398 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b802cdc0 con 0x7f17ac004d00 2023-11-09T03:47:17.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.426+0000 7f17be59c640 1 -- 172.21.15.76:0/44887398 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b8034440 con 0x7f17ac004d00 2023-11-09T03:47:17.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.426+0000 7f17be59c640 1 -- 172.21.15.76:0/44887398 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b80347b0 con 0x7f17ac004d00 2023-11-09T03:47:17.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.427+0000 7f17c50d4640 1 -- 172.21.15.76:0/44887398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17ac004d00 msgr2=0x7f17ac005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:17.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.427+0000 7f17c50d4640 1 --2- 172.21.15.76:0/44887398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17ac004d00 0x7f17ac005180 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f17b8007410 tx=0x7f17ac0057f0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.427+0000 7f17bf59e640 1 -- 172.21.15.76:0/44887398 reap_dead start 2023-11-09T03:47:17.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.427+0000 7f17c50d4640 1 -- 172.21.15.76:0/44887398 shutdown_connections 2023-11-09T03:47:17.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.427+0000 7f17c50d4640 1 -- 172.21.15.76:0/44887398 >> 172.21.15.76:0/44887398 conn(0x7f17c006ea90 msgr2=0x7f17c006ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:17.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.427+0000 7f17c50d4640 1 -- 172.21.15.76:0/44887398 shutdown_connections 2023-11-09T03:47:17.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.427+0000 7f17c50d4640 1 -- 172.21.15.76:0/44887398 wait complete. 2023-11-09T03:47:17.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.428+0000 7f17c50d4640 1 Processor -- start 2023-11-09T03:47:17.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.428+0000 7f17c50d4640 1 -- start start 2023-11-09T03:47:17.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.429+0000 7f17c50d4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17ac004d00 0x7f17c0147930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:17.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.429+0000 7f17c50d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17c00b5ab0 0x7f17c00ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:17.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.429+0000 7f17c50d4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17c00b5f10 0x7f17c00cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:17.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.429+0000 7f17c50d4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c00b47c0 con 0x7f17c00b5ab0 2023-11-09T03:47:17.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.429+0000 7f17c50d4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c01601b0 con 0x7f17c00b5f10 2023-11-09T03:47:17.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.429+0000 7f17c50d4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c0147e70 con 0x7f17ac004d00 2023-11-09T03:47:17.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.429+0000 7f17b7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17c00b5ab0 0x7f17c00ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:17.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.429+0000 7f17b7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17c00b5ab0 0x7f17c00ca610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60418/0 (socket says 172.21.15.76:60418) 2023-11-09T03:47:17.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.429+0000 7f17b7fff640 1 -- 172.21.15.76:0/59783352 learned_addr learned my addr 172.21.15.76:0/59783352 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:17.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.429+0000 7f17bed9d640 1 --2- 172.21.15.76:0/59783352 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17ac004d00 0x7f17c0147930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:17.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.429+0000 7f17bf59e640 1 --2- 172.21.15.76:0/59783352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17c00b5f10 0x7f17c00cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:17.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.429+0000 7f17b7fff640 1 -- 172.21.15.76:0/59783352 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17ac004d00 msgr2=0x7f17c0147930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:17.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.429+0000 7f17b7fff640 1 --2- 172.21.15.76:0/59783352 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17ac004d00 0x7f17c0147930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.429+0000 7f17b7fff640 1 -- 172.21.15.76:0/59783352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17c00b5f10 msgr2=0x7f17c00cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:17.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.429+0000 7f17b7fff640 1 --2- 172.21.15.76:0/59783352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17c00b5f10 0x7f17c00cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.429+0000 7f17b7fff640 1 -- 172.21.15.76:0/59783352 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c00b4a10 con 0x7f17c00b5ab0 2023-11-09T03:47:17.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.429+0000 7f17b7fff640 1 --2- 172.21.15.76:0/59783352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17c00b5ab0 0x7f17c00ca610 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f17a8007310 tx=0x7f17a80077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:17.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.429+0000 7f17b77fe640 1 -- 172.21.15.76:0/59783352 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a800f9a0 con 0x7f17c00b5ab0 2023-11-09T03:47:17.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.429+0000 7f17b77fe640 1 -- 172.21.15.76:0/59783352 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a800fb00 con 0x7f17c00b5ab0 2023-11-09T03:47:17.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.429+0000 7f17b77fe640 1 -- 172.21.15.76:0/59783352 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a800feb0 con 0x7f17c00b5ab0 2023-11-09T03:47:17.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.429+0000 7f17c50d4640 1 -- 172.21.15.76:0/59783352 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c00b4cf0 con 0x7f17c00b5ab0 2023-11-09T03:47:17.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.430+0000 7f17c50d4640 1 -- 172.21.15.76:0/59783352 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c00b52b0 con 0x7f17c00b5ab0 2023-11-09T03:47:17.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.430+0000 7f17c50d4640 1 -- 172.21.15.76:0/59783352 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c0074290 con 0x7f17c00b5ab0 2023-11-09T03:47:17.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.431+0000 7f17b77fe640 1 -- 172.21.15.76:0/59783352 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f17a801d460 con 0x7f17c00b5ab0 2023-11-09T03:47:17.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.431+0000 7f17b77fe640 1 --2- 172.21.15.76:0/59783352 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f17a006ae70 0x7f17a006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:17.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.431+0000 7f17b77fe640 1 -- 172.21.15.76:0/59783352 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f17a8089780 con 0x7f17c00b5ab0 2023-11-09T03:47:17.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.431+0000 7f17bed9d640 1 --2- 172.21.15.76:0/59783352 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f17a006ae70 0x7f17a006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:17.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.432+0000 7f17bed9d640 1 --2- 172.21.15.76:0/59783352 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f17a006ae70 0x7f17a006d330 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f17b0006f80 tx=0x7f17b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:17.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.435+0000 7f17b77fe640 1 -- 172.21.15.76:0/59783352 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f17a8054900 con 0x7f17c00b5ab0 2023-11-09T03:47:17.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.604+0000 7f17c50d4640 1 -- 172.21.15.76:0/59783352 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17c0160370 con 0x7f17c00b5ab0 2023-11-09T03:47:17.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.605+0000 7f17b77fe640 1 -- 172.21.15.76:0/59783352 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+14803 (secure 0 0 0) 0x7f17a8057f20 con 0x7f17c00b5ab0 2023-11-09T03:47:17.606 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:17.606 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":50,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:47:16.908179+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:47:17.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.607+0000 7f17c50d4640 1 -- 172.21.15.76:0/59783352 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f17a006ae70 msgr2=0x7f17a006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:17.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.607+0000 7f17c50d4640 1 --2- 172.21.15.76:0/59783352 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f17a006ae70 0x7f17a006d330 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f17b0006f80 tx=0x7f17b0008040 comp rx=0 tx=0).stop 2023-11-09T03:47:17.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.607+0000 7f17c50d4640 1 -- 172.21.15.76:0/59783352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17c00b5ab0 msgr2=0x7f17c00ca610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:17.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.607+0000 7f17c50d4640 1 --2- 172.21.15.76:0/59783352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17c00b5ab0 0x7f17c00ca610 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f17a8007310 tx=0x7f17a80077d0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.607+0000 7f17c50d4640 1 -- 172.21.15.76:0/59783352 shutdown_connections 2023-11-09T03:47:17.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.607+0000 7f17c50d4640 1 --2- 172.21.15.76:0/59783352 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f17a006ae70 0x7f17a006d330 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.607+0000 7f17c50d4640 1 --2- 172.21.15.76:0/59783352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17c00b5f10 0x7f17c00cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.607+0000 7f17c50d4640 1 --2- 172.21.15.76:0/59783352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17c00b5ab0 0x7f17c00ca610 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.607+0000 7f17c50d4640 1 --2- 172.21.15.76:0/59783352 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17ac004d00 0x7f17c0147930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.607+0000 7f17c50d4640 1 -- 172.21.15.76:0/59783352 >> 172.21.15.76:0/59783352 conn(0x7f17c006ea90 msgr2=0x7f17c00c51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:17.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.607+0000 7f17c50d4640 1 -- 172.21.15.76:0/59783352 shutdown_connections 2023-11-09T03:47:17.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.607+0000 7f17c50d4640 1 -- 172.21.15.76:0/59783352 wait complete. 2023-11-09T03:47:17.619 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-09T03:47:00.036540+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-09T03:47:17.619 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:47:17.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.722+0000 7f5c0c0e5640 1 Processor -- start 2023-11-09T03:47:17.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.723+0000 7f5c0c0e5640 1 -- start start 2023-11-09T03:47:17.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.723+0000 7f5c0c0e5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5c040cd6d0 0x7f5c04161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:17.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.723+0000 7f5c0c0e5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c040b4880 0x7f5c040b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:17.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.723+0000 7f5c0c0e5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c040cde60 con 0x7f5c04153e60 2023-11-09T03:47:17.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.723+0000 7f5c0c0e5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c04160f90 con 0x7f5c040cd6d0 2023-11-09T03:47:17.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.723+0000 7f5c0c0e5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c04161960 con 0x7f5c040b4880 2023-11-09T03:47:17.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.724+0000 7f5c09680640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5c040cd6d0 0x7f5c04161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:17.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.724+0000 7f5c09680640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5c040cd6d0 0x7f5c04161420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60432/0 (socket says 172.21.15.76:60432) 2023-11-09T03:47:17.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.724+0000 7f5c09680640 1 -- 172.21.15.76:0/1794785309 learned_addr learned my addr 172.21.15.76:0/1794785309 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:17.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.724+0000 7f5c0a682640 1 --2- 172.21.15.76:0/1794785309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c040b4880 0x7f5c040b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:17.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.724+0000 7f5c09680640 1 -- 172.21.15.76:0/1794785309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c040b4880 msgr2=0x7f5c040b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:17.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.724+0000 7f5c09680640 1 --2- 172.21.15.76:0/1794785309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c040b4880 0x7f5c040b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.724+0000 7f5c09680640 1 -- 172.21.15.76:0/1794785309 >> v1:172.21.15.113:6789/0 conn(0x7f5c04153e60 legacy=0x7f5c0415dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:17.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.724+0000 7f5c09680640 1 -- 172.21.15.76:0/1794785309 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c040b5220 con 0x7f5c040cd6d0 2023-11-09T03:47:17.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.724+0000 7f5c0a682640 1 --2- 172.21.15.76:0/1794785309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c040b4880 0x7f5c040b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:17.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.725+0000 7f5c09680640 1 --2- 172.21.15.76:0/1794785309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c040cd6d0 0x7f5c04161420 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4009870 tx=0x7f5bf402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349f2a2d2897f148 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:17.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.725+0000 7f5c08e7f640 1 -- 172.21.15.76:0/1794785309 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4030840 con 0x7f5c040cd6d0 2023-11-09T03:47:17.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.725+0000 7f5c08e7f640 1 -- 172.21.15.76:0/1794785309 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf40309a0 con 0x7f5c040cd6d0 2023-11-09T03:47:17.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.725+0000 7f5c08e7f640 1 -- 172.21.15.76:0/1794785309 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4030d30 con 0x7f5c040cd6d0 2023-11-09T03:47:17.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.725+0000 7f5c0c0e5640 1 -- 172.21.15.76:0/1794785309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c040cd6d0 msgr2=0x7f5c04161420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:17.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.725+0000 7f5c0c0e5640 1 --2- 172.21.15.76:0/1794785309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c040cd6d0 0x7f5c04161420 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4009870 tx=0x7f5bf402b980 comp rx=0 tx=0).stop 2023-11-09T03:47:17.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.725+0000 7f5c0c0e5640 1 -- 172.21.15.76:0/1794785309 shutdown_connections 2023-11-09T03:47:17.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.725+0000 7f5c0c0e5640 1 --2- 172.21.15.76:0/1794785309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c040b4880 0x7f5c040b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.725+0000 7f5c0c0e5640 1 --2- 172.21.15.76:0/1794785309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c040cd6d0 0x7f5c04161420 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.725+0000 7f5c0c0e5640 1 -- 172.21.15.76:0/1794785309 >> 172.21.15.76:0/1794785309 conn(0x7f5c040cf880 msgr2=0x7f5c040cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:17.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.725+0000 7f5c0c0e5640 1 -- 172.21.15.76:0/1794785309 shutdown_connections 2023-11-09T03:47:17.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.725+0000 7f5c0c0e5640 1 -- 172.21.15.76:0/1794785309 wait complete. 2023-11-09T03:47:17.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.727+0000 7f5c0c0e5640 1 Processor -- start 2023-11-09T03:47:17.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.727+0000 7f5c0c0e5640 1 -- start start 2023-11-09T03:47:17.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.727+0000 7f5c0c0e5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c040b4880 0x7f5c04159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:17.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.727+0000 7f5c0c0e5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c040cd6d0 0x7f5c0415fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:17.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.727+0000 7f5c0c0e5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c04153e60 0x7f5c041f28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:17.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.727+0000 7f5c0c0e5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c041597a0 con 0x7f5c04153e60 2023-11-09T03:47:17.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.727+0000 7f5c0c0e5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c04158590 con 0x7f5c040b4880 2023-11-09T03:47:17.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.727+0000 7f5c0c0e5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c041f3620 con 0x7f5c040cd6d0 2023-11-09T03:47:17.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.727+0000 7f5c0a682640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c04153e60 0x7f5c041f28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:17.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.727+0000 7f5c0a682640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c04153e60 0x7f5c041f28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60444/0 (socket says 172.21.15.76:60444) 2023-11-09T03:47:17.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.727+0000 7f5c0a682640 1 -- 172.21.15.76:0/786405782 learned_addr learned my addr 172.21.15.76:0/786405782 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:17.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.728+0000 7f5c09680640 1 --2- 172.21.15.76:0/786405782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c040cd6d0 0x7f5c0415fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:17.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.728+0000 7f5c09e81640 1 --2- 172.21.15.76:0/786405782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c040b4880 0x7f5c04159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:17.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.728+0000 7f5c0a682640 1 -- 172.21.15.76:0/786405782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c040cd6d0 msgr2=0x7f5c0415fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:17.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.728+0000 7f5c0a682640 1 --2- 172.21.15.76:0/786405782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c040cd6d0 0x7f5c0415fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.728+0000 7f5c0a682640 1 -- 172.21.15.76:0/786405782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c040b4880 msgr2=0x7f5c04159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:17.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.728+0000 7f5c0a682640 1 --2- 172.21.15.76:0/786405782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c040b4880 0x7f5c04159260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.728+0000 7f5c0a682640 1 -- 172.21.15.76:0/786405782 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf40094d0 con 0x7f5c04153e60 2023-11-09T03:47:17.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.728+0000 7f5c0a682640 1 --2- 172.21.15.76:0/786405782 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c04153e60 0x7f5c041f28f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00e7e0 tx=0x7f5bfc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:17.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.728+0000 7f5bfa7fc640 1 -- 172.21.15.76:0/786405782 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc004030 con 0x7f5c04153e60 2023-11-09T03:47:17.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.728+0000 7f5bfa7fc640 1 -- 172.21.15.76:0/786405782 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bfc004190 con 0x7f5c04153e60 2023-11-09T03:47:17.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.728+0000 7f5c0c0e5640 1 -- 172.21.15.76:0/786405782 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c041587a0 con 0x7f5c04153e60 2023-11-09T03:47:17.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.728+0000 7f5bfa7fc640 1 -- 172.21.15.76:0/786405782 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc004500 con 0x7f5c04153e60 2023-11-09T03:47:17.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.728+0000 7f5c0c0e5640 1 -- 172.21.15.76:0/786405782 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c04158d60 con 0x7f5c04153e60 2023-11-09T03:47:17.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.729+0000 7f5c0c0e5640 1 -- 172.21.15.76:0/786405782 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c04074290 con 0x7f5c04153e60 2023-11-09T03:47:17.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.729+0000 7f5bfa7fc640 1 -- 172.21.15.76:0/786405782 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5bfc00c6d0 con 0x7f5c04153e60 2023-11-09T03:47:17.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.730+0000 7f5bfa7fc640 1 --2- 172.21.15.76:0/786405782 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5be806ae70 0x7f5be806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:17.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.730+0000 7f5bfa7fc640 1 -- 172.21.15.76:0/786405782 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f5bfc089150 con 0x7f5c04153e60 2023-11-09T03:47:17.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.730+0000 7f5c09e81640 1 --2- 172.21.15.76:0/786405782 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5be806ae70 0x7f5be806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:17.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.730+0000 7f5c09e81640 1 --2- 172.21.15.76:0/786405782 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5be806ae70 0x7f5be806d330 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5c000012e0 tx=0x7f5c00009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:17.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.734+0000 7f5bfa7fc640 1 -- 172.21.15.76:0/786405782 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5bfc0542b0 con 0x7f5c04153e60 2023-11-09T03:47:17.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.900+0000 7f5c0c0e5640 1 -- 172.21.15.76:0/786405782 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5c040d0b80 con 0x7f5c04153e60 2023-11-09T03:47:17.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.901+0000 7f5bfa7fc640 1 -- 172.21.15.76:0/786405782 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5bfc0578d0 con 0x7f5c04153e60 2023-11-09T03:47:17.901 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:47:17.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.902+0000 7f5c0c0e5640 1 -- 172.21.15.76:0/786405782 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5be806ae70 msgr2=0x7f5be806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:17.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.902+0000 7f5c0c0e5640 1 --2- 172.21.15.76:0/786405782 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5be806ae70 0x7f5be806d330 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5c000012e0 tx=0x7f5c00009040 comp rx=0 tx=0).stop 2023-11-09T03:47:17.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.902+0000 7f5c0c0e5640 1 -- 172.21.15.76:0/786405782 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c04153e60 msgr2=0x7f5c041f28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:17.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.902+0000 7f5c0c0e5640 1 --2- 172.21.15.76:0/786405782 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c04153e60 0x7f5c041f28f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00e7e0 tx=0x7f5bfc00eca0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.902+0000 7f5c0c0e5640 1 -- 172.21.15.76:0/786405782 shutdown_connections 2023-11-09T03:47:17.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.902+0000 7f5c0c0e5640 1 --2- 172.21.15.76:0/786405782 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5be806ae70 0x7f5be806d330 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.902+0000 7f5c0c0e5640 1 --2- 172.21.15.76:0/786405782 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c04153e60 0x7f5c041f28f0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.902+0000 7f5c0c0e5640 1 --2- 172.21.15.76:0/786405782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c040cd6d0 0x7f5c0415fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.902+0000 7f5c0c0e5640 1 --2- 172.21.15.76:0/786405782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c040b4880 0x7f5c04159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:17.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.902+0000 7f5c0c0e5640 1 -- 172.21.15.76:0/786405782 >> 172.21.15.76:0/786405782 conn(0x7f5c040cf880 msgr2=0x7f5c040d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:17.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.902+0000 7f5c0c0e5640 1 -- 172.21.15.76:0/786405782 shutdown_connections 2023-11-09T03:47:17.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:17.903+0000 7f5c0c0e5640 1 -- 172.21.15.76:0/786405782 wait complete. 2023-11-09T03:47:17.915 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-09T03:47:18.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.017+0000 7fafccaf2640 1 Processor -- start 2023-11-09T03:47:18.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.017+0000 7fafccaf2640 1 -- start start 2023-11-09T03:47:18.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.018+0000 7fafccaf2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fafc8153e60 0x7fafc815dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:18.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.018+0000 7fafccaf2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fafc80b4880 0x7fafc80b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:18.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.018+0000 7fafccaf2640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc80cde60 con 0x7fafc80cd6d0 2023-11-09T03:47:18.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.018+0000 7fafccaf2640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc8160f90 con 0x7fafc80b4880 2023-11-09T03:47:18.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.018+0000 7fafccaf2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc80b5200 con 0x7fafc8153e60 2023-11-09T03:47:18.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.018+0000 7fafc6d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fafc80b4880 0x7fafc80b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:18.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.018+0000 7fafbdd9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fafc80cd6d0 0x7fafc8161420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57194/0 (socket says 172.21.15.76:57194) 2023-11-09T03:47:18.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.018+0000 7fafbdd9b640 1 -- 172.21.15.76:0/3806127581 learned_addr learned my addr 172.21.15.76:0/3806127581 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:18.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.019+0000 7fafc6d9d640 1 -- 172.21.15.76:0/3806127581 >> v2:172.21.15.113:3300/0 conn(0x7fafc8153e60 msgr2=0x7fafc815dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:18.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.019+0000 7fafc659c640 1 --2- 172.21.15.76:0/3806127581 >> v2:172.21.15.113:3300/0 conn(0x7fafc8153e60 0x7fafc815dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:18.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.019+0000 7fafc6d9d640 1 --2- 172.21.15.76:0/3806127581 >> v2:172.21.15.113:3300/0 conn(0x7fafc8153e60 0x7fafc815dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.019+0000 7fafc5d9b640 1 -- 172.21.15.76:0/3806127581 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3463036443 0 0) 0x7fafc80cde60 con 0x7fafc80cd6d0 2023-11-09T03:47:18.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.019+0000 7fafc6d9d640 1 -- 172.21.15.76:0/3806127581 >> v1:172.21.15.76:6789/0 conn(0x7fafc80cd6d0 legacy=0x7fafc8161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:18.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.019+0000 7fafc6d9d640 1 -- 172.21.15.76:0/3806127581 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc815e400 con 0x7fafc80b4880 2023-11-09T03:47:18.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.019+0000 7fafc659c640 1 --2- 172.21.15.76:0/3806127581 >> v2:172.21.15.113:3300/0 conn(0x7fafc8153e60 0x7fafc815dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:18.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.019+0000 7fafc6d9d640 1 --2- 172.21.15.76:0/3806127581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafc80b4880 0x7fafc80b4cc0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fafc808abe0 tx=0x7fafc002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385b4a628174a514 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:18.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.019+0000 7fafc5d9b640 1 -- 172.21.15.76:0/3806127581 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc002cc80 con 0x7fafc80b4880 2023-11-09T03:47:18.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.019+0000 7fafc5d9b640 1 -- 172.21.15.76:0/3806127581 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc002cde0 con 0x7fafc80b4880 2023-11-09T03:47:18.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.019+0000 7fafc5d9b640 1 -- 172.21.15.76:0/3806127581 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc0034660 con 0x7fafc80b4880 2023-11-09T03:47:18.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.020+0000 7fafccaf2640 1 -- 172.21.15.76:0/3806127581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafc80b4880 msgr2=0x7fafc80b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:18.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.020+0000 7fafccaf2640 1 --2- 172.21.15.76:0/3806127581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafc80b4880 0x7fafc80b4cc0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fafc808abe0 tx=0x7fafc002bed0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.020+0000 7fafccaf2640 1 -- 172.21.15.76:0/3806127581 shutdown_connections 2023-11-09T03:47:18.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.020+0000 7fafccaf2640 1 --2- 172.21.15.76:0/3806127581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafc80b4880 0x7fafc80b4cc0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.020+0000 7fafccaf2640 1 --2- 172.21.15.76:0/3806127581 >> v2:172.21.15.113:3300/0 conn(0x7fafc8153e60 0x7fafc815dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.020+0000 7fafccaf2640 1 -- 172.21.15.76:0/3806127581 >> 172.21.15.76:0/3806127581 conn(0x7fafc80cf880 msgr2=0x7fafc80cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:18.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.020+0000 7fafccaf2640 1 -- 172.21.15.76:0/3806127581 shutdown_connections 2023-11-09T03:47:18.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.020+0000 7fafccaf2640 1 -- 172.21.15.76:0/3806127581 wait complete. 2023-11-09T03:47:18.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.021+0000 7fafccaf2640 1 Processor -- start 2023-11-09T03:47:18.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.021+0000 7fafccaf2640 1 -- start start 2023-11-09T03:47:18.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.022+0000 7fafccaf2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafc80b4880 0x7fafc8145110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:18.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.022+0000 7fafccaf2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafc80cd6d0 0x7fafc81525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:18.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.022+0000 7fafccaf2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fafc8153e60 0x7fafc8156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:18.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.022+0000 7fafccaf2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc8155890 con 0x7fafc80b4880 2023-11-09T03:47:18.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.022+0000 7fafccaf2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc8152b20 con 0x7fafc80cd6d0 2023-11-09T03:47:18.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.022+0000 7fafccaf2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc8155ac0 con 0x7fafc8153e60 2023-11-09T03:47:18.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.022+0000 7fafc659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafc80b4880 0x7fafc8145110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:18.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.022+0000 7fafc659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafc80b4880 0x7fafc8145110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60450/0 (socket says 172.21.15.76:60450) 2023-11-09T03:47:18.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.022+0000 7fafc659c640 1 -- 172.21.15.76:0/3914492456 learned_addr learned my addr 172.21.15.76:0/3914492456 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:18.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.022+0000 7fafbdd9b640 1 --2- 172.21.15.76:0/3914492456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafc80cd6d0 0x7fafc81525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:18.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.022+0000 7fafc6d9d640 1 --2- 172.21.15.76:0/3914492456 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fafc8153e60 0x7fafc8156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:18.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.023+0000 7fafc659c640 1 -- 172.21.15.76:0/3914492456 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fafc8153e60 msgr2=0x7fafc8156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:18.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.023+0000 7fafc659c640 1 --2- 172.21.15.76:0/3914492456 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fafc8153e60 0x7fafc8156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.023+0000 7fafc659c640 1 -- 172.21.15.76:0/3914492456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafc80cd6d0 msgr2=0x7fafc81525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:18.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.023+0000 7fafc659c640 1 --2- 172.21.15.76:0/3914492456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafc80cd6d0 0x7fafc81525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.023+0000 7fafc659c640 1 -- 172.21.15.76:0/3914492456 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc0009a70 con 0x7fafc80b4880 2023-11-09T03:47:18.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.023+0000 7fafc659c640 1 --2- 172.21.15.76:0/3914492456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafc80b4880 0x7fafc8145110 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fafb800c7f0 tx=0x7fafb800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:18.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.023+0000 7fafbf7fe640 1 -- 172.21.15.76:0/3914492456 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb8007820 con 0x7fafc80b4880 2023-11-09T03:47:18.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.023+0000 7fafbf7fe640 1 -- 172.21.15.76:0/3914492456 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb8007980 con 0x7fafc80b4880 2023-11-09T03:47:18.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.023+0000 7fafbf7fe640 1 -- 172.21.15.76:0/3914492456 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb8007cf0 con 0x7fafc80b4880 2023-11-09T03:47:18.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.023+0000 7fafccaf2640 1 -- 172.21.15.76:0/3914492456 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc8155da0 con 0x7fafc80b4880 2023-11-09T03:47:18.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.023+0000 7fafccaf2640 1 -- 172.21.15.76:0/3914492456 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc8156360 con 0x7fafc80b4880 2023-11-09T03:47:18.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.024+0000 7fafccaf2640 1 -- 172.21.15.76:0/3914492456 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafc80c5e10 con 0x7fafc80b4880 2023-11-09T03:47:18.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.024+0000 7fafbf7fe640 1 -- 172.21.15.76:0/3914492456 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fafb80113f0 con 0x7fafc80b4880 2023-11-09T03:47:18.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.025+0000 7fafbf7fe640 1 --2- 172.21.15.76:0/3914492456 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fafa806ae70 0x7fafa806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:18.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.025+0000 7fafbf7fe640 1 -- 172.21.15.76:0/3914492456 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fafb8089760 con 0x7fafc80b4880 2023-11-09T03:47:18.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.025+0000 7fafbdd9b640 1 --2- 172.21.15.76:0/3914492456 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fafa806ae70 0x7fafa806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:18.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.025+0000 7fafbdd9b640 1 --2- 172.21.15.76:0/3914492456 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fafa806ae70 0x7fafa806d330 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fafc815f560 tx=0x7fafb0007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:18.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.029+0000 7fafbf7fe640 1 -- 172.21.15.76:0/3914492456 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fafb8054a00 con 0x7fafc80b4880 2023-11-09T03:47:18.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.195+0000 7fafccaf2640 1 -- 172.21.15.76:0/3914492456 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7fafc8074290 con 0x7fafc80b4880 2023-11-09T03:47:18.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.196+0000 7fafbf7fe640 1 -- 172.21.15.76:0/3914492456 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v50) v1 ==== 103+0+11 (secure 0 0 0) 0x7fafb801f020 con 0x7fafc80b4880 2023-11-09T03:47:18.196 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:47:18.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.197+0000 7fafccaf2640 1 -- 172.21.15.76:0/3914492456 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fafa806ae70 msgr2=0x7fafa806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:18.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.197+0000 7fafccaf2640 1 --2- 172.21.15.76:0/3914492456 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fafa806ae70 0x7fafa806d330 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fafc815f560 tx=0x7fafb0007d50 comp rx=0 tx=0).stop 2023-11-09T03:47:18.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.197+0000 7fafccaf2640 1 -- 172.21.15.76:0/3914492456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafc80b4880 msgr2=0x7fafc8145110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:18.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.197+0000 7fafccaf2640 1 --2- 172.21.15.76:0/3914492456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafc80b4880 0x7fafc8145110 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fafb800c7f0 tx=0x7fafb800ccb0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.197+0000 7fafccaf2640 1 -- 172.21.15.76:0/3914492456 shutdown_connections 2023-11-09T03:47:18.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.197+0000 7fafccaf2640 1 --2- 172.21.15.76:0/3914492456 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fafa806ae70 0x7fafa806d330 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.197+0000 7fafccaf2640 1 --2- 172.21.15.76:0/3914492456 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fafc8153e60 0x7fafc8156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.197+0000 7fafccaf2640 1 --2- 172.21.15.76:0/3914492456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafc80cd6d0 0x7fafc81525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.197+0000 7fafccaf2640 1 --2- 172.21.15.76:0/3914492456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafc80b4880 0x7fafc8145110 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.197+0000 7fafccaf2640 1 -- 172.21.15.76:0/3914492456 >> 172.21.15.76:0/3914492456 conn(0x7fafc80cf880 msgr2=0x7fafc815eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:18.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.198+0000 7fafccaf2640 1 -- 172.21.15.76:0/3914492456 shutdown_connections 2023-11-09T03:47:18.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.198+0000 7fafccaf2640 1 -- 172.21.15.76:0/3914492456 wait complete. 2023-11-09T03:47:18.209 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-09T03:47:18.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.310+0000 7efce86b4640 1 Processor -- start 2023-11-09T03:47:18.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.310+0000 7efce86b4640 1 -- start start 2023-11-09T03:47:18.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.311+0000 7efce86b4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efce00b0ae0 0x7efce00ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:18.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.311+0000 7efce86b4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efce006db20 0x7efce015d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:18.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.311+0000 7efce86b4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efce00b4110 con 0x7efce0159c70 2023-11-09T03:47:18.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.311+0000 7efce86b4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efce00b49b0 con 0x7efce00b0ae0 2023-11-09T03:47:18.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.311+0000 7efce86b4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce00b43f0 con 0x7efce006db20 2023-11-09T03:47:18.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.311+0000 7efce6c51640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efce006db20 0x7efce015d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:18.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.311+0000 7efce6c51640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efce006db20 0x7efce015d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:57658/0 (socket says 172.21.15.76:57658) 2023-11-09T03:47:18.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.311+0000 7efce5c4f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efce00b0ae0 0x7efce00ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:18.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.311+0000 7efce6c51640 1 -- 172.21.15.76:0/2380265469 learned_addr learned my addr 172.21.15.76:0/2380265469 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:18.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.311+0000 7efce6c51640 1 -- 172.21.15.76:0/2380265469 >> v2:172.21.15.113:3300/0 conn(0x7efce00b0ae0 msgr2=0x7efce00ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:18.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.311+0000 7efce6c51640 1 --2- 172.21.15.76:0/2380265469 >> v2:172.21.15.113:3300/0 conn(0x7efce00b0ae0 0x7efce00ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.311+0000 7efce6c51640 1 -- 172.21.15.76:0/2380265469 >> v1:172.21.15.113:6789/0 conn(0x7efce0159c70 legacy=0x7efce00b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:18.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.311+0000 7efce6c51640 1 -- 172.21.15.76:0/2380265469 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce00b46d0 con 0x7efce006db20 2023-11-09T03:47:18.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.312+0000 7efce6c51640 1 --2- 172.21.15.76:0/2380265469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efce006db20 0x7efce015d7d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efcd8009e50 tx=0x7efcd802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=602732c7da526f0a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:18.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.312+0000 7efce544e640 1 -- 172.21.15.76:0/2380265469 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd802cb00 con 0x7efce006db20 2023-11-09T03:47:18.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.312+0000 7efce544e640 1 -- 172.21.15.76:0/2380265469 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd802cc60 con 0x7efce006db20 2023-11-09T03:47:18.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.312+0000 7efce544e640 1 -- 172.21.15.76:0/2380265469 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd80343f0 con 0x7efce006db20 2023-11-09T03:47:18.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.313+0000 7efce86b4640 1 -- 172.21.15.76:0/2380265469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efce006db20 msgr2=0x7efce015d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:18.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.313+0000 7efce86b4640 1 --2- 172.21.15.76:0/2380265469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efce006db20 0x7efce015d7d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efcd8009e50 tx=0x7efcd802bf60 comp rx=0 tx=0).stop 2023-11-09T03:47:18.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.313+0000 7efce86b4640 1 -- 172.21.15.76:0/2380265469 shutdown_connections 2023-11-09T03:47:18.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.313+0000 7efce86b4640 1 --2- 172.21.15.76:0/2380265469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efce006db20 0x7efce015d7d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.313+0000 7efce86b4640 1 --2- 172.21.15.76:0/2380265469 >> v2:172.21.15.113:3300/0 conn(0x7efce00b0ae0 0x7efce00ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.313+0000 7efce86b4640 1 -- 172.21.15.76:0/2380265469 >> 172.21.15.76:0/2380265469 conn(0x7efce00ce950 msgr2=0x7efce00ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:18.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.313+0000 7efce86b4640 1 -- 172.21.15.76:0/2380265469 shutdown_connections 2023-11-09T03:47:18.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.313+0000 7efce86b4640 1 -- 172.21.15.76:0/2380265469 wait complete. 2023-11-09T03:47:18.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.314+0000 7efce86b4640 1 Processor -- start 2023-11-09T03:47:18.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.314+0000 7efce86b4640 1 -- start start 2023-11-09T03:47:18.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.315+0000 7efce86b4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efce006db20 0x7efce00c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:18.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.315+0000 7efce86b4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efce00b0ae0 0x7efce01550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:18.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.315+0000 7efce86b4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efce0159c70 0x7efce00c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:18.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.315+0000 7efce86b4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce015c420 con 0x7efce00b0ae0 2023-11-09T03:47:18.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.315+0000 7efce86b4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce01f1100 con 0x7efce0159c70 2023-11-09T03:47:18.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.315+0000 7efce86b4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce0155620 con 0x7efce006db20 2023-11-09T03:47:18.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.315+0000 7efce5c4f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efce00b0ae0 0x7efce01550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:18.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.315+0000 7efce5c4f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efce00b0ae0 0x7efce01550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60456/0 (socket says 172.21.15.76:60456) 2023-11-09T03:47:18.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.315+0000 7efce5c4f640 1 -- 172.21.15.76:0/2795590631 learned_addr learned my addr 172.21.15.76:0/2795590631 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:18.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.315+0000 7efce6450640 1 --2- 172.21.15.76:0/2795590631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efce006db20 0x7efce00c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:18.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.315+0000 7efce5c4f640 1 -- 172.21.15.76:0/2795590631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efce006db20 msgr2=0x7efce00c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:18.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.315+0000 7efce5c4f640 1 --2- 172.21.15.76:0/2795590631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efce006db20 0x7efce00c8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.315+0000 7efce6c51640 1 --2- 172.21.15.76:0/2795590631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efce0159c70 0x7efce00c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:18.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.315+0000 7efce5c4f640 1 -- 172.21.15.76:0/2795590631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efce0159c70 msgr2=0x7efce00c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:18.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.315+0000 7efce5c4f640 1 --2- 172.21.15.76:0/2795590631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efce0159c70 0x7efce00c6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.315+0000 7efce5c4f640 1 -- 172.21.15.76:0/2795590631 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd8009b30 con 0x7efce00b0ae0 2023-11-09T03:47:18.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.315+0000 7efce5c4f640 1 --2- 172.21.15.76:0/2795590631 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efce00b0ae0 0x7efce01550e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7efcd0010bc0 tx=0x7efcd000e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:18.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.316+0000 7efcd6ffd640 1 -- 172.21.15.76:0/2795590631 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0009670 con 0x7efce00b0ae0 2023-11-09T03:47:18.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.316+0000 7efcd6ffd640 1 -- 172.21.15.76:0/2795590631 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd0011650 con 0x7efce00b0ae0 2023-11-09T03:47:18.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.316+0000 7efce86b4640 1 -- 172.21.15.76:0/2795590631 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce01f04d0 con 0x7efce00b0ae0 2023-11-09T03:47:18.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.316+0000 7efcd6ffd640 1 -- 172.21.15.76:0/2795590631 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0011a00 con 0x7efce00b0ae0 2023-11-09T03:47:18.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.316+0000 7efce86b4640 1 -- 172.21.15.76:0/2795590631 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce01f0ab0 con 0x7efce00b0ae0 2023-11-09T03:47:18.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.317+0000 7efce86b4640 1 -- 172.21.15.76:0/2795590631 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce0074290 con 0x7efce00b0ae0 2023-11-09T03:47:18.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.317+0000 7efcd6ffd640 1 -- 172.21.15.76:0/2795590631 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efcd0011b60 con 0x7efce00b0ae0 2023-11-09T03:47:18.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.317+0000 7efcd6ffd640 1 --2- 172.21.15.76:0/2795590631 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efcc406ae70 0x7efcc406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:18.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.318+0000 7efcd6ffd640 1 -- 172.21.15.76:0/2795590631 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5530+0+0 (secure 0 0 0) 0x7efcd008ba60 con 0x7efce00b0ae0 2023-11-09T03:47:18.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.318+0000 7efce6450640 1 --2- 172.21.15.76:0/2795590631 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efcc406ae70 0x7efcc406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:18.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.318+0000 7efce6450640 1 --2- 172.21.15.76:0/2795590631 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efcc406ae70 0x7efcc406d330 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7efcdc001500 tx=0x7efcdc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:18.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.322+0000 7efcd6ffd640 1 -- 172.21.15.76:0/2795590631 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efcd0056bc0 con 0x7efce00b0ae0 2023-11-09T03:47:18.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.491+0000 7efce86b4640 1 -- 172.21.15.76:0/2795590631 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7efce01f1240 con 0x7efce00b0ae0 2023-11-09T03:47:18.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.491+0000 7efcd6ffd640 1 -- 172.21.15.76:0/2795590631 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v50) v1 ==== 84+0+137 (secure 0 0 0) 0x7efcd001e020 con 0x7efce00b0ae0 2023-11-09T03:47:18.492 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:18.493 INFO:teuthology.orchestra.run.smithi076.stdout:[ 2023-11-09T03:47:18.493 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:18.493 INFO:teuthology.orchestra.run.smithi076.stdout: "poolnum": 1, 2023-11-09T03:47:18.493 INFO:teuthology.orchestra.run.smithi076.stdout: "poolname": ".mgr" 2023-11-09T03:47:18.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:18.494 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:18.494 INFO:teuthology.orchestra.run.smithi076.stdout: "poolnum": 7, 2023-11-09T03:47:18.494 INFO:teuthology.orchestra.run.smithi076.stdout: "poolname": "snap-test-pool" 2023-11-09T03:47:18.495 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:18.495 INFO:teuthology.orchestra.run.smithi076.stdout:] 2023-11-09T03:47:18.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.493+0000 7efce86b4640 1 -- 172.21.15.76:0/2795590631 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efcc406ae70 msgr2=0x7efcc406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:18.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.493+0000 7efce86b4640 1 --2- 172.21.15.76:0/2795590631 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efcc406ae70 0x7efcc406d330 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7efcdc001500 tx=0x7efcdc009040 comp rx=0 tx=0).stop 2023-11-09T03:47:18.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.493+0000 7efce86b4640 1 -- 172.21.15.76:0/2795590631 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efce00b0ae0 msgr2=0x7efce01550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:18.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.493+0000 7efce86b4640 1 --2- 172.21.15.76:0/2795590631 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efce00b0ae0 0x7efce01550e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7efcd0010bc0 tx=0x7efcd000e730 comp rx=0 tx=0).stop 2023-11-09T03:47:18.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.493+0000 7efce86b4640 1 -- 172.21.15.76:0/2795590631 shutdown_connections 2023-11-09T03:47:18.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.493+0000 7efce86b4640 1 --2- 172.21.15.76:0/2795590631 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efcc406ae70 0x7efcc406d330 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.493+0000 7efce86b4640 1 --2- 172.21.15.76:0/2795590631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efce0159c70 0x7efce00c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.493+0000 7efce86b4640 1 --2- 172.21.15.76:0/2795590631 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efce00b0ae0 0x7efce01550e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.493+0000 7efce86b4640 1 --2- 172.21.15.76:0/2795590631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efce006db20 0x7efce00c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.493+0000 7efce86b4640 1 -- 172.21.15.76:0/2795590631 >> 172.21.15.76:0/2795590631 conn(0x7efce00ce950 msgr2=0x7efce00b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:18.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.494+0000 7efce86b4640 1 -- 172.21.15.76:0/2795590631 shutdown_connections 2023-11-09T03:47:18.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.494+0000 7efce86b4640 1 -- 172.21.15.76:0/2795590631 wait complete. 2023-11-09T03:47:18.505 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-09T03:47:18.505 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-09T03:47:18.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.610+0000 7f961cb97640 1 Processor -- start 2023-11-09T03:47:18.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.610+0000 7f961cb97640 1 -- start start 2023-11-09T03:47:18.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.611+0000 7f961cb97640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f961815f510 0x7f96180cb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:18.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.611+0000 7f961cb97640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f961815c0d0 con 0x7f961806dd50 2023-11-09T03:47:18.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.611+0000 7f961cb97640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f961806cf90 con 0x7f9618151ea0 2023-11-09T03:47:18.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.611+0000 7f961cb97640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f961806d270 con 0x7f961815f510 2023-11-09T03:47:18.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.611+0000 7f960ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f961815f510 0x7f96180cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:18.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.611+0000 7f9616d9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f961806dd50 0x7f96180c9010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57198/0 (socket says 172.21.15.76:57198) 2023-11-09T03:47:18.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.611+0000 7f9616d9d640 1 -- 172.21.15.76:0/4173236030 learned_addr learned my addr 172.21.15.76:0/4173236030 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:18.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.611+0000 7f960ffff640 1 -- 172.21.15.76:0/4173236030 >> v1:172.21.15.113:6789/0 conn(0x7f9618151ea0 legacy=0x7f961815b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:18.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.611+0000 7f960ffff640 1 -- 172.21.15.76:0/4173236030 >> v1:172.21.15.76:6789/0 conn(0x7f961806dd50 legacy=0x7f96180c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:18.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.611+0000 7f9615d9b640 1 -- 172.21.15.76:0/4173236030 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 977248224 0 0) 0x7f961815c0d0 con 0x7f961806dd50 2023-11-09T03:47:18.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.611+0000 7f960ffff640 1 -- 172.21.15.76:0/4173236030 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961806d550 con 0x7f961815f510 2023-11-09T03:47:18.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.612+0000 7f960ffff640 1 --2- 172.21.15.76:0/4173236030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f961815f510 0x7f96180cb150 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f96000097a0 tx=0x7f960002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86577d721a7dee4c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:18.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.612+0000 7f9615d9b640 1 -- 172.21.15.76:0/4173236030 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9600031830 con 0x7f961815f510 2023-11-09T03:47:18.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.612+0000 7f9615d9b640 1 -- 172.21.15.76:0/4173236030 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9600031990 con 0x7f961815f510 2023-11-09T03:47:18.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.612+0000 7f9615d9b640 1 -- 172.21.15.76:0/4173236030 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9600031d20 con 0x7f961815f510 2023-11-09T03:47:18.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.612+0000 7f961cb97640 1 -- 172.21.15.76:0/4173236030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f961815f510 msgr2=0x7f96180cb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:18.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.612+0000 7f961cb97640 1 --2- 172.21.15.76:0/4173236030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f961815f510 0x7f96180cb150 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f96000097a0 tx=0x7f960002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.613+0000 7f961cb97640 1 -- 172.21.15.76:0/4173236030 shutdown_connections 2023-11-09T03:47:18.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.613+0000 7f961cb97640 1 --2- 172.21.15.76:0/4173236030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f961815f510 0x7f96180cb150 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.613+0000 7f961cb97640 1 -- 172.21.15.76:0/4173236030 >> 172.21.15.76:0/4173236030 conn(0x7f96180b05d0 msgr2=0x7f96180b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:18.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.613+0000 7f961cb97640 1 -- 172.21.15.76:0/4173236030 shutdown_connections 2023-11-09T03:47:18.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.613+0000 7f961cb97640 1 -- 172.21.15.76:0/4173236030 wait complete. 2023-11-09T03:47:18.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.614+0000 7f961cb97640 1 Processor -- start 2023-11-09T03:47:18.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.614+0000 7f961cb97640 1 -- start start 2023-11-09T03:47:18.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.615+0000 7f961cb97640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f961806dd50 0x7f96180c3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:18.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.615+0000 7f961cb97640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9618151ea0 0x7f9618159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:18.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.615+0000 7f961cb97640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f961815f510 0x7f96181f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:18.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.615+0000 7f961cb97640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96180c6d70 con 0x7f961806dd50 2023-11-09T03:47:18.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.615+0000 7f961cb97640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96180c3d80 con 0x7f961815f510 2023-11-09T03:47:18.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.615+0000 7f961cb97640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96180c4000 con 0x7f9618151ea0 2023-11-09T03:47:18.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.615+0000 7f961659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f961806dd50 0x7f96180c3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:18.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.615+0000 7f961659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f961806dd50 0x7f96180c3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60472/0 (socket says 172.21.15.76:60472) 2023-11-09T03:47:18.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.615+0000 7f961659c640 1 -- 172.21.15.76:0/1096160304 learned_addr learned my addr 172.21.15.76:0/1096160304 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:18.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.615+0000 7f9616d9d640 1 --2- 172.21.15.76:0/1096160304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f961815f510 0x7f96181f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:18.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.615+0000 7f960ffff640 1 --2- 172.21.15.76:0/1096160304 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9618151ea0 0x7f9618159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:18.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.615+0000 7f961659c640 1 -- 172.21.15.76:0/1096160304 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9618151ea0 msgr2=0x7f9618159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:18.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.615+0000 7f961659c640 1 --2- 172.21.15.76:0/1096160304 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9618151ea0 0x7f9618159f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.615+0000 7f961659c640 1 -- 172.21.15.76:0/1096160304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f961815f510 msgr2=0x7f96181f38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:18.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.615+0000 7f961659c640 1 --2- 172.21.15.76:0/1096160304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f961815f510 0x7f96181f38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.615+0000 7f961659c640 1 -- 172.21.15.76:0/1096160304 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9600009400 con 0x7f961806dd50 2023-11-09T03:47:18.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.615+0000 7f9616d9d640 1 --2- 172.21.15.76:0/1096160304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f961815f510 0x7f96181f38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:18.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.615+0000 7f960ffff640 1 --2- 172.21.15.76:0/1096160304 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9618151ea0 0x7f9618159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:18.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.615+0000 7f961659c640 1 --2- 172.21.15.76:0/1096160304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f961806dd50 0x7f96180c3840 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f960800c430 tx=0x7f960800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:18.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.615+0000 7f960effd640 1 -- 172.21.15.76:0/1096160304 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9608011790 con 0x7f961806dd50 2023-11-09T03:47:18.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.615+0000 7f960effd640 1 -- 172.21.15.76:0/1096160304 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96080118f0 con 0x7f961806dd50 2023-11-09T03:47:18.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.615+0000 7f961cb97640 1 -- 172.21.15.76:0/1096160304 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96180c42e0 con 0x7f961806dd50 2023-11-09T03:47:18.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.615+0000 7f960effd640 1 -- 172.21.15.76:0/1096160304 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9608011ce0 con 0x7f961806dd50 2023-11-09T03:47:18.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.615+0000 7f961cb97640 1 -- 172.21.15.76:0/1096160304 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96180c48a0 con 0x7f961806dd50 2023-11-09T03:47:18.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.616+0000 7f961cb97640 1 -- 172.21.15.76:0/1096160304 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96180c6f30 con 0x7f961806dd50 2023-11-09T03:47:18.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.617+0000 7f960effd640 1 -- 172.21.15.76:0/1096160304 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f960801c460 con 0x7f961806dd50 2023-11-09T03:47:18.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.617+0000 7f960effd640 1 --2- 172.21.15.76:0/1096160304 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f95f806aea0 0x7f95f806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:18.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.617+0000 7f960effd640 1 -- 172.21.15.76:0/1096160304 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f96080894f0 con 0x7f961806dd50 2023-11-09T03:47:18.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.617+0000 7f960ffff640 1 --2- 172.21.15.76:0/1096160304 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f95f806aea0 0x7f95f806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:18.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.618+0000 7f960ffff640 1 --2- 172.21.15.76:0/1096160304 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f95f806aea0 0x7f95f806d360 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9600036000 tx=0x7f960003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:18.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.621+0000 7f960effd640 1 -- 172.21.15.76:0/1096160304 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f96180c6f30 con 0x7f961806dd50 2023-11-09T03:47:18.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.787+0000 7f961cb97640 1 -- 172.21.15.76:0/1096160304 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f96180c4a90 con 0x7f961806dd50 2023-11-09T03:47:18.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.913+0000 7f960effd640 1 -- 172.21.15.76:0/1096160304 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v51) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9608054650 con 0x7f961806dd50 2023-11-09T03:47:18.914 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' created 2023-11-09T03:47:18.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.916+0000 7f961cb97640 1 -- 172.21.15.76:0/1096160304 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f95f806aea0 msgr2=0x7f95f806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:18.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.916+0000 7f961cb97640 1 --2- 172.21.15.76:0/1096160304 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f95f806aea0 0x7f95f806d360 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9600036000 tx=0x7f960003c040 comp rx=0 tx=0).stop 2023-11-09T03:47:18.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.916+0000 7f961cb97640 1 -- 172.21.15.76:0/1096160304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f961806dd50 msgr2=0x7f96180c3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:18.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.916+0000 7f961cb97640 1 --2- 172.21.15.76:0/1096160304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f961806dd50 0x7f96180c3840 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f960800c430 tx=0x7f960800c8f0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.916+0000 7f961cb97640 1 -- 172.21.15.76:0/1096160304 shutdown_connections 2023-11-09T03:47:18.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.916+0000 7f961cb97640 1 --2- 172.21.15.76:0/1096160304 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f95f806aea0 0x7f95f806d360 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.916+0000 7f961cb97640 1 --2- 172.21.15.76:0/1096160304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f961815f510 0x7f96181f38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.916+0000 7f961cb97640 1 --2- 172.21.15.76:0/1096160304 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9618151ea0 0x7f9618159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.916+0000 7f961cb97640 1 --2- 172.21.15.76:0/1096160304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f961806dd50 0x7f96180c3840 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:18.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.916+0000 7f961cb97640 1 -- 172.21.15.76:0/1096160304 >> 172.21.15.76:0/1096160304 conn(0x7f96180b05d0 msgr2=0x7f961815b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:18.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.916+0000 7f961cb97640 1 -- 172.21.15.76:0/1096160304 shutdown_connections 2023-11-09T03:47:18.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:18.916+0000 7f961cb97640 1 -- 172.21.15.76:0/1096160304 wait complete. 2023-11-09T03:47:18.930 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-09T03:47:19.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.037+0000 7fe160c64640 1 Processor -- start 2023-11-09T03:47:19.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.037+0000 7fe160c64640 1 -- start start 2023-11-09T03:47:19.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.037+0000 7fe160c64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe15c159c00 0x7fe15c0cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:19.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.037+0000 7fe160c64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe15c0ae1d0 0x7fe15c0c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:19.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.037+0000 7fe160c64640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe15c15aff0 con 0x7fe15c0c48e0 2023-11-09T03:47:19.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.038+0000 7fe160c64640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe15c0c52d0 con 0x7fe15c159c00 2023-11-09T03:47:19.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.038+0000 7fe160c64640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c15a3e0 con 0x7fe15c0ae1d0 2023-11-09T03:47:19.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.038+0000 7fe15ad9d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe15c0ae1d0 0x7fe15c0c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:19.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.038+0000 7fe15a59c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe15c159c00 0x7fe15c0cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:19.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.038+0000 7fe151d9b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe15c0c48e0 0x7fe15c0adac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:53100/0 (socket says 172.21.15.76:53100) 2023-11-09T03:47:19.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.038+0000 7fe15ad9d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe15c0ae1d0 0x7fe15c0c4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:57694/0 (socket says 172.21.15.76:57694) 2023-11-09T03:47:19.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.038+0000 7fe151d9b640 1 -- 172.21.15.76:0/2399915114 learned_addr learned my addr 172.21.15.76:0/2399915114 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:19.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.038+0000 7fe159d9b640 1 -- 172.21.15.76:0/2399915114 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1233634309 0 0) 0x7fe15c15aff0 con 0x7fe15c0c48e0 2023-11-09T03:47:19.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.038+0000 7fe159d9b640 1 -- 172.21.15.76:0/2399915114 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe148003520 con 0x7fe15c0c48e0 2023-11-09T03:47:19.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.038+0000 7fe15ad9d640 1 -- 172.21.15.76:0/2399915114 >> v2:172.21.15.113:3300/0 conn(0x7fe15c159c00 msgr2=0x7fe15c0cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:19.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.038+0000 7fe15ad9d640 1 --2- 172.21.15.76:0/2399915114 >> v2:172.21.15.113:3300/0 conn(0x7fe15c159c00 0x7fe15c0cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:19.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.038+0000 7fe15ad9d640 1 -- 172.21.15.76:0/2399915114 >> v1:172.21.15.113:6789/0 conn(0x7fe15c0c48e0 legacy=0x7fe15c0adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:19.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.038+0000 7fe15ad9d640 1 -- 172.21.15.76:0/2399915114 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15c15a6c0 con 0x7fe15c0ae1d0 2023-11-09T03:47:19.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.038+0000 7fe15a59c640 1 --2- 172.21.15.76:0/2399915114 >> v2:172.21.15.113:3300/0 conn(0x7fe15c159c00 0x7fe15c0cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:47:19.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.039+0000 7fe15ad9d640 1 --2- 172.21.15.76:0/2399915114 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe15c0ae1d0 0x7fe15c0c4d90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe154009d90 tx=0x7fe15402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4730d4fa111acde3 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:19.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.039+0000 7fe159d9b640 1 -- 172.21.15.76:0/2399915114 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15402cbc0 con 0x7fe15c0ae1d0 2023-11-09T03:47:19.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.039+0000 7fe159d9b640 1 -- 172.21.15.76:0/2399915114 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15402cd20 con 0x7fe15c0ae1d0 2023-11-09T03:47:19.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.039+0000 7fe159d9b640 1 -- 172.21.15.76:0/2399915114 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe154034640 con 0x7fe15c0ae1d0 2023-11-09T03:47:19.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.040+0000 7fe160c64640 1 -- 172.21.15.76:0/2399915114 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe15c0ae1d0 msgr2=0x7fe15c0c4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:19.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.040+0000 7fe160c64640 1 --2- 172.21.15.76:0/2399915114 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe15c0ae1d0 0x7fe15c0c4d90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe154009d90 tx=0x7fe15402be10 comp rx=0 tx=0).stop 2023-11-09T03:47:19.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.040+0000 7fe160c64640 1 -- 172.21.15.76:0/2399915114 shutdown_connections 2023-11-09T03:47:19.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.040+0000 7fe160c64640 1 --2- 172.21.15.76:0/2399915114 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe15c0ae1d0 0x7fe15c0c4d90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:19.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.040+0000 7fe160c64640 1 --2- 172.21.15.76:0/2399915114 >> v2:172.21.15.113:3300/0 conn(0x7fe15c159c00 0x7fe15c0cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:19.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.040+0000 7fe160c64640 1 -- 172.21.15.76:0/2399915114 >> 172.21.15.76:0/2399915114 conn(0x7fe15c0ce950 msgr2=0x7fe15c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:19.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.040+0000 7fe160c64640 1 -- 172.21.15.76:0/2399915114 shutdown_connections 2023-11-09T03:47:19.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.040+0000 7fe160c64640 1 -- 172.21.15.76:0/2399915114 wait complete. 2023-11-09T03:47:19.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.041+0000 7fe160c64640 1 Processor -- start 2023-11-09T03:47:19.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.041+0000 7fe160c64640 1 -- start start 2023-11-09T03:47:19.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.042+0000 7fe160c64640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe15c0ae1d0 0x7fe15c156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:19.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.042+0000 7fe160c64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe15c0c48e0 0x7fe15c0c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:19.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.042+0000 7fe160c64640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe15c159c00 0x7fe15c151890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:19.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.042+0000 7fe160c64640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c155980 con 0x7fe15c0ae1d0 2023-11-09T03:47:19.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.042+0000 7fe160c64640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c155c60 con 0x7fe15c159c00 2023-11-09T03:47:19.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.042+0000 7fe160c64640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c155f40 con 0x7fe15c0c48e0 2023-11-09T03:47:19.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.042+0000 7fe15a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe15c0ae1d0 0x7fe15c156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:19.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.042+0000 7fe15a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe15c0ae1d0 0x7fe15c156a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60480/0 (socket says 172.21.15.76:60480) 2023-11-09T03:47:19.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.042+0000 7fe15a59c640 1 -- 172.21.15.76:0/1477219508 learned_addr learned my addr 172.21.15.76:0/1477219508 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:19.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.042+0000 7fe151d9b640 1 --2- 172.21.15.76:0/1477219508 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe15c0c48e0 0x7fe15c0c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:19.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.042+0000 7fe15ad9d640 1 --2- 172.21.15.76:0/1477219508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe15c159c00 0x7fe15c151890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:19.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.042+0000 7fe15a59c640 1 -- 172.21.15.76:0/1477219508 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe15c0c48e0 msgr2=0x7fe15c0c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:19.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.042+0000 7fe15a59c640 1 --2- 172.21.15.76:0/1477219508 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe15c0c48e0 0x7fe15c0c7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:19.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.042+0000 7fe15a59c640 1 -- 172.21.15.76:0/1477219508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe15c159c00 msgr2=0x7fe15c151890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:19.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.042+0000 7fe15a59c640 1 --2- 172.21.15.76:0/1477219508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe15c159c00 0x7fe15c151890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:19.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.042+0000 7fe15a59c640 1 -- 172.21.15.76:0/1477219508 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14c009400 con 0x7fe15c0ae1d0 2023-11-09T03:47:19.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.043+0000 7fe15a59c640 1 --2- 172.21.15.76:0/1477219508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe15c0ae1d0 0x7fe15c156a00 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe14c00e960 tx=0x7fe14c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:19.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.043+0000 7fe1537fe640 1 -- 172.21.15.76:0/1477219508 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c00ca00 con 0x7fe15c0ae1d0 2023-11-09T03:47:19.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.043+0000 7fe1537fe640 1 -- 172.21.15.76:0/1477219508 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14c00cb60 con 0x7fe15c0ae1d0 2023-11-09T03:47:19.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.043+0000 7fe160c64640 1 -- 172.21.15.76:0/1477219508 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe154009a70 con 0x7fe15c0ae1d0 2023-11-09T03:47:19.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.043+0000 7fe1537fe640 1 -- 172.21.15.76:0/1477219508 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c013470 con 0x7fe15c0ae1d0 2023-11-09T03:47:19.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.043+0000 7fe160c64640 1 -- 172.21.15.76:0/1477219508 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe15c156500 con 0x7fe15c0ae1d0 2023-11-09T03:47:19.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.044+0000 7fe160c64640 1 -- 172.21.15.76:0/1477219508 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe15c074290 con 0x7fe15c0ae1d0 2023-11-09T03:47:19.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.044+0000 7fe1537fe640 1 -- 172.21.15.76:0/1477219508 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe14c0135d0 con 0x7fe15c0ae1d0 2023-11-09T03:47:19.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.045+0000 7fe1537fe640 1 --2- 172.21.15.76:0/1477219508 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe13c06ae70 0x7fe13c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:19.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.045+0000 7fe1537fe640 1 -- 172.21.15.76:0/1477219508 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5900+0+0 (secure 0 0 0) 0x7fe14c089830 con 0x7fe15c0ae1d0 2023-11-09T03:47:19.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.045+0000 7fe151d9b640 1 --2- 172.21.15.76:0/1477219508 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe13c06ae70 0x7fe13c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:19.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.046+0000 7fe151d9b640 1 --2- 172.21.15.76:0/1477219508 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe13c06ae70 0x7fe13c06d330 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe15c1557e0 tx=0x7fe144009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:19.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.049+0000 7fe1537fe640 1 -- 172.21.15.76:0/1477219508 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe14c0548e0 con 0x7fe15c0ae1d0 2023-11-09T03:47:19.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.216+0000 7fe160c64640 1 -- 172.21.15.76:0/1477219508 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe15c15c9a0 con 0x7fe15c0ae1d0 2023-11-09T03:47:19.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.923+0000 7fe1537fe640 1 -- 172.21.15.76:0/1477219508 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v52) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe14c057f00 con 0x7fe15c0ae1d0 2023-11-09T03:47:19.924 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' created 2023-11-09T03:47:19.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.925+0000 7fe160c64640 1 -- 172.21.15.76:0/1477219508 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe13c06ae70 msgr2=0x7fe13c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:19.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.925+0000 7fe160c64640 1 --2- 172.21.15.76:0/1477219508 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe13c06ae70 0x7fe13c06d330 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe15c1557e0 tx=0x7fe144009040 comp rx=0 tx=0).stop 2023-11-09T03:47:19.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.925+0000 7fe160c64640 1 -- 172.21.15.76:0/1477219508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe15c0ae1d0 msgr2=0x7fe15c156a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:19.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.925+0000 7fe160c64640 1 --2- 172.21.15.76:0/1477219508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe15c0ae1d0 0x7fe15c156a00 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe14c00e960 tx=0x7fe14c00ee20 comp rx=0 tx=0).stop 2023-11-09T03:47:19.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.925+0000 7fe160c64640 1 -- 172.21.15.76:0/1477219508 shutdown_connections 2023-11-09T03:47:19.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.925+0000 7fe160c64640 1 --2- 172.21.15.76:0/1477219508 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe13c06ae70 0x7fe13c06d330 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:19.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.925+0000 7fe160c64640 1 --2- 172.21.15.76:0/1477219508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe15c159c00 0x7fe15c151890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:19.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.925+0000 7fe160c64640 1 --2- 172.21.15.76:0/1477219508 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe15c0c48e0 0x7fe15c0c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:19.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.925+0000 7fe160c64640 1 --2- 172.21.15.76:0/1477219508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe15c0ae1d0 0x7fe15c156a00 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:19.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.925+0000 7fe160c64640 1 -- 172.21.15.76:0/1477219508 >> 172.21.15.76:0/1477219508 conn(0x7fe15c0ce950 msgr2=0x7fe15c154510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:19.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.925+0000 7fe160c64640 1 -- 172.21.15.76:0/1477219508 shutdown_connections 2023-11-09T03:47:19.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:19.925+0000 7fe160c64640 1 -- 172.21.15.76:0/1477219508 wait complete. 2023-11-09T03:47:19.945 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-09T03:47:20.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.050+0000 7fefcdc87640 1 Processor -- start 2023-11-09T03:47:20.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.050+0000 7fefcdc87640 1 -- start start 2023-11-09T03:47:20.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.051+0000 7fefcdc87640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fefc80cdc20 0x7fefc8165950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:20.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.051+0000 7fefcdc87640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefc80d0280 con 0x7fefc815e430 2023-11-09T03:47:20.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.051+0000 7fefcdc87640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefc80cf670 con 0x7fefc80d0710 2023-11-09T03:47:20.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.051+0000 7fefcdc87640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefc80cf950 con 0x7fefc80cdc20 2023-11-09T03:47:20.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.051+0000 7fefc7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fefc80cdc20 0x7fefc8165950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:20.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.051+0000 7fefc77fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fefc815e430 0x7fefc80cec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57202/0 (socket says 172.21.15.76:57202) 2023-11-09T03:47:20.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.051+0000 7fefc77fe640 1 -- 172.21.15.76:0/3093831910 learned_addr learned my addr 172.21.15.76:0/3093831910 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:20.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.052+0000 7fefc7fff640 1 -- 172.21.15.76:0/3093831910 >> v1:172.21.15.113:6789/0 conn(0x7fefc80d0710 legacy=0x7fefc80cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:20.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.052+0000 7fefc7fff640 1 -- 172.21.15.76:0/3093831910 >> v1:172.21.15.76:6789/0 conn(0x7fefc815e430 legacy=0x7fefc80cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:20.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.052+0000 7fefc7fff640 1 -- 172.21.15.76:0/3093831910 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc80cfc30 con 0x7fefc80cdc20 2023-11-09T03:47:20.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.052+0000 7fefc6ffd640 1 -- 172.21.15.76:0/3093831910 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897325096 0 0) 0x7fefc80d0280 con 0x7fefc815e430 2023-11-09T03:47:20.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.052+0000 7fefc7fff640 1 --2- 172.21.15.76:0/3093831910 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefc80cdc20 0x7fefc8165950 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fefc0009d90 tx=0x7fefc002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76111dd6e3396680 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:20.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.053+0000 7fefc6ffd640 1 -- 172.21.15.76:0/3093831910 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc002cc60 con 0x7fefc80cdc20 2023-11-09T03:47:20.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.053+0000 7fefc6ffd640 1 -- 172.21.15.76:0/3093831910 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc002cdc0 con 0x7fefc80cdc20 2023-11-09T03:47:20.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.053+0000 7fefc6ffd640 1 -- 172.21.15.76:0/3093831910 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc0034660 con 0x7fefc80cdc20 2023-11-09T03:47:20.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.053+0000 7fefcdc87640 1 -- 172.21.15.76:0/3093831910 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefc80cdc20 msgr2=0x7fefc8165950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:20.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.053+0000 7fefcdc87640 1 --2- 172.21.15.76:0/3093831910 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefc80cdc20 0x7fefc8165950 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fefc0009d90 tx=0x7fefc002beb0 comp rx=0 tx=0).stop 2023-11-09T03:47:20.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.053+0000 7fefcdc87640 1 -- 172.21.15.76:0/3093831910 shutdown_connections 2023-11-09T03:47:20.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.053+0000 7fefcdc87640 1 --2- 172.21.15.76:0/3093831910 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefc80cdc20 0x7fefc8165950 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:20.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.053+0000 7fefcdc87640 1 -- 172.21.15.76:0/3093831910 >> 172.21.15.76:0/3093831910 conn(0x7fefc806ea20 msgr2=0x7fefc806ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:20.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.053+0000 7fefcdc87640 1 -- 172.21.15.76:0/3093831910 shutdown_connections 2023-11-09T03:47:20.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.053+0000 7fefcdc87640 1 -- 172.21.15.76:0/3093831910 wait complete. 2023-11-09T03:47:20.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.054+0000 7fefcdc87640 1 Processor -- start 2023-11-09T03:47:20.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.054+0000 7fefcdc87640 1 -- start start 2023-11-09T03:47:20.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.055+0000 7fefcdc87640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefc80cdc20 0x7fefc80b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:20.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.055+0000 7fefcdc87640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefc80d0710 0x7fefc806c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:20.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.055+0000 7fefcdc87640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fefc815e430 0x7fefc8159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:20.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.055+0000 7fefcdc87640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc80c8280 con 0x7fefc80d0710 2023-11-09T03:47:20.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.055+0000 7fefcdc87640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc80c7c70 con 0x7fefc80cdc20 2023-11-09T03:47:20.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.055+0000 7fefcdc87640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefc80b5190 con 0x7fefc815e430 2023-11-09T03:47:20.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.055+0000 7fefbeffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefc80d0710 0x7fefc806c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:20.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.055+0000 7fefbeffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefc80d0710 0x7fefc806c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60488/0 (socket says 172.21.15.76:60488) 2023-11-09T03:47:20.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.055+0000 7fefbeffd640 1 -- 172.21.15.76:0/1324507408 learned_addr learned my addr 172.21.15.76:0/1324507408 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:20.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.055+0000 7fefc77fe640 1 --2- 172.21.15.76:0/1324507408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefc80cdc20 0x7fefc80b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:20.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.056+0000 7fefc7fff640 1 --2- 172.21.15.76:0/1324507408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fefc815e430 0x7fefc8159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:20.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.056+0000 7fefbeffd640 1 -- 172.21.15.76:0/1324507408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fefc815e430 msgr2=0x7fefc8159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:20.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.056+0000 7fefbeffd640 1 --2- 172.21.15.76:0/1324507408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fefc815e430 0x7fefc8159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:20.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.056+0000 7fefbeffd640 1 -- 172.21.15.76:0/1324507408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefc80cdc20 msgr2=0x7fefc80b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:20.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.056+0000 7fefbeffd640 1 --2- 172.21.15.76:0/1324507408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefc80cdc20 0x7fefc80b4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:20.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.056+0000 7fefbeffd640 1 -- 172.21.15.76:0/1324507408 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc0009a70 con 0x7fefc80d0710 2023-11-09T03:47:20.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.056+0000 7fefc77fe640 1 --2- 172.21.15.76:0/1324507408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefc80cdc20 0x7fefc80b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:20.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.056+0000 7fefbeffd640 1 --2- 172.21.15.76:0/1324507408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefc80d0710 0x7fefc806c340 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fefb000c430 tx=0x7fefb000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:20.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.056+0000 7fefc4ff9640 1 -- 172.21.15.76:0/1324507408 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb0011790 con 0x7fefc80d0710 2023-11-09T03:47:20.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.056+0000 7fefc4ff9640 1 -- 172.21.15.76:0/1324507408 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb00118f0 con 0x7fefc80d0710 2023-11-09T03:47:20.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.056+0000 7fefc4ff9640 1 -- 172.21.15.76:0/1324507408 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb0011ce0 con 0x7fefc80d0710 2023-11-09T03:47:20.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.056+0000 7fefcdc87640 1 -- 172.21.15.76:0/1324507408 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefc80b5470 con 0x7fefc80d0710 2023-11-09T03:47:20.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.056+0000 7fefcdc87640 1 -- 172.21.15.76:0/1324507408 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefc80af190 con 0x7fefc80d0710 2023-11-09T03:47:20.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.057+0000 7fefcdc87640 1 -- 172.21.15.76:0/1324507408 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefc80c8430 con 0x7fefc80d0710 2023-11-09T03:47:20.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.057+0000 7fefc4ff9640 1 -- 172.21.15.76:0/1324507408 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fefb001c460 con 0x7fefc80d0710 2023-11-09T03:47:20.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.058+0000 7fefc4ff9640 1 --2- 172.21.15.76:0/1324507408 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fefa806ae70 0x7fefa806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:20.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.058+0000 7fefc4ff9640 1 -- 172.21.15.76:0/1324507408 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6282+0+0 (secure 0 0 0) 0x7fefb0089800 con 0x7fefc80d0710 2023-11-09T03:47:20.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.058+0000 7fefc77fe640 1 --2- 172.21.15.76:0/1324507408 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fefa806ae70 0x7fefa806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:20.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.058+0000 7fefc77fe640 1 --2- 172.21.15.76:0/1324507408 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fefa806ae70 0x7fefa806d330 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fefb8000f60 tx=0x7fefb8007ca0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:20.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.062+0000 7fefc4ff9640 1 -- 172.21.15.76:0/1324507408 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fefc80c8430 con 0x7fefc80d0710 2023-11-09T03:47:20.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.262+0000 7fefcdc87640 1 -- 172.21.15.76:0/1324507408 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fefc8074290 con 0x7fefc80d0710 2023-11-09T03:47:20.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.935+0000 7fefc4ff9640 1 -- 172.21.15.76:0/1324507408 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v20) v1 ==== 172+0+0 (secure 0 0 0) 0x7fefb0054670 con 0x7fefc80d0710 2023-11-09T03:47:20.935 INFO:teuthology.orchestra.run.smithi076.stderr:new fs with metadata pool 8 and data pool 9 2023-11-09T03:47:20.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.939+0000 7fefcdc87640 1 -- 172.21.15.76:0/1324507408 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fefa806ae70 msgr2=0x7fefa806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:20.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.939+0000 7fefcdc87640 1 --2- 172.21.15.76:0/1324507408 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fefa806ae70 0x7fefa806d330 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fefb8000f60 tx=0x7fefb8007ca0 comp rx=0 tx=0).stop 2023-11-09T03:47:20.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.939+0000 7fefcdc87640 1 -- 172.21.15.76:0/1324507408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefc80d0710 msgr2=0x7fefc806c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:20.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.939+0000 7fefcdc87640 1 --2- 172.21.15.76:0/1324507408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefc80d0710 0x7fefc806c340 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fefb000c430 tx=0x7fefb000c8f0 comp rx=0 tx=0).stop 2023-11-09T03:47:20.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.945+0000 7fefcdc87640 1 -- 172.21.15.76:0/1324507408 shutdown_connections 2023-11-09T03:47:20.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.945+0000 7fefcdc87640 1 --2- 172.21.15.76:0/1324507408 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fefa806ae70 0x7fefa806d330 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:20.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.945+0000 7fefcdc87640 1 --2- 172.21.15.76:0/1324507408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fefc815e430 0x7fefc8159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:20.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.945+0000 7fefcdc87640 1 --2- 172.21.15.76:0/1324507408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefc80d0710 0x7fefc806c340 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:20.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.945+0000 7fefcdc87640 1 --2- 172.21.15.76:0/1324507408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefc80cdc20 0x7fefc80b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:20.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.945+0000 7fefcdc87640 1 -- 172.21.15.76:0/1324507408 >> 172.21.15.76:0/1324507408 conn(0x7fefc806ea20 msgr2=0x7fefc80c6680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:20.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.948+0000 7fefcdc87640 1 -- 172.21.15.76:0/1324507408 shutdown_connections 2023-11-09T03:47:20.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:20.948+0000 7fefcdc87640 1 -- 172.21.15.76:0/1324507408 wait complete. 2023-11-09T03:47:20.964 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:47:21.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.066+0000 7fb1f816c640 1 Processor -- start 2023-11-09T03:47:21.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.066+0000 7fb1f816c640 1 -- start start 2023-11-09T03:47:21.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.067+0000 7fb1f816c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1f00ce240 0x7fb1f0157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:21.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.067+0000 7fb1f816c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1f015e000 con 0x7fb1f0157540 2023-11-09T03:47:21.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.067+0000 7fb1f816c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1f015e2e0 con 0x7fb1f00cde60 2023-11-09T03:47:21.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.067+0000 7fb1f816c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f015e5c0 con 0x7fb1f00ce240 2023-11-09T03:47:21.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.067+0000 7fb1f6709640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb1f0157540 0x7fb1f00d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57218/0 (socket says 172.21.15.76:57218) 2023-11-09T03:47:21.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.067+0000 7fb1f6709640 1 -- 172.21.15.76:0/906793565 learned_addr learned my addr 172.21.15.76:0/906793565 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:21.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.067+0000 7fb1f5707640 1 --2- 172.21.15.76:0/906793565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1f00ce240 0x7fb1f0157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:21.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.067+0000 7fb1f4f06640 1 -- 172.21.15.76:0/906793565 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 513004909 0 0) 0x7fb1f015e000 con 0x7fb1f0157540 2023-11-09T03:47:21.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.067+0000 7fb1f4f06640 1 -- 172.21.15.76:0/906793565 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1dc003520 con 0x7fb1f0157540 2023-11-09T03:47:21.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.067+0000 7fb1f4f06640 1 -- 172.21.15.76:0/906793565 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 74128199 0 0) 0x7fb1f015e2e0 con 0x7fb1f00cde60 2023-11-09T03:47:21.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.067+0000 7fb1f4f06640 1 -- 172.21.15.76:0/906793565 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1f015e000 con 0x7fb1f00cde60 2023-11-09T03:47:21.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.067+0000 7fb1f4f06640 1 -- 172.21.15.76:0/906793565 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1967745867 0 0) 0x7fb1dc003520 con 0x7fb1f0157540 2023-11-09T03:47:21.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.067+0000 7fb1f4f06640 1 -- 172.21.15.76:0/906793565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1f00ce240 msgr2=0x7fb1f0157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:21.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.067+0000 7fb1f4f06640 1 --2- 172.21.15.76:0/906793565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1f00ce240 0x7fb1f0157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:21.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.067+0000 7fb1f4f06640 1 -- 172.21.15.76:0/906793565 >> v1:172.21.15.113:6789/0 conn(0x7fb1f00cde60 legacy=0x7fb1f015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:21.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.067+0000 7fb1f4f06640 1 -- 172.21.15.76:0/906793565 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f00d1120 con 0x7fb1f0157540 2023-11-09T03:47:21.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.067+0000 7fb1f4f06640 1 -- 172.21.15.76:0/906793565 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fb1e80019f0 con 0x7fb1f0157540 2023-11-09T03:47:21.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.067+0000 7fb1f4f06640 1 -- 172.21.15.76:0/906793565 >> v1:172.21.15.76:6789/0 conn(0x7fb1f0157540 legacy=0x7fb1f00d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:21.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.067+0000 7fb1f4f06640 1 --2- 172.21.15.76:0/906793565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1dc003b30 0x7fb1dc003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:21.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.067+0000 7fb1f4f06640 1 --2- 172.21.15.76:0/906793565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1dc0047c0 0x7fb1dc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:21.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.067+0000 7fb1f4f06640 1 --2- 172.21.15.76:0/906793565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1dc005180 0x7fb1dc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:21.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.067+0000 7fb1f4f06640 1 -- 172.21.15.76:0/906793565 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f015e5c0 con 0x7fb1dc0047c0 2023-11-09T03:47:21.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.067+0000 7fb1f4f06640 1 -- 172.21.15.76:0/906793565 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc00bf70 con 0x7fb1dc003b30 2023-11-09T03:47:21.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.067+0000 7fb1f4f06640 1 -- 172.21.15.76:0/906793565 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc00c130 con 0x7fb1dc005180 2023-11-09T03:47:21.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.067+0000 7fb1f5707640 1 --2- 172.21.15.76:0/906793565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1f00ce240 0x7fb1f0157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:47:21.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.068+0000 7fb1f5707640 1 --2- 172.21.15.76:0/906793565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1dc0047c0 0x7fb1dc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:21.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.068+0000 7fb1f5f08640 1 --2- 172.21.15.76:0/906793565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1dc003b30 0x7fb1dc003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:21.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.068+0000 7fb1f6709640 1 --2- 172.21.15.76:0/906793565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1dc005180 0x7fb1dc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:21.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.068+0000 7fb1f5707640 1 -- 172.21.15.76:0/906793565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1dc005180 msgr2=0x7fb1dc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:21.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.068+0000 7fb1f5707640 1 --2- 172.21.15.76:0/906793565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1dc005180 0x7fb1dc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:21.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.068+0000 7fb1f5707640 1 -- 172.21.15.76:0/906793565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1dc003b30 msgr2=0x7fb1dc003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:21.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.068+0000 7fb1f5707640 1 --2- 172.21.15.76:0/906793565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1dc003b30 0x7fb1dc003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:21.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.068+0000 7fb1f5707640 1 -- 172.21.15.76:0/906793565 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1dc003520 con 0x7fb1dc0047c0 2023-11-09T03:47:21.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.069+0000 7fb1f5707640 1 --2- 172.21.15.76:0/906793565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1dc0047c0 0x7fb1dc004c40 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0009b00 tx=0x7fb1dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ae85c640eab5d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:21.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.069+0000 7fb1f4f06640 1 -- 172.21.15.76:0/906793565 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0004030 con 0x7fb1dc0047c0 2023-11-09T03:47:21.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.069+0000 7fb1f4f06640 1 -- 172.21.15.76:0/906793565 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e0004190 con 0x7fb1dc0047c0 2023-11-09T03:47:21.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.069+0000 7fb1f4f06640 1 -- 172.21.15.76:0/906793565 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0004520 con 0x7fb1dc0047c0 2023-11-09T03:47:21.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.069+0000 7fb1f816c640 1 -- 172.21.15.76:0/906793565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1dc0047c0 msgr2=0x7fb1dc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:21.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.069+0000 7fb1f816c640 1 --2- 172.21.15.76:0/906793565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1dc0047c0 0x7fb1dc004c40 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0009b00 tx=0x7fb1dc0057f0 comp rx=0 tx=0).stop 2023-11-09T03:47:21.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.069+0000 7fb1f6709640 1 -- 172.21.15.76:0/906793565 reap_dead start 2023-11-09T03:47:21.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.069+0000 7fb1f816c640 1 -- 172.21.15.76:0/906793565 shutdown_connections 2023-11-09T03:47:21.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.069+0000 7fb1f816c640 1 -- 172.21.15.76:0/906793565 >> 172.21.15.76:0/906793565 conn(0x7fb1f0071f30 msgr2=0x7fb1f00cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:21.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.069+0000 7fb1f816c640 1 -- 172.21.15.76:0/906793565 shutdown_connections 2023-11-09T03:47:21.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.069+0000 7fb1f816c640 1 -- 172.21.15.76:0/906793565 wait complete. 2023-11-09T03:47:21.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.070+0000 7fb1f816c640 1 Processor -- start 2023-11-09T03:47:21.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.070+0000 7fb1f816c640 1 -- start start 2023-11-09T03:47:21.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.071+0000 7fb1f816c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1dc0047c0 0x7fb1f01534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:21.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.071+0000 7fb1f816c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1f0157b30 0x7fb1f0159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:21.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.071+0000 7fb1f816c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1f01595a0 0x7fb1f01f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:21.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.071+0000 7fb1f816c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f00ccd50 con 0x7fb1dc0047c0 2023-11-09T03:47:21.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.071+0000 7fb1f816c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f0157fc0 con 0x7fb1f0157b30 2023-11-09T03:47:21.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.071+0000 7fb1f816c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f01582a0 con 0x7fb1f01595a0 2023-11-09T03:47:21.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.071+0000 7fb1f5f08640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1dc0047c0 0x7fb1f01534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:21.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.071+0000 7fb1f5f08640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1dc0047c0 0x7fb1f01534e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60502/0 (socket says 172.21.15.76:60502) 2023-11-09T03:47:21.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.071+0000 7fb1f5f08640 1 -- 172.21.15.76:0/2722436533 learned_addr learned my addr 172.21.15.76:0/2722436533 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:21.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.071+0000 7fb1f6709640 1 --2- 172.21.15.76:0/2722436533 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1f01595a0 0x7fb1f01f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:21.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.071+0000 7fb1f5707640 1 --2- 172.21.15.76:0/2722436533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1f0157b30 0x7fb1f0159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:21.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.071+0000 7fb1f5f08640 1 -- 172.21.15.76:0/2722436533 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1f01595a0 msgr2=0x7fb1f01f37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:21.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.071+0000 7fb1f5f08640 1 --2- 172.21.15.76:0/2722436533 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1f01595a0 0x7fb1f01f37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:21.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.071+0000 7fb1f5f08640 1 -- 172.21.15.76:0/2722436533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1f0157b30 msgr2=0x7fb1f0159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:21.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.071+0000 7fb1f5f08640 1 --2- 172.21.15.76:0/2722436533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1f0157b30 0x7fb1f0159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:21.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.071+0000 7fb1f5f08640 1 -- 172.21.15.76:0/2722436533 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f0158580 con 0x7fb1dc0047c0 2023-11-09T03:47:21.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.072+0000 7fb1f5707640 1 --2- 172.21.15.76:0/2722436533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1f0157b30 0x7fb1f0159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:21.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.072+0000 7fb1f5f08640 1 --2- 172.21.15.76:0/2722436533 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1dc0047c0 0x7fb1f01534e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00f7a0 tx=0x7fb1ec00fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:21.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.072+0000 7fb1e67fc640 1 -- 172.21.15.76:0/2722436533 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec00fe50 con 0x7fb1dc0047c0 2023-11-09T03:47:21.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.072+0000 7fb1e67fc640 1 -- 172.21.15.76:0/2722436533 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ec0052e0 con 0x7fb1dc0047c0 2023-11-09T03:47:21.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.072+0000 7fb1e67fc640 1 -- 172.21.15.76:0/2722436533 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec00d970 con 0x7fb1dc0047c0 2023-11-09T03:47:21.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.072+0000 7fb1f816c640 1 -- 172.21.15.76:0/2722436533 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1f0158860 con 0x7fb1dc0047c0 2023-11-09T03:47:21.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.072+0000 7fb1f816c640 1 -- 172.21.15.76:0/2722436533 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f01f3d00 con 0x7fb1dc0047c0 2023-11-09T03:47:21.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.073+0000 7fb1f816c640 1 -- 172.21.15.76:0/2722436533 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1f0158d80 con 0x7fb1dc0047c0 2023-11-09T03:47:21.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.073+0000 7fb1e67fc640 1 -- 172.21.15.76:0/2722436533 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb1ec00dad0 con 0x7fb1dc0047c0 2023-11-09T03:47:21.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.074+0000 7fb1e67fc640 1 --2- 172.21.15.76:0/2722436533 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb1d406aea0 0x7fb1d406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:21.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.074+0000 7fb1e67fc640 1 -- 172.21.15.76:0/2722436533 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb1ec0898f0 con 0x7fb1dc0047c0 2023-11-09T03:47:21.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.074+0000 7fb1f5707640 1 --2- 172.21.15.76:0/2722436533 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb1d406aea0 0x7fb1d406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:21.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.074+0000 7fb1f5707640 1 --2- 172.21.15.76:0/2722436533 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb1d406aea0 0x7fb1d406d360 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0035000 tx=0x7fb1e0030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:21.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.078+0000 7fb1e67fc640 1 -- 172.21.15.76:0/2722436533 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb1f0158d80 con 0x7fb1dc0047c0 2023-11-09T03:47:21.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.245+0000 7fb1f816c640 1 -- 172.21.15.76:0/2722436533 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1f0158d80 con 0x7fb1dc0047c0 2023-11-09T03:47:21.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.247+0000 7fb1e67fc640 1 -- 172.21.15.76:0/2722436533 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+18524 (secure 0 0 0) 0x7fb1f0158d80 con 0x7fb1dc0047c0 2023-11-09T03:47:21.246 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:21.247 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":53,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:47:20.264884+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:47:18.807110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-09T03:47:19.235022+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:47:21.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.249+0000 7fb1f816c640 1 -- 172.21.15.76:0/2722436533 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb1d406aea0 msgr2=0x7fb1d406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:21.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.249+0000 7fb1f816c640 1 --2- 172.21.15.76:0/2722436533 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb1d406aea0 0x7fb1d406d360 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0035000 tx=0x7fb1e0030000 comp rx=0 tx=0).stop 2023-11-09T03:47:21.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.249+0000 7fb1f816c640 1 -- 172.21.15.76:0/2722436533 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1dc0047c0 msgr2=0x7fb1f01534e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:21.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.249+0000 7fb1f816c640 1 --2- 172.21.15.76:0/2722436533 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1dc0047c0 0x7fb1f01534e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00f7a0 tx=0x7fb1ec00fc60 comp rx=0 tx=0).stop 2023-11-09T03:47:21.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.249+0000 7fb1f816c640 1 -- 172.21.15.76:0/2722436533 shutdown_connections 2023-11-09T03:47:21.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.249+0000 7fb1f816c640 1 --2- 172.21.15.76:0/2722436533 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb1d406aea0 0x7fb1d406d360 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:21.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.249+0000 7fb1f816c640 1 --2- 172.21.15.76:0/2722436533 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1f01595a0 0x7fb1f01f37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:21.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.249+0000 7fb1f816c640 1 --2- 172.21.15.76:0/2722436533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1f0157b30 0x7fb1f0159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:21.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.249+0000 7fb1f816c640 1 --2- 172.21.15.76:0/2722436533 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1dc0047c0 0x7fb1f01534e0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:21.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.249+0000 7fb1f816c640 1 -- 172.21.15.76:0/2722436533 >> 172.21.15.76:0/2722436533 conn(0x7fb1f0071f30 msgr2=0x7fb1f00d1770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:21.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.249+0000 7fb1f816c640 1 -- 172.21.15.76:0/2722436533 shutdown_connections 2023-11-09T03:47:21.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.249+0000 7fb1f816c640 1 -- 172.21.15.76:0/2722436533 wait complete. 2023-11-09T03:47:21.259 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:47:21.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.360+0000 7fa43c183640 1 Processor -- start 2023-11-09T03:47:21.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.360+0000 7fa43c183640 1 -- start start 2023-11-09T03:47:21.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.361+0000 7fa43c183640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa434159c70 0x7fa4340ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:21.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.361+0000 7fa43c183640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa43406db20 0x7fa4340b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:21.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.361+0000 7fa43c183640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4340b4110 con 0x7fa4340b0910 2023-11-09T03:47:21.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.361+0000 7fa43c183640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4340b49b0 con 0x7fa43406db20 2023-11-09T03:47:21.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.361+0000 7fa43c183640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4340b0da0 con 0x7fa434159c70 2023-11-09T03:47:21.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.361+0000 7fa439f1f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa434159c70 0x7fa4340ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:21.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.361+0000 7fa43971e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa43406db20 0x7fa4340b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:21.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.361+0000 7fa43971e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa43406db20 0x7fa4340b03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:53688/0 (socket says 172.21.15.76:53688) 2023-11-09T03:47:21.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.361+0000 7fa43a720640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa4340b0910 0x7fa43415d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:53120/0 (socket says 172.21.15.76:53120) 2023-11-09T03:47:21.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.361+0000 7fa43971e640 1 -- 172.21.15.76:0/1237890280 learned_addr learned my addr 172.21.15.76:0/1237890280 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:21.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.361+0000 7fa439f1f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa434159c70 0x7fa4340ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:57766/0 (socket says 172.21.15.76:57766) 2023-11-09T03:47:21.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.361+0000 7fa43971e640 1 -- 172.21.15.76:0/1237890280 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa434159c70 msgr2=0x7fa4340ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:21.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.361+0000 7fa43971e640 1 --2- 172.21.15.76:0/1237890280 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa434159c70 0x7fa4340ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:21.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.361+0000 7fa43971e640 1 -- 172.21.15.76:0/1237890280 >> v1:172.21.15.113:6789/0 conn(0x7fa4340b0910 legacy=0x7fa43415d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:21.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.361+0000 7fa43971e640 1 -- 172.21.15.76:0/1237890280 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4340b43f0 con 0x7fa43406db20 2023-11-09T03:47:21.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.361+0000 7fa438f1d640 1 -- 172.21.15.76:0/1237890280 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2093254421 0 0) 0x7fa4340b4110 con 0x7fa4340b0910 2023-11-09T03:47:21.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.361+0000 7fa439f1f640 1 --2- 172.21.15.76:0/1237890280 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa434159c70 0x7fa4340ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:47:21.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.362+0000 7fa43971e640 1 --2- 172.21.15.76:0/1237890280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa43406db20 0x7fa4340b03d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa424009870 tx=0x7fa42402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfcb6ce3c9899bf4 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:21.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.362+0000 7fa438f1d640 1 -- 172.21.15.76:0/1237890280 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa424030840 con 0x7fa43406db20 2023-11-09T03:47:21.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.362+0000 7fa438f1d640 1 -- 172.21.15.76:0/1237890280 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4240309a0 con 0x7fa43406db20 2023-11-09T03:47:21.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.362+0000 7fa438f1d640 1 -- 172.21.15.76:0/1237890280 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa424030d30 con 0x7fa43406db20 2023-11-09T03:47:21.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.363+0000 7fa43c183640 1 -- 172.21.15.76:0/1237890280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa43406db20 msgr2=0x7fa4340b03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:21.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.363+0000 7fa43c183640 1 --2- 172.21.15.76:0/1237890280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa43406db20 0x7fa4340b03d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa424009870 tx=0x7fa42402b980 comp rx=0 tx=0).stop 2023-11-09T03:47:21.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.363+0000 7fa43c183640 1 -- 172.21.15.76:0/1237890280 shutdown_connections 2023-11-09T03:47:21.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.363+0000 7fa43c183640 1 --2- 172.21.15.76:0/1237890280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa43406db20 0x7fa4340b03d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:21.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.363+0000 7fa43c183640 1 --2- 172.21.15.76:0/1237890280 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa434159c70 0x7fa4340ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:21.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.363+0000 7fa43c183640 1 -- 172.21.15.76:0/1237890280 >> 172.21.15.76:0/1237890280 conn(0x7fa4340ce950 msgr2=0x7fa4340ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:21.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.363+0000 7fa43c183640 1 -- 172.21.15.76:0/1237890280 shutdown_connections 2023-11-09T03:47:21.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.363+0000 7fa43c183640 1 -- 172.21.15.76:0/1237890280 wait complete. 2023-11-09T03:47:21.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.364+0000 7fa43c183640 1 Processor -- start 2023-11-09T03:47:21.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.364+0000 7fa43c183640 1 -- start start 2023-11-09T03:47:21.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.365+0000 7fa43c183640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa43406db20 0x7fa434155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:21.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.365+0000 7fa43c183640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4340b0910 0x7fa434147bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:21.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.365+0000 7fa43c183640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa434159c70 0x7fa434070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:21.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.365+0000 7fa43c183640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa434155bd0 con 0x7fa434159c70 2023-11-09T03:47:21.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.365+0000 7fa43c183640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4341f0640 con 0x7fa43406db20 2023-11-09T03:47:21.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.365+0000 7fa43c183640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4341511f0 con 0x7fa4340b0910 2023-11-09T03:47:21.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.365+0000 7fa43a720640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa434159c70 0x7fa434070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:21.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.365+0000 7fa43a720640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa434159c70 0x7fa434070710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60506/0 (socket says 172.21.15.76:60506) 2023-11-09T03:47:21.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.365+0000 7fa43a720640 1 -- 172.21.15.76:0/752240441 learned_addr learned my addr 172.21.15.76:0/752240441 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:21.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.365+0000 7fa439f1f640 1 --2- 172.21.15.76:0/752240441 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa43406db20 0x7fa434155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:21.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.365+0000 7fa43971e640 1 --2- 172.21.15.76:0/752240441 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4340b0910 0x7fa434147bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:21.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.365+0000 7fa43a720640 1 -- 172.21.15.76:0/752240441 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4340b0910 msgr2=0x7fa434147bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:21.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.365+0000 7fa43a720640 1 --2- 172.21.15.76:0/752240441 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4340b0910 0x7fa434147bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:21.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.365+0000 7fa43a720640 1 -- 172.21.15.76:0/752240441 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa43406db20 msgr2=0x7fa434155280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:21.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.365+0000 7fa43a720640 1 --2- 172.21.15.76:0/752240441 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa43406db20 0x7fa434155280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:21.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.365+0000 7fa43a720640 1 -- 172.21.15.76:0/752240441 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa430009400 con 0x7fa434159c70 2023-11-09T03:47:21.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.366+0000 7fa43a720640 1 --2- 172.21.15.76:0/752240441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa434159c70 0x7fa434070710 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fa42c00c7a0 tx=0x7fa42c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:21.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.366+0000 7fa42a7fc640 1 -- 172.21.15.76:0/752240441 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42c00a640 con 0x7fa434159c70 2023-11-09T03:47:21.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.366+0000 7fa42a7fc640 1 -- 172.21.15.76:0/752240441 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42c00a7a0 con 0x7fa434159c70 2023-11-09T03:47:21.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.366+0000 7fa43c183640 1 -- 172.21.15.76:0/752240441 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4240094d0 con 0x7fa434159c70 2023-11-09T03:47:21.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.366+0000 7fa42a7fc640 1 -- 172.21.15.76:0/752240441 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42c00abd0 con 0x7fa434159c70 2023-11-09T03:47:21.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.366+0000 7fa43c183640 1 -- 172.21.15.76:0/752240441 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa434155820 con 0x7fa434159c70 2023-11-09T03:47:21.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.367+0000 7fa43c183640 1 -- 172.21.15.76:0/752240441 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa434155d40 con 0x7fa434159c70 2023-11-09T03:47:21.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.367+0000 7fa42a7fc640 1 -- 172.21.15.76:0/752240441 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa42c00ad30 con 0x7fa434159c70 2023-11-09T03:47:21.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.367+0000 7fa42a7fc640 1 --2- 172.21.15.76:0/752240441 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa41806ae70 0x7fa41806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:21.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.368+0000 7fa42a7fc640 1 -- 172.21.15.76:0/752240441 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fa42c089050 con 0x7fa434159c70 2023-11-09T03:47:21.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.368+0000 7fa439f1f640 1 --2- 172.21.15.76:0/752240441 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa41806ae70 0x7fa41806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:21.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.368+0000 7fa439f1f640 1 --2- 172.21.15.76:0/752240441 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa41806ae70 0x7fa41806d330 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa430009b80 tx=0x7fa430008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:21.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.372+0000 7fa42a7fc640 1 -- 172.21.15.76:0/752240441 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa434155d40 con 0x7fa434159c70 2023-11-09T03:47:21.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.541+0000 7fa43c183640 1 -- 172.21.15.76:0/752240441 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa434155d10 con 0x7fa434159c70 2023-11-09T03:47:21.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.542+0000 7fa42a7fc640 1 -- 172.21.15.76:0/752240441 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+18524 (secure 0 0 0) 0x7fa42c018090 con 0x7fa434159c70 2023-11-09T03:47:21.542 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:21.542 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":53,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:47:20.264884+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:47:18.807110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-09T03:47:19.235022+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:47:21.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.544+0000 7fa43c183640 1 -- 172.21.15.76:0/752240441 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa41806ae70 msgr2=0x7fa41806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:21.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.544+0000 7fa43c183640 1 --2- 172.21.15.76:0/752240441 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa41806ae70 0x7fa41806d330 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa430009b80 tx=0x7fa430008040 comp rx=0 tx=0).stop 2023-11-09T03:47:21.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.544+0000 7fa43c183640 1 -- 172.21.15.76:0/752240441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa434159c70 msgr2=0x7fa434070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:21.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.544+0000 7fa43c183640 1 --2- 172.21.15.76:0/752240441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa434159c70 0x7fa434070710 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fa42c00c7a0 tx=0x7fa42c00cc60 comp rx=0 tx=0).stop 2023-11-09T03:47:21.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.544+0000 7fa43c183640 1 -- 172.21.15.76:0/752240441 shutdown_connections 2023-11-09T03:47:21.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.544+0000 7fa43c183640 1 --2- 172.21.15.76:0/752240441 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa41806ae70 0x7fa41806d330 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:21.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.544+0000 7fa43c183640 1 --2- 172.21.15.76:0/752240441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa434159c70 0x7fa434070710 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:21.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.544+0000 7fa43c183640 1 --2- 172.21.15.76:0/752240441 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4340b0910 0x7fa434147bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:21.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.544+0000 7fa43c183640 1 --2- 172.21.15.76:0/752240441 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa43406db20 0x7fa434155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:21.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.544+0000 7fa43c183640 1 -- 172.21.15.76:0/752240441 >> 172.21.15.76:0/752240441 conn(0x7fa4340ce950 msgr2=0x7fa43415cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:21.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.544+0000 7fa43c183640 1 -- 172.21.15.76:0/752240441 shutdown_connections 2023-11-09T03:47:21.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.544+0000 7fa43c183640 1 -- 172.21.15.76:0/752240441 wait complete. 2023-11-09T03:47:21.554 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-09T03:47:21.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.658+0000 7f24a95c0640 1 Processor -- start 2023-11-09T03:47:21.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.658+0000 7f24a95c0640 1 -- start start 2023-11-09T03:47:21.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.659+0000 7f24a95c0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f24a4153850 0x7f24a415a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:21.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.659+0000 7f24a95c0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f24a40cc520 0x7f24a4070fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:21.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.659+0000 7f24a95c0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24a40c3860 con 0x7f24a40cc9d0 2023-11-09T03:47:21.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.659+0000 7f24a95c0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a4070e20 con 0x7f24a4153850 2023-11-09T03:47:21.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.659+0000 7f24a95c0640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a40c9250 con 0x7f24a40cc520 2023-11-09T03:47:21.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.659+0000 7f24a2ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f24a4153850 0x7f24a415a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:21.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.659+0000 7f24a37fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f24a40cc9d0 0x7f24a4070710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57232/0 (socket says 172.21.15.76:57232) 2023-11-09T03:47:21.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.659+0000 7f24a37fe640 1 -- 172.21.15.76:0/3948682487 learned_addr learned my addr 172.21.15.76:0/3948682487 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:21.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.660+0000 7f24a27fc640 1 --2- 172.21.15.76:0/3948682487 >> v2:172.21.15.113:3300/0 conn(0x7f24a40cc520 0x7f24a4070fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:21.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.660+0000 7f24a2ffd640 1 -- 172.21.15.76:0/3948682487 >> v2:172.21.15.113:3300/0 conn(0x7f24a40cc520 msgr2=0x7f24a4070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:21.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.660+0000 7f24a2ffd640 1 --2- 172.21.15.76:0/3948682487 >> v2:172.21.15.113:3300/0 conn(0x7f24a40cc520 0x7f24a4070fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:21.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.660+0000 7f24a2ffd640 1 -- 172.21.15.76:0/3948682487 >> v1:172.21.15.76:6789/0 conn(0x7f24a40cc9d0 legacy=0x7f24a4070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:21.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.660+0000 7f24a2ffd640 1 -- 172.21.15.76:0/3948682487 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a4153c30 con 0x7f24a4153850 2023-11-09T03:47:21.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.660+0000 7f24a1ffb640 1 -- 172.21.15.76:0/3948682487 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2789889766 0 0) 0x7f24a40c3860 con 0x7f24a40cc9d0 2023-11-09T03:47:21.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.660+0000 7f24a2ffd640 1 --2- 172.21.15.76:0/3948682487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24a4153850 0x7f24a415a1c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f24a4070560 tx=0x7f249402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f713814c66ec00a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:21.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.661+0000 7f24a1ffb640 1 -- 172.21.15.76:0/3948682487 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249402cc80 con 0x7f24a4153850 2023-11-09T03:47:21.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.661+0000 7f24a1ffb640 1 -- 172.21.15.76:0/3948682487 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249402cde0 con 0x7f24a4153850 2023-11-09T03:47:21.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.661+0000 7f24a1ffb640 1 -- 172.21.15.76:0/3948682487 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2494034660 con 0x7f24a4153850 2023-11-09T03:47:21.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.661+0000 7f24a95c0640 1 -- 172.21.15.76:0/3948682487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24a4153850 msgr2=0x7f24a415a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:21.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.661+0000 7f24a95c0640 1 --2- 172.21.15.76:0/3948682487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24a4153850 0x7f24a415a1c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f24a4070560 tx=0x7f249402bed0 comp rx=0 tx=0).stop 2023-11-09T03:47:21.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.661+0000 7f24a95c0640 1 -- 172.21.15.76:0/3948682487 shutdown_connections 2023-11-09T03:47:21.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.661+0000 7f24a95c0640 1 --2- 172.21.15.76:0/3948682487 >> v2:172.21.15.113:3300/0 conn(0x7f24a40cc520 0x7f24a4070fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:21.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.661+0000 7f24a95c0640 1 --2- 172.21.15.76:0/3948682487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24a4153850 0x7f24a415a1c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:21.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.661+0000 7f24a95c0640 1 -- 172.21.15.76:0/3948682487 >> 172.21.15.76:0/3948682487 conn(0x7f24a40ce950 msgr2=0x7f24a40ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:21.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.661+0000 7f24a95c0640 1 -- 172.21.15.76:0/3948682487 shutdown_connections 2023-11-09T03:47:21.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.661+0000 7f24a95c0640 1 -- 172.21.15.76:0/3948682487 wait complete. 2023-11-09T03:47:21.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.662+0000 7f24a95c0640 1 Processor -- start 2023-11-09T03:47:21.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.662+0000 7f24a95c0640 1 -- start start 2023-11-09T03:47:21.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.663+0000 7f24a95c0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24a40cc520 0x7f24a414db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:21.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.663+0000 7f24a95c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24a40cc9d0 0x7f24a414aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:21.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.663+0000 7f24a95c0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24a4153850 0x7f24a414c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:21.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.663+0000 7f24a95c0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a414c500 con 0x7f24a40cc9d0 2023-11-09T03:47:21.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.663+0000 7f24a95c0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a414b540 con 0x7f24a4153850 2023-11-09T03:47:21.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.663+0000 7f24a95c0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a414aff0 con 0x7f24a40cc520 2023-11-09T03:47:21.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.663+0000 7f24a27fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24a40cc9d0 0x7f24a414aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:21.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.663+0000 7f24a27fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24a40cc9d0 0x7f24a414aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60528/0 (socket says 172.21.15.76:60528) 2023-11-09T03:47:21.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.663+0000 7f24a27fc640 1 -- 172.21.15.76:0/3749152006 learned_addr learned my addr 172.21.15.76:0/3749152006 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:21.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.663+0000 7f24a2ffd640 1 --2- 172.21.15.76:0/3749152006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24a40cc520 0x7f24a414db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:21.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.663+0000 7f24a37fe640 1 --2- 172.21.15.76:0/3749152006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24a4153850 0x7f24a414c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:21.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.664+0000 7f24a27fc640 1 -- 172.21.15.76:0/3749152006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24a40cc520 msgr2=0x7f24a414db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:21.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.664+0000 7f24a27fc640 1 --2- 172.21.15.76:0/3749152006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24a40cc520 0x7f24a414db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:21.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.664+0000 7f24a27fc640 1 -- 172.21.15.76:0/3749152006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24a4153850 msgr2=0x7f24a414c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:21.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.664+0000 7f24a27fc640 1 --2- 172.21.15.76:0/3749152006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24a4153850 0x7f24a414c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:21.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.664+0000 7f24a27fc640 1 -- 172.21.15.76:0/3749152006 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2494009a70 con 0x7f24a40cc9d0 2023-11-09T03:47:21.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.664+0000 7f24a37fe640 1 --2- 172.21.15.76:0/3749152006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24a4153850 0x7f24a414c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:21.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.664+0000 7f24a27fc640 1 --2- 172.21.15.76:0/3749152006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24a40cc9d0 0x7f24a414aab0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f249800d860 tx=0x7f249800dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:21.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.664+0000 7f248b7fe640 1 -- 172.21.15.76:0/3749152006 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249800de50 con 0x7f24a40cc9d0 2023-11-09T03:47:21.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.664+0000 7f248b7fe640 1 -- 172.21.15.76:0/3749152006 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249800b8d0 con 0x7f24a40cc9d0 2023-11-09T03:47:21.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.664+0000 7f24a95c0640 1 -- 172.21.15.76:0/3749152006 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a414b260 con 0x7f24a40cc9d0 2023-11-09T03:47:21.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.664+0000 7f248b7fe640 1 -- 172.21.15.76:0/3749152006 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249800bc60 con 0x7f24a40cc9d0 2023-11-09T03:47:21.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.664+0000 7f24a95c0640 1 -- 172.21.15.76:0/3749152006 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a414c120 con 0x7f24a40cc9d0 2023-11-09T03:47:21.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.665+0000 7f24a95c0640 1 -- 172.21.15.76:0/3749152006 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a4074290 con 0x7f24a40cc9d0 2023-11-09T03:47:21.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.665+0000 7f248b7fe640 1 -- 172.21.15.76:0/3749152006 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f249801e460 con 0x7f24a40cc9d0 2023-11-09T03:47:21.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.666+0000 7f248b7fe640 1 --2- 172.21.15.76:0/3749152006 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f249006ae70 0x7f249006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:21.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.666+0000 7f248b7fe640 1 -- 172.21.15.76:0/3749152006 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2498089d20 con 0x7f24a40cc9d0 2023-11-09T03:47:21.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.666+0000 7f24a2ffd640 1 --2- 172.21.15.76:0/3749152006 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f249006ae70 0x7f249006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:21.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.667+0000 7f24a2ffd640 1 --2- 172.21.15.76:0/3749152006 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f249006ae70 0x7f249006d330 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2494036000 tx=0x7f2494030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:21.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.670+0000 7f248b7fe640 1 -- 172.21.15.76:0/3749152006 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2498054bc0 con 0x7f24a40cc9d0 2023-11-09T03:47:21.871 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:21.871+0000 7f24a95c0640 1 -- 172.21.15.76:0/3749152006 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f24a4153070 con 0x7f24a40cc9d0 2023-11-09T03:47:22.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.407+0000 7f248b7fe640 1 -- 172.21.15.76:0/3749152006 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v22) v1 ==== 118+0+0 (secure 0 0 0) 0x7f24980581e0 con 0x7f24a40cc9d0 2023-11-09T03:47:22.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.409+0000 7f24a95c0640 1 -- 172.21.15.76:0/3749152006 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f249006ae70 msgr2=0x7f249006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:22.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.409+0000 7f24a95c0640 1 --2- 172.21.15.76:0/3749152006 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f249006ae70 0x7f249006d330 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2494036000 tx=0x7f2494030000 comp rx=0 tx=0).stop 2023-11-09T03:47:22.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.409+0000 7f24a95c0640 1 -- 172.21.15.76:0/3749152006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24a40cc9d0 msgr2=0x7f24a414aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:22.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.409+0000 7f24a95c0640 1 --2- 172.21.15.76:0/3749152006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24a40cc9d0 0x7f24a414aab0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f249800d860 tx=0x7f249800dd20 comp rx=0 tx=0).stop 2023-11-09T03:47:22.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.409+0000 7f24a95c0640 1 -- 172.21.15.76:0/3749152006 shutdown_connections 2023-11-09T03:47:22.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.409+0000 7f24a95c0640 1 --2- 172.21.15.76:0/3749152006 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f249006ae70 0x7f249006d330 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:22.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.409+0000 7f24a95c0640 1 --2- 172.21.15.76:0/3749152006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24a4153850 0x7f24a414c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:22.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.409+0000 7f24a95c0640 1 --2- 172.21.15.76:0/3749152006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24a40cc9d0 0x7f24a414aab0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:22.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.409+0000 7f24a95c0640 1 --2- 172.21.15.76:0/3749152006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24a40cc520 0x7f24a414db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:22.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.409+0000 7f24a95c0640 1 -- 172.21.15.76:0/3749152006 >> 172.21.15.76:0/3749152006 conn(0x7f24a40ce950 msgr2=0x7f24a41573b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:22.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.409+0000 7f24a95c0640 1 -- 172.21.15.76:0/3749152006 shutdown_connections 2023-11-09T03:47:22.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.409+0000 7f24a95c0640 1 -- 172.21.15.76:0/3749152006 wait complete. 2023-11-09T03:47:22.421 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-09T03:47:22.421 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-09T03:47:22.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.528+0000 7fe3a06a2640 1 Processor -- start 2023-11-09T03:47:22.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.528+0000 7fe3a06a2640 1 -- start start 2023-11-09T03:47:22.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.529+0000 7fe3a06a2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe39815ad30 0x7fe3980adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:22.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.529+0000 7fe3a06a2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe3980ad730 0x7fe398070710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:22.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.529+0000 7fe3a06a2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe398070c50 0x7fe3980c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:22.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.529+0000 7fe3a06a2640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3980ae260 con 0x7fe3980ad730 2023-11-09T03:47:22.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.529+0000 7fe3a06a2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3980710e0 con 0x7fe398070c50 2023-11-09T03:47:22.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.529+0000 7fe3a06a2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3980c3900 con 0x7fe39815ad30 2023-11-09T03:47:22.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.529+0000 7fe39dc3d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe3980ad730 0x7fe398070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:22.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.529+0000 7fe39dc3d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe3980ad730 0x7fe398070710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60538/0 (socket says 172.21.15.76:60538) 2023-11-09T03:47:22.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.529+0000 7fe39dc3d640 1 -- 172.21.15.76:0/2841496978 learned_addr learned my addr 172.21.15.76:0/2841496978 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:22.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.529+0000 7fe39e43e640 1 --2- 172.21.15.76:0/2841496978 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe39815ad30 0x7fe3980adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:22.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.529+0000 7fe39ec3f640 1 --2- 172.21.15.76:0/2841496978 >> v2:172.21.15.113:3300/0 conn(0x7fe398070c50 0x7fe3980c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:22.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.529+0000 7fe39dc3d640 1 -- 172.21.15.76:0/2841496978 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe39815ad30 msgr2=0x7fe3980adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:22.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.529+0000 7fe39dc3d640 1 --2- 172.21.15.76:0/2841496978 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe39815ad30 0x7fe3980adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:22.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.529+0000 7fe39dc3d640 1 -- 172.21.15.76:0/2841496978 >> v2:172.21.15.113:3300/0 conn(0x7fe398070c50 msgr2=0x7fe3980c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:22.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.529+0000 7fe39dc3d640 1 --2- 172.21.15.76:0/2841496978 >> v2:172.21.15.113:3300/0 conn(0x7fe398070c50 0x7fe3980c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:22.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.529+0000 7fe39dc3d640 1 -- 172.21.15.76:0/2841496978 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3980c3be0 con 0x7fe3980ad730 2023-11-09T03:47:22.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.529+0000 7fe39e43e640 1 --2- 172.21.15.76:0/2841496978 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe39815ad30 0x7fe3980adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:22.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.529+0000 7fe39ec3f640 1 --2- 172.21.15.76:0/2841496978 >> v2:172.21.15.113:3300/0 conn(0x7fe398070c50 0x7fe3980c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:22.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.530+0000 7fe39dc3d640 1 --2- 172.21.15.76:0/2841496978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3980ad730 0x7fe398070710 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe388009870 tx=0x7fe38802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b3142cb7174f9d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:22.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.530+0000 7fe39d43c640 1 -- 172.21.15.76:0/2841496978 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3880317c0 con 0x7fe3980ad730 2023-11-09T03:47:22.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.530+0000 7fe39d43c640 1 -- 172.21.15.76:0/2841496978 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe388031920 con 0x7fe3980ad730 2023-11-09T03:47:22.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.530+0000 7fe39d43c640 1 -- 172.21.15.76:0/2841496978 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe388031cb0 con 0x7fe3980ad730 2023-11-09T03:47:22.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.531+0000 7fe3a06a2640 1 -- 172.21.15.76:0/2841496978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3980ad730 msgr2=0x7fe398070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:22.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.531+0000 7fe3a06a2640 1 --2- 172.21.15.76:0/2841496978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3980ad730 0x7fe398070710 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe388009870 tx=0x7fe38802b860 comp rx=0 tx=0).stop 2023-11-09T03:47:22.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.531+0000 7fe3a06a2640 1 -- 172.21.15.76:0/2841496978 shutdown_connections 2023-11-09T03:47:22.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.531+0000 7fe3a06a2640 1 --2- 172.21.15.76:0/2841496978 >> v2:172.21.15.113:3300/0 conn(0x7fe398070c50 0x7fe3980c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:22.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.531+0000 7fe3a06a2640 1 --2- 172.21.15.76:0/2841496978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3980ad730 0x7fe398070710 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:22.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.531+0000 7fe3a06a2640 1 --2- 172.21.15.76:0/2841496978 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe39815ad30 0x7fe3980adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:22.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.531+0000 7fe3a06a2640 1 -- 172.21.15.76:0/2841496978 >> 172.21.15.76:0/2841496978 conn(0x7fe3980ce950 msgr2=0x7fe3980ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:22.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.531+0000 7fe3a06a2640 1 -- 172.21.15.76:0/2841496978 shutdown_connections 2023-11-09T03:47:22.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.531+0000 7fe3a06a2640 1 -- 172.21.15.76:0/2841496978 wait complete. 2023-11-09T03:47:22.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.532+0000 7fe3a06a2640 1 Processor -- start 2023-11-09T03:47:22.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.532+0000 7fe3a06a2640 1 -- start start 2023-11-09T03:47:22.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.532+0000 7fe3a06a2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe398070c50 0x7fe3981efc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:22.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.533+0000 7fe3a06a2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3980ad730 0x7fe398155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:22.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.533+0000 7fe3a06a2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe39815ad30 0x7fe398155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:22.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.533+0000 7fe3a06a2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3980b26b0 con 0x7fe39815ad30 2023-11-09T03:47:22.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.533+0000 7fe3a06a2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3981577b0 con 0x7fe398070c50 2023-11-09T03:47:22.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.533+0000 7fe3a06a2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3980b1da0 con 0x7fe3980ad730 2023-11-09T03:47:22.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.533+0000 7fe39ec3f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe39815ad30 0x7fe398155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:22.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.533+0000 7fe39ec3f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe39815ad30 0x7fe398155e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60546/0 (socket says 172.21.15.76:60546) 2023-11-09T03:47:22.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.533+0000 7fe39ec3f640 1 -- 172.21.15.76:0/2815538485 learned_addr learned my addr 172.21.15.76:0/2815538485 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:22.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.533+0000 7fe39e43e640 1 --2- 172.21.15.76:0/2815538485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe398070c50 0x7fe3981efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:22.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.533+0000 7fe39dc3d640 1 --2- 172.21.15.76:0/2815538485 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3980ad730 0x7fe398155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.533+0000 7fe39ec3f640 1 -- 172.21.15.76:0/2815538485 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3980ad730 msgr2=0x7fe398155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.533+0000 7fe39ec3f640 1 --2- 172.21.15.76:0/2815538485 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3980ad730 0x7fe398155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.533+0000 7fe39ec3f640 1 -- 172.21.15.76:0/2815538485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe398070c50 msgr2=0x7fe3981efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.533+0000 7fe39ec3f640 1 --2- 172.21.15.76:0/2815538485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe398070c50 0x7fe3981efc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:22.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.533+0000 7fe39ec3f640 1 -- 172.21.15.76:0/2815538485 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3880094d0 con 0x7fe39815ad30 2023-11-09T03:47:22.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.533+0000 7fe39ec3f640 1 --2- 172.21.15.76:0/2815538485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe39815ad30 0x7fe398155e60 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe39000cef0 tx=0x7fe39000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:22.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.533+0000 7fe38effd640 1 -- 172.21.15.76:0/2815538485 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39000d840 con 0x7fe39815ad30 2023-11-09T03:47:22.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.533+0000 7fe38effd640 1 -- 172.21.15.76:0/2815538485 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39000d9a0 con 0x7fe39815ad30 2023-11-09T03:47:22.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.533+0000 7fe38effd640 1 -- 172.21.15.76:0/2815538485 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39000dd50 con 0x7fe39815ad30 2023-11-09T03:47:22.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.533+0000 7fe3a06a2640 1 -- 172.21.15.76:0/2815538485 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3981f01a0 con 0x7fe39815ad30 2023-11-09T03:47:22.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.533+0000 7fe3a06a2640 1 -- 172.21.15.76:0/2815538485 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3980b1ff0 con 0x7fe39815ad30 2023-11-09T03:47:22.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.534+0000 7fe3a06a2640 1 -- 172.21.15.76:0/2815538485 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3980b27f0 con 0x7fe39815ad30 2023-11-09T03:47:22.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.535+0000 7fe38effd640 1 -- 172.21.15.76:0/2815538485 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe39001e3b0 con 0x7fe39815ad30 2023-11-09T03:47:22.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.535+0000 7fe38effd640 1 --2- 172.21.15.76:0/2815538485 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe37c06ae70 0x7fe37c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:22.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.535+0000 7fe38effd640 1 -- 172.21.15.76:0/2815538485 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe39008a250 con 0x7fe39815ad30 2023-11-09T03:47:22.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.535+0000 7fe39e43e640 1 --2- 172.21.15.76:0/2815538485 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe37c06ae70 0x7fe37c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:22.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.536+0000 7fe39e43e640 1 --2- 172.21.15.76:0/2815538485 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe37c06ae70 0x7fe37c06d330 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe39400aac0 tx=0x7fe394008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:22.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.539+0000 7fe38effd640 1 -- 172.21.15.76:0/2815538485 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe3980b27f0 con 0x7fe39815ad30 2023-11-09T03:47:22.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:22.741+0000 7fe3a06a2640 1 -- 172.21.15.76:0/2815538485 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe3980b27f0 con 0x7fe39815ad30 2023-11-09T03:47:23.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.411+0000 7fe38effd640 1 -- 172.21.15.76:0/2815538485 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v23) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe3980b27f0 con 0x7fe39815ad30 2023-11-09T03:47:23.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.414+0000 7fe3a06a2640 1 -- 172.21.15.76:0/2815538485 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe37c06ae70 msgr2=0x7fe37c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:23.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.414+0000 7fe3a06a2640 1 --2- 172.21.15.76:0/2815538485 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe37c06ae70 0x7fe37c06d330 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe39400aac0 tx=0x7fe394008040 comp rx=0 tx=0).stop 2023-11-09T03:47:23.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.414+0000 7fe3a06a2640 1 -- 172.21.15.76:0/2815538485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe39815ad30 msgr2=0x7fe398155e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:23.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.414+0000 7fe3a06a2640 1 --2- 172.21.15.76:0/2815538485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe39815ad30 0x7fe398155e60 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe39000cef0 tx=0x7fe39000a710 comp rx=0 tx=0).stop 2023-11-09T03:47:23.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.414+0000 7fe3a06a2640 1 -- 172.21.15.76:0/2815538485 shutdown_connections 2023-11-09T03:47:23.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.414+0000 7fe3a06a2640 1 --2- 172.21.15.76:0/2815538485 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe37c06ae70 0x7fe37c06d330 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:23.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.414+0000 7fe3a06a2640 1 --2- 172.21.15.76:0/2815538485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe39815ad30 0x7fe398155e60 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:23.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.414+0000 7fe3a06a2640 1 --2- 172.21.15.76:0/2815538485 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3980ad730 0x7fe398155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:23.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.414+0000 7fe3a06a2640 1 --2- 172.21.15.76:0/2815538485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe398070c50 0x7fe3981efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:23.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.414+0000 7fe3a06a2640 1 -- 172.21.15.76:0/2815538485 >> 172.21.15.76:0/2815538485 conn(0x7fe3980ce950 msgr2=0x7fe3980b0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:23.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.414+0000 7fe3a06a2640 1 -- 172.21.15.76:0/2815538485 shutdown_connections 2023-11-09T03:47:23.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.414+0000 7fe3a06a2640 1 -- 172.21.15.76:0/2815538485 wait complete. 2023-11-09T03:47:23.425 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-09T03:47:23.426 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:47:23.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.529+0000 7fc67d9e2640 1 Processor -- start 2023-11-09T03:47:23.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.529+0000 7fc67d9e2640 1 -- start start 2023-11-09T03:47:23.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.530+0000 7fc67d9e2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc67815ce70 0x7fc67806d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:23.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.530+0000 7fc67d9e2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc67815d250 0x7fc6780c9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:23.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.530+0000 7fc67d9e2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc6780c9d70 0x7fc6780ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:23.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.530+0000 7fc67d9e2640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6780cdc10 con 0x7fc6780c9d70 2023-11-09T03:47:23.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.530+0000 7fc67d9e2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc67806a120 con 0x7fc67815ce70 2023-11-09T03:47:23.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.530+0000 7fc67d9e2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc67806def0 con 0x7fc67815d250 2023-11-09T03:47:23.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.530+0000 7fc6777fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc6780c9d70 0x7fc6780ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:23.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.530+0000 7fc6777fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc6780c9d70 0x7fc6780ccdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42458/0 (socket says 172.21.15.76:42458) 2023-11-09T03:47:23.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.530+0000 7fc6777fe640 1 -- 172.21.15.76:0/1782129163 learned_addr learned my addr 172.21.15.76:0/1782129163 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:23.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.531+0000 7fc676ffd640 1 --2- 172.21.15.76:0/1782129163 >> v2:172.21.15.113:3300/0 conn(0x7fc67815ce70 0x7fc67806d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:23.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.531+0000 7fc6767fc640 1 --2- 172.21.15.76:0/1782129163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc67815d250 0x7fc6780c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:23.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.531+0000 7fc6777fe640 1 -- 172.21.15.76:0/1782129163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc67815d250 msgr2=0x7fc6780c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:23.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.531+0000 7fc6777fe640 1 --2- 172.21.15.76:0/1782129163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc67815d250 0x7fc6780c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:23.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.531+0000 7fc6777fe640 1 -- 172.21.15.76:0/1782129163 >> v2:172.21.15.113:3300/0 conn(0x7fc67815ce70 msgr2=0x7fc67806d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:23.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.531+0000 7fc6777fe640 1 --2- 172.21.15.76:0/1782129163 >> v2:172.21.15.113:3300/0 conn(0x7fc67815ce70 0x7fc67806d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:23.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.531+0000 7fc6777fe640 1 -- 172.21.15.76:0/1782129163 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc67806a370 con 0x7fc6780c9d70 2023-11-09T03:47:23.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.531+0000 7fc6767fc640 1 --2- 172.21.15.76:0/1782129163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc67815d250 0x7fc6780c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:23.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.531+0000 7fc676ffd640 1 --2- 172.21.15.76:0/1782129163 >> v2:172.21.15.113:3300/0 conn(0x7fc67815ce70 0x7fc67806d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:23.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.531+0000 7fc6777fe640 1 --2- 172.21.15.76:0/1782129163 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6780c9d70 0x7fc6780ccdb0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc678065ea0 tx=0x7fc66802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774ed45853746db5 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:23.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.532+0000 7fc675ffb640 1 -- 172.21.15.76:0/1782129163 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc668031860 con 0x7fc6780c9d70 2023-11-09T03:47:23.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.532+0000 7fc675ffb640 1 -- 172.21.15.76:0/1782129163 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6680319c0 con 0x7fc6780c9d70 2023-11-09T03:47:23.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.532+0000 7fc675ffb640 1 -- 172.21.15.76:0/1782129163 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc668031d50 con 0x7fc6780c9d70 2023-11-09T03:47:23.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.532+0000 7fc67d9e2640 1 -- 172.21.15.76:0/1782129163 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6780c9d70 msgr2=0x7fc6780ccdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:23.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.532+0000 7fc67d9e2640 1 --2- 172.21.15.76:0/1782129163 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6780c9d70 0x7fc6780ccdb0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc678065ea0 tx=0x7fc66802b900 comp rx=0 tx=0).stop 2023-11-09T03:47:23.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.532+0000 7fc67d9e2640 1 -- 172.21.15.76:0/1782129163 shutdown_connections 2023-11-09T03:47:23.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.532+0000 7fc67d9e2640 1 --2- 172.21.15.76:0/1782129163 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6780c9d70 0x7fc6780ccdb0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:23.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.532+0000 7fc67d9e2640 1 --2- 172.21.15.76:0/1782129163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc67815d250 0x7fc6780c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:23.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.532+0000 7fc67d9e2640 1 --2- 172.21.15.76:0/1782129163 >> v2:172.21.15.113:3300/0 conn(0x7fc67815ce70 0x7fc67806d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:23.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.532+0000 7fc67d9e2640 1 -- 172.21.15.76:0/1782129163 >> 172.21.15.76:0/1782129163 conn(0x7fc6780b0cd0 msgr2=0x7fc6780b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:23.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.532+0000 7fc67d9e2640 1 -- 172.21.15.76:0/1782129163 shutdown_connections 2023-11-09T03:47:23.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.532+0000 7fc67d9e2640 1 -- 172.21.15.76:0/1782129163 wait complete. 2023-11-09T03:47:23.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.533+0000 7fc67d9e2640 1 Processor -- start 2023-11-09T03:47:23.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.534+0000 7fc67d9e2640 1 -- start start 2023-11-09T03:47:23.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.534+0000 7fc67d9e2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc67815ce70 0x7fc67815aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:23.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.534+0000 7fc67d9e2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc67815d250 0x7fc6781e76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:23.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.534+0000 7fc67d9e2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6781e7c30 0x7fc6781e4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:23.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.534+0000 7fc67d9e2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6781e80b0 con 0x7fc67815d250 2023-11-09T03:47:23.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.534+0000 7fc67d9e2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6781e5ca0 con 0x7fc67815ce70 2023-11-09T03:47:23.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.534+0000 7fc67d9e2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6781e5f80 con 0x7fc6781e7c30 2023-11-09T03:47:23.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.534+0000 7fc6767fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc67815d250 0x7fc6781e76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:23.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.534+0000 7fc6767fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc67815d250 0x7fc6781e76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42472/0 (socket says 172.21.15.76:42472) 2023-11-09T03:47:23.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.534+0000 7fc6767fc640 1 -- 172.21.15.76:0/2437316467 learned_addr learned my addr 172.21.15.76:0/2437316467 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:23.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.534+0000 7fc6777fe640 1 --2- 172.21.15.76:0/2437316467 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6781e7c30 0x7fc6781e4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:23.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.535+0000 7fc676ffd640 1 --2- 172.21.15.76:0/2437316467 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc67815ce70 0x7fc67815aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:23.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.535+0000 7fc6767fc640 1 -- 172.21.15.76:0/2437316467 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6781e7c30 msgr2=0x7fc6781e4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:23.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.535+0000 7fc6767fc640 1 --2- 172.21.15.76:0/2437316467 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6781e7c30 0x7fc6781e4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:23.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.535+0000 7fc6767fc640 1 -- 172.21.15.76:0/2437316467 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc67815ce70 msgr2=0x7fc67815aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:23.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.535+0000 7fc6767fc640 1 --2- 172.21.15.76:0/2437316467 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc67815ce70 0x7fc67815aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:23.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.535+0000 7fc6767fc640 1 -- 172.21.15.76:0/2437316467 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6680094d0 con 0x7fc67815d250 2023-11-09T03:47:23.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.535+0000 7fc6767fc640 1 --2- 172.21.15.76:0/2437316467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc67815d250 0x7fc6781e76f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc66c00cde0 tx=0x7fc66c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:23.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.535+0000 7fc65ffff640 1 -- 172.21.15.76:0/2437316467 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c00f680 con 0x7fc67815d250 2023-11-09T03:47:23.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.535+0000 7fc65ffff640 1 -- 172.21.15.76:0/2437316467 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66c00f7e0 con 0x7fc67815d250 2023-11-09T03:47:23.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.535+0000 7fc67d9e2640 1 -- 172.21.15.76:0/2437316467 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6781e6260 con 0x7fc67815d250 2023-11-09T03:47:23.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.535+0000 7fc65ffff640 1 -- 172.21.15.76:0/2437316467 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c00fb90 con 0x7fc67815d250 2023-11-09T03:47:23.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.535+0000 7fc67d9e2640 1 -- 172.21.15.76:0/2437316467 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6781e6820 con 0x7fc67815d250 2023-11-09T03:47:23.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.536+0000 7fc67d9e2640 1 -- 172.21.15.76:0/2437316467 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc678074290 con 0x7fc67815d250 2023-11-09T03:47:23.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.536+0000 7fc65ffff640 1 -- 172.21.15.76:0/2437316467 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc66c00b040 con 0x7fc67815d250 2023-11-09T03:47:23.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.537+0000 7fc65ffff640 1 --2- 172.21.15.76:0/2437316467 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc66406ae70 0x7fc66406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:23.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.537+0000 7fc65ffff640 1 -- 172.21.15.76:0/2437316467 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc66c08a2b0 con 0x7fc67815d250 2023-11-09T03:47:23.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.537+0000 7fc676ffd640 1 --2- 172.21.15.76:0/2437316467 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc66406ae70 0x7fc66406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:23.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.537+0000 7fc676ffd640 1 --2- 172.21.15.76:0/2437316467 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc66406ae70 0x7fc66406d330 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc670006b10 tx=0x7fc670008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:23.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.541+0000 7fc65ffff640 1 -- 172.21.15.76:0/2437316467 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc66c055150 con 0x7fc67815d250 2023-11-09T03:47:23.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.741+0000 7fc67d9e2640 1 -- 172.21.15.76:0/2437316467 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc678066170 con 0x7fc67815d250 2023-11-09T03:47:23.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.741+0000 7fc65ffff640 1 -- 172.21.15.76:0/2437316467 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fc66c058770 con 0x7fc67815d250 2023-11-09T03:47:23.741 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:23.741 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5099,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1844478483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1844478483},{"type":"v1","addr":"172.21.15.113:6836","nonce":1844478483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5102,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1111318612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1111318612},{"type":"v1","addr":"172.21.15.113:6837","nonce":1111318612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":6072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/2037805253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2037805253},{"type":"v1","addr":"172.21.15.76:6834","nonce":2037805253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:47:20.265190+0000","modified":"2023-11-09T03:47:22.742442+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6075},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6075":{"gid":6075,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/3710140065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":3710140065},{"type":"v1","addr":"172.21.15.76:6835","nonce":3710140065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-09T03:47:23.742 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 23 2023-11-09T03:47:23.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.743+0000 7fc67d9e2640 1 -- 172.21.15.76:0/2437316467 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc66406ae70 msgr2=0x7fc66406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:23.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.743+0000 7fc67d9e2640 1 --2- 172.21.15.76:0/2437316467 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc66406ae70 0x7fc66406d330 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc670006b10 tx=0x7fc670008040 comp rx=0 tx=0).stop 2023-11-09T03:47:23.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.743+0000 7fc67d9e2640 1 -- 172.21.15.76:0/2437316467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc67815d250 msgr2=0x7fc6781e76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:23.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.743+0000 7fc67d9e2640 1 --2- 172.21.15.76:0/2437316467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc67815d250 0x7fc6781e76f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc66c00cde0 tx=0x7fc66c00a650 comp rx=0 tx=0).stop 2023-11-09T03:47:23.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.743+0000 7fc67d9e2640 1 -- 172.21.15.76:0/2437316467 shutdown_connections 2023-11-09T03:47:23.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.743+0000 7fc67d9e2640 1 --2- 172.21.15.76:0/2437316467 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc66406ae70 0x7fc66406d330 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:23.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.743+0000 7fc67d9e2640 1 --2- 172.21.15.76:0/2437316467 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6781e7c30 0x7fc6781e4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:23.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.743+0000 7fc67d9e2640 1 --2- 172.21.15.76:0/2437316467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc67815d250 0x7fc6781e76f0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:23.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.743+0000 7fc67d9e2640 1 --2- 172.21.15.76:0/2437316467 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc67815ce70 0x7fc67815aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:23.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.743+0000 7fc67d9e2640 1 -- 172.21.15.76:0/2437316467 >> 172.21.15.76:0/2437316467 conn(0x7fc6780b0cd0 msgr2=0x7fc6780acf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:23.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.743+0000 7fc67d9e2640 1 -- 172.21.15.76:0/2437316467 shutdown_connections 2023-11-09T03:47:23.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.743+0000 7fc67d9e2640 1 -- 172.21.15.76:0/2437316467 wait complete. 2023-11-09T03:47:23.753 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:47:23.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.854+0000 7fb170eda640 1 Processor -- start 2023-11-09T03:47:23.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.854+0000 7fb170eda640 1 -- start start 2023-11-09T03:47:23.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.855+0000 7fb170eda640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb16c0cc490 0x7fb16c0cda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:23.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.855+0000 7fb170eda640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb16c070630 0x7fb16c070a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:23.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.855+0000 7fb170eda640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb16c070fd0 con 0x7fb16c0712f0 2023-11-09T03:47:23.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.855+0000 7fb170eda640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb16c0cc990 con 0x7fb16c0cc490 2023-11-09T03:47:23.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.855+0000 7fb170eda640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c0ccc70 con 0x7fb16c070630 2023-11-09T03:47:23.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.855+0000 7fb16a59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb16c0cc490 0x7fb16c0cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:23.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.855+0000 7fb163fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb16c0712f0 0x7fb16c0ca7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47978/0 (socket says 172.21.15.76:47978) 2023-11-09T03:47:23.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.855+0000 7fb163fff640 1 -- 172.21.15.76:0/3291837308 learned_addr learned my addr 172.21.15.76:0/3291837308 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:23.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.855+0000 7fb16a59c640 1 -- 172.21.15.76:0/3291837308 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb16c070630 msgr2=0x7fb16c070a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:23.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.855+0000 7fb16ad9d640 1 --2- 172.21.15.76:0/3291837308 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb16c070630 0x7fb16c070a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:23.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.855+0000 7fb16a59c640 1 --2- 172.21.15.76:0/3291837308 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb16c070630 0x7fb16c070a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:23.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.855+0000 7fb16a59c640 1 -- 172.21.15.76:0/3291837308 >> v1:172.21.15.76:6789/0 conn(0x7fb16c0712f0 legacy=0x7fb16c0ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:23.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.855+0000 7fb169d9b640 1 -- 172.21.15.76:0/3291837308 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824217 0 0) 0x7fb16c070fd0 con 0x7fb16c0712f0 2023-11-09T03:47:23.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.855+0000 7fb16a59c640 1 -- 172.21.15.76:0/3291837308 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb16c0c9ce0 con 0x7fb16c0cc490 2023-11-09T03:47:23.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.856+0000 7fb16a59c640 1 --2- 172.21.15.76:0/3291837308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb16c0cc490 0x7fb16c0cda60 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb15c0097a0 tx=0x7fb15c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff499c28764b4ce server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:23.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.856+0000 7fb169d9b640 1 -- 172.21.15.76:0/3291837308 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15c031830 con 0x7fb16c0cc490 2023-11-09T03:47:23.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.856+0000 7fb169d9b640 1 -- 172.21.15.76:0/3291837308 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb15c031990 con 0x7fb16c0cc490 2023-11-09T03:47:23.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.856+0000 7fb169d9b640 1 -- 172.21.15.76:0/3291837308 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15c031d20 con 0x7fb16c0cc490 2023-11-09T03:47:23.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.856+0000 7fb170eda640 1 -- 172.21.15.76:0/3291837308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb16c0cc490 msgr2=0x7fb16c0cda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:23.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.856+0000 7fb170eda640 1 --2- 172.21.15.76:0/3291837308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb16c0cc490 0x7fb16c0cda60 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb15c0097a0 tx=0x7fb15c02b8d0 comp rx=0 tx=0).stop 2023-11-09T03:47:23.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.857+0000 7fb170eda640 1 -- 172.21.15.76:0/3291837308 shutdown_connections 2023-11-09T03:47:23.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.857+0000 7fb170eda640 1 --2- 172.21.15.76:0/3291837308 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb16c070630 0x7fb16c070a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:23.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.857+0000 7fb170eda640 1 --2- 172.21.15.76:0/3291837308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb16c0cc490 0x7fb16c0cda60 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:23.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.857+0000 7fb170eda640 1 -- 172.21.15.76:0/3291837308 >> 172.21.15.76:0/3291837308 conn(0x7fb16c0ce950 msgr2=0x7fb16c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:23.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.857+0000 7fb170eda640 1 -- 172.21.15.76:0/3291837308 shutdown_connections 2023-11-09T03:47:23.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.857+0000 7fb170eda640 1 -- 172.21.15.76:0/3291837308 wait complete. 2023-11-09T03:47:23.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.858+0000 7fb170eda640 1 Processor -- start 2023-11-09T03:47:23.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.858+0000 7fb170eda640 1 -- start start 2023-11-09T03:47:23.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.859+0000 7fb170eda640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb16c070630 0x7fb16c15b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:23.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.859+0000 7fb170eda640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb16c0712f0 0x7fb16c0ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:23.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.859+0000 7fb170eda640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb16c0cc490 0x7fb16c0c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:23.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.859+0000 7fb170eda640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c0ac8f0 con 0x7fb16c0712f0 2023-11-09T03:47:23.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.859+0000 7fb170eda640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c15aac0 con 0x7fb16c070630 2023-11-09T03:47:23.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.859+0000 7fb170eda640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c15ad60 con 0x7fb16c0cc490 2023-11-09T03:47:23.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.859+0000 7fb163fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb16c0712f0 0x7fb16c0ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:23.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.859+0000 7fb163fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb16c0712f0 0x7fb16c0ac3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42494/0 (socket says 172.21.15.76:42494) 2023-11-09T03:47:23.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.859+0000 7fb163fff640 1 -- 172.21.15.76:0/839432033 learned_addr learned my addr 172.21.15.76:0/839432033 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:23.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.859+0000 7fb16a59c640 1 --2- 172.21.15.76:0/839432033 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb16c070630 0x7fb16c15b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:23.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.859+0000 7fb16ad9d640 1 --2- 172.21.15.76:0/839432033 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb16c0cc490 0x7fb16c0c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:23.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.859+0000 7fb163fff640 1 -- 172.21.15.76:0/839432033 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb16c0cc490 msgr2=0x7fb16c0c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:23.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.859+0000 7fb163fff640 1 --2- 172.21.15.76:0/839432033 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb16c0cc490 0x7fb16c0c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:23.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.859+0000 7fb163fff640 1 -- 172.21.15.76:0/839432033 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb16c070630 msgr2=0x7fb16c15b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:23.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.859+0000 7fb163fff640 1 --2- 172.21.15.76:0/839432033 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb16c070630 0x7fb16c15b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:23.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.859+0000 7fb163fff640 1 -- 172.21.15.76:0/839432033 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb15c009400 con 0x7fb16c0712f0 2023-11-09T03:47:23.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.859+0000 7fb16a59c640 1 --2- 172.21.15.76:0/839432033 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb16c070630 0x7fb16c15b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:23.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.859+0000 7fb163fff640 1 --2- 172.21.15.76:0/839432033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb16c0712f0 0x7fb16c0ac3b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb15400c4d0 tx=0x7fb15400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:23.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.859+0000 7fb162ffd640 1 -- 172.21.15.76:0/839432033 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1540074e0 con 0x7fb16c0712f0 2023-11-09T03:47:23.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.859+0000 7fb162ffd640 1 -- 172.21.15.76:0/839432033 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb154007660 con 0x7fb16c0712f0 2023-11-09T03:47:23.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.859+0000 7fb170eda640 1 -- 172.21.15.76:0/839432033 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb16c15b040 con 0x7fb16c0712f0 2023-11-09T03:47:23.869 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.859+0000 7fb162ffd640 1 -- 172.21.15.76:0/839432033 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb154004a20 con 0x7fb16c0712f0 2023-11-09T03:47:23.869 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.860+0000 7fb170eda640 1 -- 172.21.15.76:0/839432033 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb16c0acb00 con 0x7fb16c0712f0 2023-11-09T03:47:23.869 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.861+0000 7fb170eda640 1 -- 172.21.15.76:0/839432033 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb16c1565a0 con 0x7fb16c0712f0 2023-11-09T03:47:23.869 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.861+0000 7fb162ffd640 1 -- 172.21.15.76:0/839432033 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb154004b80 con 0x7fb16c0712f0 2023-11-09T03:47:23.869 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.861+0000 7fb162ffd640 1 --2- 172.21.15.76:0/839432033 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb14c06ae70 0x7fb14c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:23.870 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.861+0000 7fb162ffd640 1 -- 172.21.15.76:0/839432033 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb154088e10 con 0x7fb16c0712f0 2023-11-09T03:47:23.870 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.862+0000 7fb16a59c640 1 --2- 172.21.15.76:0/839432033 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb14c06ae70 0x7fb14c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:23.870 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.862+0000 7fb16a59c640 1 --2- 172.21.15.76:0/839432033 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb14c06ae70 0x7fb14c06d330 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb15c036000 tx=0x7fb15c03c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:23.871 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:23.866+0000 7fb162ffd640 1 -- 172.21.15.76:0/839432033 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb16c1565a0 con 0x7fb16c0712f0 2023-11-09T03:47:24.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.032+0000 7fb170eda640 1 -- 172.21.15.76:0/839432033 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb16c1565a0 con 0x7fb16c0712f0 2023-11-09T03:47:24.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.033+0000 7fb162ffd640 1 -- 172.21.15.76:0/839432033 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+18511 (secure 0 0 0) 0x7fb16c1565a0 con 0x7fb16c0712f0 2023-11-09T03:47:24.033 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:24.033 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":54,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:47:21.936243+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:47:18.807110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-09T03:47:19.235022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:47:24.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.035+0000 7fb170eda640 1 -- 172.21.15.76:0/839432033 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb14c06ae70 msgr2=0x7fb14c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:24.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.035+0000 7fb170eda640 1 --2- 172.21.15.76:0/839432033 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb14c06ae70 0x7fb14c06d330 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb15c036000 tx=0x7fb15c03c040 comp rx=0 tx=0).stop 2023-11-09T03:47:24.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.035+0000 7fb170eda640 1 -- 172.21.15.76:0/839432033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb16c0712f0 msgr2=0x7fb16c0ac3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:24.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.035+0000 7fb170eda640 1 --2- 172.21.15.76:0/839432033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb16c0712f0 0x7fb16c0ac3b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb15400c4d0 tx=0x7fb15400c990 comp rx=0 tx=0).stop 2023-11-09T03:47:24.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.035+0000 7fb170eda640 1 -- 172.21.15.76:0/839432033 shutdown_connections 2023-11-09T03:47:24.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.035+0000 7fb170eda640 1 --2- 172.21.15.76:0/839432033 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb14c06ae70 0x7fb14c06d330 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:24.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.035+0000 7fb170eda640 1 --2- 172.21.15.76:0/839432033 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb16c0cc490 0x7fb16c0c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:24.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.035+0000 7fb170eda640 1 --2- 172.21.15.76:0/839432033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb16c0712f0 0x7fb16c0ac3b0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:24.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.035+0000 7fb170eda640 1 --2- 172.21.15.76:0/839432033 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb16c070630 0x7fb16c15b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:24.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.035+0000 7fb170eda640 1 -- 172.21.15.76:0/839432033 >> 172.21.15.76:0/839432033 conn(0x7fb16c0ce950 msgr2=0x7fb16c0af580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:24.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.035+0000 7fb170eda640 1 -- 172.21.15.76:0/839432033 shutdown_connections 2023-11-09T03:47:24.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.035+0000 7fb170eda640 1 -- 172.21.15.76:0/839432033 wait complete. 2023-11-09T03:47:24.045 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-09T03:47:24.046 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-09T03:47:24.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.146+0000 7ffada365640 1 Processor -- start 2023-11-09T03:47:24.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.146+0000 7ffada365640 1 -- start start 2023-11-09T03:47:24.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.147+0000 7ffada365640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffad40b42c0 0x7ffad40c9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:24.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.147+0000 7ffada365640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffad40c9c40 0x7ffad40b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:24.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.147+0000 7ffada365640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad40ca090 con 0x7ffad415ad80 2023-11-09T03:47:24.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.147+0000 7ffada365640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffad406cfa0 con 0x7ffad40c9c40 2023-11-09T03:47:24.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.147+0000 7ffada365640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad406d280 con 0x7ffad40b42c0 2023-11-09T03:47:24.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.147+0000 7ffad8902640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffad40c9c40 0x7ffad40b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:24.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.147+0000 7ffad8902640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffad40c9c40 0x7ffad40b1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42510/0 (socket says 172.21.15.76:42510) 2023-11-09T03:47:24.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.147+0000 7ffad8902640 1 -- 172.21.15.76:0/2507823759 learned_addr learned my addr 172.21.15.76:0/2507823759 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:24.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.147+0000 7ffad37fe640 1 --2- 172.21.15.76:0/2507823759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffad40b42c0 0x7ffad40c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:24.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.147+0000 7ffad8902640 1 -- 172.21.15.76:0/2507823759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffad40b42c0 msgr2=0x7ffad40c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:24.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.147+0000 7ffad8902640 1 --2- 172.21.15.76:0/2507823759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffad40b42c0 0x7ffad40c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:24.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.148+0000 7ffad8902640 1 -- 172.21.15.76:0/2507823759 >> v1:172.21.15.113:6789/0 conn(0x7ffad415ad80 legacy=0x7ffad406c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:24.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.148+0000 7ffad8902640 1 -- 172.21.15.76:0/2507823759 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad406d560 con 0x7ffad40c9c40 2023-11-09T03:47:24.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.148+0000 7ffad8902640 1 --2- 172.21.15.76:0/2507823759 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffad40c9c40 0x7ffad40b1a40 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ffacc009e50 tx=0x7ffacc02bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92034992e39bdecf server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:24.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.148+0000 7ffad2ffd640 1 -- 172.21.15.76:0/2507823759 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc02cb90 con 0x7ffad40c9c40 2023-11-09T03:47:24.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.148+0000 7ffad2ffd640 1 -- 172.21.15.76:0/2507823759 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffacc02ccf0 con 0x7ffad40c9c40 2023-11-09T03:47:24.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.148+0000 7ffad2ffd640 1 -- 172.21.15.76:0/2507823759 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc034610 con 0x7ffad40c9c40 2023-11-09T03:47:24.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.149+0000 7ffada365640 1 -- 172.21.15.76:0/2507823759 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffad40c9c40 msgr2=0x7ffad40b1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:24.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.149+0000 7ffada365640 1 --2- 172.21.15.76:0/2507823759 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffad40c9c40 0x7ffad40b1a40 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ffacc009e50 tx=0x7ffacc02bff0 comp rx=0 tx=0).stop 2023-11-09T03:47:24.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.149+0000 7ffada365640 1 -- 172.21.15.76:0/2507823759 shutdown_connections 2023-11-09T03:47:24.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.149+0000 7ffada365640 1 --2- 172.21.15.76:0/2507823759 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffad40c9c40 0x7ffad40b1a40 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:24.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.149+0000 7ffada365640 1 --2- 172.21.15.76:0/2507823759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffad40b42c0 0x7ffad40c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:24.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.149+0000 7ffada365640 1 -- 172.21.15.76:0/2507823759 >> 172.21.15.76:0/2507823759 conn(0x7ffad40ce950 msgr2=0x7ffad40ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:24.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.149+0000 7ffada365640 1 -- 172.21.15.76:0/2507823759 shutdown_connections 2023-11-09T03:47:24.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.149+0000 7ffada365640 1 -- 172.21.15.76:0/2507823759 wait complete. 2023-11-09T03:47:24.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.150+0000 7ffada365640 1 Processor -- start 2023-11-09T03:47:24.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.150+0000 7ffada365640 1 -- start start 2023-11-09T03:47:24.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.151+0000 7ffada365640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffad40b42c0 0x7ffad40c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:24.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.151+0000 7ffada365640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffad40c9c40 0x7ffad4156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:24.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.151+0000 7ffada365640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffad415ad80 0x7ffad40a74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:24.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.151+0000 7ffada365640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad40c69f0 con 0x7ffad40c9c40 2023-11-09T03:47:24.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.151+0000 7ffada365640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad40c5d70 con 0x7ffad40b42c0 2023-11-09T03:47:24.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.151+0000 7ffada365640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad40c6050 con 0x7ffad415ad80 2023-11-09T03:47:24.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.151+0000 7ffad37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffad40c9c40 0x7ffad4156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:24.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.151+0000 7ffad37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffad40c9c40 0x7ffad4156580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42524/0 (socket says 172.21.15.76:42524) 2023-11-09T03:47:24.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.151+0000 7ffad37fe640 1 -- 172.21.15.76:0/3970868752 learned_addr learned my addr 172.21.15.76:0/3970868752 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:24.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.151+0000 7ffad3fff640 1 --2- 172.21.15.76:0/3970868752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffad40b42c0 0x7ffad40c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:24.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.151+0000 7ffad8902640 1 --2- 172.21.15.76:0/3970868752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffad415ad80 0x7ffad40a74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:24.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.151+0000 7ffad37fe640 1 -- 172.21.15.76:0/3970868752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffad415ad80 msgr2=0x7ffad40a74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:24.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.152+0000 7ffad37fe640 1 --2- 172.21.15.76:0/3970868752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffad415ad80 0x7ffad40a74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:24.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.152+0000 7ffad37fe640 1 -- 172.21.15.76:0/3970868752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffad40b42c0 msgr2=0x7ffad40c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:24.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.152+0000 7ffad37fe640 1 --2- 172.21.15.76:0/3970868752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffad40b42c0 0x7ffad40c8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:24.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.152+0000 7ffad37fe640 1 -- 172.21.15.76:0/3970868752 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffacc009b30 con 0x7ffad40c9c40 2023-11-09T03:47:24.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.152+0000 7ffad8902640 1 --2- 172.21.15.76:0/3970868752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffad415ad80 0x7ffad40a74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:24.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.152+0000 7ffad37fe640 1 --2- 172.21.15.76:0/3970868752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffad40c9c40 0x7ffad4156580 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ffac800d5d0 tx=0x7ffac800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:24.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.152+0000 7ffad0ff9640 1 -- 172.21.15.76:0/3970868752 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac80125f0 con 0x7ffad40c9c40 2023-11-09T03:47:24.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.152+0000 7ffad0ff9640 1 -- 172.21.15.76:0/3970868752 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac8012770 con 0x7ffad40c9c40 2023-11-09T03:47:24.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.152+0000 7ffad0ff9640 1 -- 172.21.15.76:0/3970868752 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac8012b20 con 0x7ffad40c9c40 2023-11-09T03:47:24.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.152+0000 7ffada365640 1 -- 172.21.15.76:0/3970868752 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad40c6330 con 0x7ffad40c9c40 2023-11-09T03:47:24.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.152+0000 7ffada365640 1 -- 172.21.15.76:0/3970868752 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad40c8b60 con 0x7ffad40c9c40 2023-11-09T03:47:24.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.153+0000 7ffada365640 1 -- 172.21.15.76:0/3970868752 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad4074290 con 0x7ffad40c9c40 2023-11-09T03:47:24.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.153+0000 7ffad0ff9640 1 -- 172.21.15.76:0/3970868752 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffac8012c80 con 0x7ffad40c9c40 2023-11-09T03:47:24.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.154+0000 7ffad0ff9640 1 --2- 172.21.15.76:0/3970868752 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ffac006ae70 0x7ffac006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:24.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.154+0000 7ffad0ff9640 1 -- 172.21.15.76:0/3970868752 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ffac8089d00 con 0x7ffad40c9c40 2023-11-09T03:47:24.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.154+0000 7ffad3fff640 1 --2- 172.21.15.76:0/3970868752 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ffac006ae70 0x7ffac006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:24.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.154+0000 7ffad3fff640 1 --2- 172.21.15.76:0/3970868752 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ffac006ae70 0x7ffac006d330 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ffac4000f60 tx=0x7ffac4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:24.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.158+0000 7ffad0ff9640 1 -- 172.21.15.76:0/3970868752 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffac8054ba0 con 0x7ffad40c9c40 2023-11-09T03:47:24.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.320+0000 7ffada365640 1 -- 172.21.15.76:0/3970868752 --> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffad40b2590 con 0x7ffac006ae70 2023-11-09T03:47:24.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.330+0000 7ffad0ff9640 1 -- 172.21.15.76:0/3970868752 <== mgr.4105 v2:172.21.15.113:6832/1264193042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+369232 (secure 0 0 0) 0x7ffad40b2590 con 0x7ffac006ae70 2023-11-09T03:47:24.333 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:24.336 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2023-11-09T03:47:24.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.333+0000 7ffada365640 1 -- 172.21.15.76:0/3970868752 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ffac006ae70 msgr2=0x7ffac006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:24.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.333+0000 7ffada365640 1 --2- 172.21.15.76:0/3970868752 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ffac006ae70 0x7ffac006d330 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ffac4000f60 tx=0x7ffac4009040 comp rx=0 tx=0).stop 2023-11-09T03:47:24.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.333+0000 7ffada365640 1 -- 172.21.15.76:0/3970868752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffad40c9c40 msgr2=0x7ffad4156580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:24.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.333+0000 7ffada365640 1 --2- 172.21.15.76:0/3970868752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffad40c9c40 0x7ffad4156580 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ffac800d5d0 tx=0x7ffac800da90 comp rx=0 tx=0).stop 2023-11-09T03:47:24.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.334+0000 7ffada365640 1 -- 172.21.15.76:0/3970868752 shutdown_connections 2023-11-09T03:47:24.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.334+0000 7ffada365640 1 --2- 172.21.15.76:0/3970868752 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ffac006ae70 0x7ffac006d330 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:24.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.334+0000 7ffada365640 1 --2- 172.21.15.76:0/3970868752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffad415ad80 0x7ffad40a74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:24.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.334+0000 7ffada365640 1 --2- 172.21.15.76:0/3970868752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffad40c9c40 0x7ffad4156580 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:24.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.334+0000 7ffada365640 1 --2- 172.21.15.76:0/3970868752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffad40b42c0 0x7ffad40c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:24.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.334+0000 7ffada365640 1 -- 172.21.15.76:0/3970868752 >> 172.21.15.76:0/3970868752 conn(0x7ffad40ce950 msgr2=0x7ffad40b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:24.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.334+0000 7ffada365640 1 -- 172.21.15.76:0/3970868752 shutdown_connections 2023-11-09T03:47:24.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.334+0000 7ffada365640 1 -- 172.21.15.76:0/3970868752 wait complete. 2023-11-09T03:47:24.344 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2023-11-09T03:47:22.777354+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":246,"acting":246,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":217,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222364,"kb_used_data":6172,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752356,"statfs":{"total":773094113280,"available":772866412544,"internally_reserved":0,"allocated":6320128,"data_stored":3471228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001064"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248379+0000","last_change":"2023-11-09T03:47:20.941252+0000","last_active":"2023-11-09T03:47:22.248379+0000","last_peered":"2023-11-09T03:47:22.248379+0000","last_clean":"2023-11-09T03:47:22.248379+0000","last_became_active":"2023-11-09T03:47:20.940995+0000","last_became_peered":"2023-11-09T03:47:20.940995+0000","last_unstale":"2023-11-09T03:47:22.248379+0000","last_undegraded":"2023-11-09T03:47:22.248379+0000","last_fullsized":"2023-11-09T03:47:22.248379+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:44:27.004461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248080+0000","last_change":"2023-11-09T03:47:20.950878+0000","last_active":"2023-11-09T03:47:22.248080+0000","last_peered":"2023-11-09T03:47:22.248080+0000","last_clean":"2023-11-09T03:47:22.248080+0000","last_became_active":"2023-11-09T03:47:20.950611+0000","last_became_peered":"2023-11-09T03:47:20.950611+0000","last_unstale":"2023-11-09T03:47:22.248080+0000","last_undegraded":"2023-11-09T03:47:22.248080+0000","last_fullsized":"2023-11-09T03:47:22.248080+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:04:18.340460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026954+0000","last_change":"2023-11-09T03:47:20.945634+0000","last_active":"2023-11-09T03:47:22.026954+0000","last_peered":"2023-11-09T03:47:22.026954+0000","last_clean":"2023-11-09T03:47:22.026954+0000","last_became_active":"2023-11-09T03:47:20.945370+0000","last_became_peered":"2023-11-09T03:47:20.945370+0000","last_unstale":"2023-11-09T03:47:22.026954+0000","last_undegraded":"2023-11-09T03:47:22.026954+0000","last_fullsized":"2023-11-09T03:47:22.026954+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:57.187676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940818+0000","last_change":"2023-11-09T03:47:20.941855+0000","last_active":"2023-11-09T03:47:21.940818+0000","last_peered":"2023-11-09T03:47:21.940818+0000","last_clean":"2023-11-09T03:47:21.940818+0000","last_became_active":"2023-11-09T03:47:20.941598+0000","last_became_peered":"2023-11-09T03:47:20.941598+0000","last_unstale":"2023-11-09T03:47:21.940818+0000","last_undegraded":"2023-11-09T03:47:21.940818+0000","last_fullsized":"2023-11-09T03:47:21.940818+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:14:00.724882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248512+0000","last_change":"2023-11-09T03:47:20.947154+0000","last_active":"2023-11-09T03:47:22.248512+0000","last_peered":"2023-11-09T03:47:22.248512+0000","last_clean":"2023-11-09T03:47:22.248512+0000","last_became_active":"2023-11-09T03:47:20.946878+0000","last_became_peered":"2023-11-09T03:47:20.946878+0000","last_unstale":"2023-11-09T03:47:22.248512+0000","last_undegraded":"2023-11-09T03:47:22.248512+0000","last_fullsized":"2023-11-09T03:47:22.248512+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:41:16.327295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247898+0000","last_change":"2023-11-09T03:47:20.941698+0000","last_active":"2023-11-09T03:47:22.247898+0000","last_peered":"2023-11-09T03:47:22.247898+0000","last_clean":"2023-11-09T03:47:22.247898+0000","last_became_active":"2023-11-09T03:47:20.941432+0000","last_became_peered":"2023-11-09T03:47:20.941432+0000","last_unstale":"2023-11-09T03:47:22.247898+0000","last_undegraded":"2023-11-09T03:47:22.247898+0000","last_fullsized":"2023-11-09T03:47:22.247898+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:12:48.690396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247980+0000","last_change":"2023-11-09T03:47:20.948132+0000","last_active":"2023-11-09T03:47:22.247980+0000","last_peered":"2023-11-09T03:47:22.247980+0000","last_clean":"2023-11-09T03:47:22.247980+0000","last_became_active":"2023-11-09T03:47:20.947854+0000","last_became_peered":"2023-11-09T03:47:20.947854+0000","last_unstale":"2023-11-09T03:47:22.247980+0000","last_undegraded":"2023-11-09T03:47:22.247980+0000","last_fullsized":"2023-11-09T03:47:22.247980+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:38:09.677347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248627+0000","last_change":"2023-11-09T03:47:20.944502+0000","last_active":"2023-11-09T03:47:22.248627+0000","last_peered":"2023-11-09T03:47:22.248627+0000","last_clean":"2023-11-09T03:47:22.248627+0000","last_became_active":"2023-11-09T03:47:20.944214+0000","last_became_peered":"2023-11-09T03:47:20.944214+0000","last_unstale":"2023-11-09T03:47:22.248627+0000","last_undegraded":"2023-11-09T03:47:22.248627+0000","last_fullsized":"2023-11-09T03:47:22.248627+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:22:14.432893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248644+0000","last_change":"2023-11-09T03:47:20.954329+0000","last_active":"2023-11-09T03:47:22.248644+0000","last_peered":"2023-11-09T03:47:22.248644+0000","last_clean":"2023-11-09T03:47:22.248644+0000","last_became_active":"2023-11-09T03:47:20.953654+0000","last_became_peered":"2023-11-09T03:47:20.953654+0000","last_unstale":"2023-11-09T03:47:22.248644+0000","last_undegraded":"2023-11-09T03:47:22.248644+0000","last_fullsized":"2023-11-09T03:47:22.248644+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:48:27.582736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939384+0000","last_change":"2023-11-09T03:47:20.943196+0000","last_active":"2023-11-09T03:47:21.939384+0000","last_peered":"2023-11-09T03:47:21.939384+0000","last_clean":"2023-11-09T03:47:21.939384+0000","last_became_active":"2023-11-09T03:47:20.942943+0000","last_became_peered":"2023-11-09T03:47:20.942943+0000","last_unstale":"2023-11-09T03:47:21.939384+0000","last_undegraded":"2023-11-09T03:47:21.939384+0000","last_fullsized":"2023-11-09T03:47:21.939384+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:36:04.069418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939364+0000","last_change":"2023-11-09T03:47:20.943821+0000","last_active":"2023-11-09T03:47:21.939364+0000","last_peered":"2023-11-09T03:47:21.939364+0000","last_clean":"2023-11-09T03:47:21.939364+0000","last_became_active":"2023-11-09T03:47:20.943571+0000","last_became_peered":"2023-11-09T03:47:20.943571+0000","last_unstale":"2023-11-09T03:47:21.939364+0000","last_undegraded":"2023-11-09T03:47:21.939364+0000","last_fullsized":"2023-11-09T03:47:21.939364+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:26:12.103741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940598+0000","last_change":"2023-11-09T03:47:20.942582+0000","last_active":"2023-11-09T03:47:21.940598+0000","last_peered":"2023-11-09T03:47:21.940598+0000","last_clean":"2023-11-09T03:47:21.940598+0000","last_became_active":"2023-11-09T03:47:20.942214+0000","last_became_peered":"2023-11-09T03:47:20.942214+0000","last_unstale":"2023-11-09T03:47:21.940598+0000","last_undegraded":"2023-11-09T03:47:21.940598+0000","last_fullsized":"2023-11-09T03:47:21.940598+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:51:53.879857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-09T03:47:19.915800+0000","last_change":"2023-11-09T03:47:19.915800+0000","last_active":"2023-11-09T03:47:19.915800+0000","last_peered":"2023-11-09T03:47:19.915800+0000","last_clean":"2023-11-09T03:47:19.915800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:47:19.915800+0000","last_undegraded":"2023-11-09T03:47:19.915800+0000","last_fullsized":"2023-11-09T03:47:19.915800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248066+0000","last_change":"2023-11-09T03:47:20.944425+0000","last_active":"2023-11-09T03:47:22.248066+0000","last_peered":"2023-11-09T03:47:22.248066+0000","last_clean":"2023-11-09T03:47:22.248066+0000","last_became_active":"2023-11-09T03:47:20.944159+0000","last_became_peered":"2023-11-09T03:47:20.944159+0000","last_unstale":"2023-11-09T03:47:22.248066+0000","last_undegraded":"2023-11-09T03:47:22.248066+0000","last_fullsized":"2023-11-09T03:47:22.248066+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:26:12.103741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248162+0000","last_change":"2023-11-09T03:47:20.954225+0000","last_active":"2023-11-09T03:47:22.248162+0000","last_peered":"2023-11-09T03:47:22.248162+0000","last_clean":"2023-11-09T03:47:22.248162+0000","last_became_active":"2023-11-09T03:47:20.954004+0000","last_became_peered":"2023-11-09T03:47:20.954004+0000","last_unstale":"2023-11-09T03:47:22.248162+0000","last_undegraded":"2023-11-09T03:47:22.248162+0000","last_fullsized":"2023-11-09T03:47:22.248162+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:36:04.069418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247697+0000","last_change":"2023-11-09T03:47:20.940209+0000","last_active":"2023-11-09T03:47:22.247697+0000","last_peered":"2023-11-09T03:47:22.247697+0000","last_clean":"2023-11-09T03:47:22.247697+0000","last_became_active":"2023-11-09T03:47:20.939956+0000","last_became_peered":"2023-11-09T03:47:20.939956+0000","last_unstale":"2023-11-09T03:47:22.247697+0000","last_undegraded":"2023-11-09T03:47:22.247697+0000","last_fullsized":"2023-11-09T03:47:22.247697+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:52.683555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248360+0000","last_change":"2023-11-09T03:47:20.942850+0000","last_active":"2023-11-09T03:47:22.248360+0000","last_peered":"2023-11-09T03:47:22.248360+0000","last_clean":"2023-11-09T03:47:22.248360+0000","last_became_active":"2023-11-09T03:47:20.942596+0000","last_became_peered":"2023-11-09T03:47:20.942596+0000","last_unstale":"2023-11-09T03:47:22.248360+0000","last_undegraded":"2023-11-09T03:47:22.248360+0000","last_fullsized":"2023-11-09T03:47:22.248360+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:14:28.441176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.027085+0000","last_change":"2023-11-09T03:47:20.954037+0000","last_active":"2023-11-09T03:47:22.027085+0000","last_peered":"2023-11-09T03:47:22.027085+0000","last_clean":"2023-11-09T03:47:22.027085+0000","last_became_active":"2023-11-09T03:47:20.953777+0000","last_became_peered":"2023-11-09T03:47:20.953777+0000","last_unstale":"2023-11-09T03:47:22.027085+0000","last_undegraded":"2023-11-09T03:47:22.027085+0000","last_fullsized":"2023-11-09T03:47:22.027085+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:55:28.077165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248233+0000","last_change":"2023-11-09T03:47:20.941849+0000","last_active":"2023-11-09T03:47:22.248233+0000","last_peered":"2023-11-09T03:47:22.248233+0000","last_clean":"2023-11-09T03:47:22.248233+0000","last_became_active":"2023-11-09T03:47:20.941602+0000","last_became_peered":"2023-11-09T03:47:20.941602+0000","last_unstale":"2023-11-09T03:47:22.248233+0000","last_undegraded":"2023-11-09T03:47:22.248233+0000","last_fullsized":"2023-11-09T03:47:22.248233+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:23.398916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248619+0000","last_change":"2023-11-09T03:47:20.951388+0000","last_active":"2023-11-09T03:47:22.248619+0000","last_peered":"2023-11-09T03:47:22.248619+0000","last_clean":"2023-11-09T03:47:22.248619+0000","last_became_active":"2023-11-09T03:47:20.951123+0000","last_became_peered":"2023-11-09T03:47:20.951123+0000","last_unstale":"2023-11-09T03:47:22.248619+0000","last_undegraded":"2023-11-09T03:47:22.248619+0000","last_fullsized":"2023-11-09T03:47:22.248619+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:55:54.333416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026719+0000","last_change":"2023-11-09T03:47:20.948114+0000","last_active":"2023-11-09T03:47:22.026719+0000","last_peered":"2023-11-09T03:47:22.026719+0000","last_clean":"2023-11-09T03:47:22.026719+0000","last_became_active":"2023-11-09T03:47:20.947849+0000","last_became_peered":"2023-11-09T03:47:20.947849+0000","last_unstale":"2023-11-09T03:47:22.026719+0000","last_undegraded":"2023-11-09T03:47:22.026719+0000","last_fullsized":"2023-11-09T03:47:22.026719+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:52.683555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247955+0000","last_change":"2023-11-09T03:47:20.945723+0000","last_active":"2023-11-09T03:47:22.247955+0000","last_peered":"2023-11-09T03:47:22.247955+0000","last_clean":"2023-11-09T03:47:22.247955+0000","last_became_active":"2023-11-09T03:47:20.945447+0000","last_became_peered":"2023-11-09T03:47:20.945447+0000","last_unstale":"2023-11-09T03:47:22.247955+0000","last_undegraded":"2023-11-09T03:47:22.247955+0000","last_fullsized":"2023-11-09T03:47:22.247955+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:41:48.124870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026634+0000","last_change":"2023-11-09T03:47:20.944344+0000","last_active":"2023-11-09T03:47:22.026634+0000","last_peered":"2023-11-09T03:47:22.026634+0000","last_clean":"2023-11-09T03:47:22.026634+0000","last_became_active":"2023-11-09T03:47:20.943991+0000","last_became_peered":"2023-11-09T03:47:20.943991+0000","last_unstale":"2023-11-09T03:47:22.026634+0000","last_undegraded":"2023-11-09T03:47:22.026634+0000","last_fullsized":"2023-11-09T03:47:22.026634+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:08:17.239107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-09T03:47:19.915800+0000","last_change":"2023-11-09T03:47:19.915800+0000","last_active":"2023-11-09T03:47:19.915800+0000","last_peered":"2023-11-09T03:47:19.915800+0000","last_clean":"2023-11-09T03:47:19.915800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:47:19.915800+0000","last_undegraded":"2023-11-09T03:47:19.915800+0000","last_fullsized":"2023-11-09T03:47:19.915800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248496+0000","last_change":"2023-11-09T03:47:20.952835+0000","last_active":"2023-11-09T03:47:22.248496+0000","last_peered":"2023-11-09T03:47:22.248496+0000","last_clean":"2023-11-09T03:47:22.248496+0000","last_became_active":"2023-11-09T03:47:20.952578+0000","last_became_peered":"2023-11-09T03:47:20.952578+0000","last_unstale":"2023-11-09T03:47:22.248496+0000","last_undegraded":"2023-11-09T03:47:22.248496+0000","last_fullsized":"2023-11-09T03:47:22.248496+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:40:53.848669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.941032+0000","last_change":"2023-11-09T03:47:20.944379+0000","last_active":"2023-11-09T03:47:21.941032+0000","last_peered":"2023-11-09T03:47:21.941032+0000","last_clean":"2023-11-09T03:47:21.941032+0000","last_became_active":"2023-11-09T03:47:20.944134+0000","last_became_peered":"2023-11-09T03:47:20.944134+0000","last_unstale":"2023-11-09T03:47:21.941032+0000","last_undegraded":"2023-11-09T03:47:21.941032+0000","last_fullsized":"2023-11-09T03:47:21.941032+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:55:28.077165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248970+0000","last_change":"2023-11-09T03:47:20.940786+0000","last_active":"2023-11-09T03:47:22.248970+0000","last_peered":"2023-11-09T03:47:22.248970+0000","last_clean":"2023-11-09T03:47:22.248970+0000","last_became_active":"2023-11-09T03:47:20.940529+0000","last_became_peered":"2023-11-09T03:47:20.940529+0000","last_unstale":"2023-11-09T03:47:22.248970+0000","last_undegraded":"2023-11-09T03:47:22.248970+0000","last_fullsized":"2023-11-09T03:47:22.248970+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:40.639821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248129+0000","last_change":"2023-11-09T03:47:20.949337+0000","last_active":"2023-11-09T03:47:22.248129+0000","last_peered":"2023-11-09T03:47:22.248129+0000","last_clean":"2023-11-09T03:47:22.248129+0000","last_became_active":"2023-11-09T03:47:20.949088+0000","last_became_peered":"2023-11-09T03:47:20.949088+0000","last_unstale":"2023-11-09T03:47:22.248129+0000","last_undegraded":"2023-11-09T03:47:22.248129+0000","last_fullsized":"2023-11-09T03:47:22.248129+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:52:12.660646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248774+0000","last_change":"2023-11-09T03:47:20.948003+0000","last_active":"2023-11-09T03:47:22.248774+0000","last_peered":"2023-11-09T03:47:22.248774+0000","last_clean":"2023-11-09T03:47:22.248774+0000","last_became_active":"2023-11-09T03:47:20.947690+0000","last_became_peered":"2023-11-09T03:47:20.947690+0000","last_unstale":"2023-11-09T03:47:22.248774+0000","last_undegraded":"2023-11-09T03:47:22.248774+0000","last_fullsized":"2023-11-09T03:47:22.248774+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:28:18.013453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026610+0000","last_change":"2023-11-09T03:47:20.946260+0000","last_active":"2023-11-09T03:47:22.026610+0000","last_peered":"2023-11-09T03:47:22.026610+0000","last_clean":"2023-11-09T03:47:22.026610+0000","last_became_active":"2023-11-09T03:47:20.945986+0000","last_became_peered":"2023-11-09T03:47:20.945986+0000","last_unstale":"2023-11-09T03:47:22.026610+0000","last_undegraded":"2023-11-09T03:47:22.026610+0000","last_fullsized":"2023-11-09T03:47:22.026610+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:59:29.154623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247961+0000","last_change":"2023-11-09T03:47:20.952176+0000","last_active":"2023-11-09T03:47:22.247961+0000","last_peered":"2023-11-09T03:47:22.247961+0000","last_clean":"2023-11-09T03:47:22.247961+0000","last_became_active":"2023-11-09T03:47:20.951921+0000","last_became_peered":"2023-11-09T03:47:20.951921+0000","last_unstale":"2023-11-09T03:47:22.247961+0000","last_undegraded":"2023-11-09T03:47:22.247961+0000","last_fullsized":"2023-11-09T03:47:22.247961+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:22.376023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247669+0000","last_change":"2023-11-09T03:47:20.943940+0000","last_active":"2023-11-09T03:47:22.247669+0000","last_peered":"2023-11-09T03:47:22.247669+0000","last_clean":"2023-11-09T03:47:22.247669+0000","last_became_active":"2023-11-09T03:47:20.943708+0000","last_became_peered":"2023-11-09T03:47:20.943708+0000","last_unstale":"2023-11-09T03:47:22.247669+0000","last_undegraded":"2023-11-09T03:47:22.247669+0000","last_fullsized":"2023-11-09T03:47:22.247669+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:08:17.239107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026790+0000","last_change":"2023-11-09T03:47:20.942360+0000","last_active":"2023-11-09T03:47:22.026790+0000","last_peered":"2023-11-09T03:47:22.026790+0000","last_clean":"2023-11-09T03:47:22.026790+0000","last_became_active":"2023-11-09T03:47:20.942078+0000","last_became_peered":"2023-11-09T03:47:20.942078+0000","last_unstale":"2023-11-09T03:47:22.026790+0000","last_undegraded":"2023-11-09T03:47:22.026790+0000","last_fullsized":"2023-11-09T03:47:22.026790+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:12:48.690396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-09T03:47:18.911117+0000","last_change":"2023-11-09T03:47:18.911117+0000","last_active":"2023-11-09T03:47:18.911117+0000","last_peered":"2023-11-09T03:47:18.911117+0000","last_clean":"2023-11-09T03:47:18.911117+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:47:18.911117+0000","last_undegraded":"2023-11-09T03:47:18.911117+0000","last_fullsized":"2023-11-09T03:47:18.911117+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":27,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.928645+0000","last_change":"2023-11-09T03:47:02.581917+0000","last_active":"2023-11-09T03:47:20.928645+0000","last_peered":"2023-11-09T03:47:20.928645+0000","last_clean":"2023-11-09T03:47:20.928645+0000","last_became_active":"2023-11-09T03:47:01.588082+0000","last_became_peered":"2023-11-09T03:47:01.588082+0000","last_unstale":"2023-11-09T03:47:20.928645+0000","last_undegraded":"2023-11-09T03:47:20.928645+0000","last_fullsized":"2023-11-09T03:47:20.928645+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:44:07.653557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034538899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940245+0000","last_change":"2023-11-09T03:47:20.938900+0000","last_active":"2023-11-09T03:47:21.940245+0000","last_peered":"2023-11-09T03:47:21.940245+0000","last_clean":"2023-11-09T03:47:21.940245+0000","last_became_active":"2023-11-09T03:47:20.938654+0000","last_became_peered":"2023-11-09T03:47:20.938654+0000","last_unstale":"2023-11-09T03:47:21.940245+0000","last_undegraded":"2023-11-09T03:47:21.940245+0000","last_fullsized":"2023-11-09T03:47:21.940245+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:40.639821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248651+0000","last_change":"2023-11-09T03:47:19.945834+0000","last_active":"2023-11-09T03:47:22.248651+0000","last_peered":"2023-11-09T03:47:22.248651+0000","last_clean":"2023-11-09T03:47:22.248651+0000","last_became_active":"2023-11-09T03:47:19.945484+0000","last_became_peered":"2023-11-09T03:47:19.945484+0000","last_unstale":"2023-11-09T03:47:22.248651+0000","last_undegraded":"2023-11-09T03:47:22.248651+0000","last_fullsized":"2023-11-09T03:47:22.248651+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:57:21.887088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.930614+0000","last_change":"2023-11-09T03:47:03.029063+0000","last_active":"2023-11-09T03:47:20.930614+0000","last_peered":"2023-11-09T03:47:20.930614+0000","last_clean":"2023-11-09T03:47:20.930614+0000","last_became_active":"2023-11-09T03:47:01.593582+0000","last_became_peered":"2023-11-09T03:47:01.593582+0000","last_unstale":"2023-11-09T03:47:20.930614+0000","last_undegraded":"2023-11-09T03:47:20.930614+0000","last_fullsized":"2023-11-09T03:47:20.930614+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:56:20.649937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041855100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026721+0000","last_change":"2023-11-09T03:47:20.944419+0000","last_active":"2023-11-09T03:47:22.026721+0000","last_peered":"2023-11-09T03:47:22.026721+0000","last_clean":"2023-11-09T03:47:22.026721+0000","last_became_active":"2023-11-09T03:47:20.944141+0000","last_became_peered":"2023-11-09T03:47:20.944141+0000","last_unstale":"2023-11-09T03:47:22.026721+0000","last_undegraded":"2023-11-09T03:47:22.026721+0000","last_fullsized":"2023-11-09T03:47:22.026721+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:06:17.676049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"53'8","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026516+0000","last_change":"2023-11-09T03:47:19.946625+0000","last_active":"2023-11-09T03:47:22.026516+0000","last_peered":"2023-11-09T03:47:22.026516+0000","last_clean":"2023-11-09T03:47:22.026516+0000","last_became_active":"2023-11-09T03:47:19.946350+0000","last_became_peered":"2023-11-09T03:47:19.946350+0000","last_unstale":"2023-11-09T03:47:22.026516+0000","last_undegraded":"2023-11-09T03:47:22.026516+0000","last_fullsized":"2023-11-09T03:47:22.026516+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:32.646643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":21,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-09T03:47:15.976974+0000","last_change":"2023-11-09T03:47:02.619336+0000","last_active":"2023-11-09T03:47:15.976974+0000","last_peered":"2023-11-09T03:47:15.976974+0000","last_clean":"2023-11-09T03:47:15.976974+0000","last_became_active":"2023-11-09T03:47:01.587749+0000","last_became_peered":"2023-11-09T03:47:01.587749+0000","last_unstale":"2023-11-09T03:47:15.976974+0000","last_undegraded":"2023-11-09T03:47:15.976974+0000","last_fullsized":"2023-11-09T03:47:15.976974+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063032099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247912+0000","last_change":"2023-11-09T03:47:20.948391+0000","last_active":"2023-11-09T03:47:22.247912+0000","last_peered":"2023-11-09T03:47:22.247912+0000","last_clean":"2023-11-09T03:47:22.247912+0000","last_became_active":"2023-11-09T03:47:20.948103+0000","last_became_peered":"2023-11-09T03:47:20.948103+0000","last_unstale":"2023-11-09T03:47:22.247912+0000","last_undegraded":"2023-11-09T03:47:22.247912+0000","last_fullsized":"2023-11-09T03:47:22.247912+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:33.651326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940202+0000","last_change":"2023-11-09T03:47:19.944514+0000","last_active":"2023-11-09T03:47:21.940202+0000","last_peered":"2023-11-09T03:47:21.940202+0000","last_clean":"2023-11-09T03:47:21.940202+0000","last_became_active":"2023-11-09T03:47:19.944230+0000","last_became_peered":"2023-11-09T03:47:19.944230+0000","last_unstale":"2023-11-09T03:47:21.940202+0000","last_undegraded":"2023-11-09T03:47:21.940202+0000","last_fullsized":"2023-11-09T03:47:21.940202+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:08:16.234424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":27,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.928406+0000","last_change":"2023-11-09T03:47:02.581658+0000","last_active":"2023-11-09T03:47:20.928406+0000","last_peered":"2023-11-09T03:47:20.928406+0000","last_clean":"2023-11-09T03:47:20.928406+0000","last_became_active":"2023-11-09T03:47:01.588612+0000","last_became_peered":"2023-11-09T03:47:01.588612+0000","last_unstale":"2023-11-09T03:47:20.928406+0000","last_undegraded":"2023-11-09T03:47:20.928406+0000","last_fullsized":"2023-11-09T03:47:20.928406+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045238500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.027066+0000","last_change":"2023-11-09T03:47:20.952326+0000","last_active":"2023-11-09T03:47:22.027066+0000","last_peered":"2023-11-09T03:47:22.027066+0000","last_clean":"2023-11-09T03:47:22.027066+0000","last_became_active":"2023-11-09T03:47:20.952035+0000","last_became_peered":"2023-11-09T03:47:20.952035+0000","last_unstale":"2023-11-09T03:47:22.027066+0000","last_undegraded":"2023-11-09T03:47:22.027066+0000","last_fullsized":"2023-11-09T03:47:22.027066+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:14:00.724882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248147+0000","last_change":"2023-11-09T03:47:19.947282+0000","last_active":"2023-11-09T03:47:22.248147+0000","last_peered":"2023-11-09T03:47:22.248147+0000","last_clean":"2023-11-09T03:47:22.248147+0000","last_became_active":"2023-11-09T03:47:19.947034+0000","last_became_peered":"2023-11-09T03:47:19.947034+0000","last_unstale":"2023-11-09T03:47:22.248147+0000","last_undegraded":"2023-11-09T03:47:22.248147+0000","last_fullsized":"2023-11-09T03:47:22.248147+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:13:59.720199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":21,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-09T03:47:15.977118+0000","last_change":"2023-11-09T03:47:02.618828+0000","last_active":"2023-11-09T03:47:15.977118+0000","last_peered":"2023-11-09T03:47:15.977118+0000","last_clean":"2023-11-09T03:47:15.977118+0000","last_became_active":"2023-11-09T03:47:01.590679+0000","last_became_peered":"2023-11-09T03:47:01.590679+0000","last_unstale":"2023-11-09T03:47:15.977118+0000","last_undegraded":"2023-11-09T03:47:15.977118+0000","last_fullsized":"2023-11-09T03:47:15.977118+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:06:30.982987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070668,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248700+0000","last_change":"2023-11-09T03:47:20.954224+0000","last_active":"2023-11-09T03:47:22.248700+0000","last_peered":"2023-11-09T03:47:22.248700+0000","last_clean":"2023-11-09T03:47:22.248700+0000","last_became_active":"2023-11-09T03:47:20.953950+0000","last_became_peered":"2023-11-09T03:47:20.953950+0000","last_unstale":"2023-11-09T03:47:22.248700+0000","last_undegraded":"2023-11-09T03:47:22.248700+0000","last_fullsized":"2023-11-09T03:47:22.248700+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:58:09.769432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.027101+0000","last_change":"2023-11-09T03:47:19.947929+0000","last_active":"2023-11-09T03:47:22.027101+0000","last_peered":"2023-11-09T03:47:22.027101+0000","last_clean":"2023-11-09T03:47:22.027101+0000","last_became_active":"2023-11-09T03:47:19.947663+0000","last_became_peered":"2023-11-09T03:47:19.947663+0000","last_unstale":"2023-11-09T03:47:22.027101+0000","last_undegraded":"2023-11-09T03:47:22.027101+0000","last_fullsized":"2023-11-09T03:47:22.027101+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:02:12.613580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931259+0000","last_change":"2023-11-09T03:47:03.146528+0000","last_active":"2023-11-09T03:47:20.931259+0000","last_peered":"2023-11-09T03:47:20.931259+0000","last_clean":"2023-11-09T03:47:20.931259+0000","last_became_active":"2023-11-09T03:47:01.587060+0000","last_became_peered":"2023-11-09T03:47:01.587060+0000","last_unstale":"2023-11-09T03:47:20.931259+0000","last_undegraded":"2023-11-09T03:47:20.931259+0000","last_fullsized":"2023-11-09T03:47:20.931259+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:39:15.238496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079064999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.027243+0000","last_change":"2023-11-09T03:47:20.954274+0000","last_active":"2023-11-09T03:47:22.027243+0000","last_peered":"2023-11-09T03:47:22.027243+0000","last_clean":"2023-11-09T03:47:22.027243+0000","last_became_active":"2023-11-09T03:47:20.953976+0000","last_became_peered":"2023-11-09T03:47:20.953976+0000","last_unstale":"2023-11-09T03:47:22.027243+0000","last_undegraded":"2023-11-09T03:47:22.027243+0000","last_fullsized":"2023-11-09T03:47:22.027243+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:05:32.662676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"53'6","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248315+0000","last_change":"2023-11-09T03:47:19.948464+0000","last_active":"2023-11-09T03:47:22.248315+0000","last_peered":"2023-11-09T03:47:22.248315+0000","last_clean":"2023-11-09T03:47:22.248315+0000","last_became_active":"2023-11-09T03:47:19.948254+0000","last_became_peered":"2023-11-09T03:47:19.948254+0000","last_unstale":"2023-11-09T03:47:22.248315+0000","last_undegraded":"2023-11-09T03:47:22.248315+0000","last_fullsized":"2023-11-09T03:47:22.248315+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:21:03.454473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931191+0000","last_change":"2023-11-09T03:47:03.148246+0000","last_active":"2023-11-09T03:47:20.931191+0000","last_peered":"2023-11-09T03:47:20.931191+0000","last_clean":"2023-11-09T03:47:20.931191+0000","last_became_active":"2023-11-09T03:47:01.590988+0000","last_became_peered":"2023-11-09T03:47:01.590988+0000","last_unstale":"2023-11-09T03:47:20.931191+0000","last_undegraded":"2023-11-09T03:47:20.931191+0000","last_fullsized":"2023-11-09T03:47:20.931191+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:07:20.785445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055651099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939172+0000","last_change":"2023-11-09T03:47:20.945226+0000","last_active":"2023-11-09T03:47:21.939172+0000","last_peered":"2023-11-09T03:47:21.939172+0000","last_clean":"2023-11-09T03:47:21.939172+0000","last_became_active":"2023-11-09T03:47:20.944978+0000","last_became_peered":"2023-11-09T03:47:20.944978+0000","last_unstale":"2023-11-09T03:47:21.939172+0000","last_undegraded":"2023-11-09T03:47:21.939172+0000","last_fullsized":"2023-11-09T03:47:21.939172+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:33.651326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"53'3","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026995+0000","last_change":"2023-11-09T03:47:19.945101+0000","last_active":"2023-11-09T03:47:22.026995+0000","last_peered":"2023-11-09T03:47:22.026995+0000","last_clean":"2023-11-09T03:47:22.026995+0000","last_became_active":"2023-11-09T03:47:19.944829+0000","last_became_peered":"2023-11-09T03:47:19.944829+0000","last_unstale":"2023-11-09T03:47:22.026995+0000","last_undegraded":"2023-11-09T03:47:22.026995+0000","last_fullsized":"2023-11-09T03:47:22.026995+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:49:12.247865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.939514+0000","last_change":"2023-11-09T03:47:02.582754+0000","last_active":"2023-11-09T03:47:20.939514+0000","last_peered":"2023-11-09T03:47:20.939514+0000","last_clean":"2023-11-09T03:47:20.939514+0000","last_became_active":"2023-11-09T03:47:01.589715+0000","last_became_peered":"2023-11-09T03:47:01.589715+0000","last_unstale":"2023-11-09T03:47:20.939514+0000","last_undegraded":"2023-11-09T03:47:20.939514+0000","last_fullsized":"2023-11-09T03:47:20.939514+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099342799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248560+0000","last_change":"2023-11-09T03:47:20.953974+0000","last_active":"2023-11-09T03:47:22.248560+0000","last_peered":"2023-11-09T03:47:22.248560+0000","last_clean":"2023-11-09T03:47:22.248560+0000","last_became_active":"2023-11-09T03:47:20.953714+0000","last_became_peered":"2023-11-09T03:47:20.953714+0000","last_unstale":"2023-11-09T03:47:22.248560+0000","last_undegraded":"2023-11-09T03:47:22.248560+0000","last_fullsized":"2023-11-09T03:47:22.248560+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:40:44.918031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"53'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248845+0000","last_change":"2023-11-09T03:47:19.943054+0000","last_active":"2023-11-09T03:47:22.248845+0000","last_peered":"2023-11-09T03:47:22.248845+0000","last_clean":"2023-11-09T03:47:22.248845+0000","last_became_active":"2023-11-09T03:47:19.942787+0000","last_became_peered":"2023-11-09T03:47:19.942787+0000","last_unstale":"2023-11-09T03:47:22.248845+0000","last_undegraded":"2023-11-09T03:47:22.248845+0000","last_fullsized":"2023-11-09T03:47:22.248845+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:58:08.764749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.939087+0000","last_change":"2023-11-09T03:47:02.724896+0000","last_active":"2023-11-09T03:47:20.939087+0000","last_peered":"2023-11-09T03:47:20.939087+0000","last_clean":"2023-11-09T03:47:20.939087+0000","last_became_active":"2023-11-09T03:47:01.592784+0000","last_became_peered":"2023-11-09T03:47:01.592784+0000","last_unstale":"2023-11-09T03:47:20.939087+0000","last_undegraded":"2023-11-09T03:47:20.939087+0000","last_fullsized":"2023-11-09T03:47:20.939087+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:51:32.016204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056683500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248073+0000","last_change":"2023-11-09T03:47:20.945066+0000","last_active":"2023-11-09T03:47:22.248073+0000","last_peered":"2023-11-09T03:47:22.248073+0000","last_clean":"2023-11-09T03:47:22.248073+0000","last_became_active":"2023-11-09T03:47:20.944812+0000","last_became_peered":"2023-11-09T03:47:20.944812+0000","last_unstale":"2023-11-09T03:47:22.248073+0000","last_undegraded":"2023-11-09T03:47:22.248073+0000","last_fullsized":"2023-11-09T03:47:22.248073+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:58:30.558728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"53'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248249+0000","last_change":"2023-11-09T03:47:19.947954+0000","last_active":"2023-11-09T03:47:22.248249+0000","last_peered":"2023-11-09T03:47:22.248249+0000","last_clean":"2023-11-09T03:47:22.248249+0000","last_became_active":"2023-11-09T03:47:19.947687+0000","last_became_peered":"2023-11-09T03:47:19.947687+0000","last_unstale":"2023-11-09T03:47:22.248249+0000","last_undegraded":"2023-11-09T03:47:22.248249+0000","last_fullsized":"2023-11-09T03:47:22.248249+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:52:58.422763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.933388+0000","last_change":"2023-11-09T03:47:03.146527+0000","last_active":"2023-11-09T03:47:20.933388+0000","last_peered":"2023-11-09T03:47:20.933388+0000","last_clean":"2023-11-09T03:47:20.933388+0000","last_became_active":"2023-11-09T03:47:01.587858+0000","last_became_peered":"2023-11-09T03:47:01.587858+0000","last_unstale":"2023-11-09T03:47:20.933388+0000","last_undegraded":"2023-11-09T03:47:20.933388+0000","last_fullsized":"2023-11-09T03:47:20.933388+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:56:04.201126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084439599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248163+0000","last_change":"2023-11-09T03:47:20.943815+0000","last_active":"2023-11-09T03:47:22.248163+0000","last_peered":"2023-11-09T03:47:22.248163+0000","last_clean":"2023-11-09T03:47:22.248163+0000","last_became_active":"2023-11-09T03:47:20.943403+0000","last_became_peered":"2023-11-09T03:47:20.943403+0000","last_unstale":"2023-11-09T03:47:22.248163+0000","last_undegraded":"2023-11-09T03:47:22.248163+0000","last_fullsized":"2023-11-09T03:47:22.248163+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:59:34.738692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247950+0000","last_change":"2023-11-09T03:47:19.949858+0000","last_active":"2023-11-09T03:47:22.247950+0000","last_peered":"2023-11-09T03:47:22.247950+0000","last_clean":"2023-11-09T03:47:22.247950+0000","last_became_active":"2023-11-09T03:47:19.949636+0000","last_became_peered":"2023-11-09T03:47:19.949636+0000","last_unstale":"2023-11-09T03:47:22.247950+0000","last_undegraded":"2023-11-09T03:47:22.247950+0000","last_fullsized":"2023-11-09T03:47:22.247950+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:24.100578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.935999+0000","last_change":"2023-11-09T03:47:02.725223+0000","last_active":"2023-11-09T03:47:20.935999+0000","last_peered":"2023-11-09T03:47:20.935999+0000","last_clean":"2023-11-09T03:47:20.935999+0000","last_became_active":"2023-11-09T03:47:01.590635+0000","last_became_peered":"2023-11-09T03:47:01.590635+0000","last_unstale":"2023-11-09T03:47:20.935999+0000","last_undegraded":"2023-11-09T03:47:20.935999+0000","last_fullsized":"2023-11-09T03:47:20.935999+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:05.754357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051685100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-09T03:47:19.915800+0000","last_change":"2023-11-09T03:47:19.915800+0000","last_active":"2023-11-09T03:47:19.915800+0000","last_peered":"2023-11-09T03:47:19.915800+0000","last_clean":"2023-11-09T03:47:19.915800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:47:19.915800+0000","last_undegraded":"2023-11-09T03:47:19.915800+0000","last_fullsized":"2023-11-09T03:47:19.915800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940244+0000","last_change":"2023-11-09T03:47:19.942483+0000","last_active":"2023-11-09T03:47:21.940244+0000","last_peered":"2023-11-09T03:47:21.940244+0000","last_clean":"2023-11-09T03:47:21.940244+0000","last_became_active":"2023-11-09T03:47:19.942157+0000","last_became_peered":"2023-11-09T03:47:19.942157+0000","last_unstale":"2023-11-09T03:47:21.940244+0000","last_undegraded":"2023-11-09T03:47:21.940244+0000","last_fullsized":"2023-11-09T03:47:21.940244+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:59:28.149940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.937687+0000","last_change":"2023-11-09T03:47:02.582739+0000","last_active":"2023-11-09T03:47:20.937687+0000","last_peered":"2023-11-09T03:47:20.937687+0000","last_clean":"2023-11-09T03:47:20.937687+0000","last_became_active":"2023-11-09T03:47:01.592743+0000","last_became_peered":"2023-11-09T03:47:01.592743+0000","last_unstale":"2023-11-09T03:47:20.937687+0000","last_undegraded":"2023-11-09T03:47:20.937687+0000","last_fullsized":"2023-11-09T03:47:20.937687+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051802700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939444+0000","last_change":"2023-11-09T03:47:20.939366+0000","last_active":"2023-11-09T03:47:21.939444+0000","last_peered":"2023-11-09T03:47:21.939444+0000","last_clean":"2023-11-09T03:47:21.939444+0000","last_became_active":"2023-11-09T03:47:20.938258+0000","last_became_peered":"2023-11-09T03:47:20.938258+0000","last_unstale":"2023-11-09T03:47:21.939444+0000","last_undegraded":"2023-11-09T03:47:21.939444+0000","last_fullsized":"2023-11-09T03:47:21.939444+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:48:27.582736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"53'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248057+0000","last_change":"2023-11-09T03:47:19.946306+0000","last_active":"2023-11-09T03:47:22.248057+0000","last_peered":"2023-11-09T03:47:22.248057+0000","last_clean":"2023-11-09T03:47:22.248057+0000","last_became_active":"2023-11-09T03:47:19.946062+0000","last_became_peered":"2023-11-09T03:47:19.946062+0000","last_unstale":"2023-11-09T03:47:22.248057+0000","last_undegraded":"2023-11-09T03:47:22.248057+0000","last_fullsized":"2023-11-09T03:47:22.248057+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:51:52.875174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.935441+0000","last_change":"2023-11-09T03:47:02.724953+0000","last_active":"2023-11-09T03:47:20.935441+0000","last_peered":"2023-11-09T03:47:20.935441+0000","last_clean":"2023-11-09T03:47:20.935441+0000","last_became_active":"2023-11-09T03:47:01.587133+0000","last_became_peered":"2023-11-09T03:47:01.587133+0000","last_unstale":"2023-11-09T03:47:20.935441+0000","last_undegraded":"2023-11-09T03:47:20.935441+0000","last_fullsized":"2023-11-09T03:47:20.935441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:23:48.521161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066497599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249034+0000","last_change":"2023-11-09T03:47:20.942122+0000","last_active":"2023-11-09T03:47:22.249034+0000","last_peered":"2023-11-09T03:47:22.249034+0000","last_clean":"2023-11-09T03:47:22.249034+0000","last_became_active":"2023-11-09T03:47:20.941870+0000","last_became_peered":"2023-11-09T03:47:20.941870+0000","last_unstale":"2023-11-09T03:47:22.249034+0000","last_undegraded":"2023-11-09T03:47:22.249034+0000","last_fullsized":"2023-11-09T03:47:22.249034+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:57.187676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"53'4","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249334+0000","last_change":"2023-11-09T03:47:19.945714+0000","last_active":"2023-11-09T03:47:22.249334+0000","last_peered":"2023-11-09T03:47:22.249334+0000","last_clean":"2023-11-09T03:47:22.249334+0000","last_became_active":"2023-11-09T03:47:19.945455+0000","last_became_peered":"2023-11-09T03:47:19.945455+0000","last_unstale":"2023-11-09T03:47:22.249334+0000","last_undegraded":"2023-11-09T03:47:22.249334+0000","last_fullsized":"2023-11-09T03:47:22.249334+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:38:08.672664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.935591+0000","last_change":"2023-11-09T03:47:03.146217+0000","last_active":"2023-11-09T03:47:20.935591+0000","last_peered":"2023-11-09T03:47:20.935591+0000","last_clean":"2023-11-09T03:47:20.935591+0000","last_became_active":"2023-11-09T03:47:01.587598+0000","last_became_peered":"2023-11-09T03:47:01.587598+0000","last_unstale":"2023-11-09T03:47:20.935591+0000","last_undegraded":"2023-11-09T03:47:20.935591+0000","last_fullsized":"2023-11-09T03:47:20.935591+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:15:22.723455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090126400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249094+0000","last_change":"2023-11-09T03:47:20.950285+0000","last_active":"2023-11-09T03:47:22.249094+0000","last_peered":"2023-11-09T03:47:22.249094+0000","last_clean":"2023-11-09T03:47:22.249094+0000","last_became_active":"2023-11-09T03:47:20.949722+0000","last_became_peered":"2023-11-09T03:47:20.949722+0000","last_unstale":"2023-11-09T03:47:22.249094+0000","last_undegraded":"2023-11-09T03:47:22.249094+0000","last_fullsized":"2023-11-09T03:47:22.249094+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:51:53.879857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940663+0000","last_change":"2023-11-09T03:47:19.947977+0000","last_active":"2023-11-09T03:47:21.940663+0000","last_peered":"2023-11-09T03:47:21.940663+0000","last_clean":"2023-11-09T03:47:21.940663+0000","last_became_active":"2023-11-09T03:47:19.947549+0000","last_became_peered":"2023-11-09T03:47:19.947549+0000","last_unstale":"2023-11-09T03:47:21.940663+0000","last_undegraded":"2023-11-09T03:47:21.940663+0000","last_fullsized":"2023-11-09T03:47:21.940663+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:51.678872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931267+0000","last_change":"2023-11-09T03:47:03.148283+0000","last_active":"2023-11-09T03:47:20.931267+0000","last_peered":"2023-11-09T03:47:20.931267+0000","last_clean":"2023-11-09T03:47:20.931267+0000","last_became_active":"2023-11-09T03:47:01.588841+0000","last_became_peered":"2023-11-09T03:47:01.588841+0000","last_unstale":"2023-11-09T03:47:20.931267+0000","last_undegraded":"2023-11-09T03:47:20.931267+0000","last_fullsized":"2023-11-09T03:47:20.931267+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:47:44.880741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000436081,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249189+0000","last_change":"2023-11-09T03:47:20.937043+0000","last_active":"2023-11-09T03:47:22.249189+0000","last_peered":"2023-11-09T03:47:22.249189+0000","last_clean":"2023-11-09T03:47:22.249189+0000","last_became_active":"2023-11-09T03:47:20.936765+0000","last_became_peered":"2023-11-09T03:47:20.936765+0000","last_unstale":"2023-11-09T03:47:22.249189+0000","last_undegraded":"2023-11-09T03:47:22.249189+0000","last_fullsized":"2023-11-09T03:47:22.249189+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:14:00.724882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249189+0000","last_change":"2023-11-09T03:47:19.946355+0000","last_active":"2023-11-09T03:47:22.249189+0000","last_peered":"2023-11-09T03:47:22.249189+0000","last_clean":"2023-11-09T03:47:22.249189+0000","last_became_active":"2023-11-09T03:47:19.945071+0000","last_became_peered":"2023-11-09T03:47:19.945071+0000","last_unstale":"2023-11-09T03:47:22.249189+0000","last_undegraded":"2023-11-09T03:47:22.249189+0000","last_fullsized":"2023-11-09T03:47:22.249189+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:07:31.918577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":21,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-09T03:47:15.977070+0000","last_change":"2023-11-09T03:47:02.618856+0000","last_active":"2023-11-09T03:47:15.977070+0000","last_peered":"2023-11-09T03:47:15.977070+0000","last_clean":"2023-11-09T03:47:15.977070+0000","last_became_active":"2023-11-09T03:47:01.590430+0000","last_became_peered":"2023-11-09T03:47:01.590430+0000","last_unstale":"2023-11-09T03:47:15.977070+0000","last_undegraded":"2023-11-09T03:47:15.977070+0000","last_fullsized":"2023-11-09T03:47:15.977070+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:57:18.470305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055030200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026961+0000","last_change":"2023-11-09T03:47:20.948028+0000","last_active":"2023-11-09T03:47:22.026961+0000","last_peered":"2023-11-09T03:47:22.026961+0000","last_clean":"2023-11-09T03:47:22.026961+0000","last_became_active":"2023-11-09T03:47:20.947752+0000","last_became_peered":"2023-11-09T03:47:20.947752+0000","last_unstale":"2023-11-09T03:47:22.026961+0000","last_undegraded":"2023-11-09T03:47:22.026961+0000","last_fullsized":"2023-11-09T03:47:22.026961+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:51:53.879857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248095+0000","last_change":"2023-11-09T03:47:19.948154+0000","last_active":"2023-11-09T03:47:22.248095+0000","last_peered":"2023-11-09T03:47:22.248095+0000","last_clean":"2023-11-09T03:47:22.248095+0000","last_became_active":"2023-11-09T03:47:19.947812+0000","last_became_peered":"2023-11-09T03:47:19.947812+0000","last_unstale":"2023-11-09T03:47:22.248095+0000","last_undegraded":"2023-11-09T03:47:22.248095+0000","last_fullsized":"2023-11-09T03:47:22.248095+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:28:13.618166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":21,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-09T03:47:15.977099+0000","last_change":"2023-11-09T03:47:02.619144+0000","last_active":"2023-11-09T03:47:15.977099+0000","last_peered":"2023-11-09T03:47:15.977099+0000","last_clean":"2023-11-09T03:47:15.977099+0000","last_became_active":"2023-11-09T03:47:01.590672+0000","last_became_peered":"2023-11-09T03:47:01.590672+0000","last_unstale":"2023-11-09T03:47:15.977099+0000","last_undegraded":"2023-11-09T03:47:15.977099+0000","last_fullsized":"2023-11-09T03:47:15.977099+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:03.025119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060997500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248188+0000","last_change":"2023-11-09T03:47:20.950877+0000","last_active":"2023-11-09T03:47:22.248188+0000","last_peered":"2023-11-09T03:47:22.248188+0000","last_clean":"2023-11-09T03:47:22.248188+0000","last_became_active":"2023-11-09T03:47:20.950616+0000","last_became_peered":"2023-11-09T03:47:20.950616+0000","last_unstale":"2023-11-09T03:47:22.248188+0000","last_undegraded":"2023-11-09T03:47:22.248188+0000","last_fullsized":"2023-11-09T03:47:22.248188+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:30.970511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939086+0000","last_change":"2023-11-09T03:47:19.940824+0000","last_active":"2023-11-09T03:47:21.939086+0000","last_peered":"2023-11-09T03:47:21.939086+0000","last_clean":"2023-11-09T03:47:21.939086+0000","last_became_active":"2023-11-09T03:47:19.940553+0000","last_became_peered":"2023-11-09T03:47:19.940553+0000","last_unstale":"2023-11-09T03:47:21.939086+0000","last_undegraded":"2023-11-09T03:47:21.939086+0000","last_fullsized":"2023-11-09T03:47:21.939086+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:52:58.422763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":21,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-09T03:47:15.976969+0000","last_change":"2023-11-09T03:47:02.619079+0000","last_active":"2023-11-09T03:47:15.976969+0000","last_peered":"2023-11-09T03:47:15.976969+0000","last_clean":"2023-11-09T03:47:15.976969+0000","last_became_active":"2023-11-09T03:47:01.591421+0000","last_became_peered":"2023-11-09T03:47:01.591421+0000","last_unstale":"2023-11-09T03:47:15.976969+0000","last_undegraded":"2023-11-09T03:47:15.976969+0000","last_fullsized":"2023-11-09T03:47:15.976969+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:18:58.230606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068736399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248346+0000","last_change":"2023-11-09T03:47:20.943642+0000","last_active":"2023-11-09T03:47:22.248346+0000","last_peered":"2023-11-09T03:47:22.248346+0000","last_clean":"2023-11-09T03:47:22.248346+0000","last_became_active":"2023-11-09T03:47:20.943101+0000","last_became_peered":"2023-11-09T03:47:20.943101+0000","last_unstale":"2023-11-09T03:47:22.248346+0000","last_undegraded":"2023-11-09T03:47:22.248346+0000","last_fullsized":"2023-11-09T03:47:22.248346+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:06:11.677208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248583+0000","last_change":"2023-11-09T03:47:19.943530+0000","last_active":"2023-11-09T03:47:22.248583+0000","last_peered":"2023-11-09T03:47:22.248583+0000","last_clean":"2023-11-09T03:47:22.248583+0000","last_became_active":"2023-11-09T03:47:19.943303+0000","last_became_peered":"2023-11-09T03:47:19.943303+0000","last_unstale":"2023-11-09T03:47:22.248583+0000","last_undegraded":"2023-11-09T03:47:22.248583+0000","last_fullsized":"2023-11-09T03:47:22.248583+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:35.294223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931280+0000","last_change":"2023-11-09T03:47:03.148529+0000","last_active":"2023-11-09T03:47:20.931280+0000","last_peered":"2023-11-09T03:47:20.931280+0000","last_clean":"2023-11-09T03:47:20.931280+0000","last_became_active":"2023-11-09T03:47:01.584148+0000","last_became_peered":"2023-11-09T03:47:01.584148+0000","last_unstale":"2023-11-09T03:47:20.931280+0000","last_undegraded":"2023-11-09T03:47:20.931280+0000","last_fullsized":"2023-11-09T03:47:20.931280+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:40:00.237248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061475600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.938959+0000","last_change":"2023-11-09T03:47:20.942889+0000","last_active":"2023-11-09T03:47:21.938959+0000","last_peered":"2023-11-09T03:47:21.938959+0000","last_clean":"2023-11-09T03:47:21.938959+0000","last_became_active":"2023-11-09T03:47:20.941326+0000","last_became_peered":"2023-11-09T03:47:20.941326+0000","last_unstale":"2023-11-09T03:47:21.938959+0000","last_undegraded":"2023-11-09T03:47:21.938959+0000","last_fullsized":"2023-11-09T03:47:21.938959+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:56:23.552030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.938908+0000","last_change":"2023-11-09T03:47:19.942969+0000","last_active":"2023-11-09T03:47:21.938908+0000","last_peered":"2023-11-09T03:47:21.938908+0000","last_clean":"2023-11-09T03:47:21.938908+0000","last_became_active":"2023-11-09T03:47:19.942726+0000","last_became_peered":"2023-11-09T03:47:19.942726+0000","last_unstale":"2023-11-09T03:47:21.938908+0000","last_undegraded":"2023-11-09T03:47:21.938908+0000","last_fullsized":"2023-11-09T03:47:21.938908+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:51:50.362425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.933718+0000","last_change":"2023-11-09T03:47:03.148217+0000","last_active":"2023-11-09T03:47:20.933718+0000","last_peered":"2023-11-09T03:47:20.933718+0000","last_clean":"2023-11-09T03:47:20.933718+0000","last_became_active":"2023-11-09T03:47:01.593424+0000","last_became_peered":"2023-11-09T03:47:01.593424+0000","last_unstale":"2023-11-09T03:47:20.933718+0000","last_undegraded":"2023-11-09T03:47:20.933718+0000","last_fullsized":"2023-11-09T03:47:20.933718+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:19:32.693691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001769865,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.941149+0000","last_change":"2023-11-09T03:47:20.942781+0000","last_active":"2023-11-09T03:47:21.941149+0000","last_peered":"2023-11-09T03:47:21.941149+0000","last_clean":"2023-11-09T03:47:21.941149+0000","last_became_active":"2023-11-09T03:47:20.942498+0000","last_became_peered":"2023-11-09T03:47:20.942498+0000","last_unstale":"2023-11-09T03:47:21.941149+0000","last_undegraded":"2023-11-09T03:47:21.941149+0000","last_fullsized":"2023-11-09T03:47:21.941149+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:23:50.235725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249315+0000","last_change":"2023-11-09T03:47:19.944843+0000","last_active":"2023-11-09T03:47:22.249315+0000","last_peered":"2023-11-09T03:47:22.249315+0000","last_clean":"2023-11-09T03:47:22.249315+0000","last_became_active":"2023-11-09T03:47:19.944583+0000","last_became_peered":"2023-11-09T03:47:19.944583+0000","last_unstale":"2023-11-09T03:47:22.249315+0000","last_undegraded":"2023-11-09T03:47:22.249315+0000","last_fullsized":"2023-11-09T03:47:22.249315+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:46:53.255789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931015+0000","last_change":"2023-11-09T03:47:03.146470+0000","last_active":"2023-11-09T03:47:20.931015+0000","last_peered":"2023-11-09T03:47:20.931015+0000","last_clean":"2023-11-09T03:47:20.931015+0000","last_became_active":"2023-11-09T03:47:01.587304+0000","last_became_peered":"2023-11-09T03:47:01.587304+0000","last_unstale":"2023-11-09T03:47:20.931015+0000","last_undegraded":"2023-11-09T03:47:20.931015+0000","last_fullsized":"2023-11-09T03:47:20.931015+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:58:23.339287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00110395,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.941057+0000","last_change":"2023-11-09T03:47:20.941612+0000","last_active":"2023-11-09T03:47:21.941057+0000","last_peered":"2023-11-09T03:47:21.941057+0000","last_clean":"2023-11-09T03:47:21.941057+0000","last_became_active":"2023-11-09T03:47:20.941335+0000","last_became_peered":"2023-11-09T03:47:20.941335+0000","last_unstale":"2023-11-09T03:47:21.941057+0000","last_undegraded":"2023-11-09T03:47:21.941057+0000","last_fullsized":"2023-11-09T03:47:21.941057+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:05:32.662676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939435+0000","last_change":"2023-11-09T03:47:19.941173+0000","last_active":"2023-11-09T03:47:21.939435+0000","last_peered":"2023-11-09T03:47:21.939435+0000","last_clean":"2023-11-09T03:47:21.939435+0000","last_became_active":"2023-11-09T03:47:19.940928+0000","last_became_peered":"2023-11-09T03:47:19.940928+0000","last_unstale":"2023-11-09T03:47:21.939435+0000","last_undegraded":"2023-11-09T03:47:21.939435+0000","last_fullsized":"2023-11-09T03:47:21.939435+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:28:13.618166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.934203+0000","last_change":"2023-11-09T03:47:03.147367+0000","last_active":"2023-11-09T03:47:20.934203+0000","last_peered":"2023-11-09T03:47:20.934203+0000","last_clean":"2023-11-09T03:47:20.934203+0000","last_became_active":"2023-11-09T03:47:01.588359+0000","last_became_peered":"2023-11-09T03:47:01.588359+0000","last_unstale":"2023-11-09T03:47:20.934203+0000","last_undegraded":"2023-11-09T03:47:20.934203+0000","last_fullsized":"2023-11-09T03:47:20.934203+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:21:34.303068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014106660000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249223+0000","last_change":"2023-11-09T03:47:20.950444+0000","last_active":"2023-11-09T03:47:22.249223+0000","last_peered":"2023-11-09T03:47:22.249223+0000","last_clean":"2023-11-09T03:47:22.249223+0000","last_became_active":"2023-11-09T03:47:20.950038+0000","last_became_peered":"2023-11-09T03:47:20.950038+0000","last_unstale":"2023-11-09T03:47:22.249223+0000","last_undegraded":"2023-11-09T03:47:22.249223+0000","last_fullsized":"2023-11-09T03:47:22.249223+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:55:28.077165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940688+0000","last_change":"2023-11-09T03:47:19.943935+0000","last_active":"2023-11-09T03:47:21.940688+0000","last_peered":"2023-11-09T03:47:21.940688+0000","last_clean":"2023-11-09T03:47:21.940688+0000","last_became_active":"2023-11-09T03:47:19.943125+0000","last_became_peered":"2023-11-09T03:47:19.943125+0000","last_unstale":"2023-11-09T03:47:21.940688+0000","last_undegraded":"2023-11-09T03:47:21.940688+0000","last_fullsized":"2023-11-09T03:47:21.940688+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:06:16.671366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":27,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.928179+0000","last_change":"2023-11-09T03:47:02.581682+0000","last_active":"2023-11-09T03:47:20.928179+0000","last_peered":"2023-11-09T03:47:20.928179+0000","last_clean":"2023-11-09T03:47:20.928179+0000","last_became_active":"2023-11-09T03:47:01.593051+0000","last_became_peered":"2023-11-09T03:47:01.593051+0000","last_unstale":"2023-11-09T03:47:20.928179+0000","last_undegraded":"2023-11-09T03:47:20.928179+0000","last_fullsized":"2023-11-09T03:47:20.928179+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047277799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939213+0000","last_change":"2023-11-09T03:47:20.945402+0000","last_active":"2023-11-09T03:47:21.939213+0000","last_peered":"2023-11-09T03:47:21.939213+0000","last_clean":"2023-11-09T03:47:21.939213+0000","last_became_active":"2023-11-09T03:47:20.945145+0000","last_became_peered":"2023-11-09T03:47:20.945145+0000","last_unstale":"2023-11-09T03:47:21.939213+0000","last_undegraded":"2023-11-09T03:47:21.939213+0000","last_fullsized":"2023-11-09T03:47:21.939213+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:38:09.677347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-09T03:47:18.911117+0000","last_change":"2023-11-09T03:47:18.911117+0000","last_active":"2023-11-09T03:47:18.911117+0000","last_peered":"2023-11-09T03:47:18.911117+0000","last_clean":"2023-11-09T03:47:18.911117+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:47:18.911117+0000","last_undegraded":"2023-11-09T03:47:18.911117+0000","last_fullsized":"2023-11-09T03:47:18.911117+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.933457+0000","last_change":"2023-11-09T03:47:03.146309+0000","last_active":"2023-11-09T03:47:20.933457+0000","last_peered":"2023-11-09T03:47:20.933457+0000","last_clean":"2023-11-09T03:47:20.933457+0000","last_became_active":"2023-11-09T03:47:01.587560+0000","last_became_peered":"2023-11-09T03:47:01.587560+0000","last_unstale":"2023-11-09T03:47:20.933457+0000","last_undegraded":"2023-11-09T03:47:20.933457+0000","last_fullsized":"2023-11-09T03:47:20.933457+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:27:55.271945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070988400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249299+0000","last_change":"2023-11-09T03:47:20.941432+0000","last_active":"2023-11-09T03:47:22.249299+0000","last_peered":"2023-11-09T03:47:22.249299+0000","last_clean":"2023-11-09T03:47:22.249299+0000","last_became_active":"2023-11-09T03:47:20.941186+0000","last_became_peered":"2023-11-09T03:47:20.941186+0000","last_unstale":"2023-11-09T03:47:22.249299+0000","last_undegraded":"2023-11-09T03:47:22.249299+0000","last_fullsized":"2023-11-09T03:47:22.249299+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:25:38.423194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939102+0000","last_change":"2023-11-09T03:47:19.947334+0000","last_active":"2023-11-09T03:47:21.939102+0000","last_peered":"2023-11-09T03:47:21.939102+0000","last_clean":"2023-11-09T03:47:21.939102+0000","last_became_active":"2023-11-09T03:47:19.947091+0000","last_became_peered":"2023-11-09T03:47:19.947091+0000","last_unstale":"2023-11-09T03:47:21.939102+0000","last_undegraded":"2023-11-09T03:47:21.939102+0000","last_fullsized":"2023-11-09T03:47:21.939102+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:24:06.867382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.939549+0000","last_change":"2023-11-09T03:47:02.725125+0000","last_active":"2023-11-09T03:47:20.939549+0000","last_peered":"2023-11-09T03:47:20.939549+0000","last_clean":"2023-11-09T03:47:20.939549+0000","last_became_active":"2023-11-09T03:47:01.591521+0000","last_became_peered":"2023-11-09T03:47:01.591521+0000","last_unstale":"2023-11-09T03:47:20.939549+0000","last_undegraded":"2023-11-09T03:47:20.939549+0000","last_fullsized":"2023-11-09T03:47:20.939549+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:52:40.076542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000758675,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940341+0000","last_change":"2023-11-09T03:47:20.938307+0000","last_active":"2023-11-09T03:47:21.940341+0000","last_peered":"2023-11-09T03:47:21.940341+0000","last_clean":"2023-11-09T03:47:21.940341+0000","last_became_active":"2023-11-09T03:47:20.937853+0000","last_became_peered":"2023-11-09T03:47:20.937853+0000","last_unstale":"2023-11-09T03:47:21.940341+0000","last_undegraded":"2023-11-09T03:47:21.940341+0000","last_fullsized":"2023-11-09T03:47:21.940341+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:57.187676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248805+0000","last_change":"2023-11-09T03:47:19.945739+0000","last_active":"2023-11-09T03:47:22.248805+0000","last_peered":"2023-11-09T03:47:22.248805+0000","last_clean":"2023-11-09T03:47:22.248805+0000","last_became_active":"2023-11-09T03:47:19.944990+0000","last_became_peered":"2023-11-09T03:47:19.944990+0000","last_unstale":"2023-11-09T03:47:22.248805+0000","last_undegraded":"2023-11-09T03:47:22.248805+0000","last_fullsized":"2023-11-09T03:47:22.248805+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:53:27.482165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.932096+0000","last_change":"2023-11-09T03:47:03.147289+0000","last_active":"2023-11-09T03:47:20.932096+0000","last_peered":"2023-11-09T03:47:20.932096+0000","last_clean":"2023-11-09T03:47:20.932096+0000","last_became_active":"2023-11-09T03:47:01.587784+0000","last_became_peered":"2023-11-09T03:47:01.587784+0000","last_unstale":"2023-11-09T03:47:20.932096+0000","last_undegraded":"2023-11-09T03:47:20.932096+0000","last_fullsized":"2023-11-09T03:47:20.932096+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:11:23.824112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038781299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247714+0000","last_change":"2023-11-09T03:47:20.937750+0000","last_active":"2023-11-09T03:47:22.247714+0000","last_peered":"2023-11-09T03:47:22.247714+0000","last_clean":"2023-11-09T03:47:22.247714+0000","last_became_active":"2023-11-09T03:47:20.937473+0000","last_became_peered":"2023-11-09T03:47:20.937473+0000","last_unstale":"2023-11-09T03:47:22.247714+0000","last_undegraded":"2023-11-09T03:47:22.247714+0000","last_fullsized":"2023-11-09T03:47:22.247714+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:06:17.676049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940071+0000","last_change":"2023-11-09T03:47:19.948440+0000","last_active":"2023-11-09T03:47:21.940071+0000","last_peered":"2023-11-09T03:47:21.940071+0000","last_clean":"2023-11-09T03:47:21.940071+0000","last_became_active":"2023-11-09T03:47:19.948199+0000","last_became_peered":"2023-11-09T03:47:19.948199+0000","last_unstale":"2023-11-09T03:47:21.940071+0000","last_undegraded":"2023-11-09T03:47:21.940071+0000","last_fullsized":"2023-11-09T03:47:21.940071+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:34:52.006539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":21,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-09T03:47:15.976947+0000","last_change":"2023-11-09T03:47:02.619346+0000","last_active":"2023-11-09T03:47:15.976947+0000","last_peered":"2023-11-09T03:47:15.976947+0000","last_clean":"2023-11-09T03:47:15.976947+0000","last_became_active":"2023-11-09T03:47:01.590389+0000","last_became_peered":"2023-11-09T03:47:01.590389+0000","last_unstale":"2023-11-09T03:47:15.976947+0000","last_undegraded":"2023-11-09T03:47:15.976947+0000","last_fullsized":"2023-11-09T03:47:15.976947+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000384072,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.941022+0000","last_change":"2023-11-09T03:47:20.937215+0000","last_active":"2023-11-09T03:47:21.941022+0000","last_peered":"2023-11-09T03:47:21.941022+0000","last_clean":"2023-11-09T03:47:21.941022+0000","last_became_active":"2023-11-09T03:47:20.936913+0000","last_became_peered":"2023-11-09T03:47:20.936913+0000","last_unstale":"2023-11-09T03:47:21.941022+0000","last_undegraded":"2023-11-09T03:47:21.941022+0000","last_fullsized":"2023-11-09T03:47:21.941022+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:25:38.423194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249234+0000","last_change":"2023-11-09T03:47:19.948214+0000","last_active":"2023-11-09T03:47:22.249234+0000","last_peered":"2023-11-09T03:47:22.249234+0000","last_clean":"2023-11-09T03:47:22.249234+0000","last_became_active":"2023-11-09T03:47:19.947966+0000","last_became_peered":"2023-11-09T03:47:19.947966+0000","last_unstale":"2023-11-09T03:47:22.249234+0000","last_undegraded":"2023-11-09T03:47:22.249234+0000","last_fullsized":"2023-11-09T03:47:22.249234+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:53:19.322920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.938897+0000","last_change":"2023-11-09T03:47:02.582393+0000","last_active":"2023-11-09T03:47:20.938897+0000","last_peered":"2023-11-09T03:47:20.938897+0000","last_clean":"2023-11-09T03:47:20.938897+0000","last_became_active":"2023-11-09T03:47:01.591752+0000","last_became_peered":"2023-11-09T03:47:01.591752+0000","last_unstale":"2023-11-09T03:47:20.938897+0000","last_undegraded":"2023-11-09T03:47:20.938897+0000","last_fullsized":"2023-11-09T03:47:20.938897+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:03.025119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067021599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.027114+0000","last_change":"2023-11-09T03:47:20.946342+0000","last_active":"2023-11-09T03:47:22.027114+0000","last_peered":"2023-11-09T03:47:22.027114+0000","last_clean":"2023-11-09T03:47:22.027114+0000","last_became_active":"2023-11-09T03:47:20.946014+0000","last_became_peered":"2023-11-09T03:47:20.946014+0000","last_unstale":"2023-11-09T03:47:22.027114+0000","last_undegraded":"2023-11-09T03:47:22.027114+0000","last_fullsized":"2023-11-09T03:47:22.027114+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:25:38.423194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"53'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248629+0000","last_change":"2023-11-09T03:47:19.944449+0000","last_active":"2023-11-09T03:47:22.248629+0000","last_peered":"2023-11-09T03:47:22.248629+0000","last_clean":"2023-11-09T03:47:22.248629+0000","last_became_active":"2023-11-09T03:47:19.944203+0000","last_became_peered":"2023-11-09T03:47:19.944203+0000","last_unstale":"2023-11-09T03:47:22.248629+0000","last_undegraded":"2023-11-09T03:47:22.248629+0000","last_fullsized":"2023-11-09T03:47:22.248629+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:51.678872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.929927+0000","last_change":"2023-11-09T03:47:03.028744+0000","last_active":"2023-11-09T03:47:20.929927+0000","last_peered":"2023-11-09T03:47:20.929927+0000","last_clean":"2023-11-09T03:47:20.929927+0000","last_became_active":"2023-11-09T03:47:01.593890+0000","last_became_peered":"2023-11-09T03:47:01.593890+0000","last_unstale":"2023-11-09T03:47:20.929927+0000","last_undegraded":"2023-11-09T03:47:20.929927+0000","last_fullsized":"2023-11-09T03:47:20.929927+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:05.754357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053008299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026854+0000","last_change":"2023-11-09T03:47:20.952365+0000","last_active":"2023-11-09T03:47:22.026854+0000","last_peered":"2023-11-09T03:47:22.026854+0000","last_clean":"2023-11-09T03:47:22.026854+0000","last_became_active":"2023-11-09T03:47:20.952080+0000","last_became_peered":"2023-11-09T03:47:20.952080+0000","last_unstale":"2023-11-09T03:47:22.026854+0000","last_undegraded":"2023-11-09T03:47:22.026854+0000","last_fullsized":"2023-11-09T03:47:22.026854+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:40.639821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249192+0000","last_change":"2023-11-09T03:47:19.946712+0000","last_active":"2023-11-09T03:47:22.249192+0000","last_peered":"2023-11-09T03:47:22.249192+0000","last_clean":"2023-11-09T03:47:22.249192+0000","last_became_active":"2023-11-09T03:47:19.946456+0000","last_became_peered":"2023-11-09T03:47:19.946456+0000","last_unstale":"2023-11-09T03:47:22.249192+0000","last_undegraded":"2023-11-09T03:47:22.249192+0000","last_fullsized":"2023-11-09T03:47:22.249192+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:39:07.929390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.936277+0000","last_change":"2023-11-09T03:47:03.146783+0000","last_active":"2023-11-09T03:47:20.936277+0000","last_peered":"2023-11-09T03:47:20.936277+0000","last_clean":"2023-11-09T03:47:20.936277+0000","last_became_active":"2023-11-09T03:47:01.586923+0000","last_became_peered":"2023-11-09T03:47:01.586923+0000","last_unstale":"2023-11-09T03:47:20.936277+0000","last_undegraded":"2023-11-09T03:47:20.936277+0000","last_fullsized":"2023-11-09T03:47:20.936277+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:37:50.326442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060384700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248492+0000","last_change":"2023-11-09T03:47:20.946978+0000","last_active":"2023-11-09T03:47:22.248492+0000","last_peered":"2023-11-09T03:47:22.248492+0000","last_clean":"2023-11-09T03:47:22.248492+0000","last_became_active":"2023-11-09T03:47:20.946667+0000","last_became_peered":"2023-11-09T03:47:20.946667+0000","last_unstale":"2023-11-09T03:47:22.248492+0000","last_undegraded":"2023-11-09T03:47:22.248492+0000","last_fullsized":"2023-11-09T03:47:22.248492+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:02:01.425453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"53'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940910+0000","last_change":"2023-11-09T03:47:19.945244+0000","last_active":"2023-11-09T03:47:21.940910+0000","last_peered":"2023-11-09T03:47:21.940910+0000","last_clean":"2023-11-09T03:47:21.940910+0000","last_became_active":"2023-11-09T03:47:19.944955+0000","last_became_peered":"2023-11-09T03:47:19.944955+0000","last_unstale":"2023-11-09T03:47:21.940910+0000","last_undegraded":"2023-11-09T03:47:21.940910+0000","last_fullsized":"2023-11-09T03:47:21.940910+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:12:47.685713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.933539+0000","last_change":"2023-11-09T03:47:03.146580+0000","last_active":"2023-11-09T03:47:20.933539+0000","last_peered":"2023-11-09T03:47:20.933539+0000","last_clean":"2023-11-09T03:47:20.933539+0000","last_became_active":"2023-11-09T03:47:01.589201+0000","last_became_peered":"2023-11-09T03:47:01.589201+0000","last_unstale":"2023-11-09T03:47:20.933539+0000","last_undegraded":"2023-11-09T03:47:20.933539+0000","last_fullsized":"2023-11-09T03:47:20.933539+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:14.300421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049706999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248489+0000","last_change":"2023-11-09T03:47:20.949430+0000","last_active":"2023-11-09T03:47:22.248489+0000","last_peered":"2023-11-09T03:47:22.248489+0000","last_clean":"2023-11-09T03:47:22.248489+0000","last_became_active":"2023-11-09T03:47:20.949177+0000","last_became_peered":"2023-11-09T03:47:20.949177+0000","last_unstale":"2023-11-09T03:47:22.248489+0000","last_undegraded":"2023-11-09T03:47:22.248489+0000","last_fullsized":"2023-11-09T03:47:22.248489+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:04:05.315163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-09T03:47:18.911117+0000","last_change":"2023-11-09T03:47:18.911117+0000","last_active":"2023-11-09T03:47:18.911117+0000","last_peered":"2023-11-09T03:47:18.911117+0000","last_clean":"2023-11-09T03:47:18.911117+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:47:18.911117+0000","last_undegraded":"2023-11-09T03:47:18.911117+0000","last_fullsized":"2023-11-09T03:47:18.911117+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931584+0000","last_change":"2023-11-09T03:47:03.147851+0000","last_active":"2023-11-09T03:47:20.931584+0000","last_peered":"2023-11-09T03:47:20.931584+0000","last_clean":"2023-11-09T03:47:20.931584+0000","last_became_active":"2023-11-09T03:47:01.589893+0000","last_became_peered":"2023-11-09T03:47:01.589893+0000","last_unstale":"2023-11-09T03:47:20.931584+0000","last_undegraded":"2023-11-09T03:47:20.931584+0000","last_fullsized":"2023-11-09T03:47:20.931584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:08:17.183462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029151399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":194,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026619+0000","last_change":"2023-11-09T03:45:21.030926+0000","last_active":"2023-11-09T03:47:22.026619+0000","last_peered":"2023-11-09T03:47:22.026619+0000","last_clean":"2023-11-09T03:47:22.026619+0000","last_became_active":"2023-11-09T03:45:21.029801+0000","last_became_peered":"2023-11-09T03:45:21.029801+0000","last_unstale":"2023-11-09T03:47:22.026619+0000","last_undegraded":"2023-11-09T03:47:22.026619+0000","last_fullsized":"2023-11-09T03:47:22.026619+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_clean_scrub_stamp":"2023-11-09T03:45:19.768433+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:12:59.153713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":122,"acting":122,"num_store_stats":7},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":58,"acting":58,"num_store_stats":7},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28240,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343600,"statfs":{"total":96636764160,"available":96607846400,"internally_reserved":0,"allocated":1245184,"data_stored":881539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.501,"15min":0.498},"min":{"1min":0.34899999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.622},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.41199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.71499999999999997}]},{"osd":1,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.33000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.38100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.73099999999999998}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.52300000000000002,"15min":0.51400000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.68700000000000006,"15min":0.70799999999999996},"min":{"1min":0.41299999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.80700000000000005,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.63500000000000001}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.48699999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.68999999999999995}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.441},"min":{"1min":0.34100000000000003,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.49199999999999999,"15min":0.47899999999999998},"min":{"1min":0.38700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.38700000000000001}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.47199999999999998,"15min":0.45600000000000002},"min":{"1min":0.37,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.51100000000000001,"15min":0.497},"min":{"1min":0.39600000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.58699999999999997}]},{"osd":6,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55100000000000005,"15min":0.54100000000000004},"min":{"1min":0.44400000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.434,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.44}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":593920,"data_stored":263160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.622}]},{"osd":1,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.504}]},{"osd":3,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.60099999999999998}]},{"osd":4,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.38},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.65600000000000003}]},{"osd":5,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.35399999999999998}]},{"osd":7,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.55900000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":651264,"data_stored":291663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.496,"15min":0.48599999999999999},"min":{"1min":0.38700000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.876},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.503,"15min":0.498},"min":{"1min":0.42399999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.71799999999999997,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45400000000000001}]},{"osd":1,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.439,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.69999999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.62},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.46700000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.76200000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.64100000000000001}]},{"osd":2,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.33300000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.67900000000000005,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.499,"5min":0.371,"15min":0.371},"max":{"1min":0.72799999999999998,"5min":0.745,"15min":0.745},"last":0.71799999999999997}]},{"osd":3,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.39700000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.76500000000000001,"5min":0.88,"15min":0.88},"last":0.76400000000000001}]},{"osd":4,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.78100000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.495},{"interface":"front","average":{"1min":0.504,"5min":0.48399999999999999,"15min":0.48099999999999998},"min":{"1min":0.33100000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.79000000000000004}]},{"osd":6,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.50900000000000001,"15min":0.51100000000000001},"min":{"1min":0.33400000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.74199999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51300000000000001,"15min":0.51100000000000001},"min":{"1min":0.35499999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66900000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.753}]},{"osd":7,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.38900000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.73499999999999999,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61899999999999999,"15min":0.62},"min":{"1min":0.41199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.752,"5min":1.01,"15min":1.01},"last":0.84899999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":643072,"data_stored":285650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67600000000000005,"5min":1.034,"15min":1.034},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.53600000000000003,"15min":0.54000000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68300000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.46899999999999997}]},{"osd":1,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.56899999999999995,"15min":0.57699999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68100000000000005,"5min":1.093,"15min":1.093},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.36899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72399999999999998,"5min":0.871,"15min":0.871},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76800000000000002,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70099999999999996,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.44700000000000001}]},{"osd":3,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71699999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.502},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54900000000000004,"15min":0.55200000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86599999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.433}]},{"osd":5,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54400000000000004,"15min":0.54300000000000004},"min":{"1min":0.37,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76000000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.33100000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.36899999999999999}]},{"osd":6,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.33800000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.51700000000000002}]},{"osd":7,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.62,"15min":0.63},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77700000000000002,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.95599999999999996,"5min":1.113,"15min":1.113},"last":0.54300000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":651264,"data_stored":291067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.5,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.215},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51100000000000001,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.23300000000000001}]},{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32900000000000001,"15min":0.33100000000000002},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.54200000000000004,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.19800000000000001},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.371,"15min":0.375},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.53100000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.26700000000000002}]},{"osd":2,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.27200000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.59099999999999997,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.56100000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.31900000000000001}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.70899999999999996,"5min":0.87,"15min":0.87},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60999999999999999,"15min":0.61599999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.75600000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.62,"15min":0.625},"min":{"1min":0.45700000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79900000000000004,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61699999999999999,"15min":0.623},"min":{"1min":0.42599999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.74299999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.621,"15min":0.627},"min":{"1min":0.42399999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81200000000000006,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59999999999999998,"15min":0.60399999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76600000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.66000000000000003}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77200000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63500000000000001,"15min":0.64200000000000002},"min":{"1min":0.433,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81299999999999994,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.58299999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":651264,"data_stored":291067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.46000000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.33400000000000002,"15min":0.33100000000000002},"min":{"1min":0.247,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.27600000000000002}]},{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32500000000000001,"15min":0.32600000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.495,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.249},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.35899999999999999,"15min":0.36299999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.48599999999999999,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.33200000000000002}]},{"osd":3,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.36199999999999999,"15min":0.36399999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.45700000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40200000000000002,"15min":0.40400000000000003},"min":{"1min":0.30299999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.52300000000000002,"5min":0.754,"15min":0.754},"last":0.35299999999999998}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72299999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81299999999999994,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.50900000000000001}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.39800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.75700000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.59599999999999997,"15min":0.60099999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.753,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.498}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60399999999999998,"15min":0.60899999999999999},"min":{"1min":0.44600000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.76100000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.71199999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.57699999999999996}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60699999999999998,"15min":0.61199999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.745,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.80000000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":647168,"data_stored":291101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.28699999999999998,"5min":0.31,"15min":0.314},"min":{"1min":0.20799999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.38900000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.315,"15min":0.311},"min":{"1min":0.24099999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.43099999999999999}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.311,"15min":0.309},"min":{"1min":0.253,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.378,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.34599999999999997}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36899999999999999,"15min":0.371},"min":{"1min":0.23899999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.47699999999999998,"5min":0.505,"15min":0.505},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.37,"15min":0.371},"min":{"1min":0.30299999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.45400000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.36599999999999999}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.41899999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.747,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.44400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.77600000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.62}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56200000000000006,"15min":0.55700000000000005},"min":{"1min":0.46100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.44500000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.72599999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.60699999999999998}]},{"osd":6,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.47799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56699999999999995,"15min":0.56799999999999995},"min":{"1min":0.433,"5min":0.379,"15min":0.379},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.70399999999999996}]},{"osd":7,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.42899999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.69799999999999995,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.72899999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.63400000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28228,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343612,"statfs":{"total":96636764160,"available":96607858688,"internally_reserved":0,"allocated":1236992,"data_stored":875981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.31900000000000001,"15min":0.32300000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.38200000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.29599999999999999,"5min":0.35499999999999998,"15min":0.36499999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.44500000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.28799999999999998}]},{"osd":2,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.35699999999999998,"15min":0.36399999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.44700000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.36699999999999999,"15min":0.371},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.47699999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.251}]},{"osd":3,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.38500000000000001,"15min":0.39100000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.46400000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.41499999999999998,"15min":0.42099999999999999},"min":{"1min":0.309,"5min":0.307,"15min":0.307},"max":{"1min":0.48999999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.379}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58899999999999997,"15min":0.59299999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60599999999999998,"15min":0.60899999999999999},"min":{"1min":0.436,"5min":0.318,"15min":0.318},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.621}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.36199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76800000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.629,"15min":0.63600000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78900000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.52100000000000002}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.64100000000000001,"15min":0.64700000000000002},"min":{"1min":0.42499999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81000000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.627,"15min":0.63200000000000001},"min":{"1min":0.41699999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59699999999999998}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.44400000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.42999999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.71799999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.67000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-09T03:47:24.351 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-09T03:47:24.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.485+0000 7fe69a4d4640 1 Processor -- start 2023-11-09T03:47:24.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.486+0000 7fe69a4d4640 1 -- start start 2023-11-09T03:47:24.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.486+0000 7fe69a4d4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69415ae40 0x7fe69415bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:24.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.486+0000 7fe69a4d4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe6940cda30 0x7fe6940ced70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:24.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.486+0000 7fe69a4d4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6940cc450 con 0x7fe6940cbc00 2023-11-09T03:47:24.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.486+0000 7fe69a4d4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe69415f400 con 0x7fe6940cda30 2023-11-09T03:47:24.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.486+0000 7fe69a4d4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe69415b220 con 0x7fe69415ae40 2023-11-09T03:47:24.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.487+0000 7fe693fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69415ae40 0x7fe69415bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:24.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.487+0000 7fe693fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69415ae40 0x7fe69415bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:45378/0 (socket says 172.21.15.76:45378) 2023-11-09T03:47:24.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.487+0000 7fe693fff640 1 -- 172.21.15.76:0/3336386743 learned_addr learned my addr 172.21.15.76:0/3336386743 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:24.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.487+0000 7fe698a71640 1 --2- 172.21.15.76:0/3336386743 >> v2:172.21.15.113:3300/0 conn(0x7fe6940cda30 0x7fe6940ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:24.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.487+0000 7fe693fff640 1 -- 172.21.15.76:0/3336386743 >> v2:172.21.15.113:3300/0 conn(0x7fe6940cda30 msgr2=0x7fe6940ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:24.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.487+0000 7fe693fff640 1 --2- 172.21.15.76:0/3336386743 >> v2:172.21.15.113:3300/0 conn(0x7fe6940cda30 0x7fe6940ced70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:24.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.487+0000 7fe693fff640 1 -- 172.21.15.76:0/3336386743 >> v1:172.21.15.113:6789/0 conn(0x7fe6940cbc00 legacy=0x7fe6940cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:24.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.487+0000 7fe693fff640 1 -- 172.21.15.76:0/3336386743 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69415f890 con 0x7fe69415ae40 2023-11-09T03:47:24.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.488+0000 7fe693fff640 1 --2- 172.21.15.76:0/3336386743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69415ae40 0x7fe69415bfb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe684009870 tx=0x7fe68402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4129e874c812189d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:24.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.488+0000 7fe692ffd640 1 -- 172.21.15.76:0/3336386743 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684030840 con 0x7fe69415ae40 2023-11-09T03:47:24.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.488+0000 7fe692ffd640 1 -- 172.21.15.76:0/3336386743 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6840309a0 con 0x7fe69415ae40 2023-11-09T03:47:24.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.488+0000 7fe692ffd640 1 -- 172.21.15.76:0/3336386743 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684030d30 con 0x7fe69415ae40 2023-11-09T03:47:24.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.489+0000 7fe69a4d4640 1 -- 172.21.15.76:0/3336386743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69415ae40 msgr2=0x7fe69415bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:24.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.489+0000 7fe69a4d4640 1 --2- 172.21.15.76:0/3336386743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69415ae40 0x7fe69415bfb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe684009870 tx=0x7fe68402b980 comp rx=0 tx=0).stop 2023-11-09T03:47:24.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.489+0000 7fe69a4d4640 1 -- 172.21.15.76:0/3336386743 shutdown_connections 2023-11-09T03:47:24.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.489+0000 7fe69a4d4640 1 --2- 172.21.15.76:0/3336386743 >> v2:172.21.15.113:3300/0 conn(0x7fe6940cda30 0x7fe6940ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:24.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.489+0000 7fe69a4d4640 1 --2- 172.21.15.76:0/3336386743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69415ae40 0x7fe69415bfb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:24.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.489+0000 7fe69a4d4640 1 -- 172.21.15.76:0/3336386743 >> 172.21.15.76:0/3336386743 conn(0x7fe69406ea90 msgr2=0x7fe69406ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:24.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.489+0000 7fe69a4d4640 1 -- 172.21.15.76:0/3336386743 shutdown_connections 2023-11-09T03:47:24.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.489+0000 7fe69a4d4640 1 -- 172.21.15.76:0/3336386743 wait complete. 2023-11-09T03:47:24.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.490+0000 7fe69a4d4640 1 Processor -- start 2023-11-09T03:47:24.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.490+0000 7fe69a4d4640 1 -- start start 2023-11-09T03:47:24.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.491+0000 7fe69a4d4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6940cbc00 0x7fe6940c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:24.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.491+0000 7fe69a4d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6940cda30 0x7fe6940c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:24.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.491+0000 7fe69a4d4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69415ae40 0x7fe6940c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:24.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.491+0000 7fe69a4d4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6940c5530 con 0x7fe6940cda30 2023-11-09T03:47:24.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.491+0000 7fe69a4d4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6941f4130 con 0x7fe6940cbc00 2023-11-09T03:47:24.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.491+0000 7fe69a4d4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6940c7420 con 0x7fe69415ae40 2023-11-09T03:47:24.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.491+0000 7fe6937fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6940cda30 0x7fe6940c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:24.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.491+0000 7fe6937fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6940cda30 0x7fe6940c68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42534/0 (socket says 172.21.15.76:42534) 2023-11-09T03:47:24.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.491+0000 7fe6937fe640 1 -- 172.21.15.76:0/3024501596 learned_addr learned my addr 172.21.15.76:0/3024501596 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:24.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.491+0000 7fe693fff640 1 --2- 172.21.15.76:0/3024501596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6940cbc00 0x7fe6940c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:24.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.491+0000 7fe698a71640 1 --2- 172.21.15.76:0/3024501596 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69415ae40 0x7fe6940c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:24.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.491+0000 7fe6937fe640 1 -- 172.21.15.76:0/3024501596 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69415ae40 msgr2=0x7fe6940c9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:24.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.491+0000 7fe6937fe640 1 --2- 172.21.15.76:0/3024501596 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69415ae40 0x7fe6940c9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:24.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.491+0000 7fe6937fe640 1 -- 172.21.15.76:0/3024501596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6940cbc00 msgr2=0x7fe6940c46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:24.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.491+0000 7fe6937fe640 1 --2- 172.21.15.76:0/3024501596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6940cbc00 0x7fe6940c46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:24.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.491+0000 7fe6937fe640 1 -- 172.21.15.76:0/3024501596 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6840094d0 con 0x7fe6940cda30 2023-11-09T03:47:24.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.491+0000 7fe6937fe640 1 --2- 172.21.15.76:0/3024501596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6940cda30 0x7fe6940c68b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe68800c480 tx=0x7fe68800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:24.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.492+0000 7fe690ff9640 1 -- 172.21.15.76:0/3024501596 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6880052d0 con 0x7fe6940cda30 2023-11-09T03:47:24.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.492+0000 7fe690ff9640 1 -- 172.21.15.76:0/3024501596 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe688005430 con 0x7fe6940cda30 2023-11-09T03:47:24.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.492+0000 7fe690ff9640 1 -- 172.21.15.76:0/3024501596 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe688012870 con 0x7fe6940cda30 2023-11-09T03:47:24.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.492+0000 7fe69a4d4640 1 -- 172.21.15.76:0/3024501596 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6940c4c90 con 0x7fe6940cda30 2023-11-09T03:47:24.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.492+0000 7fe69a4d4640 1 -- 172.21.15.76:0/3024501596 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6940c5250 con 0x7fe6940cda30 2023-11-09T03:47:24.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.493+0000 7fe69a4d4640 1 -- 172.21.15.76:0/3024501596 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6940ce2d0 con 0x7fe6940cda30 2023-11-09T03:47:24.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.493+0000 7fe690ff9640 1 -- 172.21.15.76:0/3024501596 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe6880129d0 con 0x7fe6940cda30 2023-11-09T03:47:24.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.493+0000 7fe690ff9640 1 --2- 172.21.15.76:0/3024501596 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe68006aea0 0x7fe68006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:24.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.494+0000 7fe690ff9640 1 -- 172.21.15.76:0/3024501596 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe688089130 con 0x7fe6940cda30 2023-11-09T03:47:24.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.494+0000 7fe693fff640 1 --2- 172.21.15.76:0/3024501596 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe68006aea0 0x7fe68006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:24.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.494+0000 7fe693fff640 1 --2- 172.21.15.76:0/3024501596 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe68006aea0 0x7fe68006d360 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe684036000 tx=0x7fe68403b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:24.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.498+0000 7fe690ff9640 1 -- 172.21.15.76:0/3024501596 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe688054000 con 0x7fe6940cda30 2023-11-09T03:47:24.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.660+0000 7fe69a4d4640 1 -- 172.21.15.76:0/3024501596 --> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe69415edb0 con 0x7fe68006aea0 2023-11-09T03:47:24.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.669+0000 7fe690ff9640 1 -- 172.21.15.76:0/3024501596 <== mgr.4105 v2:172.21.15.113:6832/1264193042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+369232 (secure 0 0 0) 0x7fe68403f040 con 0x7fe68006aea0 2023-11-09T03:47:24.672 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:24.674 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2023-11-09T03:47:24.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.673+0000 7fe69a4d4640 1 -- 172.21.15.76:0/3024501596 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe68006aea0 msgr2=0x7fe68006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:24.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.673+0000 7fe69a4d4640 1 --2- 172.21.15.76:0/3024501596 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe68006aea0 0x7fe68006d360 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe684036000 tx=0x7fe68403b040 comp rx=0 tx=0).stop 2023-11-09T03:47:24.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.673+0000 7fe69a4d4640 1 -- 172.21.15.76:0/3024501596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6940cda30 msgr2=0x7fe6940c68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:24.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.673+0000 7fe69a4d4640 1 --2- 172.21.15.76:0/3024501596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6940cda30 0x7fe6940c68b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe68800c480 tx=0x7fe68800c940 comp rx=0 tx=0).stop 2023-11-09T03:47:24.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.673+0000 7fe69a4d4640 1 -- 172.21.15.76:0/3024501596 shutdown_connections 2023-11-09T03:47:24.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.673+0000 7fe69a4d4640 1 --2- 172.21.15.76:0/3024501596 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe68006aea0 0x7fe68006d360 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:24.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.673+0000 7fe69a4d4640 1 --2- 172.21.15.76:0/3024501596 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69415ae40 0x7fe6940c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:24.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.673+0000 7fe69a4d4640 1 --2- 172.21.15.76:0/3024501596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6940cda30 0x7fe6940c68b0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:24.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.673+0000 7fe69a4d4640 1 --2- 172.21.15.76:0/3024501596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6940cbc00 0x7fe6940c46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:24.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.673+0000 7fe69a4d4640 1 -- 172.21.15.76:0/3024501596 >> 172.21.15.76:0/3024501596 conn(0x7fe69406ea90 msgr2=0x7fe69415dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:24.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.673+0000 7fe69a4d4640 1 -- 172.21.15.76:0/3024501596 shutdown_connections 2023-11-09T03:47:24.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.673+0000 7fe69a4d4640 1 -- 172.21.15.76:0/3024501596 wait complete. 2023-11-09T03:47:24.683 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2023-11-09T03:47:22.777354+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":246,"acting":246,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":217,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222364,"kb_used_data":6172,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752356,"statfs":{"total":773094113280,"available":772866412544,"internally_reserved":0,"allocated":6320128,"data_stored":3471228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001064"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248379+0000","last_change":"2023-11-09T03:47:20.941252+0000","last_active":"2023-11-09T03:47:22.248379+0000","last_peered":"2023-11-09T03:47:22.248379+0000","last_clean":"2023-11-09T03:47:22.248379+0000","last_became_active":"2023-11-09T03:47:20.940995+0000","last_became_peered":"2023-11-09T03:47:20.940995+0000","last_unstale":"2023-11-09T03:47:22.248379+0000","last_undegraded":"2023-11-09T03:47:22.248379+0000","last_fullsized":"2023-11-09T03:47:22.248379+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:44:27.004461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248080+0000","last_change":"2023-11-09T03:47:20.950878+0000","last_active":"2023-11-09T03:47:22.248080+0000","last_peered":"2023-11-09T03:47:22.248080+0000","last_clean":"2023-11-09T03:47:22.248080+0000","last_became_active":"2023-11-09T03:47:20.950611+0000","last_became_peered":"2023-11-09T03:47:20.950611+0000","last_unstale":"2023-11-09T03:47:22.248080+0000","last_undegraded":"2023-11-09T03:47:22.248080+0000","last_fullsized":"2023-11-09T03:47:22.248080+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:04:18.340460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026954+0000","last_change":"2023-11-09T03:47:20.945634+0000","last_active":"2023-11-09T03:47:22.026954+0000","last_peered":"2023-11-09T03:47:22.026954+0000","last_clean":"2023-11-09T03:47:22.026954+0000","last_became_active":"2023-11-09T03:47:20.945370+0000","last_became_peered":"2023-11-09T03:47:20.945370+0000","last_unstale":"2023-11-09T03:47:22.026954+0000","last_undegraded":"2023-11-09T03:47:22.026954+0000","last_fullsized":"2023-11-09T03:47:22.026954+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:57.187676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940818+0000","last_change":"2023-11-09T03:47:20.941855+0000","last_active":"2023-11-09T03:47:21.940818+0000","last_peered":"2023-11-09T03:47:21.940818+0000","last_clean":"2023-11-09T03:47:21.940818+0000","last_became_active":"2023-11-09T03:47:20.941598+0000","last_became_peered":"2023-11-09T03:47:20.941598+0000","last_unstale":"2023-11-09T03:47:21.940818+0000","last_undegraded":"2023-11-09T03:47:21.940818+0000","last_fullsized":"2023-11-09T03:47:21.940818+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:14:00.724882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248512+0000","last_change":"2023-11-09T03:47:20.947154+0000","last_active":"2023-11-09T03:47:22.248512+0000","last_peered":"2023-11-09T03:47:22.248512+0000","last_clean":"2023-11-09T03:47:22.248512+0000","last_became_active":"2023-11-09T03:47:20.946878+0000","last_became_peered":"2023-11-09T03:47:20.946878+0000","last_unstale":"2023-11-09T03:47:22.248512+0000","last_undegraded":"2023-11-09T03:47:22.248512+0000","last_fullsized":"2023-11-09T03:47:22.248512+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:41:16.327295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247898+0000","last_change":"2023-11-09T03:47:20.941698+0000","last_active":"2023-11-09T03:47:22.247898+0000","last_peered":"2023-11-09T03:47:22.247898+0000","last_clean":"2023-11-09T03:47:22.247898+0000","last_became_active":"2023-11-09T03:47:20.941432+0000","last_became_peered":"2023-11-09T03:47:20.941432+0000","last_unstale":"2023-11-09T03:47:22.247898+0000","last_undegraded":"2023-11-09T03:47:22.247898+0000","last_fullsized":"2023-11-09T03:47:22.247898+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:12:48.690396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247980+0000","last_change":"2023-11-09T03:47:20.948132+0000","last_active":"2023-11-09T03:47:22.247980+0000","last_peered":"2023-11-09T03:47:22.247980+0000","last_clean":"2023-11-09T03:47:22.247980+0000","last_became_active":"2023-11-09T03:47:20.947854+0000","last_became_peered":"2023-11-09T03:47:20.947854+0000","last_unstale":"2023-11-09T03:47:22.247980+0000","last_undegraded":"2023-11-09T03:47:22.247980+0000","last_fullsized":"2023-11-09T03:47:22.247980+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:38:09.677347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248627+0000","last_change":"2023-11-09T03:47:20.944502+0000","last_active":"2023-11-09T03:47:22.248627+0000","last_peered":"2023-11-09T03:47:22.248627+0000","last_clean":"2023-11-09T03:47:22.248627+0000","last_became_active":"2023-11-09T03:47:20.944214+0000","last_became_peered":"2023-11-09T03:47:20.944214+0000","last_unstale":"2023-11-09T03:47:22.248627+0000","last_undegraded":"2023-11-09T03:47:22.248627+0000","last_fullsized":"2023-11-09T03:47:22.248627+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:22:14.432893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248644+0000","last_change":"2023-11-09T03:47:20.954329+0000","last_active":"2023-11-09T03:47:22.248644+0000","last_peered":"2023-11-09T03:47:22.248644+0000","last_clean":"2023-11-09T03:47:22.248644+0000","last_became_active":"2023-11-09T03:47:20.953654+0000","last_became_peered":"2023-11-09T03:47:20.953654+0000","last_unstale":"2023-11-09T03:47:22.248644+0000","last_undegraded":"2023-11-09T03:47:22.248644+0000","last_fullsized":"2023-11-09T03:47:22.248644+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:48:27.582736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939384+0000","last_change":"2023-11-09T03:47:20.943196+0000","last_active":"2023-11-09T03:47:21.939384+0000","last_peered":"2023-11-09T03:47:21.939384+0000","last_clean":"2023-11-09T03:47:21.939384+0000","last_became_active":"2023-11-09T03:47:20.942943+0000","last_became_peered":"2023-11-09T03:47:20.942943+0000","last_unstale":"2023-11-09T03:47:21.939384+0000","last_undegraded":"2023-11-09T03:47:21.939384+0000","last_fullsized":"2023-11-09T03:47:21.939384+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:36:04.069418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939364+0000","last_change":"2023-11-09T03:47:20.943821+0000","last_active":"2023-11-09T03:47:21.939364+0000","last_peered":"2023-11-09T03:47:21.939364+0000","last_clean":"2023-11-09T03:47:21.939364+0000","last_became_active":"2023-11-09T03:47:20.943571+0000","last_became_peered":"2023-11-09T03:47:20.943571+0000","last_unstale":"2023-11-09T03:47:21.939364+0000","last_undegraded":"2023-11-09T03:47:21.939364+0000","last_fullsized":"2023-11-09T03:47:21.939364+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:26:12.103741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940598+0000","last_change":"2023-11-09T03:47:20.942582+0000","last_active":"2023-11-09T03:47:21.940598+0000","last_peered":"2023-11-09T03:47:21.940598+0000","last_clean":"2023-11-09T03:47:21.940598+0000","last_became_active":"2023-11-09T03:47:20.942214+0000","last_became_peered":"2023-11-09T03:47:20.942214+0000","last_unstale":"2023-11-09T03:47:21.940598+0000","last_undegraded":"2023-11-09T03:47:21.940598+0000","last_fullsized":"2023-11-09T03:47:21.940598+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:51:53.879857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-09T03:47:19.915800+0000","last_change":"2023-11-09T03:47:19.915800+0000","last_active":"2023-11-09T03:47:19.915800+0000","last_peered":"2023-11-09T03:47:19.915800+0000","last_clean":"2023-11-09T03:47:19.915800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:47:19.915800+0000","last_undegraded":"2023-11-09T03:47:19.915800+0000","last_fullsized":"2023-11-09T03:47:19.915800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248066+0000","last_change":"2023-11-09T03:47:20.944425+0000","last_active":"2023-11-09T03:47:22.248066+0000","last_peered":"2023-11-09T03:47:22.248066+0000","last_clean":"2023-11-09T03:47:22.248066+0000","last_became_active":"2023-11-09T03:47:20.944159+0000","last_became_peered":"2023-11-09T03:47:20.944159+0000","last_unstale":"2023-11-09T03:47:22.248066+0000","last_undegraded":"2023-11-09T03:47:22.248066+0000","last_fullsized":"2023-11-09T03:47:22.248066+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:26:12.103741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248162+0000","last_change":"2023-11-09T03:47:20.954225+0000","last_active":"2023-11-09T03:47:22.248162+0000","last_peered":"2023-11-09T03:47:22.248162+0000","last_clean":"2023-11-09T03:47:22.248162+0000","last_became_active":"2023-11-09T03:47:20.954004+0000","last_became_peered":"2023-11-09T03:47:20.954004+0000","last_unstale":"2023-11-09T03:47:22.248162+0000","last_undegraded":"2023-11-09T03:47:22.248162+0000","last_fullsized":"2023-11-09T03:47:22.248162+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:36:04.069418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247697+0000","last_change":"2023-11-09T03:47:20.940209+0000","last_active":"2023-11-09T03:47:22.247697+0000","last_peered":"2023-11-09T03:47:22.247697+0000","last_clean":"2023-11-09T03:47:22.247697+0000","last_became_active":"2023-11-09T03:47:20.939956+0000","last_became_peered":"2023-11-09T03:47:20.939956+0000","last_unstale":"2023-11-09T03:47:22.247697+0000","last_undegraded":"2023-11-09T03:47:22.247697+0000","last_fullsized":"2023-11-09T03:47:22.247697+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:52.683555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248360+0000","last_change":"2023-11-09T03:47:20.942850+0000","last_active":"2023-11-09T03:47:22.248360+0000","last_peered":"2023-11-09T03:47:22.248360+0000","last_clean":"2023-11-09T03:47:22.248360+0000","last_became_active":"2023-11-09T03:47:20.942596+0000","last_became_peered":"2023-11-09T03:47:20.942596+0000","last_unstale":"2023-11-09T03:47:22.248360+0000","last_undegraded":"2023-11-09T03:47:22.248360+0000","last_fullsized":"2023-11-09T03:47:22.248360+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:14:28.441176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.027085+0000","last_change":"2023-11-09T03:47:20.954037+0000","last_active":"2023-11-09T03:47:22.027085+0000","last_peered":"2023-11-09T03:47:22.027085+0000","last_clean":"2023-11-09T03:47:22.027085+0000","last_became_active":"2023-11-09T03:47:20.953777+0000","last_became_peered":"2023-11-09T03:47:20.953777+0000","last_unstale":"2023-11-09T03:47:22.027085+0000","last_undegraded":"2023-11-09T03:47:22.027085+0000","last_fullsized":"2023-11-09T03:47:22.027085+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:55:28.077165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248233+0000","last_change":"2023-11-09T03:47:20.941849+0000","last_active":"2023-11-09T03:47:22.248233+0000","last_peered":"2023-11-09T03:47:22.248233+0000","last_clean":"2023-11-09T03:47:22.248233+0000","last_became_active":"2023-11-09T03:47:20.941602+0000","last_became_peered":"2023-11-09T03:47:20.941602+0000","last_unstale":"2023-11-09T03:47:22.248233+0000","last_undegraded":"2023-11-09T03:47:22.248233+0000","last_fullsized":"2023-11-09T03:47:22.248233+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:23.398916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248619+0000","last_change":"2023-11-09T03:47:20.951388+0000","last_active":"2023-11-09T03:47:22.248619+0000","last_peered":"2023-11-09T03:47:22.248619+0000","last_clean":"2023-11-09T03:47:22.248619+0000","last_became_active":"2023-11-09T03:47:20.951123+0000","last_became_peered":"2023-11-09T03:47:20.951123+0000","last_unstale":"2023-11-09T03:47:22.248619+0000","last_undegraded":"2023-11-09T03:47:22.248619+0000","last_fullsized":"2023-11-09T03:47:22.248619+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:55:54.333416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026719+0000","last_change":"2023-11-09T03:47:20.948114+0000","last_active":"2023-11-09T03:47:22.026719+0000","last_peered":"2023-11-09T03:47:22.026719+0000","last_clean":"2023-11-09T03:47:22.026719+0000","last_became_active":"2023-11-09T03:47:20.947849+0000","last_became_peered":"2023-11-09T03:47:20.947849+0000","last_unstale":"2023-11-09T03:47:22.026719+0000","last_undegraded":"2023-11-09T03:47:22.026719+0000","last_fullsized":"2023-11-09T03:47:22.026719+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:52.683555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247955+0000","last_change":"2023-11-09T03:47:20.945723+0000","last_active":"2023-11-09T03:47:22.247955+0000","last_peered":"2023-11-09T03:47:22.247955+0000","last_clean":"2023-11-09T03:47:22.247955+0000","last_became_active":"2023-11-09T03:47:20.945447+0000","last_became_peered":"2023-11-09T03:47:20.945447+0000","last_unstale":"2023-11-09T03:47:22.247955+0000","last_undegraded":"2023-11-09T03:47:22.247955+0000","last_fullsized":"2023-11-09T03:47:22.247955+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:41:48.124870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026634+0000","last_change":"2023-11-09T03:47:20.944344+0000","last_active":"2023-11-09T03:47:22.026634+0000","last_peered":"2023-11-09T03:47:22.026634+0000","last_clean":"2023-11-09T03:47:22.026634+0000","last_became_active":"2023-11-09T03:47:20.943991+0000","last_became_peered":"2023-11-09T03:47:20.943991+0000","last_unstale":"2023-11-09T03:47:22.026634+0000","last_undegraded":"2023-11-09T03:47:22.026634+0000","last_fullsized":"2023-11-09T03:47:22.026634+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:08:17.239107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-09T03:47:19.915800+0000","last_change":"2023-11-09T03:47:19.915800+0000","last_active":"2023-11-09T03:47:19.915800+0000","last_peered":"2023-11-09T03:47:19.915800+0000","last_clean":"2023-11-09T03:47:19.915800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:47:19.915800+0000","last_undegraded":"2023-11-09T03:47:19.915800+0000","last_fullsized":"2023-11-09T03:47:19.915800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248496+0000","last_change":"2023-11-09T03:47:20.952835+0000","last_active":"2023-11-09T03:47:22.248496+0000","last_peered":"2023-11-09T03:47:22.248496+0000","last_clean":"2023-11-09T03:47:22.248496+0000","last_became_active":"2023-11-09T03:47:20.952578+0000","last_became_peered":"2023-11-09T03:47:20.952578+0000","last_unstale":"2023-11-09T03:47:22.248496+0000","last_undegraded":"2023-11-09T03:47:22.248496+0000","last_fullsized":"2023-11-09T03:47:22.248496+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:40:53.848669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.941032+0000","last_change":"2023-11-09T03:47:20.944379+0000","last_active":"2023-11-09T03:47:21.941032+0000","last_peered":"2023-11-09T03:47:21.941032+0000","last_clean":"2023-11-09T03:47:21.941032+0000","last_became_active":"2023-11-09T03:47:20.944134+0000","last_became_peered":"2023-11-09T03:47:20.944134+0000","last_unstale":"2023-11-09T03:47:21.941032+0000","last_undegraded":"2023-11-09T03:47:21.941032+0000","last_fullsized":"2023-11-09T03:47:21.941032+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:55:28.077165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248970+0000","last_change":"2023-11-09T03:47:20.940786+0000","last_active":"2023-11-09T03:47:22.248970+0000","last_peered":"2023-11-09T03:47:22.248970+0000","last_clean":"2023-11-09T03:47:22.248970+0000","last_became_active":"2023-11-09T03:47:20.940529+0000","last_became_peered":"2023-11-09T03:47:20.940529+0000","last_unstale":"2023-11-09T03:47:22.248970+0000","last_undegraded":"2023-11-09T03:47:22.248970+0000","last_fullsized":"2023-11-09T03:47:22.248970+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:40.639821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248129+0000","last_change":"2023-11-09T03:47:20.949337+0000","last_active":"2023-11-09T03:47:22.248129+0000","last_peered":"2023-11-09T03:47:22.248129+0000","last_clean":"2023-11-09T03:47:22.248129+0000","last_became_active":"2023-11-09T03:47:20.949088+0000","last_became_peered":"2023-11-09T03:47:20.949088+0000","last_unstale":"2023-11-09T03:47:22.248129+0000","last_undegraded":"2023-11-09T03:47:22.248129+0000","last_fullsized":"2023-11-09T03:47:22.248129+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:52:12.660646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248774+0000","last_change":"2023-11-09T03:47:20.948003+0000","last_active":"2023-11-09T03:47:22.248774+0000","last_peered":"2023-11-09T03:47:22.248774+0000","last_clean":"2023-11-09T03:47:22.248774+0000","last_became_active":"2023-11-09T03:47:20.947690+0000","last_became_peered":"2023-11-09T03:47:20.947690+0000","last_unstale":"2023-11-09T03:47:22.248774+0000","last_undegraded":"2023-11-09T03:47:22.248774+0000","last_fullsized":"2023-11-09T03:47:22.248774+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:28:18.013453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026610+0000","last_change":"2023-11-09T03:47:20.946260+0000","last_active":"2023-11-09T03:47:22.026610+0000","last_peered":"2023-11-09T03:47:22.026610+0000","last_clean":"2023-11-09T03:47:22.026610+0000","last_became_active":"2023-11-09T03:47:20.945986+0000","last_became_peered":"2023-11-09T03:47:20.945986+0000","last_unstale":"2023-11-09T03:47:22.026610+0000","last_undegraded":"2023-11-09T03:47:22.026610+0000","last_fullsized":"2023-11-09T03:47:22.026610+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:59:29.154623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247961+0000","last_change":"2023-11-09T03:47:20.952176+0000","last_active":"2023-11-09T03:47:22.247961+0000","last_peered":"2023-11-09T03:47:22.247961+0000","last_clean":"2023-11-09T03:47:22.247961+0000","last_became_active":"2023-11-09T03:47:20.951921+0000","last_became_peered":"2023-11-09T03:47:20.951921+0000","last_unstale":"2023-11-09T03:47:22.247961+0000","last_undegraded":"2023-11-09T03:47:22.247961+0000","last_fullsized":"2023-11-09T03:47:22.247961+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:22.376023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247669+0000","last_change":"2023-11-09T03:47:20.943940+0000","last_active":"2023-11-09T03:47:22.247669+0000","last_peered":"2023-11-09T03:47:22.247669+0000","last_clean":"2023-11-09T03:47:22.247669+0000","last_became_active":"2023-11-09T03:47:20.943708+0000","last_became_peered":"2023-11-09T03:47:20.943708+0000","last_unstale":"2023-11-09T03:47:22.247669+0000","last_undegraded":"2023-11-09T03:47:22.247669+0000","last_fullsized":"2023-11-09T03:47:22.247669+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:08:17.239107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026790+0000","last_change":"2023-11-09T03:47:20.942360+0000","last_active":"2023-11-09T03:47:22.026790+0000","last_peered":"2023-11-09T03:47:22.026790+0000","last_clean":"2023-11-09T03:47:22.026790+0000","last_became_active":"2023-11-09T03:47:20.942078+0000","last_became_peered":"2023-11-09T03:47:20.942078+0000","last_unstale":"2023-11-09T03:47:22.026790+0000","last_undegraded":"2023-11-09T03:47:22.026790+0000","last_fullsized":"2023-11-09T03:47:22.026790+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:12:48.690396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-09T03:47:18.911117+0000","last_change":"2023-11-09T03:47:18.911117+0000","last_active":"2023-11-09T03:47:18.911117+0000","last_peered":"2023-11-09T03:47:18.911117+0000","last_clean":"2023-11-09T03:47:18.911117+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:47:18.911117+0000","last_undegraded":"2023-11-09T03:47:18.911117+0000","last_fullsized":"2023-11-09T03:47:18.911117+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":27,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.928645+0000","last_change":"2023-11-09T03:47:02.581917+0000","last_active":"2023-11-09T03:47:20.928645+0000","last_peered":"2023-11-09T03:47:20.928645+0000","last_clean":"2023-11-09T03:47:20.928645+0000","last_became_active":"2023-11-09T03:47:01.588082+0000","last_became_peered":"2023-11-09T03:47:01.588082+0000","last_unstale":"2023-11-09T03:47:20.928645+0000","last_undegraded":"2023-11-09T03:47:20.928645+0000","last_fullsized":"2023-11-09T03:47:20.928645+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:44:07.653557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034538899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940245+0000","last_change":"2023-11-09T03:47:20.938900+0000","last_active":"2023-11-09T03:47:21.940245+0000","last_peered":"2023-11-09T03:47:21.940245+0000","last_clean":"2023-11-09T03:47:21.940245+0000","last_became_active":"2023-11-09T03:47:20.938654+0000","last_became_peered":"2023-11-09T03:47:20.938654+0000","last_unstale":"2023-11-09T03:47:21.940245+0000","last_undegraded":"2023-11-09T03:47:21.940245+0000","last_fullsized":"2023-11-09T03:47:21.940245+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:40.639821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248651+0000","last_change":"2023-11-09T03:47:19.945834+0000","last_active":"2023-11-09T03:47:22.248651+0000","last_peered":"2023-11-09T03:47:22.248651+0000","last_clean":"2023-11-09T03:47:22.248651+0000","last_became_active":"2023-11-09T03:47:19.945484+0000","last_became_peered":"2023-11-09T03:47:19.945484+0000","last_unstale":"2023-11-09T03:47:22.248651+0000","last_undegraded":"2023-11-09T03:47:22.248651+0000","last_fullsized":"2023-11-09T03:47:22.248651+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:57:21.887088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.930614+0000","last_change":"2023-11-09T03:47:03.029063+0000","last_active":"2023-11-09T03:47:20.930614+0000","last_peered":"2023-11-09T03:47:20.930614+0000","last_clean":"2023-11-09T03:47:20.930614+0000","last_became_active":"2023-11-09T03:47:01.593582+0000","last_became_peered":"2023-11-09T03:47:01.593582+0000","last_unstale":"2023-11-09T03:47:20.930614+0000","last_undegraded":"2023-11-09T03:47:20.930614+0000","last_fullsized":"2023-11-09T03:47:20.930614+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:56:20.649937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041855100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026721+0000","last_change":"2023-11-09T03:47:20.944419+0000","last_active":"2023-11-09T03:47:22.026721+0000","last_peered":"2023-11-09T03:47:22.026721+0000","last_clean":"2023-11-09T03:47:22.026721+0000","last_became_active":"2023-11-09T03:47:20.944141+0000","last_became_peered":"2023-11-09T03:47:20.944141+0000","last_unstale":"2023-11-09T03:47:22.026721+0000","last_undegraded":"2023-11-09T03:47:22.026721+0000","last_fullsized":"2023-11-09T03:47:22.026721+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:06:17.676049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"53'8","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026516+0000","last_change":"2023-11-09T03:47:19.946625+0000","last_active":"2023-11-09T03:47:22.026516+0000","last_peered":"2023-11-09T03:47:22.026516+0000","last_clean":"2023-11-09T03:47:22.026516+0000","last_became_active":"2023-11-09T03:47:19.946350+0000","last_became_peered":"2023-11-09T03:47:19.946350+0000","last_unstale":"2023-11-09T03:47:22.026516+0000","last_undegraded":"2023-11-09T03:47:22.026516+0000","last_fullsized":"2023-11-09T03:47:22.026516+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:32.646643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":21,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-09T03:47:15.976974+0000","last_change":"2023-11-09T03:47:02.619336+0000","last_active":"2023-11-09T03:47:15.976974+0000","last_peered":"2023-11-09T03:47:15.976974+0000","last_clean":"2023-11-09T03:47:15.976974+0000","last_became_active":"2023-11-09T03:47:01.587749+0000","last_became_peered":"2023-11-09T03:47:01.587749+0000","last_unstale":"2023-11-09T03:47:15.976974+0000","last_undegraded":"2023-11-09T03:47:15.976974+0000","last_fullsized":"2023-11-09T03:47:15.976974+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063032099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247912+0000","last_change":"2023-11-09T03:47:20.948391+0000","last_active":"2023-11-09T03:47:22.247912+0000","last_peered":"2023-11-09T03:47:22.247912+0000","last_clean":"2023-11-09T03:47:22.247912+0000","last_became_active":"2023-11-09T03:47:20.948103+0000","last_became_peered":"2023-11-09T03:47:20.948103+0000","last_unstale":"2023-11-09T03:47:22.247912+0000","last_undegraded":"2023-11-09T03:47:22.247912+0000","last_fullsized":"2023-11-09T03:47:22.247912+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:33.651326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940202+0000","last_change":"2023-11-09T03:47:19.944514+0000","last_active":"2023-11-09T03:47:21.940202+0000","last_peered":"2023-11-09T03:47:21.940202+0000","last_clean":"2023-11-09T03:47:21.940202+0000","last_became_active":"2023-11-09T03:47:19.944230+0000","last_became_peered":"2023-11-09T03:47:19.944230+0000","last_unstale":"2023-11-09T03:47:21.940202+0000","last_undegraded":"2023-11-09T03:47:21.940202+0000","last_fullsized":"2023-11-09T03:47:21.940202+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:08:16.234424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":27,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.928406+0000","last_change":"2023-11-09T03:47:02.581658+0000","last_active":"2023-11-09T03:47:20.928406+0000","last_peered":"2023-11-09T03:47:20.928406+0000","last_clean":"2023-11-09T03:47:20.928406+0000","last_became_active":"2023-11-09T03:47:01.588612+0000","last_became_peered":"2023-11-09T03:47:01.588612+0000","last_unstale":"2023-11-09T03:47:20.928406+0000","last_undegraded":"2023-11-09T03:47:20.928406+0000","last_fullsized":"2023-11-09T03:47:20.928406+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045238500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.027066+0000","last_change":"2023-11-09T03:47:20.952326+0000","last_active":"2023-11-09T03:47:22.027066+0000","last_peered":"2023-11-09T03:47:22.027066+0000","last_clean":"2023-11-09T03:47:22.027066+0000","last_became_active":"2023-11-09T03:47:20.952035+0000","last_became_peered":"2023-11-09T03:47:20.952035+0000","last_unstale":"2023-11-09T03:47:22.027066+0000","last_undegraded":"2023-11-09T03:47:22.027066+0000","last_fullsized":"2023-11-09T03:47:22.027066+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:14:00.724882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248147+0000","last_change":"2023-11-09T03:47:19.947282+0000","last_active":"2023-11-09T03:47:22.248147+0000","last_peered":"2023-11-09T03:47:22.248147+0000","last_clean":"2023-11-09T03:47:22.248147+0000","last_became_active":"2023-11-09T03:47:19.947034+0000","last_became_peered":"2023-11-09T03:47:19.947034+0000","last_unstale":"2023-11-09T03:47:22.248147+0000","last_undegraded":"2023-11-09T03:47:22.248147+0000","last_fullsized":"2023-11-09T03:47:22.248147+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:13:59.720199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":21,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-09T03:47:15.977118+0000","last_change":"2023-11-09T03:47:02.618828+0000","last_active":"2023-11-09T03:47:15.977118+0000","last_peered":"2023-11-09T03:47:15.977118+0000","last_clean":"2023-11-09T03:47:15.977118+0000","last_became_active":"2023-11-09T03:47:01.590679+0000","last_became_peered":"2023-11-09T03:47:01.590679+0000","last_unstale":"2023-11-09T03:47:15.977118+0000","last_undegraded":"2023-11-09T03:47:15.977118+0000","last_fullsized":"2023-11-09T03:47:15.977118+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:06:30.982987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070668,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248700+0000","last_change":"2023-11-09T03:47:20.954224+0000","last_active":"2023-11-09T03:47:22.248700+0000","last_peered":"2023-11-09T03:47:22.248700+0000","last_clean":"2023-11-09T03:47:22.248700+0000","last_became_active":"2023-11-09T03:47:20.953950+0000","last_became_peered":"2023-11-09T03:47:20.953950+0000","last_unstale":"2023-11-09T03:47:22.248700+0000","last_undegraded":"2023-11-09T03:47:22.248700+0000","last_fullsized":"2023-11-09T03:47:22.248700+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:58:09.769432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.027101+0000","last_change":"2023-11-09T03:47:19.947929+0000","last_active":"2023-11-09T03:47:22.027101+0000","last_peered":"2023-11-09T03:47:22.027101+0000","last_clean":"2023-11-09T03:47:22.027101+0000","last_became_active":"2023-11-09T03:47:19.947663+0000","last_became_peered":"2023-11-09T03:47:19.947663+0000","last_unstale":"2023-11-09T03:47:22.027101+0000","last_undegraded":"2023-11-09T03:47:22.027101+0000","last_fullsized":"2023-11-09T03:47:22.027101+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:02:12.613580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931259+0000","last_change":"2023-11-09T03:47:03.146528+0000","last_active":"2023-11-09T03:47:20.931259+0000","last_peered":"2023-11-09T03:47:20.931259+0000","last_clean":"2023-11-09T03:47:20.931259+0000","last_became_active":"2023-11-09T03:47:01.587060+0000","last_became_peered":"2023-11-09T03:47:01.587060+0000","last_unstale":"2023-11-09T03:47:20.931259+0000","last_undegraded":"2023-11-09T03:47:20.931259+0000","last_fullsized":"2023-11-09T03:47:20.931259+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:39:15.238496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079064999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.027243+0000","last_change":"2023-11-09T03:47:20.954274+0000","last_active":"2023-11-09T03:47:22.027243+0000","last_peered":"2023-11-09T03:47:22.027243+0000","last_clean":"2023-11-09T03:47:22.027243+0000","last_became_active":"2023-11-09T03:47:20.953976+0000","last_became_peered":"2023-11-09T03:47:20.953976+0000","last_unstale":"2023-11-09T03:47:22.027243+0000","last_undegraded":"2023-11-09T03:47:22.027243+0000","last_fullsized":"2023-11-09T03:47:22.027243+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:05:32.662676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"53'6","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248315+0000","last_change":"2023-11-09T03:47:19.948464+0000","last_active":"2023-11-09T03:47:22.248315+0000","last_peered":"2023-11-09T03:47:22.248315+0000","last_clean":"2023-11-09T03:47:22.248315+0000","last_became_active":"2023-11-09T03:47:19.948254+0000","last_became_peered":"2023-11-09T03:47:19.948254+0000","last_unstale":"2023-11-09T03:47:22.248315+0000","last_undegraded":"2023-11-09T03:47:22.248315+0000","last_fullsized":"2023-11-09T03:47:22.248315+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:21:03.454473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931191+0000","last_change":"2023-11-09T03:47:03.148246+0000","last_active":"2023-11-09T03:47:20.931191+0000","last_peered":"2023-11-09T03:47:20.931191+0000","last_clean":"2023-11-09T03:47:20.931191+0000","last_became_active":"2023-11-09T03:47:01.590988+0000","last_became_peered":"2023-11-09T03:47:01.590988+0000","last_unstale":"2023-11-09T03:47:20.931191+0000","last_undegraded":"2023-11-09T03:47:20.931191+0000","last_fullsized":"2023-11-09T03:47:20.931191+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:07:20.785445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055651099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939172+0000","last_change":"2023-11-09T03:47:20.945226+0000","last_active":"2023-11-09T03:47:21.939172+0000","last_peered":"2023-11-09T03:47:21.939172+0000","last_clean":"2023-11-09T03:47:21.939172+0000","last_became_active":"2023-11-09T03:47:20.944978+0000","last_became_peered":"2023-11-09T03:47:20.944978+0000","last_unstale":"2023-11-09T03:47:21.939172+0000","last_undegraded":"2023-11-09T03:47:21.939172+0000","last_fullsized":"2023-11-09T03:47:21.939172+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:33.651326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"53'3","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026995+0000","last_change":"2023-11-09T03:47:19.945101+0000","last_active":"2023-11-09T03:47:22.026995+0000","last_peered":"2023-11-09T03:47:22.026995+0000","last_clean":"2023-11-09T03:47:22.026995+0000","last_became_active":"2023-11-09T03:47:19.944829+0000","last_became_peered":"2023-11-09T03:47:19.944829+0000","last_unstale":"2023-11-09T03:47:22.026995+0000","last_undegraded":"2023-11-09T03:47:22.026995+0000","last_fullsized":"2023-11-09T03:47:22.026995+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:49:12.247865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.939514+0000","last_change":"2023-11-09T03:47:02.582754+0000","last_active":"2023-11-09T03:47:20.939514+0000","last_peered":"2023-11-09T03:47:20.939514+0000","last_clean":"2023-11-09T03:47:20.939514+0000","last_became_active":"2023-11-09T03:47:01.589715+0000","last_became_peered":"2023-11-09T03:47:01.589715+0000","last_unstale":"2023-11-09T03:47:20.939514+0000","last_undegraded":"2023-11-09T03:47:20.939514+0000","last_fullsized":"2023-11-09T03:47:20.939514+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099342799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248560+0000","last_change":"2023-11-09T03:47:20.953974+0000","last_active":"2023-11-09T03:47:22.248560+0000","last_peered":"2023-11-09T03:47:22.248560+0000","last_clean":"2023-11-09T03:47:22.248560+0000","last_became_active":"2023-11-09T03:47:20.953714+0000","last_became_peered":"2023-11-09T03:47:20.953714+0000","last_unstale":"2023-11-09T03:47:22.248560+0000","last_undegraded":"2023-11-09T03:47:22.248560+0000","last_fullsized":"2023-11-09T03:47:22.248560+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:40:44.918031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"53'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248845+0000","last_change":"2023-11-09T03:47:19.943054+0000","last_active":"2023-11-09T03:47:22.248845+0000","last_peered":"2023-11-09T03:47:22.248845+0000","last_clean":"2023-11-09T03:47:22.248845+0000","last_became_active":"2023-11-09T03:47:19.942787+0000","last_became_peered":"2023-11-09T03:47:19.942787+0000","last_unstale":"2023-11-09T03:47:22.248845+0000","last_undegraded":"2023-11-09T03:47:22.248845+0000","last_fullsized":"2023-11-09T03:47:22.248845+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:58:08.764749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.939087+0000","last_change":"2023-11-09T03:47:02.724896+0000","last_active":"2023-11-09T03:47:20.939087+0000","last_peered":"2023-11-09T03:47:20.939087+0000","last_clean":"2023-11-09T03:47:20.939087+0000","last_became_active":"2023-11-09T03:47:01.592784+0000","last_became_peered":"2023-11-09T03:47:01.592784+0000","last_unstale":"2023-11-09T03:47:20.939087+0000","last_undegraded":"2023-11-09T03:47:20.939087+0000","last_fullsized":"2023-11-09T03:47:20.939087+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:51:32.016204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056683500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248073+0000","last_change":"2023-11-09T03:47:20.945066+0000","last_active":"2023-11-09T03:47:22.248073+0000","last_peered":"2023-11-09T03:47:22.248073+0000","last_clean":"2023-11-09T03:47:22.248073+0000","last_became_active":"2023-11-09T03:47:20.944812+0000","last_became_peered":"2023-11-09T03:47:20.944812+0000","last_unstale":"2023-11-09T03:47:22.248073+0000","last_undegraded":"2023-11-09T03:47:22.248073+0000","last_fullsized":"2023-11-09T03:47:22.248073+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:58:30.558728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"53'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248249+0000","last_change":"2023-11-09T03:47:19.947954+0000","last_active":"2023-11-09T03:47:22.248249+0000","last_peered":"2023-11-09T03:47:22.248249+0000","last_clean":"2023-11-09T03:47:22.248249+0000","last_became_active":"2023-11-09T03:47:19.947687+0000","last_became_peered":"2023-11-09T03:47:19.947687+0000","last_unstale":"2023-11-09T03:47:22.248249+0000","last_undegraded":"2023-11-09T03:47:22.248249+0000","last_fullsized":"2023-11-09T03:47:22.248249+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:52:58.422763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.933388+0000","last_change":"2023-11-09T03:47:03.146527+0000","last_active":"2023-11-09T03:47:20.933388+0000","last_peered":"2023-11-09T03:47:20.933388+0000","last_clean":"2023-11-09T03:47:20.933388+0000","last_became_active":"2023-11-09T03:47:01.587858+0000","last_became_peered":"2023-11-09T03:47:01.587858+0000","last_unstale":"2023-11-09T03:47:20.933388+0000","last_undegraded":"2023-11-09T03:47:20.933388+0000","last_fullsized":"2023-11-09T03:47:20.933388+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:56:04.201126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084439599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248163+0000","last_change":"2023-11-09T03:47:20.943815+0000","last_active":"2023-11-09T03:47:22.248163+0000","last_peered":"2023-11-09T03:47:22.248163+0000","last_clean":"2023-11-09T03:47:22.248163+0000","last_became_active":"2023-11-09T03:47:20.943403+0000","last_became_peered":"2023-11-09T03:47:20.943403+0000","last_unstale":"2023-11-09T03:47:22.248163+0000","last_undegraded":"2023-11-09T03:47:22.248163+0000","last_fullsized":"2023-11-09T03:47:22.248163+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:59:34.738692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247950+0000","last_change":"2023-11-09T03:47:19.949858+0000","last_active":"2023-11-09T03:47:22.247950+0000","last_peered":"2023-11-09T03:47:22.247950+0000","last_clean":"2023-11-09T03:47:22.247950+0000","last_became_active":"2023-11-09T03:47:19.949636+0000","last_became_peered":"2023-11-09T03:47:19.949636+0000","last_unstale":"2023-11-09T03:47:22.247950+0000","last_undegraded":"2023-11-09T03:47:22.247950+0000","last_fullsized":"2023-11-09T03:47:22.247950+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:24.100578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.935999+0000","last_change":"2023-11-09T03:47:02.725223+0000","last_active":"2023-11-09T03:47:20.935999+0000","last_peered":"2023-11-09T03:47:20.935999+0000","last_clean":"2023-11-09T03:47:20.935999+0000","last_became_active":"2023-11-09T03:47:01.590635+0000","last_became_peered":"2023-11-09T03:47:01.590635+0000","last_unstale":"2023-11-09T03:47:20.935999+0000","last_undegraded":"2023-11-09T03:47:20.935999+0000","last_fullsized":"2023-11-09T03:47:20.935999+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:05.754357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051685100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-09T03:47:19.915800+0000","last_change":"2023-11-09T03:47:19.915800+0000","last_active":"2023-11-09T03:47:19.915800+0000","last_peered":"2023-11-09T03:47:19.915800+0000","last_clean":"2023-11-09T03:47:19.915800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:47:19.915800+0000","last_undegraded":"2023-11-09T03:47:19.915800+0000","last_fullsized":"2023-11-09T03:47:19.915800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940244+0000","last_change":"2023-11-09T03:47:19.942483+0000","last_active":"2023-11-09T03:47:21.940244+0000","last_peered":"2023-11-09T03:47:21.940244+0000","last_clean":"2023-11-09T03:47:21.940244+0000","last_became_active":"2023-11-09T03:47:19.942157+0000","last_became_peered":"2023-11-09T03:47:19.942157+0000","last_unstale":"2023-11-09T03:47:21.940244+0000","last_undegraded":"2023-11-09T03:47:21.940244+0000","last_fullsized":"2023-11-09T03:47:21.940244+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:59:28.149940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.937687+0000","last_change":"2023-11-09T03:47:02.582739+0000","last_active":"2023-11-09T03:47:20.937687+0000","last_peered":"2023-11-09T03:47:20.937687+0000","last_clean":"2023-11-09T03:47:20.937687+0000","last_became_active":"2023-11-09T03:47:01.592743+0000","last_became_peered":"2023-11-09T03:47:01.592743+0000","last_unstale":"2023-11-09T03:47:20.937687+0000","last_undegraded":"2023-11-09T03:47:20.937687+0000","last_fullsized":"2023-11-09T03:47:20.937687+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051802700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939444+0000","last_change":"2023-11-09T03:47:20.939366+0000","last_active":"2023-11-09T03:47:21.939444+0000","last_peered":"2023-11-09T03:47:21.939444+0000","last_clean":"2023-11-09T03:47:21.939444+0000","last_became_active":"2023-11-09T03:47:20.938258+0000","last_became_peered":"2023-11-09T03:47:20.938258+0000","last_unstale":"2023-11-09T03:47:21.939444+0000","last_undegraded":"2023-11-09T03:47:21.939444+0000","last_fullsized":"2023-11-09T03:47:21.939444+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:48:27.582736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"53'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248057+0000","last_change":"2023-11-09T03:47:19.946306+0000","last_active":"2023-11-09T03:47:22.248057+0000","last_peered":"2023-11-09T03:47:22.248057+0000","last_clean":"2023-11-09T03:47:22.248057+0000","last_became_active":"2023-11-09T03:47:19.946062+0000","last_became_peered":"2023-11-09T03:47:19.946062+0000","last_unstale":"2023-11-09T03:47:22.248057+0000","last_undegraded":"2023-11-09T03:47:22.248057+0000","last_fullsized":"2023-11-09T03:47:22.248057+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:51:52.875174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.935441+0000","last_change":"2023-11-09T03:47:02.724953+0000","last_active":"2023-11-09T03:47:20.935441+0000","last_peered":"2023-11-09T03:47:20.935441+0000","last_clean":"2023-11-09T03:47:20.935441+0000","last_became_active":"2023-11-09T03:47:01.587133+0000","last_became_peered":"2023-11-09T03:47:01.587133+0000","last_unstale":"2023-11-09T03:47:20.935441+0000","last_undegraded":"2023-11-09T03:47:20.935441+0000","last_fullsized":"2023-11-09T03:47:20.935441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:23:48.521161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066497599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249034+0000","last_change":"2023-11-09T03:47:20.942122+0000","last_active":"2023-11-09T03:47:22.249034+0000","last_peered":"2023-11-09T03:47:22.249034+0000","last_clean":"2023-11-09T03:47:22.249034+0000","last_became_active":"2023-11-09T03:47:20.941870+0000","last_became_peered":"2023-11-09T03:47:20.941870+0000","last_unstale":"2023-11-09T03:47:22.249034+0000","last_undegraded":"2023-11-09T03:47:22.249034+0000","last_fullsized":"2023-11-09T03:47:22.249034+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:57.187676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"53'4","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249334+0000","last_change":"2023-11-09T03:47:19.945714+0000","last_active":"2023-11-09T03:47:22.249334+0000","last_peered":"2023-11-09T03:47:22.249334+0000","last_clean":"2023-11-09T03:47:22.249334+0000","last_became_active":"2023-11-09T03:47:19.945455+0000","last_became_peered":"2023-11-09T03:47:19.945455+0000","last_unstale":"2023-11-09T03:47:22.249334+0000","last_undegraded":"2023-11-09T03:47:22.249334+0000","last_fullsized":"2023-11-09T03:47:22.249334+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:38:08.672664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.935591+0000","last_change":"2023-11-09T03:47:03.146217+0000","last_active":"2023-11-09T03:47:20.935591+0000","last_peered":"2023-11-09T03:47:20.935591+0000","last_clean":"2023-11-09T03:47:20.935591+0000","last_became_active":"2023-11-09T03:47:01.587598+0000","last_became_peered":"2023-11-09T03:47:01.587598+0000","last_unstale":"2023-11-09T03:47:20.935591+0000","last_undegraded":"2023-11-09T03:47:20.935591+0000","last_fullsized":"2023-11-09T03:47:20.935591+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:15:22.723455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090126400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249094+0000","last_change":"2023-11-09T03:47:20.950285+0000","last_active":"2023-11-09T03:47:22.249094+0000","last_peered":"2023-11-09T03:47:22.249094+0000","last_clean":"2023-11-09T03:47:22.249094+0000","last_became_active":"2023-11-09T03:47:20.949722+0000","last_became_peered":"2023-11-09T03:47:20.949722+0000","last_unstale":"2023-11-09T03:47:22.249094+0000","last_undegraded":"2023-11-09T03:47:22.249094+0000","last_fullsized":"2023-11-09T03:47:22.249094+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:51:53.879857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940663+0000","last_change":"2023-11-09T03:47:19.947977+0000","last_active":"2023-11-09T03:47:21.940663+0000","last_peered":"2023-11-09T03:47:21.940663+0000","last_clean":"2023-11-09T03:47:21.940663+0000","last_became_active":"2023-11-09T03:47:19.947549+0000","last_became_peered":"2023-11-09T03:47:19.947549+0000","last_unstale":"2023-11-09T03:47:21.940663+0000","last_undegraded":"2023-11-09T03:47:21.940663+0000","last_fullsized":"2023-11-09T03:47:21.940663+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:51.678872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931267+0000","last_change":"2023-11-09T03:47:03.148283+0000","last_active":"2023-11-09T03:47:20.931267+0000","last_peered":"2023-11-09T03:47:20.931267+0000","last_clean":"2023-11-09T03:47:20.931267+0000","last_became_active":"2023-11-09T03:47:01.588841+0000","last_became_peered":"2023-11-09T03:47:01.588841+0000","last_unstale":"2023-11-09T03:47:20.931267+0000","last_undegraded":"2023-11-09T03:47:20.931267+0000","last_fullsized":"2023-11-09T03:47:20.931267+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:47:44.880741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000436081,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249189+0000","last_change":"2023-11-09T03:47:20.937043+0000","last_active":"2023-11-09T03:47:22.249189+0000","last_peered":"2023-11-09T03:47:22.249189+0000","last_clean":"2023-11-09T03:47:22.249189+0000","last_became_active":"2023-11-09T03:47:20.936765+0000","last_became_peered":"2023-11-09T03:47:20.936765+0000","last_unstale":"2023-11-09T03:47:22.249189+0000","last_undegraded":"2023-11-09T03:47:22.249189+0000","last_fullsized":"2023-11-09T03:47:22.249189+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:14:00.724882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249189+0000","last_change":"2023-11-09T03:47:19.946355+0000","last_active":"2023-11-09T03:47:22.249189+0000","last_peered":"2023-11-09T03:47:22.249189+0000","last_clean":"2023-11-09T03:47:22.249189+0000","last_became_active":"2023-11-09T03:47:19.945071+0000","last_became_peered":"2023-11-09T03:47:19.945071+0000","last_unstale":"2023-11-09T03:47:22.249189+0000","last_undegraded":"2023-11-09T03:47:22.249189+0000","last_fullsized":"2023-11-09T03:47:22.249189+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:07:31.918577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":21,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-09T03:47:15.977070+0000","last_change":"2023-11-09T03:47:02.618856+0000","last_active":"2023-11-09T03:47:15.977070+0000","last_peered":"2023-11-09T03:47:15.977070+0000","last_clean":"2023-11-09T03:47:15.977070+0000","last_became_active":"2023-11-09T03:47:01.590430+0000","last_became_peered":"2023-11-09T03:47:01.590430+0000","last_unstale":"2023-11-09T03:47:15.977070+0000","last_undegraded":"2023-11-09T03:47:15.977070+0000","last_fullsized":"2023-11-09T03:47:15.977070+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:57:18.470305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055030200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026961+0000","last_change":"2023-11-09T03:47:20.948028+0000","last_active":"2023-11-09T03:47:22.026961+0000","last_peered":"2023-11-09T03:47:22.026961+0000","last_clean":"2023-11-09T03:47:22.026961+0000","last_became_active":"2023-11-09T03:47:20.947752+0000","last_became_peered":"2023-11-09T03:47:20.947752+0000","last_unstale":"2023-11-09T03:47:22.026961+0000","last_undegraded":"2023-11-09T03:47:22.026961+0000","last_fullsized":"2023-11-09T03:47:22.026961+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:51:53.879857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248095+0000","last_change":"2023-11-09T03:47:19.948154+0000","last_active":"2023-11-09T03:47:22.248095+0000","last_peered":"2023-11-09T03:47:22.248095+0000","last_clean":"2023-11-09T03:47:22.248095+0000","last_became_active":"2023-11-09T03:47:19.947812+0000","last_became_peered":"2023-11-09T03:47:19.947812+0000","last_unstale":"2023-11-09T03:47:22.248095+0000","last_undegraded":"2023-11-09T03:47:22.248095+0000","last_fullsized":"2023-11-09T03:47:22.248095+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:28:13.618166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":21,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-09T03:47:15.977099+0000","last_change":"2023-11-09T03:47:02.619144+0000","last_active":"2023-11-09T03:47:15.977099+0000","last_peered":"2023-11-09T03:47:15.977099+0000","last_clean":"2023-11-09T03:47:15.977099+0000","last_became_active":"2023-11-09T03:47:01.590672+0000","last_became_peered":"2023-11-09T03:47:01.590672+0000","last_unstale":"2023-11-09T03:47:15.977099+0000","last_undegraded":"2023-11-09T03:47:15.977099+0000","last_fullsized":"2023-11-09T03:47:15.977099+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:03.025119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060997500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248188+0000","last_change":"2023-11-09T03:47:20.950877+0000","last_active":"2023-11-09T03:47:22.248188+0000","last_peered":"2023-11-09T03:47:22.248188+0000","last_clean":"2023-11-09T03:47:22.248188+0000","last_became_active":"2023-11-09T03:47:20.950616+0000","last_became_peered":"2023-11-09T03:47:20.950616+0000","last_unstale":"2023-11-09T03:47:22.248188+0000","last_undegraded":"2023-11-09T03:47:22.248188+0000","last_fullsized":"2023-11-09T03:47:22.248188+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:30.970511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939086+0000","last_change":"2023-11-09T03:47:19.940824+0000","last_active":"2023-11-09T03:47:21.939086+0000","last_peered":"2023-11-09T03:47:21.939086+0000","last_clean":"2023-11-09T03:47:21.939086+0000","last_became_active":"2023-11-09T03:47:19.940553+0000","last_became_peered":"2023-11-09T03:47:19.940553+0000","last_unstale":"2023-11-09T03:47:21.939086+0000","last_undegraded":"2023-11-09T03:47:21.939086+0000","last_fullsized":"2023-11-09T03:47:21.939086+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:52:58.422763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":21,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-09T03:47:15.976969+0000","last_change":"2023-11-09T03:47:02.619079+0000","last_active":"2023-11-09T03:47:15.976969+0000","last_peered":"2023-11-09T03:47:15.976969+0000","last_clean":"2023-11-09T03:47:15.976969+0000","last_became_active":"2023-11-09T03:47:01.591421+0000","last_became_peered":"2023-11-09T03:47:01.591421+0000","last_unstale":"2023-11-09T03:47:15.976969+0000","last_undegraded":"2023-11-09T03:47:15.976969+0000","last_fullsized":"2023-11-09T03:47:15.976969+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:18:58.230606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068736399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248346+0000","last_change":"2023-11-09T03:47:20.943642+0000","last_active":"2023-11-09T03:47:22.248346+0000","last_peered":"2023-11-09T03:47:22.248346+0000","last_clean":"2023-11-09T03:47:22.248346+0000","last_became_active":"2023-11-09T03:47:20.943101+0000","last_became_peered":"2023-11-09T03:47:20.943101+0000","last_unstale":"2023-11-09T03:47:22.248346+0000","last_undegraded":"2023-11-09T03:47:22.248346+0000","last_fullsized":"2023-11-09T03:47:22.248346+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:06:11.677208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248583+0000","last_change":"2023-11-09T03:47:19.943530+0000","last_active":"2023-11-09T03:47:22.248583+0000","last_peered":"2023-11-09T03:47:22.248583+0000","last_clean":"2023-11-09T03:47:22.248583+0000","last_became_active":"2023-11-09T03:47:19.943303+0000","last_became_peered":"2023-11-09T03:47:19.943303+0000","last_unstale":"2023-11-09T03:47:22.248583+0000","last_undegraded":"2023-11-09T03:47:22.248583+0000","last_fullsized":"2023-11-09T03:47:22.248583+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:35.294223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931280+0000","last_change":"2023-11-09T03:47:03.148529+0000","last_active":"2023-11-09T03:47:20.931280+0000","last_peered":"2023-11-09T03:47:20.931280+0000","last_clean":"2023-11-09T03:47:20.931280+0000","last_became_active":"2023-11-09T03:47:01.584148+0000","last_became_peered":"2023-11-09T03:47:01.584148+0000","last_unstale":"2023-11-09T03:47:20.931280+0000","last_undegraded":"2023-11-09T03:47:20.931280+0000","last_fullsized":"2023-11-09T03:47:20.931280+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:40:00.237248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061475600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.938959+0000","last_change":"2023-11-09T03:47:20.942889+0000","last_active":"2023-11-09T03:47:21.938959+0000","last_peered":"2023-11-09T03:47:21.938959+0000","last_clean":"2023-11-09T03:47:21.938959+0000","last_became_active":"2023-11-09T03:47:20.941326+0000","last_became_peered":"2023-11-09T03:47:20.941326+0000","last_unstale":"2023-11-09T03:47:21.938959+0000","last_undegraded":"2023-11-09T03:47:21.938959+0000","last_fullsized":"2023-11-09T03:47:21.938959+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:56:23.552030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.938908+0000","last_change":"2023-11-09T03:47:19.942969+0000","last_active":"2023-11-09T03:47:21.938908+0000","last_peered":"2023-11-09T03:47:21.938908+0000","last_clean":"2023-11-09T03:47:21.938908+0000","last_became_active":"2023-11-09T03:47:19.942726+0000","last_became_peered":"2023-11-09T03:47:19.942726+0000","last_unstale":"2023-11-09T03:47:21.938908+0000","last_undegraded":"2023-11-09T03:47:21.938908+0000","last_fullsized":"2023-11-09T03:47:21.938908+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:51:50.362425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.933718+0000","last_change":"2023-11-09T03:47:03.148217+0000","last_active":"2023-11-09T03:47:20.933718+0000","last_peered":"2023-11-09T03:47:20.933718+0000","last_clean":"2023-11-09T03:47:20.933718+0000","last_became_active":"2023-11-09T03:47:01.593424+0000","last_became_peered":"2023-11-09T03:47:01.593424+0000","last_unstale":"2023-11-09T03:47:20.933718+0000","last_undegraded":"2023-11-09T03:47:20.933718+0000","last_fullsized":"2023-11-09T03:47:20.933718+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:19:32.693691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001769865,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.941149+0000","last_change":"2023-11-09T03:47:20.942781+0000","last_active":"2023-11-09T03:47:21.941149+0000","last_peered":"2023-11-09T03:47:21.941149+0000","last_clean":"2023-11-09T03:47:21.941149+0000","last_became_active":"2023-11-09T03:47:20.942498+0000","last_became_peered":"2023-11-09T03:47:20.942498+0000","last_unstale":"2023-11-09T03:47:21.941149+0000","last_undegraded":"2023-11-09T03:47:21.941149+0000","last_fullsized":"2023-11-09T03:47:21.941149+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:23:50.235725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249315+0000","last_change":"2023-11-09T03:47:19.944843+0000","last_active":"2023-11-09T03:47:22.249315+0000","last_peered":"2023-11-09T03:47:22.249315+0000","last_clean":"2023-11-09T03:47:22.249315+0000","last_became_active":"2023-11-09T03:47:19.944583+0000","last_became_peered":"2023-11-09T03:47:19.944583+0000","last_unstale":"2023-11-09T03:47:22.249315+0000","last_undegraded":"2023-11-09T03:47:22.249315+0000","last_fullsized":"2023-11-09T03:47:22.249315+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:46:53.255789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931015+0000","last_change":"2023-11-09T03:47:03.146470+0000","last_active":"2023-11-09T03:47:20.931015+0000","last_peered":"2023-11-09T03:47:20.931015+0000","last_clean":"2023-11-09T03:47:20.931015+0000","last_became_active":"2023-11-09T03:47:01.587304+0000","last_became_peered":"2023-11-09T03:47:01.587304+0000","last_unstale":"2023-11-09T03:47:20.931015+0000","last_undegraded":"2023-11-09T03:47:20.931015+0000","last_fullsized":"2023-11-09T03:47:20.931015+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:58:23.339287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00110395,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.941057+0000","last_change":"2023-11-09T03:47:20.941612+0000","last_active":"2023-11-09T03:47:21.941057+0000","last_peered":"2023-11-09T03:47:21.941057+0000","last_clean":"2023-11-09T03:47:21.941057+0000","last_became_active":"2023-11-09T03:47:20.941335+0000","last_became_peered":"2023-11-09T03:47:20.941335+0000","last_unstale":"2023-11-09T03:47:21.941057+0000","last_undegraded":"2023-11-09T03:47:21.941057+0000","last_fullsized":"2023-11-09T03:47:21.941057+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:05:32.662676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939435+0000","last_change":"2023-11-09T03:47:19.941173+0000","last_active":"2023-11-09T03:47:21.939435+0000","last_peered":"2023-11-09T03:47:21.939435+0000","last_clean":"2023-11-09T03:47:21.939435+0000","last_became_active":"2023-11-09T03:47:19.940928+0000","last_became_peered":"2023-11-09T03:47:19.940928+0000","last_unstale":"2023-11-09T03:47:21.939435+0000","last_undegraded":"2023-11-09T03:47:21.939435+0000","last_fullsized":"2023-11-09T03:47:21.939435+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:28:13.618166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.934203+0000","last_change":"2023-11-09T03:47:03.147367+0000","last_active":"2023-11-09T03:47:20.934203+0000","last_peered":"2023-11-09T03:47:20.934203+0000","last_clean":"2023-11-09T03:47:20.934203+0000","last_became_active":"2023-11-09T03:47:01.588359+0000","last_became_peered":"2023-11-09T03:47:01.588359+0000","last_unstale":"2023-11-09T03:47:20.934203+0000","last_undegraded":"2023-11-09T03:47:20.934203+0000","last_fullsized":"2023-11-09T03:47:20.934203+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:21:34.303068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014106660000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249223+0000","last_change":"2023-11-09T03:47:20.950444+0000","last_active":"2023-11-09T03:47:22.249223+0000","last_peered":"2023-11-09T03:47:22.249223+0000","last_clean":"2023-11-09T03:47:22.249223+0000","last_became_active":"2023-11-09T03:47:20.950038+0000","last_became_peered":"2023-11-09T03:47:20.950038+0000","last_unstale":"2023-11-09T03:47:22.249223+0000","last_undegraded":"2023-11-09T03:47:22.249223+0000","last_fullsized":"2023-11-09T03:47:22.249223+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:55:28.077165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940688+0000","last_change":"2023-11-09T03:47:19.943935+0000","last_active":"2023-11-09T03:47:21.940688+0000","last_peered":"2023-11-09T03:47:21.940688+0000","last_clean":"2023-11-09T03:47:21.940688+0000","last_became_active":"2023-11-09T03:47:19.943125+0000","last_became_peered":"2023-11-09T03:47:19.943125+0000","last_unstale":"2023-11-09T03:47:21.940688+0000","last_undegraded":"2023-11-09T03:47:21.940688+0000","last_fullsized":"2023-11-09T03:47:21.940688+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:06:16.671366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":27,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.928179+0000","last_change":"2023-11-09T03:47:02.581682+0000","last_active":"2023-11-09T03:47:20.928179+0000","last_peered":"2023-11-09T03:47:20.928179+0000","last_clean":"2023-11-09T03:47:20.928179+0000","last_became_active":"2023-11-09T03:47:01.593051+0000","last_became_peered":"2023-11-09T03:47:01.593051+0000","last_unstale":"2023-11-09T03:47:20.928179+0000","last_undegraded":"2023-11-09T03:47:20.928179+0000","last_fullsized":"2023-11-09T03:47:20.928179+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047277799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939213+0000","last_change":"2023-11-09T03:47:20.945402+0000","last_active":"2023-11-09T03:47:21.939213+0000","last_peered":"2023-11-09T03:47:21.939213+0000","last_clean":"2023-11-09T03:47:21.939213+0000","last_became_active":"2023-11-09T03:47:20.945145+0000","last_became_peered":"2023-11-09T03:47:20.945145+0000","last_unstale":"2023-11-09T03:47:21.939213+0000","last_undegraded":"2023-11-09T03:47:21.939213+0000","last_fullsized":"2023-11-09T03:47:21.939213+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:38:09.677347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-09T03:47:18.911117+0000","last_change":"2023-11-09T03:47:18.911117+0000","last_active":"2023-11-09T03:47:18.911117+0000","last_peered":"2023-11-09T03:47:18.911117+0000","last_clean":"2023-11-09T03:47:18.911117+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:47:18.911117+0000","last_undegraded":"2023-11-09T03:47:18.911117+0000","last_fullsized":"2023-11-09T03:47:18.911117+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.933457+0000","last_change":"2023-11-09T03:47:03.146309+0000","last_active":"2023-11-09T03:47:20.933457+0000","last_peered":"2023-11-09T03:47:20.933457+0000","last_clean":"2023-11-09T03:47:20.933457+0000","last_became_active":"2023-11-09T03:47:01.587560+0000","last_became_peered":"2023-11-09T03:47:01.587560+0000","last_unstale":"2023-11-09T03:47:20.933457+0000","last_undegraded":"2023-11-09T03:47:20.933457+0000","last_fullsized":"2023-11-09T03:47:20.933457+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:27:55.271945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070988400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249299+0000","last_change":"2023-11-09T03:47:20.941432+0000","last_active":"2023-11-09T03:47:22.249299+0000","last_peered":"2023-11-09T03:47:22.249299+0000","last_clean":"2023-11-09T03:47:22.249299+0000","last_became_active":"2023-11-09T03:47:20.941186+0000","last_became_peered":"2023-11-09T03:47:20.941186+0000","last_unstale":"2023-11-09T03:47:22.249299+0000","last_undegraded":"2023-11-09T03:47:22.249299+0000","last_fullsized":"2023-11-09T03:47:22.249299+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:25:38.423194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939102+0000","last_change":"2023-11-09T03:47:19.947334+0000","last_active":"2023-11-09T03:47:21.939102+0000","last_peered":"2023-11-09T03:47:21.939102+0000","last_clean":"2023-11-09T03:47:21.939102+0000","last_became_active":"2023-11-09T03:47:19.947091+0000","last_became_peered":"2023-11-09T03:47:19.947091+0000","last_unstale":"2023-11-09T03:47:21.939102+0000","last_undegraded":"2023-11-09T03:47:21.939102+0000","last_fullsized":"2023-11-09T03:47:21.939102+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:24:06.867382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.939549+0000","last_change":"2023-11-09T03:47:02.725125+0000","last_active":"2023-11-09T03:47:20.939549+0000","last_peered":"2023-11-09T03:47:20.939549+0000","last_clean":"2023-11-09T03:47:20.939549+0000","last_became_active":"2023-11-09T03:47:01.591521+0000","last_became_peered":"2023-11-09T03:47:01.591521+0000","last_unstale":"2023-11-09T03:47:20.939549+0000","last_undegraded":"2023-11-09T03:47:20.939549+0000","last_fullsized":"2023-11-09T03:47:20.939549+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:52:40.076542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000758675,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940341+0000","last_change":"2023-11-09T03:47:20.938307+0000","last_active":"2023-11-09T03:47:21.940341+0000","last_peered":"2023-11-09T03:47:21.940341+0000","last_clean":"2023-11-09T03:47:21.940341+0000","last_became_active":"2023-11-09T03:47:20.937853+0000","last_became_peered":"2023-11-09T03:47:20.937853+0000","last_unstale":"2023-11-09T03:47:21.940341+0000","last_undegraded":"2023-11-09T03:47:21.940341+0000","last_fullsized":"2023-11-09T03:47:21.940341+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:57.187676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248805+0000","last_change":"2023-11-09T03:47:19.945739+0000","last_active":"2023-11-09T03:47:22.248805+0000","last_peered":"2023-11-09T03:47:22.248805+0000","last_clean":"2023-11-09T03:47:22.248805+0000","last_became_active":"2023-11-09T03:47:19.944990+0000","last_became_peered":"2023-11-09T03:47:19.944990+0000","last_unstale":"2023-11-09T03:47:22.248805+0000","last_undegraded":"2023-11-09T03:47:22.248805+0000","last_fullsized":"2023-11-09T03:47:22.248805+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:53:27.482165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.932096+0000","last_change":"2023-11-09T03:47:03.147289+0000","last_active":"2023-11-09T03:47:20.932096+0000","last_peered":"2023-11-09T03:47:20.932096+0000","last_clean":"2023-11-09T03:47:20.932096+0000","last_became_active":"2023-11-09T03:47:01.587784+0000","last_became_peered":"2023-11-09T03:47:01.587784+0000","last_unstale":"2023-11-09T03:47:20.932096+0000","last_undegraded":"2023-11-09T03:47:20.932096+0000","last_fullsized":"2023-11-09T03:47:20.932096+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:11:23.824112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038781299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247714+0000","last_change":"2023-11-09T03:47:20.937750+0000","last_active":"2023-11-09T03:47:22.247714+0000","last_peered":"2023-11-09T03:47:22.247714+0000","last_clean":"2023-11-09T03:47:22.247714+0000","last_became_active":"2023-11-09T03:47:20.937473+0000","last_became_peered":"2023-11-09T03:47:20.937473+0000","last_unstale":"2023-11-09T03:47:22.247714+0000","last_undegraded":"2023-11-09T03:47:22.247714+0000","last_fullsized":"2023-11-09T03:47:22.247714+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:06:17.676049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940071+0000","last_change":"2023-11-09T03:47:19.948440+0000","last_active":"2023-11-09T03:47:21.940071+0000","last_peered":"2023-11-09T03:47:21.940071+0000","last_clean":"2023-11-09T03:47:21.940071+0000","last_became_active":"2023-11-09T03:47:19.948199+0000","last_became_peered":"2023-11-09T03:47:19.948199+0000","last_unstale":"2023-11-09T03:47:21.940071+0000","last_undegraded":"2023-11-09T03:47:21.940071+0000","last_fullsized":"2023-11-09T03:47:21.940071+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:34:52.006539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":21,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-09T03:47:15.976947+0000","last_change":"2023-11-09T03:47:02.619346+0000","last_active":"2023-11-09T03:47:15.976947+0000","last_peered":"2023-11-09T03:47:15.976947+0000","last_clean":"2023-11-09T03:47:15.976947+0000","last_became_active":"2023-11-09T03:47:01.590389+0000","last_became_peered":"2023-11-09T03:47:01.590389+0000","last_unstale":"2023-11-09T03:47:15.976947+0000","last_undegraded":"2023-11-09T03:47:15.976947+0000","last_fullsized":"2023-11-09T03:47:15.976947+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000384072,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.941022+0000","last_change":"2023-11-09T03:47:20.937215+0000","last_active":"2023-11-09T03:47:21.941022+0000","last_peered":"2023-11-09T03:47:21.941022+0000","last_clean":"2023-11-09T03:47:21.941022+0000","last_became_active":"2023-11-09T03:47:20.936913+0000","last_became_peered":"2023-11-09T03:47:20.936913+0000","last_unstale":"2023-11-09T03:47:21.941022+0000","last_undegraded":"2023-11-09T03:47:21.941022+0000","last_fullsized":"2023-11-09T03:47:21.941022+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:25:38.423194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249234+0000","last_change":"2023-11-09T03:47:19.948214+0000","last_active":"2023-11-09T03:47:22.249234+0000","last_peered":"2023-11-09T03:47:22.249234+0000","last_clean":"2023-11-09T03:47:22.249234+0000","last_became_active":"2023-11-09T03:47:19.947966+0000","last_became_peered":"2023-11-09T03:47:19.947966+0000","last_unstale":"2023-11-09T03:47:22.249234+0000","last_undegraded":"2023-11-09T03:47:22.249234+0000","last_fullsized":"2023-11-09T03:47:22.249234+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:53:19.322920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.938897+0000","last_change":"2023-11-09T03:47:02.582393+0000","last_active":"2023-11-09T03:47:20.938897+0000","last_peered":"2023-11-09T03:47:20.938897+0000","last_clean":"2023-11-09T03:47:20.938897+0000","last_became_active":"2023-11-09T03:47:01.591752+0000","last_became_peered":"2023-11-09T03:47:01.591752+0000","last_unstale":"2023-11-09T03:47:20.938897+0000","last_undegraded":"2023-11-09T03:47:20.938897+0000","last_fullsized":"2023-11-09T03:47:20.938897+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:03.025119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067021599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.027114+0000","last_change":"2023-11-09T03:47:20.946342+0000","last_active":"2023-11-09T03:47:22.027114+0000","last_peered":"2023-11-09T03:47:22.027114+0000","last_clean":"2023-11-09T03:47:22.027114+0000","last_became_active":"2023-11-09T03:47:20.946014+0000","last_became_peered":"2023-11-09T03:47:20.946014+0000","last_unstale":"2023-11-09T03:47:22.027114+0000","last_undegraded":"2023-11-09T03:47:22.027114+0000","last_fullsized":"2023-11-09T03:47:22.027114+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:25:38.423194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"53'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248629+0000","last_change":"2023-11-09T03:47:19.944449+0000","last_active":"2023-11-09T03:47:22.248629+0000","last_peered":"2023-11-09T03:47:22.248629+0000","last_clean":"2023-11-09T03:47:22.248629+0000","last_became_active":"2023-11-09T03:47:19.944203+0000","last_became_peered":"2023-11-09T03:47:19.944203+0000","last_unstale":"2023-11-09T03:47:22.248629+0000","last_undegraded":"2023-11-09T03:47:22.248629+0000","last_fullsized":"2023-11-09T03:47:22.248629+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:51.678872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.929927+0000","last_change":"2023-11-09T03:47:03.028744+0000","last_active":"2023-11-09T03:47:20.929927+0000","last_peered":"2023-11-09T03:47:20.929927+0000","last_clean":"2023-11-09T03:47:20.929927+0000","last_became_active":"2023-11-09T03:47:01.593890+0000","last_became_peered":"2023-11-09T03:47:01.593890+0000","last_unstale":"2023-11-09T03:47:20.929927+0000","last_undegraded":"2023-11-09T03:47:20.929927+0000","last_fullsized":"2023-11-09T03:47:20.929927+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:05.754357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053008299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026854+0000","last_change":"2023-11-09T03:47:20.952365+0000","last_active":"2023-11-09T03:47:22.026854+0000","last_peered":"2023-11-09T03:47:22.026854+0000","last_clean":"2023-11-09T03:47:22.026854+0000","last_became_active":"2023-11-09T03:47:20.952080+0000","last_became_peered":"2023-11-09T03:47:20.952080+0000","last_unstale":"2023-11-09T03:47:22.026854+0000","last_undegraded":"2023-11-09T03:47:22.026854+0000","last_fullsized":"2023-11-09T03:47:22.026854+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:40.639821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249192+0000","last_change":"2023-11-09T03:47:19.946712+0000","last_active":"2023-11-09T03:47:22.249192+0000","last_peered":"2023-11-09T03:47:22.249192+0000","last_clean":"2023-11-09T03:47:22.249192+0000","last_became_active":"2023-11-09T03:47:19.946456+0000","last_became_peered":"2023-11-09T03:47:19.946456+0000","last_unstale":"2023-11-09T03:47:22.249192+0000","last_undegraded":"2023-11-09T03:47:22.249192+0000","last_fullsized":"2023-11-09T03:47:22.249192+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:39:07.929390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.936277+0000","last_change":"2023-11-09T03:47:03.146783+0000","last_active":"2023-11-09T03:47:20.936277+0000","last_peered":"2023-11-09T03:47:20.936277+0000","last_clean":"2023-11-09T03:47:20.936277+0000","last_became_active":"2023-11-09T03:47:01.586923+0000","last_became_peered":"2023-11-09T03:47:01.586923+0000","last_unstale":"2023-11-09T03:47:20.936277+0000","last_undegraded":"2023-11-09T03:47:20.936277+0000","last_fullsized":"2023-11-09T03:47:20.936277+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:37:50.326442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060384700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248492+0000","last_change":"2023-11-09T03:47:20.946978+0000","last_active":"2023-11-09T03:47:22.248492+0000","last_peered":"2023-11-09T03:47:22.248492+0000","last_clean":"2023-11-09T03:47:22.248492+0000","last_became_active":"2023-11-09T03:47:20.946667+0000","last_became_peered":"2023-11-09T03:47:20.946667+0000","last_unstale":"2023-11-09T03:47:22.248492+0000","last_undegraded":"2023-11-09T03:47:22.248492+0000","last_fullsized":"2023-11-09T03:47:22.248492+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:02:01.425453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"53'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940910+0000","last_change":"2023-11-09T03:47:19.945244+0000","last_active":"2023-11-09T03:47:21.940910+0000","last_peered":"2023-11-09T03:47:21.940910+0000","last_clean":"2023-11-09T03:47:21.940910+0000","last_became_active":"2023-11-09T03:47:19.944955+0000","last_became_peered":"2023-11-09T03:47:19.944955+0000","last_unstale":"2023-11-09T03:47:21.940910+0000","last_undegraded":"2023-11-09T03:47:21.940910+0000","last_fullsized":"2023-11-09T03:47:21.940910+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:12:47.685713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.933539+0000","last_change":"2023-11-09T03:47:03.146580+0000","last_active":"2023-11-09T03:47:20.933539+0000","last_peered":"2023-11-09T03:47:20.933539+0000","last_clean":"2023-11-09T03:47:20.933539+0000","last_became_active":"2023-11-09T03:47:01.589201+0000","last_became_peered":"2023-11-09T03:47:01.589201+0000","last_unstale":"2023-11-09T03:47:20.933539+0000","last_undegraded":"2023-11-09T03:47:20.933539+0000","last_fullsized":"2023-11-09T03:47:20.933539+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:14.300421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049706999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248489+0000","last_change":"2023-11-09T03:47:20.949430+0000","last_active":"2023-11-09T03:47:22.248489+0000","last_peered":"2023-11-09T03:47:22.248489+0000","last_clean":"2023-11-09T03:47:22.248489+0000","last_became_active":"2023-11-09T03:47:20.949177+0000","last_became_peered":"2023-11-09T03:47:20.949177+0000","last_unstale":"2023-11-09T03:47:22.248489+0000","last_undegraded":"2023-11-09T03:47:22.248489+0000","last_fullsized":"2023-11-09T03:47:22.248489+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:04:05.315163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-09T03:47:18.911117+0000","last_change":"2023-11-09T03:47:18.911117+0000","last_active":"2023-11-09T03:47:18.911117+0000","last_peered":"2023-11-09T03:47:18.911117+0000","last_clean":"2023-11-09T03:47:18.911117+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:47:18.911117+0000","last_undegraded":"2023-11-09T03:47:18.911117+0000","last_fullsized":"2023-11-09T03:47:18.911117+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931584+0000","last_change":"2023-11-09T03:47:03.147851+0000","last_active":"2023-11-09T03:47:20.931584+0000","last_peered":"2023-11-09T03:47:20.931584+0000","last_clean":"2023-11-09T03:47:20.931584+0000","last_became_active":"2023-11-09T03:47:01.589893+0000","last_became_peered":"2023-11-09T03:47:01.589893+0000","last_unstale":"2023-11-09T03:47:20.931584+0000","last_undegraded":"2023-11-09T03:47:20.931584+0000","last_fullsized":"2023-11-09T03:47:20.931584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:08:17.183462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029151399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":194,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026619+0000","last_change":"2023-11-09T03:45:21.030926+0000","last_active":"2023-11-09T03:47:22.026619+0000","last_peered":"2023-11-09T03:47:22.026619+0000","last_clean":"2023-11-09T03:47:22.026619+0000","last_became_active":"2023-11-09T03:45:21.029801+0000","last_became_peered":"2023-11-09T03:45:21.029801+0000","last_unstale":"2023-11-09T03:47:22.026619+0000","last_undegraded":"2023-11-09T03:47:22.026619+0000","last_fullsized":"2023-11-09T03:47:22.026619+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_clean_scrub_stamp":"2023-11-09T03:45:19.768433+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:12:59.153713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":122,"acting":122,"num_store_stats":7},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":58,"acting":58,"num_store_stats":7},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28240,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343600,"statfs":{"total":96636764160,"available":96607846400,"internally_reserved":0,"allocated":1245184,"data_stored":881539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.501,"15min":0.498},"min":{"1min":0.34899999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.622},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.41199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.71499999999999997}]},{"osd":1,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.33000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.38100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.73099999999999998}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.52300000000000002,"15min":0.51400000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.68700000000000006,"15min":0.70799999999999996},"min":{"1min":0.41299999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.80700000000000005,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.63500000000000001}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.48699999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.68999999999999995}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.441},"min":{"1min":0.34100000000000003,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.49199999999999999,"15min":0.47899999999999998},"min":{"1min":0.38700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.38700000000000001}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.47199999999999998,"15min":0.45600000000000002},"min":{"1min":0.37,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.51100000000000001,"15min":0.497},"min":{"1min":0.39600000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.58699999999999997}]},{"osd":6,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55100000000000005,"15min":0.54100000000000004},"min":{"1min":0.44400000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.434,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.44}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":593920,"data_stored":263160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.622}]},{"osd":1,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.504}]},{"osd":3,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.60099999999999998}]},{"osd":4,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.38},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.65600000000000003}]},{"osd":5,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.35399999999999998}]},{"osd":7,"last update":"Thu Nov 9 03:46:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.55900000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":651264,"data_stored":291663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.496,"15min":0.48599999999999999},"min":{"1min":0.38700000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.876},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.503,"15min":0.498},"min":{"1min":0.42399999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.71799999999999997,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45400000000000001}]},{"osd":1,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.439,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.69999999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.62},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.46700000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.76200000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.64100000000000001}]},{"osd":2,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.33300000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.67900000000000005,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.499,"5min":0.371,"15min":0.371},"max":{"1min":0.72799999999999998,"5min":0.745,"15min":0.745},"last":0.71799999999999997}]},{"osd":3,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.39700000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.76500000000000001,"5min":0.88,"15min":0.88},"last":0.76400000000000001}]},{"osd":4,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.78100000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.495},{"interface":"front","average":{"1min":0.504,"5min":0.48399999999999999,"15min":0.48099999999999998},"min":{"1min":0.33100000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.79000000000000004}]},{"osd":6,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.50900000000000001,"15min":0.51100000000000001},"min":{"1min":0.33400000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.74199999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51300000000000001,"15min":0.51100000000000001},"min":{"1min":0.35499999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66900000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.753}]},{"osd":7,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.38900000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.73499999999999999,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61899999999999999,"15min":0.62},"min":{"1min":0.41199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.752,"5min":1.01,"15min":1.01},"last":0.84899999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":643072,"data_stored":285650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67600000000000005,"5min":1.034,"15min":1.034},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.53600000000000003,"15min":0.54000000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68300000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.46899999999999997}]},{"osd":1,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.56899999999999995,"15min":0.57699999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68100000000000005,"5min":1.093,"15min":1.093},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.36899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72399999999999998,"5min":0.871,"15min":0.871},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76800000000000002,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70099999999999996,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.44700000000000001}]},{"osd":3,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71699999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.502},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54900000000000004,"15min":0.55200000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86599999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.433}]},{"osd":5,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54400000000000004,"15min":0.54300000000000004},"min":{"1min":0.37,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76000000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.33100000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.36899999999999999}]},{"osd":6,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.33800000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.51700000000000002}]},{"osd":7,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.62,"15min":0.63},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77700000000000002,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.95599999999999996,"5min":1.113,"15min":1.113},"last":0.54300000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":651264,"data_stored":291067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.5,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.215},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51100000000000001,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.23300000000000001}]},{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32900000000000001,"15min":0.33100000000000002},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.54200000000000004,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.19800000000000001},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.371,"15min":0.375},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.53100000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.26700000000000002}]},{"osd":2,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.27200000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.59099999999999997,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.56100000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.31900000000000001}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.70899999999999996,"5min":0.87,"15min":0.87},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60999999999999999,"15min":0.61599999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.75600000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.62,"15min":0.625},"min":{"1min":0.45700000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79900000000000004,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61699999999999999,"15min":0.623},"min":{"1min":0.42599999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.74299999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.621,"15min":0.627},"min":{"1min":0.42399999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81200000000000006,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59999999999999998,"15min":0.60399999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76600000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.66000000000000003}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77200000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63500000000000001,"15min":0.64200000000000002},"min":{"1min":0.433,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81299999999999994,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.58299999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":651264,"data_stored":291067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.46000000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.33400000000000002,"15min":0.33100000000000002},"min":{"1min":0.247,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.27600000000000002}]},{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32500000000000001,"15min":0.32600000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.495,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.249},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.35899999999999999,"15min":0.36299999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.48599999999999999,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.33200000000000002}]},{"osd":3,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.36199999999999999,"15min":0.36399999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.45700000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40200000000000002,"15min":0.40400000000000003},"min":{"1min":0.30299999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.52300000000000002,"5min":0.754,"15min":0.754},"last":0.35299999999999998}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72299999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81299999999999994,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.50900000000000001}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.39800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.75700000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.59599999999999997,"15min":0.60099999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.753,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.498}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60399999999999998,"15min":0.60899999999999999},"min":{"1min":0.44600000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.76100000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.71199999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.57699999999999996}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60699999999999998,"15min":0.61199999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.745,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.80000000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":647168,"data_stored":291101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.28699999999999998,"5min":0.31,"15min":0.314},"min":{"1min":0.20799999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.38900000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.315,"15min":0.311},"min":{"1min":0.24099999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.43099999999999999}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.311,"15min":0.309},"min":{"1min":0.253,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.378,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.34599999999999997}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36899999999999999,"15min":0.371},"min":{"1min":0.23899999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.47699999999999998,"5min":0.505,"15min":0.505},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.37,"15min":0.371},"min":{"1min":0.30299999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.45400000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.36599999999999999}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.41899999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.747,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.44400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.77600000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.62}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56200000000000006,"15min":0.55700000000000005},"min":{"1min":0.46100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.44500000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.72599999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.60699999999999998}]},{"osd":6,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.47799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56699999999999995,"15min":0.56799999999999995},"min":{"1min":0.433,"5min":0.379,"15min":0.379},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.70399999999999996}]},{"osd":7,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.42899999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.69799999999999995,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.72899999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.63400000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28228,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343612,"statfs":{"total":96636764160,"available":96607858688,"internally_reserved":0,"allocated":1236992,"data_stored":875981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.31900000000000001,"15min":0.32300000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.38200000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.29599999999999999,"5min":0.35499999999999998,"15min":0.36499999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.44500000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.28799999999999998}]},{"osd":2,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.35699999999999998,"15min":0.36399999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.44700000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.36699999999999999,"15min":0.371},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.47699999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.251}]},{"osd":3,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.38500000000000001,"15min":0.39100000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.46400000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.41499999999999998,"15min":0.42099999999999999},"min":{"1min":0.309,"5min":0.307,"15min":0.307},"max":{"1min":0.48999999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.379}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58899999999999997,"15min":0.59299999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60599999999999998,"15min":0.60899999999999999},"min":{"1min":0.436,"5min":0.318,"15min":0.318},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.621}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.36199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76800000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.629,"15min":0.63600000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78900000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.52100000000000002}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.64100000000000001,"15min":0.64700000000000002},"min":{"1min":0.42499999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81000000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.627,"15min":0.63200000000000001},"min":{"1min":0.41699999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59699999999999998}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.44400000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.42999999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.71799999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.67000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-09T03:47:24.690 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-09T03:47:24.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.824+0000 7f9aeb8e5640 1 Processor -- start 2023-11-09T03:47:24.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.824+0000 7f9aeb8e5640 1 -- start start 2023-11-09T03:47:24.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.825+0000 7f9aeb8e5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9ae415abf0 0x7f9ae40ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:24.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.825+0000 7f9aeb8e5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ae40cdd30 con 0x7f9ae40cb230 2023-11-09T03:47:24.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.825+0000 7f9aeb8e5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ae40cd120 con 0x7f9ae40b4190 2023-11-09T03:47:24.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.825+0000 7f9aeb8e5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae40ce700 con 0x7f9ae415abf0 2023-11-09T03:47:24.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.825+0000 7f9ae9681640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9ae415abf0 0x7f9ae40ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:24.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.825+0000 7f9ae9e82640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f9ae40cb230 0x7f9ae40cb660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47990/0 (socket says 172.21.15.76:47990) 2023-11-09T03:47:24.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.825+0000 7f9ae9e82640 1 -- 172.21.15.76:0/2544025557 learned_addr learned my addr 172.21.15.76:0/2544025557 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:24.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.825+0000 7f9ae9681640 1 -- 172.21.15.76:0/2544025557 >> v1:172.21.15.113:6789/0 conn(0x7f9ae40b4190 legacy=0x7f9ae40cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:24.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.825+0000 7f9ae9681640 1 -- 172.21.15.76:0/2544025557 >> v1:172.21.15.76:6789/0 conn(0x7f9ae40cb230 legacy=0x7f9ae40cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:24.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.825+0000 7f9adbfff640 1 -- 172.21.15.76:0/2544025557 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4151400556 0 0) 0x7f9ae40cdd30 con 0x7f9ae40cb230 2023-11-09T03:47:24.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.825+0000 7f9ae9681640 1 -- 172.21.15.76:0/2544025557 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae40cd400 con 0x7f9ae415abf0 2023-11-09T03:47:24.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.826+0000 7f9ae9681640 1 --2- 172.21.15.76:0/2544025557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ae415abf0 0x7f9ae40ce1c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00097a0 tx=0x7f9ae002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcff1aafac6ae4d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:24.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.826+0000 7f9adbfff640 1 -- 172.21.15.76:0/2544025557 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0030610 con 0x7f9ae415abf0 2023-11-09T03:47:24.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.826+0000 7f9adbfff640 1 -- 172.21.15.76:0/2544025557 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae0030770 con 0x7f9ae415abf0 2023-11-09T03:47:24.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.826+0000 7f9adbfff640 1 -- 172.21.15.76:0/2544025557 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0030b00 con 0x7f9ae415abf0 2023-11-09T03:47:24.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.827+0000 7f9aeb8e5640 1 -- 172.21.15.76:0/2544025557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ae415abf0 msgr2=0x7f9ae40ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:24.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.827+0000 7f9aeb8e5640 1 --2- 172.21.15.76:0/2544025557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ae415abf0 0x7f9ae40ce1c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00097a0 tx=0x7f9ae002b960 comp rx=0 tx=0).stop 2023-11-09T03:47:24.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.827+0000 7f9aeb8e5640 1 -- 172.21.15.76:0/2544025557 shutdown_connections 2023-11-09T03:47:24.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.827+0000 7f9aeb8e5640 1 --2- 172.21.15.76:0/2544025557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ae415abf0 0x7f9ae40ce1c0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:24.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.827+0000 7f9aeb8e5640 1 -- 172.21.15.76:0/2544025557 >> 172.21.15.76:0/2544025557 conn(0x7f9ae40b03c0 msgr2=0x7f9ae40b07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:24.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.827+0000 7f9aeb8e5640 1 -- 172.21.15.76:0/2544025557 shutdown_connections 2023-11-09T03:47:24.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.827+0000 7f9aeb8e5640 1 -- 172.21.15.76:0/2544025557 wait complete. 2023-11-09T03:47:24.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.828+0000 7f9aeb8e5640 1 Processor -- start 2023-11-09T03:47:24.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.828+0000 7f9aeb8e5640 1 -- start start 2023-11-09T03:47:24.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.829+0000 7f9aeb8e5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ae40b4190 0x7f9ae40c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:24.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.829+0000 7f9aeb8e5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ae40cb230 0x7f9ae414cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:24.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.829+0000 7f9aeb8e5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ae40ac9f0 0x7f9ae40aec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:24.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.829+0000 7f9aeb8e5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae406c7c0 con 0x7f9ae40cb230 2023-11-09T03:47:24.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.829+0000 7f9aeb8e5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae40acea0 con 0x7f9ae40b4190 2023-11-09T03:47:24.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.829+0000 7f9aeb8e5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae40c4a70 con 0x7f9ae40ac9f0 2023-11-09T03:47:24.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.829+0000 7f9ae8e80640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ae40cb230 0x7f9ae414cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:24.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.829+0000 7f9ae8e80640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ae40cb230 0x7f9ae414cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42556/0 (socket says 172.21.15.76:42556) 2023-11-09T03:47:24.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.829+0000 7f9ae8e80640 1 -- 172.21.15.76:0/291525072 learned_addr learned my addr 172.21.15.76:0/291525072 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:24.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.829+0000 7f9ae9681640 1 --2- 172.21.15.76:0/291525072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ae40b4190 0x7f9ae40c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:24.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.829+0000 7f9ae8e80640 1 -- 172.21.15.76:0/291525072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ae40ac9f0 msgr2=0x7f9ae40aec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:24.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.829+0000 7f9ae8e80640 1 --2- 172.21.15.76:0/291525072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ae40ac9f0 0x7f9ae40aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:24.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.829+0000 7f9ae8e80640 1 -- 172.21.15.76:0/291525072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ae40b4190 msgr2=0x7f9ae40c43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:24.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.829+0000 7f9ae8e80640 1 --2- 172.21.15.76:0/291525072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ae40b4190 0x7f9ae40c43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:24.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.829+0000 7f9ae8e80640 1 -- 172.21.15.76:0/291525072 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae0009400 con 0x7f9ae40cb230 2023-11-09T03:47:24.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.829+0000 7f9ae9681640 1 --2- 172.21.15.76:0/291525072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ae40b4190 0x7f9ae40c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:24.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.829+0000 7f9ae8e80640 1 --2- 172.21.15.76:0/291525072 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ae40cb230 0x7f9ae414cdd0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400c450 tx=0x7f9ad400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:24.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.830+0000 7f9ad9ffb640 1 -- 172.21.15.76:0/291525072 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad40052d0 con 0x7f9ae40cb230 2023-11-09T03:47:24.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.830+0000 7f9ad9ffb640 1 -- 172.21.15.76:0/291525072 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad4005450 con 0x7f9ae40cb230 2023-11-09T03:47:24.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.830+0000 7f9aeb8e5640 1 -- 172.21.15.76:0/291525072 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae40c4d50 con 0x7f9ae40cb230 2023-11-09T03:47:24.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.830+0000 7f9ad9ffb640 1 -- 172.21.15.76:0/291525072 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4003880 con 0x7f9ae40cb230 2023-11-09T03:47:24.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.830+0000 7f9aeb8e5640 1 -- 172.21.15.76:0/291525072 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae406ca10 con 0x7f9ae40cb230 2023-11-09T03:47:24.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.831+0000 7f9aeb8e5640 1 -- 172.21.15.76:0/291525072 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae406cc30 con 0x7f9ae40cb230 2023-11-09T03:47:24.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.831+0000 7f9ad9ffb640 1 -- 172.21.15.76:0/291525072 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9ad40039e0 con 0x7f9ae40cb230 2023-11-09T03:47:24.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.832+0000 7f9ad9ffb640 1 --2- 172.21.15.76:0/291525072 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9ac806ae70 0x7f9ac806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:24.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.832+0000 7f9ad9ffb640 1 -- 172.21.15.76:0/291525072 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9ad40893d0 con 0x7f9ae40cb230 2023-11-09T03:47:24.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.832+0000 7f9ae9681640 1 --2- 172.21.15.76:0/291525072 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9ac806ae70 0x7f9ac806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:24.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.833+0000 7f9ae9681640 1 --2- 172.21.15.76:0/291525072 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9ac806ae70 0x7f9ac806d330 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0036000 tx=0x7f9ae003b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:24.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.836+0000 7f9ad9ffb640 1 -- 172.21.15.76:0/291525072 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9ad4054270 con 0x7f9ae40cb230 2023-11-09T03:47:24.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:24.998+0000 7f9aeb8e5640 1 -- 172.21.15.76:0/291525072 --> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9ae40cd9c0 con 0x7f9ac806ae70 2023-11-09T03:47:25.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.007+0000 7f9ad9ffb640 1 -- 172.21.15.76:0/291525072 <== mgr.4105 v2:172.21.15.113:6832/1264193042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+370368 (secure 0 0 0) 0x7f9ae003f040 con 0x7f9ac806ae70 2023-11-09T03:47:25.010 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:25.013 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2023-11-09T03:47:25.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.011+0000 7f9aeb8e5640 1 -- 172.21.15.76:0/291525072 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9ac806ae70 msgr2=0x7f9ac806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:25.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.011+0000 7f9aeb8e5640 1 --2- 172.21.15.76:0/291525072 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9ac806ae70 0x7f9ac806d330 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0036000 tx=0x7f9ae003b040 comp rx=0 tx=0).stop 2023-11-09T03:47:25.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.011+0000 7f9aeb8e5640 1 -- 172.21.15.76:0/291525072 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ae40cb230 msgr2=0x7f9ae414cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:25.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.011+0000 7f9aeb8e5640 1 --2- 172.21.15.76:0/291525072 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ae40cb230 0x7f9ae414cdd0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400c450 tx=0x7f9ad400c910 comp rx=0 tx=0).stop 2023-11-09T03:47:25.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.011+0000 7f9aeb8e5640 1 -- 172.21.15.76:0/291525072 shutdown_connections 2023-11-09T03:47:25.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.011+0000 7f9aeb8e5640 1 --2- 172.21.15.76:0/291525072 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9ac806ae70 0x7f9ac806d330 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:25.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.011+0000 7f9aeb8e5640 1 --2- 172.21.15.76:0/291525072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ae40ac9f0 0x7f9ae40aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:25.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.011+0000 7f9aeb8e5640 1 --2- 172.21.15.76:0/291525072 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ae40cb230 0x7f9ae414cdd0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:25.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.011+0000 7f9aeb8e5640 1 --2- 172.21.15.76:0/291525072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ae40b4190 0x7f9ae40c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:25.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.011+0000 7f9aeb8e5640 1 -- 172.21.15.76:0/291525072 >> 172.21.15.76:0/291525072 conn(0x7f9ae40b03c0 msgr2=0x7f9ae40c98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:25.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.011+0000 7f9aeb8e5640 1 -- 172.21.15.76:0/291525072 shutdown_connections 2023-11-09T03:47:25.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.011+0000 7f9aeb8e5640 1 -- 172.21.15.76:0/291525072 wait complete. 2023-11-09T03:47:25.021 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2023-11-09T03:47:24.777879+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":234,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222420,"kb_used_data":6220,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752300,"statfs":{"total":773094113280,"available":772866355200,"internally_reserved":0,"allocated":6369280,"data_stored":3499079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.861678"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248379+0000","last_change":"2023-11-09T03:47:20.941252+0000","last_active":"2023-11-09T03:47:22.248379+0000","last_peered":"2023-11-09T03:47:22.248379+0000","last_clean":"2023-11-09T03:47:22.248379+0000","last_became_active":"2023-11-09T03:47:20.940995+0000","last_became_peered":"2023-11-09T03:47:20.940995+0000","last_unstale":"2023-11-09T03:47:22.248379+0000","last_undegraded":"2023-11-09T03:47:22.248379+0000","last_fullsized":"2023-11-09T03:47:22.248379+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:44:27.004461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248080+0000","last_change":"2023-11-09T03:47:20.950878+0000","last_active":"2023-11-09T03:47:22.248080+0000","last_peered":"2023-11-09T03:47:22.248080+0000","last_clean":"2023-11-09T03:47:22.248080+0000","last_became_active":"2023-11-09T03:47:20.950611+0000","last_became_peered":"2023-11-09T03:47:20.950611+0000","last_unstale":"2023-11-09T03:47:22.248080+0000","last_undegraded":"2023-11-09T03:47:22.248080+0000","last_fullsized":"2023-11-09T03:47:22.248080+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:04:18.340460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026954+0000","last_change":"2023-11-09T03:47:20.945634+0000","last_active":"2023-11-09T03:47:22.026954+0000","last_peered":"2023-11-09T03:47:22.026954+0000","last_clean":"2023-11-09T03:47:22.026954+0000","last_became_active":"2023-11-09T03:47:20.945370+0000","last_became_peered":"2023-11-09T03:47:20.945370+0000","last_unstale":"2023-11-09T03:47:22.026954+0000","last_undegraded":"2023-11-09T03:47:22.026954+0000","last_fullsized":"2023-11-09T03:47:22.026954+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:57.187676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940818+0000","last_change":"2023-11-09T03:47:20.941855+0000","last_active":"2023-11-09T03:47:21.940818+0000","last_peered":"2023-11-09T03:47:21.940818+0000","last_clean":"2023-11-09T03:47:21.940818+0000","last_became_active":"2023-11-09T03:47:20.941598+0000","last_became_peered":"2023-11-09T03:47:20.941598+0000","last_unstale":"2023-11-09T03:47:21.940818+0000","last_undegraded":"2023-11-09T03:47:21.940818+0000","last_fullsized":"2023-11-09T03:47:21.940818+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:14:00.724882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248512+0000","last_change":"2023-11-09T03:47:20.947154+0000","last_active":"2023-11-09T03:47:22.248512+0000","last_peered":"2023-11-09T03:47:22.248512+0000","last_clean":"2023-11-09T03:47:22.248512+0000","last_became_active":"2023-11-09T03:47:20.946878+0000","last_became_peered":"2023-11-09T03:47:20.946878+0000","last_unstale":"2023-11-09T03:47:22.248512+0000","last_undegraded":"2023-11-09T03:47:22.248512+0000","last_fullsized":"2023-11-09T03:47:22.248512+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:41:16.327295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247898+0000","last_change":"2023-11-09T03:47:20.941698+0000","last_active":"2023-11-09T03:47:22.247898+0000","last_peered":"2023-11-09T03:47:22.247898+0000","last_clean":"2023-11-09T03:47:22.247898+0000","last_became_active":"2023-11-09T03:47:20.941432+0000","last_became_peered":"2023-11-09T03:47:20.941432+0000","last_unstale":"2023-11-09T03:47:22.247898+0000","last_undegraded":"2023-11-09T03:47:22.247898+0000","last_fullsized":"2023-11-09T03:47:22.247898+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:12:48.690396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247980+0000","last_change":"2023-11-09T03:47:20.948132+0000","last_active":"2023-11-09T03:47:22.247980+0000","last_peered":"2023-11-09T03:47:22.247980+0000","last_clean":"2023-11-09T03:47:22.247980+0000","last_became_active":"2023-11-09T03:47:20.947854+0000","last_became_peered":"2023-11-09T03:47:20.947854+0000","last_unstale":"2023-11-09T03:47:22.247980+0000","last_undegraded":"2023-11-09T03:47:22.247980+0000","last_fullsized":"2023-11-09T03:47:22.247980+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:38:09.677347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248627+0000","last_change":"2023-11-09T03:47:20.944502+0000","last_active":"2023-11-09T03:47:22.248627+0000","last_peered":"2023-11-09T03:47:22.248627+0000","last_clean":"2023-11-09T03:47:22.248627+0000","last_became_active":"2023-11-09T03:47:20.944214+0000","last_became_peered":"2023-11-09T03:47:20.944214+0000","last_unstale":"2023-11-09T03:47:22.248627+0000","last_undegraded":"2023-11-09T03:47:22.248627+0000","last_fullsized":"2023-11-09T03:47:22.248627+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:22:14.432893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248644+0000","last_change":"2023-11-09T03:47:20.954329+0000","last_active":"2023-11-09T03:47:22.248644+0000","last_peered":"2023-11-09T03:47:22.248644+0000","last_clean":"2023-11-09T03:47:22.248644+0000","last_became_active":"2023-11-09T03:47:20.953654+0000","last_became_peered":"2023-11-09T03:47:20.953654+0000","last_unstale":"2023-11-09T03:47:22.248644+0000","last_undegraded":"2023-11-09T03:47:22.248644+0000","last_fullsized":"2023-11-09T03:47:22.248644+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:48:27.582736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939384+0000","last_change":"2023-11-09T03:47:20.943196+0000","last_active":"2023-11-09T03:47:21.939384+0000","last_peered":"2023-11-09T03:47:21.939384+0000","last_clean":"2023-11-09T03:47:21.939384+0000","last_became_active":"2023-11-09T03:47:20.942943+0000","last_became_peered":"2023-11-09T03:47:20.942943+0000","last_unstale":"2023-11-09T03:47:21.939384+0000","last_undegraded":"2023-11-09T03:47:21.939384+0000","last_fullsized":"2023-11-09T03:47:21.939384+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:36:04.069418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939364+0000","last_change":"2023-11-09T03:47:20.943821+0000","last_active":"2023-11-09T03:47:21.939364+0000","last_peered":"2023-11-09T03:47:21.939364+0000","last_clean":"2023-11-09T03:47:21.939364+0000","last_became_active":"2023-11-09T03:47:20.943571+0000","last_became_peered":"2023-11-09T03:47:20.943571+0000","last_unstale":"2023-11-09T03:47:21.939364+0000","last_undegraded":"2023-11-09T03:47:21.939364+0000","last_fullsized":"2023-11-09T03:47:21.939364+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:26:12.103741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940598+0000","last_change":"2023-11-09T03:47:20.942582+0000","last_active":"2023-11-09T03:47:21.940598+0000","last_peered":"2023-11-09T03:47:21.940598+0000","last_clean":"2023-11-09T03:47:21.940598+0000","last_became_active":"2023-11-09T03:47:20.942214+0000","last_became_peered":"2023-11-09T03:47:20.942214+0000","last_unstale":"2023-11-09T03:47:21.940598+0000","last_undegraded":"2023-11-09T03:47:21.940598+0000","last_fullsized":"2023-11-09T03:47:21.940598+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:51:53.879857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248688+0000","last_change":"2023-11-09T03:47:20.942261+0000","last_active":"2023-11-09T03:47:22.248688+0000","last_peered":"2023-11-09T03:47:22.248688+0000","last_clean":"2023-11-09T03:47:22.248688+0000","last_became_active":"2023-11-09T03:47:20.941996+0000","last_became_peered":"2023-11-09T03:47:20.941996+0000","last_unstale":"2023-11-09T03:47:22.248688+0000","last_undegraded":"2023-11-09T03:47:22.248688+0000","last_fullsized":"2023-11-09T03:47:22.248688+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:21:04.459156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248066+0000","last_change":"2023-11-09T03:47:20.944425+0000","last_active":"2023-11-09T03:47:22.248066+0000","last_peered":"2023-11-09T03:47:22.248066+0000","last_clean":"2023-11-09T03:47:22.248066+0000","last_became_active":"2023-11-09T03:47:20.944159+0000","last_became_peered":"2023-11-09T03:47:20.944159+0000","last_unstale":"2023-11-09T03:47:22.248066+0000","last_undegraded":"2023-11-09T03:47:22.248066+0000","last_fullsized":"2023-11-09T03:47:22.248066+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:26:12.103741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248162+0000","last_change":"2023-11-09T03:47:20.954225+0000","last_active":"2023-11-09T03:47:22.248162+0000","last_peered":"2023-11-09T03:47:22.248162+0000","last_clean":"2023-11-09T03:47:22.248162+0000","last_became_active":"2023-11-09T03:47:20.954004+0000","last_became_peered":"2023-11-09T03:47:20.954004+0000","last_unstale":"2023-11-09T03:47:22.248162+0000","last_undegraded":"2023-11-09T03:47:22.248162+0000","last_fullsized":"2023-11-09T03:47:22.248162+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:36:04.069418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247697+0000","last_change":"2023-11-09T03:47:20.940209+0000","last_active":"2023-11-09T03:47:22.247697+0000","last_peered":"2023-11-09T03:47:22.247697+0000","last_clean":"2023-11-09T03:47:22.247697+0000","last_became_active":"2023-11-09T03:47:20.939956+0000","last_became_peered":"2023-11-09T03:47:20.939956+0000","last_unstale":"2023-11-09T03:47:22.247697+0000","last_undegraded":"2023-11-09T03:47:22.247697+0000","last_fullsized":"2023-11-09T03:47:22.247697+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:52.683555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248360+0000","last_change":"2023-11-09T03:47:20.942850+0000","last_active":"2023-11-09T03:47:22.248360+0000","last_peered":"2023-11-09T03:47:22.248360+0000","last_clean":"2023-11-09T03:47:22.248360+0000","last_became_active":"2023-11-09T03:47:20.942596+0000","last_became_peered":"2023-11-09T03:47:20.942596+0000","last_unstale":"2023-11-09T03:47:22.248360+0000","last_undegraded":"2023-11-09T03:47:22.248360+0000","last_fullsized":"2023-11-09T03:47:22.248360+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:14:28.441176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.027085+0000","last_change":"2023-11-09T03:47:20.954037+0000","last_active":"2023-11-09T03:47:22.027085+0000","last_peered":"2023-11-09T03:47:22.027085+0000","last_clean":"2023-11-09T03:47:22.027085+0000","last_became_active":"2023-11-09T03:47:20.953777+0000","last_became_peered":"2023-11-09T03:47:20.953777+0000","last_unstale":"2023-11-09T03:47:22.027085+0000","last_undegraded":"2023-11-09T03:47:22.027085+0000","last_fullsized":"2023-11-09T03:47:22.027085+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:55:28.077165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248233+0000","last_change":"2023-11-09T03:47:20.941849+0000","last_active":"2023-11-09T03:47:22.248233+0000","last_peered":"2023-11-09T03:47:22.248233+0000","last_clean":"2023-11-09T03:47:22.248233+0000","last_became_active":"2023-11-09T03:47:20.941602+0000","last_became_peered":"2023-11-09T03:47:20.941602+0000","last_unstale":"2023-11-09T03:47:22.248233+0000","last_undegraded":"2023-11-09T03:47:22.248233+0000","last_fullsized":"2023-11-09T03:47:22.248233+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:23.398916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248619+0000","last_change":"2023-11-09T03:47:20.951388+0000","last_active":"2023-11-09T03:47:22.248619+0000","last_peered":"2023-11-09T03:47:22.248619+0000","last_clean":"2023-11-09T03:47:22.248619+0000","last_became_active":"2023-11-09T03:47:20.951123+0000","last_became_peered":"2023-11-09T03:47:20.951123+0000","last_unstale":"2023-11-09T03:47:22.248619+0000","last_undegraded":"2023-11-09T03:47:22.248619+0000","last_fullsized":"2023-11-09T03:47:22.248619+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:55:54.333416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026719+0000","last_change":"2023-11-09T03:47:20.948114+0000","last_active":"2023-11-09T03:47:22.026719+0000","last_peered":"2023-11-09T03:47:22.026719+0000","last_clean":"2023-11-09T03:47:22.026719+0000","last_became_active":"2023-11-09T03:47:20.947849+0000","last_became_peered":"2023-11-09T03:47:20.947849+0000","last_unstale":"2023-11-09T03:47:22.026719+0000","last_undegraded":"2023-11-09T03:47:22.026719+0000","last_fullsized":"2023-11-09T03:47:22.026719+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:52.683555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247955+0000","last_change":"2023-11-09T03:47:20.945723+0000","last_active":"2023-11-09T03:47:22.247955+0000","last_peered":"2023-11-09T03:47:22.247955+0000","last_clean":"2023-11-09T03:47:22.247955+0000","last_became_active":"2023-11-09T03:47:20.945447+0000","last_became_peered":"2023-11-09T03:47:20.945447+0000","last_unstale":"2023-11-09T03:47:22.247955+0000","last_undegraded":"2023-11-09T03:47:22.247955+0000","last_fullsized":"2023-11-09T03:47:22.247955+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:41:48.124870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026634+0000","last_change":"2023-11-09T03:47:20.944344+0000","last_active":"2023-11-09T03:47:22.026634+0000","last_peered":"2023-11-09T03:47:22.026634+0000","last_clean":"2023-11-09T03:47:22.026634+0000","last_became_active":"2023-11-09T03:47:20.943991+0000","last_became_peered":"2023-11-09T03:47:20.943991+0000","last_unstale":"2023-11-09T03:47:22.026634+0000","last_undegraded":"2023-11-09T03:47:22.026634+0000","last_fullsized":"2023-11-09T03:47:22.026634+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:08:17.239107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248007+0000","last_change":"2023-11-09T03:47:20.949489+0000","last_active":"2023-11-09T03:47:22.248007+0000","last_peered":"2023-11-09T03:47:22.248007+0000","last_clean":"2023-11-09T03:47:22.248007+0000","last_became_active":"2023-11-09T03:47:20.949211+0000","last_became_peered":"2023-11-09T03:47:20.949211+0000","last_unstale":"2023-11-09T03:47:22.248007+0000","last_undegraded":"2023-11-09T03:47:22.248007+0000","last_fullsized":"2023-11-09T03:47:22.248007+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:56:40.000841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248496+0000","last_change":"2023-11-09T03:47:20.952835+0000","last_active":"2023-11-09T03:47:22.248496+0000","last_peered":"2023-11-09T03:47:22.248496+0000","last_clean":"2023-11-09T03:47:22.248496+0000","last_became_active":"2023-11-09T03:47:20.952578+0000","last_became_peered":"2023-11-09T03:47:20.952578+0000","last_unstale":"2023-11-09T03:47:22.248496+0000","last_undegraded":"2023-11-09T03:47:22.248496+0000","last_fullsized":"2023-11-09T03:47:22.248496+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:40:53.848669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.941032+0000","last_change":"2023-11-09T03:47:20.944379+0000","last_active":"2023-11-09T03:47:21.941032+0000","last_peered":"2023-11-09T03:47:21.941032+0000","last_clean":"2023-11-09T03:47:21.941032+0000","last_became_active":"2023-11-09T03:47:20.944134+0000","last_became_peered":"2023-11-09T03:47:20.944134+0000","last_unstale":"2023-11-09T03:47:21.941032+0000","last_undegraded":"2023-11-09T03:47:21.941032+0000","last_fullsized":"2023-11-09T03:47:21.941032+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:55:28.077165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248970+0000","last_change":"2023-11-09T03:47:20.940786+0000","last_active":"2023-11-09T03:47:22.248970+0000","last_peered":"2023-11-09T03:47:22.248970+0000","last_clean":"2023-11-09T03:47:22.248970+0000","last_became_active":"2023-11-09T03:47:20.940529+0000","last_became_peered":"2023-11-09T03:47:20.940529+0000","last_unstale":"2023-11-09T03:47:22.248970+0000","last_undegraded":"2023-11-09T03:47:22.248970+0000","last_fullsized":"2023-11-09T03:47:22.248970+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:40.639821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248129+0000","last_change":"2023-11-09T03:47:20.949337+0000","last_active":"2023-11-09T03:47:22.248129+0000","last_peered":"2023-11-09T03:47:22.248129+0000","last_clean":"2023-11-09T03:47:22.248129+0000","last_became_active":"2023-11-09T03:47:20.949088+0000","last_became_peered":"2023-11-09T03:47:20.949088+0000","last_unstale":"2023-11-09T03:47:22.248129+0000","last_undegraded":"2023-11-09T03:47:22.248129+0000","last_fullsized":"2023-11-09T03:47:22.248129+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:52:12.660646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248774+0000","last_change":"2023-11-09T03:47:20.948003+0000","last_active":"2023-11-09T03:47:22.248774+0000","last_peered":"2023-11-09T03:47:22.248774+0000","last_clean":"2023-11-09T03:47:22.248774+0000","last_became_active":"2023-11-09T03:47:20.947690+0000","last_became_peered":"2023-11-09T03:47:20.947690+0000","last_unstale":"2023-11-09T03:47:22.248774+0000","last_undegraded":"2023-11-09T03:47:22.248774+0000","last_fullsized":"2023-11-09T03:47:22.248774+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:28:18.013453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026610+0000","last_change":"2023-11-09T03:47:20.946260+0000","last_active":"2023-11-09T03:47:22.026610+0000","last_peered":"2023-11-09T03:47:22.026610+0000","last_clean":"2023-11-09T03:47:22.026610+0000","last_became_active":"2023-11-09T03:47:20.945986+0000","last_became_peered":"2023-11-09T03:47:20.945986+0000","last_unstale":"2023-11-09T03:47:22.026610+0000","last_undegraded":"2023-11-09T03:47:22.026610+0000","last_fullsized":"2023-11-09T03:47:22.026610+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:59:29.154623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247961+0000","last_change":"2023-11-09T03:47:20.952176+0000","last_active":"2023-11-09T03:47:22.247961+0000","last_peered":"2023-11-09T03:47:22.247961+0000","last_clean":"2023-11-09T03:47:22.247961+0000","last_became_active":"2023-11-09T03:47:20.951921+0000","last_became_peered":"2023-11-09T03:47:20.951921+0000","last_unstale":"2023-11-09T03:47:22.247961+0000","last_undegraded":"2023-11-09T03:47:22.247961+0000","last_fullsized":"2023-11-09T03:47:22.247961+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:22.376023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247669+0000","last_change":"2023-11-09T03:47:20.943940+0000","last_active":"2023-11-09T03:47:22.247669+0000","last_peered":"2023-11-09T03:47:22.247669+0000","last_clean":"2023-11-09T03:47:22.247669+0000","last_became_active":"2023-11-09T03:47:20.943708+0000","last_became_peered":"2023-11-09T03:47:20.943708+0000","last_unstale":"2023-11-09T03:47:22.247669+0000","last_undegraded":"2023-11-09T03:47:22.247669+0000","last_fullsized":"2023-11-09T03:47:22.247669+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:08:17.239107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026790+0000","last_change":"2023-11-09T03:47:20.942360+0000","last_active":"2023-11-09T03:47:22.026790+0000","last_peered":"2023-11-09T03:47:22.026790+0000","last_clean":"2023-11-09T03:47:22.026790+0000","last_became_active":"2023-11-09T03:47:20.942078+0000","last_became_peered":"2023-11-09T03:47:20.942078+0000","last_unstale":"2023-11-09T03:47:22.026790+0000","last_undegraded":"2023-11-09T03:47:22.026790+0000","last_fullsized":"2023-11-09T03:47:22.026790+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:12:48.690396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248715+0000","last_change":"2023-11-09T03:47:19.943955+0000","last_active":"2023-11-09T03:47:22.248715+0000","last_peered":"2023-11-09T03:47:22.248715+0000","last_clean":"2023-11-09T03:47:22.248715+0000","last_became_active":"2023-11-09T03:47:19.943692+0000","last_became_peered":"2023-11-09T03:47:19.943692+0000","last_unstale":"2023-11-09T03:47:22.248715+0000","last_undegraded":"2023-11-09T03:47:22.248715+0000","last_fullsized":"2023-11-09T03:47:22.248715+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:24.100578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":27,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.928645+0000","last_change":"2023-11-09T03:47:02.581917+0000","last_active":"2023-11-09T03:47:20.928645+0000","last_peered":"2023-11-09T03:47:20.928645+0000","last_clean":"2023-11-09T03:47:20.928645+0000","last_became_active":"2023-11-09T03:47:01.588082+0000","last_became_peered":"2023-11-09T03:47:01.588082+0000","last_unstale":"2023-11-09T03:47:20.928645+0000","last_undegraded":"2023-11-09T03:47:20.928645+0000","last_fullsized":"2023-11-09T03:47:20.928645+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:44:07.653557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034538899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940245+0000","last_change":"2023-11-09T03:47:20.938900+0000","last_active":"2023-11-09T03:47:21.940245+0000","last_peered":"2023-11-09T03:47:21.940245+0000","last_clean":"2023-11-09T03:47:21.940245+0000","last_became_active":"2023-11-09T03:47:20.938654+0000","last_became_peered":"2023-11-09T03:47:20.938654+0000","last_unstale":"2023-11-09T03:47:21.940245+0000","last_undegraded":"2023-11-09T03:47:21.940245+0000","last_fullsized":"2023-11-09T03:47:21.940245+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:40.639821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248651+0000","last_change":"2023-11-09T03:47:19.945834+0000","last_active":"2023-11-09T03:47:22.248651+0000","last_peered":"2023-11-09T03:47:22.248651+0000","last_clean":"2023-11-09T03:47:22.248651+0000","last_became_active":"2023-11-09T03:47:19.945484+0000","last_became_peered":"2023-11-09T03:47:19.945484+0000","last_unstale":"2023-11-09T03:47:22.248651+0000","last_undegraded":"2023-11-09T03:47:22.248651+0000","last_fullsized":"2023-11-09T03:47:22.248651+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:57:21.887088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.930614+0000","last_change":"2023-11-09T03:47:03.029063+0000","last_active":"2023-11-09T03:47:20.930614+0000","last_peered":"2023-11-09T03:47:20.930614+0000","last_clean":"2023-11-09T03:47:20.930614+0000","last_became_active":"2023-11-09T03:47:01.593582+0000","last_became_peered":"2023-11-09T03:47:01.593582+0000","last_unstale":"2023-11-09T03:47:20.930614+0000","last_undegraded":"2023-11-09T03:47:20.930614+0000","last_fullsized":"2023-11-09T03:47:20.930614+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:56:20.649937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041855100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026721+0000","last_change":"2023-11-09T03:47:20.944419+0000","last_active":"2023-11-09T03:47:22.026721+0000","last_peered":"2023-11-09T03:47:22.026721+0000","last_clean":"2023-11-09T03:47:22.026721+0000","last_became_active":"2023-11-09T03:47:20.944141+0000","last_became_peered":"2023-11-09T03:47:20.944141+0000","last_unstale":"2023-11-09T03:47:22.026721+0000","last_undegraded":"2023-11-09T03:47:22.026721+0000","last_fullsized":"2023-11-09T03:47:22.026721+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:06:17.676049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"53'8","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026516+0000","last_change":"2023-11-09T03:47:19.946625+0000","last_active":"2023-11-09T03:47:22.026516+0000","last_peered":"2023-11-09T03:47:22.026516+0000","last_clean":"2023-11-09T03:47:22.026516+0000","last_became_active":"2023-11-09T03:47:19.946350+0000","last_became_peered":"2023-11-09T03:47:19.946350+0000","last_unstale":"2023-11-09T03:47:22.026516+0000","last_undegraded":"2023-11-09T03:47:22.026516+0000","last_fullsized":"2023-11-09T03:47:22.026516+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:32.646643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.934959+0000","last_change":"2023-11-09T03:47:02.619336+0000","last_active":"2023-11-09T03:47:20.934959+0000","last_peered":"2023-11-09T03:47:20.934959+0000","last_clean":"2023-11-09T03:47:20.934959+0000","last_became_active":"2023-11-09T03:47:01.587749+0000","last_became_peered":"2023-11-09T03:47:01.587749+0000","last_unstale":"2023-11-09T03:47:20.934959+0000","last_undegraded":"2023-11-09T03:47:20.934959+0000","last_fullsized":"2023-11-09T03:47:20.934959+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063032099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247912+0000","last_change":"2023-11-09T03:47:20.948391+0000","last_active":"2023-11-09T03:47:22.247912+0000","last_peered":"2023-11-09T03:47:22.247912+0000","last_clean":"2023-11-09T03:47:22.247912+0000","last_became_active":"2023-11-09T03:47:20.948103+0000","last_became_peered":"2023-11-09T03:47:20.948103+0000","last_unstale":"2023-11-09T03:47:22.247912+0000","last_undegraded":"2023-11-09T03:47:22.247912+0000","last_fullsized":"2023-11-09T03:47:22.247912+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:33.651326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940202+0000","last_change":"2023-11-09T03:47:19.944514+0000","last_active":"2023-11-09T03:47:21.940202+0000","last_peered":"2023-11-09T03:47:21.940202+0000","last_clean":"2023-11-09T03:47:21.940202+0000","last_became_active":"2023-11-09T03:47:19.944230+0000","last_became_peered":"2023-11-09T03:47:19.944230+0000","last_unstale":"2023-11-09T03:47:21.940202+0000","last_undegraded":"2023-11-09T03:47:21.940202+0000","last_fullsized":"2023-11-09T03:47:21.940202+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:08:16.234424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":27,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.928406+0000","last_change":"2023-11-09T03:47:02.581658+0000","last_active":"2023-11-09T03:47:20.928406+0000","last_peered":"2023-11-09T03:47:20.928406+0000","last_clean":"2023-11-09T03:47:20.928406+0000","last_became_active":"2023-11-09T03:47:01.588612+0000","last_became_peered":"2023-11-09T03:47:01.588612+0000","last_unstale":"2023-11-09T03:47:20.928406+0000","last_undegraded":"2023-11-09T03:47:20.928406+0000","last_fullsized":"2023-11-09T03:47:20.928406+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045238500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.027066+0000","last_change":"2023-11-09T03:47:20.952326+0000","last_active":"2023-11-09T03:47:22.027066+0000","last_peered":"2023-11-09T03:47:22.027066+0000","last_clean":"2023-11-09T03:47:22.027066+0000","last_became_active":"2023-11-09T03:47:20.952035+0000","last_became_peered":"2023-11-09T03:47:20.952035+0000","last_unstale":"2023-11-09T03:47:22.027066+0000","last_undegraded":"2023-11-09T03:47:22.027066+0000","last_fullsized":"2023-11-09T03:47:22.027066+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:14:00.724882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248147+0000","last_change":"2023-11-09T03:47:19.947282+0000","last_active":"2023-11-09T03:47:22.248147+0000","last_peered":"2023-11-09T03:47:22.248147+0000","last_clean":"2023-11-09T03:47:22.248147+0000","last_became_active":"2023-11-09T03:47:19.947034+0000","last_became_peered":"2023-11-09T03:47:19.947034+0000","last_unstale":"2023-11-09T03:47:22.248147+0000","last_undegraded":"2023-11-09T03:47:22.248147+0000","last_fullsized":"2023-11-09T03:47:22.248147+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:13:59.720199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.935070+0000","last_change":"2023-11-09T03:47:02.618828+0000","last_active":"2023-11-09T03:47:20.935070+0000","last_peered":"2023-11-09T03:47:20.935070+0000","last_clean":"2023-11-09T03:47:20.935070+0000","last_became_active":"2023-11-09T03:47:01.590679+0000","last_became_peered":"2023-11-09T03:47:01.590679+0000","last_unstale":"2023-11-09T03:47:20.935070+0000","last_undegraded":"2023-11-09T03:47:20.935070+0000","last_fullsized":"2023-11-09T03:47:20.935070+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:06:30.982987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070668,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248700+0000","last_change":"2023-11-09T03:47:20.954224+0000","last_active":"2023-11-09T03:47:22.248700+0000","last_peered":"2023-11-09T03:47:22.248700+0000","last_clean":"2023-11-09T03:47:22.248700+0000","last_became_active":"2023-11-09T03:47:20.953950+0000","last_became_peered":"2023-11-09T03:47:20.953950+0000","last_unstale":"2023-11-09T03:47:22.248700+0000","last_undegraded":"2023-11-09T03:47:22.248700+0000","last_fullsized":"2023-11-09T03:47:22.248700+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:58:09.769432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.027101+0000","last_change":"2023-11-09T03:47:19.947929+0000","last_active":"2023-11-09T03:47:22.027101+0000","last_peered":"2023-11-09T03:47:22.027101+0000","last_clean":"2023-11-09T03:47:22.027101+0000","last_became_active":"2023-11-09T03:47:19.947663+0000","last_became_peered":"2023-11-09T03:47:19.947663+0000","last_unstale":"2023-11-09T03:47:22.027101+0000","last_undegraded":"2023-11-09T03:47:22.027101+0000","last_fullsized":"2023-11-09T03:47:22.027101+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:02:12.613580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931259+0000","last_change":"2023-11-09T03:47:03.146528+0000","last_active":"2023-11-09T03:47:20.931259+0000","last_peered":"2023-11-09T03:47:20.931259+0000","last_clean":"2023-11-09T03:47:20.931259+0000","last_became_active":"2023-11-09T03:47:01.587060+0000","last_became_peered":"2023-11-09T03:47:01.587060+0000","last_unstale":"2023-11-09T03:47:20.931259+0000","last_undegraded":"2023-11-09T03:47:20.931259+0000","last_fullsized":"2023-11-09T03:47:20.931259+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:39:15.238496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079064999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.027243+0000","last_change":"2023-11-09T03:47:20.954274+0000","last_active":"2023-11-09T03:47:22.027243+0000","last_peered":"2023-11-09T03:47:22.027243+0000","last_clean":"2023-11-09T03:47:22.027243+0000","last_became_active":"2023-11-09T03:47:20.953976+0000","last_became_peered":"2023-11-09T03:47:20.953976+0000","last_unstale":"2023-11-09T03:47:22.027243+0000","last_undegraded":"2023-11-09T03:47:22.027243+0000","last_fullsized":"2023-11-09T03:47:22.027243+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:05:32.662676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"53'6","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248315+0000","last_change":"2023-11-09T03:47:19.948464+0000","last_active":"2023-11-09T03:47:22.248315+0000","last_peered":"2023-11-09T03:47:22.248315+0000","last_clean":"2023-11-09T03:47:22.248315+0000","last_became_active":"2023-11-09T03:47:19.948254+0000","last_became_peered":"2023-11-09T03:47:19.948254+0000","last_unstale":"2023-11-09T03:47:22.248315+0000","last_undegraded":"2023-11-09T03:47:22.248315+0000","last_fullsized":"2023-11-09T03:47:22.248315+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:21:03.454473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931191+0000","last_change":"2023-11-09T03:47:03.148246+0000","last_active":"2023-11-09T03:47:20.931191+0000","last_peered":"2023-11-09T03:47:20.931191+0000","last_clean":"2023-11-09T03:47:20.931191+0000","last_became_active":"2023-11-09T03:47:01.590988+0000","last_became_peered":"2023-11-09T03:47:01.590988+0000","last_unstale":"2023-11-09T03:47:20.931191+0000","last_undegraded":"2023-11-09T03:47:20.931191+0000","last_fullsized":"2023-11-09T03:47:20.931191+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:07:20.785445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055651099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939172+0000","last_change":"2023-11-09T03:47:20.945226+0000","last_active":"2023-11-09T03:47:21.939172+0000","last_peered":"2023-11-09T03:47:21.939172+0000","last_clean":"2023-11-09T03:47:21.939172+0000","last_became_active":"2023-11-09T03:47:20.944978+0000","last_became_peered":"2023-11-09T03:47:20.944978+0000","last_unstale":"2023-11-09T03:47:21.939172+0000","last_undegraded":"2023-11-09T03:47:21.939172+0000","last_fullsized":"2023-11-09T03:47:21.939172+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:33.651326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"53'3","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026995+0000","last_change":"2023-11-09T03:47:19.945101+0000","last_active":"2023-11-09T03:47:22.026995+0000","last_peered":"2023-11-09T03:47:22.026995+0000","last_clean":"2023-11-09T03:47:22.026995+0000","last_became_active":"2023-11-09T03:47:19.944829+0000","last_became_peered":"2023-11-09T03:47:19.944829+0000","last_unstale":"2023-11-09T03:47:22.026995+0000","last_undegraded":"2023-11-09T03:47:22.026995+0000","last_fullsized":"2023-11-09T03:47:22.026995+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:49:12.247865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.939514+0000","last_change":"2023-11-09T03:47:02.582754+0000","last_active":"2023-11-09T03:47:20.939514+0000","last_peered":"2023-11-09T03:47:20.939514+0000","last_clean":"2023-11-09T03:47:20.939514+0000","last_became_active":"2023-11-09T03:47:01.589715+0000","last_became_peered":"2023-11-09T03:47:01.589715+0000","last_unstale":"2023-11-09T03:47:20.939514+0000","last_undegraded":"2023-11-09T03:47:20.939514+0000","last_fullsized":"2023-11-09T03:47:20.939514+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099342799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248560+0000","last_change":"2023-11-09T03:47:20.953974+0000","last_active":"2023-11-09T03:47:22.248560+0000","last_peered":"2023-11-09T03:47:22.248560+0000","last_clean":"2023-11-09T03:47:22.248560+0000","last_became_active":"2023-11-09T03:47:20.953714+0000","last_became_peered":"2023-11-09T03:47:20.953714+0000","last_unstale":"2023-11-09T03:47:22.248560+0000","last_undegraded":"2023-11-09T03:47:22.248560+0000","last_fullsized":"2023-11-09T03:47:22.248560+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:40:44.918031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"53'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248845+0000","last_change":"2023-11-09T03:47:19.943054+0000","last_active":"2023-11-09T03:47:22.248845+0000","last_peered":"2023-11-09T03:47:22.248845+0000","last_clean":"2023-11-09T03:47:22.248845+0000","last_became_active":"2023-11-09T03:47:19.942787+0000","last_became_peered":"2023-11-09T03:47:19.942787+0000","last_unstale":"2023-11-09T03:47:22.248845+0000","last_undegraded":"2023-11-09T03:47:22.248845+0000","last_fullsized":"2023-11-09T03:47:22.248845+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:58:08.764749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.939087+0000","last_change":"2023-11-09T03:47:02.724896+0000","last_active":"2023-11-09T03:47:20.939087+0000","last_peered":"2023-11-09T03:47:20.939087+0000","last_clean":"2023-11-09T03:47:20.939087+0000","last_became_active":"2023-11-09T03:47:01.592784+0000","last_became_peered":"2023-11-09T03:47:01.592784+0000","last_unstale":"2023-11-09T03:47:20.939087+0000","last_undegraded":"2023-11-09T03:47:20.939087+0000","last_fullsized":"2023-11-09T03:47:20.939087+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:51:32.016204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056683500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248073+0000","last_change":"2023-11-09T03:47:20.945066+0000","last_active":"2023-11-09T03:47:22.248073+0000","last_peered":"2023-11-09T03:47:22.248073+0000","last_clean":"2023-11-09T03:47:22.248073+0000","last_became_active":"2023-11-09T03:47:20.944812+0000","last_became_peered":"2023-11-09T03:47:20.944812+0000","last_unstale":"2023-11-09T03:47:22.248073+0000","last_undegraded":"2023-11-09T03:47:22.248073+0000","last_fullsized":"2023-11-09T03:47:22.248073+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:58:30.558728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"53'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248249+0000","last_change":"2023-11-09T03:47:19.947954+0000","last_active":"2023-11-09T03:47:22.248249+0000","last_peered":"2023-11-09T03:47:22.248249+0000","last_clean":"2023-11-09T03:47:22.248249+0000","last_became_active":"2023-11-09T03:47:19.947687+0000","last_became_peered":"2023-11-09T03:47:19.947687+0000","last_unstale":"2023-11-09T03:47:22.248249+0000","last_undegraded":"2023-11-09T03:47:22.248249+0000","last_fullsized":"2023-11-09T03:47:22.248249+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:52:58.422763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.933388+0000","last_change":"2023-11-09T03:47:03.146527+0000","last_active":"2023-11-09T03:47:20.933388+0000","last_peered":"2023-11-09T03:47:20.933388+0000","last_clean":"2023-11-09T03:47:20.933388+0000","last_became_active":"2023-11-09T03:47:01.587858+0000","last_became_peered":"2023-11-09T03:47:01.587858+0000","last_unstale":"2023-11-09T03:47:20.933388+0000","last_undegraded":"2023-11-09T03:47:20.933388+0000","last_fullsized":"2023-11-09T03:47:20.933388+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:56:04.201126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084439599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248163+0000","last_change":"2023-11-09T03:47:20.943815+0000","last_active":"2023-11-09T03:47:22.248163+0000","last_peered":"2023-11-09T03:47:22.248163+0000","last_clean":"2023-11-09T03:47:22.248163+0000","last_became_active":"2023-11-09T03:47:20.943403+0000","last_became_peered":"2023-11-09T03:47:20.943403+0000","last_unstale":"2023-11-09T03:47:22.248163+0000","last_undegraded":"2023-11-09T03:47:22.248163+0000","last_fullsized":"2023-11-09T03:47:22.248163+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:59:34.738692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247950+0000","last_change":"2023-11-09T03:47:19.949858+0000","last_active":"2023-11-09T03:47:22.247950+0000","last_peered":"2023-11-09T03:47:22.247950+0000","last_clean":"2023-11-09T03:47:22.247950+0000","last_became_active":"2023-11-09T03:47:19.949636+0000","last_became_peered":"2023-11-09T03:47:19.949636+0000","last_unstale":"2023-11-09T03:47:22.247950+0000","last_undegraded":"2023-11-09T03:47:22.247950+0000","last_fullsized":"2023-11-09T03:47:22.247950+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:24.100578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.935999+0000","last_change":"2023-11-09T03:47:02.725223+0000","last_active":"2023-11-09T03:47:20.935999+0000","last_peered":"2023-11-09T03:47:20.935999+0000","last_clean":"2023-11-09T03:47:20.935999+0000","last_became_active":"2023-11-09T03:47:01.590635+0000","last_became_peered":"2023-11-09T03:47:01.590635+0000","last_unstale":"2023-11-09T03:47:20.935999+0000","last_undegraded":"2023-11-09T03:47:20.935999+0000","last_fullsized":"2023-11-09T03:47:20.935999+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:05.754357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051685100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248165+0000","last_change":"2023-11-09T03:47:20.948694+0000","last_active":"2023-11-09T03:47:22.248165+0000","last_peered":"2023-11-09T03:47:22.248165+0000","last_clean":"2023-11-09T03:47:22.248165+0000","last_became_active":"2023-11-09T03:47:20.948405+0000","last_became_peered":"2023-11-09T03:47:20.948405+0000","last_unstale":"2023-11-09T03:47:22.248165+0000","last_undegraded":"2023-11-09T03:47:22.248165+0000","last_fullsized":"2023-11-09T03:47:22.248165+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:30:50.805289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940244+0000","last_change":"2023-11-09T03:47:19.942483+0000","last_active":"2023-11-09T03:47:21.940244+0000","last_peered":"2023-11-09T03:47:21.940244+0000","last_clean":"2023-11-09T03:47:21.940244+0000","last_became_active":"2023-11-09T03:47:19.942157+0000","last_became_peered":"2023-11-09T03:47:19.942157+0000","last_unstale":"2023-11-09T03:47:21.940244+0000","last_undegraded":"2023-11-09T03:47:21.940244+0000","last_fullsized":"2023-11-09T03:47:21.940244+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:59:28.149940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.937687+0000","last_change":"2023-11-09T03:47:02.582739+0000","last_active":"2023-11-09T03:47:20.937687+0000","last_peered":"2023-11-09T03:47:20.937687+0000","last_clean":"2023-11-09T03:47:20.937687+0000","last_became_active":"2023-11-09T03:47:01.592743+0000","last_became_peered":"2023-11-09T03:47:01.592743+0000","last_unstale":"2023-11-09T03:47:20.937687+0000","last_undegraded":"2023-11-09T03:47:20.937687+0000","last_fullsized":"2023-11-09T03:47:20.937687+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051802700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939444+0000","last_change":"2023-11-09T03:47:20.939366+0000","last_active":"2023-11-09T03:47:21.939444+0000","last_peered":"2023-11-09T03:47:21.939444+0000","last_clean":"2023-11-09T03:47:21.939444+0000","last_became_active":"2023-11-09T03:47:20.938258+0000","last_became_peered":"2023-11-09T03:47:20.938258+0000","last_unstale":"2023-11-09T03:47:21.939444+0000","last_undegraded":"2023-11-09T03:47:21.939444+0000","last_fullsized":"2023-11-09T03:47:21.939444+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:48:27.582736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"53'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248057+0000","last_change":"2023-11-09T03:47:19.946306+0000","last_active":"2023-11-09T03:47:22.248057+0000","last_peered":"2023-11-09T03:47:22.248057+0000","last_clean":"2023-11-09T03:47:22.248057+0000","last_became_active":"2023-11-09T03:47:19.946062+0000","last_became_peered":"2023-11-09T03:47:19.946062+0000","last_unstale":"2023-11-09T03:47:22.248057+0000","last_undegraded":"2023-11-09T03:47:22.248057+0000","last_fullsized":"2023-11-09T03:47:22.248057+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:51:52.875174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.935441+0000","last_change":"2023-11-09T03:47:02.724953+0000","last_active":"2023-11-09T03:47:20.935441+0000","last_peered":"2023-11-09T03:47:20.935441+0000","last_clean":"2023-11-09T03:47:20.935441+0000","last_became_active":"2023-11-09T03:47:01.587133+0000","last_became_peered":"2023-11-09T03:47:01.587133+0000","last_unstale":"2023-11-09T03:47:20.935441+0000","last_undegraded":"2023-11-09T03:47:20.935441+0000","last_fullsized":"2023-11-09T03:47:20.935441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:23:48.521161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066497599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249034+0000","last_change":"2023-11-09T03:47:20.942122+0000","last_active":"2023-11-09T03:47:22.249034+0000","last_peered":"2023-11-09T03:47:22.249034+0000","last_clean":"2023-11-09T03:47:22.249034+0000","last_became_active":"2023-11-09T03:47:20.941870+0000","last_became_peered":"2023-11-09T03:47:20.941870+0000","last_unstale":"2023-11-09T03:47:22.249034+0000","last_undegraded":"2023-11-09T03:47:22.249034+0000","last_fullsized":"2023-11-09T03:47:22.249034+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:57.187676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"53'4","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249334+0000","last_change":"2023-11-09T03:47:19.945714+0000","last_active":"2023-11-09T03:47:22.249334+0000","last_peered":"2023-11-09T03:47:22.249334+0000","last_clean":"2023-11-09T03:47:22.249334+0000","last_became_active":"2023-11-09T03:47:19.945455+0000","last_became_peered":"2023-11-09T03:47:19.945455+0000","last_unstale":"2023-11-09T03:47:22.249334+0000","last_undegraded":"2023-11-09T03:47:22.249334+0000","last_fullsized":"2023-11-09T03:47:22.249334+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:38:08.672664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.935591+0000","last_change":"2023-11-09T03:47:03.146217+0000","last_active":"2023-11-09T03:47:20.935591+0000","last_peered":"2023-11-09T03:47:20.935591+0000","last_clean":"2023-11-09T03:47:20.935591+0000","last_became_active":"2023-11-09T03:47:01.587598+0000","last_became_peered":"2023-11-09T03:47:01.587598+0000","last_unstale":"2023-11-09T03:47:20.935591+0000","last_undegraded":"2023-11-09T03:47:20.935591+0000","last_fullsized":"2023-11-09T03:47:20.935591+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:15:22.723455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090126400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249094+0000","last_change":"2023-11-09T03:47:20.950285+0000","last_active":"2023-11-09T03:47:22.249094+0000","last_peered":"2023-11-09T03:47:22.249094+0000","last_clean":"2023-11-09T03:47:22.249094+0000","last_became_active":"2023-11-09T03:47:20.949722+0000","last_became_peered":"2023-11-09T03:47:20.949722+0000","last_unstale":"2023-11-09T03:47:22.249094+0000","last_undegraded":"2023-11-09T03:47:22.249094+0000","last_fullsized":"2023-11-09T03:47:22.249094+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:51:53.879857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940663+0000","last_change":"2023-11-09T03:47:19.947977+0000","last_active":"2023-11-09T03:47:21.940663+0000","last_peered":"2023-11-09T03:47:21.940663+0000","last_clean":"2023-11-09T03:47:21.940663+0000","last_became_active":"2023-11-09T03:47:19.947549+0000","last_became_peered":"2023-11-09T03:47:19.947549+0000","last_unstale":"2023-11-09T03:47:21.940663+0000","last_undegraded":"2023-11-09T03:47:21.940663+0000","last_fullsized":"2023-11-09T03:47:21.940663+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:51.678872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931267+0000","last_change":"2023-11-09T03:47:03.148283+0000","last_active":"2023-11-09T03:47:20.931267+0000","last_peered":"2023-11-09T03:47:20.931267+0000","last_clean":"2023-11-09T03:47:20.931267+0000","last_became_active":"2023-11-09T03:47:01.588841+0000","last_became_peered":"2023-11-09T03:47:01.588841+0000","last_unstale":"2023-11-09T03:47:20.931267+0000","last_undegraded":"2023-11-09T03:47:20.931267+0000","last_fullsized":"2023-11-09T03:47:20.931267+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:47:44.880741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000436081,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249189+0000","last_change":"2023-11-09T03:47:20.937043+0000","last_active":"2023-11-09T03:47:22.249189+0000","last_peered":"2023-11-09T03:47:22.249189+0000","last_clean":"2023-11-09T03:47:22.249189+0000","last_became_active":"2023-11-09T03:47:20.936765+0000","last_became_peered":"2023-11-09T03:47:20.936765+0000","last_unstale":"2023-11-09T03:47:22.249189+0000","last_undegraded":"2023-11-09T03:47:22.249189+0000","last_fullsized":"2023-11-09T03:47:22.249189+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:14:00.724882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249189+0000","last_change":"2023-11-09T03:47:19.946355+0000","last_active":"2023-11-09T03:47:22.249189+0000","last_peered":"2023-11-09T03:47:22.249189+0000","last_clean":"2023-11-09T03:47:22.249189+0000","last_became_active":"2023-11-09T03:47:19.945071+0000","last_became_peered":"2023-11-09T03:47:19.945071+0000","last_unstale":"2023-11-09T03:47:22.249189+0000","last_undegraded":"2023-11-09T03:47:22.249189+0000","last_fullsized":"2023-11-09T03:47:22.249189+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:07:31.918577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.937170+0000","last_change":"2023-11-09T03:47:02.618856+0000","last_active":"2023-11-09T03:47:20.937170+0000","last_peered":"2023-11-09T03:47:20.937170+0000","last_clean":"2023-11-09T03:47:20.937170+0000","last_became_active":"2023-11-09T03:47:01.590430+0000","last_became_peered":"2023-11-09T03:47:01.590430+0000","last_unstale":"2023-11-09T03:47:20.937170+0000","last_undegraded":"2023-11-09T03:47:20.937170+0000","last_fullsized":"2023-11-09T03:47:20.937170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:57:18.470305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055030200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026961+0000","last_change":"2023-11-09T03:47:20.948028+0000","last_active":"2023-11-09T03:47:22.026961+0000","last_peered":"2023-11-09T03:47:22.026961+0000","last_clean":"2023-11-09T03:47:22.026961+0000","last_became_active":"2023-11-09T03:47:20.947752+0000","last_became_peered":"2023-11-09T03:47:20.947752+0000","last_unstale":"2023-11-09T03:47:22.026961+0000","last_undegraded":"2023-11-09T03:47:22.026961+0000","last_fullsized":"2023-11-09T03:47:22.026961+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:51:53.879857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248095+0000","last_change":"2023-11-09T03:47:19.948154+0000","last_active":"2023-11-09T03:47:22.248095+0000","last_peered":"2023-11-09T03:47:22.248095+0000","last_clean":"2023-11-09T03:47:22.248095+0000","last_became_active":"2023-11-09T03:47:19.947812+0000","last_became_peered":"2023-11-09T03:47:19.947812+0000","last_unstale":"2023-11-09T03:47:22.248095+0000","last_undegraded":"2023-11-09T03:47:22.248095+0000","last_fullsized":"2023-11-09T03:47:22.248095+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:28:13.618166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.935365+0000","last_change":"2023-11-09T03:47:02.619144+0000","last_active":"2023-11-09T03:47:20.935365+0000","last_peered":"2023-11-09T03:47:20.935365+0000","last_clean":"2023-11-09T03:47:20.935365+0000","last_became_active":"2023-11-09T03:47:01.590672+0000","last_became_peered":"2023-11-09T03:47:01.590672+0000","last_unstale":"2023-11-09T03:47:20.935365+0000","last_undegraded":"2023-11-09T03:47:20.935365+0000","last_fullsized":"2023-11-09T03:47:20.935365+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:03.025119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060997500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248188+0000","last_change":"2023-11-09T03:47:20.950877+0000","last_active":"2023-11-09T03:47:22.248188+0000","last_peered":"2023-11-09T03:47:22.248188+0000","last_clean":"2023-11-09T03:47:22.248188+0000","last_became_active":"2023-11-09T03:47:20.950616+0000","last_became_peered":"2023-11-09T03:47:20.950616+0000","last_unstale":"2023-11-09T03:47:22.248188+0000","last_undegraded":"2023-11-09T03:47:22.248188+0000","last_fullsized":"2023-11-09T03:47:22.248188+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:30.970511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939086+0000","last_change":"2023-11-09T03:47:19.940824+0000","last_active":"2023-11-09T03:47:21.939086+0000","last_peered":"2023-11-09T03:47:21.939086+0000","last_clean":"2023-11-09T03:47:21.939086+0000","last_became_active":"2023-11-09T03:47:19.940553+0000","last_became_peered":"2023-11-09T03:47:19.940553+0000","last_unstale":"2023-11-09T03:47:21.939086+0000","last_undegraded":"2023-11-09T03:47:21.939086+0000","last_fullsized":"2023-11-09T03:47:21.939086+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:52:58.422763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.934932+0000","last_change":"2023-11-09T03:47:02.619079+0000","last_active":"2023-11-09T03:47:20.934932+0000","last_peered":"2023-11-09T03:47:20.934932+0000","last_clean":"2023-11-09T03:47:20.934932+0000","last_became_active":"2023-11-09T03:47:01.591421+0000","last_became_peered":"2023-11-09T03:47:01.591421+0000","last_unstale":"2023-11-09T03:47:20.934932+0000","last_undegraded":"2023-11-09T03:47:20.934932+0000","last_fullsized":"2023-11-09T03:47:20.934932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:18:58.230606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068736399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248346+0000","last_change":"2023-11-09T03:47:20.943642+0000","last_active":"2023-11-09T03:47:22.248346+0000","last_peered":"2023-11-09T03:47:22.248346+0000","last_clean":"2023-11-09T03:47:22.248346+0000","last_became_active":"2023-11-09T03:47:20.943101+0000","last_became_peered":"2023-11-09T03:47:20.943101+0000","last_unstale":"2023-11-09T03:47:22.248346+0000","last_undegraded":"2023-11-09T03:47:22.248346+0000","last_fullsized":"2023-11-09T03:47:22.248346+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:06:11.677208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248583+0000","last_change":"2023-11-09T03:47:19.943530+0000","last_active":"2023-11-09T03:47:22.248583+0000","last_peered":"2023-11-09T03:47:22.248583+0000","last_clean":"2023-11-09T03:47:22.248583+0000","last_became_active":"2023-11-09T03:47:19.943303+0000","last_became_peered":"2023-11-09T03:47:19.943303+0000","last_unstale":"2023-11-09T03:47:22.248583+0000","last_undegraded":"2023-11-09T03:47:22.248583+0000","last_fullsized":"2023-11-09T03:47:22.248583+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:35.294223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931280+0000","last_change":"2023-11-09T03:47:03.148529+0000","last_active":"2023-11-09T03:47:20.931280+0000","last_peered":"2023-11-09T03:47:20.931280+0000","last_clean":"2023-11-09T03:47:20.931280+0000","last_became_active":"2023-11-09T03:47:01.584148+0000","last_became_peered":"2023-11-09T03:47:01.584148+0000","last_unstale":"2023-11-09T03:47:20.931280+0000","last_undegraded":"2023-11-09T03:47:20.931280+0000","last_fullsized":"2023-11-09T03:47:20.931280+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:40:00.237248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061475600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.938959+0000","last_change":"2023-11-09T03:47:20.942889+0000","last_active":"2023-11-09T03:47:21.938959+0000","last_peered":"2023-11-09T03:47:21.938959+0000","last_clean":"2023-11-09T03:47:21.938959+0000","last_became_active":"2023-11-09T03:47:20.941326+0000","last_became_peered":"2023-11-09T03:47:20.941326+0000","last_unstale":"2023-11-09T03:47:21.938959+0000","last_undegraded":"2023-11-09T03:47:21.938959+0000","last_fullsized":"2023-11-09T03:47:21.938959+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:56:23.552030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.938908+0000","last_change":"2023-11-09T03:47:19.942969+0000","last_active":"2023-11-09T03:47:21.938908+0000","last_peered":"2023-11-09T03:47:21.938908+0000","last_clean":"2023-11-09T03:47:21.938908+0000","last_became_active":"2023-11-09T03:47:19.942726+0000","last_became_peered":"2023-11-09T03:47:19.942726+0000","last_unstale":"2023-11-09T03:47:21.938908+0000","last_undegraded":"2023-11-09T03:47:21.938908+0000","last_fullsized":"2023-11-09T03:47:21.938908+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:51:50.362425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.933718+0000","last_change":"2023-11-09T03:47:03.148217+0000","last_active":"2023-11-09T03:47:20.933718+0000","last_peered":"2023-11-09T03:47:20.933718+0000","last_clean":"2023-11-09T03:47:20.933718+0000","last_became_active":"2023-11-09T03:47:01.593424+0000","last_became_peered":"2023-11-09T03:47:01.593424+0000","last_unstale":"2023-11-09T03:47:20.933718+0000","last_undegraded":"2023-11-09T03:47:20.933718+0000","last_fullsized":"2023-11-09T03:47:20.933718+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:19:32.693691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001769865,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.941149+0000","last_change":"2023-11-09T03:47:20.942781+0000","last_active":"2023-11-09T03:47:21.941149+0000","last_peered":"2023-11-09T03:47:21.941149+0000","last_clean":"2023-11-09T03:47:21.941149+0000","last_became_active":"2023-11-09T03:47:20.942498+0000","last_became_peered":"2023-11-09T03:47:20.942498+0000","last_unstale":"2023-11-09T03:47:21.941149+0000","last_undegraded":"2023-11-09T03:47:21.941149+0000","last_fullsized":"2023-11-09T03:47:21.941149+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:23:50.235725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249315+0000","last_change":"2023-11-09T03:47:19.944843+0000","last_active":"2023-11-09T03:47:22.249315+0000","last_peered":"2023-11-09T03:47:22.249315+0000","last_clean":"2023-11-09T03:47:22.249315+0000","last_became_active":"2023-11-09T03:47:19.944583+0000","last_became_peered":"2023-11-09T03:47:19.944583+0000","last_unstale":"2023-11-09T03:47:22.249315+0000","last_undegraded":"2023-11-09T03:47:22.249315+0000","last_fullsized":"2023-11-09T03:47:22.249315+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:46:53.255789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931015+0000","last_change":"2023-11-09T03:47:03.146470+0000","last_active":"2023-11-09T03:47:20.931015+0000","last_peered":"2023-11-09T03:47:20.931015+0000","last_clean":"2023-11-09T03:47:20.931015+0000","last_became_active":"2023-11-09T03:47:01.587304+0000","last_became_peered":"2023-11-09T03:47:01.587304+0000","last_unstale":"2023-11-09T03:47:20.931015+0000","last_undegraded":"2023-11-09T03:47:20.931015+0000","last_fullsized":"2023-11-09T03:47:20.931015+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:58:23.339287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00110395,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.941057+0000","last_change":"2023-11-09T03:47:20.941612+0000","last_active":"2023-11-09T03:47:21.941057+0000","last_peered":"2023-11-09T03:47:21.941057+0000","last_clean":"2023-11-09T03:47:21.941057+0000","last_became_active":"2023-11-09T03:47:20.941335+0000","last_became_peered":"2023-11-09T03:47:20.941335+0000","last_unstale":"2023-11-09T03:47:21.941057+0000","last_undegraded":"2023-11-09T03:47:21.941057+0000","last_fullsized":"2023-11-09T03:47:21.941057+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:05:32.662676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939435+0000","last_change":"2023-11-09T03:47:19.941173+0000","last_active":"2023-11-09T03:47:21.939435+0000","last_peered":"2023-11-09T03:47:21.939435+0000","last_clean":"2023-11-09T03:47:21.939435+0000","last_became_active":"2023-11-09T03:47:19.940928+0000","last_became_peered":"2023-11-09T03:47:19.940928+0000","last_unstale":"2023-11-09T03:47:21.939435+0000","last_undegraded":"2023-11-09T03:47:21.939435+0000","last_fullsized":"2023-11-09T03:47:21.939435+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:28:13.618166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.934203+0000","last_change":"2023-11-09T03:47:03.147367+0000","last_active":"2023-11-09T03:47:20.934203+0000","last_peered":"2023-11-09T03:47:20.934203+0000","last_clean":"2023-11-09T03:47:20.934203+0000","last_became_active":"2023-11-09T03:47:01.588359+0000","last_became_peered":"2023-11-09T03:47:01.588359+0000","last_unstale":"2023-11-09T03:47:20.934203+0000","last_undegraded":"2023-11-09T03:47:20.934203+0000","last_fullsized":"2023-11-09T03:47:20.934203+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:21:34.303068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014106660000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249223+0000","last_change":"2023-11-09T03:47:20.950444+0000","last_active":"2023-11-09T03:47:22.249223+0000","last_peered":"2023-11-09T03:47:22.249223+0000","last_clean":"2023-11-09T03:47:22.249223+0000","last_became_active":"2023-11-09T03:47:20.950038+0000","last_became_peered":"2023-11-09T03:47:20.950038+0000","last_unstale":"2023-11-09T03:47:22.249223+0000","last_undegraded":"2023-11-09T03:47:22.249223+0000","last_fullsized":"2023-11-09T03:47:22.249223+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:55:28.077165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940688+0000","last_change":"2023-11-09T03:47:19.943935+0000","last_active":"2023-11-09T03:47:21.940688+0000","last_peered":"2023-11-09T03:47:21.940688+0000","last_clean":"2023-11-09T03:47:21.940688+0000","last_became_active":"2023-11-09T03:47:19.943125+0000","last_became_peered":"2023-11-09T03:47:19.943125+0000","last_unstale":"2023-11-09T03:47:21.940688+0000","last_undegraded":"2023-11-09T03:47:21.940688+0000","last_fullsized":"2023-11-09T03:47:21.940688+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:06:16.671366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":27,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.928179+0000","last_change":"2023-11-09T03:47:02.581682+0000","last_active":"2023-11-09T03:47:20.928179+0000","last_peered":"2023-11-09T03:47:20.928179+0000","last_clean":"2023-11-09T03:47:20.928179+0000","last_became_active":"2023-11-09T03:47:01.593051+0000","last_became_peered":"2023-11-09T03:47:01.593051+0000","last_unstale":"2023-11-09T03:47:20.928179+0000","last_undegraded":"2023-11-09T03:47:20.928179+0000","last_fullsized":"2023-11-09T03:47:20.928179+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047277799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939213+0000","last_change":"2023-11-09T03:47:20.945402+0000","last_active":"2023-11-09T03:47:21.939213+0000","last_peered":"2023-11-09T03:47:21.939213+0000","last_clean":"2023-11-09T03:47:21.939213+0000","last_became_active":"2023-11-09T03:47:20.945145+0000","last_became_peered":"2023-11-09T03:47:20.945145+0000","last_unstale":"2023-11-09T03:47:21.939213+0000","last_undegraded":"2023-11-09T03:47:21.939213+0000","last_fullsized":"2023-11-09T03:47:21.939213+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:38:09.677347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248640+0000","last_change":"2023-11-09T03:47:19.942655+0000","last_active":"2023-11-09T03:47:22.248640+0000","last_peered":"2023-11-09T03:47:22.248640+0000","last_clean":"2023-11-09T03:47:22.248640+0000","last_became_active":"2023-11-09T03:47:19.942380+0000","last_became_peered":"2023-11-09T03:47:19.942380+0000","last_unstale":"2023-11-09T03:47:22.248640+0000","last_undegraded":"2023-11-09T03:47:22.248640+0000","last_fullsized":"2023-11-09T03:47:22.248640+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:52:58.422763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.933457+0000","last_change":"2023-11-09T03:47:03.146309+0000","last_active":"2023-11-09T03:47:20.933457+0000","last_peered":"2023-11-09T03:47:20.933457+0000","last_clean":"2023-11-09T03:47:20.933457+0000","last_became_active":"2023-11-09T03:47:01.587560+0000","last_became_peered":"2023-11-09T03:47:01.587560+0000","last_unstale":"2023-11-09T03:47:20.933457+0000","last_undegraded":"2023-11-09T03:47:20.933457+0000","last_fullsized":"2023-11-09T03:47:20.933457+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:27:55.271945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070988400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249299+0000","last_change":"2023-11-09T03:47:20.941432+0000","last_active":"2023-11-09T03:47:22.249299+0000","last_peered":"2023-11-09T03:47:22.249299+0000","last_clean":"2023-11-09T03:47:22.249299+0000","last_became_active":"2023-11-09T03:47:20.941186+0000","last_became_peered":"2023-11-09T03:47:20.941186+0000","last_unstale":"2023-11-09T03:47:22.249299+0000","last_undegraded":"2023-11-09T03:47:22.249299+0000","last_fullsized":"2023-11-09T03:47:22.249299+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:25:38.423194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939102+0000","last_change":"2023-11-09T03:47:19.947334+0000","last_active":"2023-11-09T03:47:21.939102+0000","last_peered":"2023-11-09T03:47:21.939102+0000","last_clean":"2023-11-09T03:47:21.939102+0000","last_became_active":"2023-11-09T03:47:19.947091+0000","last_became_peered":"2023-11-09T03:47:19.947091+0000","last_unstale":"2023-11-09T03:47:21.939102+0000","last_undegraded":"2023-11-09T03:47:21.939102+0000","last_fullsized":"2023-11-09T03:47:21.939102+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:24:06.867382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.939549+0000","last_change":"2023-11-09T03:47:02.725125+0000","last_active":"2023-11-09T03:47:20.939549+0000","last_peered":"2023-11-09T03:47:20.939549+0000","last_clean":"2023-11-09T03:47:20.939549+0000","last_became_active":"2023-11-09T03:47:01.591521+0000","last_became_peered":"2023-11-09T03:47:01.591521+0000","last_unstale":"2023-11-09T03:47:20.939549+0000","last_undegraded":"2023-11-09T03:47:20.939549+0000","last_fullsized":"2023-11-09T03:47:20.939549+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:52:40.076542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000758675,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940341+0000","last_change":"2023-11-09T03:47:20.938307+0000","last_active":"2023-11-09T03:47:21.940341+0000","last_peered":"2023-11-09T03:47:21.940341+0000","last_clean":"2023-11-09T03:47:21.940341+0000","last_became_active":"2023-11-09T03:47:20.937853+0000","last_became_peered":"2023-11-09T03:47:20.937853+0000","last_unstale":"2023-11-09T03:47:21.940341+0000","last_undegraded":"2023-11-09T03:47:21.940341+0000","last_fullsized":"2023-11-09T03:47:21.940341+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:57.187676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248805+0000","last_change":"2023-11-09T03:47:19.945739+0000","last_active":"2023-11-09T03:47:22.248805+0000","last_peered":"2023-11-09T03:47:22.248805+0000","last_clean":"2023-11-09T03:47:22.248805+0000","last_became_active":"2023-11-09T03:47:19.944990+0000","last_became_peered":"2023-11-09T03:47:19.944990+0000","last_unstale":"2023-11-09T03:47:22.248805+0000","last_undegraded":"2023-11-09T03:47:22.248805+0000","last_fullsized":"2023-11-09T03:47:22.248805+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:53:27.482165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.932096+0000","last_change":"2023-11-09T03:47:03.147289+0000","last_active":"2023-11-09T03:47:20.932096+0000","last_peered":"2023-11-09T03:47:20.932096+0000","last_clean":"2023-11-09T03:47:20.932096+0000","last_became_active":"2023-11-09T03:47:01.587784+0000","last_became_peered":"2023-11-09T03:47:01.587784+0000","last_unstale":"2023-11-09T03:47:20.932096+0000","last_undegraded":"2023-11-09T03:47:20.932096+0000","last_fullsized":"2023-11-09T03:47:20.932096+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:11:23.824112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038781299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247714+0000","last_change":"2023-11-09T03:47:20.937750+0000","last_active":"2023-11-09T03:47:22.247714+0000","last_peered":"2023-11-09T03:47:22.247714+0000","last_clean":"2023-11-09T03:47:22.247714+0000","last_became_active":"2023-11-09T03:47:20.937473+0000","last_became_peered":"2023-11-09T03:47:20.937473+0000","last_unstale":"2023-11-09T03:47:22.247714+0000","last_undegraded":"2023-11-09T03:47:22.247714+0000","last_fullsized":"2023-11-09T03:47:22.247714+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:06:17.676049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940071+0000","last_change":"2023-11-09T03:47:19.948440+0000","last_active":"2023-11-09T03:47:21.940071+0000","last_peered":"2023-11-09T03:47:21.940071+0000","last_clean":"2023-11-09T03:47:21.940071+0000","last_became_active":"2023-11-09T03:47:19.948199+0000","last_became_peered":"2023-11-09T03:47:19.948199+0000","last_unstale":"2023-11-09T03:47:21.940071+0000","last_undegraded":"2023-11-09T03:47:21.940071+0000","last_fullsized":"2023-11-09T03:47:21.940071+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:34:52.006539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.935047+0000","last_change":"2023-11-09T03:47:02.619346+0000","last_active":"2023-11-09T03:47:20.935047+0000","last_peered":"2023-11-09T03:47:20.935047+0000","last_clean":"2023-11-09T03:47:20.935047+0000","last_became_active":"2023-11-09T03:47:01.590389+0000","last_became_peered":"2023-11-09T03:47:01.590389+0000","last_unstale":"2023-11-09T03:47:20.935047+0000","last_undegraded":"2023-11-09T03:47:20.935047+0000","last_fullsized":"2023-11-09T03:47:20.935047+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000384072,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.941022+0000","last_change":"2023-11-09T03:47:20.937215+0000","last_active":"2023-11-09T03:47:21.941022+0000","last_peered":"2023-11-09T03:47:21.941022+0000","last_clean":"2023-11-09T03:47:21.941022+0000","last_became_active":"2023-11-09T03:47:20.936913+0000","last_became_peered":"2023-11-09T03:47:20.936913+0000","last_unstale":"2023-11-09T03:47:21.941022+0000","last_undegraded":"2023-11-09T03:47:21.941022+0000","last_fullsized":"2023-11-09T03:47:21.941022+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:25:38.423194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249234+0000","last_change":"2023-11-09T03:47:19.948214+0000","last_active":"2023-11-09T03:47:22.249234+0000","last_peered":"2023-11-09T03:47:22.249234+0000","last_clean":"2023-11-09T03:47:22.249234+0000","last_became_active":"2023-11-09T03:47:19.947966+0000","last_became_peered":"2023-11-09T03:47:19.947966+0000","last_unstale":"2023-11-09T03:47:22.249234+0000","last_undegraded":"2023-11-09T03:47:22.249234+0000","last_fullsized":"2023-11-09T03:47:22.249234+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:53:19.322920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.938897+0000","last_change":"2023-11-09T03:47:02.582393+0000","last_active":"2023-11-09T03:47:20.938897+0000","last_peered":"2023-11-09T03:47:20.938897+0000","last_clean":"2023-11-09T03:47:20.938897+0000","last_became_active":"2023-11-09T03:47:01.591752+0000","last_became_peered":"2023-11-09T03:47:01.591752+0000","last_unstale":"2023-11-09T03:47:20.938897+0000","last_undegraded":"2023-11-09T03:47:20.938897+0000","last_fullsized":"2023-11-09T03:47:20.938897+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:03.025119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067021599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.027114+0000","last_change":"2023-11-09T03:47:20.946342+0000","last_active":"2023-11-09T03:47:22.027114+0000","last_peered":"2023-11-09T03:47:22.027114+0000","last_clean":"2023-11-09T03:47:22.027114+0000","last_became_active":"2023-11-09T03:47:20.946014+0000","last_became_peered":"2023-11-09T03:47:20.946014+0000","last_unstale":"2023-11-09T03:47:22.027114+0000","last_undegraded":"2023-11-09T03:47:22.027114+0000","last_fullsized":"2023-11-09T03:47:22.027114+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:25:38.423194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"53'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248629+0000","last_change":"2023-11-09T03:47:19.944449+0000","last_active":"2023-11-09T03:47:22.248629+0000","last_peered":"2023-11-09T03:47:22.248629+0000","last_clean":"2023-11-09T03:47:22.248629+0000","last_became_active":"2023-11-09T03:47:19.944203+0000","last_became_peered":"2023-11-09T03:47:19.944203+0000","last_unstale":"2023-11-09T03:47:22.248629+0000","last_undegraded":"2023-11-09T03:47:22.248629+0000","last_fullsized":"2023-11-09T03:47:22.248629+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:51.678872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.929927+0000","last_change":"2023-11-09T03:47:03.028744+0000","last_active":"2023-11-09T03:47:20.929927+0000","last_peered":"2023-11-09T03:47:20.929927+0000","last_clean":"2023-11-09T03:47:20.929927+0000","last_became_active":"2023-11-09T03:47:01.593890+0000","last_became_peered":"2023-11-09T03:47:01.593890+0000","last_unstale":"2023-11-09T03:47:20.929927+0000","last_undegraded":"2023-11-09T03:47:20.929927+0000","last_fullsized":"2023-11-09T03:47:20.929927+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:05.754357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053008299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026854+0000","last_change":"2023-11-09T03:47:20.952365+0000","last_active":"2023-11-09T03:47:22.026854+0000","last_peered":"2023-11-09T03:47:22.026854+0000","last_clean":"2023-11-09T03:47:22.026854+0000","last_became_active":"2023-11-09T03:47:20.952080+0000","last_became_peered":"2023-11-09T03:47:20.952080+0000","last_unstale":"2023-11-09T03:47:22.026854+0000","last_undegraded":"2023-11-09T03:47:22.026854+0000","last_fullsized":"2023-11-09T03:47:22.026854+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:40.639821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249192+0000","last_change":"2023-11-09T03:47:19.946712+0000","last_active":"2023-11-09T03:47:22.249192+0000","last_peered":"2023-11-09T03:47:22.249192+0000","last_clean":"2023-11-09T03:47:22.249192+0000","last_became_active":"2023-11-09T03:47:19.946456+0000","last_became_peered":"2023-11-09T03:47:19.946456+0000","last_unstale":"2023-11-09T03:47:22.249192+0000","last_undegraded":"2023-11-09T03:47:22.249192+0000","last_fullsized":"2023-11-09T03:47:22.249192+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:39:07.929390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.936277+0000","last_change":"2023-11-09T03:47:03.146783+0000","last_active":"2023-11-09T03:47:20.936277+0000","last_peered":"2023-11-09T03:47:20.936277+0000","last_clean":"2023-11-09T03:47:20.936277+0000","last_became_active":"2023-11-09T03:47:01.586923+0000","last_became_peered":"2023-11-09T03:47:01.586923+0000","last_unstale":"2023-11-09T03:47:20.936277+0000","last_undegraded":"2023-11-09T03:47:20.936277+0000","last_fullsized":"2023-11-09T03:47:20.936277+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:37:50.326442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060384700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248492+0000","last_change":"2023-11-09T03:47:20.946978+0000","last_active":"2023-11-09T03:47:22.248492+0000","last_peered":"2023-11-09T03:47:22.248492+0000","last_clean":"2023-11-09T03:47:22.248492+0000","last_became_active":"2023-11-09T03:47:20.946667+0000","last_became_peered":"2023-11-09T03:47:20.946667+0000","last_unstale":"2023-11-09T03:47:22.248492+0000","last_undegraded":"2023-11-09T03:47:22.248492+0000","last_fullsized":"2023-11-09T03:47:22.248492+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:02:01.425453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"53'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940910+0000","last_change":"2023-11-09T03:47:19.945244+0000","last_active":"2023-11-09T03:47:21.940910+0000","last_peered":"2023-11-09T03:47:21.940910+0000","last_clean":"2023-11-09T03:47:21.940910+0000","last_became_active":"2023-11-09T03:47:19.944955+0000","last_became_peered":"2023-11-09T03:47:19.944955+0000","last_unstale":"2023-11-09T03:47:21.940910+0000","last_undegraded":"2023-11-09T03:47:21.940910+0000","last_fullsized":"2023-11-09T03:47:21.940910+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:12:47.685713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.933539+0000","last_change":"2023-11-09T03:47:03.146580+0000","last_active":"2023-11-09T03:47:20.933539+0000","last_peered":"2023-11-09T03:47:20.933539+0000","last_clean":"2023-11-09T03:47:20.933539+0000","last_became_active":"2023-11-09T03:47:01.589201+0000","last_became_peered":"2023-11-09T03:47:01.589201+0000","last_unstale":"2023-11-09T03:47:20.933539+0000","last_undegraded":"2023-11-09T03:47:20.933539+0000","last_fullsized":"2023-11-09T03:47:20.933539+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:14.300421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049706999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248489+0000","last_change":"2023-11-09T03:47:20.949430+0000","last_active":"2023-11-09T03:47:22.248489+0000","last_peered":"2023-11-09T03:47:22.248489+0000","last_clean":"2023-11-09T03:47:22.248489+0000","last_became_active":"2023-11-09T03:47:20.949177+0000","last_became_peered":"2023-11-09T03:47:20.949177+0000","last_unstale":"2023-11-09T03:47:22.248489+0000","last_undegraded":"2023-11-09T03:47:22.248489+0000","last_fullsized":"2023-11-09T03:47:22.248489+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:04:05.315163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248363+0000","last_change":"2023-11-09T03:47:19.946526+0000","last_active":"2023-11-09T03:47:22.248363+0000","last_peered":"2023-11-09T03:47:22.248363+0000","last_clean":"2023-11-09T03:47:22.248363+0000","last_became_active":"2023-11-09T03:47:19.946263+0000","last_became_peered":"2023-11-09T03:47:19.946263+0000","last_unstale":"2023-11-09T03:47:22.248363+0000","last_undegraded":"2023-11-09T03:47:22.248363+0000","last_fullsized":"2023-11-09T03:47:22.248363+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:24:06.867382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931584+0000","last_change":"2023-11-09T03:47:03.147851+0000","last_active":"2023-11-09T03:47:20.931584+0000","last_peered":"2023-11-09T03:47:20.931584+0000","last_clean":"2023-11-09T03:47:20.931584+0000","last_became_active":"2023-11-09T03:47:01.589893+0000","last_became_peered":"2023-11-09T03:47:01.589893+0000","last_unstale":"2023-11-09T03:47:20.931584+0000","last_undegraded":"2023-11-09T03:47:20.931584+0000","last_fullsized":"2023-11-09T03:47:20.931584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:08:17.183462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029151399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":194,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026619+0000","last_change":"2023-11-09T03:45:21.030926+0000","last_active":"2023-11-09T03:47:22.026619+0000","last_peered":"2023-11-09T03:47:22.026619+0000","last_clean":"2023-11-09T03:47:22.026619+0000","last_became_active":"2023-11-09T03:45:21.029801+0000","last_became_peered":"2023-11-09T03:45:21.029801+0000","last_unstale":"2023-11-09T03:47:22.026619+0000","last_undegraded":"2023-11-09T03:47:22.026619+0000","last_fullsized":"2023-11-09T03:47:22.026619+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_clean_scrub_stamp":"2023-11-09T03:45:19.768433+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:12:59.153713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28240,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343600,"statfs":{"total":96636764160,"available":96607846400,"internally_reserved":0,"allocated":1245184,"data_stored":881539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.501,"15min":0.498},"min":{"1min":0.34899999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.622},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.41199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.71499999999999997}]},{"osd":1,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.33000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.38100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.73099999999999998}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.52300000000000002,"15min":0.51400000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.68700000000000006,"15min":0.70799999999999996},"min":{"1min":0.41299999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.80700000000000005,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.63500000000000001}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.48699999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.68999999999999995}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.441},"min":{"1min":0.34100000000000003,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.49199999999999999,"15min":0.47899999999999998},"min":{"1min":0.38700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.38700000000000001}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.47199999999999998,"15min":0.45600000000000002},"min":{"1min":0.37,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.51100000000000001,"15min":0.497},"min":{"1min":0.39600000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.58699999999999997}]},{"osd":6,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55100000000000005,"15min":0.54100000000000004},"min":{"1min":0.44400000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.434,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.44}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":643072,"data_stored":291011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.499,"15min":0.501},"min":{"1min":0.35099999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63900000000000001,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.50600000000000001,"15min":0.495},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.69899999999999995}]},{"osd":1,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.495}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.375,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.53400000000000003}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.58899999999999997}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.503,"15min":0.503},"min":{"1min":0.316,"5min":0.313,"15min":0.313},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.68899999999999995}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.436,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55500000000000005,"15min":0.55800000000000005},"min":{"1min":0.34300000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.56699999999999995}]},{"osd":7,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.42899999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.44400000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.79400000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":651264,"data_stored":291663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.496,"15min":0.48599999999999999},"min":{"1min":0.38700000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.876},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.503,"15min":0.498},"min":{"1min":0.42399999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.71799999999999997,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45400000000000001}]},{"osd":1,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.439,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.69999999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.62},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.46700000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.76200000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.64100000000000001}]},{"osd":2,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.33300000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.67900000000000005,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.499,"5min":0.371,"15min":0.371},"max":{"1min":0.72799999999999998,"5min":0.745,"15min":0.745},"last":0.71799999999999997}]},{"osd":3,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.39700000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.76500000000000001,"5min":0.88,"15min":0.88},"last":0.76400000000000001}]},{"osd":4,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.78100000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.495},{"interface":"front","average":{"1min":0.504,"5min":0.48399999999999999,"15min":0.48099999999999998},"min":{"1min":0.33100000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.79000000000000004}]},{"osd":6,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.50900000000000001,"15min":0.51100000000000001},"min":{"1min":0.33400000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.74199999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51300000000000001,"15min":0.51100000000000001},"min":{"1min":0.35499999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66900000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.753}]},{"osd":7,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.38900000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.73499999999999999,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61899999999999999,"15min":0.62},"min":{"1min":0.41199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.752,"5min":1.01,"15min":1.01},"last":0.84899999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":643072,"data_stored":285650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67600000000000005,"5min":1.034,"15min":1.034},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.53600000000000003,"15min":0.54000000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68300000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.46899999999999997}]},{"osd":1,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.56899999999999995,"15min":0.57699999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68100000000000005,"5min":1.093,"15min":1.093},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.36899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72399999999999998,"5min":0.871,"15min":0.871},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76800000000000002,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70099999999999996,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.44700000000000001}]},{"osd":3,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71699999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.502},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54900000000000004,"15min":0.55200000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86599999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.433}]},{"osd":5,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54400000000000004,"15min":0.54300000000000004},"min":{"1min":0.37,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76000000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.33100000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.36899999999999999}]},{"osd":6,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.33800000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.51700000000000002}]},{"osd":7,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.62,"15min":0.63},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77700000000000002,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.95599999999999996,"5min":1.113,"15min":1.113},"last":0.54300000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":651264,"data_stored":291067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.5,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.215},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51100000000000001,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.23300000000000001}]},{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32900000000000001,"15min":0.33100000000000002},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.54200000000000004,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.19800000000000001},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.371,"15min":0.375},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.53100000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.26700000000000002}]},{"osd":2,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.27200000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.59099999999999997,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.56100000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.31900000000000001}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.70899999999999996,"5min":0.87,"15min":0.87},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60999999999999999,"15min":0.61599999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.75600000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.62,"15min":0.625},"min":{"1min":0.45700000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79900000000000004,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61699999999999999,"15min":0.623},"min":{"1min":0.42599999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.74299999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.621,"15min":0.627},"min":{"1min":0.42399999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81200000000000006,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59999999999999998,"15min":0.60399999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76600000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.66000000000000003}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77200000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63500000000000001,"15min":0.64200000000000002},"min":{"1min":0.433,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81299999999999994,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.58299999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":651264,"data_stored":291067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.46000000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.33400000000000002,"15min":0.33100000000000002},"min":{"1min":0.247,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.27600000000000002}]},{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32500000000000001,"15min":0.32600000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.495,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.249},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.35899999999999999,"15min":0.36299999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.48599999999999999,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.33200000000000002}]},{"osd":3,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.36199999999999999,"15min":0.36399999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.45700000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40200000000000002,"15min":0.40400000000000003},"min":{"1min":0.30299999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.52300000000000002,"5min":0.754,"15min":0.754},"last":0.35299999999999998}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72299999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81299999999999994,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.50900000000000001}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.39800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.75700000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.59599999999999997,"15min":0.60099999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.753,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.498}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60399999999999998,"15min":0.60899999999999999},"min":{"1min":0.44600000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.76100000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.71199999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.57699999999999996}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60699999999999998,"15min":0.61199999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.745,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.80000000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":647168,"data_stored":291101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.28699999999999998,"5min":0.31,"15min":0.314},"min":{"1min":0.20799999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.38900000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.315,"15min":0.311},"min":{"1min":0.24099999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.43099999999999999}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.311,"15min":0.309},"min":{"1min":0.253,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.378,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.34599999999999997}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36899999999999999,"15min":0.371},"min":{"1min":0.23899999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.47699999999999998,"5min":0.505,"15min":0.505},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.37,"15min":0.371},"min":{"1min":0.30299999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.45400000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.36599999999999999}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.41899999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.747,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.44400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.77600000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.62}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56200000000000006,"15min":0.55700000000000005},"min":{"1min":0.46100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.44500000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.72599999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.60699999999999998}]},{"osd":6,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.47799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56699999999999995,"15min":0.56799999999999995},"min":{"1min":0.433,"5min":0.379,"15min":0.379},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.70399999999999996}]},{"osd":7,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.42899999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.69799999999999995,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.72899999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.63400000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28228,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343612,"statfs":{"total":96636764160,"available":96607858688,"internally_reserved":0,"allocated":1236992,"data_stored":875981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.31900000000000001,"15min":0.32300000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.38200000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.29599999999999999,"5min":0.35499999999999998,"15min":0.36499999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.44500000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.28799999999999998}]},{"osd":2,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.35699999999999998,"15min":0.36399999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.44700000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.36699999999999999,"15min":0.371},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.47699999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.251}]},{"osd":3,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.38500000000000001,"15min":0.39100000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.46400000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.41499999999999998,"15min":0.42099999999999999},"min":{"1min":0.309,"5min":0.307,"15min":0.307},"max":{"1min":0.48999999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.379}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58899999999999997,"15min":0.59299999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60599999999999998,"15min":0.60899999999999999},"min":{"1min":0.436,"5min":0.318,"15min":0.318},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.621}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.36199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76800000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.629,"15min":0.63600000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78900000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.52100000000000002}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.64100000000000001,"15min":0.64700000000000002},"min":{"1min":0.42499999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81000000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.627,"15min":0.63200000000000001},"min":{"1min":0.41699999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59699999999999998}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.44400000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.42999999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.71799999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.67000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-09T03:47:25.028 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-09T03:47:25.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.160+0000 7f04eb24f640 1 Processor -- start 2023-11-09T03:47:25.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.160+0000 7f04eb24f640 1 -- start start 2023-11-09T03:47:25.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.161+0000 7f04eb24f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04e415ad80 0x7f04e406c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:25.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.161+0000 7f04eb24f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f04e40c9e10 0x7f04e40b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:25.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.161+0000 7f04eb24f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04e406cd00 con 0x7f04e40b42c0 2023-11-09T03:47:25.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.161+0000 7f04eb24f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04e406cfa0 con 0x7f04e40c9e10 2023-11-09T03:47:25.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.161+0000 7f04eb24f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04e406d280 con 0x7f04e415ad80 2023-11-09T03:47:25.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.161+0000 7f04e97ec640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f04e40c9e10 0x7f04e40b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:25.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.161+0000 7f04e3fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f04e40b42c0 0x7f04e40c9700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48004/0 (socket says 172.21.15.76:48004) 2023-11-09T03:47:25.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.161+0000 7f04e3fff640 1 -- 172.21.15.76:0/3883339643 learned_addr learned my addr 172.21.15.76:0/3883339643 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:25.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.162+0000 7f04e8feb640 1 --2- 172.21.15.76:0/3883339643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04e415ad80 0x7f04e406c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:25.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.162+0000 7f04e97ec640 1 -- 172.21.15.76:0/3883339643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04e415ad80 msgr2=0x7f04e406c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:25.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.162+0000 7f04e97ec640 1 --2- 172.21.15.76:0/3883339643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04e415ad80 0x7f04e406c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:25.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.162+0000 7f04e97ec640 1 -- 172.21.15.76:0/3883339643 >> v1:172.21.15.76:6789/0 conn(0x7f04e40b42c0 legacy=0x7f04e40c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:25.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.162+0000 7f04e37fe640 1 -- 172.21.15.76:0/3883339643 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2730088418 0 0) 0x7f04e406cd00 con 0x7f04e40b42c0 2023-11-09T03:47:25.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.162+0000 7f04e97ec640 1 -- 172.21.15.76:0/3883339643 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e406d560 con 0x7f04e40c9e10 2023-11-09T03:47:25.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.162+0000 7f04e97ec640 1 --2- 172.21.15.76:0/3883339643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04e40c9e10 0x7f04e40b1a40 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f04dc009d90 tx=0x7f04dc02bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3583fad60182be0d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:25.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.163+0000 7f04e37fe640 1 -- 172.21.15.76:0/3883339643 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04dc02cce0 con 0x7f04e40c9e10 2023-11-09T03:47:25.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.163+0000 7f04e37fe640 1 -- 172.21.15.76:0/3883339643 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04dc02ce40 con 0x7f04e40c9e10 2023-11-09T03:47:25.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.163+0000 7f04e37fe640 1 -- 172.21.15.76:0/3883339643 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04dc034660 con 0x7f04e40c9e10 2023-11-09T03:47:25.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.163+0000 7f04eb24f640 1 -- 172.21.15.76:0/3883339643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04e40c9e10 msgr2=0x7f04e40b1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:25.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.163+0000 7f04eb24f640 1 --2- 172.21.15.76:0/3883339643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04e40c9e10 0x7f04e40b1a40 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f04dc009d90 tx=0x7f04dc02bf30 comp rx=0 tx=0).stop 2023-11-09T03:47:25.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.163+0000 7f04eb24f640 1 -- 172.21.15.76:0/3883339643 shutdown_connections 2023-11-09T03:47:25.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.163+0000 7f04eb24f640 1 --2- 172.21.15.76:0/3883339643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04e40c9e10 0x7f04e40b1a40 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:25.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.163+0000 7f04eb24f640 1 --2- 172.21.15.76:0/3883339643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04e415ad80 0x7f04e406c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:25.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.163+0000 7f04eb24f640 1 -- 172.21.15.76:0/3883339643 >> 172.21.15.76:0/3883339643 conn(0x7f04e40ce950 msgr2=0x7f04e40ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:25.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.163+0000 7f04eb24f640 1 -- 172.21.15.76:0/3883339643 shutdown_connections 2023-11-09T03:47:25.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.163+0000 7f04eb24f640 1 -- 172.21.15.76:0/3883339643 wait complete. 2023-11-09T03:47:25.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.164+0000 7f04eb24f640 1 Processor -- start 2023-11-09T03:47:25.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.164+0000 7f04eb24f640 1 -- start start 2023-11-09T03:47:25.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.165+0000 7f04eb24f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04e40b42c0 0x7f04e4153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:25.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.165+0000 7f04eb24f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04e40c9e10 0x7f04e4156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:25.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.165+0000 7f04eb24f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04e415ad80 0x7f04e4154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:25.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.165+0000 7f04eb24f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e4153fa0 con 0x7f04e40c9e10 2023-11-09T03:47:25.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.165+0000 7f04eb24f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e4155290 con 0x7f04e415ad80 2023-11-09T03:47:25.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.165+0000 7f04eb24f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04e4155570 con 0x7f04e40b42c0 2023-11-09T03:47:25.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.165+0000 7f04e3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04e40c9e10 0x7f04e4156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:25.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.165+0000 7f04e3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04e40c9e10 0x7f04e4156330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42572/0 (socket says 172.21.15.76:42572) 2023-11-09T03:47:25.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.165+0000 7f04e3fff640 1 -- 172.21.15.76:0/256278604 learned_addr learned my addr 172.21.15.76:0/256278604 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:25.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.166+0000 7f04e97ec640 1 --2- 172.21.15.76:0/256278604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04e415ad80 0x7f04e4154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:25.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.166+0000 7f04e8feb640 1 --2- 172.21.15.76:0/256278604 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04e40b42c0 0x7f04e4153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:25.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.166+0000 7f04e3fff640 1 -- 172.21.15.76:0/256278604 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04e40b42c0 msgr2=0x7f04e4153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:25.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.166+0000 7f04e3fff640 1 --2- 172.21.15.76:0/256278604 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04e40b42c0 0x7f04e4153a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:25.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.166+0000 7f04e3fff640 1 -- 172.21.15.76:0/256278604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04e415ad80 msgr2=0x7f04e4154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:25.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.166+0000 7f04e3fff640 1 --2- 172.21.15.76:0/256278604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04e415ad80 0x7f04e4154a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:25.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.166+0000 7f04e3fff640 1 -- 172.21.15.76:0/256278604 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04dc009a70 con 0x7f04e40c9e10 2023-11-09T03:47:25.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.166+0000 7f04e8feb640 1 --2- 172.21.15.76:0/256278604 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04e40b42c0 0x7f04e4153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:25.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.166+0000 7f04e97ec640 1 --2- 172.21.15.76:0/256278604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04e415ad80 0x7f04e4154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:25.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.166+0000 7f04e3fff640 1 --2- 172.21.15.76:0/256278604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04e40c9e10 0x7f04e4156330 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f04d000c4a0 tx=0x7f04d000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:25.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.166+0000 7f04e17fa640 1 -- 172.21.15.76:0/256278604 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d0005ce0 con 0x7f04e40c9e10 2023-11-09T03:47:25.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.166+0000 7f04e17fa640 1 -- 172.21.15.76:0/256278604 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d0005e40 con 0x7f04e40c9e10 2023-11-09T03:47:25.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.166+0000 7f04eb24f640 1 -- 172.21.15.76:0/256278604 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04e4155850 con 0x7f04e40c9e10 2023-11-09T03:47:25.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.166+0000 7f04e17fa640 1 -- 172.21.15.76:0/256278604 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d00072b0 con 0x7f04e40c9e10 2023-11-09T03:47:25.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.166+0000 7f04eb24f640 1 -- 172.21.15.76:0/256278604 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e4155e10 con 0x7f04e40c9e10 2023-11-09T03:47:25.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.167+0000 7f04eb24f640 1 -- 172.21.15.76:0/256278604 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04e4156000 con 0x7f04e40c9e10 2023-11-09T03:47:25.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.167+0000 7f04e17fa640 1 -- 172.21.15.76:0/256278604 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f04d0007410 con 0x7f04e40c9e10 2023-11-09T03:47:25.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.168+0000 7f04e17fa640 1 --2- 172.21.15.76:0/256278604 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f04c806aea0 0x7f04c806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:25.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.168+0000 7f04e17fa640 1 -- 172.21.15.76:0/256278604 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f04d0088e00 con 0x7f04e40c9e10 2023-11-09T03:47:25.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.168+0000 7f04e8feb640 1 --2- 172.21.15.76:0/256278604 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f04c806aea0 0x7f04c806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:25.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.168+0000 7f04e8feb640 1 --2- 172.21.15.76:0/256278604 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f04c806aea0 0x7f04c806d360 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f04d8006fd0 tx=0x7f04d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:25.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.172+0000 7f04e17fa640 1 -- 172.21.15.76:0/256278604 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f04d0053c10 con 0x7f04e40c9e10 2023-11-09T03:47:25.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.401+0000 7f04eb24f640 1 -- 172.21.15.76:0/256278604 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f04e4074290 con 0x7f04e40c9e10 2023-11-09T03:47:25.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.402+0000 7f04e17fa640 1 -- 172.21.15.76:0/256278604 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1235 (secure 0 0 0) 0x7f04d0017080 con 0x7f04e40c9e10 2023-11-09T03:47:25.401 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:25.401 INFO:teuthology.orchestra.run.smithi076.stdout:{"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","health":{"status":"HEALTH_WARN","checks":{"POOL_APP_NOT_ENABLED":{"severity":"HEALTH_WARN","summary":{"message":"1 pool(s) do not have an application enabled","count":1},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":138,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":54,"num_osds":8,"num_up_osds":8,"osd_up_since":1699501517,"num_in_osds":8,"osd_in_since":1699501508,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":123},{"state_name":"unknown","count":6}],"num_pgs":129,"num_pools":4,"num_objects":23,"data_bytes":592778,"bytes_used":227700736,"bytes_avail":772866412544,"bytes_total":773094113280,"unknown_pgs_ratio":0.046511627733707428,"write_bytes_sec":3583,"read_op_per_sec":0,"write_op_per_sec":10},"fsmap":{"epoch":23,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":6075}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2023-11-09T03:46:36.765927+0000","services":{}},"progress_events":{}} 2023-11-09T03:47:25.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.403+0000 7f04eb24f640 1 -- 172.21.15.76:0/256278604 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f04c806aea0 msgr2=0x7f04c806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:25.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.403+0000 7f04eb24f640 1 --2- 172.21.15.76:0/256278604 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f04c806aea0 0x7f04c806d360 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f04d8006fd0 tx=0x7f04d8008040 comp rx=0 tx=0).stop 2023-11-09T03:47:25.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.403+0000 7f04eb24f640 1 -- 172.21.15.76:0/256278604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04e40c9e10 msgr2=0x7f04e4156330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:25.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.403+0000 7f04eb24f640 1 --2- 172.21.15.76:0/256278604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04e40c9e10 0x7f04e4156330 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f04d000c4a0 tx=0x7f04d000c960 comp rx=0 tx=0).stop 2023-11-09T03:47:25.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.404+0000 7f04eb24f640 1 -- 172.21.15.76:0/256278604 shutdown_connections 2023-11-09T03:47:25.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.404+0000 7f04eb24f640 1 --2- 172.21.15.76:0/256278604 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f04c806aea0 0x7f04c806d360 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:25.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.404+0000 7f04eb24f640 1 --2- 172.21.15.76:0/256278604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04e415ad80 0x7f04e4154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:25.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.404+0000 7f04eb24f640 1 --2- 172.21.15.76:0/256278604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04e40c9e10 0x7f04e4156330 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:25.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.404+0000 7f04eb24f640 1 --2- 172.21.15.76:0/256278604 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04e40b42c0 0x7f04e4153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:25.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.404+0000 7f04eb24f640 1 -- 172.21.15.76:0/256278604 >> 172.21.15.76:0/256278604 conn(0x7f04e40ce950 msgr2=0x7f04e406c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:25.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.404+0000 7f04eb24f640 1 -- 172.21.15.76:0/256278604 shutdown_connections 2023-11-09T03:47:25.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.404+0000 7f04eb24f640 1 -- 172.21.15.76:0/256278604 wait complete. 2023-11-09T03:47:25.414 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-09T03:47:25.414 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-09T03:47:25.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.515+0000 7fbbf73e2640 1 Processor -- start 2023-11-09T03:47:25.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.515+0000 7fbbf73e2640 1 -- start start 2023-11-09T03:47:25.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.516+0000 7fbbf73e2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbf00cda60 0x7fbbf00ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:25.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.516+0000 7fbbf73e2640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf00cc990 con 0x7fbbf0070d40 2023-11-09T03:47:25.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.516+0000 7fbbf73e2640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf00ccc70 con 0x7fbbf00b0cd0 2023-11-09T03:47:25.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.516+0000 7fbbf73e2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf00c9ce0 con 0x7fbbf00cda60 2023-11-09T03:47:25.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.516+0000 7fbbf597f640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fbbf0070d40 0x7fbbf006c200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48018/0 (socket says 172.21.15.76:48018) 2023-11-09T03:47:25.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.516+0000 7fbbf597f640 1 -- 172.21.15.76:0/1757853402 learned_addr learned my addr 172.21.15.76:0/1757853402 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:25.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.516+0000 7fbbf517e640 1 --2- 172.21.15.76:0/1757853402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbf00cda60 0x7fbbf00ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:25.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.516+0000 7fbbe7fff640 1 -- 172.21.15.76:0/1757853402 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 137230434 0 0) 0x7fbbf00cc990 con 0x7fbbf0070d40 2023-11-09T03:47:25.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.516+0000 7fbbe7fff640 1 -- 172.21.15.76:0/1757853402 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbdc003520 con 0x7fbbf0070d40 2023-11-09T03:47:25.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.516+0000 7fbbe7fff640 1 -- 172.21.15.76:0/1757853402 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2199964220 0 0) 0x7fbbdc003520 con 0x7fbbf0070d40 2023-11-09T03:47:25.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.516+0000 7fbbe7fff640 1 -- 172.21.15.76:0/1757853402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbf00cda60 msgr2=0x7fbbf00ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:25.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.516+0000 7fbbe7fff640 1 --2- 172.21.15.76:0/1757853402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbf00cda60 0x7fbbf00ca7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:25.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.516+0000 7fbbe7fff640 1 -- 172.21.15.76:0/1757853402 >> v1:172.21.15.113:6789/0 conn(0x7fbbf00b0cd0 legacy=0x7fbbf0070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:25.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.516+0000 7fbbe7fff640 1 -- 172.21.15.76:0/1757853402 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf00c9fc0 con 0x7fbbf0070d40 2023-11-09T03:47:25.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.516+0000 7fbbe7fff640 1 -- 172.21.15.76:0/1757853402 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fbbe80019f0 con 0x7fbbf0070d40 2023-11-09T03:47:25.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.516+0000 7fbbe7fff640 1 -- 172.21.15.76:0/1757853402 >> v1:172.21.15.76:6789/0 conn(0x7fbbf0070d40 legacy=0x7fbbf006c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:25.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.516+0000 7fbbe7fff640 1 --2- 172.21.15.76:0/1757853402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbdc0039a0 0x7fbbdc003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:25.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.516+0000 7fbbe7fff640 1 --2- 172.21.15.76:0/1757853402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbf00cda60 0x7fbbdc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:25.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.516+0000 7fbbe7fff640 1 --2- 172.21.15.76:0/1757853402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbdc004d00 0x7fbbdc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:25.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.516+0000 7fbbe7fff640 1 -- 172.21.15.76:0/1757853402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf00c9ce0 con 0x7fbbdc004d00 2023-11-09T03:47:25.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.516+0000 7fbbe7fff640 1 -- 172.21.15.76:0/1757853402 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc00b980 con 0x7fbbdc0039a0 2023-11-09T03:47:25.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.516+0000 7fbbe7fff640 1 -- 172.21.15.76:0/1757853402 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc00bb40 con 0x7fbbf00cda60 2023-11-09T03:47:25.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.517+0000 7fbbf597f640 1 --2- 172.21.15.76:0/1757853402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbdc004d00 0x7fbbdc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:25.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.517+0000 7fbbf517e640 1 --2- 172.21.15.76:0/1757853402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbdc0039a0 0x7fbbdc003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:25.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.517+0000 7fbbf497d640 1 --2- 172.21.15.76:0/1757853402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbf00cda60 0x7fbbdc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:25.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.517+0000 7fbbf597f640 1 -- 172.21.15.76:0/1757853402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbf00cda60 msgr2=0x7fbbdc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:25.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.517+0000 7fbbf597f640 1 --2- 172.21.15.76:0/1757853402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbf00cda60 0x7fbbdc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:25.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.517+0000 7fbbf597f640 1 -- 172.21.15.76:0/1757853402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbdc0039a0 msgr2=0x7fbbdc003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:25.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.517+0000 7fbbf597f640 1 --2- 172.21.15.76:0/1757853402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbdc0039a0 0x7fbbdc003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:25.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.517+0000 7fbbf597f640 1 -- 172.21.15.76:0/1757853402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbdc003520 con 0x7fbbdc004d00 2023-11-09T03:47:25.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.518+0000 7fbbf597f640 1 --2- 172.21.15.76:0/1757853402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbdc004d00 0x7fbbdc005180 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbbe800a830 tx=0x7fbbdc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36e2995dd22deaa server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:25.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.518+0000 7fbbe7fff640 1 -- 172.21.15.76:0/1757853402 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe802cea0 con 0x7fbbdc004d00 2023-11-09T03:47:25.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.518+0000 7fbbe7fff640 1 -- 172.21.15.76:0/1757853402 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe802f070 con 0x7fbbdc004d00 2023-11-09T03:47:25.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.518+0000 7fbbe7fff640 1 -- 172.21.15.76:0/1757853402 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8034890 con 0x7fbbdc004d00 2023-11-09T03:47:25.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.518+0000 7fbbf73e2640 1 -- 172.21.15.76:0/1757853402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbdc004d00 msgr2=0x7fbbdc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:25.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.518+0000 7fbbf73e2640 1 --2- 172.21.15.76:0/1757853402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbdc004d00 0x7fbbdc005180 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbbe800a830 tx=0x7fbbdc0057f0 comp rx=0 tx=0).stop 2023-11-09T03:47:25.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.518+0000 7fbbf597f640 1 -- 172.21.15.76:0/1757853402 reap_dead start 2023-11-09T03:47:25.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.518+0000 7fbbf73e2640 1 -- 172.21.15.76:0/1757853402 shutdown_connections 2023-11-09T03:47:25.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.518+0000 7fbbf73e2640 1 -- 172.21.15.76:0/1757853402 >> 172.21.15.76:0/1757853402 conn(0x7fbbf00ce950 msgr2=0x7fbbf00ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:25.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.518+0000 7fbbf73e2640 1 -- 172.21.15.76:0/1757853402 shutdown_connections 2023-11-09T03:47:25.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.518+0000 7fbbf73e2640 1 -- 172.21.15.76:0/1757853402 wait complete. 2023-11-09T03:47:25.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.520+0000 7fbbf73e2640 1 Processor -- start 2023-11-09T03:47:25.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.520+0000 7fbbf73e2640 1 -- start start 2023-11-09T03:47:25.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.520+0000 7fbbf73e2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbdc004d00 0x7fbbf0157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:25.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.520+0000 7fbbf73e2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbf00b13c0 0x7fbbf0147840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:25.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.520+0000 7fbbf73e2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbf0147d80 0x7fbbf0144460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:25.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.520+0000 7fbbf73e2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf01470d0 con 0x7fbbf00b13c0 2023-11-09T03:47:25.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.520+0000 7fbbf73e2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf01473b0 con 0x7fbbdc004d00 2023-11-09T03:47:25.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.520+0000 7fbbf73e2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf00b1820 con 0x7fbbf0147d80 2023-11-09T03:47:25.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.520+0000 7fbbf497d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbf00b13c0 0x7fbbf0147840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:25.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.520+0000 7fbbf497d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbf00b13c0 0x7fbbf0147840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42592/0 (socket says 172.21.15.76:42592) 2023-11-09T03:47:25.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.520+0000 7fbbf497d640 1 -- 172.21.15.76:0/1361058064 learned_addr learned my addr 172.21.15.76:0/1361058064 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:25.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.521+0000 7fbbf517e640 1 --2- 172.21.15.76:0/1361058064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbdc004d00 0x7fbbf0157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:25.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.521+0000 7fbbf597f640 1 --2- 172.21.15.76:0/1361058064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbf0147d80 0x7fbbf0144460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:25.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.521+0000 7fbbf497d640 1 -- 172.21.15.76:0/1361058064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbf0147d80 msgr2=0x7fbbf0144460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:25.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.521+0000 7fbbf497d640 1 --2- 172.21.15.76:0/1361058064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbf0147d80 0x7fbbf0144460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:25.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.521+0000 7fbbf497d640 1 -- 172.21.15.76:0/1361058064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbdc004d00 msgr2=0x7fbbf0157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:25.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.521+0000 7fbbf497d640 1 --2- 172.21.15.76:0/1361058064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbdc004d00 0x7fbbf0157130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:25.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.521+0000 7fbbf497d640 1 -- 172.21.15.76:0/1361058064 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf00b1af0 con 0x7fbbf00b13c0 2023-11-09T03:47:25.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.521+0000 7fbbf517e640 1 --2- 172.21.15.76:0/1361058064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbdc004d00 0x7fbbf0157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:25.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.521+0000 7fbbf597f640 1 --2- 172.21.15.76:0/1361058064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbf0147d80 0x7fbbf0144460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:25.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.521+0000 7fbbf497d640 1 --2- 172.21.15.76:0/1361058064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbf00b13c0 0x7fbbf0147840 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00090a0 tx=0x7fbbe0009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:25.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.521+0000 7fbbe5ffb640 1 -- 172.21.15.76:0/1361058064 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0007990 con 0x7fbbf00b13c0 2023-11-09T03:47:25.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.521+0000 7fbbe5ffb640 1 -- 172.21.15.76:0/1361058064 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe00035f0 con 0x7fbbf00b13c0 2023-11-09T03:47:25.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.521+0000 7fbbf73e2640 1 -- 172.21.15.76:0/1361058064 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf00b1dd0 con 0x7fbbf00b13c0 2023-11-09T03:47:25.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.521+0000 7fbbe5ffb640 1 -- 172.21.15.76:0/1361058064 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe00039a0 con 0x7fbbf00b13c0 2023-11-09T03:47:25.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.521+0000 7fbbf73e2640 1 -- 172.21.15.76:0/1361058064 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf01449a0 con 0x7fbbf00b13c0 2023-11-09T03:47:25.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.522+0000 7fbbf73e2640 1 -- 172.21.15.76:0/1361058064 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf00b22f0 con 0x7fbbf00b13c0 2023-11-09T03:47:25.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.522+0000 7fbbe5ffb640 1 -- 172.21.15.76:0/1361058064 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbbe0003b00 con 0x7fbbf00b13c0 2023-11-09T03:47:25.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.523+0000 7fbbe5ffb640 1 --2- 172.21.15.76:0/1361058064 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbbd406ae70 0x7fbbd406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:25.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.523+0000 7fbbe5ffb640 1 -- 172.21.15.76:0/1361058064 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fbbe0089540 con 0x7fbbf00b13c0 2023-11-09T03:47:25.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.523+0000 7fbbf517e640 1 --2- 172.21.15.76:0/1361058064 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbbd406ae70 0x7fbbd406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:25.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.523+0000 7fbbf517e640 1 --2- 172.21.15.76:0/1361058064 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbbd406ae70 0x7fbbd406d330 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbbec0045c0 tx=0x7fbbec00a2a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:25.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.527+0000 7fbbe5ffb640 1 -- 172.21.15.76:0/1361058064 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbbe00543e0 con 0x7fbbf00b13c0 2023-11-09T03:47:25.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.693+0000 7fbbf73e2640 1 -- 172.21.15.76:0/1361058064 --> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbbf00ca2a0 con 0x7fbbd406ae70 2023-11-09T03:47:25.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.702+0000 7fbbe5ffb640 1 -- 172.21.15.76:0/1361058064 <== mgr.4105 v2:172.21.15.113:6832/1264193042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+370368 (secure 0 0 0) 0x7fbbec018070 con 0x7fbbd406ae70 2023-11-09T03:47:25.706 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:25.708 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2023-11-09T03:47:25.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.706+0000 7fbbf73e2640 1 -- 172.21.15.76:0/1361058064 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbbd406ae70 msgr2=0x7fbbd406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:25.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.706+0000 7fbbf73e2640 1 --2- 172.21.15.76:0/1361058064 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbbd406ae70 0x7fbbd406d330 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbbec0045c0 tx=0x7fbbec00a2a0 comp rx=0 tx=0).stop 2023-11-09T03:47:25.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.706+0000 7fbbf73e2640 1 -- 172.21.15.76:0/1361058064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbf00b13c0 msgr2=0x7fbbf0147840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:25.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.706+0000 7fbbf73e2640 1 --2- 172.21.15.76:0/1361058064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbf00b13c0 0x7fbbf0147840 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00090a0 tx=0x7fbbe0009560 comp rx=0 tx=0).stop 2023-11-09T03:47:25.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.706+0000 7fbbf73e2640 1 -- 172.21.15.76:0/1361058064 shutdown_connections 2023-11-09T03:47:25.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.706+0000 7fbbf73e2640 1 --2- 172.21.15.76:0/1361058064 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbbd406ae70 0x7fbbd406d330 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:25.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.706+0000 7fbbf73e2640 1 --2- 172.21.15.76:0/1361058064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbf0147d80 0x7fbbf0144460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:25.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.706+0000 7fbbf73e2640 1 --2- 172.21.15.76:0/1361058064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbf00b13c0 0x7fbbf0147840 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:25.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.706+0000 7fbbf73e2640 1 --2- 172.21.15.76:0/1361058064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbdc004d00 0x7fbbf0157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:25.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.706+0000 7fbbf73e2640 1 -- 172.21.15.76:0/1361058064 >> 172.21.15.76:0/1361058064 conn(0x7fbbf00ce950 msgr2=0x7fbbf00ab8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:25.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.706+0000 7fbbf73e2640 1 -- 172.21.15.76:0/1361058064 shutdown_connections 2023-11-09T03:47:25.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:25.706+0000 7fbbf73e2640 1 -- 172.21.15.76:0/1361058064 wait complete. 2023-11-09T03:47:25.716 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2023-11-09T03:47:24.777879+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":234,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222420,"kb_used_data":6220,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752300,"statfs":{"total":773094113280,"available":772866355200,"internally_reserved":0,"allocated":6369280,"data_stored":3499079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.861678"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248379+0000","last_change":"2023-11-09T03:47:20.941252+0000","last_active":"2023-11-09T03:47:22.248379+0000","last_peered":"2023-11-09T03:47:22.248379+0000","last_clean":"2023-11-09T03:47:22.248379+0000","last_became_active":"2023-11-09T03:47:20.940995+0000","last_became_peered":"2023-11-09T03:47:20.940995+0000","last_unstale":"2023-11-09T03:47:22.248379+0000","last_undegraded":"2023-11-09T03:47:22.248379+0000","last_fullsized":"2023-11-09T03:47:22.248379+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:44:27.004461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248080+0000","last_change":"2023-11-09T03:47:20.950878+0000","last_active":"2023-11-09T03:47:22.248080+0000","last_peered":"2023-11-09T03:47:22.248080+0000","last_clean":"2023-11-09T03:47:22.248080+0000","last_became_active":"2023-11-09T03:47:20.950611+0000","last_became_peered":"2023-11-09T03:47:20.950611+0000","last_unstale":"2023-11-09T03:47:22.248080+0000","last_undegraded":"2023-11-09T03:47:22.248080+0000","last_fullsized":"2023-11-09T03:47:22.248080+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:04:18.340460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026954+0000","last_change":"2023-11-09T03:47:20.945634+0000","last_active":"2023-11-09T03:47:22.026954+0000","last_peered":"2023-11-09T03:47:22.026954+0000","last_clean":"2023-11-09T03:47:22.026954+0000","last_became_active":"2023-11-09T03:47:20.945370+0000","last_became_peered":"2023-11-09T03:47:20.945370+0000","last_unstale":"2023-11-09T03:47:22.026954+0000","last_undegraded":"2023-11-09T03:47:22.026954+0000","last_fullsized":"2023-11-09T03:47:22.026954+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:57.187676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940818+0000","last_change":"2023-11-09T03:47:20.941855+0000","last_active":"2023-11-09T03:47:21.940818+0000","last_peered":"2023-11-09T03:47:21.940818+0000","last_clean":"2023-11-09T03:47:21.940818+0000","last_became_active":"2023-11-09T03:47:20.941598+0000","last_became_peered":"2023-11-09T03:47:20.941598+0000","last_unstale":"2023-11-09T03:47:21.940818+0000","last_undegraded":"2023-11-09T03:47:21.940818+0000","last_fullsized":"2023-11-09T03:47:21.940818+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:14:00.724882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248512+0000","last_change":"2023-11-09T03:47:20.947154+0000","last_active":"2023-11-09T03:47:22.248512+0000","last_peered":"2023-11-09T03:47:22.248512+0000","last_clean":"2023-11-09T03:47:22.248512+0000","last_became_active":"2023-11-09T03:47:20.946878+0000","last_became_peered":"2023-11-09T03:47:20.946878+0000","last_unstale":"2023-11-09T03:47:22.248512+0000","last_undegraded":"2023-11-09T03:47:22.248512+0000","last_fullsized":"2023-11-09T03:47:22.248512+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:41:16.327295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247898+0000","last_change":"2023-11-09T03:47:20.941698+0000","last_active":"2023-11-09T03:47:22.247898+0000","last_peered":"2023-11-09T03:47:22.247898+0000","last_clean":"2023-11-09T03:47:22.247898+0000","last_became_active":"2023-11-09T03:47:20.941432+0000","last_became_peered":"2023-11-09T03:47:20.941432+0000","last_unstale":"2023-11-09T03:47:22.247898+0000","last_undegraded":"2023-11-09T03:47:22.247898+0000","last_fullsized":"2023-11-09T03:47:22.247898+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:12:48.690396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247980+0000","last_change":"2023-11-09T03:47:20.948132+0000","last_active":"2023-11-09T03:47:22.247980+0000","last_peered":"2023-11-09T03:47:22.247980+0000","last_clean":"2023-11-09T03:47:22.247980+0000","last_became_active":"2023-11-09T03:47:20.947854+0000","last_became_peered":"2023-11-09T03:47:20.947854+0000","last_unstale":"2023-11-09T03:47:22.247980+0000","last_undegraded":"2023-11-09T03:47:22.247980+0000","last_fullsized":"2023-11-09T03:47:22.247980+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:38:09.677347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248627+0000","last_change":"2023-11-09T03:47:20.944502+0000","last_active":"2023-11-09T03:47:22.248627+0000","last_peered":"2023-11-09T03:47:22.248627+0000","last_clean":"2023-11-09T03:47:22.248627+0000","last_became_active":"2023-11-09T03:47:20.944214+0000","last_became_peered":"2023-11-09T03:47:20.944214+0000","last_unstale":"2023-11-09T03:47:22.248627+0000","last_undegraded":"2023-11-09T03:47:22.248627+0000","last_fullsized":"2023-11-09T03:47:22.248627+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:22:14.432893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248644+0000","last_change":"2023-11-09T03:47:20.954329+0000","last_active":"2023-11-09T03:47:22.248644+0000","last_peered":"2023-11-09T03:47:22.248644+0000","last_clean":"2023-11-09T03:47:22.248644+0000","last_became_active":"2023-11-09T03:47:20.953654+0000","last_became_peered":"2023-11-09T03:47:20.953654+0000","last_unstale":"2023-11-09T03:47:22.248644+0000","last_undegraded":"2023-11-09T03:47:22.248644+0000","last_fullsized":"2023-11-09T03:47:22.248644+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:48:27.582736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939384+0000","last_change":"2023-11-09T03:47:20.943196+0000","last_active":"2023-11-09T03:47:21.939384+0000","last_peered":"2023-11-09T03:47:21.939384+0000","last_clean":"2023-11-09T03:47:21.939384+0000","last_became_active":"2023-11-09T03:47:20.942943+0000","last_became_peered":"2023-11-09T03:47:20.942943+0000","last_unstale":"2023-11-09T03:47:21.939384+0000","last_undegraded":"2023-11-09T03:47:21.939384+0000","last_fullsized":"2023-11-09T03:47:21.939384+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:36:04.069418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939364+0000","last_change":"2023-11-09T03:47:20.943821+0000","last_active":"2023-11-09T03:47:21.939364+0000","last_peered":"2023-11-09T03:47:21.939364+0000","last_clean":"2023-11-09T03:47:21.939364+0000","last_became_active":"2023-11-09T03:47:20.943571+0000","last_became_peered":"2023-11-09T03:47:20.943571+0000","last_unstale":"2023-11-09T03:47:21.939364+0000","last_undegraded":"2023-11-09T03:47:21.939364+0000","last_fullsized":"2023-11-09T03:47:21.939364+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:26:12.103741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940598+0000","last_change":"2023-11-09T03:47:20.942582+0000","last_active":"2023-11-09T03:47:21.940598+0000","last_peered":"2023-11-09T03:47:21.940598+0000","last_clean":"2023-11-09T03:47:21.940598+0000","last_became_active":"2023-11-09T03:47:20.942214+0000","last_became_peered":"2023-11-09T03:47:20.942214+0000","last_unstale":"2023-11-09T03:47:21.940598+0000","last_undegraded":"2023-11-09T03:47:21.940598+0000","last_fullsized":"2023-11-09T03:47:21.940598+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:51:53.879857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248688+0000","last_change":"2023-11-09T03:47:20.942261+0000","last_active":"2023-11-09T03:47:22.248688+0000","last_peered":"2023-11-09T03:47:22.248688+0000","last_clean":"2023-11-09T03:47:22.248688+0000","last_became_active":"2023-11-09T03:47:20.941996+0000","last_became_peered":"2023-11-09T03:47:20.941996+0000","last_unstale":"2023-11-09T03:47:22.248688+0000","last_undegraded":"2023-11-09T03:47:22.248688+0000","last_fullsized":"2023-11-09T03:47:22.248688+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:21:04.459156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248066+0000","last_change":"2023-11-09T03:47:20.944425+0000","last_active":"2023-11-09T03:47:22.248066+0000","last_peered":"2023-11-09T03:47:22.248066+0000","last_clean":"2023-11-09T03:47:22.248066+0000","last_became_active":"2023-11-09T03:47:20.944159+0000","last_became_peered":"2023-11-09T03:47:20.944159+0000","last_unstale":"2023-11-09T03:47:22.248066+0000","last_undegraded":"2023-11-09T03:47:22.248066+0000","last_fullsized":"2023-11-09T03:47:22.248066+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:26:12.103741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248162+0000","last_change":"2023-11-09T03:47:20.954225+0000","last_active":"2023-11-09T03:47:22.248162+0000","last_peered":"2023-11-09T03:47:22.248162+0000","last_clean":"2023-11-09T03:47:22.248162+0000","last_became_active":"2023-11-09T03:47:20.954004+0000","last_became_peered":"2023-11-09T03:47:20.954004+0000","last_unstale":"2023-11-09T03:47:22.248162+0000","last_undegraded":"2023-11-09T03:47:22.248162+0000","last_fullsized":"2023-11-09T03:47:22.248162+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:36:04.069418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247697+0000","last_change":"2023-11-09T03:47:20.940209+0000","last_active":"2023-11-09T03:47:22.247697+0000","last_peered":"2023-11-09T03:47:22.247697+0000","last_clean":"2023-11-09T03:47:22.247697+0000","last_became_active":"2023-11-09T03:47:20.939956+0000","last_became_peered":"2023-11-09T03:47:20.939956+0000","last_unstale":"2023-11-09T03:47:22.247697+0000","last_undegraded":"2023-11-09T03:47:22.247697+0000","last_fullsized":"2023-11-09T03:47:22.247697+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:52.683555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248360+0000","last_change":"2023-11-09T03:47:20.942850+0000","last_active":"2023-11-09T03:47:22.248360+0000","last_peered":"2023-11-09T03:47:22.248360+0000","last_clean":"2023-11-09T03:47:22.248360+0000","last_became_active":"2023-11-09T03:47:20.942596+0000","last_became_peered":"2023-11-09T03:47:20.942596+0000","last_unstale":"2023-11-09T03:47:22.248360+0000","last_undegraded":"2023-11-09T03:47:22.248360+0000","last_fullsized":"2023-11-09T03:47:22.248360+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:14:28.441176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.027085+0000","last_change":"2023-11-09T03:47:20.954037+0000","last_active":"2023-11-09T03:47:22.027085+0000","last_peered":"2023-11-09T03:47:22.027085+0000","last_clean":"2023-11-09T03:47:22.027085+0000","last_became_active":"2023-11-09T03:47:20.953777+0000","last_became_peered":"2023-11-09T03:47:20.953777+0000","last_unstale":"2023-11-09T03:47:22.027085+0000","last_undegraded":"2023-11-09T03:47:22.027085+0000","last_fullsized":"2023-11-09T03:47:22.027085+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:55:28.077165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248233+0000","last_change":"2023-11-09T03:47:20.941849+0000","last_active":"2023-11-09T03:47:22.248233+0000","last_peered":"2023-11-09T03:47:22.248233+0000","last_clean":"2023-11-09T03:47:22.248233+0000","last_became_active":"2023-11-09T03:47:20.941602+0000","last_became_peered":"2023-11-09T03:47:20.941602+0000","last_unstale":"2023-11-09T03:47:22.248233+0000","last_undegraded":"2023-11-09T03:47:22.248233+0000","last_fullsized":"2023-11-09T03:47:22.248233+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:23.398916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248619+0000","last_change":"2023-11-09T03:47:20.951388+0000","last_active":"2023-11-09T03:47:22.248619+0000","last_peered":"2023-11-09T03:47:22.248619+0000","last_clean":"2023-11-09T03:47:22.248619+0000","last_became_active":"2023-11-09T03:47:20.951123+0000","last_became_peered":"2023-11-09T03:47:20.951123+0000","last_unstale":"2023-11-09T03:47:22.248619+0000","last_undegraded":"2023-11-09T03:47:22.248619+0000","last_fullsized":"2023-11-09T03:47:22.248619+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:55:54.333416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026719+0000","last_change":"2023-11-09T03:47:20.948114+0000","last_active":"2023-11-09T03:47:22.026719+0000","last_peered":"2023-11-09T03:47:22.026719+0000","last_clean":"2023-11-09T03:47:22.026719+0000","last_became_active":"2023-11-09T03:47:20.947849+0000","last_became_peered":"2023-11-09T03:47:20.947849+0000","last_unstale":"2023-11-09T03:47:22.026719+0000","last_undegraded":"2023-11-09T03:47:22.026719+0000","last_fullsized":"2023-11-09T03:47:22.026719+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:52.683555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247955+0000","last_change":"2023-11-09T03:47:20.945723+0000","last_active":"2023-11-09T03:47:22.247955+0000","last_peered":"2023-11-09T03:47:22.247955+0000","last_clean":"2023-11-09T03:47:22.247955+0000","last_became_active":"2023-11-09T03:47:20.945447+0000","last_became_peered":"2023-11-09T03:47:20.945447+0000","last_unstale":"2023-11-09T03:47:22.247955+0000","last_undegraded":"2023-11-09T03:47:22.247955+0000","last_fullsized":"2023-11-09T03:47:22.247955+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:41:48.124870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026634+0000","last_change":"2023-11-09T03:47:20.944344+0000","last_active":"2023-11-09T03:47:22.026634+0000","last_peered":"2023-11-09T03:47:22.026634+0000","last_clean":"2023-11-09T03:47:22.026634+0000","last_became_active":"2023-11-09T03:47:20.943991+0000","last_became_peered":"2023-11-09T03:47:20.943991+0000","last_unstale":"2023-11-09T03:47:22.026634+0000","last_undegraded":"2023-11-09T03:47:22.026634+0000","last_fullsized":"2023-11-09T03:47:22.026634+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:08:17.239107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248007+0000","last_change":"2023-11-09T03:47:20.949489+0000","last_active":"2023-11-09T03:47:22.248007+0000","last_peered":"2023-11-09T03:47:22.248007+0000","last_clean":"2023-11-09T03:47:22.248007+0000","last_became_active":"2023-11-09T03:47:20.949211+0000","last_became_peered":"2023-11-09T03:47:20.949211+0000","last_unstale":"2023-11-09T03:47:22.248007+0000","last_undegraded":"2023-11-09T03:47:22.248007+0000","last_fullsized":"2023-11-09T03:47:22.248007+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:56:40.000841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248496+0000","last_change":"2023-11-09T03:47:20.952835+0000","last_active":"2023-11-09T03:47:22.248496+0000","last_peered":"2023-11-09T03:47:22.248496+0000","last_clean":"2023-11-09T03:47:22.248496+0000","last_became_active":"2023-11-09T03:47:20.952578+0000","last_became_peered":"2023-11-09T03:47:20.952578+0000","last_unstale":"2023-11-09T03:47:22.248496+0000","last_undegraded":"2023-11-09T03:47:22.248496+0000","last_fullsized":"2023-11-09T03:47:22.248496+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:40:53.848669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.941032+0000","last_change":"2023-11-09T03:47:20.944379+0000","last_active":"2023-11-09T03:47:21.941032+0000","last_peered":"2023-11-09T03:47:21.941032+0000","last_clean":"2023-11-09T03:47:21.941032+0000","last_became_active":"2023-11-09T03:47:20.944134+0000","last_became_peered":"2023-11-09T03:47:20.944134+0000","last_unstale":"2023-11-09T03:47:21.941032+0000","last_undegraded":"2023-11-09T03:47:21.941032+0000","last_fullsized":"2023-11-09T03:47:21.941032+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:55:28.077165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248970+0000","last_change":"2023-11-09T03:47:20.940786+0000","last_active":"2023-11-09T03:47:22.248970+0000","last_peered":"2023-11-09T03:47:22.248970+0000","last_clean":"2023-11-09T03:47:22.248970+0000","last_became_active":"2023-11-09T03:47:20.940529+0000","last_became_peered":"2023-11-09T03:47:20.940529+0000","last_unstale":"2023-11-09T03:47:22.248970+0000","last_undegraded":"2023-11-09T03:47:22.248970+0000","last_fullsized":"2023-11-09T03:47:22.248970+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:40.639821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248129+0000","last_change":"2023-11-09T03:47:20.949337+0000","last_active":"2023-11-09T03:47:22.248129+0000","last_peered":"2023-11-09T03:47:22.248129+0000","last_clean":"2023-11-09T03:47:22.248129+0000","last_became_active":"2023-11-09T03:47:20.949088+0000","last_became_peered":"2023-11-09T03:47:20.949088+0000","last_unstale":"2023-11-09T03:47:22.248129+0000","last_undegraded":"2023-11-09T03:47:22.248129+0000","last_fullsized":"2023-11-09T03:47:22.248129+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:52:12.660646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248774+0000","last_change":"2023-11-09T03:47:20.948003+0000","last_active":"2023-11-09T03:47:22.248774+0000","last_peered":"2023-11-09T03:47:22.248774+0000","last_clean":"2023-11-09T03:47:22.248774+0000","last_became_active":"2023-11-09T03:47:20.947690+0000","last_became_peered":"2023-11-09T03:47:20.947690+0000","last_unstale":"2023-11-09T03:47:22.248774+0000","last_undegraded":"2023-11-09T03:47:22.248774+0000","last_fullsized":"2023-11-09T03:47:22.248774+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:28:18.013453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026610+0000","last_change":"2023-11-09T03:47:20.946260+0000","last_active":"2023-11-09T03:47:22.026610+0000","last_peered":"2023-11-09T03:47:22.026610+0000","last_clean":"2023-11-09T03:47:22.026610+0000","last_became_active":"2023-11-09T03:47:20.945986+0000","last_became_peered":"2023-11-09T03:47:20.945986+0000","last_unstale":"2023-11-09T03:47:22.026610+0000","last_undegraded":"2023-11-09T03:47:22.026610+0000","last_fullsized":"2023-11-09T03:47:22.026610+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:59:29.154623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247961+0000","last_change":"2023-11-09T03:47:20.952176+0000","last_active":"2023-11-09T03:47:22.247961+0000","last_peered":"2023-11-09T03:47:22.247961+0000","last_clean":"2023-11-09T03:47:22.247961+0000","last_became_active":"2023-11-09T03:47:20.951921+0000","last_became_peered":"2023-11-09T03:47:20.951921+0000","last_unstale":"2023-11-09T03:47:22.247961+0000","last_undegraded":"2023-11-09T03:47:22.247961+0000","last_fullsized":"2023-11-09T03:47:22.247961+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:22.376023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247669+0000","last_change":"2023-11-09T03:47:20.943940+0000","last_active":"2023-11-09T03:47:22.247669+0000","last_peered":"2023-11-09T03:47:22.247669+0000","last_clean":"2023-11-09T03:47:22.247669+0000","last_became_active":"2023-11-09T03:47:20.943708+0000","last_became_peered":"2023-11-09T03:47:20.943708+0000","last_unstale":"2023-11-09T03:47:22.247669+0000","last_undegraded":"2023-11-09T03:47:22.247669+0000","last_fullsized":"2023-11-09T03:47:22.247669+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:08:17.239107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026790+0000","last_change":"2023-11-09T03:47:20.942360+0000","last_active":"2023-11-09T03:47:22.026790+0000","last_peered":"2023-11-09T03:47:22.026790+0000","last_clean":"2023-11-09T03:47:22.026790+0000","last_became_active":"2023-11-09T03:47:20.942078+0000","last_became_peered":"2023-11-09T03:47:20.942078+0000","last_unstale":"2023-11-09T03:47:22.026790+0000","last_undegraded":"2023-11-09T03:47:22.026790+0000","last_fullsized":"2023-11-09T03:47:22.026790+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:12:48.690396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248715+0000","last_change":"2023-11-09T03:47:19.943955+0000","last_active":"2023-11-09T03:47:22.248715+0000","last_peered":"2023-11-09T03:47:22.248715+0000","last_clean":"2023-11-09T03:47:22.248715+0000","last_became_active":"2023-11-09T03:47:19.943692+0000","last_became_peered":"2023-11-09T03:47:19.943692+0000","last_unstale":"2023-11-09T03:47:22.248715+0000","last_undegraded":"2023-11-09T03:47:22.248715+0000","last_fullsized":"2023-11-09T03:47:22.248715+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:24.100578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":27,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.928645+0000","last_change":"2023-11-09T03:47:02.581917+0000","last_active":"2023-11-09T03:47:20.928645+0000","last_peered":"2023-11-09T03:47:20.928645+0000","last_clean":"2023-11-09T03:47:20.928645+0000","last_became_active":"2023-11-09T03:47:01.588082+0000","last_became_peered":"2023-11-09T03:47:01.588082+0000","last_unstale":"2023-11-09T03:47:20.928645+0000","last_undegraded":"2023-11-09T03:47:20.928645+0000","last_fullsized":"2023-11-09T03:47:20.928645+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:44:07.653557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034538899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940245+0000","last_change":"2023-11-09T03:47:20.938900+0000","last_active":"2023-11-09T03:47:21.940245+0000","last_peered":"2023-11-09T03:47:21.940245+0000","last_clean":"2023-11-09T03:47:21.940245+0000","last_became_active":"2023-11-09T03:47:20.938654+0000","last_became_peered":"2023-11-09T03:47:20.938654+0000","last_unstale":"2023-11-09T03:47:21.940245+0000","last_undegraded":"2023-11-09T03:47:21.940245+0000","last_fullsized":"2023-11-09T03:47:21.940245+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:40.639821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248651+0000","last_change":"2023-11-09T03:47:19.945834+0000","last_active":"2023-11-09T03:47:22.248651+0000","last_peered":"2023-11-09T03:47:22.248651+0000","last_clean":"2023-11-09T03:47:22.248651+0000","last_became_active":"2023-11-09T03:47:19.945484+0000","last_became_peered":"2023-11-09T03:47:19.945484+0000","last_unstale":"2023-11-09T03:47:22.248651+0000","last_undegraded":"2023-11-09T03:47:22.248651+0000","last_fullsized":"2023-11-09T03:47:22.248651+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:57:21.887088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.930614+0000","last_change":"2023-11-09T03:47:03.029063+0000","last_active":"2023-11-09T03:47:20.930614+0000","last_peered":"2023-11-09T03:47:20.930614+0000","last_clean":"2023-11-09T03:47:20.930614+0000","last_became_active":"2023-11-09T03:47:01.593582+0000","last_became_peered":"2023-11-09T03:47:01.593582+0000","last_unstale":"2023-11-09T03:47:20.930614+0000","last_undegraded":"2023-11-09T03:47:20.930614+0000","last_fullsized":"2023-11-09T03:47:20.930614+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:56:20.649937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041855100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026721+0000","last_change":"2023-11-09T03:47:20.944419+0000","last_active":"2023-11-09T03:47:22.026721+0000","last_peered":"2023-11-09T03:47:22.026721+0000","last_clean":"2023-11-09T03:47:22.026721+0000","last_became_active":"2023-11-09T03:47:20.944141+0000","last_became_peered":"2023-11-09T03:47:20.944141+0000","last_unstale":"2023-11-09T03:47:22.026721+0000","last_undegraded":"2023-11-09T03:47:22.026721+0000","last_fullsized":"2023-11-09T03:47:22.026721+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:06:17.676049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"53'8","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026516+0000","last_change":"2023-11-09T03:47:19.946625+0000","last_active":"2023-11-09T03:47:22.026516+0000","last_peered":"2023-11-09T03:47:22.026516+0000","last_clean":"2023-11-09T03:47:22.026516+0000","last_became_active":"2023-11-09T03:47:19.946350+0000","last_became_peered":"2023-11-09T03:47:19.946350+0000","last_unstale":"2023-11-09T03:47:22.026516+0000","last_undegraded":"2023-11-09T03:47:22.026516+0000","last_fullsized":"2023-11-09T03:47:22.026516+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:32.646643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.934959+0000","last_change":"2023-11-09T03:47:02.619336+0000","last_active":"2023-11-09T03:47:20.934959+0000","last_peered":"2023-11-09T03:47:20.934959+0000","last_clean":"2023-11-09T03:47:20.934959+0000","last_became_active":"2023-11-09T03:47:01.587749+0000","last_became_peered":"2023-11-09T03:47:01.587749+0000","last_unstale":"2023-11-09T03:47:20.934959+0000","last_undegraded":"2023-11-09T03:47:20.934959+0000","last_fullsized":"2023-11-09T03:47:20.934959+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063032099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247912+0000","last_change":"2023-11-09T03:47:20.948391+0000","last_active":"2023-11-09T03:47:22.247912+0000","last_peered":"2023-11-09T03:47:22.247912+0000","last_clean":"2023-11-09T03:47:22.247912+0000","last_became_active":"2023-11-09T03:47:20.948103+0000","last_became_peered":"2023-11-09T03:47:20.948103+0000","last_unstale":"2023-11-09T03:47:22.247912+0000","last_undegraded":"2023-11-09T03:47:22.247912+0000","last_fullsized":"2023-11-09T03:47:22.247912+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:33.651326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940202+0000","last_change":"2023-11-09T03:47:19.944514+0000","last_active":"2023-11-09T03:47:21.940202+0000","last_peered":"2023-11-09T03:47:21.940202+0000","last_clean":"2023-11-09T03:47:21.940202+0000","last_became_active":"2023-11-09T03:47:19.944230+0000","last_became_peered":"2023-11-09T03:47:19.944230+0000","last_unstale":"2023-11-09T03:47:21.940202+0000","last_undegraded":"2023-11-09T03:47:21.940202+0000","last_fullsized":"2023-11-09T03:47:21.940202+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:08:16.234424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":27,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.928406+0000","last_change":"2023-11-09T03:47:02.581658+0000","last_active":"2023-11-09T03:47:20.928406+0000","last_peered":"2023-11-09T03:47:20.928406+0000","last_clean":"2023-11-09T03:47:20.928406+0000","last_became_active":"2023-11-09T03:47:01.588612+0000","last_became_peered":"2023-11-09T03:47:01.588612+0000","last_unstale":"2023-11-09T03:47:20.928406+0000","last_undegraded":"2023-11-09T03:47:20.928406+0000","last_fullsized":"2023-11-09T03:47:20.928406+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045238500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.027066+0000","last_change":"2023-11-09T03:47:20.952326+0000","last_active":"2023-11-09T03:47:22.027066+0000","last_peered":"2023-11-09T03:47:22.027066+0000","last_clean":"2023-11-09T03:47:22.027066+0000","last_became_active":"2023-11-09T03:47:20.952035+0000","last_became_peered":"2023-11-09T03:47:20.952035+0000","last_unstale":"2023-11-09T03:47:22.027066+0000","last_undegraded":"2023-11-09T03:47:22.027066+0000","last_fullsized":"2023-11-09T03:47:22.027066+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:14:00.724882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248147+0000","last_change":"2023-11-09T03:47:19.947282+0000","last_active":"2023-11-09T03:47:22.248147+0000","last_peered":"2023-11-09T03:47:22.248147+0000","last_clean":"2023-11-09T03:47:22.248147+0000","last_became_active":"2023-11-09T03:47:19.947034+0000","last_became_peered":"2023-11-09T03:47:19.947034+0000","last_unstale":"2023-11-09T03:47:22.248147+0000","last_undegraded":"2023-11-09T03:47:22.248147+0000","last_fullsized":"2023-11-09T03:47:22.248147+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:13:59.720199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.935070+0000","last_change":"2023-11-09T03:47:02.618828+0000","last_active":"2023-11-09T03:47:20.935070+0000","last_peered":"2023-11-09T03:47:20.935070+0000","last_clean":"2023-11-09T03:47:20.935070+0000","last_became_active":"2023-11-09T03:47:01.590679+0000","last_became_peered":"2023-11-09T03:47:01.590679+0000","last_unstale":"2023-11-09T03:47:20.935070+0000","last_undegraded":"2023-11-09T03:47:20.935070+0000","last_fullsized":"2023-11-09T03:47:20.935070+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:06:30.982987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070668,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248700+0000","last_change":"2023-11-09T03:47:20.954224+0000","last_active":"2023-11-09T03:47:22.248700+0000","last_peered":"2023-11-09T03:47:22.248700+0000","last_clean":"2023-11-09T03:47:22.248700+0000","last_became_active":"2023-11-09T03:47:20.953950+0000","last_became_peered":"2023-11-09T03:47:20.953950+0000","last_unstale":"2023-11-09T03:47:22.248700+0000","last_undegraded":"2023-11-09T03:47:22.248700+0000","last_fullsized":"2023-11-09T03:47:22.248700+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:58:09.769432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.027101+0000","last_change":"2023-11-09T03:47:19.947929+0000","last_active":"2023-11-09T03:47:22.027101+0000","last_peered":"2023-11-09T03:47:22.027101+0000","last_clean":"2023-11-09T03:47:22.027101+0000","last_became_active":"2023-11-09T03:47:19.947663+0000","last_became_peered":"2023-11-09T03:47:19.947663+0000","last_unstale":"2023-11-09T03:47:22.027101+0000","last_undegraded":"2023-11-09T03:47:22.027101+0000","last_fullsized":"2023-11-09T03:47:22.027101+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:02:12.613580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931259+0000","last_change":"2023-11-09T03:47:03.146528+0000","last_active":"2023-11-09T03:47:20.931259+0000","last_peered":"2023-11-09T03:47:20.931259+0000","last_clean":"2023-11-09T03:47:20.931259+0000","last_became_active":"2023-11-09T03:47:01.587060+0000","last_became_peered":"2023-11-09T03:47:01.587060+0000","last_unstale":"2023-11-09T03:47:20.931259+0000","last_undegraded":"2023-11-09T03:47:20.931259+0000","last_fullsized":"2023-11-09T03:47:20.931259+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:39:15.238496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079064999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.027243+0000","last_change":"2023-11-09T03:47:20.954274+0000","last_active":"2023-11-09T03:47:22.027243+0000","last_peered":"2023-11-09T03:47:22.027243+0000","last_clean":"2023-11-09T03:47:22.027243+0000","last_became_active":"2023-11-09T03:47:20.953976+0000","last_became_peered":"2023-11-09T03:47:20.953976+0000","last_unstale":"2023-11-09T03:47:22.027243+0000","last_undegraded":"2023-11-09T03:47:22.027243+0000","last_fullsized":"2023-11-09T03:47:22.027243+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:05:32.662676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"53'6","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248315+0000","last_change":"2023-11-09T03:47:19.948464+0000","last_active":"2023-11-09T03:47:22.248315+0000","last_peered":"2023-11-09T03:47:22.248315+0000","last_clean":"2023-11-09T03:47:22.248315+0000","last_became_active":"2023-11-09T03:47:19.948254+0000","last_became_peered":"2023-11-09T03:47:19.948254+0000","last_unstale":"2023-11-09T03:47:22.248315+0000","last_undegraded":"2023-11-09T03:47:22.248315+0000","last_fullsized":"2023-11-09T03:47:22.248315+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:21:03.454473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931191+0000","last_change":"2023-11-09T03:47:03.148246+0000","last_active":"2023-11-09T03:47:20.931191+0000","last_peered":"2023-11-09T03:47:20.931191+0000","last_clean":"2023-11-09T03:47:20.931191+0000","last_became_active":"2023-11-09T03:47:01.590988+0000","last_became_peered":"2023-11-09T03:47:01.590988+0000","last_unstale":"2023-11-09T03:47:20.931191+0000","last_undegraded":"2023-11-09T03:47:20.931191+0000","last_fullsized":"2023-11-09T03:47:20.931191+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:07:20.785445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055651099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939172+0000","last_change":"2023-11-09T03:47:20.945226+0000","last_active":"2023-11-09T03:47:21.939172+0000","last_peered":"2023-11-09T03:47:21.939172+0000","last_clean":"2023-11-09T03:47:21.939172+0000","last_became_active":"2023-11-09T03:47:20.944978+0000","last_became_peered":"2023-11-09T03:47:20.944978+0000","last_unstale":"2023-11-09T03:47:21.939172+0000","last_undegraded":"2023-11-09T03:47:21.939172+0000","last_fullsized":"2023-11-09T03:47:21.939172+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:33.651326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"53'3","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026995+0000","last_change":"2023-11-09T03:47:19.945101+0000","last_active":"2023-11-09T03:47:22.026995+0000","last_peered":"2023-11-09T03:47:22.026995+0000","last_clean":"2023-11-09T03:47:22.026995+0000","last_became_active":"2023-11-09T03:47:19.944829+0000","last_became_peered":"2023-11-09T03:47:19.944829+0000","last_unstale":"2023-11-09T03:47:22.026995+0000","last_undegraded":"2023-11-09T03:47:22.026995+0000","last_fullsized":"2023-11-09T03:47:22.026995+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:49:12.247865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.939514+0000","last_change":"2023-11-09T03:47:02.582754+0000","last_active":"2023-11-09T03:47:20.939514+0000","last_peered":"2023-11-09T03:47:20.939514+0000","last_clean":"2023-11-09T03:47:20.939514+0000","last_became_active":"2023-11-09T03:47:01.589715+0000","last_became_peered":"2023-11-09T03:47:01.589715+0000","last_unstale":"2023-11-09T03:47:20.939514+0000","last_undegraded":"2023-11-09T03:47:20.939514+0000","last_fullsized":"2023-11-09T03:47:20.939514+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099342799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248560+0000","last_change":"2023-11-09T03:47:20.953974+0000","last_active":"2023-11-09T03:47:22.248560+0000","last_peered":"2023-11-09T03:47:22.248560+0000","last_clean":"2023-11-09T03:47:22.248560+0000","last_became_active":"2023-11-09T03:47:20.953714+0000","last_became_peered":"2023-11-09T03:47:20.953714+0000","last_unstale":"2023-11-09T03:47:22.248560+0000","last_undegraded":"2023-11-09T03:47:22.248560+0000","last_fullsized":"2023-11-09T03:47:22.248560+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:40:44.918031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"53'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248845+0000","last_change":"2023-11-09T03:47:19.943054+0000","last_active":"2023-11-09T03:47:22.248845+0000","last_peered":"2023-11-09T03:47:22.248845+0000","last_clean":"2023-11-09T03:47:22.248845+0000","last_became_active":"2023-11-09T03:47:19.942787+0000","last_became_peered":"2023-11-09T03:47:19.942787+0000","last_unstale":"2023-11-09T03:47:22.248845+0000","last_undegraded":"2023-11-09T03:47:22.248845+0000","last_fullsized":"2023-11-09T03:47:22.248845+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:58:08.764749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.939087+0000","last_change":"2023-11-09T03:47:02.724896+0000","last_active":"2023-11-09T03:47:20.939087+0000","last_peered":"2023-11-09T03:47:20.939087+0000","last_clean":"2023-11-09T03:47:20.939087+0000","last_became_active":"2023-11-09T03:47:01.592784+0000","last_became_peered":"2023-11-09T03:47:01.592784+0000","last_unstale":"2023-11-09T03:47:20.939087+0000","last_undegraded":"2023-11-09T03:47:20.939087+0000","last_fullsized":"2023-11-09T03:47:20.939087+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:51:32.016204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056683500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248073+0000","last_change":"2023-11-09T03:47:20.945066+0000","last_active":"2023-11-09T03:47:22.248073+0000","last_peered":"2023-11-09T03:47:22.248073+0000","last_clean":"2023-11-09T03:47:22.248073+0000","last_became_active":"2023-11-09T03:47:20.944812+0000","last_became_peered":"2023-11-09T03:47:20.944812+0000","last_unstale":"2023-11-09T03:47:22.248073+0000","last_undegraded":"2023-11-09T03:47:22.248073+0000","last_fullsized":"2023-11-09T03:47:22.248073+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:58:30.558728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"53'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248249+0000","last_change":"2023-11-09T03:47:19.947954+0000","last_active":"2023-11-09T03:47:22.248249+0000","last_peered":"2023-11-09T03:47:22.248249+0000","last_clean":"2023-11-09T03:47:22.248249+0000","last_became_active":"2023-11-09T03:47:19.947687+0000","last_became_peered":"2023-11-09T03:47:19.947687+0000","last_unstale":"2023-11-09T03:47:22.248249+0000","last_undegraded":"2023-11-09T03:47:22.248249+0000","last_fullsized":"2023-11-09T03:47:22.248249+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:52:58.422763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.933388+0000","last_change":"2023-11-09T03:47:03.146527+0000","last_active":"2023-11-09T03:47:20.933388+0000","last_peered":"2023-11-09T03:47:20.933388+0000","last_clean":"2023-11-09T03:47:20.933388+0000","last_became_active":"2023-11-09T03:47:01.587858+0000","last_became_peered":"2023-11-09T03:47:01.587858+0000","last_unstale":"2023-11-09T03:47:20.933388+0000","last_undegraded":"2023-11-09T03:47:20.933388+0000","last_fullsized":"2023-11-09T03:47:20.933388+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:56:04.201126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084439599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248163+0000","last_change":"2023-11-09T03:47:20.943815+0000","last_active":"2023-11-09T03:47:22.248163+0000","last_peered":"2023-11-09T03:47:22.248163+0000","last_clean":"2023-11-09T03:47:22.248163+0000","last_became_active":"2023-11-09T03:47:20.943403+0000","last_became_peered":"2023-11-09T03:47:20.943403+0000","last_unstale":"2023-11-09T03:47:22.248163+0000","last_undegraded":"2023-11-09T03:47:22.248163+0000","last_fullsized":"2023-11-09T03:47:22.248163+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:59:34.738692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247950+0000","last_change":"2023-11-09T03:47:19.949858+0000","last_active":"2023-11-09T03:47:22.247950+0000","last_peered":"2023-11-09T03:47:22.247950+0000","last_clean":"2023-11-09T03:47:22.247950+0000","last_became_active":"2023-11-09T03:47:19.949636+0000","last_became_peered":"2023-11-09T03:47:19.949636+0000","last_unstale":"2023-11-09T03:47:22.247950+0000","last_undegraded":"2023-11-09T03:47:22.247950+0000","last_fullsized":"2023-11-09T03:47:22.247950+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:24.100578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.935999+0000","last_change":"2023-11-09T03:47:02.725223+0000","last_active":"2023-11-09T03:47:20.935999+0000","last_peered":"2023-11-09T03:47:20.935999+0000","last_clean":"2023-11-09T03:47:20.935999+0000","last_became_active":"2023-11-09T03:47:01.590635+0000","last_became_peered":"2023-11-09T03:47:01.590635+0000","last_unstale":"2023-11-09T03:47:20.935999+0000","last_undegraded":"2023-11-09T03:47:20.935999+0000","last_fullsized":"2023-11-09T03:47:20.935999+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:05.754357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051685100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248165+0000","last_change":"2023-11-09T03:47:20.948694+0000","last_active":"2023-11-09T03:47:22.248165+0000","last_peered":"2023-11-09T03:47:22.248165+0000","last_clean":"2023-11-09T03:47:22.248165+0000","last_became_active":"2023-11-09T03:47:20.948405+0000","last_became_peered":"2023-11-09T03:47:20.948405+0000","last_unstale":"2023-11-09T03:47:22.248165+0000","last_undegraded":"2023-11-09T03:47:22.248165+0000","last_fullsized":"2023-11-09T03:47:22.248165+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:30:50.805289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940244+0000","last_change":"2023-11-09T03:47:19.942483+0000","last_active":"2023-11-09T03:47:21.940244+0000","last_peered":"2023-11-09T03:47:21.940244+0000","last_clean":"2023-11-09T03:47:21.940244+0000","last_became_active":"2023-11-09T03:47:19.942157+0000","last_became_peered":"2023-11-09T03:47:19.942157+0000","last_unstale":"2023-11-09T03:47:21.940244+0000","last_undegraded":"2023-11-09T03:47:21.940244+0000","last_fullsized":"2023-11-09T03:47:21.940244+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:59:28.149940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.937687+0000","last_change":"2023-11-09T03:47:02.582739+0000","last_active":"2023-11-09T03:47:20.937687+0000","last_peered":"2023-11-09T03:47:20.937687+0000","last_clean":"2023-11-09T03:47:20.937687+0000","last_became_active":"2023-11-09T03:47:01.592743+0000","last_became_peered":"2023-11-09T03:47:01.592743+0000","last_unstale":"2023-11-09T03:47:20.937687+0000","last_undegraded":"2023-11-09T03:47:20.937687+0000","last_fullsized":"2023-11-09T03:47:20.937687+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051802700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939444+0000","last_change":"2023-11-09T03:47:20.939366+0000","last_active":"2023-11-09T03:47:21.939444+0000","last_peered":"2023-11-09T03:47:21.939444+0000","last_clean":"2023-11-09T03:47:21.939444+0000","last_became_active":"2023-11-09T03:47:20.938258+0000","last_became_peered":"2023-11-09T03:47:20.938258+0000","last_unstale":"2023-11-09T03:47:21.939444+0000","last_undegraded":"2023-11-09T03:47:21.939444+0000","last_fullsized":"2023-11-09T03:47:21.939444+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:48:27.582736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"53'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248057+0000","last_change":"2023-11-09T03:47:19.946306+0000","last_active":"2023-11-09T03:47:22.248057+0000","last_peered":"2023-11-09T03:47:22.248057+0000","last_clean":"2023-11-09T03:47:22.248057+0000","last_became_active":"2023-11-09T03:47:19.946062+0000","last_became_peered":"2023-11-09T03:47:19.946062+0000","last_unstale":"2023-11-09T03:47:22.248057+0000","last_undegraded":"2023-11-09T03:47:22.248057+0000","last_fullsized":"2023-11-09T03:47:22.248057+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:51:52.875174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.935441+0000","last_change":"2023-11-09T03:47:02.724953+0000","last_active":"2023-11-09T03:47:20.935441+0000","last_peered":"2023-11-09T03:47:20.935441+0000","last_clean":"2023-11-09T03:47:20.935441+0000","last_became_active":"2023-11-09T03:47:01.587133+0000","last_became_peered":"2023-11-09T03:47:01.587133+0000","last_unstale":"2023-11-09T03:47:20.935441+0000","last_undegraded":"2023-11-09T03:47:20.935441+0000","last_fullsized":"2023-11-09T03:47:20.935441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:23:48.521161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066497599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249034+0000","last_change":"2023-11-09T03:47:20.942122+0000","last_active":"2023-11-09T03:47:22.249034+0000","last_peered":"2023-11-09T03:47:22.249034+0000","last_clean":"2023-11-09T03:47:22.249034+0000","last_became_active":"2023-11-09T03:47:20.941870+0000","last_became_peered":"2023-11-09T03:47:20.941870+0000","last_unstale":"2023-11-09T03:47:22.249034+0000","last_undegraded":"2023-11-09T03:47:22.249034+0000","last_fullsized":"2023-11-09T03:47:22.249034+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:57.187676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"53'4","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249334+0000","last_change":"2023-11-09T03:47:19.945714+0000","last_active":"2023-11-09T03:47:22.249334+0000","last_peered":"2023-11-09T03:47:22.249334+0000","last_clean":"2023-11-09T03:47:22.249334+0000","last_became_active":"2023-11-09T03:47:19.945455+0000","last_became_peered":"2023-11-09T03:47:19.945455+0000","last_unstale":"2023-11-09T03:47:22.249334+0000","last_undegraded":"2023-11-09T03:47:22.249334+0000","last_fullsized":"2023-11-09T03:47:22.249334+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:38:08.672664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.935591+0000","last_change":"2023-11-09T03:47:03.146217+0000","last_active":"2023-11-09T03:47:20.935591+0000","last_peered":"2023-11-09T03:47:20.935591+0000","last_clean":"2023-11-09T03:47:20.935591+0000","last_became_active":"2023-11-09T03:47:01.587598+0000","last_became_peered":"2023-11-09T03:47:01.587598+0000","last_unstale":"2023-11-09T03:47:20.935591+0000","last_undegraded":"2023-11-09T03:47:20.935591+0000","last_fullsized":"2023-11-09T03:47:20.935591+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:15:22.723455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090126400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249094+0000","last_change":"2023-11-09T03:47:20.950285+0000","last_active":"2023-11-09T03:47:22.249094+0000","last_peered":"2023-11-09T03:47:22.249094+0000","last_clean":"2023-11-09T03:47:22.249094+0000","last_became_active":"2023-11-09T03:47:20.949722+0000","last_became_peered":"2023-11-09T03:47:20.949722+0000","last_unstale":"2023-11-09T03:47:22.249094+0000","last_undegraded":"2023-11-09T03:47:22.249094+0000","last_fullsized":"2023-11-09T03:47:22.249094+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:51:53.879857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940663+0000","last_change":"2023-11-09T03:47:19.947977+0000","last_active":"2023-11-09T03:47:21.940663+0000","last_peered":"2023-11-09T03:47:21.940663+0000","last_clean":"2023-11-09T03:47:21.940663+0000","last_became_active":"2023-11-09T03:47:19.947549+0000","last_became_peered":"2023-11-09T03:47:19.947549+0000","last_unstale":"2023-11-09T03:47:21.940663+0000","last_undegraded":"2023-11-09T03:47:21.940663+0000","last_fullsized":"2023-11-09T03:47:21.940663+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:51.678872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931267+0000","last_change":"2023-11-09T03:47:03.148283+0000","last_active":"2023-11-09T03:47:20.931267+0000","last_peered":"2023-11-09T03:47:20.931267+0000","last_clean":"2023-11-09T03:47:20.931267+0000","last_became_active":"2023-11-09T03:47:01.588841+0000","last_became_peered":"2023-11-09T03:47:01.588841+0000","last_unstale":"2023-11-09T03:47:20.931267+0000","last_undegraded":"2023-11-09T03:47:20.931267+0000","last_fullsized":"2023-11-09T03:47:20.931267+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:47:44.880741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000436081,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249189+0000","last_change":"2023-11-09T03:47:20.937043+0000","last_active":"2023-11-09T03:47:22.249189+0000","last_peered":"2023-11-09T03:47:22.249189+0000","last_clean":"2023-11-09T03:47:22.249189+0000","last_became_active":"2023-11-09T03:47:20.936765+0000","last_became_peered":"2023-11-09T03:47:20.936765+0000","last_unstale":"2023-11-09T03:47:22.249189+0000","last_undegraded":"2023-11-09T03:47:22.249189+0000","last_fullsized":"2023-11-09T03:47:22.249189+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:14:00.724882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249189+0000","last_change":"2023-11-09T03:47:19.946355+0000","last_active":"2023-11-09T03:47:22.249189+0000","last_peered":"2023-11-09T03:47:22.249189+0000","last_clean":"2023-11-09T03:47:22.249189+0000","last_became_active":"2023-11-09T03:47:19.945071+0000","last_became_peered":"2023-11-09T03:47:19.945071+0000","last_unstale":"2023-11-09T03:47:22.249189+0000","last_undegraded":"2023-11-09T03:47:22.249189+0000","last_fullsized":"2023-11-09T03:47:22.249189+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:07:31.918577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.937170+0000","last_change":"2023-11-09T03:47:02.618856+0000","last_active":"2023-11-09T03:47:20.937170+0000","last_peered":"2023-11-09T03:47:20.937170+0000","last_clean":"2023-11-09T03:47:20.937170+0000","last_became_active":"2023-11-09T03:47:01.590430+0000","last_became_peered":"2023-11-09T03:47:01.590430+0000","last_unstale":"2023-11-09T03:47:20.937170+0000","last_undegraded":"2023-11-09T03:47:20.937170+0000","last_fullsized":"2023-11-09T03:47:20.937170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:57:18.470305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055030200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026961+0000","last_change":"2023-11-09T03:47:20.948028+0000","last_active":"2023-11-09T03:47:22.026961+0000","last_peered":"2023-11-09T03:47:22.026961+0000","last_clean":"2023-11-09T03:47:22.026961+0000","last_became_active":"2023-11-09T03:47:20.947752+0000","last_became_peered":"2023-11-09T03:47:20.947752+0000","last_unstale":"2023-11-09T03:47:22.026961+0000","last_undegraded":"2023-11-09T03:47:22.026961+0000","last_fullsized":"2023-11-09T03:47:22.026961+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:51:53.879857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248095+0000","last_change":"2023-11-09T03:47:19.948154+0000","last_active":"2023-11-09T03:47:22.248095+0000","last_peered":"2023-11-09T03:47:22.248095+0000","last_clean":"2023-11-09T03:47:22.248095+0000","last_became_active":"2023-11-09T03:47:19.947812+0000","last_became_peered":"2023-11-09T03:47:19.947812+0000","last_unstale":"2023-11-09T03:47:22.248095+0000","last_undegraded":"2023-11-09T03:47:22.248095+0000","last_fullsized":"2023-11-09T03:47:22.248095+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:28:13.618166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.935365+0000","last_change":"2023-11-09T03:47:02.619144+0000","last_active":"2023-11-09T03:47:20.935365+0000","last_peered":"2023-11-09T03:47:20.935365+0000","last_clean":"2023-11-09T03:47:20.935365+0000","last_became_active":"2023-11-09T03:47:01.590672+0000","last_became_peered":"2023-11-09T03:47:01.590672+0000","last_unstale":"2023-11-09T03:47:20.935365+0000","last_undegraded":"2023-11-09T03:47:20.935365+0000","last_fullsized":"2023-11-09T03:47:20.935365+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:03.025119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060997500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248188+0000","last_change":"2023-11-09T03:47:20.950877+0000","last_active":"2023-11-09T03:47:22.248188+0000","last_peered":"2023-11-09T03:47:22.248188+0000","last_clean":"2023-11-09T03:47:22.248188+0000","last_became_active":"2023-11-09T03:47:20.950616+0000","last_became_peered":"2023-11-09T03:47:20.950616+0000","last_unstale":"2023-11-09T03:47:22.248188+0000","last_undegraded":"2023-11-09T03:47:22.248188+0000","last_fullsized":"2023-11-09T03:47:22.248188+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:30.970511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939086+0000","last_change":"2023-11-09T03:47:19.940824+0000","last_active":"2023-11-09T03:47:21.939086+0000","last_peered":"2023-11-09T03:47:21.939086+0000","last_clean":"2023-11-09T03:47:21.939086+0000","last_became_active":"2023-11-09T03:47:19.940553+0000","last_became_peered":"2023-11-09T03:47:19.940553+0000","last_unstale":"2023-11-09T03:47:21.939086+0000","last_undegraded":"2023-11-09T03:47:21.939086+0000","last_fullsized":"2023-11-09T03:47:21.939086+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:52:58.422763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.934932+0000","last_change":"2023-11-09T03:47:02.619079+0000","last_active":"2023-11-09T03:47:20.934932+0000","last_peered":"2023-11-09T03:47:20.934932+0000","last_clean":"2023-11-09T03:47:20.934932+0000","last_became_active":"2023-11-09T03:47:01.591421+0000","last_became_peered":"2023-11-09T03:47:01.591421+0000","last_unstale":"2023-11-09T03:47:20.934932+0000","last_undegraded":"2023-11-09T03:47:20.934932+0000","last_fullsized":"2023-11-09T03:47:20.934932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:18:58.230606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068736399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248346+0000","last_change":"2023-11-09T03:47:20.943642+0000","last_active":"2023-11-09T03:47:22.248346+0000","last_peered":"2023-11-09T03:47:22.248346+0000","last_clean":"2023-11-09T03:47:22.248346+0000","last_became_active":"2023-11-09T03:47:20.943101+0000","last_became_peered":"2023-11-09T03:47:20.943101+0000","last_unstale":"2023-11-09T03:47:22.248346+0000","last_undegraded":"2023-11-09T03:47:22.248346+0000","last_fullsized":"2023-11-09T03:47:22.248346+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:06:11.677208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248583+0000","last_change":"2023-11-09T03:47:19.943530+0000","last_active":"2023-11-09T03:47:22.248583+0000","last_peered":"2023-11-09T03:47:22.248583+0000","last_clean":"2023-11-09T03:47:22.248583+0000","last_became_active":"2023-11-09T03:47:19.943303+0000","last_became_peered":"2023-11-09T03:47:19.943303+0000","last_unstale":"2023-11-09T03:47:22.248583+0000","last_undegraded":"2023-11-09T03:47:22.248583+0000","last_fullsized":"2023-11-09T03:47:22.248583+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:35.294223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931280+0000","last_change":"2023-11-09T03:47:03.148529+0000","last_active":"2023-11-09T03:47:20.931280+0000","last_peered":"2023-11-09T03:47:20.931280+0000","last_clean":"2023-11-09T03:47:20.931280+0000","last_became_active":"2023-11-09T03:47:01.584148+0000","last_became_peered":"2023-11-09T03:47:01.584148+0000","last_unstale":"2023-11-09T03:47:20.931280+0000","last_undegraded":"2023-11-09T03:47:20.931280+0000","last_fullsized":"2023-11-09T03:47:20.931280+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:40:00.237248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061475600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.938959+0000","last_change":"2023-11-09T03:47:20.942889+0000","last_active":"2023-11-09T03:47:21.938959+0000","last_peered":"2023-11-09T03:47:21.938959+0000","last_clean":"2023-11-09T03:47:21.938959+0000","last_became_active":"2023-11-09T03:47:20.941326+0000","last_became_peered":"2023-11-09T03:47:20.941326+0000","last_unstale":"2023-11-09T03:47:21.938959+0000","last_undegraded":"2023-11-09T03:47:21.938959+0000","last_fullsized":"2023-11-09T03:47:21.938959+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:56:23.552030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.938908+0000","last_change":"2023-11-09T03:47:19.942969+0000","last_active":"2023-11-09T03:47:21.938908+0000","last_peered":"2023-11-09T03:47:21.938908+0000","last_clean":"2023-11-09T03:47:21.938908+0000","last_became_active":"2023-11-09T03:47:19.942726+0000","last_became_peered":"2023-11-09T03:47:19.942726+0000","last_unstale":"2023-11-09T03:47:21.938908+0000","last_undegraded":"2023-11-09T03:47:21.938908+0000","last_fullsized":"2023-11-09T03:47:21.938908+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:51:50.362425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.933718+0000","last_change":"2023-11-09T03:47:03.148217+0000","last_active":"2023-11-09T03:47:20.933718+0000","last_peered":"2023-11-09T03:47:20.933718+0000","last_clean":"2023-11-09T03:47:20.933718+0000","last_became_active":"2023-11-09T03:47:01.593424+0000","last_became_peered":"2023-11-09T03:47:01.593424+0000","last_unstale":"2023-11-09T03:47:20.933718+0000","last_undegraded":"2023-11-09T03:47:20.933718+0000","last_fullsized":"2023-11-09T03:47:20.933718+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:19:32.693691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001769865,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.941149+0000","last_change":"2023-11-09T03:47:20.942781+0000","last_active":"2023-11-09T03:47:21.941149+0000","last_peered":"2023-11-09T03:47:21.941149+0000","last_clean":"2023-11-09T03:47:21.941149+0000","last_became_active":"2023-11-09T03:47:20.942498+0000","last_became_peered":"2023-11-09T03:47:20.942498+0000","last_unstale":"2023-11-09T03:47:21.941149+0000","last_undegraded":"2023-11-09T03:47:21.941149+0000","last_fullsized":"2023-11-09T03:47:21.941149+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:23:50.235725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249315+0000","last_change":"2023-11-09T03:47:19.944843+0000","last_active":"2023-11-09T03:47:22.249315+0000","last_peered":"2023-11-09T03:47:22.249315+0000","last_clean":"2023-11-09T03:47:22.249315+0000","last_became_active":"2023-11-09T03:47:19.944583+0000","last_became_peered":"2023-11-09T03:47:19.944583+0000","last_unstale":"2023-11-09T03:47:22.249315+0000","last_undegraded":"2023-11-09T03:47:22.249315+0000","last_fullsized":"2023-11-09T03:47:22.249315+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:46:53.255789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931015+0000","last_change":"2023-11-09T03:47:03.146470+0000","last_active":"2023-11-09T03:47:20.931015+0000","last_peered":"2023-11-09T03:47:20.931015+0000","last_clean":"2023-11-09T03:47:20.931015+0000","last_became_active":"2023-11-09T03:47:01.587304+0000","last_became_peered":"2023-11-09T03:47:01.587304+0000","last_unstale":"2023-11-09T03:47:20.931015+0000","last_undegraded":"2023-11-09T03:47:20.931015+0000","last_fullsized":"2023-11-09T03:47:20.931015+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:58:23.339287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00110395,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.941057+0000","last_change":"2023-11-09T03:47:20.941612+0000","last_active":"2023-11-09T03:47:21.941057+0000","last_peered":"2023-11-09T03:47:21.941057+0000","last_clean":"2023-11-09T03:47:21.941057+0000","last_became_active":"2023-11-09T03:47:20.941335+0000","last_became_peered":"2023-11-09T03:47:20.941335+0000","last_unstale":"2023-11-09T03:47:21.941057+0000","last_undegraded":"2023-11-09T03:47:21.941057+0000","last_fullsized":"2023-11-09T03:47:21.941057+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:05:32.662676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939435+0000","last_change":"2023-11-09T03:47:19.941173+0000","last_active":"2023-11-09T03:47:21.939435+0000","last_peered":"2023-11-09T03:47:21.939435+0000","last_clean":"2023-11-09T03:47:21.939435+0000","last_became_active":"2023-11-09T03:47:19.940928+0000","last_became_peered":"2023-11-09T03:47:19.940928+0000","last_unstale":"2023-11-09T03:47:21.939435+0000","last_undegraded":"2023-11-09T03:47:21.939435+0000","last_fullsized":"2023-11-09T03:47:21.939435+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:28:13.618166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.934203+0000","last_change":"2023-11-09T03:47:03.147367+0000","last_active":"2023-11-09T03:47:20.934203+0000","last_peered":"2023-11-09T03:47:20.934203+0000","last_clean":"2023-11-09T03:47:20.934203+0000","last_became_active":"2023-11-09T03:47:01.588359+0000","last_became_peered":"2023-11-09T03:47:01.588359+0000","last_unstale":"2023-11-09T03:47:20.934203+0000","last_undegraded":"2023-11-09T03:47:20.934203+0000","last_fullsized":"2023-11-09T03:47:20.934203+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:21:34.303068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014106660000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249223+0000","last_change":"2023-11-09T03:47:20.950444+0000","last_active":"2023-11-09T03:47:22.249223+0000","last_peered":"2023-11-09T03:47:22.249223+0000","last_clean":"2023-11-09T03:47:22.249223+0000","last_became_active":"2023-11-09T03:47:20.950038+0000","last_became_peered":"2023-11-09T03:47:20.950038+0000","last_unstale":"2023-11-09T03:47:22.249223+0000","last_undegraded":"2023-11-09T03:47:22.249223+0000","last_fullsized":"2023-11-09T03:47:22.249223+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:55:28.077165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940688+0000","last_change":"2023-11-09T03:47:19.943935+0000","last_active":"2023-11-09T03:47:21.940688+0000","last_peered":"2023-11-09T03:47:21.940688+0000","last_clean":"2023-11-09T03:47:21.940688+0000","last_became_active":"2023-11-09T03:47:19.943125+0000","last_became_peered":"2023-11-09T03:47:19.943125+0000","last_unstale":"2023-11-09T03:47:21.940688+0000","last_undegraded":"2023-11-09T03:47:21.940688+0000","last_fullsized":"2023-11-09T03:47:21.940688+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:06:16.671366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":27,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.928179+0000","last_change":"2023-11-09T03:47:02.581682+0000","last_active":"2023-11-09T03:47:20.928179+0000","last_peered":"2023-11-09T03:47:20.928179+0000","last_clean":"2023-11-09T03:47:20.928179+0000","last_became_active":"2023-11-09T03:47:01.593051+0000","last_became_peered":"2023-11-09T03:47:01.593051+0000","last_unstale":"2023-11-09T03:47:20.928179+0000","last_undegraded":"2023-11-09T03:47:20.928179+0000","last_fullsized":"2023-11-09T03:47:20.928179+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047277799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939213+0000","last_change":"2023-11-09T03:47:20.945402+0000","last_active":"2023-11-09T03:47:21.939213+0000","last_peered":"2023-11-09T03:47:21.939213+0000","last_clean":"2023-11-09T03:47:21.939213+0000","last_became_active":"2023-11-09T03:47:20.945145+0000","last_became_peered":"2023-11-09T03:47:20.945145+0000","last_unstale":"2023-11-09T03:47:21.939213+0000","last_undegraded":"2023-11-09T03:47:21.939213+0000","last_fullsized":"2023-11-09T03:47:21.939213+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:38:09.677347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248640+0000","last_change":"2023-11-09T03:47:19.942655+0000","last_active":"2023-11-09T03:47:22.248640+0000","last_peered":"2023-11-09T03:47:22.248640+0000","last_clean":"2023-11-09T03:47:22.248640+0000","last_became_active":"2023-11-09T03:47:19.942380+0000","last_became_peered":"2023-11-09T03:47:19.942380+0000","last_unstale":"2023-11-09T03:47:22.248640+0000","last_undegraded":"2023-11-09T03:47:22.248640+0000","last_fullsized":"2023-11-09T03:47:22.248640+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:52:58.422763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.933457+0000","last_change":"2023-11-09T03:47:03.146309+0000","last_active":"2023-11-09T03:47:20.933457+0000","last_peered":"2023-11-09T03:47:20.933457+0000","last_clean":"2023-11-09T03:47:20.933457+0000","last_became_active":"2023-11-09T03:47:01.587560+0000","last_became_peered":"2023-11-09T03:47:01.587560+0000","last_unstale":"2023-11-09T03:47:20.933457+0000","last_undegraded":"2023-11-09T03:47:20.933457+0000","last_fullsized":"2023-11-09T03:47:20.933457+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:27:55.271945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070988400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249299+0000","last_change":"2023-11-09T03:47:20.941432+0000","last_active":"2023-11-09T03:47:22.249299+0000","last_peered":"2023-11-09T03:47:22.249299+0000","last_clean":"2023-11-09T03:47:22.249299+0000","last_became_active":"2023-11-09T03:47:20.941186+0000","last_became_peered":"2023-11-09T03:47:20.941186+0000","last_unstale":"2023-11-09T03:47:22.249299+0000","last_undegraded":"2023-11-09T03:47:22.249299+0000","last_fullsized":"2023-11-09T03:47:22.249299+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:25:38.423194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939102+0000","last_change":"2023-11-09T03:47:19.947334+0000","last_active":"2023-11-09T03:47:21.939102+0000","last_peered":"2023-11-09T03:47:21.939102+0000","last_clean":"2023-11-09T03:47:21.939102+0000","last_became_active":"2023-11-09T03:47:19.947091+0000","last_became_peered":"2023-11-09T03:47:19.947091+0000","last_unstale":"2023-11-09T03:47:21.939102+0000","last_undegraded":"2023-11-09T03:47:21.939102+0000","last_fullsized":"2023-11-09T03:47:21.939102+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:24:06.867382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.939549+0000","last_change":"2023-11-09T03:47:02.725125+0000","last_active":"2023-11-09T03:47:20.939549+0000","last_peered":"2023-11-09T03:47:20.939549+0000","last_clean":"2023-11-09T03:47:20.939549+0000","last_became_active":"2023-11-09T03:47:01.591521+0000","last_became_peered":"2023-11-09T03:47:01.591521+0000","last_unstale":"2023-11-09T03:47:20.939549+0000","last_undegraded":"2023-11-09T03:47:20.939549+0000","last_fullsized":"2023-11-09T03:47:20.939549+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:52:40.076542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000758675,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940341+0000","last_change":"2023-11-09T03:47:20.938307+0000","last_active":"2023-11-09T03:47:21.940341+0000","last_peered":"2023-11-09T03:47:21.940341+0000","last_clean":"2023-11-09T03:47:21.940341+0000","last_became_active":"2023-11-09T03:47:20.937853+0000","last_became_peered":"2023-11-09T03:47:20.937853+0000","last_unstale":"2023-11-09T03:47:21.940341+0000","last_undegraded":"2023-11-09T03:47:21.940341+0000","last_fullsized":"2023-11-09T03:47:21.940341+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:57.187676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248805+0000","last_change":"2023-11-09T03:47:19.945739+0000","last_active":"2023-11-09T03:47:22.248805+0000","last_peered":"2023-11-09T03:47:22.248805+0000","last_clean":"2023-11-09T03:47:22.248805+0000","last_became_active":"2023-11-09T03:47:19.944990+0000","last_became_peered":"2023-11-09T03:47:19.944990+0000","last_unstale":"2023-11-09T03:47:22.248805+0000","last_undegraded":"2023-11-09T03:47:22.248805+0000","last_fullsized":"2023-11-09T03:47:22.248805+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:53:27.482165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.932096+0000","last_change":"2023-11-09T03:47:03.147289+0000","last_active":"2023-11-09T03:47:20.932096+0000","last_peered":"2023-11-09T03:47:20.932096+0000","last_clean":"2023-11-09T03:47:20.932096+0000","last_became_active":"2023-11-09T03:47:01.587784+0000","last_became_peered":"2023-11-09T03:47:01.587784+0000","last_unstale":"2023-11-09T03:47:20.932096+0000","last_undegraded":"2023-11-09T03:47:20.932096+0000","last_fullsized":"2023-11-09T03:47:20.932096+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:11:23.824112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038781299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247714+0000","last_change":"2023-11-09T03:47:20.937750+0000","last_active":"2023-11-09T03:47:22.247714+0000","last_peered":"2023-11-09T03:47:22.247714+0000","last_clean":"2023-11-09T03:47:22.247714+0000","last_became_active":"2023-11-09T03:47:20.937473+0000","last_became_peered":"2023-11-09T03:47:20.937473+0000","last_unstale":"2023-11-09T03:47:22.247714+0000","last_undegraded":"2023-11-09T03:47:22.247714+0000","last_fullsized":"2023-11-09T03:47:22.247714+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:06:17.676049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940071+0000","last_change":"2023-11-09T03:47:19.948440+0000","last_active":"2023-11-09T03:47:21.940071+0000","last_peered":"2023-11-09T03:47:21.940071+0000","last_clean":"2023-11-09T03:47:21.940071+0000","last_became_active":"2023-11-09T03:47:19.948199+0000","last_became_peered":"2023-11-09T03:47:19.948199+0000","last_unstale":"2023-11-09T03:47:21.940071+0000","last_undegraded":"2023-11-09T03:47:21.940071+0000","last_fullsized":"2023-11-09T03:47:21.940071+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:34:52.006539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.935047+0000","last_change":"2023-11-09T03:47:02.619346+0000","last_active":"2023-11-09T03:47:20.935047+0000","last_peered":"2023-11-09T03:47:20.935047+0000","last_clean":"2023-11-09T03:47:20.935047+0000","last_became_active":"2023-11-09T03:47:01.590389+0000","last_became_peered":"2023-11-09T03:47:01.590389+0000","last_unstale":"2023-11-09T03:47:20.935047+0000","last_undegraded":"2023-11-09T03:47:20.935047+0000","last_fullsized":"2023-11-09T03:47:20.935047+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000384072,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.941022+0000","last_change":"2023-11-09T03:47:20.937215+0000","last_active":"2023-11-09T03:47:21.941022+0000","last_peered":"2023-11-09T03:47:21.941022+0000","last_clean":"2023-11-09T03:47:21.941022+0000","last_became_active":"2023-11-09T03:47:20.936913+0000","last_became_peered":"2023-11-09T03:47:20.936913+0000","last_unstale":"2023-11-09T03:47:21.941022+0000","last_undegraded":"2023-11-09T03:47:21.941022+0000","last_fullsized":"2023-11-09T03:47:21.941022+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:25:38.423194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249234+0000","last_change":"2023-11-09T03:47:19.948214+0000","last_active":"2023-11-09T03:47:22.249234+0000","last_peered":"2023-11-09T03:47:22.249234+0000","last_clean":"2023-11-09T03:47:22.249234+0000","last_became_active":"2023-11-09T03:47:19.947966+0000","last_became_peered":"2023-11-09T03:47:19.947966+0000","last_unstale":"2023-11-09T03:47:22.249234+0000","last_undegraded":"2023-11-09T03:47:22.249234+0000","last_fullsized":"2023-11-09T03:47:22.249234+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:53:19.322920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.938897+0000","last_change":"2023-11-09T03:47:02.582393+0000","last_active":"2023-11-09T03:47:20.938897+0000","last_peered":"2023-11-09T03:47:20.938897+0000","last_clean":"2023-11-09T03:47:20.938897+0000","last_became_active":"2023-11-09T03:47:01.591752+0000","last_became_peered":"2023-11-09T03:47:01.591752+0000","last_unstale":"2023-11-09T03:47:20.938897+0000","last_undegraded":"2023-11-09T03:47:20.938897+0000","last_fullsized":"2023-11-09T03:47:20.938897+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:03.025119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067021599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.027114+0000","last_change":"2023-11-09T03:47:20.946342+0000","last_active":"2023-11-09T03:47:22.027114+0000","last_peered":"2023-11-09T03:47:22.027114+0000","last_clean":"2023-11-09T03:47:22.027114+0000","last_became_active":"2023-11-09T03:47:20.946014+0000","last_became_peered":"2023-11-09T03:47:20.946014+0000","last_unstale":"2023-11-09T03:47:22.027114+0000","last_undegraded":"2023-11-09T03:47:22.027114+0000","last_fullsized":"2023-11-09T03:47:22.027114+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:25:38.423194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"53'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248629+0000","last_change":"2023-11-09T03:47:19.944449+0000","last_active":"2023-11-09T03:47:22.248629+0000","last_peered":"2023-11-09T03:47:22.248629+0000","last_clean":"2023-11-09T03:47:22.248629+0000","last_became_active":"2023-11-09T03:47:19.944203+0000","last_became_peered":"2023-11-09T03:47:19.944203+0000","last_unstale":"2023-11-09T03:47:22.248629+0000","last_undegraded":"2023-11-09T03:47:22.248629+0000","last_fullsized":"2023-11-09T03:47:22.248629+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:51.678872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.929927+0000","last_change":"2023-11-09T03:47:03.028744+0000","last_active":"2023-11-09T03:47:20.929927+0000","last_peered":"2023-11-09T03:47:20.929927+0000","last_clean":"2023-11-09T03:47:20.929927+0000","last_became_active":"2023-11-09T03:47:01.593890+0000","last_became_peered":"2023-11-09T03:47:01.593890+0000","last_unstale":"2023-11-09T03:47:20.929927+0000","last_undegraded":"2023-11-09T03:47:20.929927+0000","last_fullsized":"2023-11-09T03:47:20.929927+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:05.754357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053008299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026854+0000","last_change":"2023-11-09T03:47:20.952365+0000","last_active":"2023-11-09T03:47:22.026854+0000","last_peered":"2023-11-09T03:47:22.026854+0000","last_clean":"2023-11-09T03:47:22.026854+0000","last_became_active":"2023-11-09T03:47:20.952080+0000","last_became_peered":"2023-11-09T03:47:20.952080+0000","last_unstale":"2023-11-09T03:47:22.026854+0000","last_undegraded":"2023-11-09T03:47:22.026854+0000","last_fullsized":"2023-11-09T03:47:22.026854+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:40.639821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249192+0000","last_change":"2023-11-09T03:47:19.946712+0000","last_active":"2023-11-09T03:47:22.249192+0000","last_peered":"2023-11-09T03:47:22.249192+0000","last_clean":"2023-11-09T03:47:22.249192+0000","last_became_active":"2023-11-09T03:47:19.946456+0000","last_became_peered":"2023-11-09T03:47:19.946456+0000","last_unstale":"2023-11-09T03:47:22.249192+0000","last_undegraded":"2023-11-09T03:47:22.249192+0000","last_fullsized":"2023-11-09T03:47:22.249192+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:39:07.929390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.936277+0000","last_change":"2023-11-09T03:47:03.146783+0000","last_active":"2023-11-09T03:47:20.936277+0000","last_peered":"2023-11-09T03:47:20.936277+0000","last_clean":"2023-11-09T03:47:20.936277+0000","last_became_active":"2023-11-09T03:47:01.586923+0000","last_became_peered":"2023-11-09T03:47:01.586923+0000","last_unstale":"2023-11-09T03:47:20.936277+0000","last_undegraded":"2023-11-09T03:47:20.936277+0000","last_fullsized":"2023-11-09T03:47:20.936277+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:37:50.326442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060384700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248492+0000","last_change":"2023-11-09T03:47:20.946978+0000","last_active":"2023-11-09T03:47:22.248492+0000","last_peered":"2023-11-09T03:47:22.248492+0000","last_clean":"2023-11-09T03:47:22.248492+0000","last_became_active":"2023-11-09T03:47:20.946667+0000","last_became_peered":"2023-11-09T03:47:20.946667+0000","last_unstale":"2023-11-09T03:47:22.248492+0000","last_undegraded":"2023-11-09T03:47:22.248492+0000","last_fullsized":"2023-11-09T03:47:22.248492+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:02:01.425453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"53'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940910+0000","last_change":"2023-11-09T03:47:19.945244+0000","last_active":"2023-11-09T03:47:21.940910+0000","last_peered":"2023-11-09T03:47:21.940910+0000","last_clean":"2023-11-09T03:47:21.940910+0000","last_became_active":"2023-11-09T03:47:19.944955+0000","last_became_peered":"2023-11-09T03:47:19.944955+0000","last_unstale":"2023-11-09T03:47:21.940910+0000","last_undegraded":"2023-11-09T03:47:21.940910+0000","last_fullsized":"2023-11-09T03:47:21.940910+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:12:47.685713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.933539+0000","last_change":"2023-11-09T03:47:03.146580+0000","last_active":"2023-11-09T03:47:20.933539+0000","last_peered":"2023-11-09T03:47:20.933539+0000","last_clean":"2023-11-09T03:47:20.933539+0000","last_became_active":"2023-11-09T03:47:01.589201+0000","last_became_peered":"2023-11-09T03:47:01.589201+0000","last_unstale":"2023-11-09T03:47:20.933539+0000","last_undegraded":"2023-11-09T03:47:20.933539+0000","last_fullsized":"2023-11-09T03:47:20.933539+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:14.300421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049706999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248489+0000","last_change":"2023-11-09T03:47:20.949430+0000","last_active":"2023-11-09T03:47:22.248489+0000","last_peered":"2023-11-09T03:47:22.248489+0000","last_clean":"2023-11-09T03:47:22.248489+0000","last_became_active":"2023-11-09T03:47:20.949177+0000","last_became_peered":"2023-11-09T03:47:20.949177+0000","last_unstale":"2023-11-09T03:47:22.248489+0000","last_undegraded":"2023-11-09T03:47:22.248489+0000","last_fullsized":"2023-11-09T03:47:22.248489+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:04:05.315163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248363+0000","last_change":"2023-11-09T03:47:19.946526+0000","last_active":"2023-11-09T03:47:22.248363+0000","last_peered":"2023-11-09T03:47:22.248363+0000","last_clean":"2023-11-09T03:47:22.248363+0000","last_became_active":"2023-11-09T03:47:19.946263+0000","last_became_peered":"2023-11-09T03:47:19.946263+0000","last_unstale":"2023-11-09T03:47:22.248363+0000","last_undegraded":"2023-11-09T03:47:22.248363+0000","last_fullsized":"2023-11-09T03:47:22.248363+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:24:06.867382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931584+0000","last_change":"2023-11-09T03:47:03.147851+0000","last_active":"2023-11-09T03:47:20.931584+0000","last_peered":"2023-11-09T03:47:20.931584+0000","last_clean":"2023-11-09T03:47:20.931584+0000","last_became_active":"2023-11-09T03:47:01.589893+0000","last_became_peered":"2023-11-09T03:47:01.589893+0000","last_unstale":"2023-11-09T03:47:20.931584+0000","last_undegraded":"2023-11-09T03:47:20.931584+0000","last_fullsized":"2023-11-09T03:47:20.931584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:08:17.183462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029151399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":194,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026619+0000","last_change":"2023-11-09T03:45:21.030926+0000","last_active":"2023-11-09T03:47:22.026619+0000","last_peered":"2023-11-09T03:47:22.026619+0000","last_clean":"2023-11-09T03:47:22.026619+0000","last_became_active":"2023-11-09T03:45:21.029801+0000","last_became_peered":"2023-11-09T03:45:21.029801+0000","last_unstale":"2023-11-09T03:47:22.026619+0000","last_undegraded":"2023-11-09T03:47:22.026619+0000","last_fullsized":"2023-11-09T03:47:22.026619+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_clean_scrub_stamp":"2023-11-09T03:45:19.768433+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:12:59.153713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28240,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343600,"statfs":{"total":96636764160,"available":96607846400,"internally_reserved":0,"allocated":1245184,"data_stored":881539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.501,"15min":0.498},"min":{"1min":0.34899999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.622},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.41199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.71499999999999997}]},{"osd":1,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.33000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.38100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.73099999999999998}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.52300000000000002,"15min":0.51400000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.68700000000000006,"15min":0.70799999999999996},"min":{"1min":0.41299999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.80700000000000005,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.63500000000000001}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.48699999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.68999999999999995}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.441},"min":{"1min":0.34100000000000003,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.49199999999999999,"15min":0.47899999999999998},"min":{"1min":0.38700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.38700000000000001}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.47199999999999998,"15min":0.45600000000000002},"min":{"1min":0.37,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.51100000000000001,"15min":0.497},"min":{"1min":0.39600000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.58699999999999997}]},{"osd":6,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55100000000000005,"15min":0.54100000000000004},"min":{"1min":0.44400000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.434,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.44}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":643072,"data_stored":291011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.499,"15min":0.501},"min":{"1min":0.35099999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63900000000000001,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.50600000000000001,"15min":0.495},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.69899999999999995}]},{"osd":1,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.495}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.375,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.53400000000000003}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.58899999999999997}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.503,"15min":0.503},"min":{"1min":0.316,"5min":0.313,"15min":0.313},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.68899999999999995}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.436,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55500000000000005,"15min":0.55800000000000005},"min":{"1min":0.34300000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.56699999999999995}]},{"osd":7,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.42899999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.44400000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.79400000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":651264,"data_stored":291663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.496,"15min":0.48599999999999999},"min":{"1min":0.38700000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.876},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.503,"15min":0.498},"min":{"1min":0.42399999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.71799999999999997,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45400000000000001}]},{"osd":1,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.439,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.69999999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.62},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.46700000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.76200000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.64100000000000001}]},{"osd":2,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.33300000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.67900000000000005,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.499,"5min":0.371,"15min":0.371},"max":{"1min":0.72799999999999998,"5min":0.745,"15min":0.745},"last":0.71799999999999997}]},{"osd":3,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.39700000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.76500000000000001,"5min":0.88,"15min":0.88},"last":0.76400000000000001}]},{"osd":4,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.78100000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.495},{"interface":"front","average":{"1min":0.504,"5min":0.48399999999999999,"15min":0.48099999999999998},"min":{"1min":0.33100000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.79000000000000004}]},{"osd":6,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.50900000000000001,"15min":0.51100000000000001},"min":{"1min":0.33400000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.74199999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51300000000000001,"15min":0.51100000000000001},"min":{"1min":0.35499999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66900000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.753}]},{"osd":7,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.38900000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.73499999999999999,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61899999999999999,"15min":0.62},"min":{"1min":0.41199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.752,"5min":1.01,"15min":1.01},"last":0.84899999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":643072,"data_stored":285650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67600000000000005,"5min":1.034,"15min":1.034},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.53600000000000003,"15min":0.54000000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68300000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.46899999999999997}]},{"osd":1,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.56899999999999995,"15min":0.57699999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68100000000000005,"5min":1.093,"15min":1.093},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.36899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72399999999999998,"5min":0.871,"15min":0.871},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76800000000000002,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70099999999999996,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.44700000000000001}]},{"osd":3,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71699999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.502},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54900000000000004,"15min":0.55200000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86599999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.433}]},{"osd":5,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54400000000000004,"15min":0.54300000000000004},"min":{"1min":0.37,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76000000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.33100000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.36899999999999999}]},{"osd":6,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.33800000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.51700000000000002}]},{"osd":7,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.62,"15min":0.63},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77700000000000002,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.95599999999999996,"5min":1.113,"15min":1.113},"last":0.54300000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":651264,"data_stored":291067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.5,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.215},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51100000000000001,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.23300000000000001}]},{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32900000000000001,"15min":0.33100000000000002},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.54200000000000004,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.19800000000000001},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.371,"15min":0.375},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.53100000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.26700000000000002}]},{"osd":2,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.27200000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.59099999999999997,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.56100000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.31900000000000001}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.70899999999999996,"5min":0.87,"15min":0.87},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60999999999999999,"15min":0.61599999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.75600000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.62,"15min":0.625},"min":{"1min":0.45700000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79900000000000004,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61699999999999999,"15min":0.623},"min":{"1min":0.42599999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.74299999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.621,"15min":0.627},"min":{"1min":0.42399999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81200000000000006,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59999999999999998,"15min":0.60399999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76600000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.66000000000000003}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77200000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63500000000000001,"15min":0.64200000000000002},"min":{"1min":0.433,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81299999999999994,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.58299999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":651264,"data_stored":291067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.46000000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.33400000000000002,"15min":0.33100000000000002},"min":{"1min":0.247,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.27600000000000002}]},{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32500000000000001,"15min":0.32600000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.495,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.249},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.35899999999999999,"15min":0.36299999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.48599999999999999,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.33200000000000002}]},{"osd":3,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.36199999999999999,"15min":0.36399999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.45700000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40200000000000002,"15min":0.40400000000000003},"min":{"1min":0.30299999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.52300000000000002,"5min":0.754,"15min":0.754},"last":0.35299999999999998}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72299999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81299999999999994,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.50900000000000001}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.39800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.75700000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.59599999999999997,"15min":0.60099999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.753,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.498}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60399999999999998,"15min":0.60899999999999999},"min":{"1min":0.44600000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.76100000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.71199999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.57699999999999996}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60699999999999998,"15min":0.61199999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.745,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.80000000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":647168,"data_stored":291101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.28699999999999998,"5min":0.31,"15min":0.314},"min":{"1min":0.20799999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.38900000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.315,"15min":0.311},"min":{"1min":0.24099999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.43099999999999999}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.311,"15min":0.309},"min":{"1min":0.253,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.378,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.34599999999999997}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36899999999999999,"15min":0.371},"min":{"1min":0.23899999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.47699999999999998,"5min":0.505,"15min":0.505},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.37,"15min":0.371},"min":{"1min":0.30299999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.45400000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.36599999999999999}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.41899999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.747,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.44400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.77600000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.62}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56200000000000006,"15min":0.55700000000000005},"min":{"1min":0.46100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.44500000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.72599999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.60699999999999998}]},{"osd":6,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.47799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56699999999999995,"15min":0.56799999999999995},"min":{"1min":0.433,"5min":0.379,"15min":0.379},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.70399999999999996}]},{"osd":7,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.42899999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.69799999999999995,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.72899999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.63400000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28228,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343612,"statfs":{"total":96636764160,"available":96607858688,"internally_reserved":0,"allocated":1236992,"data_stored":875981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.31900000000000001,"15min":0.32300000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.38200000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.29599999999999999,"5min":0.35499999999999998,"15min":0.36499999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.44500000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.28799999999999998}]},{"osd":2,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.35699999999999998,"15min":0.36399999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.44700000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.36699999999999999,"15min":0.371},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.47699999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.251}]},{"osd":3,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.38500000000000001,"15min":0.39100000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.46400000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.41499999999999998,"15min":0.42099999999999999},"min":{"1min":0.309,"5min":0.307,"15min":0.307},"max":{"1min":0.48999999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.379}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58899999999999997,"15min":0.59299999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60599999999999998,"15min":0.60899999999999999},"min":{"1min":0.436,"5min":0.318,"15min":0.318},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.621}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.36199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76800000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.629,"15min":0.63600000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78900000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.52100000000000002}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.64100000000000001,"15min":0.64700000000000002},"min":{"1min":0.42499999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81000000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.627,"15min":0.63200000000000001},"min":{"1min":0.41699999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59699999999999998}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.44400000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.42999999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.71799999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.67000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-09T03:47:26.194 INFO:tasks.ceph.mds.c.smithi076.stderr:2023-11-09T03:47:26.195+0000 7fb441e91640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:47:28.723 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-09T03:47:28.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.834+0000 7fbc31010640 1 Processor -- start 2023-11-09T03:47:28.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.834+0000 7fbc31010640 1 -- start start 2023-11-09T03:47:28.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.835+0000 7fbc31010640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbc2c0b07f0 0x7fbc2c0b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:28.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.835+0000 7fbc31010640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc2c0b1f80 con 0x7fbc2c0b23f0 2023-11-09T03:47:28.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.835+0000 7fbc31010640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc2c0b1190 con 0x7fbc2c15af10 2023-11-09T03:47:28.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.835+0000 7fbc31010640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc2c06e690 con 0x7fbc2c0b07f0 2023-11-09T03:47:28.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.835+0000 7fbc23fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbc2c0b07f0 0x7fbc2c0b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:28.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.835+0000 7fbc2b59e640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fbc2c0b23f0 0x7fbc2c0b2880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48030/0 (socket says 172.21.15.76:48030) 2023-11-09T03:47:28.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.835+0000 7fbc2b59e640 1 -- 172.21.15.76:0/2065554606 learned_addr learned my addr 172.21.15.76:0/2065554606 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:28.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.836+0000 7fbc23fff640 1 -- 172.21.15.76:0/2065554606 >> v1:172.21.15.113:6789/0 conn(0x7fbc2c15af10 legacy=0x7fbc2c0c9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:28.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.836+0000 7fbc23fff640 1 -- 172.21.15.76:0/2065554606 >> v1:172.21.15.76:6789/0 conn(0x7fbc2c0b23f0 legacy=0x7fbc2c0b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:28.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.836+0000 7fbc23fff640 1 -- 172.21.15.76:0/2065554606 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2c06db10 con 0x7fbc2c0b07f0 2023-11-09T03:47:28.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.836+0000 7fbc2a59c640 1 -- 172.21.15.76:0/2065554606 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2921460955 0 0) 0x7fbc2c0b1f80 con 0x7fbc2c0b23f0 2023-11-09T03:47:28.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.836+0000 7fbc23fff640 1 --2- 172.21.15.76:0/2065554606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc2c0b07f0 0x7fbc2c0b0c50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbc140097a0 tx=0x7fbc1402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d3f3ac2cf77a93 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:28.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.836+0000 7fbc2a59c640 1 -- 172.21.15.76:0/2065554606 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc14031830 con 0x7fbc2c0b07f0 2023-11-09T03:47:28.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.836+0000 7fbc2a59c640 1 -- 172.21.15.76:0/2065554606 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc14031990 con 0x7fbc2c0b07f0 2023-11-09T03:47:28.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.836+0000 7fbc2a59c640 1 -- 172.21.15.76:0/2065554606 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc14031d20 con 0x7fbc2c0b07f0 2023-11-09T03:47:28.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.837+0000 7fbc31010640 1 -- 172.21.15.76:0/2065554606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc2c0b07f0 msgr2=0x7fbc2c0b0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:28.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.837+0000 7fbc31010640 1 --2- 172.21.15.76:0/2065554606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc2c0b07f0 0x7fbc2c0b0c50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbc140097a0 tx=0x7fbc1402b8d0 comp rx=0 tx=0).stop 2023-11-09T03:47:28.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.837+0000 7fbc31010640 1 -- 172.21.15.76:0/2065554606 shutdown_connections 2023-11-09T03:47:28.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.837+0000 7fbc31010640 1 --2- 172.21.15.76:0/2065554606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc2c0b07f0 0x7fbc2c0b0c50 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:28.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.837+0000 7fbc31010640 1 -- 172.21.15.76:0/2065554606 >> 172.21.15.76:0/2065554606 conn(0x7fbc2c0cb7f0 msgr2=0x7fbc2c0cbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:28.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.837+0000 7fbc31010640 1 -- 172.21.15.76:0/2065554606 shutdown_connections 2023-11-09T03:47:28.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.837+0000 7fbc31010640 1 -- 172.21.15.76:0/2065554606 wait complete. 2023-11-09T03:47:28.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.838+0000 7fbc31010640 1 Processor -- start 2023-11-09T03:47:28.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.838+0000 7fbc31010640 1 -- start start 2023-11-09T03:47:28.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.839+0000 7fbc31010640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc2c0b07f0 0x7fbc2c1e0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:28.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.839+0000 7fbc31010640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc2c0b23f0 0x7fbc2c156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:28.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.839+0000 7fbc31010640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc2c15af10 0x7fbc2c1f0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:28.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.839+0000 7fbc31010640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c1f1270 con 0x7fbc2c0b07f0 2023-11-09T03:47:28.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.839+0000 7fbc31010640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c1f1550 con 0x7fbc2c0b23f0 2023-11-09T03:47:28.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.839+0000 7fbc31010640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c1e0000 con 0x7fbc2c15af10 2023-11-09T03:47:28.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.839+0000 7fbc2ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc2c0b07f0 0x7fbc2c1e0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:28.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.839+0000 7fbc2ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc2c0b07f0 0x7fbc2c1e0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42610/0 (socket says 172.21.15.76:42610) 2023-11-09T03:47:28.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.839+0000 7fbc2ad9d640 1 -- 172.21.15.76:0/2242902754 learned_addr learned my addr 172.21.15.76:0/2242902754 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:28.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.840+0000 7fbc23fff640 1 --2- 172.21.15.76:0/2242902754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc2c0b23f0 0x7fbc2c156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:28.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.840+0000 7fbc2ad9d640 1 -- 172.21.15.76:0/2242902754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc2c15af10 msgr2=0x7fbc2c1f0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:28.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.840+0000 7fbc2ad9d640 1 --2- 172.21.15.76:0/2242902754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc2c15af10 0x7fbc2c1f0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:28.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.840+0000 7fbc2ad9d640 1 -- 172.21.15.76:0/2242902754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc2c0b23f0 msgr2=0x7fbc2c156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:28.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.840+0000 7fbc2ad9d640 1 --2- 172.21.15.76:0/2242902754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc2c0b23f0 0x7fbc2c156760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:28.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.840+0000 7fbc2ad9d640 1 -- 172.21.15.76:0/2242902754 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc14009400 con 0x7fbc2c0b07f0 2023-11-09T03:47:28.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.840+0000 7fbc23fff640 1 --2- 172.21.15.76:0/2242902754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc2c0b23f0 0x7fbc2c156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:28.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.840+0000 7fbc2ad9d640 1 --2- 172.21.15.76:0/2242902754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc2c0b07f0 0x7fbc2c1e0ae0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c00c430 tx=0x7fbc1c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:28.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.840+0000 7fbc237fe640 1 -- 172.21.15.76:0/2242902754 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c011790 con 0x7fbc2c0b07f0 2023-11-09T03:47:28.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.840+0000 7fbc237fe640 1 -- 172.21.15.76:0/2242902754 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1c0118f0 con 0x7fbc2c0b07f0 2023-11-09T03:47:28.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.840+0000 7fbc31010640 1 -- 172.21.15.76:0/2242902754 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc2c1e02e0 con 0x7fbc2c0b07f0 2023-11-09T03:47:28.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.840+0000 7fbc237fe640 1 -- 172.21.15.76:0/2242902754 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c011ce0 con 0x7fbc2c0b07f0 2023-11-09T03:47:28.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.840+0000 7fbc31010640 1 -- 172.21.15.76:0/2242902754 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc2c1547e0 con 0x7fbc2c0b07f0 2023-11-09T03:47:28.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.841+0000 7fbc31010640 1 -- 172.21.15.76:0/2242902754 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc2c074290 con 0x7fbc2c0b07f0 2023-11-09T03:47:28.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.841+0000 7fbc237fe640 1 -- 172.21.15.76:0/2242902754 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbc1c01c460 con 0x7fbc2c0b07f0 2023-11-09T03:47:28.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.842+0000 7fbc237fe640 1 --2- 172.21.15.76:0/2242902754 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbc0c06ae70 0x7fbc0c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:28.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.842+0000 7fbc237fe640 1 -- 172.21.15.76:0/2242902754 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fbc1c089870 con 0x7fbc2c0b07f0 2023-11-09T03:47:28.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.842+0000 7fbc23fff640 1 --2- 172.21.15.76:0/2242902754 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbc0c06ae70 0x7fbc0c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:28.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.843+0000 7fbc23fff640 1 --2- 172.21.15.76:0/2242902754 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbc0c06ae70 0x7fbc0c06d330 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbc14036000 tx=0x7fbc1403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:28.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:28.846+0000 7fbc237fe640 1 -- 172.21.15.76:0/2242902754 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbc1c054680 con 0x7fbc2c0b07f0 2023-11-09T03:47:29.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.013+0000 7fbc31010640 1 -- 172.21.15.76:0/2242902754 --> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbc2c0b0080 con 0x7fbc0c06ae70 2023-11-09T03:47:29.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.022+0000 7fbc237fe640 1 -- 172.21.15.76:0/2242902754 <== mgr.4105 v2:172.21.15.113:6832/1264193042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+370169 (secure 0 0 0) 0x7fbc2c0b0080 con 0x7fbc0c06ae70 2023-11-09T03:47:29.025 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:29.027 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2023-11-09T03:47:29.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.026+0000 7fbc31010640 1 -- 172.21.15.76:0/2242902754 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbc0c06ae70 msgr2=0x7fbc0c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:29.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.026+0000 7fbc31010640 1 --2- 172.21.15.76:0/2242902754 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbc0c06ae70 0x7fbc0c06d330 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbc14036000 tx=0x7fbc1403c040 comp rx=0 tx=0).stop 2023-11-09T03:47:29.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.026+0000 7fbc31010640 1 -- 172.21.15.76:0/2242902754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc2c0b07f0 msgr2=0x7fbc2c1e0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:29.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.026+0000 7fbc31010640 1 --2- 172.21.15.76:0/2242902754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc2c0b07f0 0x7fbc2c1e0ae0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c00c430 tx=0x7fbc1c00c8f0 comp rx=0 tx=0).stop 2023-11-09T03:47:29.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.026+0000 7fbc31010640 1 -- 172.21.15.76:0/2242902754 shutdown_connections 2023-11-09T03:47:29.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.026+0000 7fbc31010640 1 --2- 172.21.15.76:0/2242902754 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbc0c06ae70 0x7fbc0c06d330 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:29.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.026+0000 7fbc31010640 1 --2- 172.21.15.76:0/2242902754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc2c15af10 0x7fbc2c1f0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:29.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.026+0000 7fbc31010640 1 --2- 172.21.15.76:0/2242902754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc2c0b23f0 0x7fbc2c156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:29.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.026+0000 7fbc31010640 1 --2- 172.21.15.76:0/2242902754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc2c0b07f0 0x7fbc2c1e0ae0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:29.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.026+0000 7fbc31010640 1 -- 172.21.15.76:0/2242902754 >> 172.21.15.76:0/2242902754 conn(0x7fbc2c0cb7f0 msgr2=0x7fbc2c0cbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:29.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.026+0000 7fbc31010640 1 -- 172.21.15.76:0/2242902754 shutdown_connections 2023-11-09T03:47:29.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.026+0000 7fbc31010640 1 -- 172.21.15.76:0/2242902754 wait complete. 2023-11-09T03:47:29.036 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2023-11-09T03:47:28.778806+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":258,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222452,"kb_used_data":6244,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752268,"statfs":{"total":773094113280,"available":772866322432,"internally_reserved":0,"allocated":6393856,"data_stored":3513025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001866"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248379+0000","last_change":"2023-11-09T03:47:20.941252+0000","last_active":"2023-11-09T03:47:22.248379+0000","last_peered":"2023-11-09T03:47:22.248379+0000","last_clean":"2023-11-09T03:47:22.248379+0000","last_became_active":"2023-11-09T03:47:20.940995+0000","last_became_peered":"2023-11-09T03:47:20.940995+0000","last_unstale":"2023-11-09T03:47:22.248379+0000","last_undegraded":"2023-11-09T03:47:22.248379+0000","last_fullsized":"2023-11-09T03:47:22.248379+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:44:27.004461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248080+0000","last_change":"2023-11-09T03:47:20.950878+0000","last_active":"2023-11-09T03:47:22.248080+0000","last_peered":"2023-11-09T03:47:22.248080+0000","last_clean":"2023-11-09T03:47:22.248080+0000","last_became_active":"2023-11-09T03:47:20.950611+0000","last_became_peered":"2023-11-09T03:47:20.950611+0000","last_unstale":"2023-11-09T03:47:22.248080+0000","last_undegraded":"2023-11-09T03:47:22.248080+0000","last_fullsized":"2023-11-09T03:47:22.248080+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:04:18.340460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026954+0000","last_change":"2023-11-09T03:47:20.945634+0000","last_active":"2023-11-09T03:47:22.026954+0000","last_peered":"2023-11-09T03:47:22.026954+0000","last_clean":"2023-11-09T03:47:22.026954+0000","last_became_active":"2023-11-09T03:47:20.945370+0000","last_became_peered":"2023-11-09T03:47:20.945370+0000","last_unstale":"2023-11-09T03:47:22.026954+0000","last_undegraded":"2023-11-09T03:47:22.026954+0000","last_fullsized":"2023-11-09T03:47:22.026954+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:57.187676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940818+0000","last_change":"2023-11-09T03:47:20.941855+0000","last_active":"2023-11-09T03:47:21.940818+0000","last_peered":"2023-11-09T03:47:21.940818+0000","last_clean":"2023-11-09T03:47:21.940818+0000","last_became_active":"2023-11-09T03:47:20.941598+0000","last_became_peered":"2023-11-09T03:47:20.941598+0000","last_unstale":"2023-11-09T03:47:21.940818+0000","last_undegraded":"2023-11-09T03:47:21.940818+0000","last_fullsized":"2023-11-09T03:47:21.940818+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:14:00.724882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248512+0000","last_change":"2023-11-09T03:47:20.947154+0000","last_active":"2023-11-09T03:47:22.248512+0000","last_peered":"2023-11-09T03:47:22.248512+0000","last_clean":"2023-11-09T03:47:22.248512+0000","last_became_active":"2023-11-09T03:47:20.946878+0000","last_became_peered":"2023-11-09T03:47:20.946878+0000","last_unstale":"2023-11-09T03:47:22.248512+0000","last_undegraded":"2023-11-09T03:47:22.248512+0000","last_fullsized":"2023-11-09T03:47:22.248512+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:41:16.327295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247898+0000","last_change":"2023-11-09T03:47:20.941698+0000","last_active":"2023-11-09T03:47:22.247898+0000","last_peered":"2023-11-09T03:47:22.247898+0000","last_clean":"2023-11-09T03:47:22.247898+0000","last_became_active":"2023-11-09T03:47:20.941432+0000","last_became_peered":"2023-11-09T03:47:20.941432+0000","last_unstale":"2023-11-09T03:47:22.247898+0000","last_undegraded":"2023-11-09T03:47:22.247898+0000","last_fullsized":"2023-11-09T03:47:22.247898+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:12:48.690396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247980+0000","last_change":"2023-11-09T03:47:20.948132+0000","last_active":"2023-11-09T03:47:22.247980+0000","last_peered":"2023-11-09T03:47:22.247980+0000","last_clean":"2023-11-09T03:47:22.247980+0000","last_became_active":"2023-11-09T03:47:20.947854+0000","last_became_peered":"2023-11-09T03:47:20.947854+0000","last_unstale":"2023-11-09T03:47:22.247980+0000","last_undegraded":"2023-11-09T03:47:22.247980+0000","last_fullsized":"2023-11-09T03:47:22.247980+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:38:09.677347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248627+0000","last_change":"2023-11-09T03:47:20.944502+0000","last_active":"2023-11-09T03:47:22.248627+0000","last_peered":"2023-11-09T03:47:22.248627+0000","last_clean":"2023-11-09T03:47:22.248627+0000","last_became_active":"2023-11-09T03:47:20.944214+0000","last_became_peered":"2023-11-09T03:47:20.944214+0000","last_unstale":"2023-11-09T03:47:22.248627+0000","last_undegraded":"2023-11-09T03:47:22.248627+0000","last_fullsized":"2023-11-09T03:47:22.248627+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:22:14.432893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248644+0000","last_change":"2023-11-09T03:47:20.954329+0000","last_active":"2023-11-09T03:47:22.248644+0000","last_peered":"2023-11-09T03:47:22.248644+0000","last_clean":"2023-11-09T03:47:22.248644+0000","last_became_active":"2023-11-09T03:47:20.953654+0000","last_became_peered":"2023-11-09T03:47:20.953654+0000","last_unstale":"2023-11-09T03:47:22.248644+0000","last_undegraded":"2023-11-09T03:47:22.248644+0000","last_fullsized":"2023-11-09T03:47:22.248644+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:48:27.582736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939384+0000","last_change":"2023-11-09T03:47:20.943196+0000","last_active":"2023-11-09T03:47:21.939384+0000","last_peered":"2023-11-09T03:47:21.939384+0000","last_clean":"2023-11-09T03:47:21.939384+0000","last_became_active":"2023-11-09T03:47:20.942943+0000","last_became_peered":"2023-11-09T03:47:20.942943+0000","last_unstale":"2023-11-09T03:47:21.939384+0000","last_undegraded":"2023-11-09T03:47:21.939384+0000","last_fullsized":"2023-11-09T03:47:21.939384+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:36:04.069418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939364+0000","last_change":"2023-11-09T03:47:20.943821+0000","last_active":"2023-11-09T03:47:21.939364+0000","last_peered":"2023-11-09T03:47:21.939364+0000","last_clean":"2023-11-09T03:47:21.939364+0000","last_became_active":"2023-11-09T03:47:20.943571+0000","last_became_peered":"2023-11-09T03:47:20.943571+0000","last_unstale":"2023-11-09T03:47:21.939364+0000","last_undegraded":"2023-11-09T03:47:21.939364+0000","last_fullsized":"2023-11-09T03:47:21.939364+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:26:12.103741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940598+0000","last_change":"2023-11-09T03:47:20.942582+0000","last_active":"2023-11-09T03:47:21.940598+0000","last_peered":"2023-11-09T03:47:21.940598+0000","last_clean":"2023-11-09T03:47:21.940598+0000","last_became_active":"2023-11-09T03:47:20.942214+0000","last_became_peered":"2023-11-09T03:47:20.942214+0000","last_unstale":"2023-11-09T03:47:21.940598+0000","last_undegraded":"2023-11-09T03:47:21.940598+0000","last_fullsized":"2023-11-09T03:47:21.940598+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:51:53.879857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248688+0000","last_change":"2023-11-09T03:47:20.942261+0000","last_active":"2023-11-09T03:47:22.248688+0000","last_peered":"2023-11-09T03:47:22.248688+0000","last_clean":"2023-11-09T03:47:22.248688+0000","last_became_active":"2023-11-09T03:47:20.941996+0000","last_became_peered":"2023-11-09T03:47:20.941996+0000","last_unstale":"2023-11-09T03:47:22.248688+0000","last_undegraded":"2023-11-09T03:47:22.248688+0000","last_fullsized":"2023-11-09T03:47:22.248688+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:21:04.459156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248066+0000","last_change":"2023-11-09T03:47:20.944425+0000","last_active":"2023-11-09T03:47:22.248066+0000","last_peered":"2023-11-09T03:47:22.248066+0000","last_clean":"2023-11-09T03:47:22.248066+0000","last_became_active":"2023-11-09T03:47:20.944159+0000","last_became_peered":"2023-11-09T03:47:20.944159+0000","last_unstale":"2023-11-09T03:47:22.248066+0000","last_undegraded":"2023-11-09T03:47:22.248066+0000","last_fullsized":"2023-11-09T03:47:22.248066+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:26:12.103741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248162+0000","last_change":"2023-11-09T03:47:20.954225+0000","last_active":"2023-11-09T03:47:22.248162+0000","last_peered":"2023-11-09T03:47:22.248162+0000","last_clean":"2023-11-09T03:47:22.248162+0000","last_became_active":"2023-11-09T03:47:20.954004+0000","last_became_peered":"2023-11-09T03:47:20.954004+0000","last_unstale":"2023-11-09T03:47:22.248162+0000","last_undegraded":"2023-11-09T03:47:22.248162+0000","last_fullsized":"2023-11-09T03:47:22.248162+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:36:04.069418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247697+0000","last_change":"2023-11-09T03:47:20.940209+0000","last_active":"2023-11-09T03:47:22.247697+0000","last_peered":"2023-11-09T03:47:22.247697+0000","last_clean":"2023-11-09T03:47:22.247697+0000","last_became_active":"2023-11-09T03:47:20.939956+0000","last_became_peered":"2023-11-09T03:47:20.939956+0000","last_unstale":"2023-11-09T03:47:22.247697+0000","last_undegraded":"2023-11-09T03:47:22.247697+0000","last_fullsized":"2023-11-09T03:47:22.247697+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:52.683555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248360+0000","last_change":"2023-11-09T03:47:20.942850+0000","last_active":"2023-11-09T03:47:22.248360+0000","last_peered":"2023-11-09T03:47:22.248360+0000","last_clean":"2023-11-09T03:47:22.248360+0000","last_became_active":"2023-11-09T03:47:20.942596+0000","last_became_peered":"2023-11-09T03:47:20.942596+0000","last_unstale":"2023-11-09T03:47:22.248360+0000","last_undegraded":"2023-11-09T03:47:22.248360+0000","last_fullsized":"2023-11-09T03:47:22.248360+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:14:28.441176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.027085+0000","last_change":"2023-11-09T03:47:20.954037+0000","last_active":"2023-11-09T03:47:22.027085+0000","last_peered":"2023-11-09T03:47:22.027085+0000","last_clean":"2023-11-09T03:47:22.027085+0000","last_became_active":"2023-11-09T03:47:20.953777+0000","last_became_peered":"2023-11-09T03:47:20.953777+0000","last_unstale":"2023-11-09T03:47:22.027085+0000","last_undegraded":"2023-11-09T03:47:22.027085+0000","last_fullsized":"2023-11-09T03:47:22.027085+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:55:28.077165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248233+0000","last_change":"2023-11-09T03:47:20.941849+0000","last_active":"2023-11-09T03:47:22.248233+0000","last_peered":"2023-11-09T03:47:22.248233+0000","last_clean":"2023-11-09T03:47:22.248233+0000","last_became_active":"2023-11-09T03:47:20.941602+0000","last_became_peered":"2023-11-09T03:47:20.941602+0000","last_unstale":"2023-11-09T03:47:22.248233+0000","last_undegraded":"2023-11-09T03:47:22.248233+0000","last_fullsized":"2023-11-09T03:47:22.248233+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:23.398916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248619+0000","last_change":"2023-11-09T03:47:20.951388+0000","last_active":"2023-11-09T03:47:22.248619+0000","last_peered":"2023-11-09T03:47:22.248619+0000","last_clean":"2023-11-09T03:47:22.248619+0000","last_became_active":"2023-11-09T03:47:20.951123+0000","last_became_peered":"2023-11-09T03:47:20.951123+0000","last_unstale":"2023-11-09T03:47:22.248619+0000","last_undegraded":"2023-11-09T03:47:22.248619+0000","last_fullsized":"2023-11-09T03:47:22.248619+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:55:54.333416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026719+0000","last_change":"2023-11-09T03:47:20.948114+0000","last_active":"2023-11-09T03:47:22.026719+0000","last_peered":"2023-11-09T03:47:22.026719+0000","last_clean":"2023-11-09T03:47:22.026719+0000","last_became_active":"2023-11-09T03:47:20.947849+0000","last_became_peered":"2023-11-09T03:47:20.947849+0000","last_unstale":"2023-11-09T03:47:22.026719+0000","last_undegraded":"2023-11-09T03:47:22.026719+0000","last_fullsized":"2023-11-09T03:47:22.026719+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:52.683555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247955+0000","last_change":"2023-11-09T03:47:20.945723+0000","last_active":"2023-11-09T03:47:22.247955+0000","last_peered":"2023-11-09T03:47:22.247955+0000","last_clean":"2023-11-09T03:47:22.247955+0000","last_became_active":"2023-11-09T03:47:20.945447+0000","last_became_peered":"2023-11-09T03:47:20.945447+0000","last_unstale":"2023-11-09T03:47:22.247955+0000","last_undegraded":"2023-11-09T03:47:22.247955+0000","last_fullsized":"2023-11-09T03:47:22.247955+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:41:48.124870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026634+0000","last_change":"2023-11-09T03:47:20.944344+0000","last_active":"2023-11-09T03:47:22.026634+0000","last_peered":"2023-11-09T03:47:22.026634+0000","last_clean":"2023-11-09T03:47:22.026634+0000","last_became_active":"2023-11-09T03:47:20.943991+0000","last_became_peered":"2023-11-09T03:47:20.943991+0000","last_unstale":"2023-11-09T03:47:22.026634+0000","last_undegraded":"2023-11-09T03:47:22.026634+0000","last_fullsized":"2023-11-09T03:47:22.026634+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:08:17.239107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248007+0000","last_change":"2023-11-09T03:47:20.949489+0000","last_active":"2023-11-09T03:47:22.248007+0000","last_peered":"2023-11-09T03:47:22.248007+0000","last_clean":"2023-11-09T03:47:22.248007+0000","last_became_active":"2023-11-09T03:47:20.949211+0000","last_became_peered":"2023-11-09T03:47:20.949211+0000","last_unstale":"2023-11-09T03:47:22.248007+0000","last_undegraded":"2023-11-09T03:47:22.248007+0000","last_fullsized":"2023-11-09T03:47:22.248007+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:56:40.000841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248496+0000","last_change":"2023-11-09T03:47:20.952835+0000","last_active":"2023-11-09T03:47:22.248496+0000","last_peered":"2023-11-09T03:47:22.248496+0000","last_clean":"2023-11-09T03:47:22.248496+0000","last_became_active":"2023-11-09T03:47:20.952578+0000","last_became_peered":"2023-11-09T03:47:20.952578+0000","last_unstale":"2023-11-09T03:47:22.248496+0000","last_undegraded":"2023-11-09T03:47:22.248496+0000","last_fullsized":"2023-11-09T03:47:22.248496+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:40:53.848669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.941032+0000","last_change":"2023-11-09T03:47:20.944379+0000","last_active":"2023-11-09T03:47:21.941032+0000","last_peered":"2023-11-09T03:47:21.941032+0000","last_clean":"2023-11-09T03:47:21.941032+0000","last_became_active":"2023-11-09T03:47:20.944134+0000","last_became_peered":"2023-11-09T03:47:20.944134+0000","last_unstale":"2023-11-09T03:47:21.941032+0000","last_undegraded":"2023-11-09T03:47:21.941032+0000","last_fullsized":"2023-11-09T03:47:21.941032+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:55:28.077165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248970+0000","last_change":"2023-11-09T03:47:20.940786+0000","last_active":"2023-11-09T03:47:22.248970+0000","last_peered":"2023-11-09T03:47:22.248970+0000","last_clean":"2023-11-09T03:47:22.248970+0000","last_became_active":"2023-11-09T03:47:20.940529+0000","last_became_peered":"2023-11-09T03:47:20.940529+0000","last_unstale":"2023-11-09T03:47:22.248970+0000","last_undegraded":"2023-11-09T03:47:22.248970+0000","last_fullsized":"2023-11-09T03:47:22.248970+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:40.639821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248129+0000","last_change":"2023-11-09T03:47:20.949337+0000","last_active":"2023-11-09T03:47:22.248129+0000","last_peered":"2023-11-09T03:47:22.248129+0000","last_clean":"2023-11-09T03:47:22.248129+0000","last_became_active":"2023-11-09T03:47:20.949088+0000","last_became_peered":"2023-11-09T03:47:20.949088+0000","last_unstale":"2023-11-09T03:47:22.248129+0000","last_undegraded":"2023-11-09T03:47:22.248129+0000","last_fullsized":"2023-11-09T03:47:22.248129+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:52:12.660646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248774+0000","last_change":"2023-11-09T03:47:20.948003+0000","last_active":"2023-11-09T03:47:22.248774+0000","last_peered":"2023-11-09T03:47:22.248774+0000","last_clean":"2023-11-09T03:47:22.248774+0000","last_became_active":"2023-11-09T03:47:20.947690+0000","last_became_peered":"2023-11-09T03:47:20.947690+0000","last_unstale":"2023-11-09T03:47:22.248774+0000","last_undegraded":"2023-11-09T03:47:22.248774+0000","last_fullsized":"2023-11-09T03:47:22.248774+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:28:18.013453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026610+0000","last_change":"2023-11-09T03:47:20.946260+0000","last_active":"2023-11-09T03:47:22.026610+0000","last_peered":"2023-11-09T03:47:22.026610+0000","last_clean":"2023-11-09T03:47:22.026610+0000","last_became_active":"2023-11-09T03:47:20.945986+0000","last_became_peered":"2023-11-09T03:47:20.945986+0000","last_unstale":"2023-11-09T03:47:22.026610+0000","last_undegraded":"2023-11-09T03:47:22.026610+0000","last_fullsized":"2023-11-09T03:47:22.026610+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:59:29.154623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247961+0000","last_change":"2023-11-09T03:47:20.952176+0000","last_active":"2023-11-09T03:47:22.247961+0000","last_peered":"2023-11-09T03:47:22.247961+0000","last_clean":"2023-11-09T03:47:22.247961+0000","last_became_active":"2023-11-09T03:47:20.951921+0000","last_became_peered":"2023-11-09T03:47:20.951921+0000","last_unstale":"2023-11-09T03:47:22.247961+0000","last_undegraded":"2023-11-09T03:47:22.247961+0000","last_fullsized":"2023-11-09T03:47:22.247961+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:22.376023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247669+0000","last_change":"2023-11-09T03:47:20.943940+0000","last_active":"2023-11-09T03:47:22.247669+0000","last_peered":"2023-11-09T03:47:22.247669+0000","last_clean":"2023-11-09T03:47:22.247669+0000","last_became_active":"2023-11-09T03:47:20.943708+0000","last_became_peered":"2023-11-09T03:47:20.943708+0000","last_unstale":"2023-11-09T03:47:22.247669+0000","last_undegraded":"2023-11-09T03:47:22.247669+0000","last_fullsized":"2023-11-09T03:47:22.247669+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:08:17.239107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026790+0000","last_change":"2023-11-09T03:47:20.942360+0000","last_active":"2023-11-09T03:47:22.026790+0000","last_peered":"2023-11-09T03:47:22.026790+0000","last_clean":"2023-11-09T03:47:22.026790+0000","last_became_active":"2023-11-09T03:47:20.942078+0000","last_became_peered":"2023-11-09T03:47:20.942078+0000","last_unstale":"2023-11-09T03:47:22.026790+0000","last_undegraded":"2023-11-09T03:47:22.026790+0000","last_fullsized":"2023-11-09T03:47:22.026790+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:12:48.690396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248715+0000","last_change":"2023-11-09T03:47:19.943955+0000","last_active":"2023-11-09T03:47:22.248715+0000","last_peered":"2023-11-09T03:47:22.248715+0000","last_clean":"2023-11-09T03:47:22.248715+0000","last_became_active":"2023-11-09T03:47:19.943692+0000","last_became_peered":"2023-11-09T03:47:19.943692+0000","last_unstale":"2023-11-09T03:47:22.248715+0000","last_undegraded":"2023-11-09T03:47:22.248715+0000","last_fullsized":"2023-11-09T03:47:22.248715+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:24.100578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":27,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.928645+0000","last_change":"2023-11-09T03:47:02.581917+0000","last_active":"2023-11-09T03:47:20.928645+0000","last_peered":"2023-11-09T03:47:20.928645+0000","last_clean":"2023-11-09T03:47:20.928645+0000","last_became_active":"2023-11-09T03:47:01.588082+0000","last_became_peered":"2023-11-09T03:47:01.588082+0000","last_unstale":"2023-11-09T03:47:20.928645+0000","last_undegraded":"2023-11-09T03:47:20.928645+0000","last_fullsized":"2023-11-09T03:47:20.928645+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:44:07.653557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034538899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940245+0000","last_change":"2023-11-09T03:47:20.938900+0000","last_active":"2023-11-09T03:47:21.940245+0000","last_peered":"2023-11-09T03:47:21.940245+0000","last_clean":"2023-11-09T03:47:21.940245+0000","last_became_active":"2023-11-09T03:47:20.938654+0000","last_became_peered":"2023-11-09T03:47:20.938654+0000","last_unstale":"2023-11-09T03:47:21.940245+0000","last_undegraded":"2023-11-09T03:47:21.940245+0000","last_fullsized":"2023-11-09T03:47:21.940245+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:40.639821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248651+0000","last_change":"2023-11-09T03:47:19.945834+0000","last_active":"2023-11-09T03:47:22.248651+0000","last_peered":"2023-11-09T03:47:22.248651+0000","last_clean":"2023-11-09T03:47:22.248651+0000","last_became_active":"2023-11-09T03:47:19.945484+0000","last_became_peered":"2023-11-09T03:47:19.945484+0000","last_unstale":"2023-11-09T03:47:22.248651+0000","last_undegraded":"2023-11-09T03:47:22.248651+0000","last_fullsized":"2023-11-09T03:47:22.248651+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:57:21.887088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.930614+0000","last_change":"2023-11-09T03:47:03.029063+0000","last_active":"2023-11-09T03:47:20.930614+0000","last_peered":"2023-11-09T03:47:20.930614+0000","last_clean":"2023-11-09T03:47:20.930614+0000","last_became_active":"2023-11-09T03:47:01.593582+0000","last_became_peered":"2023-11-09T03:47:01.593582+0000","last_unstale":"2023-11-09T03:47:20.930614+0000","last_undegraded":"2023-11-09T03:47:20.930614+0000","last_fullsized":"2023-11-09T03:47:20.930614+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:56:20.649937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041855100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026721+0000","last_change":"2023-11-09T03:47:20.944419+0000","last_active":"2023-11-09T03:47:22.026721+0000","last_peered":"2023-11-09T03:47:22.026721+0000","last_clean":"2023-11-09T03:47:22.026721+0000","last_became_active":"2023-11-09T03:47:20.944141+0000","last_became_peered":"2023-11-09T03:47:20.944141+0000","last_unstale":"2023-11-09T03:47:22.026721+0000","last_undegraded":"2023-11-09T03:47:22.026721+0000","last_fullsized":"2023-11-09T03:47:22.026721+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:06:17.676049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"53'8","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026516+0000","last_change":"2023-11-09T03:47:19.946625+0000","last_active":"2023-11-09T03:47:22.026516+0000","last_peered":"2023-11-09T03:47:22.026516+0000","last_clean":"2023-11-09T03:47:22.026516+0000","last_became_active":"2023-11-09T03:47:19.946350+0000","last_became_peered":"2023-11-09T03:47:19.946350+0000","last_unstale":"2023-11-09T03:47:22.026516+0000","last_undegraded":"2023-11-09T03:47:22.026516+0000","last_fullsized":"2023-11-09T03:47:22.026516+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:32.646643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.934959+0000","last_change":"2023-11-09T03:47:02.619336+0000","last_active":"2023-11-09T03:47:20.934959+0000","last_peered":"2023-11-09T03:47:20.934959+0000","last_clean":"2023-11-09T03:47:20.934959+0000","last_became_active":"2023-11-09T03:47:01.587749+0000","last_became_peered":"2023-11-09T03:47:01.587749+0000","last_unstale":"2023-11-09T03:47:20.934959+0000","last_undegraded":"2023-11-09T03:47:20.934959+0000","last_fullsized":"2023-11-09T03:47:20.934959+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063032099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247912+0000","last_change":"2023-11-09T03:47:20.948391+0000","last_active":"2023-11-09T03:47:22.247912+0000","last_peered":"2023-11-09T03:47:22.247912+0000","last_clean":"2023-11-09T03:47:22.247912+0000","last_became_active":"2023-11-09T03:47:20.948103+0000","last_became_peered":"2023-11-09T03:47:20.948103+0000","last_unstale":"2023-11-09T03:47:22.247912+0000","last_undegraded":"2023-11-09T03:47:22.247912+0000","last_fullsized":"2023-11-09T03:47:22.247912+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:33.651326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940202+0000","last_change":"2023-11-09T03:47:19.944514+0000","last_active":"2023-11-09T03:47:21.940202+0000","last_peered":"2023-11-09T03:47:21.940202+0000","last_clean":"2023-11-09T03:47:21.940202+0000","last_became_active":"2023-11-09T03:47:19.944230+0000","last_became_peered":"2023-11-09T03:47:19.944230+0000","last_unstale":"2023-11-09T03:47:21.940202+0000","last_undegraded":"2023-11-09T03:47:21.940202+0000","last_fullsized":"2023-11-09T03:47:21.940202+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:08:16.234424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":27,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.928406+0000","last_change":"2023-11-09T03:47:02.581658+0000","last_active":"2023-11-09T03:47:20.928406+0000","last_peered":"2023-11-09T03:47:20.928406+0000","last_clean":"2023-11-09T03:47:20.928406+0000","last_became_active":"2023-11-09T03:47:01.588612+0000","last_became_peered":"2023-11-09T03:47:01.588612+0000","last_unstale":"2023-11-09T03:47:20.928406+0000","last_undegraded":"2023-11-09T03:47:20.928406+0000","last_fullsized":"2023-11-09T03:47:20.928406+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045238500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.027066+0000","last_change":"2023-11-09T03:47:20.952326+0000","last_active":"2023-11-09T03:47:22.027066+0000","last_peered":"2023-11-09T03:47:22.027066+0000","last_clean":"2023-11-09T03:47:22.027066+0000","last_became_active":"2023-11-09T03:47:20.952035+0000","last_became_peered":"2023-11-09T03:47:20.952035+0000","last_unstale":"2023-11-09T03:47:22.027066+0000","last_undegraded":"2023-11-09T03:47:22.027066+0000","last_fullsized":"2023-11-09T03:47:22.027066+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:14:00.724882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248147+0000","last_change":"2023-11-09T03:47:19.947282+0000","last_active":"2023-11-09T03:47:22.248147+0000","last_peered":"2023-11-09T03:47:22.248147+0000","last_clean":"2023-11-09T03:47:22.248147+0000","last_became_active":"2023-11-09T03:47:19.947034+0000","last_became_peered":"2023-11-09T03:47:19.947034+0000","last_unstale":"2023-11-09T03:47:22.248147+0000","last_undegraded":"2023-11-09T03:47:22.248147+0000","last_fullsized":"2023-11-09T03:47:22.248147+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:13:59.720199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.935070+0000","last_change":"2023-11-09T03:47:02.618828+0000","last_active":"2023-11-09T03:47:20.935070+0000","last_peered":"2023-11-09T03:47:20.935070+0000","last_clean":"2023-11-09T03:47:20.935070+0000","last_became_active":"2023-11-09T03:47:01.590679+0000","last_became_peered":"2023-11-09T03:47:01.590679+0000","last_unstale":"2023-11-09T03:47:20.935070+0000","last_undegraded":"2023-11-09T03:47:20.935070+0000","last_fullsized":"2023-11-09T03:47:20.935070+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:06:30.982987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070668,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248700+0000","last_change":"2023-11-09T03:47:20.954224+0000","last_active":"2023-11-09T03:47:22.248700+0000","last_peered":"2023-11-09T03:47:22.248700+0000","last_clean":"2023-11-09T03:47:22.248700+0000","last_became_active":"2023-11-09T03:47:20.953950+0000","last_became_peered":"2023-11-09T03:47:20.953950+0000","last_unstale":"2023-11-09T03:47:22.248700+0000","last_undegraded":"2023-11-09T03:47:22.248700+0000","last_fullsized":"2023-11-09T03:47:22.248700+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:58:09.769432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.027101+0000","last_change":"2023-11-09T03:47:19.947929+0000","last_active":"2023-11-09T03:47:22.027101+0000","last_peered":"2023-11-09T03:47:22.027101+0000","last_clean":"2023-11-09T03:47:22.027101+0000","last_became_active":"2023-11-09T03:47:19.947663+0000","last_became_peered":"2023-11-09T03:47:19.947663+0000","last_unstale":"2023-11-09T03:47:22.027101+0000","last_undegraded":"2023-11-09T03:47:22.027101+0000","last_fullsized":"2023-11-09T03:47:22.027101+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:02:12.613580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931259+0000","last_change":"2023-11-09T03:47:03.146528+0000","last_active":"2023-11-09T03:47:20.931259+0000","last_peered":"2023-11-09T03:47:20.931259+0000","last_clean":"2023-11-09T03:47:20.931259+0000","last_became_active":"2023-11-09T03:47:01.587060+0000","last_became_peered":"2023-11-09T03:47:01.587060+0000","last_unstale":"2023-11-09T03:47:20.931259+0000","last_undegraded":"2023-11-09T03:47:20.931259+0000","last_fullsized":"2023-11-09T03:47:20.931259+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:39:15.238496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079064999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.027243+0000","last_change":"2023-11-09T03:47:20.954274+0000","last_active":"2023-11-09T03:47:22.027243+0000","last_peered":"2023-11-09T03:47:22.027243+0000","last_clean":"2023-11-09T03:47:22.027243+0000","last_became_active":"2023-11-09T03:47:20.953976+0000","last_became_peered":"2023-11-09T03:47:20.953976+0000","last_unstale":"2023-11-09T03:47:22.027243+0000","last_undegraded":"2023-11-09T03:47:22.027243+0000","last_fullsized":"2023-11-09T03:47:22.027243+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:05:32.662676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"53'6","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248315+0000","last_change":"2023-11-09T03:47:19.948464+0000","last_active":"2023-11-09T03:47:22.248315+0000","last_peered":"2023-11-09T03:47:22.248315+0000","last_clean":"2023-11-09T03:47:22.248315+0000","last_became_active":"2023-11-09T03:47:19.948254+0000","last_became_peered":"2023-11-09T03:47:19.948254+0000","last_unstale":"2023-11-09T03:47:22.248315+0000","last_undegraded":"2023-11-09T03:47:22.248315+0000","last_fullsized":"2023-11-09T03:47:22.248315+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:21:03.454473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931191+0000","last_change":"2023-11-09T03:47:03.148246+0000","last_active":"2023-11-09T03:47:20.931191+0000","last_peered":"2023-11-09T03:47:20.931191+0000","last_clean":"2023-11-09T03:47:20.931191+0000","last_became_active":"2023-11-09T03:47:01.590988+0000","last_became_peered":"2023-11-09T03:47:01.590988+0000","last_unstale":"2023-11-09T03:47:20.931191+0000","last_undegraded":"2023-11-09T03:47:20.931191+0000","last_fullsized":"2023-11-09T03:47:20.931191+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:07:20.785445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055651099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939172+0000","last_change":"2023-11-09T03:47:20.945226+0000","last_active":"2023-11-09T03:47:21.939172+0000","last_peered":"2023-11-09T03:47:21.939172+0000","last_clean":"2023-11-09T03:47:21.939172+0000","last_became_active":"2023-11-09T03:47:20.944978+0000","last_became_peered":"2023-11-09T03:47:20.944978+0000","last_unstale":"2023-11-09T03:47:21.939172+0000","last_undegraded":"2023-11-09T03:47:21.939172+0000","last_fullsized":"2023-11-09T03:47:21.939172+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:33.651326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"53'3","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026995+0000","last_change":"2023-11-09T03:47:19.945101+0000","last_active":"2023-11-09T03:47:22.026995+0000","last_peered":"2023-11-09T03:47:22.026995+0000","last_clean":"2023-11-09T03:47:22.026995+0000","last_became_active":"2023-11-09T03:47:19.944829+0000","last_became_peered":"2023-11-09T03:47:19.944829+0000","last_unstale":"2023-11-09T03:47:22.026995+0000","last_undegraded":"2023-11-09T03:47:22.026995+0000","last_fullsized":"2023-11-09T03:47:22.026995+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:49:12.247865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.939514+0000","last_change":"2023-11-09T03:47:02.582754+0000","last_active":"2023-11-09T03:47:20.939514+0000","last_peered":"2023-11-09T03:47:20.939514+0000","last_clean":"2023-11-09T03:47:20.939514+0000","last_became_active":"2023-11-09T03:47:01.589715+0000","last_became_peered":"2023-11-09T03:47:01.589715+0000","last_unstale":"2023-11-09T03:47:20.939514+0000","last_undegraded":"2023-11-09T03:47:20.939514+0000","last_fullsized":"2023-11-09T03:47:20.939514+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099342799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248560+0000","last_change":"2023-11-09T03:47:20.953974+0000","last_active":"2023-11-09T03:47:22.248560+0000","last_peered":"2023-11-09T03:47:22.248560+0000","last_clean":"2023-11-09T03:47:22.248560+0000","last_became_active":"2023-11-09T03:47:20.953714+0000","last_became_peered":"2023-11-09T03:47:20.953714+0000","last_unstale":"2023-11-09T03:47:22.248560+0000","last_undegraded":"2023-11-09T03:47:22.248560+0000","last_fullsized":"2023-11-09T03:47:22.248560+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:40:44.918031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"53'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248845+0000","last_change":"2023-11-09T03:47:19.943054+0000","last_active":"2023-11-09T03:47:22.248845+0000","last_peered":"2023-11-09T03:47:22.248845+0000","last_clean":"2023-11-09T03:47:22.248845+0000","last_became_active":"2023-11-09T03:47:19.942787+0000","last_became_peered":"2023-11-09T03:47:19.942787+0000","last_unstale":"2023-11-09T03:47:22.248845+0000","last_undegraded":"2023-11-09T03:47:22.248845+0000","last_fullsized":"2023-11-09T03:47:22.248845+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:58:08.764749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.939087+0000","last_change":"2023-11-09T03:47:02.724896+0000","last_active":"2023-11-09T03:47:20.939087+0000","last_peered":"2023-11-09T03:47:20.939087+0000","last_clean":"2023-11-09T03:47:20.939087+0000","last_became_active":"2023-11-09T03:47:01.592784+0000","last_became_peered":"2023-11-09T03:47:01.592784+0000","last_unstale":"2023-11-09T03:47:20.939087+0000","last_undegraded":"2023-11-09T03:47:20.939087+0000","last_fullsized":"2023-11-09T03:47:20.939087+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:51:32.016204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056683500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248073+0000","last_change":"2023-11-09T03:47:20.945066+0000","last_active":"2023-11-09T03:47:22.248073+0000","last_peered":"2023-11-09T03:47:22.248073+0000","last_clean":"2023-11-09T03:47:22.248073+0000","last_became_active":"2023-11-09T03:47:20.944812+0000","last_became_peered":"2023-11-09T03:47:20.944812+0000","last_unstale":"2023-11-09T03:47:22.248073+0000","last_undegraded":"2023-11-09T03:47:22.248073+0000","last_fullsized":"2023-11-09T03:47:22.248073+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:58:30.558728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"53'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248249+0000","last_change":"2023-11-09T03:47:19.947954+0000","last_active":"2023-11-09T03:47:22.248249+0000","last_peered":"2023-11-09T03:47:22.248249+0000","last_clean":"2023-11-09T03:47:22.248249+0000","last_became_active":"2023-11-09T03:47:19.947687+0000","last_became_peered":"2023-11-09T03:47:19.947687+0000","last_unstale":"2023-11-09T03:47:22.248249+0000","last_undegraded":"2023-11-09T03:47:22.248249+0000","last_fullsized":"2023-11-09T03:47:22.248249+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:52:58.422763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.933388+0000","last_change":"2023-11-09T03:47:03.146527+0000","last_active":"2023-11-09T03:47:20.933388+0000","last_peered":"2023-11-09T03:47:20.933388+0000","last_clean":"2023-11-09T03:47:20.933388+0000","last_became_active":"2023-11-09T03:47:01.587858+0000","last_became_peered":"2023-11-09T03:47:01.587858+0000","last_unstale":"2023-11-09T03:47:20.933388+0000","last_undegraded":"2023-11-09T03:47:20.933388+0000","last_fullsized":"2023-11-09T03:47:20.933388+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:56:04.201126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084439599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248163+0000","last_change":"2023-11-09T03:47:20.943815+0000","last_active":"2023-11-09T03:47:22.248163+0000","last_peered":"2023-11-09T03:47:22.248163+0000","last_clean":"2023-11-09T03:47:22.248163+0000","last_became_active":"2023-11-09T03:47:20.943403+0000","last_became_peered":"2023-11-09T03:47:20.943403+0000","last_unstale":"2023-11-09T03:47:22.248163+0000","last_undegraded":"2023-11-09T03:47:22.248163+0000","last_fullsized":"2023-11-09T03:47:22.248163+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:59:34.738692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247950+0000","last_change":"2023-11-09T03:47:19.949858+0000","last_active":"2023-11-09T03:47:22.247950+0000","last_peered":"2023-11-09T03:47:22.247950+0000","last_clean":"2023-11-09T03:47:22.247950+0000","last_became_active":"2023-11-09T03:47:19.949636+0000","last_became_peered":"2023-11-09T03:47:19.949636+0000","last_unstale":"2023-11-09T03:47:22.247950+0000","last_undegraded":"2023-11-09T03:47:22.247950+0000","last_fullsized":"2023-11-09T03:47:22.247950+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:24.100578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.935999+0000","last_change":"2023-11-09T03:47:02.725223+0000","last_active":"2023-11-09T03:47:20.935999+0000","last_peered":"2023-11-09T03:47:20.935999+0000","last_clean":"2023-11-09T03:47:20.935999+0000","last_became_active":"2023-11-09T03:47:01.590635+0000","last_became_peered":"2023-11-09T03:47:01.590635+0000","last_unstale":"2023-11-09T03:47:20.935999+0000","last_undegraded":"2023-11-09T03:47:20.935999+0000","last_fullsized":"2023-11-09T03:47:20.935999+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:05.754357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051685100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248165+0000","last_change":"2023-11-09T03:47:20.948694+0000","last_active":"2023-11-09T03:47:22.248165+0000","last_peered":"2023-11-09T03:47:22.248165+0000","last_clean":"2023-11-09T03:47:22.248165+0000","last_became_active":"2023-11-09T03:47:20.948405+0000","last_became_peered":"2023-11-09T03:47:20.948405+0000","last_unstale":"2023-11-09T03:47:22.248165+0000","last_undegraded":"2023-11-09T03:47:22.248165+0000","last_fullsized":"2023-11-09T03:47:22.248165+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:30:50.805289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940244+0000","last_change":"2023-11-09T03:47:19.942483+0000","last_active":"2023-11-09T03:47:21.940244+0000","last_peered":"2023-11-09T03:47:21.940244+0000","last_clean":"2023-11-09T03:47:21.940244+0000","last_became_active":"2023-11-09T03:47:19.942157+0000","last_became_peered":"2023-11-09T03:47:19.942157+0000","last_unstale":"2023-11-09T03:47:21.940244+0000","last_undegraded":"2023-11-09T03:47:21.940244+0000","last_fullsized":"2023-11-09T03:47:21.940244+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:59:28.149940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.937687+0000","last_change":"2023-11-09T03:47:02.582739+0000","last_active":"2023-11-09T03:47:20.937687+0000","last_peered":"2023-11-09T03:47:20.937687+0000","last_clean":"2023-11-09T03:47:20.937687+0000","last_became_active":"2023-11-09T03:47:01.592743+0000","last_became_peered":"2023-11-09T03:47:01.592743+0000","last_unstale":"2023-11-09T03:47:20.937687+0000","last_undegraded":"2023-11-09T03:47:20.937687+0000","last_fullsized":"2023-11-09T03:47:20.937687+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051802700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939444+0000","last_change":"2023-11-09T03:47:20.939366+0000","last_active":"2023-11-09T03:47:21.939444+0000","last_peered":"2023-11-09T03:47:21.939444+0000","last_clean":"2023-11-09T03:47:21.939444+0000","last_became_active":"2023-11-09T03:47:20.938258+0000","last_became_peered":"2023-11-09T03:47:20.938258+0000","last_unstale":"2023-11-09T03:47:21.939444+0000","last_undegraded":"2023-11-09T03:47:21.939444+0000","last_fullsized":"2023-11-09T03:47:21.939444+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:48:27.582736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"53'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248057+0000","last_change":"2023-11-09T03:47:19.946306+0000","last_active":"2023-11-09T03:47:22.248057+0000","last_peered":"2023-11-09T03:47:22.248057+0000","last_clean":"2023-11-09T03:47:22.248057+0000","last_became_active":"2023-11-09T03:47:19.946062+0000","last_became_peered":"2023-11-09T03:47:19.946062+0000","last_unstale":"2023-11-09T03:47:22.248057+0000","last_undegraded":"2023-11-09T03:47:22.248057+0000","last_fullsized":"2023-11-09T03:47:22.248057+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:51:52.875174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.935441+0000","last_change":"2023-11-09T03:47:02.724953+0000","last_active":"2023-11-09T03:47:20.935441+0000","last_peered":"2023-11-09T03:47:20.935441+0000","last_clean":"2023-11-09T03:47:20.935441+0000","last_became_active":"2023-11-09T03:47:01.587133+0000","last_became_peered":"2023-11-09T03:47:01.587133+0000","last_unstale":"2023-11-09T03:47:20.935441+0000","last_undegraded":"2023-11-09T03:47:20.935441+0000","last_fullsized":"2023-11-09T03:47:20.935441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:23:48.521161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066497599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249034+0000","last_change":"2023-11-09T03:47:20.942122+0000","last_active":"2023-11-09T03:47:22.249034+0000","last_peered":"2023-11-09T03:47:22.249034+0000","last_clean":"2023-11-09T03:47:22.249034+0000","last_became_active":"2023-11-09T03:47:20.941870+0000","last_became_peered":"2023-11-09T03:47:20.941870+0000","last_unstale":"2023-11-09T03:47:22.249034+0000","last_undegraded":"2023-11-09T03:47:22.249034+0000","last_fullsized":"2023-11-09T03:47:22.249034+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:57.187676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"53'4","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249334+0000","last_change":"2023-11-09T03:47:19.945714+0000","last_active":"2023-11-09T03:47:22.249334+0000","last_peered":"2023-11-09T03:47:22.249334+0000","last_clean":"2023-11-09T03:47:22.249334+0000","last_became_active":"2023-11-09T03:47:19.945455+0000","last_became_peered":"2023-11-09T03:47:19.945455+0000","last_unstale":"2023-11-09T03:47:22.249334+0000","last_undegraded":"2023-11-09T03:47:22.249334+0000","last_fullsized":"2023-11-09T03:47:22.249334+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:38:08.672664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.935591+0000","last_change":"2023-11-09T03:47:03.146217+0000","last_active":"2023-11-09T03:47:20.935591+0000","last_peered":"2023-11-09T03:47:20.935591+0000","last_clean":"2023-11-09T03:47:20.935591+0000","last_became_active":"2023-11-09T03:47:01.587598+0000","last_became_peered":"2023-11-09T03:47:01.587598+0000","last_unstale":"2023-11-09T03:47:20.935591+0000","last_undegraded":"2023-11-09T03:47:20.935591+0000","last_fullsized":"2023-11-09T03:47:20.935591+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:15:22.723455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090126400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249094+0000","last_change":"2023-11-09T03:47:20.950285+0000","last_active":"2023-11-09T03:47:22.249094+0000","last_peered":"2023-11-09T03:47:22.249094+0000","last_clean":"2023-11-09T03:47:22.249094+0000","last_became_active":"2023-11-09T03:47:20.949722+0000","last_became_peered":"2023-11-09T03:47:20.949722+0000","last_unstale":"2023-11-09T03:47:22.249094+0000","last_undegraded":"2023-11-09T03:47:22.249094+0000","last_fullsized":"2023-11-09T03:47:22.249094+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:51:53.879857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940663+0000","last_change":"2023-11-09T03:47:19.947977+0000","last_active":"2023-11-09T03:47:21.940663+0000","last_peered":"2023-11-09T03:47:21.940663+0000","last_clean":"2023-11-09T03:47:21.940663+0000","last_became_active":"2023-11-09T03:47:19.947549+0000","last_became_peered":"2023-11-09T03:47:19.947549+0000","last_unstale":"2023-11-09T03:47:21.940663+0000","last_undegraded":"2023-11-09T03:47:21.940663+0000","last_fullsized":"2023-11-09T03:47:21.940663+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:51.678872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931267+0000","last_change":"2023-11-09T03:47:03.148283+0000","last_active":"2023-11-09T03:47:20.931267+0000","last_peered":"2023-11-09T03:47:20.931267+0000","last_clean":"2023-11-09T03:47:20.931267+0000","last_became_active":"2023-11-09T03:47:01.588841+0000","last_became_peered":"2023-11-09T03:47:01.588841+0000","last_unstale":"2023-11-09T03:47:20.931267+0000","last_undegraded":"2023-11-09T03:47:20.931267+0000","last_fullsized":"2023-11-09T03:47:20.931267+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:47:44.880741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000436081,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249189+0000","last_change":"2023-11-09T03:47:20.937043+0000","last_active":"2023-11-09T03:47:22.249189+0000","last_peered":"2023-11-09T03:47:22.249189+0000","last_clean":"2023-11-09T03:47:22.249189+0000","last_became_active":"2023-11-09T03:47:20.936765+0000","last_became_peered":"2023-11-09T03:47:20.936765+0000","last_unstale":"2023-11-09T03:47:22.249189+0000","last_undegraded":"2023-11-09T03:47:22.249189+0000","last_fullsized":"2023-11-09T03:47:22.249189+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:14:00.724882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249189+0000","last_change":"2023-11-09T03:47:19.946355+0000","last_active":"2023-11-09T03:47:22.249189+0000","last_peered":"2023-11-09T03:47:22.249189+0000","last_clean":"2023-11-09T03:47:22.249189+0000","last_became_active":"2023-11-09T03:47:19.945071+0000","last_became_peered":"2023-11-09T03:47:19.945071+0000","last_unstale":"2023-11-09T03:47:22.249189+0000","last_undegraded":"2023-11-09T03:47:22.249189+0000","last_fullsized":"2023-11-09T03:47:22.249189+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:07:31.918577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.937170+0000","last_change":"2023-11-09T03:47:02.618856+0000","last_active":"2023-11-09T03:47:20.937170+0000","last_peered":"2023-11-09T03:47:20.937170+0000","last_clean":"2023-11-09T03:47:20.937170+0000","last_became_active":"2023-11-09T03:47:01.590430+0000","last_became_peered":"2023-11-09T03:47:01.590430+0000","last_unstale":"2023-11-09T03:47:20.937170+0000","last_undegraded":"2023-11-09T03:47:20.937170+0000","last_fullsized":"2023-11-09T03:47:20.937170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:57:18.470305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055030200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026961+0000","last_change":"2023-11-09T03:47:20.948028+0000","last_active":"2023-11-09T03:47:22.026961+0000","last_peered":"2023-11-09T03:47:22.026961+0000","last_clean":"2023-11-09T03:47:22.026961+0000","last_became_active":"2023-11-09T03:47:20.947752+0000","last_became_peered":"2023-11-09T03:47:20.947752+0000","last_unstale":"2023-11-09T03:47:22.026961+0000","last_undegraded":"2023-11-09T03:47:22.026961+0000","last_fullsized":"2023-11-09T03:47:22.026961+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:51:53.879857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248095+0000","last_change":"2023-11-09T03:47:19.948154+0000","last_active":"2023-11-09T03:47:22.248095+0000","last_peered":"2023-11-09T03:47:22.248095+0000","last_clean":"2023-11-09T03:47:22.248095+0000","last_became_active":"2023-11-09T03:47:19.947812+0000","last_became_peered":"2023-11-09T03:47:19.947812+0000","last_unstale":"2023-11-09T03:47:22.248095+0000","last_undegraded":"2023-11-09T03:47:22.248095+0000","last_fullsized":"2023-11-09T03:47:22.248095+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:28:13.618166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.935365+0000","last_change":"2023-11-09T03:47:02.619144+0000","last_active":"2023-11-09T03:47:20.935365+0000","last_peered":"2023-11-09T03:47:20.935365+0000","last_clean":"2023-11-09T03:47:20.935365+0000","last_became_active":"2023-11-09T03:47:01.590672+0000","last_became_peered":"2023-11-09T03:47:01.590672+0000","last_unstale":"2023-11-09T03:47:20.935365+0000","last_undegraded":"2023-11-09T03:47:20.935365+0000","last_fullsized":"2023-11-09T03:47:20.935365+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:03.025119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060997500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248188+0000","last_change":"2023-11-09T03:47:20.950877+0000","last_active":"2023-11-09T03:47:22.248188+0000","last_peered":"2023-11-09T03:47:22.248188+0000","last_clean":"2023-11-09T03:47:22.248188+0000","last_became_active":"2023-11-09T03:47:20.950616+0000","last_became_peered":"2023-11-09T03:47:20.950616+0000","last_unstale":"2023-11-09T03:47:22.248188+0000","last_undegraded":"2023-11-09T03:47:22.248188+0000","last_fullsized":"2023-11-09T03:47:22.248188+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:30.970511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939086+0000","last_change":"2023-11-09T03:47:19.940824+0000","last_active":"2023-11-09T03:47:21.939086+0000","last_peered":"2023-11-09T03:47:21.939086+0000","last_clean":"2023-11-09T03:47:21.939086+0000","last_became_active":"2023-11-09T03:47:19.940553+0000","last_became_peered":"2023-11-09T03:47:19.940553+0000","last_unstale":"2023-11-09T03:47:21.939086+0000","last_undegraded":"2023-11-09T03:47:21.939086+0000","last_fullsized":"2023-11-09T03:47:21.939086+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:52:58.422763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.934932+0000","last_change":"2023-11-09T03:47:02.619079+0000","last_active":"2023-11-09T03:47:20.934932+0000","last_peered":"2023-11-09T03:47:20.934932+0000","last_clean":"2023-11-09T03:47:20.934932+0000","last_became_active":"2023-11-09T03:47:01.591421+0000","last_became_peered":"2023-11-09T03:47:01.591421+0000","last_unstale":"2023-11-09T03:47:20.934932+0000","last_undegraded":"2023-11-09T03:47:20.934932+0000","last_fullsized":"2023-11-09T03:47:20.934932+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:18:58.230606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068736399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248346+0000","last_change":"2023-11-09T03:47:20.943642+0000","last_active":"2023-11-09T03:47:22.248346+0000","last_peered":"2023-11-09T03:47:22.248346+0000","last_clean":"2023-11-09T03:47:22.248346+0000","last_became_active":"2023-11-09T03:47:20.943101+0000","last_became_peered":"2023-11-09T03:47:20.943101+0000","last_unstale":"2023-11-09T03:47:22.248346+0000","last_undegraded":"2023-11-09T03:47:22.248346+0000","last_fullsized":"2023-11-09T03:47:22.248346+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:06:11.677208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248583+0000","last_change":"2023-11-09T03:47:19.943530+0000","last_active":"2023-11-09T03:47:22.248583+0000","last_peered":"2023-11-09T03:47:22.248583+0000","last_clean":"2023-11-09T03:47:22.248583+0000","last_became_active":"2023-11-09T03:47:19.943303+0000","last_became_peered":"2023-11-09T03:47:19.943303+0000","last_unstale":"2023-11-09T03:47:22.248583+0000","last_undegraded":"2023-11-09T03:47:22.248583+0000","last_fullsized":"2023-11-09T03:47:22.248583+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:35.294223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931280+0000","last_change":"2023-11-09T03:47:03.148529+0000","last_active":"2023-11-09T03:47:20.931280+0000","last_peered":"2023-11-09T03:47:20.931280+0000","last_clean":"2023-11-09T03:47:20.931280+0000","last_became_active":"2023-11-09T03:47:01.584148+0000","last_became_peered":"2023-11-09T03:47:01.584148+0000","last_unstale":"2023-11-09T03:47:20.931280+0000","last_undegraded":"2023-11-09T03:47:20.931280+0000","last_fullsized":"2023-11-09T03:47:20.931280+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:40:00.237248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061475600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.938959+0000","last_change":"2023-11-09T03:47:20.942889+0000","last_active":"2023-11-09T03:47:21.938959+0000","last_peered":"2023-11-09T03:47:21.938959+0000","last_clean":"2023-11-09T03:47:21.938959+0000","last_became_active":"2023-11-09T03:47:20.941326+0000","last_became_peered":"2023-11-09T03:47:20.941326+0000","last_unstale":"2023-11-09T03:47:21.938959+0000","last_undegraded":"2023-11-09T03:47:21.938959+0000","last_fullsized":"2023-11-09T03:47:21.938959+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:56:23.552030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.938908+0000","last_change":"2023-11-09T03:47:19.942969+0000","last_active":"2023-11-09T03:47:21.938908+0000","last_peered":"2023-11-09T03:47:21.938908+0000","last_clean":"2023-11-09T03:47:21.938908+0000","last_became_active":"2023-11-09T03:47:19.942726+0000","last_became_peered":"2023-11-09T03:47:19.942726+0000","last_unstale":"2023-11-09T03:47:21.938908+0000","last_undegraded":"2023-11-09T03:47:21.938908+0000","last_fullsized":"2023-11-09T03:47:21.938908+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:51:50.362425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.933718+0000","last_change":"2023-11-09T03:47:03.148217+0000","last_active":"2023-11-09T03:47:20.933718+0000","last_peered":"2023-11-09T03:47:20.933718+0000","last_clean":"2023-11-09T03:47:20.933718+0000","last_became_active":"2023-11-09T03:47:01.593424+0000","last_became_peered":"2023-11-09T03:47:01.593424+0000","last_unstale":"2023-11-09T03:47:20.933718+0000","last_undegraded":"2023-11-09T03:47:20.933718+0000","last_fullsized":"2023-11-09T03:47:20.933718+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:19:32.693691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001769865,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.941149+0000","last_change":"2023-11-09T03:47:20.942781+0000","last_active":"2023-11-09T03:47:21.941149+0000","last_peered":"2023-11-09T03:47:21.941149+0000","last_clean":"2023-11-09T03:47:21.941149+0000","last_became_active":"2023-11-09T03:47:20.942498+0000","last_became_peered":"2023-11-09T03:47:20.942498+0000","last_unstale":"2023-11-09T03:47:21.941149+0000","last_undegraded":"2023-11-09T03:47:21.941149+0000","last_fullsized":"2023-11-09T03:47:21.941149+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:23:50.235725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.774163+0000","last_change":"2023-11-09T03:47:19.944843+0000","last_active":"2023-11-09T03:47:22.774163+0000","last_peered":"2023-11-09T03:47:22.774163+0000","last_clean":"2023-11-09T03:47:22.774163+0000","last_became_active":"2023-11-09T03:47:19.944583+0000","last_became_peered":"2023-11-09T03:47:19.944583+0000","last_unstale":"2023-11-09T03:47:22.774163+0000","last_undegraded":"2023-11-09T03:47:22.774163+0000","last_fullsized":"2023-11-09T03:47:22.774163+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:46:53.255789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931015+0000","last_change":"2023-11-09T03:47:03.146470+0000","last_active":"2023-11-09T03:47:20.931015+0000","last_peered":"2023-11-09T03:47:20.931015+0000","last_clean":"2023-11-09T03:47:20.931015+0000","last_became_active":"2023-11-09T03:47:01.587304+0000","last_became_peered":"2023-11-09T03:47:01.587304+0000","last_unstale":"2023-11-09T03:47:20.931015+0000","last_undegraded":"2023-11-09T03:47:20.931015+0000","last_fullsized":"2023-11-09T03:47:20.931015+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:58:23.339287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00110395,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.941057+0000","last_change":"2023-11-09T03:47:20.941612+0000","last_active":"2023-11-09T03:47:21.941057+0000","last_peered":"2023-11-09T03:47:21.941057+0000","last_clean":"2023-11-09T03:47:21.941057+0000","last_became_active":"2023-11-09T03:47:20.941335+0000","last_became_peered":"2023-11-09T03:47:20.941335+0000","last_unstale":"2023-11-09T03:47:21.941057+0000","last_undegraded":"2023-11-09T03:47:21.941057+0000","last_fullsized":"2023-11-09T03:47:21.941057+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:05:32.662676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939435+0000","last_change":"2023-11-09T03:47:19.941173+0000","last_active":"2023-11-09T03:47:21.939435+0000","last_peered":"2023-11-09T03:47:21.939435+0000","last_clean":"2023-11-09T03:47:21.939435+0000","last_became_active":"2023-11-09T03:47:19.940928+0000","last_became_peered":"2023-11-09T03:47:19.940928+0000","last_unstale":"2023-11-09T03:47:21.939435+0000","last_undegraded":"2023-11-09T03:47:21.939435+0000","last_fullsized":"2023-11-09T03:47:21.939435+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:28:13.618166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.934203+0000","last_change":"2023-11-09T03:47:03.147367+0000","last_active":"2023-11-09T03:47:20.934203+0000","last_peered":"2023-11-09T03:47:20.934203+0000","last_clean":"2023-11-09T03:47:20.934203+0000","last_became_active":"2023-11-09T03:47:01.588359+0000","last_became_peered":"2023-11-09T03:47:01.588359+0000","last_unstale":"2023-11-09T03:47:20.934203+0000","last_undegraded":"2023-11-09T03:47:20.934203+0000","last_fullsized":"2023-11-09T03:47:20.934203+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:21:34.303068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014106660000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249223+0000","last_change":"2023-11-09T03:47:20.950444+0000","last_active":"2023-11-09T03:47:22.249223+0000","last_peered":"2023-11-09T03:47:22.249223+0000","last_clean":"2023-11-09T03:47:22.249223+0000","last_became_active":"2023-11-09T03:47:20.950038+0000","last_became_peered":"2023-11-09T03:47:20.950038+0000","last_unstale":"2023-11-09T03:47:22.249223+0000","last_undegraded":"2023-11-09T03:47:22.249223+0000","last_fullsized":"2023-11-09T03:47:22.249223+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:55:28.077165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940688+0000","last_change":"2023-11-09T03:47:19.943935+0000","last_active":"2023-11-09T03:47:21.940688+0000","last_peered":"2023-11-09T03:47:21.940688+0000","last_clean":"2023-11-09T03:47:21.940688+0000","last_became_active":"2023-11-09T03:47:19.943125+0000","last_became_peered":"2023-11-09T03:47:19.943125+0000","last_unstale":"2023-11-09T03:47:21.940688+0000","last_undegraded":"2023-11-09T03:47:21.940688+0000","last_fullsized":"2023-11-09T03:47:21.940688+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:06:16.671366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":27,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.928179+0000","last_change":"2023-11-09T03:47:02.581682+0000","last_active":"2023-11-09T03:47:20.928179+0000","last_peered":"2023-11-09T03:47:20.928179+0000","last_clean":"2023-11-09T03:47:20.928179+0000","last_became_active":"2023-11-09T03:47:01.593051+0000","last_became_peered":"2023-11-09T03:47:01.593051+0000","last_unstale":"2023-11-09T03:47:20.928179+0000","last_undegraded":"2023-11-09T03:47:20.928179+0000","last_fullsized":"2023-11-09T03:47:20.928179+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047277799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939213+0000","last_change":"2023-11-09T03:47:20.945402+0000","last_active":"2023-11-09T03:47:21.939213+0000","last_peered":"2023-11-09T03:47:21.939213+0000","last_clean":"2023-11-09T03:47:21.939213+0000","last_became_active":"2023-11-09T03:47:20.945145+0000","last_became_peered":"2023-11-09T03:47:20.945145+0000","last_unstale":"2023-11-09T03:47:21.939213+0000","last_undegraded":"2023-11-09T03:47:21.939213+0000","last_fullsized":"2023-11-09T03:47:21.939213+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:38:09.677347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248640+0000","last_change":"2023-11-09T03:47:19.942655+0000","last_active":"2023-11-09T03:47:22.248640+0000","last_peered":"2023-11-09T03:47:22.248640+0000","last_clean":"2023-11-09T03:47:22.248640+0000","last_became_active":"2023-11-09T03:47:19.942380+0000","last_became_peered":"2023-11-09T03:47:19.942380+0000","last_unstale":"2023-11-09T03:47:22.248640+0000","last_undegraded":"2023-11-09T03:47:22.248640+0000","last_fullsized":"2023-11-09T03:47:22.248640+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:52:58.422763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.933457+0000","last_change":"2023-11-09T03:47:03.146309+0000","last_active":"2023-11-09T03:47:20.933457+0000","last_peered":"2023-11-09T03:47:20.933457+0000","last_clean":"2023-11-09T03:47:20.933457+0000","last_became_active":"2023-11-09T03:47:01.587560+0000","last_became_peered":"2023-11-09T03:47:01.587560+0000","last_unstale":"2023-11-09T03:47:20.933457+0000","last_undegraded":"2023-11-09T03:47:20.933457+0000","last_fullsized":"2023-11-09T03:47:20.933457+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:27:55.271945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070988400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249299+0000","last_change":"2023-11-09T03:47:20.941432+0000","last_active":"2023-11-09T03:47:22.249299+0000","last_peered":"2023-11-09T03:47:22.249299+0000","last_clean":"2023-11-09T03:47:22.249299+0000","last_became_active":"2023-11-09T03:47:20.941186+0000","last_became_peered":"2023-11-09T03:47:20.941186+0000","last_unstale":"2023-11-09T03:47:22.249299+0000","last_undegraded":"2023-11-09T03:47:22.249299+0000","last_fullsized":"2023-11-09T03:47:22.249299+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:25:38.423194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.939102+0000","last_change":"2023-11-09T03:47:19.947334+0000","last_active":"2023-11-09T03:47:21.939102+0000","last_peered":"2023-11-09T03:47:21.939102+0000","last_clean":"2023-11-09T03:47:21.939102+0000","last_became_active":"2023-11-09T03:47:19.947091+0000","last_became_peered":"2023-11-09T03:47:19.947091+0000","last_unstale":"2023-11-09T03:47:21.939102+0000","last_undegraded":"2023-11-09T03:47:21.939102+0000","last_fullsized":"2023-11-09T03:47:21.939102+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:24:06.867382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.939549+0000","last_change":"2023-11-09T03:47:02.725125+0000","last_active":"2023-11-09T03:47:20.939549+0000","last_peered":"2023-11-09T03:47:20.939549+0000","last_clean":"2023-11-09T03:47:20.939549+0000","last_became_active":"2023-11-09T03:47:01.591521+0000","last_became_peered":"2023-11-09T03:47:01.591521+0000","last_unstale":"2023-11-09T03:47:20.939549+0000","last_undegraded":"2023-11-09T03:47:20.939549+0000","last_fullsized":"2023-11-09T03:47:20.939549+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:52:40.076542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000758675,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940341+0000","last_change":"2023-11-09T03:47:20.938307+0000","last_active":"2023-11-09T03:47:21.940341+0000","last_peered":"2023-11-09T03:47:21.940341+0000","last_clean":"2023-11-09T03:47:21.940341+0000","last_became_active":"2023-11-09T03:47:20.937853+0000","last_became_peered":"2023-11-09T03:47:20.937853+0000","last_unstale":"2023-11-09T03:47:21.940341+0000","last_undegraded":"2023-11-09T03:47:21.940341+0000","last_fullsized":"2023-11-09T03:47:21.940341+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:57.187676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248805+0000","last_change":"2023-11-09T03:47:19.945739+0000","last_active":"2023-11-09T03:47:22.248805+0000","last_peered":"2023-11-09T03:47:22.248805+0000","last_clean":"2023-11-09T03:47:22.248805+0000","last_became_active":"2023-11-09T03:47:19.944990+0000","last_became_peered":"2023-11-09T03:47:19.944990+0000","last_unstale":"2023-11-09T03:47:22.248805+0000","last_undegraded":"2023-11-09T03:47:22.248805+0000","last_fullsized":"2023-11-09T03:47:22.248805+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:53:27.482165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.932096+0000","last_change":"2023-11-09T03:47:03.147289+0000","last_active":"2023-11-09T03:47:20.932096+0000","last_peered":"2023-11-09T03:47:20.932096+0000","last_clean":"2023-11-09T03:47:20.932096+0000","last_became_active":"2023-11-09T03:47:01.587784+0000","last_became_peered":"2023-11-09T03:47:01.587784+0000","last_unstale":"2023-11-09T03:47:20.932096+0000","last_undegraded":"2023-11-09T03:47:20.932096+0000","last_fullsized":"2023-11-09T03:47:20.932096+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:11:23.824112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038781299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.247714+0000","last_change":"2023-11-09T03:47:20.937750+0000","last_active":"2023-11-09T03:47:22.247714+0000","last_peered":"2023-11-09T03:47:22.247714+0000","last_clean":"2023-11-09T03:47:22.247714+0000","last_became_active":"2023-11-09T03:47:20.937473+0000","last_became_peered":"2023-11-09T03:47:20.937473+0000","last_unstale":"2023-11-09T03:47:22.247714+0000","last_undegraded":"2023-11-09T03:47:22.247714+0000","last_fullsized":"2023-11-09T03:47:22.247714+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:06:17.676049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940071+0000","last_change":"2023-11-09T03:47:19.948440+0000","last_active":"2023-11-09T03:47:21.940071+0000","last_peered":"2023-11-09T03:47:21.940071+0000","last_clean":"2023-11-09T03:47:21.940071+0000","last_became_active":"2023-11-09T03:47:19.948199+0000","last_became_peered":"2023-11-09T03:47:19.948199+0000","last_unstale":"2023-11-09T03:47:21.940071+0000","last_undegraded":"2023-11-09T03:47:21.940071+0000","last_fullsized":"2023-11-09T03:47:21.940071+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:34:52.006539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.935047+0000","last_change":"2023-11-09T03:47:02.619346+0000","last_active":"2023-11-09T03:47:20.935047+0000","last_peered":"2023-11-09T03:47:20.935047+0000","last_clean":"2023-11-09T03:47:20.935047+0000","last_became_active":"2023-11-09T03:47:01.590389+0000","last_became_peered":"2023-11-09T03:47:01.590389+0000","last_unstale":"2023-11-09T03:47:20.935047+0000","last_undegraded":"2023-11-09T03:47:20.935047+0000","last_fullsized":"2023-11-09T03:47:20.935047+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000384072,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.941022+0000","last_change":"2023-11-09T03:47:20.937215+0000","last_active":"2023-11-09T03:47:21.941022+0000","last_peered":"2023-11-09T03:47:21.941022+0000","last_clean":"2023-11-09T03:47:21.941022+0000","last_became_active":"2023-11-09T03:47:20.936913+0000","last_became_peered":"2023-11-09T03:47:20.936913+0000","last_unstale":"2023-11-09T03:47:21.941022+0000","last_undegraded":"2023-11-09T03:47:21.941022+0000","last_fullsized":"2023-11-09T03:47:21.941022+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:25:38.423194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249234+0000","last_change":"2023-11-09T03:47:19.948214+0000","last_active":"2023-11-09T03:47:22.249234+0000","last_peered":"2023-11-09T03:47:22.249234+0000","last_clean":"2023-11-09T03:47:22.249234+0000","last_became_active":"2023-11-09T03:47:19.947966+0000","last_became_peered":"2023-11-09T03:47:19.947966+0000","last_unstale":"2023-11-09T03:47:22.249234+0000","last_undegraded":"2023-11-09T03:47:22.249234+0000","last_fullsized":"2023-11-09T03:47:22.249234+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:53:19.322920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.938897+0000","last_change":"2023-11-09T03:47:02.582393+0000","last_active":"2023-11-09T03:47:20.938897+0000","last_peered":"2023-11-09T03:47:20.938897+0000","last_clean":"2023-11-09T03:47:20.938897+0000","last_became_active":"2023-11-09T03:47:01.591752+0000","last_became_peered":"2023-11-09T03:47:01.591752+0000","last_unstale":"2023-11-09T03:47:20.938897+0000","last_undegraded":"2023-11-09T03:47:20.938897+0000","last_fullsized":"2023-11-09T03:47:20.938897+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:03.025119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067021599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.027114+0000","last_change":"2023-11-09T03:47:20.946342+0000","last_active":"2023-11-09T03:47:22.027114+0000","last_peered":"2023-11-09T03:47:22.027114+0000","last_clean":"2023-11-09T03:47:22.027114+0000","last_became_active":"2023-11-09T03:47:20.946014+0000","last_became_peered":"2023-11-09T03:47:20.946014+0000","last_unstale":"2023-11-09T03:47:22.027114+0000","last_undegraded":"2023-11-09T03:47:22.027114+0000","last_fullsized":"2023-11-09T03:47:22.027114+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:25:38.423194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"53'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248629+0000","last_change":"2023-11-09T03:47:19.944449+0000","last_active":"2023-11-09T03:47:22.248629+0000","last_peered":"2023-11-09T03:47:22.248629+0000","last_clean":"2023-11-09T03:47:22.248629+0000","last_became_active":"2023-11-09T03:47:19.944203+0000","last_became_peered":"2023-11-09T03:47:19.944203+0000","last_unstale":"2023-11-09T03:47:22.248629+0000","last_undegraded":"2023-11-09T03:47:22.248629+0000","last_fullsized":"2023-11-09T03:47:22.248629+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:51.678872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":25,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.929927+0000","last_change":"2023-11-09T03:47:03.028744+0000","last_active":"2023-11-09T03:47:20.929927+0000","last_peered":"2023-11-09T03:47:20.929927+0000","last_clean":"2023-11-09T03:47:20.929927+0000","last_became_active":"2023-11-09T03:47:01.593890+0000","last_became_peered":"2023-11-09T03:47:01.593890+0000","last_unstale":"2023-11-09T03:47:20.929927+0000","last_undegraded":"2023-11-09T03:47:20.929927+0000","last_fullsized":"2023-11-09T03:47:20.929927+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:05.754357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053008299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026854+0000","last_change":"2023-11-09T03:47:20.952365+0000","last_active":"2023-11-09T03:47:22.026854+0000","last_peered":"2023-11-09T03:47:22.026854+0000","last_clean":"2023-11-09T03:47:22.026854+0000","last_became_active":"2023-11-09T03:47:20.952080+0000","last_became_peered":"2023-11-09T03:47:20.952080+0000","last_unstale":"2023-11-09T03:47:22.026854+0000","last_undegraded":"2023-11-09T03:47:22.026854+0000","last_fullsized":"2023-11-09T03:47:22.026854+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:40.639821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.249192+0000","last_change":"2023-11-09T03:47:19.946712+0000","last_active":"2023-11-09T03:47:22.249192+0000","last_peered":"2023-11-09T03:47:22.249192+0000","last_clean":"2023-11-09T03:47:22.249192+0000","last_became_active":"2023-11-09T03:47:19.946456+0000","last_became_peered":"2023-11-09T03:47:19.946456+0000","last_unstale":"2023-11-09T03:47:22.249192+0000","last_undegraded":"2023-11-09T03:47:22.249192+0000","last_fullsized":"2023-11-09T03:47:22.249192+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:39:07.929390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.936277+0000","last_change":"2023-11-09T03:47:03.146783+0000","last_active":"2023-11-09T03:47:20.936277+0000","last_peered":"2023-11-09T03:47:20.936277+0000","last_clean":"2023-11-09T03:47:20.936277+0000","last_became_active":"2023-11-09T03:47:01.586923+0000","last_became_peered":"2023-11-09T03:47:01.586923+0000","last_unstale":"2023-11-09T03:47:20.936277+0000","last_undegraded":"2023-11-09T03:47:20.936277+0000","last_fullsized":"2023-11-09T03:47:20.936277+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:37:50.326442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060384700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248492+0000","last_change":"2023-11-09T03:47:20.946978+0000","last_active":"2023-11-09T03:47:22.248492+0000","last_peered":"2023-11-09T03:47:22.248492+0000","last_clean":"2023-11-09T03:47:22.248492+0000","last_became_active":"2023-11-09T03:47:20.946667+0000","last_became_peered":"2023-11-09T03:47:20.946667+0000","last_unstale":"2023-11-09T03:47:22.248492+0000","last_undegraded":"2023-11-09T03:47:22.248492+0000","last_fullsized":"2023-11-09T03:47:22.248492+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:02:01.425453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"53'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:21.940910+0000","last_change":"2023-11-09T03:47:19.945244+0000","last_active":"2023-11-09T03:47:21.940910+0000","last_peered":"2023-11-09T03:47:21.940910+0000","last_clean":"2023-11-09T03:47:21.940910+0000","last_became_active":"2023-11-09T03:47:19.944955+0000","last_became_peered":"2023-11-09T03:47:19.944955+0000","last_unstale":"2023-11-09T03:47:21.940910+0000","last_undegraded":"2023-11-09T03:47:21.940910+0000","last_fullsized":"2023-11-09T03:47:21.940910+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:12:47.685713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.933539+0000","last_change":"2023-11-09T03:47:03.146580+0000","last_active":"2023-11-09T03:47:20.933539+0000","last_peered":"2023-11-09T03:47:20.933539+0000","last_clean":"2023-11-09T03:47:20.933539+0000","last_became_active":"2023-11-09T03:47:01.589201+0000","last_became_peered":"2023-11-09T03:47:01.589201+0000","last_unstale":"2023-11-09T03:47:20.933539+0000","last_undegraded":"2023-11-09T03:47:20.933539+0000","last_fullsized":"2023-11-09T03:47:20.933539+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:14.300421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049706999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248489+0000","last_change":"2023-11-09T03:47:20.949430+0000","last_active":"2023-11-09T03:47:22.248489+0000","last_peered":"2023-11-09T03:47:22.248489+0000","last_clean":"2023-11-09T03:47:22.248489+0000","last_became_active":"2023-11-09T03:47:20.949177+0000","last_became_peered":"2023-11-09T03:47:20.949177+0000","last_unstale":"2023-11-09T03:47:22.248489+0000","last_undegraded":"2023-11-09T03:47:22.248489+0000","last_fullsized":"2023-11-09T03:47:22.248489+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:19.915800+0000","last_clean_scrub_stamp":"2023-11-09T03:47:19.915800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:04:05.315163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.248363+0000","last_change":"2023-11-09T03:47:19.946526+0000","last_active":"2023-11-09T03:47:22.248363+0000","last_peered":"2023-11-09T03:47:22.248363+0000","last_clean":"2023-11-09T03:47:22.248363+0000","last_became_active":"2023-11-09T03:47:19.946263+0000","last_became_peered":"2023-11-09T03:47:19.946263+0000","last_unstale":"2023-11-09T03:47:22.248363+0000","last_undegraded":"2023-11-09T03:47:22.248363+0000","last_fullsized":"2023-11-09T03:47:22.248363+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:18.911117+0000","last_clean_scrub_stamp":"2023-11-09T03:47:18.911117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:24:06.867382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":24,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-09T03:47:20.931584+0000","last_change":"2023-11-09T03:47:03.147851+0000","last_active":"2023-11-09T03:47:20.931584+0000","last_peered":"2023-11-09T03:47:20.931584+0000","last_clean":"2023-11-09T03:47:20.931584+0000","last_became_active":"2023-11-09T03:47:01.589893+0000","last_became_peered":"2023-11-09T03:47:01.589893+0000","last_unstale":"2023-11-09T03:47:20.931584+0000","last_undegraded":"2023-11-09T03:47:20.931584+0000","last_fullsized":"2023-11-09T03:47:20.931584+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:08:17.183462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029151399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":194,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-09T03:47:22.026619+0000","last_change":"2023-11-09T03:45:21.030926+0000","last_active":"2023-11-09T03:47:22.026619+0000","last_peered":"2023-11-09T03:47:22.026619+0000","last_clean":"2023-11-09T03:47:22.026619+0000","last_became_active":"2023-11-09T03:45:21.029801+0000","last_became_peered":"2023-11-09T03:45:21.029801+0000","last_unstale":"2023-11-09T03:47:22.026619+0000","last_undegraded":"2023-11-09T03:47:22.026619+0000","last_fullsized":"2023-11-09T03:47:22.026619+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_clean_scrub_stamp":"2023-11-09T03:45:19.768433+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:12:59.153713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28240,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343600,"statfs":{"total":96636764160,"available":96607846400,"internally_reserved":0,"allocated":1245184,"data_stored":881539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.501,"15min":0.498},"min":{"1min":0.34899999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.41199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.38700000000000001}]},{"osd":1,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.33000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.502},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.38100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.32800000000000001}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.52300000000000002,"15min":0.51400000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.375},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.68700000000000006,"15min":0.70799999999999996},"min":{"1min":0.41299999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.80700000000000005,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.52400000000000002}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.48699999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.54800000000000004}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.441},"min":{"1min":0.34100000000000003,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.49199999999999999,"15min":0.47899999999999998},"min":{"1min":0.38700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.627}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.47199999999999998,"15min":0.45600000000000002},"min":{"1min":0.37,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.51100000000000001,"15min":0.497},"min":{"1min":0.39600000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.60799999999999998}]},{"osd":6,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55100000000000005,"15min":0.54100000000000004},"min":{"1min":0.44400000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.434,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.65700000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":643072,"data_stored":291011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.499,"15min":0.501},"min":{"1min":0.35099999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63900000000000001,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.503},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.50600000000000001,"15min":0.495},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.66000000000000003}]},{"osd":1,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.621},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.56299999999999994}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.375,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.60699999999999998}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.51900000000000002}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.503,"15min":0.503},"min":{"1min":0.316,"5min":0.313,"15min":0.313},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.436}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.436,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55500000000000005,"15min":0.55800000000000005},"min":{"1min":0.34300000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.45600000000000002}]},{"osd":7,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.42899999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.44400000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.63600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":651264,"data_stored":291663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.496,"15min":0.48599999999999999},"min":{"1min":0.38700000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.503,"15min":0.498},"min":{"1min":0.42399999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.71799999999999997,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.55600000000000005}]},{"osd":1,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.439,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.69999999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.629},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.46700000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.76200000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.62}]},{"osd":2,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.33300000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.67900000000000005,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.499,"5min":0.371,"15min":0.371},"max":{"1min":0.72799999999999998,"5min":0.745,"15min":0.745},"last":0.57799999999999996}]},{"osd":3,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.39700000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.76500000000000001,"5min":0.88,"15min":0.88},"last":0.60399999999999998}]},{"osd":4,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.78100000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.432},{"interface":"front","average":{"1min":0.504,"5min":0.48399999999999999,"15min":0.48099999999999998},"min":{"1min":0.33100000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.47599999999999998}]},{"osd":6,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.50900000000000001,"15min":0.51100000000000001},"min":{"1min":0.33400000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.74199999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51300000000000001,"15min":0.51100000000000001},"min":{"1min":0.35499999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66900000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.51700000000000002}]},{"osd":7,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.38900000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.73499999999999999,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61899999999999999,"15min":0.62},"min":{"1min":0.41199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.752,"5min":1.01,"15min":1.01},"last":0.67800000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":655360,"data_stored":292623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67600000000000005,"5min":1.034,"15min":1.034},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.53600000000000003,"15min":0.54000000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68300000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.57099999999999995}]},{"osd":1,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.56899999999999995,"15min":0.57699999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68100000000000005,"5min":1.093,"15min":1.093},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.36899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72399999999999998,"5min":0.871,"15min":0.871},"last":0.70399999999999996}]},{"osd":2,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76800000000000002,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.44},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70099999999999996,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.65500000000000003}]},{"osd":3,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71699999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54900000000000004,"15min":0.55200000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86599999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.55100000000000005}]},{"osd":5,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54400000000000004,"15min":0.54300000000000004},"min":{"1min":0.37,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76000000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.33100000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.68300000000000005}]},{"osd":6,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.33800000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.5},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.72599999999999998}]},{"osd":7,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.62,"15min":0.63},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77700000000000002,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.95599999999999996,"5min":1.113,"15min":1.113},"last":0.58599999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":651264,"data_stored":291067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.5,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.22},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51100000000000001,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.253}]},{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32900000000000001,"15min":0.33100000000000002},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.54200000000000004,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.371,"15min":0.375},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.53100000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.34300000000000003}]},{"osd":2,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.27200000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.59099999999999997,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.56100000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.317}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.70899999999999996,"5min":0.87,"15min":0.87},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60999999999999999,"15min":0.61599999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.75600000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.55500000000000005}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.62,"15min":0.625},"min":{"1min":0.45700000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79900000000000004,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61699999999999999,"15min":0.623},"min":{"1min":0.42599999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.74299999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.66600000000000004}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.621,"15min":0.627},"min":{"1min":0.42399999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81200000000000006,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59999999999999998,"15min":0.60399999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76600000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.56599999999999995}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77200000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63500000000000001,"15min":0.64200000000000002},"min":{"1min":0.433,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81299999999999994,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.53600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":651264,"data_stored":291067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.46000000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.376},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.33400000000000002,"15min":0.33100000000000002},"min":{"1min":0.247,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.29499999999999998}]},{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32500000000000001,"15min":0.32600000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.495,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.314},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.35899999999999999,"15min":0.36299999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.48599999999999999,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.35099999999999998}]},{"osd":3,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.36199999999999999,"15min":0.36399999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.45700000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40200000000000002,"15min":0.40400000000000003},"min":{"1min":0.30299999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.52300000000000002,"5min":0.754,"15min":0.754},"last":0.41299999999999998}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72299999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81299999999999994,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.498}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.39800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.75700000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.59599999999999997,"15min":0.60099999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.753,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.59399999999999997}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60399999999999998,"15min":0.60899999999999999},"min":{"1min":0.44600000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.76100000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.71199999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.51200000000000001}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60699999999999998,"15min":0.61199999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.745,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.80000000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.53500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":647168,"data_stored":291101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.28699999999999998,"5min":0.31,"15min":0.314},"min":{"1min":0.20799999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.38900000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.315,"15min":0.311},"min":{"1min":0.24099999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.63}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.311,"15min":0.309},"min":{"1min":0.253,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.378,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.53300000000000003}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36899999999999999,"15min":0.371},"min":{"1min":0.23899999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.47699999999999998,"5min":0.505,"15min":0.505},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.37,"15min":0.371},"min":{"1min":0.30299999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.45400000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.48699999999999999}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.41899999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.747,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.44400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.77600000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.91000000000000003}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56200000000000006,"15min":0.55700000000000005},"min":{"1min":0.46100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.873},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.44500000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.72599999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.80500000000000005}]},{"osd":6,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.47799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56699999999999995,"15min":0.56799999999999995},"min":{"1min":0.433,"5min":0.379,"15min":0.379},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.78600000000000003}]},{"osd":7,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.42899999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.69799999999999995,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.72899999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.89300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28244,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343596,"statfs":{"total":96636764160,"available":96607842304,"internally_reserved":0,"allocated":1249280,"data_stored":882954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.31900000000000001,"15min":0.32300000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.38200000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.29599999999999999,"5min":0.35499999999999998,"15min":0.36499999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.44500000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.26100000000000001}]},{"osd":2,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.35699999999999998,"15min":0.36399999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.44700000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.36699999999999999,"15min":0.371},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.47699999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.32600000000000001}]},{"osd":3,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.38500000000000001,"15min":0.39100000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.46400000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.313},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.41499999999999998,"15min":0.42099999999999999},"min":{"1min":0.309,"5min":0.307,"15min":0.307},"max":{"1min":0.48999999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.41799999999999998}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58899999999999997,"15min":0.59299999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60599999999999998,"15min":0.60899999999999999},"min":{"1min":0.436,"5min":0.318,"15min":0.318},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.54100000000000004}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.36199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76800000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.629,"15min":0.63600000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78900000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.52000000000000002}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.64100000000000001,"15min":0.64700000000000002},"min":{"1min":0.42499999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81000000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.627,"15min":0.63200000000000001},"min":{"1min":0.41699999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.63}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.44400000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.42999999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.71799999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.505}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-09T03:47:29.043 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-09T03:47:29.043 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-09T03:47:29.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.183+0000 7f959ab78640 1 Processor -- start 2023-11-09T03:47:29.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.183+0000 7f959ab78640 1 -- start start 2023-11-09T03:47:29.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.184+0000 7f959ab78640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9594070e20 0x7f95940adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:29.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.184+0000 7f959ab78640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f95940ad730 0x7f959415e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:29.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.184+0000 7f959ab78640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95940c3900 con 0x7f959415ad30 2023-11-09T03:47:29.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.184+0000 7f959ab78640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95940ae260 con 0x7f95940ad730 2023-11-09T03:47:29.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.184+0000 7f959ab78640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95940c3be0 con 0x7f9594070e20 2023-11-09T03:47:29.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.184+0000 7f9598914640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f959415ad30 0x7f9594070710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48032/0 (socket says 172.21.15.76:48032) 2023-11-09T03:47:29.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.184+0000 7f9598914640 1 -- 172.21.15.76:0/983945881 learned_addr learned my addr 172.21.15.76:0/983945881 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:29.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.184+0000 7f9599115640 1 --2- 172.21.15.76:0/983945881 >> v2:172.21.15.113:3300/0 conn(0x7f95940ad730 0x7f959415e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:29.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.184+0000 7f9593fff640 1 --2- 172.21.15.76:0/983945881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9594070e20 0x7f95940adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:29.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.184+0000 7f95937fe640 1 -- 172.21.15.76:0/983945881 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1442892062 0 0) 0x7f95940c3900 con 0x7f959415ad30 2023-11-09T03:47:29.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.184+0000 7f95937fe640 1 -- 172.21.15.76:0/983945881 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f957c003520 con 0x7f959415ad30 2023-11-09T03:47:29.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.185+0000 7f95937fe640 1 -- 172.21.15.76:0/983945881 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1938604871 0 0) 0x7f957c003520 con 0x7f959415ad30 2023-11-09T03:47:29.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.185+0000 7f95937fe640 1 -- 172.21.15.76:0/983945881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9594070e20 msgr2=0x7f95940adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:29.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.185+0000 7f95937fe640 1 --2- 172.21.15.76:0/983945881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9594070e20 0x7f95940adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:29.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.185+0000 7f95937fe640 1 -- 172.21.15.76:0/983945881 >> v2:172.21.15.113:3300/0 conn(0x7f95940ad730 msgr2=0x7f959415e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:29.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.185+0000 7f95937fe640 1 --2- 172.21.15.76:0/983945881 >> v2:172.21.15.113:3300/0 conn(0x7f95940ad730 0x7f959415e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:29.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.185+0000 7f95937fe640 1 -- 172.21.15.76:0/983945881 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95940c3ec0 con 0x7f959415ad30 2023-11-09T03:47:29.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.185+0000 7f95937fe640 1 -- 172.21.15.76:0/983945881 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f9584001670 con 0x7f959415ad30 2023-11-09T03:47:29.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.185+0000 7f95937fe640 1 -- 172.21.15.76:0/983945881 >> v1:172.21.15.76:6789/0 conn(0x7f959415ad30 legacy=0x7f9594070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:29.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.185+0000 7f95937fe640 1 --2- 172.21.15.76:0/983945881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9594070e20 0x7f957c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:29.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.185+0000 7f95937fe640 1 --2- 172.21.15.76:0/983945881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f957c003ef0 0x7f957c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:29.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.185+0000 7f95937fe640 1 --2- 172.21.15.76:0/983945881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f957c004d00 0x7f957c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:29.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.185+0000 7f95937fe640 1 -- 172.21.15.76:0/983945881 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95940ae260 con 0x7f957c003ef0 2023-11-09T03:47:29.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.185+0000 7f95937fe640 1 -- 172.21.15.76:0/983945881 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95940c3be0 con 0x7f957c004d00 2023-11-09T03:47:29.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.185+0000 7f95937fe640 1 -- 172.21.15.76:0/983945881 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f957c00b9d0 con 0x7f9594070e20 2023-11-09T03:47:29.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.185+0000 7f9593fff640 1 --2- 172.21.15.76:0/983945881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f957c003ef0 0x7f957c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:29.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.185+0000 7f9599115640 1 --2- 172.21.15.76:0/983945881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f957c004d00 0x7f957c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:29.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.185+0000 7f9598914640 1 --2- 172.21.15.76:0/983945881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9594070e20 0x7f957c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:29.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.185+0000 7f9593fff640 1 -- 172.21.15.76:0/983945881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9594070e20 msgr2=0x7f957c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:29.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.185+0000 7f9593fff640 1 --2- 172.21.15.76:0/983945881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9594070e20 0x7f957c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:29.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.185+0000 7f9593fff640 1 -- 172.21.15.76:0/983945881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f957c004d00 msgr2=0x7f957c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:29.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.185+0000 7f9593fff640 1 --2- 172.21.15.76:0/983945881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f957c004d00 0x7f957c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:29.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.185+0000 7f9593fff640 1 -- 172.21.15.76:0/983945881 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957c003520 con 0x7f957c003ef0 2023-11-09T03:47:29.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.185+0000 7f9598914640 1 --2- 172.21.15.76:0/983945881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9594070e20 0x7f957c003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:29.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.186+0000 7f9593fff640 1 --2- 172.21.15.76:0/983945881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f957c003ef0 0x7f957c0047c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9588009af0 tx=0x7f957c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1288206203ebd6fd server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:29.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.186+0000 7f95937fe640 1 -- 172.21.15.76:0/983945881 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958802c6b0 con 0x7f957c003ef0 2023-11-09T03:47:29.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.186+0000 7f95937fe640 1 -- 172.21.15.76:0/983945881 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958802c810 con 0x7f957c003ef0 2023-11-09T03:47:29.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.186+0000 7f95937fe640 1 -- 172.21.15.76:0/983945881 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958802cba0 con 0x7f957c003ef0 2023-11-09T03:47:29.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.186+0000 7f959ab78640 1 -- 172.21.15.76:0/983945881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f957c003ef0 msgr2=0x7f957c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:29.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.186+0000 7f959ab78640 1 --2- 172.21.15.76:0/983945881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f957c003ef0 0x7f957c0047c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9588009af0 tx=0x7f957c0057f0 comp rx=0 tx=0).stop 2023-11-09T03:47:29.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.186+0000 7f9599115640 1 -- 172.21.15.76:0/983945881 reap_dead start 2023-11-09T03:47:29.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.186+0000 7f959ab78640 1 -- 172.21.15.76:0/983945881 shutdown_connections 2023-11-09T03:47:29.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.186+0000 7f959ab78640 1 -- 172.21.15.76:0/983945881 >> 172.21.15.76:0/983945881 conn(0x7f95940ce950 msgr2=0x7f95940ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:29.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.186+0000 7f959ab78640 1 -- 172.21.15.76:0/983945881 shutdown_connections 2023-11-09T03:47:29.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.186+0000 7f959ab78640 1 -- 172.21.15.76:0/983945881 wait complete. 2023-11-09T03:47:29.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.188+0000 7f959ab78640 1 Processor -- start 2023-11-09T03:47:29.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.188+0000 7f959ab78640 1 -- start start 2023-11-09T03:47:29.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.188+0000 7f959ab78640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f957c003ef0 0x7f95940b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:29.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.188+0000 7f959ab78640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9594151770 0x7f95941510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:29.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.188+0000 7f959ab78640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95940c80f0 0x7f95940c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:29.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.189+0000 7f959ab78640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95940b2160 con 0x7f9594151770 2023-11-09T03:47:29.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.189+0000 7f959ab78640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9594151600 con 0x7f957c003ef0 2023-11-09T03:47:29.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.189+0000 7f959ab78640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95940c8ae0 con 0x7f95940c80f0 2023-11-09T03:47:29.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.189+0000 7f9593fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9594151770 0x7f95941510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:29.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.189+0000 7f9593fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9594151770 0x7f95941510c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42624/0 (socket says 172.21.15.76:42624) 2023-11-09T03:47:29.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.189+0000 7f9593fff640 1 -- 172.21.15.76:0/2413389968 learned_addr learned my addr 172.21.15.76:0/2413389968 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:29.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.189+0000 7f9598914640 1 --2- 172.21.15.76:0/2413389968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f957c003ef0 0x7f95940b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:29.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.189+0000 7f9599115640 1 --2- 172.21.15.76:0/2413389968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95940c80f0 0x7f95940c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:29.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.189+0000 7f9593fff640 1 -- 172.21.15.76:0/2413389968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95940c80f0 msgr2=0x7f95940c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:29.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.189+0000 7f9593fff640 1 --2- 172.21.15.76:0/2413389968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95940c80f0 0x7f95940c85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:29.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.189+0000 7f9593fff640 1 -- 172.21.15.76:0/2413389968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f957c003ef0 msgr2=0x7f95940b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:29.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.189+0000 7f9593fff640 1 --2- 172.21.15.76:0/2413389968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f957c003ef0 0x7f95940b2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:29.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.189+0000 7f9593fff640 1 -- 172.21.15.76:0/2413389968 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95940b3cc0 con 0x7f9594151770 2023-11-09T03:47:29.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.189+0000 7f9593fff640 1 --2- 172.21.15.76:0/2413389968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9594151770 0x7f95941510c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f9588037040 tx=0x7f9588004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:29.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.189+0000 7f95917fa640 1 -- 172.21.15.76:0/2413389968 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9588003810 con 0x7f9594151770 2023-11-09T03:47:29.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.189+0000 7f95917fa640 1 -- 172.21.15.76:0/2413389968 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9588003970 con 0x7f9594151770 2023-11-09T03:47:29.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.189+0000 7f959ab78640 1 -- 172.21.15.76:0/2413389968 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95940b2c40 con 0x7f9594151770 2023-11-09T03:47:29.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.189+0000 7f95917fa640 1 -- 172.21.15.76:0/2413389968 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9588003ce0 con 0x7f9594151770 2023-11-09T03:47:29.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.189+0000 7f959ab78640 1 -- 172.21.15.76:0/2413389968 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95940b3200 con 0x7f9594151770 2023-11-09T03:47:29.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.190+0000 7f959ab78640 1 -- 172.21.15.76:0/2413389968 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9594074290 con 0x7f9594151770 2023-11-09T03:47:29.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.191+0000 7f95917fa640 1 -- 172.21.15.76:0/2413389968 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9588040450 con 0x7f9594151770 2023-11-09T03:47:29.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.191+0000 7f95917fa640 1 --2- 172.21.15.76:0/2413389968 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f958006aea0 0x7f958006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:29.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.191+0000 7f95917fa640 1 -- 172.21.15.76:0/2413389968 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f95880ab9a0 con 0x7f9594151770 2023-11-09T03:47:29.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.191+0000 7f9598914640 1 --2- 172.21.15.76:0/2413389968 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f958006aea0 0x7f958006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:29.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.192+0000 7f9598914640 1 --2- 172.21.15.76:0/2413389968 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f958006aea0 0x7f958006d360 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9584000fb0 tx=0x7f9584009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:29.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.195+0000 7f95917fa640 1 -- 172.21.15.76:0/2413389968 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9588076960 con 0x7f9594151770 2023-11-09T03:47:29.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.427+0000 7f959ab78640 1 -- 172.21.15.76:0/2413389968 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f9594071970 con 0x7f9594151770 2023-11-09T03:47:29.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.429+0000 7f95917fa640 1 -- 172.21.15.76:0/2413389968 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7f95880440b0 con 0x7f9594151770 2023-11-09T03:47:29.428 INFO:teuthology.orchestra.run.smithi076.stdout:[client.0] 2023-11-09T03:47:29.428 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQChVUxlHj22CxAACNj5X24hdME+GNTLw4RahQ== 2023-11-09T03:47:29.429 INFO:teuthology.orchestra.run.smithi076.stdout: caps mds = "allow" 2023-11-09T03:47:29.429 INFO:teuthology.orchestra.run.smithi076.stdout: caps mon = "allow r" 2023-11-09T03:47:29.429 INFO:teuthology.orchestra.run.smithi076.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-09T03:47:29.429 INFO:teuthology.orchestra.run.smithi076.stderr:updated caps for client.0 2023-11-09T03:47:29.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.431+0000 7f959ab78640 1 -- 172.21.15.76:0/2413389968 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f958006aea0 msgr2=0x7f958006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:29.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.431+0000 7f959ab78640 1 --2- 172.21.15.76:0/2413389968 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f958006aea0 0x7f958006d360 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9584000fb0 tx=0x7f9584009040 comp rx=0 tx=0).stop 2023-11-09T03:47:29.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.431+0000 7f959ab78640 1 -- 172.21.15.76:0/2413389968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9594151770 msgr2=0x7f95941510c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:29.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.431+0000 7f959ab78640 1 --2- 172.21.15.76:0/2413389968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9594151770 0x7f95941510c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f9588037040 tx=0x7f9588004730 comp rx=0 tx=0).stop 2023-11-09T03:47:29.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.431+0000 7f959ab78640 1 -- 172.21.15.76:0/2413389968 shutdown_connections 2023-11-09T03:47:29.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.431+0000 7f959ab78640 1 --2- 172.21.15.76:0/2413389968 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f958006aea0 0x7f958006d360 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:29.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.431+0000 7f959ab78640 1 --2- 172.21.15.76:0/2413389968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95940c80f0 0x7f95940c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:29.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.431+0000 7f959ab78640 1 --2- 172.21.15.76:0/2413389968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9594151770 0x7f95941510c0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:29.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.431+0000 7f959ab78640 1 --2- 172.21.15.76:0/2413389968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f957c003ef0 0x7f95940b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:29.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.431+0000 7f959ab78640 1 -- 172.21.15.76:0/2413389968 >> 172.21.15.76:0/2413389968 conn(0x7f95940ce950 msgr2=0x7f95940c9810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:29.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.431+0000 7f959ab78640 1 -- 172.21.15.76:0/2413389968 shutdown_connections 2023-11-09T03:47:29.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.431+0000 7f959ab78640 1 -- 172.21.15.76:0/2413389968 wait complete. 2023-11-09T03:47:29.441 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:47:29.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.541+0000 7f0b9b439640 1 Processor -- start 2023-11-09T03:47:29.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.542+0000 7f0b9b439640 1 -- start start 2023-11-09T03:47:29.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.542+0000 7f0b9b439640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b940ce370 0x7f0b9406db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:29.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.542+0000 7f0b9b439640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b9406e060 con 0x7f0b94159c70 2023-11-09T03:47:29.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.542+0000 7f0b9b439640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b940b4110 con 0x7f0b940b0ae0 2023-11-09T03:47:29.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.542+0000 7f0b9b439640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b940b49b0 con 0x7f0b940ce370 2023-11-09T03:47:29.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.543+0000 7f0b991d5640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0b94159c70 0x7f0b940b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48044/0 (socket says 172.21.15.76:48044) 2023-11-09T03:47:29.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.543+0000 7f0b991d5640 1 -- 172.21.15.76:0/1877029463 learned_addr learned my addr 172.21.15.76:0/1877029463 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:29.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.543+0000 7f0b999d6640 1 --2- 172.21.15.76:0/1877029463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b940ce370 0x7f0b9406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:29.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.543+0000 7f0b8bfff640 1 -- 172.21.15.76:0/1877029463 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3025844532 0 0) 0x7f0b9406e060 con 0x7f0b94159c70 2023-11-09T03:47:29.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.543+0000 7f0b8bfff640 1 -- 172.21.15.76:0/1877029463 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b80005730 con 0x7f0b94159c70 2023-11-09T03:47:29.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.543+0000 7f0b8bfff640 1 -- 172.21.15.76:0/1877029463 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4127739163 0 0) 0x7f0b80005730 con 0x7f0b94159c70 2023-11-09T03:47:29.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.543+0000 7f0b8bfff640 1 -- 172.21.15.76:0/1877029463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b940ce370 msgr2=0x7f0b9406db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:29.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.543+0000 7f0b8bfff640 1 --2- 172.21.15.76:0/1877029463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b940ce370 0x7f0b9406db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:29.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.543+0000 7f0b8bfff640 1 -- 172.21.15.76:0/1877029463 >> v1:172.21.15.113:6789/0 conn(0x7f0b940b0ae0 legacy=0x7f0b9415d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:29.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.543+0000 7f0b8bfff640 1 -- 172.21.15.76:0/1877029463 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b940b43f0 con 0x7f0b94159c70 2023-11-09T03:47:29.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.543+0000 7f0b8bfff640 1 -- 172.21.15.76:0/1877029463 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f0b84001670 con 0x7f0b94159c70 2023-11-09T03:47:29.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.543+0000 7f0b8bfff640 1 -- 172.21.15.76:0/1877029463 >> v1:172.21.15.76:6789/0 conn(0x7f0b94159c70 legacy=0x7f0b940b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:29.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.543+0000 7f0b8bfff640 1 --2- 172.21.15.76:0/1877029463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b80005bb0 0x7f0b80005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:29.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.543+0000 7f0b8bfff640 1 --2- 172.21.15.76:0/1877029463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b940ce370 0x7f0b800069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:29.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.543+0000 7f0b8bfff640 1 --2- 172.21.15.76:0/1877029463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b80006f10 0x7f0b80007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:29.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.543+0000 7f0b8bfff640 1 -- 172.21.15.76:0/1877029463 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b940b49b0 con 0x7f0b80006f10 2023-11-09T03:47:29.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.543+0000 7f0b8bfff640 1 -- 172.21.15.76:0/1877029463 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8000dc40 con 0x7f0b80005bb0 2023-11-09T03:47:29.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.543+0000 7f0b8bfff640 1 -- 172.21.15.76:0/1877029463 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8000de00 con 0x7f0b940ce370 2023-11-09T03:47:29.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.543+0000 7f0b999d6640 1 --2- 172.21.15.76:0/1877029463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b80006f10 0x7f0b80007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:29.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.543+0000 7f0b991d5640 1 --2- 172.21.15.76:0/1877029463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b80005bb0 0x7f0b80005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:29.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.543+0000 7f0b989d4640 1 --2- 172.21.15.76:0/1877029463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b940ce370 0x7f0b800069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:29.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.544+0000 7f0b999d6640 1 -- 172.21.15.76:0/1877029463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b940ce370 msgr2=0x7f0b800069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:29.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.544+0000 7f0b999d6640 1 --2- 172.21.15.76:0/1877029463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b940ce370 0x7f0b800069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:29.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.544+0000 7f0b999d6640 1 -- 172.21.15.76:0/1877029463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b80005bb0 msgr2=0x7f0b80005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:29.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.544+0000 7f0b999d6640 1 --2- 172.21.15.76:0/1877029463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b80005bb0 0x7f0b80005fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:29.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.544+0000 7f0b999d6640 1 -- 172.21.15.76:0/1877029463 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b80005730 con 0x7f0b80006f10 2023-11-09T03:47:29.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.544+0000 7f0b999d6640 1 --2- 172.21.15.76:0/1877029463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b80006f10 0x7f0b80007390 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009f90 tx=0x7f0b80007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f878b69798fecf server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:29.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.545+0000 7f0b8bfff640 1 -- 172.21.15.76:0/1877029463 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c02cbd0 con 0x7f0b80006f10 2023-11-09T03:47:29.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.545+0000 7f0b8bfff640 1 -- 172.21.15.76:0/1877029463 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8c00f040 con 0x7f0b80006f10 2023-11-09T03:47:29.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.545+0000 7f0b8bfff640 1 -- 172.21.15.76:0/1877029463 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c034440 con 0x7f0b80006f10 2023-11-09T03:47:29.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.545+0000 7f0b9b439640 1 -- 172.21.15.76:0/1877029463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b80006f10 msgr2=0x7f0b80007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:29.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.545+0000 7f0b9b439640 1 --2- 172.21.15.76:0/1877029463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b80006f10 0x7f0b80007390 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009f90 tx=0x7f0b80007a00 comp rx=0 tx=0).stop 2023-11-09T03:47:29.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.545+0000 7f0b999d6640 1 -- 172.21.15.76:0/1877029463 reap_dead start 2023-11-09T03:47:29.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.545+0000 7f0b9b439640 1 -- 172.21.15.76:0/1877029463 shutdown_connections 2023-11-09T03:47:29.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.545+0000 7f0b9b439640 1 -- 172.21.15.76:0/1877029463 >> 172.21.15.76:0/1877029463 conn(0x7f0b940ce950 msgr2=0x7f0b940ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:29.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.545+0000 7f0b9b439640 1 -- 172.21.15.76:0/1877029463 shutdown_connections 2023-11-09T03:47:29.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.545+0000 7f0b9b439640 1 -- 172.21.15.76:0/1877029463 wait complete. 2023-11-09T03:47:29.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.546+0000 7f0b9b439640 1 Processor -- start 2023-11-09T03:47:29.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.546+0000 7f0b9b439640 1 -- start start 2023-11-09T03:47:29.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.547+0000 7f0b9b439640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b94147e80 0x7f0b9415ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:29.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.547+0000 7f0b9b439640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b941545a0 0x7f0b940ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:29.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.547+0000 7f0b9b439640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b940adc20 0x7f0b9415c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:29.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.547+0000 7f0b9b439640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b940ac310 con 0x7f0b940adc20 2023-11-09T03:47:29.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.547+0000 7f0b9b439640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9415c270 con 0x7f0b94147e80 2023-11-09T03:47:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.547+0000 7f0b9b439640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9415b390 con 0x7f0b941545a0 2023-11-09T03:47:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.547+0000 7f0b999d6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b940adc20 0x7f0b9415c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.547+0000 7f0b999d6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b940adc20 0x7f0b9415c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42632/0 (socket says 172.21.15.76:42632) 2023-11-09T03:47:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.547+0000 7f0b999d6640 1 -- 172.21.15.76:0/3540072157 learned_addr learned my addr 172.21.15.76:0/3540072157 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.547+0000 7f0b999d6640 1 -- 172.21.15.76:0/3540072157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b941545a0 msgr2=0x7f0b940ad6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.547+0000 7f0b999d6640 1 --2- 172.21.15.76:0/3540072157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b941545a0 0x7f0b940ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.547+0000 7f0b999d6640 1 -- 172.21.15.76:0/3540072157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b94147e80 msgr2=0x7f0b9415ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.547+0000 7f0b999d6640 1 --2- 172.21.15.76:0/3540072157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b94147e80 0x7f0b9415ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.547+0000 7f0b999d6640 1 -- 172.21.15.76:0/3540072157 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b94148260 con 0x7f0b940adc20 2023-11-09T03:47:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.548+0000 7f0b999d6640 1 --2- 172.21.15.76:0/3540072157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b940adc20 0x7f0b9415c700 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c037040 tx=0x7f0b8c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.548+0000 7f0b89ffb640 1 -- 172.21.15.76:0/3540072157 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c043070 con 0x7f0b940adc20 2023-11-09T03:47:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.548+0000 7f0b89ffb640 1 -- 172.21.15.76:0/3540072157 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8c003990 con 0x7f0b940adc20 2023-11-09T03:47:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.548+0000 7f0b9b439640 1 -- 172.21.15.76:0/3540072157 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b9415b660 con 0x7f0b940adc20 2023-11-09T03:47:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.548+0000 7f0b89ffb640 1 -- 172.21.15.76:0/3540072157 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c003d40 con 0x7f0b940adc20 2023-11-09T03:47:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.548+0000 7f0b9b439640 1 -- 172.21.15.76:0/3540072157 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b9415bc20 con 0x7f0b940adc20 2023-11-09T03:47:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.549+0000 7f0b9b439640 1 -- 172.21.15.76:0/3540072157 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b94074290 con 0x7f0b940adc20 2023-11-09T03:47:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.549+0000 7f0b89ffb640 1 -- 172.21.15.76:0/3540072157 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0b8c034860 con 0x7f0b940adc20 2023-11-09T03:47:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.550+0000 7f0b89ffb640 1 --2- 172.21.15.76:0/3540072157 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0b7806ae70 0x7f0b7806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.550+0000 7f0b89ffb640 1 -- 172.21.15.76:0/3540072157 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f0b8c0ab3a0 con 0x7f0b940adc20 2023-11-09T03:47:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.550+0000 7f0b991d5640 1 --2- 172.21.15.76:0/3540072157 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0b7806ae70 0x7f0b7806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.551+0000 7f0b991d5640 1 --2- 172.21.15.76:0/3540072157 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0b7806ae70 0x7f0b7806d330 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0b84004840 tx=0x7f0b84000ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:29.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.554+0000 7f0b89ffb640 1 -- 172.21.15.76:0/3540072157 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0b8c0761b0 con 0x7f0b940adc20 2023-11-09T03:47:29.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.754+0000 7f0b9b439640 1 -- 172.21.15.76:0/3540072157 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b9415c3b0 con 0x7f0b940adc20 2023-11-09T03:47:29.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.755+0000 7f0b89ffb640 1 -- 172.21.15.76:0/3540072157 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f0b8c0797d0 con 0x7f0b940adc20 2023-11-09T03:47:29.754 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:29.755 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5099,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1844478483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1844478483},{"type":"v1","addr":"172.21.15.113:6836","nonce":1844478483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5102,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1111318612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1111318612},{"type":"v1","addr":"172.21.15.113:6837","nonce":1111318612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":6072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/2037805253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2037805253},{"type":"v1","addr":"172.21.15.76:6834","nonce":2037805253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:47:20.265190+0000","modified":"2023-11-09T03:47:22.742442+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6075},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6075":{"gid":6075,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/3710140065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":3710140065},{"type":"v1","addr":"172.21.15.76:6835","nonce":3710140065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-09T03:47:29.755 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 23 2023-11-09T03:47:29.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.756+0000 7f0b9b439640 1 -- 172.21.15.76:0/3540072157 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0b7806ae70 msgr2=0x7f0b7806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:29.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.756+0000 7f0b9b439640 1 --2- 172.21.15.76:0/3540072157 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0b7806ae70 0x7f0b7806d330 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0b84004840 tx=0x7f0b84000ed0 comp rx=0 tx=0).stop 2023-11-09T03:47:29.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.756+0000 7f0b9b439640 1 -- 172.21.15.76:0/3540072157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b940adc20 msgr2=0x7f0b9415c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:29.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.756+0000 7f0b9b439640 1 --2- 172.21.15.76:0/3540072157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b940adc20 0x7f0b9415c700 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c037040 tx=0x7f0b8c009750 comp rx=0 tx=0).stop 2023-11-09T03:47:29.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.757+0000 7f0b9b439640 1 -- 172.21.15.76:0/3540072157 shutdown_connections 2023-11-09T03:47:29.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.757+0000 7f0b9b439640 1 --2- 172.21.15.76:0/3540072157 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0b7806ae70 0x7f0b7806d330 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:29.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.757+0000 7f0b9b439640 1 --2- 172.21.15.76:0/3540072157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b940adc20 0x7f0b9415c700 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:29.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.757+0000 7f0b9b439640 1 --2- 172.21.15.76:0/3540072157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b941545a0 0x7f0b940ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:29.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.757+0000 7f0b9b439640 1 --2- 172.21.15.76:0/3540072157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b94147e80 0x7f0b9415ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:29.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.757+0000 7f0b9b439640 1 -- 172.21.15.76:0/3540072157 >> 172.21.15.76:0/3540072157 conn(0x7f0b940ce950 msgr2=0x7f0b940caed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:29.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.757+0000 7f0b9b439640 1 -- 172.21.15.76:0/3540072157 shutdown_connections 2023-11-09T03:47:29.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:29.757+0000 7f0b9b439640 1 -- 172.21.15.76:0/3540072157 wait complete. 2023-11-09T03:47:30.169 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-09T03:47:20.265190+0000', 'modified': '2023-11-09T03:47:22.742442+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6075}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6075': {'gid': 6075, 'name': 'c', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6835/3710140065', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6833', 'nonce': 3710140065}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 3710140065}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-09T03:47:30.169 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-09T03:47:30.170 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-09T03:47:30.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.273+0000 7f15af9a9640 1 Processor -- start 2023-11-09T03:47:30.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.273+0000 7f15af9a9640 1 -- start start 2023-11-09T03:47:30.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.274+0000 7f15af9a9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f15a8070e20 0x7f15a80adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:30.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.274+0000 7f15af9a9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a80c3900 con 0x7f15a815ad30 2023-11-09T03:47:30.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.274+0000 7f15af9a9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a80c3be0 con 0x7f15a80ad730 2023-11-09T03:47:30.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.274+0000 7f15af9a9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a80ae260 con 0x7f15a8070e20 2023-11-09T03:47:30.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.274+0000 7f15acf44640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f15a8070e20 0x7f15a80adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:30.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.274+0000 7f15ad745640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f15a815ad30 0x7f15a8070710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48052/0 (socket says 172.21.15.76:48052) 2023-11-09T03:47:30.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.274+0000 7f15ad745640 1 -- 172.21.15.76:0/2352370880 learned_addr learned my addr 172.21.15.76:0/2352370880 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:30.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.275+0000 7f15acf44640 1 -- 172.21.15.76:0/2352370880 >> v1:172.21.15.113:6789/0 conn(0x7f15a80ad730 legacy=0x7f15a815e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:30.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.275+0000 7f15acf44640 1 -- 172.21.15.76:0/2352370880 >> v1:172.21.15.76:6789/0 conn(0x7f15a815ad30 legacy=0x7f15a8070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:30.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.275+0000 7f159bfff640 1 -- 172.21.15.76:0/2352370880 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1777137257 0 0) 0x7f15a80c3900 con 0x7f15a815ad30 2023-11-09T03:47:30.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.275+0000 7f15acf44640 1 -- 172.21.15.76:0/2352370880 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a80c3ec0 con 0x7f15a8070e20 2023-11-09T03:47:30.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.275+0000 7f15acf44640 1 --2- 172.21.15.76:0/2352370880 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15a8070e20 0x7f15a80adc80 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f159c0097a0 tx=0x7f159c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c36509a32f423d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:30.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.276+0000 7f159bfff640 1 -- 172.21.15.76:0/2352370880 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c031830 con 0x7f15a8070e20 2023-11-09T03:47:30.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.276+0000 7f159bfff640 1 -- 172.21.15.76:0/2352370880 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159c031990 con 0x7f15a8070e20 2023-11-09T03:47:30.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.276+0000 7f159bfff640 1 -- 172.21.15.76:0/2352370880 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c031d20 con 0x7f15a8070e20 2023-11-09T03:47:30.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.276+0000 7f15af9a9640 1 -- 172.21.15.76:0/2352370880 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15a8070e20 msgr2=0x7f15a80adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:30.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.276+0000 7f15af9a9640 1 --2- 172.21.15.76:0/2352370880 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15a8070e20 0x7f15a80adc80 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f159c0097a0 tx=0x7f159c02b8d0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.276+0000 7f15af9a9640 1 -- 172.21.15.76:0/2352370880 shutdown_connections 2023-11-09T03:47:30.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.276+0000 7f15af9a9640 1 --2- 172.21.15.76:0/2352370880 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15a8070e20 0x7f15a80adc80 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.276+0000 7f15af9a9640 1 -- 172.21.15.76:0/2352370880 >> 172.21.15.76:0/2352370880 conn(0x7f15a80ce950 msgr2=0x7f15a80ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:30.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.276+0000 7f15af9a9640 1 -- 172.21.15.76:0/2352370880 shutdown_connections 2023-11-09T03:47:30.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.276+0000 7f15af9a9640 1 -- 172.21.15.76:0/2352370880 wait complete. 2023-11-09T03:47:30.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.277+0000 7f15af9a9640 1 Processor -- start 2023-11-09T03:47:30.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.277+0000 7f15af9a9640 1 -- start start 2023-11-09T03:47:30.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.278+0000 7f15af9a9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15a8070e20 0x7f15a80cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:30.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.278+0000 7f15af9a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15a80ad730 0x7f15a8146380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:30.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.278+0000 7f15af9a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15a815ad30 0x7f15a8159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:30.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.278+0000 7f15af9a9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a80cdb10 con 0x7f15a8070e20 2023-11-09T03:47:30.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.278+0000 7f15af9a9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a815bb80 con 0x7f15a815ad30 2023-11-09T03:47:30.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.278+0000 7f15af9a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a81468c0 con 0x7f15a80ad730 2023-11-09T03:47:30.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.278+0000 7f15ad745640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15a8070e20 0x7f15a80cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:30.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.278+0000 7f15ad745640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15a8070e20 0x7f15a80cd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42650/0 (socket says 172.21.15.76:42650) 2023-11-09T03:47:30.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.278+0000 7f15ad745640 1 -- 172.21.15.76:0/415330215 learned_addr learned my addr 172.21.15.76:0/415330215 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:30.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.278+0000 7f15acf44640 1 --2- 172.21.15.76:0/415330215 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15a80ad730 0x7f15a8146380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:30.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.278+0000 7f15adf46640 1 --2- 172.21.15.76:0/415330215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15a815ad30 0x7f15a8159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:30.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.278+0000 7f15ad745640 1 -- 172.21.15.76:0/415330215 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15a80ad730 msgr2=0x7f15a8146380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:30.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.278+0000 7f15ad745640 1 --2- 172.21.15.76:0/415330215 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15a80ad730 0x7f15a8146380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.278+0000 7f15ad745640 1 -- 172.21.15.76:0/415330215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15a815ad30 msgr2=0x7f15a8159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:30.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.278+0000 7f15ad745640 1 --2- 172.21.15.76:0/415330215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15a815ad30 0x7f15a8159120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.278+0000 7f15ad745640 1 -- 172.21.15.76:0/415330215 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f159c009400 con 0x7f15a8070e20 2023-11-09T03:47:30.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.278+0000 7f15acf44640 1 --2- 172.21.15.76:0/415330215 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15a80ad730 0x7f15a8146380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:30.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.279+0000 7f15ad745640 1 --2- 172.21.15.76:0/415330215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15a8070e20 0x7f15a80cd5d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f15a400c3f0 tx=0x7f15a400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:30.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.279+0000 7f1599ffb640 1 -- 172.21.15.76:0/415330215 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a40074e0 con 0x7f15a8070e20 2023-11-09T03:47:30.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.279+0000 7f1599ffb640 1 -- 172.21.15.76:0/415330215 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a4007640 con 0x7f15a8070e20 2023-11-09T03:47:30.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.279+0000 7f15af9a9640 1 -- 172.21.15.76:0/415330215 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a80ce5a0 con 0x7f15a8070e20 2023-11-09T03:47:30.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.279+0000 7f1599ffb640 1 -- 172.21.15.76:0/415330215 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a4004a20 con 0x7f15a8070e20 2023-11-09T03:47:30.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.279+0000 7f15af9a9640 1 -- 172.21.15.76:0/415330215 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a80abb00 con 0x7f15a8070e20 2023-11-09T03:47:30.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.280+0000 7f1599ffb640 1 -- 172.21.15.76:0/415330215 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f15a4004b80 con 0x7f15a8070e20 2023-11-09T03:47:30.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.281+0000 7f1599ffb640 1 --2- 172.21.15.76:0/415330215 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f158c06ae70 0x7f158c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:30.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.281+0000 7f1599ffb640 1 -- 172.21.15.76:0/415330215 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f15a4089630 con 0x7f15a8070e20 2023-11-09T03:47:30.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.281+0000 7f15acf44640 1 --2- 172.21.15.76:0/415330215 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f158c06ae70 0x7f158c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:30.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.282+0000 7f15acf44640 1 --2- 172.21.15.76:0/415330215 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f158c06ae70 0x7f158c06d330 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f159c036000 tx=0x7f159c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:30.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.282+0000 7f15b261b740 1 Processor -- start 2023-11-09T03:47:30.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.282+0000 7f15b261b740 1 -- start start 2023-11-09T03:47:30.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.282+0000 7f15b261b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56541e66c850 0x56541e63f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:30.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.282+0000 7f15b261b740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56541e66cc30 0x56541e66f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:30.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.282+0000 7f15b261b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56541e63fc90 0x56541e66fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:30.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.282+0000 7f15b261b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541e550740 con 0x56541e66c850 2023-11-09T03:47:30.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.282+0000 7f15b261b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541e5b28d0 con 0x56541e63fc90 2023-11-09T03:47:30.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.282+0000 7f15b261b740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x56541e5affc0 con 0x56541e66cc30 2023-11-09T03:47:30.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.282+0000 7f15ad745640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56541e66c850 0x56541e63f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:30.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.282+0000 7f15ad745640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56541e66c850 0x56541e63f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42654/0 (socket says 172.21.15.76:42654) 2023-11-09T03:47:30.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.282+0000 7f15ad745640 1 -- 172.21.15.76:0/2329366901 learned_addr learned my addr 172.21.15.76:0/2329366901 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:30.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.283+0000 7f15ad745640 1 -- 172.21.15.76:0/2329366901 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56541e66cc30 msgr2=0x56541e66f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:30.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.283+0000 7f15acf44640 1 --2- 172.21.15.76:0/2329366901 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56541e66cc30 0x56541e66f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:30.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.283+0000 7f15adf46640 1 --2- 172.21.15.76:0/2329366901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56541e63fc90 0x56541e66fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:30.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.283+0000 7f15ad745640 1 --2- 172.21.15.76:0/2329366901 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56541e66cc30 0x56541e66f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.283+0000 7f15ad745640 1 -- 172.21.15.76:0/2329366901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56541e63fc90 msgr2=0x56541e66fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:30.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.283+0000 7f15ad745640 1 --2- 172.21.15.76:0/2329366901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56541e63fc90 0x56541e66fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.283+0000 7f15ad745640 1 -- 172.21.15.76:0/2329366901 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56541e56eca0 con 0x56541e66c850 2023-11-09T03:47:30.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.283+0000 7f15acf44640 1 --2- 172.21.15.76:0/2329366901 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56541e66cc30 0x56541e66f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:30.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.283+0000 7f15adf46640 1 --2- 172.21.15.76:0/2329366901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56541e63fc90 0x56541e66fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:30.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.283+0000 7f15ad745640 1 --2- 172.21.15.76:0/2329366901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56541e66c850 0x56541e63f750 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f15a40878f0 tx=0x7f15a4058010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d3e41c10fe1470 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:30.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.283+0000 7f1592ffd640 1 -- 172.21.15.76:0/2329366901 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a405c2e0 con 0x56541e66c850 2023-11-09T03:47:30.298 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.283+0000 7f1592ffd640 1 -- 172.21.15.76:0/2329366901 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a4060490 con 0x56541e66c850 2023-11-09T03:47:30.298 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.283+0000 7f1592ffd640 1 -- 172.21.15.76:0/2329366901 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a40548a0 con 0x56541e66c850 2023-11-09T03:47:30.298 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.284+0000 7f15b261b740 1 -- 172.21.15.76:0/2329366901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56541e66c850 msgr2=0x56541e63f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:30.298 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.284+0000 7f15b261b740 1 --2- 172.21.15.76:0/2329366901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56541e66c850 0x56541e63f750 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f15a40878f0 tx=0x7f15a4058010 comp rx=0 tx=0).stop 2023-11-09T03:47:30.298 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.284+0000 7f15b261b740 1 -- 172.21.15.76:0/2329366901 shutdown_connections 2023-11-09T03:47:30.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.284+0000 7f15b261b740 1 --2- 172.21.15.76:0/2329366901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56541e63fc90 0x56541e66fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.284+0000 7f15b261b740 1 --2- 172.21.15.76:0/2329366901 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56541e66cc30 0x56541e66f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.284+0000 7f15b261b740 1 --2- 172.21.15.76:0/2329366901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56541e66c850 0x56541e63f750 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.284+0000 7f15b261b740 1 -- 172.21.15.76:0/2329366901 >> 172.21.15.76:0/2329366901 conn(0x56541e641a60 msgr2=0x56541e68a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:30.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.284+0000 7f15b261b740 1 -- 172.21.15.76:0/2329366901 shutdown_connections 2023-11-09T03:47:30.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.284+0000 7f15b261b740 1 -- 172.21.15.76:0/2329366901 wait complete. 2023-11-09T03:47:30.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.284+0000 7f15b261b740 1 -- start start 2023-11-09T03:47:30.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.284+0000 7f15b261b740 1 Processor -- start 2023-11-09T03:47:30.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.285+0000 7f15b261b740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56541e66c850 0x56541e672030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:30.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.285+0000 7f15b261b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56541e66cc30 0x56541e69d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:30.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.285+0000 7f15b261b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56541e69def0 0x56541e683cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:30.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.285+0000 7f15b261b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541e5ab8b0 con 0x56541e66cc30 2023-11-09T03:47:30.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.285+0000 7f15b261b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541e565c80 con 0x56541e69def0 2023-11-09T03:47:30.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.285+0000 7f15b261b740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x56541e565920 con 0x56541e66c850 2023-11-09T03:47:30.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.285+0000 7f15b261b740 10 client.0 _ll_register_callbacks cb 0x56541e2bdac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:47:30.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.285+0000 7f15acf44640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56541e66cc30 0x56541e69d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:30.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.285+0000 7f15acf44640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56541e66cc30 0x56541e69d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42656/0 (socket says 172.21.15.76:42656) 2023-11-09T03:47:30.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.285+0000 7f15acf44640 1 -- 172.21.15.76:0/492504807 learned_addr learned my addr 172.21.15.76:0/492504807 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:30.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.285+0000 7f15ad745640 1 --2- 172.21.15.76:0/492504807 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56541e66c850 0x56541e672030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:30.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.285+0000 7f15adf46640 1 --2- 172.21.15.76:0/492504807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56541e69def0 0x56541e683cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:30.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.285+0000 7f15acf44640 1 -- 172.21.15.76:0/492504807 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56541e66c850 msgr2=0x56541e672030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:30.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.285+0000 7f15acf44640 1 --2- 172.21.15.76:0/492504807 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56541e66c850 0x56541e672030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.285+0000 7f15acf44640 1 -- 172.21.15.76:0/492504807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56541e69def0 msgr2=0x56541e683cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:30.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.285+0000 7f15acf44640 1 --2- 172.21.15.76:0/492504807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56541e69def0 0x56541e683cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.285+0000 7f15acf44640 1 -- 172.21.15.76:0/492504807 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a80abb00 con 0x56541e66cc30 2023-11-09T03:47:30.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.286+0000 7f15acf44640 1 --2- 172.21.15.76:0/492504807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56541e66cc30 0x56541e69d9b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f159c009890 tx=0x7f159c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f5352393134168 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:30.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.286+0000 7f1587fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:47:30.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.286+0000 7f1587fff640 1 -- 172.21.15.76:0/492504807 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c04b070 con 0x56541e66cc30 2023-11-09T03:47:30.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.286+0000 7f15b261b740 1 -- 172.21.15.76:0/492504807 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56541e557bf0 con 0x56541e66cc30 2023-11-09T03:47:30.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.286+0000 7f1587fff640 1 -- 172.21.15.76:0/492504807 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f159c04f410 con 0x56541e66cc30 2023-11-09T03:47:30.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.286+0000 7f1587fff640 1 -- 172.21.15.76:0/492504807 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f159c03f070 con 0x56541e66cc30 2023-11-09T03:47:30.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.286+0000 7f15b261b740 10 client.6255 fetch_fsmap learned FSMap version 23 2023-11-09T03:47:30.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.286+0000 7f15b261b740 1 -- 172.21.15.76:0/492504807 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x56541e5a9f30 con 0x56541e66cc30 2023-11-09T03:47:30.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.286+0000 7f1587fff640 1 -- 172.21.15.76:0/492504807 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56541e557bf0 con 0x56541e66cc30 2023-11-09T03:47:30.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.286+0000 7f1587fff640 1 -- 172.21.15.76:0/492504807 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f159c04f5d0 con 0x56541e66cc30 2023-11-09T03:47:30.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.286+0000 7f15b261b740 10 client.6255 fetch_fsmap finished waiting for FSMap version 23 2023-11-09T03:47:30.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.286+0000 7f15b261b740 10 client.6255 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-09T03:47:30.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.286+0000 7f15b261b740 20 client.6255 populate_metadata read hostname 'smithi076' 2023-11-09T03:47:30.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.286+0000 7f15b261b740 1 --2- 172.21.15.76:0/492504807 >> [v2:172.21.15.76:6833/3710140065,v1:172.21.15.76:6835/3710140065] conn(0x56541e65d060 0x56541e65d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:30.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.286+0000 7f15b261b740 4 client.6255 mds_command: new command op to 6075 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:47:30.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.286+0000 7f15b261b740 1 -- 172.21.15.76:0/492504807 --> [v2:172.21.15.76:6833/3710140065,v1:172.21.15.76:6835/3710140065] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56541e5a4890 con 0x56541e65d060 2023-11-09T03:47:30.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.286+0000 7f15ad745640 1 --2- 172.21.15.76:0/492504807 >> [v2:172.21.15.76:6833/3710140065,v1:172.21.15.76:6835/3710140065] conn(0x56541e65d060 0x56541e65d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:30.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.287+0000 7f1587fff640 10 client.6255 ms_handle_connect on v2:172.21.15.76:6833/3710140065 2023-11-09T03:47:30.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.287+0000 7f15ad745640 1 --2- 172.21.15.76:0/492504807 >> [v2:172.21.15.76:6833/3710140065,v1:172.21.15.76:6835/3710140065] conn(0x56541e65d060 0x56541e65d460 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6201f18753644c5 server_cookie=4a50493310ab9491 in_seq=0 out_seq=0 2023-11-09T03:47:30.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.287+0000 7f1587fff640 1 -- 172.21.15.76:0/492504807 <== mds.0 v2:172.21.15.76:6833/3710140065 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56541e5a4890 con 0x56541e65d060 2023-11-09T03:47:30.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.287+0000 7f1587fff640 10 client.6255 handle_command_reply: tid=0 2023-11-09T03:47:30.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.287+0000 7f15b261b740 1 client.6255 shutdown 2023-11-09T03:47:30.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.287+0000 7f15b261b740 1 -- 172.21.15.76:0/492504807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56541e66cc30 msgr2=0x56541e69d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:30.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.287+0000 7f15b261b740 1 --2- 172.21.15.76:0/492504807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56541e66cc30 0x56541e69d9b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f159c009890 tx=0x7f159c004730 comp rx=0 tx=0).stop 2023-11-09T03:47:30.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.287+0000 7f15b261b740 1 -- 172.21.15.76:0/492504807 shutdown_connections 2023-11-09T03:47:30.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.287+0000 7f15b261b740 1 --2- 172.21.15.76:0/492504807 >> [v2:172.21.15.76:6833/3710140065,v1:172.21.15.76:6835/3710140065] conn(0x56541e65d060 0x56541e65d460 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.287+0000 7f15b261b740 1 --2- 172.21.15.76:0/492504807 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56541e69def0 0x56541e683cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.287+0000 7f1587fff640 0 client.6255 ms_handle_reset on v2:172.21.15.76:6833/3710140065 2023-11-09T03:47:30.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.287+0000 7f15b261b740 1 --2- 172.21.15.76:0/492504807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56541e66cc30 0x56541e69d9b0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.287+0000 7f15b261b740 1 --2- 172.21.15.76:0/492504807 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56541e66c850 0x56541e672030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.287+0000 7f15b261b740 1 -- 172.21.15.76:0/492504807 >> 172.21.15.76:0/492504807 conn(0x56541e63fc90 msgr2=0x56541e66f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:30.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.288+0000 7f15b261b740 1 -- 172.21.15.76:0/492504807 shutdown_connections 2023-11-09T03:47:30.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.288+0000 7f15b261b740 1 -- 172.21.15.76:0/492504807 wait complete. 2023-11-09T03:47:30.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.288+0000 7f15b261b740 20 client.6255 trim_cache size 0 max 16384 2023-11-09T03:47:30.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.303+0000 7f15b261b740 1 Processor -- start 2023-11-09T03:47:30.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.303+0000 7f15b261b740 1 -- start start 2023-11-09T03:47:30.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.304+0000 7f15b261b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56541e66c850 0x56541e66b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:30.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.304+0000 7f15b261b740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56541e66cc30 0x56541e66f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:30.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.304+0000 7f15b261b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56541e69def0 0x56541e66f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:30.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.304+0000 7f15b261b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541e52b280 con 0x56541e66c850 2023-11-09T03:47:30.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.304+0000 7f15b261b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541e69a080 con 0x56541e69def0 2023-11-09T03:47:30.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.304+0000 7f15b261b740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x56541e672cc0 con 0x56541e66cc30 2023-11-09T03:47:30.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.304+0000 7f15ad745640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56541e66c850 0x56541e66b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:30.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.304+0000 7f15ad745640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56541e66c850 0x56541e66b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42664/0 (socket says 172.21.15.76:42664) 2023-11-09T03:47:30.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.304+0000 7f15ad745640 1 -- 172.21.15.76:0/1856772374 learned_addr learned my addr 172.21.15.76:0/1856772374 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:30.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.304+0000 7f15ad745640 1 -- 172.21.15.76:0/1856772374 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56541e66cc30 msgr2=0x56541e66f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:30.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.304+0000 7f15ad745640 1 --2- 172.21.15.76:0/1856772374 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56541e66cc30 0x56541e66f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.304+0000 7f15ad745640 1 -- 172.21.15.76:0/1856772374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56541e69def0 msgr2=0x56541e66f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:30.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.304+0000 7f15ad745640 1 --2- 172.21.15.76:0/1856772374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56541e69def0 0x56541e66f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.304+0000 7f15ad745640 1 -- 172.21.15.76:0/1856772374 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56541e547d10 con 0x56541e66c850 2023-11-09T03:47:30.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.304+0000 7f15ad745640 1 --2- 172.21.15.76:0/1856772374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56541e66c850 0x56541e66b110 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f15a4057e90 tx=0x7f15a4089030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b064a1847bdbb05 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:30.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.304+0000 7f15937fe640 1 -- 172.21.15.76:0/1856772374 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a4087040 con 0x56541e66c850 2023-11-09T03:47:30.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.304+0000 7f15937fe640 1 -- 172.21.15.76:0/1856772374 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a405c2e0 con 0x56541e66c850 2023-11-09T03:47:30.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.304+0000 7f15937fe640 1 -- 172.21.15.76:0/1856772374 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a40548a0 con 0x56541e66c850 2023-11-09T03:47:30.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.305+0000 7f15b261b740 1 -- 172.21.15.76:0/1856772374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56541e66c850 msgr2=0x56541e66b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:30.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.305+0000 7f15b261b740 1 --2- 172.21.15.76:0/1856772374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56541e66c850 0x56541e66b110 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f15a4057e90 tx=0x7f15a4089030 comp rx=0 tx=0).stop 2023-11-09T03:47:30.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.305+0000 7f15b261b740 1 -- 172.21.15.76:0/1856772374 shutdown_connections 2023-11-09T03:47:30.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.305+0000 7f15b261b740 1 --2- 172.21.15.76:0/1856772374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56541e69def0 0x56541e66f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.305+0000 7f15b261b740 1 --2- 172.21.15.76:0/1856772374 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56541e66cc30 0x56541e66f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.305+0000 7f15b261b740 1 --2- 172.21.15.76:0/1856772374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56541e66c850 0x56541e66b110 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.305+0000 7f15b261b740 1 -- 172.21.15.76:0/1856772374 >> 172.21.15.76:0/1856772374 conn(0x56541e63fc90 msgr2=0x56541e68e480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:30.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.305+0000 7f15b261b740 1 -- 172.21.15.76:0/1856772374 shutdown_connections 2023-11-09T03:47:30.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.305+0000 7f15b261b740 1 -- 172.21.15.76:0/1856772374 wait complete. 2023-11-09T03:47:30.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.305+0000 7f15b261b740 1 -- start start 2023-11-09T03:47:30.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.305+0000 7f15b261b740 1 Processor -- start 2023-11-09T03:47:30.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.306+0000 7f15b261b740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56541e66c850 0x56541e66f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:30.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.306+0000 7f15b261b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56541e66cc30 0x56541e6fa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:30.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.306+0000 7f15b261b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56541e69def0 0x56541e6fa900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:30.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.306+0000 7f15b261b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541e700f50 con 0x56541e69def0 2023-11-09T03:47:30.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.306+0000 7f15b261b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541e69d050 con 0x56541e66cc30 2023-11-09T03:47:30.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.306+0000 7f15b261b740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x56541e69d290 con 0x56541e66c850 2023-11-09T03:47:30.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.306+0000 7f15b261b740 10 client.0 _ll_register_callbacks cb 0x7f15a4057f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:47:30.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.306+0000 7f15adf46640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56541e69def0 0x56541e6fa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:30.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.306+0000 7f15adf46640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56541e69def0 0x56541e6fa900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42668/0 (socket says 172.21.15.76:42668) 2023-11-09T03:47:30.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.306+0000 7f15adf46640 1 -- 172.21.15.76:0/2932334818 learned_addr learned my addr 172.21.15.76:0/2932334818 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:30.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.307+0000 7f15ad745640 1 --2- 172.21.15.76:0/2932334818 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56541e66c850 0x56541e66f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:30.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.307+0000 7f15acf44640 1 --2- 172.21.15.76:0/2932334818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56541e66cc30 0x56541e6fa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:30.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.307+0000 7f15adf46640 1 -- 172.21.15.76:0/2932334818 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56541e66c850 msgr2=0x56541e66f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:30.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.307+0000 7f15adf46640 1 --2- 172.21.15.76:0/2932334818 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56541e66c850 0x56541e66f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.307+0000 7f15adf46640 1 -- 172.21.15.76:0/2932334818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56541e66cc30 msgr2=0x56541e6fa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:30.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.307+0000 7f15adf46640 1 --2- 172.21.15.76:0/2932334818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56541e66cc30 0x56541e6fa3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.307+0000 7f15adf46640 1 -- 172.21.15.76:0/2932334818 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56541e56eca0 con 0x56541e69def0 2023-11-09T03:47:30.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.307+0000 7f15adf46640 1 --2- 172.21.15.76:0/2932334818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56541e69def0 0x56541e6fa900 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f15a00104a0 tx=0x7f15a0010960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13cabbe0f6bec7fe server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:30.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.307+0000 7f1587fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:47:30.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.307+0000 7f1587fff640 1 -- 172.21.15.76:0/2932334818 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a0003d80 con 0x56541e69def0 2023-11-09T03:47:30.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.307+0000 7f1587fff640 1 -- 172.21.15.76:0/2932334818 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f15a000c050 con 0x56541e69def0 2023-11-09T03:47:30.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.307+0000 7f15b261b740 1 -- 172.21.15.76:0/2932334818 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f15a4060490 con 0x56541e69def0 2023-11-09T03:47:30.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.307+0000 7f1587fff640 1 -- 172.21.15.76:0/2932334818 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15a00050d0 con 0x56541e69def0 2023-11-09T03:47:30.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.307+0000 7f15b261b740 10 client.6261 fetch_fsmap learned FSMap version 23 2023-11-09T03:47:30.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.307+0000 7f15b261b740 1 -- 172.21.15.76:0/2932334818 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x56541e56d500 con 0x56541e69def0 2023-11-09T03:47:30.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.307+0000 7f1587fff640 1 -- 172.21.15.76:0/2932334818 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a4060490 con 0x56541e69def0 2023-11-09T03:47:30.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.307+0000 7f1587fff640 1 -- 172.21.15.76:0/2932334818 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f15a0012040 con 0x56541e69def0 2023-11-09T03:47:30.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.307+0000 7f15b261b740 10 client.6261 fetch_fsmap finished waiting for FSMap version 23 2023-11-09T03:47:30.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.307+0000 7f15b261b740 10 client.6261 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-09T03:47:30.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.307+0000 7f15b261b740 20 client.6261 populate_metadata read hostname 'smithi076' 2023-11-09T03:47:30.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.307+0000 7f15b261b740 1 --2- 172.21.15.76:0/2932334818 >> [v2:172.21.15.76:6833/3710140065,v1:172.21.15.76:6835/3710140065] conn(0x56541e644380 0x56541e644760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:30.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.307+0000 7f15b261b740 4 client.6261 mds_command: new command op to 6075 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-09T03:47:30.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.307+0000 7f15b261b740 1 -- 172.21.15.76:0/2932334818 --> [v2:172.21.15.76:6833/3710140065,v1:172.21.15.76:6835/3710140065] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56541e5a45b0 con 0x56541e644380 2023-11-09T03:47:30.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.308+0000 7f15ad745640 1 --2- 172.21.15.76:0/2932334818 >> [v2:172.21.15.76:6833/3710140065,v1:172.21.15.76:6835/3710140065] conn(0x56541e644380 0x56541e644760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:30.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.308+0000 7f1587fff640 10 client.6261 ms_handle_connect on v2:172.21.15.76:6833/3710140065 2023-11-09T03:47:30.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.308+0000 7f15ad745640 1 --2- 172.21.15.76:0/2932334818 >> [v2:172.21.15.76:6833/3710140065,v1:172.21.15.76:6835/3710140065] conn(0x56541e644380 0x56541e644760 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4396d9de3b18e019 server_cookie=dc4783f0c533e91b in_seq=0 out_seq=0 2023-11-09T03:47:30.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.308+0000 7f1587fff640 1 -- 172.21.15.76:0/2932334818 <== mds.0 v2:172.21.15.76:6833/3710140065 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x56541e5a45b0 con 0x56541e644380 2023-11-09T03:47:30.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.308+0000 7f1587fff640 10 client.6261 handle_command_reply: tid=0 2023-11-09T03:47:30.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.308+0000 7f15b261b740 1 client.6261 shutdown 2023-11-09T03:47:30.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.308+0000 7f15b261b740 1 -- 172.21.15.76:0/2932334818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56541e69def0 msgr2=0x56541e6fa900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:30.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.308+0000 7f15b261b740 1 --2- 172.21.15.76:0/2932334818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56541e69def0 0x56541e6fa900 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f15a00104a0 tx=0x7f15a0010960 comp rx=0 tx=0).stop 2023-11-09T03:47:30.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.308+0000 7f15b261b740 1 -- 172.21.15.76:0/2932334818 shutdown_connections 2023-11-09T03:47:30.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.308+0000 7f15b261b740 1 --2- 172.21.15.76:0/2932334818 >> [v2:172.21.15.76:6833/3710140065,v1:172.21.15.76:6835/3710140065] conn(0x56541e644380 0x56541e644760 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.308+0000 7f15b261b740 1 --2- 172.21.15.76:0/2932334818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56541e69def0 0x56541e6fa900 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.308+0000 7f15b261b740 1 --2- 172.21.15.76:0/2932334818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56541e66cc30 0x56541e6fa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.308+0000 7f15b261b740 1 --2- 172.21.15.76:0/2932334818 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56541e66c850 0x56541e66f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.308+0000 7f1587fff640 0 client.6261 ms_handle_reset on v2:172.21.15.76:6833/3710140065 2023-11-09T03:47:30.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.308+0000 7f15b261b740 1 -- 172.21.15.76:0/2932334818 >> 172.21.15.76:0/2932334818 conn(0x56541e63fc90 msgr2=0x56541e68e410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:30.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.308+0000 7f15b261b740 1 -- 172.21.15.76:0/2932334818 shutdown_connections 2023-11-09T03:47:30.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.308+0000 7f15b261b740 1 -- 172.21.15.76:0/2932334818 wait complete. 2023-11-09T03:47:30.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.309+0000 7f15b261b740 20 client.6261 trim_cache size 0 max 16384 2023-11-09T03:47:30.330 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:47:30.331 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "c213b26f-d0da-45d5-8e34-bb75c096d4f2", 2023-11-09T03:47:30.331 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2023-11-09T03:47:30.331 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 6075, 2023-11-09T03:47:30.331 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2023-11-09T03:47:30.331 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2023-11-09T03:47:30.331 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2023-11-09T03:47:30.332 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 9.3719270170000009, 2023-11-09T03:47:30.332 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 23, 2023-11-09T03:47:30.332 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 54, 2023-11-09T03:47:30.332 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 54, 2023-11-09T03:47:30.332 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 14.446261746999999 2023-11-09T03:47:30.333 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:47:30.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.309+0000 7f15af9a9640 1 -- 172.21.15.76:0/415330215 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f158c06ae70 msgr2=0x7f158c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:30.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.309+0000 7f15af9a9640 1 --2- 172.21.15.76:0/415330215 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f158c06ae70 0x7f158c06d330 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f159c036000 tx=0x7f159c008030 comp rx=0 tx=0).stop 2023-11-09T03:47:30.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.309+0000 7f15af9a9640 1 -- 172.21.15.76:0/415330215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15a8070e20 msgr2=0x7f15a80cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.309+0000 7f15af9a9640 1 --2- 172.21.15.76:0/415330215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15a8070e20 0x7f15a80cd5d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f15a400c3f0 tx=0x7f15a400c8b0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.309+0000 7f15af9a9640 1 -- 172.21.15.76:0/415330215 shutdown_connections 2023-11-09T03:47:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.309+0000 7f15af9a9640 1 --2- 172.21.15.76:0/415330215 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f158c06ae70 0x7f158c06d330 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.309+0000 7f15af9a9640 1 --2- 172.21.15.76:0/415330215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15a815ad30 0x7f15a8159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.309+0000 7f15af9a9640 1 --2- 172.21.15.76:0/415330215 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15a80ad730 0x7f15a8146380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.309+0000 7f15af9a9640 1 --2- 172.21.15.76:0/415330215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15a8070e20 0x7f15a80cd5d0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.309+0000 7f15af9a9640 1 -- 172.21.15.76:0/415330215 >> 172.21.15.76:0/415330215 conn(0x7f15a80ce950 msgr2=0x7f15a8072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.309+0000 7f15af9a9640 1 -- 172.21.15.76:0/415330215 shutdown_connections 2023-11-09T03:47:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.309+0000 7f15af9a9640 1 -- 172.21.15.76:0/415330215 wait complete. 2023-11-09T03:47:30.336 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-09T03:47:30.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.438+0000 7f0425a56640 1 Processor -- start 2023-11-09T03:47:30.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.438+0000 7f0425a56640 1 -- start start 2023-11-09T03:47:30.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.439+0000 7f0425a56640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f04200af060 0x7f04200b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:30.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.439+0000 7f0425a56640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f042006ef40 0x7f04200cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:30.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.439+0000 7f0425a56640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04200b1590 con 0x7f04200aec80 2023-11-09T03:47:30.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.439+0000 7f0425a56640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0420155a40 con 0x7f04200af060 2023-11-09T03:47:30.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.439+0000 7f0425a56640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f042006a8e0 con 0x7f042006ef40 2023-11-09T03:47:30.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.439+0000 7f041effd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f04200aec80 0x7f042006a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48056/0 (socket says 172.21.15.76:48056) 2023-11-09T03:47:30.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.439+0000 7f041effd640 1 -- 172.21.15.76:0/3737320538 learned_addr learned my addr 172.21.15.76:0/3737320538 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:30.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.439+0000 7f0417fff640 1 --2- 172.21.15.76:0/3737320538 >> v2:172.21.15.113:3300/0 conn(0x7f04200af060 0x7f04200b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:30.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.439+0000 7f041e7fc640 1 -- 172.21.15.76:0/3737320538 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3132723416 0 0) 0x7f04200b1590 con 0x7f04200aec80 2023-11-09T03:47:30.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.439+0000 7f041e7fc640 1 -- 172.21.15.76:0/3737320538 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f040c003520 con 0x7f04200aec80 2023-11-09T03:47:30.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.439+0000 7f041f7fe640 1 --2- 172.21.15.76:0/3737320538 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f042006ef40 0x7f04200cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:30.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.439+0000 7f041e7fc640 1 -- 172.21.15.76:0/3737320538 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3922062604 0 0) 0x7f040c003520 con 0x7f04200aec80 2023-11-09T03:47:30.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.439+0000 7f041e7fc640 1 -- 172.21.15.76:0/3737320538 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f042006ef40 msgr2=0x7f04200cb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:30.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.439+0000 7f041e7fc640 1 --2- 172.21.15.76:0/3737320538 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f042006ef40 0x7f04200cb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.439+0000 7f041e7fc640 1 -- 172.21.15.76:0/3737320538 >> v2:172.21.15.113:3300/0 conn(0x7f04200af060 msgr2=0x7f04200b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:30.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.439+0000 7f041e7fc640 1 --2- 172.21.15.76:0/3737320538 >> v2:172.21.15.113:3300/0 conn(0x7f04200af060 0x7f04200b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.439+0000 7f041e7fc640 1 -- 172.21.15.76:0/3737320538 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04200b1800 con 0x7f04200aec80 2023-11-09T03:47:30.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.439+0000 7f041e7fc640 1 -- 172.21.15.76:0/3737320538 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f04100017f0 con 0x7f04200aec80 2023-11-09T03:47:30.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.439+0000 7f041f7fe640 1 --2- 172.21.15.76:0/3737320538 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f042006ef40 0x7f04200cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:30.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.439+0000 7f041e7fc640 1 -- 172.21.15.76:0/3737320538 >> v1:172.21.15.76:6789/0 conn(0x7f04200aec80 legacy=0x7f042006a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:30.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.439+0000 7f041e7fc640 1 --2- 172.21.15.76:0/3737320538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f040c003970 0x7f040c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:30.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.439+0000 7f041e7fc640 1 --2- 172.21.15.76:0/3737320538 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f040c0047c0 0x7f040c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:30.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.439+0000 7f041e7fc640 1 --2- 172.21.15.76:0/3737320538 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f042006ef40 0x7f040c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:30.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.439+0000 7f041e7fc640 1 -- 172.21.15.76:0/3737320538 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0420155a40 con 0x7f040c003970 2023-11-09T03:47:30.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.439+0000 7f041e7fc640 1 -- 172.21.15.76:0/3737320538 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042006a8e0 con 0x7f040c0047c0 2023-11-09T03:47:30.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.439+0000 7f041e7fc640 1 -- 172.21.15.76:0/3737320538 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f040c00b980 con 0x7f042006ef40 2023-11-09T03:47:30.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.440+0000 7f041effd640 1 --2- 172.21.15.76:0/3737320538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f040c003970 0x7f040c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:30.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.440+0000 7f041f7fe640 1 --2- 172.21.15.76:0/3737320538 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f042006ef40 0x7f040c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:30.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.440+0000 7f0417fff640 1 --2- 172.21.15.76:0/3737320538 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f040c0047c0 0x7f040c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:30.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.440+0000 7f041effd640 1 -- 172.21.15.76:0/3737320538 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f042006ef40 msgr2=0x7f040c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:30.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.440+0000 7f041effd640 1 --2- 172.21.15.76:0/3737320538 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f042006ef40 0x7f040c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.440+0000 7f041effd640 1 -- 172.21.15.76:0/3737320538 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f040c0047c0 msgr2=0x7f040c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:30.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.440+0000 7f041effd640 1 --2- 172.21.15.76:0/3737320538 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f040c0047c0 0x7f040c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.440+0000 7f041effd640 1 -- 172.21.15.76:0/3737320538 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040c003520 con 0x7f040c003970 2023-11-09T03:47:30.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.441+0000 7f041effd640 1 --2- 172.21.15.76:0/3737320538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f040c003970 0x7f040c003d90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f041000c1c0 tx=0x7f040c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcba9ea98d26b084 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:30.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.441+0000 7f041e7fc640 1 -- 172.21.15.76:0/3737320538 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0410007270 con 0x7f040c003970 2023-11-09T03:47:30.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.441+0000 7f041e7fc640 1 -- 172.21.15.76:0/3737320538 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04100073f0 con 0x7f040c003970 2023-11-09T03:47:30.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.441+0000 7f041e7fc640 1 -- 172.21.15.76:0/3737320538 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041002c8b0 con 0x7f040c003970 2023-11-09T03:47:30.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.441+0000 7f0425a56640 1 -- 172.21.15.76:0/3737320538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f040c003970 msgr2=0x7f040c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:30.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.441+0000 7f0425a56640 1 --2- 172.21.15.76:0/3737320538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f040c003970 0x7f040c003d90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f041000c1c0 tx=0x7f040c0057f0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.441+0000 7f041f7fe640 1 -- 172.21.15.76:0/3737320538 reap_dead start 2023-11-09T03:47:30.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.441+0000 7f0425a56640 1 -- 172.21.15.76:0/3737320538 shutdown_connections 2023-11-09T03:47:30.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.441+0000 7f0425a56640 1 -- 172.21.15.76:0/3737320538 >> 172.21.15.76:0/3737320538 conn(0x7f0420098d40 msgr2=0x7f04200cfa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:30.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.441+0000 7f0425a56640 1 -- 172.21.15.76:0/3737320538 shutdown_connections 2023-11-09T03:47:30.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.441+0000 7f0425a56640 1 -- 172.21.15.76:0/3737320538 wait complete. 2023-11-09T03:47:30.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.442+0000 7f0425a56640 1 Processor -- start 2023-11-09T03:47:30.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.443+0000 7f0425a56640 1 -- start start 2023-11-09T03:47:30.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.443+0000 7f0425a56640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f040c003970 0x7f0420153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:30.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.443+0000 7f0425a56640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04200ceda0 0x7f04200cf220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:30.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.443+0000 7f0425a56640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04200b36e0 0x7f04200b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:30.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.443+0000 7f0425a56640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04200b40a0 con 0x7f04200b36e0 2023-11-09T03:47:30.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.443+0000 7f0425a56640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042009bbd0 con 0x7f040c003970 2023-11-09T03:47:30.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.443+0000 7f0425a56640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04200cd990 con 0x7f04200ceda0 2023-11-09T03:47:30.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.443+0000 7f041f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04200b36e0 0x7f04200b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:30.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.443+0000 7f041f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04200b36e0 0x7f04200b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42692/0 (socket says 172.21.15.76:42692) 2023-11-09T03:47:30.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.443+0000 7f041f7fe640 1 -- 172.21.15.76:0/1991844975 learned_addr learned my addr 172.21.15.76:0/1991844975 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:30.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.443+0000 7f041effd640 1 --2- 172.21.15.76:0/1991844975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f040c003970 0x7f0420153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:30.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.443+0000 7f0417fff640 1 --2- 172.21.15.76:0/1991844975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04200ceda0 0x7f04200cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:30.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.444+0000 7f041f7fe640 1 -- 172.21.15.76:0/1991844975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04200ceda0 msgr2=0x7f04200cf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:30.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.444+0000 7f041f7fe640 1 --2- 172.21.15.76:0/1991844975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04200ceda0 0x7f04200cf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.444+0000 7f041f7fe640 1 -- 172.21.15.76:0/1991844975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f040c003970 msgr2=0x7f0420153ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:30.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.444+0000 7f041f7fe640 1 --2- 172.21.15.76:0/1991844975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f040c003970 0x7f0420153ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.444+0000 7f041f7fe640 1 -- 172.21.15.76:0/1991844975 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04200cdfe0 con 0x7f04200b36e0 2023-11-09T03:47:30.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.444+0000 7f041f7fe640 1 --2- 172.21.15.76:0/1991844975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04200b36e0 0x7f04200b3b60 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f042015fb20 tx=0x7f0418004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:30.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.444+0000 7f041cff9640 1 -- 172.21.15.76:0/1991844975 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0418007cf0 con 0x7f04200b36e0 2023-11-09T03:47:30.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.444+0000 7f041cff9640 1 -- 172.21.15.76:0/1991844975 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041800f050 con 0x7f04200b36e0 2023-11-09T03:47:30.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.444+0000 7f0425a56640 1 -- 172.21.15.76:0/1991844975 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04200ce2c0 con 0x7f04200b36e0 2023-11-09T03:47:30.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.444+0000 7f041cff9640 1 -- 172.21.15.76:0/1991844975 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0418014650 con 0x7f04200b36e0 2023-11-09T03:47:30.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.444+0000 7f0425a56640 1 -- 172.21.15.76:0/1991844975 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04200ce880 con 0x7f04200b36e0 2023-11-09T03:47:30.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.445+0000 7f0425a56640 1 -- 172.21.15.76:0/1991844975 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0420074290 con 0x7f04200b36e0 2023-11-09T03:47:30.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.445+0000 7f041cff9640 1 -- 172.21.15.76:0/1991844975 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f04180147b0 con 0x7f04200b36e0 2023-11-09T03:47:30.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.446+0000 7f041cff9640 1 --2- 172.21.15.76:0/1991844975 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f040006ae70 0x7f040006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:30.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.446+0000 7f041cff9640 1 -- 172.21.15.76:0/1991844975 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f0418089dc0 con 0x7f04200b36e0 2023-11-09T03:47:30.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.446+0000 7f041effd640 1 --2- 172.21.15.76:0/1991844975 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f040006ae70 0x7f040006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:30.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.446+0000 7f041effd640 1 --2- 172.21.15.76:0/1991844975 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f040006ae70 0x7f040006d330 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0410034000 tx=0x7f0410007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:30.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.450+0000 7f041cff9640 1 -- 172.21.15.76:0/1991844975 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0418054bd0 con 0x7f04200b36e0 2023-11-09T03:47:30.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.650+0000 7f0425a56640 1 -- 172.21.15.76:0/1991844975 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f04200cea40 con 0x7f04200b36e0 2023-11-09T03:47:30.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.650+0000 7f041cff9640 1 -- 172.21.15.76:0/1991844975 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7f04180581f0 con 0x7f04200b36e0 2023-11-09T03:47:30.650 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-09T03:47:30.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.652+0000 7f0425a56640 1 -- 172.21.15.76:0/1991844975 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f040006ae70 msgr2=0x7f040006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:30.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.652+0000 7f0425a56640 1 --2- 172.21.15.76:0/1991844975 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f040006ae70 0x7f040006d330 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0410034000 tx=0x7f0410007000 comp rx=0 tx=0).stop 2023-11-09T03:47:30.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.652+0000 7f0425a56640 1 -- 172.21.15.76:0/1991844975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04200b36e0 msgr2=0x7f04200b3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:30.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.652+0000 7f0425a56640 1 --2- 172.21.15.76:0/1991844975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04200b36e0 0x7f04200b3b60 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f042015fb20 tx=0x7f0418004700 comp rx=0 tx=0).stop 2023-11-09T03:47:30.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.652+0000 7f0425a56640 1 -- 172.21.15.76:0/1991844975 shutdown_connections 2023-11-09T03:47:30.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.652+0000 7f0425a56640 1 --2- 172.21.15.76:0/1991844975 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f040006ae70 0x7f040006d330 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.652+0000 7f0425a56640 1 --2- 172.21.15.76:0/1991844975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04200b36e0 0x7f04200b3b60 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.652+0000 7f0425a56640 1 --2- 172.21.15.76:0/1991844975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04200ceda0 0x7f04200cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.652+0000 7f0425a56640 1 --2- 172.21.15.76:0/1991844975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f040c003970 0x7f0420153ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:30.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.652+0000 7f0425a56640 1 -- 172.21.15.76:0/1991844975 >> 172.21.15.76:0/1991844975 conn(0x7f0420098d40 msgr2=0x7f04200b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:30.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.652+0000 7f0425a56640 1 -- 172.21.15.76:0/1991844975 shutdown_connections 2023-11-09T03:47:30.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:30.652+0000 7f0425a56640 1 -- 172.21.15.76:0/1991844975 wait complete. 2023-11-09T03:47:30.662 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-09T03:47:30.662 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-09T03:47:30.663 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2023-11-09T03:47:30.663 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-09T03:47:30.663 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-09T03:47:30.663 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-09T03:47:30.664 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-09T03:47:30.664 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-09T03:47:30.664 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:47:30.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:47:30.665 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2023-11-09T03:47:30.680 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-09T03:47:30.680 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-09T03:47:30.681 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2023-11-09T03:47:30.681 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:47:30.681 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2023-11-09T03:47:30.681 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:47:30.681 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:47:30.682 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:47:30.682 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2023-11-09T03:47:30.682 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:47:30.682 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:47:30.682 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2023-11-09T03:47:30.683 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:47:30.683 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:47:30.683 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-09T03:47:30.683 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:47:30.683 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-09T03:47:30.684 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42498sec preferred_lft 42498sec 2023-11-09T03:47:30.684 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2023-11-09T03:47:30.684 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:47:30.684 INFO:teuthology.orchestra.run.smithi076.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-09T03:47:30.684 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2e:15:bd:03:a8:2e brd ff:ff:ff:ff:ff:ff 2023-11-09T03:47:30.684 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-09T03:47:30.685 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:47:30.685 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::bcda:38ff:fed4:fdb0/64 scope link 2023-11-09T03:47:30.685 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:47:30.685 INFO:teuthology.orchestra.run.smithi076.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-09T03:47:30.685 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 6a:8d:94:d7:e5:00 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:47:30.686 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::688d:94ff:fed7:e500/64 scope link 2023-11-09T03:47:30.686 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:47:30.686 INFO:teuthology.orchestra.run.smithi076.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-09T03:47:30.686 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2e:15:bd:03:a8:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:47:30.686 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::2c15:bdff:fe03:a82e/64 scope link 2023-11-09T03:47:30.686 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:47:30.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:47:30.687 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-09T03:47:30.747 INFO:teuthology.orchestra.run.smithi076.stdout:1 2023-11-09T03:47:30.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:47:30.748 DEBUG:teuthology.orchestra.run.smithi076:> route 2023-11-09T03:47:30.794 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:47:30.803 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2023-11-09T03:47:30.804 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-09T03:47:30.804 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-09T03:47:30.804 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-09T03:47:30.804 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-09T03:47:30.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:47:30.805 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-09T03:47:30.805 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-09T03:47:30.806 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-09T03:47:30.806 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-09T03:47:30.806 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-09T03:47:30.807 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-09T03:47:30.846 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:47:30.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:47:30.900 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2023-11-09T03:47:30.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-09T03:47:30.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-09T03:47:30.955 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:47:30.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:47:30.955 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2023-11-09T03:47:31.021 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-09T03:47:31.021 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:47:31.021 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-09T03:47:31.076 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:47:31.076 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:47:31.076 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-09T03:47:31.130 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:47:31.185 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:47:31.185 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:47:31.186 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:47:31.186 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:47:31.186 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:47:31.186 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:47:31.187 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:47:31.187 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:47:31.187 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:47:31.187 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:47:31.187 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:47:31.187 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:47:31.188 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:47:31.188 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:47:31.188 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:47:31.188 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:47:31.188 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:47:31.188 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:47:31.189 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:47:31.189 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:47:31.189 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:47:31.189 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:47:31.189 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:47:31.190 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:47:31.190 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:47:31.190 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:47:31.190 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:47:31.190 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:47:31.190 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:47:31.191 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:47:31.191 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:47:31.191 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:47:31.191 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:47:31.191 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:47:31.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:47:31.192 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:47:31.240 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-09T03:47:31.241 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-09T03:47:31.283 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:47:31.298 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:47:31.299 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:47:31.299 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:47:31.299 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:47:31.299 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:47:31.299 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:47:31.300 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:47:31.300 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:47:31.300 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:47:31.300 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:47:31.300 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:47:31.301 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:47:31.301 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:47:31.301 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:47:31.301 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:47:31.301 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:47:31.301 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:47:31.302 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:47:31.302 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:47:31.302 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:47:31.302 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:47:31.302 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:47:31.303 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:47:31.303 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:47:31.303 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:47:31.303 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:47:31.303 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:47:31.304 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:47:31.304 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:47:31.304 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:47:31.304 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:47:31.304 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:47:31.305 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:47:31.305 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:47:31.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:47:31.305 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:47:31.375 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:47:31.376+0000 7f0ac8a39480 -1 init, newargv = 0x55d489287f30 newargc=15 2023-11-09T03:47:31.375 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[107936]: starting ceph client 2023-11-09T03:47:31.379 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[107936]: starting fuse 2023-11-09T03:47:32.455 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:47:32.473 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:47:32.473 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:47:32.473 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:47:32.474 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:47:32.474 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:47:32.474 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:47:32.474 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:47:32.474 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:47:32.475 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:47:32.475 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:47:32.475 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:47:32.475 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:47:32.475 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:47:32.475 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:47:32.476 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:47:32.476 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:47:32.476 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:47:32.476 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:47:32.476 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:47:32.477 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:47:32.477 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:47:32.477 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:47:32.477 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:47:32.477 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:47:32.477 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:47:32.478 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:47:32.478 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:47:32.478 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:47:32.478 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:47:32.478 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:47:32.479 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:47:32.479 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:47:32.479 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:47:32.479 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:47:32.479 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-09T03:47:32.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:47:32.481 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:47:32.537 INFO:teuthology.orchestra.run.smithi076.stdout:43 2023-11-09T03:47:32.537 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-09T03:47:32.538 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer -- python3 -c ' 2023-11-09T03:47:32.538 DEBUG:teuthology.orchestra.run.smithi076:> import glob 2023-11-09T03:47:32.539 DEBUG:teuthology.orchestra.run.smithi076:> import re 2023-11-09T03:47:32.539 DEBUG:teuthology.orchestra.run.smithi076:> import os 2023-11-09T03:47:32.539 DEBUG:teuthology.orchestra.run.smithi076:> import subprocess 2023-11-09T03:47:32.540 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:47:32.540 DEBUG:teuthology.orchestra.run.smithi076:> def _find_admin_socket(client_name): 2023-11-09T03:47:32.540 DEBUG:teuthology.orchestra.run.smithi076:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-09T03:47:32.541 DEBUG:teuthology.orchestra.run.smithi076:> files = glob.glob(asok_path) 2023-11-09T03:47:32.541 DEBUG:teuthology.orchestra.run.smithi076:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-09T03:47:32.541 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:47:32.542 DEBUG:teuthology.orchestra.run.smithi076:> # Given a non-glob path, it better be there 2023-11-09T03:47:32.542 DEBUG:teuthology.orchestra.run.smithi076:> if "*" not in asok_path: 2023-11-09T03:47:32.542 DEBUG:teuthology.orchestra.run.smithi076:> assert(len(files) == 1) 2023-11-09T03:47:32.543 DEBUG:teuthology.orchestra.run.smithi076:> return files[0] 2023-11-09T03:47:32.543 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:47:32.544 DEBUG:teuthology.orchestra.run.smithi076:> for f in files: 2023-11-09T03:47:32.544 DEBUG:teuthology.orchestra.run.smithi076:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-09T03:47:32.544 DEBUG:teuthology.orchestra.run.smithi076:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-09T03:47:32.545 DEBUG:teuthology.orchestra.run.smithi076:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-09T03:47:32.545 DEBUG:teuthology.orchestra.run.smithi076:> contents = proc_f.read() 2023-11-09T03:47:32.545 DEBUG:teuthology.orchestra.run.smithi076:> if mountpoint in contents: 2023-11-09T03:47:32.546 DEBUG:teuthology.orchestra.run.smithi076:> return f 2023-11-09T03:47:32.546 DEBUG:teuthology.orchestra.run.smithi076:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-09T03:47:32.546 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:47:32.547 DEBUG:teuthology.orchestra.run.smithi076:> print(_find_admin_socket("client.0")) 2023-11-09T03:47:32.547 DEBUG:teuthology.orchestra.run.smithi076:> ' 2023-11-09T03:47:32.661 INFO:teuthology.orchestra.run.smithi076.stdout:/var/run/ceph/ceph-client.0.107936.asok 2023-11-09T03:47:32.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:47:32.672 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.107936.asok 2023-11-09T03:47:32.672 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:47:32.672 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.107936.asok status 2023-11-09T03:47:32.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:47:32.840 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:47:32.840 INFO:teuthology.orchestra.run.smithi076.stdout: "metadata": { 2023-11-09T03:47:32.841 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_sha1": "7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4", 2023-11-09T03:47:32.841 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev)", 2023-11-09T03:47:32.841 INFO:teuthology.orchestra.run.smithi076.stdout: "entity_id": "0", 2023-11-09T03:47:32.841 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": "smithi076", 2023-11-09T03:47:32.841 INFO:teuthology.orchestra.run.smithi076.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-09T03:47:32.841 INFO:teuthology.orchestra.run.smithi076.stdout: "pid": "107936", 2023-11-09T03:47:32.842 INFO:teuthology.orchestra.run.smithi076.stdout: "root": "/" 2023-11-09T03:47:32.842 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:32.842 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_count": 0, 2023-11-09T03:47:32.842 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_pinned_count": 0, 2023-11-09T03:47:32.842 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 6273, 2023-11-09T03:47:32.843 INFO:teuthology.orchestra.run.smithi076.stdout: "inst": { 2023-11-09T03:47:32.843 INFO:teuthology.orchestra.run.smithi076.stdout: "name": { 2023-11-09T03:47:32.843 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "client", 2023-11-09T03:47:32.843 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 6273 2023-11-09T03:47:32.843 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:32.843 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2023-11-09T03:47:32.844 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2023-11-09T03:47:32.844 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2023-11-09T03:47:32.844 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 1554082326 2023-11-09T03:47:32.844 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:32.844 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:32.844 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2023-11-09T03:47:32.845 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2023-11-09T03:47:32.845 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2023-11-09T03:47:32.845 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 1554082326 2023-11-09T03:47:32.845 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:32.845 INFO:teuthology.orchestra.run.smithi076.stdout: "inst_str": "client.6273 192.168.0.1:0/1554082326", 2023-11-09T03:47:32.845 INFO:teuthology.orchestra.run.smithi076.stdout: "addr_str": "192.168.0.1:0/1554082326", 2023-11-09T03:47:32.846 INFO:teuthology.orchestra.run.smithi076.stdout: "inode_count": 1, 2023-11-09T03:47:32.846 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_epoch": 23, 2023-11-09T03:47:32.846 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch": 54, 2023-11-09T03:47:32.846 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch_barrier": 0, 2023-11-09T03:47:32.846 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklisted": false, 2023-11-09T03:47:32.847 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs" 2023-11-09T03:47:32.847 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:47:32.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:47:32.855 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:47:32.855 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-11-09T03:47:32.855 DEBUG:teuthology.orchestra.run.smithi076:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-09T03:47:32.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:47:32.915 INFO:teuthology.orchestra.run.smithi076.stdout:fuseblk 2023-11-09T03:47:32.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:47:32.921 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-09T03:47:32.922 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:47:32.922 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-09T03:47:32.981 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:47:32.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:47:32.988 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data mksnap snap1 2023-11-09T03:47:33.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.029+0000 7f6c6323af40 1 Processor -- start 2023-11-09T03:47:33.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.029+0000 7f6c6323af40 1 -- start start 2023-11-09T03:47:33.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.030+0000 7f6c6323af40 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555b35055c50 0x555b3509fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:33.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.030+0000 7f6c6323af40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555b35086780 0x555b3509f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:33.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.030+0000 7f6c6323af40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b35114d30 0x555b3509f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:33.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.030+0000 7f6c6323af40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b350063b0 con 0x555b35114d30 2023-11-09T03:47:33.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.030+0000 7f6c6323af40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b34fa0650 con 0x555b35086780 2023-11-09T03:47:33.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.030+0000 7f6c6323af40 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b34fa02f0 con 0x555b35055c50 2023-11-09T03:47:33.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.030+0000 7f6c60de3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b35114d30 0x555b3509f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:33.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.030+0000 7f6c60de3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b35114d30 0x555b3509f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48070/0 (socket says 172.21.15.76:48070) 2023-11-09T03:47:33.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.030+0000 7f6c60de3640 1 -- 172.21.15.76:0/2612382874 learned_addr learned my addr 172.21.15.76:0/2612382874 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:33.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.031+0000 7f6c5fde1640 1 --2- 172.21.15.76:0/2612382874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555b35086780 0x555b3509f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:33.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.031+0000 7f6c605e2640 1 --2- 172.21.15.76:0/2612382874 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555b35055c50 0x555b3509fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:33.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.031+0000 7f6c60de3640 1 -- 172.21.15.76:0/2612382874 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555b35055c50 msgr2=0x555b3509fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:33.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.031+0000 7f6c60de3640 1 --2- 172.21.15.76:0/2612382874 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555b35055c50 0x555b3509fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.031+0000 7f6c60de3640 1 -- 172.21.15.76:0/2612382874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555b35086780 msgr2=0x555b3509f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:33.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.031+0000 7f6c60de3640 1 --2- 172.21.15.76:0/2612382874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555b35086780 0x555b3509f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.031+0000 7f6c60de3640 1 -- 172.21.15.76:0/2612382874 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5000c390 con 0x555b35114d30 2023-11-09T03:47:33.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.031+0000 7f6c5fde1640 1 --2- 172.21.15.76:0/2612382874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555b35086780 0x555b3509f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:33.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.031+0000 7f6c605e2640 1 --2- 172.21.15.76:0/2612382874 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555b35055c50 0x555b3509fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:33.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.031+0000 7f6c60de3640 1 --2- 172.21.15.76:0/2612382874 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b35114d30 0x555b3509f550 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6c5001f950 tx=0x7f6c50050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dbe51ac9cd09a2d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:33.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.031+0000 7f6c5eddf640 1 -- 172.21.15.76:0/2612382874 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5004ebb0 con 0x555b35114d30 2023-11-09T03:47:33.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.031+0000 7f6c5eddf640 1 -- 172.21.15.76:0/2612382874 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5004ed10 con 0x555b35114d30 2023-11-09T03:47:33.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.031+0000 7f6c5eddf640 1 -- 172.21.15.76:0/2612382874 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5004f620 con 0x555b35114d30 2023-11-09T03:47:33.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.031+0000 7f6c6323af40 1 -- 172.21.15.76:0/2612382874 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b35114d30 msgr2=0x555b3509f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:33.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.031+0000 7f6c6323af40 1 --2- 172.21.15.76:0/2612382874 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b35114d30 0x555b3509f550 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6c5001f950 tx=0x7f6c50050000 comp rx=0 tx=0).stop 2023-11-09T03:47:33.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.031+0000 7f6c6323af40 1 -- 172.21.15.76:0/2612382874 shutdown_connections 2023-11-09T03:47:33.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.031+0000 7f6c6323af40 1 --2- 172.21.15.76:0/2612382874 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b35114d30 0x555b3509f550 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.031+0000 7f6c6323af40 1 --2- 172.21.15.76:0/2612382874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555b35086780 0x555b3509f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.031+0000 7f6c6323af40 1 --2- 172.21.15.76:0/2612382874 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555b35055c50 0x555b3509fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.031+0000 7f6c6323af40 1 -- 172.21.15.76:0/2612382874 >> 172.21.15.76:0/2612382874 conn(0x555b35082f40 msgr2=0x555b35084010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:33.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.031+0000 7f6c6323af40 1 -- 172.21.15.76:0/2612382874 shutdown_connections 2023-11-09T03:47:33.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.031+0000 7f6c6323af40 1 -- 172.21.15.76:0/2612382874 wait complete. 2023-11-09T03:47:33.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.032+0000 7f6c6323af40 1 Processor -- start 2023-11-09T03:47:33.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.032+0000 7f6c6323af40 1 -- start start 2023-11-09T03:47:33.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:47:33.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.033+0000 7f6c6323af40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b35055c50 0x555b35086cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:33.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.033+0000 7f6c6323af40 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555b35086780 0x555b351113c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:33.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.033+0000 7f6c6323af40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555b35114d30 0x555b35084010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:33.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.033+0000 7f6c6323af40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b34f804f0 con 0x555b35055c50 2023-11-09T03:47:33.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.033+0000 7f6c6323af40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b35113200 con 0x555b35114d30 2023-11-09T03:47:33.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.033+0000 7f6c6323af40 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b35087230 con 0x555b35086780 2023-11-09T03:47:33.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.033+0000 7f6c605e2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b35055c50 0x555b35086cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:33.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.033+0000 7f6c605e2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b35055c50 0x555b35086cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48074/0 (socket says 172.21.15.76:48074) 2023-11-09T03:47:33.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.033+0000 7f6c605e2640 1 -- 172.21.15.76:0/3638580860 learned_addr learned my addr 172.21.15.76:0/3638580860 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:33.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.033+0000 7f6c5fde1640 1 --2- 172.21.15.76:0/3638580860 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555b35086780 0x555b351113c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:33.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.033+0000 7f6c60de3640 1 --2- 172.21.15.76:0/3638580860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555b35114d30 0x555b35084010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:33.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.033+0000 7f6c605e2640 1 -- 172.21.15.76:0/3638580860 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555b35086780 msgr2=0x555b351113c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:33.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.033+0000 7f6c605e2640 1 --2- 172.21.15.76:0/3638580860 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555b35086780 0x555b351113c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.033+0000 7f6c605e2640 1 -- 172.21.15.76:0/3638580860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555b35114d30 msgr2=0x555b35084010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:33.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.033+0000 7f6c605e2640 1 --2- 172.21.15.76:0/3638580860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555b35114d30 0x555b35084010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.033+0000 7f6c605e2640 1 -- 172.21.15.76:0/3638580860 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b34fc6770 con 0x555b35055c50 2023-11-09T03:47:33.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.033+0000 7f6c5fde1640 1 --2- 172.21.15.76:0/3638580860 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555b35086780 0x555b351113c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:33.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.033+0000 7f6c60de3640 1 --2- 172.21.15.76:0/3638580860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555b35114d30 0x555b35084010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:33.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.033+0000 7f6c605e2640 1 --2- 172.21.15.76:0/3638580860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b35055c50 0x555b35086cf0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800cef0 tx=0x7f6c5800a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:33.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.033+0000 7f6c5dddd640 1 -- 172.21.15.76:0/3638580860 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5800ae00 con 0x555b35055c50 2023-11-09T03:47:33.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.033+0000 7f6c5dddd640 1 -- 172.21.15.76:0/3638580860 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5800f400 con 0x555b35055c50 2023-11-09T03:47:33.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.033+0000 7f6c6323af40 1 -- 172.21.15.76:0/3638580860 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555b34fac780 con 0x555b35055c50 2023-11-09T03:47:33.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.033+0000 7f6c5dddd640 1 -- 172.21.15.76:0/3638580860 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5800f770 con 0x555b35055c50 2023-11-09T03:47:33.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.034+0000 7f6c6323af40 1 -- 172.21.15.76:0/3638580860 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555b34f9f930 con 0x555b35055c50 2023-11-09T03:47:33.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.035+0000 7f6c5dddd640 1 -- 172.21.15.76:0/3638580860 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6c5800d040 con 0x555b35055c50 2023-11-09T03:47:33.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.035+0000 7f6c5dddd640 1 --2- 172.21.15.76:0/3638580860 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6c4c069850 0x7f6c4c06bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:33.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.035+0000 7f6c5dddd640 1 -- 172.21.15.76:0/3638580860 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6c58088010 con 0x555b35055c50 2023-11-09T03:47:33.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.035+0000 7f6c6323af40 1 -- 172.21.15.76:0/3638580860 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- pool_op(create snap pool 9 tid 1 name snap1 v0) v4 -- 0x555b35010330 con 0x555b35055c50 2023-11-09T03:47:33.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.035+0000 7f6c5fde1640 1 --2- 172.21.15.76:0/3638580860 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6c4c069850 0x7f6c4c06bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:33.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.035+0000 7f6c5dddd640 1 -- 172.21.15.76:0/3638580860 <== mon.0 v2:172.21.15.76:3300/0 6 ==== pool_op_reply(tid 1 (95) Operation not supported v54) v1 ==== 43+0+0 (secure 0 0 0) 0x7f6c58013020 con 0x555b35055c50 2023-11-09T03:47:33.046 INFO:teuthology.orchestra.run.smithi076.stderr:error creating pool cephfs_data snapshot snap1: (95) Operation not supported 2023-11-09T03:47:33.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.036+0000 7f6c6323af40 1 -- 172.21.15.76:0/3638580860 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6c4c069850 msgr2=0x7f6c4c06bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:33.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.036+0000 7f6c6323af40 1 --2- 172.21.15.76:0/3638580860 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6c4c069850 0x7f6c4c06bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.036+0000 7f6c6323af40 1 -- 172.21.15.76:0/3638580860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b35055c50 msgr2=0x555b35086cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:33.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.036+0000 7f6c6323af40 1 --2- 172.21.15.76:0/3638580860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b35055c50 0x555b35086cf0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800cef0 tx=0x7f6c5800a5d0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.036+0000 7f6c6323af40 1 -- 172.21.15.76:0/3638580860 shutdown_connections 2023-11-09T03:47:33.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.036+0000 7f6c6323af40 1 --2- 172.21.15.76:0/3638580860 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6c4c069850 0x7f6c4c06bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.036+0000 7f6c6323af40 1 --2- 172.21.15.76:0/3638580860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555b35114d30 0x555b35084010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.036+0000 7f6c6323af40 1 --2- 172.21.15.76:0/3638580860 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555b35086780 0x555b351113c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.036+0000 7f6c6323af40 1 --2- 172.21.15.76:0/3638580860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b35055c50 0x555b35086cf0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.036+0000 7f6c6323af40 1 -- 172.21.15.76:0/3638580860 >> 172.21.15.76:0/3638580860 conn(0x555b35082f40 msgr2=0x555b3512e390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:33.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.036+0000 7f6c6323af40 1 -- 172.21.15.76:0/3638580860 shutdown_connections 2023-11-09T03:47:33.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.036+0000 7f6c6323af40 1 -- 172.21.15.76:0/3638580860 wait complete. 2023-11-09T03:47:33.049 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata mksnap snap2 2023-11-09T03:47:33.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.123+0000 7f40b1384f40 1 Processor -- start 2023-11-09T03:47:33.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.123+0000 7f40b1384f40 1 -- start start 2023-11-09T03:47:33.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.123+0000 7f40b1384f40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55691a49c530 0x55691a55d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:33.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.123+0000 7f40b1384f40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55691a49c910 0x55691a575180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:33.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.123+0000 7f40b1384f40 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55691a573ad0 0x55691a56ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:33.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.123+0000 7f40b1384f40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691a44a3b0 con 0x55691a49c910 2023-11-09T03:47:33.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.123+0000 7f40b1384f40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691a3e4650 con 0x55691a49c530 2023-11-09T03:47:33.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.123+0000 7f40b1384f40 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55691a3e42f0 con 0x55691a573ad0 2023-11-09T03:47:33.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.124+0000 7f40a6685640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55691a49c910 0x55691a575180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:33.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.124+0000 7f40aee86640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55691a49c530 0x55691a55d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:33.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.124+0000 7f40a6685640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55691a49c910 0x55691a575180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48088/0 (socket says 172.21.15.76:48088) 2023-11-09T03:47:33.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.124+0000 7f40af687640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55691a573ad0 0x55691a56ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:33.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.124+0000 7f40a6685640 1 -- 172.21.15.76:0/3403888373 learned_addr learned my addr 172.21.15.76:0/3403888373 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:33.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.124+0000 7f40a6685640 1 -- 172.21.15.76:0/3403888373 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55691a573ad0 msgr2=0x55691a56ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:33.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.124+0000 7f40a6685640 1 --2- 172.21.15.76:0/3403888373 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55691a573ad0 0x55691a56ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.124+0000 7f40a6685640 1 -- 172.21.15.76:0/3403888373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55691a49c530 msgr2=0x55691a55d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:33.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.124+0000 7f40a6685640 1 --2- 172.21.15.76:0/3403888373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55691a49c530 0x55691a55d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.124+0000 7f40a6685640 1 -- 172.21.15.76:0/3403888373 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4098005d20 con 0x55691a49c910 2023-11-09T03:47:33.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.124+0000 7f40a6685640 1 --2- 172.21.15.76:0/3403888373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55691a49c910 0x55691a575180 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f4098009a00 tx=0x7f4098005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b55457289310ee1 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:33.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.124+0000 7f40ade84640 1 -- 172.21.15.76:0/3403888373 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098017070 con 0x55691a49c910 2023-11-09T03:47:33.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.124+0000 7f40ade84640 1 -- 172.21.15.76:0/3403888373 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40980088b0 con 0x55691a49c910 2023-11-09T03:47:33.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.124+0000 7f40ade84640 1 -- 172.21.15.76:0/3403888373 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409800f040 con 0x55691a49c910 2023-11-09T03:47:33.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.125+0000 7f40b1384f40 1 -- 172.21.15.76:0/3403888373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55691a49c910 msgr2=0x55691a575180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:33.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.125+0000 7f40b1384f40 1 --2- 172.21.15.76:0/3403888373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55691a49c910 0x55691a575180 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f4098009a00 tx=0x7f4098005a00 comp rx=0 tx=0).stop 2023-11-09T03:47:33.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.125+0000 7f40b1384f40 1 -- 172.21.15.76:0/3403888373 shutdown_connections 2023-11-09T03:47:33.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.125+0000 7f40b1384f40 1 --2- 172.21.15.76:0/3403888373 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55691a573ad0 0x55691a56ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.125+0000 7f40b1384f40 1 --2- 172.21.15.76:0/3403888373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55691a49c910 0x55691a575180 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.125+0000 7f40b1384f40 1 --2- 172.21.15.76:0/3403888373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55691a49c530 0x55691a55d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.125+0000 7f40b1384f40 1 -- 172.21.15.76:0/3403888373 >> 172.21.15.76:0/3403888373 conn(0x55691a4c6f40 msgr2=0x55691a573f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:33.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.125+0000 7f40b1384f40 1 -- 172.21.15.76:0/3403888373 shutdown_connections 2023-11-09T03:47:33.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.125+0000 7f40b1384f40 1 -- 172.21.15.76:0/3403888373 wait complete. 2023-11-09T03:47:33.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.125+0000 7f40b1384f40 1 Processor -- start 2023-11-09T03:47:33.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.125+0000 7f40b1384f40 1 -- start start 2023-11-09T03:47:33.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:47:33.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.126+0000 7f40b1384f40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55691a49c530 0x55691a55bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:33.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.126+0000 7f40b1384f40 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55691a49c910 0x55691a55c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:33.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.126+0000 7f40b1384f40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55691a573ad0 0x55691a56e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:33.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.126+0000 7f40b1384f40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691a3c44f0 con 0x55691a573ad0 2023-11-09T03:47:33.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.126+0000 7f40b1384f40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691a4dee00 con 0x55691a49c530 2023-11-09T03:47:33.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.126+0000 7f40b1384f40 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55691a56da90 con 0x55691a49c910 2023-11-09T03:47:33.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.126+0000 7f40af687640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55691a573ad0 0x55691a56e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:33.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.126+0000 7f40af687640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55691a573ad0 0x55691a56e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48094/0 (socket says 172.21.15.76:48094) 2023-11-09T03:47:33.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.126+0000 7f40af687640 1 -- 172.21.15.76:0/1662519890 learned_addr learned my addr 172.21.15.76:0/1662519890 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:33.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.126+0000 7f40aee86640 1 --2- 172.21.15.76:0/1662519890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55691a49c530 0x55691a55bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:33.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.126+0000 7f40a6685640 1 --2- 172.21.15.76:0/1662519890 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55691a49c910 0x55691a55c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:33.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.126+0000 7f40af687640 1 -- 172.21.15.76:0/1662519890 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55691a49c910 msgr2=0x55691a55c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:33.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.126+0000 7f40af687640 1 --2- 172.21.15.76:0/1662519890 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55691a49c910 0x55691a55c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.126+0000 7f40af687640 1 -- 172.21.15.76:0/1662519890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55691a49c530 msgr2=0x55691a55bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:33.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.126+0000 7f40af687640 1 --2- 172.21.15.76:0/1662519890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55691a49c530 0x55691a55bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.126+0000 7f40af687640 1 -- 172.21.15.76:0/1662519890 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4098009600 con 0x55691a573ad0 2023-11-09T03:47:33.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.126+0000 7f40a6685640 1 --2- 172.21.15.76:0/1662519890 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55691a49c910 0x55691a55c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:33.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.126+0000 7f40aee86640 1 --2- 172.21.15.76:0/1662519890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55691a49c530 0x55691a55bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:33.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.126+0000 7f40af687640 1 --2- 172.21.15.76:0/1662519890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55691a573ad0 0x55691a56e530 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f40a800a3f0 tx=0x7f40a8003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:33.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.127+0000 7f40ace82640 1 -- 172.21.15.76:0/1662519890 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8052d80 con 0x55691a573ad0 2023-11-09T03:47:33.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.127+0000 7f40ace82640 1 -- 172.21.15.76:0/1662519890 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a8067410 con 0x55691a573ad0 2023-11-09T03:47:33.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.127+0000 7f40b1384f40 1 -- 172.21.15.76:0/1662519890 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a800a100 con 0x55691a573ad0 2023-11-09T03:47:33.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.127+0000 7f40ace82640 1 -- 172.21.15.76:0/1662519890 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a80677a0 con 0x55691a573ad0 2023-11-09T03:47:33.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.127+0000 7f40b1384f40 1 -- 172.21.15.76:0/1662519890 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55691a3f0780 con 0x55691a573ad0 2023-11-09T03:47:33.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.128+0000 7f40ace82640 1 -- 172.21.15.76:0/1662519890 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f40a805d410 con 0x55691a573ad0 2023-11-09T03:47:33.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.128+0000 7f40ace82640 1 --2- 172.21.15.76:0/1662519890 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4090069820 0x7f409006bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:33.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.128+0000 7f40ace82640 1 -- 172.21.15.76:0/1662519890 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f40a80caae0 con 0x55691a573ad0 2023-11-09T03:47:33.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.128+0000 7f40aee86640 1 --2- 172.21.15.76:0/1662519890 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4090069820 0x7f409006bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:33.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.128+0000 7f40b1384f40 1 -- 172.21.15.76:0/1662519890 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- pool_op(create snap pool 8 tid 1 name snap2 v0) v4 -- 0x55691a454330 con 0x55691a573ad0 2023-11-09T03:47:33.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.129+0000 7f40ace82640 1 -- 172.21.15.76:0/1662519890 <== mon.0 v2:172.21.15.76:3300/0 6 ==== pool_op_reply(tid 1 (95) Operation not supported v54) v1 ==== 43+0+0 (secure 0 0 0) 0x7f40a806c020 con 0x55691a573ad0 2023-11-09T03:47:33.139 INFO:teuthology.orchestra.run.smithi076.stderr:error creating pool cephfs_metadata snapshot snap2: (95) Operation not supported 2023-11-09T03:47:33.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.129+0000 7f40b1384f40 1 -- 172.21.15.76:0/1662519890 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4090069820 msgr2=0x7f409006bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:33.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.129+0000 7f40b1384f40 1 --2- 172.21.15.76:0/1662519890 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4090069820 0x7f409006bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.129+0000 7f40b1384f40 1 -- 172.21.15.76:0/1662519890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55691a573ad0 msgr2=0x55691a56e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:33.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.129+0000 7f40b1384f40 1 --2- 172.21.15.76:0/1662519890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55691a573ad0 0x55691a56e530 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f40a800a3f0 tx=0x7f40a8003330 comp rx=0 tx=0).stop 2023-11-09T03:47:33.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.129+0000 7f40aee86640 1 --2- 172.21.15.76:0/1662519890 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4090069820 0x7f409006bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:47:33.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.129+0000 7f40b1384f40 1 -- 172.21.15.76:0/1662519890 shutdown_connections 2023-11-09T03:47:33.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.129+0000 7f40b1384f40 1 --2- 172.21.15.76:0/1662519890 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4090069820 0x7f409006bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.129+0000 7f40b1384f40 1 --2- 172.21.15.76:0/1662519890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55691a573ad0 0x55691a56e530 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.129+0000 7f40b1384f40 1 --2- 172.21.15.76:0/1662519890 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55691a49c910 0x55691a55c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.129+0000 7f40b1384f40 1 --2- 172.21.15.76:0/1662519890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55691a49c530 0x55691a55bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.129+0000 7f40b1384f40 1 -- 172.21.15.76:0/1662519890 >> 172.21.15.76:0/1662519890 conn(0x55691a4c6f40 msgr2=0x55691a576720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:33.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.129+0000 7f40b1384f40 1 -- 172.21.15.76:0/1662519890 shutdown_connections 2023-11-09T03:47:33.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.129+0000 7f40b1384f40 1 -- 172.21.15.76:0/1662519890 wait complete. 2023-11-09T03:47:33.142 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool mksnap cephfs_data snap3 2023-11-09T03:47:33.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.277+0000 7fb3733cc640 1 Processor -- start 2023-11-09T03:47:33.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.278+0000 7fb3733cc640 1 -- start start 2023-11-09T03:47:33.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.278+0000 7fb3733cc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb36c159c00 0x7fb36c0cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:33.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.278+0000 7fb3733cc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb36c0c48e0 0x7fb36c0c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:33.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.278+0000 7fb3733cc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb36c15aff0 con 0x7fb36c0adac0 2023-11-09T03:47:33.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.278+0000 7fb3733cc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb36c0c52d0 con 0x7fb36c159c00 2023-11-09T03:47:33.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.278+0000 7fb3733cc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c15a3e0 con 0x7fb36c0c48e0 2023-11-09T03:47:33.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.279+0000 7fb370967640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb36c0c48e0 0x7fb36c0c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:33.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.279+0000 7fb371168640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb36c159c00 0x7fb36c0cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:33.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.279+0000 7fb371969640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb36c0adac0 0x7fb36c0adf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:46734/0 (socket says 172.21.15.76:46734) 2023-11-09T03:47:33.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.279+0000 7fb371168640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb36c159c00 0x7fb36c0cc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:36668/0 (socket says 172.21.15.76:36668) 2023-11-09T03:47:33.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.279+0000 7fb370967640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb36c0c48e0 0x7fb36c0c4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:47734/0 (socket says 172.21.15.76:47734) 2023-11-09T03:47:33.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.279+0000 7fb371969640 1 -- 172.21.15.76:0/417626933 learned_addr learned my addr 172.21.15.76:0/417626933 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:33.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.279+0000 7fb371168640 1 -- 172.21.15.76:0/417626933 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb36c0c48e0 msgr2=0x7fb36c0c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:33.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.279+0000 7fb371168640 1 --2- 172.21.15.76:0/417626933 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb36c0c48e0 0x7fb36c0c4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.279+0000 7fb371168640 1 -- 172.21.15.76:0/417626933 >> v1:172.21.15.113:6789/0 conn(0x7fb36c0adac0 legacy=0x7fb36c0adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:33.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.279+0000 7fb371168640 1 -- 172.21.15.76:0/417626933 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36c15a6c0 con 0x7fb36c159c00 2023-11-09T03:47:33.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.280+0000 7fb371168640 1 --2- 172.21.15.76:0/417626933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb36c159c00 0x7fb36c0cc890 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb3640097a0 tx=0x7fb36402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db8690bbd0f97c03 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:33.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.280+0000 7fb36bfff640 1 -- 172.21.15.76:0/417626933 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb364031830 con 0x7fb36c159c00 2023-11-09T03:47:33.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.280+0000 7fb36bfff640 1 -- 172.21.15.76:0/417626933 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb364031990 con 0x7fb36c159c00 2023-11-09T03:47:33.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.280+0000 7fb36bfff640 1 -- 172.21.15.76:0/417626933 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb364031d20 con 0x7fb36c159c00 2023-11-09T03:47:33.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.281+0000 7fb3733cc640 1 -- 172.21.15.76:0/417626933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb36c159c00 msgr2=0x7fb36c0cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:33.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.281+0000 7fb3733cc640 1 --2- 172.21.15.76:0/417626933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb36c159c00 0x7fb36c0cc890 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb3640097a0 tx=0x7fb36402b8d0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.281+0000 7fb3733cc640 1 -- 172.21.15.76:0/417626933 shutdown_connections 2023-11-09T03:47:33.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.281+0000 7fb3733cc640 1 --2- 172.21.15.76:0/417626933 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb36c0c48e0 0x7fb36c0c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.281+0000 7fb3733cc640 1 --2- 172.21.15.76:0/417626933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb36c159c00 0x7fb36c0cc890 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.281+0000 7fb3733cc640 1 -- 172.21.15.76:0/417626933 >> 172.21.15.76:0/417626933 conn(0x7fb36c0ce950 msgr2=0x7fb36c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:33.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.281+0000 7fb3733cc640 1 -- 172.21.15.76:0/417626933 shutdown_connections 2023-11-09T03:47:33.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.281+0000 7fb3733cc640 1 -- 172.21.15.76:0/417626933 wait complete. 2023-11-09T03:47:33.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.282+0000 7fb3733cc640 1 Processor -- start 2023-11-09T03:47:33.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.282+0000 7fb3733cc640 1 -- start start 2023-11-09T03:47:33.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.283+0000 7fb3733cc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb36c0adac0 0x7fb36c1f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:33.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.283+0000 7fb3733cc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb36c0c48e0 0x7fb36c0c3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:33.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.283+0000 7fb3733cc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb36c159c00 0x7fb36c1f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:33.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.283+0000 7fb3733cc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c1f6b10 con 0x7fb36c159c00 2023-11-09T03:47:33.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.283+0000 7fb3733cc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c1f6830 con 0x7fb36c0c48e0 2023-11-09T03:47:33.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.283+0000 7fb3733cc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c1f27e0 con 0x7fb36c0adac0 2023-11-09T03:47:33.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.283+0000 7fb371969640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb36c159c00 0x7fb36c1f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:33.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.283+0000 7fb371969640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb36c159c00 0x7fb36c1f5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48102/0 (socket says 172.21.15.76:48102) 2023-11-09T03:47:33.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.283+0000 7fb371969640 1 -- 172.21.15.76:0/3148942579 learned_addr learned my addr 172.21.15.76:0/3148942579 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:33.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.283+0000 7fb371168640 1 --2- 172.21.15.76:0/3148942579 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb36c0adac0 0x7fb36c1f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:33.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.283+0000 7fb370967640 1 --2- 172.21.15.76:0/3148942579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb36c0c48e0 0x7fb36c0c3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:33.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.283+0000 7fb371969640 1 -- 172.21.15.76:0/3148942579 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb36c0adac0 msgr2=0x7fb36c1f32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:33.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.283+0000 7fb371969640 1 --2- 172.21.15.76:0/3148942579 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb36c0adac0 0x7fb36c1f32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.283+0000 7fb371969640 1 -- 172.21.15.76:0/3148942579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb36c0c48e0 msgr2=0x7fb36c0c3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:33.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.283+0000 7fb371969640 1 --2- 172.21.15.76:0/3148942579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb36c0c48e0 0x7fb36c0c3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.283+0000 7fb371969640 1 -- 172.21.15.76:0/3148942579 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb364009400 con 0x7fb36c159c00 2023-11-09T03:47:33.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.283+0000 7fb371969640 1 --2- 172.21.15.76:0/3148942579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb36c159c00 0x7fb36c1f5cc0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fb35c00ca80 tx=0x7fb35c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:33.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.283+0000 7fb369ffb640 1 -- 172.21.15.76:0/3148942579 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35c00ac30 con 0x7fb36c159c00 2023-11-09T03:47:33.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.283+0000 7fb369ffb640 1 -- 172.21.15.76:0/3148942579 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35c00ad90 con 0x7fb36c159c00 2023-11-09T03:47:33.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.283+0000 7fb369ffb640 1 -- 172.21.15.76:0/3148942579 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35c012600 con 0x7fb36c159c00 2023-11-09T03:47:33.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.283+0000 7fb3733cc640 1 -- 172.21.15.76:0/3148942579 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb36c1f2ac0 con 0x7fb36c159c00 2023-11-09T03:47:33.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.284+0000 7fb3733cc640 1 -- 172.21.15.76:0/3148942579 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb36c14ef70 con 0x7fb36c159c00 2023-11-09T03:47:33.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.285+0000 7fb3733cc640 1 -- 172.21.15.76:0/3148942579 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb36c1f2fe0 con 0x7fb36c159c00 2023-11-09T03:47:33.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.285+0000 7fb369ffb640 1 -- 172.21.15.76:0/3148942579 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb35c007070 con 0x7fb36c159c00 2023-11-09T03:47:33.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.285+0000 7fb369ffb640 1 --2- 172.21.15.76:0/3148942579 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb35806ae70 0x7fb35806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:33.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.285+0000 7fb369ffb640 1 -- 172.21.15.76:0/3148942579 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb35c089740 con 0x7fb36c159c00 2023-11-09T03:47:33.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.286+0000 7fb371168640 1 --2- 172.21.15.76:0/3148942579 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb35806ae70 0x7fb35806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:33.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.286+0000 7fb371168640 1 --2- 172.21.15.76:0/3148942579 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb35806ae70 0x7fb35806d330 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb364036000 tx=0x7fb36403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:33.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.290+0000 7fb369ffb640 1 -- 172.21.15.76:0/3148942579 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb36c1f2fe0 con 0x7fb36c159c00 2023-11-09T03:47:33.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.458+0000 7fb3733cc640 1 -- 172.21.15.76:0/3148942579 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool mksnap", "pool": "cephfs_data", "snap": "snap3"} v 0) v1 -- 0x7fb36c1f2fe0 con 0x7fb36c159c00 2023-11-09T03:47:33.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.458+0000 7fb369ffb640 1 -- 172.21.15.76:0/3148942579 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool mksnap", "pool": "cephfs_data", "snap": "snap3"}]=-95 (95) Operation not supported v54) v1 ==== 131+0+0 (secure 0 0 0) 0x7fb36c1f2fe0 con 0x7fb36c159c00 2023-11-09T03:47:33.458 INFO:teuthology.orchestra.run.smithi076.stderr:Error ENOTSUP: (95) Operation not supported 2023-11-09T03:47:33.463 DEBUG:teuthology.orchestra.run:got remote process result: 95 2023-11-09T03:47:33.464 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool mksnap cephfs_metadata snap4 2023-11-09T03:47:33.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.567+0000 7f432b539640 1 Processor -- start 2023-11-09T03:47:33.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.567+0000 7f432b539640 1 -- start start 2023-11-09T03:47:33.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.568+0000 7f432b539640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4324159c00 0x7f43240cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:33.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.568+0000 7f432b539640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f43240c48e0 0x7f43240c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:33.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.568+0000 7f432b539640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f432415aff0 con 0x7f43240adac0 2023-11-09T03:47:33.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.568+0000 7f432b539640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43240c52d0 con 0x7f43240c48e0 2023-11-09T03:47:33.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.568+0000 7f432b539640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f432415a3e0 con 0x7f4324159c00 2023-11-09T03:47:33.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.568+0000 7f4328ad4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f43240c48e0 0x7f43240c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:33.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.568+0000 7f4328ad4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f43240c48e0 0x7f43240c4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48104/0 (socket says 172.21.15.76:48104) 2023-11-09T03:47:33.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.568+0000 7f4328ad4640 1 -- 172.21.15.76:0/1422556410 learned_addr learned my addr 172.21.15.76:0/1422556410 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:33.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.568+0000 7f43292d5640 1 --2- 172.21.15.76:0/1422556410 >> v2:172.21.15.113:3300/0 conn(0x7f4324159c00 0x7f43240cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:33.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.568+0000 7f4328ad4640 1 -- 172.21.15.76:0/1422556410 >> v2:172.21.15.113:3300/0 conn(0x7f4324159c00 msgr2=0x7f43240cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:33.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.568+0000 7f4328ad4640 1 --2- 172.21.15.76:0/1422556410 >> v2:172.21.15.113:3300/0 conn(0x7f4324159c00 0x7f43240cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.568+0000 7f4328ad4640 1 -- 172.21.15.76:0/1422556410 >> v1:172.21.15.113:6789/0 conn(0x7f43240adac0 legacy=0x7f43240adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:33.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.568+0000 7f4328ad4640 1 -- 172.21.15.76:0/1422556410 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432415a6c0 con 0x7f43240c48e0 2023-11-09T03:47:33.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.568+0000 7f43292d5640 1 --2- 172.21.15.76:0/1422556410 >> v2:172.21.15.113:3300/0 conn(0x7f4324159c00 0x7f43240cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:33.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.569+0000 7f4328ad4640 1 --2- 172.21.15.76:0/1422556410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43240c48e0 0x7f43240c4d90 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f43240cc4d0 tx=0x7f431402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5782bfc0b3ec9127 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:33.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.569+0000 7f431bfff640 1 -- 172.21.15.76:0/1422556410 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43140317e0 con 0x7f43240c48e0 2023-11-09T03:47:33.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.569+0000 7f431bfff640 1 -- 172.21.15.76:0/1422556410 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4314031940 con 0x7f43240c48e0 2023-11-09T03:47:33.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.569+0000 7f431bfff640 1 -- 172.21.15.76:0/1422556410 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314031cd0 con 0x7f43240c48e0 2023-11-09T03:47:33.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.570+0000 7f432b539640 1 -- 172.21.15.76:0/1422556410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43240c48e0 msgr2=0x7f43240c4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:33.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.570+0000 7f432b539640 1 --2- 172.21.15.76:0/1422556410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43240c48e0 0x7f43240c4d90 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f43240cc4d0 tx=0x7f431402b880 comp rx=0 tx=0).stop 2023-11-09T03:47:33.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.570+0000 7f432b539640 1 -- 172.21.15.76:0/1422556410 shutdown_connections 2023-11-09T03:47:33.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.570+0000 7f432b539640 1 --2- 172.21.15.76:0/1422556410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43240c48e0 0x7f43240c4d90 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.570+0000 7f432b539640 1 --2- 172.21.15.76:0/1422556410 >> v2:172.21.15.113:3300/0 conn(0x7f4324159c00 0x7f43240cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.570+0000 7f432b539640 1 -- 172.21.15.76:0/1422556410 >> 172.21.15.76:0/1422556410 conn(0x7f43240ce950 msgr2=0x7f43240ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:33.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.570+0000 7f432b539640 1 -- 172.21.15.76:0/1422556410 shutdown_connections 2023-11-09T03:47:33.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.570+0000 7f432b539640 1 -- 172.21.15.76:0/1422556410 wait complete. 2023-11-09T03:47:33.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.571+0000 7f432b539640 1 Processor -- start 2023-11-09T03:47:33.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.571+0000 7f432b539640 1 -- start start 2023-11-09T03:47:33.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.572+0000 7f432b539640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43240adac0 0x7f4324144440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:33.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.572+0000 7f432b539640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43240c48e0 0x7f432414e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:33.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.572+0000 7f432b539640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4324159c00 0x7f4324150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:33.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.572+0000 7f432b539640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432414f650 con 0x7f43240adac0 2023-11-09T03:47:33.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.572+0000 7f432b539640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432414ea40 con 0x7f43240c48e0 2023-11-09T03:47:33.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.572+0000 7f432b539640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432414ed20 con 0x7f4324159c00 2023-11-09T03:47:33.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.572+0000 7f43292d5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43240adac0 0x7f4324144440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:33.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.572+0000 7f43292d5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43240adac0 0x7f4324144440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48112/0 (socket says 172.21.15.76:48112) 2023-11-09T03:47:33.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.572+0000 7f43292d5640 1 -- 172.21.15.76:0/396382439 learned_addr learned my addr 172.21.15.76:0/396382439 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:33.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.572+0000 7f4328ad4640 1 --2- 172.21.15.76:0/396382439 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43240c48e0 0x7f432414e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:33.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.572+0000 7f4329ad6640 1 --2- 172.21.15.76:0/396382439 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4324159c00 0x7f4324150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:33.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.572+0000 7f43292d5640 1 -- 172.21.15.76:0/396382439 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4324159c00 msgr2=0x7f4324150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:33.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.572+0000 7f43292d5640 1 --2- 172.21.15.76:0/396382439 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4324159c00 0x7f4324150550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.572+0000 7f43292d5640 1 -- 172.21.15.76:0/396382439 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43240c48e0 msgr2=0x7f432414e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:33.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.572+0000 7f43292d5640 1 --2- 172.21.15.76:0/396382439 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43240c48e0 0x7f432414e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:33.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.572+0000 7f43292d5640 1 -- 172.21.15.76:0/396382439 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43140094d0 con 0x7f43240adac0 2023-11-09T03:47:33.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.572+0000 7f4328ad4640 1 --2- 172.21.15.76:0/396382439 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43240c48e0 0x7f432414e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:33.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.572+0000 7f43292d5640 1 --2- 172.21.15.76:0/396382439 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43240adac0 0x7f4324144440 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f432000eb40 tx=0x7f432000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:33.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.572+0000 7f4319ffb640 1 -- 172.21.15.76:0/396382439 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4320007c10 con 0x7f43240adac0 2023-11-09T03:47:33.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.572+0000 7f4319ffb640 1 -- 172.21.15.76:0/396382439 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4320007d70 con 0x7f43240adac0 2023-11-09T03:47:33.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.572+0000 7f432b539640 1 -- 172.21.15.76:0/396382439 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f432414f000 con 0x7f43240adac0 2023-11-09T03:47:33.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.572+0000 7f4319ffb640 1 -- 172.21.15.76:0/396382439 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4320010680 con 0x7f43240adac0 2023-11-09T03:47:33.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.573+0000 7f432b539640 1 -- 172.21.15.76:0/396382439 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4324144980 con 0x7f43240adac0 2023-11-09T03:47:33.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.573+0000 7f432b539640 1 -- 172.21.15.76:0/396382439 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4324074290 con 0x7f43240adac0 2023-11-09T03:47:33.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.574+0000 7f4319ffb640 1 -- 172.21.15.76:0/396382439 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f43200107e0 con 0x7f43240adac0 2023-11-09T03:47:33.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.574+0000 7f4319ffb640 1 --2- 172.21.15.76:0/396382439 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f430806ae70 0x7f430806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:33.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.574+0000 7f4319ffb640 1 -- 172.21.15.76:0/396382439 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4320014070 con 0x7f43240adac0 2023-11-09T03:47:33.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.574+0000 7f4328ad4640 1 --2- 172.21.15.76:0/396382439 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f430806ae70 0x7f430806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:33.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.575+0000 7f4328ad4640 1 --2- 172.21.15.76:0/396382439 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f430806ae70 0x7f430806d330 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4314036000 tx=0x7f431403d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:33.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.578+0000 7f4319ffb640 1 -- 172.21.15.76:0/396382439 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4320051ca0 con 0x7f43240adac0 2023-11-09T03:47:33.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.747+0000 7f432b539640 1 -- 172.21.15.76:0/396382439 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool mksnap", "pool": "cephfs_metadata", "snap": "snap4"} v 0) v1 -- 0x7f43240725c0 con 0x7f43240adac0 2023-11-09T03:47:33.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:33.747+0000 7f4319ffb640 1 -- 172.21.15.76:0/396382439 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool mksnap", "pool": "cephfs_metadata", "snap": "snap4"}]=-95 (95) Operation not supported v54) v1 ==== 135+0+0 (secure 0 0 0) 0x7f4320056c00 con 0x7f43240adac0 2023-11-09T03:47:33.746 INFO:teuthology.orchestra.run.smithi076.stderr:Error ENOTSUP: (95) Operation not supported 2023-11-09T03:47:33.751 DEBUG:teuthology.orchestra.run:got remote process result: 95 2023-11-09T03:47:33.752 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-09T03:47:33.753 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-09T03:47:33.803 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-09T03:47:33.855 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-09T03:47:33.855 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:47:33.855 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:47:33.871 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:47:34.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.011+0000 7f7f5554c640 1 Processor -- start 2023-11-09T03:47:34.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.011+0000 7f7f5554c640 1 -- start start 2023-11-09T03:47:34.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.012+0000 7f7f5554c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7f500cda60 0x7f7f500ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:34.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.012+0000 7f7f5554c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f50070d40 0x7f7f5006c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:34.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.012+0000 7f7f5554c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f500cc990 con 0x7f7f500b0cd0 2023-11-09T03:47:34.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.012+0000 7f7f5554c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f500ccc70 con 0x7f7f500cda60 2023-11-09T03:47:34.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.012+0000 7f7f5554c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f500c9ce0 con 0x7f7f50070d40 2023-11-09T03:47:34.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.012+0000 7f7f4effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7f500cda60 0x7f7f500ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:34.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.012+0000 7f7f4e7fc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f7f500b0cd0 0x7f7f50070630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47214/0 (socket says 172.21.15.76:47214) 2023-11-09T03:47:34.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.012+0000 7f7f4e7fc640 1 -- 172.21.15.76:0/4225437650 learned_addr learned my addr 172.21.15.76:0/4225437650 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:34.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.012+0000 7f7f4f7fe640 1 --2- 172.21.15.76:0/4225437650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f50070d40 0x7f7f5006c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:34.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.012+0000 7f7f4effd640 1 -- 172.21.15.76:0/4225437650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f50070d40 msgr2=0x7f7f5006c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:34.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.012+0000 7f7f4effd640 1 --2- 172.21.15.76:0/4225437650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f50070d40 0x7f7f5006c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.012+0000 7f7f4effd640 1 -- 172.21.15.76:0/4225437650 >> v1:172.21.15.76:6789/0 conn(0x7f7f500b0cd0 legacy=0x7f7f50070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:34.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.012+0000 7f7f4effd640 1 -- 172.21.15.76:0/4225437650 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f500c9fc0 con 0x7f7f500cda60 2023-11-09T03:47:34.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.012+0000 7f7f4dffb640 1 -- 172.21.15.76:0/4225437650 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3936827531 0 0) 0x7f7f500cc990 con 0x7f7f500b0cd0 2023-11-09T03:47:34.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.013+0000 7f7f4effd640 1 --2- 172.21.15.76:0/4225437650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f500cda60 0x7f7f500ca7a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7f400097a0 tx=0x7f7f4002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444735002cb59ea2 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:34.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.013+0000 7f7f4dffb640 1 -- 172.21.15.76:0/4225437650 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f40031830 con 0x7f7f500cda60 2023-11-09T03:47:34.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.013+0000 7f7f4dffb640 1 -- 172.21.15.76:0/4225437650 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f40031990 con 0x7f7f500cda60 2023-11-09T03:47:34.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.013+0000 7f7f4dffb640 1 -- 172.21.15.76:0/4225437650 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f40031d20 con 0x7f7f500cda60 2023-11-09T03:47:34.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.014+0000 7f7f5554c640 1 -- 172.21.15.76:0/4225437650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f500cda60 msgr2=0x7f7f500ca7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:34.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.014+0000 7f7f5554c640 1 --2- 172.21.15.76:0/4225437650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f500cda60 0x7f7f500ca7a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7f400097a0 tx=0x7f7f4002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.014+0000 7f7f5554c640 1 -- 172.21.15.76:0/4225437650 shutdown_connections 2023-11-09T03:47:34.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.014+0000 7f7f5554c640 1 --2- 172.21.15.76:0/4225437650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f50070d40 0x7f7f5006c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.014+0000 7f7f5554c640 1 --2- 172.21.15.76:0/4225437650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f500cda60 0x7f7f500ca7a0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.014+0000 7f7f5554c640 1 -- 172.21.15.76:0/4225437650 >> 172.21.15.76:0/4225437650 conn(0x7f7f500ce950 msgr2=0x7f7f500ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:34.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.014+0000 7f7f5554c640 1 -- 172.21.15.76:0/4225437650 shutdown_connections 2023-11-09T03:47:34.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.014+0000 7f7f5554c640 1 -- 172.21.15.76:0/4225437650 wait complete. 2023-11-09T03:47:34.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.015+0000 7f7f5554c640 1 Processor -- start 2023-11-09T03:47:34.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.015+0000 7f7f5554c640 1 -- start start 2023-11-09T03:47:34.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.016+0000 7f7f5554c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f50070d40 0x7f7f50158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:34.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.016+0000 7f7f5554c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f500b0cd0 0x7f7f500b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:34.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.016+0000 7f7f5554c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f500cda60 0x7f7f501e4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:34.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.016+0000 7f7f5554c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f501e42f0 con 0x7f7f50070d40 2023-11-09T03:47:34.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.016+0000 7f7f5554c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f500b3af0 con 0x7f7f500b0cd0 2023-11-09T03:47:34.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.016+0000 7f7f5554c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f501e3670 con 0x7f7f500cda60 2023-11-09T03:47:34.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.016+0000 7f7f4effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f50070d40 0x7f7f50158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:34.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.016+0000 7f7f4effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f50070d40 0x7f7f50158730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48136/0 (socket says 172.21.15.76:48136) 2023-11-09T03:47:34.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.016+0000 7f7f4effd640 1 -- 172.21.15.76:0/3600166310 learned_addr learned my addr 172.21.15.76:0/3600166310 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:34.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.016+0000 7f7f4f7fe640 1 --2- 172.21.15.76:0/3600166310 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f500cda60 0x7f7f501e4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:34.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.016+0000 7f7f4e7fc640 1 --2- 172.21.15.76:0/3600166310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f500b0cd0 0x7f7f500b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:34.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.016+0000 7f7f4effd640 1 -- 172.21.15.76:0/3600166310 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f500cda60 msgr2=0x7f7f501e4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:34.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.016+0000 7f7f4effd640 1 --2- 172.21.15.76:0/3600166310 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f500cda60 0x7f7f501e4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.016+0000 7f7f4effd640 1 -- 172.21.15.76:0/3600166310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f500b0cd0 msgr2=0x7f7f500b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:34.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.016+0000 7f7f4effd640 1 --2- 172.21.15.76:0/3600166310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f500b0cd0 0x7f7f500b35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.016+0000 7f7f4effd640 1 -- 172.21.15.76:0/3600166310 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f40009400 con 0x7f7f50070d40 2023-11-09T03:47:34.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.017+0000 7f7f4effd640 1 --2- 172.21.15.76:0/3600166310 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f50070d40 0x7f7f50158730 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7f40000c00 tx=0x7f7f400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:34.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.017+0000 7f7f377fe640 1 -- 172.21.15.76:0/3600166310 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4003aa60 con 0x7f7f50070d40 2023-11-09T03:47:34.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.017+0000 7f7f377fe640 1 -- 172.21.15.76:0/3600166310 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4003abc0 con 0x7f7f50070d40 2023-11-09T03:47:34.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.017+0000 7f7f5554c640 1 -- 172.21.15.76:0/3600166310 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f501e3950 con 0x7f7f50070d40 2023-11-09T03:47:34.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.017+0000 7f7f377fe640 1 -- 172.21.15.76:0/3600166310 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f400443f0 con 0x7f7f50070d40 2023-11-09T03:47:34.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.017+0000 7f7f5554c640 1 -- 172.21.15.76:0/3600166310 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f501e3f10 con 0x7f7f50070d40 2023-11-09T03:47:34.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.018+0000 7f7f5554c640 1 -- 172.21.15.76:0/3600166310 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f5015c700 con 0x7f7f50070d40 2023-11-09T03:47:34.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.018+0000 7f7f377fe640 1 -- 172.21.15.76:0/3600166310 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7f4004d450 con 0x7f7f50070d40 2023-11-09T03:47:34.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.018+0000 7f7f377fe640 1 --2- 172.21.15.76:0/3600166310 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7f3c06ae70 0x7f7f3c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:34.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.019+0000 7f7f377fe640 1 -- 172.21.15.76:0/3600166310 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7f400ab950 con 0x7f7f50070d40 2023-11-09T03:47:34.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.019+0000 7f7f4e7fc640 1 --2- 172.21.15.76:0/3600166310 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7f3c06ae70 0x7f7f3c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:34.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.019+0000 7f7f4e7fc640 1 --2- 172.21.15.76:0/3600166310 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7f3c06ae70 0x7f7f3c06d330 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7f44007fd0 tx=0x7f7f44009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:34.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.023+0000 7f7f377fe640 1 -- 172.21.15.76:0/3600166310 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7f400767f0 con 0x7f7f50070d40 2023-11-09T03:47:34.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.189+0000 7f7f5554c640 1 -- 172.21.15.76:0/3600166310 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f50074290 con 0x7f7f50070d40 2023-11-09T03:47:34.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.190+0000 7f7f377fe640 1 -- 172.21.15.76:0/3600166310 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+18511 (secure 0 0 0) 0x7f7f40035020 con 0x7f7f50070d40 2023-11-09T03:47:34.190 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:34.190 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":54,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:47:21.936243+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:47:18.807110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-09T03:47:19.235022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:47:34.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.192+0000 7f7f5554c640 1 -- 172.21.15.76:0/3600166310 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7f3c06ae70 msgr2=0x7f7f3c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:34.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.192+0000 7f7f5554c640 1 --2- 172.21.15.76:0/3600166310 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7f3c06ae70 0x7f7f3c06d330 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7f44007fd0 tx=0x7f7f44009040 comp rx=0 tx=0).stop 2023-11-09T03:47:34.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.192+0000 7f7f5554c640 1 -- 172.21.15.76:0/3600166310 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f50070d40 msgr2=0x7f7f50158730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:34.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.192+0000 7f7f5554c640 1 --2- 172.21.15.76:0/3600166310 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f50070d40 0x7f7f50158730 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7f40000c00 tx=0x7f7f400046c0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.192+0000 7f7f5554c640 1 -- 172.21.15.76:0/3600166310 shutdown_connections 2023-11-09T03:47:34.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.192+0000 7f7f5554c640 1 --2- 172.21.15.76:0/3600166310 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7f3c06ae70 0x7f7f3c06d330 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.192+0000 7f7f5554c640 1 --2- 172.21.15.76:0/3600166310 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f500cda60 0x7f7f501e4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.192+0000 7f7f5554c640 1 --2- 172.21.15.76:0/3600166310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f500b0cd0 0x7f7f500b35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.192+0000 7f7f5554c640 1 --2- 172.21.15.76:0/3600166310 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f50070d40 0x7f7f50158730 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.192+0000 7f7f5554c640 1 -- 172.21.15.76:0/3600166310 >> 172.21.15.76:0/3600166310 conn(0x7f7f500ce950 msgr2=0x7f7f5006d310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:34.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.192+0000 7f7f5554c640 1 -- 172.21.15.76:0/3600166310 shutdown_connections 2023-11-09T03:47:34.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.192+0000 7f7f5554c640 1 -- 172.21.15.76:0/3600166310 wait complete. 2023-11-09T03:47:34.203 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-09T03:47:00.036540+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:47:18.807110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:47:19.235022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-09T03:47:34.203 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:47:34.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.307+0000 7fb285ada640 1 Processor -- start 2023-11-09T03:47:34.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.307+0000 7fb285ada640 1 -- start start 2023-11-09T03:47:34.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.308+0000 7fb285ada640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb280159c00 0x7fb2800cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:34.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.308+0000 7fb285ada640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb2800ae1d0 0x7fb2800c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:34.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.308+0000 7fb285ada640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb28015aff0 con 0x7fb2800c48e0 2023-11-09T03:47:34.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.308+0000 7fb285ada640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2800c52a0 con 0x7fb2800ae1d0 2023-11-09T03:47:34.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.308+0000 7fb285ada640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb28015a3e0 con 0x7fb280159c00 2023-11-09T03:47:34.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.308+0000 7fb27f7fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb280159c00 0x7fb2800cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:34.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.308+0000 7fb27ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb2800ae1d0 0x7fb2800c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:34.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.308+0000 7fb27ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb2800ae1d0 0x7fb2800c4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:36704/0 (socket says 172.21.15.76:36704) 2023-11-09T03:47:34.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.308+0000 7fb27ffff640 1 -- 172.21.15.76:0/860220472 learned_addr learned my addr 172.21.15.76:0/860220472 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:34.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.309+0000 7fb27f7fe640 1 -- 172.21.15.76:0/860220472 >> v2:172.21.15.113:3300/0 conn(0x7fb2800ae1d0 msgr2=0x7fb2800c4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:34.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.309+0000 7fb27f7fe640 1 --2- 172.21.15.76:0/860220472 >> v2:172.21.15.113:3300/0 conn(0x7fb2800ae1d0 0x7fb2800c4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.309+0000 7fb27f7fe640 1 -- 172.21.15.76:0/860220472 >> v1:172.21.15.113:6789/0 conn(0x7fb2800c48e0 legacy=0x7fb2800adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:34.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.309+0000 7fb27f7fe640 1 -- 172.21.15.76:0/860220472 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb28015a6c0 con 0x7fb280159c00 2023-11-09T03:47:34.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.309+0000 7fb27ffff640 1 --2- 172.21.15.76:0/860220472 >> v2:172.21.15.113:3300/0 conn(0x7fb2800ae1d0 0x7fb2800c4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:47:34.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.309+0000 7fb27f7fe640 1 --2- 172.21.15.76:0/860220472 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb280159c00 0x7fb2800cc890 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb27000ba40 tx=0x7fb27002e080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89e238fd9e908ac1 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:34.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.310+0000 7fb27e7fc640 1 -- 172.21.15.76:0/860220472 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27002ee30 con 0x7fb280159c00 2023-11-09T03:47:34.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.310+0000 7fb27e7fc640 1 -- 172.21.15.76:0/860220472 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb270036440 con 0x7fb280159c00 2023-11-09T03:47:34.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.310+0000 7fb27e7fc640 1 -- 172.21.15.76:0/860220472 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2700367d0 con 0x7fb280159c00 2023-11-09T03:47:34.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.310+0000 7fb285ada640 1 -- 172.21.15.76:0/860220472 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb280159c00 msgr2=0x7fb2800cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:34.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.310+0000 7fb285ada640 1 --2- 172.21.15.76:0/860220472 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb280159c00 0x7fb2800cc890 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb27000ba40 tx=0x7fb27002e080 comp rx=0 tx=0).stop 2023-11-09T03:47:34.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.310+0000 7fb285ada640 1 -- 172.21.15.76:0/860220472 shutdown_connections 2023-11-09T03:47:34.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.310+0000 7fb285ada640 1 --2- 172.21.15.76:0/860220472 >> v2:172.21.15.113:3300/0 conn(0x7fb2800ae1d0 0x7fb2800c4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.310+0000 7fb285ada640 1 --2- 172.21.15.76:0/860220472 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb280159c00 0x7fb2800cc890 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.310+0000 7fb285ada640 1 -- 172.21.15.76:0/860220472 >> 172.21.15.76:0/860220472 conn(0x7fb2800ce950 msgr2=0x7fb2800ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:34.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.310+0000 7fb285ada640 1 -- 172.21.15.76:0/860220472 shutdown_connections 2023-11-09T03:47:34.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.310+0000 7fb285ada640 1 -- 172.21.15.76:0/860220472 wait complete. 2023-11-09T03:47:34.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.311+0000 7fb285ada640 1 Processor -- start 2023-11-09T03:47:34.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.312+0000 7fb285ada640 1 -- start start 2023-11-09T03:47:34.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.312+0000 7fb285ada640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2800ae1d0 0x7fb2801534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:34.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.312+0000 7fb285ada640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2800c48e0 0x7fb280155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:34.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.312+0000 7fb285ada640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb280159c00 0x7fb2801e5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:34.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.312+0000 7fb285ada640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2801e3080 con 0x7fb280159c00 2023-11-09T03:47:34.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.312+0000 7fb285ada640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2801e2d10 con 0x7fb2800ae1d0 2023-11-09T03:47:34.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.312+0000 7fb285ada640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb280153a30 con 0x7fb2800c48e0 2023-11-09T03:47:34.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.312+0000 7fb27ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb280159c00 0x7fb2801e5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:34.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.312+0000 7fb27ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb280159c00 0x7fb2801e5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48144/0 (socket says 172.21.15.76:48144) 2023-11-09T03:47:34.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.312+0000 7fb27ffff640 1 -- 172.21.15.76:0/3658155949 learned_addr learned my addr 172.21.15.76:0/3658155949 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:34.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.313+0000 7fb27f7fe640 1 --2- 172.21.15.76:0/3658155949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2800ae1d0 0x7fb2801534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:34.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.313+0000 7fb27effd640 1 --2- 172.21.15.76:0/3658155949 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2800c48e0 0x7fb280155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:34.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.313+0000 7fb27ffff640 1 -- 172.21.15.76:0/3658155949 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2800c48e0 msgr2=0x7fb280155a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:34.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.313+0000 7fb27ffff640 1 --2- 172.21.15.76:0/3658155949 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2800c48e0 0x7fb280155a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.313+0000 7fb27ffff640 1 -- 172.21.15.76:0/3658155949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2800ae1d0 msgr2=0x7fb2801534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:34.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.313+0000 7fb27ffff640 1 --2- 172.21.15.76:0/3658155949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2800ae1d0 0x7fb2801534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.313+0000 7fb27ffff640 1 -- 172.21.15.76:0/3658155949 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb278009b30 con 0x7fb280159c00 2023-11-09T03:47:34.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.313+0000 7fb27ffff640 1 --2- 172.21.15.76:0/3658155949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb280159c00 0x7fb2801e5580 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb278009f10 tx=0x7fb27800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:34.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.313+0000 7fb267fff640 1 -- 172.21.15.76:0/3658155949 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb278018070 con 0x7fb280159c00 2023-11-09T03:47:34.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.313+0000 7fb267fff640 1 -- 172.21.15.76:0/3658155949 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27800f040 con 0x7fb280159c00 2023-11-09T03:47:34.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.313+0000 7fb267fff640 1 -- 172.21.15.76:0/3658155949 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27800ce60 con 0x7fb280159c00 2023-11-09T03:47:34.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.313+0000 7fb285ada640 1 -- 172.21.15.76:0/3658155949 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb27000b6a0 con 0x7fb280159c00 2023-11-09T03:47:34.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.313+0000 7fb285ada640 1 -- 172.21.15.76:0/3658155949 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb280156720 con 0x7fb280159c00 2023-11-09T03:47:34.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.314+0000 7fb285ada640 1 -- 172.21.15.76:0/3658155949 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2801e31c0 con 0x7fb280159c00 2023-11-09T03:47:34.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.314+0000 7fb267fff640 1 -- 172.21.15.76:0/3658155949 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb278014480 con 0x7fb280159c00 2023-11-09T03:47:34.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.315+0000 7fb267fff640 1 --2- 172.21.15.76:0/3658155949 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb26c06ae70 0x7fb26c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:34.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.315+0000 7fb267fff640 1 -- 172.21.15.76:0/3658155949 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb278089b30 con 0x7fb280159c00 2023-11-09T03:47:34.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.315+0000 7fb27f7fe640 1 --2- 172.21.15.76:0/3658155949 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb26c06ae70 0x7fb26c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:34.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.315+0000 7fb27f7fe640 1 --2- 172.21.15.76:0/3658155949 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb26c06ae70 0x7fb26c06d330 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb270038000 tx=0x7fb270032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:34.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.319+0000 7fb267fff640 1 -- 172.21.15.76:0/3658155949 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb2801e31c0 con 0x7fb280159c00 2023-11-09T03:47:34.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.486+0000 7fb285ada640 1 -- 172.21.15.76:0/3658155949 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb2801e31c0 con 0x7fb280159c00 2023-11-09T03:47:34.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.486+0000 7fb267fff640 1 -- 172.21.15.76:0/3658155949 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb2801e31c0 con 0x7fb280159c00 2023-11-09T03:47:34.486 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:47:34.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.488+0000 7fb285ada640 1 -- 172.21.15.76:0/3658155949 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb26c06ae70 msgr2=0x7fb26c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:34.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.488+0000 7fb285ada640 1 --2- 172.21.15.76:0/3658155949 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb26c06ae70 0x7fb26c06d330 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb270038000 tx=0x7fb270032000 comp rx=0 tx=0).stop 2023-11-09T03:47:34.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.488+0000 7fb285ada640 1 -- 172.21.15.76:0/3658155949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb280159c00 msgr2=0x7fb2801e5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:34.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.488+0000 7fb285ada640 1 --2- 172.21.15.76:0/3658155949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb280159c00 0x7fb2801e5580 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb278009f10 tx=0x7fb27800c490 comp rx=0 tx=0).stop 2023-11-09T03:47:34.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.488+0000 7fb285ada640 1 -- 172.21.15.76:0/3658155949 shutdown_connections 2023-11-09T03:47:34.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.488+0000 7fb285ada640 1 --2- 172.21.15.76:0/3658155949 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb26c06ae70 0x7fb26c06d330 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.488+0000 7fb285ada640 1 --2- 172.21.15.76:0/3658155949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb280159c00 0x7fb2801e5580 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.488+0000 7fb285ada640 1 --2- 172.21.15.76:0/3658155949 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2800c48e0 0x7fb280155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.488+0000 7fb285ada640 1 --2- 172.21.15.76:0/3658155949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2800ae1d0 0x7fb2801534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.488+0000 7fb285ada640 1 -- 172.21.15.76:0/3658155949 >> 172.21.15.76:0/3658155949 conn(0x7fb2800ce950 msgr2=0x7fb280153fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:34.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.488+0000 7fb285ada640 1 -- 172.21.15.76:0/3658155949 shutdown_connections 2023-11-09T03:47:34.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.488+0000 7fb285ada640 1 -- 172.21.15.76:0/3658155949 wait complete. 2023-11-09T03:47:34.498 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-09T03:47:34.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.602+0000 7f20fd0e5640 1 Processor -- start 2023-11-09T03:47:34.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.602+0000 7f20fd0e5640 1 -- start start 2023-11-09T03:47:34.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.602+0000 7f20fd0e5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f20f815ad40 0x7f20f806b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:34.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.602+0000 7f20fd0e5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20f815b120 0x7f20f815f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:34.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.602+0000 7f20fd0e5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f20f80c9b60 0x7f20f8070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:34.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.602+0000 7f20fd0e5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f806bc40 con 0x7f20f815ad40 2023-11-09T03:47:34.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.602+0000 7f20fd0e5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f8065df0 con 0x7f20f80c9b60 2023-11-09T03:47:34.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.602+0000 7f20fd0e5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f815f910 con 0x7f20f815b120 2023-11-09T03:47:34.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.603+0000 7f20f6d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f20f815ad40 0x7f20f806b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:34.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.603+0000 7f20f6d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f20f815ad40 0x7f20f806b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48160/0 (socket says 172.21.15.76:48160) 2023-11-09T03:47:34.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.603+0000 7f20f6d9d640 1 -- 172.21.15.76:0/319157879 learned_addr learned my addr 172.21.15.76:0/319157879 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:34.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.603+0000 7f20ee59c640 1 --2- 172.21.15.76:0/319157879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20f815b120 0x7f20f815f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:34.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.603+0000 7f20f759e640 1 --2- 172.21.15.76:0/319157879 >> v2:172.21.15.113:3300/0 conn(0x7f20f80c9b60 0x7f20f8070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:34.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.603+0000 7f20f6d9d640 1 -- 172.21.15.76:0/319157879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20f815b120 msgr2=0x7f20f815f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:34.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.603+0000 7f20f6d9d640 1 --2- 172.21.15.76:0/319157879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20f815b120 0x7f20f815f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.603+0000 7f20f6d9d640 1 -- 172.21.15.76:0/319157879 >> v2:172.21.15.113:3300/0 conn(0x7f20f80c9b60 msgr2=0x7f20f8070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:34.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.603+0000 7f20f6d9d640 1 --2- 172.21.15.76:0/319157879 >> v2:172.21.15.113:3300/0 conn(0x7f20f80c9b60 0x7f20f8070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.603+0000 7f20f6d9d640 1 -- 172.21.15.76:0/319157879 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f8066040 con 0x7f20f815ad40 2023-11-09T03:47:34.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.604+0000 7f20f6d9d640 1 --2- 172.21.15.76:0/319157879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20f815ad40 0x7f20f806b700 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f20f806e130 tx=0x7f20e802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8caa2e9e3f0eaf80 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:34.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.604+0000 7f20f659c640 1 -- 172.21.15.76:0/319157879 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8031870 con 0x7f20f815ad40 2023-11-09T03:47:34.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.604+0000 7f20f659c640 1 -- 172.21.15.76:0/319157879 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e80319d0 con 0x7f20f815ad40 2023-11-09T03:47:34.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.604+0000 7f20f659c640 1 -- 172.21.15.76:0/319157879 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8031d60 con 0x7f20f815ad40 2023-11-09T03:47:34.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.604+0000 7f20fd0e5640 1 -- 172.21.15.76:0/319157879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20f815ad40 msgr2=0x7f20f806b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:34.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.604+0000 7f20fd0e5640 1 --2- 172.21.15.76:0/319157879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20f815ad40 0x7f20f806b700 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f20f806e130 tx=0x7f20e802b910 comp rx=0 tx=0).stop 2023-11-09T03:47:34.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.604+0000 7f20fd0e5640 1 -- 172.21.15.76:0/319157879 shutdown_connections 2023-11-09T03:47:34.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.604+0000 7f20fd0e5640 1 --2- 172.21.15.76:0/319157879 >> v2:172.21.15.113:3300/0 conn(0x7f20f80c9b60 0x7f20f8070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.604+0000 7f20fd0e5640 1 --2- 172.21.15.76:0/319157879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20f815b120 0x7f20f815f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.604+0000 7f20fd0e5640 1 --2- 172.21.15.76:0/319157879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20f815ad40 0x7f20f806b700 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.604+0000 7f20fd0e5640 1 -- 172.21.15.76:0/319157879 >> 172.21.15.76:0/319157879 conn(0x7f20f80cbc90 msgr2=0x7f20f80cc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:34.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.604+0000 7f20fd0e5640 1 -- 172.21.15.76:0/319157879 shutdown_connections 2023-11-09T03:47:34.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.604+0000 7f20fd0e5640 1 -- 172.21.15.76:0/319157879 wait complete. 2023-11-09T03:47:34.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.606+0000 7f20fd0e5640 1 Processor -- start 2023-11-09T03:47:34.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.606+0000 7f20fd0e5640 1 -- start start 2023-11-09T03:47:34.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.607+0000 7f20fd0e5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20f80c9b60 0x7f20f80c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:34.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.607+0000 7f20fd0e5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20f815ad40 0x7f20f814df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:34.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.607+0000 7f20fd0e5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20f815b120 0x7f20f80c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:34.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.607+0000 7f20fd0e5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f80c6ef0 con 0x7f20f815b120 2023-11-09T03:47:34.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.607+0000 7f20fd0e5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f80c62e0 con 0x7f20f815ad40 2023-11-09T03:47:34.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.607+0000 7f20fd0e5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f80c65c0 con 0x7f20f80c9b60 2023-11-09T03:47:34.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.607+0000 7f20f759e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20f815b120 0x7f20f80c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:34.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.607+0000 7f20f759e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20f815b120 0x7f20f80c4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48164/0 (socket says 172.21.15.76:48164) 2023-11-09T03:47:34.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.607+0000 7f20f759e640 1 -- 172.21.15.76:0/2502859996 learned_addr learned my addr 172.21.15.76:0/2502859996 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:34.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.607+0000 7f20ee59c640 1 --2- 172.21.15.76:0/2502859996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20f815ad40 0x7f20f814df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:34.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.607+0000 7f20f6d9d640 1 --2- 172.21.15.76:0/2502859996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20f80c9b60 0x7f20f80c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:34.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.607+0000 7f20f759e640 1 -- 172.21.15.76:0/2502859996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20f80c9b60 msgr2=0x7f20f80c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:34.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.607+0000 7f20f759e640 1 --2- 172.21.15.76:0/2502859996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20f80c9b60 0x7f20f80c8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.607+0000 7f20f759e640 1 -- 172.21.15.76:0/2502859996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20f815ad40 msgr2=0x7f20f814df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:34.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.607+0000 7f20f759e640 1 --2- 172.21.15.76:0/2502859996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20f815ad40 0x7f20f814df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.607+0000 7f20f759e640 1 -- 172.21.15.76:0/2502859996 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e80094d0 con 0x7f20f815b120 2023-11-09T03:47:34.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.607+0000 7f20ee59c640 1 --2- 172.21.15.76:0/2502859996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20f815ad40 0x7f20f814df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:34.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.607+0000 7f20f6d9d640 1 --2- 172.21.15.76:0/2502859996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20f80c9b60 0x7f20f80c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:34.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.607+0000 7f20f759e640 1 --2- 172.21.15.76:0/2502859996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20f815b120 0x7f20f80c4c60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f20f000cb80 tx=0x7f20f000a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:34.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.607+0000 7f20effff640 1 -- 172.21.15.76:0/2502859996 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f000ce70 con 0x7f20f815b120 2023-11-09T03:47:34.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.607+0000 7f20effff640 1 -- 172.21.15.76:0/2502859996 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f000abd0 con 0x7f20f815b120 2023-11-09T03:47:34.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.607+0000 7f20fd0e5640 1 -- 172.21.15.76:0/2502859996 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f80c68a0 con 0x7f20f815b120 2023-11-09T03:47:34.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.607+0000 7f20effff640 1 -- 172.21.15.76:0/2502859996 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f0003990 con 0x7f20f815b120 2023-11-09T03:47:34.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.607+0000 7f20fd0e5640 1 -- 172.21.15.76:0/2502859996 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f814f0e0 con 0x7f20f815b120 2023-11-09T03:47:34.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.608+0000 7f20fd0e5640 1 -- 172.21.15.76:0/2502859996 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f80c70a0 con 0x7f20f815b120 2023-11-09T03:47:34.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.609+0000 7f20effff640 1 -- 172.21.15.76:0/2502859996 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f20f0011410 con 0x7f20f815b120 2023-11-09T03:47:34.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.609+0000 7f20effff640 1 --2- 172.21.15.76:0/2502859996 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f20d806ae70 0x7f20d806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:34.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.609+0000 7f20effff640 1 -- 172.21.15.76:0/2502859996 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f20f0089600 con 0x7f20f815b120 2023-11-09T03:47:34.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.610+0000 7f20f6d9d640 1 --2- 172.21.15.76:0/2502859996 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f20d806ae70 0x7f20d806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:34.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.610+0000 7f20f6d9d640 1 --2- 172.21.15.76:0/2502859996 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f20d806ae70 0x7f20d806d330 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f20e8036000 tx=0x7f20e803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:34.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.613+0000 7f20effff640 1 -- 172.21.15.76:0/2502859996 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f20f80c70a0 con 0x7f20f815b120 2023-11-09T03:47:34.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.780+0000 7f20fd0e5640 1 -- 172.21.15.76:0/2502859996 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f20f80c9840 con 0x7f20f815b120 2023-11-09T03:47:34.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.780+0000 7f20effff640 1 -- 172.21.15.76:0/2502859996 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v54) v1 ==== 103+0+11 (secure 0 0 0) 0x7f20f001e020 con 0x7f20f815b120 2023-11-09T03:47:34.780 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:47:34.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.782+0000 7f20fd0e5640 1 -- 172.21.15.76:0/2502859996 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f20d806ae70 msgr2=0x7f20d806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:34.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.782+0000 7f20fd0e5640 1 --2- 172.21.15.76:0/2502859996 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f20d806ae70 0x7f20d806d330 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f20e8036000 tx=0x7f20e803c040 comp rx=0 tx=0).stop 2023-11-09T03:47:34.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.782+0000 7f20fd0e5640 1 -- 172.21.15.76:0/2502859996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20f815b120 msgr2=0x7f20f80c4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:34.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.782+0000 7f20fd0e5640 1 --2- 172.21.15.76:0/2502859996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20f815b120 0x7f20f80c4c60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f20f000cb80 tx=0x7f20f000a4c0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.782+0000 7f20fd0e5640 1 -- 172.21.15.76:0/2502859996 shutdown_connections 2023-11-09T03:47:34.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.782+0000 7f20fd0e5640 1 --2- 172.21.15.76:0/2502859996 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f20d806ae70 0x7f20d806d330 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.782+0000 7f20fd0e5640 1 --2- 172.21.15.76:0/2502859996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20f815b120 0x7f20f80c4c60 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.782+0000 7f20fd0e5640 1 --2- 172.21.15.76:0/2502859996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20f815ad40 0x7f20f814df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.782+0000 7f20fd0e5640 1 --2- 172.21.15.76:0/2502859996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f20f80c9b60 0x7f20f80c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.782+0000 7f20fd0e5640 1 -- 172.21.15.76:0/2502859996 >> 172.21.15.76:0/2502859996 conn(0x7f20f80cbc90 msgr2=0x7f20f81552a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:34.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.783+0000 7f20fd0e5640 1 -- 172.21.15.76:0/2502859996 shutdown_connections 2023-11-09T03:47:34.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.783+0000 7f20fd0e5640 1 -- 172.21.15.76:0/2502859996 wait complete. 2023-11-09T03:47:34.793 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:47:34.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.896+0000 7f7047c51640 1 Processor -- start 2023-11-09T03:47:34.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.896+0000 7f7047c51640 1 -- start start 2023-11-09T03:47:34.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.897+0000 7f7047c51640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7040153850 0x7f704015a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:34.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.897+0000 7f7047c51640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f70400cc9d0 0x7f70400c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:34.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.897+0000 7f7047c51640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70400c3860 con 0x7f70400cc520 2023-11-09T03:47:34.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.897+0000 7f7047c51640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70400c9250 con 0x7f70400cc9d0 2023-11-09T03:47:34.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.897+0000 7f7047c51640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7040153c30 con 0x7f7040153850 2023-11-09T03:47:34.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.897+0000 7f70461ee640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f70400cc9d0 0x7f70400c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:34.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.897+0000 7f70451ec640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f70400cc520 0x7f7040070fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47216/0 (socket says 172.21.15.76:47216) 2023-11-09T03:47:34.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.897+0000 7f70451ec640 1 -- 172.21.15.76:0/2061946970 learned_addr learned my addr 172.21.15.76:0/2061946970 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:34.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.897+0000 7f70459ed640 1 --2- 172.21.15.76:0/2061946970 >> v2:172.21.15.113:3300/0 conn(0x7f7040153850 0x7f704015a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:34.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.897+0000 7f70461ee640 1 -- 172.21.15.76:0/2061946970 >> v2:172.21.15.113:3300/0 conn(0x7f7040153850 msgr2=0x7f704015a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:34.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.897+0000 7f70461ee640 1 --2- 172.21.15.76:0/2061946970 >> v2:172.21.15.113:3300/0 conn(0x7f7040153850 0x7f704015a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.897+0000 7f70461ee640 1 -- 172.21.15.76:0/2061946970 >> v1:172.21.15.76:6789/0 conn(0x7f70400cc520 legacy=0x7f7040070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:34.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.897+0000 7f70461ee640 1 -- 172.21.15.76:0/2061946970 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704015a700 con 0x7f70400cc9d0 2023-11-09T03:47:34.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.898+0000 7f70461ee640 1 --2- 172.21.15.76:0/2061946970 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70400cc9d0 0x7f70400c3cf0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f70400c5710 tx=0x7f703802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3383266100fe49 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:34.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.898+0000 7f70449eb640 1 -- 172.21.15.76:0/2061946970 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703802cc80 con 0x7f70400cc9d0 2023-11-09T03:47:34.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.898+0000 7f70449eb640 1 -- 172.21.15.76:0/2061946970 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703802cde0 con 0x7f70400cc9d0 2023-11-09T03:47:34.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.898+0000 7f70449eb640 1 -- 172.21.15.76:0/2061946970 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7038034660 con 0x7f70400cc9d0 2023-11-09T03:47:34.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.899+0000 7f7047c51640 1 -- 172.21.15.76:0/2061946970 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70400cc9d0 msgr2=0x7f70400c3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:34.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.899+0000 7f7047c51640 1 --2- 172.21.15.76:0/2061946970 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70400cc9d0 0x7f70400c3cf0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f70400c5710 tx=0x7f703802bed0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.899+0000 7f7047c51640 1 -- 172.21.15.76:0/2061946970 shutdown_connections 2023-11-09T03:47:34.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.899+0000 7f7047c51640 1 --2- 172.21.15.76:0/2061946970 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70400cc9d0 0x7f70400c3cf0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.899+0000 7f7047c51640 1 --2- 172.21.15.76:0/2061946970 >> v2:172.21.15.113:3300/0 conn(0x7f7040153850 0x7f704015a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.899+0000 7f7047c51640 1 -- 172.21.15.76:0/2061946970 >> 172.21.15.76:0/2061946970 conn(0x7f70400ce950 msgr2=0x7f70400ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:34.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.899+0000 7f7047c51640 1 -- 172.21.15.76:0/2061946970 shutdown_connections 2023-11-09T03:47:34.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.899+0000 7f7047c51640 1 -- 172.21.15.76:0/2061946970 wait complete. 2023-11-09T03:47:34.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.900+0000 7f7047c51640 1 Processor -- start 2023-11-09T03:47:34.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.900+0000 7f7047c51640 1 -- start start 2023-11-09T03:47:34.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.901+0000 7f7047c51640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70400cc520 0x7f70401472d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:34.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.901+0000 7f7047c51640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70400cc9d0 0x7f70400b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:34.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.901+0000 7f7047c51640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7040153850 0x7f70400b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:34.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.901+0000 7f7047c51640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70400b2d20 con 0x7f70400cc9d0 2023-11-09T03:47:34.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.901+0000 7f7047c51640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7040147810 con 0x7f7040153850 2023-11-09T03:47:34.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.901+0000 7f7047c51640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70400b20a0 con 0x7f70400cc520 2023-11-09T03:47:34.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.901+0000 7f70451ec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70400cc9d0 0x7f70400b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:34.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.901+0000 7f70451ec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70400cc9d0 0x7f70400b1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48176/0 (socket says 172.21.15.76:48176) 2023-11-09T03:47:34.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.901+0000 7f70451ec640 1 -- 172.21.15.76:0/2324201330 learned_addr learned my addr 172.21.15.76:0/2324201330 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:34.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.901+0000 7f70459ed640 1 --2- 172.21.15.76:0/2324201330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70400cc520 0x7f70401472d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:34.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.901+0000 7f70461ee640 1 --2- 172.21.15.76:0/2324201330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7040153850 0x7f70400b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:34.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.901+0000 7f70451ec640 1 -- 172.21.15.76:0/2324201330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70400cc520 msgr2=0x7f70401472d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:34.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.901+0000 7f70451ec640 1 --2- 172.21.15.76:0/2324201330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70400cc520 0x7f70401472d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.901+0000 7f70451ec640 1 -- 172.21.15.76:0/2324201330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7040153850 msgr2=0x7f70400b1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:34.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.901+0000 7f70451ec640 1 --2- 172.21.15.76:0/2324201330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7040153850 0x7f70400b1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:34.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.901+0000 7f70451ec640 1 -- 172.21.15.76:0/2324201330 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7038009a70 con 0x7f70400cc9d0 2023-11-09T03:47:34.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.902+0000 7f70451ec640 1 --2- 172.21.15.76:0/2324201330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70400cc9d0 0x7f70400b1530 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f703000c4a0 tx=0x7f703000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:34.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.902+0000 7f70367fc640 1 -- 172.21.15.76:0/2324201330 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7030005ce0 con 0x7f70400cc9d0 2023-11-09T03:47:34.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.902+0000 7f70367fc640 1 -- 172.21.15.76:0/2324201330 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7030005e40 con 0x7f70400cc9d0 2023-11-09T03:47:34.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.902+0000 7f70367fc640 1 -- 172.21.15.76:0/2324201330 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7030007270 con 0x7f70400cc9d0 2023-11-09T03:47:34.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.902+0000 7f7047c51640 1 -- 172.21.15.76:0/2324201330 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70400b2380 con 0x7f70400cc9d0 2023-11-09T03:47:34.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.902+0000 7f7047c51640 1 -- 172.21.15.76:0/2324201330 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70400b2940 con 0x7f70400cc9d0 2023-11-09T03:47:34.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.903+0000 7f7047c51640 1 -- 172.21.15.76:0/2324201330 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70400c4a20 con 0x7f70400cc9d0 2023-11-09T03:47:34.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.903+0000 7f70367fc640 1 -- 172.21.15.76:0/2324201330 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7030007490 con 0x7f70400cc9d0 2023-11-09T03:47:34.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.904+0000 7f70367fc640 1 --2- 172.21.15.76:0/2324201330 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f702406ae70 0x7f702406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:34.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.904+0000 7f70367fc640 1 -- 172.21.15.76:0/2324201330 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7030088cc0 con 0x7f70400cc9d0 2023-11-09T03:47:34.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.904+0000 7f70459ed640 1 --2- 172.21.15.76:0/2324201330 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f702406ae70 0x7f702406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:34.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.904+0000 7f70459ed640 1 --2- 172.21.15.76:0/2324201330 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f702406ae70 0x7f702406d330 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f703c00baf0 tx=0x7f703c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:34.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:34.908+0000 7f70367fc640 1 -- 172.21.15.76:0/2324201330 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7030053b80 con 0x7f70400cc9d0 2023-11-09T03:47:35.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.075+0000 7f7047c51640 1 -- 172.21.15.76:0/2324201330 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7040074290 con 0x7f70400cc9d0 2023-11-09T03:47:35.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.076+0000 7f70367fc640 1 -- 172.21.15.76:0/2324201330 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7f70300170a0 con 0x7f70400cc9d0 2023-11-09T03:47:35.075 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:47:35.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.078+0000 7f7047c51640 1 -- 172.21.15.76:0/2324201330 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f702406ae70 msgr2=0x7f702406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:35.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.078+0000 7f7047c51640 1 --2- 172.21.15.76:0/2324201330 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f702406ae70 0x7f702406d330 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f703c00baf0 tx=0x7f703c008040 comp rx=0 tx=0).stop 2023-11-09T03:47:35.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.078+0000 7f7047c51640 1 -- 172.21.15.76:0/2324201330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70400cc9d0 msgr2=0x7f70400b1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:35.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.078+0000 7f7047c51640 1 --2- 172.21.15.76:0/2324201330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70400cc9d0 0x7f70400b1530 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f703000c4a0 tx=0x7f703000c960 comp rx=0 tx=0).stop 2023-11-09T03:47:35.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.078+0000 7f7047c51640 1 -- 172.21.15.76:0/2324201330 shutdown_connections 2023-11-09T03:47:35.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.078+0000 7f7047c51640 1 --2- 172.21.15.76:0/2324201330 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f702406ae70 0x7f702406d330 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:35.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.078+0000 7f7047c51640 1 --2- 172.21.15.76:0/2324201330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7040153850 0x7f70400b1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:35.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.078+0000 7f7047c51640 1 --2- 172.21.15.76:0/2324201330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70400cc9d0 0x7f70400b1530 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:35.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.078+0000 7f7047c51640 1 --2- 172.21.15.76:0/2324201330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70400cc520 0x7f70401472d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:35.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.078+0000 7f7047c51640 1 -- 172.21.15.76:0/2324201330 >> 172.21.15.76:0/2324201330 conn(0x7f70400ce950 msgr2=0x7f7040070610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:35.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.078+0000 7f7047c51640 1 -- 172.21.15.76:0/2324201330 shutdown_connections 2023-11-09T03:47:35.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.078+0000 7f7047c51640 1 -- 172.21.15.76:0/2324201330 wait complete. 2023-11-09T03:47:35.088 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:47:35.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.191+0000 7f21a98e1640 1 Processor -- start 2023-11-09T03:47:35.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.191+0000 7f21a98e1640 1 -- start start 2023-11-09T03:47:35.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.192+0000 7f21a98e1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f21a415e3f0 0x7f21a40b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:35.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.192+0000 7f21a98e1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f21a40cc300 0x7f21a40b1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:35.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.192+0000 7f21a98e1640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21a40caa20 con 0x7f21a40ca0a0 2023-11-09T03:47:35.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.192+0000 7f21a98e1640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a40cb230 con 0x7f21a40cc300 2023-11-09T03:47:35.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.192+0000 7f21a98e1640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a40ca530 con 0x7f21a415e3f0 2023-11-09T03:47:35.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.192+0000 7f21a37fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f21a40cc300 0x7f21a40b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:35.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.192+0000 7f219bfff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f21a40ca0a0 0x7f21a40cc920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47224/0 (socket says 172.21.15.76:47224) 2023-11-09T03:47:35.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.192+0000 7f21a37fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f21a40cc300 0x7f21a40b1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48192/0 (socket says 172.21.15.76:48192) 2023-11-09T03:47:35.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.192+0000 7f219bfff640 1 -- 172.21.15.76:0/3618759920 learned_addr learned my addr 172.21.15.76:0/3618759920 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:35.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.192+0000 7f21a2ffd640 1 --2- 172.21.15.76:0/3618759920 >> v2:172.21.15.113:3300/0 conn(0x7f21a415e3f0 0x7f21a40b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:35.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.192+0000 7f21a37fe640 1 -- 172.21.15.76:0/3618759920 >> v2:172.21.15.113:3300/0 conn(0x7f21a415e3f0 msgr2=0x7f21a40b2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:35.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.192+0000 7f21a37fe640 1 --2- 172.21.15.76:0/3618759920 >> v2:172.21.15.113:3300/0 conn(0x7f21a415e3f0 0x7f21a40b2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:35.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.192+0000 7f21a37fe640 1 -- 172.21.15.76:0/3618759920 >> v1:172.21.15.76:6789/0 conn(0x7f21a40ca0a0 legacy=0x7f21a40cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:35.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.192+0000 7f21a37fe640 1 -- 172.21.15.76:0/3618759920 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a40cac70 con 0x7f21a40cc300 2023-11-09T03:47:35.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.192+0000 7f21a27fc640 1 -- 172.21.15.76:0/3618759920 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2408900362 0 0) 0x7f21a40caa20 con 0x7f21a40ca0a0 2023-11-09T03:47:35.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.193+0000 7f21a37fe640 1 --2- 172.21.15.76:0/3618759920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21a40cc300 0x7f21a40b1890 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f21a40b5bc0 tx=0x7f219c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d8cc14e11b2c23 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:35.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.193+0000 7f21a27fc640 1 -- 172.21.15.76:0/3618759920 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c02cc80 con 0x7f21a40cc300 2023-11-09T03:47:35.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.193+0000 7f21a27fc640 1 -- 172.21.15.76:0/3618759920 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219c02cde0 con 0x7f21a40cc300 2023-11-09T03:47:35.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.193+0000 7f21a27fc640 1 -- 172.21.15.76:0/3618759920 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c034660 con 0x7f21a40cc300 2023-11-09T03:47:35.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.193+0000 7f21a98e1640 1 -- 172.21.15.76:0/3618759920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21a40cc300 msgr2=0x7f21a40b1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:35.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.193+0000 7f21a98e1640 1 --2- 172.21.15.76:0/3618759920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21a40cc300 0x7f21a40b1890 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f21a40b5bc0 tx=0x7f219c02bed0 comp rx=0 tx=0).stop 2023-11-09T03:47:35.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.194+0000 7f21a98e1640 1 -- 172.21.15.76:0/3618759920 shutdown_connections 2023-11-09T03:47:35.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.194+0000 7f21a98e1640 1 --2- 172.21.15.76:0/3618759920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21a40cc300 0x7f21a40b1890 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:35.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.194+0000 7f21a98e1640 1 --2- 172.21.15.76:0/3618759920 >> v2:172.21.15.113:3300/0 conn(0x7f21a415e3f0 0x7f21a40b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:35.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.194+0000 7f21a98e1640 1 -- 172.21.15.76:0/3618759920 >> 172.21.15.76:0/3618759920 conn(0x7f21a4071820 msgr2=0x7f21a4071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:35.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.194+0000 7f21a98e1640 1 -- 172.21.15.76:0/3618759920 shutdown_connections 2023-11-09T03:47:35.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.194+0000 7f21a98e1640 1 -- 172.21.15.76:0/3618759920 wait complete. 2023-11-09T03:47:35.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.195+0000 7f21a98e1640 1 Processor -- start 2023-11-09T03:47:35.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.195+0000 7f21a98e1640 1 -- start start 2023-11-09T03:47:35.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.195+0000 7f21a98e1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21a40cc300 0x7f21a4159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:35.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.195+0000 7f21a98e1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21a415e3f0 0x7f21a40b3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:35.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.195+0000 7f21a98e1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21a41f4a90 0x7f21a41efcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:35.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.196+0000 7f21a98e1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a41f3da0 con 0x7f21a41f4a90 2023-11-09T03:47:35.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.196+0000 7f21a98e1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a4148970 con 0x7f21a415e3f0 2023-11-09T03:47:35.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.196+0000 7f21a98e1640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a415a130 con 0x7f21a40cc300 2023-11-09T03:47:35.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.196+0000 7f21a37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21a41f4a90 0x7f21a41efcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:35.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.196+0000 7f21a37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21a41f4a90 0x7f21a41efcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48194/0 (socket says 172.21.15.76:48194) 2023-11-09T03:47:35.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.196+0000 7f21a37fe640 1 -- 172.21.15.76:0/2521456330 learned_addr learned my addr 172.21.15.76:0/2521456330 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:35.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.196+0000 7f21a2ffd640 1 --2- 172.21.15.76:0/2521456330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21a40cc300 0x7f21a4159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:35.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.196+0000 7f219bfff640 1 --2- 172.21.15.76:0/2521456330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21a415e3f0 0x7f21a40b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:35.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.196+0000 7f21a37fe640 1 -- 172.21.15.76:0/2521456330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21a40cc300 msgr2=0x7f21a4159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:35.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.196+0000 7f21a37fe640 1 --2- 172.21.15.76:0/2521456330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21a40cc300 0x7f21a4159870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:35.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.196+0000 7f21a37fe640 1 -- 172.21.15.76:0/2521456330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21a415e3f0 msgr2=0x7f21a40b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:35.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.196+0000 7f21a37fe640 1 --2- 172.21.15.76:0/2521456330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21a415e3f0 0x7f21a40b3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:35.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.196+0000 7f21a37fe640 1 -- 172.21.15.76:0/2521456330 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219c009a70 con 0x7f21a41f4a90 2023-11-09T03:47:35.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.196+0000 7f219bfff640 1 --2- 172.21.15.76:0/2521456330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21a415e3f0 0x7f21a40b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:35.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.196+0000 7f21a37fe640 1 --2- 172.21.15.76:0/2521456330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21a41f4a90 0x7f21a41efcd0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f219c009e80 tx=0x7f219c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:35.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.196+0000 7f21a0ff9640 1 -- 172.21.15.76:0/2521456330 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c040650 con 0x7f21a41f4a90 2023-11-09T03:47:35.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.196+0000 7f21a0ff9640 1 -- 172.21.15.76:0/2521456330 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219c0407b0 con 0x7f21a41f4a90 2023-11-09T03:47:35.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.196+0000 7f21a98e1640 1 -- 172.21.15.76:0/2521456330 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a41f3fb0 con 0x7f21a41f4a90 2023-11-09T03:47:35.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.196+0000 7f21a0ff9640 1 -- 172.21.15.76:0/2521456330 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c040b60 con 0x7f21a41f4a90 2023-11-09T03:47:35.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.196+0000 7f21a98e1640 1 -- 172.21.15.76:0/2521456330 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a41f4570 con 0x7f21a41f4a90 2023-11-09T03:47:35.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.197+0000 7f21a98e1640 1 -- 172.21.15.76:0/2521456330 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21a4074290 con 0x7f21a41f4a90 2023-11-09T03:47:35.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.197+0000 7f21a0ff9640 1 -- 172.21.15.76:0/2521456330 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f219c040cc0 con 0x7f21a41f4a90 2023-11-09T03:47:35.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.198+0000 7f21a0ff9640 1 --2- 172.21.15.76:0/2521456330 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f218406ae70 0x7f218406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:35.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.198+0000 7f21a0ff9640 1 -- 172.21.15.76:0/2521456330 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f219c0ac5d0 con 0x7f21a41f4a90 2023-11-09T03:47:35.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.198+0000 7f21a2ffd640 1 --2- 172.21.15.76:0/2521456330 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f218406ae70 0x7f218406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:35.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.199+0000 7f21a2ffd640 1 --2- 172.21.15.76:0/2521456330 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f218406ae70 0x7f218406d330 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f21940079a0 tx=0x7f2194008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:35.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.202+0000 7f21a0ff9640 1 -- 172.21.15.76:0/2521456330 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f219c0773e0 con 0x7f21a41f4a90 2023-11-09T03:47:35.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.368+0000 7f21a98e1640 1 -- 172.21.15.76:0/2521456330 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f21a415adf0 con 0x7f21a41f4a90 2023-11-09T03:47:35.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.369+0000 7f21a0ff9640 1 -- 172.21.15.76:0/2521456330 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7f219c009450 con 0x7f21a41f4a90 2023-11-09T03:47:35.369 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:47:35.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.370+0000 7f21a98e1640 1 -- 172.21.15.76:0/2521456330 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f218406ae70 msgr2=0x7f218406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:35.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.370+0000 7f21a98e1640 1 --2- 172.21.15.76:0/2521456330 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f218406ae70 0x7f218406d330 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f21940079a0 tx=0x7f2194008040 comp rx=0 tx=0).stop 2023-11-09T03:47:35.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.370+0000 7f21a98e1640 1 -- 172.21.15.76:0/2521456330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21a41f4a90 msgr2=0x7f21a41efcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:35.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.370+0000 7f21a98e1640 1 --2- 172.21.15.76:0/2521456330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21a41f4a90 0x7f21a41efcd0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f219c009e80 tx=0x7f219c004a40 comp rx=0 tx=0).stop 2023-11-09T03:47:35.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.371+0000 7f21a98e1640 1 -- 172.21.15.76:0/2521456330 shutdown_connections 2023-11-09T03:47:35.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.371+0000 7f21a98e1640 1 --2- 172.21.15.76:0/2521456330 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f218406ae70 0x7f218406d330 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:35.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.371+0000 7f21a98e1640 1 --2- 172.21.15.76:0/2521456330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21a41f4a90 0x7f21a41efcd0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:35.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.371+0000 7f21a98e1640 1 --2- 172.21.15.76:0/2521456330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21a415e3f0 0x7f21a40b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:35.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.371+0000 7f21a98e1640 1 --2- 172.21.15.76:0/2521456330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21a40cc300 0x7f21a4159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:35.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.371+0000 7f21a98e1640 1 -- 172.21.15.76:0/2521456330 >> 172.21.15.76:0/2521456330 conn(0x7f21a4071820 msgr2=0x7f21a40cc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:35.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.371+0000 7f21a98e1640 1 -- 172.21.15.76:0/2521456330 shutdown_connections 2023-11-09T03:47:35.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.371+0000 7f21a98e1640 1 -- 172.21.15.76:0/2521456330 wait complete. 2023-11-09T03:47:35.381 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-09T03:47:35.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.484+0000 7f4d4cae5640 1 Processor -- start 2023-11-09T03:47:35.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.484+0000 7f4d4cae5640 1 -- start start 2023-11-09T03:47:35.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.485+0000 7f4d4cae5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4d480cc520 0x7f4d48070fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:35.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.485+0000 7f4d4cae5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4d480cc9d0 0x7f4d480714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:35.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.485+0000 7f4d4cae5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d480c3860 con 0x7f4d48153850 2023-11-09T03:47:35.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.485+0000 7f4d4cae5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d480c9250 con 0x7f4d480cc9d0 2023-11-09T03:47:35.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.485+0000 7f4d4cae5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d48153c30 con 0x7f4d480cc520 2023-11-09T03:47:35.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.485+0000 7f4d46d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4d480cc9d0 0x7f4d480714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:35.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.485+0000 7f4d46d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4d480cc9d0 0x7f4d480714e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48204/0 (socket says 172.21.15.76:48204) 2023-11-09T03:47:35.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.485+0000 7f4d46d9d640 1 -- 172.21.15.76:0/1672242214 learned_addr learned my addr 172.21.15.76:0/1672242214 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:35.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.485+0000 7f4d46d9d640 1 -- 172.21.15.76:0/1672242214 >> v2:172.21.15.113:3300/0 conn(0x7f4d480cc520 msgr2=0x7f4d48070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:35.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.485+0000 7f4d46d9d640 1 --2- 172.21.15.76:0/1672242214 >> v2:172.21.15.113:3300/0 conn(0x7f4d480cc520 0x7f4d48070fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:35.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.485+0000 7f4d46d9d640 1 -- 172.21.15.76:0/1672242214 >> v1:172.21.15.113:6789/0 conn(0x7f4d48153850 legacy=0x7f4d4815a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:35.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.485+0000 7f4d46d9d640 1 -- 172.21.15.76:0/1672242214 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d480ce590 con 0x7f4d480cc9d0 2023-11-09T03:47:35.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.486+0000 7f4d46d9d640 1 --2- 172.21.15.76:0/1672242214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d480cc9d0 0x7f4d480714e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f4d48070560 tx=0x7f4d4002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f305228e602eb41b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:35.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.486+0000 7f4d45d9b640 1 -- 172.21.15.76:0/1672242214 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4002ca80 con 0x7f4d480cc9d0 2023-11-09T03:47:35.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.486+0000 7f4d45d9b640 1 -- 172.21.15.76:0/1672242214 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d4002cbe0 con 0x7f4d480cc9d0 2023-11-09T03:47:35.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.486+0000 7f4d45d9b640 1 -- 172.21.15.76:0/1672242214 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d400343f0 con 0x7f4d480cc9d0 2023-11-09T03:47:35.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.486+0000 7f4d4cae5640 1 -- 172.21.15.76:0/1672242214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d480cc9d0 msgr2=0x7f4d480714e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:35.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.486+0000 7f4d4cae5640 1 --2- 172.21.15.76:0/1672242214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d480cc9d0 0x7f4d480714e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f4d48070560 tx=0x7f4d4002bee0 comp rx=0 tx=0).stop 2023-11-09T03:47:35.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.487+0000 7f4d4cae5640 1 -- 172.21.15.76:0/1672242214 shutdown_connections 2023-11-09T03:47:35.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.487+0000 7f4d4cae5640 1 --2- 172.21.15.76:0/1672242214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d480cc9d0 0x7f4d480714e0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:35.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.487+0000 7f4d4cae5640 1 --2- 172.21.15.76:0/1672242214 >> v2:172.21.15.113:3300/0 conn(0x7f4d480cc520 0x7f4d48070fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:35.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.487+0000 7f4d4cae5640 1 -- 172.21.15.76:0/1672242214 >> 172.21.15.76:0/1672242214 conn(0x7f4d480ce950 msgr2=0x7f4d480ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:35.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.487+0000 7f4d4cae5640 1 -- 172.21.15.76:0/1672242214 shutdown_connections 2023-11-09T03:47:35.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.487+0000 7f4d4cae5640 1 -- 172.21.15.76:0/1672242214 wait complete. 2023-11-09T03:47:35.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.488+0000 7f4d4cae5640 1 Processor -- start 2023-11-09T03:47:35.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.488+0000 7f4d4cae5640 1 -- start start 2023-11-09T03:47:35.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.489+0000 7f4d4cae5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d480cc520 0x7f4d4814d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:35.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.489+0000 7f4d4cae5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d480cc9d0 0x7f4d480b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:35.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.489+0000 7f4d4cae5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d48153850 0x7f4d4814a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:35.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.489+0000 7f4d4cae5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d480b1a70 con 0x7f4d480cc9d0 2023-11-09T03:47:35.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.489+0000 7f4d4cae5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d480b1d10 con 0x7f4d480cc520 2023-11-09T03:47:35.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.489+0000 7f4d4cae5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d480b1ff0 con 0x7f4d48153850 2023-11-09T03:47:35.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.489+0000 7f4d3ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d480cc9d0 0x7f4d480b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:35.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.489+0000 7f4d3ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d480cc9d0 0x7f4d480b1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48210/0 (socket says 172.21.15.76:48210) 2023-11-09T03:47:35.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.489+0000 7f4d3ffff640 1 -- 172.21.15.76:0/829777074 learned_addr learned my addr 172.21.15.76:0/829777074 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:35.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.489+0000 7f4d4659c640 1 --2- 172.21.15.76:0/829777074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d480cc520 0x7f4d4814d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:35.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.489+0000 7f4d46d9d640 1 --2- 172.21.15.76:0/829777074 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d48153850 0x7f4d4814a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:35.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.489+0000 7f4d3ffff640 1 -- 172.21.15.76:0/829777074 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d48153850 msgr2=0x7f4d4814a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:35.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.489+0000 7f4d3ffff640 1 --2- 172.21.15.76:0/829777074 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d48153850 0x7f4d4814a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:35.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.489+0000 7f4d3ffff640 1 -- 172.21.15.76:0/829777074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d480cc520 msgr2=0x7f4d4814d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:35.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.489+0000 7f4d3ffff640 1 --2- 172.21.15.76:0/829777074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d480cc520 0x7f4d4814d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:35.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.489+0000 7f4d3ffff640 1 -- 172.21.15.76:0/829777074 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d40009b30 con 0x7f4d480cc9d0 2023-11-09T03:47:35.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.489+0000 7f4d3ffff640 1 --2- 172.21.15.76:0/829777074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d480cc9d0 0x7f4d480b1530 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f4d3000b410 tx=0x7f4d3000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:35.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.489+0000 7f4d3effd640 1 -- 172.21.15.76:0/829777074 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d30011830 con 0x7f4d480cc9d0 2023-11-09T03:47:35.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.489+0000 7f4d3effd640 1 -- 172.21.15.76:0/829777074 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d300119b0 con 0x7f4d480cc9d0 2023-11-09T03:47:35.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.489+0000 7f4d4cae5640 1 -- 172.21.15.76:0/829777074 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d480b22d0 con 0x7f4d480cc9d0 2023-11-09T03:47:35.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.489+0000 7f4d3effd640 1 -- 172.21.15.76:0/829777074 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d30011d60 con 0x7f4d480cc9d0 2023-11-09T03:47:35.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.489+0000 7f4d4cae5640 1 -- 172.21.15.76:0/829777074 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d480b2890 con 0x7f4d480cc9d0 2023-11-09T03:47:35.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.490+0000 7f4d4cae5640 1 -- 172.21.15.76:0/829777074 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d480b2a80 con 0x7f4d480cc9d0 2023-11-09T03:47:35.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.491+0000 7f4d3effd640 1 -- 172.21.15.76:0/829777074 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4d3001a400 con 0x7f4d480cc9d0 2023-11-09T03:47:35.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.491+0000 7f4d3effd640 1 --2- 172.21.15.76:0/829777074 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d2806ae70 0x7f4d2806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:35.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.491+0000 7f4d3effd640 1 -- 172.21.15.76:0/829777074 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4d30089a30 con 0x7f4d480cc9d0 2023-11-09T03:47:35.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.491+0000 7f4d4659c640 1 --2- 172.21.15.76:0/829777074 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d2806ae70 0x7f4d2806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:35.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.492+0000 7f4d4659c640 1 --2- 172.21.15.76:0/829777074 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d2806ae70 0x7f4d2806d330 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4d38007fd0 tx=0x7f4d38009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:35.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.495+0000 7f4d3effd640 1 -- 172.21.15.76:0/829777074 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4d30054840 con 0x7f4d480cc9d0 2023-11-09T03:47:35.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.672+0000 7f4d4cae5640 1 -- 172.21.15.76:0/829777074 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4d48074290 con 0x7f4d480cc9d0 2023-11-09T03:47:35.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.672+0000 7f4d3effd640 1 -- 172.21.15.76:0/829777074 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4d30015020 con 0x7f4d480cc9d0 2023-11-09T03:47:35.672 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2023-11-09T03:47:35.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.674+0000 7f4d4cae5640 1 -- 172.21.15.76:0/829777074 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d2806ae70 msgr2=0x7f4d2806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:35.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.674+0000 7f4d4cae5640 1 --2- 172.21.15.76:0/829777074 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d2806ae70 0x7f4d2806d330 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4d38007fd0 tx=0x7f4d38009040 comp rx=0 tx=0).stop 2023-11-09T03:47:35.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.674+0000 7f4d4cae5640 1 -- 172.21.15.76:0/829777074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d480cc9d0 msgr2=0x7f4d480b1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:35.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.674+0000 7f4d4cae5640 1 --2- 172.21.15.76:0/829777074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d480cc9d0 0x7f4d480b1530 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f4d3000b410 tx=0x7f4d3000b8d0 comp rx=0 tx=0).stop 2023-11-09T03:47:35.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.674+0000 7f4d4cae5640 1 -- 172.21.15.76:0/829777074 shutdown_connections 2023-11-09T03:47:35.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.674+0000 7f4d4cae5640 1 --2- 172.21.15.76:0/829777074 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d2806ae70 0x7f4d2806d330 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:35.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.674+0000 7f4d4cae5640 1 --2- 172.21.15.76:0/829777074 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d48153850 0x7f4d4814a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:35.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.674+0000 7f4d4cae5640 1 --2- 172.21.15.76:0/829777074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d480cc9d0 0x7f4d480b1530 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:35.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.674+0000 7f4d4cae5640 1 --2- 172.21.15.76:0/829777074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d480cc520 0x7f4d4814d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:35.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.674+0000 7f4d4cae5640 1 -- 172.21.15.76:0/829777074 >> 172.21.15.76:0/829777074 conn(0x7f4d480ce950 msgr2=0x7f4d4806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:35.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.674+0000 7f4d4cae5640 1 -- 172.21.15.76:0/829777074 shutdown_connections 2023-11-09T03:47:35.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35.674+0000 7f4d4cae5640 1 -- 172.21.15.76:0/829777074 wait complete. 2023-11-09T03:47:35.684 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi076.front.sepia.ceph.com... 2023-11-09T03:47:35.685 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:47:35.685 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-09T03:47:35.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:47:35.745 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[107936]: fuse finished with error 0 and tester_r 0 2023-11-09T03:47:35.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:47:35.751 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:47:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:47:35.752 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T03:47:35.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:47:35.753 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-09T03:47:35.769 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:47:35.769 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:47:35.789 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2440 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-09T03:47:35.789 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2023-11-09T03:47:35.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:47:35.790 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-09T03:47:35.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:47:35.848 INFO:teuthology.orchestra.run.smithi113.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-09T03:47:35.848 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:47:35.849 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:47:35.864 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056381,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14761 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248868k,nr_inodes=812217,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-09T03:47:35.865 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T03:47:35.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:47:35.866 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-09T03:47:35.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:47:35.920 INFO:teuthology.orchestra.run.smithi076.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-09T03:47:35.921 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:47:35.921 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:47:35.941 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2440 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-09T03:47:35.942 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2023-11-09T03:47:35.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:47:35.943 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-09T03:47:35.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:47:36.000 INFO:teuthology.orchestra.run.smithi113.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-09T03:47:36.000 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:47:36.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.102+0000 7fb0f95a9640 1 Processor -- start 2023-11-09T03:47:36.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.102+0000 7fb0f95a9640 1 -- start start 2023-11-09T03:47:36.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.102+0000 7fb0f95a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0f40cde60 0x7fb0f415d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:36.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.102+0000 7fb0f95a9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb0f40ce240 0x7fb0f40d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:36.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.102+0000 7fb0f95a9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f415e000 con 0x7fb0f415daa0 2023-11-09T03:47:36.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.102+0000 7fb0f95a9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f415e5c0 con 0x7fb0f40ce240 2023-11-09T03:47:36.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.102+0000 7fb0f95a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f415e2e0 con 0x7fb0f40cde60 2023-11-09T03:47:36.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.102+0000 7fb0f37fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb0f40ce240 0x7fb0f40d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:36.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.103+0000 7fb0f37fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb0f40ce240 0x7fb0f40d1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48220/0 (socket says 172.21.15.76:48220) 2023-11-09T03:47:36.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.103+0000 7fb0f37fe640 1 -- 172.21.15.76:0/3814743816 learned_addr learned my addr 172.21.15.76:0/3814743816 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:36.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.103+0000 7fb0f2ffd640 1 --2- 172.21.15.76:0/3814743816 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0f40cde60 0x7fb0f415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:36.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.103+0000 7fb0f37fe640 1 -- 172.21.15.76:0/3814743816 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0f40cde60 msgr2=0x7fb0f415d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:36.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.103+0000 7fb0f37fe640 1 --2- 172.21.15.76:0/3814743816 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0f40cde60 0x7fb0f415d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.103+0000 7fb0f37fe640 1 -- 172.21.15.76:0/3814743816 >> v1:172.21.15.113:6789/0 conn(0x7fb0f415daa0 legacy=0x7fb0f4157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:36.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.103+0000 7fb0f37fe640 1 -- 172.21.15.76:0/3814743816 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f40d1120 con 0x7fb0f40ce240 2023-11-09T03:47:36.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.103+0000 7fb0f37fe640 1 --2- 172.21.15.76:0/3814743816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0f40ce240 0x7fb0f40d1920 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009870 tx=0x7fb0e402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95948eb24220fd4 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:36.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.104+0000 7fb0f1ffb640 1 -- 172.21.15.76:0/3814743816 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4031640 con 0x7fb0f40ce240 2023-11-09T03:47:36.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.104+0000 7fb0f1ffb640 1 -- 172.21.15.76:0/3814743816 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e40317a0 con 0x7fb0f40ce240 2023-11-09T03:47:36.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.104+0000 7fb0f1ffb640 1 -- 172.21.15.76:0/3814743816 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4031b30 con 0x7fb0f40ce240 2023-11-09T03:47:36.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.104+0000 7fb0f95a9640 1 -- 172.21.15.76:0/3814743816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0f40ce240 msgr2=0x7fb0f40d1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:36.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.104+0000 7fb0f95a9640 1 --2- 172.21.15.76:0/3814743816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0f40ce240 0x7fb0f40d1920 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009870 tx=0x7fb0e402b8f0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.104+0000 7fb0f95a9640 1 -- 172.21.15.76:0/3814743816 shutdown_connections 2023-11-09T03:47:36.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.104+0000 7fb0f95a9640 1 --2- 172.21.15.76:0/3814743816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0f40ce240 0x7fb0f40d1920 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.104+0000 7fb0f95a9640 1 --2- 172.21.15.76:0/3814743816 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0f40cde60 0x7fb0f415d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.104+0000 7fb0f95a9640 1 -- 172.21.15.76:0/3814743816 >> 172.21.15.76:0/3814743816 conn(0x7fb0f4071f30 msgr2=0x7fb0f40cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:36.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.104+0000 7fb0f95a9640 1 -- 172.21.15.76:0/3814743816 shutdown_connections 2023-11-09T03:47:36.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.104+0000 7fb0f95a9640 1 -- 172.21.15.76:0/3814743816 wait complete. 2023-11-09T03:47:36.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.106+0000 7fb0f95a9640 1 Processor -- start 2023-11-09T03:47:36.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.106+0000 7fb0f95a9640 1 -- start start 2023-11-09T03:47:36.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.106+0000 7fb0f95a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0f40cde60 0x7fb0f414e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:36.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.106+0000 7fb0f95a9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0f40ce240 0x7fb0f41511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:36.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.106+0000 7fb0f95a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0f415daa0 0x7fb0f4151710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:36.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.106+0000 7fb0f95a9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f4153eb0 con 0x7fb0f40ce240 2023-11-09T03:47:36.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.106+0000 7fb0f95a9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f41f5c70 con 0x7fb0f40cde60 2023-11-09T03:47:36.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.106+0000 7fb0f95a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f41f6510 con 0x7fb0f415daa0 2023-11-09T03:47:36.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.106+0000 7fb0f27fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0f40ce240 0x7fb0f41511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:36.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.106+0000 7fb0f27fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0f40ce240 0x7fb0f41511d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48224/0 (socket says 172.21.15.76:48224) 2023-11-09T03:47:36.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.106+0000 7fb0f27fc640 1 -- 172.21.15.76:0/1913307635 learned_addr learned my addr 172.21.15.76:0/1913307635 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:36.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.107+0000 7fb0f37fe640 1 --2- 172.21.15.76:0/1913307635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0f415daa0 0x7fb0f4151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:36.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.107+0000 7fb0f2ffd640 1 --2- 172.21.15.76:0/1913307635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0f40cde60 0x7fb0f414e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:36.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.107+0000 7fb0f27fc640 1 -- 172.21.15.76:0/1913307635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0f415daa0 msgr2=0x7fb0f4151710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:36.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.107+0000 7fb0f27fc640 1 --2- 172.21.15.76:0/1913307635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0f415daa0 0x7fb0f4151710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.107+0000 7fb0f27fc640 1 -- 172.21.15.76:0/1913307635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0f40cde60 msgr2=0x7fb0f414e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:36.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.107+0000 7fb0f27fc640 1 --2- 172.21.15.76:0/1913307635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0f40cde60 0x7fb0f414e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.107+0000 7fb0f27fc640 1 -- 172.21.15.76:0/1913307635 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e40094d0 con 0x7fb0f40ce240 2023-11-09T03:47:36.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.107+0000 7fb0f27fc640 1 --2- 172.21.15.76:0/1913307635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0f40ce240 0x7fb0f41511d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00b3f0 tx=0x7fb0dc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:36.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.107+0000 7fb0eb7fe640 1 -- 172.21.15.76:0/1913307635 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc011830 con 0x7fb0f40ce240 2023-11-09T03:47:36.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.107+0000 7fb0eb7fe640 1 -- 172.21.15.76:0/1913307635 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0dc011990 con 0x7fb0f40ce240 2023-11-09T03:47:36.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.107+0000 7fb0f95a9640 1 -- 172.21.15.76:0/1913307635 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f40cb260 con 0x7fb0f40ce240 2023-11-09T03:47:36.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.107+0000 7fb0eb7fe640 1 -- 172.21.15.76:0/1913307635 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc011d40 con 0x7fb0f40ce240 2023-11-09T03:47:36.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.107+0000 7fb0f95a9640 1 -- 172.21.15.76:0/1913307635 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f41f61a0 con 0x7fb0f40ce240 2023-11-09T03:47:36.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.108+0000 7fb0f95a9640 1 -- 172.21.15.76:0/1913307635 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f41f6360 con 0x7fb0f40ce240 2023-11-09T03:47:36.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.108+0000 7fb0eb7fe640 1 -- 172.21.15.76:0/1913307635 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb0dc01a400 con 0x7fb0f40ce240 2023-11-09T03:47:36.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.109+0000 7fb0eb7fe640 1 --2- 172.21.15.76:0/1913307635 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb0d406ae70 0x7fb0d406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:36.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.109+0000 7fb0eb7fe640 1 -- 172.21.15.76:0/1913307635 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb0dc089620 con 0x7fb0f40ce240 2023-11-09T03:47:36.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.109+0000 7fb0f2ffd640 1 --2- 172.21.15.76:0/1913307635 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb0d406ae70 0x7fb0d406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:36.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.109+0000 7fb0f2ffd640 1 --2- 172.21.15.76:0/1913307635 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb0d406ae70 0x7fb0d406d330 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009f30 tx=0x7fb0ec00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:36.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.113+0000 7fb0eb7fe640 1 -- 172.21.15.76:0/1913307635 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb0dc0544c0 con 0x7fb0f40ce240 2023-11-09T03:47:36.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.313+0000 7fb0f95a9640 1 -- 172.21.15.76:0/1913307635 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0f4074290 con 0x7fb0f40ce240 2023-11-09T03:47:36.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.314+0000 7fb0eb7fe640 1 -- 172.21.15.76:0/1913307635 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fb0dc057ae0 con 0x7fb0f40ce240 2023-11-09T03:47:36.314 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:36.315 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5099,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1844478483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1844478483},{"type":"v1","addr":"172.21.15.113:6836","nonce":1844478483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5102,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1111318612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1111318612},{"type":"v1","addr":"172.21.15.113:6837","nonce":1111318612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":6072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/2037805253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2037805253},{"type":"v1","addr":"172.21.15.76:6834","nonce":2037805253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:47:20.265190+0000","modified":"2023-11-09T03:47:22.742442+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6075},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6075":{"gid":6075,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/3710140065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":3710140065},{"type":"v1","addr":"172.21.15.76:6835","nonce":3710140065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-09T03:47:36.317 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 23 2023-11-09T03:47:36.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.316+0000 7fb0f95a9640 1 -- 172.21.15.76:0/1913307635 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb0d406ae70 msgr2=0x7fb0d406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:36.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.316+0000 7fb0f95a9640 1 --2- 172.21.15.76:0/1913307635 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb0d406ae70 0x7fb0d406d330 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009f30 tx=0x7fb0ec00b040 comp rx=0 tx=0).stop 2023-11-09T03:47:36.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.316+0000 7fb0f95a9640 1 -- 172.21.15.76:0/1913307635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0f40ce240 msgr2=0x7fb0f41511d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:36.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.316+0000 7fb0f95a9640 1 --2- 172.21.15.76:0/1913307635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0f40ce240 0x7fb0f41511d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00b3f0 tx=0x7fb0dc00b8b0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.316+0000 7fb0f95a9640 1 -- 172.21.15.76:0/1913307635 shutdown_connections 2023-11-09T03:47:36.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.316+0000 7fb0f95a9640 1 --2- 172.21.15.76:0/1913307635 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb0d406ae70 0x7fb0d406d330 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.316+0000 7fb0f95a9640 1 --2- 172.21.15.76:0/1913307635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0f415daa0 0x7fb0f4151710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.316+0000 7fb0f95a9640 1 --2- 172.21.15.76:0/1913307635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0f40ce240 0x7fb0f41511d0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.316+0000 7fb0f95a9640 1 --2- 172.21.15.76:0/1913307635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0f40cde60 0x7fb0f414e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.316+0000 7fb0f95a9640 1 -- 172.21.15.76:0/1913307635 >> 172.21.15.76:0/1913307635 conn(0x7fb0f4071f30 msgr2=0x7fb0f40b7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:36.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.316+0000 7fb0f95a9640 1 -- 172.21.15.76:0/1913307635 shutdown_connections 2023-11-09T03:47:36.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.316+0000 7fb0f95a9640 1 -- 172.21.15.76:0/1913307635 wait complete. 2023-11-09T03:47:36.326 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:47:36.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.427+0000 7f16430d4640 1 Processor -- start 2023-11-09T03:47:36.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.427+0000 7f16430d4640 1 -- start start 2023-11-09T03:47:36.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.428+0000 7f16430d4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f163c159c70 0x7f163c0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:36.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.428+0000 7f16430d4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f163c06db20 0x7f163c0b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:36.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.428+0000 7f16430d4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f163c0b0910 0x7f163c15d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:36.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.428+0000 7f16430d4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f163c0b49b0 con 0x7f163c159c70 2023-11-09T03:47:36.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.428+0000 7f16430d4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f163c0b0da0 con 0x7f163c06db20 2023-11-09T03:47:36.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.428+0000 7f16430d4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163c0b4110 con 0x7f163c0b0910 2023-11-09T03:47:36.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.428+0000 7f1640e70640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f163c159c70 0x7f163c0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:36.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.428+0000 7f1640e70640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f163c159c70 0x7f163c0ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48228/0 (socket says 172.21.15.76:48228) 2023-11-09T03:47:36.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.428+0000 7f1640e70640 1 -- 172.21.15.76:0/314125525 learned_addr learned my addr 172.21.15.76:0/314125525 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:36.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.428+0000 7f1640e70640 1 -- 172.21.15.76:0/314125525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f163c0b0910 msgr2=0x7f163c15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:36.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.428+0000 7f1640e70640 1 --2- 172.21.15.76:0/314125525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f163c0b0910 0x7f163c15d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.428+0000 7f1640e70640 1 -- 172.21.15.76:0/314125525 >> v2:172.21.15.113:3300/0 conn(0x7f163c06db20 msgr2=0x7f163c0b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:36.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.428+0000 7f1640e70640 1 --2- 172.21.15.76:0/314125525 >> v2:172.21.15.113:3300/0 conn(0x7f163c06db20 0x7f163c0b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.428+0000 7f1640e70640 1 -- 172.21.15.76:0/314125525 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163c0b43f0 con 0x7f163c159c70 2023-11-09T03:47:36.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.429+0000 7f1640e70640 1 --2- 172.21.15.76:0/314125525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f163c159c70 0x7f163c0ce370 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f163c0cf800 tx=0x7f162c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d1925ba07c8ed0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:36.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.429+0000 7f163b7fe640 1 -- 172.21.15.76:0/314125525 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c031870 con 0x7f163c159c70 2023-11-09T03:47:36.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.429+0000 7f163b7fe640 1 -- 172.21.15.76:0/314125525 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162c0319d0 con 0x7f163c159c70 2023-11-09T03:47:36.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.429+0000 7f163b7fe640 1 -- 172.21.15.76:0/314125525 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c031d60 con 0x7f163c159c70 2023-11-09T03:47:36.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.430+0000 7f16430d4640 1 -- 172.21.15.76:0/314125525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f163c159c70 msgr2=0x7f163c0ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:36.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.430+0000 7f16430d4640 1 --2- 172.21.15.76:0/314125525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f163c159c70 0x7f163c0ce370 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f163c0cf800 tx=0x7f162c02b910 comp rx=0 tx=0).stop 2023-11-09T03:47:36.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.430+0000 7f16430d4640 1 -- 172.21.15.76:0/314125525 shutdown_connections 2023-11-09T03:47:36.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.430+0000 7f16430d4640 1 --2- 172.21.15.76:0/314125525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f163c0b0910 0x7f163c15d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.430+0000 7f16430d4640 1 --2- 172.21.15.76:0/314125525 >> v2:172.21.15.113:3300/0 conn(0x7f163c06db20 0x7f163c0b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.430+0000 7f16430d4640 1 --2- 172.21.15.76:0/314125525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f163c159c70 0x7f163c0ce370 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.430+0000 7f16430d4640 1 -- 172.21.15.76:0/314125525 >> 172.21.15.76:0/314125525 conn(0x7f163c0ce950 msgr2=0x7f163c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:36.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.430+0000 7f16430d4640 1 -- 172.21.15.76:0/314125525 shutdown_connections 2023-11-09T03:47:36.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.430+0000 7f16430d4640 1 -- 172.21.15.76:0/314125525 wait complete. 2023-11-09T03:47:36.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.431+0000 7f16430d4640 1 Processor -- start 2023-11-09T03:47:36.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.431+0000 7f16430d4640 1 -- start start 2023-11-09T03:47:36.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.432+0000 7f16430d4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f163c06db20 0x7f163c14eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:36.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.432+0000 7f16430d4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f163c0b0910 0x7f163c0c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:36.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.432+0000 7f16430d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f163c159c70 0x7f163c151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:36.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.432+0000 7f16430d4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c150790 con 0x7f163c159c70 2023-11-09T03:47:36.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.432+0000 7f16430d4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c14f070 con 0x7f163c06db20 2023-11-09T03:47:36.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.432+0000 7f16430d4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163c0c8780 con 0x7f163c0b0910 2023-11-09T03:47:36.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.432+0000 7f1641671640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f163c159c70 0x7f163c151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:36.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.432+0000 7f1641671640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f163c159c70 0x7f163c151de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48242/0 (socket says 172.21.15.76:48242) 2023-11-09T03:47:36.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.432+0000 7f1641671640 1 -- 172.21.15.76:0/3642467224 learned_addr learned my addr 172.21.15.76:0/3642467224 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:36.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.432+0000 7f1640e70640 1 --2- 172.21.15.76:0/3642467224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f163c06db20 0x7f163c14eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:36.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.432+0000 7f163bfff640 1 --2- 172.21.15.76:0/3642467224 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f163c0b0910 0x7f163c0c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:36.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.432+0000 7f1641671640 1 -- 172.21.15.76:0/3642467224 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f163c0b0910 msgr2=0x7f163c0c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:36.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.432+0000 7f1641671640 1 --2- 172.21.15.76:0/3642467224 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f163c0b0910 0x7f163c0c8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.432+0000 7f1641671640 1 -- 172.21.15.76:0/3642467224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f163c06db20 msgr2=0x7f163c14eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:36.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.432+0000 7f1641671640 1 --2- 172.21.15.76:0/3642467224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f163c06db20 0x7f163c14eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.432+0000 7f1641671640 1 -- 172.21.15.76:0/3642467224 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f162c0094d0 con 0x7f163c159c70 2023-11-09T03:47:36.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.433+0000 7f1641671640 1 --2- 172.21.15.76:0/3642467224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f163c159c70 0x7f163c151de0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f163400b970 tx=0x7f163400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:36.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.433+0000 7f16397fa640 1 -- 172.21.15.76:0/3642467224 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163400cb40 con 0x7f163c159c70 2023-11-09T03:47:36.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.433+0000 7f16397fa640 1 -- 172.21.15.76:0/3642467224 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163400ccc0 con 0x7f163c159c70 2023-11-09T03:47:36.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.433+0000 7f16430d4640 1 -- 172.21.15.76:0/3642467224 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f163c156490 con 0x7f163c159c70 2023-11-09T03:47:36.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.433+0000 7f16397fa640 1 -- 172.21.15.76:0/3642467224 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1634014680 con 0x7f163c159c70 2023-11-09T03:47:36.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.433+0000 7f16430d4640 1 -- 172.21.15.76:0/3642467224 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f163c150cc0 con 0x7f163c159c70 2023-11-09T03:47:36.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.434+0000 7f16430d4640 1 -- 172.21.15.76:0/3642467224 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f163c074290 con 0x7f163c159c70 2023-11-09T03:47:36.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.434+0000 7f16397fa640 1 -- 172.21.15.76:0/3642467224 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f163401d3b0 con 0x7f163c159c70 2023-11-09T03:47:36.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.435+0000 7f16397fa640 1 --2- 172.21.15.76:0/3642467224 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f162806ae70 0x7f162806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:36.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.435+0000 7f16397fa640 1 -- 172.21.15.76:0/3642467224 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f16340897d0 con 0x7f163c159c70 2023-11-09T03:47:36.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.435+0000 7f1640e70640 1 --2- 172.21.15.76:0/3642467224 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f162806ae70 0x7f162806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:36.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.436+0000 7f1640e70640 1 --2- 172.21.15.76:0/3642467224 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f162806ae70 0x7f162806d330 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f162c036000 tx=0x7f162c03c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:36.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.439+0000 7f16397fa640 1 -- 172.21.15.76:0/3642467224 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1634054670 con 0x7f163c159c70 2023-11-09T03:47:36.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.608+0000 7f16430d4640 1 -- 172.21.15.76:0/3642467224 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f163c06d3f0 con 0x7f163c159c70 2023-11-09T03:47:36.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.609+0000 7f16397fa640 1 -- 172.21.15.76:0/3642467224 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+18511 (secure 0 0 0) 0x7f1634019090 con 0x7f163c159c70 2023-11-09T03:47:36.609 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:36.610 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":54,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:47:21.936243+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:47:18.807110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-09T03:47:19.235022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:47:36.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.611+0000 7f16430d4640 1 -- 172.21.15.76:0/3642467224 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f162806ae70 msgr2=0x7f162806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:36.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.611+0000 7f16430d4640 1 --2- 172.21.15.76:0/3642467224 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f162806ae70 0x7f162806d330 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f162c036000 tx=0x7f162c03c040 comp rx=0 tx=0).stop 2023-11-09T03:47:36.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.611+0000 7f16430d4640 1 -- 172.21.15.76:0/3642467224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f163c159c70 msgr2=0x7f163c151de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:36.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.611+0000 7f16430d4640 1 --2- 172.21.15.76:0/3642467224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f163c159c70 0x7f163c151de0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f163400b970 tx=0x7f163400be30 comp rx=0 tx=0).stop 2023-11-09T03:47:36.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.611+0000 7f16430d4640 1 -- 172.21.15.76:0/3642467224 shutdown_connections 2023-11-09T03:47:36.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.611+0000 7f16430d4640 1 --2- 172.21.15.76:0/3642467224 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f162806ae70 0x7f162806d330 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.611+0000 7f16430d4640 1 --2- 172.21.15.76:0/3642467224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f163c159c70 0x7f163c151de0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.611+0000 7f16430d4640 1 --2- 172.21.15.76:0/3642467224 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f163c0b0910 0x7f163c0c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.611+0000 7f16430d4640 1 --2- 172.21.15.76:0/3642467224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f163c06db20 0x7f163c14eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.611+0000 7f16430d4640 1 -- 172.21.15.76:0/3642467224 >> 172.21.15.76:0/3642467224 conn(0x7f163c0ce950 msgr2=0x7f163c15a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:36.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.611+0000 7f16430d4640 1 -- 172.21.15.76:0/3642467224 shutdown_connections 2023-11-09T03:47:36.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.611+0000 7f16430d4640 1 -- 172.21.15.76:0/3642467224 wait complete. 2023-11-09T03:47:36.623 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-09T03:47:00.036540+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:47:18.807110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:47:19.235022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-09T03:47:36.624 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:47:36.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.728+0000 7fb5e3bfc640 1 Processor -- start 2023-11-09T03:47:36.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.728+0000 7fb5e3bfc640 1 -- start start 2023-11-09T03:47:36.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.729+0000 7fb5e3bfc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb5dc0c4ff0 0x7fb5dc0cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:36.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.729+0000 7fb5e3bfc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5dc0adac0 0x7fb5dc0adf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:36.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.729+0000 7fb5e3bfc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5dc15aff0 con 0x7fb5dc159c00 2023-11-09T03:47:36.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.729+0000 7fb5e3bfc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5dc0ae460 con 0x7fb5dc0c4ff0 2023-11-09T03:47:36.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.729+0000 7fb5e3bfc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc15a3e0 con 0x7fb5dc0adac0 2023-11-09T03:47:36.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.729+0000 7fb5e1998640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb5dc159c00 0x7fb5dc0c48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47238/0 (socket says 172.21.15.76:47238) 2023-11-09T03:47:36.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.729+0000 7fb5e1197640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb5dc0c4ff0 0x7fb5dc0cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:36.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.729+0000 7fb5e1998640 1 -- 172.21.15.76:0/1244093249 learned_addr learned my addr 172.21.15.76:0/1244093249 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:36.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.730+0000 7fb5e2199640 1 --2- 172.21.15.76:0/1244093249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5dc0adac0 0x7fb5dc0adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:36.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.730+0000 7fb5e1197640 1 -- 172.21.15.76:0/1244093249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5dc0adac0 msgr2=0x7fb5dc0adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:36.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.730+0000 7fb5e1197640 1 --2- 172.21.15.76:0/1244093249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5dc0adac0 0x7fb5dc0adf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.730+0000 7fb5e1197640 1 -- 172.21.15.76:0/1244093249 >> v1:172.21.15.76:6789/0 conn(0x7fb5dc159c00 legacy=0x7fb5dc0c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:36.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.730+0000 7fb5e0996640 1 -- 172.21.15.76:0/1244093249 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 40264052 0 0) 0x7fb5dc15aff0 con 0x7fb5dc159c00 2023-11-09T03:47:36.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.730+0000 7fb5e1197640 1 -- 172.21.15.76:0/1244093249 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5dc15a6c0 con 0x7fb5dc0c4ff0 2023-11-09T03:47:36.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.730+0000 7fb5e2199640 1 --2- 172.21.15.76:0/1244093249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5dc0adac0 0x7fb5dc0adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:36.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.730+0000 7fb5e1197640 1 --2- 172.21.15.76:0/1244093249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5dc0c4ff0 0x7fb5dc0cc890 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc0097a0 tx=0x7fb5cc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa9a6f8177ef3a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:36.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.730+0000 7fb5e0996640 1 -- 172.21.15.76:0/1244093249 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc031830 con 0x7fb5dc0c4ff0 2023-11-09T03:47:36.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.730+0000 7fb5e0996640 1 -- 172.21.15.76:0/1244093249 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5cc031990 con 0x7fb5dc0c4ff0 2023-11-09T03:47:36.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.731+0000 7fb5e0996640 1 -- 172.21.15.76:0/1244093249 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc031d20 con 0x7fb5dc0c4ff0 2023-11-09T03:47:36.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.731+0000 7fb5e3bfc640 1 -- 172.21.15.76:0/1244093249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5dc0c4ff0 msgr2=0x7fb5dc0cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:36.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.731+0000 7fb5e3bfc640 1 --2- 172.21.15.76:0/1244093249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5dc0c4ff0 0x7fb5dc0cc890 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc0097a0 tx=0x7fb5cc02b8d0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.731+0000 7fb5e3bfc640 1 -- 172.21.15.76:0/1244093249 shutdown_connections 2023-11-09T03:47:36.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.731+0000 7fb5e3bfc640 1 --2- 172.21.15.76:0/1244093249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5dc0adac0 0x7fb5dc0adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.731+0000 7fb5e3bfc640 1 --2- 172.21.15.76:0/1244093249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5dc0c4ff0 0x7fb5dc0cc890 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.731+0000 7fb5e3bfc640 1 -- 172.21.15.76:0/1244093249 >> 172.21.15.76:0/1244093249 conn(0x7fb5dc0ce950 msgr2=0x7fb5dc0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:36.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.731+0000 7fb5e3bfc640 1 -- 172.21.15.76:0/1244093249 shutdown_connections 2023-11-09T03:47:36.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.731+0000 7fb5e3bfc640 1 -- 172.21.15.76:0/1244093249 wait complete. 2023-11-09T03:47:36.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.732+0000 7fb5e3bfc640 1 Processor -- start 2023-11-09T03:47:36.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.732+0000 7fb5e3bfc640 1 -- start start 2023-11-09T03:47:36.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.733+0000 7fb5e3bfc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5dc0adac0 0x7fb5dc1493b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:36.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.733+0000 7fb5e3bfc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5dc0c4ff0 0x7fb5dc14bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:36.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.733+0000 7fb5e3bfc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5dc159c00 0x7fb5dc06c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:36.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.733+0000 7fb5e3bfc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc14e0b0 con 0x7fb5dc0c4ff0 2023-11-09T03:47:36.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.733+0000 7fb5e3bfc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc14d4a0 con 0x7fb5dc159c00 2023-11-09T03:47:36.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.733+0000 7fb5e3bfc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc14d780 con 0x7fb5dc0adac0 2023-11-09T03:47:36.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.733+0000 7fb5e1197640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5dc0c4ff0 0x7fb5dc14bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:36.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.733+0000 7fb5e1197640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5dc0c4ff0 0x7fb5dc14bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48264/0 (socket says 172.21.15.76:48264) 2023-11-09T03:47:36.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.733+0000 7fb5e1197640 1 -- 172.21.15.76:0/2066036998 learned_addr learned my addr 172.21.15.76:0/2066036998 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:36.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.733+0000 7fb5e1998640 1 --2- 172.21.15.76:0/2066036998 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5dc0adac0 0x7fb5dc1493b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:36.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.733+0000 7fb5e2199640 1 --2- 172.21.15.76:0/2066036998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5dc159c00 0x7fb5dc06c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:36.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.733+0000 7fb5e1197640 1 -- 172.21.15.76:0/2066036998 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5dc0adac0 msgr2=0x7fb5dc1493b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:36.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.733+0000 7fb5e1197640 1 --2- 172.21.15.76:0/2066036998 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5dc0adac0 0x7fb5dc1493b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.733+0000 7fb5e1197640 1 -- 172.21.15.76:0/2066036998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5dc159c00 msgr2=0x7fb5dc06c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:36.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.733+0000 7fb5e1197640 1 --2- 172.21.15.76:0/2066036998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5dc159c00 0x7fb5dc06c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.733+0000 7fb5e1197640 1 -- 172.21.15.76:0/2066036998 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5cc009400 con 0x7fb5dc0c4ff0 2023-11-09T03:47:36.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.733+0000 7fb5e1998640 1 --2- 172.21.15.76:0/2066036998 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5dc0adac0 0x7fb5dc1493b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:36.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.734+0000 7fb5e1197640 1 --2- 172.21.15.76:0/2066036998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5dc0c4ff0 0x7fb5dc14bc80 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc000c00 tx=0x7fb5cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:36.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.734+0000 7fb5d27fc640 1 -- 172.21.15.76:0/2066036998 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc004230 con 0x7fb5dc0c4ff0 2023-11-09T03:47:36.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.734+0000 7fb5d27fc640 1 -- 172.21.15.76:0/2066036998 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5cc004390 con 0x7fb5dc0c4ff0 2023-11-09T03:47:36.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.734+0000 7fb5e3bfc640 1 -- 172.21.15.76:0/2066036998 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5dc14da60 con 0x7fb5dc0c4ff0 2023-11-09T03:47:36.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.734+0000 7fb5d27fc640 1 -- 172.21.15.76:0/2066036998 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc03a920 con 0x7fb5dc0c4ff0 2023-11-09T03:47:36.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.734+0000 7fb5e3bfc640 1 -- 172.21.15.76:0/2066036998 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5dc14c1c0 con 0x7fb5dc0c4ff0 2023-11-09T03:47:36.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.735+0000 7fb5e3bfc640 1 -- 172.21.15.76:0/2066036998 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5dc074290 con 0x7fb5dc0c4ff0 2023-11-09T03:47:36.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.735+0000 7fb5d27fc640 1 -- 172.21.15.76:0/2066036998 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb5cc03aa80 con 0x7fb5dc0c4ff0 2023-11-09T03:47:36.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.735+0000 7fb5d27fc640 1 --2- 172.21.15.76:0/2066036998 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb5c006ae70 0x7fb5c006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:36.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.736+0000 7fb5d27fc640 1 -- 172.21.15.76:0/2066036998 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb5cc0ac040 con 0x7fb5dc0c4ff0 2023-11-09T03:47:36.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.736+0000 7fb5e1998640 1 --2- 172.21.15.76:0/2066036998 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb5c006ae70 0x7fb5c006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:36.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.736+0000 7fb5e1998640 1 --2- 172.21.15.76:0/2066036998 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb5c006ae70 0x7fb5c006d330 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8007fd0 tx=0x7fb5d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:36.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.740+0000 7fb5d27fc640 1 -- 172.21.15.76:0/2066036998 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb5cc076e70 con 0x7fb5dc0c4ff0 2023-11-09T03:47:36.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.906+0000 7fb5e3bfc640 1 -- 172.21.15.76:0/2066036998 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb5dc0b45a0 con 0x7fb5dc0c4ff0 2023-11-09T03:47:36.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.906+0000 7fb5d27fc640 1 -- 172.21.15.76:0/2066036998 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb5cc035020 con 0x7fb5dc0c4ff0 2023-11-09T03:47:36.906 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:47:36.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.908+0000 7fb5e3bfc640 1 -- 172.21.15.76:0/2066036998 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb5c006ae70 msgr2=0x7fb5c006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:36.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.908+0000 7fb5e3bfc640 1 --2- 172.21.15.76:0/2066036998 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb5c006ae70 0x7fb5c006d330 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8007fd0 tx=0x7fb5d8009040 comp rx=0 tx=0).stop 2023-11-09T03:47:36.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.908+0000 7fb5e3bfc640 1 -- 172.21.15.76:0/2066036998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5dc0c4ff0 msgr2=0x7fb5dc14bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:36.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.908+0000 7fb5e3bfc640 1 --2- 172.21.15.76:0/2066036998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5dc0c4ff0 0x7fb5dc14bc80 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc000c00 tx=0x7fb5cc0046c0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.908+0000 7fb5e3bfc640 1 -- 172.21.15.76:0/2066036998 shutdown_connections 2023-11-09T03:47:36.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.908+0000 7fb5e3bfc640 1 --2- 172.21.15.76:0/2066036998 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb5c006ae70 0x7fb5c006d330 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.908+0000 7fb5e3bfc640 1 --2- 172.21.15.76:0/2066036998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5dc159c00 0x7fb5dc06c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.908+0000 7fb5e3bfc640 1 --2- 172.21.15.76:0/2066036998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5dc0c4ff0 0x7fb5dc14bc80 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.908+0000 7fb5e3bfc640 1 --2- 172.21.15.76:0/2066036998 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5dc0adac0 0x7fb5dc1493b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:36.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.908+0000 7fb5e3bfc640 1 -- 172.21.15.76:0/2066036998 >> 172.21.15.76:0/2066036998 conn(0x7fb5dc0ce950 msgr2=0x7fb5dc06dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:36.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.908+0000 7fb5e3bfc640 1 -- 172.21.15.76:0/2066036998 shutdown_connections 2023-11-09T03:47:36.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:36.908+0000 7fb5e3bfc640 1 -- 172.21.15.76:0/2066036998 wait complete. 2023-11-09T03:47:36.918 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-09T03:47:37.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.024+0000 7f237f590640 1 Processor -- start 2023-11-09T03:47:37.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.024+0000 7f237f590640 1 -- start start 2023-11-09T03:47:37.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.025+0000 7f237f590640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f23780ae1d0 0x7f23780cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:37.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.025+0000 7f237f590640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f237815aff0 con 0x7f23780c4ff0 2023-11-09T03:47:37.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.025+0000 7f237f590640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f237815a3e0 con 0x7f2378159c00 2023-11-09T03:47:37.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.025+0000 7f237f590640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f237815a6c0 con 0x7f23780ae1d0 2023-11-09T03:47:37.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.025+0000 7f237cb2b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f23780c4ff0 0x7f23780adac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47244/0 (socket says 172.21.15.76:47244) 2023-11-09T03:47:37.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.025+0000 7f237cb2b640 1 -- 172.21.15.76:0/1031638367 learned_addr learned my addr 172.21.15.76:0/1031638367 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:37.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.025+0000 7f237db2d640 1 --2- 172.21.15.76:0/1031638367 >> v2:172.21.15.113:3300/0 conn(0x7f23780ae1d0 0x7f23780cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:37.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.025+0000 7f236ffff640 1 -- 172.21.15.76:0/1031638367 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1621649941 0 0) 0x7f237815aff0 con 0x7f23780c4ff0 2023-11-09T03:47:37.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.025+0000 7f236ffff640 1 -- 172.21.15.76:0/1031638367 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2364003520 con 0x7f23780c4ff0 2023-11-09T03:47:37.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.025+0000 7f236ffff640 1 -- 172.21.15.76:0/1031638367 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3848853744 0 0) 0x7f2364003520 con 0x7f23780c4ff0 2023-11-09T03:47:37.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.025+0000 7f236ffff640 1 -- 172.21.15.76:0/1031638367 >> v2:172.21.15.113:3300/0 conn(0x7f23780ae1d0 msgr2=0x7f23780cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:37.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.025+0000 7f236ffff640 1 --2- 172.21.15.76:0/1031638367 >> v2:172.21.15.113:3300/0 conn(0x7f23780ae1d0 0x7f23780cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.025+0000 7f236ffff640 1 -- 172.21.15.76:0/1031638367 >> v1:172.21.15.113:6789/0 conn(0x7f2378159c00 legacy=0x7f23780c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:37.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.025+0000 7f236ffff640 1 -- 172.21.15.76:0/1031638367 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237815a9a0 con 0x7f23780c4ff0 2023-11-09T03:47:37.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.025+0000 7f236ffff640 1 -- 172.21.15.76:0/1031638367 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f2368001670 con 0x7f23780c4ff0 2023-11-09T03:47:37.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.025+0000 7f236ffff640 1 -- 172.21.15.76:0/1031638367 >> v1:172.21.15.76:6789/0 conn(0x7f23780c4ff0 legacy=0x7f23780adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:37.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.025+0000 7f236ffff640 1 --2- 172.21.15.76:0/1031638367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2364003970 0x7f2364003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:37.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.025+0000 7f236ffff640 1 --2- 172.21.15.76:0/1031638367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23640047c0 0x7f2364004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:37.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.025+0000 7f236ffff640 1 --2- 172.21.15.76:0/1031638367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2364005180 0x7f236400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:37.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.025+0000 7f236ffff640 1 -- 172.21.15.76:0/1031638367 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237815a6c0 con 0x7f23640047c0 2023-11-09T03:47:37.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.025+0000 7f236ffff640 1 -- 172.21.15.76:0/1031638367 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236400c020 con 0x7f2364005180 2023-11-09T03:47:37.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.025+0000 7f236ffff640 1 -- 172.21.15.76:0/1031638367 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236400c1e0 con 0x7f2364003970 2023-11-09T03:47:37.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.026+0000 7f237cb2b640 1 --2- 172.21.15.76:0/1031638367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23640047c0 0x7f2364004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:37.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.026+0000 7f237d32c640 1 --2- 172.21.15.76:0/1031638367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2364003970 0x7f2364003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:37.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.026+0000 7f237db2d640 1 --2- 172.21.15.76:0/1031638367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2364005180 0x7f236400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:37.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.026+0000 7f237cb2b640 1 -- 172.21.15.76:0/1031638367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2364003970 msgr2=0x7f2364003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:37.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.026+0000 7f237cb2b640 1 --2- 172.21.15.76:0/1031638367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2364003970 0x7f2364003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.026+0000 7f237cb2b640 1 -- 172.21.15.76:0/1031638367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2364005180 msgr2=0x7f236400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:37.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.026+0000 7f237cb2b640 1 --2- 172.21.15.76:0/1031638367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2364005180 0x7f236400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.026+0000 7f237cb2b640 1 -- 172.21.15.76:0/1031638367 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2364003520 con 0x7f23640047c0 2023-11-09T03:47:37.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.026+0000 7f237db2d640 1 --2- 172.21.15.76:0/1031638367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2364005180 0x7f236400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:37.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.026+0000 7f237db2d640 1 -- 172.21.15.76:0/1031638367 reap_dead start 2023-11-09T03:47:37.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.027+0000 7f237cb2b640 1 --2- 172.21.15.76:0/1031638367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23640047c0 0x7f2364004c40 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f23780b4a60 tx=0x7f23640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92938651dca0f8e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:37.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.027+0000 7f236ffff640 1 -- 172.21.15.76:0/1031638367 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2368005ad0 con 0x7f23640047c0 2023-11-09T03:47:37.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.027+0000 7f236ffff640 1 -- 172.21.15.76:0/1031638367 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2368005c30 con 0x7f23640047c0 2023-11-09T03:47:37.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.027+0000 7f236ffff640 1 -- 172.21.15.76:0/1031638367 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236802c810 con 0x7f23640047c0 2023-11-09T03:47:37.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.027+0000 7f237f590640 1 -- 172.21.15.76:0/1031638367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23640047c0 msgr2=0x7f2364004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:37.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.027+0000 7f237f590640 1 --2- 172.21.15.76:0/1031638367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23640047c0 0x7f2364004c40 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f23780b4a60 tx=0x7f23640057f0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.027+0000 7f237f590640 1 -- 172.21.15.76:0/1031638367 shutdown_connections 2023-11-09T03:47:37.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.027+0000 7f237f590640 1 --2- 172.21.15.76:0/1031638367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23640047c0 0x7f2364004c40 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.027+0000 7f237f590640 1 -- 172.21.15.76:0/1031638367 >> 172.21.15.76:0/1031638367 conn(0x7f23780ce950 msgr2=0x7f23780ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:37.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.027+0000 7f237f590640 1 -- 172.21.15.76:0/1031638367 shutdown_connections 2023-11-09T03:47:37.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.027+0000 7f237f590640 1 -- 172.21.15.76:0/1031638367 wait complete. 2023-11-09T03:47:37.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.028+0000 7f237f590640 1 Processor -- start 2023-11-09T03:47:37.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.029+0000 7f237f590640 1 -- start start 2023-11-09T03:47:37.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.029+0000 7f237f590640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23640047c0 0x7f2378146e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:37.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.029+0000 7f237f590640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23781499f0 0x7f23780c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:37.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.029+0000 7f237f590640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23781e4630 0x7f23781e4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:37.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.029+0000 7f237f590640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23780c5a20 con 0x7f23781499f0 2023-11-09T03:47:37.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.029+0000 7f237f590640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23781e4ff0 con 0x7f23640047c0 2023-11-09T03:47:37.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.029+0000 7f237f590640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2378147380 con 0x7f23781e4630 2023-11-09T03:47:37.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.029+0000 7f237cb2b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23781499f0 0x7f23780c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:37.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.029+0000 7f237cb2b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23781499f0 0x7f23780c82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48284/0 (socket says 172.21.15.76:48284) 2023-11-09T03:47:37.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.029+0000 7f237cb2b640 1 -- 172.21.15.76:0/2474246911 learned_addr learned my addr 172.21.15.76:0/2474246911 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:37.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.029+0000 7f237d32c640 1 --2- 172.21.15.76:0/2474246911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23640047c0 0x7f2378146e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:37.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.029+0000 7f237db2d640 1 --2- 172.21.15.76:0/2474246911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23781e4630 0x7f23781e4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:37.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.030+0000 7f237cb2b640 1 -- 172.21.15.76:0/2474246911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23781e4630 msgr2=0x7f23781e4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:37.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.030+0000 7f237cb2b640 1 --2- 172.21.15.76:0/2474246911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23781e4630 0x7f23781e4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.030+0000 7f237cb2b640 1 -- 172.21.15.76:0/2474246911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23640047c0 msgr2=0x7f2378146e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:37.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.030+0000 7f237cb2b640 1 --2- 172.21.15.76:0/2474246911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23640047c0 0x7f2378146e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.030+0000 7f237cb2b640 1 -- 172.21.15.76:0/2474246911 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23781481c0 con 0x7f23781499f0 2023-11-09T03:47:37.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.030+0000 7f237cb2b640 1 --2- 172.21.15.76:0/2474246911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23781499f0 0x7f23780c82e0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f2368001930 tx=0x7f2368001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:37.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.030+0000 7f236dffb640 1 -- 172.21.15.76:0/2474246911 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236803c070 con 0x7f23781499f0 2023-11-09T03:47:37.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.030+0000 7f236dffb640 1 -- 172.21.15.76:0/2474246911 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236802ca20 con 0x7f23781499f0 2023-11-09T03:47:37.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.030+0000 7f236dffb640 1 -- 172.21.15.76:0/2474246911 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236802cd90 con 0x7f23781499f0 2023-11-09T03:47:37.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.030+0000 7f237f590640 1 -- 172.21.15.76:0/2474246911 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23780c8820 con 0x7f23781499f0 2023-11-09T03:47:37.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.030+0000 7f237f590640 1 -- 172.21.15.76:0/2474246911 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2378147890 con 0x7f23781499f0 2023-11-09T03:47:37.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.031+0000 7f237f590640 1 -- 172.21.15.76:0/2474246911 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2378148350 con 0x7f23781499f0 2023-11-09T03:47:37.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.031+0000 7f236dffb640 1 -- 172.21.15.76:0/2474246911 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2368032070 con 0x7f23781499f0 2023-11-09T03:47:37.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.032+0000 7f236dffb640 1 --2- 172.21.15.76:0/2474246911 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f235c06aea0 0x7f235c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:37.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.032+0000 7f236dffb640 1 -- 172.21.15.76:0/2474246911 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f23680aa610 con 0x7f23781499f0 2023-11-09T03:47:37.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.032+0000 7f237d32c640 1 --2- 172.21.15.76:0/2474246911 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f235c06aea0 0x7f235c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:37.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.032+0000 7f237d32c640 1 --2- 172.21.15.76:0/2474246911 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f235c06aea0 0x7f235c06d360 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2374000c00 tx=0x7f2374009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:37.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.036+0000 7f236dffb640 1 -- 172.21.15.76:0/2474246911 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2368075680 con 0x7f23781499f0 2023-11-09T03:47:37.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.201+0000 7f237f590640 1 -- 172.21.15.76:0/2474246911 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f23780b4820 con 0x7f23781499f0 2023-11-09T03:47:37.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.201+0000 7f236dffb640 1 -- 172.21.15.76:0/2474246911 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v54) v1 ==== 103+0+11 (secure 0 0 0) 0x7f23680450a0 con 0x7f23781499f0 2023-11-09T03:47:37.201 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:47:37.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.203+0000 7f237f590640 1 -- 172.21.15.76:0/2474246911 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f235c06aea0 msgr2=0x7f235c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:37.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.203+0000 7f237f590640 1 --2- 172.21.15.76:0/2474246911 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f235c06aea0 0x7f235c06d360 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2374000c00 tx=0x7f2374009040 comp rx=0 tx=0).stop 2023-11-09T03:47:37.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.203+0000 7f237f590640 1 -- 172.21.15.76:0/2474246911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23781499f0 msgr2=0x7f23780c82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:37.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.203+0000 7f237f590640 1 --2- 172.21.15.76:0/2474246911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23781499f0 0x7f23780c82e0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f2368001930 tx=0x7f2368001a00 comp rx=0 tx=0).stop 2023-11-09T03:47:37.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.203+0000 7f237f590640 1 -- 172.21.15.76:0/2474246911 shutdown_connections 2023-11-09T03:47:37.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.203+0000 7f237f590640 1 --2- 172.21.15.76:0/2474246911 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f235c06aea0 0x7f235c06d360 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.203+0000 7f237f590640 1 --2- 172.21.15.76:0/2474246911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23781e4630 0x7f23781e4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.203+0000 7f237f590640 1 --2- 172.21.15.76:0/2474246911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23781499f0 0x7f23780c82e0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.203+0000 7f237f590640 1 --2- 172.21.15.76:0/2474246911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23640047c0 0x7f2378146e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.203+0000 7f237f590640 1 -- 172.21.15.76:0/2474246911 >> 172.21.15.76:0/2474246911 conn(0x7f23780ce950 msgr2=0x7f23780708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:37.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.203+0000 7f237f590640 1 -- 172.21.15.76:0/2474246911 shutdown_connections 2023-11-09T03:47:37.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.203+0000 7f237f590640 1 -- 172.21.15.76:0/2474246911 wait complete. 2023-11-09T03:47:37.213 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:47:37.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.317+0000 7f422eff5640 1 Processor -- start 2023-11-09T03:47:37.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.317+0000 7f422eff5640 1 -- start start 2023-11-09T03:47:37.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.317+0000 7f422eff5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4228153e60 0x7f422815dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:37.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.317+0000 7f422eff5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f42280cd6d0 0x7f4228161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:37.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.317+0000 7f422eff5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f42280b4880 0x7f42280b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:37.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.317+0000 7f422eff5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42280cde60 con 0x7f42280b4880 2023-11-09T03:47:37.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.317+0000 7f422eff5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4228160f90 con 0x7f4228153e60 2023-11-09T03:47:37.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.317+0000 7f422eff5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42280b51e0 con 0x7f42280cd6d0 2023-11-09T03:47:37.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.317+0000 7f422d592640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f42280b4880 0x7f42280b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:37.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.318+0000 7f422d592640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f42280b4880 0x7f42280b4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48298/0 (socket says 172.21.15.76:48298) 2023-11-09T03:47:37.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.318+0000 7f422d592640 1 -- 172.21.15.76:0/364673099 learned_addr learned my addr 172.21.15.76:0/364673099 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:37.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.318+0000 7f422cd91640 1 --2- 172.21.15.76:0/364673099 >> v2:172.21.15.113:3300/0 conn(0x7f4228153e60 0x7f422815dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:37.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.318+0000 7f4227fff640 1 --2- 172.21.15.76:0/364673099 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f42280cd6d0 0x7f4228161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:37.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.318+0000 7f422d592640 1 -- 172.21.15.76:0/364673099 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f42280cd6d0 msgr2=0x7f4228161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:37.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.318+0000 7f422d592640 1 --2- 172.21.15.76:0/364673099 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f42280cd6d0 0x7f4228161420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.318+0000 7f422d592640 1 -- 172.21.15.76:0/364673099 >> v2:172.21.15.113:3300/0 conn(0x7f4228153e60 msgr2=0x7f422815dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:37.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.318+0000 7f422d592640 1 --2- 172.21.15.76:0/364673099 >> v2:172.21.15.113:3300/0 conn(0x7f4228153e60 0x7f422815dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.318+0000 7f422d592640 1 -- 172.21.15.76:0/364673099 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422815e400 con 0x7f42280b4880 2023-11-09T03:47:37.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.318+0000 7f422d592640 1 --2- 172.21.15.76:0/364673099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42280b4880 0x7f42280b4ca0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f422808abe0 tx=0x7f422002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd6273b6bfc2f86 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:37.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.319+0000 7f42277fe640 1 -- 172.21.15.76:0/364673099 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422002ca70 con 0x7f42280b4880 2023-11-09T03:47:37.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.319+0000 7f42277fe640 1 -- 172.21.15.76:0/364673099 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422002cbd0 con 0x7f42280b4880 2023-11-09T03:47:37.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.319+0000 7f42277fe640 1 -- 172.21.15.76:0/364673099 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42200343f0 con 0x7f42280b4880 2023-11-09T03:47:37.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.319+0000 7f422eff5640 1 -- 172.21.15.76:0/364673099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42280b4880 msgr2=0x7f42280b4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:37.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.319+0000 7f422eff5640 1 --2- 172.21.15.76:0/364673099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42280b4880 0x7f42280b4ca0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f422808abe0 tx=0x7f422002bed0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.319+0000 7f422eff5640 1 -- 172.21.15.76:0/364673099 shutdown_connections 2023-11-09T03:47:37.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.319+0000 7f422eff5640 1 --2- 172.21.15.76:0/364673099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42280b4880 0x7f42280b4ca0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.319+0000 7f422eff5640 1 --2- 172.21.15.76:0/364673099 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f42280cd6d0 0x7f4228161420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.319+0000 7f422eff5640 1 --2- 172.21.15.76:0/364673099 >> v2:172.21.15.113:3300/0 conn(0x7f4228153e60 0x7f422815dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.319+0000 7f422eff5640 1 -- 172.21.15.76:0/364673099 >> 172.21.15.76:0/364673099 conn(0x7f42280cf880 msgr2=0x7f42280cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:37.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.319+0000 7f422eff5640 1 -- 172.21.15.76:0/364673099 shutdown_connections 2023-11-09T03:47:37.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.319+0000 7f422eff5640 1 -- 172.21.15.76:0/364673099 wait complete. 2023-11-09T03:47:37.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.320+0000 7f422eff5640 1 Processor -- start 2023-11-09T03:47:37.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.320+0000 7f422eff5640 1 -- start start 2023-11-09T03:47:37.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.321+0000 7f422eff5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42280b4880 0x7f4228159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:37.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.321+0000 7f422eff5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42280cd6d0 0x7f42280c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:37.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.321+0000 7f422eff5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4228153e60 0x7f42281f5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:37.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.321+0000 7f422eff5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422815a3a0 con 0x7f42280cd6d0 2023-11-09T03:47:37.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.321+0000 7f422eff5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42280c97d0 con 0x7f42280b4880 2023-11-09T03:47:37.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.321+0000 7f422eff5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42280c7d00 con 0x7f4228153e60 2023-11-09T03:47:37.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.321+0000 7f4227fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42280cd6d0 0x7f42280c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:37.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.321+0000 7f4227fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42280cd6d0 0x7f42280c9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48302/0 (socket says 172.21.15.76:48302) 2023-11-09T03:47:37.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.321+0000 7f4227fff640 1 -- 172.21.15.76:0/2049871087 learned_addr learned my addr 172.21.15.76:0/2049871087 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:37.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.322+0000 7f422d592640 1 --2- 172.21.15.76:0/2049871087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4228153e60 0x7f42281f5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:37.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.322+0000 7f422cd91640 1 --2- 172.21.15.76:0/2049871087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42280b4880 0x7f4228159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:37.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.322+0000 7f4227fff640 1 -- 172.21.15.76:0/2049871087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4228153e60 msgr2=0x7f42281f5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:37.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.322+0000 7f4227fff640 1 --2- 172.21.15.76:0/2049871087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4228153e60 0x7f42281f5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.322+0000 7f4227fff640 1 -- 172.21.15.76:0/2049871087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42280b4880 msgr2=0x7f4228159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:37.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.322+0000 7f4227fff640 1 --2- 172.21.15.76:0/2049871087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42280b4880 0x7f4228159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.322+0000 7f4227fff640 1 -- 172.21.15.76:0/2049871087 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4220009b30 con 0x7f42280cd6d0 2023-11-09T03:47:37.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.322+0000 7f422d592640 1 --2- 172.21.15.76:0/2049871087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4228153e60 0x7f42281f5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:37.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.322+0000 7f422cd91640 1 --2- 172.21.15.76:0/2049871087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42280b4880 0x7f4228159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:37.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.322+0000 7f4227fff640 1 --2- 172.21.15.76:0/2049871087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42280cd6d0 0x7f42280c9290 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f421400c7f0 tx=0x7f421400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:37.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.322+0000 7f42257fa640 1 -- 172.21.15.76:0/2049871087 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4214004220 con 0x7f42280cd6d0 2023-11-09T03:47:37.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.322+0000 7f42257fa640 1 -- 172.21.15.76:0/2049871087 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4214004380 con 0x7f42280cd6d0 2023-11-09T03:47:37.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.322+0000 7f422eff5640 1 -- 172.21.15.76:0/2049871087 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42280c6c60 con 0x7f42280cd6d0 2023-11-09T03:47:37.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.322+0000 7f42257fa640 1 -- 172.21.15.76:0/2049871087 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4214007860 con 0x7f42280cd6d0 2023-11-09T03:47:37.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.322+0000 7f422eff5640 1 -- 172.21.15.76:0/2049871087 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42280c7220 con 0x7f42280cd6d0 2023-11-09T03:47:37.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.323+0000 7f422eff5640 1 -- 172.21.15.76:0/2049871087 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4228074290 con 0x7f42280cd6d0 2023-11-09T03:47:37.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.323+0000 7f42257fa640 1 -- 172.21.15.76:0/2049871087 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4214007bd0 con 0x7f42280cd6d0 2023-11-09T03:47:37.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.324+0000 7f42257fa640 1 --2- 172.21.15.76:0/2049871087 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f420c06ae70 0x7f420c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:37.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.324+0000 7f42257fa640 1 -- 172.21.15.76:0/2049871087 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f42140898a0 con 0x7f42280cd6d0 2023-11-09T03:47:37.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.324+0000 7f422cd91640 1 --2- 172.21.15.76:0/2049871087 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f420c06ae70 0x7f420c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:37.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.324+0000 7f422cd91640 1 --2- 172.21.15.76:0/2049871087 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f420c06ae70 0x7f420c06d330 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f421c00b820 tx=0x7f421c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:37.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.328+0000 7f42257fa640 1 -- 172.21.15.76:0/2049871087 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4214054620 con 0x7f42280cd6d0 2023-11-09T03:47:37.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.493+0000 7f422eff5640 1 -- 172.21.15.76:0/2049871087 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f42280d0b80 con 0x7f42280cd6d0 2023-11-09T03:47:37.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.493+0000 7f42257fa640 1 -- 172.21.15.76:0/2049871087 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7f421401e0a0 con 0x7f42280cd6d0 2023-11-09T03:47:37.494 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:47:37.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.495+0000 7f422eff5640 1 -- 172.21.15.76:0/2049871087 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f420c06ae70 msgr2=0x7f420c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:37.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.495+0000 7f422eff5640 1 --2- 172.21.15.76:0/2049871087 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f420c06ae70 0x7f420c06d330 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f421c00b820 tx=0x7f421c008040 comp rx=0 tx=0).stop 2023-11-09T03:47:37.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.495+0000 7f422eff5640 1 -- 172.21.15.76:0/2049871087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42280cd6d0 msgr2=0x7f42280c9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:37.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.495+0000 7f422eff5640 1 --2- 172.21.15.76:0/2049871087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42280cd6d0 0x7f42280c9290 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f421400c7f0 tx=0x7f421400ccb0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.495+0000 7f422eff5640 1 -- 172.21.15.76:0/2049871087 shutdown_connections 2023-11-09T03:47:37.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.495+0000 7f422eff5640 1 --2- 172.21.15.76:0/2049871087 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f420c06ae70 0x7f420c06d330 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.495+0000 7f422eff5640 1 --2- 172.21.15.76:0/2049871087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4228153e60 0x7f42281f5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.495+0000 7f422eff5640 1 --2- 172.21.15.76:0/2049871087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42280cd6d0 0x7f42280c9290 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.495+0000 7f422eff5640 1 --2- 172.21.15.76:0/2049871087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42280b4880 0x7f4228159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.495+0000 7f422eff5640 1 -- 172.21.15.76:0/2049871087 >> 172.21.15.76:0/2049871087 conn(0x7f42280cf880 msgr2=0x7f42280d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:37.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.495+0000 7f422eff5640 1 -- 172.21.15.76:0/2049871087 shutdown_connections 2023-11-09T03:47:37.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.495+0000 7f422eff5640 1 -- 172.21.15.76:0/2049871087 wait complete. 2023-11-09T03:47:37.506 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:47:37.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.609+0000 7f54d74db640 1 Processor -- start 2023-11-09T03:47:37.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.610+0000 7f54d74db640 1 -- start start 2023-11-09T03:47:37.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.610+0000 7f54d74db640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54d0153e60 0x7f54d015dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:37.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.610+0000 7f54d74db640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54d00cd6d0 0x7f54d0161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:37.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.610+0000 7f54d74db640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54d00cde60 con 0x7f54d00b4880 2023-11-09T03:47:37.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.610+0000 7f54d74db640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54d0160f90 con 0x7f54d00cd6d0 2023-11-09T03:47:37.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.610+0000 7f54d74db640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d015e400 con 0x7f54d0153e60 2023-11-09T03:47:37.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.611+0000 7f54d5277640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54d0153e60 0x7f54d015dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:37.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.611+0000 7f54d5277640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54d0153e60 0x7f54d015dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:47928/0 (socket says 172.21.15.76:47928) 2023-11-09T03:47:37.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.611+0000 7f54d5277640 1 -- 172.21.15.76:0/213277987 learned_addr learned my addr 172.21.15.76:0/213277987 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:37.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.611+0000 7f54d4a76640 1 --2- 172.21.15.76:0/213277987 >> v2:172.21.15.113:3300/0 conn(0x7f54d00cd6d0 0x7f54d0161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:37.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.611+0000 7f54d5277640 1 -- 172.21.15.76:0/213277987 >> v2:172.21.15.113:3300/0 conn(0x7f54d00cd6d0 msgr2=0x7f54d0161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:37.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.611+0000 7f54d5277640 1 --2- 172.21.15.76:0/213277987 >> v2:172.21.15.113:3300/0 conn(0x7f54d00cd6d0 0x7f54d0161420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.611+0000 7f54d5277640 1 -- 172.21.15.76:0/213277987 >> v1:172.21.15.113:6789/0 conn(0x7f54d00b4880 legacy=0x7f54d006b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:37.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.611+0000 7f54d5277640 1 -- 172.21.15.76:0/213277987 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d0161960 con 0x7f54d0153e60 2023-11-09T03:47:37.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.612+0000 7f54d5277640 1 --2- 172.21.15.76:0/213277987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54d0153e60 0x7f54d015dec0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f54c0009870 tx=0x7f54c002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb0bfa4a528d9827 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:37.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.612+0000 7f54c7fff640 1 -- 172.21.15.76:0/213277987 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c0030840 con 0x7f54d0153e60 2023-11-09T03:47:37.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.612+0000 7f54c7fff640 1 -- 172.21.15.76:0/213277987 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c00309a0 con 0x7f54d0153e60 2023-11-09T03:47:37.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.612+0000 7f54c7fff640 1 -- 172.21.15.76:0/213277987 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c0030d30 con 0x7f54d0153e60 2023-11-09T03:47:37.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.612+0000 7f54d74db640 1 -- 172.21.15.76:0/213277987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54d0153e60 msgr2=0x7f54d015dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:37.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.612+0000 7f54d74db640 1 --2- 172.21.15.76:0/213277987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54d0153e60 0x7f54d015dec0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f54c0009870 tx=0x7f54c002b980 comp rx=0 tx=0).stop 2023-11-09T03:47:37.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.613+0000 7f54d74db640 1 -- 172.21.15.76:0/213277987 shutdown_connections 2023-11-09T03:47:37.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.613+0000 7f54d74db640 1 --2- 172.21.15.76:0/213277987 >> v2:172.21.15.113:3300/0 conn(0x7f54d00cd6d0 0x7f54d0161420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.613+0000 7f54d74db640 1 --2- 172.21.15.76:0/213277987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54d0153e60 0x7f54d015dec0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.613+0000 7f54d74db640 1 -- 172.21.15.76:0/213277987 >> 172.21.15.76:0/213277987 conn(0x7f54d00cf880 msgr2=0x7f54d00cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:37.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.613+0000 7f54d74db640 1 -- 172.21.15.76:0/213277987 shutdown_connections 2023-11-09T03:47:37.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.613+0000 7f54d74db640 1 -- 172.21.15.76:0/213277987 wait complete. 2023-11-09T03:47:37.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.614+0000 7f54d74db640 1 Processor -- start 2023-11-09T03:47:37.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.614+0000 7f54d74db640 1 -- start start 2023-11-09T03:47:37.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.614+0000 7f54d74db640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54d00b4880 0x7f54d0145c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:37.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.614+0000 7f54d74db640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54d00cd6d0 0x7f54d015a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:37.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.614+0000 7f54d74db640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54d0153e60 0x7f54d0154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:37.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.615+0000 7f54d74db640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d015bb30 con 0x7f54d00b4880 2023-11-09T03:47:37.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.615+0000 7f54d74db640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d015aca0 con 0x7f54d00cd6d0 2023-11-09T03:47:37.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.615+0000 7f54d74db640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d0155220 con 0x7f54d0153e60 2023-11-09T03:47:37.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.615+0000 7f54d5277640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54d00b4880 0x7f54d0145c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:37.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.615+0000 7f54d5277640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54d00b4880 0x7f54d0145c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48314/0 (socket says 172.21.15.76:48314) 2023-11-09T03:47:37.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.615+0000 7f54d5277640 1 -- 172.21.15.76:0/1134279158 learned_addr learned my addr 172.21.15.76:0/1134279158 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:37.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.615+0000 7f54d5a78640 1 --2- 172.21.15.76:0/1134279158 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54d0153e60 0x7f54d0154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:37.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.615+0000 7f54d4a76640 1 --2- 172.21.15.76:0/1134279158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54d00cd6d0 0x7f54d015a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:37.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.615+0000 7f54d5277640 1 -- 172.21.15.76:0/1134279158 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54d0153e60 msgr2=0x7f54d0154ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:37.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.615+0000 7f54d5277640 1 --2- 172.21.15.76:0/1134279158 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54d0153e60 0x7f54d0154ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.615+0000 7f54d5277640 1 -- 172.21.15.76:0/1134279158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54d00cd6d0 msgr2=0x7f54d015a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:37.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.615+0000 7f54d5277640 1 --2- 172.21.15.76:0/1134279158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54d00cd6d0 0x7f54d015a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.615+0000 7f54d5277640 1 -- 172.21.15.76:0/1134279158 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c00094d0 con 0x7f54d00b4880 2023-11-09T03:47:37.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.615+0000 7f54d5277640 1 --2- 172.21.15.76:0/1134279158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54d00b4880 0x7f54d0145c10 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f54c0000c00 tx=0x7f54c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:37.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.615+0000 7f54c5ffb640 1 -- 172.21.15.76:0/1134279158 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c0039a90 con 0x7f54d00b4880 2023-11-09T03:47:37.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.615+0000 7f54c5ffb640 1 -- 172.21.15.76:0/1134279158 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c0039bf0 con 0x7f54d00b4880 2023-11-09T03:47:37.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.615+0000 7f54d74db640 1 -- 172.21.15.76:0/1134279158 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54d015aeb0 con 0x7f54d00b4880 2023-11-09T03:47:37.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.615+0000 7f54c5ffb640 1 -- 172.21.15.76:0/1134279158 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c0043410 con 0x7f54d00b4880 2023-11-09T03:47:37.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.616+0000 7f54d74db640 1 -- 172.21.15.76:0/1134279158 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54d015b470 con 0x7f54d00b4880 2023-11-09T03:47:37.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.617+0000 7f54d74db640 1 -- 172.21.15.76:0/1134279158 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54d0074290 con 0x7f54d00b4880 2023-11-09T03:47:37.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.617+0000 7f54c5ffb640 1 -- 172.21.15.76:0/1134279158 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f54c002f390 con 0x7f54d00b4880 2023-11-09T03:47:37.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.617+0000 7f54c5ffb640 1 --2- 172.21.15.76:0/1134279158 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f54b406ae70 0x7f54b406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:37.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.618+0000 7f54c5ffb640 1 -- 172.21.15.76:0/1134279158 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f54c00ab9c0 con 0x7f54d00b4880 2023-11-09T03:47:37.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.618+0000 7f54d4a76640 1 --2- 172.21.15.76:0/1134279158 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f54b406ae70 0x7f54b406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:37.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.618+0000 7f54d4a76640 1 --2- 172.21.15.76:0/1134279158 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f54b406ae70 0x7f54b406d330 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f54d00c92e0 tx=0x7f54cc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:37.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.622+0000 7f54c5ffb640 1 -- 172.21.15.76:0/1134279158 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f54c0076800 con 0x7f54d00b4880 2023-11-09T03:47:37.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.788+0000 7f54d74db640 1 -- 172.21.15.76:0/1134279158 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f54d00b7fb0 con 0x7f54d00b4880 2023-11-09T03:47:37.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.789+0000 7f54c5ffb640 1 -- 172.21.15.76:0/1134279158 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7f54c0079e20 con 0x7f54d00b4880 2023-11-09T03:47:37.788 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:47:37.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.791+0000 7f54d74db640 1 -- 172.21.15.76:0/1134279158 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f54b406ae70 msgr2=0x7f54b406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:37.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.791+0000 7f54d74db640 1 --2- 172.21.15.76:0/1134279158 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f54b406ae70 0x7f54b406d330 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f54d00c92e0 tx=0x7f54cc009190 comp rx=0 tx=0).stop 2023-11-09T03:47:37.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.791+0000 7f54d74db640 1 -- 172.21.15.76:0/1134279158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54d00b4880 msgr2=0x7f54d0145c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:37.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.791+0000 7f54d74db640 1 --2- 172.21.15.76:0/1134279158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54d00b4880 0x7f54d0145c10 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f54c0000c00 tx=0x7f54c0004730 comp rx=0 tx=0).stop 2023-11-09T03:47:37.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.791+0000 7f54d74db640 1 -- 172.21.15.76:0/1134279158 shutdown_connections 2023-11-09T03:47:37.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.791+0000 7f54d74db640 1 --2- 172.21.15.76:0/1134279158 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f54b406ae70 0x7f54b406d330 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.791+0000 7f54d74db640 1 --2- 172.21.15.76:0/1134279158 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54d0153e60 0x7f54d0154ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.791+0000 7f54d74db640 1 --2- 172.21.15.76:0/1134279158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54d00cd6d0 0x7f54d015a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.791+0000 7f54d74db640 1 --2- 172.21.15.76:0/1134279158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54d00b4880 0x7f54d0145c10 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.791+0000 7f54d74db640 1 -- 172.21.15.76:0/1134279158 >> 172.21.15.76:0/1134279158 conn(0x7f54d00cf880 msgr2=0x7f54d015ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:37.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.791+0000 7f54d74db640 1 -- 172.21.15.76:0/1134279158 shutdown_connections 2023-11-09T03:47:37.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.791+0000 7f54d74db640 1 -- 172.21.15.76:0/1134279158 wait complete. 2023-11-09T03:47:37.801 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:47:37.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.901+0000 7f154027c640 1 Processor -- start 2023-11-09T03:47:37.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.901+0000 7f154027c640 1 -- start start 2023-11-09T03:47:37.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.902+0000 7f154027c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f15380cde60 0x7f153815d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:37.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.902+0000 7f154027c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15380ce240 0x7f15380d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:37.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.902+0000 7f154027c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f153815e000 con 0x7f153815daa0 2023-11-09T03:47:37.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.902+0000 7f154027c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f153815e5c0 con 0x7f15380cde60 2023-11-09T03:47:37.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.902+0000 7f154027c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f153815e2e0 con 0x7f15380ce240 2023-11-09T03:47:37.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.902+0000 7f153e018640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f15380cde60 0x7f153815d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:37.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.902+0000 7f153e018640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f15380cde60 0x7f153815d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48324/0 (socket says 172.21.15.76:48324) 2023-11-09T03:47:37.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.903+0000 7f153e018640 1 -- 172.21.15.76:0/585909200 learned_addr learned my addr 172.21.15.76:0/585909200 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:37.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.903+0000 7f153e819640 1 --2- 172.21.15.76:0/585909200 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15380ce240 0x7f15380d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:37.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.903+0000 7f153e018640 1 -- 172.21.15.76:0/585909200 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15380ce240 msgr2=0x7f15380d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:37.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.903+0000 7f153e018640 1 --2- 172.21.15.76:0/585909200 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15380ce240 0x7f15380d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.903+0000 7f153e018640 1 -- 172.21.15.76:0/585909200 >> v1:172.21.15.113:6789/0 conn(0x7f153815daa0 legacy=0x7f1538157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:37.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.903+0000 7f153e018640 1 -- 172.21.15.76:0/585909200 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15380d1120 con 0x7f15380cde60 2023-11-09T03:47:37.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.904+0000 7f153e018640 1 --2- 172.21.15.76:0/585909200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15380cde60 0x7f153815d560 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1534009e50 tx=0x7f153402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa7fb376a1fd7ca server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:37.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.904+0000 7f153d016640 1 -- 172.21.15.76:0/585909200 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153402cb00 con 0x7f15380cde60 2023-11-09T03:47:37.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.904+0000 7f153d016640 1 -- 172.21.15.76:0/585909200 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153402cc60 con 0x7f15380cde60 2023-11-09T03:47:37.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.904+0000 7f153d016640 1 -- 172.21.15.76:0/585909200 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15340343f0 con 0x7f15380cde60 2023-11-09T03:47:37.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.904+0000 7f154027c640 1 -- 172.21.15.76:0/585909200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15380cde60 msgr2=0x7f153815d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:37.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.904+0000 7f154027c640 1 --2- 172.21.15.76:0/585909200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15380cde60 0x7f153815d560 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1534009e50 tx=0x7f153402bf60 comp rx=0 tx=0).stop 2023-11-09T03:47:37.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.904+0000 7f154027c640 1 -- 172.21.15.76:0/585909200 shutdown_connections 2023-11-09T03:47:37.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.904+0000 7f154027c640 1 --2- 172.21.15.76:0/585909200 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15380ce240 0x7f15380d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.904+0000 7f154027c640 1 --2- 172.21.15.76:0/585909200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15380cde60 0x7f153815d560 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.904+0000 7f154027c640 1 -- 172.21.15.76:0/585909200 >> 172.21.15.76:0/585909200 conn(0x7f1538071f30 msgr2=0x7f15380cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:37.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.904+0000 7f154027c640 1 -- 172.21.15.76:0/585909200 shutdown_connections 2023-11-09T03:47:37.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.904+0000 7f154027c640 1 -- 172.21.15.76:0/585909200 wait complete. 2023-11-09T03:47:37.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.906+0000 7f154027c640 1 Processor -- start 2023-11-09T03:47:37.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.906+0000 7f154027c640 1 -- start start 2023-11-09T03:47:37.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.906+0000 7f154027c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15380cde60 0x7f1538156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:37.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.906+0000 7f154027c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15380ce240 0x7f1538146db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:37.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.906+0000 7f154027c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f153815daa0 0x7f15381f6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:37.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.906+0000 7f154027c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1538157d70 con 0x7f15380ce240 2023-11-09T03:47:37.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.906+0000 7f154027c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15381574d0 con 0x7f15380cde60 2023-11-09T03:47:37.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.906+0000 7f154027c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15381577b0 con 0x7f153815daa0 2023-11-09T03:47:37.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.906+0000 7f153d817640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15380ce240 0x7f1538146db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:37.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.906+0000 7f153d817640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15380ce240 0x7f1538146db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48326/0 (socket says 172.21.15.76:48326) 2023-11-09T03:47:37.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.906+0000 7f153d817640 1 -- 172.21.15.76:0/3013976075 learned_addr learned my addr 172.21.15.76:0/3013976075 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:37.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.907+0000 7f153e819640 1 --2- 172.21.15.76:0/3013976075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f153815daa0 0x7f15381f6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:37.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.907+0000 7f153e018640 1 --2- 172.21.15.76:0/3013976075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15380cde60 0x7f1538156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:37.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.907+0000 7f153d817640 1 -- 172.21.15.76:0/3013976075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f153815daa0 msgr2=0x7f15381f6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:37.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.907+0000 7f153d817640 1 --2- 172.21.15.76:0/3013976075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f153815daa0 0x7f15381f6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.907+0000 7f153d817640 1 -- 172.21.15.76:0/3013976075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15380cde60 msgr2=0x7f1538156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:37.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.907+0000 7f153d817640 1 --2- 172.21.15.76:0/3013976075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15380cde60 0x7f1538156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:37.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.907+0000 7f153d817640 1 -- 172.21.15.76:0/3013976075 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1534009b30 con 0x7f15380ce240 2023-11-09T03:47:37.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.907+0000 7f153d817640 1 --2- 172.21.15.76:0/3013976075 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15380ce240 0x7f1538146db0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f152800c3c0 tx=0x7f152800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:37.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.907+0000 7f152effd640 1 -- 172.21.15.76:0/3013976075 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15280052d0 con 0x7f15380ce240 2023-11-09T03:47:37.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.907+0000 7f152effd640 1 -- 172.21.15.76:0/3013976075 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1528005450 con 0x7f15380ce240 2023-11-09T03:47:37.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.907+0000 7f154027c640 1 -- 172.21.15.76:0/3013976075 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1538157a90 con 0x7f15380ce240 2023-11-09T03:47:37.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.907+0000 7f152effd640 1 -- 172.21.15.76:0/3013976075 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1528003830 con 0x7f15380ce240 2023-11-09T03:47:37.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.907+0000 7f154027c640 1 -- 172.21.15.76:0/3013976075 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15381472f0 con 0x7f15380ce240 2023-11-09T03:47:37.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.908+0000 7f154027c640 1 -- 172.21.15.76:0/3013976075 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1538147970 con 0x7f15380ce240 2023-11-09T03:47:37.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.908+0000 7f152effd640 1 -- 172.21.15.76:0/3013976075 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1528003990 con 0x7f15380ce240 2023-11-09T03:47:37.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.909+0000 7f152effd640 1 --2- 172.21.15.76:0/3013976075 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f151c06ae70 0x7f151c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:37.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.909+0000 7f152effd640 1 -- 172.21.15.76:0/3013976075 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1528088da0 con 0x7f15380ce240 2023-11-09T03:47:37.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.909+0000 7f153e018640 1 --2- 172.21.15.76:0/3013976075 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f151c06ae70 0x7f151c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:37.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.909+0000 7f153e018640 1 --2- 172.21.15.76:0/3013976075 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f151c06ae70 0x7f151c06d330 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1534035000 tx=0x7f1534030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:37.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:37.913+0000 7f152effd640 1 -- 172.21.15.76:0/3013976075 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1538147970 con 0x7f15380ce240 2023-11-09T03:47:38.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.113+0000 7f154027c640 1 -- 172.21.15.76:0/3013976075 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1538147970 con 0x7f15380ce240 2023-11-09T03:47:38.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.114+0000 7f152effd640 1 -- 172.21.15.76:0/3013976075 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f1538147970 con 0x7f15380ce240 2023-11-09T03:47:38.114 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:38.115 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5099,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1844478483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1844478483},{"type":"v1","addr":"172.21.15.113:6836","nonce":1844478483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5102,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1111318612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1111318612},{"type":"v1","addr":"172.21.15.113:6837","nonce":1111318612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":6072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/2037805253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2037805253},{"type":"v1","addr":"172.21.15.76:6834","nonce":2037805253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:47:20.265190+0000","modified":"2023-11-09T03:47:22.742442+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6075},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6075":{"gid":6075,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/3710140065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":3710140065},{"type":"v1","addr":"172.21.15.76:6835","nonce":3710140065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-09T03:47:38.115 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 23 2023-11-09T03:47:38.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.116+0000 7f154027c640 1 -- 172.21.15.76:0/3013976075 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f151c06ae70 msgr2=0x7f151c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:38.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.116+0000 7f154027c640 1 --2- 172.21.15.76:0/3013976075 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f151c06ae70 0x7f151c06d330 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1534035000 tx=0x7f1534030000 comp rx=0 tx=0).stop 2023-11-09T03:47:38.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.116+0000 7f154027c640 1 -- 172.21.15.76:0/3013976075 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15380ce240 msgr2=0x7f1538146db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:38.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.116+0000 7f154027c640 1 --2- 172.21.15.76:0/3013976075 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15380ce240 0x7f1538146db0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f152800c3c0 tx=0x7f152800c880 comp rx=0 tx=0).stop 2023-11-09T03:47:38.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.116+0000 7f154027c640 1 -- 172.21.15.76:0/3013976075 shutdown_connections 2023-11-09T03:47:38.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.116+0000 7f154027c640 1 --2- 172.21.15.76:0/3013976075 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f151c06ae70 0x7f151c06d330 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.116+0000 7f154027c640 1 --2- 172.21.15.76:0/3013976075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f153815daa0 0x7f15381f6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.116+0000 7f154027c640 1 --2- 172.21.15.76:0/3013976075 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15380ce240 0x7f1538146db0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.116+0000 7f154027c640 1 --2- 172.21.15.76:0/3013976075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15380cde60 0x7f1538156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.116+0000 7f154027c640 1 -- 172.21.15.76:0/3013976075 >> 172.21.15.76:0/3013976075 conn(0x7f1538071f30 msgr2=0x7f15380c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:38.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.116+0000 7f154027c640 1 -- 172.21.15.76:0/3013976075 shutdown_connections 2023-11-09T03:47:38.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.116+0000 7f154027c640 1 -- 172.21.15.76:0/3013976075 wait complete. 2023-11-09T03:47:38.127 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:47:38.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.226+0000 7fe2337ee640 1 Processor -- start 2023-11-09T03:47:38.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.226+0000 7fe2337ee640 1 -- start start 2023-11-09T03:47:38.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.227+0000 7fe2337ee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe22c157710 0x7fe22c167b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:38.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.227+0000 7fe2337ee640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe22c15e000 con 0x7fe22c0ce240 2023-11-09T03:47:38.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.227+0000 7fe2337ee640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe22c15e2e0 con 0x7fe22c0cde60 2023-11-09T03:47:38.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.227+0000 7fe2337ee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe22c15e5c0 con 0x7fe22c157710 2023-11-09T03:47:38.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.227+0000 7fe230d89640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe22c0ce240 0x7fe22c157000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47260/0 (socket says 172.21.15.76:47260) 2023-11-09T03:47:38.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.227+0000 7fe230d89640 1 -- 172.21.15.76:0/2737096575 learned_addr learned my addr 172.21.15.76:0/2737096575 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:38.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.228+0000 7fe231d8b640 1 --2- 172.21.15.76:0/2737096575 >> v2:172.21.15.113:3300/0 conn(0x7fe22c157710 0x7fe22c167b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:38.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.228+0000 7fe223fff640 1 -- 172.21.15.76:0/2737096575 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 74620134 0 0) 0x7fe22c15e000 con 0x7fe22c0ce240 2023-11-09T03:47:38.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.228+0000 7fe223fff640 1 -- 172.21.15.76:0/2737096575 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe218003520 con 0x7fe22c0ce240 2023-11-09T03:47:38.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.228+0000 7fe223fff640 1 -- 172.21.15.76:0/2737096575 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1846767303 0 0) 0x7fe218003520 con 0x7fe22c0ce240 2023-11-09T03:47:38.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.228+0000 7fe223fff640 1 -- 172.21.15.76:0/2737096575 >> v2:172.21.15.113:3300/0 conn(0x7fe22c157710 msgr2=0x7fe22c167b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:38.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.228+0000 7fe223fff640 1 --2- 172.21.15.76:0/2737096575 >> v2:172.21.15.113:3300/0 conn(0x7fe22c157710 0x7fe22c167b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.228+0000 7fe223fff640 1 -- 172.21.15.76:0/2737096575 >> v1:172.21.15.113:6789/0 conn(0x7fe22c0cde60 legacy=0x7fe22c15d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:38.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.228+0000 7fe223fff640 1 -- 172.21.15.76:0/2737096575 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22c0d1120 con 0x7fe22c0ce240 2023-11-09T03:47:38.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.228+0000 7fe223fff640 1 -- 172.21.15.76:0/2737096575 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fe21c001670 con 0x7fe22c0ce240 2023-11-09T03:47:38.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.228+0000 7fe223fff640 1 -- 172.21.15.76:0/2737096575 >> v1:172.21.15.76:6789/0 conn(0x7fe22c0ce240 legacy=0x7fe22c157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:38.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.228+0000 7fe223fff640 1 --2- 172.21.15.76:0/2737096575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe218003970 0x7fe218003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:38.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.228+0000 7fe223fff640 1 --2- 172.21.15.76:0/2737096575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2180047c0 0x7fe218004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:38.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.228+0000 7fe223fff640 1 --2- 172.21.15.76:0/2737096575 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe218005180 0x7fe21800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:38.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.228+0000 7fe223fff640 1 -- 172.21.15.76:0/2737096575 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c15e5c0 con 0x7fe218003970 2023-11-09T03:47:38.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.228+0000 7fe223fff640 1 -- 172.21.15.76:0/2737096575 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21800c020 con 0x7fe2180047c0 2023-11-09T03:47:38.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.228+0000 7fe223fff640 1 -- 172.21.15.76:0/2737096575 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe21800c1e0 con 0x7fe218005180 2023-11-09T03:47:38.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.228+0000 7fe23158a640 1 --2- 172.21.15.76:0/2737096575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe218003970 0x7fe218003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:38.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.228+0000 7fe230d89640 1 --2- 172.21.15.76:0/2737096575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2180047c0 0x7fe218004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:38.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.228+0000 7fe231d8b640 1 --2- 172.21.15.76:0/2737096575 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe218005180 0x7fe21800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:38.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.228+0000 7fe23158a640 1 -- 172.21.15.76:0/2737096575 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe218005180 msgr2=0x7fe21800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:38.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.228+0000 7fe23158a640 1 --2- 172.21.15.76:0/2737096575 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe218005180 0x7fe21800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.228+0000 7fe23158a640 1 -- 172.21.15.76:0/2737096575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2180047c0 msgr2=0x7fe218004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:38.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.228+0000 7fe23158a640 1 --2- 172.21.15.76:0/2737096575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2180047c0 0x7fe218004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.228+0000 7fe23158a640 1 -- 172.21.15.76:0/2737096575 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe218003520 con 0x7fe218003970 2023-11-09T03:47:38.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.228+0000 7fe231d8b640 1 -- 172.21.15.76:0/2737096575 reap_dead start 2023-11-09T03:47:38.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.229+0000 7fe23158a640 1 --2- 172.21.15.76:0/2737096575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe218003970 0x7fe218003d90 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fe228001750 tx=0x7fe2180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba35f0b5edc782ce server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:38.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.229+0000 7fe223fff640 1 -- 172.21.15.76:0/2737096575 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2280052d0 con 0x7fe218003970 2023-11-09T03:47:38.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.229+0000 7fe223fff640 1 -- 172.21.15.76:0/2737096575 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22802e050 con 0x7fe218003970 2023-11-09T03:47:38.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.229+0000 7fe223fff640 1 -- 172.21.15.76:0/2737096575 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe228003830 con 0x7fe218003970 2023-11-09T03:47:38.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.230+0000 7fe2337ee640 1 -- 172.21.15.76:0/2737096575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe218003970 msgr2=0x7fe218003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:38.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.230+0000 7fe2337ee640 1 --2- 172.21.15.76:0/2737096575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe218003970 0x7fe218003d90 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fe228001750 tx=0x7fe2180057f0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.230+0000 7fe2337ee640 1 -- 172.21.15.76:0/2737096575 shutdown_connections 2023-11-09T03:47:38.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.230+0000 7fe2337ee640 1 --2- 172.21.15.76:0/2737096575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe218003970 0x7fe218003d90 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.230+0000 7fe2337ee640 1 -- 172.21.15.76:0/2737096575 >> 172.21.15.76:0/2737096575 conn(0x7fe22c071f30 msgr2=0x7fe22c0cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:38.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.230+0000 7fe2337ee640 1 -- 172.21.15.76:0/2737096575 shutdown_connections 2023-11-09T03:47:38.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.230+0000 7fe2337ee640 1 -- 172.21.15.76:0/2737096575 wait complete. 2023-11-09T03:47:38.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.231+0000 7fe2337ee640 1 Processor -- start 2023-11-09T03:47:38.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.231+0000 7fe2337ee640 1 -- start start 2023-11-09T03:47:38.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.232+0000 7fe2337ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe218003970 0x7fe22c1525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:38.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.232+0000 7fe2337ee640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe22c157c20 0x7fe22c0d2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:38.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.232+0000 7fe2337ee640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe22c1580a0 0x7fe22c0d3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:38.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.232+0000 7fe2337ee640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c1f4540 con 0x7fe22c1580a0 2023-11-09T03:47:38.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.232+0000 7fe2337ee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c1f3930 con 0x7fe218003970 2023-11-09T03:47:38.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.232+0000 7fe2337ee640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c1f3c10 con 0x7fe22c157c20 2023-11-09T03:47:38.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.232+0000 7fe231d8b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe22c1580a0 0x7fe22c0d3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:38.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.232+0000 7fe231d8b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe22c1580a0 0x7fe22c0d3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48340/0 (socket says 172.21.15.76:48340) 2023-11-09T03:47:38.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.232+0000 7fe231d8b640 1 -- 172.21.15.76:0/209845877 learned_addr learned my addr 172.21.15.76:0/209845877 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:38.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.232+0000 7fe230d89640 1 --2- 172.21.15.76:0/209845877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe22c157c20 0x7fe22c0d2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:38.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.232+0000 7fe23158a640 1 --2- 172.21.15.76:0/209845877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe218003970 0x7fe22c1525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:38.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.232+0000 7fe231d8b640 1 -- 172.21.15.76:0/209845877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe22c157c20 msgr2=0x7fe22c0d2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:38.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.232+0000 7fe231d8b640 1 --2- 172.21.15.76:0/209845877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe22c157c20 0x7fe22c0d2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.232+0000 7fe231d8b640 1 -- 172.21.15.76:0/209845877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe218003970 msgr2=0x7fe22c1525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:38.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.232+0000 7fe231d8b640 1 --2- 172.21.15.76:0/209845877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe218003970 0x7fe22c1525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.232+0000 7fe231d8b640 1 -- 172.21.15.76:0/209845877 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22c1f3ef0 con 0x7fe22c1580a0 2023-11-09T03:47:38.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.232+0000 7fe230d89640 1 --2- 172.21.15.76:0/209845877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe22c157c20 0x7fe22c0d2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:38.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.232+0000 7fe23158a640 1 --2- 172.21.15.76:0/209845877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe218003970 0x7fe22c1525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:38.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.232+0000 7fe231d8b640 1 --2- 172.21.15.76:0/209845877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe22c1580a0 0x7fe22c0d3340 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe22c0cf5c0 tx=0x7fe22400f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:38.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.232+0000 7fe221ffb640 1 -- 172.21.15.76:0/209845877 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22400e040 con 0x7fe22c1580a0 2023-11-09T03:47:38.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.232+0000 7fe221ffb640 1 -- 172.21.15.76:0/209845877 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22400f040 con 0x7fe22c1580a0 2023-11-09T03:47:38.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.232+0000 7fe2337ee640 1 -- 172.21.15.76:0/209845877 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe22c1f41d0 con 0x7fe22c1580a0 2023-11-09T03:47:38.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.232+0000 7fe221ffb640 1 -- 172.21.15.76:0/209845877 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe224015430 con 0x7fe22c1580a0 2023-11-09T03:47:38.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.233+0000 7fe2337ee640 1 -- 172.21.15.76:0/209845877 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe22c158780 con 0x7fe22c1580a0 2023-11-09T03:47:38.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.234+0000 7fe2337ee640 1 -- 172.21.15.76:0/209845877 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe22c1f46f0 con 0x7fe22c1580a0 2023-11-09T03:47:38.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.234+0000 7fe221ffb640 1 -- 172.21.15.76:0/209845877 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe224015590 con 0x7fe22c1580a0 2023-11-09T03:47:38.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.234+0000 7fe221ffb640 1 --2- 172.21.15.76:0/209845877 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe21006aea0 0x7fe21006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:38.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.234+0000 7fe221ffb640 1 -- 172.21.15.76:0/209845877 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe22408a1c0 con 0x7fe22c1580a0 2023-11-09T03:47:38.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.235+0000 7fe23158a640 1 --2- 172.21.15.76:0/209845877 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe21006aea0 0x7fe21006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:38.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.235+0000 7fe23158a640 1 --2- 172.21.15.76:0/209845877 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe21006aea0 0x7fe21006d360 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe228033000 tx=0x7fe228009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:38.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.238+0000 7fe221ffb640 1 -- 172.21.15.76:0/209845877 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe22c1f46f0 con 0x7fe22c1580a0 2023-11-09T03:47:38.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.406+0000 7fe2337ee640 1 -- 172.21.15.76:0/209845877 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe22c074290 con 0x7fe22c1580a0 2023-11-09T03:47:38.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.408+0000 7fe221ffb640 1 -- 172.21.15.76:0/209845877 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+18511 (secure 0 0 0) 0x7fe224055060 con 0x7fe22c1580a0 2023-11-09T03:47:38.409 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:38.409 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":54,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:47:21.936243+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:47:18.807110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-09T03:47:19.235022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:47:38.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.410+0000 7fe2337ee640 1 -- 172.21.15.76:0/209845877 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe21006aea0 msgr2=0x7fe21006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:38.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.410+0000 7fe2337ee640 1 --2- 172.21.15.76:0/209845877 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe21006aea0 0x7fe21006d360 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe228033000 tx=0x7fe228009030 comp rx=0 tx=0).stop 2023-11-09T03:47:38.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.410+0000 7fe2337ee640 1 -- 172.21.15.76:0/209845877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe22c1580a0 msgr2=0x7fe22c0d3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:38.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.410+0000 7fe2337ee640 1 --2- 172.21.15.76:0/209845877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe22c1580a0 0x7fe22c0d3340 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe22c0cf5c0 tx=0x7fe22400f1b0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.410+0000 7fe2337ee640 1 -- 172.21.15.76:0/209845877 shutdown_connections 2023-11-09T03:47:38.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.410+0000 7fe2337ee640 1 --2- 172.21.15.76:0/209845877 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe21006aea0 0x7fe21006d360 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.410+0000 7fe2337ee640 1 --2- 172.21.15.76:0/209845877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe22c1580a0 0x7fe22c0d3340 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.410+0000 7fe2337ee640 1 --2- 172.21.15.76:0/209845877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe22c157c20 0x7fe22c0d2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.410+0000 7fe2337ee640 1 --2- 172.21.15.76:0/209845877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe218003970 0x7fe22c1525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.410+0000 7fe2337ee640 1 -- 172.21.15.76:0/209845877 >> 172.21.15.76:0/209845877 conn(0x7fe22c071f30 msgr2=0x7fe22c0b4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:38.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.410+0000 7fe2337ee640 1 -- 172.21.15.76:0/209845877 shutdown_connections 2023-11-09T03:47:38.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.410+0000 7fe2337ee640 1 -- 172.21.15.76:0/209845877 wait complete. 2023-11-09T03:47:38.421 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-09T03:47:38.422 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:47:38.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.520+0000 7f4d6a376640 1 Processor -- start 2023-11-09T03:47:38.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.521+0000 7f4d6a376640 1 -- start start 2023-11-09T03:47:38.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.521+0000 7f4d6a376640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4d640cde60 0x7f4d6415d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:38.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.521+0000 7f4d6a376640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d6415e000 con 0x7f4d640ce240 2023-11-09T03:47:38.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.521+0000 7f4d6a376640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d6415e2e0 con 0x7f4d6415daa0 2023-11-09T03:47:38.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.521+0000 7f4d6a376640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d6415e5c0 con 0x7f4d640cde60 2023-11-09T03:47:38.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.521+0000 7f4d63fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4d640cde60 0x7f4d6415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:38.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.521+0000 7f4d68913640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f4d640ce240 0x7f4d640d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47274/0 (socket says 172.21.15.76:47274) 2023-11-09T03:47:38.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.521+0000 7f4d68913640 1 -- 172.21.15.76:0/4255296049 learned_addr learned my addr 172.21.15.76:0/4255296049 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:38.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.522+0000 7f4d63fff640 1 -- 172.21.15.76:0/4255296049 >> v1:172.21.15.113:6789/0 conn(0x7f4d6415daa0 legacy=0x7f4d64157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:38.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.522+0000 7f4d63fff640 1 -- 172.21.15.76:0/4255296049 >> v1:172.21.15.76:6789/0 conn(0x7f4d640ce240 legacy=0x7f4d640d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:38.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.522+0000 7f4d63fff640 1 -- 172.21.15.76:0/4255296049 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d640d1120 con 0x7f4d640cde60 2023-11-09T03:47:38.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.522+0000 7f4d63fff640 1 --2- 172.21.15.76:0/4255296049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d640cde60 0x7f4d6415d560 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4d540097a0 tx=0x7f4d5402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892da58802e8c991 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:38.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.523+0000 7f4d62ffd640 1 -- 172.21.15.76:0/4255296049 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d54031830 con 0x7f4d640cde60 2023-11-09T03:47:38.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.523+0000 7f4d62ffd640 1 -- 172.21.15.76:0/4255296049 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d54031990 con 0x7f4d640cde60 2023-11-09T03:47:38.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.523+0000 7f4d62ffd640 1 -- 172.21.15.76:0/4255296049 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d54031d20 con 0x7f4d640cde60 2023-11-09T03:47:38.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.523+0000 7f4d6a376640 1 -- 172.21.15.76:0/4255296049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d640cde60 msgr2=0x7f4d6415d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:38.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.523+0000 7f4d6a376640 1 --2- 172.21.15.76:0/4255296049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d640cde60 0x7f4d6415d560 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4d540097a0 tx=0x7f4d5402b8d0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.523+0000 7f4d6a376640 1 -- 172.21.15.76:0/4255296049 shutdown_connections 2023-11-09T03:47:38.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.523+0000 7f4d6a376640 1 --2- 172.21.15.76:0/4255296049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d640cde60 0x7f4d6415d560 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.523+0000 7f4d6a376640 1 -- 172.21.15.76:0/4255296049 >> 172.21.15.76:0/4255296049 conn(0x7f4d64071f30 msgr2=0x7f4d640cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:38.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.523+0000 7f4d6a376640 1 -- 172.21.15.76:0/4255296049 shutdown_connections 2023-11-09T03:47:38.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.523+0000 7f4d6a376640 1 -- 172.21.15.76:0/4255296049 wait complete. 2023-11-09T03:47:38.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.524+0000 7f4d6a376640 1 Processor -- start 2023-11-09T03:47:38.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.525+0000 7f4d6a376640 1 -- start start 2023-11-09T03:47:38.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.525+0000 7f4d6a376640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d640cde60 0x7f4d640cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:38.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.525+0000 7f4d6a376640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d640ce240 0x7f4d640cb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:38.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.525+0000 7f4d6a376640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d6415daa0 0x7f4d6414a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:38.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.525+0000 7f4d6a376640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d640ca380 con 0x7f4d6415daa0 2023-11-09T03:47:38.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.525+0000 7f4d6a376640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d640ca660 con 0x7f4d640cde60 2023-11-09T03:47:38.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.525+0000 7f4d6a376640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d640ca940 con 0x7f4d640ce240 2023-11-09T03:47:38.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.525+0000 7f4d68913640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d6415daa0 0x7f4d6414a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:38.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.525+0000 7f4d68913640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d6415daa0 0x7f4d6414a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48364/0 (socket says 172.21.15.76:48364) 2023-11-09T03:47:38.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.525+0000 7f4d68913640 1 -- 172.21.15.76:0/2255312560 learned_addr learned my addr 172.21.15.76:0/2255312560 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:38.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.526+0000 7f4d63fff640 1 --2- 172.21.15.76:0/2255312560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d640cde60 0x7f4d640cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:38.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.526+0000 7f4d637fe640 1 --2- 172.21.15.76:0/2255312560 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d640ce240 0x7f4d640cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:38.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.526+0000 7f4d68913640 1 -- 172.21.15.76:0/2255312560 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d640ce240 msgr2=0x7f4d640cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:38.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.526+0000 7f4d68913640 1 --2- 172.21.15.76:0/2255312560 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d640ce240 0x7f4d640cb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.526+0000 7f4d68913640 1 -- 172.21.15.76:0/2255312560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d640cde60 msgr2=0x7f4d640cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:38.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.526+0000 7f4d68913640 1 --2- 172.21.15.76:0/2255312560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d640cde60 0x7f4d640cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.526+0000 7f4d68913640 1 -- 172.21.15.76:0/2255312560 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d54009400 con 0x7f4d6415daa0 2023-11-09T03:47:38.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.526+0000 7f4d637fe640 1 --2- 172.21.15.76:0/2255312560 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d640ce240 0x7f4d640cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:38.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.526+0000 7f4d68913640 1 --2- 172.21.15.76:0/2255312560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d6415daa0 0x7f4d6414a5c0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00caf0 tx=0x7f4d5c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:38.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.526+0000 7f4d60ff9640 1 -- 172.21.15.76:0/2255312560 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c00aca0 con 0x7f4d6415daa0 2023-11-09T03:47:38.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.526+0000 7f4d60ff9640 1 -- 172.21.15.76:0/2255312560 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5c00ae00 con 0x7f4d6415daa0 2023-11-09T03:47:38.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.526+0000 7f4d6a376640 1 -- 172.21.15.76:0/2255312560 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d640cac20 con 0x7f4d6415daa0 2023-11-09T03:47:38.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.526+0000 7f4d60ff9640 1 -- 172.21.15.76:0/2255312560 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c012600 con 0x7f4d6415daa0 2023-11-09T03:47:38.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.526+0000 7f4d6a376640 1 -- 172.21.15.76:0/2255312560 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6414ab00 con 0x7f4d6415daa0 2023-11-09T03:47:38.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.527+0000 7f4d6a376640 1 -- 172.21.15.76:0/2255312560 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d640cb140 con 0x7f4d6415daa0 2023-11-09T03:47:38.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.528+0000 7f4d60ff9640 1 -- 172.21.15.76:0/2255312560 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4d5c007070 con 0x7f4d6415daa0 2023-11-09T03:47:38.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.528+0000 7f4d60ff9640 1 --2- 172.21.15.76:0/2255312560 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d5006ae70 0x7f4d5006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:38.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.528+0000 7f4d60ff9640 1 -- 172.21.15.76:0/2255312560 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4d5c089680 con 0x7f4d6415daa0 2023-11-09T03:47:38.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.528+0000 7f4d63fff640 1 --2- 172.21.15.76:0/2255312560 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d5006ae70 0x7f4d5006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:38.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.529+0000 7f4d63fff640 1 --2- 172.21.15.76:0/2255312560 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d5006ae70 0x7f4d5006d330 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4d54036000 tx=0x7f4d5403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:38.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.532+0000 7f4d60ff9640 1 -- 172.21.15.76:0/2255312560 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4d5c054540 con 0x7f4d6415daa0 2023-11-09T03:47:38.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.733+0000 7f4d6a376640 1 -- 172.21.15.76:0/2255312560 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d64074290 con 0x7f4d6415daa0 2023-11-09T03:47:38.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.734+0000 7f4d60ff9640 1 -- 172.21.15.76:0/2255312560 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f4d5c057b60 con 0x7f4d6415daa0 2023-11-09T03:47:38.734 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:38.735 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5099,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1844478483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1844478483},{"type":"v1","addr":"172.21.15.113:6836","nonce":1844478483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5102,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1111318612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1111318612},{"type":"v1","addr":"172.21.15.113:6837","nonce":1111318612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":6072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/2037805253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2037805253},{"type":"v1","addr":"172.21.15.76:6834","nonce":2037805253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:47:20.265190+0000","modified":"2023-11-09T03:47:22.742442+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6075},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6075":{"gid":6075,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/3710140065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":3710140065},{"type":"v1","addr":"172.21.15.76:6835","nonce":3710140065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-09T03:47:38.735 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 23 2023-11-09T03:47:38.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.736+0000 7f4d6a376640 1 -- 172.21.15.76:0/2255312560 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d5006ae70 msgr2=0x7f4d5006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:38.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.736+0000 7f4d6a376640 1 --2- 172.21.15.76:0/2255312560 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d5006ae70 0x7f4d5006d330 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4d54036000 tx=0x7f4d5403c040 comp rx=0 tx=0).stop 2023-11-09T03:47:38.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.736+0000 7f4d6a376640 1 -- 172.21.15.76:0/2255312560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d6415daa0 msgr2=0x7f4d6414a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:38.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.736+0000 7f4d6a376640 1 --2- 172.21.15.76:0/2255312560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d6415daa0 0x7f4d6414a5c0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00caf0 tx=0x7f4d5c00a060 comp rx=0 tx=0).stop 2023-11-09T03:47:38.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.736+0000 7f4d6a376640 1 -- 172.21.15.76:0/2255312560 shutdown_connections 2023-11-09T03:47:38.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.736+0000 7f4d6a376640 1 --2- 172.21.15.76:0/2255312560 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d5006ae70 0x7f4d5006d330 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.736+0000 7f4d6a376640 1 --2- 172.21.15.76:0/2255312560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d6415daa0 0x7f4d6414a5c0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.736+0000 7f4d6a376640 1 --2- 172.21.15.76:0/2255312560 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d640ce240 0x7f4d640cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.736+0000 7f4d6a376640 1 --2- 172.21.15.76:0/2255312560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d640cde60 0x7f4d640cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.736+0000 7f4d6a376640 1 -- 172.21.15.76:0/2255312560 >> 172.21.15.76:0/2255312560 conn(0x7f4d64071f30 msgr2=0x7f4d640d2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:38.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.736+0000 7f4d6a376640 1 -- 172.21.15.76:0/2255312560 shutdown_connections 2023-11-09T03:47:38.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.736+0000 7f4d6a376640 1 -- 172.21.15.76:0/2255312560 wait complete. 2023-11-09T03:47:38.747 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:47:38.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.847+0000 7fd246a07640 1 Processor -- start 2023-11-09T03:47:38.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.847+0000 7fd246a07640 1 -- start start 2023-11-09T03:47:38.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.848+0000 7fd246a07640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd2400ce370 0x7fd24006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:38.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.848+0000 7fd246a07640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd24006e060 con 0x7fd2400b0ae0 2023-11-09T03:47:38.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.848+0000 7fd246a07640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2400b4110 con 0x7fd240159c70 2023-11-09T03:47:38.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.848+0000 7fd246a07640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2400b49b0 con 0x7fd2400ce370 2023-11-09T03:47:38.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.848+0000 7fd23f7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd2400b0ae0 0x7fd24015d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47276/0 (socket says 172.21.15.76:47276) 2023-11-09T03:47:38.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.848+0000 7fd23f7fe640 1 -- 172.21.15.76:0/2569497259 learned_addr learned my addr 172.21.15.76:0/2569497259 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:38.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.848+0000 7fd244fa4640 1 --2- 172.21.15.76:0/2569497259 >> v2:172.21.15.113:3300/0 conn(0x7fd2400ce370 0x7fd24006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:38.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.848+0000 7fd23effd640 1 -- 172.21.15.76:0/2569497259 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3514458130 0 0) 0x7fd24006e060 con 0x7fd2400b0ae0 2023-11-09T03:47:38.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.848+0000 7fd23effd640 1 -- 172.21.15.76:0/2569497259 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd228003520 con 0x7fd2400b0ae0 2023-11-09T03:47:38.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.848+0000 7fd23effd640 1 -- 172.21.15.76:0/2569497259 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2008239411 0 0) 0x7fd228003520 con 0x7fd2400b0ae0 2023-11-09T03:47:38.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.848+0000 7fd23effd640 1 -- 172.21.15.76:0/2569497259 >> v2:172.21.15.113:3300/0 conn(0x7fd2400ce370 msgr2=0x7fd24006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:38.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.848+0000 7fd23effd640 1 --2- 172.21.15.76:0/2569497259 >> v2:172.21.15.113:3300/0 conn(0x7fd2400ce370 0x7fd24006db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.848+0000 7fd23effd640 1 -- 172.21.15.76:0/2569497259 >> v1:172.21.15.113:6789/0 conn(0x7fd240159c70 legacy=0x7fd2400b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:38.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.848+0000 7fd23effd640 1 -- 172.21.15.76:0/2569497259 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2400b43f0 con 0x7fd2400b0ae0 2023-11-09T03:47:38.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.848+0000 7fd23effd640 1 -- 172.21.15.76:0/2569497259 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fd234001670 con 0x7fd2400b0ae0 2023-11-09T03:47:38.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.848+0000 7fd23effd640 1 -- 172.21.15.76:0/2569497259 >> v1:172.21.15.76:6789/0 conn(0x7fd2400b0ae0 legacy=0x7fd24015d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:38.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.848+0000 7fd23effd640 1 --2- 172.21.15.76:0/2569497259 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd228003970 0x7fd228003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.848+0000 7fd23effd640 1 --2- 172.21.15.76:0/2569497259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2280047c0 0x7fd228004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.848+0000 7fd23effd640 1 --2- 172.21.15.76:0/2569497259 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd228005180 0x7fd22800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.848+0000 7fd23effd640 1 -- 172.21.15.76:0/2569497259 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2400b49b0 con 0x7fd228003970 2023-11-09T03:47:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.848+0000 7fd23effd640 1 -- 172.21.15.76:0/2569497259 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22800c050 con 0x7fd2280047c0 2023-11-09T03:47:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.848+0000 7fd23effd640 1 -- 172.21.15.76:0/2569497259 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd22800c210 con 0x7fd228005180 2023-11-09T03:47:38.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.849+0000 7fd23ffff640 1 --2- 172.21.15.76:0/2569497259 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd228003970 0x7fd228003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:38.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.849+0000 7fd23f7fe640 1 --2- 172.21.15.76:0/2569497259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2280047c0 0x7fd228004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:38.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.849+0000 7fd244fa4640 1 --2- 172.21.15.76:0/2569497259 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd228005180 0x7fd22800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:38.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.849+0000 7fd23ffff640 1 -- 172.21.15.76:0/2569497259 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd228005180 msgr2=0x7fd22800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:38.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.849+0000 7fd23ffff640 1 --2- 172.21.15.76:0/2569497259 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd228005180 0x7fd22800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.849+0000 7fd23ffff640 1 -- 172.21.15.76:0/2569497259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2280047c0 msgr2=0x7fd228004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:38.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.849+0000 7fd23ffff640 1 --2- 172.21.15.76:0/2569497259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2280047c0 0x7fd228004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.849+0000 7fd23ffff640 1 -- 172.21.15.76:0/2569497259 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd228003520 con 0x7fd228003970 2023-11-09T03:47:38.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.849+0000 7fd23f7fe640 1 --2- 172.21.15.76:0/2569497259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2280047c0 0x7fd228004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:38.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.849+0000 7fd244fa4640 1 -- 172.21.15.76:0/2569497259 reap_dead start 2023-11-09T03:47:38.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.849+0000 7fd23ffff640 1 --2- 172.21.15.76:0/2569497259 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd228003970 0x7fd228003db0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd230001750 tx=0x7fd2280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a04b27b8cb27f31 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:38.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.850+0000 7fd23effd640 1 -- 172.21.15.76:0/2569497259 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2300052d0 con 0x7fd228003970 2023-11-09T03:47:38.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.850+0000 7fd23effd640 1 -- 172.21.15.76:0/2569497259 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23002e050 con 0x7fd228003970 2023-11-09T03:47:38.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.850+0000 7fd23effd640 1 -- 172.21.15.76:0/2569497259 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2300035f0 con 0x7fd228003970 2023-11-09T03:47:38.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.850+0000 7fd246a07640 1 -- 172.21.15.76:0/2569497259 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd228003970 msgr2=0x7fd228003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:38.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.850+0000 7fd246a07640 1 --2- 172.21.15.76:0/2569497259 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd228003970 0x7fd228003db0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd230001750 tx=0x7fd2280057f0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.850+0000 7fd246a07640 1 -- 172.21.15.76:0/2569497259 shutdown_connections 2023-11-09T03:47:38.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.850+0000 7fd246a07640 1 --2- 172.21.15.76:0/2569497259 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd228003970 0x7fd228003db0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.850+0000 7fd246a07640 1 -- 172.21.15.76:0/2569497259 >> 172.21.15.76:0/2569497259 conn(0x7fd2400ce950 msgr2=0x7fd2400ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:38.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.850+0000 7fd246a07640 1 -- 172.21.15.76:0/2569497259 shutdown_connections 2023-11-09T03:47:38.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.850+0000 7fd246a07640 1 -- 172.21.15.76:0/2569497259 wait complete. 2023-11-09T03:47:38.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.851+0000 7fd246a07640 1 Processor -- start 2023-11-09T03:47:38.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.852+0000 7fd246a07640 1 -- start start 2023-11-09T03:47:38.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.852+0000 7fd246a07640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd228003970 0x7fd240152470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:38.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.852+0000 7fd246a07640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2401f24b0 0x7fd2401f2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:38.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.852+0000 7fd246a07640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2400c6280 0x7fd2400c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:38.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.852+0000 7fd246a07640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2400ce7e0 con 0x7fd228003970 2023-11-09T03:47:38.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.852+0000 7fd246a07640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2401f3cc0 con 0x7fd2400c6280 2023-11-09T03:47:38.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.852+0000 7fd246a07640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2401f36d0 con 0x7fd2401f24b0 2023-11-09T03:47:38.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.852+0000 7fd23ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd228003970 0x7fd240152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:38.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.852+0000 7fd23ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd228003970 0x7fd240152470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48372/0 (socket says 172.21.15.76:48372) 2023-11-09T03:47:38.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.852+0000 7fd23ffff640 1 -- 172.21.15.76:0/585608515 learned_addr learned my addr 172.21.15.76:0/585608515 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:38.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.852+0000 7fd23f7fe640 1 --2- 172.21.15.76:0/585608515 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2401f24b0 0x7fd2401f2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:38.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.852+0000 7fd244fa4640 1 --2- 172.21.15.76:0/585608515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2400c6280 0x7fd2400c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:38.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.853+0000 7fd23ffff640 1 -- 172.21.15.76:0/585608515 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2401f24b0 msgr2=0x7fd2401f2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:38.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.853+0000 7fd23ffff640 1 --2- 172.21.15.76:0/585608515 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2401f24b0 0x7fd2401f2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.853+0000 7fd23ffff640 1 -- 172.21.15.76:0/585608515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2400c6280 msgr2=0x7fd2400c6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:38.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.853+0000 7fd23ffff640 1 --2- 172.21.15.76:0/585608515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2400c6280 0x7fd2400c6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:38.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.853+0000 7fd23ffff640 1 -- 172.21.15.76:0/585608515 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2401f3040 con 0x7fd228003970 2023-11-09T03:47:38.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.853+0000 7fd23ffff640 1 --2- 172.21.15.76:0/585608515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd228003970 0x7fd240152470 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd230000c00 tx=0x7fd230001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:38.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.853+0000 7fd23cff9640 1 -- 172.21.15.76:0/585608515 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2300076c0 con 0x7fd228003970 2023-11-09T03:47:38.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.853+0000 7fd23cff9640 1 -- 172.21.15.76:0/585608515 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd230007a30 con 0x7fd228003970 2023-11-09T03:47:38.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.853+0000 7fd23cff9640 1 -- 172.21.15.76:0/585608515 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd230006be0 con 0x7fd228003970 2023-11-09T03:47:38.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.853+0000 7fd246a07640 1 -- 172.21.15.76:0/585608515 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2401f3320 con 0x7fd228003970 2023-11-09T03:47:38.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.853+0000 7fd246a07640 1 -- 172.21.15.76:0/585608515 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2401529b0 con 0x7fd228003970 2023-11-09T03:47:38.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.854+0000 7fd246a07640 1 -- 172.21.15.76:0/585608515 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd240074290 con 0x7fd228003970 2023-11-09T03:47:38.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.854+0000 7fd23cff9640 1 -- 172.21.15.76:0/585608515 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd230006d40 con 0x7fd228003970 2023-11-09T03:47:38.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.855+0000 7fd23cff9640 1 --2- 172.21.15.76:0/585608515 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd22c06ae70 0x7fd22c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:38.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.855+0000 7fd23cff9640 1 -- 172.21.15.76:0/585608515 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd2300ab4e0 con 0x7fd228003970 2023-11-09T03:47:38.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.855+0000 7fd23f7fe640 1 --2- 172.21.15.76:0/585608515 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd22c06ae70 0x7fd22c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:38.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.855+0000 7fd23f7fe640 1 --2- 172.21.15.76:0/585608515 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd22c06ae70 0x7fd22c06d330 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd2401566a0 tx=0x7fd234009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:38.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:38.859+0000 7fd23cff9640 1 -- 172.21.15.76:0/585608515 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd2300762f0 con 0x7fd228003970 2023-11-09T03:47:39.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.060+0000 7fd246a07640 1 -- 172.21.15.76:0/585608515 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd24006d3f0 con 0x7fd228003970 2023-11-09T03:47:39.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.061+0000 7fd23cff9640 1 -- 172.21.15.76:0/585608515 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fd230079910 con 0x7fd228003970 2023-11-09T03:47:39.061 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:39.061 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5099,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1844478483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1844478483},{"type":"v1","addr":"172.21.15.113:6836","nonce":1844478483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5102,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1111318612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1111318612},{"type":"v1","addr":"172.21.15.113:6837","nonce":1111318612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":6072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/2037805253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2037805253},{"type":"v1","addr":"172.21.15.76:6834","nonce":2037805253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:47:20.265190+0000","modified":"2023-11-09T03:47:22.742442+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6075},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6075":{"gid":6075,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/3710140065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":3710140065},{"type":"v1","addr":"172.21.15.76:6835","nonce":3710140065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-09T03:47:39.062 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 23 2023-11-09T03:47:39.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.063+0000 7fd246a07640 1 -- 172.21.15.76:0/585608515 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd22c06ae70 msgr2=0x7fd22c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:39.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.063+0000 7fd246a07640 1 --2- 172.21.15.76:0/585608515 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd22c06ae70 0x7fd22c06d330 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd2401566a0 tx=0x7fd234009040 comp rx=0 tx=0).stop 2023-11-09T03:47:39.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.063+0000 7fd246a07640 1 -- 172.21.15.76:0/585608515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd228003970 msgr2=0x7fd240152470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:39.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.063+0000 7fd246a07640 1 --2- 172.21.15.76:0/585608515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd228003970 0x7fd240152470 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd230000c00 tx=0x7fd230001120 comp rx=0 tx=0).stop 2023-11-09T03:47:39.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.063+0000 7fd246a07640 1 -- 172.21.15.76:0/585608515 shutdown_connections 2023-11-09T03:47:39.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.063+0000 7fd246a07640 1 --2- 172.21.15.76:0/585608515 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd22c06ae70 0x7fd22c06d330 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.063+0000 7fd246a07640 1 --2- 172.21.15.76:0/585608515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2400c6280 0x7fd2400c6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.063+0000 7fd246a07640 1 --2- 172.21.15.76:0/585608515 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2401f24b0 0x7fd2401f2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.063+0000 7fd246a07640 1 --2- 172.21.15.76:0/585608515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd228003970 0x7fd240152470 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.063+0000 7fd246a07640 1 -- 172.21.15.76:0/585608515 >> 172.21.15.76:0/585608515 conn(0x7fd2400ce950 msgr2=0x7fd2400ca430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:39.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.063+0000 7fd246a07640 1 -- 172.21.15.76:0/585608515 shutdown_connections 2023-11-09T03:47:39.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.063+0000 7fd246a07640 1 -- 172.21.15.76:0/585608515 wait complete. 2023-11-09T03:47:39.074 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:47:39.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.174+0000 7f682ef97640 1 Processor -- start 2023-11-09T03:47:39.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.174+0000 7f682ef97640 1 -- start start 2023-11-09T03:47:39.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.174+0000 7f682ef97640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f68280cde60 0x7f682815d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:39.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.174+0000 7f682ef97640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68280ce240 0x7f68280d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:39.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.174+0000 7f682ef97640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f682815e000 con 0x7f682815daa0 2023-11-09T03:47:39.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.174+0000 7f682ef97640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f682815e5c0 con 0x7f68280cde60 2023-11-09T03:47:39.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.174+0000 7f682ef97640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f682815e2e0 con 0x7f68280ce240 2023-11-09T03:47:39.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.174+0000 7f682cd33640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f68280cde60 0x7f682815d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:39.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.175+0000 7f682cd33640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f68280cde60 0x7f682815d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48374/0 (socket says 172.21.15.76:48374) 2023-11-09T03:47:39.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.175+0000 7f682cd33640 1 -- 172.21.15.76:0/3917377707 learned_addr learned my addr 172.21.15.76:0/3917377707 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:39.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.175+0000 7f682d534640 1 --2- 172.21.15.76:0/3917377707 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68280ce240 0x7f68280d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:39.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.175+0000 7f682cd33640 1 -- 172.21.15.76:0/3917377707 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68280ce240 msgr2=0x7f68280d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:39.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.175+0000 7f682cd33640 1 --2- 172.21.15.76:0/3917377707 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68280ce240 0x7f68280d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.175+0000 7f682cd33640 1 -- 172.21.15.76:0/3917377707 >> v1:172.21.15.113:6789/0 conn(0x7f682815daa0 legacy=0x7f6828157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:39.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.175+0000 7f682cd33640 1 -- 172.21.15.76:0/3917377707 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68280d1120 con 0x7f68280cde60 2023-11-09T03:47:39.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.175+0000 7f682cd33640 1 --2- 172.21.15.76:0/3917377707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68280cde60 0x7f682815d560 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f681c009870 tx=0x7f681c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac98e7d422f3f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:39.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.176+0000 7f68277fe640 1 -- 172.21.15.76:0/3917377707 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c031640 con 0x7f68280cde60 2023-11-09T03:47:39.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.176+0000 7f68277fe640 1 -- 172.21.15.76:0/3917377707 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681c0317a0 con 0x7f68280cde60 2023-11-09T03:47:39.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.176+0000 7f68277fe640 1 -- 172.21.15.76:0/3917377707 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c031b30 con 0x7f68280cde60 2023-11-09T03:47:39.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.176+0000 7f682ef97640 1 -- 172.21.15.76:0/3917377707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68280cde60 msgr2=0x7f682815d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:39.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.176+0000 7f682ef97640 1 --2- 172.21.15.76:0/3917377707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68280cde60 0x7f682815d560 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f681c009870 tx=0x7f681c02b8f0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.176+0000 7f682ef97640 1 -- 172.21.15.76:0/3917377707 shutdown_connections 2023-11-09T03:47:39.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.176+0000 7f682ef97640 1 --2- 172.21.15.76:0/3917377707 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68280ce240 0x7f68280d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.176+0000 7f682ef97640 1 --2- 172.21.15.76:0/3917377707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68280cde60 0x7f682815d560 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.176+0000 7f682ef97640 1 -- 172.21.15.76:0/3917377707 >> 172.21.15.76:0/3917377707 conn(0x7f6828071f30 msgr2=0x7f68280cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:39.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.176+0000 7f682ef97640 1 -- 172.21.15.76:0/3917377707 shutdown_connections 2023-11-09T03:47:39.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.176+0000 7f682ef97640 1 -- 172.21.15.76:0/3917377707 wait complete. 2023-11-09T03:47:39.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.177+0000 7f682ef97640 1 Processor -- start 2023-11-09T03:47:39.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.177+0000 7f682ef97640 1 -- start start 2023-11-09T03:47:39.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.178+0000 7f682ef97640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68280cde60 0x7f682815aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:39.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.178+0000 7f682ef97640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68280ce240 0x7f682815c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:39.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.178+0000 7f682ef97640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f682815daa0 0x7f68280cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:39.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.178+0000 7f682ef97640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6828140f00 con 0x7f682815daa0 2023-11-09T03:47:39.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.178+0000 7f682ef97640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6828157bc0 con 0x7f68280ce240 2023-11-09T03:47:39.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.178+0000 7f682ef97640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6828157ea0 con 0x7f68280cde60 2023-11-09T03:47:39.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.178+0000 7f682d534640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f682815daa0 0x7f68280cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:39.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.178+0000 7f682d534640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f682815daa0 0x7f68280cbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48376/0 (socket says 172.21.15.76:48376) 2023-11-09T03:47:39.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.178+0000 7f682d534640 1 -- 172.21.15.76:0/1983724150 learned_addr learned my addr 172.21.15.76:0/1983724150 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:39.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.179+0000 7f682cd33640 1 --2- 172.21.15.76:0/1983724150 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68280cde60 0x7f682815aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:39.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.179+0000 7f682d534640 1 -- 172.21.15.76:0/1983724150 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68280cde60 msgr2=0x7f682815aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:39.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.179+0000 7f682d534640 1 --2- 172.21.15.76:0/1983724150 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68280cde60 0x7f682815aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.179+0000 7f6827fff640 1 --2- 172.21.15.76:0/1983724150 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68280ce240 0x7f682815c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:39.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.179+0000 7f682d534640 1 -- 172.21.15.76:0/1983724150 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68280ce240 msgr2=0x7f682815c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:39.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.179+0000 7f682d534640 1 --2- 172.21.15.76:0/1983724150 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68280ce240 0x7f682815c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.179+0000 7f682d534640 1 -- 172.21.15.76:0/1983724150 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681c0094d0 con 0x7f682815daa0 2023-11-09T03:47:39.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.179+0000 7f682d534640 1 --2- 172.21.15.76:0/1983724150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f682815daa0 0x7f68280cbf00 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f6820007e30 tx=0x7f682000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:39.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.179+0000 7f68257fa640 1 -- 172.21.15.76:0/1983724150 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6820014620 con 0x7f682815daa0 2023-11-09T03:47:39.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.179+0000 7f68257fa640 1 -- 172.21.15.76:0/1983724150 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682000ee90 con 0x7f682815daa0 2023-11-09T03:47:39.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.179+0000 7f68257fa640 1 -- 172.21.15.76:0/1983724150 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68200149c0 con 0x7f682815daa0 2023-11-09T03:47:39.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.179+0000 7f682ef97640 1 -- 172.21.15.76:0/1983724150 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6828158180 con 0x7f682815daa0 2023-11-09T03:47:39.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.179+0000 7f682ef97640 1 -- 172.21.15.76:0/1983724150 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68281577e0 con 0x7f682815daa0 2023-11-09T03:47:39.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.180+0000 7f682ef97640 1 -- 172.21.15.76:0/1983724150 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6828158370 con 0x7f682815daa0 2023-11-09T03:47:39.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.181+0000 7f68257fa640 1 -- 172.21.15.76:0/1983724150 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6820014b20 con 0x7f682815daa0 2023-11-09T03:47:39.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.181+0000 7f68257fa640 1 --2- 172.21.15.76:0/1983724150 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f680c06ae70 0x7f680c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:39.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.181+0000 7f68257fa640 1 -- 172.21.15.76:0/1983724150 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f682008a210 con 0x7f682815daa0 2023-11-09T03:47:39.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.181+0000 7f682cd33640 1 --2- 172.21.15.76:0/1983724150 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f680c06ae70 0x7f680c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:39.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.182+0000 7f682cd33640 1 --2- 172.21.15.76:0/1983724150 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f680c06ae70 0x7f680c06d330 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f681c036000 tx=0x7f681c03c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:39.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.185+0000 7f68257fa640 1 -- 172.21.15.76:0/1983724150 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6828158370 con 0x7f682815daa0 2023-11-09T03:47:39.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.353+0000 7f682ef97640 1 -- 172.21.15.76:0/1983724150 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6828141350 con 0x7f682815daa0 2023-11-09T03:47:39.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.354+0000 7f68257fa640 1 -- 172.21.15.76:0/1983724150 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+18511 (secure 0 0 0) 0x7f6828141350 con 0x7f682815daa0 2023-11-09T03:47:39.355 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:39.355 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":54,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:47:21.936243+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:47:18.807110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-09T03:47:19.235022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:47:39.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.356+0000 7f682ef97640 1 -- 172.21.15.76:0/1983724150 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f680c06ae70 msgr2=0x7f680c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:39.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.356+0000 7f682ef97640 1 --2- 172.21.15.76:0/1983724150 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f680c06ae70 0x7f680c06d330 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f681c036000 tx=0x7f681c03c040 comp rx=0 tx=0).stop 2023-11-09T03:47:39.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.356+0000 7f682ef97640 1 -- 172.21.15.76:0/1983724150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f682815daa0 msgr2=0x7f68280cbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:39.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.356+0000 7f682ef97640 1 --2- 172.21.15.76:0/1983724150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f682815daa0 0x7f68280cbf00 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f6820007e30 tx=0x7f682000e4b0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.356+0000 7f682ef97640 1 -- 172.21.15.76:0/1983724150 shutdown_connections 2023-11-09T03:47:39.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.356+0000 7f682ef97640 1 --2- 172.21.15.76:0/1983724150 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f680c06ae70 0x7f680c06d330 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.356+0000 7f682ef97640 1 --2- 172.21.15.76:0/1983724150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f682815daa0 0x7f68280cbf00 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.356+0000 7f682ef97640 1 --2- 172.21.15.76:0/1983724150 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68280ce240 0x7f682815c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.356+0000 7f682ef97640 1 --2- 172.21.15.76:0/1983724150 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68280cde60 0x7f682815aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.356+0000 7f682ef97640 1 -- 172.21.15.76:0/1983724150 >> 172.21.15.76:0/1983724150 conn(0x7f6828071f30 msgr2=0x7f68280b6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:39.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.356+0000 7f682ef97640 1 -- 172.21.15.76:0/1983724150 shutdown_connections 2023-11-09T03:47:39.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.356+0000 7f682ef97640 1 -- 172.21.15.76:0/1983724150 wait complete. 2023-11-09T03:47:39.368 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-09T03:47:39.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.469+0000 7fb206289640 1 Processor -- start 2023-11-09T03:47:39.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.469+0000 7fb206289640 1 -- start start 2023-11-09T03:47:39.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.470+0000 7fb206289640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb2000cccc0 0x7fb20006c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:39.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.470+0000 7fb206289640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb20006ca40 con 0x7fb200160430 2023-11-09T03:47:39.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.470+0000 7fb206289640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb20006cd20 con 0x7fb2000b2d30 2023-11-09T03:47:39.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.470+0000 7fb206289640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2000cb580 con 0x7fb2000cccc0 2023-11-09T03:47:39.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.470+0000 7fb1fffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb200160430 0x7fb2000b2620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47284/0 (socket says 172.21.15.76:47284) 2023-11-09T03:47:39.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.470+0000 7fb1fffff640 1 -- 172.21.15.76:0/2507526670 learned_addr learned my addr 172.21.15.76:0/2507526670 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:39.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.471+0000 7fb1feffd640 1 -- 172.21.15.76:0/2507526670 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2451723236 0 0) 0x7fb20006ca40 con 0x7fb200160430 2023-11-09T03:47:39.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.471+0000 7fb1feffd640 1 -- 172.21.15.76:0/2507526670 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1e8003520 con 0x7fb200160430 2023-11-09T03:47:39.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.471+0000 7fb204826640 1 --2- 172.21.15.76:0/2507526670 >> v2:172.21.15.113:3300/0 conn(0x7fb2000cccc0 0x7fb20006c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:39.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.471+0000 7fb1feffd640 1 -- 172.21.15.76:0/2507526670 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 710440650 0 0) 0x7fb1e8003520 con 0x7fb200160430 2023-11-09T03:47:39.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.471+0000 7fb1feffd640 1 -- 172.21.15.76:0/2507526670 >> v2:172.21.15.113:3300/0 conn(0x7fb2000cccc0 msgr2=0x7fb20006c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:39.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.471+0000 7fb1feffd640 1 --2- 172.21.15.76:0/2507526670 >> v2:172.21.15.113:3300/0 conn(0x7fb2000cccc0 0x7fb20006c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.471+0000 7fb1feffd640 1 -- 172.21.15.76:0/2507526670 >> v1:172.21.15.113:6789/0 conn(0x7fb2000b2d30 legacy=0x7fb20006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:39.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.471+0000 7fb1feffd640 1 -- 172.21.15.76:0/2507526670 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20006d000 con 0x7fb200160430 2023-11-09T03:47:39.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.471+0000 7fb1feffd640 1 -- 172.21.15.76:0/2507526670 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fb1f0001670 con 0x7fb200160430 2023-11-09T03:47:39.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.471+0000 7fb1feffd640 1 -- 172.21.15.76:0/2507526670 >> v1:172.21.15.76:6789/0 conn(0x7fb200160430 legacy=0x7fb2000b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:39.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.471+0000 7fb1feffd640 1 --2- 172.21.15.76:0/2507526670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1e8003970 0x7fb1e8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:39.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.471+0000 7fb1feffd640 1 --2- 172.21.15.76:0/2507526670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1e80047c0 0x7fb1e8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:39.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.471+0000 7fb1feffd640 1 --2- 172.21.15.76:0/2507526670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1e8005180 0x7fb1e800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:39.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.471+0000 7fb1feffd640 1 -- 172.21.15.76:0/2507526670 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2000cb580 con 0x7fb1e8005180 2023-11-09T03:47:39.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.471+0000 7fb1feffd640 1 -- 172.21.15.76:0/2507526670 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e800c0d0 con 0x7fb1e8003970 2023-11-09T03:47:39.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.471+0000 7fb1feffd640 1 -- 172.21.15.76:0/2507526670 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e800c290 con 0x7fb1e80047c0 2023-11-09T03:47:39.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.471+0000 7fb204826640 1 --2- 172.21.15.76:0/2507526670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1e8005180 0x7fb1e800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:39.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.471+0000 7fb1fffff640 1 --2- 172.21.15.76:0/2507526670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1e8003970 0x7fb1e8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:39.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.471+0000 7fb1ff7fe640 1 --2- 172.21.15.76:0/2507526670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1e80047c0 0x7fb1e8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:39.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.471+0000 7fb204826640 1 -- 172.21.15.76:0/2507526670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1e80047c0 msgr2=0x7fb1e8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:39.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.471+0000 7fb204826640 1 --2- 172.21.15.76:0/2507526670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1e80047c0 0x7fb1e8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.471+0000 7fb204826640 1 -- 172.21.15.76:0/2507526670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1e8003970 msgr2=0x7fb1e8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:39.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.471+0000 7fb204826640 1 --2- 172.21.15.76:0/2507526670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1e8003970 0x7fb1e8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.471+0000 7fb204826640 1 -- 172.21.15.76:0/2507526670 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e8003520 con 0x7fb1e8005180 2023-11-09T03:47:39.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.471+0000 7fb1fffff640 1 --2- 172.21.15.76:0/2507526670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1e8003970 0x7fb1e8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:39.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.472+0000 7fb204826640 1 -- 172.21.15.76:0/2507526670 reap_dead start 2023-11-09T03:47:39.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.472+0000 7fb204826640 1 --2- 172.21.15.76:0/2507526670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1e8005180 0x7fb1e800b980 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8007360 tx=0x7fb1e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18386453dbd75ee server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:39.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.472+0000 7fb1feffd640 1 -- 172.21.15.76:0/2507526670 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f802ccf0 con 0x7fb1e8005180 2023-11-09T03:47:39.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.472+0000 7fb1feffd640 1 -- 172.21.15.76:0/2507526670 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f802ce90 con 0x7fb1e8005180 2023-11-09T03:47:39.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.472+0000 7fb1feffd640 1 -- 172.21.15.76:0/2507526670 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8003bd0 con 0x7fb1e8005180 2023-11-09T03:47:39.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.473+0000 7fb206289640 1 -- 172.21.15.76:0/2507526670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1e8005180 msgr2=0x7fb1e800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:39.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.473+0000 7fb206289640 1 --2- 172.21.15.76:0/2507526670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1e8005180 0x7fb1e800b980 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8007360 tx=0x7fb1e80057f0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.473+0000 7fb206289640 1 -- 172.21.15.76:0/2507526670 shutdown_connections 2023-11-09T03:47:39.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.473+0000 7fb206289640 1 --2- 172.21.15.76:0/2507526670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1e8005180 0x7fb1e800b980 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.473+0000 7fb206289640 1 -- 172.21.15.76:0/2507526670 >> 172.21.15.76:0/2507526670 conn(0x7fb20006ea20 msgr2=0x7fb20006ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:39.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.473+0000 7fb206289640 1 -- 172.21.15.76:0/2507526670 shutdown_connections 2023-11-09T03:47:39.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.473+0000 7fb206289640 1 -- 172.21.15.76:0/2507526670 wait complete. 2023-11-09T03:47:39.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.474+0000 7fb206289640 1 Processor -- start 2023-11-09T03:47:39.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.474+0000 7fb206289640 1 -- start start 2023-11-09T03:47:39.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.475+0000 7fb206289640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1e8005180 0x7fb2000ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:39.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.475+0000 7fb206289640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb20015f570 0x7fb200154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:39.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.475+0000 7fb206289640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2000cd180 0x7fb2000cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:39.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.475+0000 7fb206289640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb200153b60 con 0x7fb20015f570 2023-11-09T03:47:39.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.475+0000 7fb206289640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20015f9f0 con 0x7fb2000cd180 2023-11-09T03:47:39.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.475+0000 7fb206289640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2000cdb40 con 0x7fb1e8005180 2023-11-09T03:47:39.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.475+0000 7fb1ff7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb20015f570 0x7fb200154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:39.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.475+0000 7fb1ff7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb20015f570 0x7fb200154350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48396/0 (socket says 172.21.15.76:48396) 2023-11-09T03:47:39.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.475+0000 7fb1ff7fe640 1 -- 172.21.15.76:0/3955526455 learned_addr learned my addr 172.21.15.76:0/3955526455 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:39.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.475+0000 7fb1ff7fe640 1 -- 172.21.15.76:0/3955526455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1e8005180 msgr2=0x7fb2000ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:39.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.475+0000 7fb1fffff640 1 --2- 172.21.15.76:0/3955526455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1e8005180 0x7fb2000ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:39.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.475+0000 7fb1ff7fe640 1 --2- 172.21.15.76:0/3955526455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1e8005180 0x7fb2000ca720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.475+0000 7fb204826640 1 --2- 172.21.15.76:0/3955526455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2000cd180 0x7fb2000cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:39.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.475+0000 7fb1ff7fe640 1 -- 172.21.15.76:0/3955526455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2000cd180 msgr2=0x7fb2000cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:39.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.475+0000 7fb1ff7fe640 1 --2- 172.21.15.76:0/3955526455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2000cd180 0x7fb2000cd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.475+0000 7fb1ff7fe640 1 -- 172.21.15.76:0/3955526455 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb200154890 con 0x7fb20015f570 2023-11-09T03:47:39.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.475+0000 7fb1ff7fe640 1 --2- 172.21.15.76:0/3955526455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb20015f570 0x7fb200154350 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb1f400e480 tx=0x7fb1f400e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:39.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.476+0000 7fb1fcff9640 1 -- 172.21.15.76:0/3955526455 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f40052e0 con 0x7fb20015f570 2023-11-09T03:47:39.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.476+0000 7fb1fcff9640 1 -- 172.21.15.76:0/3955526455 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f4005440 con 0x7fb20015f570 2023-11-09T03:47:39.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.476+0000 7fb206289640 1 -- 172.21.15.76:0/3955526455 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2000c8f70 con 0x7fb20015f570 2023-11-09T03:47:39.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.476+0000 7fb1fcff9640 1 -- 172.21.15.76:0/3955526455 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4003830 con 0x7fb20015f570 2023-11-09T03:47:39.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.476+0000 7fb206289640 1 -- 172.21.15.76:0/3955526455 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2000c9530 con 0x7fb20015f570 2023-11-09T03:47:39.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.477+0000 7fb206289640 1 -- 172.21.15.76:0/3955526455 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb200074290 con 0x7fb20015f570 2023-11-09T03:47:39.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.477+0000 7fb1fcff9640 1 -- 172.21.15.76:0/3955526455 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb1f4003990 con 0x7fb20015f570 2023-11-09T03:47:39.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.477+0000 7fb1fcff9640 1 --2- 172.21.15.76:0/3955526455 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb1ec06aea0 0x7fb1ec06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:39.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.477+0000 7fb1fcff9640 1 -- 172.21.15.76:0/3955526455 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb1f4089500 con 0x7fb20015f570 2023-11-09T03:47:39.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.478+0000 7fb1fffff640 1 --2- 172.21.15.76:0/3955526455 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb1ec06aea0 0x7fb1ec06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:39.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.478+0000 7fb1fffff640 1 --2- 172.21.15.76:0/3955526455 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb1ec06aea0 0x7fb1ec06d360 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb2001f5b90 tx=0x7fb1f0000ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:39.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.482+0000 7fb1fcff9640 1 -- 172.21.15.76:0/3955526455 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb1f40543a0 con 0x7fb20015f570 2023-11-09T03:47:39.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.683+0000 7fb206289640 1 -- 172.21.15.76:0/3955526455 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb2000b6b20 con 0x7fb20015f570 2023-11-09T03:47:39.684 INFO:tasks.ceph.mon.a.smithi076.stderr:2023-11-09T03:47:39.684+0000 7fe4a393d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-09T03:47:39.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.685+0000 7fb1fcff9640 1 -- 172.21.15.76:0/3955526455 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb1f40579c0 con 0x7fb20015f570 2023-11-09T03:47:39.686 INFO:teuthology.orchestra.run.smithi076.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-09T03:47:39.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.688+0000 7fb206289640 1 -- 172.21.15.76:0/3955526455 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb1ec06aea0 msgr2=0x7fb1ec06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:39.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.688+0000 7fb206289640 1 --2- 172.21.15.76:0/3955526455 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb1ec06aea0 0x7fb1ec06d360 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb2001f5b90 tx=0x7fb1f0000ed0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.688+0000 7fb206289640 1 -- 172.21.15.76:0/3955526455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb20015f570 msgr2=0x7fb200154350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:39.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.688+0000 7fb206289640 1 --2- 172.21.15.76:0/3955526455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb20015f570 0x7fb200154350 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb1f400e480 tx=0x7fb1f400e940 comp rx=0 tx=0).stop 2023-11-09T03:47:39.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.688+0000 7fb206289640 1 -- 172.21.15.76:0/3955526455 shutdown_connections 2023-11-09T03:47:39.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.688+0000 7fb206289640 1 --2- 172.21.15.76:0/3955526455 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb1ec06aea0 0x7fb1ec06d360 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.688+0000 7fb206289640 1 --2- 172.21.15.76:0/3955526455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2000cd180 0x7fb2000cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.688+0000 7fb206289640 1 --2- 172.21.15.76:0/3955526455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb20015f570 0x7fb200154350 secure :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb1f400e480 tx=0x7fb1f400e940 comp rx=0 tx=0).stop 2023-11-09T03:47:39.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.688+0000 7fb206289640 1 --2- 172.21.15.76:0/3955526455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1e8005180 0x7fb2000ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.688+0000 7fb206289640 1 -- 172.21.15.76:0/3955526455 >> 172.21.15.76:0/3955526455 conn(0x7fb20006ea20 msgr2=0x7fb20015a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:39.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.688+0000 7fb206289640 1 -- 172.21.15.76:0/3955526455 shutdown_connections 2023-11-09T03:47:39.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.688+0000 7fb206289640 1 -- 172.21.15.76:0/3955526455 wait complete. 2023-11-09T03:47:39.691 INFO:tasks.ceph.mds.c.smithi076.stderr: -630> 2023-11-09T03:47:26.195+0000 7fb441e91640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:47:39.701 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-09T03:47:39.713 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2023-11-09T03:47:39.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.804+0000 7f0017d55640 1 Processor -- start 2023-11-09T03:47:39.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.804+0000 7f0017d55640 1 -- start start 2023-11-09T03:47:39.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.805+0000 7f0017d55640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0010153e60 0x7f001015dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:39.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.805+0000 7f0017d55640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f00100cd6d0 0x7f0010161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:39.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.805+0000 7f0017d55640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00100cde60 con 0x7f00100b4880 2023-11-09T03:47:39.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.805+0000 7f0017d55640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0010160f90 con 0x7f00100cd6d0 2023-11-09T03:47:39.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.805+0000 7f0017d55640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f001015e400 con 0x7f0010153e60 2023-11-09T03:47:39.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.805+0000 7f00152f0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f00100cd6d0 0x7f0010161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:39.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.805+0000 7f00152f0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f00100cd6d0 0x7f0010161420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48416/0 (socket says 172.21.15.76:48416) 2023-11-09T03:47:39.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.805+0000 7f00152f0640 1 -- 172.21.15.76:0/3815548541 learned_addr learned my addr 172.21.15.76:0/3815548541 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:39.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.806+0000 7f0015af1640 1 --2- 172.21.15.76:0/3815548541 >> v2:172.21.15.113:3300/0 conn(0x7f0010153e60 0x7f001015dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:39.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.806+0000 7f00152f0640 1 -- 172.21.15.76:0/3815548541 >> v2:172.21.15.113:3300/0 conn(0x7f0010153e60 msgr2=0x7f001015dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:39.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.806+0000 7f00152f0640 1 --2- 172.21.15.76:0/3815548541 >> v2:172.21.15.113:3300/0 conn(0x7f0010153e60 0x7f001015dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.806+0000 7f00152f0640 1 -- 172.21.15.76:0/3815548541 >> v1:172.21.15.113:6789/0 conn(0x7f00100b4880 legacy=0x7f001006b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:39.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.806+0000 7f00152f0640 1 -- 172.21.15.76:0/3815548541 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0010161960 con 0x7f00100cd6d0 2023-11-09T03:47:39.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.806+0000 7f0015af1640 1 --2- 172.21.15.76:0/3815548541 >> v2:172.21.15.113:3300/0 conn(0x7f0010153e60 0x7f001015dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:39.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.806+0000 7f00152f0640 1 --2- 172.21.15.76:0/3815548541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00100cd6d0 0x7f0010161420 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f001008abe0 tx=0x7f000002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e659cf668b235a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:39.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.807+0000 7f0014aef640 1 -- 172.21.15.76:0/3815548541 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0000031870 con 0x7f00100cd6d0 2023-11-09T03:47:39.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.807+0000 7f0014aef640 1 -- 172.21.15.76:0/3815548541 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00000319d0 con 0x7f00100cd6d0 2023-11-09T03:47:39.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.807+0000 7f0014aef640 1 -- 172.21.15.76:0/3815548541 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0000031d60 con 0x7f00100cd6d0 2023-11-09T03:47:39.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.807+0000 7f0017d55640 1 -- 172.21.15.76:0/3815548541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00100cd6d0 msgr2=0x7f0010161420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:39.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.807+0000 7f0017d55640 1 --2- 172.21.15.76:0/3815548541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00100cd6d0 0x7f0010161420 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f001008abe0 tx=0x7f000002b910 comp rx=0 tx=0).stop 2023-11-09T03:47:39.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.807+0000 7f0017d55640 1 -- 172.21.15.76:0/3815548541 shutdown_connections 2023-11-09T03:47:39.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.807+0000 7f0017d55640 1 --2- 172.21.15.76:0/3815548541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00100cd6d0 0x7f0010161420 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.807+0000 7f0017d55640 1 --2- 172.21.15.76:0/3815548541 >> v2:172.21.15.113:3300/0 conn(0x7f0010153e60 0x7f001015dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.807+0000 7f0017d55640 1 -- 172.21.15.76:0/3815548541 >> 172.21.15.76:0/3815548541 conn(0x7f00100cf880 msgr2=0x7f00100cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:39.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.807+0000 7f0017d55640 1 -- 172.21.15.76:0/3815548541 shutdown_connections 2023-11-09T03:47:39.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.807+0000 7f0017d55640 1 -- 172.21.15.76:0/3815548541 wait complete. 2023-11-09T03:47:39.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.808+0000 7f0017d55640 1 Processor -- start 2023-11-09T03:47:39.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.808+0000 7f0017d55640 1 -- start start 2023-11-09T03:47:39.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.809+0000 7f0017d55640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00100b4880 0x7f00100cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:39.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.809+0000 7f0017d55640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00100cd6d0 0x7f001015f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:39.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.809+0000 7f0017d55640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0010153e60 0x7f001015fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:39.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.809+0000 7f0017d55640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00100cb9b0 con 0x7f0010153e60 2023-11-09T03:47:39.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.809+0000 7f0017d55640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00100cade0 con 0x7f00100cd6d0 2023-11-09T03:47:39.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.809+0000 7f0017d55640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00100cb010 con 0x7f00100b4880 2023-11-09T03:47:39.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.809+0000 7f00162f2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0010153e60 0x7f001015fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:39.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.809+0000 7f00162f2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0010153e60 0x7f001015fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48432/0 (socket says 172.21.15.76:48432) 2023-11-09T03:47:39.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.809+0000 7f00162f2640 1 -- 172.21.15.76:0/3544792549 learned_addr learned my addr 172.21.15.76:0/3544792549 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:39.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.809+0000 7f00162f2640 1 -- 172.21.15.76:0/3544792549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00100b4880 msgr2=0x7f00100cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:39.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.809+0000 7f00162f2640 1 --2- 172.21.15.76:0/3544792549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00100b4880 0x7f00100cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.810+0000 7f00162f2640 1 -- 172.21.15.76:0/3544792549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00100cd6d0 msgr2=0x7f001015f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:39.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.810+0000 7f00162f2640 1 --2- 172.21.15.76:0/3544792549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00100cd6d0 0x7f001015f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:39.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.810+0000 7f00162f2640 1 -- 172.21.15.76:0/3544792549 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00000094d0 con 0x7f0010153e60 2023-11-09T03:47:39.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.810+0000 7f00162f2640 1 --2- 172.21.15.76:0/3544792549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0010153e60 0x7f001015fc00 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f000800c780 tx=0x7f000800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:39.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.810+0000 7f00067fc640 1 -- 172.21.15.76:0/3544792549 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0008004b60 con 0x7f0010153e60 2023-11-09T03:47:39.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.810+0000 7f00067fc640 1 -- 172.21.15.76:0/3544792549 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000800ce90 con 0x7f0010153e60 2023-11-09T03:47:39.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.810+0000 7f0017d55640 1 -- 172.21.15.76:0/3544792549 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00100cb2f0 con 0x7f0010153e60 2023-11-09T03:47:39.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.810+0000 7f00067fc640 1 -- 172.21.15.76:0/3544792549 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000800a850 con 0x7f0010153e60 2023-11-09T03:47:39.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.810+0000 7f0017d55640 1 -- 172.21.15.76:0/3544792549 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00100c9380 con 0x7f0010153e60 2023-11-09T03:47:39.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.811+0000 7f0017d55640 1 -- 172.21.15.76:0/3544792549 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00100cbaf0 con 0x7f0010153e60 2023-11-09T03:47:39.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.811+0000 7f00067fc640 1 -- 172.21.15.76:0/3544792549 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f000800a9b0 con 0x7f0010153e60 2023-11-09T03:47:39.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.812+0000 7f00067fc640 1 --2- 172.21.15.76:0/3544792549 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efff406ae70 0x7efff406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:39.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.812+0000 7f00067fc640 1 -- 172.21.15.76:0/3544792549 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6468+0+0 (secure 0 0 0) 0x7f00080899e0 con 0x7f0010153e60 2023-11-09T03:47:39.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.812+0000 7f0015af1640 1 --2- 172.21.15.76:0/3544792549 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efff406ae70 0x7efff406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:39.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.812+0000 7f0015af1640 1 --2- 172.21.15.76:0/3544792549 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efff406ae70 0x7efff406d330 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f000c006fd0 tx=0x7f000c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:39.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:39.816+0000 7f00067fc640 1 -- 172.21.15.76:0/3544792549 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f00100cbaf0 con 0x7f0010153e60 2023-11-09T03:47:40.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.023+0000 7f0017d55640 1 -- 172.21.15.76:0/3544792549 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f00100cbaf0 con 0x7f0010153e60 2023-11-09T03:47:40.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.689+0000 7f00067fc640 1 -- 172.21.15.76:0/3544792549 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f00100cbaf0 con 0x7f0010153e60 2023-11-09T03:47:40.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.691+0000 7f0017d55640 1 -- 172.21.15.76:0/3544792549 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efff406ae70 msgr2=0x7efff406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:40.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.691+0000 7f0017d55640 1 --2- 172.21.15.76:0/3544792549 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efff406ae70 0x7efff406d330 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f000c006fd0 tx=0x7f000c008040 comp rx=0 tx=0).stop 2023-11-09T03:47:40.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.691+0000 7f0017d55640 1 -- 172.21.15.76:0/3544792549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0010153e60 msgr2=0x7f001015fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:40.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.691+0000 7f0017d55640 1 --2- 172.21.15.76:0/3544792549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0010153e60 0x7f001015fc00 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f000800c780 tx=0x7f000800cc40 comp rx=0 tx=0).stop 2023-11-09T03:47:40.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.691+0000 7f0017d55640 1 -- 172.21.15.76:0/3544792549 shutdown_connections 2023-11-09T03:47:40.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.691+0000 7f0017d55640 1 --2- 172.21.15.76:0/3544792549 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efff406ae70 0x7efff406d330 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:40.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.691+0000 7f0017d55640 1 --2- 172.21.15.76:0/3544792549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0010153e60 0x7f001015fc00 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:40.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.691+0000 7f0017d55640 1 --2- 172.21.15.76:0/3544792549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00100cd6d0 0x7f001015f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:40.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.691+0000 7f0017d55640 1 --2- 172.21.15.76:0/3544792549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00100b4880 0x7f00100cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:40.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.691+0000 7f0017d55640 1 -- 172.21.15.76:0/3544792549 >> 172.21.15.76:0/3544792549 conn(0x7f00100cf880 msgr2=0x7f001015bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:40.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.691+0000 7f0017d55640 1 -- 172.21.15.76:0/3544792549 shutdown_connections 2023-11-09T03:47:40.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.691+0000 7f0017d55640 1 -- 172.21.15.76:0/3544792549 wait complete. 2023-11-09T03:47:40.704 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-09T03:47:40.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.849+0000 7fecd8044640 1 Processor -- start 2023-11-09T03:47:40.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.850+0000 7fecd8044640 1 -- start start 2023-11-09T03:47:40.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.850+0000 7fecd8044640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecd00cb680 0x7fecd00cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:40.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.850+0000 7fecd8044640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fecd00cda80 0x7fecd015d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:40.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.850+0000 7fecd8044640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd0161870 con 0x7fecd015d1b0 2023-11-09T03:47:40.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.850+0000 7fecd8044640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecd00cdf10 con 0x7fecd00cda80 2023-11-09T03:47:40.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.850+0000 7fecd8044640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd0160c60 con 0x7fecd00cb680 2023-11-09T03:47:40.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.851+0000 7fecd5de0640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fecd015d1b0 0x7fecd00ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47286/0 (socket says 172.21.15.76:47286) 2023-11-09T03:47:40.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.851+0000 7fecd5de0640 1 -- 172.21.15.76:0/3435271639 learned_addr learned my addr 172.21.15.76:0/3435271639 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:40.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.851+0000 7fecd55df640 1 --2- 172.21.15.76:0/3435271639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecd00cb680 0x7fecd00cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:40.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.851+0000 7fecd65e1640 1 --2- 172.21.15.76:0/3435271639 >> v2:172.21.15.113:3300/0 conn(0x7fecd00cda80 0x7fecd015d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:40.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.851+0000 7fecd4dde640 1 -- 172.21.15.76:0/3435271639 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2722485082 0 0) 0x7fecd0161870 con 0x7fecd015d1b0 2023-11-09T03:47:40.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.851+0000 7fecd4dde640 1 -- 172.21.15.76:0/3435271639 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecb8003520 con 0x7fecd015d1b0 2023-11-09T03:47:40.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.851+0000 7fecd4dde640 1 -- 172.21.15.76:0/3435271639 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 902441734 0 0) 0x7fecb8003520 con 0x7fecd015d1b0 2023-11-09T03:47:40.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.851+0000 7fecd4dde640 1 -- 172.21.15.76:0/3435271639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecd00cb680 msgr2=0x7fecd00cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:40.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.851+0000 7fecd4dde640 1 --2- 172.21.15.76:0/3435271639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecd00cb680 0x7fecd00cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:40.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.851+0000 7fecd4dde640 1 -- 172.21.15.76:0/3435271639 >> v2:172.21.15.113:3300/0 conn(0x7fecd00cda80 msgr2=0x7fecd015d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:40.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.851+0000 7fecd4dde640 1 --2- 172.21.15.76:0/3435271639 >> v2:172.21.15.113:3300/0 conn(0x7fecd00cda80 0x7fecd015d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:40.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.851+0000 7fecd4dde640 1 -- 172.21.15.76:0/3435271639 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd0160f40 con 0x7fecd015d1b0 2023-11-09T03:47:40.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.851+0000 7fecd4dde640 1 -- 172.21.15.76:0/3435271639 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fecc80019f0 con 0x7fecd015d1b0 2023-11-09T03:47:40.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.851+0000 7fecd4dde640 1 -- 172.21.15.76:0/3435271639 >> v1:172.21.15.76:6789/0 conn(0x7fecd015d1b0 legacy=0x7fecd00ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:40.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.851+0000 7fecd4dde640 1 --2- 172.21.15.76:0/3435271639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fecb8003970 0x7fecb8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:40.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.851+0000 7fecd4dde640 1 --2- 172.21.15.76:0/3435271639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecd00cb680 0x7fecb80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:40.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.851+0000 7fecd4dde640 1 --2- 172.21.15.76:0/3435271639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecb8004d00 0x7fecb8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:40.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.851+0000 7fecd4dde640 1 -- 172.21.15.76:0/3435271639 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd00cdf10 con 0x7fecb8003970 2023-11-09T03:47:40.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.851+0000 7fecd4dde640 1 -- 172.21.15.76:0/3435271639 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd0160c60 con 0x7fecb8004d00 2023-11-09T03:47:40.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.851+0000 7fecd4dde640 1 -- 172.21.15.76:0/3435271639 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb800b980 con 0x7fecd00cb680 2023-11-09T03:47:40.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.851+0000 7fecd5de0640 1 --2- 172.21.15.76:0/3435271639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fecb8003970 0x7fecb8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:40.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.851+0000 7fecd55df640 1 --2- 172.21.15.76:0/3435271639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecd00cb680 0x7fecb80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:40.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.851+0000 7fecd65e1640 1 --2- 172.21.15.76:0/3435271639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecb8004d00 0x7fecb8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:40.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.852+0000 7fecd5de0640 1 -- 172.21.15.76:0/3435271639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecd00cb680 msgr2=0x7fecb80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:40.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.852+0000 7fecd5de0640 1 --2- 172.21.15.76:0/3435271639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecd00cb680 0x7fecb80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:40.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.852+0000 7fecd5de0640 1 -- 172.21.15.76:0/3435271639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecb8004d00 msgr2=0x7fecb8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:40.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.852+0000 7fecd5de0640 1 --2- 172.21.15.76:0/3435271639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecb8004d00 0x7fecb8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:40.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.852+0000 7fecd5de0640 1 -- 172.21.15.76:0/3435271639 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb8003520 con 0x7fecb8003970 2023-11-09T03:47:40.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.852+0000 7fecd65e1640 1 --2- 172.21.15.76:0/3435271639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecb8004d00 0x7fecb8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:40.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.852+0000 7fecd5de0640 1 --2- 172.21.15.76:0/3435271639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fecb8003970 0x7fecb8003d90 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fecc800a860 tx=0x7fecb80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ed29e79f01f6b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:40.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.852+0000 7fecd4dde640 1 -- 172.21.15.76:0/3435271639 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc8034630 con 0x7fecb8003970 2023-11-09T03:47:40.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.852+0000 7fecd4dde640 1 -- 172.21.15.76:0/3435271639 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc802f070 con 0x7fecb8003970 2023-11-09T03:47:40.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.852+0000 7fecd4dde640 1 -- 172.21.15.76:0/3435271639 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc8034a10 con 0x7fecb8003970 2023-11-09T03:47:40.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.853+0000 7fecd8044640 1 -- 172.21.15.76:0/3435271639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fecb8003970 msgr2=0x7fecb8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:40.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.853+0000 7fecd8044640 1 --2- 172.21.15.76:0/3435271639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fecb8003970 0x7fecb8003d90 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fecc800a860 tx=0x7fecb80057f0 comp rx=0 tx=0).stop 2023-11-09T03:47:40.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.853+0000 7fecd65e1640 1 -- 172.21.15.76:0/3435271639 reap_dead start 2023-11-09T03:47:40.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.853+0000 7fecd8044640 1 -- 172.21.15.76:0/3435271639 shutdown_connections 2023-11-09T03:47:40.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.853+0000 7fecd8044640 1 -- 172.21.15.76:0/3435271639 >> 172.21.15.76:0/3435271639 conn(0x7fecd006ea20 msgr2=0x7fecd006ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:40.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.853+0000 7fecd8044640 1 -- 172.21.15.76:0/3435271639 shutdown_connections 2023-11-09T03:47:40.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.853+0000 7fecd8044640 1 -- 172.21.15.76:0/3435271639 wait complete. 2023-11-09T03:47:40.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.854+0000 7fecd8044640 1 Processor -- start 2023-11-09T03:47:40.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.854+0000 7fecd8044640 1 -- start start 2023-11-09T03:47:40.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.855+0000 7fecd8044640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecb8003970 0x7fecd01562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:40.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.855+0000 7fecd8044640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecd00b62d0 0x7fecd00cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:40.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.855+0000 7fecd8044640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fecd00ccda0 0x7fecd01f61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:40.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.855+0000 7fecd8044640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd00cd250 con 0x7fecd00ccda0 2023-11-09T03:47:40.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.855+0000 7fecd8044640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd01f6f40 con 0x7fecb8003970 2023-11-09T03:47:40.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.855+0000 7fecd8044640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd00b5510 con 0x7fecd00b62d0 2023-11-09T03:47:40.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.855+0000 7fecd65e1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fecd00ccda0 0x7fecd01f61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:40.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.855+0000 7fecd65e1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fecd00ccda0 0x7fecd01f61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48446/0 (socket says 172.21.15.76:48446) 2023-11-09T03:47:40.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.855+0000 7fecd65e1640 1 -- 172.21.15.76:0/3321708401 learned_addr learned my addr 172.21.15.76:0/3321708401 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:40.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.855+0000 7fecd55df640 1 --2- 172.21.15.76:0/3321708401 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecd00b62d0 0x7fecd00cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:40.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.855+0000 7fecd5de0640 1 --2- 172.21.15.76:0/3321708401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecb8003970 0x7fecd01562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:40.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.855+0000 7fecd65e1640 1 -- 172.21.15.76:0/3321708401 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecd00b62d0 msgr2=0x7fecd00cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:40.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.855+0000 7fecd65e1640 1 --2- 172.21.15.76:0/3321708401 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecd00b62d0 0x7fecd00cc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:40.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.855+0000 7fecd65e1640 1 -- 172.21.15.76:0/3321708401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecb8003970 msgr2=0x7fecd01562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:40.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.855+0000 7fecd65e1640 1 --2- 172.21.15.76:0/3321708401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecb8003970 0x7fecd01562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:40.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.855+0000 7fecd65e1640 1 -- 172.21.15.76:0/3321708401 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd00b57f0 con 0x7fecd00ccda0 2023-11-09T03:47:40.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.855+0000 7fecd65e1640 1 --2- 172.21.15.76:0/3321708401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fecd00ccda0 0x7fecd01f61d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fecd006f950 tx=0x7fecc0011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:40.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.856+0000 7fecce7fc640 1 -- 172.21.15.76:0/3321708401 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc0004500 con 0x7fecd00ccda0 2023-11-09T03:47:40.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.856+0000 7fecce7fc640 1 -- 172.21.15.76:0/3321708401 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc0003610 con 0x7fecd00ccda0 2023-11-09T03:47:40.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.856+0000 7fecd8044640 1 -- 172.21.15.76:0/3321708401 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd00b5ad0 con 0x7fecd00ccda0 2023-11-09T03:47:40.869 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.856+0000 7fecce7fc640 1 -- 172.21.15.76:0/3321708401 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc00039c0 con 0x7fecd00ccda0 2023-11-09T03:47:40.869 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.856+0000 7fecd8044640 1 -- 172.21.15.76:0/3321708401 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd0156830 con 0x7fecd00ccda0 2023-11-09T03:47:40.869 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.857+0000 7fecd8044640 1 -- 172.21.15.76:0/3321708401 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecd00b5ff0 con 0x7fecd00ccda0 2023-11-09T03:47:40.869 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.857+0000 7fecce7fc640 1 -- 172.21.15.76:0/3321708401 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fecc0003b20 con 0x7fecd00ccda0 2023-11-09T03:47:40.870 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.857+0000 7fecce7fc640 1 --2- 172.21.15.76:0/3321708401 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fecbc06aea0 0x7fecbc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:40.870 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.858+0000 7fecce7fc640 1 -- 172.21.15.76:0/3321708401 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6468+0+0 (secure 0 0 0) 0x7fecc0089bd0 con 0x7fecd00ccda0 2023-11-09T03:47:40.870 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.858+0000 7fecd5de0640 1 --2- 172.21.15.76:0/3321708401 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fecbc06aea0 0x7fecbc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:40.870 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.858+0000 7fecd5de0640 1 --2- 172.21.15.76:0/3321708401 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fecbc06aea0 0x7fecbc06d360 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fecc8034000 tx=0x7fecc8009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:40.870 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:40.862+0000 7fecce7fc640 1 -- 172.21.15.76:0/3321708401 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fecd00b5ff0 con 0x7fecd00ccda0 2023-11-09T03:47:41.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.030+0000 7fecd8044640 1 -- 172.21.15.76:0/3321708401 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fecd00b5ff0 con 0x7fecd00ccda0 2023-11-09T03:47:41.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.697+0000 7fecce7fc640 1 -- 172.21.15.76:0/3321708401 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v56) v1 ==== 181+0+0 (secure 0 0 0) 0x7fecd00b5ff0 con 0x7fecd00ccda0 2023-11-09T03:47:41.698 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' removed 2023-11-09T03:47:41.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.700+0000 7fecd8044640 1 -- 172.21.15.76:0/3321708401 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fecbc06aea0 msgr2=0x7fecbc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:41.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.700+0000 7fecd8044640 1 --2- 172.21.15.76:0/3321708401 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fecbc06aea0 0x7fecbc06d360 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fecc8034000 tx=0x7fecc8009030 comp rx=0 tx=0).stop 2023-11-09T03:47:41.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.700+0000 7fecd8044640 1 -- 172.21.15.76:0/3321708401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fecd00ccda0 msgr2=0x7fecd01f61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:41.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.700+0000 7fecd8044640 1 --2- 172.21.15.76:0/3321708401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fecd00ccda0 0x7fecd01f61d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fecd006f950 tx=0x7fecc0011920 comp rx=0 tx=0).stop 2023-11-09T03:47:41.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.700+0000 7fecd8044640 1 -- 172.21.15.76:0/3321708401 shutdown_connections 2023-11-09T03:47:41.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.700+0000 7fecd8044640 1 --2- 172.21.15.76:0/3321708401 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fecbc06aea0 0x7fecbc06d360 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:41.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.700+0000 7fecd8044640 1 --2- 172.21.15.76:0/3321708401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fecd00ccda0 0x7fecd01f61d0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:41.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.700+0000 7fecd8044640 1 --2- 172.21.15.76:0/3321708401 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecd00b62d0 0x7fecd00cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:41.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.700+0000 7fecd8044640 1 --2- 172.21.15.76:0/3321708401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecb8003970 0x7fecd01562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:41.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.700+0000 7fecd8044640 1 -- 172.21.15.76:0/3321708401 >> 172.21.15.76:0/3321708401 conn(0x7fecd006ea20 msgr2=0x7fecd006c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:41.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.700+0000 7fecd8044640 1 -- 172.21.15.76:0/3321708401 shutdown_connections 2023-11-09T03:47:41.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.700+0000 7fecd8044640 1 -- 172.21.15.76:0/3321708401 wait complete. 2023-11-09T03:47:41.714 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-09T03:47:41.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.820+0000 7f5424bd7640 1 Processor -- start 2023-11-09T03:47:41.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.820+0000 7f5424bd7640 1 -- start start 2023-11-09T03:47:41.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.820+0000 7f5424bd7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f542015e420 0x7f54200ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:41.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.820+0000 7f5424bd7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54200c5890 con 0x7f54200adc70 2023-11-09T03:47:41.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.820+0000 7f5424bd7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54200cd8e0 con 0x7f5420159be0 2023-11-09T03:47:41.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.820+0000 7f5424bd7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54200c5670 con 0x7f542015e420 2023-11-09T03:47:41.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.821+0000 7f541ed9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f54200adc70 0x7f54200c3840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47290/0 (socket says 172.21.15.76:47290) 2023-11-09T03:47:41.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.821+0000 7f541ed9d640 1 -- 172.21.15.76:0/5946599 learned_addr learned my addr 172.21.15.76:0/5946599 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:41.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.821+0000 7f5415d9b640 1 --2- 172.21.15.76:0/5946599 >> v2:172.21.15.113:3300/0 conn(0x7f542015e420 0x7f54200ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:41.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.821+0000 7f541dd9b640 1 -- 172.21.15.76:0/5946599 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1584503761 0 0) 0x7f54200c5890 con 0x7f54200adc70 2023-11-09T03:47:41.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.821+0000 7f541dd9b640 1 -- 172.21.15.76:0/5946599 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f540c003520 con 0x7f54200adc70 2023-11-09T03:47:41.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.821+0000 7f541dd9b640 1 -- 172.21.15.76:0/5946599 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3455419253 0 0) 0x7f540c003520 con 0x7f54200adc70 2023-11-09T03:47:41.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.821+0000 7f541dd9b640 1 -- 172.21.15.76:0/5946599 >> v2:172.21.15.113:3300/0 conn(0x7f542015e420 msgr2=0x7f54200ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:41.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.821+0000 7f541dd9b640 1 --2- 172.21.15.76:0/5946599 >> v2:172.21.15.113:3300/0 conn(0x7f542015e420 0x7f54200ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:41.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.821+0000 7f541dd9b640 1 -- 172.21.15.76:0/5946599 >> v1:172.21.15.113:6789/0 conn(0x7f5420159be0 legacy=0x7f54200c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:41.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.821+0000 7f541dd9b640 1 -- 172.21.15.76:0/5946599 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54200cdbc0 con 0x7f54200adc70 2023-11-09T03:47:41.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.821+0000 7f541dd9b640 1 -- 172.21.15.76:0/5946599 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f54180019f0 con 0x7f54200adc70 2023-11-09T03:47:41.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.821+0000 7f541dd9b640 1 -- 172.21.15.76:0/5946599 >> v1:172.21.15.76:6789/0 conn(0x7f54200adc70 legacy=0x7f54200c3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:41.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.821+0000 7f541dd9b640 1 --2- 172.21.15.76:0/5946599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f540c003970 0x7f540c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:41.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.821+0000 7f541dd9b640 1 --2- 172.21.15.76:0/5946599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f540c0047c0 0x7f540c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:41.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.821+0000 7f541dd9b640 1 --2- 172.21.15.76:0/5946599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f540c005180 0x7f540c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:41.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.821+0000 7f541dd9b640 1 -- 172.21.15.76:0/5946599 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54200c5670 con 0x7f540c005180 2023-11-09T03:47:41.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.821+0000 7f541dd9b640 1 -- 172.21.15.76:0/5946599 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540c00c120 con 0x7f540c0047c0 2023-11-09T03:47:41.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.821+0000 7f541dd9b640 1 -- 172.21.15.76:0/5946599 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540c00c2e0 con 0x7f540c003970 2023-11-09T03:47:41.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.821+0000 7f541ed9d640 1 --2- 172.21.15.76:0/5946599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f540c005180 0x7f540c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:41.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.821+0000 7f541e59c640 1 --2- 172.21.15.76:0/5946599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f540c003970 0x7f540c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:41.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.821+0000 7f5415d9b640 1 --2- 172.21.15.76:0/5946599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f540c0047c0 0x7f540c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:41.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.821+0000 7f541ed9d640 1 -- 172.21.15.76:0/5946599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f540c003970 msgr2=0x7f540c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:41.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.821+0000 7f541ed9d640 1 --2- 172.21.15.76:0/5946599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f540c003970 0x7f540c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:41.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.821+0000 7f541ed9d640 1 -- 172.21.15.76:0/5946599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f540c0047c0 msgr2=0x7f540c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:41.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.821+0000 7f541ed9d640 1 --2- 172.21.15.76:0/5946599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f540c0047c0 0x7f540c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:41.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.821+0000 7f541ed9d640 1 -- 172.21.15.76:0/5946599 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540c003520 con 0x7f540c005180 2023-11-09T03:47:41.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.822+0000 7f541ed9d640 1 -- 172.21.15.76:0/5946599 reap_dead start 2023-11-09T03:47:41.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.822+0000 7f541ed9d640 1 --2- 172.21.15.76:0/5946599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f540c005180 0x7f540c00b980 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f541800a830 tx=0x7f540c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e918252abc13a89 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:41.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.822+0000 7f541dd9b640 1 -- 172.21.15.76:0/5946599 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54180072d0 con 0x7f540c005180 2023-11-09T03:47:41.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.822+0000 7f541dd9b640 1 -- 172.21.15.76:0/5946599 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541802f070 con 0x7f540c005180 2023-11-09T03:47:41.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.822+0000 7f541dd9b640 1 -- 172.21.15.76:0/5946599 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5418007690 con 0x7f540c005180 2023-11-09T03:47:41.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.823+0000 7f5424bd7640 1 -- 172.21.15.76:0/5946599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f540c005180 msgr2=0x7f540c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:41.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.823+0000 7f5424bd7640 1 --2- 172.21.15.76:0/5946599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f540c005180 0x7f540c00b980 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f541800a830 tx=0x7f540c0057f0 comp rx=0 tx=0).stop 2023-11-09T03:47:41.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.823+0000 7f5424bd7640 1 -- 172.21.15.76:0/5946599 shutdown_connections 2023-11-09T03:47:41.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.823+0000 7f5424bd7640 1 --2- 172.21.15.76:0/5946599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f540c005180 0x7f540c00b980 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:41.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.823+0000 7f5424bd7640 1 -- 172.21.15.76:0/5946599 >> 172.21.15.76:0/5946599 conn(0x7f54200b05d0 msgr2=0x7f54200b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:41.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.823+0000 7f5424bd7640 1 -- 172.21.15.76:0/5946599 shutdown_connections 2023-11-09T03:47:41.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.823+0000 7f5424bd7640 1 -- 172.21.15.76:0/5946599 wait complete. 2023-11-09T03:47:41.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.824+0000 7f5424bd7640 1 Processor -- start 2023-11-09T03:47:41.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.824+0000 7f5424bd7640 1 -- start start 2023-11-09T03:47:41.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.825+0000 7f5424bd7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f540c005180 0x7f5420151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:41.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.825+0000 7f5424bd7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f542014a370 0x7f54200b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:41.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.825+0000 7f5424bd7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54200b2c20 0x7f54200cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:41.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.825+0000 7f5424bd7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54200b30a0 con 0x7f542014a370 2023-11-09T03:47:41.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.825+0000 7f5424bd7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54200b3380 con 0x7f540c005180 2023-11-09T03:47:41.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.825+0000 7f5424bd7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54200b3660 con 0x7f54200b2c20 2023-11-09T03:47:41.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.825+0000 7f5415d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f542014a370 0x7f54200b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:41.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.825+0000 7f5415d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f542014a370 0x7f54200b26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48464/0 (socket says 172.21.15.76:48464) 2023-11-09T03:47:41.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.825+0000 7f5415d9b640 1 -- 172.21.15.76:0/209925849 learned_addr learned my addr 172.21.15.76:0/209925849 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:41.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.825+0000 7f541e59c640 1 --2- 172.21.15.76:0/209925849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f540c005180 0x7f5420151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:41.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.825+0000 7f541ed9d640 1 --2- 172.21.15.76:0/209925849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54200b2c20 0x7f54200cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:41.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.825+0000 7f5415d9b640 1 -- 172.21.15.76:0/209925849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54200b2c20 msgr2=0x7f54200cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:41.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.825+0000 7f5415d9b640 1 --2- 172.21.15.76:0/209925849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54200b2c20 0x7f54200cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:41.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.825+0000 7f5415d9b640 1 -- 172.21.15.76:0/209925849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f540c005180 msgr2=0x7f5420151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:41.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.825+0000 7f5415d9b640 1 --2- 172.21.15.76:0/209925849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f540c005180 0x7f5420151960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:41.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.825+0000 7f5415d9b640 1 -- 172.21.15.76:0/209925849 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54200c5c10 con 0x7f542014a370 2023-11-09T03:47:41.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.825+0000 7f541ed9d640 1 --2- 172.21.15.76:0/209925849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54200b2c20 0x7f54200cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:41.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.826+0000 7f5415d9b640 1 --2- 172.21.15.76:0/209925849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f542014a370 0x7f54200b26e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f5408011590 tx=0x7f5408011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:41.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.826+0000 7f54177fe640 1 -- 172.21.15.76:0/209925849 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5408004220 con 0x7f542014a370 2023-11-09T03:47:41.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.826+0000 7f54177fe640 1 -- 172.21.15.76:0/209925849 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5408004380 con 0x7f542014a370 2023-11-09T03:47:41.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.826+0000 7f54177fe640 1 -- 172.21.15.76:0/209925849 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5408015930 con 0x7f542014a370 2023-11-09T03:47:41.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.826+0000 7f5424bd7640 1 -- 172.21.15.76:0/209925849 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54200b3d80 con 0x7f542014a370 2023-11-09T03:47:41.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.826+0000 7f5424bd7640 1 -- 172.21.15.76:0/209925849 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542014a9b0 con 0x7f542014a370 2023-11-09T03:47:41.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.827+0000 7f5424bd7640 1 -- 172.21.15.76:0/209925849 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54200c5a70 con 0x7f542014a370 2023-11-09T03:47:41.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.827+0000 7f54177fe640 1 -- 172.21.15.76:0/209925849 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5408015a90 con 0x7f542014a370 2023-11-09T03:47:41.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.827+0000 7f54177fe640 1 --2- 172.21.15.76:0/209925849 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f540006ae70 0x7f540006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:41.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.828+0000 7f54177fe640 1 -- 172.21.15.76:0/209925849 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6030+0+0 (secure 0 0 0) 0x7f5408089790 con 0x7f542014a370 2023-11-09T03:47:41.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.828+0000 7f541e59c640 1 --2- 172.21.15.76:0/209925849 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f540006ae70 0x7f540006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:41.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.828+0000 7f541e59c640 1 --2- 172.21.15.76:0/209925849 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f540006ae70 0x7f540006d330 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5410007fd0 tx=0x7f5410009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:41.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:41.832+0000 7f54177fe640 1 -- 172.21.15.76:0/209925849 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f54200c5a70 con 0x7f542014a370 2023-11-09T03:47:42.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.003+0000 7f5424bd7640 1 -- 172.21.15.76:0/209925849 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f54200c5a70 con 0x7f542014a370 2023-11-09T03:47:42.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.699+0000 7f54177fe640 1 -- 172.21.15.76:0/209925849 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v57) v1 ==== 169+0+0 (secure 0 0 0) 0x7f54200c5a70 con 0x7f542014a370 2023-11-09T03:47:42.700 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' removed 2023-11-09T03:47:42.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.704+0000 7f5424bd7640 1 -- 172.21.15.76:0/209925849 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f540006ae70 msgr2=0x7f540006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:42.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.704+0000 7f5424bd7640 1 --2- 172.21.15.76:0/209925849 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f540006ae70 0x7f540006d330 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5410007fd0 tx=0x7f5410009040 comp rx=0 tx=0).stop 2023-11-09T03:47:42.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.704+0000 7f5424bd7640 1 -- 172.21.15.76:0/209925849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f542014a370 msgr2=0x7f54200b26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:42.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.704+0000 7f5424bd7640 1 --2- 172.21.15.76:0/209925849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f542014a370 0x7f54200b26e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f5408011590 tx=0x7f5408011a50 comp rx=0 tx=0).stop 2023-11-09T03:47:42.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.705+0000 7f5424bd7640 1 -- 172.21.15.76:0/209925849 shutdown_connections 2023-11-09T03:47:42.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.705+0000 7f5424bd7640 1 --2- 172.21.15.76:0/209925849 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f540006ae70 0x7f540006d330 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:42.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.705+0000 7f5424bd7640 1 --2- 172.21.15.76:0/209925849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54200b2c20 0x7f54200cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:42.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.705+0000 7f5424bd7640 1 --2- 172.21.15.76:0/209925849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f542014a370 0x7f54200b26e0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:42.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.705+0000 7f5424bd7640 1 --2- 172.21.15.76:0/209925849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f540c005180 0x7f5420151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:42.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.705+0000 7f5424bd7640 1 -- 172.21.15.76:0/209925849 >> 172.21.15.76:0/209925849 conn(0x7f54200b05d0 msgr2=0x7f54200cb730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:42.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.705+0000 7f5424bd7640 1 -- 172.21.15.76:0/209925849 shutdown_connections 2023-11-09T03:47:42.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.705+0000 7f5424bd7640 1 -- 172.21.15.76:0/209925849 wait complete. 2023-11-09T03:47:42.722 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-09T03:47:42.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.829+0000 7f73b9826640 1 Processor -- start 2023-11-09T03:47:42.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.829+0000 7f73b9826640 1 -- start start 2023-11-09T03:47:42.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.829+0000 7f73b9826640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f73b4160430 0x7f73b40b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:42.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.829+0000 7f73b9826640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f73b40b2b60 0x7f73b406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:42.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.829+0000 7f73b9826640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73b406c1d0 0x7f73b4160b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:42.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.829+0000 7f73b9826640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73b40cb580 con 0x7f73b40b2b60 2023-11-09T03:47:42.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.829+0000 7f73b9826640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73b40b2fc0 con 0x7f73b4160430 2023-11-09T03:47:42.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.829+0000 7f73b9826640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b406ca40 con 0x7f73b406c1d0 2023-11-09T03:47:42.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.829+0000 7f73b27fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f73b40b2b60 0x7f73b406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:42.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.830+0000 7f73b27fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f73b40b2b60 0x7f73b406bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47634/0 (socket says 172.21.15.76:47634) 2023-11-09T03:47:42.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.830+0000 7f73b27fc640 1 -- 172.21.15.76:0/714169973 learned_addr learned my addr 172.21.15.76:0/714169973 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:42.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.830+0000 7f73b27fc640 1 -- 172.21.15.76:0/714169973 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73b406c1d0 msgr2=0x7f73b4160b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:42.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.830+0000 7f73b27fc640 1 --2- 172.21.15.76:0/714169973 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73b406c1d0 0x7f73b4160b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:42.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.830+0000 7f73b27fc640 1 -- 172.21.15.76:0/714169973 >> v2:172.21.15.113:3300/0 conn(0x7f73b4160430 msgr2=0x7f73b40b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:42.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.830+0000 7f73b27fc640 1 --2- 172.21.15.76:0/714169973 >> v2:172.21.15.113:3300/0 conn(0x7f73b4160430 0x7f73b40b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:42.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.830+0000 7f73b27fc640 1 -- 172.21.15.76:0/714169973 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b406cd20 con 0x7f73b40b2b60 2023-11-09T03:47:42.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.830+0000 7f73b27fc640 1 --2- 172.21.15.76:0/714169973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73b40b2b60 0x7f73b406bc90 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f73b406f6d0 tx=0x7f73a802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff75e36dab78dff9 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:42.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.831+0000 7f73b1ffb640 1 -- 172.21.15.76:0/714169973 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a8031870 con 0x7f73b40b2b60 2023-11-09T03:47:42.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.831+0000 7f73b1ffb640 1 -- 172.21.15.76:0/714169973 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a80319d0 con 0x7f73b40b2b60 2023-11-09T03:47:42.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.831+0000 7f73b1ffb640 1 -- 172.21.15.76:0/714169973 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a8031d60 con 0x7f73b40b2b60 2023-11-09T03:47:42.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.831+0000 7f73b9826640 1 -- 172.21.15.76:0/714169973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73b40b2b60 msgr2=0x7f73b406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:42.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.831+0000 7f73b9826640 1 --2- 172.21.15.76:0/714169973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73b40b2b60 0x7f73b406bc90 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f73b406f6d0 tx=0x7f73a802b910 comp rx=0 tx=0).stop 2023-11-09T03:47:42.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.831+0000 7f73b9826640 1 -- 172.21.15.76:0/714169973 shutdown_connections 2023-11-09T03:47:42.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.831+0000 7f73b9826640 1 --2- 172.21.15.76:0/714169973 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73b406c1d0 0x7f73b4160b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:42.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.831+0000 7f73b9826640 1 --2- 172.21.15.76:0/714169973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73b40b2b60 0x7f73b406bc90 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:42.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.831+0000 7f73b9826640 1 --2- 172.21.15.76:0/714169973 >> v2:172.21.15.113:3300/0 conn(0x7f73b4160430 0x7f73b40b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:42.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.831+0000 7f73b9826640 1 -- 172.21.15.76:0/714169973 >> 172.21.15.76:0/714169973 conn(0x7f73b406ea20 msgr2=0x7f73b406ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:42.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.831+0000 7f73b9826640 1 -- 172.21.15.76:0/714169973 shutdown_connections 2023-11-09T03:47:42.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.831+0000 7f73b9826640 1 -- 172.21.15.76:0/714169973 wait complete. 2023-11-09T03:47:42.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.832+0000 7f73b9826640 1 Processor -- start 2023-11-09T03:47:42.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.833+0000 7f73b9826640 1 -- start start 2023-11-09T03:47:42.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.833+0000 7f73b9826640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73b406c1d0 0x7f73b40ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:42.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.833+0000 7f73b9826640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73b40b2b60 0x7f73b4153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:42.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.833+0000 7f73b9826640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73b4160430 0x7f73b4154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:42.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.833+0000 7f73b9826640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b4154580 con 0x7f73b4160430 2023-11-09T03:47:42.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.833+0000 7f73b9826640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b41f2df0 con 0x7f73b40b2b60 2023-11-09T03:47:42.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.833+0000 7f73b9826640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b41f30d0 con 0x7f73b406c1d0 2023-11-09T03:47:42.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.833+0000 7f73b37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73b4160430 0x7f73b4154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:42.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.833+0000 7f73b37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73b4160430 0x7f73b4154040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47640/0 (socket says 172.21.15.76:47640) 2023-11-09T03:47:42.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.833+0000 7f73b37fe640 1 -- 172.21.15.76:0/36617276 learned_addr learned my addr 172.21.15.76:0/36617276 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:42.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.834+0000 7f73b2ffd640 1 --2- 172.21.15.76:0/36617276 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73b406c1d0 0x7f73b40ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:42.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.834+0000 7f73b27fc640 1 --2- 172.21.15.76:0/36617276 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73b40b2b60 0x7f73b4153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:42.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.834+0000 7f73b37fe640 1 -- 172.21.15.76:0/36617276 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73b406c1d0 msgr2=0x7f73b40ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:42.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.834+0000 7f73b37fe640 1 --2- 172.21.15.76:0/36617276 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73b406c1d0 0x7f73b40ccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:42.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.834+0000 7f73b37fe640 1 -- 172.21.15.76:0/36617276 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73b40b2b60 msgr2=0x7f73b4153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:42.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.834+0000 7f73b37fe640 1 --2- 172.21.15.76:0/36617276 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73b40b2b60 0x7f73b4153b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:42.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.834+0000 7f73b37fe640 1 -- 172.21.15.76:0/36617276 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a80094d0 con 0x7f73b4160430 2023-11-09T03:47:42.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.834+0000 7f73b37fe640 1 --2- 172.21.15.76:0/36617276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73b4160430 0x7f73b4154040 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f73ac00b970 tx=0x7f73ac00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:42.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.834+0000 7f739bfff640 1 -- 172.21.15.76:0/36617276 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ac00cb40 con 0x7f73b4160430 2023-11-09T03:47:42.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.834+0000 7f739bfff640 1 -- 172.21.15.76:0/36617276 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73ac00ccc0 con 0x7f73b4160430 2023-11-09T03:47:42.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.834+0000 7f739bfff640 1 -- 172.21.15.76:0/36617276 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ac014680 con 0x7f73b4160430 2023-11-09T03:47:42.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.834+0000 7f73b9826640 1 -- 172.21.15.76:0/36617276 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b41f33b0 con 0x7f73b4160430 2023-11-09T03:47:42.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.834+0000 7f73b9826640 1 -- 172.21.15.76:0/36617276 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b41f2aa0 con 0x7f73b4160430 2023-11-09T03:47:42.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.835+0000 7f73b9826640 1 -- 172.21.15.76:0/36617276 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b41f38d0 con 0x7f73b4160430 2023-11-09T03:47:42.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.835+0000 7f739bfff640 1 -- 172.21.15.76:0/36617276 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f73ac01d3b0 con 0x7f73b4160430 2023-11-09T03:47:42.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.836+0000 7f739bfff640 1 --2- 172.21.15.76:0/36617276 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f73a006ae70 0x7f73a006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:42.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.836+0000 7f739bfff640 1 -- 172.21.15.76:0/36617276 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f73ac089550 con 0x7f73b4160430 2023-11-09T03:47:42.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.836+0000 7f73b2ffd640 1 --2- 172.21.15.76:0/36617276 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f73a006ae70 0x7f73a006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:42.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.837+0000 7f73b2ffd640 1 --2- 172.21.15.76:0/36617276 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f73a006ae70 0x7f73a006d330 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f73a400a810 tx=0x7f73a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:42.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:42.840+0000 7f739bfff640 1 -- 172.21.15.76:0/36617276 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f73b41f38d0 con 0x7f73b4160430 2023-11-09T03:47:43.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.015+0000 7f73b9826640 1 -- 172.21.15.76:0/36617276 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f73b41f38d0 con 0x7f73b4160430 2023-11-09T03:47:43.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.018+0000 7f739bfff640 1 -- 172.21.15.76:0/36617276 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v14) v1 ==== 104+0+0 (secure 0 0 0) 0x7f73b41f38d0 con 0x7f73b4160430 2023-11-09T03:47:43.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.020+0000 7f73b9826640 1 -- 172.21.15.76:0/36617276 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f73a006ae70 msgr2=0x7f73a006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:43.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.020+0000 7f73b9826640 1 --2- 172.21.15.76:0/36617276 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f73a006ae70 0x7f73a006d330 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f73a400a810 tx=0x7f73a4008040 comp rx=0 tx=0).stop 2023-11-09T03:47:43.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.020+0000 7f73b9826640 1 -- 172.21.15.76:0/36617276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73b4160430 msgr2=0x7f73b4154040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:43.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.020+0000 7f73b9826640 1 --2- 172.21.15.76:0/36617276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73b4160430 0x7f73b4154040 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f73ac00b970 tx=0x7f73ac00be30 comp rx=0 tx=0).stop 2023-11-09T03:47:43.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.020+0000 7f73b9826640 1 -- 172.21.15.76:0/36617276 shutdown_connections 2023-11-09T03:47:43.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.020+0000 7f73b9826640 1 --2- 172.21.15.76:0/36617276 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f73a006ae70 0x7f73a006d330 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:43.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.020+0000 7f73b9826640 1 --2- 172.21.15.76:0/36617276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73b4160430 0x7f73b4154040 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:43.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.020+0000 7f73b9826640 1 --2- 172.21.15.76:0/36617276 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73b40b2b60 0x7f73b4153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:43.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.020+0000 7f73b9826640 1 --2- 172.21.15.76:0/36617276 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73b406c1d0 0x7f73b40ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:43.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.020+0000 7f73b9826640 1 -- 172.21.15.76:0/36617276 >> 172.21.15.76:0/36617276 conn(0x7f73b406ea20 msgr2=0x7f73b40b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:43.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.020+0000 7f73b9826640 1 -- 172.21.15.76:0/36617276 shutdown_connections 2023-11-09T03:47:43.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.020+0000 7f73b9826640 1 -- 172.21.15.76:0/36617276 wait complete. 2023-11-09T03:47:43.031 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_disallow_monitor_managed_snaps_for_fs_pools' 2023-11-09T03:47:43.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.131+0000 7f414d264640 1 Processor -- start 2023-11-09T03:47:43.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.131+0000 7f414d264640 1 -- start start 2023-11-09T03:47:43.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.132+0000 7f414d264640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f41480c99b0 0x7f414806d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:43.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.132+0000 7f414d264640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f414806d550 0x7f414806a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:43.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.132+0000 7f414d264640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41480cab30 con 0x7f414815ae70 2023-11-09T03:47:43.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.132+0000 7f414d264640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f414806d9e0 con 0x7f414806d550 2023-11-09T03:47:43.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.132+0000 7f414d264640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41480b5110 con 0x7f41480c99b0 2023-11-09T03:47:43.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.132+0000 7f41477fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f414806d550 0x7f414806a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:43.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.132+0000 7f41477fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f414806d550 0x7f414806a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47650/0 (socket says 172.21.15.76:47650) 2023-11-09T03:47:43.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.132+0000 7f41477fe640 1 -- 172.21.15.76:0/1369235886 learned_addr learned my addr 172.21.15.76:0/1369235886 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:43.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.132+0000 7f41477fe640 1 -- 172.21.15.76:0/1369235886 >> v2:172.21.15.113:3300/0 conn(0x7f41480c99b0 msgr2=0x7f414806d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:43.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.132+0000 7f41467fc640 1 --2- 172.21.15.76:0/1369235886 >> v2:172.21.15.113:3300/0 conn(0x7f41480c99b0 0x7f414806d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:43.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.132+0000 7f41477fe640 1 --2- 172.21.15.76:0/1369235886 >> v2:172.21.15.113:3300/0 conn(0x7f41480c99b0 0x7f414806d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:43.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.132+0000 7f41477fe640 1 -- 172.21.15.76:0/1369235886 >> v1:172.21.15.113:6789/0 conn(0x7f414815ae70 legacy=0x7f414815f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:43.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.132+0000 7f41477fe640 1 -- 172.21.15.76:0/1369235886 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41480c9eb0 con 0x7f414806d550 2023-11-09T03:47:43.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.133+0000 7f41477fe640 1 --2- 172.21.15.76:0/1369235886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f414806d550 0x7f414806a8d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f41480cba10 tx=0x7f414002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b16bfd918c135d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:43.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.133+0000 7f4145ffb640 1 -- 172.21.15.76:0/1369235886 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414002cb20 con 0x7f414806d550 2023-11-09T03:47:43.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.133+0000 7f4145ffb640 1 -- 172.21.15.76:0/1369235886 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f414002cc80 con 0x7f414806d550 2023-11-09T03:47:43.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.133+0000 7f4145ffb640 1 -- 172.21.15.76:0/1369235886 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41400343f0 con 0x7f414806d550 2023-11-09T03:47:43.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.134+0000 7f414d264640 1 -- 172.21.15.76:0/1369235886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f414806d550 msgr2=0x7f414806a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:43.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.134+0000 7f414d264640 1 --2- 172.21.15.76:0/1369235886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f414806d550 0x7f414806a8d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f41480cba10 tx=0x7f414002bf80 comp rx=0 tx=0).stop 2023-11-09T03:47:43.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.134+0000 7f414d264640 1 -- 172.21.15.76:0/1369235886 shutdown_connections 2023-11-09T03:47:43.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.134+0000 7f414d264640 1 --2- 172.21.15.76:0/1369235886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f414806d550 0x7f414806a8d0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:43.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.134+0000 7f414d264640 1 --2- 172.21.15.76:0/1369235886 >> v2:172.21.15.113:3300/0 conn(0x7f41480c99b0 0x7f414806d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:43.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.134+0000 7f414d264640 1 -- 172.21.15.76:0/1369235886 >> 172.21.15.76:0/1369235886 conn(0x7f41480cf4b0 msgr2=0x7f41480cf890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:43.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.134+0000 7f414d264640 1 -- 172.21.15.76:0/1369235886 shutdown_connections 2023-11-09T03:47:43.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.134+0000 7f414d264640 1 -- 172.21.15.76:0/1369235886 wait complete. 2023-11-09T03:47:43.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.135+0000 7f414d264640 1 Processor -- start 2023-11-09T03:47:43.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.135+0000 7f414d264640 1 -- start start 2023-11-09T03:47:43.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.136+0000 7f414d264640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f414806d550 0x7f41481f3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:43.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.136+0000 7f414d264640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41480c99b0 0x7f41480b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:43.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.136+0000 7f414d264640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f414815ae70 0x7f4148146f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:43.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.136+0000 7f414d264640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41481f4310 con 0x7f414815ae70 2023-11-09T03:47:43.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.136+0000 7f414d264640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41481f6aa0 con 0x7f414806d550 2023-11-09T03:47:43.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.136+0000 7f414d264640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41481f6d80 con 0x7f41480c99b0 2023-11-09T03:47:43.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.136+0000 7f41477fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f414815ae70 0x7f4148146f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:43.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.136+0000 7f41477fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f414815ae70 0x7f4148146f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47666/0 (socket says 172.21.15.76:47666) 2023-11-09T03:47:43.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.136+0000 7f41477fe640 1 -- 172.21.15.76:0/3248701527 learned_addr learned my addr 172.21.15.76:0/3248701527 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:43.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.136+0000 7f41467fc640 1 --2- 172.21.15.76:0/3248701527 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41480c99b0 0x7f41480b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:43.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.136+0000 7f4146ffd640 1 --2- 172.21.15.76:0/3248701527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f414806d550 0x7f41481f3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:43.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.136+0000 7f41477fe640 1 -- 172.21.15.76:0/3248701527 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41480c99b0 msgr2=0x7f41480b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:43.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.136+0000 7f41477fe640 1 --2- 172.21.15.76:0/3248701527 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41480c99b0 0x7f41480b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:43.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.136+0000 7f41477fe640 1 -- 172.21.15.76:0/3248701527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f414806d550 msgr2=0x7f41481f3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:43.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.136+0000 7f41477fe640 1 --2- 172.21.15.76:0/3248701527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f414806d550 0x7f41481f3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:43.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.136+0000 7f41477fe640 1 -- 172.21.15.76:0/3248701527 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4140009b30 con 0x7f414815ae70 2023-11-09T03:47:43.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.136+0000 7f41467fc640 1 --2- 172.21.15.76:0/3248701527 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41480c99b0 0x7f41480b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:43.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.136+0000 7f4146ffd640 1 --2- 172.21.15.76:0/3248701527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f414806d550 0x7f41481f3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:43.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.136+0000 7f41477fe640 1 --2- 172.21.15.76:0/3248701527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f414815ae70 0x7f4148146f00 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4140036040 tx=0x7f4140004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:43.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.137+0000 7f412f7fe640 1 -- 172.21.15.76:0/3248701527 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4140007e70 con 0x7f414815ae70 2023-11-09T03:47:43.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.137+0000 7f412f7fe640 1 -- 172.21.15.76:0/3248701527 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4140009510 con 0x7f414815ae70 2023-11-09T03:47:43.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.137+0000 7f414d264640 1 -- 172.21.15.76:0/3248701527 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41481f7060 con 0x7f414815ae70 2023-11-09T03:47:43.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.137+0000 7f412f7fe640 1 -- 172.21.15.76:0/3248701527 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414003e690 con 0x7f414815ae70 2023-11-09T03:47:43.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.137+0000 7f414d264640 1 -- 172.21.15.76:0/3248701527 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41481f7620 con 0x7f414815ae70 2023-11-09T03:47:43.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.138+0000 7f414d264640 1 -- 172.21.15.76:0/3248701527 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41481f7840 con 0x7f414815ae70 2023-11-09T03:47:43.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.138+0000 7f412f7fe640 1 -- 172.21.15.76:0/3248701527 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f414003e7f0 con 0x7f414815ae70 2023-11-09T03:47:43.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.138+0000 7f412f7fe640 1 --2- 172.21.15.76:0/3248701527 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f413406aea0 0x7f413406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:43.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.139+0000 7f412f7fe640 1 -- 172.21.15.76:0/3248701527 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f41400ac1b0 con 0x7f414815ae70 2023-11-09T03:47:43.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.139+0000 7f4146ffd640 1 --2- 172.21.15.76:0/3248701527 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f413406aea0 0x7f413406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:43.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.139+0000 7f4146ffd640 1 --2- 172.21.15.76:0/3248701527 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f413406aea0 0x7f413406d360 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f413800a810 tx=0x7f4138008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:43.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.143+0000 7f412f7fe640 1 -- 172.21.15.76:0/3248701527 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f41400772c0 con 0x7f414815ae70 2023-11-09T03:47:43.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.375+0000 7f414d264640 1 -- 172.21.15.76:0/3248701527 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_disallow_monitor_managed_snaps_for_fs_pools"]} v 0) v1 -- 0x7f414806ee80 con 0x7f414815ae70 2023-11-09T03:47:43.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.704+0000 7f412f7fe640 1 -- 172.21.15.76:0/3248701527 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_disallow_monitor_managed_snaps_for_fs_pools"]}]=0 v146) v1 ==== 178+0+0 (secure 0 0 0) 0x7f414007a8e0 con 0x7f414815ae70 2023-11-09T03:47:43.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.706+0000 7f414d264640 1 -- 172.21.15.76:0/3248701527 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f413406aea0 msgr2=0x7f413406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:43.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.706+0000 7f414d264640 1 --2- 172.21.15.76:0/3248701527 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f413406aea0 0x7f413406d360 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f413800a810 tx=0x7f4138008040 comp rx=0 tx=0).stop 2023-11-09T03:47:43.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.706+0000 7f414d264640 1 -- 172.21.15.76:0/3248701527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f414815ae70 msgr2=0x7f4148146f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:43.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.706+0000 7f414d264640 1 --2- 172.21.15.76:0/3248701527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f414815ae70 0x7f4148146f00 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4140036040 tx=0x7f4140004a40 comp rx=0 tx=0).stop 2023-11-09T03:47:43.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.706+0000 7f414d264640 1 -- 172.21.15.76:0/3248701527 shutdown_connections 2023-11-09T03:47:43.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.706+0000 7f414d264640 1 --2- 172.21.15.76:0/3248701527 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f413406aea0 0x7f413406d360 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:43.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.706+0000 7f414d264640 1 --2- 172.21.15.76:0/3248701527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f414815ae70 0x7f4148146f00 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:43.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.706+0000 7f414d264640 1 --2- 172.21.15.76:0/3248701527 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41480c99b0 0x7f41480b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:43.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.706+0000 7f414d264640 1 --2- 172.21.15.76:0/3248701527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f414806d550 0x7f41481f3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:43.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.706+0000 7f414d264640 1 -- 172.21.15.76:0/3248701527 >> 172.21.15.76:0/3248701527 conn(0x7f41480cf4b0 msgr2=0x7f41480c63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:43.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.706+0000 7f414d264640 1 -- 172.21.15.76:0/3248701527 shutdown_connections 2023-11-09T03:47:43.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.706+0000 7f414d264640 1 -- 172.21.15.76:0/3248701527 wait complete. 2023-11-09T03:47:43.717 INFO:tasks.cephfs_test_runner:Test that creation of monitor managed snaps fails for pools attached ... ok 2023-11-09T03:47:43.717 INFO:tasks.cephfs_test_runner:Starting test: test_using_pool_with_snap_fails_fs_creation (tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools) Test that using a pool with snaps for fs creation fails 2023-11-09T03:47:43.718 INFO:tasks.cephfs_test_runner:test_using_pool_with_snap_fails_fs_creation (tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools) 2023-11-09T03:47:43.718 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:47:43.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.818+0000 7fb68e090640 1 Processor -- start 2023-11-09T03:47:43.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.818+0000 7fb68e090640 1 -- start start 2023-11-09T03:47:43.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.819+0000 7fb68e090640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb688157710 0x7fb688167b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:43.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.819+0000 7fb68e090640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb68815e000 con 0x7fb6880cde60 2023-11-09T03:47:43.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.819+0000 7fb68e090640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb68815e2e0 con 0x7fb6880ce240 2023-11-09T03:47:43.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.819+0000 7fb68e090640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb68815e5c0 con 0x7fb688157710 2023-11-09T03:47:43.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.819+0000 7fb687fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb688157710 0x7fb688167b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:43.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.819+0000 7fb6877fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb6880cde60 0x7fb68815d560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57362/0 (socket says 172.21.15.76:57362) 2023-11-09T03:47:43.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.819+0000 7fb6877fe640 1 -- 172.21.15.76:0/3705165993 learned_addr learned my addr 172.21.15.76:0/3705165993 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:43.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.820+0000 7fb687fff640 1 -- 172.21.15.76:0/3705165993 >> v1:172.21.15.113:6789/0 conn(0x7fb6880ce240 legacy=0x7fb688157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:43.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.820+0000 7fb687fff640 1 -- 172.21.15.76:0/3705165993 >> v1:172.21.15.76:6789/0 conn(0x7fb6880cde60 legacy=0x7fb68815d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:43.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.820+0000 7fb687fff640 1 -- 172.21.15.76:0/3705165993 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6880d1120 con 0x7fb688157710 2023-11-09T03:47:43.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.820+0000 7fb6867fc640 1 -- 172.21.15.76:0/3705165993 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3149073957 0 0) 0x7fb68815e000 con 0x7fb6880cde60 2023-11-09T03:47:43.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.820+0000 7fb687fff640 1 --2- 172.21.15.76:0/3705165993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb688157710 0x7fb688167b90 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fb680009d90 tx=0x7fb68002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9a49a6a24b364c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:43.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.821+0000 7fb6867fc640 1 -- 172.21.15.76:0/3705165993 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68002cd00 con 0x7fb688157710 2023-11-09T03:47:43.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.821+0000 7fb6867fc640 1 -- 172.21.15.76:0/3705165993 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68002ce60 con 0x7fb688157710 2023-11-09T03:47:43.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.821+0000 7fb6867fc640 1 -- 172.21.15.76:0/3705165993 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb680034660 con 0x7fb688157710 2023-11-09T03:47:43.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.821+0000 7fb68e090640 1 -- 172.21.15.76:0/3705165993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb688157710 msgr2=0x7fb688167b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:43.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.821+0000 7fb68e090640 1 --2- 172.21.15.76:0/3705165993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb688157710 0x7fb688167b90 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fb680009d90 tx=0x7fb68002bf50 comp rx=0 tx=0).stop 2023-11-09T03:47:43.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.821+0000 7fb68e090640 1 -- 172.21.15.76:0/3705165993 shutdown_connections 2023-11-09T03:47:43.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.821+0000 7fb68e090640 1 --2- 172.21.15.76:0/3705165993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb688157710 0x7fb688167b90 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:43.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.821+0000 7fb68e090640 1 -- 172.21.15.76:0/3705165993 >> 172.21.15.76:0/3705165993 conn(0x7fb688071f30 msgr2=0x7fb6880cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:43.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.821+0000 7fb68e090640 1 -- 172.21.15.76:0/3705165993 shutdown_connections 2023-11-09T03:47:43.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.821+0000 7fb68e090640 1 -- 172.21.15.76:0/3705165993 wait complete. 2023-11-09T03:47:43.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.822+0000 7fb68e090640 1 Processor -- start 2023-11-09T03:47:43.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.822+0000 7fb68e090640 1 -- start start 2023-11-09T03:47:43.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.823+0000 7fb68e090640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6880cde60 0x7fb6881548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:43.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.823+0000 7fb68e090640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6880ce240 0x7fb68815bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:43.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.823+0000 7fb68e090640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb688157710 0x7fb6880d1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:43.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.823+0000 7fb68e090640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68815a8b0 con 0x7fb688157710 2023-11-09T03:47:43.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.823+0000 7fb68e090640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb688158160 con 0x7fb6880ce240 2023-11-09T03:47:43.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.823+0000 7fb68e090640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb68815c7a0 con 0x7fb6880cde60 2023-11-09T03:47:43.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.823+0000 7fb687fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb688157710 0x7fb6880d1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:43.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.823+0000 7fb687fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb688157710 0x7fb6880d1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47692/0 (socket says 172.21.15.76:47692) 2023-11-09T03:47:43.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.823+0000 7fb687fff640 1 -- 172.21.15.76:0/796269781 learned_addr learned my addr 172.21.15.76:0/796269781 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:43.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.824+0000 7fb6877fe640 1 --2- 172.21.15.76:0/796269781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6880cde60 0x7fb6881548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:43.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.824+0000 7fb686ffd640 1 --2- 172.21.15.76:0/796269781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6880ce240 0x7fb68815bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:43.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.824+0000 7fb687fff640 1 -- 172.21.15.76:0/796269781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6880cde60 msgr2=0x7fb6881548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:43.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.824+0000 7fb687fff640 1 --2- 172.21.15.76:0/796269781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6880cde60 0x7fb6881548b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:43.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.824+0000 7fb687fff640 1 -- 172.21.15.76:0/796269781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6880ce240 msgr2=0x7fb68815bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:43.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.824+0000 7fb687fff640 1 --2- 172.21.15.76:0/796269781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6880ce240 0x7fb68815bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:43.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.824+0000 7fb687fff640 1 -- 172.21.15.76:0/796269781 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb680009a70 con 0x7fb688157710 2023-11-09T03:47:43.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.824+0000 7fb687fff640 1 --2- 172.21.15.76:0/796269781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb688157710 0x7fb6880d1820 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fb680037040 tx=0x7fb680004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:43.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.824+0000 7fb66ffff640 1 -- 172.21.15.76:0/796269781 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68003d600 con 0x7fb688157710 2023-11-09T03:47:43.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.824+0000 7fb66ffff640 1 -- 172.21.15.76:0/796269781 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68003d780 con 0x7fb688157710 2023-11-09T03:47:43.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.824+0000 7fb68e090640 1 -- 172.21.15.76:0/796269781 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb68815ca80 con 0x7fb688157710 2023-11-09T03:47:43.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.824+0000 7fb66ffff640 1 -- 172.21.15.76:0/796269781 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68003db30 con 0x7fb688157710 2023-11-09T03:47:43.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.824+0000 7fb68e090640 1 -- 172.21.15.76:0/796269781 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6880cd4f0 con 0x7fb688157710 2023-11-09T03:47:43.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.825+0000 7fb68e090640 1 -- 172.21.15.76:0/796269781 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb688157be0 con 0x7fb688157710 2023-11-09T03:47:43.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.825+0000 7fb66ffff640 1 -- 172.21.15.76:0/796269781 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb68003b040 con 0x7fb688157710 2023-11-09T03:47:43.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.826+0000 7fb66ffff640 1 --2- 172.21.15.76:0/796269781 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb67406aea0 0x7fb67406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:43.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.826+0000 7fb66ffff640 1 -- 172.21.15.76:0/796269781 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fb6800ab2b0 con 0x7fb688157710 2023-11-09T03:47:43.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.826+0000 7fb6877fe640 1 --2- 172.21.15.76:0/796269781 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb67406aea0 0x7fb67406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:43.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.826+0000 7fb6877fe640 1 --2- 172.21.15.76:0/796269781 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb67406aea0 0x7fb67406d360 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb678007fd0 tx=0x7fb678009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:43.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.830+0000 7fb66ffff640 1 -- 172.21.15.76:0/796269781 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb688157be0 con 0x7fb688157710 2023-11-09T03:47:43.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.998+0000 7fb68e090640 1 -- 172.21.15.76:0/796269781 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6881582a0 con 0x7fb688157710 2023-11-09T03:47:43.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:43.998+0000 7fb66ffff640 1 -- 172.21.15.76:0/796269781 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+14932 (secure 0 0 0) 0x7fb6881582a0 con 0x7fb688157710 2023-11-09T03:47:43.999 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:43.999 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":57,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:47:42.699230+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:6835/3710140065":"2023-11-10T03:47:39.685599+0000","172.21.15.76:6833/3710140065":"2023-11-10T03:47:39.685599+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:47:44.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.000+0000 7fb68e090640 1 -- 172.21.15.76:0/796269781 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb67406aea0 msgr2=0x7fb67406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:44.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.000+0000 7fb68e090640 1 --2- 172.21.15.76:0/796269781 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb67406aea0 0x7fb67406d360 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb678007fd0 tx=0x7fb678009040 comp rx=0 tx=0).stop 2023-11-09T03:47:44.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.000+0000 7fb68e090640 1 -- 172.21.15.76:0/796269781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb688157710 msgr2=0x7fb6880d1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:44.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.000+0000 7fb68e090640 1 --2- 172.21.15.76:0/796269781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb688157710 0x7fb6880d1820 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fb680037040 tx=0x7fb680004a40 comp rx=0 tx=0).stop 2023-11-09T03:47:44.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.000+0000 7fb68e090640 1 -- 172.21.15.76:0/796269781 shutdown_connections 2023-11-09T03:47:44.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.001+0000 7fb68e090640 1 --2- 172.21.15.76:0/796269781 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb67406aea0 0x7fb67406d360 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.001+0000 7fb68e090640 1 --2- 172.21.15.76:0/796269781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb688157710 0x7fb6880d1820 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.001+0000 7fb68e090640 1 --2- 172.21.15.76:0/796269781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6880ce240 0x7fb68815bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.001+0000 7fb68e090640 1 --2- 172.21.15.76:0/796269781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6880cde60 0x7fb6881548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.001+0000 7fb68e090640 1 -- 172.21.15.76:0/796269781 >> 172.21.15.76:0/796269781 conn(0x7fb688071f30 msgr2=0x7fb6880d09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:44.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.001+0000 7fb68e090640 1 -- 172.21.15.76:0/796269781 shutdown_connections 2023-11-09T03:47:44.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.001+0000 7fb68e090640 1 -- 172.21.15.76:0/796269781 wait complete. 2023-11-09T03:47:44.012 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-09T03:47:00.036540+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-09T03:47:44.012 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:47:44.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.115+0000 7f2a2d8d4640 1 Processor -- start 2023-11-09T03:47:44.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.115+0000 7f2a2d8d4640 1 -- start start 2023-11-09T03:47:44.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.115+0000 7f2a2d8d4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2a281605b0 0x7f2a280ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:44.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.115+0000 7f2a2d8d4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a280b1a40 0x7f2a2806a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:44.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.115+0000 7f2a2d8d4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a28160990 con 0x7f2a280ca560 2023-11-09T03:47:44.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.115+0000 7f2a2d8d4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a280cc7c0 con 0x7f2a281605b0 2023-11-09T03:47:44.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.115+0000 7f2a2d8d4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a280cd750 con 0x7f2a280b1a40 2023-11-09T03:47:44.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.116+0000 7f2a277fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2a280ca560 0x7f2a28160be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57378/0 (socket says 172.21.15.76:57378) 2023-11-09T03:47:44.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.116+0000 7f2a277fe640 1 -- 172.21.15.76:0/1195535677 learned_addr learned my addr 172.21.15.76:0/1195535677 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:44.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.116+0000 7f2a26ffd640 1 --2- 172.21.15.76:0/1195535677 >> v2:172.21.15.113:3300/0 conn(0x7f2a281605b0 0x7f2a280ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:44.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.116+0000 7f2a267fc640 1 --2- 172.21.15.76:0/1195535677 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a280b1a40 0x7f2a2806a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:44.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.116+0000 7f2a25ffb640 1 -- 172.21.15.76:0/1195535677 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 52608367 0 0) 0x7f2a28160990 con 0x7f2a280ca560 2023-11-09T03:47:44.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.116+0000 7f2a25ffb640 1 -- 172.21.15.76:0/1195535677 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a14003520 con 0x7f2a280ca560 2023-11-09T03:47:44.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.116+0000 7f2a25ffb640 1 -- 172.21.15.76:0/1195535677 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3872406435 0 0) 0x7f2a14003520 con 0x7f2a280ca560 2023-11-09T03:47:44.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.116+0000 7f2a25ffb640 1 -- 172.21.15.76:0/1195535677 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a280b1a40 msgr2=0x7f2a2806a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:44.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.116+0000 7f2a25ffb640 1 --2- 172.21.15.76:0/1195535677 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a280b1a40 0x7f2a2806a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.116+0000 7f2a25ffb640 1 -- 172.21.15.76:0/1195535677 >> v2:172.21.15.113:3300/0 conn(0x7f2a281605b0 msgr2=0x7f2a280ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:44.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.116+0000 7f2a25ffb640 1 --2- 172.21.15.76:0/1195535677 >> v2:172.21.15.113:3300/0 conn(0x7f2a281605b0 0x7f2a280ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.116+0000 7f2a25ffb640 1 -- 172.21.15.76:0/1195535677 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a280cc4e0 con 0x7f2a280ca560 2023-11-09T03:47:44.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.116+0000 7f2a25ffb640 1 -- 172.21.15.76:0/1195535677 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f2a200019f0 con 0x7f2a280ca560 2023-11-09T03:47:44.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.116+0000 7f2a25ffb640 1 -- 172.21.15.76:0/1195535677 >> v1:172.21.15.76:6789/0 conn(0x7f2a280ca560 legacy=0x7f2a28160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:44.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.116+0000 7f2a25ffb640 1 --2- 172.21.15.76:0/1195535677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a14003970 0x7f2a14003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:44.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.116+0000 7f2a25ffb640 1 --2- 172.21.15.76:0/1195535677 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a140047c0 0x7f2a14004c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:44.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.116+0000 7f2a25ffb640 1 --2- 172.21.15.76:0/1195535677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a14005140 0x7f2a1400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:44.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.116+0000 7f2a25ffb640 1 -- 172.21.15.76:0/1195535677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a280cc7c0 con 0x7f2a14005140 2023-11-09T03:47:44.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.116+0000 7f2a25ffb640 1 -- 172.21.15.76:0/1195535677 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a280cd750 con 0x7f2a14003970 2023-11-09T03:47:44.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.116+0000 7f2a25ffb640 1 -- 172.21.15.76:0/1195535677 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1400bec0 con 0x7f2a140047c0 2023-11-09T03:47:44.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.116+0000 7f2a277fe640 1 --2- 172.21.15.76:0/1195535677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a14005140 0x7f2a1400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:44.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.116+0000 7f2a26ffd640 1 --2- 172.21.15.76:0/1195535677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a14003970 0x7f2a14003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:44.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.116+0000 7f2a267fc640 1 --2- 172.21.15.76:0/1195535677 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a140047c0 0x7f2a14004c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:44.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.116+0000 7f2a277fe640 1 -- 172.21.15.76:0/1195535677 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a140047c0 msgr2=0x7f2a14004c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:44.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.116+0000 7f2a277fe640 1 --2- 172.21.15.76:0/1195535677 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a140047c0 0x7f2a14004c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.116+0000 7f2a277fe640 1 -- 172.21.15.76:0/1195535677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a14003970 msgr2=0x7f2a14003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:44.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.116+0000 7f2a277fe640 1 --2- 172.21.15.76:0/1195535677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a14003970 0x7f2a14003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.116+0000 7f2a277fe640 1 -- 172.21.15.76:0/1195535677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a14003520 con 0x7f2a14005140 2023-11-09T03:47:44.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.117+0000 7f2a277fe640 1 --2- 172.21.15.76:0/1195535677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a14005140 0x7f2a1400b980 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f2a2000a830 tx=0x7f2a140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f981b9b3826661 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:44.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.117+0000 7f2a25ffb640 1 -- 172.21.15.76:0/1195535677 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a20034630 con 0x7f2a14005140 2023-11-09T03:47:44.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.117+0000 7f2a25ffb640 1 -- 172.21.15.76:0/1195535677 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2002f070 con 0x7f2a14005140 2023-11-09T03:47:44.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.117+0000 7f2a25ffb640 1 -- 172.21.15.76:0/1195535677 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a20034a10 con 0x7f2a14005140 2023-11-09T03:47:44.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.118+0000 7f2a2d8d4640 1 -- 172.21.15.76:0/1195535677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a14005140 msgr2=0x7f2a1400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:44.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.118+0000 7f2a2d8d4640 1 --2- 172.21.15.76:0/1195535677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a14005140 0x7f2a1400b980 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f2a2000a830 tx=0x7f2a140057f0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.118+0000 7f2a277fe640 1 -- 172.21.15.76:0/1195535677 reap_dead start 2023-11-09T03:47:44.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.118+0000 7f2a2d8d4640 1 -- 172.21.15.76:0/1195535677 shutdown_connections 2023-11-09T03:47:44.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.118+0000 7f2a2d8d4640 1 -- 172.21.15.76:0/1195535677 >> 172.21.15.76:0/1195535677 conn(0x7f2a28071820 msgr2=0x7f2a28071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:44.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.118+0000 7f2a2d8d4640 1 -- 172.21.15.76:0/1195535677 shutdown_connections 2023-11-09T03:47:44.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.118+0000 7f2a2d8d4640 1 -- 172.21.15.76:0/1195535677 wait complete. 2023-11-09T03:47:44.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.119+0000 7f2a2d8d4640 1 Processor -- start 2023-11-09T03:47:44.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.119+0000 7f2a2d8d4640 1 -- start start 2023-11-09T03:47:44.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.120+0000 7f2a2d8d4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a14005140 0x7f2a2815ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:44.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.120+0000 7f2a2d8d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a280b2230 0x7f2a2814ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:44.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.120+0000 7f2a2d8d4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a280b3820 0x7f2a280b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:44.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.120+0000 7f2a2d8d4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2815b580 con 0x7f2a280b2230 2023-11-09T03:47:44.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.120+0000 7f2a2d8d4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a280b3390 con 0x7f2a280b3820 2023-11-09T03:47:44.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.120+0000 7f2a2d8d4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2815f2b0 con 0x7f2a14005140 2023-11-09T03:47:44.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.120+0000 7f2a267fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a280b2230 0x7f2a2814ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:44.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.120+0000 7f2a267fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a280b2230 0x7f2a2814ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47708/0 (socket says 172.21.15.76:47708) 2023-11-09T03:47:44.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.120+0000 7f2a267fc640 1 -- 172.21.15.76:0/2369927811 learned_addr learned my addr 172.21.15.76:0/2369927811 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:44.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.120+0000 7f2a26ffd640 1 --2- 172.21.15.76:0/2369927811 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a14005140 0x7f2a2815ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:44.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.120+0000 7f2a277fe640 1 --2- 172.21.15.76:0/2369927811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a280b3820 0x7f2a280b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:44.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.120+0000 7f2a267fc640 1 -- 172.21.15.76:0/2369927811 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a14005140 msgr2=0x7f2a2815ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:44.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.120+0000 7f2a267fc640 1 --2- 172.21.15.76:0/2369927811 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a14005140 0x7f2a2815ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.120+0000 7f2a267fc640 1 -- 172.21.15.76:0/2369927811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a280b3820 msgr2=0x7f2a280b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:44.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.120+0000 7f2a267fc640 1 --2- 172.21.15.76:0/2369927811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a280b3820 0x7f2a280b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.120+0000 7f2a267fc640 1 -- 172.21.15.76:0/2369927811 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2814efa0 con 0x7f2a280b2230 2023-11-09T03:47:44.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.120+0000 7f2a26ffd640 1 --2- 172.21.15.76:0/2369927811 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a14005140 0x7f2a2815ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:44.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.120+0000 7f2a277fe640 1 --2- 172.21.15.76:0/2369927811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a280b3820 0x7f2a280b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:44.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.121+0000 7f2a267fc640 1 --2- 172.21.15.76:0/2369927811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a280b2230 0x7f2a2814ea60 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f2a18011630 tx=0x7f2a18011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:44.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.121+0000 7f2a1ffff640 1 -- 172.21.15.76:0/2369927811 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1800f920 con 0x7f2a280b2230 2023-11-09T03:47:44.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.121+0000 7f2a1ffff640 1 -- 172.21.15.76:0/2369927811 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a1800fa80 con 0x7f2a280b2230 2023-11-09T03:47:44.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.121+0000 7f2a1ffff640 1 -- 172.21.15.76:0/2369927811 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1800fdf0 con 0x7f2a280b2230 2023-11-09T03:47:44.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.121+0000 7f2a2d8d4640 1 -- 172.21.15.76:0/2369927811 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a280b2780 con 0x7f2a280b2230 2023-11-09T03:47:44.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.121+0000 7f2a2d8d4640 1 -- 172.21.15.76:0/2369927811 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a280b2d40 con 0x7f2a280b2230 2023-11-09T03:47:44.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.122+0000 7f2a2d8d4640 1 -- 172.21.15.76:0/2369927811 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a280672a0 con 0x7f2a280b2230 2023-11-09T03:47:44.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.122+0000 7f2a1ffff640 1 -- 172.21.15.76:0/2369927811 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2a18015460 con 0x7f2a280b2230 2023-11-09T03:47:44.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.123+0000 7f2a1ffff640 1 --2- 172.21.15.76:0/2369927811 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2a0806ae70 0x7f2a0806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:44.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.123+0000 7f2a1ffff640 1 -- 172.21.15.76:0/2369927811 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f2a18089b40 con 0x7f2a280b2230 2023-11-09T03:47:44.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.123+0000 7f2a26ffd640 1 --2- 172.21.15.76:0/2369927811 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2a0806ae70 0x7f2a0806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:44.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.123+0000 7f2a26ffd640 1 --2- 172.21.15.76:0/2369927811 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2a0806ae70 0x7f2a0806d330 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009bc0 tx=0x7f2a10008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:44.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.127+0000 7f2a1ffff640 1 -- 172.21.15.76:0/2369927811 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2a18054bc0 con 0x7f2a280b2230 2023-11-09T03:47:44.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.292+0000 7f2a2d8d4640 1 -- 172.21.15.76:0/2369927811 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2a28074290 con 0x7f2a280b2230 2023-11-09T03:47:44.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.293+0000 7f2a1ffff640 1 -- 172.21.15.76:0/2369927811 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2a18022020 con 0x7f2a280b2230 2023-11-09T03:47:44.293 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:47:44.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.295+0000 7f2a2d8d4640 1 -- 172.21.15.76:0/2369927811 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2a0806ae70 msgr2=0x7f2a0806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:44.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.295+0000 7f2a2d8d4640 1 --2- 172.21.15.76:0/2369927811 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2a0806ae70 0x7f2a0806d330 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009bc0 tx=0x7f2a10008040 comp rx=0 tx=0).stop 2023-11-09T03:47:44.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.295+0000 7f2a2d8d4640 1 -- 172.21.15.76:0/2369927811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a280b2230 msgr2=0x7f2a2814ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:44.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.295+0000 7f2a2d8d4640 1 --2- 172.21.15.76:0/2369927811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a280b2230 0x7f2a2814ea60 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f2a18011630 tx=0x7f2a18011af0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.295+0000 7f2a2d8d4640 1 -- 172.21.15.76:0/2369927811 shutdown_connections 2023-11-09T03:47:44.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.295+0000 7f2a2d8d4640 1 --2- 172.21.15.76:0/2369927811 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2a0806ae70 0x7f2a0806d330 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.295+0000 7f2a2d8d4640 1 --2- 172.21.15.76:0/2369927811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a280b3820 0x7f2a280b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.295+0000 7f2a2d8d4640 1 --2- 172.21.15.76:0/2369927811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a280b2230 0x7f2a2814ea60 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.295+0000 7f2a2d8d4640 1 --2- 172.21.15.76:0/2369927811 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a14005140 0x7f2a2815ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.295+0000 7f2a2d8d4640 1 -- 172.21.15.76:0/2369927811 >> 172.21.15.76:0/2369927811 conn(0x7f2a28071820 msgr2=0x7f2a280c9850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:44.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.295+0000 7f2a2d8d4640 1 -- 172.21.15.76:0/2369927811 shutdown_connections 2023-11-09T03:47:44.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.295+0000 7f2a2d8d4640 1 -- 172.21.15.76:0/2369927811 wait complete. 2023-11-09T03:47:44.306 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-09T03:47:44.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.408+0000 7f8cee925640 1 Processor -- start 2023-11-09T03:47:44.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.408+0000 7f8cee925640 1 -- start start 2023-11-09T03:47:44.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.408+0000 7f8cee925640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8ce8159c00 0x7f8ce80cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:44.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.408+0000 7f8cee925640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8ce80c48e0 0x7f8ce80c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:44.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.408+0000 7f8cee925640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ce815aff0 con 0x7f8ce80adac0 2023-11-09T03:47:44.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.408+0000 7f8cee925640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ce80c52d0 con 0x7f8ce80c48e0 2023-11-09T03:47:44.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.408+0000 7f8cee925640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ce815a3e0 con 0x7f8ce8159c00 2023-11-09T03:47:44.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.409+0000 7f8ce77fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8ce80c48e0 0x7f8ce80c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:44.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.409+0000 7f8ce77fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8ce80c48e0 0x7f8ce80c4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47716/0 (socket says 172.21.15.76:47716) 2023-11-09T03:47:44.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.409+0000 7f8ce77fe640 1 -- 172.21.15.76:0/1172071469 learned_addr learned my addr 172.21.15.76:0/1172071469 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:44.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.409+0000 7f8ce7fff640 1 --2- 172.21.15.76:0/1172071469 >> v2:172.21.15.113:3300/0 conn(0x7f8ce8159c00 0x7f8ce80cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:44.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.409+0000 7f8ce77fe640 1 -- 172.21.15.76:0/1172071469 >> v2:172.21.15.113:3300/0 conn(0x7f8ce8159c00 msgr2=0x7f8ce80cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:44.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.409+0000 7f8ce77fe640 1 --2- 172.21.15.76:0/1172071469 >> v2:172.21.15.113:3300/0 conn(0x7f8ce8159c00 0x7f8ce80cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.409+0000 7f8ce77fe640 1 -- 172.21.15.76:0/1172071469 >> v1:172.21.15.113:6789/0 conn(0x7f8ce80adac0 legacy=0x7f8ce80adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:44.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.409+0000 7f8ce77fe640 1 -- 172.21.15.76:0/1172071469 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce815a6c0 con 0x7f8ce80c48e0 2023-11-09T03:47:44.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.410+0000 7f8ce77fe640 1 --2- 172.21.15.76:0/1172071469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ce80c48e0 0x7f8ce80c4d90 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8ce80cc4d0 tx=0x7f8cdc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af354d706c0bb4ee server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:44.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.410+0000 7f8ce6ffd640 1 -- 172.21.15.76:0/1172071469 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc031870 con 0x7f8ce80c48e0 2023-11-09T03:47:44.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.410+0000 7f8ce6ffd640 1 -- 172.21.15.76:0/1172071469 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cdc0319d0 con 0x7f8ce80c48e0 2023-11-09T03:47:44.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.410+0000 7f8ce6ffd640 1 -- 172.21.15.76:0/1172071469 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc031d60 con 0x7f8ce80c48e0 2023-11-09T03:47:44.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.410+0000 7f8cee925640 1 -- 172.21.15.76:0/1172071469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ce80c48e0 msgr2=0x7f8ce80c4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:44.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.410+0000 7f8cee925640 1 --2- 172.21.15.76:0/1172071469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ce80c48e0 0x7f8ce80c4d90 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8ce80cc4d0 tx=0x7f8cdc02b910 comp rx=0 tx=0).stop 2023-11-09T03:47:44.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.410+0000 7f8cee925640 1 -- 172.21.15.76:0/1172071469 shutdown_connections 2023-11-09T03:47:44.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.410+0000 7f8cee925640 1 --2- 172.21.15.76:0/1172071469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ce80c48e0 0x7f8ce80c4d90 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.410+0000 7f8cee925640 1 --2- 172.21.15.76:0/1172071469 >> v2:172.21.15.113:3300/0 conn(0x7f8ce8159c00 0x7f8ce80cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.410+0000 7f8cee925640 1 -- 172.21.15.76:0/1172071469 >> 172.21.15.76:0/1172071469 conn(0x7f8ce80ce950 msgr2=0x7f8ce80ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:44.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.410+0000 7f8cee925640 1 -- 172.21.15.76:0/1172071469 shutdown_connections 2023-11-09T03:47:44.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.410+0000 7f8cee925640 1 -- 172.21.15.76:0/1172071469 wait complete. 2023-11-09T03:47:44.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.411+0000 7f8cee925640 1 Processor -- start 2023-11-09T03:47:44.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.412+0000 7f8cee925640 1 -- start start 2023-11-09T03:47:44.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.412+0000 7f8cee925640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ce80adac0 0x7f8ce81f3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:44.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.412+0000 7f8cee925640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ce80c48e0 0x7f8ce80c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:44.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.412+0000 7f8cee925640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ce8159c00 0x7f8ce8152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:44.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.412+0000 7f8cee925640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce81524f0 con 0x7f8ce80c48e0 2023-11-09T03:47:44.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.412+0000 7f8cee925640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce81f4400 con 0x7f8ce8159c00 2023-11-09T03:47:44.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.412+0000 7f8cee925640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce81f3e40 con 0x7f8ce80adac0 2023-11-09T03:47:44.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.413+0000 7f8ce77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ce80c48e0 0x7f8ce80c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:44.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.413+0000 7f8ce77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ce80c48e0 0x7f8ce80c7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47722/0 (socket says 172.21.15.76:47722) 2023-11-09T03:47:44.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.413+0000 7f8ce77fe640 1 -- 172.21.15.76:0/3035033658 learned_addr learned my addr 172.21.15.76:0/3035033658 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:44.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.413+0000 7f8ce7fff640 1 --2- 172.21.15.76:0/3035033658 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ce80adac0 0x7f8ce81f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:44.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.413+0000 7f8cecec2640 1 --2- 172.21.15.76:0/3035033658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ce8159c00 0x7f8ce8152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:44.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.413+0000 7f8ce77fe640 1 -- 172.21.15.76:0/3035033658 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ce80adac0 msgr2=0x7f8ce81f3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:44.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.413+0000 7f8ce77fe640 1 --2- 172.21.15.76:0/3035033658 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ce80adac0 0x7f8ce81f3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.413+0000 7f8ce77fe640 1 -- 172.21.15.76:0/3035033658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ce8159c00 msgr2=0x7f8ce8152910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:44.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.413+0000 7f8ce77fe640 1 --2- 172.21.15.76:0/3035033658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ce8159c00 0x7f8ce8152910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.413+0000 7f8ce77fe640 1 -- 172.21.15.76:0/3035033658 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cdc0094d0 con 0x7f8ce80c48e0 2023-11-09T03:47:44.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.413+0000 7f8cecec2640 1 --2- 172.21.15.76:0/3035033658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ce8159c00 0x7f8ce8152910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:44.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.413+0000 7f8ce77fe640 1 --2- 172.21.15.76:0/3035033658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ce80c48e0 0x7f8ce80c7ca0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc000c00 tx=0x7f8cdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:44.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.413+0000 7f8ce4ff9640 1 -- 172.21.15.76:0/3035033658 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc03ad30 con 0x7f8ce80c48e0 2023-11-09T03:47:44.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.413+0000 7f8ce4ff9640 1 -- 172.21.15.76:0/3035033658 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cdc0443f0 con 0x7f8ce80c48e0 2023-11-09T03:47:44.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.413+0000 7f8ce4ff9640 1 -- 172.21.15.76:0/3035033658 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc0447a0 con 0x7f8ce80c48e0 2023-11-09T03:47:44.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.413+0000 7f8cee925640 1 -- 172.21.15.76:0/3035033658 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ce81f4120 con 0x7f8ce80c48e0 2023-11-09T03:47:44.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.413+0000 7f8cee925640 1 -- 172.21.15.76:0/3035033658 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ce81f3a50 con 0x7f8ce80c48e0 2023-11-09T03:47:44.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.414+0000 7f8cee925640 1 -- 172.21.15.76:0/3035033658 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ce8152630 con 0x7f8ce80c48e0 2023-11-09T03:47:44.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.415+0000 7f8ce4ff9640 1 -- 172.21.15.76:0/3035033658 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8cdc044900 con 0x7f8ce80c48e0 2023-11-09T03:47:44.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.415+0000 7f8ce4ff9640 1 --2- 172.21.15.76:0/3035033658 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8cd406ae70 0x7f8cd406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:44.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.415+0000 7f8ce4ff9640 1 -- 172.21.15.76:0/3035033658 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f8cdc02d040 con 0x7f8ce80c48e0 2023-11-09T03:47:44.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.415+0000 7f8ce7fff640 1 --2- 172.21.15.76:0/3035033658 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8cd406ae70 0x7f8cd406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:44.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.416+0000 7f8ce7fff640 1 --2- 172.21.15.76:0/3035033658 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8cd406ae70 0x7f8cd406d330 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8006fd0 tx=0x7f8cd8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:44.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.419+0000 7f8ce4ff9640 1 -- 172.21.15.76:0/3035033658 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8ce8152630 con 0x7f8ce80c48e0 2023-11-09T03:47:44.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.584+0000 7f8cee925640 1 -- 172.21.15.76:0/3035033658 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f8ce8152630 con 0x7f8ce80c48e0 2023-11-09T03:47:44.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.584+0000 7f8ce4ff9640 1 -- 172.21.15.76:0/3035033658 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v57) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8ce8152630 con 0x7f8ce80c48e0 2023-11-09T03:47:44.585 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:47:44.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.586+0000 7f8cee925640 1 -- 172.21.15.76:0/3035033658 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8cd406ae70 msgr2=0x7f8cd406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:44.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.586+0000 7f8cee925640 1 --2- 172.21.15.76:0/3035033658 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8cd406ae70 0x7f8cd406d330 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8006fd0 tx=0x7f8cd8008040 comp rx=0 tx=0).stop 2023-11-09T03:47:44.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.586+0000 7f8cee925640 1 -- 172.21.15.76:0/3035033658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ce80c48e0 msgr2=0x7f8ce80c7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:44.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.586+0000 7f8cee925640 1 --2- 172.21.15.76:0/3035033658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ce80c48e0 0x7f8ce80c7ca0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc000c00 tx=0x7f8cdc0046c0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.586+0000 7f8cee925640 1 -- 172.21.15.76:0/3035033658 shutdown_connections 2023-11-09T03:47:44.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.586+0000 7f8cee925640 1 --2- 172.21.15.76:0/3035033658 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8cd406ae70 0x7f8cd406d330 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.586+0000 7f8cee925640 1 --2- 172.21.15.76:0/3035033658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ce8159c00 0x7f8ce8152910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.586+0000 7f8cee925640 1 --2- 172.21.15.76:0/3035033658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ce80c48e0 0x7f8ce80c7ca0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.586+0000 7f8cee925640 1 --2- 172.21.15.76:0/3035033658 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ce80adac0 0x7f8ce81f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.586+0000 7f8cee925640 1 -- 172.21.15.76:0/3035033658 >> 172.21.15.76:0/3035033658 conn(0x7f8ce80ce950 msgr2=0x7f8ce80cb340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:44.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.586+0000 7f8cee925640 1 -- 172.21.15.76:0/3035033658 shutdown_connections 2023-11-09T03:47:44.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.586+0000 7f8cee925640 1 -- 172.21.15.76:0/3035033658 wait complete. 2023-11-09T03:47:44.597 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_using_pool_with_snap_fails_fs_creation' 2023-11-09T03:47:44.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.697+0000 7f96a1ad4640 1 Processor -- start 2023-11-09T03:47:44.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.697+0000 7f96a1ad4640 1 -- start start 2023-11-09T03:47:44.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.698+0000 7f96a1ad4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f969c159c70 0x7f969c0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:44.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.698+0000 7f96a1ad4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f969c06db20 0x7f969c0b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:44.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.698+0000 7f96a1ad4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f969c0b4110 con 0x7f969c0b0910 2023-11-09T03:47:44.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.698+0000 7f96a1ad4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f969c0b49b0 con 0x7f969c06db20 2023-11-09T03:47:44.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.698+0000 7f96a1ad4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f969c0b0da0 con 0x7f969c159c70 2023-11-09T03:47:44.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.698+0000 7f969affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f969c06db20 0x7f969c0b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:44.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.698+0000 7f969bfff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f969c0b0910 0x7f969c15d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57392/0 (socket says 172.21.15.76:57392) 2023-11-09T03:47:44.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.698+0000 7f969bfff640 1 -- 172.21.15.76:0/840351569 learned_addr learned my addr 172.21.15.76:0/840351569 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:44.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.698+0000 7f969b7fe640 1 --2- 172.21.15.76:0/840351569 >> v2:172.21.15.113:3300/0 conn(0x7f969c159c70 0x7f969c0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:44.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.698+0000 7f969affd640 1 -- 172.21.15.76:0/840351569 >> v2:172.21.15.113:3300/0 conn(0x7f969c159c70 msgr2=0x7f969c0ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:44.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.698+0000 7f969affd640 1 --2- 172.21.15.76:0/840351569 >> v2:172.21.15.113:3300/0 conn(0x7f969c159c70 0x7f969c0ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.698+0000 7f969affd640 1 -- 172.21.15.76:0/840351569 >> v1:172.21.15.76:6789/0 conn(0x7f969c0b0910 legacy=0x7f969c15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:44.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.698+0000 7f969a7fc640 1 -- 172.21.15.76:0/840351569 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 310864772 0 0) 0x7f969c0b4110 con 0x7f969c0b0910 2023-11-09T03:47:44.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.698+0000 7f969affd640 1 -- 172.21.15.76:0/840351569 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969c0b43f0 con 0x7f969c06db20 2023-11-09T03:47:44.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.699+0000 7f969affd640 1 --2- 172.21.15.76:0/840351569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f969c06db20 0x7f969c0b03d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f969c0cc440 tx=0x7f969002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae37e51f766a1980 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:44.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.699+0000 7f969a7fc640 1 -- 172.21.15.76:0/840351569 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9690031830 con 0x7f969c06db20 2023-11-09T03:47:44.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.699+0000 7f969a7fc640 1 -- 172.21.15.76:0/840351569 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9690031990 con 0x7f969c06db20 2023-11-09T03:47:44.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.699+0000 7f969a7fc640 1 -- 172.21.15.76:0/840351569 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9690031d20 con 0x7f969c06db20 2023-11-09T03:47:44.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.700+0000 7f96a1ad4640 1 -- 172.21.15.76:0/840351569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f969c06db20 msgr2=0x7f969c0b03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:44.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.700+0000 7f96a1ad4640 1 --2- 172.21.15.76:0/840351569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f969c06db20 0x7f969c0b03d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f969c0cc440 tx=0x7f969002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.700+0000 7f96a1ad4640 1 -- 172.21.15.76:0/840351569 shutdown_connections 2023-11-09T03:47:44.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.700+0000 7f96a1ad4640 1 --2- 172.21.15.76:0/840351569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f969c06db20 0x7f969c0b03d0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.700+0000 7f96a1ad4640 1 --2- 172.21.15.76:0/840351569 >> v2:172.21.15.113:3300/0 conn(0x7f969c159c70 0x7f969c0ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.700+0000 7f96a1ad4640 1 -- 172.21.15.76:0/840351569 >> 172.21.15.76:0/840351569 conn(0x7f969c0ce950 msgr2=0x7f969c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:44.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.700+0000 7f96a1ad4640 1 -- 172.21.15.76:0/840351569 shutdown_connections 2023-11-09T03:47:44.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.700+0000 7f96a1ad4640 1 -- 172.21.15.76:0/840351569 wait complete. 2023-11-09T03:47:44.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.701+0000 7f96a1ad4640 1 Processor -- start 2023-11-09T03:47:44.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.701+0000 7f96a1ad4640 1 -- start start 2023-11-09T03:47:44.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.702+0000 7f96a1ad4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f969c06db20 0x7f969c156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:44.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.702+0000 7f96a1ad4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f969c0b0910 0x7f969c1555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:44.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.702+0000 7f96a1ad4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f969c159c70 0x7f969c153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:44.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.702+0000 7f96a1ad4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c155b00 con 0x7f969c159c70 2023-11-09T03:47:44.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.702+0000 7f96a1ad4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c154520 con 0x7f969c06db20 2023-11-09T03:47:44.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.702+0000 7f96a1ad4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969c154800 con 0x7f969c0b0910 2023-11-09T03:47:44.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.702+0000 7f969bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f969c159c70 0x7f969c153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:44.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.702+0000 7f969bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f969c159c70 0x7f969c153cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47742/0 (socket says 172.21.15.76:47742) 2023-11-09T03:47:44.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.702+0000 7f969bfff640 1 -- 172.21.15.76:0/490651726 learned_addr learned my addr 172.21.15.76:0/490651726 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:44.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.702+0000 7f969affd640 1 --2- 172.21.15.76:0/490651726 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f969c0b0910 0x7f969c1555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:44.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.702+0000 7f969b7fe640 1 --2- 172.21.15.76:0/490651726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f969c06db20 0x7f969c156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:44.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.702+0000 7f969bfff640 1 -- 172.21.15.76:0/490651726 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f969c0b0910 msgr2=0x7f969c1555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:44.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.702+0000 7f969bfff640 1 --2- 172.21.15.76:0/490651726 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f969c0b0910 0x7f969c1555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.702+0000 7f969bfff640 1 -- 172.21.15.76:0/490651726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f969c06db20 msgr2=0x7f969c156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:44.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.702+0000 7f969bfff640 1 --2- 172.21.15.76:0/490651726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f969c06db20 0x7f969c156720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:44.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.702+0000 7f969bfff640 1 -- 172.21.15.76:0/490651726 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9690009400 con 0x7f969c159c70 2023-11-09T03:47:44.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.702+0000 7f969b7fe640 1 --2- 172.21.15.76:0/490651726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f969c06db20 0x7f969c156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:44.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.702+0000 7f969bfff640 1 --2- 172.21.15.76:0/490651726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f969c159c70 0x7f969c153cd0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f969400c7a0 tx=0x7f969400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:44.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.703+0000 7f9683fff640 1 -- 172.21.15.76:0/490651726 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969400a640 con 0x7f969c159c70 2023-11-09T03:47:44.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.703+0000 7f9683fff640 1 -- 172.21.15.76:0/490651726 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969400a7a0 con 0x7f969c159c70 2023-11-09T03:47:44.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.703+0000 7f96a1ad4640 1 -- 172.21.15.76:0/490651726 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f969c154ae0 con 0x7f969c159c70 2023-11-09T03:47:44.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.703+0000 7f9683fff640 1 -- 172.21.15.76:0/490651726 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969400abd0 con 0x7f969c159c70 2023-11-09T03:47:44.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.703+0000 7f96a1ad4640 1 -- 172.21.15.76:0/490651726 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969c1550a0 con 0x7f969c159c70 2023-11-09T03:47:44.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.704+0000 7f96a1ad4640 1 -- 172.21.15.76:0/490651726 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f969c074290 con 0x7f969c159c70 2023-11-09T03:47:44.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.704+0000 7f9683fff640 1 -- 172.21.15.76:0/490651726 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f969400ad30 con 0x7f969c159c70 2023-11-09T03:47:44.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.704+0000 7f9683fff640 1 --2- 172.21.15.76:0/490651726 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f968806ae70 0x7f968806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:44.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.705+0000 7f9683fff640 1 -- 172.21.15.76:0/490651726 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f9694088d00 con 0x7f969c159c70 2023-11-09T03:47:44.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.705+0000 7f969b7fe640 1 --2- 172.21.15.76:0/490651726 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f968806ae70 0x7f968806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:44.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.705+0000 7f969b7fe640 1 --2- 172.21.15.76:0/490651726 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f968806ae70 0x7f968806d330 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f968c006fd0 tx=0x7f968c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:44.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.709+0000 7f9683fff640 1 -- 172.21.15.76:0/490651726 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9694053ec0 con 0x7f969c159c70 2023-11-09T03:47:44.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:44.941+0000 7f96a1ad4640 1 -- 172.21.15.76:0/490651726 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_using_pool_with_snap_fails_fs_creation"]} v 0) v1 -- 0x7f969c0ced30 con 0x7f969c159c70 2023-11-09T03:47:45.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.718+0000 7f9683fff640 1 -- 172.21.15.76:0/490651726 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_using_pool_with_snap_fails_fs_creation"]}]=0 v148) v1 ==== 176+0+0 (secure 0 0 0) 0x7f9694018020 con 0x7f969c159c70 2023-11-09T03:47:45.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.721+0000 7f96a1ad4640 1 -- 172.21.15.76:0/490651726 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f968806ae70 msgr2=0x7f968806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:45.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.721+0000 7f96a1ad4640 1 --2- 172.21.15.76:0/490651726 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f968806ae70 0x7f968806d330 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f968c006fd0 tx=0x7f968c008040 comp rx=0 tx=0).stop 2023-11-09T03:47:45.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.721+0000 7f96a1ad4640 1 -- 172.21.15.76:0/490651726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f969c159c70 msgr2=0x7f969c153cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:45.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.721+0000 7f96a1ad4640 1 --2- 172.21.15.76:0/490651726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f969c159c70 0x7f969c153cd0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f969400c7a0 tx=0x7f969400cc60 comp rx=0 tx=0).stop 2023-11-09T03:47:45.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.721+0000 7f96a1ad4640 1 -- 172.21.15.76:0/490651726 shutdown_connections 2023-11-09T03:47:45.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.721+0000 7f96a1ad4640 1 --2- 172.21.15.76:0/490651726 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f968806ae70 0x7f968806d330 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:45.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.721+0000 7f96a1ad4640 1 --2- 172.21.15.76:0/490651726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f969c159c70 0x7f969c153cd0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:45.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.721+0000 7f96a1ad4640 1 --2- 172.21.15.76:0/490651726 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f969c0b0910 0x7f969c1555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:45.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.721+0000 7f96a1ad4640 1 --2- 172.21.15.76:0/490651726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f969c06db20 0x7f969c156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:45.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.721+0000 7f96a1ad4640 1 -- 172.21.15.76:0/490651726 >> 172.21.15.76:0/490651726 conn(0x7f969c0ce950 msgr2=0x7f969c0caeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:45.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.721+0000 7f96a1ad4640 1 -- 172.21.15.76:0/490651726 shutdown_connections 2023-11-09T03:47:45.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.721+0000 7f96a1ad4640 1 -- 172.21.15.76:0/490651726 wait complete. 2023-11-09T03:47:45.735 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-09T03:47:45.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.842+0000 7fd903232640 1 Processor -- start 2023-11-09T03:47:45.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.842+0000 7fd903232640 1 -- start start 2023-11-09T03:47:45.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.842+0000 7fd903232640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd8fc0cd6d0 0x7fd8fc161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:45.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.842+0000 7fd903232640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8fc0b4880 0x7fd8fc0b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:45.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.842+0000 7fd903232640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8fc0cde60 con 0x7fd8fc153e60 2023-11-09T03:47:45.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.842+0000 7fd903232640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8fc160f90 con 0x7fd8fc0cd6d0 2023-11-09T03:47:45.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.842+0000 7fd903232640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc161960 con 0x7fd8fc0b4880 2023-11-09T03:47:45.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.843+0000 7fd9017cf640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8fc0b4880 0x7fd8fc0b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:45.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.843+0000 7fd8fbfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd8fc0cd6d0 0x7fd8fc161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:45.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.843+0000 7fd9017cf640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8fc0b4880 0x7fd8fc0b4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:58666/0 (socket says 172.21.15.76:58666) 2023-11-09T03:47:45.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.843+0000 7fd900fce640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd8fc153e60 0x7fd8fc15dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:50732/0 (socket says 172.21.15.76:50732) 2023-11-09T03:47:45.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.843+0000 7fd9017cf640 1 -- 172.21.15.76:0/70759640 learned_addr learned my addr 172.21.15.76:0/70759640 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:45.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.843+0000 7fd8fbfff640 1 -- 172.21.15.76:0/70759640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8fc0b4880 msgr2=0x7fd8fc0b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:45.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.843+0000 7fd8fbfff640 1 --2- 172.21.15.76:0/70759640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8fc0b4880 0x7fd8fc0b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:45.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.843+0000 7fd8fbfff640 1 -- 172.21.15.76:0/70759640 >> v1:172.21.15.113:6789/0 conn(0x7fd8fc153e60 legacy=0x7fd8fc15dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:45.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.843+0000 7fd8fb7fe640 1 -- 172.21.15.76:0/70759640 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1296535797 0 0) 0x7fd8fc0cde60 con 0x7fd8fc153e60 2023-11-09T03:47:45.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.843+0000 7fd8fbfff640 1 -- 172.21.15.76:0/70759640 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8fc0b5220 con 0x7fd8fc0cd6d0 2023-11-09T03:47:45.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.843+0000 7fd9017cf640 1 --2- 172.21.15.76:0/70759640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8fc0b4880 0x7fd8fc0b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:47:45.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.844+0000 7fd8fbfff640 1 --2- 172.21.15.76:0/70759640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8fc0cd6d0 0x7fd8fc161420 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd8e40097a0 tx=0x7fd8e402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b99daa53d5bf03de server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:45.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.844+0000 7fd8fb7fe640 1 -- 172.21.15.76:0/70759640 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4031830 con 0x7fd8fc0cd6d0 2023-11-09T03:47:45.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.844+0000 7fd8fb7fe640 1 -- 172.21.15.76:0/70759640 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e4031990 con 0x7fd8fc0cd6d0 2023-11-09T03:47:45.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.844+0000 7fd8fb7fe640 1 -- 172.21.15.76:0/70759640 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4031d20 con 0x7fd8fc0cd6d0 2023-11-09T03:47:45.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.845+0000 7fd903232640 1 -- 172.21.15.76:0/70759640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8fc0cd6d0 msgr2=0x7fd8fc161420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:45.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.845+0000 7fd903232640 1 --2- 172.21.15.76:0/70759640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8fc0cd6d0 0x7fd8fc161420 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd8e40097a0 tx=0x7fd8e402b8d0 comp rx=0 tx=0).stop 2023-11-09T03:47:45.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.845+0000 7fd903232640 1 -- 172.21.15.76:0/70759640 shutdown_connections 2023-11-09T03:47:45.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.845+0000 7fd903232640 1 --2- 172.21.15.76:0/70759640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8fc0b4880 0x7fd8fc0b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:45.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.845+0000 7fd903232640 1 --2- 172.21.15.76:0/70759640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8fc0cd6d0 0x7fd8fc161420 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:45.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.845+0000 7fd903232640 1 -- 172.21.15.76:0/70759640 >> 172.21.15.76:0/70759640 conn(0x7fd8fc0cf880 msgr2=0x7fd8fc0cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:45.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.845+0000 7fd903232640 1 -- 172.21.15.76:0/70759640 shutdown_connections 2023-11-09T03:47:45.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.845+0000 7fd903232640 1 -- 172.21.15.76:0/70759640 wait complete. 2023-11-09T03:47:45.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.846+0000 7fd903232640 1 Processor -- start 2023-11-09T03:47:45.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.846+0000 7fd903232640 1 -- start start 2023-11-09T03:47:45.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.847+0000 7fd903232640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8fc0b4880 0x7fd8fc153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:45.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.847+0000 7fd903232640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8fc0cd6d0 0x7fd8fc155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:45.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.847+0000 7fd903232640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8fc153e60 0x7fd8fc0cb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:45.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.847+0000 7fd903232640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc153850 con 0x7fd8fc0b4880 2023-11-09T03:47:45.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.847+0000 7fd903232640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc0ca500 con 0x7fd8fc153e60 2023-11-09T03:47:45.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.847+0000 7fd903232640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc0cbca0 con 0x7fd8fc0cd6d0 2023-11-09T03:47:45.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.847+0000 7fd900fce640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8fc0b4880 0x7fd8fc153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:45.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.847+0000 7fd900fce640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8fc0b4880 0x7fd8fc153310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47758/0 (socket says 172.21.15.76:47758) 2023-11-09T03:47:45.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.847+0000 7fd900fce640 1 -- 172.21.15.76:0/3774707970 learned_addr learned my addr 172.21.15.76:0/3774707970 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:45.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.847+0000 7fd9017cf640 1 --2- 172.21.15.76:0/3774707970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8fc153e60 0x7fd8fc0cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:45.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.847+0000 7fd8fbfff640 1 --2- 172.21.15.76:0/3774707970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8fc0cd6d0 0x7fd8fc155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:45.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.847+0000 7fd900fce640 1 -- 172.21.15.76:0/3774707970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8fc0cd6d0 msgr2=0x7fd8fc155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:45.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.847+0000 7fd900fce640 1 --2- 172.21.15.76:0/3774707970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8fc0cd6d0 0x7fd8fc155840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:45.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.847+0000 7fd900fce640 1 -- 172.21.15.76:0/3774707970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8fc153e60 msgr2=0x7fd8fc0cb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:45.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.847+0000 7fd900fce640 1 --2- 172.21.15.76:0/3774707970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8fc153e60 0x7fd8fc0cb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:45.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.847+0000 7fd900fce640 1 -- 172.21.15.76:0/3774707970 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ec0094d0 con 0x7fd8fc0b4880 2023-11-09T03:47:45.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.847+0000 7fd8fbfff640 1 --2- 172.21.15.76:0/3774707970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8fc0cd6d0 0x7fd8fc155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:45.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.848+0000 7fd900fce640 1 --2- 172.21.15.76:0/3774707970 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8fc0b4880 0x7fd8fc153310 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fd8f400c7c0 tx=0x7fd8f400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:45.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.848+0000 7fd8f97fa640 1 -- 172.21.15.76:0/3774707970 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f400a640 con 0x7fd8fc0b4880 2023-11-09T03:47:45.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.848+0000 7fd8f97fa640 1 -- 172.21.15.76:0/3774707970 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f400a7c0 con 0x7fd8fc0b4880 2023-11-09T03:47:45.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.848+0000 7fd903232640 1 -- 172.21.15.76:0/3774707970 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e4009400 con 0x7fd8fc0b4880 2023-11-09T03:47:45.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.848+0000 7fd8f97fa640 1 -- 172.21.15.76:0/3774707970 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f400ab70 con 0x7fd8fc0b4880 2023-11-09T03:47:45.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.848+0000 7fd903232640 1 -- 172.21.15.76:0/3774707970 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8fc158440 con 0x7fd8fc0b4880 2023-11-09T03:47:45.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.849+0000 7fd903232640 1 -- 172.21.15.76:0/3774707970 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8fc074290 con 0x7fd8fc0b4880 2023-11-09T03:47:45.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.849+0000 7fd8f97fa640 1 -- 172.21.15.76:0/3774707970 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd8f400acd0 con 0x7fd8fc0b4880 2023-11-09T03:47:45.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.849+0000 7fd8f97fa640 1 --2- 172.21.15.76:0/3774707970 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd8dc06ae70 0x7fd8dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:45.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.849+0000 7fd8f97fa640 1 -- 172.21.15.76:0/3774707970 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fd8f4088d00 con 0x7fd8fc0b4880 2023-11-09T03:47:45.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.850+0000 7fd8fbfff640 1 --2- 172.21.15.76:0/3774707970 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd8dc06ae70 0x7fd8dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:45.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.850+0000 7fd8fbfff640 1 --2- 172.21.15.76:0/3774707970 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd8dc06ae70 0x7fd8dc06d330 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4036000 tx=0x7fd8e4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:45.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:45.854+0000 7fd8f97fa640 1 -- 172.21.15.76:0/3774707970 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd8f4053f50 con 0x7fd8fc0b4880 2023-11-09T03:47:46.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.020+0000 7fd903232640 1 -- 172.21.15.76:0/3774707970 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd8fc0d0b80 con 0x7fd8fc0b4880 2023-11-09T03:47:46.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.023+0000 7fd8f97fa640 1 -- 172.21.15.76:0/3774707970 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v15)=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd8f4018020 con 0x7fd8fc0b4880 2023-11-09T03:47:46.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.025+0000 7fd903232640 1 -- 172.21.15.76:0/3774707970 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd8dc06ae70 msgr2=0x7fd8dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:46.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.025+0000 7fd903232640 1 --2- 172.21.15.76:0/3774707970 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd8dc06ae70 0x7fd8dc06d330 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4036000 tx=0x7fd8e4008030 comp rx=0 tx=0).stop 2023-11-09T03:47:46.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.025+0000 7fd903232640 1 -- 172.21.15.76:0/3774707970 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8fc0b4880 msgr2=0x7fd8fc153310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:46.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.025+0000 7fd903232640 1 --2- 172.21.15.76:0/3774707970 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8fc0b4880 0x7fd8fc153310 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fd8f400c7c0 tx=0x7fd8f400cc80 comp rx=0 tx=0).stop 2023-11-09T03:47:46.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.025+0000 7fd903232640 1 -- 172.21.15.76:0/3774707970 shutdown_connections 2023-11-09T03:47:46.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.025+0000 7fd903232640 1 --2- 172.21.15.76:0/3774707970 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd8dc06ae70 0x7fd8dc06d330 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:46.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.025+0000 7fd903232640 1 --2- 172.21.15.76:0/3774707970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8fc153e60 0x7fd8fc0cb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:46.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.025+0000 7fd903232640 1 --2- 172.21.15.76:0/3774707970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8fc0cd6d0 0x7fd8fc155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:46.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.025+0000 7fd903232640 1 --2- 172.21.15.76:0/3774707970 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8fc0b4880 0x7fd8fc153310 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:46.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.025+0000 7fd903232640 1 -- 172.21.15.76:0/3774707970 >> 172.21.15.76:0/3774707970 conn(0x7fd8fc0cf880 msgr2=0x7fd8fc0d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:46.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.025+0000 7fd903232640 1 -- 172.21.15.76:0/3774707970 shutdown_connections 2023-11-09T03:47:46.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.025+0000 7fd903232640 1 -- 172.21.15.76:0/3774707970 wait complete. 2023-11-09T03:47:46.036 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-09T03:47:46.036 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-09T03:47:46.078 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-09T03:47:46.126 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-09T03:47:46.127 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:47:46.127 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:47:46.142 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056381,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14761 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248868k,nr_inodes=812217,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-09T03:47:46.142 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:47:46.143 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:47:46.160 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2440 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-09T03:47:46.160 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:47:46.160 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:47:46.196 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056381,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14761 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248868k,nr_inodes=812217,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-09T03:47:46.197 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:47:46.197 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:47:46.216 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2440 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-09T03:47:46.216 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-09T03:47:46.217 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:47:46.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.336+0000 7f1dc6e39640 1 Processor -- start 2023-11-09T03:47:46.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.336+0000 7f1dc6e39640 1 -- start start 2023-11-09T03:47:46.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.337+0000 7f1dc6e39640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1dc0159c70 0x7f1dc00ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:46.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.337+0000 7f1dc6e39640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1dc006db20 0x7f1dc00b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:46.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.337+0000 7f1dc6e39640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1dc00b0910 0x7f1dc015d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:46.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.337+0000 7f1dc6e39640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dc00b49b0 con 0x7f1dc0159c70 2023-11-09T03:47:46.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.337+0000 7f1dc6e39640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dc00b0da0 con 0x7f1dc00b0910 2023-11-09T03:47:46.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.337+0000 7f1dc6e39640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc00b4110 con 0x7f1dc006db20 2023-11-09T03:47:46.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.337+0000 7f1dc4bd5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1dc0159c70 0x7f1dc00ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:46.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.337+0000 7f1dc4bd5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1dc0159c70 0x7f1dc00ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47770/0 (socket says 172.21.15.76:47770) 2023-11-09T03:47:46.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.337+0000 7f1dc4bd5640 1 -- 172.21.15.76:0/3043371074 learned_addr learned my addr 172.21.15.76:0/3043371074 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:46.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.338+0000 7f1dc4bd5640 1 -- 172.21.15.76:0/3043371074 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1dc006db20 msgr2=0x7f1dc00b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:46.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.338+0000 7f1dbffff640 1 --2- 172.21.15.76:0/3043371074 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1dc006db20 0x7f1dc00b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:46.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.338+0000 7f1dc53d6640 1 --2- 172.21.15.76:0/3043371074 >> v2:172.21.15.113:3300/0 conn(0x7f1dc00b0910 0x7f1dc015d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:46.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.338+0000 7f1dc4bd5640 1 --2- 172.21.15.76:0/3043371074 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1dc006db20 0x7f1dc00b03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:46.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.338+0000 7f1dc4bd5640 1 -- 172.21.15.76:0/3043371074 >> v2:172.21.15.113:3300/0 conn(0x7f1dc00b0910 msgr2=0x7f1dc015d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:46.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.338+0000 7f1dc4bd5640 1 --2- 172.21.15.76:0/3043371074 >> v2:172.21.15.113:3300/0 conn(0x7f1dc00b0910 0x7f1dc015d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:46.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.338+0000 7f1dc4bd5640 1 -- 172.21.15.76:0/3043371074 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc00b43f0 con 0x7f1dc0159c70 2023-11-09T03:47:46.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.338+0000 7f1dc4bd5640 1 --2- 172.21.15.76:0/3043371074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dc0159c70 0x7f1dc00ce370 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f1dc00ca130 tx=0x7f1db002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f0a2e5d8ee3780 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:46.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.339+0000 7f1dbf7fe640 1 -- 172.21.15.76:0/3043371074 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db0031870 con 0x7f1dc0159c70 2023-11-09T03:47:46.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.339+0000 7f1dbf7fe640 1 -- 172.21.15.76:0/3043371074 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1db00319d0 con 0x7f1dc0159c70 2023-11-09T03:47:46.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.339+0000 7f1dbf7fe640 1 -- 172.21.15.76:0/3043371074 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db0031d60 con 0x7f1dc0159c70 2023-11-09T03:47:46.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.339+0000 7f1dc6e39640 1 -- 172.21.15.76:0/3043371074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dc0159c70 msgr2=0x7f1dc00ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:46.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.339+0000 7f1dc6e39640 1 --2- 172.21.15.76:0/3043371074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dc0159c70 0x7f1dc00ce370 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f1dc00ca130 tx=0x7f1db002b910 comp rx=0 tx=0).stop 2023-11-09T03:47:46.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.339+0000 7f1dc6e39640 1 -- 172.21.15.76:0/3043371074 shutdown_connections 2023-11-09T03:47:46.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.339+0000 7f1dc6e39640 1 --2- 172.21.15.76:0/3043371074 >> v2:172.21.15.113:3300/0 conn(0x7f1dc00b0910 0x7f1dc015d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:46.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.339+0000 7f1dc6e39640 1 --2- 172.21.15.76:0/3043371074 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1dc006db20 0x7f1dc00b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:46.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.339+0000 7f1dc6e39640 1 --2- 172.21.15.76:0/3043371074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dc0159c70 0x7f1dc00ce370 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:46.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.339+0000 7f1dc6e39640 1 -- 172.21.15.76:0/3043371074 >> 172.21.15.76:0/3043371074 conn(0x7f1dc00ce950 msgr2=0x7f1dc00ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:46.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.339+0000 7f1dc6e39640 1 -- 172.21.15.76:0/3043371074 shutdown_connections 2023-11-09T03:47:46.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.339+0000 7f1dc6e39640 1 -- 172.21.15.76:0/3043371074 wait complete. 2023-11-09T03:47:46.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.340+0000 7f1dc6e39640 1 Processor -- start 2023-11-09T03:47:46.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.340+0000 7f1dc6e39640 1 -- start start 2023-11-09T03:47:46.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.341+0000 7f1dc6e39640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1dc006db20 0x7f1dc015b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:46.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.341+0000 7f1dc6e39640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1dc00b0910 0x7f1dc01f29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:46.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.341+0000 7f1dc6e39640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dc0159c70 0x7f1dc01f2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:46.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.341+0000 7f1dc6e39640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc0151a00 con 0x7f1dc0159c70 2023-11-09T03:47:46.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.341+0000 7f1dc6e39640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc015c700 con 0x7f1dc006db20 2023-11-09T03:47:46.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.341+0000 7f1dc6e39640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc015baf0 con 0x7f1dc00b0910 2023-11-09T03:47:46.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.341+0000 7f1dc53d6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dc0159c70 0x7f1dc01f2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:46.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.341+0000 7f1dc53d6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dc0159c70 0x7f1dc01f2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47786/0 (socket says 172.21.15.76:47786) 2023-11-09T03:47:46.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.341+0000 7f1dc53d6640 1 -- 172.21.15.76:0/1405843322 learned_addr learned my addr 172.21.15.76:0/1405843322 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:46.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.341+0000 7f1dc53d6640 1 -- 172.21.15.76:0/1405843322 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1dc00b0910 msgr2=0x7f1dc01f29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:46.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.341+0000 7f1dc53d6640 1 --2- 172.21.15.76:0/1405843322 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1dc00b0910 0x7f1dc01f29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:46.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.341+0000 7f1dc53d6640 1 -- 172.21.15.76:0/1405843322 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1dc006db20 msgr2=0x7f1dc015b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:46.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.341+0000 7f1dc53d6640 1 --2- 172.21.15.76:0/1405843322 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1dc006db20 0x7f1dc015b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:46.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.341+0000 7f1dc53d6640 1 -- 172.21.15.76:0/1405843322 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db00094d0 con 0x7f1dc0159c70 2023-11-09T03:47:46.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.342+0000 7f1dc53d6640 1 --2- 172.21.15.76:0/1405843322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dc0159c70 0x7f1dc01f2ee0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1db800cd70 tx=0x7f1db800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:46.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.342+0000 7f1dbd7fa640 1 -- 172.21.15.76:0/1405843322 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db8003b90 con 0x7f1dc0159c70 2023-11-09T03:47:46.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.342+0000 7f1dbd7fa640 1 -- 172.21.15.76:0/1405843322 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1db8003cf0 con 0x7f1dc0159c70 2023-11-09T03:47:46.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.342+0000 7f1dc6e39640 1 -- 172.21.15.76:0/1405843322 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc015bdd0 con 0x7f1dc0159c70 2023-11-09T03:47:46.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.342+0000 7f1dbd7fa640 1 -- 172.21.15.76:0/1405843322 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db8004240 con 0x7f1dc0159c70 2023-11-09T03:47:46.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.342+0000 7f1dc6e39640 1 -- 172.21.15.76:0/1405843322 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc015c390 con 0x7f1dc0159c70 2023-11-09T03:47:46.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.343+0000 7f1dc6e39640 1 -- 172.21.15.76:0/1405843322 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dc0074290 con 0x7f1dc0159c70 2023-11-09T03:47:46.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.343+0000 7f1dbd7fa640 1 -- 172.21.15.76:0/1405843322 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1db80043a0 con 0x7f1dc0159c70 2023-11-09T03:47:46.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.344+0000 7f1dbd7fa640 1 --2- 172.21.15.76:0/1405843322 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1dac06ae70 0x7f1dac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:46.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.344+0000 7f1dbd7fa640 1 -- 172.21.15.76:0/1405843322 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f1db8012070 con 0x7f1dc0159c70 2023-11-09T03:47:46.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.344+0000 7f1dc4bd5640 1 --2- 172.21.15.76:0/1405843322 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1dac06ae70 0x7f1dac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:46.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.344+0000 7f1dc4bd5640 1 --2- 172.21.15.76:0/1405843322 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1dac06ae70 0x7f1dac06d330 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1db0036000 tx=0x7f1db0030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:46.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.348+0000 7f1dbd7fa640 1 -- 172.21.15.76:0/1405843322 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1db8055080 con 0x7f1dc0159c70 2023-11-09T03:47:46.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.549+0000 7f1dc6e39640 1 -- 172.21.15.76:0/1405843322 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1dc015c840 con 0x7f1dc0159c70 2023-11-09T03:47:46.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.550+0000 7f1dbd7fa640 1 -- 172.21.15.76:0/1405843322 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f1db80586a0 con 0x7f1dc0159c70 2023-11-09T03:47:46.550 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:46.551 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":25,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5099,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1844478483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1844478483},{"type":"v1","addr":"172.21.15.113:6836","nonce":1844478483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5102,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1111318612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1111318612},{"type":"v1","addr":"172.21.15.113:6837","nonce":1111318612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":6072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/2037805253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2037805253},{"type":"v1","addr":"172.21.15.76:6834","nonce":2037805253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":6423,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3794700971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":3794700971},{"type":"v1","addr":"172.21.15.76:6835","nonce":3794700971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[]} 2023-11-09T03:47:46.551 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 25 2023-11-09T03:47:46.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.551+0000 7f1dc6e39640 1 -- 172.21.15.76:0/1405843322 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1dac06ae70 msgr2=0x7f1dac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:46.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.551+0000 7f1dc6e39640 1 --2- 172.21.15.76:0/1405843322 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1dac06ae70 0x7f1dac06d330 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1db0036000 tx=0x7f1db0030000 comp rx=0 tx=0).stop 2023-11-09T03:47:46.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.552+0000 7f1dc6e39640 1 -- 172.21.15.76:0/1405843322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dc0159c70 msgr2=0x7f1dc01f2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:46.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.552+0000 7f1dc6e39640 1 --2- 172.21.15.76:0/1405843322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dc0159c70 0x7f1dc01f2ee0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1db800cd70 tx=0x7f1db800a660 comp rx=0 tx=0).stop 2023-11-09T03:47:46.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.552+0000 7f1dc6e39640 1 -- 172.21.15.76:0/1405843322 shutdown_connections 2023-11-09T03:47:46.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.552+0000 7f1dc6e39640 1 --2- 172.21.15.76:0/1405843322 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1dac06ae70 0x7f1dac06d330 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:46.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.552+0000 7f1dc6e39640 1 --2- 172.21.15.76:0/1405843322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dc0159c70 0x7f1dc01f2ee0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:46.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.552+0000 7f1dc6e39640 1 --2- 172.21.15.76:0/1405843322 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1dc00b0910 0x7f1dc01f29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:46.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.552+0000 7f1dc6e39640 1 --2- 172.21.15.76:0/1405843322 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1dc006db20 0x7f1dc015b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:46.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.552+0000 7f1dc6e39640 1 -- 172.21.15.76:0/1405843322 >> 172.21.15.76:0/1405843322 conn(0x7f1dc00ce950 msgr2=0x7f1dc00b1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:46.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.552+0000 7f1dc6e39640 1 -- 172.21.15.76:0/1405843322 shutdown_connections 2023-11-09T03:47:46.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.552+0000 7f1dc6e39640 1 -- 172.21.15.76:0/1405843322 wait complete. 2023-11-09T03:47:46.563 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-09T03:47:46.563 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-09T03:47:46.564 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-09T03:47:46.564 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:47:46.564 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-09T03:47:46.564 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-09T03:47:46.565 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-09T03:47:46.565 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:47:46.565 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-09T03:47:46.565 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-09T03:47:46.566 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-09T03:47:46.566 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:47:46.566 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-09T03:47:46.566 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-09T03:47:46.567 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-09T03:47:46.567 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:47:46.573 INFO:tasks.ceph.mds.a:Stopped 2023-11-09T03:47:46.573 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-09T03:47:46.574 INFO:tasks.ceph.mds.c:Stopped 2023-11-09T03:47:46.575 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-09T03:47:46.576 INFO:tasks.ceph.mds.b:Stopped 2023-11-09T03:47:46.577 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-09T03:47:46.578 INFO:tasks.ceph.mds.d:Stopped 2023-11-09T03:47:46.578 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-09T03:47:46.616 INFO:tasks.ceph.mds.a:Started 2023-11-09T03:47:46.616 DEBUG:teuthology.parallel:result is None 2023-11-09T03:47:46.617 INFO:tasks.ceph.mds.c:Started 2023-11-09T03:47:46.617 DEBUG:teuthology.parallel:result is None 2023-11-09T03:47:46.620 INFO:tasks.ceph.mds.b:Started 2023-11-09T03:47:46.620 DEBUG:teuthology.parallel:result is None 2023-11-09T03:47:46.622 INFO:tasks.ceph.mds.d:Started 2023-11-09T03:47:46.622 DEBUG:teuthology.parallel:result is None 2023-11-09T03:47:46.623 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-09T03:47:46.691 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2023-11-09T03:47:46.692 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2023-11-09T03:47:46.702 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2023-11-09T03:47:46.702 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2023-11-09T03:47:46.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.729+0000 7fa1d86c2640 1 Processor -- start 2023-11-09T03:47:46.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.729+0000 7fa1d86c2640 1 -- start start 2023-11-09T03:47:46.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.730+0000 7fa1d86c2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1d0153e60 0x7fa1d015dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:46.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.730+0000 7fa1d86c2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa1d00b4880 0x7fa1d00b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:46.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.730+0000 7fa1d86c2640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1d00cde60 con 0x7fa1d00cd6d0 2023-11-09T03:47:46.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.730+0000 7fa1d86c2640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1d00b5230 con 0x7fa1d00b4880 2023-11-09T03:47:46.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.730+0000 7fa1d86c2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d0160f90 con 0x7fa1d0153e60 2023-11-09T03:47:46.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.730+0000 7fa1d6c5f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa1d00b4880 0x7fa1d00b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:46.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.730+0000 7fa1d6c5f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa1d00b4880 0x7fa1d00b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47834/0 (socket says 172.21.15.76:47834) 2023-11-09T03:47:46.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.730+0000 7fa1d6c5f640 1 -- 172.21.15.76:0/590270176 learned_addr learned my addr 172.21.15.76:0/590270176 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:46.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.730+0000 7fa1d5c5d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa1d00cd6d0 0x7fa1d0161420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:50764/0 (socket says 172.21.15.76:50764) 2023-11-09T03:47:46.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.730+0000 7fa1d645e640 1 --2- 172.21.15.76:0/590270176 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1d0153e60 0x7fa1d015dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:46.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.730+0000 7fa1d6c5f640 1 -- 172.21.15.76:0/590270176 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1d0153e60 msgr2=0x7fa1d015dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:46.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.730+0000 7fa1d6c5f640 1 --2- 172.21.15.76:0/590270176 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1d0153e60 0x7fa1d015dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:46.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.730+0000 7fa1d6c5f640 1 -- 172.21.15.76:0/590270176 >> v1:172.21.15.113:6789/0 conn(0x7fa1d00cd6d0 legacy=0x7fa1d0161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:46.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.730+0000 7fa1d6c5f640 1 -- 172.21.15.76:0/590270176 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d015e400 con 0x7fa1d00b4880 2023-11-09T03:47:46.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.731+0000 7fa1d6c5f640 1 --2- 172.21.15.76:0/590270176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d00b4880 0x7fa1d00b4cf0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009e50 tx=0x7fa1c802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716417bf7f179869 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:46.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.731+0000 7fa1d545c640 1 -- 172.21.15.76:0/590270176 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c802cb00 con 0x7fa1d00b4880 2023-11-09T03:47:46.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.731+0000 7fa1d545c640 1 -- 172.21.15.76:0/590270176 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c802cc60 con 0x7fa1d00b4880 2023-11-09T03:47:46.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.731+0000 7fa1d545c640 1 -- 172.21.15.76:0/590270176 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c80343f0 con 0x7fa1d00b4880 2023-11-09T03:47:46.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.732+0000 7fa1d86c2640 1 -- 172.21.15.76:0/590270176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d00b4880 msgr2=0x7fa1d00b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:46.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.732+0000 7fa1d86c2640 1 --2- 172.21.15.76:0/590270176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d00b4880 0x7fa1d00b4cf0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009e50 tx=0x7fa1c802bf60 comp rx=0 tx=0).stop 2023-11-09T03:47:46.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.732+0000 7fa1d86c2640 1 -- 172.21.15.76:0/590270176 shutdown_connections 2023-11-09T03:47:46.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.732+0000 7fa1d86c2640 1 --2- 172.21.15.76:0/590270176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d00b4880 0x7fa1d00b4cf0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:46.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.732+0000 7fa1d86c2640 1 --2- 172.21.15.76:0/590270176 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1d0153e60 0x7fa1d015dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:46.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.732+0000 7fa1d86c2640 1 -- 172.21.15.76:0/590270176 >> 172.21.15.76:0/590270176 conn(0x7fa1d00cf880 msgr2=0x7fa1d00cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:46.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.732+0000 7fa1d86c2640 1 -- 172.21.15.76:0/590270176 shutdown_connections 2023-11-09T03:47:46.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.732+0000 7fa1d86c2640 1 -- 172.21.15.76:0/590270176 wait complete. 2023-11-09T03:47:46.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.733+0000 7fa1d86c2640 1 Processor -- start 2023-11-09T03:47:46.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.733+0000 7fa1d86c2640 1 -- start start 2023-11-09T03:47:46.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.734+0000 7fa1d86c2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1d00b4880 0x7fa1d0150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:46.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.734+0000 7fa1d86c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d00cd6d0 0x7fa1d00c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:46.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.734+0000 7fa1d86c2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d0153e60 0x7fa1d00c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:46.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.734+0000 7fa1d86c2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d01525a0 con 0x7fa1d00cd6d0 2023-11-09T03:47:46.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.734+0000 7fa1d86c2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d0151a20 con 0x7fa1d0153e60 2023-11-09T03:47:46.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.734+0000 7fa1d5c5d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d00cd6d0 0x7fa1d00c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:46.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.734+0000 7fa1d86c2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d0151d00 con 0x7fa1d00b4880 2023-11-09T03:47:46.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.734+0000 7fa1d5c5d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d00cd6d0 0x7fa1d00c6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47840/0 (socket says 172.21.15.76:47840) 2023-11-09T03:47:46.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.734+0000 7fa1d5c5d640 1 -- 172.21.15.76:0/2018901904 learned_addr learned my addr 172.21.15.76:0/2018901904 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:46.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.734+0000 7fa1d645e640 1 --2- 172.21.15.76:0/2018901904 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1d00b4880 0x7fa1d0150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:46.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.734+0000 7fa1d6c5f640 1 --2- 172.21.15.76:0/2018901904 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d0153e60 0x7fa1d00c6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:46.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.735+0000 7fa1d5c5d640 1 -- 172.21.15.76:0/2018901904 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1d00b4880 msgr2=0x7fa1d0150030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:46.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.735+0000 7fa1d5c5d640 1 --2- 172.21.15.76:0/2018901904 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1d00b4880 0x7fa1d0150030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:46.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.735+0000 7fa1d5c5d640 1 -- 172.21.15.76:0/2018901904 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d0153e60 msgr2=0x7fa1d00c6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:46.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.735+0000 7fa1d5c5d640 1 --2- 172.21.15.76:0/2018901904 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d0153e60 0x7fa1d00c6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:46.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.735+0000 7fa1d5c5d640 1 -- 172.21.15.76:0/2018901904 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c8009b30 con 0x7fa1d00cd6d0 2023-11-09T03:47:46.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.735+0000 7fa1d5c5d640 1 --2- 172.21.15.76:0/2018901904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d00cd6d0 0x7fa1d00c6930 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000c450 tx=0x7fa1c000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:46.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.736+0000 7fa1c6ffd640 1 -- 172.21.15.76:0/2018901904 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0007480 con 0x7fa1d00cd6d0 2023-11-09T03:47:46.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.736+0000 7fa1c6ffd640 1 -- 172.21.15.76:0/2018901904 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c00075e0 con 0x7fa1d00cd6d0 2023-11-09T03:47:46.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.736+0000 7fa1c6ffd640 1 -- 172.21.15.76:0/2018901904 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0004a00 con 0x7fa1d00cd6d0 2023-11-09T03:47:46.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.737+0000 7fa1d86c2640 1 -- 172.21.15.76:0/2018901904 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1d0151fe0 con 0x7fa1d00cd6d0 2023-11-09T03:47:46.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.737+0000 7fa1d86c2640 1 -- 172.21.15.76:0/2018901904 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1d0150570 con 0x7fa1d00cd6d0 2023-11-09T03:47:46.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.738+0000 7fa1d86c2640 1 -- 172.21.15.76:0/2018901904 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1d0140240 con 0x7fa1d00cd6d0 2023-11-09T03:47:46.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.739+0000 7fa1c6ffd640 1 -- 172.21.15.76:0/2018901904 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa1c0004b60 con 0x7fa1d00cd6d0 2023-11-09T03:47:46.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.739+0000 7fa1c6ffd640 1 --2- 172.21.15.76:0/2018901904 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa1b406ae70 0x7fa1b406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:46.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.739+0000 7fa1c6ffd640 1 -- 172.21.15.76:0/2018901904 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fa1c0088600 con 0x7fa1d00cd6d0 2023-11-09T03:47:46.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.739+0000 7fa1d645e640 1 --2- 172.21.15.76:0/2018901904 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa1b406ae70 0x7fa1b406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:46.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.740+0000 7fa1d645e640 1 --2- 172.21.15.76:0/2018901904 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa1b406ae70 0x7fa1b406d330 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc009aa0 tx=0x7fa1cc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:46.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.744+0000 7fa1c6ffd640 1 -- 172.21.15.76:0/2018901904 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa1d0140240 con 0x7fa1d00cd6d0 2023-11-09T03:47:46.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:46.924+0000 7fa1d86c2640 1 -- 172.21.15.76:0/2018901904 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa1d0140240 con 0x7fa1d00cd6d0 2023-11-09T03:47:47.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.752+0000 7fa1c6ffd640 1 -- 172.21.15.76:0/2018901904 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v62) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa1d0140240 con 0x7fa1d00cd6d0 2023-11-09T03:47:47.753 INFO:teuthology.orchestra.run.smithi076.stderr: removed all blocklist entries 2023-11-09T03:47:47.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.755+0000 7fa1d86c2640 1 -- 172.21.15.76:0/2018901904 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa1b406ae70 msgr2=0x7fa1b406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:47.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.755+0000 7fa1d86c2640 1 --2- 172.21.15.76:0/2018901904 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa1b406ae70 0x7fa1b406d330 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc009aa0 tx=0x7fa1cc009190 comp rx=0 tx=0).stop 2023-11-09T03:47:47.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.755+0000 7fa1d86c2640 1 -- 172.21.15.76:0/2018901904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d00cd6d0 msgr2=0x7fa1d00c6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:47.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.755+0000 7fa1d86c2640 1 --2- 172.21.15.76:0/2018901904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d00cd6d0 0x7fa1d00c6930 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000c450 tx=0x7fa1c000c910 comp rx=0 tx=0).stop 2023-11-09T03:47:47.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.755+0000 7fa1d86c2640 1 -- 172.21.15.76:0/2018901904 shutdown_connections 2023-11-09T03:47:47.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.755+0000 7fa1d86c2640 1 --2- 172.21.15.76:0/2018901904 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa1b406ae70 0x7fa1b406d330 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:47.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.755+0000 7fa1d86c2640 1 --2- 172.21.15.76:0/2018901904 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d0153e60 0x7fa1d00c6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:47.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.755+0000 7fa1d86c2640 1 --2- 172.21.15.76:0/2018901904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d00cd6d0 0x7fa1d00c6930 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:47.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.755+0000 7fa1d86c2640 1 --2- 172.21.15.76:0/2018901904 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1d00b4880 0x7fa1d0150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:47.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.755+0000 7fa1d86c2640 1 -- 172.21.15.76:0/2018901904 >> 172.21.15.76:0/2018901904 conn(0x7fa1d00cf880 msgr2=0x7fa1d015fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:47.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.755+0000 7fa1d86c2640 1 -- 172.21.15.76:0/2018901904 shutdown_connections 2023-11-09T03:47:47.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.755+0000 7fa1d86c2640 1 -- 172.21.15.76:0/2018901904 wait complete. 2023-11-09T03:47:47.768 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-09T03:47:47.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.873+0000 7fc2fa94c640 1 Processor -- start 2023-11-09T03:47:47.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.873+0000 7fc2fa94c640 1 -- start start 2023-11-09T03:47:47.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.874+0000 7fc2fa94c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc2f4159c70 0x7fc2f40ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:47.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.874+0000 7fc2fa94c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2f406db20 0x7fc2f40b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:47.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.874+0000 7fc2fa94c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc2f40b0910 0x7fc2f415d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:47.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.874+0000 7fc2fa94c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2f40b49b0 con 0x7fc2f40b0910 2023-11-09T03:47:47.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.874+0000 7fc2fa94c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2f40b0da0 con 0x7fc2f4159c70 2023-11-09T03:47:47.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.874+0000 7fc2fa94c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f40b4110 con 0x7fc2f406db20 2023-11-09T03:47:47.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.874+0000 7fc2f8ee9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc2f40b0910 0x7fc2f415d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:47.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.874+0000 7fc2f8ee9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc2f40b0910 0x7fc2f415d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47842/0 (socket says 172.21.15.76:47842) 2023-11-09T03:47:47.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.874+0000 7fc2f8ee9640 1 -- 172.21.15.76:0/3305837018 learned_addr learned my addr 172.21.15.76:0/3305837018 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:47.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.875+0000 7fc2f3fff640 1 --2- 172.21.15.76:0/3305837018 >> v2:172.21.15.113:3300/0 conn(0x7fc2f4159c70 0x7fc2f40ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:47.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.875+0000 7fc2f37fe640 1 --2- 172.21.15.76:0/3305837018 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2f406db20 0x7fc2f40b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:47.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.875+0000 7fc2f8ee9640 1 -- 172.21.15.76:0/3305837018 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2f406db20 msgr2=0x7fc2f40b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:47.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.875+0000 7fc2f8ee9640 1 --2- 172.21.15.76:0/3305837018 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2f406db20 0x7fc2f40b03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:47.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.875+0000 7fc2f8ee9640 1 -- 172.21.15.76:0/3305837018 >> v2:172.21.15.113:3300/0 conn(0x7fc2f4159c70 msgr2=0x7fc2f40ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:47.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.875+0000 7fc2f8ee9640 1 --2- 172.21.15.76:0/3305837018 >> v2:172.21.15.113:3300/0 conn(0x7fc2f4159c70 0x7fc2f40ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:47.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.875+0000 7fc2f8ee9640 1 -- 172.21.15.76:0/3305837018 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f40b43f0 con 0x7fc2f40b0910 2023-11-09T03:47:47.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.875+0000 7fc2f3fff640 1 --2- 172.21.15.76:0/3305837018 >> v2:172.21.15.113:3300/0 conn(0x7fc2f4159c70 0x7fc2f40ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:47.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.875+0000 7fc2f8ee9640 1 --2- 172.21.15.76:0/3305837018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2f40b0910 0x7fc2f415d7d0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fc2f40cc440 tx=0x7fc2ec02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8239a325a327aa76 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:47.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.876+0000 7fc2f2ffd640 1 -- 172.21.15.76:0/3305837018 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec02ca70 con 0x7fc2f40b0910 2023-11-09T03:47:47.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.876+0000 7fc2f2ffd640 1 -- 172.21.15.76:0/3305837018 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ec02cbd0 con 0x7fc2f40b0910 2023-11-09T03:47:47.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.876+0000 7fc2f2ffd640 1 -- 172.21.15.76:0/3305837018 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec0343f0 con 0x7fc2f40b0910 2023-11-09T03:47:47.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.876+0000 7fc2fa94c640 1 -- 172.21.15.76:0/3305837018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2f40b0910 msgr2=0x7fc2f415d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:47.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.876+0000 7fc2fa94c640 1 --2- 172.21.15.76:0/3305837018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2f40b0910 0x7fc2f415d7d0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fc2f40cc440 tx=0x7fc2ec02bed0 comp rx=0 tx=0).stop 2023-11-09T03:47:47.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.876+0000 7fc2fa94c640 1 -- 172.21.15.76:0/3305837018 shutdown_connections 2023-11-09T03:47:47.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.876+0000 7fc2fa94c640 1 --2- 172.21.15.76:0/3305837018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2f40b0910 0x7fc2f415d7d0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:47.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.876+0000 7fc2fa94c640 1 --2- 172.21.15.76:0/3305837018 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2f406db20 0x7fc2f40b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:47.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.876+0000 7fc2fa94c640 1 --2- 172.21.15.76:0/3305837018 >> v2:172.21.15.113:3300/0 conn(0x7fc2f4159c70 0x7fc2f40ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:47.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.876+0000 7fc2fa94c640 1 -- 172.21.15.76:0/3305837018 >> 172.21.15.76:0/3305837018 conn(0x7fc2f40ce950 msgr2=0x7fc2f40ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:47.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.876+0000 7fc2fa94c640 1 -- 172.21.15.76:0/3305837018 shutdown_connections 2023-11-09T03:47:47.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.876+0000 7fc2fa94c640 1 -- 172.21.15.76:0/3305837018 wait complete. 2023-11-09T03:47:47.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.877+0000 7fc2fa94c640 1 Processor -- start 2023-11-09T03:47:47.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.878+0000 7fc2fa94c640 1 -- start start 2023-11-09T03:47:47.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.878+0000 7fc2fa94c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2f406db20 0x7fc2f414f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:47.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.878+0000 7fc2fa94c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2f40b0910 0x7fc2f41533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:47.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.878+0000 7fc2fa94c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2f4159c70 0x7fc2f40cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:47.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.878+0000 7fc2fa94c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f41f6820 con 0x7fc2f40b0910 2023-11-09T03:47:47.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.878+0000 7fc2fa94c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f414f550 con 0x7fc2f4159c70 2023-11-09T03:47:47.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.878+0000 7fc2fa94c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f41f5e80 con 0x7fc2f406db20 2023-11-09T03:47:47.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.878+0000 7fc2f37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2f40b0910 0x7fc2f41533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:47.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.878+0000 7fc2f37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2f40b0910 0x7fc2f41533c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47854/0 (socket says 172.21.15.76:47854) 2023-11-09T03:47:47.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.878+0000 7fc2f37fe640 1 -- 172.21.15.76:0/609186534 learned_addr learned my addr 172.21.15.76:0/609186534 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:47.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.879+0000 7fc2f8ee9640 1 --2- 172.21.15.76:0/609186534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2f4159c70 0x7fc2f40cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:47.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.879+0000 7fc2f3fff640 1 --2- 172.21.15.76:0/609186534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2f406db20 0x7fc2f414f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:47.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.879+0000 7fc2f37fe640 1 -- 172.21.15.76:0/609186534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2f406db20 msgr2=0x7fc2f414f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:47.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.879+0000 7fc2f37fe640 1 --2- 172.21.15.76:0/609186534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2f406db20 0x7fc2f414f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:47.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.879+0000 7fc2f37fe640 1 -- 172.21.15.76:0/609186534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2f4159c70 msgr2=0x7fc2f40cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:47.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.879+0000 7fc2f37fe640 1 --2- 172.21.15.76:0/609186534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2f4159c70 0x7fc2f40cca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:47.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.879+0000 7fc2f37fe640 1 -- 172.21.15.76:0/609186534 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ec009b30 con 0x7fc2f40b0910 2023-11-09T03:47:47.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.879+0000 7fc2f8ee9640 1 --2- 172.21.15.76:0/609186534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2f4159c70 0x7fc2f40cca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:47.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.879+0000 7fc2f37fe640 1 --2- 172.21.15.76:0/609186534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2f40b0910 0x7fc2f41533c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800d800 tx=0x7fc2e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:47.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.879+0000 7fc2f0ff9640 1 -- 172.21.15.76:0/609186534 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e80096a0 con 0x7fc2f40b0910 2023-11-09T03:47:47.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.879+0000 7fc2f0ff9640 1 -- 172.21.15.76:0/609186534 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e8009800 con 0x7fc2f40b0910 2023-11-09T03:47:47.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.879+0000 7fc2fa94c640 1 -- 172.21.15.76:0/609186534 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f41f6160 con 0x7fc2f40b0910 2023-11-09T03:47:47.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.879+0000 7fc2f0ff9640 1 -- 172.21.15.76:0/609186534 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8009b90 con 0x7fc2f40b0910 2023-11-09T03:47:47.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.879+0000 7fc2fa94c640 1 -- 172.21.15.76:0/609186534 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f41f6c40 con 0x7fc2f40b0910 2023-11-09T03:47:47.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.880+0000 7fc2fa94c640 1 -- 172.21.15.76:0/609186534 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2f41f6a90 con 0x7fc2f40b0910 2023-11-09T03:47:47.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.880+0000 7fc2f0ff9640 1 -- 172.21.15.76:0/609186534 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc2e8009cf0 con 0x7fc2f40b0910 2023-11-09T03:47:47.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.881+0000 7fc2f0ff9640 1 --2- 172.21.15.76:0/609186534 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc2e006ae70 0x7fc2e006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:47.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.881+0000 7fc2f0ff9640 1 -- 172.21.15.76:0/609186534 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fc2e8089950 con 0x7fc2f40b0910 2023-11-09T03:47:47.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.881+0000 7fc2f3fff640 1 --2- 172.21.15.76:0/609186534 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc2e006ae70 0x7fc2e006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:47.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.882+0000 7fc2f3fff640 1 --2- 172.21.15.76:0/609186534 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc2e006ae70 0x7fc2e006d330 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc2e40096d0 tx=0x7fc2e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:47.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:47.886+0000 7fc2f0ff9640 1 -- 172.21.15.76:0/609186534 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc2e8054b40 con 0x7fc2f40b0910 2023-11-09T03:47:48.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.114+0000 7fc2fa94c640 1 -- 172.21.15.76:0/609186534 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc2f4074290 con 0x7fc2f40b0910 2023-11-09T03:47:48.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.114+0000 7fc2f0ff9640 1 -- 172.21.15.76:0/609186534 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc2e800b440 con 0x7fc2f40b0910 2023-11-09T03:47:48.115 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:48.115 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:47:48.115 INFO:teuthology.orchestra.run.smithi076.stdout: "auth_dump": [ 2023-11-09T03:47:48.116 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:48.116 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.a", 2023-11-09T03:47:48.116 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCgVUxlydtlLxAAi0bnTEOD+fpGL1cYic3caA==", 2023-11-09T03:47:48.116 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:48.116 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:47:48.116 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2023-11-09T03:47:48.117 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2023-11-09T03:47:48.117 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:48.117 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:48.117 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:48.117 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:48.118 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.b", 2023-11-09T03:47:48.118 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxl2+QnABAA7CTWWEbPPXkollRFDlLxPA==", 2023-11-09T03:47:48.118 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:48.118 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:47:48.118 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2023-11-09T03:47:48.118 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2023-11-09T03:47:48.119 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:48.119 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:48.119 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:48.119 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:48.119 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.c", 2023-11-09T03:47:48.120 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCgVUxlrTu9NhAAhg4GpZWZTmVPCP3ffjPo/Q==", 2023-11-09T03:47:48.120 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:48.120 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:47:48.120 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2023-11-09T03:47:48.120 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2023-11-09T03:47:48.120 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:48.121 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:48.121 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:48.121 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:48.121 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.d", 2023-11-09T03:47:48.121 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxl4IZ8BxAA8zGFE+1tlHkT5UyzL0nWxQ==", 2023-11-09T03:47:48.121 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:48.122 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:47:48.122 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2023-11-09T03:47:48.122 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2023-11-09T03:47:48.122 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:48.122 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:48.122 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:48.123 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:48.123 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.0", 2023-11-09T03:47:48.123 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCkVUxlXM5LKhAAM253+XEj5YSwmQywwDieTg==", 2023-11-09T03:47:48.123 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:48.123 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:47:48.123 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:47:48.124 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:48.124 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:48.124 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:48.124 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:48.124 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.1", 2023-11-09T03:47:48.125 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCnVUxlgoDVKBAAkmNFo4E0bAQ1UxkCfQAbjw==", 2023-11-09T03:47:48.125 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:48.125 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:47:48.125 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:47:48.125 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:48.125 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:48.126 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:48.126 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:48.126 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.2", 2023-11-09T03:47:48.126 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCqVUxlzQjVJhAAOoU5IdN86BmcPz71mTq7ng==", 2023-11-09T03:47:48.126 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:48.127 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:47:48.127 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:47:48.127 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:48.127 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:48.127 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:48.127 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:48.128 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.3", 2023-11-09T03:47:48.128 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCtVUxlhY6VJBAAmCO/ZS8vBa7enyuXX5Y+hw==", 2023-11-09T03:47:48.128 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:48.128 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:47:48.128 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:47:48.128 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:48.129 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:48.129 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:48.129 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:48.129 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.4", 2023-11-09T03:47:48.129 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCzVUxl79egLxAAnKBOQb/9EQyaHaQt7SawIg==", 2023-11-09T03:47:48.129 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:48.130 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:47:48.130 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:47:48.130 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:48.130 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:48.130 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:48.131 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:48.131 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.5", 2023-11-09T03:47:48.131 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQC2VUxlIfUgLhAAVPurhJCc8cmpt11pQjzL9w==", 2023-11-09T03:47:48.131 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:48.131 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:47:48.131 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:47:48.132 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:48.132 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:48.132 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:48.132 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:48.132 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.6", 2023-11-09T03:47:48.132 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQC5VUxlrI58LBAAWxZotpACR4e8vFj4IiaEKA==", 2023-11-09T03:47:48.133 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:48.133 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:47:48.133 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:47:48.133 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:48.133 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:48.133 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:48.134 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:48.134 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.7", 2023-11-09T03:47:48.134 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQC8VUxl0bA3KxAAwnKNJKqYIcGRLLhQHUaU9w==", 2023-11-09T03:47:48.134 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:48.134 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:47:48.135 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:47:48.135 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:48.135 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:48.135 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:48.135 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:48.135 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.0", 2023-11-09T03:47:48.136 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxlHj22CxAACNj5X24hdME+GNTLw4RahQ==", 2023-11-09T03:47:48.136 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:48.136 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:47:48.136 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow r", 2023-11-09T03:47:48.136 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-09T03:47:48.136 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:48.137 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:48.137 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:48.137 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.1", 2023-11-09T03:47:48.137 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxljXv4FBAAiTOhfIReQWTy6fbujbBrrw==", 2023-11-09T03:47:48.137 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:48.138 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:47:48.138 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2023-11-09T03:47:48.138 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2023-11-09T03:47:48.138 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2023-11-09T03:47:48.138 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:48.138 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:48.139 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:48.139 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.2", 2023-11-09T03:47:48.139 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxlGVSSERAAQts5odubZ5paZwg8US3QrQ==", 2023-11-09T03:47:48.139 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:48.139 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:47:48.139 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2023-11-09T03:47:48.140 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2023-11-09T03:47:48.140 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2023-11-09T03:47:48.140 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:48.140 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:48.140 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:48.141 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.3", 2023-11-09T03:47:48.141 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxlgmDHGhAA7r98HNQEMnjbqW6RjE5aig==", 2023-11-09T03:47:48.141 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:48.141 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:47:48.141 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2023-11-09T03:47:48.141 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2023-11-09T03:47:48.142 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2023-11-09T03:47:48.142 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:48.142 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:48.142 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:48.142 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.admin", 2023-11-09T03:47:48.143 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCgVUxlV4l8EhAAnrgeLaTaR99vNYGCJjI1TA==", 2023-11-09T03:47:48.143 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:48.143 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2023-11-09T03:47:48.143 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2023-11-09T03:47:48.143 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2023-11-09T03:47:48.143 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:48.144 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:48.144 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:48.144 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:48.144 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mgr.x", 2023-11-09T03:47:48.144 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCgVUxl7JryKxAABFx9CRqSbbJu8SIyi3Va8w==", 2023-11-09T03:47:48.144 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:48.145 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2023-11-09T03:47:48.145 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile mgr", 2023-11-09T03:47:48.145 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:48.145 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:48.145 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:48.145 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:48.146 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mgr.y", 2023-11-09T03:47:48.146 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCgVUxlb9L/JxAAu3IOXTn8TGvbzxYVta3yew==", 2023-11-09T03:47:48.146 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:47:48.146 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2023-11-09T03:47:48.146 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile mgr", 2023-11-09T03:47:48.147 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:47:48.147 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:48.147 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:48.147 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2023-11-09T03:47:48.147 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:47:48.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.116+0000 7fc2fa94c640 1 -- 172.21.15.76:0/609186534 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc2e006ae70 msgr2=0x7fc2e006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:48.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.116+0000 7fc2fa94c640 1 --2- 172.21.15.76:0/609186534 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc2e006ae70 0x7fc2e006d330 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc2e40096d0 tx=0x7fc2e4008040 comp rx=0 tx=0).stop 2023-11-09T03:47:48.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.116+0000 7fc2fa94c640 1 -- 172.21.15.76:0/609186534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2f40b0910 msgr2=0x7fc2f41533c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:48.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.116+0000 7fc2fa94c640 1 --2- 172.21.15.76:0/609186534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2f40b0910 0x7fc2f41533c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800d800 tx=0x7fc2e800dcc0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.116+0000 7fc2fa94c640 1 -- 172.21.15.76:0/609186534 shutdown_connections 2023-11-09T03:47:48.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.116+0000 7fc2fa94c640 1 --2- 172.21.15.76:0/609186534 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc2e006ae70 0x7fc2e006d330 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.116+0000 7fc2fa94c640 1 --2- 172.21.15.76:0/609186534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2f4159c70 0x7fc2f40cca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.116+0000 7fc2fa94c640 1 --2- 172.21.15.76:0/609186534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2f40b0910 0x7fc2f41533c0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.116+0000 7fc2fa94c640 1 --2- 172.21.15.76:0/609186534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2f406db20 0x7fc2f414f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.116+0000 7fc2fa94c640 1 -- 172.21.15.76:0/609186534 >> 172.21.15.76:0/609186534 conn(0x7fc2f40ce950 msgr2=0x7fc2f415e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:48.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.116+0000 7fc2fa94c640 1 -- 172.21.15.76:0/609186534 shutdown_connections 2023-11-09T03:47:48.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.116+0000 7fc2fa94c640 1 -- 172.21.15.76:0/609186534 wait complete. 2023-11-09T03:47:48.151 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:47:48.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.250+0000 7fd37cc44640 1 Processor -- start 2023-11-09T03:47:48.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.251+0000 7fd37cc44640 1 -- start start 2023-11-09T03:47:48.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.251+0000 7fd37cc44640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3780b0ae0 0x7fd3780ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:48.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.251+0000 7fd37cc44640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd37806db20 0x7fd37815d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:48.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.251+0000 7fd37cc44640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3780b4110 con 0x7fd378159c70 2023-11-09T03:47:48.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.251+0000 7fd37cc44640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3780b49b0 con 0x7fd37806db20 2023-11-09T03:47:48.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.251+0000 7fd37cc44640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3780b43f0 con 0x7fd3780b0ae0 2023-11-09T03:47:48.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.252+0000 7fd375d9b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3780b0ae0 0x7fd3780ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:48.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.252+0000 7fd376d9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd37806db20 0x7fd37815d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:48.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.252+0000 7fd375d9b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3780b0ae0 0x7fd3780ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:58768/0 (socket says 172.21.15.76:58768) 2023-11-09T03:47:48.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.252+0000 7fd375d9b640 1 -- 172.21.15.76:0/3919154163 learned_addr learned my addr 172.21.15.76:0/3919154163 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:48.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.252+0000 7fd376d9d640 1 -- 172.21.15.76:0/3919154163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3780b0ae0 msgr2=0x7fd3780ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:48.255 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.252+0000 7fd376d9d640 1 --2- 172.21.15.76:0/3919154163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3780b0ae0 0x7fd3780ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.255 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.252+0000 7fd376d9d640 1 -- 172.21.15.76:0/3919154163 >> v1:172.21.15.113:6789/0 conn(0x7fd378159c70 legacy=0x7fd3780b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:48.255 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.252+0000 7fd376d9d640 1 -- 172.21.15.76:0/3919154163 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3780b46d0 con 0x7fd37806db20 2023-11-09T03:47:48.255 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.252+0000 7fd375d9b640 1 --2- 172.21.15.76:0/3919154163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3780b0ae0 0x7fd3780ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:47:48.255 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.253+0000 7fd376d9d640 1 --2- 172.21.15.76:0/3919154163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd37806db20 0x7fd37815d7d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd370009d90 tx=0x7fd37002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e03f42ecb18c1605 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:48.255 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.253+0000 7fd37559a640 1 -- 172.21.15.76:0/3919154163 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37002cc50 con 0x7fd37806db20 2023-11-09T03:47:48.256 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.253+0000 7fd37559a640 1 -- 172.21.15.76:0/3919154163 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37002cdb0 con 0x7fd37806db20 2023-11-09T03:47:48.256 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.253+0000 7fd37559a640 1 -- 172.21.15.76:0/3919154163 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd370034660 con 0x7fd37806db20 2023-11-09T03:47:48.258 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.254+0000 7fd37cc44640 1 -- 172.21.15.76:0/3919154163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd37806db20 msgr2=0x7fd37815d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:48.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.254+0000 7fd37cc44640 1 --2- 172.21.15.76:0/3919154163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd37806db20 0x7fd37815d7d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd370009d90 tx=0x7fd37002bea0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.254+0000 7fd37cc44640 1 -- 172.21.15.76:0/3919154163 shutdown_connections 2023-11-09T03:47:48.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.254+0000 7fd37cc44640 1 --2- 172.21.15.76:0/3919154163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd37806db20 0x7fd37815d7d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.254+0000 7fd37cc44640 1 --2- 172.21.15.76:0/3919154163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3780b0ae0 0x7fd3780ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.254+0000 7fd37cc44640 1 -- 172.21.15.76:0/3919154163 >> 172.21.15.76:0/3919154163 conn(0x7fd3780ce950 msgr2=0x7fd3780ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:48.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.254+0000 7fd37cc44640 1 -- 172.21.15.76:0/3919154163 shutdown_connections 2023-11-09T03:47:48.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.254+0000 7fd37cc44640 1 -- 172.21.15.76:0/3919154163 wait complete. 2023-11-09T03:47:48.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.255+0000 7fd37cc44640 1 Processor -- start 2023-11-09T03:47:48.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.255+0000 7fd37cc44640 1 -- start start 2023-11-09T03:47:48.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.256+0000 7fd37cc44640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd37806db20 0x7fd37814ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:48.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.256+0000 7fd37cc44640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3780b0ae0 0x7fd37815d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:48.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.256+0000 7fd37cc44640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd378159c70 0x7fd37815da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:48.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.256+0000 7fd37cc44640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3780a7800 con 0x7fd37806db20 2023-11-09T03:47:48.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.256+0000 7fd37cc44640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37815c4c0 con 0x7fd378159c70 2023-11-09T03:47:48.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.256+0000 7fd37cc44640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37815c7a0 con 0x7fd3780b0ae0 2023-11-09T03:47:48.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.256+0000 7fd37659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd37806db20 0x7fd37814ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:48.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.256+0000 7fd37659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd37806db20 0x7fd37814ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47870/0 (socket says 172.21.15.76:47870) 2023-11-09T03:47:48.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.256+0000 7fd37659c640 1 -- 172.21.15.76:0/92770185 learned_addr learned my addr 172.21.15.76:0/92770185 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:48.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.256+0000 7fd376d9d640 1 --2- 172.21.15.76:0/92770185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd378159c70 0x7fd37815da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:48.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.257+0000 7fd375d9b640 1 --2- 172.21.15.76:0/92770185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3780b0ae0 0x7fd37815d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:48.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.257+0000 7fd37659c640 1 -- 172.21.15.76:0/92770185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3780b0ae0 msgr2=0x7fd37815d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:48.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.257+0000 7fd37659c640 1 --2- 172.21.15.76:0/92770185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3780b0ae0 0x7fd37815d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.257+0000 7fd37659c640 1 -- 172.21.15.76:0/92770185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd378159c70 msgr2=0x7fd37815da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:48.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.257+0000 7fd37659c640 1 --2- 172.21.15.76:0/92770185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd378159c70 0x7fd37815da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.257+0000 7fd37659c640 1 -- 172.21.15.76:0/92770185 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3680094d0 con 0x7fd37806db20 2023-11-09T03:47:48.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.257+0000 7fd37659c640 1 --2- 172.21.15.76:0/92770185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd37806db20 0x7fd37814ae20 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fd36c00c3c0 tx=0x7fd36c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:48.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.257+0000 7fd366ffd640 1 -- 172.21.15.76:0/92770185 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c0052d0 con 0x7fd37806db20 2023-11-09T03:47:48.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.257+0000 7fd366ffd640 1 -- 172.21.15.76:0/92770185 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd36c005450 con 0x7fd37806db20 2023-11-09T03:47:48.264 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.257+0000 7fd37cc44640 1 -- 172.21.15.76:0/92770185 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd370009a70 con 0x7fd37806db20 2023-11-09T03:47:48.264 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.257+0000 7fd366ffd640 1 -- 172.21.15.76:0/92770185 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c007260 con 0x7fd37806db20 2023-11-09T03:47:48.264 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.257+0000 7fd37cc44640 1 -- 172.21.15.76:0/92770185 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd37815cd60 con 0x7fd37806db20 2023-11-09T03:47:48.264 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.258+0000 7fd37cc44640 1 -- 172.21.15.76:0/92770185 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd378074290 con 0x7fd37806db20 2023-11-09T03:47:48.264 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.258+0000 7fd366ffd640 1 -- 172.21.15.76:0/92770185 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd36c0073c0 con 0x7fd37806db20 2023-11-09T03:47:48.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.259+0000 7fd366ffd640 1 --2- 172.21.15.76:0/92770185 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd35806ae70 0x7fd35806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:48.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.259+0000 7fd366ffd640 1 -- 172.21.15.76:0/92770185 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fd36c088990 con 0x7fd37806db20 2023-11-09T03:47:48.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.259+0000 7fd375d9b640 1 --2- 172.21.15.76:0/92770185 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd35806ae70 0x7fd35806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:48.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.260+0000 7fd375d9b640 1 --2- 172.21.15.76:0/92770185 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd35806ae70 0x7fd35806d330 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd378158190 tx=0x7fd368008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:48.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.263+0000 7fd366ffd640 1 -- 172.21.15.76:0/92770185 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd36c053a60 con 0x7fd37806db20 2023-11-09T03:47:48.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.433+0000 7fd37cc44640 1 -- 172.21.15.76:0/92770185 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd37806d3f0 con 0x7fd37806db20 2023-11-09T03:47:48.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.434+0000 7fd366ffd640 1 -- 172.21.15.76:0/92770185 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+14803 (secure 0 0 0) 0x7fd36c057080 con 0x7fd37806db20 2023-11-09T03:47:48.435 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:48.435 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":62,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:47:47.745981+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:47:48.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.436+0000 7fd37cc44640 1 -- 172.21.15.76:0/92770185 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd35806ae70 msgr2=0x7fd35806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:48.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.436+0000 7fd37cc44640 1 --2- 172.21.15.76:0/92770185 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd35806ae70 0x7fd35806d330 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd378158190 tx=0x7fd368008040 comp rx=0 tx=0).stop 2023-11-09T03:47:48.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.436+0000 7fd37cc44640 1 -- 172.21.15.76:0/92770185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd37806db20 msgr2=0x7fd37814ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:48.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.436+0000 7fd37cc44640 1 --2- 172.21.15.76:0/92770185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd37806db20 0x7fd37814ae20 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fd36c00c3c0 tx=0x7fd36c00c880 comp rx=0 tx=0).stop 2023-11-09T03:47:48.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.436+0000 7fd37cc44640 1 -- 172.21.15.76:0/92770185 shutdown_connections 2023-11-09T03:47:48.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.436+0000 7fd37cc44640 1 --2- 172.21.15.76:0/92770185 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd35806ae70 0x7fd35806d330 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.436+0000 7fd37cc44640 1 --2- 172.21.15.76:0/92770185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd378159c70 0x7fd37815da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.436+0000 7fd37cc44640 1 --2- 172.21.15.76:0/92770185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3780b0ae0 0x7fd37815d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.436+0000 7fd37cc44640 1 --2- 172.21.15.76:0/92770185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd37806db20 0x7fd37814ae20 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.436+0000 7fd37cc44640 1 -- 172.21.15.76:0/92770185 >> 172.21.15.76:0/92770185 conn(0x7fd3780ce950 msgr2=0x7fd3780b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:48.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.436+0000 7fd37cc44640 1 -- 172.21.15.76:0/92770185 shutdown_connections 2023-11-09T03:47:48.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.436+0000 7fd37cc44640 1 -- 172.21.15.76:0/92770185 wait complete. 2023-11-09T03:47:48.447 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-09T03:47:00.036540+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-09T03:47:48.448 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:47:48.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.549+0000 7fbc03925640 1 Processor -- start 2023-11-09T03:47:48.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.550+0000 7fbc03925640 1 -- start start 2023-11-09T03:47:48.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.550+0000 7fbc03925640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbbfc153e60 0x7fbbfc15dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:48.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.550+0000 7fbc03925640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbfc0b4880 0x7fbbfc0b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:48.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.550+0000 7fbc03925640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbfc0cde60 con 0x7fbbfc0cd6d0 2023-11-09T03:47:48.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.550+0000 7fbc03925640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbfc160f90 con 0x7fbbfc153e60 2023-11-09T03:47:48.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.550+0000 7fbc03925640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc0b5200 con 0x7fbbfc0b4880 2023-11-09T03:47:48.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.551+0000 7fbc016c1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbbfc153e60 0x7fbbfc15dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:48.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.551+0000 7fbc01ec2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbfc0b4880 0x7fbbfc0b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:48.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.551+0000 7fbc00ec0640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbbfc0cd6d0 0x7fbbfc161420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:50776/0 (socket says 172.21.15.76:50776) 2023-11-09T03:47:48.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.551+0000 7fbc00ec0640 1 -- 172.21.15.76:0/380875465 learned_addr learned my addr 172.21.15.76:0/380875465 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:48.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.551+0000 7fbc016c1640 1 -- 172.21.15.76:0/380875465 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbfc0b4880 msgr2=0x7fbbfc0b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:48.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.551+0000 7fbc016c1640 1 --2- 172.21.15.76:0/380875465 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbfc0b4880 0x7fbbfc0b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.551+0000 7fbc016c1640 1 -- 172.21.15.76:0/380875465 >> v1:172.21.15.113:6789/0 conn(0x7fbbfc0cd6d0 legacy=0x7fbbfc161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:48.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.551+0000 7fbc016c1640 1 -- 172.21.15.76:0/380875465 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbfc15e400 con 0x7fbbfc153e60 2023-11-09T03:47:48.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.551+0000 7fbc01ec2640 1 --2- 172.21.15.76:0/380875465 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbfc0b4880 0x7fbbfc0b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:47:48.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.552+0000 7fbc016c1640 1 --2- 172.21.15.76:0/380875465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbfc153e60 0x7fbbfc15dec0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009d90 tx=0x7fbbf802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a492534c72eeef9d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:48.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.552+0000 7fbbf3fff640 1 -- 172.21.15.76:0/380875465 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf802cc50 con 0x7fbbfc153e60 2023-11-09T03:47:48.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.552+0000 7fbbf3fff640 1 -- 172.21.15.76:0/380875465 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf802cdb0 con 0x7fbbfc153e60 2023-11-09T03:47:48.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.552+0000 7fbbf3fff640 1 -- 172.21.15.76:0/380875465 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8034660 con 0x7fbbfc153e60 2023-11-09T03:47:48.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.553+0000 7fbc03925640 1 -- 172.21.15.76:0/380875465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbfc153e60 msgr2=0x7fbbfc15dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:48.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.553+0000 7fbc03925640 1 --2- 172.21.15.76:0/380875465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbfc153e60 0x7fbbfc15dec0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009d90 tx=0x7fbbf802bea0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.553+0000 7fbc03925640 1 -- 172.21.15.76:0/380875465 shutdown_connections 2023-11-09T03:47:48.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.553+0000 7fbc03925640 1 --2- 172.21.15.76:0/380875465 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbfc0b4880 0x7fbbfc0b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.553+0000 7fbc03925640 1 --2- 172.21.15.76:0/380875465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbfc153e60 0x7fbbfc15dec0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.553+0000 7fbc03925640 1 -- 172.21.15.76:0/380875465 >> 172.21.15.76:0/380875465 conn(0x7fbbfc0cf880 msgr2=0x7fbbfc0cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:48.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.553+0000 7fbc03925640 1 -- 172.21.15.76:0/380875465 shutdown_connections 2023-11-09T03:47:48.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.553+0000 7fbc03925640 1 -- 172.21.15.76:0/380875465 wait complete. 2023-11-09T03:47:48.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.554+0000 7fbc03925640 1 Processor -- start 2023-11-09T03:47:48.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.554+0000 7fbc03925640 1 -- start start 2023-11-09T03:47:48.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.555+0000 7fbc03925640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbfc0b4880 0x7fbbfc14caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:48.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.555+0000 7fbc03925640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbfc0cd6d0 0x7fbbfc0c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:48.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.555+0000 7fbc03925640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbfc153e60 0x7fbbfc0cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:48.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.555+0000 7fbc03925640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc0cb5d0 con 0x7fbbfc153e60 2023-11-09T03:47:48.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.555+0000 7fbc03925640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc14cfe0 con 0x7fbbfc0cd6d0 2023-11-09T03:47:48.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.555+0000 7fbc03925640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc0ca950 con 0x7fbbfc0b4880 2023-11-09T03:47:48.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.555+0000 7fbc01ec2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbfc153e60 0x7fbbfc0cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:48.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.555+0000 7fbc01ec2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbfc153e60 0x7fbbfc0cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47884/0 (socket says 172.21.15.76:47884) 2023-11-09T03:47:48.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.555+0000 7fbc01ec2640 1 -- 172.21.15.76:0/3585184211 learned_addr learned my addr 172.21.15.76:0/3585184211 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:48.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.555+0000 7fbc00ec0640 1 --2- 172.21.15.76:0/3585184211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbfc0cd6d0 0x7fbbfc0c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:48.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.555+0000 7fbc016c1640 1 --2- 172.21.15.76:0/3585184211 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbfc0b4880 0x7fbbfc14caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:48.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.555+0000 7fbc01ec2640 1 -- 172.21.15.76:0/3585184211 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbfc0b4880 msgr2=0x7fbbfc14caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:48.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.555+0000 7fbc01ec2640 1 --2- 172.21.15.76:0/3585184211 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbfc0b4880 0x7fbbfc14caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.555+0000 7fbc01ec2640 1 -- 172.21.15.76:0/3585184211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbfc0cd6d0 msgr2=0x7fbbfc0c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:48.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.555+0000 7fbc01ec2640 1 --2- 172.21.15.76:0/3585184211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbfc0cd6d0 0x7fbbfc0c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.555+0000 7fbc01ec2640 1 -- 172.21.15.76:0/3585184211 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf4009400 con 0x7fbbfc153e60 2023-11-09T03:47:48.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.555+0000 7fbc00ec0640 1 --2- 172.21.15.76:0/3585184211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbfc0cd6d0 0x7fbbfc0c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:48.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.555+0000 7fbc01ec2640 1 --2- 172.21.15.76:0/3585184211 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbfc153e60 0x7fbbfc0cc270 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400e930 tx=0x7fbbf400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:48.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.555+0000 7fbbf1ffb640 1 -- 172.21.15.76:0/3585184211 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4009700 con 0x7fbbfc153e60 2023-11-09T03:47:48.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.555+0000 7fbbf1ffb640 1 -- 172.21.15.76:0/3585184211 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf400ca00 con 0x7fbbfc153e60 2023-11-09T03:47:48.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.555+0000 7fbbf1ffb640 1 -- 172.21.15.76:0/3585184211 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf400cd90 con 0x7fbbfc153e60 2023-11-09T03:47:48.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.555+0000 7fbc03925640 1 -- 172.21.15.76:0/3585184211 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf8009a70 con 0x7fbbfc153e60 2023-11-09T03:47:48.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.556+0000 7fbc03925640 1 -- 172.21.15.76:0/3585184211 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbfc0caf10 con 0x7fbbfc153e60 2023-11-09T03:47:48.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.557+0000 7fbc03925640 1 -- 172.21.15.76:0/3585184211 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbfc0cb710 con 0x7fbbfc153e60 2023-11-09T03:47:48.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.557+0000 7fbbf1ffb640 1 -- 172.21.15.76:0/3585184211 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbbf4013470 con 0x7fbbfc153e60 2023-11-09T03:47:48.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.557+0000 7fbbf1ffb640 1 --2- 172.21.15.76:0/3585184211 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbbe006aea0 0x7fbbe006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:48.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.558+0000 7fbbf1ffb640 1 -- 172.21.15.76:0/3585184211 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fbbf4089120 con 0x7fbbfc153e60 2023-11-09T03:47:48.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.558+0000 7fbc016c1640 1 --2- 172.21.15.76:0/3585184211 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbbe006aea0 0x7fbbe006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:48.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.558+0000 7fbc016c1640 1 --2- 172.21.15.76:0/3585184211 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbbe006aea0 0x7fbbe006d360 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8036000 tx=0x7fbbf8030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:48.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.562+0000 7fbbf1ffb640 1 -- 172.21.15.76:0/3585184211 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbbfc0cb710 con 0x7fbbfc153e60 2023-11-09T03:47:48.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.728+0000 7fbc03925640 1 -- 172.21.15.76:0/3585184211 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbbfc0cb710 con 0x7fbbfc153e60 2023-11-09T03:47:48.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.728+0000 7fbbf1ffb640 1 -- 172.21.15.76:0/3585184211 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbbfc0cb710 con 0x7fbbfc153e60 2023-11-09T03:47:48.729 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:47:48.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.730+0000 7fbc03925640 1 -- 172.21.15.76:0/3585184211 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbbe006aea0 msgr2=0x7fbbe006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:48.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.730+0000 7fbc03925640 1 --2- 172.21.15.76:0/3585184211 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbbe006aea0 0x7fbbe006d360 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8036000 tx=0x7fbbf8030000 comp rx=0 tx=0).stop 2023-11-09T03:47:48.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.730+0000 7fbc03925640 1 -- 172.21.15.76:0/3585184211 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbfc153e60 msgr2=0x7fbbfc0cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:48.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.730+0000 7fbc03925640 1 --2- 172.21.15.76:0/3585184211 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbfc153e60 0x7fbbfc0cc270 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400e930 tx=0x7fbbf400edf0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.730+0000 7fbc03925640 1 -- 172.21.15.76:0/3585184211 shutdown_connections 2023-11-09T03:47:48.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.730+0000 7fbc03925640 1 --2- 172.21.15.76:0/3585184211 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbbe006aea0 0x7fbbe006d360 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.730+0000 7fbc03925640 1 --2- 172.21.15.76:0/3585184211 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbbfc153e60 0x7fbbfc0cc270 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.730+0000 7fbc03925640 1 --2- 172.21.15.76:0/3585184211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbfc0cd6d0 0x7fbbfc0c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.730+0000 7fbc03925640 1 --2- 172.21.15.76:0/3585184211 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbfc0b4880 0x7fbbfc14caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.730+0000 7fbc03925640 1 -- 172.21.15.76:0/3585184211 >> 172.21.15.76:0/3585184211 conn(0x7fbbfc0cf880 msgr2=0x7fbbfc0cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:48.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.730+0000 7fbc03925640 1 -- 172.21.15.76:0/3585184211 shutdown_connections 2023-11-09T03:47:48.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.730+0000 7fbc03925640 1 -- 172.21.15.76:0/3585184211 wait complete. 2023-11-09T03:47:48.741 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-09T03:47:48.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.844+0000 7f0a884cf640 1 Processor -- start 2023-11-09T03:47:48.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.844+0000 7f0a884cf640 1 -- start start 2023-11-09T03:47:48.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.845+0000 7f0a884cf640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a8015ad40 0x7f0a8006b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:48.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.845+0000 7f0a884cf640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a80065df0 con 0x7f0a800c9b60 2023-11-09T03:47:48.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.845+0000 7f0a884cf640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a80066040 con 0x7f0a8015b120 2023-11-09T03:47:48.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.845+0000 7f0a884cf640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8006bc40 con 0x7f0a8015ad40 2023-11-09T03:47:48.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.845+0000 7f0a86a6c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0a800c9b60 0x7f0a80070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57408/0 (socket says 172.21.15.76:57408) 2023-11-09T03:47:48.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.845+0000 7f0a86a6c640 1 -- 172.21.15.76:0/2354189500 learned_addr learned my addr 172.21.15.76:0/2354189500 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:48.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.845+0000 7f0a85269640 1 -- 172.21.15.76:0/2354189500 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1554273269 0 0) 0x7f0a80065df0 con 0x7f0a800c9b60 2023-11-09T03:47:48.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.845+0000 7f0a85269640 1 -- 172.21.15.76:0/2354189500 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a6c003520 con 0x7f0a800c9b60 2023-11-09T03:47:48.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.846+0000 7f0a8626b640 1 --2- 172.21.15.76:0/2354189500 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a8015ad40 0x7f0a8006b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:48.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.846+0000 7f0a85269640 1 -- 172.21.15.76:0/2354189500 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2638732382 0 0) 0x7f0a6c003520 con 0x7f0a800c9b60 2023-11-09T03:47:48.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.846+0000 7f0a85269640 1 -- 172.21.15.76:0/2354189500 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a8015ad40 msgr2=0x7f0a8006b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:48.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.846+0000 7f0a85269640 1 --2- 172.21.15.76:0/2354189500 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a8015ad40 0x7f0a8006b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.846+0000 7f0a85269640 1 -- 172.21.15.76:0/2354189500 >> v1:172.21.15.113:6789/0 conn(0x7f0a8015b120 legacy=0x7f0a8015f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:48.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.846+0000 7f0a85269640 1 -- 172.21.15.76:0/2354189500 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a80066320 con 0x7f0a800c9b60 2023-11-09T03:47:48.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.846+0000 7f0a85269640 1 -- 172.21.15.76:0/2354189500 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f0a780019f0 con 0x7f0a800c9b60 2023-11-09T03:47:48.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.846+0000 7f0a85269640 1 -- 172.21.15.76:0/2354189500 >> v1:172.21.15.76:6789/0 conn(0x7f0a800c9b60 legacy=0x7f0a80070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:48.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.846+0000 7f0a85269640 1 --2- 172.21.15.76:0/2354189500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a6c0039a0 0x7f0a6c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:48.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.846+0000 7f0a85269640 1 --2- 172.21.15.76:0/2354189500 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a8015ad40 0x7f0a6c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:48.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.846+0000 7f0a85269640 1 --2- 172.21.15.76:0/2354189500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a6c004d00 0x7f0a6c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:48.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.846+0000 7f0a85269640 1 -- 172.21.15.76:0/2354189500 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8006bc40 con 0x7f0a6c0039a0 2023-11-09T03:47:48.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.846+0000 7f0a85269640 1 -- 172.21.15.76:0/2354189500 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c00bb10 con 0x7f0a6c004d00 2023-11-09T03:47:48.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.846+0000 7f0a85269640 1 -- 172.21.15.76:0/2354189500 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c00bca0 con 0x7f0a8015ad40 2023-11-09T03:47:48.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.846+0000 7f0a8626b640 1 --2- 172.21.15.76:0/2354189500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a6c0039a0 0x7f0a6c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:48.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.846+0000 7f0a85a6a640 1 --2- 172.21.15.76:0/2354189500 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a8015ad40 0x7f0a6c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:48.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.846+0000 7f0a86a6c640 1 --2- 172.21.15.76:0/2354189500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a6c004d00 0x7f0a6c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:48.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.846+0000 7f0a8626b640 1 -- 172.21.15.76:0/2354189500 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a8015ad40 msgr2=0x7f0a6c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:48.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.846+0000 7f0a8626b640 1 --2- 172.21.15.76:0/2354189500 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a8015ad40 0x7f0a6c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.846+0000 7f0a8626b640 1 -- 172.21.15.76:0/2354189500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a6c004d00 msgr2=0x7f0a6c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:48.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.846+0000 7f0a8626b640 1 --2- 172.21.15.76:0/2354189500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a6c004d00 0x7f0a6c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.846+0000 7f0a8626b640 1 -- 172.21.15.76:0/2354189500 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6c003520 con 0x7f0a6c0039a0 2023-11-09T03:47:48.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.846+0000 7f0a86a6c640 1 --2- 172.21.15.76:0/2354189500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a6c004d00 0x7f0a6c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:48.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.847+0000 7f0a8626b640 1 --2- 172.21.15.76:0/2354189500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a6c0039a0 0x7f0a6c003dc0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00ba60 tx=0x7f0a6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b63296588ecf5 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:48.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.847+0000 7f0a85269640 1 -- 172.21.15.76:0/2354189500 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c032630 con 0x7f0a6c0039a0 2023-11-09T03:47:48.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.847+0000 7f0a85269640 1 -- 172.21.15.76:0/2354189500 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7c032790 con 0x7f0a6c0039a0 2023-11-09T03:47:48.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.847+0000 7f0a85269640 1 -- 172.21.15.76:0/2354189500 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c032b20 con 0x7f0a6c0039a0 2023-11-09T03:47:48.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.847+0000 7f0a884cf640 1 -- 172.21.15.76:0/2354189500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a6c0039a0 msgr2=0x7f0a6c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:48.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.847+0000 7f0a884cf640 1 --2- 172.21.15.76:0/2354189500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a6c0039a0 0x7f0a6c003dc0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00ba60 tx=0x7f0a6c0057f0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.847+0000 7f0a86a6c640 1 -- 172.21.15.76:0/2354189500 reap_dead start 2023-11-09T03:47:48.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.848+0000 7f0a884cf640 1 -- 172.21.15.76:0/2354189500 shutdown_connections 2023-11-09T03:47:48.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.848+0000 7f0a884cf640 1 -- 172.21.15.76:0/2354189500 >> 172.21.15.76:0/2354189500 conn(0x7f0a800cbc90 msgr2=0x7f0a800cc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:48.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.848+0000 7f0a884cf640 1 -- 172.21.15.76:0/2354189500 shutdown_connections 2023-11-09T03:47:48.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.848+0000 7f0a884cf640 1 -- 172.21.15.76:0/2354189500 wait complete. 2023-11-09T03:47:48.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.849+0000 7f0a884cf640 1 Processor -- start 2023-11-09T03:47:48.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.849+0000 7f0a884cf640 1 -- start start 2023-11-09T03:47:48.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.849+0000 7f0a884cf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a6c0039a0 0x7f0a800cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:48.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.849+0000 7f0a884cf640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a800ce890 0x7f0a801f5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:48.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.849+0000 7f0a884cf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a80156620 0x7f0a800c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:48.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.849+0000 7f0a884cf640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a800cd820 con 0x7f0a6c0039a0 2023-11-09T03:47:48.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.850+0000 7f0a884cf640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a800ce180 con 0x7f0a80156620 2023-11-09T03:47:48.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.850+0000 7f0a884cf640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a801f60f0 con 0x7f0a800ce890 2023-11-09T03:47:48.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.850+0000 7f0a8626b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a6c0039a0 0x7f0a800cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:48.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.850+0000 7f0a8626b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a6c0039a0 0x7f0a800cdc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47904/0 (socket says 172.21.15.76:47904) 2023-11-09T03:47:48.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.850+0000 7f0a8626b640 1 -- 172.21.15.76:0/2838080778 learned_addr learned my addr 172.21.15.76:0/2838080778 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:48.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.850+0000 7f0a86a6c640 1 --2- 172.21.15.76:0/2838080778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a80156620 0x7f0a800c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:48.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.850+0000 7f0a85a6a640 1 --2- 172.21.15.76:0/2838080778 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a800ce890 0x7f0a801f5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:48.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.850+0000 7f0a8626b640 1 -- 172.21.15.76:0/2838080778 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a800ce890 msgr2=0x7f0a801f5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:48.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.850+0000 7f0a8626b640 1 --2- 172.21.15.76:0/2838080778 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a800ce890 0x7f0a801f5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.850+0000 7f0a8626b640 1 -- 172.21.15.76:0/2838080778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a80156620 msgr2=0x7f0a800c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:48.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.850+0000 7f0a8626b640 1 --2- 172.21.15.76:0/2838080778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a80156620 0x7f0a800c4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:48.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.850+0000 7f0a8626b640 1 -- 172.21.15.76:0/2838080778 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a800ced20 con 0x7f0a6c0039a0 2023-11-09T03:47:48.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.850+0000 7f0a86a6c640 1 --2- 172.21.15.76:0/2838080778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a80156620 0x7f0a800c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:48.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.850+0000 7f0a8626b640 1 --2- 172.21.15.76:0/2838080778 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a6c0039a0 0x7f0a800cdc40 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c035040 tx=0x7f0a7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:48.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.850+0000 7f0a76ffd640 1 -- 172.21.15.76:0/2838080778 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c004220 con 0x7f0a6c0039a0 2023-11-09T03:47:48.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.850+0000 7f0a76ffd640 1 -- 172.21.15.76:0/2838080778 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7c004380 con 0x7f0a6c0039a0 2023-11-09T03:47:48.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.850+0000 7f0a884cf640 1 -- 172.21.15.76:0/2838080778 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a800cce80 con 0x7f0a6c0039a0 2023-11-09T03:47:48.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.850+0000 7f0a76ffd640 1 -- 172.21.15.76:0/2838080778 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c03bab0 con 0x7f0a6c0039a0 2023-11-09T03:47:48.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.850+0000 7f0a884cf640 1 -- 172.21.15.76:0/2838080778 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a800cd440 con 0x7f0a6c0039a0 2023-11-09T03:47:48.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.851+0000 7f0a884cf640 1 -- 172.21.15.76:0/2838080778 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a80074290 con 0x7f0a6c0039a0 2023-11-09T03:47:48.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.852+0000 7f0a76ffd640 1 -- 172.21.15.76:0/2838080778 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0a7c03bc10 con 0x7f0a6c0039a0 2023-11-09T03:47:48.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.852+0000 7f0a76ffd640 1 --2- 172.21.15.76:0/2838080778 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a6406aea0 0x7f0a6406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:48.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.852+0000 7f0a76ffd640 1 -- 172.21.15.76:0/2838080778 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f0a7c0abc40 con 0x7f0a6c0039a0 2023-11-09T03:47:48.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.852+0000 7f0a85a6a640 1 --2- 172.21.15.76:0/2838080778 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a6406aea0 0x7f0a6406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:48.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.853+0000 7f0a85a6a640 1 --2- 172.21.15.76:0/2838080778 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a6406aea0 0x7f0a6406d360 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0a800ce4f0 tx=0x7f0a70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:48.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:48.856+0000 7f0a76ffd640 1 -- 172.21.15.76:0/2838080778 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0a7c076e30 con 0x7f0a6c0039a0 2023-11-09T03:47:49.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.022+0000 7f0a884cf640 1 -- 172.21.15.76:0/2838080778 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f0a80074290 con 0x7f0a6c0039a0 2023-11-09T03:47:49.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.023+0000 7f0a76ffd640 1 -- 172.21.15.76:0/2838080778 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v62) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0a7c036020 con 0x7f0a6c0039a0 2023-11-09T03:47:49.023 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:47:49.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.025+0000 7f0a884cf640 1 -- 172.21.15.76:0/2838080778 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a6406aea0 msgr2=0x7f0a6406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:49.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.025+0000 7f0a884cf640 1 --2- 172.21.15.76:0/2838080778 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a6406aea0 0x7f0a6406d360 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0a800ce4f0 tx=0x7f0a70009040 comp rx=0 tx=0).stop 2023-11-09T03:47:49.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.025+0000 7f0a884cf640 1 -- 172.21.15.76:0/2838080778 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a6c0039a0 msgr2=0x7f0a800cdc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:49.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.025+0000 7f0a884cf640 1 --2- 172.21.15.76:0/2838080778 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a6c0039a0 0x7f0a800cdc40 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c035040 tx=0x7f0a7c004730 comp rx=0 tx=0).stop 2023-11-09T03:47:49.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.025+0000 7f0a884cf640 1 -- 172.21.15.76:0/2838080778 shutdown_connections 2023-11-09T03:47:49.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.025+0000 7f0a884cf640 1 --2- 172.21.15.76:0/2838080778 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a6406aea0 0x7f0a6406d360 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.025+0000 7f0a884cf640 1 --2- 172.21.15.76:0/2838080778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a80156620 0x7f0a800c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.025+0000 7f0a884cf640 1 --2- 172.21.15.76:0/2838080778 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a800ce890 0x7f0a801f5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.025+0000 7f0a884cf640 1 --2- 172.21.15.76:0/2838080778 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a6c0039a0 0x7f0a800cdc40 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.025+0000 7f0a884cf640 1 -- 172.21.15.76:0/2838080778 >> 172.21.15.76:0/2838080778 conn(0x7f0a800cbc90 msgr2=0x7f0a8006a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:49.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.025+0000 7f0a884cf640 1 -- 172.21.15.76:0/2838080778 shutdown_connections 2023-11-09T03:47:49.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.025+0000 7f0a884cf640 1 -- 172.21.15.76:0/2838080778 wait complete. 2023-11-09T03:47:49.036 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-09T03:47:49.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.136+0000 7fc946fb6640 1 Processor -- start 2023-11-09T03:47:49.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.136+0000 7fc946fb6640 1 -- start start 2023-11-09T03:47:49.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.136+0000 7fc946fb6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9400cda30 0x7fc940165ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:49.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.136+0000 7fc946fb6640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9400cc450 con 0x7fc94015bfb0 2023-11-09T03:47:49.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.136+0000 7fc946fb6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc94015f400 con 0x7fc94015ae40 2023-11-09T03:47:49.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.136+0000 7fc946fb6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc94015b220 con 0x7fc9400cda30 2023-11-09T03:47:49.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.137+0000 7fc93ffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc94015bfb0 0x7fc9400cd320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57416/0 (socket says 172.21.15.76:57416) 2023-11-09T03:47:49.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.137+0000 7fc93ffff640 1 -- 172.21.15.76:0/2510535554 learned_addr learned my addr 172.21.15.76:0/2510535554 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:49.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.137+0000 7fc945553640 1 --2- 172.21.15.76:0/2510535554 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9400cda30 0x7fc940165ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:49.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.137+0000 7fc93f7fe640 1 -- 172.21.15.76:0/2510535554 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4217796956 0 0) 0x7fc934001110 con 0x7fc94015bfb0 2023-11-09T03:47:49.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.137+0000 7fc93f7fe640 1 -- 172.21.15.76:0/2510535554 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc928003520 con 0x7fc94015bfb0 2023-11-09T03:47:49.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.137+0000 7fc93f7fe640 1 -- 172.21.15.76:0/2510535554 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2038730606 0 0) 0x7fc94015f400 con 0x7fc94015ae40 2023-11-09T03:47:49.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.137+0000 7fc93f7fe640 1 -- 172.21.15.76:0/2510535554 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc934001110 con 0x7fc94015ae40 2023-11-09T03:47:49.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.137+0000 7fc93f7fe640 1 -- 172.21.15.76:0/2510535554 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 178836627 0 0) 0x7fc928003520 con 0x7fc94015bfb0 2023-11-09T03:47:49.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.137+0000 7fc93f7fe640 1 -- 172.21.15.76:0/2510535554 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9400cda30 msgr2=0x7fc940165ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:49.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.137+0000 7fc93f7fe640 1 --2- 172.21.15.76:0/2510535554 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9400cda30 0x7fc940165ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.137+0000 7fc93f7fe640 1 -- 172.21.15.76:0/2510535554 >> v1:172.21.15.113:6789/0 conn(0x7fc94015ae40 legacy=0x7fc9400cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:49.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.137+0000 7fc93f7fe640 1 -- 172.21.15.76:0/2510535554 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc94015f890 con 0x7fc94015bfb0 2023-11-09T03:47:49.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.137+0000 7fc93f7fe640 1 -- 172.21.15.76:0/2510535554 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fc934001800 con 0x7fc94015bfb0 2023-11-09T03:47:49.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.137+0000 7fc945553640 1 --2- 172.21.15.76:0/2510535554 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9400cda30 0x7fc940165ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:47:49.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.137+0000 7fc93f7fe640 1 -- 172.21.15.76:0/2510535554 >> v1:172.21.15.76:6789/0 conn(0x7fc94015bfb0 legacy=0x7fc9400cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:49.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.137+0000 7fc93f7fe640 1 --2- 172.21.15.76:0/2510535554 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9400cda30 0x7fc928003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:49.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.137+0000 7fc93f7fe640 1 --2- 172.21.15.76:0/2510535554 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc928004070 0x7fc9280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:49.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.137+0000 7fc93f7fe640 1 --2- 172.21.15.76:0/2510535554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc928004d00 0x7fc928005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:49.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.137+0000 7fc93f7fe640 1 -- 172.21.15.76:0/2510535554 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94015b220 con 0x7fc928004d00 2023-11-09T03:47:49.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.137+0000 7fc93f7fe640 1 -- 172.21.15.76:0/2510535554 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92800ba30 con 0x7fc928004070 2023-11-09T03:47:49.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.137+0000 7fc93f7fe640 1 -- 172.21.15.76:0/2510535554 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92800bbf0 con 0x7fc9400cda30 2023-11-09T03:47:49.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.137+0000 7fc945553640 1 --2- 172.21.15.76:0/2510535554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc928004d00 0x7fc928005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:49.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.137+0000 7fc93ffff640 1 --2- 172.21.15.76:0/2510535554 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc928004070 0x7fc9280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:49.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.137+0000 7fc944d52640 1 --2- 172.21.15.76:0/2510535554 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9400cda30 0x7fc928003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:49.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.138+0000 7fc945553640 1 -- 172.21.15.76:0/2510535554 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9400cda30 msgr2=0x7fc928003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:49.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.138+0000 7fc945553640 1 --2- 172.21.15.76:0/2510535554 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9400cda30 0x7fc928003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.138+0000 7fc945553640 1 -- 172.21.15.76:0/2510535554 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc928004070 msgr2=0x7fc9280047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:49.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.138+0000 7fc945553640 1 --2- 172.21.15.76:0/2510535554 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc928004070 0x7fc9280047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.138+0000 7fc945553640 1 -- 172.21.15.76:0/2510535554 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc930009400 con 0x7fc928004d00 2023-11-09T03:47:49.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.138+0000 7fc944d52640 1 --2- 172.21.15.76:0/2510535554 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9400cda30 0x7fc928003b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:49.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.138+0000 7fc945553640 1 --2- 172.21.15.76:0/2510535554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc928004d00 0x7fc928005180 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fc930009c70 tx=0x7fc9280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d860eb78a4866a6b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:49.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.138+0000 7fc93f7fe640 1 -- 172.21.15.76:0/2510535554 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93002c910 con 0x7fc928004d00 2023-11-09T03:47:49.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.138+0000 7fc93f7fe640 1 -- 172.21.15.76:0/2510535554 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93002ca70 con 0x7fc928004d00 2023-11-09T03:47:49.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.138+0000 7fc93f7fe640 1 -- 172.21.15.76:0/2510535554 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93002cde0 con 0x7fc928004d00 2023-11-09T03:47:49.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.139+0000 7fc946fb6640 1 -- 172.21.15.76:0/2510535554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc928004d00 msgr2=0x7fc928005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:49.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.139+0000 7fc946fb6640 1 --2- 172.21.15.76:0/2510535554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc928004d00 0x7fc928005180 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fc930009c70 tx=0x7fc9280057f0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.139+0000 7fc945553640 1 -- 172.21.15.76:0/2510535554 reap_dead start 2023-11-09T03:47:49.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.139+0000 7fc946fb6640 1 -- 172.21.15.76:0/2510535554 shutdown_connections 2023-11-09T03:47:49.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.139+0000 7fc946fb6640 1 -- 172.21.15.76:0/2510535554 >> 172.21.15.76:0/2510535554 conn(0x7fc94006ea90 msgr2=0x7fc94006ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:49.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.139+0000 7fc946fb6640 1 -- 172.21.15.76:0/2510535554 shutdown_connections 2023-11-09T03:47:49.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.139+0000 7fc946fb6640 1 -- 172.21.15.76:0/2510535554 wait complete. 2023-11-09T03:47:49.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.140+0000 7fc946fb6640 1 Processor -- start 2023-11-09T03:47:49.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.140+0000 7fc946fb6640 1 -- start start 2023-11-09T03:47:49.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.141+0000 7fc946fb6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9400c6470 0x7fc9400b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:49.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.141+0000 7fc946fb6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9400c6850 0x7fc9400b6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:49.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.141+0000 7fc946fb6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9401e66a0 0x7fc9401e6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:49.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.141+0000 7fc946fb6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9401f4ef0 con 0x7fc9400c6470 2023-11-09T03:47:49.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.141+0000 7fc946fb6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9401e7410 con 0x7fc9401e66a0 2023-11-09T03:47:49.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.141+0000 7fc946fb6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9401f5310 con 0x7fc9400c6850 2023-11-09T03:47:49.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.141+0000 7fc944d52640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9400c6470 0x7fc9400b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:49.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.141+0000 7fc944d52640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9400c6470 0x7fc9400b5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47918/0 (socket says 172.21.15.76:47918) 2023-11-09T03:47:49.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.141+0000 7fc944d52640 1 -- 172.21.15.76:0/1874793993 learned_addr learned my addr 172.21.15.76:0/1874793993 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:49.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.141+0000 7fc945553640 1 --2- 172.21.15.76:0/1874793993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9401e66a0 0x7fc9401e6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:49.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.141+0000 7fc93ffff640 1 --2- 172.21.15.76:0/1874793993 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9400c6850 0x7fc9400b6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:49.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.141+0000 7fc944d52640 1 -- 172.21.15.76:0/1874793993 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9400c6850 msgr2=0x7fc9400b6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:49.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.141+0000 7fc944d52640 1 --2- 172.21.15.76:0/1874793993 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9400c6850 0x7fc9400b6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.141+0000 7fc944d52640 1 -- 172.21.15.76:0/1874793993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9401e66a0 msgr2=0x7fc9401e6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:49.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.141+0000 7fc944d52640 1 --2- 172.21.15.76:0/1874793993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9401e66a0 0x7fc9401e6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.141+0000 7fc944d52640 1 -- 172.21.15.76:0/1874793993 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9401f4550 con 0x7fc9400c6470 2023-11-09T03:47:49.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.141+0000 7fc93ffff640 1 --2- 172.21.15.76:0/1874793993 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9400c6850 0x7fc9400b6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:49.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.142+0000 7fc944d52640 1 --2- 172.21.15.76:0/1874793993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9400c6470 0x7fc9400b5b00 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fc93800eb20 tx=0x7fc938002130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:49.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.142+0000 7fc93d7fa640 1 -- 172.21.15.76:0/1874793993 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc938017070 con 0x7fc9400c6470 2023-11-09T03:47:49.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.142+0000 7fc93d7fa640 1 -- 172.21.15.76:0/1874793993 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc938003b80 con 0x7fc9400c6470 2023-11-09T03:47:49.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.142+0000 7fc946fb6640 1 -- 172.21.15.76:0/1874793993 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9401f4830 con 0x7fc9400c6470 2023-11-09T03:47:49.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.142+0000 7fc93d7fa640 1 -- 172.21.15.76:0/1874793993 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc938003cf0 con 0x7fc9400c6470 2023-11-09T03:47:49.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.142+0000 7fc946fb6640 1 -- 172.21.15.76:0/1874793993 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9401e7090 con 0x7fc9400c6470 2023-11-09T03:47:49.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.143+0000 7fc946fb6640 1 -- 172.21.15.76:0/1874793993 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9400ce2d0 con 0x7fc9400c6470 2023-11-09T03:47:49.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.143+0000 7fc93d7fa640 1 -- 172.21.15.76:0/1874793993 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc938006b50 con 0x7fc9400c6470 2023-11-09T03:47:49.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.143+0000 7fc93d7fa640 1 --2- 172.21.15.76:0/1874793993 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc92c06aea0 0x7fc92c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:49.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.144+0000 7fc93d7fa640 1 -- 172.21.15.76:0/1874793993 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fc938089180 con 0x7fc9400c6470 2023-11-09T03:47:49.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.144+0000 7fc93ffff640 1 --2- 172.21.15.76:0/1874793993 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc92c06aea0 0x7fc92c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:49.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.144+0000 7fc93ffff640 1 --2- 172.21.15.76:0/1874793993 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc92c06aea0 0x7fc92c06d360 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc9400cfec0 tx=0x7fc934009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:49.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.148+0000 7fc93d7fa640 1 -- 172.21.15.76:0/1874793993 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc938054370 con 0x7fc9400c6470 2023-11-09T03:47:49.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.315+0000 7fc946fb6640 1 -- 172.21.15.76:0/1874793993 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc9401f5030 con 0x7fc9400c6470 2023-11-09T03:47:49.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.316+0000 7fc93d7fa640 1 -- 172.21.15.76:0/1874793993 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v62) v1 ==== 84+0+137 (secure 0 0 0) 0x7fc9401f5030 con 0x7fc9400c6470 2023-11-09T03:47:49.316 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:49.317 INFO:teuthology.orchestra.run.smithi076.stdout:[ 2023-11-09T03:47:49.317 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:49.317 INFO:teuthology.orchestra.run.smithi076.stdout: "poolnum": 1, 2023-11-09T03:47:49.317 INFO:teuthology.orchestra.run.smithi076.stdout: "poolname": ".mgr" 2023-11-09T03:47:49.317 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:49.318 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:47:49.318 INFO:teuthology.orchestra.run.smithi076.stdout: "poolnum": 7, 2023-11-09T03:47:49.318 INFO:teuthology.orchestra.run.smithi076.stdout: "poolname": "snap-test-pool" 2023-11-09T03:47:49.318 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:49.318 INFO:teuthology.orchestra.run.smithi076.stdout:] 2023-11-09T03:47:49.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.318+0000 7fc946fb6640 1 -- 172.21.15.76:0/1874793993 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc92c06aea0 msgr2=0x7fc92c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:49.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.318+0000 7fc946fb6640 1 --2- 172.21.15.76:0/1874793993 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc92c06aea0 0x7fc92c06d360 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc9400cfec0 tx=0x7fc934009040 comp rx=0 tx=0).stop 2023-11-09T03:47:49.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.318+0000 7fc946fb6640 1 -- 172.21.15.76:0/1874793993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9400c6470 msgr2=0x7fc9400b5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:49.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.318+0000 7fc946fb6640 1 --2- 172.21.15.76:0/1874793993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9400c6470 0x7fc9400b5b00 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fc93800eb20 tx=0x7fc938002130 comp rx=0 tx=0).stop 2023-11-09T03:47:49.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.318+0000 7fc946fb6640 1 -- 172.21.15.76:0/1874793993 shutdown_connections 2023-11-09T03:47:49.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.318+0000 7fc946fb6640 1 --2- 172.21.15.76:0/1874793993 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc92c06aea0 0x7fc92c06d360 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.318+0000 7fc946fb6640 1 --2- 172.21.15.76:0/1874793993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9401e66a0 0x7fc9401e6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.318+0000 7fc946fb6640 1 --2- 172.21.15.76:0/1874793993 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9400c6850 0x7fc9400b6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.318+0000 7fc946fb6640 1 --2- 172.21.15.76:0/1874793993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9400c6470 0x7fc9400b5b00 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.318+0000 7fc946fb6640 1 -- 172.21.15.76:0/1874793993 >> 172.21.15.76:0/1874793993 conn(0x7fc94006ea90 msgr2=0x7fc9400d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:49.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.318+0000 7fc946fb6640 1 -- 172.21.15.76:0/1874793993 shutdown_connections 2023-11-09T03:47:49.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.318+0000 7fc946fb6640 1 -- 172.21.15.76:0/1874793993 wait complete. 2023-11-09T03:47:49.329 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-09T03:47:49.329 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-09T03:47:49.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.432+0000 7fea8c232640 1 Processor -- start 2023-11-09T03:47:49.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.432+0000 7fea8c232640 1 -- start start 2023-11-09T03:47:49.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.433+0000 7fea8c232640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fea84159be0 0x7fea840c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:49.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.433+0000 7fea8c232640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea8415e420 0x7fea840ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:49.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.433+0000 7fea8c232640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea840c4850 con 0x7fea840adc70 2023-11-09T03:47:49.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.433+0000 7fea8c232640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea840ae100 con 0x7fea84159be0 2023-11-09T03:47:49.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.433+0000 7fea8c232640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea840c5890 con 0x7fea8415e420 2023-11-09T03:47:49.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.433+0000 7fea89fce640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fea84159be0 0x7fea840c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:49.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.433+0000 7fea89fce640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fea84159be0 0x7fea840c4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47924/0 (socket says 172.21.15.76:47924) 2023-11-09T03:47:49.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.433+0000 7fea89fce640 1 -- 172.21.15.76:0/1353962364 learned_addr learned my addr 172.21.15.76:0/1353962364 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:49.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.433+0000 7fea897cd640 1 --2- 172.21.15.76:0/1353962364 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea8415e420 0x7fea840ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:49.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.433+0000 7fea89fce640 1 -- 172.21.15.76:0/1353962364 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea8415e420 msgr2=0x7fea840ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:49.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.433+0000 7fea89fce640 1 --2- 172.21.15.76:0/1353962364 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea8415e420 0x7fea840ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.433+0000 7fea89fce640 1 -- 172.21.15.76:0/1353962364 >> v1:172.21.15.113:6789/0 conn(0x7fea840adc70 legacy=0x7fea840cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:49.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.433+0000 7fea89fce640 1 -- 172.21.15.76:0/1353962364 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea840cd8e0 con 0x7fea84159be0 2023-11-09T03:47:49.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.434+0000 7fea89fce640 1 --2- 172.21.15.76:0/1353962364 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea84159be0 0x7fea840c4f60 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fea8000bb10 tx=0x7fea8002e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f85507af4b9ef31 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:49.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.434+0000 7fea88fcc640 1 -- 172.21.15.76:0/1353962364 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8002ee70 con 0x7fea84159be0 2023-11-09T03:47:49.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.434+0000 7fea88fcc640 1 -- 172.21.15.76:0/1353962364 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea80036440 con 0x7fea84159be0 2023-11-09T03:47:49.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.434+0000 7fea88fcc640 1 -- 172.21.15.76:0/1353962364 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea800367d0 con 0x7fea84159be0 2023-11-09T03:47:49.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.435+0000 7fea8c232640 1 -- 172.21.15.76:0/1353962364 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea84159be0 msgr2=0x7fea840c4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:49.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.435+0000 7fea8c232640 1 --2- 172.21.15.76:0/1353962364 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea84159be0 0x7fea840c4f60 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fea8000bb10 tx=0x7fea8002e0c0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.435+0000 7fea8c232640 1 -- 172.21.15.76:0/1353962364 shutdown_connections 2023-11-09T03:47:49.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.435+0000 7fea8c232640 1 --2- 172.21.15.76:0/1353962364 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea8415e420 0x7fea840ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.435+0000 7fea8c232640 1 --2- 172.21.15.76:0/1353962364 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea84159be0 0x7fea840c4f60 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.435+0000 7fea8c232640 1 -- 172.21.15.76:0/1353962364 >> 172.21.15.76:0/1353962364 conn(0x7fea840b05d0 msgr2=0x7fea840b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:49.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.435+0000 7fea8c232640 1 -- 172.21.15.76:0/1353962364 shutdown_connections 2023-11-09T03:47:49.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.435+0000 7fea8c232640 1 -- 172.21.15.76:0/1353962364 wait complete. 2023-11-09T03:47:49.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.436+0000 7fea8c232640 1 Processor -- start 2023-11-09T03:47:49.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.436+0000 7fea8c232640 1 -- start start 2023-11-09T03:47:49.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.436+0000 7fea8c232640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea840adc70 0x7fea84150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:49.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.437+0000 7fea8c232640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea84159be0 0x7fea840c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:49.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.437+0000 7fea8c232640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea8415e420 0x7fea840cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:49.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.437+0000 7fea8c232640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea841f6ec0 con 0x7fea840adc70 2023-11-09T03:47:49.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.437+0000 7fea8c232640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea840c6a00 con 0x7fea8415e420 2023-11-09T03:47:49.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.437+0000 7fea8c232640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea840c6ce0 con 0x7fea84159be0 2023-11-09T03:47:49.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.437+0000 7fea89fce640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea840adc70 0x7fea84150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:49.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.437+0000 7fea89fce640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea840adc70 0x7fea84150450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47926/0 (socket says 172.21.15.76:47926) 2023-11-09T03:47:49.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.437+0000 7fea89fce640 1 -- 172.21.15.76:0/1576501329 learned_addr learned my addr 172.21.15.76:0/1576501329 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:49.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.437+0000 7fea8a7cf640 1 --2- 172.21.15.76:0/1576501329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea8415e420 0x7fea840cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:49.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.437+0000 7fea897cd640 1 --2- 172.21.15.76:0/1576501329 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea84159be0 0x7fea840c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:49.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.437+0000 7fea89fce640 1 -- 172.21.15.76:0/1576501329 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea84159be0 msgr2=0x7fea840c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:49.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.437+0000 7fea89fce640 1 --2- 172.21.15.76:0/1576501329 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea84159be0 0x7fea840c77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.437+0000 7fea89fce640 1 -- 172.21.15.76:0/1576501329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea8415e420 msgr2=0x7fea840cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:49.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.437+0000 7fea89fce640 1 --2- 172.21.15.76:0/1576501329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea8415e420 0x7fea840cc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.437+0000 7fea89fce640 1 -- 172.21.15.76:0/1576501329 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea8000b770 con 0x7fea840adc70 2023-11-09T03:47:49.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.437+0000 7fea89fce640 1 --2- 172.21.15.76:0/1576501329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea840adc70 0x7fea84150450 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fea8002e100 tx=0x7fea800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:49.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.437+0000 7fea7a7fc640 1 -- 172.21.15.76:0/1576501329 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea800348d0 con 0x7fea840adc70 2023-11-09T03:47:49.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.437+0000 7fea7a7fc640 1 -- 172.21.15.76:0/1576501329 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea80034a50 con 0x7fea840adc70 2023-11-09T03:47:49.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.437+0000 7fea8c232640 1 -- 172.21.15.76:0/1576501329 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea840c6fc0 con 0x7fea840adc70 2023-11-09T03:47:49.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.437+0000 7fea7a7fc640 1 -- 172.21.15.76:0/1576501329 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea80034e00 con 0x7fea840adc70 2023-11-09T03:47:49.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.437+0000 7fea8c232640 1 -- 172.21.15.76:0/1576501329 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea84150990 con 0x7fea840adc70 2023-11-09T03:47:49.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.438+0000 7fea8c232640 1 -- 172.21.15.76:0/1576501329 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea8404e490 con 0x7fea840adc70 2023-11-09T03:47:49.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.439+0000 7fea7a7fc640 1 -- 172.21.15.76:0/1576501329 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fea8004a3b0 con 0x7fea840adc70 2023-11-09T03:47:49.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.439+0000 7fea7a7fc640 1 --2- 172.21.15.76:0/1576501329 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fea6806ae70 0x7fea6806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:49.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.439+0000 7fea7a7fc640 1 -- 172.21.15.76:0/1576501329 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fea800aba70 con 0x7fea840adc70 2023-11-09T03:47:49.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.439+0000 7fea897cd640 1 --2- 172.21.15.76:0/1576501329 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fea6806ae70 0x7fea6806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:49.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.440+0000 7fea897cd640 1 --2- 172.21.15.76:0/1576501329 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fea6806ae70 0x7fea6806d330 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fea840c8a90 tx=0x7fea74008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:49.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.443+0000 7fea7a7fc640 1 -- 172.21.15.76:0/1576501329 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fea80076c80 con 0x7fea840adc70 2023-11-09T03:47:49.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.611+0000 7fea8c232640 1 -- 172.21.15.76:0/1576501329 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fea84070ce0 con 0x7fea840adc70 2023-11-09T03:47:49.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.759+0000 7fea7a7fc640 1 -- 172.21.15.76:0/1576501329 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v63) v1 ==== 138+0+0 (secure 0 0 0) 0x7fea800460b0 con 0x7fea840adc70 2023-11-09T03:47:49.760 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' created 2023-11-09T03:47:49.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.762+0000 7fea8c232640 1 -- 172.21.15.76:0/1576501329 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fea6806ae70 msgr2=0x7fea6806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:49.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.762+0000 7fea8c232640 1 --2- 172.21.15.76:0/1576501329 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fea6806ae70 0x7fea6806d330 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fea840c8a90 tx=0x7fea74008040 comp rx=0 tx=0).stop 2023-11-09T03:47:49.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.762+0000 7fea8c232640 1 -- 172.21.15.76:0/1576501329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea840adc70 msgr2=0x7fea84150450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:49.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.762+0000 7fea8c232640 1 --2- 172.21.15.76:0/1576501329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea840adc70 0x7fea84150450 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fea8002e100 tx=0x7fea800046c0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.762+0000 7fea8c232640 1 -- 172.21.15.76:0/1576501329 shutdown_connections 2023-11-09T03:47:49.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.762+0000 7fea8c232640 1 --2- 172.21.15.76:0/1576501329 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fea6806ae70 0x7fea6806d330 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.762+0000 7fea8c232640 1 --2- 172.21.15.76:0/1576501329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea8415e420 0x7fea840cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.762+0000 7fea8c232640 1 --2- 172.21.15.76:0/1576501329 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea84159be0 0x7fea840c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.762+0000 7fea8c232640 1 --2- 172.21.15.76:0/1576501329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea840adc70 0x7fea84150450 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.762+0000 7fea8c232640 1 -- 172.21.15.76:0/1576501329 >> 172.21.15.76:0/1576501329 conn(0x7fea840b05d0 msgr2=0x7fea840c3840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:49.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.762+0000 7fea8c232640 1 -- 172.21.15.76:0/1576501329 shutdown_connections 2023-11-09T03:47:49.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.762+0000 7fea8c232640 1 -- 172.21.15.76:0/1576501329 wait complete. 2023-11-09T03:47:49.776 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-09T03:47:49.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.882+0000 7f49086fc640 1 Processor -- start 2023-11-09T03:47:49.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.882+0000 7f49086fc640 1 -- start start 2023-11-09T03:47:49.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.883+0000 7f49086fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f49000ad730 0x7f49000adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:49.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.883+0000 7f49086fc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49000c3900 con 0x7f490015ad30 2023-11-09T03:47:49.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.883+0000 7f49086fc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49000c3be0 con 0x7f4900070e20 2023-11-09T03:47:49.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.883+0000 7f49086fc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49000ae260 con 0x7f49000ad730 2023-11-09T03:47:49.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.883+0000 7f4906498640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f490015ad30 0x7f4900070710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57424/0 (socket says 172.21.15.76:57424) 2023-11-09T03:47:49.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.883+0000 7f4906498640 1 -- 172.21.15.76:0/118613359 learned_addr learned my addr 172.21.15.76:0/118613359 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:49.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.883+0000 7f4906c99640 1 --2- 172.21.15.76:0/118613359 >> v2:172.21.15.76:3300/0 conn(0x7f49000ad730 0x7f49000adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:49.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.883+0000 7f4905496640 1 -- 172.21.15.76:0/118613359 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2147410165 0 0) 0x7f49000c3900 con 0x7f490015ad30 2023-11-09T03:47:49.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.883+0000 7f4905496640 1 -- 172.21.15.76:0/118613359 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48ec003520 con 0x7f490015ad30 2023-11-09T03:47:49.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.883+0000 7f4906c99640 1 -- 172.21.15.76:0/118613359 >> v1:172.21.15.113:6789/0 conn(0x7f4900070e20 legacy=0x7f490015e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:49.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.883+0000 7f4906c99640 1 -- 172.21.15.76:0/118613359 >> v1:172.21.15.76:6789/0 conn(0x7f490015ad30 legacy=0x7f4900070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:49.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.883+0000 7f4906c99640 1 -- 172.21.15.76:0/118613359 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49000c3ec0 con 0x7f49000ad730 2023-11-09T03:47:49.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.884+0000 7f4906c99640 1 --2- 172.21.15.76:0/118613359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49000ad730 0x7f49000adc80 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009d90 tx=0x7f48f802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d483ed42f21c41 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:49.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.884+0000 7f4905496640 1 -- 172.21.15.76:0/118613359 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f802cc60 con 0x7f49000ad730 2023-11-09T03:47:49.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.884+0000 7f4905496640 1 -- 172.21.15.76:0/118613359 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f802cdc0 con 0x7f49000ad730 2023-11-09T03:47:49.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.884+0000 7f4905496640 1 -- 172.21.15.76:0/118613359 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8034660 con 0x7f49000ad730 2023-11-09T03:47:49.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.885+0000 7f49086fc640 1 -- 172.21.15.76:0/118613359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49000ad730 msgr2=0x7f49000adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:49.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.885+0000 7f49086fc640 1 --2- 172.21.15.76:0/118613359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49000ad730 0x7f49000adc80 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009d90 tx=0x7f48f802beb0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.885+0000 7f49086fc640 1 -- 172.21.15.76:0/118613359 shutdown_connections 2023-11-09T03:47:49.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.885+0000 7f49086fc640 1 --2- 172.21.15.76:0/118613359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49000ad730 0x7f49000adc80 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.885+0000 7f49086fc640 1 -- 172.21.15.76:0/118613359 >> 172.21.15.76:0/118613359 conn(0x7f49000ce950 msgr2=0x7f49000ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:49.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.885+0000 7f49086fc640 1 -- 172.21.15.76:0/118613359 shutdown_connections 2023-11-09T03:47:49.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.885+0000 7f49086fc640 1 -- 172.21.15.76:0/118613359 wait complete. 2023-11-09T03:47:49.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.886+0000 7f49086fc640 1 Processor -- start 2023-11-09T03:47:49.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.886+0000 7f49086fc640 1 -- start start 2023-11-09T03:47:49.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.887+0000 7f49086fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4900070e20 0x7f4900145470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:49.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.887+0000 7f49086fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49000ad730 0x7f490015d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:49.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.887+0000 7f49086fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f490015ad30 0x7f490006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:49.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.887+0000 7f49086fc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49001459b0 con 0x7f49000ad730 2023-11-09T03:47:49.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.887+0000 7f49086fc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49001f1a10 con 0x7f490015ad30 2023-11-09T03:47:49.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.887+0000 7f49086fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49001f0e00 con 0x7f4900070e20 2023-11-09T03:47:49.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.887+0000 7f4905c97640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49000ad730 0x7f490015d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:49.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.887+0000 7f4905c97640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49000ad730 0x7f490015d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47946/0 (socket says 172.21.15.76:47946) 2023-11-09T03:47:49.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.887+0000 7f4905c97640 1 -- 172.21.15.76:0/1691738073 learned_addr learned my addr 172.21.15.76:0/1691738073 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:49.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.887+0000 7f4906c99640 1 --2- 172.21.15.76:0/1691738073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f490015ad30 0x7f490006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:49.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.887+0000 7f4906498640 1 --2- 172.21.15.76:0/1691738073 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4900070e20 0x7f4900145470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:49.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.887+0000 7f4905c97640 1 -- 172.21.15.76:0/1691738073 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4900070e20 msgr2=0x7f4900145470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:49.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.887+0000 7f4905c97640 1 --2- 172.21.15.76:0/1691738073 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4900070e20 0x7f4900145470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.887+0000 7f4905c97640 1 -- 172.21.15.76:0/1691738073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f490015ad30 msgr2=0x7f490006d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:49.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.887+0000 7f4905c97640 1 --2- 172.21.15.76:0/1691738073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f490015ad30 0x7f490006d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:49.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.887+0000 7f4905c97640 1 -- 172.21.15.76:0/1691738073 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f8009a70 con 0x7f49000ad730 2023-11-09T03:47:49.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.887+0000 7f4906498640 1 --2- 172.21.15.76:0/1691738073 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4900070e20 0x7f4900145470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:49.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.887+0000 7f4905c97640 1 --2- 172.21.15.76:0/1691738073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49000ad730 0x7f490015d830 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f48f000c450 tx=0x7f48f000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:49.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.887+0000 7f48f6ffd640 1 -- 172.21.15.76:0/1691738073 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f00052d0 con 0x7f49000ad730 2023-11-09T03:47:49.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.887+0000 7f48f6ffd640 1 -- 172.21.15.76:0/1691738073 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f0005430 con 0x7f49000ad730 2023-11-09T03:47:49.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.887+0000 7f48f6ffd640 1 -- 172.21.15.76:0/1691738073 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f0012870 con 0x7f49000ad730 2023-11-09T03:47:49.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.887+0000 7f49086fc640 1 -- 172.21.15.76:0/1691738073 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49001f10e0 con 0x7f49000ad730 2023-11-09T03:47:49.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.888+0000 7f49086fc640 1 -- 172.21.15.76:0/1691738073 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49001f16a0 con 0x7f49000ad730 2023-11-09T03:47:49.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.888+0000 7f49086fc640 1 -- 172.21.15.76:0/1691738073 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4900074290 con 0x7f49000ad730 2023-11-09T03:47:49.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.889+0000 7f48f6ffd640 1 -- 172.21.15.76:0/1691738073 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f48f00129d0 con 0x7f49000ad730 2023-11-09T03:47:49.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.889+0000 7f48f6ffd640 1 --2- 172.21.15.76:0/1691738073 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f48e406ae70 0x7f48e406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:49.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.889+0000 7f48f6ffd640 1 -- 172.21.15.76:0/1691738073 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5900+0+0 (secure 0 0 0) 0x7f48f00892f0 con 0x7f49000ad730 2023-11-09T03:47:49.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.890+0000 7f4906498640 1 --2- 172.21.15.76:0/1691738073 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f48e406ae70 0x7f48e406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:49.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.890+0000 7f4906498640 1 --2- 172.21.15.76:0/1691738073 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f48e406ae70 0x7f48e406d330 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0011e0 tx=0x7f48fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:49.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:49.894+0000 7f48f6ffd640 1 -- 172.21.15.76:0/1691738073 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f48f0054250 con 0x7f49000ad730 2023-11-09T03:47:50.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.062+0000 7f49086fc640 1 -- 172.21.15.76:0/1691738073 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f49001f1b50 con 0x7f49000ad730 2023-11-09T03:47:50.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.762+0000 7f48f6ffd640 1 -- 172.21.15.76:0/1691738073 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v64) v1 ==== 170+0+0 (secure 0 0 0) 0x7f48f0057870 con 0x7f49000ad730 2023-11-09T03:47:50.762 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' created 2023-11-09T03:47:50.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.764+0000 7f49086fc640 1 -- 172.21.15.76:0/1691738073 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f48e406ae70 msgr2=0x7f48e406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:50.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.764+0000 7f49086fc640 1 --2- 172.21.15.76:0/1691738073 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f48e406ae70 0x7f48e406d330 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0011e0 tx=0x7f48fc009040 comp rx=0 tx=0).stop 2023-11-09T03:47:50.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.764+0000 7f49086fc640 1 -- 172.21.15.76:0/1691738073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49000ad730 msgr2=0x7f490015d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:50.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.764+0000 7f49086fc640 1 --2- 172.21.15.76:0/1691738073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49000ad730 0x7f490015d830 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f48f000c450 tx=0x7f48f000c910 comp rx=0 tx=0).stop 2023-11-09T03:47:50.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.764+0000 7f49086fc640 1 -- 172.21.15.76:0/1691738073 shutdown_connections 2023-11-09T03:47:50.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.764+0000 7f49086fc640 1 --2- 172.21.15.76:0/1691738073 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f48e406ae70 0x7f48e406d330 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:50.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.764+0000 7f49086fc640 1 --2- 172.21.15.76:0/1691738073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f490015ad30 0x7f490006d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:50.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.764+0000 7f49086fc640 1 --2- 172.21.15.76:0/1691738073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49000ad730 0x7f490015d830 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:50.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.764+0000 7f49086fc640 1 --2- 172.21.15.76:0/1691738073 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4900070e20 0x7f4900145470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:50.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.764+0000 7f49086fc640 1 -- 172.21.15.76:0/1691738073 >> 172.21.15.76:0/1691738073 conn(0x7f49000ce950 msgr2=0x7f49000af5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:50.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.765+0000 7f49086fc640 1 -- 172.21.15.76:0/1691738073 shutdown_connections 2023-11-09T03:47:50.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.765+0000 7f49086fc640 1 -- 172.21.15.76:0/1691738073 wait complete. 2023-11-09T03:47:50.791 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-09T03:47:50.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.898+0000 7f9c49555640 1 Processor -- start 2023-11-09T03:47:50.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.898+0000 7f9c49555640 1 -- start start 2023-11-09T03:47:50.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.899+0000 7f9c49555640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9c44160430 0x7f9c440b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:50.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.899+0000 7f9c49555640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c440b2b60 0x7f9c4406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:50.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.899+0000 7f9c49555640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c440b2fc0 con 0x7f9c4406c1d0 2023-11-09T03:47:50.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.899+0000 7f9c49555640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c440cb580 con 0x7f9c44160430 2023-11-09T03:47:50.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.899+0000 7f9c49555640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4406ca40 con 0x7f9c440b2b60 2023-11-09T03:47:50.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.899+0000 7f9c437fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f9c4406c1d0 0x7f9c44160b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57428/0 (socket says 172.21.15.76:57428) 2023-11-09T03:47:50.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.899+0000 7f9c437fe640 1 -- 172.21.15.76:0/3162484604 learned_addr learned my addr 172.21.15.76:0/3162484604 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:50.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.899+0000 7f9c42ffd640 1 --2- 172.21.15.76:0/3162484604 >> v2:172.21.15.113:3300/0 conn(0x7f9c44160430 0x7f9c440b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:50.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.899+0000 7f9c427fc640 1 --2- 172.21.15.76:0/3162484604 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c440b2b60 0x7f9c4406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:50.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.899+0000 7f9c41ffb640 1 -- 172.21.15.76:0/3162484604 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3016898518 0 0) 0x7f9c440b2fc0 con 0x7f9c4406c1d0 2023-11-09T03:47:50.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.899+0000 7f9c41ffb640 1 -- 172.21.15.76:0/3162484604 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c2c003520 con 0x7f9c4406c1d0 2023-11-09T03:47:50.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.900+0000 7f9c41ffb640 1 -- 172.21.15.76:0/3162484604 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2867140670 0 0) 0x7f9c2c003520 con 0x7f9c4406c1d0 2023-11-09T03:47:50.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.900+0000 7f9c427fc640 1 -- 172.21.15.76:0/3162484604 >> v2:172.21.15.113:3300/0 conn(0x7f9c44160430 msgr2=0x7f9c440b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:50.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.900+0000 7f9c427fc640 1 --2- 172.21.15.76:0/3162484604 >> v2:172.21.15.113:3300/0 conn(0x7f9c44160430 0x7f9c440b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:50.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.900+0000 7f9c427fc640 1 -- 172.21.15.76:0/3162484604 >> v1:172.21.15.76:6789/0 conn(0x7f9c4406c1d0 legacy=0x7f9c44160b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:50.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.900+0000 7f9c427fc640 1 -- 172.21.15.76:0/3162484604 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4406cd20 con 0x7f9c440b2b60 2023-11-09T03:47:50.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.900+0000 7f9c42ffd640 1 --2- 172.21.15.76:0/3162484604 >> v2:172.21.15.113:3300/0 conn(0x7f9c44160430 0x7f9c440b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:47:50.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.900+0000 7f9c427fc640 1 --2- 172.21.15.76:0/3162484604 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c440b2b60 0x7f9c4406bc90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9c380097a0 tx=0x7f9c3802b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e33dc11649b4440 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:50.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.901+0000 7f9c41ffb640 1 -- 172.21.15.76:0/3162484604 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c001670 con 0x7f9c440b2b60 2023-11-09T03:47:50.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.901+0000 7f9c41ffb640 1 -- 172.21.15.76:0/3162484604 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3802e050 con 0x7f9c440b2b60 2023-11-09T03:47:50.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.901+0000 7f9c41ffb640 1 -- 172.21.15.76:0/3162484604 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c38030710 con 0x7f9c440b2b60 2023-11-09T03:47:50.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.901+0000 7f9c49555640 1 -- 172.21.15.76:0/3162484604 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c440b2b60 msgr2=0x7f9c4406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:50.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.901+0000 7f9c49555640 1 --2- 172.21.15.76:0/3162484604 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c440b2b60 0x7f9c4406bc90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9c380097a0 tx=0x7f9c3802b790 comp rx=0 tx=0).stop 2023-11-09T03:47:50.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.901+0000 7f9c49555640 1 -- 172.21.15.76:0/3162484604 shutdown_connections 2023-11-09T03:47:50.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.901+0000 7f9c49555640 1 --2- 172.21.15.76:0/3162484604 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c440b2b60 0x7f9c4406bc90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:50.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.901+0000 7f9c49555640 1 --2- 172.21.15.76:0/3162484604 >> v2:172.21.15.113:3300/0 conn(0x7f9c44160430 0x7f9c440b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:50.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.901+0000 7f9c49555640 1 -- 172.21.15.76:0/3162484604 >> 172.21.15.76:0/3162484604 conn(0x7f9c4406ea20 msgr2=0x7f9c4406ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:50.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.901+0000 7f9c49555640 1 -- 172.21.15.76:0/3162484604 shutdown_connections 2023-11-09T03:47:50.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.901+0000 7f9c49555640 1 -- 172.21.15.76:0/3162484604 wait complete. 2023-11-09T03:47:50.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.902+0000 7f9c49555640 1 Processor -- start 2023-11-09T03:47:50.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.903+0000 7f9c49555640 1 -- start start 2023-11-09T03:47:50.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.903+0000 7f9c49555640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c4406c1d0 0x7f9c44152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:50.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.903+0000 7f9c49555640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c440b2b60 0x7f9c440c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:50.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.903+0000 7f9c49555640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c44160430 0x7f9c440ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:50.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.903+0000 7f9c49555640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c440c9e30 con 0x7f9c4406c1d0 2023-11-09T03:47:50.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.903+0000 7f9c49555640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c440ca110 con 0x7f9c44160430 2023-11-09T03:47:50.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.903+0000 7f9c49555640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c440ca3f0 con 0x7f9c440b2b60 2023-11-09T03:47:50.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.903+0000 7f9c42ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c4406c1d0 0x7f9c44152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:50.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.903+0000 7f9c42ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c4406c1d0 0x7f9c44152270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47948/0 (socket says 172.21.15.76:47948) 2023-11-09T03:47:50.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.903+0000 7f9c42ffd640 1 -- 172.21.15.76:0/161035939 learned_addr learned my addr 172.21.15.76:0/161035939 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:50.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.903+0000 7f9c437fe640 1 --2- 172.21.15.76:0/161035939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c44160430 0x7f9c440ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:50.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.903+0000 7f9c427fc640 1 --2- 172.21.15.76:0/161035939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c440b2b60 0x7f9c440c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:50.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.904+0000 7f9c42ffd640 1 -- 172.21.15.76:0/161035939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c440b2b60 msgr2=0x7f9c440c5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:50.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.904+0000 7f9c42ffd640 1 --2- 172.21.15.76:0/161035939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c440b2b60 0x7f9c440c5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:50.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.904+0000 7f9c42ffd640 1 -- 172.21.15.76:0/161035939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c44160430 msgr2=0x7f9c440ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:50.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.904+0000 7f9c42ffd640 1 --2- 172.21.15.76:0/161035939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c44160430 0x7f9c440ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:50.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.904+0000 7f9c42ffd640 1 -- 172.21.15.76:0/161035939 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c34009a70 con 0x7f9c4406c1d0 2023-11-09T03:47:50.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.904+0000 7f9c42ffd640 1 --2- 172.21.15.76:0/161035939 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c4406c1d0 0x7f9c44152270 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400efc0 tx=0x7f9c3400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:50.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.904+0000 7f9c2b7fe640 1 -- 172.21.15.76:0/161035939 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c34014680 con 0x7f9c4406c1d0 2023-11-09T03:47:50.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.904+0000 7f9c2b7fe640 1 -- 172.21.15.76:0/161035939 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3400f040 con 0x7f9c4406c1d0 2023-11-09T03:47:50.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.904+0000 7f9c49555640 1 -- 172.21.15.76:0/161035939 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c38009400 con 0x7f9c4406c1d0 2023-11-09T03:47:50.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.904+0000 7f9c2b7fe640 1 -- 172.21.15.76:0/161035939 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c34014a00 con 0x7f9c4406c1d0 2023-11-09T03:47:50.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.904+0000 7f9c49555640 1 -- 172.21.15.76:0/161035939 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c440ca9b0 con 0x7f9c4406c1d0 2023-11-09T03:47:50.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.905+0000 7f9c49555640 1 -- 172.21.15.76:0/161035939 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c44074290 con 0x7f9c4406c1d0 2023-11-09T03:47:50.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.906+0000 7f9c2b7fe640 1 -- 172.21.15.76:0/161035939 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9c34014b60 con 0x7f9c4406c1d0 2023-11-09T03:47:50.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.906+0000 7f9c2b7fe640 1 --2- 172.21.15.76:0/161035939 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c3006ae70 0x7f9c3006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:50.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.906+0000 7f9c2b7fe640 1 -- 172.21.15.76:0/161035939 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f9c3408a6b0 con 0x7f9c4406c1d0 2023-11-09T03:47:50.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.906+0000 7f9c427fc640 1 --2- 172.21.15.76:0/161035939 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c3006ae70 0x7f9c3006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:50.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.907+0000 7f9c427fc640 1 --2- 172.21.15.76:0/161035939 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c3006ae70 0x7f9c3006d330 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9c38034000 tx=0x7f9c3802f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:50.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:50.910+0000 7f9c2b7fe640 1 -- 172.21.15.76:0/161035939 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9c340555d0 con 0x7f9c4406c1d0 2023-11-09T03:47:51.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.111+0000 7f9c49555640 1 -- 172.21.15.76:0/161035939 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9c440c9f70 con 0x7f9c4406c1d0 2023-11-09T03:47:51.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.774+0000 7f9c2b7fe640 1 -- 172.21.15.76:0/161035939 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v27) v1 ==== 174+0+0 (secure 0 0 0) 0x7f9c440c9f70 con 0x7f9c4406c1d0 2023-11-09T03:47:51.775 INFO:teuthology.orchestra.run.smithi076.stderr:new fs with metadata pool 10 and data pool 11 2023-11-09T03:47:51.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.776+0000 7f9c49555640 1 -- 172.21.15.76:0/161035939 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c3006ae70 msgr2=0x7f9c3006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:51.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.776+0000 7f9c49555640 1 --2- 172.21.15.76:0/161035939 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c3006ae70 0x7f9c3006d330 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9c38034000 tx=0x7f9c3802f000 comp rx=0 tx=0).stop 2023-11-09T03:47:51.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.776+0000 7f9c49555640 1 -- 172.21.15.76:0/161035939 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c4406c1d0 msgr2=0x7f9c44152270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:51.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.776+0000 7f9c49555640 1 --2- 172.21.15.76:0/161035939 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c4406c1d0 0x7f9c44152270 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400efc0 tx=0x7f9c3400c490 comp rx=0 tx=0).stop 2023-11-09T03:47:51.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.776+0000 7f9c49555640 1 -- 172.21.15.76:0/161035939 shutdown_connections 2023-11-09T03:47:51.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.776+0000 7f9c49555640 1 --2- 172.21.15.76:0/161035939 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c3006ae70 0x7f9c3006d330 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:51.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.776+0000 7f9c49555640 1 --2- 172.21.15.76:0/161035939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c44160430 0x7f9c440ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:51.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.776+0000 7f9c49555640 1 --2- 172.21.15.76:0/161035939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c440b2b60 0x7f9c440c5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:51.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.776+0000 7f9c49555640 1 --2- 172.21.15.76:0/161035939 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c4406c1d0 0x7f9c44152270 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:51.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.776+0000 7f9c49555640 1 -- 172.21.15.76:0/161035939 >> 172.21.15.76:0/161035939 conn(0x7f9c4406ea20 msgr2=0x7f9c440b0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:51.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.777+0000 7f9c49555640 1 -- 172.21.15.76:0/161035939 shutdown_connections 2023-11-09T03:47:51.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.777+0000 7f9c49555640 1 -- 172.21.15.76:0/161035939 wait complete. 2023-11-09T03:47:51.792 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:47:51.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.892+0000 7faa47061640 1 Processor -- start 2023-11-09T03:47:51.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.892+0000 7faa47061640 1 -- start start 2023-11-09T03:47:51.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.893+0000 7faa47061640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa4006a1b0 0x7faa4006b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:51.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.893+0000 7faa47061640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa4006cf60 con 0x7faa400c9a40 2023-11-09T03:47:51.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.893+0000 7faa47061640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa4015b160 con 0x7faa4015ad80 2023-11-09T03:47:51.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.893+0000 7faa47061640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa4006b9b0 con 0x7faa4006a1b0 2023-11-09T03:47:51.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.893+0000 7faa455fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7faa400c9a40 0x7faa4006dce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57436/0 (socket says 172.21.15.76:57436) 2023-11-09T03:47:51.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.893+0000 7faa455fe640 1 -- 172.21.15.76:0/755369805 learned_addr learned my addr 172.21.15.76:0/755369805 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:51.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.893+0000 7faa3ffff640 1 --2- 172.21.15.76:0/755369805 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa4006a1b0 0x7faa4006b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:51.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.893+0000 7faa3f7fe640 1 -- 172.21.15.76:0/755369805 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1818270915 0 0) 0x7faa4006cf60 con 0x7faa400c9a40 2023-11-09T03:47:51.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.893+0000 7faa3f7fe640 1 -- 172.21.15.76:0/755369805 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa28003520 con 0x7faa400c9a40 2023-11-09T03:47:51.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.894+0000 7faa3f7fe640 1 -- 172.21.15.76:0/755369805 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3075369321 0 0) 0x7faa28003520 con 0x7faa400c9a40 2023-11-09T03:47:51.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.894+0000 7faa3f7fe640 1 -- 172.21.15.76:0/755369805 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa4006a1b0 msgr2=0x7faa4006b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:51.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.894+0000 7faa3f7fe640 1 --2- 172.21.15.76:0/755369805 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa4006a1b0 0x7faa4006b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:51.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.894+0000 7faa3f7fe640 1 -- 172.21.15.76:0/755369805 >> v1:172.21.15.113:6789/0 conn(0x7faa4015ad80 legacy=0x7faa400abe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:51.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.894+0000 7faa3f7fe640 1 -- 172.21.15.76:0/755369805 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa4006d200 con 0x7faa400c9a40 2023-11-09T03:47:51.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.894+0000 7faa3f7fe640 1 -- 172.21.15.76:0/755369805 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7faa380019f0 con 0x7faa400c9a40 2023-11-09T03:47:51.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.894+0000 7faa3f7fe640 1 -- 172.21.15.76:0/755369805 >> v1:172.21.15.76:6789/0 conn(0x7faa400c9a40 legacy=0x7faa4006dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:51.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.894+0000 7faa3f7fe640 1 --2- 172.21.15.76:0/755369805 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa280039a0 0x7faa28003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:51.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.894+0000 7faa3f7fe640 1 --2- 172.21.15.76:0/755369805 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa28004350 0x7faa28004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:51.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.894+0000 7faa3f7fe640 1 --2- 172.21.15.76:0/755369805 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa4006a1b0 0x7faa28005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:51.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.894+0000 7faa3f7fe640 1 -- 172.21.15.76:0/755369805 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4006b9b0 con 0x7faa28004350 2023-11-09T03:47:51.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.894+0000 7faa3f7fe640 1 -- 172.21.15.76:0/755369805 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2800b980 con 0x7faa280039a0 2023-11-09T03:47:51.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.894+0000 7faa3f7fe640 1 -- 172.21.15.76:0/755369805 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2800bb40 con 0x7faa4006a1b0 2023-11-09T03:47:51.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.894+0000 7faa3ffff640 1 --2- 172.21.15.76:0/755369805 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa28004350 0x7faa28004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:51.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.894+0000 7faa44dfd640 1 --2- 172.21.15.76:0/755369805 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa280039a0 0x7faa28003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:51.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.894+0000 7faa455fe640 1 --2- 172.21.15.76:0/755369805 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa4006a1b0 0x7faa28005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:51.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.894+0000 7faa3ffff640 1 -- 172.21.15.76:0/755369805 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa4006a1b0 msgr2=0x7faa28005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:51.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.894+0000 7faa3ffff640 1 --2- 172.21.15.76:0/755369805 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa4006a1b0 0x7faa28005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:51.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.894+0000 7faa3ffff640 1 -- 172.21.15.76:0/755369805 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa280039a0 msgr2=0x7faa28003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:51.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.894+0000 7faa3ffff640 1 --2- 172.21.15.76:0/755369805 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa280039a0 0x7faa28003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:51.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.894+0000 7faa3ffff640 1 -- 172.21.15.76:0/755369805 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa28003520 con 0x7faa28004350 2023-11-09T03:47:51.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.895+0000 7faa3ffff640 1 --2- 172.21.15.76:0/755369805 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa28004350 0x7faa28004820 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7faa34009a90 tx=0x7faa280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4764bab52f2db7 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:51.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.895+0000 7faa3f7fe640 1 -- 172.21.15.76:0/755369805 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3402c680 con 0x7faa28004350 2023-11-09T03:47:51.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.895+0000 7faa3f7fe640 1 -- 172.21.15.76:0/755369805 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3402c7e0 con 0x7faa28004350 2023-11-09T03:47:51.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.895+0000 7faa3f7fe640 1 -- 172.21.15.76:0/755369805 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3402cb70 con 0x7faa28004350 2023-11-09T03:47:51.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.896+0000 7faa47061640 1 -- 172.21.15.76:0/755369805 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa28004350 msgr2=0x7faa28004820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:51.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.896+0000 7faa47061640 1 --2- 172.21.15.76:0/755369805 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa28004350 0x7faa28004820 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7faa34009a90 tx=0x7faa280057f0 comp rx=0 tx=0).stop 2023-11-09T03:47:51.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.896+0000 7faa455fe640 1 -- 172.21.15.76:0/755369805 reap_dead start 2023-11-09T03:47:51.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.896+0000 7faa47061640 1 -- 172.21.15.76:0/755369805 shutdown_connections 2023-11-09T03:47:51.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.896+0000 7faa47061640 1 -- 172.21.15.76:0/755369805 >> 172.21.15.76:0/755369805 conn(0x7faa400b0cd0 msgr2=0x7faa400b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:51.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.896+0000 7faa47061640 1 -- 172.21.15.76:0/755369805 shutdown_connections 2023-11-09T03:47:51.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.896+0000 7faa47061640 1 -- 172.21.15.76:0/755369805 wait complete. 2023-11-09T03:47:51.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.897+0000 7faa47061640 1 Processor -- start 2023-11-09T03:47:51.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.897+0000 7faa47061640 1 -- start start 2023-11-09T03:47:51.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.898+0000 7faa47061640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa28004350 0x7faa4014fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:51.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.898+0000 7faa47061640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa401599b0 0x7faa40159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:51.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.898+0000 7faa47061640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa40154300 0x7faa40154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:51.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.898+0000 7faa47061640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa40154cc0 con 0x7faa28004350 2023-11-09T03:47:51.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.898+0000 7faa47061640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa401f4d20 con 0x7faa40154300 2023-11-09T03:47:51.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.898+0000 7faa47061640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa401539f0 con 0x7faa401599b0 2023-11-09T03:47:51.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.898+0000 7faa44dfd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa28004350 0x7faa4014fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:51.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.898+0000 7faa44dfd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa28004350 0x7faa4014fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47978/0 (socket says 172.21.15.76:47978) 2023-11-09T03:47:51.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.898+0000 7faa44dfd640 1 -- 172.21.15.76:0/662380343 learned_addr learned my addr 172.21.15.76:0/662380343 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:51.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.898+0000 7faa3ffff640 1 --2- 172.21.15.76:0/662380343 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa401599b0 0x7faa40159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:51.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.898+0000 7faa455fe640 1 --2- 172.21.15.76:0/662380343 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa40154300 0x7faa40154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:51.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.898+0000 7faa44dfd640 1 -- 172.21.15.76:0/662380343 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa401599b0 msgr2=0x7faa40159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:51.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.898+0000 7faa44dfd640 1 --2- 172.21.15.76:0/662380343 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa401599b0 0x7faa40159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:51.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.898+0000 7faa44dfd640 1 -- 172.21.15.76:0/662380343 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa40154300 msgr2=0x7faa40154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:51.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.898+0000 7faa44dfd640 1 --2- 172.21.15.76:0/662380343 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa40154300 0x7faa40154780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:51.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.898+0000 7faa44dfd640 1 -- 172.21.15.76:0/662380343 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa4015a520 con 0x7faa28004350 2023-11-09T03:47:51.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.898+0000 7faa455fe640 1 --2- 172.21.15.76:0/662380343 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa40154300 0x7faa40154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:51.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.898+0000 7faa3ffff640 1 --2- 172.21.15.76:0/662380343 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa401599b0 0x7faa40159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:51.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.898+0000 7faa44dfd640 1 --2- 172.21.15.76:0/662380343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa28004350 0x7faa4014fda0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7faa3000f8b0 tx=0x7faa3000fd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:51.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.898+0000 7faa3d7fa640 1 -- 172.21.15.76:0/662380343 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa30001550 con 0x7faa28004350 2023-11-09T03:47:51.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.898+0000 7faa3d7fa640 1 -- 172.21.15.76:0/662380343 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa300052e0 con 0x7faa28004350 2023-11-09T03:47:51.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.898+0000 7faa47061640 1 -- 172.21.15.76:0/662380343 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa401f3f60 con 0x7faa28004350 2023-11-09T03:47:51.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.898+0000 7faa3d7fa640 1 -- 172.21.15.76:0/662380343 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3000d970 con 0x7faa28004350 2023-11-09T03:47:51.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.898+0000 7faa47061640 1 -- 172.21.15.76:0/662380343 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa401f4520 con 0x7faa28004350 2023-11-09T03:47:51.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.899+0000 7faa47061640 1 -- 172.21.15.76:0/662380343 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa40074290 con 0x7faa28004350 2023-11-09T03:47:51.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.900+0000 7faa3d7fa640 1 -- 172.21.15.76:0/662380343 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faa3000dad0 con 0x7faa28004350 2023-11-09T03:47:51.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.900+0000 7faa3d7fa640 1 --2- 172.21.15.76:0/662380343 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faa2c06ae70 0x7faa2c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:51.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.900+0000 7faa3d7fa640 1 -- 172.21.15.76:0/662380343 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6382+0+0 (secure 0 0 0) 0x7faa30089950 con 0x7faa28004350 2023-11-09T03:47:51.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.901+0000 7faa3ffff640 1 --2- 172.21.15.76:0/662380343 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faa2c06ae70 0x7faa2c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:51.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.901+0000 7faa3ffff640 1 --2- 172.21.15.76:0/662380343 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faa2c06ae70 0x7faa2c06d330 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7faa34009a90 tx=0x7faa34030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:51.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:51.905+0000 7faa3d7fa640 1 -- 172.21.15.76:0/662380343 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faa300547f0 con 0x7faa28004350 2023-11-09T03:47:52.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.073+0000 7faa47061640 1 -- 172.21.15.76:0/662380343 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa40066320 con 0x7faa28004350 2023-11-09T03:47:52.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.074+0000 7faa3d7fa640 1 -- 172.21.15.76:0/662380343 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18527 (secure 0 0 0) 0x7faa30057e10 con 0x7faa28004350 2023-11-09T03:47:52.075 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:52.075 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":65,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:47:51.113829+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:47:49.630346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-09T03:47:50.082207+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:47:52.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.076+0000 7faa47061640 1 -- 172.21.15.76:0/662380343 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faa2c06ae70 msgr2=0x7faa2c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:52.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.076+0000 7faa47061640 1 --2- 172.21.15.76:0/662380343 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faa2c06ae70 0x7faa2c06d330 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7faa34009a90 tx=0x7faa34030000 comp rx=0 tx=0).stop 2023-11-09T03:47:52.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.076+0000 7faa47061640 1 -- 172.21.15.76:0/662380343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa28004350 msgr2=0x7faa4014fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:52.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.076+0000 7faa47061640 1 --2- 172.21.15.76:0/662380343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa28004350 0x7faa4014fda0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7faa3000f8b0 tx=0x7faa3000fd70 comp rx=0 tx=0).stop 2023-11-09T03:47:52.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.076+0000 7faa47061640 1 -- 172.21.15.76:0/662380343 shutdown_connections 2023-11-09T03:47:52.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.076+0000 7faa47061640 1 --2- 172.21.15.76:0/662380343 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faa2c06ae70 0x7faa2c06d330 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.076+0000 7faa47061640 1 --2- 172.21.15.76:0/662380343 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa40154300 0x7faa40154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.076+0000 7faa47061640 1 --2- 172.21.15.76:0/662380343 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa401599b0 0x7faa40159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.076+0000 7faa47061640 1 --2- 172.21.15.76:0/662380343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa28004350 0x7faa4014fda0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.076+0000 7faa47061640 1 -- 172.21.15.76:0/662380343 >> 172.21.15.76:0/662380343 conn(0x7faa400b0cd0 msgr2=0x7faa400b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:52.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.076+0000 7faa47061640 1 -- 172.21.15.76:0/662380343 shutdown_connections 2023-11-09T03:47:52.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.076+0000 7faa47061640 1 -- 172.21.15.76:0/662380343 wait complete. 2023-11-09T03:47:52.087 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:47:52.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.187+0000 7ff58bf25640 1 Processor -- start 2023-11-09T03:47:52.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.187+0000 7ff58bf25640 1 -- start start 2023-11-09T03:47:52.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.188+0000 7ff58bf25640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff5840ade20 0x7ff58406c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:52.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.188+0000 7ff58bf25640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff5840cce10 0x7ff5840cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:52.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.188+0000 7ff58bf25640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5840cd7a0 con 0x7ff5840c1e80 2023-11-09T03:47:52.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.188+0000 7ff58bf25640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5841617c0 con 0x7ff5840cce10 2023-11-09T03:47:52.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.188+0000 7ff58bf25640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5840ae200 con 0x7ff5840ade20 2023-11-09T03:47:52.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.188+0000 7ff58a4c2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff5840cce10 0x7ff5840cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:52.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.188+0000 7ff58a4c2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff5840cce10 0x7ff5840cd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47984/0 (socket says 172.21.15.76:47984) 2023-11-09T03:47:52.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.188+0000 7ff58a4c2640 1 -- 172.21.15.76:0/24178974 learned_addr learned my addr 172.21.15.76:0/24178974 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:52.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.188+0000 7ff589cc1640 1 --2- 172.21.15.76:0/24178974 >> v2:172.21.15.113:3300/0 conn(0x7ff5840ade20 0x7ff58406c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:52.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.189+0000 7ff58a4c2640 1 -- 172.21.15.76:0/24178974 >> v2:172.21.15.113:3300/0 conn(0x7ff5840ade20 msgr2=0x7ff58406c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:52.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.189+0000 7ff58a4c2640 1 --2- 172.21.15.76:0/24178974 >> v2:172.21.15.113:3300/0 conn(0x7ff5840ade20 0x7ff58406c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.189+0000 7ff58a4c2640 1 -- 172.21.15.76:0/24178974 >> v1:172.21.15.113:6789/0 conn(0x7ff5840c1e80 legacy=0x7ff5840c92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:52.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.189+0000 7ff58a4c2640 1 -- 172.21.15.76:0/24178974 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5840c9d10 con 0x7ff5840cce10 2023-11-09T03:47:52.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.189+0000 7ff58a4c2640 1 --2- 172.21.15.76:0/24178974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5840cce10 0x7ff5840cd260 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff5840cc440 tx=0x7ff58002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e32b3b8cca157c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:52.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.189+0000 7ff588cbf640 1 -- 172.21.15.76:0/24178974 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58002cb20 con 0x7ff5840cce10 2023-11-09T03:47:52.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.189+0000 7ff588cbf640 1 -- 172.21.15.76:0/24178974 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58002cc80 con 0x7ff5840cce10 2023-11-09T03:47:52.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.189+0000 7ff588cbf640 1 -- 172.21.15.76:0/24178974 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5800343f0 con 0x7ff5840cce10 2023-11-09T03:47:52.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.190+0000 7ff58bf25640 1 -- 172.21.15.76:0/24178974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5840cce10 msgr2=0x7ff5840cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:52.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.190+0000 7ff58bf25640 1 --2- 172.21.15.76:0/24178974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5840cce10 0x7ff5840cd260 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff5840cc440 tx=0x7ff58002bf80 comp rx=0 tx=0).stop 2023-11-09T03:47:52.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.190+0000 7ff58bf25640 1 -- 172.21.15.76:0/24178974 shutdown_connections 2023-11-09T03:47:52.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.190+0000 7ff58bf25640 1 --2- 172.21.15.76:0/24178974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5840cce10 0x7ff5840cd260 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.190+0000 7ff58bf25640 1 --2- 172.21.15.76:0/24178974 >> v2:172.21.15.113:3300/0 conn(0x7ff5840ade20 0x7ff58406c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.190+0000 7ff58bf25640 1 -- 172.21.15.76:0/24178974 >> 172.21.15.76:0/24178974 conn(0x7ff5840ce950 msgr2=0x7ff5840ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:52.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.190+0000 7ff58bf25640 1 -- 172.21.15.76:0/24178974 shutdown_connections 2023-11-09T03:47:52.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.190+0000 7ff58bf25640 1 -- 172.21.15.76:0/24178974 wait complete. 2023-11-09T03:47:52.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.191+0000 7ff58bf25640 1 Processor -- start 2023-11-09T03:47:52.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.191+0000 7ff58bf25640 1 -- start start 2023-11-09T03:47:52.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.192+0000 7ff58bf25640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5840ade20 0x7ff584157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:52.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.192+0000 7ff58bf25640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5840c1e80 0x7ff584161900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:52.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.192+0000 7ff58bf25640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5840cce10 0x7ff58415f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:52.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.192+0000 7ff58bf25640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff584162940 con 0x7ff5840cce10 2023-11-09T03:47:52.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.192+0000 7ff58bf25640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff584157e60 con 0x7ff5840c1e80 2023-11-09T03:47:52.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.192+0000 7ff58bf25640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff584162380 con 0x7ff5840ade20 2023-11-09T03:47:52.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.192+0000 7ff58a4c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5840cce10 0x7ff58415f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:52.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.192+0000 7ff58a4c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5840cce10 0x7ff58415f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47990/0 (socket says 172.21.15.76:47990) 2023-11-09T03:47:52.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.192+0000 7ff58a4c2640 1 -- 172.21.15.76:0/3616383971 learned_addr learned my addr 172.21.15.76:0/3616383971 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:52.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.192+0000 7ff5894c0640 1 --2- 172.21.15.76:0/3616383971 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5840c1e80 0x7ff584161900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:52.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.192+0000 7ff589cc1640 1 --2- 172.21.15.76:0/3616383971 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5840ade20 0x7ff584157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:52.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.192+0000 7ff58a4c2640 1 -- 172.21.15.76:0/3616383971 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5840ade20 msgr2=0x7ff584157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:52.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.192+0000 7ff58a4c2640 1 --2- 172.21.15.76:0/3616383971 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5840ade20 0x7ff584157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.192+0000 7ff58a4c2640 1 -- 172.21.15.76:0/3616383971 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5840c1e80 msgr2=0x7ff584161900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:52.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.192+0000 7ff58a4c2640 1 --2- 172.21.15.76:0/3616383971 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5840c1e80 0x7ff584161900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.192+0000 7ff58a4c2640 1 -- 172.21.15.76:0/3616383971 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff580009b30 con 0x7ff5840cce10 2023-11-09T03:47:52.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.192+0000 7ff589cc1640 1 --2- 172.21.15.76:0/3616383971 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5840ade20 0x7ff584157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:52.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.193+0000 7ff58a4c2640 1 --2- 172.21.15.76:0/3616383971 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5840cce10 0x7ff58415f510 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7ff580009f40 tx=0x7ff580004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:52.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.193+0000 7ff57a7fc640 1 -- 172.21.15.76:0/3616383971 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff580009510 con 0x7ff5840cce10 2023-11-09T03:47:52.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.193+0000 7ff57a7fc640 1 -- 172.21.15.76:0/3616383971 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58002c920 con 0x7ff5840cce10 2023-11-09T03:47:52.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.193+0000 7ff58bf25640 1 -- 172.21.15.76:0/3616383971 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff584162660 con 0x7ff5840cce10 2023-11-09T03:47:52.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.193+0000 7ff57a7fc640 1 -- 172.21.15.76:0/3616383971 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58003e690 con 0x7ff5840cce10 2023-11-09T03:47:52.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.193+0000 7ff58bf25640 1 -- 172.21.15.76:0/3616383971 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff584157540 con 0x7ff5840cce10 2023-11-09T03:47:52.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.194+0000 7ff58bf25640 1 -- 172.21.15.76:0/3616383971 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff584157ff0 con 0x7ff5840cce10 2023-11-09T03:47:52.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.194+0000 7ff57a7fc640 1 -- 172.21.15.76:0/3616383971 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff58003e7f0 con 0x7ff5840cce10 2023-11-09T03:47:52.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.195+0000 7ff57a7fc640 1 --2- 172.21.15.76:0/3616383971 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff56806ae70 0x7ff56806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:52.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.195+0000 7ff57a7fc640 1 -- 172.21.15.76:0/3616383971 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff5800ac330 con 0x7ff5840cce10 2023-11-09T03:47:52.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.195+0000 7ff589cc1640 1 --2- 172.21.15.76:0/3616383971 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff56806ae70 0x7ff56806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:52.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.195+0000 7ff589cc1640 1 --2- 172.21.15.76:0/3616383971 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff56806ae70 0x7ff56806d330 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009ac0 tx=0x7ff57c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:52.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.199+0000 7ff57a7fc640 1 -- 172.21.15.76:0/3616383971 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff5800771d0 con 0x7ff5840cce10 2023-11-09T03:47:52.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.367+0000 7ff58bf25640 1 -- 172.21.15.76:0/3616383971 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff584074290 con 0x7ff5840cce10 2023-11-09T03:47:52.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.368+0000 7ff57a7fc640 1 -- 172.21.15.76:0/3616383971 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18527 (secure 0 0 0) 0x7ff58007a7f0 con 0x7ff5840cce10 2023-11-09T03:47:52.369 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:52.369 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":65,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:47:51.113829+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:47:49.630346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-09T03:47:50.082207+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:47:52.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.370+0000 7ff58bf25640 1 -- 172.21.15.76:0/3616383971 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff56806ae70 msgr2=0x7ff56806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:52.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.370+0000 7ff58bf25640 1 --2- 172.21.15.76:0/3616383971 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff56806ae70 0x7ff56806d330 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009ac0 tx=0x7ff57c008040 comp rx=0 tx=0).stop 2023-11-09T03:47:52.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.370+0000 7ff58bf25640 1 -- 172.21.15.76:0/3616383971 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5840cce10 msgr2=0x7ff58415f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:52.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.370+0000 7ff58bf25640 1 --2- 172.21.15.76:0/3616383971 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5840cce10 0x7ff58415f510 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7ff580009f40 tx=0x7ff580004a40 comp rx=0 tx=0).stop 2023-11-09T03:47:52.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.371+0000 7ff58bf25640 1 -- 172.21.15.76:0/3616383971 shutdown_connections 2023-11-09T03:47:52.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.371+0000 7ff58bf25640 1 --2- 172.21.15.76:0/3616383971 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff56806ae70 0x7ff56806d330 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.371+0000 7ff58bf25640 1 --2- 172.21.15.76:0/3616383971 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5840cce10 0x7ff58415f510 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.371+0000 7ff58bf25640 1 --2- 172.21.15.76:0/3616383971 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5840c1e80 0x7ff584161900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.371+0000 7ff58bf25640 1 --2- 172.21.15.76:0/3616383971 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5840ade20 0x7ff584157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.371+0000 7ff58bf25640 1 -- 172.21.15.76:0/3616383971 >> 172.21.15.76:0/3616383971 conn(0x7ff5840ce950 msgr2=0x7ff584160f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:52.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.371+0000 7ff58bf25640 1 -- 172.21.15.76:0/3616383971 shutdown_connections 2023-11-09T03:47:52.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.371+0000 7ff58bf25640 1 -- 172.21.15.76:0/3616383971 wait complete. 2023-11-09T03:47:52.382 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-09T03:47:52.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.485+0000 7f6508900640 1 Processor -- start 2023-11-09T03:47:52.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.485+0000 7f6508900640 1 -- start start 2023-11-09T03:47:52.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.486+0000 7f6508900640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f65040c4ff0 0x7f65040cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:52.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.486+0000 7f6508900640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f65040adac0 0x7f65040adf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:52.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.486+0000 7f6508900640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f650415aff0 con 0x7f6504159c00 2023-11-09T03:47:52.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.486+0000 7f6508900640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65040ae460 con 0x7f65040c4ff0 2023-11-09T03:47:52.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.486+0000 7f6508900640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f650415a3e0 con 0x7f65040adac0 2023-11-09T03:47:52.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.486+0000 7f64fbfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f65040c4ff0 0x7f65040cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:52.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.486+0000 7f650259c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f6504159c00 0x7f65040c48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57448/0 (socket says 172.21.15.76:57448) 2023-11-09T03:47:52.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.486+0000 7f650259c640 1 -- 172.21.15.76:0/1645539265 learned_addr learned my addr 172.21.15.76:0/1645539265 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:52.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.486+0000 7f6502d9d640 1 --2- 172.21.15.76:0/1645539265 >> v2:172.21.15.113:3300/0 conn(0x7f65040adac0 0x7f65040adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:52.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.486+0000 7f64fbfff640 1 -- 172.21.15.76:0/1645539265 >> v2:172.21.15.113:3300/0 conn(0x7f65040adac0 msgr2=0x7f65040adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:52.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.486+0000 7f64fbfff640 1 --2- 172.21.15.76:0/1645539265 >> v2:172.21.15.113:3300/0 conn(0x7f65040adac0 0x7f65040adf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.486+0000 7f64fbfff640 1 -- 172.21.15.76:0/1645539265 >> v1:172.21.15.76:6789/0 conn(0x7f6504159c00 legacy=0x7f65040c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:52.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.486+0000 7f64fbfff640 1 -- 172.21.15.76:0/1645539265 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f650415a6c0 con 0x7f65040c4ff0 2023-11-09T03:47:52.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.487+0000 7f64fbfff640 1 --2- 172.21.15.76:0/1645539265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65040c4ff0 0x7f65040cc890 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f65040c59a0 tx=0x7f64ec02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3dabd2d8c361dc server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:52.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.487+0000 7f6501d9b640 1 -- 172.21.15.76:0/1645539265 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec031830 con 0x7f65040c4ff0 2023-11-09T03:47:52.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.487+0000 7f6501d9b640 1 -- 172.21.15.76:0/1645539265 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64ec031990 con 0x7f65040c4ff0 2023-11-09T03:47:52.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.487+0000 7f6501d9b640 1 -- 172.21.15.76:0/1645539265 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec031d20 con 0x7f65040c4ff0 2023-11-09T03:47:52.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.488+0000 7f6508900640 1 -- 172.21.15.76:0/1645539265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65040c4ff0 msgr2=0x7f65040cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:52.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.488+0000 7f6508900640 1 --2- 172.21.15.76:0/1645539265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65040c4ff0 0x7f65040cc890 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f65040c59a0 tx=0x7f64ec02b8d0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.488+0000 7f6508900640 1 -- 172.21.15.76:0/1645539265 shutdown_connections 2023-11-09T03:47:52.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.488+0000 7f6508900640 1 --2- 172.21.15.76:0/1645539265 >> v2:172.21.15.113:3300/0 conn(0x7f65040adac0 0x7f65040adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.488+0000 7f6508900640 1 --2- 172.21.15.76:0/1645539265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65040c4ff0 0x7f65040cc890 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.488+0000 7f6508900640 1 -- 172.21.15.76:0/1645539265 >> 172.21.15.76:0/1645539265 conn(0x7f65040ce950 msgr2=0x7f65040ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:52.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.488+0000 7f6508900640 1 -- 172.21.15.76:0/1645539265 shutdown_connections 2023-11-09T03:47:52.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.488+0000 7f6508900640 1 -- 172.21.15.76:0/1645539265 wait complete. 2023-11-09T03:47:52.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.489+0000 7f6508900640 1 Processor -- start 2023-11-09T03:47:52.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.489+0000 7f6508900640 1 -- start start 2023-11-09T03:47:52.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.490+0000 7f6508900640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65040adac0 0x7f650415bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:52.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.490+0000 7f6508900640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65040c4ff0 0x7f650414d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:52.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.490+0000 7f6508900640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6504159c00 0x7f650414d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:52.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.490+0000 7f6508900640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6504158340 con 0x7f65040adac0 2023-11-09T03:47:52.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.490+0000 7f6508900640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6504157730 con 0x7f65040c4ff0 2023-11-09T03:47:52.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.490+0000 7f6508900640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6504157a10 con 0x7f6504159c00 2023-11-09T03:47:52.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.490+0000 7f650259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65040adac0 0x7f650415bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:52.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.490+0000 7f650259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65040adac0 0x7f650415bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48006/0 (socket says 172.21.15.76:48006) 2023-11-09T03:47:52.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.490+0000 7f650259c640 1 -- 172.21.15.76:0/1736369635 learned_addr learned my addr 172.21.15.76:0/1736369635 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:52.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.490+0000 7f64fbfff640 1 --2- 172.21.15.76:0/1736369635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65040c4ff0 0x7f650414d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:52.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.490+0000 7f6502d9d640 1 --2- 172.21.15.76:0/1736369635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6504159c00 0x7f650414d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:52.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.490+0000 7f650259c640 1 -- 172.21.15.76:0/1736369635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6504159c00 msgr2=0x7f650414d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:52.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.490+0000 7f650259c640 1 --2- 172.21.15.76:0/1736369635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6504159c00 0x7f650414d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.490+0000 7f650259c640 1 -- 172.21.15.76:0/1736369635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65040c4ff0 msgr2=0x7f650414d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:52.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.490+0000 7f650259c640 1 --2- 172.21.15.76:0/1736369635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65040c4ff0 0x7f650414d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.490+0000 7f650259c640 1 -- 172.21.15.76:0/1736369635 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ec009400 con 0x7f65040adac0 2023-11-09T03:47:52.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.490+0000 7f6502d9d640 1 --2- 172.21.15.76:0/1736369635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6504159c00 0x7f650414d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:52.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.490+0000 7f650259c640 1 --2- 172.21.15.76:0/1736369635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65040adac0 0x7f650415bd50 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f64f400c480 tx=0x7f64f400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:52.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.490+0000 7f64faffd640 1 -- 172.21.15.76:0/1736369635 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f4005ce0 con 0x7f65040adac0 2023-11-09T03:47:52.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.490+0000 7f64faffd640 1 -- 172.21.15.76:0/1736369635 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f4005e40 con 0x7f65040adac0 2023-11-09T03:47:52.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.490+0000 7f6508900640 1 -- 172.21.15.76:0/1736369635 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6504157cf0 con 0x7f65040adac0 2023-11-09T03:47:52.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.490+0000 7f64faffd640 1 -- 172.21.15.76:0/1736369635 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f40072f0 con 0x7f65040adac0 2023-11-09T03:47:52.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.491+0000 7f6508900640 1 -- 172.21.15.76:0/1736369635 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f650415c290 con 0x7f65040adac0 2023-11-09T03:47:52.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.492+0000 7f6508900640 1 -- 172.21.15.76:0/1736369635 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f650415c480 con 0x7f65040adac0 2023-11-09T03:47:52.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.492+0000 7f64faffd640 1 -- 172.21.15.76:0/1736369635 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f64f4007510 con 0x7f65040adac0 2023-11-09T03:47:52.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.492+0000 7f64faffd640 1 --2- 172.21.15.76:0/1736369635 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f64e406ae40 0x7f64e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:52.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.492+0000 7f64faffd640 1 -- 172.21.15.76:0/1736369635 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f64f4088ce0 con 0x7f65040adac0 2023-11-09T03:47:52.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.493+0000 7f64fbfff640 1 --2- 172.21.15.76:0/1736369635 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f64e406ae40 0x7f64e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:52.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.493+0000 7f64fbfff640 1 --2- 172.21.15.76:0/1736369635 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f64e406ae40 0x7f64e406d300 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f64ec036000 tx=0x7f64ec03c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:52.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.497+0000 7f64faffd640 1 -- 172.21.15.76:0/1736369635 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f64f4053c30 con 0x7f65040adac0 2023-11-09T03:47:52.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.699+0000 7f6508900640 1 -- 172.21.15.76:0/1736369635 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6504074290 con 0x7f65040adac0 2023-11-09T03:47:52.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.781+0000 7f64faffd640 1 -- 172.21.15.76:0/1736369635 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v28) v1 ==== 118+0+0 (secure 0 0 0) 0x7f64f4017020 con 0x7f65040adac0 2023-11-09T03:47:52.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.784+0000 7f6508900640 1 -- 172.21.15.76:0/1736369635 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f64e406ae40 msgr2=0x7f64e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:52.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.784+0000 7f6508900640 1 --2- 172.21.15.76:0/1736369635 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f64e406ae40 0x7f64e406d300 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f64ec036000 tx=0x7f64ec03c040 comp rx=0 tx=0).stop 2023-11-09T03:47:52.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.784+0000 7f6508900640 1 -- 172.21.15.76:0/1736369635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65040adac0 msgr2=0x7f650415bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:52.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.784+0000 7f6508900640 1 --2- 172.21.15.76:0/1736369635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65040adac0 0x7f650415bd50 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f64f400c480 tx=0x7f64f400c940 comp rx=0 tx=0).stop 2023-11-09T03:47:52.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.784+0000 7f6508900640 1 -- 172.21.15.76:0/1736369635 shutdown_connections 2023-11-09T03:47:52.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.784+0000 7f6508900640 1 --2- 172.21.15.76:0/1736369635 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f64e406ae40 0x7f64e406d300 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.784+0000 7f6508900640 1 --2- 172.21.15.76:0/1736369635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6504159c00 0x7f650414d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.784+0000 7f6508900640 1 --2- 172.21.15.76:0/1736369635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65040c4ff0 0x7f650414d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.784+0000 7f6508900640 1 --2- 172.21.15.76:0/1736369635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65040adac0 0x7f650415bd50 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.784+0000 7f6508900640 1 -- 172.21.15.76:0/1736369635 >> 172.21.15.76:0/1736369635 conn(0x7f65040ce950 msgr2=0x7f65040c7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:52.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.784+0000 7f6508900640 1 -- 172.21.15.76:0/1736369635 shutdown_connections 2023-11-09T03:47:52.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.784+0000 7f6508900640 1 -- 172.21.15.76:0/1736369635 wait complete. 2023-11-09T03:47:52.796 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-09T03:47:52.796 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-09T03:47:52.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.898+0000 7f2257056640 1 Processor -- start 2023-11-09T03:47:52.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.898+0000 7f2257056640 1 -- start start 2023-11-09T03:47:52.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.899+0000 7f2257056640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2250153850 0x7f225015a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:52.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.899+0000 7f2257056640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22500cc9d0 0x7f22500c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:52.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.899+0000 7f2257056640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22500c3860 con 0x7f22500cc520 2023-11-09T03:47:52.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.899+0000 7f2257056640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22500c9250 con 0x7f2250153850 2023-11-09T03:47:52.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.899+0000 7f2257056640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2250153c30 con 0x7f22500cc9d0 2023-11-09T03:47:52.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.899+0000 7f224ffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f22500cc520 0x7f2250070fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57948/0 (socket says 172.21.15.76:57948) 2023-11-09T03:47:52.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.899+0000 7f224ffff640 1 -- 172.21.15.76:0/1276659421 learned_addr learned my addr 172.21.15.76:0/1276659421 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:52.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.899+0000 7f22555f3640 1 --2- 172.21.15.76:0/1276659421 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22500cc9d0 0x7f22500c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:52.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.899+0000 7f2254df2640 1 --2- 172.21.15.76:0/1276659421 >> v2:172.21.15.113:3300/0 conn(0x7f2250153850 0x7f225015a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:52.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.900+0000 7f224f7fe640 1 -- 172.21.15.76:0/1276659421 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1898939126 0 0) 0x7f22500c3860 con 0x7f22500cc520 2023-11-09T03:47:52.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.900+0000 7f224f7fe640 1 -- 172.21.15.76:0/1276659421 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2238005730 con 0x7f22500cc520 2023-11-09T03:47:52.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.900+0000 7f224f7fe640 1 -- 172.21.15.76:0/1276659421 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4063115630 0 0) 0x7f2238005730 con 0x7f22500cc520 2023-11-09T03:47:52.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.900+0000 7f224f7fe640 1 -- 172.21.15.76:0/1276659421 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22500cc9d0 msgr2=0x7f22500c3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:52.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.900+0000 7f224f7fe640 1 --2- 172.21.15.76:0/1276659421 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22500cc9d0 0x7f22500c3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.900+0000 7f224f7fe640 1 -- 172.21.15.76:0/1276659421 >> v2:172.21.15.113:3300/0 conn(0x7f2250153850 msgr2=0x7f225015a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:52.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.900+0000 7f224f7fe640 1 --2- 172.21.15.76:0/1276659421 >> v2:172.21.15.113:3300/0 conn(0x7f2250153850 0x7f225015a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.900+0000 7f224f7fe640 1 -- 172.21.15.76:0/1276659421 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f225015a700 con 0x7f22500cc520 2023-11-09T03:47:52.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.900+0000 7f224f7fe640 1 -- 172.21.15.76:0/1276659421 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f2244001670 con 0x7f22500cc520 2023-11-09T03:47:52.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.900+0000 7f224f7fe640 1 -- 172.21.15.76:0/1276659421 >> v1:172.21.15.76:6789/0 conn(0x7f22500cc520 legacy=0x7f2250070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:52.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.900+0000 7f224f7fe640 1 --2- 172.21.15.76:0/1276659421 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22500cc9d0 0x7f2238005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:52.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.900+0000 7f224f7fe640 1 --2- 172.21.15.76:0/1276659421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2238006100 0x7f22380069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:52.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.900+0000 7f224f7fe640 1 --2- 172.21.15.76:0/1276659421 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2238006f10 0x7f2238007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:52.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.900+0000 7f224f7fe640 1 -- 172.21.15.76:0/1276659421 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22500c9250 con 0x7f2238006f10 2023-11-09T03:47:52.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.900+0000 7f224f7fe640 1 -- 172.21.15.76:0/1276659421 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2250153c30 con 0x7f2238006100 2023-11-09T03:47:52.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.900+0000 7f224f7fe640 1 -- 172.21.15.76:0/1276659421 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f223800db90 con 0x7f22500cc9d0 2023-11-09T03:47:52.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.900+0000 7f22555f3640 1 --2- 172.21.15.76:0/1276659421 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2238006f10 0x7f2238007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:52.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.900+0000 7f2254df2640 1 --2- 172.21.15.76:0/1276659421 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22500cc9d0 0x7f2238005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:52.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.900+0000 7f224ffff640 1 --2- 172.21.15.76:0/1276659421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2238006100 0x7f22380069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:52.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.900+0000 7f22555f3640 1 -- 172.21.15.76:0/1276659421 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22500cc9d0 msgr2=0x7f2238005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:52.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.900+0000 7f22555f3640 1 --2- 172.21.15.76:0/1276659421 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22500cc9d0 0x7f2238005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.900+0000 7f22555f3640 1 -- 172.21.15.76:0/1276659421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2238006100 msgr2=0x7f22380069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:52.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.900+0000 7f22555f3640 1 --2- 172.21.15.76:0/1276659421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2238006100 0x7f22380069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.900+0000 7f22555f3640 1 -- 172.21.15.76:0/1276659421 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2238005730 con 0x7f2238006f10 2023-11-09T03:47:52.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.900+0000 7f224ffff640 1 --2- 172.21.15.76:0/1276659421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2238006100 0x7f22380069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:52.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.901+0000 7f22555f3640 1 --2- 172.21.15.76:0/1276659421 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2238006f10 0x7f2238007390 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f2248008330 tx=0x7f2238007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529f3e875f0c1a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:52.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.901+0000 7f224f7fe640 1 -- 172.21.15.76:0/1276659421 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224800f040 con 0x7f2238006f10 2023-11-09T03:47:52.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.901+0000 7f224f7fe640 1 -- 172.21.15.76:0/1276659421 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2248030070 con 0x7f2238006f10 2023-11-09T03:47:52.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.901+0000 7f224f7fe640 1 -- 172.21.15.76:0/1276659421 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224802cdf0 con 0x7f2238006f10 2023-11-09T03:47:52.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.902+0000 7f2257056640 1 -- 172.21.15.76:0/1276659421 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2238006f10 msgr2=0x7f2238007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:52.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.902+0000 7f2257056640 1 --2- 172.21.15.76:0/1276659421 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2238006f10 0x7f2238007390 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f2248008330 tx=0x7f2238007a00 comp rx=0 tx=0).stop 2023-11-09T03:47:52.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.902+0000 7f22555f3640 1 -- 172.21.15.76:0/1276659421 reap_dead start 2023-11-09T03:47:52.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.902+0000 7f2257056640 1 -- 172.21.15.76:0/1276659421 shutdown_connections 2023-11-09T03:47:52.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.902+0000 7f2257056640 1 -- 172.21.15.76:0/1276659421 >> 172.21.15.76:0/1276659421 conn(0x7f22500ce950 msgr2=0x7f22500ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:52.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.902+0000 7f2257056640 1 -- 172.21.15.76:0/1276659421 shutdown_connections 2023-11-09T03:47:52.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.902+0000 7f2257056640 1 -- 172.21.15.76:0/1276659421 wait complete. 2023-11-09T03:47:52.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.903+0000 7f2257056640 1 Processor -- start 2023-11-09T03:47:52.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.903+0000 7f2257056640 1 -- start start 2023-11-09T03:47:52.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.904+0000 7f2257056640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f225014e600 0x7f225014fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:52.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.904+0000 7f2257056640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2250150170 0x7f22501ee720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:52.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.904+0000 7f2257056640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22501eec60 0x7f22500c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:52.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.904+0000 7f2257056640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22500c5a20 con 0x7f2250150170 2023-11-09T03:47:52.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.904+0000 7f2257056640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22500c5fe0 con 0x7f225014e600 2023-11-09T03:47:52.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.904+0000 7f2257056640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22501ef210 con 0x7f22501eec60 2023-11-09T03:47:52.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.904+0000 7f224ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2250150170 0x7f22501ee720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:52.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.904+0000 7f224ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2250150170 0x7f22501ee720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41074/0 (socket says 172.21.15.76:41074) 2023-11-09T03:47:52.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.904+0000 7f224ffff640 1 -- 172.21.15.76:0/1249784690 learned_addr learned my addr 172.21.15.76:0/1249784690 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:52.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.904+0000 7f2254df2640 1 --2- 172.21.15.76:0/1249784690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f225014e600 0x7f225014fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:52.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.904+0000 7f224ffff640 1 -- 172.21.15.76:0/1249784690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22501eec60 msgr2=0x7f22500c73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:52.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.904+0000 7f224ffff640 1 --2- 172.21.15.76:0/1249784690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22501eec60 0x7f22500c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.904+0000 7f224ffff640 1 -- 172.21.15.76:0/1249784690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f225014e600 msgr2=0x7f225014fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:52.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.904+0000 7f224ffff640 1 --2- 172.21.15.76:0/1249784690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f225014e600 0x7f225014fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:52.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.904+0000 7f224ffff640 1 -- 172.21.15.76:0/1249784690 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22501ef4f0 con 0x7f2250150170 2023-11-09T03:47:52.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.904+0000 7f224ffff640 1 --2- 172.21.15.76:0/1249784690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2250150170 0x7f22501ee720 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f2244000ff0 tx=0x7f224400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:52.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.904+0000 7f224d7fa640 1 -- 172.21.15.76:0/1249784690 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224400c820 con 0x7f2250150170 2023-11-09T03:47:52.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.904+0000 7f224d7fa640 1 -- 172.21.15.76:0/1249784690 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f224400c980 con 0x7f2250150170 2023-11-09T03:47:52.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.904+0000 7f2257056640 1 -- 172.21.15.76:0/1249784690 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22500c5c70 con 0x7f2250150170 2023-11-09T03:47:52.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.904+0000 7f224d7fa640 1 -- 172.21.15.76:0/1249784690 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224400ccf0 con 0x7f2250150170 2023-11-09T03:47:52.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.904+0000 7f2257056640 1 -- 172.21.15.76:0/1249784690 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2250158fd0 con 0x7f2250150170 2023-11-09T03:47:52.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.905+0000 7f2257056640 1 -- 172.21.15.76:0/1249784690 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22500c6150 con 0x7f2250150170 2023-11-09T03:47:52.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.906+0000 7f224d7fa640 1 -- 172.21.15.76:0/1249784690 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f224401d460 con 0x7f2250150170 2023-11-09T03:47:52.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.906+0000 7f224d7fa640 1 --2- 172.21.15.76:0/1249784690 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f223c06ae70 0x7f223c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:52.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.906+0000 7f224d7fa640 1 -- 172.21.15.76:0/1249784690 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2244089830 con 0x7f2250150170 2023-11-09T03:47:52.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.906+0000 7f2254df2640 1 --2- 172.21.15.76:0/1249784690 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f223c06ae70 0x7f223c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:52.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.907+0000 7f2254df2640 1 --2- 172.21.15.76:0/1249784690 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f223c06ae70 0x7f223c06d330 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2240009a60 tx=0x7f2240008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:52.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:52.911+0000 7f224d7fa640 1 -- 172.21.15.76:0/1249784690 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f22440546d0 con 0x7f2250150170 2023-11-09T03:47:53.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.112+0000 7f2257056640 1 -- 172.21.15.76:0/1249784690 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2250074290 con 0x7f2250150170 2023-11-09T03:47:53.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.795+0000 7f224d7fa640 1 -- 172.21.15.76:0/1249784690 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v29) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2244057cf0 con 0x7f2250150170 2023-11-09T03:47:53.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.798+0000 7f2257056640 1 -- 172.21.15.76:0/1249784690 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f223c06ae70 msgr2=0x7f223c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:53.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.798+0000 7f2257056640 1 --2- 172.21.15.76:0/1249784690 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f223c06ae70 0x7f223c06d330 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2240009a60 tx=0x7f2240008040 comp rx=0 tx=0).stop 2023-11-09T03:47:53.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.798+0000 7f2257056640 1 -- 172.21.15.76:0/1249784690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2250150170 msgr2=0x7f22501ee720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:53.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.798+0000 7f2257056640 1 --2- 172.21.15.76:0/1249784690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2250150170 0x7f22501ee720 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f2244000ff0 tx=0x7f224400ed50 comp rx=0 tx=0).stop 2023-11-09T03:47:53.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.798+0000 7f2257056640 1 -- 172.21.15.76:0/1249784690 shutdown_connections 2023-11-09T03:47:53.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.798+0000 7f2257056640 1 --2- 172.21.15.76:0/1249784690 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f223c06ae70 0x7f223c06d330 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:53.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.798+0000 7f2257056640 1 --2- 172.21.15.76:0/1249784690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22501eec60 0x7f22500c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:53.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.798+0000 7f2257056640 1 --2- 172.21.15.76:0/1249784690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2250150170 0x7f22501ee720 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:53.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.798+0000 7f2257056640 1 --2- 172.21.15.76:0/1249784690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f225014e600 0x7f225014fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:53.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.798+0000 7f2257056640 1 -- 172.21.15.76:0/1249784690 >> 172.21.15.76:0/1249784690 conn(0x7f22500ce950 msgr2=0x7f22500cd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:53.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.798+0000 7f2257056640 1 -- 172.21.15.76:0/1249784690 shutdown_connections 2023-11-09T03:47:53.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.798+0000 7f2257056640 1 -- 172.21.15.76:0/1249784690 wait complete. 2023-11-09T03:47:53.809 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-09T03:47:53.810 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:47:53.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.913+0000 7fadd2487640 1 Processor -- start 2023-11-09T03:47:53.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.913+0000 7fadd2487640 1 -- start start 2023-11-09T03:47:53.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.914+0000 7fadd2487640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fadcc15ad80 0x7fadcc06c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:53.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.914+0000 7fadd2487640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadcc06cd00 con 0x7fadcc0b42c0 2023-11-09T03:47:53.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.914+0000 7fadd2487640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadcc06cfa0 con 0x7fadcc0c9e10 2023-11-09T03:47:53.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.914+0000 7fadd2487640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadcc06d280 con 0x7fadcc15ad80 2023-11-09T03:47:53.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.914+0000 7fadcbfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fadcc15ad80 0x7fadcc06c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:53.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.914+0000 7fadcb7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fadcc0b42c0 0x7fadcc0c9700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57960/0 (socket says 172.21.15.76:57960) 2023-11-09T03:47:53.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.914+0000 7fadcb7fe640 1 -- 172.21.15.76:0/3183778516 learned_addr learned my addr 172.21.15.76:0/3183778516 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:53.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.914+0000 7fadcbfff640 1 -- 172.21.15.76:0/3183778516 >> v1:172.21.15.113:6789/0 conn(0x7fadcc0c9e10 legacy=0x7fadcc0b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:53.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.914+0000 7fadcbfff640 1 -- 172.21.15.76:0/3183778516 >> v1:172.21.15.76:6789/0 conn(0x7fadcc0b42c0 legacy=0x7fadcc0c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:53.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.914+0000 7fadcaffd640 1 -- 172.21.15.76:0/3183778516 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1428583359 0 0) 0x7fadcc06cd00 con 0x7fadcc0b42c0 2023-11-09T03:47:53.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.914+0000 7fadcbfff640 1 -- 172.21.15.76:0/3183778516 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadcc06d560 con 0x7fadcc15ad80 2023-11-09T03:47:53.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.915+0000 7fadcbfff640 1 --2- 172.21.15.76:0/3183778516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadcc15ad80 0x7fadcc06c7c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0097a0 tx=0x7fadbc02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520261b4ecb1380a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:53.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.915+0000 7fadcaffd640 1 -- 172.21.15.76:0/3183778516 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc030610 con 0x7fadcc15ad80 2023-11-09T03:47:53.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.915+0000 7fadcaffd640 1 -- 172.21.15.76:0/3183778516 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadbc030770 con 0x7fadcc15ad80 2023-11-09T03:47:53.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.915+0000 7fadcaffd640 1 -- 172.21.15.76:0/3183778516 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc030b00 con 0x7fadcc15ad80 2023-11-09T03:47:53.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.916+0000 7fadd2487640 1 -- 172.21.15.76:0/3183778516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadcc15ad80 msgr2=0x7fadcc06c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:53.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.916+0000 7fadd2487640 1 --2- 172.21.15.76:0/3183778516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadcc15ad80 0x7fadcc06c7c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0097a0 tx=0x7fadbc02b960 comp rx=0 tx=0).stop 2023-11-09T03:47:53.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.916+0000 7fadd2487640 1 -- 172.21.15.76:0/3183778516 shutdown_connections 2023-11-09T03:47:53.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.916+0000 7fadd2487640 1 --2- 172.21.15.76:0/3183778516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadcc15ad80 0x7fadcc06c7c0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:53.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.916+0000 7fadd2487640 1 -- 172.21.15.76:0/3183778516 >> 172.21.15.76:0/3183778516 conn(0x7fadcc0ce950 msgr2=0x7fadcc0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:53.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.916+0000 7fadd2487640 1 -- 172.21.15.76:0/3183778516 shutdown_connections 2023-11-09T03:47:53.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.916+0000 7fadd2487640 1 -- 172.21.15.76:0/3183778516 wait complete. 2023-11-09T03:47:53.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.917+0000 7fadd2487640 1 Processor -- start 2023-11-09T03:47:53.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.917+0000 7fadd2487640 1 -- start start 2023-11-09T03:47:53.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.918+0000 7fadd2487640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadcc0b42c0 0x7fadcc1541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:53.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.918+0000 7fadd2487640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadcc0c9e10 0x7fadcc1f2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:53.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.918+0000 7fadd2487640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadcc15ad80 0x7fadcc15cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:53.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.918+0000 7fadd2487640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc15e2a0 con 0x7fadcc0c9e10 2023-11-09T03:47:53.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.918+0000 7fadd2487640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc15da00 con 0x7fadcc15ad80 2023-11-09T03:47:53.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.918+0000 7fadd2487640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc15dce0 con 0x7fadcc0b42c0 2023-11-09T03:47:53.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.918+0000 7fadcb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadcc0c9e10 0x7fadcc1f2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:53.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.918+0000 7fadcb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadcc0c9e10 0x7fadcc1f2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41084/0 (socket says 172.21.15.76:41084) 2023-11-09T03:47:53.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.918+0000 7fadcb7fe640 1 -- 172.21.15.76:0/1768992948 learned_addr learned my addr 172.21.15.76:0/1768992948 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:53.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.918+0000 7fadcb7fe640 1 -- 172.21.15.76:0/1768992948 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadcc0b42c0 msgr2=0x7fadcc1541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:53.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.918+0000 7fadcb7fe640 1 --2- 172.21.15.76:0/1768992948 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadcc0b42c0 0x7fadcc1541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:53.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.918+0000 7fadcb7fe640 1 -- 172.21.15.76:0/1768992948 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadcc15ad80 msgr2=0x7fadcc15cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:53.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.918+0000 7fadcb7fe640 1 --2- 172.21.15.76:0/1768992948 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadcc15ad80 0x7fadcc15cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:53.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.918+0000 7fadcb7fe640 1 -- 172.21.15.76:0/1768992948 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadbc009400 con 0x7fadcc0c9e10 2023-11-09T03:47:53.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.918+0000 7fadcb7fe640 1 --2- 172.21.15.76:0/1768992948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadcc0c9e10 0x7fadcc1f2570 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fadc000c4d0 tx=0x7fadc000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:53.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.918+0000 7fadc8ff9640 1 -- 172.21.15.76:0/1768992948 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc00074e0 con 0x7fadcc0c9e10 2023-11-09T03:47:53.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.918+0000 7fadc8ff9640 1 -- 172.21.15.76:0/1768992948 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc0007660 con 0x7fadcc0c9e10 2023-11-09T03:47:53.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.918+0000 7fadd2487640 1 -- 172.21.15.76:0/1768992948 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadcc15dfc0 con 0x7fadcc0c9e10 2023-11-09T03:47:53.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.918+0000 7fadc8ff9640 1 -- 172.21.15.76:0/1768992948 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0004a20 con 0x7fadcc0c9e10 2023-11-09T03:47:53.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.919+0000 7fadd2487640 1 -- 172.21.15.76:0/1768992948 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadcc1f2ab0 con 0x7fadcc0c9e10 2023-11-09T03:47:53.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.919+0000 7fadd2487640 1 -- 172.21.15.76:0/1768992948 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadcc0ccb60 con 0x7fadcc0c9e10 2023-11-09T03:47:53.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.920+0000 7fadc8ff9640 1 -- 172.21.15.76:0/1768992948 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fadc0004b80 con 0x7fadcc0c9e10 2023-11-09T03:47:53.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.920+0000 7fadc8ff9640 1 --2- 172.21.15.76:0/1768992948 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fadb806ae70 0x7fadb806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:53.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.920+0000 7fadc8ff9640 1 -- 172.21.15.76:0/1768992948 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fadc0088d50 con 0x7fadcc0c9e10 2023-11-09T03:47:53.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.921+0000 7fadcbfff640 1 --2- 172.21.15.76:0/1768992948 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fadb806ae70 0x7fadb806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:53.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.921+0000 7fadcbfff640 1 --2- 172.21.15.76:0/1768992948 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fadb806ae70 0x7fadb806d330 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fadbc036000 tx=0x7fadbc030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:53.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:53.924+0000 7fadc8ff9640 1 -- 172.21.15.76:0/1768992948 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fadcc0ccb60 con 0x7fadcc0c9e10 2023-11-09T03:47:54.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.124+0000 7fadd2487640 1 -- 172.21.15.76:0/1768992948 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fadcc0ccb60 con 0x7fadcc0c9e10 2023-11-09T03:47:54.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.125+0000 7fadc8ff9640 1 -- 172.21.15.76:0/1768992948 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fadcc0ccb60 con 0x7fadcc0c9e10 2023-11-09T03:47:54.124 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:54.124 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2800733670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2800733670},{"type":"v1","addr":"172.21.15.113:6836","nonce":2800733670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":5791,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1301905851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1301905851},{"type":"v1","addr":"172.21.15.113:6837","nonce":1301905851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":6498,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/968739949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":968739949},{"type":"v1","addr":"172.21.15.76:6833","nonce":968739949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:47:51.114132+0000","modified":"2023-11-09T03:47:53.114512+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6501},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6501":{"gid":6501,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/1576689316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1576689316},{"type":"v1","addr":"172.21.15.76:6835","nonce":1576689316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-09T03:47:54.125 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 29 2023-11-09T03:47:54.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.127+0000 7fadd2487640 1 -- 172.21.15.76:0/1768992948 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fadb806ae70 msgr2=0x7fadb806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:54.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.127+0000 7fadd2487640 1 --2- 172.21.15.76:0/1768992948 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fadb806ae70 0x7fadb806d330 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fadbc036000 tx=0x7fadbc030000 comp rx=0 tx=0).stop 2023-11-09T03:47:54.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.127+0000 7fadd2487640 1 -- 172.21.15.76:0/1768992948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadcc0c9e10 msgr2=0x7fadcc1f2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:54.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.127+0000 7fadd2487640 1 --2- 172.21.15.76:0/1768992948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadcc0c9e10 0x7fadcc1f2570 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fadc000c4d0 tx=0x7fadc000c990 comp rx=0 tx=0).stop 2023-11-09T03:47:54.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.127+0000 7fadd2487640 1 -- 172.21.15.76:0/1768992948 shutdown_connections 2023-11-09T03:47:54.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.127+0000 7fadd2487640 1 --2- 172.21.15.76:0/1768992948 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fadb806ae70 0x7fadb806d330 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.127+0000 7fadd2487640 1 --2- 172.21.15.76:0/1768992948 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadcc15ad80 0x7fadcc15cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.127+0000 7fadd2487640 1 --2- 172.21.15.76:0/1768992948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadcc0c9e10 0x7fadcc1f2570 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.127+0000 7fadd2487640 1 --2- 172.21.15.76:0/1768992948 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadcc0b42c0 0x7fadcc1541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.127+0000 7fadd2487640 1 -- 172.21.15.76:0/1768992948 >> 172.21.15.76:0/1768992948 conn(0x7fadcc0ce950 msgr2=0x7fadcc0b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:54.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.127+0000 7fadd2487640 1 -- 172.21.15.76:0/1768992948 shutdown_connections 2023-11-09T03:47:54.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.127+0000 7fadd2487640 1 -- 172.21.15.76:0/1768992948 wait complete. 2023-11-09T03:47:54.137 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:47:54.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.238+0000 7feb30d90640 1 Processor -- start 2023-11-09T03:47:54.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.238+0000 7feb30d90640 1 -- start start 2023-11-09T03:47:54.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.238+0000 7feb30d90640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feb2c0b0ae0 0x7feb2c0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:54.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.238+0000 7feb30d90640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb2c06db20 0x7feb2c15d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:54.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.239+0000 7feb30d90640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2c0b4110 con 0x7feb2c159c70 2023-11-09T03:47:54.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.239+0000 7feb30d90640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb2c0b49b0 con 0x7feb2c0b0ae0 2023-11-09T03:47:54.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.239+0000 7feb30d90640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c0b43f0 con 0x7feb2c06db20 2023-11-09T03:47:54.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.239+0000 7feb29d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feb2c0b0ae0 0x7feb2c0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:54.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.239+0000 7feb2ad9d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb2c06db20 0x7feb2c15d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:54.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.239+0000 7feb2a59c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7feb2c159c70 0x7feb2c0b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:57446/0 (socket says 172.21.15.76:57446) 2023-11-09T03:47:54.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.239+0000 7feb2a59c640 1 -- 172.21.15.76:0/2232211156 learned_addr learned my addr 172.21.15.76:0/2232211156 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:54.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.239+0000 7feb29d9b640 1 -- 172.21.15.76:0/2232211156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb2c06db20 msgr2=0x7feb2c15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:54.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.239+0000 7feb29d9b640 1 --2- 172.21.15.76:0/2232211156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb2c06db20 0x7feb2c15d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.239+0000 7feb29d9b640 1 -- 172.21.15.76:0/2232211156 >> v1:172.21.15.113:6789/0 conn(0x7feb2c159c70 legacy=0x7feb2c0b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:54.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.239+0000 7feb29d9b640 1 -- 172.21.15.76:0/2232211156 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2c0b46d0 con 0x7feb2c0b0ae0 2023-11-09T03:47:54.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.239+0000 7feb2ad9d640 1 --2- 172.21.15.76:0/2232211156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb2c06db20 0x7feb2c15d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:47:54.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.240+0000 7feb29d9b640 1 --2- 172.21.15.76:0/2232211156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb2c0b0ae0 0x7feb2c0ce370 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feb200097a0 tx=0x7feb2002b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cc8a88bc04224b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:54.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.240+0000 7feb2959a640 1 -- 172.21.15.76:0/2232211156 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb20030710 con 0x7feb2c0b0ae0 2023-11-09T03:47:54.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.240+0000 7feb2959a640 1 -- 172.21.15.76:0/2232211156 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2002e050 con 0x7feb2c0b0ae0 2023-11-09T03:47:54.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.240+0000 7feb2959a640 1 -- 172.21.15.76:0/2232211156 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb20030a90 con 0x7feb2c0b0ae0 2023-11-09T03:47:54.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.241+0000 7feb30d90640 1 -- 172.21.15.76:0/2232211156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb2c0b0ae0 msgr2=0x7feb2c0ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:54.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.241+0000 7feb30d90640 1 --2- 172.21.15.76:0/2232211156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb2c0b0ae0 0x7feb2c0ce370 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feb200097a0 tx=0x7feb2002b710 comp rx=0 tx=0).stop 2023-11-09T03:47:54.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.241+0000 7feb30d90640 1 -- 172.21.15.76:0/2232211156 shutdown_connections 2023-11-09T03:47:54.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.241+0000 7feb30d90640 1 --2- 172.21.15.76:0/2232211156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb2c06db20 0x7feb2c15d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.241+0000 7feb30d90640 1 --2- 172.21.15.76:0/2232211156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb2c0b0ae0 0x7feb2c0ce370 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.241+0000 7feb30d90640 1 -- 172.21.15.76:0/2232211156 >> 172.21.15.76:0/2232211156 conn(0x7feb2c0ce950 msgr2=0x7feb2c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:54.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.241+0000 7feb30d90640 1 -- 172.21.15.76:0/2232211156 shutdown_connections 2023-11-09T03:47:54.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.241+0000 7feb30d90640 1 -- 172.21.15.76:0/2232211156 wait complete. 2023-11-09T03:47:54.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.242+0000 7feb30d90640 1 Processor -- start 2023-11-09T03:47:54.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.242+0000 7feb30d90640 1 -- start start 2023-11-09T03:47:54.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.243+0000 7feb30d90640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb2c06db20 0x7feb2c156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:54.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.243+0000 7feb30d90640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb2c0b0ae0 0x7feb2c0ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:54.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.243+0000 7feb30d90640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb2c159c70 0x7feb2c1553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:54.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.243+0000 7feb30d90640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c0aecb0 con 0x7feb2c06db20 2023-11-09T03:47:54.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.243+0000 7feb30d90640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c0ae0a0 con 0x7feb2c0b0ae0 2023-11-09T03:47:54.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.243+0000 7feb30d90640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c0ae380 con 0x7feb2c159c70 2023-11-09T03:47:54.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.243+0000 7feb2a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb2c06db20 0x7feb2c156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:54.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.243+0000 7feb2a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb2c06db20 0x7feb2c156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41094/0 (socket says 172.21.15.76:41094) 2023-11-09T03:47:54.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.243+0000 7feb2a59c640 1 -- 172.21.15.76:0/3189206059 learned_addr learned my addr 172.21.15.76:0/3189206059 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:54.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.243+0000 7feb2ad9d640 1 --2- 172.21.15.76:0/3189206059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb2c159c70 0x7feb2c1553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:54.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.243+0000 7feb29d9b640 1 --2- 172.21.15.76:0/3189206059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb2c0b0ae0 0x7feb2c0ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:54.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.243+0000 7feb2a59c640 1 -- 172.21.15.76:0/3189206059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb2c159c70 msgr2=0x7feb2c1553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:54.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.244+0000 7feb2a59c640 1 --2- 172.21.15.76:0/3189206059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb2c159c70 0x7feb2c1553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.244+0000 7feb2a59c640 1 -- 172.21.15.76:0/3189206059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb2c0b0ae0 msgr2=0x7feb2c0ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:54.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.244+0000 7feb2a59c640 1 --2- 172.21.15.76:0/3189206059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb2c0b0ae0 0x7feb2c0ad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.244+0000 7feb2a59c640 1 -- 172.21.15.76:0/3189206059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb1c009400 con 0x7feb2c06db20 2023-11-09T03:47:54.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.244+0000 7feb2a59c640 1 --2- 172.21.15.76:0/3189206059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb2c06db20 0x7feb2c156570 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7feb24002080 tx=0x7feb2400cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:54.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.244+0000 7feb1affd640 1 -- 172.21.15.76:0/3189206059 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb24004d70 con 0x7feb2c06db20 2023-11-09T03:47:54.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.244+0000 7feb1affd640 1 -- 172.21.15.76:0/3189206059 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2400a700 con 0x7feb2c06db20 2023-11-09T03:47:54.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.244+0000 7feb30d90640 1 -- 172.21.15.76:0/3189206059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb20009400 con 0x7feb2c06db20 2023-11-09T03:47:54.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.244+0000 7feb1affd640 1 -- 172.21.15.76:0/3189206059 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2400aaf0 con 0x7feb2c06db20 2023-11-09T03:47:54.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.244+0000 7feb30d90640 1 -- 172.21.15.76:0/3189206059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb2c0ae940 con 0x7feb2c06db20 2023-11-09T03:47:54.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.245+0000 7feb30d90640 1 -- 172.21.15.76:0/3189206059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb2c074290 con 0x7feb2c06db20 2023-11-09T03:47:54.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.245+0000 7feb1affd640 1 -- 172.21.15.76:0/3189206059 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feb2400ac50 con 0x7feb2c06db20 2023-11-09T03:47:54.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.245+0000 7feb1affd640 1 --2- 172.21.15.76:0/3189206059 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7feb0c06ae70 0x7feb0c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:54.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.246+0000 7feb1affd640 1 -- 172.21.15.76:0/3189206059 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7feb2408ba30 con 0x7feb2c06db20 2023-11-09T03:47:54.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.246+0000 7feb29d9b640 1 --2- 172.21.15.76:0/3189206059 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7feb0c06ae70 0x7feb0c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:54.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.246+0000 7feb29d9b640 1 --2- 172.21.15.76:0/3189206059 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7feb0c06ae70 0x7feb0c06d330 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7feb20034000 tx=0x7feb20008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:54.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.250+0000 7feb1affd640 1 -- 172.21.15.76:0/3189206059 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feb24056840 con 0x7feb2c06db20 2023-11-09T03:47:54.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.418+0000 7feb30d90640 1 -- 172.21.15.76:0/3189206059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb2c0ad530 con 0x7feb2c06db20 2023-11-09T03:47:54.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.419+0000 7feb1affd640 1 -- 172.21.15.76:0/3189206059 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+18514 (secure 0 0 0) 0x7feb24015070 con 0x7feb2c06db20 2023-11-09T03:47:54.419 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:54.419 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":66,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:47:52.409039+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:47:49.630346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-09T03:47:50.082207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:47:54.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.421+0000 7feb30d90640 1 -- 172.21.15.76:0/3189206059 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7feb0c06ae70 msgr2=0x7feb0c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:54.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.421+0000 7feb30d90640 1 --2- 172.21.15.76:0/3189206059 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7feb0c06ae70 0x7feb0c06d330 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7feb20034000 tx=0x7feb20008030 comp rx=0 tx=0).stop 2023-11-09T03:47:54.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.421+0000 7feb30d90640 1 -- 172.21.15.76:0/3189206059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb2c06db20 msgr2=0x7feb2c156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:54.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.421+0000 7feb30d90640 1 --2- 172.21.15.76:0/3189206059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb2c06db20 0x7feb2c156570 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7feb24002080 tx=0x7feb2400cec0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.421+0000 7feb30d90640 1 -- 172.21.15.76:0/3189206059 shutdown_connections 2023-11-09T03:47:54.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.421+0000 7feb30d90640 1 --2- 172.21.15.76:0/3189206059 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7feb0c06ae70 0x7feb0c06d330 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.421+0000 7feb30d90640 1 --2- 172.21.15.76:0/3189206059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb2c159c70 0x7feb2c1553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.421+0000 7feb30d90640 1 --2- 172.21.15.76:0/3189206059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb2c0b0ae0 0x7feb2c0ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.421+0000 7feb30d90640 1 --2- 172.21.15.76:0/3189206059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb2c06db20 0x7feb2c156570 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.421+0000 7feb30d90640 1 -- 172.21.15.76:0/3189206059 >> 172.21.15.76:0/3189206059 conn(0x7feb2c0ce950 msgr2=0x7feb2c1e63d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:54.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.421+0000 7feb30d90640 1 -- 172.21.15.76:0/3189206059 shutdown_connections 2023-11-09T03:47:54.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.421+0000 7feb30d90640 1 -- 172.21.15.76:0/3189206059 wait complete. 2023-11-09T03:47:54.432 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-09T03:47:54.432 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-09T03:47:54.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.533+0000 7efd1d705640 1 Processor -- start 2023-11-09T03:47:54.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.533+0000 7efd1d705640 1 -- start start 2023-11-09T03:47:54.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.534+0000 7efd1d705640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd1815cd20 0x7efd181613b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:54.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.534+0000 7efd1d705640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efd1806cae0 0x7efd180b2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:54.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.534+0000 7efd1d705640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd1806c650 con 0x7efd1806cf40 2023-11-09T03:47:54.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.534+0000 7efd1d705640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd1806b830 con 0x7efd1806cae0 2023-11-09T03:47:54.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.534+0000 7efd1d705640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd181618f0 con 0x7efd1815cd20 2023-11-09T03:47:54.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.534+0000 7efd177fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7efd1806cf40 0x7efd180cb490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57966/0 (socket says 172.21.15.76:57966) 2023-11-09T03:47:54.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.534+0000 7efd177fe640 1 -- 172.21.15.76:0/935079683 learned_addr learned my addr 172.21.15.76:0/935079683 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:54.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.535+0000 7efd167fc640 1 -- 172.21.15.76:0/935079683 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 608044185 0 0) 0x7efd1806c650 con 0x7efd1806cf40 2023-11-09T03:47:54.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.535+0000 7efd167fc640 1 -- 172.21.15.76:0/935079683 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd04003520 con 0x7efd1806cf40 2023-11-09T03:47:54.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.535+0000 7efd16ffd640 1 --2- 172.21.15.76:0/935079683 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd1815cd20 0x7efd181613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:54.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.535+0000 7efd0ffff640 1 --2- 172.21.15.76:0/935079683 >> v2:172.21.15.113:3300/0 conn(0x7efd1806cae0 0x7efd180b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:54.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.535+0000 7efd167fc640 1 -- 172.21.15.76:0/935079683 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1760645453 0 0) 0x7efd04003520 con 0x7efd1806cf40 2023-11-09T03:47:54.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.535+0000 7efd167fc640 1 -- 172.21.15.76:0/935079683 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd1815cd20 msgr2=0x7efd181613b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:54.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.535+0000 7efd167fc640 1 --2- 172.21.15.76:0/935079683 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd1815cd20 0x7efd181613b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.535+0000 7efd167fc640 1 -- 172.21.15.76:0/935079683 >> v2:172.21.15.113:3300/0 conn(0x7efd1806cae0 msgr2=0x7efd180b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:54.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.535+0000 7efd167fc640 1 --2- 172.21.15.76:0/935079683 >> v2:172.21.15.113:3300/0 conn(0x7efd1806cae0 0x7efd180b2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.535+0000 7efd167fc640 1 -- 172.21.15.76:0/935079683 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1806ba40 con 0x7efd1806cf40 2023-11-09T03:47:54.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.535+0000 7efd167fc640 1 -- 172.21.15.76:0/935079683 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7efd100019f0 con 0x7efd1806cf40 2023-11-09T03:47:54.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.535+0000 7efd167fc640 1 -- 172.21.15.76:0/935079683 >> v1:172.21.15.76:6789/0 conn(0x7efd1806cf40 legacy=0x7efd180cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:54.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.535+0000 7efd167fc640 1 --2- 172.21.15.76:0/935079683 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd1815cd20 0x7efd04003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:54.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.535+0000 7efd167fc640 1 --2- 172.21.15.76:0/935079683 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd04003ef0 0x7efd040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:54.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.535+0000 7efd167fc640 1 --2- 172.21.15.76:0/935079683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd04004d00 0x7efd04005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:54.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.535+0000 7efd167fc640 1 -- 172.21.15.76:0/935079683 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1806b830 con 0x7efd04003ef0 2023-11-09T03:47:54.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.535+0000 7efd167fc640 1 -- 172.21.15.76:0/935079683 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd181618f0 con 0x7efd04004d00 2023-11-09T03:47:54.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.535+0000 7efd167fc640 1 -- 172.21.15.76:0/935079683 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd0400b980 con 0x7efd1815cd20 2023-11-09T03:47:54.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.535+0000 7efd0ffff640 1 --2- 172.21.15.76:0/935079683 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd04003ef0 0x7efd040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:54.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.535+0000 7efd177fe640 1 --2- 172.21.15.76:0/935079683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd04004d00 0x7efd04005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:54.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.535+0000 7efd16ffd640 1 --2- 172.21.15.76:0/935079683 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd1815cd20 0x7efd04003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:54.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.535+0000 7efd0ffff640 1 -- 172.21.15.76:0/935079683 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd1815cd20 msgr2=0x7efd04003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:54.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.535+0000 7efd0ffff640 1 --2- 172.21.15.76:0/935079683 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd1815cd20 0x7efd04003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.535+0000 7efd0ffff640 1 -- 172.21.15.76:0/935079683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd04004d00 msgr2=0x7efd04005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:54.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.535+0000 7efd0ffff640 1 --2- 172.21.15.76:0/935079683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd04004d00 0x7efd04005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.535+0000 7efd0ffff640 1 -- 172.21.15.76:0/935079683 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd04003520 con 0x7efd04003ef0 2023-11-09T03:47:54.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.535+0000 7efd177fe640 1 --2- 172.21.15.76:0/935079683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd04004d00 0x7efd04005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:54.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.536+0000 7efd0ffff640 1 --2- 172.21.15.76:0/935079683 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd04003ef0 0x7efd040047c0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7efd00009ac0 tx=0x7efd040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39eb3e9a1f44a06 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:54.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.536+0000 7efd167fc640 1 -- 172.21.15.76:0/935079683 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0002c6d0 con 0x7efd04003ef0 2023-11-09T03:47:54.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.536+0000 7efd167fc640 1 -- 172.21.15.76:0/935079683 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0002c830 con 0x7efd04003ef0 2023-11-09T03:47:54.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.536+0000 7efd167fc640 1 -- 172.21.15.76:0/935079683 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0002cbc0 con 0x7efd04003ef0 2023-11-09T03:47:54.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.537+0000 7efd1d705640 1 -- 172.21.15.76:0/935079683 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd04003ef0 msgr2=0x7efd040047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:54.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.537+0000 7efd1d705640 1 --2- 172.21.15.76:0/935079683 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd04003ef0 0x7efd040047c0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7efd00009ac0 tx=0x7efd040057f0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.537+0000 7efd177fe640 1 -- 172.21.15.76:0/935079683 reap_dead start 2023-11-09T03:47:54.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.537+0000 7efd1d705640 1 -- 172.21.15.76:0/935079683 shutdown_connections 2023-11-09T03:47:54.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.537+0000 7efd1d705640 1 -- 172.21.15.76:0/935079683 >> 172.21.15.76:0/935079683 conn(0x7efd180cd600 msgr2=0x7efd180cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:54.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.537+0000 7efd1d705640 1 -- 172.21.15.76:0/935079683 shutdown_connections 2023-11-09T03:47:54.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.537+0000 7efd1d705640 1 -- 172.21.15.76:0/935079683 wait complete. 2023-11-09T03:47:54.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.538+0000 7efd1d705640 1 Processor -- start 2023-11-09T03:47:54.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.538+0000 7efd1d705640 1 -- start start 2023-11-09T03:47:54.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.539+0000 7efd1d705640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd04003ef0 0x7efd180c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:54.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.539+0000 7efd1d705640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd181f6a10 0x7efd181f3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:54.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.539+0000 7efd1d705640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd181f5e70 0x7efd181f62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:54.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.539+0000 7efd1d705640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd181f6830 con 0x7efd181f5e70 2023-11-09T03:47:54.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.539+0000 7efd1d705640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd180b41d0 con 0x7efd04003ef0 2023-11-09T03:47:54.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.539+0000 7efd1d705640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd180ca640 con 0x7efd181f6a10 2023-11-09T03:47:54.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.539+0000 7efd177fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd181f5e70 0x7efd181f62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:54.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.539+0000 7efd177fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd181f5e70 0x7efd181f62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41112/0 (socket says 172.21.15.76:41112) 2023-11-09T03:47:54.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.539+0000 7efd177fe640 1 -- 172.21.15.76:0/2900813698 learned_addr learned my addr 172.21.15.76:0/2900813698 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:54.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.539+0000 7efd16ffd640 1 --2- 172.21.15.76:0/2900813698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd04003ef0 0x7efd180c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:54.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.539+0000 7efd177fe640 1 -- 172.21.15.76:0/2900813698 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd181f6a10 msgr2=0x7efd181f3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:54.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.539+0000 7efd0ffff640 1 --2- 172.21.15.76:0/2900813698 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd181f6a10 0x7efd181f3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:54.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.539+0000 7efd177fe640 1 --2- 172.21.15.76:0/2900813698 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd181f6a10 0x7efd181f3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.539+0000 7efd177fe640 1 -- 172.21.15.76:0/2900813698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd04003ef0 msgr2=0x7efd180c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:54.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.539+0000 7efd177fe640 1 --2- 172.21.15.76:0/2900813698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd04003ef0 0x7efd180c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.539+0000 7efd177fe640 1 -- 172.21.15.76:0/2900813698 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd180b38a0 con 0x7efd181f5e70 2023-11-09T03:47:54.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.539+0000 7efd16ffd640 1 --2- 172.21.15.76:0/2900813698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd04003ef0 0x7efd180c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:54.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.539+0000 7efd177fe640 1 --2- 172.21.15.76:0/2900813698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd181f5e70 0x7efd181f62f0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7efd100013d0 tx=0x7efd1000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:54.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.539+0000 7efd0f7fe640 1 -- 172.21.15.76:0/2900813698 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd10007a90 con 0x7efd181f5e70 2023-11-09T03:47:54.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.539+0000 7efd0f7fe640 1 -- 172.21.15.76:0/2900813698 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd1000ed40 con 0x7efd181f5e70 2023-11-09T03:47:54.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.539+0000 7efd1d705640 1 -- 172.21.15.76:0/2900813698 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd180b3b80 con 0x7efd181f5e70 2023-11-09T03:47:54.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.539+0000 7efd0f7fe640 1 -- 172.21.15.76:0/2900813698 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd10014670 con 0x7efd181f5e70 2023-11-09T03:47:54.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.539+0000 7efd1d705640 1 -- 172.21.15.76:0/2900813698 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd180ca280 con 0x7efd181f5e70 2023-11-09T03:47:54.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.540+0000 7efd1d705640 1 -- 172.21.15.76:0/2900813698 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd180ca410 con 0x7efd181f5e70 2023-11-09T03:47:54.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.541+0000 7efd0f7fe640 1 -- 172.21.15.76:0/2900813698 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efd100147d0 con 0x7efd181f5e70 2023-11-09T03:47:54.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.541+0000 7efd0f7fe640 1 --2- 172.21.15.76:0/2900813698 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efcf806aea0 0x7efcf806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:54.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.541+0000 7efd0f7fe640 1 -- 172.21.15.76:0/2900813698 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7efd10089590 con 0x7efd181f5e70 2023-11-09T03:47:54.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.542+0000 7efd16ffd640 1 --2- 172.21.15.76:0/2900813698 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efcf806aea0 0x7efcf806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:54.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.542+0000 7efd16ffd640 1 --2- 172.21.15.76:0/2900813698 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efcf806aea0 0x7efcf806d360 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7efd180b36c0 tx=0x7efd08008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:54.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.546+0000 7efd0f7fe640 1 -- 172.21.15.76:0/2900813698 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efd10054430 con 0x7efd181f5e70 2023-11-09T03:47:54.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.708+0000 7efd1d705640 1 -- 172.21.15.76:0/2900813698 --> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efd1806bd20 con 0x7efcf806aea0 2023-11-09T03:47:54.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.718+0000 7efd0f7fe640 1 -- 172.21.15.76:0/2900813698 <== mgr.4105 v2:172.21.15.113:6832/1264193042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+370461 (secure 0 0 0) 0x7efd08005990 con 0x7efcf806aea0 2023-11-09T03:47:54.721 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:54.723 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2023-11-09T03:47:54.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.721+0000 7efd1d705640 1 -- 172.21.15.76:0/2900813698 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efcf806aea0 msgr2=0x7efcf806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:54.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.721+0000 7efd1d705640 1 --2- 172.21.15.76:0/2900813698 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efcf806aea0 0x7efcf806d360 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7efd180b36c0 tx=0x7efd08008040 comp rx=0 tx=0).stop 2023-11-09T03:47:54.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.721+0000 7efd1d705640 1 -- 172.21.15.76:0/2900813698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd181f5e70 msgr2=0x7efd181f62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:54.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.721+0000 7efd1d705640 1 --2- 172.21.15.76:0/2900813698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd181f5e70 0x7efd181f62f0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7efd100013d0 tx=0x7efd1000e530 comp rx=0 tx=0).stop 2023-11-09T03:47:54.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.721+0000 7efd1d705640 1 -- 172.21.15.76:0/2900813698 shutdown_connections 2023-11-09T03:47:54.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.721+0000 7efd1d705640 1 --2- 172.21.15.76:0/2900813698 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efcf806aea0 0x7efcf806d360 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.721+0000 7efd1d705640 1 --2- 172.21.15.76:0/2900813698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd181f5e70 0x7efd181f62f0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.721+0000 7efd1d705640 1 --2- 172.21.15.76:0/2900813698 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd181f6a10 0x7efd181f3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.721+0000 7efd1d705640 1 --2- 172.21.15.76:0/2900813698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd04003ef0 0x7efd180c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.721+0000 7efd1d705640 1 -- 172.21.15.76:0/2900813698 >> 172.21.15.76:0/2900813698 conn(0x7efd180cd600 msgr2=0x7efd1815dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:54.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.721+0000 7efd1d705640 1 -- 172.21.15.76:0/2900813698 shutdown_connections 2023-11-09T03:47:54.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.721+0000 7efd1d705640 1 -- 172.21.15.76:0/2900813698 wait complete. 2023-11-09T03:47:54.732 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2023-11-09T03:47:52.784815+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223580,"kb_used_data":7364,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751140,"statfs":{"total":773094113280,"available":772865167360,"internally_reserved":0,"allocated":7540736,"data_stored":4076385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000766"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.433191+0000","last_change":"2023-11-09T03:47:51.431585+0000","last_active":"2023-11-09T03:47:51.433191+0000","last_peered":"2023-11-09T03:47:51.433191+0000","last_clean":"2023-11-09T03:47:51.433191+0000","last_became_active":"2023-11-09T03:47:51.430932+0000","last_became_peered":"2023-11-09T03:47:51.430932+0000","last_unstale":"2023-11-09T03:47:51.433191+0000","last_undegraded":"2023-11-09T03:47:51.433191+0000","last_fullsized":"2023-11-09T03:47:51.433191+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:21:20.038282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.437544+0000","last_change":"2023-11-09T03:47:51.436525+0000","last_active":"2023-11-09T03:47:51.437544+0000","last_peered":"2023-11-09T03:47:51.437544+0000","last_clean":"2023-11-09T03:47:51.437544+0000","last_became_active":"2023-11-09T03:47:51.436248+0000","last_became_peered":"2023-11-09T03:47:51.436248+0000","last_unstale":"2023-11-09T03:47:51.437544+0000","last_undegraded":"2023-11-09T03:47:51.437544+0000","last_fullsized":"2023-11-09T03:47:51.437544+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:52:24.725477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.423880+0000","last_change":"2023-11-09T03:47:51.419938+0000","last_active":"2023-11-09T03:47:51.423880+0000","last_peered":"2023-11-09T03:47:51.423880+0000","last_clean":"2023-11-09T03:47:51.423880+0000","last_became_active":"2023-11-09T03:47:51.419181+0000","last_became_peered":"2023-11-09T03:47:51.419181+0000","last_unstale":"2023-11-09T03:47:51.423880+0000","last_undegraded":"2023-11-09T03:47:51.423880+0000","last_fullsized":"2023-11-09T03:47:51.423880+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:27:39.529115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.427045+0000","last_change":"2023-11-09T03:47:51.425618+0000","last_active":"2023-11-09T03:47:51.427045+0000","last_peered":"2023-11-09T03:47:51.427045+0000","last_clean":"2023-11-09T03:47:51.427045+0000","last_became_active":"2023-11-09T03:47:51.425344+0000","last_became_peered":"2023-11-09T03:47:51.425344+0000","last_unstale":"2023-11-09T03:47:51.427045+0000","last_undegraded":"2023-11-09T03:47:51.427045+0000","last_fullsized":"2023-11-09T03:47:51.427045+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:06:42.522828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.432897+0000","last_change":"2023-11-09T03:47:51.430203+0000","last_active":"2023-11-09T03:47:51.432897+0000","last_peered":"2023-11-09T03:47:51.432897+0000","last_clean":"2023-11-09T03:47:51.432897+0000","last_became_active":"2023-11-09T03:47:51.429837+0000","last_became_peered":"2023-11-09T03:47:51.429837+0000","last_unstale":"2023-11-09T03:47:51.432897+0000","last_undegraded":"2023-11-09T03:47:51.432897+0000","last_fullsized":"2023-11-09T03:47:51.432897+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:26:42.949360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.435746+0000","last_change":"2023-11-09T03:47:51.434475+0000","last_active":"2023-11-09T03:47:51.435746+0000","last_peered":"2023-11-09T03:47:51.435746+0000","last_clean":"2023-11-09T03:47:51.435746+0000","last_became_active":"2023-11-09T03:47:51.434185+0000","last_became_peered":"2023-11-09T03:47:51.434185+0000","last_unstale":"2023-11-09T03:47:51.435746+0000","last_undegraded":"2023-11-09T03:47:51.435746+0000","last_fullsized":"2023-11-09T03:47:51.435746+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:05:01.816130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.432495+0000","last_change":"2023-11-09T03:47:51.428082+0000","last_active":"2023-11-09T03:47:51.432495+0000","last_peered":"2023-11-09T03:47:51.432495+0000","last_clean":"2023-11-09T03:47:51.432495+0000","last_became_active":"2023-11-09T03:47:51.427814+0000","last_became_peered":"2023-11-09T03:47:51.427814+0000","last_unstale":"2023-11-09T03:47:51.432495+0000","last_undegraded":"2023-11-09T03:47:51.432495+0000","last_fullsized":"2023-11-09T03:47:51.432495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:42:31.986727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.433050+0000","last_change":"2023-11-09T03:47:51.431870+0000","last_active":"2023-11-09T03:47:51.433050+0000","last_peered":"2023-11-09T03:47:51.433050+0000","last_clean":"2023-11-09T03:47:51.433050+0000","last_became_active":"2023-11-09T03:47:51.431385+0000","last_became_peered":"2023-11-09T03:47:51.431385+0000","last_unstale":"2023-11-09T03:47:51.433050+0000","last_undegraded":"2023-11-09T03:47:51.433050+0000","last_fullsized":"2023-11-09T03:47:51.433050+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:59:01.404347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.421594+0000","last_change":"2023-11-09T03:47:51.419475+0000","last_active":"2023-11-09T03:47:51.421594+0000","last_peered":"2023-11-09T03:47:51.421594+0000","last_clean":"2023-11-09T03:47:51.421594+0000","last_became_active":"2023-11-09T03:47:51.418167+0000","last_became_peered":"2023-11-09T03:47:51.418167+0000","last_unstale":"2023-11-09T03:47:51.421594+0000","last_undegraded":"2023-11-09T03:47:51.421594+0000","last_fullsized":"2023-11-09T03:47:51.421594+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:58:40.615051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.413230+0000","last_change":"2023-11-09T03:47:51.425933+0000","last_active":"2023-11-09T03:47:52.413230+0000","last_peered":"2023-11-09T03:47:52.413230+0000","last_clean":"2023-11-09T03:47:52.413230+0000","last_became_active":"2023-11-09T03:47:51.425597+0000","last_became_peered":"2023-11-09T03:47:51.425597+0000","last_unstale":"2023-11-09T03:47:52.413230+0000","last_undegraded":"2023-11-09T03:47:52.413230+0000","last_fullsized":"2023-11-09T03:47:52.413230+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:09:11.376416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.431687+0000","last_change":"2023-11-09T03:47:51.430610+0000","last_active":"2023-11-09T03:47:51.431687+0000","last_peered":"2023-11-09T03:47:51.431687+0000","last_clean":"2023-11-09T03:47:51.431687+0000","last_became_active":"2023-11-09T03:47:51.429575+0000","last_became_peered":"2023-11-09T03:47:51.429575+0000","last_unstale":"2023-11-09T03:47:51.431687+0000","last_undegraded":"2023-11-09T03:47:51.431687+0000","last_fullsized":"2023-11-09T03:47:51.431687+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:38:52.289915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.422504+0000","last_change":"2023-11-09T03:47:51.420829+0000","last_active":"2023-11-09T03:47:51.422504+0000","last_peered":"2023-11-09T03:47:51.422504+0000","last_clean":"2023-11-09T03:47:51.422504+0000","last_became_active":"2023-11-09T03:47:51.420557+0000","last_became_peered":"2023-11-09T03:47:51.420557+0000","last_unstale":"2023-11-09T03:47:51.422504+0000","last_undegraded":"2023-11-09T03:47:51.422504+0000","last_fullsized":"2023-11-09T03:47:51.422504+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:00:05.584311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.414736+0000","last_change":"2023-11-09T03:47:51.426249+0000","last_active":"2023-11-09T03:47:52.414736+0000","last_peered":"2023-11-09T03:47:52.414736+0000","last_clean":"2023-11-09T03:47:52.414736+0000","last_became_active":"2023-11-09T03:47:51.425990+0000","last_became_peered":"2023-11-09T03:47:51.425990+0000","last_unstale":"2023-11-09T03:47:52.414736+0000","last_undegraded":"2023-11-09T03:47:52.414736+0000","last_fullsized":"2023-11-09T03:47:52.414736+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:20:51.046991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.425940+0000","last_change":"2023-11-09T03:47:51.425017+0000","last_active":"2023-11-09T03:47:51.425940+0000","last_peered":"2023-11-09T03:47:51.425940+0000","last_clean":"2023-11-09T03:47:51.425940+0000","last_became_active":"2023-11-09T03:47:51.424755+0000","last_became_peered":"2023-11-09T03:47:51.424755+0000","last_unstale":"2023-11-09T03:47:51.425940+0000","last_undegraded":"2023-11-09T03:47:51.425940+0000","last_fullsized":"2023-11-09T03:47:51.425940+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:53:51.173223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.414815+0000","last_change":"2023-11-09T03:47:51.418529+0000","last_active":"2023-11-09T03:47:52.414815+0000","last_peered":"2023-11-09T03:47:52.414815+0000","last_clean":"2023-11-09T03:47:52.414815+0000","last_became_active":"2023-11-09T03:47:51.418260+0000","last_became_peered":"2023-11-09T03:47:51.418260+0000","last_unstale":"2023-11-09T03:47:52.414815+0000","last_undegraded":"2023-11-09T03:47:52.414815+0000","last_fullsized":"2023-11-09T03:47:52.414815+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:16:23.544065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.412904+0000","last_change":"2023-11-09T03:47:51.430458+0000","last_active":"2023-11-09T03:47:52.412904+0000","last_peered":"2023-11-09T03:47:52.412904+0000","last_clean":"2023-11-09T03:47:52.412904+0000","last_became_active":"2023-11-09T03:47:51.430165+0000","last_became_peered":"2023-11-09T03:47:51.430165+0000","last_unstale":"2023-11-09T03:47:52.412904+0000","last_undegraded":"2023-11-09T03:47:52.412904+0000","last_fullsized":"2023-11-09T03:47:52.412904+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:08:34.417859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.422230+0000","last_change":"2023-11-09T03:47:51.420761+0000","last_active":"2023-11-09T03:47:51.422230+0000","last_peered":"2023-11-09T03:47:51.422230+0000","last_clean":"2023-11-09T03:47:51.422230+0000","last_became_active":"2023-11-09T03:47:51.420484+0000","last_became_peered":"2023-11-09T03:47:51.420484+0000","last_unstale":"2023-11-09T03:47:51.422230+0000","last_undegraded":"2023-11-09T03:47:51.422230+0000","last_fullsized":"2023-11-09T03:47:51.422230+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:36:34.915038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.426869+0000","last_change":"2023-11-09T03:47:51.425219+0000","last_active":"2023-11-09T03:47:51.426869+0000","last_peered":"2023-11-09T03:47:51.426869+0000","last_clean":"2023-11-09T03:47:51.426869+0000","last_became_active":"2023-11-09T03:47:51.424984+0000","last_became_peered":"2023-11-09T03:47:51.424984+0000","last_unstale":"2023-11-09T03:47:51.426869+0000","last_undegraded":"2023-11-09T03:47:51.426869+0000","last_fullsized":"2023-11-09T03:47:51.426869+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:53:59.332467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.431510+0000","last_change":"2023-11-09T03:47:51.429393+0000","last_active":"2023-11-09T03:47:51.431510+0000","last_peered":"2023-11-09T03:47:51.431510+0000","last_clean":"2023-11-09T03:47:51.431510+0000","last_became_active":"2023-11-09T03:47:51.429115+0000","last_became_peered":"2023-11-09T03:47:51.429115+0000","last_unstale":"2023-11-09T03:47:51.431510+0000","last_undegraded":"2023-11-09T03:47:51.431510+0000","last_fullsized":"2023-11-09T03:47:51.431510+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:55:58.922785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.414645+0000","last_change":"2023-11-09T03:47:51.424978+0000","last_active":"2023-11-09T03:47:52.414645+0000","last_peered":"2023-11-09T03:47:52.414645+0000","last_clean":"2023-11-09T03:47:52.414645+0000","last_became_active":"2023-11-09T03:47:51.424712+0000","last_became_peered":"2023-11-09T03:47:51.424712+0000","last_unstale":"2023-11-09T03:47:52.414645+0000","last_undegraded":"2023-11-09T03:47:52.414645+0000","last_fullsized":"2023-11-09T03:47:52.414645+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:41.735881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.413179+0000","last_change":"2023-11-09T03:47:51.425271+0000","last_active":"2023-11-09T03:47:52.413179+0000","last_peered":"2023-11-09T03:47:52.413179+0000","last_clean":"2023-11-09T03:47:52.413179+0000","last_became_active":"2023-11-09T03:47:51.424721+0000","last_became_peered":"2023-11-09T03:47:51.424721+0000","last_unstale":"2023-11-09T03:47:52.413179+0000","last_undegraded":"2023-11-09T03:47:52.413179+0000","last_fullsized":"2023-11-09T03:47:52.413179+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:09:21.626828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.413155+0000","last_change":"2023-11-09T03:47:51.426715+0000","last_active":"2023-11-09T03:47:52.413155+0000","last_peered":"2023-11-09T03:47:52.413155+0000","last_clean":"2023-11-09T03:47:52.413155+0000","last_became_active":"2023-11-09T03:47:51.426415+0000","last_became_peered":"2023-11-09T03:47:51.426415+0000","last_unstale":"2023-11-09T03:47:52.413155+0000","last_undegraded":"2023-11-09T03:47:52.413155+0000","last_fullsized":"2023-11-09T03:47:52.413155+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:43:42.601800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.430510+0000","last_change":"2023-11-09T03:47:51.427292+0000","last_active":"2023-11-09T03:47:51.430510+0000","last_peered":"2023-11-09T03:47:51.430510+0000","last_clean":"2023-11-09T03:47:51.430510+0000","last_became_active":"2023-11-09T03:47:51.427049+0000","last_became_peered":"2023-11-09T03:47:51.427049+0000","last_unstale":"2023-11-09T03:47:51.430510+0000","last_undegraded":"2023-11-09T03:47:51.430510+0000","last_fullsized":"2023-11-09T03:47:51.430510+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:36:47.931275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.436619+0000","last_change":"2023-11-09T03:47:51.434689+0000","last_active":"2023-11-09T03:47:51.436619+0000","last_peered":"2023-11-09T03:47:51.436619+0000","last_clean":"2023-11-09T03:47:51.436619+0000","last_became_active":"2023-11-09T03:47:51.434411+0000","last_became_peered":"2023-11-09T03:47:51.434411+0000","last_unstale":"2023-11-09T03:47:51.436619+0000","last_undegraded":"2023-11-09T03:47:51.436619+0000","last_fullsized":"2023-11-09T03:47:51.436619+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:53:23.159911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.413487+0000","last_change":"2023-11-09T03:47:51.424006+0000","last_active":"2023-11-09T03:47:52.413487+0000","last_peered":"2023-11-09T03:47:52.413487+0000","last_clean":"2023-11-09T03:47:52.413487+0000","last_became_active":"2023-11-09T03:47:51.422581+0000","last_became_peered":"2023-11-09T03:47:51.422581+0000","last_unstale":"2023-11-09T03:47:52.413487+0000","last_undegraded":"2023-11-09T03:47:52.413487+0000","last_fullsized":"2023-11-09T03:47:52.413487+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:52:09.449985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.428703+0000","last_change":"2023-11-09T03:47:51.426203+0000","last_active":"2023-11-09T03:47:51.428703+0000","last_peered":"2023-11-09T03:47:51.428703+0000","last_clean":"2023-11-09T03:47:51.428703+0000","last_became_active":"2023-11-09T03:47:51.425355+0000","last_became_peered":"2023-11-09T03:47:51.425355+0000","last_unstale":"2023-11-09T03:47:51.428703+0000","last_undegraded":"2023-11-09T03:47:51.428703+0000","last_fullsized":"2023-11-09T03:47:51.428703+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:32:30.681190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.425427+0000","last_change":"2023-11-09T03:47:51.423308+0000","last_active":"2023-11-09T03:47:51.425427+0000","last_peered":"2023-11-09T03:47:51.425427+0000","last_clean":"2023-11-09T03:47:51.425427+0000","last_became_active":"2023-11-09T03:47:51.423067+0000","last_became_peered":"2023-11-09T03:47:51.423067+0000","last_unstale":"2023-11-09T03:47:51.425427+0000","last_undegraded":"2023-11-09T03:47:51.425427+0000","last_fullsized":"2023-11-09T03:47:51.425427+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:12:17.509338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.413202+0000","last_change":"2023-11-09T03:47:51.431335+0000","last_active":"2023-11-09T03:47:52.413202+0000","last_peered":"2023-11-09T03:47:52.413202+0000","last_clean":"2023-11-09T03:47:52.413202+0000","last_became_active":"2023-11-09T03:47:51.428252+0000","last_became_peered":"2023-11-09T03:47:51.428252+0000","last_unstale":"2023-11-09T03:47:52.413202+0000","last_undegraded":"2023-11-09T03:47:52.413202+0000","last_fullsized":"2023-11-09T03:47:52.413202+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:27:32.178108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.427921+0000","last_change":"2023-11-09T03:47:51.425727+0000","last_active":"2023-11-09T03:47:51.427921+0000","last_peered":"2023-11-09T03:47:51.427921+0000","last_clean":"2023-11-09T03:47:51.427921+0000","last_became_active":"2023-11-09T03:47:51.425517+0000","last_became_peered":"2023-11-09T03:47:51.425517+0000","last_unstale":"2023-11-09T03:47:51.427921+0000","last_undegraded":"2023-11-09T03:47:51.427921+0000","last_fullsized":"2023-11-09T03:47:51.427921+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:06:18.914118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.428649+0000","last_change":"2023-11-09T03:47:51.426023+0000","last_active":"2023-11-09T03:47:51.428649+0000","last_peered":"2023-11-09T03:47:51.428649+0000","last_clean":"2023-11-09T03:47:51.428649+0000","last_became_active":"2023-11-09T03:47:51.425767+0000","last_became_peered":"2023-11-09T03:47:51.425767+0000","last_unstale":"2023-11-09T03:47:51.428649+0000","last_undegraded":"2023-11-09T03:47:51.428649+0000","last_fullsized":"2023-11-09T03:47:51.428649+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:36:47.931275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.430145+0000","last_change":"2023-11-09T03:47:51.427277+0000","last_active":"2023-11-09T03:47:51.430145+0000","last_peered":"2023-11-09T03:47:51.430145+0000","last_clean":"2023-11-09T03:47:51.430145+0000","last_became_active":"2023-11-09T03:47:51.426955+0000","last_became_peered":"2023-11-09T03:47:51.426955+0000","last_unstale":"2023-11-09T03:47:51.430145+0000","last_undegraded":"2023-11-09T03:47:51.430145+0000","last_fullsized":"2023-11-09T03:47:51.430145+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:38:52.289915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.431683+0000","last_change":"2023-11-09T03:47:51.427399+0000","last_active":"2023-11-09T03:47:51.431683+0000","last_peered":"2023-11-09T03:47:51.431683+0000","last_clean":"2023-11-09T03:47:51.431683+0000","last_became_active":"2023-11-09T03:47:51.427133+0000","last_became_peered":"2023-11-09T03:47:51.427133+0000","last_unstale":"2023-11-09T03:47:51.431683+0000","last_undegraded":"2023-11-09T03:47:51.431683+0000","last_fullsized":"2023-11-09T03:47:51.431683+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:18.970489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.421470+0000","last_change":"2023-11-09T03:47:51.419191+0000","last_active":"2023-11-09T03:47:51.421470+0000","last_peered":"2023-11-09T03:47:51.421470+0000","last_clean":"2023-11-09T03:47:51.421470+0000","last_became_active":"2023-11-09T03:47:51.418955+0000","last_became_peered":"2023-11-09T03:47:51.418955+0000","last_unstale":"2023-11-09T03:47:51.421470+0000","last_undegraded":"2023-11-09T03:47:51.421470+0000","last_fullsized":"2023-11-09T03:47:51.421470+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:56:54.397650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"65'8","reported_seq":20,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.412909+0000","last_change":"2023-11-09T03:47:50.776715+0000","last_active":"2023-11-09T03:47:52.412909+0000","last_peered":"2023-11-09T03:47:52.412909+0000","last_clean":"2023-11-09T03:47:52.412909+0000","last_became_active":"2023-11-09T03:47:50.776449+0000","last_became_peered":"2023-11-09T03:47:50.776449+0000","last_unstale":"2023-11-09T03:47:52.412909+0000","last_undegraded":"2023-11-09T03:47:52.412909+0000","last_fullsized":"2023-11-09T03:47:52.412909+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:14:58.280559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.771810+0000","last_change":"2023-11-09T03:47:02.581917+0000","last_active":"2023-11-09T03:47:50.771810+0000","last_peered":"2023-11-09T03:47:50.771810+0000","last_clean":"2023-11-09T03:47:50.771810+0000","last_became_active":"2023-11-09T03:47:01.588082+0000","last_became_peered":"2023-11-09T03:47:01.588082+0000","last_unstale":"2023-11-09T03:47:50.771810+0000","last_undegraded":"2023-11-09T03:47:50.771810+0000","last_fullsized":"2023-11-09T03:47:50.771810+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:44:07.653557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034538899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.414593+0000","last_change":"2023-11-09T03:47:51.425687+0000","last_active":"2023-11-09T03:47:52.414593+0000","last_peered":"2023-11-09T03:47:52.414593+0000","last_clean":"2023-11-09T03:47:52.414593+0000","last_became_active":"2023-11-09T03:47:51.425337+0000","last_became_peered":"2023-11-09T03:47:51.425337+0000","last_unstale":"2023-11-09T03:47:52.414593+0000","last_undegraded":"2023-11-09T03:47:52.414593+0000","last_fullsized":"2023-11-09T03:47:52.414593+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:49:15.018709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.421104+0000","last_change":"2023-11-09T03:47:50.780392+0000","last_active":"2023-11-09T03:47:51.421104+0000","last_peered":"2023-11-09T03:47:51.421104+0000","last_clean":"2023-11-09T03:47:51.421104+0000","last_became_active":"2023-11-09T03:47:50.780121+0000","last_became_peered":"2023-11-09T03:47:50.780121+0000","last_unstale":"2023-11-09T03:47:51.421104+0000","last_undegraded":"2023-11-09T03:47:51.421104+0000","last_fullsized":"2023-11-09T03:47:51.421104+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:44:59.854706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":33,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.772789+0000","last_change":"2023-11-09T03:47:03.029063+0000","last_active":"2023-11-09T03:47:50.772789+0000","last_peered":"2023-11-09T03:47:50.772789+0000","last_clean":"2023-11-09T03:47:50.772789+0000","last_became_active":"2023-11-09T03:47:01.593582+0000","last_became_peered":"2023-11-09T03:47:01.593582+0000","last_unstale":"2023-11-09T03:47:50.772789+0000","last_undegraded":"2023-11-09T03:47:50.772789+0000","last_fullsized":"2023-11-09T03:47:50.772789+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:56:20.649937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041855100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.432338+0000","last_change":"2023-11-09T03:47:51.428315+0000","last_active":"2023-11-09T03:47:51.432338+0000","last_peered":"2023-11-09T03:47:51.432338+0000","last_clean":"2023-11-09T03:47:51.432338+0000","last_became_active":"2023-11-09T03:47:51.428074+0000","last_became_peered":"2023-11-09T03:47:51.428074+0000","last_unstale":"2023-11-09T03:47:51.432338+0000","last_undegraded":"2023-11-09T03:47:51.432338+0000","last_fullsized":"2023-11-09T03:47:51.432338+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:11:17.062291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.422640+0000","last_change":"2023-11-09T03:47:50.785900+0000","last_active":"2023-11-09T03:47:51.422640+0000","last_peered":"2023-11-09T03:47:51.422640+0000","last_clean":"2023-11-09T03:47:51.422640+0000","last_became_active":"2023-11-09T03:47:50.785601+0000","last_became_peered":"2023-11-09T03:47:50.785601+0000","last_unstale":"2023-11-09T03:47:51.422640+0000","last_undegraded":"2023-11-09T03:47:51.422640+0000","last_fullsized":"2023-11-09T03:47:51.422640+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:06:41.516591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.783141+0000","last_change":"2023-11-09T03:47:02.619336+0000","last_active":"2023-11-09T03:47:50.783141+0000","last_peered":"2023-11-09T03:47:50.783141+0000","last_clean":"2023-11-09T03:47:50.783141+0000","last_became_active":"2023-11-09T03:47:01.587749+0000","last_became_peered":"2023-11-09T03:47:01.587749+0000","last_unstale":"2023-11-09T03:47:50.783141+0000","last_undegraded":"2023-11-09T03:47:50.783141+0000","last_fullsized":"2023-11-09T03:47:50.783141+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063032099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.434012+0000","last_change":"2023-11-09T03:47:51.432431+0000","last_active":"2023-11-09T03:47:51.434012+0000","last_peered":"2023-11-09T03:47:51.434012+0000","last_clean":"2023-11-09T03:47:51.434012+0000","last_became_active":"2023-11-09T03:47:51.432164+0000","last_became_peered":"2023-11-09T03:47:51.432164+0000","last_unstale":"2023-11-09T03:47:51.434012+0000","last_undegraded":"2023-11-09T03:47:51.434012+0000","last_fullsized":"2023-11-09T03:47:51.434012+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:11:35.906808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"65'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.786911+0000","last_change":"2023-11-09T03:47:50.785328+0000","last_active":"2023-11-09T03:47:51.786911+0000","last_peered":"2023-11-09T03:47:51.786911+0000","last_clean":"2023-11-09T03:47:51.786911+0000","last_became_active":"2023-11-09T03:47:50.784280+0000","last_became_peered":"2023-11-09T03:47:50.784280+0000","last_unstale":"2023-11-09T03:47:51.786911+0000","last_undegraded":"2023-11-09T03:47:51.786911+0000","last_fullsized":"2023-11-09T03:47:51.786911+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:05:00.809894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.771140+0000","last_change":"2023-11-09T03:47:02.581658+0000","last_active":"2023-11-09T03:47:50.771140+0000","last_peered":"2023-11-09T03:47:50.771140+0000","last_clean":"2023-11-09T03:47:50.771140+0000","last_became_active":"2023-11-09T03:47:01.588612+0000","last_became_peered":"2023-11-09T03:47:01.588612+0000","last_unstale":"2023-11-09T03:47:50.771140+0000","last_undegraded":"2023-11-09T03:47:50.771140+0000","last_fullsized":"2023-11-09T03:47:50.771140+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045238500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.425900+0000","last_change":"2023-11-09T03:47:51.424216+0000","last_active":"2023-11-09T03:47:51.425900+0000","last_peered":"2023-11-09T03:47:51.425900+0000","last_clean":"2023-11-09T03:47:51.425900+0000","last_became_active":"2023-11-09T03:47:51.423782+0000","last_became_peered":"2023-11-09T03:47:51.423782+0000","last_unstale":"2023-11-09T03:47:51.425900+0000","last_undegraded":"2023-11-09T03:47:51.425900+0000","last_fullsized":"2023-11-09T03:47:51.425900+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:49:44.098168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"65'6","reported_seq":17,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.792705+0000","last_change":"2023-11-09T03:47:50.783046+0000","last_active":"2023-11-09T03:47:51.792705+0000","last_peered":"2023-11-09T03:47:51.792705+0000","last_clean":"2023-11-09T03:47:51.792705+0000","last_became_active":"2023-11-09T03:47:50.782715+0000","last_became_peered":"2023-11-09T03:47:50.782715+0000","last_unstale":"2023-11-09T03:47:51.792705+0000","last_undegraded":"2023-11-09T03:47:51.792705+0000","last_fullsized":"2023-11-09T03:47:51.792705+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:14:58.280559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.781573+0000","last_change":"2023-11-09T03:47:02.618828+0000","last_active":"2023-11-09T03:47:50.781573+0000","last_peered":"2023-11-09T03:47:50.781573+0000","last_clean":"2023-11-09T03:47:50.781573+0000","last_became_active":"2023-11-09T03:47:01.590679+0000","last_became_peered":"2023-11-09T03:47:01.590679+0000","last_unstale":"2023-11-09T03:47:50.781573+0000","last_undegraded":"2023-11-09T03:47:50.781573+0000","last_fullsized":"2023-11-09T03:47:50.781573+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:06:30.982987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070668,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.435787+0000","last_change":"2023-11-09T03:47:51.434706+0000","last_active":"2023-11-09T03:47:51.435787+0000","last_peered":"2023-11-09T03:47:51.435787+0000","last_clean":"2023-11-09T03:47:51.435787+0000","last_became_active":"2023-11-09T03:47:51.434251+0000","last_became_peered":"2023-11-09T03:47:51.434251+0000","last_unstale":"2023-11-09T03:47:51.435787+0000","last_undegraded":"2023-11-09T03:47:51.435787+0000","last_fullsized":"2023-11-09T03:47:51.435787+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:14:31.570502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"65'3","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.786663+0000","last_change":"2023-11-09T03:47:50.777327+0000","last_active":"2023-11-09T03:47:51.786663+0000","last_peered":"2023-11-09T03:47:51.786663+0000","last_clean":"2023-11-09T03:47:51.786663+0000","last_became_active":"2023-11-09T03:47:50.777076+0000","last_became_peered":"2023-11-09T03:47:50.777076+0000","last_unstale":"2023-11-09T03:47:51.786663+0000","last_undegraded":"2023-11-09T03:47:51.786663+0000","last_fullsized":"2023-11-09T03:47:51.786663+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:53:33.101892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":34,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.410485+0000","last_change":"2023-11-09T03:47:03.146528+0000","last_active":"2023-11-09T03:47:51.410485+0000","last_peered":"2023-11-09T03:47:51.410485+0000","last_clean":"2023-11-09T03:47:51.410485+0000","last_became_active":"2023-11-09T03:47:01.587060+0000","last_became_peered":"2023-11-09T03:47:01.587060+0000","last_unstale":"2023-11-09T03:47:51.410485+0000","last_undegraded":"2023-11-09T03:47:51.410485+0000","last_fullsized":"2023-11-09T03:47:51.410485+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:39:15.238496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079064999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.433820+0000","last_change":"2023-11-09T03:47:51.433039+0000","last_active":"2023-11-09T03:47:51.433820+0000","last_peered":"2023-11-09T03:47:51.433820+0000","last_clean":"2023-11-09T03:47:51.433820+0000","last_became_active":"2023-11-09T03:47:51.432780+0000","last_became_peered":"2023-11-09T03:47:51.432780+0000","last_unstale":"2023-11-09T03:47:51.433820+0000","last_undegraded":"2023-11-09T03:47:51.433820+0000","last_fullsized":"2023-11-09T03:47:51.433820+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:42:31.986727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.419179+0000","last_change":"2023-11-09T03:47:50.785375+0000","last_active":"2023-11-09T03:47:51.419179+0000","last_peered":"2023-11-09T03:47:51.419179+0000","last_clean":"2023-11-09T03:47:51.419179+0000","last_became_active":"2023-11-09T03:47:50.785090+0000","last_became_peered":"2023-11-09T03:47:50.785090+0000","last_unstale":"2023-11-09T03:47:51.419179+0000","last_undegraded":"2023-11-09T03:47:51.419179+0000","last_fullsized":"2023-11-09T03:47:51.419179+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:53:58.326231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":34,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.410680+0000","last_change":"2023-11-09T03:47:03.148246+0000","last_active":"2023-11-09T03:47:51.410680+0000","last_peered":"2023-11-09T03:47:51.410680+0000","last_clean":"2023-11-09T03:47:51.410680+0000","last_became_active":"2023-11-09T03:47:01.590988+0000","last_became_peered":"2023-11-09T03:47:01.590988+0000","last_unstale":"2023-11-09T03:47:51.410680+0000","last_undegraded":"2023-11-09T03:47:51.410680+0000","last_fullsized":"2023-11-09T03:47:51.410680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:07:20.785445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055651099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.434893+0000","last_change":"2023-11-09T03:47:51.433756+0000","last_active":"2023-11-09T03:47:51.434893+0000","last_peered":"2023-11-09T03:47:51.434893+0000","last_clean":"2023-11-09T03:47:51.434893+0000","last_became_active":"2023-11-09T03:47:51.432762+0000","last_became_peered":"2023-11-09T03:47:51.432762+0000","last_unstale":"2023-11-09T03:47:51.434893+0000","last_undegraded":"2023-11-09T03:47:51.434893+0000","last_fullsized":"2023-11-09T03:47:51.434893+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:21:20.038282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.413342+0000","last_change":"2023-11-09T03:47:50.780186+0000","last_active":"2023-11-09T03:47:52.413342+0000","last_peered":"2023-11-09T03:47:52.413342+0000","last_clean":"2023-11-09T03:47:52.413342+0000","last_became_active":"2023-11-09T03:47:50.779756+0000","last_became_peered":"2023-11-09T03:47:50.779756+0000","last_unstale":"2023-11-09T03:47:52.413342+0000","last_undegraded":"2023-11-09T03:47:52.413342+0000","last_fullsized":"2023-11-09T03:47:52.413342+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:41:14.757414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":36,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.413429+0000","last_change":"2023-11-09T03:47:02.582754+0000","last_active":"2023-11-09T03:47:51.413429+0000","last_peered":"2023-11-09T03:47:51.413429+0000","last_clean":"2023-11-09T03:47:51.413429+0000","last_became_active":"2023-11-09T03:47:01.589715+0000","last_became_peered":"2023-11-09T03:47:01.589715+0000","last_unstale":"2023-11-09T03:47:51.413429+0000","last_undegraded":"2023-11-09T03:47:51.413429+0000","last_fullsized":"2023-11-09T03:47:51.413429+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099342799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.414558+0000","last_change":"2023-11-09T03:47:51.421718+0000","last_active":"2023-11-09T03:47:52.414558+0000","last_peered":"2023-11-09T03:47:52.414558+0000","last_clean":"2023-11-09T03:47:52.414558+0000","last_became_active":"2023-11-09T03:47:51.421451+0000","last_became_peered":"2023-11-09T03:47:51.421451+0000","last_unstale":"2023-11-09T03:47:52.414558+0000","last_undegraded":"2023-11-09T03:47:52.414558+0000","last_fullsized":"2023-11-09T03:47:52.414558+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:01:59.285602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.413736+0000","last_change":"2023-11-09T03:47:50.779901+0000","last_active":"2023-11-09T03:47:51.413736+0000","last_peered":"2023-11-09T03:47:51.413736+0000","last_clean":"2023-11-09T03:47:51.413736+0000","last_became_active":"2023-11-09T03:47:50.778699+0000","last_became_peered":"2023-11-09T03:47:50.778699+0000","last_unstale":"2023-11-09T03:47:51.413736+0000","last_undegraded":"2023-11-09T03:47:51.413736+0000","last_fullsized":"2023-11-09T03:47:51.413736+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:48:57.422120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.774628+0000","last_change":"2023-11-09T03:47:02.724896+0000","last_active":"2023-11-09T03:47:50.774628+0000","last_peered":"2023-11-09T03:47:50.774628+0000","last_clean":"2023-11-09T03:47:50.774628+0000","last_became_active":"2023-11-09T03:47:01.592784+0000","last_became_peered":"2023-11-09T03:47:01.592784+0000","last_unstale":"2023-11-09T03:47:50.774628+0000","last_undegraded":"2023-11-09T03:47:50.774628+0000","last_fullsized":"2023-11-09T03:47:50.774628+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:51:32.016204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056683500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.432861+0000","last_change":"2023-11-09T03:47:51.430153+0000","last_active":"2023-11-09T03:47:51.432861+0000","last_peered":"2023-11-09T03:47:51.432861+0000","last_clean":"2023-11-09T03:47:51.432861+0000","last_became_active":"2023-11-09T03:47:51.429887+0000","last_became_peered":"2023-11-09T03:47:51.429887+0000","last_unstale":"2023-11-09T03:47:51.432861+0000","last_undegraded":"2023-11-09T03:47:51.432861+0000","last_fullsized":"2023-11-09T03:47:51.432861+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:17:04.496946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.413110+0000","last_change":"2023-11-09T03:47:50.785443+0000","last_active":"2023-11-09T03:47:51.413110+0000","last_peered":"2023-11-09T03:47:51.413110+0000","last_clean":"2023-11-09T03:47:51.413110+0000","last_became_active":"2023-11-09T03:47:50.784798+0000","last_became_peered":"2023-11-09T03:47:50.784798+0000","last_unstale":"2023-11-09T03:47:51.413110+0000","last_undegraded":"2023-11-09T03:47:51.413110+0000","last_fullsized":"2023-11-09T03:47:51.413110+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:59:00.398111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":35,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.765459+0000","last_change":"2023-11-09T03:47:03.146527+0000","last_active":"2023-11-09T03:47:50.765459+0000","last_peered":"2023-11-09T03:47:50.765459+0000","last_clean":"2023-11-09T03:47:50.765459+0000","last_became_active":"2023-11-09T03:47:01.587858+0000","last_became_peered":"2023-11-09T03:47:01.587858+0000","last_unstale":"2023-11-09T03:47:50.765459+0000","last_undegraded":"2023-11-09T03:47:50.765459+0000","last_fullsized":"2023-11-09T03:47:50.765459+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:56:04.201126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084439599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.413634+0000","last_change":"2023-11-09T03:47:51.424215+0000","last_active":"2023-11-09T03:47:52.413634+0000","last_peered":"2023-11-09T03:47:52.413634+0000","last_clean":"2023-11-09T03:47:52.413634+0000","last_became_active":"2023-11-09T03:47:51.423001+0000","last_became_peered":"2023-11-09T03:47:51.423001+0000","last_unstale":"2023-11-09T03:47:52.413634+0000","last_undegraded":"2023-11-09T03:47:52.413634+0000","last_fullsized":"2023-11-09T03:47:52.413634+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:13:45.340453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"65'1","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.785920+0000","last_change":"2023-11-09T03:47:50.782111+0000","last_active":"2023-11-09T03:47:51.785920+0000","last_peered":"2023-11-09T03:47:51.785920+0000","last_clean":"2023-11-09T03:47:51.785920+0000","last_became_active":"2023-11-09T03:47:50.781833+0000","last_became_peered":"2023-11-09T03:47:50.781833+0000","last_unstale":"2023-11-09T03:47:51.785920+0000","last_undegraded":"2023-11-09T03:47:51.785920+0000","last_fullsized":"2023-11-09T03:47:51.785920+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:04:35.154546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.776045+0000","last_change":"2023-11-09T03:47:02.725223+0000","last_active":"2023-11-09T03:47:50.776045+0000","last_peered":"2023-11-09T03:47:50.776045+0000","last_clean":"2023-11-09T03:47:50.776045+0000","last_became_active":"2023-11-09T03:47:01.590635+0000","last_became_peered":"2023-11-09T03:47:01.590635+0000","last_unstale":"2023-11-09T03:47:50.776045+0000","last_undegraded":"2023-11-09T03:47:50.776045+0000","last_fullsized":"2023-11-09T03:47:50.776045+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:05.754357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051685100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.428631+0000","last_change":"2023-11-09T03:47:51.425520+0000","last_active":"2023-11-09T03:47:51.428631+0000","last_peered":"2023-11-09T03:47:51.428631+0000","last_clean":"2023-11-09T03:47:51.428631+0000","last_became_active":"2023-11-09T03:47:51.425239+0000","last_became_peered":"2023-11-09T03:47:51.425239+0000","last_unstale":"2023-11-09T03:47:51.428631+0000","last_undegraded":"2023-11-09T03:47:51.428631+0000","last_fullsized":"2023-11-09T03:47:51.428631+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:11:35.906808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"65'1","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.785787+0000","last_change":"2023-11-09T03:47:50.778115+0000","last_active":"2023-11-09T03:47:51.785787+0000","last_peered":"2023-11-09T03:47:51.785787+0000","last_clean":"2023-11-09T03:47:51.785787+0000","last_became_active":"2023-11-09T03:47:50.777666+0000","last_became_peered":"2023-11-09T03:47:50.777666+0000","last_unstale":"2023-11-09T03:47:51.785787+0000","last_undegraded":"2023-11-09T03:47:51.785787+0000","last_fullsized":"2023-11-09T03:47:51.785787+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:19:44.308319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":36,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.414324+0000","last_change":"2023-11-09T03:47:02.582739+0000","last_active":"2023-11-09T03:47:51.414324+0000","last_peered":"2023-11-09T03:47:51.414324+0000","last_clean":"2023-11-09T03:47:51.414324+0000","last_became_active":"2023-11-09T03:47:01.592743+0000","last_became_peered":"2023-11-09T03:47:01.592743+0000","last_unstale":"2023-11-09T03:47:51.414324+0000","last_undegraded":"2023-11-09T03:47:51.414324+0000","last_fullsized":"2023-11-09T03:47:51.414324+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051802700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.430708+0000","last_change":"2023-11-09T03:47:51.425934+0000","last_active":"2023-11-09T03:47:51.430708+0000","last_peered":"2023-11-09T03:47:51.430708+0000","last_clean":"2023-11-09T03:47:51.430708+0000","last_became_active":"2023-11-09T03:47:51.425667+0000","last_became_peered":"2023-11-09T03:47:51.425667+0000","last_unstale":"2023-11-09T03:47:51.430708+0000","last_undegraded":"2023-11-09T03:47:51.430708+0000","last_fullsized":"2023-11-09T03:47:51.430708+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:32:30.681190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"65'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.786124+0000","last_change":"2023-11-09T03:47:50.783099+0000","last_active":"2023-11-09T03:47:51.786124+0000","last_peered":"2023-11-09T03:47:51.786124+0000","last_clean":"2023-11-09T03:47:51.786124+0000","last_became_active":"2023-11-09T03:47:50.782082+0000","last_became_peered":"2023-11-09T03:47:50.782082+0000","last_unstale":"2023-11-09T03:47:51.786124+0000","last_undegraded":"2023-11-09T03:47:51.786124+0000","last_fullsized":"2023-11-09T03:47:51.786124+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:04:48.179843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.773010+0000","last_change":"2023-11-09T03:47:02.724953+0000","last_active":"2023-11-09T03:47:50.773010+0000","last_peered":"2023-11-09T03:47:50.773010+0000","last_clean":"2023-11-09T03:47:50.773010+0000","last_became_active":"2023-11-09T03:47:01.587133+0000","last_became_peered":"2023-11-09T03:47:01.587133+0000","last_unstale":"2023-11-09T03:47:50.773010+0000","last_undegraded":"2023-11-09T03:47:50.773010+0000","last_fullsized":"2023-11-09T03:47:50.773010+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:23:48.521161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066497599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.438370+0000","last_change":"2023-11-09T03:47:51.436760+0000","last_active":"2023-11-09T03:47:51.438370+0000","last_peered":"2023-11-09T03:47:51.438370+0000","last_clean":"2023-11-09T03:47:51.438370+0000","last_became_active":"2023-11-09T03:47:51.436265+0000","last_became_peered":"2023-11-09T03:47:51.436265+0000","last_unstale":"2023-11-09T03:47:51.438370+0000","last_undegraded":"2023-11-09T03:47:51.438370+0000","last_fullsized":"2023-11-09T03:47:51.438370+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:57:53.737390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"65'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.413573+0000","last_change":"2023-11-09T03:47:50.786349+0000","last_active":"2023-11-09T03:47:52.413573+0000","last_peered":"2023-11-09T03:47:52.413573+0000","last_clean":"2023-11-09T03:47:52.413573+0000","last_became_active":"2023-11-09T03:47:50.786079+0000","last_became_peered":"2023-11-09T03:47:50.786079+0000","last_unstale":"2023-11-09T03:47:52.413573+0000","last_undegraded":"2023-11-09T03:47:52.413573+0000","last_fullsized":"2023-11-09T03:47:52.413573+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:56:24.172800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":35,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.765230+0000","last_change":"2023-11-09T03:47:03.146217+0000","last_active":"2023-11-09T03:47:50.765230+0000","last_peered":"2023-11-09T03:47:50.765230+0000","last_clean":"2023-11-09T03:47:50.765230+0000","last_became_active":"2023-11-09T03:47:01.587598+0000","last_became_peered":"2023-11-09T03:47:01.587598+0000","last_unstale":"2023-11-09T03:47:50.765230+0000","last_undegraded":"2023-11-09T03:47:50.765230+0000","last_fullsized":"2023-11-09T03:47:50.765230+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:15:22.723455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090126400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.423912+0000","last_change":"2023-11-09T03:47:51.421511+0000","last_active":"2023-11-09T03:47:51.423912+0000","last_peered":"2023-11-09T03:47:51.423912+0000","last_clean":"2023-11-09T03:47:51.423912+0000","last_became_active":"2023-11-09T03:47:51.421262+0000","last_became_peered":"2023-11-09T03:47:51.421262+0000","last_unstale":"2023-11-09T03:47:51.423912+0000","last_undegraded":"2023-11-09T03:47:51.423912+0000","last_fullsized":"2023-11-09T03:47:51.423912+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:38:40.522966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"65'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.414900+0000","last_change":"2023-11-09T03:47:50.783271+0000","last_active":"2023-11-09T03:47:52.414900+0000","last_peered":"2023-11-09T03:47:52.414900+0000","last_clean":"2023-11-09T03:47:52.414900+0000","last_became_active":"2023-11-09T03:47:50.782546+0000","last_became_peered":"2023-11-09T03:47:50.782546+0000","last_unstale":"2023-11-09T03:47:52.414900+0000","last_undegraded":"2023-11-09T03:47:52.414900+0000","last_fullsized":"2023-11-09T03:47:52.414900+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:21:19.032046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":34,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.410784+0000","last_change":"2023-11-09T03:47:03.148283+0000","last_active":"2023-11-09T03:47:51.410784+0000","last_peered":"2023-11-09T03:47:51.410784+0000","last_clean":"2023-11-09T03:47:51.410784+0000","last_became_active":"2023-11-09T03:47:01.588841+0000","last_became_peered":"2023-11-09T03:47:01.588841+0000","last_unstale":"2023-11-09T03:47:51.410784+0000","last_undegraded":"2023-11-09T03:47:51.410784+0000","last_fullsized":"2023-11-09T03:47:51.410784+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:47:44.880741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000436081,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.432369+0000","last_change":"2023-11-09T03:47:51.430694+0000","last_active":"2023-11-09T03:47:51.432369+0000","last_peered":"2023-11-09T03:47:51.432369+0000","last_clean":"2023-11-09T03:47:51.432369+0000","last_became_active":"2023-11-09T03:47:51.430302+0000","last_became_peered":"2023-11-09T03:47:51.430302+0000","last_unstale":"2023-11-09T03:47:51.432369+0000","last_undegraded":"2023-11-09T03:47:51.432369+0000","last_fullsized":"2023-11-09T03:47:51.432369+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:11:17.062291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"65'4","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.789961+0000","last_change":"2023-11-09T03:47:50.783769+0000","last_active":"2023-11-09T03:47:51.789961+0000","last_peered":"2023-11-09T03:47:51.789961+0000","last_clean":"2023-11-09T03:47:51.789961+0000","last_became_active":"2023-11-09T03:47:50.783497+0000","last_became_peered":"2023-11-09T03:47:50.783497+0000","last_unstale":"2023-11-09T03:47:51.789961+0000","last_undegraded":"2023-11-09T03:47:51.789961+0000","last_fullsized":"2023-11-09T03:47:51.789961+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:36:59.189560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.781364+0000","last_change":"2023-11-09T03:47:02.618856+0000","last_active":"2023-11-09T03:47:50.781364+0000","last_peered":"2023-11-09T03:47:50.781364+0000","last_clean":"2023-11-09T03:47:50.781364+0000","last_became_active":"2023-11-09T03:47:01.590430+0000","last_became_peered":"2023-11-09T03:47:01.590430+0000","last_unstale":"2023-11-09T03:47:50.781364+0000","last_undegraded":"2023-11-09T03:47:50.781364+0000","last_fullsized":"2023-11-09T03:47:50.781364+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:57:18.470305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055030200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.426831+0000","last_change":"2023-11-09T03:47:51.425418+0000","last_active":"2023-11-09T03:47:51.426831+0000","last_peered":"2023-11-09T03:47:51.426831+0000","last_clean":"2023-11-09T03:47:51.426831+0000","last_became_active":"2023-11-09T03:47:51.425164+0000","last_became_peered":"2023-11-09T03:47:51.425164+0000","last_unstale":"2023-11-09T03:47:51.426831+0000","last_undegraded":"2023-11-09T03:47:51.426831+0000","last_fullsized":"2023-11-09T03:47:51.426831+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:06:03.508296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.422207+0000","last_change":"2023-11-09T03:47:50.786002+0000","last_active":"2023-11-09T03:47:51.422207+0000","last_peered":"2023-11-09T03:47:51.422207+0000","last_clean":"2023-11-09T03:47:51.422207+0000","last_became_active":"2023-11-09T03:47:50.785336+0000","last_became_peered":"2023-11-09T03:47:50.785336+0000","last_unstale":"2023-11-09T03:47:51.422207+0000","last_undegraded":"2023-11-09T03:47:51.422207+0000","last_fullsized":"2023-11-09T03:47:51.422207+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:00:04.578075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.781812+0000","last_change":"2023-11-09T03:47:02.619144+0000","last_active":"2023-11-09T03:47:50.781812+0000","last_peered":"2023-11-09T03:47:50.781812+0000","last_clean":"2023-11-09T03:47:50.781812+0000","last_became_active":"2023-11-09T03:47:01.590672+0000","last_became_peered":"2023-11-09T03:47:01.590672+0000","last_unstale":"2023-11-09T03:47:50.781812+0000","last_undegraded":"2023-11-09T03:47:50.781812+0000","last_fullsized":"2023-11-09T03:47:50.781812+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:03.025119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060997500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.426561+0000","last_change":"2023-11-09T03:47:51.423417+0000","last_active":"2023-11-09T03:47:51.426561+0000","last_peered":"2023-11-09T03:47:51.426561+0000","last_clean":"2023-11-09T03:47:51.426561+0000","last_became_active":"2023-11-09T03:47:51.423176+0000","last_became_peered":"2023-11-09T03:47:51.423176+0000","last_unstale":"2023-11-09T03:47:51.426561+0000","last_undegraded":"2023-11-09T03:47:51.426561+0000","last_fullsized":"2023-11-09T03:47:51.426561+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:57:53.737390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.413416+0000","last_change":"2023-11-09T03:47:50.775788+0000","last_active":"2023-11-09T03:47:52.413416+0000","last_peered":"2023-11-09T03:47:52.413416+0000","last_clean":"2023-11-09T03:47:52.413416+0000","last_became_active":"2023-11-09T03:47:50.775509+0000","last_became_peered":"2023-11-09T03:47:50.775509+0000","last_unstale":"2023-11-09T03:47:52.413416+0000","last_undegraded":"2023-11-09T03:47:52.413416+0000","last_fullsized":"2023-11-09T03:47:52.413416+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:06:41.516591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.781724+0000","last_change":"2023-11-09T03:47:02.619079+0000","last_active":"2023-11-09T03:47:50.781724+0000","last_peered":"2023-11-09T03:47:50.781724+0000","last_clean":"2023-11-09T03:47:50.781724+0000","last_became_active":"2023-11-09T03:47:01.591421+0000","last_became_peered":"2023-11-09T03:47:01.591421+0000","last_unstale":"2023-11-09T03:47:50.781724+0000","last_undegraded":"2023-11-09T03:47:50.781724+0000","last_fullsized":"2023-11-09T03:47:50.781724+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:18:58.230606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068736399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.435687+0000","last_change":"2023-11-09T03:47:51.433858+0000","last_active":"2023-11-09T03:47:51.435687+0000","last_peered":"2023-11-09T03:47:51.435687+0000","last_clean":"2023-11-09T03:47:51.435687+0000","last_became_active":"2023-11-09T03:47:51.433561+0000","last_became_peered":"2023-11-09T03:47:51.433561+0000","last_unstale":"2023-11-09T03:47:51.435687+0000","last_undegraded":"2023-11-09T03:47:51.435687+0000","last_fullsized":"2023-11-09T03:47:51.435687+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:07:03.975863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.413057+0000","last_change":"2023-11-09T03:47:50.785136+0000","last_active":"2023-11-09T03:47:51.413057+0000","last_peered":"2023-11-09T03:47:51.413057+0000","last_clean":"2023-11-09T03:47:51.413057+0000","last_became_active":"2023-11-09T03:47:50.784764+0000","last_became_peered":"2023-11-09T03:47:50.784764+0000","last_unstale":"2023-11-09T03:47:51.413057+0000","last_undegraded":"2023-11-09T03:47:51.413057+0000","last_fullsized":"2023-11-09T03:47:51.413057+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:17.964253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":34,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.411240+0000","last_change":"2023-11-09T03:47:03.148529+0000","last_active":"2023-11-09T03:47:51.411240+0000","last_peered":"2023-11-09T03:47:51.411240+0000","last_clean":"2023-11-09T03:47:51.411240+0000","last_became_active":"2023-11-09T03:47:01.584148+0000","last_became_peered":"2023-11-09T03:47:01.584148+0000","last_unstale":"2023-11-09T03:47:51.411240+0000","last_undegraded":"2023-11-09T03:47:51.411240+0000","last_fullsized":"2023-11-09T03:47:51.411240+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:40:00.237248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061475600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.424192+0000","last_change":"2023-11-09T03:47:51.422333+0000","last_active":"2023-11-09T03:47:51.424192+0000","last_peered":"2023-11-09T03:47:51.424192+0000","last_clean":"2023-11-09T03:47:51.424192+0000","last_became_active":"2023-11-09T03:47:51.421811+0000","last_became_peered":"2023-11-09T03:47:51.421811+0000","last_unstale":"2023-11-09T03:47:51.424192+0000","last_undegraded":"2023-11-09T03:47:51.424192+0000","last_fullsized":"2023-11-09T03:47:51.424192+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:05:37.375241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.415322+0000","last_change":"2023-11-09T03:47:50.782177+0000","last_active":"2023-11-09T03:47:51.415322+0000","last_peered":"2023-11-09T03:47:51.415322+0000","last_clean":"2023-11-09T03:47:51.415322+0000","last_became_active":"2023-11-09T03:47:50.781308+0000","last_became_peered":"2023-11-09T03:47:50.781308+0000","last_unstale":"2023-11-09T03:47:51.415322+0000","last_undegraded":"2023-11-09T03:47:51.415322+0000","last_fullsized":"2023-11-09T03:47:51.415322+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:43:39.503427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.771407+0000","last_change":"2023-11-09T03:47:03.148217+0000","last_active":"2023-11-09T03:47:50.771407+0000","last_peered":"2023-11-09T03:47:50.771407+0000","last_clean":"2023-11-09T03:47:50.771407+0000","last_became_active":"2023-11-09T03:47:01.593424+0000","last_became_peered":"2023-11-09T03:47:01.593424+0000","last_unstale":"2023-11-09T03:47:50.771407+0000","last_undegraded":"2023-11-09T03:47:50.771407+0000","last_fullsized":"2023-11-09T03:47:50.771407+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:19:32.693691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001769865,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.430726+0000","last_change":"2023-11-09T03:47:51.427859+0000","last_active":"2023-11-09T03:47:51.430726+0000","last_peered":"2023-11-09T03:47:51.430726+0000","last_clean":"2023-11-09T03:47:51.430726+0000","last_became_active":"2023-11-09T03:47:51.427608+0000","last_became_peered":"2023-11-09T03:47:51.427608+0000","last_unstale":"2023-11-09T03:47:51.430726+0000","last_undegraded":"2023-11-09T03:47:51.430726+0000","last_fullsized":"2023-11-09T03:47:51.430726+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:01:59.285602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.414730+0000","last_change":"2023-11-09T03:47:50.780364+0000","last_active":"2023-11-09T03:47:52.414730+0000","last_peered":"2023-11-09T03:47:52.414730+0000","last_clean":"2023-11-09T03:47:52.414730+0000","last_became_active":"2023-11-09T03:47:50.780015+0000","last_became_peered":"2023-11-09T03:47:50.780015+0000","last_unstale":"2023-11-09T03:47:52.414730+0000","last_undegraded":"2023-11-09T03:47:52.414730+0000","last_fullsized":"2023-11-09T03:47:52.414730+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:38:51.283678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":34,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.411140+0000","last_change":"2023-11-09T03:47:03.146470+0000","last_active":"2023-11-09T03:47:51.411140+0000","last_peered":"2023-11-09T03:47:51.411140+0000","last_clean":"2023-11-09T03:47:51.411140+0000","last_became_active":"2023-11-09T03:47:01.587304+0000","last_became_peered":"2023-11-09T03:47:01.587304+0000","last_unstale":"2023-11-09T03:47:51.411140+0000","last_undegraded":"2023-11-09T03:47:51.411140+0000","last_fullsized":"2023-11-09T03:47:51.411140+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:58:23.339287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00110395,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.437478+0000","last_change":"2023-11-09T03:47:51.435550+0000","last_active":"2023-11-09T03:47:51.437478+0000","last_peered":"2023-11-09T03:47:51.437478+0000","last_clean":"2023-11-09T03:47:51.437478+0000","last_became_active":"2023-11-09T03:47:51.435197+0000","last_became_peered":"2023-11-09T03:47:51.435197+0000","last_unstale":"2023-11-09T03:47:51.437478+0000","last_undegraded":"2023-11-09T03:47:51.437478+0000","last_fullsized":"2023-11-09T03:47:51.437478+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:52:03.847868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.415203+0000","last_change":"2023-11-09T03:47:50.782930+0000","last_active":"2023-11-09T03:47:51.415203+0000","last_peered":"2023-11-09T03:47:51.415203+0000","last_clean":"2023-11-09T03:47:51.415203+0000","last_became_active":"2023-11-09T03:47:50.782659+0000","last_became_peered":"2023-11-09T03:47:50.782659+0000","last_unstale":"2023-11-09T03:47:51.415203+0000","last_undegraded":"2023-11-09T03:47:51.415203+0000","last_fullsized":"2023-11-09T03:47:51.415203+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:47:24.099855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.770672+0000","last_change":"2023-11-09T03:47:03.147367+0000","last_active":"2023-11-09T03:47:50.770672+0000","last_peered":"2023-11-09T03:47:50.770672+0000","last_clean":"2023-11-09T03:47:50.770672+0000","last_became_active":"2023-11-09T03:47:01.588359+0000","last_became_peered":"2023-11-09T03:47:01.588359+0000","last_unstale":"2023-11-09T03:47:50.770672+0000","last_undegraded":"2023-11-09T03:47:50.770672+0000","last_fullsized":"2023-11-09T03:47:50.770672+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:21:34.303068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014106660000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.431545+0000","last_change":"2023-11-09T03:47:51.428133+0000","last_active":"2023-11-09T03:47:51.431545+0000","last_peered":"2023-11-09T03:47:51.431545+0000","last_clean":"2023-11-09T03:47:51.431545+0000","last_became_active":"2023-11-09T03:47:51.427786+0000","last_became_peered":"2023-11-09T03:47:51.427786+0000","last_unstale":"2023-11-09T03:47:51.431545+0000","last_undegraded":"2023-11-09T03:47:51.431545+0000","last_fullsized":"2023-11-09T03:47:51.431545+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:52:03.847868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.422000+0000","last_change":"2023-11-09T03:47:50.781009+0000","last_active":"2023-11-09T03:47:51.422000+0000","last_peered":"2023-11-09T03:47:51.422000+0000","last_clean":"2023-11-09T03:47:51.422000+0000","last_became_active":"2023-11-09T03:47:50.780172+0000","last_became_peered":"2023-11-09T03:47:50.780172+0000","last_unstale":"2023-11-09T03:47:51.422000+0000","last_undegraded":"2023-11-09T03:47:51.422000+0000","last_fullsized":"2023-11-09T03:47:51.422000+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:01:58.279366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.774013+0000","last_change":"2023-11-09T03:47:02.581682+0000","last_active":"2023-11-09T03:47:50.774013+0000","last_peered":"2023-11-09T03:47:50.774013+0000","last_clean":"2023-11-09T03:47:50.774013+0000","last_became_active":"2023-11-09T03:47:01.593051+0000","last_became_peered":"2023-11-09T03:47:01.593051+0000","last_unstale":"2023-11-09T03:47:50.774013+0000","last_undegraded":"2023-11-09T03:47:50.774013+0000","last_fullsized":"2023-11-09T03:47:50.774013+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047277799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.414917+0000","last_change":"2023-11-09T03:47:51.419244+0000","last_active":"2023-11-09T03:47:52.414917+0000","last_peered":"2023-11-09T03:47:52.414917+0000","last_clean":"2023-11-09T03:47:52.414917+0000","last_became_active":"2023-11-09T03:47:51.418983+0000","last_became_peered":"2023-11-09T03:47:51.418983+0000","last_unstale":"2023-11-09T03:47:52.414917+0000","last_undegraded":"2023-11-09T03:47:52.414917+0000","last_fullsized":"2023-11-09T03:47:52.414917+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:06:27.135244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"65'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.786082+0000","last_change":"2023-11-09T03:47:50.778993+0000","last_active":"2023-11-09T03:47:51.786082+0000","last_peered":"2023-11-09T03:47:51.786082+0000","last_clean":"2023-11-09T03:47:51.786082+0000","last_became_active":"2023-11-09T03:47:50.778713+0000","last_became_peered":"2023-11-09T03:47:50.778713+0000","last_unstale":"2023-11-09T03:47:51.786082+0000","last_undegraded":"2023-11-09T03:47:51.786082+0000","last_fullsized":"2023-11-09T03:47:51.786082+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:53:33.810634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":35,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.765274+0000","last_change":"2023-11-09T03:47:03.146309+0000","last_active":"2023-11-09T03:47:50.765274+0000","last_peered":"2023-11-09T03:47:50.765274+0000","last_clean":"2023-11-09T03:47:50.765274+0000","last_became_active":"2023-11-09T03:47:01.587560+0000","last_became_peered":"2023-11-09T03:47:01.587560+0000","last_unstale":"2023-11-09T03:47:50.765274+0000","last_undegraded":"2023-11-09T03:47:50.765274+0000","last_fullsized":"2023-11-09T03:47:50.765274+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:27:55.271945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070988400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.428243+0000","last_change":"2023-11-09T03:47:51.426419+0000","last_active":"2023-11-09T03:47:51.428243+0000","last_peered":"2023-11-09T03:47:51.428243+0000","last_clean":"2023-11-09T03:47:51.428243+0000","last_became_active":"2023-11-09T03:47:51.426178+0000","last_became_peered":"2023-11-09T03:47:51.426178+0000","last_unstale":"2023-11-09T03:47:51.428243+0000","last_undegraded":"2023-11-09T03:47:51.428243+0000","last_fullsized":"2023-11-09T03:47:51.428243+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:51:09.098748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.414443+0000","last_change":"2023-11-09T03:47:50.783293+0000","last_active":"2023-11-09T03:47:52.414443+0000","last_peered":"2023-11-09T03:47:52.414443+0000","last_clean":"2023-11-09T03:47:52.414443+0000","last_became_active":"2023-11-09T03:47:50.782667+0000","last_became_peered":"2023-11-09T03:47:50.782667+0000","last_unstale":"2023-11-09T03:47:52.414443+0000","last_undegraded":"2023-11-09T03:47:52.414443+0000","last_fullsized":"2023-11-09T03:47:52.414443+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:11:34.900572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.774708+0000","last_change":"2023-11-09T03:47:02.725125+0000","last_active":"2023-11-09T03:47:50.774708+0000","last_peered":"2023-11-09T03:47:50.774708+0000","last_clean":"2023-11-09T03:47:50.774708+0000","last_became_active":"2023-11-09T03:47:01.591521+0000","last_became_peered":"2023-11-09T03:47:01.591521+0000","last_unstale":"2023-11-09T03:47:50.774708+0000","last_undegraded":"2023-11-09T03:47:50.774708+0000","last_fullsized":"2023-11-09T03:47:50.774708+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:52:40.076542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000758675,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.428252+0000","last_change":"2023-11-09T03:47:51.426821+0000","last_active":"2023-11-09T03:47:51.428252+0000","last_peered":"2023-11-09T03:47:51.428252+0000","last_clean":"2023-11-09T03:47:51.428252+0000","last_became_active":"2023-11-09T03:47:51.426516+0000","last_became_peered":"2023-11-09T03:47:51.426516+0000","last_unstale":"2023-11-09T03:47:51.428252+0000","last_undegraded":"2023-11-09T03:47:51.428252+0000","last_fullsized":"2023-11-09T03:47:51.428252+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:15:26.625990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.421152+0000","last_change":"2023-11-09T03:47:50.782705+0000","last_active":"2023-11-09T03:47:51.421152+0000","last_peered":"2023-11-09T03:47:51.421152+0000","last_clean":"2023-11-09T03:47:51.421152+0000","last_became_active":"2023-11-09T03:47:50.782427+0000","last_became_peered":"2023-11-09T03:47:50.782427+0000","last_unstale":"2023-11-09T03:47:51.421152+0000","last_undegraded":"2023-11-09T03:47:51.421152+0000","last_fullsized":"2023-11-09T03:47:51.421152+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:36:59.189560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":34,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.410970+0000","last_change":"2023-11-09T03:47:03.147289+0000","last_active":"2023-11-09T03:47:51.410970+0000","last_peered":"2023-11-09T03:47:51.410970+0000","last_clean":"2023-11-09T03:47:51.410970+0000","last_became_active":"2023-11-09T03:47:01.587784+0000","last_became_peered":"2023-11-09T03:47:01.587784+0000","last_unstale":"2023-11-09T03:47:51.410970+0000","last_undegraded":"2023-11-09T03:47:51.410970+0000","last_fullsized":"2023-11-09T03:47:51.410970+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:11:23.824112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038781299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.424282+0000","last_change":"2023-11-09T03:47:51.421317+0000","last_active":"2023-11-09T03:47:51.424282+0000","last_peered":"2023-11-09T03:47:51.424282+0000","last_clean":"2023-11-09T03:47:51.424282+0000","last_became_active":"2023-11-09T03:47:51.421044+0000","last_became_peered":"2023-11-09T03:47:51.421044+0000","last_unstale":"2023-11-09T03:47:51.424282+0000","last_undegraded":"2023-11-09T03:47:51.424282+0000","last_fullsized":"2023-11-09T03:47:51.424282+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:50:46.643929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.422774+0000","last_change":"2023-11-09T03:47:50.783770+0000","last_active":"2023-11-09T03:47:51.422774+0000","last_peered":"2023-11-09T03:47:51.422774+0000","last_clean":"2023-11-09T03:47:51.422774+0000","last_became_active":"2023-11-09T03:47:50.781901+0000","last_became_peered":"2023-11-09T03:47:50.781901+0000","last_unstale":"2023-11-09T03:47:51.422774+0000","last_undegraded":"2023-11-09T03:47:51.422774+0000","last_fullsized":"2023-11-09T03:47:51.422774+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:14:58.280559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.783243+0000","last_change":"2023-11-09T03:47:02.619346+0000","last_active":"2023-11-09T03:47:50.783243+0000","last_peered":"2023-11-09T03:47:50.783243+0000","last_clean":"2023-11-09T03:47:50.783243+0000","last_became_active":"2023-11-09T03:47:01.590389+0000","last_became_peered":"2023-11-09T03:47:01.590389+0000","last_unstale":"2023-11-09T03:47:50.783243+0000","last_undegraded":"2023-11-09T03:47:50.783243+0000","last_fullsized":"2023-11-09T03:47:50.783243+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000384072,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.435734+0000","last_change":"2023-11-09T03:47:51.431055+0000","last_active":"2023-11-09T03:47:51.435734+0000","last_peered":"2023-11-09T03:47:51.435734+0000","last_clean":"2023-11-09T03:47:51.435734+0000","last_became_active":"2023-11-09T03:47:51.430848+0000","last_became_peered":"2023-11-09T03:47:51.430848+0000","last_unstale":"2023-11-09T03:47:51.435734+0000","last_undegraded":"2023-11-09T03:47:51.435734+0000","last_fullsized":"2023-11-09T03:47:51.435734+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:02:44.463883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.416938+0000","last_change":"2023-11-09T03:47:50.783797+0000","last_active":"2023-11-09T03:47:51.416938+0000","last_peered":"2023-11-09T03:47:51.416938+0000","last_clean":"2023-11-09T03:47:51.416938+0000","last_became_active":"2023-11-09T03:47:50.782590+0000","last_became_peered":"2023-11-09T03:47:50.782590+0000","last_unstale":"2023-11-09T03:47:51.416938+0000","last_undegraded":"2023-11-09T03:47:51.416938+0000","last_fullsized":"2023-11-09T03:47:51.416938+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:00:16.605114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":36,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.413841+0000","last_change":"2023-11-09T03:47:02.582393+0000","last_active":"2023-11-09T03:47:51.413841+0000","last_peered":"2023-11-09T03:47:51.413841+0000","last_clean":"2023-11-09T03:47:51.413841+0000","last_became_active":"2023-11-09T03:47:01.591752+0000","last_became_peered":"2023-11-09T03:47:01.591752+0000","last_unstale":"2023-11-09T03:47:51.413841+0000","last_undegraded":"2023-11-09T03:47:51.413841+0000","last_fullsized":"2023-11-09T03:47:51.413841+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:03.025119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067021599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.424184+0000","last_change":"2023-11-09T03:47:51.422732+0000","last_active":"2023-11-09T03:47:51.424184+0000","last_peered":"2023-11-09T03:47:51.424184+0000","last_clean":"2023-11-09T03:47:51.424184+0000","last_became_active":"2023-11-09T03:47:51.422448+0000","last_became_peered":"2023-11-09T03:47:51.422448+0000","last_unstale":"2023-11-09T03:47:51.424184+0000","last_undegraded":"2023-11-09T03:47:51.424184+0000","last_fullsized":"2023-11-09T03:47:51.424184+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:04:49.186079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"65'1","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.786064+0000","last_change":"2023-11-09T03:47:50.785733+0000","last_active":"2023-11-09T03:47:51.786064+0000","last_peered":"2023-11-09T03:47:51.786064+0000","last_clean":"2023-11-09T03:47:51.786064+0000","last_became_active":"2023-11-09T03:47:50.785453+0000","last_became_peered":"2023-11-09T03:47:50.785453+0000","last_unstale":"2023-11-09T03:47:51.786064+0000","last_undegraded":"2023-11-09T03:47:51.786064+0000","last_fullsized":"2023-11-09T03:47:51.786064+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:24:20.075108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":33,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.772682+0000","last_change":"2023-11-09T03:47:03.028744+0000","last_active":"2023-11-09T03:47:50.772682+0000","last_peered":"2023-11-09T03:47:50.772682+0000","last_clean":"2023-11-09T03:47:50.772682+0000","last_became_active":"2023-11-09T03:47:01.593890+0000","last_became_peered":"2023-11-09T03:47:01.593890+0000","last_unstale":"2023-11-09T03:47:50.772682+0000","last_undegraded":"2023-11-09T03:47:50.772682+0000","last_fullsized":"2023-11-09T03:47:50.772682+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:05.754357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053008299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.431416+0000","last_change":"2023-11-09T03:47:51.428956+0000","last_active":"2023-11-09T03:47:51.431416+0000","last_peered":"2023-11-09T03:47:51.431416+0000","last_clean":"2023-11-09T03:47:51.431416+0000","last_became_active":"2023-11-09T03:47:51.428694+0000","last_became_peered":"2023-11-09T03:47:51.428694+0000","last_unstale":"2023-11-09T03:47:51.431416+0000","last_undegraded":"2023-11-09T03:47:51.431416+0000","last_fullsized":"2023-11-09T03:47:51.431416+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:45:00.860943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"65'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.413334+0000","last_change":"2023-11-09T03:47:50.786278+0000","last_active":"2023-11-09T03:47:52.413334+0000","last_peered":"2023-11-09T03:47:52.413334+0000","last_clean":"2023-11-09T03:47:52.413334+0000","last_became_active":"2023-11-09T03:47:50.786018+0000","last_became_peered":"2023-11-09T03:47:50.786018+0000","last_unstale":"2023-11-09T03:47:52.413334+0000","last_undegraded":"2023-11-09T03:47:52.413334+0000","last_fullsized":"2023-11-09T03:47:52.413334+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:22:44.272276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":35,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.765378+0000","last_change":"2023-11-09T03:47:03.146783+0000","last_active":"2023-11-09T03:47:50.765378+0000","last_peered":"2023-11-09T03:47:50.765378+0000","last_clean":"2023-11-09T03:47:50.765378+0000","last_became_active":"2023-11-09T03:47:01.586923+0000","last_became_peered":"2023-11-09T03:47:01.586923+0000","last_unstale":"2023-11-09T03:47:50.765378+0000","last_undegraded":"2023-11-09T03:47:50.765378+0000","last_fullsized":"2023-11-09T03:47:50.765378+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:37:50.326442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060384700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.413326+0000","last_change":"2023-11-09T03:47:51.423642+0000","last_active":"2023-11-09T03:47:52.413326+0000","last_peered":"2023-11-09T03:47:52.413326+0000","last_clean":"2023-11-09T03:47:52.413326+0000","last_became_active":"2023-11-09T03:47:51.423363+0000","last_became_peered":"2023-11-09T03:47:51.423363+0000","last_unstale":"2023-11-09T03:47:52.413326+0000","last_undegraded":"2023-11-09T03:47:52.413326+0000","last_fullsized":"2023-11-09T03:47:52.413326+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:26:42.350849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"65'1","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.786002+0000","last_change":"2023-11-09T03:47:50.786181+0000","last_active":"2023-11-09T03:47:51.786002+0000","last_peered":"2023-11-09T03:47:51.786002+0000","last_clean":"2023-11-09T03:47:51.786002+0000","last_became_active":"2023-11-09T03:47:50.785921+0000","last_became_peered":"2023-11-09T03:47:50.785921+0000","last_unstale":"2023-11-09T03:47:51.786002+0000","last_undegraded":"2023-11-09T03:47:51.786002+0000","last_fullsized":"2023-11-09T03:47:51.786002+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:26:08.262577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":35,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.765257+0000","last_change":"2023-11-09T03:47:03.146580+0000","last_active":"2023-11-09T03:47:50.765257+0000","last_peered":"2023-11-09T03:47:50.765257+0000","last_clean":"2023-11-09T03:47:50.765257+0000","last_became_active":"2023-11-09T03:47:01.589201+0000","last_became_peered":"2023-11-09T03:47:01.589201+0000","last_unstale":"2023-11-09T03:47:50.765257+0000","last_undegraded":"2023-11-09T03:47:50.765257+0000","last_fullsized":"2023-11-09T03:47:50.765257+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:14.300421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049706999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.428274+0000","last_change":"2023-11-09T03:47:51.427080+0000","last_active":"2023-11-09T03:47:51.428274+0000","last_peered":"2023-11-09T03:47:51.428274+0000","last_clean":"2023-11-09T03:47:51.428274+0000","last_became_active":"2023-11-09T03:47:51.426164+0000","last_became_peered":"2023-11-09T03:47:51.426164+0000","last_unstale":"2023-11-09T03:47:51.428274+0000","last_undegraded":"2023-11-09T03:47:51.428274+0000","last_fullsized":"2023-11-09T03:47:51.428274+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:35:45.209321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"65'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.785881+0000","last_change":"2023-11-09T03:47:50.781515+0000","last_active":"2023-11-09T03:47:51.785881+0000","last_peered":"2023-11-09T03:47:51.785881+0000","last_clean":"2023-11-09T03:47:51.785881+0000","last_became_active":"2023-11-09T03:47:50.781152+0000","last_became_peered":"2023-11-09T03:47:50.781152+0000","last_unstale":"2023-11-09T03:47:51.785881+0000","last_undegraded":"2023-11-09T03:47:51.785881+0000","last_fullsized":"2023-11-09T03:47:51.785881+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:15:06.138290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":34,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.411123+0000","last_change":"2023-11-09T03:47:03.147851+0000","last_active":"2023-11-09T03:47:51.411123+0000","last_peered":"2023-11-09T03:47:51.411123+0000","last_clean":"2023-11-09T03:47:51.411123+0000","last_became_active":"2023-11-09T03:47:01.589893+0000","last_became_peered":"2023-11-09T03:47:01.589893+0000","last_unstale":"2023-11-09T03:47:51.411123+0000","last_undegraded":"2023-11-09T03:47:51.411123+0000","last_fullsized":"2023-11-09T03:47:51.411123+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:08:17.183462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029151399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":202,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.775020+0000","last_change":"2023-11-09T03:45:21.030926+0000","last_active":"2023-11-09T03:47:50.775020+0000","last_peered":"2023-11-09T03:47:50.775020+0000","last_clean":"2023-11-09T03:47:50.775020+0000","last_became_active":"2023-11-09T03:45:21.029801+0000","last_became_peered":"2023-11-09T03:45:21.029801+0000","last_unstale":"2023-11-09T03:47:50.775020+0000","last_undegraded":"2023-11-09T03:47:50.775020+0000","last_fullsized":"2023-11-09T03:47:50.775020+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_clean_scrub_stamp":"2023-11-09T03:45:19.768433+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:12:59.153713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28380,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343460,"statfs":{"total":96636764160,"available":96607703040,"internally_reserved":0,"allocated":1388544,"data_stored":952415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.501,"15min":0.498},"min":{"1min":0.34899999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.41199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.54900000000000004}]},{"osd":1,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.33000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.38100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.51800000000000002}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.52300000000000002,"15min":0.51400000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.68700000000000006,"15min":0.70799999999999996},"min":{"1min":0.41299999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.80700000000000005,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.505}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.48699999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.58999999999999997}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.441},"min":{"1min":0.34100000000000003,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.49199999999999999,"15min":0.47899999999999998},"min":{"1min":0.38700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.57699999999999996}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.47199999999999998,"15min":0.45600000000000002},"min":{"1min":0.37,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.51100000000000001,"15min":0.497},"min":{"1min":0.39600000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55100000000000005,"15min":0.54100000000000004},"min":{"1min":0.44400000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.434,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.60599999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":794624,"data_stored":361523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.499,"15min":0.501},"min":{"1min":0.35099999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63900000000000001,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.50600000000000001,"15min":0.495},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.73199999999999998}]},{"osd":1,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.65300000000000002}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.375,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.70399999999999996}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.753},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.67500000000000004}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.503,"15min":0.503},"min":{"1min":0.316,"5min":0.313,"15min":0.313},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.56399999999999995}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.436,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55500000000000005,"15min":0.55800000000000005},"min":{"1min":0.34300000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.69299999999999995}]},{"osd":7,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.42899999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.44400000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.66300000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":361521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.496,"15min":0.48599999999999999},"min":{"1min":0.38700000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.503,"15min":0.498},"min":{"1min":0.42399999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.71799999999999997,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.51600000000000001}]},{"osd":1,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.439,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.69999999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.46700000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.76200000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.53100000000000003}]},{"osd":2,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.33300000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.67900000000000005,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.499,"5min":0.371,"15min":0.371},"max":{"1min":0.72799999999999998,"5min":0.745,"15min":0.745},"last":0.503}]},{"osd":3,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.39700000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.76500000000000001,"5min":0.88,"15min":0.88},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.78100000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.48399999999999999,"15min":0.48099999999999998},"min":{"1min":0.33100000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.48499999999999999}]},{"osd":6,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.50900000000000001,"15min":0.51100000000000001},"min":{"1min":0.33400000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.74199999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.442},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51300000000000001,"15min":0.51100000000000001},"min":{"1min":0.35499999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66900000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.46200000000000002}]},{"osd":7,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.38900000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.73499999999999999,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61899999999999999,"15min":0.62},"min":{"1min":0.41199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.752,"5min":1.01,"15min":1.01},"last":0.60199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":361501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67600000000000005,"5min":1.034,"15min":1.034},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.53600000000000003,"15min":0.54000000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68300000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.51100000000000001}]},{"osd":1,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.56899999999999995,"15min":0.57699999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68100000000000005,"5min":1.093,"15min":1.093},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.36899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72399999999999998,"5min":0.871,"15min":0.871},"last":0.53500000000000003}]},{"osd":2,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76800000000000002,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70099999999999996,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.69199999999999995}]},{"osd":3,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71699999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54900000000000004,"15min":0.55200000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86599999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.45400000000000001}]},{"osd":5,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54400000000000004,"15min":0.54300000000000004},"min":{"1min":0.37,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76000000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.33100000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.72899999999999998}]},{"osd":6,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.33800000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.622},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.65300000000000002}]},{"osd":7,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.62,"15min":0.63},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77700000000000002,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.95599999999999996,"5min":1.113,"15min":1.113},"last":0.66600000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":794624,"data_stored":361543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.5,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.313},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51100000000000001,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.47799999999999998}]},{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32900000000000001,"15min":0.33100000000000002},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.54200000000000004,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.371,"15min":0.375},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.53100000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.35499999999999998}]},{"osd":2,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.27200000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.59099999999999997,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.56100000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.45700000000000002}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.70899999999999996,"5min":0.87,"15min":0.87},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60999999999999999,"15min":0.61599999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.75600000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.83299999999999996}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.62,"15min":0.625},"min":{"1min":0.45700000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79900000000000004,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61699999999999999,"15min":0.623},"min":{"1min":0.42599999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.74299999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.82099999999999995}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.621,"15min":0.627},"min":{"1min":0.42399999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81200000000000006,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59999999999999998,"15min":0.60399999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76600000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.63200000000000001}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77200000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63500000000000001,"15min":0.64200000000000002},"min":{"1min":0.433,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81299999999999994,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.81299999999999994}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":798720,"data_stored":363043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.46000000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.33400000000000002,"15min":0.33100000000000002},"min":{"1min":0.247,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.23300000000000001}]},{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32500000000000001,"15min":0.32600000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.495,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.35899999999999999,"15min":0.36299999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.48599999999999999,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.28599999999999998}]},{"osd":3,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.36199999999999999,"15min":0.36399999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.45700000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.25},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40200000000000002,"15min":0.40400000000000003},"min":{"1min":0.30299999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.52300000000000002,"5min":0.754,"15min":0.754},"last":0.39000000000000001}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72299999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81299999999999994,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.40300000000000002}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.39800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.75700000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.59599999999999997,"15min":0.60099999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.753,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.434}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60399999999999998,"15min":0.60899999999999999},"min":{"1min":0.44600000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.76100000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.71199999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.498}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60699999999999998,"15min":0.61199999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.745,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.371},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.80000000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.46100000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":806912,"data_stored":363659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.28699999999999998,"5min":0.31,"15min":0.314},"min":{"1min":0.20799999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.38900000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.315,"15min":0.311},"min":{"1min":0.24099999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.22500000000000001}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.311,"15min":0.309},"min":{"1min":0.253,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.378,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.253},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.34000000000000002}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36899999999999999,"15min":0.371},"min":{"1min":0.23899999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.47699999999999998,"5min":0.505,"15min":0.505},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.37,"15min":0.371},"min":{"1min":0.30299999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.45400000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.27100000000000002}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.41899999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.747,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.44400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.77600000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.51000000000000001}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56200000000000006,"15min":0.55700000000000005},"min":{"1min":0.46100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.44500000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.72599999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.44400000000000001}]},{"osd":6,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.47799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56699999999999995,"15min":0.56799999999999995},"min":{"1min":0.433,"5min":0.379,"15min":0.379},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.45400000000000001}]},{"osd":7,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.42899999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.69799999999999995,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.72899999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.49199999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28368,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343472,"statfs":{"total":96636764160,"available":96607715328,"internally_reserved":0,"allocated":1376256,"data_stored":951180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.31900000000000001,"15min":0.32300000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.38200000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.25},{"interface":"front","average":{"1min":0.29599999999999999,"5min":0.35499999999999998,"15min":0.36499999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.44500000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.375}]},{"osd":2,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.35699999999999998,"15min":0.36399999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.44700000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.36699999999999999,"15min":0.371},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.47699999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.33700000000000002}]},{"osd":3,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.38500000000000001,"15min":0.39100000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.46400000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.41499999999999998,"15min":0.42099999999999999},"min":{"1min":0.309,"5min":0.307,"15min":0.307},"max":{"1min":0.48999999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.32900000000000001}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58899999999999997,"15min":0.59299999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60599999999999998,"15min":0.60899999999999999},"min":{"1min":0.436,"5min":0.318,"15min":0.318},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.51100000000000001}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.36199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76800000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.629,"15min":0.63600000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78900000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.497}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.64100000000000001,"15min":0.64700000000000002},"min":{"1min":0.42499999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81000000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.627,"15min":0.63200000000000001},"min":{"1min":0.41699999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.55900000000000005}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.44400000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.42999999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.71799999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-09T03:47:54.740 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-09T03:47:54.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.873+0000 7fd915c4b640 1 Processor -- start 2023-11-09T03:47:54.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.873+0000 7fd915c4b640 1 -- start start 2023-11-09T03:47:54.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.874+0000 7fd915c4b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd91015ab70 0x7fd9100cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:54.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.874+0000 7fd915c4b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd91006eba0 0x7fd91006a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:54.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.874+0000 7fd915c4b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd91006a660 0x7fd9100ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:54.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.874+0000 7fd915c4b640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9100cba70 con 0x7fd91006eba0 2023-11-09T03:47:54.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.874+0000 7fd915c4b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd91015d650 con 0x7fd91015ab70 2023-11-09T03:47:54.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.874+0000 7fd915c4b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91015e620 con 0x7fd91006a660 2023-11-09T03:47:54.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.874+0000 7fd90effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd91006eba0 0x7fd91006a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:54.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.874+0000 7fd90effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd91006eba0 0x7fd91006a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41124/0 (socket says 172.21.15.76:41124) 2023-11-09T03:47:54.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.874+0000 7fd90effd640 1 -- 172.21.15.76:0/807274821 learned_addr learned my addr 172.21.15.76:0/807274821 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:54.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.874+0000 7fd90effd640 1 -- 172.21.15.76:0/807274821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd91006a660 msgr2=0x7fd9100ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:54.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.874+0000 7fd90effd640 1 --2- 172.21.15.76:0/807274821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd91006a660 0x7fd9100ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.874+0000 7fd90effd640 1 -- 172.21.15.76:0/807274821 >> v2:172.21.15.113:3300/0 conn(0x7fd91015ab70 msgr2=0x7fd9100cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:54.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.874+0000 7fd90f7fe640 1 --2- 172.21.15.76:0/807274821 >> v2:172.21.15.113:3300/0 conn(0x7fd91015ab70 0x7fd9100cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:54.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.874+0000 7fd90effd640 1 --2- 172.21.15.76:0/807274821 >> v2:172.21.15.113:3300/0 conn(0x7fd91015ab70 0x7fd9100cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.874+0000 7fd90effd640 1 -- 172.21.15.76:0/807274821 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91006aa90 con 0x7fd91006eba0 2023-11-09T03:47:54.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.875+0000 7fd90effd640 1 --2- 172.21.15.76:0/807274821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd91006eba0 0x7fd91006a120 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fd9100b2600 tx=0x7fd90402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f10f4a3e82f826 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:54.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.875+0000 7fd90e7fc640 1 -- 172.21.15.76:0/807274821 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90402e070 con 0x7fd91006eba0 2023-11-09T03:47:54.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.875+0000 7fd90e7fc640 1 -- 172.21.15.76:0/807274821 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd904004030 con 0x7fd91006eba0 2023-11-09T03:47:54.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.875+0000 7fd90e7fc640 1 -- 172.21.15.76:0/807274821 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd904004420 con 0x7fd91006eba0 2023-11-09T03:47:54.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.876+0000 7fd915c4b640 1 -- 172.21.15.76:0/807274821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd91006eba0 msgr2=0x7fd91006a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:54.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.876+0000 7fd915c4b640 1 --2- 172.21.15.76:0/807274821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd91006eba0 0x7fd91006a120 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fd9100b2600 tx=0x7fd90402b840 comp rx=0 tx=0).stop 2023-11-09T03:47:54.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.876+0000 7fd915c4b640 1 -- 172.21.15.76:0/807274821 shutdown_connections 2023-11-09T03:47:54.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.876+0000 7fd915c4b640 1 --2- 172.21.15.76:0/807274821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd91006a660 0x7fd9100ae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.876+0000 7fd915c4b640 1 --2- 172.21.15.76:0/807274821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd91006eba0 0x7fd91006a120 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.876+0000 7fd915c4b640 1 --2- 172.21.15.76:0/807274821 >> v2:172.21.15.113:3300/0 conn(0x7fd91015ab70 0x7fd9100cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.876+0000 7fd915c4b640 1 -- 172.21.15.76:0/807274821 >> 172.21.15.76:0/807274821 conn(0x7fd910098d40 msgr2=0x7fd9100cfa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:54.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.876+0000 7fd915c4b640 1 -- 172.21.15.76:0/807274821 shutdown_connections 2023-11-09T03:47:54.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.876+0000 7fd915c4b640 1 -- 172.21.15.76:0/807274821 wait complete. 2023-11-09T03:47:54.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.877+0000 7fd915c4b640 1 Processor -- start 2023-11-09T03:47:54.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.877+0000 7fd915c4b640 1 -- start start 2023-11-09T03:47:54.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.878+0000 7fd915c4b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd91006a660 0x7fd91014ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:54.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.878+0000 7fd915c4b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd91006eba0 0x7fd9100997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:54.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.878+0000 7fd915c4b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd91015ab70 0x7fd91009a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:54.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.878+0000 7fd915c4b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91014f4b0 con 0x7fd91006a660 2023-11-09T03:47:54.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.878+0000 7fd915c4b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9100b3380 con 0x7fd91006eba0 2023-11-09T03:47:54.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.878+0000 7fd915c4b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9100b3660 con 0x7fd91015ab70 2023-11-09T03:47:54.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.878+0000 7fd90f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd91006a660 0x7fd91014ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:54.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.878+0000 7fd90f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd91006a660 0x7fd91014ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41140/0 (socket says 172.21.15.76:41140) 2023-11-09T03:47:54.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.878+0000 7fd90f7fe640 1 -- 172.21.15.76:0/1477492038 learned_addr learned my addr 172.21.15.76:0/1477492038 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:54.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.878+0000 7fd90effd640 1 --2- 172.21.15.76:0/1477492038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd91006eba0 0x7fd9100997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:54.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.878+0000 7fd90ffff640 1 --2- 172.21.15.76:0/1477492038 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd91015ab70 0x7fd91009a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:54.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.878+0000 7fd90f7fe640 1 -- 172.21.15.76:0/1477492038 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd91015ab70 msgr2=0x7fd91009a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:54.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.878+0000 7fd90f7fe640 1 --2- 172.21.15.76:0/1477492038 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd91015ab70 0x7fd91009a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.878+0000 7fd90f7fe640 1 -- 172.21.15.76:0/1477492038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd91006eba0 msgr2=0x7fd9100997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:54.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.878+0000 7fd90f7fe640 1 --2- 172.21.15.76:0/1477492038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd91006eba0 0x7fd9100997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:54.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.878+0000 7fd90f7fe640 1 -- 172.21.15.76:0/1477492038 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd904009460 con 0x7fd91006a660 2023-11-09T03:47:54.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.878+0000 7fd90effd640 1 --2- 172.21.15.76:0/1477492038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd91006eba0 0x7fd9100997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:54.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.878+0000 7fd90ffff640 1 --2- 172.21.15.76:0/1477492038 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd91015ab70 0x7fd91009a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:54.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.879+0000 7fd90f7fe640 1 --2- 172.21.15.76:0/1477492038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd91006a660 0x7fd91014ef70 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fd90000c7f0 tx=0x7fd90000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:54.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.879+0000 7fd8f7fff640 1 -- 172.21.15.76:0/1477492038 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd900007820 con 0x7fd91006a660 2023-11-09T03:47:54.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.879+0000 7fd8f7fff640 1 -- 172.21.15.76:0/1477492038 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd900007980 con 0x7fd91006a660 2023-11-09T03:47:54.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.879+0000 7fd915c4b640 1 -- 172.21.15.76:0/1477492038 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9100b3940 con 0x7fd91006a660 2023-11-09T03:47:54.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.879+0000 7fd8f7fff640 1 -- 172.21.15.76:0/1477492038 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd900007d30 con 0x7fd91006a660 2023-11-09T03:47:54.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.879+0000 7fd915c4b640 1 -- 172.21.15.76:0/1477492038 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd910099cf0 con 0x7fd91006a660 2023-11-09T03:47:54.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.880+0000 7fd915c4b640 1 -- 172.21.15.76:0/1477492038 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd910074290 con 0x7fd91006a660 2023-11-09T03:47:54.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.880+0000 7fd8f7fff640 1 -- 172.21.15.76:0/1477492038 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd900004020 con 0x7fd91006a660 2023-11-09T03:47:54.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.880+0000 7fd8f7fff640 1 --2- 172.21.15.76:0/1477492038 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd8fc06ae70 0x7fd8fc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:54.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.881+0000 7fd8f7fff640 1 -- 172.21.15.76:0/1477492038 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd900089b80 con 0x7fd91006a660 2023-11-09T03:47:54.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.881+0000 7fd90effd640 1 --2- 172.21.15.76:0/1477492038 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd8fc06ae70 0x7fd8fc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:54.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.881+0000 7fd90effd640 1 --2- 172.21.15.76:0/1477492038 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd8fc06ae70 0x7fd8fc06d330 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd904034000 tx=0x7fd904008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:54.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:54.885+0000 7fd8f7fff640 1 -- 172.21.15.76:0/1477492038 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd900054a20 con 0x7fd91006a660 2023-11-09T03:47:55.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.049+0000 7fd915c4b640 1 -- 172.21.15.76:0/1477492038 --> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd91015d860 con 0x7fd8fc06ae70 2023-11-09T03:47:55.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.059+0000 7fd8f7fff640 1 -- 172.21.15.76:0/1477492038 <== mgr.4105 v2:172.21.15.113:6832/1264193042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+370461 (secure 0 0 0) 0x7fd904042090 con 0x7fd8fc06ae70 2023-11-09T03:47:55.061 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:55.063 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2023-11-09T03:47:55.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.062+0000 7fd915c4b640 1 -- 172.21.15.76:0/1477492038 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd8fc06ae70 msgr2=0x7fd8fc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:55.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.062+0000 7fd915c4b640 1 --2- 172.21.15.76:0/1477492038 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd8fc06ae70 0x7fd8fc06d330 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd904034000 tx=0x7fd904008030 comp rx=0 tx=0).stop 2023-11-09T03:47:55.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.062+0000 7fd915c4b640 1 -- 172.21.15.76:0/1477492038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd91006a660 msgr2=0x7fd91014ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:55.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.062+0000 7fd915c4b640 1 --2- 172.21.15.76:0/1477492038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd91006a660 0x7fd91014ef70 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fd90000c7f0 tx=0x7fd90000ccb0 comp rx=0 tx=0).stop 2023-11-09T03:47:55.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.062+0000 7fd915c4b640 1 -- 172.21.15.76:0/1477492038 shutdown_connections 2023-11-09T03:47:55.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.062+0000 7fd915c4b640 1 --2- 172.21.15.76:0/1477492038 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd8fc06ae70 0x7fd8fc06d330 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:55.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.062+0000 7fd915c4b640 1 --2- 172.21.15.76:0/1477492038 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd91015ab70 0x7fd91009a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:55.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.062+0000 7fd915c4b640 1 --2- 172.21.15.76:0/1477492038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd91006eba0 0x7fd9100997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:55.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.062+0000 7fd915c4b640 1 --2- 172.21.15.76:0/1477492038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd91006a660 0x7fd91014ef70 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:55.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.062+0000 7fd915c4b640 1 -- 172.21.15.76:0/1477492038 >> 172.21.15.76:0/1477492038 conn(0x7fd910098d40 msgr2=0x7fd91015c260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:55.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.062+0000 7fd915c4b640 1 -- 172.21.15.76:0/1477492038 shutdown_connections 2023-11-09T03:47:55.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.062+0000 7fd915c4b640 1 -- 172.21.15.76:0/1477492038 wait complete. 2023-11-09T03:47:55.072 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2023-11-09T03:47:54.785586+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223580,"kb_used_data":7364,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751140,"statfs":{"total":773094113280,"available":772865167360,"internally_reserved":0,"allocated":7540736,"data_stored":4076385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.023253"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.433191+0000","last_change":"2023-11-09T03:47:51.431585+0000","last_active":"2023-11-09T03:47:51.433191+0000","last_peered":"2023-11-09T03:47:51.433191+0000","last_clean":"2023-11-09T03:47:51.433191+0000","last_became_active":"2023-11-09T03:47:51.430932+0000","last_became_peered":"2023-11-09T03:47:51.430932+0000","last_unstale":"2023-11-09T03:47:51.433191+0000","last_undegraded":"2023-11-09T03:47:51.433191+0000","last_fullsized":"2023-11-09T03:47:51.433191+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:21:20.038282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.437544+0000","last_change":"2023-11-09T03:47:51.436525+0000","last_active":"2023-11-09T03:47:51.437544+0000","last_peered":"2023-11-09T03:47:51.437544+0000","last_clean":"2023-11-09T03:47:51.437544+0000","last_became_active":"2023-11-09T03:47:51.436248+0000","last_became_peered":"2023-11-09T03:47:51.436248+0000","last_unstale":"2023-11-09T03:47:51.437544+0000","last_undegraded":"2023-11-09T03:47:51.437544+0000","last_fullsized":"2023-11-09T03:47:51.437544+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:52:24.725477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.423880+0000","last_change":"2023-11-09T03:47:51.419938+0000","last_active":"2023-11-09T03:47:51.423880+0000","last_peered":"2023-11-09T03:47:51.423880+0000","last_clean":"2023-11-09T03:47:51.423880+0000","last_became_active":"2023-11-09T03:47:51.419181+0000","last_became_peered":"2023-11-09T03:47:51.419181+0000","last_unstale":"2023-11-09T03:47:51.423880+0000","last_undegraded":"2023-11-09T03:47:51.423880+0000","last_fullsized":"2023-11-09T03:47:51.423880+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:27:39.529115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.427045+0000","last_change":"2023-11-09T03:47:51.425618+0000","last_active":"2023-11-09T03:47:51.427045+0000","last_peered":"2023-11-09T03:47:51.427045+0000","last_clean":"2023-11-09T03:47:51.427045+0000","last_became_active":"2023-11-09T03:47:51.425344+0000","last_became_peered":"2023-11-09T03:47:51.425344+0000","last_unstale":"2023-11-09T03:47:51.427045+0000","last_undegraded":"2023-11-09T03:47:51.427045+0000","last_fullsized":"2023-11-09T03:47:51.427045+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:06:42.522828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.432897+0000","last_change":"2023-11-09T03:47:51.430203+0000","last_active":"2023-11-09T03:47:51.432897+0000","last_peered":"2023-11-09T03:47:51.432897+0000","last_clean":"2023-11-09T03:47:51.432897+0000","last_became_active":"2023-11-09T03:47:51.429837+0000","last_became_peered":"2023-11-09T03:47:51.429837+0000","last_unstale":"2023-11-09T03:47:51.432897+0000","last_undegraded":"2023-11-09T03:47:51.432897+0000","last_fullsized":"2023-11-09T03:47:51.432897+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:26:42.949360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.435746+0000","last_change":"2023-11-09T03:47:51.434475+0000","last_active":"2023-11-09T03:47:51.435746+0000","last_peered":"2023-11-09T03:47:51.435746+0000","last_clean":"2023-11-09T03:47:51.435746+0000","last_became_active":"2023-11-09T03:47:51.434185+0000","last_became_peered":"2023-11-09T03:47:51.434185+0000","last_unstale":"2023-11-09T03:47:51.435746+0000","last_undegraded":"2023-11-09T03:47:51.435746+0000","last_fullsized":"2023-11-09T03:47:51.435746+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:05:01.816130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.432495+0000","last_change":"2023-11-09T03:47:51.428082+0000","last_active":"2023-11-09T03:47:51.432495+0000","last_peered":"2023-11-09T03:47:51.432495+0000","last_clean":"2023-11-09T03:47:51.432495+0000","last_became_active":"2023-11-09T03:47:51.427814+0000","last_became_peered":"2023-11-09T03:47:51.427814+0000","last_unstale":"2023-11-09T03:47:51.432495+0000","last_undegraded":"2023-11-09T03:47:51.432495+0000","last_fullsized":"2023-11-09T03:47:51.432495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:42:31.986727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.433050+0000","last_change":"2023-11-09T03:47:51.431870+0000","last_active":"2023-11-09T03:47:51.433050+0000","last_peered":"2023-11-09T03:47:51.433050+0000","last_clean":"2023-11-09T03:47:51.433050+0000","last_became_active":"2023-11-09T03:47:51.431385+0000","last_became_peered":"2023-11-09T03:47:51.431385+0000","last_unstale":"2023-11-09T03:47:51.433050+0000","last_undegraded":"2023-11-09T03:47:51.433050+0000","last_fullsized":"2023-11-09T03:47:51.433050+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:59:01.404347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.421594+0000","last_change":"2023-11-09T03:47:51.419475+0000","last_active":"2023-11-09T03:47:51.421594+0000","last_peered":"2023-11-09T03:47:51.421594+0000","last_clean":"2023-11-09T03:47:51.421594+0000","last_became_active":"2023-11-09T03:47:51.418167+0000","last_became_peered":"2023-11-09T03:47:51.418167+0000","last_unstale":"2023-11-09T03:47:51.421594+0000","last_undegraded":"2023-11-09T03:47:51.421594+0000","last_fullsized":"2023-11-09T03:47:51.421594+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:58:40.615051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.413230+0000","last_change":"2023-11-09T03:47:51.425933+0000","last_active":"2023-11-09T03:47:52.413230+0000","last_peered":"2023-11-09T03:47:52.413230+0000","last_clean":"2023-11-09T03:47:52.413230+0000","last_became_active":"2023-11-09T03:47:51.425597+0000","last_became_peered":"2023-11-09T03:47:51.425597+0000","last_unstale":"2023-11-09T03:47:52.413230+0000","last_undegraded":"2023-11-09T03:47:52.413230+0000","last_fullsized":"2023-11-09T03:47:52.413230+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:09:11.376416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.431687+0000","last_change":"2023-11-09T03:47:51.430610+0000","last_active":"2023-11-09T03:47:51.431687+0000","last_peered":"2023-11-09T03:47:51.431687+0000","last_clean":"2023-11-09T03:47:51.431687+0000","last_became_active":"2023-11-09T03:47:51.429575+0000","last_became_peered":"2023-11-09T03:47:51.429575+0000","last_unstale":"2023-11-09T03:47:51.431687+0000","last_undegraded":"2023-11-09T03:47:51.431687+0000","last_fullsized":"2023-11-09T03:47:51.431687+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:38:52.289915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.422504+0000","last_change":"2023-11-09T03:47:51.420829+0000","last_active":"2023-11-09T03:47:51.422504+0000","last_peered":"2023-11-09T03:47:51.422504+0000","last_clean":"2023-11-09T03:47:51.422504+0000","last_became_active":"2023-11-09T03:47:51.420557+0000","last_became_peered":"2023-11-09T03:47:51.420557+0000","last_unstale":"2023-11-09T03:47:51.422504+0000","last_undegraded":"2023-11-09T03:47:51.422504+0000","last_fullsized":"2023-11-09T03:47:51.422504+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:00:05.584311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.414736+0000","last_change":"2023-11-09T03:47:51.426249+0000","last_active":"2023-11-09T03:47:52.414736+0000","last_peered":"2023-11-09T03:47:52.414736+0000","last_clean":"2023-11-09T03:47:52.414736+0000","last_became_active":"2023-11-09T03:47:51.425990+0000","last_became_peered":"2023-11-09T03:47:51.425990+0000","last_unstale":"2023-11-09T03:47:52.414736+0000","last_undegraded":"2023-11-09T03:47:52.414736+0000","last_fullsized":"2023-11-09T03:47:52.414736+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:20:51.046991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.425940+0000","last_change":"2023-11-09T03:47:51.425017+0000","last_active":"2023-11-09T03:47:51.425940+0000","last_peered":"2023-11-09T03:47:51.425940+0000","last_clean":"2023-11-09T03:47:51.425940+0000","last_became_active":"2023-11-09T03:47:51.424755+0000","last_became_peered":"2023-11-09T03:47:51.424755+0000","last_unstale":"2023-11-09T03:47:51.425940+0000","last_undegraded":"2023-11-09T03:47:51.425940+0000","last_fullsized":"2023-11-09T03:47:51.425940+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:53:51.173223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.414815+0000","last_change":"2023-11-09T03:47:51.418529+0000","last_active":"2023-11-09T03:47:52.414815+0000","last_peered":"2023-11-09T03:47:52.414815+0000","last_clean":"2023-11-09T03:47:52.414815+0000","last_became_active":"2023-11-09T03:47:51.418260+0000","last_became_peered":"2023-11-09T03:47:51.418260+0000","last_unstale":"2023-11-09T03:47:52.414815+0000","last_undegraded":"2023-11-09T03:47:52.414815+0000","last_fullsized":"2023-11-09T03:47:52.414815+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:16:23.544065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.412904+0000","last_change":"2023-11-09T03:47:51.430458+0000","last_active":"2023-11-09T03:47:52.412904+0000","last_peered":"2023-11-09T03:47:52.412904+0000","last_clean":"2023-11-09T03:47:52.412904+0000","last_became_active":"2023-11-09T03:47:51.430165+0000","last_became_peered":"2023-11-09T03:47:51.430165+0000","last_unstale":"2023-11-09T03:47:52.412904+0000","last_undegraded":"2023-11-09T03:47:52.412904+0000","last_fullsized":"2023-11-09T03:47:52.412904+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:08:34.417859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.422230+0000","last_change":"2023-11-09T03:47:51.420761+0000","last_active":"2023-11-09T03:47:51.422230+0000","last_peered":"2023-11-09T03:47:51.422230+0000","last_clean":"2023-11-09T03:47:51.422230+0000","last_became_active":"2023-11-09T03:47:51.420484+0000","last_became_peered":"2023-11-09T03:47:51.420484+0000","last_unstale":"2023-11-09T03:47:51.422230+0000","last_undegraded":"2023-11-09T03:47:51.422230+0000","last_fullsized":"2023-11-09T03:47:51.422230+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:36:34.915038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.426869+0000","last_change":"2023-11-09T03:47:51.425219+0000","last_active":"2023-11-09T03:47:51.426869+0000","last_peered":"2023-11-09T03:47:51.426869+0000","last_clean":"2023-11-09T03:47:51.426869+0000","last_became_active":"2023-11-09T03:47:51.424984+0000","last_became_peered":"2023-11-09T03:47:51.424984+0000","last_unstale":"2023-11-09T03:47:51.426869+0000","last_undegraded":"2023-11-09T03:47:51.426869+0000","last_fullsized":"2023-11-09T03:47:51.426869+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:53:59.332467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.431510+0000","last_change":"2023-11-09T03:47:51.429393+0000","last_active":"2023-11-09T03:47:51.431510+0000","last_peered":"2023-11-09T03:47:51.431510+0000","last_clean":"2023-11-09T03:47:51.431510+0000","last_became_active":"2023-11-09T03:47:51.429115+0000","last_became_peered":"2023-11-09T03:47:51.429115+0000","last_unstale":"2023-11-09T03:47:51.431510+0000","last_undegraded":"2023-11-09T03:47:51.431510+0000","last_fullsized":"2023-11-09T03:47:51.431510+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:55:58.922785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.414645+0000","last_change":"2023-11-09T03:47:51.424978+0000","last_active":"2023-11-09T03:47:52.414645+0000","last_peered":"2023-11-09T03:47:52.414645+0000","last_clean":"2023-11-09T03:47:52.414645+0000","last_became_active":"2023-11-09T03:47:51.424712+0000","last_became_peered":"2023-11-09T03:47:51.424712+0000","last_unstale":"2023-11-09T03:47:52.414645+0000","last_undegraded":"2023-11-09T03:47:52.414645+0000","last_fullsized":"2023-11-09T03:47:52.414645+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:41.735881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.413179+0000","last_change":"2023-11-09T03:47:51.425271+0000","last_active":"2023-11-09T03:47:52.413179+0000","last_peered":"2023-11-09T03:47:52.413179+0000","last_clean":"2023-11-09T03:47:52.413179+0000","last_became_active":"2023-11-09T03:47:51.424721+0000","last_became_peered":"2023-11-09T03:47:51.424721+0000","last_unstale":"2023-11-09T03:47:52.413179+0000","last_undegraded":"2023-11-09T03:47:52.413179+0000","last_fullsized":"2023-11-09T03:47:52.413179+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:09:21.626828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.413155+0000","last_change":"2023-11-09T03:47:51.426715+0000","last_active":"2023-11-09T03:47:52.413155+0000","last_peered":"2023-11-09T03:47:52.413155+0000","last_clean":"2023-11-09T03:47:52.413155+0000","last_became_active":"2023-11-09T03:47:51.426415+0000","last_became_peered":"2023-11-09T03:47:51.426415+0000","last_unstale":"2023-11-09T03:47:52.413155+0000","last_undegraded":"2023-11-09T03:47:52.413155+0000","last_fullsized":"2023-11-09T03:47:52.413155+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:43:42.601800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.430510+0000","last_change":"2023-11-09T03:47:51.427292+0000","last_active":"2023-11-09T03:47:51.430510+0000","last_peered":"2023-11-09T03:47:51.430510+0000","last_clean":"2023-11-09T03:47:51.430510+0000","last_became_active":"2023-11-09T03:47:51.427049+0000","last_became_peered":"2023-11-09T03:47:51.427049+0000","last_unstale":"2023-11-09T03:47:51.430510+0000","last_undegraded":"2023-11-09T03:47:51.430510+0000","last_fullsized":"2023-11-09T03:47:51.430510+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:36:47.931275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.436619+0000","last_change":"2023-11-09T03:47:51.434689+0000","last_active":"2023-11-09T03:47:51.436619+0000","last_peered":"2023-11-09T03:47:51.436619+0000","last_clean":"2023-11-09T03:47:51.436619+0000","last_became_active":"2023-11-09T03:47:51.434411+0000","last_became_peered":"2023-11-09T03:47:51.434411+0000","last_unstale":"2023-11-09T03:47:51.436619+0000","last_undegraded":"2023-11-09T03:47:51.436619+0000","last_fullsized":"2023-11-09T03:47:51.436619+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:53:23.159911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.413487+0000","last_change":"2023-11-09T03:47:51.424006+0000","last_active":"2023-11-09T03:47:52.413487+0000","last_peered":"2023-11-09T03:47:52.413487+0000","last_clean":"2023-11-09T03:47:52.413487+0000","last_became_active":"2023-11-09T03:47:51.422581+0000","last_became_peered":"2023-11-09T03:47:51.422581+0000","last_unstale":"2023-11-09T03:47:52.413487+0000","last_undegraded":"2023-11-09T03:47:52.413487+0000","last_fullsized":"2023-11-09T03:47:52.413487+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:52:09.449985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.428703+0000","last_change":"2023-11-09T03:47:51.426203+0000","last_active":"2023-11-09T03:47:51.428703+0000","last_peered":"2023-11-09T03:47:51.428703+0000","last_clean":"2023-11-09T03:47:51.428703+0000","last_became_active":"2023-11-09T03:47:51.425355+0000","last_became_peered":"2023-11-09T03:47:51.425355+0000","last_unstale":"2023-11-09T03:47:51.428703+0000","last_undegraded":"2023-11-09T03:47:51.428703+0000","last_fullsized":"2023-11-09T03:47:51.428703+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:32:30.681190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.425427+0000","last_change":"2023-11-09T03:47:51.423308+0000","last_active":"2023-11-09T03:47:51.425427+0000","last_peered":"2023-11-09T03:47:51.425427+0000","last_clean":"2023-11-09T03:47:51.425427+0000","last_became_active":"2023-11-09T03:47:51.423067+0000","last_became_peered":"2023-11-09T03:47:51.423067+0000","last_unstale":"2023-11-09T03:47:51.425427+0000","last_undegraded":"2023-11-09T03:47:51.425427+0000","last_fullsized":"2023-11-09T03:47:51.425427+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:12:17.509338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.413202+0000","last_change":"2023-11-09T03:47:51.431335+0000","last_active":"2023-11-09T03:47:52.413202+0000","last_peered":"2023-11-09T03:47:52.413202+0000","last_clean":"2023-11-09T03:47:52.413202+0000","last_became_active":"2023-11-09T03:47:51.428252+0000","last_became_peered":"2023-11-09T03:47:51.428252+0000","last_unstale":"2023-11-09T03:47:52.413202+0000","last_undegraded":"2023-11-09T03:47:52.413202+0000","last_fullsized":"2023-11-09T03:47:52.413202+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:27:32.178108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.427921+0000","last_change":"2023-11-09T03:47:51.425727+0000","last_active":"2023-11-09T03:47:51.427921+0000","last_peered":"2023-11-09T03:47:51.427921+0000","last_clean":"2023-11-09T03:47:51.427921+0000","last_became_active":"2023-11-09T03:47:51.425517+0000","last_became_peered":"2023-11-09T03:47:51.425517+0000","last_unstale":"2023-11-09T03:47:51.427921+0000","last_undegraded":"2023-11-09T03:47:51.427921+0000","last_fullsized":"2023-11-09T03:47:51.427921+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:06:18.914118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.428649+0000","last_change":"2023-11-09T03:47:51.426023+0000","last_active":"2023-11-09T03:47:51.428649+0000","last_peered":"2023-11-09T03:47:51.428649+0000","last_clean":"2023-11-09T03:47:51.428649+0000","last_became_active":"2023-11-09T03:47:51.425767+0000","last_became_peered":"2023-11-09T03:47:51.425767+0000","last_unstale":"2023-11-09T03:47:51.428649+0000","last_undegraded":"2023-11-09T03:47:51.428649+0000","last_fullsized":"2023-11-09T03:47:51.428649+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:36:47.931275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.430145+0000","last_change":"2023-11-09T03:47:51.427277+0000","last_active":"2023-11-09T03:47:51.430145+0000","last_peered":"2023-11-09T03:47:51.430145+0000","last_clean":"2023-11-09T03:47:51.430145+0000","last_became_active":"2023-11-09T03:47:51.426955+0000","last_became_peered":"2023-11-09T03:47:51.426955+0000","last_unstale":"2023-11-09T03:47:51.430145+0000","last_undegraded":"2023-11-09T03:47:51.430145+0000","last_fullsized":"2023-11-09T03:47:51.430145+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:38:52.289915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.431683+0000","last_change":"2023-11-09T03:47:51.427399+0000","last_active":"2023-11-09T03:47:51.431683+0000","last_peered":"2023-11-09T03:47:51.431683+0000","last_clean":"2023-11-09T03:47:51.431683+0000","last_became_active":"2023-11-09T03:47:51.427133+0000","last_became_peered":"2023-11-09T03:47:51.427133+0000","last_unstale":"2023-11-09T03:47:51.431683+0000","last_undegraded":"2023-11-09T03:47:51.431683+0000","last_fullsized":"2023-11-09T03:47:51.431683+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:18.970489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.421470+0000","last_change":"2023-11-09T03:47:51.419191+0000","last_active":"2023-11-09T03:47:51.421470+0000","last_peered":"2023-11-09T03:47:51.421470+0000","last_clean":"2023-11-09T03:47:51.421470+0000","last_became_active":"2023-11-09T03:47:51.418955+0000","last_became_peered":"2023-11-09T03:47:51.418955+0000","last_unstale":"2023-11-09T03:47:51.421470+0000","last_undegraded":"2023-11-09T03:47:51.421470+0000","last_fullsized":"2023-11-09T03:47:51.421470+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:56:54.397650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"65'8","reported_seq":20,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.412909+0000","last_change":"2023-11-09T03:47:50.776715+0000","last_active":"2023-11-09T03:47:52.412909+0000","last_peered":"2023-11-09T03:47:52.412909+0000","last_clean":"2023-11-09T03:47:52.412909+0000","last_became_active":"2023-11-09T03:47:50.776449+0000","last_became_peered":"2023-11-09T03:47:50.776449+0000","last_unstale":"2023-11-09T03:47:52.412909+0000","last_undegraded":"2023-11-09T03:47:52.412909+0000","last_fullsized":"2023-11-09T03:47:52.412909+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:14:58.280559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.771810+0000","last_change":"2023-11-09T03:47:02.581917+0000","last_active":"2023-11-09T03:47:50.771810+0000","last_peered":"2023-11-09T03:47:50.771810+0000","last_clean":"2023-11-09T03:47:50.771810+0000","last_became_active":"2023-11-09T03:47:01.588082+0000","last_became_peered":"2023-11-09T03:47:01.588082+0000","last_unstale":"2023-11-09T03:47:50.771810+0000","last_undegraded":"2023-11-09T03:47:50.771810+0000","last_fullsized":"2023-11-09T03:47:50.771810+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:44:07.653557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034538899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.414593+0000","last_change":"2023-11-09T03:47:51.425687+0000","last_active":"2023-11-09T03:47:52.414593+0000","last_peered":"2023-11-09T03:47:52.414593+0000","last_clean":"2023-11-09T03:47:52.414593+0000","last_became_active":"2023-11-09T03:47:51.425337+0000","last_became_peered":"2023-11-09T03:47:51.425337+0000","last_unstale":"2023-11-09T03:47:52.414593+0000","last_undegraded":"2023-11-09T03:47:52.414593+0000","last_fullsized":"2023-11-09T03:47:52.414593+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:49:15.018709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.421104+0000","last_change":"2023-11-09T03:47:50.780392+0000","last_active":"2023-11-09T03:47:51.421104+0000","last_peered":"2023-11-09T03:47:51.421104+0000","last_clean":"2023-11-09T03:47:51.421104+0000","last_became_active":"2023-11-09T03:47:50.780121+0000","last_became_peered":"2023-11-09T03:47:50.780121+0000","last_unstale":"2023-11-09T03:47:51.421104+0000","last_undegraded":"2023-11-09T03:47:51.421104+0000","last_fullsized":"2023-11-09T03:47:51.421104+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:44:59.854706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":33,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.772789+0000","last_change":"2023-11-09T03:47:03.029063+0000","last_active":"2023-11-09T03:47:50.772789+0000","last_peered":"2023-11-09T03:47:50.772789+0000","last_clean":"2023-11-09T03:47:50.772789+0000","last_became_active":"2023-11-09T03:47:01.593582+0000","last_became_peered":"2023-11-09T03:47:01.593582+0000","last_unstale":"2023-11-09T03:47:50.772789+0000","last_undegraded":"2023-11-09T03:47:50.772789+0000","last_fullsized":"2023-11-09T03:47:50.772789+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:56:20.649937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041855100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.432338+0000","last_change":"2023-11-09T03:47:51.428315+0000","last_active":"2023-11-09T03:47:51.432338+0000","last_peered":"2023-11-09T03:47:51.432338+0000","last_clean":"2023-11-09T03:47:51.432338+0000","last_became_active":"2023-11-09T03:47:51.428074+0000","last_became_peered":"2023-11-09T03:47:51.428074+0000","last_unstale":"2023-11-09T03:47:51.432338+0000","last_undegraded":"2023-11-09T03:47:51.432338+0000","last_fullsized":"2023-11-09T03:47:51.432338+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:11:17.062291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.422640+0000","last_change":"2023-11-09T03:47:50.785900+0000","last_active":"2023-11-09T03:47:51.422640+0000","last_peered":"2023-11-09T03:47:51.422640+0000","last_clean":"2023-11-09T03:47:51.422640+0000","last_became_active":"2023-11-09T03:47:50.785601+0000","last_became_peered":"2023-11-09T03:47:50.785601+0000","last_unstale":"2023-11-09T03:47:51.422640+0000","last_undegraded":"2023-11-09T03:47:51.422640+0000","last_fullsized":"2023-11-09T03:47:51.422640+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:06:41.516591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.783141+0000","last_change":"2023-11-09T03:47:02.619336+0000","last_active":"2023-11-09T03:47:50.783141+0000","last_peered":"2023-11-09T03:47:50.783141+0000","last_clean":"2023-11-09T03:47:50.783141+0000","last_became_active":"2023-11-09T03:47:01.587749+0000","last_became_peered":"2023-11-09T03:47:01.587749+0000","last_unstale":"2023-11-09T03:47:50.783141+0000","last_undegraded":"2023-11-09T03:47:50.783141+0000","last_fullsized":"2023-11-09T03:47:50.783141+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063032099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.434012+0000","last_change":"2023-11-09T03:47:51.432431+0000","last_active":"2023-11-09T03:47:51.434012+0000","last_peered":"2023-11-09T03:47:51.434012+0000","last_clean":"2023-11-09T03:47:51.434012+0000","last_became_active":"2023-11-09T03:47:51.432164+0000","last_became_peered":"2023-11-09T03:47:51.432164+0000","last_unstale":"2023-11-09T03:47:51.434012+0000","last_undegraded":"2023-11-09T03:47:51.434012+0000","last_fullsized":"2023-11-09T03:47:51.434012+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:11:35.906808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"65'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.786911+0000","last_change":"2023-11-09T03:47:50.785328+0000","last_active":"2023-11-09T03:47:51.786911+0000","last_peered":"2023-11-09T03:47:51.786911+0000","last_clean":"2023-11-09T03:47:51.786911+0000","last_became_active":"2023-11-09T03:47:50.784280+0000","last_became_peered":"2023-11-09T03:47:50.784280+0000","last_unstale":"2023-11-09T03:47:51.786911+0000","last_undegraded":"2023-11-09T03:47:51.786911+0000","last_fullsized":"2023-11-09T03:47:51.786911+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:05:00.809894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.771140+0000","last_change":"2023-11-09T03:47:02.581658+0000","last_active":"2023-11-09T03:47:50.771140+0000","last_peered":"2023-11-09T03:47:50.771140+0000","last_clean":"2023-11-09T03:47:50.771140+0000","last_became_active":"2023-11-09T03:47:01.588612+0000","last_became_peered":"2023-11-09T03:47:01.588612+0000","last_unstale":"2023-11-09T03:47:50.771140+0000","last_undegraded":"2023-11-09T03:47:50.771140+0000","last_fullsized":"2023-11-09T03:47:50.771140+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045238500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.425900+0000","last_change":"2023-11-09T03:47:51.424216+0000","last_active":"2023-11-09T03:47:51.425900+0000","last_peered":"2023-11-09T03:47:51.425900+0000","last_clean":"2023-11-09T03:47:51.425900+0000","last_became_active":"2023-11-09T03:47:51.423782+0000","last_became_peered":"2023-11-09T03:47:51.423782+0000","last_unstale":"2023-11-09T03:47:51.425900+0000","last_undegraded":"2023-11-09T03:47:51.425900+0000","last_fullsized":"2023-11-09T03:47:51.425900+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:49:44.098168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"65'6","reported_seq":17,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.792705+0000","last_change":"2023-11-09T03:47:50.783046+0000","last_active":"2023-11-09T03:47:51.792705+0000","last_peered":"2023-11-09T03:47:51.792705+0000","last_clean":"2023-11-09T03:47:51.792705+0000","last_became_active":"2023-11-09T03:47:50.782715+0000","last_became_peered":"2023-11-09T03:47:50.782715+0000","last_unstale":"2023-11-09T03:47:51.792705+0000","last_undegraded":"2023-11-09T03:47:51.792705+0000","last_fullsized":"2023-11-09T03:47:51.792705+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:14:58.280559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.781573+0000","last_change":"2023-11-09T03:47:02.618828+0000","last_active":"2023-11-09T03:47:50.781573+0000","last_peered":"2023-11-09T03:47:50.781573+0000","last_clean":"2023-11-09T03:47:50.781573+0000","last_became_active":"2023-11-09T03:47:01.590679+0000","last_became_peered":"2023-11-09T03:47:01.590679+0000","last_unstale":"2023-11-09T03:47:50.781573+0000","last_undegraded":"2023-11-09T03:47:50.781573+0000","last_fullsized":"2023-11-09T03:47:50.781573+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:06:30.982987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070668,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.435787+0000","last_change":"2023-11-09T03:47:51.434706+0000","last_active":"2023-11-09T03:47:51.435787+0000","last_peered":"2023-11-09T03:47:51.435787+0000","last_clean":"2023-11-09T03:47:51.435787+0000","last_became_active":"2023-11-09T03:47:51.434251+0000","last_became_peered":"2023-11-09T03:47:51.434251+0000","last_unstale":"2023-11-09T03:47:51.435787+0000","last_undegraded":"2023-11-09T03:47:51.435787+0000","last_fullsized":"2023-11-09T03:47:51.435787+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:14:31.570502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"65'3","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.786663+0000","last_change":"2023-11-09T03:47:50.777327+0000","last_active":"2023-11-09T03:47:51.786663+0000","last_peered":"2023-11-09T03:47:51.786663+0000","last_clean":"2023-11-09T03:47:51.786663+0000","last_became_active":"2023-11-09T03:47:50.777076+0000","last_became_peered":"2023-11-09T03:47:50.777076+0000","last_unstale":"2023-11-09T03:47:51.786663+0000","last_undegraded":"2023-11-09T03:47:51.786663+0000","last_fullsized":"2023-11-09T03:47:51.786663+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:53:33.101892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":34,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.410485+0000","last_change":"2023-11-09T03:47:03.146528+0000","last_active":"2023-11-09T03:47:51.410485+0000","last_peered":"2023-11-09T03:47:51.410485+0000","last_clean":"2023-11-09T03:47:51.410485+0000","last_became_active":"2023-11-09T03:47:01.587060+0000","last_became_peered":"2023-11-09T03:47:01.587060+0000","last_unstale":"2023-11-09T03:47:51.410485+0000","last_undegraded":"2023-11-09T03:47:51.410485+0000","last_fullsized":"2023-11-09T03:47:51.410485+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:39:15.238496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079064999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.433820+0000","last_change":"2023-11-09T03:47:51.433039+0000","last_active":"2023-11-09T03:47:51.433820+0000","last_peered":"2023-11-09T03:47:51.433820+0000","last_clean":"2023-11-09T03:47:51.433820+0000","last_became_active":"2023-11-09T03:47:51.432780+0000","last_became_peered":"2023-11-09T03:47:51.432780+0000","last_unstale":"2023-11-09T03:47:51.433820+0000","last_undegraded":"2023-11-09T03:47:51.433820+0000","last_fullsized":"2023-11-09T03:47:51.433820+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:42:31.986727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.419179+0000","last_change":"2023-11-09T03:47:50.785375+0000","last_active":"2023-11-09T03:47:51.419179+0000","last_peered":"2023-11-09T03:47:51.419179+0000","last_clean":"2023-11-09T03:47:51.419179+0000","last_became_active":"2023-11-09T03:47:50.785090+0000","last_became_peered":"2023-11-09T03:47:50.785090+0000","last_unstale":"2023-11-09T03:47:51.419179+0000","last_undegraded":"2023-11-09T03:47:51.419179+0000","last_fullsized":"2023-11-09T03:47:51.419179+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:53:58.326231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":34,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.410680+0000","last_change":"2023-11-09T03:47:03.148246+0000","last_active":"2023-11-09T03:47:51.410680+0000","last_peered":"2023-11-09T03:47:51.410680+0000","last_clean":"2023-11-09T03:47:51.410680+0000","last_became_active":"2023-11-09T03:47:01.590988+0000","last_became_peered":"2023-11-09T03:47:01.590988+0000","last_unstale":"2023-11-09T03:47:51.410680+0000","last_undegraded":"2023-11-09T03:47:51.410680+0000","last_fullsized":"2023-11-09T03:47:51.410680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:07:20.785445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055651099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.434893+0000","last_change":"2023-11-09T03:47:51.433756+0000","last_active":"2023-11-09T03:47:51.434893+0000","last_peered":"2023-11-09T03:47:51.434893+0000","last_clean":"2023-11-09T03:47:51.434893+0000","last_became_active":"2023-11-09T03:47:51.432762+0000","last_became_peered":"2023-11-09T03:47:51.432762+0000","last_unstale":"2023-11-09T03:47:51.434893+0000","last_undegraded":"2023-11-09T03:47:51.434893+0000","last_fullsized":"2023-11-09T03:47:51.434893+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:21:20.038282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.413342+0000","last_change":"2023-11-09T03:47:50.780186+0000","last_active":"2023-11-09T03:47:52.413342+0000","last_peered":"2023-11-09T03:47:52.413342+0000","last_clean":"2023-11-09T03:47:52.413342+0000","last_became_active":"2023-11-09T03:47:50.779756+0000","last_became_peered":"2023-11-09T03:47:50.779756+0000","last_unstale":"2023-11-09T03:47:52.413342+0000","last_undegraded":"2023-11-09T03:47:52.413342+0000","last_fullsized":"2023-11-09T03:47:52.413342+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:41:14.757414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":36,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.413429+0000","last_change":"2023-11-09T03:47:02.582754+0000","last_active":"2023-11-09T03:47:51.413429+0000","last_peered":"2023-11-09T03:47:51.413429+0000","last_clean":"2023-11-09T03:47:51.413429+0000","last_became_active":"2023-11-09T03:47:01.589715+0000","last_became_peered":"2023-11-09T03:47:01.589715+0000","last_unstale":"2023-11-09T03:47:51.413429+0000","last_undegraded":"2023-11-09T03:47:51.413429+0000","last_fullsized":"2023-11-09T03:47:51.413429+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099342799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.414558+0000","last_change":"2023-11-09T03:47:51.421718+0000","last_active":"2023-11-09T03:47:52.414558+0000","last_peered":"2023-11-09T03:47:52.414558+0000","last_clean":"2023-11-09T03:47:52.414558+0000","last_became_active":"2023-11-09T03:47:51.421451+0000","last_became_peered":"2023-11-09T03:47:51.421451+0000","last_unstale":"2023-11-09T03:47:52.414558+0000","last_undegraded":"2023-11-09T03:47:52.414558+0000","last_fullsized":"2023-11-09T03:47:52.414558+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:01:59.285602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.413736+0000","last_change":"2023-11-09T03:47:50.779901+0000","last_active":"2023-11-09T03:47:51.413736+0000","last_peered":"2023-11-09T03:47:51.413736+0000","last_clean":"2023-11-09T03:47:51.413736+0000","last_became_active":"2023-11-09T03:47:50.778699+0000","last_became_peered":"2023-11-09T03:47:50.778699+0000","last_unstale":"2023-11-09T03:47:51.413736+0000","last_undegraded":"2023-11-09T03:47:51.413736+0000","last_fullsized":"2023-11-09T03:47:51.413736+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:48:57.422120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.774628+0000","last_change":"2023-11-09T03:47:02.724896+0000","last_active":"2023-11-09T03:47:50.774628+0000","last_peered":"2023-11-09T03:47:50.774628+0000","last_clean":"2023-11-09T03:47:50.774628+0000","last_became_active":"2023-11-09T03:47:01.592784+0000","last_became_peered":"2023-11-09T03:47:01.592784+0000","last_unstale":"2023-11-09T03:47:50.774628+0000","last_undegraded":"2023-11-09T03:47:50.774628+0000","last_fullsized":"2023-11-09T03:47:50.774628+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:51:32.016204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056683500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.432861+0000","last_change":"2023-11-09T03:47:51.430153+0000","last_active":"2023-11-09T03:47:51.432861+0000","last_peered":"2023-11-09T03:47:51.432861+0000","last_clean":"2023-11-09T03:47:51.432861+0000","last_became_active":"2023-11-09T03:47:51.429887+0000","last_became_peered":"2023-11-09T03:47:51.429887+0000","last_unstale":"2023-11-09T03:47:51.432861+0000","last_undegraded":"2023-11-09T03:47:51.432861+0000","last_fullsized":"2023-11-09T03:47:51.432861+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:17:04.496946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.413110+0000","last_change":"2023-11-09T03:47:50.785443+0000","last_active":"2023-11-09T03:47:51.413110+0000","last_peered":"2023-11-09T03:47:51.413110+0000","last_clean":"2023-11-09T03:47:51.413110+0000","last_became_active":"2023-11-09T03:47:50.784798+0000","last_became_peered":"2023-11-09T03:47:50.784798+0000","last_unstale":"2023-11-09T03:47:51.413110+0000","last_undegraded":"2023-11-09T03:47:51.413110+0000","last_fullsized":"2023-11-09T03:47:51.413110+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:59:00.398111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":35,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.765459+0000","last_change":"2023-11-09T03:47:03.146527+0000","last_active":"2023-11-09T03:47:50.765459+0000","last_peered":"2023-11-09T03:47:50.765459+0000","last_clean":"2023-11-09T03:47:50.765459+0000","last_became_active":"2023-11-09T03:47:01.587858+0000","last_became_peered":"2023-11-09T03:47:01.587858+0000","last_unstale":"2023-11-09T03:47:50.765459+0000","last_undegraded":"2023-11-09T03:47:50.765459+0000","last_fullsized":"2023-11-09T03:47:50.765459+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:56:04.201126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084439599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.413634+0000","last_change":"2023-11-09T03:47:51.424215+0000","last_active":"2023-11-09T03:47:52.413634+0000","last_peered":"2023-11-09T03:47:52.413634+0000","last_clean":"2023-11-09T03:47:52.413634+0000","last_became_active":"2023-11-09T03:47:51.423001+0000","last_became_peered":"2023-11-09T03:47:51.423001+0000","last_unstale":"2023-11-09T03:47:52.413634+0000","last_undegraded":"2023-11-09T03:47:52.413634+0000","last_fullsized":"2023-11-09T03:47:52.413634+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:13:45.340453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"65'1","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.785920+0000","last_change":"2023-11-09T03:47:50.782111+0000","last_active":"2023-11-09T03:47:51.785920+0000","last_peered":"2023-11-09T03:47:51.785920+0000","last_clean":"2023-11-09T03:47:51.785920+0000","last_became_active":"2023-11-09T03:47:50.781833+0000","last_became_peered":"2023-11-09T03:47:50.781833+0000","last_unstale":"2023-11-09T03:47:51.785920+0000","last_undegraded":"2023-11-09T03:47:51.785920+0000","last_fullsized":"2023-11-09T03:47:51.785920+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:04:35.154546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.776045+0000","last_change":"2023-11-09T03:47:02.725223+0000","last_active":"2023-11-09T03:47:50.776045+0000","last_peered":"2023-11-09T03:47:50.776045+0000","last_clean":"2023-11-09T03:47:50.776045+0000","last_became_active":"2023-11-09T03:47:01.590635+0000","last_became_peered":"2023-11-09T03:47:01.590635+0000","last_unstale":"2023-11-09T03:47:50.776045+0000","last_undegraded":"2023-11-09T03:47:50.776045+0000","last_fullsized":"2023-11-09T03:47:50.776045+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:05.754357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051685100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.428631+0000","last_change":"2023-11-09T03:47:51.425520+0000","last_active":"2023-11-09T03:47:51.428631+0000","last_peered":"2023-11-09T03:47:51.428631+0000","last_clean":"2023-11-09T03:47:51.428631+0000","last_became_active":"2023-11-09T03:47:51.425239+0000","last_became_peered":"2023-11-09T03:47:51.425239+0000","last_unstale":"2023-11-09T03:47:51.428631+0000","last_undegraded":"2023-11-09T03:47:51.428631+0000","last_fullsized":"2023-11-09T03:47:51.428631+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:11:35.906808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"65'1","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.785787+0000","last_change":"2023-11-09T03:47:50.778115+0000","last_active":"2023-11-09T03:47:51.785787+0000","last_peered":"2023-11-09T03:47:51.785787+0000","last_clean":"2023-11-09T03:47:51.785787+0000","last_became_active":"2023-11-09T03:47:50.777666+0000","last_became_peered":"2023-11-09T03:47:50.777666+0000","last_unstale":"2023-11-09T03:47:51.785787+0000","last_undegraded":"2023-11-09T03:47:51.785787+0000","last_fullsized":"2023-11-09T03:47:51.785787+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:19:44.308319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":36,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.414324+0000","last_change":"2023-11-09T03:47:02.582739+0000","last_active":"2023-11-09T03:47:51.414324+0000","last_peered":"2023-11-09T03:47:51.414324+0000","last_clean":"2023-11-09T03:47:51.414324+0000","last_became_active":"2023-11-09T03:47:01.592743+0000","last_became_peered":"2023-11-09T03:47:01.592743+0000","last_unstale":"2023-11-09T03:47:51.414324+0000","last_undegraded":"2023-11-09T03:47:51.414324+0000","last_fullsized":"2023-11-09T03:47:51.414324+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051802700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.430708+0000","last_change":"2023-11-09T03:47:51.425934+0000","last_active":"2023-11-09T03:47:51.430708+0000","last_peered":"2023-11-09T03:47:51.430708+0000","last_clean":"2023-11-09T03:47:51.430708+0000","last_became_active":"2023-11-09T03:47:51.425667+0000","last_became_peered":"2023-11-09T03:47:51.425667+0000","last_unstale":"2023-11-09T03:47:51.430708+0000","last_undegraded":"2023-11-09T03:47:51.430708+0000","last_fullsized":"2023-11-09T03:47:51.430708+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:32:30.681190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"65'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.786124+0000","last_change":"2023-11-09T03:47:50.783099+0000","last_active":"2023-11-09T03:47:51.786124+0000","last_peered":"2023-11-09T03:47:51.786124+0000","last_clean":"2023-11-09T03:47:51.786124+0000","last_became_active":"2023-11-09T03:47:50.782082+0000","last_became_peered":"2023-11-09T03:47:50.782082+0000","last_unstale":"2023-11-09T03:47:51.786124+0000","last_undegraded":"2023-11-09T03:47:51.786124+0000","last_fullsized":"2023-11-09T03:47:51.786124+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:04:48.179843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.773010+0000","last_change":"2023-11-09T03:47:02.724953+0000","last_active":"2023-11-09T03:47:50.773010+0000","last_peered":"2023-11-09T03:47:50.773010+0000","last_clean":"2023-11-09T03:47:50.773010+0000","last_became_active":"2023-11-09T03:47:01.587133+0000","last_became_peered":"2023-11-09T03:47:01.587133+0000","last_unstale":"2023-11-09T03:47:50.773010+0000","last_undegraded":"2023-11-09T03:47:50.773010+0000","last_fullsized":"2023-11-09T03:47:50.773010+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:23:48.521161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066497599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.438370+0000","last_change":"2023-11-09T03:47:51.436760+0000","last_active":"2023-11-09T03:47:51.438370+0000","last_peered":"2023-11-09T03:47:51.438370+0000","last_clean":"2023-11-09T03:47:51.438370+0000","last_became_active":"2023-11-09T03:47:51.436265+0000","last_became_peered":"2023-11-09T03:47:51.436265+0000","last_unstale":"2023-11-09T03:47:51.438370+0000","last_undegraded":"2023-11-09T03:47:51.438370+0000","last_fullsized":"2023-11-09T03:47:51.438370+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:57:53.737390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"65'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.413573+0000","last_change":"2023-11-09T03:47:50.786349+0000","last_active":"2023-11-09T03:47:52.413573+0000","last_peered":"2023-11-09T03:47:52.413573+0000","last_clean":"2023-11-09T03:47:52.413573+0000","last_became_active":"2023-11-09T03:47:50.786079+0000","last_became_peered":"2023-11-09T03:47:50.786079+0000","last_unstale":"2023-11-09T03:47:52.413573+0000","last_undegraded":"2023-11-09T03:47:52.413573+0000","last_fullsized":"2023-11-09T03:47:52.413573+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:56:24.172800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":35,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.765230+0000","last_change":"2023-11-09T03:47:03.146217+0000","last_active":"2023-11-09T03:47:50.765230+0000","last_peered":"2023-11-09T03:47:50.765230+0000","last_clean":"2023-11-09T03:47:50.765230+0000","last_became_active":"2023-11-09T03:47:01.587598+0000","last_became_peered":"2023-11-09T03:47:01.587598+0000","last_unstale":"2023-11-09T03:47:50.765230+0000","last_undegraded":"2023-11-09T03:47:50.765230+0000","last_fullsized":"2023-11-09T03:47:50.765230+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:15:22.723455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090126400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.423912+0000","last_change":"2023-11-09T03:47:51.421511+0000","last_active":"2023-11-09T03:47:51.423912+0000","last_peered":"2023-11-09T03:47:51.423912+0000","last_clean":"2023-11-09T03:47:51.423912+0000","last_became_active":"2023-11-09T03:47:51.421262+0000","last_became_peered":"2023-11-09T03:47:51.421262+0000","last_unstale":"2023-11-09T03:47:51.423912+0000","last_undegraded":"2023-11-09T03:47:51.423912+0000","last_fullsized":"2023-11-09T03:47:51.423912+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:38:40.522966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"65'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.414900+0000","last_change":"2023-11-09T03:47:50.783271+0000","last_active":"2023-11-09T03:47:52.414900+0000","last_peered":"2023-11-09T03:47:52.414900+0000","last_clean":"2023-11-09T03:47:52.414900+0000","last_became_active":"2023-11-09T03:47:50.782546+0000","last_became_peered":"2023-11-09T03:47:50.782546+0000","last_unstale":"2023-11-09T03:47:52.414900+0000","last_undegraded":"2023-11-09T03:47:52.414900+0000","last_fullsized":"2023-11-09T03:47:52.414900+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:21:19.032046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":34,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.410784+0000","last_change":"2023-11-09T03:47:03.148283+0000","last_active":"2023-11-09T03:47:51.410784+0000","last_peered":"2023-11-09T03:47:51.410784+0000","last_clean":"2023-11-09T03:47:51.410784+0000","last_became_active":"2023-11-09T03:47:01.588841+0000","last_became_peered":"2023-11-09T03:47:01.588841+0000","last_unstale":"2023-11-09T03:47:51.410784+0000","last_undegraded":"2023-11-09T03:47:51.410784+0000","last_fullsized":"2023-11-09T03:47:51.410784+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:47:44.880741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000436081,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.432369+0000","last_change":"2023-11-09T03:47:51.430694+0000","last_active":"2023-11-09T03:47:51.432369+0000","last_peered":"2023-11-09T03:47:51.432369+0000","last_clean":"2023-11-09T03:47:51.432369+0000","last_became_active":"2023-11-09T03:47:51.430302+0000","last_became_peered":"2023-11-09T03:47:51.430302+0000","last_unstale":"2023-11-09T03:47:51.432369+0000","last_undegraded":"2023-11-09T03:47:51.432369+0000","last_fullsized":"2023-11-09T03:47:51.432369+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:11:17.062291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"65'4","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.789961+0000","last_change":"2023-11-09T03:47:50.783769+0000","last_active":"2023-11-09T03:47:51.789961+0000","last_peered":"2023-11-09T03:47:51.789961+0000","last_clean":"2023-11-09T03:47:51.789961+0000","last_became_active":"2023-11-09T03:47:50.783497+0000","last_became_peered":"2023-11-09T03:47:50.783497+0000","last_unstale":"2023-11-09T03:47:51.789961+0000","last_undegraded":"2023-11-09T03:47:51.789961+0000","last_fullsized":"2023-11-09T03:47:51.789961+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:36:59.189560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.781364+0000","last_change":"2023-11-09T03:47:02.618856+0000","last_active":"2023-11-09T03:47:50.781364+0000","last_peered":"2023-11-09T03:47:50.781364+0000","last_clean":"2023-11-09T03:47:50.781364+0000","last_became_active":"2023-11-09T03:47:01.590430+0000","last_became_peered":"2023-11-09T03:47:01.590430+0000","last_unstale":"2023-11-09T03:47:50.781364+0000","last_undegraded":"2023-11-09T03:47:50.781364+0000","last_fullsized":"2023-11-09T03:47:50.781364+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:57:18.470305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055030200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.426831+0000","last_change":"2023-11-09T03:47:51.425418+0000","last_active":"2023-11-09T03:47:51.426831+0000","last_peered":"2023-11-09T03:47:51.426831+0000","last_clean":"2023-11-09T03:47:51.426831+0000","last_became_active":"2023-11-09T03:47:51.425164+0000","last_became_peered":"2023-11-09T03:47:51.425164+0000","last_unstale":"2023-11-09T03:47:51.426831+0000","last_undegraded":"2023-11-09T03:47:51.426831+0000","last_fullsized":"2023-11-09T03:47:51.426831+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:06:03.508296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.422207+0000","last_change":"2023-11-09T03:47:50.786002+0000","last_active":"2023-11-09T03:47:51.422207+0000","last_peered":"2023-11-09T03:47:51.422207+0000","last_clean":"2023-11-09T03:47:51.422207+0000","last_became_active":"2023-11-09T03:47:50.785336+0000","last_became_peered":"2023-11-09T03:47:50.785336+0000","last_unstale":"2023-11-09T03:47:51.422207+0000","last_undegraded":"2023-11-09T03:47:51.422207+0000","last_fullsized":"2023-11-09T03:47:51.422207+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:00:04.578075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.781812+0000","last_change":"2023-11-09T03:47:02.619144+0000","last_active":"2023-11-09T03:47:50.781812+0000","last_peered":"2023-11-09T03:47:50.781812+0000","last_clean":"2023-11-09T03:47:50.781812+0000","last_became_active":"2023-11-09T03:47:01.590672+0000","last_became_peered":"2023-11-09T03:47:01.590672+0000","last_unstale":"2023-11-09T03:47:50.781812+0000","last_undegraded":"2023-11-09T03:47:50.781812+0000","last_fullsized":"2023-11-09T03:47:50.781812+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:03.025119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060997500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.426561+0000","last_change":"2023-11-09T03:47:51.423417+0000","last_active":"2023-11-09T03:47:51.426561+0000","last_peered":"2023-11-09T03:47:51.426561+0000","last_clean":"2023-11-09T03:47:51.426561+0000","last_became_active":"2023-11-09T03:47:51.423176+0000","last_became_peered":"2023-11-09T03:47:51.423176+0000","last_unstale":"2023-11-09T03:47:51.426561+0000","last_undegraded":"2023-11-09T03:47:51.426561+0000","last_fullsized":"2023-11-09T03:47:51.426561+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:57:53.737390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.413416+0000","last_change":"2023-11-09T03:47:50.775788+0000","last_active":"2023-11-09T03:47:52.413416+0000","last_peered":"2023-11-09T03:47:52.413416+0000","last_clean":"2023-11-09T03:47:52.413416+0000","last_became_active":"2023-11-09T03:47:50.775509+0000","last_became_peered":"2023-11-09T03:47:50.775509+0000","last_unstale":"2023-11-09T03:47:52.413416+0000","last_undegraded":"2023-11-09T03:47:52.413416+0000","last_fullsized":"2023-11-09T03:47:52.413416+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:06:41.516591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.781724+0000","last_change":"2023-11-09T03:47:02.619079+0000","last_active":"2023-11-09T03:47:50.781724+0000","last_peered":"2023-11-09T03:47:50.781724+0000","last_clean":"2023-11-09T03:47:50.781724+0000","last_became_active":"2023-11-09T03:47:01.591421+0000","last_became_peered":"2023-11-09T03:47:01.591421+0000","last_unstale":"2023-11-09T03:47:50.781724+0000","last_undegraded":"2023-11-09T03:47:50.781724+0000","last_fullsized":"2023-11-09T03:47:50.781724+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:18:58.230606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068736399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.435687+0000","last_change":"2023-11-09T03:47:51.433858+0000","last_active":"2023-11-09T03:47:51.435687+0000","last_peered":"2023-11-09T03:47:51.435687+0000","last_clean":"2023-11-09T03:47:51.435687+0000","last_became_active":"2023-11-09T03:47:51.433561+0000","last_became_peered":"2023-11-09T03:47:51.433561+0000","last_unstale":"2023-11-09T03:47:51.435687+0000","last_undegraded":"2023-11-09T03:47:51.435687+0000","last_fullsized":"2023-11-09T03:47:51.435687+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:07:03.975863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.413057+0000","last_change":"2023-11-09T03:47:50.785136+0000","last_active":"2023-11-09T03:47:51.413057+0000","last_peered":"2023-11-09T03:47:51.413057+0000","last_clean":"2023-11-09T03:47:51.413057+0000","last_became_active":"2023-11-09T03:47:50.784764+0000","last_became_peered":"2023-11-09T03:47:50.784764+0000","last_unstale":"2023-11-09T03:47:51.413057+0000","last_undegraded":"2023-11-09T03:47:51.413057+0000","last_fullsized":"2023-11-09T03:47:51.413057+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:17.964253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":34,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.411240+0000","last_change":"2023-11-09T03:47:03.148529+0000","last_active":"2023-11-09T03:47:51.411240+0000","last_peered":"2023-11-09T03:47:51.411240+0000","last_clean":"2023-11-09T03:47:51.411240+0000","last_became_active":"2023-11-09T03:47:01.584148+0000","last_became_peered":"2023-11-09T03:47:01.584148+0000","last_unstale":"2023-11-09T03:47:51.411240+0000","last_undegraded":"2023-11-09T03:47:51.411240+0000","last_fullsized":"2023-11-09T03:47:51.411240+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:40:00.237248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061475600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.424192+0000","last_change":"2023-11-09T03:47:51.422333+0000","last_active":"2023-11-09T03:47:51.424192+0000","last_peered":"2023-11-09T03:47:51.424192+0000","last_clean":"2023-11-09T03:47:51.424192+0000","last_became_active":"2023-11-09T03:47:51.421811+0000","last_became_peered":"2023-11-09T03:47:51.421811+0000","last_unstale":"2023-11-09T03:47:51.424192+0000","last_undegraded":"2023-11-09T03:47:51.424192+0000","last_fullsized":"2023-11-09T03:47:51.424192+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:05:37.375241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.415322+0000","last_change":"2023-11-09T03:47:50.782177+0000","last_active":"2023-11-09T03:47:51.415322+0000","last_peered":"2023-11-09T03:47:51.415322+0000","last_clean":"2023-11-09T03:47:51.415322+0000","last_became_active":"2023-11-09T03:47:50.781308+0000","last_became_peered":"2023-11-09T03:47:50.781308+0000","last_unstale":"2023-11-09T03:47:51.415322+0000","last_undegraded":"2023-11-09T03:47:51.415322+0000","last_fullsized":"2023-11-09T03:47:51.415322+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:43:39.503427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.771407+0000","last_change":"2023-11-09T03:47:03.148217+0000","last_active":"2023-11-09T03:47:50.771407+0000","last_peered":"2023-11-09T03:47:50.771407+0000","last_clean":"2023-11-09T03:47:50.771407+0000","last_became_active":"2023-11-09T03:47:01.593424+0000","last_became_peered":"2023-11-09T03:47:01.593424+0000","last_unstale":"2023-11-09T03:47:50.771407+0000","last_undegraded":"2023-11-09T03:47:50.771407+0000","last_fullsized":"2023-11-09T03:47:50.771407+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:19:32.693691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001769865,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.430726+0000","last_change":"2023-11-09T03:47:51.427859+0000","last_active":"2023-11-09T03:47:51.430726+0000","last_peered":"2023-11-09T03:47:51.430726+0000","last_clean":"2023-11-09T03:47:51.430726+0000","last_became_active":"2023-11-09T03:47:51.427608+0000","last_became_peered":"2023-11-09T03:47:51.427608+0000","last_unstale":"2023-11-09T03:47:51.430726+0000","last_undegraded":"2023-11-09T03:47:51.430726+0000","last_fullsized":"2023-11-09T03:47:51.430726+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:01:59.285602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.414730+0000","last_change":"2023-11-09T03:47:50.780364+0000","last_active":"2023-11-09T03:47:52.414730+0000","last_peered":"2023-11-09T03:47:52.414730+0000","last_clean":"2023-11-09T03:47:52.414730+0000","last_became_active":"2023-11-09T03:47:50.780015+0000","last_became_peered":"2023-11-09T03:47:50.780015+0000","last_unstale":"2023-11-09T03:47:52.414730+0000","last_undegraded":"2023-11-09T03:47:52.414730+0000","last_fullsized":"2023-11-09T03:47:52.414730+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:38:51.283678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":34,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.411140+0000","last_change":"2023-11-09T03:47:03.146470+0000","last_active":"2023-11-09T03:47:51.411140+0000","last_peered":"2023-11-09T03:47:51.411140+0000","last_clean":"2023-11-09T03:47:51.411140+0000","last_became_active":"2023-11-09T03:47:01.587304+0000","last_became_peered":"2023-11-09T03:47:01.587304+0000","last_unstale":"2023-11-09T03:47:51.411140+0000","last_undegraded":"2023-11-09T03:47:51.411140+0000","last_fullsized":"2023-11-09T03:47:51.411140+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:58:23.339287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00110395,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.437478+0000","last_change":"2023-11-09T03:47:51.435550+0000","last_active":"2023-11-09T03:47:51.437478+0000","last_peered":"2023-11-09T03:47:51.437478+0000","last_clean":"2023-11-09T03:47:51.437478+0000","last_became_active":"2023-11-09T03:47:51.435197+0000","last_became_peered":"2023-11-09T03:47:51.435197+0000","last_unstale":"2023-11-09T03:47:51.437478+0000","last_undegraded":"2023-11-09T03:47:51.437478+0000","last_fullsized":"2023-11-09T03:47:51.437478+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:52:03.847868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.415203+0000","last_change":"2023-11-09T03:47:50.782930+0000","last_active":"2023-11-09T03:47:51.415203+0000","last_peered":"2023-11-09T03:47:51.415203+0000","last_clean":"2023-11-09T03:47:51.415203+0000","last_became_active":"2023-11-09T03:47:50.782659+0000","last_became_peered":"2023-11-09T03:47:50.782659+0000","last_unstale":"2023-11-09T03:47:51.415203+0000","last_undegraded":"2023-11-09T03:47:51.415203+0000","last_fullsized":"2023-11-09T03:47:51.415203+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:47:24.099855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.770672+0000","last_change":"2023-11-09T03:47:03.147367+0000","last_active":"2023-11-09T03:47:50.770672+0000","last_peered":"2023-11-09T03:47:50.770672+0000","last_clean":"2023-11-09T03:47:50.770672+0000","last_became_active":"2023-11-09T03:47:01.588359+0000","last_became_peered":"2023-11-09T03:47:01.588359+0000","last_unstale":"2023-11-09T03:47:50.770672+0000","last_undegraded":"2023-11-09T03:47:50.770672+0000","last_fullsized":"2023-11-09T03:47:50.770672+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:21:34.303068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014106660000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.431545+0000","last_change":"2023-11-09T03:47:51.428133+0000","last_active":"2023-11-09T03:47:51.431545+0000","last_peered":"2023-11-09T03:47:51.431545+0000","last_clean":"2023-11-09T03:47:51.431545+0000","last_became_active":"2023-11-09T03:47:51.427786+0000","last_became_peered":"2023-11-09T03:47:51.427786+0000","last_unstale":"2023-11-09T03:47:51.431545+0000","last_undegraded":"2023-11-09T03:47:51.431545+0000","last_fullsized":"2023-11-09T03:47:51.431545+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:52:03.847868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.422000+0000","last_change":"2023-11-09T03:47:50.781009+0000","last_active":"2023-11-09T03:47:51.422000+0000","last_peered":"2023-11-09T03:47:51.422000+0000","last_clean":"2023-11-09T03:47:51.422000+0000","last_became_active":"2023-11-09T03:47:50.780172+0000","last_became_peered":"2023-11-09T03:47:50.780172+0000","last_unstale":"2023-11-09T03:47:51.422000+0000","last_undegraded":"2023-11-09T03:47:51.422000+0000","last_fullsized":"2023-11-09T03:47:51.422000+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:01:58.279366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.774013+0000","last_change":"2023-11-09T03:47:02.581682+0000","last_active":"2023-11-09T03:47:50.774013+0000","last_peered":"2023-11-09T03:47:50.774013+0000","last_clean":"2023-11-09T03:47:50.774013+0000","last_became_active":"2023-11-09T03:47:01.593051+0000","last_became_peered":"2023-11-09T03:47:01.593051+0000","last_unstale":"2023-11-09T03:47:50.774013+0000","last_undegraded":"2023-11-09T03:47:50.774013+0000","last_fullsized":"2023-11-09T03:47:50.774013+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047277799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.414917+0000","last_change":"2023-11-09T03:47:51.419244+0000","last_active":"2023-11-09T03:47:52.414917+0000","last_peered":"2023-11-09T03:47:52.414917+0000","last_clean":"2023-11-09T03:47:52.414917+0000","last_became_active":"2023-11-09T03:47:51.418983+0000","last_became_peered":"2023-11-09T03:47:51.418983+0000","last_unstale":"2023-11-09T03:47:52.414917+0000","last_undegraded":"2023-11-09T03:47:52.414917+0000","last_fullsized":"2023-11-09T03:47:52.414917+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:06:27.135244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"65'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.786082+0000","last_change":"2023-11-09T03:47:50.778993+0000","last_active":"2023-11-09T03:47:51.786082+0000","last_peered":"2023-11-09T03:47:51.786082+0000","last_clean":"2023-11-09T03:47:51.786082+0000","last_became_active":"2023-11-09T03:47:50.778713+0000","last_became_peered":"2023-11-09T03:47:50.778713+0000","last_unstale":"2023-11-09T03:47:51.786082+0000","last_undegraded":"2023-11-09T03:47:51.786082+0000","last_fullsized":"2023-11-09T03:47:51.786082+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:53:33.810634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":35,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.765274+0000","last_change":"2023-11-09T03:47:03.146309+0000","last_active":"2023-11-09T03:47:50.765274+0000","last_peered":"2023-11-09T03:47:50.765274+0000","last_clean":"2023-11-09T03:47:50.765274+0000","last_became_active":"2023-11-09T03:47:01.587560+0000","last_became_peered":"2023-11-09T03:47:01.587560+0000","last_unstale":"2023-11-09T03:47:50.765274+0000","last_undegraded":"2023-11-09T03:47:50.765274+0000","last_fullsized":"2023-11-09T03:47:50.765274+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:27:55.271945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070988400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.428243+0000","last_change":"2023-11-09T03:47:51.426419+0000","last_active":"2023-11-09T03:47:51.428243+0000","last_peered":"2023-11-09T03:47:51.428243+0000","last_clean":"2023-11-09T03:47:51.428243+0000","last_became_active":"2023-11-09T03:47:51.426178+0000","last_became_peered":"2023-11-09T03:47:51.426178+0000","last_unstale":"2023-11-09T03:47:51.428243+0000","last_undegraded":"2023-11-09T03:47:51.428243+0000","last_fullsized":"2023-11-09T03:47:51.428243+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:51:09.098748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.414443+0000","last_change":"2023-11-09T03:47:50.783293+0000","last_active":"2023-11-09T03:47:52.414443+0000","last_peered":"2023-11-09T03:47:52.414443+0000","last_clean":"2023-11-09T03:47:52.414443+0000","last_became_active":"2023-11-09T03:47:50.782667+0000","last_became_peered":"2023-11-09T03:47:50.782667+0000","last_unstale":"2023-11-09T03:47:52.414443+0000","last_undegraded":"2023-11-09T03:47:52.414443+0000","last_fullsized":"2023-11-09T03:47:52.414443+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:11:34.900572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.774708+0000","last_change":"2023-11-09T03:47:02.725125+0000","last_active":"2023-11-09T03:47:50.774708+0000","last_peered":"2023-11-09T03:47:50.774708+0000","last_clean":"2023-11-09T03:47:50.774708+0000","last_became_active":"2023-11-09T03:47:01.591521+0000","last_became_peered":"2023-11-09T03:47:01.591521+0000","last_unstale":"2023-11-09T03:47:50.774708+0000","last_undegraded":"2023-11-09T03:47:50.774708+0000","last_fullsized":"2023-11-09T03:47:50.774708+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:52:40.076542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000758675,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.428252+0000","last_change":"2023-11-09T03:47:51.426821+0000","last_active":"2023-11-09T03:47:51.428252+0000","last_peered":"2023-11-09T03:47:51.428252+0000","last_clean":"2023-11-09T03:47:51.428252+0000","last_became_active":"2023-11-09T03:47:51.426516+0000","last_became_peered":"2023-11-09T03:47:51.426516+0000","last_unstale":"2023-11-09T03:47:51.428252+0000","last_undegraded":"2023-11-09T03:47:51.428252+0000","last_fullsized":"2023-11-09T03:47:51.428252+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:15:26.625990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.421152+0000","last_change":"2023-11-09T03:47:50.782705+0000","last_active":"2023-11-09T03:47:51.421152+0000","last_peered":"2023-11-09T03:47:51.421152+0000","last_clean":"2023-11-09T03:47:51.421152+0000","last_became_active":"2023-11-09T03:47:50.782427+0000","last_became_peered":"2023-11-09T03:47:50.782427+0000","last_unstale":"2023-11-09T03:47:51.421152+0000","last_undegraded":"2023-11-09T03:47:51.421152+0000","last_fullsized":"2023-11-09T03:47:51.421152+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:36:59.189560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":34,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.410970+0000","last_change":"2023-11-09T03:47:03.147289+0000","last_active":"2023-11-09T03:47:51.410970+0000","last_peered":"2023-11-09T03:47:51.410970+0000","last_clean":"2023-11-09T03:47:51.410970+0000","last_became_active":"2023-11-09T03:47:01.587784+0000","last_became_peered":"2023-11-09T03:47:01.587784+0000","last_unstale":"2023-11-09T03:47:51.410970+0000","last_undegraded":"2023-11-09T03:47:51.410970+0000","last_fullsized":"2023-11-09T03:47:51.410970+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:11:23.824112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038781299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.424282+0000","last_change":"2023-11-09T03:47:51.421317+0000","last_active":"2023-11-09T03:47:51.424282+0000","last_peered":"2023-11-09T03:47:51.424282+0000","last_clean":"2023-11-09T03:47:51.424282+0000","last_became_active":"2023-11-09T03:47:51.421044+0000","last_became_peered":"2023-11-09T03:47:51.421044+0000","last_unstale":"2023-11-09T03:47:51.424282+0000","last_undegraded":"2023-11-09T03:47:51.424282+0000","last_fullsized":"2023-11-09T03:47:51.424282+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:50:46.643929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.422774+0000","last_change":"2023-11-09T03:47:50.783770+0000","last_active":"2023-11-09T03:47:51.422774+0000","last_peered":"2023-11-09T03:47:51.422774+0000","last_clean":"2023-11-09T03:47:51.422774+0000","last_became_active":"2023-11-09T03:47:50.781901+0000","last_became_peered":"2023-11-09T03:47:50.781901+0000","last_unstale":"2023-11-09T03:47:51.422774+0000","last_undegraded":"2023-11-09T03:47:51.422774+0000","last_fullsized":"2023-11-09T03:47:51.422774+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:14:58.280559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":34,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.783243+0000","last_change":"2023-11-09T03:47:02.619346+0000","last_active":"2023-11-09T03:47:50.783243+0000","last_peered":"2023-11-09T03:47:50.783243+0000","last_clean":"2023-11-09T03:47:50.783243+0000","last_became_active":"2023-11-09T03:47:01.590389+0000","last_became_peered":"2023-11-09T03:47:01.590389+0000","last_unstale":"2023-11-09T03:47:50.783243+0000","last_undegraded":"2023-11-09T03:47:50.783243+0000","last_fullsized":"2023-11-09T03:47:50.783243+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000384072,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.435734+0000","last_change":"2023-11-09T03:47:51.431055+0000","last_active":"2023-11-09T03:47:51.435734+0000","last_peered":"2023-11-09T03:47:51.435734+0000","last_clean":"2023-11-09T03:47:51.435734+0000","last_became_active":"2023-11-09T03:47:51.430848+0000","last_became_peered":"2023-11-09T03:47:51.430848+0000","last_unstale":"2023-11-09T03:47:51.435734+0000","last_undegraded":"2023-11-09T03:47:51.435734+0000","last_fullsized":"2023-11-09T03:47:51.435734+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:02:44.463883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.416938+0000","last_change":"2023-11-09T03:47:50.783797+0000","last_active":"2023-11-09T03:47:51.416938+0000","last_peered":"2023-11-09T03:47:51.416938+0000","last_clean":"2023-11-09T03:47:51.416938+0000","last_became_active":"2023-11-09T03:47:50.782590+0000","last_became_peered":"2023-11-09T03:47:50.782590+0000","last_unstale":"2023-11-09T03:47:51.416938+0000","last_undegraded":"2023-11-09T03:47:51.416938+0000","last_fullsized":"2023-11-09T03:47:51.416938+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:00:16.605114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":36,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.413841+0000","last_change":"2023-11-09T03:47:02.582393+0000","last_active":"2023-11-09T03:47:51.413841+0000","last_peered":"2023-11-09T03:47:51.413841+0000","last_clean":"2023-11-09T03:47:51.413841+0000","last_became_active":"2023-11-09T03:47:01.591752+0000","last_became_peered":"2023-11-09T03:47:01.591752+0000","last_unstale":"2023-11-09T03:47:51.413841+0000","last_undegraded":"2023-11-09T03:47:51.413841+0000","last_fullsized":"2023-11-09T03:47:51.413841+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:03.025119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067021599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.424184+0000","last_change":"2023-11-09T03:47:51.422732+0000","last_active":"2023-11-09T03:47:51.424184+0000","last_peered":"2023-11-09T03:47:51.424184+0000","last_clean":"2023-11-09T03:47:51.424184+0000","last_became_active":"2023-11-09T03:47:51.422448+0000","last_became_peered":"2023-11-09T03:47:51.422448+0000","last_unstale":"2023-11-09T03:47:51.424184+0000","last_undegraded":"2023-11-09T03:47:51.424184+0000","last_fullsized":"2023-11-09T03:47:51.424184+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:04:49.186079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"65'1","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.786064+0000","last_change":"2023-11-09T03:47:50.785733+0000","last_active":"2023-11-09T03:47:51.786064+0000","last_peered":"2023-11-09T03:47:51.786064+0000","last_clean":"2023-11-09T03:47:51.786064+0000","last_became_active":"2023-11-09T03:47:50.785453+0000","last_became_peered":"2023-11-09T03:47:50.785453+0000","last_unstale":"2023-11-09T03:47:51.786064+0000","last_undegraded":"2023-11-09T03:47:51.786064+0000","last_fullsized":"2023-11-09T03:47:51.786064+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:24:20.075108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":33,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.772682+0000","last_change":"2023-11-09T03:47:03.028744+0000","last_active":"2023-11-09T03:47:50.772682+0000","last_peered":"2023-11-09T03:47:50.772682+0000","last_clean":"2023-11-09T03:47:50.772682+0000","last_became_active":"2023-11-09T03:47:01.593890+0000","last_became_peered":"2023-11-09T03:47:01.593890+0000","last_unstale":"2023-11-09T03:47:50.772682+0000","last_undegraded":"2023-11-09T03:47:50.772682+0000","last_fullsized":"2023-11-09T03:47:50.772682+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:05.754357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053008299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.431416+0000","last_change":"2023-11-09T03:47:51.428956+0000","last_active":"2023-11-09T03:47:51.431416+0000","last_peered":"2023-11-09T03:47:51.431416+0000","last_clean":"2023-11-09T03:47:51.431416+0000","last_became_active":"2023-11-09T03:47:51.428694+0000","last_became_peered":"2023-11-09T03:47:51.428694+0000","last_unstale":"2023-11-09T03:47:51.431416+0000","last_undegraded":"2023-11-09T03:47:51.431416+0000","last_fullsized":"2023-11-09T03:47:51.431416+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:45:00.860943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"65'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.413334+0000","last_change":"2023-11-09T03:47:50.786278+0000","last_active":"2023-11-09T03:47:52.413334+0000","last_peered":"2023-11-09T03:47:52.413334+0000","last_clean":"2023-11-09T03:47:52.413334+0000","last_became_active":"2023-11-09T03:47:50.786018+0000","last_became_peered":"2023-11-09T03:47:50.786018+0000","last_unstale":"2023-11-09T03:47:52.413334+0000","last_undegraded":"2023-11-09T03:47:52.413334+0000","last_fullsized":"2023-11-09T03:47:52.413334+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:22:44.272276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":35,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.765378+0000","last_change":"2023-11-09T03:47:03.146783+0000","last_active":"2023-11-09T03:47:50.765378+0000","last_peered":"2023-11-09T03:47:50.765378+0000","last_clean":"2023-11-09T03:47:50.765378+0000","last_became_active":"2023-11-09T03:47:01.586923+0000","last_became_peered":"2023-11-09T03:47:01.586923+0000","last_unstale":"2023-11-09T03:47:50.765378+0000","last_undegraded":"2023-11-09T03:47:50.765378+0000","last_fullsized":"2023-11-09T03:47:50.765378+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:37:50.326442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060384700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-09T03:47:52.413326+0000","last_change":"2023-11-09T03:47:51.423642+0000","last_active":"2023-11-09T03:47:52.413326+0000","last_peered":"2023-11-09T03:47:52.413326+0000","last_clean":"2023-11-09T03:47:52.413326+0000","last_became_active":"2023-11-09T03:47:51.423363+0000","last_became_peered":"2023-11-09T03:47:51.423363+0000","last_unstale":"2023-11-09T03:47:52.413326+0000","last_undegraded":"2023-11-09T03:47:52.413326+0000","last_fullsized":"2023-11-09T03:47:52.413326+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:26:42.350849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"65'1","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.786002+0000","last_change":"2023-11-09T03:47:50.786181+0000","last_active":"2023-11-09T03:47:51.786002+0000","last_peered":"2023-11-09T03:47:51.786002+0000","last_clean":"2023-11-09T03:47:51.786002+0000","last_became_active":"2023-11-09T03:47:50.785921+0000","last_became_peered":"2023-11-09T03:47:50.785921+0000","last_unstale":"2023-11-09T03:47:51.786002+0000","last_undegraded":"2023-11-09T03:47:51.786002+0000","last_fullsized":"2023-11-09T03:47:51.786002+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:26:08.262577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":35,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.765257+0000","last_change":"2023-11-09T03:47:03.146580+0000","last_active":"2023-11-09T03:47:50.765257+0000","last_peered":"2023-11-09T03:47:50.765257+0000","last_clean":"2023-11-09T03:47:50.765257+0000","last_became_active":"2023-11-09T03:47:01.589201+0000","last_became_peered":"2023-11-09T03:47:01.589201+0000","last_unstale":"2023-11-09T03:47:50.765257+0000","last_undegraded":"2023-11-09T03:47:50.765257+0000","last_fullsized":"2023-11-09T03:47:50.765257+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:14.300421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049706999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.428274+0000","last_change":"2023-11-09T03:47:51.427080+0000","last_active":"2023-11-09T03:47:51.428274+0000","last_peered":"2023-11-09T03:47:51.428274+0000","last_clean":"2023-11-09T03:47:51.428274+0000","last_became_active":"2023-11-09T03:47:51.426164+0000","last_became_peered":"2023-11-09T03:47:51.426164+0000","last_unstale":"2023-11-09T03:47:51.428274+0000","last_undegraded":"2023-11-09T03:47:51.428274+0000","last_fullsized":"2023-11-09T03:47:51.428274+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:50.761419+0000","last_clean_scrub_stamp":"2023-11-09T03:47:50.761419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:35:45.209321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"65'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.785881+0000","last_change":"2023-11-09T03:47:50.781515+0000","last_active":"2023-11-09T03:47:51.785881+0000","last_peered":"2023-11-09T03:47:51.785881+0000","last_clean":"2023-11-09T03:47:51.785881+0000","last_became_active":"2023-11-09T03:47:50.781152+0000","last_became_peered":"2023-11-09T03:47:50.781152+0000","last_unstale":"2023-11-09T03:47:51.785881+0000","last_undegraded":"2023-11-09T03:47:51.785881+0000","last_fullsized":"2023-11-09T03:47:51.785881+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:49.755183+0000","last_clean_scrub_stamp":"2023-11-09T03:47:49.755183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:15:06.138290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":34,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-09T03:47:51.411123+0000","last_change":"2023-11-09T03:47:03.147851+0000","last_active":"2023-11-09T03:47:51.411123+0000","last_peered":"2023-11-09T03:47:51.411123+0000","last_clean":"2023-11-09T03:47:51.411123+0000","last_became_active":"2023-11-09T03:47:01.589893+0000","last_became_peered":"2023-11-09T03:47:01.589893+0000","last_unstale":"2023-11-09T03:47:51.411123+0000","last_undegraded":"2023-11-09T03:47:51.411123+0000","last_fullsized":"2023-11-09T03:47:51.411123+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:08:17.183462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029151399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":202,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-09T03:47:50.775020+0000","last_change":"2023-11-09T03:45:21.030926+0000","last_active":"2023-11-09T03:47:50.775020+0000","last_peered":"2023-11-09T03:47:50.775020+0000","last_clean":"2023-11-09T03:47:50.775020+0000","last_became_active":"2023-11-09T03:45:21.029801+0000","last_became_peered":"2023-11-09T03:45:21.029801+0000","last_unstale":"2023-11-09T03:47:50.775020+0000","last_undegraded":"2023-11-09T03:47:50.775020+0000","last_fullsized":"2023-11-09T03:47:50.775020+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_clean_scrub_stamp":"2023-11-09T03:45:19.768433+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:12:59.153713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28380,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343460,"statfs":{"total":96636764160,"available":96607703040,"internally_reserved":0,"allocated":1388544,"data_stored":952415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.501,"15min":0.498},"min":{"1min":0.34899999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.41199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.54900000000000004}]},{"osd":1,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.33000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.38100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.51800000000000002}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.52300000000000002,"15min":0.51400000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.68700000000000006,"15min":0.70799999999999996},"min":{"1min":0.41299999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.80700000000000005,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.505}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.48699999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.58999999999999997}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.441},"min":{"1min":0.34100000000000003,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.49199999999999999,"15min":0.47899999999999998},"min":{"1min":0.38700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.57699999999999996}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.47199999999999998,"15min":0.45600000000000002},"min":{"1min":0.37,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.51100000000000001,"15min":0.497},"min":{"1min":0.39600000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55100000000000005,"15min":0.54100000000000004},"min":{"1min":0.44400000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.434,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.60599999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":794624,"data_stored":361523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.499,"15min":0.501},"min":{"1min":0.35099999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63900000000000001,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.50600000000000001,"15min":0.495},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.73199999999999998}]},{"osd":1,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.65300000000000002}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.375,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.70399999999999996}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.753},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.67500000000000004}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.503,"15min":0.503},"min":{"1min":0.316,"5min":0.313,"15min":0.313},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.56399999999999995}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.436,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55500000000000005,"15min":0.55800000000000005},"min":{"1min":0.34300000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.69299999999999995}]},{"osd":7,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.42899999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.44400000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.66300000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":361521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.496,"15min":0.48599999999999999},"min":{"1min":0.38700000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.503,"15min":0.498},"min":{"1min":0.42399999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.71799999999999997,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.51600000000000001}]},{"osd":1,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.439,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.69999999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.46700000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.76200000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.53100000000000003}]},{"osd":2,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.33300000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.67900000000000005,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.499,"5min":0.371,"15min":0.371},"max":{"1min":0.72799999999999998,"5min":0.745,"15min":0.745},"last":0.503}]},{"osd":3,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.39700000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.76500000000000001,"5min":0.88,"15min":0.88},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.78100000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.48399999999999999,"15min":0.48099999999999998},"min":{"1min":0.33100000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.48499999999999999}]},{"osd":6,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.50900000000000001,"15min":0.51100000000000001},"min":{"1min":0.33400000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.74199999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.442},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51300000000000001,"15min":0.51100000000000001},"min":{"1min":0.35499999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66900000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.46200000000000002}]},{"osd":7,"last update":"Thu Nov 9 03:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.38900000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.73499999999999999,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61899999999999999,"15min":0.62},"min":{"1min":0.41199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.752,"5min":1.01,"15min":1.01},"last":0.60199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":361501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67600000000000005,"5min":1.034,"15min":1.034},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.53600000000000003,"15min":0.54000000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68300000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.51100000000000001}]},{"osd":1,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.56899999999999995,"15min":0.57699999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68100000000000005,"5min":1.093,"15min":1.093},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.36899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72399999999999998,"5min":0.871,"15min":0.871},"last":0.53500000000000003}]},{"osd":2,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76800000000000002,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70099999999999996,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.69199999999999995}]},{"osd":3,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71699999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54900000000000004,"15min":0.55200000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86599999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.45400000000000001}]},{"osd":5,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54400000000000004,"15min":0.54300000000000004},"min":{"1min":0.37,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76000000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.33100000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.72899999999999998}]},{"osd":6,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.33800000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.622},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.65300000000000002}]},{"osd":7,"last update":"Thu Nov 9 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.62,"15min":0.63},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77700000000000002,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.95599999999999996,"5min":1.113,"15min":1.113},"last":0.66600000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":794624,"data_stored":361543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.5,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.313},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51100000000000001,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.47799999999999998}]},{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32900000000000001,"15min":0.33100000000000002},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.54200000000000004,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.371,"15min":0.375},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.53100000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.35499999999999998}]},{"osd":2,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.27200000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.59099999999999997,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.56100000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.45700000000000002}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.70899999999999996,"5min":0.87,"15min":0.87},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60999999999999999,"15min":0.61599999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.75600000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.83299999999999996}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.62,"15min":0.625},"min":{"1min":0.45700000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79900000000000004,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61699999999999999,"15min":0.623},"min":{"1min":0.42599999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.74299999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.82099999999999995}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.621,"15min":0.627},"min":{"1min":0.42399999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81200000000000006,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59999999999999998,"15min":0.60399999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76600000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.63200000000000001}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77200000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63500000000000001,"15min":0.64200000000000002},"min":{"1min":0.433,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81299999999999994,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.81299999999999994}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":798720,"data_stored":363043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.46000000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.33400000000000002,"15min":0.33100000000000002},"min":{"1min":0.247,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.23300000000000001}]},{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32500000000000001,"15min":0.32600000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.495,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.35899999999999999,"15min":0.36299999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.48599999999999999,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.28599999999999998}]},{"osd":3,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.36199999999999999,"15min":0.36399999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.45700000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.25},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40200000000000002,"15min":0.40400000000000003},"min":{"1min":0.30299999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.52300000000000002,"5min":0.754,"15min":0.754},"last":0.39000000000000001}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72299999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81299999999999994,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.40300000000000002}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.39800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.75700000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.59599999999999997,"15min":0.60099999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.753,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.434}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60399999999999998,"15min":0.60899999999999999},"min":{"1min":0.44600000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.76100000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.71199999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.498}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60699999999999998,"15min":0.61199999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.745,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.371},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.80000000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.46100000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":806912,"data_stored":363659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.28699999999999998,"5min":0.31,"15min":0.314},"min":{"1min":0.20799999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.38900000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.315,"15min":0.311},"min":{"1min":0.24099999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.22500000000000001}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.311,"15min":0.309},"min":{"1min":0.253,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.378,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.253},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.34000000000000002}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36899999999999999,"15min":0.371},"min":{"1min":0.23899999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.47699999999999998,"5min":0.505,"15min":0.505},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.37,"15min":0.371},"min":{"1min":0.30299999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.45400000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.27100000000000002}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.41899999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.747,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.44400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.77600000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.51000000000000001}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56200000000000006,"15min":0.55700000000000005},"min":{"1min":0.46100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.44500000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.72599999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.44400000000000001}]},{"osd":6,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.47799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56699999999999995,"15min":0.56799999999999995},"min":{"1min":0.433,"5min":0.379,"15min":0.379},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.45400000000000001}]},{"osd":7,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.42899999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.69799999999999995,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.72899999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.49199999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28368,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343472,"statfs":{"total":96636764160,"available":96607715328,"internally_reserved":0,"allocated":1376256,"data_stored":951180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.31900000000000001,"15min":0.32300000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.38200000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.25},{"interface":"front","average":{"1min":0.29599999999999999,"5min":0.35499999999999998,"15min":0.36499999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.44500000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.375}]},{"osd":2,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.35699999999999998,"15min":0.36399999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.44700000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.36699999999999999,"15min":0.371},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.47699999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.33700000000000002}]},{"osd":3,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.38500000000000001,"15min":0.39100000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.46400000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.41499999999999998,"15min":0.42099999999999999},"min":{"1min":0.309,"5min":0.307,"15min":0.307},"max":{"1min":0.48999999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.32900000000000001}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58899999999999997,"15min":0.59299999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60599999999999998,"15min":0.60899999999999999},"min":{"1min":0.436,"5min":0.318,"15min":0.318},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.51100000000000001}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.36199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76800000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.629,"15min":0.63600000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78900000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.497}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.64100000000000001,"15min":0.64700000000000002},"min":{"1min":0.42499999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81000000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.627,"15min":0.63200000000000001},"min":{"1min":0.41699999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.55900000000000005}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.44400000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.42999999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.71799999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-09T03:47:55.079 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-09T03:47:55.080 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-09T03:47:55.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.219+0000 7f522b761640 1 Processor -- start 2023-11-09T03:47:55.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.220+0000 7f522b761640 1 -- start start 2023-11-09T03:47:55.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.220+0000 7f522b761640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52240cd6d0 0x7f5224161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:55.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.220+0000 7f522b761640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f52240b4880 0x7f52240b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:55.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.220+0000 7f522b761640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52240cde60 con 0x7f5224153e60 2023-11-09T03:47:55.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.220+0000 7f522b761640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5224160f90 con 0x7f52240b4880 2023-11-09T03:47:55.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.220+0000 7f522b761640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52240b51e0 con 0x7f52240cd6d0 2023-11-09T03:47:55.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.221+0000 7f5229cfe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f52240b4880 0x7f52240b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:55.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.221+0000 7f5228cfc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52240cd6d0 0x7f5224161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:55.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.221+0000 7f52294fd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f5224153e60 0x7f522415dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:57456/0 (socket says 172.21.15.76:57456) 2023-11-09T03:47:55.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.221+0000 7f52294fd640 1 -- 172.21.15.76:0/1863725801 learned_addr learned my addr 172.21.15.76:0/1863725801 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:55.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.221+0000 7f5229cfe640 1 -- 172.21.15.76:0/1863725801 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52240cd6d0 msgr2=0x7f5224161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:55.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.221+0000 7f5229cfe640 1 --2- 172.21.15.76:0/1863725801 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52240cd6d0 0x7f5224161420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:55.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.221+0000 7f5229cfe640 1 -- 172.21.15.76:0/1863725801 >> v1:172.21.15.113:6789/0 conn(0x7f5224153e60 legacy=0x7f522415dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:55.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.221+0000 7f521bfff640 1 -- 172.21.15.76:0/1863725801 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 185947174 0 0) 0x7f52240cde60 con 0x7f5224153e60 2023-11-09T03:47:55.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.221+0000 7f5229cfe640 1 -- 172.21.15.76:0/1863725801 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5224161960 con 0x7f52240b4880 2023-11-09T03:47:55.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.221+0000 7f5228cfc640 1 --2- 172.21.15.76:0/1863725801 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52240cd6d0 0x7f5224161420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:47:55.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.222+0000 7f5229cfe640 1 --2- 172.21.15.76:0/1863725801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52240b4880 0x7f52240b4ca0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f521c0097a0 tx=0x7f521c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bdd6dd89d399307 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:55.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.222+0000 7f521bfff640 1 -- 172.21.15.76:0/1863725801 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c02d040 con 0x7f52240b4880 2023-11-09T03:47:55.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.222+0000 7f521bfff640 1 -- 172.21.15.76:0/1863725801 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521c031810 con 0x7f52240b4880 2023-11-09T03:47:55.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.222+0000 7f521bfff640 1 -- 172.21.15.76:0/1863725801 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c031ba0 con 0x7f52240b4880 2023-11-09T03:47:55.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.223+0000 7f522b761640 1 -- 172.21.15.76:0/1863725801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52240b4880 msgr2=0x7f52240b4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:55.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.223+0000 7f522b761640 1 --2- 172.21.15.76:0/1863725801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52240b4880 0x7f52240b4ca0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f521c0097a0 tx=0x7f521c02b8b0 comp rx=0 tx=0).stop 2023-11-09T03:47:55.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.223+0000 7f522b761640 1 -- 172.21.15.76:0/1863725801 shutdown_connections 2023-11-09T03:47:55.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.223+0000 7f522b761640 1 --2- 172.21.15.76:0/1863725801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52240b4880 0x7f52240b4ca0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:55.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.223+0000 7f522b761640 1 --2- 172.21.15.76:0/1863725801 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52240cd6d0 0x7f5224161420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:55.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.223+0000 7f522b761640 1 -- 172.21.15.76:0/1863725801 >> 172.21.15.76:0/1863725801 conn(0x7f52240cf880 msgr2=0x7f52240cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:55.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.223+0000 7f522b761640 1 -- 172.21.15.76:0/1863725801 shutdown_connections 2023-11-09T03:47:55.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.223+0000 7f522b761640 1 -- 172.21.15.76:0/1863725801 wait complete. 2023-11-09T03:47:55.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.224+0000 7f522b761640 1 Processor -- start 2023-11-09T03:47:55.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.224+0000 7f522b761640 1 -- start start 2023-11-09T03:47:55.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.225+0000 7f522b761640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52240b4880 0x7f5224159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:55.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.225+0000 7f522b761640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52240cd6d0 0x7f522415a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:55.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.225+0000 7f522b761640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5224153e60 0x7f5224155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:55.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.225+0000 7f522b761640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52240c9270 con 0x7f5224153e60 2023-11-09T03:47:55.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.225+0000 7f522b761640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52240c8350 con 0x7f52240cd6d0 2023-11-09T03:47:55.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.225+0000 7f522b761640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52240c85f0 con 0x7f52240b4880 2023-11-09T03:47:55.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.225+0000 7f5229cfe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5224153e60 0x7f5224155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:55.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.225+0000 7f5229cfe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5224153e60 0x7f5224155e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41146/0 (socket says 172.21.15.76:41146) 2023-11-09T03:47:55.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.225+0000 7f5229cfe640 1 -- 172.21.15.76:0/1700266231 learned_addr learned my addr 172.21.15.76:0/1700266231 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:55.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.225+0000 7f5228cfc640 1 --2- 172.21.15.76:0/1700266231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52240cd6d0 0x7f522415a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:55.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.225+0000 7f52294fd640 1 --2- 172.21.15.76:0/1700266231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52240b4880 0x7f5224159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:55.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.225+0000 7f5229cfe640 1 -- 172.21.15.76:0/1700266231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52240b4880 msgr2=0x7f5224159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:55.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.225+0000 7f5229cfe640 1 --2- 172.21.15.76:0/1700266231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52240b4880 0x7f5224159cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:55.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.225+0000 7f5229cfe640 1 -- 172.21.15.76:0/1700266231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52240cd6d0 msgr2=0x7f522415a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:55.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.225+0000 7f5229cfe640 1 --2- 172.21.15.76:0/1700266231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52240cd6d0 0x7f522415a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:55.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.225+0000 7f5229cfe640 1 -- 172.21.15.76:0/1700266231 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5214009400 con 0x7f5224153e60 2023-11-09T03:47:55.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.225+0000 7f5228cfc640 1 --2- 172.21.15.76:0/1700266231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52240cd6d0 0x7f522415a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:47:55.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.226+0000 7f5229cfe640 1 --2- 172.21.15.76:0/1700266231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5224153e60 0x7f5224155e50 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f521c0097a0 tx=0x7f521c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:55.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.226+0000 7f5219ffb640 1 -- 172.21.15.76:0/1700266231 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c006e10 con 0x7f5224153e60 2023-11-09T03:47:55.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.226+0000 7f5219ffb640 1 -- 172.21.15.76:0/1700266231 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521c03bb60 con 0x7f5224153e60 2023-11-09T03:47:55.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.226+0000 7f522b761640 1 -- 172.21.15.76:0/1700266231 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f521c009400 con 0x7f5224153e60 2023-11-09T03:47:55.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.226+0000 7f5219ffb640 1 -- 172.21.15.76:0/1700266231 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c004030 con 0x7f5224153e60 2023-11-09T03:47:55.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.226+0000 7f522b761640 1 -- 172.21.15.76:0/1700266231 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52240c8bb0 con 0x7f5224153e60 2023-11-09T03:47:55.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.227+0000 7f522b761640 1 -- 172.21.15.76:0/1700266231 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5224155b30 con 0x7f5224153e60 2023-11-09T03:47:55.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.228+0000 7f5219ffb640 1 -- 172.21.15.76:0/1700266231 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f521c043070 con 0x7f5224153e60 2023-11-09T03:47:55.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.228+0000 7f5219ffb640 1 --2- 172.21.15.76:0/1700266231 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f520806ae70 0x7f520806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:55.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.228+0000 7f5219ffb640 1 -- 172.21.15.76:0/1700266231 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f521c0ab660 con 0x7f5224153e60 2023-11-09T03:47:55.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.228+0000 7f52294fd640 1 --2- 172.21.15.76:0/1700266231 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f520806ae70 0x7f520806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:55.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.229+0000 7f52294fd640 1 --2- 172.21.15.76:0/1700266231 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f520806ae70 0x7f520806d330 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f52200012e0 tx=0x7f5220001630 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:55.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.232+0000 7f5219ffb640 1 -- 172.21.15.76:0/1700266231 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f521c0766d0 con 0x7f5224153e60 2023-11-09T03:47:55.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.461+0000 7f522b761640 1 -- 172.21.15.76:0/1700266231 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5224074290 con 0x7f5224153e60 2023-11-09T03:47:55.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.463+0000 7f5219ffb640 1 -- 172.21.15.76:0/1700266231 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f521c080020 con 0x7f5224153e60 2023-11-09T03:47:55.462 INFO:teuthology.orchestra.run.smithi076.stdout:[client.0] 2023-11-09T03:47:55.462 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQChVUxlHj22CxAACNj5X24hdME+GNTLw4RahQ== 2023-11-09T03:47:55.463 INFO:teuthology.orchestra.run.smithi076.stdout: caps mds = "allow" 2023-11-09T03:47:55.463 INFO:teuthology.orchestra.run.smithi076.stdout: caps mon = "allow r" 2023-11-09T03:47:55.463 INFO:teuthology.orchestra.run.smithi076.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-09T03:47:55.464 INFO:teuthology.orchestra.run.smithi076.stderr:updated caps for client.0 2023-11-09T03:47:55.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.464+0000 7f522b761640 1 -- 172.21.15.76:0/1700266231 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f520806ae70 msgr2=0x7f520806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:55.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.464+0000 7f522b761640 1 --2- 172.21.15.76:0/1700266231 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f520806ae70 0x7f520806d330 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f52200012e0 tx=0x7f5220001630 comp rx=0 tx=0).stop 2023-11-09T03:47:55.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.464+0000 7f522b761640 1 -- 172.21.15.76:0/1700266231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5224153e60 msgr2=0x7f5224155e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:55.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.464+0000 7f522b761640 1 --2- 172.21.15.76:0/1700266231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5224153e60 0x7f5224155e50 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f521c0097a0 tx=0x7f521c0046c0 comp rx=0 tx=0).stop 2023-11-09T03:47:55.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.465+0000 7f522b761640 1 -- 172.21.15.76:0/1700266231 shutdown_connections 2023-11-09T03:47:55.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.465+0000 7f522b761640 1 --2- 172.21.15.76:0/1700266231 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f520806ae70 0x7f520806d330 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:55.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.465+0000 7f522b761640 1 --2- 172.21.15.76:0/1700266231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5224153e60 0x7f5224155e50 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:55.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.465+0000 7f522b761640 1 --2- 172.21.15.76:0/1700266231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52240cd6d0 0x7f522415a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:55.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.465+0000 7f522b761640 1 --2- 172.21.15.76:0/1700266231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52240b4880 0x7f5224159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:55.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.465+0000 7f522b761640 1 -- 172.21.15.76:0/1700266231 >> 172.21.15.76:0/1700266231 conn(0x7f52240cf880 msgr2=0x7f52240d0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:55.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.465+0000 7f522b761640 1 -- 172.21.15.76:0/1700266231 shutdown_connections 2023-11-09T03:47:55.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.465+0000 7f522b761640 1 -- 172.21.15.76:0/1700266231 wait complete. 2023-11-09T03:47:55.475 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:47:55.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.575+0000 7f70978d4640 1 Processor -- start 2023-11-09T03:47:55.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.576+0000 7f70978d4640 1 -- start start 2023-11-09T03:47:55.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.576+0000 7f70978d4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70900cde60 0x7f709015d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:55.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.576+0000 7f70978d4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f709015daa0 0x7f7090157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:55.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.576+0000 7f70978d4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f709015e000 con 0x7f70900ce240 2023-11-09T03:47:55.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.576+0000 7f70978d4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f709015e5c0 con 0x7f709015daa0 2023-11-09T03:47:55.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.576+0000 7f70978d4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709015e2e0 con 0x7f70900cde60 2023-11-09T03:47:55.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.576+0000 7f7095e71640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f70900ce240 0x7f709006b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57974/0 (socket says 172.21.15.76:57974) 2023-11-09T03:47:55.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.576+0000 7f7095e71640 1 -- 172.21.15.76:0/3492625053 learned_addr learned my addr 172.21.15.76:0/3492625053 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:55.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.577+0000 7f7087fff640 1 -- 172.21.15.76:0/3492625053 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1060315021 0 0) 0x7f709015e000 con 0x7f70900ce240 2023-11-09T03:47:55.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.577+0000 7f7087fff640 1 -- 172.21.15.76:0/3492625053 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f707c003520 con 0x7f70900ce240 2023-11-09T03:47:55.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.577+0000 7f7094e6f640 1 --2- 172.21.15.76:0/3492625053 >> v2:172.21.15.113:3300/0 conn(0x7f709015daa0 0x7f7090157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:55.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.577+0000 7f7095670640 1 --2- 172.21.15.76:0/3492625053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70900cde60 0x7f709015d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:55.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.577+0000 7f7087fff640 1 -- 172.21.15.76:0/3492625053 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1615458432 0 0) 0x7f707c003520 con 0x7f70900ce240 2023-11-09T03:47:55.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.577+0000 7f7087fff640 1 -- 172.21.15.76:0/3492625053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70900cde60 msgr2=0x7f709015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:55.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.577+0000 7f7087fff640 1 --2- 172.21.15.76:0/3492625053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70900cde60 0x7f709015d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:55.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.577+0000 7f7087fff640 1 -- 172.21.15.76:0/3492625053 >> v2:172.21.15.113:3300/0 conn(0x7f709015daa0 msgr2=0x7f7090157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:55.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.577+0000 7f7087fff640 1 --2- 172.21.15.76:0/3492625053 >> v2:172.21.15.113:3300/0 conn(0x7f709015daa0 0x7f7090157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:55.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.577+0000 7f7087fff640 1 -- 172.21.15.76:0/3492625053 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70900d1120 con 0x7f70900ce240 2023-11-09T03:47:55.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.577+0000 7f7087fff640 1 -- 172.21.15.76:0/3492625053 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f70880019f0 con 0x7f70900ce240 2023-11-09T03:47:55.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.577+0000 7f7087fff640 1 -- 172.21.15.76:0/3492625053 >> v1:172.21.15.76:6789/0 conn(0x7f70900ce240 legacy=0x7f709006b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:55.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.577+0000 7f7087fff640 1 --2- 172.21.15.76:0/3492625053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f707c003970 0x7f707c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:55.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.577+0000 7f7087fff640 1 --2- 172.21.15.76:0/3492625053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f707c0047c0 0x7f707c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:55.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.577+0000 7f7087fff640 1 --2- 172.21.15.76:0/3492625053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70900cde60 0x7f707c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:55.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.577+0000 7f7087fff640 1 -- 172.21.15.76:0/3492625053 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709015e5c0 con 0x7f707c003970 2023-11-09T03:47:55.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.577+0000 7f7087fff640 1 -- 172.21.15.76:0/3492625053 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709015e2e0 con 0x7f707c0047c0 2023-11-09T03:47:55.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.577+0000 7f7087fff640 1 -- 172.21.15.76:0/3492625053 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f707c00b980 con 0x7f70900cde60 2023-11-09T03:47:55.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.577+0000 7f7095670640 1 --2- 172.21.15.76:0/3492625053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f707c003970 0x7f707c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:55.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.577+0000 7f7094e6f640 1 --2- 172.21.15.76:0/3492625053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f707c0047c0 0x7f707c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:55.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.577+0000 7f7095e71640 1 --2- 172.21.15.76:0/3492625053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70900cde60 0x7f707c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:55.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.577+0000 7f7095670640 1 -- 172.21.15.76:0/3492625053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70900cde60 msgr2=0x7f707c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:55.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.577+0000 7f7095670640 1 --2- 172.21.15.76:0/3492625053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70900cde60 0x7f707c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:55.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.577+0000 7f7095670640 1 -- 172.21.15.76:0/3492625053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f707c0047c0 msgr2=0x7f707c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:55.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.577+0000 7f7095670640 1 --2- 172.21.15.76:0/3492625053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f707c0047c0 0x7f707c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:55.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.577+0000 7f7095670640 1 -- 172.21.15.76:0/3492625053 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c003520 con 0x7f707c003970 2023-11-09T03:47:55.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.577+0000 7f7095e71640 1 --2- 172.21.15.76:0/3492625053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70900cde60 0x7f707c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:55.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.577+0000 7f7094e6f640 1 --2- 172.21.15.76:0/3492625053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f707c0047c0 0x7f707c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:55.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.578+0000 7f7095670640 1 --2- 172.21.15.76:0/3492625053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f707c003970 0x7f707c003d90 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f7080009ac0 tx=0x7f707c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d03461e100a9d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:55.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.578+0000 7f7087fff640 1 -- 172.21.15.76:0/3492625053 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708002c6b0 con 0x7f707c003970 2023-11-09T03:47:55.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.578+0000 7f7087fff640 1 -- 172.21.15.76:0/3492625053 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f708002c810 con 0x7f707c003970 2023-11-09T03:47:55.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.578+0000 7f7087fff640 1 -- 172.21.15.76:0/3492625053 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708002cba0 con 0x7f707c003970 2023-11-09T03:47:55.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.579+0000 7f70978d4640 1 -- 172.21.15.76:0/3492625053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f707c003970 msgr2=0x7f707c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:55.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.579+0000 7f70978d4640 1 --2- 172.21.15.76:0/3492625053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f707c003970 0x7f707c003d90 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f7080009ac0 tx=0x7f707c0057f0 comp rx=0 tx=0).stop 2023-11-09T03:47:55.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.579+0000 7f7095e71640 1 -- 172.21.15.76:0/3492625053 reap_dead start 2023-11-09T03:47:55.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.579+0000 7f70978d4640 1 -- 172.21.15.76:0/3492625053 shutdown_connections 2023-11-09T03:47:55.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.579+0000 7f70978d4640 1 -- 172.21.15.76:0/3492625053 >> 172.21.15.76:0/3492625053 conn(0x7f7090071f30 msgr2=0x7f70900cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:55.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.579+0000 7f70978d4640 1 -- 172.21.15.76:0/3492625053 shutdown_connections 2023-11-09T03:47:55.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.579+0000 7f70978d4640 1 -- 172.21.15.76:0/3492625053 wait complete. 2023-11-09T03:47:55.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.580+0000 7f70978d4640 1 Processor -- start 2023-11-09T03:47:55.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.580+0000 7f70978d4640 1 -- start start 2023-11-09T03:47:55.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.581+0000 7f70978d4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f707c003970 0x7f709014e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:55.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.581+0000 7f70978d4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70901580f0 0x7f7090154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:55.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.581+0000 7f70978d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70900cc9d0 0x7f70900cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:55.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.581+0000 7f70978d4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70900cd390 con 0x7f70900cc9d0 2023-11-09T03:47:55.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.581+0000 7f70978d4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70901f2690 con 0x7f70901580f0 2023-11-09T03:47:55.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.581+0000 7f70978d4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7090154e00 con 0x7f707c003970 2023-11-09T03:47:55.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.581+0000 7f7095e71640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70900cc9d0 0x7f70900cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:55.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.581+0000 7f7095e71640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70900cc9d0 0x7f70900cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41170/0 (socket says 172.21.15.76:41170) 2023-11-09T03:47:55.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.581+0000 7f7095e71640 1 -- 172.21.15.76:0/310161434 learned_addr learned my addr 172.21.15.76:0/310161434 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:55.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.581+0000 7f7095e71640 1 -- 172.21.15.76:0/310161434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f707c003970 msgr2=0x7f709014e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:55.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.581+0000 7f7095e71640 1 --2- 172.21.15.76:0/310161434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f707c003970 0x7f709014e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:55.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.581+0000 7f7095e71640 1 -- 172.21.15.76:0/310161434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70901580f0 msgr2=0x7f7090154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:55.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.581+0000 7f7095e71640 1 --2- 172.21.15.76:0/310161434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70901580f0 0x7f7090154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:55.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.581+0000 7f7095e71640 1 -- 172.21.15.76:0/310161434 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70901550e0 con 0x7f70900cc9d0 2023-11-09T03:47:55.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.581+0000 7f7095e71640 1 --2- 172.21.15.76:0/310161434 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70900cc9d0 0x7f70900cce50 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f709006c790 tx=0x7f708800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:55.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.581+0000 7f7085ffb640 1 -- 172.21.15.76:0/310161434 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7088001990 con 0x7f70900cc9d0 2023-11-09T03:47:55.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.582+0000 7f7085ffb640 1 -- 172.21.15.76:0/310161434 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f708800bb80 con 0x7f70900cc9d0 2023-11-09T03:47:55.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.582+0000 7f70978d4640 1 -- 172.21.15.76:0/310161434 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70901553c0 con 0x7f70900cc9d0 2023-11-09T03:47:55.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.582+0000 7f7085ffb640 1 -- 172.21.15.76:0/310161434 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7088007070 con 0x7f70900cc9d0 2023-11-09T03:47:55.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.582+0000 7f70978d4640 1 -- 172.21.15.76:0/310161434 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f709014e820 con 0x7f70900cc9d0 2023-11-09T03:47:55.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.583+0000 7f70978d4640 1 -- 172.21.15.76:0/310161434 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70901f2850 con 0x7f70900cc9d0 2023-11-09T03:47:55.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.583+0000 7f7085ffb640 1 -- 172.21.15.76:0/310161434 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f70880071d0 con 0x7f70900cc9d0 2023-11-09T03:47:55.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.583+0000 7f7085ffb640 1 --2- 172.21.15.76:0/310161434 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f707406aea0 0x7f707406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:55.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.583+0000 7f7085ffb640 1 -- 172.21.15.76:0/310161434 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7088089c90 con 0x7f70900cc9d0 2023-11-09T03:47:55.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.584+0000 7f7095670640 1 --2- 172.21.15.76:0/310161434 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f707406aea0 0x7f707406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:55.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.584+0000 7f7095670640 1 --2- 172.21.15.76:0/310161434 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f707406aea0 0x7f707406d360 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7080036000 tx=0x7f708002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:55.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.588+0000 7f7085ffb640 1 -- 172.21.15.76:0/310161434 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7088054aa0 con 0x7f70900cc9d0 2023-11-09T03:47:55.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.787+0000 7f70978d4640 1 -- 172.21.15.76:0/310161434 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7090074290 con 0x7f70900cc9d0 2023-11-09T03:47:55.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.788+0000 7f7085ffb640 1 -- 172.21.15.76:0/310161434 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f70880580c0 con 0x7f70900cc9d0 2023-11-09T03:47:55.788 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:47:55.788 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2800733670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2800733670},{"type":"v1","addr":"172.21.15.113:6836","nonce":2800733670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":5791,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1301905851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1301905851},{"type":"v1","addr":"172.21.15.113:6837","nonce":1301905851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":6498,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/968739949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":968739949},{"type":"v1","addr":"172.21.15.76:6833","nonce":968739949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:47:51.114132+0000","modified":"2023-11-09T03:47:53.114512+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6501},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6501":{"gid":6501,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/1576689316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1576689316},{"type":"v1","addr":"172.21.15.76:6835","nonce":1576689316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-09T03:47:55.788 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 29 2023-11-09T03:47:55.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.790+0000 7f70978d4640 1 -- 172.21.15.76:0/310161434 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f707406aea0 msgr2=0x7f707406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:55.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.790+0000 7f70978d4640 1 --2- 172.21.15.76:0/310161434 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f707406aea0 0x7f707406d360 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7080036000 tx=0x7f708002e000 comp rx=0 tx=0).stop 2023-11-09T03:47:55.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.790+0000 7f70978d4640 1 -- 172.21.15.76:0/310161434 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70900cc9d0 msgr2=0x7f70900cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:55.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.790+0000 7f70978d4640 1 --2- 172.21.15.76:0/310161434 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70900cc9d0 0x7f70900cce50 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f709006c790 tx=0x7f708800b4a0 comp rx=0 tx=0).stop 2023-11-09T03:47:55.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.790+0000 7f70978d4640 1 -- 172.21.15.76:0/310161434 shutdown_connections 2023-11-09T03:47:55.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.790+0000 7f70978d4640 1 --2- 172.21.15.76:0/310161434 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f707406aea0 0x7f707406d360 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:55.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.790+0000 7f70978d4640 1 --2- 172.21.15.76:0/310161434 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70900cc9d0 0x7f70900cce50 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:55.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.790+0000 7f70978d4640 1 --2- 172.21.15.76:0/310161434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70901580f0 0x7f7090154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:55.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.790+0000 7f70978d4640 1 --2- 172.21.15.76:0/310161434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f707c003970 0x7f709014e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:55.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.790+0000 7f70978d4640 1 -- 172.21.15.76:0/310161434 >> 172.21.15.76:0/310161434 conn(0x7f7090071f30 msgr2=0x7f70900d09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:55.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.790+0000 7f70978d4640 1 -- 172.21.15.76:0/310161434 shutdown_connections 2023-11-09T03:47:55.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:55.790+0000 7f70978d4640 1 -- 172.21.15.76:0/310161434 wait complete. 2023-11-09T03:47:56.202 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-09T03:47:51.114132+0000', 'modified': '2023-11-09T03:47:53.114512+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6501}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6501': {'gid': 6501, 'name': 'c', 'rank': 0, 'incarnation': 27, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6835/1576689316', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 1576689316}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 1576689316}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-09T03:47:56.203 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-09T03:47:56.203 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-09T03:47:56.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.305+0000 7f84eb955640 1 Processor -- start 2023-11-09T03:47:56.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.306+0000 7f84eb955640 1 -- start start 2023-11-09T03:47:56.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.306+0000 7f84eb955640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f84e4153e60 0x7f84e415dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:56.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.306+0000 7f84eb955640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84e40cde60 con 0x7f84e40cd6d0 2023-11-09T03:47:56.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.306+0000 7f84eb955640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84e4160f90 con 0x7f84e40b4880 2023-11-09T03:47:56.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.306+0000 7f84eb955640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84e415e400 con 0x7f84e4153e60 2023-11-09T03:47:56.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.307+0000 7f84e8ef0640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f84e40cd6d0 0x7f84e4161420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57982/0 (socket says 172.21.15.76:57982) 2023-11-09T03:47:56.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.307+0000 7f84e8ef0640 1 -- 172.21.15.76:0/2618745250 learned_addr learned my addr 172.21.15.76:0/2618745250 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:56.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.307+0000 7f84e96f1640 1 --2- 172.21.15.76:0/2618745250 >> v2:172.21.15.113:3300/0 conn(0x7f84e4153e60 0x7f84e415dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:56.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.307+0000 7f84d7fff640 1 -- 172.21.15.76:0/2618745250 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 515504191 0 0) 0x7f84e40cde60 con 0x7f84e40cd6d0 2023-11-09T03:47:56.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.307+0000 7f84d7fff640 1 -- 172.21.15.76:0/2618745250 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84d0003520 con 0x7f84e40cd6d0 2023-11-09T03:47:56.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.307+0000 7f84d7fff640 1 -- 172.21.15.76:0/2618745250 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2558822806 0 0) 0x7f84d0003520 con 0x7f84e40cd6d0 2023-11-09T03:47:56.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.307+0000 7f84d7fff640 1 -- 172.21.15.76:0/2618745250 >> v2:172.21.15.113:3300/0 conn(0x7f84e4153e60 msgr2=0x7f84e415dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:56.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.307+0000 7f84d7fff640 1 --2- 172.21.15.76:0/2618745250 >> v2:172.21.15.113:3300/0 conn(0x7f84e4153e60 0x7f84e415dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.307+0000 7f84d7fff640 1 -- 172.21.15.76:0/2618745250 >> v1:172.21.15.113:6789/0 conn(0x7f84e40b4880 legacy=0x7f84e40d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:56.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.307+0000 7f84d7fff640 1 -- 172.21.15.76:0/2618745250 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e40ce090 con 0x7f84e40cd6d0 2023-11-09T03:47:56.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.307+0000 7f84d7fff640 1 -- 172.21.15.76:0/2618745250 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f84e0001670 con 0x7f84e40cd6d0 2023-11-09T03:47:56.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.307+0000 7f84d7fff640 1 -- 172.21.15.76:0/2618745250 >> v1:172.21.15.76:6789/0 conn(0x7f84e40cd6d0 legacy=0x7f84e4161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:56.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.307+0000 7f84d7fff640 1 --2- 172.21.15.76:0/2618745250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84d0003970 0x7f84d0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:56.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.307+0000 7f84d7fff640 1 --2- 172.21.15.76:0/2618745250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84d00047c0 0x7f84d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:56.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.307+0000 7f84d7fff640 1 --2- 172.21.15.76:0/2618745250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84d0005180 0x7f84d000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:56.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.307+0000 7f84d7fff640 1 -- 172.21.15.76:0/2618745250 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e415e400 con 0x7f84d00047c0 2023-11-09T03:47:56.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.307+0000 7f84d7fff640 1 -- 172.21.15.76:0/2618745250 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d000c020 con 0x7f84d0005180 2023-11-09T03:47:56.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.307+0000 7f84d7fff640 1 -- 172.21.15.76:0/2618745250 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d000c1e0 con 0x7f84d0003970 2023-11-09T03:47:56.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.307+0000 7f84e8ef0640 1 --2- 172.21.15.76:0/2618745250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84d00047c0 0x7f84d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:56.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.307+0000 7f84e9ef2640 1 --2- 172.21.15.76:0/2618745250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84d0005180 0x7f84d000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:56.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.307+0000 7f84e96f1640 1 --2- 172.21.15.76:0/2618745250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84d0003970 0x7f84d0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:56.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.307+0000 7f84e8ef0640 1 -- 172.21.15.76:0/2618745250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84d0003970 msgr2=0x7f84d0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:56.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.307+0000 7f84e8ef0640 1 --2- 172.21.15.76:0/2618745250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84d0003970 0x7f84d0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.307+0000 7f84e8ef0640 1 -- 172.21.15.76:0/2618745250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84d0005180 msgr2=0x7f84d000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:56.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.307+0000 7f84e8ef0640 1 --2- 172.21.15.76:0/2618745250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84d0005180 0x7f84d000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.307+0000 7f84e8ef0640 1 -- 172.21.15.76:0/2618745250 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d0003520 con 0x7f84d00047c0 2023-11-09T03:47:56.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.308+0000 7f84e9ef2640 1 --2- 172.21.15.76:0/2618745250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84d0005180 0x7f84d000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:56.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.308+0000 7f84e96f1640 1 --2- 172.21.15.76:0/2618745250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84d0003970 0x7f84d0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:56.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.308+0000 7f84e9ef2640 1 -- 172.21.15.76:0/2618745250 reap_dead start 2023-11-09T03:47:56.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.308+0000 7f84e8ef0640 1 --2- 172.21.15.76:0/2618745250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84d00047c0 0x7f84d0004c40 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f84e000c210 tx=0x7f84d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a3e40f3d17ac29 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:56.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.308+0000 7f84d7fff640 1 -- 172.21.15.76:0/2618745250 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0005ad0 con 0x7f84d00047c0 2023-11-09T03:47:56.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.308+0000 7f84d7fff640 1 -- 172.21.15.76:0/2618745250 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e0005c30 con 0x7f84d00047c0 2023-11-09T03:47:56.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.308+0000 7f84d7fff640 1 -- 172.21.15.76:0/2618745250 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e002cad0 con 0x7f84d00047c0 2023-11-09T03:47:56.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.309+0000 7f84eb955640 1 -- 172.21.15.76:0/2618745250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84d00047c0 msgr2=0x7f84d0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:56.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.309+0000 7f84eb955640 1 --2- 172.21.15.76:0/2618745250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84d00047c0 0x7f84d0004c40 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f84e000c210 tx=0x7f84d00057f0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.309+0000 7f84eb955640 1 -- 172.21.15.76:0/2618745250 shutdown_connections 2023-11-09T03:47:56.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.309+0000 7f84eb955640 1 --2- 172.21.15.76:0/2618745250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84d00047c0 0x7f84d0004c40 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.309+0000 7f84eb955640 1 -- 172.21.15.76:0/2618745250 >> 172.21.15.76:0/2618745250 conn(0x7f84e40cf880 msgr2=0x7f84e40cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:56.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.309+0000 7f84eb955640 1 -- 172.21.15.76:0/2618745250 shutdown_connections 2023-11-09T03:47:56.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.309+0000 7f84eb955640 1 -- 172.21.15.76:0/2618745250 wait complete. 2023-11-09T03:47:56.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.310+0000 7f84eb955640 1 Processor -- start 2023-11-09T03:47:56.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.310+0000 7f84eb955640 1 -- start start 2023-11-09T03:47:56.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.311+0000 7f84eb955640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84d00047c0 0x7f84e4156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:56.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.311+0000 7f84eb955640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84e40c7340 0x7f84e40c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:56.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.311+0000 7f84eb955640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84e4156850 0x7f84e40c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:56.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.311+0000 7f84eb955640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e40ca700 con 0x7f84e40c7340 2023-11-09T03:47:56.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.311+0000 7f84eb955640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e40c77d0 con 0x7f84e4156850 2023-11-09T03:47:56.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.311+0000 7f84eb955640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e40c7ab0 con 0x7f84d00047c0 2023-11-09T03:47:56.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.311+0000 7f84e8ef0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84e40c7340 0x7f84e40c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:56.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.311+0000 7f84e8ef0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84e40c7340 0x7f84e40c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41178/0 (socket says 172.21.15.76:41178) 2023-11-09T03:47:56.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.311+0000 7f84e8ef0640 1 -- 172.21.15.76:0/1850598752 learned_addr learned my addr 172.21.15.76:0/1850598752 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:56.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.311+0000 7f84e9ef2640 1 --2- 172.21.15.76:0/1850598752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84e4156850 0x7f84e40c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:56.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.311+0000 7f84e96f1640 1 --2- 172.21.15.76:0/1850598752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84d00047c0 0x7f84e4156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:56.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.311+0000 7f84e8ef0640 1 -- 172.21.15.76:0/1850598752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84d00047c0 msgr2=0x7f84e4156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:56.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.311+0000 7f84e8ef0640 1 --2- 172.21.15.76:0/1850598752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84d00047c0 0x7f84e4156310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.311+0000 7f84e8ef0640 1 -- 172.21.15.76:0/1850598752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84e4156850 msgr2=0x7f84e40c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:56.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.311+0000 7f84e8ef0640 1 --2- 172.21.15.76:0/1850598752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84e4156850 0x7f84e40c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.311+0000 7f84e8ef0640 1 -- 172.21.15.76:0/1850598752 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e40c7d90 con 0x7f84e40c7340 2023-11-09T03:47:56.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.311+0000 7f84e8ef0640 1 --2- 172.21.15.76:0/1850598752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84e40c7340 0x7f84e40c8dd0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f84e0001930 tx=0x7f84e002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:56.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.311+0000 7f84d5ffb640 1 -- 172.21.15.76:0/1850598752 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e003c040 con 0x7f84e40c7340 2023-11-09T03:47:56.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.311+0000 7f84d5ffb640 1 -- 172.21.15.76:0/1850598752 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e002cd60 con 0x7f84e40c7340 2023-11-09T03:47:56.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.312+0000 7f84eb955640 1 -- 172.21.15.76:0/1850598752 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e40c8070 con 0x7f84e40c7340 2023-11-09T03:47:56.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.312+0000 7f84d5ffb640 1 -- 172.21.15.76:0/1850598752 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e00072b0 con 0x7f84e40c7340 2023-11-09T03:47:56.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.312+0000 7f84eb955640 1 -- 172.21.15.76:0/1850598752 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e4150870 con 0x7f84e40c7340 2023-11-09T03:47:56.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.313+0000 7f84d5ffb640 1 -- 172.21.15.76:0/1850598752 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f84e0032070 con 0x7f84e40c7340 2023-11-09T03:47:56.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.313+0000 7f84d5ffb640 1 --2- 172.21.15.76:0/1850598752 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f84c806aea0 0x7f84c806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:56.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.314+0000 7f84d5ffb640 1 -- 172.21.15.76:0/1850598752 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f84e00aadc0 con 0x7f84e40c7340 2023-11-09T03:47:56.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.314+0000 7f84e96f1640 1 --2- 172.21.15.76:0/1850598752 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f84c806aea0 0x7f84c806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:56.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.314+0000 7f84e96f1640 1 --2- 172.21.15.76:0/1850598752 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f84c806aea0 0x7f84c806d360 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f84e40687c0 tx=0x7f84d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:56.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.314+0000 7f84ee5c1740 1 Processor -- start 2023-11-09T03:47:56.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.314+0000 7f84ee5c1740 1 -- start start 2023-11-09T03:47:56.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.315+0000 7f84ee5c1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5604c120c850 0x5604c11df750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:56.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.315+0000 7f84ee5c1740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5604c120cc30 0x5604c120f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:56.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.315+0000 7f84ee5c1740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604c11dfc90 0x5604c120fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:56.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.315+0000 7f84ee5c1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c10f0740 con 0x5604c120c850 2023-11-09T03:47:56.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.315+0000 7f84ee5c1740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c11528d0 con 0x5604c11dfc90 2023-11-09T03:47:56.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.315+0000 7f84ee5c1740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5604c114ffc0 con 0x5604c120cc30 2023-11-09T03:47:56.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.315+0000 7f84e96f1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5604c120c850 0x5604c11df750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:56.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.315+0000 7f84e96f1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5604c120c850 0x5604c11df750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41186/0 (socket says 172.21.15.76:41186) 2023-11-09T03:47:56.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.315+0000 7f84e96f1640 1 -- 172.21.15.76:0/2098879679 learned_addr learned my addr 172.21.15.76:0/2098879679 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:56.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.315+0000 7f84e9ef2640 1 --2- 172.21.15.76:0/2098879679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604c11dfc90 0x5604c120fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:56.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.315+0000 7f84e8ef0640 1 --2- 172.21.15.76:0/2098879679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5604c120cc30 0x5604c120f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:56.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.315+0000 7f84e96f1640 1 -- 172.21.15.76:0/2098879679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5604c120cc30 msgr2=0x5604c120f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:56.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.315+0000 7f84e96f1640 1 --2- 172.21.15.76:0/2098879679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5604c120cc30 0x5604c120f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.315+0000 7f84e96f1640 1 -- 172.21.15.76:0/2098879679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604c11dfc90 msgr2=0x5604c120fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:56.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.315+0000 7f84e96f1640 1 --2- 172.21.15.76:0/2098879679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604c11dfc90 0x5604c120fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.315+0000 7f84e96f1640 1 -- 172.21.15.76:0/2098879679 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604c110eca0 con 0x5604c120c850 2023-11-09T03:47:56.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.316+0000 7f84e96f1640 1 --2- 172.21.15.76:0/2098879679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5604c120c850 0x5604c11df750 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f84d80044c0 tx=0x7f84d8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c51a69b77de2756 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:56.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.316+0000 7f84ceffd640 1 -- 172.21.15.76:0/2098879679 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d8003b90 con 0x5604c120c850 2023-11-09T03:47:56.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.316+0000 7f84ceffd640 1 -- 172.21.15.76:0/2098879679 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d8003cf0 con 0x5604c120c850 2023-11-09T03:47:56.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.316+0000 7f84ceffd640 1 -- 172.21.15.76:0/2098879679 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d8016660 con 0x5604c120c850 2023-11-09T03:47:56.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.316+0000 7f84ee5c1740 1 -- 172.21.15.76:0/2098879679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5604c120c850 msgr2=0x5604c11df750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:56.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.316+0000 7f84ee5c1740 1 --2- 172.21.15.76:0/2098879679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5604c120c850 0x5604c11df750 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f84d80044c0 tx=0x7f84d8016000 comp rx=0 tx=0).stop 2023-11-09T03:47:56.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.316+0000 7f84ee5c1740 1 -- 172.21.15.76:0/2098879679 shutdown_connections 2023-11-09T03:47:56.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.316+0000 7f84ee5c1740 1 --2- 172.21.15.76:0/2098879679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604c11dfc90 0x5604c120fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.316+0000 7f84ee5c1740 1 --2- 172.21.15.76:0/2098879679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5604c120cc30 0x5604c120f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.316+0000 7f84ee5c1740 1 --2- 172.21.15.76:0/2098879679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5604c120c850 0x5604c11df750 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.316+0000 7f84ee5c1740 1 -- 172.21.15.76:0/2098879679 >> 172.21.15.76:0/2098879679 conn(0x5604c11e1a60 msgr2=0x5604c122a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:56.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.316+0000 7f84ee5c1740 1 -- 172.21.15.76:0/2098879679 shutdown_connections 2023-11-09T03:47:56.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.316+0000 7f84ee5c1740 1 -- 172.21.15.76:0/2098879679 wait complete. 2023-11-09T03:47:56.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.317+0000 7f84ee5c1740 1 -- start start 2023-11-09T03:47:56.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.317+0000 7f84ee5c1740 1 Processor -- start 2023-11-09T03:47:56.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.318+0000 7f84ee5c1740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5604c120c850 0x5604c11ffb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:56.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.318+0000 7f84ee5c1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5604c120cc30 0x5604c1216630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:56.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.318+0000 7f84ee5c1740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604c1216b70 0x5604c1211a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:56.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.318+0000 7f84ee5c1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c114b8b0 con 0x5604c120cc30 2023-11-09T03:47:56.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.318+0000 7f84ee5c1740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c1105c80 con 0x5604c1216b70 2023-11-09T03:47:56.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.318+0000 7f84ee5c1740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5604c1105920 con 0x5604c120c850 2023-11-09T03:47:56.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.318+0000 7f84ee5c1740 10 client.0 _ll_register_callbacks cb 0x5604c0e5dac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:47:56.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.318+0000 7f84e8ef0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5604c120cc30 0x5604c1216630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:56.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.318+0000 7f84e8ef0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5604c120cc30 0x5604c1216630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41192/0 (socket says 172.21.15.76:41192) 2023-11-09T03:47:56.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.318+0000 7f84e8ef0640 1 -- 172.21.15.76:0/3246994990 learned_addr learned my addr 172.21.15.76:0/3246994990 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:56.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.318+0000 7f84e96f1640 1 --2- 172.21.15.76:0/3246994990 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5604c120c850 0x5604c11ffb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:56.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.318+0000 7f84e9ef2640 1 --2- 172.21.15.76:0/3246994990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604c1216b70 0x5604c1211a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:56.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.318+0000 7f84e8ef0640 1 -- 172.21.15.76:0/3246994990 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5604c120c850 msgr2=0x5604c11ffb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:56.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.318+0000 7f84e8ef0640 1 --2- 172.21.15.76:0/3246994990 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5604c120c850 0x5604c11ffb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.318+0000 7f84e8ef0640 1 -- 172.21.15.76:0/3246994990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604c1216b70 msgr2=0x5604c1211a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:56.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.318+0000 7f84e8ef0640 1 --2- 172.21.15.76:0/3246994990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604c1216b70 0x5604c1211a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.318+0000 7f84e8ef0640 1 -- 172.21.15.76:0/3246994990 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d8004170 con 0x5604c120cc30 2023-11-09T03:47:56.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.318+0000 7f84e9ef2640 1 --2- 172.21.15.76:0/3246994990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604c1216b70 0x5604c1211a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:56.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.318+0000 7f84e8ef0640 1 --2- 172.21.15.76:0/3246994990 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5604c120cc30 0x5604c1216630 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f84e002c7a0 tx=0x7f84e0049f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a41e60e2fc80368 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:56.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.318+0000 7f84c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:47:56.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.318+0000 7f84c3fff640 1 -- 172.21.15.76:0/3246994990 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e00911f0 con 0x5604c120cc30 2023-11-09T03:47:56.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.319+0000 7f84c3fff640 1 -- 172.21.15.76:0/3246994990 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f84e0091030 con 0x5604c120cc30 2023-11-09T03:47:56.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.319+0000 7f84ee5c1740 1 -- 172.21.15.76:0/3246994990 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604c10f7bf0 con 0x5604c120cc30 2023-11-09T03:47:56.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.319+0000 7f84c3fff640 1 -- 172.21.15.76:0/3246994990 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84e007d9e0 con 0x5604c120cc30 2023-11-09T03:47:56.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.319+0000 7f84ee5c1740 10 client.6636 fetch_fsmap learned FSMap version 29 2023-11-09T03:47:56.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.319+0000 7f84ee5c1740 1 -- 172.21.15.76:0/3246994990 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5604c1149f30 con 0x5604c120cc30 2023-11-09T03:47:56.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.319+0000 7f84c3fff640 1 -- 172.21.15.76:0/3246994990 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5604c10f7bf0 con 0x5604c120cc30 2023-11-09T03:47:56.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.319+0000 7f84c3fff640 1 -- 172.21.15.76:0/3246994990 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f84e0075fa0 con 0x5604c120cc30 2023-11-09T03:47:56.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.319+0000 7f84ee5c1740 10 client.6636 fetch_fsmap finished waiting for FSMap version 29 2023-11-09T03:47:56.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.319+0000 7f84ee5c1740 10 client.6636 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-09T03:47:56.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.319+0000 7f84ee5c1740 20 client.6636 populate_metadata read hostname 'smithi076' 2023-11-09T03:47:56.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.319+0000 7f84ee5c1740 1 --2- 172.21.15.76:0/3246994990 >> [v2:172.21.15.76:6834/1576689316,v1:172.21.15.76:6835/1576689316] conn(0x5604c1215aa0 0x5604c123de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:56.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.319+0000 7f84ee5c1740 4 client.6636 mds_command: new command op to 6501 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:47:56.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.319+0000 7f84ee5c1740 1 -- 172.21.15.76:0/3246994990 --> [v2:172.21.15.76:6834/1576689316,v1:172.21.15.76:6835/1576689316] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604c1144890 con 0x5604c1215aa0 2023-11-09T03:47:56.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.319+0000 7f84e96f1640 1 --2- 172.21.15.76:0/3246994990 >> [v2:172.21.15.76:6834/1576689316,v1:172.21.15.76:6835/1576689316] conn(0x5604c1215aa0 0x5604c123de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:56.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.319+0000 7f84c3fff640 10 client.6636 ms_handle_connect on v2:172.21.15.76:6834/1576689316 2023-11-09T03:47:56.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.319+0000 7f84e96f1640 1 --2- 172.21.15.76:0/3246994990 >> [v2:172.21.15.76:6834/1576689316,v1:172.21.15.76:6835/1576689316] conn(0x5604c1215aa0 0x5604c123de00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8387af6e7435ead server_cookie=f457702b5fb07bd8 in_seq=0 out_seq=0 2023-11-09T03:47:56.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.320+0000 7f84c3fff640 1 -- 172.21.15.76:0/3246994990 <== mds.0 v2:172.21.15.76:6834/1576689316 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5604c1144890 con 0x5604c1215aa0 2023-11-09T03:47:56.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.320+0000 7f84c3fff640 10 client.6636 handle_command_reply: tid=0 2023-11-09T03:47:56.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.320+0000 7f84ee5c1740 1 client.6636 shutdown 2023-11-09T03:47:56.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.320+0000 7f84ee5c1740 1 -- 172.21.15.76:0/3246994990 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5604c120cc30 msgr2=0x5604c1216630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:56.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.320+0000 7f84ee5c1740 1 --2- 172.21.15.76:0/3246994990 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5604c120cc30 0x5604c1216630 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f84e002c7a0 tx=0x7f84e0049f50 comp rx=0 tx=0).stop 2023-11-09T03:47:56.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.320+0000 7f84ee5c1740 1 -- 172.21.15.76:0/3246994990 shutdown_connections 2023-11-09T03:47:56.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.320+0000 7f84ee5c1740 1 --2- 172.21.15.76:0/3246994990 >> [v2:172.21.15.76:6834/1576689316,v1:172.21.15.76:6835/1576689316] conn(0x5604c1215aa0 0x5604c123de00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.320+0000 7f84ee5c1740 1 --2- 172.21.15.76:0/3246994990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604c1216b70 0x5604c1211a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.320+0000 7f84ee5c1740 1 --2- 172.21.15.76:0/3246994990 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5604c120cc30 0x5604c1216630 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.320+0000 7f84c3fff640 0 client.6636 ms_handle_reset on v2:172.21.15.76:6834/1576689316 2023-11-09T03:47:56.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.320+0000 7f84ee5c1740 1 --2- 172.21.15.76:0/3246994990 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5604c120c850 0x5604c11ffb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.320+0000 7f84ee5c1740 1 -- 172.21.15.76:0/3246994990 >> 172.21.15.76:0/3246994990 conn(0x5604c11dfc90 msgr2=0x5604c120f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:56.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.320+0000 7f84ee5c1740 1 -- 172.21.15.76:0/3246994990 shutdown_connections 2023-11-09T03:47:56.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.320+0000 7f84ee5c1740 1 -- 172.21.15.76:0/3246994990 wait complete. 2023-11-09T03:47:56.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.320+0000 7f84ee5c1740 20 client.6636 trim_cache size 0 max 16384 2023-11-09T03:47:56.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.336+0000 7f84ee5c1740 1 Processor -- start 2023-11-09T03:47:56.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.336+0000 7f84ee5c1740 1 -- start start 2023-11-09T03:47:56.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.336+0000 7f84ee5c1740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5604c120c850 0x5604c124d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:56.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.336+0000 7f84ee5c1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5604c120cc30 0x5604c120a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:56.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.337+0000 7f84ee5c1740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604c1216b70 0x5604c1216070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:56.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.337+0000 7f84ee5c1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c10cb280 con 0x5604c120cc30 2023-11-09T03:47:56.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.337+0000 7f84ee5c1740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c11ff2c0 con 0x5604c1216b70 2023-11-09T03:47:56.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.337+0000 7f84ee5c1740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5604c124dbc0 con 0x5604c120c850 2023-11-09T03:47:56.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.337+0000 7f84e8ef0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5604c120cc30 0x5604c120a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:56.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.337+0000 7f84e8ef0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5604c120cc30 0x5604c120a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41198/0 (socket says 172.21.15.76:41198) 2023-11-09T03:47:56.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.337+0000 7f84e8ef0640 1 -- 172.21.15.76:0/586155983 learned_addr learned my addr 172.21.15.76:0/586155983 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:56.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.337+0000 7f84e8ef0640 1 -- 172.21.15.76:0/586155983 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5604c120c850 msgr2=0x5604c124d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:56.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.337+0000 7f84e8ef0640 1 --2- 172.21.15.76:0/586155983 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5604c120c850 0x5604c124d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.337+0000 7f84e8ef0640 1 -- 172.21.15.76:0/586155983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604c1216b70 msgr2=0x5604c1216070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:56.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.337+0000 7f84e8ef0640 1 --2- 172.21.15.76:0/586155983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604c1216b70 0x5604c1216070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.337+0000 7f84e8ef0640 1 -- 172.21.15.76:0/586155983 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604c10e7d10 con 0x5604c120cc30 2023-11-09T03:47:56.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.337+0000 7f84e8ef0640 1 --2- 172.21.15.76:0/586155983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5604c120cc30 0x5604c120a280 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f84e000c210 tx=0x7f84e008e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1576668dcafd3a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:56.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.337+0000 7f84cf7fe640 1 -- 172.21.15.76:0/586155983 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e007d9e0 con 0x5604c120cc30 2023-11-09T03:47:56.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.337+0000 7f84cf7fe640 1 -- 172.21.15.76:0/586155983 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e0079660 con 0x5604c120cc30 2023-11-09T03:47:56.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.337+0000 7f84cf7fe640 1 -- 172.21.15.76:0/586155983 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0081700 con 0x5604c120cc30 2023-11-09T03:47:56.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.338+0000 7f84ee5c1740 1 -- 172.21.15.76:0/586155983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5604c120cc30 msgr2=0x5604c120a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:56.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.338+0000 7f84ee5c1740 1 --2- 172.21.15.76:0/586155983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5604c120cc30 0x5604c120a280 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f84e000c210 tx=0x7f84e008e000 comp rx=0 tx=0).stop 2023-11-09T03:47:56.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.338+0000 7f84ee5c1740 1 -- 172.21.15.76:0/586155983 shutdown_connections 2023-11-09T03:47:56.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.338+0000 7f84ee5c1740 1 --2- 172.21.15.76:0/586155983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604c1216b70 0x5604c1216070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.338+0000 7f84ee5c1740 1 --2- 172.21.15.76:0/586155983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5604c120cc30 0x5604c120a280 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.338+0000 7f84ee5c1740 1 --2- 172.21.15.76:0/586155983 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5604c120c850 0x5604c124d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.338+0000 7f84ee5c1740 1 -- 172.21.15.76:0/586155983 >> 172.21.15.76:0/586155983 conn(0x5604c11dfc90 msgr2=0x5604c11ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:56.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.338+0000 7f84ee5c1740 1 -- 172.21.15.76:0/586155983 shutdown_connections 2023-11-09T03:47:56.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.338+0000 7f84ee5c1740 1 -- 172.21.15.76:0/586155983 wait complete. 2023-11-09T03:47:56.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.338+0000 7f84ee5c1740 1 -- start start 2023-11-09T03:47:56.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.338+0000 7f84ee5c1740 1 Processor -- start 2023-11-09T03:47:56.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.339+0000 7f84ee5c1740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5604c120c850 0x5604c12945b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:56.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.339+0000 7f84ee5c1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5604c120cc30 0x5604c120ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:56.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.339+0000 7f84ee5c1740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604c1216b70 0x5604c12956a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:56.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.339+0000 7f84ee5c1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c11db060 con 0x5604c120cc30 2023-11-09T03:47:56.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.339+0000 7f84ee5c1740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c1214b80 con 0x5604c1216b70 2023-11-09T03:47:56.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.339+0000 7f84ee5c1740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5604c11dac10 con 0x5604c120c850 2023-11-09T03:47:56.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.339+0000 7f84ee5c1740 10 client.0 _ll_register_callbacks cb 0x7f84d801b040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:47:56.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.339+0000 7f84e8ef0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5604c120cc30 0x5604c120ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:56.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.339+0000 7f84e8ef0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5604c120cc30 0x5604c120ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41210/0 (socket says 172.21.15.76:41210) 2023-11-09T03:47:56.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.339+0000 7f84e8ef0640 1 -- 172.21.15.76:0/3464395376 learned_addr learned my addr 172.21.15.76:0/3464395376 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:56.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.340+0000 7f84e96f1640 1 --2- 172.21.15.76:0/3464395376 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5604c120c850 0x5604c12945b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:56.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.340+0000 7f84e9ef2640 1 --2- 172.21.15.76:0/3464395376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604c1216b70 0x5604c12956a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:56.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.340+0000 7f84e8ef0640 1 -- 172.21.15.76:0/3464395376 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5604c120c850 msgr2=0x5604c12945b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:56.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.340+0000 7f84e8ef0640 1 --2- 172.21.15.76:0/3464395376 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5604c120c850 0x5604c12945b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.340+0000 7f84e8ef0640 1 -- 172.21.15.76:0/3464395376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604c1216b70 msgr2=0x5604c12956a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:56.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.340+0000 7f84e8ef0640 1 --2- 172.21.15.76:0/3464395376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604c1216b70 0x5604c12956a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.340+0000 7f84e8ef0640 1 -- 172.21.15.76:0/3464395376 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5604c1149f30 con 0x5604c120cc30 2023-11-09T03:47:56.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.340+0000 7f84e96f1640 1 --2- 172.21.15.76:0/3464395376 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5604c120c850 0x5604c12945b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:56.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.340+0000 7f84e9ef2640 1 --2- 172.21.15.76:0/3464395376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604c1216b70 0x5604c12956a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:47:56.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.340+0000 7f84e8ef0640 1 --2- 172.21.15.76:0/3464395376 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5604c120cc30 0x5604c120ad90 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f84e002c7a0 tx=0x7f84e0001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1517a971b48ca83e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:56.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.340+0000 7f84c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:47:56.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.340+0000 7f84c3fff640 1 -- 172.21.15.76:0/3464395376 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e007d9e0 con 0x5604c120cc30 2023-11-09T03:47:56.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.340+0000 7f84c3fff640 1 -- 172.21.15.76:0/3464395376 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f84e0091040 con 0x5604c120cc30 2023-11-09T03:47:56.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.340+0000 7f84ee5c1740 1 -- 172.21.15.76:0/3464395376 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84d8003cf0 con 0x5604c120cc30 2023-11-09T03:47:56.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.340+0000 7f84c3fff640 1 -- 172.21.15.76:0/3464395376 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84e0098240 con 0x5604c120cc30 2023-11-09T03:47:56.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.340+0000 7f84ee5c1740 10 client.6642 fetch_fsmap learned FSMap version 29 2023-11-09T03:47:56.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.340+0000 7f84ee5c1740 1 -- 172.21.15.76:0/3464395376 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5604c110d500 con 0x5604c120cc30 2023-11-09T03:47:56.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.340+0000 7f84c3fff640 1 -- 172.21.15.76:0/3464395376 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d8003cf0 con 0x5604c120cc30 2023-11-09T03:47:56.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.340+0000 7f84c3fff640 1 -- 172.21.15.76:0/3464395376 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f84e007abc0 con 0x5604c120cc30 2023-11-09T03:47:56.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.340+0000 7f84ee5c1740 10 client.6642 fetch_fsmap finished waiting for FSMap version 29 2023-11-09T03:47:56.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.340+0000 7f84ee5c1740 10 client.6642 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-09T03:47:56.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.340+0000 7f84ee5c1740 20 client.6642 populate_metadata read hostname 'smithi076' 2023-11-09T03:47:56.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.340+0000 7f84ee5c1740 1 --2- 172.21.15.76:0/3464395376 >> [v2:172.21.15.76:6834/1576689316,v1:172.21.15.76:6835/1576689316] conn(0x5604c1210a30 0x5604c1210e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:56.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.340+0000 7f84ee5c1740 4 client.6642 mds_command: new command op to 6501 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-09T03:47:56.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.340+0000 7f84ee5c1740 1 -- 172.21.15.76:0/3464395376 --> [v2:172.21.15.76:6834/1576689316,v1:172.21.15.76:6835/1576689316] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5604c11445b0 con 0x5604c1210a30 2023-11-09T03:47:56.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.340+0000 7f84e96f1640 1 --2- 172.21.15.76:0/3464395376 >> [v2:172.21.15.76:6834/1576689316,v1:172.21.15.76:6835/1576689316] conn(0x5604c1210a30 0x5604c1210e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:56.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.341+0000 7f84c3fff640 10 client.6642 ms_handle_connect on v2:172.21.15.76:6834/1576689316 2023-11-09T03:47:56.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.341+0000 7f84e96f1640 1 --2- 172.21.15.76:0/3464395376 >> [v2:172.21.15.76:6834/1576689316,v1:172.21.15.76:6835/1576689316] conn(0x5604c1210a30 0x5604c1210e10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8063093fca610aa server_cookie=692638a1f408f8ee in_seq=0 out_seq=0 2023-11-09T03:47:56.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.341+0000 7f84c3fff640 1 -- 172.21.15.76:0/3464395376 <== mds.0 v2:172.21.15.76:6834/1576689316 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5604c11445b0 con 0x5604c1210a30 2023-11-09T03:47:56.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.341+0000 7f84c3fff640 10 client.6642 handle_command_reply: tid=0 2023-11-09T03:47:56.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.341+0000 7f84ee5c1740 1 client.6642 shutdown 2023-11-09T03:47:56.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.341+0000 7f84ee5c1740 1 -- 172.21.15.76:0/3464395376 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5604c120cc30 msgr2=0x5604c120ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:56.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.341+0000 7f84ee5c1740 1 --2- 172.21.15.76:0/3464395376 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5604c120cc30 0x5604c120ad90 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f84e002c7a0 tx=0x7f84e0001a00 comp rx=0 tx=0).stop 2023-11-09T03:47:56.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.341+0000 7f84ee5c1740 1 -- 172.21.15.76:0/3464395376 shutdown_connections 2023-11-09T03:47:56.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.341+0000 7f84ee5c1740 1 --2- 172.21.15.76:0/3464395376 >> [v2:172.21.15.76:6834/1576689316,v1:172.21.15.76:6835/1576689316] conn(0x5604c1210a30 0x5604c1210e10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.341+0000 7f84ee5c1740 1 --2- 172.21.15.76:0/3464395376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604c1216b70 0x5604c12956a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.341+0000 7f84ee5c1740 1 --2- 172.21.15.76:0/3464395376 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5604c120cc30 0x5604c120ad90 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.341+0000 7f84c3fff640 0 client.6642 ms_handle_reset on v2:172.21.15.76:6834/1576689316 2023-11-09T03:47:56.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.341+0000 7f84ee5c1740 1 --2- 172.21.15.76:0/3464395376 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5604c120c850 0x5604c12945b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.341+0000 7f84ee5c1740 1 -- 172.21.15.76:0/3464395376 >> 172.21.15.76:0/3464395376 conn(0x5604c11dfc90 msgr2=0x5604c1216070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:56.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.341+0000 7f84ee5c1740 1 -- 172.21.15.76:0/3464395376 shutdown_connections 2023-11-09T03:47:56.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.341+0000 7f84ee5c1740 1 -- 172.21.15.76:0/3464395376 wait complete. 2023-11-09T03:47:56.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.341+0000 7f84ee5c1740 20 client.6642 trim_cache size 0 max 16384 2023-11-09T03:47:56.363 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:47:56.364 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "c213b26f-d0da-45d5-8e34-bb75c096d4f2", 2023-11-09T03:47:56.364 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2023-11-09T03:47:56.364 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 6501, 2023-11-09T03:47:56.364 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2023-11-09T03:47:56.364 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2023-11-09T03:47:56.365 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2023-11-09T03:47:56.365 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 4.5657224530000002, 2023-11-09T03:47:56.365 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 29, 2023-11-09T03:47:56.365 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 66, 2023-11-09T03:47:56.365 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 66, 2023-11-09T03:47:56.365 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 9.6460531609999993 2023-11-09T03:47:56.366 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:47:56.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.342+0000 7f84eb955640 1 -- 172.21.15.76:0/1850598752 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f84c806aea0 msgr2=0x7f84c806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:56.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.342+0000 7f84eb955640 1 --2- 172.21.15.76:0/1850598752 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f84c806aea0 0x7f84c806d360 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f84e40687c0 tx=0x7f84d8008040 comp rx=0 tx=0).stop 2023-11-09T03:47:56.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.342+0000 7f84eb955640 1 -- 172.21.15.76:0/1850598752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84e40c7340 msgr2=0x7f84e40c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:56.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.342+0000 7f84eb955640 1 --2- 172.21.15.76:0/1850598752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84e40c7340 0x7f84e40c8dd0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f84e0001930 tx=0x7f84e002e070 comp rx=0 tx=0).stop 2023-11-09T03:47:56.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.342+0000 7f84eb955640 1 -- 172.21.15.76:0/1850598752 shutdown_connections 2023-11-09T03:47:56.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.342+0000 7f84eb955640 1 --2- 172.21.15.76:0/1850598752 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f84c806aea0 0x7f84c806d360 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.342+0000 7f84eb955640 1 --2- 172.21.15.76:0/1850598752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84e4156850 0x7f84e40c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.342+0000 7f84eb955640 1 --2- 172.21.15.76:0/1850598752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84e40c7340 0x7f84e40c8dd0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.342+0000 7f84eb955640 1 --2- 172.21.15.76:0/1850598752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84d00047c0 0x7f84e4156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.342+0000 7f84eb955640 1 -- 172.21.15.76:0/1850598752 >> 172.21.15.76:0/1850598752 conn(0x7f84e40cf880 msgr2=0x7f84e415fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:56.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.342+0000 7f84eb955640 1 -- 172.21.15.76:0/1850598752 shutdown_connections 2023-11-09T03:47:56.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.342+0000 7f84eb955640 1 -- 172.21.15.76:0/1850598752 wait complete. 2023-11-09T03:47:56.369 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-09T03:47:56.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.470+0000 7f1886500640 1 Processor -- start 2023-11-09T03:47:56.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.470+0000 7f1886500640 1 -- start start 2023-11-09T03:47:56.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.471+0000 7f1886500640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f18800b0cd0 0x7f1880070630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:56.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.471+0000 7f1886500640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f18800cde40 0x7f1880070b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:56.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.471+0000 7f1886500640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18800cc990 con 0x7f18800cda60 2023-11-09T03:47:56.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.471+0000 7f1886500640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18800ccc70 con 0x7f18800cde40 2023-11-09T03:47:56.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.471+0000 7f1886500640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18800c9ce0 con 0x7f18800b0cd0 2023-11-09T03:47:56.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.471+0000 7f1884a9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f18800cde40 0x7f1880070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:56.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.471+0000 7f187ffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f18800cda60 0x7f18800ca7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57986/0 (socket says 172.21.15.76:57986) 2023-11-09T03:47:56.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.471+0000 7f187ffff640 1 -- 172.21.15.76:0/3913589614 learned_addr learned my addr 172.21.15.76:0/3913589614 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:56.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.471+0000 7f187f7fe640 1 --2- 172.21.15.76:0/3913589614 >> v2:172.21.15.113:3300/0 conn(0x7f18800b0cd0 0x7f1880070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:56.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.471+0000 7f1884a9d640 1 -- 172.21.15.76:0/3913589614 >> v2:172.21.15.113:3300/0 conn(0x7f18800b0cd0 msgr2=0x7f1880070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:56.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.471+0000 7f1884a9d640 1 --2- 172.21.15.76:0/3913589614 >> v2:172.21.15.113:3300/0 conn(0x7f18800b0cd0 0x7f1880070630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.471+0000 7f1884a9d640 1 -- 172.21.15.76:0/3913589614 >> v1:172.21.15.76:6789/0 conn(0x7f18800cda60 legacy=0x7f18800ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:56.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.471+0000 7f1884a9d640 1 -- 172.21.15.76:0/3913589614 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18800c9fc0 con 0x7f18800cde40 2023-11-09T03:47:56.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.471+0000 7f187effd640 1 -- 172.21.15.76:0/3913589614 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528430517 0 0) 0x7f18800cc990 con 0x7f18800cda60 2023-11-09T03:47:56.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.472+0000 7f1884a9d640 1 --2- 172.21.15.76:0/3913589614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18800cde40 0x7f1880070b70 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f188006e330 tx=0x7f187802cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f1bc71f3525fe server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:56.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.472+0000 7f187effd640 1 -- 172.21.15.76:0/3913589614 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187802db70 con 0x7f18800cde40 2023-11-09T03:47:56.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.472+0000 7f187effd640 1 -- 172.21.15.76:0/3913589614 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187802dcd0 con 0x7f18800cde40 2023-11-09T03:47:56.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.472+0000 7f187effd640 1 -- 172.21.15.76:0/3913589614 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1878008660 con 0x7f18800cde40 2023-11-09T03:47:56.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.473+0000 7f1886500640 1 -- 172.21.15.76:0/3913589614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18800cde40 msgr2=0x7f1880070b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:56.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.473+0000 7f1886500640 1 --2- 172.21.15.76:0/3913589614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18800cde40 0x7f1880070b70 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f188006e330 tx=0x7f187802cfd0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.473+0000 7f1886500640 1 -- 172.21.15.76:0/3913589614 shutdown_connections 2023-11-09T03:47:56.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.473+0000 7f1886500640 1 --2- 172.21.15.76:0/3913589614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18800cde40 0x7f1880070b70 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.473+0000 7f1886500640 1 --2- 172.21.15.76:0/3913589614 >> v2:172.21.15.113:3300/0 conn(0x7f18800b0cd0 0x7f1880070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.473+0000 7f1886500640 1 -- 172.21.15.76:0/3913589614 >> 172.21.15.76:0/3913589614 conn(0x7f18800ce950 msgr2=0x7f18800ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:56.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.473+0000 7f1886500640 1 -- 172.21.15.76:0/3913589614 shutdown_connections 2023-11-09T03:47:56.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.473+0000 7f1886500640 1 -- 172.21.15.76:0/3913589614 wait complete. 2023-11-09T03:47:56.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.474+0000 7f1886500640 1 Processor -- start 2023-11-09T03:47:56.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.474+0000 7f1886500640 1 -- start start 2023-11-09T03:47:56.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.475+0000 7f1886500640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18800b0cd0 0x7f1880152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:56.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.475+0000 7f1886500640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18800cda60 0x7f18801e23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:56.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.475+0000 7f1886500640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18800cde40 0x7f18801e28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:56.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.475+0000 7f1886500640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18800ac6c0 con 0x7f18800b0cd0 2023-11-09T03:47:56.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.475+0000 7f1886500640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18800ac9a0 con 0x7f18800cde40 2023-11-09T03:47:56.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.475+0000 7f1886500640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18800acc80 con 0x7f18800cda60 2023-11-09T03:47:56.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.475+0000 7f187ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18800b0cd0 0x7f1880152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:56.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.475+0000 7f187ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18800b0cd0 0x7f1880152fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41234/0 (socket says 172.21.15.76:41234) 2023-11-09T03:47:56.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.475+0000 7f187ffff640 1 -- 172.21.15.76:0/1962686626 learned_addr learned my addr 172.21.15.76:0/1962686626 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:56.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.475+0000 7f187ffff640 1 -- 172.21.15.76:0/1962686626 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18800cda60 msgr2=0x7f18801e23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:56.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.475+0000 7f187ffff640 1 --2- 172.21.15.76:0/1962686626 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18800cda60 0x7f18801e23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.475+0000 7f187ffff640 1 -- 172.21.15.76:0/1962686626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18800cde40 msgr2=0x7f18801e28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:56.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.475+0000 7f187ffff640 1 --2- 172.21.15.76:0/1962686626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18800cde40 0x7f18801e28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.475+0000 7f187ffff640 1 -- 172.21.15.76:0/1962686626 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f187800ab80 con 0x7f18800b0cd0 2023-11-09T03:47:56.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.475+0000 7f187ffff640 1 --2- 172.21.15.76:0/1962686626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18800b0cd0 0x7f1880152fc0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f187000c4d0 tx=0x7f187000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:56.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.476+0000 7f187cff9640 1 -- 172.21.15.76:0/1962686626 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18700074e0 con 0x7f18800b0cd0 2023-11-09T03:47:56.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.476+0000 7f187cff9640 1 -- 172.21.15.76:0/1962686626 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1870007660 con 0x7f18800b0cd0 2023-11-09T03:47:56.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.476+0000 7f187cff9640 1 -- 172.21.15.76:0/1962686626 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1870004a20 con 0x7f18800b0cd0 2023-11-09T03:47:56.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.476+0000 7f1886500640 1 -- 172.21.15.76:0/1962686626 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18800acf60 con 0x7f18800b0cd0 2023-11-09T03:47:56.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.476+0000 7f1886500640 1 -- 172.21.15.76:0/1962686626 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18800ab950 con 0x7f18800b0cd0 2023-11-09T03:47:56.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.477+0000 7f1886500640 1 -- 172.21.15.76:0/1962686626 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18800cc470 con 0x7f18800b0cd0 2023-11-09T03:47:56.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.477+0000 7f187cff9640 1 -- 172.21.15.76:0/1962686626 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1870004b80 con 0x7f18800b0cd0 2023-11-09T03:47:56.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.477+0000 7f187cff9640 1 --2- 172.21.15.76:0/1962686626 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f186c06ae70 0x7f186c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:56.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.477+0000 7f187cff9640 1 -- 172.21.15.76:0/1962686626 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f18700889b0 con 0x7f18800b0cd0 2023-11-09T03:47:56.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.478+0000 7f187f7fe640 1 --2- 172.21.15.76:0/1962686626 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f186c06ae70 0x7f186c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:56.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.478+0000 7f187f7fe640 1 --2- 172.21.15.76:0/1962686626 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f186c06ae70 0x7f186c06d330 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f18801e1bd0 tx=0x7f1874008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:56.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.482+0000 7f187cff9640 1 -- 172.21.15.76:0/1962686626 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f18800cc470 con 0x7f18800b0cd0 2023-11-09T03:47:56.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.683+0000 7f1886500640 1 -- 172.21.15.76:0/1962686626 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f18800c6860 con 0x7f18800b0cd0 2023-11-09T03:47:56.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.684+0000 7f187cff9640 1 -- 172.21.15.76:0/1962686626 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) v1 ==== 53+0+73 (secure 0 0 0) 0x7f18800c6860 con 0x7f18800b0cd0 2023-11-09T03:47:56.683 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-09T03:47:56.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.686+0000 7f1886500640 1 -- 172.21.15.76:0/1962686626 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f186c06ae70 msgr2=0x7f186c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:56.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.686+0000 7f1886500640 1 --2- 172.21.15.76:0/1962686626 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f186c06ae70 0x7f186c06d330 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f18801e1bd0 tx=0x7f1874008040 comp rx=0 tx=0).stop 2023-11-09T03:47:56.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.686+0000 7f1886500640 1 -- 172.21.15.76:0/1962686626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18800b0cd0 msgr2=0x7f1880152fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:56.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.686+0000 7f1886500640 1 --2- 172.21.15.76:0/1962686626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18800b0cd0 0x7f1880152fc0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f187000c4d0 tx=0x7f187000c990 comp rx=0 tx=0).stop 2023-11-09T03:47:56.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.686+0000 7f1886500640 1 -- 172.21.15.76:0/1962686626 shutdown_connections 2023-11-09T03:47:56.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.686+0000 7f1886500640 1 --2- 172.21.15.76:0/1962686626 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f186c06ae70 0x7f186c06d330 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.686+0000 7f1886500640 1 --2- 172.21.15.76:0/1962686626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18800cde40 0x7f18801e28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.686+0000 7f1886500640 1 --2- 172.21.15.76:0/1962686626 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18800cda60 0x7f18801e23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.686+0000 7f1886500640 1 --2- 172.21.15.76:0/1962686626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18800b0cd0 0x7f1880152fc0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:56.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.686+0000 7f1886500640 1 -- 172.21.15.76:0/1962686626 >> 172.21.15.76:0/1962686626 conn(0x7f18800ce950 msgr2=0x7f188006d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:56.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.686+0000 7f1886500640 1 -- 172.21.15.76:0/1962686626 shutdown_connections 2023-11-09T03:47:56.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:56.686+0000 7f1886500640 1 -- 172.21.15.76:0/1962686626 wait complete. 2023-11-09T03:47:56.696 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-09T03:47:56.696 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-09T03:47:56.696 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2023-11-09T03:47:56.697 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-09T03:47:56.697 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-09T03:47:56.697 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-09T03:47:56.698 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-09T03:47:56.698 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-09T03:47:56.698 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:47:56.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:47:56.699 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2023-11-09T03:47:56.714 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-09T03:47:56.714 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-09T03:47:56.715 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2023-11-09T03:47:56.715 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:47:56.715 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2023-11-09T03:47:56.715 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:47:56.715 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:47:56.715 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:47:56.716 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2023-11-09T03:47:56.716 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:47:56.716 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:47:56.716 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2023-11-09T03:47:56.716 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:47:56.716 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:47:56.717 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-09T03:47:56.717 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:47:56.717 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-09T03:47:56.717 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42472sec preferred_lft 42472sec 2023-11-09T03:47:56.717 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2023-11-09T03:47:56.717 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:47:56.718 INFO:teuthology.orchestra.run.smithi076.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-09T03:47:56.718 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2e:15:bd:03:a8:2e brd ff:ff:ff:ff:ff:ff 2023-11-09T03:47:56.718 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-09T03:47:56.718 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:47:56.718 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::bcda:38ff:fed4:fdb0/64 scope link 2023-11-09T03:47:56.718 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:47:56.719 INFO:teuthology.orchestra.run.smithi076.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-09T03:47:56.719 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 6a:8d:94:d7:e5:00 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:47:56.719 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::688d:94ff:fed7:e500/64 scope link 2023-11-09T03:47:56.719 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:47:56.719 INFO:teuthology.orchestra.run.smithi076.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-09T03:47:56.719 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2e:15:bd:03:a8:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:47:56.720 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::2c15:bdff:fe03:a82e/64 scope link 2023-11-09T03:47:56.720 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:47:56.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:47:56.720 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-09T03:47:56.779 INFO:teuthology.orchestra.run.smithi076.stdout:1 2023-11-09T03:47:56.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:47:56.781 DEBUG:teuthology.orchestra.run.smithi076:> route 2023-11-09T03:47:56.836 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2023-11-09T03:47:56.836 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-09T03:47:56.836 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-09T03:47:56.837 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-09T03:47:56.837 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-09T03:47:56.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:47:56.837 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-09T03:47:56.838 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-09T03:47:56.838 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-09T03:47:56.838 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-09T03:47:56.839 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-09T03:47:56.839 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-09T03:47:56.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:47:56.931 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2023-11-09T03:47:56.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-09T03:47:56.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-09T03:47:56.946 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:47:56.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:47:56.947 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2023-11-09T03:47:57.012 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-09T03:47:57.013 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:47:57.013 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-09T03:47:57.067 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:47:57.068 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:47:57.068 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-09T03:47:57.122 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:47:57.177 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:47:57.177 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:47:57.177 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:47:57.178 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:47:57.178 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:47:57.178 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:47:57.178 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:47:57.178 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:47:57.178 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:47:57.179 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:47:57.179 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:47:57.179 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:47:57.179 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:47:57.179 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:47:57.179 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:47:57.180 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:47:57.180 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:47:57.180 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:47:57.180 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:47:57.180 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:47:57.181 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:47:57.181 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:47:57.181 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:47:57.181 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:47:57.181 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:47:57.181 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:47:57.182 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:47:57.182 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:47:57.182 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:47:57.182 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:47:57.182 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:47:57.182 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:47:57.183 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:47:57.183 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:47:57.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:47:57.183 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:47:57.231 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-09T03:47:57.231 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-09T03:47:57.273 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:47:57.288 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:47:57.288 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:47:57.288 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:47:57.289 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:47:57.289 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:47:57.289 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:47:57.289 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:47:57.289 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:47:57.290 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:47:57.290 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:47:57.290 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:47:57.290 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:47:57.290 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:47:57.290 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:47:57.291 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:47:57.291 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:47:57.291 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:47:57.291 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:47:57.291 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:47:57.291 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:47:57.292 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:47:57.292 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:47:57.292 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:47:57.292 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:47:57.292 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:47:57.293 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:47:57.293 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:47:57.293 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:47:57.293 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:47:57.293 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:47:57.293 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:47:57.294 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:47:57.294 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:47:57.294 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:47:57.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:47:57.295 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:47:57.363 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:47:57.364+0000 7faeba034480 -1 init, newargv = 0x55f65d73db70 newargc=15 2023-11-09T03:47:57.364 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[110864]: starting ceph client 2023-11-09T03:47:57.367 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[110864]: starting fuse 2023-11-09T03:47:58.444 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:47:58.463 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:47:58.463 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:47:58.464 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:47:58.464 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:47:58.464 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:47:58.464 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:47:58.464 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:47:58.465 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:47:58.465 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:47:58.465 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:47:58.465 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:47:58.465 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:47:58.466 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:47:58.466 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:47:58.466 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:47:58.466 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:47:58.466 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:47:58.467 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:47:58.467 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:47:58.467 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:47:58.467 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:47:58.467 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:47:58.468 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:47:58.468 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:47:58.468 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:47:58.468 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:47:58.468 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:47:58.468 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:47:58.469 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:47:58.469 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:47:58.469 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:47:58.469 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:47:58.469 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:47:58.469 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:47:58.470 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-09T03:47:58.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:47:58.471 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:47:58.528 INFO:teuthology.orchestra.run.smithi076.stdout:43 2023-11-09T03:47:58.529 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-09T03:47:58.529 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer -- python3 -c ' 2023-11-09T03:47:58.530 DEBUG:teuthology.orchestra.run.smithi076:> import glob 2023-11-09T03:47:58.530 DEBUG:teuthology.orchestra.run.smithi076:> import re 2023-11-09T03:47:58.531 DEBUG:teuthology.orchestra.run.smithi076:> import os 2023-11-09T03:47:58.531 DEBUG:teuthology.orchestra.run.smithi076:> import subprocess 2023-11-09T03:47:58.531 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:47:58.532 DEBUG:teuthology.orchestra.run.smithi076:> def _find_admin_socket(client_name): 2023-11-09T03:47:58.532 DEBUG:teuthology.orchestra.run.smithi076:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-09T03:47:58.532 DEBUG:teuthology.orchestra.run.smithi076:> files = glob.glob(asok_path) 2023-11-09T03:47:58.533 DEBUG:teuthology.orchestra.run.smithi076:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-09T03:47:58.533 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:47:58.533 DEBUG:teuthology.orchestra.run.smithi076:> # Given a non-glob path, it better be there 2023-11-09T03:47:58.534 DEBUG:teuthology.orchestra.run.smithi076:> if "*" not in asok_path: 2023-11-09T03:47:58.534 DEBUG:teuthology.orchestra.run.smithi076:> assert(len(files) == 1) 2023-11-09T03:47:58.534 DEBUG:teuthology.orchestra.run.smithi076:> return files[0] 2023-11-09T03:47:58.535 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:47:58.535 DEBUG:teuthology.orchestra.run.smithi076:> for f in files: 2023-11-09T03:47:58.536 DEBUG:teuthology.orchestra.run.smithi076:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-09T03:47:58.536 DEBUG:teuthology.orchestra.run.smithi076:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-09T03:47:58.536 DEBUG:teuthology.orchestra.run.smithi076:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-09T03:47:58.536 DEBUG:teuthology.orchestra.run.smithi076:> contents = proc_f.read() 2023-11-09T03:47:58.537 DEBUG:teuthology.orchestra.run.smithi076:> if mountpoint in contents: 2023-11-09T03:47:58.537 DEBUG:teuthology.orchestra.run.smithi076:> return f 2023-11-09T03:47:58.538 DEBUG:teuthology.orchestra.run.smithi076:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-09T03:47:58.538 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:47:58.538 DEBUG:teuthology.orchestra.run.smithi076:> print(_find_admin_socket("client.0")) 2023-11-09T03:47:58.539 DEBUG:teuthology.orchestra.run.smithi076:> ' 2023-11-09T03:47:58.655 INFO:teuthology.orchestra.run.smithi076.stdout:/var/run/ceph/ceph-client.0.110864.asok 2023-11-09T03:47:58.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:47:58.666 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.110864.asok 2023-11-09T03:47:58.666 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:47:58.666 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.110864.asok status 2023-11-09T03:47:58.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:47:58.835 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:47:58.835 INFO:teuthology.orchestra.run.smithi076.stdout: "metadata": { 2023-11-09T03:47:58.836 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_sha1": "7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4", 2023-11-09T03:47:58.836 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev)", 2023-11-09T03:47:58.836 INFO:teuthology.orchestra.run.smithi076.stdout: "entity_id": "0", 2023-11-09T03:47:58.836 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": "smithi076", 2023-11-09T03:47:58.836 INFO:teuthology.orchestra.run.smithi076.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-09T03:47:58.837 INFO:teuthology.orchestra.run.smithi076.stdout: "pid": "110864", 2023-11-09T03:47:58.837 INFO:teuthology.orchestra.run.smithi076.stdout: "root": "/" 2023-11-09T03:47:58.837 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:58.837 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_count": 0, 2023-11-09T03:47:58.837 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_pinned_count": 0, 2023-11-09T03:47:58.837 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 6657, 2023-11-09T03:47:58.838 INFO:teuthology.orchestra.run.smithi076.stdout: "inst": { 2023-11-09T03:47:58.838 INFO:teuthology.orchestra.run.smithi076.stdout: "name": { 2023-11-09T03:47:58.838 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "client", 2023-11-09T03:47:58.838 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 6657 2023-11-09T03:47:58.838 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:58.838 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2023-11-09T03:47:58.839 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2023-11-09T03:47:58.839 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2023-11-09T03:47:58.839 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2792820638 2023-11-09T03:47:58.839 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:47:58.839 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:58.839 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2023-11-09T03:47:58.840 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2023-11-09T03:47:58.840 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2023-11-09T03:47:58.840 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2792820638 2023-11-09T03:47:58.840 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:47:58.840 INFO:teuthology.orchestra.run.smithi076.stdout: "inst_str": "client.6657 192.168.0.1:0/2792820638", 2023-11-09T03:47:58.840 INFO:teuthology.orchestra.run.smithi076.stdout: "addr_str": "192.168.0.1:0/2792820638", 2023-11-09T03:47:58.841 INFO:teuthology.orchestra.run.smithi076.stdout: "inode_count": 1, 2023-11-09T03:47:58.841 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_epoch": 29, 2023-11-09T03:47:58.841 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch": 66, 2023-11-09T03:47:58.841 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch_barrier": 0, 2023-11-09T03:47:58.841 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklisted": false, 2023-11-09T03:47:58.841 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs" 2023-11-09T03:47:58.842 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:47:58.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:47:58.850 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:47:58.850 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-11-09T03:47:58.851 DEBUG:teuthology.orchestra.run.smithi076:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-09T03:47:58.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:47:58.909 INFO:teuthology.orchestra.run.smithi076.stdout:fuseblk 2023-11-09T03:47:58.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:47:58.916 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-09T03:47:58.916 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:47:58.917 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-09T03:47:58.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:47:58.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:47:58.983 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create test_data_pool 2023-11-09T03:47:59.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.085+0000 7f8c50064640 1 Processor -- start 2023-11-09T03:47:59.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.085+0000 7f8c50064640 1 -- start start 2023-11-09T03:47:59.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.086+0000 7f8c50064640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c480cd6d0 0x7f8c48161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:59.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.086+0000 7f8c50064640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8c480b4880 0x7f8c480b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:59.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.086+0000 7f8c50064640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c480cde60 con 0x7f8c48153e60 2023-11-09T03:47:59.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.086+0000 7f8c50064640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c48160f90 con 0x7f8c480b4880 2023-11-09T03:47:59.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.086+0000 7f8c50064640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c480b51e0 con 0x7f8c480cd6d0 2023-11-09T03:47:59.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.086+0000 7f8c4de00640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8c48153e60 0x7f8c4815dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:58000/0 (socket says 172.21.15.76:58000) 2023-11-09T03:47:59.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.086+0000 7f8c4de00640 1 -- 172.21.15.76:0/594720343 learned_addr learned my addr 172.21.15.76:0/594720343 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:59.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.086+0000 7f8c4e601640 1 --2- 172.21.15.76:0/594720343 >> v2:172.21.15.76:3300/0 conn(0x7f8c480b4880 0x7f8c480b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:59.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.086+0000 7f8c4d5ff640 1 --2- 172.21.15.76:0/594720343 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c480cd6d0 0x7f8c48161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:59.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.086+0000 7f8c4cdfe640 1 -- 172.21.15.76:0/594720343 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1083915804 0 0) 0x7f8c480cde60 con 0x7f8c48153e60 2023-11-09T03:47:59.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.086+0000 7f8c4cdfe640 1 -- 172.21.15.76:0/594720343 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c34003520 con 0x7f8c48153e60 2023-11-09T03:47:59.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.086+0000 7f8c4e601640 1 -- 172.21.15.76:0/594720343 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c480cd6d0 msgr2=0x7f8c48161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:59.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.086+0000 7f8c4e601640 1 --2- 172.21.15.76:0/594720343 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c480cd6d0 0x7f8c48161420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:59.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.086+0000 7f8c4e601640 1 -- 172.21.15.76:0/594720343 >> v1:172.21.15.76:6789/0 conn(0x7f8c48153e60 legacy=0x7f8c4815dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:59.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.086+0000 7f8c4cdfe640 1 -- 172.21.15.76:0/594720343 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1735942972 0 0) 0x7f8c34003520 con 0x7f8c48153e60 2023-11-09T03:47:59.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.086+0000 7f8c4e601640 1 -- 172.21.15.76:0/594720343 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c48161960 con 0x7f8c480b4880 2023-11-09T03:47:59.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.086+0000 7f8c4d5ff640 1 --2- 172.21.15.76:0/594720343 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c480cd6d0 0x7f8c48161420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:59.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.087+0000 7f8c4e601640 1 --2- 172.21.15.76:0/594720343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c480b4880 0x7f8c480b4ca0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f8c40009d90 tx=0x7f8c4002bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78755284db25ad40 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:59.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.087+0000 7f8c4cdfe640 1 -- 172.21.15.76:0/594720343 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c44001670 con 0x7f8c480b4880 2023-11-09T03:47:59.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.087+0000 7f8c4cdfe640 1 -- 172.21.15.76:0/594720343 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4002c8a0 con 0x7f8c480b4880 2023-11-09T03:47:59.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.087+0000 7f8c4cdfe640 1 -- 172.21.15.76:0/594720343 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4002cc30 con 0x7f8c480b4880 2023-11-09T03:47:59.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.088+0000 7f8c50064640 1 -- 172.21.15.76:0/594720343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c480b4880 msgr2=0x7f8c480b4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:59.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.088+0000 7f8c50064640 1 --2- 172.21.15.76:0/594720343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c480b4880 0x7f8c480b4ca0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f8c40009d90 tx=0x7f8c4002bd70 comp rx=0 tx=0).stop 2023-11-09T03:47:59.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.088+0000 7f8c50064640 1 -- 172.21.15.76:0/594720343 shutdown_connections 2023-11-09T03:47:59.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.088+0000 7f8c50064640 1 --2- 172.21.15.76:0/594720343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c480b4880 0x7f8c480b4ca0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:59.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.088+0000 7f8c50064640 1 --2- 172.21.15.76:0/594720343 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c480cd6d0 0x7f8c48161420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:59.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.088+0000 7f8c50064640 1 -- 172.21.15.76:0/594720343 >> 172.21.15.76:0/594720343 conn(0x7f8c480cf880 msgr2=0x7f8c480cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:59.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.088+0000 7f8c50064640 1 -- 172.21.15.76:0/594720343 shutdown_connections 2023-11-09T03:47:59.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.088+0000 7f8c50064640 1 -- 172.21.15.76:0/594720343 wait complete. 2023-11-09T03:47:59.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.089+0000 7f8c50064640 1 Processor -- start 2023-11-09T03:47:59.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.089+0000 7f8c50064640 1 -- start start 2023-11-09T03:47:59.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.090+0000 7f8c50064640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c480b4880 0x7f8c481533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:59.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.090+0000 7f8c50064640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c480cd6d0 0x7f8c480c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:59.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.090+0000 7f8c50064640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c48153e60 0x7f8c480b7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:59.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.090+0000 7f8c50064640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c481f24c0 con 0x7f8c480cd6d0 2023-11-09T03:47:59.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.090+0000 7f8c50064640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c480ca990 con 0x7f8c48153e60 2023-11-09T03:47:59.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.090+0000 7f8c50064640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c480cac70 con 0x7f8c480b4880 2023-11-09T03:47:59.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.090+0000 7f8c4d5ff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c480cd6d0 0x7f8c480c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:59.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.090+0000 7f8c4d5ff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c480cd6d0 0x7f8c480c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41254/0 (socket says 172.21.15.76:41254) 2023-11-09T03:47:59.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.090+0000 7f8c4d5ff640 1 -- 172.21.15.76:0/3522217799 learned_addr learned my addr 172.21.15.76:0/3522217799 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:59.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.090+0000 7f8c4de00640 1 --2- 172.21.15.76:0/3522217799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c480b4880 0x7f8c481533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:59.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.090+0000 7f8c4e601640 1 --2- 172.21.15.76:0/3522217799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c48153e60 0x7f8c480b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:59.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.090+0000 7f8c4d5ff640 1 -- 172.21.15.76:0/3522217799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c480b4880 msgr2=0x7f8c481533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:59.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.090+0000 7f8c4d5ff640 1 --2- 172.21.15.76:0/3522217799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c480b4880 0x7f8c481533c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:59.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.090+0000 7f8c4d5ff640 1 -- 172.21.15.76:0/3522217799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c48153e60 msgr2=0x7f8c480b7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:59.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.090+0000 7f8c4d5ff640 1 --2- 172.21.15.76:0/3522217799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c48153e60 0x7f8c480b7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:59.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.090+0000 7f8c4d5ff640 1 -- 172.21.15.76:0/3522217799 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c40009a70 con 0x7f8c480cd6d0 2023-11-09T03:47:59.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.090+0000 7f8c4de00640 1 --2- 172.21.15.76:0/3522217799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c480b4880 0x7f8c481533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:47:59.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.090+0000 7f8c4d5ff640 1 --2- 172.21.15.76:0/3522217799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c480cd6d0 0x7f8c480c9ed0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800e7e0 tx=0x7f8c3800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:59.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.090+0000 7f8c3e7fc640 1 -- 172.21.15.76:0/3522217799 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38004030 con 0x7f8c480cd6d0 2023-11-09T03:47:59.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.090+0000 7f8c3e7fc640 1 -- 172.21.15.76:0/3522217799 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c38004190 con 0x7f8c480cd6d0 2023-11-09T03:47:59.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.090+0000 7f8c50064640 1 -- 172.21.15.76:0/3522217799 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c480caf50 con 0x7f8c480cd6d0 2023-11-09T03:47:59.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.090+0000 7f8c3e7fc640 1 -- 172.21.15.76:0/3522217799 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38004520 con 0x7f8c480cd6d0 2023-11-09T03:47:59.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.091+0000 7f8c50064640 1 -- 172.21.15.76:0/3522217799 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c480ca410 con 0x7f8c480cd6d0 2023-11-09T03:47:59.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.092+0000 7f8c50064640 1 -- 172.21.15.76:0/3522217799 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c48074290 con 0x7f8c480cd6d0 2023-11-09T03:47:59.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.092+0000 7f8c3e7fc640 1 -- 172.21.15.76:0/3522217799 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8c3800c6d0 con 0x7f8c480cd6d0 2023-11-09T03:47:59.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.092+0000 7f8c3e7fc640 1 --2- 172.21.15.76:0/3522217799 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8c2c06ae70 0x7f8c2c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:59.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.092+0000 7f8c3e7fc640 1 -- 172.21.15.76:0/3522217799 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8c38089610 con 0x7f8c480cd6d0 2023-11-09T03:47:59.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.093+0000 7f8c4de00640 1 --2- 172.21.15.76:0/3522217799 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8c2c06ae70 0x7f8c2c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:59.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.093+0000 7f8c4de00640 1 --2- 172.21.15.76:0/3522217799 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8c2c06ae70 0x7f8c2c06d330 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8c440017a0 tx=0x7f8c44009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:59.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.096+0000 7f8c3e7fc640 1 -- 172.21.15.76:0/3522217799 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8c38054420 con 0x7f8c480cd6d0 2023-11-09T03:47:59.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.262+0000 7f8c50064640 1 -- 172.21.15.76:0/3522217799 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "test_data_pool"} v 0) v1 -- 0x7f8c48153910 con 0x7f8c480cd6d0 2023-11-09T03:47:59.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.421+0000 7f8c3e7fc640 1 -- 172.21.15.76:0/3522217799 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "test_data_pool"}]=0 pool 'test_data_pool' created v67) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8c38057a40 con 0x7f8c480cd6d0 2023-11-09T03:47:59.421 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'test_data_pool' created 2023-11-09T03:47:59.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.424+0000 7f8c50064640 1 -- 172.21.15.76:0/3522217799 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8c2c06ae70 msgr2=0x7f8c2c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:59.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.424+0000 7f8c50064640 1 --2- 172.21.15.76:0/3522217799 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8c2c06ae70 0x7f8c2c06d330 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8c440017a0 tx=0x7f8c44009040 comp rx=0 tx=0).stop 2023-11-09T03:47:59.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.424+0000 7f8c50064640 1 -- 172.21.15.76:0/3522217799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c480cd6d0 msgr2=0x7f8c480c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:59.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.424+0000 7f8c50064640 1 --2- 172.21.15.76:0/3522217799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c480cd6d0 0x7f8c480c9ed0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800e7e0 tx=0x7f8c3800eca0 comp rx=0 tx=0).stop 2023-11-09T03:47:59.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.424+0000 7f8c50064640 1 -- 172.21.15.76:0/3522217799 shutdown_connections 2023-11-09T03:47:59.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.424+0000 7f8c50064640 1 --2- 172.21.15.76:0/3522217799 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8c2c06ae70 0x7f8c2c06d330 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:59.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.424+0000 7f8c50064640 1 --2- 172.21.15.76:0/3522217799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c48153e60 0x7f8c480b7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:59.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.424+0000 7f8c50064640 1 --2- 172.21.15.76:0/3522217799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c480cd6d0 0x7f8c480c9ed0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:59.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.424+0000 7f8c50064640 1 --2- 172.21.15.76:0/3522217799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c480b4880 0x7f8c481533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:59.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.424+0000 7f8c50064640 1 -- 172.21.15.76:0/3522217799 >> 172.21.15.76:0/3522217799 conn(0x7f8c480cf880 msgr2=0x7f8c4815beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:59.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.425+0000 7f8c50064640 1 -- 172.21.15.76:0/3522217799 shutdown_connections 2023-11-09T03:47:59.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.426+0000 7f8c50064640 1 -- 172.21.15.76:0/3522217799 wait complete. 2023-11-09T03:47:59.439 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create test_metadata_pool 2023-11-09T03:47:59.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.543+0000 7fade0064640 1 Processor -- start 2023-11-09T03:47:59.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.543+0000 7fade0064640 1 -- start start 2023-11-09T03:47:59.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.544+0000 7fade0064640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadd8159c00 0x7fadd80cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:59.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.544+0000 7fade0064640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fadd80ae1d0 0x7fadd80c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:59.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.544+0000 7fade0064640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadd815aff0 con 0x7fadd80c48e0 2023-11-09T03:47:59.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.544+0000 7fade0064640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadd80c52a0 con 0x7fadd80ae1d0 2023-11-09T03:47:59.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.544+0000 7fade0064640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd815a3e0 con 0x7fadd8159c00 2023-11-09T03:47:59.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.544+0000 7faddde00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadd8159c00 0x7fadd80cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:59.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.544+0000 7faddd5ff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fadd80c48e0 0x7fadd80adac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:57476/0 (socket says 172.21.15.76:57476) 2023-11-09T03:47:59.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.544+0000 7faddde00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadd8159c00 0x7fadd80cc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:60760/0 (socket says 172.21.15.76:60760) 2023-11-09T03:47:59.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.544+0000 7faddd5ff640 1 -- 172.21.15.76:0/2051656955 learned_addr learned my addr 172.21.15.76:0/2051656955 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:47:59.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.544+0000 7fadde601640 1 --2- 172.21.15.76:0/2051656955 >> v2:172.21.15.113:3300/0 conn(0x7fadd80ae1d0 0x7fadd80c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:59.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.545+0000 7faddde00640 1 -- 172.21.15.76:0/2051656955 >> v2:172.21.15.113:3300/0 conn(0x7fadd80ae1d0 msgr2=0x7fadd80c4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:47:59.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.545+0000 7faddde00640 1 --2- 172.21.15.76:0/2051656955 >> v2:172.21.15.113:3300/0 conn(0x7fadd80ae1d0 0x7fadd80c4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:59.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.545+0000 7faddde00640 1 -- 172.21.15.76:0/2051656955 >> v1:172.21.15.113:6789/0 conn(0x7fadd80c48e0 legacy=0x7fadd80adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:59.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.545+0000 7faddde00640 1 -- 172.21.15.76:0/2051656955 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd815a6c0 con 0x7fadd8159c00 2023-11-09T03:47:59.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.545+0000 7faddcdfe640 1 -- 172.21.15.76:0/2051656955 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2770275608 0 0) 0x7fadd815aff0 con 0x7fadd80c48e0 2023-11-09T03:47:59.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.545+0000 7fadde601640 1 --2- 172.21.15.76:0/2051656955 >> v2:172.21.15.113:3300/0 conn(0x7fadd80ae1d0 0x7fadd80c4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:47:59.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.546+0000 7faddde00640 1 --2- 172.21.15.76:0/2051656955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadd8159c00 0x7fadd80cc890 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009d90 tx=0x7fadd402beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=150d8fff069b73f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:59.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.546+0000 7faddcdfe640 1 -- 172.21.15.76:0/2051656955 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd402cc60 con 0x7fadd8159c00 2023-11-09T03:47:59.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.546+0000 7faddcdfe640 1 -- 172.21.15.76:0/2051656955 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd402cdc0 con 0x7fadd8159c00 2023-11-09T03:47:59.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.546+0000 7faddcdfe640 1 -- 172.21.15.76:0/2051656955 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd4034660 con 0x7fadd8159c00 2023-11-09T03:47:59.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.546+0000 7fade0064640 1 -- 172.21.15.76:0/2051656955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadd8159c00 msgr2=0x7fadd80cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:59.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.546+0000 7fade0064640 1 --2- 172.21.15.76:0/2051656955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadd8159c00 0x7fadd80cc890 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009d90 tx=0x7fadd402beb0 comp rx=0 tx=0).stop 2023-11-09T03:47:59.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.546+0000 7fade0064640 1 -- 172.21.15.76:0/2051656955 shutdown_connections 2023-11-09T03:47:59.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.546+0000 7fade0064640 1 --2- 172.21.15.76:0/2051656955 >> v2:172.21.15.113:3300/0 conn(0x7fadd80ae1d0 0x7fadd80c4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:59.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.546+0000 7fade0064640 1 --2- 172.21.15.76:0/2051656955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadd8159c00 0x7fadd80cc890 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:59.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.546+0000 7fade0064640 1 -- 172.21.15.76:0/2051656955 >> 172.21.15.76:0/2051656955 conn(0x7fadd80ce950 msgr2=0x7fadd80ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:47:59.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.546+0000 7fade0064640 1 -- 172.21.15.76:0/2051656955 shutdown_connections 2023-11-09T03:47:59.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.547+0000 7fade0064640 1 -- 172.21.15.76:0/2051656955 wait complete. 2023-11-09T03:47:59.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.548+0000 7fade0064640 1 Processor -- start 2023-11-09T03:47:59.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.548+0000 7fade0064640 1 -- start start 2023-11-09T03:47:59.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.548+0000 7fade0064640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadd80ae1d0 0x7fadd81452b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:59.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.548+0000 7fade0064640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadd80c48e0 0x7fadd81589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:59.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.548+0000 7fade0064640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadd8159c00 0x7fadd8158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:59.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.548+0000 7fade0064640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd8148ba0 con 0x7fadd8159c00 2023-11-09T03:47:59.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.548+0000 7fade0064640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd81457f0 con 0x7fadd80c48e0 2023-11-09T03:47:59.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.548+0000 7fade0064640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd81488c0 con 0x7fadd80ae1d0 2023-11-09T03:47:59.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.548+0000 7fadde601640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadd8159c00 0x7fadd8158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:59.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.548+0000 7fadde601640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadd8159c00 0x7fadd8158f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41262/0 (socket says 172.21.15.76:41262) 2023-11-09T03:47:59.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.549+0000 7fadde601640 1 -- 172.21.15.76:0/2110598156 learned_addr learned my addr 172.21.15.76:0/2110598156 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:47:59.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.549+0000 7faddd5ff640 1 --2- 172.21.15.76:0/2110598156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadd80c48e0 0x7fadd81589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:59.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.549+0000 7faddde00640 1 --2- 172.21.15.76:0/2110598156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadd80ae1d0 0x7fadd81452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:59.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.549+0000 7fadde601640 1 -- 172.21.15.76:0/2110598156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadd80ae1d0 msgr2=0x7fadd81452b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:59.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.549+0000 7fadde601640 1 --2- 172.21.15.76:0/2110598156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadd80ae1d0 0x7fadd81452b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:59.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.549+0000 7fadde601640 1 -- 172.21.15.76:0/2110598156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadd80c48e0 msgr2=0x7fadd81589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:47:59.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.549+0000 7fadde601640 1 --2- 172.21.15.76:0/2110598156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadd80c48e0 0x7fadd81589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:47:59.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.549+0000 7fadde601640 1 -- 172.21.15.76:0/2110598156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd0009400 con 0x7fadd8159c00 2023-11-09T03:47:59.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.549+0000 7fadde601640 1 --2- 172.21.15.76:0/2110598156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadd8159c00 0x7fadd8158f30 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fadd000e880 tx=0x7fadd000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:59.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.549+0000 7fadce7fc640 1 -- 172.21.15.76:0/2110598156 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd0009660 con 0x7fadd8159c00 2023-11-09T03:47:59.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.549+0000 7fade0064640 1 -- 172.21.15.76:0/2110598156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd4009a70 con 0x7fadd8159c00 2023-11-09T03:47:59.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.549+0000 7fadce7fc640 1 -- 172.21.15.76:0/2110598156 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd0004020 con 0x7fadd8159c00 2023-11-09T03:47:59.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.549+0000 7fadce7fc640 1 -- 172.21.15.76:0/2110598156 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd00043b0 con 0x7fadd8159c00 2023-11-09T03:47:59.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.549+0000 7fade0064640 1 -- 172.21.15.76:0/2110598156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd81f3ed0 con 0x7fadd8159c00 2023-11-09T03:47:59.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.550+0000 7fade0064640 1 -- 172.21.15.76:0/2110598156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd81f3a50 con 0x7fadd8159c00 2023-11-09T03:47:59.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.551+0000 7fadce7fc640 1 -- 172.21.15.76:0/2110598156 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fadd000c940 con 0x7fadd8159c00 2023-11-09T03:47:59.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.551+0000 7fadce7fc640 1 --2- 172.21.15.76:0/2110598156 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fadbc06ae70 0x7fadbc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:47:59.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.551+0000 7fadce7fc640 1 -- 172.21.15.76:0/2110598156 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fadd00898e0 con 0x7fadd8159c00 2023-11-09T03:47:59.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.551+0000 7faddde00640 1 --2- 172.21.15.76:0/2110598156 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fadbc06ae70 0x7fadbc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:47:59.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.552+0000 7faddde00640 1 --2- 172.21.15.76:0/2110598156 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fadbc06ae70 0x7fadbc06d330 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fadd4036000 tx=0x7fadd4030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:47:59.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.555+0000 7fadce7fc640 1 -- 172.21.15.76:0/2110598156 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fadd81f3a50 con 0x7fadd8159c00 2023-11-09T03:47:59.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:47:59.723+0000 7fade0064640 1 -- 172.21.15.76:0/2110598156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "test_metadata_pool"} v 0) v1 -- 0x7fadd81f3a50 con 0x7fadd8159c00 2023-11-09T03:48:00.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.428+0000 7fadce7fc640 1 -- 172.21.15.76:0/2110598156 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "test_metadata_pool"}]=0 pool 'test_metadata_pool' created v68) v1 ==== 126+0+0 (secure 0 0 0) 0x7fadd81f3a50 con 0x7fadd8159c00 2023-11-09T03:48:00.428 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'test_metadata_pool' created 2023-11-09T03:48:00.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.432+0000 7fade0064640 1 -- 172.21.15.76:0/2110598156 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fadbc06ae70 msgr2=0x7fadbc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:00.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.432+0000 7fade0064640 1 --2- 172.21.15.76:0/2110598156 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fadbc06ae70 0x7fadbc06d330 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fadd4036000 tx=0x7fadd4030000 comp rx=0 tx=0).stop 2023-11-09T03:48:00.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.432+0000 7fade0064640 1 -- 172.21.15.76:0/2110598156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadd8159c00 msgr2=0x7fadd8158f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:00.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.432+0000 7fade0064640 1 --2- 172.21.15.76:0/2110598156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadd8159c00 0x7fadd8158f30 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fadd000e880 tx=0x7fadd000ed40 comp rx=0 tx=0).stop 2023-11-09T03:48:00.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.433+0000 7fade0064640 1 -- 172.21.15.76:0/2110598156 shutdown_connections 2023-11-09T03:48:00.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.433+0000 7fade0064640 1 --2- 172.21.15.76:0/2110598156 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fadbc06ae70 0x7fadbc06d330 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:00.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.433+0000 7fade0064640 1 --2- 172.21.15.76:0/2110598156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadd8159c00 0x7fadd8158f30 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:00.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.433+0000 7fade0064640 1 --2- 172.21.15.76:0/2110598156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadd80c48e0 0x7fadd81589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:00.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.433+0000 7fade0064640 1 --2- 172.21.15.76:0/2110598156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadd80ae1d0 0x7fadd81452b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:00.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.433+0000 7fade0064640 1 -- 172.21.15.76:0/2110598156 >> 172.21.15.76:0/2110598156 conn(0x7fadd80ce950 msgr2=0x7fadd815c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:00.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.435+0000 7fade0064640 1 -- 172.21.15.76:0/2110598156 shutdown_connections 2023-11-09T03:48:00.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.437+0000 7fade0064640 1 -- 172.21.15.76:0/2110598156 wait complete. 2023-11-09T03:48:00.456 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool test_data_pool mksnap snap4 2023-11-09T03:48:00.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.500+0000 7fce6553cf40 1 Processor -- start 2023-11-09T03:48:00.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.500+0000 7fce6553cf40 1 -- start start 2023-11-09T03:48:00.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.500+0000 7fce6553cf40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce4c002260 0x55a5d8bb14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:00.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.500+0000 7fce6553cf40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5d8bb3df0 0x55a5d8b03210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:00.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.500+0000 7fce6553cf40 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a5d8b88dc0 0x55a5d8ad6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:00.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.500+0000 7fce6553cf40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5d8a873b0 con 0x7fce4c002260 2023-11-09T03:48:00.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.500+0000 7fce6553cf40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5d8a21650 con 0x55a5d8bb3df0 2023-11-09T03:48:00.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.500+0000 7fce6553cf40 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5d8a212f0 con 0x55a5d8b88dc0 2023-11-09T03:48:00.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.500+0000 7fce6303e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce4c002260 0x55a5d8bb14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:00.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.501+0000 7fce6303e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce4c002260 0x55a5d8bb14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41268/0 (socket says 172.21.15.76:41268) 2023-11-09T03:48:00.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.501+0000 7fce6303e640 1 -- 172.21.15.76:0/3747178023 learned_addr learned my addr 172.21.15.76:0/3747178023 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:00.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.501+0000 7fce6283d640 1 --2- 172.21.15.76:0/3747178023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5d8bb3df0 0x55a5d8b03210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:00.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.501+0000 7fce6383f640 1 --2- 172.21.15.76:0/3747178023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a5d8b88dc0 0x55a5d8ad6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:00.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.501+0000 7fce6303e640 1 -- 172.21.15.76:0/3747178023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a5d8b88dc0 msgr2=0x55a5d8ad6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:00.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.501+0000 7fce6303e640 1 --2- 172.21.15.76:0/3747178023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a5d8b88dc0 0x55a5d8ad6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:00.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.501+0000 7fce6303e640 1 -- 172.21.15.76:0/3747178023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5d8bb3df0 msgr2=0x55a5d8b03210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:00.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.501+0000 7fce6303e640 1 --2- 172.21.15.76:0/3747178023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5d8bb3df0 0x55a5d8b03210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:00.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.501+0000 7fce6303e640 1 -- 172.21.15.76:0/3747178023 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5d8a2d780 con 0x7fce4c002260 2023-11-09T03:48:00.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.501+0000 7fce6303e640 1 --2- 172.21.15.76:0/3747178023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce4c002260 0x55a5d8bb14f0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fce54056040 tx=0x7fce54003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca4dfaee61ff717 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:00.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.501+0000 7fce6183b640 1 -- 172.21.15.76:0/3747178023 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce54005050 con 0x7fce4c002260 2023-11-09T03:48:00.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.501+0000 7fce6183b640 1 -- 172.21.15.76:0/3747178023 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce540051b0 con 0x7fce4c002260 2023-11-09T03:48:00.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.501+0000 7fce6183b640 1 -- 172.21.15.76:0/3747178023 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce54005520 con 0x7fce4c002260 2023-11-09T03:48:00.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.502+0000 7fce6553cf40 1 -- 172.21.15.76:0/3747178023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce4c002260 msgr2=0x55a5d8bb14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:00.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.502+0000 7fce6553cf40 1 --2- 172.21.15.76:0/3747178023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce4c002260 0x55a5d8bb14f0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fce54056040 tx=0x7fce54003330 comp rx=0 tx=0).stop 2023-11-09T03:48:00.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.502+0000 7fce6553cf40 1 -- 172.21.15.76:0/3747178023 shutdown_connections 2023-11-09T03:48:00.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.502+0000 7fce6553cf40 1 --2- 172.21.15.76:0/3747178023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a5d8b88dc0 0x55a5d8ad6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:00.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.502+0000 7fce6553cf40 1 --2- 172.21.15.76:0/3747178023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5d8bb3df0 0x55a5d8b03210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:00.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.502+0000 7fce6553cf40 1 --2- 172.21.15.76:0/3747178023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce4c002260 0x55a5d8bb14f0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:00.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.502+0000 7fce6553cf40 1 -- 172.21.15.76:0/3747178023 >> 172.21.15.76:0/3747178023 conn(0x55a5d8b03f40 msgr2=0x55a5d8bad040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:00.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.502+0000 7fce6553cf40 1 -- 172.21.15.76:0/3747178023 shutdown_connections 2023-11-09T03:48:00.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.502+0000 7fce6553cf40 1 -- 172.21.15.76:0/3747178023 wait complete. 2023-11-09T03:48:00.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.502+0000 7fce6553cf40 1 Processor -- start 2023-11-09T03:48:00.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.502+0000 7fce6553cf40 1 -- start start 2023-11-09T03:48:00.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.503+0000 7fce6553cf40 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a5d8b88dc0 0x55a5d8b8fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:00.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.503+0000 7fce6553cf40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5d8bb3df0 0x55a5d8baf420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:00.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.503+0000 7fce6553cf40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce4c002260 0x55a5d8b03400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:00.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.503+0000 7fce6553cf40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5d8a014f0 con 0x7fce4c002260 2023-11-09T03:48:00.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.503+0000 7fce6553cf40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5d8ba90e0 con 0x55a5d8bb3df0 2023-11-09T03:48:00.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.503+0000 7fce6553cf40 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5d8ba8e00 con 0x55a5d8b88dc0 2023-11-09T03:48:00.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.503+0000 7fce6383f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce4c002260 0x55a5d8b03400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:00.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.503+0000 7fce6383f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce4c002260 0x55a5d8b03400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41278/0 (socket says 172.21.15.76:41278) 2023-11-09T03:48:00.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.503+0000 7fce6383f640 1 -- 172.21.15.76:0/122369636 learned_addr learned my addr 172.21.15.76:0/122369636 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:00.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.503+0000 7fce6283d640 1 --2- 172.21.15.76:0/122369636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5d8bb3df0 0x55a5d8baf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:00.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.503+0000 7fce6303e640 1 --2- 172.21.15.76:0/122369636 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a5d8b88dc0 0x55a5d8b8fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:00.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.503+0000 7fce6383f640 1 -- 172.21.15.76:0/122369636 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a5d8b88dc0 msgr2=0x55a5d8b8fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:00.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.503+0000 7fce6383f640 1 --2- 172.21.15.76:0/122369636 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a5d8b88dc0 0x55a5d8b8fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:00.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.503+0000 7fce6383f640 1 -- 172.21.15.76:0/122369636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5d8bb3df0 msgr2=0x55a5d8baf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:00.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.504+0000 7fce6383f640 1 --2- 172.21.15.76:0/122369636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5d8bb3df0 0x55a5d8baf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:00.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.504+0000 7fce6383f640 1 -- 172.21.15.76:0/122369636 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c001e10 con 0x7fce4c002260 2023-11-09T03:48:00.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.504+0000 7fce6383f640 1 --2- 172.21.15.76:0/122369636 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce4c002260 0x55a5d8b03400 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x55a5d8b1ec80 tx=0x7fce5c00e7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:00.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.504+0000 7fce60839640 1 -- 172.21.15.76:0/122369636 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c00d7f0 con 0x7fce4c002260 2023-11-09T03:48:00.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.504+0000 7fce60839640 1 -- 172.21.15.76:0/122369636 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5c00eed0 con 0x7fce4c002260 2023-11-09T03:48:00.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.504+0000 7fce60839640 1 -- 172.21.15.76:0/122369636 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c00db50 con 0x7fce4c002260 2023-11-09T03:48:00.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.504+0000 7fce6553cf40 1 -- 172.21.15.76:0/122369636 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a5d8a20930 con 0x7fce4c002260 2023-11-09T03:48:00.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.504+0000 7fce6553cf40 1 -- 172.21.15.76:0/122369636 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5d8a20390 con 0x7fce4c002260 2023-11-09T03:48:00.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.505+0000 7fce60839640 1 -- 172.21.15.76:0/122369636 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fce5c00dcb0 con 0x7fce4c002260 2023-11-09T03:48:00.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.506+0000 7fce60839640 1 --2- 172.21.15.76:0/122369636 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fce50069820 0x7fce5006bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:00.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.506+0000 7fce60839640 1 -- 172.21.15.76:0/122369636 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7092+0+0 (secure 0 0 0) 0x7fce5c089160 con 0x7fce4c002260 2023-11-09T03:48:00.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.506+0000 7fce6303e640 1 --2- 172.21.15.76:0/122369636 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fce50069820 0x7fce5006bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:00.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.506+0000 7fce6553cf40 1 -- 172.21.15.76:0/122369636 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- pool_op(create snap pool 12 tid 1 name snap4 v0) v4 -- 0x55a5d8a91330 con 0x7fce4c002260 2023-11-09T03:48:00.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:00.506+0000 7fce6303e640 1 --2- 172.21.15.76:0/122369636 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fce50069820 0x7fce5006bce0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fce54056040 tx=0x7fce5404c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:00.913 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:48:00.916 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:48:01.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.440+0000 7fce60839640 1 -- 172.21.15.76:0/122369636 <== mon.0 v2:172.21.15.76:3300/0 6 ==== pool_op_reply(tid 1 (0) Success v69) v1 ==== 43+0+0 (secure 0 0 0) 0x7fce5c01e080 con 0x7fce4c002260 2023-11-09T03:48:01.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.440+0000 7fce60839640 1 -- 172.21.15.76:0/122369636 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=69}) v3 -- 0x7fce50073f00 con 0x7fce4c002260 2023-11-09T03:48:01.440 INFO:teuthology.orchestra.run.smithi076.stdout:created pool test_data_pool snap snap4 2023-11-09T03:48:01.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.440+0000 7fce60839640 1 -- 172.21.15.76:0/122369636 <== mon.0 v2:172.21.15.76:3300/0 7 ==== osd_map(69..69 src has 1..69) v4 ==== 718+0+0 (secure 0 0 0) 0x7fce5c057aa0 con 0x7fce4c002260 2023-11-09T03:48:01.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.441+0000 7fce6553cf40 1 -- 172.21.15.76:0/122369636 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fce50069820 msgr2=0x7fce5006bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:01.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.441+0000 7fce6553cf40 1 --2- 172.21.15.76:0/122369636 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fce50069820 0x7fce5006bce0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fce54056040 tx=0x7fce5404c000 comp rx=0 tx=0).stop 2023-11-09T03:48:01.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.441+0000 7fce6553cf40 1 -- 172.21.15.76:0/122369636 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce4c002260 msgr2=0x55a5d8b03400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:01.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.441+0000 7fce6553cf40 1 --2- 172.21.15.76:0/122369636 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce4c002260 0x55a5d8b03400 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x55a5d8b1ec80 tx=0x7fce5c00e7b0 comp rx=0 tx=0).stop 2023-11-09T03:48:01.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.441+0000 7fce6553cf40 1 -- 172.21.15.76:0/122369636 shutdown_connections 2023-11-09T03:48:01.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.441+0000 7fce6553cf40 1 --2- 172.21.15.76:0/122369636 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fce50069820 0x7fce5006bce0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:01.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.441+0000 7fce6553cf40 1 --2- 172.21.15.76:0/122369636 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce4c002260 0x55a5d8b03400 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:01.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.441+0000 7fce6553cf40 1 --2- 172.21.15.76:0/122369636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5d8bb3df0 0x55a5d8baf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:01.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.441+0000 7fce6553cf40 1 --2- 172.21.15.76:0/122369636 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a5d8b88dc0 0x55a5d8b8fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:01.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.441+0000 7fce6553cf40 1 -- 172.21.15.76:0/122369636 >> 172.21.15.76:0/122369636 conn(0x55a5d8b03f40 msgr2=0x55a5d8ad5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:01.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.442+0000 7fce6553cf40 1 -- 172.21.15.76:0/122369636 shutdown_connections 2023-11-09T03:48:01.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.442+0000 7fce6553cf40 1 -- 172.21.15.76:0/122369636 wait complete. 2023-11-09T03:48:01.447 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new testfs test_metadata_pool test_data_pool 2023-11-09T03:48:01.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.561+0000 7efc7a4b4640 1 Processor -- start 2023-11-09T03:48:01.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.561+0000 7efc7a4b4640 1 -- start start 2023-11-09T03:48:01.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.562+0000 7efc7a4b4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc74160430 0x7efc740b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:01.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.562+0000 7efc7a4b4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efc7406c3a0 0x7efc740cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:01.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.562+0000 7efc7a4b4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc740b2fc0 con 0x7efc740b2b60 2023-11-09T03:48:01.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.562+0000 7efc7a4b4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc740cb580 con 0x7efc7406c3a0 2023-11-09T03:48:01.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.562+0000 7efc7a4b4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc7406ca40 con 0x7efc74160430 2023-11-09T03:48:01.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.563+0000 7efc78a51640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efc7406c3a0 0x7efc740cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:01.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.563+0000 7efc78a51640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efc7406c3a0 0x7efc740cbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:56932/0 (socket says 172.21.15.76:56932) 2023-11-09T03:48:01.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.563+0000 7efc73fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc74160430 0x7efc740b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:01.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.563+0000 7efc78a51640 1 -- 172.21.15.76:0/3933918715 learned_addr learned my addr 172.21.15.76:0/3933918715 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:01.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.563+0000 7efc737fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7efc740b2b60 0x7efc7406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:57492/0 (socket says 172.21.15.76:57492) 2023-11-09T03:48:01.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.563+0000 7efc78a51640 1 -- 172.21.15.76:0/3933918715 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc74160430 msgr2=0x7efc740b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:01.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.563+0000 7efc78a51640 1 --2- 172.21.15.76:0/3933918715 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc74160430 0x7efc740b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:01.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.563+0000 7efc78a51640 1 -- 172.21.15.76:0/3933918715 >> v1:172.21.15.113:6789/0 conn(0x7efc740b2b60 legacy=0x7efc7406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:01.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.563+0000 7efc78a51640 1 -- 172.21.15.76:0/3933918715 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc7406c810 con 0x7efc7406c3a0 2023-11-09T03:48:01.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.563+0000 7efc73fff640 1 --2- 172.21.15.76:0/3933918715 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc74160430 0x7efc740b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:48:01.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.564+0000 7efc78a51640 1 --2- 172.21.15.76:0/3933918715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc7406c3a0 0x7efc740cbcb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efc6c009e50 tx=0x7efc6c02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=802dc6047991982d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:01.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.564+0000 7efc72ffd640 1 -- 172.21.15.76:0/3933918715 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c02cb00 con 0x7efc7406c3a0 2023-11-09T03:48:01.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.564+0000 7efc72ffd640 1 -- 172.21.15.76:0/3933918715 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc6c02cc60 con 0x7efc7406c3a0 2023-11-09T03:48:01.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.564+0000 7efc72ffd640 1 -- 172.21.15.76:0/3933918715 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c0343f0 con 0x7efc7406c3a0 2023-11-09T03:48:01.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.564+0000 7efc7a4b4640 1 -- 172.21.15.76:0/3933918715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc7406c3a0 msgr2=0x7efc740cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:01.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.564+0000 7efc7a4b4640 1 --2- 172.21.15.76:0/3933918715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc7406c3a0 0x7efc740cbcb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efc6c009e50 tx=0x7efc6c02bf60 comp rx=0 tx=0).stop 2023-11-09T03:48:01.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.565+0000 7efc7a4b4640 1 -- 172.21.15.76:0/3933918715 shutdown_connections 2023-11-09T03:48:01.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.565+0000 7efc7a4b4640 1 --2- 172.21.15.76:0/3933918715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc7406c3a0 0x7efc740cbcb0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:01.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.565+0000 7efc7a4b4640 1 --2- 172.21.15.76:0/3933918715 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc74160430 0x7efc740b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:01.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.565+0000 7efc7a4b4640 1 -- 172.21.15.76:0/3933918715 >> 172.21.15.76:0/3933918715 conn(0x7efc7406ea20 msgr2=0x7efc7406ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:01.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.565+0000 7efc7a4b4640 1 -- 172.21.15.76:0/3933918715 shutdown_connections 2023-11-09T03:48:01.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.565+0000 7efc7a4b4640 1 -- 172.21.15.76:0/3933918715 wait complete. 2023-11-09T03:48:01.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.566+0000 7efc7a4b4640 1 Processor -- start 2023-11-09T03:48:01.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.566+0000 7efc7a4b4640 1 -- start start 2023-11-09T03:48:01.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.567+0000 7efc7a4b4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc7406c3a0 0x7efc740c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:01.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.567+0000 7efc7a4b4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc740b2b60 0x7efc740b4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:01.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.567+0000 7efc7a4b4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc74160430 0x7efc74151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:01.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.567+0000 7efc7a4b4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7415b8e0 con 0x7efc7406c3a0 2023-11-09T03:48:01.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.567+0000 7efc7a4b4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc74152460 con 0x7efc740b2b60 2023-11-09T03:48:01.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.567+0000 7efc7a4b4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc740b4e30 con 0x7efc74160430 2023-11-09T03:48:01.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.567+0000 7efc73fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc7406c3a0 0x7efc740c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:01.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.567+0000 7efc73fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc7406c3a0 0x7efc740c6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41286/0 (socket says 172.21.15.76:41286) 2023-11-09T03:48:01.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.567+0000 7efc73fff640 1 -- 172.21.15.76:0/1771647283 learned_addr learned my addr 172.21.15.76:0/1771647283 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:01.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.567+0000 7efc737fe640 1 --2- 172.21.15.76:0/1771647283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc740b2b60 0x7efc740b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:01.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.567+0000 7efc78a51640 1 --2- 172.21.15.76:0/1771647283 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc74160430 0x7efc74151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:01.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.567+0000 7efc73fff640 1 -- 172.21.15.76:0/1771647283 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc74160430 msgr2=0x7efc74151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:01.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.567+0000 7efc73fff640 1 --2- 172.21.15.76:0/1771647283 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc74160430 0x7efc74151f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:01.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.567+0000 7efc73fff640 1 -- 172.21.15.76:0/1771647283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc740b2b60 msgr2=0x7efc740b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:01.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.567+0000 7efc73fff640 1 --2- 172.21.15.76:0/1771647283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc740b2b60 0x7efc740b4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:01.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.567+0000 7efc73fff640 1 -- 172.21.15.76:0/1771647283 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc64009400 con 0x7efc7406c3a0 2023-11-09T03:48:01.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.567+0000 7efc73fff640 1 --2- 172.21.15.76:0/1771647283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc7406c3a0 0x7efc740c6f10 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7efc6400ebe0 tx=0x7efc6400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:01.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.567+0000 7efc70ff9640 1 -- 172.21.15.76:0/1771647283 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc64009760 con 0x7efc7406c3a0 2023-11-09T03:48:01.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.567+0000 7efc70ff9640 1 -- 172.21.15.76:0/1771647283 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc6400cc30 con 0x7efc7406c3a0 2023-11-09T03:48:01.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.567+0000 7efc7a4b4640 1 -- 172.21.15.76:0/1771647283 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc6c009b30 con 0x7efc7406c3a0 2023-11-09T03:48:01.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.567+0000 7efc70ff9640 1 -- 172.21.15.76:0/1771647283 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6400eee0 con 0x7efc7406c3a0 2023-11-09T03:48:01.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.567+0000 7efc7a4b4640 1 -- 172.21.15.76:0/1771647283 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc740b53f0 con 0x7efc7406c3a0 2023-11-09T03:48:01.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.569+0000 7efc7a4b4640 1 -- 172.21.15.76:0/1771647283 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc74066920 con 0x7efc7406c3a0 2023-11-09T03:48:01.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.569+0000 7efc70ff9640 1 -- 172.21.15.76:0/1771647283 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efc64013470 con 0x7efc7406c3a0 2023-11-09T03:48:01.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.569+0000 7efc70ff9640 1 --2- 172.21.15.76:0/1771647283 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efc6006ae70 0x7efc6006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:01.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.569+0000 7efc70ff9640 1 -- 172.21.15.76:0/1771647283 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7131+0+0 (secure 0 0 0) 0x7efc64089e40 con 0x7efc7406c3a0 2023-11-09T03:48:01.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.570+0000 7efc737fe640 1 --2- 172.21.15.76:0/1771647283 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efc6006ae70 0x7efc6006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:01.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.570+0000 7efc737fe640 1 --2- 172.21.15.76:0/1771647283 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efc6006ae70 0x7efc6006d330 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efc740b41a0 tx=0x7efc68000ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:01.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.574+0000 7efc70ff9640 1 -- 172.21.15.76:0/1771647283 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efc64054a10 con 0x7efc7406c3a0 2023-11-09T03:48:01.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.778+0000 7efc7a4b4640 1 -- 172.21.15.76:0/1771647283 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "testfs", "metadata": "test_metadata_pool", "data": "test_data_pool"} v 0) v1 -- 0x7efc74074290 con 0x7efc7406c3a0 2023-11-09T03:48:01.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.778+0000 7efc70ff9640 1 -- 172.21.15.76:0/1771647283 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "testfs", "metadata": "test_metadata_pool", "data": "test_data_pool"}]=-95 pool(12) already has mon-managed snaps; can't attach pool to fs v29) v1 ==== 198+0+0 (secure 0 0 0) 0x7efc640170c0 con 0x7efc7406c3a0 2023-11-09T03:48:01.778 INFO:teuthology.orchestra.run.smithi076.stderr:Error ENOTSUP: pool(12) already has mon-managed snaps; can't attach pool to fs 2023-11-09T03:48:01.782 DEBUG:teuthology.orchestra.run:got remote process result: 95 2023-11-09T03:48:01.783 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool test_data_pool rmsnap snap4 2023-11-09T03:48:01.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.824+0000 7f9f51640f40 1 Processor -- start 2023-11-09T03:48:01.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.824+0000 7f9f51640f40 1 -- start start 2023-11-09T03:48:01.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.824+0000 7f9f51640f40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558600285d20 0x55860035ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:01.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.824+0000 7f9f51640f40 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55860028aa60 0x558600342410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:01.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.824+0000 7f9f51640f40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558600345000 0x558600361d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:01.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.825+0000 7f9f51640f40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586001d1650 con 0x558600345000 2023-11-09T03:48:01.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.825+0000 7f9f51640f40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586001d12f0 con 0x558600285d20 2023-11-09T03:48:01.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.825+0000 7f9f51640f40 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586001b14f0 con 0x55860028aa60 2023-11-09T03:48:01.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.825+0000 7f9f4f20a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558600345000 0x558600361d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:01.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.825+0000 7f9f4f20a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558600345000 0x558600361d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41310/0 (socket says 172.21.15.76:41310) 2023-11-09T03:48:01.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.825+0000 7f9f4f20a640 1 -- 172.21.15.76:0/2628914870 learned_addr learned my addr 172.21.15.76:0/2628914870 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:01.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.825+0000 7f9f4ea09640 1 --2- 172.21.15.76:0/2628914870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558600285d20 0x55860035ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:01.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.825+0000 7f9f4f20a640 1 -- 172.21.15.76:0/2628914870 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55860028aa60 msgr2=0x558600342410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:01.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.825+0000 7f9f4f20a640 1 --2- 172.21.15.76:0/2628914870 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55860028aa60 0x558600342410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:01.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.825+0000 7f9f4f20a640 1 -- 172.21.15.76:0/2628914870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558600285d20 msgr2=0x55860035ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:01.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.825+0000 7f9f4f20a640 1 --2- 172.21.15.76:0/2628914870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558600285d20 0x55860035ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:01.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.825+0000 7f9f4f20a640 1 -- 172.21.15.76:0/2628914870 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f4800a1c0 con 0x558600345000 2023-11-09T03:48:01.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.825+0000 7f9f4f20a640 1 --2- 172.21.15.76:0/2628914870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558600345000 0x558600361d60 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f9f480445d0 tx=0x7f9f4804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b62be2ff1f3722 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:01.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.825+0000 7f9f4d206640 1 -- 172.21.15.76:0/2628914870 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48053950 con 0x558600345000 2023-11-09T03:48:01.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.825+0000 7f9f4d206640 1 -- 172.21.15.76:0/2628914870 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f48053ad0 con 0x558600345000 2023-11-09T03:48:01.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.825+0000 7f9f4d206640 1 -- 172.21.15.76:0/2628914870 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48053e80 con 0x558600345000 2023-11-09T03:48:01.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.826+0000 7f9f51640f40 1 -- 172.21.15.76:0/2628914870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558600345000 msgr2=0x558600361d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:01.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.826+0000 7f9f51640f40 1 --2- 172.21.15.76:0/2628914870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558600345000 0x558600361d60 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f9f480445d0 tx=0x7f9f4804c000 comp rx=0 tx=0).stop 2023-11-09T03:48:01.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.826+0000 7f9f51640f40 1 -- 172.21.15.76:0/2628914870 shutdown_connections 2023-11-09T03:48:01.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.826+0000 7f9f51640f40 1 --2- 172.21.15.76:0/2628914870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558600345000 0x558600361d60 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:01.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.826+0000 7f9f51640f40 1 --2- 172.21.15.76:0/2628914870 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55860028aa60 0x558600342410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:01.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.826+0000 7f9f51640f40 1 --2- 172.21.15.76:0/2628914870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558600285d20 0x55860035ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:01.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.826+0000 7f9f51640f40 1 -- 172.21.15.76:0/2628914870 >> 172.21.15.76:0/2628914870 conn(0x5586002cf530 msgr2=0x5586002ce4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:01.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.826+0000 7f9f51640f40 1 -- 172.21.15.76:0/2628914870 shutdown_connections 2023-11-09T03:48:01.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.826+0000 7f9f51640f40 1 -- 172.21.15.76:0/2628914870 wait complete. 2023-11-09T03:48:01.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.826+0000 7f9f51640f40 1 Processor -- start 2023-11-09T03:48:01.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.826+0000 7f9f51640f40 1 -- start start 2023-11-09T03:48:01.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.827+0000 7f9f51640f40 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x558600285d20 0x5586002ce070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:01.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.827+0000 7f9f51640f40 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55860028aa60 0x558600340330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:01.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.827+0000 7f9f51640f40 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558600345000 0x5586003546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:01.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.827+0000 7f9f51640f40 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586002ce8c0 con 0x55860028aa60 2023-11-09T03:48:01.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.827+0000 7f9f51640f40 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586002ceba0 con 0x558600345000 2023-11-09T03:48:01.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.827+0000 7f9f51640f40 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x558600351220 con 0x558600285d20 2023-11-09T03:48:01.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.827+0000 7f9f4e208640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55860028aa60 0x558600340330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:01.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.827+0000 7f9f4e208640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55860028aa60 0x558600340330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41326/0 (socket says 172.21.15.76:41326) 2023-11-09T03:48:01.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.827+0000 7f9f4e208640 1 -- 172.21.15.76:0/2223899845 learned_addr learned my addr 172.21.15.76:0/2223899845 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:01.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.827+0000 7f9f4f20a640 1 --2- 172.21.15.76:0/2223899845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558600345000 0x5586003546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:01.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.827+0000 7f9f4ea09640 1 --2- 172.21.15.76:0/2223899845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x558600285d20 0x5586002ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:01.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.827+0000 7f9f4e208640 1 -- 172.21.15.76:0/2223899845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x558600285d20 msgr2=0x5586002ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:01.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.827+0000 7f9f4e208640 1 --2- 172.21.15.76:0/2223899845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x558600285d20 0x5586002ce070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:01.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.827+0000 7f9f4e208640 1 -- 172.21.15.76:0/2223899845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558600345000 msgr2=0x5586003546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:01.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.827+0000 7f9f4e208640 1 --2- 172.21.15.76:0/2223899845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558600345000 0x5586003546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:01.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.827+0000 7f9f4e208640 1 -- 172.21.15.76:0/2223899845 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586001d05a0 con 0x55860028aa60 2023-11-09T03:48:01.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.827+0000 7f9f4f20a640 1 --2- 172.21.15.76:0/2223899845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558600345000 0x5586003546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:01.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.828+0000 7f9f4e208640 1 --2- 172.21.15.76:0/2223899845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55860028aa60 0x558600340330 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400af20 tx=0x7f9f440087a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:01.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.828+0000 7f9f37fff640 1 -- 172.21.15.76:0/2223899845 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4400b810 con 0x55860028aa60 2023-11-09T03:48:01.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.828+0000 7f9f37fff640 1 -- 172.21.15.76:0/2223899845 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4400b970 con 0x55860028aa60 2023-11-09T03:48:01.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.828+0000 7f9f37fff640 1 -- 172.21.15.76:0/2223899845 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4400bd20 con 0x55860028aa60 2023-11-09T03:48:01.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.828+0000 7f9f51640f40 1 -- 172.21.15.76:0/2223899845 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5586001f7770 con 0x55860028aa60 2023-11-09T03:48:01.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.828+0000 7f9f51640f40 1 -- 172.21.15.76:0/2223899845 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586001dd780 con 0x55860028aa60 2023-11-09T03:48:01.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.829+0000 7f9f37fff640 1 -- 172.21.15.76:0/2223899845 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9f44011070 con 0x55860028aa60 2023-11-09T03:48:01.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.829+0000 7f9f37fff640 1 --2- 172.21.15.76:0/2223899845 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9f3c069820 0x7f9f3c06bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:01.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.829+0000 7f9f37fff640 1 -- 172.21.15.76:0/2223899845 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7131+0+0 (secure 0 0 0) 0x7f9f440884c0 con 0x55860028aa60 2023-11-09T03:48:01.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.830+0000 7f9f4ea09640 1 --2- 172.21.15.76:0/2223899845 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9f3c069820 0x7f9f3c06bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:01.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.830+0000 7f9f51640f40 1 -- 172.21.15.76:0/2223899845 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- pool_op(delete snap pool 12 tid 1 name snap4 v0) v4 -- 0x558600241330 con 0x55860028aa60 2023-11-09T03:48:01.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:01.830+0000 7f9f4ea09640 1 --2- 172.21.15.76:0/2223899845 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9f3c069820 0x7f9f3c06bce0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9f400075f0 tx=0x7f9f40006800 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:02.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.441+0000 7f9f37fff640 1 -- 172.21.15.76:0/2223899845 <== mon.0 v2:172.21.15.76:3300/0 6 ==== pool_op_reply(tid 1 (0) Success v70) v1 ==== 43+0+0 (secure 0 0 0) 0x7f9f44053070 con 0x55860028aa60 2023-11-09T03:48:02.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.441+0000 7f9f37fff640 1 -- 172.21.15.76:0/2223899845 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f9f3c073f60 con 0x55860028aa60 2023-11-09T03:48:02.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.441+0000 7f9f37fff640 1 -- 172.21.15.76:0/2223899845 <== mon.0 v2:172.21.15.76:3300/0 7 ==== osd_map(70..70 src has 1..70) v4 ==== 970+0+0 (secure 0 0 0) 0x7f9f440062f0 con 0x55860028aa60 2023-11-09T03:48:02.441 INFO:teuthology.orchestra.run.smithi076.stdout:removed pool test_data_pool snap snap4 2023-11-09T03:48:02.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.442+0000 7f9f51640f40 1 -- 172.21.15.76:0/2223899845 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9f3c069820 msgr2=0x7f9f3c06bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:02.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.442+0000 7f9f51640f40 1 --2- 172.21.15.76:0/2223899845 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9f3c069820 0x7f9f3c06bce0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9f400075f0 tx=0x7f9f40006800 comp rx=0 tx=0).stop 2023-11-09T03:48:02.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.442+0000 7f9f51640f40 1 -- 172.21.15.76:0/2223899845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55860028aa60 msgr2=0x558600340330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:02.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.442+0000 7f9f51640f40 1 --2- 172.21.15.76:0/2223899845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55860028aa60 0x558600340330 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400af20 tx=0x7f9f440087a0 comp rx=0 tx=0).stop 2023-11-09T03:48:02.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.442+0000 7f9f51640f40 1 -- 172.21.15.76:0/2223899845 shutdown_connections 2023-11-09T03:48:02.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.442+0000 7f9f51640f40 1 --2- 172.21.15.76:0/2223899845 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9f3c069820 0x7f9f3c06bce0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:02.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.442+0000 7f9f51640f40 1 --2- 172.21.15.76:0/2223899845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558600345000 0x5586003546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:02.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.442+0000 7f9f51640f40 1 --2- 172.21.15.76:0/2223899845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55860028aa60 0x558600340330 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:02.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.442+0000 7f9f51640f40 1 --2- 172.21.15.76:0/2223899845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x558600285d20 0x5586002ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:02.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.442+0000 7f9f51640f40 1 -- 172.21.15.76:0/2223899845 >> 172.21.15.76:0/2223899845 conn(0x5586002cf530 msgr2=0x558600361d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:02.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.442+0000 7f9f51640f40 1 -- 172.21.15.76:0/2223899845 shutdown_connections 2023-11-09T03:48:02.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.442+0000 7f9f51640f40 1 -- 172.21.15.76:0/2223899845 wait complete. 2023-11-09T03:48:02.446 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete test_data_pool 2023-11-09T03:48:02.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.599+0000 7efdcf4d4640 1 Processor -- start 2023-11-09T03:48:02.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.599+0000 7efdcf4d4640 1 -- start start 2023-11-09T03:48:02.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.600+0000 7efdcf4d4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efdc80cd6d0 0x7efdc8161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:02.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.600+0000 7efdcf4d4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdc80cde60 con 0x7efdc80b4880 2023-11-09T03:48:02.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.600+0000 7efdcf4d4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdc8160f90 con 0x7efdc8153e60 2023-11-09T03:48:02.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.600+0000 7efdcf4d4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdc8161960 con 0x7efdc80cd6d0 2023-11-09T03:48:02.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.600+0000 7efdcda71640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7efdc80b4880 0x7efdc80d0fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59918/0 (socket says 172.21.15.76:59918) 2023-11-09T03:48:02.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.600+0000 7efdcda71640 1 -- 172.21.15.76:0/1229109478 learned_addr learned my addr 172.21.15.76:0/1229109478 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:02.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.600+0000 7efdcca6f640 1 --2- 172.21.15.76:0/1229109478 >> v2:172.21.15.113:3300/0 conn(0x7efdc80cd6d0 0x7efdc8161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:02.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.600+0000 7efdbffff640 1 -- 172.21.15.76:0/1229109478 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2861963876 0 0) 0x7efdc80cde60 con 0x7efdc80b4880 2023-11-09T03:48:02.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.600+0000 7efdbffff640 1 -- 172.21.15.76:0/1229109478 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdb4003520 con 0x7efdc80b4880 2023-11-09T03:48:02.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.600+0000 7efdbffff640 1 -- 172.21.15.76:0/1229109478 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4135649077 0 0) 0x7efdc8160f90 con 0x7efdc8153e60 2023-11-09T03:48:02.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.600+0000 7efdbffff640 1 -- 172.21.15.76:0/1229109478 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdc80cde60 con 0x7efdc8153e60 2023-11-09T03:48:02.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.600+0000 7efdbffff640 1 -- 172.21.15.76:0/1229109478 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1545890022 0 0) 0x7efdb4003520 con 0x7efdc80b4880 2023-11-09T03:48:02.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.600+0000 7efdbffff640 1 -- 172.21.15.76:0/1229109478 >> v2:172.21.15.113:3300/0 conn(0x7efdc80cd6d0 msgr2=0x7efdc8161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:02.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.600+0000 7efdbffff640 1 --2- 172.21.15.76:0/1229109478 >> v2:172.21.15.113:3300/0 conn(0x7efdc80cd6d0 0x7efdc8161420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:02.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.600+0000 7efdbffff640 1 -- 172.21.15.76:0/1229109478 >> v1:172.21.15.113:6789/0 conn(0x7efdc8153e60 legacy=0x7efdc815dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:02.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.600+0000 7efdbffff640 1 -- 172.21.15.76:0/1229109478 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc80ce090 con 0x7efdc80b4880 2023-11-09T03:48:02.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.600+0000 7efdbffff640 1 -- 172.21.15.76:0/1229109478 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7efdc00019f0 con 0x7efdc80b4880 2023-11-09T03:48:02.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.600+0000 7efdbffff640 1 -- 172.21.15.76:0/1229109478 >> v1:172.21.15.76:6789/0 conn(0x7efdc80b4880 legacy=0x7efdc80d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:02.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.600+0000 7efdbffff640 1 --2- 172.21.15.76:0/1229109478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdb4003b00 0x7efdb4003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:02.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.600+0000 7efdbffff640 1 --2- 172.21.15.76:0/1229109478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdb40047c0 0x7efdb4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:02.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.600+0000 7efdbffff640 1 --2- 172.21.15.76:0/1229109478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdb4005180 0x7efdb400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:02.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.601+0000 7efdbffff640 1 -- 172.21.15.76:0/1229109478 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc8161960 con 0x7efdb4003b00 2023-11-09T03:48:02.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.601+0000 7efdbffff640 1 -- 172.21.15.76:0/1229109478 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb400c000 con 0x7efdb40047c0 2023-11-09T03:48:02.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.601+0000 7efdbffff640 1 -- 172.21.15.76:0/1229109478 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb400c1c0 con 0x7efdb4005180 2023-11-09T03:48:02.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.601+0000 7efdcd270640 1 --2- 172.21.15.76:0/1229109478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdb4003b00 0x7efdb4003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:02.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.601+0000 7efdcca6f640 1 --2- 172.21.15.76:0/1229109478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdb40047c0 0x7efdb4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:02.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.601+0000 7efdcda71640 1 --2- 172.21.15.76:0/1229109478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdb4005180 0x7efdb400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:02.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.601+0000 7efdcd270640 1 -- 172.21.15.76:0/1229109478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdb4005180 msgr2=0x7efdb400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:02.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.601+0000 7efdcd270640 1 --2- 172.21.15.76:0/1229109478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdb4005180 0x7efdb400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:02.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.601+0000 7efdcd270640 1 -- 172.21.15.76:0/1229109478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdb40047c0 msgr2=0x7efdb4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:02.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.601+0000 7efdcd270640 1 --2- 172.21.15.76:0/1229109478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdb40047c0 0x7efdb4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:02.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.601+0000 7efdcd270640 1 -- 172.21.15.76:0/1229109478 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb4003520 con 0x7efdb4003b00 2023-11-09T03:48:02.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.601+0000 7efdcda71640 1 --2- 172.21.15.76:0/1229109478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdb4005180 0x7efdb400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:02.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.601+0000 7efdcda71640 1 -- 172.21.15.76:0/1229109478 reap_dead start 2023-11-09T03:48:02.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.602+0000 7efdcd270640 1 --2- 172.21.15.76:0/1229109478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdb4003b00 0x7efdb4003f00 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7efdc40018d0 tx=0x7efdb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e80376d6d95f39 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:02.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.602+0000 7efdbffff640 1 -- 172.21.15.76:0/1229109478 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc40052d0 con 0x7efdb4003b00 2023-11-09T03:48:02.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.602+0000 7efdbffff640 1 -- 172.21.15.76:0/1229109478 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc4005430 con 0x7efdb4003b00 2023-11-09T03:48:02.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.602+0000 7efdbffff640 1 -- 172.21.15.76:0/1229109478 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc4003810 con 0x7efdb4003b00 2023-11-09T03:48:02.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.602+0000 7efdcf4d4640 1 -- 172.21.15.76:0/1229109478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdb4003b00 msgr2=0x7efdb4003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:02.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.602+0000 7efdcf4d4640 1 --2- 172.21.15.76:0/1229109478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdb4003b00 0x7efdb4003f00 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7efdc40018d0 tx=0x7efdb40057f0 comp rx=0 tx=0).stop 2023-11-09T03:48:02.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.602+0000 7efdcf4d4640 1 -- 172.21.15.76:0/1229109478 shutdown_connections 2023-11-09T03:48:02.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.602+0000 7efdcf4d4640 1 --2- 172.21.15.76:0/1229109478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdb4003b00 0x7efdb4003f00 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:02.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.602+0000 7efdcf4d4640 1 -- 172.21.15.76:0/1229109478 >> 172.21.15.76:0/1229109478 conn(0x7efdc80cf880 msgr2=0x7efdc80cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:02.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.602+0000 7efdcf4d4640 1 -- 172.21.15.76:0/1229109478 shutdown_connections 2023-11-09T03:48:02.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.602+0000 7efdcf4d4640 1 -- 172.21.15.76:0/1229109478 wait complete. 2023-11-09T03:48:02.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.604+0000 7efdcf4d4640 1 Processor -- start 2023-11-09T03:48:02.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.604+0000 7efdcf4d4640 1 -- start start 2023-11-09T03:48:02.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.604+0000 7efdcf4d4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdb4003b00 0x7efdc81f6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:02.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.604+0000 7efdcf4d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdc8152ee0 0x7efdc80cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:02.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.604+0000 7efdcf4d4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdc80b65a0 0x7efdc80b6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:02.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.604+0000 7efdcf4d4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc8154710 con 0x7efdc8152ee0 2023-11-09T03:48:02.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.604+0000 7efdcf4d4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc80b6f60 con 0x7efdc80b65a0 2023-11-09T03:48:02.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.604+0000 7efdcf4d4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc80ccda0 con 0x7efdb4003b00 2023-11-09T03:48:02.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.604+0000 7efdcca6f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdc8152ee0 0x7efdc80cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:02.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.604+0000 7efdcca6f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdc8152ee0 0x7efdc80cbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50500/0 (socket says 172.21.15.76:50500) 2023-11-09T03:48:02.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.604+0000 7efdcca6f640 1 -- 172.21.15.76:0/3491697730 learned_addr learned my addr 172.21.15.76:0/3491697730 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:02.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.605+0000 7efdcda71640 1 --2- 172.21.15.76:0/3491697730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdc80b65a0 0x7efdc80b6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:02.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.605+0000 7efdcd270640 1 --2- 172.21.15.76:0/3491697730 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdb4003b00 0x7efdc81f6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:02.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.605+0000 7efdcca6f640 1 -- 172.21.15.76:0/3491697730 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdb4003b00 msgr2=0x7efdc81f6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:02.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.605+0000 7efdcca6f640 1 --2- 172.21.15.76:0/3491697730 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdb4003b00 0x7efdc81f6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:02.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.605+0000 7efdcca6f640 1 -- 172.21.15.76:0/3491697730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdc80b65a0 msgr2=0x7efdc80b6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:02.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.605+0000 7efdcca6f640 1 --2- 172.21.15.76:0/3491697730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdc80b65a0 0x7efdc80b6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:02.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.605+0000 7efdcca6f640 1 -- 172.21.15.76:0/3491697730 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc80cc500 con 0x7efdc8152ee0 2023-11-09T03:48:02.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.605+0000 7efdcda71640 1 --2- 172.21.15.76:0/3491697730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdc80b65a0 0x7efdc80b6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:02.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.605+0000 7efdcca6f640 1 --2- 172.21.15.76:0/3491697730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdc8152ee0 0x7efdc80cbcd0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7efdb8011a70 tx=0x7efdb8011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:02.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.605+0000 7efdbdffb640 1 -- 172.21.15.76:0/3491697730 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb8009680 con 0x7efdc8152ee0 2023-11-09T03:48:02.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.605+0000 7efdbdffb640 1 -- 172.21.15.76:0/3491697730 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb800fb50 con 0x7efdc8152ee0 2023-11-09T03:48:02.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.605+0000 7efdcf4d4640 1 -- 172.21.15.76:0/3491697730 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdc80cc7e0 con 0x7efdc8152ee0 2023-11-09T03:48:02.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.605+0000 7efdbdffb640 1 -- 172.21.15.76:0/3491697730 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb800fee0 con 0x7efdc8152ee0 2023-11-09T03:48:02.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.605+0000 7efdcf4d4640 1 -- 172.21.15.76:0/3491697730 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdc80cc210 con 0x7efdc8152ee0 2023-11-09T03:48:02.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.606+0000 7efdcf4d4640 1 -- 172.21.15.76:0/3491697730 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdc8074290 con 0x7efdc8152ee0 2023-11-09T03:48:02.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.607+0000 7efdbdffb640 1 -- 172.21.15.76:0/3491697730 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efdb8014440 con 0x7efdc8152ee0 2023-11-09T03:48:02.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.607+0000 7efdbdffb640 1 --2- 172.21.15.76:0/3491697730 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efdac06ae70 0x7efdac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:02.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.607+0000 7efdbdffb640 1 -- 172.21.15.76:0/3491697730 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7148+0+0 (secure 0 0 0) 0x7efdb808a4d0 con 0x7efdc8152ee0 2023-11-09T03:48:02.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.607+0000 7efdcd270640 1 --2- 172.21.15.76:0/3491697730 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efdac06ae70 0x7efdac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:02.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.608+0000 7efdcd270640 1 --2- 172.21.15.76:0/3491697730 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efdac06ae70 0x7efdac06d330 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efdc4034000 tx=0x7efdc4031000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:02.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.612+0000 7efdbdffb640 1 -- 172.21.15.76:0/3491697730 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efdb8054fe0 con 0x7efdc8152ee0 2023-11-09T03:48:02.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.780+0000 7efdcf4d4640 1 -- 172.21.15.76:0/3491697730 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "test_data_pool"} v 0) v1 -- 0x7efdc80d1d00 con 0x7efdc8152ee0 2023-11-09T03:48:02.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.781+0000 7efdbdffb640 1 -- 172.21.15.76:0/3491697730 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "test_data_pool"}]=-1 WARNING: this will *PERMANENTLY DESTROY* all data stored in pool test_data_pool. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. v70) v1 ==== 296+0+0 (secure 0 0 0) 0x7efdb8058600 con 0x7efdc8152ee0 2023-11-09T03:48:02.780 INFO:teuthology.orchestra.run.smithi076.stderr:Error EPERM: WARNING: this will *PERMANENTLY DESTROY* all data stored in pool test_data_pool. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. 2023-11-09T03:48:02.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:48:02.786 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete test_metadata_pool 2023-11-09T03:48:02.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.888+0000 7f732358e640 1 Processor -- start 2023-11-09T03:48:02.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.888+0000 7f732358e640 1 -- start start 2023-11-09T03:48:02.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.889+0000 7f732358e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f731c0c4ff0 0x7f731c0cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:02.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.889+0000 7f732358e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f731c0adac0 0x7f731c0adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:02.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.889+0000 7f732358e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f731c15aff0 con 0x7f731c159c00 2023-11-09T03:48:02.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.889+0000 7f732358e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f731c0ae450 con 0x7f731c0adac0 2023-11-09T03:48:02.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.889+0000 7f732358e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f731c15a3e0 con 0x7f731c0c4ff0 2023-11-09T03:48:02.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.889+0000 7f7321b2b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f731c0adac0 0x7f731c0adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:02.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.889+0000 7f732132a640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f731c159c00 0x7f731c0c48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59932/0 (socket says 172.21.15.76:59932) 2023-11-09T03:48:02.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.889+0000 7f732132a640 1 -- 172.21.15.76:0/1895920016 learned_addr learned my addr 172.21.15.76:0/1895920016 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:02.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.889+0000 7f7321b2b640 1 -- 172.21.15.76:0/1895920016 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f731c0c4ff0 msgr2=0x7f731c0cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:02.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.889+0000 7f7320b29640 1 --2- 172.21.15.76:0/1895920016 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f731c0c4ff0 0x7f731c0cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:02.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.889+0000 7f7321b2b640 1 --2- 172.21.15.76:0/1895920016 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f731c0c4ff0 0x7f731c0cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:02.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.889+0000 7f7313fff640 1 -- 172.21.15.76:0/1895920016 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3898490511 0 0) 0x7f731c15aff0 con 0x7f731c159c00 2023-11-09T03:48:02.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.889+0000 7f7321b2b640 1 -- 172.21.15.76:0/1895920016 >> v1:172.21.15.76:6789/0 conn(0x7f731c159c00 legacy=0x7f731c0c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:02.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.889+0000 7f7321b2b640 1 -- 172.21.15.76:0/1895920016 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f731c15a6c0 con 0x7f731c0adac0 2023-11-09T03:48:02.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.889+0000 7f7320b29640 1 --2- 172.21.15.76:0/1895920016 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f731c0c4ff0 0x7f731c0cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:02.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.890+0000 7f7321b2b640 1 --2- 172.21.15.76:0/1895920016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f731c0adac0 0x7f731c0adf10 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f7314009d90 tx=0x7f731402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc297c2bde8bdb0a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:02.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.890+0000 7f7313fff640 1 -- 172.21.15.76:0/1895920016 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731402cc50 con 0x7f731c0adac0 2023-11-09T03:48:02.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.890+0000 7f7313fff640 1 -- 172.21.15.76:0/1895920016 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731402cdb0 con 0x7f731c0adac0 2023-11-09T03:48:02.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.890+0000 7f7313fff640 1 -- 172.21.15.76:0/1895920016 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7314034660 con 0x7f731c0adac0 2023-11-09T03:48:02.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.891+0000 7f732358e640 1 -- 172.21.15.76:0/1895920016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f731c0adac0 msgr2=0x7f731c0adf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:02.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.891+0000 7f732358e640 1 --2- 172.21.15.76:0/1895920016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f731c0adac0 0x7f731c0adf10 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f7314009d90 tx=0x7f731402bea0 comp rx=0 tx=0).stop 2023-11-09T03:48:02.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.891+0000 7f732358e640 1 -- 172.21.15.76:0/1895920016 shutdown_connections 2023-11-09T03:48:02.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.891+0000 7f732358e640 1 --2- 172.21.15.76:0/1895920016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f731c0adac0 0x7f731c0adf10 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:02.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.891+0000 7f732358e640 1 --2- 172.21.15.76:0/1895920016 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f731c0c4ff0 0x7f731c0cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:02.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.891+0000 7f732358e640 1 -- 172.21.15.76:0/1895920016 >> 172.21.15.76:0/1895920016 conn(0x7f731c0ce950 msgr2=0x7f731c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:02.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.891+0000 7f732358e640 1 -- 172.21.15.76:0/1895920016 shutdown_connections 2023-11-09T03:48:02.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.891+0000 7f732358e640 1 -- 172.21.15.76:0/1895920016 wait complete. 2023-11-09T03:48:02.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.892+0000 7f732358e640 1 Processor -- start 2023-11-09T03:48:02.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.892+0000 7f732358e640 1 -- start start 2023-11-09T03:48:02.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.893+0000 7f732358e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f731c0adac0 0x7f731c0b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:02.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.893+0000 7f732358e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f731c0c4ff0 0x7f731c1f3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:02.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.893+0000 7f732358e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f731c159c00 0x7f731c1f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:02.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.893+0000 7f732358e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731c0b1970 con 0x7f731c0c4ff0 2023-11-09T03:48:02.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.893+0000 7f732358e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731c0b46a0 con 0x7f731c159c00 2023-11-09T03:48:02.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.893+0000 7f732358e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f731c0af070 con 0x7f731c0adac0 2023-11-09T03:48:02.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.893+0000 7f7320b29640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f731c0c4ff0 0x7f731c1f3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:02.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.893+0000 7f7320b29640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f731c0c4ff0 0x7f731c1f3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50512/0 (socket says 172.21.15.76:50512) 2023-11-09T03:48:02.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.893+0000 7f7320b29640 1 -- 172.21.15.76:0/550733568 learned_addr learned my addr 172.21.15.76:0/550733568 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:02.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.893+0000 7f7320b29640 1 -- 172.21.15.76:0/550733568 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f731c0adac0 msgr2=0x7f731c0b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:02.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.893+0000 7f732132a640 1 --2- 172.21.15.76:0/550733568 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f731c0adac0 0x7f731c0b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:02.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.893+0000 7f7320b29640 1 --2- 172.21.15.76:0/550733568 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f731c0adac0 0x7f731c0b0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:02.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.893+0000 7f7320b29640 1 -- 172.21.15.76:0/550733568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f731c159c00 msgr2=0x7f731c1f40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:02.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.893+0000 7f7320b29640 1 --2- 172.21.15.76:0/550733568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f731c159c00 0x7f731c1f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:02.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.893+0000 7f7320b29640 1 -- 172.21.15.76:0/550733568 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7314009a70 con 0x7f731c0c4ff0 2023-11-09T03:48:02.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.893+0000 7f7320b29640 1 --2- 172.21.15.76:0/550733568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f731c0c4ff0 0x7f731c1f3ba0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f730c00c7f0 tx=0x7f730c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:02.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.893+0000 7f7311ffb640 1 -- 172.21.15.76:0/550733568 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730c00cea0 con 0x7f731c0c4ff0 2023-11-09T03:48:02.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.893+0000 7f7311ffb640 1 -- 172.21.15.76:0/550733568 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730c004230 con 0x7f731c0c4ff0 2023-11-09T03:48:02.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.893+0000 7f732358e640 1 -- 172.21.15.76:0/550733568 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f731c0af350 con 0x7f731c0c4ff0 2023-11-09T03:48:02.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.893+0000 7f7311ffb640 1 -- 172.21.15.76:0/550733568 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730c007830 con 0x7f731c0c4ff0 2023-11-09T03:48:02.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.894+0000 7f732358e640 1 -- 172.21.15.76:0/550733568 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f731c0af910 con 0x7f731c0c4ff0 2023-11-09T03:48:02.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.895+0000 7f732358e640 1 -- 172.21.15.76:0/550733568 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f731c074290 con 0x7f731c0c4ff0 2023-11-09T03:48:02.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.895+0000 7f7311ffb640 1 -- 172.21.15.76:0/550733568 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f730c007990 con 0x7f731c0c4ff0 2023-11-09T03:48:02.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.895+0000 7f7311ffb640 1 --2- 172.21.15.76:0/550733568 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f730006ae70 0x7f730006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:02.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.895+0000 7f7311ffb640 1 -- 172.21.15.76:0/550733568 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7148+0+0 (secure 0 0 0) 0x7f730c089ab0 con 0x7f731c0c4ff0 2023-11-09T03:48:02.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.896+0000 7f732132a640 1 --2- 172.21.15.76:0/550733568 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f730006ae70 0x7f730006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:02.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.896+0000 7f732132a640 1 --2- 172.21.15.76:0/550733568 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f730006ae70 0x7f730006d330 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7318000f60 tx=0x7f7318009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:02.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:02.899+0000 7f7311ffb640 1 -- 172.21.15.76:0/550733568 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f730c054650 con 0x7f731c0c4ff0 2023-11-09T03:48:03.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.065+0000 7f732358e640 1 -- 172.21.15.76:0/550733568 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "test_metadata_pool"} v 0) v1 -- 0x7f731c0c9990 con 0x7f731c0c4ff0 2023-11-09T03:48:03.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.065+0000 7f7311ffb640 1 -- 172.21.15.76:0/550733568 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "test_metadata_pool"}]=-1 WARNING: this will *PERMANENTLY DESTROY* all data stored in pool test_metadata_pool. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. v70) v1 ==== 304+0+0 (secure 0 0 0) 0x7f730c087e60 con 0x7f731c0c4ff0 2023-11-09T03:48:03.065 INFO:teuthology.orchestra.run.smithi076.stderr:Error EPERM: WARNING: this will *PERMANENTLY DESTROY* all data stored in pool test_metadata_pool. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. 2023-11-09T03:48:03.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:48:03.070 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-09T03:48:03.070 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-09T03:48:03.120 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-09T03:48:03.166 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-09T03:48:03.167 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:48:03.167 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:48:03.182 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:48:03.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.322+0000 7ff00e264640 1 Processor -- start 2023-11-09T03:48:03.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.322+0000 7ff00e264640 1 -- start start 2023-11-09T03:48:03.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.323+0000 7ff00e264640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff00815cfd0 0x7ff0080b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:03.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.323+0000 7ff00e264640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0080714b0 con 0x7ff008070e90 2023-11-09T03:48:03.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.323+0000 7ff00e264640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff008071790 con 0x7ff0080b0910 2023-11-09T03:48:03.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.323+0000 7ff00e264640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0080ab8e0 con 0x7ff00815cfd0 2023-11-09T03:48:03.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.323+0000 7ff007fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff00815cfd0 0x7ff0080b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:03.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.323+0000 7ff007fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff00815cfd0 0x7ff0080b03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50516/0 (socket says 172.21.15.76:50516) 2023-11-09T03:48:03.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.323+0000 7ff007fff640 1 -- 172.21.15.76:0/1753018198 learned_addr learned my addr 172.21.15.76:0/1753018198 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:03.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.323+0000 7ff007fff640 1 -- 172.21.15.76:0/1753018198 >> v1:172.21.15.113:6789/0 conn(0x7ff0080b0910 legacy=0x7ff008070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:03.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.323+0000 7ff007fff640 1 -- 172.21.15.76:0/1753018198 >> v1:172.21.15.76:6789/0 conn(0x7ff008070e90 legacy=0x7ff00806ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:03.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.323+0000 7ff007fff640 1 -- 172.21.15.76:0/1753018198 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff00806c690 con 0x7ff00815cfd0 2023-11-09T03:48:03.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.323+0000 7ff006ffd640 1 -- 172.21.15.76:0/1753018198 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2198252745 0 0) 0x7ff0080714b0 con 0x7ff008070e90 2023-11-09T03:48:03.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.324+0000 7ff007fff640 1 --2- 172.21.15.76:0/1753018198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff00815cfd0 0x7ff0080b03d0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7feff8009870 tx=0x7feff802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e730498a89a001d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:03.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.324+0000 7ff006ffd640 1 -- 172.21.15.76:0/1753018198 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff802f840 con 0x7ff00815cfd0 2023-11-09T03:48:03.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.324+0000 7ff006ffd640 1 -- 172.21.15.76:0/1753018198 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff802f9a0 con 0x7ff00815cfd0 2023-11-09T03:48:03.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.324+0000 7ff006ffd640 1 -- 172.21.15.76:0/1753018198 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff802fd30 con 0x7ff00815cfd0 2023-11-09T03:48:03.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.325+0000 7ff00e264640 1 -- 172.21.15.76:0/1753018198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff00815cfd0 msgr2=0x7ff0080b03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:03.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.325+0000 7ff00e264640 1 --2- 172.21.15.76:0/1753018198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff00815cfd0 0x7ff0080b03d0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7feff8009870 tx=0x7feff802ba20 comp rx=0 tx=0).stop 2023-11-09T03:48:03.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.325+0000 7ff00e264640 1 -- 172.21.15.76:0/1753018198 shutdown_connections 2023-11-09T03:48:03.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.325+0000 7ff00e264640 1 --2- 172.21.15.76:0/1753018198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff00815cfd0 0x7ff0080b03d0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:03.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.325+0000 7ff00e264640 1 -- 172.21.15.76:0/1753018198 >> 172.21.15.76:0/1753018198 conn(0x7ff0080ce950 msgr2=0x7ff0080ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:03.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.325+0000 7ff00e264640 1 -- 172.21.15.76:0/1753018198 shutdown_connections 2023-11-09T03:48:03.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.325+0000 7ff00e264640 1 -- 172.21.15.76:0/1753018198 wait complete. 2023-11-09T03:48:03.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.327+0000 7ff00e264640 1 Processor -- start 2023-11-09T03:48:03.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.327+0000 7ff00e264640 1 -- start start 2023-11-09T03:48:03.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.328+0000 7ff00e264640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff008070e90 0x7ff00814be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:03.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.328+0000 7ff00e264640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0080b0910 0x7ff0081f6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:03.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.328+0000 7ff00e264640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff00815cfd0 0x7ff00815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:03.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.328+0000 7ff00e264640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00814c3d0 con 0x7ff0080b0910 2023-11-09T03:48:03.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.328+0000 7ff00e264640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00814b440 con 0x7ff00815cfd0 2023-11-09T03:48:03.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.328+0000 7ff00e264640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0081f6e40 con 0x7ff008070e90 2023-11-09T03:48:03.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.328+0000 7ff0077fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0080b0910 0x7ff0081f6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:03.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.328+0000 7ff0077fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0080b0910 0x7ff0081f6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50532/0 (socket says 172.21.15.76:50532) 2023-11-09T03:48:03.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.328+0000 7ff0077fe640 1 -- 172.21.15.76:0/2103388766 learned_addr learned my addr 172.21.15.76:0/2103388766 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:03.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.328+0000 7ff00c801640 1 --2- 172.21.15.76:0/2103388766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff00815cfd0 0x7ff00815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:03.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.328+0000 7ff007fff640 1 --2- 172.21.15.76:0/2103388766 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff008070e90 0x7ff00814be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:03.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.328+0000 7ff0077fe640 1 -- 172.21.15.76:0/2103388766 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff008070e90 msgr2=0x7ff00814be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:03.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.328+0000 7ff0077fe640 1 --2- 172.21.15.76:0/2103388766 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff008070e90 0x7ff00814be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:03.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.328+0000 7ff0077fe640 1 -- 172.21.15.76:0/2103388766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff00815cfd0 msgr2=0x7ff00815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:03.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.328+0000 7ff0077fe640 1 --2- 172.21.15.76:0/2103388766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff00815cfd0 0x7ff00815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:03.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.328+0000 7ff0077fe640 1 -- 172.21.15.76:0/2103388766 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff80094d0 con 0x7ff0080b0910 2023-11-09T03:48:03.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.328+0000 7ff0077fe640 1 --2- 172.21.15.76:0/2103388766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0080b0910 0x7ff0081f6900 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7feff000c450 tx=0x7feff000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:03.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.329+0000 7ff004ff9640 1 -- 172.21.15.76:0/2103388766 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff00052d0 con 0x7ff0080b0910 2023-11-09T03:48:03.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.329+0000 7ff004ff9640 1 -- 172.21.15.76:0/2103388766 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff0005430 con 0x7ff0080b0910 2023-11-09T03:48:03.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.329+0000 7ff00e264640 1 -- 172.21.15.76:0/2103388766 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0080c9420 con 0x7ff0080b0910 2023-11-09T03:48:03.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.329+0000 7ff004ff9640 1 -- 172.21.15.76:0/2103388766 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff0012870 con 0x7ff0080b0910 2023-11-09T03:48:03.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.329+0000 7ff00e264640 1 -- 172.21.15.76:0/2103388766 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff00814b690 con 0x7ff0080b0910 2023-11-09T03:48:03.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.330+0000 7ff00e264640 1 -- 172.21.15.76:0/2103388766 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0080c9940 con 0x7ff0080b0910 2023-11-09T03:48:03.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.330+0000 7ff004ff9640 1 -- 172.21.15.76:0/2103388766 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feff00129d0 con 0x7ff0080b0910 2023-11-09T03:48:03.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.330+0000 7ff004ff9640 1 --2- 172.21.15.76:0/2103388766 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fefe806ae70 0x7fefe806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:03.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.331+0000 7ff004ff9640 1 -- 172.21.15.76:0/2103388766 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7148+0+0 (secure 0 0 0) 0x7feff00895a0 con 0x7ff0080b0910 2023-11-09T03:48:03.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.331+0000 7ff007fff640 1 --2- 172.21.15.76:0/2103388766 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fefe806ae70 0x7fefe806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:03.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.331+0000 7ff007fff640 1 --2- 172.21.15.76:0/2103388766 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fefe806ae70 0x7fefe806d330 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7feff8036000 tx=0x7feff8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:03.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.335+0000 7ff004ff9640 1 -- 172.21.15.76:0/2103388766 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feff0054160 con 0x7ff0080b0910 2023-11-09T03:48:03.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.504+0000 7ff00e264640 1 -- 172.21.15.76:0/2103388766 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff008074290 con 0x7ff0080b0910 2023-11-09T03:48:03.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.505+0000 7ff004ff9640 1 -- 172.21.15.76:0/2103388766 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+22089 (secure 0 0 0) 0x7feff0057780 con 0x7ff0080b0910 2023-11-09T03:48:03.505 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:03.505 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":70,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:48:02.436030+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:47:49.630346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-09T03:47:50.082207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-09T03:47:59.280601+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-09T03:47:59.742039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":12,"snaps":[{"begin":1,"length":1}]}],"new_removed_snaps":[{"pool":12,"snaps":[{"begin":1,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:48:03.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.507+0000 7ff00e264640 1 -- 172.21.15.76:0/2103388766 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fefe806ae70 msgr2=0x7fefe806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:03.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.507+0000 7ff00e264640 1 --2- 172.21.15.76:0/2103388766 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fefe806ae70 0x7fefe806d330 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7feff8036000 tx=0x7feff8008030 comp rx=0 tx=0).stop 2023-11-09T03:48:03.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.507+0000 7ff00e264640 1 -- 172.21.15.76:0/2103388766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0080b0910 msgr2=0x7ff0081f6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:03.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.507+0000 7ff00e264640 1 --2- 172.21.15.76:0/2103388766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0080b0910 0x7ff0081f6900 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7feff000c450 tx=0x7feff000c910 comp rx=0 tx=0).stop 2023-11-09T03:48:03.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.507+0000 7ff00e264640 1 -- 172.21.15.76:0/2103388766 shutdown_connections 2023-11-09T03:48:03.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.507+0000 7ff00e264640 1 --2- 172.21.15.76:0/2103388766 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fefe806ae70 0x7fefe806d330 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:03.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.507+0000 7ff00e264640 1 --2- 172.21.15.76:0/2103388766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff00815cfd0 0x7ff00815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:03.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.507+0000 7ff00e264640 1 --2- 172.21.15.76:0/2103388766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0080b0910 0x7ff0081f6900 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:03.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.507+0000 7ff00e264640 1 --2- 172.21.15.76:0/2103388766 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff008070e90 0x7ff00814be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:03.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.507+0000 7ff00e264640 1 -- 172.21.15.76:0/2103388766 >> 172.21.15.76:0/2103388766 conn(0x7ff0080ce950 msgr2=0x7ff0080c79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:03.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.508+0000 7ff00e264640 1 -- 172.21.15.76:0/2103388766 shutdown_connections 2023-11-09T03:48:03.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.508+0000 7ff00e264640 1 -- 172.21.15.76:0/2103388766 wait complete. 2023-11-09T03:48:03.518 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-09T03:47:00.036540+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:47:49.630346+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:47:50.082207+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_data_pool', 'create_time': '2023-11-09T03:47:59.280601+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 70, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-09T03:47:59.742039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-09T03:48:03.518 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:48:03.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.622+0000 7fd05bc7c640 1 Processor -- start 2023-11-09T03:48:03.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.622+0000 7fd05bc7c640 1 -- start start 2023-11-09T03:48:03.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.623+0000 7fd05bc7c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd054159c00 0x7fd0540cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:03.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.623+0000 7fd05bc7c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd0540ae1d0 0x7fd0540c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:03.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.623+0000 7fd05bc7c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd05415aff0 con 0x7fd0540c48e0 2023-11-09T03:48:03.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.623+0000 7fd05bc7c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0540c52a0 con 0x7fd0540ae1d0 2023-11-09T03:48:03.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.623+0000 7fd05bc7c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd05415a3e0 con 0x7fd054159c00 2023-11-09T03:48:03.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.623+0000 7fd05a219640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd0540ae1d0 0x7fd0540c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:03.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.623+0000 7fd05a219640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd0540ae1d0 0x7fd0540c4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50544/0 (socket says 172.21.15.76:50544) 2023-11-09T03:48:03.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.623+0000 7fd05a219640 1 -- 172.21.15.76:0/3323467886 learned_addr learned my addr 172.21.15.76:0/3323467886 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:03.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.623+0000 7fd05a219640 1 -- 172.21.15.76:0/3323467886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd054159c00 msgr2=0x7fd0540cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:03.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.623+0000 7fd059a18640 1 --2- 172.21.15.76:0/3323467886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd054159c00 0x7fd0540cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:03.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.623+0000 7fd05a219640 1 --2- 172.21.15.76:0/3323467886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd054159c00 0x7fd0540cc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:03.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.623+0000 7fd05a219640 1 -- 172.21.15.76:0/3323467886 >> v1:172.21.15.113:6789/0 conn(0x7fd0540c48e0 legacy=0x7fd0540adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:03.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.623+0000 7fd05a219640 1 -- 172.21.15.76:0/3323467886 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd05415a6c0 con 0x7fd0540ae1d0 2023-11-09T03:48:03.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.624+0000 7fd05a219640 1 --2- 172.21.15.76:0/3323467886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0540ae1d0 0x7fd0540c4d60 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009e50 tx=0x7fd04c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14e1ee99a6a66f server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:03.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.624+0000 7fd058a16640 1 -- 172.21.15.76:0/3323467886 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c02ca70 con 0x7fd0540ae1d0 2023-11-09T03:48:03.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.624+0000 7fd058a16640 1 -- 172.21.15.76:0/3323467886 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04c02cbd0 con 0x7fd0540ae1d0 2023-11-09T03:48:03.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.624+0000 7fd058a16640 1 -- 172.21.15.76:0/3323467886 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c0343f0 con 0x7fd0540ae1d0 2023-11-09T03:48:03.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.625+0000 7fd05bc7c640 1 -- 172.21.15.76:0/3323467886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0540ae1d0 msgr2=0x7fd0540c4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:03.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.625+0000 7fd05bc7c640 1 --2- 172.21.15.76:0/3323467886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0540ae1d0 0x7fd0540c4d60 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009e50 tx=0x7fd04c02bed0 comp rx=0 tx=0).stop 2023-11-09T03:48:03.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.625+0000 7fd05bc7c640 1 -- 172.21.15.76:0/3323467886 shutdown_connections 2023-11-09T03:48:03.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.625+0000 7fd05bc7c640 1 --2- 172.21.15.76:0/3323467886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0540ae1d0 0x7fd0540c4d60 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:03.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.625+0000 7fd05bc7c640 1 --2- 172.21.15.76:0/3323467886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd054159c00 0x7fd0540cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:03.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.625+0000 7fd05bc7c640 1 -- 172.21.15.76:0/3323467886 >> 172.21.15.76:0/3323467886 conn(0x7fd0540ce950 msgr2=0x7fd0540ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:03.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.625+0000 7fd05bc7c640 1 -- 172.21.15.76:0/3323467886 shutdown_connections 2023-11-09T03:48:03.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.625+0000 7fd05bc7c640 1 -- 172.21.15.76:0/3323467886 wait complete. 2023-11-09T03:48:03.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.626+0000 7fd05bc7c640 1 Processor -- start 2023-11-09T03:48:03.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.626+0000 7fd05bc7c640 1 -- start start 2023-11-09T03:48:03.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.627+0000 7fd05bc7c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0540ae1d0 0x7fd0541e7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:03.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.627+0000 7fd05bc7c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0540c48e0 0x7fd05413bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:03.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.627+0000 7fd05bc7c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd054159c00 0x7fd0540ac270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:03.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.627+0000 7fd05bc7c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05413c640 con 0x7fd054159c00 2023-11-09T03:48:03.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.627+0000 7fd05bc7c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0540acac0 con 0x7fd0540ae1d0 2023-11-09T03:48:03.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.627+0000 7fd05bc7c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0540acda0 con 0x7fd0540c48e0 2023-11-09T03:48:03.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.627+0000 7fd05a219640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd054159c00 0x7fd0540ac270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:03.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.627+0000 7fd05a219640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd054159c00 0x7fd0540ac270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50548/0 (socket says 172.21.15.76:50548) 2023-11-09T03:48:03.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.627+0000 7fd05a219640 1 -- 172.21.15.76:0/4273802241 learned_addr learned my addr 172.21.15.76:0/4273802241 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:03.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.627+0000 7fd059a18640 1 --2- 172.21.15.76:0/4273802241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0540ae1d0 0x7fd0541e7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:03.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.627+0000 7fd059217640 1 --2- 172.21.15.76:0/4273802241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0540c48e0 0x7fd05413bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:03.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.627+0000 7fd05a219640 1 -- 172.21.15.76:0/4273802241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0540c48e0 msgr2=0x7fd05413bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:03.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.627+0000 7fd05a219640 1 --2- 172.21.15.76:0/4273802241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0540c48e0 0x7fd05413bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:03.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.627+0000 7fd05a219640 1 -- 172.21.15.76:0/4273802241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0540ae1d0 msgr2=0x7fd0541e7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:03.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.627+0000 7fd05a219640 1 --2- 172.21.15.76:0/4273802241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0540ae1d0 0x7fd0541e7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:03.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.627+0000 7fd05a219640 1 -- 172.21.15.76:0/4273802241 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c009b30 con 0x7fd054159c00 2023-11-09T03:48:03.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.627+0000 7fd059a18640 1 --2- 172.21.15.76:0/4273802241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0540ae1d0 0x7fd0541e7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:03.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.627+0000 7fd05a219640 1 --2- 172.21.15.76:0/4273802241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd054159c00 0x7fd0540ac270 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009e50 tx=0x7fd04c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:03.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.628+0000 7fd04a7fc640 1 -- 172.21.15.76:0/4273802241 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c009510 con 0x7fd054159c00 2023-11-09T03:48:03.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.628+0000 7fd04a7fc640 1 -- 172.21.15.76:0/4273802241 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04c03e450 con 0x7fd054159c00 2023-11-09T03:48:03.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.628+0000 7fd05bc7c640 1 -- 172.21.15.76:0/4273802241 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0540ad080 con 0x7fd054159c00 2023-11-09T03:48:03.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.628+0000 7fd04a7fc640 1 -- 172.21.15.76:0/4273802241 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c03e800 con 0x7fd054159c00 2023-11-09T03:48:03.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.628+0000 7fd05bc7c640 1 -- 172.21.15.76:0/4273802241 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0540ac7b0 con 0x7fd054159c00 2023-11-09T03:48:03.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.629+0000 7fd05bc7c640 1 -- 172.21.15.76:0/4273802241 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd05413ca30 con 0x7fd054159c00 2023-11-09T03:48:03.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.629+0000 7fd04a7fc640 1 -- 172.21.15.76:0/4273802241 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd04c03e960 con 0x7fd054159c00 2023-11-09T03:48:03.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.629+0000 7fd04a7fc640 1 --2- 172.21.15.76:0/4273802241 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd03806ae70 0x7fd03806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:03.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.630+0000 7fd04a7fc640 1 -- 172.21.15.76:0/4273802241 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7148+0+0 (secure 0 0 0) 0x7fd04c0ac9e0 con 0x7fd054159c00 2023-11-09T03:48:03.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.630+0000 7fd059a18640 1 --2- 172.21.15.76:0/4273802241 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd03806ae70 0x7fd03806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:03.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.630+0000 7fd059a18640 1 --2- 172.21.15.76:0/4273802241 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd03806ae70 0x7fd03806d330 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd05000a9d0 tx=0x7fd050008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:03.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.634+0000 7fd04a7fc640 1 -- 172.21.15.76:0/4273802241 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd04c077630 con 0x7fd054159c00 2023-11-09T03:48:03.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.801+0000 7fd05bc7c640 1 -- 172.21.15.76:0/4273802241 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd054074290 con 0x7fd054159c00 2023-11-09T03:48:03.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.802+0000 7fd04a7fc640 1 -- 172.21.15.76:0/4273802241 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd04c07ac50 con 0x7fd054159c00 2023-11-09T03:48:03.801 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:48:03.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.803+0000 7fd05bc7c640 1 -- 172.21.15.76:0/4273802241 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd03806ae70 msgr2=0x7fd03806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:03.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.803+0000 7fd05bc7c640 1 --2- 172.21.15.76:0/4273802241 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd03806ae70 0x7fd03806d330 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd05000a9d0 tx=0x7fd050008040 comp rx=0 tx=0).stop 2023-11-09T03:48:03.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.803+0000 7fd05bc7c640 1 -- 172.21.15.76:0/4273802241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd054159c00 msgr2=0x7fd0540ac270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:03.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.803+0000 7fd05bc7c640 1 --2- 172.21.15.76:0/4273802241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd054159c00 0x7fd0540ac270 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009e50 tx=0x7fd04c004a40 comp rx=0 tx=0).stop 2023-11-09T03:48:03.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.803+0000 7fd05bc7c640 1 -- 172.21.15.76:0/4273802241 shutdown_connections 2023-11-09T03:48:03.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.803+0000 7fd05bc7c640 1 --2- 172.21.15.76:0/4273802241 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd03806ae70 0x7fd03806d330 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:03.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.803+0000 7fd05bc7c640 1 --2- 172.21.15.76:0/4273802241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd054159c00 0x7fd0540ac270 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:03.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.803+0000 7fd05bc7c640 1 --2- 172.21.15.76:0/4273802241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0540c48e0 0x7fd05413bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:03.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.803+0000 7fd05bc7c640 1 --2- 172.21.15.76:0/4273802241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0540ae1d0 0x7fd0541e7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:03.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.803+0000 7fd05bc7c640 1 -- 172.21.15.76:0/4273802241 >> 172.21.15.76:0/4273802241 conn(0x7fd0540ce950 msgr2=0x7fd0540c7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:03.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.803+0000 7fd05bc7c640 1 -- 172.21.15.76:0/4273802241 shutdown_connections 2023-11-09T03:48:03.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.803+0000 7fd05bc7c640 1 -- 172.21.15.76:0/4273802241 wait complete. 2023-11-09T03:48:03.813 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-09T03:48:03.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.917+0000 7fd17ac7c640 1 Processor -- start 2023-11-09T03:48:03.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.917+0000 7fd17ac7c640 1 -- start start 2023-11-09T03:48:03.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.918+0000 7fd17ac7c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd174153850 0x7fd17415a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:03.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.918+0000 7fd17ac7c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd1740cc520 0x7fd174070fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:03.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.918+0000 7fd17ac7c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1740c3860 con 0x7fd1740cc9d0 2023-11-09T03:48:03.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.918+0000 7fd17ac7c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd174070e20 con 0x7fd1740cc520 2023-11-09T03:48:03.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.918+0000 7fd17ac7c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1740c9250 con 0x7fd174153850 2023-11-09T03:48:03.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.918+0000 7fd173fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd1740cc520 0x7fd174070fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:03.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.918+0000 7fd179219640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd1740cc9d0 0x7fd174070710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59954/0 (socket says 172.21.15.76:59954) 2023-11-09T03:48:03.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.918+0000 7fd173fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd1740cc520 0x7fd174070fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50558/0 (socket says 172.21.15.76:50558) 2023-11-09T03:48:03.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.918+0000 7fd179219640 1 -- 172.21.15.76:0/4287079607 learned_addr learned my addr 172.21.15.76:0/4287079607 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:03.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.918+0000 7fd178a18640 1 --2- 172.21.15.76:0/4287079607 >> v2:172.21.15.113:3300/0 conn(0x7fd174153850 0x7fd17415a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:03.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.918+0000 7fd173fff640 1 -- 172.21.15.76:0/4287079607 >> v2:172.21.15.113:3300/0 conn(0x7fd174153850 msgr2=0x7fd17415a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:03.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.918+0000 7fd173fff640 1 --2- 172.21.15.76:0/4287079607 >> v2:172.21.15.113:3300/0 conn(0x7fd174153850 0x7fd17415a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:03.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.918+0000 7fd173fff640 1 -- 172.21.15.76:0/4287079607 >> v1:172.21.15.76:6789/0 conn(0x7fd1740cc9d0 legacy=0x7fd174070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:03.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.918+0000 7fd173fff640 1 -- 172.21.15.76:0/4287079607 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd174153c30 con 0x7fd1740cc520 2023-11-09T03:48:03.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.919+0000 7fd173fff640 1 --2- 172.21.15.76:0/4287079607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1740cc520 0x7fd174070fa0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fd1740c5710 tx=0x7fd16802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fec07e3e2465314 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:03.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.919+0000 7fd1737fe640 1 -- 172.21.15.76:0/4287079607 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168031830 con 0x7fd1740cc520 2023-11-09T03:48:03.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.919+0000 7fd1737fe640 1 -- 172.21.15.76:0/4287079607 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd168031990 con 0x7fd1740cc520 2023-11-09T03:48:03.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.919+0000 7fd1737fe640 1 -- 172.21.15.76:0/4287079607 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168031d20 con 0x7fd1740cc520 2023-11-09T03:48:03.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.920+0000 7fd17ac7c640 1 -- 172.21.15.76:0/4287079607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1740cc520 msgr2=0x7fd174070fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:03.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.920+0000 7fd17ac7c640 1 --2- 172.21.15.76:0/4287079607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1740cc520 0x7fd174070fa0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fd1740c5710 tx=0x7fd16802b8d0 comp rx=0 tx=0).stop 2023-11-09T03:48:03.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.920+0000 7fd17ac7c640 1 -- 172.21.15.76:0/4287079607 shutdown_connections 2023-11-09T03:48:03.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.920+0000 7fd17ac7c640 1 --2- 172.21.15.76:0/4287079607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1740cc520 0x7fd174070fa0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:03.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.920+0000 7fd17ac7c640 1 --2- 172.21.15.76:0/4287079607 >> v2:172.21.15.113:3300/0 conn(0x7fd174153850 0x7fd17415a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:03.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.920+0000 7fd17ac7c640 1 -- 172.21.15.76:0/4287079607 >> 172.21.15.76:0/4287079607 conn(0x7fd1740ce950 msgr2=0x7fd1740ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:03.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.920+0000 7fd17ac7c640 1 -- 172.21.15.76:0/4287079607 shutdown_connections 2023-11-09T03:48:03.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.920+0000 7fd17ac7c640 1 -- 172.21.15.76:0/4287079607 wait complete. 2023-11-09T03:48:03.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.921+0000 7fd17ac7c640 1 Processor -- start 2023-11-09T03:48:03.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.921+0000 7fd17ac7c640 1 -- start start 2023-11-09T03:48:03.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.922+0000 7fd17ac7c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1740cc520 0x7fd174149d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:03.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.922+0000 7fd17ac7c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1740cc9d0 0x7fd17414d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:03.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.922+0000 7fd17ac7c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd174153850 0x7fd1740b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:03.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.922+0000 7fd17ac7c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1740c7070 con 0x7fd1740cc9d0 2023-11-09T03:48:03.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.922+0000 7fd17ac7c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1740c8820 con 0x7fd1740cc520 2023-11-09T03:48:03.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.922+0000 7fd17ac7c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17414de50 con 0x7fd174153850 2023-11-09T03:48:03.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.922+0000 7fd173fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1740cc9d0 0x7fd17414d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:03.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.922+0000 7fd173fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1740cc9d0 0x7fd17414d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50570/0 (socket says 172.21.15.76:50570) 2023-11-09T03:48:03.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.922+0000 7fd173fff640 1 -- 172.21.15.76:0/590073172 learned_addr learned my addr 172.21.15.76:0/590073172 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:03.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.922+0000 7fd178a18640 1 --2- 172.21.15.76:0/590073172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1740cc520 0x7fd174149d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:03.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.922+0000 7fd173fff640 1 -- 172.21.15.76:0/590073172 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd174153850 msgr2=0x7fd1740b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:03.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.922+0000 7fd173fff640 1 --2- 172.21.15.76:0/590073172 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd174153850 0x7fd1740b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:03.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.922+0000 7fd173fff640 1 -- 172.21.15.76:0/590073172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1740cc520 msgr2=0x7fd174149d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:03.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.922+0000 7fd173fff640 1 --2- 172.21.15.76:0/590073172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1740cc520 0x7fd174149d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:03.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.922+0000 7fd173fff640 1 -- 172.21.15.76:0/590073172 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd168009400 con 0x7fd1740cc9d0 2023-11-09T03:48:03.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.923+0000 7fd173fff640 1 --2- 172.21.15.76:0/590073172 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1740cc9d0 0x7fd17414d7e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fd168009890 tx=0x7fd1680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:03.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.923+0000 7fd1717fa640 1 -- 172.21.15.76:0/590073172 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16803ba70 con 0x7fd1740cc9d0 2023-11-09T03:48:03.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.923+0000 7fd1717fa640 1 -- 172.21.15.76:0/590073172 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16803bbf0 con 0x7fd1740cc9d0 2023-11-09T03:48:03.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.923+0000 7fd1717fa640 1 -- 172.21.15.76:0/590073172 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168045400 con 0x7fd1740cc9d0 2023-11-09T03:48:03.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.923+0000 7fd17ac7c640 1 -- 172.21.15.76:0/590073172 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1740c6a20 con 0x7fd1740cc9d0 2023-11-09T03:48:03.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.923+0000 7fd17ac7c640 1 -- 172.21.15.76:0/590073172 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd17414e130 con 0x7fd1740cc9d0 2023-11-09T03:48:03.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.924+0000 7fd17ac7c640 1 -- 172.21.15.76:0/590073172 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1740c6880 con 0x7fd1740cc9d0 2023-11-09T03:48:03.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.924+0000 7fd1717fa640 1 -- 172.21.15.76:0/590073172 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd16804e440 con 0x7fd1740cc9d0 2023-11-09T03:48:03.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.925+0000 7fd1717fa640 1 --2- 172.21.15.76:0/590073172 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd16006ae70 0x7fd16006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:03.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.925+0000 7fd1717fa640 1 -- 172.21.15.76:0/590073172 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7148+0+0 (secure 0 0 0) 0x7fd1680abe50 con 0x7fd1740cc9d0 2023-11-09T03:48:03.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.925+0000 7fd178a18640 1 --2- 172.21.15.76:0/590073172 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd16006ae70 0x7fd16006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:03.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.925+0000 7fd178a18640 1 --2- 172.21.15.76:0/590073172 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd16006ae70 0x7fd16006d330 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd16400b7c0 tx=0x7fd164008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:03.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:03.929+0000 7fd1717fa640 1 -- 172.21.15.76:0/590073172 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd1740c6880 con 0x7fd1740cc9d0 2023-11-09T03:48:04.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.094+0000 7fd17ac7c640 1 -- 172.21.15.76:0/590073172 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7fd1740c6880 con 0x7fd1740cc9d0 2023-11-09T03:48:04.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.094+0000 7fd1717fa640 1 -- 172.21.15.76:0/590073172 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v70) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd1740c6880 con 0x7fd1740cc9d0 2023-11-09T03:48:04.094 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:48:04.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.096+0000 7fd17ac7c640 1 -- 172.21.15.76:0/590073172 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd16006ae70 msgr2=0x7fd16006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:04.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.096+0000 7fd17ac7c640 1 --2- 172.21.15.76:0/590073172 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd16006ae70 0x7fd16006d330 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd16400b7c0 tx=0x7fd164008040 comp rx=0 tx=0).stop 2023-11-09T03:48:04.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.096+0000 7fd17ac7c640 1 -- 172.21.15.76:0/590073172 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1740cc9d0 msgr2=0x7fd17414d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:04.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.096+0000 7fd17ac7c640 1 --2- 172.21.15.76:0/590073172 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1740cc9d0 0x7fd17414d7e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fd168009890 tx=0x7fd1680046c0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.096+0000 7fd17ac7c640 1 -- 172.21.15.76:0/590073172 shutdown_connections 2023-11-09T03:48:04.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.096+0000 7fd17ac7c640 1 --2- 172.21.15.76:0/590073172 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd16006ae70 0x7fd16006d330 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.096+0000 7fd17ac7c640 1 --2- 172.21.15.76:0/590073172 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd174153850 0x7fd1740b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.096+0000 7fd17ac7c640 1 --2- 172.21.15.76:0/590073172 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1740cc9d0 0x7fd17414d7e0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.096+0000 7fd17ac7c640 1 --2- 172.21.15.76:0/590073172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1740cc520 0x7fd174149d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.096+0000 7fd17ac7c640 1 -- 172.21.15.76:0/590073172 >> 172.21.15.76:0/590073172 conn(0x7fd1740ce950 msgr2=0x7fd17406d730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:04.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.096+0000 7fd17ac7c640 1 -- 172.21.15.76:0/590073172 shutdown_connections 2023-11-09T03:48:04.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.096+0000 7fd17ac7c640 1 -- 172.21.15.76:0/590073172 wait complete. 2023-11-09T03:48:04.106 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:48:04.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.209+0000 7fae8fb39640 1 Processor -- start 2023-11-09T03:48:04.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.210+0000 7fae8fb39640 1 -- start start 2023-11-09T03:48:04.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.210+0000 7fae8fb39640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fae88159c00 0x7fae880cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:04.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.210+0000 7fae8fb39640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fae880c48e0 0x7fae880c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:04.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.210+0000 7fae8fb39640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae880adac0 0x7fae880adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:04.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.211+0000 7fae8fb39640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae880c52d0 con 0x7fae88159c00 2023-11-09T03:48:04.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.211+0000 7fae8fb39640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae8815aff0 con 0x7fae880c48e0 2023-11-09T03:48:04.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.211+0000 7fae8fb39640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae8815a3e0 con 0x7fae880adac0 2023-11-09T03:48:04.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.211+0000 7fae8d8d5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fae88159c00 0x7fae880cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:04.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.211+0000 7fae8d8d5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fae88159c00 0x7fae880cc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50586/0 (socket says 172.21.15.76:50586) 2023-11-09T03:48:04.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.211+0000 7fae8d8d5640 1 -- 172.21.15.76:0/1523590565 learned_addr learned my addr 172.21.15.76:0/1523590565 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:04.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.211+0000 7fae8d0d4640 1 --2- 172.21.15.76:0/1523590565 >> v2:172.21.15.113:3300/0 conn(0x7fae880c48e0 0x7fae880c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:04.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.211+0000 7fae8e0d6640 1 --2- 172.21.15.76:0/1523590565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae880adac0 0x7fae880adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:04.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.211+0000 7fae8d8d5640 1 -- 172.21.15.76:0/1523590565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae880adac0 msgr2=0x7fae880adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:04.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.211+0000 7fae8d8d5640 1 --2- 172.21.15.76:0/1523590565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae880adac0 0x7fae880adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.211+0000 7fae8d8d5640 1 -- 172.21.15.76:0/1523590565 >> v2:172.21.15.113:3300/0 conn(0x7fae880c48e0 msgr2=0x7fae880c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:04.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.211+0000 7fae8d8d5640 1 --2- 172.21.15.76:0/1523590565 >> v2:172.21.15.113:3300/0 conn(0x7fae880c48e0 0x7fae880c4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.211+0000 7fae8d8d5640 1 -- 172.21.15.76:0/1523590565 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae8815a6c0 con 0x7fae88159c00 2023-11-09T03:48:04.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.212+0000 7fae8d8d5640 1 --2- 172.21.15.76:0/1523590565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae88159c00 0x7fae880cc890 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fae8806ebb0 tx=0x7fae7c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e47e0b6c8d0a5a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:04.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.212+0000 7fae8c8d3640 1 -- 172.21.15.76:0/1523590565 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c031870 con 0x7fae88159c00 2023-11-09T03:48:04.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.212+0000 7fae8c8d3640 1 -- 172.21.15.76:0/1523590565 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae7c0319d0 con 0x7fae88159c00 2023-11-09T03:48:04.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.212+0000 7fae8c8d3640 1 -- 172.21.15.76:0/1523590565 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c031d60 con 0x7fae88159c00 2023-11-09T03:48:04.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.212+0000 7fae8fb39640 1 -- 172.21.15.76:0/1523590565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae88159c00 msgr2=0x7fae880cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:04.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.212+0000 7fae8fb39640 1 --2- 172.21.15.76:0/1523590565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae88159c00 0x7fae880cc890 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fae8806ebb0 tx=0x7fae7c02b910 comp rx=0 tx=0).stop 2023-11-09T03:48:04.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.212+0000 7fae8fb39640 1 -- 172.21.15.76:0/1523590565 shutdown_connections 2023-11-09T03:48:04.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.212+0000 7fae8fb39640 1 --2- 172.21.15.76:0/1523590565 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae880adac0 0x7fae880adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.212+0000 7fae8fb39640 1 --2- 172.21.15.76:0/1523590565 >> v2:172.21.15.113:3300/0 conn(0x7fae880c48e0 0x7fae880c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.212+0000 7fae8fb39640 1 --2- 172.21.15.76:0/1523590565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae88159c00 0x7fae880cc890 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.212+0000 7fae8fb39640 1 -- 172.21.15.76:0/1523590565 >> 172.21.15.76:0/1523590565 conn(0x7fae880ce950 msgr2=0x7fae880ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:04.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.212+0000 7fae8fb39640 1 -- 172.21.15.76:0/1523590565 shutdown_connections 2023-11-09T03:48:04.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.212+0000 7fae8fb39640 1 -- 172.21.15.76:0/1523590565 wait complete. 2023-11-09T03:48:04.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.214+0000 7fae8fb39640 1 Processor -- start 2023-11-09T03:48:04.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.214+0000 7fae8fb39640 1 -- start start 2023-11-09T03:48:04.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.214+0000 7fae8fb39640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae880adac0 0x7fae881e35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:04.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.214+0000 7fae8fb39640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae880c48e0 0x7fae8814fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:04.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.214+0000 7fae8fb39640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae88159c00 0x7fae88156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:04.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.214+0000 7fae8fb39640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae881e2370 con 0x7fae880c48e0 2023-11-09T03:48:04.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.214+0000 7fae8fb39640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae881e3b20 con 0x7fae880adac0 2023-11-09T03:48:04.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.214+0000 7fae8fb39640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae880c6de0 con 0x7fae88159c00 2023-11-09T03:48:04.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.214+0000 7fae8d0d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae880c48e0 0x7fae8814fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:04.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.214+0000 7fae8d0d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae880c48e0 0x7fae8814fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50590/0 (socket says 172.21.15.76:50590) 2023-11-09T03:48:04.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.214+0000 7fae8d0d4640 1 -- 172.21.15.76:0/486417476 learned_addr learned my addr 172.21.15.76:0/486417476 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:04.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.215+0000 7fae8d8d5640 1 --2- 172.21.15.76:0/486417476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae880adac0 0x7fae881e35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:04.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.215+0000 7fae8e0d6640 1 --2- 172.21.15.76:0/486417476 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae88159c00 0x7fae88156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:04.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.215+0000 7fae8d0d4640 1 -- 172.21.15.76:0/486417476 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae88159c00 msgr2=0x7fae88156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:04.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.215+0000 7fae8d0d4640 1 --2- 172.21.15.76:0/486417476 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae88159c00 0x7fae88156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.215+0000 7fae8d0d4640 1 -- 172.21.15.76:0/486417476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae880adac0 msgr2=0x7fae881e35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:04.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.215+0000 7fae8d0d4640 1 --2- 172.21.15.76:0/486417476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae880adac0 0x7fae881e35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.215+0000 7fae8d0d4640 1 -- 172.21.15.76:0/486417476 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7c0094d0 con 0x7fae880c48e0 2023-11-09T03:48:04.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.215+0000 7fae8d0d4640 1 --2- 172.21.15.76:0/486417476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae880c48e0 0x7fae8814fc80 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fae8400d830 tx=0x7fae8400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:04.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.215+0000 7fae7a7fc640 1 -- 172.21.15.76:0/486417476 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8400b6c0 con 0x7fae880c48e0 2023-11-09T03:48:04.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.215+0000 7fae7a7fc640 1 -- 172.21.15.76:0/486417476 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8400b820 con 0x7fae880c48e0 2023-11-09T03:48:04.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.215+0000 7fae7a7fc640 1 -- 172.21.15.76:0/486417476 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8400bbb0 con 0x7fae880c48e0 2023-11-09T03:48:04.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.215+0000 7fae8fb39640 1 -- 172.21.15.76:0/486417476 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae880c70c0 con 0x7fae880c48e0 2023-11-09T03:48:04.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.215+0000 7fae8fb39640 1 -- 172.21.15.76:0/486417476 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae880c7680 con 0x7fae880c48e0 2023-11-09T03:48:04.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.216+0000 7fae8fb39640 1 -- 172.21.15.76:0/486417476 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae88074290 con 0x7fae880c48e0 2023-11-09T03:48:04.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.217+0000 7fae7a7fc640 1 -- 172.21.15.76:0/486417476 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fae8401e460 con 0x7fae880c48e0 2023-11-09T03:48:04.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.217+0000 7fae7a7fc640 1 --2- 172.21.15.76:0/486417476 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae6c06ae70 0x7fae6c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:04.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.217+0000 7fae7a7fc640 1 -- 172.21.15.76:0/486417476 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7148+0+0 (secure 0 0 0) 0x7fae8408a360 con 0x7fae880c48e0 2023-11-09T03:48:04.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.217+0000 7fae8d8d5640 1 --2- 172.21.15.76:0/486417476 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae6c06ae70 0x7fae6c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:04.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.218+0000 7fae8d8d5640 1 --2- 172.21.15.76:0/486417476 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae6c06ae70 0x7fae6c06d330 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fae7c036000 tx=0x7fae7c03c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:04.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.221+0000 7fae7a7fc640 1 -- 172.21.15.76:0/486417476 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fae84054e70 con 0x7fae880c48e0 2023-11-09T03:48:04.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.386+0000 7fae8fb39640 1 -- 172.21.15.76:0/486417476 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fae88070d30 con 0x7fae880c48e0 2023-11-09T03:48:04.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.387+0000 7fae7a7fc640 1 -- 172.21.15.76:0/486417476 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) v1 ==== 104+0+11 (secure 0 0 0) 0x7fae84058490 con 0x7fae880c48e0 2023-11-09T03:48:04.387 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:48:04.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.389+0000 7fae8fb39640 1 -- 172.21.15.76:0/486417476 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae6c06ae70 msgr2=0x7fae6c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:04.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.389+0000 7fae8fb39640 1 --2- 172.21.15.76:0/486417476 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae6c06ae70 0x7fae6c06d330 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fae7c036000 tx=0x7fae7c03c040 comp rx=0 tx=0).stop 2023-11-09T03:48:04.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.389+0000 7fae8fb39640 1 -- 172.21.15.76:0/486417476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae880c48e0 msgr2=0x7fae8814fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:04.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.389+0000 7fae8fb39640 1 --2- 172.21.15.76:0/486417476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae880c48e0 0x7fae8814fc80 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fae8400d830 tx=0x7fae8400dcf0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.389+0000 7fae8fb39640 1 -- 172.21.15.76:0/486417476 shutdown_connections 2023-11-09T03:48:04.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.389+0000 7fae8fb39640 1 --2- 172.21.15.76:0/486417476 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae6c06ae70 0x7fae6c06d330 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.389+0000 7fae8fb39640 1 --2- 172.21.15.76:0/486417476 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae88159c00 0x7fae88156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.389+0000 7fae8fb39640 1 --2- 172.21.15.76:0/486417476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae880c48e0 0x7fae8814fc80 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.389+0000 7fae8fb39640 1 --2- 172.21.15.76:0/486417476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae880adac0 0x7fae881e35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.389+0000 7fae8fb39640 1 -- 172.21.15.76:0/486417476 >> 172.21.15.76:0/486417476 conn(0x7fae880ce950 msgr2=0x7fae881e5900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:04.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.389+0000 7fae8fb39640 1 -- 172.21.15.76:0/486417476 shutdown_connections 2023-11-09T03:48:04.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.389+0000 7fae8fb39640 1 -- 172.21.15.76:0/486417476 wait complete. 2023-11-09T03:48:04.399 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:48:04.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.502+0000 7f5a883c9640 1 Processor -- start 2023-11-09T03:48:04.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.502+0000 7f5a883c9640 1 -- start start 2023-11-09T03:48:04.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.503+0000 7f5a883c9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5a80070ff0 0x7f5a800cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:04.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.503+0000 7f5a883c9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5a80066000 0x7f5a8006a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:04.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.503+0000 7f5a883c9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a800c9bf0 con 0x7f5a8015d1b0 2023-11-09T03:48:04.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.503+0000 7f5a883c9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a800cd890 con 0x7f5a80066000 2023-11-09T03:48:04.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.503+0000 7f5a883c9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a800cdb30 con 0x7f5a80070ff0 2023-11-09T03:48:04.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.503+0000 7f5a86966640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5a80066000 0x7f5a8006a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:04.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.503+0000 7f5a86966640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5a80066000 0x7f5a8006a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50594/0 (socket says 172.21.15.76:50594) 2023-11-09T03:48:04.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.503+0000 7f5a86966640 1 -- 172.21.15.76:0/4268615492 learned_addr learned my addr 172.21.15.76:0/4268615492 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:04.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.503+0000 7f5a86165640 1 --2- 172.21.15.76:0/4268615492 >> v2:172.21.15.113:3300/0 conn(0x7f5a80070ff0 0x7f5a800cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:04.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.503+0000 7f5a86966640 1 -- 172.21.15.76:0/4268615492 >> v2:172.21.15.113:3300/0 conn(0x7f5a80070ff0 msgr2=0x7f5a800cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:04.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.503+0000 7f5a86966640 1 --2- 172.21.15.76:0/4268615492 >> v2:172.21.15.113:3300/0 conn(0x7f5a80070ff0 0x7f5a800cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.503+0000 7f5a86966640 1 -- 172.21.15.76:0/4268615492 >> v1:172.21.15.113:6789/0 conn(0x7f5a8015d1b0 legacy=0x7f5a800713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:04.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.503+0000 7f5a86966640 1 -- 172.21.15.76:0/4268615492 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a800cde10 con 0x7f5a80066000 2023-11-09T03:48:04.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.503+0000 7f5a86165640 1 --2- 172.21.15.76:0/4268615492 >> v2:172.21.15.113:3300/0 conn(0x7f5a80070ff0 0x7f5a800cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:04.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.504+0000 7f5a86966640 1 --2- 172.21.15.76:0/4268615492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a80066000 0x7f5a8006a500 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f5a8006e7f0 tx=0x7f5a7802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef255f3b4097866 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:04.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.504+0000 7f5a85163640 1 -- 172.21.15.76:0/4268615492 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7802cb20 con 0x7f5a80066000 2023-11-09T03:48:04.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.504+0000 7f5a85163640 1 -- 172.21.15.76:0/4268615492 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a7802cc80 con 0x7f5a80066000 2023-11-09T03:48:04.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.504+0000 7f5a85163640 1 -- 172.21.15.76:0/4268615492 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a780343f0 con 0x7f5a80066000 2023-11-09T03:48:04.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.505+0000 7f5a883c9640 1 -- 172.21.15.76:0/4268615492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a80066000 msgr2=0x7f5a8006a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:04.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.505+0000 7f5a883c9640 1 --2- 172.21.15.76:0/4268615492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a80066000 0x7f5a8006a500 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f5a8006e7f0 tx=0x7f5a7802bf80 comp rx=0 tx=0).stop 2023-11-09T03:48:04.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.505+0000 7f5a883c9640 1 -- 172.21.15.76:0/4268615492 shutdown_connections 2023-11-09T03:48:04.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.505+0000 7f5a883c9640 1 --2- 172.21.15.76:0/4268615492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a80066000 0x7f5a8006a500 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.505+0000 7f5a883c9640 1 --2- 172.21.15.76:0/4268615492 >> v2:172.21.15.113:3300/0 conn(0x7f5a80070ff0 0x7f5a800cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.505+0000 7f5a883c9640 1 -- 172.21.15.76:0/4268615492 >> 172.21.15.76:0/4268615492 conn(0x7f5a800cbc90 msgr2=0x7f5a800cc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:04.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.505+0000 7f5a883c9640 1 -- 172.21.15.76:0/4268615492 shutdown_connections 2023-11-09T03:48:04.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.505+0000 7f5a883c9640 1 -- 172.21.15.76:0/4268615492 wait complete. 2023-11-09T03:48:04.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.506+0000 7f5a883c9640 1 Processor -- start 2023-11-09T03:48:04.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.506+0000 7f5a883c9640 1 -- start start 2023-11-09T03:48:04.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.507+0000 7f5a883c9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a80066000 0x7f5a801f5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:04.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.507+0000 7f5a883c9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a80070ff0 0x7f5a800b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:04.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.507+0000 7f5a883c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a8015d1b0 0x7f5a801f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:04.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.507+0000 7f5a883c9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a801f3a70 con 0x7f5a8015d1b0 2023-11-09T03:48:04.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.507+0000 7f5a883c9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a801f3740 con 0x7f5a80066000 2023-11-09T03:48:04.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.507+0000 7f5a883c9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a801f63c0 con 0x7f5a80070ff0 2023-11-09T03:48:04.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.507+0000 7f5a86966640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a8015d1b0 0x7f5a801f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:04.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.507+0000 7f5a86966640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a8015d1b0 0x7f5a801f3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50602/0 (socket says 172.21.15.76:50602) 2023-11-09T03:48:04.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.507+0000 7f5a86966640 1 -- 172.21.15.76:0/1772297740 learned_addr learned my addr 172.21.15.76:0/1772297740 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:04.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.507+0000 7f5a86966640 1 -- 172.21.15.76:0/1772297740 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a80070ff0 msgr2=0x7f5a800b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:04.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.507+0000 7f5a85964640 1 --2- 172.21.15.76:0/1772297740 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a80070ff0 0x7f5a800b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:04.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.507+0000 7f5a86165640 1 --2- 172.21.15.76:0/1772297740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a80066000 0x7f5a801f5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:04.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.507+0000 7f5a86966640 1 --2- 172.21.15.76:0/1772297740 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a80070ff0 0x7f5a800b2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.507+0000 7f5a86966640 1 -- 172.21.15.76:0/1772297740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a80066000 msgr2=0x7f5a801f5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:04.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.507+0000 7f5a86966640 1 --2- 172.21.15.76:0/1772297740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a80066000 0x7f5a801f5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.507+0000 7f5a86966640 1 -- 172.21.15.76:0/1772297740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a78009b30 con 0x7f5a8015d1b0 2023-11-09T03:48:04.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.507+0000 7f5a85964640 1 --2- 172.21.15.76:0/1772297740 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a80070ff0 0x7f5a800b2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:04.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.507+0000 7f5a86165640 1 --2- 172.21.15.76:0/1772297740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a80066000 0x7f5a801f5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:04.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.507+0000 7f5a86966640 1 --2- 172.21.15.76:0/1772297740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a8015d1b0 0x7f5a801f3f00 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f5a78036040 tx=0x7f5a78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:04.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.507+0000 7f5a76ffd640 1 -- 172.21.15.76:0/1772297740 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a78007e70 con 0x7f5a8015d1b0 2023-11-09T03:48:04.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.507+0000 7f5a76ffd640 1 -- 172.21.15.76:0/1772297740 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a78009510 con 0x7f5a8015d1b0 2023-11-09T03:48:04.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.507+0000 7f5a76ffd640 1 -- 172.21.15.76:0/1772297740 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7803e690 con 0x7f5a8015d1b0 2023-11-09T03:48:04.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.507+0000 7f5a883c9640 1 -- 172.21.15.76:0/1772297740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a801f66a0 con 0x7f5a8015d1b0 2023-11-09T03:48:04.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.508+0000 7f5a883c9640 1 -- 172.21.15.76:0/1772297740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a801f60b0 con 0x7f5a8015d1b0 2023-11-09T03:48:04.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.508+0000 7f5a883c9640 1 -- 172.21.15.76:0/1772297740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a801f3be0 con 0x7f5a8015d1b0 2023-11-09T03:48:04.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.509+0000 7f5a76ffd640 1 -- 172.21.15.76:0/1772297740 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5a7803e7f0 con 0x7f5a8015d1b0 2023-11-09T03:48:04.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.509+0000 7f5a76ffd640 1 --2- 172.21.15.76:0/1772297740 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5a6406ae70 0x7f5a6406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:04.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.509+0000 7f5a76ffd640 1 -- 172.21.15.76:0/1772297740 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7148+0+0 (secure 0 0 0) 0x7f5a780ac820 con 0x7f5a8015d1b0 2023-11-09T03:48:04.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.509+0000 7f5a86165640 1 --2- 172.21.15.76:0/1772297740 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5a6406ae70 0x7f5a6406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:04.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.510+0000 7f5a86165640 1 --2- 172.21.15.76:0/1772297740 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5a6406ae70 0x7f5a6406d330 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c006fd0 tx=0x7f5a7c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:04.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.513+0000 7f5a76ffd640 1 -- 172.21.15.76:0/1772297740 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5a78077330 con 0x7f5a8015d1b0 2023-11-09T03:48:04.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.679+0000 7f5a883c9640 1 -- 172.21.15.76:0/1772297740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5a80074290 con 0x7f5a8015d1b0 2023-11-09T03:48:04.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.679+0000 7f5a76ffd640 1 -- 172.21.15.76:0/1772297740 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5a7807a950 con 0x7f5a8015d1b0 2023-11-09T03:48:04.679 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:48:04.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.681+0000 7f5a883c9640 1 -- 172.21.15.76:0/1772297740 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5a6406ae70 msgr2=0x7f5a6406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:04.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.681+0000 7f5a883c9640 1 --2- 172.21.15.76:0/1772297740 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5a6406ae70 0x7f5a6406d330 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c006fd0 tx=0x7f5a7c008040 comp rx=0 tx=0).stop 2023-11-09T03:48:04.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.681+0000 7f5a883c9640 1 -- 172.21.15.76:0/1772297740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a8015d1b0 msgr2=0x7f5a801f3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:04.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.681+0000 7f5a883c9640 1 --2- 172.21.15.76:0/1772297740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a8015d1b0 0x7f5a801f3f00 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f5a78036040 tx=0x7f5a78004a40 comp rx=0 tx=0).stop 2023-11-09T03:48:04.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.681+0000 7f5a883c9640 1 -- 172.21.15.76:0/1772297740 shutdown_connections 2023-11-09T03:48:04.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.681+0000 7f5a883c9640 1 --2- 172.21.15.76:0/1772297740 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5a6406ae70 0x7f5a6406d330 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.681+0000 7f5a883c9640 1 --2- 172.21.15.76:0/1772297740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a8015d1b0 0x7f5a801f3f00 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.681+0000 7f5a883c9640 1 --2- 172.21.15.76:0/1772297740 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a80070ff0 0x7f5a800b2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.681+0000 7f5a883c9640 1 --2- 172.21.15.76:0/1772297740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a80066000 0x7f5a801f5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.681+0000 7f5a883c9640 1 -- 172.21.15.76:0/1772297740 >> 172.21.15.76:0/1772297740 conn(0x7f5a800cbc90 msgr2=0x7f5a800c9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:04.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.681+0000 7f5a883c9640 1 -- 172.21.15.76:0/1772297740 shutdown_connections 2023-11-09T03:48:04.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.681+0000 7f5a883c9640 1 -- 172.21.15.76:0/1772297740 wait complete. 2023-11-09T03:48:04.691 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-09T03:48:04.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.794+0000 7f190647c640 1 Processor -- start 2023-11-09T03:48:04.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.794+0000 7f190647c640 1 -- start start 2023-11-09T03:48:04.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.795+0000 7f190647c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f19000d0710 0x7f19000cd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:04.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.795+0000 7f190647c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f19000cda50 0x7f19000b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:04.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.795+0000 7f190647c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19000d0280 con 0x7f190015e430 2023-11-09T03:48:04.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.795+0000 7f190647c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19000cf670 con 0x7f19000d0710 2023-11-09T03:48:04.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.795+0000 7f190647c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19000cf950 con 0x7f19000cda50 2023-11-09T03:48:04.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.795+0000 7f18fffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f190015e430 0x7f19000cec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59956/0 (socket says 172.21.15.76:59956) 2023-11-09T03:48:04.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.795+0000 7f18fffff640 1 -- 172.21.15.76:0/4291660303 learned_addr learned my addr 172.21.15.76:0/4291660303 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:04.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.795+0000 7f18ff7fe640 1 --2- 172.21.15.76:0/4291660303 >> v2:172.21.15.76:3300/0 conn(0x7f19000d0710 0x7f19000cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:04.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.796+0000 7f1904a19640 1 --2- 172.21.15.76:0/4291660303 >> v2:172.21.15.113:3300/0 conn(0x7f19000cda50 0x7f19000b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:04.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.796+0000 7f18feffd640 1 -- 172.21.15.76:0/4291660303 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1831358049 0 0) 0x7f19000d0280 con 0x7f190015e430 2023-11-09T03:48:04.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.796+0000 7f18feffd640 1 -- 172.21.15.76:0/4291660303 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18e8003520 con 0x7f190015e430 2023-11-09T03:48:04.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.796+0000 7f18ff7fe640 1 -- 172.21.15.76:0/4291660303 >> v2:172.21.15.113:3300/0 conn(0x7f19000cda50 msgr2=0x7f19000b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:04.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.796+0000 7f18ff7fe640 1 --2- 172.21.15.76:0/4291660303 >> v2:172.21.15.113:3300/0 conn(0x7f19000cda50 0x7f19000b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.796+0000 7f18ff7fe640 1 -- 172.21.15.76:0/4291660303 >> v1:172.21.15.76:6789/0 conn(0x7f190015e430 legacy=0x7f19000cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:04.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.796+0000 7f18ff7fe640 1 -- 172.21.15.76:0/4291660303 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19000cfc30 con 0x7f19000d0710 2023-11-09T03:48:04.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.796+0000 7f1904a19640 1 --2- 172.21.15.76:0/4291660303 >> v2:172.21.15.113:3300/0 conn(0x7f19000cda50 0x7f19000b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:04.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.796+0000 7f18ff7fe640 1 --2- 172.21.15.76:0/4291660303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19000d0710 0x7f19000cd510 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f19000b6250 tx=0x7f18f402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a641cedbbc03ea6 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:04.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.797+0000 7f18feffd640 1 -- 172.21.15.76:0/4291660303 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f4031830 con 0x7f19000d0710 2023-11-09T03:48:04.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.797+0000 7f18feffd640 1 -- 172.21.15.76:0/4291660303 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f4031990 con 0x7f19000d0710 2023-11-09T03:48:04.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.797+0000 7f18feffd640 1 -- 172.21.15.76:0/4291660303 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f4031d20 con 0x7f19000d0710 2023-11-09T03:48:04.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.797+0000 7f190647c640 1 -- 172.21.15.76:0/4291660303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19000d0710 msgr2=0x7f19000cd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:04.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.797+0000 7f190647c640 1 --2- 172.21.15.76:0/4291660303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19000d0710 0x7f19000cd510 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f19000b6250 tx=0x7f18f402b8d0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.797+0000 7f190647c640 1 -- 172.21.15.76:0/4291660303 shutdown_connections 2023-11-09T03:48:04.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.797+0000 7f190647c640 1 --2- 172.21.15.76:0/4291660303 >> v2:172.21.15.113:3300/0 conn(0x7f19000cda50 0x7f19000b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.797+0000 7f190647c640 1 --2- 172.21.15.76:0/4291660303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19000d0710 0x7f19000cd510 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.797+0000 7f190647c640 1 -- 172.21.15.76:0/4291660303 >> 172.21.15.76:0/4291660303 conn(0x7f190006ea20 msgr2=0x7f190006ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:04.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.797+0000 7f190647c640 1 -- 172.21.15.76:0/4291660303 shutdown_connections 2023-11-09T03:48:04.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.797+0000 7f190647c640 1 -- 172.21.15.76:0/4291660303 wait complete. 2023-11-09T03:48:04.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.798+0000 7f190647c640 1 Processor -- start 2023-11-09T03:48:04.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.798+0000 7f190647c640 1 -- start start 2023-11-09T03:48:04.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.799+0000 7f190647c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19000cda50 0x7f19000c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:04.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.799+0000 7f190647c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19000d0710 0x7f190014ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:04.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.799+0000 7f190647c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f190015e430 0x7f19000ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:04.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.799+0000 7f190647c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19000c9b60 con 0x7f190015e430 2023-11-09T03:48:04.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.799+0000 7f190647c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19000c8ee0 con 0x7f19000cda50 2023-11-09T03:48:04.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.799+0000 7f190647c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19000c91c0 con 0x7f19000d0710 2023-11-09T03:48:04.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.799+0000 7f1904a19640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f190015e430 0x7f19000ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:04.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.799+0000 7f1904a19640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f190015e430 0x7f19000ca7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50618/0 (socket says 172.21.15.76:50618) 2023-11-09T03:48:04.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.799+0000 7f1904a19640 1 -- 172.21.15.76:0/2023963364 learned_addr learned my addr 172.21.15.76:0/2023963364 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:04.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.799+0000 7f1904a19640 1 -- 172.21.15.76:0/2023963364 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19000d0710 msgr2=0x7f190014ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:04.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.799+0000 7f18ff7fe640 1 --2- 172.21.15.76:0/2023963364 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19000d0710 0x7f190014ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:04.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.800+0000 7f18fffff640 1 --2- 172.21.15.76:0/2023963364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19000cda50 0x7f19000c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:04.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.800+0000 7f1904a19640 1 --2- 172.21.15.76:0/2023963364 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19000d0710 0x7f190014ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.800+0000 7f1904a19640 1 -- 172.21.15.76:0/2023963364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19000cda50 msgr2=0x7f19000c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:04.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.800+0000 7f1904a19640 1 --2- 172.21.15.76:0/2023963364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19000cda50 0x7f19000c9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.800+0000 7f1904a19640 1 -- 172.21.15.76:0/2023963364 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f4009400 con 0x7f190015e430 2023-11-09T03:48:04.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.800+0000 7f18ff7fe640 1 --2- 172.21.15.76:0/2023963364 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19000d0710 0x7f190014ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:04.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.800+0000 7f18fffff640 1 --2- 172.21.15.76:0/2023963364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19000cda50 0x7f19000c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:04.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.800+0000 7f1904a19640 1 --2- 172.21.15.76:0/2023963364 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f190015e430 0x7f19000ca7c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f18f800cd50 tx=0x7f18f800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:04.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.800+0000 7f18fcff9640 1 -- 172.21.15.76:0/2023963364 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f800f680 con 0x7f190015e430 2023-11-09T03:48:04.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.800+0000 7f18fcff9640 1 -- 172.21.15.76:0/2023963364 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f800f800 con 0x7f190015e430 2023-11-09T03:48:04.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.800+0000 7f190647c640 1 -- 172.21.15.76:0/2023963364 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19000c94a0 con 0x7f190015e430 2023-11-09T03:48:04.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.800+0000 7f18fcff9640 1 -- 172.21.15.76:0/2023963364 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f800fbb0 con 0x7f190015e430 2023-11-09T03:48:04.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.800+0000 7f190647c640 1 -- 172.21.15.76:0/2023963364 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190014f240 con 0x7f190015e430 2023-11-09T03:48:04.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.801+0000 7f190647c640 1 -- 172.21.15.76:0/2023963364 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19000ca4c0 con 0x7f190015e430 2023-11-09T03:48:04.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.801+0000 7f18fcff9640 1 -- 172.21.15.76:0/2023963364 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f18f800b040 con 0x7f190015e430 2023-11-09T03:48:04.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.802+0000 7f18fcff9640 1 --2- 172.21.15.76:0/2023963364 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f18ec06ae70 0x7f18ec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:04.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.802+0000 7f18fcff9640 1 -- 172.21.15.76:0/2023963364 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7148+0+0 (secure 0 0 0) 0x7f18f808a580 con 0x7f190015e430 2023-11-09T03:48:04.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.802+0000 7f18fffff640 1 --2- 172.21.15.76:0/2023963364 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f18ec06ae70 0x7f18ec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:04.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.802+0000 7f18fffff640 1 --2- 172.21.15.76:0/2023963364 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f18ec06ae70 0x7f18ec06d330 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f18f00011e0 tx=0x7f18f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:04.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.806+0000 7f18fcff9640 1 -- 172.21.15.76:0/2023963364 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f18f8055120 con 0x7f190015e430 2023-11-09T03:48:04.970 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.971+0000 7f190647c640 1 -- 172.21.15.76:0/2023963364 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7f19000c9ca0 con 0x7f190015e430 2023-11-09T03:48:04.970 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.971+0000 7f18fcff9640 1 -- 172.21.15.76:0/2023963364 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v70) v1 ==== 103+0+11 (secure 0 0 0) 0x7f19000c9ca0 con 0x7f190015e430 2023-11-09T03:48:04.971 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:48:04.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.973+0000 7f190647c640 1 -- 172.21.15.76:0/2023963364 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f18ec06ae70 msgr2=0x7f18ec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:04.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.973+0000 7f190647c640 1 --2- 172.21.15.76:0/2023963364 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f18ec06ae70 0x7f18ec06d330 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f18f00011e0 tx=0x7f18f0009040 comp rx=0 tx=0).stop 2023-11-09T03:48:04.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.973+0000 7f190647c640 1 -- 172.21.15.76:0/2023963364 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f190015e430 msgr2=0x7f19000ca7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:04.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.973+0000 7f190647c640 1 --2- 172.21.15.76:0/2023963364 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f190015e430 0x7f19000ca7c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f18f800cd50 tx=0x7f18f800a650 comp rx=0 tx=0).stop 2023-11-09T03:48:04.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.973+0000 7f190647c640 1 -- 172.21.15.76:0/2023963364 shutdown_connections 2023-11-09T03:48:04.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.973+0000 7f190647c640 1 --2- 172.21.15.76:0/2023963364 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f18ec06ae70 0x7f18ec06d330 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.973+0000 7f190647c640 1 --2- 172.21.15.76:0/2023963364 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f190015e430 0x7f19000ca7c0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.973+0000 7f190647c640 1 --2- 172.21.15.76:0/2023963364 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f19000d0710 0x7f190014ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.973+0000 7f190647c640 1 --2- 172.21.15.76:0/2023963364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19000cda50 0x7f19000c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:04.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.973+0000 7f190647c640 1 -- 172.21.15.76:0/2023963364 >> 172.21.15.76:0/2023963364 conn(0x7f190006ea20 msgr2=0x7f19000c55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:04.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.973+0000 7f190647c640 1 -- 172.21.15.76:0/2023963364 shutdown_connections 2023-11-09T03:48:04.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:04.973+0000 7f190647c640 1 -- 172.21.15.76:0/2023963364 wait complete. 2023-11-09T03:48:04.983 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-09T03:48:05.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.086+0000 7f1c963a5640 1 Processor -- start 2023-11-09T03:48:05.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.086+0000 7f1c963a5640 1 -- start start 2023-11-09T03:48:05.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.087+0000 7f1c963a5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1c900c4ff0 0x7f1c900cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:05.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.087+0000 7f1c963a5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1c900adac0 0x7f1c900adf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:05.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.087+0000 7f1c963a5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c9015aff0 con 0x7f1c90159c00 2023-11-09T03:48:05.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.087+0000 7f1c963a5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c900ae460 con 0x7f1c900c4ff0 2023-11-09T03:48:05.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.087+0000 7f1c963a5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c9015a3e0 con 0x7f1c900adac0 2023-11-09T03:48:05.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.087+0000 7f1c8f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1c900c4ff0 0x7f1c900cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:05.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.087+0000 7f1c8ffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f1c90159c00 0x7f1c900c48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59960/0 (socket says 172.21.15.76:59960) 2023-11-09T03:48:05.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.087+0000 7f1c8ffff640 1 -- 172.21.15.76:0/1999263347 learned_addr learned my addr 172.21.15.76:0/1999263347 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:05.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.087+0000 7f1c94942640 1 --2- 172.21.15.76:0/1999263347 >> v2:172.21.15.113:3300/0 conn(0x7f1c900adac0 0x7f1c900adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:05.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.087+0000 7f1c8f7fe640 1 -- 172.21.15.76:0/1999263347 >> v2:172.21.15.113:3300/0 conn(0x7f1c900adac0 msgr2=0x7f1c900adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:05.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.087+0000 7f1c8f7fe640 1 --2- 172.21.15.76:0/1999263347 >> v2:172.21.15.113:3300/0 conn(0x7f1c900adac0 0x7f1c900adf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:05.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.087+0000 7f1c8f7fe640 1 -- 172.21.15.76:0/1999263347 >> v1:172.21.15.76:6789/0 conn(0x7f1c90159c00 legacy=0x7f1c900c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:05.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.087+0000 7f1c8f7fe640 1 -- 172.21.15.76:0/1999263347 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9015a6c0 con 0x7f1c900c4ff0 2023-11-09T03:48:05.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.087+0000 7f1c8effd640 1 -- 172.21.15.76:0/1999263347 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 853511139 0 0) 0x7f1c9015aff0 con 0x7f1c90159c00 2023-11-09T03:48:05.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.087+0000 7f1c94942640 1 --2- 172.21.15.76:0/1999263347 >> v2:172.21.15.113:3300/0 conn(0x7f1c900adac0 0x7f1c900adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:05.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.088+0000 7f1c8f7fe640 1 --2- 172.21.15.76:0/1999263347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c900c4ff0 0x7f1c900cc890 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f1c900c59a0 tx=0x7f1c8402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cf1aed76a5b475 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:05.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.088+0000 7f1c8effd640 1 -- 172.21.15.76:0/1999263347 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c84031830 con 0x7f1c900c4ff0 2023-11-09T03:48:05.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.088+0000 7f1c8effd640 1 -- 172.21.15.76:0/1999263347 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c84031990 con 0x7f1c900c4ff0 2023-11-09T03:48:05.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.088+0000 7f1c8effd640 1 -- 172.21.15.76:0/1999263347 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c84031d20 con 0x7f1c900c4ff0 2023-11-09T03:48:05.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.088+0000 7f1c963a5640 1 -- 172.21.15.76:0/1999263347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c900c4ff0 msgr2=0x7f1c900cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:05.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.088+0000 7f1c963a5640 1 --2- 172.21.15.76:0/1999263347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c900c4ff0 0x7f1c900cc890 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f1c900c59a0 tx=0x7f1c8402b8d0 comp rx=0 tx=0).stop 2023-11-09T03:48:05.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.089+0000 7f1c963a5640 1 -- 172.21.15.76:0/1999263347 shutdown_connections 2023-11-09T03:48:05.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.089+0000 7f1c963a5640 1 --2- 172.21.15.76:0/1999263347 >> v2:172.21.15.113:3300/0 conn(0x7f1c900adac0 0x7f1c900adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:05.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.089+0000 7f1c963a5640 1 --2- 172.21.15.76:0/1999263347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c900c4ff0 0x7f1c900cc890 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:05.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.089+0000 7f1c963a5640 1 -- 172.21.15.76:0/1999263347 >> 172.21.15.76:0/1999263347 conn(0x7f1c900ce950 msgr2=0x7f1c900ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:05.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.089+0000 7f1c963a5640 1 -- 172.21.15.76:0/1999263347 shutdown_connections 2023-11-09T03:48:05.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.089+0000 7f1c963a5640 1 -- 172.21.15.76:0/1999263347 wait complete. 2023-11-09T03:48:05.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.090+0000 7f1c963a5640 1 Processor -- start 2023-11-09T03:48:05.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.090+0000 7f1c963a5640 1 -- start start 2023-11-09T03:48:05.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.091+0000 7f1c963a5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c900adac0 0x7f1c9014a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:05.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.091+0000 7f1c963a5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c900c4ff0 0x7f1c90157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:05.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.091+0000 7f1c963a5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c90159c00 0x7f1c90158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:05.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.091+0000 7f1c963a5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c900a74c0 con 0x7f1c90159c00 2023-11-09T03:48:05.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.091+0000 7f1c963a5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c901570f0 con 0x7f1c900adac0 2023-11-09T03:48:05.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.091+0000 7f1c963a5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c900a7710 con 0x7f1c900c4ff0 2023-11-09T03:48:05.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.091+0000 7f1c94942640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c90159c00 0x7f1c90158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:05.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.091+0000 7f1c94942640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c90159c00 0x7f1c90158100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50640/0 (socket says 172.21.15.76:50640) 2023-11-09T03:48:05.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.091+0000 7f1c94942640 1 -- 172.21.15.76:0/1995062968 learned_addr learned my addr 172.21.15.76:0/1995062968 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:05.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.091+0000 7f1c8ffff640 1 --2- 172.21.15.76:0/1995062968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c900adac0 0x7f1c9014a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:05.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.091+0000 7f1c8f7fe640 1 --2- 172.21.15.76:0/1995062968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c900c4ff0 0x7f1c90157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:05.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.091+0000 7f1c94942640 1 -- 172.21.15.76:0/1995062968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c900c4ff0 msgr2=0x7f1c90157bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:05.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.091+0000 7f1c94942640 1 --2- 172.21.15.76:0/1995062968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c900c4ff0 0x7f1c90157bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:05.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.091+0000 7f1c94942640 1 -- 172.21.15.76:0/1995062968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c900adac0 msgr2=0x7f1c9014a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:05.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.091+0000 7f1c94942640 1 --2- 172.21.15.76:0/1995062968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c900adac0 0x7f1c9014a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:05.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.091+0000 7f1c94942640 1 -- 172.21.15.76:0/1995062968 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c84009400 con 0x7f1c90159c00 2023-11-09T03:48:05.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.091+0000 7f1c8f7fe640 1 --2- 172.21.15.76:0/1995062968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c900c4ff0 0x7f1c90157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:05.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.091+0000 7f1c94942640 1 --2- 172.21.15.76:0/1995062968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c90159c00 0x7f1c90158100 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800cef0 tx=0x7f1c8800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:05.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.091+0000 7f1c8cff9640 1 -- 172.21.15.76:0/1995062968 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8800d840 con 0x7f1c90159c00 2023-11-09T03:48:05.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.091+0000 7f1c8cff9640 1 -- 172.21.15.76:0/1995062968 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c8800d9c0 con 0x7f1c90159c00 2023-11-09T03:48:05.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.091+0000 7f1c963a5640 1 -- 172.21.15.76:0/1995062968 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c900a79f0 con 0x7f1c90159c00 2023-11-09T03:48:05.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.091+0000 7f1c8cff9640 1 -- 172.21.15.76:0/1995062968 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8800dd70 con 0x7f1c90159c00 2023-11-09T03:48:05.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.091+0000 7f1c963a5640 1 -- 172.21.15.76:0/1995062968 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c9014a8e0 con 0x7f1c90159c00 2023-11-09T03:48:05.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.092+0000 7f1c963a5640 1 -- 172.21.15.76:0/1995062968 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c900a7f10 con 0x7f1c90159c00 2023-11-09T03:48:05.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.093+0000 7f1c8cff9640 1 -- 172.21.15.76:0/1995062968 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1c8801e3b0 con 0x7f1c90159c00 2023-11-09T03:48:05.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.093+0000 7f1c8cff9640 1 --2- 172.21.15.76:0/1995062968 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1c7c06ae70 0x7f1c7c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:05.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.093+0000 7f1c8cff9640 1 -- 172.21.15.76:0/1995062968 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7148+0+0 (secure 0 0 0) 0x7f1c8808a600 con 0x7f1c90159c00 2023-11-09T03:48:05.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.094+0000 7f1c8ffff640 1 --2- 172.21.15.76:0/1995062968 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1c7c06ae70 0x7f1c7c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:05.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.094+0000 7f1c8ffff640 1 --2- 172.21.15.76:0/1995062968 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1c7c06ae70 0x7f1c7c06d330 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1c80000f60 tx=0x7f1c80007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:05.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.097+0000 7f1c8cff9640 1 -- 172.21.15.76:0/1995062968 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1c900a7f10 con 0x7f1c90159c00 2023-11-09T03:48:05.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.263+0000 7f1c963a5640 1 -- 172.21.15.76:0/1995062968 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7f1c900a7f10 con 0x7f1c90159c00 2023-11-09T03:48:05.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.264+0000 7f1c8cff9640 1 -- 172.21.15.76:0/1995062968 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v70) v1 ==== 107+0+11 (secure 0 0 0) 0x7f1c900a7f10 con 0x7f1c90159c00 2023-11-09T03:48:05.264 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:48:05.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.265+0000 7f1c963a5640 1 -- 172.21.15.76:0/1995062968 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1c7c06ae70 msgr2=0x7f1c7c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:05.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.265+0000 7f1c963a5640 1 --2- 172.21.15.76:0/1995062968 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1c7c06ae70 0x7f1c7c06d330 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1c80000f60 tx=0x7f1c80007c00 comp rx=0 tx=0).stop 2023-11-09T03:48:05.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.265+0000 7f1c963a5640 1 -- 172.21.15.76:0/1995062968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c90159c00 msgr2=0x7f1c90158100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:05.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.265+0000 7f1c963a5640 1 --2- 172.21.15.76:0/1995062968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c90159c00 0x7f1c90158100 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800cef0 tx=0x7f1c8800a710 comp rx=0 tx=0).stop 2023-11-09T03:48:05.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.266+0000 7f1c963a5640 1 -- 172.21.15.76:0/1995062968 shutdown_connections 2023-11-09T03:48:05.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.266+0000 7f1c963a5640 1 --2- 172.21.15.76:0/1995062968 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1c7c06ae70 0x7f1c7c06d330 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:05.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.266+0000 7f1c963a5640 1 --2- 172.21.15.76:0/1995062968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c90159c00 0x7f1c90158100 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:05.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.266+0000 7f1c963a5640 1 --2- 172.21.15.76:0/1995062968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c900c4ff0 0x7f1c90157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:05.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.266+0000 7f1c963a5640 1 --2- 172.21.15.76:0/1995062968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c900adac0 0x7f1c9014a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:05.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.266+0000 7f1c963a5640 1 -- 172.21.15.76:0/1995062968 >> 172.21.15.76:0/1995062968 conn(0x7f1c900ce950 msgr2=0x7f1c9006d950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:05.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.266+0000 7f1c963a5640 1 -- 172.21.15.76:0/1995062968 shutdown_connections 2023-11-09T03:48:05.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.266+0000 7f1c963a5640 1 -- 172.21.15.76:0/1995062968 wait complete. 2023-11-09T03:48:05.276 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-09T03:48:05.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.377+0000 7f25389c9640 1 Processor -- start 2023-11-09T03:48:05.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.377+0000 7f25389c9640 1 -- start start 2023-11-09T03:48:05.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.378+0000 7f25389c9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f25340cd6d0 0x7f2534161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:05.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.378+0000 7f25389c9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25340b4880 0x7f25340b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:05.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.378+0000 7f25389c9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25340cde60 con 0x7f2534153e60 2023-11-09T03:48:05.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.378+0000 7f25389c9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2534160f90 con 0x7f25340cd6d0 2023-11-09T03:48:05.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.378+0000 7f25389c9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2534161960 con 0x7f25340b4880 2023-11-09T03:48:05.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.378+0000 7f2531d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f25340cd6d0 0x7f2534161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:05.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.378+0000 7f253259c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2534153e60 0x7f253415dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59974/0 (socket says 172.21.15.76:59974) 2023-11-09T03:48:05.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.378+0000 7f253259c640 1 -- 172.21.15.76:0/2619192442 learned_addr learned my addr 172.21.15.76:0/2619192442 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:05.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.379+0000 7f2532d9d640 1 --2- 172.21.15.76:0/2619192442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25340b4880 0x7f25340b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:05.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.379+0000 7f2531d9b640 1 -- 172.21.15.76:0/2619192442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25340b4880 msgr2=0x7f25340b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:05.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.379+0000 7f2531d9b640 1 --2- 172.21.15.76:0/2619192442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25340b4880 0x7f25340b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:05.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.379+0000 7f2531d9b640 1 -- 172.21.15.76:0/2619192442 >> v1:172.21.15.76:6789/0 conn(0x7f2534153e60 legacy=0x7f253415dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:05.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.379+0000 7f253159a640 1 -- 172.21.15.76:0/2619192442 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2086281962 0 0) 0x7f25340cde60 con 0x7f2534153e60 2023-11-09T03:48:05.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.379+0000 7f2531d9b640 1 -- 172.21.15.76:0/2619192442 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25340b5220 con 0x7f25340cd6d0 2023-11-09T03:48:05.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.379+0000 7f2531d9b640 1 --2- 172.21.15.76:0/2619192442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25340cd6d0 0x7f2534161420 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f25280097a0 tx=0x7f252802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a683a583806635 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:05.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.380+0000 7f253159a640 1 -- 172.21.15.76:0/2619192442 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2528031830 con 0x7f25340cd6d0 2023-11-09T03:48:05.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.380+0000 7f253159a640 1 -- 172.21.15.76:0/2619192442 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2528031990 con 0x7f25340cd6d0 2023-11-09T03:48:05.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.380+0000 7f253159a640 1 -- 172.21.15.76:0/2619192442 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2528031d20 con 0x7f25340cd6d0 2023-11-09T03:48:05.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.380+0000 7f25389c9640 1 -- 172.21.15.76:0/2619192442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25340cd6d0 msgr2=0x7f2534161420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:05.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.380+0000 7f25389c9640 1 --2- 172.21.15.76:0/2619192442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25340cd6d0 0x7f2534161420 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f25280097a0 tx=0x7f252802b8d0 comp rx=0 tx=0).stop 2023-11-09T03:48:05.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.380+0000 7f25389c9640 1 -- 172.21.15.76:0/2619192442 shutdown_connections 2023-11-09T03:48:05.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.380+0000 7f25389c9640 1 --2- 172.21.15.76:0/2619192442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25340b4880 0x7f25340b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:05.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.380+0000 7f25389c9640 1 --2- 172.21.15.76:0/2619192442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25340cd6d0 0x7f2534161420 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:05.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.380+0000 7f25389c9640 1 -- 172.21.15.76:0/2619192442 >> 172.21.15.76:0/2619192442 conn(0x7f25340cf880 msgr2=0x7f25340cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:05.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.380+0000 7f25389c9640 1 -- 172.21.15.76:0/2619192442 shutdown_connections 2023-11-09T03:48:05.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.380+0000 7f25389c9640 1 -- 172.21.15.76:0/2619192442 wait complete. 2023-11-09T03:48:05.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.382+0000 7f25389c9640 1 Processor -- start 2023-11-09T03:48:05.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.382+0000 7f25389c9640 1 -- start start 2023-11-09T03:48:05.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.382+0000 7f25389c9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25340b4880 0x7f2534155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:05.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.382+0000 7f25389c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25340cd6d0 0x7f2534159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:05.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.382+0000 7f25389c9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2534153e60 0x7f25341f4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:05.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.382+0000 7f25389c9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2534155780 con 0x7f25340cd6d0 2023-11-09T03:48:05.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.382+0000 7f25389c9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25341f3c00 con 0x7f2534153e60 2023-11-09T03:48:05.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.382+0000 7f25389c9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25341f3ee0 con 0x7f25340b4880 2023-11-09T03:48:05.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.382+0000 7f2531d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25340cd6d0 0x7f2534159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:05.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.382+0000 7f2531d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25340cd6d0 0x7f2534159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50668/0 (socket says 172.21.15.76:50668) 2023-11-09T03:48:05.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.383+0000 7f2531d9b640 1 -- 172.21.15.76:0/109301482 learned_addr learned my addr 172.21.15.76:0/109301482 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:05.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.383+0000 7f253259c640 1 --2- 172.21.15.76:0/109301482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25340b4880 0x7f2534155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:05.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.383+0000 7f2532d9d640 1 --2- 172.21.15.76:0/109301482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2534153e60 0x7f25341f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:05.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.383+0000 7f2531d9b640 1 -- 172.21.15.76:0/109301482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25340b4880 msgr2=0x7f2534155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:05.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.383+0000 7f2531d9b640 1 --2- 172.21.15.76:0/109301482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25340b4880 0x7f2534155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:05.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.383+0000 7f2531d9b640 1 -- 172.21.15.76:0/109301482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2534153e60 msgr2=0x7f25341f4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:05.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.383+0000 7f2531d9b640 1 --2- 172.21.15.76:0/109301482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2534153e60 0x7f25341f4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:05.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.383+0000 7f2531d9b640 1 -- 172.21.15.76:0/109301482 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2528009400 con 0x7f25340cd6d0 2023-11-09T03:48:05.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.383+0000 7f2531d9b640 1 --2- 172.21.15.76:0/109301482 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25340cd6d0 0x7f2534159ed0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f252802bee0 tx=0x7f25280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:05.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.383+0000 7f2522ffd640 1 -- 172.21.15.76:0/109301482 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252803a6f0 con 0x7f25340cd6d0 2023-11-09T03:48:05.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.383+0000 7f2522ffd640 1 -- 172.21.15.76:0/109301482 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252803a870 con 0x7f25340cd6d0 2023-11-09T03:48:05.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.383+0000 7f25389c9640 1 -- 172.21.15.76:0/109301482 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25341f41c0 con 0x7f25340cd6d0 2023-11-09T03:48:05.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.383+0000 7f2522ffd640 1 -- 172.21.15.76:0/109301482 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252803ac20 con 0x7f25340cd6d0 2023-11-09T03:48:05.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.383+0000 7f25389c9640 1 -- 172.21.15.76:0/109301482 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25341f4780 con 0x7f25340cd6d0 2023-11-09T03:48:05.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.384+0000 7f25389c9640 1 -- 172.21.15.76:0/109301482 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25341f49a0 con 0x7f25340cd6d0 2023-11-09T03:48:05.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.385+0000 7f2522ffd640 1 -- 172.21.15.76:0/109301482 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f252804c3b0 con 0x7f25340cd6d0 2023-11-09T03:48:05.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.385+0000 7f2522ffd640 1 --2- 172.21.15.76:0/109301482 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f251406aea0 0x7f251406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:05.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.385+0000 7f2522ffd640 1 -- 172.21.15.76:0/109301482 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7148+0+0 (secure 0 0 0) 0x7f25280ab720 con 0x7f25340cd6d0 2023-11-09T03:48:05.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.385+0000 7f253259c640 1 --2- 172.21.15.76:0/109301482 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f251406aea0 0x7f251406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:05.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.386+0000 7f253259c640 1 --2- 172.21.15.76:0/109301482 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f251406aea0 0x7f251406d360 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f25240012e0 tx=0x7f2524001630 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:05.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.389+0000 7f2522ffd640 1 -- 172.21.15.76:0/109301482 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f25280762e0 con 0x7f25340cd6d0 2023-11-09T03:48:05.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.568+0000 7f25389c9640 1 -- 172.21.15.76:0/109301482 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2534074290 con 0x7f25340cd6d0 2023-11-09T03:48:05.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.568+0000 7f2522ffd640 1 -- 172.21.15.76:0/109301482 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v70) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2528079900 con 0x7f25340cd6d0 2023-11-09T03:48:05.568 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2023-11-09T03:48:05.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.570+0000 7f25389c9640 1 -- 172.21.15.76:0/109301482 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f251406aea0 msgr2=0x7f251406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:05.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.570+0000 7f25389c9640 1 --2- 172.21.15.76:0/109301482 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f251406aea0 0x7f251406d360 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f25240012e0 tx=0x7f2524001630 comp rx=0 tx=0).stop 2023-11-09T03:48:05.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.570+0000 7f25389c9640 1 -- 172.21.15.76:0/109301482 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25340cd6d0 msgr2=0x7f2534159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:05.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.570+0000 7f25389c9640 1 --2- 172.21.15.76:0/109301482 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25340cd6d0 0x7f2534159ed0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f252802bee0 tx=0x7f25280046c0 comp rx=0 tx=0).stop 2023-11-09T03:48:05.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.570+0000 7f25389c9640 1 -- 172.21.15.76:0/109301482 shutdown_connections 2023-11-09T03:48:05.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.570+0000 7f25389c9640 1 --2- 172.21.15.76:0/109301482 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f251406aea0 0x7f251406d360 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:05.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.570+0000 7f25389c9640 1 --2- 172.21.15.76:0/109301482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2534153e60 0x7f25341f4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:05.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.570+0000 7f25389c9640 1 --2- 172.21.15.76:0/109301482 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25340cd6d0 0x7f2534159ed0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:05.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.570+0000 7f25389c9640 1 --2- 172.21.15.76:0/109301482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25340b4880 0x7f2534155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:05.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.570+0000 7f25389c9640 1 -- 172.21.15.76:0/109301482 >> 172.21.15.76:0/109301482 conn(0x7f25340cf880 msgr2=0x7f253415fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:05.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.570+0000 7f25389c9640 1 -- 172.21.15.76:0/109301482 shutdown_connections 2023-11-09T03:48:05.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.570+0000 7f25389c9640 1 -- 172.21.15.76:0/109301482 wait complete. 2023-11-09T03:48:05.580 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi076.front.sepia.ceph.com... 2023-11-09T03:48:05.580 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:48:05.581 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-09T03:48:05.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:48:05.641 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[110864]: fuse finished with error 0 and tester_r 0 2023-11-09T03:48:05.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:48:05.646 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:48:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:48:05.648 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T03:48:05.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:48:05.649 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-09T03:48:05.664 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:48:05.664 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:48:05.683 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2440 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-09T03:48:05.683 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2023-11-09T03:48:05.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:48:05.684 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-09T03:48:05.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:48:05.741 INFO:teuthology.orchestra.run.smithi113.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-09T03:48:05.742 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:48:05.742 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:48:05.757 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056381,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14761 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248868k,nr_inodes=812217,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-09T03:48:05.757 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T03:48:05.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:48:05.758 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-09T03:48:05.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:48:05.812 INFO:teuthology.orchestra.run.smithi076.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-09T03:48:05.812 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:48:05.813 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:48:05.831 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2440 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-09T03:48:05.831 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2023-11-09T03:48:05.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:48:05.832 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-09T03:48:05.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:48:05.889 INFO:teuthology.orchestra.run.smithi113.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-09T03:48:05.889 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:48:05.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.990+0000 7f76e5e90640 1 Processor -- start 2023-11-09T03:48:05.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.990+0000 7f76e5e90640 1 -- start start 2023-11-09T03:48:05.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.991+0000 7f76e5e90640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f76e015ad80 0x7f76e006c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:05.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.991+0000 7f76e5e90640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76e006cd00 con 0x7f76e00b42c0 2023-11-09T03:48:05.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.991+0000 7f76e5e90640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76e006cfa0 con 0x7f76e00c9e10 2023-11-09T03:48:05.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.991+0000 7f76e5e90640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76e006d280 con 0x7f76e015ad80 2023-11-09T03:48:05.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.991+0000 7f76deffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f76e00b42c0 0x7f76e00c9700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59978/0 (socket says 172.21.15.76:59978) 2023-11-09T03:48:05.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.991+0000 7f76deffd640 1 -- 172.21.15.76:0/3532675942 learned_addr learned my addr 172.21.15.76:0/3532675942 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:05.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.991+0000 7f76de7fc640 1 -- 172.21.15.76:0/3532675942 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3972465166 0 0) 0x7f76e006cd00 con 0x7f76e00b42c0 2023-11-09T03:48:05.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.991+0000 7f76de7fc640 1 -- 172.21.15.76:0/3532675942 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76c8003520 con 0x7f76e00b42c0 2023-11-09T03:48:05.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.991+0000 7f76df7fe640 1 --2- 172.21.15.76:0/3532675942 >> v2:172.21.15.113:3300/0 conn(0x7f76e015ad80 0x7f76e006c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:05.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.991+0000 7f76de7fc640 1 -- 172.21.15.76:0/3532675942 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 256758944 0 0) 0x7f76c8003520 con 0x7f76e00b42c0 2023-11-09T03:48:05.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.991+0000 7f76de7fc640 1 -- 172.21.15.76:0/3532675942 >> v2:172.21.15.113:3300/0 conn(0x7f76e015ad80 msgr2=0x7f76e006c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:05.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.991+0000 7f76de7fc640 1 --2- 172.21.15.76:0/3532675942 >> v2:172.21.15.113:3300/0 conn(0x7f76e015ad80 0x7f76e006c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:05.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.991+0000 7f76de7fc640 1 -- 172.21.15.76:0/3532675942 >> v1:172.21.15.113:6789/0 conn(0x7f76e00c9e10 legacy=0x7f76e00b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:05.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.991+0000 7f76de7fc640 1 -- 172.21.15.76:0/3532675942 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e006d560 con 0x7f76e00b42c0 2023-11-09T03:48:05.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.991+0000 7f76de7fc640 1 -- 172.21.15.76:0/3532675942 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f76d4001670 con 0x7f76e00b42c0 2023-11-09T03:48:05.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.991+0000 7f76de7fc640 1 -- 172.21.15.76:0/3532675942 >> v1:172.21.15.76:6789/0 conn(0x7f76e00b42c0 legacy=0x7f76e00c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:05.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.991+0000 7f76de7fc640 1 --2- 172.21.15.76:0/3532675942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76c8003970 0x7f76c8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:05.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.991+0000 7f76de7fc640 1 --2- 172.21.15.76:0/3532675942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76c80047c0 0x7f76c8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:05.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.991+0000 7f76de7fc640 1 --2- 172.21.15.76:0/3532675942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76c8005180 0x7f76c800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:05.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.991+0000 7f76de7fc640 1 -- 172.21.15.76:0/3532675942 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e006d280 con 0x7f76c8003970 2023-11-09T03:48:05.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.991+0000 7f76de7fc640 1 -- 172.21.15.76:0/3532675942 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c800c030 con 0x7f76c8005180 2023-11-09T03:48:05.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.991+0000 7f76de7fc640 1 -- 172.21.15.76:0/3532675942 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c800c1f0 con 0x7f76c80047c0 2023-11-09T03:48:05.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.991+0000 7f76df7fe640 1 --2- 172.21.15.76:0/3532675942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76c8003970 0x7f76c8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:05.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.992+0000 7f76dffff640 1 --2- 172.21.15.76:0/3532675942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76c8005180 0x7f76c800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:05.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.992+0000 7f76deffd640 1 --2- 172.21.15.76:0/3532675942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76c80047c0 0x7f76c8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:05.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.992+0000 7f76df7fe640 1 -- 172.21.15.76:0/3532675942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76c80047c0 msgr2=0x7f76c8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:05.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.992+0000 7f76df7fe640 1 --2- 172.21.15.76:0/3532675942 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76c80047c0 0x7f76c8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:05.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.992+0000 7f76df7fe640 1 -- 172.21.15.76:0/3532675942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76c8005180 msgr2=0x7f76c800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:06.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.992+0000 7f76df7fe640 1 --2- 172.21.15.76:0/3532675942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76c8005180 0x7f76c800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.992+0000 7f76dffff640 1 -- 172.21.15.76:0/3532675942 reap_dead start 2023-11-09T03:48:06.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.992+0000 7f76df7fe640 1 -- 172.21.15.76:0/3532675942 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c8003520 con 0x7f76c8003970 2023-11-09T03:48:06.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.992+0000 7f76df7fe640 1 --2- 172.21.15.76:0/3532675942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76c8003970 0x7f76c8003d90 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f76d0009ac0 tx=0x7f76c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85219c14ae97ef87 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:06.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.992+0000 7f76de7fc640 1 -- 172.21.15.76:0/3532675942 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d002c6b0 con 0x7f76c8003970 2023-11-09T03:48:06.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.993+0000 7f76de7fc640 1 -- 172.21.15.76:0/3532675942 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d002c810 con 0x7f76c8003970 2023-11-09T03:48:06.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.993+0000 7f76de7fc640 1 -- 172.21.15.76:0/3532675942 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d002cba0 con 0x7f76c8003970 2023-11-09T03:48:06.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.993+0000 7f76e5e90640 1 -- 172.21.15.76:0/3532675942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76c8003970 msgr2=0x7f76c8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:06.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.993+0000 7f76e5e90640 1 --2- 172.21.15.76:0/3532675942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76c8003970 0x7f76c8003d90 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f76d0009ac0 tx=0x7f76c80057f0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.993+0000 7f76e5e90640 1 -- 172.21.15.76:0/3532675942 shutdown_connections 2023-11-09T03:48:06.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.993+0000 7f76e5e90640 1 --2- 172.21.15.76:0/3532675942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76c8003970 0x7f76c8003d90 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.993+0000 7f76e5e90640 1 -- 172.21.15.76:0/3532675942 >> 172.21.15.76:0/3532675942 conn(0x7f76e00ce950 msgr2=0x7f76e00ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:06.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.993+0000 7f76e5e90640 1 -- 172.21.15.76:0/3532675942 shutdown_connections 2023-11-09T03:48:06.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.993+0000 7f76e5e90640 1 -- 172.21.15.76:0/3532675942 wait complete. 2023-11-09T03:48:06.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.994+0000 7f76e5e90640 1 Processor -- start 2023-11-09T03:48:06.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.994+0000 7f76e5e90640 1 -- start start 2023-11-09T03:48:06.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.995+0000 7f76e5e90640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76c8003970 0x7f76e00a8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:06.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.995+0000 7f76e5e90640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76e01512f0 0x7f76e015d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:06.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.995+0000 7f76e5e90640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76e00c6640 0x7f76e015d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:06.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.995+0000 7f76e5e90640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e00a8850 con 0x7f76e00c6640 2023-11-09T03:48:06.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.995+0000 7f76e5e90640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e00c6ed0 con 0x7f76c8003970 2023-11-09T03:48:06.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.995+0000 7f76e5e90640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e00c71b0 con 0x7f76e01512f0 2023-11-09T03:48:06.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.995+0000 7f76dffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76e00c6640 0x7f76e015d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:06.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.995+0000 7f76dffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76e00c6640 0x7f76e015d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50692/0 (socket says 172.21.15.76:50692) 2023-11-09T03:48:06.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.995+0000 7f76dffff640 1 -- 172.21.15.76:0/212035126 learned_addr learned my addr 172.21.15.76:0/212035126 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:06.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.995+0000 7f76df7fe640 1 --2- 172.21.15.76:0/212035126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76c8003970 0x7f76e00a8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:06.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.995+0000 7f76dffff640 1 -- 172.21.15.76:0/212035126 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76e01512f0 msgr2=0x7f76e015d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:06.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.995+0000 7f76deffd640 1 --2- 172.21.15.76:0/212035126 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76e01512f0 0x7f76e015d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:06.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.995+0000 7f76dffff640 1 --2- 172.21.15.76:0/212035126 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76e01512f0 0x7f76e015d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.995+0000 7f76dffff640 1 -- 172.21.15.76:0/212035126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76c8003970 msgr2=0x7f76e00a8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:06.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.995+0000 7f76dffff640 1 --2- 172.21.15.76:0/212035126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76c8003970 0x7f76e00a8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.995+0000 7f76dffff640 1 -- 172.21.15.76:0/212035126 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e00c7490 con 0x7f76e00c6640 2023-11-09T03:48:06.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.995+0000 7f76df7fe640 1 --2- 172.21.15.76:0/212035126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76c8003970 0x7f76e00a8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:06.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.996+0000 7f76dffff640 1 --2- 172.21.15.76:0/212035126 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76e00c6640 0x7f76e015d870 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f76d8007fd0 tx=0x7f76d800e980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:06.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.996+0000 7f76c7fff640 1 -- 172.21.15.76:0/212035126 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8006d60 con 0x7f76e00c6640 2023-11-09T03:48:06.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.996+0000 7f76c7fff640 1 -- 172.21.15.76:0/212035126 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d8003970 con 0x7f76e00c6640 2023-11-09T03:48:06.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.996+0000 7f76e5e90640 1 -- 172.21.15.76:0/212035126 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e00c7770 con 0x7f76e00c6640 2023-11-09T03:48:06.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.996+0000 7f76c7fff640 1 -- 172.21.15.76:0/212035126 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8003d20 con 0x7f76e00c6640 2023-11-09T03:48:06.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.996+0000 7f76e5e90640 1 -- 172.21.15.76:0/212035126 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e0154d60 con 0x7f76e00c6640 2023-11-09T03:48:06.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.997+0000 7f76e5e90640 1 -- 172.21.15.76:0/212035126 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e00a7d10 con 0x7f76e00c6640 2023-11-09T03:48:06.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.997+0000 7f76c7fff640 1 -- 172.21.15.76:0/212035126 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f76d8005070 con 0x7f76e00c6640 2023-11-09T03:48:06.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.997+0000 7f76c7fff640 1 --2- 172.21.15.76:0/212035126 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f76cc06aea0 0x7f76cc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:06.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.998+0000 7f76c7fff640 1 -- 172.21.15.76:0/212035126 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7148+0+0 (secure 0 0 0) 0x7f76d80892e0 con 0x7f76e00c6640 2023-11-09T03:48:06.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.998+0000 7f76df7fe640 1 --2- 172.21.15.76:0/212035126 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f76cc06aea0 0x7f76cc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:06.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:05.998+0000 7f76df7fe640 1 --2- 172.21.15.76:0/212035126 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f76cc06aea0 0x7f76cc06d360 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f76d0036000 tx=0x7f76d0030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:06.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.002+0000 7f76c7fff640 1 -- 172.21.15.76:0/212035126 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f76d8053df0 con 0x7f76e00c6640 2023-11-09T03:48:06.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.206+0000 7f76e5e90640 1 -- 172.21.15.76:0/212035126 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76e0074290 con 0x7f76e00c6640 2023-11-09T03:48:06.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.207+0000 7f76c7fff640 1 -- 172.21.15.76:0/212035126 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f76d8057410 con 0x7f76e00c6640 2023-11-09T03:48:06.207 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:06.207 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2800733670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2800733670},{"type":"v1","addr":"172.21.15.113:6836","nonce":2800733670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":5791,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1301905851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1301905851},{"type":"v1","addr":"172.21.15.113:6837","nonce":1301905851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":6498,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/968739949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":968739949},{"type":"v1","addr":"172.21.15.76:6833","nonce":968739949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:47:51.114132+0000","modified":"2023-11-09T03:47:53.114512+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6501},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6501":{"gid":6501,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/1576689316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1576689316},{"type":"v1","addr":"172.21.15.76:6835","nonce":1576689316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-09T03:48:06.208 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 29 2023-11-09T03:48:06.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.209+0000 7f76e5e90640 1 -- 172.21.15.76:0/212035126 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f76cc06aea0 msgr2=0x7f76cc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:06.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.209+0000 7f76e5e90640 1 --2- 172.21.15.76:0/212035126 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f76cc06aea0 0x7f76cc06d360 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f76d0036000 tx=0x7f76d0030000 comp rx=0 tx=0).stop 2023-11-09T03:48:06.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.209+0000 7f76e5e90640 1 -- 172.21.15.76:0/212035126 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76e00c6640 msgr2=0x7f76e015d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:06.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.209+0000 7f76e5e90640 1 --2- 172.21.15.76:0/212035126 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76e00c6640 0x7f76e015d870 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f76d8007fd0 tx=0x7f76d800e980 comp rx=0 tx=0).stop 2023-11-09T03:48:06.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.209+0000 7f76e5e90640 1 -- 172.21.15.76:0/212035126 shutdown_connections 2023-11-09T03:48:06.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.209+0000 7f76e5e90640 1 --2- 172.21.15.76:0/212035126 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f76cc06aea0 0x7f76cc06d360 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.209+0000 7f76e5e90640 1 --2- 172.21.15.76:0/212035126 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76e00c6640 0x7f76e015d870 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.209+0000 7f76e5e90640 1 --2- 172.21.15.76:0/212035126 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76e01512f0 0x7f76e015d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.209+0000 7f76e5e90640 1 --2- 172.21.15.76:0/212035126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76c8003970 0x7f76e00a8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.209+0000 7f76e5e90640 1 -- 172.21.15.76:0/212035126 >> 172.21.15.76:0/212035126 conn(0x7f76e00ce950 msgr2=0x7f76e00b0780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:06.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.209+0000 7f76e5e90640 1 -- 172.21.15.76:0/212035126 shutdown_connections 2023-11-09T03:48:06.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.209+0000 7f76e5e90640 1 -- 172.21.15.76:0/212035126 wait complete. 2023-11-09T03:48:06.220 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:48:06.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.320+0000 7f7569d3c640 1 Processor -- start 2023-11-09T03:48:06.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.320+0000 7f7569d3c640 1 -- start start 2023-11-09T03:48:06.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.321+0000 7f7569d3c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f75640b0ae0 0x7f75640ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:06.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.321+0000 7f7569d3c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f756406db20 0x7f756415d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:06.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.321+0000 7f7569d3c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75640b4110 con 0x7f7564159c70 2023-11-09T03:48:06.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.321+0000 7f7569d3c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75640b49b0 con 0x7f75640b0ae0 2023-11-09T03:48:06.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.321+0000 7f7569d3c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75640b43f0 con 0x7f756406db20 2023-11-09T03:48:06.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.321+0000 7f75637fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f7564159c70 0x7f75640b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59982/0 (socket says 172.21.15.76:59982) 2023-11-09T03:48:06.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.321+0000 7f75637fe640 1 -- 172.21.15.76:0/1735816409 learned_addr learned my addr 172.21.15.76:0/1735816409 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:06.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.321+0000 7f7562ffd640 1 --2- 172.21.15.76:0/1735816409 >> v2:172.21.15.113:3300/0 conn(0x7f75640b0ae0 0x7f75640ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:06.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.321+0000 7f7563fff640 1 --2- 172.21.15.76:0/1735816409 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f756406db20 0x7f756415d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:06.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.321+0000 7f75627fc640 1 -- 172.21.15.76:0/1735816409 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1558592525 0 0) 0x7f75640b4110 con 0x7f7564159c70 2023-11-09T03:48:06.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.321+0000 7f75627fc640 1 -- 172.21.15.76:0/1735816409 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f754c005730 con 0x7f7564159c70 2023-11-09T03:48:06.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.322+0000 7f75627fc640 1 -- 172.21.15.76:0/1735816409 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3669859315 0 0) 0x7f754c005730 con 0x7f7564159c70 2023-11-09T03:48:06.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.322+0000 7f7562ffd640 1 -- 172.21.15.76:0/1735816409 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f756406db20 msgr2=0x7f756415d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:06.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.322+0000 7f7562ffd640 1 --2- 172.21.15.76:0/1735816409 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f756406db20 0x7f756415d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.322+0000 7f7562ffd640 1 -- 172.21.15.76:0/1735816409 >> v1:172.21.15.76:6789/0 conn(0x7f7564159c70 legacy=0x7f75640b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:06.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.322+0000 7f7562ffd640 1 -- 172.21.15.76:0/1735816409 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75640b46d0 con 0x7f75640b0ae0 2023-11-09T03:48:06.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.322+0000 7f7563fff640 1 --2- 172.21.15.76:0/1735816409 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f756406db20 0x7f756415d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:48:06.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.322+0000 7f7562ffd640 1 --2- 172.21.15.76:0/1735816409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75640b0ae0 0x7f75640ce370 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f75580097a0 tx=0x7f755802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3369df6009e969c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:06.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.323+0000 7f75627fc640 1 -- 172.21.15.76:0/1735816409 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7554001670 con 0x7f75640b0ae0 2023-11-09T03:48:06.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.323+0000 7f75627fc640 1 -- 172.21.15.76:0/1735816409 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755802e050 con 0x7f75640b0ae0 2023-11-09T03:48:06.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.323+0000 7f75627fc640 1 -- 172.21.15.76:0/1735816409 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558030710 con 0x7f75640b0ae0 2023-11-09T03:48:06.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.323+0000 7f7569d3c640 1 -- 172.21.15.76:0/1735816409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75640b0ae0 msgr2=0x7f75640ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:06.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.323+0000 7f7569d3c640 1 --2- 172.21.15.76:0/1735816409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75640b0ae0 0x7f75640ce370 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f75580097a0 tx=0x7f755802b790 comp rx=0 tx=0).stop 2023-11-09T03:48:06.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.323+0000 7f7569d3c640 1 -- 172.21.15.76:0/1735816409 shutdown_connections 2023-11-09T03:48:06.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.323+0000 7f7569d3c640 1 --2- 172.21.15.76:0/1735816409 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f756406db20 0x7f756415d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.323+0000 7f7569d3c640 1 --2- 172.21.15.76:0/1735816409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75640b0ae0 0x7f75640ce370 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.323+0000 7f7569d3c640 1 -- 172.21.15.76:0/1735816409 >> 172.21.15.76:0/1735816409 conn(0x7f75640ce950 msgr2=0x7f75640ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:06.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.323+0000 7f7569d3c640 1 -- 172.21.15.76:0/1735816409 shutdown_connections 2023-11-09T03:48:06.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.323+0000 7f7569d3c640 1 -- 172.21.15.76:0/1735816409 wait complete. 2023-11-09T03:48:06.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.324+0000 7f7569d3c640 1 Processor -- start 2023-11-09T03:48:06.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.324+0000 7f7569d3c640 1 -- start start 2023-11-09T03:48:06.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.325+0000 7f7569d3c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f756406db20 0x7f75640aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:06.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.325+0000 7f7569d3c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75640b0ae0 0x7f756415a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:06.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.325+0000 7f7569d3c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7564159c70 0x7f7564140cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:06.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.325+0000 7f7569d3c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756415ab80 con 0x7f7564159c70 2023-11-09T03:48:06.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.325+0000 7f7569d3c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75640c8d60 con 0x7f75640b0ae0 2023-11-09T03:48:06.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.325+0000 7f7569d3c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75641510c0 con 0x7f756406db20 2023-11-09T03:48:06.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.325+0000 7f7563fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7564159c70 0x7f7564140cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:06.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.325+0000 7f7563fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7564159c70 0x7f7564140cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50704/0 (socket says 172.21.15.76:50704) 2023-11-09T03:48:06.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.325+0000 7f7563fff640 1 -- 172.21.15.76:0/3087689258 learned_addr learned my addr 172.21.15.76:0/3087689258 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:06.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.326+0000 7f75637fe640 1 --2- 172.21.15.76:0/3087689258 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f756406db20 0x7f75640aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:06.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.326+0000 7f7562ffd640 1 --2- 172.21.15.76:0/3087689258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75640b0ae0 0x7f756415a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:06.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.326+0000 7f7563fff640 1 -- 172.21.15.76:0/3087689258 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f756406db20 msgr2=0x7f75640aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:06.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.326+0000 7f7563fff640 1 --2- 172.21.15.76:0/3087689258 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f756406db20 0x7f75640aecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.326+0000 7f7563fff640 1 -- 172.21.15.76:0/3087689258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75640b0ae0 msgr2=0x7f756415a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:06.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.326+0000 7f7563fff640 1 --2- 172.21.15.76:0/3087689258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75640b0ae0 0x7f756415a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.326+0000 7f7563fff640 1 -- 172.21.15.76:0/3087689258 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c009a70 con 0x7f7564159c70 2023-11-09T03:48:06.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.326+0000 7f7562ffd640 1 --2- 172.21.15.76:0/3087689258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75640b0ae0 0x7f756415a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:06.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.326+0000 7f7563fff640 1 --2- 172.21.15.76:0/3087689258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7564159c70 0x7f7564140cc0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f755c009f00 tx=0x7f755c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:06.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.326+0000 7f754bfff640 1 -- 172.21.15.76:0/3087689258 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c009d70 con 0x7f7564159c70 2023-11-09T03:48:06.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.326+0000 7f754bfff640 1 -- 172.21.15.76:0/3087689258 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755c00f040 con 0x7f7564159c70 2023-11-09T03:48:06.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.326+0000 7f754bfff640 1 -- 172.21.15.76:0/3087689258 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c003bb0 con 0x7f7564159c70 2023-11-09T03:48:06.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.326+0000 7f7569d3c640 1 -- 172.21.15.76:0/3087689258 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7558009400 con 0x7f7564159c70 2023-11-09T03:48:06.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.326+0000 7f7569d3c640 1 -- 172.21.15.76:0/3087689258 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75640c5e10 con 0x7f7564159c70 2023-11-09T03:48:06.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.327+0000 7f7569d3c640 1 -- 172.21.15.76:0/3087689258 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7564074290 con 0x7f7564159c70 2023-11-09T03:48:06.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.327+0000 7f754bfff640 1 -- 172.21.15.76:0/3087689258 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f755c014480 con 0x7f7564159c70 2023-11-09T03:48:06.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.328+0000 7f754bfff640 1 --2- 172.21.15.76:0/3087689258 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f755006ae70 0x7f755006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:06.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.328+0000 7f754bfff640 1 -- 172.21.15.76:0/3087689258 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7148+0+0 (secure 0 0 0) 0x7f755c08a820 con 0x7f7564159c70 2023-11-09T03:48:06.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.328+0000 7f75637fe640 1 --2- 172.21.15.76:0/3087689258 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f755006ae70 0x7f755006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:06.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.328+0000 7f75637fe640 1 --2- 172.21.15.76:0/3087689258 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f755006ae70 0x7f755006d330 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f75540011e0 tx=0x7f7554009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:06.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.332+0000 7f754bfff640 1 -- 172.21.15.76:0/3087689258 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f755c0553c0 con 0x7f7564159c70 2023-11-09T03:48:06.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.501+0000 7f7569d3c640 1 -- 172.21.15.76:0/3087689258 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75640ce730 con 0x7f7564159c70 2023-11-09T03:48:06.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.502+0000 7f754bfff640 1 -- 172.21.15.76:0/3087689258 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+22089 (secure 0 0 0) 0x7f755c0589e0 con 0x7f7564159c70 2023-11-09T03:48:06.502 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:06.502 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":70,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:48:02.436030+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:47:49.630346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-09T03:47:50.082207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-09T03:47:59.280601+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-09T03:47:59.742039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":12,"snaps":[{"begin":1,"length":1}]}],"new_removed_snaps":[{"pool":12,"snaps":[{"begin":1,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:48:06.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.504+0000 7f7569d3c640 1 -- 172.21.15.76:0/3087689258 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f755006ae70 msgr2=0x7f755006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:06.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.504+0000 7f7569d3c640 1 --2- 172.21.15.76:0/3087689258 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f755006ae70 0x7f755006d330 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f75540011e0 tx=0x7f7554009040 comp rx=0 tx=0).stop 2023-11-09T03:48:06.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.504+0000 7f7569d3c640 1 -- 172.21.15.76:0/3087689258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7564159c70 msgr2=0x7f7564140cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:06.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.504+0000 7f7569d3c640 1 --2- 172.21.15.76:0/3087689258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7564159c70 0x7f7564140cc0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f755c009f00 tx=0x7f755c00c490 comp rx=0 tx=0).stop 2023-11-09T03:48:06.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.504+0000 7f7569d3c640 1 -- 172.21.15.76:0/3087689258 shutdown_connections 2023-11-09T03:48:06.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.504+0000 7f7569d3c640 1 --2- 172.21.15.76:0/3087689258 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f755006ae70 0x7f755006d330 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.504+0000 7f7569d3c640 1 --2- 172.21.15.76:0/3087689258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7564159c70 0x7f7564140cc0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.504+0000 7f7569d3c640 1 --2- 172.21.15.76:0/3087689258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75640b0ae0 0x7f756415a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.504+0000 7f7569d3c640 1 --2- 172.21.15.76:0/3087689258 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f756406db20 0x7f75640aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.504+0000 7f7569d3c640 1 -- 172.21.15.76:0/3087689258 >> 172.21.15.76:0/3087689258 conn(0x7f75640ce950 msgr2=0x7f75640cb060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:06.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.504+0000 7f7569d3c640 1 -- 172.21.15.76:0/3087689258 shutdown_connections 2023-11-09T03:48:06.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.504+0000 7f7569d3c640 1 -- 172.21.15.76:0/3087689258 wait complete. 2023-11-09T03:48:06.515 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-09T03:47:00.036540+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:47:49.630346+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:47:50.082207+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_data_pool', 'create_time': '2023-11-09T03:47:59.280601+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 70, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-09T03:47:59.742039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-09T03:48:06.516 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:48:06.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.620+0000 7fec318a2640 1 Processor -- start 2023-11-09T03:48:06.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.620+0000 7fec318a2640 1 -- start start 2023-11-09T03:48:06.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.621+0000 7fec318a2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fec2c0b4880 0x7fec2c0b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:06.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.621+0000 7fec318a2640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec2c0cde60 con 0x7fec2c153e60 2023-11-09T03:48:06.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.621+0000 7fec318a2640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec2c160f90 con 0x7fec2c0cd6d0 2023-11-09T03:48:06.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.621+0000 7fec318a2640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec2c0b5200 con 0x7fec2c0b4880 2023-11-09T03:48:06.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.621+0000 7fec2b7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fec2c0b4880 0x7fec2c0b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:06.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.621+0000 7fec2affd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fec2c153e60 0x7fec2c15dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59994/0 (socket says 172.21.15.76:59994) 2023-11-09T03:48:06.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.621+0000 7fec2affd640 1 -- 172.21.15.76:0/1819675052 learned_addr learned my addr 172.21.15.76:0/1819675052 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:06.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.621+0000 7fec2b7fe640 1 -- 172.21.15.76:0/1819675052 >> v1:172.21.15.113:6789/0 conn(0x7fec2c0cd6d0 legacy=0x7fec2c161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:06.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.621+0000 7fec2b7fe640 1 -- 172.21.15.76:0/1819675052 >> v1:172.21.15.76:6789/0 conn(0x7fec2c153e60 legacy=0x7fec2c15dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:06.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.621+0000 7fec2b7fe640 1 -- 172.21.15.76:0/1819675052 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2c0ce090 con 0x7fec2c0b4880 2023-11-09T03:48:06.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.622+0000 7fec2b7fe640 1 --2- 172.21.15.76:0/1819675052 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec2c0b4880 0x7fec2c0b4cc0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fec240097a0 tx=0x7fec2402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1001bdf184ee464 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:06.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.622+0000 7fec29ffb640 1 -- 172.21.15.76:0/1819675052 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec24031830 con 0x7fec2c0b4880 2023-11-09T03:48:06.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.622+0000 7fec29ffb640 1 -- 172.21.15.76:0/1819675052 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec24031990 con 0x7fec2c0b4880 2023-11-09T03:48:06.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.622+0000 7fec29ffb640 1 -- 172.21.15.76:0/1819675052 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec24031d20 con 0x7fec2c0b4880 2023-11-09T03:48:06.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.623+0000 7fec318a2640 1 -- 172.21.15.76:0/1819675052 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec2c0b4880 msgr2=0x7fec2c0b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:06.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.623+0000 7fec318a2640 1 --2- 172.21.15.76:0/1819675052 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec2c0b4880 0x7fec2c0b4cc0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fec240097a0 tx=0x7fec2402b8d0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.623+0000 7fec318a2640 1 -- 172.21.15.76:0/1819675052 shutdown_connections 2023-11-09T03:48:06.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.623+0000 7fec318a2640 1 --2- 172.21.15.76:0/1819675052 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec2c0b4880 0x7fec2c0b4cc0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.623+0000 7fec318a2640 1 -- 172.21.15.76:0/1819675052 >> 172.21.15.76:0/1819675052 conn(0x7fec2c0cf880 msgr2=0x7fec2c0cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:06.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.623+0000 7fec318a2640 1 -- 172.21.15.76:0/1819675052 shutdown_connections 2023-11-09T03:48:06.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.623+0000 7fec318a2640 1 -- 172.21.15.76:0/1819675052 wait complete. 2023-11-09T03:48:06.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.624+0000 7fec318a2640 1 Processor -- start 2023-11-09T03:48:06.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.624+0000 7fec318a2640 1 -- start start 2023-11-09T03:48:06.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.625+0000 7fec318a2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec2c0b4880 0x7fec2c152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:06.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.625+0000 7fec318a2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec2c0cd6d0 0x7fec2c154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:06.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.625+0000 7fec318a2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec2c153e60 0x7fec2c156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:06.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.625+0000 7fec318a2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c157870 con 0x7fec2c0b4880 2023-11-09T03:48:06.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.625+0000 7fec318a2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c1572b0 con 0x7fec2c153e60 2023-11-09T03:48:06.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.625+0000 7fec318a2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c157590 con 0x7fec2c0cd6d0 2023-11-09T03:48:06.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.625+0000 7fec2affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec2c0b4880 0x7fec2c152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:06.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.625+0000 7fec2affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec2c0b4880 0x7fec2c152f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50722/0 (socket says 172.21.15.76:50722) 2023-11-09T03:48:06.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.625+0000 7fec2affd640 1 -- 172.21.15.76:0/3056066613 learned_addr learned my addr 172.21.15.76:0/3056066613 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:06.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.625+0000 7fec2b7fe640 1 --2- 172.21.15.76:0/3056066613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec2c153e60 0x7fec2c156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:06.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.625+0000 7fec2a7fc640 1 --2- 172.21.15.76:0/3056066613 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec2c0cd6d0 0x7fec2c154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:06.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.625+0000 7fec2affd640 1 -- 172.21.15.76:0/3056066613 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec2c0cd6d0 msgr2=0x7fec2c154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:06.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.625+0000 7fec2affd640 1 --2- 172.21.15.76:0/3056066613 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec2c0cd6d0 0x7fec2c154760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.625+0000 7fec2affd640 1 -- 172.21.15.76:0/3056066613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec2c153e60 msgr2=0x7fec2c156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:06.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.625+0000 7fec2affd640 1 --2- 172.21.15.76:0/3056066613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec2c153e60 0x7fec2c156a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.625+0000 7fec2affd640 1 -- 172.21.15.76:0/3056066613 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec24009400 con 0x7fec2c0b4880 2023-11-09T03:48:06.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.625+0000 7fec2affd640 1 --2- 172.21.15.76:0/3056066613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec2c0b4880 0x7fec2c152f70 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fec1c00cbc0 tx=0x7fec1c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:06.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.625+0000 7fec13fff640 1 -- 172.21.15.76:0/3056066613 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1c00aca0 con 0x7fec2c0b4880 2023-11-09T03:48:06.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.625+0000 7fec13fff640 1 -- 172.21.15.76:0/3056066613 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1c00ae00 con 0x7fec2c0b4880 2023-11-09T03:48:06.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.625+0000 7fec318a2640 1 -- 172.21.15.76:0/3056066613 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec2c0c6b50 con 0x7fec2c0b4880 2023-11-09T03:48:06.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.625+0000 7fec13fff640 1 -- 172.21.15.76:0/3056066613 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1c012600 con 0x7fec2c0b4880 2023-11-09T03:48:06.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.625+0000 7fec318a2640 1 -- 172.21.15.76:0/3056066613 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec2c156f90 con 0x7fec2c0b4880 2023-11-09T03:48:06.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.626+0000 7fec318a2640 1 -- 172.21.15.76:0/3056066613 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec2c0c69b0 con 0x7fec2c0b4880 2023-11-09T03:48:06.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.627+0000 7fec13fff640 1 -- 172.21.15.76:0/3056066613 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fec1c007070 con 0x7fec2c0b4880 2023-11-09T03:48:06.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.627+0000 7fec13fff640 1 --2- 172.21.15.76:0/3056066613 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fec1806ae70 0x7fec1806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:06.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.627+0000 7fec13fff640 1 -- 172.21.15.76:0/3056066613 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7148+0+0 (secure 0 0 0) 0x7fec1c089910 con 0x7fec2c0b4880 2023-11-09T03:48:06.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.627+0000 7fec2a7fc640 1 --2- 172.21.15.76:0/3056066613 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fec1806ae70 0x7fec1806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:06.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.628+0000 7fec2a7fc640 1 --2- 172.21.15.76:0/3056066613 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fec1806ae70 0x7fec1806d330 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fec2c161b30 tx=0x7fec20009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:06.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.631+0000 7fec13fff640 1 -- 172.21.15.76:0/3056066613 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fec2c0c69b0 con 0x7fec2c0b4880 2023-11-09T03:48:06.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.799+0000 7fec318a2640 1 -- 172.21.15.76:0/3056066613 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fec2c0c69b0 con 0x7fec2c0b4880 2023-11-09T03:48:06.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.799+0000 7fec13fff640 1 -- 172.21.15.76:0/3056066613 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7fec1c017020 con 0x7fec2c0b4880 2023-11-09T03:48:06.799 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:48:06.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.801+0000 7fec318a2640 1 -- 172.21.15.76:0/3056066613 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fec1806ae70 msgr2=0x7fec1806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:06.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.801+0000 7fec318a2640 1 --2- 172.21.15.76:0/3056066613 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fec1806ae70 0x7fec1806d330 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fec2c161b30 tx=0x7fec20009040 comp rx=0 tx=0).stop 2023-11-09T03:48:06.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.801+0000 7fec318a2640 1 -- 172.21.15.76:0/3056066613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec2c0b4880 msgr2=0x7fec2c152f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:06.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.801+0000 7fec318a2640 1 --2- 172.21.15.76:0/3056066613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec2c0b4880 0x7fec2c152f70 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fec1c00cbc0 tx=0x7fec1c00a060 comp rx=0 tx=0).stop 2023-11-09T03:48:06.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.801+0000 7fec318a2640 1 -- 172.21.15.76:0/3056066613 shutdown_connections 2023-11-09T03:48:06.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.801+0000 7fec318a2640 1 --2- 172.21.15.76:0/3056066613 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fec1806ae70 0x7fec1806d330 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.801+0000 7fec318a2640 1 --2- 172.21.15.76:0/3056066613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec2c153e60 0x7fec2c156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.801+0000 7fec318a2640 1 --2- 172.21.15.76:0/3056066613 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec2c0cd6d0 0x7fec2c154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.801+0000 7fec318a2640 1 --2- 172.21.15.76:0/3056066613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec2c0b4880 0x7fec2c152f70 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.801+0000 7fec318a2640 1 -- 172.21.15.76:0/3056066613 >> 172.21.15.76:0/3056066613 conn(0x7fec2c0cf880 msgr2=0x7fec2c15bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:06.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.801+0000 7fec318a2640 1 -- 172.21.15.76:0/3056066613 shutdown_connections 2023-11-09T03:48:06.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.801+0000 7fec318a2640 1 -- 172.21.15.76:0/3056066613 wait complete. 2023-11-09T03:48:06.812 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-09T03:48:06.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.915+0000 7f0d3c7e7640 1 Processor -- start 2023-11-09T03:48:06.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.915+0000 7f0d3c7e7640 1 -- start start 2023-11-09T03:48:06.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.916+0000 7f0d3c7e7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0d34153850 0x7f0d3415a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:06.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.916+0000 7f0d3c7e7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0d340cc9d0 0x7f0d340c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:06.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.916+0000 7f0d3c7e7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d340c3860 con 0x7f0d340cc520 2023-11-09T03:48:06.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.916+0000 7f0d3c7e7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d340c9250 con 0x7f0d34153850 2023-11-09T03:48:06.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.916+0000 7f0d3c7e7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d34153c30 con 0x7f0d340cc9d0 2023-11-09T03:48:06.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.916+0000 7f0d3a583640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0d34153850 0x7f0d3415a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:06.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.916+0000 7f0d39d82640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0d340cc520 0x7f0d34070fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59996/0 (socket says 172.21.15.76:59996) 2023-11-09T03:48:06.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.916+0000 7f0d39d82640 1 -- 172.21.15.76:0/2044054125 learned_addr learned my addr 172.21.15.76:0/2044054125 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:06.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.917+0000 7f0d3ad84640 1 --2- 172.21.15.76:0/2044054125 >> v2:172.21.15.113:3300/0 conn(0x7f0d340cc9d0 0x7f0d340c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:06.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.917+0000 7f0d3a583640 1 -- 172.21.15.76:0/2044054125 >> v2:172.21.15.113:3300/0 conn(0x7f0d340cc9d0 msgr2=0x7f0d340c3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:06.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.917+0000 7f0d3a583640 1 --2- 172.21.15.76:0/2044054125 >> v2:172.21.15.113:3300/0 conn(0x7f0d340cc9d0 0x7f0d340c3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.917+0000 7f0d3a583640 1 -- 172.21.15.76:0/2044054125 >> v1:172.21.15.76:6789/0 conn(0x7f0d340cc520 legacy=0x7f0d34070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:06.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.917+0000 7f0d3a583640 1 -- 172.21.15.76:0/2044054125 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3415a700 con 0x7f0d34153850 2023-11-09T03:48:06.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.917+0000 7f0d39581640 1 -- 172.21.15.76:0/2044054125 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4289248780 0 0) 0x7f0d340c3860 con 0x7f0d340cc520 2023-11-09T03:48:06.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.917+0000 7f0d3ad84640 1 --2- 172.21.15.76:0/2044054125 >> v2:172.21.15.113:3300/0 conn(0x7f0d340cc9d0 0x7f0d340c3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:06.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.917+0000 7f0d3a583640 1 --2- 172.21.15.76:0/2044054125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d34153850 0x7f0d3415a1c0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f0d340ca7c0 tx=0x7f0d3002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d68efad932a0d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:06.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.918+0000 7f0d39581640 1 -- 172.21.15.76:0/2044054125 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3002cc80 con 0x7f0d34153850 2023-11-09T03:48:06.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.918+0000 7f0d39581640 1 -- 172.21.15.76:0/2044054125 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d3002cde0 con 0x7f0d34153850 2023-11-09T03:48:06.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.918+0000 7f0d39581640 1 -- 172.21.15.76:0/2044054125 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d30034660 con 0x7f0d34153850 2023-11-09T03:48:06.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.918+0000 7f0d3c7e7640 1 -- 172.21.15.76:0/2044054125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d34153850 msgr2=0x7f0d3415a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:06.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.918+0000 7f0d3c7e7640 1 --2- 172.21.15.76:0/2044054125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d34153850 0x7f0d3415a1c0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f0d340ca7c0 tx=0x7f0d3002bed0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.918+0000 7f0d3c7e7640 1 -- 172.21.15.76:0/2044054125 shutdown_connections 2023-11-09T03:48:06.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.918+0000 7f0d3c7e7640 1 --2- 172.21.15.76:0/2044054125 >> v2:172.21.15.113:3300/0 conn(0x7f0d340cc9d0 0x7f0d340c3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.918+0000 7f0d3c7e7640 1 --2- 172.21.15.76:0/2044054125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d34153850 0x7f0d3415a1c0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.918+0000 7f0d3c7e7640 1 -- 172.21.15.76:0/2044054125 >> 172.21.15.76:0/2044054125 conn(0x7f0d340ce950 msgr2=0x7f0d340ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:06.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.918+0000 7f0d3c7e7640 1 -- 172.21.15.76:0/2044054125 shutdown_connections 2023-11-09T03:48:06.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.918+0000 7f0d3c7e7640 1 -- 172.21.15.76:0/2044054125 wait complete. 2023-11-09T03:48:06.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.919+0000 7f0d3c7e7640 1 Processor -- start 2023-11-09T03:48:06.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.919+0000 7f0d3c7e7640 1 -- start start 2023-11-09T03:48:06.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.920+0000 7f0d3c7e7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d340cc520 0x7f0d341f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:06.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.920+0000 7f0d3c7e7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d340cc9d0 0x7f0d341f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:06.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.920+0000 7f0d3c7e7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d34153850 0x7f0d341e19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:06.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.920+0000 7f0d3c7e7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d341f28e0 con 0x7f0d340cc9d0 2023-11-09T03:48:06.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.920+0000 7f0d3c7e7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d34151020 con 0x7f0d34153850 2023-11-09T03:48:06.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.920+0000 7f0d3c7e7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d34151300 con 0x7f0d340cc520 2023-11-09T03:48:06.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.920+0000 7f0d39d82640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d340cc9d0 0x7f0d341f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:06.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.920+0000 7f0d39d82640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d340cc9d0 0x7f0d341f4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50740/0 (socket says 172.21.15.76:50740) 2023-11-09T03:48:06.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.920+0000 7f0d39d82640 1 -- 172.21.15.76:0/2363510622 learned_addr learned my addr 172.21.15.76:0/2363510622 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:06.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.920+0000 7f0d3ad84640 1 --2- 172.21.15.76:0/2363510622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d34153850 0x7f0d341e19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:06.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.920+0000 7f0d3a583640 1 --2- 172.21.15.76:0/2363510622 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d340cc520 0x7f0d341f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:06.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.920+0000 7f0d39d82640 1 -- 172.21.15.76:0/2363510622 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d340cc520 msgr2=0x7f0d341f23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:06.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.920+0000 7f0d39d82640 1 --2- 172.21.15.76:0/2363510622 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d340cc520 0x7f0d341f23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.920+0000 7f0d39d82640 1 -- 172.21.15.76:0/2363510622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d34153850 msgr2=0x7f0d341e19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:06.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.920+0000 7f0d39d82640 1 --2- 172.21.15.76:0/2363510622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d34153850 0x7f0d341e19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:06.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.921+0000 7f0d39d82640 1 -- 172.21.15.76:0/2363510622 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d30009a70 con 0x7f0d340cc9d0 2023-11-09T03:48:06.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.921+0000 7f0d3a583640 1 --2- 172.21.15.76:0/2363510622 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d340cc520 0x7f0d341f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:06.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.921+0000 7f0d39d82640 1 --2- 172.21.15.76:0/2363510622 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d340cc9d0 0x7f0d341f4f70 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400c4d0 tx=0x7f0d2400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:06.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.921+0000 7f0d2affd640 1 -- 172.21.15.76:0/2363510622 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d240074e0 con 0x7f0d340cc9d0 2023-11-09T03:48:06.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.921+0000 7f0d2affd640 1 -- 172.21.15.76:0/2363510622 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d24007660 con 0x7f0d340cc9d0 2023-11-09T03:48:06.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.921+0000 7f0d2affd640 1 -- 172.21.15.76:0/2363510622 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24004a20 con 0x7f0d340cc9d0 2023-11-09T03:48:06.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.921+0000 7f0d3c7e7640 1 -- 172.21.15.76:0/2363510622 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d341515e0 con 0x7f0d340cc9d0 2023-11-09T03:48:06.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.921+0000 7f0d3c7e7640 1 -- 172.21.15.76:0/2363510622 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d34151ba0 con 0x7f0d340cc9d0 2023-11-09T03:48:06.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.922+0000 7f0d3c7e7640 1 -- 172.21.15.76:0/2363510622 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d34074290 con 0x7f0d340cc9d0 2023-11-09T03:48:06.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.922+0000 7f0d2affd640 1 -- 172.21.15.76:0/2363510622 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0d24004b80 con 0x7f0d340cc9d0 2023-11-09T03:48:06.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.923+0000 7f0d2affd640 1 --2- 172.21.15.76:0/2363510622 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0d1806ae70 0x7f0d1806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:06.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.923+0000 7f0d2affd640 1 -- 172.21.15.76:0/2363510622 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7148+0+0 (secure 0 0 0) 0x7f0d24088d50 con 0x7f0d340cc9d0 2023-11-09T03:48:06.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.923+0000 7f0d3a583640 1 --2- 172.21.15.76:0/2363510622 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0d1806ae70 0x7f0d1806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:06.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.923+0000 7f0d3a583640 1 --2- 172.21.15.76:0/2363510622 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0d1806ae70 0x7f0d1806d330 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0d30036000 tx=0x7f0d30030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:06.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:06.927+0000 7f0d2affd640 1 -- 172.21.15.76:0/2363510622 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0d2408c030 con 0x7f0d340cc9d0 2023-11-09T03:48:07.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.091+0000 7f0d3c7e7640 1 -- 172.21.15.76:0/2363510622 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f0d340c5010 con 0x7f0d340cc9d0 2023-11-09T03:48:07.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.092+0000 7f0d2affd640 1 -- 172.21.15.76:0/2363510622 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v70) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0d240180a0 con 0x7f0d340cc9d0 2023-11-09T03:48:07.092 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:48:07.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.094+0000 7f0d3c7e7640 1 -- 172.21.15.76:0/2363510622 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0d1806ae70 msgr2=0x7f0d1806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:07.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.094+0000 7f0d3c7e7640 1 --2- 172.21.15.76:0/2363510622 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0d1806ae70 0x7f0d1806d330 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0d30036000 tx=0x7f0d30030000 comp rx=0 tx=0).stop 2023-11-09T03:48:07.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.094+0000 7f0d3c7e7640 1 -- 172.21.15.76:0/2363510622 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d340cc9d0 msgr2=0x7f0d341f4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:07.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.094+0000 7f0d3c7e7640 1 --2- 172.21.15.76:0/2363510622 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d340cc9d0 0x7f0d341f4f70 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400c4d0 tx=0x7f0d2400c990 comp rx=0 tx=0).stop 2023-11-09T03:48:07.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.094+0000 7f0d3c7e7640 1 -- 172.21.15.76:0/2363510622 shutdown_connections 2023-11-09T03:48:07.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.094+0000 7f0d3c7e7640 1 --2- 172.21.15.76:0/2363510622 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0d1806ae70 0x7f0d1806d330 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.094+0000 7f0d3c7e7640 1 --2- 172.21.15.76:0/2363510622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d34153850 0x7f0d341e19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.094+0000 7f0d3c7e7640 1 --2- 172.21.15.76:0/2363510622 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d340cc9d0 0x7f0d341f4f70 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.094+0000 7f0d3c7e7640 1 --2- 172.21.15.76:0/2363510622 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d340cc520 0x7f0d341f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.094+0000 7f0d3c7e7640 1 -- 172.21.15.76:0/2363510622 >> 172.21.15.76:0/2363510622 conn(0x7f0d340ce950 msgr2=0x7f0d340ccdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:07.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.094+0000 7f0d3c7e7640 1 -- 172.21.15.76:0/2363510622 shutdown_connections 2023-11-09T03:48:07.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.094+0000 7f0d3c7e7640 1 -- 172.21.15.76:0/2363510622 wait complete. 2023-11-09T03:48:07.104 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:48:07.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.207+0000 7efcdb27c640 1 Processor -- start 2023-11-09T03:48:07.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.207+0000 7efcdb27c640 1 -- start start 2023-11-09T03:48:07.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.208+0000 7efcdb27c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efcd4160430 0x7efcd40b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:07.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.208+0000 7efcdb27c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd40b2fc0 con 0x7efcd40b2b60 2023-11-09T03:48:07.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.208+0000 7efcdb27c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd406c810 con 0x7efcd406c3a0 2023-11-09T03:48:07.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.208+0000 7efcdb27c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd40cb580 con 0x7efcd4160430 2023-11-09T03:48:07.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.208+0000 7efcd8817640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7efcd40b2b60 0x7efcd406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:60008/0 (socket says 172.21.15.76:60008) 2023-11-09T03:48:07.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.208+0000 7efcd8817640 1 -- 172.21.15.76:0/3331455366 learned_addr learned my addr 172.21.15.76:0/3331455366 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:07.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.208+0000 7efccbfff640 1 -- 172.21.15.76:0/3331455366 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 104317693 0 0) 0x7efcd40b2fc0 con 0x7efcd40b2b60 2023-11-09T03:48:07.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.209+0000 7efccbfff640 1 -- 172.21.15.76:0/3331455366 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcc0003520 con 0x7efcd40b2b60 2023-11-09T03:48:07.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.209+0000 7efcd9018640 1 --2- 172.21.15.76:0/3331455366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efcd4160430 0x7efcd40b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:07.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.209+0000 7efccbfff640 1 -- 172.21.15.76:0/3331455366 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1263062773 0 0) 0x7efcc0003520 con 0x7efcd40b2b60 2023-11-09T03:48:07.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.209+0000 7efccbfff640 1 -- 172.21.15.76:0/3331455366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efcd4160430 msgr2=0x7efcd40b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:07.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.209+0000 7efccbfff640 1 --2- 172.21.15.76:0/3331455366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efcd4160430 0x7efcd40b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.209+0000 7efccbfff640 1 -- 172.21.15.76:0/3331455366 >> v1:172.21.15.113:6789/0 conn(0x7efcd406c3a0 legacy=0x7efcd40cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:07.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.209+0000 7efccbfff640 1 -- 172.21.15.76:0/3331455366 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd406ca40 con 0x7efcd40b2b60 2023-11-09T03:48:07.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.209+0000 7efccbfff640 1 -- 172.21.15.76:0/3331455366 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7efcc4001670 con 0x7efcd40b2b60 2023-11-09T03:48:07.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.209+0000 7efccbfff640 1 -- 172.21.15.76:0/3331455366 >> v1:172.21.15.76:6789/0 conn(0x7efcd40b2b60 legacy=0x7efcd406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:07.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.209+0000 7efccbfff640 1 --2- 172.21.15.76:0/3331455366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efcd4160430 0x7efcc00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:07.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.209+0000 7efccbfff640 1 --2- 172.21.15.76:0/3331455366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcc0003f20 0x7efcc00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:07.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.209+0000 7efccbfff640 1 --2- 172.21.15.76:0/3331455366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcc0004d00 0x7efcc0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:07.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.209+0000 7efccbfff640 1 -- 172.21.15.76:0/3331455366 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd40cb580 con 0x7efcc0004d00 2023-11-09T03:48:07.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.209+0000 7efccbfff640 1 -- 172.21.15.76:0/3331455366 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc000ba60 con 0x7efcc0003f20 2023-11-09T03:48:07.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.209+0000 7efccbfff640 1 -- 172.21.15.76:0/3331455366 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc000bc20 con 0x7efcd4160430 2023-11-09T03:48:07.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.209+0000 7efcd9819640 1 --2- 172.21.15.76:0/3331455366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcc0004d00 0x7efcc0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:07.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.209+0000 7efcd9018640 1 --2- 172.21.15.76:0/3331455366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efcd4160430 0x7efcc00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:07.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.209+0000 7efcd8817640 1 --2- 172.21.15.76:0/3331455366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcc0003f20 0x7efcc00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:07.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.209+0000 7efcd9819640 1 -- 172.21.15.76:0/3331455366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efcd4160430 msgr2=0x7efcc00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:07.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.209+0000 7efcd9819640 1 --2- 172.21.15.76:0/3331455366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efcd4160430 0x7efcc00039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.209+0000 7efcd9819640 1 -- 172.21.15.76:0/3331455366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcc0003f20 msgr2=0x7efcc00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:07.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.209+0000 7efcd9819640 1 --2- 172.21.15.76:0/3331455366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcc0003f20 0x7efcc00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.209+0000 7efcd9819640 1 -- 172.21.15.76:0/3331455366 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc0003520 con 0x7efcc0004d00 2023-11-09T03:48:07.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.210+0000 7efcd9819640 1 --2- 172.21.15.76:0/3331455366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcc0004d00 0x7efcc0005180 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7efccc001750 tx=0x7efcc00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78a342bc6a968c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:07.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.210+0000 7efccbfff640 1 -- 172.21.15.76:0/3331455366 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc0052d0 con 0x7efcc0004d00 2023-11-09T03:48:07.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.210+0000 7efccbfff640 1 -- 172.21.15.76:0/3331455366 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efccc02e050 con 0x7efcc0004d00 2023-11-09T03:48:07.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.210+0000 7efccbfff640 1 -- 172.21.15.76:0/3331455366 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc003830 con 0x7efcc0004d00 2023-11-09T03:48:07.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.211+0000 7efcdb27c640 1 -- 172.21.15.76:0/3331455366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcc0004d00 msgr2=0x7efcc0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:07.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.211+0000 7efcdb27c640 1 --2- 172.21.15.76:0/3331455366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcc0004d00 0x7efcc0005180 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7efccc001750 tx=0x7efcc00057f0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.211+0000 7efcd9819640 1 -- 172.21.15.76:0/3331455366 reap_dead start 2023-11-09T03:48:07.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.211+0000 7efcdb27c640 1 -- 172.21.15.76:0/3331455366 shutdown_connections 2023-11-09T03:48:07.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.211+0000 7efcdb27c640 1 -- 172.21.15.76:0/3331455366 >> 172.21.15.76:0/3331455366 conn(0x7efcd406ea20 msgr2=0x7efcd406ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:07.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.211+0000 7efcdb27c640 1 -- 172.21.15.76:0/3331455366 shutdown_connections 2023-11-09T03:48:07.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.211+0000 7efcdb27c640 1 -- 172.21.15.76:0/3331455366 wait complete. 2023-11-09T03:48:07.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.212+0000 7efcdb27c640 1 Processor -- start 2023-11-09T03:48:07.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.212+0000 7efcdb27c640 1 -- start start 2023-11-09T03:48:07.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.213+0000 7efcdb27c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcd4150930 0x7efcd4150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:07.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.213+0000 7efcdb27c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcd40b37c0 0x7efcd40b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:07.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.213+0000 7efcdb27c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efcd4160ff0 0x7efcd4161470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:07.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.213+0000 7efcdb27c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd40b3c40 con 0x7efcd4150930 2023-11-09T03:48:07.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.213+0000 7efcdb27c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd41619b0 con 0x7efcd40b37c0 2023-11-09T03:48:07.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.213+0000 7efcdb27c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd40b5e90 con 0x7efcd4160ff0 2023-11-09T03:48:07.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.213+0000 7efcd9018640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcd4150930 0x7efcd4150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:07.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.213+0000 7efcd9018640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcd4150930 0x7efcd4150fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50770/0 (socket says 172.21.15.76:50770) 2023-11-09T03:48:07.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.213+0000 7efcd9018640 1 -- 172.21.15.76:0/247212285 learned_addr learned my addr 172.21.15.76:0/247212285 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:07.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.213+0000 7efcd9018640 1 -- 172.21.15.76:0/247212285 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efcd4160ff0 msgr2=0x7efcd4161470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:07.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.213+0000 7efcd9018640 1 --2- 172.21.15.76:0/247212285 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efcd4160ff0 0x7efcd4161470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.213+0000 7efcd9018640 1 -- 172.21.15.76:0/247212285 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcd40b37c0 msgr2=0x7efcd40b6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:07.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.213+0000 7efcd9018640 1 --2- 172.21.15.76:0/247212285 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcd40b37c0 0x7efcd40b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.213+0000 7efcd9018640 1 -- 172.21.15.76:0/247212285 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd4154950 con 0x7efcd4150930 2023-11-09T03:48:07.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.213+0000 7efcd9018640 1 --2- 172.21.15.76:0/247212285 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcd4150930 0x7efcd4150fc0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7efcd0011e60 tx=0x7efcd000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:07.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.213+0000 7efcc9ffb640 1 -- 172.21.15.76:0/247212285 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd000e040 con 0x7efcd4150930 2023-11-09T03:48:07.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.213+0000 7efcc9ffb640 1 -- 172.21.15.76:0/247212285 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd000f040 con 0x7efcd4150930 2023-11-09T03:48:07.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.213+0000 7efcc9ffb640 1 -- 172.21.15.76:0/247212285 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0015830 con 0x7efcd4150930 2023-11-09T03:48:07.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.213+0000 7efcdb27c640 1 -- 172.21.15.76:0/247212285 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd40b3e30 con 0x7efcd4150930 2023-11-09T03:48:07.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.214+0000 7efcdb27c640 1 -- 172.21.15.76:0/247212285 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd40b5840 con 0x7efcd4150930 2023-11-09T03:48:07.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.215+0000 7efcdb27c640 1 -- 172.21.15.76:0/247212285 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd4074290 con 0x7efcd4150930 2023-11-09T03:48:07.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.215+0000 7efcc9ffb640 1 -- 172.21.15.76:0/247212285 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efcd0015990 con 0x7efcd4150930 2023-11-09T03:48:07.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.215+0000 7efcc9ffb640 1 --2- 172.21.15.76:0/247212285 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efcb806ae70 0x7efcb806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:07.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.215+0000 7efcc9ffb640 1 -- 172.21.15.76:0/247212285 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7148+0+0 (secure 0 0 0) 0x7efcd008a8c0 con 0x7efcd4150930 2023-11-09T03:48:07.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.216+0000 7efcd8817640 1 --2- 172.21.15.76:0/247212285 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efcb806ae70 0x7efcb806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:07.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.216+0000 7efcd8817640 1 --2- 172.21.15.76:0/247212285 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efcb806ae70 0x7efcb806d330 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7efcc4001c90 tx=0x7efcc4001b20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:07.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.220+0000 7efcc9ffb640 1 -- 172.21.15.76:0/247212285 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efcd00553d0 con 0x7efcd4150930 2023-11-09T03:48:07.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.384+0000 7efcdb27c640 1 -- 172.21.15.76:0/247212285 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efcd40cc9d0 con 0x7efcd4150930 2023-11-09T03:48:07.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.384+0000 7efcc9ffb640 1 -- 172.21.15.76:0/247212285 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) v1 ==== 104+0+11 (secure 0 0 0) 0x7efcd00589f0 con 0x7efcd4150930 2023-11-09T03:48:07.384 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:48:07.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.386+0000 7efcdb27c640 1 -- 172.21.15.76:0/247212285 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efcb806ae70 msgr2=0x7efcb806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:07.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.386+0000 7efcdb27c640 1 --2- 172.21.15.76:0/247212285 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efcb806ae70 0x7efcb806d330 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7efcc4001c90 tx=0x7efcc4001b20 comp rx=0 tx=0).stop 2023-11-09T03:48:07.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.386+0000 7efcdb27c640 1 -- 172.21.15.76:0/247212285 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcd4150930 msgr2=0x7efcd4150fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:07.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.386+0000 7efcdb27c640 1 --2- 172.21.15.76:0/247212285 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcd4150930 0x7efcd4150fc0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7efcd0011e60 tx=0x7efcd000f580 comp rx=0 tx=0).stop 2023-11-09T03:48:07.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.386+0000 7efcdb27c640 1 -- 172.21.15.76:0/247212285 shutdown_connections 2023-11-09T03:48:07.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.386+0000 7efcdb27c640 1 --2- 172.21.15.76:0/247212285 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efcb806ae70 0x7efcb806d330 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.386+0000 7efcdb27c640 1 --2- 172.21.15.76:0/247212285 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efcd4160ff0 0x7efcd4161470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.386+0000 7efcdb27c640 1 --2- 172.21.15.76:0/247212285 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcd40b37c0 0x7efcd40b6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.386+0000 7efcdb27c640 1 --2- 172.21.15.76:0/247212285 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcd4150930 0x7efcd4150fc0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.386+0000 7efcdb27c640 1 -- 172.21.15.76:0/247212285 >> 172.21.15.76:0/247212285 conn(0x7efcd406ea20 msgr2=0x7efcd415b320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:07.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.386+0000 7efcdb27c640 1 -- 172.21.15.76:0/247212285 shutdown_connections 2023-11-09T03:48:07.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.386+0000 7efcdb27c640 1 -- 172.21.15.76:0/247212285 wait complete. 2023-11-09T03:48:07.396 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:48:07.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.501+0000 7f1cc7fb6640 1 Processor -- start 2023-11-09T03:48:07.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.501+0000 7f1cc7fb6640 1 -- start start 2023-11-09T03:48:07.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.502+0000 7f1cc7fb6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc0159c00 0x7f1cc00cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:07.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.502+0000 7f1cc7fb6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc00ae1d0 0x7f1cc00c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:07.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.502+0000 7f1cc7fb6640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc015aff0 con 0x7f1cc00c48e0 2023-11-09T03:48:07.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.502+0000 7f1cc7fb6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cc00c52d0 con 0x7f1cc0159c00 2023-11-09T03:48:07.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.502+0000 7f1cc7fb6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc015a3e0 con 0x7f1cc00ae1d0 2023-11-09T03:48:07.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.502+0000 7f1cc5551640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f1cc00c48e0 0x7f1cc00adac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:60020/0 (socket says 172.21.15.76:60020) 2023-11-09T03:48:07.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.502+0000 7f1cc5551640 1 -- 172.21.15.76:0/3987916865 learned_addr learned my addr 172.21.15.76:0/3987916865 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:07.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.502+0000 7f1cc6553640 1 --2- 172.21.15.76:0/3987916865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc00ae1d0 0x7f1cc00c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:07.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.502+0000 7f1cc5d52640 1 --2- 172.21.15.76:0/3987916865 >> v2:172.21.15.113:3300/0 conn(0x7f1cc0159c00 0x7f1cc00cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:07.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.502+0000 7f1cc4d50640 1 -- 172.21.15.76:0/3987916865 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3609132047 0 0) 0x7f1cc015aff0 con 0x7f1cc00c48e0 2023-11-09T03:48:07.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.502+0000 7f1cc4d50640 1 -- 172.21.15.76:0/3987916865 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cac003520 con 0x7f1cc00c48e0 2023-11-09T03:48:07.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.503+0000 7f1cc4d50640 1 -- 172.21.15.76:0/3987916865 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2162831163 0 0) 0x7f1cac003520 con 0x7f1cc00c48e0 2023-11-09T03:48:07.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.503+0000 7f1cc4d50640 1 -- 172.21.15.76:0/3987916865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc00ae1d0 msgr2=0x7f1cc00c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:07.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.503+0000 7f1cc4d50640 1 --2- 172.21.15.76:0/3987916865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc00ae1d0 0x7f1cc00c4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.503+0000 7f1cc4d50640 1 -- 172.21.15.76:0/3987916865 >> v2:172.21.15.113:3300/0 conn(0x7f1cc0159c00 msgr2=0x7f1cc00cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:07.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.503+0000 7f1cc4d50640 1 --2- 172.21.15.76:0/3987916865 >> v2:172.21.15.113:3300/0 conn(0x7f1cc0159c00 0x7f1cc00cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.503+0000 7f1cc4d50640 1 -- 172.21.15.76:0/3987916865 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc015a6c0 con 0x7f1cc00c48e0 2023-11-09T03:48:07.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.503+0000 7f1cc4d50640 1 -- 172.21.15.76:0/3987916865 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f1cb0001670 con 0x7f1cc00c48e0 2023-11-09T03:48:07.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.503+0000 7f1cc4d50640 1 -- 172.21.15.76:0/3987916865 >> v1:172.21.15.76:6789/0 conn(0x7f1cc00c48e0 legacy=0x7f1cc00adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:07.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.503+0000 7f1cc4d50640 1 --2- 172.21.15.76:0/3987916865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc00ae1d0 0x7f1cac003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:07.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.503+0000 7f1cc4d50640 1 --2- 172.21.15.76:0/3987916865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cac003ef0 0x7f1cac0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:07.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.503+0000 7f1cc4d50640 1 --2- 172.21.15.76:0/3987916865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cac004d00 0x7f1cac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:07.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.503+0000 7f1cc4d50640 1 -- 172.21.15.76:0/3987916865 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc00c52d0 con 0x7f1cac004d00 2023-11-09T03:48:07.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.503+0000 7f1cc4d50640 1 -- 172.21.15.76:0/3987916865 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc015a3e0 con 0x7f1cac003ef0 2023-11-09T03:48:07.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.503+0000 7f1cc4d50640 1 -- 172.21.15.76:0/3987916865 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac00b9d0 con 0x7f1cc00ae1d0 2023-11-09T03:48:07.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.503+0000 7f1cc6553640 1 --2- 172.21.15.76:0/3987916865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cac004d00 0x7f1cac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:07.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.503+0000 7f1cc5d52640 1 --2- 172.21.15.76:0/3987916865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc00ae1d0 0x7f1cac003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:07.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.503+0000 7f1cc5551640 1 --2- 172.21.15.76:0/3987916865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cac003ef0 0x7f1cac0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:07.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.503+0000 7f1cc6553640 1 -- 172.21.15.76:0/3987916865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc00ae1d0 msgr2=0x7f1cac003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:07.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.503+0000 7f1cc6553640 1 --2- 172.21.15.76:0/3987916865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc00ae1d0 0x7f1cac003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.503+0000 7f1cc6553640 1 -- 172.21.15.76:0/3987916865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cac003ef0 msgr2=0x7f1cac0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:07.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.503+0000 7f1cc6553640 1 --2- 172.21.15.76:0/3987916865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cac003ef0 0x7f1cac0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.503+0000 7f1cc6553640 1 -- 172.21.15.76:0/3987916865 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cac003520 con 0x7f1cac004d00 2023-11-09T03:48:07.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.504+0000 7f1cc6553640 1 --2- 172.21.15.76:0/3987916865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cac004d00 0x7f1cac005180 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8009af0 tx=0x7f1cac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2db7f43e5960ed8 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:07.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.504+0000 7f1cc4d50640 1 -- 172.21.15.76:0/3987916865 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8036440 con 0x7f1cac004d00 2023-11-09T03:48:07.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.504+0000 7f1cc4d50640 1 -- 172.21.15.76:0/3987916865 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb80365a0 con 0x7f1cac004d00 2023-11-09T03:48:07.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.504+0000 7f1cc4d50640 1 -- 172.21.15.76:0/3987916865 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8036930 con 0x7f1cac004d00 2023-11-09T03:48:07.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.504+0000 7f1cc7fb6640 1 -- 172.21.15.76:0/3987916865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cac004d00 msgr2=0x7f1cac005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:07.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.504+0000 7f1cc7fb6640 1 --2- 172.21.15.76:0/3987916865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cac004d00 0x7f1cac005180 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8009af0 tx=0x7f1cac0057f0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.504+0000 7f1cc6553640 1 -- 172.21.15.76:0/3987916865 reap_dead start 2023-11-09T03:48:07.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.504+0000 7f1cc7fb6640 1 -- 172.21.15.76:0/3987916865 shutdown_connections 2023-11-09T03:48:07.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.504+0000 7f1cc7fb6640 1 -- 172.21.15.76:0/3987916865 >> 172.21.15.76:0/3987916865 conn(0x7f1cc00ce950 msgr2=0x7f1cc00ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:07.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.505+0000 7f1cc7fb6640 1 -- 172.21.15.76:0/3987916865 shutdown_connections 2023-11-09T03:48:07.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.505+0000 7f1cc7fb6640 1 -- 172.21.15.76:0/3987916865 wait complete. 2023-11-09T03:48:07.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.506+0000 7f1cc7fb6640 1 Processor -- start 2023-11-09T03:48:07.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.506+0000 7f1cc7fb6640 1 -- start start 2023-11-09T03:48:07.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.506+0000 7f1cc7fb6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cc0149490 0x7f1cc00af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:07.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.506+0000 7f1cc7fb6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cc01547e0 0x7f1cc00b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:07.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.506+0000 7f1cc7fb6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc00b1600 0x7f1cc006cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:07.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.506+0000 7f1cc7fb6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc00b0c30 con 0x7f1cc0149490 2023-11-09T03:48:07.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.506+0000 7f1cc7fb6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc00b0020 con 0x7f1cc01547e0 2023-11-09T03:48:07.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.506+0000 7f1cc7fb6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc00b0300 con 0x7f1cc00b1600 2023-11-09T03:48:07.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.506+0000 7f1cc5d52640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cc0149490 0x7f1cc00af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:07.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.507+0000 7f1cc5d52640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cc0149490 0x7f1cc00af7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50788/0 (socket says 172.21.15.76:50788) 2023-11-09T03:48:07.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.507+0000 7f1cc5d52640 1 -- 172.21.15.76:0/1914934580 learned_addr learned my addr 172.21.15.76:0/1914934580 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:07.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.507+0000 7f1cc5551640 1 --2- 172.21.15.76:0/1914934580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cc01547e0 0x7f1cc00b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:07.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.507+0000 7f1cc5d52640 1 -- 172.21.15.76:0/1914934580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc00b1600 msgr2=0x7f1cc006cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:07.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.507+0000 7f1cc5d52640 1 --2- 172.21.15.76:0/1914934580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc00b1600 0x7f1cc006cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.507+0000 7f1cc5d52640 1 -- 172.21.15.76:0/1914934580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cc01547e0 msgr2=0x7f1cc00b10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:07.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.507+0000 7f1cc5d52640 1 --2- 172.21.15.76:0/1914934580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cc01547e0 0x7f1cc00b10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.507+0000 7f1cc5d52640 1 -- 172.21.15.76:0/1914934580 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc00b05e0 con 0x7f1cc0149490 2023-11-09T03:48:07.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.507+0000 7f1cc5d52640 1 --2- 172.21.15.76:0/1914934580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cc0149490 0x7f1cc00af7f0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc011e30 tx=0x7f1cbc00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:07.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.507+0000 7f1cb67fc640 1 -- 172.21.15.76:0/1914934580 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc00e040 con 0x7f1cc0149490 2023-11-09T03:48:07.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.507+0000 7f1cb67fc640 1 -- 172.21.15.76:0/1914934580 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cbc00f040 con 0x7f1cc0149490 2023-11-09T03:48:07.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.507+0000 7f1cc7fb6640 1 -- 172.21.15.76:0/1914934580 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc00b08c0 con 0x7f1cc0149490 2023-11-09T03:48:07.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.507+0000 7f1cb67fc640 1 -- 172.21.15.76:0/1914934580 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc009c00 con 0x7f1cc0149490 2023-11-09T03:48:07.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.507+0000 7f1cc7fb6640 1 -- 172.21.15.76:0/1914934580 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc0149870 con 0x7f1cc0149490 2023-11-09T03:48:07.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.508+0000 7f1cc7fb6640 1 -- 172.21.15.76:0/1914934580 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc0074290 con 0x7f1cc0149490 2023-11-09T03:48:07.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.508+0000 7f1cb67fc640 1 -- 172.21.15.76:0/1914934580 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1cbc015850 con 0x7f1cc0149490 2023-11-09T03:48:07.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.509+0000 7f1cb67fc640 1 --2- 172.21.15.76:0/1914934580 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1ca406aea0 0x7f1ca406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:07.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.509+0000 7f1cb67fc640 1 -- 172.21.15.76:0/1914934580 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7148+0+0 (secure 0 0 0) 0x7f1cbc08aae0 con 0x7f1cc0149490 2023-11-09T03:48:07.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.509+0000 7f1cc5551640 1 --2- 172.21.15.76:0/1914934580 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1ca406aea0 0x7f1ca406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:07.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.510+0000 7f1cc5551640 1 --2- 172.21.15.76:0/1914934580 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1ca406aea0 0x7f1ca406d360 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1cc006dc50 tx=0x7f1cb0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:07.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.513+0000 7f1cb67fc640 1 -- 172.21.15.76:0/1914934580 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1cbc0555f0 con 0x7f1cc0149490 2023-11-09T03:48:07.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.680+0000 7f1cc7fb6640 1 -- 172.21.15.76:0/1914934580 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1cc00ce010 con 0x7f1cc0149490 2023-11-09T03:48:07.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.680+0000 7f1cb67fc640 1 -- 172.21.15.76:0/1914934580 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1cbc058c10 con 0x7f1cc0149490 2023-11-09T03:48:07.680 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:48:07.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.682+0000 7f1cc7fb6640 1 -- 172.21.15.76:0/1914934580 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1ca406aea0 msgr2=0x7f1ca406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:07.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.682+0000 7f1cc7fb6640 1 --2- 172.21.15.76:0/1914934580 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1ca406aea0 0x7f1ca406d360 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1cc006dc50 tx=0x7f1cb0009040 comp rx=0 tx=0).stop 2023-11-09T03:48:07.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.682+0000 7f1cc7fb6640 1 -- 172.21.15.76:0/1914934580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cc0149490 msgr2=0x7f1cc00af7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:07.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.682+0000 7f1cc7fb6640 1 --2- 172.21.15.76:0/1914934580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cc0149490 0x7f1cc00af7f0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc011e30 tx=0x7f1cbc00f580 comp rx=0 tx=0).stop 2023-11-09T03:48:07.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.682+0000 7f1cc7fb6640 1 -- 172.21.15.76:0/1914934580 shutdown_connections 2023-11-09T03:48:07.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.682+0000 7f1cc7fb6640 1 --2- 172.21.15.76:0/1914934580 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1ca406aea0 0x7f1ca406d360 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.682+0000 7f1cc7fb6640 1 --2- 172.21.15.76:0/1914934580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc00b1600 0x7f1cc006cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.682+0000 7f1cc7fb6640 1 --2- 172.21.15.76:0/1914934580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cc01547e0 0x7f1cc00b10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.682+0000 7f1cc7fb6640 1 --2- 172.21.15.76:0/1914934580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cc0149490 0x7f1cc00af7f0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.682+0000 7f1cc7fb6640 1 -- 172.21.15.76:0/1914934580 >> 172.21.15.76:0/1914934580 conn(0x7f1cc00ce950 msgr2=0x7f1cc00710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:07.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.682+0000 7f1cc7fb6640 1 -- 172.21.15.76:0/1914934580 shutdown_connections 2023-11-09T03:48:07.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.682+0000 7f1cc7fb6640 1 -- 172.21.15.76:0/1914934580 wait complete. 2023-11-09T03:48:07.692 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-09T03:48:07.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.797+0000 7f5cee07c640 1 Processor -- start 2023-11-09T03:48:07.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.797+0000 7f5cee07c640 1 -- start start 2023-11-09T03:48:07.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.798+0000 7f5cee07c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ce815b140 0x7f5ce806e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:07.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.798+0000 7f5cee07c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5ce8067090 0x7f5ce80b2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:07.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.798+0000 7f5cee07c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce80b24f0 con 0x7f5ce806cbd0 2023-11-09T03:48:07.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.798+0000 7f5cee07c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ce80ce220 con 0x7f5ce8067090 2023-11-09T03:48:07.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.798+0000 7f5cee07c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce80cdc60 con 0x7f5ce815b140 2023-11-09T03:48:07.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.798+0000 7f5ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5ce8067090 0x7f5ce80b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:07.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.798+0000 7f5ce77fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f5ce806cbd0 0x7f5ce806cfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:60034/0 (socket says 172.21.15.76:60034) 2023-11-09T03:48:07.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.798+0000 7f5ce77fe640 1 -- 172.21.15.76:0/185434656 learned_addr learned my addr 172.21.15.76:0/185434656 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:07.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.798+0000 7f5ce7fff640 1 -- 172.21.15.76:0/185434656 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ce815b140 msgr2=0x7f5ce806e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:07.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.798+0000 7f5ce6ffd640 1 --2- 172.21.15.76:0/185434656 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ce815b140 0x7f5ce806e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:07.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.798+0000 7f5ce7fff640 1 --2- 172.21.15.76:0/185434656 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ce815b140 0x7f5ce806e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.798+0000 7f5ce7fff640 1 -- 172.21.15.76:0/185434656 >> v1:172.21.15.76:6789/0 conn(0x7f5ce806cbd0 legacy=0x7f5ce806cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:07.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.798+0000 7f5ce67fc640 1 -- 172.21.15.76:0/185434656 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 46783473 0 0) 0x7f5ce80b24f0 con 0x7f5ce806cbd0 2023-11-09T03:48:07.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.798+0000 7f5ce7fff640 1 -- 172.21.15.76:0/185434656 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce80cdf40 con 0x7f5ce8067090 2023-11-09T03:48:07.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.798+0000 7f5ce6ffd640 1 --2- 172.21.15.76:0/185434656 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ce815b140 0x7f5ce806e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:07.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.799+0000 7f5ce7fff640 1 --2- 172.21.15.76:0/185434656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ce8067090 0x7f5ce80b2980 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009d90 tx=0x7f5ce002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0697ee17a9c794 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:07.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.799+0000 7f5ce67fc640 1 -- 172.21.15.76:0/185434656 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce002cbc0 con 0x7f5ce8067090 2023-11-09T03:48:07.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.799+0000 7f5ce67fc640 1 -- 172.21.15.76:0/185434656 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce002cd20 con 0x7f5ce8067090 2023-11-09T03:48:07.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.799+0000 7f5ce67fc640 1 -- 172.21.15.76:0/185434656 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0034660 con 0x7f5ce8067090 2023-11-09T03:48:07.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.800+0000 7f5cee07c640 1 -- 172.21.15.76:0/185434656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ce8067090 msgr2=0x7f5ce80b2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:07.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.800+0000 7f5cee07c640 1 --2- 172.21.15.76:0/185434656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ce8067090 0x7f5ce80b2980 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009d90 tx=0x7f5ce002be10 comp rx=0 tx=0).stop 2023-11-09T03:48:07.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.800+0000 7f5cee07c640 1 -- 172.21.15.76:0/185434656 shutdown_connections 2023-11-09T03:48:07.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.800+0000 7f5cee07c640 1 --2- 172.21.15.76:0/185434656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ce8067090 0x7f5ce80b2980 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.800+0000 7f5cee07c640 1 --2- 172.21.15.76:0/185434656 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ce815b140 0x7f5ce806e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.800+0000 7f5cee07c640 1 -- 172.21.15.76:0/185434656 >> 172.21.15.76:0/185434656 conn(0x7f5ce806a120 msgr2=0x7f5ce806a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:07.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.800+0000 7f5cee07c640 1 -- 172.21.15.76:0/185434656 shutdown_connections 2023-11-09T03:48:07.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.800+0000 7f5cee07c640 1 -- 172.21.15.76:0/185434656 wait complete. 2023-11-09T03:48:07.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.801+0000 7f5cee07c640 1 Processor -- start 2023-11-09T03:48:07.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.801+0000 7f5cee07c640 1 -- start start 2023-11-09T03:48:07.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.802+0000 7f5cee07c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ce8067090 0x7f5ce80c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:07.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.802+0000 7f5cee07c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ce806cbd0 0x7f5ce813de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:07.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.802+0000 7f5cee07c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ce815b140 0x7f5ce81e45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:07.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.802+0000 7f5cee07c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce815bfb0 con 0x7f5ce815b140 2023-11-09T03:48:07.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.802+0000 7f5cee07c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce80c23a0 con 0x7f5ce806cbd0 2023-11-09T03:48:07.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.802+0000 7f5cee07c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce814f7c0 con 0x7f5ce8067090 2023-11-09T03:48:07.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.802+0000 7f5ce7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ce815b140 0x7f5ce81e45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:07.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.802+0000 7f5ce7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ce815b140 0x7f5ce81e45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50812/0 (socket says 172.21.15.76:50812) 2023-11-09T03:48:07.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.802+0000 7f5ce7fff640 1 -- 172.21.15.76:0/3723220698 learned_addr learned my addr 172.21.15.76:0/3723220698 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:07.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.802+0000 7f5ce77fe640 1 --2- 172.21.15.76:0/3723220698 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ce8067090 0x7f5ce80c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:07.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.802+0000 7f5ce6ffd640 1 --2- 172.21.15.76:0/3723220698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ce806cbd0 0x7f5ce813de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:07.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.802+0000 7f5ce7fff640 1 -- 172.21.15.76:0/3723220698 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ce8067090 msgr2=0x7f5ce80c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:07.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.802+0000 7f5ce7fff640 1 --2- 172.21.15.76:0/3723220698 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ce8067090 0x7f5ce80c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.802+0000 7f5ce7fff640 1 -- 172.21.15.76:0/3723220698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ce806cbd0 msgr2=0x7f5ce813de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:07.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.802+0000 7f5ce7fff640 1 --2- 172.21.15.76:0/3723220698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ce806cbd0 0x7f5ce813de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.802+0000 7f5ce7fff640 1 -- 172.21.15.76:0/3723220698 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce0009a70 con 0x7f5ce815b140 2023-11-09T03:48:07.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.802+0000 7f5ce7fff640 1 --2- 172.21.15.76:0/3723220698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ce815b140 0x7f5ce81e45e0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009eb0 tx=0x7f5ce0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:07.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.802+0000 7f5ccffff640 1 -- 172.21.15.76:0/3723220698 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0040650 con 0x7f5ce815b140 2023-11-09T03:48:07.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.802+0000 7f5ccffff640 1 -- 172.21.15.76:0/3723220698 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce002ce90 con 0x7f5ce815b140 2023-11-09T03:48:07.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.802+0000 7f5cee07c640 1 -- 172.21.15.76:0/3723220698 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ce81e3860 con 0x7f5ce815b140 2023-11-09T03:48:07.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.802+0000 7f5ccffff640 1 -- 172.21.15.76:0/3723220698 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0040a10 con 0x7f5ce815b140 2023-11-09T03:48:07.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.802+0000 7f5cee07c640 1 -- 172.21.15.76:0/3723220698 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ce81e3de0 con 0x7f5ce815b140 2023-11-09T03:48:07.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.803+0000 7f5cee07c640 1 -- 172.21.15.76:0/3723220698 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ce8066100 con 0x7f5ce815b140 2023-11-09T03:48:07.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.804+0000 7f5ccffff640 1 -- 172.21.15.76:0/3723220698 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5ce0040b70 con 0x7f5ce815b140 2023-11-09T03:48:07.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.804+0000 7f5ccffff640 1 --2- 172.21.15.76:0/3723220698 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5cd406ae70 0x7f5cd406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:07.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.804+0000 7f5ccffff640 1 -- 172.21.15.76:0/3723220698 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7148+0+0 (secure 0 0 0) 0x7f5ce00acce0 con 0x7f5ce815b140 2023-11-09T03:48:07.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.804+0000 7f5ce77fe640 1 --2- 172.21.15.76:0/3723220698 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5cd406ae70 0x7f5cd406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:07.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.805+0000 7f5ce77fe640 1 --2- 172.21.15.76:0/3723220698 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5cd406ae70 0x7f5cd406d330 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8000f60 tx=0x7f5cd8007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:07.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.808+0000 7f5ccffff640 1 -- 172.21.15.76:0/3723220698 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ce00779a0 con 0x7f5ce815b140 2023-11-09T03:48:07.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.975+0000 7f5cee07c640 1 -- 172.21.15.76:0/3723220698 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7f5ce80c91a0 con 0x7f5ce815b140 2023-11-09T03:48:07.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.976+0000 7f5ccffff640 1 -- 172.21.15.76:0/3723220698 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v70) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5ce0039020 con 0x7f5ce815b140 2023-11-09T03:48:07.976 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:48:07.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.978+0000 7f5cee07c640 1 -- 172.21.15.76:0/3723220698 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5cd406ae70 msgr2=0x7f5cd406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:07.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.978+0000 7f5cee07c640 1 --2- 172.21.15.76:0/3723220698 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5cd406ae70 0x7f5cd406d330 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8000f60 tx=0x7f5cd8007b50 comp rx=0 tx=0).stop 2023-11-09T03:48:07.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.978+0000 7f5cee07c640 1 -- 172.21.15.76:0/3723220698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ce815b140 msgr2=0x7f5ce81e45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:07.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.978+0000 7f5cee07c640 1 --2- 172.21.15.76:0/3723220698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ce815b140 0x7f5ce81e45e0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009eb0 tx=0x7f5ce0004a40 comp rx=0 tx=0).stop 2023-11-09T03:48:07.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.978+0000 7f5cee07c640 1 -- 172.21.15.76:0/3723220698 shutdown_connections 2023-11-09T03:48:07.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.978+0000 7f5cee07c640 1 --2- 172.21.15.76:0/3723220698 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5cd406ae70 0x7f5cd406d330 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.978+0000 7f5cee07c640 1 --2- 172.21.15.76:0/3723220698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ce815b140 0x7f5ce81e45e0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.978+0000 7f5cee07c640 1 --2- 172.21.15.76:0/3723220698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ce806cbd0 0x7f5ce813de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.978+0000 7f5cee07c640 1 --2- 172.21.15.76:0/3723220698 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ce8067090 0x7f5ce80c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:07.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.978+0000 7f5cee07c640 1 -- 172.21.15.76:0/3723220698 >> 172.21.15.76:0/3723220698 conn(0x7f5ce806a120 msgr2=0x7f5ce8159c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:07.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.978+0000 7f5cee07c640 1 -- 172.21.15.76:0/3723220698 shutdown_connections 2023-11-09T03:48:07.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:07.978+0000 7f5cee07c640 1 -- 172.21.15.76:0/3723220698 wait complete. 2023-11-09T03:48:07.988 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-09T03:48:08.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.092+0000 7f27ebb5e640 1 Processor -- start 2023-11-09T03:48:08.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.092+0000 7f27ebb5e640 1 -- start start 2023-11-09T03:48:08.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.093+0000 7f27ebb5e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f27e4159c00 0x7f27e40cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:08.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.093+0000 7f27ebb5e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f27e40c48e0 0x7f27e40c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:08.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.093+0000 7f27ebb5e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e415aff0 con 0x7f27e40adac0 2023-11-09T03:48:08.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.093+0000 7f27ebb5e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e40c52d0 con 0x7f27e4159c00 2023-11-09T03:48:08.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.093+0000 7f27ebb5e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e415a3e0 con 0x7f27e40c48e0 2023-11-09T03:48:08.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.093+0000 7f27e98fa640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f27e4159c00 0x7f27e40cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:08.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.093+0000 7f27e98fa640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f27e4159c00 0x7f27e40cc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50824/0 (socket says 172.21.15.76:50824) 2023-11-09T03:48:08.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.093+0000 7f27e98fa640 1 -- 172.21.15.76:0/215066647 learned_addr learned my addr 172.21.15.76:0/215066647 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:08.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.093+0000 7f27e98fa640 1 -- 172.21.15.76:0/215066647 >> v2:172.21.15.113:3300/0 conn(0x7f27e40c48e0 msgr2=0x7f27e40c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:08.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.093+0000 7f27e90f9640 1 --2- 172.21.15.76:0/215066647 >> v2:172.21.15.113:3300/0 conn(0x7f27e40c48e0 0x7f27e40c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:08.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.093+0000 7f27e98fa640 1 --2- 172.21.15.76:0/215066647 >> v2:172.21.15.113:3300/0 conn(0x7f27e40c48e0 0x7f27e40c4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.093+0000 7f27e98fa640 1 -- 172.21.15.76:0/215066647 >> v1:172.21.15.113:6789/0 conn(0x7f27e40adac0 legacy=0x7f27e40adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:08.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.093+0000 7f27e98fa640 1 -- 172.21.15.76:0/215066647 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e415a6c0 con 0x7f27e4159c00 2023-11-09T03:48:08.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.093+0000 7f27e90f9640 1 --2- 172.21.15.76:0/215066647 >> v2:172.21.15.113:3300/0 conn(0x7f27e40c48e0 0x7f27e40c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:08.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.094+0000 7f27e98fa640 1 --2- 172.21.15.76:0/215066647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27e4159c00 0x7f27e40cc890 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f27e406ebb0 tx=0x7f27e002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=888ddc2010aaa7c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:08.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.094+0000 7f27e88f8640 1 -- 172.21.15.76:0/215066647 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e00317e0 con 0x7f27e4159c00 2023-11-09T03:48:08.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.094+0000 7f27e88f8640 1 -- 172.21.15.76:0/215066647 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e0031940 con 0x7f27e4159c00 2023-11-09T03:48:08.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.094+0000 7f27e88f8640 1 -- 172.21.15.76:0/215066647 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e0031cd0 con 0x7f27e4159c00 2023-11-09T03:48:08.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.094+0000 7f27ebb5e640 1 -- 172.21.15.76:0/215066647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27e4159c00 msgr2=0x7f27e40cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:08.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.094+0000 7f27ebb5e640 1 --2- 172.21.15.76:0/215066647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27e4159c00 0x7f27e40cc890 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f27e406ebb0 tx=0x7f27e002b880 comp rx=0 tx=0).stop 2023-11-09T03:48:08.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.095+0000 7f27ebb5e640 1 -- 172.21.15.76:0/215066647 shutdown_connections 2023-11-09T03:48:08.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.095+0000 7f27ebb5e640 1 --2- 172.21.15.76:0/215066647 >> v2:172.21.15.113:3300/0 conn(0x7f27e40c48e0 0x7f27e40c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.095+0000 7f27ebb5e640 1 --2- 172.21.15.76:0/215066647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27e4159c00 0x7f27e40cc890 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.095+0000 7f27ebb5e640 1 -- 172.21.15.76:0/215066647 >> 172.21.15.76:0/215066647 conn(0x7f27e40ce950 msgr2=0x7f27e40ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:08.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.095+0000 7f27ebb5e640 1 -- 172.21.15.76:0/215066647 shutdown_connections 2023-11-09T03:48:08.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.095+0000 7f27ebb5e640 1 -- 172.21.15.76:0/215066647 wait complete. 2023-11-09T03:48:08.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.096+0000 7f27ebb5e640 1 Processor -- start 2023-11-09T03:48:08.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.096+0000 7f27ebb5e640 1 -- start start 2023-11-09T03:48:08.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.096+0000 7f27ebb5e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27e40adac0 0x7f27e415bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:08.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.097+0000 7f27ebb5e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27e40c48e0 0x7f27e413d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:08.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.097+0000 7f27ebb5e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27e4159c00 0x7f27e40c6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:08.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.097+0000 7f27ebb5e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e414ad00 con 0x7f27e4159c00 2023-11-09T03:48:08.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.097+0000 7f27ebb5e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e414a0f0 con 0x7f27e40adac0 2023-11-09T03:48:08.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.097+0000 7f27ebb5e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e414a3d0 con 0x7f27e40c48e0 2023-11-09T03:48:08.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.097+0000 7f27ea0fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27e4159c00 0x7f27e40c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:08.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.097+0000 7f27ea0fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27e4159c00 0x7f27e40c6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50840/0 (socket says 172.21.15.76:50840) 2023-11-09T03:48:08.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.097+0000 7f27ea0fb640 1 -- 172.21.15.76:0/2742274968 learned_addr learned my addr 172.21.15.76:0/2742274968 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:08.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.097+0000 7f27e98fa640 1 --2- 172.21.15.76:0/2742274968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27e40adac0 0x7f27e415bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:08.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.097+0000 7f27e90f9640 1 --2- 172.21.15.76:0/2742274968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27e40c48e0 0x7f27e413d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:08.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.097+0000 7f27ea0fb640 1 -- 172.21.15.76:0/2742274968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27e40c48e0 msgr2=0x7f27e413d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:08.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.097+0000 7f27ea0fb640 1 --2- 172.21.15.76:0/2742274968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27e40c48e0 0x7f27e413d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.097+0000 7f27ea0fb640 1 -- 172.21.15.76:0/2742274968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27e40adac0 msgr2=0x7f27e415bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:08.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.097+0000 7f27ea0fb640 1 --2- 172.21.15.76:0/2742274968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27e40adac0 0x7f27e415bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.097+0000 7f27ea0fb640 1 -- 172.21.15.76:0/2742274968 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e00094d0 con 0x7f27e4159c00 2023-11-09T03:48:08.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.097+0000 7f27ea0fb640 1 --2- 172.21.15.76:0/2742274968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27e4159c00 0x7f27e40c6410 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00c780 tx=0x7f27dc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:08.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.097+0000 7f27da7fc640 1 -- 172.21.15.76:0/2742274968 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc004b60 con 0x7f27e4159c00 2023-11-09T03:48:08.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.097+0000 7f27da7fc640 1 -- 172.21.15.76:0/2742274968 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27dc00ce90 con 0x7f27e4159c00 2023-11-09T03:48:08.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.097+0000 7f27da7fc640 1 -- 172.21.15.76:0/2742274968 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc00a850 con 0x7f27e4159c00 2023-11-09T03:48:08.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.097+0000 7f27ebb5e640 1 -- 172.21.15.76:0/2742274968 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e414a6b0 con 0x7f27e4159c00 2023-11-09T03:48:08.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.097+0000 7f27ebb5e640 1 -- 172.21.15.76:0/2742274968 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e40c8770 con 0x7f27e4159c00 2023-11-09T03:48:08.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.098+0000 7f27ebb5e640 1 -- 172.21.15.76:0/2742274968 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e414aeb0 con 0x7f27e4159c00 2023-11-09T03:48:08.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.099+0000 7f27da7fc640 1 -- 172.21.15.76:0/2742274968 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f27dc00a9b0 con 0x7f27e4159c00 2023-11-09T03:48:08.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.099+0000 7f27da7fc640 1 --2- 172.21.15.76:0/2742274968 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f27c806ae70 0x7f27c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:08.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.099+0000 7f27da7fc640 1 -- 172.21.15.76:0/2742274968 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7148+0+0 (secure 0 0 0) 0x7f27dc089d20 con 0x7f27e4159c00 2023-11-09T03:48:08.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.099+0000 7f27e98fa640 1 --2- 172.21.15.76:0/2742274968 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f27c806ae70 0x7f27c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:08.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.100+0000 7f27e98fa640 1 --2- 172.21.15.76:0/2742274968 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f27c806ae70 0x7f27c806d330 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f27e0036000 tx=0x7f27e003d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:08.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.103+0000 7f27da7fc640 1 -- 172.21.15.76:0/2742274968 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f27e414aeb0 con 0x7f27e4159c00 2023-11-09T03:48:08.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.269+0000 7f27ebb5e640 1 -- 172.21.15.76:0/2742274968 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7f27e4074290 con 0x7f27e4159c00 2023-11-09T03:48:08.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.269+0000 7f27da7fc640 1 -- 172.21.15.76:0/2742274968 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v70) v1 ==== 107+0+11 (secure 0 0 0) 0x7f27dc054830 con 0x7f27e4159c00 2023-11-09T03:48:08.269 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:48:08.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.271+0000 7f27ebb5e640 1 -- 172.21.15.76:0/2742274968 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f27c806ae70 msgr2=0x7f27c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:08.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.271+0000 7f27ebb5e640 1 --2- 172.21.15.76:0/2742274968 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f27c806ae70 0x7f27c806d330 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f27e0036000 tx=0x7f27e003d040 comp rx=0 tx=0).stop 2023-11-09T03:48:08.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.271+0000 7f27ebb5e640 1 -- 172.21.15.76:0/2742274968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27e4159c00 msgr2=0x7f27e40c6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:08.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.271+0000 7f27ebb5e640 1 --2- 172.21.15.76:0/2742274968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27e4159c00 0x7f27e40c6410 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00c780 tx=0x7f27dc00cc40 comp rx=0 tx=0).stop 2023-11-09T03:48:08.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.271+0000 7f27ebb5e640 1 -- 172.21.15.76:0/2742274968 shutdown_connections 2023-11-09T03:48:08.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.271+0000 7f27ebb5e640 1 --2- 172.21.15.76:0/2742274968 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f27c806ae70 0x7f27c806d330 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.271+0000 7f27ebb5e640 1 --2- 172.21.15.76:0/2742274968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27e4159c00 0x7f27e40c6410 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.271+0000 7f27ebb5e640 1 --2- 172.21.15.76:0/2742274968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27e40c48e0 0x7f27e413d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.271+0000 7f27ebb5e640 1 --2- 172.21.15.76:0/2742274968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27e40adac0 0x7f27e415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.271+0000 7f27ebb5e640 1 -- 172.21.15.76:0/2742274968 >> 172.21.15.76:0/2742274968 conn(0x7f27e40ce950 msgr2=0x7f27e40cb340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:08.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.271+0000 7f27ebb5e640 1 -- 172.21.15.76:0/2742274968 shutdown_connections 2023-11-09T03:48:08.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.271+0000 7f27ebb5e640 1 -- 172.21.15.76:0/2742274968 wait complete. 2023-11-09T03:48:08.281 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:48:08.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.383+0000 7fbdc8bd0640 1 Processor -- start 2023-11-09T03:48:08.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.383+0000 7fbdc8bd0640 1 -- start start 2023-11-09T03:48:08.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.383+0000 7fbdc8bd0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdc40cde60 0x7fbdc415d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:08.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.383+0000 7fbdc8bd0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdc415daa0 0x7fbdc4157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:08.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.383+0000 7fbdc8bd0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbdc40ce240 0x7fbdc4157540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:08.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.383+0000 7fbdc8bd0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc415e5c0 con 0x7fbdc40ce240 2023-11-09T03:48:08.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.383+0000 7fbdc8bd0640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc4157a80 con 0x7fbdc40cde60 2023-11-09T03:48:08.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.383+0000 7fbdc8bd0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc415e000 con 0x7fbdc415daa0 2023-11-09T03:48:08.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.384+0000 7fbdc2d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbdc40ce240 0x7fbdc4157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:08.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.384+0000 7fbdc2d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbdc40ce240 0x7fbdc4157540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50844/0 (socket says 172.21.15.76:50844) 2023-11-09T03:48:08.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.384+0000 7fbdc2d9d640 1 -- 172.21.15.76:0/2988181475 learned_addr learned my addr 172.21.15.76:0/2988181475 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:08.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.384+0000 7fbdc259c640 1 --2- 172.21.15.76:0/2988181475 >> v2:172.21.15.113:3300/0 conn(0x7fbdc40cde60 0x7fbdc415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:08.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.384+0000 7fbdbbfff640 1 --2- 172.21.15.76:0/2988181475 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdc415daa0 0x7fbdc4157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:08.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.384+0000 7fbdc2d9d640 1 -- 172.21.15.76:0/2988181475 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdc415daa0 msgr2=0x7fbdc4157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:08.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.384+0000 7fbdc2d9d640 1 --2- 172.21.15.76:0/2988181475 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdc415daa0 0x7fbdc4157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.384+0000 7fbdc2d9d640 1 -- 172.21.15.76:0/2988181475 >> v2:172.21.15.113:3300/0 conn(0x7fbdc40cde60 msgr2=0x7fbdc415d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:08.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.384+0000 7fbdc2d9d640 1 --2- 172.21.15.76:0/2988181475 >> v2:172.21.15.113:3300/0 conn(0x7fbdc40cde60 0x7fbdc415d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.384+0000 7fbdc2d9d640 1 -- 172.21.15.76:0/2988181475 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc415e2e0 con 0x7fbdc40ce240 2023-11-09T03:48:08.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.385+0000 7fbdc2d9d640 1 --2- 172.21.15.76:0/2988181475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdc40ce240 0x7fbdc4157540 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40b5550 tx=0x7fbdbc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2192a7d3c06f298 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:08.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.385+0000 7fbdc1d9b640 1 -- 172.21.15.76:0/2988181475 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc02ca70 con 0x7fbdc40ce240 2023-11-09T03:48:08.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.385+0000 7fbdc1d9b640 1 -- 172.21.15.76:0/2988181475 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdbc02cbd0 con 0x7fbdc40ce240 2023-11-09T03:48:08.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.385+0000 7fbdc1d9b640 1 -- 172.21.15.76:0/2988181475 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc0343f0 con 0x7fbdc40ce240 2023-11-09T03:48:08.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.385+0000 7fbdc8bd0640 1 -- 172.21.15.76:0/2988181475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdc40ce240 msgr2=0x7fbdc4157540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:08.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.385+0000 7fbdc8bd0640 1 --2- 172.21.15.76:0/2988181475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdc40ce240 0x7fbdc4157540 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40b5550 tx=0x7fbdbc02bed0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.385+0000 7fbdc8bd0640 1 -- 172.21.15.76:0/2988181475 shutdown_connections 2023-11-09T03:48:08.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.385+0000 7fbdc8bd0640 1 --2- 172.21.15.76:0/2988181475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdc40ce240 0x7fbdc4157540 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.385+0000 7fbdc8bd0640 1 --2- 172.21.15.76:0/2988181475 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdc415daa0 0x7fbdc4157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.385+0000 7fbdc8bd0640 1 --2- 172.21.15.76:0/2988181475 >> v2:172.21.15.113:3300/0 conn(0x7fbdc40cde60 0x7fbdc415d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.385+0000 7fbdc8bd0640 1 -- 172.21.15.76:0/2988181475 >> 172.21.15.76:0/2988181475 conn(0x7fbdc4071f30 msgr2=0x7fbdc40cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:08.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.385+0000 7fbdc8bd0640 1 -- 172.21.15.76:0/2988181475 shutdown_connections 2023-11-09T03:48:08.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.385+0000 7fbdc8bd0640 1 -- 172.21.15.76:0/2988181475 wait complete. 2023-11-09T03:48:08.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.387+0000 7fbdc8bd0640 1 Processor -- start 2023-11-09T03:48:08.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.387+0000 7fbdc8bd0640 1 -- start start 2023-11-09T03:48:08.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.387+0000 7fbdc8bd0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdc40cde60 0x7fbdc41f1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:08.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.387+0000 7fbdc8bd0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdc40ce240 0x7fbdc40c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:08.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.387+0000 7fbdc8bd0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdc415daa0 0x7fbdc40c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:08.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.387+0000 7fbdc8bd0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc40c76d0 con 0x7fbdc40cde60 2023-11-09T03:48:08.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.387+0000 7fbdc8bd0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc40c79b0 con 0x7fbdc415daa0 2023-11-09T03:48:08.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.387+0000 7fbdc8bd0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc40c7c90 con 0x7fbdc40ce240 2023-11-09T03:48:08.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.387+0000 7fbdc259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdc40cde60 0x7fbdc41f1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:08.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.387+0000 7fbdc259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdc40cde60 0x7fbdc41f1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50856/0 (socket says 172.21.15.76:50856) 2023-11-09T03:48:08.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.387+0000 7fbdc259c640 1 -- 172.21.15.76:0/3615061059 learned_addr learned my addr 172.21.15.76:0/3615061059 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:08.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.388+0000 7fbdbbfff640 1 --2- 172.21.15.76:0/3615061059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdc40ce240 0x7fbdc40c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:08.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.388+0000 7fbdc2d9d640 1 --2- 172.21.15.76:0/3615061059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdc415daa0 0x7fbdc40c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:08.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.388+0000 7fbdc259c640 1 -- 172.21.15.76:0/3615061059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdc40ce240 msgr2=0x7fbdc40c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:08.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.388+0000 7fbdc259c640 1 --2- 172.21.15.76:0/3615061059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdc40ce240 0x7fbdc40c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.388+0000 7fbdc259c640 1 -- 172.21.15.76:0/3615061059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdc415daa0 msgr2=0x7fbdc40c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:08.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.388+0000 7fbdc259c640 1 --2- 172.21.15.76:0/3615061059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdc415daa0 0x7fbdc40c9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.388+0000 7fbdc259c640 1 -- 172.21.15.76:0/3615061059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdbc009b30 con 0x7fbdc40cde60 2023-11-09T03:48:08.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.388+0000 7fbdc259c640 1 --2- 172.21.15.76:0/3615061059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdc40cde60 0x7fbdc41f1e00 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400d800 tx=0x7fbdb400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:08.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.388+0000 7fbdbaffd640 1 -- 172.21.15.76:0/3615061059 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb40096a0 con 0x7fbdc40cde60 2023-11-09T03:48:08.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.388+0000 7fbdbaffd640 1 -- 172.21.15.76:0/3615061059 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb4009800 con 0x7fbdc40cde60 2023-11-09T03:48:08.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.388+0000 7fbdc8bd0640 1 -- 172.21.15.76:0/3615061059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc40c7f70 con 0x7fbdc40cde60 2023-11-09T03:48:08.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.388+0000 7fbdbaffd640 1 -- 172.21.15.76:0/3615061059 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4009b90 con 0x7fbdc40cde60 2023-11-09T03:48:08.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.388+0000 7fbdc8bd0640 1 -- 172.21.15.76:0/3615061059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc41f2340 con 0x7fbdc40cde60 2023-11-09T03:48:08.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.389+0000 7fbdc8bd0640 1 -- 172.21.15.76:0/3615061059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc4074290 con 0x7fbdc40cde60 2023-11-09T03:48:08.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.389+0000 7fbdbaffd640 1 -- 172.21.15.76:0/3615061059 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbdb4009cf0 con 0x7fbdc40cde60 2023-11-09T03:48:08.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.390+0000 7fbdbaffd640 1 --2- 172.21.15.76:0/3615061059 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbda406ae70 0x7fbda406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:08.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.390+0000 7fbdbaffd640 1 -- 172.21.15.76:0/3615061059 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7148+0+0 (secure 0 0 0) 0x7fbdb408a030 con 0x7fbdc40cde60 2023-11-09T03:48:08.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.390+0000 7fbdbbfff640 1 --2- 172.21.15.76:0/3615061059 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbda406ae70 0x7fbda406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:08.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.391+0000 7fbdbbfff640 1 --2- 172.21.15.76:0/3615061059 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbda406ae70 0x7fbda406d330 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbdac006fd0 tx=0x7fbdac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:08.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.394+0000 7fbdbaffd640 1 -- 172.21.15.76:0/3615061059 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbdb4054b40 con 0x7fbdc40cde60 2023-11-09T03:48:08.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.598+0000 7fbdc8bd0640 1 -- 172.21.15.76:0/3615061059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbdc40b8530 con 0x7fbdc40cde60 2023-11-09T03:48:08.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.598+0000 7fbdbaffd640 1 -- 172.21.15.76:0/3615061059 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fbdb400b440 con 0x7fbdc40cde60 2023-11-09T03:48:08.598 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:08.598 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2800733670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2800733670},{"type":"v1","addr":"172.21.15.113:6836","nonce":2800733670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":5791,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1301905851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1301905851},{"type":"v1","addr":"172.21.15.113:6837","nonce":1301905851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":6498,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/968739949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":968739949},{"type":"v1","addr":"172.21.15.76:6833","nonce":968739949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:47:51.114132+0000","modified":"2023-11-09T03:47:53.114512+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6501},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6501":{"gid":6501,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/1576689316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1576689316},{"type":"v1","addr":"172.21.15.76:6835","nonce":1576689316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-09T03:48:08.599 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 29 2023-11-09T03:48:08.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.600+0000 7fbdc8bd0640 1 -- 172.21.15.76:0/3615061059 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbda406ae70 msgr2=0x7fbda406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:08.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.600+0000 7fbdc8bd0640 1 --2- 172.21.15.76:0/3615061059 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbda406ae70 0x7fbda406d330 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbdac006fd0 tx=0x7fbdac008040 comp rx=0 tx=0).stop 2023-11-09T03:48:08.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.600+0000 7fbdc8bd0640 1 -- 172.21.15.76:0/3615061059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdc40cde60 msgr2=0x7fbdc41f1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:08.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.600+0000 7fbdc8bd0640 1 --2- 172.21.15.76:0/3615061059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdc40cde60 0x7fbdc41f1e00 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400d800 tx=0x7fbdb400dcc0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.600+0000 7fbdc8bd0640 1 -- 172.21.15.76:0/3615061059 shutdown_connections 2023-11-09T03:48:08.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.600+0000 7fbdc8bd0640 1 --2- 172.21.15.76:0/3615061059 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbda406ae70 0x7fbda406d330 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.600+0000 7fbdc8bd0640 1 --2- 172.21.15.76:0/3615061059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdc415daa0 0x7fbdc40c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.600+0000 7fbdc8bd0640 1 --2- 172.21.15.76:0/3615061059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdc40ce240 0x7fbdc40c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.600+0000 7fbdc8bd0640 1 --2- 172.21.15.76:0/3615061059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdc40cde60 0x7fbdc41f1e00 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.600+0000 7fbdc8bd0640 1 -- 172.21.15.76:0/3615061059 >> 172.21.15.76:0/3615061059 conn(0x7fbdc4071f30 msgr2=0x7fbdc40b5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:08.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.600+0000 7fbdc8bd0640 1 -- 172.21.15.76:0/3615061059 shutdown_connections 2023-11-09T03:48:08.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.600+0000 7fbdc8bd0640 1 -- 172.21.15.76:0/3615061059 wait complete. 2023-11-09T03:48:08.611 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:48:08.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.711+0000 7f510ce07640 1 Processor -- start 2023-11-09T03:48:08.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.711+0000 7f510ce07640 1 -- start start 2023-11-09T03:48:08.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.711+0000 7f510ce07640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5108159c70 0x7f51080ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:08.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.711+0000 7f510ce07640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f51080b0ae0 0x7f510815d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:08.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.711+0000 7f510ce07640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51080b4110 con 0x7f510806db20 2023-11-09T03:48:08.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.711+0000 7f510ce07640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51080b49b0 con 0x7f51080b0ae0 2023-11-09T03:48:08.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.711+0000 7f510ce07640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51080b43f0 con 0x7f5108159c70 2023-11-09T03:48:08.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.711+0000 7f50fffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f510806db20 0x7f51080b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:60050/0 (socket says 172.21.15.76:60050) 2023-11-09T03:48:08.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.711+0000 7f50fffff640 1 -- 172.21.15.76:0/2159018832 learned_addr learned my addr 172.21.15.76:0/2159018832 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:08.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.712+0000 7f5105d9b640 1 -- 172.21.15.76:0/2159018832 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3815210229 0 0) 0x7f51080b4110 con 0x7f510806db20 2023-11-09T03:48:08.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.712+0000 7f5106d9d640 1 --2- 172.21.15.76:0/2159018832 >> v2:172.21.15.113:3300/0 conn(0x7f51080b0ae0 0x7f510815d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:08.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.712+0000 7f510659c640 1 --2- 172.21.15.76:0/2159018832 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5108159c70 0x7f51080ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:08.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.712+0000 7f5105d9b640 1 -- 172.21.15.76:0/2159018832 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50f4005730 con 0x7f510806db20 2023-11-09T03:48:08.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.712+0000 7f5105d9b640 1 -- 172.21.15.76:0/2159018832 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2504167989 0 0) 0x7f50f4005730 con 0x7f510806db20 2023-11-09T03:48:08.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.712+0000 7f5105d9b640 1 -- 172.21.15.76:0/2159018832 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5108159c70 msgr2=0x7f51080ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:08.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.712+0000 7f5105d9b640 1 --2- 172.21.15.76:0/2159018832 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5108159c70 0x7f51080ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.712+0000 7f5105d9b640 1 -- 172.21.15.76:0/2159018832 >> v2:172.21.15.113:3300/0 conn(0x7f51080b0ae0 msgr2=0x7f510815d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:08.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.712+0000 7f5105d9b640 1 --2- 172.21.15.76:0/2159018832 >> v2:172.21.15.113:3300/0 conn(0x7f51080b0ae0 0x7f510815d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.712+0000 7f5105d9b640 1 -- 172.21.15.76:0/2159018832 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51080b46d0 con 0x7f510806db20 2023-11-09T03:48:08.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.712+0000 7f5105d9b640 1 -- 172.21.15.76:0/2159018832 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f50f0001670 con 0x7f510806db20 2023-11-09T03:48:08.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.712+0000 7f5105d9b640 1 -- 172.21.15.76:0/2159018832 >> v1:172.21.15.76:6789/0 conn(0x7f510806db20 legacy=0x7f51080b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:08.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.712+0000 7f5105d9b640 1 --2- 172.21.15.76:0/2159018832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50f4005b80 0x7f50f4005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:08.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.712+0000 7f5105d9b640 1 --2- 172.21.15.76:0/2159018832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50f40069d0 0x7f50f4006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:08.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.712+0000 7f5105d9b640 1 --2- 172.21.15.76:0/2159018832 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5108159c70 0x7f50f4007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:08.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.712+0000 7f5105d9b640 1 -- 172.21.15.76:0/2159018832 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51080b49b0 con 0x7f50f40069d0 2023-11-09T03:48:08.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.712+0000 7f5105d9b640 1 -- 172.21.15.76:0/2159018832 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51080b43f0 con 0x7f50f4005b80 2023-11-09T03:48:08.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.712+0000 7f5105d9b640 1 -- 172.21.15.76:0/2159018832 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f400dc40 con 0x7f5108159c70 2023-11-09T03:48:08.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.712+0000 7f50fffff640 1 --2- 172.21.15.76:0/2159018832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50f40069d0 0x7f50f4006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:08.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.712+0000 7f510659c640 1 --2- 172.21.15.76:0/2159018832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50f4005b80 0x7f50f4005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:08.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.712+0000 7f5106d9d640 1 --2- 172.21.15.76:0/2159018832 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5108159c70 0x7f50f4007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:08.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.712+0000 7f50fffff640 1 -- 172.21.15.76:0/2159018832 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5108159c70 msgr2=0x7f50f4007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:08.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.712+0000 7f50fffff640 1 --2- 172.21.15.76:0/2159018832 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5108159c70 0x7f50f4007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.712+0000 7f50fffff640 1 -- 172.21.15.76:0/2159018832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50f4005b80 msgr2=0x7f50f4005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:08.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.712+0000 7f50fffff640 1 --2- 172.21.15.76:0/2159018832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50f4005b80 0x7f50f4005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.712+0000 7f50fffff640 1 -- 172.21.15.76:0/2159018832 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f4005730 con 0x7f50f40069d0 2023-11-09T03:48:08.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.713+0000 7f50fffff640 1 --2- 172.21.15.76:0/2159018832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50f40069d0 0x7f50f4006e50 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f50f0001050 tx=0x7f50f4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc1808374b321b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:08.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.713+0000 7f5105d9b640 1 -- 172.21.15.76:0/2159018832 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f0005ad0 con 0x7f50f40069d0 2023-11-09T03:48:08.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.713+0000 7f5105d9b640 1 -- 172.21.15.76:0/2159018832 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f0005c30 con 0x7f50f40069d0 2023-11-09T03:48:08.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.713+0000 7f5105d9b640 1 -- 172.21.15.76:0/2159018832 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f002c870 con 0x7f50f40069d0 2023-11-09T03:48:08.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.714+0000 7f510ce07640 1 -- 172.21.15.76:0/2159018832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50f40069d0 msgr2=0x7f50f4006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:08.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.714+0000 7f510ce07640 1 --2- 172.21.15.76:0/2159018832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50f40069d0 0x7f50f4006e50 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f50f0001050 tx=0x7f50f4007a00 comp rx=0 tx=0).stop 2023-11-09T03:48:08.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.714+0000 7f5106d9d640 1 -- 172.21.15.76:0/2159018832 reap_dead start 2023-11-09T03:48:08.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.714+0000 7f510ce07640 1 -- 172.21.15.76:0/2159018832 shutdown_connections 2023-11-09T03:48:08.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.714+0000 7f510ce07640 1 -- 172.21.15.76:0/2159018832 >> 172.21.15.76:0/2159018832 conn(0x7f51080ce950 msgr2=0x7f51080ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:08.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.714+0000 7f510ce07640 1 -- 172.21.15.76:0/2159018832 shutdown_connections 2023-11-09T03:48:08.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.714+0000 7f510ce07640 1 -- 172.21.15.76:0/2159018832 wait complete. 2023-11-09T03:48:08.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.715+0000 7f510ce07640 1 Processor -- start 2023-11-09T03:48:08.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.715+0000 7f510ce07640 1 -- start start 2023-11-09T03:48:08.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.716+0000 7f510ce07640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50f40069d0 0x7f51080c4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:08.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.716+0000 7f510ce07640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5108140300 0x7f51080c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:08.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.716+0000 7f510ce07640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f510815a290 0x7f510815a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:08.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.716+0000 7f510ce07640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51080c6f20 con 0x7f510815a290 2023-11-09T03:48:08.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.716+0000 7f510ce07640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510815ac50 con 0x7f50f40069d0 2023-11-09T03:48:08.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.716+0000 7f510ce07640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51080c7d20 con 0x7f5108140300 2023-11-09T03:48:08.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.716+0000 7f5106d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f510815a290 0x7f510815a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:08.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.716+0000 7f5106d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f510815a290 0x7f510815a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50886/0 (socket says 172.21.15.76:50886) 2023-11-09T03:48:08.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.716+0000 7f5106d9d640 1 -- 172.21.15.76:0/148220194 learned_addr learned my addr 172.21.15.76:0/148220194 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:08.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.716+0000 7f510659c640 1 --2- 172.21.15.76:0/148220194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50f40069d0 0x7f51080c4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:08.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.716+0000 7f50fffff640 1 --2- 172.21.15.76:0/148220194 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5108140300 0x7f51080c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:08.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.716+0000 7f5106d9d640 1 -- 172.21.15.76:0/148220194 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5108140300 msgr2=0x7f51080c69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:08.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.716+0000 7f5106d9d640 1 --2- 172.21.15.76:0/148220194 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5108140300 0x7f51080c69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.716+0000 7f5106d9d640 1 -- 172.21.15.76:0/148220194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50f40069d0 msgr2=0x7f51080c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:08.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.716+0000 7f5106d9d640 1 --2- 172.21.15.76:0/148220194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50f40069d0 0x7f51080c4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.716+0000 7f5106d9d640 1 -- 172.21.15.76:0/148220194 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5108140780 con 0x7f510815a290 2023-11-09T03:48:08.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.716+0000 7f5106d9d640 1 --2- 172.21.15.76:0/148220194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f510815a290 0x7f510815a710 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f51080b1390 tx=0x7f5100007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:08.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.717+0000 7f50feffd640 1 -- 172.21.15.76:0/148220194 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51000046a0 con 0x7f510815a290 2023-11-09T03:48:08.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.717+0000 7f50feffd640 1 -- 172.21.15.76:0/148220194 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5100004800 con 0x7f510815a290 2023-11-09T03:48:08.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.717+0000 7f510ce07640 1 -- 172.21.15.76:0/148220194 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51080c7110 con 0x7f510815a290 2023-11-09T03:48:08.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.717+0000 7f50feffd640 1 -- 172.21.15.76:0/148220194 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5100015660 con 0x7f510815a290 2023-11-09T03:48:08.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.717+0000 7f510ce07640 1 -- 172.21.15.76:0/148220194 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51080c76d0 con 0x7f510815a290 2023-11-09T03:48:08.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.718+0000 7f510ce07640 1 -- 172.21.15.76:0/148220194 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5108074290 con 0x7f510815a290 2023-11-09T03:48:08.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.718+0000 7f50feffd640 1 -- 172.21.15.76:0/148220194 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f51000157c0 con 0x7f510815a290 2023-11-09T03:48:08.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.718+0000 7f50feffd640 1 --2- 172.21.15.76:0/148220194 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f50e806ae70 0x7f50e806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:08.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.718+0000 7f50feffd640 1 -- 172.21.15.76:0/148220194 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7148+0+0 (secure 0 0 0) 0x7f510008a480 con 0x7f510815a290 2023-11-09T03:48:08.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.719+0000 7f510659c640 1 --2- 172.21.15.76:0/148220194 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f50e806ae70 0x7f50e806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:08.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.719+0000 7f510659c640 1 --2- 172.21.15.76:0/148220194 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f50e806ae70 0x7f50e806d330 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f50f8006fd0 tx=0x7f50f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:08.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.723+0000 7f50feffd640 1 -- 172.21.15.76:0/148220194 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5100054f90 con 0x7f510815a290 2023-11-09T03:48:08.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.890+0000 7f510ce07640 1 -- 172.21.15.76:0/148220194 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f510806d3f0 con 0x7f510815a290 2023-11-09T03:48:08.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.892+0000 7f50feffd640 1 -- 172.21.15.76:0/148220194 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+22089 (secure 0 0 0) 0x7f51000585b0 con 0x7f510815a290 2023-11-09T03:48:08.891 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:08.892 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":70,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:48:02.436030+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:47:49.630346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-09T03:47:50.082207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-09T03:47:59.280601+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-09T03:47:59.742039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":12,"snaps":[{"begin":1,"length":1}]}],"new_removed_snaps":[{"pool":12,"snaps":[{"begin":1,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:48:08.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.893+0000 7f510ce07640 1 -- 172.21.15.76:0/148220194 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f50e806ae70 msgr2=0x7f50e806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:08.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.893+0000 7f510ce07640 1 --2- 172.21.15.76:0/148220194 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f50e806ae70 0x7f50e806d330 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f50f8006fd0 tx=0x7f50f8008040 comp rx=0 tx=0).stop 2023-11-09T03:48:08.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.893+0000 7f510ce07640 1 -- 172.21.15.76:0/148220194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f510815a290 msgr2=0x7f510815a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:08.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.893+0000 7f510ce07640 1 --2- 172.21.15.76:0/148220194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f510815a290 0x7f510815a710 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f51080b1390 tx=0x7f5100007700 comp rx=0 tx=0).stop 2023-11-09T03:48:08.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.893+0000 7f510ce07640 1 -- 172.21.15.76:0/148220194 shutdown_connections 2023-11-09T03:48:08.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.893+0000 7f510ce07640 1 --2- 172.21.15.76:0/148220194 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f50e806ae70 0x7f50e806d330 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.893+0000 7f510ce07640 1 --2- 172.21.15.76:0/148220194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f510815a290 0x7f510815a710 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.893+0000 7f510ce07640 1 --2- 172.21.15.76:0/148220194 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5108140300 0x7f51080c69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.893+0000 7f510ce07640 1 --2- 172.21.15.76:0/148220194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50f40069d0 0x7f51080c4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:08.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.893+0000 7f510ce07640 1 -- 172.21.15.76:0/148220194 >> 172.21.15.76:0/148220194 conn(0x7f51080ce950 msgr2=0x7f51080ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:08.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.893+0000 7f510ce07640 1 -- 172.21.15.76:0/148220194 shutdown_connections 2023-11-09T03:48:08.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:08.894+0000 7f510ce07640 1 -- 172.21.15.76:0/148220194 wait complete. 2023-11-09T03:48:08.904 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-09T03:48:08.904 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:48:09.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.005+0000 7fea8fafc640 1 Processor -- start 2023-11-09T03:48:09.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.005+0000 7fea8fafc640 1 -- start start 2023-11-09T03:48:09.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.006+0000 7fea8fafc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fea880b0ae0 0x7fea880ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.006+0000 7fea8fafc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea8806db20 0x7fea8815d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.006+0000 7fea8fafc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea880b4110 con 0x7fea88159c70 2023-11-09T03:48:09.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.006+0000 7fea8fafc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea880b49b0 con 0x7fea880b0ae0 2023-11-09T03:48:09.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.006+0000 7fea8fafc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea880b43f0 con 0x7fea8806db20 2023-11-09T03:48:09.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.006+0000 7fea8d097640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fea880b0ae0 0x7fea880ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.006+0000 7fea8d898640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fea88159c70 0x7fea880b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:60058/0 (socket says 172.21.15.76:60058) 2023-11-09T03:48:09.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.006+0000 7fea8d097640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fea880b0ae0 0x7fea880ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50890/0 (socket says 172.21.15.76:50890) 2023-11-09T03:48:09.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.006+0000 7fea8d898640 1 -- 172.21.15.76:0/1239340851 learned_addr learned my addr 172.21.15.76:0/1239340851 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:09.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.006+0000 7fea8e099640 1 --2- 172.21.15.76:0/1239340851 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea8806db20 0x7fea8815d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.006+0000 7fea8d097640 1 -- 172.21.15.76:0/1239340851 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea8806db20 msgr2=0x7fea8815d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:09.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.006+0000 7fea8d097640 1 --2- 172.21.15.76:0/1239340851 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea8806db20 0x7fea8815d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.006+0000 7fea8d097640 1 -- 172.21.15.76:0/1239340851 >> v1:172.21.15.76:6789/0 conn(0x7fea88159c70 legacy=0x7fea880b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:09.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.006+0000 7fea8d097640 1 -- 172.21.15.76:0/1239340851 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea880b46d0 con 0x7fea880b0ae0 2023-11-09T03:48:09.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.006+0000 7fea8c896640 1 -- 172.21.15.76:0/1239340851 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2611692010 0 0) 0x7fea880b4110 con 0x7fea88159c70 2023-11-09T03:48:09.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.006+0000 7fea8e099640 1 --2- 172.21.15.76:0/1239340851 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea8806db20 0x7fea8815d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:09.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.007+0000 7fea8d097640 1 --2- 172.21.15.76:0/1239340851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea880b0ae0 0x7fea880ce370 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fea780097a0 tx=0x7fea7802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5cc8712b64192c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:09.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.007+0000 7fea8c896640 1 -- 172.21.15.76:0/1239340851 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea78031830 con 0x7fea880b0ae0 2023-11-09T03:48:09.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.007+0000 7fea8c896640 1 -- 172.21.15.76:0/1239340851 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea78031990 con 0x7fea880b0ae0 2023-11-09T03:48:09.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.007+0000 7fea8c896640 1 -- 172.21.15.76:0/1239340851 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea78031d20 con 0x7fea880b0ae0 2023-11-09T03:48:09.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.008+0000 7fea8fafc640 1 -- 172.21.15.76:0/1239340851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea880b0ae0 msgr2=0x7fea880ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:09.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.008+0000 7fea8fafc640 1 --2- 172.21.15.76:0/1239340851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea880b0ae0 0x7fea880ce370 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fea780097a0 tx=0x7fea7802b8d0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.008+0000 7fea8fafc640 1 -- 172.21.15.76:0/1239340851 shutdown_connections 2023-11-09T03:48:09.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.008+0000 7fea8fafc640 1 --2- 172.21.15.76:0/1239340851 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea8806db20 0x7fea8815d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.008+0000 7fea8fafc640 1 --2- 172.21.15.76:0/1239340851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea880b0ae0 0x7fea880ce370 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.008+0000 7fea8fafc640 1 -- 172.21.15.76:0/1239340851 >> 172.21.15.76:0/1239340851 conn(0x7fea880ce950 msgr2=0x7fea880ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:09.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.008+0000 7fea8fafc640 1 -- 172.21.15.76:0/1239340851 shutdown_connections 2023-11-09T03:48:09.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.008+0000 7fea8fafc640 1 -- 172.21.15.76:0/1239340851 wait complete. 2023-11-09T03:48:09.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.009+0000 7fea8fafc640 1 Processor -- start 2023-11-09T03:48:09.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.009+0000 7fea8fafc640 1 -- start start 2023-11-09T03:48:09.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.010+0000 7fea8fafc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea8806db20 0x7fea88153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.010+0000 7fea8fafc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea880b0ae0 0x7fea88154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.010+0000 7fea8fafc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea88159c70 0x7fea881524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.010+0000 7fea8fafc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea88156700 con 0x7fea880b0ae0 2023-11-09T03:48:09.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.010+0000 7fea8fafc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea88155200 con 0x7fea88159c70 2023-11-09T03:48:09.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.010+0000 7fea8fafc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea88154f60 con 0x7fea8806db20 2023-11-09T03:48:09.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.010+0000 7fea8d097640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea880b0ae0 0x7fea88154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.010+0000 7fea8d097640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea880b0ae0 0x7fea88154a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50892/0 (socket says 172.21.15.76:50892) 2023-11-09T03:48:09.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.010+0000 7fea8d097640 1 -- 172.21.15.76:0/3736802836 learned_addr learned my addr 172.21.15.76:0/3736802836 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:09.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.010+0000 7fea8d898640 1 --2- 172.21.15.76:0/3736802836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea8806db20 0x7fea88153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.010+0000 7fea8e099640 1 --2- 172.21.15.76:0/3736802836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea88159c70 0x7fea881524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.010+0000 7fea8d097640 1 -- 172.21.15.76:0/3736802836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea8806db20 msgr2=0x7fea88153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:09.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.010+0000 7fea8d097640 1 --2- 172.21.15.76:0/3736802836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea8806db20 0x7fea88153020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.010+0000 7fea8d097640 1 -- 172.21.15.76:0/3736802836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea88159c70 msgr2=0x7fea881524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:09.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.010+0000 7fea8d097640 1 --2- 172.21.15.76:0/3736802836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea88159c70 0x7fea881524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.010+0000 7fea8d097640 1 -- 172.21.15.76:0/3736802836 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea78009400 con 0x7fea880b0ae0 2023-11-09T03:48:09.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.010+0000 7fea8d898640 1 --2- 172.21.15.76:0/3736802836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea8806db20 0x7fea88153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:09.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.010+0000 7fea8e099640 1 --2- 172.21.15.76:0/3736802836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea88159c70 0x7fea881524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:09.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.010+0000 7fea8d097640 1 --2- 172.21.15.76:0/3736802836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea880b0ae0 0x7fea88154a20 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fea7802bee0 tx=0x7fea780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:09.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.011+0000 7fea7e7fc640 1 -- 172.21.15.76:0/3736802836 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7803a6f0 con 0x7fea880b0ae0 2023-11-09T03:48:09.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.011+0000 7fea7e7fc640 1 -- 172.21.15.76:0/3736802836 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7803a870 con 0x7fea880b0ae0 2023-11-09T03:48:09.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.011+0000 7fea8fafc640 1 -- 172.21.15.76:0/3736802836 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea88156930 con 0x7fea880b0ae0 2023-11-09T03:48:09.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.011+0000 7fea7e7fc640 1 -- 172.21.15.76:0/3736802836 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7803ac20 con 0x7fea880b0ae0 2023-11-09T03:48:09.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.011+0000 7fea8fafc640 1 -- 172.21.15.76:0/3736802836 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea88156ef0 con 0x7fea880b0ae0 2023-11-09T03:48:09.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.012+0000 7fea8fafc640 1 -- 172.21.15.76:0/3736802836 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea88074290 con 0x7fea880b0ae0 2023-11-09T03:48:09.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.012+0000 7fea7e7fc640 1 -- 172.21.15.76:0/3736802836 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fea7804c3b0 con 0x7fea880b0ae0 2023-11-09T03:48:09.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.012+0000 7fea7e7fc640 1 --2- 172.21.15.76:0/3736802836 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fea6c06aea0 0x7fea6c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.013+0000 7fea7e7fc640 1 -- 172.21.15.76:0/3736802836 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7148+0+0 (secure 0 0 0) 0x7fea780ab930 con 0x7fea880b0ae0 2023-11-09T03:48:09.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.013+0000 7fea8d898640 1 --2- 172.21.15.76:0/3736802836 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fea6c06aea0 0x7fea6c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.013+0000 7fea8d898640 1 --2- 172.21.15.76:0/3736802836 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fea6c06aea0 0x7fea6c06d360 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fea84000f60 tx=0x7fea84009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:09.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.017+0000 7fea7e7fc640 1 -- 172.21.15.76:0/3736802836 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fea78076560 con 0x7fea880b0ae0 2023-11-09T03:48:09.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.215+0000 7fea8fafc640 1 -- 172.21.15.76:0/3736802836 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea8806d3f0 con 0x7fea880b0ae0 2023-11-09T03:48:09.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.215+0000 7fea7e7fc640 1 -- 172.21.15.76:0/3736802836 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fea78079b80 con 0x7fea880b0ae0 2023-11-09T03:48:09.215 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:09.216 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2800733670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2800733670},{"type":"v1","addr":"172.21.15.113:6836","nonce":2800733670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":5791,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1301905851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1301905851},{"type":"v1","addr":"172.21.15.113:6837","nonce":1301905851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":6498,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/968739949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":968739949},{"type":"v1","addr":"172.21.15.76:6833","nonce":968739949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:47:51.114132+0000","modified":"2023-11-09T03:47:53.114512+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6501},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6501":{"gid":6501,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/1576689316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1576689316},{"type":"v1","addr":"172.21.15.76:6835","nonce":1576689316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-09T03:48:09.216 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 29 2023-11-09T03:48:09.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.217+0000 7fea8fafc640 1 -- 172.21.15.76:0/3736802836 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fea6c06aea0 msgr2=0x7fea6c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:09.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.217+0000 7fea8fafc640 1 --2- 172.21.15.76:0/3736802836 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fea6c06aea0 0x7fea6c06d360 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fea84000f60 tx=0x7fea84009040 comp rx=0 tx=0).stop 2023-11-09T03:48:09.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.217+0000 7fea8fafc640 1 -- 172.21.15.76:0/3736802836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea880b0ae0 msgr2=0x7fea88154a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:09.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.217+0000 7fea8fafc640 1 --2- 172.21.15.76:0/3736802836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea880b0ae0 0x7fea88154a20 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fea7802bee0 tx=0x7fea780046c0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.217+0000 7fea8fafc640 1 -- 172.21.15.76:0/3736802836 shutdown_connections 2023-11-09T03:48:09.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.217+0000 7fea8fafc640 1 --2- 172.21.15.76:0/3736802836 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fea6c06aea0 0x7fea6c06d360 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.217+0000 7fea8fafc640 1 --2- 172.21.15.76:0/3736802836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea88159c70 0x7fea881524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.217+0000 7fea8fafc640 1 --2- 172.21.15.76:0/3736802836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea880b0ae0 0x7fea88154a20 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.217+0000 7fea8fafc640 1 --2- 172.21.15.76:0/3736802836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea8806db20 0x7fea88153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.217+0000 7fea8fafc640 1 -- 172.21.15.76:0/3736802836 >> 172.21.15.76:0/3736802836 conn(0x7fea880ce950 msgr2=0x7fea880cd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:09.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.217+0000 7fea8fafc640 1 -- 172.21.15.76:0/3736802836 shutdown_connections 2023-11-09T03:48:09.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.217+0000 7fea8fafc640 1 -- 172.21.15.76:0/3736802836 wait complete. 2023-11-09T03:48:09.227 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:48:09.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.329+0000 7fa9db190640 1 Processor -- start 2023-11-09T03:48:09.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.329+0000 7fa9db190640 1 -- start start 2023-11-09T03:48:09.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.330+0000 7fa9db190640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa9d40b0ae0 0x7fa9d4070780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.330+0000 7fa9db190640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d40714b0 con 0x7fa9d4070cc0 2023-11-09T03:48:09.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.330+0000 7fa9db190640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d4071790 con 0x7fa9d415cfd0 2023-11-09T03:48:09.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.330+0000 7fa9db190640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9d40ab8e0 con 0x7fa9d40b0ae0 2023-11-09T03:48:09.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.330+0000 7fa9d3fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa9d40b0ae0 0x7fa9d4070780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.330+0000 7fa9d972d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa9d4070cc0 0x7fa9d406ce90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:60060/0 (socket says 172.21.15.76:60060) 2023-11-09T03:48:09.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.330+0000 7fa9d972d640 1 -- 172.21.15.76:0/3257638289 learned_addr learned my addr 172.21.15.76:0/3257638289 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:09.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.330+0000 7fa9d3fff640 1 -- 172.21.15.76:0/3257638289 >> v1:172.21.15.113:6789/0 conn(0x7fa9d415cfd0 legacy=0x7fa9d40b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:09.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.330+0000 7fa9d3fff640 1 -- 172.21.15.76:0/3257638289 >> v1:172.21.15.76:6789/0 conn(0x7fa9d4070cc0 legacy=0x7fa9d406ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:09.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.330+0000 7fa9d3fff640 1 -- 172.21.15.76:0/3257638289 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d406c690 con 0x7fa9d40b0ae0 2023-11-09T03:48:09.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.330+0000 7fa9d37fe640 1 -- 172.21.15.76:0/3257638289 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256171692 0 0) 0x7fa9d40714b0 con 0x7fa9d4070cc0 2023-11-09T03:48:09.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.331+0000 7fa9d3fff640 1 --2- 172.21.15.76:0/3257638289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9d40b0ae0 0x7fa9d4070780 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fa9c00097a0 tx=0x7fa9c002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964385a27ce92e26 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:09.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.331+0000 7fa9d37fe640 1 -- 172.21.15.76:0/3257638289 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0031830 con 0x7fa9d40b0ae0 2023-11-09T03:48:09.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.331+0000 7fa9d37fe640 1 -- 172.21.15.76:0/3257638289 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c0031990 con 0x7fa9d40b0ae0 2023-11-09T03:48:09.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.331+0000 7fa9d37fe640 1 -- 172.21.15.76:0/3257638289 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0031d20 con 0x7fa9d40b0ae0 2023-11-09T03:48:09.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.332+0000 7fa9db190640 1 -- 172.21.15.76:0/3257638289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9d40b0ae0 msgr2=0x7fa9d4070780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:09.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.332+0000 7fa9db190640 1 --2- 172.21.15.76:0/3257638289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9d40b0ae0 0x7fa9d4070780 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fa9c00097a0 tx=0x7fa9c002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.332+0000 7fa9db190640 1 -- 172.21.15.76:0/3257638289 shutdown_connections 2023-11-09T03:48:09.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.332+0000 7fa9db190640 1 --2- 172.21.15.76:0/3257638289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9d40b0ae0 0x7fa9d4070780 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.332+0000 7fa9db190640 1 -- 172.21.15.76:0/3257638289 >> 172.21.15.76:0/3257638289 conn(0x7fa9d40ce950 msgr2=0x7fa9d40ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:09.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.332+0000 7fa9db190640 1 -- 172.21.15.76:0/3257638289 shutdown_connections 2023-11-09T03:48:09.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.332+0000 7fa9db190640 1 -- 172.21.15.76:0/3257638289 wait complete. 2023-11-09T03:48:09.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.333+0000 7fa9db190640 1 Processor -- start 2023-11-09T03:48:09.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.333+0000 7fa9db190640 1 -- start start 2023-11-09T03:48:09.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.334+0000 7fa9db190640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9d4070cc0 0x7fa9d4151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.334+0000 7fa9db190640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9d40b0ae0 0x7fa9d40cad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.334+0000 7fa9db190640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9d415cfd0 0x7fa9d40cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.334+0000 7fa9db190640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d40c95d0 con 0x7fa9d415cfd0 2023-11-09T03:48:09.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.334+0000 7fa9db190640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d40c30a0 con 0x7fa9d4070cc0 2023-11-09T03:48:09.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.334+0000 7fa9db190640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d40c3380 con 0x7fa9d40b0ae0 2023-11-09T03:48:09.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.334+0000 7fa9d972d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9d415cfd0 0x7fa9d40cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.334+0000 7fa9d972d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9d415cfd0 0x7fa9d40cb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50920/0 (socket says 172.21.15.76:50920) 2023-11-09T03:48:09.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.334+0000 7fa9d972d640 1 -- 172.21.15.76:0/1953950693 learned_addr learned my addr 172.21.15.76:0/1953950693 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:09.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.334+0000 7fa9d8f2c640 1 --2- 172.21.15.76:0/1953950693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9d4070cc0 0x7fa9d4151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.334+0000 7fa9d3fff640 1 --2- 172.21.15.76:0/1953950693 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9d40b0ae0 0x7fa9d40cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.334+0000 7fa9d972d640 1 -- 172.21.15.76:0/1953950693 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9d40b0ae0 msgr2=0x7fa9d40cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:09.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.334+0000 7fa9d972d640 1 --2- 172.21.15.76:0/1953950693 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9d40b0ae0 0x7fa9d40cad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.334+0000 7fa9d972d640 1 -- 172.21.15.76:0/1953950693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9d4070cc0 msgr2=0x7fa9d4151530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:09.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.334+0000 7fa9d972d640 1 --2- 172.21.15.76:0/1953950693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9d4070cc0 0x7fa9d4151530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.334+0000 7fa9d972d640 1 -- 172.21.15.76:0/1953950693 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c0009400 con 0x7fa9d415cfd0 2023-11-09T03:48:09.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.334+0000 7fa9d972d640 1 --2- 172.21.15.76:0/1953950693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9d415cfd0 0x7fa9d40cb2d0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00c7a0 tx=0x7fa9cc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:09.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.334+0000 7fa9d17fa640 1 -- 172.21.15.76:0/1953950693 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc00a640 con 0x7fa9d415cfd0 2023-11-09T03:48:09.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.334+0000 7fa9d17fa640 1 -- 172.21.15.76:0/1953950693 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9cc00a7a0 con 0x7fa9d415cfd0 2023-11-09T03:48:09.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.334+0000 7fa9d17fa640 1 -- 172.21.15.76:0/1953950693 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc00abd0 con 0x7fa9d415cfd0 2023-11-09T03:48:09.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.334+0000 7fa9db190640 1 -- 172.21.15.76:0/1953950693 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d40c3660 con 0x7fa9d415cfd0 2023-11-09T03:48:09.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.335+0000 7fa9db190640 1 -- 172.21.15.76:0/1953950693 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d40c92a0 con 0x7fa9d415cfd0 2023-11-09T03:48:09.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.335+0000 7fa9db190640 1 -- 172.21.15.76:0/1953950693 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d40c3850 con 0x7fa9d415cfd0 2023-11-09T03:48:09.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.336+0000 7fa9d17fa640 1 -- 172.21.15.76:0/1953950693 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa9cc00ad30 con 0x7fa9d415cfd0 2023-11-09T03:48:09.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.336+0000 7fa9d17fa640 1 --2- 172.21.15.76:0/1953950693 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa9b806ae70 0x7fa9b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.336+0000 7fa9d17fa640 1 -- 172.21.15.76:0/1953950693 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7148+0+0 (secure 0 0 0) 0x7fa9cc089140 con 0x7fa9d415cfd0 2023-11-09T03:48:09.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.336+0000 7fa9d8f2c640 1 --2- 172.21.15.76:0/1953950693 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa9b806ae70 0x7fa9b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.337+0000 7fa9d8f2c640 1 --2- 172.21.15.76:0/1953950693 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa9b806ae70 0x7fa9b806d330 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8006fd0 tx=0x7fa9c8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:09.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.340+0000 7fa9d17fa640 1 -- 172.21.15.76:0/1953950693 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa9cc053c50 con 0x7fa9d415cfd0 2023-11-09T03:48:09.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.544+0000 7fa9db190640 1 -- 172.21.15.76:0/1953950693 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9d4074290 con 0x7fa9d415cfd0 2023-11-09T03:48:09.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.544+0000 7fa9d17fa640 1 -- 172.21.15.76:0/1953950693 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fa9cc018090 con 0x7fa9d415cfd0 2023-11-09T03:48:09.544 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:09.544 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2800733670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2800733670},{"type":"v1","addr":"172.21.15.113:6836","nonce":2800733670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":5791,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1301905851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1301905851},{"type":"v1","addr":"172.21.15.113:6837","nonce":1301905851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":6498,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/968739949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":968739949},{"type":"v1","addr":"172.21.15.76:6833","nonce":968739949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:47:51.114132+0000","modified":"2023-11-09T03:47:53.114512+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6501},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6501":{"gid":6501,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/1576689316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1576689316},{"type":"v1","addr":"172.21.15.76:6835","nonce":1576689316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-09T03:48:09.546 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 29 2023-11-09T03:48:09.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.546+0000 7fa9db190640 1 -- 172.21.15.76:0/1953950693 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa9b806ae70 msgr2=0x7fa9b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:09.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.546+0000 7fa9db190640 1 --2- 172.21.15.76:0/1953950693 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa9b806ae70 0x7fa9b806d330 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8006fd0 tx=0x7fa9c8008040 comp rx=0 tx=0).stop 2023-11-09T03:48:09.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.546+0000 7fa9db190640 1 -- 172.21.15.76:0/1953950693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9d415cfd0 msgr2=0x7fa9d40cb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:09.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.546+0000 7fa9db190640 1 --2- 172.21.15.76:0/1953950693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9d415cfd0 0x7fa9d40cb2d0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00c7a0 tx=0x7fa9cc00cc60 comp rx=0 tx=0).stop 2023-11-09T03:48:09.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.546+0000 7fa9db190640 1 -- 172.21.15.76:0/1953950693 shutdown_connections 2023-11-09T03:48:09.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.546+0000 7fa9db190640 1 --2- 172.21.15.76:0/1953950693 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa9b806ae70 0x7fa9b806d330 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.546+0000 7fa9db190640 1 --2- 172.21.15.76:0/1953950693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9d415cfd0 0x7fa9d40cb2d0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.546+0000 7fa9db190640 1 --2- 172.21.15.76:0/1953950693 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9d40b0ae0 0x7fa9d40cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.546+0000 7fa9db190640 1 --2- 172.21.15.76:0/1953950693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9d4070cc0 0x7fa9d4151530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.546+0000 7fa9db190640 1 -- 172.21.15.76:0/1953950693 >> 172.21.15.76:0/1953950693 conn(0x7fa9d40ce950 msgr2=0x7fa9d40afe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:09.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.546+0000 7fa9db190640 1 -- 172.21.15.76:0/1953950693 shutdown_connections 2023-11-09T03:48:09.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.546+0000 7fa9db190640 1 -- 172.21.15.76:0/1953950693 wait complete. 2023-11-09T03:48:09.556 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:48:09.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.658+0000 7fc65b2c2640 1 Processor -- start 2023-11-09T03:48:09.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.658+0000 7fc65b2c2640 1 -- start start 2023-11-09T03:48:09.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.659+0000 7fc65b2c2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc654159c70 0x7fc6540ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.659+0000 7fc65b2c2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc6540b0ae0 0x7fc65415d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.659+0000 7fc65b2c2640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6540b4110 con 0x7fc65406db20 2023-11-09T03:48:09.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.659+0000 7fc65b2c2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6540b49b0 con 0x7fc6540b0ae0 2023-11-09T03:48:09.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.659+0000 7fc65b2c2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540b43f0 con 0x7fc654159c70 2023-11-09T03:48:09.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.659+0000 7fc65885d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc65406db20 0x7fc6540b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:60068/0 (socket says 172.21.15.76:60068) 2023-11-09T03:48:09.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.659+0000 7fc65885d640 1 -- 172.21.15.76:0/327655911 learned_addr learned my addr 172.21.15.76:0/327655911 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:09.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.659+0000 7fc64bfff640 1 -- 172.21.15.76:0/327655911 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3169047545 0 0) 0x7fc6540b4110 con 0x7fc65406db20 2023-11-09T03:48:09.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.659+0000 7fc64bfff640 1 -- 172.21.15.76:0/327655911 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc640003520 con 0x7fc65406db20 2023-11-09T03:48:09.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.659+0000 7fc65985f640 1 --2- 172.21.15.76:0/327655911 >> v2:172.21.15.113:3300/0 conn(0x7fc6540b0ae0 0x7fc65415d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.659+0000 7fc65905e640 1 --2- 172.21.15.76:0/327655911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc654159c70 0x7fc6540ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.659+0000 7fc64bfff640 1 -- 172.21.15.76:0/327655911 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3189113432 0 0) 0x7fc640003520 con 0x7fc65406db20 2023-11-09T03:48:09.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.659+0000 7fc64bfff640 1 -- 172.21.15.76:0/327655911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc654159c70 msgr2=0x7fc6540ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:09.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.659+0000 7fc64bfff640 1 --2- 172.21.15.76:0/327655911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc654159c70 0x7fc6540ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.659+0000 7fc64bfff640 1 -- 172.21.15.76:0/327655911 >> v2:172.21.15.113:3300/0 conn(0x7fc6540b0ae0 msgr2=0x7fc65415d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:09.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.659+0000 7fc64bfff640 1 --2- 172.21.15.76:0/327655911 >> v2:172.21.15.113:3300/0 conn(0x7fc6540b0ae0 0x7fc65415d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.659+0000 7fc64bfff640 1 -- 172.21.15.76:0/327655911 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6540b46d0 con 0x7fc65406db20 2023-11-09T03:48:09.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.659+0000 7fc64bfff640 1 -- 172.21.15.76:0/327655911 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fc644001670 con 0x7fc65406db20 2023-11-09T03:48:09.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.659+0000 7fc64bfff640 1 -- 172.21.15.76:0/327655911 >> v1:172.21.15.76:6789/0 conn(0x7fc65406db20 legacy=0x7fc6540b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:09.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.659+0000 7fc64bfff640 1 --2- 172.21.15.76:0/327655911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc640003970 0x7fc640003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.659+0000 7fc64bfff640 1 --2- 172.21.15.76:0/327655911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc654159c70 0x7fc6400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.659+0000 7fc64bfff640 1 --2- 172.21.15.76:0/327655911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc640004d00 0x7fc640005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.659+0000 7fc64bfff640 1 -- 172.21.15.76:0/327655911 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540b49b0 con 0x7fc640004d00 2023-11-09T03:48:09.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.659+0000 7fc64bfff640 1 -- 172.21.15.76:0/327655911 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540b43f0 con 0x7fc640003970 2023-11-09T03:48:09.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.659+0000 7fc64bfff640 1 -- 172.21.15.76:0/327655911 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc64000b980 con 0x7fc654159c70 2023-11-09T03:48:09.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.660+0000 7fc65985f640 1 --2- 172.21.15.76:0/327655911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc640004d00 0x7fc640005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.660+0000 7fc65905e640 1 --2- 172.21.15.76:0/327655911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc640003970 0x7fc640003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.660+0000 7fc65885d640 1 --2- 172.21.15.76:0/327655911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc654159c70 0x7fc6400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.660+0000 7fc65985f640 1 -- 172.21.15.76:0/327655911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc654159c70 msgr2=0x7fc6400047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:09.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.660+0000 7fc65985f640 1 --2- 172.21.15.76:0/327655911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc654159c70 0x7fc6400047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.660+0000 7fc65985f640 1 -- 172.21.15.76:0/327655911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc640003970 msgr2=0x7fc640003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:09.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.660+0000 7fc65985f640 1 --2- 172.21.15.76:0/327655911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc640003970 0x7fc640003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.660+0000 7fc65985f640 1 -- 172.21.15.76:0/327655911 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc640003520 con 0x7fc640004d00 2023-11-09T03:48:09.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.661+0000 7fc65985f640 1 --2- 172.21.15.76:0/327655911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc640004d00 0x7fc640005180 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fc64c009fc0 tx=0x7fc6400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87038b325743287c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:09.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.661+0000 7fc64bfff640 1 -- 172.21.15.76:0/327655911 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64c02cc60 con 0x7fc640004d00 2023-11-09T03:48:09.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.661+0000 7fc64bfff640 1 -- 172.21.15.76:0/327655911 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64c00f040 con 0x7fc640004d00 2023-11-09T03:48:09.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.661+0000 7fc64bfff640 1 -- 172.21.15.76:0/327655911 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64c034440 con 0x7fc640004d00 2023-11-09T03:48:09.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.661+0000 7fc65b2c2640 1 -- 172.21.15.76:0/327655911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc640004d00 msgr2=0x7fc640005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:09.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.661+0000 7fc65b2c2640 1 --2- 172.21.15.76:0/327655911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc640004d00 0x7fc640005180 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fc64c009fc0 tx=0x7fc6400057f0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.661+0000 7fc65985f640 1 -- 172.21.15.76:0/327655911 reap_dead start 2023-11-09T03:48:09.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.661+0000 7fc65b2c2640 1 -- 172.21.15.76:0/327655911 shutdown_connections 2023-11-09T03:48:09.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.661+0000 7fc65b2c2640 1 -- 172.21.15.76:0/327655911 >> 172.21.15.76:0/327655911 conn(0x7fc6540ce950 msgr2=0x7fc6540ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:09.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.661+0000 7fc65b2c2640 1 -- 172.21.15.76:0/327655911 shutdown_connections 2023-11-09T03:48:09.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.661+0000 7fc65b2c2640 1 -- 172.21.15.76:0/327655911 wait complete. 2023-11-09T03:48:09.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.662+0000 7fc65b2c2640 1 Processor -- start 2023-11-09T03:48:09.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.663+0000 7fc65b2c2640 1 -- start start 2023-11-09T03:48:09.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.663+0000 7fc65b2c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc640004d00 0x7fc6540c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.663+0000 7fc65b2c2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6540c4bf0 0x7fc654151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.663+0000 7fc65b2c2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6540c5050 0x7fc6541521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.663+0000 7fc65b2c2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc654143db0 con 0x7fc640004d00 2023-11-09T03:48:09.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.663+0000 7fc65b2c2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540c7df0 con 0x7fc6540c4bf0 2023-11-09T03:48:09.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.663+0000 7fc65b2c2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540c80d0 con 0x7fc6540c5050 2023-11-09T03:48:09.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.663+0000 7fc65905e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc640004d00 0x7fc6540c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.663+0000 7fc65905e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc640004d00 0x7fc6540c6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50940/0 (socket says 172.21.15.76:50940) 2023-11-09T03:48:09.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.663+0000 7fc65905e640 1 -- 172.21.15.76:0/906743890 learned_addr learned my addr 172.21.15.76:0/906743890 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:09.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.664+0000 7fc65885d640 1 --2- 172.21.15.76:0/906743890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6540c4bf0 0x7fc654151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.664+0000 7fc65985f640 1 --2- 172.21.15.76:0/906743890 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6540c5050 0x7fc6541521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.664+0000 7fc65905e640 1 -- 172.21.15.76:0/906743890 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6540c5050 msgr2=0x7fc6541521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:09.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.664+0000 7fc65905e640 1 --2- 172.21.15.76:0/906743890 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6540c5050 0x7fc6541521b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.664+0000 7fc65905e640 1 -- 172.21.15.76:0/906743890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6540c4bf0 msgr2=0x7fc654151c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:09.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.664+0000 7fc65905e640 1 --2- 172.21.15.76:0/906743890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6540c4bf0 0x7fc654151c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.664+0000 7fc65905e640 1 -- 172.21.15.76:0/906743890 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6540c83b0 con 0x7fc640004d00 2023-11-09T03:48:09.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.664+0000 7fc65885d640 1 --2- 172.21.15.76:0/906743890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6540c4bf0 0x7fc654151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:09.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.664+0000 7fc65985f640 1 --2- 172.21.15.76:0/906743890 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6540c5050 0x7fc6541521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:09.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.664+0000 7fc65905e640 1 --2- 172.21.15.76:0/906743890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc640004d00 0x7fc6540c6030 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fc6500118e0 tx=0x7fc650011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:09.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.664+0000 7fc649ffb640 1 -- 172.21.15.76:0/906743890 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65000faf0 con 0x7fc640004d00 2023-11-09T03:48:09.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.664+0000 7fc649ffb640 1 -- 172.21.15.76:0/906743890 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65000fc50 con 0x7fc640004d00 2023-11-09T03:48:09.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.664+0000 7fc649ffb640 1 -- 172.21.15.76:0/906743890 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc650014440 con 0x7fc640004d00 2023-11-09T03:48:09.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.664+0000 7fc65b2c2640 1 -- 172.21.15.76:0/906743890 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6540c8690 con 0x7fc640004d00 2023-11-09T03:48:09.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.664+0000 7fc65b2c2640 1 -- 172.21.15.76:0/906743890 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6540c8e90 con 0x7fc640004d00 2023-11-09T03:48:09.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.665+0000 7fc65b2c2640 1 -- 172.21.15.76:0/906743890 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc654074290 con 0x7fc640004d00 2023-11-09T03:48:09.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.666+0000 7fc649ffb640 1 -- 172.21.15.76:0/906743890 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc6500145a0 con 0x7fc640004d00 2023-11-09T03:48:09.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.666+0000 7fc649ffb640 1 --2- 172.21.15.76:0/906743890 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc63806ae70 0x7fc63806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.666+0000 7fc649ffb640 1 -- 172.21.15.76:0/906743890 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7148+0+0 (secure 0 0 0) 0x7fc65008a4f0 con 0x7fc640004d00 2023-11-09T03:48:09.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.666+0000 7fc65885d640 1 --2- 172.21.15.76:0/906743890 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc63806ae70 0x7fc63806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.667+0000 7fc65885d640 1 --2- 172.21.15.76:0/906743890 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc63806ae70 0x7fc63806d330 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc65414f2e0 tx=0x7fc644009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:09.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.670+0000 7fc649ffb640 1 -- 172.21.15.76:0/906743890 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc650055000 con 0x7fc640004d00 2023-11-09T03:48:09.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.839+0000 7fc65b2c2640 1 -- 172.21.15.76:0/906743890 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc654158720 con 0x7fc640004d00 2023-11-09T03:48:09.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.840+0000 7fc649ffb640 1 -- 172.21.15.76:0/906743890 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+22089 (secure 0 0 0) 0x7fc650058620 con 0x7fc640004d00 2023-11-09T03:48:09.840 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:09.840 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":70,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:48:02.436030+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:47:49.630346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-09T03:47:50.082207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-09T03:47:59.280601+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-09T03:47:59.742039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":12,"snaps":[{"begin":1,"length":1}]}],"new_removed_snaps":[{"pool":12,"snaps":[{"begin":1,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:48:09.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.842+0000 7fc65b2c2640 1 -- 172.21.15.76:0/906743890 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc63806ae70 msgr2=0x7fc63806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:09.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.842+0000 7fc65b2c2640 1 --2- 172.21.15.76:0/906743890 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc63806ae70 0x7fc63806d330 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc65414f2e0 tx=0x7fc644009040 comp rx=0 tx=0).stop 2023-11-09T03:48:09.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.842+0000 7fc65b2c2640 1 -- 172.21.15.76:0/906743890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc640004d00 msgr2=0x7fc6540c6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:09.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.842+0000 7fc65b2c2640 1 --2- 172.21.15.76:0/906743890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc640004d00 0x7fc6540c6030 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fc6500118e0 tx=0x7fc650011da0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.842+0000 7fc65b2c2640 1 -- 172.21.15.76:0/906743890 shutdown_connections 2023-11-09T03:48:09.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.842+0000 7fc65b2c2640 1 --2- 172.21.15.76:0/906743890 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc63806ae70 0x7fc63806d330 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.842+0000 7fc65b2c2640 1 --2- 172.21.15.76:0/906743890 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6540c5050 0x7fc6541521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.842+0000 7fc65b2c2640 1 --2- 172.21.15.76:0/906743890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6540c4bf0 0x7fc654151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.842+0000 7fc65b2c2640 1 --2- 172.21.15.76:0/906743890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc640004d00 0x7fc6540c6030 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.842+0000 7fc65b2c2640 1 -- 172.21.15.76:0/906743890 >> 172.21.15.76:0/906743890 conn(0x7fc6540ce950 msgr2=0x7fc654159020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:09.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.842+0000 7fc65b2c2640 1 -- 172.21.15.76:0/906743890 shutdown_connections 2023-11-09T03:48:09.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.842+0000 7fc65b2c2640 1 -- 172.21.15.76:0/906743890 wait complete. 2023-11-09T03:48:09.853 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-09T03:48:09.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.954+0000 7f20982c2640 1 Processor -- start 2023-11-09T03:48:09.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.954+0000 7f20982c2640 1 -- start start 2023-11-09T03:48:09.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.955+0000 7f20982c2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2090159c00 0x7f20900cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.955+0000 7f20982c2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f20900ae1d0 0x7f20900c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.955+0000 7f20982c2640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f209015aff0 con 0x7f20900c48e0 2023-11-09T03:48:09.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.955+0000 7f20982c2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20900c52a0 con 0x7f20900ae1d0 2023-11-09T03:48:09.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.955+0000 7f20982c2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209015a3e0 con 0x7f2090159c00 2023-11-09T03:48:09.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.955+0000 7f209585d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f20900c48e0 0x7f20900adac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:60076/0 (socket says 172.21.15.76:60076) 2023-11-09T03:48:09.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.955+0000 7f209585d640 1 -- 172.21.15.76:0/2085465203 learned_addr learned my addr 172.21.15.76:0/2085465203 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:09.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.955+0000 7f209685f640 1 --2- 172.21.15.76:0/2085465203 >> v2:172.21.15.113:3300/0 conn(0x7f20900ae1d0 0x7f20900c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.955+0000 7f209605e640 1 --2- 172.21.15.76:0/2085465203 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2090159c00 0x7f20900cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.955+0000 7f209505c640 1 -- 172.21.15.76:0/2085465203 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 442446290 0 0) 0x7f209015aff0 con 0x7f20900c48e0 2023-11-09T03:48:09.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.955+0000 7f209505c640 1 -- 172.21.15.76:0/2085465203 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f207c003520 con 0x7f20900c48e0 2023-11-09T03:48:09.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.956+0000 7f209505c640 1 -- 172.21.15.76:0/2085465203 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3330729333 0 0) 0x7f207c003520 con 0x7f20900c48e0 2023-11-09T03:48:09.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.956+0000 7f209505c640 1 -- 172.21.15.76:0/2085465203 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2090159c00 msgr2=0x7f20900cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:09.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.956+0000 7f209505c640 1 --2- 172.21.15.76:0/2085465203 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2090159c00 0x7f20900cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.956+0000 7f209505c640 1 -- 172.21.15.76:0/2085465203 >> v2:172.21.15.113:3300/0 conn(0x7f20900ae1d0 msgr2=0x7f20900c4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:09.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.956+0000 7f209505c640 1 --2- 172.21.15.76:0/2085465203 >> v2:172.21.15.113:3300/0 conn(0x7f20900ae1d0 0x7f20900c4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.956+0000 7f209505c640 1 -- 172.21.15.76:0/2085465203 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209015a6c0 con 0x7f20900c48e0 2023-11-09T03:48:09.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.956+0000 7f209685f640 1 --2- 172.21.15.76:0/2085465203 >> v2:172.21.15.113:3300/0 conn(0x7f20900ae1d0 0x7f20900c4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:48:09.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.956+0000 7f209505c640 1 -- 172.21.15.76:0/2085465203 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f2080001670 con 0x7f20900c48e0 2023-11-09T03:48:09.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.956+0000 7f209505c640 1 -- 172.21.15.76:0/2085465203 >> v1:172.21.15.76:6789/0 conn(0x7f20900c48e0 legacy=0x7f20900adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:09.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.956+0000 7f209505c640 1 --2- 172.21.15.76:0/2085465203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f207c003970 0x7f207c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.956+0000 7f209505c640 1 --2- 172.21.15.76:0/2085465203 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2090159c00 0x7f207c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.956+0000 7f209505c640 1 --2- 172.21.15.76:0/2085465203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f207c004d00 0x7f207c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.956+0000 7f209505c640 1 -- 172.21.15.76:0/2085465203 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20900c52a0 con 0x7f207c004d00 2023-11-09T03:48:09.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.956+0000 7f209505c640 1 -- 172.21.15.76:0/2085465203 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209015a3e0 con 0x7f207c003970 2023-11-09T03:48:09.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.956+0000 7f209505c640 1 -- 172.21.15.76:0/2085465203 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f207c00b980 con 0x7f2090159c00 2023-11-09T03:48:09.963 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.956+0000 7f209685f640 1 --2- 172.21.15.76:0/2085465203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f207c004d00 0x7f207c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.963 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.956+0000 7f209605e640 1 --2- 172.21.15.76:0/2085465203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f207c003970 0x7f207c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.963 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.956+0000 7f209585d640 1 --2- 172.21.15.76:0/2085465203 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2090159c00 0x7f207c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.963 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.956+0000 7f209685f640 1 -- 172.21.15.76:0/2085465203 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2090159c00 msgr2=0x7f207c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:09.963 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.956+0000 7f209685f640 1 --2- 172.21.15.76:0/2085465203 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2090159c00 0x7f207c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.956+0000 7f209685f640 1 -- 172.21.15.76:0/2085465203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f207c003970 msgr2=0x7f207c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:09.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.956+0000 7f209685f640 1 --2- 172.21.15.76:0/2085465203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f207c003970 0x7f207c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.956+0000 7f209685f640 1 -- 172.21.15.76:0/2085465203 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f207c003520 con 0x7f207c004d00 2023-11-09T03:48:09.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.956+0000 7f209605e640 1 --2- 172.21.15.76:0/2085465203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f207c003970 0x7f207c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:09.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.956+0000 7f209585d640 1 --2- 172.21.15.76:0/2085465203 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2090159c00 0x7f207c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:09.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.957+0000 7f209685f640 1 --2- 172.21.15.76:0/2085465203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f207c004d00 0x7f207c005180 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f208c009c90 tx=0x7f207c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7612f315d3e0062 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:09.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.957+0000 7f209505c640 1 -- 172.21.15.76:0/2085465203 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c02c980 con 0x7f207c004d00 2023-11-09T03:48:09.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.957+0000 7f209505c640 1 -- 172.21.15.76:0/2085465203 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208c02cae0 con 0x7f207c004d00 2023-11-09T03:48:09.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.957+0000 7f209505c640 1 -- 172.21.15.76:0/2085465203 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c02ce90 con 0x7f207c004d00 2023-11-09T03:48:09.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.957+0000 7f20982c2640 1 -- 172.21.15.76:0/2085465203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f207c004d00 msgr2=0x7f207c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:09.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.957+0000 7f20982c2640 1 --2- 172.21.15.76:0/2085465203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f207c004d00 0x7f207c005180 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f208c009c90 tx=0x7f207c0057f0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.957+0000 7f209685f640 1 -- 172.21.15.76:0/2085465203 reap_dead start 2023-11-09T03:48:09.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.958+0000 7f20982c2640 1 -- 172.21.15.76:0/2085465203 shutdown_connections 2023-11-09T03:48:09.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.958+0000 7f20982c2640 1 -- 172.21.15.76:0/2085465203 >> 172.21.15.76:0/2085465203 conn(0x7f20900ce950 msgr2=0x7f20900ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:09.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.958+0000 7f20982c2640 1 -- 172.21.15.76:0/2085465203 shutdown_connections 2023-11-09T03:48:09.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.958+0000 7f20982c2640 1 -- 172.21.15.76:0/2085465203 wait complete. 2023-11-09T03:48:09.971 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.959+0000 7f20982c2640 1 Processor -- start 2023-11-09T03:48:09.971 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.959+0000 7f20982c2640 1 -- start start 2023-11-09T03:48:09.971 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.959+0000 7f20982c2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f207c004d00 0x7f209014a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.959+0000 7f20982c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2090157e60 0x7f20901582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.960+0000 7f20982c2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2090155b00 0x7f2090155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.960+0000 7f20982c2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2090158820 con 0x7f2090157e60 2023-11-09T03:48:09.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.960+0000 7f20982c2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20901568c0 con 0x7f207c004d00 2023-11-09T03:48:09.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.960+0000 7f20982c2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2090156ba0 con 0x7f2090155b00 2023-11-09T03:48:09.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.960+0000 7f209585d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2090157e60 0x7f20901582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.960+0000 7f209585d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2090157e60 0x7f20901582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50966/0 (socket says 172.21.15.76:50966) 2023-11-09T03:48:09.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.960+0000 7f209585d640 1 -- 172.21.15.76:0/2374501549 learned_addr learned my addr 172.21.15.76:0/2374501549 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:09.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.960+0000 7f209605e640 1 --2- 172.21.15.76:0/2374501549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f207c004d00 0x7f209014a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.960+0000 7f209685f640 1 --2- 172.21.15.76:0/2374501549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2090155b00 0x7f2090155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.960+0000 7f209585d640 1 -- 172.21.15.76:0/2374501549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2090155b00 msgr2=0x7f2090155f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:09.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.960+0000 7f209585d640 1 --2- 172.21.15.76:0/2374501549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2090155b00 0x7f2090155f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.960+0000 7f209585d640 1 -- 172.21.15.76:0/2374501549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f207c004d00 msgr2=0x7f209014a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:09.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.960+0000 7f209585d640 1 --2- 172.21.15.76:0/2374501549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f207c004d00 0x7f209014a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:09.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.960+0000 7f209585d640 1 -- 172.21.15.76:0/2374501549 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2090156e80 con 0x7f2090157e60 2023-11-09T03:48:09.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.960+0000 7f209605e640 1 --2- 172.21.15.76:0/2374501549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f207c004d00 0x7f209014a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:09.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.960+0000 7f209585d640 1 --2- 172.21.15.76:0/2374501549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2090157e60 0x7f20901582e0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f2080001200 tx=0x7f208000ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:09.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.960+0000 7f2086ffd640 1 -- 172.21.15.76:0/2374501549 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2080004ab0 con 0x7f2090157e60 2023-11-09T03:48:09.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.960+0000 7f2086ffd640 1 -- 172.21.15.76:0/2374501549 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2080004c10 con 0x7f2090157e60 2023-11-09T03:48:09.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.960+0000 7f2086ffd640 1 -- 172.21.15.76:0/2374501549 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2080005050 con 0x7f2090157e60 2023-11-09T03:48:09.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.960+0000 7f20982c2640 1 -- 172.21.15.76:0/2374501549 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2090157160 con 0x7f2090157e60 2023-11-09T03:48:09.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.960+0000 7f20982c2640 1 -- 172.21.15.76:0/2374501549 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209014a900 con 0x7f2090157e60 2023-11-09T03:48:09.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.961+0000 7f20982c2640 1 -- 172.21.15.76:0/2374501549 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20900ae640 con 0x7f2090157e60 2023-11-09T03:48:09.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.962+0000 7f2086ffd640 1 -- 172.21.15.76:0/2374501549 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f20800051b0 con 0x7f2090157e60 2023-11-09T03:48:09.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.962+0000 7f2086ffd640 1 --2- 172.21.15.76:0/2374501549 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f207406aea0 0x7f207406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:09.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.962+0000 7f2086ffd640 1 -- 172.21.15.76:0/2374501549 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7148+0+0 (secure 0 0 0) 0x7f2080089670 con 0x7f2090157e60 2023-11-09T03:48:09.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.962+0000 7f209605e640 1 --2- 172.21.15.76:0/2374501549 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f207406aea0 0x7f207406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:09.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.963+0000 7f209605e640 1 --2- 172.21.15.76:0/2374501549 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f207406aea0 0x7f207406d360 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2088007520 tx=0x7f208800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:09.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:09.966+0000 7f2086ffd640 1 -- 172.21.15.76:0/2374501549 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2080054230 con 0x7f2090157e60 2023-11-09T03:48:10.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.170+0000 7f20982c2640 1 -- 172.21.15.76:0/2374501549 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2090074290 con 0x7f2090157e60 2023-11-09T03:48:10.170 INFO:tasks.ceph.mon.a.smithi076.stderr:2023-11-09T03:48:10.170+0000 7fe4a393d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-09T03:48:10.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.172+0000 7f2086ffd640 1 -- 172.21.15.76:0/2374501549 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2080057850 con 0x7f2090157e60 2023-11-09T03:48:10.171 INFO:teuthology.orchestra.run.smithi076.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-09T03:48:10.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.174+0000 7f20982c2640 1 -- 172.21.15.76:0/2374501549 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f207406aea0 msgr2=0x7f207406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:10.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.174+0000 7f20982c2640 1 --2- 172.21.15.76:0/2374501549 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f207406aea0 0x7f207406d360 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2088007520 tx=0x7f208800b040 comp rx=0 tx=0).stop 2023-11-09T03:48:10.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.174+0000 7f20982c2640 1 -- 172.21.15.76:0/2374501549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2090157e60 msgr2=0x7f20901582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:10.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.174+0000 7f20982c2640 1 --2- 172.21.15.76:0/2374501549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2090157e60 0x7f20901582e0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f2080001200 tx=0x7f208000ef80 comp rx=0 tx=0).stop 2023-11-09T03:48:10.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.174+0000 7f20982c2640 1 -- 172.21.15.76:0/2374501549 shutdown_connections 2023-11-09T03:48:10.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.174+0000 7f20982c2640 1 --2- 172.21.15.76:0/2374501549 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f207406aea0 0x7f207406d360 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:10.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.174+0000 7f20982c2640 1 --2- 172.21.15.76:0/2374501549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2090155b00 0x7f2090155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:10.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.174+0000 7f20982c2640 1 --2- 172.21.15.76:0/2374501549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2090157e60 0x7f20901582e0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:10.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.174+0000 7f20982c2640 1 --2- 172.21.15.76:0/2374501549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f207c004d00 0x7f209014a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:10.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.174+0000 7f20982c2640 1 -- 172.21.15.76:0/2374501549 >> 172.21.15.76:0/2374501549 conn(0x7f20900ce950 msgr2=0x7f20900c5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:10.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.175+0000 7f20982c2640 1 -- 172.21.15.76:0/2374501549 shutdown_connections 2023-11-09T03:48:10.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.175+0000 7f20982c2640 1 -- 172.21.15.76:0/2374501549 wait complete. 2023-11-09T03:48:10.186 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-09T03:48:10.199 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2023-11-09T03:48:10.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.289+0000 7fcc20555640 1 Processor -- start 2023-11-09T03:48:10.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.289+0000 7fcc20555640 1 -- start start 2023-11-09T03:48:10.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.290+0000 7fcc20555640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcc180c4ff0 0x7fcc180cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:10.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.290+0000 7fcc20555640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc1815aff0 con 0x7fcc180adac0 2023-11-09T03:48:10.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.290+0000 7fcc20555640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc1815a3e0 con 0x7fcc18159c00 2023-11-09T03:48:10.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.290+0000 7fcc20555640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc1815a6c0 con 0x7fcc180c4ff0 2023-11-09T03:48:10.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.290+0000 7fcc1eaf2640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fcc180adac0 0x7fcc180adf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:60094/0 (socket says 172.21.15.76:60094) 2023-11-09T03:48:10.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.290+0000 7fcc1eaf2640 1 -- 172.21.15.76:0/2803239004 learned_addr learned my addr 172.21.15.76:0/2803239004 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:10.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.290+0000 7fcc1d2ef640 1 -- 172.21.15.76:0/2803239004 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3954821604 0 0) 0x7fcc1815aff0 con 0x7fcc180adac0 2023-11-09T03:48:10.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.290+0000 7fcc1daf0640 1 --2- 172.21.15.76:0/2803239004 >> v2:172.21.15.113:3300/0 conn(0x7fcc180c4ff0 0x7fcc180cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:10.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.290+0000 7fcc1d2ef640 1 -- 172.21.15.76:0/2803239004 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc04003520 con 0x7fcc180adac0 2023-11-09T03:48:10.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.291+0000 7fcc1d2ef640 1 -- 172.21.15.76:0/2803239004 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3816025166 0 0) 0x7fcc04003520 con 0x7fcc180adac0 2023-11-09T03:48:10.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.291+0000 7fcc1d2ef640 1 -- 172.21.15.76:0/2803239004 >> v2:172.21.15.113:3300/0 conn(0x7fcc180c4ff0 msgr2=0x7fcc180cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:10.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.291+0000 7fcc1d2ef640 1 --2- 172.21.15.76:0/2803239004 >> v2:172.21.15.113:3300/0 conn(0x7fcc180c4ff0 0x7fcc180cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:10.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.291+0000 7fcc1d2ef640 1 -- 172.21.15.76:0/2803239004 >> v1:172.21.15.113:6789/0 conn(0x7fcc18159c00 legacy=0x7fcc180c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:10.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.291+0000 7fcc1d2ef640 1 -- 172.21.15.76:0/2803239004 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1815a9a0 con 0x7fcc180adac0 2023-11-09T03:48:10.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.291+0000 7fcc1d2ef640 1 -- 172.21.15.76:0/2803239004 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fcc100019f0 con 0x7fcc180adac0 2023-11-09T03:48:10.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.291+0000 7fcc1d2ef640 1 -- 172.21.15.76:0/2803239004 >> v1:172.21.15.76:6789/0 conn(0x7fcc180adac0 legacy=0x7fcc180adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:10.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.291+0000 7fcc1d2ef640 1 --2- 172.21.15.76:0/2803239004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc04003970 0x7fcc04003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:10.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.291+0000 7fcc1d2ef640 1 --2- 172.21.15.76:0/2803239004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc040047c0 0x7fcc04004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:10.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.291+0000 7fcc1d2ef640 1 --2- 172.21.15.76:0/2803239004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc04005180 0x7fcc0400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:10.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.291+0000 7fcc1d2ef640 1 -- 172.21.15.76:0/2803239004 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1815a6c0 con 0x7fcc040047c0 2023-11-09T03:48:10.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.291+0000 7fcc1d2ef640 1 -- 172.21.15.76:0/2803239004 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0400c020 con 0x7fcc04003970 2023-11-09T03:48:10.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.291+0000 7fcc1d2ef640 1 -- 172.21.15.76:0/2803239004 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0400c1e0 con 0x7fcc04005180 2023-11-09T03:48:10.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.291+0000 7fcc1daf0640 1 --2- 172.21.15.76:0/2803239004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc040047c0 0x7fcc04004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:10.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.291+0000 7fcc1e2f1640 1 --2- 172.21.15.76:0/2803239004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc04003970 0x7fcc04003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:10.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.291+0000 7fcc1eaf2640 1 --2- 172.21.15.76:0/2803239004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc04005180 0x7fcc0400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:10.298 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.291+0000 7fcc1daf0640 1 -- 172.21.15.76:0/2803239004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc04005180 msgr2=0x7fcc0400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:10.298 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.291+0000 7fcc1daf0640 1 --2- 172.21.15.76:0/2803239004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc04005180 0x7fcc0400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:10.298 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.291+0000 7fcc1daf0640 1 -- 172.21.15.76:0/2803239004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc04003970 msgr2=0x7fcc04003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:10.298 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.291+0000 7fcc1daf0640 1 --2- 172.21.15.76:0/2803239004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc04003970 0x7fcc04003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:10.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.291+0000 7fcc1daf0640 1 -- 172.21.15.76:0/2803239004 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc04003520 con 0x7fcc040047c0 2023-11-09T03:48:10.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.291+0000 7fcc1eaf2640 1 -- 172.21.15.76:0/2803239004 reap_dead start 2023-11-09T03:48:10.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.291+0000 7fcc1e2f1640 1 --2- 172.21.15.76:0/2803239004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc04003970 0x7fcc04003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:10.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.292+0000 7fcc1daf0640 1 --2- 172.21.15.76:0/2803239004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc040047c0 0x7fcc04004c40 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fcc180cfa50 tx=0x7fcc040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee57b253c5cec68 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:10.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.292+0000 7fcc1d2ef640 1 -- 172.21.15.76:0/2803239004 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1402c630 con 0x7fcc040047c0 2023-11-09T03:48:10.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.292+0000 7fcc1d2ef640 1 -- 172.21.15.76:0/2803239004 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1402c790 con 0x7fcc040047c0 2023-11-09T03:48:10.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.292+0000 7fcc1d2ef640 1 -- 172.21.15.76:0/2803239004 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1402cb40 con 0x7fcc040047c0 2023-11-09T03:48:10.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.293+0000 7fcc20555640 1 -- 172.21.15.76:0/2803239004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc040047c0 msgr2=0x7fcc04004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:10.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.293+0000 7fcc20555640 1 --2- 172.21.15.76:0/2803239004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc040047c0 0x7fcc04004c40 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fcc180cfa50 tx=0x7fcc040057f0 comp rx=0 tx=0).stop 2023-11-09T03:48:10.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.293+0000 7fcc20555640 1 -- 172.21.15.76:0/2803239004 shutdown_connections 2023-11-09T03:48:10.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.293+0000 7fcc20555640 1 --2- 172.21.15.76:0/2803239004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc040047c0 0x7fcc04004c40 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:10.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.293+0000 7fcc20555640 1 -- 172.21.15.76:0/2803239004 >> 172.21.15.76:0/2803239004 conn(0x7fcc180ce950 msgr2=0x7fcc180ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:10.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.293+0000 7fcc20555640 1 -- 172.21.15.76:0/2803239004 shutdown_connections 2023-11-09T03:48:10.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.293+0000 7fcc20555640 1 -- 172.21.15.76:0/2803239004 wait complete. 2023-11-09T03:48:10.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.294+0000 7fcc20555640 1 Processor -- start 2023-11-09T03:48:10.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.294+0000 7fcc20555640 1 -- start start 2023-11-09T03:48:10.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.294+0000 7fcc20555640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc040047c0 0x7fcc1814f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:10.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.295+0000 7fcc20555640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc18157140 0x7fcc181575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:10.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.295+0000 7fcc20555640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc1806d760 0x7fcc1806dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:10.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.295+0000 7fcc20555640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc181f4830 con 0x7fcc040047c0 2023-11-09T03:48:10.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.295+0000 7fcc20555640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc18157b00 con 0x7fcc18157140 2023-11-09T03:48:10.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.295+0000 7fcc20555640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1806e120 con 0x7fcc1806d760 2023-11-09T03:48:10.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.295+0000 7fcc1e2f1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc040047c0 0x7fcc1814f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:10.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.295+0000 7fcc1e2f1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc040047c0 0x7fcc1814f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50998/0 (socket says 172.21.15.76:50998) 2023-11-09T03:48:10.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.295+0000 7fcc1e2f1640 1 -- 172.21.15.76:0/635426232 learned_addr learned my addr 172.21.15.76:0/635426232 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:10.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.295+0000 7fcc1daf0640 1 --2- 172.21.15.76:0/635426232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc18157140 0x7fcc181575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:10.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.295+0000 7fcc1eaf2640 1 --2- 172.21.15.76:0/635426232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc1806d760 0x7fcc1806dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:10.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.295+0000 7fcc1e2f1640 1 -- 172.21.15.76:0/635426232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc1806d760 msgr2=0x7fcc1806dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:10.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.295+0000 7fcc1e2f1640 1 --2- 172.21.15.76:0/635426232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc1806d760 0x7fcc1806dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:10.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.295+0000 7fcc1e2f1640 1 -- 172.21.15.76:0/635426232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc18157140 msgr2=0x7fcc181575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:10.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.295+0000 7fcc1e2f1640 1 --2- 172.21.15.76:0/635426232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc18157140 0x7fcc181575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:10.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.295+0000 7fcc1e2f1640 1 -- 172.21.15.76:0/635426232 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc181530b0 con 0x7fcc040047c0 2023-11-09T03:48:10.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.295+0000 7fcc1daf0640 1 --2- 172.21.15.76:0/635426232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc18157140 0x7fcc181575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:10.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.295+0000 7fcc1e2f1640 1 --2- 172.21.15.76:0/635426232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc040047c0 0x7fcc1814f250 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800e4e0 tx=0x7fcc0800e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:10.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.295+0000 7fcc0effd640 1 -- 172.21.15.76:0/635426232 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc08017070 con 0x7fcc040047c0 2023-11-09T03:48:10.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.295+0000 7fcc0effd640 1 -- 172.21.15.76:0/635426232 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc08007e90 con 0x7fcc040047c0 2023-11-09T03:48:10.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.295+0000 7fcc20555640 1 -- 172.21.15.76:0/635426232 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc18157d20 con 0x7fcc040047c0 2023-11-09T03:48:10.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.295+0000 7fcc0effd640 1 -- 172.21.15.76:0/635426232 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc080052e0 con 0x7fcc040047c0 2023-11-09T03:48:10.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.295+0000 7fcc20555640 1 -- 172.21.15.76:0/635426232 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc181f3d50 con 0x7fcc040047c0 2023-11-09T03:48:10.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.296+0000 7fcc20555640 1 -- 172.21.15.76:0/635426232 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc18074290 con 0x7fcc040047c0 2023-11-09T03:48:10.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.297+0000 7fcc0effd640 1 -- 172.21.15.76:0/635426232 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcc08007020 con 0x7fcc040047c0 2023-11-09T03:48:10.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.297+0000 7fcc0effd640 1 --2- 172.21.15.76:0/635426232 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcbfc06aea0 0x7fcbfc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:10.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.297+0000 7fcc0effd640 1 -- 172.21.15.76:0/635426232 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7206+0+0 (secure 0 0 0) 0x7fcc080890b0 con 0x7fcc040047c0 2023-11-09T03:48:10.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.297+0000 7fcc1daf0640 1 --2- 172.21.15.76:0/635426232 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcbfc06aea0 0x7fcbfc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:10.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.298+0000 7fcc1daf0640 1 --2- 172.21.15.76:0/635426232 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcbfc06aea0 0x7fcbfc06d360 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcc14036000 tx=0x7fcc14030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:10.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.302+0000 7fcc0effd640 1 -- 172.21.15.76:0/635426232 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcc08053c10 con 0x7fcc040047c0 2023-11-09T03:48:10.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:10.503+0000 7fcc20555640 1 -- 172.21.15.76:0/635426232 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcc180ca8d0 con 0x7fcc040047c0 2023-11-09T03:48:11.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.172+0000 7fcc0effd640 1 -- 172.21.15.76:0/635426232 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcc08057230 con 0x7fcc040047c0 2023-11-09T03:48:11.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.174+0000 7fcc20555640 1 -- 172.21.15.76:0/635426232 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcbfc06aea0 msgr2=0x7fcbfc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:11.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.174+0000 7fcc20555640 1 --2- 172.21.15.76:0/635426232 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcbfc06aea0 0x7fcbfc06d360 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcc14036000 tx=0x7fcc14030000 comp rx=0 tx=0).stop 2023-11-09T03:48:11.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.174+0000 7fcc20555640 1 -- 172.21.15.76:0/635426232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc040047c0 msgr2=0x7fcc1814f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:11.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.174+0000 7fcc20555640 1 --2- 172.21.15.76:0/635426232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc040047c0 0x7fcc1814f250 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800e4e0 tx=0x7fcc0800e9a0 comp rx=0 tx=0).stop 2023-11-09T03:48:11.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.175+0000 7fcc20555640 1 -- 172.21.15.76:0/635426232 shutdown_connections 2023-11-09T03:48:11.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.175+0000 7fcc20555640 1 --2- 172.21.15.76:0/635426232 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcbfc06aea0 0x7fcbfc06d360 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:11.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.175+0000 7fcc20555640 1 --2- 172.21.15.76:0/635426232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc1806d760 0x7fcc1806dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:11.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.175+0000 7fcc20555640 1 --2- 172.21.15.76:0/635426232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc18157140 0x7fcc181575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:11.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.175+0000 7fcc20555640 1 --2- 172.21.15.76:0/635426232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc040047c0 0x7fcc1814f250 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:11.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.175+0000 7fcc20555640 1 -- 172.21.15.76:0/635426232 >> 172.21.15.76:0/635426232 conn(0x7fcc180ce950 msgr2=0x7fcc1815afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:11.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.175+0000 7fcc20555640 1 -- 172.21.15.76:0/635426232 shutdown_connections 2023-11-09T03:48:11.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.175+0000 7fcc20555640 1 -- 172.21.15.76:0/635426232 wait complete. 2023-11-09T03:48:11.188 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-09T03:48:11.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.336+0000 7f80ba4a2640 1 Processor -- start 2023-11-09T03:48:11.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.336+0000 7f80ba4a2640 1 -- start start 2023-11-09T03:48:11.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.337+0000 7f80ba4a2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f80b4151ea0 0x7f80b415b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:11.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.337+0000 7f80ba4a2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f80b415f510 0x7f80b40cb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:11.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.337+0000 7f80ba4a2640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b406cf90 con 0x7f80b406dd50 2023-11-09T03:48:11.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.337+0000 7f80ba4a2640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b406d270 con 0x7f80b415f510 2023-11-09T03:48:11.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.337+0000 7f80ba4a2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b406d550 con 0x7f80b4151ea0 2023-11-09T03:48:11.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.337+0000 7f80b37fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f80b415f510 0x7f80b40cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:11.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.337+0000 7f80b37fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f80b415f510 0x7f80b40cb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51014/0 (socket says 172.21.15.76:51014) 2023-11-09T03:48:11.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.337+0000 7f80b37fe640 1 -- 172.21.15.76:0/1217533231 learned_addr learned my addr 172.21.15.76:0/1217533231 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:11.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.337+0000 7f80b37fe640 1 -- 172.21.15.76:0/1217533231 >> v2:172.21.15.113:3300/0 conn(0x7f80b4151ea0 msgr2=0x7f80b415b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:11.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.337+0000 7f80b3fff640 1 --2- 172.21.15.76:0/1217533231 >> v2:172.21.15.113:3300/0 conn(0x7f80b4151ea0 0x7f80b415b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:11.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.337+0000 7f80b37fe640 1 --2- 172.21.15.76:0/1217533231 >> v2:172.21.15.113:3300/0 conn(0x7f80b4151ea0 0x7f80b415b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:11.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.337+0000 7f80b37fe640 1 -- 172.21.15.76:0/1217533231 >> v1:172.21.15.113:6789/0 conn(0x7f80b406dd50 legacy=0x7f80b4070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:11.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.337+0000 7f80b37fe640 1 -- 172.21.15.76:0/1217533231 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b406d830 con 0x7f80b415f510 2023-11-09T03:48:11.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.337+0000 7f80b3fff640 1 --2- 172.21.15.76:0/1217533231 >> v2:172.21.15.113:3300/0 conn(0x7f80b4151ea0 0x7f80b415b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:11.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.338+0000 7f80b37fe640 1 --2- 172.21.15.76:0/1217533231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80b415f510 0x7f80b40cb150 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f80b40cfac0 tx=0x7f80a802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a3a745ede8337d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:11.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.338+0000 7f80b2ffd640 1 -- 172.21.15.76:0/1217533231 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8031870 con 0x7f80b415f510 2023-11-09T03:48:11.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.338+0000 7f80b2ffd640 1 -- 172.21.15.76:0/1217533231 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a80319d0 con 0x7f80b415f510 2023-11-09T03:48:11.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.338+0000 7f80b2ffd640 1 -- 172.21.15.76:0/1217533231 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8031d60 con 0x7f80b415f510 2023-11-09T03:48:11.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.339+0000 7f80ba4a2640 1 -- 172.21.15.76:0/1217533231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80b415f510 msgr2=0x7f80b40cb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:11.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.339+0000 7f80ba4a2640 1 --2- 172.21.15.76:0/1217533231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80b415f510 0x7f80b40cb150 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f80b40cfac0 tx=0x7f80a802b910 comp rx=0 tx=0).stop 2023-11-09T03:48:11.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.339+0000 7f80ba4a2640 1 -- 172.21.15.76:0/1217533231 shutdown_connections 2023-11-09T03:48:11.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.339+0000 7f80ba4a2640 1 --2- 172.21.15.76:0/1217533231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80b415f510 0x7f80b40cb150 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:11.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.339+0000 7f80ba4a2640 1 --2- 172.21.15.76:0/1217533231 >> v2:172.21.15.113:3300/0 conn(0x7f80b4151ea0 0x7f80b415b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:11.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.339+0000 7f80ba4a2640 1 -- 172.21.15.76:0/1217533231 >> 172.21.15.76:0/1217533231 conn(0x7f80b40b05d0 msgr2=0x7f80b40b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:11.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.339+0000 7f80ba4a2640 1 -- 172.21.15.76:0/1217533231 shutdown_connections 2023-11-09T03:48:11.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.339+0000 7f80ba4a2640 1 -- 172.21.15.76:0/1217533231 wait complete. 2023-11-09T03:48:11.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.340+0000 7f80ba4a2640 1 Processor -- start 2023-11-09T03:48:11.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.340+0000 7f80ba4a2640 1 -- start start 2023-11-09T03:48:11.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.341+0000 7f80ba4a2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80b406dd50 0x7f80b40b2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:11.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.341+0000 7f80ba4a2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80b4151ea0 0x7f80b40c6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:11.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.341+0000 7f80ba4a2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80b415f510 0x7f80b41f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:11.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.341+0000 7f80ba4a2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b40b2560 con 0x7f80b406dd50 2023-11-09T03:48:11.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.341+0000 7f80ba4a2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b41f6c40 con 0x7f80b4151ea0 2023-11-09T03:48:11.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.341+0000 7f80ba4a2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b40c6c90 con 0x7f80b415f510 2023-11-09T03:48:11.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.341+0000 7f80b3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80b406dd50 0x7f80b40b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:11.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.341+0000 7f80b3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80b406dd50 0x7f80b40b2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51018/0 (socket says 172.21.15.76:51018) 2023-11-09T03:48:11.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.341+0000 7f80b3fff640 1 -- 172.21.15.76:0/1582398232 learned_addr learned my addr 172.21.15.76:0/1582398232 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:11.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.341+0000 7f80b37fe640 1 --2- 172.21.15.76:0/1582398232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80b4151ea0 0x7f80b40c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:11.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.341+0000 7f80b8a3f640 1 --2- 172.21.15.76:0/1582398232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80b415f510 0x7f80b41f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:11.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.341+0000 7f80b3fff640 1 -- 172.21.15.76:0/1582398232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80b415f510 msgr2=0x7f80b41f1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:11.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.341+0000 7f80b3fff640 1 --2- 172.21.15.76:0/1582398232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80b415f510 0x7f80b41f1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:11.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.341+0000 7f80b3fff640 1 -- 172.21.15.76:0/1582398232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80b4151ea0 msgr2=0x7f80b40c6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:11.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.341+0000 7f80b3fff640 1 --2- 172.21.15.76:0/1582398232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80b4151ea0 0x7f80b40c6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:11.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.341+0000 7f80b3fff640 1 -- 172.21.15.76:0/1582398232 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a80094d0 con 0x7f80b406dd50 2023-11-09T03:48:11.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.341+0000 7f80b37fe640 1 --2- 172.21.15.76:0/1582398232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80b4151ea0 0x7f80b40c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:11.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.341+0000 7f80b8a3f640 1 --2- 172.21.15.76:0/1582398232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80b415f510 0x7f80b41f1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:11.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.341+0000 7f80b3fff640 1 --2- 172.21.15.76:0/1582398232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80b406dd50 0x7f80b40b2020 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f80a400c820 tx=0x7f80a400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:11.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.342+0000 7f80b0ff9640 1 -- 172.21.15.76:0/1582398232 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a4007a30 con 0x7f80b406dd50 2023-11-09T03:48:11.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.342+0000 7f80b0ff9640 1 -- 172.21.15.76:0/1582398232 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a4007b90 con 0x7f80b406dd50 2023-11-09T03:48:11.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.342+0000 7f80b0ff9640 1 -- 172.21.15.76:0/1582398232 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a400ce70 con 0x7f80b406dd50 2023-11-09T03:48:11.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.342+0000 7f80ba4a2640 1 -- 172.21.15.76:0/1582398232 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b41f5fc0 con 0x7f80b406dd50 2023-11-09T03:48:11.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.342+0000 7f80ba4a2640 1 -- 172.21.15.76:0/1582398232 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b41f6580 con 0x7f80b406dd50 2023-11-09T03:48:11.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.343+0000 7f80ba4a2640 1 -- 172.21.15.76:0/1582398232 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b404e490 con 0x7f80b406dd50 2023-11-09T03:48:11.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.343+0000 7f80b0ff9640 1 -- 172.21.15.76:0/1582398232 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f80a40113f0 con 0x7f80b406dd50 2023-11-09T03:48:11.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.343+0000 7f80b0ff9640 1 --2- 172.21.15.76:0/1582398232 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f80a006ae70 0x7f80a006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:11.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.343+0000 7f80b0ff9640 1 -- 172.21.15.76:0/1582398232 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7206+0+0 (secure 0 0 0) 0x7f80a4089d00 con 0x7f80b406dd50 2023-11-09T03:48:11.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.344+0000 7f80b37fe640 1 --2- 172.21.15.76:0/1582398232 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f80a006ae70 0x7f80a006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:11.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.344+0000 7f80b37fe640 1 --2- 172.21.15.76:0/1582398232 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f80a006ae70 0x7f80a006d330 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f80a8036000 tx=0x7f80a803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:11.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.348+0000 7f80b0ff9640 1 -- 172.21.15.76:0/1582398232 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f80a4054860 con 0x7f80b406dd50 2023-11-09T03:48:11.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:11.517+0000 7f80ba4a2640 1 -- 172.21.15.76:0/1582398232 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f80b415e210 con 0x7f80b406dd50 2023-11-09T03:48:12.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.486+0000 7f80b0ff9640 1 -- 172.21.15.76:0/1582398232 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v73) v1 ==== 181+0+0 (secure 0 0 0) 0x7f80a401f0d0 con 0x7f80b406dd50 2023-11-09T03:48:12.487 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' removed 2023-11-09T03:48:12.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.490+0000 7f80ba4a2640 1 -- 172.21.15.76:0/1582398232 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f80a006ae70 msgr2=0x7f80a006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:12.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.490+0000 7f80ba4a2640 1 --2- 172.21.15.76:0/1582398232 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f80a006ae70 0x7f80a006d330 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f80a8036000 tx=0x7f80a803c040 comp rx=0 tx=0).stop 2023-11-09T03:48:12.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.490+0000 7f80ba4a2640 1 -- 172.21.15.76:0/1582398232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80b406dd50 msgr2=0x7f80b40b2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:12.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.490+0000 7f80ba4a2640 1 --2- 172.21.15.76:0/1582398232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80b406dd50 0x7f80b40b2020 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f80a400c820 tx=0x7f80a400cc20 comp rx=0 tx=0).stop 2023-11-09T03:48:12.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.491+0000 7f80ba4a2640 1 -- 172.21.15.76:0/1582398232 shutdown_connections 2023-11-09T03:48:12.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.491+0000 7f80ba4a2640 1 --2- 172.21.15.76:0/1582398232 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f80a006ae70 0x7f80a006d330 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:12.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.491+0000 7f80ba4a2640 1 --2- 172.21.15.76:0/1582398232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80b415f510 0x7f80b41f1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:12.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.491+0000 7f80ba4a2640 1 --2- 172.21.15.76:0/1582398232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80b4151ea0 0x7f80b40c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:12.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.491+0000 7f80ba4a2640 1 --2- 172.21.15.76:0/1582398232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80b406dd50 0x7f80b40b2020 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:12.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.491+0000 7f80ba4a2640 1 -- 172.21.15.76:0/1582398232 >> 172.21.15.76:0/1582398232 conn(0x7f80b40b05d0 msgr2=0x7f80b40ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:12.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.491+0000 7f80ba4a2640 1 -- 172.21.15.76:0/1582398232 shutdown_connections 2023-11-09T03:48:12.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.491+0000 7f80ba4a2640 1 -- 172.21.15.76:0/1582398232 wait complete. 2023-11-09T03:48:12.506 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-09T03:48:12.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.615+0000 7f636054c640 1 Processor -- start 2023-11-09T03:48:12.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.615+0000 7f636054c640 1 -- start start 2023-11-09T03:48:12.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.616+0000 7f636054c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6358159be0 0x7f63580c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:12.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.616+0000 7f636054c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63580c5890 con 0x7f635815e420 2023-11-09T03:48:12.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.616+0000 7f636054c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63580cd8e0 con 0x7f63580ade40 2023-11-09T03:48:12.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.616+0000 7f636054c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63580cdbc0 con 0x7f6358159be0 2023-11-09T03:48:12.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.616+0000 7f635dae7640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f635815e420 0x7f63580ad730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40284/0 (socket says 172.21.15.76:40284) 2023-11-09T03:48:12.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.616+0000 7f635dae7640 1 -- 172.21.15.76:0/4293551316 learned_addr learned my addr 172.21.15.76:0/4293551316 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:12.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.616+0000 7f635e2e8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6358159be0 0x7f63580c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:12.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.617+0000 7f635e2e8640 1 -- 172.21.15.76:0/4293551316 >> v1:172.21.15.113:6789/0 conn(0x7f63580ade40 legacy=0x7f63580c3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:12.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.617+0000 7f635e2e8640 1 -- 172.21.15.76:0/4293551316 >> v1:172.21.15.76:6789/0 conn(0x7f635815e420 legacy=0x7f63580ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:12.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.617+0000 7f635e2e8640 1 -- 172.21.15.76:0/4293551316 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63580cdea0 con 0x7f6358159be0 2023-11-09T03:48:12.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.617+0000 7f635e2e8640 1 --2- 172.21.15.76:0/4293551316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6358159be0 0x7f63580c4f60 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f6354009d90 tx=0x7f635402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2cbc0e8129fd35d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:12.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.617+0000 7f635d2e6640 1 -- 172.21.15.76:0/4293551316 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635402cc60 con 0x7f6358159be0 2023-11-09T03:48:12.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.617+0000 7f635d2e6640 1 -- 172.21.15.76:0/4293551316 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635402cdc0 con 0x7f6358159be0 2023-11-09T03:48:12.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.617+0000 7f635d2e6640 1 -- 172.21.15.76:0/4293551316 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6354034660 con 0x7f6358159be0 2023-11-09T03:48:12.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.618+0000 7f636054c640 1 -- 172.21.15.76:0/4293551316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6358159be0 msgr2=0x7f63580c4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:12.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.618+0000 7f636054c640 1 --2- 172.21.15.76:0/4293551316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6358159be0 0x7f63580c4f60 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f6354009d90 tx=0x7f635402beb0 comp rx=0 tx=0).stop 2023-11-09T03:48:12.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.618+0000 7f636054c640 1 -- 172.21.15.76:0/4293551316 shutdown_connections 2023-11-09T03:48:12.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.618+0000 7f636054c640 1 --2- 172.21.15.76:0/4293551316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6358159be0 0x7f63580c4f60 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:12.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.618+0000 7f636054c640 1 -- 172.21.15.76:0/4293551316 >> 172.21.15.76:0/4293551316 conn(0x7f63580b05d0 msgr2=0x7f63580b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:12.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.618+0000 7f636054c640 1 -- 172.21.15.76:0/4293551316 shutdown_connections 2023-11-09T03:48:12.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.618+0000 7f636054c640 1 -- 172.21.15.76:0/4293551316 wait complete. 2023-11-09T03:48:12.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.619+0000 7f636054c640 1 Processor -- start 2023-11-09T03:48:12.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.619+0000 7f636054c640 1 -- start start 2023-11-09T03:48:12.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.620+0000 7f636054c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63580ade40 0x7f6358149500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:12.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.620+0000 7f636054c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6358159be0 0x7f635815ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:12.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.620+0000 7f636054c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f635815e420 0x7f635815cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:12.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.620+0000 7f636054c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6358149a40 con 0x7f635815e420 2023-11-09T03:48:12.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.620+0000 7f636054c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635815b7c0 con 0x7f6358159be0 2023-11-09T03:48:12.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.620+0000 7f636054c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635815ba60 con 0x7f63580ade40 2023-11-09T03:48:12.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.620+0000 7f635eae9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f635815e420 0x7f635815cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:12.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.620+0000 7f635eae9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f635815e420 0x7f635815cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51188/0 (socket says 172.21.15.76:51188) 2023-11-09T03:48:12.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.620+0000 7f635eae9640 1 -- 172.21.15.76:0/3097200446 learned_addr learned my addr 172.21.15.76:0/3097200446 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:12.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.620+0000 7f635dae7640 1 --2- 172.21.15.76:0/3097200446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6358159be0 0x7f635815ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:12.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.620+0000 7f635eae9640 1 -- 172.21.15.76:0/3097200446 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63580ade40 msgr2=0x7f6358149500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:12.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.620+0000 7f635eae9640 1 --2- 172.21.15.76:0/3097200446 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63580ade40 0x7f6358149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:12.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.620+0000 7f635eae9640 1 -- 172.21.15.76:0/3097200446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6358159be0 msgr2=0x7f635815ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:12.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.620+0000 7f635eae9640 1 --2- 172.21.15.76:0/3097200446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6358159be0 0x7f635815ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:12.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.620+0000 7f635eae9640 1 -- 172.21.15.76:0/3097200446 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6354009a70 con 0x7f635815e420 2023-11-09T03:48:12.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.621+0000 7f635eae9640 1 --2- 172.21.15.76:0/3097200446 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f635815e420 0x7f635815cb00 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f6350001e00 tx=0x7f635000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:12.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.621+0000 7f634effd640 1 -- 172.21.15.76:0/3097200446 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63500052d0 con 0x7f635815e420 2023-11-09T03:48:12.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.621+0000 7f634effd640 1 -- 172.21.15.76:0/3097200446 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6350005430 con 0x7f635815e420 2023-11-09T03:48:12.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.621+0000 7f636054c640 1 -- 172.21.15.76:0/3097200446 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f635815bd40 con 0x7f635815e420 2023-11-09T03:48:12.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.621+0000 7f634effd640 1 -- 172.21.15.76:0/3097200446 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63500108e0 con 0x7f635815e420 2023-11-09T03:48:12.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.621+0000 7f636054c640 1 -- 172.21.15.76:0/3097200446 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f635815c300 con 0x7f635815e420 2023-11-09T03:48:12.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.622+0000 7f636054c640 1 -- 172.21.15.76:0/3097200446 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f635804e490 con 0x7f635815e420 2023-11-09T03:48:12.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.622+0000 7f634effd640 1 -- 172.21.15.76:0/3097200446 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6350010a40 con 0x7f635815e420 2023-11-09T03:48:12.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.622+0000 7f634effd640 1 --2- 172.21.15.76:0/3097200446 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f633c06ae70 0x7f633c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:12.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.623+0000 7f634effd640 1 -- 172.21.15.76:0/3097200446 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6798+0+0 (secure 0 0 0) 0x7f635008b7c0 con 0x7f635815e420 2023-11-09T03:48:12.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.623+0000 7f635e2e8640 1 --2- 172.21.15.76:0/3097200446 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f633c06ae70 0x7f633c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:12.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.623+0000 7f635e2e8640 1 --2- 172.21.15.76:0/3097200446 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f633c06ae70 0x7f633c06d330 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6354036000 tx=0x7f635402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:12.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.627+0000 7f634effd640 1 -- 172.21.15.76:0/3097200446 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6350056430 con 0x7f635815e420 2023-11-09T03:48:12.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:12.794+0000 7f636054c640 1 -- 172.21.15.76:0/3097200446 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f63580c4d40 con 0x7f635815e420 2023-11-09T03:48:13.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.491+0000 7f634effd640 1 -- 172.21.15.76:0/3097200446 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v74) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6350059a50 con 0x7f635815e420 2023-11-09T03:48:13.492 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' removed 2023-11-09T03:48:13.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.494+0000 7f636054c640 1 -- 172.21.15.76:0/3097200446 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f633c06ae70 msgr2=0x7f633c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:13.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.494+0000 7f636054c640 1 --2- 172.21.15.76:0/3097200446 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f633c06ae70 0x7f633c06d330 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6354036000 tx=0x7f635402e000 comp rx=0 tx=0).stop 2023-11-09T03:48:13.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.494+0000 7f636054c640 1 -- 172.21.15.76:0/3097200446 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f635815e420 msgr2=0x7f635815cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:13.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.494+0000 7f636054c640 1 --2- 172.21.15.76:0/3097200446 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f635815e420 0x7f635815cb00 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f6350001e00 tx=0x7f635000c850 comp rx=0 tx=0).stop 2023-11-09T03:48:13.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.494+0000 7f636054c640 1 -- 172.21.15.76:0/3097200446 shutdown_connections 2023-11-09T03:48:13.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.494+0000 7f636054c640 1 --2- 172.21.15.76:0/3097200446 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f633c06ae70 0x7f633c06d330 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:13.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.494+0000 7f636054c640 1 --2- 172.21.15.76:0/3097200446 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f635815e420 0x7f635815cb00 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:13.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.494+0000 7f636054c640 1 --2- 172.21.15.76:0/3097200446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6358159be0 0x7f635815ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:13.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.494+0000 7f636054c640 1 --2- 172.21.15.76:0/3097200446 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63580ade40 0x7f6358149500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:13.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.494+0000 7f636054c640 1 -- 172.21.15.76:0/3097200446 >> 172.21.15.76:0/3097200446 conn(0x7f63580b05d0 msgr2=0x7f63580b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:13.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.494+0000 7f636054c640 1 -- 172.21.15.76:0/3097200446 shutdown_connections 2023-11-09T03:48:13.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.494+0000 7f636054c640 1 -- 172.21.15.76:0/3097200446 wait complete. 2023-11-09T03:48:13.507 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-09T03:48:13.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.615+0000 7f9691244640 1 Processor -- start 2023-11-09T03:48:13.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.616+0000 7f9691244640 1 -- start start 2023-11-09T03:48:13.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.616+0000 7f9691244640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f968c159c00 0x7f968c0cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:13.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.616+0000 7f9691244640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f968c0c48e0 0x7f968c0c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:13.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.616+0000 7f9691244640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f968c15aff0 con 0x7f968c0adac0 2023-11-09T03:48:13.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.616+0000 7f9691244640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f968c0c52d0 con 0x7f968c0c48e0 2023-11-09T03:48:13.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.616+0000 7f9691244640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f968c15a3e0 con 0x7f968c159c00 2023-11-09T03:48:13.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.616+0000 7f968a59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f968c0c48e0 0x7f968c0c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:13.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.616+0000 7f968a59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f968c0c48e0 0x7f968c0c4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51194/0 (socket says 172.21.15.76:51194) 2023-11-09T03:48:13.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.616+0000 7f968a59c640 1 -- 172.21.15.76:0/2834731881 learned_addr learned my addr 172.21.15.76:0/2834731881 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:13.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.617+0000 7f968ad9d640 1 --2- 172.21.15.76:0/2834731881 >> v2:172.21.15.113:3300/0 conn(0x7f968c159c00 0x7f968c0cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:13.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.617+0000 7f968a59c640 1 -- 172.21.15.76:0/2834731881 >> v2:172.21.15.113:3300/0 conn(0x7f968c159c00 msgr2=0x7f968c0cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:13.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.617+0000 7f968a59c640 1 --2- 172.21.15.76:0/2834731881 >> v2:172.21.15.113:3300/0 conn(0x7f968c159c00 0x7f968c0cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:13.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.617+0000 7f968a59c640 1 -- 172.21.15.76:0/2834731881 >> v1:172.21.15.113:6789/0 conn(0x7f968c0adac0 legacy=0x7f968c0adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:13.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.617+0000 7f968a59c640 1 -- 172.21.15.76:0/2834731881 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968c15a6c0 con 0x7f968c0c48e0 2023-11-09T03:48:13.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.617+0000 7f968a59c640 1 --2- 172.21.15.76:0/2834731881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f968c0c48e0 0x7f968c0c4d90 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f968c0cc4d0 tx=0x7f967c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6caad230d32c466 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:13.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.618+0000 7f9689d9b640 1 -- 172.21.15.76:0/2834731881 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c031870 con 0x7f968c0c48e0 2023-11-09T03:48:13.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.618+0000 7f9689d9b640 1 -- 172.21.15.76:0/2834731881 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967c0319d0 con 0x7f968c0c48e0 2023-11-09T03:48:13.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.618+0000 7f9689d9b640 1 -- 172.21.15.76:0/2834731881 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c031d60 con 0x7f968c0c48e0 2023-11-09T03:48:13.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.618+0000 7f9691244640 1 -- 172.21.15.76:0/2834731881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f968c0c48e0 msgr2=0x7f968c0c4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:13.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.618+0000 7f9691244640 1 --2- 172.21.15.76:0/2834731881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f968c0c48e0 0x7f968c0c4d90 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f968c0cc4d0 tx=0x7f967c02b910 comp rx=0 tx=0).stop 2023-11-09T03:48:13.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.618+0000 7f9691244640 1 -- 172.21.15.76:0/2834731881 shutdown_connections 2023-11-09T03:48:13.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.618+0000 7f9691244640 1 --2- 172.21.15.76:0/2834731881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f968c0c48e0 0x7f968c0c4d90 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:13.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.618+0000 7f9691244640 1 --2- 172.21.15.76:0/2834731881 >> v2:172.21.15.113:3300/0 conn(0x7f968c159c00 0x7f968c0cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:13.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.618+0000 7f9691244640 1 -- 172.21.15.76:0/2834731881 >> 172.21.15.76:0/2834731881 conn(0x7f968c0ce950 msgr2=0x7f968c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:13.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.618+0000 7f9691244640 1 -- 172.21.15.76:0/2834731881 shutdown_connections 2023-11-09T03:48:13.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.618+0000 7f9691244640 1 -- 172.21.15.76:0/2834731881 wait complete. 2023-11-09T03:48:13.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.620+0000 7f9691244640 1 Processor -- start 2023-11-09T03:48:13.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.620+0000 7f9691244640 1 -- start start 2023-11-09T03:48:13.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.620+0000 7f9691244640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f968c0adac0 0x7f968c1492f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:13.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.620+0000 7f9691244640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f968c0c48e0 0x7f968c0cb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:13.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.620+0000 7f9691244640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f968c159c00 0x7f968c0cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:13.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.620+0000 7f9691244640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c153150 con 0x7f968c0c48e0 2023-11-09T03:48:13.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.620+0000 7f9691244640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c0ab970 con 0x7f968c0adac0 2023-11-09T03:48:13.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.620+0000 7f9691244640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f968c0abc10 con 0x7f968c159c00 2023-11-09T03:48:13.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.620+0000 7f968a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f968c0c48e0 0x7f968c0cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:13.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.620+0000 7f968a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f968c0c48e0 0x7f968c0cb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51210/0 (socket says 172.21.15.76:51210) 2023-11-09T03:48:13.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.620+0000 7f968a59c640 1 -- 172.21.15.76:0/1820998709 learned_addr learned my addr 172.21.15.76:0/1820998709 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:13.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.621+0000 7f968b59e640 1 --2- 172.21.15.76:0/1820998709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f968c159c00 0x7f968c0cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:13.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.621+0000 7f968ad9d640 1 --2- 172.21.15.76:0/1820998709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f968c0adac0 0x7f968c1492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:13.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.621+0000 7f968a59c640 1 -- 172.21.15.76:0/1820998709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f968c159c00 msgr2=0x7f968c0cb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:13.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.621+0000 7f968a59c640 1 --2- 172.21.15.76:0/1820998709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f968c159c00 0x7f968c0cb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:13.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.621+0000 7f968a59c640 1 -- 172.21.15.76:0/1820998709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f968c0adac0 msgr2=0x7f968c1492f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:13.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.621+0000 7f968a59c640 1 --2- 172.21.15.76:0/1820998709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f968c0adac0 0x7f968c1492f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:13.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.621+0000 7f968a59c640 1 -- 172.21.15.76:0/1820998709 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f967c0094d0 con 0x7f968c0c48e0 2023-11-09T03:48:13.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.621+0000 7f968a59c640 1 --2- 172.21.15.76:0/1820998709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f968c0c48e0 0x7f968c0cb340 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f967c02b950 tx=0x7f967c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:13.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.621+0000 7f96737fe640 1 -- 172.21.15.76:0/1820998709 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c03ad30 con 0x7f968c0c48e0 2023-11-09T03:48:13.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.621+0000 7f96737fe640 1 -- 172.21.15.76:0/1820998709 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967c0443f0 con 0x7f968c0c48e0 2023-11-09T03:48:13.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.621+0000 7f9691244640 1 -- 172.21.15.76:0/1820998709 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968c0abef0 con 0x7f968c0c48e0 2023-11-09T03:48:13.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.621+0000 7f96737fe640 1 -- 172.21.15.76:0/1820998709 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c0447a0 con 0x7f968c0c48e0 2023-11-09T03:48:13.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.621+0000 7f9691244640 1 -- 172.21.15.76:0/1820998709 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968c0ac4b0 con 0x7f968c0c48e0 2023-11-09T03:48:13.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.622+0000 7f9691244640 1 -- 172.21.15.76:0/1820998709 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f968c0ac6a0 con 0x7f968c0c48e0 2023-11-09T03:48:13.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.622+0000 7f96737fe640 1 -- 172.21.15.76:0/1820998709 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f967c04d450 con 0x7f968c0c48e0 2023-11-09T03:48:13.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.623+0000 7f96737fe640 1 --2- 172.21.15.76:0/1820998709 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f967806ae70 0x7f967806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:13.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.623+0000 7f96737fe640 1 -- 172.21.15.76:0/1820998709 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f967c02d040 con 0x7f968c0c48e0 2023-11-09T03:48:13.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.623+0000 7f968ad9d640 1 --2- 172.21.15.76:0/1820998709 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f967806ae70 0x7f967806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:13.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.623+0000 7f968ad9d640 1 --2- 172.21.15.76:0/1820998709 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f967806ae70 0x7f967806d330 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9680006fd0 tx=0x7f9680008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:13.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.627+0000 7f96737fe640 1 -- 172.21.15.76:0/1820998709 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f967c076cd0 con 0x7f968c0c48e0 2023-11-09T03:48:13.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.793+0000 7f9691244640 1 -- 172.21.15.76:0/1820998709 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f968c074290 con 0x7f968c0c48e0 2023-11-09T03:48:13.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.796+0000 7f96737fe640 1 -- 172.21.15.76:0/1820998709 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v16) v1 ==== 104+0+0 (secure 0 0 0) 0x7f967c035020 con 0x7f968c0c48e0 2023-11-09T03:48:13.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.798+0000 7f9691244640 1 -- 172.21.15.76:0/1820998709 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f967806ae70 msgr2=0x7f967806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:13.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.798+0000 7f9691244640 1 --2- 172.21.15.76:0/1820998709 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f967806ae70 0x7f967806d330 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9680006fd0 tx=0x7f9680008040 comp rx=0 tx=0).stop 2023-11-09T03:48:13.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.798+0000 7f9691244640 1 -- 172.21.15.76:0/1820998709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f968c0c48e0 msgr2=0x7f968c0cb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:13.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.798+0000 7f9691244640 1 --2- 172.21.15.76:0/1820998709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f968c0c48e0 0x7f968c0cb340 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f967c02b950 tx=0x7f967c0046c0 comp rx=0 tx=0).stop 2023-11-09T03:48:13.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.798+0000 7f9691244640 1 -- 172.21.15.76:0/1820998709 shutdown_connections 2023-11-09T03:48:13.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.798+0000 7f9691244640 1 --2- 172.21.15.76:0/1820998709 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f967806ae70 0x7f967806d330 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:13.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.798+0000 7f9691244640 1 --2- 172.21.15.76:0/1820998709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f968c159c00 0x7f968c0cb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:13.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.798+0000 7f9691244640 1 --2- 172.21.15.76:0/1820998709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f968c0c48e0 0x7f968c0cb340 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:13.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.798+0000 7f9691244640 1 --2- 172.21.15.76:0/1820998709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f968c0adac0 0x7f968c1492f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:13.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.798+0000 7f9691244640 1 -- 172.21.15.76:0/1820998709 >> 172.21.15.76:0/1820998709 conn(0x7f968c0ce950 msgr2=0x7f968c0adea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:13.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.798+0000 7f9691244640 1 -- 172.21.15.76:0/1820998709 shutdown_connections 2023-11-09T03:48:13.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.798+0000 7f9691244640 1 -- 172.21.15.76:0/1820998709 wait complete. 2023-11-09T03:48:13.810 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_using_pool_with_snap_fails_fs_creation' 2023-11-09T03:48:13.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.911+0000 7f0a7230f640 1 Processor -- start 2023-11-09T03:48:13.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.911+0000 7f0a7230f640 1 -- start start 2023-11-09T03:48:13.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.911+0000 7f0a7230f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0a6c159c70 0x7f0a6c0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:13.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.911+0000 7f0a7230f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a6c06db20 0x7f0a6c0b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:13.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.911+0000 7f0a7230f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0a6c0b0910 0x7f0a6c15d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:13.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.911+0000 7f0a7230f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a6c0b49b0 con 0x7f0a6c0b0910 2023-11-09T03:48:13.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.911+0000 7f0a7230f640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a6c0b0da0 con 0x7f0a6c159c70 2023-11-09T03:48:13.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.911+0000 7f0a7230f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c0b4110 con 0x7f0a6c06db20 2023-11-09T03:48:13.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.912+0000 7f0a708ac640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0a6c0b0910 0x7f0a6c15d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:13.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.912+0000 7f0a708ac640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0a6c0b0910 0x7f0a6c15d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51214/0 (socket says 172.21.15.76:51214) 2023-11-09T03:48:13.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.912+0000 7f0a708ac640 1 -- 172.21.15.76:0/2333899513 learned_addr learned my addr 172.21.15.76:0/2333899513 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:13.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.912+0000 7f0a6bfff640 1 --2- 172.21.15.76:0/2333899513 >> v2:172.21.15.113:3300/0 conn(0x7f0a6c159c70 0x7f0a6c0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:13.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.912+0000 7f0a6b7fe640 1 --2- 172.21.15.76:0/2333899513 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a6c06db20 0x7f0a6c0b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:13.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.912+0000 7f0a708ac640 1 -- 172.21.15.76:0/2333899513 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a6c06db20 msgr2=0x7f0a6c0b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:13.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.912+0000 7f0a708ac640 1 --2- 172.21.15.76:0/2333899513 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a6c06db20 0x7f0a6c0b03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:13.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.912+0000 7f0a708ac640 1 -- 172.21.15.76:0/2333899513 >> v2:172.21.15.113:3300/0 conn(0x7f0a6c159c70 msgr2=0x7f0a6c0ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:13.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.912+0000 7f0a708ac640 1 --2- 172.21.15.76:0/2333899513 >> v2:172.21.15.113:3300/0 conn(0x7f0a6c159c70 0x7f0a6c0ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:13.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.912+0000 7f0a708ac640 1 -- 172.21.15.76:0/2333899513 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6c0b43f0 con 0x7f0a6c0b0910 2023-11-09T03:48:13.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.913+0000 7f0a708ac640 1 --2- 172.21.15.76:0/2333899513 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a6c0b0910 0x7f0a6c15d7d0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0cc440 tx=0x7f0a6402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3923216dfe43b1b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:13.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.913+0000 7f0a6affd640 1 -- 172.21.15.76:0/2333899513 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6402ca70 con 0x7f0a6c0b0910 2023-11-09T03:48:13.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.913+0000 7f0a6affd640 1 -- 172.21.15.76:0/2333899513 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a6402cbd0 con 0x7f0a6c0b0910 2023-11-09T03:48:13.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.913+0000 7f0a6affd640 1 -- 172.21.15.76:0/2333899513 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a640343f0 con 0x7f0a6c0b0910 2023-11-09T03:48:13.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.913+0000 7f0a7230f640 1 -- 172.21.15.76:0/2333899513 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a6c0b0910 msgr2=0x7f0a6c15d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:13.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.913+0000 7f0a7230f640 1 --2- 172.21.15.76:0/2333899513 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a6c0b0910 0x7f0a6c15d7d0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0cc440 tx=0x7f0a6402bed0 comp rx=0 tx=0).stop 2023-11-09T03:48:13.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.913+0000 7f0a7230f640 1 -- 172.21.15.76:0/2333899513 shutdown_connections 2023-11-09T03:48:13.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.913+0000 7f0a7230f640 1 --2- 172.21.15.76:0/2333899513 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a6c0b0910 0x7f0a6c15d7d0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:13.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.913+0000 7f0a7230f640 1 --2- 172.21.15.76:0/2333899513 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a6c06db20 0x7f0a6c0b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:13.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.913+0000 7f0a7230f640 1 --2- 172.21.15.76:0/2333899513 >> v2:172.21.15.113:3300/0 conn(0x7f0a6c159c70 0x7f0a6c0ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:13.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.913+0000 7f0a7230f640 1 -- 172.21.15.76:0/2333899513 >> 172.21.15.76:0/2333899513 conn(0x7f0a6c0ce950 msgr2=0x7f0a6c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:13.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.913+0000 7f0a7230f640 1 -- 172.21.15.76:0/2333899513 shutdown_connections 2023-11-09T03:48:13.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.913+0000 7f0a7230f640 1 -- 172.21.15.76:0/2333899513 wait complete. 2023-11-09T03:48:13.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.914+0000 7f0a7230f640 1 Processor -- start 2023-11-09T03:48:13.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.915+0000 7f0a7230f640 1 -- start start 2023-11-09T03:48:13.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.915+0000 7f0a7230f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a6c06db20 0x7f0a6c0cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:13.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.915+0000 7f0a7230f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a6c0b0910 0x7f0a6c0ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:13.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.915+0000 7f0a7230f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a6c159c70 0x7f0a6c0c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:13.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.915+0000 7f0a7230f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c1f3bf0 con 0x7f0a6c06db20 2023-11-09T03:48:13.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.915+0000 7f0a7230f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c1f1350 con 0x7f0a6c0b0910 2023-11-09T03:48:13.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.915+0000 7f0a7230f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c1f3e40 con 0x7f0a6c159c70 2023-11-09T03:48:13.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.915+0000 7f0a6bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a6c06db20 0x7f0a6c0cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:13.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.915+0000 7f0a6bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a6c06db20 0x7f0a6c0cca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51224/0 (socket says 172.21.15.76:51224) 2023-11-09T03:48:13.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.916+0000 7f0a6bfff640 1 -- 172.21.15.76:0/730821791 learned_addr learned my addr 172.21.15.76:0/730821791 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:13.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.916+0000 7f0a6b7fe640 1 --2- 172.21.15.76:0/730821791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a6c0b0910 0x7f0a6c0ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:13.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.916+0000 7f0a708ac640 1 --2- 172.21.15.76:0/730821791 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a6c159c70 0x7f0a6c0c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:13.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.916+0000 7f0a6bfff640 1 -- 172.21.15.76:0/730821791 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a6c159c70 msgr2=0x7f0a6c0c69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:13.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.916+0000 7f0a6bfff640 1 --2- 172.21.15.76:0/730821791 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a6c159c70 0x7f0a6c0c69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:13.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.916+0000 7f0a6bfff640 1 -- 172.21.15.76:0/730821791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a6c0b0910 msgr2=0x7f0a6c0ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:13.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.916+0000 7f0a6bfff640 1 --2- 172.21.15.76:0/730821791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a6c0b0910 0x7f0a6c0ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:13.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.916+0000 7f0a6bfff640 1 -- 172.21.15.76:0/730821791 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a64009b30 con 0x7f0a6c06db20 2023-11-09T03:48:13.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.916+0000 7f0a6bfff640 1 --2- 172.21.15.76:0/730821791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a6c06db20 0x7f0a6c0cca50 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c00d5f0 tx=0x7f0a5c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:13.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.916+0000 7f0a68ff9640 1 -- 172.21.15.76:0/730821791 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c0125f0 con 0x7f0a6c06db20 2023-11-09T03:48:13.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.916+0000 7f0a68ff9640 1 -- 172.21.15.76:0/730821791 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a5c012750 con 0x7f0a6c06db20 2023-11-09T03:48:13.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.916+0000 7f0a7230f640 1 -- 172.21.15.76:0/730821791 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a6c1f4120 con 0x7f0a6c06db20 2023-11-09T03:48:13.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.916+0000 7f0a68ff9640 1 -- 172.21.15.76:0/730821791 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c012b00 con 0x7f0a6c06db20 2023-11-09T03:48:13.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.916+0000 7f0a7230f640 1 -- 172.21.15.76:0/730821791 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a6c1f4920 con 0x7f0a6c06db20 2023-11-09T03:48:13.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.917+0000 7f0a7230f640 1 -- 172.21.15.76:0/730821791 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a6c1f4ae0 con 0x7f0a6c06db20 2023-11-09T03:48:13.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.917+0000 7f0a68ff9640 1 -- 172.21.15.76:0/730821791 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0a5c012c60 con 0x7f0a6c06db20 2023-11-09T03:48:13.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.918+0000 7f0a68ff9640 1 --2- 172.21.15.76:0/730821791 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a5806ae70 0x7f0a5806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:13.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.918+0000 7f0a68ff9640 1 -- 172.21.15.76:0/730821791 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f0a5c089c30 con 0x7f0a6c06db20 2023-11-09T03:48:13.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.918+0000 7f0a6b7fe640 1 --2- 172.21.15.76:0/730821791 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a5806ae70 0x7f0a5806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:13.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.919+0000 7f0a6b7fe640 1 --2- 172.21.15.76:0/730821791 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a5806ae70 0x7f0a5806d330 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c155210 tx=0x7f0a60008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:13.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:13.922+0000 7f0a68ff9640 1 -- 172.21.15.76:0/730821791 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0a5c054b00 con 0x7f0a6c06db20 2023-11-09T03:48:14.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.153+0000 7f0a7230f640 1 -- 172.21.15.76:0/730821791 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_using_pool_with_snap_fails_fs_creation"]} v 0) v1 -- 0x7f0a6c074290 con 0x7f0a6c06db20 2023-11-09T03:48:14.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.493+0000 7f0a68ff9640 1 -- 172.21.15.76:0/730821791 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_using_pool_with_snap_fails_fs_creation"]}]=0 v176) v1 ==== 173+0+0 (secure 0 0 0) 0x7f0a5c058120 con 0x7f0a6c06db20 2023-11-09T03:48:14.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.495+0000 7f0a7230f640 1 -- 172.21.15.76:0/730821791 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a5806ae70 msgr2=0x7f0a5806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:14.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.495+0000 7f0a7230f640 1 --2- 172.21.15.76:0/730821791 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a5806ae70 0x7f0a5806d330 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c155210 tx=0x7f0a60008040 comp rx=0 tx=0).stop 2023-11-09T03:48:14.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.495+0000 7f0a7230f640 1 -- 172.21.15.76:0/730821791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a6c06db20 msgr2=0x7f0a6c0cca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:14.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.495+0000 7f0a7230f640 1 --2- 172.21.15.76:0/730821791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a6c06db20 0x7f0a6c0cca50 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c00d5f0 tx=0x7f0a5c00dab0 comp rx=0 tx=0).stop 2023-11-09T03:48:14.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.495+0000 7f0a7230f640 1 -- 172.21.15.76:0/730821791 shutdown_connections 2023-11-09T03:48:14.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.495+0000 7f0a7230f640 1 --2- 172.21.15.76:0/730821791 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a5806ae70 0x7f0a5806d330 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:14.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.495+0000 7f0a7230f640 1 --2- 172.21.15.76:0/730821791 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a6c159c70 0x7f0a6c0c69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:14.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.495+0000 7f0a7230f640 1 --2- 172.21.15.76:0/730821791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a6c0b0910 0x7f0a6c0ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:14.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.495+0000 7f0a7230f640 1 --2- 172.21.15.76:0/730821791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a6c06db20 0x7f0a6c0cca50 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:14.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.495+0000 7f0a7230f640 1 -- 172.21.15.76:0/730821791 >> 172.21.15.76:0/730821791 conn(0x7f0a6c0ce950 msgr2=0x7f0a6c15e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:14.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.495+0000 7f0a7230f640 1 -- 172.21.15.76:0/730821791 shutdown_connections 2023-11-09T03:48:14.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.495+0000 7f0a7230f640 1 -- 172.21.15.76:0/730821791 wait complete. 2023-11-09T03:48:14.507 INFO:tasks.cephfs_test_runner:Test that using a pool with snaps for fs creation fails ... ok 2023-11-09T03:48:14.507 INFO:tasks.cephfs_test_runner:Starting test: test_allow_new_snaps_config (tasks.cephfs.test_snapshots.TestSnapshots) Check whether 'allow_new_snaps' setting works 2023-11-09T03:48:14.508 INFO:tasks.cephfs_test_runner:test_allow_new_snaps_config (tasks.cephfs.test_snapshots.TestSnapshots) 2023-11-09T03:48:14.508 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:48:14.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.607+0000 7f8cb8d78640 1 Processor -- start 2023-11-09T03:48:14.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.607+0000 7f8cb8d78640 1 -- start start 2023-11-09T03:48:14.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.608+0000 7f8cb8d78640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8cb40cda60 0x7f8cb40ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:14.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.608+0000 7f8cb8d78640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8cb4070d40 0x7f8cb406c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:14.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.608+0000 7f8cb8d78640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cb40cc990 con 0x7f8cb40b0cd0 2023-11-09T03:48:14.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.608+0000 7f8cb8d78640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cb40ccc70 con 0x7f8cb40cda60 2023-11-09T03:48:14.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.608+0000 7f8cb8d78640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cb40c9ce0 con 0x7f8cb4070d40 2023-11-09T03:48:14.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.609+0000 7f8cb259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8cb40cda60 0x7f8cb40ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:14.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.609+0000 7f8cb259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8cb40cda60 0x7f8cb40ca7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51240/0 (socket says 172.21.15.76:51240) 2023-11-09T03:48:14.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.609+0000 7f8cb259c640 1 -- 172.21.15.76:0/1992969471 learned_addr learned my addr 172.21.15.76:0/1992969471 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:14.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.609+0000 7f8cb2d9d640 1 --2- 172.21.15.76:0/1992969471 >> v2:172.21.15.113:3300/0 conn(0x7f8cb4070d40 0x7f8cb406c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:14.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.609+0000 7f8cb259c640 1 -- 172.21.15.76:0/1992969471 >> v2:172.21.15.113:3300/0 conn(0x7f8cb4070d40 msgr2=0x7f8cb406c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:14.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.609+0000 7f8cb259c640 1 --2- 172.21.15.76:0/1992969471 >> v2:172.21.15.113:3300/0 conn(0x7f8cb4070d40 0x7f8cb406c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:14.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.609+0000 7f8cb259c640 1 -- 172.21.15.76:0/1992969471 >> v1:172.21.15.113:6789/0 conn(0x7f8cb40b0cd0 legacy=0x7f8cb4070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:14.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.609+0000 7f8cb259c640 1 -- 172.21.15.76:0/1992969471 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb40c9fc0 con 0x7f8cb40cda60 2023-11-09T03:48:14.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.609+0000 7f8cb2d9d640 1 --2- 172.21.15.76:0/1992969471 >> v2:172.21.15.113:3300/0 conn(0x7f8cb4070d40 0x7f8cb406c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:14.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.610+0000 7f8cb259c640 1 --2- 172.21.15.76:0/1992969471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cb40cda60 0x7f8cb40ca7a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f8cb40c93d0 tx=0x7f8cac02df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0fcc3e7e152c9bd server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:14.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.610+0000 7f8cb159a640 1 -- 172.21.15.76:0/1992969471 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac02ed00 con 0x7f8cb40cda60 2023-11-09T03:48:14.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.610+0000 7f8cb159a640 1 -- 172.21.15.76:0/1992969471 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cac032070 con 0x7f8cb40cda60 2023-11-09T03:48:14.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.610+0000 7f8cb159a640 1 -- 172.21.15.76:0/1992969471 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac038620 con 0x7f8cb40cda60 2023-11-09T03:48:14.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.610+0000 7f8cb8d78640 1 -- 172.21.15.76:0/1992969471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cb40cda60 msgr2=0x7f8cb40ca7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:14.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.610+0000 7f8cb8d78640 1 --2- 172.21.15.76:0/1992969471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cb40cda60 0x7f8cb40ca7a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f8cb40c93d0 tx=0x7f8cac02df50 comp rx=0 tx=0).stop 2023-11-09T03:48:14.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.610+0000 7f8cb8d78640 1 -- 172.21.15.76:0/1992969471 shutdown_connections 2023-11-09T03:48:14.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.610+0000 7f8cb8d78640 1 --2- 172.21.15.76:0/1992969471 >> v2:172.21.15.113:3300/0 conn(0x7f8cb4070d40 0x7f8cb406c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:14.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.610+0000 7f8cb8d78640 1 --2- 172.21.15.76:0/1992969471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cb40cda60 0x7f8cb40ca7a0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:14.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.610+0000 7f8cb8d78640 1 -- 172.21.15.76:0/1992969471 >> 172.21.15.76:0/1992969471 conn(0x7f8cb40ce950 msgr2=0x7f8cb40ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:14.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.610+0000 7f8cb8d78640 1 -- 172.21.15.76:0/1992969471 shutdown_connections 2023-11-09T03:48:14.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.610+0000 7f8cb8d78640 1 -- 172.21.15.76:0/1992969471 wait complete. 2023-11-09T03:48:14.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.612+0000 7f8cb8d78640 1 Processor -- start 2023-11-09T03:48:14.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.612+0000 7f8cb8d78640 1 -- start start 2023-11-09T03:48:14.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.612+0000 7f8cb8d78640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8cb4070d40 0x7f8cb4156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:14.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.612+0000 7f8cb8d78640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cb40b0cd0 0x7f8cb40b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:14.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.612+0000 7f8cb8d78640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8cb40cda60 0x7f8cb40b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:14.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.612+0000 7f8cb8d78640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb4157320 con 0x7f8cb40b0cd0 2023-11-09T03:48:14.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.612+0000 7f8cb8d78640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb40b1c10 con 0x7f8cb4070d40 2023-11-09T03:48:14.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.612+0000 7f8cb8d78640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb40b1ef0 con 0x7f8cb40cda60 2023-11-09T03:48:14.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.613+0000 7f8cb1d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cb40b0cd0 0x7f8cb40b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:14.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.613+0000 7f8cb1d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cb40b0cd0 0x7f8cb40b13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51252/0 (socket says 172.21.15.76:51252) 2023-11-09T03:48:14.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.613+0000 7f8cb1d9b640 1 -- 172.21.15.76:0/1831204355 learned_addr learned my addr 172.21.15.76:0/1831204355 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:14.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.613+0000 7f8cb259c640 1 --2- 172.21.15.76:0/1831204355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8cb4070d40 0x7f8cb4156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:14.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.613+0000 7f8cb2d9d640 1 --2- 172.21.15.76:0/1831204355 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8cb40cda60 0x7f8cb40b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:14.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.613+0000 7f8cb1d9b640 1 -- 172.21.15.76:0/1831204355 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8cb40cda60 msgr2=0x7f8cb40b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:14.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.613+0000 7f8cb1d9b640 1 --2- 172.21.15.76:0/1831204355 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8cb40cda60 0x7f8cb40b2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:14.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.613+0000 7f8cb1d9b640 1 -- 172.21.15.76:0/1831204355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8cb4070d40 msgr2=0x7f8cb4156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:14.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.613+0000 7f8cb1d9b640 1 --2- 172.21.15.76:0/1831204355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8cb4070d40 0x7f8cb4156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:14.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.613+0000 7f8cb1d9b640 1 -- 172.21.15.76:0/1831204355 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac00b760 con 0x7f8cb40b0cd0 2023-11-09T03:48:14.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.613+0000 7f8cb259c640 1 --2- 172.21.15.76:0/1831204355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8cb4070d40 0x7f8cb4156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:14.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.613+0000 7f8cb1d9b640 1 --2- 172.21.15.76:0/1831204355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cb40b0cd0 0x7f8cb40b13e0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800c7a0 tx=0x7f8ca800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:14.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.613+0000 7f8ca2ffd640 1 -- 172.21.15.76:0/1831204355 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8006d50 con 0x7f8cb40b0cd0 2023-11-09T03:48:14.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.613+0000 7f8ca2ffd640 1 -- 172.21.15.76:0/1831204355 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca8003970 con 0x7f8cb40b0cd0 2023-11-09T03:48:14.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.613+0000 7f8cb8d78640 1 -- 172.21.15.76:0/1831204355 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb40b21d0 con 0x7f8cb40b0cd0 2023-11-09T03:48:14.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.613+0000 7f8ca2ffd640 1 -- 172.21.15.76:0/1831204355 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8003d60 con 0x7f8cb40b0cd0 2023-11-09T03:48:14.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.613+0000 7f8cb8d78640 1 -- 172.21.15.76:0/1831204355 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb40b1920 con 0x7f8cb40b0cd0 2023-11-09T03:48:14.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.614+0000 7f8cb8d78640 1 -- 172.21.15.76:0/1831204355 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cb4074290 con 0x7f8cb40b0cd0 2023-11-09T03:48:14.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.615+0000 7f8ca2ffd640 1 -- 172.21.15.76:0/1831204355 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8ca800a850 con 0x7f8cb40b0cd0 2023-11-09T03:48:14.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.615+0000 7f8ca2ffd640 1 --2- 172.21.15.76:0/1831204355 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8c9406ae70 0x7f8c9406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:14.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.615+0000 7f8ca2ffd640 1 -- 172.21.15.76:0/1831204355 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f8ca8089410 con 0x7f8cb40b0cd0 2023-11-09T03:48:14.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.615+0000 7f8cb259c640 1 --2- 172.21.15.76:0/1831204355 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8c9406ae70 0x7f8c9406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:14.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.616+0000 7f8cb259c640 1 --2- 172.21.15.76:0/1831204355 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8c9406ae70 0x7f8c9406d330 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8cac039000 tx=0x7f8cac034000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:14.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.619+0000 7f8ca2ffd640 1 -- 172.21.15.76:0/1831204355 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8ca80542e0 con 0x7f8cb40b0cd0 2023-11-09T03:48:14.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.787+0000 7f8cb8d78640 1 -- 172.21.15.76:0/1831204355 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8cb40b48d0 con 0x7f8cb40b0cd0 2023-11-09T03:48:14.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.789+0000 7f8ca2ffd640 1 -- 172.21.15.76:0/1831204355 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+18420 (secure 0 0 0) 0x7f8ca8057900 con 0x7f8cb40b0cd0 2023-11-09T03:48:14.789 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:14.790 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":74,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:48:13.488785+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-09T03:47:59.280601+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-09T03:47:59.742039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:6835/1576689316":"2023-11-10T03:48:10.171047+0000","172.21.15.76:6834/1576689316":"2023-11-10T03:48:10.171047+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:48:14.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.791+0000 7f8cb8d78640 1 -- 172.21.15.76:0/1831204355 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8c9406ae70 msgr2=0x7f8c9406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:14.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.791+0000 7f8cb8d78640 1 --2- 172.21.15.76:0/1831204355 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8c9406ae70 0x7f8c9406d330 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8cac039000 tx=0x7f8cac034000 comp rx=0 tx=0).stop 2023-11-09T03:48:14.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.791+0000 7f8cb8d78640 1 -- 172.21.15.76:0/1831204355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cb40b0cd0 msgr2=0x7f8cb40b13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:14.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.791+0000 7f8cb8d78640 1 --2- 172.21.15.76:0/1831204355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cb40b0cd0 0x7f8cb40b13e0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800c7a0 tx=0x7f8ca800cc60 comp rx=0 tx=0).stop 2023-11-09T03:48:14.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.791+0000 7f8cb8d78640 1 -- 172.21.15.76:0/1831204355 shutdown_connections 2023-11-09T03:48:14.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.791+0000 7f8cb8d78640 1 --2- 172.21.15.76:0/1831204355 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8c9406ae70 0x7f8c9406d330 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:14.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.791+0000 7f8cb8d78640 1 --2- 172.21.15.76:0/1831204355 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8cb40cda60 0x7f8cb40b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:14.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.791+0000 7f8cb8d78640 1 --2- 172.21.15.76:0/1831204355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cb40b0cd0 0x7f8cb40b13e0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:14.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.791+0000 7f8cb8d78640 1 --2- 172.21.15.76:0/1831204355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8cb4070d40 0x7f8cb4156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:14.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.791+0000 7f8cb8d78640 1 -- 172.21.15.76:0/1831204355 >> 172.21.15.76:0/1831204355 conn(0x7f8cb40ce950 msgr2=0x7f8cb40ac260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:14.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.791+0000 7f8cb8d78640 1 -- 172.21.15.76:0/1831204355 shutdown_connections 2023-11-09T03:48:14.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.791+0000 7f8cb8d78640 1 -- 172.21.15.76:0/1831204355 wait complete. 2023-11-09T03:48:14.802 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-09T03:47:00.036540+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_data_pool', 'create_time': '2023-11-09T03:47:59.280601+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 70, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-09T03:47:59.742039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-09T03:48:14.803 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:48:14.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.905+0000 7fa3684a9640 1 Processor -- start 2023-11-09T03:48:14.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.905+0000 7fa3684a9640 1 -- start start 2023-11-09T03:48:14.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.906+0000 7fa3684a9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa3600c4ff0 0x7fa3600cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:14.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.906+0000 7fa3684a9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa3600adac0 0x7fa3600adf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:14.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.906+0000 7fa3684a9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa36015aff0 con 0x7fa360159c00 2023-11-09T03:48:14.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.906+0000 7fa3684a9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3600ae460 con 0x7fa3600c4ff0 2023-11-09T03:48:14.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.906+0000 7fa3684a9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa36015a3e0 con 0x7fa3600adac0 2023-11-09T03:48:14.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.907+0000 7fa365a44640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa3600c4ff0 0x7fa3600cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:14.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.907+0000 7fa366245640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa360159c00 0x7fa3600c48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40292/0 (socket says 172.21.15.76:40292) 2023-11-09T03:48:14.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.907+0000 7fa366245640 1 -- 172.21.15.76:0/1195282734 learned_addr learned my addr 172.21.15.76:0/1195282734 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:14.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.907+0000 7fa366a46640 1 --2- 172.21.15.76:0/1195282734 >> v2:172.21.15.113:3300/0 conn(0x7fa3600adac0 0x7fa3600adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:14.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.907+0000 7fa365a44640 1 -- 172.21.15.76:0/1195282734 >> v2:172.21.15.113:3300/0 conn(0x7fa3600adac0 msgr2=0x7fa3600adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:14.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.907+0000 7fa365a44640 1 --2- 172.21.15.76:0/1195282734 >> v2:172.21.15.113:3300/0 conn(0x7fa3600adac0 0x7fa3600adf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:14.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.907+0000 7fa365a44640 1 -- 172.21.15.76:0/1195282734 >> v1:172.21.15.76:6789/0 conn(0x7fa360159c00 legacy=0x7fa3600c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:14.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.907+0000 7fa365a44640 1 -- 172.21.15.76:0/1195282734 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36015a6c0 con 0x7fa3600c4ff0 2023-11-09T03:48:14.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.908+0000 7fa365a44640 1 --2- 172.21.15.76:0/1195282734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3600c4ff0 0x7fa3600cc890 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa3600c59a0 tx=0x7fa35c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c8ff139ea327b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:14.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.908+0000 7fa365243640 1 -- 172.21.15.76:0/1195282734 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c031830 con 0x7fa3600c4ff0 2023-11-09T03:48:14.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.908+0000 7fa365243640 1 -- 172.21.15.76:0/1195282734 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35c031990 con 0x7fa3600c4ff0 2023-11-09T03:48:14.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.908+0000 7fa365243640 1 -- 172.21.15.76:0/1195282734 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c031d20 con 0x7fa3600c4ff0 2023-11-09T03:48:14.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.908+0000 7fa3684a9640 1 -- 172.21.15.76:0/1195282734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3600c4ff0 msgr2=0x7fa3600cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:14.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.908+0000 7fa3684a9640 1 --2- 172.21.15.76:0/1195282734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3600c4ff0 0x7fa3600cc890 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa3600c59a0 tx=0x7fa35c02b8d0 comp rx=0 tx=0).stop 2023-11-09T03:48:14.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.908+0000 7fa3684a9640 1 -- 172.21.15.76:0/1195282734 shutdown_connections 2023-11-09T03:48:14.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.908+0000 7fa3684a9640 1 --2- 172.21.15.76:0/1195282734 >> v2:172.21.15.113:3300/0 conn(0x7fa3600adac0 0x7fa3600adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:14.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.908+0000 7fa3684a9640 1 --2- 172.21.15.76:0/1195282734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3600c4ff0 0x7fa3600cc890 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:14.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.908+0000 7fa3684a9640 1 -- 172.21.15.76:0/1195282734 >> 172.21.15.76:0/1195282734 conn(0x7fa3600ce950 msgr2=0x7fa3600ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:14.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.908+0000 7fa3684a9640 1 -- 172.21.15.76:0/1195282734 shutdown_connections 2023-11-09T03:48:14.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.908+0000 7fa3684a9640 1 -- 172.21.15.76:0/1195282734 wait complete. 2023-11-09T03:48:14.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.909+0000 7fa3684a9640 1 Processor -- start 2023-11-09T03:48:14.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.910+0000 7fa3684a9640 1 -- start start 2023-11-09T03:48:14.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.910+0000 7fa3684a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3600adac0 0x7fa360145b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:14.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.910+0000 7fa3684a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3600c4ff0 0x7fa360145370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:14.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.910+0000 7fa3684a9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa360159c00 0x7fa360149f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:14.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.910+0000 7fa3684a9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36014be80 con 0x7fa360159c00 2023-11-09T03:48:14.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.910+0000 7fa3684a9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36014e090 con 0x7fa3600c4ff0 2023-11-09T03:48:14.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.910+0000 7fa3684a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36014aa50 con 0x7fa3600adac0 2023-11-09T03:48:14.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.910+0000 7fa366a46640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa360159c00 0x7fa360149f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:14.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.910+0000 7fa366a46640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa360159c00 0x7fa360149f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51270/0 (socket says 172.21.15.76:51270) 2023-11-09T03:48:14.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.910+0000 7fa366a46640 1 -- 172.21.15.76:0/3302362343 learned_addr learned my addr 172.21.15.76:0/3302362343 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:14.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.911+0000 7fa365a44640 1 --2- 172.21.15.76:0/3302362343 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3600c4ff0 0x7fa360145370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:14.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.911+0000 7fa366245640 1 --2- 172.21.15.76:0/3302362343 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3600adac0 0x7fa360145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:14.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.911+0000 7fa366a46640 1 -- 172.21.15.76:0/3302362343 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3600adac0 msgr2=0x7fa360145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:14.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.911+0000 7fa366a46640 1 --2- 172.21.15.76:0/3302362343 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3600adac0 0x7fa360145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:14.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.911+0000 7fa366a46640 1 -- 172.21.15.76:0/3302362343 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3600c4ff0 msgr2=0x7fa360145370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:14.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.911+0000 7fa366a46640 1 --2- 172.21.15.76:0/3302362343 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3600c4ff0 0x7fa360145370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:14.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.911+0000 7fa366a46640 1 -- 172.21.15.76:0/3302362343 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c009400 con 0x7fa360159c00 2023-11-09T03:48:14.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.911+0000 7fa366245640 1 --2- 172.21.15.76:0/3302362343 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3600adac0 0x7fa360145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:14.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.911+0000 7fa366a46640 1 --2- 172.21.15.76:0/3302362343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa360159c00 0x7fa360149f90 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa35800efc0 tx=0x7fa35800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:14.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.911+0000 7fa352ffd640 1 -- 172.21.15.76:0/3302362343 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa358014480 con 0x7fa360159c00 2023-11-09T03:48:14.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.911+0000 7fa352ffd640 1 -- 172.21.15.76:0/3302362343 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35800f040 con 0x7fa360159c00 2023-11-09T03:48:14.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.911+0000 7fa3684a9640 1 -- 172.21.15.76:0/3302362343 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36014ad30 con 0x7fa360159c00 2023-11-09T03:48:14.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.911+0000 7fa352ffd640 1 -- 172.21.15.76:0/3302362343 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa358014820 con 0x7fa360159c00 2023-11-09T03:48:14.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.911+0000 7fa3684a9640 1 -- 172.21.15.76:0/3302362343 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36014e610 con 0x7fa360159c00 2023-11-09T03:48:14.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.912+0000 7fa3684a9640 1 -- 172.21.15.76:0/3302362343 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa36014e800 con 0x7fa360159c00 2023-11-09T03:48:14.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.912+0000 7fa352ffd640 1 -- 172.21.15.76:0/3302362343 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa358014980 con 0x7fa360159c00 2023-11-09T03:48:14.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.913+0000 7fa352ffd640 1 --2- 172.21.15.76:0/3302362343 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa34406ae70 0x7fa34406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:14.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.913+0000 7fa352ffd640 1 -- 172.21.15.76:0/3302362343 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6326+0+0 (secure 0 0 0) 0x7fa35808a720 con 0x7fa360159c00 2023-11-09T03:48:14.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.913+0000 7fa366245640 1 --2- 172.21.15.76:0/3302362343 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa34406ae70 0x7fa34406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:14.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.913+0000 7fa366245640 1 --2- 172.21.15.76:0/3302362343 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa34406ae70 0x7fa34406d330 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa354005880 tx=0x7fa354009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:14.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:14.917+0000 7fa352ffd640 1 -- 172.21.15.76:0/3302362343 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa358055560 con 0x7fa360159c00 2023-11-09T03:48:15.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.082+0000 7fa3684a9640 1 -- 172.21.15.76:0/3302362343 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa360074290 con 0x7fa360159c00 2023-11-09T03:48:15.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.082+0000 7fa352ffd640 1 -- 172.21.15.76:0/3302362343 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa358018020 con 0x7fa360159c00 2023-11-09T03:48:15.083 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:48:15.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.084+0000 7fa3684a9640 1 -- 172.21.15.76:0/3302362343 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa34406ae70 msgr2=0x7fa34406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:15.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.084+0000 7fa3684a9640 1 --2- 172.21.15.76:0/3302362343 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa34406ae70 0x7fa34406d330 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa354005880 tx=0x7fa354009040 comp rx=0 tx=0).stop 2023-11-09T03:48:15.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.084+0000 7fa3684a9640 1 -- 172.21.15.76:0/3302362343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa360159c00 msgr2=0x7fa360149f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:15.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.084+0000 7fa3684a9640 1 --2- 172.21.15.76:0/3302362343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa360159c00 0x7fa360149f90 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa35800efc0 tx=0x7fa35800c490 comp rx=0 tx=0).stop 2023-11-09T03:48:15.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.084+0000 7fa3684a9640 1 -- 172.21.15.76:0/3302362343 shutdown_connections 2023-11-09T03:48:15.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.084+0000 7fa3684a9640 1 --2- 172.21.15.76:0/3302362343 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa34406ae70 0x7fa34406d330 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.084+0000 7fa3684a9640 1 --2- 172.21.15.76:0/3302362343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa360159c00 0x7fa360149f90 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.084+0000 7fa3684a9640 1 --2- 172.21.15.76:0/3302362343 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3600c4ff0 0x7fa360145370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.084+0000 7fa3684a9640 1 --2- 172.21.15.76:0/3302362343 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3600adac0 0x7fa360145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.084+0000 7fa3684a9640 1 -- 172.21.15.76:0/3302362343 >> 172.21.15.76:0/3302362343 conn(0x7fa3600ce950 msgr2=0x7fa3600708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:15.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.084+0000 7fa3684a9640 1 -- 172.21.15.76:0/3302362343 shutdown_connections 2023-11-09T03:48:15.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.084+0000 7fa3684a9640 1 -- 172.21.15.76:0/3302362343 wait complete. 2023-11-09T03:48:15.095 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-09T03:48:15.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.197+0000 7f039aa5d640 1 Processor -- start 2023-11-09T03:48:15.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.197+0000 7f039aa5d640 1 -- start start 2023-11-09T03:48:15.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.198+0000 7f039aa5d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f039415ad30 0x7f03940adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:15.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.198+0000 7f039aa5d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0394070e20 0x7f039415e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:15.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.198+0000 7f039aa5d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03940c3900 con 0x7f03940ad730 2023-11-09T03:48:15.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.198+0000 7f039aa5d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03940ae260 con 0x7f039415ad30 2023-11-09T03:48:15.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.198+0000 7f039aa5d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03940c3be0 con 0x7f0394070e20 2023-11-09T03:48:15.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.198+0000 7f0393fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f039415ad30 0x7f03940adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:15.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.198+0000 7f03937fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f03940ad730 0x7f0394070710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40296/0 (socket says 172.21.15.76:40296) 2023-11-09T03:48:15.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.198+0000 7f03937fe640 1 -- 172.21.15.76:0/3753902564 learned_addr learned my addr 172.21.15.76:0/3753902564 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:15.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.198+0000 7f0398ffa640 1 --2- 172.21.15.76:0/3753902564 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0394070e20 0x7f039415e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:15.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.198+0000 7f0393fff640 1 -- 172.21.15.76:0/3753902564 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0394070e20 msgr2=0x7f039415e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:15.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.198+0000 7f0393fff640 1 --2- 172.21.15.76:0/3753902564 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0394070e20 0x7f039415e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.198+0000 7f0393fff640 1 -- 172.21.15.76:0/3753902564 >> v1:172.21.15.76:6789/0 conn(0x7f03940ad730 legacy=0x7f0394070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:15.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.198+0000 7f0393fff640 1 -- 172.21.15.76:0/3753902564 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03940c3ec0 con 0x7f039415ad30 2023-11-09T03:48:15.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.199+0000 7f0393fff640 1 --2- 172.21.15.76:0/3753902564 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f039415ad30 0x7f03940adc80 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f03840097a0 tx=0x7f038402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0424d7640dab9a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:15.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.199+0000 7f0392ffd640 1 -- 172.21.15.76:0/3753902564 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038402e070 con 0x7f039415ad30 2023-11-09T03:48:15.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.199+0000 7f0392ffd640 1 -- 172.21.15.76:0/3753902564 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03840315a0 con 0x7f039415ad30 2023-11-09T03:48:15.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.199+0000 7f0392ffd640 1 -- 172.21.15.76:0/3753902564 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0384031930 con 0x7f039415ad30 2023-11-09T03:48:15.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.199+0000 7f039aa5d640 1 -- 172.21.15.76:0/3753902564 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f039415ad30 msgr2=0x7f03940adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:15.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.199+0000 7f039aa5d640 1 --2- 172.21.15.76:0/3753902564 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f039415ad30 0x7f03940adc80 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f03840097a0 tx=0x7f038402b840 comp rx=0 tx=0).stop 2023-11-09T03:48:15.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.200+0000 7f039aa5d640 1 -- 172.21.15.76:0/3753902564 shutdown_connections 2023-11-09T03:48:15.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.200+0000 7f039aa5d640 1 --2- 172.21.15.76:0/3753902564 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0394070e20 0x7f039415e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.200+0000 7f039aa5d640 1 --2- 172.21.15.76:0/3753902564 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f039415ad30 0x7f03940adc80 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.200+0000 7f039aa5d640 1 -- 172.21.15.76:0/3753902564 >> 172.21.15.76:0/3753902564 conn(0x7f03940ce950 msgr2=0x7f03940ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:15.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.200+0000 7f039aa5d640 1 -- 172.21.15.76:0/3753902564 shutdown_connections 2023-11-09T03:48:15.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.200+0000 7f039aa5d640 1 -- 172.21.15.76:0/3753902564 wait complete. 2023-11-09T03:48:15.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.201+0000 7f039aa5d640 1 Processor -- start 2023-11-09T03:48:15.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.201+0000 7f039aa5d640 1 -- start start 2023-11-09T03:48:15.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.201+0000 7f039aa5d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0394070e20 0x7f0394153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:15.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.201+0000 7f039aa5d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f03940ad730 0x7f03940afb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:15.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.202+0000 7f039aa5d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f039415ad30 0x7f03940b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:15.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.202+0000 7f039aa5d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03940b0070 con 0x7f0394070e20 2023-11-09T03:48:15.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.202+0000 7f039aa5d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0394154460 con 0x7f039415ad30 2023-11-09T03:48:15.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.202+0000 7f039aa5d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0394154740 con 0x7f03940ad730 2023-11-09T03:48:15.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.202+0000 7f0393fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0394070e20 0x7f0394153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:15.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.202+0000 7f0393fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0394070e20 0x7f0394153df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51282/0 (socket says 172.21.15.76:51282) 2023-11-09T03:48:15.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.202+0000 7f0393fff640 1 -- 172.21.15.76:0/1774571844 learned_addr learned my addr 172.21.15.76:0/1774571844 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:15.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.202+0000 7f03937fe640 1 --2- 172.21.15.76:0/1774571844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f03940ad730 0x7f03940afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:15.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.202+0000 7f0393fff640 1 -- 172.21.15.76:0/1774571844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f03940ad730 msgr2=0x7f03940afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:15.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.202+0000 7f0393fff640 1 --2- 172.21.15.76:0/1774571844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f03940ad730 0x7f03940afb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.202+0000 7f0393fff640 1 -- 172.21.15.76:0/1774571844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f039415ad30 msgr2=0x7f03940b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:15.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.202+0000 7f0393fff640 1 --2- 172.21.15.76:0/1774571844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f039415ad30 0x7f03940b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.202+0000 7f0393fff640 1 -- 172.21.15.76:0/1774571844 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0384009400 con 0x7f0394070e20 2023-11-09T03:48:15.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.202+0000 7f0393fff640 1 --2- 172.21.15.76:0/1774571844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0394070e20 0x7f0394153df0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f0384034040 tx=0x7f03840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:15.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.202+0000 7f0390ff9640 1 -- 172.21.15.76:0/1774571844 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0384032900 con 0x7f0394070e20 2023-11-09T03:48:15.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.202+0000 7f0390ff9640 1 -- 172.21.15.76:0/1774571844 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0384032a60 con 0x7f0394070e20 2023-11-09T03:48:15.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.202+0000 7f039aa5d640 1 -- 172.21.15.76:0/1774571844 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0394154a20 con 0x7f0394070e20 2023-11-09T03:48:15.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.202+0000 7f0390ff9640 1 -- 172.21.15.76:0/1774571844 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0384032e10 con 0x7f0394070e20 2023-11-09T03:48:15.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.202+0000 7f039aa5d640 1 -- 172.21.15.76:0/1774571844 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0394154fe0 con 0x7f0394070e20 2023-11-09T03:48:15.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.203+0000 7f039aa5d640 1 -- 172.21.15.76:0/1774571844 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0394074290 con 0x7f0394070e20 2023-11-09T03:48:15.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.204+0000 7f0390ff9640 1 -- 172.21.15.76:0/1774571844 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f038404e430 con 0x7f0394070e20 2023-11-09T03:48:15.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.204+0000 7f0390ff9640 1 --2- 172.21.15.76:0/1774571844 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f038006aea0 0x7f038006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:15.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.204+0000 7f0390ff9640 1 -- 172.21.15.76:0/1774571844 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f03840ab9e0 con 0x7f0394070e20 2023-11-09T03:48:15.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.204+0000 7f03937fe640 1 --2- 172.21.15.76:0/1774571844 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f038006aea0 0x7f038006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:15.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.205+0000 7f03937fe640 1 --2- 172.21.15.76:0/1774571844 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f038006aea0 0x7f038006d360 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f039406dd30 tx=0x7f0388009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:15.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.208+0000 7f0390ff9640 1 -- 172.21.15.76:0/1774571844 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0384076820 con 0x7f0394070e20 2023-11-09T03:48:15.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.373+0000 7f039aa5d640 1 -- 172.21.15.76:0/1774571844 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f0394071970 con 0x7f0394070e20 2023-11-09T03:48:15.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.374+0000 7f0390ff9640 1 -- 172.21.15.76:0/1774571844 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v74) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0384079e40 con 0x7f0394070e20 2023-11-09T03:48:15.375 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:48:15.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.376+0000 7f039aa5d640 1 -- 172.21.15.76:0/1774571844 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f038006aea0 msgr2=0x7f038006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:15.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.376+0000 7f039aa5d640 1 --2- 172.21.15.76:0/1774571844 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f038006aea0 0x7f038006d360 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f039406dd30 tx=0x7f0388009040 comp rx=0 tx=0).stop 2023-11-09T03:48:15.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.376+0000 7f039aa5d640 1 -- 172.21.15.76:0/1774571844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0394070e20 msgr2=0x7f0394153df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:15.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.376+0000 7f039aa5d640 1 --2- 172.21.15.76:0/1774571844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0394070e20 0x7f0394153df0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f0384034040 tx=0x7f03840046c0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.376+0000 7f039aa5d640 1 -- 172.21.15.76:0/1774571844 shutdown_connections 2023-11-09T03:48:15.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.376+0000 7f039aa5d640 1 --2- 172.21.15.76:0/1774571844 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f038006aea0 0x7f038006d360 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.376+0000 7f039aa5d640 1 --2- 172.21.15.76:0/1774571844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f039415ad30 0x7f03940b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.376+0000 7f039aa5d640 1 --2- 172.21.15.76:0/1774571844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f03940ad730 0x7f03940afb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.376+0000 7f039aa5d640 1 --2- 172.21.15.76:0/1774571844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0394070e20 0x7f0394153df0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.376+0000 7f039aa5d640 1 -- 172.21.15.76:0/1774571844 >> 172.21.15.76:0/1774571844 conn(0x7f03940ce950 msgr2=0x7f039415b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:15.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.376+0000 7f039aa5d640 1 -- 172.21.15.76:0/1774571844 shutdown_connections 2023-11-09T03:48:15.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.376+0000 7f039aa5d640 1 -- 172.21.15.76:0/1774571844 wait complete. 2023-11-09T03:48:15.387 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-09T03:48:15.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.489+0000 7f34991fb640 1 Processor -- start 2023-11-09T03:48:15.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.489+0000 7f34991fb640 1 -- start start 2023-11-09T03:48:15.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.490+0000 7f34991fb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3494153e60 0x7f349415dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:15.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.490+0000 7f34991fb640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f34940cd6d0 0x7f3494161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:15.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.490+0000 7f34991fb640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34940cde60 con 0x7f34940b4880 2023-11-09T03:48:15.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.490+0000 7f34991fb640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3494160f90 con 0x7f34940cd6d0 2023-11-09T03:48:15.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.490+0000 7f34991fb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349415e400 con 0x7f3494153e60 2023-11-09T03:48:15.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.490+0000 7f349259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f34940cd6d0 0x7f3494161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:15.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.490+0000 7f349259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f34940cd6d0 0x7f3494161420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51286/0 (socket says 172.21.15.76:51286) 2023-11-09T03:48:15.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.490+0000 7f349259c640 1 -- 172.21.15.76:0/335784208 learned_addr learned my addr 172.21.15.76:0/335784208 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:15.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.490+0000 7f349259c640 1 -- 172.21.15.76:0/335784208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3494153e60 msgr2=0x7f349415dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:15.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.491+0000 7f349259c640 1 --2- 172.21.15.76:0/335784208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3494153e60 0x7f349415dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.491+0000 7f349259c640 1 -- 172.21.15.76:0/335784208 >> v1:172.21.15.113:6789/0 conn(0x7f34940b4880 legacy=0x7f349406b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:15.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.491+0000 7f349259c640 1 -- 172.21.15.76:0/335784208 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3494161960 con 0x7f34940cd6d0 2023-11-09T03:48:15.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.491+0000 7f349259c640 1 --2- 172.21.15.76:0/335784208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34940cd6d0 0x7f3494161420 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3488009870 tx=0x7f348802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e177cdd37b0d7a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:15.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.491+0000 7f3491d9b640 1 -- 172.21.15.76:0/335784208 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3488031640 con 0x7f34940cd6d0 2023-11-09T03:48:15.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.491+0000 7f3491d9b640 1 -- 172.21.15.76:0/335784208 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34880317a0 con 0x7f34940cd6d0 2023-11-09T03:48:15.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.491+0000 7f3491d9b640 1 -- 172.21.15.76:0/335784208 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3488031b30 con 0x7f34940cd6d0 2023-11-09T03:48:15.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.492+0000 7f34991fb640 1 -- 172.21.15.76:0/335784208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34940cd6d0 msgr2=0x7f3494161420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:15.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.492+0000 7f34991fb640 1 --2- 172.21.15.76:0/335784208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34940cd6d0 0x7f3494161420 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3488009870 tx=0x7f348802b8f0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.492+0000 7f34991fb640 1 -- 172.21.15.76:0/335784208 shutdown_connections 2023-11-09T03:48:15.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.492+0000 7f34991fb640 1 --2- 172.21.15.76:0/335784208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34940cd6d0 0x7f3494161420 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.492+0000 7f34991fb640 1 --2- 172.21.15.76:0/335784208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3494153e60 0x7f349415dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.492+0000 7f34991fb640 1 -- 172.21.15.76:0/335784208 >> 172.21.15.76:0/335784208 conn(0x7f34940cf880 msgr2=0x7f34940cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:15.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.492+0000 7f34991fb640 1 -- 172.21.15.76:0/335784208 shutdown_connections 2023-11-09T03:48:15.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.492+0000 7f34991fb640 1 -- 172.21.15.76:0/335784208 wait complete. 2023-11-09T03:48:15.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.494+0000 7f34991fb640 1 Processor -- start 2023-11-09T03:48:15.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.494+0000 7f34991fb640 1 -- start start 2023-11-09T03:48:15.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.494+0000 7f34991fb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34940b4880 0x7f34941583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:15.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.494+0000 7f34991fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34940cd6d0 0x7f349415e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:15.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.494+0000 7f34991fb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3494153e60 0x7f349415eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:15.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.494+0000 7f34991fb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34941f2640 con 0x7f34940cd6d0 2023-11-09T03:48:15.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.494+0000 7f34991fb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34941f2850 con 0x7f34940b4880 2023-11-09T03:48:15.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.494+0000 7f34991fb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34941f2b30 con 0x7f3494153e60 2023-11-09T03:48:15.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.494+0000 7f349259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34940cd6d0 0x7f349415e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:15.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.494+0000 7f349259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34940cd6d0 0x7f349415e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51292/0 (socket says 172.21.15.76:51292) 2023-11-09T03:48:15.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.494+0000 7f349259c640 1 -- 172.21.15.76:0/252953887 learned_addr learned my addr 172.21.15.76:0/252953887 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:15.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.494+0000 7f3492d9d640 1 --2- 172.21.15.76:0/252953887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34940b4880 0x7f34941583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:15.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.495+0000 7f349359e640 1 --2- 172.21.15.76:0/252953887 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3494153e60 0x7f349415eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:15.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.495+0000 7f349259c640 1 -- 172.21.15.76:0/252953887 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3494153e60 msgr2=0x7f349415eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:15.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.495+0000 7f349259c640 1 --2- 172.21.15.76:0/252953887 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3494153e60 0x7f349415eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.495+0000 7f349259c640 1 -- 172.21.15.76:0/252953887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34940b4880 msgr2=0x7f34941583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:15.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.495+0000 7f349259c640 1 --2- 172.21.15.76:0/252953887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34940b4880 0x7f34941583b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.495+0000 7f349259c640 1 -- 172.21.15.76:0/252953887 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34880094d0 con 0x7f34940cd6d0 2023-11-09T03:48:15.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.495+0000 7f349359e640 1 --2- 172.21.15.76:0/252953887 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3494153e60 0x7f349415eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:15.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.495+0000 7f349259c640 1 --2- 172.21.15.76:0/252953887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34940cd6d0 0x7f349415e5d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f3488009870 tx=0x7f34880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:15.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.495+0000 7f347b7fe640 1 -- 172.21.15.76:0/252953887 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348803ac60 con 0x7f34940cd6d0 2023-11-09T03:48:15.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.496+0000 7f347b7fe640 1 -- 172.21.15.76:0/252953887 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348803adc0 con 0x7f34940cd6d0 2023-11-09T03:48:15.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.496+0000 7f34991fb640 1 -- 172.21.15.76:0/252953887 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34941f2e10 con 0x7f34940cd6d0 2023-11-09T03:48:15.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.496+0000 7f347b7fe640 1 -- 172.21.15.76:0/252953887 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3488044630 con 0x7f34940cd6d0 2023-11-09T03:48:15.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.496+0000 7f34991fb640 1 -- 172.21.15.76:0/252953887 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34941f33d0 con 0x7f34940cd6d0 2023-11-09T03:48:15.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.497+0000 7f34991fb640 1 -- 172.21.15.76:0/252953887 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3494074290 con 0x7f34940cd6d0 2023-11-09T03:48:15.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.497+0000 7f347b7fe640 1 -- 172.21.15.76:0/252953887 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f348802f050 con 0x7f34940cd6d0 2023-11-09T03:48:15.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.498+0000 7f347b7fe640 1 --2- 172.21.15.76:0/252953887 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f348006ae70 0x7f348006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:15.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.498+0000 7f347b7fe640 1 -- 172.21.15.76:0/252953887 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f34880ab7f0 con 0x7f34940cd6d0 2023-11-09T03:48:15.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.498+0000 7f3492d9d640 1 --2- 172.21.15.76:0/252953887 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f348006ae70 0x7f348006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:15.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.498+0000 7f3492d9d640 1 --2- 172.21.15.76:0/252953887 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f348006ae70 0x7f348006d330 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f34840098e0 tx=0x7f3484009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:15.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.503+0000 7f347b7fe640 1 -- 172.21.15.76:0/252953887 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3488076620 con 0x7f34940cd6d0 2023-11-09T03:48:15.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.668+0000 7f34991fb640 1 -- 172.21.15.76:0/252953887 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7f34940d0b80 con 0x7f34940cd6d0 2023-11-09T03:48:15.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.668+0000 7f347b7fe640 1 -- 172.21.15.76:0/252953887 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v74) v1 ==== 103+0+11 (secure 0 0 0) 0x7f34880800a0 con 0x7f34940cd6d0 2023-11-09T03:48:15.669 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:48:15.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.670+0000 7f34991fb640 1 -- 172.21.15.76:0/252953887 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f348006ae70 msgr2=0x7f348006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:15.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.670+0000 7f34991fb640 1 --2- 172.21.15.76:0/252953887 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f348006ae70 0x7f348006d330 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f34840098e0 tx=0x7f3484009060 comp rx=0 tx=0).stop 2023-11-09T03:48:15.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.670+0000 7f34991fb640 1 -- 172.21.15.76:0/252953887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34940cd6d0 msgr2=0x7f349415e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:15.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.670+0000 7f34991fb640 1 --2- 172.21.15.76:0/252953887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34940cd6d0 0x7f349415e5d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f3488009870 tx=0x7f34880046c0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.670+0000 7f34991fb640 1 -- 172.21.15.76:0/252953887 shutdown_connections 2023-11-09T03:48:15.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.670+0000 7f34991fb640 1 --2- 172.21.15.76:0/252953887 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f348006ae70 0x7f348006d330 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.670+0000 7f34991fb640 1 --2- 172.21.15.76:0/252953887 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3494153e60 0x7f349415eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.670+0000 7f34991fb640 1 --2- 172.21.15.76:0/252953887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34940cd6d0 0x7f349415e5d0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.670+0000 7f34991fb640 1 --2- 172.21.15.76:0/252953887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34940b4880 0x7f34941583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.670+0000 7f34991fb640 1 -- 172.21.15.76:0/252953887 >> 172.21.15.76:0/252953887 conn(0x7f34940cf880 msgr2=0x7f34940cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:15.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.670+0000 7f34991fb640 1 -- 172.21.15.76:0/252953887 shutdown_connections 2023-11-09T03:48:15.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.670+0000 7f34991fb640 1 -- 172.21.15.76:0/252953887 wait complete. 2023-11-09T03:48:15.681 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-09T03:48:15.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.784+0000 7fc4c7ae5640 1 Processor -- start 2023-11-09T03:48:15.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.784+0000 7fc4c7ae5640 1 -- start start 2023-11-09T03:48:15.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.785+0000 7fc4c7ae5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4c015e5a0 0x7fc4c006b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:15.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.785+0000 7fc4c7ae5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc4c00c9af0 0x7fc4c0070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:15.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.785+0000 7fc4c7ae5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c0066530 con 0x7fc4c015e1c0 2023-11-09T03:48:15.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.785+0000 7fc4c7ae5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4c015f9e0 con 0x7fc4c00c9af0 2023-11-09T03:48:15.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.785+0000 7fc4c7ae5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c006bc40 con 0x7fc4c015e5a0 2023-11-09T03:48:15.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.785+0000 7fc4c6082640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc4c00c9af0 0x7fc4c0070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:15.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.785+0000 7fc4c6082640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc4c00c9af0 0x7fc4c0070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51296/0 (socket says 172.21.15.76:51296) 2023-11-09T03:48:15.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.785+0000 7fc4c6082640 1 -- 172.21.15.76:0/508847752 learned_addr learned my addr 172.21.15.76:0/508847752 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:15.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.785+0000 7fc4c5080640 1 --2- 172.21.15.76:0/508847752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4c015e5a0 0x7fc4c006b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:15.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.785+0000 7fc4c6082640 1 -- 172.21.15.76:0/508847752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4c015e5a0 msgr2=0x7fc4c006b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:15.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.785+0000 7fc4c6082640 1 --2- 172.21.15.76:0/508847752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4c015e5a0 0x7fc4c006b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.785+0000 7fc4c6082640 1 -- 172.21.15.76:0/508847752 >> v1:172.21.15.113:6789/0 conn(0x7fc4c015e1c0 legacy=0x7fc4c015f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:15.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.785+0000 7fc4c6082640 1 -- 172.21.15.76:0/508847752 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c0065f70 con 0x7fc4c00c9af0 2023-11-09T03:48:15.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.785+0000 7fc4c5080640 1 --2- 172.21.15.76:0/508847752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4c015e5a0 0x7fc4c006b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:15.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.786+0000 7fc4c6082640 1 --2- 172.21.15.76:0/508847752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4c00c9af0 0x7fc4c0070ff0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009e50 tx=0x7fc4b802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b137f376dbcdc07 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:15.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.786+0000 7fc4c487f640 1 -- 172.21.15.76:0/508847752 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b802ca70 con 0x7fc4c00c9af0 2023-11-09T03:48:15.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.786+0000 7fc4c487f640 1 -- 172.21.15.76:0/508847752 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b802cbd0 con 0x7fc4c00c9af0 2023-11-09T03:48:15.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.786+0000 7fc4c487f640 1 -- 172.21.15.76:0/508847752 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b80343f0 con 0x7fc4c00c9af0 2023-11-09T03:48:15.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.786+0000 7fc4c7ae5640 1 -- 172.21.15.76:0/508847752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4c00c9af0 msgr2=0x7fc4c0070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:15.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.786+0000 7fc4c7ae5640 1 --2- 172.21.15.76:0/508847752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4c00c9af0 0x7fc4c0070ff0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009e50 tx=0x7fc4b802bed0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.786+0000 7fc4c7ae5640 1 -- 172.21.15.76:0/508847752 shutdown_connections 2023-11-09T03:48:15.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.786+0000 7fc4c7ae5640 1 --2- 172.21.15.76:0/508847752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4c00c9af0 0x7fc4c0070ff0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.786+0000 7fc4c7ae5640 1 --2- 172.21.15.76:0/508847752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4c015e5a0 0x7fc4c006b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.786+0000 7fc4c7ae5640 1 -- 172.21.15.76:0/508847752 >> 172.21.15.76:0/508847752 conn(0x7fc4c00cbc90 msgr2=0x7fc4c00cc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:15.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.787+0000 7fc4c7ae5640 1 -- 172.21.15.76:0/508847752 shutdown_connections 2023-11-09T03:48:15.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.787+0000 7fc4c7ae5640 1 -- 172.21.15.76:0/508847752 wait complete. 2023-11-09T03:48:15.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.788+0000 7fc4c7ae5640 1 Processor -- start 2023-11-09T03:48:15.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.788+0000 7fc4c7ae5640 1 -- start start 2023-11-09T03:48:15.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.788+0000 7fc4c7ae5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4c00c9af0 0x7fc4c0153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:15.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.788+0000 7fc4c7ae5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4c015e1c0 0x7fc4c00c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:15.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.788+0000 7fc4c7ae5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4c015e5a0 0x7fc4c00c5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:15.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.788+0000 7fc4c7ae5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c00cdd60 con 0x7fc4c015e5a0 2023-11-09T03:48:15.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.788+0000 7fc4c7ae5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c0154480 con 0x7fc4c00c9af0 2023-11-09T03:48:15.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.788+0000 7fc4c7ae5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c0154680 con 0x7fc4c015e1c0 2023-11-09T03:48:15.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.788+0000 7fc4c6082640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4c015e5a0 0x7fc4c00c5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:15.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.788+0000 7fc4c6082640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4c015e5a0 0x7fc4c00c5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51310/0 (socket says 172.21.15.76:51310) 2023-11-09T03:48:15.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.788+0000 7fc4c6082640 1 -- 172.21.15.76:0/2334245411 learned_addr learned my addr 172.21.15.76:0/2334245411 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:15.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.789+0000 7fc4c5881640 1 --2- 172.21.15.76:0/2334245411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4c00c9af0 0x7fc4c0153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:15.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.789+0000 7fc4c5080640 1 --2- 172.21.15.76:0/2334245411 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4c015e1c0 0x7fc4c00c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:15.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.789+0000 7fc4c6082640 1 -- 172.21.15.76:0/2334245411 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4c015e1c0 msgr2=0x7fc4c00c4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:15.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.789+0000 7fc4c6082640 1 --2- 172.21.15.76:0/2334245411 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4c015e1c0 0x7fc4c00c4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.789+0000 7fc4c6082640 1 -- 172.21.15.76:0/2334245411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4c00c9af0 msgr2=0x7fc4c0153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:15.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.789+0000 7fc4c6082640 1 --2- 172.21.15.76:0/2334245411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4c00c9af0 0x7fc4c0153f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.789+0000 7fc4c6082640 1 -- 172.21.15.76:0/2334245411 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b8009b30 con 0x7fc4c015e5a0 2023-11-09T03:48:15.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.789+0000 7fc4c6082640 1 --2- 172.21.15.76:0/2334245411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4c015e5a0 0x7fc4c00c5290 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009e50 tx=0x7fc4b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:15.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.789+0000 7fc4b67fc640 1 -- 172.21.15.76:0/2334245411 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8009510 con 0x7fc4c015e5a0 2023-11-09T03:48:15.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.789+0000 7fc4b67fc640 1 -- 172.21.15.76:0/2334245411 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b803e450 con 0x7fc4c015e5a0 2023-11-09T03:48:15.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.789+0000 7fc4c7ae5640 1 -- 172.21.15.76:0/2334245411 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c00c8890 con 0x7fc4c015e5a0 2023-11-09T03:48:15.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.789+0000 7fc4b67fc640 1 -- 172.21.15.76:0/2334245411 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b803e800 con 0x7fc4c015e5a0 2023-11-09T03:48:15.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.789+0000 7fc4c7ae5640 1 -- 172.21.15.76:0/2334245411 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c00c7db0 con 0x7fc4c015e5a0 2023-11-09T03:48:15.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.790+0000 7fc4c7ae5640 1 -- 172.21.15.76:0/2334245411 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c0074290 con 0x7fc4c015e5a0 2023-11-09T03:48:15.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.791+0000 7fc4b67fc640 1 -- 172.21.15.76:0/2334245411 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc4b803e960 con 0x7fc4c015e5a0 2023-11-09T03:48:15.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.791+0000 7fc4b67fc640 1 --2- 172.21.15.76:0/2334245411 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc4a406ae70 0x7fc4a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:15.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.791+0000 7fc4b67fc640 1 -- 172.21.15.76:0/2334245411 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6326+0+0 (secure 0 0 0) 0x7fc4b80ac600 con 0x7fc4c015e5a0 2023-11-09T03:48:15.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.791+0000 7fc4c5881640 1 --2- 172.21.15.76:0/2334245411 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc4a406ae70 0x7fc4a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:15.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.792+0000 7fc4c5881640 1 --2- 172.21.15.76:0/2334245411 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc4a406ae70 0x7fc4a406d330 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc000f60 tx=0x7fc4bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:15.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.795+0000 7fc4b67fc640 1 -- 172.21.15.76:0/2334245411 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc4b8077440 con 0x7fc4c015e5a0 2023-11-09T03:48:15.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.962+0000 7fc4c7ae5640 1 -- 172.21.15.76:0/2334245411 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7fc4c00ccac0 con 0x7fc4c015e5a0 2023-11-09T03:48:15.963 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.962+0000 7fc4b67fc640 1 -- 172.21.15.76:0/2334245411 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v74) v1 ==== 107+0+11 (secure 0 0 0) 0x7fc4b807aa60 con 0x7fc4c015e5a0 2023-11-09T03:48:15.963 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:48:15.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.964+0000 7fc4c7ae5640 1 -- 172.21.15.76:0/2334245411 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc4a406ae70 msgr2=0x7fc4a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:15.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.964+0000 7fc4c7ae5640 1 --2- 172.21.15.76:0/2334245411 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc4a406ae70 0x7fc4a406d330 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc000f60 tx=0x7fc4bc009040 comp rx=0 tx=0).stop 2023-11-09T03:48:15.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.964+0000 7fc4c7ae5640 1 -- 172.21.15.76:0/2334245411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4c015e5a0 msgr2=0x7fc4c00c5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:15.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.964+0000 7fc4c7ae5640 1 --2- 172.21.15.76:0/2334245411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4c015e5a0 0x7fc4c00c5290 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009e50 tx=0x7fc4b8004a40 comp rx=0 tx=0).stop 2023-11-09T03:48:15.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.964+0000 7fc4c7ae5640 1 -- 172.21.15.76:0/2334245411 shutdown_connections 2023-11-09T03:48:15.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.964+0000 7fc4c7ae5640 1 --2- 172.21.15.76:0/2334245411 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc4a406ae70 0x7fc4a406d330 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.964+0000 7fc4c7ae5640 1 --2- 172.21.15.76:0/2334245411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4c015e5a0 0x7fc4c00c5290 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.964+0000 7fc4c7ae5640 1 --2- 172.21.15.76:0/2334245411 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4c015e1c0 0x7fc4c00c4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.964+0000 7fc4c7ae5640 1 --2- 172.21.15.76:0/2334245411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4c00c9af0 0x7fc4c0153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:15.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.964+0000 7fc4c7ae5640 1 -- 172.21.15.76:0/2334245411 >> 172.21.15.76:0/2334245411 conn(0x7fc4c00cbc90 msgr2=0x7fc4c0159de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:15.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.964+0000 7fc4c7ae5640 1 -- 172.21.15.76:0/2334245411 shutdown_connections 2023-11-09T03:48:15.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:15.964+0000 7fc4c7ae5640 1 -- 172.21.15.76:0/2334245411 wait complete. 2023-11-09T03:48:15.975 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_allow_new_snaps_config' 2023-11-09T03:48:16.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.075+0000 7f2ac3e7c640 1 Processor -- start 2023-11-09T03:48:16.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.075+0000 7f2ac3e7c640 1 -- start start 2023-11-09T03:48:16.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.076+0000 7f2ac3e7c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2abc159b80 0x7f2abc0b4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:16.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.076+0000 7f2ac3e7c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2abc159f60 0x7f2abc0ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:16.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.076+0000 7f2ac3e7c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2abc0ade60 con 0x7f2abc0ad910 2023-11-09T03:48:16.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.076+0000 7f2ac3e7c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2abc0ae140 con 0x7f2abc159f60 2023-11-09T03:48:16.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.076+0000 7f2ac3e7c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2abc06dc70 con 0x7f2abc159b80 2023-11-09T03:48:16.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.076+0000 7f2ac2419640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2abc159f60 0x7f2abc0ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:16.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.076+0000 7f2ac1417640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2abc0ad910 0x7f2abc0ce1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40310/0 (socket says 172.21.15.76:40310) 2023-11-09T03:48:16.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.076+0000 7f2ac1417640 1 -- 172.21.15.76:0/289465849 learned_addr learned my addr 172.21.15.76:0/289465849 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:16.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.076+0000 7f2ac2419640 1 -- 172.21.15.76:0/289465849 >> v2:172.21.15.113:3300/0 conn(0x7f2abc159b80 msgr2=0x7f2abc0b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:16.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.076+0000 7f2ac1c18640 1 --2- 172.21.15.76:0/289465849 >> v2:172.21.15.113:3300/0 conn(0x7f2abc159b80 0x7f2abc0b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:16.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.076+0000 7f2ac2419640 1 --2- 172.21.15.76:0/289465849 >> v2:172.21.15.113:3300/0 conn(0x7f2abc159b80 0x7f2abc0b4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:16.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.076+0000 7f2ac2419640 1 -- 172.21.15.76:0/289465849 >> v1:172.21.15.76:6789/0 conn(0x7f2abc0ad910 legacy=0x7f2abc0ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:16.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.076+0000 7f2ac0c16640 1 -- 172.21.15.76:0/289465849 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2943971342 0 0) 0x7f2abc0ade60 con 0x7f2abc0ad910 2023-11-09T03:48:16.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.077+0000 7f2ac2419640 1 -- 172.21.15.76:0/289465849 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2abc06df50 con 0x7f2abc159f60 2023-11-09T03:48:16.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.077+0000 7f2ac2419640 1 --2- 172.21.15.76:0/289465849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2abc159f60 0x7f2abc0ca6b0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0ce970 tx=0x7f2ab402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ab78fdd87e6e78 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:16.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.077+0000 7f2ac0c16640 1 -- 172.21.15.76:0/289465849 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab402cd00 con 0x7f2abc159f60 2023-11-09T03:48:16.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.077+0000 7f2ac0c16640 1 -- 172.21.15.76:0/289465849 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab402ce60 con 0x7f2abc159f60 2023-11-09T03:48:16.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.077+0000 7f2ac0c16640 1 -- 172.21.15.76:0/289465849 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4034660 con 0x7f2abc159f60 2023-11-09T03:48:16.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.078+0000 7f2ac3e7c640 1 -- 172.21.15.76:0/289465849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2abc159f60 msgr2=0x7f2abc0ca6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:16.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.078+0000 7f2ac3e7c640 1 --2- 172.21.15.76:0/289465849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2abc159f60 0x7f2abc0ca6b0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0ce970 tx=0x7f2ab402bf50 comp rx=0 tx=0).stop 2023-11-09T03:48:16.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.078+0000 7f2ac3e7c640 1 -- 172.21.15.76:0/289465849 shutdown_connections 2023-11-09T03:48:16.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.078+0000 7f2ac3e7c640 1 --2- 172.21.15.76:0/289465849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2abc159f60 0x7f2abc0ca6b0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:16.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.078+0000 7f2ac3e7c640 1 --2- 172.21.15.76:0/289465849 >> v2:172.21.15.113:3300/0 conn(0x7f2abc159b80 0x7f2abc0b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:16.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.078+0000 7f2ac3e7c640 1 -- 172.21.15.76:0/289465849 >> 172.21.15.76:0/289465849 conn(0x7f2abc0cb490 msgr2=0x7f2abc0cb870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:16.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.078+0000 7f2ac3e7c640 1 -- 172.21.15.76:0/289465849 shutdown_connections 2023-11-09T03:48:16.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.078+0000 7f2ac3e7c640 1 -- 172.21.15.76:0/289465849 wait complete. 2023-11-09T03:48:16.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.079+0000 7f2ac3e7c640 1 Processor -- start 2023-11-09T03:48:16.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.079+0000 7f2ac3e7c640 1 -- start start 2023-11-09T03:48:16.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.080+0000 7f2ac3e7c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2abc0ad910 0x7f2abc156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:16.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.080+0000 7f2ac3e7c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2abc159b80 0x7f2abc0afad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:16.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.080+0000 7f2ac3e7c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2abc159f60 0x7f2abc0ce950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:16.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.080+0000 7f2ac3e7c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc1f3700 con 0x7f2abc0ad910 2023-11-09T03:48:16.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.080+0000 7f2ac3e7c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc1f2ac0 con 0x7f2abc159f60 2023-11-09T03:48:16.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.080+0000 7f2ac3e7c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc156780 con 0x7f2abc159b80 2023-11-09T03:48:16.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.080+0000 7f2ac1c18640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2abc0ad910 0x7f2abc156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:16.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.080+0000 7f2ac1c18640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2abc0ad910 0x7f2abc156240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51330/0 (socket says 172.21.15.76:51330) 2023-11-09T03:48:16.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.080+0000 7f2ac1c18640 1 -- 172.21.15.76:0/639211263 learned_addr learned my addr 172.21.15.76:0/639211263 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:16.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.080+0000 7f2ac1c18640 1 -- 172.21.15.76:0/639211263 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2abc159b80 msgr2=0x7f2abc0afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:16.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.080+0000 7f2ac1c18640 1 --2- 172.21.15.76:0/639211263 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2abc159b80 0x7f2abc0afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:16.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.080+0000 7f2ac1c18640 1 -- 172.21.15.76:0/639211263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2abc159f60 msgr2=0x7f2abc0ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:16.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.080+0000 7f2ac2419640 1 --2- 172.21.15.76:0/639211263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2abc159f60 0x7f2abc0ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:16.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.080+0000 7f2ac1c18640 1 --2- 172.21.15.76:0/639211263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2abc159f60 0x7f2abc0ce950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:16.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.080+0000 7f2ac1c18640 1 -- 172.21.15.76:0/639211263 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab4009a70 con 0x7f2abc0ad910 2023-11-09T03:48:16.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.081+0000 7f2ac1c18640 1 --2- 172.21.15.76:0/639211263 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2abc0ad910 0x7f2abc156240 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800c7f0 tx=0x7f2ab800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:16.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.081+0000 7f2ab27fc640 1 -- 172.21.15.76:0/639211263 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8004220 con 0x7f2abc0ad910 2023-11-09T03:48:16.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.081+0000 7f2ab27fc640 1 -- 172.21.15.76:0/639211263 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab80043a0 con 0x7f2abc0ad910 2023-11-09T03:48:16.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.081+0000 7f2ac3e7c640 1 -- 172.21.15.76:0/639211263 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2abc1f2d60 con 0x7f2abc0ad910 2023-11-09T03:48:16.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.081+0000 7f2ab27fc640 1 -- 172.21.15.76:0/639211263 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8007860 con 0x7f2abc0ad910 2023-11-09T03:48:16.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.081+0000 7f2ac3e7c640 1 -- 172.21.15.76:0/639211263 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2abc1f3320 con 0x7f2abc0ad910 2023-11-09T03:48:16.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.082+0000 7f2ac3e7c640 1 -- 172.21.15.76:0/639211263 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2abc1f3840 con 0x7f2abc0ad910 2023-11-09T03:48:16.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.082+0000 7f2ab27fc640 1 -- 172.21.15.76:0/639211263 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2ab8007bd0 con 0x7f2abc0ad910 2023-11-09T03:48:16.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.083+0000 7f2ab27fc640 1 --2- 172.21.15.76:0/639211263 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2aa006ae70 0x7f2aa006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:16.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.083+0000 7f2ab27fc640 1 -- 172.21.15.76:0/639211263 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f2ab80897f0 con 0x7f2abc0ad910 2023-11-09T03:48:16.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.083+0000 7f2ac1417640 1 --2- 172.21.15.76:0/639211263 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2aa006ae70 0x7f2aa006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:16.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.083+0000 7f2ac1417640 1 --2- 172.21.15.76:0/639211263 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2aa006ae70 0x7f2aa006d330 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2aac007fd0 tx=0x7f2aac007920 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:16.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.087+0000 7f2ab27fc640 1 -- 172.21.15.76:0/639211263 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2abc1f3840 con 0x7f2abc0ad910 2023-11-09T03:48:16.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.317+0000 7f2ac3e7c640 1 -- 172.21.15.76:0/639211263 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_allow_new_snaps_config"]} v 0) v1 -- 0x7f2abc1f3840 con 0x7f2abc0ad910 2023-11-09T03:48:16.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.500+0000 7f2ab27fc640 1 -- 172.21.15.76:0/639211263 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_allow_new_snaps_config"]}]=0 v178) v1 ==== 151+0+0 (secure 0 0 0) 0x7f2abc1f3840 con 0x7f2abc0ad910 2023-11-09T03:48:16.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.502+0000 7f2ac3e7c640 1 -- 172.21.15.76:0/639211263 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2aa006ae70 msgr2=0x7f2aa006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:16.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.502+0000 7f2ac3e7c640 1 --2- 172.21.15.76:0/639211263 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2aa006ae70 0x7f2aa006d330 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2aac007fd0 tx=0x7f2aac007920 comp rx=0 tx=0).stop 2023-11-09T03:48:16.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.502+0000 7f2ac3e7c640 1 -- 172.21.15.76:0/639211263 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2abc0ad910 msgr2=0x7f2abc156240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:16.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.502+0000 7f2ac3e7c640 1 --2- 172.21.15.76:0/639211263 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2abc0ad910 0x7f2abc156240 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800c7f0 tx=0x7f2ab800ccb0 comp rx=0 tx=0).stop 2023-11-09T03:48:16.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.502+0000 7f2ac3e7c640 1 -- 172.21.15.76:0/639211263 shutdown_connections 2023-11-09T03:48:16.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.502+0000 7f2ac3e7c640 1 --2- 172.21.15.76:0/639211263 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2aa006ae70 0x7f2aa006d330 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:16.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.502+0000 7f2ac3e7c640 1 --2- 172.21.15.76:0/639211263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2abc159f60 0x7f2abc0ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:16.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.502+0000 7f2ac3e7c640 1 --2- 172.21.15.76:0/639211263 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2abc159b80 0x7f2abc0afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:16.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.502+0000 7f2ac3e7c640 1 --2- 172.21.15.76:0/639211263 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2abc0ad910 0x7f2abc156240 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:16.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.502+0000 7f2ac3e7c640 1 -- 172.21.15.76:0/639211263 >> 172.21.15.76:0/639211263 conn(0x7f2abc0cb490 msgr2=0x7f2abc0cd380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:16.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.502+0000 7f2ac3e7c640 1 -- 172.21.15.76:0/639211263 shutdown_connections 2023-11-09T03:48:16.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.502+0000 7f2ac3e7c640 1 -- 172.21.15.76:0/639211263 wait complete. 2023-11-09T03:48:16.514 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-09T03:48:16.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.617+0000 7f7866539640 1 Processor -- start 2023-11-09T03:48:16.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.617+0000 7f7866539640 1 -- start start 2023-11-09T03:48:16.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.617+0000 7f7866539640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7860153e60 0x7f786015dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:16.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.617+0000 7f7866539640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f78600cd6d0 0x7f7860161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:16.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.617+0000 7f7866539640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78600cde60 con 0x7f78600b4880 2023-11-09T03:48:16.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.617+0000 7f7866539640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7860160f90 con 0x7f78600cd6d0 2023-11-09T03:48:16.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.617+0000 7f7866539640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786015e400 con 0x7f7860153e60 2023-11-09T03:48:16.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.618+0000 7f785f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f78600cd6d0 0x7f7860161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:16.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.618+0000 7f785f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f78600cd6d0 0x7f7860161420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51334/0 (socket says 172.21.15.76:51334) 2023-11-09T03:48:16.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.618+0000 7f785f7fe640 1 -- 172.21.15.76:0/3733286378 learned_addr learned my addr 172.21.15.76:0/3733286378 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:16.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.618+0000 7f785ffff640 1 --2- 172.21.15.76:0/3733286378 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7860153e60 0x7f786015dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:16.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.618+0000 7f785f7fe640 1 -- 172.21.15.76:0/3733286378 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7860153e60 msgr2=0x7f786015dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:16.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.618+0000 7f785f7fe640 1 --2- 172.21.15.76:0/3733286378 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7860153e60 0x7f786015dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:16.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.618+0000 7f785f7fe640 1 -- 172.21.15.76:0/3733286378 >> v1:172.21.15.113:6789/0 conn(0x7f78600b4880 legacy=0x7f786006b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:16.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.618+0000 7f785f7fe640 1 -- 172.21.15.76:0/3733286378 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7860161960 con 0x7f78600cd6d0 2023-11-09T03:48:16.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.619+0000 7f785f7fe640 1 --2- 172.21.15.76:0/3733286378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78600cd6d0 0x7f7860161420 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f7854009870 tx=0x7f785402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32fe4f6160228e4a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:16.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.619+0000 7f785effd640 1 -- 172.21.15.76:0/3733286378 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7854030840 con 0x7f78600cd6d0 2023-11-09T03:48:16.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.619+0000 7f785effd640 1 -- 172.21.15.76:0/3733286378 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78540309a0 con 0x7f78600cd6d0 2023-11-09T03:48:16.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.619+0000 7f785effd640 1 -- 172.21.15.76:0/3733286378 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7854030d30 con 0x7f78600cd6d0 2023-11-09T03:48:16.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.619+0000 7f7866539640 1 -- 172.21.15.76:0/3733286378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78600cd6d0 msgr2=0x7f7860161420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:16.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.619+0000 7f7866539640 1 --2- 172.21.15.76:0/3733286378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78600cd6d0 0x7f7860161420 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f7854009870 tx=0x7f785402b980 comp rx=0 tx=0).stop 2023-11-09T03:48:16.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.619+0000 7f7866539640 1 -- 172.21.15.76:0/3733286378 shutdown_connections 2023-11-09T03:48:16.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.619+0000 7f7866539640 1 --2- 172.21.15.76:0/3733286378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78600cd6d0 0x7f7860161420 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:16.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.619+0000 7f7866539640 1 --2- 172.21.15.76:0/3733286378 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7860153e60 0x7f786015dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:16.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.619+0000 7f7866539640 1 -- 172.21.15.76:0/3733286378 >> 172.21.15.76:0/3733286378 conn(0x7f78600cf880 msgr2=0x7f78600cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:16.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.619+0000 7f7866539640 1 -- 172.21.15.76:0/3733286378 shutdown_connections 2023-11-09T03:48:16.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.619+0000 7f7866539640 1 -- 172.21.15.76:0/3733286378 wait complete. 2023-11-09T03:48:16.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.620+0000 7f7866539640 1 Processor -- start 2023-11-09T03:48:16.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.621+0000 7f7866539640 1 -- start start 2023-11-09T03:48:16.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.621+0000 7f7866539640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78600b4880 0x7f78600c9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:16.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.621+0000 7f7866539640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78600cd6d0 0x7f7860159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:16.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.621+0000 7f7866539640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7860153e60 0x7f78600ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:16.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.621+0000 7f7866539640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78600c97c0 con 0x7f7860153e60 2023-11-09T03:48:16.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.621+0000 7f7866539640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7860158bc0 con 0x7f78600cd6d0 2023-11-09T03:48:16.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.621+0000 7f7866539640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7860159a20 con 0x7f78600b4880 2023-11-09T03:48:16.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.621+0000 7f7864ad6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7860153e60 0x7f78600ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:16.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.621+0000 7f7864ad6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7860153e60 0x7f78600ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51346/0 (socket says 172.21.15.76:51346) 2023-11-09T03:48:16.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.621+0000 7f7864ad6640 1 -- 172.21.15.76:0/2541246384 learned_addr learned my addr 172.21.15.76:0/2541246384 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:16.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.622+0000 7f785f7fe640 1 --2- 172.21.15.76:0/2541246384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78600cd6d0 0x7f7860159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:16.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.622+0000 7f7864ad6640 1 -- 172.21.15.76:0/2541246384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78600b4880 msgr2=0x7f78600c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:16.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.622+0000 7f7864ad6640 1 --2- 172.21.15.76:0/2541246384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78600b4880 0x7f78600c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:16.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.622+0000 7f7864ad6640 1 -- 172.21.15.76:0/2541246384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78600cd6d0 msgr2=0x7f7860159eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:16.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.622+0000 7f7864ad6640 1 --2- 172.21.15.76:0/2541246384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78600cd6d0 0x7f7860159eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:16.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.622+0000 7f7864ad6640 1 -- 172.21.15.76:0/2541246384 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78540094d0 con 0x7f7860153e60 2023-11-09T03:48:16.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.622+0000 7f7864ad6640 1 --2- 172.21.15.76:0/2541246384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7860153e60 0x7f78600ccbc0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f785800c7a0 tx=0x7f785800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:16.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.622+0000 7f785cff9640 1 -- 172.21.15.76:0/2541246384 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858003b70 con 0x7f7860153e60 2023-11-09T03:48:16.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.622+0000 7f785cff9640 1 -- 172.21.15.76:0/2541246384 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7858003cf0 con 0x7f7860153e60 2023-11-09T03:48:16.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.622+0000 7f7866539640 1 -- 172.21.15.76:0/2541246384 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7860158e10 con 0x7f7860153e60 2023-11-09T03:48:16.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.622+0000 7f785cff9640 1 -- 172.21.15.76:0/2541246384 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785800aa90 con 0x7f7860153e60 2023-11-09T03:48:16.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.622+0000 7f7866539640 1 -- 172.21.15.76:0/2541246384 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78601593d0 con 0x7f7860153e60 2023-11-09T03:48:16.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.623+0000 7f7866539640 1 -- 172.21.15.76:0/2541246384 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7860074290 con 0x7f7860153e60 2023-11-09T03:48:16.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.623+0000 7f785cff9640 1 -- 172.21.15.76:0/2541246384 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f785800abf0 con 0x7f7860153e60 2023-11-09T03:48:16.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.624+0000 7f785cff9640 1 --2- 172.21.15.76:0/2541246384 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f784c06ae70 0x7f784c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:16.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.624+0000 7f785cff9640 1 -- 172.21.15.76:0/2541246384 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f7858089650 con 0x7f7860153e60 2023-11-09T03:48:16.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.624+0000 7f785ffff640 1 --2- 172.21.15.76:0/2541246384 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f784c06ae70 0x7f784c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:16.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.624+0000 7f785ffff640 1 --2- 172.21.15.76:0/2541246384 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f784c06ae70 0x7f784c06d330 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7850006fd0 tx=0x7f7850008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:16.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.628+0000 7f785cff9640 1 -- 172.21.15.76:0/2541246384 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7858054520 con 0x7f7860153e60 2023-11-09T03:48:16.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.793+0000 7f7866539640 1 -- 172.21.15.76:0/2541246384 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7860159b60 con 0x7f7860153e60 2023-11-09T03:48:16.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.798+0000 7f785cff9640 1 -- 172.21.15.76:0/2541246384 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v17)=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7860159b60 con 0x7f7860153e60 2023-11-09T03:48:16.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.800+0000 7f7866539640 1 -- 172.21.15.76:0/2541246384 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f784c06ae70 msgr2=0x7f784c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:16.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.800+0000 7f7866539640 1 --2- 172.21.15.76:0/2541246384 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f784c06ae70 0x7f784c06d330 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7850006fd0 tx=0x7f7850008040 comp rx=0 tx=0).stop 2023-11-09T03:48:16.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.800+0000 7f7866539640 1 -- 172.21.15.76:0/2541246384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7860153e60 msgr2=0x7f78600ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:16.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.800+0000 7f7866539640 1 --2- 172.21.15.76:0/2541246384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7860153e60 0x7f78600ccbc0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f785800c7a0 tx=0x7f785800cc60 comp rx=0 tx=0).stop 2023-11-09T03:48:16.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.800+0000 7f7866539640 1 -- 172.21.15.76:0/2541246384 shutdown_connections 2023-11-09T03:48:16.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.800+0000 7f7866539640 1 --2- 172.21.15.76:0/2541246384 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f784c06ae70 0x7f784c06d330 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:16.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.800+0000 7f7866539640 1 --2- 172.21.15.76:0/2541246384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7860153e60 0x7f78600ccbc0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:16.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.800+0000 7f7866539640 1 --2- 172.21.15.76:0/2541246384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78600cd6d0 0x7f7860159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:16.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.800+0000 7f7866539640 1 --2- 172.21.15.76:0/2541246384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78600b4880 0x7f78600c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:16.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.800+0000 7f7866539640 1 -- 172.21.15.76:0/2541246384 >> 172.21.15.76:0/2541246384 conn(0x7f78600cf880 msgr2=0x7f78600cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:16.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.801+0000 7f7866539640 1 -- 172.21.15.76:0/2541246384 shutdown_connections 2023-11-09T03:48:16.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:16.801+0000 7f7866539640 1 -- 172.21.15.76:0/2541246384 wait complete. 2023-11-09T03:48:16.812 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-09T03:48:16.812 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-09T03:48:16.851 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-09T03:48:16.896 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-09T03:48:16.897 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:48:16.897 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:48:16.912 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056381,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14761 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248868k,nr_inodes=812217,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-09T03:48:16.912 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:48:16.913 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:48:16.930 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2440 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-09T03:48:16.930 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:48:16.930 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:48:16.967 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056381,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14761 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248868k,nr_inodes=812217,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-09T03:48:16.967 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:48:16.967 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:48:16.986 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2440 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-09T03:48:16.986 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-09T03:48:16.987 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:48:17.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.105+0000 7fbcb0aec640 1 Processor -- start 2023-11-09T03:48:17.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.105+0000 7fbcb0aec640 1 -- start start 2023-11-09T03:48:17.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.106+0000 7fbcb0aec640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbcac157710 0x7fbcac167b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:17.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.106+0000 7fbcb0aec640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcac15e000 con 0x7fbcac0ce240 2023-11-09T03:48:17.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.106+0000 7fbcb0aec640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcac15e2e0 con 0x7fbcac0cde60 2023-11-09T03:48:17.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.106+0000 7fbcb0aec640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcac15e5c0 con 0x7fbcac157710 2023-11-09T03:48:17.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.106+0000 7fbcaad9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbcac157710 0x7fbcac167b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:17.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.106+0000 7fbca9d9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fbcac0ce240 0x7fbcac157000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40318/0 (socket says 172.21.15.76:40318) 2023-11-09T03:48:17.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.106+0000 7fbca9d9b640 1 -- 172.21.15.76:0/555659759 learned_addr learned my addr 172.21.15.76:0/555659759 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:17.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.107+0000 7fbcaad9d640 1 -- 172.21.15.76:0/555659759 >> v1:172.21.15.113:6789/0 conn(0x7fbcac0cde60 legacy=0x7fbcac15d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:17.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.107+0000 7fbcaad9d640 1 -- 172.21.15.76:0/555659759 >> v1:172.21.15.76:6789/0 conn(0x7fbcac0ce240 legacy=0x7fbcac157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:17.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.107+0000 7fbcaad9d640 1 -- 172.21.15.76:0/555659759 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcac0d1120 con 0x7fbcac157710 2023-11-09T03:48:17.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.107+0000 7fbca959a640 1 -- 172.21.15.76:0/555659759 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 429724824 0 0) 0x7fbcac15e000 con 0x7fbcac0ce240 2023-11-09T03:48:17.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.107+0000 7fbcaad9d640 1 --2- 172.21.15.76:0/555659759 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcac157710 0x7fbcac167b90 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fbca40097a0 tx=0x7fbca402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fd61f258603184 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:17.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.108+0000 7fbca959a640 1 -- 172.21.15.76:0/555659759 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca4031830 con 0x7fbcac157710 2023-11-09T03:48:17.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.108+0000 7fbca959a640 1 -- 172.21.15.76:0/555659759 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca4031990 con 0x7fbcac157710 2023-11-09T03:48:17.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.108+0000 7fbca959a640 1 -- 172.21.15.76:0/555659759 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca4031d20 con 0x7fbcac157710 2023-11-09T03:48:17.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.108+0000 7fbcb0aec640 1 -- 172.21.15.76:0/555659759 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcac157710 msgr2=0x7fbcac167b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:17.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.108+0000 7fbcb0aec640 1 --2- 172.21.15.76:0/555659759 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcac157710 0x7fbcac167b90 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fbca40097a0 tx=0x7fbca402b8d0 comp rx=0 tx=0).stop 2023-11-09T03:48:17.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.108+0000 7fbcb0aec640 1 -- 172.21.15.76:0/555659759 shutdown_connections 2023-11-09T03:48:17.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.108+0000 7fbcb0aec640 1 --2- 172.21.15.76:0/555659759 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcac157710 0x7fbcac167b90 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:17.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.108+0000 7fbcb0aec640 1 -- 172.21.15.76:0/555659759 >> 172.21.15.76:0/555659759 conn(0x7fbcac071f30 msgr2=0x7fbcac0cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:17.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.108+0000 7fbcb0aec640 1 -- 172.21.15.76:0/555659759 shutdown_connections 2023-11-09T03:48:17.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.108+0000 7fbcb0aec640 1 -- 172.21.15.76:0/555659759 wait complete. 2023-11-09T03:48:17.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.109+0000 7fbcb0aec640 1 Processor -- start 2023-11-09T03:48:17.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.109+0000 7fbcb0aec640 1 -- start start 2023-11-09T03:48:17.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.110+0000 7fbcb0aec640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcac0cde60 0x7fbcac0c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:17.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.110+0000 7fbcb0aec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcac0ce240 0x7fbcac0b6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:17.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.110+0000 7fbcb0aec640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbcac157710 0x7fbcac14ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:17.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.110+0000 7fbcb0aec640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac15ad00 con 0x7fbcac0ce240 2023-11-09T03:48:17.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.110+0000 7fbcb0aec640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac15af50 con 0x7fbcac0cde60 2023-11-09T03:48:17.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.110+0000 7fbcb0aec640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac15b230 con 0x7fbcac157710 2023-11-09T03:48:17.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.110+0000 7fbca9d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcac0ce240 0x7fbcac0b6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:17.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.110+0000 7fbca9d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcac0ce240 0x7fbcac0b6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51354/0 (socket says 172.21.15.76:51354) 2023-11-09T03:48:17.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.110+0000 7fbca9d9b640 1 -- 172.21.15.76:0/1399160776 learned_addr learned my addr 172.21.15.76:0/1399160776 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:17.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.111+0000 7fbcaa59c640 1 --2- 172.21.15.76:0/1399160776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcac0cde60 0x7fbcac0c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:17.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.111+0000 7fbca9d9b640 1 -- 172.21.15.76:0/1399160776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbcac157710 msgr2=0x7fbcac14ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:17.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.111+0000 7fbca9d9b640 1 --2- 172.21.15.76:0/1399160776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbcac157710 0x7fbcac14ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:17.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.111+0000 7fbca9d9b640 1 -- 172.21.15.76:0/1399160776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcac0cde60 msgr2=0x7fbcac0c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:17.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.111+0000 7fbca9d9b640 1 --2- 172.21.15.76:0/1399160776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcac0cde60 0x7fbcac0c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:17.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.111+0000 7fbca9d9b640 1 -- 172.21.15.76:0/1399160776 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca4009400 con 0x7fbcac0ce240 2023-11-09T03:48:17.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.111+0000 7fbcaa59c640 1 --2- 172.21.15.76:0/1399160776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcac0cde60 0x7fbcac0c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:17.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.111+0000 7fbca9d9b640 1 --2- 172.21.15.76:0/1399160776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcac0ce240 0x7fbcac0b6780 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fbca000c4d0 tx=0x7fbca000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:17.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.111+0000 7fbc9affd640 1 -- 172.21.15.76:0/1399160776 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca00074e0 con 0x7fbcac0ce240 2023-11-09T03:48:17.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.111+0000 7fbc9affd640 1 -- 172.21.15.76:0/1399160776 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca0007660 con 0x7fbcac0ce240 2023-11-09T03:48:17.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.111+0000 7fbcb0aec640 1 -- 172.21.15.76:0/1399160776 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcac15b510 con 0x7fbcac0ce240 2023-11-09T03:48:17.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.111+0000 7fbc9affd640 1 -- 172.21.15.76:0/1399160776 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca0004a20 con 0x7fbcac0ce240 2023-11-09T03:48:17.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.111+0000 7fbcb0aec640 1 -- 172.21.15.76:0/1399160776 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcac15bad0 con 0x7fbcac0ce240 2023-11-09T03:48:17.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.112+0000 7fbcb0aec640 1 -- 172.21.15.76:0/1399160776 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcac15bcf0 con 0x7fbcac0ce240 2023-11-09T03:48:17.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.112+0000 7fbc9affd640 1 -- 172.21.15.76:0/1399160776 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbca0004b80 con 0x7fbcac0ce240 2023-11-09T03:48:17.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.113+0000 7fbc9affd640 1 --2- 172.21.15.76:0/1399160776 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbc8c06ae70 0x7fbc8c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:17.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.113+0000 7fbc9affd640 1 -- 172.21.15.76:0/1399160776 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6326+0+0 (secure 0 0 0) 0x7fbca0088db0 con 0x7fbcac0ce240 2023-11-09T03:48:17.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.113+0000 7fbcaa59c640 1 --2- 172.21.15.76:0/1399160776 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbc8c06ae70 0x7fbc8c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:17.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.113+0000 7fbcaa59c640 1 --2- 172.21.15.76:0/1399160776 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbc8c06ae70 0x7fbc8c06d330 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c004520 tx=0x7fbc9c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:17.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.117+0000 7fbc9affd640 1 -- 172.21.15.76:0/1399160776 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbca0053bf0 con 0x7fbcac0ce240 2023-11-09T03:48:17.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.315+0000 7fbcb0aec640 1 -- 172.21.15.76:0/1399160776 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbcac156ad0 con 0x7fbcac0ce240 2023-11-09T03:48:17.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.315+0000 7fbc9affd640 1 -- 172.21.15.76:0/1399160776 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+3702 (secure 0 0 0) 0x7fbca0018020 con 0x7fbcac0ce240 2023-11-09T03:48:17.316 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:17.316 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":31,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2800733670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2800733670},{"type":"v1","addr":"172.21.15.113:6836","nonce":2800733670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":5791,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1301905851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1301905851},{"type":"v1","addr":"172.21.15.113:6837","nonce":1301905851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":6498,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/968739949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":968739949},{"type":"v1","addr":"172.21.15.76:6833","nonce":968739949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":6876,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1260068698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1260068698},{"type":"v1","addr":"172.21.15.76:6835","nonce":1260068698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[]} 2023-11-09T03:48:17.316 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 31 2023-11-09T03:48:17.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.317+0000 7fbcb0aec640 1 -- 172.21.15.76:0/1399160776 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbc8c06ae70 msgr2=0x7fbc8c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:17.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.317+0000 7fbcb0aec640 1 --2- 172.21.15.76:0/1399160776 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbc8c06ae70 0x7fbc8c06d330 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c004520 tx=0x7fbc9c009040 comp rx=0 tx=0).stop 2023-11-09T03:48:17.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.317+0000 7fbcb0aec640 1 -- 172.21.15.76:0/1399160776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcac0ce240 msgr2=0x7fbcac0b6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:17.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.317+0000 7fbcb0aec640 1 --2- 172.21.15.76:0/1399160776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcac0ce240 0x7fbcac0b6780 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fbca000c4d0 tx=0x7fbca000c990 comp rx=0 tx=0).stop 2023-11-09T03:48:17.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.317+0000 7fbcb0aec640 1 -- 172.21.15.76:0/1399160776 shutdown_connections 2023-11-09T03:48:17.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.317+0000 7fbcb0aec640 1 --2- 172.21.15.76:0/1399160776 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbc8c06ae70 0x7fbc8c06d330 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:17.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.317+0000 7fbcb0aec640 1 --2- 172.21.15.76:0/1399160776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbcac157710 0x7fbcac14ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:17.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.317+0000 7fbcb0aec640 1 --2- 172.21.15.76:0/1399160776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcac0ce240 0x7fbcac0b6780 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:17.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.317+0000 7fbcb0aec640 1 --2- 172.21.15.76:0/1399160776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcac0cde60 0x7fbcac0c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:17.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.317+0000 7fbcb0aec640 1 -- 172.21.15.76:0/1399160776 >> 172.21.15.76:0/1399160776 conn(0x7fbcac071f30 msgr2=0x7fbcac0d09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:17.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.317+0000 7fbcb0aec640 1 -- 172.21.15.76:0/1399160776 shutdown_connections 2023-11-09T03:48:17.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.317+0000 7fbcb0aec640 1 -- 172.21.15.76:0/1399160776 wait complete. 2023-11-09T03:48:17.328 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-09T03:48:17.329 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-09T03:48:17.329 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-09T03:48:17.329 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:48:17.329 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-09T03:48:17.330 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-09T03:48:17.330 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-09T03:48:17.330 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:48:17.331 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-09T03:48:17.331 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-09T03:48:17.331 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-09T03:48:17.331 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:48:17.332 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-09T03:48:17.332 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-09T03:48:17.332 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-09T03:48:17.332 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:48:17.338 INFO:tasks.ceph.mds.a:Stopped 2023-11-09T03:48:17.338 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-09T03:48:17.340 INFO:tasks.ceph.mds.c:Stopped 2023-11-09T03:48:17.340 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-09T03:48:17.343 INFO:tasks.ceph.mds.b:Stopped 2023-11-09T03:48:17.343 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-09T03:48:17.344 INFO:tasks.ceph.mds.d:Stopped 2023-11-09T03:48:17.344 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-09T03:48:17.382 INFO:tasks.ceph.mds.a:Started 2023-11-09T03:48:17.382 DEBUG:teuthology.parallel:result is None 2023-11-09T03:48:17.383 INFO:tasks.ceph.mds.c:Started 2023-11-09T03:48:17.384 DEBUG:teuthology.parallel:result is None 2023-11-09T03:48:17.386 INFO:tasks.ceph.mds.b:Started 2023-11-09T03:48:17.387 DEBUG:teuthology.parallel:result is None 2023-11-09T03:48:17.387 INFO:tasks.ceph.mds.d:Started 2023-11-09T03:48:17.388 DEBUG:teuthology.parallel:result is None 2023-11-09T03:48:17.388 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-09T03:48:17.458 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2023-11-09T03:48:17.459 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2023-11-09T03:48:17.472 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2023-11-09T03:48:17.472 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2023-11-09T03:48:17.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.494+0000 7f8a8471d640 1 Processor -- start 2023-11-09T03:48:17.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.494+0000 7f8a8471d640 1 -- start start 2023-11-09T03:48:17.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.495+0000 7f8a8471d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8a7c153850 0x7f8a7c15a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:17.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.495+0000 7f8a8471d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8a7c0cc520 0x7f8a7c070fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:17.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.495+0000 7f8a8471d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a7c0c3860 con 0x7f8a7c0cc9d0 2023-11-09T03:48:17.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.495+0000 7f8a8471d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a7c0c9250 con 0x7f8a7c0cc520 2023-11-09T03:48:17.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.495+0000 7f8a8471d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a7c153c30 con 0x7f8a7c153850 2023-11-09T03:48:17.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.495+0000 7f8a81cb8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8a7c0cc520 0x7f8a7c070fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:17.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.496+0000 7f8a82cba640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8a7c0cc9d0 0x7f8a7c0c3cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40336/0 (socket says 172.21.15.76:40336) 2023-11-09T03:48:17.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.496+0000 7f8a82cba640 1 -- 172.21.15.76:0/1063762405 learned_addr learned my addr 172.21.15.76:0/1063762405 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:17.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.496+0000 7f8a824b9640 1 --2- 172.21.15.76:0/1063762405 >> v2:172.21.15.113:3300/0 conn(0x7f8a7c153850 0x7f8a7c15a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:17.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.496+0000 7f8a81cb8640 1 -- 172.21.15.76:0/1063762405 >> v2:172.21.15.113:3300/0 conn(0x7f8a7c153850 msgr2=0x7f8a7c15a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:17.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.496+0000 7f8a81cb8640 1 --2- 172.21.15.76:0/1063762405 >> v2:172.21.15.113:3300/0 conn(0x7f8a7c153850 0x7f8a7c15a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:17.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.496+0000 7f8a81cb8640 1 -- 172.21.15.76:0/1063762405 >> v1:172.21.15.76:6789/0 conn(0x7f8a7c0cc9d0 legacy=0x7f8a7c0c3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:17.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.496+0000 7f8a81cb8640 1 -- 172.21.15.76:0/1063762405 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7c15a700 con 0x7f8a7c0cc520 2023-11-09T03:48:17.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.496+0000 7f8a814b7640 1 -- 172.21.15.76:0/1063762405 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 462323414 0 0) 0x7f8a7c0c3860 con 0x7f8a7c0cc9d0 2023-11-09T03:48:17.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.497+0000 7f8a81cb8640 1 --2- 172.21.15.76:0/1063762405 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a7c0cc520 0x7f8a7c070fa0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c0c5710 tx=0x7f8a7802cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e1625454786271 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:17.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.497+0000 7f8a814b7640 1 -- 172.21.15.76:0/1063762405 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7802db50 con 0x7f8a7c0cc520 2023-11-09T03:48:17.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.497+0000 7f8a814b7640 1 -- 172.21.15.76:0/1063762405 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7802dcb0 con 0x7f8a7c0cc520 2023-11-09T03:48:17.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.497+0000 7f8a814b7640 1 -- 172.21.15.76:0/1063762405 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a78035660 con 0x7f8a7c0cc520 2023-11-09T03:48:17.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.497+0000 7f8a8471d640 1 -- 172.21.15.76:0/1063762405 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a7c0cc520 msgr2=0x7f8a7c070fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:17.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.497+0000 7f8a8471d640 1 --2- 172.21.15.76:0/1063762405 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a7c0cc520 0x7f8a7c070fa0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c0c5710 tx=0x7f8a7802cda0 comp rx=0 tx=0).stop 2023-11-09T03:48:17.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.497+0000 7f8a8471d640 1 -- 172.21.15.76:0/1063762405 shutdown_connections 2023-11-09T03:48:17.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.497+0000 7f8a8471d640 1 --2- 172.21.15.76:0/1063762405 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a7c0cc520 0x7f8a7c070fa0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:17.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.497+0000 7f8a8471d640 1 --2- 172.21.15.76:0/1063762405 >> v2:172.21.15.113:3300/0 conn(0x7f8a7c153850 0x7f8a7c15a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:17.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.497+0000 7f8a8471d640 1 -- 172.21.15.76:0/1063762405 >> 172.21.15.76:0/1063762405 conn(0x7f8a7c0ce950 msgr2=0x7f8a7c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:17.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.497+0000 7f8a8471d640 1 -- 172.21.15.76:0/1063762405 shutdown_connections 2023-11-09T03:48:17.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.497+0000 7f8a8471d640 1 -- 172.21.15.76:0/1063762405 wait complete. 2023-11-09T03:48:17.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.498+0000 7f8a8471d640 1 Processor -- start 2023-11-09T03:48:17.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.498+0000 7f8a8471d640 1 -- start start 2023-11-09T03:48:17.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.499+0000 7f8a8471d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a7c0cc520 0x7f8a7c14d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:17.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.499+0000 7f8a8471d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a7c0cc9d0 0x7f8a7c0b3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:17.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.499+0000 7f8a8471d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a7c153850 0x7f8a7c0b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:17.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.499+0000 7f8a8471d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c150010 con 0x7f8a7c0cc9d0 2023-11-09T03:48:17.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.499+0000 7f8a8471d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c1502f0 con 0x7f8a7c0cc520 2023-11-09T03:48:17.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.499+0000 7f8a8471d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c1505d0 con 0x7f8a7c153850 2023-11-09T03:48:17.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.499+0000 7f8a81cb8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a7c0cc9d0 0x7f8a7c0b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:17.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.499+0000 7f8a81cb8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a7c0cc9d0 0x7f8a7c0b3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51410/0 (socket says 172.21.15.76:51410) 2023-11-09T03:48:17.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.499+0000 7f8a81cb8640 1 -- 172.21.15.76:0/2826161031 learned_addr learned my addr 172.21.15.76:0/2826161031 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:17.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.499+0000 7f8a824b9640 1 --2- 172.21.15.76:0/2826161031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a7c0cc520 0x7f8a7c14d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:17.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.499+0000 7f8a82cba640 1 --2- 172.21.15.76:0/2826161031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a7c153850 0x7f8a7c0b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:17.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.500+0000 7f8a81cb8640 1 -- 172.21.15.76:0/2826161031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a7c153850 msgr2=0x7f8a7c0b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:17.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.500+0000 7f8a81cb8640 1 --2- 172.21.15.76:0/2826161031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a7c153850 0x7f8a7c0b3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:17.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.500+0000 7f8a81cb8640 1 -- 172.21.15.76:0/2826161031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a7c0cc520 msgr2=0x7f8a7c14d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:17.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.500+0000 7f8a81cb8640 1 --2- 172.21.15.76:0/2826161031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a7c0cc520 0x7f8a7c14d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:17.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.500+0000 7f8a81cb8640 1 -- 172.21.15.76:0/2826161031 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7800a5a0 con 0x7f8a7c0cc9d0 2023-11-09T03:48:17.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.500+0000 7f8a81cb8640 1 --2- 172.21.15.76:0/2826161031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a7c0cc9d0 0x7f8a7c0b3740 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f8a78038040 tx=0x7f8a780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:17.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.500+0000 7f8a72ffd640 1 -- 172.21.15.76:0/2826161031 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7803e610 con 0x7f8a7c0cc9d0 2023-11-09T03:48:17.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.500+0000 7f8a72ffd640 1 -- 172.21.15.76:0/2826161031 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7803e770 con 0x7f8a7c0cc9d0 2023-11-09T03:48:17.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.500+0000 7f8a8471d640 1 -- 172.21.15.76:0/2826161031 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a7c1508b0 con 0x7f8a7c0cc9d0 2023-11-09T03:48:17.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.500+0000 7f8a72ffd640 1 -- 172.21.15.76:0/2826161031 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7803eae0 con 0x7f8a7c0cc9d0 2023-11-09T03:48:17.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.500+0000 7f8a8471d640 1 -- 172.21.15.76:0/2826161031 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a7c14d8f0 con 0x7f8a7c0cc9d0 2023-11-09T03:48:17.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.501+0000 7f8a8471d640 1 -- 172.21.15.76:0/2826161031 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a7c0cef90 con 0x7f8a7c0cc9d0 2023-11-09T03:48:17.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.502+0000 7f8a72ffd640 1 -- 172.21.15.76:0/2826161031 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8a7803c040 con 0x7f8a7c0cc9d0 2023-11-09T03:48:17.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.502+0000 7f8a72ffd640 1 --2- 172.21.15.76:0/2826161031 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8a6006aec0 0x7f8a6006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:17.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.502+0000 7f8a72ffd640 1 -- 172.21.15.76:0/2826161031 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f8a780ab740 con 0x7f8a7c0cc9d0 2023-11-09T03:48:17.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.502+0000 7f8a824b9640 1 --2- 172.21.15.76:0/2826161031 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8a6006aec0 0x7f8a6006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:17.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.503+0000 7f8a824b9640 1 --2- 172.21.15.76:0/2826161031 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8a6006aec0 0x7f8a6006d380 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c0079a0 tx=0x7f8a6c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:17.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.511+0000 7f8a72ffd640 1 -- 172.21.15.76:0/2826161031 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8a780767a0 con 0x7f8a7c0cc9d0 2023-11-09T03:48:17.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:17.690+0000 7f8a8471d640 1 -- 172.21.15.76:0/2826161031 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8a7c074290 con 0x7f8a7c0cc9d0 2023-11-09T03:48:18.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.522+0000 7f8a72ffd640 1 -- 172.21.15.76:0/2826161031 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v79) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8a78079dc0 con 0x7f8a7c0cc9d0 2023-11-09T03:48:18.524 INFO:teuthology.orchestra.run.smithi076.stderr: removed all blocklist entries 2023-11-09T03:48:18.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.525+0000 7f8a8471d640 1 -- 172.21.15.76:0/2826161031 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8a6006aec0 msgr2=0x7f8a6006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:18.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.525+0000 7f8a8471d640 1 --2- 172.21.15.76:0/2826161031 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8a6006aec0 0x7f8a6006d380 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c0079a0 tx=0x7f8a6c008040 comp rx=0 tx=0).stop 2023-11-09T03:48:18.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.525+0000 7f8a8471d640 1 -- 172.21.15.76:0/2826161031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a7c0cc9d0 msgr2=0x7f8a7c0b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:18.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.525+0000 7f8a8471d640 1 --2- 172.21.15.76:0/2826161031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a7c0cc9d0 0x7f8a7c0b3740 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f8a78038040 tx=0x7f8a780046c0 comp rx=0 tx=0).stop 2023-11-09T03:48:18.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.525+0000 7f8a8471d640 1 -- 172.21.15.76:0/2826161031 shutdown_connections 2023-11-09T03:48:18.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.525+0000 7f8a8471d640 1 --2- 172.21.15.76:0/2826161031 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8a6006aec0 0x7f8a6006d380 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:18.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.525+0000 7f8a8471d640 1 --2- 172.21.15.76:0/2826161031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8a7c153850 0x7f8a7c0b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:18.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.525+0000 7f8a8471d640 1 --2- 172.21.15.76:0/2826161031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a7c0cc9d0 0x7f8a7c0b3740 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:18.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.525+0000 7f8a8471d640 1 --2- 172.21.15.76:0/2826161031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8a7c0cc520 0x7f8a7c14d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:18.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.525+0000 7f8a8471d640 1 -- 172.21.15.76:0/2826161031 >> 172.21.15.76:0/2826161031 conn(0x7f8a7c0ce950 msgr2=0x7f8a7c0c3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:18.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.525+0000 7f8a8471d640 1 -- 172.21.15.76:0/2826161031 shutdown_connections 2023-11-09T03:48:18.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.525+0000 7f8a8471d640 1 -- 172.21.15.76:0/2826161031 wait complete. 2023-11-09T03:48:18.539 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-09T03:48:18.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.644+0000 7f3347539640 1 Processor -- start 2023-11-09T03:48:18.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.644+0000 7f3347539640 1 -- start start 2023-11-09T03:48:18.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.645+0000 7f3347539640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f334006a830 0x7f33400b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:18.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.645+0000 7f3347539640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f334015dd10 con 0x7f33400cce10 2023-11-09T03:48:18.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.645+0000 7f3347539640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f334015d720 con 0x7f33400b18b0 2023-11-09T03:48:18.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.645+0000 7f3347539640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f334006fe00 con 0x7f334006a830 2023-11-09T03:48:18.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.645+0000 7f33452d5640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f33400cce10 0x7f334006a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40350/0 (socket says 172.21.15.76:40350) 2023-11-09T03:48:18.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.645+0000 7f33452d5640 1 -- 172.21.15.76:0/1641653877 learned_addr learned my addr 172.21.15.76:0/1641653877 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:18.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.645+0000 7f3344ad4640 1 --2- 172.21.15.76:0/1641653877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f334006a830 0x7f33400b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:18.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.646+0000 7f3337fff640 1 -- 172.21.15.76:0/1641653877 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1560310073 0 0) 0x7f334015dd10 con 0x7f33400cce10 2023-11-09T03:48:18.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.646+0000 7f3337fff640 1 -- 172.21.15.76:0/1641653877 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f332c003520 con 0x7f33400cce10 2023-11-09T03:48:18.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.646+0000 7f3337fff640 1 -- 172.21.15.76:0/1641653877 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 436863974 0 0) 0x7f332c003520 con 0x7f33400cce10 2023-11-09T03:48:18.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.646+0000 7f3337fff640 1 -- 172.21.15.76:0/1641653877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f334006a830 msgr2=0x7f33400b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:18.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.646+0000 7f3337fff640 1 --2- 172.21.15.76:0/1641653877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f334006a830 0x7f33400b1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:18.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.646+0000 7f3337fff640 1 -- 172.21.15.76:0/1641653877 >> v1:172.21.15.113:6789/0 conn(0x7f33400b18b0 legacy=0x7f334015ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:18.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.646+0000 7f3337fff640 1 -- 172.21.15.76:0/1641653877 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33400700e0 con 0x7f33400cce10 2023-11-09T03:48:18.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.646+0000 7f3337fff640 1 -- 172.21.15.76:0/1641653877 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f333c0019f0 con 0x7f33400cce10 2023-11-09T03:48:18.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.646+0000 7f3337fff640 1 -- 172.21.15.76:0/1641653877 >> v1:172.21.15.76:6789/0 conn(0x7f33400cce10 legacy=0x7f334006a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:18.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.646+0000 7f3337fff640 1 --2- 172.21.15.76:0/1641653877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f332c0039a0 0x7f332c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:18.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.646+0000 7f3337fff640 1 --2- 172.21.15.76:0/1641653877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f332c0047c0 0x7f332c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:18.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.646+0000 7f3337fff640 1 --2- 172.21.15.76:0/1641653877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f334006a830 0x7f332c005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:18.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.646+0000 7f3337fff640 1 -- 172.21.15.76:0/1641653877 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334006fe00 con 0x7f332c0047c0 2023-11-09T03:48:18.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.646+0000 7f3337fff640 1 -- 172.21.15.76:0/1641653877 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332c00bb40 con 0x7f332c0039a0 2023-11-09T03:48:18.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.646+0000 7f3337fff640 1 -- 172.21.15.76:0/1641653877 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332c00bd00 con 0x7f334006a830 2023-11-09T03:48:18.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.646+0000 7f3344ad4640 1 --2- 172.21.15.76:0/1641653877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f332c0047c0 0x7f332c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:18.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.646+0000 7f3345ad6640 1 --2- 172.21.15.76:0/1641653877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f334006a830 0x7f332c005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:18.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.646+0000 7f33452d5640 1 --2- 172.21.15.76:0/1641653877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f332c0039a0 0x7f332c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:18.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.646+0000 7f3344ad4640 1 -- 172.21.15.76:0/1641653877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f334006a830 msgr2=0x7f332c005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:18.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.646+0000 7f3344ad4640 1 --2- 172.21.15.76:0/1641653877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f334006a830 0x7f332c005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:18.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.646+0000 7f3344ad4640 1 -- 172.21.15.76:0/1641653877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f332c0039a0 msgr2=0x7f332c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:18.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.646+0000 7f3344ad4640 1 --2- 172.21.15.76:0/1641653877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f332c0039a0 0x7f332c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:18.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.646+0000 7f3344ad4640 1 -- 172.21.15.76:0/1641653877 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f332c003520 con 0x7f332c0047c0 2023-11-09T03:48:18.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.647+0000 7f3344ad4640 1 --2- 172.21.15.76:0/1641653877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f332c0047c0 0x7f332c004c40 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f3330009ac0 tx=0x7f332c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b946ae9239c63916 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:18.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.647+0000 7f3337fff640 1 -- 172.21.15.76:0/1641653877 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333002c6b0 con 0x7f332c0047c0 2023-11-09T03:48:18.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.647+0000 7f3337fff640 1 -- 172.21.15.76:0/1641653877 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333002c810 con 0x7f332c0047c0 2023-11-09T03:48:18.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.647+0000 7f3337fff640 1 -- 172.21.15.76:0/1641653877 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333002cba0 con 0x7f332c0047c0 2023-11-09T03:48:18.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.648+0000 7f3347539640 1 -- 172.21.15.76:0/1641653877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f332c0047c0 msgr2=0x7f332c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:18.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.648+0000 7f3347539640 1 --2- 172.21.15.76:0/1641653877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f332c0047c0 0x7f332c004c40 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f3330009ac0 tx=0x7f332c0057f0 comp rx=0 tx=0).stop 2023-11-09T03:48:18.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.648+0000 7f3345ad6640 1 -- 172.21.15.76:0/1641653877 reap_dead start 2023-11-09T03:48:18.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.648+0000 7f3347539640 1 -- 172.21.15.76:0/1641653877 shutdown_connections 2023-11-09T03:48:18.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.648+0000 7f3347539640 1 -- 172.21.15.76:0/1641653877 >> 172.21.15.76:0/1641653877 conn(0x7f334006d8b0 msgr2=0x7f334006dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:18.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.648+0000 7f3347539640 1 -- 172.21.15.76:0/1641653877 shutdown_connections 2023-11-09T03:48:18.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.648+0000 7f3347539640 1 -- 172.21.15.76:0/1641653877 wait complete. 2023-11-09T03:48:18.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.649+0000 7f3347539640 1 Processor -- start 2023-11-09T03:48:18.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.649+0000 7f3347539640 1 -- start start 2023-11-09T03:48:18.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.650+0000 7f3347539640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f332c0047c0 0x7f33400c34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:18.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.650+0000 7f3347539640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33400c9520 0x7f33400add20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:18.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.650+0000 7f3347539640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3340151820 0x7f3340151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:18.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.650+0000 7f3347539640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33400c3e10 con 0x7f332c0047c0 2023-11-09T03:48:18.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.650+0000 7f3347539640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33400c40f0 con 0x7f33400c9520 2023-11-09T03:48:18.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.650+0000 7f3347539640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33400c43d0 con 0x7f3340151820 2023-11-09T03:48:18.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.650+0000 7f33452d5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f332c0047c0 0x7f33400c34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:18.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.650+0000 7f33452d5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f332c0047c0 0x7f33400c34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51430/0 (socket says 172.21.15.76:51430) 2023-11-09T03:48:18.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.650+0000 7f33452d5640 1 -- 172.21.15.76:0/278482495 learned_addr learned my addr 172.21.15.76:0/278482495 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:18.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.650+0000 7f3344ad4640 1 --2- 172.21.15.76:0/278482495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33400c9520 0x7f33400add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:18.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.650+0000 7f3345ad6640 1 --2- 172.21.15.76:0/278482495 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3340151820 0x7f3340151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:18.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.650+0000 7f33452d5640 1 -- 172.21.15.76:0/278482495 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3340151820 msgr2=0x7f3340151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:18.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.650+0000 7f33452d5640 1 --2- 172.21.15.76:0/278482495 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3340151820 0x7f3340151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:18.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.650+0000 7f33452d5640 1 -- 172.21.15.76:0/278482495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33400c9520 msgr2=0x7f33400add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:18.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.650+0000 7f33452d5640 1 --2- 172.21.15.76:0/278482495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33400c9520 0x7f33400add20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:18.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.650+0000 7f33452d5640 1 -- 172.21.15.76:0/278482495 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33400c46b0 con 0x7f332c0047c0 2023-11-09T03:48:18.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.650+0000 7f3345ad6640 1 --2- 172.21.15.76:0/278482495 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3340151820 0x7f3340151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:18.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.650+0000 7f33452d5640 1 --2- 172.21.15.76:0/278482495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f332c0047c0 0x7f33400c34c0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f333c007970 tx=0x7f333c00a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:18.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.650+0000 7f3335ffb640 1 -- 172.21.15.76:0/278482495 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c00ad90 con 0x7f332c0047c0 2023-11-09T03:48:18.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.650+0000 7f3335ffb640 1 -- 172.21.15.76:0/278482495 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333c0133f0 con 0x7f332c0047c0 2023-11-09T03:48:18.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.650+0000 7f3347539640 1 -- 172.21.15.76:0/278482495 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33400c4990 con 0x7f332c0047c0 2023-11-09T03:48:18.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.650+0000 7f3335ffb640 1 -- 172.21.15.76:0/278482495 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c0137a0 con 0x7f332c0047c0 2023-11-09T03:48:18.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.651+0000 7f3347539640 1 -- 172.21.15.76:0/278482495 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33400c3a00 con 0x7f332c0047c0 2023-11-09T03:48:18.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.651+0000 7f3347539640 1 -- 172.21.15.76:0/278482495 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33400c4b80 con 0x7f332c0047c0 2023-11-09T03:48:18.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.652+0000 7f3335ffb640 1 -- 172.21.15.76:0/278482495 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f333c013900 con 0x7f332c0047c0 2023-11-09T03:48:18.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.652+0000 7f3335ffb640 1 --2- 172.21.15.76:0/278482495 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f332406ae70 0x7f332406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:18.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.652+0000 7f3335ffb640 1 -- 172.21.15.76:0/278482495 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6240+0+0 (secure 0 0 0) 0x7f333c08a530 con 0x7f332c0047c0 2023-11-09T03:48:18.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.653+0000 7f3344ad4640 1 --2- 172.21.15.76:0/278482495 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f332406ae70 0x7f332406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:18.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.653+0000 7f3344ad4640 1 --2- 172.21.15.76:0/278482495 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f332406ae70 0x7f332406d330 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3330036000 tx=0x7f3330030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:18.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.657+0000 7f3335ffb640 1 -- 172.21.15.76:0/278482495 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f333c0553d0 con 0x7f332c0047c0 2023-11-09T03:48:18.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.883+0000 7f3347539640 1 -- 172.21.15.76:0/278482495 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f33400b59a0 con 0x7f332c0047c0 2023-11-09T03:48:18.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.883+0000 7f3335ffb640 1 -- 172.21.15.76:0/278482495 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f333c0589f0 con 0x7f332c0047c0 2023-11-09T03:48:18.884 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:18.885 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:48:18.885 INFO:teuthology.orchestra.run.smithi076.stdout: "auth_dump": [ 2023-11-09T03:48:18.885 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:18.885 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.a", 2023-11-09T03:48:18.885 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCgVUxlydtlLxAAi0bnTEOD+fpGL1cYic3caA==", 2023-11-09T03:48:18.886 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:18.886 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:48:18.886 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2023-11-09T03:48:18.886 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2023-11-09T03:48:18.886 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:18.887 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:18.887 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:18.887 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:18.887 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.b", 2023-11-09T03:48:18.887 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxl2+QnABAA7CTWWEbPPXkollRFDlLxPA==", 2023-11-09T03:48:18.887 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:18.888 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:48:18.888 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2023-11-09T03:48:18.888 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2023-11-09T03:48:18.888 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:18.888 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:18.889 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:18.889 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:18.889 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.c", 2023-11-09T03:48:18.889 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCgVUxlrTu9NhAAhg4GpZWZTmVPCP3ffjPo/Q==", 2023-11-09T03:48:18.889 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:18.890 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:48:18.890 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2023-11-09T03:48:18.890 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2023-11-09T03:48:18.890 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:18.890 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:18.890 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:18.891 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:18.891 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.d", 2023-11-09T03:48:18.891 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxl4IZ8BxAA8zGFE+1tlHkT5UyzL0nWxQ==", 2023-11-09T03:48:18.891 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:18.891 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:48:18.892 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2023-11-09T03:48:18.892 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2023-11-09T03:48:18.892 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:18.892 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:18.892 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:18.893 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:18.893 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.0", 2023-11-09T03:48:18.893 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCkVUxlXM5LKhAAM253+XEj5YSwmQywwDieTg==", 2023-11-09T03:48:18.893 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:18.893 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:48:18.894 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:48:18.894 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:18.894 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:18.894 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:18.894 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:18.895 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.1", 2023-11-09T03:48:18.895 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCnVUxlgoDVKBAAkmNFo4E0bAQ1UxkCfQAbjw==", 2023-11-09T03:48:18.895 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:18.895 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:48:18.895 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:48:18.896 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:18.896 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:18.896 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:18.896 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:18.896 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.2", 2023-11-09T03:48:18.897 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCqVUxlzQjVJhAAOoU5IdN86BmcPz71mTq7ng==", 2023-11-09T03:48:18.897 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:18.897 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:48:18.897 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:48:18.897 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:18.898 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:18.898 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:18.898 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:18.898 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.3", 2023-11-09T03:48:18.898 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCtVUxlhY6VJBAAmCO/ZS8vBa7enyuXX5Y+hw==", 2023-11-09T03:48:18.899 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:18.899 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:48:18.899 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:48:18.899 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:18.899 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:18.900 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:18.900 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:18.900 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.4", 2023-11-09T03:48:18.900 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCzVUxl79egLxAAnKBOQb/9EQyaHaQt7SawIg==", 2023-11-09T03:48:18.900 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:18.900 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:48:18.901 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:48:18.901 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:18.901 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:18.901 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:18.901 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:18.901 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.5", 2023-11-09T03:48:18.902 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQC2VUxlIfUgLhAAVPurhJCc8cmpt11pQjzL9w==", 2023-11-09T03:48:18.902 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:18.902 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:48:18.902 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:48:18.902 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:18.903 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:18.903 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:18.903 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:18.903 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.6", 2023-11-09T03:48:18.903 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQC5VUxlrI58LBAAWxZotpACR4e8vFj4IiaEKA==", 2023-11-09T03:48:18.904 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:18.904 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:48:18.904 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:48:18.904 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:18.904 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:18.904 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:18.905 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:18.905 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.7", 2023-11-09T03:48:18.905 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQC8VUxl0bA3KxAAwnKNJKqYIcGRLLhQHUaU9w==", 2023-11-09T03:48:18.905 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:18.905 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:48:18.906 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:48:18.906 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:18.906 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:18.906 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:18.906 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:18.907 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.0", 2023-11-09T03:48:18.907 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxlHj22CxAACNj5X24hdME+GNTLw4RahQ==", 2023-11-09T03:48:18.907 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:18.907 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:48:18.907 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow r", 2023-11-09T03:48:18.907 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-09T03:48:18.908 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:18.908 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:18.908 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:18.908 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.1", 2023-11-09T03:48:18.908 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxljXv4FBAAiTOhfIReQWTy6fbujbBrrw==", 2023-11-09T03:48:18.909 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:18.909 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:48:18.909 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2023-11-09T03:48:18.909 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2023-11-09T03:48:18.909 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2023-11-09T03:48:18.910 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:18.910 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:18.910 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:18.910 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.2", 2023-11-09T03:48:18.910 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxlGVSSERAAQts5odubZ5paZwg8US3QrQ==", 2023-11-09T03:48:18.910 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:18.911 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:48:18.911 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2023-11-09T03:48:18.911 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2023-11-09T03:48:18.911 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2023-11-09T03:48:18.911 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:18.912 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:18.912 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:18.912 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.3", 2023-11-09T03:48:18.912 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxlgmDHGhAA7r98HNQEMnjbqW6RjE5aig==", 2023-11-09T03:48:18.912 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:18.913 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:48:18.913 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2023-11-09T03:48:18.913 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2023-11-09T03:48:18.913 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2023-11-09T03:48:18.913 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:18.913 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:18.914 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:18.914 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.admin", 2023-11-09T03:48:18.914 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCgVUxlV4l8EhAAnrgeLaTaR99vNYGCJjI1TA==", 2023-11-09T03:48:18.914 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:18.914 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2023-11-09T03:48:18.915 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2023-11-09T03:48:18.915 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2023-11-09T03:48:18.915 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:18.915 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:18.915 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:18.916 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:18.916 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mgr.x", 2023-11-09T03:48:18.916 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCgVUxl7JryKxAABFx9CRqSbbJu8SIyi3Va8w==", 2023-11-09T03:48:18.916 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:18.916 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2023-11-09T03:48:18.916 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile mgr", 2023-11-09T03:48:18.917 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:18.917 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:18.917 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:18.917 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:18.917 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mgr.y", 2023-11-09T03:48:18.918 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCgVUxlb9L/JxAAu3IOXTn8TGvbzxYVta3yew==", 2023-11-09T03:48:18.918 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:18.918 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2023-11-09T03:48:18.918 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile mgr", 2023-11-09T03:48:18.918 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:18.919 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:18.919 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:18.919 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2023-11-09T03:48:18.919 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:48:18.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.885+0000 7f3347539640 1 -- 172.21.15.76:0/278482495 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f332406ae70 msgr2=0x7f332406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:18.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.885+0000 7f3347539640 1 --2- 172.21.15.76:0/278482495 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f332406ae70 0x7f332406d330 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3330036000 tx=0x7f3330030000 comp rx=0 tx=0).stop 2023-11-09T03:48:18.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.885+0000 7f3347539640 1 -- 172.21.15.76:0/278482495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f332c0047c0 msgr2=0x7f33400c34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:18.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.885+0000 7f3347539640 1 --2- 172.21.15.76:0/278482495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f332c0047c0 0x7f33400c34c0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f333c007970 tx=0x7f333c00a6e0 comp rx=0 tx=0).stop 2023-11-09T03:48:18.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.885+0000 7f3347539640 1 -- 172.21.15.76:0/278482495 shutdown_connections 2023-11-09T03:48:18.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.885+0000 7f3347539640 1 --2- 172.21.15.76:0/278482495 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f332406ae70 0x7f332406d330 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:18.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.885+0000 7f3347539640 1 --2- 172.21.15.76:0/278482495 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3340151820 0x7f3340151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:18.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.885+0000 7f3347539640 1 --2- 172.21.15.76:0/278482495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33400c9520 0x7f33400add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:18.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.885+0000 7f3347539640 1 --2- 172.21.15.76:0/278482495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f332c0047c0 0x7f33400c34c0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:18.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.885+0000 7f3347539640 1 -- 172.21.15.76:0/278482495 >> 172.21.15.76:0/278482495 conn(0x7f334006d8b0 msgr2=0x7f33400aeb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:18.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.886+0000 7f3347539640 1 -- 172.21.15.76:0/278482495 shutdown_connections 2023-11-09T03:48:18.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:18.886+0000 7f3347539640 1 -- 172.21.15.76:0/278482495 wait complete. 2023-11-09T03:48:18.923 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:48:19.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.022+0000 7f63691e7640 1 Processor -- start 2023-11-09T03:48:19.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.022+0000 7f63691e7640 1 -- start start 2023-11-09T03:48:19.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.023+0000 7f63691e7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63640cde60 0x7f636415d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:19.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.023+0000 7f63691e7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f636415daa0 0x7f6364157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:19.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.023+0000 7f63691e7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f636415e000 con 0x7f63640ce240 2023-11-09T03:48:19.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.023+0000 7f63691e7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f636415e5c0 con 0x7f636415daa0 2023-11-09T03:48:19.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.023+0000 7f63691e7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f636415e2e0 con 0x7f63640cde60 2023-11-09T03:48:19.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.023+0000 7f636259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f636415daa0 0x7f6364157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:19.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.023+0000 7f636359e640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f63640ce240 0x7f636406b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40356/0 (socket says 172.21.15.76:40356) 2023-11-09T03:48:19.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.023+0000 7f636259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f636415daa0 0x7f6364157000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51438/0 (socket says 172.21.15.76:51438) 2023-11-09T03:48:19.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.023+0000 7f636359e640 1 -- 172.21.15.76:0/1898298519 learned_addr learned my addr 172.21.15.76:0/1898298519 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:19.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.024+0000 7f6362d9d640 1 --2- 172.21.15.76:0/1898298519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63640cde60 0x7f636415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:19.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.024+0000 7f636259c640 1 -- 172.21.15.76:0/1898298519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63640cde60 msgr2=0x7f636415d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:19.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.024+0000 7f636259c640 1 --2- 172.21.15.76:0/1898298519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63640cde60 0x7f636415d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.024+0000 7f636259c640 1 -- 172.21.15.76:0/1898298519 >> v1:172.21.15.76:6789/0 conn(0x7f63640ce240 legacy=0x7f636406b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:19.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.024+0000 7f636259c640 1 -- 172.21.15.76:0/1898298519 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63640d1120 con 0x7f636415daa0 2023-11-09T03:48:19.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.024+0000 7f6361d9b640 1 -- 172.21.15.76:0/1898298519 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4206315689 0 0) 0x7f636415e000 con 0x7f63640ce240 2023-11-09T03:48:19.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.024+0000 7f636259c640 1 --2- 172.21.15.76:0/1898298519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f636415daa0 0x7f6364157000 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f63580097a0 tx=0x7f635802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312af3d931333d75 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:19.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.025+0000 7f6361d9b640 1 -- 172.21.15.76:0/1898298519 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6358031830 con 0x7f636415daa0 2023-11-09T03:48:19.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.025+0000 7f6361d9b640 1 -- 172.21.15.76:0/1898298519 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6358031990 con 0x7f636415daa0 2023-11-09T03:48:19.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.025+0000 7f6361d9b640 1 -- 172.21.15.76:0/1898298519 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6358031d20 con 0x7f636415daa0 2023-11-09T03:48:19.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.025+0000 7f63691e7640 1 -- 172.21.15.76:0/1898298519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f636415daa0 msgr2=0x7f6364157000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:19.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.025+0000 7f63691e7640 1 --2- 172.21.15.76:0/1898298519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f636415daa0 0x7f6364157000 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f63580097a0 tx=0x7f635802b8d0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.025+0000 7f63691e7640 1 -- 172.21.15.76:0/1898298519 shutdown_connections 2023-11-09T03:48:19.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.025+0000 7f63691e7640 1 --2- 172.21.15.76:0/1898298519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f636415daa0 0x7f6364157000 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.025+0000 7f63691e7640 1 --2- 172.21.15.76:0/1898298519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63640cde60 0x7f636415d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.025+0000 7f63691e7640 1 -- 172.21.15.76:0/1898298519 >> 172.21.15.76:0/1898298519 conn(0x7f6364071f30 msgr2=0x7f63640cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:19.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.025+0000 7f63691e7640 1 -- 172.21.15.76:0/1898298519 shutdown_connections 2023-11-09T03:48:19.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.025+0000 7f63691e7640 1 -- 172.21.15.76:0/1898298519 wait complete. 2023-11-09T03:48:19.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.027+0000 7f63691e7640 1 Processor -- start 2023-11-09T03:48:19.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.027+0000 7f63691e7640 1 -- start start 2023-11-09T03:48:19.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.027+0000 7f63691e7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63640cde60 0x7f63640c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:19.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.027+0000 7f63691e7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63640ce240 0x7f6364153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:19.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.027+0000 7f63691e7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f636415daa0 0x7f6364151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:19.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.027+0000 7f63691e7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6364152d50 con 0x7f636415daa0 2023-11-09T03:48:19.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.027+0000 7f63691e7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63641523d0 con 0x7f63640cde60 2023-11-09T03:48:19.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.027+0000 7f63691e7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63641f6b80 con 0x7f63640ce240 2023-11-09T03:48:19.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.027+0000 7f636359e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f636415daa0 0x7f6364151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:19.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.027+0000 7f636359e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f636415daa0 0x7f6364151020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51450/0 (socket says 172.21.15.76:51450) 2023-11-09T03:48:19.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.027+0000 7f636359e640 1 -- 172.21.15.76:0/3730758006 learned_addr learned my addr 172.21.15.76:0/3730758006 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:19.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.028+0000 7f6362d9d640 1 --2- 172.21.15.76:0/3730758006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63640cde60 0x7f63640c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:19.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.028+0000 7f636259c640 1 --2- 172.21.15.76:0/3730758006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63640ce240 0x7f6364153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:19.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.028+0000 7f636359e640 1 -- 172.21.15.76:0/3730758006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63640ce240 msgr2=0x7f6364153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:19.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.028+0000 7f636359e640 1 --2- 172.21.15.76:0/3730758006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63640ce240 0x7f6364153170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.028+0000 7f636359e640 1 -- 172.21.15.76:0/3730758006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63640cde60 msgr2=0x7f63640c5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:19.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.028+0000 7f636359e640 1 --2- 172.21.15.76:0/3730758006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63640cde60 0x7f63640c5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.028+0000 7f636359e640 1 -- 172.21.15.76:0/3730758006 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6358009400 con 0x7f636415daa0 2023-11-09T03:48:19.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.028+0000 7f636359e640 1 --2- 172.21.15.76:0/3730758006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f636415daa0 0x7f6364151020 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f635c00c450 tx=0x7f635c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:19.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.028+0000 7f634b7fe640 1 -- 172.21.15.76:0/3730758006 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c0074e0 con 0x7f636415daa0 2023-11-09T03:48:19.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.028+0000 7f634b7fe640 1 -- 172.21.15.76:0/3730758006 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635c007660 con 0x7f636415daa0 2023-11-09T03:48:19.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.028+0000 7f63691e7640 1 -- 172.21.15.76:0/3730758006 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6364152690 con 0x7f636415daa0 2023-11-09T03:48:19.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.028+0000 7f634b7fe640 1 -- 172.21.15.76:0/3730758006 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c004a20 con 0x7f636415daa0 2023-11-09T03:48:19.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.028+0000 7f63691e7640 1 -- 172.21.15.76:0/3730758006 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63640c5f40 con 0x7f636415daa0 2023-11-09T03:48:19.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.029+0000 7f63691e7640 1 -- 172.21.15.76:0/3730758006 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6364152e90 con 0x7f636415daa0 2023-11-09T03:48:19.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.030+0000 7f634b7fe640 1 -- 172.21.15.76:0/3730758006 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f635c004b80 con 0x7f636415daa0 2023-11-09T03:48:19.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.030+0000 7f634b7fe640 1 --2- 172.21.15.76:0/3730758006 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f635006ae70 0x7f635006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:19.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.030+0000 7f634b7fe640 1 -- 172.21.15.76:0/3730758006 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6240+0+0 (secure 0 0 0) 0x7f635c088c00 con 0x7f636415daa0 2023-11-09T03:48:19.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.030+0000 7f6362d9d640 1 --2- 172.21.15.76:0/3730758006 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f635006ae70 0x7f635006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:19.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.031+0000 7f6362d9d640 1 --2- 172.21.15.76:0/3730758006 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f635006ae70 0x7f635006d330 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6354003b80 tx=0x7f63540098d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:19.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.034+0000 7f634b7fe640 1 -- 172.21.15.76:0/3730758006 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6364152e90 con 0x7f636415daa0 2023-11-09T03:48:19.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.202+0000 7f63691e7640 1 -- 172.21.15.76:0/3730758006 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6364152e90 con 0x7f636415daa0 2023-11-09T03:48:19.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.203+0000 7f634b7fe640 1 -- 172.21.15.76:0/3730758006 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18291 (secure 0 0 0) 0x7f6364152e90 con 0x7f636415daa0 2023-11-09T03:48:19.204 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:19.204 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":79,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:48:18.522650+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-09T03:47:59.280601+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-09T03:47:59.742039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:48:19.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.205+0000 7f63691e7640 1 -- 172.21.15.76:0/3730758006 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f635006ae70 msgr2=0x7f635006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:19.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.205+0000 7f63691e7640 1 --2- 172.21.15.76:0/3730758006 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f635006ae70 0x7f635006d330 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6354003b80 tx=0x7f63540098d0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.205+0000 7f63691e7640 1 -- 172.21.15.76:0/3730758006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f636415daa0 msgr2=0x7f6364151020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:19.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.205+0000 7f63691e7640 1 --2- 172.21.15.76:0/3730758006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f636415daa0 0x7f6364151020 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f635c00c450 tx=0x7f635c00c910 comp rx=0 tx=0).stop 2023-11-09T03:48:19.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.205+0000 7f63691e7640 1 -- 172.21.15.76:0/3730758006 shutdown_connections 2023-11-09T03:48:19.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.205+0000 7f63691e7640 1 --2- 172.21.15.76:0/3730758006 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f635006ae70 0x7f635006d330 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.205+0000 7f63691e7640 1 --2- 172.21.15.76:0/3730758006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f636415daa0 0x7f6364151020 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.205+0000 7f63691e7640 1 --2- 172.21.15.76:0/3730758006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63640ce240 0x7f6364153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.205+0000 7f63691e7640 1 --2- 172.21.15.76:0/3730758006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63640cde60 0x7f63640c5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.205+0000 7f63691e7640 1 -- 172.21.15.76:0/3730758006 >> 172.21.15.76:0/3730758006 conn(0x7f6364071f30 msgr2=0x7f63640b6050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:19.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.205+0000 7f63691e7640 1 -- 172.21.15.76:0/3730758006 shutdown_connections 2023-11-09T03:48:19.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.205+0000 7f63691e7640 1 -- 172.21.15.76:0/3730758006 wait complete. 2023-11-09T03:48:19.216 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-09T03:47:00.036540+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_data_pool', 'create_time': '2023-11-09T03:47:59.280601+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 70, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-09T03:47:59.742039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-09T03:48:19.217 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:48:19.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.320+0000 7fb299bfb640 1 Processor -- start 2023-11-09T03:48:19.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.320+0000 7fb299bfb640 1 -- start start 2023-11-09T03:48:19.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.321+0000 7fb299bfb640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb2940c4ff0 0x7fb2940cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:19.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.321+0000 7fb299bfb640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb29415aff0 con 0x7fb294159c00 2023-11-09T03:48:19.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.321+0000 7fb299bfb640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb29415a3e0 con 0x7fb2940adac0 2023-11-09T03:48:19.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.321+0000 7fb299bfb640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb29415a6c0 con 0x7fb2940c4ff0 2023-11-09T03:48:19.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.321+0000 7fb2937fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb294159c00 0x7fb2940c48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40372/0 (socket says 172.21.15.76:40372) 2023-11-09T03:48:19.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.321+0000 7fb2937fe640 1 -- 172.21.15.76:0/1682035552 learned_addr learned my addr 172.21.15.76:0/1682035552 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:19.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.322+0000 7fb2927fc640 1 -- 172.21.15.76:0/1682035552 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2802841648 0 0) 0x7fb29415aff0 con 0x7fb294159c00 2023-11-09T03:48:19.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.322+0000 7fb292ffd640 1 --2- 172.21.15.76:0/1682035552 >> v2:172.21.15.113:3300/0 conn(0x7fb2940c4ff0 0x7fb2940cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:19.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.322+0000 7fb2927fc640 1 -- 172.21.15.76:0/1682035552 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb27c003520 con 0x7fb294159c00 2023-11-09T03:48:19.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.322+0000 7fb2927fc640 1 -- 172.21.15.76:0/1682035552 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 855557866 0 0) 0x7fb27c003520 con 0x7fb294159c00 2023-11-09T03:48:19.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.322+0000 7fb2927fc640 1 -- 172.21.15.76:0/1682035552 >> v2:172.21.15.113:3300/0 conn(0x7fb2940c4ff0 msgr2=0x7fb2940cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:19.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.322+0000 7fb2927fc640 1 --2- 172.21.15.76:0/1682035552 >> v2:172.21.15.113:3300/0 conn(0x7fb2940c4ff0 0x7fb2940cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.322+0000 7fb2927fc640 1 -- 172.21.15.76:0/1682035552 >> v1:172.21.15.113:6789/0 conn(0x7fb2940adac0 legacy=0x7fb2940adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:19.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.322+0000 7fb2927fc640 1 -- 172.21.15.76:0/1682035552 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29415a9a0 con 0x7fb294159c00 2023-11-09T03:48:19.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.322+0000 7fb2927fc640 1 -- 172.21.15.76:0/1682035552 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fb284001670 con 0x7fb294159c00 2023-11-09T03:48:19.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.322+0000 7fb2927fc640 1 -- 172.21.15.76:0/1682035552 >> v1:172.21.15.76:6789/0 conn(0x7fb294159c00 legacy=0x7fb2940c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:19.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.322+0000 7fb2927fc640 1 --2- 172.21.15.76:0/1682035552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb27c003970 0x7fb27c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:19.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.322+0000 7fb2927fc640 1 --2- 172.21.15.76:0/1682035552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb27c0047c0 0x7fb27c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:19.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.322+0000 7fb2927fc640 1 --2- 172.21.15.76:0/1682035552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb27c005180 0x7fb27c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:19.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.322+0000 7fb2927fc640 1 -- 172.21.15.76:0/1682035552 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29415a6c0 con 0x7fb27c003970 2023-11-09T03:48:19.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.322+0000 7fb2927fc640 1 -- 172.21.15.76:0/1682035552 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c00c020 con 0x7fb27c0047c0 2023-11-09T03:48:19.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.322+0000 7fb2927fc640 1 -- 172.21.15.76:0/1682035552 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c00c1e0 con 0x7fb27c005180 2023-11-09T03:48:19.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.322+0000 7fb2937fe640 1 --2- 172.21.15.76:0/1682035552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb27c003970 0x7fb27c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:19.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.322+0000 7fb292ffd640 1 --2- 172.21.15.76:0/1682035552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb27c0047c0 0x7fb27c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:19.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.322+0000 7fb293fff640 1 --2- 172.21.15.76:0/1682035552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb27c005180 0x7fb27c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:19.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.322+0000 7fb2937fe640 1 -- 172.21.15.76:0/1682035552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb27c005180 msgr2=0x7fb27c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:19.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.322+0000 7fb2937fe640 1 --2- 172.21.15.76:0/1682035552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb27c005180 0x7fb27c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.322+0000 7fb2937fe640 1 -- 172.21.15.76:0/1682035552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb27c0047c0 msgr2=0x7fb27c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:19.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.322+0000 7fb2937fe640 1 --2- 172.21.15.76:0/1682035552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb27c0047c0 0x7fb27c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.322+0000 7fb2937fe640 1 -- 172.21.15.76:0/1682035552 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27c003520 con 0x7fb27c003970 2023-11-09T03:48:19.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.322+0000 7fb293fff640 1 -- 172.21.15.76:0/1682035552 reap_dead start 2023-11-09T03:48:19.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.323+0000 7fb2937fe640 1 --2- 172.21.15.76:0/1682035552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb27c003970 0x7fb27c003d90 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fb28400c240 tx=0x7fb27c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cfd77a755800ec7 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:19.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.323+0000 7fb2927fc640 1 -- 172.21.15.76:0/1682035552 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb284005ad0 con 0x7fb27c003970 2023-11-09T03:48:19.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.323+0000 7fb2927fc640 1 -- 172.21.15.76:0/1682035552 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb284005c30 con 0x7fb27c003970 2023-11-09T03:48:19.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.323+0000 7fb2927fc640 1 -- 172.21.15.76:0/1682035552 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28402caf0 con 0x7fb27c003970 2023-11-09T03:48:19.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.324+0000 7fb299bfb640 1 -- 172.21.15.76:0/1682035552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb27c003970 msgr2=0x7fb27c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:19.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.324+0000 7fb299bfb640 1 --2- 172.21.15.76:0/1682035552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb27c003970 0x7fb27c003d90 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fb28400c240 tx=0x7fb27c0057f0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.324+0000 7fb299bfb640 1 -- 172.21.15.76:0/1682035552 shutdown_connections 2023-11-09T03:48:19.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.324+0000 7fb299bfb640 1 --2- 172.21.15.76:0/1682035552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb27c003970 0x7fb27c003d90 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.324+0000 7fb299bfb640 1 -- 172.21.15.76:0/1682035552 >> 172.21.15.76:0/1682035552 conn(0x7fb2940ce950 msgr2=0x7fb2940ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:19.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.324+0000 7fb299bfb640 1 -- 172.21.15.76:0/1682035552 shutdown_connections 2023-11-09T03:48:19.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.324+0000 7fb299bfb640 1 -- 172.21.15.76:0/1682035552 wait complete. 2023-11-09T03:48:19.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.325+0000 7fb299bfb640 1 Processor -- start 2023-11-09T03:48:19.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.325+0000 7fb299bfb640 1 -- start start 2023-11-09T03:48:19.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.326+0000 7fb299bfb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb27c003970 0x7fb294146560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:19.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.326+0000 7fb299bfb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2941ee600 0x7fb294158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:19.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.326+0000 7fb299bfb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2941591b0 0x7fb2941568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:19.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.326+0000 7fb299bfb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2940c8010 con 0x7fb2941591b0 2023-11-09T03:48:19.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.326+0000 7fb299bfb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2940c7800 con 0x7fb27c003970 2023-11-09T03:48:19.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.326+0000 7fb299bfb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb294156460 con 0x7fb2941ee600 2023-11-09T03:48:19.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.326+0000 7fb293fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2941591b0 0x7fb2941568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:19.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.326+0000 7fb293fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2941591b0 0x7fb2941568f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51478/0 (socket says 172.21.15.76:51478) 2023-11-09T03:48:19.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.326+0000 7fb293fff640 1 -- 172.21.15.76:0/1951787954 learned_addr learned my addr 172.21.15.76:0/1951787954 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:19.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.326+0000 7fb2937fe640 1 --2- 172.21.15.76:0/1951787954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb27c003970 0x7fb294146560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:19.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.326+0000 7fb292ffd640 1 --2- 172.21.15.76:0/1951787954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2941ee600 0x7fb294158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:19.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.326+0000 7fb293fff640 1 -- 172.21.15.76:0/1951787954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2941ee600 msgr2=0x7fb294158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:19.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.326+0000 7fb293fff640 1 --2- 172.21.15.76:0/1951787954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2941ee600 0x7fb294158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.326+0000 7fb293fff640 1 -- 172.21.15.76:0/1951787954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb27c003970 msgr2=0x7fb294146560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:19.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.326+0000 7fb293fff640 1 --2- 172.21.15.76:0/1951787954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb27c003970 0x7fb294146560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.326+0000 7fb293fff640 1 -- 172.21.15.76:0/1951787954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2940c7a50 con 0x7fb2941591b0 2023-11-09T03:48:19.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.326+0000 7fb2937fe640 1 --2- 172.21.15.76:0/1951787954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb27c003970 0x7fb294146560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:19.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.326+0000 7fb293fff640 1 --2- 172.21.15.76:0/1951787954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2941591b0 0x7fb2941568f0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fb2940cb070 tx=0x7fb28c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:19.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.326+0000 7fb27bfff640 1 -- 172.21.15.76:0/1951787954 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c00eed0 con 0x7fb2941591b0 2023-11-09T03:48:19.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.326+0000 7fb27bfff640 1 -- 172.21.15.76:0/1951787954 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb28c003970 con 0x7fb2941591b0 2023-11-09T03:48:19.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.326+0000 7fb299bfb640 1 -- 172.21.15.76:0/1951787954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2940c7d30 con 0x7fb2941591b0 2023-11-09T03:48:19.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.326+0000 7fb27bfff640 1 -- 172.21.15.76:0/1951787954 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c003d00 con 0x7fb2941591b0 2023-11-09T03:48:19.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.327+0000 7fb299bfb640 1 -- 172.21.15.76:0/1951787954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb294150bb0 con 0x7fb2941591b0 2023-11-09T03:48:19.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.327+0000 7fb299bfb640 1 -- 172.21.15.76:0/1951787954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb294155f50 con 0x7fb2941591b0 2023-11-09T03:48:19.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.328+0000 7fb27bfff640 1 -- 172.21.15.76:0/1951787954 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb28c0050d0 con 0x7fb2941591b0 2023-11-09T03:48:19.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.328+0000 7fb27bfff640 1 --2- 172.21.15.76:0/1951787954 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb28006aea0 0x7fb28006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:19.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.328+0000 7fb27bfff640 1 -- 172.21.15.76:0/1951787954 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6240+0+0 (secure 0 0 0) 0x7fb28c089360 con 0x7fb2941591b0 2023-11-09T03:48:19.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.328+0000 7fb2937fe640 1 --2- 172.21.15.76:0/1951787954 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb28006aea0 0x7fb28006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:19.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.329+0000 7fb2937fe640 1 --2- 172.21.15.76:0/1951787954 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb28006aea0 0x7fb28006d360 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb284034000 tx=0x7fb284009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:19.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.332+0000 7fb27bfff640 1 -- 172.21.15.76:0/1951787954 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb294155f50 con 0x7fb2941591b0 2023-11-09T03:48:19.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.500+0000 7fb299bfb640 1 -- 172.21.15.76:0/1951787954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb294155f50 con 0x7fb2941591b0 2023-11-09T03:48:19.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.500+0000 7fb27bfff640 1 -- 172.21.15.76:0/1951787954 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb294155f50 con 0x7fb2941591b0 2023-11-09T03:48:19.501 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:48:19.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.502+0000 7fb299bfb640 1 -- 172.21.15.76:0/1951787954 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb28006aea0 msgr2=0x7fb28006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:19.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.502+0000 7fb299bfb640 1 --2- 172.21.15.76:0/1951787954 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb28006aea0 0x7fb28006d360 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb284034000 tx=0x7fb284009030 comp rx=0 tx=0).stop 2023-11-09T03:48:19.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.502+0000 7fb299bfb640 1 -- 172.21.15.76:0/1951787954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2941591b0 msgr2=0x7fb2941568f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:19.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.502+0000 7fb299bfb640 1 --2- 172.21.15.76:0/1951787954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2941591b0 0x7fb2941568f0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fb2940cb070 tx=0x7fb28c00eb90 comp rx=0 tx=0).stop 2023-11-09T03:48:19.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.502+0000 7fb299bfb640 1 -- 172.21.15.76:0/1951787954 shutdown_connections 2023-11-09T03:48:19.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.502+0000 7fb299bfb640 1 --2- 172.21.15.76:0/1951787954 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb28006aea0 0x7fb28006d360 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.502+0000 7fb299bfb640 1 --2- 172.21.15.76:0/1951787954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2941591b0 0x7fb2941568f0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.502+0000 7fb299bfb640 1 --2- 172.21.15.76:0/1951787954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2941ee600 0x7fb294158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.502+0000 7fb299bfb640 1 --2- 172.21.15.76:0/1951787954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb27c003970 0x7fb294146560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.502+0000 7fb299bfb640 1 -- 172.21.15.76:0/1951787954 >> 172.21.15.76:0/1951787954 conn(0x7fb2940ce950 msgr2=0x7fb29406cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:19.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.502+0000 7fb299bfb640 1 -- 172.21.15.76:0/1951787954 shutdown_connections 2023-11-09T03:48:19.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.502+0000 7fb299bfb640 1 -- 172.21.15.76:0/1951787954 wait complete. 2023-11-09T03:48:19.513 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-09T03:48:19.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.616+0000 7f09604d4640 1 Processor -- start 2023-11-09T03:48:19.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.616+0000 7f09604d4640 1 -- start start 2023-11-09T03:48:19.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.617+0000 7f09604d4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f09580ae1d0 0x7f09580cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:19.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.617+0000 7f09604d4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f095815aff0 con 0x7f0958159c00 2023-11-09T03:48:19.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.617+0000 7f09604d4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f095815a3e0 con 0x7f09580c4ff0 2023-11-09T03:48:19.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.617+0000 7f09604d4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f095815a6c0 con 0x7f09580ae1d0 2023-11-09T03:48:19.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.617+0000 7f095ea71640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f09580ae1d0 0x7f09580cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:19.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.617+0000 7f095e270640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0958159c00 0x7f09580c48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40386/0 (socket says 172.21.15.76:40386) 2023-11-09T03:48:19.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.617+0000 7f095ea71640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f09580ae1d0 0x7f09580cc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51494/0 (socket says 172.21.15.76:51494) 2023-11-09T03:48:19.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.617+0000 7f095e270640 1 -- 172.21.15.76:0/1958664928 learned_addr learned my addr 172.21.15.76:0/1958664928 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:19.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.617+0000 7f095ea71640 1 -- 172.21.15.76:0/1958664928 >> v1:172.21.15.113:6789/0 conn(0x7f09580c4ff0 legacy=0x7f09580adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:19.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.617+0000 7f095ea71640 1 -- 172.21.15.76:0/1958664928 >> v1:172.21.15.76:6789/0 conn(0x7f0958159c00 legacy=0x7f09580c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:19.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.617+0000 7f095ea71640 1 -- 172.21.15.76:0/1958664928 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095815a9a0 con 0x7f09580ae1d0 2023-11-09T03:48:19.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.618+0000 7f095ea71640 1 --2- 172.21.15.76:0/1958664928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09580ae1d0 0x7f09580cc890 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f0950009d90 tx=0x7f095002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32bb929e263ff302 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:19.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.618+0000 7f095d26e640 1 -- 172.21.15.76:0/1958664928 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095002cd00 con 0x7f09580ae1d0 2023-11-09T03:48:19.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.618+0000 7f095d26e640 1 -- 172.21.15.76:0/1958664928 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095002ce60 con 0x7f09580ae1d0 2023-11-09T03:48:19.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.618+0000 7f095d26e640 1 -- 172.21.15.76:0/1958664928 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0950034660 con 0x7f09580ae1d0 2023-11-09T03:48:19.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.618+0000 7f09604d4640 1 -- 172.21.15.76:0/1958664928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09580ae1d0 msgr2=0x7f09580cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:19.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.618+0000 7f09604d4640 1 --2- 172.21.15.76:0/1958664928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09580ae1d0 0x7f09580cc890 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f0950009d90 tx=0x7f095002bf50 comp rx=0 tx=0).stop 2023-11-09T03:48:19.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.619+0000 7f09604d4640 1 -- 172.21.15.76:0/1958664928 shutdown_connections 2023-11-09T03:48:19.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.619+0000 7f09604d4640 1 --2- 172.21.15.76:0/1958664928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09580ae1d0 0x7f09580cc890 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.619+0000 7f09604d4640 1 -- 172.21.15.76:0/1958664928 >> 172.21.15.76:0/1958664928 conn(0x7f09580ce950 msgr2=0x7f09580ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:19.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.619+0000 7f09604d4640 1 -- 172.21.15.76:0/1958664928 shutdown_connections 2023-11-09T03:48:19.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.619+0000 7f09604d4640 1 -- 172.21.15.76:0/1958664928 wait complete. 2023-11-09T03:48:19.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.620+0000 7f09604d4640 1 Processor -- start 2023-11-09T03:48:19.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.620+0000 7f09604d4640 1 -- start start 2023-11-09T03:48:19.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.620+0000 7f09604d4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09580ae1d0 0x7f09580c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:19.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.620+0000 7f09604d4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09580c4ff0 0x7f095814cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:19.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.620+0000 7f09604d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0958159c00 0x7f0958148510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:19.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.620+0000 7f09604d4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095814cff0 con 0x7f0958159c00 2023-11-09T03:48:19.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.620+0000 7f09604d4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09580c6b40 con 0x7f09580ae1d0 2023-11-09T03:48:19.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.620+0000 7f09604d4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09580c6e20 con 0x7f09580c4ff0 2023-11-09T03:48:19.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.621+0000 7f095ea71640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0958159c00 0x7f0958148510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:19.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.621+0000 7f095ea71640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0958159c00 0x7f0958148510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51506/0 (socket says 172.21.15.76:51506) 2023-11-09T03:48:19.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.621+0000 7f095ea71640 1 -- 172.21.15.76:0/3563688408 learned_addr learned my addr 172.21.15.76:0/3563688408 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:19.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.621+0000 7f095e270640 1 --2- 172.21.15.76:0/3563688408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09580ae1d0 0x7f09580c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:19.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.621+0000 7f095da6f640 1 --2- 172.21.15.76:0/3563688408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09580c4ff0 0x7f095814cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:19.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.621+0000 7f095ea71640 1 -- 172.21.15.76:0/3563688408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09580c4ff0 msgr2=0x7f095814cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:19.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.621+0000 7f095ea71640 1 --2- 172.21.15.76:0/3563688408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09580c4ff0 0x7f095814cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.621+0000 7f095ea71640 1 -- 172.21.15.76:0/3563688408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09580ae1d0 msgr2=0x7f09580c6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:19.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.621+0000 7f095ea71640 1 --2- 172.21.15.76:0/3563688408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09580ae1d0 0x7f09580c6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.621+0000 7f095ea71640 1 -- 172.21.15.76:0/3563688408 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0950009a70 con 0x7f0958159c00 2023-11-09T03:48:19.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.621+0000 7f095ea71640 1 --2- 172.21.15.76:0/3563688408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0958159c00 0x7f0958148510 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f0950009eb0 tx=0x7f0950004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:19.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.621+0000 7f094effd640 1 -- 172.21.15.76:0/3563688408 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095003e610 con 0x7f0958159c00 2023-11-09T03:48:19.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.621+0000 7f094effd640 1 -- 172.21.15.76:0/3563688408 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095003e770 con 0x7f0958159c00 2023-11-09T03:48:19.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.621+0000 7f094effd640 1 -- 172.21.15.76:0/3563688408 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095003eb00 con 0x7f0958159c00 2023-11-09T03:48:19.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.621+0000 7f09604d4640 1 -- 172.21.15.76:0/3563688408 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09580c7100 con 0x7f0958159c00 2023-11-09T03:48:19.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.621+0000 7f09604d4640 1 -- 172.21.15.76:0/3563688408 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09580c76c0 con 0x7f0958159c00 2023-11-09T03:48:19.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.622+0000 7f09604d4640 1 -- 172.21.15.76:0/3563688408 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0958074290 con 0x7f0958159c00 2023-11-09T03:48:19.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.623+0000 7f094effd640 1 -- 172.21.15.76:0/3563688408 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f095003b050 con 0x7f0958159c00 2023-11-09T03:48:19.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.623+0000 7f094effd640 1 --2- 172.21.15.76:0/3563688408 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f093c06ae70 0x7f093c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:19.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.623+0000 7f094effd640 1 -- 172.21.15.76:0/3563688408 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6240+0+0 (secure 0 0 0) 0x7f09500ac270 con 0x7f0958159c00 2023-11-09T03:48:19.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.623+0000 7f095e270640 1 --2- 172.21.15.76:0/3563688408 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f093c06ae70 0x7f093c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:19.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.624+0000 7f095e270640 1 --2- 172.21.15.76:0/3563688408 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f093c06ae70 0x7f093c06d330 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0954005880 tx=0x7f0954009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:19.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.627+0000 7f094effd640 1 -- 172.21.15.76:0/3563688408 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f09500771a0 con 0x7f0958159c00 2023-11-09T03:48:19.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.792+0000 7f09604d4640 1 -- 172.21.15.76:0/3563688408 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f095814a9c0 con 0x7f0958159c00 2023-11-09T03:48:19.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.793+0000 7f094effd640 1 -- 172.21.15.76:0/3563688408 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v79) v1 ==== 103+0+11 (secure 0 0 0) 0x7f095007a7c0 con 0x7f0958159c00 2023-11-09T03:48:19.793 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:48:19.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.794+0000 7f09604d4640 1 -- 172.21.15.76:0/3563688408 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f093c06ae70 msgr2=0x7f093c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:19.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.794+0000 7f09604d4640 1 --2- 172.21.15.76:0/3563688408 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f093c06ae70 0x7f093c06d330 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0954005880 tx=0x7f0954009040 comp rx=0 tx=0).stop 2023-11-09T03:48:19.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.794+0000 7f09604d4640 1 -- 172.21.15.76:0/3563688408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0958159c00 msgr2=0x7f0958148510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:19.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.794+0000 7f09604d4640 1 --2- 172.21.15.76:0/3563688408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0958159c00 0x7f0958148510 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f0950009eb0 tx=0x7f0950004a40 comp rx=0 tx=0).stop 2023-11-09T03:48:19.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.795+0000 7f09604d4640 1 -- 172.21.15.76:0/3563688408 shutdown_connections 2023-11-09T03:48:19.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.795+0000 7f09604d4640 1 --2- 172.21.15.76:0/3563688408 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f093c06ae70 0x7f093c06d330 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.795+0000 7f09604d4640 1 --2- 172.21.15.76:0/3563688408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0958159c00 0x7f0958148510 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.795+0000 7f09604d4640 1 --2- 172.21.15.76:0/3563688408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09580c4ff0 0x7f095814cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.795+0000 7f09604d4640 1 --2- 172.21.15.76:0/3563688408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09580ae1d0 0x7f09580c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.795+0000 7f09604d4640 1 -- 172.21.15.76:0/3563688408 >> 172.21.15.76:0/3563688408 conn(0x7f09580ce950 msgr2=0x7f095806d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:19.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.795+0000 7f09604d4640 1 -- 172.21.15.76:0/3563688408 shutdown_connections 2023-11-09T03:48:19.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.795+0000 7f09604d4640 1 -- 172.21.15.76:0/3563688408 wait complete. 2023-11-09T03:48:19.806 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-09T03:48:19.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.909+0000 7f920d3c0640 1 Processor -- start 2023-11-09T03:48:19.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.909+0000 7f920d3c0640 1 -- start start 2023-11-09T03:48:19.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.909+0000 7f920d3c0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92080cd6d0 0x7f9208161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:19.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.909+0000 7f920d3c0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92080cde60 con 0x7f92080b4880 2023-11-09T03:48:19.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.909+0000 7f920d3c0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9208160f90 con 0x7f9208153e60 2023-11-09T03:48:19.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.909+0000 7f920d3c0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9208161960 con 0x7f92080cd6d0 2023-11-09T03:48:19.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.910+0000 7f92077fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f92080b4880 0x7f92080b4ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40394/0 (socket says 172.21.15.76:40394) 2023-11-09T03:48:19.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.910+0000 7f92077fe640 1 -- 172.21.15.76:0/3555685612 learned_addr learned my addr 172.21.15.76:0/3555685612 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:19.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.910+0000 7f92067fc640 1 --2- 172.21.15.76:0/3555685612 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92080cd6d0 0x7f9208161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:19.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.910+0000 7f9205ffb640 1 -- 172.21.15.76:0/3555685612 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2225106422 0 0) 0x7f92080cde60 con 0x7f92080b4880 2023-11-09T03:48:19.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.910+0000 7f9205ffb640 1 -- 172.21.15.76:0/3555685612 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91f0003520 con 0x7f92080b4880 2023-11-09T03:48:19.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.910+0000 7f9205ffb640 1 -- 172.21.15.76:0/3555685612 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 409664297 0 0) 0x7f91f0003520 con 0x7f92080b4880 2023-11-09T03:48:19.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.910+0000 7f9205ffb640 1 -- 172.21.15.76:0/3555685612 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92080cd6d0 msgr2=0x7f9208161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:19.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.910+0000 7f9205ffb640 1 --2- 172.21.15.76:0/3555685612 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92080cd6d0 0x7f9208161420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.910+0000 7f9205ffb640 1 -- 172.21.15.76:0/3555685612 >> v1:172.21.15.113:6789/0 conn(0x7f9208153e60 legacy=0x7f920815dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:19.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.910+0000 7f9205ffb640 1 -- 172.21.15.76:0/3555685612 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92080ce090 con 0x7f92080b4880 2023-11-09T03:48:19.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.910+0000 7f9205ffb640 1 -- 172.21.15.76:0/3555685612 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f92000019f0 con 0x7f92080b4880 2023-11-09T03:48:19.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.910+0000 7f9205ffb640 1 -- 172.21.15.76:0/3555685612 >> v1:172.21.15.76:6789/0 conn(0x7f92080b4880 legacy=0x7f92080b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:19.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.910+0000 7f9205ffb640 1 --2- 172.21.15.76:0/3555685612 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92080cd6d0 0x7f91f00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:19.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.910+0000 7f9205ffb640 1 --2- 172.21.15.76:0/3555685612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91f0003f20 0x7f91f00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:19.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.910+0000 7f9205ffb640 1 --2- 172.21.15.76:0/3555685612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91f0004d00 0x7f91f0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:19.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.910+0000 7f9205ffb640 1 -- 172.21.15.76:0/3555685612 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9208161960 con 0x7f91f0004d00 2023-11-09T03:48:19.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.910+0000 7f9205ffb640 1 -- 172.21.15.76:0/3555685612 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f000ba60 con 0x7f91f0003f20 2023-11-09T03:48:19.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.910+0000 7f9205ffb640 1 -- 172.21.15.76:0/3555685612 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f000bc20 con 0x7f92080cd6d0 2023-11-09T03:48:19.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.910+0000 7f92077fe640 1 --2- 172.21.15.76:0/3555685612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91f0004d00 0x7f91f0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:19.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.910+0000 7f9206ffd640 1 --2- 172.21.15.76:0/3555685612 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92080cd6d0 0x7f91f00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:19.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.910+0000 7f92067fc640 1 --2- 172.21.15.76:0/3555685612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91f0003f20 0x7f91f00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:19.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.911+0000 7f92077fe640 1 -- 172.21.15.76:0/3555685612 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92080cd6d0 msgr2=0x7f91f00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:19.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.911+0000 7f92077fe640 1 --2- 172.21.15.76:0/3555685612 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92080cd6d0 0x7f91f00039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.911+0000 7f92077fe640 1 -- 172.21.15.76:0/3555685612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91f0003f20 msgr2=0x7f91f00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:19.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.911+0000 7f92077fe640 1 --2- 172.21.15.76:0/3555685612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91f0003f20 0x7f91f00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.911+0000 7f92077fe640 1 -- 172.21.15.76:0/3555685612 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f0003520 con 0x7f91f0004d00 2023-11-09T03:48:19.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.911+0000 7f92067fc640 1 --2- 172.21.15.76:0/3555685612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91f0003f20 0x7f91f00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:19.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.911+0000 7f92077fe640 1 --2- 172.21.15.76:0/3555685612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91f0004d00 0x7f91f0005180 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f920000a820 tx=0x7f91f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b96397bd8f4ed6 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:19.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.911+0000 7f9205ffb640 1 -- 172.21.15.76:0/3555685612 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92000072d0 con 0x7f91f0004d00 2023-11-09T03:48:19.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.911+0000 7f9205ffb640 1 -- 172.21.15.76:0/3555685612 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f920002f070 con 0x7f91f0004d00 2023-11-09T03:48:19.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.911+0000 7f9205ffb640 1 -- 172.21.15.76:0/3555685612 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92000076f0 con 0x7f91f0004d00 2023-11-09T03:48:19.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.912+0000 7f920d3c0640 1 -- 172.21.15.76:0/3555685612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91f0004d00 msgr2=0x7f91f0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:19.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.912+0000 7f920d3c0640 1 --2- 172.21.15.76:0/3555685612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91f0004d00 0x7f91f0005180 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f920000a820 tx=0x7f91f00057f0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.912+0000 7f92077fe640 1 -- 172.21.15.76:0/3555685612 reap_dead start 2023-11-09T03:48:19.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.912+0000 7f920d3c0640 1 -- 172.21.15.76:0/3555685612 shutdown_connections 2023-11-09T03:48:19.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.912+0000 7f920d3c0640 1 -- 172.21.15.76:0/3555685612 >> 172.21.15.76:0/3555685612 conn(0x7f92080cf880 msgr2=0x7f92080cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:19.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.912+0000 7f920d3c0640 1 -- 172.21.15.76:0/3555685612 shutdown_connections 2023-11-09T03:48:19.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.912+0000 7f920d3c0640 1 -- 172.21.15.76:0/3555685612 wait complete. 2023-11-09T03:48:19.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.913+0000 7f920d3c0640 1 Processor -- start 2023-11-09T03:48:19.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.913+0000 7f920d3c0640 1 -- start start 2023-11-09T03:48:19.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.914+0000 7f920d3c0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f91f0004d00 0x7f92081503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:19.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.914+0000 7f920d3c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92081f37c0 0x7f92080c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:19.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.914+0000 7f920d3c0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92080cb110 0x7f92080cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:19.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.914+0000 7f920d3c0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92080ca070 con 0x7f92081f37c0 2023-11-09T03:48:19.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.914+0000 7f920d3c0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92080ca350 con 0x7f92080cb110 2023-11-09T03:48:19.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.914+0000 7f920d3c0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92080ca630 con 0x7f91f0004d00 2023-11-09T03:48:19.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.914+0000 7f92067fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92081f37c0 0x7f92080c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:19.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.914+0000 7f92067fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92081f37c0 0x7f92080c9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51516/0 (socket says 172.21.15.76:51516) 2023-11-09T03:48:19.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.914+0000 7f92067fc640 1 -- 172.21.15.76:0/360445174 learned_addr learned my addr 172.21.15.76:0/360445174 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:19.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.914+0000 7f92067fc640 1 -- 172.21.15.76:0/360445174 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f91f0004d00 msgr2=0x7f92081503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:19.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.914+0000 7f9206ffd640 1 --2- 172.21.15.76:0/360445174 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f91f0004d00 0x7f92081503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:19.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.914+0000 7f92077fe640 1 --2- 172.21.15.76:0/360445174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92080cb110 0x7f92080cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:19.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.914+0000 7f92067fc640 1 --2- 172.21.15.76:0/360445174 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f91f0004d00 0x7f92081503e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.914+0000 7f92067fc640 1 -- 172.21.15.76:0/360445174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92080cb110 msgr2=0x7f92080cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:19.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.915+0000 7f92067fc640 1 --2- 172.21.15.76:0/360445174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92080cb110 0x7f92080cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:19.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.915+0000 7f92067fc640 1 -- 172.21.15.76:0/360445174 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92080ca910 con 0x7f92081f37c0 2023-11-09T03:48:19.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.915+0000 7f92077fe640 1 --2- 172.21.15.76:0/360445174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92080cb110 0x7f92080cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:19.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.915+0000 7f92067fc640 1 --2- 172.21.15.76:0/360445174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92081f37c0 0x7f92080c9150 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f91fc011600 tx=0x7f91fc011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:19.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.915+0000 7f91ef7fe640 1 -- 172.21.15.76:0/360445174 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc00f920 con 0x7f92081f37c0 2023-11-09T03:48:19.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.915+0000 7f91ef7fe640 1 -- 172.21.15.76:0/360445174 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91fc00fa80 con 0x7f92081f37c0 2023-11-09T03:48:19.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.915+0000 7f920d3c0640 1 -- 172.21.15.76:0/360445174 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92080cabf0 con 0x7f92081f37c0 2023-11-09T03:48:19.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.915+0000 7f91ef7fe640 1 -- 172.21.15.76:0/360445174 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc00fe10 con 0x7f92081f37c0 2023-11-09T03:48:19.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.915+0000 7f920d3c0640 1 -- 172.21.15.76:0/360445174 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92080c9690 con 0x7f92081f37c0 2023-11-09T03:48:19.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.916+0000 7f920d3c0640 1 -- 172.21.15.76:0/360445174 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92080b5c20 con 0x7f92081f37c0 2023-11-09T03:48:19.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.916+0000 7f91ef7fe640 1 -- 172.21.15.76:0/360445174 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f91fc015460 con 0x7f92081f37c0 2023-11-09T03:48:19.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.917+0000 7f91ef7fe640 1 --2- 172.21.15.76:0/360445174 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f91f406ae70 0x7f91f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:19.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.917+0000 7f91ef7fe640 1 -- 172.21.15.76:0/360445174 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6240+0+0 (secure 0 0 0) 0x7f91fc08a100 con 0x7f92081f37c0 2023-11-09T03:48:19.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.917+0000 7f9206ffd640 1 --2- 172.21.15.76:0/360445174 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f91f406ae70 0x7f91f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:19.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.918+0000 7f9206ffd640 1 --2- 172.21.15.76:0/360445174 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f91f406ae70 0x7f91f406d330 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f91f8005590 tx=0x7f91f8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:19.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:19.921+0000 7f91ef7fe640 1 -- 172.21.15.76:0/360445174 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f91fc054fa0 con 0x7f92081f37c0 2023-11-09T03:48:20.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.087+0000 7f920d3c0640 1 -- 172.21.15.76:0/360445174 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7f9208074290 con 0x7f92081f37c0 2023-11-09T03:48:20.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.088+0000 7f91ef7fe640 1 -- 172.21.15.76:0/360445174 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v79) v1 ==== 103+0+11 (secure 0 0 0) 0x7f91fc0585c0 con 0x7f92081f37c0 2023-11-09T03:48:20.089 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:48:20.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.090+0000 7f920d3c0640 1 -- 172.21.15.76:0/360445174 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f91f406ae70 msgr2=0x7f91f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:20.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.090+0000 7f920d3c0640 1 --2- 172.21.15.76:0/360445174 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f91f406ae70 0x7f91f406d330 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f91f8005590 tx=0x7f91f8009040 comp rx=0 tx=0).stop 2023-11-09T03:48:20.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.090+0000 7f920d3c0640 1 -- 172.21.15.76:0/360445174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92081f37c0 msgr2=0x7f92080c9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:20.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.090+0000 7f920d3c0640 1 --2- 172.21.15.76:0/360445174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92081f37c0 0x7f92080c9150 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f91fc011600 tx=0x7f91fc011ac0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.090+0000 7f920d3c0640 1 -- 172.21.15.76:0/360445174 shutdown_connections 2023-11-09T03:48:20.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.090+0000 7f920d3c0640 1 --2- 172.21.15.76:0/360445174 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f91f406ae70 0x7f91f406d330 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.090+0000 7f920d3c0640 1 --2- 172.21.15.76:0/360445174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92080cb110 0x7f92080cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.090+0000 7f920d3c0640 1 --2- 172.21.15.76:0/360445174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92081f37c0 0x7f92080c9150 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.090+0000 7f920d3c0640 1 --2- 172.21.15.76:0/360445174 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f91f0004d00 0x7f92081503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.090+0000 7f920d3c0640 1 -- 172.21.15.76:0/360445174 >> 172.21.15.76:0/360445174 conn(0x7f92080cf880 msgr2=0x7f92080cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:20.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.090+0000 7f920d3c0640 1 -- 172.21.15.76:0/360445174 shutdown_connections 2023-11-09T03:48:20.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.090+0000 7f920d3c0640 1 -- 172.21.15.76:0/360445174 wait complete. 2023-11-09T03:48:20.101 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-09T03:48:20.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.205+0000 7fab1b0a9640 1 Processor -- start 2023-11-09T03:48:20.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.205+0000 7fab1b0a9640 1 -- start start 2023-11-09T03:48:20.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.205+0000 7fab1b0a9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fab140b2d30 0x7fab1406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:20.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.205+0000 7fab1b0a9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab1406ca40 con 0x7fab1406c1d0 2023-11-09T03:48:20.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.205+0000 7fab1b0a9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab1406cd20 con 0x7fab14160430 2023-11-09T03:48:20.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.205+0000 7fab1b0a9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab140cb580 con 0x7fab140b2d30 2023-11-09T03:48:20.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.206+0000 7fab19646640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fab1406c1d0 0x7fab140cbcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40410/0 (socket says 172.21.15.76:40410) 2023-11-09T03:48:20.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.206+0000 7fab19646640 1 -- 172.21.15.76:0/1624610777 learned_addr learned my addr 172.21.15.76:0/1624610777 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:20.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.206+0000 7fab137fe640 1 -- 172.21.15.76:0/1624610777 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 727242053 0 0) 0x7fab1406ca40 con 0x7fab1406c1d0 2023-11-09T03:48:20.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.206+0000 7fab13fff640 1 --2- 172.21.15.76:0/1624610777 >> v2:172.21.15.113:3300/0 conn(0x7fab140b2d30 0x7fab1406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:20.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.206+0000 7fab137fe640 1 -- 172.21.15.76:0/1624610777 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab00003520 con 0x7fab1406c1d0 2023-11-09T03:48:20.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.206+0000 7fab137fe640 1 -- 172.21.15.76:0/1624610777 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 842649029 0 0) 0x7fab00003520 con 0x7fab1406c1d0 2023-11-09T03:48:20.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.206+0000 7fab137fe640 1 -- 172.21.15.76:0/1624610777 >> v2:172.21.15.113:3300/0 conn(0x7fab140b2d30 msgr2=0x7fab1406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:20.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.206+0000 7fab137fe640 1 --2- 172.21.15.76:0/1624610777 >> v2:172.21.15.113:3300/0 conn(0x7fab140b2d30 0x7fab1406bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.206+0000 7fab137fe640 1 -- 172.21.15.76:0/1624610777 >> v1:172.21.15.113:6789/0 conn(0x7fab14160430 legacy=0x7fab140b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:20.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.206+0000 7fab137fe640 1 -- 172.21.15.76:0/1624610777 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1406d000 con 0x7fab1406c1d0 2023-11-09T03:48:20.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.206+0000 7fab137fe640 1 -- 172.21.15.76:0/1624610777 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fab0c0019f0 con 0x7fab1406c1d0 2023-11-09T03:48:20.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.206+0000 7fab137fe640 1 -- 172.21.15.76:0/1624610777 >> v1:172.21.15.76:6789/0 conn(0x7fab1406c1d0 legacy=0x7fab140cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:20.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.206+0000 7fab137fe640 1 --2- 172.21.15.76:0/1624610777 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab00003970 0x7fab00003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:20.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.206+0000 7fab137fe640 1 --2- 172.21.15.76:0/1624610777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab000047c0 0x7fab00004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:20.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.206+0000 7fab137fe640 1 --2- 172.21.15.76:0/1624610777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab00005180 0x7fab0000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:20.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.206+0000 7fab137fe640 1 -- 172.21.15.76:0/1624610777 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab140cb580 con 0x7fab00005180 2023-11-09T03:48:20.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.206+0000 7fab137fe640 1 -- 172.21.15.76:0/1624610777 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0000c020 con 0x7fab000047c0 2023-11-09T03:48:20.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.206+0000 7fab137fe640 1 -- 172.21.15.76:0/1624610777 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab0000c1e0 con 0x7fab00003970 2023-11-09T03:48:20.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.206+0000 7fab19646640 1 --2- 172.21.15.76:0/1624610777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab00005180 0x7fab0000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:20.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.206+0000 7fab13fff640 1 --2- 172.21.15.76:0/1624610777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab000047c0 0x7fab00004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:20.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.206+0000 7fab18e45640 1 --2- 172.21.15.76:0/1624610777 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab00003970 0x7fab00003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:20.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.207+0000 7fab19646640 1 -- 172.21.15.76:0/1624610777 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab00003970 msgr2=0x7fab00003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:20.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.207+0000 7fab19646640 1 --2- 172.21.15.76:0/1624610777 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab00003970 0x7fab00003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.207+0000 7fab19646640 1 -- 172.21.15.76:0/1624610777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab000047c0 msgr2=0x7fab00004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:20.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.207+0000 7fab19646640 1 --2- 172.21.15.76:0/1624610777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab000047c0 0x7fab00004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.207+0000 7fab19646640 1 -- 172.21.15.76:0/1624610777 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab00003520 con 0x7fab00005180 2023-11-09T03:48:20.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.207+0000 7fab18e45640 1 --2- 172.21.15.76:0/1624610777 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab00003970 0x7fab00003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:20.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.207+0000 7fab19646640 1 -- 172.21.15.76:0/1624610777 reap_dead start 2023-11-09T03:48:20.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.207+0000 7fab19646640 1 --2- 172.21.15.76:0/1624610777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab00005180 0x7fab0000b980 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00a830 tx=0x7fab000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a5aab5e83f8700 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:20.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.207+0000 7fab137fe640 1 -- 172.21.15.76:0/1624610777 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c0072d0 con 0x7fab00005180 2023-11-09T03:48:20.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.207+0000 7fab137fe640 1 -- 172.21.15.76:0/1624610777 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0c02f070 con 0x7fab00005180 2023-11-09T03:48:20.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.207+0000 7fab137fe640 1 -- 172.21.15.76:0/1624610777 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c007690 con 0x7fab00005180 2023-11-09T03:48:20.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.208+0000 7fab1b0a9640 1 -- 172.21.15.76:0/1624610777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab00005180 msgr2=0x7fab0000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:20.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.208+0000 7fab1b0a9640 1 --2- 172.21.15.76:0/1624610777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab00005180 0x7fab0000b980 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00a830 tx=0x7fab000057f0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.208+0000 7fab1b0a9640 1 -- 172.21.15.76:0/1624610777 shutdown_connections 2023-11-09T03:48:20.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.208+0000 7fab1b0a9640 1 --2- 172.21.15.76:0/1624610777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab00005180 0x7fab0000b980 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.208+0000 7fab1b0a9640 1 -- 172.21.15.76:0/1624610777 >> 172.21.15.76:0/1624610777 conn(0x7fab1406ea20 msgr2=0x7fab1406ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:20.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.208+0000 7fab1b0a9640 1 -- 172.21.15.76:0/1624610777 shutdown_connections 2023-11-09T03:48:20.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.208+0000 7fab1b0a9640 1 -- 172.21.15.76:0/1624610777 wait complete. 2023-11-09T03:48:20.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.209+0000 7fab1b0a9640 1 Processor -- start 2023-11-09T03:48:20.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.209+0000 7fab1b0a9640 1 -- start start 2023-11-09T03:48:20.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.210+0000 7fab1b0a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab00005180 0x7fab1415f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:20.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.210+0000 7fab1b0a9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab1415f8d0 0x7fab1415ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:20.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.210+0000 7fab1b0a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab14152320 0x7fab140cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:20.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.210+0000 7fab1b0a9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab14152820 con 0x7fab1415f8d0 2023-11-09T03:48:20.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.210+0000 7fab1b0a9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab14152b00 con 0x7fab14152320 2023-11-09T03:48:20.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.210+0000 7fab1b0a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab140b4970 con 0x7fab00005180 2023-11-09T03:48:20.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.210+0000 7fab13fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab1415f8d0 0x7fab1415ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:20.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.210+0000 7fab13fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab1415f8d0 0x7fab1415ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51544/0 (socket says 172.21.15.76:51544) 2023-11-09T03:48:20.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.210+0000 7fab13fff640 1 -- 172.21.15.76:0/87091330 learned_addr learned my addr 172.21.15.76:0/87091330 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:20.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.210+0000 7fab13fff640 1 -- 172.21.15.76:0/87091330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab00005180 msgr2=0x7fab1415f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:20.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.210+0000 7fab13fff640 1 --2- 172.21.15.76:0/87091330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab00005180 0x7fab1415f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.210+0000 7fab13fff640 1 -- 172.21.15.76:0/87091330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab14152320 msgr2=0x7fab140cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:20.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.210+0000 7fab13fff640 1 --2- 172.21.15.76:0/87091330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab14152320 0x7fab140cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.210+0000 7fab13fff640 1 -- 172.21.15.76:0/87091330 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab140b4c50 con 0x7fab1415f8d0 2023-11-09T03:48:20.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.211+0000 7fab13fff640 1 --2- 172.21.15.76:0/87091330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab1415f8d0 0x7fab1415ec90 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7faafc0118e0 tx=0x7faafc011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:20.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.211+0000 7fab117fa640 1 -- 172.21.15.76:0/87091330 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc00faf0 con 0x7fab1415f8d0 2023-11-09T03:48:20.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.211+0000 7fab117fa640 1 -- 172.21.15.76:0/87091330 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faafc00fc50 con 0x7fab1415f8d0 2023-11-09T03:48:20.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.211+0000 7fab1b0a9640 1 -- 172.21.15.76:0/87091330 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab140b4f30 con 0x7fab1415f8d0 2023-11-09T03:48:20.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.211+0000 7fab117fa640 1 -- 172.21.15.76:0/87091330 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc014440 con 0x7fab1415f8d0 2023-11-09T03:48:20.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.211+0000 7fab1b0a9640 1 -- 172.21.15.76:0/87091330 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1415c0c0 con 0x7fab1415f8d0 2023-11-09T03:48:20.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.212+0000 7fab1b0a9640 1 -- 172.21.15.76:0/87091330 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab14074290 con 0x7fab1415f8d0 2023-11-09T03:48:20.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.212+0000 7fab117fa640 1 -- 172.21.15.76:0/87091330 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faafc0145a0 con 0x7fab1415f8d0 2023-11-09T03:48:20.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.212+0000 7fab117fa640 1 --2- 172.21.15.76:0/87091330 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faaf406ae70 0x7faaf406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:20.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.212+0000 7fab117fa640 1 -- 172.21.15.76:0/87091330 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6240+0+0 (secure 0 0 0) 0x7faafc08a160 con 0x7fab1415f8d0 2023-11-09T03:48:20.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.213+0000 7fab18e45640 1 --2- 172.21.15.76:0/87091330 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faaf406ae70 0x7faaf406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:20.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.213+0000 7fab18e45640 1 --2- 172.21.15.76:0/87091330 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faaf406ae70 0x7faaf406d330 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fab14161b70 tx=0x7fab04009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:20.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.217+0000 7fab117fa640 1 -- 172.21.15.76:0/87091330 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faafc055000 con 0x7fab1415f8d0 2023-11-09T03:48:20.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.381+0000 7fab1b0a9640 1 -- 172.21.15.76:0/87091330 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7fab140b6b20 con 0x7fab1415f8d0 2023-11-09T03:48:20.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.381+0000 7fab117fa640 1 -- 172.21.15.76:0/87091330 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v79) v1 ==== 107+0+11 (secure 0 0 0) 0x7faafc058620 con 0x7fab1415f8d0 2023-11-09T03:48:20.382 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:48:20.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.383+0000 7fab1b0a9640 1 -- 172.21.15.76:0/87091330 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faaf406ae70 msgr2=0x7faaf406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:20.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.383+0000 7fab1b0a9640 1 --2- 172.21.15.76:0/87091330 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faaf406ae70 0x7faaf406d330 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fab14161b70 tx=0x7fab04009040 comp rx=0 tx=0).stop 2023-11-09T03:48:20.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.383+0000 7fab1b0a9640 1 -- 172.21.15.76:0/87091330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab1415f8d0 msgr2=0x7fab1415ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:20.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.383+0000 7fab1b0a9640 1 --2- 172.21.15.76:0/87091330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab1415f8d0 0x7fab1415ec90 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7faafc0118e0 tx=0x7faafc011da0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.383+0000 7fab1b0a9640 1 -- 172.21.15.76:0/87091330 shutdown_connections 2023-11-09T03:48:20.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.383+0000 7fab1b0a9640 1 --2- 172.21.15.76:0/87091330 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faaf406ae70 0x7faaf406d330 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.383+0000 7fab1b0a9640 1 --2- 172.21.15.76:0/87091330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab14152320 0x7fab140cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.383+0000 7fab1b0a9640 1 --2- 172.21.15.76:0/87091330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab1415f8d0 0x7fab1415ec90 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.383+0000 7fab1b0a9640 1 --2- 172.21.15.76:0/87091330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab00005180 0x7fab1415f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.383+0000 7fab1b0a9640 1 -- 172.21.15.76:0/87091330 >> 172.21.15.76:0/87091330 conn(0x7fab1406ea20 msgr2=0x7fab140c7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:20.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.383+0000 7fab1b0a9640 1 -- 172.21.15.76:0/87091330 shutdown_connections 2023-11-09T03:48:20.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.383+0000 7fab1b0a9640 1 -- 172.21.15.76:0/87091330 wait complete. 2023-11-09T03:48:20.394 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-09T03:48:20.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.494+0000 7fb724ee0640 1 Processor -- start 2023-11-09T03:48:20.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.494+0000 7fb724ee0640 1 -- start start 2023-11-09T03:48:20.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.494+0000 7fb724ee0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb7200cde60 0x7fb72015d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:20.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.494+0000 7fb724ee0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb72015daa0 0x7fb720157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:20.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.494+0000 7fb724ee0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7200ce240 0x7fb720157540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:20.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.494+0000 7fb724ee0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb72015e5c0 con 0x7fb7200cde60 2023-11-09T03:48:20.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.494+0000 7fb724ee0640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb720157a80 con 0x7fb72015daa0 2023-11-09T03:48:20.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.494+0000 7fb724ee0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72015e000 con 0x7fb7200ce240 2023-11-09T03:48:20.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.495+0000 7fb71e59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb7200cde60 0x7fb72015d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:20.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.495+0000 7fb71e59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb7200cde60 0x7fb72015d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51550/0 (socket says 172.21.15.76:51550) 2023-11-09T03:48:20.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.495+0000 7fb71e59c640 1 -- 172.21.15.76:0/1619320979 learned_addr learned my addr 172.21.15.76:0/1619320979 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:20.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.495+0000 7fb71e59c640 1 -- 172.21.15.76:0/1619320979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7200ce240 msgr2=0x7fb720157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:20.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.495+0000 7fb71e59c640 1 --2- 172.21.15.76:0/1619320979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7200ce240 0x7fb720157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.495+0000 7fb71e59c640 1 -- 172.21.15.76:0/1619320979 >> v2:172.21.15.113:3300/0 conn(0x7fb72015daa0 msgr2=0x7fb720157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:20.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.495+0000 7fb71e59c640 1 --2- 172.21.15.76:0/1619320979 >> v2:172.21.15.113:3300/0 conn(0x7fb72015daa0 0x7fb720157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.495+0000 7fb71e59c640 1 -- 172.21.15.76:0/1619320979 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72015e2e0 con 0x7fb7200cde60 2023-11-09T03:48:20.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.496+0000 7fb71e59c640 1 --2- 172.21.15.76:0/1619320979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7200cde60 0x7fb72015d560 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fb7200cf5f0 tx=0x7fb71002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b53b86e5d9591c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:20.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.496+0000 7fb71dd9b640 1 -- 172.21.15.76:0/1619320979 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb710031870 con 0x7fb7200cde60 2023-11-09T03:48:20.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.496+0000 7fb71dd9b640 1 -- 172.21.15.76:0/1619320979 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7100319d0 con 0x7fb7200cde60 2023-11-09T03:48:20.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.496+0000 7fb71dd9b640 1 -- 172.21.15.76:0/1619320979 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb710031d60 con 0x7fb7200cde60 2023-11-09T03:48:20.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.496+0000 7fb724ee0640 1 -- 172.21.15.76:0/1619320979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7200cde60 msgr2=0x7fb72015d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:20.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.496+0000 7fb724ee0640 1 --2- 172.21.15.76:0/1619320979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7200cde60 0x7fb72015d560 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fb7200cf5f0 tx=0x7fb71002b910 comp rx=0 tx=0).stop 2023-11-09T03:48:20.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.496+0000 7fb724ee0640 1 -- 172.21.15.76:0/1619320979 shutdown_connections 2023-11-09T03:48:20.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.496+0000 7fb724ee0640 1 --2- 172.21.15.76:0/1619320979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7200ce240 0x7fb720157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.496+0000 7fb724ee0640 1 --2- 172.21.15.76:0/1619320979 >> v2:172.21.15.113:3300/0 conn(0x7fb72015daa0 0x7fb720157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.496+0000 7fb724ee0640 1 --2- 172.21.15.76:0/1619320979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7200cde60 0x7fb72015d560 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.496+0000 7fb724ee0640 1 -- 172.21.15.76:0/1619320979 >> 172.21.15.76:0/1619320979 conn(0x7fb720071f30 msgr2=0x7fb7200cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:20.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.496+0000 7fb724ee0640 1 -- 172.21.15.76:0/1619320979 shutdown_connections 2023-11-09T03:48:20.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.496+0000 7fb724ee0640 1 -- 172.21.15.76:0/1619320979 wait complete. 2023-11-09T03:48:20.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.498+0000 7fb724ee0640 1 Processor -- start 2023-11-09T03:48:20.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.498+0000 7fb724ee0640 1 -- start start 2023-11-09T03:48:20.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.498+0000 7fb724ee0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7200cde60 0x7fb72015b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:20.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.498+0000 7fb724ee0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7200ce240 0x7fb72015c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:20.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.498+0000 7fb724ee0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb72015daa0 0x7fb72014f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:20.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.498+0000 7fb724ee0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72015a5e0 con 0x7fb7200ce240 2023-11-09T03:48:20.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.498+0000 7fb724ee0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72015bb10 con 0x7fb7200cde60 2023-11-09T03:48:20.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.498+0000 7fb724ee0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72015a830 con 0x7fb72015daa0 2023-11-09T03:48:20.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.498+0000 7fb717fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7200ce240 0x7fb72015c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:20.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.498+0000 7fb717fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7200ce240 0x7fb72015c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51556/0 (socket says 172.21.15.76:51556) 2023-11-09T03:48:20.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.498+0000 7fb717fff640 1 -- 172.21.15.76:0/1113295073 learned_addr learned my addr 172.21.15.76:0/1113295073 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:20.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.499+0000 7fb71e59c640 1 --2- 172.21.15.76:0/1113295073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7200cde60 0x7fb72015b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:20.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.499+0000 7fb71ed9d640 1 --2- 172.21.15.76:0/1113295073 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb72015daa0 0x7fb72014f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:20.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.499+0000 7fb717fff640 1 -- 172.21.15.76:0/1113295073 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb72015daa0 msgr2=0x7fb72014f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:20.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.499+0000 7fb717fff640 1 --2- 172.21.15.76:0/1113295073 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb72015daa0 0x7fb72014f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.499+0000 7fb717fff640 1 -- 172.21.15.76:0/1113295073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7200cde60 msgr2=0x7fb72015b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:20.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.499+0000 7fb717fff640 1 --2- 172.21.15.76:0/1113295073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7200cde60 0x7fb72015b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.499+0000 7fb717fff640 1 -- 172.21.15.76:0/1113295073 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7100094d0 con 0x7fb7200ce240 2023-11-09T03:48:20.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.499+0000 7fb71e59c640 1 --2- 172.21.15.76:0/1113295073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7200cde60 0x7fb72015b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:20.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.499+0000 7fb717fff640 1 --2- 172.21.15.76:0/1113295073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7200ce240 0x7fb72015c310 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fb70800b410 tx=0x7fb70800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:20.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.499+0000 7fb716ffd640 1 -- 172.21.15.76:0/1113295073 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb708011830 con 0x7fb7200ce240 2023-11-09T03:48:20.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.499+0000 7fb716ffd640 1 -- 172.21.15.76:0/1113295073 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb708011990 con 0x7fb7200ce240 2023-11-09T03:48:20.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.499+0000 7fb724ee0640 1 -- 172.21.15.76:0/1113295073 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb72015ab10 con 0x7fb7200ce240 2023-11-09T03:48:20.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.499+0000 7fb716ffd640 1 -- 172.21.15.76:0/1113295073 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb708011d40 con 0x7fb7200ce240 2023-11-09T03:48:20.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.499+0000 7fb724ee0640 1 -- 172.21.15.76:0/1113295073 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb72015b0d0 con 0x7fb7200ce240 2023-11-09T03:48:20.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.500+0000 7fb724ee0640 1 -- 172.21.15.76:0/1113295073 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb72015b290 con 0x7fb7200ce240 2023-11-09T03:48:20.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.500+0000 7fb716ffd640 1 -- 172.21.15.76:0/1113295073 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb70801a400 con 0x7fb7200ce240 2023-11-09T03:48:20.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.501+0000 7fb716ffd640 1 --2- 172.21.15.76:0/1113295073 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb70006ae70 0x7fb70006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:20.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.501+0000 7fb716ffd640 1 -- 172.21.15.76:0/1113295073 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6240+0+0 (secure 0 0 0) 0x7fb708089a00 con 0x7fb7200ce240 2023-11-09T03:48:20.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.501+0000 7fb71e59c640 1 --2- 172.21.15.76:0/1113295073 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb70006ae70 0x7fb70006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:20.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.502+0000 7fb71e59c640 1 --2- 172.21.15.76:0/1113295073 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb70006ae70 0x7fb70006d330 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb710036000 tx=0x7fb71003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:20.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.505+0000 7fb716ffd640 1 -- 172.21.15.76:0/1113295073 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb708054810 con 0x7fb7200ce240 2023-11-09T03:48:20.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.675+0000 7fb724ee0640 1 -- 172.21.15.76:0/1113295073 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb720074290 con 0x7fb7200ce240 2023-11-09T03:48:20.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.675+0000 7fb716ffd640 1 -- 172.21.15.76:0/1113295073 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+287 (secure 0 0 0) 0x7fb708015070 con 0x7fb7200ce240 2023-11-09T03:48:20.676 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:20.676 INFO:teuthology.orchestra.run.smithi076.stdout:[ 2023-11-09T03:48:20.677 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:20.677 INFO:teuthology.orchestra.run.smithi076.stdout: "poolnum": 1, 2023-11-09T03:48:20.677 INFO:teuthology.orchestra.run.smithi076.stdout: "poolname": ".mgr" 2023-11-09T03:48:20.677 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:20.677 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:20.678 INFO:teuthology.orchestra.run.smithi076.stdout: "poolnum": 7, 2023-11-09T03:48:20.678 INFO:teuthology.orchestra.run.smithi076.stdout: "poolname": "snap-test-pool" 2023-11-09T03:48:20.678 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:20.678 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:20.678 INFO:teuthology.orchestra.run.smithi076.stdout: "poolnum": 12, 2023-11-09T03:48:20.679 INFO:teuthology.orchestra.run.smithi076.stdout: "poolname": "test_data_pool" 2023-11-09T03:48:20.679 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:20.679 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:20.679 INFO:teuthology.orchestra.run.smithi076.stdout: "poolnum": 13, 2023-11-09T03:48:20.679 INFO:teuthology.orchestra.run.smithi076.stdout: "poolname": "test_metadata_pool" 2023-11-09T03:48:20.680 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:20.680 INFO:teuthology.orchestra.run.smithi076.stdout:] 2023-11-09T03:48:20.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.677+0000 7fb724ee0640 1 -- 172.21.15.76:0/1113295073 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb70006ae70 msgr2=0x7fb70006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:20.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.677+0000 7fb724ee0640 1 --2- 172.21.15.76:0/1113295073 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb70006ae70 0x7fb70006d330 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb710036000 tx=0x7fb71003c040 comp rx=0 tx=0).stop 2023-11-09T03:48:20.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.677+0000 7fb724ee0640 1 -- 172.21.15.76:0/1113295073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7200ce240 msgr2=0x7fb72015c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:20.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.677+0000 7fb724ee0640 1 --2- 172.21.15.76:0/1113295073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7200ce240 0x7fb72015c310 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fb70800b410 tx=0x7fb70800b8d0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.677+0000 7fb724ee0640 1 -- 172.21.15.76:0/1113295073 shutdown_connections 2023-11-09T03:48:20.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.677+0000 7fb724ee0640 1 --2- 172.21.15.76:0/1113295073 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb70006ae70 0x7fb70006d330 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.677+0000 7fb724ee0640 1 --2- 172.21.15.76:0/1113295073 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb72015daa0 0x7fb72014f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.677+0000 7fb724ee0640 1 --2- 172.21.15.76:0/1113295073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7200ce240 0x7fb72015c310 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.677+0000 7fb724ee0640 1 --2- 172.21.15.76:0/1113295073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7200cde60 0x7fb72015b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.677+0000 7fb724ee0640 1 -- 172.21.15.76:0/1113295073 >> 172.21.15.76:0/1113295073 conn(0x7fb720071f30 msgr2=0x7fb720156dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:20.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.677+0000 7fb724ee0640 1 -- 172.21.15.76:0/1113295073 shutdown_connections 2023-11-09T03:48:20.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.677+0000 7fb724ee0640 1 -- 172.21.15.76:0/1113295073 wait complete. 2023-11-09T03:48:20.688 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-09T03:48:20.689 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-09T03:48:20.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.793+0000 7fb1f4ce5640 1 Processor -- start 2023-11-09T03:48:20.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.793+0000 7fb1f4ce5640 1 -- start start 2023-11-09T03:48:20.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.793+0000 7fb1f4ce5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb1f00cd9f0 0x7fb1f00cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:20.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.793+0000 7fb1f4ce5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1f015d4b0 0x7fb1f015d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:20.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.793+0000 7fb1f4ce5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1f00d1320 con 0x7fb1f00b47e0 2023-11-09T03:48:20.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.793+0000 7fb1f4ce5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1f015e4d0 con 0x7fb1f00cd9f0 2023-11-09T03:48:20.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.793+0000 7fb1f4ce5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f00ce310 con 0x7fb1f015d4b0 2023-11-09T03:48:20.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.794+0000 7fb1ee59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb1f00cd9f0 0x7fb1f00cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:20.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.794+0000 7fb1eed9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb1f00b47e0 0x7fb1f00b4c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40426/0 (socket says 172.21.15.76:40426) 2023-11-09T03:48:20.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.794+0000 7fb1eed9d640 1 -- 172.21.15.76:0/187836500 learned_addr learned my addr 172.21.15.76:0/187836500 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:20.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.794+0000 7fb1ee59c640 1 -- 172.21.15.76:0/187836500 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1f015d4b0 msgr2=0x7fb1f015d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:20.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.794+0000 7fb1ee59c640 1 --2- 172.21.15.76:0/187836500 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1f015d4b0 0x7fb1f015d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.794+0000 7fb1ee59c640 1 -- 172.21.15.76:0/187836500 >> v1:172.21.15.76:6789/0 conn(0x7fb1f00b47e0 legacy=0x7fb1f00b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:20.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.794+0000 7fb1ee59c640 1 -- 172.21.15.76:0/187836500 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f015df10 con 0x7fb1f00cd9f0 2023-11-09T03:48:20.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.794+0000 7fb1edd9b640 1 -- 172.21.15.76:0/187836500 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 881741784 0 0) 0x7fb1f00d1320 con 0x7fb1f00b47e0 2023-11-09T03:48:20.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.795+0000 7fb1ee59c640 1 --2- 172.21.15.76:0/187836500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1f00cd9f0 0x7fb1f00cddd0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fb1e00097a0 tx=0x7fb1e002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a02d48aa7948b0a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:20.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.795+0000 7fb1edd9b640 1 -- 172.21.15.76:0/187836500 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0030610 con 0x7fb1f00cd9f0 2023-11-09T03:48:20.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.795+0000 7fb1edd9b640 1 -- 172.21.15.76:0/187836500 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e0030770 con 0x7fb1f00cd9f0 2023-11-09T03:48:20.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.795+0000 7fb1edd9b640 1 -- 172.21.15.76:0/187836500 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0030b00 con 0x7fb1f00cd9f0 2023-11-09T03:48:20.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.795+0000 7fb1f4ce5640 1 -- 172.21.15.76:0/187836500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1f00cd9f0 msgr2=0x7fb1f00cddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:20.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.795+0000 7fb1f4ce5640 1 --2- 172.21.15.76:0/187836500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1f00cd9f0 0x7fb1f00cddd0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fb1e00097a0 tx=0x7fb1e002b960 comp rx=0 tx=0).stop 2023-11-09T03:48:20.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.795+0000 7fb1f4ce5640 1 -- 172.21.15.76:0/187836500 shutdown_connections 2023-11-09T03:48:20.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.795+0000 7fb1f4ce5640 1 --2- 172.21.15.76:0/187836500 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1f015d4b0 0x7fb1f015d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.795+0000 7fb1f4ce5640 1 --2- 172.21.15.76:0/187836500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1f00cd9f0 0x7fb1f00cddd0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.795+0000 7fb1f4ce5640 1 -- 172.21.15.76:0/187836500 >> 172.21.15.76:0/187836500 conn(0x7fb1f0071c40 msgr2=0x7fb1f0072050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:20.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.795+0000 7fb1f4ce5640 1 -- 172.21.15.76:0/187836500 shutdown_connections 2023-11-09T03:48:20.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.795+0000 7fb1f4ce5640 1 -- 172.21.15.76:0/187836500 wait complete. 2023-11-09T03:48:20.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.797+0000 7fb1f4ce5640 1 Processor -- start 2023-11-09T03:48:20.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.797+0000 7fb1f4ce5640 1 -- start start 2023-11-09T03:48:20.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.797+0000 7fb1f4ce5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1f00b47e0 0x7fb1f01579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:20.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.797+0000 7fb1f4ce5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1f00cd9f0 0x7fb1f00cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:20.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.797+0000 7fb1f4ce5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1f015d4b0 0x7fb1f01f6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:20.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.797+0000 7fb1f4ce5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f01582f0 con 0x7fb1f015d4b0 2023-11-09T03:48:20.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.797+0000 7fb1f4ce5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f01585d0 con 0x7fb1f00cd9f0 2023-11-09T03:48:20.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.797+0000 7fb1f4ce5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f01588b0 con 0x7fb1f00b47e0 2023-11-09T03:48:20.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.797+0000 7fb1eed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1f015d4b0 0x7fb1f01f6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:20.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.797+0000 7fb1eed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1f015d4b0 0x7fb1f01f6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51574/0 (socket says 172.21.15.76:51574) 2023-11-09T03:48:20.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.797+0000 7fb1eed9d640 1 -- 172.21.15.76:0/1221607735 learned_addr learned my addr 172.21.15.76:0/1221607735 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:20.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.798+0000 7fb1ee59c640 1 --2- 172.21.15.76:0/1221607735 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1f00b47e0 0x7fb1f01579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:20.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.798+0000 7fb1e7fff640 1 --2- 172.21.15.76:0/1221607735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1f00cd9f0 0x7fb1f00cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:20.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.798+0000 7fb1eed9d640 1 -- 172.21.15.76:0/1221607735 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1f00b47e0 msgr2=0x7fb1f01579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:20.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.798+0000 7fb1eed9d640 1 --2- 172.21.15.76:0/1221607735 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1f00b47e0 0x7fb1f01579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.798+0000 7fb1eed9d640 1 -- 172.21.15.76:0/1221607735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1f00cd9f0 msgr2=0x7fb1f00cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:20.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.798+0000 7fb1eed9d640 1 --2- 172.21.15.76:0/1221607735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1f00cd9f0 0x7fb1f00cc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:20.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.798+0000 7fb1eed9d640 1 -- 172.21.15.76:0/1221607735 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e0009400 con 0x7fb1f015d4b0 2023-11-09T03:48:20.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.798+0000 7fb1eed9d640 1 --2- 172.21.15.76:0/1221607735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1f015d4b0 0x7fb1f01f6520 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fb1e800c780 tx=0x7fb1e800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:20.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.798+0000 7fb1e6ffd640 1 -- 172.21.15.76:0/1221607735 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e800ce90 con 0x7fb1f015d4b0 2023-11-09T03:48:20.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.798+0000 7fb1e6ffd640 1 -- 172.21.15.76:0/1221607735 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e800a640 con 0x7fb1f015d4b0 2023-11-09T03:48:20.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.798+0000 7fb1e6ffd640 1 -- 172.21.15.76:0/1221607735 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e800a9d0 con 0x7fb1f015d4b0 2023-11-09T03:48:20.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.798+0000 7fb1f4ce5640 1 -- 172.21.15.76:0/1221607735 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1f0158b90 con 0x7fb1f015d4b0 2023-11-09T03:48:20.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.798+0000 7fb1f4ce5640 1 -- 172.21.15.76:0/1221607735 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f0159390 con 0x7fb1f015d4b0 2023-11-09T03:48:20.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.799+0000 7fb1f4ce5640 1 -- 172.21.15.76:0/1221607735 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1f01590b0 con 0x7fb1f015d4b0 2023-11-09T03:48:20.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.800+0000 7fb1e6ffd640 1 -- 172.21.15.76:0/1221607735 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb1e800ab30 con 0x7fb1f015d4b0 2023-11-09T03:48:20.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.800+0000 7fb1e6ffd640 1 --2- 172.21.15.76:0/1221607735 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb1d006aea0 0x7fb1d006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:20.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.800+0000 7fb1e6ffd640 1 -- 172.21.15.76:0/1221607735 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6240+0+0 (secure 0 0 0) 0x7fb1e8088b50 con 0x7fb1f015d4b0 2023-11-09T03:48:20.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.800+0000 7fb1ee59c640 1 --2- 172.21.15.76:0/1221607735 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb1d006aea0 0x7fb1d006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:20.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.801+0000 7fb1ee59c640 1 --2- 172.21.15.76:0/1221607735 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb1d006aea0 0x7fb1d006d360 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0036000 tx=0x7fb1e003b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:20.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.804+0000 7fb1e6ffd640 1 -- 172.21.15.76:0/1221607735 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb1f01590b0 con 0x7fb1f015d4b0 2023-11-09T03:48:20.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:20.973+0000 7fb1f4ce5640 1 -- 172.21.15.76:0/1221607735 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb1f01590b0 con 0x7fb1f015d4b0 2023-11-09T03:48:21.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.535+0000 7fb1e6ffd640 1 -- 172.21.15.76:0/1221607735 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb1f01590b0 con 0x7fb1f015d4b0 2023-11-09T03:48:21.536 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' created 2023-11-09T03:48:21.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.538+0000 7fb1f4ce5640 1 -- 172.21.15.76:0/1221607735 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb1d006aea0 msgr2=0x7fb1d006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:21.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.538+0000 7fb1f4ce5640 1 --2- 172.21.15.76:0/1221607735 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb1d006aea0 0x7fb1d006d360 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0036000 tx=0x7fb1e003b040 comp rx=0 tx=0).stop 2023-11-09T03:48:21.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.538+0000 7fb1f4ce5640 1 -- 172.21.15.76:0/1221607735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1f015d4b0 msgr2=0x7fb1f01f6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:21.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.538+0000 7fb1f4ce5640 1 --2- 172.21.15.76:0/1221607735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1f015d4b0 0x7fb1f01f6520 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fb1e800c780 tx=0x7fb1e800cc40 comp rx=0 tx=0).stop 2023-11-09T03:48:21.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.538+0000 7fb1f4ce5640 1 -- 172.21.15.76:0/1221607735 shutdown_connections 2023-11-09T03:48:21.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.538+0000 7fb1f4ce5640 1 --2- 172.21.15.76:0/1221607735 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb1d006aea0 0x7fb1d006d360 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:21.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.538+0000 7fb1f4ce5640 1 --2- 172.21.15.76:0/1221607735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1f015d4b0 0x7fb1f01f6520 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:21.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.538+0000 7fb1f4ce5640 1 --2- 172.21.15.76:0/1221607735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1f00cd9f0 0x7fb1f00cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:21.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.538+0000 7fb1f4ce5640 1 --2- 172.21.15.76:0/1221607735 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1f00b47e0 0x7fb1f01579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:21.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.538+0000 7fb1f4ce5640 1 -- 172.21.15.76:0/1221607735 >> 172.21.15.76:0/1221607735 conn(0x7fb1f0071c40 msgr2=0x7fb1f00cf770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:21.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.538+0000 7fb1f4ce5640 1 -- 172.21.15.76:0/1221607735 shutdown_connections 2023-11-09T03:48:21.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.538+0000 7fb1f4ce5640 1 -- 172.21.15.76:0/1221607735 wait complete. 2023-11-09T03:48:21.555 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-09T03:48:21.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.664+0000 7f9d8c258640 1 Processor -- start 2023-11-09T03:48:21.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.664+0000 7f9d8c258640 1 -- start start 2023-11-09T03:48:21.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.665+0000 7f9d8c258640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9d8406e990 0x7f9d8406ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:21.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.665+0000 7f9d8c258640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d8406ad60 0x7f9d840ca470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:21.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.665+0000 7f9d8c258640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d8415a2f0 con 0x7f9d840ccba0 2023-11-09T03:48:21.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.665+0000 7f9d8c258640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d8406d0e0 con 0x7f9d8406e990 2023-11-09T03:48:21.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.665+0000 7f9d8c258640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d840cea00 con 0x7f9d8406ad60 2023-11-09T03:48:21.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.665+0000 7f9d897f3640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f9d840ccba0 0x7f9d840c9d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40442/0 (socket says 172.21.15.76:40442) 2023-11-09T03:48:21.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.665+0000 7f9d897f3640 1 -- 172.21.15.76:0/4144825540 learned_addr learned my addr 172.21.15.76:0/4144825540 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:21.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.665+0000 7f9d89ff4640 1 --2- 172.21.15.76:0/4144825540 >> v2:172.21.15.113:3300/0 conn(0x7f9d8406e990 0x7f9d8406ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:21.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.665+0000 7f9d8a7f5640 1 --2- 172.21.15.76:0/4144825540 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d8406ad60 0x7f9d840ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:21.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.665+0000 7f9d88ff2640 1 -- 172.21.15.76:0/4144825540 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1965293224 0 0) 0x7f9d8415a2f0 con 0x7f9d840ccba0 2023-11-09T03:48:21.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.665+0000 7f9d88ff2640 1 -- 172.21.15.76:0/4144825540 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d70005730 con 0x7f9d840ccba0 2023-11-09T03:48:21.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.666+0000 7f9d88ff2640 1 -- 172.21.15.76:0/4144825540 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4218286269 0 0) 0x7f9d70005730 con 0x7f9d840ccba0 2023-11-09T03:48:21.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.666+0000 7f9d88ff2640 1 -- 172.21.15.76:0/4144825540 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d8406ad60 msgr2=0x7f9d840ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:21.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.666+0000 7f9d88ff2640 1 --2- 172.21.15.76:0/4144825540 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d8406ad60 0x7f9d840ca470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:21.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.666+0000 7f9d88ff2640 1 -- 172.21.15.76:0/4144825540 >> v2:172.21.15.113:3300/0 conn(0x7f9d8406e990 msgr2=0x7f9d8406ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:21.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.666+0000 7f9d88ff2640 1 --2- 172.21.15.76:0/4144825540 >> v2:172.21.15.113:3300/0 conn(0x7f9d8406e990 0x7f9d8406ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:21.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.666+0000 7f9d88ff2640 1 -- 172.21.15.76:0/4144825540 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8406f2b0 con 0x7f9d840ccba0 2023-11-09T03:48:21.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.666+0000 7f9d88ff2640 1 -- 172.21.15.76:0/4144825540 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f9d80001670 con 0x7f9d840ccba0 2023-11-09T03:48:21.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.666+0000 7f9d88ff2640 1 -- 172.21.15.76:0/4144825540 >> v1:172.21.15.76:6789/0 conn(0x7f9d840ccba0 legacy=0x7f9d840c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:21.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.666+0000 7f9d88ff2640 1 --2- 172.21.15.76:0/4144825540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d70005b80 0x7f9d70005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:21.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.666+0000 7f9d88ff2640 1 --2- 172.21.15.76:0/4144825540 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d700069d0 0x7f9d70006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:21.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.666+0000 7f9d88ff2640 1 --2- 172.21.15.76:0/4144825540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d70007390 0x7f9d7000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:21.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.666+0000 7f9d88ff2640 1 -- 172.21.15.76:0/4144825540 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8406d0e0 con 0x7f9d70007390 2023-11-09T03:48:21.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.666+0000 7f9d88ff2640 1 -- 172.21.15.76:0/4144825540 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d840cea00 con 0x7f9d70005b80 2023-11-09T03:48:21.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.666+0000 7f9d88ff2640 1 -- 172.21.15.76:0/4144825540 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7000e1b0 con 0x7f9d700069d0 2023-11-09T03:48:21.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.666+0000 7f9d8a7f5640 1 --2- 172.21.15.76:0/4144825540 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d8406ad60 0x7f9d840ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:48:21.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.666+0000 7f9d89ff4640 1 --2- 172.21.15.76:0/4144825540 >> v2:172.21.15.113:3300/0 conn(0x7f9d8406e990 0x7f9d8406ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:48:21.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.666+0000 7f9d8a7f5640 1 --2- 172.21.15.76:0/4144825540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d70007390 0x7f9d7000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:21.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.666+0000 7f9d897f3640 1 --2- 172.21.15.76:0/4144825540 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d700069d0 0x7f9d70006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:21.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.666+0000 7f9d89ff4640 1 --2- 172.21.15.76:0/4144825540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d70005b80 0x7f9d70005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:21.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.666+0000 7f9d8a7f5640 1 -- 172.21.15.76:0/4144825540 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d700069d0 msgr2=0x7f9d70006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:21.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.666+0000 7f9d8a7f5640 1 --2- 172.21.15.76:0/4144825540 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d700069d0 0x7f9d70006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:21.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.666+0000 7f9d8a7f5640 1 -- 172.21.15.76:0/4144825540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d70005b80 msgr2=0x7f9d70005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:21.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.666+0000 7f9d8a7f5640 1 --2- 172.21.15.76:0/4144825540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d70005b80 0x7f9d70005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:21.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.666+0000 7f9d8a7f5640 1 -- 172.21.15.76:0/4144825540 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d70005730 con 0x7f9d70007390 2023-11-09T03:48:21.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.667+0000 7f9d8a7f5640 1 --2- 172.21.15.76:0/4144825540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d70007390 0x7f9d7000db90 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f9d8406b140 tx=0x7f9d70007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33abebabe974608 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:21.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.667+0000 7f9d88ff2640 1 -- 172.21.15.76:0/4144825540 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c02ccf0 con 0x7f9d70007390 2023-11-09T03:48:21.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.667+0000 7f9d88ff2640 1 -- 172.21.15.76:0/4144825540 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7c00f040 con 0x7f9d70007390 2023-11-09T03:48:21.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.667+0000 7f9d88ff2640 1 -- 172.21.15.76:0/4144825540 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c003bb0 con 0x7f9d70007390 2023-11-09T03:48:21.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.668+0000 7f9d8c258640 1 -- 172.21.15.76:0/4144825540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d70007390 msgr2=0x7f9d7000db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:21.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.668+0000 7f9d8c258640 1 --2- 172.21.15.76:0/4144825540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d70007390 0x7f9d7000db90 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f9d8406b140 tx=0x7f9d70007a00 comp rx=0 tx=0).stop 2023-11-09T03:48:21.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.668+0000 7f9d8a7f5640 1 -- 172.21.15.76:0/4144825540 reap_dead start 2023-11-09T03:48:21.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.668+0000 7f9d8c258640 1 -- 172.21.15.76:0/4144825540 shutdown_connections 2023-11-09T03:48:21.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.668+0000 7f9d8c258640 1 -- 172.21.15.76:0/4144825540 >> 172.21.15.76:0/4144825540 conn(0x7f9d8406a120 msgr2=0x7f9d8406a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:21.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.668+0000 7f9d8c258640 1 -- 172.21.15.76:0/4144825540 shutdown_connections 2023-11-09T03:48:21.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.668+0000 7f9d8c258640 1 -- 172.21.15.76:0/4144825540 wait complete. 2023-11-09T03:48:21.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.669+0000 7f9d8c258640 1 Processor -- start 2023-11-09T03:48:21.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.669+0000 7f9d8c258640 1 -- start start 2023-11-09T03:48:21.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.669+0000 7f9d8c258640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d70007390 0x7f9d841f26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:21.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.669+0000 7f9d8c258640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d84147020 0x7f9d8415c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:21.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.669+0000 7f9d8c258640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d8415cb50 0x7f9d841ee6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:21.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.670+0000 7f9d8c258640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d841f2bf0 con 0x7f9d70007390 2023-11-09T03:48:21.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.670+0000 7f9d8c258640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8415cfd0 con 0x7f9d84147020 2023-11-09T03:48:21.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.670+0000 7f9d8c258640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d840c6f30 con 0x7f9d8415cb50 2023-11-09T03:48:21.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.670+0000 7f9d89ff4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d70007390 0x7f9d841f26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:21.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.670+0000 7f9d89ff4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d70007390 0x7f9d841f26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51580/0 (socket says 172.21.15.76:51580) 2023-11-09T03:48:21.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.670+0000 7f9d89ff4640 1 -- 172.21.15.76:0/2753598108 learned_addr learned my addr 172.21.15.76:0/2753598108 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:21.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.670+0000 7f9d8a7f5640 1 --2- 172.21.15.76:0/2753598108 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d8415cb50 0x7f9d841ee6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:21.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.670+0000 7f9d897f3640 1 --2- 172.21.15.76:0/2753598108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d84147020 0x7f9d8415c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:21.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.670+0000 7f9d89ff4640 1 -- 172.21.15.76:0/2753598108 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d8415cb50 msgr2=0x7f9d841ee6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:21.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.670+0000 7f9d89ff4640 1 --2- 172.21.15.76:0/2753598108 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d8415cb50 0x7f9d841ee6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:21.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.670+0000 7f9d89ff4640 1 -- 172.21.15.76:0/2753598108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d84147020 msgr2=0x7f9d8415c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:21.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.670+0000 7f9d89ff4640 1 --2- 172.21.15.76:0/2753598108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d84147020 0x7f9d8415c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:21.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.670+0000 7f9d89ff4640 1 -- 172.21.15.76:0/2753598108 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d840c6d20 con 0x7f9d70007390 2023-11-09T03:48:21.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.670+0000 7f9d89ff4640 1 --2- 172.21.15.76:0/2753598108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d70007390 0x7f9d841f26b0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f9d74011900 tx=0x7f9d74011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:21.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.670+0000 7f9d7a7fc640 1 -- 172.21.15.76:0/2753598108 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7400fb70 con 0x7f9d70007390 2023-11-09T03:48:21.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.670+0000 7f9d7a7fc640 1 -- 172.21.15.76:0/2753598108 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7400fcd0 con 0x7f9d70007390 2023-11-09T03:48:21.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.670+0000 7f9d8c258640 1 -- 172.21.15.76:0/2753598108 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d840c61a0 con 0x7f9d70007390 2023-11-09T03:48:21.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.670+0000 7f9d7a7fc640 1 -- 172.21.15.76:0/2753598108 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d74015460 con 0x7f9d70007390 2023-11-09T03:48:21.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.670+0000 7f9d8c258640 1 -- 172.21.15.76:0/2753598108 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d840c6760 con 0x7f9d70007390 2023-11-09T03:48:21.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.671+0000 7f9d8c258640 1 -- 172.21.15.76:0/2753598108 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d84074290 con 0x7f9d70007390 2023-11-09T03:48:21.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.672+0000 7f9d7a7fc640 1 -- 172.21.15.76:0/2753598108 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9d740155c0 con 0x7f9d70007390 2023-11-09T03:48:21.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.672+0000 7f9d7a7fc640 1 --2- 172.21.15.76:0/2753598108 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9d6806aea0 0x7f9d6806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:21.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.673+0000 7f9d7a7fc640 1 -- 172.21.15.76:0/2753598108 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6610+0+0 (secure 0 0 0) 0x7f9d7408a1e0 con 0x7f9d70007390 2023-11-09T03:48:21.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.673+0000 7f9d897f3640 1 --2- 172.21.15.76:0/2753598108 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9d6806aea0 0x7f9d6806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:21.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.673+0000 7f9d897f3640 1 --2- 172.21.15.76:0/2753598108 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9d6806aea0 0x7f9d6806d360 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9d840b03a0 tx=0x7f9d800078f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:21.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.676+0000 7f9d7a7fc640 1 -- 172.21.15.76:0/2753598108 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9d74055090 con 0x7f9d70007390 2023-11-09T03:48:21.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:21.843+0000 7f9d8c258640 1 -- 172.21.15.76:0/2753598108 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9d840c9060 con 0x7f9d70007390 2023-11-09T03:48:22.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.540+0000 7f9d7a7fc640 1 -- 172.21.15.76:0/2753598108 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9d740586b0 con 0x7f9d70007390 2023-11-09T03:48:22.541 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' created 2023-11-09T03:48:22.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.543+0000 7f9d8c258640 1 -- 172.21.15.76:0/2753598108 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9d6806aea0 msgr2=0x7f9d6806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:22.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.543+0000 7f9d8c258640 1 --2- 172.21.15.76:0/2753598108 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9d6806aea0 0x7f9d6806d360 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9d840b03a0 tx=0x7f9d800078f0 comp rx=0 tx=0).stop 2023-11-09T03:48:22.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.543+0000 7f9d8c258640 1 -- 172.21.15.76:0/2753598108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d70007390 msgr2=0x7f9d841f26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:22.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.543+0000 7f9d8c258640 1 --2- 172.21.15.76:0/2753598108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d70007390 0x7f9d841f26b0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f9d74011900 tx=0x7f9d74011dc0 comp rx=0 tx=0).stop 2023-11-09T03:48:22.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.543+0000 7f9d8c258640 1 -- 172.21.15.76:0/2753598108 shutdown_connections 2023-11-09T03:48:22.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.543+0000 7f9d8c258640 1 --2- 172.21.15.76:0/2753598108 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9d6806aea0 0x7f9d6806d360 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:22.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.543+0000 7f9d8c258640 1 --2- 172.21.15.76:0/2753598108 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d8415cb50 0x7f9d841ee6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:22.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.543+0000 7f9d8c258640 1 --2- 172.21.15.76:0/2753598108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d84147020 0x7f9d8415c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:22.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.543+0000 7f9d8c258640 1 --2- 172.21.15.76:0/2753598108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d70007390 0x7f9d841f26b0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:22.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.543+0000 7f9d8c258640 1 -- 172.21.15.76:0/2753598108 >> 172.21.15.76:0/2753598108 conn(0x7f9d8406a120 msgr2=0x7f9d8415a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:22.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.544+0000 7f9d8c258640 1 -- 172.21.15.76:0/2753598108 shutdown_connections 2023-11-09T03:48:22.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.544+0000 7f9d8c258640 1 -- 172.21.15.76:0/2753598108 wait complete. 2023-11-09T03:48:22.573 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-09T03:48:22.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.678+0000 7fe74c4ec640 1 Processor -- start 2023-11-09T03:48:22.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.678+0000 7fe74c4ec640 1 -- start start 2023-11-09T03:48:22.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.679+0000 7fe74c4ec640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe7440cd6d0 0x7fe744161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:22.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.679+0000 7fe74c4ec640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7440cde60 con 0x7fe7440b4880 2023-11-09T03:48:22.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.679+0000 7fe74c4ec640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe744160f90 con 0x7fe744153e60 2023-11-09T03:48:22.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.679+0000 7fe74c4ec640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe744161960 con 0x7fe7440cd6d0 2023-11-09T03:48:22.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.679+0000 7fe74aa89640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe7440b4880 0x7fe7440d0fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59004/0 (socket says 172.21.15.76:59004) 2023-11-09T03:48:22.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.679+0000 7fe74aa89640 1 -- 172.21.15.76:0/1458464272 learned_addr learned my addr 172.21.15.76:0/1458464272 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:22.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.680+0000 7fe749286640 1 -- 172.21.15.76:0/1458464272 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 926747437 0 0) 0x7fe7440cde60 con 0x7fe7440b4880 2023-11-09T03:48:22.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.680+0000 7fe749a87640 1 --2- 172.21.15.76:0/1458464272 >> v2:172.21.15.113:3300/0 conn(0x7fe7440cd6d0 0x7fe744161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:22.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.680+0000 7fe749286640 1 -- 172.21.15.76:0/1458464272 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe730003520 con 0x7fe7440b4880 2023-11-09T03:48:22.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.680+0000 7fe749286640 1 -- 172.21.15.76:0/1458464272 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 760050061 0 0) 0x7fe730003520 con 0x7fe7440b4880 2023-11-09T03:48:22.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.680+0000 7fe749286640 1 -- 172.21.15.76:0/1458464272 >> v2:172.21.15.113:3300/0 conn(0x7fe7440cd6d0 msgr2=0x7fe744161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:22.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.680+0000 7fe749286640 1 --2- 172.21.15.76:0/1458464272 >> v2:172.21.15.113:3300/0 conn(0x7fe7440cd6d0 0x7fe744161420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:22.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.680+0000 7fe749286640 1 -- 172.21.15.76:0/1458464272 >> v1:172.21.15.113:6789/0 conn(0x7fe744153e60 legacy=0x7fe74415dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:22.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.680+0000 7fe749286640 1 -- 172.21.15.76:0/1458464272 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7440ce090 con 0x7fe7440b4880 2023-11-09T03:48:22.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.680+0000 7fe749286640 1 -- 172.21.15.76:0/1458464272 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fe73c0019f0 con 0x7fe7440b4880 2023-11-09T03:48:22.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.680+0000 7fe749286640 1 -- 172.21.15.76:0/1458464272 >> v1:172.21.15.76:6789/0 conn(0x7fe7440b4880 legacy=0x7fe7440d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:22.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.680+0000 7fe749286640 1 --2- 172.21.15.76:0/1458464272 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe730003970 0x7fe730003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:22.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.680+0000 7fe749286640 1 --2- 172.21.15.76:0/1458464272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7300047c0 0x7fe730004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:22.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.680+0000 7fe749286640 1 --2- 172.21.15.76:0/1458464272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe730005180 0x7fe73000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:22.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.680+0000 7fe749286640 1 -- 172.21.15.76:0/1458464272 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe744161960 con 0x7fe730005180 2023-11-09T03:48:22.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.680+0000 7fe749286640 1 -- 172.21.15.76:0/1458464272 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73000c020 con 0x7fe7300047c0 2023-11-09T03:48:22.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.680+0000 7fe749286640 1 -- 172.21.15.76:0/1458464272 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe73000c1e0 con 0x7fe730003970 2023-11-09T03:48:22.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.680+0000 7fe74aa89640 1 --2- 172.21.15.76:0/1458464272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe730005180 0x7fe73000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:22.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.680+0000 7fe749a87640 1 --2- 172.21.15.76:0/1458464272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7300047c0 0x7fe730004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:22.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.680+0000 7fe74a288640 1 --2- 172.21.15.76:0/1458464272 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe730003970 0x7fe730003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:22.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.680+0000 7fe74aa89640 1 -- 172.21.15.76:0/1458464272 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe730003970 msgr2=0x7fe730003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:22.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.680+0000 7fe74aa89640 1 --2- 172.21.15.76:0/1458464272 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe730003970 0x7fe730003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:22.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.680+0000 7fe74aa89640 1 -- 172.21.15.76:0/1458464272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7300047c0 msgr2=0x7fe730004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:22.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.680+0000 7fe74aa89640 1 --2- 172.21.15.76:0/1458464272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7300047c0 0x7fe730004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:22.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.680+0000 7fe74aa89640 1 -- 172.21.15.76:0/1458464272 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe730003520 con 0x7fe730005180 2023-11-09T03:48:22.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.680+0000 7fe749a87640 1 --2- 172.21.15.76:0/1458464272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7300047c0 0x7fe730004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:22.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.681+0000 7fe74aa89640 1 -- 172.21.15.76:0/1458464272 reap_dead start 2023-11-09T03:48:22.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.681+0000 7fe74aa89640 1 --2- 172.21.15.76:0/1458464272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe730005180 0x7fe73000b980 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fe73c00a830 tx=0x7fe7300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9cd67887191b1d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:22.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.681+0000 7fe749286640 1 -- 172.21.15.76:0/1458464272 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73c0072d0 con 0x7fe730005180 2023-11-09T03:48:22.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.681+0000 7fe749286640 1 -- 172.21.15.76:0/1458464272 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73c02f070 con 0x7fe730005180 2023-11-09T03:48:22.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.681+0000 7fe749286640 1 -- 172.21.15.76:0/1458464272 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73c0076b0 con 0x7fe730005180 2023-11-09T03:48:22.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.682+0000 7fe74c4ec640 1 -- 172.21.15.76:0/1458464272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe730005180 msgr2=0x7fe73000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:22.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.682+0000 7fe74c4ec640 1 --2- 172.21.15.76:0/1458464272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe730005180 0x7fe73000b980 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fe73c00a830 tx=0x7fe7300057f0 comp rx=0 tx=0).stop 2023-11-09T03:48:22.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.682+0000 7fe74c4ec640 1 -- 172.21.15.76:0/1458464272 shutdown_connections 2023-11-09T03:48:22.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.682+0000 7fe74c4ec640 1 --2- 172.21.15.76:0/1458464272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe730005180 0x7fe73000b980 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:22.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.682+0000 7fe74c4ec640 1 -- 172.21.15.76:0/1458464272 >> 172.21.15.76:0/1458464272 conn(0x7fe7440cf880 msgr2=0x7fe7440cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:22.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.682+0000 7fe74c4ec640 1 -- 172.21.15.76:0/1458464272 shutdown_connections 2023-11-09T03:48:22.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.682+0000 7fe74c4ec640 1 -- 172.21.15.76:0/1458464272 wait complete. 2023-11-09T03:48:22.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.683+0000 7fe74c4ec640 1 Processor -- start 2023-11-09T03:48:22.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.683+0000 7fe74c4ec640 1 -- start start 2023-11-09T03:48:22.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.684+0000 7fe74c4ec640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe730005180 0x7fe7440c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:22.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.684+0000 7fe74c4ec640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe744148650 0x7fe7441f2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:22.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.684+0000 7fe74c4ec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe744154e80 0x7fe7440c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:22.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.684+0000 7fe74c4ec640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe744146be0 con 0x7fe744154e80 2023-11-09T03:48:22.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.684+0000 7fe74c4ec640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7441f2680 con 0x7fe730005180 2023-11-09T03:48:22.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.684+0000 7fe74c4ec640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe744155660 con 0x7fe744148650 2023-11-09T03:48:22.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.684+0000 7fe74aa89640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe744154e80 0x7fe7440c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:22.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.684+0000 7fe74aa89640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe744154e80 0x7fe7440c8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37680/0 (socket says 172.21.15.76:37680) 2023-11-09T03:48:22.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.684+0000 7fe74aa89640 1 -- 172.21.15.76:0/3957481736 learned_addr learned my addr 172.21.15.76:0/3957481736 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:22.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.684+0000 7fe74aa89640 1 -- 172.21.15.76:0/3957481736 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe744148650 msgr2=0x7fe7441f2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:22.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.684+0000 7fe74aa89640 1 --2- 172.21.15.76:0/3957481736 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe744148650 0x7fe7441f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:22.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.684+0000 7fe74aa89640 1 -- 172.21.15.76:0/3957481736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe730005180 msgr2=0x7fe7440c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:22.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.684+0000 7fe74aa89640 1 --2- 172.21.15.76:0/3957481736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe730005180 0x7fe7440c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:22.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.684+0000 7fe74aa89640 1 -- 172.21.15.76:0/3957481736 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe744155380 con 0x7fe744154e80 2023-11-09T03:48:22.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.684+0000 7fe74aa89640 1 --2- 172.21.15.76:0/3957481736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe744154e80 0x7fe7440c8ad0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fe73c00a950 tx=0x7fe73c001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:22.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.684+0000 7fe73affd640 1 -- 172.21.15.76:0/3957481736 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73c0019b0 con 0x7fe744154e80 2023-11-09T03:48:22.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.684+0000 7fe73affd640 1 -- 172.21.15.76:0/3957481736 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73c004b60 con 0x7fe744154e80 2023-11-09T03:48:22.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.685+0000 7fe73affd640 1 -- 172.21.15.76:0/3957481736 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73c007400 con 0x7fe744154e80 2023-11-09T03:48:22.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.685+0000 7fe74c4ec640 1 -- 172.21.15.76:0/3957481736 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe744146590 con 0x7fe744154e80 2023-11-09T03:48:22.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.685+0000 7fe74c4ec640 1 -- 172.21.15.76:0/3957481736 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe744147fb0 con 0x7fe744154e80 2023-11-09T03:48:22.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.686+0000 7fe74c4ec640 1 -- 172.21.15.76:0/3957481736 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7441463f0 con 0x7fe744154e80 2023-11-09T03:48:22.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.686+0000 7fe73affd640 1 -- 172.21.15.76:0/3957481736 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe73c031030 con 0x7fe744154e80 2023-11-09T03:48:22.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.687+0000 7fe73affd640 1 --2- 172.21.15.76:0/3957481736 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe72806ae70 0x7fe72806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:22.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.687+0000 7fe73affd640 1 -- 172.21.15.76:0/3957481736 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6992+0+0 (secure 0 0 0) 0x7fe73c0aad20 con 0x7fe744154e80 2023-11-09T03:48:22.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.687+0000 7fe74a288640 1 --2- 172.21.15.76:0/3957481736 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe72806ae70 0x7fe72806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:22.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.687+0000 7fe74a288640 1 --2- 172.21.15.76:0/3957481736 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe72806ae70 0x7fe72806d330 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe7400014f0 tx=0x7fe740009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:22.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.691+0000 7fe73affd640 1 -- 172.21.15.76:0/3957481736 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe7441463f0 con 0x7fe744154e80 2023-11-09T03:48:22.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:22.892+0000 7fe74c4ec640 1 -- 172.21.15.76:0/3957481736 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe7441463f0 con 0x7fe744154e80 2023-11-09T03:48:23.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.548+0000 7fe73affd640 1 -- 172.21.15.76:0/3957481736 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v33) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe7441463f0 con 0x7fe744154e80 2023-11-09T03:48:23.550 INFO:teuthology.orchestra.run.smithi076.stderr:new fs with metadata pool 14 and data pool 15 2023-11-09T03:48:23.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.563+0000 7fe74c4ec640 1 -- 172.21.15.76:0/3957481736 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe72806ae70 msgr2=0x7fe72806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:23.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.564+0000 7fe74c4ec640 1 --2- 172.21.15.76:0/3957481736 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe72806ae70 0x7fe72806d330 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe7400014f0 tx=0x7fe740009040 comp rx=0 tx=0).stop 2023-11-09T03:48:23.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.564+0000 7fe74c4ec640 1 -- 172.21.15.76:0/3957481736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe744154e80 msgr2=0x7fe7440c8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:23.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.564+0000 7fe74c4ec640 1 --2- 172.21.15.76:0/3957481736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe744154e80 0x7fe7440c8ad0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fe73c00a950 tx=0x7fe73c001d80 comp rx=0 tx=0).stop 2023-11-09T03:48:23.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.564+0000 7fe74c4ec640 1 -- 172.21.15.76:0/3957481736 shutdown_connections 2023-11-09T03:48:23.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.564+0000 7fe74c4ec640 1 --2- 172.21.15.76:0/3957481736 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe72806ae70 0x7fe72806d330 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:23.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.564+0000 7fe74c4ec640 1 --2- 172.21.15.76:0/3957481736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe744154e80 0x7fe7440c8ad0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:23.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.565+0000 7fe74c4ec640 1 --2- 172.21.15.76:0/3957481736 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe744148650 0x7fe7441f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:23.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.565+0000 7fe74c4ec640 1 --2- 172.21.15.76:0/3957481736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe730005180 0x7fe7440c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:23.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.565+0000 7fe74c4ec640 1 -- 172.21.15.76:0/3957481736 >> 172.21.15.76:0/3957481736 conn(0x7fe7440cf880 msgr2=0x7fe7440b7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:23.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.565+0000 7fe74c4ec640 1 -- 172.21.15.76:0/3957481736 shutdown_connections 2023-11-09T03:48:23.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.565+0000 7fe74c4ec640 1 -- 172.21.15.76:0/3957481736 wait complete. 2023-11-09T03:48:23.585 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:48:23.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.687+0000 7f9e79efc640 1 Processor -- start 2023-11-09T03:48:23.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.687+0000 7f9e79efc640 1 -- start start 2023-11-09T03:48:23.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.688+0000 7f9e79efc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9e740ce370 0x7f9e7406db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:23.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.688+0000 7f9e79efc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e7406e060 con 0x7f9e74159c70 2023-11-09T03:48:23.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.688+0000 7f9e79efc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e740b4110 con 0x7f9e740b0ae0 2023-11-09T03:48:23.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.688+0000 7f9e79efc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e740b49b0 con 0x7f9e740ce370 2023-11-09T03:48:23.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.688+0000 7f9e73fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9e740ce370 0x7f9e7406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:23.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.688+0000 7f9e737fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f9e74159c70 0x7f9e740b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59020/0 (socket says 172.21.15.76:59020) 2023-11-09T03:48:23.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.688+0000 7f9e737fe640 1 -- 172.21.15.76:0/1905825332 learned_addr learned my addr 172.21.15.76:0/1905825332 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:23.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.688+0000 7f9e73fff640 1 -- 172.21.15.76:0/1905825332 >> v1:172.21.15.113:6789/0 conn(0x7f9e740b0ae0 legacy=0x7f9e7415d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:23.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.688+0000 7f9e73fff640 1 -- 172.21.15.76:0/1905825332 >> v1:172.21.15.76:6789/0 conn(0x7f9e74159c70 legacy=0x7f9e740b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:23.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.688+0000 7f9e73fff640 1 -- 172.21.15.76:0/1905825332 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e740b43f0 con 0x7f9e740ce370 2023-11-09T03:48:23.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.688+0000 7f9e727fc640 1 -- 172.21.15.76:0/1905825332 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1158734772 0 0) 0x7f9e7406e060 con 0x7f9e74159c70 2023-11-09T03:48:23.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.689+0000 7f9e73fff640 1 --2- 172.21.15.76:0/1905825332 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e740ce370 0x7f9e7406db20 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f9e7406e1f0 tx=0x7f9e6c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8ddb6484479e81 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:23.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.689+0000 7f9e727fc640 1 -- 172.21.15.76:0/1905825332 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c02cc50 con 0x7f9e740ce370 2023-11-09T03:48:23.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.689+0000 7f9e727fc640 1 -- 172.21.15.76:0/1905825332 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6c02cdb0 con 0x7f9e740ce370 2023-11-09T03:48:23.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.689+0000 7f9e727fc640 1 -- 172.21.15.76:0/1905825332 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c034660 con 0x7f9e740ce370 2023-11-09T03:48:23.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.690+0000 7f9e79efc640 1 -- 172.21.15.76:0/1905825332 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e740ce370 msgr2=0x7f9e7406db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:23.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.690+0000 7f9e79efc640 1 --2- 172.21.15.76:0/1905825332 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e740ce370 0x7f9e7406db20 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f9e7406e1f0 tx=0x7f9e6c02bea0 comp rx=0 tx=0).stop 2023-11-09T03:48:23.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.690+0000 7f9e79efc640 1 -- 172.21.15.76:0/1905825332 shutdown_connections 2023-11-09T03:48:23.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.690+0000 7f9e79efc640 1 --2- 172.21.15.76:0/1905825332 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e740ce370 0x7f9e7406db20 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:23.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.690+0000 7f9e79efc640 1 -- 172.21.15.76:0/1905825332 >> 172.21.15.76:0/1905825332 conn(0x7f9e740ce950 msgr2=0x7f9e740ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:23.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.690+0000 7f9e79efc640 1 -- 172.21.15.76:0/1905825332 shutdown_connections 2023-11-09T03:48:23.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.690+0000 7f9e79efc640 1 -- 172.21.15.76:0/1905825332 wait complete. 2023-11-09T03:48:23.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.691+0000 7f9e79efc640 1 Processor -- start 2023-11-09T03:48:23.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.691+0000 7f9e79efc640 1 -- start start 2023-11-09T03:48:23.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.692+0000 7f9e79efc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e740b0ae0 0x7f9e741439b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:23.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.692+0000 7f9e79efc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e740ce370 0x7f9e74143ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:23.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.692+0000 7f9e79efc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9e74159c70 0x7f9e74146760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:23.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.692+0000 7f9e79efc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e740ce7e0 con 0x7f9e740b0ae0 2023-11-09T03:48:23.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.692+0000 7f9e79efc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e74144430 con 0x7f9e740ce370 2023-11-09T03:48:23.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.692+0000 7f9e79efc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7414c3e0 con 0x7f9e74159c70 2023-11-09T03:48:23.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.692+0000 7f9e737fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e740b0ae0 0x7f9e741439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:23.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.692+0000 7f9e737fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e740b0ae0 0x7f9e741439b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37708/0 (socket says 172.21.15.76:37708) 2023-11-09T03:48:23.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.692+0000 7f9e737fe640 1 -- 172.21.15.76:0/1121213078 learned_addr learned my addr 172.21.15.76:0/1121213078 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:23.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.692+0000 7f9e72ffd640 1 --2- 172.21.15.76:0/1121213078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e740ce370 0x7f9e74143ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:23.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.692+0000 7f9e73fff640 1 --2- 172.21.15.76:0/1121213078 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9e74159c70 0x7f9e74146760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:23.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.692+0000 7f9e737fe640 1 -- 172.21.15.76:0/1121213078 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9e74159c70 msgr2=0x7f9e74146760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:23.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.692+0000 7f9e737fe640 1 --2- 172.21.15.76:0/1121213078 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9e74159c70 0x7f9e74146760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:23.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.692+0000 7f9e737fe640 1 -- 172.21.15.76:0/1121213078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e740ce370 msgr2=0x7f9e74143ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:23.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.692+0000 7f9e737fe640 1 --2- 172.21.15.76:0/1121213078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e740ce370 0x7f9e74143ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:23.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.692+0000 7f9e737fe640 1 -- 172.21.15.76:0/1121213078 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6c009a70 con 0x7f9e740b0ae0 2023-11-09T03:48:23.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.692+0000 7f9e737fe640 1 --2- 172.21.15.76:0/1121213078 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e740b0ae0 0x7f9e741439b0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f9e6400c460 tx=0x7f9e6400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:23.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.693+0000 7f9e5bfff640 1 -- 172.21.15.76:0/1121213078 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e640074e0 con 0x7f9e740b0ae0 2023-11-09T03:48:23.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.693+0000 7f9e5bfff640 1 -- 172.21.15.76:0/1121213078 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e64007660 con 0x7f9e740b0ae0 2023-11-09T03:48:23.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.693+0000 7f9e5bfff640 1 -- 172.21.15.76:0/1121213078 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e64004a20 con 0x7f9e740b0ae0 2023-11-09T03:48:23.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.693+0000 7f9e79efc640 1 -- 172.21.15.76:0/1121213078 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e7414c6c0 con 0x7f9e740b0ae0 2023-11-09T03:48:23.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.693+0000 7f9e79efc640 1 -- 172.21.15.76:0/1121213078 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e74147ff0 con 0x7f9e740b0ae0 2023-11-09T03:48:23.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.694+0000 7f9e79efc640 1 -- 172.21.15.76:0/1121213078 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e74074290 con 0x7f9e740b0ae0 2023-11-09T03:48:23.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.694+0000 7f9e5bfff640 1 -- 172.21.15.76:0/1121213078 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9e64004b80 con 0x7f9e740b0ae0 2023-11-09T03:48:23.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.694+0000 7f9e5bfff640 1 --2- 172.21.15.76:0/1121213078 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9e6006ae70 0x7f9e6006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:23.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.694+0000 7f9e5bfff640 1 -- 172.21.15.76:0/1121213078 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f9e64089910 con 0x7f9e740b0ae0 2023-11-09T03:48:23.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.695+0000 7f9e72ffd640 1 --2- 172.21.15.76:0/1121213078 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9e6006ae70 0x7f9e6006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:23.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.695+0000 7f9e72ffd640 1 --2- 172.21.15.76:0/1121213078 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9e6006ae70 0x7f9e6006d330 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9e7413fbc0 tx=0x7f9e68009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:23.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.699+0000 7f9e5bfff640 1 -- 172.21.15.76:0/1121213078 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9e64054450 con 0x7f9e740b0ae0 2023-11-09T03:48:23.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.868+0000 7f9e79efc640 1 -- 172.21.15.76:0/1121213078 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e7406d3f0 con 0x7f9e740b0ae0 2023-11-09T03:48:23.869 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.869+0000 7f9e5bfff640 1 -- 172.21.15.76:0/1121213078 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+21954 (secure 0 0 0) 0x7f9e64057a70 con 0x7f9e740b0ae0 2023-11-09T03:48:23.870 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:23.870 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":82,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:48:22.894700+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-09T03:47:59.280601+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-09T03:47:59.742039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:48:20.992533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-11-09T03:48:21.862178+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:48:23.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.871+0000 7f9e79efc640 1 -- 172.21.15.76:0/1121213078 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9e6006ae70 msgr2=0x7f9e6006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:23.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.871+0000 7f9e79efc640 1 --2- 172.21.15.76:0/1121213078 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9e6006ae70 0x7f9e6006d330 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9e7413fbc0 tx=0x7f9e68009040 comp rx=0 tx=0).stop 2023-11-09T03:48:23.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.871+0000 7f9e79efc640 1 -- 172.21.15.76:0/1121213078 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e740b0ae0 msgr2=0x7f9e741439b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:23.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.871+0000 7f9e79efc640 1 --2- 172.21.15.76:0/1121213078 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e740b0ae0 0x7f9e741439b0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f9e6400c460 tx=0x7f9e6400c920 comp rx=0 tx=0).stop 2023-11-09T03:48:23.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.871+0000 7f9e79efc640 1 -- 172.21.15.76:0/1121213078 shutdown_connections 2023-11-09T03:48:23.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.871+0000 7f9e79efc640 1 --2- 172.21.15.76:0/1121213078 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9e6006ae70 0x7f9e6006d330 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:23.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.871+0000 7f9e79efc640 1 --2- 172.21.15.76:0/1121213078 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9e74159c70 0x7f9e74146760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:23.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.871+0000 7f9e79efc640 1 --2- 172.21.15.76:0/1121213078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e740ce370 0x7f9e74143ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:23.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.871+0000 7f9e79efc640 1 --2- 172.21.15.76:0/1121213078 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e740b0ae0 0x7f9e741439b0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:23.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.871+0000 7f9e79efc640 1 -- 172.21.15.76:0/1121213078 >> 172.21.15.76:0/1121213078 conn(0x7f9e740ce950 msgr2=0x7f9e740b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:23.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.871+0000 7f9e79efc640 1 -- 172.21.15.76:0/1121213078 shutdown_connections 2023-11-09T03:48:23.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.871+0000 7f9e79efc640 1 -- 172.21.15.76:0/1121213078 wait complete. 2023-11-09T03:48:23.883 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:48:23.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.983+0000 7f50dbbb6640 1 Processor -- start 2023-11-09T03:48:23.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.983+0000 7f50dbbb6640 1 -- start start 2023-11-09T03:48:23.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.984+0000 7f50dbbb6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f50d415aa30 0x7f50d40c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:23.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.984+0000 7f50dbbb6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f50d40ca280 0x7f50d406c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:23.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.984+0000 7f50dbbb6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d415bc30 con 0x7f50d406cc30 2023-11-09T03:48:23.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.984+0000 7f50dbbb6640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50d406d080 con 0x7f50d40ca280 2023-11-09T03:48:23.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.984+0000 7f50dbbb6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50d415bf10 con 0x7f50d415aa30 2023-11-09T03:48:23.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.984+0000 7f50d9151640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f50d40ca280 0x7f50d406c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:23.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.984+0000 7f50d9151640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f50d40ca280 0x7f50d406c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37714/0 (socket says 172.21.15.76:37714) 2023-11-09T03:48:23.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.984+0000 7f50d9151640 1 -- 172.21.15.76:0/376348479 learned_addr learned my addr 172.21.15.76:0/376348479 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:23.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.984+0000 7f50d9952640 1 --2- 172.21.15.76:0/376348479 >> v2:172.21.15.113:3300/0 conn(0x7f50d415aa30 0x7f50d40c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:23.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.984+0000 7f50d9151640 1 -- 172.21.15.76:0/376348479 >> v2:172.21.15.113:3300/0 conn(0x7f50d415aa30 msgr2=0x7f50d40c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:23.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.984+0000 7f50d9151640 1 --2- 172.21.15.76:0/376348479 >> v2:172.21.15.113:3300/0 conn(0x7f50d415aa30 0x7f50d40c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:23.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.984+0000 7f50d9151640 1 -- 172.21.15.76:0/376348479 >> v1:172.21.15.113:6789/0 conn(0x7f50d406cc30 legacy=0x7f50d40b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:23.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.984+0000 7f50d9151640 1 -- 172.21.15.76:0/376348479 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d40c9540 con 0x7f50d40ca280 2023-11-09T03:48:23.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.985+0000 7f50d9151640 1 --2- 172.21.15.76:0/376348479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50d40ca280 0x7f50d406c6f0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f50d40cc780 tx=0x7f50c402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451b14f5d58efcf2 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:23.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.985+0000 7f50d8950640 1 -- 172.21.15.76:0/376348479 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c4030860 con 0x7f50d40ca280 2023-11-09T03:48:23.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.985+0000 7f50d8950640 1 -- 172.21.15.76:0/376348479 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c40309c0 con 0x7f50d40ca280 2023-11-09T03:48:23.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.985+0000 7f50d8950640 1 -- 172.21.15.76:0/376348479 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c4030d50 con 0x7f50d40ca280 2023-11-09T03:48:23.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.986+0000 7f50dbbb6640 1 -- 172.21.15.76:0/376348479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50d40ca280 msgr2=0x7f50d406c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:23.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.986+0000 7f50dbbb6640 1 --2- 172.21.15.76:0/376348479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50d40ca280 0x7f50d406c6f0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f50d40cc780 tx=0x7f50c402b9a0 comp rx=0 tx=0).stop 2023-11-09T03:48:23.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.986+0000 7f50dbbb6640 1 -- 172.21.15.76:0/376348479 shutdown_connections 2023-11-09T03:48:23.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.986+0000 7f50dbbb6640 1 --2- 172.21.15.76:0/376348479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50d40ca280 0x7f50d406c6f0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:23.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.986+0000 7f50dbbb6640 1 --2- 172.21.15.76:0/376348479 >> v2:172.21.15.113:3300/0 conn(0x7f50d415aa30 0x7f50d40c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:23.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.986+0000 7f50dbbb6640 1 -- 172.21.15.76:0/376348479 >> 172.21.15.76:0/376348479 conn(0x7f50d40cb7f0 msgr2=0x7f50d40cbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:23.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.986+0000 7f50dbbb6640 1 -- 172.21.15.76:0/376348479 shutdown_connections 2023-11-09T03:48:23.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.986+0000 7f50dbbb6640 1 -- 172.21.15.76:0/376348479 wait complete. 2023-11-09T03:48:23.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.987+0000 7f50dbbb6640 1 Processor -- start 2023-11-09T03:48:23.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.987+0000 7f50dbbb6640 1 -- start start 2023-11-09T03:48:23.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.987+0000 7f50dbbb6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50d406cc30 0x7f50d40c28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:23.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.987+0000 7f50dbbb6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50d415aa30 0x7f50d40c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:23.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.987+0000 7f50dbbb6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50d40c6c00 0x7f50d41556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:23.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.987+0000 7f50dbbb6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d40af4c0 con 0x7f50d40c6c00 2023-11-09T03:48:23.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.988+0000 7f50dbbb6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d41511e0 con 0x7f50d415aa30 2023-11-09T03:48:23.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.988+0000 7f50dbbb6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d40c2de0 con 0x7f50d406cc30 2023-11-09T03:48:23.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.988+0000 7f50da153640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50d40c6c00 0x7f50d41556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:23.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.988+0000 7f50da153640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50d40c6c00 0x7f50d41556a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37716/0 (socket says 172.21.15.76:37716) 2023-11-09T03:48:23.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.988+0000 7f50da153640 1 -- 172.21.15.76:0/4245146488 learned_addr learned my addr 172.21.15.76:0/4245146488 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:23.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.988+0000 7f50d9151640 1 --2- 172.21.15.76:0/4245146488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50d415aa30 0x7f50d40c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:23.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.988+0000 7f50d9952640 1 --2- 172.21.15.76:0/4245146488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50d406cc30 0x7f50d40c28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:23.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.988+0000 7f50da153640 1 -- 172.21.15.76:0/4245146488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50d406cc30 msgr2=0x7f50d40c28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:23.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.988+0000 7f50da153640 1 --2- 172.21.15.76:0/4245146488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50d406cc30 0x7f50d40c28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:23.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.988+0000 7f50da153640 1 -- 172.21.15.76:0/4245146488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50d415aa30 msgr2=0x7f50d40c66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:23.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.988+0000 7f50da153640 1 --2- 172.21.15.76:0/4245146488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50d415aa30 0x7f50d40c66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:23.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.988+0000 7f50da153640 1 -- 172.21.15.76:0/4245146488 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c40094d0 con 0x7f50d40c6c00 2023-11-09T03:48:23.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.988+0000 7f50d9151640 1 --2- 172.21.15.76:0/4245146488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50d415aa30 0x7f50d40c66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:23.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.988+0000 7f50d9952640 1 --2- 172.21.15.76:0/4245146488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50d406cc30 0x7f50d40c28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:23.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.988+0000 7f50da153640 1 --2- 172.21.15.76:0/4245146488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50d40c6c00 0x7f50d41556a0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00c760 tx=0x7f50cc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:23.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.988+0000 7f50ca7fc640 1 -- 172.21.15.76:0/4245146488 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc003b70 con 0x7f50d40c6c00 2023-11-09T03:48:23.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.988+0000 7f50ca7fc640 1 -- 172.21.15.76:0/4245146488 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50cc003cf0 con 0x7f50d40c6c00 2023-11-09T03:48:23.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.988+0000 7f50dbbb6640 1 -- 172.21.15.76:0/4245146488 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d40ae840 con 0x7f50d40c6c00 2023-11-09T03:48:23.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.988+0000 7f50ca7fc640 1 -- 172.21.15.76:0/4245146488 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc00aa30 con 0x7f50d40c6c00 2023-11-09T03:48:23.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.988+0000 7f50dbbb6640 1 -- 172.21.15.76:0/4245146488 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d40aee00 con 0x7f50d40c6c00 2023-11-09T03:48:23.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.989+0000 7f50dbbb6640 1 -- 172.21.15.76:0/4245146488 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d40af600 con 0x7f50d40c6c00 2023-11-09T03:48:23.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.990+0000 7f50ca7fc640 1 -- 172.21.15.76:0/4245146488 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f50cc00ab90 con 0x7f50d40c6c00 2023-11-09T03:48:23.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.990+0000 7f50ca7fc640 1 --2- 172.21.15.76:0/4245146488 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f50b806ae70 0x7f50b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:23.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.990+0000 7f50ca7fc640 1 -- 172.21.15.76:0/4245146488 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f50cc089830 con 0x7f50d40c6c00 2023-11-09T03:48:23.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.991+0000 7f50d9952640 1 --2- 172.21.15.76:0/4245146488 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f50b806ae70 0x7f50b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:23.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.991+0000 7f50d9952640 1 --2- 172.21.15.76:0/4245146488 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f50b806ae70 0x7f50b806d330 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f50d0009a20 tx=0x7f50d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:23.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:23.995+0000 7f50ca7fc640 1 -- 172.21.15.76:0/4245146488 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f50d40af600 con 0x7f50d40c6c00 2023-11-09T03:48:24.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.163+0000 7f50dbbb6640 1 -- 172.21.15.76:0/4245146488 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50d40af600 con 0x7f50d40c6c00 2023-11-09T03:48:24.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.164+0000 7f50ca7fc640 1 -- 172.21.15.76:0/4245146488 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+21954 (secure 0 0 0) 0x7f50d40af600 con 0x7f50d40c6c00 2023-11-09T03:48:24.165 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:24.165 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":82,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:48:22.894700+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-09T03:47:59.280601+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-09T03:47:59.742039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:48:20.992533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-11-09T03:48:21.862178+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:48:24.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.166+0000 7f50dbbb6640 1 -- 172.21.15.76:0/4245146488 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f50b806ae70 msgr2=0x7f50b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:24.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.166+0000 7f50dbbb6640 1 --2- 172.21.15.76:0/4245146488 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f50b806ae70 0x7f50b806d330 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f50d0009a20 tx=0x7f50d0008040 comp rx=0 tx=0).stop 2023-11-09T03:48:24.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.166+0000 7f50dbbb6640 1 -- 172.21.15.76:0/4245146488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50d40c6c00 msgr2=0x7f50d41556a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:24.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.166+0000 7f50dbbb6640 1 --2- 172.21.15.76:0/4245146488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50d40c6c00 0x7f50d41556a0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00c760 tx=0x7f50cc00cc20 comp rx=0 tx=0).stop 2023-11-09T03:48:24.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.166+0000 7f50dbbb6640 1 -- 172.21.15.76:0/4245146488 shutdown_connections 2023-11-09T03:48:24.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.166+0000 7f50dbbb6640 1 --2- 172.21.15.76:0/4245146488 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f50b806ae70 0x7f50b806d330 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:24.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.166+0000 7f50dbbb6640 1 --2- 172.21.15.76:0/4245146488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50d40c6c00 0x7f50d41556a0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:24.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.166+0000 7f50dbbb6640 1 --2- 172.21.15.76:0/4245146488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50d415aa30 0x7f50d40c66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:24.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.166+0000 7f50dbbb6640 1 --2- 172.21.15.76:0/4245146488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50d406cc30 0x7f50d40c28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:24.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.166+0000 7f50dbbb6640 1 -- 172.21.15.76:0/4245146488 >> 172.21.15.76:0/4245146488 conn(0x7f50d40cb7f0 msgr2=0x7f50d415c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:24.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.166+0000 7f50dbbb6640 1 -- 172.21.15.76:0/4245146488 shutdown_connections 2023-11-09T03:48:24.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.166+0000 7f50dbbb6640 1 -- 172.21.15.76:0/4245146488 wait complete. 2023-11-09T03:48:24.178 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-09T03:48:24.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.280+0000 7f43719d5640 1 Processor -- start 2023-11-09T03:48:24.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.280+0000 7f43719d5640 1 -- start start 2023-11-09T03:48:24.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.280+0000 7f43719d5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f436c153e60 0x7f436c15dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:24.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.280+0000 7f43719d5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f436c0cd6d0 0x7f436c161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:24.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.280+0000 7f43719d5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f436c0b4880 0x7f436c0b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:24.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.280+0000 7f43719d5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f436c0cde60 con 0x7f436c153e60 2023-11-09T03:48:24.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.280+0000 7f43719d5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f436c0b5220 con 0x7f436c0cd6d0 2023-11-09T03:48:24.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.280+0000 7f43719d5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436c160f90 con 0x7f436c0b4880 2023-11-09T03:48:24.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.281+0000 7f436affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f436c153e60 0x7f436c15dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:24.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.281+0000 7f436affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f436c153e60 0x7f436c15dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37726/0 (socket says 172.21.15.76:37726) 2023-11-09T03:48:24.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.281+0000 7f436affd640 1 -- 172.21.15.76:0/2648710168 learned_addr learned my addr 172.21.15.76:0/2648710168 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:24.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.281+0000 7f436a7fc640 1 --2- 172.21.15.76:0/2648710168 >> v2:172.21.15.113:3300/0 conn(0x7f436c0cd6d0 0x7f436c161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:24.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.281+0000 7f436b7fe640 1 --2- 172.21.15.76:0/2648710168 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f436c0b4880 0x7f436c0b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:24.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.281+0000 7f436affd640 1 -- 172.21.15.76:0/2648710168 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f436c0b4880 msgr2=0x7f436c0b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:24.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.281+0000 7f436affd640 1 --2- 172.21.15.76:0/2648710168 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f436c0b4880 0x7f436c0b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:24.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.281+0000 7f436affd640 1 -- 172.21.15.76:0/2648710168 >> v2:172.21.15.113:3300/0 conn(0x7f436c0cd6d0 msgr2=0x7f436c161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:24.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.281+0000 7f436affd640 1 --2- 172.21.15.76:0/2648710168 >> v2:172.21.15.113:3300/0 conn(0x7f436c0cd6d0 0x7f436c161420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:24.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.281+0000 7f436affd640 1 -- 172.21.15.76:0/2648710168 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f436c15e400 con 0x7f436c153e60 2023-11-09T03:48:24.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.282+0000 7f436affd640 1 --2- 172.21.15.76:0/2648710168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f436c153e60 0x7f436c15dec0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f436c0c9e80 tx=0x7f435c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736e3419c2861c2a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:24.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.282+0000 7f4369ffb640 1 -- 172.21.15.76:0/2648710168 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c031640 con 0x7f436c153e60 2023-11-09T03:48:24.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.282+0000 7f4369ffb640 1 -- 172.21.15.76:0/2648710168 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f435c0317a0 con 0x7f436c153e60 2023-11-09T03:48:24.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.282+0000 7f4369ffb640 1 -- 172.21.15.76:0/2648710168 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c031b30 con 0x7f436c153e60 2023-11-09T03:48:24.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.282+0000 7f43719d5640 1 -- 172.21.15.76:0/2648710168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f436c153e60 msgr2=0x7f436c15dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:24.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.282+0000 7f43719d5640 1 --2- 172.21.15.76:0/2648710168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f436c153e60 0x7f436c15dec0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f436c0c9e80 tx=0x7f435c02b8f0 comp rx=0 tx=0).stop 2023-11-09T03:48:24.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.282+0000 7f43719d5640 1 -- 172.21.15.76:0/2648710168 shutdown_connections 2023-11-09T03:48:24.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.282+0000 7f43719d5640 1 --2- 172.21.15.76:0/2648710168 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f436c0b4880 0x7f436c0b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:24.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.282+0000 7f43719d5640 1 --2- 172.21.15.76:0/2648710168 >> v2:172.21.15.113:3300/0 conn(0x7f436c0cd6d0 0x7f436c161420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:24.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.282+0000 7f43719d5640 1 --2- 172.21.15.76:0/2648710168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f436c153e60 0x7f436c15dec0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:24.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.282+0000 7f43719d5640 1 -- 172.21.15.76:0/2648710168 >> 172.21.15.76:0/2648710168 conn(0x7f436c0cf880 msgr2=0x7f436c0cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:24.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.282+0000 7f43719d5640 1 -- 172.21.15.76:0/2648710168 shutdown_connections 2023-11-09T03:48:24.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.282+0000 7f43719d5640 1 -- 172.21.15.76:0/2648710168 wait complete. 2023-11-09T03:48:24.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.284+0000 7f43719d5640 1 Processor -- start 2023-11-09T03:48:24.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.284+0000 7f43719d5640 1 -- start start 2023-11-09T03:48:24.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.284+0000 7f43719d5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f436c0b4880 0x7f436c144b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:24.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.284+0000 7f43719d5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f436c0cd6d0 0x7f436c0cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:24.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.284+0000 7f43719d5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f436c153e60 0x7f436c155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:24.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.284+0000 7f43719d5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c1450d0 con 0x7f436c0b4880 2023-11-09T03:48:24.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.284+0000 7f43719d5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c1424b0 con 0x7f436c0cd6d0 2023-11-09T03:48:24.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.284+0000 7f43719d5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436c142790 con 0x7f436c153e60 2023-11-09T03:48:24.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.285+0000 7f436affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f436c0b4880 0x7f436c144b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:24.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.285+0000 7f436affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f436c0b4880 0x7f436c144b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37742/0 (socket says 172.21.15.76:37742) 2023-11-09T03:48:24.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.285+0000 7f436affd640 1 -- 172.21.15.76:0/1296357149 learned_addr learned my addr 172.21.15.76:0/1296357149 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:24.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.285+0000 7f436a7fc640 1 --2- 172.21.15.76:0/1296357149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f436c0cd6d0 0x7f436c0cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:24.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.285+0000 7f436b7fe640 1 --2- 172.21.15.76:0/1296357149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f436c153e60 0x7f436c155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:24.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.285+0000 7f436affd640 1 -- 172.21.15.76:0/1296357149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f436c153e60 msgr2=0x7f436c155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:24.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.285+0000 7f436affd640 1 --2- 172.21.15.76:0/1296357149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f436c153e60 0x7f436c155290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:24.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.285+0000 7f436affd640 1 -- 172.21.15.76:0/1296357149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f436c0cd6d0 msgr2=0x7f436c0cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:24.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.285+0000 7f436affd640 1 --2- 172.21.15.76:0/1296357149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f436c0cd6d0 0x7f436c0cc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:24.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.285+0000 7f436affd640 1 -- 172.21.15.76:0/1296357149 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f435c0094d0 con 0x7f436c0b4880 2023-11-09T03:48:24.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.285+0000 7f436b7fe640 1 --2- 172.21.15.76:0/1296357149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f436c153e60 0x7f436c155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:24.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.285+0000 7f436affd640 1 --2- 172.21.15.76:0/1296357149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f436c0b4880 0x7f436c144b90 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f435c000c00 tx=0x7f435c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:24.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.285+0000 7f4353fff640 1 -- 172.21.15.76:0/1296357149 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c03ac60 con 0x7f436c0b4880 2023-11-09T03:48:24.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.285+0000 7f4353fff640 1 -- 172.21.15.76:0/1296357149 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f435c03adc0 con 0x7f436c0b4880 2023-11-09T03:48:24.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.285+0000 7f43719d5640 1 -- 172.21.15.76:0/1296357149 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436c142a70 con 0x7f436c0b4880 2023-11-09T03:48:24.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.285+0000 7f4353fff640 1 -- 172.21.15.76:0/1296357149 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c044630 con 0x7f436c0b4880 2023-11-09T03:48:24.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.285+0000 7f43719d5640 1 -- 172.21.15.76:0/1296357149 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436c143030 con 0x7f436c0b4880 2023-11-09T03:48:24.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.286+0000 7f43719d5640 1 -- 172.21.15.76:0/1296357149 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f436c143220 con 0x7f436c0b4880 2023-11-09T03:48:24.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.287+0000 7f4353fff640 1 -- 172.21.15.76:0/1296357149 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f435c02f050 con 0x7f436c0b4880 2023-11-09T03:48:24.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.287+0000 7f4353fff640 1 --2- 172.21.15.76:0/1296357149 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f435806ae70 0x7f435806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:24.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.287+0000 7f4353fff640 1 -- 172.21.15.76:0/1296357149 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f435c0aba80 con 0x7f436c0b4880 2023-11-09T03:48:24.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.287+0000 7f436a7fc640 1 --2- 172.21.15.76:0/1296357149 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f435806ae70 0x7f435806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:24.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.288+0000 7f436a7fc640 1 --2- 172.21.15.76:0/1296357149 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f435806ae70 0x7f435806d330 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f436c0cd0b0 tx=0x7f4360008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:24.298 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.292+0000 7f4353fff640 1 -- 172.21.15.76:0/1296357149 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f435c076530 con 0x7f436c0b4880 2023-11-09T03:48:24.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.494+0000 7f43719d5640 1 -- 172.21.15.76:0/1296357149 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f436c074290 con 0x7f436c0b4880 2023-11-09T03:48:24.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.549+0000 7f4353fff640 1 -- 172.21.15.76:0/1296357149 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) v1 ==== 118+0+0 (secure 0 0 0) 0x7f435c0a9e60 con 0x7f436c0b4880 2023-11-09T03:48:24.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.551+0000 7f43719d5640 1 -- 172.21.15.76:0/1296357149 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f435806ae70 msgr2=0x7f435806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:24.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.551+0000 7f43719d5640 1 --2- 172.21.15.76:0/1296357149 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f435806ae70 0x7f435806d330 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f436c0cd0b0 tx=0x7f4360008040 comp rx=0 tx=0).stop 2023-11-09T03:48:24.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.551+0000 7f43719d5640 1 -- 172.21.15.76:0/1296357149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f436c0b4880 msgr2=0x7f436c144b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:24.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.551+0000 7f43719d5640 1 --2- 172.21.15.76:0/1296357149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f436c0b4880 0x7f436c144b90 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f435c000c00 tx=0x7f435c0046c0 comp rx=0 tx=0).stop 2023-11-09T03:48:24.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.551+0000 7f43719d5640 1 -- 172.21.15.76:0/1296357149 shutdown_connections 2023-11-09T03:48:24.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.551+0000 7f43719d5640 1 --2- 172.21.15.76:0/1296357149 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f435806ae70 0x7f435806d330 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:24.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.551+0000 7f43719d5640 1 --2- 172.21.15.76:0/1296357149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f436c153e60 0x7f436c155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:24.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.551+0000 7f43719d5640 1 --2- 172.21.15.76:0/1296357149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f436c0cd6d0 0x7f436c0cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:24.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.551+0000 7f43719d5640 1 --2- 172.21.15.76:0/1296357149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f436c0b4880 0x7f436c144b90 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:24.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.551+0000 7f43719d5640 1 -- 172.21.15.76:0/1296357149 >> 172.21.15.76:0/1296357149 conn(0x7f436c0cf880 msgr2=0x7f436c0cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:24.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.551+0000 7f43719d5640 1 -- 172.21.15.76:0/1296357149 shutdown_connections 2023-11-09T03:48:24.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.551+0000 7f43719d5640 1 -- 172.21.15.76:0/1296357149 wait complete. 2023-11-09T03:48:24.564 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-09T03:48:24.564 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-09T03:48:24.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.666+0000 7fe6b32db640 1 Processor -- start 2023-11-09T03:48:24.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.667+0000 7fe6b32db640 1 -- start start 2023-11-09T03:48:24.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.667+0000 7fe6b32db640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe6ac153850 0x7fe6ac15a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:24.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.667+0000 7fe6b32db640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6ac0c3860 con 0x7fe6ac0cc520 2023-11-09T03:48:24.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.667+0000 7fe6b32db640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6ac0c9250 con 0x7fe6ac0cc9d0 2023-11-09T03:48:24.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.667+0000 7fe6b32db640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6ac153c30 con 0x7fe6ac153850 2023-11-09T03:48:24.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.667+0000 7fe6b0876640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe6ac0cc520 0x7fe6ac0c3cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59026/0 (socket says 172.21.15.76:59026) 2023-11-09T03:48:24.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.668+0000 7fe6b0876640 1 -- 172.21.15.76:0/595337667 learned_addr learned my addr 172.21.15.76:0/595337667 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:24.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.668+0000 7fe6b1077640 1 --2- 172.21.15.76:0/595337667 >> v2:172.21.15.113:3300/0 conn(0x7fe6ac153850 0x7fe6ac15a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:24.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.668+0000 7fe6a3fff640 1 -- 172.21.15.76:0/595337667 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1974941638 0 0) 0x7fe6ac0c3860 con 0x7fe6ac0cc520 2023-11-09T03:48:24.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.668+0000 7fe6a3fff640 1 -- 172.21.15.76:0/595337667 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe698003520 con 0x7fe6ac0cc520 2023-11-09T03:48:24.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.668+0000 7fe6a3fff640 1 -- 172.21.15.76:0/595337667 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3067098529 0 0) 0x7fe698003520 con 0x7fe6ac0cc520 2023-11-09T03:48:24.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.668+0000 7fe6a3fff640 1 -- 172.21.15.76:0/595337667 >> v2:172.21.15.113:3300/0 conn(0x7fe6ac153850 msgr2=0x7fe6ac15a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:24.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.668+0000 7fe6a3fff640 1 --2- 172.21.15.76:0/595337667 >> v2:172.21.15.113:3300/0 conn(0x7fe6ac153850 0x7fe6ac15a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:24.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.668+0000 7fe6a3fff640 1 -- 172.21.15.76:0/595337667 >> v1:172.21.15.113:6789/0 conn(0x7fe6ac0cc9d0 legacy=0x7fe6ac070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:24.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.668+0000 7fe6a3fff640 1 -- 172.21.15.76:0/595337667 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ac15a700 con 0x7fe6ac0cc520 2023-11-09T03:48:24.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.668+0000 7fe6a3fff640 1 -- 172.21.15.76:0/595337667 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fe69c001670 con 0x7fe6ac0cc520 2023-11-09T03:48:24.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.668+0000 7fe6a3fff640 1 -- 172.21.15.76:0/595337667 >> v1:172.21.15.76:6789/0 conn(0x7fe6ac0cc520 legacy=0x7fe6ac0c3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:24.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.668+0000 7fe6a3fff640 1 --2- 172.21.15.76:0/595337667 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe698003970 0x7fe698003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:24.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.668+0000 7fe6a3fff640 1 --2- 172.21.15.76:0/595337667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6980047c0 0x7fe698004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:24.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.668+0000 7fe6a3fff640 1 --2- 172.21.15.76:0/595337667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe698005180 0x7fe69800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:24.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.668+0000 7fe6a3fff640 1 -- 172.21.15.76:0/595337667 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac153c30 con 0x7fe698005180 2023-11-09T03:48:24.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.668+0000 7fe6a3fff640 1 -- 172.21.15.76:0/595337667 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69800c020 con 0x7fe6980047c0 2023-11-09T03:48:24.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.668+0000 7fe6a3fff640 1 -- 172.21.15.76:0/595337667 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe69800c1e0 con 0x7fe698003970 2023-11-09T03:48:24.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.668+0000 7fe6b1878640 1 --2- 172.21.15.76:0/595337667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe698005180 0x7fe69800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:24.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.668+0000 7fe6b1077640 1 --2- 172.21.15.76:0/595337667 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe698003970 0x7fe698003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:24.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.668+0000 7fe6b0876640 1 --2- 172.21.15.76:0/595337667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6980047c0 0x7fe698004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:24.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.668+0000 7fe6b1878640 1 -- 172.21.15.76:0/595337667 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe698003970 msgr2=0x7fe698003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:24.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.668+0000 7fe6b1878640 1 --2- 172.21.15.76:0/595337667 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe698003970 0x7fe698003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:24.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.668+0000 7fe6b1878640 1 -- 172.21.15.76:0/595337667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6980047c0 msgr2=0x7fe698004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:24.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.668+0000 7fe6b1878640 1 --2- 172.21.15.76:0/595337667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6980047c0 0x7fe698004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:24.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.669+0000 7fe6b1878640 1 -- 172.21.15.76:0/595337667 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe698003520 con 0x7fe698005180 2023-11-09T03:48:24.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.669+0000 7fe6b1878640 1 -- 172.21.15.76:0/595337667 reap_dead start 2023-11-09T03:48:24.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.669+0000 7fe6b1878640 1 --2- 172.21.15.76:0/595337667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe698005180 0x7fe69800b980 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4001750 tx=0x7fe6980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a1bbf799f521d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:24.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.669+0000 7fe6a3fff640 1 -- 172.21.15.76:0/595337667 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a40052d0 con 0x7fe698005180 2023-11-09T03:48:24.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.669+0000 7fe6a3fff640 1 -- 172.21.15.76:0/595337667 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a402e050 con 0x7fe698005180 2023-11-09T03:48:24.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.669+0000 7fe6a3fff640 1 -- 172.21.15.76:0/595337667 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4003830 con 0x7fe698005180 2023-11-09T03:48:24.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.670+0000 7fe6b32db640 1 -- 172.21.15.76:0/595337667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe698005180 msgr2=0x7fe69800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:24.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.670+0000 7fe6b32db640 1 --2- 172.21.15.76:0/595337667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe698005180 0x7fe69800b980 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4001750 tx=0x7fe6980057f0 comp rx=0 tx=0).stop 2023-11-09T03:48:24.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.670+0000 7fe6b32db640 1 -- 172.21.15.76:0/595337667 shutdown_connections 2023-11-09T03:48:24.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.670+0000 7fe6b32db640 1 --2- 172.21.15.76:0/595337667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe698005180 0x7fe69800b980 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:24.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.670+0000 7fe6b32db640 1 -- 172.21.15.76:0/595337667 >> 172.21.15.76:0/595337667 conn(0x7fe6ac0ce950 msgr2=0x7fe6ac0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:24.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.670+0000 7fe6b32db640 1 -- 172.21.15.76:0/595337667 shutdown_connections 2023-11-09T03:48:24.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.670+0000 7fe6b32db640 1 -- 172.21.15.76:0/595337667 wait complete. 2023-11-09T03:48:24.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.671+0000 7fe6b32db640 1 Processor -- start 2023-11-09T03:48:24.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.671+0000 7fe6b32db640 1 -- start start 2023-11-09T03:48:24.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.672+0000 7fe6b32db640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe698005180 0x7fe6ac1ee3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:24.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.672+0000 7fe6b32db640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6ac0b1530 0x7fe6ac143290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:24.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.672+0000 7fe6b32db640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6ac158400 0x7fe6ac158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:24.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.672+0000 7fe6b32db640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac158dc0 con 0x7fe6ac0b1530 2023-11-09T03:48:24.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.672+0000 7fe6b32db640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac150a70 con 0x7fe698005180 2023-11-09T03:48:24.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.672+0000 7fe6b32db640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac1ee8e0 con 0x7fe6ac158400 2023-11-09T03:48:24.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.672+0000 7fe6b0876640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6ac0b1530 0x7fe6ac143290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:24.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.672+0000 7fe6b0876640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6ac0b1530 0x7fe6ac143290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37768/0 (socket says 172.21.15.76:37768) 2023-11-09T03:48:24.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.672+0000 7fe6b0876640 1 -- 172.21.15.76:0/284157580 learned_addr learned my addr 172.21.15.76:0/284157580 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:24.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.672+0000 7fe6b1077640 1 --2- 172.21.15.76:0/284157580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe698005180 0x7fe6ac1ee3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:24.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.672+0000 7fe6b0876640 1 -- 172.21.15.76:0/284157580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6ac158400 msgr2=0x7fe6ac158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:24.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.672+0000 7fe6b0876640 1 --2- 172.21.15.76:0/284157580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6ac158400 0x7fe6ac158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:24.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.672+0000 7fe6b0876640 1 -- 172.21.15.76:0/284157580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe698005180 msgr2=0x7fe6ac1ee3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:24.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.672+0000 7fe6b0876640 1 --2- 172.21.15.76:0/284157580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe698005180 0x7fe6ac1ee3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:24.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.672+0000 7fe6b0876640 1 -- 172.21.15.76:0/284157580 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ac1eeb80 con 0x7fe6ac0b1530 2023-11-09T03:48:24.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.672+0000 7fe6b1077640 1 --2- 172.21.15.76:0/284157580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe698005180 0x7fe6ac1ee3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:24.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.672+0000 7fe6b0876640 1 --2- 172.21.15.76:0/284157580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6ac0b1530 0x7fe6ac143290 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fe69c0072c0 tx=0x7fe69c007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:24.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.672+0000 7fe6a1ffb640 1 -- 172.21.15.76:0/284157580 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c00f8e0 con 0x7fe6ac0b1530 2023-11-09T03:48:24.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.673+0000 7fe6a1ffb640 1 -- 172.21.15.76:0/284157580 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69c00fa40 con 0x7fe6ac0b1530 2023-11-09T03:48:24.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.673+0000 7fe6b32db640 1 -- 172.21.15.76:0/284157580 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6ac1eee60 con 0x7fe6ac0b1530 2023-11-09T03:48:24.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.673+0000 7fe6a1ffb640 1 -- 172.21.15.76:0/284157580 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c00fdf0 con 0x7fe6ac0b1530 2023-11-09T03:48:24.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.673+0000 7fe6b32db640 1 -- 172.21.15.76:0/284157580 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6ac1551c0 con 0x7fe6ac0b1530 2023-11-09T03:48:24.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.674+0000 7fe6b32db640 1 -- 172.21.15.76:0/284157580 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6ac1ef380 con 0x7fe6ac0b1530 2023-11-09T03:48:24.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.674+0000 7fe6a1ffb640 1 -- 172.21.15.76:0/284157580 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe69c01d460 con 0x7fe6ac0b1530 2023-11-09T03:48:24.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.675+0000 7fe6a1ffb640 1 --2- 172.21.15.76:0/284157580 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe69006aea0 0x7fe69006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:24.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.675+0000 7fe6a1ffb640 1 -- 172.21.15.76:0/284157580 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7fe69c089c90 con 0x7fe6ac0b1530 2023-11-09T03:48:24.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.675+0000 7fe6b1077640 1 --2- 172.21.15.76:0/284157580 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe69006aea0 0x7fe69006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:24.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.675+0000 7fe6b1077640 1 --2- 172.21.15.76:0/284157580 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe69006aea0 0x7fe69006d360 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8007440 tx=0x7fe6a800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:24.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.679+0000 7fe6a1ffb640 1 -- 172.21.15.76:0/284157580 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe6ac1ef380 con 0x7fe6ac0b1530 2023-11-09T03:48:24.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:24.882+0000 7fe6b32db640 1 -- 172.21.15.76:0/284157580 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe6ac155020 con 0x7fe6ac0b1530 2023-11-09T03:48:25.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.560+0000 7fe6a1ffb640 1 -- 172.21.15.76:0/284157580 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe6ac155020 con 0x7fe6ac0b1530 2023-11-09T03:48:25.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.563+0000 7fe6b32db640 1 -- 172.21.15.76:0/284157580 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe69006aea0 msgr2=0x7fe69006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:25.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.563+0000 7fe6b32db640 1 --2- 172.21.15.76:0/284157580 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe69006aea0 0x7fe69006d360 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8007440 tx=0x7fe6a800b040 comp rx=0 tx=0).stop 2023-11-09T03:48:25.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.563+0000 7fe6b32db640 1 -- 172.21.15.76:0/284157580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6ac0b1530 msgr2=0x7fe6ac143290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:25.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.563+0000 7fe6b32db640 1 --2- 172.21.15.76:0/284157580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6ac0b1530 0x7fe6ac143290 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fe69c0072c0 tx=0x7fe69c007780 comp rx=0 tx=0).stop 2023-11-09T03:48:25.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.563+0000 7fe6b32db640 1 -- 172.21.15.76:0/284157580 shutdown_connections 2023-11-09T03:48:25.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.563+0000 7fe6b32db640 1 --2- 172.21.15.76:0/284157580 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe69006aea0 0x7fe69006d360 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:25.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.563+0000 7fe6b32db640 1 --2- 172.21.15.76:0/284157580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6ac158400 0x7fe6ac158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:25.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.563+0000 7fe6b32db640 1 --2- 172.21.15.76:0/284157580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6ac0b1530 0x7fe6ac143290 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:25.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.563+0000 7fe6b32db640 1 --2- 172.21.15.76:0/284157580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe698005180 0x7fe6ac1ee3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:25.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.563+0000 7fe6b32db640 1 -- 172.21.15.76:0/284157580 >> 172.21.15.76:0/284157580 conn(0x7fe6ac0ce950 msgr2=0x7fe6ac1526f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:25.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.563+0000 7fe6b32db640 1 -- 172.21.15.76:0/284157580 shutdown_connections 2023-11-09T03:48:25.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.563+0000 7fe6b32db640 1 -- 172.21.15.76:0/284157580 wait complete. 2023-11-09T03:48:25.577 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-09T03:48:25.577 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:48:25.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.683+0000 7f919e087640 1 Processor -- start 2023-11-09T03:48:25.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.683+0000 7f919e087640 1 -- start start 2023-11-09T03:48:25.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.684+0000 7f919e087640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9198159c70 0x7f91980ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:25.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.684+0000 7f919e087640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f919806db20 0x7f91980b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:25.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.684+0000 7f919e087640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91980b4110 con 0x7f91980b0910 2023-11-09T03:48:25.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.684+0000 7f919e087640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91980b49b0 con 0x7f9198159c70 2023-11-09T03:48:25.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.684+0000 7f919e087640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91980b0da0 con 0x7f919806db20 2023-11-09T03:48:25.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.684+0000 7f91977fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9198159c70 0x7f91980ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:25.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.684+0000 7f91977fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9198159c70 0x7f91980ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37784/0 (socket says 172.21.15.76:37784) 2023-11-09T03:48:25.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.684+0000 7f91977fe640 1 -- 172.21.15.76:0/1627065548 learned_addr learned my addr 172.21.15.76:0/1627065548 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:25.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.684+0000 7f9196ffd640 1 --2- 172.21.15.76:0/1627065548 >> v2:172.21.15.113:3300/0 conn(0x7f919806db20 0x7f91980b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:25.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.684+0000 7f91977fe640 1 -- 172.21.15.76:0/1627065548 >> v2:172.21.15.113:3300/0 conn(0x7f919806db20 msgr2=0x7f91980b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:25.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.684+0000 7f91977fe640 1 --2- 172.21.15.76:0/1627065548 >> v2:172.21.15.113:3300/0 conn(0x7f919806db20 0x7f91980b03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:25.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.684+0000 7f91977fe640 1 -- 172.21.15.76:0/1627065548 >> v1:172.21.15.113:6789/0 conn(0x7f91980b0910 legacy=0x7f919815d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:25.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.684+0000 7f91977fe640 1 -- 172.21.15.76:0/1627065548 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91980b43f0 con 0x7f9198159c70 2023-11-09T03:48:25.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.684+0000 7f9196ffd640 1 --2- 172.21.15.76:0/1627065548 >> v2:172.21.15.113:3300/0 conn(0x7f919806db20 0x7f91980b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:25.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.685+0000 7f91977fe640 1 --2- 172.21.15.76:0/1627065548 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9198159c70 0x7f91980ce370 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f91980cf800 tx=0x7f918802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf0dd315ed1069e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:25.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.685+0000 7f91967fc640 1 -- 172.21.15.76:0/1627065548 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9188031870 con 0x7f9198159c70 2023-11-09T03:48:25.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.685+0000 7f91967fc640 1 -- 172.21.15.76:0/1627065548 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91880319d0 con 0x7f9198159c70 2023-11-09T03:48:25.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.685+0000 7f91967fc640 1 -- 172.21.15.76:0/1627065548 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9188031d60 con 0x7f9198159c70 2023-11-09T03:48:25.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.686+0000 7f919e087640 1 -- 172.21.15.76:0/1627065548 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9198159c70 msgr2=0x7f91980ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:25.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.686+0000 7f919e087640 1 --2- 172.21.15.76:0/1627065548 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9198159c70 0x7f91980ce370 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f91980cf800 tx=0x7f918802b910 comp rx=0 tx=0).stop 2023-11-09T03:48:25.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.686+0000 7f919e087640 1 -- 172.21.15.76:0/1627065548 shutdown_connections 2023-11-09T03:48:25.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.686+0000 7f919e087640 1 --2- 172.21.15.76:0/1627065548 >> v2:172.21.15.113:3300/0 conn(0x7f919806db20 0x7f91980b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:25.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.686+0000 7f919e087640 1 --2- 172.21.15.76:0/1627065548 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9198159c70 0x7f91980ce370 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:25.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.686+0000 7f919e087640 1 -- 172.21.15.76:0/1627065548 >> 172.21.15.76:0/1627065548 conn(0x7f91980ce950 msgr2=0x7f91980ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:25.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.686+0000 7f919e087640 1 -- 172.21.15.76:0/1627065548 shutdown_connections 2023-11-09T03:48:25.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.686+0000 7f919e087640 1 -- 172.21.15.76:0/1627065548 wait complete. 2023-11-09T03:48:25.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.687+0000 7f919e087640 1 Processor -- start 2023-11-09T03:48:25.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.687+0000 7f919e087640 1 -- start start 2023-11-09T03:48:25.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.688+0000 7f919e087640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f919806db20 0x7f9198156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:25.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.688+0000 7f919e087640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f91980b0910 0x7f9198146ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:25.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.688+0000 7f919e087640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9198159c70 0x7f91980c5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:25.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.688+0000 7f919e087640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9198147220 con 0x7f919806db20 2023-11-09T03:48:25.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.688+0000 7f919e087640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91980c8b60 con 0x7f9198159c70 2023-11-09T03:48:25.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.688+0000 7f919e087640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91980c7f50 con 0x7f91980b0910 2023-11-09T03:48:25.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.688+0000 7f91977fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f919806db20 0x7f9198156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:25.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.688+0000 7f91977fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f919806db20 0x7f9198156210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37792/0 (socket says 172.21.15.76:37792) 2023-11-09T03:48:25.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.688+0000 7f91977fe640 1 -- 172.21.15.76:0/633289646 learned_addr learned my addr 172.21.15.76:0/633289646 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:25.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.688+0000 7f91977fe640 1 -- 172.21.15.76:0/633289646 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f91980b0910 msgr2=0x7f9198146ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:25.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.688+0000 7f91977fe640 1 --2- 172.21.15.76:0/633289646 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f91980b0910 0x7f9198146ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:25.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.688+0000 7f91977fe640 1 -- 172.21.15.76:0/633289646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9198159c70 msgr2=0x7f91980c5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:25.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.688+0000 7f9197fff640 1 --2- 172.21.15.76:0/633289646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9198159c70 0x7f91980c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:25.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.688+0000 7f91977fe640 1 --2- 172.21.15.76:0/633289646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9198159c70 0x7f91980c5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:25.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.688+0000 7f91977fe640 1 -- 172.21.15.76:0/633289646 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91880094d0 con 0x7f919806db20 2023-11-09T03:48:25.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.689+0000 7f91977fe640 1 --2- 172.21.15.76:0/633289646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f919806db20 0x7f9198156210 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f9188000c00 tx=0x7f91880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:25.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.689+0000 7f917ffff640 1 -- 172.21.15.76:0/633289646 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918803ad10 con 0x7f919806db20 2023-11-09T03:48:25.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.689+0000 7f917ffff640 1 -- 172.21.15.76:0/633289646 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f918803ae90 con 0x7f919806db20 2023-11-09T03:48:25.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.689+0000 7f919e087640 1 -- 172.21.15.76:0/633289646 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91980c8230 con 0x7f919806db20 2023-11-09T03:48:25.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.689+0000 7f917ffff640 1 -- 172.21.15.76:0/633289646 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9188044630 con 0x7f919806db20 2023-11-09T03:48:25.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.689+0000 7f919e087640 1 -- 172.21.15.76:0/633289646 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91980c87f0 con 0x7f919806db20 2023-11-09T03:48:25.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.690+0000 7f919e087640 1 -- 172.21.15.76:0/633289646 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9198074290 con 0x7f919806db20 2023-11-09T03:48:25.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.690+0000 7f917ffff640 1 -- 172.21.15.76:0/633289646 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f918804d450 con 0x7f919806db20 2023-11-09T03:48:25.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.691+0000 7f917ffff640 1 --2- 172.21.15.76:0/633289646 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f918406aea0 0x7f918406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:25.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.691+0000 7f917ffff640 1 -- 172.21.15.76:0/633289646 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f918802d040 con 0x7f919806db20 2023-11-09T03:48:25.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.691+0000 7f9196ffd640 1 --2- 172.21.15.76:0/633289646 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f918406aea0 0x7f918406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:25.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.692+0000 7f9196ffd640 1 --2- 172.21.15.76:0/633289646 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f918406aea0 0x7f918406d360 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f918c006fd0 tx=0x7f918c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:25.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.695+0000 7f917ffff640 1 -- 172.21.15.76:0/633289646 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9188076d90 con 0x7f919806db20 2023-11-09T03:48:25.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.895+0000 7f919e087640 1 -- 172.21.15.76:0/633289646 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f919806d3f0 con 0x7f919806db20 2023-11-09T03:48:25.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.896+0000 7f917ffff640 1 -- 172.21.15.76:0/633289646 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f918807a3b0 con 0x7f919806db20 2023-11-09T03:48:25.897 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:25.897 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":35,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6097,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2408659865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2408659865},{"type":"v1","addr":"172.21.15.113:6836","nonce":2408659865}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6100,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1873078383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1873078383},{"type":"v1","addr":"172.21.15.113:6837","nonce":1873078383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/3549714130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3549714130},{"type":"v1","addr":"172.21.15.76:6834","nonce":3549714130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:48:22.895004+0000","modified":"2023-11-09T03:48:24.884028+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6984":{"gid":6984,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/2165197913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":2165197913},{"type":"v1","addr":"172.21.15.76:6835","nonce":2165197913}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-09T03:48:25.897 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 35 2023-11-09T03:48:25.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.898+0000 7f919e087640 1 -- 172.21.15.76:0/633289646 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f918406aea0 msgr2=0x7f918406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:25.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.898+0000 7f919e087640 1 --2- 172.21.15.76:0/633289646 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f918406aea0 0x7f918406d360 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f918c006fd0 tx=0x7f918c008040 comp rx=0 tx=0).stop 2023-11-09T03:48:25.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.898+0000 7f919e087640 1 -- 172.21.15.76:0/633289646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f919806db20 msgr2=0x7f9198156210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:25.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.898+0000 7f919e087640 1 --2- 172.21.15.76:0/633289646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f919806db20 0x7f9198156210 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f9188000c00 tx=0x7f91880046c0 comp rx=0 tx=0).stop 2023-11-09T03:48:25.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.898+0000 7f919e087640 1 -- 172.21.15.76:0/633289646 shutdown_connections 2023-11-09T03:48:25.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.898+0000 7f919e087640 1 --2- 172.21.15.76:0/633289646 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f918406aea0 0x7f918406d360 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:25.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.898+0000 7f919e087640 1 --2- 172.21.15.76:0/633289646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9198159c70 0x7f91980c5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:25.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.898+0000 7f919e087640 1 --2- 172.21.15.76:0/633289646 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f91980b0910 0x7f9198146ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:25.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.898+0000 7f919e087640 1 --2- 172.21.15.76:0/633289646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f919806db20 0x7f9198156210 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:25.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.898+0000 7f919e087640 1 -- 172.21.15.76:0/633289646 >> 172.21.15.76:0/633289646 conn(0x7f91980ce950 msgr2=0x7f919815cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:25.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.898+0000 7f919e087640 1 -- 172.21.15.76:0/633289646 shutdown_connections 2023-11-09T03:48:25.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:25.898+0000 7f919e087640 1 -- 172.21.15.76:0/633289646 wait complete. 2023-11-09T03:48:25.909 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:48:26.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.008+0000 7f7c2a81a640 1 Processor -- start 2023-11-09T03:48:26.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.008+0000 7f7c2a81a640 1 -- start start 2023-11-09T03:48:26.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.009+0000 7f7c2a81a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7c240cda60 0x7f7c240ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:26.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.009+0000 7f7c2a81a640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c240cc990 con 0x7f7c24070d40 2023-11-09T03:48:26.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.009+0000 7f7c2a81a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c240ccc70 con 0x7f7c240b0cd0 2023-11-09T03:48:26.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.009+0000 7f7c2a81a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c240c9ce0 con 0x7f7c240cda60 2023-11-09T03:48:26.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.009+0000 7f7c28db7640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f7c24070d40 0x7f7c2406c200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59036/0 (socket says 172.21.15.76:59036) 2023-11-09T03:48:26.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.009+0000 7f7c28db7640 1 -- 172.21.15.76:0/435252769 learned_addr learned my addr 172.21.15.76:0/435252769 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:26.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.010+0000 7f7c23fff640 1 --2- 172.21.15.76:0/435252769 >> v2:172.21.15.113:3300/0 conn(0x7f7c240cda60 0x7f7c240ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:26.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.010+0000 7f7c22ffd640 1 -- 172.21.15.76:0/435252769 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2068067133 0 0) 0x7f7c240cc990 con 0x7f7c24070d40 2023-11-09T03:48:26.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.010+0000 7f7c22ffd640 1 -- 172.21.15.76:0/435252769 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c0c003520 con 0x7f7c24070d40 2023-11-09T03:48:26.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.010+0000 7f7c22ffd640 1 -- 172.21.15.76:0/435252769 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2091540860 0 0) 0x7f7c0c003520 con 0x7f7c24070d40 2023-11-09T03:48:26.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.010+0000 7f7c22ffd640 1 -- 172.21.15.76:0/435252769 >> v2:172.21.15.113:3300/0 conn(0x7f7c240cda60 msgr2=0x7f7c240ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:26.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.010+0000 7f7c22ffd640 1 --2- 172.21.15.76:0/435252769 >> v2:172.21.15.113:3300/0 conn(0x7f7c240cda60 0x7f7c240ca7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.010+0000 7f7c22ffd640 1 -- 172.21.15.76:0/435252769 >> v1:172.21.15.113:6789/0 conn(0x7f7c240b0cd0 legacy=0x7f7c24070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:26.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.010+0000 7f7c22ffd640 1 -- 172.21.15.76:0/435252769 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c240c9fc0 con 0x7f7c24070d40 2023-11-09T03:48:26.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.010+0000 7f7c22ffd640 1 -- 172.21.15.76:0/435252769 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f7c1c0019f0 con 0x7f7c24070d40 2023-11-09T03:48:26.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.010+0000 7f7c22ffd640 1 -- 172.21.15.76:0/435252769 >> v1:172.21.15.76:6789/0 conn(0x7f7c24070d40 legacy=0x7f7c2406c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:26.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.010+0000 7f7c22ffd640 1 --2- 172.21.15.76:0/435252769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c0c003970 0x7f7c0c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:26.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.010+0000 7f7c22ffd640 1 --2- 172.21.15.76:0/435252769 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c0c0047c0 0x7f7c0c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:26.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.010+0000 7f7c22ffd640 1 --2- 172.21.15.76:0/435252769 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c0c005180 0x7f7c0c00ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:26.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.010+0000 7f7c22ffd640 1 -- 172.21.15.76:0/435252769 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c240c9ce0 con 0x7f7c0c0047c0 2023-11-09T03:48:26.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.010+0000 7f7c22ffd640 1 -- 172.21.15.76:0/435252769 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c00d130 con 0x7f7c0c003970 2023-11-09T03:48:26.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.010+0000 7f7c22ffd640 1 -- 172.21.15.76:0/435252769 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c00d2f0 con 0x7f7c0c005180 2023-11-09T03:48:26.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.010+0000 7f7c237fe640 1 --2- 172.21.15.76:0/435252769 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c0c0047c0 0x7f7c0c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:26.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.010+0000 7f7c23fff640 1 --2- 172.21.15.76:0/435252769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c0c003970 0x7f7c0c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:26.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.010+0000 7f7c28db7640 1 --2- 172.21.15.76:0/435252769 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c0c005180 0x7f7c0c00ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:26.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.010+0000 7f7c237fe640 1 -- 172.21.15.76:0/435252769 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c0c005180 msgr2=0x7f7c0c00ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:26.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.010+0000 7f7c237fe640 1 --2- 172.21.15.76:0/435252769 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c0c005180 0x7f7c0c00ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.010+0000 7f7c237fe640 1 -- 172.21.15.76:0/435252769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c0c003970 msgr2=0x7f7c0c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:26.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.010+0000 7f7c237fe640 1 --2- 172.21.15.76:0/435252769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c0c003970 0x7f7c0c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.010+0000 7f7c237fe640 1 -- 172.21.15.76:0/435252769 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c0c003520 con 0x7f7c0c0047c0 2023-11-09T03:48:26.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.010+0000 7f7c28db7640 1 --2- 172.21.15.76:0/435252769 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c0c005180 0x7f7c0c00ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:26.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.010+0000 7f7c28db7640 1 -- 172.21.15.76:0/435252769 reap_dead start 2023-11-09T03:48:26.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.011+0000 7f7c237fe640 1 --2- 172.21.15.76:0/435252769 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c0c0047c0 0x7f7c0c004c40 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f7c18001720 tx=0x7f7c0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f37c1e48e7dd92 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:26.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.011+0000 7f7c22ffd640 1 -- 172.21.15.76:0/435252769 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c180052d0 con 0x7f7c0c0047c0 2023-11-09T03:48:26.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.011+0000 7f7c22ffd640 1 -- 172.21.15.76:0/435252769 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1802e050 con 0x7f7c0c0047c0 2023-11-09T03:48:26.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.011+0000 7f7c22ffd640 1 -- 172.21.15.76:0/435252769 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c18003830 con 0x7f7c0c0047c0 2023-11-09T03:48:26.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.012+0000 7f7c2a81a640 1 -- 172.21.15.76:0/435252769 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c0c0047c0 msgr2=0x7f7c0c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:26.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.012+0000 7f7c2a81a640 1 --2- 172.21.15.76:0/435252769 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c0c0047c0 0x7f7c0c004c40 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f7c18001720 tx=0x7f7c0c0057f0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.012+0000 7f7c2a81a640 1 -- 172.21.15.76:0/435252769 shutdown_connections 2023-11-09T03:48:26.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.012+0000 7f7c2a81a640 1 --2- 172.21.15.76:0/435252769 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c0c0047c0 0x7f7c0c004c40 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.012+0000 7f7c2a81a640 1 -- 172.21.15.76:0/435252769 >> 172.21.15.76:0/435252769 conn(0x7f7c240ce950 msgr2=0x7f7c240ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:26.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.012+0000 7f7c2a81a640 1 -- 172.21.15.76:0/435252769 shutdown_connections 2023-11-09T03:48:26.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.012+0000 7f7c2a81a640 1 -- 172.21.15.76:0/435252769 wait complete. 2023-11-09T03:48:26.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.013+0000 7f7c2a81a640 1 Processor -- start 2023-11-09T03:48:26.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.013+0000 7f7c2a81a640 1 -- start start 2023-11-09T03:48:26.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.014+0000 7f7c2a81a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c0c0047c0 0x7f7c2414b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:26.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.014+0000 7f7c2a81a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c241e2590 0x7f7c241e29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:26.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.014+0000 7f7c2a81a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c241eef50 0x7f7c241ef3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:26.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.014+0000 7f7c2a81a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c241e2f30 con 0x7f7c241eef50 2023-11-09T03:48:26.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.014+0000 7f7c2a81a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2414beb0 con 0x7f7c0c0047c0 2023-11-09T03:48:26.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.014+0000 7f7c2a81a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c240c84f0 con 0x7f7c241e2590 2023-11-09T03:48:26.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.014+0000 7f7c28db7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c241eef50 0x7f7c241ef3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:26.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.014+0000 7f7c28db7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c241eef50 0x7f7c241ef3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37818/0 (socket says 172.21.15.76:37818) 2023-11-09T03:48:26.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.014+0000 7f7c28db7640 1 -- 172.21.15.76:0/2626143669 learned_addr learned my addr 172.21.15.76:0/2626143669 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:26.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.014+0000 7f7c237fe640 1 --2- 172.21.15.76:0/2626143669 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c241e2590 0x7f7c241e29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:26.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.014+0000 7f7c23fff640 1 --2- 172.21.15.76:0/2626143669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c0c0047c0 0x7f7c2414b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:26.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.014+0000 7f7c28db7640 1 -- 172.21.15.76:0/2626143669 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c241e2590 msgr2=0x7f7c241e29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:26.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.014+0000 7f7c28db7640 1 --2- 172.21.15.76:0/2626143669 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c241e2590 0x7f7c241e29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.014+0000 7f7c28db7640 1 -- 172.21.15.76:0/2626143669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c0c0047c0 msgr2=0x7f7c2414b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:26.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.014+0000 7f7c28db7640 1 --2- 172.21.15.76:0/2626143669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c0c0047c0 0x7f7c2414b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.014+0000 7f7c28db7640 1 -- 172.21.15.76:0/2626143669 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c241e3110 con 0x7f7c241eef50 2023-11-09T03:48:26.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.014+0000 7f7c28db7640 1 --2- 172.21.15.76:0/2626143669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c241eef50 0x7f7c241ef3d0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f7c2415ce70 tx=0x7f7c1c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:26.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.014+0000 7f7c20ff9640 1 -- 172.21.15.76:0/2626143669 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c00eb90 con 0x7f7c241eef50 2023-11-09T03:48:26.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.015+0000 7f7c20ff9640 1 -- 172.21.15.76:0/2626143669 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1c00ed10 con 0x7f7c241eef50 2023-11-09T03:48:26.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.015+0000 7f7c2a81a640 1 -- 172.21.15.76:0/2626143669 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c241e33f0 con 0x7f7c241eef50 2023-11-09T03:48:26.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.015+0000 7f7c20ff9640 1 -- 172.21.15.76:0/2626143669 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c013660 con 0x7f7c241eef50 2023-11-09T03:48:26.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.015+0000 7f7c2a81a640 1 -- 172.21.15.76:0/2626143669 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c241e39b0 con 0x7f7c241eef50 2023-11-09T03:48:26.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.015+0000 7f7c2a81a640 1 -- 172.21.15.76:0/2626143669 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c24074290 con 0x7f7c241eef50 2023-11-09T03:48:26.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.016+0000 7f7c20ff9640 1 -- 172.21.15.76:0/2626143669 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7c1c0137c0 con 0x7f7c241eef50 2023-11-09T03:48:26.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.016+0000 7f7c20ff9640 1 --2- 172.21.15.76:0/2626143669 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7c1006aea0 0x7f7c1006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:26.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.016+0000 7f7c20ff9640 1 -- 172.21.15.76:0/2626143669 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f7c1c089f40 con 0x7f7c241eef50 2023-11-09T03:48:26.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.016+0000 7f7c23fff640 1 --2- 172.21.15.76:0/2626143669 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7c1006aea0 0x7f7c1006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:26.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.017+0000 7f7c23fff640 1 --2- 172.21.15.76:0/2626143669 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7c1006aea0 0x7f7c1006d360 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7c14009b00 tx=0x7f7c14008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:26.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.020+0000 7f7c20ff9640 1 -- 172.21.15.76:0/2626143669 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7c1c054a80 con 0x7f7c241eef50 2023-11-09T03:48:26.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.187+0000 7f7c2a81a640 1 -- 172.21.15.76:0/2626143669 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c240b48d0 con 0x7f7c241eef50 2023-11-09T03:48:26.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.189+0000 7f7c20ff9640 1 -- 172.21.15.76:0/2626143669 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+21941 (secure 0 0 0) 0x7f7c1c0580a0 con 0x7f7c241eef50 2023-11-09T03:48:26.189 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:26.190 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":83,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:48:24.549937+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-09T03:47:59.280601+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-09T03:47:59.742039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:48:20.992533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-11-09T03:48:21.862178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:48:26.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.191+0000 7f7c2a81a640 1 -- 172.21.15.76:0/2626143669 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7c1006aea0 msgr2=0x7f7c1006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:26.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.191+0000 7f7c2a81a640 1 --2- 172.21.15.76:0/2626143669 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7c1006aea0 0x7f7c1006d360 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7c14009b00 tx=0x7f7c14008040 comp rx=0 tx=0).stop 2023-11-09T03:48:26.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.191+0000 7f7c2a81a640 1 -- 172.21.15.76:0/2626143669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c241eef50 msgr2=0x7f7c241ef3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:26.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.191+0000 7f7c2a81a640 1 --2- 172.21.15.76:0/2626143669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c241eef50 0x7f7c241ef3d0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f7c2415ce70 tx=0x7f7c1c00e530 comp rx=0 tx=0).stop 2023-11-09T03:48:26.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.191+0000 7f7c2a81a640 1 -- 172.21.15.76:0/2626143669 shutdown_connections 2023-11-09T03:48:26.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.191+0000 7f7c2a81a640 1 --2- 172.21.15.76:0/2626143669 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7c1006aea0 0x7f7c1006d360 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.191+0000 7f7c2a81a640 1 --2- 172.21.15.76:0/2626143669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c241eef50 0x7f7c241ef3d0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.191+0000 7f7c2a81a640 1 --2- 172.21.15.76:0/2626143669 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c241e2590 0x7f7c241e29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.191+0000 7f7c2a81a640 1 --2- 172.21.15.76:0/2626143669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c0c0047c0 0x7f7c2414b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.191+0000 7f7c2a81a640 1 -- 172.21.15.76:0/2626143669 >> 172.21.15.76:0/2626143669 conn(0x7f7c240ce950 msgr2=0x7f7c240b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:26.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.191+0000 7f7c2a81a640 1 -- 172.21.15.76:0/2626143669 shutdown_connections 2023-11-09T03:48:26.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.191+0000 7f7c2a81a640 1 -- 172.21.15.76:0/2626143669 wait complete. 2023-11-09T03:48:26.202 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-09T03:48:26.203 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-09T03:48:26.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.302+0000 7fad89d4c640 1 Processor -- start 2023-11-09T03:48:26.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.302+0000 7fad89d4c640 1 -- start start 2023-11-09T03:48:26.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.302+0000 7fad89d4c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fad840cde60 0x7fad8415d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:26.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.303+0000 7fad89d4c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad8415e000 con 0x7fad8415daa0 2023-11-09T03:48:26.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.303+0000 7fad89d4c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad8415e2e0 con 0x7fad840ce240 2023-11-09T03:48:26.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.303+0000 7fad89d4c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad8415e5c0 con 0x7fad840cde60 2023-11-09T03:48:26.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.303+0000 7fad837fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fad840cde60 0x7fad8415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:26.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.303+0000 7fad82ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fad8415daa0 0x7fad84157000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59040/0 (socket says 172.21.15.76:59040) 2023-11-09T03:48:26.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.303+0000 7fad82ffd640 1 -- 172.21.15.76:0/2158432984 learned_addr learned my addr 172.21.15.76:0/2158432984 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:26.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.303+0000 7fad837fe640 1 -- 172.21.15.76:0/2158432984 >> v1:172.21.15.113:6789/0 conn(0x7fad840ce240 legacy=0x7fad840d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:26.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.303+0000 7fad837fe640 1 -- 172.21.15.76:0/2158432984 >> v1:172.21.15.76:6789/0 conn(0x7fad8415daa0 legacy=0x7fad84157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:26.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.303+0000 7fad827fc640 1 -- 172.21.15.76:0/2158432984 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 556029337 0 0) 0x7fad8415e000 con 0x7fad8415daa0 2023-11-09T03:48:26.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.303+0000 7fad837fe640 1 -- 172.21.15.76:0/2158432984 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad840d1120 con 0x7fad840cde60 2023-11-09T03:48:26.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.305+0000 7fad837fe640 1 --2- 172.21.15.76:0/2158432984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad840cde60 0x7fad8415d560 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fad740097a0 tx=0x7fad7402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa20bae76d1d391e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:26.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.305+0000 7fad827fc640 1 -- 172.21.15.76:0/2158432984 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad74031830 con 0x7fad840cde60 2023-11-09T03:48:26.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.305+0000 7fad827fc640 1 -- 172.21.15.76:0/2158432984 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad74031990 con 0x7fad840cde60 2023-11-09T03:48:26.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.305+0000 7fad827fc640 1 -- 172.21.15.76:0/2158432984 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad74031d20 con 0x7fad840cde60 2023-11-09T03:48:26.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.305+0000 7fad89d4c640 1 -- 172.21.15.76:0/2158432984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad840cde60 msgr2=0x7fad8415d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:26.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.305+0000 7fad89d4c640 1 --2- 172.21.15.76:0/2158432984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad840cde60 0x7fad8415d560 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fad740097a0 tx=0x7fad7402b8d0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.305+0000 7fad89d4c640 1 -- 172.21.15.76:0/2158432984 shutdown_connections 2023-11-09T03:48:26.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.305+0000 7fad89d4c640 1 --2- 172.21.15.76:0/2158432984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad840cde60 0x7fad8415d560 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.305+0000 7fad89d4c640 1 -- 172.21.15.76:0/2158432984 >> 172.21.15.76:0/2158432984 conn(0x7fad84071f30 msgr2=0x7fad840cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:26.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.305+0000 7fad89d4c640 1 -- 172.21.15.76:0/2158432984 shutdown_connections 2023-11-09T03:48:26.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.305+0000 7fad89d4c640 1 -- 172.21.15.76:0/2158432984 wait complete. 2023-11-09T03:48:26.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.306+0000 7fad89d4c640 1 Processor -- start 2023-11-09T03:48:26.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.307+0000 7fad89d4c640 1 -- start start 2023-11-09T03:48:26.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.307+0000 7fad89d4c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad840cde60 0x7fad8414a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:26.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.307+0000 7fad89d4c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad840ce240 0x7fad840b6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:26.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.307+0000 7fad89d4c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad8415daa0 0x7fad84148e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:26.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.307+0000 7fad89d4c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8414a7e0 con 0x7fad8415daa0 2023-11-09T03:48:26.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.307+0000 7fad89d4c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad84147dc0 con 0x7fad840ce240 2023-11-09T03:48:26.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.307+0000 7fad89d4c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad841480a0 con 0x7fad840cde60 2023-11-09T03:48:26.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.307+0000 7fad83fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad8415daa0 0x7fad84148e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:26.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.307+0000 7fad83fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad8415daa0 0x7fad84148e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37838/0 (socket says 172.21.15.76:37838) 2023-11-09T03:48:26.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.308+0000 7fad83fff640 1 -- 172.21.15.76:0/104064 learned_addr learned my addr 172.21.15.76:0/104064 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:26.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.308+0000 7fad82ffd640 1 --2- 172.21.15.76:0/104064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad840ce240 0x7fad840b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:26.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.308+0000 7fad837fe640 1 --2- 172.21.15.76:0/104064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad840cde60 0x7fad8414a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:26.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.308+0000 7fad83fff640 1 -- 172.21.15.76:0/104064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad840cde60 msgr2=0x7fad8414a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:26.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.308+0000 7fad83fff640 1 --2- 172.21.15.76:0/104064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad840cde60 0x7fad8414a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.308+0000 7fad83fff640 1 -- 172.21.15.76:0/104064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad840ce240 msgr2=0x7fad840b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:26.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.308+0000 7fad83fff640 1 --2- 172.21.15.76:0/104064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad840ce240 0x7fad840b6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.308+0000 7fad83fff640 1 -- 172.21.15.76:0/104064 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad74009400 con 0x7fad8415daa0 2023-11-09T03:48:26.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.308+0000 7fad83fff640 1 --2- 172.21.15.76:0/104064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad8415daa0 0x7fad84148e60 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00c780 tx=0x7fad7c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:26.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.308+0000 7fad6bfff640 1 -- 172.21.15.76:0/104064 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c003d80 con 0x7fad8415daa0 2023-11-09T03:48:26.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.308+0000 7fad6bfff640 1 -- 172.21.15.76:0/104064 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7c00ce30 con 0x7fad8415daa0 2023-11-09T03:48:26.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.308+0000 7fad6bfff640 1 -- 172.21.15.76:0/104064 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c005290 con 0x7fad8415daa0 2023-11-09T03:48:26.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.308+0000 7fad89d4c640 1 -- 172.21.15.76:0/104064 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad84148380 con 0x7fad8415daa0 2023-11-09T03:48:26.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.308+0000 7fad89d4c640 1 -- 172.21.15.76:0/104064 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad84148940 con 0x7fad8415daa0 2023-11-09T03:48:26.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.309+0000 7fad89d4c640 1 -- 172.21.15.76:0/104064 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad8415d560 con 0x7fad8415daa0 2023-11-09T03:48:26.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.310+0000 7fad6bfff640 1 -- 172.21.15.76:0/104064 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fad7c006b10 con 0x7fad8415daa0 2023-11-09T03:48:26.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.310+0000 7fad6bfff640 1 --2- 172.21.15.76:0/104064 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fad7006ae70 0x7fad7006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:26.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.310+0000 7fad6bfff640 1 -- 172.21.15.76:0/104064 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7fad7c088e00 con 0x7fad8415daa0 2023-11-09T03:48:26.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.310+0000 7fad837fe640 1 --2- 172.21.15.76:0/104064 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fad7006ae70 0x7fad7006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:26.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.311+0000 7fad837fe640 1 --2- 172.21.15.76:0/104064 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fad7006ae70 0x7fad7006d330 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fad74036000 tx=0x7fad7403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:26.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.313+0000 7fad6bfff640 1 -- 172.21.15.76:0/104064 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fad7c053a80 con 0x7fad8415daa0 2023-11-09T03:48:26.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.476+0000 7fad89d4c640 1 -- 172.21.15.76:0/104064 --> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fad840d0dd0 con 0x7fad7006ae70 2023-11-09T03:48:26.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.490+0000 7fad6bfff640 1 -- 172.21.15.76:0/104064 <== mgr.4105 v2:172.21.15.113:6832/1264193042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+526928 (secure 0 0 0) 0x7fad840d0dd0 con 0x7fad7006ae70 2023-11-09T03:48:26.494 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:26.497 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2023-11-09T03:48:26.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.494+0000 7fad89d4c640 1 -- 172.21.15.76:0/104064 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fad7006ae70 msgr2=0x7fad7006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:26.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.494+0000 7fad89d4c640 1 --2- 172.21.15.76:0/104064 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fad7006ae70 0x7fad7006d330 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fad74036000 tx=0x7fad7403c040 comp rx=0 tx=0).stop 2023-11-09T03:48:26.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.494+0000 7fad89d4c640 1 -- 172.21.15.76:0/104064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad8415daa0 msgr2=0x7fad84148e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:26.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.494+0000 7fad89d4c640 1 --2- 172.21.15.76:0/104064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad8415daa0 0x7fad84148e60 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00c780 tx=0x7fad7c00cc40 comp rx=0 tx=0).stop 2023-11-09T03:48:26.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.494+0000 7fad89d4c640 1 -- 172.21.15.76:0/104064 shutdown_connections 2023-11-09T03:48:26.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.494+0000 7fad89d4c640 1 --2- 172.21.15.76:0/104064 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fad7006ae70 0x7fad7006d330 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.494+0000 7fad89d4c640 1 --2- 172.21.15.76:0/104064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad8415daa0 0x7fad84148e60 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.494+0000 7fad89d4c640 1 --2- 172.21.15.76:0/104064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad840ce240 0x7fad840b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.494+0000 7fad89d4c640 1 --2- 172.21.15.76:0/104064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad840cde60 0x7fad8414a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.494+0000 7fad89d4c640 1 -- 172.21.15.76:0/104064 >> 172.21.15.76:0/104064 conn(0x7fad84071f30 msgr2=0x7fad840d2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:26.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.494+0000 7fad89d4c640 1 -- 172.21.15.76:0/104064 shutdown_connections 2023-11-09T03:48:26.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.494+0000 7fad89d4c640 1 -- 172.21.15.76:0/104064 wait complete. 2023-11-09T03:48:26.504 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2023-11-09T03:48:24.793867+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":204,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237468,"kb_used_data":8828,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754737252,"statfs":{"total":773094113280,"available":772850946048,"internally_reserved":0,"allocated":9039872,"data_stored":4943241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001348"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549889+0000","last_change":"2023-11-09T03:48:22.549889+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549889+0000","last_undegraded":"2023-11-09T03:48:22.549889+0000","last_fullsized":"2023-11-09T03:48:22.549889+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.554480+0000","last_change":"2023-11-09T03:48:22.554480+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.554480+0000","last_undegraded":"2023-11-09T03:48:22.554480+0000","last_fullsized":"2023-11-09T03:48:22.554480+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549408+0000","last_change":"2023-11-09T03:48:22.549408+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549408+0000","last_undegraded":"2023-11-09T03:48:22.549408+0000","last_fullsized":"2023-11-09T03:48:22.549408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550714+0000","last_change":"2023-11-09T03:48:22.550714+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550714+0000","last_undegraded":"2023-11-09T03:48:22.550714+0000","last_fullsized":"2023-11-09T03:48:22.550714+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550039+0000","last_change":"2023-11-09T03:48:22.550039+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550039+0000","last_undegraded":"2023-11-09T03:48:22.550039+0000","last_fullsized":"2023-11-09T03:48:22.550039+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549702+0000","last_change":"2023-11-09T03:48:22.549702+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549702+0000","last_undegraded":"2023-11-09T03:48:22.549702+0000","last_fullsized":"2023-11-09T03:48:22.549702+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550846+0000","last_change":"2023-11-09T03:48:22.550846+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550846+0000","last_undegraded":"2023-11-09T03:48:22.550846+0000","last_fullsized":"2023-11-09T03:48:22.550846+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.554400+0000","last_change":"2023-11-09T03:48:22.554400+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.554400+0000","last_undegraded":"2023-11-09T03:48:22.554400+0000","last_fullsized":"2023-11-09T03:48:22.554400+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.547291+0000","last_change":"2023-11-09T03:48:22.547291+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.547291+0000","last_undegraded":"2023-11-09T03:48:22.547291+0000","last_fullsized":"2023-11-09T03:48:22.547291+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551875+0000","last_change":"2023-11-09T03:48:22.551875+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551875+0000","last_undegraded":"2023-11-09T03:48:22.551875+0000","last_fullsized":"2023-11-09T03:48:22.551875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.554690+0000","last_change":"2023-11-09T03:48:22.554690+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.554690+0000","last_undegraded":"2023-11-09T03:48:22.554690+0000","last_fullsized":"2023-11-09T03:48:22.554690+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.547393+0000","last_change":"2023-11-09T03:48:22.547393+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.547393+0000","last_undegraded":"2023-11-09T03:48:22.547393+0000","last_fullsized":"2023-11-09T03:48:22.547393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550250+0000","last_change":"2023-11-09T03:48:22.550250+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550250+0000","last_undegraded":"2023-11-09T03:48:22.550250+0000","last_fullsized":"2023-11-09T03:48:22.550250+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.558130+0000","last_change":"2023-11-09T03:48:22.558130+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.558130+0000","last_undegraded":"2023-11-09T03:48:22.558130+0000","last_fullsized":"2023-11-09T03:48:22.558130+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549957+0000","last_change":"2023-11-09T03:48:22.549957+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549957+0000","last_undegraded":"2023-11-09T03:48:22.549957+0000","last_fullsized":"2023-11-09T03:48:22.549957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.553338+0000","last_change":"2023-11-09T03:48:22.553338+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.553338+0000","last_undegraded":"2023-11-09T03:48:22.553338+0000","last_fullsized":"2023-11-09T03:48:22.553338+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.554384+0000","last_change":"2023-11-09T03:48:22.554384+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.554384+0000","last_undegraded":"2023-11-09T03:48:22.554384+0000","last_fullsized":"2023-11-09T03:48:22.554384+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550716+0000","last_change":"2023-11-09T03:48:22.550716+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550716+0000","last_undegraded":"2023-11-09T03:48:22.550716+0000","last_fullsized":"2023-11-09T03:48:22.550716+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.557699+0000","last_change":"2023-11-09T03:48:22.557699+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.557699+0000","last_undegraded":"2023-11-09T03:48:22.557699+0000","last_fullsized":"2023-11-09T03:48:22.557699+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.546650+0000","last_change":"2023-11-09T03:48:22.546650+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.546650+0000","last_undegraded":"2023-11-09T03:48:22.546650+0000","last_fullsized":"2023-11-09T03:48:22.546650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.555079+0000","last_change":"2023-11-09T03:48:22.555079+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.555079+0000","last_undegraded":"2023-11-09T03:48:22.555079+0000","last_fullsized":"2023-11-09T03:48:22.555079+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.554040+0000","last_change":"2023-11-09T03:48:22.554040+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.554040+0000","last_undegraded":"2023-11-09T03:48:22.554040+0000","last_fullsized":"2023-11-09T03:48:22.554040+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.553445+0000","last_change":"2023-11-09T03:48:22.553445+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.553445+0000","last_undegraded":"2023-11-09T03:48:22.553445+0000","last_fullsized":"2023-11-09T03:48:22.553445+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551333+0000","last_change":"2023-11-09T03:48:22.551333+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551333+0000","last_undegraded":"2023-11-09T03:48:22.551333+0000","last_fullsized":"2023-11-09T03:48:22.551333+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.553219+0000","last_change":"2023-11-09T03:48:22.553219+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.553219+0000","last_undegraded":"2023-11-09T03:48:22.553219+0000","last_fullsized":"2023-11-09T03:48:22.553219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.552812+0000","last_change":"2023-11-09T03:48:22.552812+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.552812+0000","last_undegraded":"2023-11-09T03:48:22.552812+0000","last_fullsized":"2023-11-09T03:48:22.552812+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.552951+0000","last_change":"2023-11-09T03:48:22.552951+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.552951+0000","last_undegraded":"2023-11-09T03:48:22.552951+0000","last_fullsized":"2023-11-09T03:48:22.552951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.552510+0000","last_change":"2023-11-09T03:48:22.552510+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.552510+0000","last_undegraded":"2023-11-09T03:48:22.552510+0000","last_fullsized":"2023-11-09T03:48:22.552510+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.552644+0000","last_change":"2023-11-09T03:48:22.552644+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.552644+0000","last_undegraded":"2023-11-09T03:48:22.552644+0000","last_fullsized":"2023-11-09T03:48:22.552644+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551900+0000","last_change":"2023-11-09T03:48:22.551900+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551900+0000","last_undegraded":"2023-11-09T03:48:22.551900+0000","last_fullsized":"2023-11-09T03:48:22.551900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549131+0000","last_change":"2023-11-09T03:48:22.549131+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549131+0000","last_undegraded":"2023-11-09T03:48:22.549131+0000","last_fullsized":"2023-11-09T03:48:22.549131+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.548919+0000","last_change":"2023-11-09T03:48:22.548919+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.548919+0000","last_undegraded":"2023-11-09T03:48:22.548919+0000","last_fullsized":"2023-11-09T03:48:22.548919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.554246+0000","last_change":"2023-11-09T03:48:22.554246+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.554246+0000","last_undegraded":"2023-11-09T03:48:22.554246+0000","last_fullsized":"2023-11-09T03:48:22.554246+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.568615+0000","last_change":"2023-11-09T03:48:22.562765+0000","last_active":"2023-11-09T03:48:22.568615+0000","last_peered":"2023-11-09T03:48:22.568615+0000","last_clean":"2023-11-09T03:48:22.568615+0000","last_became_active":"2023-11-09T03:48:22.562455+0000","last_became_peered":"2023-11-09T03:48:22.562455+0000","last_unstale":"2023-11-09T03:48:22.568615+0000","last_undegraded":"2023-11-09T03:48:22.568615+0000","last_fullsized":"2023-11-09T03:48:22.568615+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:55:13.543207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545671+0000","last_change":"2023-11-09T03:48:01.449077+0000","last_active":"2023-11-09T03:48:21.545671+0000","last_peered":"2023-11-09T03:48:21.545671+0000","last_clean":"2023-11-09T03:48:21.545671+0000","last_became_active":"2023-11-09T03:48:01.448806+0000","last_became_peered":"2023-11-09T03:48:01.448806+0000","last_unstale":"2023-11-09T03:48:21.545671+0000","last_undegraded":"2023-11-09T03:48:21.545671+0000","last_fullsized":"2023-11-09T03:48:21.545671+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:16:00.220961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542885+0000","last_change":"2023-11-09T03:48:02.611756+0000","last_active":"2023-11-09T03:48:21.542885+0000","last_peered":"2023-11-09T03:48:21.542885+0000","last_clean":"2023-11-09T03:48:21.542885+0000","last_became_active":"2023-11-09T03:48:00.445880+0000","last_became_peered":"2023-11-09T03:48:00.445880+0000","last_unstale":"2023-11-09T03:48:21.542885+0000","last_undegraded":"2023-11-09T03:48:21.542885+0000","last_fullsized":"2023-11-09T03:48:21.542885+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:15:35.281296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085514900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539506+0000","last_change":"2023-11-09T03:47:02.581917+0000","last_active":"2023-11-09T03:48:21.539506+0000","last_peered":"2023-11-09T03:48:21.539506+0000","last_clean":"2023-11-09T03:48:21.539506+0000","last_became_active":"2023-11-09T03:47:01.588082+0000","last_became_peered":"2023-11-09T03:47:01.588082+0000","last_unstale":"2023-11-09T03:48:21.539506+0000","last_undegraded":"2023-11-09T03:48:21.539506+0000","last_fullsized":"2023-11-09T03:48:21.539506+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:44:07.653557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034538899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.553249+0000","last_change":"2023-11-09T03:48:22.553249+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.553249+0000","last_undegraded":"2023-11-09T03:48:22.553249+0000","last_fullsized":"2023-11-09T03:48:22.553249+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.571401+0000","last_change":"2023-11-09T03:48:22.566342+0000","last_active":"2023-11-09T03:48:22.571401+0000","last_peered":"2023-11-09T03:48:22.571401+0000","last_clean":"2023-11-09T03:48:22.571401+0000","last_became_active":"2023-11-09T03:48:22.566028+0000","last_became_peered":"2023-11-09T03:48:22.566028+0000","last_unstale":"2023-11-09T03:48:22.571401+0000","last_undegraded":"2023-11-09T03:48:22.571401+0000","last_fullsized":"2023-11-09T03:48:22.571401+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:31.602666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542096+0000","last_change":"2023-11-09T03:48:01.456245+0000","last_active":"2023-11-09T03:48:21.542096+0000","last_peered":"2023-11-09T03:48:21.542096+0000","last_clean":"2023-11-09T03:48:21.542096+0000","last_became_active":"2023-11-09T03:48:01.455944+0000","last_became_peered":"2023-11-09T03:48:01.455944+0000","last_unstale":"2023-11-09T03:48:21.542096+0000","last_undegraded":"2023-11-09T03:48:21.542096+0000","last_fullsized":"2023-11-09T03:48:21.542096+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:06:14.849196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543120+0000","last_change":"2023-11-09T03:48:02.612341+0000","last_active":"2023-11-09T03:48:21.543120+0000","last_peered":"2023-11-09T03:48:21.543120+0000","last_clean":"2023-11-09T03:48:21.543120+0000","last_became_active":"2023-11-09T03:48:00.444426+0000","last_became_peered":"2023-11-09T03:48:00.444426+0000","last_unstale":"2023-11-09T03:48:21.543120+0000","last_undegraded":"2023-11-09T03:48:21.543120+0000","last_fullsized":"2023-11-09T03:48:21.543120+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:19:53.969861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010812420000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545670+0000","last_change":"2023-11-09T03:47:03.029063+0000","last_active":"2023-11-09T03:48:21.545670+0000","last_peered":"2023-11-09T03:48:21.545670+0000","last_clean":"2023-11-09T03:48:21.545670+0000","last_became_active":"2023-11-09T03:47:01.593582+0000","last_became_peered":"2023-11-09T03:47:01.593582+0000","last_unstale":"2023-11-09T03:48:21.545670+0000","last_undegraded":"2023-11-09T03:48:21.545670+0000","last_fullsized":"2023-11-09T03:48:21.545670+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:56:20.649937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041855100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.546617+0000","last_change":"2023-11-09T03:48:22.546617+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.546617+0000","last_undegraded":"2023-11-09T03:48:22.546617+0000","last_fullsized":"2023-11-09T03:48:22.546617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.564376+0000","last_change":"2023-11-09T03:48:22.562238+0000","last_active":"2023-11-09T03:48:22.564376+0000","last_peered":"2023-11-09T03:48:22.564376+0000","last_clean":"2023-11-09T03:48:22.564376+0000","last_became_active":"2023-11-09T03:48:22.561984+0000","last_became_peered":"2023-11-09T03:48:22.561984+0000","last_unstale":"2023-11-09T03:48:22.564376+0000","last_undegraded":"2023-11-09T03:48:22.564376+0000","last_fullsized":"2023-11-09T03:48:22.564376+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:28:06.557471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541062+0000","last_change":"2023-11-09T03:48:01.450271+0000","last_active":"2023-11-09T03:48:21.541062+0000","last_peered":"2023-11-09T03:48:21.541062+0000","last_clean":"2023-11-09T03:48:21.541062+0000","last_became_active":"2023-11-09T03:48:01.449998+0000","last_became_peered":"2023-11-09T03:48:01.449998+0000","last_unstale":"2023-11-09T03:48:21.541062+0000","last_undegraded":"2023-11-09T03:48:21.541062+0000","last_fullsized":"2023-11-09T03:48:21.541062+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:12:27.170650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545599+0000","last_change":"2023-11-09T03:48:02.852114+0000","last_active":"2023-11-09T03:48:21.545599+0000","last_peered":"2023-11-09T03:48:21.545599+0000","last_clean":"2023-11-09T03:48:21.545599+0000","last_became_active":"2023-11-09T03:48:00.447203+0000","last_became_peered":"2023-11-09T03:48:00.447203+0000","last_unstale":"2023-11-09T03:48:21.545599+0000","last_undegraded":"2023-11-09T03:48:21.545599+0000","last_fullsized":"2023-11-09T03:48:21.545599+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:11:36.826430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049189500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543495+0000","last_change":"2023-11-09T03:47:02.619336+0000","last_active":"2023-11-09T03:48:21.543495+0000","last_peered":"2023-11-09T03:48:21.543495+0000","last_clean":"2023-11-09T03:48:21.543495+0000","last_became_active":"2023-11-09T03:47:01.587749+0000","last_became_peered":"2023-11-09T03:47:01.587749+0000","last_unstale":"2023-11-09T03:48:21.543495+0000","last_undegraded":"2023-11-09T03:48:21.543495+0000","last_fullsized":"2023-11-09T03:48:21.543495+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063032099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551692+0000","last_change":"2023-11-09T03:48:22.551692+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551692+0000","last_undegraded":"2023-11-09T03:48:22.551692+0000","last_fullsized":"2023-11-09T03:48:22.551692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.559640+0000","last_change":"2023-11-09T03:48:22.557530+0000","last_active":"2023-11-09T03:48:22.559640+0000","last_peered":"2023-11-09T03:48:22.559640+0000","last_clean":"2023-11-09T03:48:22.559640+0000","last_became_active":"2023-11-09T03:48:22.556685+0000","last_became_peered":"2023-11-09T03:48:22.556685+0000","last_unstale":"2023-11-09T03:48:22.559640+0000","last_undegraded":"2023-11-09T03:48:22.559640+0000","last_fullsized":"2023-11-09T03:48:22.559640+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:06:49.685032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547581+0000","last_change":"2023-11-09T03:48:01.454147+0000","last_active":"2023-11-09T03:48:21.547581+0000","last_peered":"2023-11-09T03:48:21.547581+0000","last_clean":"2023-11-09T03:48:21.547581+0000","last_became_active":"2023-11-09T03:48:01.453868+0000","last_became_peered":"2023-11-09T03:48:01.453868+0000","last_unstale":"2023-11-09T03:48:21.547581+0000","last_undegraded":"2023-11-09T03:48:21.547581+0000","last_fullsized":"2023-11-09T03:48:21.547581+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:05:09.655318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542848+0000","last_change":"2023-11-09T03:48:02.612000+0000","last_active":"2023-11-09T03:48:21.542848+0000","last_peered":"2023-11-09T03:48:21.542848+0000","last_clean":"2023-11-09T03:48:21.542848+0000","last_became_active":"2023-11-09T03:48:00.441667+0000","last_became_peered":"2023-11-09T03:48:00.441667+0000","last_unstale":"2023-11-09T03:48:21.542848+0000","last_undegraded":"2023-11-09T03:48:21.542848+0000","last_fullsized":"2023-11-09T03:48:21.542848+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:12:26.164644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000966781,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539495+0000","last_change":"2023-11-09T03:47:02.581658+0000","last_active":"2023-11-09T03:48:21.539495+0000","last_peered":"2023-11-09T03:48:21.539495+0000","last_clean":"2023-11-09T03:48:21.539495+0000","last_became_active":"2023-11-09T03:47:01.588612+0000","last_became_peered":"2023-11-09T03:47:01.588612+0000","last_unstale":"2023-11-09T03:48:21.539495+0000","last_undegraded":"2023-11-09T03:48:21.539495+0000","last_fullsized":"2023-11-09T03:48:21.539495+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045238500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.547531+0000","last_change":"2023-11-09T03:48:22.547531+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.547531+0000","last_undegraded":"2023-11-09T03:48:22.547531+0000","last_fullsized":"2023-11-09T03:48:22.547531+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.560046+0000","last_change":"2023-11-09T03:48:22.557828+0000","last_active":"2023-11-09T03:48:22.560046+0000","last_peered":"2023-11-09T03:48:22.560046+0000","last_clean":"2023-11-09T03:48:22.560046+0000","last_became_active":"2023-11-09T03:48:22.557588+0000","last_became_peered":"2023-11-09T03:48:22.557588+0000","last_unstale":"2023-11-09T03:48:22.560046+0000","last_undegraded":"2023-11-09T03:48:22.560046+0000","last_fullsized":"2023-11-09T03:48:22.560046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:36:15.980235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545124+0000","last_change":"2023-11-09T03:48:01.449834+0000","last_active":"2023-11-09T03:48:21.545124+0000","last_peered":"2023-11-09T03:48:21.545124+0000","last_clean":"2023-11-09T03:48:21.545124+0000","last_became_active":"2023-11-09T03:48:01.449552+0000","last_became_peered":"2023-11-09T03:48:01.449552+0000","last_unstale":"2023-11-09T03:48:21.545124+0000","last_undegraded":"2023-11-09T03:48:21.545124+0000","last_fullsized":"2023-11-09T03:48:21.545124+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:11:37.832436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542114+0000","last_change":"2023-11-09T03:48:02.682197+0000","last_active":"2023-11-09T03:48:21.542114+0000","last_peered":"2023-11-09T03:48:21.542114+0000","last_clean":"2023-11-09T03:48:21.542114+0000","last_became_active":"2023-11-09T03:48:00.445616+0000","last_became_peered":"2023-11-09T03:48:00.445616+0000","last_unstale":"2023-11-09T03:48:21.542114+0000","last_undegraded":"2023-11-09T03:48:21.542114+0000","last_fullsized":"2023-11-09T03:48:21.542114+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:57:03.474084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000416494,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542622+0000","last_change":"2023-11-09T03:47:02.618828+0000","last_active":"2023-11-09T03:48:21.542622+0000","last_peered":"2023-11-09T03:48:21.542622+0000","last_clean":"2023-11-09T03:48:21.542622+0000","last_became_active":"2023-11-09T03:47:01.590679+0000","last_became_peered":"2023-11-09T03:47:01.590679+0000","last_unstale":"2023-11-09T03:48:21.542622+0000","last_undegraded":"2023-11-09T03:48:21.542622+0000","last_fullsized":"2023-11-09T03:48:21.542622+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:06:30.982987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070668,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.546525+0000","last_change":"2023-11-09T03:48:22.546525+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.546525+0000","last_undegraded":"2023-11-09T03:48:22.546525+0000","last_fullsized":"2023-11-09T03:48:22.546525+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.560446+0000","last_change":"2023-11-09T03:48:22.556590+0000","last_active":"2023-11-09T03:48:22.560446+0000","last_peered":"2023-11-09T03:48:22.560446+0000","last_clean":"2023-11-09T03:48:22.560446+0000","last_became_active":"2023-11-09T03:48:22.556295+0000","last_became_peered":"2023-11-09T03:48:22.556295+0000","last_unstale":"2023-11-09T03:48:22.560446+0000","last_undegraded":"2023-11-09T03:48:22.560446+0000","last_fullsized":"2023-11-09T03:48:22.560446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:09:09.041542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.546994+0000","last_change":"2023-11-09T03:48:01.447647+0000","last_active":"2023-11-09T03:48:21.546994+0000","last_peered":"2023-11-09T03:48:21.546994+0000","last_clean":"2023-11-09T03:48:21.546994+0000","last_became_active":"2023-11-09T03:48:01.447302+0000","last_became_peered":"2023-11-09T03:48:01.447302+0000","last_unstale":"2023-11-09T03:48:21.546994+0000","last_undegraded":"2023-11-09T03:48:21.546994+0000","last_fullsized":"2023-11-09T03:48:21.546994+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:08:47.931939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539407+0000","last_change":"2023-11-09T03:48:03.327731+0000","last_active":"2023-11-09T03:48:21.539407+0000","last_peered":"2023-11-09T03:48:21.539407+0000","last_clean":"2023-11-09T03:48:21.539407+0000","last_became_active":"2023-11-09T03:48:00.449660+0000","last_became_peered":"2023-11-09T03:48:00.449660+0000","last_unstale":"2023-11-09T03:48:21.539407+0000","last_undegraded":"2023-11-09T03:48:21.539407+0000","last_fullsized":"2023-11-09T03:48:21.539407+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:02:07.940908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037816599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.546874+0000","last_change":"2023-11-09T03:47:03.146528+0000","last_active":"2023-11-09T03:48:21.546874+0000","last_peered":"2023-11-09T03:48:21.546874+0000","last_clean":"2023-11-09T03:48:21.546874+0000","last_became_active":"2023-11-09T03:47:01.587060+0000","last_became_peered":"2023-11-09T03:47:01.587060+0000","last_unstale":"2023-11-09T03:48:21.546874+0000","last_undegraded":"2023-11-09T03:48:21.546874+0000","last_fullsized":"2023-11-09T03:48:21.546874+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:39:15.238496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079064999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.547973+0000","last_change":"2023-11-09T03:48:22.547973+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.547973+0000","last_undegraded":"2023-11-09T03:48:22.547973+0000","last_fullsized":"2023-11-09T03:48:22.547973+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.569334+0000","last_change":"2023-11-09T03:48:22.565708+0000","last_active":"2023-11-09T03:48:22.569334+0000","last_peered":"2023-11-09T03:48:22.569334+0000","last_clean":"2023-11-09T03:48:22.569334+0000","last_became_active":"2023-11-09T03:48:22.563961+0000","last_became_peered":"2023-11-09T03:48:22.563961+0000","last_unstale":"2023-11-09T03:48:22.569334+0000","last_undegraded":"2023-11-09T03:48:22.569334+0000","last_fullsized":"2023-11-09T03:48:22.569334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:03:03.041986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539139+0000","last_change":"2023-11-09T03:48:01.453939+0000","last_active":"2023-11-09T03:48:21.539139+0000","last_peered":"2023-11-09T03:48:21.539139+0000","last_clean":"2023-11-09T03:48:21.539139+0000","last_became_active":"2023-11-09T03:48:01.453600+0000","last_became_peered":"2023-11-09T03:48:01.453600+0000","last_unstale":"2023-11-09T03:48:21.539139+0000","last_undegraded":"2023-11-09T03:48:21.539139+0000","last_fullsized":"2023-11-09T03:48:21.539139+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:45:10.522254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547099+0000","last_change":"2023-11-09T03:48:02.440910+0000","last_active":"2023-11-09T03:48:21.547099+0000","last_peered":"2023-11-09T03:48:21.547099+0000","last_clean":"2023-11-09T03:48:21.547099+0000","last_became_active":"2023-11-09T03:48:00.443791+0000","last_became_peered":"2023-11-09T03:48:00.443791+0000","last_unstale":"2023-11-09T03:48:21.547099+0000","last_undegraded":"2023-11-09T03:48:21.547099+0000","last_fullsized":"2023-11-09T03:48:21.547099+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:11:36.826430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069223400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547414+0000","last_change":"2023-11-09T03:47:03.148246+0000","last_active":"2023-11-09T03:48:21.547414+0000","last_peered":"2023-11-09T03:48:21.547414+0000","last_clean":"2023-11-09T03:48:21.547414+0000","last_became_active":"2023-11-09T03:47:01.590988+0000","last_became_peered":"2023-11-09T03:47:01.590988+0000","last_unstale":"2023-11-09T03:48:21.547414+0000","last_undegraded":"2023-11-09T03:48:21.547414+0000","last_fullsized":"2023-11-09T03:48:21.547414+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:07:20.785445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055651099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551090+0000","last_change":"2023-11-09T03:48:22.551090+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551090+0000","last_undegraded":"2023-11-09T03:48:22.551090+0000","last_fullsized":"2023-11-09T03:48:22.551090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.569365+0000","last_change":"2023-11-09T03:48:22.565920+0000","last_active":"2023-11-09T03:48:22.569365+0000","last_peered":"2023-11-09T03:48:22.569365+0000","last_clean":"2023-11-09T03:48:22.569365+0000","last_became_active":"2023-11-09T03:48:22.564592+0000","last_became_peered":"2023-11-09T03:48:22.564592+0000","last_unstale":"2023-11-09T03:48:22.569365+0000","last_undegraded":"2023-11-09T03:48:22.569365+0000","last_fullsized":"2023-11-09T03:48:22.569365+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:41:46.534565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539107+0000","last_change":"2023-11-09T03:48:01.450557+0000","last_active":"2023-11-09T03:48:21.539107+0000","last_peered":"2023-11-09T03:48:21.539107+0000","last_clean":"2023-11-09T03:48:21.539107+0000","last_became_active":"2023-11-09T03:48:01.450291+0000","last_became_peered":"2023-11-09T03:48:01.450291+0000","last_unstale":"2023-11-09T03:48:21.539107+0000","last_undegraded":"2023-11-09T03:48:21.539107+0000","last_fullsized":"2023-11-09T03:48:21.539107+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:21:29.699594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545002+0000","last_change":"2023-11-09T03:48:02.852010+0000","last_active":"2023-11-09T03:48:21.545002+0000","last_peered":"2023-11-09T03:48:21.545002+0000","last_clean":"2023-11-09T03:48:21.545002+0000","last_became_active":"2023-11-09T03:48:00.445267+0000","last_became_peered":"2023-11-09T03:48:00.445267+0000","last_unstale":"2023-11-09T03:48:21.545002+0000","last_undegraded":"2023-11-09T03:48:21.545002+0000","last_fullsized":"2023-11-09T03:48:21.545002+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:17.989628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064559199999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":50,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540983+0000","last_change":"2023-11-09T03:47:02.582754+0000","last_active":"2023-11-09T03:48:21.540983+0000","last_peered":"2023-11-09T03:48:21.540983+0000","last_clean":"2023-11-09T03:48:21.540983+0000","last_became_active":"2023-11-09T03:47:01.589715+0000","last_became_peered":"2023-11-09T03:47:01.589715+0000","last_unstale":"2023-11-09T03:48:21.540983+0000","last_undegraded":"2023-11-09T03:48:21.540983+0000","last_fullsized":"2023-11-09T03:48:21.540983+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099342799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.546539+0000","last_change":"2023-11-09T03:48:22.546539+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.546539+0000","last_undegraded":"2023-11-09T03:48:22.546539+0000","last_fullsized":"2023-11-09T03:48:22.546539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.571909+0000","last_change":"2023-11-09T03:48:22.566859+0000","last_active":"2023-11-09T03:48:22.571909+0000","last_peered":"2023-11-09T03:48:22.571909+0000","last_clean":"2023-11-09T03:48:22.571909+0000","last_became_active":"2023-11-09T03:48:22.566080+0000","last_became_peered":"2023-11-09T03:48:22.566080+0000","last_unstale":"2023-11-09T03:48:22.571909+0000","last_undegraded":"2023-11-09T03:48:22.571909+0000","last_fullsized":"2023-11-09T03:48:22.571909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:14:56.468255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543312+0000","last_change":"2023-11-09T03:48:01.452456+0000","last_active":"2023-11-09T03:48:21.543312+0000","last_peered":"2023-11-09T03:48:21.543312+0000","last_clean":"2023-11-09T03:48:21.543312+0000","last_became_active":"2023-11-09T03:48:01.452059+0000","last_became_peered":"2023-11-09T03:48:01.452059+0000","last_unstale":"2023-11-09T03:48:21.543312+0000","last_undegraded":"2023-11-09T03:48:21.543312+0000","last_fullsized":"2023-11-09T03:48:21.543312+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:57:56.855869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541181+0000","last_change":"2023-11-09T03:48:02.442557+0000","last_active":"2023-11-09T03:48:21.541181+0000","last_peered":"2023-11-09T03:48:21.541181+0000","last_clean":"2023-11-09T03:48:21.541181+0000","last_became_active":"2023-11-09T03:48:00.444823+0000","last_became_peered":"2023-11-09T03:48:00.444823+0000","last_unstale":"2023-11-09T03:48:21.541181+0000","last_undegraded":"2023-11-09T03:48:21.541181+0000","last_fullsized":"2023-11-09T03:48:21.541181+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:19:53.969861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075076299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":47,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545453+0000","last_change":"2023-11-09T03:47:02.724896+0000","last_active":"2023-11-09T03:48:21.545453+0000","last_peered":"2023-11-09T03:48:21.545453+0000","last_clean":"2023-11-09T03:48:21.545453+0000","last_became_active":"2023-11-09T03:47:01.592784+0000","last_became_peered":"2023-11-09T03:47:01.592784+0000","last_unstale":"2023-11-09T03:48:21.545453+0000","last_undegraded":"2023-11-09T03:48:21.545453+0000","last_fullsized":"2023-11-09T03:48:21.545453+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:51:32.016204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056683500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.548469+0000","last_change":"2023-11-09T03:48:22.548469+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.548469+0000","last_undegraded":"2023-11-09T03:48:22.548469+0000","last_fullsized":"2023-11-09T03:48:22.548469+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.571850+0000","last_change":"2023-11-09T03:48:22.565279+0000","last_active":"2023-11-09T03:48:22.571850+0000","last_peered":"2023-11-09T03:48:22.571850+0000","last_clean":"2023-11-09T03:48:22.571850+0000","last_became_active":"2023-11-09T03:48:22.564998+0000","last_became_peered":"2023-11-09T03:48:22.564998+0000","last_unstale":"2023-11-09T03:48:22.571850+0000","last_undegraded":"2023-11-09T03:48:22.571850+0000","last_fullsized":"2023-11-09T03:48:22.571850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:24:48.220122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540896+0000","last_change":"2023-11-09T03:48:01.452167+0000","last_active":"2023-11-09T03:48:21.540896+0000","last_peered":"2023-11-09T03:48:21.540896+0000","last_clean":"2023-11-09T03:48:21.540896+0000","last_became_active":"2023-11-09T03:48:01.451780+0000","last_became_peered":"2023-11-09T03:48:01.451780+0000","last_unstale":"2023-11-09T03:48:21.540896+0000","last_undegraded":"2023-11-09T03:48:21.540896+0000","last_fullsized":"2023-11-09T03:48:21.540896+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:15:36.287302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543137+0000","last_change":"2023-11-09T03:48:02.612246+0000","last_active":"2023-11-09T03:48:21.543137+0000","last_peered":"2023-11-09T03:48:21.543137+0000","last_clean":"2023-11-09T03:48:21.543137+0000","last_became_active":"2023-11-09T03:48:00.444629+0000","last_became_peered":"2023-11-09T03:48:00.444629+0000","last_unstale":"2023-11-09T03:48:21.543137+0000","last_undegraded":"2023-11-09T03:48:21.543137+0000","last_fullsized":"2023-11-09T03:48:21.543137+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:51:17.754054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098167299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541889+0000","last_change":"2023-11-09T03:47:03.146527+0000","last_active":"2023-11-09T03:48:21.541889+0000","last_peered":"2023-11-09T03:48:21.541889+0000","last_clean":"2023-11-09T03:48:21.541889+0000","last_became_active":"2023-11-09T03:47:01.587858+0000","last_became_peered":"2023-11-09T03:47:01.587858+0000","last_unstale":"2023-11-09T03:48:21.541889+0000","last_undegraded":"2023-11-09T03:48:21.541889+0000","last_fullsized":"2023-11-09T03:48:21.541889+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:56:04.201126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084439599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.548328+0000","last_change":"2023-11-09T03:48:22.548328+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.548328+0000","last_undegraded":"2023-11-09T03:48:22.548328+0000","last_fullsized":"2023-11-09T03:48:22.548328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.558055+0000","last_change":"2023-11-09T03:48:22.553920+0000","last_active":"2023-11-09T03:48:22.558055+0000","last_peered":"2023-11-09T03:48:22.558055+0000","last_clean":"2023-11-09T03:48:22.558055+0000","last_became_active":"2023-11-09T03:48:22.552644+0000","last_became_peered":"2023-11-09T03:48:22.552644+0000","last_unstale":"2023-11-09T03:48:22.558055+0000","last_undegraded":"2023-11-09T03:48:22.558055+0000","last_fullsized":"2023-11-09T03:48:22.558055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:59.646501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545255+0000","last_change":"2023-11-09T03:48:01.450685+0000","last_active":"2023-11-09T03:48:21.545255+0000","last_peered":"2023-11-09T03:48:21.545255+0000","last_clean":"2023-11-09T03:48:21.545255+0000","last_became_active":"2023-11-09T03:48:01.450445+0000","last_became_peered":"2023-11-09T03:48:01.450445+0000","last_unstale":"2023-11-09T03:48:21.545255+0000","last_undegraded":"2023-11-09T03:48:21.545255+0000","last_fullsized":"2023-11-09T03:48:21.545255+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:16:00.220961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547739+0000","last_change":"2023-11-09T03:48:02.441928+0000","last_active":"2023-11-09T03:48:21.547739+0000","last_peered":"2023-11-09T03:48:21.547739+0000","last_clean":"2023-11-09T03:48:21.547739+0000","last_became_active":"2023-11-09T03:48:00.444417+0000","last_became_peered":"2023-11-09T03:48:00.444417+0000","last_unstale":"2023-11-09T03:48:21.547739+0000","last_undegraded":"2023-11-09T03:48:21.547739+0000","last_fullsized":"2023-11-09T03:48:21.547739+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:55.450096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072428900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":47,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545707+0000","last_change":"2023-11-09T03:47:02.725223+0000","last_active":"2023-11-09T03:48:21.545707+0000","last_peered":"2023-11-09T03:48:21.545707+0000","last_clean":"2023-11-09T03:48:21.545707+0000","last_became_active":"2023-11-09T03:47:01.590635+0000","last_became_peered":"2023-11-09T03:47:01.590635+0000","last_unstale":"2023-11-09T03:48:21.545707+0000","last_undegraded":"2023-11-09T03:48:21.545707+0000","last_fullsized":"2023-11-09T03:48:21.545707+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:05.754357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051685100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551472+0000","last_change":"2023-11-09T03:48:22.551472+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551472+0000","last_undegraded":"2023-11-09T03:48:22.551472+0000","last_fullsized":"2023-11-09T03:48:22.551472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.569184+0000","last_change":"2023-11-09T03:48:22.563841+0000","last_active":"2023-11-09T03:48:22.569184+0000","last_peered":"2023-11-09T03:48:22.569184+0000","last_clean":"2023-11-09T03:48:22.569184+0000","last_became_active":"2023-11-09T03:48:22.563590+0000","last_became_peered":"2023-11-09T03:48:22.563590+0000","last_unstale":"2023-11-09T03:48:22.569184+0000","last_undegraded":"2023-11-09T03:48:22.569184+0000","last_fullsized":"2023-11-09T03:48:22.569184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:10:36.225903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542779+0000","last_change":"2023-11-09T03:48:01.453143+0000","last_active":"2023-11-09T03:48:21.542779+0000","last_peered":"2023-11-09T03:48:21.542779+0000","last_clean":"2023-11-09T03:48:21.542779+0000","last_became_active":"2023-11-09T03:48:01.452873+0000","last_became_peered":"2023-11-09T03:48:01.452873+0000","last_unstale":"2023-11-09T03:48:21.542779+0000","last_undegraded":"2023-11-09T03:48:21.542779+0000","last_fullsized":"2023-11-09T03:48:21.542779+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:06:28.575429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.548086+0000","last_change":"2023-11-09T03:48:02.442416+0000","last_active":"2023-11-09T03:48:21.548086+0000","last_peered":"2023-11-09T03:48:21.548086+0000","last_clean":"2023-11-09T03:48:21.548086+0000","last_became_active":"2023-11-09T03:48:00.443765+0000","last_became_peered":"2023-11-09T03:48:00.443765+0000","last_unstale":"2023-11-09T03:48:21.548086+0000","last_undegraded":"2023-11-09T03:48:21.548086+0000","last_fullsized":"2023-11-09T03:48:21.548086+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:41:22.321811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012468850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":50,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541230+0000","last_change":"2023-11-09T03:47:02.582739+0000","last_active":"2023-11-09T03:48:21.541230+0000","last_peered":"2023-11-09T03:48:21.541230+0000","last_clean":"2023-11-09T03:48:21.541230+0000","last_became_active":"2023-11-09T03:47:01.592743+0000","last_became_peered":"2023-11-09T03:47:01.592743+0000","last_unstale":"2023-11-09T03:48:21.541230+0000","last_undegraded":"2023-11-09T03:48:21.541230+0000","last_fullsized":"2023-11-09T03:48:21.541230+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051802700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.546563+0000","last_change":"2023-11-09T03:48:22.546563+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.546563+0000","last_undegraded":"2023-11-09T03:48:22.546563+0000","last_fullsized":"2023-11-09T03:48:22.546563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.561070+0000","last_change":"2023-11-09T03:48:22.559947+0000","last_active":"2023-11-09T03:48:22.561070+0000","last_peered":"2023-11-09T03:48:22.561070+0000","last_clean":"2023-11-09T03:48:22.561070+0000","last_became_active":"2023-11-09T03:48:22.559667+0000","last_became_peered":"2023-11-09T03:48:22.559667+0000","last_unstale":"2023-11-09T03:48:22.561070+0000","last_undegraded":"2023-11-09T03:48:22.561070+0000","last_fullsized":"2023-11-09T03:48:22.561070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:57.844631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540463+0000","last_change":"2023-11-09T03:48:01.442195+0000","last_active":"2023-11-09T03:48:21.540463+0000","last_peered":"2023-11-09T03:48:21.540463+0000","last_clean":"2023-11-09T03:48:21.540463+0000","last_became_active":"2023-11-09T03:48:01.441887+0000","last_became_peered":"2023-11-09T03:48:01.441887+0000","last_unstale":"2023-11-09T03:48:21.540463+0000","last_undegraded":"2023-11-09T03:48:21.540463+0000","last_fullsized":"2023-11-09T03:48:21.540463+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:06:28.575429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540528+0000","last_change":"2023-11-09T03:48:02.442061+0000","last_active":"2023-11-09T03:48:21.540528+0000","last_peered":"2023-11-09T03:48:21.540528+0000","last_clean":"2023-11-09T03:48:21.540528+0000","last_became_active":"2023-11-09T03:48:00.445921+0000","last_became_peered":"2023-11-09T03:48:00.445921+0000","last_unstale":"2023-11-09T03:48:21.540528+0000","last_undegraded":"2023-11-09T03:48:21.540528+0000","last_fullsized":"2023-11-09T03:48:21.540528+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:50:55.299235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012719770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":47,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545111+0000","last_change":"2023-11-09T03:47:02.724953+0000","last_active":"2023-11-09T03:48:21.545111+0000","last_peered":"2023-11-09T03:48:21.545111+0000","last_clean":"2023-11-09T03:48:21.545111+0000","last_became_active":"2023-11-09T03:47:01.587133+0000","last_became_peered":"2023-11-09T03:47:01.587133+0000","last_unstale":"2023-11-09T03:48:21.545111+0000","last_undegraded":"2023-11-09T03:48:21.545111+0000","last_fullsized":"2023-11-09T03:48:21.545111+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:23:48.521161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066497599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.547867+0000","last_change":"2023-11-09T03:48:22.547867+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.547867+0000","last_undegraded":"2023-11-09T03:48:22.547867+0000","last_fullsized":"2023-11-09T03:48:22.547867+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.563196+0000","last_change":"2023-11-09T03:48:22.556449+0000","last_active":"2023-11-09T03:48:22.563196+0000","last_peered":"2023-11-09T03:48:22.563196+0000","last_clean":"2023-11-09T03:48:22.563196+0000","last_became_active":"2023-11-09T03:48:22.556170+0000","last_became_peered":"2023-11-09T03:48:22.556170+0000","last_unstale":"2023-11-09T03:48:22.563196+0000","last_undegraded":"2023-11-09T03:48:22.563196+0000","last_fullsized":"2023-11-09T03:48:22.563196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:09:09.041542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539364+0000","last_change":"2023-11-09T03:48:01.450507+0000","last_active":"2023-11-09T03:48:21.539364+0000","last_peered":"2023-11-09T03:48:21.539364+0000","last_clean":"2023-11-09T03:48:21.539364+0000","last_became_active":"2023-11-09T03:48:01.450191+0000","last_became_peered":"2023-11-09T03:48:01.450191+0000","last_unstale":"2023-11-09T03:48:21.539364+0000","last_undegraded":"2023-11-09T03:48:21.539364+0000","last_fullsized":"2023-11-09T03:48:21.539364+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:32:40.342502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540558+0000","last_change":"2023-11-09T03:48:02.442120+0000","last_active":"2023-11-09T03:48:21.540558+0000","last_peered":"2023-11-09T03:48:21.540558+0000","last_clean":"2023-11-09T03:48:21.540558+0000","last_became_active":"2023-11-09T03:48:00.445504+0000","last_became_peered":"2023-11-09T03:48:00.445504+0000","last_unstale":"2023-11-09T03:48:21.540558+0000","last_undegraded":"2023-11-09T03:48:21.540558+0000","last_fullsized":"2023-11-09T03:48:21.540558+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:35:53.864627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001095766,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541998+0000","last_change":"2023-11-09T03:47:03.146217+0000","last_active":"2023-11-09T03:48:21.541998+0000","last_peered":"2023-11-09T03:48:21.541998+0000","last_clean":"2023-11-09T03:48:21.541998+0000","last_became_active":"2023-11-09T03:47:01.587598+0000","last_became_peered":"2023-11-09T03:47:01.587598+0000","last_unstale":"2023-11-09T03:48:21.541998+0000","last_undegraded":"2023-11-09T03:48:21.541998+0000","last_fullsized":"2023-11-09T03:48:21.541998+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:15:22.723455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090126400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550235+0000","last_change":"2023-11-09T03:48:22.550235+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550235+0000","last_undegraded":"2023-11-09T03:48:22.550235+0000","last_fullsized":"2023-11-09T03:48:22.550235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.564011+0000","last_change":"2023-11-09T03:48:22.561567+0000","last_active":"2023-11-09T03:48:22.564011+0000","last_peered":"2023-11-09T03:48:22.564011+0000","last_clean":"2023-11-09T03:48:22.564011+0000","last_became_active":"2023-11-09T03:48:22.561266+0000","last_became_peered":"2023-11-09T03:48:22.561266+0000","last_unstale":"2023-11-09T03:48:22.564011+0000","last_undegraded":"2023-11-09T03:48:22.564011+0000","last_fullsized":"2023-11-09T03:48:22.564011+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:21.472203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542405+0000","last_change":"2023-11-09T03:48:01.454878+0000","last_active":"2023-11-09T03:48:21.542405+0000","last_peered":"2023-11-09T03:48:21.542405+0000","last_clean":"2023-11-09T03:48:21.542405+0000","last_became_active":"2023-11-09T03:48:01.454591+0000","last_became_peered":"2023-11-09T03:48:01.454591+0000","last_unstale":"2023-11-09T03:48:21.542405+0000","last_undegraded":"2023-11-09T03:48:21.542405+0000","last_fullsized":"2023-11-09T03:48:21.542405+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:38.536898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539170+0000","last_change":"2023-11-09T03:48:03.327800+0000","last_active":"2023-11-09T03:48:21.539170+0000","last_peered":"2023-11-09T03:48:21.539170+0000","last_clean":"2023-11-09T03:48:21.539170+0000","last_became_active":"2023-11-09T03:48:00.446576+0000","last_became_peered":"2023-11-09T03:48:00.446576+0000","last_unstale":"2023-11-09T03:48:21.539170+0000","last_undegraded":"2023-11-09T03:48:21.539170+0000","last_fullsized":"2023-11-09T03:48:21.539170+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:37:08.851102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054679700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547427+0000","last_change":"2023-11-09T03:47:03.148283+0000","last_active":"2023-11-09T03:48:21.547427+0000","last_peered":"2023-11-09T03:48:21.547427+0000","last_clean":"2023-11-09T03:48:21.547427+0000","last_became_active":"2023-11-09T03:47:01.588841+0000","last_became_peered":"2023-11-09T03:47:01.588841+0000","last_unstale":"2023-11-09T03:48:21.547427+0000","last_undegraded":"2023-11-09T03:48:21.547427+0000","last_fullsized":"2023-11-09T03:48:21.547427+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:47:44.880741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000436081,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.552336+0000","last_change":"2023-11-09T03:48:22.552336+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.552336+0000","last_undegraded":"2023-11-09T03:48:22.552336+0000","last_fullsized":"2023-11-09T03:48:22.552336+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.558220+0000","last_change":"2023-11-09T03:48:22.556480+0000","last_active":"2023-11-09T03:48:22.558220+0000","last_peered":"2023-11-09T03:48:22.558220+0000","last_clean":"2023-11-09T03:48:22.558220+0000","last_became_active":"2023-11-09T03:48:22.556059+0000","last_became_peered":"2023-11-09T03:48:22.556059+0000","last_unstale":"2023-11-09T03:48:22.558220+0000","last_undegraded":"2023-11-09T03:48:22.558220+0000","last_fullsized":"2023-11-09T03:48:22.558220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:51:17.414844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.544682+0000","last_change":"2023-11-09T03:48:01.450581+0000","last_active":"2023-11-09T03:48:21.544682+0000","last_peered":"2023-11-09T03:48:21.544682+0000","last_clean":"2023-11-09T03:48:21.544682+0000","last_became_active":"2023-11-09T03:48:01.450280+0000","last_became_peered":"2023-11-09T03:48:01.450280+0000","last_unstale":"2023-11-09T03:48:21.544682+0000","last_undegraded":"2023-11-09T03:48:21.544682+0000","last_fullsized":"2023-11-09T03:48:21.544682+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:18.995634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.546874+0000","last_change":"2023-11-09T03:48:02.440934+0000","last_active":"2023-11-09T03:48:21.546874+0000","last_peered":"2023-11-09T03:48:21.546874+0000","last_clean":"2023-11-09T03:48:21.546874+0000","last_became_active":"2023-11-09T03:48:00.444394+0000","last_became_peered":"2023-11-09T03:48:00.444394+0000","last_unstale":"2023-11-09T03:48:21.546874+0000","last_undegraded":"2023-11-09T03:48:21.546874+0000","last_fullsized":"2023-11-09T03:48:21.546874+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:15:59.214955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063908300000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542547+0000","last_change":"2023-11-09T03:47:02.618856+0000","last_active":"2023-11-09T03:48:21.542547+0000","last_peered":"2023-11-09T03:48:21.542547+0000","last_clean":"2023-11-09T03:48:21.542547+0000","last_became_active":"2023-11-09T03:47:01.590430+0000","last_became_peered":"2023-11-09T03:47:01.590430+0000","last_unstale":"2023-11-09T03:48:21.542547+0000","last_undegraded":"2023-11-09T03:48:21.542547+0000","last_fullsized":"2023-11-09T03:48:21.542547+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:57:18.470305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055030200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550790+0000","last_change":"2023-11-09T03:48:22.550790+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550790+0000","last_undegraded":"2023-11-09T03:48:22.550790+0000","last_fullsized":"2023-11-09T03:48:22.550790+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.563239+0000","last_change":"2023-11-09T03:48:22.559687+0000","last_active":"2023-11-09T03:48:22.563239+0000","last_peered":"2023-11-09T03:48:22.563239+0000","last_clean":"2023-11-09T03:48:22.563239+0000","last_became_active":"2023-11-09T03:48:22.559404+0000","last_became_peered":"2023-11-09T03:48:22.559404+0000","last_unstale":"2023-11-09T03:48:22.563239+0000","last_undegraded":"2023-11-09T03:48:22.563239+0000","last_fullsized":"2023-11-09T03:48:22.563239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:39:53.462957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545586+0000","last_change":"2023-11-09T03:48:01.453600+0000","last_active":"2023-11-09T03:48:21.545586+0000","last_peered":"2023-11-09T03:48:21.545586+0000","last_clean":"2023-11-09T03:48:21.545586+0000","last_became_active":"2023-11-09T03:48:01.453330+0000","last_became_peered":"2023-11-09T03:48:01.453330+0000","last_unstale":"2023-11-09T03:48:21.545586+0000","last_undegraded":"2023-11-09T03:48:21.545586+0000","last_fullsized":"2023-11-09T03:48:21.545586+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:08:47.931939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539485+0000","last_change":"2023-11-09T03:48:03.328210+0000","last_active":"2023-11-09T03:48:21.539485+0000","last_peered":"2023-11-09T03:48:21.539485+0000","last_clean":"2023-11-09T03:48:21.539485+0000","last_became_active":"2023-11-09T03:48:00.450002+0000","last_became_peered":"2023-11-09T03:48:00.450002+0000","last_unstale":"2023-11-09T03:48:21.539485+0000","last_undegraded":"2023-11-09T03:48:21.539485+0000","last_fullsized":"2023-11-09T03:48:21.539485+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:50.391187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036521900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542921+0000","last_change":"2023-11-09T03:47:02.619144+0000","last_active":"2023-11-09T03:48:21.542921+0000","last_peered":"2023-11-09T03:48:21.542921+0000","last_clean":"2023-11-09T03:48:21.542921+0000","last_became_active":"2023-11-09T03:47:01.590672+0000","last_became_peered":"2023-11-09T03:47:01.590672+0000","last_unstale":"2023-11-09T03:48:21.542921+0000","last_undegraded":"2023-11-09T03:48:21.542921+0000","last_fullsized":"2023-11-09T03:48:21.542921+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:03.025119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060997500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.547130+0000","last_change":"2023-11-09T03:48:22.547130+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.547130+0000","last_undegraded":"2023-11-09T03:48:22.547130+0000","last_fullsized":"2023-11-09T03:48:22.547130+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.571874+0000","last_change":"2023-11-09T03:48:22.566419+0000","last_active":"2023-11-09T03:48:22.571874+0000","last_peered":"2023-11-09T03:48:22.571874+0000","last_clean":"2023-11-09T03:48:22.571874+0000","last_became_active":"2023-11-09T03:48:22.566144+0000","last_became_peered":"2023-11-09T03:48:22.566144+0000","last_unstale":"2023-11-09T03:48:22.571874+0000","last_undegraded":"2023-11-09T03:48:22.571874+0000","last_fullsized":"2023-11-09T03:48:22.571874+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:43:01.915119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541019+0000","last_change":"2023-11-09T03:48:01.447278+0000","last_active":"2023-11-09T03:48:21.541019+0000","last_peered":"2023-11-09T03:48:21.541019+0000","last_clean":"2023-11-09T03:48:21.541019+0000","last_became_active":"2023-11-09T03:48:01.446991+0000","last_became_peered":"2023-11-09T03:48:01.446991+0000","last_unstale":"2023-11-09T03:48:21.541019+0000","last_undegraded":"2023-11-09T03:48:21.541019+0000","last_fullsized":"2023-11-09T03:48:21.541019+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:27:49.190426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539429+0000","last_change":"2023-11-09T03:48:03.328278+0000","last_active":"2023-11-09T03:48:21.539429+0000","last_peered":"2023-11-09T03:48:21.539429+0000","last_clean":"2023-11-09T03:48:21.539429+0000","last_became_active":"2023-11-09T03:48:00.443213+0000","last_became_peered":"2023-11-09T03:48:00.443213+0000","last_unstale":"2023-11-09T03:48:21.539429+0000","last_undegraded":"2023-11-09T03:48:21.539429+0000","last_fullsized":"2023-11-09T03:48:21.539429+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:20:59.702297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030886100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543013+0000","last_change":"2023-11-09T03:47:02.619079+0000","last_active":"2023-11-09T03:48:21.543013+0000","last_peered":"2023-11-09T03:48:21.543013+0000","last_clean":"2023-11-09T03:48:21.543013+0000","last_became_active":"2023-11-09T03:47:01.591421+0000","last_became_peered":"2023-11-09T03:47:01.591421+0000","last_unstale":"2023-11-09T03:48:21.543013+0000","last_undegraded":"2023-11-09T03:48:21.543013+0000","last_fullsized":"2023-11-09T03:48:21.543013+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:18:58.230606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068736399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.547470+0000","last_change":"2023-11-09T03:48:22.547470+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.547470+0000","last_undegraded":"2023-11-09T03:48:22.547470+0000","last_fullsized":"2023-11-09T03:48:22.547470+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.574695+0000","last_change":"2023-11-09T03:48:22.569861+0000","last_active":"2023-11-09T03:48:22.574695+0000","last_peered":"2023-11-09T03:48:22.574695+0000","last_clean":"2023-11-09T03:48:22.574695+0000","last_became_active":"2023-11-09T03:48:22.569611+0000","last_became_peered":"2023-11-09T03:48:22.569611+0000","last_unstale":"2023-11-09T03:48:22.574695+0000","last_undegraded":"2023-11-09T03:48:22.574695+0000","last_fullsized":"2023-11-09T03:48:22.574695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:36:57.293292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541146+0000","last_change":"2023-11-09T03:48:01.449069+0000","last_active":"2023-11-09T03:48:21.541146+0000","last_peered":"2023-11-09T03:48:21.541146+0000","last_clean":"2023-11-09T03:48:21.541146+0000","last_became_active":"2023-11-09T03:48:01.448810+0000","last_became_peered":"2023-11-09T03:48:01.448810+0000","last_unstale":"2023-11-09T03:48:21.541146+0000","last_undegraded":"2023-11-09T03:48:21.541146+0000","last_fullsized":"2023-11-09T03:48:21.541146+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:56:51.158572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542867+0000","last_change":"2023-11-09T03:48:02.612067+0000","last_active":"2023-11-09T03:48:21.542867+0000","last_peered":"2023-11-09T03:48:21.542867+0000","last_clean":"2023-11-09T03:48:21.542867+0000","last_became_active":"2023-11-09T03:48:00.443296+0000","last_became_peered":"2023-11-09T03:48:00.443296+0000","last_unstale":"2023-11-09T03:48:21.542867+0000","last_undegraded":"2023-11-09T03:48:21.542867+0000","last_fullsized":"2023-11-09T03:48:21.542867+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:27:48.184420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010423330000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547943+0000","last_change":"2023-11-09T03:47:03.148529+0000","last_active":"2023-11-09T03:48:21.547943+0000","last_peered":"2023-11-09T03:48:21.547943+0000","last_clean":"2023-11-09T03:48:21.547943+0000","last_became_active":"2023-11-09T03:47:01.584148+0000","last_became_peered":"2023-11-09T03:47:01.584148+0000","last_unstale":"2023-11-09T03:48:21.547943+0000","last_undegraded":"2023-11-09T03:48:21.547943+0000","last_fullsized":"2023-11-09T03:48:21.547943+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:40:00.237248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061475600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550722+0000","last_change":"2023-11-09T03:48:22.550722+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550722+0000","last_undegraded":"2023-11-09T03:48:22.550722+0000","last_fullsized":"2023-11-09T03:48:22.550722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.563077+0000","last_change":"2023-11-09T03:48:22.558072+0000","last_active":"2023-11-09T03:48:22.563077+0000","last_peered":"2023-11-09T03:48:22.563077+0000","last_clean":"2023-11-09T03:48:22.563077+0000","last_became_active":"2023-11-09T03:48:22.557800+0000","last_became_peered":"2023-11-09T03:48:22.557800+0000","last_unstale":"2023-11-09T03:48:22.563077+0000","last_undegraded":"2023-11-09T03:48:22.563077+0000","last_fullsized":"2023-11-09T03:48:22.563077+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:00:20.811244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542948+0000","last_change":"2023-11-09T03:48:01.451750+0000","last_active":"2023-11-09T03:48:21.542948+0000","last_peered":"2023-11-09T03:48:21.542948+0000","last_clean":"2023-11-09T03:48:21.542948+0000","last_became_active":"2023-11-09T03:48:01.451393+0000","last_became_peered":"2023-11-09T03:48:01.451393+0000","last_unstale":"2023-11-09T03:48:21.542948+0000","last_undegraded":"2023-11-09T03:48:21.542948+0000","last_fullsized":"2023-11-09T03:48:21.542948+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:56:51.158572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542743+0000","last_change":"2023-11-09T03:48:02.611721+0000","last_active":"2023-11-09T03:48:21.542743+0000","last_peered":"2023-11-09T03:48:21.542743+0000","last_clean":"2023-11-09T03:48:21.542743+0000","last_became_active":"2023-11-09T03:48:00.439557+0000","last_became_peered":"2023-11-09T03:48:00.439557+0000","last_unstale":"2023-11-09T03:48:21.542743+0000","last_undegraded":"2023-11-09T03:48:21.542743+0000","last_fullsized":"2023-11-09T03:48:21.542743+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:35:53.864627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001037187,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543088+0000","last_change":"2023-11-09T03:47:03.148217+0000","last_active":"2023-11-09T03:48:21.543088+0000","last_peered":"2023-11-09T03:48:21.543088+0000","last_clean":"2023-11-09T03:48:21.543088+0000","last_became_active":"2023-11-09T03:47:01.593424+0000","last_became_peered":"2023-11-09T03:47:01.593424+0000","last_unstale":"2023-11-09T03:48:21.543088+0000","last_undegraded":"2023-11-09T03:48:21.543088+0000","last_fullsized":"2023-11-09T03:48:21.543088+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:19:32.693691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001769865,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551131+0000","last_change":"2023-11-09T03:48:22.551131+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551131+0000","last_undegraded":"2023-11-09T03:48:22.551131+0000","last_fullsized":"2023-11-09T03:48:22.551131+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.569383+0000","last_change":"2023-11-09T03:48:22.566073+0000","last_active":"2023-11-09T03:48:22.569383+0000","last_peered":"2023-11-09T03:48:22.569383+0000","last_clean":"2023-11-09T03:48:22.569383+0000","last_became_active":"2023-11-09T03:48:22.564106+0000","last_became_peered":"2023-11-09T03:48:22.564106+0000","last_unstale":"2023-11-09T03:48:22.569383+0000","last_undegraded":"2023-11-09T03:48:22.569383+0000","last_fullsized":"2023-11-09T03:48:22.569383+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:05:06.931697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547292+0000","last_change":"2023-11-09T03:48:01.447617+0000","last_active":"2023-11-09T03:48:21.547292+0000","last_peered":"2023-11-09T03:48:21.547292+0000","last_clean":"2023-11-09T03:48:21.547292+0000","last_became_active":"2023-11-09T03:48:01.447329+0000","last_became_peered":"2023-11-09T03:48:01.447329+0000","last_unstale":"2023-11-09T03:48:21.547292+0000","last_undegraded":"2023-11-09T03:48:21.547292+0000","last_fullsized":"2023-11-09T03:48:21.547292+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:54:52.433604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540889+0000","last_change":"2023-11-09T03:48:02.442427+0000","last_active":"2023-11-09T03:48:21.540889+0000","last_peered":"2023-11-09T03:48:21.540889+0000","last_clean":"2023-11-09T03:48:21.540889+0000","last_became_active":"2023-11-09T03:48:00.444856+0000","last_became_peered":"2023-11-09T03:48:00.444856+0000","last_unstale":"2023-11-09T03:48:21.540889+0000","last_undegraded":"2023-11-09T03:48:21.540889+0000","last_fullsized":"2023-11-09T03:48:21.540889+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:53:42.763434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011927730000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.546971+0000","last_change":"2023-11-09T03:47:03.146470+0000","last_active":"2023-11-09T03:48:21.546971+0000","last_peered":"2023-11-09T03:48:21.546971+0000","last_clean":"2023-11-09T03:48:21.546971+0000","last_became_active":"2023-11-09T03:47:01.587304+0000","last_became_peered":"2023-11-09T03:47:01.587304+0000","last_unstale":"2023-11-09T03:48:21.546971+0000","last_undegraded":"2023-11-09T03:48:21.546971+0000","last_fullsized":"2023-11-09T03:48:21.546971+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:58:23.339287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00110395,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.548843+0000","last_change":"2023-11-09T03:48:22.548843+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.548843+0000","last_undegraded":"2023-11-09T03:48:22.548843+0000","last_fullsized":"2023-11-09T03:48:22.548843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.564450+0000","last_change":"2023-11-09T03:48:22.561952+0000","last_active":"2023-11-09T03:48:22.564450+0000","last_peered":"2023-11-09T03:48:22.564450+0000","last_clean":"2023-11-09T03:48:22.564450+0000","last_became_active":"2023-11-09T03:48:22.561647+0000","last_became_peered":"2023-11-09T03:48:22.561647+0000","last_unstale":"2023-11-09T03:48:22.564450+0000","last_undegraded":"2023-11-09T03:48:22.564450+0000","last_fullsized":"2023-11-09T03:48:22.564450+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:52.879173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545482+0000","last_change":"2023-11-09T03:48:01.449406+0000","last_active":"2023-11-09T03:48:21.545482+0000","last_peered":"2023-11-09T03:48:21.545482+0000","last_clean":"2023-11-09T03:48:21.545482+0000","last_became_active":"2023-11-09T03:48:01.449130+0000","last_became_peered":"2023-11-09T03:48:01.449130+0000","last_unstale":"2023-11-09T03:48:21.545482+0000","last_undegraded":"2023-11-09T03:48:21.545482+0000","last_fullsized":"2023-11-09T03:48:21.545482+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:20:36.303940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547362+0000","last_change":"2023-11-09T03:48:02.441801+0000","last_active":"2023-11-09T03:48:21.547362+0000","last_peered":"2023-11-09T03:48:21.547362+0000","last_clean":"2023-11-09T03:48:21.547362+0000","last_became_active":"2023-11-09T03:48:00.444975+0000","last_became_peered":"2023-11-09T03:48:00.444975+0000","last_unstale":"2023-11-09T03:48:21.547362+0000","last_undegraded":"2023-11-09T03:48:21.547362+0000","last_fullsized":"2023-11-09T03:48:21.547362+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:39:31.347349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061165099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542743+0000","last_change":"2023-11-09T03:47:03.147367+0000","last_active":"2023-11-09T03:48:21.542743+0000","last_peered":"2023-11-09T03:48:21.542743+0000","last_clean":"2023-11-09T03:48:21.542743+0000","last_became_active":"2023-11-09T03:47:01.588359+0000","last_became_peered":"2023-11-09T03:47:01.588359+0000","last_unstale":"2023-11-09T03:48:21.542743+0000","last_undegraded":"2023-11-09T03:48:21.542743+0000","last_fullsized":"2023-11-09T03:48:21.542743+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:21:34.303068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014106660000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.546731+0000","last_change":"2023-11-09T03:48:22.546731+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.546731+0000","last_undegraded":"2023-11-09T03:48:22.546731+0000","last_fullsized":"2023-11-09T03:48:22.546731+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.563046+0000","last_change":"2023-11-09T03:48:22.556245+0000","last_active":"2023-11-09T03:48:22.563046+0000","last_peered":"2023-11-09T03:48:22.563046+0000","last_clean":"2023-11-09T03:48:22.563046+0000","last_became_active":"2023-11-09T03:48:22.555974+0000","last_became_peered":"2023-11-09T03:48:22.555974+0000","last_unstale":"2023-11-09T03:48:22.563046+0000","last_undegraded":"2023-11-09T03:48:22.563046+0000","last_fullsized":"2023-11-09T03:48:22.563046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:55:13.543207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542383+0000","last_change":"2023-11-09T03:48:01.451866+0000","last_active":"2023-11-09T03:48:21.542383+0000","last_peered":"2023-11-09T03:48:21.542383+0000","last_clean":"2023-11-09T03:48:21.542383+0000","last_became_active":"2023-11-09T03:48:01.451350+0000","last_became_peered":"2023-11-09T03:48:01.451350+0000","last_unstale":"2023-11-09T03:48:21.542383+0000","last_undegraded":"2023-11-09T03:48:21.542383+0000","last_fullsized":"2023-11-09T03:48:21.542383+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:36.735029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540171+0000","last_change":"2023-11-09T03:48:02.441648+0000","last_active":"2023-11-09T03:48:21.540171+0000","last_peered":"2023-11-09T03:48:21.540171+0000","last_clean":"2023-11-09T03:48:21.540171+0000","last_became_active":"2023-11-09T03:48:00.448541+0000","last_became_peered":"2023-11-09T03:48:00.448541+0000","last_unstale":"2023-11-09T03:48:21.540171+0000","last_undegraded":"2023-11-09T03:48:21.540171+0000","last_fullsized":"2023-11-09T03:48:21.540171+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:35.729023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539334+0000","last_change":"2023-11-09T03:47:02.581682+0000","last_active":"2023-11-09T03:48:21.539334+0000","last_peered":"2023-11-09T03:48:21.539334+0000","last_clean":"2023-11-09T03:48:21.539334+0000","last_became_active":"2023-11-09T03:47:01.593051+0000","last_became_peered":"2023-11-09T03:47:01.593051+0000","last_unstale":"2023-11-09T03:48:21.539334+0000","last_undegraded":"2023-11-09T03:48:21.539334+0000","last_fullsized":"2023-11-09T03:48:21.539334+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047277799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549020+0000","last_change":"2023-11-09T03:48:22.549020+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549020+0000","last_undegraded":"2023-11-09T03:48:22.549020+0000","last_fullsized":"2023-11-09T03:48:22.549020+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.569210+0000","last_change":"2023-11-09T03:48:22.563908+0000","last_active":"2023-11-09T03:48:22.569210+0000","last_peered":"2023-11-09T03:48:22.569210+0000","last_clean":"2023-11-09T03:48:22.569210+0000","last_became_active":"2023-11-09T03:48:22.563651+0000","last_became_peered":"2023-11-09T03:48:22.563651+0000","last_unstale":"2023-11-09T03:48:22.569210+0000","last_undegraded":"2023-11-09T03:48:22.569210+0000","last_fullsized":"2023-11-09T03:48:22.569210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:52.879173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.544927+0000","last_change":"2023-11-09T03:48:01.455784+0000","last_active":"2023-11-09T03:48:21.544927+0000","last_peered":"2023-11-09T03:48:21.544927+0000","last_clean":"2023-11-09T03:48:21.544927+0000","last_became_active":"2023-11-09T03:48:01.455532+0000","last_became_peered":"2023-11-09T03:48:01.455532+0000","last_unstale":"2023-11-09T03:48:21.544927+0000","last_undegraded":"2023-11-09T03:48:21.544927+0000","last_fullsized":"2023-11-09T03:48:21.544927+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:21:23.156761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542535+0000","last_change":"2023-11-09T03:48:02.611298+0000","last_active":"2023-11-09T03:48:21.542535+0000","last_peered":"2023-11-09T03:48:21.542535+0000","last_clean":"2023-11-09T03:48:21.542535+0000","last_became_active":"2023-11-09T03:48:00.441261+0000","last_became_peered":"2023-11-09T03:48:00.441261+0000","last_unstale":"2023-11-09T03:48:21.542535+0000","last_undegraded":"2023-11-09T03:48:21.542535+0000","last_fullsized":"2023-11-09T03:48:21.542535+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:50:55.299235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081460199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542200+0000","last_change":"2023-11-09T03:47:03.146309+0000","last_active":"2023-11-09T03:48:21.542200+0000","last_peered":"2023-11-09T03:48:21.542200+0000","last_clean":"2023-11-09T03:48:21.542200+0000","last_became_active":"2023-11-09T03:47:01.587560+0000","last_became_peered":"2023-11-09T03:47:01.587560+0000","last_unstale":"2023-11-09T03:48:21.542200+0000","last_undegraded":"2023-11-09T03:48:21.542200+0000","last_fullsized":"2023-11-09T03:48:21.542200+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:27:55.271945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070988400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551507+0000","last_change":"2023-11-09T03:48:22.551507+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551507+0000","last_undegraded":"2023-11-09T03:48:22.551507+0000","last_fullsized":"2023-11-09T03:48:22.551507+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.563543+0000","last_change":"2023-11-09T03:48:22.561557+0000","last_active":"2023-11-09T03:48:22.563543+0000","last_peered":"2023-11-09T03:48:22.563543+0000","last_clean":"2023-11-09T03:48:22.563543+0000","last_became_active":"2023-11-09T03:48:22.561249+0000","last_became_peered":"2023-11-09T03:48:22.561249+0000","last_unstale":"2023-11-09T03:48:22.563543+0000","last_undegraded":"2023-11-09T03:48:22.563543+0000","last_fullsized":"2023-11-09T03:48:22.563543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:15:26.050548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543219+0000","last_change":"2023-11-09T03:48:01.452477+0000","last_active":"2023-11-09T03:48:21.543219+0000","last_peered":"2023-11-09T03:48:21.543219+0000","last_clean":"2023-11-09T03:48:21.543219+0000","last_became_active":"2023-11-09T03:48:01.452186+0000","last_became_peered":"2023-11-09T03:48:01.452186+0000","last_unstale":"2023-11-09T03:48:21.543219+0000","last_undegraded":"2023-11-09T03:48:21.543219+0000","last_fullsized":"2023-11-09T03:48:21.543219+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:32:21.825778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547861+0000","last_change":"2023-11-09T03:48:02.442101+0000","last_active":"2023-11-09T03:48:21.547861+0000","last_peered":"2023-11-09T03:48:21.547861+0000","last_clean":"2023-11-09T03:48:21.547861+0000","last_became_active":"2023-11-09T03:48:00.449354+0000","last_became_peered":"2023-11-09T03:48:00.449354+0000","last_unstale":"2023-11-09T03:48:21.547861+0000","last_undegraded":"2023-11-09T03:48:21.547861+0000","last_fullsized":"2023-11-09T03:48:21.547861+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:06:32.906582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093455000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":47,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545438+0000","last_change":"2023-11-09T03:47:02.725125+0000","last_active":"2023-11-09T03:48:21.545438+0000","last_peered":"2023-11-09T03:48:21.545438+0000","last_clean":"2023-11-09T03:48:21.545438+0000","last_became_active":"2023-11-09T03:47:01.591521+0000","last_became_peered":"2023-11-09T03:47:01.591521+0000","last_unstale":"2023-11-09T03:48:21.545438+0000","last_undegraded":"2023-11-09T03:48:21.545438+0000","last_fullsized":"2023-11-09T03:48:21.545438+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:52:40.076542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000758675,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550819+0000","last_change":"2023-11-09T03:48:22.550819+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550819+0000","last_undegraded":"2023-11-09T03:48:22.550819+0000","last_fullsized":"2023-11-09T03:48:22.550819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.565704+0000","last_change":"2023-11-09T03:48:22.561189+0000","last_active":"2023-11-09T03:48:22.565704+0000","last_peered":"2023-11-09T03:48:22.565704+0000","last_clean":"2023-11-09T03:48:22.565704+0000","last_became_active":"2023-11-09T03:48:22.560938+0000","last_became_peered":"2023-11-09T03:48:22.560938+0000","last_unstale":"2023-11-09T03:48:22.565704+0000","last_undegraded":"2023-11-09T03:48:22.565704+0000","last_fullsized":"2023-11-09T03:48:22.565704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:05:30.764921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545418+0000","last_change":"2023-11-09T03:48:01.448129+0000","last_active":"2023-11-09T03:48:21.545418+0000","last_peered":"2023-11-09T03:48:21.545418+0000","last_clean":"2023-11-09T03:48:21.545418+0000","last_became_active":"2023-11-09T03:48:01.447844+0000","last_became_peered":"2023-11-09T03:48:01.447844+0000","last_unstale":"2023-11-09T03:48:21.545418+0000","last_undegraded":"2023-11-09T03:48:21.545418+0000","last_fullsized":"2023-11-09T03:48:21.545418+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:54:52.433604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543211+0000","last_change":"2023-11-09T03:48:02.612450+0000","last_active":"2023-11-09T03:48:21.543211+0000","last_peered":"2023-11-09T03:48:21.543211+0000","last_clean":"2023-11-09T03:48:21.543211+0000","last_became_active":"2023-11-09T03:48:00.441855+0000","last_became_peered":"2023-11-09T03:48:00.441855+0000","last_unstale":"2023-11-09T03:48:21.543211+0000","last_undegraded":"2023-11-09T03:48:21.543211+0000","last_fullsized":"2023-11-09T03:48:21.543211+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:53:42.763434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095368899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547816+0000","last_change":"2023-11-09T03:47:03.147289+0000","last_active":"2023-11-09T03:48:21.547816+0000","last_peered":"2023-11-09T03:48:21.547816+0000","last_clean":"2023-11-09T03:48:21.547816+0000","last_became_active":"2023-11-09T03:47:01.587784+0000","last_became_peered":"2023-11-09T03:47:01.587784+0000","last_unstale":"2023-11-09T03:48:21.547816+0000","last_undegraded":"2023-11-09T03:48:21.547816+0000","last_fullsized":"2023-11-09T03:48:21.547816+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:11:23.824112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038781299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.552840+0000","last_change":"2023-11-09T03:48:22.552840+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.552840+0000","last_undegraded":"2023-11-09T03:48:22.552840+0000","last_fullsized":"2023-11-09T03:48:22.552840+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.568633+0000","last_change":"2023-11-09T03:48:22.564955+0000","last_active":"2023-11-09T03:48:22.568633+0000","last_peered":"2023-11-09T03:48:22.568633+0000","last_clean":"2023-11-09T03:48:22.568633+0000","last_became_active":"2023-11-09T03:48:22.564677+0000","last_became_peered":"2023-11-09T03:48:22.564677+0000","last_unstale":"2023-11-09T03:48:22.568633+0000","last_undegraded":"2023-11-09T03:48:22.568633+0000","last_fullsized":"2023-11-09T03:48:22.568633+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:16:21.330564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541065+0000","last_change":"2023-11-09T03:48:01.449617+0000","last_active":"2023-11-09T03:48:21.541065+0000","last_peered":"2023-11-09T03:48:21.541065+0000","last_clean":"2023-11-09T03:48:21.541065+0000","last_became_active":"2023-11-09T03:48:01.449328+0000","last_became_peered":"2023-11-09T03:48:01.449328+0000","last_unstale":"2023-11-09T03:48:21.541065+0000","last_undegraded":"2023-11-09T03:48:21.541065+0000","last_fullsized":"2023-11-09T03:48:21.541065+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:51:18.760059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547661+0000","last_change":"2023-11-09T03:48:02.442215+0000","last_active":"2023-11-09T03:48:21.547661+0000","last_peered":"2023-11-09T03:48:21.547661+0000","last_clean":"2023-11-09T03:48:21.547661+0000","last_became_active":"2023-11-09T03:48:00.444720+0000","last_became_peered":"2023-11-09T03:48:00.444720+0000","last_unstale":"2023-11-09T03:48:21.547661+0000","last_undegraded":"2023-11-09T03:48:21.547661+0000","last_fullsized":"2023-11-09T03:48:21.547661+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:55:55.735668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001407176,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543390+0000","last_change":"2023-11-09T03:47:02.619346+0000","last_active":"2023-11-09T03:48:21.543390+0000","last_peered":"2023-11-09T03:48:21.543390+0000","last_clean":"2023-11-09T03:48:21.543390+0000","last_became_active":"2023-11-09T03:47:01.590389+0000","last_became_peered":"2023-11-09T03:47:01.590389+0000","last_unstale":"2023-11-09T03:48:21.543390+0000","last_undegraded":"2023-11-09T03:48:21.543390+0000","last_fullsized":"2023-11-09T03:48:21.543390+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000384072,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551555+0000","last_change":"2023-11-09T03:48:22.551555+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551555+0000","last_undegraded":"2023-11-09T03:48:22.551555+0000","last_fullsized":"2023-11-09T03:48:22.551555+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.569120+0000","last_change":"2023-11-09T03:48:22.562601+0000","last_active":"2023-11-09T03:48:22.569120+0000","last_peered":"2023-11-09T03:48:22.569120+0000","last_clean":"2023-11-09T03:48:22.569120+0000","last_became_active":"2023-11-09T03:48:22.562326+0000","last_became_peered":"2023-11-09T03:48:22.562326+0000","last_unstale":"2023-11-09T03:48:22.569120+0000","last_undegraded":"2023-11-09T03:48:22.569120+0000","last_fullsized":"2023-11-09T03:48:22.569120+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:28:06.557471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545053+0000","last_change":"2023-11-09T03:48:01.450018+0000","last_active":"2023-11-09T03:48:21.545053+0000","last_peered":"2023-11-09T03:48:21.545053+0000","last_clean":"2023-11-09T03:48:21.545053+0000","last_became_active":"2023-11-09T03:48:01.449749+0000","last_became_peered":"2023-11-09T03:48:01.449749+0000","last_unstale":"2023-11-09T03:48:21.545053+0000","last_undegraded":"2023-11-09T03:48:21.545053+0000","last_fullsized":"2023-11-09T03:48:21.545053+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:21:23.156761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542795+0000","last_change":"2023-11-09T03:48:02.524216+0000","last_active":"2023-11-09T03:48:21.542795+0000","last_peered":"2023-11-09T03:48:21.542795+0000","last_clean":"2023-11-09T03:48:21.542795+0000","last_became_active":"2023-11-09T03:48:00.446972+0000","last_became_peered":"2023-11-09T03:48:00.446972+0000","last_unstale":"2023-11-09T03:48:21.542795+0000","last_undegraded":"2023-11-09T03:48:21.542795+0000","last_fullsized":"2023-11-09T03:48:21.542795+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:54:07.987773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000825255,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":50,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540373+0000","last_change":"2023-11-09T03:47:02.582393+0000","last_active":"2023-11-09T03:48:21.540373+0000","last_peered":"2023-11-09T03:48:21.540373+0000","last_clean":"2023-11-09T03:48:21.540373+0000","last_became_active":"2023-11-09T03:47:01.591752+0000","last_became_peered":"2023-11-09T03:47:01.591752+0000","last_unstale":"2023-11-09T03:48:21.540373+0000","last_undegraded":"2023-11-09T03:48:21.540373+0000","last_fullsized":"2023-11-09T03:48:21.540373+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:03.025119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067021599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549641+0000","last_change":"2023-11-09T03:48:22.549641+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549641+0000","last_undegraded":"2023-11-09T03:48:22.549641+0000","last_fullsized":"2023-11-09T03:48:22.549641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.566064+0000","last_change":"2023-11-09T03:48:22.562752+0000","last_active":"2023-11-09T03:48:22.566064+0000","last_peered":"2023-11-09T03:48:22.566064+0000","last_clean":"2023-11-09T03:48:22.566064+0000","last_became_active":"2023-11-09T03:48:22.562474+0000","last_became_peered":"2023-11-09T03:48:22.562474+0000","last_unstale":"2023-11-09T03:48:22.566064+0000","last_undegraded":"2023-11-09T03:48:22.566064+0000","last_fullsized":"2023-11-09T03:48:22.566064+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:35:27.804836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542868+0000","last_change":"2023-11-09T03:48:01.451462+0000","last_active":"2023-11-09T03:48:21.542868+0000","last_peered":"2023-11-09T03:48:21.542868+0000","last_clean":"2023-11-09T03:48:21.542868+0000","last_became_active":"2023-11-09T03:48:01.451188+0000","last_became_peered":"2023-11-09T03:48:01.451188+0000","last_unstale":"2023-11-09T03:48:21.542868+0000","last_undegraded":"2023-11-09T03:48:21.542868+0000","last_fullsized":"2023-11-09T03:48:21.542868+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:28.631801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542221+0000","last_change":"2023-11-09T03:48:02.682290+0000","last_active":"2023-11-09T03:48:21.542221+0000","last_peered":"2023-11-09T03:48:21.542221+0000","last_clean":"2023-11-09T03:48:21.542221+0000","last_became_active":"2023-11-09T03:48:00.444246+0000","last_became_peered":"2023-11-09T03:48:00.444246+0000","last_unstale":"2023-11-09T03:48:21.542221+0000","last_undegraded":"2023-11-09T03:48:21.542221+0000","last_fullsized":"2023-11-09T03:48:21.542221+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:09.487058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000423359,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.544622+0000","last_change":"2023-11-09T03:47:03.028744+0000","last_active":"2023-11-09T03:48:21.544622+0000","last_peered":"2023-11-09T03:48:21.544622+0000","last_clean":"2023-11-09T03:48:21.544622+0000","last_became_active":"2023-11-09T03:47:01.593890+0000","last_became_peered":"2023-11-09T03:47:01.593890+0000","last_unstale":"2023-11-09T03:48:21.544622+0000","last_undegraded":"2023-11-09T03:48:21.544622+0000","last_fullsized":"2023-11-09T03:48:21.544622+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:05.754357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053008299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550087+0000","last_change":"2023-11-09T03:48:22.550087+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550087+0000","last_undegraded":"2023-11-09T03:48:22.550087+0000","last_fullsized":"2023-11-09T03:48:22.550087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.566105+0000","last_change":"2023-11-09T03:48:22.560645+0000","last_active":"2023-11-09T03:48:22.566105+0000","last_peered":"2023-11-09T03:48:22.566105+0000","last_clean":"2023-11-09T03:48:22.566105+0000","last_became_active":"2023-11-09T03:48:22.560328+0000","last_became_peered":"2023-11-09T03:48:22.560328+0000","last_unstale":"2023-11-09T03:48:22.566105+0000","last_undegraded":"2023-11-09T03:48:22.566105+0000","last_fullsized":"2023-11-09T03:48:22.566105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:59.646501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.544881+0000","last_change":"2023-11-09T03:48:01.450144+0000","last_active":"2023-11-09T03:48:21.544881+0000","last_peered":"2023-11-09T03:48:21.544881+0000","last_clean":"2023-11-09T03:48:21.544881+0000","last_became_active":"2023-11-09T03:48:01.449859+0000","last_became_peered":"2023-11-09T03:48:01.449859+0000","last_unstale":"2023-11-09T03:48:21.544881+0000","last_undegraded":"2023-11-09T03:48:21.544881+0000","last_fullsized":"2023-11-09T03:48:21.544881+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:57:56.855869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542191+0000","last_change":"2023-11-09T03:48:02.681869+0000","last_active":"2023-11-09T03:48:21.542191+0000","last_peered":"2023-11-09T03:48:21.542191+0000","last_clean":"2023-11-09T03:48:21.542191+0000","last_became_active":"2023-11-09T03:48:00.444928+0000","last_became_peered":"2023-11-09T03:48:00.444928+0000","last_unstale":"2023-11-09T03:48:21.542191+0000","last_undegraded":"2023-11-09T03:48:21.542191+0000","last_fullsized":"2023-11-09T03:48:21.542191+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:59:58.695636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044822599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542292+0000","last_change":"2023-11-09T03:47:03.146783+0000","last_active":"2023-11-09T03:48:21.542292+0000","last_peered":"2023-11-09T03:48:21.542292+0000","last_clean":"2023-11-09T03:48:21.542292+0000","last_became_active":"2023-11-09T03:47:01.586923+0000","last_became_peered":"2023-11-09T03:47:01.586923+0000","last_unstale":"2023-11-09T03:48:21.542292+0000","last_undegraded":"2023-11-09T03:48:21.542292+0000","last_fullsized":"2023-11-09T03:48:21.542292+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:37:50.326442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060384700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549765+0000","last_change":"2023-11-09T03:48:22.549765+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549765+0000","last_undegraded":"2023-11-09T03:48:22.549765+0000","last_fullsized":"2023-11-09T03:48:22.549765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.571830+0000","last_change":"2023-11-09T03:48:22.565142+0000","last_active":"2023-11-09T03:48:22.571830+0000","last_peered":"2023-11-09T03:48:22.571830+0000","last_clean":"2023-11-09T03:48:22.571830+0000","last_became_active":"2023-11-09T03:48:22.564856+0000","last_became_peered":"2023-11-09T03:48:22.564856+0000","last_unstale":"2023-11-09T03:48:22.571830+0000","last_undegraded":"2023-11-09T03:48:22.571830+0000","last_fullsized":"2023-11-09T03:48:22.571830+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:25:04.111498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542575+0000","last_change":"2023-11-09T03:48:01.448383+0000","last_active":"2023-11-09T03:48:21.542575+0000","last_peered":"2023-11-09T03:48:21.542575+0000","last_clean":"2023-11-09T03:48:21.542575+0000","last_became_active":"2023-11-09T03:48:01.448050+0000","last_became_peered":"2023-11-09T03:48:01.448050+0000","last_unstale":"2023-11-09T03:48:21.542575+0000","last_undegraded":"2023-11-09T03:48:21.542575+0000","last_fullsized":"2023-11-09T03:48:21.542575+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:53:44.478182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541748+0000","last_change":"2023-11-09T03:48:02.523794+0000","last_active":"2023-11-09T03:48:21.541748+0000","last_peered":"2023-11-09T03:48:21.541748+0000","last_clean":"2023-11-09T03:48:21.541748+0000","last_became_active":"2023-11-09T03:48:00.440255+0000","last_became_peered":"2023-11-09T03:48:00.440255+0000","last_unstale":"2023-11-09T03:48:21.541748+0000","last_undegraded":"2023-11-09T03:48:21.541748+0000","last_fullsized":"2023-11-09T03:48:21.541748+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:58:02.392696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064572299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542252+0000","last_change":"2023-11-09T03:47:03.146580+0000","last_active":"2023-11-09T03:48:21.542252+0000","last_peered":"2023-11-09T03:48:21.542252+0000","last_clean":"2023-11-09T03:48:21.542252+0000","last_became_active":"2023-11-09T03:47:01.589201+0000","last_became_peered":"2023-11-09T03:47:01.589201+0000","last_unstale":"2023-11-09T03:48:21.542252+0000","last_undegraded":"2023-11-09T03:48:21.542252+0000","last_fullsized":"2023-11-09T03:48:21.542252+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:14.300421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049706999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551049+0000","last_change":"2023-11-09T03:48:22.551049+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551049+0000","last_undegraded":"2023-11-09T03:48:22.551049+0000","last_fullsized":"2023-11-09T03:48:22.551049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.566108+0000","last_change":"2023-11-09T03:48:22.562609+0000","last_active":"2023-11-09T03:48:22.566108+0000","last_peered":"2023-11-09T03:48:22.566108+0000","last_clean":"2023-11-09T03:48:22.566108+0000","last_became_active":"2023-11-09T03:48:22.562328+0000","last_became_peered":"2023-11-09T03:48:22.562328+0000","last_unstale":"2023-11-09T03:48:22.566108+0000","last_undegraded":"2023-11-09T03:48:22.566108+0000","last_fullsized":"2023-11-09T03:48:22.566108+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:57:25.589693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539239+0000","last_change":"2023-11-09T03:48:01.451284+0000","last_active":"2023-11-09T03:48:21.539239+0000","last_peered":"2023-11-09T03:48:21.539239+0000","last_clean":"2023-11-09T03:48:21.539239+0000","last_became_active":"2023-11-09T03:48:01.450954+0000","last_became_peered":"2023-11-09T03:48:01.450954+0000","last_unstale":"2023-11-09T03:48:21.539239+0000","last_undegraded":"2023-11-09T03:48:21.539239+0000","last_fullsized":"2023-11-09T03:48:21.539239+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:49:24.680020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545344+0000","last_change":"2023-11-09T03:48:02.733038+0000","last_active":"2023-11-09T03:48:21.545344+0000","last_peered":"2023-11-09T03:48:21.545344+0000","last_clean":"2023-11-09T03:48:21.545344+0000","last_became_active":"2023-11-09T03:48:00.446661+0000","last_became_peered":"2023-11-09T03:48:00.446661+0000","last_unstale":"2023-11-09T03:48:21.545344+0000","last_undegraded":"2023-11-09T03:48:21.545344+0000","last_fullsized":"2023-11-09T03:48:21.545344+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:05:08.649312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040102300000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547989+0000","last_change":"2023-11-09T03:47:03.147851+0000","last_active":"2023-11-09T03:48:21.547989+0000","last_peered":"2023-11-09T03:48:21.547989+0000","last_clean":"2023-11-09T03:48:21.547989+0000","last_became_active":"2023-11-09T03:47:01.589893+0000","last_became_peered":"2023-11-09T03:47:01.589893+0000","last_unstale":"2023-11-09T03:48:21.547989+0000","last_undegraded":"2023-11-09T03:48:21.547989+0000","last_fullsized":"2023-11-09T03:48:21.547989+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:08:17.183462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029151399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":215,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545358+0000","last_change":"2023-11-09T03:45:21.030926+0000","last_active":"2023-11-09T03:48:21.545358+0000","last_peered":"2023-11-09T03:48:21.545358+0000","last_clean":"2023-11-09T03:48:21.545358+0000","last_became_active":"2023-11-09T03:45:21.029801+0000","last_became_peered":"2023-11-09T03:45:21.029801+0000","last_unstale":"2023-11-09T03:48:21.545358+0000","last_undegraded":"2023-11-09T03:48:21.545358+0000","last_fullsized":"2023-11-09T03:48:21.545358+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_clean_scrub_stamp":"2023-11-09T03:45:19.768433+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:12:59.153713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1572864,"data_stored":1058844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.501,"15min":0.498},"min":{"1min":0.34899999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.41199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.59899999999999998}]},{"osd":1,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.33000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.38100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.57799999999999996}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.52300000000000002,"15min":0.51400000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.68700000000000006,"15min":0.70799999999999996},"min":{"1min":0.41299999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.80700000000000005,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.628}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.48699999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61599999999999999}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.441},"min":{"1min":0.34100000000000003,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.49199999999999999,"15min":0.47899999999999998},"min":{"1min":0.38700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.42799999999999999}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.47199999999999998,"15min":0.45600000000000002},"min":{"1min":0.37,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.51100000000000001,"15min":0.497},"min":{"1min":0.39600000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.54800000000000004}]},{"osd":6,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55100000000000005,"15min":0.54100000000000004},"min":{"1min":0.44400000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.496},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.434,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.48099999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":991232,"data_stored":476134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.499,"15min":0.501},"min":{"1min":0.35099999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63900000000000001,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.50600000000000001,"15min":0.495},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.51100000000000001}]},{"osd":1,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.375,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.58299999999999996}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.503,"15min":0.503},"min":{"1min":0.316,"5min":0.313,"15min":0.313},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.437},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.45700000000000002}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.436,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55500000000000005,"15min":0.55800000000000005},"min":{"1min":0.34300000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.42899999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.44400000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.56100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":978944,"data_stored":468476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.504,"15min":0.48899999999999999},"min":{"1min":0.36199999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.503},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.376,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.61299999999999999}]},{"osd":1,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88300000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.36599999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.84199999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.65900000000000003}]},{"osd":2,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.45800000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.55600000000000005},"min":{"1min":0.40300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.71099999999999997}]},{"osd":3,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57099999999999995,"15min":0.56499999999999995},"min":{"1min":0.39400000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55200000000000005,"15min":0.55100000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.76100000000000001}]},{"osd":4,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.49199999999999999,"15min":0.48399999999999999},"min":{"1min":0.32400000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.48999999999999999,"15min":0.48299999999999998},"min":{"1min":0.29699999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.57899999999999996}]},{"osd":6,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.50600000000000001,"15min":0.51000000000000001},"min":{"1min":0.33700000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.755,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.53300000000000003,"15min":0.51700000000000002},"min":{"1min":0.38300000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.64400000000000002}]},{"osd":7,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.34699999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.81200000000000006,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.61899999999999999,"15min":0.62},"min":{"1min":0.432,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84899999999999998,"5min":1.01,"15min":1.01},"last":0.79000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":978944,"data_stored":468476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69399999999999995,"5min":1.034,"15min":1.034},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.38800000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78300000000000003,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.34000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.74299999999999999,"5min":1.093,"15min":1.093},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54200000000000004,"15min":0.54100000000000004},"min":{"1min":0.36399999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71899999999999997,"5min":0.871,"15min":0.871},"last":0.54900000000000004}]},{"osd":2,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54100000000000004,"15min":0.55400000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.69199999999999995,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54800000000000004,"15min":0.54500000000000004},"min":{"1min":0.39700000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.51800000000000002}]},{"osd":3,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.433,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.70199999999999996,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.376,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72499999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.379}]},{"osd":5,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.33300000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.78700000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.318,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81100000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.35799999999999998}]},{"osd":6,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.75800000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57899999999999996,"15min":0.58399999999999996},"min":{"1min":0.39600000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.75900000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.39600000000000002}]},{"osd":7,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61599999999999999,"15min":0.629},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83499999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61099999999999999,"15min":0.61899999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79300000000000004,"5min":1.113,"15min":1.113},"last":0.45500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":978944,"data_stored":468476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.32100000000000001,"15min":0.32300000000000001},"min":{"1min":0.22,"5min":0.215,"15min":0.215},"max":{"1min":0.40200000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.313,"5min":0.32300000000000001,"15min":0.32400000000000001},"min":{"1min":0.23999999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.47799999999999998,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.23999999999999999}]},{"osd":1,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.32600000000000001,"15min":0.33000000000000002},"min":{"1min":0.23599999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.42899999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.36699999999999999,"15min":0.373},"min":{"1min":0.28100000000000003,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.47899999999999998,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.36599999999999999}]},{"osd":2,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.37,"15min":0.372},"min":{"1min":0.27200000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.439,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.312},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.379,"15min":0.38300000000000001},"min":{"1min":0.29799999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.45700000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.38400000000000001}]},{"osd":4,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58899999999999997,"15min":0.59499999999999997},"min":{"1min":0.42299999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79000000000000004,"5min":0.87,"15min":0.87},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.59699999999999998,"15min":0.61099999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.434}]},{"osd":5,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.60499999999999998,"15min":0.62},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86599999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60699999999999998,"15min":0.61899999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.82099999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.443}]},{"osd":6,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60999999999999999,"15min":0.623},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78200000000000003,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.45400000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.80300000000000005,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.55200000000000005}]},{"osd":7,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.61099999999999999,"15min":0.62},"min":{"1min":0.41099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.70899999999999996,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.621,"15min":0.63700000000000001},"min":{"1min":0.46100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81299999999999994,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.46100000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":991232,"data_stored":476134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.46000000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.33400000000000002,"15min":0.33100000000000002},"min":{"1min":0.247,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.29399999999999998}]},{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32500000000000001,"15min":0.32600000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.495,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.35899999999999999,"15min":0.36299999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.48599999999999999,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.36699999999999999}]},{"osd":3,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.36199999999999999,"15min":0.36399999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.45700000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.312},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40200000000000002,"15min":0.40400000000000003},"min":{"1min":0.30299999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.52300000000000002,"5min":0.754,"15min":0.754},"last":0.39900000000000002}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72299999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81299999999999994,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.46000000000000002}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.39800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.75700000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.59599999999999997,"15min":0.60099999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.753,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.498}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60399999999999998,"15min":0.60899999999999999},"min":{"1min":0.44600000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.76100000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.71199999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60699999999999998,"15min":0.61199999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.745,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.80000000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.47299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":978944,"data_stored":468476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.28699999999999998,"5min":0.31,"15min":0.314},"min":{"1min":0.20799999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.38900000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"last":0.309},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.315,"15min":0.311},"min":{"1min":0.24099999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.29299999999999998}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.311,"15min":0.309},"min":{"1min":0.253,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.378,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.34999999999999998}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36899999999999999,"15min":0.371},"min":{"1min":0.23899999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.47699999999999998,"5min":0.505,"15min":0.505},"last":0.379},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.37,"15min":0.371},"min":{"1min":0.30299999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.45400000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.41899999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.747,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.44400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.77600000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.58699999999999997}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56200000000000006,"15min":0.55700000000000005},"min":{"1min":0.46100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.44500000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.72599999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.47799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56699999999999995,"15min":0.56799999999999995},"min":{"1min":0.433,"5min":0.379,"15min":0.379},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.53300000000000003}]},{"osd":7,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.42899999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.69799999999999995,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.72899999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.54700000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1568768,"data_stored":1058225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.31900000000000001,"15min":0.32300000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.38200000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.29599999999999999,"5min":0.35499999999999998,"15min":0.36499999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.44500000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.27300000000000002}]},{"osd":2,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.35699999999999998,"15min":0.36399999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.44700000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.376},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.36699999999999999,"15min":0.371},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.47699999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.316}]},{"osd":3,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.38500000000000001,"15min":0.39100000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.46400000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.41499999999999998,"15min":0.42099999999999999},"min":{"1min":0.309,"5min":0.307,"15min":0.307},"max":{"1min":0.48999999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58899999999999997,"15min":0.59299999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60599999999999998,"15min":0.60899999999999999},"min":{"1min":0.436,"5min":0.318,"15min":0.318},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.60299999999999998}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.36199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76800000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.629,"15min":0.63600000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78900000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.64100000000000001,"15min":0.64700000000000002},"min":{"1min":0.42499999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81000000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.627,"15min":0.63200000000000001},"min":{"1min":0.41699999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.44400000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.42999999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.71799999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.55600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-09T03:48:26.514 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-09T03:48:26.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.648+0000 7f2e0f007640 1 Processor -- start 2023-11-09T03:48:26.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.648+0000 7f2e0f007640 1 -- start start 2023-11-09T03:48:26.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.648+0000 7f2e0f007640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2e080cde60 0x7f2e0815d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:26.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.648+0000 7f2e0f007640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e0815daa0 0x7f2e08157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:26.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.648+0000 7f2e0f007640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e0815e000 con 0x7f2e080ce240 2023-11-09T03:48:26.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.648+0000 7f2e0f007640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e0815e5c0 con 0x7f2e080cde60 2023-11-09T03:48:26.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.648+0000 7f2e0f007640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0815e2e0 con 0x7f2e0815daa0 2023-11-09T03:48:26.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.649+0000 7f2e0cda3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2e080cde60 0x7f2e0815d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:26.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.649+0000 7f2e0cda3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2e080cde60 0x7f2e0815d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37850/0 (socket says 172.21.15.76:37850) 2023-11-09T03:48:26.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.649+0000 7f2e0cda3640 1 -- 172.21.15.76:0/1309954450 learned_addr learned my addr 172.21.15.76:0/1309954450 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:26.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.649+0000 7f2e07fff640 1 --2- 172.21.15.76:0/1309954450 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e0815daa0 0x7f2e08157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:26.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.649+0000 7f2e0cda3640 1 -- 172.21.15.76:0/1309954450 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e0815daa0 msgr2=0x7f2e08157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:26.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.649+0000 7f2e0cda3640 1 --2- 172.21.15.76:0/1309954450 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e0815daa0 0x7f2e08157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.649+0000 7f2e0cda3640 1 -- 172.21.15.76:0/1309954450 >> v1:172.21.15.113:6789/0 conn(0x7f2e080ce240 legacy=0x7f2e0806b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:26.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.649+0000 7f2e0cda3640 1 -- 172.21.15.76:0/1309954450 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e080d1120 con 0x7f2e080cde60 2023-11-09T03:48:26.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.650+0000 7f2e0cda3640 1 --2- 172.21.15.76:0/1309954450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e080cde60 0x7f2e0815d560 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009870 tx=0x7f2dfc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5cc2bc147836c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:26.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.650+0000 7f2e077fe640 1 -- 172.21.15.76:0/1309954450 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc030840 con 0x7f2e080cde60 2023-11-09T03:48:26.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.650+0000 7f2e077fe640 1 -- 172.21.15.76:0/1309954450 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dfc0309a0 con 0x7f2e080cde60 2023-11-09T03:48:26.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.650+0000 7f2e077fe640 1 -- 172.21.15.76:0/1309954450 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc030d30 con 0x7f2e080cde60 2023-11-09T03:48:26.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.650+0000 7f2e0f007640 1 -- 172.21.15.76:0/1309954450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e080cde60 msgr2=0x7f2e0815d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:26.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.650+0000 7f2e0f007640 1 --2- 172.21.15.76:0/1309954450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e080cde60 0x7f2e0815d560 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009870 tx=0x7f2dfc02b980 comp rx=0 tx=0).stop 2023-11-09T03:48:26.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.650+0000 7f2e0f007640 1 -- 172.21.15.76:0/1309954450 shutdown_connections 2023-11-09T03:48:26.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.650+0000 7f2e0f007640 1 --2- 172.21.15.76:0/1309954450 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e0815daa0 0x7f2e08157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.650+0000 7f2e0f007640 1 --2- 172.21.15.76:0/1309954450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e080cde60 0x7f2e0815d560 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.650+0000 7f2e0f007640 1 -- 172.21.15.76:0/1309954450 >> 172.21.15.76:0/1309954450 conn(0x7f2e08071f30 msgr2=0x7f2e080cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:26.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.650+0000 7f2e0f007640 1 -- 172.21.15.76:0/1309954450 shutdown_connections 2023-11-09T03:48:26.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.650+0000 7f2e0f007640 1 -- 172.21.15.76:0/1309954450 wait complete. 2023-11-09T03:48:26.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.652+0000 7f2e0f007640 1 Processor -- start 2023-11-09T03:48:26.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.652+0000 7f2e0f007640 1 -- start start 2023-11-09T03:48:26.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.652+0000 7f2e0f007640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e080cde60 0x7f2e081f2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:26.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.652+0000 7f2e0f007640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e080ce240 0x7f2e0815c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:26.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.652+0000 7f2e0f007640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e0815daa0 0x7f2e0815aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:26.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.652+0000 7f2e0f007640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e081f3530 con 0x7f2e0815daa0 2023-11-09T03:48:26.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.652+0000 7f2e0f007640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e081f2a60 con 0x7f2e080ce240 2023-11-09T03:48:26.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.652+0000 7f2e0f007640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0815cc80 con 0x7f2e080cde60 2023-11-09T03:48:26.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.653+0000 7f2e0d5a4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e0815daa0 0x7f2e0815aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:26.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.653+0000 7f2e0d5a4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e0815daa0 0x7f2e0815aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37858/0 (socket says 172.21.15.76:37858) 2023-11-09T03:48:26.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.653+0000 7f2e0d5a4640 1 -- 172.21.15.76:0/3596161157 learned_addr learned my addr 172.21.15.76:0/3596161157 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:26.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.653+0000 7f2e0cda3640 1 --2- 172.21.15.76:0/3596161157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e080cde60 0x7f2e081f2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:26.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.653+0000 7f2e07fff640 1 --2- 172.21.15.76:0/3596161157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e080ce240 0x7f2e0815c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:26.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.653+0000 7f2e0d5a4640 1 -- 172.21.15.76:0/3596161157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e080cde60 msgr2=0x7f2e081f2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:26.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.653+0000 7f2e0d5a4640 1 --2- 172.21.15.76:0/3596161157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e080cde60 0x7f2e081f2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.653+0000 7f2e0d5a4640 1 -- 172.21.15.76:0/3596161157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e080ce240 msgr2=0x7f2e0815c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:26.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.653+0000 7f2e0d5a4640 1 --2- 172.21.15.76:0/3596161157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e080ce240 0x7f2e0815c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.653+0000 7f2e0d5a4640 1 -- 172.21.15.76:0/3596161157 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dfc0094d0 con 0x7f2e0815daa0 2023-11-09T03:48:26.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.653+0000 7f2e0d5a4640 1 --2- 172.21.15.76:0/3596161157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e0815daa0 0x7f2e0815aa40 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f2e0000c7a0 tx=0x7f2e0000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:26.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.653+0000 7f2e057fa640 1 -- 172.21.15.76:0/3596161157 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e00003b70 con 0x7f2e0815daa0 2023-11-09T03:48:26.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.653+0000 7f2e057fa640 1 -- 172.21.15.76:0/3596161157 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e00003cf0 con 0x7f2e0815daa0 2023-11-09T03:48:26.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.653+0000 7f2e0f007640 1 -- 172.21.15.76:0/3596161157 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e081f2c90 con 0x7f2e0815daa0 2023-11-09T03:48:26.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.653+0000 7f2e057fa640 1 -- 172.21.15.76:0/3596161157 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0000aa90 con 0x7f2e0815daa0 2023-11-09T03:48:26.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.653+0000 7f2e0f007640 1 -- 172.21.15.76:0/3596161157 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e081f3250 con 0x7f2e0815daa0 2023-11-09T03:48:26.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.654+0000 7f2e0f007640 1 -- 172.21.15.76:0/3596161157 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e08074290 con 0x7f2e0815daa0 2023-11-09T03:48:26.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.655+0000 7f2e057fa640 1 -- 172.21.15.76:0/3596161157 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2e0000abf0 con 0x7f2e0815daa0 2023-11-09T03:48:26.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.655+0000 7f2e057fa640 1 --2- 172.21.15.76:0/3596161157 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2dec06ae70 0x7f2dec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:26.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.655+0000 7f2e057fa640 1 -- 172.21.15.76:0/3596161157 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f2e00089970 con 0x7f2e0815daa0 2023-11-09T03:48:26.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.655+0000 7f2e0cda3640 1 --2- 172.21.15.76:0/3596161157 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2dec06ae70 0x7f2dec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:26.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.656+0000 7f2e0cda3640 1 --2- 172.21.15.76:0/3596161157 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2dec06ae70 0x7f2dec06d330 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc036000 tx=0x7f2dfc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:26.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.660+0000 7f2e057fa640 1 -- 172.21.15.76:0/3596161157 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2e00054540 con 0x7f2e0815daa0 2023-11-09T03:48:26.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.821+0000 7f2e0f007640 1 -- 172.21.15.76:0/3596161157 --> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2e081559c0 con 0x7f2dec06ae70 2023-11-09T03:48:26.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.834+0000 7f2e057fa640 1 -- 172.21.15.76:0/3596161157 <== mgr.4105 v2:172.21.15.113:6832/1264193042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+526928 (secure 0 0 0) 0x7f2e081559c0 con 0x7f2dec06ae70 2023-11-09T03:48:26.839 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:26.843 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2023-11-09T03:48:26.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.838+0000 7f2e0f007640 1 -- 172.21.15.76:0/3596161157 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2dec06ae70 msgr2=0x7f2dec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:26.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.838+0000 7f2e0f007640 1 --2- 172.21.15.76:0/3596161157 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2dec06ae70 0x7f2dec06d330 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc036000 tx=0x7f2dfc008030 comp rx=0 tx=0).stop 2023-11-09T03:48:26.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.838+0000 7f2e0f007640 1 -- 172.21.15.76:0/3596161157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e0815daa0 msgr2=0x7f2e0815aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:26.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.838+0000 7f2e0f007640 1 --2- 172.21.15.76:0/3596161157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e0815daa0 0x7f2e0815aa40 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f2e0000c7a0 tx=0x7f2e0000cc60 comp rx=0 tx=0).stop 2023-11-09T03:48:26.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.838+0000 7f2e0f007640 1 -- 172.21.15.76:0/3596161157 shutdown_connections 2023-11-09T03:48:26.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.838+0000 7f2e0f007640 1 --2- 172.21.15.76:0/3596161157 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2dec06ae70 0x7f2dec06d330 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.838+0000 7f2e0f007640 1 --2- 172.21.15.76:0/3596161157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e0815daa0 0x7f2e0815aa40 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.838+0000 7f2e0f007640 1 --2- 172.21.15.76:0/3596161157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e080ce240 0x7f2e0815c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.838+0000 7f2e0f007640 1 --2- 172.21.15.76:0/3596161157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e080cde60 0x7f2e081f2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.838+0000 7f2e0f007640 1 -- 172.21.15.76:0/3596161157 >> 172.21.15.76:0/3596161157 conn(0x7f2e08071f30 msgr2=0x7f2e080c6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:26.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.838+0000 7f2e0f007640 1 -- 172.21.15.76:0/3596161157 shutdown_connections 2023-11-09T03:48:26.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.838+0000 7f2e0f007640 1 -- 172.21.15.76:0/3596161157 wait complete. 2023-11-09T03:48:26.848 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2023-11-09T03:48:26.794284+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":204,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237468,"kb_used_data":8828,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754737252,"statfs":{"total":773094113280,"available":772850946048,"internally_reserved":0,"allocated":9039872,"data_stored":4943241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.256434"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549889+0000","last_change":"2023-11-09T03:48:22.549889+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549889+0000","last_undegraded":"2023-11-09T03:48:22.549889+0000","last_fullsized":"2023-11-09T03:48:22.549889+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.554480+0000","last_change":"2023-11-09T03:48:22.554480+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.554480+0000","last_undegraded":"2023-11-09T03:48:22.554480+0000","last_fullsized":"2023-11-09T03:48:22.554480+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549408+0000","last_change":"2023-11-09T03:48:22.549408+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549408+0000","last_undegraded":"2023-11-09T03:48:22.549408+0000","last_fullsized":"2023-11-09T03:48:22.549408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550714+0000","last_change":"2023-11-09T03:48:22.550714+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550714+0000","last_undegraded":"2023-11-09T03:48:22.550714+0000","last_fullsized":"2023-11-09T03:48:22.550714+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550039+0000","last_change":"2023-11-09T03:48:22.550039+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550039+0000","last_undegraded":"2023-11-09T03:48:22.550039+0000","last_fullsized":"2023-11-09T03:48:22.550039+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549702+0000","last_change":"2023-11-09T03:48:22.549702+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549702+0000","last_undegraded":"2023-11-09T03:48:22.549702+0000","last_fullsized":"2023-11-09T03:48:22.549702+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550846+0000","last_change":"2023-11-09T03:48:22.550846+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550846+0000","last_undegraded":"2023-11-09T03:48:22.550846+0000","last_fullsized":"2023-11-09T03:48:22.550846+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.554400+0000","last_change":"2023-11-09T03:48:22.554400+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.554400+0000","last_undegraded":"2023-11-09T03:48:22.554400+0000","last_fullsized":"2023-11-09T03:48:22.554400+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.547291+0000","last_change":"2023-11-09T03:48:22.547291+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.547291+0000","last_undegraded":"2023-11-09T03:48:22.547291+0000","last_fullsized":"2023-11-09T03:48:22.547291+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551875+0000","last_change":"2023-11-09T03:48:22.551875+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551875+0000","last_undegraded":"2023-11-09T03:48:22.551875+0000","last_fullsized":"2023-11-09T03:48:22.551875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.554690+0000","last_change":"2023-11-09T03:48:22.554690+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.554690+0000","last_undegraded":"2023-11-09T03:48:22.554690+0000","last_fullsized":"2023-11-09T03:48:22.554690+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.547393+0000","last_change":"2023-11-09T03:48:22.547393+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.547393+0000","last_undegraded":"2023-11-09T03:48:22.547393+0000","last_fullsized":"2023-11-09T03:48:22.547393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550250+0000","last_change":"2023-11-09T03:48:22.550250+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550250+0000","last_undegraded":"2023-11-09T03:48:22.550250+0000","last_fullsized":"2023-11-09T03:48:22.550250+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.558130+0000","last_change":"2023-11-09T03:48:22.558130+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.558130+0000","last_undegraded":"2023-11-09T03:48:22.558130+0000","last_fullsized":"2023-11-09T03:48:22.558130+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549957+0000","last_change":"2023-11-09T03:48:22.549957+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549957+0000","last_undegraded":"2023-11-09T03:48:22.549957+0000","last_fullsized":"2023-11-09T03:48:22.549957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.553338+0000","last_change":"2023-11-09T03:48:22.553338+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.553338+0000","last_undegraded":"2023-11-09T03:48:22.553338+0000","last_fullsized":"2023-11-09T03:48:22.553338+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.554384+0000","last_change":"2023-11-09T03:48:22.554384+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.554384+0000","last_undegraded":"2023-11-09T03:48:22.554384+0000","last_fullsized":"2023-11-09T03:48:22.554384+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550716+0000","last_change":"2023-11-09T03:48:22.550716+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550716+0000","last_undegraded":"2023-11-09T03:48:22.550716+0000","last_fullsized":"2023-11-09T03:48:22.550716+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.557699+0000","last_change":"2023-11-09T03:48:22.557699+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.557699+0000","last_undegraded":"2023-11-09T03:48:22.557699+0000","last_fullsized":"2023-11-09T03:48:22.557699+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.546650+0000","last_change":"2023-11-09T03:48:22.546650+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.546650+0000","last_undegraded":"2023-11-09T03:48:22.546650+0000","last_fullsized":"2023-11-09T03:48:22.546650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.555079+0000","last_change":"2023-11-09T03:48:22.555079+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.555079+0000","last_undegraded":"2023-11-09T03:48:22.555079+0000","last_fullsized":"2023-11-09T03:48:22.555079+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.554040+0000","last_change":"2023-11-09T03:48:22.554040+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.554040+0000","last_undegraded":"2023-11-09T03:48:22.554040+0000","last_fullsized":"2023-11-09T03:48:22.554040+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.553445+0000","last_change":"2023-11-09T03:48:22.553445+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.553445+0000","last_undegraded":"2023-11-09T03:48:22.553445+0000","last_fullsized":"2023-11-09T03:48:22.553445+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551333+0000","last_change":"2023-11-09T03:48:22.551333+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551333+0000","last_undegraded":"2023-11-09T03:48:22.551333+0000","last_fullsized":"2023-11-09T03:48:22.551333+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.553219+0000","last_change":"2023-11-09T03:48:22.553219+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.553219+0000","last_undegraded":"2023-11-09T03:48:22.553219+0000","last_fullsized":"2023-11-09T03:48:22.553219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.552812+0000","last_change":"2023-11-09T03:48:22.552812+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.552812+0000","last_undegraded":"2023-11-09T03:48:22.552812+0000","last_fullsized":"2023-11-09T03:48:22.552812+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.552951+0000","last_change":"2023-11-09T03:48:22.552951+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.552951+0000","last_undegraded":"2023-11-09T03:48:22.552951+0000","last_fullsized":"2023-11-09T03:48:22.552951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.552510+0000","last_change":"2023-11-09T03:48:22.552510+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.552510+0000","last_undegraded":"2023-11-09T03:48:22.552510+0000","last_fullsized":"2023-11-09T03:48:22.552510+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.552644+0000","last_change":"2023-11-09T03:48:22.552644+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.552644+0000","last_undegraded":"2023-11-09T03:48:22.552644+0000","last_fullsized":"2023-11-09T03:48:22.552644+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551900+0000","last_change":"2023-11-09T03:48:22.551900+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551900+0000","last_undegraded":"2023-11-09T03:48:22.551900+0000","last_fullsized":"2023-11-09T03:48:22.551900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549131+0000","last_change":"2023-11-09T03:48:22.549131+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549131+0000","last_undegraded":"2023-11-09T03:48:22.549131+0000","last_fullsized":"2023-11-09T03:48:22.549131+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.548919+0000","last_change":"2023-11-09T03:48:22.548919+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.548919+0000","last_undegraded":"2023-11-09T03:48:22.548919+0000","last_fullsized":"2023-11-09T03:48:22.548919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.554246+0000","last_change":"2023-11-09T03:48:22.554246+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.554246+0000","last_undegraded":"2023-11-09T03:48:22.554246+0000","last_fullsized":"2023-11-09T03:48:22.554246+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.568615+0000","last_change":"2023-11-09T03:48:22.562765+0000","last_active":"2023-11-09T03:48:22.568615+0000","last_peered":"2023-11-09T03:48:22.568615+0000","last_clean":"2023-11-09T03:48:22.568615+0000","last_became_active":"2023-11-09T03:48:22.562455+0000","last_became_peered":"2023-11-09T03:48:22.562455+0000","last_unstale":"2023-11-09T03:48:22.568615+0000","last_undegraded":"2023-11-09T03:48:22.568615+0000","last_fullsized":"2023-11-09T03:48:22.568615+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:55:13.543207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545671+0000","last_change":"2023-11-09T03:48:01.449077+0000","last_active":"2023-11-09T03:48:21.545671+0000","last_peered":"2023-11-09T03:48:21.545671+0000","last_clean":"2023-11-09T03:48:21.545671+0000","last_became_active":"2023-11-09T03:48:01.448806+0000","last_became_peered":"2023-11-09T03:48:01.448806+0000","last_unstale":"2023-11-09T03:48:21.545671+0000","last_undegraded":"2023-11-09T03:48:21.545671+0000","last_fullsized":"2023-11-09T03:48:21.545671+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:16:00.220961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542885+0000","last_change":"2023-11-09T03:48:02.611756+0000","last_active":"2023-11-09T03:48:21.542885+0000","last_peered":"2023-11-09T03:48:21.542885+0000","last_clean":"2023-11-09T03:48:21.542885+0000","last_became_active":"2023-11-09T03:48:00.445880+0000","last_became_peered":"2023-11-09T03:48:00.445880+0000","last_unstale":"2023-11-09T03:48:21.542885+0000","last_undegraded":"2023-11-09T03:48:21.542885+0000","last_fullsized":"2023-11-09T03:48:21.542885+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:15:35.281296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085514900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539506+0000","last_change":"2023-11-09T03:47:02.581917+0000","last_active":"2023-11-09T03:48:21.539506+0000","last_peered":"2023-11-09T03:48:21.539506+0000","last_clean":"2023-11-09T03:48:21.539506+0000","last_became_active":"2023-11-09T03:47:01.588082+0000","last_became_peered":"2023-11-09T03:47:01.588082+0000","last_unstale":"2023-11-09T03:48:21.539506+0000","last_undegraded":"2023-11-09T03:48:21.539506+0000","last_fullsized":"2023-11-09T03:48:21.539506+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:44:07.653557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034538899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.553249+0000","last_change":"2023-11-09T03:48:22.553249+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.553249+0000","last_undegraded":"2023-11-09T03:48:22.553249+0000","last_fullsized":"2023-11-09T03:48:22.553249+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.571401+0000","last_change":"2023-11-09T03:48:22.566342+0000","last_active":"2023-11-09T03:48:22.571401+0000","last_peered":"2023-11-09T03:48:22.571401+0000","last_clean":"2023-11-09T03:48:22.571401+0000","last_became_active":"2023-11-09T03:48:22.566028+0000","last_became_peered":"2023-11-09T03:48:22.566028+0000","last_unstale":"2023-11-09T03:48:22.571401+0000","last_undegraded":"2023-11-09T03:48:22.571401+0000","last_fullsized":"2023-11-09T03:48:22.571401+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:31.602666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542096+0000","last_change":"2023-11-09T03:48:01.456245+0000","last_active":"2023-11-09T03:48:21.542096+0000","last_peered":"2023-11-09T03:48:21.542096+0000","last_clean":"2023-11-09T03:48:21.542096+0000","last_became_active":"2023-11-09T03:48:01.455944+0000","last_became_peered":"2023-11-09T03:48:01.455944+0000","last_unstale":"2023-11-09T03:48:21.542096+0000","last_undegraded":"2023-11-09T03:48:21.542096+0000","last_fullsized":"2023-11-09T03:48:21.542096+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:06:14.849196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543120+0000","last_change":"2023-11-09T03:48:02.612341+0000","last_active":"2023-11-09T03:48:21.543120+0000","last_peered":"2023-11-09T03:48:21.543120+0000","last_clean":"2023-11-09T03:48:21.543120+0000","last_became_active":"2023-11-09T03:48:00.444426+0000","last_became_peered":"2023-11-09T03:48:00.444426+0000","last_unstale":"2023-11-09T03:48:21.543120+0000","last_undegraded":"2023-11-09T03:48:21.543120+0000","last_fullsized":"2023-11-09T03:48:21.543120+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:19:53.969861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010812420000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545670+0000","last_change":"2023-11-09T03:47:03.029063+0000","last_active":"2023-11-09T03:48:21.545670+0000","last_peered":"2023-11-09T03:48:21.545670+0000","last_clean":"2023-11-09T03:48:21.545670+0000","last_became_active":"2023-11-09T03:47:01.593582+0000","last_became_peered":"2023-11-09T03:47:01.593582+0000","last_unstale":"2023-11-09T03:48:21.545670+0000","last_undegraded":"2023-11-09T03:48:21.545670+0000","last_fullsized":"2023-11-09T03:48:21.545670+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:56:20.649937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041855100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.546617+0000","last_change":"2023-11-09T03:48:22.546617+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.546617+0000","last_undegraded":"2023-11-09T03:48:22.546617+0000","last_fullsized":"2023-11-09T03:48:22.546617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.564376+0000","last_change":"2023-11-09T03:48:22.562238+0000","last_active":"2023-11-09T03:48:22.564376+0000","last_peered":"2023-11-09T03:48:22.564376+0000","last_clean":"2023-11-09T03:48:22.564376+0000","last_became_active":"2023-11-09T03:48:22.561984+0000","last_became_peered":"2023-11-09T03:48:22.561984+0000","last_unstale":"2023-11-09T03:48:22.564376+0000","last_undegraded":"2023-11-09T03:48:22.564376+0000","last_fullsized":"2023-11-09T03:48:22.564376+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:28:06.557471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541062+0000","last_change":"2023-11-09T03:48:01.450271+0000","last_active":"2023-11-09T03:48:21.541062+0000","last_peered":"2023-11-09T03:48:21.541062+0000","last_clean":"2023-11-09T03:48:21.541062+0000","last_became_active":"2023-11-09T03:48:01.449998+0000","last_became_peered":"2023-11-09T03:48:01.449998+0000","last_unstale":"2023-11-09T03:48:21.541062+0000","last_undegraded":"2023-11-09T03:48:21.541062+0000","last_fullsized":"2023-11-09T03:48:21.541062+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:12:27.170650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545599+0000","last_change":"2023-11-09T03:48:02.852114+0000","last_active":"2023-11-09T03:48:21.545599+0000","last_peered":"2023-11-09T03:48:21.545599+0000","last_clean":"2023-11-09T03:48:21.545599+0000","last_became_active":"2023-11-09T03:48:00.447203+0000","last_became_peered":"2023-11-09T03:48:00.447203+0000","last_unstale":"2023-11-09T03:48:21.545599+0000","last_undegraded":"2023-11-09T03:48:21.545599+0000","last_fullsized":"2023-11-09T03:48:21.545599+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:11:36.826430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049189500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543495+0000","last_change":"2023-11-09T03:47:02.619336+0000","last_active":"2023-11-09T03:48:21.543495+0000","last_peered":"2023-11-09T03:48:21.543495+0000","last_clean":"2023-11-09T03:48:21.543495+0000","last_became_active":"2023-11-09T03:47:01.587749+0000","last_became_peered":"2023-11-09T03:47:01.587749+0000","last_unstale":"2023-11-09T03:48:21.543495+0000","last_undegraded":"2023-11-09T03:48:21.543495+0000","last_fullsized":"2023-11-09T03:48:21.543495+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063032099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551692+0000","last_change":"2023-11-09T03:48:22.551692+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551692+0000","last_undegraded":"2023-11-09T03:48:22.551692+0000","last_fullsized":"2023-11-09T03:48:22.551692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.559640+0000","last_change":"2023-11-09T03:48:22.557530+0000","last_active":"2023-11-09T03:48:22.559640+0000","last_peered":"2023-11-09T03:48:22.559640+0000","last_clean":"2023-11-09T03:48:22.559640+0000","last_became_active":"2023-11-09T03:48:22.556685+0000","last_became_peered":"2023-11-09T03:48:22.556685+0000","last_unstale":"2023-11-09T03:48:22.559640+0000","last_undegraded":"2023-11-09T03:48:22.559640+0000","last_fullsized":"2023-11-09T03:48:22.559640+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:06:49.685032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547581+0000","last_change":"2023-11-09T03:48:01.454147+0000","last_active":"2023-11-09T03:48:21.547581+0000","last_peered":"2023-11-09T03:48:21.547581+0000","last_clean":"2023-11-09T03:48:21.547581+0000","last_became_active":"2023-11-09T03:48:01.453868+0000","last_became_peered":"2023-11-09T03:48:01.453868+0000","last_unstale":"2023-11-09T03:48:21.547581+0000","last_undegraded":"2023-11-09T03:48:21.547581+0000","last_fullsized":"2023-11-09T03:48:21.547581+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:05:09.655318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542848+0000","last_change":"2023-11-09T03:48:02.612000+0000","last_active":"2023-11-09T03:48:21.542848+0000","last_peered":"2023-11-09T03:48:21.542848+0000","last_clean":"2023-11-09T03:48:21.542848+0000","last_became_active":"2023-11-09T03:48:00.441667+0000","last_became_peered":"2023-11-09T03:48:00.441667+0000","last_unstale":"2023-11-09T03:48:21.542848+0000","last_undegraded":"2023-11-09T03:48:21.542848+0000","last_fullsized":"2023-11-09T03:48:21.542848+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:12:26.164644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000966781,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539495+0000","last_change":"2023-11-09T03:47:02.581658+0000","last_active":"2023-11-09T03:48:21.539495+0000","last_peered":"2023-11-09T03:48:21.539495+0000","last_clean":"2023-11-09T03:48:21.539495+0000","last_became_active":"2023-11-09T03:47:01.588612+0000","last_became_peered":"2023-11-09T03:47:01.588612+0000","last_unstale":"2023-11-09T03:48:21.539495+0000","last_undegraded":"2023-11-09T03:48:21.539495+0000","last_fullsized":"2023-11-09T03:48:21.539495+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045238500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.547531+0000","last_change":"2023-11-09T03:48:22.547531+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.547531+0000","last_undegraded":"2023-11-09T03:48:22.547531+0000","last_fullsized":"2023-11-09T03:48:22.547531+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.560046+0000","last_change":"2023-11-09T03:48:22.557828+0000","last_active":"2023-11-09T03:48:22.560046+0000","last_peered":"2023-11-09T03:48:22.560046+0000","last_clean":"2023-11-09T03:48:22.560046+0000","last_became_active":"2023-11-09T03:48:22.557588+0000","last_became_peered":"2023-11-09T03:48:22.557588+0000","last_unstale":"2023-11-09T03:48:22.560046+0000","last_undegraded":"2023-11-09T03:48:22.560046+0000","last_fullsized":"2023-11-09T03:48:22.560046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:36:15.980235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545124+0000","last_change":"2023-11-09T03:48:01.449834+0000","last_active":"2023-11-09T03:48:21.545124+0000","last_peered":"2023-11-09T03:48:21.545124+0000","last_clean":"2023-11-09T03:48:21.545124+0000","last_became_active":"2023-11-09T03:48:01.449552+0000","last_became_peered":"2023-11-09T03:48:01.449552+0000","last_unstale":"2023-11-09T03:48:21.545124+0000","last_undegraded":"2023-11-09T03:48:21.545124+0000","last_fullsized":"2023-11-09T03:48:21.545124+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:11:37.832436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542114+0000","last_change":"2023-11-09T03:48:02.682197+0000","last_active":"2023-11-09T03:48:21.542114+0000","last_peered":"2023-11-09T03:48:21.542114+0000","last_clean":"2023-11-09T03:48:21.542114+0000","last_became_active":"2023-11-09T03:48:00.445616+0000","last_became_peered":"2023-11-09T03:48:00.445616+0000","last_unstale":"2023-11-09T03:48:21.542114+0000","last_undegraded":"2023-11-09T03:48:21.542114+0000","last_fullsized":"2023-11-09T03:48:21.542114+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:57:03.474084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000416494,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542622+0000","last_change":"2023-11-09T03:47:02.618828+0000","last_active":"2023-11-09T03:48:21.542622+0000","last_peered":"2023-11-09T03:48:21.542622+0000","last_clean":"2023-11-09T03:48:21.542622+0000","last_became_active":"2023-11-09T03:47:01.590679+0000","last_became_peered":"2023-11-09T03:47:01.590679+0000","last_unstale":"2023-11-09T03:48:21.542622+0000","last_undegraded":"2023-11-09T03:48:21.542622+0000","last_fullsized":"2023-11-09T03:48:21.542622+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:06:30.982987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070668,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.546525+0000","last_change":"2023-11-09T03:48:22.546525+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.546525+0000","last_undegraded":"2023-11-09T03:48:22.546525+0000","last_fullsized":"2023-11-09T03:48:22.546525+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.560446+0000","last_change":"2023-11-09T03:48:22.556590+0000","last_active":"2023-11-09T03:48:22.560446+0000","last_peered":"2023-11-09T03:48:22.560446+0000","last_clean":"2023-11-09T03:48:22.560446+0000","last_became_active":"2023-11-09T03:48:22.556295+0000","last_became_peered":"2023-11-09T03:48:22.556295+0000","last_unstale":"2023-11-09T03:48:22.560446+0000","last_undegraded":"2023-11-09T03:48:22.560446+0000","last_fullsized":"2023-11-09T03:48:22.560446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:09:09.041542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.546994+0000","last_change":"2023-11-09T03:48:01.447647+0000","last_active":"2023-11-09T03:48:21.546994+0000","last_peered":"2023-11-09T03:48:21.546994+0000","last_clean":"2023-11-09T03:48:21.546994+0000","last_became_active":"2023-11-09T03:48:01.447302+0000","last_became_peered":"2023-11-09T03:48:01.447302+0000","last_unstale":"2023-11-09T03:48:21.546994+0000","last_undegraded":"2023-11-09T03:48:21.546994+0000","last_fullsized":"2023-11-09T03:48:21.546994+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:08:47.931939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539407+0000","last_change":"2023-11-09T03:48:03.327731+0000","last_active":"2023-11-09T03:48:21.539407+0000","last_peered":"2023-11-09T03:48:21.539407+0000","last_clean":"2023-11-09T03:48:21.539407+0000","last_became_active":"2023-11-09T03:48:00.449660+0000","last_became_peered":"2023-11-09T03:48:00.449660+0000","last_unstale":"2023-11-09T03:48:21.539407+0000","last_undegraded":"2023-11-09T03:48:21.539407+0000","last_fullsized":"2023-11-09T03:48:21.539407+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:02:07.940908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037816599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.546874+0000","last_change":"2023-11-09T03:47:03.146528+0000","last_active":"2023-11-09T03:48:21.546874+0000","last_peered":"2023-11-09T03:48:21.546874+0000","last_clean":"2023-11-09T03:48:21.546874+0000","last_became_active":"2023-11-09T03:47:01.587060+0000","last_became_peered":"2023-11-09T03:47:01.587060+0000","last_unstale":"2023-11-09T03:48:21.546874+0000","last_undegraded":"2023-11-09T03:48:21.546874+0000","last_fullsized":"2023-11-09T03:48:21.546874+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:39:15.238496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079064999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.547973+0000","last_change":"2023-11-09T03:48:22.547973+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.547973+0000","last_undegraded":"2023-11-09T03:48:22.547973+0000","last_fullsized":"2023-11-09T03:48:22.547973+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.569334+0000","last_change":"2023-11-09T03:48:22.565708+0000","last_active":"2023-11-09T03:48:22.569334+0000","last_peered":"2023-11-09T03:48:22.569334+0000","last_clean":"2023-11-09T03:48:22.569334+0000","last_became_active":"2023-11-09T03:48:22.563961+0000","last_became_peered":"2023-11-09T03:48:22.563961+0000","last_unstale":"2023-11-09T03:48:22.569334+0000","last_undegraded":"2023-11-09T03:48:22.569334+0000","last_fullsized":"2023-11-09T03:48:22.569334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:03:03.041986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539139+0000","last_change":"2023-11-09T03:48:01.453939+0000","last_active":"2023-11-09T03:48:21.539139+0000","last_peered":"2023-11-09T03:48:21.539139+0000","last_clean":"2023-11-09T03:48:21.539139+0000","last_became_active":"2023-11-09T03:48:01.453600+0000","last_became_peered":"2023-11-09T03:48:01.453600+0000","last_unstale":"2023-11-09T03:48:21.539139+0000","last_undegraded":"2023-11-09T03:48:21.539139+0000","last_fullsized":"2023-11-09T03:48:21.539139+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:45:10.522254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547099+0000","last_change":"2023-11-09T03:48:02.440910+0000","last_active":"2023-11-09T03:48:21.547099+0000","last_peered":"2023-11-09T03:48:21.547099+0000","last_clean":"2023-11-09T03:48:21.547099+0000","last_became_active":"2023-11-09T03:48:00.443791+0000","last_became_peered":"2023-11-09T03:48:00.443791+0000","last_unstale":"2023-11-09T03:48:21.547099+0000","last_undegraded":"2023-11-09T03:48:21.547099+0000","last_fullsized":"2023-11-09T03:48:21.547099+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:11:36.826430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069223400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547414+0000","last_change":"2023-11-09T03:47:03.148246+0000","last_active":"2023-11-09T03:48:21.547414+0000","last_peered":"2023-11-09T03:48:21.547414+0000","last_clean":"2023-11-09T03:48:21.547414+0000","last_became_active":"2023-11-09T03:47:01.590988+0000","last_became_peered":"2023-11-09T03:47:01.590988+0000","last_unstale":"2023-11-09T03:48:21.547414+0000","last_undegraded":"2023-11-09T03:48:21.547414+0000","last_fullsized":"2023-11-09T03:48:21.547414+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:07:20.785445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055651099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551090+0000","last_change":"2023-11-09T03:48:22.551090+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551090+0000","last_undegraded":"2023-11-09T03:48:22.551090+0000","last_fullsized":"2023-11-09T03:48:22.551090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.569365+0000","last_change":"2023-11-09T03:48:22.565920+0000","last_active":"2023-11-09T03:48:22.569365+0000","last_peered":"2023-11-09T03:48:22.569365+0000","last_clean":"2023-11-09T03:48:22.569365+0000","last_became_active":"2023-11-09T03:48:22.564592+0000","last_became_peered":"2023-11-09T03:48:22.564592+0000","last_unstale":"2023-11-09T03:48:22.569365+0000","last_undegraded":"2023-11-09T03:48:22.569365+0000","last_fullsized":"2023-11-09T03:48:22.569365+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:41:46.534565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539107+0000","last_change":"2023-11-09T03:48:01.450557+0000","last_active":"2023-11-09T03:48:21.539107+0000","last_peered":"2023-11-09T03:48:21.539107+0000","last_clean":"2023-11-09T03:48:21.539107+0000","last_became_active":"2023-11-09T03:48:01.450291+0000","last_became_peered":"2023-11-09T03:48:01.450291+0000","last_unstale":"2023-11-09T03:48:21.539107+0000","last_undegraded":"2023-11-09T03:48:21.539107+0000","last_fullsized":"2023-11-09T03:48:21.539107+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:21:29.699594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545002+0000","last_change":"2023-11-09T03:48:02.852010+0000","last_active":"2023-11-09T03:48:21.545002+0000","last_peered":"2023-11-09T03:48:21.545002+0000","last_clean":"2023-11-09T03:48:21.545002+0000","last_became_active":"2023-11-09T03:48:00.445267+0000","last_became_peered":"2023-11-09T03:48:00.445267+0000","last_unstale":"2023-11-09T03:48:21.545002+0000","last_undegraded":"2023-11-09T03:48:21.545002+0000","last_fullsized":"2023-11-09T03:48:21.545002+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:17.989628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064559199999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":50,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540983+0000","last_change":"2023-11-09T03:47:02.582754+0000","last_active":"2023-11-09T03:48:21.540983+0000","last_peered":"2023-11-09T03:48:21.540983+0000","last_clean":"2023-11-09T03:48:21.540983+0000","last_became_active":"2023-11-09T03:47:01.589715+0000","last_became_peered":"2023-11-09T03:47:01.589715+0000","last_unstale":"2023-11-09T03:48:21.540983+0000","last_undegraded":"2023-11-09T03:48:21.540983+0000","last_fullsized":"2023-11-09T03:48:21.540983+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099342799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.546539+0000","last_change":"2023-11-09T03:48:22.546539+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.546539+0000","last_undegraded":"2023-11-09T03:48:22.546539+0000","last_fullsized":"2023-11-09T03:48:22.546539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.571909+0000","last_change":"2023-11-09T03:48:22.566859+0000","last_active":"2023-11-09T03:48:22.571909+0000","last_peered":"2023-11-09T03:48:22.571909+0000","last_clean":"2023-11-09T03:48:22.571909+0000","last_became_active":"2023-11-09T03:48:22.566080+0000","last_became_peered":"2023-11-09T03:48:22.566080+0000","last_unstale":"2023-11-09T03:48:22.571909+0000","last_undegraded":"2023-11-09T03:48:22.571909+0000","last_fullsized":"2023-11-09T03:48:22.571909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:14:56.468255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543312+0000","last_change":"2023-11-09T03:48:01.452456+0000","last_active":"2023-11-09T03:48:21.543312+0000","last_peered":"2023-11-09T03:48:21.543312+0000","last_clean":"2023-11-09T03:48:21.543312+0000","last_became_active":"2023-11-09T03:48:01.452059+0000","last_became_peered":"2023-11-09T03:48:01.452059+0000","last_unstale":"2023-11-09T03:48:21.543312+0000","last_undegraded":"2023-11-09T03:48:21.543312+0000","last_fullsized":"2023-11-09T03:48:21.543312+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:57:56.855869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541181+0000","last_change":"2023-11-09T03:48:02.442557+0000","last_active":"2023-11-09T03:48:21.541181+0000","last_peered":"2023-11-09T03:48:21.541181+0000","last_clean":"2023-11-09T03:48:21.541181+0000","last_became_active":"2023-11-09T03:48:00.444823+0000","last_became_peered":"2023-11-09T03:48:00.444823+0000","last_unstale":"2023-11-09T03:48:21.541181+0000","last_undegraded":"2023-11-09T03:48:21.541181+0000","last_fullsized":"2023-11-09T03:48:21.541181+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:19:53.969861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075076299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":47,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545453+0000","last_change":"2023-11-09T03:47:02.724896+0000","last_active":"2023-11-09T03:48:21.545453+0000","last_peered":"2023-11-09T03:48:21.545453+0000","last_clean":"2023-11-09T03:48:21.545453+0000","last_became_active":"2023-11-09T03:47:01.592784+0000","last_became_peered":"2023-11-09T03:47:01.592784+0000","last_unstale":"2023-11-09T03:48:21.545453+0000","last_undegraded":"2023-11-09T03:48:21.545453+0000","last_fullsized":"2023-11-09T03:48:21.545453+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:51:32.016204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056683500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.548469+0000","last_change":"2023-11-09T03:48:22.548469+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.548469+0000","last_undegraded":"2023-11-09T03:48:22.548469+0000","last_fullsized":"2023-11-09T03:48:22.548469+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.571850+0000","last_change":"2023-11-09T03:48:22.565279+0000","last_active":"2023-11-09T03:48:22.571850+0000","last_peered":"2023-11-09T03:48:22.571850+0000","last_clean":"2023-11-09T03:48:22.571850+0000","last_became_active":"2023-11-09T03:48:22.564998+0000","last_became_peered":"2023-11-09T03:48:22.564998+0000","last_unstale":"2023-11-09T03:48:22.571850+0000","last_undegraded":"2023-11-09T03:48:22.571850+0000","last_fullsized":"2023-11-09T03:48:22.571850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:24:48.220122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540896+0000","last_change":"2023-11-09T03:48:01.452167+0000","last_active":"2023-11-09T03:48:21.540896+0000","last_peered":"2023-11-09T03:48:21.540896+0000","last_clean":"2023-11-09T03:48:21.540896+0000","last_became_active":"2023-11-09T03:48:01.451780+0000","last_became_peered":"2023-11-09T03:48:01.451780+0000","last_unstale":"2023-11-09T03:48:21.540896+0000","last_undegraded":"2023-11-09T03:48:21.540896+0000","last_fullsized":"2023-11-09T03:48:21.540896+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:15:36.287302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543137+0000","last_change":"2023-11-09T03:48:02.612246+0000","last_active":"2023-11-09T03:48:21.543137+0000","last_peered":"2023-11-09T03:48:21.543137+0000","last_clean":"2023-11-09T03:48:21.543137+0000","last_became_active":"2023-11-09T03:48:00.444629+0000","last_became_peered":"2023-11-09T03:48:00.444629+0000","last_unstale":"2023-11-09T03:48:21.543137+0000","last_undegraded":"2023-11-09T03:48:21.543137+0000","last_fullsized":"2023-11-09T03:48:21.543137+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:51:17.754054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098167299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541889+0000","last_change":"2023-11-09T03:47:03.146527+0000","last_active":"2023-11-09T03:48:21.541889+0000","last_peered":"2023-11-09T03:48:21.541889+0000","last_clean":"2023-11-09T03:48:21.541889+0000","last_became_active":"2023-11-09T03:47:01.587858+0000","last_became_peered":"2023-11-09T03:47:01.587858+0000","last_unstale":"2023-11-09T03:48:21.541889+0000","last_undegraded":"2023-11-09T03:48:21.541889+0000","last_fullsized":"2023-11-09T03:48:21.541889+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:56:04.201126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084439599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.548328+0000","last_change":"2023-11-09T03:48:22.548328+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.548328+0000","last_undegraded":"2023-11-09T03:48:22.548328+0000","last_fullsized":"2023-11-09T03:48:22.548328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.558055+0000","last_change":"2023-11-09T03:48:22.553920+0000","last_active":"2023-11-09T03:48:22.558055+0000","last_peered":"2023-11-09T03:48:22.558055+0000","last_clean":"2023-11-09T03:48:22.558055+0000","last_became_active":"2023-11-09T03:48:22.552644+0000","last_became_peered":"2023-11-09T03:48:22.552644+0000","last_unstale":"2023-11-09T03:48:22.558055+0000","last_undegraded":"2023-11-09T03:48:22.558055+0000","last_fullsized":"2023-11-09T03:48:22.558055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:59.646501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545255+0000","last_change":"2023-11-09T03:48:01.450685+0000","last_active":"2023-11-09T03:48:21.545255+0000","last_peered":"2023-11-09T03:48:21.545255+0000","last_clean":"2023-11-09T03:48:21.545255+0000","last_became_active":"2023-11-09T03:48:01.450445+0000","last_became_peered":"2023-11-09T03:48:01.450445+0000","last_unstale":"2023-11-09T03:48:21.545255+0000","last_undegraded":"2023-11-09T03:48:21.545255+0000","last_fullsized":"2023-11-09T03:48:21.545255+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:16:00.220961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547739+0000","last_change":"2023-11-09T03:48:02.441928+0000","last_active":"2023-11-09T03:48:21.547739+0000","last_peered":"2023-11-09T03:48:21.547739+0000","last_clean":"2023-11-09T03:48:21.547739+0000","last_became_active":"2023-11-09T03:48:00.444417+0000","last_became_peered":"2023-11-09T03:48:00.444417+0000","last_unstale":"2023-11-09T03:48:21.547739+0000","last_undegraded":"2023-11-09T03:48:21.547739+0000","last_fullsized":"2023-11-09T03:48:21.547739+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:55.450096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072428900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":47,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545707+0000","last_change":"2023-11-09T03:47:02.725223+0000","last_active":"2023-11-09T03:48:21.545707+0000","last_peered":"2023-11-09T03:48:21.545707+0000","last_clean":"2023-11-09T03:48:21.545707+0000","last_became_active":"2023-11-09T03:47:01.590635+0000","last_became_peered":"2023-11-09T03:47:01.590635+0000","last_unstale":"2023-11-09T03:48:21.545707+0000","last_undegraded":"2023-11-09T03:48:21.545707+0000","last_fullsized":"2023-11-09T03:48:21.545707+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:05.754357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051685100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551472+0000","last_change":"2023-11-09T03:48:22.551472+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551472+0000","last_undegraded":"2023-11-09T03:48:22.551472+0000","last_fullsized":"2023-11-09T03:48:22.551472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.569184+0000","last_change":"2023-11-09T03:48:22.563841+0000","last_active":"2023-11-09T03:48:22.569184+0000","last_peered":"2023-11-09T03:48:22.569184+0000","last_clean":"2023-11-09T03:48:22.569184+0000","last_became_active":"2023-11-09T03:48:22.563590+0000","last_became_peered":"2023-11-09T03:48:22.563590+0000","last_unstale":"2023-11-09T03:48:22.569184+0000","last_undegraded":"2023-11-09T03:48:22.569184+0000","last_fullsized":"2023-11-09T03:48:22.569184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:10:36.225903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542779+0000","last_change":"2023-11-09T03:48:01.453143+0000","last_active":"2023-11-09T03:48:21.542779+0000","last_peered":"2023-11-09T03:48:21.542779+0000","last_clean":"2023-11-09T03:48:21.542779+0000","last_became_active":"2023-11-09T03:48:01.452873+0000","last_became_peered":"2023-11-09T03:48:01.452873+0000","last_unstale":"2023-11-09T03:48:21.542779+0000","last_undegraded":"2023-11-09T03:48:21.542779+0000","last_fullsized":"2023-11-09T03:48:21.542779+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:06:28.575429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.548086+0000","last_change":"2023-11-09T03:48:02.442416+0000","last_active":"2023-11-09T03:48:21.548086+0000","last_peered":"2023-11-09T03:48:21.548086+0000","last_clean":"2023-11-09T03:48:21.548086+0000","last_became_active":"2023-11-09T03:48:00.443765+0000","last_became_peered":"2023-11-09T03:48:00.443765+0000","last_unstale":"2023-11-09T03:48:21.548086+0000","last_undegraded":"2023-11-09T03:48:21.548086+0000","last_fullsized":"2023-11-09T03:48:21.548086+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:41:22.321811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012468850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":50,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541230+0000","last_change":"2023-11-09T03:47:02.582739+0000","last_active":"2023-11-09T03:48:21.541230+0000","last_peered":"2023-11-09T03:48:21.541230+0000","last_clean":"2023-11-09T03:48:21.541230+0000","last_became_active":"2023-11-09T03:47:01.592743+0000","last_became_peered":"2023-11-09T03:47:01.592743+0000","last_unstale":"2023-11-09T03:48:21.541230+0000","last_undegraded":"2023-11-09T03:48:21.541230+0000","last_fullsized":"2023-11-09T03:48:21.541230+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051802700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.546563+0000","last_change":"2023-11-09T03:48:22.546563+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.546563+0000","last_undegraded":"2023-11-09T03:48:22.546563+0000","last_fullsized":"2023-11-09T03:48:22.546563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.561070+0000","last_change":"2023-11-09T03:48:22.559947+0000","last_active":"2023-11-09T03:48:22.561070+0000","last_peered":"2023-11-09T03:48:22.561070+0000","last_clean":"2023-11-09T03:48:22.561070+0000","last_became_active":"2023-11-09T03:48:22.559667+0000","last_became_peered":"2023-11-09T03:48:22.559667+0000","last_unstale":"2023-11-09T03:48:22.561070+0000","last_undegraded":"2023-11-09T03:48:22.561070+0000","last_fullsized":"2023-11-09T03:48:22.561070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:57.844631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540463+0000","last_change":"2023-11-09T03:48:01.442195+0000","last_active":"2023-11-09T03:48:21.540463+0000","last_peered":"2023-11-09T03:48:21.540463+0000","last_clean":"2023-11-09T03:48:21.540463+0000","last_became_active":"2023-11-09T03:48:01.441887+0000","last_became_peered":"2023-11-09T03:48:01.441887+0000","last_unstale":"2023-11-09T03:48:21.540463+0000","last_undegraded":"2023-11-09T03:48:21.540463+0000","last_fullsized":"2023-11-09T03:48:21.540463+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:06:28.575429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540528+0000","last_change":"2023-11-09T03:48:02.442061+0000","last_active":"2023-11-09T03:48:21.540528+0000","last_peered":"2023-11-09T03:48:21.540528+0000","last_clean":"2023-11-09T03:48:21.540528+0000","last_became_active":"2023-11-09T03:48:00.445921+0000","last_became_peered":"2023-11-09T03:48:00.445921+0000","last_unstale":"2023-11-09T03:48:21.540528+0000","last_undegraded":"2023-11-09T03:48:21.540528+0000","last_fullsized":"2023-11-09T03:48:21.540528+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:50:55.299235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012719770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":47,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545111+0000","last_change":"2023-11-09T03:47:02.724953+0000","last_active":"2023-11-09T03:48:21.545111+0000","last_peered":"2023-11-09T03:48:21.545111+0000","last_clean":"2023-11-09T03:48:21.545111+0000","last_became_active":"2023-11-09T03:47:01.587133+0000","last_became_peered":"2023-11-09T03:47:01.587133+0000","last_unstale":"2023-11-09T03:48:21.545111+0000","last_undegraded":"2023-11-09T03:48:21.545111+0000","last_fullsized":"2023-11-09T03:48:21.545111+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:23:48.521161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066497599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.547867+0000","last_change":"2023-11-09T03:48:22.547867+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.547867+0000","last_undegraded":"2023-11-09T03:48:22.547867+0000","last_fullsized":"2023-11-09T03:48:22.547867+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.563196+0000","last_change":"2023-11-09T03:48:22.556449+0000","last_active":"2023-11-09T03:48:22.563196+0000","last_peered":"2023-11-09T03:48:22.563196+0000","last_clean":"2023-11-09T03:48:22.563196+0000","last_became_active":"2023-11-09T03:48:22.556170+0000","last_became_peered":"2023-11-09T03:48:22.556170+0000","last_unstale":"2023-11-09T03:48:22.563196+0000","last_undegraded":"2023-11-09T03:48:22.563196+0000","last_fullsized":"2023-11-09T03:48:22.563196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:09:09.041542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539364+0000","last_change":"2023-11-09T03:48:01.450507+0000","last_active":"2023-11-09T03:48:21.539364+0000","last_peered":"2023-11-09T03:48:21.539364+0000","last_clean":"2023-11-09T03:48:21.539364+0000","last_became_active":"2023-11-09T03:48:01.450191+0000","last_became_peered":"2023-11-09T03:48:01.450191+0000","last_unstale":"2023-11-09T03:48:21.539364+0000","last_undegraded":"2023-11-09T03:48:21.539364+0000","last_fullsized":"2023-11-09T03:48:21.539364+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:32:40.342502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540558+0000","last_change":"2023-11-09T03:48:02.442120+0000","last_active":"2023-11-09T03:48:21.540558+0000","last_peered":"2023-11-09T03:48:21.540558+0000","last_clean":"2023-11-09T03:48:21.540558+0000","last_became_active":"2023-11-09T03:48:00.445504+0000","last_became_peered":"2023-11-09T03:48:00.445504+0000","last_unstale":"2023-11-09T03:48:21.540558+0000","last_undegraded":"2023-11-09T03:48:21.540558+0000","last_fullsized":"2023-11-09T03:48:21.540558+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:35:53.864627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001095766,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541998+0000","last_change":"2023-11-09T03:47:03.146217+0000","last_active":"2023-11-09T03:48:21.541998+0000","last_peered":"2023-11-09T03:48:21.541998+0000","last_clean":"2023-11-09T03:48:21.541998+0000","last_became_active":"2023-11-09T03:47:01.587598+0000","last_became_peered":"2023-11-09T03:47:01.587598+0000","last_unstale":"2023-11-09T03:48:21.541998+0000","last_undegraded":"2023-11-09T03:48:21.541998+0000","last_fullsized":"2023-11-09T03:48:21.541998+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:15:22.723455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090126400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550235+0000","last_change":"2023-11-09T03:48:22.550235+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550235+0000","last_undegraded":"2023-11-09T03:48:22.550235+0000","last_fullsized":"2023-11-09T03:48:22.550235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.564011+0000","last_change":"2023-11-09T03:48:22.561567+0000","last_active":"2023-11-09T03:48:22.564011+0000","last_peered":"2023-11-09T03:48:22.564011+0000","last_clean":"2023-11-09T03:48:22.564011+0000","last_became_active":"2023-11-09T03:48:22.561266+0000","last_became_peered":"2023-11-09T03:48:22.561266+0000","last_unstale":"2023-11-09T03:48:22.564011+0000","last_undegraded":"2023-11-09T03:48:22.564011+0000","last_fullsized":"2023-11-09T03:48:22.564011+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:21.472203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542405+0000","last_change":"2023-11-09T03:48:01.454878+0000","last_active":"2023-11-09T03:48:21.542405+0000","last_peered":"2023-11-09T03:48:21.542405+0000","last_clean":"2023-11-09T03:48:21.542405+0000","last_became_active":"2023-11-09T03:48:01.454591+0000","last_became_peered":"2023-11-09T03:48:01.454591+0000","last_unstale":"2023-11-09T03:48:21.542405+0000","last_undegraded":"2023-11-09T03:48:21.542405+0000","last_fullsized":"2023-11-09T03:48:21.542405+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:38.536898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539170+0000","last_change":"2023-11-09T03:48:03.327800+0000","last_active":"2023-11-09T03:48:21.539170+0000","last_peered":"2023-11-09T03:48:21.539170+0000","last_clean":"2023-11-09T03:48:21.539170+0000","last_became_active":"2023-11-09T03:48:00.446576+0000","last_became_peered":"2023-11-09T03:48:00.446576+0000","last_unstale":"2023-11-09T03:48:21.539170+0000","last_undegraded":"2023-11-09T03:48:21.539170+0000","last_fullsized":"2023-11-09T03:48:21.539170+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:37:08.851102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054679700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547427+0000","last_change":"2023-11-09T03:47:03.148283+0000","last_active":"2023-11-09T03:48:21.547427+0000","last_peered":"2023-11-09T03:48:21.547427+0000","last_clean":"2023-11-09T03:48:21.547427+0000","last_became_active":"2023-11-09T03:47:01.588841+0000","last_became_peered":"2023-11-09T03:47:01.588841+0000","last_unstale":"2023-11-09T03:48:21.547427+0000","last_undegraded":"2023-11-09T03:48:21.547427+0000","last_fullsized":"2023-11-09T03:48:21.547427+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:47:44.880741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000436081,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.552336+0000","last_change":"2023-11-09T03:48:22.552336+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.552336+0000","last_undegraded":"2023-11-09T03:48:22.552336+0000","last_fullsized":"2023-11-09T03:48:22.552336+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.558220+0000","last_change":"2023-11-09T03:48:22.556480+0000","last_active":"2023-11-09T03:48:22.558220+0000","last_peered":"2023-11-09T03:48:22.558220+0000","last_clean":"2023-11-09T03:48:22.558220+0000","last_became_active":"2023-11-09T03:48:22.556059+0000","last_became_peered":"2023-11-09T03:48:22.556059+0000","last_unstale":"2023-11-09T03:48:22.558220+0000","last_undegraded":"2023-11-09T03:48:22.558220+0000","last_fullsized":"2023-11-09T03:48:22.558220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:51:17.414844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.544682+0000","last_change":"2023-11-09T03:48:01.450581+0000","last_active":"2023-11-09T03:48:21.544682+0000","last_peered":"2023-11-09T03:48:21.544682+0000","last_clean":"2023-11-09T03:48:21.544682+0000","last_became_active":"2023-11-09T03:48:01.450280+0000","last_became_peered":"2023-11-09T03:48:01.450280+0000","last_unstale":"2023-11-09T03:48:21.544682+0000","last_undegraded":"2023-11-09T03:48:21.544682+0000","last_fullsized":"2023-11-09T03:48:21.544682+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:18.995634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.546874+0000","last_change":"2023-11-09T03:48:02.440934+0000","last_active":"2023-11-09T03:48:21.546874+0000","last_peered":"2023-11-09T03:48:21.546874+0000","last_clean":"2023-11-09T03:48:21.546874+0000","last_became_active":"2023-11-09T03:48:00.444394+0000","last_became_peered":"2023-11-09T03:48:00.444394+0000","last_unstale":"2023-11-09T03:48:21.546874+0000","last_undegraded":"2023-11-09T03:48:21.546874+0000","last_fullsized":"2023-11-09T03:48:21.546874+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:15:59.214955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063908300000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542547+0000","last_change":"2023-11-09T03:47:02.618856+0000","last_active":"2023-11-09T03:48:21.542547+0000","last_peered":"2023-11-09T03:48:21.542547+0000","last_clean":"2023-11-09T03:48:21.542547+0000","last_became_active":"2023-11-09T03:47:01.590430+0000","last_became_peered":"2023-11-09T03:47:01.590430+0000","last_unstale":"2023-11-09T03:48:21.542547+0000","last_undegraded":"2023-11-09T03:48:21.542547+0000","last_fullsized":"2023-11-09T03:48:21.542547+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:57:18.470305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055030200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550790+0000","last_change":"2023-11-09T03:48:22.550790+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550790+0000","last_undegraded":"2023-11-09T03:48:22.550790+0000","last_fullsized":"2023-11-09T03:48:22.550790+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.563239+0000","last_change":"2023-11-09T03:48:22.559687+0000","last_active":"2023-11-09T03:48:22.563239+0000","last_peered":"2023-11-09T03:48:22.563239+0000","last_clean":"2023-11-09T03:48:22.563239+0000","last_became_active":"2023-11-09T03:48:22.559404+0000","last_became_peered":"2023-11-09T03:48:22.559404+0000","last_unstale":"2023-11-09T03:48:22.563239+0000","last_undegraded":"2023-11-09T03:48:22.563239+0000","last_fullsized":"2023-11-09T03:48:22.563239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:39:53.462957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545586+0000","last_change":"2023-11-09T03:48:01.453600+0000","last_active":"2023-11-09T03:48:21.545586+0000","last_peered":"2023-11-09T03:48:21.545586+0000","last_clean":"2023-11-09T03:48:21.545586+0000","last_became_active":"2023-11-09T03:48:01.453330+0000","last_became_peered":"2023-11-09T03:48:01.453330+0000","last_unstale":"2023-11-09T03:48:21.545586+0000","last_undegraded":"2023-11-09T03:48:21.545586+0000","last_fullsized":"2023-11-09T03:48:21.545586+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:08:47.931939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539485+0000","last_change":"2023-11-09T03:48:03.328210+0000","last_active":"2023-11-09T03:48:21.539485+0000","last_peered":"2023-11-09T03:48:21.539485+0000","last_clean":"2023-11-09T03:48:21.539485+0000","last_became_active":"2023-11-09T03:48:00.450002+0000","last_became_peered":"2023-11-09T03:48:00.450002+0000","last_unstale":"2023-11-09T03:48:21.539485+0000","last_undegraded":"2023-11-09T03:48:21.539485+0000","last_fullsized":"2023-11-09T03:48:21.539485+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:50.391187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036521900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542921+0000","last_change":"2023-11-09T03:47:02.619144+0000","last_active":"2023-11-09T03:48:21.542921+0000","last_peered":"2023-11-09T03:48:21.542921+0000","last_clean":"2023-11-09T03:48:21.542921+0000","last_became_active":"2023-11-09T03:47:01.590672+0000","last_became_peered":"2023-11-09T03:47:01.590672+0000","last_unstale":"2023-11-09T03:48:21.542921+0000","last_undegraded":"2023-11-09T03:48:21.542921+0000","last_fullsized":"2023-11-09T03:48:21.542921+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:03.025119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060997500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.547130+0000","last_change":"2023-11-09T03:48:22.547130+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.547130+0000","last_undegraded":"2023-11-09T03:48:22.547130+0000","last_fullsized":"2023-11-09T03:48:22.547130+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.571874+0000","last_change":"2023-11-09T03:48:22.566419+0000","last_active":"2023-11-09T03:48:22.571874+0000","last_peered":"2023-11-09T03:48:22.571874+0000","last_clean":"2023-11-09T03:48:22.571874+0000","last_became_active":"2023-11-09T03:48:22.566144+0000","last_became_peered":"2023-11-09T03:48:22.566144+0000","last_unstale":"2023-11-09T03:48:22.571874+0000","last_undegraded":"2023-11-09T03:48:22.571874+0000","last_fullsized":"2023-11-09T03:48:22.571874+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:43:01.915119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541019+0000","last_change":"2023-11-09T03:48:01.447278+0000","last_active":"2023-11-09T03:48:21.541019+0000","last_peered":"2023-11-09T03:48:21.541019+0000","last_clean":"2023-11-09T03:48:21.541019+0000","last_became_active":"2023-11-09T03:48:01.446991+0000","last_became_peered":"2023-11-09T03:48:01.446991+0000","last_unstale":"2023-11-09T03:48:21.541019+0000","last_undegraded":"2023-11-09T03:48:21.541019+0000","last_fullsized":"2023-11-09T03:48:21.541019+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:27:49.190426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539429+0000","last_change":"2023-11-09T03:48:03.328278+0000","last_active":"2023-11-09T03:48:21.539429+0000","last_peered":"2023-11-09T03:48:21.539429+0000","last_clean":"2023-11-09T03:48:21.539429+0000","last_became_active":"2023-11-09T03:48:00.443213+0000","last_became_peered":"2023-11-09T03:48:00.443213+0000","last_unstale":"2023-11-09T03:48:21.539429+0000","last_undegraded":"2023-11-09T03:48:21.539429+0000","last_fullsized":"2023-11-09T03:48:21.539429+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:20:59.702297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030886100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543013+0000","last_change":"2023-11-09T03:47:02.619079+0000","last_active":"2023-11-09T03:48:21.543013+0000","last_peered":"2023-11-09T03:48:21.543013+0000","last_clean":"2023-11-09T03:48:21.543013+0000","last_became_active":"2023-11-09T03:47:01.591421+0000","last_became_peered":"2023-11-09T03:47:01.591421+0000","last_unstale":"2023-11-09T03:48:21.543013+0000","last_undegraded":"2023-11-09T03:48:21.543013+0000","last_fullsized":"2023-11-09T03:48:21.543013+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:18:58.230606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068736399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.547470+0000","last_change":"2023-11-09T03:48:22.547470+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.547470+0000","last_undegraded":"2023-11-09T03:48:22.547470+0000","last_fullsized":"2023-11-09T03:48:22.547470+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.574695+0000","last_change":"2023-11-09T03:48:22.569861+0000","last_active":"2023-11-09T03:48:22.574695+0000","last_peered":"2023-11-09T03:48:22.574695+0000","last_clean":"2023-11-09T03:48:22.574695+0000","last_became_active":"2023-11-09T03:48:22.569611+0000","last_became_peered":"2023-11-09T03:48:22.569611+0000","last_unstale":"2023-11-09T03:48:22.574695+0000","last_undegraded":"2023-11-09T03:48:22.574695+0000","last_fullsized":"2023-11-09T03:48:22.574695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:36:57.293292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541146+0000","last_change":"2023-11-09T03:48:01.449069+0000","last_active":"2023-11-09T03:48:21.541146+0000","last_peered":"2023-11-09T03:48:21.541146+0000","last_clean":"2023-11-09T03:48:21.541146+0000","last_became_active":"2023-11-09T03:48:01.448810+0000","last_became_peered":"2023-11-09T03:48:01.448810+0000","last_unstale":"2023-11-09T03:48:21.541146+0000","last_undegraded":"2023-11-09T03:48:21.541146+0000","last_fullsized":"2023-11-09T03:48:21.541146+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:56:51.158572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542867+0000","last_change":"2023-11-09T03:48:02.612067+0000","last_active":"2023-11-09T03:48:21.542867+0000","last_peered":"2023-11-09T03:48:21.542867+0000","last_clean":"2023-11-09T03:48:21.542867+0000","last_became_active":"2023-11-09T03:48:00.443296+0000","last_became_peered":"2023-11-09T03:48:00.443296+0000","last_unstale":"2023-11-09T03:48:21.542867+0000","last_undegraded":"2023-11-09T03:48:21.542867+0000","last_fullsized":"2023-11-09T03:48:21.542867+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:27:48.184420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010423330000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547943+0000","last_change":"2023-11-09T03:47:03.148529+0000","last_active":"2023-11-09T03:48:21.547943+0000","last_peered":"2023-11-09T03:48:21.547943+0000","last_clean":"2023-11-09T03:48:21.547943+0000","last_became_active":"2023-11-09T03:47:01.584148+0000","last_became_peered":"2023-11-09T03:47:01.584148+0000","last_unstale":"2023-11-09T03:48:21.547943+0000","last_undegraded":"2023-11-09T03:48:21.547943+0000","last_fullsized":"2023-11-09T03:48:21.547943+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:40:00.237248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061475600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550722+0000","last_change":"2023-11-09T03:48:22.550722+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550722+0000","last_undegraded":"2023-11-09T03:48:22.550722+0000","last_fullsized":"2023-11-09T03:48:22.550722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.563077+0000","last_change":"2023-11-09T03:48:22.558072+0000","last_active":"2023-11-09T03:48:22.563077+0000","last_peered":"2023-11-09T03:48:22.563077+0000","last_clean":"2023-11-09T03:48:22.563077+0000","last_became_active":"2023-11-09T03:48:22.557800+0000","last_became_peered":"2023-11-09T03:48:22.557800+0000","last_unstale":"2023-11-09T03:48:22.563077+0000","last_undegraded":"2023-11-09T03:48:22.563077+0000","last_fullsized":"2023-11-09T03:48:22.563077+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:00:20.811244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542948+0000","last_change":"2023-11-09T03:48:01.451750+0000","last_active":"2023-11-09T03:48:21.542948+0000","last_peered":"2023-11-09T03:48:21.542948+0000","last_clean":"2023-11-09T03:48:21.542948+0000","last_became_active":"2023-11-09T03:48:01.451393+0000","last_became_peered":"2023-11-09T03:48:01.451393+0000","last_unstale":"2023-11-09T03:48:21.542948+0000","last_undegraded":"2023-11-09T03:48:21.542948+0000","last_fullsized":"2023-11-09T03:48:21.542948+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:56:51.158572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542743+0000","last_change":"2023-11-09T03:48:02.611721+0000","last_active":"2023-11-09T03:48:21.542743+0000","last_peered":"2023-11-09T03:48:21.542743+0000","last_clean":"2023-11-09T03:48:21.542743+0000","last_became_active":"2023-11-09T03:48:00.439557+0000","last_became_peered":"2023-11-09T03:48:00.439557+0000","last_unstale":"2023-11-09T03:48:21.542743+0000","last_undegraded":"2023-11-09T03:48:21.542743+0000","last_fullsized":"2023-11-09T03:48:21.542743+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:35:53.864627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001037187,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543088+0000","last_change":"2023-11-09T03:47:03.148217+0000","last_active":"2023-11-09T03:48:21.543088+0000","last_peered":"2023-11-09T03:48:21.543088+0000","last_clean":"2023-11-09T03:48:21.543088+0000","last_became_active":"2023-11-09T03:47:01.593424+0000","last_became_peered":"2023-11-09T03:47:01.593424+0000","last_unstale":"2023-11-09T03:48:21.543088+0000","last_undegraded":"2023-11-09T03:48:21.543088+0000","last_fullsized":"2023-11-09T03:48:21.543088+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:19:32.693691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001769865,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551131+0000","last_change":"2023-11-09T03:48:22.551131+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551131+0000","last_undegraded":"2023-11-09T03:48:22.551131+0000","last_fullsized":"2023-11-09T03:48:22.551131+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.569383+0000","last_change":"2023-11-09T03:48:22.566073+0000","last_active":"2023-11-09T03:48:22.569383+0000","last_peered":"2023-11-09T03:48:22.569383+0000","last_clean":"2023-11-09T03:48:22.569383+0000","last_became_active":"2023-11-09T03:48:22.564106+0000","last_became_peered":"2023-11-09T03:48:22.564106+0000","last_unstale":"2023-11-09T03:48:22.569383+0000","last_undegraded":"2023-11-09T03:48:22.569383+0000","last_fullsized":"2023-11-09T03:48:22.569383+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:05:06.931697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547292+0000","last_change":"2023-11-09T03:48:01.447617+0000","last_active":"2023-11-09T03:48:21.547292+0000","last_peered":"2023-11-09T03:48:21.547292+0000","last_clean":"2023-11-09T03:48:21.547292+0000","last_became_active":"2023-11-09T03:48:01.447329+0000","last_became_peered":"2023-11-09T03:48:01.447329+0000","last_unstale":"2023-11-09T03:48:21.547292+0000","last_undegraded":"2023-11-09T03:48:21.547292+0000","last_fullsized":"2023-11-09T03:48:21.547292+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:54:52.433604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540889+0000","last_change":"2023-11-09T03:48:02.442427+0000","last_active":"2023-11-09T03:48:21.540889+0000","last_peered":"2023-11-09T03:48:21.540889+0000","last_clean":"2023-11-09T03:48:21.540889+0000","last_became_active":"2023-11-09T03:48:00.444856+0000","last_became_peered":"2023-11-09T03:48:00.444856+0000","last_unstale":"2023-11-09T03:48:21.540889+0000","last_undegraded":"2023-11-09T03:48:21.540889+0000","last_fullsized":"2023-11-09T03:48:21.540889+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:53:42.763434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011927730000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.546971+0000","last_change":"2023-11-09T03:47:03.146470+0000","last_active":"2023-11-09T03:48:21.546971+0000","last_peered":"2023-11-09T03:48:21.546971+0000","last_clean":"2023-11-09T03:48:21.546971+0000","last_became_active":"2023-11-09T03:47:01.587304+0000","last_became_peered":"2023-11-09T03:47:01.587304+0000","last_unstale":"2023-11-09T03:48:21.546971+0000","last_undegraded":"2023-11-09T03:48:21.546971+0000","last_fullsized":"2023-11-09T03:48:21.546971+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:58:23.339287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00110395,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.548843+0000","last_change":"2023-11-09T03:48:22.548843+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.548843+0000","last_undegraded":"2023-11-09T03:48:22.548843+0000","last_fullsized":"2023-11-09T03:48:22.548843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.564450+0000","last_change":"2023-11-09T03:48:22.561952+0000","last_active":"2023-11-09T03:48:22.564450+0000","last_peered":"2023-11-09T03:48:22.564450+0000","last_clean":"2023-11-09T03:48:22.564450+0000","last_became_active":"2023-11-09T03:48:22.561647+0000","last_became_peered":"2023-11-09T03:48:22.561647+0000","last_unstale":"2023-11-09T03:48:22.564450+0000","last_undegraded":"2023-11-09T03:48:22.564450+0000","last_fullsized":"2023-11-09T03:48:22.564450+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:52.879173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545482+0000","last_change":"2023-11-09T03:48:01.449406+0000","last_active":"2023-11-09T03:48:21.545482+0000","last_peered":"2023-11-09T03:48:21.545482+0000","last_clean":"2023-11-09T03:48:21.545482+0000","last_became_active":"2023-11-09T03:48:01.449130+0000","last_became_peered":"2023-11-09T03:48:01.449130+0000","last_unstale":"2023-11-09T03:48:21.545482+0000","last_undegraded":"2023-11-09T03:48:21.545482+0000","last_fullsized":"2023-11-09T03:48:21.545482+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:20:36.303940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547362+0000","last_change":"2023-11-09T03:48:02.441801+0000","last_active":"2023-11-09T03:48:21.547362+0000","last_peered":"2023-11-09T03:48:21.547362+0000","last_clean":"2023-11-09T03:48:21.547362+0000","last_became_active":"2023-11-09T03:48:00.444975+0000","last_became_peered":"2023-11-09T03:48:00.444975+0000","last_unstale":"2023-11-09T03:48:21.547362+0000","last_undegraded":"2023-11-09T03:48:21.547362+0000","last_fullsized":"2023-11-09T03:48:21.547362+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:39:31.347349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061165099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542743+0000","last_change":"2023-11-09T03:47:03.147367+0000","last_active":"2023-11-09T03:48:21.542743+0000","last_peered":"2023-11-09T03:48:21.542743+0000","last_clean":"2023-11-09T03:48:21.542743+0000","last_became_active":"2023-11-09T03:47:01.588359+0000","last_became_peered":"2023-11-09T03:47:01.588359+0000","last_unstale":"2023-11-09T03:48:21.542743+0000","last_undegraded":"2023-11-09T03:48:21.542743+0000","last_fullsized":"2023-11-09T03:48:21.542743+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:21:34.303068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014106660000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.546731+0000","last_change":"2023-11-09T03:48:22.546731+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.546731+0000","last_undegraded":"2023-11-09T03:48:22.546731+0000","last_fullsized":"2023-11-09T03:48:22.546731+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.563046+0000","last_change":"2023-11-09T03:48:22.556245+0000","last_active":"2023-11-09T03:48:22.563046+0000","last_peered":"2023-11-09T03:48:22.563046+0000","last_clean":"2023-11-09T03:48:22.563046+0000","last_became_active":"2023-11-09T03:48:22.555974+0000","last_became_peered":"2023-11-09T03:48:22.555974+0000","last_unstale":"2023-11-09T03:48:22.563046+0000","last_undegraded":"2023-11-09T03:48:22.563046+0000","last_fullsized":"2023-11-09T03:48:22.563046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:55:13.543207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542383+0000","last_change":"2023-11-09T03:48:01.451866+0000","last_active":"2023-11-09T03:48:21.542383+0000","last_peered":"2023-11-09T03:48:21.542383+0000","last_clean":"2023-11-09T03:48:21.542383+0000","last_became_active":"2023-11-09T03:48:01.451350+0000","last_became_peered":"2023-11-09T03:48:01.451350+0000","last_unstale":"2023-11-09T03:48:21.542383+0000","last_undegraded":"2023-11-09T03:48:21.542383+0000","last_fullsized":"2023-11-09T03:48:21.542383+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:36.735029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540171+0000","last_change":"2023-11-09T03:48:02.441648+0000","last_active":"2023-11-09T03:48:21.540171+0000","last_peered":"2023-11-09T03:48:21.540171+0000","last_clean":"2023-11-09T03:48:21.540171+0000","last_became_active":"2023-11-09T03:48:00.448541+0000","last_became_peered":"2023-11-09T03:48:00.448541+0000","last_unstale":"2023-11-09T03:48:21.540171+0000","last_undegraded":"2023-11-09T03:48:21.540171+0000","last_fullsized":"2023-11-09T03:48:21.540171+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:35.729023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539334+0000","last_change":"2023-11-09T03:47:02.581682+0000","last_active":"2023-11-09T03:48:21.539334+0000","last_peered":"2023-11-09T03:48:21.539334+0000","last_clean":"2023-11-09T03:48:21.539334+0000","last_became_active":"2023-11-09T03:47:01.593051+0000","last_became_peered":"2023-11-09T03:47:01.593051+0000","last_unstale":"2023-11-09T03:48:21.539334+0000","last_undegraded":"2023-11-09T03:48:21.539334+0000","last_fullsized":"2023-11-09T03:48:21.539334+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047277799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549020+0000","last_change":"2023-11-09T03:48:22.549020+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549020+0000","last_undegraded":"2023-11-09T03:48:22.549020+0000","last_fullsized":"2023-11-09T03:48:22.549020+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.569210+0000","last_change":"2023-11-09T03:48:22.563908+0000","last_active":"2023-11-09T03:48:22.569210+0000","last_peered":"2023-11-09T03:48:22.569210+0000","last_clean":"2023-11-09T03:48:22.569210+0000","last_became_active":"2023-11-09T03:48:22.563651+0000","last_became_peered":"2023-11-09T03:48:22.563651+0000","last_unstale":"2023-11-09T03:48:22.569210+0000","last_undegraded":"2023-11-09T03:48:22.569210+0000","last_fullsized":"2023-11-09T03:48:22.569210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:52.879173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.544927+0000","last_change":"2023-11-09T03:48:01.455784+0000","last_active":"2023-11-09T03:48:21.544927+0000","last_peered":"2023-11-09T03:48:21.544927+0000","last_clean":"2023-11-09T03:48:21.544927+0000","last_became_active":"2023-11-09T03:48:01.455532+0000","last_became_peered":"2023-11-09T03:48:01.455532+0000","last_unstale":"2023-11-09T03:48:21.544927+0000","last_undegraded":"2023-11-09T03:48:21.544927+0000","last_fullsized":"2023-11-09T03:48:21.544927+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:21:23.156761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542535+0000","last_change":"2023-11-09T03:48:02.611298+0000","last_active":"2023-11-09T03:48:21.542535+0000","last_peered":"2023-11-09T03:48:21.542535+0000","last_clean":"2023-11-09T03:48:21.542535+0000","last_became_active":"2023-11-09T03:48:00.441261+0000","last_became_peered":"2023-11-09T03:48:00.441261+0000","last_unstale":"2023-11-09T03:48:21.542535+0000","last_undegraded":"2023-11-09T03:48:21.542535+0000","last_fullsized":"2023-11-09T03:48:21.542535+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:50:55.299235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081460199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542200+0000","last_change":"2023-11-09T03:47:03.146309+0000","last_active":"2023-11-09T03:48:21.542200+0000","last_peered":"2023-11-09T03:48:21.542200+0000","last_clean":"2023-11-09T03:48:21.542200+0000","last_became_active":"2023-11-09T03:47:01.587560+0000","last_became_peered":"2023-11-09T03:47:01.587560+0000","last_unstale":"2023-11-09T03:48:21.542200+0000","last_undegraded":"2023-11-09T03:48:21.542200+0000","last_fullsized":"2023-11-09T03:48:21.542200+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:27:55.271945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070988400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551507+0000","last_change":"2023-11-09T03:48:22.551507+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551507+0000","last_undegraded":"2023-11-09T03:48:22.551507+0000","last_fullsized":"2023-11-09T03:48:22.551507+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.563543+0000","last_change":"2023-11-09T03:48:22.561557+0000","last_active":"2023-11-09T03:48:22.563543+0000","last_peered":"2023-11-09T03:48:22.563543+0000","last_clean":"2023-11-09T03:48:22.563543+0000","last_became_active":"2023-11-09T03:48:22.561249+0000","last_became_peered":"2023-11-09T03:48:22.561249+0000","last_unstale":"2023-11-09T03:48:22.563543+0000","last_undegraded":"2023-11-09T03:48:22.563543+0000","last_fullsized":"2023-11-09T03:48:22.563543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:15:26.050548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543219+0000","last_change":"2023-11-09T03:48:01.452477+0000","last_active":"2023-11-09T03:48:21.543219+0000","last_peered":"2023-11-09T03:48:21.543219+0000","last_clean":"2023-11-09T03:48:21.543219+0000","last_became_active":"2023-11-09T03:48:01.452186+0000","last_became_peered":"2023-11-09T03:48:01.452186+0000","last_unstale":"2023-11-09T03:48:21.543219+0000","last_undegraded":"2023-11-09T03:48:21.543219+0000","last_fullsized":"2023-11-09T03:48:21.543219+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:32:21.825778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547861+0000","last_change":"2023-11-09T03:48:02.442101+0000","last_active":"2023-11-09T03:48:21.547861+0000","last_peered":"2023-11-09T03:48:21.547861+0000","last_clean":"2023-11-09T03:48:21.547861+0000","last_became_active":"2023-11-09T03:48:00.449354+0000","last_became_peered":"2023-11-09T03:48:00.449354+0000","last_unstale":"2023-11-09T03:48:21.547861+0000","last_undegraded":"2023-11-09T03:48:21.547861+0000","last_fullsized":"2023-11-09T03:48:21.547861+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:06:32.906582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093455000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":47,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545438+0000","last_change":"2023-11-09T03:47:02.725125+0000","last_active":"2023-11-09T03:48:21.545438+0000","last_peered":"2023-11-09T03:48:21.545438+0000","last_clean":"2023-11-09T03:48:21.545438+0000","last_became_active":"2023-11-09T03:47:01.591521+0000","last_became_peered":"2023-11-09T03:47:01.591521+0000","last_unstale":"2023-11-09T03:48:21.545438+0000","last_undegraded":"2023-11-09T03:48:21.545438+0000","last_fullsized":"2023-11-09T03:48:21.545438+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:52:40.076542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000758675,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550819+0000","last_change":"2023-11-09T03:48:22.550819+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550819+0000","last_undegraded":"2023-11-09T03:48:22.550819+0000","last_fullsized":"2023-11-09T03:48:22.550819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.565704+0000","last_change":"2023-11-09T03:48:22.561189+0000","last_active":"2023-11-09T03:48:22.565704+0000","last_peered":"2023-11-09T03:48:22.565704+0000","last_clean":"2023-11-09T03:48:22.565704+0000","last_became_active":"2023-11-09T03:48:22.560938+0000","last_became_peered":"2023-11-09T03:48:22.560938+0000","last_unstale":"2023-11-09T03:48:22.565704+0000","last_undegraded":"2023-11-09T03:48:22.565704+0000","last_fullsized":"2023-11-09T03:48:22.565704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:05:30.764921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545418+0000","last_change":"2023-11-09T03:48:01.448129+0000","last_active":"2023-11-09T03:48:21.545418+0000","last_peered":"2023-11-09T03:48:21.545418+0000","last_clean":"2023-11-09T03:48:21.545418+0000","last_became_active":"2023-11-09T03:48:01.447844+0000","last_became_peered":"2023-11-09T03:48:01.447844+0000","last_unstale":"2023-11-09T03:48:21.545418+0000","last_undegraded":"2023-11-09T03:48:21.545418+0000","last_fullsized":"2023-11-09T03:48:21.545418+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:54:52.433604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543211+0000","last_change":"2023-11-09T03:48:02.612450+0000","last_active":"2023-11-09T03:48:21.543211+0000","last_peered":"2023-11-09T03:48:21.543211+0000","last_clean":"2023-11-09T03:48:21.543211+0000","last_became_active":"2023-11-09T03:48:00.441855+0000","last_became_peered":"2023-11-09T03:48:00.441855+0000","last_unstale":"2023-11-09T03:48:21.543211+0000","last_undegraded":"2023-11-09T03:48:21.543211+0000","last_fullsized":"2023-11-09T03:48:21.543211+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:53:42.763434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095368899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547816+0000","last_change":"2023-11-09T03:47:03.147289+0000","last_active":"2023-11-09T03:48:21.547816+0000","last_peered":"2023-11-09T03:48:21.547816+0000","last_clean":"2023-11-09T03:48:21.547816+0000","last_became_active":"2023-11-09T03:47:01.587784+0000","last_became_peered":"2023-11-09T03:47:01.587784+0000","last_unstale":"2023-11-09T03:48:21.547816+0000","last_undegraded":"2023-11-09T03:48:21.547816+0000","last_fullsized":"2023-11-09T03:48:21.547816+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:11:23.824112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038781299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.552840+0000","last_change":"2023-11-09T03:48:22.552840+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.552840+0000","last_undegraded":"2023-11-09T03:48:22.552840+0000","last_fullsized":"2023-11-09T03:48:22.552840+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.568633+0000","last_change":"2023-11-09T03:48:22.564955+0000","last_active":"2023-11-09T03:48:22.568633+0000","last_peered":"2023-11-09T03:48:22.568633+0000","last_clean":"2023-11-09T03:48:22.568633+0000","last_became_active":"2023-11-09T03:48:22.564677+0000","last_became_peered":"2023-11-09T03:48:22.564677+0000","last_unstale":"2023-11-09T03:48:22.568633+0000","last_undegraded":"2023-11-09T03:48:22.568633+0000","last_fullsized":"2023-11-09T03:48:22.568633+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:16:21.330564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541065+0000","last_change":"2023-11-09T03:48:01.449617+0000","last_active":"2023-11-09T03:48:21.541065+0000","last_peered":"2023-11-09T03:48:21.541065+0000","last_clean":"2023-11-09T03:48:21.541065+0000","last_became_active":"2023-11-09T03:48:01.449328+0000","last_became_peered":"2023-11-09T03:48:01.449328+0000","last_unstale":"2023-11-09T03:48:21.541065+0000","last_undegraded":"2023-11-09T03:48:21.541065+0000","last_fullsized":"2023-11-09T03:48:21.541065+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:51:18.760059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547661+0000","last_change":"2023-11-09T03:48:02.442215+0000","last_active":"2023-11-09T03:48:21.547661+0000","last_peered":"2023-11-09T03:48:21.547661+0000","last_clean":"2023-11-09T03:48:21.547661+0000","last_became_active":"2023-11-09T03:48:00.444720+0000","last_became_peered":"2023-11-09T03:48:00.444720+0000","last_unstale":"2023-11-09T03:48:21.547661+0000","last_undegraded":"2023-11-09T03:48:21.547661+0000","last_fullsized":"2023-11-09T03:48:21.547661+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:55:55.735668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001407176,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543390+0000","last_change":"2023-11-09T03:47:02.619346+0000","last_active":"2023-11-09T03:48:21.543390+0000","last_peered":"2023-11-09T03:48:21.543390+0000","last_clean":"2023-11-09T03:48:21.543390+0000","last_became_active":"2023-11-09T03:47:01.590389+0000","last_became_peered":"2023-11-09T03:47:01.590389+0000","last_unstale":"2023-11-09T03:48:21.543390+0000","last_undegraded":"2023-11-09T03:48:21.543390+0000","last_fullsized":"2023-11-09T03:48:21.543390+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000384072,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551555+0000","last_change":"2023-11-09T03:48:22.551555+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551555+0000","last_undegraded":"2023-11-09T03:48:22.551555+0000","last_fullsized":"2023-11-09T03:48:22.551555+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.569120+0000","last_change":"2023-11-09T03:48:22.562601+0000","last_active":"2023-11-09T03:48:22.569120+0000","last_peered":"2023-11-09T03:48:22.569120+0000","last_clean":"2023-11-09T03:48:22.569120+0000","last_became_active":"2023-11-09T03:48:22.562326+0000","last_became_peered":"2023-11-09T03:48:22.562326+0000","last_unstale":"2023-11-09T03:48:22.569120+0000","last_undegraded":"2023-11-09T03:48:22.569120+0000","last_fullsized":"2023-11-09T03:48:22.569120+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:28:06.557471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545053+0000","last_change":"2023-11-09T03:48:01.450018+0000","last_active":"2023-11-09T03:48:21.545053+0000","last_peered":"2023-11-09T03:48:21.545053+0000","last_clean":"2023-11-09T03:48:21.545053+0000","last_became_active":"2023-11-09T03:48:01.449749+0000","last_became_peered":"2023-11-09T03:48:01.449749+0000","last_unstale":"2023-11-09T03:48:21.545053+0000","last_undegraded":"2023-11-09T03:48:21.545053+0000","last_fullsized":"2023-11-09T03:48:21.545053+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:21:23.156761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542795+0000","last_change":"2023-11-09T03:48:02.524216+0000","last_active":"2023-11-09T03:48:21.542795+0000","last_peered":"2023-11-09T03:48:21.542795+0000","last_clean":"2023-11-09T03:48:21.542795+0000","last_became_active":"2023-11-09T03:48:00.446972+0000","last_became_peered":"2023-11-09T03:48:00.446972+0000","last_unstale":"2023-11-09T03:48:21.542795+0000","last_undegraded":"2023-11-09T03:48:21.542795+0000","last_fullsized":"2023-11-09T03:48:21.542795+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:54:07.987773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000825255,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":50,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540373+0000","last_change":"2023-11-09T03:47:02.582393+0000","last_active":"2023-11-09T03:48:21.540373+0000","last_peered":"2023-11-09T03:48:21.540373+0000","last_clean":"2023-11-09T03:48:21.540373+0000","last_became_active":"2023-11-09T03:47:01.591752+0000","last_became_peered":"2023-11-09T03:47:01.591752+0000","last_unstale":"2023-11-09T03:48:21.540373+0000","last_undegraded":"2023-11-09T03:48:21.540373+0000","last_fullsized":"2023-11-09T03:48:21.540373+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:03.025119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067021599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549641+0000","last_change":"2023-11-09T03:48:22.549641+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549641+0000","last_undegraded":"2023-11-09T03:48:22.549641+0000","last_fullsized":"2023-11-09T03:48:22.549641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.566064+0000","last_change":"2023-11-09T03:48:22.562752+0000","last_active":"2023-11-09T03:48:22.566064+0000","last_peered":"2023-11-09T03:48:22.566064+0000","last_clean":"2023-11-09T03:48:22.566064+0000","last_became_active":"2023-11-09T03:48:22.562474+0000","last_became_peered":"2023-11-09T03:48:22.562474+0000","last_unstale":"2023-11-09T03:48:22.566064+0000","last_undegraded":"2023-11-09T03:48:22.566064+0000","last_fullsized":"2023-11-09T03:48:22.566064+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:35:27.804836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542868+0000","last_change":"2023-11-09T03:48:01.451462+0000","last_active":"2023-11-09T03:48:21.542868+0000","last_peered":"2023-11-09T03:48:21.542868+0000","last_clean":"2023-11-09T03:48:21.542868+0000","last_became_active":"2023-11-09T03:48:01.451188+0000","last_became_peered":"2023-11-09T03:48:01.451188+0000","last_unstale":"2023-11-09T03:48:21.542868+0000","last_undegraded":"2023-11-09T03:48:21.542868+0000","last_fullsized":"2023-11-09T03:48:21.542868+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:28.631801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542221+0000","last_change":"2023-11-09T03:48:02.682290+0000","last_active":"2023-11-09T03:48:21.542221+0000","last_peered":"2023-11-09T03:48:21.542221+0000","last_clean":"2023-11-09T03:48:21.542221+0000","last_became_active":"2023-11-09T03:48:00.444246+0000","last_became_peered":"2023-11-09T03:48:00.444246+0000","last_unstale":"2023-11-09T03:48:21.542221+0000","last_undegraded":"2023-11-09T03:48:21.542221+0000","last_fullsized":"2023-11-09T03:48:21.542221+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:09.487058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000423359,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.544622+0000","last_change":"2023-11-09T03:47:03.028744+0000","last_active":"2023-11-09T03:48:21.544622+0000","last_peered":"2023-11-09T03:48:21.544622+0000","last_clean":"2023-11-09T03:48:21.544622+0000","last_became_active":"2023-11-09T03:47:01.593890+0000","last_became_peered":"2023-11-09T03:47:01.593890+0000","last_unstale":"2023-11-09T03:48:21.544622+0000","last_undegraded":"2023-11-09T03:48:21.544622+0000","last_fullsized":"2023-11-09T03:48:21.544622+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:05.754357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053008299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550087+0000","last_change":"2023-11-09T03:48:22.550087+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550087+0000","last_undegraded":"2023-11-09T03:48:22.550087+0000","last_fullsized":"2023-11-09T03:48:22.550087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.566105+0000","last_change":"2023-11-09T03:48:22.560645+0000","last_active":"2023-11-09T03:48:22.566105+0000","last_peered":"2023-11-09T03:48:22.566105+0000","last_clean":"2023-11-09T03:48:22.566105+0000","last_became_active":"2023-11-09T03:48:22.560328+0000","last_became_peered":"2023-11-09T03:48:22.560328+0000","last_unstale":"2023-11-09T03:48:22.566105+0000","last_undegraded":"2023-11-09T03:48:22.566105+0000","last_fullsized":"2023-11-09T03:48:22.566105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:59.646501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.544881+0000","last_change":"2023-11-09T03:48:01.450144+0000","last_active":"2023-11-09T03:48:21.544881+0000","last_peered":"2023-11-09T03:48:21.544881+0000","last_clean":"2023-11-09T03:48:21.544881+0000","last_became_active":"2023-11-09T03:48:01.449859+0000","last_became_peered":"2023-11-09T03:48:01.449859+0000","last_unstale":"2023-11-09T03:48:21.544881+0000","last_undegraded":"2023-11-09T03:48:21.544881+0000","last_fullsized":"2023-11-09T03:48:21.544881+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:57:56.855869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542191+0000","last_change":"2023-11-09T03:48:02.681869+0000","last_active":"2023-11-09T03:48:21.542191+0000","last_peered":"2023-11-09T03:48:21.542191+0000","last_clean":"2023-11-09T03:48:21.542191+0000","last_became_active":"2023-11-09T03:48:00.444928+0000","last_became_peered":"2023-11-09T03:48:00.444928+0000","last_unstale":"2023-11-09T03:48:21.542191+0000","last_undegraded":"2023-11-09T03:48:21.542191+0000","last_fullsized":"2023-11-09T03:48:21.542191+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:59:58.695636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044822599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542292+0000","last_change":"2023-11-09T03:47:03.146783+0000","last_active":"2023-11-09T03:48:21.542292+0000","last_peered":"2023-11-09T03:48:21.542292+0000","last_clean":"2023-11-09T03:48:21.542292+0000","last_became_active":"2023-11-09T03:47:01.586923+0000","last_became_peered":"2023-11-09T03:47:01.586923+0000","last_unstale":"2023-11-09T03:48:21.542292+0000","last_undegraded":"2023-11-09T03:48:21.542292+0000","last_fullsized":"2023-11-09T03:48:21.542292+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:37:50.326442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060384700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549765+0000","last_change":"2023-11-09T03:48:22.549765+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549765+0000","last_undegraded":"2023-11-09T03:48:22.549765+0000","last_fullsized":"2023-11-09T03:48:22.549765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.571830+0000","last_change":"2023-11-09T03:48:22.565142+0000","last_active":"2023-11-09T03:48:22.571830+0000","last_peered":"2023-11-09T03:48:22.571830+0000","last_clean":"2023-11-09T03:48:22.571830+0000","last_became_active":"2023-11-09T03:48:22.564856+0000","last_became_peered":"2023-11-09T03:48:22.564856+0000","last_unstale":"2023-11-09T03:48:22.571830+0000","last_undegraded":"2023-11-09T03:48:22.571830+0000","last_fullsized":"2023-11-09T03:48:22.571830+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:25:04.111498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542575+0000","last_change":"2023-11-09T03:48:01.448383+0000","last_active":"2023-11-09T03:48:21.542575+0000","last_peered":"2023-11-09T03:48:21.542575+0000","last_clean":"2023-11-09T03:48:21.542575+0000","last_became_active":"2023-11-09T03:48:01.448050+0000","last_became_peered":"2023-11-09T03:48:01.448050+0000","last_unstale":"2023-11-09T03:48:21.542575+0000","last_undegraded":"2023-11-09T03:48:21.542575+0000","last_fullsized":"2023-11-09T03:48:21.542575+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:53:44.478182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541748+0000","last_change":"2023-11-09T03:48:02.523794+0000","last_active":"2023-11-09T03:48:21.541748+0000","last_peered":"2023-11-09T03:48:21.541748+0000","last_clean":"2023-11-09T03:48:21.541748+0000","last_became_active":"2023-11-09T03:48:00.440255+0000","last_became_peered":"2023-11-09T03:48:00.440255+0000","last_unstale":"2023-11-09T03:48:21.541748+0000","last_undegraded":"2023-11-09T03:48:21.541748+0000","last_fullsized":"2023-11-09T03:48:21.541748+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:58:02.392696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064572299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542252+0000","last_change":"2023-11-09T03:47:03.146580+0000","last_active":"2023-11-09T03:48:21.542252+0000","last_peered":"2023-11-09T03:48:21.542252+0000","last_clean":"2023-11-09T03:48:21.542252+0000","last_became_active":"2023-11-09T03:47:01.589201+0000","last_became_peered":"2023-11-09T03:47:01.589201+0000","last_unstale":"2023-11-09T03:48:21.542252+0000","last_undegraded":"2023-11-09T03:48:21.542252+0000","last_fullsized":"2023-11-09T03:48:21.542252+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:14.300421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049706999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551049+0000","last_change":"2023-11-09T03:48:22.551049+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551049+0000","last_undegraded":"2023-11-09T03:48:22.551049+0000","last_fullsized":"2023-11-09T03:48:22.551049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.566108+0000","last_change":"2023-11-09T03:48:22.562609+0000","last_active":"2023-11-09T03:48:22.566108+0000","last_peered":"2023-11-09T03:48:22.566108+0000","last_clean":"2023-11-09T03:48:22.566108+0000","last_became_active":"2023-11-09T03:48:22.562328+0000","last_became_peered":"2023-11-09T03:48:22.562328+0000","last_unstale":"2023-11-09T03:48:22.566108+0000","last_undegraded":"2023-11-09T03:48:22.566108+0000","last_fullsized":"2023-11-09T03:48:22.566108+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:57:25.589693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539239+0000","last_change":"2023-11-09T03:48:01.451284+0000","last_active":"2023-11-09T03:48:21.539239+0000","last_peered":"2023-11-09T03:48:21.539239+0000","last_clean":"2023-11-09T03:48:21.539239+0000","last_became_active":"2023-11-09T03:48:01.450954+0000","last_became_peered":"2023-11-09T03:48:01.450954+0000","last_unstale":"2023-11-09T03:48:21.539239+0000","last_undegraded":"2023-11-09T03:48:21.539239+0000","last_fullsized":"2023-11-09T03:48:21.539239+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:49:24.680020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545344+0000","last_change":"2023-11-09T03:48:02.733038+0000","last_active":"2023-11-09T03:48:21.545344+0000","last_peered":"2023-11-09T03:48:21.545344+0000","last_clean":"2023-11-09T03:48:21.545344+0000","last_became_active":"2023-11-09T03:48:00.446661+0000","last_became_peered":"2023-11-09T03:48:00.446661+0000","last_unstale":"2023-11-09T03:48:21.545344+0000","last_undegraded":"2023-11-09T03:48:21.545344+0000","last_fullsized":"2023-11-09T03:48:21.545344+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:05:08.649312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040102300000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547989+0000","last_change":"2023-11-09T03:47:03.147851+0000","last_active":"2023-11-09T03:48:21.547989+0000","last_peered":"2023-11-09T03:48:21.547989+0000","last_clean":"2023-11-09T03:48:21.547989+0000","last_became_active":"2023-11-09T03:47:01.589893+0000","last_became_peered":"2023-11-09T03:47:01.589893+0000","last_unstale":"2023-11-09T03:48:21.547989+0000","last_undegraded":"2023-11-09T03:48:21.547989+0000","last_fullsized":"2023-11-09T03:48:21.547989+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:08:17.183462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029151399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":215,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545358+0000","last_change":"2023-11-09T03:45:21.030926+0000","last_active":"2023-11-09T03:48:21.545358+0000","last_peered":"2023-11-09T03:48:21.545358+0000","last_clean":"2023-11-09T03:48:21.545358+0000","last_became_active":"2023-11-09T03:45:21.029801+0000","last_became_peered":"2023-11-09T03:45:21.029801+0000","last_unstale":"2023-11-09T03:48:21.545358+0000","last_undegraded":"2023-11-09T03:48:21.545358+0000","last_fullsized":"2023-11-09T03:48:21.545358+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_clean_scrub_stamp":"2023-11-09T03:45:19.768433+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:12:59.153713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1572864,"data_stored":1058844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.501,"15min":0.498},"min":{"1min":0.34899999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.41199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.59899999999999998}]},{"osd":1,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.33000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.38100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.57799999999999996}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.52300000000000002,"15min":0.51400000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.68700000000000006,"15min":0.70799999999999996},"min":{"1min":0.41299999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.80700000000000005,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.628}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.48699999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61599999999999999}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.441},"min":{"1min":0.34100000000000003,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.49199999999999999,"15min":0.47899999999999998},"min":{"1min":0.38700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.42799999999999999}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.47199999999999998,"15min":0.45600000000000002},"min":{"1min":0.37,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.51100000000000001,"15min":0.497},"min":{"1min":0.39600000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.54800000000000004}]},{"osd":6,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55100000000000005,"15min":0.54100000000000004},"min":{"1min":0.44400000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.496},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.434,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.48099999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":991232,"data_stored":476134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.499,"15min":0.501},"min":{"1min":0.35099999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63900000000000001,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.50600000000000001,"15min":0.495},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.51100000000000001}]},{"osd":1,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.375,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.58299999999999996}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.503,"15min":0.503},"min":{"1min":0.316,"5min":0.313,"15min":0.313},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.437},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.45700000000000002}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.436,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55500000000000005,"15min":0.55800000000000005},"min":{"1min":0.34300000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.42899999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.44400000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.56100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":978944,"data_stored":468476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.504,"15min":0.48899999999999999},"min":{"1min":0.36199999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.503},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.376,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.61299999999999999}]},{"osd":1,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88300000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.36599999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.84199999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.65900000000000003}]},{"osd":2,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.45800000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.55600000000000005},"min":{"1min":0.40300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.71099999999999997}]},{"osd":3,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57099999999999995,"15min":0.56499999999999995},"min":{"1min":0.39400000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55200000000000005,"15min":0.55100000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.76100000000000001}]},{"osd":4,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.49199999999999999,"15min":0.48399999999999999},"min":{"1min":0.32400000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.48999999999999999,"15min":0.48299999999999998},"min":{"1min":0.29699999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.57899999999999996}]},{"osd":6,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.50600000000000001,"15min":0.51000000000000001},"min":{"1min":0.33700000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.755,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.53300000000000003,"15min":0.51700000000000002},"min":{"1min":0.38300000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.64400000000000002}]},{"osd":7,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.34699999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.81200000000000006,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.61899999999999999,"15min":0.62},"min":{"1min":0.432,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84899999999999998,"5min":1.01,"15min":1.01},"last":0.79000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":978944,"data_stored":468476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69399999999999995,"5min":1.034,"15min":1.034},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.38800000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78300000000000003,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.34000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.74299999999999999,"5min":1.093,"15min":1.093},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54200000000000004,"15min":0.54100000000000004},"min":{"1min":0.36399999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71899999999999997,"5min":0.871,"15min":0.871},"last":0.54900000000000004}]},{"osd":2,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54100000000000004,"15min":0.55400000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.69199999999999995,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54800000000000004,"15min":0.54500000000000004},"min":{"1min":0.39700000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.51800000000000002}]},{"osd":3,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.433,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.70199999999999996,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.376,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72499999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.379}]},{"osd":5,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.33300000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.78700000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.318,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81100000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.35799999999999998}]},{"osd":6,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.75800000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57899999999999996,"15min":0.58399999999999996},"min":{"1min":0.39600000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.75900000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.39600000000000002}]},{"osd":7,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61599999999999999,"15min":0.629},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83499999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61099999999999999,"15min":0.61899999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79300000000000004,"5min":1.113,"15min":1.113},"last":0.45500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":978944,"data_stored":468476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.32100000000000001,"15min":0.32300000000000001},"min":{"1min":0.22,"5min":0.215,"15min":0.215},"max":{"1min":0.40200000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.313,"5min":0.32300000000000001,"15min":0.32400000000000001},"min":{"1min":0.23999999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.47799999999999998,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.23999999999999999}]},{"osd":1,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.32600000000000001,"15min":0.33000000000000002},"min":{"1min":0.23599999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.42899999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.36699999999999999,"15min":0.373},"min":{"1min":0.28100000000000003,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.47899999999999998,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.36599999999999999}]},{"osd":2,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.37,"15min":0.372},"min":{"1min":0.27200000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.439,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.312},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.379,"15min":0.38300000000000001},"min":{"1min":0.29799999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.45700000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.38400000000000001}]},{"osd":4,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58899999999999997,"15min":0.59499999999999997},"min":{"1min":0.42299999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79000000000000004,"5min":0.87,"15min":0.87},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.59699999999999998,"15min":0.61099999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.434}]},{"osd":5,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.60499999999999998,"15min":0.62},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86599999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60699999999999998,"15min":0.61899999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.82099999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.443}]},{"osd":6,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60999999999999999,"15min":0.623},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78200000000000003,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.45400000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.80300000000000005,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.55200000000000005}]},{"osd":7,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.61099999999999999,"15min":0.62},"min":{"1min":0.41099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.70899999999999996,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.621,"15min":0.63700000000000001},"min":{"1min":0.46100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81299999999999994,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.46100000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":991232,"data_stored":476134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.46000000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.33400000000000002,"15min":0.33100000000000002},"min":{"1min":0.247,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.29399999999999998}]},{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32500000000000001,"15min":0.32600000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.495,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.35899999999999999,"15min":0.36299999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.48599999999999999,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.36699999999999999}]},{"osd":3,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.36199999999999999,"15min":0.36399999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.45700000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.312},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40200000000000002,"15min":0.40400000000000003},"min":{"1min":0.30299999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.52300000000000002,"5min":0.754,"15min":0.754},"last":0.39900000000000002}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72299999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81299999999999994,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.46000000000000002}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.39800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.75700000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.59599999999999997,"15min":0.60099999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.753,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.498}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60399999999999998,"15min":0.60899999999999999},"min":{"1min":0.44600000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.76100000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.71199999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60699999999999998,"15min":0.61199999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.745,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.80000000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.47299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":978944,"data_stored":468476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.28699999999999998,"5min":0.31,"15min":0.314},"min":{"1min":0.20799999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.38900000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"last":0.309},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.315,"15min":0.311},"min":{"1min":0.24099999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.29299999999999998}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.311,"15min":0.309},"min":{"1min":0.253,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.378,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.34999999999999998}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36899999999999999,"15min":0.371},"min":{"1min":0.23899999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.47699999999999998,"5min":0.505,"15min":0.505},"last":0.379},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.37,"15min":0.371},"min":{"1min":0.30299999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.45400000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.41899999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.747,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.44400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.77600000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.58699999999999997}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56200000000000006,"15min":0.55700000000000005},"min":{"1min":0.46100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.44500000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.72599999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.47799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56699999999999995,"15min":0.56799999999999995},"min":{"1min":0.433,"5min":0.379,"15min":0.379},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.53300000000000003}]},{"osd":7,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.42899999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.69799999999999995,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.72899999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.54700000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1568768,"data_stored":1058225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.31900000000000001,"15min":0.32300000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.38200000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.29599999999999999,"5min":0.35499999999999998,"15min":0.36499999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.44500000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.27300000000000002}]},{"osd":2,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.35699999999999998,"15min":0.36399999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.44700000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.376},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.36699999999999999,"15min":0.371},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.47699999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.316}]},{"osd":3,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.38500000000000001,"15min":0.39100000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.46400000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.41499999999999998,"15min":0.42099999999999999},"min":{"1min":0.309,"5min":0.307,"15min":0.307},"max":{"1min":0.48999999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58899999999999997,"15min":0.59299999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60599999999999998,"15min":0.60899999999999999},"min":{"1min":0.436,"5min":0.318,"15min":0.318},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.60299999999999998}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.36199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76800000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.629,"15min":0.63600000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78900000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.64100000000000001,"15min":0.64700000000000002},"min":{"1min":0.42499999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81000000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.627,"15min":0.63200000000000001},"min":{"1min":0.41699999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.44400000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.42999999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.71799999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.55600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-09T03:48:26.857 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-09T03:48:26.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.988+0000 7f076c0e5640 1 Processor -- start 2023-11-09T03:48:26.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.989+0000 7f076c0e5640 1 -- start start 2023-11-09T03:48:26.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.989+0000 7f076c0e5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f07640b0ae0 0x7f07640ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:26.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.989+0000 7f076c0e5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f076406db20 0x7f076415d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:26.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.989+0000 7f076c0e5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07640b4110 con 0x7f0764159c70 2023-11-09T03:48:26.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.989+0000 7f076c0e5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07640b49b0 con 0x7f07640b0ae0 2023-11-09T03:48:26.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.989+0000 7f076c0e5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07640b43f0 con 0x7f076406db20 2023-11-09T03:48:26.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.989+0000 7f0769e81640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0764159c70 0x7f07640b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59050/0 (socket says 172.21.15.76:59050) 2023-11-09T03:48:26.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.989+0000 7f0769e81640 1 -- 172.21.15.76:0/2382313120 learned_addr learned my addr 172.21.15.76:0/2382313120 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:26.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.990+0000 7f0768e7f640 1 -- 172.21.15.76:0/2382313120 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 8296092 0 0) 0x7f07640b4110 con 0x7f0764159c70 2023-11-09T03:48:26.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.990+0000 7f0768e7f640 1 -- 172.21.15.76:0/2382313120 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0750003520 con 0x7f0764159c70 2023-11-09T03:48:26.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.990+0000 7f0769680640 1 --2- 172.21.15.76:0/2382313120 >> v2:172.21.15.113:3300/0 conn(0x7f07640b0ae0 0x7f07640ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:26.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.990+0000 7f076a682640 1 --2- 172.21.15.76:0/2382313120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f076406db20 0x7f076415d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:26.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.990+0000 7f0768e7f640 1 -- 172.21.15.76:0/2382313120 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2562158623 0 0) 0x7f0750003520 con 0x7f0764159c70 2023-11-09T03:48:26.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.990+0000 7f0768e7f640 1 -- 172.21.15.76:0/2382313120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f076406db20 msgr2=0x7f076415d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:26.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.990+0000 7f0768e7f640 1 --2- 172.21.15.76:0/2382313120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f076406db20 0x7f076415d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.990+0000 7f0768e7f640 1 -- 172.21.15.76:0/2382313120 >> v2:172.21.15.113:3300/0 conn(0x7f07640b0ae0 msgr2=0x7f07640ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:26.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.990+0000 7f0768e7f640 1 --2- 172.21.15.76:0/2382313120 >> v2:172.21.15.113:3300/0 conn(0x7f07640b0ae0 0x7f07640ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.990+0000 7f0768e7f640 1 -- 172.21.15.76:0/2382313120 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07640b46d0 con 0x7f0764159c70 2023-11-09T03:48:26.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.990+0000 7f0768e7f640 1 -- 172.21.15.76:0/2382313120 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f07600019f0 con 0x7f0764159c70 2023-11-09T03:48:26.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.990+0000 7f0768e7f640 1 -- 172.21.15.76:0/2382313120 >> v1:172.21.15.76:6789/0 conn(0x7f0764159c70 legacy=0x7f07640b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:26.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.990+0000 7f0768e7f640 1 --2- 172.21.15.76:0/2382313120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f076406db20 0x7f0750003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:26.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.990+0000 7f0768e7f640 1 --2- 172.21.15.76:0/2382313120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0750003ef0 0x7f07500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:26.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.990+0000 7f0768e7f640 1 --2- 172.21.15.76:0/2382313120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0750004d00 0x7f0750005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:26.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.990+0000 7f0768e7f640 1 -- 172.21.15.76:0/2382313120 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07640b49b0 con 0x7f0750004d00 2023-11-09T03:48:26.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.990+0000 7f0768e7f640 1 -- 172.21.15.76:0/2382313120 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07640b43f0 con 0x7f0750003ef0 2023-11-09T03:48:26.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.990+0000 7f0768e7f640 1 -- 172.21.15.76:0/2382313120 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f075000b980 con 0x7f076406db20 2023-11-09T03:48:26.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.990+0000 7f076a682640 1 --2- 172.21.15.76:0/2382313120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0750004d00 0x7f0750005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:26.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.990+0000 7f0769e81640 1 --2- 172.21.15.76:0/2382313120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f076406db20 0x7f0750003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:26.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.990+0000 7f0769680640 1 --2- 172.21.15.76:0/2382313120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0750003ef0 0x7f07500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:26.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.990+0000 7f076a682640 1 -- 172.21.15.76:0/2382313120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f076406db20 msgr2=0x7f0750003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:26.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.990+0000 7f076a682640 1 --2- 172.21.15.76:0/2382313120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f076406db20 0x7f0750003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.990+0000 7f076a682640 1 -- 172.21.15.76:0/2382313120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0750003ef0 msgr2=0x7f07500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:26.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.990+0000 7f076a682640 1 --2- 172.21.15.76:0/2382313120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0750003ef0 0x7f07500047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.990+0000 7f076a682640 1 -- 172.21.15.76:0/2382313120 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0750003520 con 0x7f0750004d00 2023-11-09T03:48:26.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.991+0000 7f0769e81640 1 --2- 172.21.15.76:0/2382313120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f076406db20 0x7f0750003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:26.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.991+0000 7f0769680640 1 --2- 172.21.15.76:0/2382313120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0750003ef0 0x7f07500047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:26.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.991+0000 7f076a682640 1 --2- 172.21.15.76:0/2382313120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0750004d00 0x7f0750005180 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f075c009ac0 tx=0x7f07500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51f3706f431d7b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:26.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.991+0000 7f0768e7f640 1 -- 172.21.15.76:0/2382313120 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c02c6b0 con 0x7f0750004d00 2023-11-09T03:48:26.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.991+0000 7f0768e7f640 1 -- 172.21.15.76:0/2382313120 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075c02c810 con 0x7f0750004d00 2023-11-09T03:48:26.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.991+0000 7f0768e7f640 1 -- 172.21.15.76:0/2382313120 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c02cba0 con 0x7f0750004d00 2023-11-09T03:48:26.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.992+0000 7f076c0e5640 1 -- 172.21.15.76:0/2382313120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0750004d00 msgr2=0x7f0750005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:26.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.992+0000 7f076c0e5640 1 --2- 172.21.15.76:0/2382313120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0750004d00 0x7f0750005180 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f075c009ac0 tx=0x7f07500057f0 comp rx=0 tx=0).stop 2023-11-09T03:48:26.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.992+0000 7f076a682640 1 -- 172.21.15.76:0/2382313120 reap_dead start 2023-11-09T03:48:26.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.992+0000 7f076c0e5640 1 -- 172.21.15.76:0/2382313120 shutdown_connections 2023-11-09T03:48:26.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.992+0000 7f076c0e5640 1 -- 172.21.15.76:0/2382313120 >> 172.21.15.76:0/2382313120 conn(0x7f07640ce950 msgr2=0x7f07640ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:27.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.992+0000 7f076c0e5640 1 -- 172.21.15.76:0/2382313120 shutdown_connections 2023-11-09T03:48:27.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.992+0000 7f076c0e5640 1 -- 172.21.15.76:0/2382313120 wait complete. 2023-11-09T03:48:27.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.993+0000 7f076c0e5640 1 Processor -- start 2023-11-09T03:48:27.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.993+0000 7f076c0e5640 1 -- start start 2023-11-09T03:48:27.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.994+0000 7f076c0e5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0750004d00 0x7f07640af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:27.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.994+0000 7f076c0e5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f076414dee0 0x7f07641f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:27.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.994+0000 7f076c0e5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07641f6450 0x7f07640ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:27.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.994+0000 7f076c0e5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07640b07b0 con 0x7f076414dee0 2023-11-09T03:48:27.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.994+0000 7f076c0e5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076414f970 con 0x7f07641f6450 2023-11-09T03:48:27.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.994+0000 7f076c0e5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07641f68d0 con 0x7f0750004d00 2023-11-09T03:48:27.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.994+0000 7f0769680640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f076414dee0 0x7f07641f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:27.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.994+0000 7f0769680640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f076414dee0 0x7f07641f5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37874/0 (socket says 172.21.15.76:37874) 2023-11-09T03:48:27.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.994+0000 7f0769680640 1 -- 172.21.15.76:0/2208859482 learned_addr learned my addr 172.21.15.76:0/2208859482 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:27.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.994+0000 7f076a682640 1 --2- 172.21.15.76:0/2208859482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07641f6450 0x7f07640ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:27.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.994+0000 7f0769e81640 1 --2- 172.21.15.76:0/2208859482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0750004d00 0x7f07640af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:27.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.994+0000 7f0769680640 1 -- 172.21.15.76:0/2208859482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0750004d00 msgr2=0x7f07640af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:27.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.994+0000 7f0769680640 1 --2- 172.21.15.76:0/2208859482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0750004d00 0x7f07640af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:27.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.994+0000 7f0769680640 1 -- 172.21.15.76:0/2208859482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07641f6450 msgr2=0x7f07640ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:27.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.994+0000 7f0769680640 1 --2- 172.21.15.76:0/2208859482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07641f6450 0x7f07640ab8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:27.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.994+0000 7f0769680640 1 -- 172.21.15.76:0/2208859482 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f076414e340 con 0x7f076414dee0 2023-11-09T03:48:27.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.994+0000 7f076a682640 1 --2- 172.21.15.76:0/2208859482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07641f6450 0x7f07640ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:27.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.994+0000 7f0769680640 1 --2- 172.21.15.76:0/2208859482 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f076414dee0 0x7f07641f5f10 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f0754011630 tx=0x7f0754011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:27.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.994+0000 7f075a7fc640 1 -- 172.21.15.76:0/2208859482 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075400f920 con 0x7f076414dee0 2023-11-09T03:48:27.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.994+0000 7f075a7fc640 1 -- 172.21.15.76:0/2208859482 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075400fa80 con 0x7f076414dee0 2023-11-09T03:48:27.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.995+0000 7f076c0e5640 1 -- 172.21.15.76:0/2208859482 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07640b04d0 con 0x7f076414dee0 2023-11-09T03:48:27.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.995+0000 7f075a7fc640 1 -- 172.21.15.76:0/2208859482 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075400fe10 con 0x7f076414dee0 2023-11-09T03:48:27.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.995+0000 7f076c0e5640 1 -- 172.21.15.76:0/2208859482 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f076414f600 con 0x7f076414dee0 2023-11-09T03:48:27.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.995+0000 7f076c0e5640 1 -- 172.21.15.76:0/2208859482 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0764074290 con 0x7f076414dee0 2023-11-09T03:48:27.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.996+0000 7f075a7fc640 1 -- 172.21.15.76:0/2208859482 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0754015460 con 0x7f076414dee0 2023-11-09T03:48:27.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.996+0000 7f075a7fc640 1 --2- 172.21.15.76:0/2208859482 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f074806aea0 0x7f074806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:27.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.996+0000 7f075a7fc640 1 -- 172.21.15.76:0/2208859482 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f075408a3a0 con 0x7f076414dee0 2023-11-09T03:48:27.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.996+0000 7f0769e81640 1 --2- 172.21.15.76:0/2208859482 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f074806aea0 0x7f074806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:27.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:26.997+0000 7f0769e81640 1 --2- 172.21.15.76:0/2208859482 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f074806aea0 0x7f074806d360 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0760001330 tx=0x7f0760009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:27.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.000+0000 7f075a7fc640 1 -- 172.21.15.76:0/2208859482 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0754054ee0 con 0x7f076414dee0 2023-11-09T03:48:27.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.162+0000 7f076c0e5640 1 -- 172.21.15.76:0/2208859482 --> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f07640cde50 con 0x7f074806aea0 2023-11-09T03:48:27.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.175+0000 7f075a7fc640 1 -- 172.21.15.76:0/2208859482 <== mgr.4105 v2:172.21.15.113:6832/1264193042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+526928 (secure 0 0 0) 0x7f0760015460 con 0x7f074806aea0 2023-11-09T03:48:27.179 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:27.183 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2023-11-09T03:48:27.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.179+0000 7f076c0e5640 1 -- 172.21.15.76:0/2208859482 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f074806aea0 msgr2=0x7f074806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:27.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.179+0000 7f076c0e5640 1 --2- 172.21.15.76:0/2208859482 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f074806aea0 0x7f074806d360 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0760001330 tx=0x7f0760009040 comp rx=0 tx=0).stop 2023-11-09T03:48:27.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.179+0000 7f076c0e5640 1 -- 172.21.15.76:0/2208859482 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f076414dee0 msgr2=0x7f07641f5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:27.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.179+0000 7f076c0e5640 1 --2- 172.21.15.76:0/2208859482 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f076414dee0 0x7f07641f5f10 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f0754011630 tx=0x7f0754011af0 comp rx=0 tx=0).stop 2023-11-09T03:48:27.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.179+0000 7f076c0e5640 1 -- 172.21.15.76:0/2208859482 shutdown_connections 2023-11-09T03:48:27.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.179+0000 7f076c0e5640 1 --2- 172.21.15.76:0/2208859482 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f074806aea0 0x7f074806d360 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:27.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.179+0000 7f076c0e5640 1 --2- 172.21.15.76:0/2208859482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07641f6450 0x7f07640ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:27.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.179+0000 7f076c0e5640 1 --2- 172.21.15.76:0/2208859482 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f076414dee0 0x7f07641f5f10 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:27.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.179+0000 7f076c0e5640 1 --2- 172.21.15.76:0/2208859482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0750004d00 0x7f07640af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:27.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.179+0000 7f076c0e5640 1 -- 172.21.15.76:0/2208859482 >> 172.21.15.76:0/2208859482 conn(0x7f07640ce950 msgr2=0x7f07640b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:27.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.179+0000 7f076c0e5640 1 -- 172.21.15.76:0/2208859482 shutdown_connections 2023-11-09T03:48:27.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.179+0000 7f076c0e5640 1 -- 172.21.15.76:0/2208859482 wait complete. 2023-11-09T03:48:27.189 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2023-11-09T03:48:26.794284+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":204,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237468,"kb_used_data":8828,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754737252,"statfs":{"total":773094113280,"available":772850946048,"internally_reserved":0,"allocated":9039872,"data_stored":4943241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.256434"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549889+0000","last_change":"2023-11-09T03:48:22.549889+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549889+0000","last_undegraded":"2023-11-09T03:48:22.549889+0000","last_fullsized":"2023-11-09T03:48:22.549889+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.554480+0000","last_change":"2023-11-09T03:48:22.554480+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.554480+0000","last_undegraded":"2023-11-09T03:48:22.554480+0000","last_fullsized":"2023-11-09T03:48:22.554480+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549408+0000","last_change":"2023-11-09T03:48:22.549408+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549408+0000","last_undegraded":"2023-11-09T03:48:22.549408+0000","last_fullsized":"2023-11-09T03:48:22.549408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550714+0000","last_change":"2023-11-09T03:48:22.550714+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550714+0000","last_undegraded":"2023-11-09T03:48:22.550714+0000","last_fullsized":"2023-11-09T03:48:22.550714+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550039+0000","last_change":"2023-11-09T03:48:22.550039+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550039+0000","last_undegraded":"2023-11-09T03:48:22.550039+0000","last_fullsized":"2023-11-09T03:48:22.550039+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549702+0000","last_change":"2023-11-09T03:48:22.549702+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549702+0000","last_undegraded":"2023-11-09T03:48:22.549702+0000","last_fullsized":"2023-11-09T03:48:22.549702+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550846+0000","last_change":"2023-11-09T03:48:22.550846+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550846+0000","last_undegraded":"2023-11-09T03:48:22.550846+0000","last_fullsized":"2023-11-09T03:48:22.550846+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.554400+0000","last_change":"2023-11-09T03:48:22.554400+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.554400+0000","last_undegraded":"2023-11-09T03:48:22.554400+0000","last_fullsized":"2023-11-09T03:48:22.554400+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.547291+0000","last_change":"2023-11-09T03:48:22.547291+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.547291+0000","last_undegraded":"2023-11-09T03:48:22.547291+0000","last_fullsized":"2023-11-09T03:48:22.547291+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551875+0000","last_change":"2023-11-09T03:48:22.551875+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551875+0000","last_undegraded":"2023-11-09T03:48:22.551875+0000","last_fullsized":"2023-11-09T03:48:22.551875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.554690+0000","last_change":"2023-11-09T03:48:22.554690+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.554690+0000","last_undegraded":"2023-11-09T03:48:22.554690+0000","last_fullsized":"2023-11-09T03:48:22.554690+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.547393+0000","last_change":"2023-11-09T03:48:22.547393+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.547393+0000","last_undegraded":"2023-11-09T03:48:22.547393+0000","last_fullsized":"2023-11-09T03:48:22.547393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550250+0000","last_change":"2023-11-09T03:48:22.550250+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550250+0000","last_undegraded":"2023-11-09T03:48:22.550250+0000","last_fullsized":"2023-11-09T03:48:22.550250+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.558130+0000","last_change":"2023-11-09T03:48:22.558130+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.558130+0000","last_undegraded":"2023-11-09T03:48:22.558130+0000","last_fullsized":"2023-11-09T03:48:22.558130+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549957+0000","last_change":"2023-11-09T03:48:22.549957+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549957+0000","last_undegraded":"2023-11-09T03:48:22.549957+0000","last_fullsized":"2023-11-09T03:48:22.549957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.553338+0000","last_change":"2023-11-09T03:48:22.553338+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.553338+0000","last_undegraded":"2023-11-09T03:48:22.553338+0000","last_fullsized":"2023-11-09T03:48:22.553338+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.554384+0000","last_change":"2023-11-09T03:48:22.554384+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.554384+0000","last_undegraded":"2023-11-09T03:48:22.554384+0000","last_fullsized":"2023-11-09T03:48:22.554384+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550716+0000","last_change":"2023-11-09T03:48:22.550716+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550716+0000","last_undegraded":"2023-11-09T03:48:22.550716+0000","last_fullsized":"2023-11-09T03:48:22.550716+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.557699+0000","last_change":"2023-11-09T03:48:22.557699+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.557699+0000","last_undegraded":"2023-11-09T03:48:22.557699+0000","last_fullsized":"2023-11-09T03:48:22.557699+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.546650+0000","last_change":"2023-11-09T03:48:22.546650+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.546650+0000","last_undegraded":"2023-11-09T03:48:22.546650+0000","last_fullsized":"2023-11-09T03:48:22.546650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.555079+0000","last_change":"2023-11-09T03:48:22.555079+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.555079+0000","last_undegraded":"2023-11-09T03:48:22.555079+0000","last_fullsized":"2023-11-09T03:48:22.555079+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.554040+0000","last_change":"2023-11-09T03:48:22.554040+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.554040+0000","last_undegraded":"2023-11-09T03:48:22.554040+0000","last_fullsized":"2023-11-09T03:48:22.554040+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.553445+0000","last_change":"2023-11-09T03:48:22.553445+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.553445+0000","last_undegraded":"2023-11-09T03:48:22.553445+0000","last_fullsized":"2023-11-09T03:48:22.553445+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551333+0000","last_change":"2023-11-09T03:48:22.551333+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551333+0000","last_undegraded":"2023-11-09T03:48:22.551333+0000","last_fullsized":"2023-11-09T03:48:22.551333+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.553219+0000","last_change":"2023-11-09T03:48:22.553219+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.553219+0000","last_undegraded":"2023-11-09T03:48:22.553219+0000","last_fullsized":"2023-11-09T03:48:22.553219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.552812+0000","last_change":"2023-11-09T03:48:22.552812+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.552812+0000","last_undegraded":"2023-11-09T03:48:22.552812+0000","last_fullsized":"2023-11-09T03:48:22.552812+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.552951+0000","last_change":"2023-11-09T03:48:22.552951+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.552951+0000","last_undegraded":"2023-11-09T03:48:22.552951+0000","last_fullsized":"2023-11-09T03:48:22.552951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.552510+0000","last_change":"2023-11-09T03:48:22.552510+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.552510+0000","last_undegraded":"2023-11-09T03:48:22.552510+0000","last_fullsized":"2023-11-09T03:48:22.552510+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.552644+0000","last_change":"2023-11-09T03:48:22.552644+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.552644+0000","last_undegraded":"2023-11-09T03:48:22.552644+0000","last_fullsized":"2023-11-09T03:48:22.552644+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551900+0000","last_change":"2023-11-09T03:48:22.551900+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551900+0000","last_undegraded":"2023-11-09T03:48:22.551900+0000","last_fullsized":"2023-11-09T03:48:22.551900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549131+0000","last_change":"2023-11-09T03:48:22.549131+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549131+0000","last_undegraded":"2023-11-09T03:48:22.549131+0000","last_fullsized":"2023-11-09T03:48:22.549131+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.548919+0000","last_change":"2023-11-09T03:48:22.548919+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.548919+0000","last_undegraded":"2023-11-09T03:48:22.548919+0000","last_fullsized":"2023-11-09T03:48:22.548919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.554246+0000","last_change":"2023-11-09T03:48:22.554246+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.554246+0000","last_undegraded":"2023-11-09T03:48:22.554246+0000","last_fullsized":"2023-11-09T03:48:22.554246+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.568615+0000","last_change":"2023-11-09T03:48:22.562765+0000","last_active":"2023-11-09T03:48:22.568615+0000","last_peered":"2023-11-09T03:48:22.568615+0000","last_clean":"2023-11-09T03:48:22.568615+0000","last_became_active":"2023-11-09T03:48:22.562455+0000","last_became_peered":"2023-11-09T03:48:22.562455+0000","last_unstale":"2023-11-09T03:48:22.568615+0000","last_undegraded":"2023-11-09T03:48:22.568615+0000","last_fullsized":"2023-11-09T03:48:22.568615+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:55:13.543207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545671+0000","last_change":"2023-11-09T03:48:01.449077+0000","last_active":"2023-11-09T03:48:21.545671+0000","last_peered":"2023-11-09T03:48:21.545671+0000","last_clean":"2023-11-09T03:48:21.545671+0000","last_became_active":"2023-11-09T03:48:01.448806+0000","last_became_peered":"2023-11-09T03:48:01.448806+0000","last_unstale":"2023-11-09T03:48:21.545671+0000","last_undegraded":"2023-11-09T03:48:21.545671+0000","last_fullsized":"2023-11-09T03:48:21.545671+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:16:00.220961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542885+0000","last_change":"2023-11-09T03:48:02.611756+0000","last_active":"2023-11-09T03:48:21.542885+0000","last_peered":"2023-11-09T03:48:21.542885+0000","last_clean":"2023-11-09T03:48:21.542885+0000","last_became_active":"2023-11-09T03:48:00.445880+0000","last_became_peered":"2023-11-09T03:48:00.445880+0000","last_unstale":"2023-11-09T03:48:21.542885+0000","last_undegraded":"2023-11-09T03:48:21.542885+0000","last_fullsized":"2023-11-09T03:48:21.542885+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:15:35.281296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085514900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539506+0000","last_change":"2023-11-09T03:47:02.581917+0000","last_active":"2023-11-09T03:48:21.539506+0000","last_peered":"2023-11-09T03:48:21.539506+0000","last_clean":"2023-11-09T03:48:21.539506+0000","last_became_active":"2023-11-09T03:47:01.588082+0000","last_became_peered":"2023-11-09T03:47:01.588082+0000","last_unstale":"2023-11-09T03:48:21.539506+0000","last_undegraded":"2023-11-09T03:48:21.539506+0000","last_fullsized":"2023-11-09T03:48:21.539506+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:44:07.653557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034538899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.553249+0000","last_change":"2023-11-09T03:48:22.553249+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.553249+0000","last_undegraded":"2023-11-09T03:48:22.553249+0000","last_fullsized":"2023-11-09T03:48:22.553249+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.571401+0000","last_change":"2023-11-09T03:48:22.566342+0000","last_active":"2023-11-09T03:48:22.571401+0000","last_peered":"2023-11-09T03:48:22.571401+0000","last_clean":"2023-11-09T03:48:22.571401+0000","last_became_active":"2023-11-09T03:48:22.566028+0000","last_became_peered":"2023-11-09T03:48:22.566028+0000","last_unstale":"2023-11-09T03:48:22.571401+0000","last_undegraded":"2023-11-09T03:48:22.571401+0000","last_fullsized":"2023-11-09T03:48:22.571401+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:31.602666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542096+0000","last_change":"2023-11-09T03:48:01.456245+0000","last_active":"2023-11-09T03:48:21.542096+0000","last_peered":"2023-11-09T03:48:21.542096+0000","last_clean":"2023-11-09T03:48:21.542096+0000","last_became_active":"2023-11-09T03:48:01.455944+0000","last_became_peered":"2023-11-09T03:48:01.455944+0000","last_unstale":"2023-11-09T03:48:21.542096+0000","last_undegraded":"2023-11-09T03:48:21.542096+0000","last_fullsized":"2023-11-09T03:48:21.542096+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:06:14.849196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543120+0000","last_change":"2023-11-09T03:48:02.612341+0000","last_active":"2023-11-09T03:48:21.543120+0000","last_peered":"2023-11-09T03:48:21.543120+0000","last_clean":"2023-11-09T03:48:21.543120+0000","last_became_active":"2023-11-09T03:48:00.444426+0000","last_became_peered":"2023-11-09T03:48:00.444426+0000","last_unstale":"2023-11-09T03:48:21.543120+0000","last_undegraded":"2023-11-09T03:48:21.543120+0000","last_fullsized":"2023-11-09T03:48:21.543120+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:19:53.969861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010812420000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545670+0000","last_change":"2023-11-09T03:47:03.029063+0000","last_active":"2023-11-09T03:48:21.545670+0000","last_peered":"2023-11-09T03:48:21.545670+0000","last_clean":"2023-11-09T03:48:21.545670+0000","last_became_active":"2023-11-09T03:47:01.593582+0000","last_became_peered":"2023-11-09T03:47:01.593582+0000","last_unstale":"2023-11-09T03:48:21.545670+0000","last_undegraded":"2023-11-09T03:48:21.545670+0000","last_fullsized":"2023-11-09T03:48:21.545670+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:56:20.649937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041855100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.546617+0000","last_change":"2023-11-09T03:48:22.546617+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.546617+0000","last_undegraded":"2023-11-09T03:48:22.546617+0000","last_fullsized":"2023-11-09T03:48:22.546617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.564376+0000","last_change":"2023-11-09T03:48:22.562238+0000","last_active":"2023-11-09T03:48:22.564376+0000","last_peered":"2023-11-09T03:48:22.564376+0000","last_clean":"2023-11-09T03:48:22.564376+0000","last_became_active":"2023-11-09T03:48:22.561984+0000","last_became_peered":"2023-11-09T03:48:22.561984+0000","last_unstale":"2023-11-09T03:48:22.564376+0000","last_undegraded":"2023-11-09T03:48:22.564376+0000","last_fullsized":"2023-11-09T03:48:22.564376+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:28:06.557471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541062+0000","last_change":"2023-11-09T03:48:01.450271+0000","last_active":"2023-11-09T03:48:21.541062+0000","last_peered":"2023-11-09T03:48:21.541062+0000","last_clean":"2023-11-09T03:48:21.541062+0000","last_became_active":"2023-11-09T03:48:01.449998+0000","last_became_peered":"2023-11-09T03:48:01.449998+0000","last_unstale":"2023-11-09T03:48:21.541062+0000","last_undegraded":"2023-11-09T03:48:21.541062+0000","last_fullsized":"2023-11-09T03:48:21.541062+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:12:27.170650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545599+0000","last_change":"2023-11-09T03:48:02.852114+0000","last_active":"2023-11-09T03:48:21.545599+0000","last_peered":"2023-11-09T03:48:21.545599+0000","last_clean":"2023-11-09T03:48:21.545599+0000","last_became_active":"2023-11-09T03:48:00.447203+0000","last_became_peered":"2023-11-09T03:48:00.447203+0000","last_unstale":"2023-11-09T03:48:21.545599+0000","last_undegraded":"2023-11-09T03:48:21.545599+0000","last_fullsized":"2023-11-09T03:48:21.545599+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:11:36.826430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049189500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543495+0000","last_change":"2023-11-09T03:47:02.619336+0000","last_active":"2023-11-09T03:48:21.543495+0000","last_peered":"2023-11-09T03:48:21.543495+0000","last_clean":"2023-11-09T03:48:21.543495+0000","last_became_active":"2023-11-09T03:47:01.587749+0000","last_became_peered":"2023-11-09T03:47:01.587749+0000","last_unstale":"2023-11-09T03:48:21.543495+0000","last_undegraded":"2023-11-09T03:48:21.543495+0000","last_fullsized":"2023-11-09T03:48:21.543495+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063032099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551692+0000","last_change":"2023-11-09T03:48:22.551692+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551692+0000","last_undegraded":"2023-11-09T03:48:22.551692+0000","last_fullsized":"2023-11-09T03:48:22.551692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.559640+0000","last_change":"2023-11-09T03:48:22.557530+0000","last_active":"2023-11-09T03:48:22.559640+0000","last_peered":"2023-11-09T03:48:22.559640+0000","last_clean":"2023-11-09T03:48:22.559640+0000","last_became_active":"2023-11-09T03:48:22.556685+0000","last_became_peered":"2023-11-09T03:48:22.556685+0000","last_unstale":"2023-11-09T03:48:22.559640+0000","last_undegraded":"2023-11-09T03:48:22.559640+0000","last_fullsized":"2023-11-09T03:48:22.559640+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:06:49.685032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547581+0000","last_change":"2023-11-09T03:48:01.454147+0000","last_active":"2023-11-09T03:48:21.547581+0000","last_peered":"2023-11-09T03:48:21.547581+0000","last_clean":"2023-11-09T03:48:21.547581+0000","last_became_active":"2023-11-09T03:48:01.453868+0000","last_became_peered":"2023-11-09T03:48:01.453868+0000","last_unstale":"2023-11-09T03:48:21.547581+0000","last_undegraded":"2023-11-09T03:48:21.547581+0000","last_fullsized":"2023-11-09T03:48:21.547581+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:05:09.655318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542848+0000","last_change":"2023-11-09T03:48:02.612000+0000","last_active":"2023-11-09T03:48:21.542848+0000","last_peered":"2023-11-09T03:48:21.542848+0000","last_clean":"2023-11-09T03:48:21.542848+0000","last_became_active":"2023-11-09T03:48:00.441667+0000","last_became_peered":"2023-11-09T03:48:00.441667+0000","last_unstale":"2023-11-09T03:48:21.542848+0000","last_undegraded":"2023-11-09T03:48:21.542848+0000","last_fullsized":"2023-11-09T03:48:21.542848+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:12:26.164644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000966781,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539495+0000","last_change":"2023-11-09T03:47:02.581658+0000","last_active":"2023-11-09T03:48:21.539495+0000","last_peered":"2023-11-09T03:48:21.539495+0000","last_clean":"2023-11-09T03:48:21.539495+0000","last_became_active":"2023-11-09T03:47:01.588612+0000","last_became_peered":"2023-11-09T03:47:01.588612+0000","last_unstale":"2023-11-09T03:48:21.539495+0000","last_undegraded":"2023-11-09T03:48:21.539495+0000","last_fullsized":"2023-11-09T03:48:21.539495+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045238500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.547531+0000","last_change":"2023-11-09T03:48:22.547531+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.547531+0000","last_undegraded":"2023-11-09T03:48:22.547531+0000","last_fullsized":"2023-11-09T03:48:22.547531+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.560046+0000","last_change":"2023-11-09T03:48:22.557828+0000","last_active":"2023-11-09T03:48:22.560046+0000","last_peered":"2023-11-09T03:48:22.560046+0000","last_clean":"2023-11-09T03:48:22.560046+0000","last_became_active":"2023-11-09T03:48:22.557588+0000","last_became_peered":"2023-11-09T03:48:22.557588+0000","last_unstale":"2023-11-09T03:48:22.560046+0000","last_undegraded":"2023-11-09T03:48:22.560046+0000","last_fullsized":"2023-11-09T03:48:22.560046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:36:15.980235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545124+0000","last_change":"2023-11-09T03:48:01.449834+0000","last_active":"2023-11-09T03:48:21.545124+0000","last_peered":"2023-11-09T03:48:21.545124+0000","last_clean":"2023-11-09T03:48:21.545124+0000","last_became_active":"2023-11-09T03:48:01.449552+0000","last_became_peered":"2023-11-09T03:48:01.449552+0000","last_unstale":"2023-11-09T03:48:21.545124+0000","last_undegraded":"2023-11-09T03:48:21.545124+0000","last_fullsized":"2023-11-09T03:48:21.545124+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:11:37.832436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542114+0000","last_change":"2023-11-09T03:48:02.682197+0000","last_active":"2023-11-09T03:48:21.542114+0000","last_peered":"2023-11-09T03:48:21.542114+0000","last_clean":"2023-11-09T03:48:21.542114+0000","last_became_active":"2023-11-09T03:48:00.445616+0000","last_became_peered":"2023-11-09T03:48:00.445616+0000","last_unstale":"2023-11-09T03:48:21.542114+0000","last_undegraded":"2023-11-09T03:48:21.542114+0000","last_fullsized":"2023-11-09T03:48:21.542114+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:57:03.474084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000416494,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542622+0000","last_change":"2023-11-09T03:47:02.618828+0000","last_active":"2023-11-09T03:48:21.542622+0000","last_peered":"2023-11-09T03:48:21.542622+0000","last_clean":"2023-11-09T03:48:21.542622+0000","last_became_active":"2023-11-09T03:47:01.590679+0000","last_became_peered":"2023-11-09T03:47:01.590679+0000","last_unstale":"2023-11-09T03:48:21.542622+0000","last_undegraded":"2023-11-09T03:48:21.542622+0000","last_fullsized":"2023-11-09T03:48:21.542622+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:06:30.982987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070668,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.546525+0000","last_change":"2023-11-09T03:48:22.546525+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.546525+0000","last_undegraded":"2023-11-09T03:48:22.546525+0000","last_fullsized":"2023-11-09T03:48:22.546525+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.560446+0000","last_change":"2023-11-09T03:48:22.556590+0000","last_active":"2023-11-09T03:48:22.560446+0000","last_peered":"2023-11-09T03:48:22.560446+0000","last_clean":"2023-11-09T03:48:22.560446+0000","last_became_active":"2023-11-09T03:48:22.556295+0000","last_became_peered":"2023-11-09T03:48:22.556295+0000","last_unstale":"2023-11-09T03:48:22.560446+0000","last_undegraded":"2023-11-09T03:48:22.560446+0000","last_fullsized":"2023-11-09T03:48:22.560446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:09:09.041542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.546994+0000","last_change":"2023-11-09T03:48:01.447647+0000","last_active":"2023-11-09T03:48:21.546994+0000","last_peered":"2023-11-09T03:48:21.546994+0000","last_clean":"2023-11-09T03:48:21.546994+0000","last_became_active":"2023-11-09T03:48:01.447302+0000","last_became_peered":"2023-11-09T03:48:01.447302+0000","last_unstale":"2023-11-09T03:48:21.546994+0000","last_undegraded":"2023-11-09T03:48:21.546994+0000","last_fullsized":"2023-11-09T03:48:21.546994+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:08:47.931939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539407+0000","last_change":"2023-11-09T03:48:03.327731+0000","last_active":"2023-11-09T03:48:21.539407+0000","last_peered":"2023-11-09T03:48:21.539407+0000","last_clean":"2023-11-09T03:48:21.539407+0000","last_became_active":"2023-11-09T03:48:00.449660+0000","last_became_peered":"2023-11-09T03:48:00.449660+0000","last_unstale":"2023-11-09T03:48:21.539407+0000","last_undegraded":"2023-11-09T03:48:21.539407+0000","last_fullsized":"2023-11-09T03:48:21.539407+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:02:07.940908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037816599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.546874+0000","last_change":"2023-11-09T03:47:03.146528+0000","last_active":"2023-11-09T03:48:21.546874+0000","last_peered":"2023-11-09T03:48:21.546874+0000","last_clean":"2023-11-09T03:48:21.546874+0000","last_became_active":"2023-11-09T03:47:01.587060+0000","last_became_peered":"2023-11-09T03:47:01.587060+0000","last_unstale":"2023-11-09T03:48:21.546874+0000","last_undegraded":"2023-11-09T03:48:21.546874+0000","last_fullsized":"2023-11-09T03:48:21.546874+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:39:15.238496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079064999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.547973+0000","last_change":"2023-11-09T03:48:22.547973+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.547973+0000","last_undegraded":"2023-11-09T03:48:22.547973+0000","last_fullsized":"2023-11-09T03:48:22.547973+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.569334+0000","last_change":"2023-11-09T03:48:22.565708+0000","last_active":"2023-11-09T03:48:22.569334+0000","last_peered":"2023-11-09T03:48:22.569334+0000","last_clean":"2023-11-09T03:48:22.569334+0000","last_became_active":"2023-11-09T03:48:22.563961+0000","last_became_peered":"2023-11-09T03:48:22.563961+0000","last_unstale":"2023-11-09T03:48:22.569334+0000","last_undegraded":"2023-11-09T03:48:22.569334+0000","last_fullsized":"2023-11-09T03:48:22.569334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:03:03.041986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539139+0000","last_change":"2023-11-09T03:48:01.453939+0000","last_active":"2023-11-09T03:48:21.539139+0000","last_peered":"2023-11-09T03:48:21.539139+0000","last_clean":"2023-11-09T03:48:21.539139+0000","last_became_active":"2023-11-09T03:48:01.453600+0000","last_became_peered":"2023-11-09T03:48:01.453600+0000","last_unstale":"2023-11-09T03:48:21.539139+0000","last_undegraded":"2023-11-09T03:48:21.539139+0000","last_fullsized":"2023-11-09T03:48:21.539139+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:45:10.522254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547099+0000","last_change":"2023-11-09T03:48:02.440910+0000","last_active":"2023-11-09T03:48:21.547099+0000","last_peered":"2023-11-09T03:48:21.547099+0000","last_clean":"2023-11-09T03:48:21.547099+0000","last_became_active":"2023-11-09T03:48:00.443791+0000","last_became_peered":"2023-11-09T03:48:00.443791+0000","last_unstale":"2023-11-09T03:48:21.547099+0000","last_undegraded":"2023-11-09T03:48:21.547099+0000","last_fullsized":"2023-11-09T03:48:21.547099+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:11:36.826430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069223400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547414+0000","last_change":"2023-11-09T03:47:03.148246+0000","last_active":"2023-11-09T03:48:21.547414+0000","last_peered":"2023-11-09T03:48:21.547414+0000","last_clean":"2023-11-09T03:48:21.547414+0000","last_became_active":"2023-11-09T03:47:01.590988+0000","last_became_peered":"2023-11-09T03:47:01.590988+0000","last_unstale":"2023-11-09T03:48:21.547414+0000","last_undegraded":"2023-11-09T03:48:21.547414+0000","last_fullsized":"2023-11-09T03:48:21.547414+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:07:20.785445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055651099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551090+0000","last_change":"2023-11-09T03:48:22.551090+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551090+0000","last_undegraded":"2023-11-09T03:48:22.551090+0000","last_fullsized":"2023-11-09T03:48:22.551090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.569365+0000","last_change":"2023-11-09T03:48:22.565920+0000","last_active":"2023-11-09T03:48:22.569365+0000","last_peered":"2023-11-09T03:48:22.569365+0000","last_clean":"2023-11-09T03:48:22.569365+0000","last_became_active":"2023-11-09T03:48:22.564592+0000","last_became_peered":"2023-11-09T03:48:22.564592+0000","last_unstale":"2023-11-09T03:48:22.569365+0000","last_undegraded":"2023-11-09T03:48:22.569365+0000","last_fullsized":"2023-11-09T03:48:22.569365+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:41:46.534565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539107+0000","last_change":"2023-11-09T03:48:01.450557+0000","last_active":"2023-11-09T03:48:21.539107+0000","last_peered":"2023-11-09T03:48:21.539107+0000","last_clean":"2023-11-09T03:48:21.539107+0000","last_became_active":"2023-11-09T03:48:01.450291+0000","last_became_peered":"2023-11-09T03:48:01.450291+0000","last_unstale":"2023-11-09T03:48:21.539107+0000","last_undegraded":"2023-11-09T03:48:21.539107+0000","last_fullsized":"2023-11-09T03:48:21.539107+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:21:29.699594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545002+0000","last_change":"2023-11-09T03:48:02.852010+0000","last_active":"2023-11-09T03:48:21.545002+0000","last_peered":"2023-11-09T03:48:21.545002+0000","last_clean":"2023-11-09T03:48:21.545002+0000","last_became_active":"2023-11-09T03:48:00.445267+0000","last_became_peered":"2023-11-09T03:48:00.445267+0000","last_unstale":"2023-11-09T03:48:21.545002+0000","last_undegraded":"2023-11-09T03:48:21.545002+0000","last_fullsized":"2023-11-09T03:48:21.545002+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:17.989628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064559199999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":50,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540983+0000","last_change":"2023-11-09T03:47:02.582754+0000","last_active":"2023-11-09T03:48:21.540983+0000","last_peered":"2023-11-09T03:48:21.540983+0000","last_clean":"2023-11-09T03:48:21.540983+0000","last_became_active":"2023-11-09T03:47:01.589715+0000","last_became_peered":"2023-11-09T03:47:01.589715+0000","last_unstale":"2023-11-09T03:48:21.540983+0000","last_undegraded":"2023-11-09T03:48:21.540983+0000","last_fullsized":"2023-11-09T03:48:21.540983+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099342799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.546539+0000","last_change":"2023-11-09T03:48:22.546539+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.546539+0000","last_undegraded":"2023-11-09T03:48:22.546539+0000","last_fullsized":"2023-11-09T03:48:22.546539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.571909+0000","last_change":"2023-11-09T03:48:22.566859+0000","last_active":"2023-11-09T03:48:22.571909+0000","last_peered":"2023-11-09T03:48:22.571909+0000","last_clean":"2023-11-09T03:48:22.571909+0000","last_became_active":"2023-11-09T03:48:22.566080+0000","last_became_peered":"2023-11-09T03:48:22.566080+0000","last_unstale":"2023-11-09T03:48:22.571909+0000","last_undegraded":"2023-11-09T03:48:22.571909+0000","last_fullsized":"2023-11-09T03:48:22.571909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:14:56.468255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543312+0000","last_change":"2023-11-09T03:48:01.452456+0000","last_active":"2023-11-09T03:48:21.543312+0000","last_peered":"2023-11-09T03:48:21.543312+0000","last_clean":"2023-11-09T03:48:21.543312+0000","last_became_active":"2023-11-09T03:48:01.452059+0000","last_became_peered":"2023-11-09T03:48:01.452059+0000","last_unstale":"2023-11-09T03:48:21.543312+0000","last_undegraded":"2023-11-09T03:48:21.543312+0000","last_fullsized":"2023-11-09T03:48:21.543312+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:57:56.855869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541181+0000","last_change":"2023-11-09T03:48:02.442557+0000","last_active":"2023-11-09T03:48:21.541181+0000","last_peered":"2023-11-09T03:48:21.541181+0000","last_clean":"2023-11-09T03:48:21.541181+0000","last_became_active":"2023-11-09T03:48:00.444823+0000","last_became_peered":"2023-11-09T03:48:00.444823+0000","last_unstale":"2023-11-09T03:48:21.541181+0000","last_undegraded":"2023-11-09T03:48:21.541181+0000","last_fullsized":"2023-11-09T03:48:21.541181+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:19:53.969861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075076299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":47,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545453+0000","last_change":"2023-11-09T03:47:02.724896+0000","last_active":"2023-11-09T03:48:21.545453+0000","last_peered":"2023-11-09T03:48:21.545453+0000","last_clean":"2023-11-09T03:48:21.545453+0000","last_became_active":"2023-11-09T03:47:01.592784+0000","last_became_peered":"2023-11-09T03:47:01.592784+0000","last_unstale":"2023-11-09T03:48:21.545453+0000","last_undegraded":"2023-11-09T03:48:21.545453+0000","last_fullsized":"2023-11-09T03:48:21.545453+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:51:32.016204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056683500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.548469+0000","last_change":"2023-11-09T03:48:22.548469+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.548469+0000","last_undegraded":"2023-11-09T03:48:22.548469+0000","last_fullsized":"2023-11-09T03:48:22.548469+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.571850+0000","last_change":"2023-11-09T03:48:22.565279+0000","last_active":"2023-11-09T03:48:22.571850+0000","last_peered":"2023-11-09T03:48:22.571850+0000","last_clean":"2023-11-09T03:48:22.571850+0000","last_became_active":"2023-11-09T03:48:22.564998+0000","last_became_peered":"2023-11-09T03:48:22.564998+0000","last_unstale":"2023-11-09T03:48:22.571850+0000","last_undegraded":"2023-11-09T03:48:22.571850+0000","last_fullsized":"2023-11-09T03:48:22.571850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:24:48.220122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540896+0000","last_change":"2023-11-09T03:48:01.452167+0000","last_active":"2023-11-09T03:48:21.540896+0000","last_peered":"2023-11-09T03:48:21.540896+0000","last_clean":"2023-11-09T03:48:21.540896+0000","last_became_active":"2023-11-09T03:48:01.451780+0000","last_became_peered":"2023-11-09T03:48:01.451780+0000","last_unstale":"2023-11-09T03:48:21.540896+0000","last_undegraded":"2023-11-09T03:48:21.540896+0000","last_fullsized":"2023-11-09T03:48:21.540896+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:15:36.287302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543137+0000","last_change":"2023-11-09T03:48:02.612246+0000","last_active":"2023-11-09T03:48:21.543137+0000","last_peered":"2023-11-09T03:48:21.543137+0000","last_clean":"2023-11-09T03:48:21.543137+0000","last_became_active":"2023-11-09T03:48:00.444629+0000","last_became_peered":"2023-11-09T03:48:00.444629+0000","last_unstale":"2023-11-09T03:48:21.543137+0000","last_undegraded":"2023-11-09T03:48:21.543137+0000","last_fullsized":"2023-11-09T03:48:21.543137+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:51:17.754054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098167299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541889+0000","last_change":"2023-11-09T03:47:03.146527+0000","last_active":"2023-11-09T03:48:21.541889+0000","last_peered":"2023-11-09T03:48:21.541889+0000","last_clean":"2023-11-09T03:48:21.541889+0000","last_became_active":"2023-11-09T03:47:01.587858+0000","last_became_peered":"2023-11-09T03:47:01.587858+0000","last_unstale":"2023-11-09T03:48:21.541889+0000","last_undegraded":"2023-11-09T03:48:21.541889+0000","last_fullsized":"2023-11-09T03:48:21.541889+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:56:04.201126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084439599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.548328+0000","last_change":"2023-11-09T03:48:22.548328+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.548328+0000","last_undegraded":"2023-11-09T03:48:22.548328+0000","last_fullsized":"2023-11-09T03:48:22.548328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.558055+0000","last_change":"2023-11-09T03:48:22.553920+0000","last_active":"2023-11-09T03:48:22.558055+0000","last_peered":"2023-11-09T03:48:22.558055+0000","last_clean":"2023-11-09T03:48:22.558055+0000","last_became_active":"2023-11-09T03:48:22.552644+0000","last_became_peered":"2023-11-09T03:48:22.552644+0000","last_unstale":"2023-11-09T03:48:22.558055+0000","last_undegraded":"2023-11-09T03:48:22.558055+0000","last_fullsized":"2023-11-09T03:48:22.558055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:59.646501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545255+0000","last_change":"2023-11-09T03:48:01.450685+0000","last_active":"2023-11-09T03:48:21.545255+0000","last_peered":"2023-11-09T03:48:21.545255+0000","last_clean":"2023-11-09T03:48:21.545255+0000","last_became_active":"2023-11-09T03:48:01.450445+0000","last_became_peered":"2023-11-09T03:48:01.450445+0000","last_unstale":"2023-11-09T03:48:21.545255+0000","last_undegraded":"2023-11-09T03:48:21.545255+0000","last_fullsized":"2023-11-09T03:48:21.545255+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:16:00.220961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547739+0000","last_change":"2023-11-09T03:48:02.441928+0000","last_active":"2023-11-09T03:48:21.547739+0000","last_peered":"2023-11-09T03:48:21.547739+0000","last_clean":"2023-11-09T03:48:21.547739+0000","last_became_active":"2023-11-09T03:48:00.444417+0000","last_became_peered":"2023-11-09T03:48:00.444417+0000","last_unstale":"2023-11-09T03:48:21.547739+0000","last_undegraded":"2023-11-09T03:48:21.547739+0000","last_fullsized":"2023-11-09T03:48:21.547739+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:55.450096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072428900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":47,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545707+0000","last_change":"2023-11-09T03:47:02.725223+0000","last_active":"2023-11-09T03:48:21.545707+0000","last_peered":"2023-11-09T03:48:21.545707+0000","last_clean":"2023-11-09T03:48:21.545707+0000","last_became_active":"2023-11-09T03:47:01.590635+0000","last_became_peered":"2023-11-09T03:47:01.590635+0000","last_unstale":"2023-11-09T03:48:21.545707+0000","last_undegraded":"2023-11-09T03:48:21.545707+0000","last_fullsized":"2023-11-09T03:48:21.545707+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:05.754357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051685100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551472+0000","last_change":"2023-11-09T03:48:22.551472+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551472+0000","last_undegraded":"2023-11-09T03:48:22.551472+0000","last_fullsized":"2023-11-09T03:48:22.551472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.569184+0000","last_change":"2023-11-09T03:48:22.563841+0000","last_active":"2023-11-09T03:48:22.569184+0000","last_peered":"2023-11-09T03:48:22.569184+0000","last_clean":"2023-11-09T03:48:22.569184+0000","last_became_active":"2023-11-09T03:48:22.563590+0000","last_became_peered":"2023-11-09T03:48:22.563590+0000","last_unstale":"2023-11-09T03:48:22.569184+0000","last_undegraded":"2023-11-09T03:48:22.569184+0000","last_fullsized":"2023-11-09T03:48:22.569184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:10:36.225903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542779+0000","last_change":"2023-11-09T03:48:01.453143+0000","last_active":"2023-11-09T03:48:21.542779+0000","last_peered":"2023-11-09T03:48:21.542779+0000","last_clean":"2023-11-09T03:48:21.542779+0000","last_became_active":"2023-11-09T03:48:01.452873+0000","last_became_peered":"2023-11-09T03:48:01.452873+0000","last_unstale":"2023-11-09T03:48:21.542779+0000","last_undegraded":"2023-11-09T03:48:21.542779+0000","last_fullsized":"2023-11-09T03:48:21.542779+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:06:28.575429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.548086+0000","last_change":"2023-11-09T03:48:02.442416+0000","last_active":"2023-11-09T03:48:21.548086+0000","last_peered":"2023-11-09T03:48:21.548086+0000","last_clean":"2023-11-09T03:48:21.548086+0000","last_became_active":"2023-11-09T03:48:00.443765+0000","last_became_peered":"2023-11-09T03:48:00.443765+0000","last_unstale":"2023-11-09T03:48:21.548086+0000","last_undegraded":"2023-11-09T03:48:21.548086+0000","last_fullsized":"2023-11-09T03:48:21.548086+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:41:22.321811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012468850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":50,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541230+0000","last_change":"2023-11-09T03:47:02.582739+0000","last_active":"2023-11-09T03:48:21.541230+0000","last_peered":"2023-11-09T03:48:21.541230+0000","last_clean":"2023-11-09T03:48:21.541230+0000","last_became_active":"2023-11-09T03:47:01.592743+0000","last_became_peered":"2023-11-09T03:47:01.592743+0000","last_unstale":"2023-11-09T03:48:21.541230+0000","last_undegraded":"2023-11-09T03:48:21.541230+0000","last_fullsized":"2023-11-09T03:48:21.541230+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051802700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.546563+0000","last_change":"2023-11-09T03:48:22.546563+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.546563+0000","last_undegraded":"2023-11-09T03:48:22.546563+0000","last_fullsized":"2023-11-09T03:48:22.546563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.561070+0000","last_change":"2023-11-09T03:48:22.559947+0000","last_active":"2023-11-09T03:48:22.561070+0000","last_peered":"2023-11-09T03:48:22.561070+0000","last_clean":"2023-11-09T03:48:22.561070+0000","last_became_active":"2023-11-09T03:48:22.559667+0000","last_became_peered":"2023-11-09T03:48:22.559667+0000","last_unstale":"2023-11-09T03:48:22.561070+0000","last_undegraded":"2023-11-09T03:48:22.561070+0000","last_fullsized":"2023-11-09T03:48:22.561070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:57.844631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540463+0000","last_change":"2023-11-09T03:48:01.442195+0000","last_active":"2023-11-09T03:48:21.540463+0000","last_peered":"2023-11-09T03:48:21.540463+0000","last_clean":"2023-11-09T03:48:21.540463+0000","last_became_active":"2023-11-09T03:48:01.441887+0000","last_became_peered":"2023-11-09T03:48:01.441887+0000","last_unstale":"2023-11-09T03:48:21.540463+0000","last_undegraded":"2023-11-09T03:48:21.540463+0000","last_fullsized":"2023-11-09T03:48:21.540463+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:06:28.575429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540528+0000","last_change":"2023-11-09T03:48:02.442061+0000","last_active":"2023-11-09T03:48:21.540528+0000","last_peered":"2023-11-09T03:48:21.540528+0000","last_clean":"2023-11-09T03:48:21.540528+0000","last_became_active":"2023-11-09T03:48:00.445921+0000","last_became_peered":"2023-11-09T03:48:00.445921+0000","last_unstale":"2023-11-09T03:48:21.540528+0000","last_undegraded":"2023-11-09T03:48:21.540528+0000","last_fullsized":"2023-11-09T03:48:21.540528+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:50:55.299235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012719770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":47,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545111+0000","last_change":"2023-11-09T03:47:02.724953+0000","last_active":"2023-11-09T03:48:21.545111+0000","last_peered":"2023-11-09T03:48:21.545111+0000","last_clean":"2023-11-09T03:48:21.545111+0000","last_became_active":"2023-11-09T03:47:01.587133+0000","last_became_peered":"2023-11-09T03:47:01.587133+0000","last_unstale":"2023-11-09T03:48:21.545111+0000","last_undegraded":"2023-11-09T03:48:21.545111+0000","last_fullsized":"2023-11-09T03:48:21.545111+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:23:48.521161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066497599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.547867+0000","last_change":"2023-11-09T03:48:22.547867+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.547867+0000","last_undegraded":"2023-11-09T03:48:22.547867+0000","last_fullsized":"2023-11-09T03:48:22.547867+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.563196+0000","last_change":"2023-11-09T03:48:22.556449+0000","last_active":"2023-11-09T03:48:22.563196+0000","last_peered":"2023-11-09T03:48:22.563196+0000","last_clean":"2023-11-09T03:48:22.563196+0000","last_became_active":"2023-11-09T03:48:22.556170+0000","last_became_peered":"2023-11-09T03:48:22.556170+0000","last_unstale":"2023-11-09T03:48:22.563196+0000","last_undegraded":"2023-11-09T03:48:22.563196+0000","last_fullsized":"2023-11-09T03:48:22.563196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:09:09.041542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539364+0000","last_change":"2023-11-09T03:48:01.450507+0000","last_active":"2023-11-09T03:48:21.539364+0000","last_peered":"2023-11-09T03:48:21.539364+0000","last_clean":"2023-11-09T03:48:21.539364+0000","last_became_active":"2023-11-09T03:48:01.450191+0000","last_became_peered":"2023-11-09T03:48:01.450191+0000","last_unstale":"2023-11-09T03:48:21.539364+0000","last_undegraded":"2023-11-09T03:48:21.539364+0000","last_fullsized":"2023-11-09T03:48:21.539364+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:32:40.342502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540558+0000","last_change":"2023-11-09T03:48:02.442120+0000","last_active":"2023-11-09T03:48:21.540558+0000","last_peered":"2023-11-09T03:48:21.540558+0000","last_clean":"2023-11-09T03:48:21.540558+0000","last_became_active":"2023-11-09T03:48:00.445504+0000","last_became_peered":"2023-11-09T03:48:00.445504+0000","last_unstale":"2023-11-09T03:48:21.540558+0000","last_undegraded":"2023-11-09T03:48:21.540558+0000","last_fullsized":"2023-11-09T03:48:21.540558+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:35:53.864627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001095766,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541998+0000","last_change":"2023-11-09T03:47:03.146217+0000","last_active":"2023-11-09T03:48:21.541998+0000","last_peered":"2023-11-09T03:48:21.541998+0000","last_clean":"2023-11-09T03:48:21.541998+0000","last_became_active":"2023-11-09T03:47:01.587598+0000","last_became_peered":"2023-11-09T03:47:01.587598+0000","last_unstale":"2023-11-09T03:48:21.541998+0000","last_undegraded":"2023-11-09T03:48:21.541998+0000","last_fullsized":"2023-11-09T03:48:21.541998+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:15:22.723455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090126400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550235+0000","last_change":"2023-11-09T03:48:22.550235+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550235+0000","last_undegraded":"2023-11-09T03:48:22.550235+0000","last_fullsized":"2023-11-09T03:48:22.550235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.564011+0000","last_change":"2023-11-09T03:48:22.561567+0000","last_active":"2023-11-09T03:48:22.564011+0000","last_peered":"2023-11-09T03:48:22.564011+0000","last_clean":"2023-11-09T03:48:22.564011+0000","last_became_active":"2023-11-09T03:48:22.561266+0000","last_became_peered":"2023-11-09T03:48:22.561266+0000","last_unstale":"2023-11-09T03:48:22.564011+0000","last_undegraded":"2023-11-09T03:48:22.564011+0000","last_fullsized":"2023-11-09T03:48:22.564011+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:21.472203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542405+0000","last_change":"2023-11-09T03:48:01.454878+0000","last_active":"2023-11-09T03:48:21.542405+0000","last_peered":"2023-11-09T03:48:21.542405+0000","last_clean":"2023-11-09T03:48:21.542405+0000","last_became_active":"2023-11-09T03:48:01.454591+0000","last_became_peered":"2023-11-09T03:48:01.454591+0000","last_unstale":"2023-11-09T03:48:21.542405+0000","last_undegraded":"2023-11-09T03:48:21.542405+0000","last_fullsized":"2023-11-09T03:48:21.542405+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:38.536898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539170+0000","last_change":"2023-11-09T03:48:03.327800+0000","last_active":"2023-11-09T03:48:21.539170+0000","last_peered":"2023-11-09T03:48:21.539170+0000","last_clean":"2023-11-09T03:48:21.539170+0000","last_became_active":"2023-11-09T03:48:00.446576+0000","last_became_peered":"2023-11-09T03:48:00.446576+0000","last_unstale":"2023-11-09T03:48:21.539170+0000","last_undegraded":"2023-11-09T03:48:21.539170+0000","last_fullsized":"2023-11-09T03:48:21.539170+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:37:08.851102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054679700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547427+0000","last_change":"2023-11-09T03:47:03.148283+0000","last_active":"2023-11-09T03:48:21.547427+0000","last_peered":"2023-11-09T03:48:21.547427+0000","last_clean":"2023-11-09T03:48:21.547427+0000","last_became_active":"2023-11-09T03:47:01.588841+0000","last_became_peered":"2023-11-09T03:47:01.588841+0000","last_unstale":"2023-11-09T03:48:21.547427+0000","last_undegraded":"2023-11-09T03:48:21.547427+0000","last_fullsized":"2023-11-09T03:48:21.547427+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:47:44.880741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000436081,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.552336+0000","last_change":"2023-11-09T03:48:22.552336+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.552336+0000","last_undegraded":"2023-11-09T03:48:22.552336+0000","last_fullsized":"2023-11-09T03:48:22.552336+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.558220+0000","last_change":"2023-11-09T03:48:22.556480+0000","last_active":"2023-11-09T03:48:22.558220+0000","last_peered":"2023-11-09T03:48:22.558220+0000","last_clean":"2023-11-09T03:48:22.558220+0000","last_became_active":"2023-11-09T03:48:22.556059+0000","last_became_peered":"2023-11-09T03:48:22.556059+0000","last_unstale":"2023-11-09T03:48:22.558220+0000","last_undegraded":"2023-11-09T03:48:22.558220+0000","last_fullsized":"2023-11-09T03:48:22.558220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:51:17.414844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.544682+0000","last_change":"2023-11-09T03:48:01.450581+0000","last_active":"2023-11-09T03:48:21.544682+0000","last_peered":"2023-11-09T03:48:21.544682+0000","last_clean":"2023-11-09T03:48:21.544682+0000","last_became_active":"2023-11-09T03:48:01.450280+0000","last_became_peered":"2023-11-09T03:48:01.450280+0000","last_unstale":"2023-11-09T03:48:21.544682+0000","last_undegraded":"2023-11-09T03:48:21.544682+0000","last_fullsized":"2023-11-09T03:48:21.544682+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:18.995634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.546874+0000","last_change":"2023-11-09T03:48:02.440934+0000","last_active":"2023-11-09T03:48:21.546874+0000","last_peered":"2023-11-09T03:48:21.546874+0000","last_clean":"2023-11-09T03:48:21.546874+0000","last_became_active":"2023-11-09T03:48:00.444394+0000","last_became_peered":"2023-11-09T03:48:00.444394+0000","last_unstale":"2023-11-09T03:48:21.546874+0000","last_undegraded":"2023-11-09T03:48:21.546874+0000","last_fullsized":"2023-11-09T03:48:21.546874+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:15:59.214955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063908300000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542547+0000","last_change":"2023-11-09T03:47:02.618856+0000","last_active":"2023-11-09T03:48:21.542547+0000","last_peered":"2023-11-09T03:48:21.542547+0000","last_clean":"2023-11-09T03:48:21.542547+0000","last_became_active":"2023-11-09T03:47:01.590430+0000","last_became_peered":"2023-11-09T03:47:01.590430+0000","last_unstale":"2023-11-09T03:48:21.542547+0000","last_undegraded":"2023-11-09T03:48:21.542547+0000","last_fullsized":"2023-11-09T03:48:21.542547+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:57:18.470305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055030200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550790+0000","last_change":"2023-11-09T03:48:22.550790+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550790+0000","last_undegraded":"2023-11-09T03:48:22.550790+0000","last_fullsized":"2023-11-09T03:48:22.550790+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.563239+0000","last_change":"2023-11-09T03:48:22.559687+0000","last_active":"2023-11-09T03:48:22.563239+0000","last_peered":"2023-11-09T03:48:22.563239+0000","last_clean":"2023-11-09T03:48:22.563239+0000","last_became_active":"2023-11-09T03:48:22.559404+0000","last_became_peered":"2023-11-09T03:48:22.559404+0000","last_unstale":"2023-11-09T03:48:22.563239+0000","last_undegraded":"2023-11-09T03:48:22.563239+0000","last_fullsized":"2023-11-09T03:48:22.563239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:39:53.462957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545586+0000","last_change":"2023-11-09T03:48:01.453600+0000","last_active":"2023-11-09T03:48:21.545586+0000","last_peered":"2023-11-09T03:48:21.545586+0000","last_clean":"2023-11-09T03:48:21.545586+0000","last_became_active":"2023-11-09T03:48:01.453330+0000","last_became_peered":"2023-11-09T03:48:01.453330+0000","last_unstale":"2023-11-09T03:48:21.545586+0000","last_undegraded":"2023-11-09T03:48:21.545586+0000","last_fullsized":"2023-11-09T03:48:21.545586+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:08:47.931939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539485+0000","last_change":"2023-11-09T03:48:03.328210+0000","last_active":"2023-11-09T03:48:21.539485+0000","last_peered":"2023-11-09T03:48:21.539485+0000","last_clean":"2023-11-09T03:48:21.539485+0000","last_became_active":"2023-11-09T03:48:00.450002+0000","last_became_peered":"2023-11-09T03:48:00.450002+0000","last_unstale":"2023-11-09T03:48:21.539485+0000","last_undegraded":"2023-11-09T03:48:21.539485+0000","last_fullsized":"2023-11-09T03:48:21.539485+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:50.391187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036521900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542921+0000","last_change":"2023-11-09T03:47:02.619144+0000","last_active":"2023-11-09T03:48:21.542921+0000","last_peered":"2023-11-09T03:48:21.542921+0000","last_clean":"2023-11-09T03:48:21.542921+0000","last_became_active":"2023-11-09T03:47:01.590672+0000","last_became_peered":"2023-11-09T03:47:01.590672+0000","last_unstale":"2023-11-09T03:48:21.542921+0000","last_undegraded":"2023-11-09T03:48:21.542921+0000","last_fullsized":"2023-11-09T03:48:21.542921+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:03.025119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060997500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.547130+0000","last_change":"2023-11-09T03:48:22.547130+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.547130+0000","last_undegraded":"2023-11-09T03:48:22.547130+0000","last_fullsized":"2023-11-09T03:48:22.547130+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.571874+0000","last_change":"2023-11-09T03:48:22.566419+0000","last_active":"2023-11-09T03:48:22.571874+0000","last_peered":"2023-11-09T03:48:22.571874+0000","last_clean":"2023-11-09T03:48:22.571874+0000","last_became_active":"2023-11-09T03:48:22.566144+0000","last_became_peered":"2023-11-09T03:48:22.566144+0000","last_unstale":"2023-11-09T03:48:22.571874+0000","last_undegraded":"2023-11-09T03:48:22.571874+0000","last_fullsized":"2023-11-09T03:48:22.571874+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:43:01.915119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541019+0000","last_change":"2023-11-09T03:48:01.447278+0000","last_active":"2023-11-09T03:48:21.541019+0000","last_peered":"2023-11-09T03:48:21.541019+0000","last_clean":"2023-11-09T03:48:21.541019+0000","last_became_active":"2023-11-09T03:48:01.446991+0000","last_became_peered":"2023-11-09T03:48:01.446991+0000","last_unstale":"2023-11-09T03:48:21.541019+0000","last_undegraded":"2023-11-09T03:48:21.541019+0000","last_fullsized":"2023-11-09T03:48:21.541019+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:27:49.190426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539429+0000","last_change":"2023-11-09T03:48:03.328278+0000","last_active":"2023-11-09T03:48:21.539429+0000","last_peered":"2023-11-09T03:48:21.539429+0000","last_clean":"2023-11-09T03:48:21.539429+0000","last_became_active":"2023-11-09T03:48:00.443213+0000","last_became_peered":"2023-11-09T03:48:00.443213+0000","last_unstale":"2023-11-09T03:48:21.539429+0000","last_undegraded":"2023-11-09T03:48:21.539429+0000","last_fullsized":"2023-11-09T03:48:21.539429+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:20:59.702297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030886100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543013+0000","last_change":"2023-11-09T03:47:02.619079+0000","last_active":"2023-11-09T03:48:21.543013+0000","last_peered":"2023-11-09T03:48:21.543013+0000","last_clean":"2023-11-09T03:48:21.543013+0000","last_became_active":"2023-11-09T03:47:01.591421+0000","last_became_peered":"2023-11-09T03:47:01.591421+0000","last_unstale":"2023-11-09T03:48:21.543013+0000","last_undegraded":"2023-11-09T03:48:21.543013+0000","last_fullsized":"2023-11-09T03:48:21.543013+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:18:58.230606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068736399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.547470+0000","last_change":"2023-11-09T03:48:22.547470+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.547470+0000","last_undegraded":"2023-11-09T03:48:22.547470+0000","last_fullsized":"2023-11-09T03:48:22.547470+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.574695+0000","last_change":"2023-11-09T03:48:22.569861+0000","last_active":"2023-11-09T03:48:22.574695+0000","last_peered":"2023-11-09T03:48:22.574695+0000","last_clean":"2023-11-09T03:48:22.574695+0000","last_became_active":"2023-11-09T03:48:22.569611+0000","last_became_peered":"2023-11-09T03:48:22.569611+0000","last_unstale":"2023-11-09T03:48:22.574695+0000","last_undegraded":"2023-11-09T03:48:22.574695+0000","last_fullsized":"2023-11-09T03:48:22.574695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:36:57.293292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541146+0000","last_change":"2023-11-09T03:48:01.449069+0000","last_active":"2023-11-09T03:48:21.541146+0000","last_peered":"2023-11-09T03:48:21.541146+0000","last_clean":"2023-11-09T03:48:21.541146+0000","last_became_active":"2023-11-09T03:48:01.448810+0000","last_became_peered":"2023-11-09T03:48:01.448810+0000","last_unstale":"2023-11-09T03:48:21.541146+0000","last_undegraded":"2023-11-09T03:48:21.541146+0000","last_fullsized":"2023-11-09T03:48:21.541146+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:56:51.158572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542867+0000","last_change":"2023-11-09T03:48:02.612067+0000","last_active":"2023-11-09T03:48:21.542867+0000","last_peered":"2023-11-09T03:48:21.542867+0000","last_clean":"2023-11-09T03:48:21.542867+0000","last_became_active":"2023-11-09T03:48:00.443296+0000","last_became_peered":"2023-11-09T03:48:00.443296+0000","last_unstale":"2023-11-09T03:48:21.542867+0000","last_undegraded":"2023-11-09T03:48:21.542867+0000","last_fullsized":"2023-11-09T03:48:21.542867+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:27:48.184420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010423330000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547943+0000","last_change":"2023-11-09T03:47:03.148529+0000","last_active":"2023-11-09T03:48:21.547943+0000","last_peered":"2023-11-09T03:48:21.547943+0000","last_clean":"2023-11-09T03:48:21.547943+0000","last_became_active":"2023-11-09T03:47:01.584148+0000","last_became_peered":"2023-11-09T03:47:01.584148+0000","last_unstale":"2023-11-09T03:48:21.547943+0000","last_undegraded":"2023-11-09T03:48:21.547943+0000","last_fullsized":"2023-11-09T03:48:21.547943+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:40:00.237248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061475600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550722+0000","last_change":"2023-11-09T03:48:22.550722+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550722+0000","last_undegraded":"2023-11-09T03:48:22.550722+0000","last_fullsized":"2023-11-09T03:48:22.550722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.563077+0000","last_change":"2023-11-09T03:48:22.558072+0000","last_active":"2023-11-09T03:48:22.563077+0000","last_peered":"2023-11-09T03:48:22.563077+0000","last_clean":"2023-11-09T03:48:22.563077+0000","last_became_active":"2023-11-09T03:48:22.557800+0000","last_became_peered":"2023-11-09T03:48:22.557800+0000","last_unstale":"2023-11-09T03:48:22.563077+0000","last_undegraded":"2023-11-09T03:48:22.563077+0000","last_fullsized":"2023-11-09T03:48:22.563077+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:00:20.811244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542948+0000","last_change":"2023-11-09T03:48:01.451750+0000","last_active":"2023-11-09T03:48:21.542948+0000","last_peered":"2023-11-09T03:48:21.542948+0000","last_clean":"2023-11-09T03:48:21.542948+0000","last_became_active":"2023-11-09T03:48:01.451393+0000","last_became_peered":"2023-11-09T03:48:01.451393+0000","last_unstale":"2023-11-09T03:48:21.542948+0000","last_undegraded":"2023-11-09T03:48:21.542948+0000","last_fullsized":"2023-11-09T03:48:21.542948+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:56:51.158572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542743+0000","last_change":"2023-11-09T03:48:02.611721+0000","last_active":"2023-11-09T03:48:21.542743+0000","last_peered":"2023-11-09T03:48:21.542743+0000","last_clean":"2023-11-09T03:48:21.542743+0000","last_became_active":"2023-11-09T03:48:00.439557+0000","last_became_peered":"2023-11-09T03:48:00.439557+0000","last_unstale":"2023-11-09T03:48:21.542743+0000","last_undegraded":"2023-11-09T03:48:21.542743+0000","last_fullsized":"2023-11-09T03:48:21.542743+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:35:53.864627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001037187,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543088+0000","last_change":"2023-11-09T03:47:03.148217+0000","last_active":"2023-11-09T03:48:21.543088+0000","last_peered":"2023-11-09T03:48:21.543088+0000","last_clean":"2023-11-09T03:48:21.543088+0000","last_became_active":"2023-11-09T03:47:01.593424+0000","last_became_peered":"2023-11-09T03:47:01.593424+0000","last_unstale":"2023-11-09T03:48:21.543088+0000","last_undegraded":"2023-11-09T03:48:21.543088+0000","last_fullsized":"2023-11-09T03:48:21.543088+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:19:32.693691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001769865,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551131+0000","last_change":"2023-11-09T03:48:22.551131+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551131+0000","last_undegraded":"2023-11-09T03:48:22.551131+0000","last_fullsized":"2023-11-09T03:48:22.551131+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.569383+0000","last_change":"2023-11-09T03:48:22.566073+0000","last_active":"2023-11-09T03:48:22.569383+0000","last_peered":"2023-11-09T03:48:22.569383+0000","last_clean":"2023-11-09T03:48:22.569383+0000","last_became_active":"2023-11-09T03:48:22.564106+0000","last_became_peered":"2023-11-09T03:48:22.564106+0000","last_unstale":"2023-11-09T03:48:22.569383+0000","last_undegraded":"2023-11-09T03:48:22.569383+0000","last_fullsized":"2023-11-09T03:48:22.569383+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:05:06.931697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547292+0000","last_change":"2023-11-09T03:48:01.447617+0000","last_active":"2023-11-09T03:48:21.547292+0000","last_peered":"2023-11-09T03:48:21.547292+0000","last_clean":"2023-11-09T03:48:21.547292+0000","last_became_active":"2023-11-09T03:48:01.447329+0000","last_became_peered":"2023-11-09T03:48:01.447329+0000","last_unstale":"2023-11-09T03:48:21.547292+0000","last_undegraded":"2023-11-09T03:48:21.547292+0000","last_fullsized":"2023-11-09T03:48:21.547292+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:54:52.433604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540889+0000","last_change":"2023-11-09T03:48:02.442427+0000","last_active":"2023-11-09T03:48:21.540889+0000","last_peered":"2023-11-09T03:48:21.540889+0000","last_clean":"2023-11-09T03:48:21.540889+0000","last_became_active":"2023-11-09T03:48:00.444856+0000","last_became_peered":"2023-11-09T03:48:00.444856+0000","last_unstale":"2023-11-09T03:48:21.540889+0000","last_undegraded":"2023-11-09T03:48:21.540889+0000","last_fullsized":"2023-11-09T03:48:21.540889+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:53:42.763434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011927730000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.546971+0000","last_change":"2023-11-09T03:47:03.146470+0000","last_active":"2023-11-09T03:48:21.546971+0000","last_peered":"2023-11-09T03:48:21.546971+0000","last_clean":"2023-11-09T03:48:21.546971+0000","last_became_active":"2023-11-09T03:47:01.587304+0000","last_became_peered":"2023-11-09T03:47:01.587304+0000","last_unstale":"2023-11-09T03:48:21.546971+0000","last_undegraded":"2023-11-09T03:48:21.546971+0000","last_fullsized":"2023-11-09T03:48:21.546971+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:58:23.339287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00110395,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.548843+0000","last_change":"2023-11-09T03:48:22.548843+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.548843+0000","last_undegraded":"2023-11-09T03:48:22.548843+0000","last_fullsized":"2023-11-09T03:48:22.548843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.564450+0000","last_change":"2023-11-09T03:48:22.561952+0000","last_active":"2023-11-09T03:48:22.564450+0000","last_peered":"2023-11-09T03:48:22.564450+0000","last_clean":"2023-11-09T03:48:22.564450+0000","last_became_active":"2023-11-09T03:48:22.561647+0000","last_became_peered":"2023-11-09T03:48:22.561647+0000","last_unstale":"2023-11-09T03:48:22.564450+0000","last_undegraded":"2023-11-09T03:48:22.564450+0000","last_fullsized":"2023-11-09T03:48:22.564450+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:52.879173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545482+0000","last_change":"2023-11-09T03:48:01.449406+0000","last_active":"2023-11-09T03:48:21.545482+0000","last_peered":"2023-11-09T03:48:21.545482+0000","last_clean":"2023-11-09T03:48:21.545482+0000","last_became_active":"2023-11-09T03:48:01.449130+0000","last_became_peered":"2023-11-09T03:48:01.449130+0000","last_unstale":"2023-11-09T03:48:21.545482+0000","last_undegraded":"2023-11-09T03:48:21.545482+0000","last_fullsized":"2023-11-09T03:48:21.545482+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:20:36.303940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547362+0000","last_change":"2023-11-09T03:48:02.441801+0000","last_active":"2023-11-09T03:48:21.547362+0000","last_peered":"2023-11-09T03:48:21.547362+0000","last_clean":"2023-11-09T03:48:21.547362+0000","last_became_active":"2023-11-09T03:48:00.444975+0000","last_became_peered":"2023-11-09T03:48:00.444975+0000","last_unstale":"2023-11-09T03:48:21.547362+0000","last_undegraded":"2023-11-09T03:48:21.547362+0000","last_fullsized":"2023-11-09T03:48:21.547362+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:39:31.347349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061165099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542743+0000","last_change":"2023-11-09T03:47:03.147367+0000","last_active":"2023-11-09T03:48:21.542743+0000","last_peered":"2023-11-09T03:48:21.542743+0000","last_clean":"2023-11-09T03:48:21.542743+0000","last_became_active":"2023-11-09T03:47:01.588359+0000","last_became_peered":"2023-11-09T03:47:01.588359+0000","last_unstale":"2023-11-09T03:48:21.542743+0000","last_undegraded":"2023-11-09T03:48:21.542743+0000","last_fullsized":"2023-11-09T03:48:21.542743+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:21:34.303068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014106660000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.546731+0000","last_change":"2023-11-09T03:48:22.546731+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.546731+0000","last_undegraded":"2023-11-09T03:48:22.546731+0000","last_fullsized":"2023-11-09T03:48:22.546731+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.563046+0000","last_change":"2023-11-09T03:48:22.556245+0000","last_active":"2023-11-09T03:48:22.563046+0000","last_peered":"2023-11-09T03:48:22.563046+0000","last_clean":"2023-11-09T03:48:22.563046+0000","last_became_active":"2023-11-09T03:48:22.555974+0000","last_became_peered":"2023-11-09T03:48:22.555974+0000","last_unstale":"2023-11-09T03:48:22.563046+0000","last_undegraded":"2023-11-09T03:48:22.563046+0000","last_fullsized":"2023-11-09T03:48:22.563046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:55:13.543207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542383+0000","last_change":"2023-11-09T03:48:01.451866+0000","last_active":"2023-11-09T03:48:21.542383+0000","last_peered":"2023-11-09T03:48:21.542383+0000","last_clean":"2023-11-09T03:48:21.542383+0000","last_became_active":"2023-11-09T03:48:01.451350+0000","last_became_peered":"2023-11-09T03:48:01.451350+0000","last_unstale":"2023-11-09T03:48:21.542383+0000","last_undegraded":"2023-11-09T03:48:21.542383+0000","last_fullsized":"2023-11-09T03:48:21.542383+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:36.735029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540171+0000","last_change":"2023-11-09T03:48:02.441648+0000","last_active":"2023-11-09T03:48:21.540171+0000","last_peered":"2023-11-09T03:48:21.540171+0000","last_clean":"2023-11-09T03:48:21.540171+0000","last_became_active":"2023-11-09T03:48:00.448541+0000","last_became_peered":"2023-11-09T03:48:00.448541+0000","last_unstale":"2023-11-09T03:48:21.540171+0000","last_undegraded":"2023-11-09T03:48:21.540171+0000","last_fullsized":"2023-11-09T03:48:21.540171+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:35.729023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539334+0000","last_change":"2023-11-09T03:47:02.581682+0000","last_active":"2023-11-09T03:48:21.539334+0000","last_peered":"2023-11-09T03:48:21.539334+0000","last_clean":"2023-11-09T03:48:21.539334+0000","last_became_active":"2023-11-09T03:47:01.593051+0000","last_became_peered":"2023-11-09T03:47:01.593051+0000","last_unstale":"2023-11-09T03:48:21.539334+0000","last_undegraded":"2023-11-09T03:48:21.539334+0000","last_fullsized":"2023-11-09T03:48:21.539334+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047277799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549020+0000","last_change":"2023-11-09T03:48:22.549020+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549020+0000","last_undegraded":"2023-11-09T03:48:22.549020+0000","last_fullsized":"2023-11-09T03:48:22.549020+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.569210+0000","last_change":"2023-11-09T03:48:22.563908+0000","last_active":"2023-11-09T03:48:22.569210+0000","last_peered":"2023-11-09T03:48:22.569210+0000","last_clean":"2023-11-09T03:48:22.569210+0000","last_became_active":"2023-11-09T03:48:22.563651+0000","last_became_peered":"2023-11-09T03:48:22.563651+0000","last_unstale":"2023-11-09T03:48:22.569210+0000","last_undegraded":"2023-11-09T03:48:22.569210+0000","last_fullsized":"2023-11-09T03:48:22.569210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:52.879173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.544927+0000","last_change":"2023-11-09T03:48:01.455784+0000","last_active":"2023-11-09T03:48:21.544927+0000","last_peered":"2023-11-09T03:48:21.544927+0000","last_clean":"2023-11-09T03:48:21.544927+0000","last_became_active":"2023-11-09T03:48:01.455532+0000","last_became_peered":"2023-11-09T03:48:01.455532+0000","last_unstale":"2023-11-09T03:48:21.544927+0000","last_undegraded":"2023-11-09T03:48:21.544927+0000","last_fullsized":"2023-11-09T03:48:21.544927+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:21:23.156761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542535+0000","last_change":"2023-11-09T03:48:02.611298+0000","last_active":"2023-11-09T03:48:21.542535+0000","last_peered":"2023-11-09T03:48:21.542535+0000","last_clean":"2023-11-09T03:48:21.542535+0000","last_became_active":"2023-11-09T03:48:00.441261+0000","last_became_peered":"2023-11-09T03:48:00.441261+0000","last_unstale":"2023-11-09T03:48:21.542535+0000","last_undegraded":"2023-11-09T03:48:21.542535+0000","last_fullsized":"2023-11-09T03:48:21.542535+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:50:55.299235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081460199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542200+0000","last_change":"2023-11-09T03:47:03.146309+0000","last_active":"2023-11-09T03:48:21.542200+0000","last_peered":"2023-11-09T03:48:21.542200+0000","last_clean":"2023-11-09T03:48:21.542200+0000","last_became_active":"2023-11-09T03:47:01.587560+0000","last_became_peered":"2023-11-09T03:47:01.587560+0000","last_unstale":"2023-11-09T03:48:21.542200+0000","last_undegraded":"2023-11-09T03:48:21.542200+0000","last_fullsized":"2023-11-09T03:48:21.542200+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:27:55.271945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070988400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551507+0000","last_change":"2023-11-09T03:48:22.551507+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551507+0000","last_undegraded":"2023-11-09T03:48:22.551507+0000","last_fullsized":"2023-11-09T03:48:22.551507+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.563543+0000","last_change":"2023-11-09T03:48:22.561557+0000","last_active":"2023-11-09T03:48:22.563543+0000","last_peered":"2023-11-09T03:48:22.563543+0000","last_clean":"2023-11-09T03:48:22.563543+0000","last_became_active":"2023-11-09T03:48:22.561249+0000","last_became_peered":"2023-11-09T03:48:22.561249+0000","last_unstale":"2023-11-09T03:48:22.563543+0000","last_undegraded":"2023-11-09T03:48:22.563543+0000","last_fullsized":"2023-11-09T03:48:22.563543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:15:26.050548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543219+0000","last_change":"2023-11-09T03:48:01.452477+0000","last_active":"2023-11-09T03:48:21.543219+0000","last_peered":"2023-11-09T03:48:21.543219+0000","last_clean":"2023-11-09T03:48:21.543219+0000","last_became_active":"2023-11-09T03:48:01.452186+0000","last_became_peered":"2023-11-09T03:48:01.452186+0000","last_unstale":"2023-11-09T03:48:21.543219+0000","last_undegraded":"2023-11-09T03:48:21.543219+0000","last_fullsized":"2023-11-09T03:48:21.543219+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:32:21.825778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547861+0000","last_change":"2023-11-09T03:48:02.442101+0000","last_active":"2023-11-09T03:48:21.547861+0000","last_peered":"2023-11-09T03:48:21.547861+0000","last_clean":"2023-11-09T03:48:21.547861+0000","last_became_active":"2023-11-09T03:48:00.449354+0000","last_became_peered":"2023-11-09T03:48:00.449354+0000","last_unstale":"2023-11-09T03:48:21.547861+0000","last_undegraded":"2023-11-09T03:48:21.547861+0000","last_fullsized":"2023-11-09T03:48:21.547861+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:06:32.906582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093455000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":47,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545438+0000","last_change":"2023-11-09T03:47:02.725125+0000","last_active":"2023-11-09T03:48:21.545438+0000","last_peered":"2023-11-09T03:48:21.545438+0000","last_clean":"2023-11-09T03:48:21.545438+0000","last_became_active":"2023-11-09T03:47:01.591521+0000","last_became_peered":"2023-11-09T03:47:01.591521+0000","last_unstale":"2023-11-09T03:48:21.545438+0000","last_undegraded":"2023-11-09T03:48:21.545438+0000","last_fullsized":"2023-11-09T03:48:21.545438+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:52:40.076542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000758675,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550819+0000","last_change":"2023-11-09T03:48:22.550819+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550819+0000","last_undegraded":"2023-11-09T03:48:22.550819+0000","last_fullsized":"2023-11-09T03:48:22.550819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.565704+0000","last_change":"2023-11-09T03:48:22.561189+0000","last_active":"2023-11-09T03:48:22.565704+0000","last_peered":"2023-11-09T03:48:22.565704+0000","last_clean":"2023-11-09T03:48:22.565704+0000","last_became_active":"2023-11-09T03:48:22.560938+0000","last_became_peered":"2023-11-09T03:48:22.560938+0000","last_unstale":"2023-11-09T03:48:22.565704+0000","last_undegraded":"2023-11-09T03:48:22.565704+0000","last_fullsized":"2023-11-09T03:48:22.565704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:05:30.764921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545418+0000","last_change":"2023-11-09T03:48:01.448129+0000","last_active":"2023-11-09T03:48:21.545418+0000","last_peered":"2023-11-09T03:48:21.545418+0000","last_clean":"2023-11-09T03:48:21.545418+0000","last_became_active":"2023-11-09T03:48:01.447844+0000","last_became_peered":"2023-11-09T03:48:01.447844+0000","last_unstale":"2023-11-09T03:48:21.545418+0000","last_undegraded":"2023-11-09T03:48:21.545418+0000","last_fullsized":"2023-11-09T03:48:21.545418+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:54:52.433604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543211+0000","last_change":"2023-11-09T03:48:02.612450+0000","last_active":"2023-11-09T03:48:21.543211+0000","last_peered":"2023-11-09T03:48:21.543211+0000","last_clean":"2023-11-09T03:48:21.543211+0000","last_became_active":"2023-11-09T03:48:00.441855+0000","last_became_peered":"2023-11-09T03:48:00.441855+0000","last_unstale":"2023-11-09T03:48:21.543211+0000","last_undegraded":"2023-11-09T03:48:21.543211+0000","last_fullsized":"2023-11-09T03:48:21.543211+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:53:42.763434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095368899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547816+0000","last_change":"2023-11-09T03:47:03.147289+0000","last_active":"2023-11-09T03:48:21.547816+0000","last_peered":"2023-11-09T03:48:21.547816+0000","last_clean":"2023-11-09T03:48:21.547816+0000","last_became_active":"2023-11-09T03:47:01.587784+0000","last_became_peered":"2023-11-09T03:47:01.587784+0000","last_unstale":"2023-11-09T03:48:21.547816+0000","last_undegraded":"2023-11-09T03:48:21.547816+0000","last_fullsized":"2023-11-09T03:48:21.547816+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:11:23.824112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038781299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.552840+0000","last_change":"2023-11-09T03:48:22.552840+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.552840+0000","last_undegraded":"2023-11-09T03:48:22.552840+0000","last_fullsized":"2023-11-09T03:48:22.552840+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.568633+0000","last_change":"2023-11-09T03:48:22.564955+0000","last_active":"2023-11-09T03:48:22.568633+0000","last_peered":"2023-11-09T03:48:22.568633+0000","last_clean":"2023-11-09T03:48:22.568633+0000","last_became_active":"2023-11-09T03:48:22.564677+0000","last_became_peered":"2023-11-09T03:48:22.564677+0000","last_unstale":"2023-11-09T03:48:22.568633+0000","last_undegraded":"2023-11-09T03:48:22.568633+0000","last_fullsized":"2023-11-09T03:48:22.568633+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:16:21.330564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541065+0000","last_change":"2023-11-09T03:48:01.449617+0000","last_active":"2023-11-09T03:48:21.541065+0000","last_peered":"2023-11-09T03:48:21.541065+0000","last_clean":"2023-11-09T03:48:21.541065+0000","last_became_active":"2023-11-09T03:48:01.449328+0000","last_became_peered":"2023-11-09T03:48:01.449328+0000","last_unstale":"2023-11-09T03:48:21.541065+0000","last_undegraded":"2023-11-09T03:48:21.541065+0000","last_fullsized":"2023-11-09T03:48:21.541065+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:51:18.760059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547661+0000","last_change":"2023-11-09T03:48:02.442215+0000","last_active":"2023-11-09T03:48:21.547661+0000","last_peered":"2023-11-09T03:48:21.547661+0000","last_clean":"2023-11-09T03:48:21.547661+0000","last_became_active":"2023-11-09T03:48:00.444720+0000","last_became_peered":"2023-11-09T03:48:00.444720+0000","last_unstale":"2023-11-09T03:48:21.547661+0000","last_undegraded":"2023-11-09T03:48:21.547661+0000","last_fullsized":"2023-11-09T03:48:21.547661+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:55:55.735668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001407176,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543390+0000","last_change":"2023-11-09T03:47:02.619346+0000","last_active":"2023-11-09T03:48:21.543390+0000","last_peered":"2023-11-09T03:48:21.543390+0000","last_clean":"2023-11-09T03:48:21.543390+0000","last_became_active":"2023-11-09T03:47:01.590389+0000","last_became_peered":"2023-11-09T03:47:01.590389+0000","last_unstale":"2023-11-09T03:48:21.543390+0000","last_undegraded":"2023-11-09T03:48:21.543390+0000","last_fullsized":"2023-11-09T03:48:21.543390+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000384072,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551555+0000","last_change":"2023-11-09T03:48:22.551555+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551555+0000","last_undegraded":"2023-11-09T03:48:22.551555+0000","last_fullsized":"2023-11-09T03:48:22.551555+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.569120+0000","last_change":"2023-11-09T03:48:22.562601+0000","last_active":"2023-11-09T03:48:22.569120+0000","last_peered":"2023-11-09T03:48:22.569120+0000","last_clean":"2023-11-09T03:48:22.569120+0000","last_became_active":"2023-11-09T03:48:22.562326+0000","last_became_peered":"2023-11-09T03:48:22.562326+0000","last_unstale":"2023-11-09T03:48:22.569120+0000","last_undegraded":"2023-11-09T03:48:22.569120+0000","last_fullsized":"2023-11-09T03:48:22.569120+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:28:06.557471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545053+0000","last_change":"2023-11-09T03:48:01.450018+0000","last_active":"2023-11-09T03:48:21.545053+0000","last_peered":"2023-11-09T03:48:21.545053+0000","last_clean":"2023-11-09T03:48:21.545053+0000","last_became_active":"2023-11-09T03:48:01.449749+0000","last_became_peered":"2023-11-09T03:48:01.449749+0000","last_unstale":"2023-11-09T03:48:21.545053+0000","last_undegraded":"2023-11-09T03:48:21.545053+0000","last_fullsized":"2023-11-09T03:48:21.545053+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:21:23.156761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542795+0000","last_change":"2023-11-09T03:48:02.524216+0000","last_active":"2023-11-09T03:48:21.542795+0000","last_peered":"2023-11-09T03:48:21.542795+0000","last_clean":"2023-11-09T03:48:21.542795+0000","last_became_active":"2023-11-09T03:48:00.446972+0000","last_became_peered":"2023-11-09T03:48:00.446972+0000","last_unstale":"2023-11-09T03:48:21.542795+0000","last_undegraded":"2023-11-09T03:48:21.542795+0000","last_fullsized":"2023-11-09T03:48:21.542795+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:54:07.987773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000825255,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":50,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540373+0000","last_change":"2023-11-09T03:47:02.582393+0000","last_active":"2023-11-09T03:48:21.540373+0000","last_peered":"2023-11-09T03:48:21.540373+0000","last_clean":"2023-11-09T03:48:21.540373+0000","last_became_active":"2023-11-09T03:47:01.591752+0000","last_became_peered":"2023-11-09T03:47:01.591752+0000","last_unstale":"2023-11-09T03:48:21.540373+0000","last_undegraded":"2023-11-09T03:48:21.540373+0000","last_fullsized":"2023-11-09T03:48:21.540373+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:03.025119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067021599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549641+0000","last_change":"2023-11-09T03:48:22.549641+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549641+0000","last_undegraded":"2023-11-09T03:48:22.549641+0000","last_fullsized":"2023-11-09T03:48:22.549641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.566064+0000","last_change":"2023-11-09T03:48:22.562752+0000","last_active":"2023-11-09T03:48:22.566064+0000","last_peered":"2023-11-09T03:48:22.566064+0000","last_clean":"2023-11-09T03:48:22.566064+0000","last_became_active":"2023-11-09T03:48:22.562474+0000","last_became_peered":"2023-11-09T03:48:22.562474+0000","last_unstale":"2023-11-09T03:48:22.566064+0000","last_undegraded":"2023-11-09T03:48:22.566064+0000","last_fullsized":"2023-11-09T03:48:22.566064+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:35:27.804836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542868+0000","last_change":"2023-11-09T03:48:01.451462+0000","last_active":"2023-11-09T03:48:21.542868+0000","last_peered":"2023-11-09T03:48:21.542868+0000","last_clean":"2023-11-09T03:48:21.542868+0000","last_became_active":"2023-11-09T03:48:01.451188+0000","last_became_peered":"2023-11-09T03:48:01.451188+0000","last_unstale":"2023-11-09T03:48:21.542868+0000","last_undegraded":"2023-11-09T03:48:21.542868+0000","last_fullsized":"2023-11-09T03:48:21.542868+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:28.631801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542221+0000","last_change":"2023-11-09T03:48:02.682290+0000","last_active":"2023-11-09T03:48:21.542221+0000","last_peered":"2023-11-09T03:48:21.542221+0000","last_clean":"2023-11-09T03:48:21.542221+0000","last_became_active":"2023-11-09T03:48:00.444246+0000","last_became_peered":"2023-11-09T03:48:00.444246+0000","last_unstale":"2023-11-09T03:48:21.542221+0000","last_undegraded":"2023-11-09T03:48:21.542221+0000","last_fullsized":"2023-11-09T03:48:21.542221+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:09.487058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000423359,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.544622+0000","last_change":"2023-11-09T03:47:03.028744+0000","last_active":"2023-11-09T03:48:21.544622+0000","last_peered":"2023-11-09T03:48:21.544622+0000","last_clean":"2023-11-09T03:48:21.544622+0000","last_became_active":"2023-11-09T03:47:01.593890+0000","last_became_peered":"2023-11-09T03:47:01.593890+0000","last_unstale":"2023-11-09T03:48:21.544622+0000","last_undegraded":"2023-11-09T03:48:21.544622+0000","last_fullsized":"2023-11-09T03:48:21.544622+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:05.754357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053008299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550087+0000","last_change":"2023-11-09T03:48:22.550087+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550087+0000","last_undegraded":"2023-11-09T03:48:22.550087+0000","last_fullsized":"2023-11-09T03:48:22.550087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.566105+0000","last_change":"2023-11-09T03:48:22.560645+0000","last_active":"2023-11-09T03:48:22.566105+0000","last_peered":"2023-11-09T03:48:22.566105+0000","last_clean":"2023-11-09T03:48:22.566105+0000","last_became_active":"2023-11-09T03:48:22.560328+0000","last_became_peered":"2023-11-09T03:48:22.560328+0000","last_unstale":"2023-11-09T03:48:22.566105+0000","last_undegraded":"2023-11-09T03:48:22.566105+0000","last_fullsized":"2023-11-09T03:48:22.566105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:59.646501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.544881+0000","last_change":"2023-11-09T03:48:01.450144+0000","last_active":"2023-11-09T03:48:21.544881+0000","last_peered":"2023-11-09T03:48:21.544881+0000","last_clean":"2023-11-09T03:48:21.544881+0000","last_became_active":"2023-11-09T03:48:01.449859+0000","last_became_peered":"2023-11-09T03:48:01.449859+0000","last_unstale":"2023-11-09T03:48:21.544881+0000","last_undegraded":"2023-11-09T03:48:21.544881+0000","last_fullsized":"2023-11-09T03:48:21.544881+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:57:56.855869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542191+0000","last_change":"2023-11-09T03:48:02.681869+0000","last_active":"2023-11-09T03:48:21.542191+0000","last_peered":"2023-11-09T03:48:21.542191+0000","last_clean":"2023-11-09T03:48:21.542191+0000","last_became_active":"2023-11-09T03:48:00.444928+0000","last_became_peered":"2023-11-09T03:48:00.444928+0000","last_unstale":"2023-11-09T03:48:21.542191+0000","last_undegraded":"2023-11-09T03:48:21.542191+0000","last_fullsized":"2023-11-09T03:48:21.542191+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:59:58.695636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044822599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542292+0000","last_change":"2023-11-09T03:47:03.146783+0000","last_active":"2023-11-09T03:48:21.542292+0000","last_peered":"2023-11-09T03:48:21.542292+0000","last_clean":"2023-11-09T03:48:21.542292+0000","last_became_active":"2023-11-09T03:47:01.586923+0000","last_became_peered":"2023-11-09T03:47:01.586923+0000","last_unstale":"2023-11-09T03:48:21.542292+0000","last_undegraded":"2023-11-09T03:48:21.542292+0000","last_fullsized":"2023-11-09T03:48:21.542292+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:37:50.326442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060384700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549765+0000","last_change":"2023-11-09T03:48:22.549765+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549765+0000","last_undegraded":"2023-11-09T03:48:22.549765+0000","last_fullsized":"2023-11-09T03:48:22.549765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.571830+0000","last_change":"2023-11-09T03:48:22.565142+0000","last_active":"2023-11-09T03:48:22.571830+0000","last_peered":"2023-11-09T03:48:22.571830+0000","last_clean":"2023-11-09T03:48:22.571830+0000","last_became_active":"2023-11-09T03:48:22.564856+0000","last_became_peered":"2023-11-09T03:48:22.564856+0000","last_unstale":"2023-11-09T03:48:22.571830+0000","last_undegraded":"2023-11-09T03:48:22.571830+0000","last_fullsized":"2023-11-09T03:48:22.571830+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:25:04.111498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542575+0000","last_change":"2023-11-09T03:48:01.448383+0000","last_active":"2023-11-09T03:48:21.542575+0000","last_peered":"2023-11-09T03:48:21.542575+0000","last_clean":"2023-11-09T03:48:21.542575+0000","last_became_active":"2023-11-09T03:48:01.448050+0000","last_became_peered":"2023-11-09T03:48:01.448050+0000","last_unstale":"2023-11-09T03:48:21.542575+0000","last_undegraded":"2023-11-09T03:48:21.542575+0000","last_fullsized":"2023-11-09T03:48:21.542575+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:53:44.478182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541748+0000","last_change":"2023-11-09T03:48:02.523794+0000","last_active":"2023-11-09T03:48:21.541748+0000","last_peered":"2023-11-09T03:48:21.541748+0000","last_clean":"2023-11-09T03:48:21.541748+0000","last_became_active":"2023-11-09T03:48:00.440255+0000","last_became_peered":"2023-11-09T03:48:00.440255+0000","last_unstale":"2023-11-09T03:48:21.541748+0000","last_undegraded":"2023-11-09T03:48:21.541748+0000","last_fullsized":"2023-11-09T03:48:21.541748+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:58:02.392696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064572299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542252+0000","last_change":"2023-11-09T03:47:03.146580+0000","last_active":"2023-11-09T03:48:21.542252+0000","last_peered":"2023-11-09T03:48:21.542252+0000","last_clean":"2023-11-09T03:48:21.542252+0000","last_became_active":"2023-11-09T03:47:01.589201+0000","last_became_peered":"2023-11-09T03:47:01.589201+0000","last_unstale":"2023-11-09T03:48:21.542252+0000","last_undegraded":"2023-11-09T03:48:21.542252+0000","last_fullsized":"2023-11-09T03:48:21.542252+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:14.300421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049706999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551049+0000","last_change":"2023-11-09T03:48:22.551049+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551049+0000","last_undegraded":"2023-11-09T03:48:22.551049+0000","last_fullsized":"2023-11-09T03:48:22.551049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.566108+0000","last_change":"2023-11-09T03:48:22.562609+0000","last_active":"2023-11-09T03:48:22.566108+0000","last_peered":"2023-11-09T03:48:22.566108+0000","last_clean":"2023-11-09T03:48:22.566108+0000","last_became_active":"2023-11-09T03:48:22.562328+0000","last_became_peered":"2023-11-09T03:48:22.562328+0000","last_unstale":"2023-11-09T03:48:22.566108+0000","last_undegraded":"2023-11-09T03:48:22.566108+0000","last_fullsized":"2023-11-09T03:48:22.566108+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:57:25.589693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539239+0000","last_change":"2023-11-09T03:48:01.451284+0000","last_active":"2023-11-09T03:48:21.539239+0000","last_peered":"2023-11-09T03:48:21.539239+0000","last_clean":"2023-11-09T03:48:21.539239+0000","last_became_active":"2023-11-09T03:48:01.450954+0000","last_became_peered":"2023-11-09T03:48:01.450954+0000","last_unstale":"2023-11-09T03:48:21.539239+0000","last_undegraded":"2023-11-09T03:48:21.539239+0000","last_fullsized":"2023-11-09T03:48:21.539239+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:49:24.680020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545344+0000","last_change":"2023-11-09T03:48:02.733038+0000","last_active":"2023-11-09T03:48:21.545344+0000","last_peered":"2023-11-09T03:48:21.545344+0000","last_clean":"2023-11-09T03:48:21.545344+0000","last_became_active":"2023-11-09T03:48:00.446661+0000","last_became_peered":"2023-11-09T03:48:00.446661+0000","last_unstale":"2023-11-09T03:48:21.545344+0000","last_undegraded":"2023-11-09T03:48:21.545344+0000","last_fullsized":"2023-11-09T03:48:21.545344+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:05:08.649312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040102300000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547989+0000","last_change":"2023-11-09T03:47:03.147851+0000","last_active":"2023-11-09T03:48:21.547989+0000","last_peered":"2023-11-09T03:48:21.547989+0000","last_clean":"2023-11-09T03:48:21.547989+0000","last_became_active":"2023-11-09T03:47:01.589893+0000","last_became_peered":"2023-11-09T03:47:01.589893+0000","last_unstale":"2023-11-09T03:48:21.547989+0000","last_undegraded":"2023-11-09T03:48:21.547989+0000","last_fullsized":"2023-11-09T03:48:21.547989+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:08:17.183462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029151399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":215,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545358+0000","last_change":"2023-11-09T03:45:21.030926+0000","last_active":"2023-11-09T03:48:21.545358+0000","last_peered":"2023-11-09T03:48:21.545358+0000","last_clean":"2023-11-09T03:48:21.545358+0000","last_became_active":"2023-11-09T03:45:21.029801+0000","last_became_peered":"2023-11-09T03:45:21.029801+0000","last_unstale":"2023-11-09T03:48:21.545358+0000","last_undegraded":"2023-11-09T03:48:21.545358+0000","last_fullsized":"2023-11-09T03:48:21.545358+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_clean_scrub_stamp":"2023-11-09T03:45:19.768433+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:12:59.153713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1572864,"data_stored":1058844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.501,"15min":0.498},"min":{"1min":0.34899999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.41199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.59899999999999998}]},{"osd":1,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.33000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.38100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.57799999999999996}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.52300000000000002,"15min":0.51400000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.68700000000000006,"15min":0.70799999999999996},"min":{"1min":0.41299999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.80700000000000005,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.628}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.48699999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61599999999999999}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.441},"min":{"1min":0.34100000000000003,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.49199999999999999,"15min":0.47899999999999998},"min":{"1min":0.38700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.42799999999999999}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.47199999999999998,"15min":0.45600000000000002},"min":{"1min":0.37,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.51100000000000001,"15min":0.497},"min":{"1min":0.39600000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.54800000000000004}]},{"osd":6,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55100000000000005,"15min":0.54100000000000004},"min":{"1min":0.44400000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.496},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.434,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.48099999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":991232,"data_stored":476134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.499,"15min":0.501},"min":{"1min":0.35099999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63900000000000001,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.50600000000000001,"15min":0.495},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.51100000000000001}]},{"osd":1,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.375,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.58299999999999996}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.503,"15min":0.503},"min":{"1min":0.316,"5min":0.313,"15min":0.313},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.437},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.45700000000000002}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.436,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55500000000000005,"15min":0.55800000000000005},"min":{"1min":0.34300000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.42899999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.44400000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.56100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":978944,"data_stored":468476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.504,"15min":0.48899999999999999},"min":{"1min":0.36199999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.503},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.376,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.61299999999999999}]},{"osd":1,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88300000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.36599999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.84199999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.65900000000000003}]},{"osd":2,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.45800000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.55600000000000005},"min":{"1min":0.40300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.71099999999999997}]},{"osd":3,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57099999999999995,"15min":0.56499999999999995},"min":{"1min":0.39400000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55200000000000005,"15min":0.55100000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.76100000000000001}]},{"osd":4,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.49199999999999999,"15min":0.48399999999999999},"min":{"1min":0.32400000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.48999999999999999,"15min":0.48299999999999998},"min":{"1min":0.29699999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.57899999999999996}]},{"osd":6,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.50600000000000001,"15min":0.51000000000000001},"min":{"1min":0.33700000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.755,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.53300000000000003,"15min":0.51700000000000002},"min":{"1min":0.38300000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.64400000000000002}]},{"osd":7,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.34699999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.81200000000000006,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.61899999999999999,"15min":0.62},"min":{"1min":0.432,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84899999999999998,"5min":1.01,"15min":1.01},"last":0.79000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":978944,"data_stored":468476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69399999999999995,"5min":1.034,"15min":1.034},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.38800000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78300000000000003,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.34000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.74299999999999999,"5min":1.093,"15min":1.093},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54200000000000004,"15min":0.54100000000000004},"min":{"1min":0.36399999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71899999999999997,"5min":0.871,"15min":0.871},"last":0.54900000000000004}]},{"osd":2,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54100000000000004,"15min":0.55400000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.69199999999999995,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54800000000000004,"15min":0.54500000000000004},"min":{"1min":0.39700000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.51800000000000002}]},{"osd":3,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.433,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.70199999999999996,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.376,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72499999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.379}]},{"osd":5,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.33300000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.78700000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.318,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81100000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.35799999999999998}]},{"osd":6,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.75800000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57899999999999996,"15min":0.58399999999999996},"min":{"1min":0.39600000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.75900000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.39600000000000002}]},{"osd":7,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61599999999999999,"15min":0.629},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83499999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61099999999999999,"15min":0.61899999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79300000000000004,"5min":1.113,"15min":1.113},"last":0.45500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":978944,"data_stored":468476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.32100000000000001,"15min":0.32300000000000001},"min":{"1min":0.22,"5min":0.215,"15min":0.215},"max":{"1min":0.40200000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.313,"5min":0.32300000000000001,"15min":0.32400000000000001},"min":{"1min":0.23999999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.47799999999999998,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.23999999999999999}]},{"osd":1,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.32600000000000001,"15min":0.33000000000000002},"min":{"1min":0.23599999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.42899999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.36699999999999999,"15min":0.373},"min":{"1min":0.28100000000000003,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.47899999999999998,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.36599999999999999}]},{"osd":2,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.37,"15min":0.372},"min":{"1min":0.27200000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.439,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.312},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.379,"15min":0.38300000000000001},"min":{"1min":0.29799999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.45700000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.38400000000000001}]},{"osd":4,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58899999999999997,"15min":0.59499999999999997},"min":{"1min":0.42299999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79000000000000004,"5min":0.87,"15min":0.87},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.59699999999999998,"15min":0.61099999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.434}]},{"osd":5,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.60499999999999998,"15min":0.62},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86599999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60699999999999998,"15min":0.61899999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.82099999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.443}]},{"osd":6,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60999999999999999,"15min":0.623},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78200000000000003,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.45400000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.80300000000000005,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.55200000000000005}]},{"osd":7,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.61099999999999999,"15min":0.62},"min":{"1min":0.41099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.70899999999999996,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.621,"15min":0.63700000000000001},"min":{"1min":0.46100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81299999999999994,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.46100000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":991232,"data_stored":476134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.46000000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.33400000000000002,"15min":0.33100000000000002},"min":{"1min":0.247,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.29399999999999998}]},{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32500000000000001,"15min":0.32600000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.495,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.35899999999999999,"15min":0.36299999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.48599999999999999,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.36699999999999999}]},{"osd":3,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.36199999999999999,"15min":0.36399999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.45700000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.312},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40200000000000002,"15min":0.40400000000000003},"min":{"1min":0.30299999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.52300000000000002,"5min":0.754,"15min":0.754},"last":0.39900000000000002}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72299999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81299999999999994,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.46000000000000002}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.39800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.75700000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.59599999999999997,"15min":0.60099999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.753,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.498}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60399999999999998,"15min":0.60899999999999999},"min":{"1min":0.44600000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.76100000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.71199999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60699999999999998,"15min":0.61199999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.745,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.80000000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.47299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":978944,"data_stored":468476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.28699999999999998,"5min":0.31,"15min":0.314},"min":{"1min":0.20799999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.38900000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"last":0.309},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.315,"15min":0.311},"min":{"1min":0.24099999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.29299999999999998}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.311,"15min":0.309},"min":{"1min":0.253,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.378,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.34999999999999998}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36899999999999999,"15min":0.371},"min":{"1min":0.23899999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.47699999999999998,"5min":0.505,"15min":0.505},"last":0.379},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.37,"15min":0.371},"min":{"1min":0.30299999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.45400000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.41899999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.747,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.44400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.77600000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.58699999999999997}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56200000000000006,"15min":0.55700000000000005},"min":{"1min":0.46100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.44500000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.72599999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.47799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56699999999999995,"15min":0.56799999999999995},"min":{"1min":0.433,"5min":0.379,"15min":0.379},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.53300000000000003}]},{"osd":7,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.42899999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.69799999999999995,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.72899999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.54700000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1568768,"data_stored":1058225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.31900000000000001,"15min":0.32300000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.38200000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.29599999999999999,"5min":0.35499999999999998,"15min":0.36499999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.44500000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.27300000000000002}]},{"osd":2,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.35699999999999998,"15min":0.36399999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.44700000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.376},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.36699999999999999,"15min":0.371},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.47699999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.316}]},{"osd":3,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.38500000000000001,"15min":0.39100000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.46400000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.41499999999999998,"15min":0.42099999999999999},"min":{"1min":0.309,"5min":0.307,"15min":0.307},"max":{"1min":0.48999999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58899999999999997,"15min":0.59299999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60599999999999998,"15min":0.60899999999999999},"min":{"1min":0.436,"5min":0.318,"15min":0.318},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.60299999999999998}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.36199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76800000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.629,"15min":0.63600000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78900000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.64100000000000001,"15min":0.64700000000000002},"min":{"1min":0.42499999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81000000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.627,"15min":0.63200000000000001},"min":{"1min":0.41699999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.44400000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.42999999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.71799999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.55600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-09T03:48:27.199 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3f is not active+clean 2023-11-09T03:48:27.199 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.549889+0000', 'last_change': '2023-11-09T03:48:22.549889+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.549889+0000', 'last_undegraded': '2023-11-09T03:48:22.549889+0000', 'last_fullsized': '2023-11-09T03:48:22.549889+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-09T03:48:27.200 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3e is not active+clean 2023-11-09T03:48:27.200 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.554480+0000', 'last_change': '2023-11-09T03:48:22.554480+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.554480+0000', 'last_undegraded': '2023-11-09T03:48:22.554480+0000', 'last_fullsized': '2023-11-09T03:48:22.554480+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-09T03:48:27.201 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3d is not active+clean 2023-11-09T03:48:27.201 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.549408+0000', 'last_change': '2023-11-09T03:48:22.549408+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.549408+0000', 'last_undegraded': '2023-11-09T03:48:22.549408+0000', 'last_fullsized': '2023-11-09T03:48:22.549408+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-09T03:48:27.201 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3c is not active+clean 2023-11-09T03:48:27.202 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.550714+0000', 'last_change': '2023-11-09T03:48:22.550714+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.550714+0000', 'last_undegraded': '2023-11-09T03:48:22.550714+0000', 'last_fullsized': '2023-11-09T03:48:22.550714+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-09T03:48:27.202 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3b is not active+clean 2023-11-09T03:48:27.202 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.550039+0000', 'last_change': '2023-11-09T03:48:22.550039+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.550039+0000', 'last_undegraded': '2023-11-09T03:48:22.550039+0000', 'last_fullsized': '2023-11-09T03:48:22.550039+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-09T03:48:27.203 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3a is not active+clean 2023-11-09T03:48:27.203 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.549702+0000', 'last_change': '2023-11-09T03:48:22.549702+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.549702+0000', 'last_undegraded': '2023-11-09T03:48:22.549702+0000', 'last_fullsized': '2023-11-09T03:48:22.549702+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-09T03:48:27.203 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.39 is not active+clean 2023-11-09T03:48:27.204 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.550846+0000', 'last_change': '2023-11-09T03:48:22.550846+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.550846+0000', 'last_undegraded': '2023-11-09T03:48:22.550846+0000', 'last_fullsized': '2023-11-09T03:48:22.550846+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-09T03:48:27.204 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.38 is not active+clean 2023-11-09T03:48:27.205 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.554400+0000', 'last_change': '2023-11-09T03:48:22.554400+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.554400+0000', 'last_undegraded': '2023-11-09T03:48:22.554400+0000', 'last_fullsized': '2023-11-09T03:48:22.554400+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-09T03:48:27.205 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.37 is not active+clean 2023-11-09T03:48:27.205 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.547291+0000', 'last_change': '2023-11-09T03:48:22.547291+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.547291+0000', 'last_undegraded': '2023-11-09T03:48:22.547291+0000', 'last_fullsized': '2023-11-09T03:48:22.547291+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-09T03:48:27.206 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.36 is not active+clean 2023-11-09T03:48:27.206 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.551875+0000', 'last_change': '2023-11-09T03:48:22.551875+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.551875+0000', 'last_undegraded': '2023-11-09T03:48:22.551875+0000', 'last_fullsized': '2023-11-09T03:48:22.551875+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-09T03:48:27.206 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.35 is not active+clean 2023-11-09T03:48:27.207 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.554690+0000', 'last_change': '2023-11-09T03:48:22.554690+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.554690+0000', 'last_undegraded': '2023-11-09T03:48:22.554690+0000', 'last_fullsized': '2023-11-09T03:48:22.554690+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-09T03:48:27.207 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.34 is not active+clean 2023-11-09T03:48:27.207 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.547393+0000', 'last_change': '2023-11-09T03:48:22.547393+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.547393+0000', 'last_undegraded': '2023-11-09T03:48:22.547393+0000', 'last_fullsized': '2023-11-09T03:48:22.547393+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-09T03:48:27.208 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.33 is not active+clean 2023-11-09T03:48:27.208 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.550250+0000', 'last_change': '2023-11-09T03:48:22.550250+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.550250+0000', 'last_undegraded': '2023-11-09T03:48:22.550250+0000', 'last_fullsized': '2023-11-09T03:48:22.550250+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-09T03:48:27.208 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.32 is not active+clean 2023-11-09T03:48:27.209 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.558130+0000', 'last_change': '2023-11-09T03:48:22.558130+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.558130+0000', 'last_undegraded': '2023-11-09T03:48:22.558130+0000', 'last_fullsized': '2023-11-09T03:48:22.558130+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-09T03:48:27.209 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.31 is not active+clean 2023-11-09T03:48:27.209 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.549957+0000', 'last_change': '2023-11-09T03:48:22.549957+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.549957+0000', 'last_undegraded': '2023-11-09T03:48:22.549957+0000', 'last_fullsized': '2023-11-09T03:48:22.549957+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-09T03:48:27.210 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.30 is not active+clean 2023-11-09T03:48:27.210 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.553338+0000', 'last_change': '2023-11-09T03:48:22.553338+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.553338+0000', 'last_undegraded': '2023-11-09T03:48:22.553338+0000', 'last_fullsized': '2023-11-09T03:48:22.553338+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-09T03:48:27.211 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2f is not active+clean 2023-11-09T03:48:27.211 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.554384+0000', 'last_change': '2023-11-09T03:48:22.554384+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.554384+0000', 'last_undegraded': '2023-11-09T03:48:22.554384+0000', 'last_fullsized': '2023-11-09T03:48:22.554384+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-09T03:48:27.211 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2e is not active+clean 2023-11-09T03:48:27.212 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.550716+0000', 'last_change': '2023-11-09T03:48:22.550716+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.550716+0000', 'last_undegraded': '2023-11-09T03:48:22.550716+0000', 'last_fullsized': '2023-11-09T03:48:22.550716+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-09T03:48:27.212 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2d is not active+clean 2023-11-09T03:48:27.212 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.557699+0000', 'last_change': '2023-11-09T03:48:22.557699+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.557699+0000', 'last_undegraded': '2023-11-09T03:48:22.557699+0000', 'last_fullsized': '2023-11-09T03:48:22.557699+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-09T03:48:27.213 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2c is not active+clean 2023-11-09T03:48:27.213 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.546650+0000', 'last_change': '2023-11-09T03:48:22.546650+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.546650+0000', 'last_undegraded': '2023-11-09T03:48:22.546650+0000', 'last_fullsized': '2023-11-09T03:48:22.546650+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-09T03:48:27.213 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2b is not active+clean 2023-11-09T03:48:27.214 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.555079+0000', 'last_change': '2023-11-09T03:48:22.555079+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.555079+0000', 'last_undegraded': '2023-11-09T03:48:22.555079+0000', 'last_fullsized': '2023-11-09T03:48:22.555079+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-09T03:48:27.214 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2a is not active+clean 2023-11-09T03:48:27.214 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.554040+0000', 'last_change': '2023-11-09T03:48:22.554040+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.554040+0000', 'last_undegraded': '2023-11-09T03:48:22.554040+0000', 'last_fullsized': '2023-11-09T03:48:22.554040+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-09T03:48:27.215 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.29 is not active+clean 2023-11-09T03:48:27.215 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.553445+0000', 'last_change': '2023-11-09T03:48:22.553445+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.553445+0000', 'last_undegraded': '2023-11-09T03:48:22.553445+0000', 'last_fullsized': '2023-11-09T03:48:22.553445+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-09T03:48:27.216 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.28 is not active+clean 2023-11-09T03:48:27.216 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.551333+0000', 'last_change': '2023-11-09T03:48:22.551333+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.551333+0000', 'last_undegraded': '2023-11-09T03:48:22.551333+0000', 'last_fullsized': '2023-11-09T03:48:22.551333+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-09T03:48:27.216 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.27 is not active+clean 2023-11-09T03:48:27.217 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.553219+0000', 'last_change': '2023-11-09T03:48:22.553219+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.553219+0000', 'last_undegraded': '2023-11-09T03:48:22.553219+0000', 'last_fullsized': '2023-11-09T03:48:22.553219+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-09T03:48:27.217 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.26 is not active+clean 2023-11-09T03:48:27.217 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.552812+0000', 'last_change': '2023-11-09T03:48:22.552812+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.552812+0000', 'last_undegraded': '2023-11-09T03:48:22.552812+0000', 'last_fullsized': '2023-11-09T03:48:22.552812+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-09T03:48:27.218 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.25 is not active+clean 2023-11-09T03:48:27.218 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.552951+0000', 'last_change': '2023-11-09T03:48:22.552951+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.552951+0000', 'last_undegraded': '2023-11-09T03:48:22.552951+0000', 'last_fullsized': '2023-11-09T03:48:22.552951+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-09T03:48:27.218 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.24 is not active+clean 2023-11-09T03:48:27.219 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.552510+0000', 'last_change': '2023-11-09T03:48:22.552510+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.552510+0000', 'last_undegraded': '2023-11-09T03:48:22.552510+0000', 'last_fullsized': '2023-11-09T03:48:22.552510+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-09T03:48:27.219 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.23 is not active+clean 2023-11-09T03:48:27.219 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.552644+0000', 'last_change': '2023-11-09T03:48:22.552644+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.552644+0000', 'last_undegraded': '2023-11-09T03:48:22.552644+0000', 'last_fullsized': '2023-11-09T03:48:22.552644+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-09T03:48:27.220 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.22 is not active+clean 2023-11-09T03:48:27.220 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.551900+0000', 'last_change': '2023-11-09T03:48:22.551900+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.551900+0000', 'last_undegraded': '2023-11-09T03:48:22.551900+0000', 'last_fullsized': '2023-11-09T03:48:22.551900+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-09T03:48:27.220 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.21 is not active+clean 2023-11-09T03:48:27.221 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.549131+0000', 'last_change': '2023-11-09T03:48:22.549131+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.549131+0000', 'last_undegraded': '2023-11-09T03:48:22.549131+0000', 'last_fullsized': '2023-11-09T03:48:22.549131+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-09T03:48:27.221 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.20 is not active+clean 2023-11-09T03:48:27.221 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.548919+0000', 'last_change': '2023-11-09T03:48:22.548919+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.548919+0000', 'last_undegraded': '2023-11-09T03:48:22.548919+0000', 'last_fullsized': '2023-11-09T03:48:22.548919+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-09T03:48:27.222 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.15 is not active+clean 2023-11-09T03:48:27.222 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.554246+0000', 'last_change': '2023-11-09T03:48:22.554246+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.554246+0000', 'last_undegraded': '2023-11-09T03:48:22.554246+0000', 'last_fullsized': '2023-11-09T03:48:22.554246+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-09T03:48:27.223 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.14 is not active+clean 2023-11-09T03:48:27.223 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.553249+0000', 'last_change': '2023-11-09T03:48:22.553249+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.553249+0000', 'last_undegraded': '2023-11-09T03:48:22.553249+0000', 'last_fullsized': '2023-11-09T03:48:22.553249+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-09T03:48:27.223 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.17 is not active+clean 2023-11-09T03:48:27.224 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.546617+0000', 'last_change': '2023-11-09T03:48:22.546617+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.546617+0000', 'last_undegraded': '2023-11-09T03:48:22.546617+0000', 'last_fullsized': '2023-11-09T03:48:22.546617+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-09T03:48:27.224 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.16 is not active+clean 2023-11-09T03:48:27.224 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.551692+0000', 'last_change': '2023-11-09T03:48:22.551692+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.551692+0000', 'last_undegraded': '2023-11-09T03:48:22.551692+0000', 'last_fullsized': '2023-11-09T03:48:22.551692+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-09T03:48:27.225 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.11 is not active+clean 2023-11-09T03:48:27.225 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.547531+0000', 'last_change': '2023-11-09T03:48:22.547531+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.547531+0000', 'last_undegraded': '2023-11-09T03:48:22.547531+0000', 'last_fullsized': '2023-11-09T03:48:22.547531+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-09T03:48:27.225 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.10 is not active+clean 2023-11-09T03:48:27.226 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.546525+0000', 'last_change': '2023-11-09T03:48:22.546525+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.546525+0000', 'last_undegraded': '2023-11-09T03:48:22.546525+0000', 'last_fullsized': '2023-11-09T03:48:22.546525+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-09T03:48:27.226 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.13 is not active+clean 2023-11-09T03:48:27.227 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.547973+0000', 'last_change': '2023-11-09T03:48:22.547973+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.547973+0000', 'last_undegraded': '2023-11-09T03:48:22.547973+0000', 'last_fullsized': '2023-11-09T03:48:22.547973+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-09T03:48:27.227 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.12 is not active+clean 2023-11-09T03:48:27.227 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.551090+0000', 'last_change': '2023-11-09T03:48:22.551090+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.551090+0000', 'last_undegraded': '2023-11-09T03:48:22.551090+0000', 'last_fullsized': '2023-11-09T03:48:22.551090+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-09T03:48:27.228 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1d is not active+clean 2023-11-09T03:48:27.228 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.546539+0000', 'last_change': '2023-11-09T03:48:22.546539+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.546539+0000', 'last_undegraded': '2023-11-09T03:48:22.546539+0000', 'last_fullsized': '2023-11-09T03:48:22.546539+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-09T03:48:27.228 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1c is not active+clean 2023-11-09T03:48:27.229 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.548469+0000', 'last_change': '2023-11-09T03:48:22.548469+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.548469+0000', 'last_undegraded': '2023-11-09T03:48:22.548469+0000', 'last_fullsized': '2023-11-09T03:48:22.548469+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-09T03:48:27.229 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1f is not active+clean 2023-11-09T03:48:27.229 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.548328+0000', 'last_change': '2023-11-09T03:48:22.548328+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.548328+0000', 'last_undegraded': '2023-11-09T03:48:22.548328+0000', 'last_fullsized': '2023-11-09T03:48:22.548328+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-09T03:48:27.230 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1e is not active+clean 2023-11-09T03:48:27.230 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.551472+0000', 'last_change': '2023-11-09T03:48:22.551472+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.551472+0000', 'last_undegraded': '2023-11-09T03:48:22.551472+0000', 'last_fullsized': '2023-11-09T03:48:22.551472+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-09T03:48:27.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.19 is not active+clean 2023-11-09T03:48:27.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.546563+0000', 'last_change': '2023-11-09T03:48:22.546563+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.546563+0000', 'last_undegraded': '2023-11-09T03:48:22.546563+0000', 'last_fullsized': '2023-11-09T03:48:22.546563+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-09T03:48:27.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.18 is not active+clean 2023-11-09T03:48:27.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.547867+0000', 'last_change': '2023-11-09T03:48:22.547867+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.547867+0000', 'last_undegraded': '2023-11-09T03:48:22.547867+0000', 'last_fullsized': '2023-11-09T03:48:22.547867+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-09T03:48:27.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1b is not active+clean 2023-11-09T03:48:27.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.550235+0000', 'last_change': '2023-11-09T03:48:22.550235+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.550235+0000', 'last_undegraded': '2023-11-09T03:48:22.550235+0000', 'last_fullsized': '2023-11-09T03:48:22.550235+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-09T03:48:27.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1a is not active+clean 2023-11-09T03:48:27.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.552336+0000', 'last_change': '2023-11-09T03:48:22.552336+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.552336+0000', 'last_undegraded': '2023-11-09T03:48:22.552336+0000', 'last_fullsized': '2023-11-09T03:48:22.552336+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-09T03:48:27.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.5 is not active+clean 2023-11-09T03:48:27.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.550790+0000', 'last_change': '2023-11-09T03:48:22.550790+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.550790+0000', 'last_undegraded': '2023-11-09T03:48:22.550790+0000', 'last_fullsized': '2023-11-09T03:48:22.550790+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-09T03:48:27.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.4 is not active+clean 2023-11-09T03:48:27.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.547130+0000', 'last_change': '2023-11-09T03:48:22.547130+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.547130+0000', 'last_undegraded': '2023-11-09T03:48:22.547130+0000', 'last_fullsized': '2023-11-09T03:48:22.547130+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-09T03:48:27.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.7 is not active+clean 2023-11-09T03:48:27.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.547470+0000', 'last_change': '2023-11-09T03:48:22.547470+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.547470+0000', 'last_undegraded': '2023-11-09T03:48:22.547470+0000', 'last_fullsized': '2023-11-09T03:48:22.547470+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-09T03:48:27.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.6 is not active+clean 2023-11-09T03:48:27.236 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.550722+0000', 'last_change': '2023-11-09T03:48:22.550722+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.550722+0000', 'last_undegraded': '2023-11-09T03:48:22.550722+0000', 'last_fullsized': '2023-11-09T03:48:22.550722+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-09T03:48:27.236 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1 is not active+clean 2023-11-09T03:48:27.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.551131+0000', 'last_change': '2023-11-09T03:48:22.551131+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.551131+0000', 'last_undegraded': '2023-11-09T03:48:22.551131+0000', 'last_fullsized': '2023-11-09T03:48:22.551131+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-09T03:48:27.237 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.0 is not active+clean 2023-11-09T03:48:27.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.548843+0000', 'last_change': '2023-11-09T03:48:22.548843+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.548843+0000', 'last_undegraded': '2023-11-09T03:48:22.548843+0000', 'last_fullsized': '2023-11-09T03:48:22.548843+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-09T03:48:27.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3 is not active+clean 2023-11-09T03:48:27.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.546731+0000', 'last_change': '2023-11-09T03:48:22.546731+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.546731+0000', 'last_undegraded': '2023-11-09T03:48:22.546731+0000', 'last_fullsized': '2023-11-09T03:48:22.546731+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-09T03:48:27.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2 is not active+clean 2023-11-09T03:48:27.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.549020+0000', 'last_change': '2023-11-09T03:48:22.549020+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.549020+0000', 'last_undegraded': '2023-11-09T03:48:22.549020+0000', 'last_fullsized': '2023-11-09T03:48:22.549020+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-09T03:48:27.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.d is not active+clean 2023-11-09T03:48:27.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.551507+0000', 'last_change': '2023-11-09T03:48:22.551507+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.551507+0000', 'last_undegraded': '2023-11-09T03:48:22.551507+0000', 'last_fullsized': '2023-11-09T03:48:22.551507+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-09T03:48:27.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.c is not active+clean 2023-11-09T03:48:27.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.550819+0000', 'last_change': '2023-11-09T03:48:22.550819+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.550819+0000', 'last_undegraded': '2023-11-09T03:48:22.550819+0000', 'last_fullsized': '2023-11-09T03:48:22.550819+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-09T03:48:27.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.f is not active+clean 2023-11-09T03:48:27.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.552840+0000', 'last_change': '2023-11-09T03:48:22.552840+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.552840+0000', 'last_undegraded': '2023-11-09T03:48:22.552840+0000', 'last_fullsized': '2023-11-09T03:48:22.552840+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-09T03:48:27.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.9 is not active+clean 2023-11-09T03:48:27.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.551555+0000', 'last_change': '2023-11-09T03:48:22.551555+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.551555+0000', 'last_undegraded': '2023-11-09T03:48:22.551555+0000', 'last_fullsized': '2023-11-09T03:48:22.551555+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-09T03:48:27.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.8 is not active+clean 2023-11-09T03:48:27.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.549641+0000', 'last_change': '2023-11-09T03:48:22.549641+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.549641+0000', 'last_undegraded': '2023-11-09T03:48:22.549641+0000', 'last_fullsized': '2023-11-09T03:48:22.549641+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-09T03:48:27.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.b is not active+clean 2023-11-09T03:48:27.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.550087+0000', 'last_change': '2023-11-09T03:48:22.550087+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.550087+0000', 'last_undegraded': '2023-11-09T03:48:22.550087+0000', 'last_fullsized': '2023-11-09T03:48:22.550087+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-09T03:48:27.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.a is not active+clean 2023-11-09T03:48:27.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.549765+0000', 'last_change': '2023-11-09T03:48:22.549765+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.549765+0000', 'last_undegraded': '2023-11-09T03:48:22.549765+0000', 'last_fullsized': '2023-11-09T03:48:22.549765+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-09T03:48:27.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.e is not active+clean 2023-11-09T03:48:27.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 81, 'state': 'creating+peering', 'last_fresh': '2023-11-09T03:48:22.551049+0000', 'last_change': '2023-11-09T03:48:22.551049+0000', 'last_active': '2023-11-09T03:48:22.537305+0000', 'last_peered': '2023-11-09T03:48:22.537305+0000', 'last_clean': '2023-11-09T03:48:22.537305+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-09T03:48:22.551049+0000', 'last_undegraded': '2023-11-09T03:48:22.551049+0000', 'last_fullsized': '2023-11-09T03:48:22.551049+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'last_clean_scrub_stamp': '2023-11-09T03:48:22.537305+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-09T03:48:27.245 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-09T03:48:27.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.345+0000 7f6d6464b640 1 Processor -- start 2023-11-09T03:48:27.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.345+0000 7f6d6464b640 1 -- start start 2023-11-09T03:48:27.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.346+0000 7f6d6464b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6d5c0b0ae0 0x7f6d5c0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:27.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.346+0000 7f6d6464b640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d5c0b4110 con 0x7f6d5c06db20 2023-11-09T03:48:27.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.346+0000 7f6d6464b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d5c0b43f0 con 0x7f6d5c159c70 2023-11-09T03:48:27.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.346+0000 7f6d6464b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d5c0b49b0 con 0x7f6d5c0b0ae0 2023-11-09T03:48:27.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.346+0000 7f6d62be8640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f6d5c06db20 0x7f6d5c15d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59064/0 (socket says 172.21.15.76:59064) 2023-11-09T03:48:27.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.346+0000 7f6d62be8640 1 -- 172.21.15.76:0/1406732311 learned_addr learned my addr 172.21.15.76:0/1406732311 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:27.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.346+0000 7f6d613e5640 1 -- 172.21.15.76:0/1406732311 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4029043618 0 0) 0x7f6d5c0b4110 con 0x7f6d5c06db20 2023-11-09T03:48:27.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.347+0000 7f6d613e5640 1 -- 172.21.15.76:0/1406732311 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d48003520 con 0x7f6d5c06db20 2023-11-09T03:48:27.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.347+0000 7f6d61be6640 1 --2- 172.21.15.76:0/1406732311 >> v2:172.21.15.113:3300/0 conn(0x7f6d5c0b0ae0 0x7f6d5c0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:27.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.347+0000 7f6d613e5640 1 -- 172.21.15.76:0/1406732311 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2394052691 0 0) 0x7f6d48003520 con 0x7f6d5c06db20 2023-11-09T03:48:27.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.347+0000 7f6d613e5640 1 -- 172.21.15.76:0/1406732311 >> v2:172.21.15.113:3300/0 conn(0x7f6d5c0b0ae0 msgr2=0x7f6d5c0ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:27.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.347+0000 7f6d613e5640 1 --2- 172.21.15.76:0/1406732311 >> v2:172.21.15.113:3300/0 conn(0x7f6d5c0b0ae0 0x7f6d5c0ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:27.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.347+0000 7f6d613e5640 1 -- 172.21.15.76:0/1406732311 >> v1:172.21.15.113:6789/0 conn(0x7f6d5c159c70 legacy=0x7f6d5c0b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:27.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.347+0000 7f6d613e5640 1 -- 172.21.15.76:0/1406732311 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5c0b46d0 con 0x7f6d5c06db20 2023-11-09T03:48:27.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.347+0000 7f6d613e5640 1 -- 172.21.15.76:0/1406732311 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f6d540019f0 con 0x7f6d5c06db20 2023-11-09T03:48:27.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.347+0000 7f6d613e5640 1 -- 172.21.15.76:0/1406732311 >> v1:172.21.15.76:6789/0 conn(0x7f6d5c06db20 legacy=0x7f6d5c15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:27.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.347+0000 7f6d613e5640 1 --2- 172.21.15.76:0/1406732311 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d48003970 0x7f6d48003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:27.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.347+0000 7f6d613e5640 1 --2- 172.21.15.76:0/1406732311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d480047c0 0x7f6d48004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:27.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.347+0000 7f6d613e5640 1 --2- 172.21.15.76:0/1406732311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d48005180 0x7f6d4800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:27.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.347+0000 7f6d613e5640 1 -- 172.21.15.76:0/1406732311 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c0b49b0 con 0x7f6d480047c0 2023-11-09T03:48:27.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.347+0000 7f6d613e5640 1 -- 172.21.15.76:0/1406732311 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4800c020 con 0x7f6d48005180 2023-11-09T03:48:27.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.347+0000 7f6d613e5640 1 -- 172.21.15.76:0/1406732311 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4800c1e0 con 0x7f6d48003970 2023-11-09T03:48:27.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.347+0000 7f6d61be6640 1 --2- 172.21.15.76:0/1406732311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d480047c0 0x7f6d48004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:27.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.347+0000 7f6d62be8640 1 --2- 172.21.15.76:0/1406732311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d48005180 0x7f6d4800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:27.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.347+0000 7f6d623e7640 1 --2- 172.21.15.76:0/1406732311 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d48003970 0x7f6d48003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:27.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.347+0000 7f6d61be6640 1 -- 172.21.15.76:0/1406732311 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d48003970 msgr2=0x7f6d48003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:27.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.347+0000 7f6d61be6640 1 --2- 172.21.15.76:0/1406732311 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d48003970 0x7f6d48003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:27.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.347+0000 7f6d61be6640 1 -- 172.21.15.76:0/1406732311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d48005180 msgr2=0x7f6d4800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:27.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.347+0000 7f6d61be6640 1 --2- 172.21.15.76:0/1406732311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d48005180 0x7f6d4800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:27.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.347+0000 7f6d61be6640 1 -- 172.21.15.76:0/1406732311 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d48003520 con 0x7f6d480047c0 2023-11-09T03:48:27.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.347+0000 7f6d62be8640 1 --2- 172.21.15.76:0/1406732311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d48005180 0x7f6d4800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:27.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.347+0000 7f6d62be8640 1 -- 172.21.15.76:0/1406732311 reap_dead start 2023-11-09T03:48:27.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.348+0000 7f6d61be6640 1 --2- 172.21.15.76:0/1406732311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d480047c0 0x7f6d48004c40 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009ac0 tx=0x7f6d480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e4244f1e8718db5 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:27.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.348+0000 7f6d613e5640 1 -- 172.21.15.76:0/1406732311 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c02c6b0 con 0x7f6d480047c0 2023-11-09T03:48:27.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.348+0000 7f6d613e5640 1 -- 172.21.15.76:0/1406732311 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4c02c810 con 0x7f6d480047c0 2023-11-09T03:48:27.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.348+0000 7f6d613e5640 1 -- 172.21.15.76:0/1406732311 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c02cba0 con 0x7f6d480047c0 2023-11-09T03:48:27.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.349+0000 7f6d6464b640 1 -- 172.21.15.76:0/1406732311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d480047c0 msgr2=0x7f6d48004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:27.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.349+0000 7f6d6464b640 1 --2- 172.21.15.76:0/1406732311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d480047c0 0x7f6d48004c40 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009ac0 tx=0x7f6d480057f0 comp rx=0 tx=0).stop 2023-11-09T03:48:27.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.349+0000 7f6d6464b640 1 -- 172.21.15.76:0/1406732311 shutdown_connections 2023-11-09T03:48:27.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.349+0000 7f6d6464b640 1 --2- 172.21.15.76:0/1406732311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d480047c0 0x7f6d48004c40 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:27.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.349+0000 7f6d6464b640 1 -- 172.21.15.76:0/1406732311 >> 172.21.15.76:0/1406732311 conn(0x7f6d5c0ce950 msgr2=0x7f6d5c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:27.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.349+0000 7f6d6464b640 1 -- 172.21.15.76:0/1406732311 shutdown_connections 2023-11-09T03:48:27.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.349+0000 7f6d6464b640 1 -- 172.21.15.76:0/1406732311 wait complete. 2023-11-09T03:48:27.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.350+0000 7f6d6464b640 1 Processor -- start 2023-11-09T03:48:27.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.350+0000 7f6d6464b640 1 -- start start 2023-11-09T03:48:27.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.351+0000 7f6d6464b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d480047c0 0x7f6d5c0a7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:27.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.351+0000 7f6d6464b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d5c0c6370 0x7f6d5c152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:27.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.351+0000 7f6d6464b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d5c15bd00 0x7f6d5c15c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:27.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.351+0000 7f6d6464b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c15c6c0 con 0x7f6d5c15bd00 2023-11-09T03:48:27.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.351+0000 7f6d6464b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c0c76c0 con 0x7f6d5c0c6370 2023-11-09T03:48:27.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.351+0000 7f6d6464b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c0c67d0 con 0x7f6d480047c0 2023-11-09T03:48:27.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.351+0000 7f6d62be8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d5c15bd00 0x7f6d5c15c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:27.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.351+0000 7f6d62be8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d5c15bd00 0x7f6d5c15c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37890/0 (socket says 172.21.15.76:37890) 2023-11-09T03:48:27.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.351+0000 7f6d62be8640 1 -- 172.21.15.76:0/585080848 learned_addr learned my addr 172.21.15.76:0/585080848 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:27.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.351+0000 7f6d623e7640 1 --2- 172.21.15.76:0/585080848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d480047c0 0x7f6d5c0a7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:27.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.351+0000 7f6d61be6640 1 --2- 172.21.15.76:0/585080848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d5c0c6370 0x7f6d5c152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:27.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.351+0000 7f6d62be8640 1 -- 172.21.15.76:0/585080848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d480047c0 msgr2=0x7f6d5c0a7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:27.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.351+0000 7f6d62be8640 1 --2- 172.21.15.76:0/585080848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d480047c0 0x7f6d5c0a7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:27.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.351+0000 7f6d62be8640 1 -- 172.21.15.76:0/585080848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d5c0c6370 msgr2=0x7f6d5c152da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:27.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.351+0000 7f6d62be8640 1 --2- 172.21.15.76:0/585080848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d5c0c6370 0x7f6d5c152da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:27.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.351+0000 7f6d62be8640 1 -- 172.21.15.76:0/585080848 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5c0c6de0 con 0x7f6d5c15bd00 2023-11-09T03:48:27.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.351+0000 7f6d62be8640 1 --2- 172.21.15.76:0/585080848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d5c15bd00 0x7f6d5c15c180 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c0cf6e0 tx=0x7f6d5400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:27.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.351+0000 7f6d52ffd640 1 -- 172.21.15.76:0/585080848 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d54007ae0 con 0x7f6d5c15bd00 2023-11-09T03:48:27.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.351+0000 7f6d52ffd640 1 -- 172.21.15.76:0/585080848 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5400eda0 con 0x7f6d5c15bd00 2023-11-09T03:48:27.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.351+0000 7f6d6464b640 1 -- 172.21.15.76:0/585080848 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d5c0c70c0 con 0x7f6d5c15bd00 2023-11-09T03:48:27.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.351+0000 7f6d52ffd640 1 -- 172.21.15.76:0/585080848 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d54013640 con 0x7f6d5c15bd00 2023-11-09T03:48:27.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.352+0000 7f6d6464b640 1 -- 172.21.15.76:0/585080848 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d5c0c73a0 con 0x7f6d5c15bd00 2023-11-09T03:48:27.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.352+0000 7f6d6464b640 1 -- 172.21.15.76:0/585080848 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d5c074290 con 0x7f6d5c15bd00 2023-11-09T03:48:27.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.353+0000 7f6d52ffd640 1 -- 172.21.15.76:0/585080848 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6d540137a0 con 0x7f6d5c15bd00 2023-11-09T03:48:27.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.353+0000 7f6d52ffd640 1 --2- 172.21.15.76:0/585080848 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6d4006ae70 0x7f6d4006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:27.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.353+0000 7f6d52ffd640 1 -- 172.21.15.76:0/585080848 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f6d54089ef0 con 0x7f6d5c15bd00 2023-11-09T03:48:27.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.353+0000 7f6d623e7640 1 --2- 172.21.15.76:0/585080848 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6d4006ae70 0x7f6d4006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:27.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.354+0000 7f6d623e7640 1 --2- 172.21.15.76:0/585080848 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6d4006ae70 0x7f6d4006d330 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6d58007fd0 tx=0x7f6d58009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:27.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.358+0000 7f6d52ffd640 1 -- 172.21.15.76:0/585080848 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6d54054ac0 con 0x7f6d5c15bd00 2023-11-09T03:48:27.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.591+0000 7f6d6464b640 1 -- 172.21.15.76:0/585080848 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f6d5c1587c0 con 0x7f6d5c15bd00 2023-11-09T03:48:27.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.591+0000 7f6d52ffd640 1 -- 172.21.15.76:0/585080848 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f6d540580e0 con 0x7f6d5c15bd00 2023-11-09T03:48:27.591 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:27.591 INFO:teuthology.orchestra.run.smithi076.stdout:{"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","health":{"status":"HEALTH_WARN","checks":{"POOL_APP_NOT_ENABLED":{"severity":"HEALTH_WARN","summary":{"message":"3 pool(s) do not have an application enabled","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":200,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":83,"num_osds":8,"num_up_osds":8,"osd_up_since":1699501517,"num_in_osds":8,"osd_in_since":1699501508,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":129},{"state_name":"creating+peering","count":64}],"num_pgs":193,"num_pools":6,"num_objects":2,"data_bytes":590368,"bytes_used":243167232,"bytes_avail":772850946048,"bytes_total":773094113280,"inactive_pgs_ratio":0.33160620927810669},"fsmap":{"epoch":35,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"c","status":"up:active","gid":6984}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2023-11-09T03:48:10.789958+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-09T03:48:27.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.593+0000 7f6d6464b640 1 -- 172.21.15.76:0/585080848 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6d4006ae70 msgr2=0x7f6d4006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:27.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.593+0000 7f6d6464b640 1 --2- 172.21.15.76:0/585080848 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6d4006ae70 0x7f6d4006d330 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6d58007fd0 tx=0x7f6d58009040 comp rx=0 tx=0).stop 2023-11-09T03:48:27.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.593+0000 7f6d6464b640 1 -- 172.21.15.76:0/585080848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d5c15bd00 msgr2=0x7f6d5c15c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:27.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.593+0000 7f6d6464b640 1 --2- 172.21.15.76:0/585080848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d5c15bd00 0x7f6d5c15c180 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c0cf6e0 tx=0x7f6d5400e530 comp rx=0 tx=0).stop 2023-11-09T03:48:27.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.593+0000 7f6d6464b640 1 -- 172.21.15.76:0/585080848 shutdown_connections 2023-11-09T03:48:27.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.593+0000 7f6d6464b640 1 --2- 172.21.15.76:0/585080848 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6d4006ae70 0x7f6d4006d330 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:27.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.593+0000 7f6d6464b640 1 --2- 172.21.15.76:0/585080848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d5c15bd00 0x7f6d5c15c180 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:27.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.593+0000 7f6d6464b640 1 --2- 172.21.15.76:0/585080848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d5c0c6370 0x7f6d5c152da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:27.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.593+0000 7f6d6464b640 1 --2- 172.21.15.76:0/585080848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d480047c0 0x7f6d5c0a7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:27.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.593+0000 7f6d6464b640 1 -- 172.21.15.76:0/585080848 >> 172.21.15.76:0/585080848 conn(0x7f6d5c0ce950 msgr2=0x7f6d5c0ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:27.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.594+0000 7f6d6464b640 1 -- 172.21.15.76:0/585080848 shutdown_connections 2023-11-09T03:48:27.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.594+0000 7f6d6464b640 1 -- 172.21.15.76:0/585080848 wait complete. 2023-11-09T03:48:27.604 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-09T03:48:27.604 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-09T03:48:27.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.704+0000 7f0894090640 1 Processor -- start 2023-11-09T03:48:27.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.704+0000 7f0894090640 1 -- start start 2023-11-09T03:48:27.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.705+0000 7f0894090640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f088c0cde60 0x7f088c15d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:27.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.705+0000 7f0894090640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f088c0ce240 0x7f088c0d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:27.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.705+0000 7f0894090640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f088c15e000 con 0x7f088c15daa0 2023-11-09T03:48:27.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.705+0000 7f0894090640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f088c15e5c0 con 0x7f088c0cde60 2023-11-09T03:48:27.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.705+0000 7f0894090640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088c15e2e0 con 0x7f088c0ce240 2023-11-09T03:48:27.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.705+0000 7f0891e2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f088c0cde60 0x7f088c15d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:27.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.705+0000 7f089262d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f088c0ce240 0x7f088c0d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:27.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.705+0000 7f089262d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f088c0ce240 0x7f088c0d1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:54862/0 (socket says 172.21.15.76:54862) 2023-11-09T03:48:27.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.705+0000 7f0891e2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f088c0cde60 0x7f088c15d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:40922/0 (socket says 172.21.15.76:40922) 2023-11-09T03:48:27.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.705+0000 7f089162b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f088c15daa0 0x7f088c157000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:46930/0 (socket says 172.21.15.76:46930) 2023-11-09T03:48:27.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.705+0000 7f089262d640 1 -- 172.21.15.76:0/3046504883 learned_addr learned my addr 172.21.15.76:0/3046504883 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:27.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.705+0000 7f0890e2a640 1 -- 172.21.15.76:0/3046504883 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2355928820 0 0) 0x7f088c15e000 con 0x7f088c15daa0 2023-11-09T03:48:27.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.705+0000 7f0890e2a640 1 -- 172.21.15.76:0/3046504883 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0878003520 con 0x7f088c15daa0 2023-11-09T03:48:27.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.705+0000 7f0891e2c640 1 -- 172.21.15.76:0/3046504883 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f088c0ce240 msgr2=0x7f088c0d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:27.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.705+0000 7f0891e2c640 1 --2- 172.21.15.76:0/3046504883 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f088c0ce240 0x7f088c0d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:27.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.705+0000 7f0891e2c640 1 -- 172.21.15.76:0/3046504883 >> v1:172.21.15.113:6789/0 conn(0x7f088c15daa0 legacy=0x7f088c157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:27.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.706+0000 7f0891e2c640 1 -- 172.21.15.76:0/3046504883 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088c0d1120 con 0x7f088c0cde60 2023-11-09T03:48:27.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.706+0000 7f089262d640 1 --2- 172.21.15.76:0/3046504883 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f088c0ce240 0x7f088c0d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:48:27.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.706+0000 7f0891e2c640 1 --2- 172.21.15.76:0/3046504883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f088c0cde60 0x7f088c15d560 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f08880097a0 tx=0x7f088802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aedc639b52b01e01 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:27.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.706+0000 7f0890e2a640 1 -- 172.21.15.76:0/3046504883 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088802d040 con 0x7f088c0cde60 2023-11-09T03:48:27.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.706+0000 7f0890e2a640 1 -- 172.21.15.76:0/3046504883 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0888031810 con 0x7f088c0cde60 2023-11-09T03:48:27.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.706+0000 7f0890e2a640 1 -- 172.21.15.76:0/3046504883 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0888031b80 con 0x7f088c0cde60 2023-11-09T03:48:27.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.707+0000 7f0894090640 1 -- 172.21.15.76:0/3046504883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f088c0cde60 msgr2=0x7f088c15d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:27.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.707+0000 7f0894090640 1 --2- 172.21.15.76:0/3046504883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f088c0cde60 0x7f088c15d560 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f08880097a0 tx=0x7f088802b8b0 comp rx=0 tx=0).stop 2023-11-09T03:48:27.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.707+0000 7f0894090640 1 -- 172.21.15.76:0/3046504883 shutdown_connections 2023-11-09T03:48:27.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.707+0000 7f0894090640 1 --2- 172.21.15.76:0/3046504883 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f088c0ce240 0x7f088c0d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:27.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.707+0000 7f0894090640 1 --2- 172.21.15.76:0/3046504883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f088c0cde60 0x7f088c15d560 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:27.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.707+0000 7f0894090640 1 -- 172.21.15.76:0/3046504883 >> 172.21.15.76:0/3046504883 conn(0x7f088c071f30 msgr2=0x7f088c0cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:27.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.707+0000 7f0894090640 1 -- 172.21.15.76:0/3046504883 shutdown_connections 2023-11-09T03:48:27.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.707+0000 7f0894090640 1 -- 172.21.15.76:0/3046504883 wait complete. 2023-11-09T03:48:27.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.708+0000 7f0894090640 1 Processor -- start 2023-11-09T03:48:27.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.708+0000 7f0894090640 1 -- start start 2023-11-09T03:48:27.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.709+0000 7f0894090640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f088c0cde60 0x7f088c1592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:27.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.709+0000 7f0894090640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f088c0ce240 0x7f088c15b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:27.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.709+0000 7f0894090640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f088c15daa0 0x7f088c14bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:27.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.709+0000 7f0894090640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088c156e70 con 0x7f088c0ce240 2023-11-09T03:48:27.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.709+0000 7f0894090640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088c157110 con 0x7f088c0cde60 2023-11-09T03:48:27.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.709+0000 7f0894090640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088c1573f0 con 0x7f088c15daa0 2023-11-09T03:48:27.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.709+0000 7f089162b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f088c0ce240 0x7f088c15b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:27.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.709+0000 7f089162b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f088c0ce240 0x7f088c15b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37906/0 (socket says 172.21.15.76:37906) 2023-11-09T03:48:27.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.709+0000 7f089162b640 1 -- 172.21.15.76:0/2478894230 learned_addr learned my addr 172.21.15.76:0/2478894230 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:27.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.709+0000 7f089262d640 1 --2- 172.21.15.76:0/2478894230 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f088c15daa0 0x7f088c14bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:27.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.709+0000 7f0891e2c640 1 --2- 172.21.15.76:0/2478894230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f088c0cde60 0x7f088c1592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:27.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.709+0000 7f089162b640 1 -- 172.21.15.76:0/2478894230 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f088c15daa0 msgr2=0x7f088c14bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:27.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.709+0000 7f089162b640 1 --2- 172.21.15.76:0/2478894230 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f088c15daa0 0x7f088c14bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:27.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.709+0000 7f089162b640 1 -- 172.21.15.76:0/2478894230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f088c0cde60 msgr2=0x7f088c1592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:27.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.709+0000 7f089162b640 1 --2- 172.21.15.76:0/2478894230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f088c0cde60 0x7f088c1592a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:27.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.709+0000 7f089162b640 1 -- 172.21.15.76:0/2478894230 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0884009b30 con 0x7f088c0ce240 2023-11-09T03:48:27.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.710+0000 7f089162b640 1 --2- 172.21.15.76:0/2478894230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f088c0ce240 0x7f088c15b3b0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f087c00c3c0 tx=0x7f087c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:27.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.710+0000 7f08827fc640 1 -- 172.21.15.76:0/2478894230 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087c0074e0 con 0x7f088c0ce240 2023-11-09T03:48:27.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.710+0000 7f08827fc640 1 -- 172.21.15.76:0/2478894230 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087c007660 con 0x7f088c0ce240 2023-11-09T03:48:27.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.710+0000 7f0894090640 1 -- 172.21.15.76:0/2478894230 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0888009400 con 0x7f088c0ce240 2023-11-09T03:48:27.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.710+0000 7f08827fc640 1 -- 172.21.15.76:0/2478894230 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087c004a20 con 0x7f088c0ce240 2023-11-09T03:48:27.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.710+0000 7f0894090640 1 -- 172.21.15.76:0/2478894230 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f088c1579b0 con 0x7f088c0ce240 2023-11-09T03:48:27.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.711+0000 7f0894090640 1 -- 172.21.15.76:0/2478894230 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f088c074290 con 0x7f088c0ce240 2023-11-09T03:48:27.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.711+0000 7f08827fc640 1 -- 172.21.15.76:0/2478894230 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f087c004b80 con 0x7f088c0ce240 2023-11-09T03:48:27.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.711+0000 7f08827fc640 1 --2- 172.21.15.76:0/2478894230 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f087006ae70 0x7f087006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:27.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.712+0000 7f08827fc640 1 -- 172.21.15.76:0/2478894230 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f087c0895e0 con 0x7f088c0ce240 2023-11-09T03:48:27.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.712+0000 7f0891e2c640 1 --2- 172.21.15.76:0/2478894230 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f087006ae70 0x7f087006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:27.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.712+0000 7f0891e2c640 1 --2- 172.21.15.76:0/2478894230 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f087006ae70 0x7f087006d330 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0888036000 tx=0x7f088803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:27.720 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.716+0000 7f08827fc640 1 -- 172.21.15.76:0/2478894230 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f087c0540a0 con 0x7f088c0ce240 2023-11-09T03:48:27.803 INFO:tasks.ceph.mds.c.smithi076.stderr:2023-11-09T03:48:27.804+0000 7f719d522640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:48:27.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.879+0000 7f0894090640 1 -- 172.21.15.76:0/2478894230 --> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f088c0d1400 con 0x7f087006ae70 2023-11-09T03:48:27.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.892+0000 7f08827fc640 1 -- 172.21.15.76:0/2478894230 <== mgr.4105 v2:172.21.15.113:6832/1264193042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+526928 (secure 0 0 0) 0x7f0888044400 con 0x7f087006ae70 2023-11-09T03:48:27.897 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:27.900 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2023-11-09T03:48:27.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.896+0000 7f0894090640 1 -- 172.21.15.76:0/2478894230 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f087006ae70 msgr2=0x7f087006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:27.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.896+0000 7f0894090640 1 --2- 172.21.15.76:0/2478894230 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f087006ae70 0x7f087006d330 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0888036000 tx=0x7f088803c040 comp rx=0 tx=0).stop 2023-11-09T03:48:27.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.896+0000 7f0894090640 1 -- 172.21.15.76:0/2478894230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f088c0ce240 msgr2=0x7f088c15b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:27.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.896+0000 7f0894090640 1 --2- 172.21.15.76:0/2478894230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f088c0ce240 0x7f088c15b3b0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f087c00c3c0 tx=0x7f087c001e70 comp rx=0 tx=0).stop 2023-11-09T03:48:27.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.896+0000 7f0894090640 1 -- 172.21.15.76:0/2478894230 shutdown_connections 2023-11-09T03:48:27.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.896+0000 7f0894090640 1 --2- 172.21.15.76:0/2478894230 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f087006ae70 0x7f087006d330 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:27.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.896+0000 7f0894090640 1 --2- 172.21.15.76:0/2478894230 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f088c15daa0 0x7f088c14bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:27.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.897+0000 7f0894090640 1 --2- 172.21.15.76:0/2478894230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f088c0ce240 0x7f088c15b3b0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:27.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.897+0000 7f0894090640 1 --2- 172.21.15.76:0/2478894230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f088c0cde60 0x7f088c1592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:27.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.897+0000 7f0894090640 1 -- 172.21.15.76:0/2478894230 >> 172.21.15.76:0/2478894230 conn(0x7f088c071f30 msgr2=0x7f088c0b8070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:27.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.897+0000 7f0894090640 1 -- 172.21.15.76:0/2478894230 shutdown_connections 2023-11-09T03:48:27.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:27.897+0000 7f0894090640 1 -- 172.21.15.76:0/2478894230 wait complete. 2023-11-09T03:48:27.907 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2023-11-09T03:48:26.794284+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":204,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237468,"kb_used_data":8828,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754737252,"statfs":{"total":773094113280,"available":772850946048,"internally_reserved":0,"allocated":9039872,"data_stored":4943241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.256434"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549889+0000","last_change":"2023-11-09T03:48:22.549889+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549889+0000","last_undegraded":"2023-11-09T03:48:22.549889+0000","last_fullsized":"2023-11-09T03:48:22.549889+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.554480+0000","last_change":"2023-11-09T03:48:22.554480+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.554480+0000","last_undegraded":"2023-11-09T03:48:22.554480+0000","last_fullsized":"2023-11-09T03:48:22.554480+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549408+0000","last_change":"2023-11-09T03:48:22.549408+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549408+0000","last_undegraded":"2023-11-09T03:48:22.549408+0000","last_fullsized":"2023-11-09T03:48:22.549408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550714+0000","last_change":"2023-11-09T03:48:22.550714+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550714+0000","last_undegraded":"2023-11-09T03:48:22.550714+0000","last_fullsized":"2023-11-09T03:48:22.550714+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550039+0000","last_change":"2023-11-09T03:48:22.550039+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550039+0000","last_undegraded":"2023-11-09T03:48:22.550039+0000","last_fullsized":"2023-11-09T03:48:22.550039+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549702+0000","last_change":"2023-11-09T03:48:22.549702+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549702+0000","last_undegraded":"2023-11-09T03:48:22.549702+0000","last_fullsized":"2023-11-09T03:48:22.549702+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550846+0000","last_change":"2023-11-09T03:48:22.550846+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550846+0000","last_undegraded":"2023-11-09T03:48:22.550846+0000","last_fullsized":"2023-11-09T03:48:22.550846+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.554400+0000","last_change":"2023-11-09T03:48:22.554400+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.554400+0000","last_undegraded":"2023-11-09T03:48:22.554400+0000","last_fullsized":"2023-11-09T03:48:22.554400+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.547291+0000","last_change":"2023-11-09T03:48:22.547291+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.547291+0000","last_undegraded":"2023-11-09T03:48:22.547291+0000","last_fullsized":"2023-11-09T03:48:22.547291+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551875+0000","last_change":"2023-11-09T03:48:22.551875+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551875+0000","last_undegraded":"2023-11-09T03:48:22.551875+0000","last_fullsized":"2023-11-09T03:48:22.551875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.554690+0000","last_change":"2023-11-09T03:48:22.554690+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.554690+0000","last_undegraded":"2023-11-09T03:48:22.554690+0000","last_fullsized":"2023-11-09T03:48:22.554690+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.547393+0000","last_change":"2023-11-09T03:48:22.547393+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.547393+0000","last_undegraded":"2023-11-09T03:48:22.547393+0000","last_fullsized":"2023-11-09T03:48:22.547393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550250+0000","last_change":"2023-11-09T03:48:22.550250+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550250+0000","last_undegraded":"2023-11-09T03:48:22.550250+0000","last_fullsized":"2023-11-09T03:48:22.550250+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.558130+0000","last_change":"2023-11-09T03:48:22.558130+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.558130+0000","last_undegraded":"2023-11-09T03:48:22.558130+0000","last_fullsized":"2023-11-09T03:48:22.558130+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549957+0000","last_change":"2023-11-09T03:48:22.549957+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549957+0000","last_undegraded":"2023-11-09T03:48:22.549957+0000","last_fullsized":"2023-11-09T03:48:22.549957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.553338+0000","last_change":"2023-11-09T03:48:22.553338+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.553338+0000","last_undegraded":"2023-11-09T03:48:22.553338+0000","last_fullsized":"2023-11-09T03:48:22.553338+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.554384+0000","last_change":"2023-11-09T03:48:22.554384+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.554384+0000","last_undegraded":"2023-11-09T03:48:22.554384+0000","last_fullsized":"2023-11-09T03:48:22.554384+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550716+0000","last_change":"2023-11-09T03:48:22.550716+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550716+0000","last_undegraded":"2023-11-09T03:48:22.550716+0000","last_fullsized":"2023-11-09T03:48:22.550716+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.557699+0000","last_change":"2023-11-09T03:48:22.557699+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.557699+0000","last_undegraded":"2023-11-09T03:48:22.557699+0000","last_fullsized":"2023-11-09T03:48:22.557699+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.546650+0000","last_change":"2023-11-09T03:48:22.546650+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.546650+0000","last_undegraded":"2023-11-09T03:48:22.546650+0000","last_fullsized":"2023-11-09T03:48:22.546650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.555079+0000","last_change":"2023-11-09T03:48:22.555079+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.555079+0000","last_undegraded":"2023-11-09T03:48:22.555079+0000","last_fullsized":"2023-11-09T03:48:22.555079+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.554040+0000","last_change":"2023-11-09T03:48:22.554040+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.554040+0000","last_undegraded":"2023-11-09T03:48:22.554040+0000","last_fullsized":"2023-11-09T03:48:22.554040+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.553445+0000","last_change":"2023-11-09T03:48:22.553445+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.553445+0000","last_undegraded":"2023-11-09T03:48:22.553445+0000","last_fullsized":"2023-11-09T03:48:22.553445+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551333+0000","last_change":"2023-11-09T03:48:22.551333+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551333+0000","last_undegraded":"2023-11-09T03:48:22.551333+0000","last_fullsized":"2023-11-09T03:48:22.551333+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.553219+0000","last_change":"2023-11-09T03:48:22.553219+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.553219+0000","last_undegraded":"2023-11-09T03:48:22.553219+0000","last_fullsized":"2023-11-09T03:48:22.553219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.552812+0000","last_change":"2023-11-09T03:48:22.552812+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.552812+0000","last_undegraded":"2023-11-09T03:48:22.552812+0000","last_fullsized":"2023-11-09T03:48:22.552812+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.552951+0000","last_change":"2023-11-09T03:48:22.552951+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.552951+0000","last_undegraded":"2023-11-09T03:48:22.552951+0000","last_fullsized":"2023-11-09T03:48:22.552951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.552510+0000","last_change":"2023-11-09T03:48:22.552510+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.552510+0000","last_undegraded":"2023-11-09T03:48:22.552510+0000","last_fullsized":"2023-11-09T03:48:22.552510+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.552644+0000","last_change":"2023-11-09T03:48:22.552644+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.552644+0000","last_undegraded":"2023-11-09T03:48:22.552644+0000","last_fullsized":"2023-11-09T03:48:22.552644+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551900+0000","last_change":"2023-11-09T03:48:22.551900+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551900+0000","last_undegraded":"2023-11-09T03:48:22.551900+0000","last_fullsized":"2023-11-09T03:48:22.551900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549131+0000","last_change":"2023-11-09T03:48:22.549131+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549131+0000","last_undegraded":"2023-11-09T03:48:22.549131+0000","last_fullsized":"2023-11-09T03:48:22.549131+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.548919+0000","last_change":"2023-11-09T03:48:22.548919+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.548919+0000","last_undegraded":"2023-11-09T03:48:22.548919+0000","last_fullsized":"2023-11-09T03:48:22.548919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.554246+0000","last_change":"2023-11-09T03:48:22.554246+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.554246+0000","last_undegraded":"2023-11-09T03:48:22.554246+0000","last_fullsized":"2023-11-09T03:48:22.554246+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.568615+0000","last_change":"2023-11-09T03:48:22.562765+0000","last_active":"2023-11-09T03:48:22.568615+0000","last_peered":"2023-11-09T03:48:22.568615+0000","last_clean":"2023-11-09T03:48:22.568615+0000","last_became_active":"2023-11-09T03:48:22.562455+0000","last_became_peered":"2023-11-09T03:48:22.562455+0000","last_unstale":"2023-11-09T03:48:22.568615+0000","last_undegraded":"2023-11-09T03:48:22.568615+0000","last_fullsized":"2023-11-09T03:48:22.568615+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:55:13.543207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545671+0000","last_change":"2023-11-09T03:48:01.449077+0000","last_active":"2023-11-09T03:48:21.545671+0000","last_peered":"2023-11-09T03:48:21.545671+0000","last_clean":"2023-11-09T03:48:21.545671+0000","last_became_active":"2023-11-09T03:48:01.448806+0000","last_became_peered":"2023-11-09T03:48:01.448806+0000","last_unstale":"2023-11-09T03:48:21.545671+0000","last_undegraded":"2023-11-09T03:48:21.545671+0000","last_fullsized":"2023-11-09T03:48:21.545671+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:16:00.220961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542885+0000","last_change":"2023-11-09T03:48:02.611756+0000","last_active":"2023-11-09T03:48:21.542885+0000","last_peered":"2023-11-09T03:48:21.542885+0000","last_clean":"2023-11-09T03:48:21.542885+0000","last_became_active":"2023-11-09T03:48:00.445880+0000","last_became_peered":"2023-11-09T03:48:00.445880+0000","last_unstale":"2023-11-09T03:48:21.542885+0000","last_undegraded":"2023-11-09T03:48:21.542885+0000","last_fullsized":"2023-11-09T03:48:21.542885+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:15:35.281296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085514900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539506+0000","last_change":"2023-11-09T03:47:02.581917+0000","last_active":"2023-11-09T03:48:21.539506+0000","last_peered":"2023-11-09T03:48:21.539506+0000","last_clean":"2023-11-09T03:48:21.539506+0000","last_became_active":"2023-11-09T03:47:01.588082+0000","last_became_peered":"2023-11-09T03:47:01.588082+0000","last_unstale":"2023-11-09T03:48:21.539506+0000","last_undegraded":"2023-11-09T03:48:21.539506+0000","last_fullsized":"2023-11-09T03:48:21.539506+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:44:07.653557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034538899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.553249+0000","last_change":"2023-11-09T03:48:22.553249+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.553249+0000","last_undegraded":"2023-11-09T03:48:22.553249+0000","last_fullsized":"2023-11-09T03:48:22.553249+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.571401+0000","last_change":"2023-11-09T03:48:22.566342+0000","last_active":"2023-11-09T03:48:22.571401+0000","last_peered":"2023-11-09T03:48:22.571401+0000","last_clean":"2023-11-09T03:48:22.571401+0000","last_became_active":"2023-11-09T03:48:22.566028+0000","last_became_peered":"2023-11-09T03:48:22.566028+0000","last_unstale":"2023-11-09T03:48:22.571401+0000","last_undegraded":"2023-11-09T03:48:22.571401+0000","last_fullsized":"2023-11-09T03:48:22.571401+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:31.602666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542096+0000","last_change":"2023-11-09T03:48:01.456245+0000","last_active":"2023-11-09T03:48:21.542096+0000","last_peered":"2023-11-09T03:48:21.542096+0000","last_clean":"2023-11-09T03:48:21.542096+0000","last_became_active":"2023-11-09T03:48:01.455944+0000","last_became_peered":"2023-11-09T03:48:01.455944+0000","last_unstale":"2023-11-09T03:48:21.542096+0000","last_undegraded":"2023-11-09T03:48:21.542096+0000","last_fullsized":"2023-11-09T03:48:21.542096+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:06:14.849196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543120+0000","last_change":"2023-11-09T03:48:02.612341+0000","last_active":"2023-11-09T03:48:21.543120+0000","last_peered":"2023-11-09T03:48:21.543120+0000","last_clean":"2023-11-09T03:48:21.543120+0000","last_became_active":"2023-11-09T03:48:00.444426+0000","last_became_peered":"2023-11-09T03:48:00.444426+0000","last_unstale":"2023-11-09T03:48:21.543120+0000","last_undegraded":"2023-11-09T03:48:21.543120+0000","last_fullsized":"2023-11-09T03:48:21.543120+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:19:53.969861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010812420000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545670+0000","last_change":"2023-11-09T03:47:03.029063+0000","last_active":"2023-11-09T03:48:21.545670+0000","last_peered":"2023-11-09T03:48:21.545670+0000","last_clean":"2023-11-09T03:48:21.545670+0000","last_became_active":"2023-11-09T03:47:01.593582+0000","last_became_peered":"2023-11-09T03:47:01.593582+0000","last_unstale":"2023-11-09T03:48:21.545670+0000","last_undegraded":"2023-11-09T03:48:21.545670+0000","last_fullsized":"2023-11-09T03:48:21.545670+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:56:20.649937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041855100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.546617+0000","last_change":"2023-11-09T03:48:22.546617+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.546617+0000","last_undegraded":"2023-11-09T03:48:22.546617+0000","last_fullsized":"2023-11-09T03:48:22.546617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.564376+0000","last_change":"2023-11-09T03:48:22.562238+0000","last_active":"2023-11-09T03:48:22.564376+0000","last_peered":"2023-11-09T03:48:22.564376+0000","last_clean":"2023-11-09T03:48:22.564376+0000","last_became_active":"2023-11-09T03:48:22.561984+0000","last_became_peered":"2023-11-09T03:48:22.561984+0000","last_unstale":"2023-11-09T03:48:22.564376+0000","last_undegraded":"2023-11-09T03:48:22.564376+0000","last_fullsized":"2023-11-09T03:48:22.564376+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:28:06.557471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541062+0000","last_change":"2023-11-09T03:48:01.450271+0000","last_active":"2023-11-09T03:48:21.541062+0000","last_peered":"2023-11-09T03:48:21.541062+0000","last_clean":"2023-11-09T03:48:21.541062+0000","last_became_active":"2023-11-09T03:48:01.449998+0000","last_became_peered":"2023-11-09T03:48:01.449998+0000","last_unstale":"2023-11-09T03:48:21.541062+0000","last_undegraded":"2023-11-09T03:48:21.541062+0000","last_fullsized":"2023-11-09T03:48:21.541062+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:12:27.170650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545599+0000","last_change":"2023-11-09T03:48:02.852114+0000","last_active":"2023-11-09T03:48:21.545599+0000","last_peered":"2023-11-09T03:48:21.545599+0000","last_clean":"2023-11-09T03:48:21.545599+0000","last_became_active":"2023-11-09T03:48:00.447203+0000","last_became_peered":"2023-11-09T03:48:00.447203+0000","last_unstale":"2023-11-09T03:48:21.545599+0000","last_undegraded":"2023-11-09T03:48:21.545599+0000","last_fullsized":"2023-11-09T03:48:21.545599+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:11:36.826430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049189500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543495+0000","last_change":"2023-11-09T03:47:02.619336+0000","last_active":"2023-11-09T03:48:21.543495+0000","last_peered":"2023-11-09T03:48:21.543495+0000","last_clean":"2023-11-09T03:48:21.543495+0000","last_became_active":"2023-11-09T03:47:01.587749+0000","last_became_peered":"2023-11-09T03:47:01.587749+0000","last_unstale":"2023-11-09T03:48:21.543495+0000","last_undegraded":"2023-11-09T03:48:21.543495+0000","last_fullsized":"2023-11-09T03:48:21.543495+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063032099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551692+0000","last_change":"2023-11-09T03:48:22.551692+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551692+0000","last_undegraded":"2023-11-09T03:48:22.551692+0000","last_fullsized":"2023-11-09T03:48:22.551692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.559640+0000","last_change":"2023-11-09T03:48:22.557530+0000","last_active":"2023-11-09T03:48:22.559640+0000","last_peered":"2023-11-09T03:48:22.559640+0000","last_clean":"2023-11-09T03:48:22.559640+0000","last_became_active":"2023-11-09T03:48:22.556685+0000","last_became_peered":"2023-11-09T03:48:22.556685+0000","last_unstale":"2023-11-09T03:48:22.559640+0000","last_undegraded":"2023-11-09T03:48:22.559640+0000","last_fullsized":"2023-11-09T03:48:22.559640+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:06:49.685032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547581+0000","last_change":"2023-11-09T03:48:01.454147+0000","last_active":"2023-11-09T03:48:21.547581+0000","last_peered":"2023-11-09T03:48:21.547581+0000","last_clean":"2023-11-09T03:48:21.547581+0000","last_became_active":"2023-11-09T03:48:01.453868+0000","last_became_peered":"2023-11-09T03:48:01.453868+0000","last_unstale":"2023-11-09T03:48:21.547581+0000","last_undegraded":"2023-11-09T03:48:21.547581+0000","last_fullsized":"2023-11-09T03:48:21.547581+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:05:09.655318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542848+0000","last_change":"2023-11-09T03:48:02.612000+0000","last_active":"2023-11-09T03:48:21.542848+0000","last_peered":"2023-11-09T03:48:21.542848+0000","last_clean":"2023-11-09T03:48:21.542848+0000","last_became_active":"2023-11-09T03:48:00.441667+0000","last_became_peered":"2023-11-09T03:48:00.441667+0000","last_unstale":"2023-11-09T03:48:21.542848+0000","last_undegraded":"2023-11-09T03:48:21.542848+0000","last_fullsized":"2023-11-09T03:48:21.542848+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:12:26.164644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000966781,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539495+0000","last_change":"2023-11-09T03:47:02.581658+0000","last_active":"2023-11-09T03:48:21.539495+0000","last_peered":"2023-11-09T03:48:21.539495+0000","last_clean":"2023-11-09T03:48:21.539495+0000","last_became_active":"2023-11-09T03:47:01.588612+0000","last_became_peered":"2023-11-09T03:47:01.588612+0000","last_unstale":"2023-11-09T03:48:21.539495+0000","last_undegraded":"2023-11-09T03:48:21.539495+0000","last_fullsized":"2023-11-09T03:48:21.539495+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045238500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.547531+0000","last_change":"2023-11-09T03:48:22.547531+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.547531+0000","last_undegraded":"2023-11-09T03:48:22.547531+0000","last_fullsized":"2023-11-09T03:48:22.547531+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.560046+0000","last_change":"2023-11-09T03:48:22.557828+0000","last_active":"2023-11-09T03:48:22.560046+0000","last_peered":"2023-11-09T03:48:22.560046+0000","last_clean":"2023-11-09T03:48:22.560046+0000","last_became_active":"2023-11-09T03:48:22.557588+0000","last_became_peered":"2023-11-09T03:48:22.557588+0000","last_unstale":"2023-11-09T03:48:22.560046+0000","last_undegraded":"2023-11-09T03:48:22.560046+0000","last_fullsized":"2023-11-09T03:48:22.560046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:36:15.980235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545124+0000","last_change":"2023-11-09T03:48:01.449834+0000","last_active":"2023-11-09T03:48:21.545124+0000","last_peered":"2023-11-09T03:48:21.545124+0000","last_clean":"2023-11-09T03:48:21.545124+0000","last_became_active":"2023-11-09T03:48:01.449552+0000","last_became_peered":"2023-11-09T03:48:01.449552+0000","last_unstale":"2023-11-09T03:48:21.545124+0000","last_undegraded":"2023-11-09T03:48:21.545124+0000","last_fullsized":"2023-11-09T03:48:21.545124+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:11:37.832436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542114+0000","last_change":"2023-11-09T03:48:02.682197+0000","last_active":"2023-11-09T03:48:21.542114+0000","last_peered":"2023-11-09T03:48:21.542114+0000","last_clean":"2023-11-09T03:48:21.542114+0000","last_became_active":"2023-11-09T03:48:00.445616+0000","last_became_peered":"2023-11-09T03:48:00.445616+0000","last_unstale":"2023-11-09T03:48:21.542114+0000","last_undegraded":"2023-11-09T03:48:21.542114+0000","last_fullsized":"2023-11-09T03:48:21.542114+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:57:03.474084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000416494,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542622+0000","last_change":"2023-11-09T03:47:02.618828+0000","last_active":"2023-11-09T03:48:21.542622+0000","last_peered":"2023-11-09T03:48:21.542622+0000","last_clean":"2023-11-09T03:48:21.542622+0000","last_became_active":"2023-11-09T03:47:01.590679+0000","last_became_peered":"2023-11-09T03:47:01.590679+0000","last_unstale":"2023-11-09T03:48:21.542622+0000","last_undegraded":"2023-11-09T03:48:21.542622+0000","last_fullsized":"2023-11-09T03:48:21.542622+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:06:30.982987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070668,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.546525+0000","last_change":"2023-11-09T03:48:22.546525+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.546525+0000","last_undegraded":"2023-11-09T03:48:22.546525+0000","last_fullsized":"2023-11-09T03:48:22.546525+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.560446+0000","last_change":"2023-11-09T03:48:22.556590+0000","last_active":"2023-11-09T03:48:22.560446+0000","last_peered":"2023-11-09T03:48:22.560446+0000","last_clean":"2023-11-09T03:48:22.560446+0000","last_became_active":"2023-11-09T03:48:22.556295+0000","last_became_peered":"2023-11-09T03:48:22.556295+0000","last_unstale":"2023-11-09T03:48:22.560446+0000","last_undegraded":"2023-11-09T03:48:22.560446+0000","last_fullsized":"2023-11-09T03:48:22.560446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:09:09.041542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.546994+0000","last_change":"2023-11-09T03:48:01.447647+0000","last_active":"2023-11-09T03:48:21.546994+0000","last_peered":"2023-11-09T03:48:21.546994+0000","last_clean":"2023-11-09T03:48:21.546994+0000","last_became_active":"2023-11-09T03:48:01.447302+0000","last_became_peered":"2023-11-09T03:48:01.447302+0000","last_unstale":"2023-11-09T03:48:21.546994+0000","last_undegraded":"2023-11-09T03:48:21.546994+0000","last_fullsized":"2023-11-09T03:48:21.546994+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:08:47.931939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539407+0000","last_change":"2023-11-09T03:48:03.327731+0000","last_active":"2023-11-09T03:48:21.539407+0000","last_peered":"2023-11-09T03:48:21.539407+0000","last_clean":"2023-11-09T03:48:21.539407+0000","last_became_active":"2023-11-09T03:48:00.449660+0000","last_became_peered":"2023-11-09T03:48:00.449660+0000","last_unstale":"2023-11-09T03:48:21.539407+0000","last_undegraded":"2023-11-09T03:48:21.539407+0000","last_fullsized":"2023-11-09T03:48:21.539407+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:02:07.940908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037816599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.546874+0000","last_change":"2023-11-09T03:47:03.146528+0000","last_active":"2023-11-09T03:48:21.546874+0000","last_peered":"2023-11-09T03:48:21.546874+0000","last_clean":"2023-11-09T03:48:21.546874+0000","last_became_active":"2023-11-09T03:47:01.587060+0000","last_became_peered":"2023-11-09T03:47:01.587060+0000","last_unstale":"2023-11-09T03:48:21.546874+0000","last_undegraded":"2023-11-09T03:48:21.546874+0000","last_fullsized":"2023-11-09T03:48:21.546874+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:39:15.238496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079064999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.547973+0000","last_change":"2023-11-09T03:48:22.547973+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.547973+0000","last_undegraded":"2023-11-09T03:48:22.547973+0000","last_fullsized":"2023-11-09T03:48:22.547973+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.569334+0000","last_change":"2023-11-09T03:48:22.565708+0000","last_active":"2023-11-09T03:48:22.569334+0000","last_peered":"2023-11-09T03:48:22.569334+0000","last_clean":"2023-11-09T03:48:22.569334+0000","last_became_active":"2023-11-09T03:48:22.563961+0000","last_became_peered":"2023-11-09T03:48:22.563961+0000","last_unstale":"2023-11-09T03:48:22.569334+0000","last_undegraded":"2023-11-09T03:48:22.569334+0000","last_fullsized":"2023-11-09T03:48:22.569334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:03:03.041986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539139+0000","last_change":"2023-11-09T03:48:01.453939+0000","last_active":"2023-11-09T03:48:21.539139+0000","last_peered":"2023-11-09T03:48:21.539139+0000","last_clean":"2023-11-09T03:48:21.539139+0000","last_became_active":"2023-11-09T03:48:01.453600+0000","last_became_peered":"2023-11-09T03:48:01.453600+0000","last_unstale":"2023-11-09T03:48:21.539139+0000","last_undegraded":"2023-11-09T03:48:21.539139+0000","last_fullsized":"2023-11-09T03:48:21.539139+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:45:10.522254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547099+0000","last_change":"2023-11-09T03:48:02.440910+0000","last_active":"2023-11-09T03:48:21.547099+0000","last_peered":"2023-11-09T03:48:21.547099+0000","last_clean":"2023-11-09T03:48:21.547099+0000","last_became_active":"2023-11-09T03:48:00.443791+0000","last_became_peered":"2023-11-09T03:48:00.443791+0000","last_unstale":"2023-11-09T03:48:21.547099+0000","last_undegraded":"2023-11-09T03:48:21.547099+0000","last_fullsized":"2023-11-09T03:48:21.547099+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:11:36.826430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069223400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547414+0000","last_change":"2023-11-09T03:47:03.148246+0000","last_active":"2023-11-09T03:48:21.547414+0000","last_peered":"2023-11-09T03:48:21.547414+0000","last_clean":"2023-11-09T03:48:21.547414+0000","last_became_active":"2023-11-09T03:47:01.590988+0000","last_became_peered":"2023-11-09T03:47:01.590988+0000","last_unstale":"2023-11-09T03:48:21.547414+0000","last_undegraded":"2023-11-09T03:48:21.547414+0000","last_fullsized":"2023-11-09T03:48:21.547414+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:07:20.785445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055651099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551090+0000","last_change":"2023-11-09T03:48:22.551090+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551090+0000","last_undegraded":"2023-11-09T03:48:22.551090+0000","last_fullsized":"2023-11-09T03:48:22.551090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.569365+0000","last_change":"2023-11-09T03:48:22.565920+0000","last_active":"2023-11-09T03:48:22.569365+0000","last_peered":"2023-11-09T03:48:22.569365+0000","last_clean":"2023-11-09T03:48:22.569365+0000","last_became_active":"2023-11-09T03:48:22.564592+0000","last_became_peered":"2023-11-09T03:48:22.564592+0000","last_unstale":"2023-11-09T03:48:22.569365+0000","last_undegraded":"2023-11-09T03:48:22.569365+0000","last_fullsized":"2023-11-09T03:48:22.569365+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:41:46.534565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539107+0000","last_change":"2023-11-09T03:48:01.450557+0000","last_active":"2023-11-09T03:48:21.539107+0000","last_peered":"2023-11-09T03:48:21.539107+0000","last_clean":"2023-11-09T03:48:21.539107+0000","last_became_active":"2023-11-09T03:48:01.450291+0000","last_became_peered":"2023-11-09T03:48:01.450291+0000","last_unstale":"2023-11-09T03:48:21.539107+0000","last_undegraded":"2023-11-09T03:48:21.539107+0000","last_fullsized":"2023-11-09T03:48:21.539107+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:21:29.699594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545002+0000","last_change":"2023-11-09T03:48:02.852010+0000","last_active":"2023-11-09T03:48:21.545002+0000","last_peered":"2023-11-09T03:48:21.545002+0000","last_clean":"2023-11-09T03:48:21.545002+0000","last_became_active":"2023-11-09T03:48:00.445267+0000","last_became_peered":"2023-11-09T03:48:00.445267+0000","last_unstale":"2023-11-09T03:48:21.545002+0000","last_undegraded":"2023-11-09T03:48:21.545002+0000","last_fullsized":"2023-11-09T03:48:21.545002+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:17.989628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064559199999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":50,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540983+0000","last_change":"2023-11-09T03:47:02.582754+0000","last_active":"2023-11-09T03:48:21.540983+0000","last_peered":"2023-11-09T03:48:21.540983+0000","last_clean":"2023-11-09T03:48:21.540983+0000","last_became_active":"2023-11-09T03:47:01.589715+0000","last_became_peered":"2023-11-09T03:47:01.589715+0000","last_unstale":"2023-11-09T03:48:21.540983+0000","last_undegraded":"2023-11-09T03:48:21.540983+0000","last_fullsized":"2023-11-09T03:48:21.540983+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099342799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.546539+0000","last_change":"2023-11-09T03:48:22.546539+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.546539+0000","last_undegraded":"2023-11-09T03:48:22.546539+0000","last_fullsized":"2023-11-09T03:48:22.546539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.571909+0000","last_change":"2023-11-09T03:48:22.566859+0000","last_active":"2023-11-09T03:48:22.571909+0000","last_peered":"2023-11-09T03:48:22.571909+0000","last_clean":"2023-11-09T03:48:22.571909+0000","last_became_active":"2023-11-09T03:48:22.566080+0000","last_became_peered":"2023-11-09T03:48:22.566080+0000","last_unstale":"2023-11-09T03:48:22.571909+0000","last_undegraded":"2023-11-09T03:48:22.571909+0000","last_fullsized":"2023-11-09T03:48:22.571909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:14:56.468255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543312+0000","last_change":"2023-11-09T03:48:01.452456+0000","last_active":"2023-11-09T03:48:21.543312+0000","last_peered":"2023-11-09T03:48:21.543312+0000","last_clean":"2023-11-09T03:48:21.543312+0000","last_became_active":"2023-11-09T03:48:01.452059+0000","last_became_peered":"2023-11-09T03:48:01.452059+0000","last_unstale":"2023-11-09T03:48:21.543312+0000","last_undegraded":"2023-11-09T03:48:21.543312+0000","last_fullsized":"2023-11-09T03:48:21.543312+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:57:56.855869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541181+0000","last_change":"2023-11-09T03:48:02.442557+0000","last_active":"2023-11-09T03:48:21.541181+0000","last_peered":"2023-11-09T03:48:21.541181+0000","last_clean":"2023-11-09T03:48:21.541181+0000","last_became_active":"2023-11-09T03:48:00.444823+0000","last_became_peered":"2023-11-09T03:48:00.444823+0000","last_unstale":"2023-11-09T03:48:21.541181+0000","last_undegraded":"2023-11-09T03:48:21.541181+0000","last_fullsized":"2023-11-09T03:48:21.541181+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:19:53.969861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075076299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":47,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545453+0000","last_change":"2023-11-09T03:47:02.724896+0000","last_active":"2023-11-09T03:48:21.545453+0000","last_peered":"2023-11-09T03:48:21.545453+0000","last_clean":"2023-11-09T03:48:21.545453+0000","last_became_active":"2023-11-09T03:47:01.592784+0000","last_became_peered":"2023-11-09T03:47:01.592784+0000","last_unstale":"2023-11-09T03:48:21.545453+0000","last_undegraded":"2023-11-09T03:48:21.545453+0000","last_fullsized":"2023-11-09T03:48:21.545453+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:51:32.016204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056683500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.548469+0000","last_change":"2023-11-09T03:48:22.548469+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.548469+0000","last_undegraded":"2023-11-09T03:48:22.548469+0000","last_fullsized":"2023-11-09T03:48:22.548469+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.571850+0000","last_change":"2023-11-09T03:48:22.565279+0000","last_active":"2023-11-09T03:48:22.571850+0000","last_peered":"2023-11-09T03:48:22.571850+0000","last_clean":"2023-11-09T03:48:22.571850+0000","last_became_active":"2023-11-09T03:48:22.564998+0000","last_became_peered":"2023-11-09T03:48:22.564998+0000","last_unstale":"2023-11-09T03:48:22.571850+0000","last_undegraded":"2023-11-09T03:48:22.571850+0000","last_fullsized":"2023-11-09T03:48:22.571850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:24:48.220122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540896+0000","last_change":"2023-11-09T03:48:01.452167+0000","last_active":"2023-11-09T03:48:21.540896+0000","last_peered":"2023-11-09T03:48:21.540896+0000","last_clean":"2023-11-09T03:48:21.540896+0000","last_became_active":"2023-11-09T03:48:01.451780+0000","last_became_peered":"2023-11-09T03:48:01.451780+0000","last_unstale":"2023-11-09T03:48:21.540896+0000","last_undegraded":"2023-11-09T03:48:21.540896+0000","last_fullsized":"2023-11-09T03:48:21.540896+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:15:36.287302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543137+0000","last_change":"2023-11-09T03:48:02.612246+0000","last_active":"2023-11-09T03:48:21.543137+0000","last_peered":"2023-11-09T03:48:21.543137+0000","last_clean":"2023-11-09T03:48:21.543137+0000","last_became_active":"2023-11-09T03:48:00.444629+0000","last_became_peered":"2023-11-09T03:48:00.444629+0000","last_unstale":"2023-11-09T03:48:21.543137+0000","last_undegraded":"2023-11-09T03:48:21.543137+0000","last_fullsized":"2023-11-09T03:48:21.543137+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:51:17.754054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098167299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541889+0000","last_change":"2023-11-09T03:47:03.146527+0000","last_active":"2023-11-09T03:48:21.541889+0000","last_peered":"2023-11-09T03:48:21.541889+0000","last_clean":"2023-11-09T03:48:21.541889+0000","last_became_active":"2023-11-09T03:47:01.587858+0000","last_became_peered":"2023-11-09T03:47:01.587858+0000","last_unstale":"2023-11-09T03:48:21.541889+0000","last_undegraded":"2023-11-09T03:48:21.541889+0000","last_fullsized":"2023-11-09T03:48:21.541889+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:56:04.201126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084439599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.548328+0000","last_change":"2023-11-09T03:48:22.548328+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.548328+0000","last_undegraded":"2023-11-09T03:48:22.548328+0000","last_fullsized":"2023-11-09T03:48:22.548328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.558055+0000","last_change":"2023-11-09T03:48:22.553920+0000","last_active":"2023-11-09T03:48:22.558055+0000","last_peered":"2023-11-09T03:48:22.558055+0000","last_clean":"2023-11-09T03:48:22.558055+0000","last_became_active":"2023-11-09T03:48:22.552644+0000","last_became_peered":"2023-11-09T03:48:22.552644+0000","last_unstale":"2023-11-09T03:48:22.558055+0000","last_undegraded":"2023-11-09T03:48:22.558055+0000","last_fullsized":"2023-11-09T03:48:22.558055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:59.646501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545255+0000","last_change":"2023-11-09T03:48:01.450685+0000","last_active":"2023-11-09T03:48:21.545255+0000","last_peered":"2023-11-09T03:48:21.545255+0000","last_clean":"2023-11-09T03:48:21.545255+0000","last_became_active":"2023-11-09T03:48:01.450445+0000","last_became_peered":"2023-11-09T03:48:01.450445+0000","last_unstale":"2023-11-09T03:48:21.545255+0000","last_undegraded":"2023-11-09T03:48:21.545255+0000","last_fullsized":"2023-11-09T03:48:21.545255+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:16:00.220961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547739+0000","last_change":"2023-11-09T03:48:02.441928+0000","last_active":"2023-11-09T03:48:21.547739+0000","last_peered":"2023-11-09T03:48:21.547739+0000","last_clean":"2023-11-09T03:48:21.547739+0000","last_became_active":"2023-11-09T03:48:00.444417+0000","last_became_peered":"2023-11-09T03:48:00.444417+0000","last_unstale":"2023-11-09T03:48:21.547739+0000","last_undegraded":"2023-11-09T03:48:21.547739+0000","last_fullsized":"2023-11-09T03:48:21.547739+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:55.450096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072428900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":47,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545707+0000","last_change":"2023-11-09T03:47:02.725223+0000","last_active":"2023-11-09T03:48:21.545707+0000","last_peered":"2023-11-09T03:48:21.545707+0000","last_clean":"2023-11-09T03:48:21.545707+0000","last_became_active":"2023-11-09T03:47:01.590635+0000","last_became_peered":"2023-11-09T03:47:01.590635+0000","last_unstale":"2023-11-09T03:48:21.545707+0000","last_undegraded":"2023-11-09T03:48:21.545707+0000","last_fullsized":"2023-11-09T03:48:21.545707+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:05.754357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051685100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551472+0000","last_change":"2023-11-09T03:48:22.551472+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551472+0000","last_undegraded":"2023-11-09T03:48:22.551472+0000","last_fullsized":"2023-11-09T03:48:22.551472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.569184+0000","last_change":"2023-11-09T03:48:22.563841+0000","last_active":"2023-11-09T03:48:22.569184+0000","last_peered":"2023-11-09T03:48:22.569184+0000","last_clean":"2023-11-09T03:48:22.569184+0000","last_became_active":"2023-11-09T03:48:22.563590+0000","last_became_peered":"2023-11-09T03:48:22.563590+0000","last_unstale":"2023-11-09T03:48:22.569184+0000","last_undegraded":"2023-11-09T03:48:22.569184+0000","last_fullsized":"2023-11-09T03:48:22.569184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:10:36.225903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542779+0000","last_change":"2023-11-09T03:48:01.453143+0000","last_active":"2023-11-09T03:48:21.542779+0000","last_peered":"2023-11-09T03:48:21.542779+0000","last_clean":"2023-11-09T03:48:21.542779+0000","last_became_active":"2023-11-09T03:48:01.452873+0000","last_became_peered":"2023-11-09T03:48:01.452873+0000","last_unstale":"2023-11-09T03:48:21.542779+0000","last_undegraded":"2023-11-09T03:48:21.542779+0000","last_fullsized":"2023-11-09T03:48:21.542779+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:06:28.575429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.548086+0000","last_change":"2023-11-09T03:48:02.442416+0000","last_active":"2023-11-09T03:48:21.548086+0000","last_peered":"2023-11-09T03:48:21.548086+0000","last_clean":"2023-11-09T03:48:21.548086+0000","last_became_active":"2023-11-09T03:48:00.443765+0000","last_became_peered":"2023-11-09T03:48:00.443765+0000","last_unstale":"2023-11-09T03:48:21.548086+0000","last_undegraded":"2023-11-09T03:48:21.548086+0000","last_fullsized":"2023-11-09T03:48:21.548086+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:41:22.321811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012468850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":50,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541230+0000","last_change":"2023-11-09T03:47:02.582739+0000","last_active":"2023-11-09T03:48:21.541230+0000","last_peered":"2023-11-09T03:48:21.541230+0000","last_clean":"2023-11-09T03:48:21.541230+0000","last_became_active":"2023-11-09T03:47:01.592743+0000","last_became_peered":"2023-11-09T03:47:01.592743+0000","last_unstale":"2023-11-09T03:48:21.541230+0000","last_undegraded":"2023-11-09T03:48:21.541230+0000","last_fullsized":"2023-11-09T03:48:21.541230+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051802700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.546563+0000","last_change":"2023-11-09T03:48:22.546563+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.546563+0000","last_undegraded":"2023-11-09T03:48:22.546563+0000","last_fullsized":"2023-11-09T03:48:22.546563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.561070+0000","last_change":"2023-11-09T03:48:22.559947+0000","last_active":"2023-11-09T03:48:22.561070+0000","last_peered":"2023-11-09T03:48:22.561070+0000","last_clean":"2023-11-09T03:48:22.561070+0000","last_became_active":"2023-11-09T03:48:22.559667+0000","last_became_peered":"2023-11-09T03:48:22.559667+0000","last_unstale":"2023-11-09T03:48:22.561070+0000","last_undegraded":"2023-11-09T03:48:22.561070+0000","last_fullsized":"2023-11-09T03:48:22.561070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:57.844631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540463+0000","last_change":"2023-11-09T03:48:01.442195+0000","last_active":"2023-11-09T03:48:21.540463+0000","last_peered":"2023-11-09T03:48:21.540463+0000","last_clean":"2023-11-09T03:48:21.540463+0000","last_became_active":"2023-11-09T03:48:01.441887+0000","last_became_peered":"2023-11-09T03:48:01.441887+0000","last_unstale":"2023-11-09T03:48:21.540463+0000","last_undegraded":"2023-11-09T03:48:21.540463+0000","last_fullsized":"2023-11-09T03:48:21.540463+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:06:28.575429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540528+0000","last_change":"2023-11-09T03:48:02.442061+0000","last_active":"2023-11-09T03:48:21.540528+0000","last_peered":"2023-11-09T03:48:21.540528+0000","last_clean":"2023-11-09T03:48:21.540528+0000","last_became_active":"2023-11-09T03:48:00.445921+0000","last_became_peered":"2023-11-09T03:48:00.445921+0000","last_unstale":"2023-11-09T03:48:21.540528+0000","last_undegraded":"2023-11-09T03:48:21.540528+0000","last_fullsized":"2023-11-09T03:48:21.540528+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:50:55.299235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012719770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":47,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545111+0000","last_change":"2023-11-09T03:47:02.724953+0000","last_active":"2023-11-09T03:48:21.545111+0000","last_peered":"2023-11-09T03:48:21.545111+0000","last_clean":"2023-11-09T03:48:21.545111+0000","last_became_active":"2023-11-09T03:47:01.587133+0000","last_became_peered":"2023-11-09T03:47:01.587133+0000","last_unstale":"2023-11-09T03:48:21.545111+0000","last_undegraded":"2023-11-09T03:48:21.545111+0000","last_fullsized":"2023-11-09T03:48:21.545111+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:23:48.521161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066497599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.547867+0000","last_change":"2023-11-09T03:48:22.547867+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.547867+0000","last_undegraded":"2023-11-09T03:48:22.547867+0000","last_fullsized":"2023-11-09T03:48:22.547867+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.563196+0000","last_change":"2023-11-09T03:48:22.556449+0000","last_active":"2023-11-09T03:48:22.563196+0000","last_peered":"2023-11-09T03:48:22.563196+0000","last_clean":"2023-11-09T03:48:22.563196+0000","last_became_active":"2023-11-09T03:48:22.556170+0000","last_became_peered":"2023-11-09T03:48:22.556170+0000","last_unstale":"2023-11-09T03:48:22.563196+0000","last_undegraded":"2023-11-09T03:48:22.563196+0000","last_fullsized":"2023-11-09T03:48:22.563196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:09:09.041542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539364+0000","last_change":"2023-11-09T03:48:01.450507+0000","last_active":"2023-11-09T03:48:21.539364+0000","last_peered":"2023-11-09T03:48:21.539364+0000","last_clean":"2023-11-09T03:48:21.539364+0000","last_became_active":"2023-11-09T03:48:01.450191+0000","last_became_peered":"2023-11-09T03:48:01.450191+0000","last_unstale":"2023-11-09T03:48:21.539364+0000","last_undegraded":"2023-11-09T03:48:21.539364+0000","last_fullsized":"2023-11-09T03:48:21.539364+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:32:40.342502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540558+0000","last_change":"2023-11-09T03:48:02.442120+0000","last_active":"2023-11-09T03:48:21.540558+0000","last_peered":"2023-11-09T03:48:21.540558+0000","last_clean":"2023-11-09T03:48:21.540558+0000","last_became_active":"2023-11-09T03:48:00.445504+0000","last_became_peered":"2023-11-09T03:48:00.445504+0000","last_unstale":"2023-11-09T03:48:21.540558+0000","last_undegraded":"2023-11-09T03:48:21.540558+0000","last_fullsized":"2023-11-09T03:48:21.540558+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:35:53.864627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001095766,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541998+0000","last_change":"2023-11-09T03:47:03.146217+0000","last_active":"2023-11-09T03:48:21.541998+0000","last_peered":"2023-11-09T03:48:21.541998+0000","last_clean":"2023-11-09T03:48:21.541998+0000","last_became_active":"2023-11-09T03:47:01.587598+0000","last_became_peered":"2023-11-09T03:47:01.587598+0000","last_unstale":"2023-11-09T03:48:21.541998+0000","last_undegraded":"2023-11-09T03:48:21.541998+0000","last_fullsized":"2023-11-09T03:48:21.541998+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:15:22.723455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090126400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550235+0000","last_change":"2023-11-09T03:48:22.550235+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550235+0000","last_undegraded":"2023-11-09T03:48:22.550235+0000","last_fullsized":"2023-11-09T03:48:22.550235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.564011+0000","last_change":"2023-11-09T03:48:22.561567+0000","last_active":"2023-11-09T03:48:22.564011+0000","last_peered":"2023-11-09T03:48:22.564011+0000","last_clean":"2023-11-09T03:48:22.564011+0000","last_became_active":"2023-11-09T03:48:22.561266+0000","last_became_peered":"2023-11-09T03:48:22.561266+0000","last_unstale":"2023-11-09T03:48:22.564011+0000","last_undegraded":"2023-11-09T03:48:22.564011+0000","last_fullsized":"2023-11-09T03:48:22.564011+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:21.472203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542405+0000","last_change":"2023-11-09T03:48:01.454878+0000","last_active":"2023-11-09T03:48:21.542405+0000","last_peered":"2023-11-09T03:48:21.542405+0000","last_clean":"2023-11-09T03:48:21.542405+0000","last_became_active":"2023-11-09T03:48:01.454591+0000","last_became_peered":"2023-11-09T03:48:01.454591+0000","last_unstale":"2023-11-09T03:48:21.542405+0000","last_undegraded":"2023-11-09T03:48:21.542405+0000","last_fullsized":"2023-11-09T03:48:21.542405+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:38.536898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539170+0000","last_change":"2023-11-09T03:48:03.327800+0000","last_active":"2023-11-09T03:48:21.539170+0000","last_peered":"2023-11-09T03:48:21.539170+0000","last_clean":"2023-11-09T03:48:21.539170+0000","last_became_active":"2023-11-09T03:48:00.446576+0000","last_became_peered":"2023-11-09T03:48:00.446576+0000","last_unstale":"2023-11-09T03:48:21.539170+0000","last_undegraded":"2023-11-09T03:48:21.539170+0000","last_fullsized":"2023-11-09T03:48:21.539170+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:37:08.851102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054679700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547427+0000","last_change":"2023-11-09T03:47:03.148283+0000","last_active":"2023-11-09T03:48:21.547427+0000","last_peered":"2023-11-09T03:48:21.547427+0000","last_clean":"2023-11-09T03:48:21.547427+0000","last_became_active":"2023-11-09T03:47:01.588841+0000","last_became_peered":"2023-11-09T03:47:01.588841+0000","last_unstale":"2023-11-09T03:48:21.547427+0000","last_undegraded":"2023-11-09T03:48:21.547427+0000","last_fullsized":"2023-11-09T03:48:21.547427+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:47:44.880741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000436081,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.552336+0000","last_change":"2023-11-09T03:48:22.552336+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.552336+0000","last_undegraded":"2023-11-09T03:48:22.552336+0000","last_fullsized":"2023-11-09T03:48:22.552336+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.558220+0000","last_change":"2023-11-09T03:48:22.556480+0000","last_active":"2023-11-09T03:48:22.558220+0000","last_peered":"2023-11-09T03:48:22.558220+0000","last_clean":"2023-11-09T03:48:22.558220+0000","last_became_active":"2023-11-09T03:48:22.556059+0000","last_became_peered":"2023-11-09T03:48:22.556059+0000","last_unstale":"2023-11-09T03:48:22.558220+0000","last_undegraded":"2023-11-09T03:48:22.558220+0000","last_fullsized":"2023-11-09T03:48:22.558220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:51:17.414844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.544682+0000","last_change":"2023-11-09T03:48:01.450581+0000","last_active":"2023-11-09T03:48:21.544682+0000","last_peered":"2023-11-09T03:48:21.544682+0000","last_clean":"2023-11-09T03:48:21.544682+0000","last_became_active":"2023-11-09T03:48:01.450280+0000","last_became_peered":"2023-11-09T03:48:01.450280+0000","last_unstale":"2023-11-09T03:48:21.544682+0000","last_undegraded":"2023-11-09T03:48:21.544682+0000","last_fullsized":"2023-11-09T03:48:21.544682+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:18.995634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.546874+0000","last_change":"2023-11-09T03:48:02.440934+0000","last_active":"2023-11-09T03:48:21.546874+0000","last_peered":"2023-11-09T03:48:21.546874+0000","last_clean":"2023-11-09T03:48:21.546874+0000","last_became_active":"2023-11-09T03:48:00.444394+0000","last_became_peered":"2023-11-09T03:48:00.444394+0000","last_unstale":"2023-11-09T03:48:21.546874+0000","last_undegraded":"2023-11-09T03:48:21.546874+0000","last_fullsized":"2023-11-09T03:48:21.546874+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:15:59.214955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063908300000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542547+0000","last_change":"2023-11-09T03:47:02.618856+0000","last_active":"2023-11-09T03:48:21.542547+0000","last_peered":"2023-11-09T03:48:21.542547+0000","last_clean":"2023-11-09T03:48:21.542547+0000","last_became_active":"2023-11-09T03:47:01.590430+0000","last_became_peered":"2023-11-09T03:47:01.590430+0000","last_unstale":"2023-11-09T03:48:21.542547+0000","last_undegraded":"2023-11-09T03:48:21.542547+0000","last_fullsized":"2023-11-09T03:48:21.542547+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:57:18.470305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055030200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550790+0000","last_change":"2023-11-09T03:48:22.550790+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550790+0000","last_undegraded":"2023-11-09T03:48:22.550790+0000","last_fullsized":"2023-11-09T03:48:22.550790+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.563239+0000","last_change":"2023-11-09T03:48:22.559687+0000","last_active":"2023-11-09T03:48:22.563239+0000","last_peered":"2023-11-09T03:48:22.563239+0000","last_clean":"2023-11-09T03:48:22.563239+0000","last_became_active":"2023-11-09T03:48:22.559404+0000","last_became_peered":"2023-11-09T03:48:22.559404+0000","last_unstale":"2023-11-09T03:48:22.563239+0000","last_undegraded":"2023-11-09T03:48:22.563239+0000","last_fullsized":"2023-11-09T03:48:22.563239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:39:53.462957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545586+0000","last_change":"2023-11-09T03:48:01.453600+0000","last_active":"2023-11-09T03:48:21.545586+0000","last_peered":"2023-11-09T03:48:21.545586+0000","last_clean":"2023-11-09T03:48:21.545586+0000","last_became_active":"2023-11-09T03:48:01.453330+0000","last_became_peered":"2023-11-09T03:48:01.453330+0000","last_unstale":"2023-11-09T03:48:21.545586+0000","last_undegraded":"2023-11-09T03:48:21.545586+0000","last_fullsized":"2023-11-09T03:48:21.545586+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:08:47.931939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539485+0000","last_change":"2023-11-09T03:48:03.328210+0000","last_active":"2023-11-09T03:48:21.539485+0000","last_peered":"2023-11-09T03:48:21.539485+0000","last_clean":"2023-11-09T03:48:21.539485+0000","last_became_active":"2023-11-09T03:48:00.450002+0000","last_became_peered":"2023-11-09T03:48:00.450002+0000","last_unstale":"2023-11-09T03:48:21.539485+0000","last_undegraded":"2023-11-09T03:48:21.539485+0000","last_fullsized":"2023-11-09T03:48:21.539485+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:50.391187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036521900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542921+0000","last_change":"2023-11-09T03:47:02.619144+0000","last_active":"2023-11-09T03:48:21.542921+0000","last_peered":"2023-11-09T03:48:21.542921+0000","last_clean":"2023-11-09T03:48:21.542921+0000","last_became_active":"2023-11-09T03:47:01.590672+0000","last_became_peered":"2023-11-09T03:47:01.590672+0000","last_unstale":"2023-11-09T03:48:21.542921+0000","last_undegraded":"2023-11-09T03:48:21.542921+0000","last_fullsized":"2023-11-09T03:48:21.542921+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:03.025119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060997500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.547130+0000","last_change":"2023-11-09T03:48:22.547130+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.547130+0000","last_undegraded":"2023-11-09T03:48:22.547130+0000","last_fullsized":"2023-11-09T03:48:22.547130+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.571874+0000","last_change":"2023-11-09T03:48:22.566419+0000","last_active":"2023-11-09T03:48:22.571874+0000","last_peered":"2023-11-09T03:48:22.571874+0000","last_clean":"2023-11-09T03:48:22.571874+0000","last_became_active":"2023-11-09T03:48:22.566144+0000","last_became_peered":"2023-11-09T03:48:22.566144+0000","last_unstale":"2023-11-09T03:48:22.571874+0000","last_undegraded":"2023-11-09T03:48:22.571874+0000","last_fullsized":"2023-11-09T03:48:22.571874+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:43:01.915119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541019+0000","last_change":"2023-11-09T03:48:01.447278+0000","last_active":"2023-11-09T03:48:21.541019+0000","last_peered":"2023-11-09T03:48:21.541019+0000","last_clean":"2023-11-09T03:48:21.541019+0000","last_became_active":"2023-11-09T03:48:01.446991+0000","last_became_peered":"2023-11-09T03:48:01.446991+0000","last_unstale":"2023-11-09T03:48:21.541019+0000","last_undegraded":"2023-11-09T03:48:21.541019+0000","last_fullsized":"2023-11-09T03:48:21.541019+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:27:49.190426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539429+0000","last_change":"2023-11-09T03:48:03.328278+0000","last_active":"2023-11-09T03:48:21.539429+0000","last_peered":"2023-11-09T03:48:21.539429+0000","last_clean":"2023-11-09T03:48:21.539429+0000","last_became_active":"2023-11-09T03:48:00.443213+0000","last_became_peered":"2023-11-09T03:48:00.443213+0000","last_unstale":"2023-11-09T03:48:21.539429+0000","last_undegraded":"2023-11-09T03:48:21.539429+0000","last_fullsized":"2023-11-09T03:48:21.539429+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:20:59.702297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030886100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543013+0000","last_change":"2023-11-09T03:47:02.619079+0000","last_active":"2023-11-09T03:48:21.543013+0000","last_peered":"2023-11-09T03:48:21.543013+0000","last_clean":"2023-11-09T03:48:21.543013+0000","last_became_active":"2023-11-09T03:47:01.591421+0000","last_became_peered":"2023-11-09T03:47:01.591421+0000","last_unstale":"2023-11-09T03:48:21.543013+0000","last_undegraded":"2023-11-09T03:48:21.543013+0000","last_fullsized":"2023-11-09T03:48:21.543013+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:18:58.230606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068736399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.547470+0000","last_change":"2023-11-09T03:48:22.547470+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.547470+0000","last_undegraded":"2023-11-09T03:48:22.547470+0000","last_fullsized":"2023-11-09T03:48:22.547470+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.574695+0000","last_change":"2023-11-09T03:48:22.569861+0000","last_active":"2023-11-09T03:48:22.574695+0000","last_peered":"2023-11-09T03:48:22.574695+0000","last_clean":"2023-11-09T03:48:22.574695+0000","last_became_active":"2023-11-09T03:48:22.569611+0000","last_became_peered":"2023-11-09T03:48:22.569611+0000","last_unstale":"2023-11-09T03:48:22.574695+0000","last_undegraded":"2023-11-09T03:48:22.574695+0000","last_fullsized":"2023-11-09T03:48:22.574695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:36:57.293292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541146+0000","last_change":"2023-11-09T03:48:01.449069+0000","last_active":"2023-11-09T03:48:21.541146+0000","last_peered":"2023-11-09T03:48:21.541146+0000","last_clean":"2023-11-09T03:48:21.541146+0000","last_became_active":"2023-11-09T03:48:01.448810+0000","last_became_peered":"2023-11-09T03:48:01.448810+0000","last_unstale":"2023-11-09T03:48:21.541146+0000","last_undegraded":"2023-11-09T03:48:21.541146+0000","last_fullsized":"2023-11-09T03:48:21.541146+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:56:51.158572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542867+0000","last_change":"2023-11-09T03:48:02.612067+0000","last_active":"2023-11-09T03:48:21.542867+0000","last_peered":"2023-11-09T03:48:21.542867+0000","last_clean":"2023-11-09T03:48:21.542867+0000","last_became_active":"2023-11-09T03:48:00.443296+0000","last_became_peered":"2023-11-09T03:48:00.443296+0000","last_unstale":"2023-11-09T03:48:21.542867+0000","last_undegraded":"2023-11-09T03:48:21.542867+0000","last_fullsized":"2023-11-09T03:48:21.542867+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:27:48.184420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010423330000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547943+0000","last_change":"2023-11-09T03:47:03.148529+0000","last_active":"2023-11-09T03:48:21.547943+0000","last_peered":"2023-11-09T03:48:21.547943+0000","last_clean":"2023-11-09T03:48:21.547943+0000","last_became_active":"2023-11-09T03:47:01.584148+0000","last_became_peered":"2023-11-09T03:47:01.584148+0000","last_unstale":"2023-11-09T03:48:21.547943+0000","last_undegraded":"2023-11-09T03:48:21.547943+0000","last_fullsized":"2023-11-09T03:48:21.547943+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:40:00.237248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061475600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550722+0000","last_change":"2023-11-09T03:48:22.550722+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550722+0000","last_undegraded":"2023-11-09T03:48:22.550722+0000","last_fullsized":"2023-11-09T03:48:22.550722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.563077+0000","last_change":"2023-11-09T03:48:22.558072+0000","last_active":"2023-11-09T03:48:22.563077+0000","last_peered":"2023-11-09T03:48:22.563077+0000","last_clean":"2023-11-09T03:48:22.563077+0000","last_became_active":"2023-11-09T03:48:22.557800+0000","last_became_peered":"2023-11-09T03:48:22.557800+0000","last_unstale":"2023-11-09T03:48:22.563077+0000","last_undegraded":"2023-11-09T03:48:22.563077+0000","last_fullsized":"2023-11-09T03:48:22.563077+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:00:20.811244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542948+0000","last_change":"2023-11-09T03:48:01.451750+0000","last_active":"2023-11-09T03:48:21.542948+0000","last_peered":"2023-11-09T03:48:21.542948+0000","last_clean":"2023-11-09T03:48:21.542948+0000","last_became_active":"2023-11-09T03:48:01.451393+0000","last_became_peered":"2023-11-09T03:48:01.451393+0000","last_unstale":"2023-11-09T03:48:21.542948+0000","last_undegraded":"2023-11-09T03:48:21.542948+0000","last_fullsized":"2023-11-09T03:48:21.542948+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:56:51.158572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542743+0000","last_change":"2023-11-09T03:48:02.611721+0000","last_active":"2023-11-09T03:48:21.542743+0000","last_peered":"2023-11-09T03:48:21.542743+0000","last_clean":"2023-11-09T03:48:21.542743+0000","last_became_active":"2023-11-09T03:48:00.439557+0000","last_became_peered":"2023-11-09T03:48:00.439557+0000","last_unstale":"2023-11-09T03:48:21.542743+0000","last_undegraded":"2023-11-09T03:48:21.542743+0000","last_fullsized":"2023-11-09T03:48:21.542743+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:35:53.864627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001037187,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543088+0000","last_change":"2023-11-09T03:47:03.148217+0000","last_active":"2023-11-09T03:48:21.543088+0000","last_peered":"2023-11-09T03:48:21.543088+0000","last_clean":"2023-11-09T03:48:21.543088+0000","last_became_active":"2023-11-09T03:47:01.593424+0000","last_became_peered":"2023-11-09T03:47:01.593424+0000","last_unstale":"2023-11-09T03:48:21.543088+0000","last_undegraded":"2023-11-09T03:48:21.543088+0000","last_fullsized":"2023-11-09T03:48:21.543088+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:19:32.693691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001769865,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551131+0000","last_change":"2023-11-09T03:48:22.551131+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551131+0000","last_undegraded":"2023-11-09T03:48:22.551131+0000","last_fullsized":"2023-11-09T03:48:22.551131+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.569383+0000","last_change":"2023-11-09T03:48:22.566073+0000","last_active":"2023-11-09T03:48:22.569383+0000","last_peered":"2023-11-09T03:48:22.569383+0000","last_clean":"2023-11-09T03:48:22.569383+0000","last_became_active":"2023-11-09T03:48:22.564106+0000","last_became_peered":"2023-11-09T03:48:22.564106+0000","last_unstale":"2023-11-09T03:48:22.569383+0000","last_undegraded":"2023-11-09T03:48:22.569383+0000","last_fullsized":"2023-11-09T03:48:22.569383+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:05:06.931697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547292+0000","last_change":"2023-11-09T03:48:01.447617+0000","last_active":"2023-11-09T03:48:21.547292+0000","last_peered":"2023-11-09T03:48:21.547292+0000","last_clean":"2023-11-09T03:48:21.547292+0000","last_became_active":"2023-11-09T03:48:01.447329+0000","last_became_peered":"2023-11-09T03:48:01.447329+0000","last_unstale":"2023-11-09T03:48:21.547292+0000","last_undegraded":"2023-11-09T03:48:21.547292+0000","last_fullsized":"2023-11-09T03:48:21.547292+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:54:52.433604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540889+0000","last_change":"2023-11-09T03:48:02.442427+0000","last_active":"2023-11-09T03:48:21.540889+0000","last_peered":"2023-11-09T03:48:21.540889+0000","last_clean":"2023-11-09T03:48:21.540889+0000","last_became_active":"2023-11-09T03:48:00.444856+0000","last_became_peered":"2023-11-09T03:48:00.444856+0000","last_unstale":"2023-11-09T03:48:21.540889+0000","last_undegraded":"2023-11-09T03:48:21.540889+0000","last_fullsized":"2023-11-09T03:48:21.540889+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:53:42.763434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011927730000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.546971+0000","last_change":"2023-11-09T03:47:03.146470+0000","last_active":"2023-11-09T03:48:21.546971+0000","last_peered":"2023-11-09T03:48:21.546971+0000","last_clean":"2023-11-09T03:48:21.546971+0000","last_became_active":"2023-11-09T03:47:01.587304+0000","last_became_peered":"2023-11-09T03:47:01.587304+0000","last_unstale":"2023-11-09T03:48:21.546971+0000","last_undegraded":"2023-11-09T03:48:21.546971+0000","last_fullsized":"2023-11-09T03:48:21.546971+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:58:23.339287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00110395,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.548843+0000","last_change":"2023-11-09T03:48:22.548843+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.548843+0000","last_undegraded":"2023-11-09T03:48:22.548843+0000","last_fullsized":"2023-11-09T03:48:22.548843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.564450+0000","last_change":"2023-11-09T03:48:22.561952+0000","last_active":"2023-11-09T03:48:22.564450+0000","last_peered":"2023-11-09T03:48:22.564450+0000","last_clean":"2023-11-09T03:48:22.564450+0000","last_became_active":"2023-11-09T03:48:22.561647+0000","last_became_peered":"2023-11-09T03:48:22.561647+0000","last_unstale":"2023-11-09T03:48:22.564450+0000","last_undegraded":"2023-11-09T03:48:22.564450+0000","last_fullsized":"2023-11-09T03:48:22.564450+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:52.879173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545482+0000","last_change":"2023-11-09T03:48:01.449406+0000","last_active":"2023-11-09T03:48:21.545482+0000","last_peered":"2023-11-09T03:48:21.545482+0000","last_clean":"2023-11-09T03:48:21.545482+0000","last_became_active":"2023-11-09T03:48:01.449130+0000","last_became_peered":"2023-11-09T03:48:01.449130+0000","last_unstale":"2023-11-09T03:48:21.545482+0000","last_undegraded":"2023-11-09T03:48:21.545482+0000","last_fullsized":"2023-11-09T03:48:21.545482+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:20:36.303940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547362+0000","last_change":"2023-11-09T03:48:02.441801+0000","last_active":"2023-11-09T03:48:21.547362+0000","last_peered":"2023-11-09T03:48:21.547362+0000","last_clean":"2023-11-09T03:48:21.547362+0000","last_became_active":"2023-11-09T03:48:00.444975+0000","last_became_peered":"2023-11-09T03:48:00.444975+0000","last_unstale":"2023-11-09T03:48:21.547362+0000","last_undegraded":"2023-11-09T03:48:21.547362+0000","last_fullsized":"2023-11-09T03:48:21.547362+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:39:31.347349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061165099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542743+0000","last_change":"2023-11-09T03:47:03.147367+0000","last_active":"2023-11-09T03:48:21.542743+0000","last_peered":"2023-11-09T03:48:21.542743+0000","last_clean":"2023-11-09T03:48:21.542743+0000","last_became_active":"2023-11-09T03:47:01.588359+0000","last_became_peered":"2023-11-09T03:47:01.588359+0000","last_unstale":"2023-11-09T03:48:21.542743+0000","last_undegraded":"2023-11-09T03:48:21.542743+0000","last_fullsized":"2023-11-09T03:48:21.542743+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:21:34.303068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014106660000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.546731+0000","last_change":"2023-11-09T03:48:22.546731+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.546731+0000","last_undegraded":"2023-11-09T03:48:22.546731+0000","last_fullsized":"2023-11-09T03:48:22.546731+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.563046+0000","last_change":"2023-11-09T03:48:22.556245+0000","last_active":"2023-11-09T03:48:22.563046+0000","last_peered":"2023-11-09T03:48:22.563046+0000","last_clean":"2023-11-09T03:48:22.563046+0000","last_became_active":"2023-11-09T03:48:22.555974+0000","last_became_peered":"2023-11-09T03:48:22.555974+0000","last_unstale":"2023-11-09T03:48:22.563046+0000","last_undegraded":"2023-11-09T03:48:22.563046+0000","last_fullsized":"2023-11-09T03:48:22.563046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:55:13.543207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542383+0000","last_change":"2023-11-09T03:48:01.451866+0000","last_active":"2023-11-09T03:48:21.542383+0000","last_peered":"2023-11-09T03:48:21.542383+0000","last_clean":"2023-11-09T03:48:21.542383+0000","last_became_active":"2023-11-09T03:48:01.451350+0000","last_became_peered":"2023-11-09T03:48:01.451350+0000","last_unstale":"2023-11-09T03:48:21.542383+0000","last_undegraded":"2023-11-09T03:48:21.542383+0000","last_fullsized":"2023-11-09T03:48:21.542383+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:36.735029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540171+0000","last_change":"2023-11-09T03:48:02.441648+0000","last_active":"2023-11-09T03:48:21.540171+0000","last_peered":"2023-11-09T03:48:21.540171+0000","last_clean":"2023-11-09T03:48:21.540171+0000","last_became_active":"2023-11-09T03:48:00.448541+0000","last_became_peered":"2023-11-09T03:48:00.448541+0000","last_unstale":"2023-11-09T03:48:21.540171+0000","last_undegraded":"2023-11-09T03:48:21.540171+0000","last_fullsized":"2023-11-09T03:48:21.540171+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:35.729023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539334+0000","last_change":"2023-11-09T03:47:02.581682+0000","last_active":"2023-11-09T03:48:21.539334+0000","last_peered":"2023-11-09T03:48:21.539334+0000","last_clean":"2023-11-09T03:48:21.539334+0000","last_became_active":"2023-11-09T03:47:01.593051+0000","last_became_peered":"2023-11-09T03:47:01.593051+0000","last_unstale":"2023-11-09T03:48:21.539334+0000","last_undegraded":"2023-11-09T03:48:21.539334+0000","last_fullsized":"2023-11-09T03:48:21.539334+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047277799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549020+0000","last_change":"2023-11-09T03:48:22.549020+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549020+0000","last_undegraded":"2023-11-09T03:48:22.549020+0000","last_fullsized":"2023-11-09T03:48:22.549020+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.569210+0000","last_change":"2023-11-09T03:48:22.563908+0000","last_active":"2023-11-09T03:48:22.569210+0000","last_peered":"2023-11-09T03:48:22.569210+0000","last_clean":"2023-11-09T03:48:22.569210+0000","last_became_active":"2023-11-09T03:48:22.563651+0000","last_became_peered":"2023-11-09T03:48:22.563651+0000","last_unstale":"2023-11-09T03:48:22.569210+0000","last_undegraded":"2023-11-09T03:48:22.569210+0000","last_fullsized":"2023-11-09T03:48:22.569210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:52.879173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.544927+0000","last_change":"2023-11-09T03:48:01.455784+0000","last_active":"2023-11-09T03:48:21.544927+0000","last_peered":"2023-11-09T03:48:21.544927+0000","last_clean":"2023-11-09T03:48:21.544927+0000","last_became_active":"2023-11-09T03:48:01.455532+0000","last_became_peered":"2023-11-09T03:48:01.455532+0000","last_unstale":"2023-11-09T03:48:21.544927+0000","last_undegraded":"2023-11-09T03:48:21.544927+0000","last_fullsized":"2023-11-09T03:48:21.544927+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:21:23.156761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542535+0000","last_change":"2023-11-09T03:48:02.611298+0000","last_active":"2023-11-09T03:48:21.542535+0000","last_peered":"2023-11-09T03:48:21.542535+0000","last_clean":"2023-11-09T03:48:21.542535+0000","last_became_active":"2023-11-09T03:48:00.441261+0000","last_became_peered":"2023-11-09T03:48:00.441261+0000","last_unstale":"2023-11-09T03:48:21.542535+0000","last_undegraded":"2023-11-09T03:48:21.542535+0000","last_fullsized":"2023-11-09T03:48:21.542535+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:50:55.299235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081460199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542200+0000","last_change":"2023-11-09T03:47:03.146309+0000","last_active":"2023-11-09T03:48:21.542200+0000","last_peered":"2023-11-09T03:48:21.542200+0000","last_clean":"2023-11-09T03:48:21.542200+0000","last_became_active":"2023-11-09T03:47:01.587560+0000","last_became_peered":"2023-11-09T03:47:01.587560+0000","last_unstale":"2023-11-09T03:48:21.542200+0000","last_undegraded":"2023-11-09T03:48:21.542200+0000","last_fullsized":"2023-11-09T03:48:21.542200+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:27:55.271945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070988400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551507+0000","last_change":"2023-11-09T03:48:22.551507+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551507+0000","last_undegraded":"2023-11-09T03:48:22.551507+0000","last_fullsized":"2023-11-09T03:48:22.551507+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.563543+0000","last_change":"2023-11-09T03:48:22.561557+0000","last_active":"2023-11-09T03:48:22.563543+0000","last_peered":"2023-11-09T03:48:22.563543+0000","last_clean":"2023-11-09T03:48:22.563543+0000","last_became_active":"2023-11-09T03:48:22.561249+0000","last_became_peered":"2023-11-09T03:48:22.561249+0000","last_unstale":"2023-11-09T03:48:22.563543+0000","last_undegraded":"2023-11-09T03:48:22.563543+0000","last_fullsized":"2023-11-09T03:48:22.563543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:15:26.050548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543219+0000","last_change":"2023-11-09T03:48:01.452477+0000","last_active":"2023-11-09T03:48:21.543219+0000","last_peered":"2023-11-09T03:48:21.543219+0000","last_clean":"2023-11-09T03:48:21.543219+0000","last_became_active":"2023-11-09T03:48:01.452186+0000","last_became_peered":"2023-11-09T03:48:01.452186+0000","last_unstale":"2023-11-09T03:48:21.543219+0000","last_undegraded":"2023-11-09T03:48:21.543219+0000","last_fullsized":"2023-11-09T03:48:21.543219+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:32:21.825778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547861+0000","last_change":"2023-11-09T03:48:02.442101+0000","last_active":"2023-11-09T03:48:21.547861+0000","last_peered":"2023-11-09T03:48:21.547861+0000","last_clean":"2023-11-09T03:48:21.547861+0000","last_became_active":"2023-11-09T03:48:00.449354+0000","last_became_peered":"2023-11-09T03:48:00.449354+0000","last_unstale":"2023-11-09T03:48:21.547861+0000","last_undegraded":"2023-11-09T03:48:21.547861+0000","last_fullsized":"2023-11-09T03:48:21.547861+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:06:32.906582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093455000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":47,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545438+0000","last_change":"2023-11-09T03:47:02.725125+0000","last_active":"2023-11-09T03:48:21.545438+0000","last_peered":"2023-11-09T03:48:21.545438+0000","last_clean":"2023-11-09T03:48:21.545438+0000","last_became_active":"2023-11-09T03:47:01.591521+0000","last_became_peered":"2023-11-09T03:47:01.591521+0000","last_unstale":"2023-11-09T03:48:21.545438+0000","last_undegraded":"2023-11-09T03:48:21.545438+0000","last_fullsized":"2023-11-09T03:48:21.545438+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:52:40.076542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000758675,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550819+0000","last_change":"2023-11-09T03:48:22.550819+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550819+0000","last_undegraded":"2023-11-09T03:48:22.550819+0000","last_fullsized":"2023-11-09T03:48:22.550819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.565704+0000","last_change":"2023-11-09T03:48:22.561189+0000","last_active":"2023-11-09T03:48:22.565704+0000","last_peered":"2023-11-09T03:48:22.565704+0000","last_clean":"2023-11-09T03:48:22.565704+0000","last_became_active":"2023-11-09T03:48:22.560938+0000","last_became_peered":"2023-11-09T03:48:22.560938+0000","last_unstale":"2023-11-09T03:48:22.565704+0000","last_undegraded":"2023-11-09T03:48:22.565704+0000","last_fullsized":"2023-11-09T03:48:22.565704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:05:30.764921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545418+0000","last_change":"2023-11-09T03:48:01.448129+0000","last_active":"2023-11-09T03:48:21.545418+0000","last_peered":"2023-11-09T03:48:21.545418+0000","last_clean":"2023-11-09T03:48:21.545418+0000","last_became_active":"2023-11-09T03:48:01.447844+0000","last_became_peered":"2023-11-09T03:48:01.447844+0000","last_unstale":"2023-11-09T03:48:21.545418+0000","last_undegraded":"2023-11-09T03:48:21.545418+0000","last_fullsized":"2023-11-09T03:48:21.545418+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:54:52.433604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543211+0000","last_change":"2023-11-09T03:48:02.612450+0000","last_active":"2023-11-09T03:48:21.543211+0000","last_peered":"2023-11-09T03:48:21.543211+0000","last_clean":"2023-11-09T03:48:21.543211+0000","last_became_active":"2023-11-09T03:48:00.441855+0000","last_became_peered":"2023-11-09T03:48:00.441855+0000","last_unstale":"2023-11-09T03:48:21.543211+0000","last_undegraded":"2023-11-09T03:48:21.543211+0000","last_fullsized":"2023-11-09T03:48:21.543211+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:53:42.763434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095368899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547816+0000","last_change":"2023-11-09T03:47:03.147289+0000","last_active":"2023-11-09T03:48:21.547816+0000","last_peered":"2023-11-09T03:48:21.547816+0000","last_clean":"2023-11-09T03:48:21.547816+0000","last_became_active":"2023-11-09T03:47:01.587784+0000","last_became_peered":"2023-11-09T03:47:01.587784+0000","last_unstale":"2023-11-09T03:48:21.547816+0000","last_undegraded":"2023-11-09T03:48:21.547816+0000","last_fullsized":"2023-11-09T03:48:21.547816+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:11:23.824112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038781299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.552840+0000","last_change":"2023-11-09T03:48:22.552840+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.552840+0000","last_undegraded":"2023-11-09T03:48:22.552840+0000","last_fullsized":"2023-11-09T03:48:22.552840+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.568633+0000","last_change":"2023-11-09T03:48:22.564955+0000","last_active":"2023-11-09T03:48:22.568633+0000","last_peered":"2023-11-09T03:48:22.568633+0000","last_clean":"2023-11-09T03:48:22.568633+0000","last_became_active":"2023-11-09T03:48:22.564677+0000","last_became_peered":"2023-11-09T03:48:22.564677+0000","last_unstale":"2023-11-09T03:48:22.568633+0000","last_undegraded":"2023-11-09T03:48:22.568633+0000","last_fullsized":"2023-11-09T03:48:22.568633+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:16:21.330564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541065+0000","last_change":"2023-11-09T03:48:01.449617+0000","last_active":"2023-11-09T03:48:21.541065+0000","last_peered":"2023-11-09T03:48:21.541065+0000","last_clean":"2023-11-09T03:48:21.541065+0000","last_became_active":"2023-11-09T03:48:01.449328+0000","last_became_peered":"2023-11-09T03:48:01.449328+0000","last_unstale":"2023-11-09T03:48:21.541065+0000","last_undegraded":"2023-11-09T03:48:21.541065+0000","last_fullsized":"2023-11-09T03:48:21.541065+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:51:18.760059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547661+0000","last_change":"2023-11-09T03:48:02.442215+0000","last_active":"2023-11-09T03:48:21.547661+0000","last_peered":"2023-11-09T03:48:21.547661+0000","last_clean":"2023-11-09T03:48:21.547661+0000","last_became_active":"2023-11-09T03:48:00.444720+0000","last_became_peered":"2023-11-09T03:48:00.444720+0000","last_unstale":"2023-11-09T03:48:21.547661+0000","last_undegraded":"2023-11-09T03:48:21.547661+0000","last_fullsized":"2023-11-09T03:48:21.547661+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:55:55.735668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001407176,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.543390+0000","last_change":"2023-11-09T03:47:02.619346+0000","last_active":"2023-11-09T03:48:21.543390+0000","last_peered":"2023-11-09T03:48:21.543390+0000","last_clean":"2023-11-09T03:48:21.543390+0000","last_became_active":"2023-11-09T03:47:01.590389+0000","last_became_peered":"2023-11-09T03:47:01.590389+0000","last_unstale":"2023-11-09T03:48:21.543390+0000","last_undegraded":"2023-11-09T03:48:21.543390+0000","last_fullsized":"2023-11-09T03:48:21.543390+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000384072,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551555+0000","last_change":"2023-11-09T03:48:22.551555+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551555+0000","last_undegraded":"2023-11-09T03:48:22.551555+0000","last_fullsized":"2023-11-09T03:48:22.551555+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.569120+0000","last_change":"2023-11-09T03:48:22.562601+0000","last_active":"2023-11-09T03:48:22.569120+0000","last_peered":"2023-11-09T03:48:22.569120+0000","last_clean":"2023-11-09T03:48:22.569120+0000","last_became_active":"2023-11-09T03:48:22.562326+0000","last_became_peered":"2023-11-09T03:48:22.562326+0000","last_unstale":"2023-11-09T03:48:22.569120+0000","last_undegraded":"2023-11-09T03:48:22.569120+0000","last_fullsized":"2023-11-09T03:48:22.569120+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:28:06.557471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545053+0000","last_change":"2023-11-09T03:48:01.450018+0000","last_active":"2023-11-09T03:48:21.545053+0000","last_peered":"2023-11-09T03:48:21.545053+0000","last_clean":"2023-11-09T03:48:21.545053+0000","last_became_active":"2023-11-09T03:48:01.449749+0000","last_became_peered":"2023-11-09T03:48:01.449749+0000","last_unstale":"2023-11-09T03:48:21.545053+0000","last_undegraded":"2023-11-09T03:48:21.545053+0000","last_fullsized":"2023-11-09T03:48:21.545053+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:21:23.156761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542795+0000","last_change":"2023-11-09T03:48:02.524216+0000","last_active":"2023-11-09T03:48:21.542795+0000","last_peered":"2023-11-09T03:48:21.542795+0000","last_clean":"2023-11-09T03:48:21.542795+0000","last_became_active":"2023-11-09T03:48:00.446972+0000","last_became_peered":"2023-11-09T03:48:00.446972+0000","last_unstale":"2023-11-09T03:48:21.542795+0000","last_undegraded":"2023-11-09T03:48:21.542795+0000","last_fullsized":"2023-11-09T03:48:21.542795+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:54:07.987773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000825255,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":50,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.540373+0000","last_change":"2023-11-09T03:47:02.582393+0000","last_active":"2023-11-09T03:48:21.540373+0000","last_peered":"2023-11-09T03:48:21.540373+0000","last_clean":"2023-11-09T03:48:21.540373+0000","last_became_active":"2023-11-09T03:47:01.591752+0000","last_became_peered":"2023-11-09T03:47:01.591752+0000","last_unstale":"2023-11-09T03:48:21.540373+0000","last_undegraded":"2023-11-09T03:48:21.540373+0000","last_fullsized":"2023-11-09T03:48:21.540373+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:03.025119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067021599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549641+0000","last_change":"2023-11-09T03:48:22.549641+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549641+0000","last_undegraded":"2023-11-09T03:48:22.549641+0000","last_fullsized":"2023-11-09T03:48:22.549641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.566064+0000","last_change":"2023-11-09T03:48:22.562752+0000","last_active":"2023-11-09T03:48:22.566064+0000","last_peered":"2023-11-09T03:48:22.566064+0000","last_clean":"2023-11-09T03:48:22.566064+0000","last_became_active":"2023-11-09T03:48:22.562474+0000","last_became_peered":"2023-11-09T03:48:22.562474+0000","last_unstale":"2023-11-09T03:48:22.566064+0000","last_undegraded":"2023-11-09T03:48:22.566064+0000","last_fullsized":"2023-11-09T03:48:22.566064+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:35:27.804836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542868+0000","last_change":"2023-11-09T03:48:01.451462+0000","last_active":"2023-11-09T03:48:21.542868+0000","last_peered":"2023-11-09T03:48:21.542868+0000","last_clean":"2023-11-09T03:48:21.542868+0000","last_became_active":"2023-11-09T03:48:01.451188+0000","last_became_peered":"2023-11-09T03:48:01.451188+0000","last_unstale":"2023-11-09T03:48:21.542868+0000","last_undegraded":"2023-11-09T03:48:21.542868+0000","last_fullsized":"2023-11-09T03:48:21.542868+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:28.631801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542221+0000","last_change":"2023-11-09T03:48:02.682290+0000","last_active":"2023-11-09T03:48:21.542221+0000","last_peered":"2023-11-09T03:48:21.542221+0000","last_clean":"2023-11-09T03:48:21.542221+0000","last_became_active":"2023-11-09T03:48:00.444246+0000","last_became_peered":"2023-11-09T03:48:00.444246+0000","last_unstale":"2023-11-09T03:48:21.542221+0000","last_undegraded":"2023-11-09T03:48:21.542221+0000","last_fullsized":"2023-11-09T03:48:21.542221+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:09.487058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000423359,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.544622+0000","last_change":"2023-11-09T03:47:03.028744+0000","last_active":"2023-11-09T03:48:21.544622+0000","last_peered":"2023-11-09T03:48:21.544622+0000","last_clean":"2023-11-09T03:48:21.544622+0000","last_became_active":"2023-11-09T03:47:01.593890+0000","last_became_peered":"2023-11-09T03:47:01.593890+0000","last_unstale":"2023-11-09T03:48:21.544622+0000","last_undegraded":"2023-11-09T03:48:21.544622+0000","last_fullsized":"2023-11-09T03:48:21.544622+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:05.754357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053008299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.550087+0000","last_change":"2023-11-09T03:48:22.550087+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.550087+0000","last_undegraded":"2023-11-09T03:48:22.550087+0000","last_fullsized":"2023-11-09T03:48:22.550087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.566105+0000","last_change":"2023-11-09T03:48:22.560645+0000","last_active":"2023-11-09T03:48:22.566105+0000","last_peered":"2023-11-09T03:48:22.566105+0000","last_clean":"2023-11-09T03:48:22.566105+0000","last_became_active":"2023-11-09T03:48:22.560328+0000","last_became_peered":"2023-11-09T03:48:22.560328+0000","last_unstale":"2023-11-09T03:48:22.566105+0000","last_undegraded":"2023-11-09T03:48:22.566105+0000","last_fullsized":"2023-11-09T03:48:22.566105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:59.646501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.544881+0000","last_change":"2023-11-09T03:48:01.450144+0000","last_active":"2023-11-09T03:48:21.544881+0000","last_peered":"2023-11-09T03:48:21.544881+0000","last_clean":"2023-11-09T03:48:21.544881+0000","last_became_active":"2023-11-09T03:48:01.449859+0000","last_became_peered":"2023-11-09T03:48:01.449859+0000","last_unstale":"2023-11-09T03:48:21.544881+0000","last_undegraded":"2023-11-09T03:48:21.544881+0000","last_fullsized":"2023-11-09T03:48:21.544881+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:57:56.855869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542191+0000","last_change":"2023-11-09T03:48:02.681869+0000","last_active":"2023-11-09T03:48:21.542191+0000","last_peered":"2023-11-09T03:48:21.542191+0000","last_clean":"2023-11-09T03:48:21.542191+0000","last_became_active":"2023-11-09T03:48:00.444928+0000","last_became_peered":"2023-11-09T03:48:00.444928+0000","last_unstale":"2023-11-09T03:48:21.542191+0000","last_undegraded":"2023-11-09T03:48:21.542191+0000","last_fullsized":"2023-11-09T03:48:21.542191+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:59:58.695636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044822599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542292+0000","last_change":"2023-11-09T03:47:03.146783+0000","last_active":"2023-11-09T03:48:21.542292+0000","last_peered":"2023-11-09T03:48:21.542292+0000","last_clean":"2023-11-09T03:48:21.542292+0000","last_became_active":"2023-11-09T03:47:01.586923+0000","last_became_peered":"2023-11-09T03:47:01.586923+0000","last_unstale":"2023-11-09T03:48:21.542292+0000","last_undegraded":"2023-11-09T03:48:21.542292+0000","last_fullsized":"2023-11-09T03:48:21.542292+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:37:50.326442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060384700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.549765+0000","last_change":"2023-11-09T03:48:22.549765+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.549765+0000","last_undegraded":"2023-11-09T03:48:22.549765+0000","last_fullsized":"2023-11-09T03:48:22.549765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.571830+0000","last_change":"2023-11-09T03:48:22.565142+0000","last_active":"2023-11-09T03:48:22.571830+0000","last_peered":"2023-11-09T03:48:22.571830+0000","last_clean":"2023-11-09T03:48:22.571830+0000","last_became_active":"2023-11-09T03:48:22.564856+0000","last_became_peered":"2023-11-09T03:48:22.564856+0000","last_unstale":"2023-11-09T03:48:22.571830+0000","last_undegraded":"2023-11-09T03:48:22.571830+0000","last_fullsized":"2023-11-09T03:48:22.571830+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:25:04.111498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542575+0000","last_change":"2023-11-09T03:48:01.448383+0000","last_active":"2023-11-09T03:48:21.542575+0000","last_peered":"2023-11-09T03:48:21.542575+0000","last_clean":"2023-11-09T03:48:21.542575+0000","last_became_active":"2023-11-09T03:48:01.448050+0000","last_became_peered":"2023-11-09T03:48:01.448050+0000","last_unstale":"2023-11-09T03:48:21.542575+0000","last_undegraded":"2023-11-09T03:48:21.542575+0000","last_fullsized":"2023-11-09T03:48:21.542575+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:53:44.478182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.541748+0000","last_change":"2023-11-09T03:48:02.523794+0000","last_active":"2023-11-09T03:48:21.541748+0000","last_peered":"2023-11-09T03:48:21.541748+0000","last_clean":"2023-11-09T03:48:21.541748+0000","last_became_active":"2023-11-09T03:48:00.440255+0000","last_became_peered":"2023-11-09T03:48:00.440255+0000","last_unstale":"2023-11-09T03:48:21.541748+0000","last_undegraded":"2023-11-09T03:48:21.541748+0000","last_fullsized":"2023-11-09T03:48:21.541748+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:58:02.392696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064572299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.542252+0000","last_change":"2023-11-09T03:47:03.146580+0000","last_active":"2023-11-09T03:48:21.542252+0000","last_peered":"2023-11-09T03:48:21.542252+0000","last_clean":"2023-11-09T03:48:21.542252+0000","last_became_active":"2023-11-09T03:47:01.589201+0000","last_became_peered":"2023-11-09T03:47:01.589201+0000","last_unstale":"2023-11-09T03:48:21.542252+0000","last_undegraded":"2023-11-09T03:48:21.542252+0000","last_fullsized":"2023-11-09T03:48:21.542252+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:14.300421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049706999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-11-09T03:48:22.551049+0000","last_change":"2023-11-09T03:48:22.551049+0000","last_active":"2023-11-09T03:48:22.537305+0000","last_peered":"2023-11-09T03:48:22.537305+0000","last_clean":"2023-11-09T03:48:22.537305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-09T03:48:22.551049+0000","last_undegraded":"2023-11-09T03:48:22.551049+0000","last_fullsized":"2023-11-09T03:48:22.551049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-09T03:48:22.566108+0000","last_change":"2023-11-09T03:48:22.562609+0000","last_active":"2023-11-09T03:48:22.566108+0000","last_peered":"2023-11-09T03:48:22.566108+0000","last_clean":"2023-11-09T03:48:22.566108+0000","last_became_active":"2023-11-09T03:48:22.562328+0000","last_became_peered":"2023-11-09T03:48:22.562328+0000","last_unstale":"2023-11-09T03:48:22.566108+0000","last_undegraded":"2023-11-09T03:48:22.566108+0000","last_fullsized":"2023-11-09T03:48:22.566108+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:57:25.589693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.539239+0000","last_change":"2023-11-09T03:48:01.451284+0000","last_active":"2023-11-09T03:48:21.539239+0000","last_peered":"2023-11-09T03:48:21.539239+0000","last_clean":"2023-11-09T03:48:21.539239+0000","last_became_active":"2023-11-09T03:48:01.450954+0000","last_became_peered":"2023-11-09T03:48:01.450954+0000","last_unstale":"2023-11-09T03:48:21.539239+0000","last_undegraded":"2023-11-09T03:48:21.539239+0000","last_fullsized":"2023-11-09T03:48:21.539239+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:49:24.680020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545344+0000","last_change":"2023-11-09T03:48:02.733038+0000","last_active":"2023-11-09T03:48:21.545344+0000","last_peered":"2023-11-09T03:48:21.545344+0000","last_clean":"2023-11-09T03:48:21.545344+0000","last_became_active":"2023-11-09T03:48:00.446661+0000","last_became_peered":"2023-11-09T03:48:00.446661+0000","last_unstale":"2023-11-09T03:48:21.545344+0000","last_undegraded":"2023-11-09T03:48:21.545344+0000","last_fullsized":"2023-11-09T03:48:21.545344+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:05:08.649312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040102300000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.547989+0000","last_change":"2023-11-09T03:47:03.147851+0000","last_active":"2023-11-09T03:48:21.547989+0000","last_peered":"2023-11-09T03:48:21.547989+0000","last_clean":"2023-11-09T03:48:21.547989+0000","last_became_active":"2023-11-09T03:47:01.589893+0000","last_became_peered":"2023-11-09T03:47:01.589893+0000","last_unstale":"2023-11-09T03:48:21.547989+0000","last_undegraded":"2023-11-09T03:48:21.547989+0000","last_fullsized":"2023-11-09T03:48:21.547989+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:08:17.183462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029151399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":215,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-09T03:48:21.545358+0000","last_change":"2023-11-09T03:45:21.030926+0000","last_active":"2023-11-09T03:48:21.545358+0000","last_peered":"2023-11-09T03:48:21.545358+0000","last_clean":"2023-11-09T03:48:21.545358+0000","last_became_active":"2023-11-09T03:45:21.029801+0000","last_became_peered":"2023-11-09T03:45:21.029801+0000","last_unstale":"2023-11-09T03:48:21.545358+0000","last_undegraded":"2023-11-09T03:48:21.545358+0000","last_fullsized":"2023-11-09T03:48:21.545358+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_clean_scrub_stamp":"2023-11-09T03:45:19.768433+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:12:59.153713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1572864,"data_stored":1058844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.501,"15min":0.498},"min":{"1min":0.34899999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.41199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.59899999999999998}]},{"osd":1,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.33000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.38100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.57799999999999996}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.52300000000000002,"15min":0.51400000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.68700000000000006,"15min":0.70799999999999996},"min":{"1min":0.41299999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.80700000000000005,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.628}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.48699999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61599999999999999}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.441},"min":{"1min":0.34100000000000003,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.49199999999999999,"15min":0.47899999999999998},"min":{"1min":0.38700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.42799999999999999}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.47199999999999998,"15min":0.45600000000000002},"min":{"1min":0.37,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.51100000000000001,"15min":0.497},"min":{"1min":0.39600000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.54800000000000004}]},{"osd":6,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55100000000000005,"15min":0.54100000000000004},"min":{"1min":0.44400000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.496},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.434,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.48099999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":991232,"data_stored":476134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.499,"15min":0.501},"min":{"1min":0.35099999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63900000000000001,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.50600000000000001,"15min":0.495},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.51100000000000001}]},{"osd":1,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.375,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.58299999999999996}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.503,"15min":0.503},"min":{"1min":0.316,"5min":0.313,"15min":0.313},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.437},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.45700000000000002}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.436,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55500000000000005,"15min":0.55800000000000005},"min":{"1min":0.34300000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.42899999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.44400000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.56100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":978944,"data_stored":468476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.504,"15min":0.48899999999999999},"min":{"1min":0.36199999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.503},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.376,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.61299999999999999}]},{"osd":1,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88300000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.36599999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.84199999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.65900000000000003}]},{"osd":2,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.45800000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.55600000000000005},"min":{"1min":0.40300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.71099999999999997}]},{"osd":3,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57099999999999995,"15min":0.56499999999999995},"min":{"1min":0.39400000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55200000000000005,"15min":0.55100000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.76100000000000001}]},{"osd":4,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.49199999999999999,"15min":0.48399999999999999},"min":{"1min":0.32400000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.48999999999999999,"15min":0.48299999999999998},"min":{"1min":0.29699999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.57899999999999996}]},{"osd":6,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.50600000000000001,"15min":0.51000000000000001},"min":{"1min":0.33700000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.755,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.53300000000000003,"15min":0.51700000000000002},"min":{"1min":0.38300000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.64400000000000002}]},{"osd":7,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.34699999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.81200000000000006,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.61899999999999999,"15min":0.62},"min":{"1min":0.432,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84899999999999998,"5min":1.01,"15min":1.01},"last":0.79000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":978944,"data_stored":468476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69399999999999995,"5min":1.034,"15min":1.034},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.38800000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78300000000000003,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.34000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.74299999999999999,"5min":1.093,"15min":1.093},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54200000000000004,"15min":0.54100000000000004},"min":{"1min":0.36399999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71899999999999997,"5min":0.871,"15min":0.871},"last":0.54900000000000004}]},{"osd":2,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54100000000000004,"15min":0.55400000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.69199999999999995,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54800000000000004,"15min":0.54500000000000004},"min":{"1min":0.39700000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.51800000000000002}]},{"osd":3,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.433,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.70199999999999996,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.376,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72499999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.379}]},{"osd":5,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.33300000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.78700000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.318,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81100000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.35799999999999998}]},{"osd":6,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.75800000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57899999999999996,"15min":0.58399999999999996},"min":{"1min":0.39600000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.75900000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.39600000000000002}]},{"osd":7,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61599999999999999,"15min":0.629},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83499999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61099999999999999,"15min":0.61899999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79300000000000004,"5min":1.113,"15min":1.113},"last":0.45500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":978944,"data_stored":468476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.32100000000000001,"15min":0.32300000000000001},"min":{"1min":0.22,"5min":0.215,"15min":0.215},"max":{"1min":0.40200000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.313,"5min":0.32300000000000001,"15min":0.32400000000000001},"min":{"1min":0.23999999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.47799999999999998,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.23999999999999999}]},{"osd":1,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.32600000000000001,"15min":0.33000000000000002},"min":{"1min":0.23599999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.42899999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.36699999999999999,"15min":0.373},"min":{"1min":0.28100000000000003,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.47899999999999998,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.36599999999999999}]},{"osd":2,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.37,"15min":0.372},"min":{"1min":0.27200000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.439,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.312},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.379,"15min":0.38300000000000001},"min":{"1min":0.29799999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.45700000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.38400000000000001}]},{"osd":4,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58899999999999997,"15min":0.59499999999999997},"min":{"1min":0.42299999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79000000000000004,"5min":0.87,"15min":0.87},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.59699999999999998,"15min":0.61099999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.434}]},{"osd":5,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.60499999999999998,"15min":0.62},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86599999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60699999999999998,"15min":0.61899999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.82099999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.443}]},{"osd":6,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60999999999999999,"15min":0.623},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78200000000000003,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.45400000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.80300000000000005,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.55200000000000005}]},{"osd":7,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.61099999999999999,"15min":0.62},"min":{"1min":0.41099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.70899999999999996,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.621,"15min":0.63700000000000001},"min":{"1min":0.46100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81299999999999994,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.46100000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":991232,"data_stored":476134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.46000000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.33400000000000002,"15min":0.33100000000000002},"min":{"1min":0.247,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.29399999999999998}]},{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32500000000000001,"15min":0.32600000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.495,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.35899999999999999,"15min":0.36299999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.48599999999999999,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.36699999999999999}]},{"osd":3,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.36199999999999999,"15min":0.36399999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.45700000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.312},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40200000000000002,"15min":0.40400000000000003},"min":{"1min":0.30299999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.52300000000000002,"5min":0.754,"15min":0.754},"last":0.39900000000000002}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72299999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81299999999999994,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.46000000000000002}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.39800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.75700000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.59599999999999997,"15min":0.60099999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.753,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.498}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60399999999999998,"15min":0.60899999999999999},"min":{"1min":0.44600000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.76100000000000001,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.71199999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60699999999999998,"15min":0.61199999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.745,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.80000000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.47299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":978944,"data_stored":468476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.28699999999999998,"5min":0.31,"15min":0.314},"min":{"1min":0.20799999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.38900000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"last":0.309},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.315,"15min":0.311},"min":{"1min":0.24099999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"last":0.29299999999999998}]},{"osd":2,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.311,"15min":0.309},"min":{"1min":0.253,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.378,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.34999999999999998}]},{"osd":3,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36899999999999999,"15min":0.371},"min":{"1min":0.23899999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.47699999999999998,"5min":0.505,"15min":0.505},"last":0.379},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.37,"15min":0.371},"min":{"1min":0.30299999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.45400000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.41899999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.747,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.44400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.77600000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.58699999999999997}]},{"osd":5,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56200000000000006,"15min":0.55700000000000005},"min":{"1min":0.46100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.44500000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.72599999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.47799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56699999999999995,"15min":0.56799999999999995},"min":{"1min":0.433,"5min":0.379,"15min":0.379},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.53300000000000003}]},{"osd":7,"last update":"Thu Nov 9 03:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.42899999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.69799999999999995,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.72899999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.54700000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1568768,"data_stored":1058225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.31900000000000001,"15min":0.32300000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.38200000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.29599999999999999,"5min":0.35499999999999998,"15min":0.36499999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.44500000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.27300000000000002}]},{"osd":2,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.35699999999999998,"15min":0.36399999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.44700000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.376},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.36699999999999999,"15min":0.371},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.47699999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.316}]},{"osd":3,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.38500000000000001,"15min":0.39100000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.46400000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.41499999999999998,"15min":0.42099999999999999},"min":{"1min":0.309,"5min":0.307,"15min":0.307},"max":{"1min":0.48999999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58899999999999997,"15min":0.59299999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60599999999999998,"15min":0.60899999999999999},"min":{"1min":0.436,"5min":0.318,"15min":0.318},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.60299999999999998}]},{"osd":5,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.36199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76800000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.629,"15min":0.63600000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78900000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.64100000000000001,"15min":0.64700000000000002},"min":{"1min":0.42499999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81000000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.627,"15min":0.63200000000000001},"min":{"1min":0.41699999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Nov 9 03:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.44400000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.42999999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.71799999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.55600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-09T03:48:30.917 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-09T03:48:31.008 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:48:31.011 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:48:31.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.028+0000 7efd47a7c640 1 Processor -- start 2023-11-09T03:48:31.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.028+0000 7efd47a7c640 1 -- start start 2023-11-09T03:48:31.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.029+0000 7efd47a7c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd400cec10 0x7efd400b2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:31.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.029+0000 7efd47a7c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd400b3c10 con 0x7efd4015ac70 2023-11-09T03:48:31.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.029+0000 7efd47a7c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd400cf180 con 0x7efd400b2b40 2023-11-09T03:48:31.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.029+0000 7efd47a7c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd400b2fb0 con 0x7efd400cec10 2023-11-09T03:48:31.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.029+0000 7efd45818640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7efd4015ac70 0x7efd400d0330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59072/0 (socket says 172.21.15.76:59072) 2023-11-09T03:48:31.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.029+0000 7efd45818640 1 -- 172.21.15.76:0/527714229 learned_addr learned my addr 172.21.15.76:0/527714229 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:31.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.029+0000 7efd45017640 1 --2- 172.21.15.76:0/527714229 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd400cec10 0x7efd400b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:31.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.029+0000 7efd44816640 1 -- 172.21.15.76:0/527714229 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3630224857 0 0) 0x7efd400b3c10 con 0x7efd4015ac70 2023-11-09T03:48:31.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.029+0000 7efd44816640 1 -- 172.21.15.76:0/527714229 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd2c003520 con 0x7efd4015ac70 2023-11-09T03:48:31.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.030+0000 7efd44816640 1 -- 172.21.15.76:0/527714229 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1539618837 0 0) 0x7efd400cf180 con 0x7efd400b2b40 2023-11-09T03:48:31.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.030+0000 7efd44816640 1 -- 172.21.15.76:0/527714229 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd400b3c10 con 0x7efd400b2b40 2023-11-09T03:48:31.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.030+0000 7efd45017640 1 -- 172.21.15.76:0/527714229 >> v1:172.21.15.113:6789/0 conn(0x7efd400b2b40 legacy=0x7efd400b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:31.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.030+0000 7efd45017640 1 -- 172.21.15.76:0/527714229 >> v1:172.21.15.76:6789/0 conn(0x7efd4015ac70 legacy=0x7efd400d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:31.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.030+0000 7efd44816640 1 -- 172.21.15.76:0/527714229 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1245780390 0 0) 0x7efd2c003520 con 0x7efd4015ac70 2023-11-09T03:48:31.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.030+0000 7efd45017640 1 -- 172.21.15.76:0/527714229 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd400cf460 con 0x7efd400cec10 2023-11-09T03:48:31.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.031+0000 7efd45017640 1 --2- 172.21.15.76:0/527714229 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd400cec10 0x7efd400b2600 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0097d0 tx=0x7efd3c02b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26db3964e9bdf15a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:31.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.031+0000 7efd44816640 1 -- 172.21.15.76:0/527714229 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd30001810 con 0x7efd400cec10 2023-11-09T03:48:31.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.031+0000 7efd44816640 1 -- 172.21.15.76:0/527714229 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3c02e050 con 0x7efd400cec10 2023-11-09T03:48:31.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.031+0000 7efd44816640 1 -- 172.21.15.76:0/527714229 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c030730 con 0x7efd400cec10 2023-11-09T03:48:31.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.032+0000 7efd47a7c640 1 -- 172.21.15.76:0/527714229 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd400cec10 msgr2=0x7efd400b2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:31.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.032+0000 7efd47a7c640 1 --2- 172.21.15.76:0/527714229 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd400cec10 0x7efd400b2600 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0097d0 tx=0x7efd3c02b800 comp rx=0 tx=0).stop 2023-11-09T03:48:31.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.032+0000 7efd47a7c640 1 -- 172.21.15.76:0/527714229 shutdown_connections 2023-11-09T03:48:31.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.032+0000 7efd47a7c640 1 --2- 172.21.15.76:0/527714229 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd400cec10 0x7efd400b2600 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.032+0000 7efd47a7c640 1 -- 172.21.15.76:0/527714229 >> 172.21.15.76:0/527714229 conn(0x7efd400cd600 msgr2=0x7efd400cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:31.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.032+0000 7efd47a7c640 1 -- 172.21.15.76:0/527714229 shutdown_connections 2023-11-09T03:48:31.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.032+0000 7efd47a7c640 1 -- 172.21.15.76:0/527714229 wait complete. 2023-11-09T03:48:31.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.033+0000 7efd47a7c640 1 Processor -- start 2023-11-09T03:48:31.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.033+0000 7efd47a7c640 1 -- start start 2023-11-09T03:48:31.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.034+0000 7efd47a7c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd400b2b40 0x7efd400c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:31.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.034+0000 7efd47a7c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd400cec10 0x7efd4014f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:31.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.034+0000 7efd47a7c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd4015ac70 0x7efd400c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:31.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.034+0000 7efd47a7c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd40154060 con 0x7efd400b2b40 2023-11-09T03:48:31.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.034+0000 7efd47a7c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd400c5d30 con 0x7efd400cec10 2023-11-09T03:48:31.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.034+0000 7efd47a7c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd4014f8f0 con 0x7efd4015ac70 2023-11-09T03:48:31.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.035+0000 7efd45818640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd400b2b40 0x7efd400c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:31.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.035+0000 7efd45818640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd400b2b40 0x7efd400c57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37908/0 (socket says 172.21.15.76:37908) 2023-11-09T03:48:31.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.035+0000 7efd45818640 1 -- 172.21.15.76:0/3166239865 learned_addr learned my addr 172.21.15.76:0/3166239865 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:31.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.035+0000 7efd46019640 1 --2- 172.21.15.76:0/3166239865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd4015ac70 0x7efd400c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:31.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.035+0000 7efd45017640 1 --2- 172.21.15.76:0/3166239865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd400cec10 0x7efd4014f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:31.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.035+0000 7efd45818640 1 -- 172.21.15.76:0/3166239865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd4015ac70 msgr2=0x7efd400c68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:31.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.035+0000 7efd45818640 1 --2- 172.21.15.76:0/3166239865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd4015ac70 0x7efd400c68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.035+0000 7efd45818640 1 -- 172.21.15.76:0/3166239865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd400cec10 msgr2=0x7efd4014f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:31.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.035+0000 7efd45818640 1 --2- 172.21.15.76:0/3166239865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd400cec10 0x7efd4014f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.035+0000 7efd45818640 1 -- 172.21.15.76:0/3166239865 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3c009400 con 0x7efd400b2b40 2023-11-09T03:48:31.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.035+0000 7efd45017640 1 --2- 172.21.15.76:0/3166239865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd400cec10 0x7efd4014f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:31.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.035+0000 7efd45818640 1 --2- 172.21.15.76:0/3166239865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd400b2b40 0x7efd400c57f0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7efd3000c5a0 tx=0x7efd3000ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:31.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.035+0000 7efd367fc640 1 -- 172.21.15.76:0/3166239865 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd30007480 con 0x7efd400b2b40 2023-11-09T03:48:31.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.035+0000 7efd367fc640 1 -- 172.21.15.76:0/3166239865 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd30007600 con 0x7efd400b2b40 2023-11-09T03:48:31.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.035+0000 7efd47a7c640 1 -- 172.21.15.76:0/3166239865 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd400c8c20 con 0x7efd400b2b40 2023-11-09T03:48:31.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.035+0000 7efd367fc640 1 -- 172.21.15.76:0/3166239865 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd300059e0 con 0x7efd400b2b40 2023-11-09T03:48:31.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.035+0000 7efd47a7c640 1 -- 172.21.15.76:0/3166239865 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd400c91e0 con 0x7efd400b2b40 2023-11-09T03:48:31.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.036+0000 7efd47a7c640 1 -- 172.21.15.76:0/3166239865 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd40153e30 con 0x7efd400b2b40 2023-11-09T03:48:31.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.037+0000 7efd367fc640 1 -- 172.21.15.76:0/3166239865 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efd30005b40 con 0x7efd400b2b40 2023-11-09T03:48:31.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.037+0000 7efd367fc640 1 --2- 172.21.15.76:0/3166239865 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efd2406ae70 0x7efd2406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:31.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.037+0000 7efd367fc640 1 -- 172.21.15.76:0/3166239865 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7efd300892f0 con 0x7efd400b2b40 2023-11-09T03:48:31.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.037+0000 7efd45017640 1 --2- 172.21.15.76:0/3166239865 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efd2406ae70 0x7efd2406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:31.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.038+0000 7efd45017640 1 --2- 172.21.15.76:0/3166239865 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efd2406ae70 0x7efd2406d330 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7efd3c034000 tx=0x7efd3c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:31.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.041+0000 7efd367fc640 1 -- 172.21.15.76:0/3166239865 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efd40153e30 con 0x7efd400b2b40 2023-11-09T03:48:31.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.205+0000 7efd47a7c640 1 -- 172.21.15.76:0/3166239865 --> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efd400cfa20 con 0x7efd2406ae70 2023-11-09T03:48:31.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.219+0000 7efd367fc640 1 -- 172.21.15.76:0/3166239865 <== mgr.4105 v2:172.21.15.113:6832/1264193042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+532096 (secure 0 0 0) 0x7efd3c040070 con 0x7efd2406ae70 2023-11-09T03:48:31.223 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:31.227 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2023-11-09T03:48:31.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.223+0000 7efd47a7c640 1 -- 172.21.15.76:0/3166239865 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efd2406ae70 msgr2=0x7efd2406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:31.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.223+0000 7efd47a7c640 1 --2- 172.21.15.76:0/3166239865 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efd2406ae70 0x7efd2406d330 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7efd3c034000 tx=0x7efd3c008030 comp rx=0 tx=0).stop 2023-11-09T03:48:31.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.223+0000 7efd47a7c640 1 -- 172.21.15.76:0/3166239865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd400b2b40 msgr2=0x7efd400c57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:31.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.223+0000 7efd47a7c640 1 --2- 172.21.15.76:0/3166239865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd400b2b40 0x7efd400c57f0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7efd3000c5a0 tx=0x7efd3000ca60 comp rx=0 tx=0).stop 2023-11-09T03:48:31.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.223+0000 7efd47a7c640 1 -- 172.21.15.76:0/3166239865 shutdown_connections 2023-11-09T03:48:31.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.223+0000 7efd47a7c640 1 --2- 172.21.15.76:0/3166239865 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efd2406ae70 0x7efd2406d330 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.223+0000 7efd47a7c640 1 --2- 172.21.15.76:0/3166239865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd4015ac70 0x7efd400c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.223+0000 7efd47a7c640 1 --2- 172.21.15.76:0/3166239865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd400cec10 0x7efd4014f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.223+0000 7efd47a7c640 1 --2- 172.21.15.76:0/3166239865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd400b2b40 0x7efd400c57f0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.223+0000 7efd47a7c640 1 -- 172.21.15.76:0/3166239865 >> 172.21.15.76:0/3166239865 conn(0x7efd400cd600 msgr2=0x7efd400708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:31.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.223+0000 7efd47a7c640 1 -- 172.21.15.76:0/3166239865 shutdown_connections 2023-11-09T03:48:31.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.223+0000 7efd47a7c640 1 -- 172.21.15.76:0/3166239865 wait complete. 2023-11-09T03:48:31.233 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2023-11-09T03:48:30.795682+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237852,"kb_used_data":9156,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754736868,"statfs":{"total":773094113280,"available":772850552832,"internally_reserved":0,"allocated":9375744,"data_stored":5120737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002637"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.736288+0000","last_change":"2023-11-09T03:48:23.564306+0000","last_active":"2023-11-09T03:48:24.736288+0000","last_peered":"2023-11-09T03:48:24.736288+0000","last_clean":"2023-11-09T03:48:24.736288+0000","last_became_active":"2023-11-09T03:48:23.564015+0000","last_became_peered":"2023-11-09T03:48:23.564015+0000","last_unstale":"2023-11-09T03:48:24.736288+0000","last_undegraded":"2023-11-09T03:48:24.736288+0000","last_fullsized":"2023-11-09T03:48:24.736288+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:09:10.046513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.737991+0000","last_change":"2023-11-09T03:48:23.564535+0000","last_active":"2023-11-09T03:48:24.737991+0000","last_peered":"2023-11-09T03:48:24.737991+0000","last_clean":"2023-11-09T03:48:24.737991+0000","last_became_active":"2023-11-09T03:48:23.564222+0000","last_became_peered":"2023-11-09T03:48:23.564222+0000","last_unstale":"2023-11-09T03:48:24.737991+0000","last_undegraded":"2023-11-09T03:48:24.737991+0000","last_fullsized":"2023-11-09T03:48:24.737991+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:25:12.430709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.736292+0000","last_change":"2023-11-09T03:48:23.563977+0000","last_active":"2023-11-09T03:48:24.736292+0000","last_peered":"2023-11-09T03:48:24.736292+0000","last_clean":"2023-11-09T03:48:24.736292+0000","last_became_active":"2023-11-09T03:48:23.563703+0000","last_became_peered":"2023-11-09T03:48:23.563703+0000","last_unstale":"2023-11-09T03:48:24.736292+0000","last_undegraded":"2023-11-09T03:48:24.736292+0000","last_fullsized":"2023-11-09T03:48:24.736292+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:05:31.769892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.556111+0000","last_change":"2023-11-09T03:48:23.574171+0000","last_active":"2023-11-09T03:48:24.556111+0000","last_peered":"2023-11-09T03:48:24.556111+0000","last_clean":"2023-11-09T03:48:24.556111+0000","last_became_active":"2023-11-09T03:48:23.573886+0000","last_became_peered":"2023-11-09T03:48:23.573886+0000","last_unstale":"2023-11-09T03:48:24.556111+0000","last_undegraded":"2023-11-09T03:48:24.556111+0000","last_fullsized":"2023-11-09T03:48:24.556111+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:45:43.244544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.736161+0000","last_change":"2023-11-09T03:48:23.567195+0000","last_active":"2023-11-09T03:48:24.736161+0000","last_peered":"2023-11-09T03:48:24.736161+0000","last_clean":"2023-11-09T03:48:24.736161+0000","last_became_active":"2023-11-09T03:48:23.566906+0000","last_became_peered":"2023-11-09T03:48:23.566906+0000","last_unstale":"2023-11-09T03:48:24.736161+0000","last_undegraded":"2023-11-09T03:48:24.736161+0000","last_fullsized":"2023-11-09T03:48:24.736161+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:24:49.225093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.558265+0000","last_change":"2023-11-09T03:48:23.563150+0000","last_active":"2023-11-09T03:48:24.558265+0000","last_peered":"2023-11-09T03:48:24.558265+0000","last_clean":"2023-11-09T03:48:24.558265+0000","last_became_active":"2023-11-09T03:48:23.562888+0000","last_became_peered":"2023-11-09T03:48:23.562888+0000","last_unstale":"2023-11-09T03:48:24.558265+0000","last_undegraded":"2023-11-09T03:48:24.558265+0000","last_fullsized":"2023-11-09T03:48:24.558265+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:32:37.449196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.558114+0000","last_change":"2023-11-09T03:48:23.568186+0000","last_active":"2023-11-09T03:48:24.558114+0000","last_peered":"2023-11-09T03:48:24.558114+0000","last_clean":"2023-11-09T03:48:24.558114+0000","last_became_active":"2023-11-09T03:48:23.567916+0000","last_became_peered":"2023-11-09T03:48:23.567916+0000","last_unstale":"2023-11-09T03:48:24.558114+0000","last_undegraded":"2023-11-09T03:48:24.558114+0000","last_fullsized":"2023-11-09T03:48:24.558114+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:09:34.179452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.736125+0000","last_change":"2023-11-09T03:48:23.561106+0000","last_active":"2023-11-09T03:48:24.736125+0000","last_peered":"2023-11-09T03:48:24.736125+0000","last_clean":"2023-11-09T03:48:24.736125+0000","last_became_active":"2023-11-09T03:48:23.560823+0000","last_became_peered":"2023-11-09T03:48:23.560823+0000","last_unstale":"2023-11-09T03:48:24.736125+0000","last_undegraded":"2023-11-09T03:48:24.736125+0000","last_fullsized":"2023-11-09T03:48:24.736125+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:44:30.239622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.738065+0000","last_change":"2023-11-09T03:48:23.568675+0000","last_active":"2023-11-09T03:48:24.738065+0000","last_peered":"2023-11-09T03:48:24.738065+0000","last_clean":"2023-11-09T03:48:24.738065+0000","last_became_active":"2023-11-09T03:48:23.568363+0000","last_became_peered":"2023-11-09T03:48:23.568363+0000","last_unstale":"2023-11-09T03:48:24.738065+0000","last_undegraded":"2023-11-09T03:48:24.738065+0000","last_fullsized":"2023-11-09T03:48:24.738065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:28:07.562442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.736313+0000","last_change":"2023-11-09T03:48:23.567572+0000","last_active":"2023-11-09T03:48:24.736313+0000","last_peered":"2023-11-09T03:48:24.736313+0000","last_clean":"2023-11-09T03:48:24.736313+0000","last_became_active":"2023-11-09T03:48:23.567277+0000","last_became_peered":"2023-11-09T03:48:23.567277+0000","last_unstale":"2023-11-09T03:48:24.736313+0000","last_undegraded":"2023-11-09T03:48:24.736313+0000","last_fullsized":"2023-11-09T03:48:24.736313+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:43:02.920090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.558402+0000","last_change":"2023-11-09T03:48:23.566095+0000","last_active":"2023-11-09T03:48:24.558402+0000","last_peered":"2023-11-09T03:48:24.558402+0000","last_clean":"2023-11-09T03:48:24.558402+0000","last_became_active":"2023-11-09T03:48:23.565829+0000","last_became_peered":"2023-11-09T03:48:23.565829+0000","last_unstale":"2023-11-09T03:48:24.558402+0000","last_undegraded":"2023-11-09T03:48:24.558402+0000","last_fullsized":"2023-11-09T03:48:24.558402+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:33:11.589157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.737176+0000","last_change":"2023-11-09T03:48:23.560858+0000","last_active":"2023-11-09T03:48:24.737176+0000","last_peered":"2023-11-09T03:48:24.737176+0000","last_clean":"2023-11-09T03:48:24.737176+0000","last_became_active":"2023-11-09T03:48:23.560311+0000","last_became_peered":"2023-11-09T03:48:23.560311+0000","last_unstale":"2023-11-09T03:48:24.737176+0000","last_undegraded":"2023-11-09T03:48:24.737176+0000","last_fullsized":"2023-11-09T03:48:24.737176+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:22.477174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.556004+0000","last_change":"2023-11-09T03:48:23.572965+0000","last_active":"2023-11-09T03:48:24.556004+0000","last_peered":"2023-11-09T03:48:24.556004+0000","last_clean":"2023-11-09T03:48:24.556004+0000","last_became_active":"2023-11-09T03:48:23.572717+0000","last_became_peered":"2023-11-09T03:48:23.572717+0000","last_unstale":"2023-11-09T03:48:24.556004+0000","last_undegraded":"2023-11-09T03:48:24.556004+0000","last_fullsized":"2023-11-09T03:48:24.556004+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:36:19.969871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.735453+0000","last_change":"2023-11-09T03:48:23.560352+0000","last_active":"2023-11-09T03:48:24.735453+0000","last_peered":"2023-11-09T03:48:24.735453+0000","last_clean":"2023-11-09T03:48:24.735453+0000","last_became_active":"2023-11-09T03:48:23.560081+0000","last_became_peered":"2023-11-09T03:48:23.560081+0000","last_unstale":"2023-11-09T03:48:24.735453+0000","last_undegraded":"2023-11-09T03:48:24.735453+0000","last_fullsized":"2023-11-09T03:48:24.735453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:11:48.838177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.555759+0000","last_change":"2023-11-09T03:48:23.566228+0000","last_active":"2023-11-09T03:48:24.555759+0000","last_peered":"2023-11-09T03:48:24.555759+0000","last_clean":"2023-11-09T03:48:24.555759+0000","last_became_active":"2023-11-09T03:48:23.563775+0000","last_became_peered":"2023-11-09T03:48:23.563775+0000","last_unstale":"2023-11-09T03:48:24.555759+0000","last_undegraded":"2023-11-09T03:48:24.555759+0000","last_fullsized":"2023-11-09T03:48:24.555759+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:17:46.479994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.557989+0000","last_change":"2023-11-09T03:48:23.558913+0000","last_active":"2023-11-09T03:48:24.557989+0000","last_peered":"2023-11-09T03:48:24.557989+0000","last_clean":"2023-11-09T03:48:24.557989+0000","last_became_active":"2023-11-09T03:48:23.558615+0000","last_became_peered":"2023-11-09T03:48:23.558615+0000","last_unstale":"2023-11-09T03:48:24.557989+0000","last_undegraded":"2023-11-09T03:48:24.557989+0000","last_fullsized":"2023-11-09T03:48:24.557989+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:20:33.663992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.737473+0000","last_change":"2023-11-09T03:48:23.567436+0000","last_active":"2023-11-09T03:48:24.737473+0000","last_peered":"2023-11-09T03:48:24.737473+0000","last_clean":"2023-11-09T03:48:24.737473+0000","last_became_active":"2023-11-09T03:48:23.567176+0000","last_became_peered":"2023-11-09T03:48:23.567176+0000","last_unstale":"2023-11-09T03:48:24.737473+0000","last_undegraded":"2023-11-09T03:48:24.737473+0000","last_fullsized":"2023-11-09T03:48:24.737473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:20:56.927411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.556301+0000","last_change":"2023-11-09T03:48:23.575111+0000","last_active":"2023-11-09T03:48:24.556301+0000","last_peered":"2023-11-09T03:48:24.556301+0000","last_clean":"2023-11-09T03:48:24.556301+0000","last_became_active":"2023-11-09T03:48:23.574850+0000","last_became_peered":"2023-11-09T03:48:23.574850+0000","last_unstale":"2023-11-09T03:48:24.556301+0000","last_undegraded":"2023-11-09T03:48:24.556301+0000","last_fullsized":"2023-11-09T03:48:24.556301+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:36:51.450660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.737066+0000","last_change":"2023-11-09T03:48:23.560218+0000","last_active":"2023-11-09T03:48:24.737066+0000","last_peered":"2023-11-09T03:48:24.737066+0000","last_clean":"2023-11-09T03:48:24.737066+0000","last_became_active":"2023-11-09T03:48:23.559906+0000","last_became_peered":"2023-11-09T03:48:23.559906+0000","last_unstale":"2023-11-09T03:48:24.737066+0000","last_undegraded":"2023-11-09T03:48:24.737066+0000","last_fullsized":"2023-11-09T03:48:24.737066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:37:31.971682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.737318+0000","last_change":"2023-11-09T03:48:23.568439+0000","last_active":"2023-11-09T03:48:24.737318+0000","last_peered":"2023-11-09T03:48:24.737318+0000","last_clean":"2023-11-09T03:48:24.737318+0000","last_became_active":"2023-11-09T03:48:23.567718+0000","last_became_peered":"2023-11-09T03:48:23.567718+0000","last_unstale":"2023-11-09T03:48:24.737318+0000","last_undegraded":"2023-11-09T03:48:24.737318+0000","last_fullsized":"2023-11-09T03:48:24.737318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:15:27.055519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.737448+0000","last_change":"2023-11-09T03:48:23.570578+0000","last_active":"2023-11-09T03:48:24.737448+0000","last_peered":"2023-11-09T03:48:24.737448+0000","last_clean":"2023-11-09T03:48:24.737448+0000","last_became_active":"2023-11-09T03:48:23.568127+0000","last_became_peered":"2023-11-09T03:48:23.568127+0000","last_unstale":"2023-11-09T03:48:24.737448+0000","last_undegraded":"2023-11-09T03:48:24.737448+0000","last_fullsized":"2023-11-09T03:48:24.737448+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:02:31.061488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.736839+0000","last_change":"2023-11-09T03:48:23.564780+0000","last_active":"2023-11-09T03:48:24.736839+0000","last_peered":"2023-11-09T03:48:24.736839+0000","last_clean":"2023-11-09T03:48:24.736839+0000","last_became_active":"2023-11-09T03:48:23.564505+0000","last_became_peered":"2023-11-09T03:48:23.564505+0000","last_unstale":"2023-11-09T03:48:24.736839+0000","last_undegraded":"2023-11-09T03:48:24.736839+0000","last_fullsized":"2023-11-09T03:48:24.736839+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:26.570438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.735824+0000","last_change":"2023-11-09T03:48:23.569253+0000","last_active":"2023-11-09T03:48:24.735824+0000","last_peered":"2023-11-09T03:48:24.735824+0000","last_clean":"2023-11-09T03:48:24.735824+0000","last_became_active":"2023-11-09T03:48:23.568986+0000","last_became_peered":"2023-11-09T03:48:23.568986+0000","last_unstale":"2023-11-09T03:48:24.735824+0000","last_undegraded":"2023-11-09T03:48:24.735824+0000","last_fullsized":"2023-11-09T03:48:24.735824+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:50:43.991293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.558150+0000","last_change":"2023-11-09T03:48:23.562715+0000","last_active":"2023-11-09T03:48:24.558150+0000","last_peered":"2023-11-09T03:48:24.558150+0000","last_clean":"2023-11-09T03:48:24.558150+0000","last_became_active":"2023-11-09T03:48:23.562445+0000","last_became_peered":"2023-11-09T03:48:23.562445+0000","last_unstale":"2023-11-09T03:48:24.558150+0000","last_undegraded":"2023-11-09T03:48:24.558150+0000","last_fullsized":"2023-11-09T03:48:24.558150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:53:56.398002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.737646+0000","last_change":"2023-11-09T03:48:23.566699+0000","last_active":"2023-11-09T03:48:24.737646+0000","last_peered":"2023-11-09T03:48:24.737646+0000","last_clean":"2023-11-09T03:48:24.737646+0000","last_became_active":"2023-11-09T03:48:23.566422+0000","last_became_peered":"2023-11-09T03:48:23.566422+0000","last_unstale":"2023-11-09T03:48:24.737646+0000","last_undegraded":"2023-11-09T03:48:24.737646+0000","last_fullsized":"2023-11-09T03:48:24.737646+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:59:18.876273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.737876+0000","last_change":"2023-11-09T03:48:23.565120+0000","last_active":"2023-11-09T03:48:24.737876+0000","last_peered":"2023-11-09T03:48:24.737876+0000","last_clean":"2023-11-09T03:48:24.737876+0000","last_became_active":"2023-11-09T03:48:23.564849+0000","last_became_peered":"2023-11-09T03:48:23.564849+0000","last_unstale":"2023-11-09T03:48:24.737876+0000","last_undegraded":"2023-11-09T03:48:24.737876+0000","last_fullsized":"2023-11-09T03:48:24.737876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:00:41.952528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.558635+0000","last_change":"2023-11-09T03:48:23.559888+0000","last_active":"2023-11-09T03:48:24.558635+0000","last_peered":"2023-11-09T03:48:24.558635+0000","last_clean":"2023-11-09T03:48:24.558635+0000","last_became_active":"2023-11-09T03:48:23.559644+0000","last_became_peered":"2023-11-09T03:48:23.559644+0000","last_unstale":"2023-11-09T03:48:24.558635+0000","last_undegraded":"2023-11-09T03:48:24.558635+0000","last_fullsized":"2023-11-09T03:48:24.558635+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:53:59.098346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.736885+0000","last_change":"2023-11-09T03:48:23.568003+0000","last_active":"2023-11-09T03:48:24.736885+0000","last_peered":"2023-11-09T03:48:24.736885+0000","last_clean":"2023-11-09T03:48:24.736885+0000","last_became_active":"2023-11-09T03:48:23.567707+0000","last_became_peered":"2023-11-09T03:48:23.567707+0000","last_unstale":"2023-11-09T03:48:24.736885+0000","last_undegraded":"2023-11-09T03:48:24.736885+0000","last_fullsized":"2023-11-09T03:48:24.736885+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:10:37.230874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.736774+0000","last_change":"2023-11-09T03:48:23.562640+0000","last_active":"2023-11-09T03:48:24.736774+0000","last_peered":"2023-11-09T03:48:24.736774+0000","last_clean":"2023-11-09T03:48:24.736774+0000","last_became_active":"2023-11-09T03:48:23.562369+0000","last_became_peered":"2023-11-09T03:48:23.562369+0000","last_unstale":"2023-11-09T03:48:24.736774+0000","last_undegraded":"2023-11-09T03:48:24.736774+0000","last_fullsized":"2023-11-09T03:48:24.736774+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:03:57.112002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.736932+0000","last_change":"2023-11-09T03:48:23.560875+0000","last_active":"2023-11-09T03:48:24.736932+0000","last_peered":"2023-11-09T03:48:24.736932+0000","last_clean":"2023-11-09T03:48:24.736932+0000","last_became_active":"2023-11-09T03:48:23.560625+0000","last_became_peered":"2023-11-09T03:48:23.560625+0000","last_unstale":"2023-11-09T03:48:24.736932+0000","last_undegraded":"2023-11-09T03:48:24.736932+0000","last_fullsized":"2023-11-09T03:48:24.736932+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:45:32.636828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.555944+0000","last_change":"2023-11-09T03:48:23.565144+0000","last_active":"2023-11-09T03:48:24.555944+0000","last_peered":"2023-11-09T03:48:24.555944+0000","last_clean":"2023-11-09T03:48:24.555944+0000","last_became_active":"2023-11-09T03:48:23.563500+0000","last_became_peered":"2023-11-09T03:48:23.563500+0000","last_unstale":"2023-11-09T03:48:24.555944+0000","last_undegraded":"2023-11-09T03:48:24.555944+0000","last_fullsized":"2023-11-09T03:48:24.555944+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:28:55.131717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.736790+0000","last_change":"2023-11-09T03:48:23.561978+0000","last_active":"2023-11-09T03:48:24.736790+0000","last_peered":"2023-11-09T03:48:24.736790+0000","last_clean":"2023-11-09T03:48:24.736790+0000","last_became_active":"2023-11-09T03:48:23.561155+0000","last_became_peered":"2023-11-09T03:48:23.561155+0000","last_unstale":"2023-11-09T03:48:24.736790+0000","last_undegraded":"2023-11-09T03:48:24.736790+0000","last_fullsized":"2023-11-09T03:48:24.736790+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:23:18.570676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.558423+0000","last_change":"2023-11-09T03:48:23.559262+0000","last_active":"2023-11-09T03:48:24.558423+0000","last_peered":"2023-11-09T03:48:24.558423+0000","last_clean":"2023-11-09T03:48:24.558423+0000","last_became_active":"2023-11-09T03:48:23.558945+0000","last_became_peered":"2023-11-09T03:48:23.558945+0000","last_unstale":"2023-11-09T03:48:24.558423+0000","last_undegraded":"2023-11-09T03:48:24.558423+0000","last_fullsized":"2023-11-09T03:48:24.558423+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:00:37.247407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"82'6","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.584718+0000","last_change":"2023-11-09T03:48:22.562765+0000","last_active":"2023-11-09T03:48:23.584718+0000","last_peered":"2023-11-09T03:48:23.584718+0000","last_clean":"2023-11-09T03:48:23.584718+0000","last_became_active":"2023-11-09T03:48:22.562455+0000","last_became_peered":"2023-11-09T03:48:22.562455+0000","last_unstale":"2023-11-09T03:48:23.584718+0000","last_undegraded":"2023-11-09T03:48:23.584718+0000","last_fullsized":"2023-11-09T03:48:23.584718+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:57:25.589693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.557744+0000","last_change":"2023-11-09T03:48:01.449077+0000","last_active":"2023-11-09T03:48:23.557744+0000","last_peered":"2023-11-09T03:48:23.557744+0000","last_clean":"2023-11-09T03:48:23.557744+0000","last_became_active":"2023-11-09T03:48:01.448806+0000","last_became_peered":"2023-11-09T03:48:01.448806+0000","last_unstale":"2023-11-09T03:48:23.557744+0000","last_undegraded":"2023-11-09T03:48:23.557744+0000","last_fullsized":"2023-11-09T03:48:23.557744+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:16:00.220961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.553211+0000","last_change":"2023-11-09T03:48:02.611756+0000","last_active":"2023-11-09T03:48:23.553211+0000","last_peered":"2023-11-09T03:48:23.553211+0000","last_clean":"2023-11-09T03:48:23.553211+0000","last_became_active":"2023-11-09T03:48:00.445880+0000","last_became_peered":"2023-11-09T03:48:00.445880+0000","last_unstale":"2023-11-09T03:48:23.553211+0000","last_undegraded":"2023-11-09T03:48:23.553211+0000","last_fullsized":"2023-11-09T03:48:23.553211+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:15:35.281296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085514900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.552388+0000","last_change":"2023-11-09T03:47:02.581917+0000","last_active":"2023-11-09T03:48:23.552388+0000","last_peered":"2023-11-09T03:48:23.552388+0000","last_clean":"2023-11-09T03:48:23.552388+0000","last_became_active":"2023-11-09T03:47:01.588082+0000","last_became_peered":"2023-11-09T03:47:01.588082+0000","last_unstale":"2023-11-09T03:48:23.552388+0000","last_undegraded":"2023-11-09T03:48:23.552388+0000","last_fullsized":"2023-11-09T03:48:23.552388+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:44:07.653557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034538899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.736595+0000","last_change":"2023-11-09T03:48:23.566645+0000","last_active":"2023-11-09T03:48:24.736595+0000","last_peered":"2023-11-09T03:48:24.736595+0000","last_clean":"2023-11-09T03:48:24.736595+0000","last_became_active":"2023-11-09T03:48:23.566315+0000","last_became_peered":"2023-11-09T03:48:23.566315+0000","last_unstale":"2023-11-09T03:48:24.736595+0000","last_undegraded":"2023-11-09T03:48:24.736595+0000","last_fullsized":"2023-11-09T03:48:24.736595+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:14:57.473226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"82'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.576878+0000","last_change":"2023-11-09T03:48:22.566342+0000","last_active":"2023-11-09T03:48:23.576878+0000","last_peered":"2023-11-09T03:48:23.576878+0000","last_clean":"2023-11-09T03:48:23.576878+0000","last_became_active":"2023-11-09T03:48:22.566028+0000","last_became_peered":"2023-11-09T03:48:22.566028+0000","last_unstale":"2023-11-09T03:48:23.576878+0000","last_undegraded":"2023-11-09T03:48:23.576878+0000","last_fullsized":"2023-11-09T03:48:23.576878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:36:57.293292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.555509+0000","last_change":"2023-11-09T03:48:01.456245+0000","last_active":"2023-11-09T03:48:23.555509+0000","last_peered":"2023-11-09T03:48:23.555509+0000","last_clean":"2023-11-09T03:48:23.555509+0000","last_became_active":"2023-11-09T03:48:01.455944+0000","last_became_peered":"2023-11-09T03:48:01.455944+0000","last_unstale":"2023-11-09T03:48:23.555509+0000","last_undegraded":"2023-11-09T03:48:23.555509+0000","last_fullsized":"2023-11-09T03:48:23.555509+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:06:14.849196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.554646+0000","last_change":"2023-11-09T03:48:02.612341+0000","last_active":"2023-11-09T03:48:23.554646+0000","last_peered":"2023-11-09T03:48:23.554646+0000","last_clean":"2023-11-09T03:48:23.554646+0000","last_became_active":"2023-11-09T03:48:00.444426+0000","last_became_peered":"2023-11-09T03:48:00.444426+0000","last_unstale":"2023-11-09T03:48:23.554646+0000","last_undegraded":"2023-11-09T03:48:23.554646+0000","last_fullsized":"2023-11-09T03:48:23.554646+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:19:53.969861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010812420000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":48,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.560541+0000","last_change":"2023-11-09T03:47:03.029063+0000","last_active":"2023-11-09T03:48:23.560541+0000","last_peered":"2023-11-09T03:48:23.560541+0000","last_clean":"2023-11-09T03:48:23.560541+0000","last_became_active":"2023-11-09T03:47:01.593582+0000","last_became_peered":"2023-11-09T03:47:01.593582+0000","last_unstale":"2023-11-09T03:48:23.560541+0000","last_undegraded":"2023-11-09T03:48:23.560541+0000","last_fullsized":"2023-11-09T03:48:23.560541+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:56:20.649937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041855100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.737887+0000","last_change":"2023-11-09T03:48:23.565316+0000","last_active":"2023-11-09T03:48:24.737887+0000","last_peered":"2023-11-09T03:48:24.737887+0000","last_clean":"2023-11-09T03:48:24.737887+0000","last_became_active":"2023-11-09T03:48:23.564434+0000","last_became_peered":"2023-11-09T03:48:23.564434+0000","last_unstale":"2023-11-09T03:48:24.737887+0000","last_undegraded":"2023-11-09T03:48:24.737887+0000","last_fullsized":"2023-11-09T03:48:24.737887+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:43:02.920090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.554659+0000","last_change":"2023-11-09T03:48:22.562238+0000","last_active":"2023-11-09T03:48:23.554659+0000","last_peered":"2023-11-09T03:48:23.554659+0000","last_clean":"2023-11-09T03:48:23.554659+0000","last_became_active":"2023-11-09T03:48:22.561984+0000","last_became_peered":"2023-11-09T03:48:22.561984+0000","last_unstale":"2023-11-09T03:48:23.554659+0000","last_undegraded":"2023-11-09T03:48:23.554659+0000","last_fullsized":"2023-11-09T03:48:23.554659+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:32:36.444225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.549406+0000","last_change":"2023-11-09T03:48:01.450271+0000","last_active":"2023-11-09T03:48:23.549406+0000","last_peered":"2023-11-09T03:48:23.549406+0000","last_clean":"2023-11-09T03:48:23.549406+0000","last_became_active":"2023-11-09T03:48:01.449998+0000","last_became_peered":"2023-11-09T03:48:01.449998+0000","last_unstale":"2023-11-09T03:48:23.549406+0000","last_undegraded":"2023-11-09T03:48:23.549406+0000","last_fullsized":"2023-11-09T03:48:23.549406+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:12:27.170650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.559316+0000","last_change":"2023-11-09T03:48:02.852114+0000","last_active":"2023-11-09T03:48:23.559316+0000","last_peered":"2023-11-09T03:48:23.559316+0000","last_clean":"2023-11-09T03:48:23.559316+0000","last_became_active":"2023-11-09T03:48:00.447203+0000","last_became_peered":"2023-11-09T03:48:00.447203+0000","last_unstale":"2023-11-09T03:48:23.559316+0000","last_undegraded":"2023-11-09T03:48:23.559316+0000","last_fullsized":"2023-11-09T03:48:23.559316+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:11:36.826430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049189500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":48,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.554933+0000","last_change":"2023-11-09T03:47:02.619336+0000","last_active":"2023-11-09T03:48:23.554933+0000","last_peered":"2023-11-09T03:48:23.554933+0000","last_clean":"2023-11-09T03:48:23.554933+0000","last_became_active":"2023-11-09T03:47:01.587749+0000","last_became_peered":"2023-11-09T03:47:01.587749+0000","last_unstale":"2023-11-09T03:48:23.554933+0000","last_undegraded":"2023-11-09T03:48:23.554933+0000","last_fullsized":"2023-11-09T03:48:23.554933+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063032099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.737637+0000","last_change":"2023-11-09T03:48:23.556819+0000","last_active":"2023-11-09T03:48:24.737637+0000","last_peered":"2023-11-09T03:48:24.737637+0000","last_clean":"2023-11-09T03:48:24.737637+0000","last_became_active":"2023-11-09T03:48:23.556546+0000","last_became_peered":"2023-11-09T03:48:23.556546+0000","last_unstale":"2023-11-09T03:48:24.737637+0000","last_undegraded":"2023-11-09T03:48:24.737637+0000","last_fullsized":"2023-11-09T03:48:24.737637+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:58:37.494787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.552025+0000","last_change":"2023-11-09T03:48:22.557530+0000","last_active":"2023-11-09T03:48:23.552025+0000","last_peered":"2023-11-09T03:48:23.552025+0000","last_clean":"2023-11-09T03:48:23.552025+0000","last_became_active":"2023-11-09T03:48:22.556685+0000","last_became_peered":"2023-11-09T03:48:22.556685+0000","last_unstale":"2023-11-09T03:48:23.552025+0000","last_undegraded":"2023-11-09T03:48:23.552025+0000","last_fullsized":"2023-11-09T03:48:23.552025+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:58:17.965471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.546696+0000","last_change":"2023-11-09T03:48:01.454147+0000","last_active":"2023-11-09T03:48:23.546696+0000","last_peered":"2023-11-09T03:48:23.546696+0000","last_clean":"2023-11-09T03:48:23.546696+0000","last_became_active":"2023-11-09T03:48:01.453868+0000","last_became_peered":"2023-11-09T03:48:01.453868+0000","last_unstale":"2023-11-09T03:48:23.546696+0000","last_undegraded":"2023-11-09T03:48:23.546696+0000","last_fullsized":"2023-11-09T03:48:23.546696+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:05:09.655318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.553512+0000","last_change":"2023-11-09T03:48:02.612000+0000","last_active":"2023-11-09T03:48:23.553512+0000","last_peered":"2023-11-09T03:48:23.553512+0000","last_clean":"2023-11-09T03:48:23.553512+0000","last_became_active":"2023-11-09T03:48:00.441667+0000","last_became_peered":"2023-11-09T03:48:00.441667+0000","last_unstale":"2023-11-09T03:48:23.553512+0000","last_undegraded":"2023-11-09T03:48:23.553512+0000","last_fullsized":"2023-11-09T03:48:23.553512+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:12:26.164644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000966781,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.551964+0000","last_change":"2023-11-09T03:47:02.581658+0000","last_active":"2023-11-09T03:48:23.551964+0000","last_peered":"2023-11-09T03:48:23.551964+0000","last_clean":"2023-11-09T03:48:23.551964+0000","last_became_active":"2023-11-09T03:47:01.588612+0000","last_became_peered":"2023-11-09T03:47:01.588612+0000","last_unstale":"2023-11-09T03:48:23.551964+0000","last_undegraded":"2023-11-09T03:48:23.551964+0000","last_fullsized":"2023-11-09T03:48:23.551964+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045238500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.735858+0000","last_change":"2023-11-09T03:48:23.572440+0000","last_active":"2023-11-09T03:48:24.735858+0000","last_peered":"2023-11-09T03:48:24.735858+0000","last_clean":"2023-11-09T03:48:24.735858+0000","last_became_active":"2023-11-09T03:48:23.572187+0000","last_became_peered":"2023-11-09T03:48:23.572187+0000","last_unstale":"2023-11-09T03:48:24.735858+0000","last_undegraded":"2023-11-09T03:48:24.735858+0000","last_fullsized":"2023-11-09T03:48:24.735858+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:21:51.814168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"82'8","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.582275+0000","last_change":"2023-11-09T03:48:22.557828+0000","last_active":"2023-11-09T03:48:23.582275+0000","last_peered":"2023-11-09T03:48:23.582275+0000","last_clean":"2023-11-09T03:48:23.582275+0000","last_became_active":"2023-11-09T03:48:22.557588+0000","last_became_peered":"2023-11-09T03:48:22.557588+0000","last_unstale":"2023-11-09T03:48:23.582275+0000","last_undegraded":"2023-11-09T03:48:23.582275+0000","last_fullsized":"2023-11-09T03:48:23.582275+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:40.105236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.550657+0000","last_change":"2023-11-09T03:48:01.449834+0000","last_active":"2023-11-09T03:48:23.550657+0000","last_peered":"2023-11-09T03:48:23.550657+0000","last_clean":"2023-11-09T03:48:23.550657+0000","last_became_active":"2023-11-09T03:48:01.449552+0000","last_became_peered":"2023-11-09T03:48:01.449552+0000","last_unstale":"2023-11-09T03:48:23.550657+0000","last_undegraded":"2023-11-09T03:48:23.550657+0000","last_fullsized":"2023-11-09T03:48:23.550657+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:11:37.832436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.550747+0000","last_change":"2023-11-09T03:48:02.682197+0000","last_active":"2023-11-09T03:48:23.550747+0000","last_peered":"2023-11-09T03:48:23.550747+0000","last_clean":"2023-11-09T03:48:23.550747+0000","last_became_active":"2023-11-09T03:48:00.445616+0000","last_became_peered":"2023-11-09T03:48:00.445616+0000","last_unstale":"2023-11-09T03:48:23.550747+0000","last_undegraded":"2023-11-09T03:48:23.550747+0000","last_fullsized":"2023-11-09T03:48:23.550747+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:57:03.474084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000416494,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":48,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.553435+0000","last_change":"2023-11-09T03:47:02.618828+0000","last_active":"2023-11-09T03:48:23.553435+0000","last_peered":"2023-11-09T03:48:23.553435+0000","last_clean":"2023-11-09T03:48:23.553435+0000","last_became_active":"2023-11-09T03:47:01.590679+0000","last_became_peered":"2023-11-09T03:47:01.590679+0000","last_unstale":"2023-11-09T03:48:23.553435+0000","last_undegraded":"2023-11-09T03:48:23.553435+0000","last_fullsized":"2023-11-09T03:48:23.553435+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:06:30.982987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070668,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.737595+0000","last_change":"2023-11-09T03:48:23.563145+0000","last_active":"2023-11-09T03:48:24.737595+0000","last_peered":"2023-11-09T03:48:24.737595+0000","last_clean":"2023-11-09T03:48:24.737595+0000","last_became_active":"2023-11-09T03:48:23.562867+0000","last_became_peered":"2023-11-09T03:48:23.562867+0000","last_unstale":"2023-11-09T03:48:24.737595+0000","last_undegraded":"2023-11-09T03:48:24.737595+0000","last_fullsized":"2023-11-09T03:48:24.737595+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:24:49.225093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.548683+0000","last_change":"2023-11-09T03:48:22.556590+0000","last_active":"2023-11-09T03:48:23.548683+0000","last_peered":"2023-11-09T03:48:23.548683+0000","last_clean":"2023-11-09T03:48:23.548683+0000","last_became_active":"2023-11-09T03:48:22.556295+0000","last_became_peered":"2023-11-09T03:48:22.556295+0000","last_unstale":"2023-11-09T03:48:23.548683+0000","last_undegraded":"2023-11-09T03:48:23.548683+0000","last_fullsized":"2023-11-09T03:48:23.548683+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:59.646501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.547443+0000","last_change":"2023-11-09T03:48:01.447647+0000","last_active":"2023-11-09T03:48:23.547443+0000","last_peered":"2023-11-09T03:48:23.547443+0000","last_clean":"2023-11-09T03:48:23.547443+0000","last_became_active":"2023-11-09T03:48:01.447302+0000","last_became_peered":"2023-11-09T03:48:01.447302+0000","last_unstale":"2023-11-09T03:48:23.547443+0000","last_undegraded":"2023-11-09T03:48:23.547443+0000","last_fullsized":"2023-11-09T03:48:23.547443+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:08:47.931939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.551323+0000","last_change":"2023-11-09T03:48:03.327731+0000","last_active":"2023-11-09T03:48:23.551323+0000","last_peered":"2023-11-09T03:48:23.551323+0000","last_clean":"2023-11-09T03:48:23.551323+0000","last_became_active":"2023-11-09T03:48:00.449660+0000","last_became_peered":"2023-11-09T03:48:00.449660+0000","last_unstale":"2023-11-09T03:48:23.551323+0000","last_undegraded":"2023-11-09T03:48:23.551323+0000","last_fullsized":"2023-11-09T03:48:23.551323+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:02:07.940908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037816599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":48,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.547342+0000","last_change":"2023-11-09T03:47:03.146528+0000","last_active":"2023-11-09T03:48:23.547342+0000","last_peered":"2023-11-09T03:48:23.547342+0000","last_clean":"2023-11-09T03:48:23.547342+0000","last_became_active":"2023-11-09T03:47:01.587060+0000","last_became_peered":"2023-11-09T03:47:01.587060+0000","last_unstale":"2023-11-09T03:48:23.547342+0000","last_undegraded":"2023-11-09T03:48:23.547342+0000","last_fullsized":"2023-11-09T03:48:23.547342+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:39:15.238496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079064999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.735904+0000","last_change":"2023-11-09T03:48:23.560214+0000","last_active":"2023-11-09T03:48:24.735904+0000","last_peered":"2023-11-09T03:48:24.735904+0000","last_clean":"2023-11-09T03:48:24.735904+0000","last_became_active":"2023-11-09T03:48:23.559927+0000","last_became_peered":"2023-11-09T03:48:23.559927+0000","last_unstale":"2023-11-09T03:48:24.735904+0000","last_undegraded":"2023-11-09T03:48:24.735904+0000","last_fullsized":"2023-11-09T03:48:24.735904+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:55:14.548178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.553798+0000","last_change":"2023-11-09T03:48:22.565708+0000","last_active":"2023-11-09T03:48:23.553798+0000","last_peered":"2023-11-09T03:48:23.553798+0000","last_clean":"2023-11-09T03:48:23.553798+0000","last_became_active":"2023-11-09T03:48:22.563961+0000","last_became_peered":"2023-11-09T03:48:22.563961+0000","last_unstale":"2023-11-09T03:48:23.553798+0000","last_undegraded":"2023-11-09T03:48:23.553798+0000","last_fullsized":"2023-11-09T03:48:23.553798+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:53:53.930826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.549636+0000","last_change":"2023-11-09T03:48:01.453939+0000","last_active":"2023-11-09T03:48:23.549636+0000","last_peered":"2023-11-09T03:48:23.549636+0000","last_clean":"2023-11-09T03:48:23.549636+0000","last_became_active":"2023-11-09T03:48:01.453600+0000","last_became_peered":"2023-11-09T03:48:01.453600+0000","last_unstale":"2023-11-09T03:48:23.549636+0000","last_undegraded":"2023-11-09T03:48:23.549636+0000","last_fullsized":"2023-11-09T03:48:23.549636+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:45:10.522254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.546178+0000","last_change":"2023-11-09T03:48:02.440910+0000","last_active":"2023-11-09T03:48:23.546178+0000","last_peered":"2023-11-09T03:48:23.546178+0000","last_clean":"2023-11-09T03:48:23.546178+0000","last_became_active":"2023-11-09T03:48:00.443791+0000","last_became_peered":"2023-11-09T03:48:00.443791+0000","last_unstale":"2023-11-09T03:48:23.546178+0000","last_undegraded":"2023-11-09T03:48:23.546178+0000","last_fullsized":"2023-11-09T03:48:23.546178+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:11:36.826430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069223400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":48,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.547291+0000","last_change":"2023-11-09T03:47:03.148246+0000","last_active":"2023-11-09T03:48:23.547291+0000","last_peered":"2023-11-09T03:48:23.547291+0000","last_clean":"2023-11-09T03:48:23.547291+0000","last_became_active":"2023-11-09T03:47:01.590988+0000","last_became_peered":"2023-11-09T03:47:01.590988+0000","last_unstale":"2023-11-09T03:48:23.547291+0000","last_undegraded":"2023-11-09T03:48:23.547291+0000","last_fullsized":"2023-11-09T03:48:23.547291+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:07:20.785445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055651099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.555853+0000","last_change":"2023-11-09T03:48:23.562324+0000","last_active":"2023-11-09T03:48:24.555853+0000","last_peered":"2023-11-09T03:48:24.555853+0000","last_clean":"2023-11-09T03:48:24.555853+0000","last_became_active":"2023-11-09T03:48:23.562055+0000","last_became_peered":"2023-11-09T03:48:23.562055+0000","last_unstale":"2023-11-09T03:48:24.555853+0000","last_undegraded":"2023-11-09T03:48:24.555853+0000","last_fullsized":"2023-11-09T03:48:24.555853+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:52:20.339478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.582042+0000","last_change":"2023-11-09T03:48:22.565920+0000","last_active":"2023-11-09T03:48:23.582042+0000","last_peered":"2023-11-09T03:48:23.582042+0000","last_clean":"2023-11-09T03:48:23.582042+0000","last_became_active":"2023-11-09T03:48:22.564592+0000","last_became_peered":"2023-11-09T03:48:22.564592+0000","last_unstale":"2023-11-09T03:48:23.582042+0000","last_undegraded":"2023-11-09T03:48:23.582042+0000","last_fullsized":"2023-11-09T03:48:23.582042+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:12:06.677722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.556211+0000","last_change":"2023-11-09T03:48:01.450557+0000","last_active":"2023-11-09T03:48:23.556211+0000","last_peered":"2023-11-09T03:48:23.556211+0000","last_clean":"2023-11-09T03:48:23.556211+0000","last_became_active":"2023-11-09T03:48:01.450291+0000","last_became_peered":"2023-11-09T03:48:01.450291+0000","last_unstale":"2023-11-09T03:48:23.556211+0000","last_undegraded":"2023-11-09T03:48:23.556211+0000","last_fullsized":"2023-11-09T03:48:23.556211+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:21:29.699594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.559845+0000","last_change":"2023-11-09T03:48:02.852010+0000","last_active":"2023-11-09T03:48:23.559845+0000","last_peered":"2023-11-09T03:48:23.559845+0000","last_clean":"2023-11-09T03:48:23.559845+0000","last_became_active":"2023-11-09T03:48:00.445267+0000","last_became_peered":"2023-11-09T03:48:00.445267+0000","last_unstale":"2023-11-09T03:48:23.559845+0000","last_undegraded":"2023-11-09T03:48:23.559845+0000","last_fullsized":"2023-11-09T03:48:23.559845+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:17.989628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064559199999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":52,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.549574+0000","last_change":"2023-11-09T03:47:02.582754+0000","last_active":"2023-11-09T03:48:23.549574+0000","last_peered":"2023-11-09T03:48:23.549574+0000","last_clean":"2023-11-09T03:48:23.549574+0000","last_became_active":"2023-11-09T03:47:01.589715+0000","last_became_peered":"2023-11-09T03:47:01.589715+0000","last_unstale":"2023-11-09T03:48:23.549574+0000","last_undegraded":"2023-11-09T03:48:23.549574+0000","last_fullsized":"2023-11-09T03:48:23.549574+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099342799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.737721+0000","last_change":"2023-11-09T03:48:23.563992+0000","last_active":"2023-11-09T03:48:24.737721+0000","last_peered":"2023-11-09T03:48:24.737721+0000","last_clean":"2023-11-09T03:48:24.737721+0000","last_became_active":"2023-11-09T03:48:23.563724+0000","last_became_peered":"2023-11-09T03:48:23.563724+0000","last_unstale":"2023-11-09T03:48:24.737721+0000","last_undegraded":"2023-11-09T03:48:24.737721+0000","last_fullsized":"2023-11-09T03:48:24.737721+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:53.884144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.582211+0000","last_change":"2023-11-09T03:48:22.566859+0000","last_active":"2023-11-09T03:48:23.582211+0000","last_peered":"2023-11-09T03:48:23.582211+0000","last_clean":"2023-11-09T03:48:23.582211+0000","last_became_active":"2023-11-09T03:48:22.566080+0000","last_became_peered":"2023-11-09T03:48:22.566080+0000","last_unstale":"2023-11-09T03:48:23.582211+0000","last_undegraded":"2023-11-09T03:48:23.582211+0000","last_fullsized":"2023-11-09T03:48:23.582211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:00:36.242435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.554864+0000","last_change":"2023-11-09T03:48:01.452456+0000","last_active":"2023-11-09T03:48:23.554864+0000","last_peered":"2023-11-09T03:48:23.554864+0000","last_clean":"2023-11-09T03:48:23.554864+0000","last_became_active":"2023-11-09T03:48:01.452059+0000","last_became_peered":"2023-11-09T03:48:01.452059+0000","last_unstale":"2023-11-09T03:48:23.554864+0000","last_undegraded":"2023-11-09T03:48:23.554864+0000","last_fullsized":"2023-11-09T03:48:23.554864+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:57:56.855869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.549648+0000","last_change":"2023-11-09T03:48:02.442557+0000","last_active":"2023-11-09T03:48:23.549648+0000","last_peered":"2023-11-09T03:48:23.549648+0000","last_clean":"2023-11-09T03:48:23.549648+0000","last_became_active":"2023-11-09T03:48:00.444823+0000","last_became_peered":"2023-11-09T03:48:00.444823+0000","last_unstale":"2023-11-09T03:48:23.549648+0000","last_undegraded":"2023-11-09T03:48:23.549648+0000","last_fullsized":"2023-11-09T03:48:23.549648+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:19:53.969861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075076299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":49,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.552852+0000","last_change":"2023-11-09T03:47:02.724896+0000","last_active":"2023-11-09T03:48:23.552852+0000","last_peered":"2023-11-09T03:48:23.552852+0000","last_clean":"2023-11-09T03:48:23.552852+0000","last_became_active":"2023-11-09T03:47:01.592784+0000","last_became_peered":"2023-11-09T03:47:01.592784+0000","last_unstale":"2023-11-09T03:48:23.552852+0000","last_undegraded":"2023-11-09T03:48:23.552852+0000","last_fullsized":"2023-11-09T03:48:23.552852+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:51:32.016204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056683500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.735799+0000","last_change":"2023-11-09T03:48:23.561233+0000","last_active":"2023-11-09T03:48:24.735799+0000","last_peered":"2023-11-09T03:48:24.735799+0000","last_clean":"2023-11-09T03:48:24.735799+0000","last_became_active":"2023-11-09T03:48:23.560958+0000","last_became_peered":"2023-11-09T03:48:23.560958+0000","last_unstale":"2023-11-09T03:48:24.735799+0000","last_undegraded":"2023-11-09T03:48:24.735799+0000","last_fullsized":"2023-11-09T03:48:24.735799+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:11:59.947010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.583908+0000","last_change":"2023-11-09T03:48:22.565279+0000","last_active":"2023-11-09T03:48:23.583908+0000","last_peered":"2023-11-09T03:48:23.583908+0000","last_clean":"2023-11-09T03:48:23.583908+0000","last_became_active":"2023-11-09T03:48:22.564998+0000","last_became_peered":"2023-11-09T03:48:22.564998+0000","last_unstale":"2023-11-09T03:48:23.583908+0000","last_undegraded":"2023-11-09T03:48:23.583908+0000","last_fullsized":"2023-11-09T03:48:23.583908+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:09:33.174481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.549311+0000","last_change":"2023-11-09T03:48:01.452167+0000","last_active":"2023-11-09T03:48:23.549311+0000","last_peered":"2023-11-09T03:48:23.549311+0000","last_clean":"2023-11-09T03:48:23.549311+0000","last_became_active":"2023-11-09T03:48:01.451780+0000","last_became_peered":"2023-11-09T03:48:01.451780+0000","last_unstale":"2023-11-09T03:48:23.549311+0000","last_undegraded":"2023-11-09T03:48:23.549311+0000","last_fullsized":"2023-11-09T03:48:23.549311+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:15:36.287302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.554563+0000","last_change":"2023-11-09T03:48:02.612246+0000","last_active":"2023-11-09T03:48:23.554563+0000","last_peered":"2023-11-09T03:48:23.554563+0000","last_clean":"2023-11-09T03:48:23.554563+0000","last_became_active":"2023-11-09T03:48:00.444629+0000","last_became_peered":"2023-11-09T03:48:00.444629+0000","last_unstale":"2023-11-09T03:48:23.554563+0000","last_undegraded":"2023-11-09T03:48:23.554563+0000","last_fullsized":"2023-11-09T03:48:23.554563+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:51:17.754054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098167299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.558238+0000","last_change":"2023-11-09T03:47:03.146527+0000","last_active":"2023-11-09T03:48:23.558238+0000","last_peered":"2023-11-09T03:48:23.558238+0000","last_clean":"2023-11-09T03:48:23.558238+0000","last_became_active":"2023-11-09T03:47:01.587858+0000","last_became_peered":"2023-11-09T03:47:01.587858+0000","last_unstale":"2023-11-09T03:48:23.558238+0000","last_undegraded":"2023-11-09T03:48:23.558238+0000","last_fullsized":"2023-11-09T03:48:23.558238+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:56:04.201126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084439599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.556396+0000","last_change":"2023-11-09T03:48:23.565737+0000","last_active":"2023-11-09T03:48:24.556396+0000","last_peered":"2023-11-09T03:48:24.556396+0000","last_clean":"2023-11-09T03:48:24.556396+0000","last_became_active":"2023-11-09T03:48:23.565466+0000","last_became_peered":"2023-11-09T03:48:23.565466+0000","last_unstale":"2023-11-09T03:48:24.556396+0000","last_undegraded":"2023-11-09T03:48:24.556396+0000","last_fullsized":"2023-11-09T03:48:24.556396+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:23:26.242374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.578562+0000","last_change":"2023-11-09T03:48:22.553920+0000","last_active":"2023-11-09T03:48:23.578562+0000","last_peered":"2023-11-09T03:48:23.578562+0000","last_clean":"2023-11-09T03:48:23.578562+0000","last_became_active":"2023-11-09T03:48:22.552644+0000","last_became_peered":"2023-11-09T03:48:22.552644+0000","last_unstale":"2023-11-09T03:48:23.578562+0000","last_undegraded":"2023-11-09T03:48:23.578562+0000","last_fullsized":"2023-11-09T03:48:23.578562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:21.472203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.552315+0000","last_change":"2023-11-09T03:48:01.450685+0000","last_active":"2023-11-09T03:48:23.552315+0000","last_peered":"2023-11-09T03:48:23.552315+0000","last_clean":"2023-11-09T03:48:23.552315+0000","last_became_active":"2023-11-09T03:48:01.450445+0000","last_became_peered":"2023-11-09T03:48:01.450445+0000","last_unstale":"2023-11-09T03:48:23.552315+0000","last_undegraded":"2023-11-09T03:48:23.552315+0000","last_fullsized":"2023-11-09T03:48:23.552315+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:16:00.220961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.546860+0000","last_change":"2023-11-09T03:48:02.441928+0000","last_active":"2023-11-09T03:48:23.546860+0000","last_peered":"2023-11-09T03:48:23.546860+0000","last_clean":"2023-11-09T03:48:23.546860+0000","last_became_active":"2023-11-09T03:48:00.444417+0000","last_became_peered":"2023-11-09T03:48:00.444417+0000","last_unstale":"2023-11-09T03:48:23.546860+0000","last_undegraded":"2023-11-09T03:48:23.546860+0000","last_fullsized":"2023-11-09T03:48:23.546860+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:55.450096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072428900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":49,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.551502+0000","last_change":"2023-11-09T03:47:02.725223+0000","last_active":"2023-11-09T03:48:23.551502+0000","last_peered":"2023-11-09T03:48:23.551502+0000","last_clean":"2023-11-09T03:48:23.551502+0000","last_became_active":"2023-11-09T03:47:01.590635+0000","last_became_peered":"2023-11-09T03:47:01.590635+0000","last_unstale":"2023-11-09T03:48:23.551502+0000","last_undegraded":"2023-11-09T03:48:23.551502+0000","last_fullsized":"2023-11-09T03:48:23.551502+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:05.754357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051685100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.737586+0000","last_change":"2023-11-09T03:48:23.559563+0000","last_active":"2023-11-09T03:48:24.737586+0000","last_peered":"2023-11-09T03:48:24.737586+0000","last_clean":"2023-11-09T03:48:24.737586+0000","last_became_active":"2023-11-09T03:48:23.559278+0000","last_became_peered":"2023-11-09T03:48:23.559278+0000","last_unstale":"2023-11-09T03:48:24.737586+0000","last_undegraded":"2023-11-09T03:48:24.737586+0000","last_fullsized":"2023-11-09T03:48:24.737586+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:49:46.794594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"82'4","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.581916+0000","last_change":"2023-11-09T03:48:22.563841+0000","last_active":"2023-11-09T03:48:23.581916+0000","last_peered":"2023-11-09T03:48:23.581916+0000","last_clean":"2023-11-09T03:48:23.581916+0000","last_became_active":"2023-11-09T03:48:22.563590+0000","last_became_peered":"2023-11-09T03:48:22.563590+0000","last_unstale":"2023-11-09T03:48:23.581916+0000","last_undegraded":"2023-11-09T03:48:23.581916+0000","last_fullsized":"2023-11-09T03:48:23.581916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:17:45.475023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.554165+0000","last_change":"2023-11-09T03:48:01.453143+0000","last_active":"2023-11-09T03:48:23.554165+0000","last_peered":"2023-11-09T03:48:23.554165+0000","last_clean":"2023-11-09T03:48:23.554165+0000","last_became_active":"2023-11-09T03:48:01.452873+0000","last_became_peered":"2023-11-09T03:48:01.452873+0000","last_unstale":"2023-11-09T03:48:23.554165+0000","last_undegraded":"2023-11-09T03:48:23.554165+0000","last_fullsized":"2023-11-09T03:48:23.554165+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:06:28.575429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.546609+0000","last_change":"2023-11-09T03:48:02.442416+0000","last_active":"2023-11-09T03:48:23.546609+0000","last_peered":"2023-11-09T03:48:23.546609+0000","last_clean":"2023-11-09T03:48:23.546609+0000","last_became_active":"2023-11-09T03:48:00.443765+0000","last_became_peered":"2023-11-09T03:48:00.443765+0000","last_unstale":"2023-11-09T03:48:23.546609+0000","last_undegraded":"2023-11-09T03:48:23.546609+0000","last_fullsized":"2023-11-09T03:48:23.546609+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:41:22.321811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012468850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":52,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.549723+0000","last_change":"2023-11-09T03:47:02.582739+0000","last_active":"2023-11-09T03:48:23.549723+0000","last_peered":"2023-11-09T03:48:23.549723+0000","last_clean":"2023-11-09T03:48:23.549723+0000","last_became_active":"2023-11-09T03:47:01.592743+0000","last_became_peered":"2023-11-09T03:47:01.592743+0000","last_unstale":"2023-11-09T03:48:23.549723+0000","last_undegraded":"2023-11-09T03:48:23.549723+0000","last_fullsized":"2023-11-09T03:48:23.549723+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051802700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.737361+0000","last_change":"2023-11-09T03:48:23.563958+0000","last_active":"2023-11-09T03:48:24.737361+0000","last_peered":"2023-11-09T03:48:24.737361+0000","last_clean":"2023-11-09T03:48:24.737361+0000","last_became_active":"2023-11-09T03:48:23.563682+0000","last_became_peered":"2023-11-09T03:48:23.563682+0000","last_unstale":"2023-11-09T03:48:24.737361+0000","last_undegraded":"2023-11-09T03:48:24.737361+0000","last_fullsized":"2023-11-09T03:48:24.737361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:25:05.116469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.556054+0000","last_change":"2023-11-09T03:48:22.559947+0000","last_active":"2023-11-09T03:48:23.556054+0000","last_peered":"2023-11-09T03:48:23.556054+0000","last_clean":"2023-11-09T03:48:23.556054+0000","last_became_active":"2023-11-09T03:48:22.559667+0000","last_became_peered":"2023-11-09T03:48:22.559667+0000","last_unstale":"2023-11-09T03:48:23.556054+0000","last_undegraded":"2023-11-09T03:48:23.556054+0000","last_fullsized":"2023-11-09T03:48:23.556054+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:20:57.413543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.549255+0000","last_change":"2023-11-09T03:48:01.442195+0000","last_active":"2023-11-09T03:48:23.549255+0000","last_peered":"2023-11-09T03:48:23.549255+0000","last_clean":"2023-11-09T03:48:23.549255+0000","last_became_active":"2023-11-09T03:48:01.441887+0000","last_became_peered":"2023-11-09T03:48:01.441887+0000","last_unstale":"2023-11-09T03:48:23.549255+0000","last_undegraded":"2023-11-09T03:48:23.549255+0000","last_fullsized":"2023-11-09T03:48:23.549255+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:06:28.575429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.548824+0000","last_change":"2023-11-09T03:48:02.442061+0000","last_active":"2023-11-09T03:48:23.548824+0000","last_peered":"2023-11-09T03:48:23.548824+0000","last_clean":"2023-11-09T03:48:23.548824+0000","last_became_active":"2023-11-09T03:48:00.445921+0000","last_became_peered":"2023-11-09T03:48:00.445921+0000","last_unstale":"2023-11-09T03:48:23.548824+0000","last_undegraded":"2023-11-09T03:48:23.548824+0000","last_fullsized":"2023-11-09T03:48:23.548824+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:50:55.299235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012719770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":49,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.550635+0000","last_change":"2023-11-09T03:47:02.724953+0000","last_active":"2023-11-09T03:48:23.550635+0000","last_peered":"2023-11-09T03:48:23.550635+0000","last_clean":"2023-11-09T03:48:23.550635+0000","last_became_active":"2023-11-09T03:47:01.587133+0000","last_became_peered":"2023-11-09T03:47:01.587133+0000","last_unstale":"2023-11-09T03:48:23.550635+0000","last_undegraded":"2023-11-09T03:48:23.550635+0000","last_fullsized":"2023-11-09T03:48:23.550635+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:23:48.521161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066497599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.555742+0000","last_change":"2023-11-09T03:48:23.565013+0000","last_active":"2023-11-09T03:48:24.555742+0000","last_peered":"2023-11-09T03:48:24.555742+0000","last_clean":"2023-11-09T03:48:24.555742+0000","last_became_active":"2023-11-09T03:48:23.564761+0000","last_became_peered":"2023-11-09T03:48:23.564761+0000","last_unstale":"2023-11-09T03:48:24.555742+0000","last_undegraded":"2023-11-09T03:48:24.555742+0000","last_fullsized":"2023-11-09T03:48:24.555742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:53:59.098346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.552857+0000","last_change":"2023-11-09T03:48:22.556449+0000","last_active":"2023-11-09T03:48:23.552857+0000","last_peered":"2023-11-09T03:48:23.552857+0000","last_clean":"2023-11-09T03:48:23.552857+0000","last_became_active":"2023-11-09T03:48:22.556170+0000","last_became_peered":"2023-11-09T03:48:22.556170+0000","last_unstale":"2023-11-09T03:48:23.552857+0000","last_undegraded":"2023-11-09T03:48:23.552857+0000","last_fullsized":"2023-11-09T03:48:23.552857+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:59.646501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.551267+0000","last_change":"2023-11-09T03:48:01.450507+0000","last_active":"2023-11-09T03:48:23.551267+0000","last_peered":"2023-11-09T03:48:23.551267+0000","last_clean":"2023-11-09T03:48:23.551267+0000","last_became_active":"2023-11-09T03:48:01.450191+0000","last_became_peered":"2023-11-09T03:48:01.450191+0000","last_unstale":"2023-11-09T03:48:23.551267+0000","last_undegraded":"2023-11-09T03:48:23.551267+0000","last_fullsized":"2023-11-09T03:48:23.551267+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:32:40.342502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.549331+0000","last_change":"2023-11-09T03:48:02.442120+0000","last_active":"2023-11-09T03:48:23.549331+0000","last_peered":"2023-11-09T03:48:23.549331+0000","last_clean":"2023-11-09T03:48:23.549331+0000","last_became_active":"2023-11-09T03:48:00.445504+0000","last_became_peered":"2023-11-09T03:48:00.445504+0000","last_unstale":"2023-11-09T03:48:23.549331+0000","last_undegraded":"2023-11-09T03:48:23.549331+0000","last_fullsized":"2023-11-09T03:48:23.549331+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:35:53.864627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001095766,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.551126+0000","last_change":"2023-11-09T03:47:03.146217+0000","last_active":"2023-11-09T03:48:23.551126+0000","last_peered":"2023-11-09T03:48:23.551126+0000","last_clean":"2023-11-09T03:48:23.551126+0000","last_became_active":"2023-11-09T03:47:01.587598+0000","last_became_peered":"2023-11-09T03:47:01.587598+0000","last_unstale":"2023-11-09T03:48:23.551126+0000","last_undegraded":"2023-11-09T03:48:23.551126+0000","last_fullsized":"2023-11-09T03:48:23.551126+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:15:22.723455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090126400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.558430+0000","last_change":"2023-11-09T03:48:23.559047+0000","last_active":"2023-11-09T03:48:24.558430+0000","last_peered":"2023-11-09T03:48:24.558430+0000","last_clean":"2023-11-09T03:48:24.558430+0000","last_became_active":"2023-11-09T03:48:23.558784+0000","last_became_peered":"2023-11-09T03:48:23.558784+0000","last_unstale":"2023-11-09T03:48:24.558430+0000","last_undegraded":"2023-11-09T03:48:24.558430+0000","last_fullsized":"2023-11-09T03:48:24.558430+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:00:48.408855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.580885+0000","last_change":"2023-11-09T03:48:22.561567+0000","last_active":"2023-11-09T03:48:23.580885+0000","last_peered":"2023-11-09T03:48:23.580885+0000","last_clean":"2023-11-09T03:48:23.580885+0000","last_became_active":"2023-11-09T03:48:22.561266+0000","last_became_peered":"2023-11-09T03:48:22.561266+0000","last_unstale":"2023-11-09T03:48:23.580885+0000","last_undegraded":"2023-11-09T03:48:23.580885+0000","last_fullsized":"2023-11-09T03:48:23.580885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:00:47.403883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.558008+0000","last_change":"2023-11-09T03:48:01.454878+0000","last_active":"2023-11-09T03:48:23.558008+0000","last_peered":"2023-11-09T03:48:23.558008+0000","last_clean":"2023-11-09T03:48:23.558008+0000","last_became_active":"2023-11-09T03:48:01.454591+0000","last_became_peered":"2023-11-09T03:48:01.454591+0000","last_unstale":"2023-11-09T03:48:23.558008+0000","last_undegraded":"2023-11-09T03:48:23.558008+0000","last_fullsized":"2023-11-09T03:48:23.558008+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:38.536898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.556267+0000","last_change":"2023-11-09T03:48:03.327800+0000","last_active":"2023-11-09T03:48:23.556267+0000","last_peered":"2023-11-09T03:48:23.556267+0000","last_clean":"2023-11-09T03:48:23.556267+0000","last_became_active":"2023-11-09T03:48:00.446576+0000","last_became_peered":"2023-11-09T03:48:00.446576+0000","last_unstale":"2023-11-09T03:48:23.556267+0000","last_undegraded":"2023-11-09T03:48:23.556267+0000","last_fullsized":"2023-11-09T03:48:23.556267+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:37:08.851102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054679700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":48,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.547378+0000","last_change":"2023-11-09T03:47:03.148283+0000","last_active":"2023-11-09T03:48:23.547378+0000","last_peered":"2023-11-09T03:48:23.547378+0000","last_clean":"2023-11-09T03:48:23.547378+0000","last_became_active":"2023-11-09T03:47:01.588841+0000","last_became_peered":"2023-11-09T03:47:01.588841+0000","last_unstale":"2023-11-09T03:48:23.547378+0000","last_undegraded":"2023-11-09T03:48:23.547378+0000","last_fullsized":"2023-11-09T03:48:23.547378+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:47:44.880741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000436081,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.736018+0000","last_change":"2023-11-09T03:48:23.565978+0000","last_active":"2023-11-09T03:48:24.736018+0000","last_peered":"2023-11-09T03:48:24.736018+0000","last_clean":"2023-11-09T03:48:24.736018+0000","last_became_active":"2023-11-09T03:48:23.565718+0000","last_became_peered":"2023-11-09T03:48:23.565718+0000","last_unstale":"2023-11-09T03:48:24.736018+0000","last_undegraded":"2023-11-09T03:48:24.736018+0000","last_fullsized":"2023-11-09T03:48:24.736018+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:25:05.116469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.578362+0000","last_change":"2023-11-09T03:48:22.556480+0000","last_active":"2023-11-09T03:48:23.578362+0000","last_peered":"2023-11-09T03:48:23.578362+0000","last_clean":"2023-11-09T03:48:23.578362+0000","last_became_active":"2023-11-09T03:48:22.556059+0000","last_became_peered":"2023-11-09T03:48:22.556059+0000","last_unstale":"2023-11-09T03:48:23.578362+0000","last_undegraded":"2023-11-09T03:48:23.578362+0000","last_fullsized":"2023-11-09T03:48:23.578362+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:57:12.268175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.551122+0000","last_change":"2023-11-09T03:48:01.450581+0000","last_active":"2023-11-09T03:48:23.551122+0000","last_peered":"2023-11-09T03:48:23.551122+0000","last_clean":"2023-11-09T03:48:23.551122+0000","last_became_active":"2023-11-09T03:48:01.450280+0000","last_became_peered":"2023-11-09T03:48:01.450280+0000","last_unstale":"2023-11-09T03:48:23.551122+0000","last_undegraded":"2023-11-09T03:48:23.551122+0000","last_fullsized":"2023-11-09T03:48:23.551122+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:18.995634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.546756+0000","last_change":"2023-11-09T03:48:02.440934+0000","last_active":"2023-11-09T03:48:23.546756+0000","last_peered":"2023-11-09T03:48:23.546756+0000","last_clean":"2023-11-09T03:48:23.546756+0000","last_became_active":"2023-11-09T03:48:00.444394+0000","last_became_peered":"2023-11-09T03:48:00.444394+0000","last_unstale":"2023-11-09T03:48:23.546756+0000","last_undegraded":"2023-11-09T03:48:23.546756+0000","last_fullsized":"2023-11-09T03:48:23.546756+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:15:59.214955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063908300000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":48,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.552128+0000","last_change":"2023-11-09T03:47:02.618856+0000","last_active":"2023-11-09T03:48:23.552128+0000","last_peered":"2023-11-09T03:48:23.552128+0000","last_clean":"2023-11-09T03:48:23.552128+0000","last_became_active":"2023-11-09T03:47:01.590430+0000","last_became_peered":"2023-11-09T03:47:01.590430+0000","last_unstale":"2023-11-09T03:48:23.552128+0000","last_undegraded":"2023-11-09T03:48:23.552128+0000","last_fullsized":"2023-11-09T03:48:23.552128+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:57:18.470305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055030200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.736955+0000","last_change":"2023-11-09T03:48:23.565865+0000","last_active":"2023-11-09T03:48:24.736955+0000","last_peered":"2023-11-09T03:48:24.736955+0000","last_clean":"2023-11-09T03:48:24.736955+0000","last_became_active":"2023-11-09T03:48:23.565602+0000","last_became_peered":"2023-11-09T03:48:23.565602+0000","last_unstale":"2023-11-09T03:48:24.736955+0000","last_undegraded":"2023-11-09T03:48:24.736955+0000","last_fullsized":"2023-11-09T03:48:24.736955+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:36:58.298263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.554121+0000","last_change":"2023-11-09T03:48:22.559687+0000","last_active":"2023-11-09T03:48:23.554121+0000","last_peered":"2023-11-09T03:48:23.554121+0000","last_clean":"2023-11-09T03:48:23.554121+0000","last_became_active":"2023-11-09T03:48:22.559404+0000","last_became_peered":"2023-11-09T03:48:22.559404+0000","last_unstale":"2023-11-09T03:48:23.554121+0000","last_undegraded":"2023-11-09T03:48:23.554121+0000","last_fullsized":"2023-11-09T03:48:23.554121+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:21.472203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.550993+0000","last_change":"2023-11-09T03:48:01.453600+0000","last_active":"2023-11-09T03:48:23.550993+0000","last_peered":"2023-11-09T03:48:23.550993+0000","last_clean":"2023-11-09T03:48:23.550993+0000","last_became_active":"2023-11-09T03:48:01.453330+0000","last_became_peered":"2023-11-09T03:48:01.453330+0000","last_unstale":"2023-11-09T03:48:23.550993+0000","last_undegraded":"2023-11-09T03:48:23.550993+0000","last_fullsized":"2023-11-09T03:48:23.550993+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:08:47.931939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.551866+0000","last_change":"2023-11-09T03:48:03.328210+0000","last_active":"2023-11-09T03:48:23.551866+0000","last_peered":"2023-11-09T03:48:23.551866+0000","last_clean":"2023-11-09T03:48:23.551866+0000","last_became_active":"2023-11-09T03:48:00.450002+0000","last_became_peered":"2023-11-09T03:48:00.450002+0000","last_unstale":"2023-11-09T03:48:23.551866+0000","last_undegraded":"2023-11-09T03:48:23.551866+0000","last_fullsized":"2023-11-09T03:48:23.551866+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:50.391187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036521900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":48,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.553705+0000","last_change":"2023-11-09T03:47:02.619144+0000","last_active":"2023-11-09T03:48:23.553705+0000","last_peered":"2023-11-09T03:48:23.553705+0000","last_clean":"2023-11-09T03:48:23.553705+0000","last_became_active":"2023-11-09T03:47:01.590672+0000","last_became_peered":"2023-11-09T03:47:01.590672+0000","last_unstale":"2023-11-09T03:48:23.553705+0000","last_undegraded":"2023-11-09T03:48:23.553705+0000","last_fullsized":"2023-11-09T03:48:23.553705+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:03.025119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060997500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.735844+0000","last_change":"2023-11-09T03:48:23.562604+0000","last_active":"2023-11-09T03:48:24.735844+0000","last_peered":"2023-11-09T03:48:24.735844+0000","last_clean":"2023-11-09T03:48:24.735844+0000","last_became_active":"2023-11-09T03:48:23.562342+0000","last_became_peered":"2023-11-09T03:48:23.562342+0000","last_unstale":"2023-11-09T03:48:24.735844+0000","last_undegraded":"2023-11-09T03:48:24.735844+0000","last_fullsized":"2023-11-09T03:48:24.735844+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:16:22.335535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.546492+0000","last_change":"2023-11-09T03:48:22.566419+0000","last_active":"2023-11-09T03:48:23.546492+0000","last_peered":"2023-11-09T03:48:23.546492+0000","last_clean":"2023-11-09T03:48:23.546492+0000","last_became_active":"2023-11-09T03:48:22.566144+0000","last_became_peered":"2023-11-09T03:48:22.566144+0000","last_unstale":"2023-11-09T03:48:23.546492+0000","last_undegraded":"2023-11-09T03:48:23.546492+0000","last_fullsized":"2023-11-09T03:48:23.546492+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:53:55.393030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.549517+0000","last_change":"2023-11-09T03:48:01.447278+0000","last_active":"2023-11-09T03:48:23.549517+0000","last_peered":"2023-11-09T03:48:23.549517+0000","last_clean":"2023-11-09T03:48:23.549517+0000","last_became_active":"2023-11-09T03:48:01.446991+0000","last_became_peered":"2023-11-09T03:48:01.446991+0000","last_unstale":"2023-11-09T03:48:23.549517+0000","last_undegraded":"2023-11-09T03:48:23.549517+0000","last_fullsized":"2023-11-09T03:48:23.549517+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:27:49.190426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.552097+0000","last_change":"2023-11-09T03:48:03.328278+0000","last_active":"2023-11-09T03:48:23.552097+0000","last_peered":"2023-11-09T03:48:23.552097+0000","last_clean":"2023-11-09T03:48:23.552097+0000","last_became_active":"2023-11-09T03:48:00.443213+0000","last_became_peered":"2023-11-09T03:48:00.443213+0000","last_unstale":"2023-11-09T03:48:23.552097+0000","last_undegraded":"2023-11-09T03:48:23.552097+0000","last_fullsized":"2023-11-09T03:48:23.552097+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:20:59.702297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030886100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":48,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.554872+0000","last_change":"2023-11-09T03:47:02.619079+0000","last_active":"2023-11-09T03:48:23.554872+0000","last_peered":"2023-11-09T03:48:23.554872+0000","last_clean":"2023-11-09T03:48:23.554872+0000","last_became_active":"2023-11-09T03:47:01.591421+0000","last_became_peered":"2023-11-09T03:47:01.591421+0000","last_unstale":"2023-11-09T03:48:23.554872+0000","last_undegraded":"2023-11-09T03:48:23.554872+0000","last_fullsized":"2023-11-09T03:48:23.554872+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:18:58.230606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068736399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.736429+0000","last_change":"2023-11-09T03:48:23.558304+0000","last_active":"2023-11-09T03:48:24.736429+0000","last_peered":"2023-11-09T03:48:24.736429+0000","last_clean":"2023-11-09T03:48:24.736429+0000","last_became_active":"2023-11-09T03:48:23.558011+0000","last_became_peered":"2023-11-09T03:48:23.558011+0000","last_unstale":"2023-11-09T03:48:24.736429+0000","last_undegraded":"2023-11-09T03:48:24.736429+0000","last_fullsized":"2023-11-09T03:48:24.736429+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:39:54.467929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.737734+0000","last_change":"2023-11-09T03:48:22.569861+0000","last_active":"2023-11-09T03:48:24.737734+0000","last_peered":"2023-11-09T03:48:24.737734+0000","last_clean":"2023-11-09T03:48:24.737734+0000","last_became_active":"2023-11-09T03:48:22.569611+0000","last_became_peered":"2023-11-09T03:48:22.569611+0000","last_unstale":"2023-11-09T03:48:24.737734+0000","last_undegraded":"2023-11-09T03:48:24.737734+0000","last_fullsized":"2023-11-09T03:48:24.737734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:36:18.964900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.549505+0000","last_change":"2023-11-09T03:48:01.449069+0000","last_active":"2023-11-09T03:48:23.549505+0000","last_peered":"2023-11-09T03:48:23.549505+0000","last_clean":"2023-11-09T03:48:23.549505+0000","last_became_active":"2023-11-09T03:48:01.448810+0000","last_became_peered":"2023-11-09T03:48:01.448810+0000","last_unstale":"2023-11-09T03:48:23.549505+0000","last_undegraded":"2023-11-09T03:48:23.549505+0000","last_fullsized":"2023-11-09T03:48:23.549505+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:56:51.158572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.554208+0000","last_change":"2023-11-09T03:48:02.612067+0000","last_active":"2023-11-09T03:48:23.554208+0000","last_peered":"2023-11-09T03:48:23.554208+0000","last_clean":"2023-11-09T03:48:23.554208+0000","last_became_active":"2023-11-09T03:48:00.443296+0000","last_became_peered":"2023-11-09T03:48:00.443296+0000","last_unstale":"2023-11-09T03:48:23.554208+0000","last_undegraded":"2023-11-09T03:48:23.554208+0000","last_fullsized":"2023-11-09T03:48:23.554208+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:27:48.184420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010423330000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":48,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.547945+0000","last_change":"2023-11-09T03:47:03.148529+0000","last_active":"2023-11-09T03:48:23.547945+0000","last_peered":"2023-11-09T03:48:23.547945+0000","last_clean":"2023-11-09T03:48:23.547945+0000","last_became_active":"2023-11-09T03:47:01.584148+0000","last_became_peered":"2023-11-09T03:47:01.584148+0000","last_unstale":"2023-11-09T03:48:23.547945+0000","last_undegraded":"2023-11-09T03:48:23.547945+0000","last_fullsized":"2023-11-09T03:48:23.547945+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:40:00.237248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061475600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.737719+0000","last_change":"2023-11-09T03:48:23.562959+0000","last_active":"2023-11-09T03:48:24.737719+0000","last_peered":"2023-11-09T03:48:24.737719+0000","last_clean":"2023-11-09T03:48:24.737719+0000","last_became_active":"2023-11-09T03:48:23.562658+0000","last_became_peered":"2023-11-09T03:48:23.562658+0000","last_unstale":"2023-11-09T03:48:24.737719+0000","last_undegraded":"2023-11-09T03:48:24.737719+0000","last_fullsized":"2023-11-09T03:48:24.737719+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:43:57.917297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.574210+0000","last_change":"2023-11-09T03:48:22.558072+0000","last_active":"2023-11-09T03:48:23.574210+0000","last_peered":"2023-11-09T03:48:23.574210+0000","last_clean":"2023-11-09T03:48:23.574210+0000","last_became_active":"2023-11-09T03:48:22.557800+0000","last_became_peered":"2023-11-09T03:48:22.557800+0000","last_unstale":"2023-11-09T03:48:23.574210+0000","last_undegraded":"2023-11-09T03:48:23.574210+0000","last_fullsized":"2023-11-09T03:48:23.574210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:21.472203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.554798+0000","last_change":"2023-11-09T03:48:01.451750+0000","last_active":"2023-11-09T03:48:23.554798+0000","last_peered":"2023-11-09T03:48:23.554798+0000","last_clean":"2023-11-09T03:48:23.554798+0000","last_became_active":"2023-11-09T03:48:01.451393+0000","last_became_peered":"2023-11-09T03:48:01.451393+0000","last_unstale":"2023-11-09T03:48:23.554798+0000","last_undegraded":"2023-11-09T03:48:23.554798+0000","last_fullsized":"2023-11-09T03:48:23.554798+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:56:51.158572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.553137+0000","last_change":"2023-11-09T03:48:02.611721+0000","last_active":"2023-11-09T03:48:23.553137+0000","last_peered":"2023-11-09T03:48:23.553137+0000","last_clean":"2023-11-09T03:48:23.553137+0000","last_became_active":"2023-11-09T03:48:00.439557+0000","last_became_peered":"2023-11-09T03:48:00.439557+0000","last_unstale":"2023-11-09T03:48:23.553137+0000","last_undegraded":"2023-11-09T03:48:23.553137+0000","last_fullsized":"2023-11-09T03:48:23.553137+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:35:53.864627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001037187,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":48,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.554559+0000","last_change":"2023-11-09T03:47:03.148217+0000","last_active":"2023-11-09T03:48:23.554559+0000","last_peered":"2023-11-09T03:48:23.554559+0000","last_clean":"2023-11-09T03:48:23.554559+0000","last_became_active":"2023-11-09T03:47:01.593424+0000","last_became_peered":"2023-11-09T03:47:01.593424+0000","last_unstale":"2023-11-09T03:48:23.554559+0000","last_undegraded":"2023-11-09T03:48:23.554559+0000","last_fullsized":"2023-11-09T03:48:23.554559+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:19:32.693691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001769865,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.735962+0000","last_change":"2023-11-09T03:48:23.567808+0000","last_active":"2023-11-09T03:48:24.735962+0000","last_peered":"2023-11-09T03:48:24.735962+0000","last_clean":"2023-11-09T03:48:24.735962+0000","last_became_active":"2023-11-09T03:48:23.567516+0000","last_became_peered":"2023-11-09T03:48:23.567516+0000","last_unstale":"2023-11-09T03:48:24.735962+0000","last_undegraded":"2023-11-09T03:48:24.735962+0000","last_fullsized":"2023-11-09T03:48:24.735962+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:04:53.884144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.550398+0000","last_change":"2023-11-09T03:48:22.566073+0000","last_active":"2023-11-09T03:48:23.550398+0000","last_peered":"2023-11-09T03:48:23.550398+0000","last_clean":"2023-11-09T03:48:23.550398+0000","last_became_active":"2023-11-09T03:48:22.564106+0000","last_became_peered":"2023-11-09T03:48:22.564106+0000","last_unstale":"2023-11-09T03:48:23.550398+0000","last_undegraded":"2023-11-09T03:48:23.550398+0000","last_fullsized":"2023-11-09T03:48:23.550398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:29:19.629987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.547084+0000","last_change":"2023-11-09T03:48:01.447617+0000","last_active":"2023-11-09T03:48:23.547084+0000","last_peered":"2023-11-09T03:48:23.547084+0000","last_clean":"2023-11-09T03:48:23.547084+0000","last_became_active":"2023-11-09T03:48:01.447329+0000","last_became_peered":"2023-11-09T03:48:01.447329+0000","last_unstale":"2023-11-09T03:48:23.547084+0000","last_undegraded":"2023-11-09T03:48:23.547084+0000","last_fullsized":"2023-11-09T03:48:23.547084+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:54:52.433604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.549391+0000","last_change":"2023-11-09T03:48:02.442427+0000","last_active":"2023-11-09T03:48:23.549391+0000","last_peered":"2023-11-09T03:48:23.549391+0000","last_clean":"2023-11-09T03:48:23.549391+0000","last_became_active":"2023-11-09T03:48:00.444856+0000","last_became_peered":"2023-11-09T03:48:00.444856+0000","last_unstale":"2023-11-09T03:48:23.549391+0000","last_undegraded":"2023-11-09T03:48:23.549391+0000","last_fullsized":"2023-11-09T03:48:23.549391+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:53:42.763434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011927730000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":48,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.546848+0000","last_change":"2023-11-09T03:47:03.146470+0000","last_active":"2023-11-09T03:48:23.546848+0000","last_peered":"2023-11-09T03:48:23.546848+0000","last_clean":"2023-11-09T03:48:23.546848+0000","last_became_active":"2023-11-09T03:47:01.587304+0000","last_became_peered":"2023-11-09T03:47:01.587304+0000","last_unstale":"2023-11-09T03:48:23.546848+0000","last_undegraded":"2023-11-09T03:48:23.546848+0000","last_fullsized":"2023-11-09T03:48:23.546848+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:58:23.339287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00110395,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.735660+0000","last_change":"2023-11-09T03:48:23.570389+0000","last_active":"2023-11-09T03:48:24.735660+0000","last_peered":"2023-11-09T03:48:24.735660+0000","last_clean":"2023-11-09T03:48:24.735660+0000","last_became_active":"2023-11-09T03:48:23.570111+0000","last_became_peered":"2023-11-09T03:48:23.570111+0000","last_unstale":"2023-11-09T03:48:24.735660+0000","last_undegraded":"2023-11-09T03:48:24.735660+0000","last_fullsized":"2023-11-09T03:48:24.735660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:43:03.762612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.557818+0000","last_change":"2023-11-09T03:48:22.561952+0000","last_active":"2023-11-09T03:48:23.557818+0000","last_peered":"2023-11-09T03:48:23.557818+0000","last_clean":"2023-11-09T03:48:23.557818+0000","last_became_active":"2023-11-09T03:48:22.561647+0000","last_became_peered":"2023-11-09T03:48:22.561647+0000","last_unstale":"2023-11-09T03:48:23.557818+0000","last_undegraded":"2023-11-09T03:48:23.557818+0000","last_fullsized":"2023-11-09T03:48:23.557818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:00:36.242435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.557076+0000","last_change":"2023-11-09T03:48:01.449406+0000","last_active":"2023-11-09T03:48:23.557076+0000","last_peered":"2023-11-09T03:48:23.557076+0000","last_clean":"2023-11-09T03:48:23.557076+0000","last_became_active":"2023-11-09T03:48:01.449130+0000","last_became_peered":"2023-11-09T03:48:01.449130+0000","last_unstale":"2023-11-09T03:48:23.557076+0000","last_undegraded":"2023-11-09T03:48:23.557076+0000","last_fullsized":"2023-11-09T03:48:23.557076+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:20:36.303940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.547214+0000","last_change":"2023-11-09T03:48:02.441801+0000","last_active":"2023-11-09T03:48:23.547214+0000","last_peered":"2023-11-09T03:48:23.547214+0000","last_clean":"2023-11-09T03:48:23.547214+0000","last_became_active":"2023-11-09T03:48:00.444975+0000","last_became_peered":"2023-11-09T03:48:00.444975+0000","last_unstale":"2023-11-09T03:48:23.547214+0000","last_undegraded":"2023-11-09T03:48:23.547214+0000","last_fullsized":"2023-11-09T03:48:23.547214+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:39:31.347349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061165099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":48,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.550743+0000","last_change":"2023-11-09T03:47:03.147367+0000","last_active":"2023-11-09T03:48:23.550743+0000","last_peered":"2023-11-09T03:48:23.550743+0000","last_clean":"2023-11-09T03:48:23.550743+0000","last_became_active":"2023-11-09T03:47:01.588359+0000","last_became_peered":"2023-11-09T03:47:01.588359+0000","last_unstale":"2023-11-09T03:48:23.550743+0000","last_undegraded":"2023-11-09T03:48:23.550743+0000","last_fullsized":"2023-11-09T03:48:23.550743+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:21:34.303068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014106660000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.737054+0000","last_change":"2023-11-09T03:48:23.559136+0000","last_active":"2023-11-09T03:48:24.737054+0000","last_peered":"2023-11-09T03:48:24.737054+0000","last_clean":"2023-11-09T03:48:24.737054+0000","last_became_active":"2023-11-09T03:48:23.558875+0000","last_became_peered":"2023-11-09T03:48:23.558875+0000","last_unstale":"2023-11-09T03:48:24.737054+0000","last_undegraded":"2023-11-09T03:48:24.737054+0000","last_fullsized":"2023-11-09T03:48:24.737054+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:06:36.963770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.551957+0000","last_change":"2023-11-09T03:48:22.556245+0000","last_active":"2023-11-09T03:48:23.551957+0000","last_peered":"2023-11-09T03:48:23.551957+0000","last_clean":"2023-11-09T03:48:23.551957+0000","last_became_active":"2023-11-09T03:48:22.555974+0000","last_became_peered":"2023-11-09T03:48:22.555974+0000","last_unstale":"2023-11-09T03:48:23.551957+0000","last_undegraded":"2023-11-09T03:48:23.551957+0000","last_fullsized":"2023-11-09T03:48:23.551957+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:23:17.565705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.552947+0000","last_change":"2023-11-09T03:48:01.451866+0000","last_active":"2023-11-09T03:48:23.552947+0000","last_peered":"2023-11-09T03:48:23.552947+0000","last_clean":"2023-11-09T03:48:23.552947+0000","last_became_active":"2023-11-09T03:48:01.451350+0000","last_became_peered":"2023-11-09T03:48:01.451350+0000","last_unstale":"2023-11-09T03:48:23.552947+0000","last_undegraded":"2023-11-09T03:48:23.552947+0000","last_fullsized":"2023-11-09T03:48:23.552947+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:36.735029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.548008+0000","last_change":"2023-11-09T03:48:02.441648+0000","last_active":"2023-11-09T03:48:23.548008+0000","last_peered":"2023-11-09T03:48:23.548008+0000","last_clean":"2023-11-09T03:48:23.548008+0000","last_became_active":"2023-11-09T03:48:00.448541+0000","last_became_peered":"2023-11-09T03:48:00.448541+0000","last_unstale":"2023-11-09T03:48:23.548008+0000","last_undegraded":"2023-11-09T03:48:23.548008+0000","last_fullsized":"2023-11-09T03:48:23.548008+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:35.729023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.551401+0000","last_change":"2023-11-09T03:47:02.581682+0000","last_active":"2023-11-09T03:48:23.551401+0000","last_peered":"2023-11-09T03:48:23.551401+0000","last_clean":"2023-11-09T03:48:23.551401+0000","last_became_active":"2023-11-09T03:47:01.593051+0000","last_became_peered":"2023-11-09T03:47:01.593051+0000","last_unstale":"2023-11-09T03:48:23.551401+0000","last_undegraded":"2023-11-09T03:48:23.551401+0000","last_fullsized":"2023-11-09T03:48:23.551401+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047277799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.737198+0000","last_change":"2023-11-09T03:48:23.571807+0000","last_active":"2023-11-09T03:48:24.737198+0000","last_peered":"2023-11-09T03:48:24.737198+0000","last_clean":"2023-11-09T03:48:24.737198+0000","last_became_active":"2023-11-09T03:48:23.571498+0000","last_became_peered":"2023-11-09T03:48:23.571498+0000","last_unstale":"2023-11-09T03:48:24.737198+0000","last_undegraded":"2023-11-09T03:48:24.737198+0000","last_fullsized":"2023-11-09T03:48:24.737198+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:33:02.457076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.546372+0000","last_change":"2023-11-09T03:48:22.563908+0000","last_active":"2023-11-09T03:48:23.546372+0000","last_peered":"2023-11-09T03:48:23.546372+0000","last_clean":"2023-11-09T03:48:23.546372+0000","last_became_active":"2023-11-09T03:48:22.563651+0000","last_became_peered":"2023-11-09T03:48:22.563651+0000","last_unstale":"2023-11-09T03:48:23.546372+0000","last_undegraded":"2023-11-09T03:48:23.546372+0000","last_fullsized":"2023-11-09T03:48:23.546372+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:32:36.444225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.551606+0000","last_change":"2023-11-09T03:48:01.455784+0000","last_active":"2023-11-09T03:48:23.551606+0000","last_peered":"2023-11-09T03:48:23.551606+0000","last_clean":"2023-11-09T03:48:23.551606+0000","last_became_active":"2023-11-09T03:48:01.455532+0000","last_became_peered":"2023-11-09T03:48:01.455532+0000","last_unstale":"2023-11-09T03:48:23.551606+0000","last_undegraded":"2023-11-09T03:48:23.551606+0000","last_fullsized":"2023-11-09T03:48:23.551606+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:21:23.156761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.553163+0000","last_change":"2023-11-09T03:48:02.611298+0000","last_active":"2023-11-09T03:48:23.553163+0000","last_peered":"2023-11-09T03:48:23.553163+0000","last_clean":"2023-11-09T03:48:23.553163+0000","last_became_active":"2023-11-09T03:48:00.441261+0000","last_became_peered":"2023-11-09T03:48:00.441261+0000","last_unstale":"2023-11-09T03:48:23.553163+0000","last_undegraded":"2023-11-09T03:48:23.553163+0000","last_fullsized":"2023-11-09T03:48:23.553163+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:50:55.299235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081460199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.550920+0000","last_change":"2023-11-09T03:47:03.146309+0000","last_active":"2023-11-09T03:48:23.550920+0000","last_peered":"2023-11-09T03:48:23.550920+0000","last_clean":"2023-11-09T03:48:23.550920+0000","last_became_active":"2023-11-09T03:47:01.587560+0000","last_became_peered":"2023-11-09T03:47:01.587560+0000","last_unstale":"2023-11-09T03:48:23.550920+0000","last_undegraded":"2023-11-09T03:48:23.550920+0000","last_fullsized":"2023-11-09T03:48:23.550920+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:27:55.271945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070988400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.558930+0000","last_change":"2023-11-09T03:48:23.565975+0000","last_active":"2023-11-09T03:48:24.558930+0000","last_peered":"2023-11-09T03:48:24.558930+0000","last_clean":"2023-11-09T03:48:24.558930+0000","last_became_active":"2023-11-09T03:48:23.565705+0000","last_became_peered":"2023-11-09T03:48:23.565705+0000","last_unstale":"2023-11-09T03:48:24.558930+0000","last_undegraded":"2023-11-09T03:48:24.558930+0000","last_fullsized":"2023-11-09T03:48:24.558930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:17:46.479994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.548463+0000","last_change":"2023-11-09T03:48:22.561557+0000","last_active":"2023-11-09T03:48:23.548463+0000","last_peered":"2023-11-09T03:48:23.548463+0000","last_clean":"2023-11-09T03:48:23.548463+0000","last_became_active":"2023-11-09T03:48:22.561249+0000","last_became_peered":"2023-11-09T03:48:22.561249+0000","last_unstale":"2023-11-09T03:48:23.548463+0000","last_undegraded":"2023-11-09T03:48:23.548463+0000","last_fullsized":"2023-11-09T03:48:23.548463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:33:10.584186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.554792+0000","last_change":"2023-11-09T03:48:01.452477+0000","last_active":"2023-11-09T03:48:23.554792+0000","last_peered":"2023-11-09T03:48:23.554792+0000","last_clean":"2023-11-09T03:48:23.554792+0000","last_became_active":"2023-11-09T03:48:01.452186+0000","last_became_peered":"2023-11-09T03:48:01.452186+0000","last_unstale":"2023-11-09T03:48:23.554792+0000","last_undegraded":"2023-11-09T03:48:23.554792+0000","last_fullsized":"2023-11-09T03:48:23.554792+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:32:21.825778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.548265+0000","last_change":"2023-11-09T03:48:02.442101+0000","last_active":"2023-11-09T03:48:23.548265+0000","last_peered":"2023-11-09T03:48:23.548265+0000","last_clean":"2023-11-09T03:48:23.548265+0000","last_became_active":"2023-11-09T03:48:00.449354+0000","last_became_peered":"2023-11-09T03:48:00.449354+0000","last_unstale":"2023-11-09T03:48:23.548265+0000","last_undegraded":"2023-11-09T03:48:23.548265+0000","last_fullsized":"2023-11-09T03:48:23.548265+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:06:32.906582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093455000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":49,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.552922+0000","last_change":"2023-11-09T03:47:02.725125+0000","last_active":"2023-11-09T03:48:23.552922+0000","last_peered":"2023-11-09T03:48:23.552922+0000","last_clean":"2023-11-09T03:48:23.552922+0000","last_became_active":"2023-11-09T03:47:01.591521+0000","last_became_peered":"2023-11-09T03:47:01.591521+0000","last_unstale":"2023-11-09T03:48:23.552922+0000","last_undegraded":"2023-11-09T03:48:23.552922+0000","last_fullsized":"2023-11-09T03:48:23.552922+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:52:40.076542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000758675,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.736536+0000","last_change":"2023-11-09T03:48:23.562707+0000","last_active":"2023-11-09T03:48:24.736536+0000","last_peered":"2023-11-09T03:48:24.736536+0000","last_clean":"2023-11-09T03:48:24.736536+0000","last_became_active":"2023-11-09T03:48:23.562427+0000","last_became_peered":"2023-11-09T03:48:23.562427+0000","last_unstale":"2023-11-09T03:48:24.736536+0000","last_undegraded":"2023-11-09T03:48:24.736536+0000","last_fullsized":"2023-11-09T03:48:24.736536+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:24:49.225093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.574142+0000","last_change":"2023-11-09T03:48:22.561189+0000","last_active":"2023-11-09T03:48:23.574142+0000","last_peered":"2023-11-09T03:48:23.574142+0000","last_clean":"2023-11-09T03:48:23.574142+0000","last_became_active":"2023-11-09T03:48:22.560938+0000","last_became_peered":"2023-11-09T03:48:22.560938+0000","last_unstale":"2023-11-09T03:48:23.574142+0000","last_undegraded":"2023-11-09T03:48:23.574142+0000","last_fullsized":"2023-11-09T03:48:23.574142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:35:27.804836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.550904+0000","last_change":"2023-11-09T03:48:01.448129+0000","last_active":"2023-11-09T03:48:23.550904+0000","last_peered":"2023-11-09T03:48:23.550904+0000","last_clean":"2023-11-09T03:48:23.550904+0000","last_became_active":"2023-11-09T03:48:01.447844+0000","last_became_peered":"2023-11-09T03:48:01.447844+0000","last_unstale":"2023-11-09T03:48:23.550904+0000","last_undegraded":"2023-11-09T03:48:23.550904+0000","last_fullsized":"2023-11-09T03:48:23.550904+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:54:52.433604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.554716+0000","last_change":"2023-11-09T03:48:02.612450+0000","last_active":"2023-11-09T03:48:23.554716+0000","last_peered":"2023-11-09T03:48:23.554716+0000","last_clean":"2023-11-09T03:48:23.554716+0000","last_became_active":"2023-11-09T03:48:00.441855+0000","last_became_peered":"2023-11-09T03:48:00.441855+0000","last_unstale":"2023-11-09T03:48:23.554716+0000","last_undegraded":"2023-11-09T03:48:23.554716+0000","last_fullsized":"2023-11-09T03:48:23.554716+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:53:42.763434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095368899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":48,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.547455+0000","last_change":"2023-11-09T03:47:03.147289+0000","last_active":"2023-11-09T03:48:23.547455+0000","last_peered":"2023-11-09T03:48:23.547455+0000","last_clean":"2023-11-09T03:48:23.547455+0000","last_became_active":"2023-11-09T03:47:01.587784+0000","last_became_peered":"2023-11-09T03:47:01.587784+0000","last_unstale":"2023-11-09T03:48:23.547455+0000","last_undegraded":"2023-11-09T03:48:23.547455+0000","last_fullsized":"2023-11-09T03:48:23.547455+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:11:23.824112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038781299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.737976+0000","last_change":"2023-11-09T03:48:23.566137+0000","last_active":"2023-11-09T03:48:24.737976+0000","last_peered":"2023-11-09T03:48:24.737976+0000","last_clean":"2023-11-09T03:48:24.737976+0000","last_became_active":"2023-11-09T03:48:23.565825+0000","last_became_peered":"2023-11-09T03:48:23.565825+0000","last_unstale":"2023-11-09T03:48:24.737976+0000","last_undegraded":"2023-11-09T03:48:24.737976+0000","last_fullsized":"2023-11-09T03:48:24.737976+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:20:33.663992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.578830+0000","last_change":"2023-11-09T03:48:22.564955+0000","last_active":"2023-11-09T03:48:23.578830+0000","last_peered":"2023-11-09T03:48:23.578830+0000","last_clean":"2023-11-09T03:48:23.578830+0000","last_became_active":"2023-11-09T03:48:22.564677+0000","last_became_peered":"2023-11-09T03:48:22.564677+0000","last_unstale":"2023-11-09T03:48:23.578830+0000","last_undegraded":"2023-11-09T03:48:23.578830+0000","last_fullsized":"2023-11-09T03:48:23.578830+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:31.602666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.549658+0000","last_change":"2023-11-09T03:48:01.449617+0000","last_active":"2023-11-09T03:48:23.549658+0000","last_peered":"2023-11-09T03:48:23.549658+0000","last_clean":"2023-11-09T03:48:23.549658+0000","last_became_active":"2023-11-09T03:48:01.449328+0000","last_became_peered":"2023-11-09T03:48:01.449328+0000","last_unstale":"2023-11-09T03:48:23.549658+0000","last_undegraded":"2023-11-09T03:48:23.549658+0000","last_fullsized":"2023-11-09T03:48:23.549658+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:51:18.760059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.546781+0000","last_change":"2023-11-09T03:48:02.442215+0000","last_active":"2023-11-09T03:48:23.546781+0000","last_peered":"2023-11-09T03:48:23.546781+0000","last_clean":"2023-11-09T03:48:23.546781+0000","last_became_active":"2023-11-09T03:48:00.444720+0000","last_became_peered":"2023-11-09T03:48:00.444720+0000","last_unstale":"2023-11-09T03:48:23.546781+0000","last_undegraded":"2023-11-09T03:48:23.546781+0000","last_fullsized":"2023-11-09T03:48:23.546781+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:55:55.735668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001407176,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":48,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.555233+0000","last_change":"2023-11-09T03:47:02.619346+0000","last_active":"2023-11-09T03:48:23.555233+0000","last_peered":"2023-11-09T03:48:23.555233+0000","last_clean":"2023-11-09T03:48:23.555233+0000","last_became_active":"2023-11-09T03:47:01.590389+0000","last_became_peered":"2023-11-09T03:47:01.590389+0000","last_unstale":"2023-11-09T03:48:23.555233+0000","last_undegraded":"2023-11-09T03:48:23.555233+0000","last_fullsized":"2023-11-09T03:48:23.555233+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000384072,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.736362+0000","last_change":"2023-11-09T03:48:23.562902+0000","last_active":"2023-11-09T03:48:24.736362+0000","last_peered":"2023-11-09T03:48:24.736362+0000","last_clean":"2023-11-09T03:48:24.736362+0000","last_became_active":"2023-11-09T03:48:23.562626+0000","last_became_peered":"2023-11-09T03:48:23.562626+0000","last_unstale":"2023-11-09T03:48:24.736362+0000","last_undegraded":"2023-11-09T03:48:24.736362+0000","last_fullsized":"2023-11-09T03:48:24.736362+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:28:07.562442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.546034+0000","last_change":"2023-11-09T03:48:22.562601+0000","last_active":"2023-11-09T03:48:23.546034+0000","last_peered":"2023-11-09T03:48:23.546034+0000","last_clean":"2023-11-09T03:48:23.546034+0000","last_became_active":"2023-11-09T03:48:22.562326+0000","last_became_peered":"2023-11-09T03:48:22.562326+0000","last_unstale":"2023-11-09T03:48:23.546034+0000","last_undegraded":"2023-11-09T03:48:23.546034+0000","last_fullsized":"2023-11-09T03:48:23.546034+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:59:17.871302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.555248+0000","last_change":"2023-11-09T03:48:01.450018+0000","last_active":"2023-11-09T03:48:23.555248+0000","last_peered":"2023-11-09T03:48:23.555248+0000","last_clean":"2023-11-09T03:48:23.555248+0000","last_became_active":"2023-11-09T03:48:01.449749+0000","last_became_peered":"2023-11-09T03:48:01.449749+0000","last_unstale":"2023-11-09T03:48:23.555248+0000","last_undegraded":"2023-11-09T03:48:23.555248+0000","last_fullsized":"2023-11-09T03:48:23.555248+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:21:23.156761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.553934+0000","last_change":"2023-11-09T03:48:02.524216+0000","last_active":"2023-11-09T03:48:23.553934+0000","last_peered":"2023-11-09T03:48:23.553934+0000","last_clean":"2023-11-09T03:48:23.553934+0000","last_became_active":"2023-11-09T03:48:00.446972+0000","last_became_peered":"2023-11-09T03:48:00.446972+0000","last_unstale":"2023-11-09T03:48:23.553934+0000","last_undegraded":"2023-11-09T03:48:23.553934+0000","last_fullsized":"2023-11-09T03:48:23.553934+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:54:07.987773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000825255,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":52,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.548707+0000","last_change":"2023-11-09T03:47:02.582393+0000","last_active":"2023-11-09T03:48:23.548707+0000","last_peered":"2023-11-09T03:48:23.548707+0000","last_clean":"2023-11-09T03:48:23.548707+0000","last_became_active":"2023-11-09T03:47:01.591752+0000","last_became_peered":"2023-11-09T03:47:01.591752+0000","last_unstale":"2023-11-09T03:48:23.548707+0000","last_undegraded":"2023-11-09T03:48:23.548707+0000","last_fullsized":"2023-11-09T03:48:23.548707+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:03.025119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067021599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.735450+0000","last_change":"2023-11-09T03:48:23.566889+0000","last_active":"2023-11-09T03:48:24.735450+0000","last_peered":"2023-11-09T03:48:24.735450+0000","last_clean":"2023-11-09T03:48:24.735450+0000","last_became_active":"2023-11-09T03:48:23.566625+0000","last_became_peered":"2023-11-09T03:48:23.566625+0000","last_unstale":"2023-11-09T03:48:24.735450+0000","last_undegraded":"2023-11-09T03:48:24.735450+0000","last_fullsized":"2023-11-09T03:48:24.735450+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:37:19.707161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.559627+0000","last_change":"2023-11-09T03:48:22.562752+0000","last_active":"2023-11-09T03:48:23.559627+0000","last_peered":"2023-11-09T03:48:23.559627+0000","last_clean":"2023-11-09T03:48:23.559627+0000","last_became_active":"2023-11-09T03:48:22.562474+0000","last_became_peered":"2023-11-09T03:48:22.562474+0000","last_unstale":"2023-11-09T03:48:23.559627+0000","last_undegraded":"2023-11-09T03:48:23.559627+0000","last_fullsized":"2023-11-09T03:48:23.559627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:15:26.050548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.554694+0000","last_change":"2023-11-09T03:48:01.451462+0000","last_active":"2023-11-09T03:48:23.554694+0000","last_peered":"2023-11-09T03:48:23.554694+0000","last_clean":"2023-11-09T03:48:23.554694+0000","last_became_active":"2023-11-09T03:48:01.451188+0000","last_became_peered":"2023-11-09T03:48:01.451188+0000","last_unstale":"2023-11-09T03:48:23.554694+0000","last_undegraded":"2023-11-09T03:48:23.554694+0000","last_fullsized":"2023-11-09T03:48:23.554694+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:28.631801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.558930+0000","last_change":"2023-11-09T03:48:02.682290+0000","last_active":"2023-11-09T03:48:23.558930+0000","last_peered":"2023-11-09T03:48:23.558930+0000","last_clean":"2023-11-09T03:48:23.558930+0000","last_became_active":"2023-11-09T03:48:00.444246+0000","last_became_peered":"2023-11-09T03:48:00.444246+0000","last_unstale":"2023-11-09T03:48:23.558930+0000","last_undegraded":"2023-11-09T03:48:23.558930+0000","last_fullsized":"2023-11-09T03:48:23.558930+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:09.487058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000423359,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":48,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.555155+0000","last_change":"2023-11-09T03:47:03.028744+0000","last_active":"2023-11-09T03:48:23.555155+0000","last_peered":"2023-11-09T03:48:23.555155+0000","last_clean":"2023-11-09T03:48:23.555155+0000","last_became_active":"2023-11-09T03:47:01.593890+0000","last_became_peered":"2023-11-09T03:47:01.593890+0000","last_unstale":"2023-11-09T03:48:23.555155+0000","last_undegraded":"2023-11-09T03:48:23.555155+0000","last_fullsized":"2023-11-09T03:48:23.555155+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:05.754357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053008299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.737873+0000","last_change":"2023-11-09T03:48:23.573683+0000","last_active":"2023-11-09T03:48:24.737873+0000","last_peered":"2023-11-09T03:48:24.737873+0000","last_clean":"2023-11-09T03:48:24.737873+0000","last_became_active":"2023-11-09T03:48:23.573424+0000","last_became_peered":"2023-11-09T03:48:23.573424+0000","last_unstale":"2023-11-09T03:48:24.737873+0000","last_undegraded":"2023-11-09T03:48:24.737873+0000","last_fullsized":"2023-11-09T03:48:24.737873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:21:22.822876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.556997+0000","last_change":"2023-11-09T03:48:22.560645+0000","last_active":"2023-11-09T03:48:23.556997+0000","last_peered":"2023-11-09T03:48:23.556997+0000","last_clean":"2023-11-09T03:48:23.556997+0000","last_became_active":"2023-11-09T03:48:22.560328+0000","last_became_peered":"2023-11-09T03:48:22.560328+0000","last_unstale":"2023-11-09T03:48:23.556997+0000","last_undegraded":"2023-11-09T03:48:23.556997+0000","last_fullsized":"2023-11-09T03:48:23.556997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:31.602666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.559698+0000","last_change":"2023-11-09T03:48:01.450144+0000","last_active":"2023-11-09T03:48:23.559698+0000","last_peered":"2023-11-09T03:48:23.559698+0000","last_clean":"2023-11-09T03:48:23.559698+0000","last_became_active":"2023-11-09T03:48:01.449859+0000","last_became_peered":"2023-11-09T03:48:01.449859+0000","last_unstale":"2023-11-09T03:48:23.559698+0000","last_undegraded":"2023-11-09T03:48:23.559698+0000","last_fullsized":"2023-11-09T03:48:23.559698+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:57:56.855869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.551046+0000","last_change":"2023-11-09T03:48:02.681869+0000","last_active":"2023-11-09T03:48:23.551046+0000","last_peered":"2023-11-09T03:48:23.551046+0000","last_clean":"2023-11-09T03:48:23.551046+0000","last_became_active":"2023-11-09T03:48:00.444928+0000","last_became_peered":"2023-11-09T03:48:00.444928+0000","last_unstale":"2023-11-09T03:48:23.551046+0000","last_undegraded":"2023-11-09T03:48:23.551046+0000","last_fullsized":"2023-11-09T03:48:23.551046+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:59:58.695636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044822599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.559016+0000","last_change":"2023-11-09T03:47:03.146783+0000","last_active":"2023-11-09T03:48:23.559016+0000","last_peered":"2023-11-09T03:48:23.559016+0000","last_clean":"2023-11-09T03:48:23.559016+0000","last_became_active":"2023-11-09T03:47:01.586923+0000","last_became_peered":"2023-11-09T03:47:01.586923+0000","last_unstale":"2023-11-09T03:48:23.559016+0000","last_undegraded":"2023-11-09T03:48:23.559016+0000","last_fullsized":"2023-11-09T03:48:23.559016+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:37:50.326442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060384700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.735606+0000","last_change":"2023-11-09T03:48:23.568213+0000","last_active":"2023-11-09T03:48:24.735606+0000","last_peered":"2023-11-09T03:48:24.735606+0000","last_clean":"2023-11-09T03:48:24.735606+0000","last_became_active":"2023-11-09T03:48:23.567968+0000","last_became_peered":"2023-11-09T03:48:23.567968+0000","last_unstale":"2023-11-09T03:48:24.735606+0000","last_undegraded":"2023-11-09T03:48:24.735606+0000","last_fullsized":"2023-11-09T03:48:24.735606+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:39:24.065800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.583767+0000","last_change":"2023-11-09T03:48:22.565142+0000","last_active":"2023-11-09T03:48:23.583767+0000","last_peered":"2023-11-09T03:48:23.583767+0000","last_clean":"2023-11-09T03:48:23.583767+0000","last_became_active":"2023-11-09T03:48:22.564856+0000","last_became_peered":"2023-11-09T03:48:22.564856+0000","last_unstale":"2023-11-09T03:48:23.583767+0000","last_undegraded":"2023-11-09T03:48:23.583767+0000","last_fullsized":"2023-11-09T03:48:23.583767+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:52:19.334507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.550600+0000","last_change":"2023-11-09T03:48:01.448383+0000","last_active":"2023-11-09T03:48:23.550600+0000","last_peered":"2023-11-09T03:48:23.550600+0000","last_clean":"2023-11-09T03:48:23.550600+0000","last_became_active":"2023-11-09T03:48:01.448050+0000","last_became_peered":"2023-11-09T03:48:01.448050+0000","last_unstale":"2023-11-09T03:48:23.550600+0000","last_undegraded":"2023-11-09T03:48:23.550600+0000","last_fullsized":"2023-11-09T03:48:23.550600+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:53:44.478182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.552808+0000","last_change":"2023-11-09T03:48:02.523794+0000","last_active":"2023-11-09T03:48:23.552808+0000","last_peered":"2023-11-09T03:48:23.552808+0000","last_clean":"2023-11-09T03:48:23.552808+0000","last_became_active":"2023-11-09T03:48:00.440255+0000","last_became_peered":"2023-11-09T03:48:00.440255+0000","last_unstale":"2023-11-09T03:48:23.552808+0000","last_undegraded":"2023-11-09T03:48:23.552808+0000","last_fullsized":"2023-11-09T03:48:23.552808+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:58:02.392696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064572299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.550995+0000","last_change":"2023-11-09T03:47:03.146580+0000","last_active":"2023-11-09T03:48:23.550995+0000","last_peered":"2023-11-09T03:48:23.550995+0000","last_clean":"2023-11-09T03:48:23.550995+0000","last_became_active":"2023-11-09T03:47:01.589201+0000","last_became_peered":"2023-11-09T03:47:01.589201+0000","last_unstale":"2023-11-09T03:48:23.550995+0000","last_undegraded":"2023-11-09T03:48:23.550995+0000","last_fullsized":"2023-11-09T03:48:23.550995+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:14.300421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049706999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-09T03:48:24.737705+0000","last_change":"2023-11-09T03:48:23.567611+0000","last_active":"2023-11-09T03:48:24.737705+0000","last_peered":"2023-11-09T03:48:24.737705+0000","last_clean":"2023-11-09T03:48:24.737705+0000","last_became_active":"2023-11-09T03:48:23.567333+0000","last_became_peered":"2023-11-09T03:48:23.567333+0000","last_unstale":"2023-11-09T03:48:24.737705+0000","last_undegraded":"2023-11-09T03:48:24.737705+0000","last_fullsized":"2023-11-09T03:48:24.737705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:22.537305+0000","last_clean_scrub_stamp":"2023-11-09T03:48:22.537305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:50:13.511767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.577058+0000","last_change":"2023-11-09T03:48:22.562609+0000","last_active":"2023-11-09T03:48:23.577058+0000","last_peered":"2023-11-09T03:48:23.577058+0000","last_clean":"2023-11-09T03:48:23.577058+0000","last_became_active":"2023-11-09T03:48:22.562328+0000","last_became_peered":"2023-11-09T03:48:22.562328+0000","last_unstale":"2023-11-09T03:48:23.577058+0000","last_undegraded":"2023-11-09T03:48:23.577058+0000","last_fullsized":"2023-11-09T03:48:23.577058+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:21.532334+0000","last_clean_scrub_stamp":"2023-11-09T03:48:21.532334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:15:26.050548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.555772+0000","last_change":"2023-11-09T03:48:01.451284+0000","last_active":"2023-11-09T03:48:23.555772+0000","last_peered":"2023-11-09T03:48:23.555772+0000","last_clean":"2023-11-09T03:48:23.555772+0000","last_became_active":"2023-11-09T03:48:01.450954+0000","last_became_peered":"2023-11-09T03:48:01.450954+0000","last_unstale":"2023-11-09T03:48:23.555772+0000","last_undegraded":"2023-11-09T03:48:23.555772+0000","last_fullsized":"2023-11-09T03:48:23.555772+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:49:24.680020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.552837+0000","last_change":"2023-11-09T03:48:02.733038+0000","last_active":"2023-11-09T03:48:23.552837+0000","last_peered":"2023-11-09T03:48:23.552837+0000","last_clean":"2023-11-09T03:48:23.552837+0000","last_became_active":"2023-11-09T03:48:00.446661+0000","last_became_peered":"2023-11-09T03:48:00.446661+0000","last_unstale":"2023-11-09T03:48:23.552837+0000","last_undegraded":"2023-11-09T03:48:23.552837+0000","last_fullsized":"2023-11-09T03:48:23.552837+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:05:08.649312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040102300000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":48,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.548399+0000","last_change":"2023-11-09T03:47:03.147851+0000","last_active":"2023-11-09T03:48:23.548399+0000","last_peered":"2023-11-09T03:48:23.548399+0000","last_clean":"2023-11-09T03:48:23.548399+0000","last_became_active":"2023-11-09T03:47:01.589893+0000","last_became_peered":"2023-11-09T03:47:01.589893+0000","last_unstale":"2023-11-09T03:48:23.548399+0000","last_undegraded":"2023-11-09T03:48:23.548399+0000","last_fullsized":"2023-11-09T03:48:23.548399+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:08:17.183462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029151399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":217,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-09T03:48:23.551805+0000","last_change":"2023-11-09T03:45:21.030926+0000","last_active":"2023-11-09T03:48:23.551805+0000","last_peered":"2023-11-09T03:48:23.551805+0000","last_clean":"2023-11-09T03:48:23.551805+0000","last_became_active":"2023-11-09T03:45:21.029801+0000","last_became_peered":"2023-11-09T03:45:21.029801+0000","last_unstale":"2023-11-09T03:48:23.551805+0000","last_undegraded":"2023-11-09T03:48:23.551805+0000","last_fullsized":"2023-11-09T03:48:23.551805+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_clean_scrub_stamp":"2023-11-09T03:45:19.768433+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:12:59.153713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32764,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1617920,"data_stored":1082455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.503,"15min":0.499},"min":{"1min":0.36599999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.432},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53200000000000003,"15min":0.52100000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.66700000000000004,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.63300000000000001}]},{"osd":1,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.38600000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70599999999999996,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53000000000000003,"15min":0.51900000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.60799999999999998}]},{"osd":2,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52700000000000002,"15min":0.51500000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72099999999999997,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.65400000000000003,"15min":0.69699999999999995},"min":{"1min":0.41599999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.73299999999999998,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.52200000000000002,"15min":0.51300000000000001},"min":{"1min":0.42599999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.67300000000000004,"5min":0.754,"15min":0.754},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56399999999999995,"15min":0.54500000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81499999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.81499999999999995}]},{"osd":4,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.45500000000000002,"15min":0.44400000000000001},"min":{"1min":0.32000000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.5,"15min":0.48199999999999998},"min":{"1min":0.32300000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.73199999999999998,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.72999999999999998}]},{"osd":5,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.48499999999999999,"15min":0.46000000000000002},"min":{"1min":0.30599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76300000000000001,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51900000000000002,"15min":0.499},"min":{"1min":0.32700000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.79700000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.79700000000000004}]},{"osd":6,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55700000000000005,"15min":0.54300000000000004},"min":{"1min":0.38900000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.74299999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54000000000000004,"15min":0.52200000000000002},"min":{"1min":0.40400000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.83199999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1024000,"data_stored":492627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.504,"15min":0.503},"min":{"1min":0.34100000000000003,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.65700000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51400000000000001,"15min":0.498},"min":{"1min":0.36699999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73199999999999998,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.42799999999999999}]},{"osd":1,"last update":"Thu Nov 9 03:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52200000000000002,"15min":0.51200000000000001},"min":{"1min":0.35899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.39300000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.65300000000000002,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.47199999999999998}]},{"osd":2,"last update":"Thu Nov 9 03:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.42199999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.65400000000000003,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52900000000000003,"15min":0.52100000000000002},"min":{"1min":0.39900000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.70399999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.46200000000000002}]},{"osd":3,"last update":"Thu Nov 9 03:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52500000000000002,"15min":0.52400000000000002},"min":{"1min":0.39100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.42799999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.68400000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.56799999999999995}]},{"osd":4,"last update":"Thu Nov 9 03:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.497,"15min":0.501},"min":{"1min":0.32000000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.66700000000000004,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53200000000000003,"15min":0.52900000000000003},"min":{"1min":0.34599999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.76900000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.38}]},{"osd":5,"last update":"Thu Nov 9 03:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57999999999999996,"15min":0.57699999999999996},"min":{"1min":0.39500000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80800000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55000000000000004,"15min":0.55600000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.78800000000000003,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.308}]},{"osd":7,"last update":"Thu Nov 9 03:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.45200000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83199999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56699999999999995,"15min":0.55800000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71499999999999997,"5min":0.877,"15min":0.877},"last":0.49199999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1019904,"data_stored":492045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.504,"15min":0.48899999999999999},"min":{"1min":0.36199999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.376,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.57399999999999995}]},{"osd":1,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88300000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.36599999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.84199999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.53200000000000003}]},{"osd":2,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.45800000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.55600000000000005},"min":{"1min":0.40300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.59299999999999997}]},{"osd":3,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57099999999999995,"15min":0.56499999999999995},"min":{"1min":0.39400000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55200000000000005,"15min":0.55100000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.626}]},{"osd":4,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.49199999999999999,"15min":0.48399999999999999},"min":{"1min":0.32400000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.48999999999999999,"15min":0.48299999999999998},"min":{"1min":0.29699999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.70899999999999996}]},{"osd":6,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.50600000000000001,"15min":0.51000000000000001},"min":{"1min":0.33700000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.755,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.53300000000000003,"15min":0.51700000000000002},"min":{"1min":0.38300000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.64100000000000001}]},{"osd":7,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.34699999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.81200000000000006,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.61899999999999999,"15min":0.62},"min":{"1min":0.432,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84899999999999998,"5min":1.01,"15min":1.01},"last":0.72799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574889,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1036288,"data_stored":493711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69399999999999995,"5min":1.034,"15min":1.034},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.38800000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78300000000000003,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.44900000000000001}]},{"osd":1,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.34000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.74299999999999999,"5min":1.093,"15min":1.093},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54200000000000004,"15min":0.54100000000000004},"min":{"1min":0.36399999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71899999999999997,"5min":0.871,"15min":0.871},"last":0.46700000000000003}]},{"osd":2,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54100000000000004,"15min":0.55400000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.69199999999999995,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54800000000000004,"15min":0.54500000000000004},"min":{"1min":0.39700000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.40699999999999997}]},{"osd":3,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.433,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.70199999999999996,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.376,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72499999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.52600000000000002}]},{"osd":5,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.33300000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.78700000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.318,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81100000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.495}]},{"osd":6,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.75800000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57899999999999996,"15min":0.58399999999999996},"min":{"1min":0.39600000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.75900000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.54200000000000004}]},{"osd":7,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61599999999999999,"15min":0.629},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83499999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61099999999999999,"15min":0.61899999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79300000000000004,"5min":1.113,"15min":1.113},"last":0.48099999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574890,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1032192,"data_stored":493609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.32100000000000001,"15min":0.32300000000000001},"min":{"1min":0.22,"5min":0.215,"15min":0.215},"max":{"1min":0.40200000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.313,"5min":0.32300000000000001,"15min":0.32400000000000001},"min":{"1min":0.23999999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.47799999999999998,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.307}]},{"osd":1,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.32600000000000001,"15min":0.33000000000000002},"min":{"1min":0.23599999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.42899999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.36699999999999999,"15min":0.373},"min":{"1min":0.28100000000000003,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.47899999999999998,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.379}]},{"osd":2,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.37,"15min":0.372},"min":{"1min":0.27200000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.439,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.379,"15min":0.38300000000000001},"min":{"1min":0.29799999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.45700000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.34300000000000003}]},{"osd":4,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58899999999999997,"15min":0.59499999999999997},"min":{"1min":0.42299999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79000000000000004,"5min":0.87,"15min":0.87},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.59699999999999998,"15min":0.61099999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.70099999999999996}]},{"osd":5,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.60499999999999998,"15min":0.62},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86599999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60699999999999998,"15min":0.61899999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.82099999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.625}]},{"osd":6,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60999999999999999,"15min":0.623},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78200000000000003,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.45400000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.80300000000000005,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.52600000000000002}]},{"osd":7,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.61099999999999999,"15min":0.62},"min":{"1min":0.41099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.70899999999999996,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.621,"15min":0.63700000000000001},"min":{"1min":0.46100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81299999999999994,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.68400000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28024,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343816,"statfs":{"total":96636764160,"available":96608067584,"internally_reserved":0,"allocated":1019904,"data_stored":492009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:25 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32900000000000001,"15min":0.33100000000000002},"min":{"1min":0.20999999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.42599999999999999,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.21199999999999999},{"interface":"front","average":{"1min":0.31,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.23300000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.51400000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.30099999999999999}]},{"osd":1,"last update":"Thu Nov 9 03:48:25 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.32200000000000001,"15min":0.32500000000000001},"min":{"1min":0.249,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.46600000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.36399999999999999,"15min":0.36499999999999999},"min":{"1min":0.252,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.249}]},{"osd":3,"last update":"Thu Nov 9 03:48:25 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.35699999999999998,"15min":0.36299999999999999},"min":{"1min":0.25,"5min":0.22,"15min":0.22},"max":{"1min":0.54100000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.39600000000000002,"15min":0.40200000000000002},"min":{"1min":0.30299999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.495,"5min":0.754,"15min":0.754},"last":0.28399999999999997}]},{"osd":4,"last update":"Thu Nov 9 03:48:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.44800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.435}]},{"osd":5,"last update":"Thu Nov 9 03:48:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56599999999999995,"15min":0.57399999999999995},"min":{"1min":0.38,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.69799999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.434,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.40999999999999998}]},{"osd":6,"last update":"Thu Nov 9 03:48:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60599999999999998,"15min":0.60999999999999999},"min":{"1min":0.438,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59099999999999997,"15min":0.59199999999999997},"min":{"1min":0.436,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.45200000000000001}]},{"osd":7,"last update":"Thu Nov 9 03:48:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.61499999999999999,"15min":0.623},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.39600000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1015808,"data_stored":491975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.313,"15min":0.315},"min":{"1min":0.221,"5min":0.19,"15min":0.19},"max":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.29899999999999999,"5min":0.313,"15min":0.311},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.32500000000000001}]},{"osd":2,"last update":"Thu Nov 9 03:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.314,"15min":0.31},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.34399999999999997,"15min":0.34200000000000003},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.53300000000000003,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.30399999999999999}]},{"osd":3,"last update":"Thu Nov 9 03:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36699999999999999,"15min":0.371},"min":{"1min":0.26000000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.36399999999999999,"15min":0.36899999999999999},"min":{"1min":0.27100000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.376}]},{"osd":4,"last update":"Thu Nov 9 03:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.55400000000000005},"min":{"1min":0.40799999999999997,"5min":0.37,"15min":0.37},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56200000000000006,"15min":0.56100000000000005},"min":{"1min":0.41499999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.53500000000000003}]},{"osd":5,"last update":"Thu Nov 9 03:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.44400000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80500000000000005,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.46300000000000002}]},{"osd":6,"last update":"Thu Nov 9 03:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.41199999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.47699999999999998}]},{"osd":7,"last update":"Thu Nov 9 03:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59099999999999997,"15min":0.60399999999999998},"min":{"1min":0.41399999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.83099999999999996,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.499},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.57299999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32756,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339084,"statfs":{"total":96636764160,"available":96603222016,"internally_reserved":0,"allocated":1609728,"data_stored":1082306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32000000000000001,"15min":0.32400000000000001},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.34599999999999997,"15min":0.36199999999999999},"min":{"1min":0.253,"5min":0.219,"15min":0.219},"max":{"1min":0.58299999999999996,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.34699999999999998}]},{"osd":2,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35799999999999998,"15min":0.36399999999999999},"min":{"1min":0.24299999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.35999999999999999,"15min":0.36899999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.29599999999999999}]},{"osd":3,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.379,"15min":0.38900000000000001},"min":{"1min":0.27300000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.40699999999999997,"15min":0.41799999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.56599999999999995,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.317}]},{"osd":4,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58999999999999997,"15min":0.59399999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.436},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.53100000000000003}]},{"osd":5,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.59199999999999997},"min":{"1min":0.378,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61499999999999999,"15min":0.63100000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80800000000000005,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.50700000000000001}]},{"osd":6,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63400000000000001,"15min":0.64500000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88300000000000001,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.624,"15min":0.63100000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85799999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.46200000000000002}]},{"osd":7,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61299999999999999,"15min":0.621},"min":{"1min":0.39800000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.42599999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.78800000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.45100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-09T03:48:31.242 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-09T03:48:31.242 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-09T03:48:31.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.381+0000 7f490bb6c640 1 Processor -- start 2023-11-09T03:48:31.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.381+0000 7f490bb6c640 1 -- start start 2023-11-09T03:48:31.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.382+0000 7f490bb6c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4904153e60 0x7f490415dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:31.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.382+0000 7f490bb6c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f49040b4880 0x7f49040b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:31.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.382+0000 7f490bb6c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49040cde60 con 0x7f49040cd6d0 2023-11-09T03:48:31.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.382+0000 7f490bb6c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4904160f90 con 0x7f4904153e60 2023-11-09T03:48:31.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.382+0000 7f490bb6c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49040b5200 con 0x7f49040b4880 2023-11-09T03:48:31.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.382+0000 7f4909908640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4904153e60 0x7f490415dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:31.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.382+0000 7f4909908640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4904153e60 0x7f490415dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37914/0 (socket says 172.21.15.76:37914) 2023-11-09T03:48:31.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.382+0000 7f4909908640 1 -- 172.21.15.76:0/3334263357 learned_addr learned my addr 172.21.15.76:0/3334263357 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:31.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.382+0000 7f490a109640 1 --2- 172.21.15.76:0/3334263357 >> v2:172.21.15.113:3300/0 conn(0x7f49040b4880 0x7f49040b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:31.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.382+0000 7f4909908640 1 -- 172.21.15.76:0/3334263357 >> v2:172.21.15.113:3300/0 conn(0x7f49040b4880 msgr2=0x7f49040b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:31.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.382+0000 7f4909908640 1 --2- 172.21.15.76:0/3334263357 >> v2:172.21.15.113:3300/0 conn(0x7f49040b4880 0x7f49040b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.382+0000 7f4909908640 1 -- 172.21.15.76:0/3334263357 >> v1:172.21.15.113:6789/0 conn(0x7f49040cd6d0 legacy=0x7f4904161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:31.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.382+0000 7f4909908640 1 -- 172.21.15.76:0/3334263357 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490415e400 con 0x7f4904153e60 2023-11-09T03:48:31.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.383+0000 7f4909908640 1 --2- 172.21.15.76:0/3334263357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4904153e60 0x7f490415dec0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f49040b8f40 tx=0x7f48fc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2d4a705232885b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:31.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.383+0000 7f4908906640 1 -- 172.21.15.76:0/3334263357 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc02ca80 con 0x7f4904153e60 2023-11-09T03:48:31.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.383+0000 7f4908906640 1 -- 172.21.15.76:0/3334263357 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48fc02cbe0 con 0x7f4904153e60 2023-11-09T03:48:31.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.383+0000 7f4908906640 1 -- 172.21.15.76:0/3334263357 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc0343f0 con 0x7f4904153e60 2023-11-09T03:48:31.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.384+0000 7f490bb6c640 1 -- 172.21.15.76:0/3334263357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4904153e60 msgr2=0x7f490415dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:31.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.384+0000 7f490bb6c640 1 --2- 172.21.15.76:0/3334263357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4904153e60 0x7f490415dec0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f49040b8f40 tx=0x7f48fc02bee0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.384+0000 7f490bb6c640 1 -- 172.21.15.76:0/3334263357 shutdown_connections 2023-11-09T03:48:31.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.384+0000 7f490bb6c640 1 --2- 172.21.15.76:0/3334263357 >> v2:172.21.15.113:3300/0 conn(0x7f49040b4880 0x7f49040b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.384+0000 7f490bb6c640 1 --2- 172.21.15.76:0/3334263357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4904153e60 0x7f490415dec0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.384+0000 7f490bb6c640 1 -- 172.21.15.76:0/3334263357 >> 172.21.15.76:0/3334263357 conn(0x7f49040cf880 msgr2=0x7f49040cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:31.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.384+0000 7f490bb6c640 1 -- 172.21.15.76:0/3334263357 shutdown_connections 2023-11-09T03:48:31.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.384+0000 7f490bb6c640 1 -- 172.21.15.76:0/3334263357 wait complete. 2023-11-09T03:48:31.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.385+0000 7f490bb6c640 1 Processor -- start 2023-11-09T03:48:31.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.385+0000 7f490bb6c640 1 -- start start 2023-11-09T03:48:31.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.386+0000 7f490bb6c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49040b4880 0x7f49040c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:31.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.386+0000 7f490bb6c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49040cd6d0 0x7f490415a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:31.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.386+0000 7f490bb6c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4904153e60 0x7f4904158890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:31.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.386+0000 7f490bb6c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49040cc360 con 0x7f4904153e60 2023-11-09T03:48:31.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.386+0000 7f490bb6c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49040cb6e0 con 0x7f49040cd6d0 2023-11-09T03:48:31.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.386+0000 7f490bb6c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49040cb9c0 con 0x7f49040b4880 2023-11-09T03:48:31.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.386+0000 7f490a109640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4904153e60 0x7f4904158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:31.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.386+0000 7f490a109640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4904153e60 0x7f4904158890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37916/0 (socket says 172.21.15.76:37916) 2023-11-09T03:48:31.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.386+0000 7f490a109640 1 -- 172.21.15.76:0/127545384 learned_addr learned my addr 172.21.15.76:0/127545384 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:31.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.386+0000 7f4909107640 1 --2- 172.21.15.76:0/127545384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49040cd6d0 0x7f490415a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:31.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.386+0000 7f4909908640 1 --2- 172.21.15.76:0/127545384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49040b4880 0x7f49040c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:31.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.386+0000 7f490a109640 1 -- 172.21.15.76:0/127545384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49040b4880 msgr2=0x7f49040c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:31.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.386+0000 7f490a109640 1 --2- 172.21.15.76:0/127545384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49040b4880 0x7f49040c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.386+0000 7f490a109640 1 -- 172.21.15.76:0/127545384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49040cd6d0 msgr2=0x7f490415a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:31.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.386+0000 7f490a109640 1 --2- 172.21.15.76:0/127545384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49040cd6d0 0x7f490415a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.386+0000 7f490a109640 1 -- 172.21.15.76:0/127545384 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48fc009b30 con 0x7f4904153e60 2023-11-09T03:48:31.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.386+0000 7f490a109640 1 --2- 172.21.15.76:0/127545384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4904153e60 0x7f4904158890 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f490000d800 tx=0x7f490000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:31.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.386+0000 7f48fa7fc640 1 -- 172.21.15.76:0/127545384 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490000b6c0 con 0x7f4904153e60 2023-11-09T03:48:31.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.386+0000 7f48fa7fc640 1 -- 172.21.15.76:0/127545384 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490000b840 con 0x7f4904153e60 2023-11-09T03:48:31.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.386+0000 7f490bb6c640 1 -- 172.21.15.76:0/127545384 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49040cbca0 con 0x7f4904153e60 2023-11-09T03:48:31.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.386+0000 7f48fa7fc640 1 -- 172.21.15.76:0/127545384 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490000bbf0 con 0x7f4904153e60 2023-11-09T03:48:31.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.386+0000 7f490bb6c640 1 -- 172.21.15.76:0/127545384 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490415a8c0 con 0x7f4904153e60 2023-11-09T03:48:31.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.387+0000 7f490bb6c640 1 -- 172.21.15.76:0/127545384 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49040cc4a0 con 0x7f4904153e60 2023-11-09T03:48:31.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.388+0000 7f48fa7fc640 1 -- 172.21.15.76:0/127545384 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f490000bd50 con 0x7f4904153e60 2023-11-09T03:48:31.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.388+0000 7f48fa7fc640 1 --2- 172.21.15.76:0/127545384 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f48e806ae70 0x7f48e806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:31.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.388+0000 7f48fa7fc640 1 -- 172.21.15.76:0/127545384 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f490008a090 con 0x7f4904153e60 2023-11-09T03:48:31.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.388+0000 7f4909908640 1 --2- 172.21.15.76:0/127545384 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f48e806ae70 0x7f48e806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:31.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.389+0000 7f4909908640 1 --2- 172.21.15.76:0/127545384 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f48e806ae70 0x7f48e806d330 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f48fc035000 tx=0x7f48fc030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:31.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.393+0000 7f48fa7fc640 1 -- 172.21.15.76:0/127545384 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f49040cc4a0 con 0x7f4904153e60 2023-11-09T03:48:31.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.627+0000 7f490bb6c640 1 -- 172.21.15.76:0/127545384 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f49040cc4a0 con 0x7f4904153e60 2023-11-09T03:48:31.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.628+0000 7f48fa7fc640 1 -- 172.21.15.76:0/127545384 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7f49040cc4a0 con 0x7f4904153e60 2023-11-09T03:48:31.628 INFO:teuthology.orchestra.run.smithi076.stdout:[client.0] 2023-11-09T03:48:31.628 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQChVUxlHj22CxAACNj5X24hdME+GNTLw4RahQ== 2023-11-09T03:48:31.628 INFO:teuthology.orchestra.run.smithi076.stdout: caps mds = "allow" 2023-11-09T03:48:31.629 INFO:teuthology.orchestra.run.smithi076.stdout: caps mon = "allow r" 2023-11-09T03:48:31.629 INFO:teuthology.orchestra.run.smithi076.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-09T03:48:31.629 INFO:teuthology.orchestra.run.smithi076.stderr:updated caps for client.0 2023-11-09T03:48:31.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.630+0000 7f490bb6c640 1 -- 172.21.15.76:0/127545384 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f48e806ae70 msgr2=0x7f48e806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:31.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.630+0000 7f490bb6c640 1 --2- 172.21.15.76:0/127545384 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f48e806ae70 0x7f48e806d330 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f48fc035000 tx=0x7f48fc030000 comp rx=0 tx=0).stop 2023-11-09T03:48:31.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.630+0000 7f490bb6c640 1 -- 172.21.15.76:0/127545384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4904153e60 msgr2=0x7f4904158890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:31.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.630+0000 7f490bb6c640 1 --2- 172.21.15.76:0/127545384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4904153e60 0x7f4904158890 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f490000d800 tx=0x7f490000dcc0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.631+0000 7f490bb6c640 1 -- 172.21.15.76:0/127545384 shutdown_connections 2023-11-09T03:48:31.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.631+0000 7f490bb6c640 1 --2- 172.21.15.76:0/127545384 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f48e806ae70 0x7f48e806d330 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.631+0000 7f490bb6c640 1 --2- 172.21.15.76:0/127545384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4904153e60 0x7f4904158890 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.631+0000 7f490bb6c640 1 --2- 172.21.15.76:0/127545384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49040cd6d0 0x7f490415a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.631+0000 7f490bb6c640 1 --2- 172.21.15.76:0/127545384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49040b4880 0x7f49040c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.631+0000 7f490bb6c640 1 -- 172.21.15.76:0/127545384 >> 172.21.15.76:0/127545384 conn(0x7f49040cf880 msgr2=0x7f49040b80a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:31.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.631+0000 7f490bb6c640 1 -- 172.21.15.76:0/127545384 shutdown_connections 2023-11-09T03:48:31.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.631+0000 7f490bb6c640 1 -- 172.21.15.76:0/127545384 wait complete. 2023-11-09T03:48:31.641 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:48:31.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.742+0000 7fd37f010640 1 Processor -- start 2023-11-09T03:48:31.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.742+0000 7fd37f010640 1 -- start start 2023-11-09T03:48:31.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.742+0000 7fd37f010640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd378070e90 0x7fd378167d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:31.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.742+0000 7fd37f010640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3780714b0 con 0x7fd37815cfd0 2023-11-09T03:48:31.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.742+0000 7fd37f010640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd378071790 con 0x7fd3780b0ae0 2023-11-09T03:48:31.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.742+0000 7fd37f010640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3780ab8e0 con 0x7fd378070e90 2023-11-09T03:48:31.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.743+0000 7fd37cdac640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd37815cfd0 0x7fd3780b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59088/0 (socket says 172.21.15.76:59088) 2023-11-09T03:48:31.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.743+0000 7fd37cdac640 1 -- 172.21.15.76:0/269298844 learned_addr learned my addr 172.21.15.76:0/269298844 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:31.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.743+0000 7fd3777fe640 1 -- 172.21.15.76:0/269298844 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4223893339 0 0) 0x7fd3780714b0 con 0x7fd37815cfd0 2023-11-09T03:48:31.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.743+0000 7fd37d5ad640 1 --2- 172.21.15.76:0/269298844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd378070e90 0x7fd378167d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:31.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.743+0000 7fd3777fe640 1 -- 172.21.15.76:0/269298844 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd364003520 con 0x7fd37815cfd0 2023-11-09T03:48:31.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.743+0000 7fd3777fe640 1 -- 172.21.15.76:0/269298844 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 17780896 0 0) 0x7fd364003520 con 0x7fd37815cfd0 2023-11-09T03:48:31.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.743+0000 7fd3777fe640 1 -- 172.21.15.76:0/269298844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd378070e90 msgr2=0x7fd378167d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:31.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.743+0000 7fd3777fe640 1 --2- 172.21.15.76:0/269298844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd378070e90 0x7fd378167d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.743+0000 7fd3777fe640 1 -- 172.21.15.76:0/269298844 >> v1:172.21.15.113:6789/0 conn(0x7fd3780b0ae0 legacy=0x7fd378070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:31.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.743+0000 7fd3777fe640 1 -- 172.21.15.76:0/269298844 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37806c690 con 0x7fd37815cfd0 2023-11-09T03:48:31.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.743+0000 7fd3777fe640 1 -- 172.21.15.76:0/269298844 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fd3700019f0 con 0x7fd37815cfd0 2023-11-09T03:48:31.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.743+0000 7fd3777fe640 1 -- 172.21.15.76:0/269298844 >> v1:172.21.15.76:6789/0 conn(0x7fd37815cfd0 legacy=0x7fd3780b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:31.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.743+0000 7fd3777fe640 1 --2- 172.21.15.76:0/269298844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd378070e90 0x7fd3640039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:31.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.743+0000 7fd3777fe640 1 --2- 172.21.15.76:0/269298844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd364003f20 0x7fd3640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:31.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.743+0000 7fd3777fe640 1 --2- 172.21.15.76:0/269298844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd364004d00 0x7fd364005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:31.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.743+0000 7fd3777fe640 1 -- 172.21.15.76:0/269298844 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36400ba30 con 0x7fd364004d00 2023-11-09T03:48:31.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.743+0000 7fd3777fe640 1 -- 172.21.15.76:0/269298844 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36400bba0 con 0x7fd364003f20 2023-11-09T03:48:31.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.743+0000 7fd3777fe640 1 -- 172.21.15.76:0/269298844 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd36400bd60 con 0x7fd378070e90 2023-11-09T03:48:31.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.743+0000 7fd37d5ad640 1 --2- 172.21.15.76:0/269298844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd364004d00 0x7fd364005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:31.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.743+0000 7fd377fff640 1 --2- 172.21.15.76:0/269298844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd364003f20 0x7fd3640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:31.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.743+0000 7fd37cdac640 1 --2- 172.21.15.76:0/269298844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd378070e90 0x7fd3640039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:31.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.744+0000 7fd37d5ad640 1 -- 172.21.15.76:0/269298844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd378070e90 msgr2=0x7fd3640039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:31.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.744+0000 7fd37d5ad640 1 --2- 172.21.15.76:0/269298844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd378070e90 0x7fd3640039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.744+0000 7fd37d5ad640 1 -- 172.21.15.76:0/269298844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd364003f20 msgr2=0x7fd3640047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:31.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.744+0000 7fd37d5ad640 1 --2- 172.21.15.76:0/269298844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd364003f20 0x7fd3640047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.744+0000 7fd37d5ad640 1 -- 172.21.15.76:0/269298844 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd364003520 con 0x7fd364004d00 2023-11-09T03:48:31.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.744+0000 7fd37d5ad640 1 --2- 172.21.15.76:0/269298844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd364004d00 0x7fd364005180 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fd368009ac0 tx=0x7fd3640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77dd261ed0ac654 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:31.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.744+0000 7fd3777fe640 1 -- 172.21.15.76:0/269298844 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36802c6b0 con 0x7fd364004d00 2023-11-09T03:48:31.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.744+0000 7fd3777fe640 1 -- 172.21.15.76:0/269298844 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd36802c810 con 0x7fd364004d00 2023-11-09T03:48:31.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.744+0000 7fd3777fe640 1 -- 172.21.15.76:0/269298844 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36802cba0 con 0x7fd364004d00 2023-11-09T03:48:31.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.745+0000 7fd37f010640 1 -- 172.21.15.76:0/269298844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd364004d00 msgr2=0x7fd364005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:31.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.745+0000 7fd37f010640 1 --2- 172.21.15.76:0/269298844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd364004d00 0x7fd364005180 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fd368009ac0 tx=0x7fd3640057f0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.745+0000 7fd37d5ad640 1 -- 172.21.15.76:0/269298844 reap_dead start 2023-11-09T03:48:31.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.745+0000 7fd37f010640 1 -- 172.21.15.76:0/269298844 shutdown_connections 2023-11-09T03:48:31.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.745+0000 7fd37f010640 1 -- 172.21.15.76:0/269298844 >> 172.21.15.76:0/269298844 conn(0x7fd3780ce950 msgr2=0x7fd3780ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:31.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.745+0000 7fd37f010640 1 -- 172.21.15.76:0/269298844 shutdown_connections 2023-11-09T03:48:31.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.745+0000 7fd37f010640 1 -- 172.21.15.76:0/269298844 wait complete. 2023-11-09T03:48:31.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.746+0000 7fd37f010640 1 Processor -- start 2023-11-09T03:48:31.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.746+0000 7fd37f010640 1 -- start start 2023-11-09T03:48:31.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.747+0000 7fd37f010640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3781f6190 0x7fd3780c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:31.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.747+0000 7fd37f010640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3780c30b0 0x7fd3781f6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:31.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.747+0000 7fd37f010640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3780c69e0 0x7fd3780c4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:31.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.747+0000 7fd37f010640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3780cb380 con 0x7fd3780c30b0 2023-11-09T03:48:31.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.747+0000 7fd37f010640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3780c42e0 con 0x7fd3780c69e0 2023-11-09T03:48:31.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.747+0000 7fd37f010640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3781546f0 con 0x7fd3781f6190 2023-11-09T03:48:31.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.747+0000 7fd377fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3780c30b0 0x7fd3781f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:31.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.747+0000 7fd377fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3780c30b0 0x7fd3781f6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37938/0 (socket says 172.21.15.76:37938) 2023-11-09T03:48:31.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.747+0000 7fd377fff640 1 -- 172.21.15.76:0/448599629 learned_addr learned my addr 172.21.15.76:0/448599629 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:31.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.747+0000 7fd37cdac640 1 --2- 172.21.15.76:0/448599629 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3781f6190 0x7fd3780c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:31.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.747+0000 7fd37d5ad640 1 --2- 172.21.15.76:0/448599629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3780c69e0 0x7fd3780c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:31.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.747+0000 7fd377fff640 1 -- 172.21.15.76:0/448599629 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3781f6190 msgr2=0x7fd3780c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:31.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.747+0000 7fd377fff640 1 --2- 172.21.15.76:0/448599629 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3781f6190 0x7fd3780c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.747+0000 7fd377fff640 1 -- 172.21.15.76:0/448599629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3780c69e0 msgr2=0x7fd3780c4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:31.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.747+0000 7fd377fff640 1 --2- 172.21.15.76:0/448599629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3780c69e0 0x7fd3780c4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.747+0000 7fd377fff640 1 -- 172.21.15.76:0/448599629 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3780c39d0 con 0x7fd3780c30b0 2023-11-09T03:48:31.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.747+0000 7fd37cdac640 1 --2- 172.21.15.76:0/448599629 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3781f6190 0x7fd3780c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:31.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.747+0000 7fd37d5ad640 1 --2- 172.21.15.76:0/448599629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3780c69e0 0x7fd3780c4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:31.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.748+0000 7fd377fff640 1 --2- 172.21.15.76:0/448599629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3780c30b0 0x7fd3781f6570 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fd36000f820 tx=0x7fd36000fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:31.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.748+0000 7fd3757fa640 1 -- 172.21.15.76:0/448599629 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3600050d0 con 0x7fd3780c30b0 2023-11-09T03:48:31.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.748+0000 7fd3757fa640 1 -- 172.21.15.76:0/448599629 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd360005230 con 0x7fd3780c30b0 2023-11-09T03:48:31.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.748+0000 7fd3757fa640 1 -- 172.21.15.76:0/448599629 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36000d730 con 0x7fd3780c30b0 2023-11-09T03:48:31.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.748+0000 7fd37f010640 1 -- 172.21.15.76:0/448599629 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3781f6ab0 con 0x7fd3780c30b0 2023-11-09T03:48:31.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.748+0000 7fd37f010640 1 -- 172.21.15.76:0/448599629 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3780ca1e0 con 0x7fd3780c30b0 2023-11-09T03:48:31.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.749+0000 7fd37f010640 1 -- 172.21.15.76:0/448599629 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd378074290 con 0x7fd3780c30b0 2023-11-09T03:48:31.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.749+0000 7fd3757fa640 1 -- 172.21.15.76:0/448599629 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd36000d890 con 0x7fd3780c30b0 2023-11-09T03:48:31.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.750+0000 7fd3757fa640 1 --2- 172.21.15.76:0/448599629 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd35806ae70 0x7fd35806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:31.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.750+0000 7fd3757fa640 1 -- 172.21.15.76:0/448599629 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7fd360089a50 con 0x7fd3780c30b0 2023-11-09T03:48:31.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.750+0000 7fd37cdac640 1 --2- 172.21.15.76:0/448599629 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd35806ae70 0x7fd35806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:31.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.750+0000 7fd37cdac640 1 --2- 172.21.15.76:0/448599629 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd35806ae70 0x7fd35806d330 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd370004d60 tx=0x7fd370006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:31.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.754+0000 7fd3757fa640 1 -- 172.21.15.76:0/448599629 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd360054620 con 0x7fd3780c30b0 2023-11-09T03:48:31.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.952+0000 7fd37f010640 1 -- 172.21.15.76:0/448599629 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3780c37b0 con 0x7fd3780c30b0 2023-11-09T03:48:31.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.953+0000 7fd3757fa640 1 -- 172.21.15.76:0/448599629 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fd36005e030 con 0x7fd3780c30b0 2023-11-09T03:48:31.953 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:31.953 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":35,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6097,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2408659865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2408659865},{"type":"v1","addr":"172.21.15.113:6836","nonce":2408659865}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6100,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1873078383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1873078383},{"type":"v1","addr":"172.21.15.113:6837","nonce":1873078383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/3549714130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3549714130},{"type":"v1","addr":"172.21.15.76:6834","nonce":3549714130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:48:22.895004+0000","modified":"2023-11-09T03:48:24.884028+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6984":{"gid":6984,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/2165197913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":2165197913},{"type":"v1","addr":"172.21.15.76:6835","nonce":2165197913}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-09T03:48:31.953 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 35 2023-11-09T03:48:31.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.955+0000 7fd37f010640 1 -- 172.21.15.76:0/448599629 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd35806ae70 msgr2=0x7fd35806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:31.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.955+0000 7fd37f010640 1 --2- 172.21.15.76:0/448599629 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd35806ae70 0x7fd35806d330 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd370004d60 tx=0x7fd370006040 comp rx=0 tx=0).stop 2023-11-09T03:48:31.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.955+0000 7fd37f010640 1 -- 172.21.15.76:0/448599629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3780c30b0 msgr2=0x7fd3781f6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:31.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.955+0000 7fd37f010640 1 --2- 172.21.15.76:0/448599629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3780c30b0 0x7fd3781f6570 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fd36000f820 tx=0x7fd36000fce0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.955+0000 7fd37f010640 1 -- 172.21.15.76:0/448599629 shutdown_connections 2023-11-09T03:48:31.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.955+0000 7fd37f010640 1 --2- 172.21.15.76:0/448599629 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd35806ae70 0x7fd35806d330 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.955+0000 7fd37f010640 1 --2- 172.21.15.76:0/448599629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3780c69e0 0x7fd3780c4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.955+0000 7fd37f010640 1 --2- 172.21.15.76:0/448599629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3780c30b0 0x7fd3781f6570 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.955+0000 7fd37f010640 1 --2- 172.21.15.76:0/448599629 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3781f6190 0x7fd3780c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:31.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.955+0000 7fd37f010640 1 -- 172.21.15.76:0/448599629 >> 172.21.15.76:0/448599629 conn(0x7fd3780ce950 msgr2=0x7fd3780aeff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:31.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.955+0000 7fd37f010640 1 -- 172.21.15.76:0/448599629 shutdown_connections 2023-11-09T03:48:31.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:31.955+0000 7fd37f010640 1 -- 172.21.15.76:0/448599629 wait complete. 2023-11-09T03:48:32.368 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-09T03:48:22.895004+0000', 'modified': '2023-11-09T03:48:24.884028+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6984}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6984': {'gid': 6984, 'name': 'c', 'rank': 0, 'incarnation': 33, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6835/2165197913', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6833', 'nonce': 2165197913}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 2165197913}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-09T03:48:32.368 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-09T03:48:32.369 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-09T03:48:32.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.476+0000 7f11765a9640 1 Processor -- start 2023-11-09T03:48:32.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.476+0000 7f11765a9640 1 -- start start 2023-11-09T03:48:32.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.477+0000 7f11765a9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1170153850 0x7f117015a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:32.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.477+0000 7f11765a9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f11700cc520 0x7f1170070fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:32.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.477+0000 7f11765a9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11700c3860 con 0x7f11700cc9d0 2023-11-09T03:48:32.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.477+0000 7f11765a9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11700c9250 con 0x7f11700cc520 2023-11-09T03:48:32.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.477+0000 7f11765a9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1170153c30 con 0x7f1170153850 2023-11-09T03:48:32.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.477+0000 7f116f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f11700cc520 0x7f1170070fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:32.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.477+0000 7f116f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f11700cc520 0x7f1170070fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37942/0 (socket says 172.21.15.76:37942) 2023-11-09T03:48:32.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.477+0000 7f116f7fe640 1 -- 172.21.15.76:0/3074493257 learned_addr learned my addr 172.21.15.76:0/3074493257 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:32.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.477+0000 7f116ffff640 1 --2- 172.21.15.76:0/3074493257 >> v2:172.21.15.113:3300/0 conn(0x7f1170153850 0x7f117015a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:32.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.477+0000 7f116f7fe640 1 -- 172.21.15.76:0/3074493257 >> v2:172.21.15.113:3300/0 conn(0x7f1170153850 msgr2=0x7f117015a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:32.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.477+0000 7f116f7fe640 1 --2- 172.21.15.76:0/3074493257 >> v2:172.21.15.113:3300/0 conn(0x7f1170153850 0x7f117015a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.477+0000 7f116f7fe640 1 -- 172.21.15.76:0/3074493257 >> v1:172.21.15.113:6789/0 conn(0x7f11700cc9d0 legacy=0x7f11700c3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:32.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.477+0000 7f116f7fe640 1 -- 172.21.15.76:0/3074493257 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117015a700 con 0x7f11700cc520 2023-11-09T03:48:32.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.478+0000 7f116f7fe640 1 --2- 172.21.15.76:0/3074493257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11700cc520 0x7f1170070fa0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f11700c5710 tx=0x7f116402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b46f63812822168 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:32.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.478+0000 7f116effd640 1 -- 172.21.15.76:0/3074493257 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164030860 con 0x7f11700cc520 2023-11-09T03:48:32.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.478+0000 7f116effd640 1 -- 172.21.15.76:0/3074493257 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11640309c0 con 0x7f11700cc520 2023-11-09T03:48:32.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.478+0000 7f116effd640 1 -- 172.21.15.76:0/3074493257 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164030d50 con 0x7f11700cc520 2023-11-09T03:48:32.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.478+0000 7f11765a9640 1 -- 172.21.15.76:0/3074493257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11700cc520 msgr2=0x7f1170070fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:32.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.478+0000 7f11765a9640 1 --2- 172.21.15.76:0/3074493257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11700cc520 0x7f1170070fa0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f11700c5710 tx=0x7f116402b9a0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.479+0000 7f11765a9640 1 -- 172.21.15.76:0/3074493257 shutdown_connections 2023-11-09T03:48:32.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.479+0000 7f11765a9640 1 --2- 172.21.15.76:0/3074493257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11700cc520 0x7f1170070fa0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.479+0000 7f11765a9640 1 --2- 172.21.15.76:0/3074493257 >> v2:172.21.15.113:3300/0 conn(0x7f1170153850 0x7f117015a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.479+0000 7f11765a9640 1 -- 172.21.15.76:0/3074493257 >> 172.21.15.76:0/3074493257 conn(0x7f11700ce950 msgr2=0x7f11700ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:32.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.479+0000 7f11765a9640 1 -- 172.21.15.76:0/3074493257 shutdown_connections 2023-11-09T03:48:32.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.479+0000 7f11765a9640 1 -- 172.21.15.76:0/3074493257 wait complete. 2023-11-09T03:48:32.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.480+0000 7f11765a9640 1 Processor -- start 2023-11-09T03:48:32.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.480+0000 7f11765a9640 1 -- start start 2023-11-09T03:48:32.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.481+0000 7f11765a9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11700cc520 0x7f11700ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:32.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.481+0000 7f11765a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11700cc9d0 0x7f1170149400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:32.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.481+0000 7f11765a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1170153850 0x7f1170070610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:32.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.481+0000 7f11765a9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117015b5f0 con 0x7f11700cc520 2023-11-09T03:48:32.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.481+0000 7f11765a9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1170149940 con 0x7f11700cc9d0 2023-11-09T03:48:32.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.481+0000 7f11765a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1170149e00 con 0x7f1170153850 2023-11-09T03:48:32.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.481+0000 7f116ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11700cc520 0x7f11700ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:32.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.481+0000 7f116ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11700cc520 0x7f11700ab8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37954/0 (socket says 172.21.15.76:37954) 2023-11-09T03:48:32.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.481+0000 7f116ffff640 1 -- 172.21.15.76:0/3190922069 learned_addr learned my addr 172.21.15.76:0/3190922069 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:32.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.481+0000 7f116f7fe640 1 --2- 172.21.15.76:0/3190922069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11700cc9d0 0x7f1170149400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:32.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.481+0000 7f1174b46640 1 --2- 172.21.15.76:0/3190922069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1170153850 0x7f1170070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:32.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.481+0000 7f116ffff640 1 -- 172.21.15.76:0/3190922069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1170153850 msgr2=0x7f1170070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:32.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.481+0000 7f116ffff640 1 --2- 172.21.15.76:0/3190922069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1170153850 0x7f1170070610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.481+0000 7f116ffff640 1 -- 172.21.15.76:0/3190922069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11700cc9d0 msgr2=0x7f1170149400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:32.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.481+0000 7f116ffff640 1 --2- 172.21.15.76:0/3190922069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11700cc9d0 0x7f1170149400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.481+0000 7f116ffff640 1 -- 172.21.15.76:0/3190922069 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11640094d0 con 0x7f11700cc520 2023-11-09T03:48:32.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.481+0000 7f116f7fe640 1 --2- 172.21.15.76:0/3190922069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11700cc9d0 0x7f1170149400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:32.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.481+0000 7f116ffff640 1 --2- 172.21.15.76:0/3190922069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11700cc520 0x7f11700ab8d0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f116000e870 tx=0x7f116000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:32.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.481+0000 7f116cff9640 1 -- 172.21.15.76:0/3190922069 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11600094f0 con 0x7f11700cc520 2023-11-09T03:48:32.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.481+0000 7f116cff9640 1 -- 172.21.15.76:0/3190922069 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116000c940 con 0x7f11700cc520 2023-11-09T03:48:32.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.481+0000 7f11765a9640 1 -- 172.21.15.76:0/3190922069 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1170149b20 con 0x7f11700cc520 2023-11-09T03:48:32.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.481+0000 7f116cff9640 1 -- 172.21.15.76:0/3190922069 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116000ccf0 con 0x7f11700cc520 2023-11-09T03:48:32.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.481+0000 7f11765a9640 1 -- 172.21.15.76:0/3190922069 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11700ad5a0 con 0x7f11700cc520 2023-11-09T03:48:32.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.483+0000 7f116cff9640 1 -- 172.21.15.76:0/3190922069 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1160013470 con 0x7f11700cc520 2023-11-09T03:48:32.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.483+0000 7f116cff9640 1 --2- 172.21.15.76:0/3190922069 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f115c06ae70 0x7f115c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:32.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.483+0000 7f116cff9640 1 -- 172.21.15.76:0/3190922069 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f1160089ca0 con 0x7f11700cc520 2023-11-09T03:48:32.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.483+0000 7f116f7fe640 1 --2- 172.21.15.76:0/3190922069 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f115c06ae70 0x7f115c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:32.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.484+0000 7f116f7fe640 1 --2- 172.21.15.76:0/3190922069 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f115c06ae70 0x7f115c06d330 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1164036000 tx=0x7f1164008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:32.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.484+0000 7f1179218740 1 Processor -- start 2023-11-09T03:48:32.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.484+0000 7f1179218740 1 -- start start 2023-11-09T03:48:32.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.485+0000 7f1179218740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3f9b2c850 0x55f3f9aff750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:32.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.485+0000 7f1179218740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3f9b2cc30 0x55f3f9b2f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:32.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.485+0000 7f1179218740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3f9affc90 0x55f3f9b2fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:32.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.485+0000 7f1179218740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3f9a10740 con 0x55f3f9b2cc30 2023-11-09T03:48:32.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.485+0000 7f1179218740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3f9a728d0 con 0x55f3f9b2c850 2023-11-09T03:48:32.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.485+0000 7f1179218740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f3f9a6ffc0 con 0x55f3f9affc90 2023-11-09T03:48:32.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.485+0000 7f116f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3f9b2cc30 0x55f3f9b2f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:32.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.485+0000 7f116f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3f9b2cc30 0x55f3f9b2f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37960/0 (socket says 172.21.15.76:37960) 2023-11-09T03:48:32.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.485+0000 7f116f7fe640 1 -- 172.21.15.76:0/1720783665 learned_addr learned my addr 172.21.15.76:0/1720783665 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:32.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.485+0000 7f1174b46640 1 --2- 172.21.15.76:0/1720783665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3f9affc90 0x55f3f9b2fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:32.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.485+0000 7f116f7fe640 1 -- 172.21.15.76:0/1720783665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3f9affc90 msgr2=0x55f3f9b2fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:32.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.485+0000 7f116ffff640 1 --2- 172.21.15.76:0/1720783665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3f9b2c850 0x55f3f9aff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:32.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.485+0000 7f116f7fe640 1 --2- 172.21.15.76:0/1720783665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3f9affc90 0x55f3f9b2fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.485+0000 7f116f7fe640 1 -- 172.21.15.76:0/1720783665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3f9b2c850 msgr2=0x55f3f9aff750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:32.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.485+0000 7f116f7fe640 1 --2- 172.21.15.76:0/1720783665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3f9b2c850 0x55f3f9aff750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.485+0000 7f116f7fe640 1 -- 172.21.15.76:0/1720783665 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3f9a2eca0 con 0x55f3f9b2cc30 2023-11-09T03:48:32.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.485+0000 7f1174b46640 1 --2- 172.21.15.76:0/1720783665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3f9affc90 0x55f3f9b2fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:32.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.485+0000 7f116ffff640 1 --2- 172.21.15.76:0/1720783665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3f9b2c850 0x55f3f9aff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:32.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.485+0000 7f116f7fe640 1 --2- 172.21.15.76:0/1720783665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3f9b2cc30 0x55f3f9b2f5e0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f1164009930 tx=0x7f1164030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e79994ba8c68c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:32.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.486+0000 7f1155ffb640 1 -- 172.21.15.76:0/1720783665 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164045e10 con 0x55f3f9b2cc30 2023-11-09T03:48:32.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.486+0000 7f1155ffb640 1 -- 172.21.15.76:0/1720783665 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1164039450 con 0x55f3f9b2cc30 2023-11-09T03:48:32.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.486+0000 7f1155ffb640 1 -- 172.21.15.76:0/1720783665 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11640397c0 con 0x55f3f9b2cc30 2023-11-09T03:48:32.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.486+0000 7f1179218740 1 -- 172.21.15.76:0/1720783665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3f9b2cc30 msgr2=0x55f3f9b2f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:32.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.486+0000 7f1179218740 1 --2- 172.21.15.76:0/1720783665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3f9b2cc30 0x55f3f9b2f5e0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f1164009930 tx=0x7f1164030000 comp rx=0 tx=0).stop 2023-11-09T03:48:32.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.486+0000 7f1179218740 1 -- 172.21.15.76:0/1720783665 shutdown_connections 2023-11-09T03:48:32.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.486+0000 7f1179218740 1 --2- 172.21.15.76:0/1720783665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3f9affc90 0x55f3f9b2fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.486+0000 7f1179218740 1 --2- 172.21.15.76:0/1720783665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3f9b2cc30 0x55f3f9b2f5e0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.486+0000 7f1179218740 1 --2- 172.21.15.76:0/1720783665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3f9b2c850 0x55f3f9aff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.486+0000 7f1179218740 1 -- 172.21.15.76:0/1720783665 >> 172.21.15.76:0/1720783665 conn(0x55f3f9b01a60 msgr2=0x55f3f9b4a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:32.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.486+0000 7f1179218740 1 -- 172.21.15.76:0/1720783665 shutdown_connections 2023-11-09T03:48:32.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.486+0000 7f1179218740 1 -- 172.21.15.76:0/1720783665 wait complete. 2023-11-09T03:48:32.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.487+0000 7f1179218740 1 -- start start 2023-11-09T03:48:32.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.487+0000 7f1179218740 1 Processor -- start 2023-11-09T03:48:32.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.488+0000 7f1179218740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3f9b2c850 0x55f3f9b5e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:32.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.488+0000 7f1179218740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3f9b2cc30 0x55f3f9afe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:32.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.488+0000 7f1179218740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3f9b1d670 0x55f3f9b1daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:32.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.488+0000 7f1179218740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3f9a6b8b0 con 0x55f3f9b2cc30 2023-11-09T03:48:32.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.488+0000 7f1179218740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3f9a25c80 con 0x55f3f9b2c850 2023-11-09T03:48:32.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.488+0000 7f1179218740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f3f9a25920 con 0x55f3f9b1d670 2023-11-09T03:48:32.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.488+0000 7f1179218740 10 client.0 _ll_register_callbacks cb 0x55f3f977dac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:48:32.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.488+0000 7f116f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3f9b2cc30 0x55f3f9afe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:32.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.488+0000 7f116f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3f9b2cc30 0x55f3f9afe8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37966/0 (socket says 172.21.15.76:37966) 2023-11-09T03:48:32.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.488+0000 7f116f7fe640 1 -- 172.21.15.76:0/406806944 learned_addr learned my addr 172.21.15.76:0/406806944 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:32.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.488+0000 7f116ffff640 1 --2- 172.21.15.76:0/406806944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3f9b2c850 0x55f3f9b5e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:32.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.488+0000 7f1174b46640 1 --2- 172.21.15.76:0/406806944 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3f9b1d670 0x55f3f9b1daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:32.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.488+0000 7f116f7fe640 1 -- 172.21.15.76:0/406806944 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3f9b1d670 msgr2=0x55f3f9b1daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:32.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.488+0000 7f116f7fe640 1 --2- 172.21.15.76:0/406806944 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3f9b1d670 0x55f3f9b1daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.488+0000 7f116f7fe640 1 -- 172.21.15.76:0/406806944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3f9b2c850 msgr2=0x55f3f9b5e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:32.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.488+0000 7f116f7fe640 1 --2- 172.21.15.76:0/406806944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3f9b2c850 0x55f3f9b5e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.488+0000 7f116f7fe640 1 -- 172.21.15.76:0/406806944 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f117015a700 con 0x55f3f9b2cc30 2023-11-09T03:48:32.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.488+0000 7f1174b46640 1 --2- 172.21.15.76:0/406806944 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3f9b1d670 0x55f3f9b1daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:32.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.488+0000 7f116ffff640 1 --2- 172.21.15.76:0/406806944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3f9b2c850 0x55f3f9b5e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:32.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.488+0000 7f116f7fe640 1 --2- 172.21.15.76:0/406806944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3f9b2cc30 0x55f3f9afe8f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f1164030850 tx=0x7f1164004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7e87c3280e0ee8 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:32.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.488+0000 7f114effd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:48:32.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.488+0000 7f114effd640 1 -- 172.21.15.76:0/406806944 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164039e40 con 0x55f3f9b2cc30 2023-11-09T03:48:32.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.488+0000 7f114effd640 1 -- 172.21.15.76:0/406806944 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f116403e040 con 0x55f3f9b2cc30 2023-11-09T03:48:32.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.488+0000 7f1179218740 1 -- 172.21.15.76:0/406806944 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f3f9a17bf0 con 0x55f3f9b2cc30 2023-11-09T03:48:32.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.489+0000 7f114effd640 1 -- 172.21.15.76:0/406806944 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f116405a460 con 0x55f3f9b2cc30 2023-11-09T03:48:32.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.489+0000 7f1179218740 10 client.7176 fetch_fsmap learned FSMap version 35 2023-11-09T03:48:32.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.489+0000 7f1179218740 1 -- 172.21.15.76:0/406806944 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x55f3f9a69f30 con 0x55f3f9b2cc30 2023-11-09T03:48:32.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.489+0000 7f114effd640 1 -- 172.21.15.76:0/406806944 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f3f9a17bf0 con 0x55f3f9b2cc30 2023-11-09T03:48:32.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.489+0000 7f114effd640 1 -- 172.21.15.76:0/406806944 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1164042050 con 0x55f3f9b2cc30 2023-11-09T03:48:32.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.489+0000 7f1179218740 10 client.7176 fetch_fsmap finished waiting for FSMap version 35 2023-11-09T03:48:32.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.489+0000 7f1179218740 10 client.7176 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-09T03:48:32.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.489+0000 7f1179218740 20 client.7176 populate_metadata read hostname 'smithi076' 2023-11-09T03:48:32.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.489+0000 7f1179218740 1 --2- 172.21.15.76:0/406806944 >> [v2:172.21.15.76:6833/2165197913,v1:172.21.15.76:6835/2165197913] conn(0x55f3f9b42b00 0x55f3f9b42ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:32.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.489+0000 7f1179218740 4 client.7176 mds_command: new command op to 6984 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:48:32.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.489+0000 7f1179218740 1 -- 172.21.15.76:0/406806944 --> [v2:172.21.15.76:6833/2165197913,v1:172.21.15.76:6835/2165197913] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f3f9a64890 con 0x55f3f9b42b00 2023-11-09T03:48:32.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.489+0000 7f116ffff640 1 --2- 172.21.15.76:0/406806944 >> [v2:172.21.15.76:6833/2165197913,v1:172.21.15.76:6835/2165197913] conn(0x55f3f9b42b00 0x55f3f9b42ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:32.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.489+0000 7f114effd640 10 client.7176 ms_handle_connect on v2:172.21.15.76:6833/2165197913 2023-11-09T03:48:32.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.489+0000 7f116ffff640 1 --2- 172.21.15.76:0/406806944 >> [v2:172.21.15.76:6833/2165197913,v1:172.21.15.76:6835/2165197913] conn(0x55f3f9b42b00 0x55f3f9b42ee0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ce6053f0f31ffe server_cookie=64190615bd5a71a2 in_seq=0 out_seq=0 2023-11-09T03:48:32.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.490+0000 7f114effd640 1 -- 172.21.15.76:0/406806944 <== mds.0 v2:172.21.15.76:6833/2165197913 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f3f9a64890 con 0x55f3f9b42b00 2023-11-09T03:48:32.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.490+0000 7f114effd640 10 client.7176 handle_command_reply: tid=0 2023-11-09T03:48:32.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.490+0000 7f1179218740 1 client.7176 shutdown 2023-11-09T03:48:32.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.490+0000 7f1179218740 1 -- 172.21.15.76:0/406806944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3f9b2cc30 msgr2=0x55f3f9afe8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:32.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.490+0000 7f1179218740 1 --2- 172.21.15.76:0/406806944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3f9b2cc30 0x55f3f9afe8f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f1164030850 tx=0x7f1164004730 comp rx=0 tx=0).stop 2023-11-09T03:48:32.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.490+0000 7f1179218740 1 -- 172.21.15.76:0/406806944 shutdown_connections 2023-11-09T03:48:32.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.490+0000 7f1179218740 1 --2- 172.21.15.76:0/406806944 >> [v2:172.21.15.76:6833/2165197913,v1:172.21.15.76:6835/2165197913] conn(0x55f3f9b42b00 0x55f3f9b42ee0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.490+0000 7f1179218740 1 --2- 172.21.15.76:0/406806944 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3f9b1d670 0x55f3f9b1daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.490+0000 7f1179218740 1 --2- 172.21.15.76:0/406806944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3f9b2cc30 0x55f3f9afe8f0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.490+0000 7f1179218740 1 --2- 172.21.15.76:0/406806944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3f9b2c850 0x55f3f9b5e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.490+0000 7f114effd640 0 client.7176 ms_handle_reset on v2:172.21.15.76:6833/2165197913 2023-11-09T03:48:32.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.490+0000 7f1179218740 1 -- 172.21.15.76:0/406806944 >> 172.21.15.76:0/406806944 conn(0x55f3f9affc90 msgr2=0x55f3f9b03600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:32.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.490+0000 7f1179218740 1 -- 172.21.15.76:0/406806944 shutdown_connections 2023-11-09T03:48:32.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.490+0000 7f1179218740 1 -- 172.21.15.76:0/406806944 wait complete. 2023-11-09T03:48:32.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.490+0000 7f1179218740 20 client.7176 trim_cache size 0 max 16384 2023-11-09T03:48:32.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.505+0000 7f1179218740 1 Processor -- start 2023-11-09T03:48:32.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.506+0000 7f1179218740 1 -- start start 2023-11-09T03:48:32.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.506+0000 7f1179218740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3f9b1d670 0x55f3f9b1cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:32.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.506+0000 7f1179218740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3f9b2c850 0x55f3f9b31aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:32.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.506+0000 7f1179218740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3f9b2cc30 0x55f3f9b31fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:32.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.506+0000 7f1179218740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3f99eb280 con 0x55f3f9b2cc30 2023-11-09T03:48:32.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.506+0000 7f1179218740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3f9afb330 con 0x55f3f9b1d670 2023-11-09T03:48:32.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.506+0000 7f1179218740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f3f9b459d0 con 0x55f3f9b2c850 2023-11-09T03:48:32.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.506+0000 7f1174b46640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3f9b2cc30 0x55f3f9b31fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:32.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.506+0000 7f1174b46640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3f9b2cc30 0x55f3f9b31fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37978/0 (socket says 172.21.15.76:37978) 2023-11-09T03:48:32.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.506+0000 7f1174b46640 1 -- 172.21.15.76:0/583257109 learned_addr learned my addr 172.21.15.76:0/583257109 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:32.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.506+0000 7f116ffff640 1 --2- 172.21.15.76:0/583257109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3f9b1d670 0x55f3f9b1cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:32.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.507+0000 7f116f7fe640 1 --2- 172.21.15.76:0/583257109 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3f9b2c850 0x55f3f9b31aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:32.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.507+0000 7f1174b46640 1 -- 172.21.15.76:0/583257109 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3f9b2c850 msgr2=0x55f3f9b31aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:32.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.507+0000 7f1174b46640 1 --2- 172.21.15.76:0/583257109 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3f9b2c850 0x55f3f9b31aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.507+0000 7f1174b46640 1 -- 172.21.15.76:0/583257109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3f9b1d670 msgr2=0x55f3f9b1cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:32.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.507+0000 7f1174b46640 1 --2- 172.21.15.76:0/583257109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3f9b1d670 0x55f3f9b1cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.507+0000 7f1174b46640 1 -- 172.21.15.76:0/583257109 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3f9a07d10 con 0x55f3f9b2cc30 2023-11-09T03:48:32.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.507+0000 7f1174b46640 1 --2- 172.21.15.76:0/583257109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3f9b2cc30 0x55f3f9b31fe0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f1168001c40 tx=0x7f1168002100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4150e0774442a82 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:32.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.507+0000 7f11567fc640 1 -- 172.21.15.76:0/583257109 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168019070 con 0x55f3f9b2cc30 2023-11-09T03:48:32.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.507+0000 7f11567fc640 1 -- 172.21.15.76:0/583257109 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116800d040 con 0x55f3f9b2cc30 2023-11-09T03:48:32.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.507+0000 7f11567fc640 1 -- 172.21.15.76:0/583257109 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168003970 con 0x55f3f9b2cc30 2023-11-09T03:48:32.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.507+0000 7f1179218740 1 -- 172.21.15.76:0/583257109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3f9b2cc30 msgr2=0x55f3f9b31fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:32.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.507+0000 7f1179218740 1 --2- 172.21.15.76:0/583257109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3f9b2cc30 0x55f3f9b31fe0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f1168001c40 tx=0x7f1168002100 comp rx=0 tx=0).stop 2023-11-09T03:48:32.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.507+0000 7f1179218740 1 -- 172.21.15.76:0/583257109 shutdown_connections 2023-11-09T03:48:32.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.507+0000 7f1179218740 1 --2- 172.21.15.76:0/583257109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3f9b2cc30 0x55f3f9b31fe0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.507+0000 7f1179218740 1 --2- 172.21.15.76:0/583257109 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3f9b2c850 0x55f3f9b31aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.507+0000 7f1179218740 1 --2- 172.21.15.76:0/583257109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3f9b1d670 0x55f3f9b1cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.507+0000 7f1179218740 1 -- 172.21.15.76:0/583257109 >> 172.21.15.76:0/583257109 conn(0x55f3f9affc90 msgr2=0x55f3f9b5e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:32.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.507+0000 7f1179218740 1 -- 172.21.15.76:0/583257109 shutdown_connections 2023-11-09T03:48:32.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.507+0000 7f1179218740 1 -- 172.21.15.76:0/583257109 wait complete. 2023-11-09T03:48:32.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.508+0000 7f1179218740 1 -- start start 2023-11-09T03:48:32.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.508+0000 7f1179218740 1 Processor -- start 2023-11-09T03:48:32.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.509+0000 7f1179218740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3f9b1d670 0x55f3f9b31aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:32.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.509+0000 7f1179218740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3f9b2c850 0x55f3f9b31fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:32.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.509+0000 7f1179218740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3f9b2cc30 0x55f3f9bb5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:32.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.509+0000 7f1179218740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3f9bb6a20 con 0x55f3f9b1d670 2023-11-09T03:48:32.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.509+0000 7f1179218740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3f9b4cb30 con 0x55f3f9b2c850 2023-11-09T03:48:32.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.509+0000 7f1179218740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f3f9b4d060 con 0x55f3f9b2cc30 2023-11-09T03:48:32.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.509+0000 7f1179218740 10 client.0 _ll_register_callbacks cb 0x7f1160009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:48:32.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.509+0000 7f116ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3f9b1d670 0x55f3f9b31aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:32.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.509+0000 7f116ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3f9b1d670 0x55f3f9b31aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37988/0 (socket says 172.21.15.76:37988) 2023-11-09T03:48:32.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.509+0000 7f116ffff640 1 -- 172.21.15.76:0/4011571024 learned_addr learned my addr 172.21.15.76:0/4011571024 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:32.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.509+0000 7f1174b46640 1 --2- 172.21.15.76:0/4011571024 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3f9b2cc30 0x55f3f9bb5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:32.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.509+0000 7f116f7fe640 1 --2- 172.21.15.76:0/4011571024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3f9b2c850 0x55f3f9b31fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:32.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.509+0000 7f116ffff640 1 -- 172.21.15.76:0/4011571024 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3f9b2cc30 msgr2=0x55f3f9bb5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:32.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.509+0000 7f116ffff640 1 --2- 172.21.15.76:0/4011571024 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3f9b2cc30 0x55f3f9bb5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.509+0000 7f116ffff640 1 -- 172.21.15.76:0/4011571024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3f9b2c850 msgr2=0x55f3f9b31fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:32.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.509+0000 7f116ffff640 1 --2- 172.21.15.76:0/4011571024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3f9b2c850 0x55f3f9b31fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.509+0000 7f116ffff640 1 -- 172.21.15.76:0/4011571024 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11700c3860 con 0x55f3f9b1d670 2023-11-09T03:48:32.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.509+0000 7f116f7fe640 1 --2- 172.21.15.76:0/4011571024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3f9b2c850 0x55f3f9b31fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:32.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.509+0000 7f116ffff640 1 --2- 172.21.15.76:0/4011571024 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3f9b1d670 0x55f3f9b31aa0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f11600581a0 tx=0x7f116005c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e963e27e93c9f59a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:32.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.509+0000 7f114effd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:48:32.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.509+0000 7f114effd640 1 -- 172.21.15.76:0/4011571024 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11600566a0 con 0x55f3f9b1d670 2023-11-09T03:48:32.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.509+0000 7f114effd640 1 -- 172.21.15.76:0/4011571024 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f1160060310 con 0x55f3f9b1d670 2023-11-09T03:48:32.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.509+0000 7f1179218740 1 -- 172.21.15.76:0/4011571024 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f3f9b05fd0 con 0x55f3f9b1d670 2023-11-09T03:48:32.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.510+0000 7f114effd640 1 -- 172.21.15.76:0/4011571024 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11600604d0 con 0x55f3f9b1d670 2023-11-09T03:48:32.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.510+0000 7f1179218740 10 client.7182 fetch_fsmap learned FSMap version 35 2023-11-09T03:48:32.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.510+0000 7f1179218740 1 -- 172.21.15.76:0/4011571024 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x55f3f9a2d500 con 0x55f3f9b1d670 2023-11-09T03:48:32.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.510+0000 7f114effd640 1 -- 172.21.15.76:0/4011571024 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f3f9b05fd0 con 0x55f3f9b1d670 2023-11-09T03:48:32.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.510+0000 7f114effd640 1 -- 172.21.15.76:0/4011571024 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1160011070 con 0x55f3f9b1d670 2023-11-09T03:48:32.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.510+0000 7f1179218740 10 client.7182 fetch_fsmap finished waiting for FSMap version 35 2023-11-09T03:48:32.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.510+0000 7f1179218740 10 client.7182 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-09T03:48:32.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.510+0000 7f1179218740 20 client.7182 populate_metadata read hostname 'smithi076' 2023-11-09T03:48:32.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.510+0000 7f1179218740 1 --2- 172.21.15.76:0/4011571024 >> [v2:172.21.15.76:6833/2165197913,v1:172.21.15.76:6835/2165197913] conn(0x55f3f9bb5c40 0x55f3f9b4b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:32.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.510+0000 7f1179218740 4 client.7182 mds_command: new command op to 6984 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-09T03:48:32.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.510+0000 7f1179218740 1 -- 172.21.15.76:0/4011571024 --> [v2:172.21.15.76:6833/2165197913,v1:172.21.15.76:6835/2165197913] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f1164039450 con 0x55f3f9bb5c40 2023-11-09T03:48:32.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.510+0000 7f116f7fe640 1 --2- 172.21.15.76:0/4011571024 >> [v2:172.21.15.76:6833/2165197913,v1:172.21.15.76:6835/2165197913] conn(0x55f3f9bb5c40 0x55f3f9b4b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:32.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.510+0000 7f114effd640 10 client.7182 ms_handle_connect on v2:172.21.15.76:6833/2165197913 2023-11-09T03:48:32.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.510+0000 7f116f7fe640 1 --2- 172.21.15.76:0/4011571024 >> [v2:172.21.15.76:6833/2165197913,v1:172.21.15.76:6835/2165197913] conn(0x55f3f9bb5c40 0x55f3f9b4b7e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c392caf4a64d2146 server_cookie=1018b1dbca274fc2 in_seq=0 out_seq=0 2023-11-09T03:48:32.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.510+0000 7f114effd640 1 -- 172.21.15.76:0/4011571024 <== mds.0 v2:172.21.15.76:6833/2165197913 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x7f1164039450 con 0x55f3f9bb5c40 2023-11-09T03:48:32.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.510+0000 7f114effd640 10 client.7182 handle_command_reply: tid=0 2023-11-09T03:48:32.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.510+0000 7f1179218740 1 client.7182 shutdown 2023-11-09T03:48:32.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.511+0000 7f1179218740 1 -- 172.21.15.76:0/4011571024 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3f9b1d670 msgr2=0x55f3f9b31aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:32.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.511+0000 7f1179218740 1 --2- 172.21.15.76:0/4011571024 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3f9b1d670 0x55f3f9b31aa0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f11600581a0 tx=0x7f116005c680 comp rx=0 tx=0).stop 2023-11-09T03:48:32.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.511+0000 7f1179218740 1 -- 172.21.15.76:0/4011571024 shutdown_connections 2023-11-09T03:48:32.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.511+0000 7f1179218740 1 --2- 172.21.15.76:0/4011571024 >> [v2:172.21.15.76:6833/2165197913,v1:172.21.15.76:6835/2165197913] conn(0x55f3f9bb5c40 0x55f3f9b4b7e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.511+0000 7f1179218740 1 --2- 172.21.15.76:0/4011571024 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3f9b2cc30 0x55f3f9bb5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.511+0000 7f1179218740 1 --2- 172.21.15.76:0/4011571024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3f9b2c850 0x55f3f9b31fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.511+0000 7f114effd640 0 client.7182 ms_handle_reset on v2:172.21.15.76:6833/2165197913 2023-11-09T03:48:32.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.511+0000 7f1179218740 1 --2- 172.21.15.76:0/4011571024 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3f9b1d670 0x55f3f9b31aa0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.511+0000 7f1179218740 1 -- 172.21.15.76:0/4011571024 >> 172.21.15.76:0/4011571024 conn(0x55f3f9affc90 msgr2=0x55f3f9b4ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:32.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.511+0000 7f1179218740 1 -- 172.21.15.76:0/4011571024 shutdown_connections 2023-11-09T03:48:32.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.511+0000 7f1179218740 1 -- 172.21.15.76:0/4011571024 wait complete. 2023-11-09T03:48:32.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.511+0000 7f1179218740 20 client.7182 trim_cache size 0 max 16384 2023-11-09T03:48:32.534 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:48:32.534 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "c213b26f-d0da-45d5-8e34-bb75c096d4f2", 2023-11-09T03:48:32.534 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2023-11-09T03:48:32.535 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 6984, 2023-11-09T03:48:32.535 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2023-11-09T03:48:32.535 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2023-11-09T03:48:32.535 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2023-11-09T03:48:32.535 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 8.9600651930000001, 2023-11-09T03:48:32.536 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 35, 2023-11-09T03:48:32.536 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 83, 2023-11-09T03:48:32.536 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 83, 2023-11-09T03:48:32.536 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 15.049761012999999 2023-11-09T03:48:32.536 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:48:32.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.511+0000 7f11765a9640 1 -- 172.21.15.76:0/3190922069 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f115c06ae70 msgr2=0x7f115c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:32.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.511+0000 7f11765a9640 1 --2- 172.21.15.76:0/3190922069 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f115c06ae70 0x7f115c06d330 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1164036000 tx=0x7f1164008030 comp rx=0 tx=0).stop 2023-11-09T03:48:32.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.511+0000 7f11765a9640 1 -- 172.21.15.76:0/3190922069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11700cc520 msgr2=0x7f11700ab8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:32.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.511+0000 7f11765a9640 1 --2- 172.21.15.76:0/3190922069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11700cc520 0x7f11700ab8d0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f116000e870 tx=0x7f116000ed30 comp rx=0 tx=0).stop 2023-11-09T03:48:32.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.511+0000 7f11765a9640 1 -- 172.21.15.76:0/3190922069 shutdown_connections 2023-11-09T03:48:32.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.511+0000 7f11765a9640 1 --2- 172.21.15.76:0/3190922069 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f115c06ae70 0x7f115c06d330 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.511+0000 7f11765a9640 1 --2- 172.21.15.76:0/3190922069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1170153850 0x7f1170070610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.511+0000 7f11765a9640 1 --2- 172.21.15.76:0/3190922069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11700cc9d0 0x7f1170149400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.511+0000 7f11765a9640 1 --2- 172.21.15.76:0/3190922069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11700cc520 0x7f11700ab8d0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.511+0000 7f11765a9640 1 -- 172.21.15.76:0/3190922069 >> 172.21.15.76:0/3190922069 conn(0x7f11700ce950 msgr2=0x7f117014f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:32.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.511+0000 7f11765a9640 1 -- 172.21.15.76:0/3190922069 shutdown_connections 2023-11-09T03:48:32.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.511+0000 7f11765a9640 1 -- 172.21.15.76:0/3190922069 wait complete. 2023-11-09T03:48:32.540 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-09T03:48:32.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.641+0000 7f6a51f4c640 1 Processor -- start 2023-11-09T03:48:32.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.642+0000 7f6a51f4c640 1 -- start start 2023-11-09T03:48:32.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.642+0000 7f6a51f4c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6a4c0cde60 0x7f6a4c15d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:32.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.642+0000 7f6a51f4c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a4c0ce240 0x7f6a4c0d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:32.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.642+0000 7f6a51f4c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a4c15e000 con 0x7f6a4c15daa0 2023-11-09T03:48:32.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.642+0000 7f6a51f4c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a4c15e5c0 con 0x7f6a4c0cde60 2023-11-09T03:48:32.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.642+0000 7f6a51f4c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c15e2e0 con 0x7f6a4c0ce240 2023-11-09T03:48:32.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.643+0000 7f6a4b7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6a4c0cde60 0x7f6a4c15d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:32.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.643+0000 7f6a4bfff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a4c0ce240 0x7f6a4c0d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:32.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.643+0000 7f6a4affd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6a4c15daa0 0x7f6a4c157000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:48308/0 (socket says 172.21.15.76:48308) 2023-11-09T03:48:32.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.643+0000 7f6a4affd640 1 -- 172.21.15.76:0/1323658655 learned_addr learned my addr 172.21.15.76:0/1323658655 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:32.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.643+0000 7f6a4a7fc640 1 -- 172.21.15.76:0/1323658655 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 579060769 0 0) 0x7f6a4c15e000 con 0x7f6a4c15daa0 2023-11-09T03:48:32.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.643+0000 7f6a4a7fc640 1 -- 172.21.15.76:0/1323658655 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a34003520 con 0x7f6a4c15daa0 2023-11-09T03:48:32.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.643+0000 7f6a4b7fe640 1 -- 172.21.15.76:0/1323658655 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a4c0ce240 msgr2=0x7f6a4c0d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:32.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.643+0000 7f6a4b7fe640 1 --2- 172.21.15.76:0/1323658655 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a4c0ce240 0x7f6a4c0d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.643+0000 7f6a4b7fe640 1 -- 172.21.15.76:0/1323658655 >> v1:172.21.15.113:6789/0 conn(0x7f6a4c15daa0 legacy=0x7f6a4c157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:32.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.643+0000 7f6a4b7fe640 1 -- 172.21.15.76:0/1323658655 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a4c0d1120 con 0x7f6a4c0cde60 2023-11-09T03:48:32.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.643+0000 7f6a4bfff640 1 --2- 172.21.15.76:0/1323658655 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a4c0ce240 0x7f6a4c0d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:48:32.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.644+0000 7f6a4b7fe640 1 --2- 172.21.15.76:0/1323658655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4c0cde60 0x7f6a4c15d560 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c0097a0 tx=0x7f6a3c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8211de08bdcad8eb server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:32.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.644+0000 7f6a4a7fc640 1 -- 172.21.15.76:0/1323658655 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c031830 con 0x7f6a4c0cde60 2023-11-09T03:48:32.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.644+0000 7f6a4a7fc640 1 -- 172.21.15.76:0/1323658655 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3c031990 con 0x7f6a4c0cde60 2023-11-09T03:48:32.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.644+0000 7f6a4a7fc640 1 -- 172.21.15.76:0/1323658655 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c031d00 con 0x7f6a4c0cde60 2023-11-09T03:48:32.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.644+0000 7f6a51f4c640 1 -- 172.21.15.76:0/1323658655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4c0cde60 msgr2=0x7f6a4c15d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:32.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.644+0000 7f6a51f4c640 1 --2- 172.21.15.76:0/1323658655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4c0cde60 0x7f6a4c15d560 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c0097a0 tx=0x7f6a3c02b8d0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.645+0000 7f6a51f4c640 1 -- 172.21.15.76:0/1323658655 shutdown_connections 2023-11-09T03:48:32.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.645+0000 7f6a51f4c640 1 --2- 172.21.15.76:0/1323658655 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a4c0ce240 0x7f6a4c0d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.645+0000 7f6a51f4c640 1 --2- 172.21.15.76:0/1323658655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4c0cde60 0x7f6a4c15d560 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.645+0000 7f6a51f4c640 1 -- 172.21.15.76:0/1323658655 >> 172.21.15.76:0/1323658655 conn(0x7f6a4c071f30 msgr2=0x7f6a4c0cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:32.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.645+0000 7f6a51f4c640 1 -- 172.21.15.76:0/1323658655 shutdown_connections 2023-11-09T03:48:32.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.645+0000 7f6a51f4c640 1 -- 172.21.15.76:0/1323658655 wait complete. 2023-11-09T03:48:32.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.646+0000 7f6a51f4c640 1 Processor -- start 2023-11-09T03:48:32.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.646+0000 7f6a51f4c640 1 -- start start 2023-11-09T03:48:32.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.646+0000 7f6a51f4c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a4c0cde60 0x7f6a4c159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:32.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.646+0000 7f6a51f4c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a4c0ce240 0x7f6a4c14bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:32.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.647+0000 7f6a51f4c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4c15daa0 0x7f6a4c14c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:32.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.647+0000 7f6a51f4c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c1f5fe0 con 0x7f6a4c0cde60 2023-11-09T03:48:32.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.647+0000 7f6a51f4c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c14c810 con 0x7f6a4c15daa0 2023-11-09T03:48:32.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.647+0000 7f6a51f4c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c0caf40 con 0x7f6a4c0ce240 2023-11-09T03:48:32.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.647+0000 7f6a4b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a4c0cde60 0x7f6a4c159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:32.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.647+0000 7f6a4b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a4c0cde60 0x7f6a4c159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49726/0 (socket says 172.21.15.76:49726) 2023-11-09T03:48:32.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.647+0000 7f6a4b7fe640 1 -- 172.21.15.76:0/3622958310 learned_addr learned my addr 172.21.15.76:0/3622958310 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:32.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.647+0000 7f6a4affd640 1 --2- 172.21.15.76:0/3622958310 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a4c0ce240 0x7f6a4c14bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:32.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.647+0000 7f6a4bfff640 1 --2- 172.21.15.76:0/3622958310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4c15daa0 0x7f6a4c14c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:32.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.647+0000 7f6a4b7fe640 1 -- 172.21.15.76:0/3622958310 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a4c0ce240 msgr2=0x7f6a4c14bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:32.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.647+0000 7f6a4b7fe640 1 --2- 172.21.15.76:0/3622958310 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a4c0ce240 0x7f6a4c14bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.647+0000 7f6a4b7fe640 1 -- 172.21.15.76:0/3622958310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4c15daa0 msgr2=0x7f6a4c14c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:32.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.647+0000 7f6a4b7fe640 1 --2- 172.21.15.76:0/3622958310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4c15daa0 0x7f6a4c14c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.647+0000 7f6a4b7fe640 1 -- 172.21.15.76:0/3622958310 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a44009a70 con 0x7f6a4c0cde60 2023-11-09T03:48:32.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.647+0000 7f6a4b7fe640 1 --2- 172.21.15.76:0/3622958310 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a4c0cde60 0x7f6a4c159620 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c0098c0 tx=0x7f6a3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:32.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.647+0000 7f6a33fff640 1 -- 172.21.15.76:0/3622958310 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c03a630 con 0x7f6a4c0cde60 2023-11-09T03:48:32.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.647+0000 7f6a33fff640 1 -- 172.21.15.76:0/3622958310 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3c03a790 con 0x7f6a4c0cde60 2023-11-09T03:48:32.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.647+0000 7f6a33fff640 1 -- 172.21.15.76:0/3622958310 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c03ab40 con 0x7f6a4c0cde60 2023-11-09T03:48:32.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.647+0000 7f6a51f4c640 1 -- 172.21.15.76:0/3622958310 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a3c009400 con 0x7f6a4c0cde60 2023-11-09T03:48:32.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.647+0000 7f6a51f4c640 1 -- 172.21.15.76:0/3622958310 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a4c15c9e0 con 0x7f6a4c0cde60 2023-11-09T03:48:32.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.648+0000 7f6a51f4c640 1 -- 172.21.15.76:0/3622958310 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a4c0cb0b0 con 0x7f6a4c0cde60 2023-11-09T03:48:32.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.649+0000 7f6a33fff640 1 -- 172.21.15.76:0/3622958310 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6a3c03aca0 con 0x7f6a4c0cde60 2023-11-09T03:48:32.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.649+0000 7f6a33fff640 1 --2- 172.21.15.76:0/3622958310 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6a3806ae70 0x7f6a3806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:32.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.649+0000 7f6a33fff640 1 -- 172.21.15.76:0/3622958310 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f6a3c0ab490 con 0x7f6a4c0cde60 2023-11-09T03:48:32.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.650+0000 7f6a4affd640 1 --2- 172.21.15.76:0/3622958310 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6a3806ae70 0x7f6a3806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:32.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.650+0000 7f6a4affd640 1 --2- 172.21.15.76:0/3622958310 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6a3806ae70 0x7f6a3806d330 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c0c7470 tx=0x7f6a40007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:32.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.654+0000 7f6a33fff640 1 -- 172.21.15.76:0/3622958310 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6a4c0cb0b0 con 0x7f6a4c0cde60 2023-11-09T03:48:32.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.854+0000 7f6a51f4c640 1 -- 172.21.15.76:0/3622958310 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6a4c0cb0b0 con 0x7f6a4c0cde60 2023-11-09T03:48:32.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.854+0000 7f6a33fff640 1 -- 172.21.15.76:0/3622958310 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6a3c076190 con 0x7f6a4c0cde60 2023-11-09T03:48:32.854 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-09T03:48:32.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.856+0000 7f6a51f4c640 1 -- 172.21.15.76:0/3622958310 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6a3806ae70 msgr2=0x7f6a3806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:32.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.856+0000 7f6a51f4c640 1 --2- 172.21.15.76:0/3622958310 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6a3806ae70 0x7f6a3806d330 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c0c7470 tx=0x7f6a40007d30 comp rx=0 tx=0).stop 2023-11-09T03:48:32.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.856+0000 7f6a51f4c640 1 -- 172.21.15.76:0/3622958310 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a4c0cde60 msgr2=0x7f6a4c159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:32.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.856+0000 7f6a51f4c640 1 --2- 172.21.15.76:0/3622958310 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a4c0cde60 0x7f6a4c159620 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c0098c0 tx=0x7f6a3c0046c0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.856+0000 7f6a51f4c640 1 -- 172.21.15.76:0/3622958310 shutdown_connections 2023-11-09T03:48:32.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.856+0000 7f6a51f4c640 1 --2- 172.21.15.76:0/3622958310 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6a3806ae70 0x7f6a3806d330 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.856+0000 7f6a51f4c640 1 --2- 172.21.15.76:0/3622958310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a4c15daa0 0x7f6a4c14c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.856+0000 7f6a51f4c640 1 --2- 172.21.15.76:0/3622958310 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a4c0ce240 0x7f6a4c14bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.856+0000 7f6a51f4c640 1 --2- 172.21.15.76:0/3622958310 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a4c0cde60 0x7f6a4c159620 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:32.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.856+0000 7f6a51f4c640 1 -- 172.21.15.76:0/3622958310 >> 172.21.15.76:0/3622958310 conn(0x7f6a4c071f30 msgr2=0x7f6a4c0b8060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:32.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.856+0000 7f6a51f4c640 1 -- 172.21.15.76:0/3622958310 shutdown_connections 2023-11-09T03:48:32.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:32.856+0000 7f6a51f4c640 1 -- 172.21.15.76:0/3622958310 wait complete. 2023-11-09T03:48:32.866 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-09T03:48:32.867 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-09T03:48:32.867 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2023-11-09T03:48:32.867 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-09T03:48:32.868 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-09T03:48:32.868 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-09T03:48:32.868 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-09T03:48:32.868 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-09T03:48:32.869 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:48:32.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:48:32.869 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2023-11-09T03:48:32.885 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-09T03:48:32.885 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-09T03:48:32.885 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2023-11-09T03:48:32.885 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:48:32.885 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2023-11-09T03:48:32.886 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:48:32.886 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:48:32.886 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:48:32.886 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2023-11-09T03:48:32.886 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:48:32.886 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:48:32.887 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2023-11-09T03:48:32.887 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:48:32.887 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:48:32.887 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-09T03:48:32.887 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:48:32.887 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-09T03:48:32.888 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42436sec preferred_lft 42436sec 2023-11-09T03:48:32.888 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2023-11-09T03:48:32.888 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:48:32.888 INFO:teuthology.orchestra.run.smithi076.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-09T03:48:32.888 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2e:15:bd:03:a8:2e brd ff:ff:ff:ff:ff:ff 2023-11-09T03:48:32.888 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-09T03:48:32.889 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:48:32.889 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::bcda:38ff:fed4:fdb0/64 scope link 2023-11-09T03:48:32.889 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:48:32.889 INFO:teuthology.orchestra.run.smithi076.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-09T03:48:32.889 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 6a:8d:94:d7:e5:00 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:48:32.890 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::688d:94ff:fed7:e500/64 scope link 2023-11-09T03:48:32.890 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:48:32.890 INFO:teuthology.orchestra.run.smithi076.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-09T03:48:32.890 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2e:15:bd:03:a8:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:48:32.890 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::2c15:bdff:fe03:a82e/64 scope link 2023-11-09T03:48:32.890 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:48:32.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:48:32.891 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-09T03:48:32.951 INFO:teuthology.orchestra.run.smithi076.stdout:1 2023-11-09T03:48:32.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:48:32.952 DEBUG:teuthology.orchestra.run.smithi076:> route 2023-11-09T03:48:33.007 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2023-11-09T03:48:33.008 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-09T03:48:33.008 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-09T03:48:33.008 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-09T03:48:33.008 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-09T03:48:33.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:48:33.009 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-09T03:48:33.009 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-09T03:48:33.010 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-09T03:48:33.010 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-09T03:48:33.011 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-09T03:48:33.011 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-09T03:48:33.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:48:33.103 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2023-11-09T03:48:33.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-09T03:48:33.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-09T03:48:33.118 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:48:33.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:48:33.119 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2023-11-09T03:48:33.184 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-09T03:48:33.185 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:48:33.185 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-09T03:48:33.239 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:48:33.240 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:48:33.240 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-09T03:48:33.294 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:48:33.349 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:48:33.349 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:48:33.349 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:48:33.350 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:48:33.350 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:48:33.350 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:48:33.350 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:48:33.350 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:48:33.351 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:48:33.351 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:48:33.351 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:48:33.351 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:48:33.351 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:48:33.351 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:48:33.352 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:48:33.352 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:48:33.352 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:48:33.352 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:48:33.352 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:48:33.352 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:48:33.353 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:48:33.353 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:48:33.353 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:48:33.353 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:48:33.353 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:48:33.353 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:48:33.354 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:48:33.354 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:48:33.354 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:48:33.354 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:48:33.354 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:48:33.354 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:48:33.355 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:48:33.355 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:48:33.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:48:33.355 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:48:33.403 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-09T03:48:33.403 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-09T03:48:33.445 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:48:33.460 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:48:33.460 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:48:33.461 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:48:33.461 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:48:33.461 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:48:33.461 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:48:33.461 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:48:33.462 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:48:33.462 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:48:33.462 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:48:33.462 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:48:33.462 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:48:33.463 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:48:33.463 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:48:33.463 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:48:33.463 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:48:33.463 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:48:33.463 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:48:33.464 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:48:33.464 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:48:33.464 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:48:33.464 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:48:33.464 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:48:33.465 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:48:33.465 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:48:33.465 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:48:33.465 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:48:33.465 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:48:33.466 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:48:33.466 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:48:33.466 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:48:33.466 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:48:33.467 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:48:33.467 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:48:33.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:48:33.467 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:48:33.536 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:48:33.537+0000 7f9392775480 -1 init, newargv = 0x55b26d01c2f0 newargc=15 2023-11-09T03:48:33.537 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[114461]: starting ceph client 2023-11-09T03:48:33.540 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[114461]: starting fuse 2023-11-09T03:48:34.617 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:48:34.637 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:48:34.637 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:48:34.638 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:48:34.638 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:48:34.638 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:48:34.638 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:48:34.638 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:48:34.639 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:48:34.639 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:48:34.639 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:48:34.639 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:48:34.639 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:48:34.640 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:48:34.640 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:48:34.640 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:48:34.640 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:48:34.640 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:48:34.640 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:48:34.641 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:48:34.641 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:48:34.641 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:48:34.641 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:48:34.641 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:48:34.642 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:48:34.642 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:48:34.642 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:48:34.642 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:48:34.642 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:48:34.642 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:48:34.643 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:48:34.643 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:48:34.643 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:48:34.643 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:48:34.643 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:48:34.644 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-09T03:48:34.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:48:34.644 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:48:34.694 INFO:teuthology.orchestra.run.smithi076.stdout:43 2023-11-09T03:48:34.694 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-09T03:48:34.695 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer -- python3 -c ' 2023-11-09T03:48:34.695 DEBUG:teuthology.orchestra.run.smithi076:> import glob 2023-11-09T03:48:34.696 DEBUG:teuthology.orchestra.run.smithi076:> import re 2023-11-09T03:48:34.696 DEBUG:teuthology.orchestra.run.smithi076:> import os 2023-11-09T03:48:34.696 DEBUG:teuthology.orchestra.run.smithi076:> import subprocess 2023-11-09T03:48:34.697 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:48:34.697 DEBUG:teuthology.orchestra.run.smithi076:> def _find_admin_socket(client_name): 2023-11-09T03:48:34.697 DEBUG:teuthology.orchestra.run.smithi076:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-09T03:48:34.698 DEBUG:teuthology.orchestra.run.smithi076:> files = glob.glob(asok_path) 2023-11-09T03:48:34.698 DEBUG:teuthology.orchestra.run.smithi076:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-09T03:48:34.698 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:48:34.699 DEBUG:teuthology.orchestra.run.smithi076:> # Given a non-glob path, it better be there 2023-11-09T03:48:34.699 DEBUG:teuthology.orchestra.run.smithi076:> if "*" not in asok_path: 2023-11-09T03:48:34.699 DEBUG:teuthology.orchestra.run.smithi076:> assert(len(files) == 1) 2023-11-09T03:48:34.700 DEBUG:teuthology.orchestra.run.smithi076:> return files[0] 2023-11-09T03:48:34.700 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:48:34.700 DEBUG:teuthology.orchestra.run.smithi076:> for f in files: 2023-11-09T03:48:34.701 DEBUG:teuthology.orchestra.run.smithi076:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-09T03:48:34.701 DEBUG:teuthology.orchestra.run.smithi076:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-09T03:48:34.701 DEBUG:teuthology.orchestra.run.smithi076:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-09T03:48:34.702 DEBUG:teuthology.orchestra.run.smithi076:> contents = proc_f.read() 2023-11-09T03:48:34.702 DEBUG:teuthology.orchestra.run.smithi076:> if mountpoint in contents: 2023-11-09T03:48:34.702 DEBUG:teuthology.orchestra.run.smithi076:> return f 2023-11-09T03:48:34.703 DEBUG:teuthology.orchestra.run.smithi076:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-09T03:48:34.703 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:48:34.703 DEBUG:teuthology.orchestra.run.smithi076:> print(_find_admin_socket("client.0")) 2023-11-09T03:48:34.704 DEBUG:teuthology.orchestra.run.smithi076:> ' 2023-11-09T03:48:34.821 INFO:teuthology.orchestra.run.smithi076.stdout:/var/run/ceph/ceph-client.0.114461.asok 2023-11-09T03:48:34.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:48:34.831 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.114461.asok 2023-11-09T03:48:34.832 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:48:34.832 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.114461.asok status 2023-11-09T03:48:34.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:48:35.000 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:48:35.000 INFO:teuthology.orchestra.run.smithi076.stdout: "metadata": { 2023-11-09T03:48:35.000 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_sha1": "7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4", 2023-11-09T03:48:35.000 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev)", 2023-11-09T03:48:35.001 INFO:teuthology.orchestra.run.smithi076.stdout: "entity_id": "0", 2023-11-09T03:48:35.001 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": "smithi076", 2023-11-09T03:48:35.001 INFO:teuthology.orchestra.run.smithi076.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-09T03:48:35.001 INFO:teuthology.orchestra.run.smithi076.stdout: "pid": "114461", 2023-11-09T03:48:35.001 INFO:teuthology.orchestra.run.smithi076.stdout: "root": "/" 2023-11-09T03:48:35.002 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:35.002 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_count": 0, 2023-11-09T03:48:35.002 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_pinned_count": 0, 2023-11-09T03:48:35.002 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 7191, 2023-11-09T03:48:35.002 INFO:teuthology.orchestra.run.smithi076.stdout: "inst": { 2023-11-09T03:48:35.003 INFO:teuthology.orchestra.run.smithi076.stdout: "name": { 2023-11-09T03:48:35.003 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "client", 2023-11-09T03:48:35.003 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 7191 2023-11-09T03:48:35.003 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:35.003 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2023-11-09T03:48:35.004 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2023-11-09T03:48:35.004 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2023-11-09T03:48:35.004 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 1150042836 2023-11-09T03:48:35.004 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:35.004 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:35.005 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2023-11-09T03:48:35.005 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2023-11-09T03:48:35.005 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2023-11-09T03:48:35.005 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 1150042836 2023-11-09T03:48:35.005 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:35.006 INFO:teuthology.orchestra.run.smithi076.stdout: "inst_str": "client.7191 192.168.0.1:0/1150042836", 2023-11-09T03:48:35.006 INFO:teuthology.orchestra.run.smithi076.stdout: "addr_str": "192.168.0.1:0/1150042836", 2023-11-09T03:48:35.006 INFO:teuthology.orchestra.run.smithi076.stdout: "inode_count": 1, 2023-11-09T03:48:35.006 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_epoch": 35, 2023-11-09T03:48:35.006 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch": 83, 2023-11-09T03:48:35.007 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch_barrier": 0, 2023-11-09T03:48:35.007 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklisted": false, 2023-11-09T03:48:35.007 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs" 2023-11-09T03:48:35.007 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:48:35.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:48:35.015 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:48:35.015 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-11-09T03:48:35.015 DEBUG:teuthology.orchestra.run.smithi076:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-09T03:48:35.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:48:35.074 INFO:teuthology.orchestra.run.smithi076.stdout:fuseblk 2023-11-09T03:48:35.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:48:35.081 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-09T03:48:35.081 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:48:35.082 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-09T03:48:35.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:48:35.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:48:35.148 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_max_snaps_per_dir 2023-11-09T03:48:35.259 INFO:teuthology.orchestra.run.smithi076.stdout:{"mds_max_snaps_per_dir":"100"} 2023-11-09T03:48:35.259 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_max_snaps_per_dir": "100" } 2023-11-09T03:48:35.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:48:35.260 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test-allow-snaps) 2023-11-09T03:48:35.281 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps false --yes-i-really-mean-it 2023-11-09T03:48:35.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.425+0000 7fc1864b0640 1 Processor -- start 2023-11-09T03:48:35.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.425+0000 7fc1864b0640 1 -- start start 2023-11-09T03:48:35.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.426+0000 7fc1864b0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc180151ea0 0x7fc18015b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:35.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.426+0000 7fc1864b0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc18015f510 0x7fc1800cb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:35.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.426+0000 7fc1864b0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc18006cf90 con 0x7fc18006dd50 2023-11-09T03:48:35.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.426+0000 7fc1864b0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc18006d270 con 0x7fc180151ea0 2023-11-09T03:48:35.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.426+0000 7fc1864b0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18006d550 con 0x7fc18015f510 2023-11-09T03:48:35.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.426+0000 7fc17ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc180151ea0 0x7fc18015b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:35.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.426+0000 7fc184a4d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc18006dd50 0x7fc180070710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:49282/0 (socket says 172.21.15.76:49282) 2023-11-09T03:48:35.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.426+0000 7fc184a4d640 1 -- 172.21.15.76:0/2466293764 learned_addr learned my addr 172.21.15.76:0/2466293764 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:35.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.427+0000 7fc17ffff640 1 -- 172.21.15.76:0/2466293764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc18015f510 msgr2=0x7fc1800cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:35.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.427+0000 7fc17ffff640 1 --2- 172.21.15.76:0/2466293764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc18015f510 0x7fc1800cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:35.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.427+0000 7fc17ffff640 1 -- 172.21.15.76:0/2466293764 >> v1:172.21.15.76:6789/0 conn(0x7fc18006dd50 legacy=0x7fc180070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:35.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.427+0000 7fc17ffff640 1 -- 172.21.15.76:0/2466293764 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18006d830 con 0x7fc180151ea0 2023-11-09T03:48:35.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.427+0000 7fc17effd640 1 -- 172.21.15.76:0/2466293764 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4083975098 0 0) 0x7fc18006cf90 con 0x7fc18006dd50 2023-11-09T03:48:35.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.427+0000 7fc17ffff640 1 --2- 172.21.15.76:0/2466293764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc180151ea0 0x7fc18015b9c0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc1700097a0 tx=0x7fc17002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27ace132ef6b2e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:35.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.427+0000 7fc17effd640 1 -- 172.21.15.76:0/2466293764 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc170031830 con 0x7fc180151ea0 2023-11-09T03:48:35.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.427+0000 7fc17effd640 1 -- 172.21.15.76:0/2466293764 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc170031990 con 0x7fc180151ea0 2023-11-09T03:48:35.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.427+0000 7fc17effd640 1 -- 172.21.15.76:0/2466293764 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc170031d20 con 0x7fc180151ea0 2023-11-09T03:48:35.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.428+0000 7fc1864b0640 1 -- 172.21.15.76:0/2466293764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc180151ea0 msgr2=0x7fc18015b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:35.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.428+0000 7fc1864b0640 1 --2- 172.21.15.76:0/2466293764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc180151ea0 0x7fc18015b9c0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc1700097a0 tx=0x7fc17002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:48:35.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.428+0000 7fc1864b0640 1 -- 172.21.15.76:0/2466293764 shutdown_connections 2023-11-09T03:48:35.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.428+0000 7fc1864b0640 1 --2- 172.21.15.76:0/2466293764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc18015f510 0x7fc1800cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:35.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.428+0000 7fc1864b0640 1 --2- 172.21.15.76:0/2466293764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc180151ea0 0x7fc18015b9c0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:35.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.428+0000 7fc1864b0640 1 -- 172.21.15.76:0/2466293764 >> 172.21.15.76:0/2466293764 conn(0x7fc1800b05d0 msgr2=0x7fc1800b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:35.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.428+0000 7fc1864b0640 1 -- 172.21.15.76:0/2466293764 shutdown_connections 2023-11-09T03:48:35.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.428+0000 7fc1864b0640 1 -- 172.21.15.76:0/2466293764 wait complete. 2023-11-09T03:48:35.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.429+0000 7fc1864b0640 1 Processor -- start 2023-11-09T03:48:35.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.429+0000 7fc1864b0640 1 -- start start 2023-11-09T03:48:35.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.430+0000 7fc1864b0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc18006dd50 0x7fc1800c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:35.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.430+0000 7fc1864b0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc180151ea0 0x7fc1800b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:35.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.430+0000 7fc1864b0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc18015f510 0x7fc180149bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:35.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.430+0000 7fc1864b0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1800c69e0 con 0x7fc18015f510 2023-11-09T03:48:35.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.430+0000 7fc1864b0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1800b3f10 con 0x7fc18006dd50 2023-11-09T03:48:35.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.430+0000 7fc1864b0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1800c6c10 con 0x7fc180151ea0 2023-11-09T03:48:35.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.430+0000 7fc184a4d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc18015f510 0x7fc180149bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:35.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.430+0000 7fc184a4d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc18015f510 0x7fc180149bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49732/0 (socket says 172.21.15.76:49732) 2023-11-09T03:48:35.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.430+0000 7fc184a4d640 1 -- 172.21.15.76:0/3428128503 learned_addr learned my addr 172.21.15.76:0/3428128503 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:35.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.430+0000 7fc17ffff640 1 --2- 172.21.15.76:0/3428128503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc18006dd50 0x7fc1800c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:35.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.430+0000 7fc17f7fe640 1 --2- 172.21.15.76:0/3428128503 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc180151ea0 0x7fc1800b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:35.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.430+0000 7fc184a4d640 1 -- 172.21.15.76:0/3428128503 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc180151ea0 msgr2=0x7fc1800b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:35.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.430+0000 7fc184a4d640 1 --2- 172.21.15.76:0/3428128503 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc180151ea0 0x7fc1800b39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:35.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.430+0000 7fc184a4d640 1 -- 172.21.15.76:0/3428128503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc18006dd50 msgr2=0x7fc1800c64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:35.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.430+0000 7fc184a4d640 1 --2- 172.21.15.76:0/3428128503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc18006dd50 0x7fc1800c64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:35.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.430+0000 7fc184a4d640 1 -- 172.21.15.76:0/3428128503 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc170009400 con 0x7fc18015f510 2023-11-09T03:48:35.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.431+0000 7fc17ffff640 1 --2- 172.21.15.76:0/3428128503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc18006dd50 0x7fc1800c64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:35.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.431+0000 7fc184a4d640 1 --2- 172.21.15.76:0/3428128503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc18015f510 0x7fc180149bf0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fc17800c7e0 tx=0x7fc17800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:35.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.431+0000 7fc17cff9640 1 -- 172.21.15.76:0/3428128503 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17800a640 con 0x7fc18015f510 2023-11-09T03:48:35.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.431+0000 7fc17cff9640 1 -- 172.21.15.76:0/3428128503 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17800a7c0 con 0x7fc18015f510 2023-11-09T03:48:35.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.431+0000 7fc1864b0640 1 -- 172.21.15.76:0/3428128503 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1800c6ef0 con 0x7fc18015f510 2023-11-09T03:48:35.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.431+0000 7fc17cff9640 1 -- 172.21.15.76:0/3428128503 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17800ab70 con 0x7fc18015f510 2023-11-09T03:48:35.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.431+0000 7fc1864b0640 1 -- 172.21.15.76:0/3428128503 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1800c74b0 con 0x7fc18015f510 2023-11-09T03:48:35.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.432+0000 7fc1864b0640 1 -- 172.21.15.76:0/3428128503 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc18004e490 con 0x7fc18015f510 2023-11-09T03:48:35.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.432+0000 7fc17cff9640 1 -- 172.21.15.76:0/3428128503 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc17800acd0 con 0x7fc18015f510 2023-11-09T03:48:35.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.433+0000 7fc17cff9640 1 --2- 172.21.15.76:0/3428128503 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc16c06ae70 0x7fc16c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:35.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.433+0000 7fc17cff9640 1 -- 172.21.15.76:0/3428128503 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7fc178089420 con 0x7fc18015f510 2023-11-09T03:48:35.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.433+0000 7fc17ffff640 1 --2- 172.21.15.76:0/3428128503 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc16c06ae70 0x7fc16c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:35.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.433+0000 7fc17ffff640 1 --2- 172.21.15.76:0/3428128503 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc16c06ae70 0x7fc16c06d330 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc170036000 tx=0x7fc170008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:35.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.437+0000 7fc17cff9640 1 -- 172.21.15.76:0/3428128503 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc178053f60 con 0x7fc18015f510 2023-11-09T03:48:35.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:35.638+0000 7fc1864b0640 1 -- 172.21.15.76:0/3428128503 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "false", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc1800cc910 con 0x7fc18015f510 2023-11-09T03:48:36.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.607+0000 7fc17cff9640 1 -- 172.21.15.76:0/3428128503 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "false", "yes_i_really_mean_it": true}]=0 disabled new snapshots v36) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc178057580 con 0x7fc18015f510 2023-11-09T03:48:36.607 INFO:teuthology.orchestra.run.smithi076.stderr:disabled new snapshots 2023-11-09T03:48:36.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.610+0000 7fc1864b0640 1 -- 172.21.15.76:0/3428128503 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc16c06ae70 msgr2=0x7fc16c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:36.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.610+0000 7fc1864b0640 1 --2- 172.21.15.76:0/3428128503 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc16c06ae70 0x7fc16c06d330 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc170036000 tx=0x7fc170008030 comp rx=0 tx=0).stop 2023-11-09T03:48:36.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.610+0000 7fc1864b0640 1 -- 172.21.15.76:0/3428128503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc18015f510 msgr2=0x7fc180149bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:36.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.610+0000 7fc1864b0640 1 --2- 172.21.15.76:0/3428128503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc18015f510 0x7fc180149bf0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fc17800c7e0 tx=0x7fc17800cca0 comp rx=0 tx=0).stop 2023-11-09T03:48:36.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.610+0000 7fc1864b0640 1 -- 172.21.15.76:0/3428128503 shutdown_connections 2023-11-09T03:48:36.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.610+0000 7fc1864b0640 1 --2- 172.21.15.76:0/3428128503 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc16c06ae70 0x7fc16c06d330 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:36.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.610+0000 7fc1864b0640 1 --2- 172.21.15.76:0/3428128503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc18015f510 0x7fc180149bf0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:36.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.610+0000 7fc1864b0640 1 --2- 172.21.15.76:0/3428128503 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc180151ea0 0x7fc1800b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:36.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.610+0000 7fc1864b0640 1 --2- 172.21.15.76:0/3428128503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc18006dd50 0x7fc1800c64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:36.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.610+0000 7fc1864b0640 1 -- 172.21.15.76:0/3428128503 >> 172.21.15.76:0/3428128503 conn(0x7fc1800b05d0 msgr2=0x7fc180151470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:36.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.610+0000 7fc1864b0640 1 -- 172.21.15.76:0/3428128503 shutdown_connections 2023-11-09T03:48:36.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.610+0000 7fc1864b0640 1 -- 172.21.15.76:0/3428128503 wait complete. 2023-11-09T03:48:36.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:48:36.622 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test-allow-snaps/.snap/snap00) 2023-11-09T03:48:36.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:48:36.642 INFO:teuthology.orchestra.run.smithi076.stderr:mkdir: cannot create directory ‘test-allow-snaps/.snap/snap00’: Operation not permitted 2023-11-09T03:48:36.643 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2023-11-09T03:48:36.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.790+0000 7feca0f7c640 1 Processor -- start 2023-11-09T03:48:36.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.790+0000 7feca0f7c640 1 -- start start 2023-11-09T03:48:36.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.791+0000 7feca0f7c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fec9c0ade40 0x7fec9c164af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:36.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.791+0000 7feca0f7c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec9c0c4850 con 0x7fec9c159be0 2023-11-09T03:48:36.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.791+0000 7feca0f7c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec9c0c5890 con 0x7fec9c15e420 2023-11-09T03:48:36.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.791+0000 7feca0f7c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec9c0c5670 con 0x7fec9c0ade40 2023-11-09T03:48:36.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.791+0000 7fec9a59c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fec9c159be0 0x7fec9c0c4f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:49290/0 (socket says 172.21.15.76:49290) 2023-11-09T03:48:36.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.791+0000 7fec9a59c640 1 -- 172.21.15.76:0/437757764 learned_addr learned my addr 172.21.15.76:0/437757764 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:36.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.791+0000 7fec9ad9d640 1 --2- 172.21.15.76:0/437757764 >> v2:172.21.15.113:3300/0 conn(0x7fec9c0ade40 0x7fec9c164af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:36.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.791+0000 7fec99d9b640 1 -- 172.21.15.76:0/437757764 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2783244296 0 0) 0x7fec9c0c4850 con 0x7fec9c159be0 2023-11-09T03:48:36.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.791+0000 7fec99d9b640 1 -- 172.21.15.76:0/437757764 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec88003520 con 0x7fec9c159be0 2023-11-09T03:48:36.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.791+0000 7fec99d9b640 1 -- 172.21.15.76:0/437757764 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2671316223 0 0) 0x7fec9c0c5890 con 0x7fec9c15e420 2023-11-09T03:48:36.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.791+0000 7fec99d9b640 1 -- 172.21.15.76:0/437757764 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec9c0c4850 con 0x7fec9c15e420 2023-11-09T03:48:36.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.791+0000 7fec99d9b640 1 -- 172.21.15.76:0/437757764 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2994167011 0 0) 0x7fec88003520 con 0x7fec9c159be0 2023-11-09T03:48:36.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.791+0000 7fec99d9b640 1 -- 172.21.15.76:0/437757764 >> v2:172.21.15.113:3300/0 conn(0x7fec9c0ade40 msgr2=0x7fec9c164af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:36.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.791+0000 7fec99d9b640 1 --2- 172.21.15.76:0/437757764 >> v2:172.21.15.113:3300/0 conn(0x7fec9c0ade40 0x7fec9c164af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:36.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.791+0000 7fec99d9b640 1 -- 172.21.15.76:0/437757764 >> v1:172.21.15.113:6789/0 conn(0x7fec9c15e420 legacy=0x7fec9c0ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:36.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.791+0000 7fec99d9b640 1 -- 172.21.15.76:0/437757764 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9c0cd8e0 con 0x7fec9c159be0 2023-11-09T03:48:36.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.791+0000 7fec99d9b640 1 -- 172.21.15.76:0/437757764 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fec8c0015f0 con 0x7fec9c159be0 2023-11-09T03:48:36.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.791+0000 7fec99d9b640 1 -- 172.21.15.76:0/437757764 >> v1:172.21.15.76:6789/0 conn(0x7fec9c159be0 legacy=0x7fec9c0c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:36.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.791+0000 7fec99d9b640 1 --2- 172.21.15.76:0/437757764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec88003b50 0x7fec88003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:36.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.791+0000 7fec99d9b640 1 --2- 172.21.15.76:0/437757764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec880047c0 0x7fec88004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:36.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.791+0000 7fec99d9b640 1 --2- 172.21.15.76:0/437757764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec880051d0 0x7fec8800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:36.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.792+0000 7fec99d9b640 1 -- 172.21.15.76:0/437757764 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c0c5670 con 0x7fec88003b50 2023-11-09T03:48:36.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.792+0000 7fec99d9b640 1 -- 172.21.15.76:0/437757764 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8800c100 con 0x7fec880051d0 2023-11-09T03:48:36.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.792+0000 7fec99d9b640 1 -- 172.21.15.76:0/437757764 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8800c2c0 con 0x7fec880047c0 2023-11-09T03:48:36.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.792+0000 7fec9a59c640 1 --2- 172.21.15.76:0/437757764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec88003b50 0x7fec88003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:36.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.792+0000 7fec9ad9d640 1 --2- 172.21.15.76:0/437757764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec880051d0 0x7fec8800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:36.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.792+0000 7fec93fff640 1 --2- 172.21.15.76:0/437757764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec880047c0 0x7fec88004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:36.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.792+0000 7fec9a59c640 1 -- 172.21.15.76:0/437757764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec880047c0 msgr2=0x7fec88004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:36.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.792+0000 7fec9a59c640 1 --2- 172.21.15.76:0/437757764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec880047c0 0x7fec88004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:36.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.792+0000 7fec9a59c640 1 -- 172.21.15.76:0/437757764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec880051d0 msgr2=0x7fec8800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:36.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.792+0000 7fec9a59c640 1 --2- 172.21.15.76:0/437757764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec880051d0 0x7fec8800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:36.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.792+0000 7fec9a59c640 1 -- 172.21.15.76:0/437757764 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec88003520 con 0x7fec88003b50 2023-11-09T03:48:36.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.792+0000 7fec9ad9d640 1 --2- 172.21.15.76:0/437757764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec880051d0 0x7fec8800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:36.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.792+0000 7fec9ad9d640 1 -- 172.21.15.76:0/437757764 reap_dead start 2023-11-09T03:48:36.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.793+0000 7fec9a59c640 1 --2- 172.21.15.76:0/437757764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec88003b50 0x7fec88003f50 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fec8c007f40 tx=0x7fec880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13136497f0153128 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:36.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.793+0000 7fec99d9b640 1 -- 172.21.15.76:0/437757764 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c005b90 con 0x7fec88003b50 2023-11-09T03:48:36.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.793+0000 7fec99d9b640 1 -- 172.21.15.76:0/437757764 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8c005d10 con 0x7fec88003b50 2023-11-09T03:48:36.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.793+0000 7fec99d9b640 1 -- 172.21.15.76:0/437757764 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c0072b0 con 0x7fec88003b50 2023-11-09T03:48:36.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.793+0000 7feca0f7c640 1 -- 172.21.15.76:0/437757764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec88003b50 msgr2=0x7fec88003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:36.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.793+0000 7feca0f7c640 1 --2- 172.21.15.76:0/437757764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec88003b50 0x7fec88003f50 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fec8c007f40 tx=0x7fec880057f0 comp rx=0 tx=0).stop 2023-11-09T03:48:36.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.793+0000 7feca0f7c640 1 -- 172.21.15.76:0/437757764 shutdown_connections 2023-11-09T03:48:36.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.793+0000 7feca0f7c640 1 --2- 172.21.15.76:0/437757764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec88003b50 0x7fec88003f50 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:36.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.793+0000 7feca0f7c640 1 -- 172.21.15.76:0/437757764 >> 172.21.15.76:0/437757764 conn(0x7fec9c0b05d0 msgr2=0x7fec9c0b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:36.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.793+0000 7feca0f7c640 1 -- 172.21.15.76:0/437757764 shutdown_connections 2023-11-09T03:48:36.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.793+0000 7feca0f7c640 1 -- 172.21.15.76:0/437757764 wait complete. 2023-11-09T03:48:36.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.794+0000 7feca0f7c640 1 Processor -- start 2023-11-09T03:48:36.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.795+0000 7feca0f7c640 1 -- start start 2023-11-09T03:48:36.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.795+0000 7feca0f7c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec88003b50 0x7fec9c0ae660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:36.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.795+0000 7feca0f7c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec9c0c89d0 0x7fec9c0ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:36.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.795+0000 7feca0f7c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec9c0af430 0x7fec9c0abe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:36.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.795+0000 7feca0f7c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c0c78c0 con 0x7fec9c0c89d0 2023-11-09T03:48:36.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.795+0000 7feca0f7c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c0ad190 con 0x7fec88003b50 2023-11-09T03:48:36.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.795+0000 7feca0f7c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c0ad470 con 0x7fec9c0af430 2023-11-09T03:48:36.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.795+0000 7fec93fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec9c0c89d0 0x7fec9c0ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:36.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.795+0000 7fec93fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec9c0c89d0 0x7fec9c0ab8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49744/0 (socket says 172.21.15.76:49744) 2023-11-09T03:48:36.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.795+0000 7fec93fff640 1 -- 172.21.15.76:0/1716669810 learned_addr learned my addr 172.21.15.76:0/1716669810 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:36.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.796+0000 7fec9a59c640 1 --2- 172.21.15.76:0/1716669810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec88003b50 0x7fec9c0ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:36.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.796+0000 7fec9ad9d640 1 --2- 172.21.15.76:0/1716669810 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec9c0af430 0x7fec9c0abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:36.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.796+0000 7fec93fff640 1 -- 172.21.15.76:0/1716669810 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec9c0af430 msgr2=0x7fec9c0abe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:36.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.796+0000 7fec93fff640 1 --2- 172.21.15.76:0/1716669810 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec9c0af430 0x7fec9c0abe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:36.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.796+0000 7fec93fff640 1 -- 172.21.15.76:0/1716669810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec88003b50 msgr2=0x7fec9c0ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:36.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.796+0000 7fec93fff640 1 --2- 172.21.15.76:0/1716669810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec88003b50 0x7fec9c0ae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:36.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.796+0000 7fec93fff640 1 -- 172.21.15.76:0/1716669810 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9c0ad750 con 0x7fec9c0c89d0 2023-11-09T03:48:36.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.796+0000 7fec9a59c640 1 --2- 172.21.15.76:0/1716669810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec88003b50 0x7fec9c0ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:36.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.796+0000 7fec9ad9d640 1 --2- 172.21.15.76:0/1716669810 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec9c0af430 0x7fec9c0abe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:36.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.796+0000 7fec93fff640 1 --2- 172.21.15.76:0/1716669810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec9c0c89d0 0x7fec9c0ab8d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fec8400e780 tx=0x7fec8400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:36.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.796+0000 7fec92ffd640 1 -- 172.21.15.76:0/1716669810 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec840050d0 con 0x7fec9c0c89d0 2023-11-09T03:48:36.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.796+0000 7fec92ffd640 1 -- 172.21.15.76:0/1716669810 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec84005230 con 0x7fec9c0c89d0 2023-11-09T03:48:36.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.796+0000 7fec92ffd640 1 -- 172.21.15.76:0/1716669810 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8400c6c0 con 0x7fec9c0c89d0 2023-11-09T03:48:36.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.796+0000 7feca0f7c640 1 -- 172.21.15.76:0/1716669810 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec9c0c7b80 con 0x7fec9c0c89d0 2023-11-09T03:48:36.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.796+0000 7feca0f7c640 1 -- 172.21.15.76:0/1716669810 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec9c0aeba0 con 0x7fec9c0c89d0 2023-11-09T03:48:36.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.797+0000 7feca0f7c640 1 -- 172.21.15.76:0/1716669810 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec9c0aed90 con 0x7fec9c0c89d0 2023-11-09T03:48:36.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.797+0000 7fec92ffd640 1 -- 172.21.15.76:0/1716669810 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fec8400c820 con 0x7fec9c0c89d0 2023-11-09T03:48:36.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.798+0000 7fec92ffd640 1 --2- 172.21.15.76:0/1716669810 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fec7c06ae70 0x7fec7c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:36.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.798+0000 7fec92ffd640 1 -- 172.21.15.76:0/1716669810 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7fec84089b00 con 0x7fec9c0c89d0 2023-11-09T03:48:36.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.798+0000 7fec9a59c640 1 --2- 172.21.15.76:0/1716669810 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fec7c06ae70 0x7fec7c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:36.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.798+0000 7fec9a59c640 1 --2- 172.21.15.76:0/1716669810 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fec7c06ae70 0x7fec7c06d330 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fec8c007f10 tx=0x7fec8c009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:36.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:36.802+0000 7fec92ffd640 1 -- 172.21.15.76:0/1716669810 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fec8408be60 con 0x7fec9c0c89d0 2023-11-09T03:48:37.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:37.004+0000 7feca0f7c640 1 -- 172.21.15.76:0/1716669810 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fec9c0c7d70 con 0x7fec9c0c89d0 2023-11-09T03:48:37.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:37.609+0000 7fec92ffd640 1 -- 172.21.15.76:0/1716669810 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v37) v1 ==== 167+0+0 (secure 0 0 0) 0x7fec84054660 con 0x7fec9c0c89d0 2023-11-09T03:48:37.608 INFO:teuthology.orchestra.run.smithi076.stderr:enabled new snapshots 2023-11-09T03:48:37.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:37.611+0000 7feca0f7c640 1 -- 172.21.15.76:0/1716669810 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fec7c06ae70 msgr2=0x7fec7c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:37.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:37.611+0000 7feca0f7c640 1 --2- 172.21.15.76:0/1716669810 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fec7c06ae70 0x7fec7c06d330 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fec8c007f10 tx=0x7fec8c009030 comp rx=0 tx=0).stop 2023-11-09T03:48:37.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:37.611+0000 7feca0f7c640 1 -- 172.21.15.76:0/1716669810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec9c0c89d0 msgr2=0x7fec9c0ab8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:37.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:37.611+0000 7feca0f7c640 1 --2- 172.21.15.76:0/1716669810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec9c0c89d0 0x7fec9c0ab8d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fec8400e780 tx=0x7fec8400ec40 comp rx=0 tx=0).stop 2023-11-09T03:48:37.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:37.611+0000 7feca0f7c640 1 -- 172.21.15.76:0/1716669810 shutdown_connections 2023-11-09T03:48:37.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:37.611+0000 7feca0f7c640 1 --2- 172.21.15.76:0/1716669810 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fec7c06ae70 0x7fec7c06d330 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:37.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:37.611+0000 7feca0f7c640 1 --2- 172.21.15.76:0/1716669810 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec9c0af430 0x7fec9c0abe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:37.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:37.611+0000 7feca0f7c640 1 --2- 172.21.15.76:0/1716669810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec9c0c89d0 0x7fec9c0ab8d0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:37.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:37.611+0000 7feca0f7c640 1 --2- 172.21.15.76:0/1716669810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec88003b50 0x7fec9c0ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:37.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:37.611+0000 7feca0f7c640 1 -- 172.21.15.76:0/1716669810 >> 172.21.15.76:0/1716669810 conn(0x7fec9c0b05d0 msgr2=0x7fec9c06d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:37.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:37.611+0000 7feca0f7c640 1 -- 172.21.15.76:0/1716669810 shutdown_connections 2023-11-09T03:48:37.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:37.611+0000 7feca0f7c640 1 -- 172.21.15.76:0/1716669810 wait complete. 2023-11-09T03:48:37.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:48:37.624 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test-allow-snaps/.snap/snap00) 2023-11-09T03:48:37.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:48:37.646 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir test-allow-snaps/.snap/snap00) 2023-11-09T03:48:37.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:48:37.709 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir test-allow-snaps) 2023-11-09T03:48:37.765 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-09T03:48:37.766 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-09T03:48:37.849 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-09T03:48:37.902 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-09T03:48:37.902 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:48:37.903 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:48:37.918 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:48:38.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.057+0000 7f852470f640 1 Processor -- start 2023-11-09T03:48:38.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.057+0000 7f852470f640 1 -- start start 2023-11-09T03:48:38.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.058+0000 7f852470f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f851c0cde60 0x7f851c15d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:38.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.058+0000 7f852470f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f851c0ce240 0x7f851c0d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:38.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.058+0000 7f852470f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f851c15e000 con 0x7f851c15daa0 2023-11-09T03:48:38.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.058+0000 7f852470f640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f851c15e5c0 con 0x7f851c0cde60 2023-11-09T03:48:38.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.058+0000 7f852470f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851c15e2e0 con 0x7f851c0ce240 2023-11-09T03:48:38.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.059+0000 7f85224ab640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f851c0cde60 0x7f851c15d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:38.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.059+0000 7f8521caa640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f851c15daa0 0x7f851c157000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:48332/0 (socket says 172.21.15.76:48332) 2023-11-09T03:48:38.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.059+0000 7f8521caa640 1 -- 172.21.15.76:0/2632779637 learned_addr learned my addr 172.21.15.76:0/2632779637 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:38.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.059+0000 7f8522cac640 1 --2- 172.21.15.76:0/2632779637 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f851c0ce240 0x7f851c0d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:38.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.059+0000 7f85224ab640 1 -- 172.21.15.76:0/2632779637 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f851c0ce240 msgr2=0x7f851c0d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:38.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.059+0000 7f85224ab640 1 --2- 172.21.15.76:0/2632779637 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f851c0ce240 0x7f851c0d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.059+0000 7f85224ab640 1 -- 172.21.15.76:0/2632779637 >> v1:172.21.15.113:6789/0 conn(0x7f851c15daa0 legacy=0x7f851c157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:38.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.059+0000 7f85224ab640 1 -- 172.21.15.76:0/2632779637 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851c0d1120 con 0x7f851c0cde60 2023-11-09T03:48:38.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.060+0000 7f85224ab640 1 --2- 172.21.15.76:0/2632779637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f851c0cde60 0x7f851c15d560 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8518009d90 tx=0x7f851802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98c8951c4e4b1800 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:38.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.060+0000 7f85214a9640 1 -- 172.21.15.76:0/2632779637 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851802cc50 con 0x7f851c0cde60 2023-11-09T03:48:38.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.060+0000 7f85214a9640 1 -- 172.21.15.76:0/2632779637 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851802cdb0 con 0x7f851c0cde60 2023-11-09T03:48:38.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.060+0000 7f85214a9640 1 -- 172.21.15.76:0/2632779637 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8518034660 con 0x7f851c0cde60 2023-11-09T03:48:38.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.061+0000 7f852470f640 1 -- 172.21.15.76:0/2632779637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f851c0cde60 msgr2=0x7f851c15d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:38.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.061+0000 7f852470f640 1 --2- 172.21.15.76:0/2632779637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f851c0cde60 0x7f851c15d560 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8518009d90 tx=0x7f851802bea0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.061+0000 7f852470f640 1 -- 172.21.15.76:0/2632779637 shutdown_connections 2023-11-09T03:48:38.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.061+0000 7f852470f640 1 --2- 172.21.15.76:0/2632779637 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f851c0ce240 0x7f851c0d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.061+0000 7f852470f640 1 --2- 172.21.15.76:0/2632779637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f851c0cde60 0x7f851c15d560 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.061+0000 7f852470f640 1 -- 172.21.15.76:0/2632779637 >> 172.21.15.76:0/2632779637 conn(0x7f851c071f30 msgr2=0x7f851c0cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:38.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.061+0000 7f852470f640 1 -- 172.21.15.76:0/2632779637 shutdown_connections 2023-11-09T03:48:38.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.061+0000 7f852470f640 1 -- 172.21.15.76:0/2632779637 wait complete. 2023-11-09T03:48:38.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.062+0000 7f852470f640 1 Processor -- start 2023-11-09T03:48:38.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.062+0000 7f852470f640 1 -- start start 2023-11-09T03:48:38.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.063+0000 7f852470f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f851c0cde60 0x7f851c1f5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:38.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.063+0000 7f852470f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f851c0ce240 0x7f851c1f6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:38.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.063+0000 7f852470f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f851c15daa0 0x7f851c0b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:38.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.063+0000 7f852470f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c1f5df0 con 0x7f851c0cde60 2023-11-09T03:48:38.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.063+0000 7f852470f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c1f5550 con 0x7f851c0ce240 2023-11-09T03:48:38.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.063+0000 7f852470f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851c1f5830 con 0x7f851c15daa0 2023-11-09T03:48:38.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.063+0000 7f85224ab640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f851c0cde60 0x7f851c1f5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:38.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.063+0000 7f85224ab640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f851c0cde60 0x7f851c1f5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49758/0 (socket says 172.21.15.76:49758) 2023-11-09T03:48:38.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.063+0000 7f85224ab640 1 -- 172.21.15.76:0/1850027820 learned_addr learned my addr 172.21.15.76:0/1850027820 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:38.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.063+0000 7f8522cac640 1 --2- 172.21.15.76:0/1850027820 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f851c15daa0 0x7f851c0b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:38.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.063+0000 7f8521caa640 1 --2- 172.21.15.76:0/1850027820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f851c0ce240 0x7f851c1f6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:38.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.063+0000 7f85224ab640 1 -- 172.21.15.76:0/1850027820 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f851c15daa0 msgr2=0x7f851c0b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:38.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.063+0000 7f85224ab640 1 --2- 172.21.15.76:0/1850027820 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f851c15daa0 0x7f851c0b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.063+0000 7f85224ab640 1 -- 172.21.15.76:0/1850027820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f851c0ce240 msgr2=0x7f851c1f6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:38.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.063+0000 7f85224ab640 1 --2- 172.21.15.76:0/1850027820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f851c0ce240 0x7f851c1f6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.063+0000 7f85224ab640 1 -- 172.21.15.76:0/1850027820 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8518009a70 con 0x7f851c0cde60 2023-11-09T03:48:38.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.063+0000 7f8522cac640 1 --2- 172.21.15.76:0/1850027820 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f851c15daa0 0x7f851c0b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:38.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.063+0000 7f8521caa640 1 --2- 172.21.15.76:0/1850027820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f851c0ce240 0x7f851c1f6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:38.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.063+0000 7f85224ab640 1 --2- 172.21.15.76:0/1850027820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f851c0cde60 0x7f851c1f5fd0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f8518009d90 tx=0x7f8518004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:38.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.063+0000 7f8512ffd640 1 -- 172.21.15.76:0/1850027820 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8518040650 con 0x7f851c0cde60 2023-11-09T03:48:38.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.064+0000 7f852470f640 1 -- 172.21.15.76:0/1850027820 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f851c1f5b10 con 0x7f851c0cde60 2023-11-09T03:48:38.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.064+0000 7f8512ffd640 1 -- 172.21.15.76:0/1850027820 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85180407b0 con 0x7f851c0cde60 2023-11-09T03:48:38.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.064+0000 7f8512ffd640 1 -- 172.21.15.76:0/1850027820 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8518040b60 con 0x7f851c0cde60 2023-11-09T03:48:38.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.064+0000 7f852470f640 1 -- 172.21.15.76:0/1850027820 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f851c0b3fe0 con 0x7f851c0cde60 2023-11-09T03:48:38.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.064+0000 7f852470f640 1 -- 172.21.15.76:0/1850027820 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f851c1f0620 con 0x7f851c0cde60 2023-11-09T03:48:38.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.065+0000 7f8512ffd640 1 -- 172.21.15.76:0/1850027820 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8518040cc0 con 0x7f851c0cde60 2023-11-09T03:48:38.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.065+0000 7f8512ffd640 1 --2- 172.21.15.76:0/1850027820 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f850006ae70 0x7f850006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:38.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.065+0000 7f8512ffd640 1 -- 172.21.15.76:0/1850027820 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f85180ac9e0 con 0x7f851c0cde60 2023-11-09T03:48:38.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.065+0000 7f8521caa640 1 --2- 172.21.15.76:0/1850027820 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f850006ae70 0x7f850006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:38.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.066+0000 7f8521caa640 1 --2- 172.21.15.76:0/1850027820 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f850006ae70 0x7f850006d330 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f851c15ab40 tx=0x7f850c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:38.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.070+0000 7f8512ffd640 1 -- 172.21.15.76:0/1850027820 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f851c1f0620 con 0x7f851c0cde60 2023-11-09T03:48:38.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.236+0000 7f852470f640 1 -- 172.21.15.76:0/1850027820 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f851c1f0620 con 0x7f851c0cde60 2023-11-09T03:48:38.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.237+0000 7f8512ffd640 1 -- 172.21.15.76:0/1850027820 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+21941 (secure 0 0 0) 0x7f851c1f0620 con 0x7f851c0cde60 2023-11-09T03:48:38.237 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:38.238 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":83,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:48:24.549937+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-09T03:47:59.280601+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-09T03:47:59.742039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:48:20.992533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-11-09T03:48:21.862178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:48:38.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.239+0000 7f852470f640 1 -- 172.21.15.76:0/1850027820 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f850006ae70 msgr2=0x7f850006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:38.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.239+0000 7f852470f640 1 --2- 172.21.15.76:0/1850027820 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f850006ae70 0x7f850006d330 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f851c15ab40 tx=0x7f850c009040 comp rx=0 tx=0).stop 2023-11-09T03:48:38.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.239+0000 7f852470f640 1 -- 172.21.15.76:0/1850027820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f851c0cde60 msgr2=0x7f851c1f5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:38.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.239+0000 7f852470f640 1 --2- 172.21.15.76:0/1850027820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f851c0cde60 0x7f851c1f5fd0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f8518009d90 tx=0x7f8518004a40 comp rx=0 tx=0).stop 2023-11-09T03:48:38.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.239+0000 7f852470f640 1 -- 172.21.15.76:0/1850027820 shutdown_connections 2023-11-09T03:48:38.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.239+0000 7f852470f640 1 --2- 172.21.15.76:0/1850027820 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f850006ae70 0x7f850006d330 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.239+0000 7f852470f640 1 --2- 172.21.15.76:0/1850027820 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f851c15daa0 0x7f851c0b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.239+0000 7f852470f640 1 --2- 172.21.15.76:0/1850027820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f851c0ce240 0x7f851c1f6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.239+0000 7f852470f640 1 --2- 172.21.15.76:0/1850027820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f851c0cde60 0x7f851c1f5fd0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.240+0000 7f852470f640 1 -- 172.21.15.76:0/1850027820 >> 172.21.15.76:0/1850027820 conn(0x7f851c071f30 msgr2=0x7f851c0b60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:38.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.240+0000 7f852470f640 1 -- 172.21.15.76:0/1850027820 shutdown_connections 2023-11-09T03:48:38.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.240+0000 7f852470f640 1 -- 172.21.15.76:0/1850027820 wait complete. 2023-11-09T03:48:38.249 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-09T03:47:00.036540+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_data_pool', 'create_time': '2023-11-09T03:47:59.280601+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 70, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-09T03:47:59.742039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:48:20.992533+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:48:21.862178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-09T03:48:38.250 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:48:38.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.354+0000 7f6a28061640 1 Processor -- start 2023-11-09T03:48:38.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.354+0000 7f6a28061640 1 -- start start 2023-11-09T03:48:38.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.355+0000 7f6a28061640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6a20159c10 0x7f6a2006b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:38.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.355+0000 7f6a28061640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a200ccde0 0x7f6a200b2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:38.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.355+0000 7f6a28061640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a20159ff0 con 0x7f6a20065e60 2023-11-09T03:48:38.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.355+0000 7f6a28061640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a2006a590 con 0x7f6a20159c10 2023-11-09T03:48:38.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.355+0000 7f6a28061640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2006a870 con 0x7f6a200ccde0 2023-11-09T03:48:38.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.355+0000 7f6a25dfd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6a20159c10 0x7f6a2006b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:38.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.355+0000 7f6a255fc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f6a20065e60 0x7f6a200cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:49292/0 (socket says 172.21.15.76:49292) 2023-11-09T03:48:38.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.355+0000 7f6a255fc640 1 -- 172.21.15.76:0/1284774091 learned_addr learned my addr 172.21.15.76:0/1284774091 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:38.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.355+0000 7f6a265fe640 1 --2- 172.21.15.76:0/1284774091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a200ccde0 0x7f6a200b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:38.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.355+0000 7f6a25dfd640 1 -- 172.21.15.76:0/1284774091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a200ccde0 msgr2=0x7f6a200b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:38.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.355+0000 7f6a25dfd640 1 --2- 172.21.15.76:0/1284774091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a200ccde0 0x7f6a200b2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.355+0000 7f6a25dfd640 1 -- 172.21.15.76:0/1284774091 >> v1:172.21.15.76:6789/0 conn(0x7f6a20065e60 legacy=0x7f6a200cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:38.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.355+0000 7f6a24dfb640 1 -- 172.21.15.76:0/1284774091 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 321708017 0 0) 0x7f6a20159ff0 con 0x7f6a20065e60 2023-11-09T03:48:38.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.355+0000 7f6a25dfd640 1 -- 172.21.15.76:0/1284774091 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2006ab50 con 0x7f6a20159c10 2023-11-09T03:48:38.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.355+0000 7f6a265fe640 1 --2- 172.21.15.76:0/1284774091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a200ccde0 0x7f6a200b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:38.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.356+0000 7f6a25dfd640 1 --2- 172.21.15.76:0/1284774091 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a20159c10 0x7f6a2006b630 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0097a0 tx=0x7f6a1c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886dd9f5264bb74e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:38.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.356+0000 7f6a24dfb640 1 -- 172.21.15.76:0/1284774091 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c02e070 con 0x7f6a20159c10 2023-11-09T03:48:38.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.356+0000 7f6a24dfb640 1 -- 172.21.15.76:0/1284774091 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1c0315a0 con 0x7f6a20159c10 2023-11-09T03:48:38.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.356+0000 7f6a24dfb640 1 -- 172.21.15.76:0/1284774091 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c031930 con 0x7f6a20159c10 2023-11-09T03:48:38.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.357+0000 7f6a28061640 1 -- 172.21.15.76:0/1284774091 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a20159c10 msgr2=0x7f6a2006b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:38.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.357+0000 7f6a28061640 1 --2- 172.21.15.76:0/1284774091 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a20159c10 0x7f6a2006b630 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0097a0 tx=0x7f6a1c02b840 comp rx=0 tx=0).stop 2023-11-09T03:48:38.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.357+0000 7f6a28061640 1 -- 172.21.15.76:0/1284774091 shutdown_connections 2023-11-09T03:48:38.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.357+0000 7f6a28061640 1 --2- 172.21.15.76:0/1284774091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a200ccde0 0x7f6a200b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.357+0000 7f6a28061640 1 --2- 172.21.15.76:0/1284774091 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a20159c10 0x7f6a2006b630 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.357+0000 7f6a28061640 1 -- 172.21.15.76:0/1284774091 >> 172.21.15.76:0/1284774091 conn(0x7f6a200cbc90 msgr2=0x7f6a200cc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:38.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.357+0000 7f6a28061640 1 -- 172.21.15.76:0/1284774091 shutdown_connections 2023-11-09T03:48:38.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.357+0000 7f6a28061640 1 -- 172.21.15.76:0/1284774091 wait complete. 2023-11-09T03:48:38.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.358+0000 7f6a28061640 1 Processor -- start 2023-11-09T03:48:38.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.358+0000 7f6a28061640 1 -- start start 2023-11-09T03:48:38.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.359+0000 7f6a28061640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a20065e60 0x7f6a2014d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:38.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.359+0000 7f6a28061640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a200ccde0 0x7f6a200c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:38.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.359+0000 7f6a28061640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a20159c10 0x7f6a201e67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:38.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.359+0000 7f6a28061640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a200c5670 con 0x7f6a20065e60 2023-11-09T03:48:38.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.359+0000 7f6a28061640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a200c5950 con 0x7f6a200ccde0 2023-11-09T03:48:38.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.359+0000 7f6a28061640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a200c5c30 con 0x7f6a20159c10 2023-11-09T03:48:38.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.359+0000 7f6a25dfd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a20065e60 0x7f6a2014d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:38.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.359+0000 7f6a25dfd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a20065e60 0x7f6a2014d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49778/0 (socket says 172.21.15.76:49778) 2023-11-09T03:48:38.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.359+0000 7f6a25dfd640 1 -- 172.21.15.76:0/1478861794 learned_addr learned my addr 172.21.15.76:0/1478861794 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:38.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.359+0000 7f6a255fc640 1 --2- 172.21.15.76:0/1478861794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a200ccde0 0x7f6a200c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:38.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.359+0000 7f6a265fe640 1 --2- 172.21.15.76:0/1478861794 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a20159c10 0x7f6a201e67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:38.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.359+0000 7f6a25dfd640 1 -- 172.21.15.76:0/1478861794 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a20159c10 msgr2=0x7f6a201e67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:38.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.359+0000 7f6a25dfd640 1 --2- 172.21.15.76:0/1478861794 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a20159c10 0x7f6a201e67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.359+0000 7f6a25dfd640 1 -- 172.21.15.76:0/1478861794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a200ccde0 msgr2=0x7f6a200c4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:38.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.359+0000 7f6a25dfd640 1 --2- 172.21.15.76:0/1478861794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a200ccde0 0x7f6a200c4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.359+0000 7f6a25dfd640 1 -- 172.21.15.76:0/1478861794 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1c009400 con 0x7f6a20065e60 2023-11-09T03:48:38.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.359+0000 7f6a255fc640 1 --2- 172.21.15.76:0/1478861794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a200ccde0 0x7f6a200c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:38.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.359+0000 7f6a265fe640 1 --2- 172.21.15.76:0/1478861794 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a20159c10 0x7f6a201e67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:38.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.359+0000 7f6a25dfd640 1 --2- 172.21.15.76:0/1478861794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a20065e60 0x7f6a2014d280 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0097a0 tx=0x7f6a1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:38.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.359+0000 7f6a167fc640 1 -- 172.21.15.76:0/1478861794 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c004230 con 0x7f6a20065e60 2023-11-09T03:48:38.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.359+0000 7f6a167fc640 1 -- 172.21.15.76:0/1478861794 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1c0043b0 con 0x7f6a20065e60 2023-11-09T03:48:38.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.359+0000 7f6a28061640 1 -- 172.21.15.76:0/1478861794 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a20153e20 con 0x7f6a20065e60 2023-11-09T03:48:38.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.359+0000 7f6a167fc640 1 -- 172.21.15.76:0/1478861794 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c004520 con 0x7f6a20065e60 2023-11-09T03:48:38.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.359+0000 7f6a28061640 1 -- 172.21.15.76:0/1478861794 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a2014c3e0 con 0x7f6a20065e60 2023-11-09T03:48:38.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.360+0000 7f6a28061640 1 -- 172.21.15.76:0/1478861794 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a2014c5a0 con 0x7f6a20065e60 2023-11-09T03:48:38.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.361+0000 7f6a167fc640 1 -- 172.21.15.76:0/1478861794 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6a1c032880 con 0x7f6a20065e60 2023-11-09T03:48:38.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.361+0000 7f6a167fc640 1 --2- 172.21.15.76:0/1478861794 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6a0406ae70 0x7f6a0406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:38.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.361+0000 7f6a167fc640 1 -- 172.21.15.76:0/1478861794 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f6a1c0abc80 con 0x7f6a20065e60 2023-11-09T03:48:38.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.361+0000 7f6a255fc640 1 --2- 172.21.15.76:0/1478861794 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6a0406ae70 0x7f6a0406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:38.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.362+0000 7f6a255fc640 1 --2- 172.21.15.76:0/1478861794 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6a0406ae70 0x7f6a0406d330 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6a2015d5e0 tx=0x7f6a10009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:38.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.365+0000 7f6a167fc640 1 -- 172.21.15.76:0/1478861794 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6a1c0767c0 con 0x7f6a20065e60 2023-11-09T03:48:38.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.532+0000 7f6a28061640 1 -- 172.21.15.76:0/1478861794 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6a20074290 con 0x7f6a20065e60 2023-11-09T03:48:38.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.533+0000 7f6a167fc640 1 -- 172.21.15.76:0/1478861794 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6a1c079de0 con 0x7f6a20065e60 2023-11-09T03:48:38.532 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:48:38.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.534+0000 7f6a28061640 1 -- 172.21.15.76:0/1478861794 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6a0406ae70 msgr2=0x7f6a0406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:38.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.534+0000 7f6a28061640 1 --2- 172.21.15.76:0/1478861794 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6a0406ae70 0x7f6a0406d330 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6a2015d5e0 tx=0x7f6a10009040 comp rx=0 tx=0).stop 2023-11-09T03:48:38.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.534+0000 7f6a28061640 1 -- 172.21.15.76:0/1478861794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a20065e60 msgr2=0x7f6a2014d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:38.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.534+0000 7f6a28061640 1 --2- 172.21.15.76:0/1478861794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a20065e60 0x7f6a2014d280 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0097a0 tx=0x7f6a1c0046c0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.535+0000 7f6a28061640 1 -- 172.21.15.76:0/1478861794 shutdown_connections 2023-11-09T03:48:38.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.535+0000 7f6a28061640 1 --2- 172.21.15.76:0/1478861794 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6a0406ae70 0x7f6a0406d330 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.535+0000 7f6a28061640 1 --2- 172.21.15.76:0/1478861794 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6a20159c10 0x7f6a201e67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.535+0000 7f6a28061640 1 --2- 172.21.15.76:0/1478861794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a200ccde0 0x7f6a200c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.535+0000 7f6a28061640 1 --2- 172.21.15.76:0/1478861794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a20065e60 0x7f6a2014d280 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.535+0000 7f6a28061640 1 -- 172.21.15.76:0/1478861794 >> 172.21.15.76:0/1478861794 conn(0x7f6a200cbc90 msgr2=0x7f6a20158850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:38.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.535+0000 7f6a28061640 1 -- 172.21.15.76:0/1478861794 shutdown_connections 2023-11-09T03:48:38.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.535+0000 7f6a28061640 1 -- 172.21.15.76:0/1478861794 wait complete. 2023-11-09T03:48:38.544 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-09T03:48:38.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.649+0000 7fb7346e0640 1 Processor -- start 2023-11-09T03:48:38.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.649+0000 7fb7346e0640 1 -- start start 2023-11-09T03:48:38.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.650+0000 7fb7346e0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb72c153e60 0x7fb72c15dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:38.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.650+0000 7fb7346e0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb72c0b4880 0x7fb72c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:38.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.650+0000 7fb7346e0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb72c0cde60 con 0x7fb72c0cd6d0 2023-11-09T03:48:38.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.650+0000 7fb7346e0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb72c0b5230 con 0x7fb72c0b4880 2023-11-09T03:48:38.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.650+0000 7fb7346e0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c160f90 con 0x7fb72c153e60 2023-11-09T03:48:38.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.650+0000 7fb732c7d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb72c0b4880 0x7fb72c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:38.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.650+0000 7fb732c7d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb72c0b4880 0x7fb72c0b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49788/0 (socket says 172.21.15.76:49788) 2023-11-09T03:48:38.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.650+0000 7fb732c7d640 1 -- 172.21.15.76:0/2615802858 learned_addr learned my addr 172.21.15.76:0/2615802858 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:38.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.651+0000 7fb73247c640 1 --2- 172.21.15.76:0/2615802858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb72c153e60 0x7fb72c15dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:38.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.651+0000 7fb732c7d640 1 -- 172.21.15.76:0/2615802858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb72c153e60 msgr2=0x7fb72c15dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:38.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.651+0000 7fb732c7d640 1 --2- 172.21.15.76:0/2615802858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb72c153e60 0x7fb72c15dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.651+0000 7fb732c7d640 1 -- 172.21.15.76:0/2615802858 >> v1:172.21.15.113:6789/0 conn(0x7fb72c0cd6d0 legacy=0x7fb72c161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:38.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.651+0000 7fb732c7d640 1 -- 172.21.15.76:0/2615802858 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72c15e400 con 0x7fb72c0b4880 2023-11-09T03:48:38.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.651+0000 7fb732c7d640 1 --2- 172.21.15.76:0/2615802858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb72c0b4880 0x7fb72c0b4cf0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fb71c009870 tx=0x7fb71c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c1cd656e22c14e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:38.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.652+0000 7fb73147a640 1 -- 172.21.15.76:0/2615802858 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c030840 con 0x7fb72c0b4880 2023-11-09T03:48:38.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.652+0000 7fb73147a640 1 -- 172.21.15.76:0/2615802858 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71c0309a0 con 0x7fb72c0b4880 2023-11-09T03:48:38.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.652+0000 7fb73147a640 1 -- 172.21.15.76:0/2615802858 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c030d30 con 0x7fb72c0b4880 2023-11-09T03:48:38.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.652+0000 7fb7346e0640 1 -- 172.21.15.76:0/2615802858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb72c0b4880 msgr2=0x7fb72c0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:38.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.652+0000 7fb7346e0640 1 --2- 172.21.15.76:0/2615802858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb72c0b4880 0x7fb72c0b4cf0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fb71c009870 tx=0x7fb71c02b980 comp rx=0 tx=0).stop 2023-11-09T03:48:38.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.652+0000 7fb7346e0640 1 -- 172.21.15.76:0/2615802858 shutdown_connections 2023-11-09T03:48:38.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.652+0000 7fb7346e0640 1 --2- 172.21.15.76:0/2615802858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb72c0b4880 0x7fb72c0b4cf0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.652+0000 7fb7346e0640 1 --2- 172.21.15.76:0/2615802858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb72c153e60 0x7fb72c15dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.652+0000 7fb7346e0640 1 -- 172.21.15.76:0/2615802858 >> 172.21.15.76:0/2615802858 conn(0x7fb72c0cf880 msgr2=0x7fb72c0cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:38.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.652+0000 7fb7346e0640 1 -- 172.21.15.76:0/2615802858 shutdown_connections 2023-11-09T03:48:38.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.652+0000 7fb7346e0640 1 -- 172.21.15.76:0/2615802858 wait complete. 2023-11-09T03:48:38.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.653+0000 7fb7346e0640 1 Processor -- start 2023-11-09T03:48:38.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.653+0000 7fb7346e0640 1 -- start start 2023-11-09T03:48:38.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.654+0000 7fb7346e0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb72c0b4880 0x7fb72c0c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:38.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.654+0000 7fb7346e0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb72c0cd6d0 0x7fb72c150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:38.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.654+0000 7fb7346e0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb72c153e60 0x7fb72c1f5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:38.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.654+0000 7fb7346e0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c150d70 con 0x7fb72c0b4880 2023-11-09T03:48:38.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.654+0000 7fb7346e0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c0c8ef0 con 0x7fb72c0cd6d0 2023-11-09T03:48:38.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.654+0000 7fb7346e0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c0c82e0 con 0x7fb72c153e60 2023-11-09T03:48:38.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.654+0000 7fb73247c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb72c0b4880 0x7fb72c0c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:38.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.654+0000 7fb73247c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb72c0b4880 0x7fb72c0c9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49800/0 (socket says 172.21.15.76:49800) 2023-11-09T03:48:38.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.654+0000 7fb73247c640 1 -- 172.21.15.76:0/61292397 learned_addr learned my addr 172.21.15.76:0/61292397 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:38.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.654+0000 7fb73247c640 1 -- 172.21.15.76:0/61292397 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb72c153e60 msgr2=0x7fb72c1f5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:38.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.654+0000 7fb73247c640 1 --2- 172.21.15.76:0/61292397 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb72c153e60 0x7fb72c1f5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.654+0000 7fb73247c640 1 -- 172.21.15.76:0/61292397 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb72c0cd6d0 msgr2=0x7fb72c150830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:38.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.654+0000 7fb73247c640 1 --2- 172.21.15.76:0/61292397 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb72c0cd6d0 0x7fb72c150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.654+0000 7fb73247c640 1 -- 172.21.15.76:0/61292397 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb71c0094d0 con 0x7fb72c0b4880 2023-11-09T03:48:38.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.654+0000 7fb73247c640 1 --2- 172.21.15.76:0/61292397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb72c0b4880 0x7fb72c0c9380 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fb72400d620 tx=0x7fb72400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:38.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.655+0000 7fb72affd640 1 -- 172.21.15.76:0/61292397 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72400b620 con 0x7fb72c0b4880 2023-11-09T03:48:38.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.655+0000 7fb72affd640 1 -- 172.21.15.76:0/61292397 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72400b780 con 0x7fb72c0b4880 2023-11-09T03:48:38.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.655+0000 7fb72affd640 1 -- 172.21.15.76:0/61292397 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72400bb30 con 0x7fb72c0b4880 2023-11-09T03:48:38.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.655+0000 7fb7346e0640 1 -- 172.21.15.76:0/61292397 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb72c0c85c0 con 0x7fb72c0b4880 2023-11-09T03:48:38.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.655+0000 7fb7346e0640 1 -- 172.21.15.76:0/61292397 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb72c0c8b80 con 0x7fb72c0b4880 2023-11-09T03:48:38.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.656+0000 7fb7346e0640 1 -- 172.21.15.76:0/61292397 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb72c074290 con 0x7fb72c0b4880 2023-11-09T03:48:38.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.656+0000 7fb72affd640 1 -- 172.21.15.76:0/61292397 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb72400bc90 con 0x7fb72c0b4880 2023-11-09T03:48:38.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.656+0000 7fb72affd640 1 --2- 172.21.15.76:0/61292397 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb71806ae70 0x7fb71806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:38.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.656+0000 7fb72affd640 1 -- 172.21.15.76:0/61292397 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7fb72408a030 con 0x7fb72c0b4880 2023-11-09T03:48:38.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.657+0000 7fb731c7b640 1 --2- 172.21.15.76:0/61292397 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb71806ae70 0x7fb71806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:38.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.657+0000 7fb731c7b640 1 --2- 172.21.15.76:0/61292397 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb71806ae70 0x7fb71806d330 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb7200012e0 tx=0x7fb720007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:38.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.660+0000 7fb72affd640 1 -- 172.21.15.76:0/61292397 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb724054b70 con 0x7fb72c0b4880 2023-11-09T03:48:38.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.826+0000 7fb7346e0640 1 -- 172.21.15.76:0/61292397 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7fb72c0b83b0 con 0x7fb72c0b4880 2023-11-09T03:48:38.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.826+0000 7fb72affd640 1 -- 172.21.15.76:0/61292397 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v83) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb724058190 con 0x7fb72c0b4880 2023-11-09T03:48:38.826 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:48:38.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.828+0000 7fb7346e0640 1 -- 172.21.15.76:0/61292397 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb71806ae70 msgr2=0x7fb71806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:38.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.828+0000 7fb7346e0640 1 --2- 172.21.15.76:0/61292397 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb71806ae70 0x7fb71806d330 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb7200012e0 tx=0x7fb720007d30 comp rx=0 tx=0).stop 2023-11-09T03:48:38.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.828+0000 7fb7346e0640 1 -- 172.21.15.76:0/61292397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb72c0b4880 msgr2=0x7fb72c0c9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:38.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.828+0000 7fb7346e0640 1 --2- 172.21.15.76:0/61292397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb72c0b4880 0x7fb72c0c9380 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fb72400d620 tx=0x7fb72400dae0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.828+0000 7fb7346e0640 1 -- 172.21.15.76:0/61292397 shutdown_connections 2023-11-09T03:48:38.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.828+0000 7fb7346e0640 1 --2- 172.21.15.76:0/61292397 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb71806ae70 0x7fb71806d330 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.828+0000 7fb7346e0640 1 --2- 172.21.15.76:0/61292397 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb72c153e60 0x7fb72c1f5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.828+0000 7fb7346e0640 1 --2- 172.21.15.76:0/61292397 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb72c0cd6d0 0x7fb72c150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.828+0000 7fb7346e0640 1 --2- 172.21.15.76:0/61292397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb72c0b4880 0x7fb72c0c9380 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.828+0000 7fb7346e0640 1 -- 172.21.15.76:0/61292397 >> 172.21.15.76:0/61292397 conn(0x7fb72c0cf880 msgr2=0x7fb72c15fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:38.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.828+0000 7fb7346e0640 1 -- 172.21.15.76:0/61292397 shutdown_connections 2023-11-09T03:48:38.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.828+0000 7fb7346e0640 1 -- 172.21.15.76:0/61292397 wait complete. 2023-11-09T03:48:38.838 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-09T03:48:38.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.941+0000 7f743a0af640 1 Processor -- start 2023-11-09T03:48:38.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.941+0000 7f743a0af640 1 -- start start 2023-11-09T03:48:38.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.942+0000 7f743a0af640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f74340c4ff0 0x7f74340cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:38.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.942+0000 7f743a0af640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74340adac0 0x7f74340adf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:38.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.942+0000 7f743a0af640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f743415aff0 con 0x7f7434159c00 2023-11-09T03:48:38.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.942+0000 7f743a0af640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74340ae460 con 0x7f74340c4ff0 2023-11-09T03:48:38.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.942+0000 7f743a0af640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743415a3e0 con 0x7f74340adac0 2023-11-09T03:48:38.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.942+0000 7f74337fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f7434159c00 0x7f74340c48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:49294/0 (socket says 172.21.15.76:49294) 2023-11-09T03:48:38.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.942+0000 7f74337fe640 1 -- 172.21.15.76:0/1314150259 learned_addr learned my addr 172.21.15.76:0/1314150259 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:38.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.942+0000 7f7432ffd640 1 --2- 172.21.15.76:0/1314150259 >> v2:172.21.15.113:3300/0 conn(0x7f74340c4ff0 0x7f74340cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:38.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.942+0000 7f7433fff640 1 --2- 172.21.15.76:0/1314150259 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74340adac0 0x7f74340adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:38.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.942+0000 7f74327fc640 1 -- 172.21.15.76:0/1314150259 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2500231624 0 0) 0x7f743415aff0 con 0x7f7434159c00 2023-11-09T03:48:38.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.942+0000 7f74327fc640 1 -- 172.21.15.76:0/1314150259 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f741c003520 con 0x7f7434159c00 2023-11-09T03:48:38.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.942+0000 7f74327fc640 1 -- 172.21.15.76:0/1314150259 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1685217601 0 0) 0x7f741c003520 con 0x7f7434159c00 2023-11-09T03:48:38.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.942+0000 7f74327fc640 1 -- 172.21.15.76:0/1314150259 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74340adac0 msgr2=0x7f74340adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:38.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.942+0000 7f74327fc640 1 --2- 172.21.15.76:0/1314150259 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74340adac0 0x7f74340adf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.942+0000 7f74327fc640 1 -- 172.21.15.76:0/1314150259 >> v2:172.21.15.113:3300/0 conn(0x7f74340c4ff0 msgr2=0x7f74340cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:38.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.942+0000 7f74327fc640 1 --2- 172.21.15.76:0/1314150259 >> v2:172.21.15.113:3300/0 conn(0x7f74340c4ff0 0x7f74340cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.942+0000 7f74327fc640 1 -- 172.21.15.76:0/1314150259 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743415a6c0 con 0x7f7434159c00 2023-11-09T03:48:38.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.942+0000 7f74327fc640 1 -- 172.21.15.76:0/1314150259 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f7424001670 con 0x7f7434159c00 2023-11-09T03:48:38.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.942+0000 7f74327fc640 1 -- 172.21.15.76:0/1314150259 >> v1:172.21.15.76:6789/0 conn(0x7f7434159c00 legacy=0x7f74340c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:38.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.942+0000 7f74327fc640 1 --2- 172.21.15.76:0/1314150259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f741c003970 0x7f741c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:38.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.942+0000 7f74327fc640 1 --2- 172.21.15.76:0/1314150259 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74340adac0 0x7f741c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:38.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.942+0000 7f74327fc640 1 --2- 172.21.15.76:0/1314150259 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f741c004d00 0x7f741c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:38.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.942+0000 7f74327fc640 1 -- 172.21.15.76:0/1314150259 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74340ae460 con 0x7f741c004d00 2023-11-09T03:48:38.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.942+0000 7f74327fc640 1 -- 172.21.15.76:0/1314150259 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743415a3e0 con 0x7f741c003970 2023-11-09T03:48:38.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.942+0000 7f74327fc640 1 -- 172.21.15.76:0/1314150259 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f741c00b980 con 0x7f74340adac0 2023-11-09T03:48:38.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.942+0000 7f7432ffd640 1 --2- 172.21.15.76:0/1314150259 >> v2:172.21.15.113:3300/0 conn(0x7f74340c4ff0 0x7f74340cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:48:38.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.942+0000 7f7433fff640 1 --2- 172.21.15.76:0/1314150259 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f741c004d00 0x7f741c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:38.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.942+0000 7f74337fe640 1 --2- 172.21.15.76:0/1314150259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f741c003970 0x7f741c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:38.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.943+0000 7f7432ffd640 1 --2- 172.21.15.76:0/1314150259 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74340adac0 0x7f741c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:38.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.943+0000 7f7433fff640 1 -- 172.21.15.76:0/1314150259 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74340adac0 msgr2=0x7f741c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:38.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.943+0000 7f7433fff640 1 --2- 172.21.15.76:0/1314150259 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74340adac0 0x7f741c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.943+0000 7f7433fff640 1 -- 172.21.15.76:0/1314150259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f741c003970 msgr2=0x7f741c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:38.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.943+0000 7f7433fff640 1 --2- 172.21.15.76:0/1314150259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f741c003970 0x7f741c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.943+0000 7f7433fff640 1 -- 172.21.15.76:0/1314150259 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f741c003520 con 0x7f741c004d00 2023-11-09T03:48:38.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.943+0000 7f7432ffd640 1 --2- 172.21.15.76:0/1314150259 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74340adac0 0x7f741c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:38.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.943+0000 7f7433fff640 1 --2- 172.21.15.76:0/1314150259 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f741c004d00 0x7f741c005180 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f742c00e170 tx=0x7f741c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf5349e18d226ed server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:38.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.944+0000 7f74327fc640 1 -- 172.21.15.76:0/1314150259 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c02cbf0 con 0x7f741c004d00 2023-11-09T03:48:38.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.944+0000 7f74327fc640 1 -- 172.21.15.76:0/1314150259 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742c00f040 con 0x7f741c004d00 2023-11-09T03:48:38.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.944+0000 7f74327fc640 1 -- 172.21.15.76:0/1314150259 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c034440 con 0x7f741c004d00 2023-11-09T03:48:38.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.944+0000 7f743a0af640 1 -- 172.21.15.76:0/1314150259 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f741c004d00 msgr2=0x7f741c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:38.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.944+0000 7f743a0af640 1 --2- 172.21.15.76:0/1314150259 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f741c004d00 0x7f741c005180 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f742c00e170 tx=0x7f741c0057f0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.944+0000 7f7433fff640 1 -- 172.21.15.76:0/1314150259 reap_dead start 2023-11-09T03:48:38.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.944+0000 7f743a0af640 1 -- 172.21.15.76:0/1314150259 shutdown_connections 2023-11-09T03:48:38.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.944+0000 7f743a0af640 1 -- 172.21.15.76:0/1314150259 >> 172.21.15.76:0/1314150259 conn(0x7f74340ce950 msgr2=0x7f74340ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:38.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.944+0000 7f743a0af640 1 -- 172.21.15.76:0/1314150259 shutdown_connections 2023-11-09T03:48:38.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.944+0000 7f743a0af640 1 -- 172.21.15.76:0/1314150259 wait complete. 2023-11-09T03:48:38.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.945+0000 7f743a0af640 1 Processor -- start 2023-11-09T03:48:38.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.945+0000 7f743a0af640 1 -- start start 2023-11-09T03:48:38.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.946+0000 7f743a0af640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74340c7090 0x7f7434145df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:38.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.946+0000 7f743a0af640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f743414d4b0 0x7f743414d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:38.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.946+0000 7f743a0af640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74340c76f0 0x7f74340c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:38.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.946+0000 7f743a0af640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7434146e90 con 0x7f74340c7090 2023-11-09T03:48:38.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.946+0000 7f743a0af640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74340c8620 con 0x7f743414d4b0 2023-11-09T03:48:38.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.946+0000 7f743a0af640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743414de70 con 0x7f74340c76f0 2023-11-09T03:48:38.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.946+0000 7f74337fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74340c7090 0x7f7434145df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:38.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.946+0000 7f74337fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74340c7090 0x7f7434145df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49820/0 (socket says 172.21.15.76:49820) 2023-11-09T03:48:38.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.946+0000 7f74337fe640 1 -- 172.21.15.76:0/89146723 learned_addr learned my addr 172.21.15.76:0/89146723 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:38.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.946+0000 7f74337fe640 1 -- 172.21.15.76:0/89146723 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74340c76f0 msgr2=0x7f74340c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:38.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.946+0000 7f74337fe640 1 --2- 172.21.15.76:0/89146723 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74340c76f0 0x7f74340c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.946+0000 7f74337fe640 1 -- 172.21.15.76:0/89146723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f743414d4b0 msgr2=0x7f743414d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:38.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.946+0000 7f74337fe640 1 --2- 172.21.15.76:0/89146723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f743414d4b0 0x7f743414d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:38.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.946+0000 7f74337fe640 1 -- 172.21.15.76:0/89146723 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7434145960 con 0x7f74340c7090 2023-11-09T03:48:38.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.947+0000 7f74337fe640 1 --2- 172.21.15.76:0/89146723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74340c7090 0x7f7434145df0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f7424007290 tx=0x7f7424007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:38.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.947+0000 7f741bfff640 1 -- 172.21.15.76:0/89146723 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742400f9a0 con 0x7f74340c7090 2023-11-09T03:48:38.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.947+0000 7f741bfff640 1 -- 172.21.15.76:0/89146723 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742400fb20 con 0x7f74340c7090 2023-11-09T03:48:38.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.947+0000 7f743a0af640 1 -- 172.21.15.76:0/89146723 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74340c80b0 con 0x7f74340c7090 2023-11-09T03:48:38.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.947+0000 7f741bfff640 1 -- 172.21.15.76:0/89146723 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742400fed0 con 0x7f74340c7090 2023-11-09T03:48:38.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.947+0000 7f743a0af640 1 -- 172.21.15.76:0/89146723 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7434145310 con 0x7f74340c7090 2023-11-09T03:48:38.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.948+0000 7f743a0af640 1 -- 172.21.15.76:0/89146723 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7434074290 con 0x7f74340c7090 2023-11-09T03:48:38.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.948+0000 7f741bfff640 1 -- 172.21.15.76:0/89146723 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f742401d460 con 0x7f74340c7090 2023-11-09T03:48:38.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.948+0000 7f741bfff640 1 --2- 172.21.15.76:0/89146723 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f742006ae70 0x7f742006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:38.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.949+0000 7f741bfff640 1 -- 172.21.15.76:0/89146723 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f7424089db0 con 0x7f74340c7090 2023-11-09T03:48:38.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.949+0000 7f7432ffd640 1 --2- 172.21.15.76:0/89146723 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f742006ae70 0x7f742006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:38.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.949+0000 7f7432ffd640 1 --2- 172.21.15.76:0/89146723 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f742006ae70 0x7f742006d330 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f74340caed0 tx=0x7f7428008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:38.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:38.953+0000 7f741bfff640 1 -- 172.21.15.76:0/89146723 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7424054980 con 0x7f74340c7090 2023-11-09T03:48:39.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.118+0000 7f743a0af640 1 -- 172.21.15.76:0/89146723 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7f74340cf220 con 0x7f74340c7090 2023-11-09T03:48:39.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.119+0000 7f741bfff640 1 -- 172.21.15.76:0/89146723 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v83) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7424057fa0 con 0x7f74340c7090 2023-11-09T03:48:39.119 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:48:39.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.121+0000 7f743a0af640 1 -- 172.21.15.76:0/89146723 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f742006ae70 msgr2=0x7f742006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:39.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.121+0000 7f743a0af640 1 --2- 172.21.15.76:0/89146723 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f742006ae70 0x7f742006d330 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f74340caed0 tx=0x7f7428008040 comp rx=0 tx=0).stop 2023-11-09T03:48:39.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.121+0000 7f743a0af640 1 -- 172.21.15.76:0/89146723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74340c7090 msgr2=0x7f7434145df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:39.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.121+0000 7f743a0af640 1 --2- 172.21.15.76:0/89146723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74340c7090 0x7f7434145df0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f7424007290 tx=0x7f7424007750 comp rx=0 tx=0).stop 2023-11-09T03:48:39.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.121+0000 7f743a0af640 1 -- 172.21.15.76:0/89146723 shutdown_connections 2023-11-09T03:48:39.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.121+0000 7f743a0af640 1 --2- 172.21.15.76:0/89146723 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f742006ae70 0x7f742006d330 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.121+0000 7f743a0af640 1 --2- 172.21.15.76:0/89146723 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74340c76f0 0x7f74340c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.121+0000 7f743a0af640 1 --2- 172.21.15.76:0/89146723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f743414d4b0 0x7f743414d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.121+0000 7f743a0af640 1 --2- 172.21.15.76:0/89146723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74340c7090 0x7f7434145df0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.121+0000 7f743a0af640 1 -- 172.21.15.76:0/89146723 >> 172.21.15.76:0/89146723 conn(0x7f74340ce950 msgr2=0x7f743406db30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:39.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.121+0000 7f743a0af640 1 -- 172.21.15.76:0/89146723 shutdown_connections 2023-11-09T03:48:39.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.121+0000 7f743a0af640 1 -- 172.21.15.76:0/89146723 wait complete. 2023-11-09T03:48:39.131 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-09T03:48:39.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.235+0000 7fbb85365640 1 Processor -- start 2023-11-09T03:48:39.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.235+0000 7fbb85365640 1 -- start start 2023-11-09T03:48:39.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.235+0000 7fbb85365640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbb80153e60 0x7fbb8015dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:39.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.235+0000 7fbb85365640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbb800cd6d0 0x7fbb80161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:39.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.235+0000 7fbb85365640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb800b4880 0x7fbb800b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:39.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.235+0000 7fbb85365640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb800cde60 con 0x7fbb80153e60 2023-11-09T03:48:39.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.235+0000 7fbb85365640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb800b5220 con 0x7fbb800cd6d0 2023-11-09T03:48:39.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.235+0000 7fbb85365640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb80160f90 con 0x7fbb800b4880 2023-11-09T03:48:39.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.236+0000 7fbb7effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbb80153e60 0x7fbb8015dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:39.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.236+0000 7fbb7effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbb80153e60 0x7fbb8015dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49834/0 (socket says 172.21.15.76:49834) 2023-11-09T03:48:39.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.236+0000 7fbb7effd640 1 -- 172.21.15.76:0/926668577 learned_addr learned my addr 172.21.15.76:0/926668577 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:39.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.236+0000 7fbb7f7fe640 1 --2- 172.21.15.76:0/926668577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb800b4880 0x7fbb800b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:39.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.236+0000 7fbb7e7fc640 1 --2- 172.21.15.76:0/926668577 >> v2:172.21.15.113:3300/0 conn(0x7fbb800cd6d0 0x7fbb80161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:39.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.236+0000 7fbb7effd640 1 -- 172.21.15.76:0/926668577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb800b4880 msgr2=0x7fbb800b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:39.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.236+0000 7fbb7effd640 1 --2- 172.21.15.76:0/926668577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb800b4880 0x7fbb800b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.236+0000 7fbb7effd640 1 -- 172.21.15.76:0/926668577 >> v2:172.21.15.113:3300/0 conn(0x7fbb800cd6d0 msgr2=0x7fbb80161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:39.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.236+0000 7fbb7effd640 1 --2- 172.21.15.76:0/926668577 >> v2:172.21.15.113:3300/0 conn(0x7fbb800cd6d0 0x7fbb80161420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.236+0000 7fbb7effd640 1 -- 172.21.15.76:0/926668577 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8015e400 con 0x7fbb80153e60 2023-11-09T03:48:39.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.237+0000 7fbb7effd640 1 --2- 172.21.15.76:0/926668577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb80153e60 0x7fbb8015dec0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fbb800c9e80 tx=0x7fbb7002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4960dbeac689e326 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:39.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.237+0000 7fbb7dffb640 1 -- 172.21.15.76:0/926668577 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7002ca70 con 0x7fbb80153e60 2023-11-09T03:48:39.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.237+0000 7fbb7dffb640 1 -- 172.21.15.76:0/926668577 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7002cbd0 con 0x7fbb80153e60 2023-11-09T03:48:39.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.237+0000 7fbb7dffb640 1 -- 172.21.15.76:0/926668577 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb700343f0 con 0x7fbb80153e60 2023-11-09T03:48:39.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.237+0000 7fbb85365640 1 -- 172.21.15.76:0/926668577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb80153e60 msgr2=0x7fbb8015dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:39.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.237+0000 7fbb85365640 1 --2- 172.21.15.76:0/926668577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb80153e60 0x7fbb8015dec0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fbb800c9e80 tx=0x7fbb7002bed0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.237+0000 7fbb85365640 1 -- 172.21.15.76:0/926668577 shutdown_connections 2023-11-09T03:48:39.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.237+0000 7fbb85365640 1 --2- 172.21.15.76:0/926668577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb800b4880 0x7fbb800b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.237+0000 7fbb85365640 1 --2- 172.21.15.76:0/926668577 >> v2:172.21.15.113:3300/0 conn(0x7fbb800cd6d0 0x7fbb80161420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.237+0000 7fbb85365640 1 --2- 172.21.15.76:0/926668577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb80153e60 0x7fbb8015dec0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.237+0000 7fbb85365640 1 -- 172.21.15.76:0/926668577 >> 172.21.15.76:0/926668577 conn(0x7fbb800cf880 msgr2=0x7fbb800cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:39.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.237+0000 7fbb85365640 1 -- 172.21.15.76:0/926668577 shutdown_connections 2023-11-09T03:48:39.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.237+0000 7fbb85365640 1 -- 172.21.15.76:0/926668577 wait complete. 2023-11-09T03:48:39.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.239+0000 7fbb85365640 1 Processor -- start 2023-11-09T03:48:39.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.239+0000 7fbb85365640 1 -- start start 2023-11-09T03:48:39.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.239+0000 7fbb85365640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb800b4880 0x7fbb8014d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:39.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.239+0000 7fbb85365640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb800cd6d0 0x7fbb80150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:39.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.239+0000 7fbb85365640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb80153e60 0x7fbb8014da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:39.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.239+0000 7fbb85365640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8014e770 con 0x7fbb800b4880 2023-11-09T03:48:39.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.239+0000 7fbb85365640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb800ca890 con 0x7fbb800cd6d0 2023-11-09T03:48:39.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.239+0000 7fbb85365640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8014dfc0 con 0x7fbb80153e60 2023-11-09T03:48:39.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.239+0000 7fbb7effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb800b4880 0x7fbb8014d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:39.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.239+0000 7fbb7effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb800b4880 0x7fbb8014d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49846/0 (socket says 172.21.15.76:49846) 2023-11-09T03:48:39.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.239+0000 7fbb7effd640 1 -- 172.21.15.76:0/1866654573 learned_addr learned my addr 172.21.15.76:0/1866654573 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:39.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.240+0000 7fbb7e7fc640 1 --2- 172.21.15.76:0/1866654573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb800cd6d0 0x7fbb80150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:39.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.240+0000 7fbb7effd640 1 -- 172.21.15.76:0/1866654573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb80153e60 msgr2=0x7fbb8014da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:39.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.240+0000 7fbb7f7fe640 1 --2- 172.21.15.76:0/1866654573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb80153e60 0x7fbb8014da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:39.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.240+0000 7fbb7effd640 1 --2- 172.21.15.76:0/1866654573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb80153e60 0x7fbb8014da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.240+0000 7fbb7effd640 1 -- 172.21.15.76:0/1866654573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb800cd6d0 msgr2=0x7fbb80150a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:39.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.240+0000 7fbb7effd640 1 --2- 172.21.15.76:0/1866654573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb800cd6d0 0x7fbb80150a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.240+0000 7fbb7effd640 1 -- 172.21.15.76:0/1866654573 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb70009b30 con 0x7fbb800b4880 2023-11-09T03:48:39.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.240+0000 7fbb7effd640 1 --2- 172.21.15.76:0/1866654573 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb800b4880 0x7fbb8014d4d0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fbb70000f80 tx=0x7fbb70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:39.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.240+0000 7fbb677fe640 1 -- 172.21.15.76:0/1866654573 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb70009510 con 0x7fbb800b4880 2023-11-09T03:48:39.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.240+0000 7fbb677fe640 1 -- 172.21.15.76:0/1866654573 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7003e450 con 0x7fbb800b4880 2023-11-09T03:48:39.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.240+0000 7fbb85365640 1 -- 172.21.15.76:0/1866654573 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb800cadf0 con 0x7fbb800b4880 2023-11-09T03:48:39.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.240+0000 7fbb677fe640 1 -- 172.21.15.76:0/1866654573 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7003e800 con 0x7fbb800b4880 2023-11-09T03:48:39.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.240+0000 7fbb85365640 1 -- 172.21.15.76:0/1866654573 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb8014e490 con 0x7fbb800b4880 2023-11-09T03:48:39.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.241+0000 7fbb85365640 1 -- 172.21.15.76:0/1866654573 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb80074290 con 0x7fbb800b4880 2023-11-09T03:48:39.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.241+0000 7fbb677fe640 1 -- 172.21.15.76:0/1866654573 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbb7003e960 con 0x7fbb800b4880 2023-11-09T03:48:39.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.242+0000 7fbb677fe640 1 --2- 172.21.15.76:0/1866654573 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbb6c06ae70 0x7fbb6c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:39.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.242+0000 7fbb677fe640 1 -- 172.21.15.76:0/1866654573 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7fbb700ac960 con 0x7fbb800b4880 2023-11-09T03:48:39.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.242+0000 7fbb7e7fc640 1 --2- 172.21.15.76:0/1866654573 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbb6c06ae70 0x7fbb6c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:39.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.243+0000 7fbb7e7fc640 1 --2- 172.21.15.76:0/1866654573 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbb6c06ae70 0x7fbb6c06d330 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbb8015b670 tx=0x7fbb74008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:39.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.246+0000 7fbb677fe640 1 -- 172.21.15.76:0/1866654573 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbb700774a0 con 0x7fbb800b4880 2023-11-09T03:48:39.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.411+0000 7fbb85365640 1 -- 172.21.15.76:0/1866654573 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7fbb800b5000 con 0x7fbb800b4880 2023-11-09T03:48:39.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.411+0000 7fbb677fe640 1 -- 172.21.15.76:0/1866654573 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7fbb7007aac0 con 0x7fbb800b4880 2023-11-09T03:48:39.411 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:48:39.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.413+0000 7fbb85365640 1 -- 172.21.15.76:0/1866654573 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbb6c06ae70 msgr2=0x7fbb6c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:39.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.413+0000 7fbb85365640 1 --2- 172.21.15.76:0/1866654573 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbb6c06ae70 0x7fbb6c06d330 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbb8015b670 tx=0x7fbb74008040 comp rx=0 tx=0).stop 2023-11-09T03:48:39.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.413+0000 7fbb85365640 1 -- 172.21.15.76:0/1866654573 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb800b4880 msgr2=0x7fbb8014d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:39.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.413+0000 7fbb85365640 1 --2- 172.21.15.76:0/1866654573 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb800b4880 0x7fbb8014d4d0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fbb70000f80 tx=0x7fbb70004a40 comp rx=0 tx=0).stop 2023-11-09T03:48:39.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.413+0000 7fbb85365640 1 -- 172.21.15.76:0/1866654573 shutdown_connections 2023-11-09T03:48:39.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.413+0000 7fbb85365640 1 --2- 172.21.15.76:0/1866654573 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbb6c06ae70 0x7fbb6c06d330 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.413+0000 7fbb85365640 1 --2- 172.21.15.76:0/1866654573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb80153e60 0x7fbb8014da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.413+0000 7fbb85365640 1 --2- 172.21.15.76:0/1866654573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb800cd6d0 0x7fbb80150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.413+0000 7fbb85365640 1 --2- 172.21.15.76:0/1866654573 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb800b4880 0x7fbb8014d4d0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.413+0000 7fbb85365640 1 -- 172.21.15.76:0/1866654573 >> 172.21.15.76:0/1866654573 conn(0x7fbb800cf880 msgr2=0x7fbb800b8110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:39.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.413+0000 7fbb85365640 1 -- 172.21.15.76:0/1866654573 shutdown_connections 2023-11-09T03:48:39.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.413+0000 7fbb85365640 1 -- 172.21.15.76:0/1866654573 wait complete. 2023-11-09T03:48:39.424 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:48:39.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.610+0000 7fa83dc32640 1 Processor -- start 2023-11-09T03:48:39.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.610+0000 7fa83dc32640 1 -- start start 2023-11-09T03:48:39.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.611+0000 7fa83dc32640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa8380cc9d0 0x7fa8380c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:39.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.611+0000 7fa83dc32640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8380c3860 con 0x7fa838153850 2023-11-09T03:48:39.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.611+0000 7fa83dc32640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8380c9250 con 0x7fa8380cc520 2023-11-09T03:48:39.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.611+0000 7fa83dc32640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa838153c30 con 0x7fa8380cc9d0 2023-11-09T03:48:39.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.611+0000 7fa8377fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa838153850 0x7fa83815a1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:49306/0 (socket says 172.21.15.76:49306) 2023-11-09T03:48:39.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.611+0000 7fa8377fe640 1 -- 172.21.15.76:0/3858769836 learned_addr learned my addr 172.21.15.76:0/3858769836 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:39.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.611+0000 7fa837fff640 1 --2- 172.21.15.76:0/3858769836 >> v2:172.21.15.113:3300/0 conn(0x7fa8380cc9d0 0x7fa8380c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:39.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.611+0000 7fa8367fc640 1 -- 172.21.15.76:0/3858769836 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922639760 0 0) 0x7fa8380c3860 con 0x7fa838153850 2023-11-09T03:48:39.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.611+0000 7fa8367fc640 1 -- 172.21.15.76:0/3858769836 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa820003520 con 0x7fa838153850 2023-11-09T03:48:39.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.611+0000 7fa8367fc640 1 -- 172.21.15.76:0/3858769836 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3797406714 0 0) 0x7fa820003520 con 0x7fa838153850 2023-11-09T03:48:39.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.611+0000 7fa8367fc640 1 -- 172.21.15.76:0/3858769836 >> v2:172.21.15.113:3300/0 conn(0x7fa8380cc9d0 msgr2=0x7fa8380c3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:39.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.611+0000 7fa8367fc640 1 --2- 172.21.15.76:0/3858769836 >> v2:172.21.15.113:3300/0 conn(0x7fa8380cc9d0 0x7fa8380c3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.611+0000 7fa8367fc640 1 -- 172.21.15.76:0/3858769836 >> v1:172.21.15.113:6789/0 conn(0x7fa8380cc520 legacy=0x7fa838070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:39.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.612+0000 7fa8367fc640 1 -- 172.21.15.76:0/3858769836 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8380ce590 con 0x7fa838153850 2023-11-09T03:48:39.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.612+0000 7fa8367fc640 1 -- 172.21.15.76:0/3858769836 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fa828001670 con 0x7fa838153850 2023-11-09T03:48:39.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.612+0000 7fa8367fc640 1 -- 172.21.15.76:0/3858769836 >> v1:172.21.15.76:6789/0 conn(0x7fa838153850 legacy=0x7fa83815a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:39.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.612+0000 7fa8367fc640 1 --2- 172.21.15.76:0/3858769836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa820003970 0x7fa820003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:39.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.612+0000 7fa8367fc640 1 --2- 172.21.15.76:0/3858769836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8200047c0 0x7fa820004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:39.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.612+0000 7fa8367fc640 1 --2- 172.21.15.76:0/3858769836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa820005180 0x7fa82000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:39.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.612+0000 7fa8367fc640 1 -- 172.21.15.76:0/3858769836 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa838153c30 con 0x7fa820003970 2023-11-09T03:48:39.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.612+0000 7fa8367fc640 1 -- 172.21.15.76:0/3858769836 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82000c0d0 con 0x7fa820005180 2023-11-09T03:48:39.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.612+0000 7fa8367fc640 1 -- 172.21.15.76:0/3858769836 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa82000c290 con 0x7fa8200047c0 2023-11-09T03:48:39.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.612+0000 7fa837fff640 1 --2- 172.21.15.76:0/3858769836 >> v2:172.21.15.113:3300/0 conn(0x7fa8380cc9d0 0x7fa8380c3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:39.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.612+0000 7fa8377fe640 1 --2- 172.21.15.76:0/3858769836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa820003970 0x7fa820003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:39.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.612+0000 7fa836ffd640 1 --2- 172.21.15.76:0/3858769836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8200047c0 0x7fa820004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:39.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.612+0000 7fa837fff640 1 --2- 172.21.15.76:0/3858769836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa820005180 0x7fa82000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:39.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.612+0000 7fa8377fe640 1 -- 172.21.15.76:0/3858769836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8200047c0 msgr2=0x7fa820004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:39.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.612+0000 7fa8377fe640 1 --2- 172.21.15.76:0/3858769836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8200047c0 0x7fa820004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.612+0000 7fa8377fe640 1 -- 172.21.15.76:0/3858769836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa820005180 msgr2=0x7fa82000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:39.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.612+0000 7fa8377fe640 1 --2- 172.21.15.76:0/3858769836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa820005180 0x7fa82000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.612+0000 7fa8377fe640 1 -- 172.21.15.76:0/3858769836 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa820003520 con 0x7fa820003970 2023-11-09T03:48:39.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.612+0000 7fa837fff640 1 -- 172.21.15.76:0/3858769836 reap_dead start 2023-11-09T03:48:39.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.613+0000 7fa8377fe640 1 --2- 172.21.15.76:0/3858769836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa820003970 0x7fa820003d90 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa82800c240 tx=0x7fa8200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebf9d34d0d8f4e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:39.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.613+0000 7fa8367fc640 1 -- 172.21.15.76:0/3858769836 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa828005ad0 con 0x7fa820003970 2023-11-09T03:48:39.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.613+0000 7fa8367fc640 1 -- 172.21.15.76:0/3858769836 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa828005c30 con 0x7fa820003970 2023-11-09T03:48:39.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.613+0000 7fa8367fc640 1 -- 172.21.15.76:0/3858769836 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82802caf0 con 0x7fa820003970 2023-11-09T03:48:39.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.613+0000 7fa83dc32640 1 -- 172.21.15.76:0/3858769836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa820003970 msgr2=0x7fa820003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:39.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.613+0000 7fa83dc32640 1 --2- 172.21.15.76:0/3858769836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa820003970 0x7fa820003d90 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa82800c240 tx=0x7fa8200057f0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.613+0000 7fa83dc32640 1 -- 172.21.15.76:0/3858769836 shutdown_connections 2023-11-09T03:48:39.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.613+0000 7fa83dc32640 1 --2- 172.21.15.76:0/3858769836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa820003970 0x7fa820003d90 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.613+0000 7fa83dc32640 1 -- 172.21.15.76:0/3858769836 >> 172.21.15.76:0/3858769836 conn(0x7fa8380ce950 msgr2=0x7fa8380ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:39.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.613+0000 7fa83dc32640 1 -- 172.21.15.76:0/3858769836 shutdown_connections 2023-11-09T03:48:39.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.613+0000 7fa83dc32640 1 -- 172.21.15.76:0/3858769836 wait complete. 2023-11-09T03:48:39.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.615+0000 7fa83dc32640 1 Processor -- start 2023-11-09T03:48:39.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.615+0000 7fa83dc32640 1 -- start start 2023-11-09T03:48:39.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.616+0000 7fa83dc32640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa820003970 0x7fa838156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:39.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.616+0000 7fa83dc32640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa838154e10 0x7fa83806d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:39.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.616+0000 7fa83dc32640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8380c8670 0x7fa83806d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:39.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.616+0000 7fa83dc32640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa838153d70 con 0x7fa838154e10 2023-11-09T03:48:39.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.616+0000 7fa83dc32640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa838154050 con 0x7fa8380c8670 2023-11-09T03:48:39.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.616+0000 7fa83dc32640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa838154330 con 0x7fa820003970 2023-11-09T03:48:39.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.616+0000 7fa836ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa838154e10 0x7fa83806d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:39.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.616+0000 7fa836ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa838154e10 0x7fa83806d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49856/0 (socket says 172.21.15.76:49856) 2023-11-09T03:48:39.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.617+0000 7fa836ffd640 1 -- 172.21.15.76:0/1019388331 learned_addr learned my addr 172.21.15.76:0/1019388331 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:39.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.617+0000 7fa8377fe640 1 --2- 172.21.15.76:0/1019388331 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa820003970 0x7fa838156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:39.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.617+0000 7fa837fff640 1 --2- 172.21.15.76:0/1019388331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8380c8670 0x7fa83806d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:39.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.617+0000 7fa836ffd640 1 -- 172.21.15.76:0/1019388331 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa820003970 msgr2=0x7fa838156370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:39.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.617+0000 7fa836ffd640 1 --2- 172.21.15.76:0/1019388331 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa820003970 0x7fa838156370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.617+0000 7fa836ffd640 1 -- 172.21.15.76:0/1019388331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8380c8670 msgr2=0x7fa83806d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:39.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.617+0000 7fa836ffd640 1 --2- 172.21.15.76:0/1019388331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8380c8670 0x7fa83806d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.617+0000 7fa836ffd640 1 -- 172.21.15.76:0/1019388331 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa838154610 con 0x7fa838154e10 2023-11-09T03:48:39.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.617+0000 7fa836ffd640 1 --2- 172.21.15.76:0/1019388331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa838154e10 0x7fa83806d2b0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa82c00e550 tx=0x7fa82c00ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:39.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.617+0000 7fa81ffff640 1 -- 172.21.15.76:0/1019388331 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c0052e0 con 0x7fa838154e10 2023-11-09T03:48:39.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.617+0000 7fa81ffff640 1 -- 172.21.15.76:0/1019388331 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82c005440 con 0x7fa838154e10 2023-11-09T03:48:39.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.617+0000 7fa83dc32640 1 -- 172.21.15.76:0/1019388331 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8381548f0 con 0x7fa838154e10 2023-11-09T03:48:39.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.617+0000 7fa81ffff640 1 -- 172.21.15.76:0/1019388331 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c003810 con 0x7fa838154e10 2023-11-09T03:48:39.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.617+0000 7fa83dc32640 1 -- 172.21.15.76:0/1019388331 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa83814e1b0 con 0x7fa838154e10 2023-11-09T03:48:39.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.618+0000 7fa83dc32640 1 -- 172.21.15.76:0/1019388331 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa838074290 con 0x7fa838154e10 2023-11-09T03:48:39.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.619+0000 7fa81ffff640 1 -- 172.21.15.76:0/1019388331 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa82c003970 con 0x7fa838154e10 2023-11-09T03:48:39.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.619+0000 7fa81ffff640 1 --2- 172.21.15.76:0/1019388331 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa82406aea0 0x7fa82406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:39.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.619+0000 7fa81ffff640 1 -- 172.21.15.76:0/1019388331 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7fa82c089780 con 0x7fa838154e10 2023-11-09T03:48:39.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.619+0000 7fa8377fe640 1 --2- 172.21.15.76:0/1019388331 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa82406aea0 0x7fa82406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:39.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.620+0000 7fa8377fe640 1 --2- 172.21.15.76:0/1019388331 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa82406aea0 0x7fa82406d360 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa828034000 tx=0x7fa828009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:39.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.623+0000 7fa81ffff640 1 -- 172.21.15.76:0/1019388331 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa82c054400 con 0x7fa838154e10 2023-11-09T03:48:39.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.788+0000 7fa83dc32640 1 -- 172.21.15.76:0/1019388331 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa8380c5010 con 0x7fa838154e10 2023-11-09T03:48:39.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.789+0000 7fa81ffff640 1 -- 172.21.15.76:0/1019388331 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa82c057a20 con 0x7fa838154e10 2023-11-09T03:48:39.788 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:48:39.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.791+0000 7fa83dc32640 1 -- 172.21.15.76:0/1019388331 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa82406aea0 msgr2=0x7fa82406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:39.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.791+0000 7fa83dc32640 1 --2- 172.21.15.76:0/1019388331 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa82406aea0 0x7fa82406d360 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa828034000 tx=0x7fa828009030 comp rx=0 tx=0).stop 2023-11-09T03:48:39.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.791+0000 7fa83dc32640 1 -- 172.21.15.76:0/1019388331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa838154e10 msgr2=0x7fa83806d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:39.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.791+0000 7fa83dc32640 1 --2- 172.21.15.76:0/1019388331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa838154e10 0x7fa83806d2b0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa82c00e550 tx=0x7fa82c00ea10 comp rx=0 tx=0).stop 2023-11-09T03:48:39.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.791+0000 7fa83dc32640 1 -- 172.21.15.76:0/1019388331 shutdown_connections 2023-11-09T03:48:39.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.791+0000 7fa83dc32640 1 --2- 172.21.15.76:0/1019388331 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa82406aea0 0x7fa82406d360 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.791+0000 7fa83dc32640 1 --2- 172.21.15.76:0/1019388331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8380c8670 0x7fa83806d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.791+0000 7fa83dc32640 1 --2- 172.21.15.76:0/1019388331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa838154e10 0x7fa83806d2b0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.791+0000 7fa83dc32640 1 --2- 172.21.15.76:0/1019388331 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa820003970 0x7fa838156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.791+0000 7fa83dc32640 1 -- 172.21.15.76:0/1019388331 >> 172.21.15.76:0/1019388331 conn(0x7fa8380ce950 msgr2=0x7fa838070610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:39.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.791+0000 7fa83dc32640 1 -- 172.21.15.76:0/1019388331 shutdown_connections 2023-11-09T03:48:39.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.791+0000 7fa83dc32640 1 -- 172.21.15.76:0/1019388331 wait complete. 2023-11-09T03:48:39.801 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:48:39.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.904+0000 7fe69ddf5640 1 Processor -- start 2023-11-09T03:48:39.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.904+0000 7fe69ddf5640 1 -- start start 2023-11-09T03:48:39.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.904+0000 7fe69ddf5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe698159c00 0x7fe6980cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:39.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.904+0000 7fe69ddf5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe69815aff0 con 0x7fe6980c48e0 2023-11-09T03:48:39.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.904+0000 7fe69ddf5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe69815a3e0 con 0x7fe6980ae1d0 2023-11-09T03:48:39.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.904+0000 7fe69ddf5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe69815a6c0 con 0x7fe698159c00 2023-11-09T03:48:39.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.905+0000 7fe6977fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe698159c00 0x7fe6980cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:39.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.905+0000 7fe6977fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe698159c00 0x7fe6980cc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49870/0 (socket says 172.21.15.76:49870) 2023-11-09T03:48:39.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.905+0000 7fe696ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe6980c48e0 0x7fe6980adac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:49320/0 (socket says 172.21.15.76:49320) 2023-11-09T03:48:39.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.905+0000 7fe6977fe640 1 -- 172.21.15.76:0/3597667853 learned_addr learned my addr 172.21.15.76:0/3597667853 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:39.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.905+0000 7fe6977fe640 1 -- 172.21.15.76:0/3597667853 >> v1:172.21.15.113:6789/0 conn(0x7fe6980ae1d0 legacy=0x7fe6980709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:39.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.905+0000 7fe6977fe640 1 -- 172.21.15.76:0/3597667853 >> v1:172.21.15.76:6789/0 conn(0x7fe6980c48e0 legacy=0x7fe6980adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:39.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.905+0000 7fe6967fc640 1 -- 172.21.15.76:0/3597667853 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 225152429 0 0) 0x7fe69815aff0 con 0x7fe6980c48e0 2023-11-09T03:48:39.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.905+0000 7fe6977fe640 1 -- 172.21.15.76:0/3597667853 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69815a9a0 con 0x7fe698159c00 2023-11-09T03:48:39.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.906+0000 7fe6977fe640 1 --2- 172.21.15.76:0/3597667853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe698159c00 0x7fe6980cc890 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fe68c009870 tx=0x7fe68c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732bf68dc9e1ad6e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:39.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.906+0000 7fe6967fc640 1 -- 172.21.15.76:0/3597667853 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68c030860 con 0x7fe698159c00 2023-11-09T03:48:39.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.906+0000 7fe6967fc640 1 -- 172.21.15.76:0/3597667853 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68c0309c0 con 0x7fe698159c00 2023-11-09T03:48:39.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.906+0000 7fe6967fc640 1 -- 172.21.15.76:0/3597667853 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68c030d50 con 0x7fe698159c00 2023-11-09T03:48:39.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.906+0000 7fe69ddf5640 1 -- 172.21.15.76:0/3597667853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe698159c00 msgr2=0x7fe6980cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:39.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.906+0000 7fe69ddf5640 1 --2- 172.21.15.76:0/3597667853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe698159c00 0x7fe6980cc890 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fe68c009870 tx=0x7fe68c02b9a0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.906+0000 7fe69ddf5640 1 -- 172.21.15.76:0/3597667853 shutdown_connections 2023-11-09T03:48:39.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.906+0000 7fe69ddf5640 1 --2- 172.21.15.76:0/3597667853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe698159c00 0x7fe6980cc890 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.906+0000 7fe69ddf5640 1 -- 172.21.15.76:0/3597667853 >> 172.21.15.76:0/3597667853 conn(0x7fe6980ce950 msgr2=0x7fe6980ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:39.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.906+0000 7fe69ddf5640 1 -- 172.21.15.76:0/3597667853 shutdown_connections 2023-11-09T03:48:39.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.906+0000 7fe69ddf5640 1 -- 172.21.15.76:0/3597667853 wait complete. 2023-11-09T03:48:39.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.907+0000 7fe69ddf5640 1 Processor -- start 2023-11-09T03:48:39.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.908+0000 7fe69ddf5640 1 -- start start 2023-11-09T03:48:39.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.908+0000 7fe69ddf5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6980ae1d0 0x7fe698144230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:39.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.908+0000 7fe69ddf5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6980c48e0 0x7fe6981507f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:39.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.908+0000 7fe69ddf5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe698159c00 0x7fe6980c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:39.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.908+0000 7fe69ddf5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe698151de0 con 0x7fe6980c48e0 2023-11-09T03:48:39.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.908+0000 7fe69ddf5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6981511d0 con 0x7fe6980ae1d0 2023-11-09T03:48:39.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.908+0000 7fe69ddf5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6981514b0 con 0x7fe698159c00 2023-11-09T03:48:39.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.908+0000 7fe696ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6980c48e0 0x7fe6981507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:39.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.908+0000 7fe696ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6980c48e0 0x7fe6981507f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49886/0 (socket says 172.21.15.76:49886) 2023-11-09T03:48:39.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.909+0000 7fe696ffd640 1 -- 172.21.15.76:0/4139363171 learned_addr learned my addr 172.21.15.76:0/4139363171 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:39.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.909+0000 7fe6977fe640 1 --2- 172.21.15.76:0/4139363171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6980ae1d0 0x7fe698144230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:39.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.909+0000 7fe697fff640 1 --2- 172.21.15.76:0/4139363171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe698159c00 0x7fe6980c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:39.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.909+0000 7fe696ffd640 1 -- 172.21.15.76:0/4139363171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe698159c00 msgr2=0x7fe6980c7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:39.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.909+0000 7fe696ffd640 1 --2- 172.21.15.76:0/4139363171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe698159c00 0x7fe6980c7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.909+0000 7fe696ffd640 1 -- 172.21.15.76:0/4139363171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6980ae1d0 msgr2=0x7fe698144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:39.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.909+0000 7fe696ffd640 1 --2- 172.21.15.76:0/4139363171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6980ae1d0 0x7fe698144230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:39.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.909+0000 7fe696ffd640 1 -- 172.21.15.76:0/4139363171 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68c0094d0 con 0x7fe6980c48e0 2023-11-09T03:48:39.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.909+0000 7fe6977fe640 1 --2- 172.21.15.76:0/4139363171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6980ae1d0 0x7fe698144230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:39.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.909+0000 7fe696ffd640 1 --2- 172.21.15.76:0/4139363171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6980c48e0 0x7fe6981507f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fe68800c4a0 tx=0x7fe68800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:39.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.909+0000 7fe67ffff640 1 -- 172.21.15.76:0/4139363171 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6880074e0 con 0x7fe6980c48e0 2023-11-09T03:48:39.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.909+0000 7fe67ffff640 1 -- 172.21.15.76:0/4139363171 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe688007640 con 0x7fe6980c48e0 2023-11-09T03:48:39.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.909+0000 7fe67ffff640 1 -- 172.21.15.76:0/4139363171 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe688004a00 con 0x7fe6980c48e0 2023-11-09T03:48:39.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.909+0000 7fe69ddf5640 1 -- 172.21.15.76:0/4139363171 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe698151790 con 0x7fe6980c48e0 2023-11-09T03:48:39.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.909+0000 7fe69ddf5640 1 -- 172.21.15.76:0/4139363171 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe698144770 con 0x7fe6980c48e0 2023-11-09T03:48:39.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.910+0000 7fe69ddf5640 1 -- 172.21.15.76:0/4139363171 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6980c8c60 con 0x7fe6980c48e0 2023-11-09T03:48:39.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.910+0000 7fe67ffff640 1 -- 172.21.15.76:0/4139363171 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe688004b60 con 0x7fe6980c48e0 2023-11-09T03:48:39.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.911+0000 7fe67ffff640 1 --2- 172.21.15.76:0/4139363171 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe68406ae70 0x7fe68406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:39.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.911+0000 7fe67ffff640 1 -- 172.21.15.76:0/4139363171 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7fe688088dc0 con 0x7fe6980c48e0 2023-11-09T03:48:39.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.911+0000 7fe6977fe640 1 --2- 172.21.15.76:0/4139363171 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe68406ae70 0x7fe68406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:39.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.911+0000 7fe6977fe640 1 --2- 172.21.15.76:0/4139363171 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe68406ae70 0x7fe68406d330 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe68c036000 tx=0x7fe68c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:39.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:39.915+0000 7fe67ffff640 1 -- 172.21.15.76:0/4139363171 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe688053870 con 0x7fe6980c48e0 2023-11-09T03:48:40.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.080+0000 7fe69ddf5640 1 -- 172.21.15.76:0/4139363171 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe698074290 con 0x7fe6980c48e0 2023-11-09T03:48:40.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.080+0000 7fe67ffff640 1 -- 172.21.15.76:0/4139363171 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe688018070 con 0x7fe6980c48e0 2023-11-09T03:48:40.080 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:48:40.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.082+0000 7fe69ddf5640 1 -- 172.21.15.76:0/4139363171 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe68406ae70 msgr2=0x7fe68406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:40.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.082+0000 7fe69ddf5640 1 --2- 172.21.15.76:0/4139363171 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe68406ae70 0x7fe68406d330 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe68c036000 tx=0x7fe68c008030 comp rx=0 tx=0).stop 2023-11-09T03:48:40.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.082+0000 7fe69ddf5640 1 -- 172.21.15.76:0/4139363171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6980c48e0 msgr2=0x7fe6981507f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:40.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.082+0000 7fe69ddf5640 1 --2- 172.21.15.76:0/4139363171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6980c48e0 0x7fe6981507f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fe68800c4a0 tx=0x7fe68800c960 comp rx=0 tx=0).stop 2023-11-09T03:48:40.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.082+0000 7fe69ddf5640 1 -- 172.21.15.76:0/4139363171 shutdown_connections 2023-11-09T03:48:40.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.082+0000 7fe69ddf5640 1 --2- 172.21.15.76:0/4139363171 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe68406ae70 0x7fe68406d330 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:40.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.082+0000 7fe69ddf5640 1 --2- 172.21.15.76:0/4139363171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe698159c00 0x7fe6980c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:40.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.082+0000 7fe69ddf5640 1 --2- 172.21.15.76:0/4139363171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6980c48e0 0x7fe6981507f0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:40.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.082+0000 7fe69ddf5640 1 --2- 172.21.15.76:0/4139363171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6980ae1d0 0x7fe698144230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:40.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.082+0000 7fe69ddf5640 1 -- 172.21.15.76:0/4139363171 >> 172.21.15.76:0/4139363171 conn(0x7fe6980ce950 msgr2=0x7fe6980ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:40.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.082+0000 7fe69ddf5640 1 -- 172.21.15.76:0/4139363171 shutdown_connections 2023-11-09T03:48:40.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.082+0000 7fe69ddf5640 1 -- 172.21.15.76:0/4139363171 wait complete. 2023-11-09T03:48:40.092 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-09T03:48:40.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.193+0000 7f49923f5640 1 Processor -- start 2023-11-09T03:48:40.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.193+0000 7f49923f5640 1 -- start start 2023-11-09T03:48:40.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.194+0000 7f49923f5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f498c0c4ff0 0x7f498c0cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:40.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.194+0000 7f49923f5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f498c15aff0 con 0x7f498c0adac0 2023-11-09T03:48:40.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.194+0000 7f49923f5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f498c15a3e0 con 0x7f498c159c00 2023-11-09T03:48:40.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.194+0000 7f49923f5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498c15a6c0 con 0x7f498c0c4ff0 2023-11-09T03:48:40.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.194+0000 7f4990992640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f498c0adac0 0x7f498c0adf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:49334/0 (socket says 172.21.15.76:49334) 2023-11-09T03:48:40.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.194+0000 7f4990992640 1 -- 172.21.15.76:0/2994774090 learned_addr learned my addr 172.21.15.76:0/2994774090 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:40.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.194+0000 7f498affd640 1 -- 172.21.15.76:0/2994774090 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3898608456 0 0) 0x7f498c15aff0 con 0x7f498c0adac0 2023-11-09T03:48:40.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.195+0000 7f498affd640 1 -- 172.21.15.76:0/2994774090 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4974003520 con 0x7f498c0adac0 2023-11-09T03:48:40.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.195+0000 7f498b7fe640 1 --2- 172.21.15.76:0/2994774090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f498c0c4ff0 0x7f498c0cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:40.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.195+0000 7f498affd640 1 -- 172.21.15.76:0/2994774090 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2503951273 0 0) 0x7f4974003520 con 0x7f498c0adac0 2023-11-09T03:48:40.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.195+0000 7f498affd640 1 -- 172.21.15.76:0/2994774090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f498c0c4ff0 msgr2=0x7f498c0cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:40.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.195+0000 7f498affd640 1 --2- 172.21.15.76:0/2994774090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f498c0c4ff0 0x7f498c0cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:40.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.195+0000 7f498affd640 1 -- 172.21.15.76:0/2994774090 >> v1:172.21.15.113:6789/0 conn(0x7f498c159c00 legacy=0x7f498c0c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:40.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.195+0000 7f498affd640 1 -- 172.21.15.76:0/2994774090 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498c15a9a0 con 0x7f498c0adac0 2023-11-09T03:48:40.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.195+0000 7f498affd640 1 -- 172.21.15.76:0/2994774090 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f49840019f0 con 0x7f498c0adac0 2023-11-09T03:48:40.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.195+0000 7f498b7fe640 1 --2- 172.21.15.76:0/2994774090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f498c0c4ff0 0x7f498c0cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:40.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.195+0000 7f498affd640 1 -- 172.21.15.76:0/2994774090 >> v1:172.21.15.76:6789/0 conn(0x7f498c0adac0 legacy=0x7f498c0adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:40.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.195+0000 7f498affd640 1 --2- 172.21.15.76:0/2994774090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f498c0c4ff0 0x7f49740039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:40.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.195+0000 7f498affd640 1 --2- 172.21.15.76:0/2994774090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4974003f90 0x7f49740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:40.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.195+0000 7f498affd640 1 --2- 172.21.15.76:0/2994774090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4974004d00 0x7f4974005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:40.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.195+0000 7f498affd640 1 -- 172.21.15.76:0/2994774090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c15a6c0 con 0x7f4974004d00 2023-11-09T03:48:40.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.195+0000 7f498affd640 1 -- 172.21.15.76:0/2994774090 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497400bae0 con 0x7f4974003f90 2023-11-09T03:48:40.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.195+0000 7f498affd640 1 -- 172.21.15.76:0/2994774090 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f497400bca0 con 0x7f498c0c4ff0 2023-11-09T03:48:40.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.195+0000 7f4990992640 1 --2- 172.21.15.76:0/2994774090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4974004d00 0x7f4974005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:40.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.195+0000 7f498b7fe640 1 --2- 172.21.15.76:0/2994774090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4974003f90 0x7f49740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:40.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.195+0000 7f498bfff640 1 --2- 172.21.15.76:0/2994774090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f498c0c4ff0 0x7f49740039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:40.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.195+0000 7f4990992640 1 -- 172.21.15.76:0/2994774090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f498c0c4ff0 msgr2=0x7f49740039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:40.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.195+0000 7f4990992640 1 --2- 172.21.15.76:0/2994774090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f498c0c4ff0 0x7f49740039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:40.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.195+0000 7f4990992640 1 -- 172.21.15.76:0/2994774090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4974003f90 msgr2=0x7f49740047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:40.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.195+0000 7f4990992640 1 --2- 172.21.15.76:0/2994774090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4974003f90 0x7f49740047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:40.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.195+0000 7f4990992640 1 -- 172.21.15.76:0/2994774090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4974003520 con 0x7f4974004d00 2023-11-09T03:48:40.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.195+0000 7f498bfff640 1 --2- 172.21.15.76:0/2994774090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f498c0c4ff0 0x7f49740039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:40.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.196+0000 7f4990992640 1 --2- 172.21.15.76:0/2994774090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4974004d00 0x7f4974005180 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f498400a820 tx=0x7f49740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e1726ac4b82e14 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:40.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.196+0000 7f498affd640 1 -- 172.21.15.76:0/2994774090 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49840072d0 con 0x7f4974004d00 2023-11-09T03:48:40.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.196+0000 7f498affd640 1 -- 172.21.15.76:0/2994774090 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498402f070 con 0x7f4974004d00 2023-11-09T03:48:40.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.196+0000 7f498affd640 1 -- 172.21.15.76:0/2994774090 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49840076f0 con 0x7f4974004d00 2023-11-09T03:48:40.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.197+0000 7f49923f5640 1 -- 172.21.15.76:0/2994774090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4974004d00 msgr2=0x7f4974005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:40.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.197+0000 7f49923f5640 1 --2- 172.21.15.76:0/2994774090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4974004d00 0x7f4974005180 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f498400a820 tx=0x7f49740057f0 comp rx=0 tx=0).stop 2023-11-09T03:48:40.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.197+0000 7f4990992640 1 -- 172.21.15.76:0/2994774090 reap_dead start 2023-11-09T03:48:40.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.197+0000 7f49923f5640 1 -- 172.21.15.76:0/2994774090 shutdown_connections 2023-11-09T03:48:40.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.197+0000 7f49923f5640 1 -- 172.21.15.76:0/2994774090 >> 172.21.15.76:0/2994774090 conn(0x7f498c0ce950 msgr2=0x7f498c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:40.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.197+0000 7f49923f5640 1 -- 172.21.15.76:0/2994774090 shutdown_connections 2023-11-09T03:48:40.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.197+0000 7f49923f5640 1 -- 172.21.15.76:0/2994774090 wait complete. 2023-11-09T03:48:40.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.198+0000 7f49923f5640 1 Processor -- start 2023-11-09T03:48:40.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.198+0000 7f49923f5640 1 -- start start 2023-11-09T03:48:40.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.199+0000 7f49923f5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f498c144340 0x7f498c150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:40.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.199+0000 7f49923f5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f498c152b30 0x7f498c152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:40.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.199+0000 7f49923f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f498c1f39e0 0x7f498c1f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:40.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.199+0000 7f49923f5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c1534f0 con 0x7f498c152b30 2023-11-09T03:48:40.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.199+0000 7f49923f5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c1512b0 con 0x7f498c1f39e0 2023-11-09T03:48:40.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.199+0000 7f49923f5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498c151590 con 0x7f498c144340 2023-11-09T03:48:40.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.199+0000 7f498b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f498c152b30 0x7f498c152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:40.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.199+0000 7f498b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f498c152b30 0x7f498c152fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49904/0 (socket says 172.21.15.76:49904) 2023-11-09T03:48:40.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.199+0000 7f498b7fe640 1 -- 172.21.15.76:0/4099485023 learned_addr learned my addr 172.21.15.76:0/4099485023 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:40.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.199+0000 7f498bfff640 1 --2- 172.21.15.76:0/4099485023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f498c144340 0x7f498c150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:40.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.199+0000 7f4990992640 1 --2- 172.21.15.76:0/4099485023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f498c1f39e0 0x7f498c1f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:40.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.199+0000 7f498b7fe640 1 -- 172.21.15.76:0/4099485023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f498c144340 msgr2=0x7f498c150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:40.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.199+0000 7f498b7fe640 1 --2- 172.21.15.76:0/4099485023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f498c144340 0x7f498c150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:40.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.199+0000 7f498b7fe640 1 -- 172.21.15.76:0/4099485023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f498c1f39e0 msgr2=0x7f498c1f3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:40.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.199+0000 7f498b7fe640 1 --2- 172.21.15.76:0/4099485023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f498c1f39e0 0x7f498c1f3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:40.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.199+0000 7f498b7fe640 1 -- 172.21.15.76:0/4099485023 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498c151870 con 0x7f498c152b30 2023-11-09T03:48:40.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.199+0000 7f498b7fe640 1 --2- 172.21.15.76:0/4099485023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f498c152b30 0x7f498c152fb0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f4980009fd0 tx=0x7f49800119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:40.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.199+0000 7f4988ff9640 1 -- 172.21.15.76:0/4099485023 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498000c070 con 0x7f498c152b30 2023-11-09T03:48:40.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.199+0000 7f4988ff9640 1 -- 172.21.15.76:0/4099485023 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4980015930 con 0x7f498c152b30 2023-11-09T03:48:40.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.199+0000 7f49923f5640 1 -- 172.21.15.76:0/4099485023 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f498c151b50 con 0x7f498c152b30 2023-11-09T03:48:40.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.199+0000 7f4988ff9640 1 -- 172.21.15.76:0/4099485023 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4980015cc0 con 0x7f498c152b30 2023-11-09T03:48:40.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.200+0000 7f49923f5640 1 -- 172.21.15.76:0/4099485023 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f498c150ea0 con 0x7f498c152b30 2023-11-09T03:48:40.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.200+0000 7f49923f5640 1 -- 172.21.15.76:0/4099485023 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f498c074290 con 0x7f498c152b30 2023-11-09T03:48:40.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.201+0000 7f4988ff9640 1 -- 172.21.15.76:0/4099485023 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4980013400 con 0x7f498c152b30 2023-11-09T03:48:40.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.201+0000 7f4988ff9640 1 --2- 172.21.15.76:0/4099485023 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f497806aea0 0x7f497806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:40.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.201+0000 7f4988ff9640 1 -- 172.21.15.76:0/4099485023 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f4980089da0 con 0x7f498c152b30 2023-11-09T03:48:40.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.201+0000 7f498bfff640 1 --2- 172.21.15.76:0/4099485023 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f497806aea0 0x7f497806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:40.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.202+0000 7f498bfff640 1 --2- 172.21.15.76:0/4099485023 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f497806aea0 0x7f497806d360 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f498c1503e0 tx=0x7f497c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:40.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.205+0000 7f4988ff9640 1 -- 172.21.15.76:0/4099485023 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4980054970 con 0x7f498c152b30 2023-11-09T03:48:40.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.385+0000 7f49923f5640 1 -- 172.21.15.76:0/4099485023 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f498c0cd430 con 0x7f498c152b30 2023-11-09T03:48:40.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.385+0000 7f4988ff9640 1 -- 172.21.15.76:0/4099485023 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v83) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4980023080 con 0x7f498c152b30 2023-11-09T03:48:40.385 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2023-11-09T03:48:40.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.387+0000 7f49923f5640 1 -- 172.21.15.76:0/4099485023 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f497806aea0 msgr2=0x7f497806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:40.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.387+0000 7f49923f5640 1 --2- 172.21.15.76:0/4099485023 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f497806aea0 0x7f497806d360 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f498c1503e0 tx=0x7f497c009040 comp rx=0 tx=0).stop 2023-11-09T03:48:40.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.387+0000 7f49923f5640 1 -- 172.21.15.76:0/4099485023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f498c152b30 msgr2=0x7f498c152fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:40.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.387+0000 7f49923f5640 1 --2- 172.21.15.76:0/4099485023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f498c152b30 0x7f498c152fb0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f4980009fd0 tx=0x7f49800119f0 comp rx=0 tx=0).stop 2023-11-09T03:48:40.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.387+0000 7f49923f5640 1 -- 172.21.15.76:0/4099485023 shutdown_connections 2023-11-09T03:48:40.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.387+0000 7f49923f5640 1 --2- 172.21.15.76:0/4099485023 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f497806aea0 0x7f497806d360 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:40.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.387+0000 7f49923f5640 1 --2- 172.21.15.76:0/4099485023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f498c1f39e0 0x7f498c1f3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:40.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.387+0000 7f49923f5640 1 --2- 172.21.15.76:0/4099485023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f498c152b30 0x7f498c152fb0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:40.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.387+0000 7f49923f5640 1 --2- 172.21.15.76:0/4099485023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f498c144340 0x7f498c150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:40.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.387+0000 7f49923f5640 1 -- 172.21.15.76:0/4099485023 >> 172.21.15.76:0/4099485023 conn(0x7f498c0ce950 msgr2=0x7f498c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:40.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.387+0000 7f49923f5640 1 -- 172.21.15.76:0/4099485023 shutdown_connections 2023-11-09T03:48:40.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.387+0000 7f49923f5640 1 -- 172.21.15.76:0/4099485023 wait complete. 2023-11-09T03:48:40.398 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi076.front.sepia.ceph.com... 2023-11-09T03:48:40.398 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:48:40.399 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-09T03:48:40.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:48:40.459 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[114461]: fuse finished with error 0 and tester_r 0 2023-11-09T03:48:40.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:48:40.466 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T03:48:40.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:48:40.466 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-09T03:48:40.469 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:48:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:48:40.481 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:48:40.481 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:48:40.501 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2440 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-09T03:48:40.501 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2023-11-09T03:48:40.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:48:40.502 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-09T03:48:40.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:48:40.559 INFO:teuthology.orchestra.run.smithi113.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-09T03:48:40.559 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:48:40.560 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:48:40.575 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056381,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14761 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248868k,nr_inodes=812217,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-09T03:48:40.576 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T03:48:40.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:48:40.576 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-09T03:48:40.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:48:40.631 INFO:teuthology.orchestra.run.smithi076.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-09T03:48:40.631 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:48:40.631 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:48:40.649 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2440 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-09T03:48:40.650 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2023-11-09T03:48:40.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:48:40.650 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-09T03:48:40.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:48:40.707 INFO:teuthology.orchestra.run.smithi113.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-09T03:48:40.707 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:48:40.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.809+0000 7f77f51f5640 1 Processor -- start 2023-11-09T03:48:40.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.809+0000 7f77f51f5640 1 -- start start 2023-11-09T03:48:40.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.809+0000 7f77f51f5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77f00cde60 0x7f77f015d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:40.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.809+0000 7f77f51f5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f77f00ce240 0x7f77f00d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:40.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.809+0000 7f77f51f5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f015e000 con 0x7f77f015daa0 2023-11-09T03:48:40.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.809+0000 7f77f51f5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f015e5c0 con 0x7f77f00ce240 2023-11-09T03:48:40.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.809+0000 7f77f51f5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f015e2e0 con 0x7f77f00cde60 2023-11-09T03:48:40.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.810+0000 7f77ef59e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f77f00ce240 0x7f77f00d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:40.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.810+0000 7f77e7fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f77f015daa0 0x7f77f0157000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:49344/0 (socket says 172.21.15.76:49344) 2023-11-09T03:48:40.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.810+0000 7f77e7fff640 1 -- 172.21.15.76:0/18183121 learned_addr learned my addr 172.21.15.76:0/18183121 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:40.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.810+0000 7f77eed9d640 1 --2- 172.21.15.76:0/18183121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77f00cde60 0x7f77f015d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:40.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.810+0000 7f77ef59e640 1 -- 172.21.15.76:0/18183121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77f00cde60 msgr2=0x7f77f015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:40.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.810+0000 7f77ef59e640 1 --2- 172.21.15.76:0/18183121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77f00cde60 0x7f77f015d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:40.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.810+0000 7f77ef59e640 1 -- 172.21.15.76:0/18183121 >> v1:172.21.15.76:6789/0 conn(0x7f77f015daa0 legacy=0x7f77f0157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:40.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.810+0000 7f77ef59e640 1 -- 172.21.15.76:0/18183121 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f00d1120 con 0x7f77f00ce240 2023-11-09T03:48:40.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.810+0000 7f77ee59c640 1 -- 172.21.15.76:0/18183121 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1919642173 0 0) 0x7f77f015e000 con 0x7f77f015daa0 2023-11-09T03:48:40.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.811+0000 7f77ef59e640 1 --2- 172.21.15.76:0/18183121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77f00ce240 0x7f77f00d1920 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009d90 tx=0x7f77e802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f92ae4c0f99a4c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:40.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.811+0000 7f77ee59c640 1 -- 172.21.15.76:0/18183121 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e802cc50 con 0x7f77f00ce240 2023-11-09T03:48:40.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.811+0000 7f77ee59c640 1 -- 172.21.15.76:0/18183121 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e802cdb0 con 0x7f77f00ce240 2023-11-09T03:48:40.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.811+0000 7f77ee59c640 1 -- 172.21.15.76:0/18183121 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e8034660 con 0x7f77f00ce240 2023-11-09T03:48:40.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.811+0000 7f77f51f5640 1 -- 172.21.15.76:0/18183121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77f00ce240 msgr2=0x7f77f00d1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:40.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.811+0000 7f77f51f5640 1 --2- 172.21.15.76:0/18183121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77f00ce240 0x7f77f00d1920 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009d90 tx=0x7f77e802bea0 comp rx=0 tx=0).stop 2023-11-09T03:48:40.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.811+0000 7f77f51f5640 1 -- 172.21.15.76:0/18183121 shutdown_connections 2023-11-09T03:48:40.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.811+0000 7f77f51f5640 1 --2- 172.21.15.76:0/18183121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77f00ce240 0x7f77f00d1920 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:40.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.811+0000 7f77f51f5640 1 --2- 172.21.15.76:0/18183121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77f00cde60 0x7f77f015d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:40.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.811+0000 7f77f51f5640 1 -- 172.21.15.76:0/18183121 >> 172.21.15.76:0/18183121 conn(0x7f77f0071f30 msgr2=0x7f77f00cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:40.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.811+0000 7f77f51f5640 1 -- 172.21.15.76:0/18183121 shutdown_connections 2023-11-09T03:48:40.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.811+0000 7f77f51f5640 1 -- 172.21.15.76:0/18183121 wait complete. 2023-11-09T03:48:40.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.812+0000 7f77f51f5640 1 Processor -- start 2023-11-09T03:48:40.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.813+0000 7f77f51f5640 1 -- start start 2023-11-09T03:48:40.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.813+0000 7f77f51f5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77f00cde60 0x7f77f00cd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:40.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.813+0000 7f77f51f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77f00ce240 0x7f77f0152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:40.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.813+0000 7f77f51f5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77f015daa0 0x7f77f01f1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:40.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.813+0000 7f77f51f5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f0153fc0 con 0x7f77f015daa0 2023-11-09T03:48:40.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.813+0000 7f77f51f5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f01529a0 con 0x7f77f00ce240 2023-11-09T03:48:40.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.813+0000 7f77f51f5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f0152c00 con 0x7f77f00cde60 2023-11-09T03:48:40.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.813+0000 7f77ef59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77f015daa0 0x7f77f01f1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:40.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.813+0000 7f77ef59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77f015daa0 0x7f77f01f1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49922/0 (socket says 172.21.15.76:49922) 2023-11-09T03:48:40.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.813+0000 7f77ef59e640 1 -- 172.21.15.76:0/480369896 learned_addr learned my addr 172.21.15.76:0/480369896 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:40.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.814+0000 7f77e7fff640 1 --2- 172.21.15.76:0/480369896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77f00ce240 0x7f77f0152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:40.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.814+0000 7f77eed9d640 1 --2- 172.21.15.76:0/480369896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77f00cde60 0x7f77f00cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:40.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.814+0000 7f77ef59e640 1 -- 172.21.15.76:0/480369896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77f00cde60 msgr2=0x7f77f00cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:40.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.814+0000 7f77ef59e640 1 --2- 172.21.15.76:0/480369896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77f00cde60 0x7f77f00cd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:40.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.814+0000 7f77ef59e640 1 -- 172.21.15.76:0/480369896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77f00ce240 msgr2=0x7f77f0152460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:40.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.814+0000 7f77ef59e640 1 --2- 172.21.15.76:0/480369896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77f00ce240 0x7f77f0152460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:40.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.814+0000 7f77ef59e640 1 -- 172.21.15.76:0/480369896 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e8009a70 con 0x7f77f015daa0 2023-11-09T03:48:40.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.814+0000 7f77ef59e640 1 --2- 172.21.15.76:0/480369896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77f015daa0 0x7f77f01f1d70 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009d90 tx=0x7f77e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:40.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.814+0000 7f77e77fe640 1 -- 172.21.15.76:0/480369896 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e8040650 con 0x7f77f015daa0 2023-11-09T03:48:40.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.814+0000 7f77e77fe640 1 -- 172.21.15.76:0/480369896 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e80407d0 con 0x7f77f015daa0 2023-11-09T03:48:40.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.814+0000 7f77f51f5640 1 -- 172.21.15.76:0/480369896 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f0152ee0 con 0x7f77f015daa0 2023-11-09T03:48:40.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.814+0000 7f77e77fe640 1 -- 172.21.15.76:0/480369896 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e8040b80 con 0x7f77f015daa0 2023-11-09T03:48:40.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.814+0000 7f77f51f5640 1 -- 172.21.15.76:0/480369896 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f01534a0 con 0x7f77f015daa0 2023-11-09T03:48:40.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.815+0000 7f77f51f5640 1 -- 172.21.15.76:0/480369896 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f0154100 con 0x7f77f015daa0 2023-11-09T03:48:40.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.815+0000 7f77e77fe640 1 -- 172.21.15.76:0/480369896 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f77e8040ce0 con 0x7f77f015daa0 2023-11-09T03:48:40.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.816+0000 7f77e77fe640 1 --2- 172.21.15.76:0/480369896 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f77d006ae70 0x7f77d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:40.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.816+0000 7f77e77fe640 1 -- 172.21.15.76:0/480369896 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f77e80ac910 con 0x7f77f015daa0 2023-11-09T03:48:40.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.816+0000 7f77eed9d640 1 --2- 172.21.15.76:0/480369896 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f77d006ae70 0x7f77d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:40.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.816+0000 7f77eed9d640 1 --2- 172.21.15.76:0/480369896 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f77d006ae70 0x7f77d006d330 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f77e000a970 tx=0x7f77e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:40.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:40.820+0000 7f77e77fe640 1 -- 172.21.15.76:0/480369896 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f77f0154100 con 0x7f77f015daa0 2023-11-09T03:48:41.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.018+0000 7f77f51f5640 1 -- 172.21.15.76:0/480369896 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77f0154100 con 0x7f77f015daa0 2023-11-09T03:48:41.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.018+0000 7f77e77fe640 1 -- 172.21.15.76:0/480369896 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f77f0154100 con 0x7f77f015daa0 2023-11-09T03:48:41.018 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:41.019 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":37,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6097,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2408659865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2408659865},{"type":"v1","addr":"172.21.15.113:6836","nonce":2408659865}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6100,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1873078383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1873078383},{"type":"v1","addr":"172.21.15.113:6837","nonce":1873078383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/3549714130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3549714130},{"type":"v1","addr":"172.21.15.76:6834","nonce":3549714130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:22.895004+0000","modified":"2023-11-09T03:48:37.005363+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6984":{"gid":6984,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/2165197913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":2165197913},{"type":"v1","addr":"172.21.15.76:6835","nonce":2165197913}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-09T03:48:41.019 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 37 2023-11-09T03:48:41.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.020+0000 7f77f51f5640 1 -- 172.21.15.76:0/480369896 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f77d006ae70 msgr2=0x7f77d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:41.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.020+0000 7f77f51f5640 1 --2- 172.21.15.76:0/480369896 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f77d006ae70 0x7f77d006d330 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f77e000a970 tx=0x7f77e0008040 comp rx=0 tx=0).stop 2023-11-09T03:48:41.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.020+0000 7f77f51f5640 1 -- 172.21.15.76:0/480369896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77f015daa0 msgr2=0x7f77f01f1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:41.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.020+0000 7f77f51f5640 1 --2- 172.21.15.76:0/480369896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77f015daa0 0x7f77f01f1d70 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009d90 tx=0x7f77e8004a40 comp rx=0 tx=0).stop 2023-11-09T03:48:41.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.020+0000 7f77f51f5640 1 -- 172.21.15.76:0/480369896 shutdown_connections 2023-11-09T03:48:41.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.020+0000 7f77f51f5640 1 --2- 172.21.15.76:0/480369896 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f77d006ae70 0x7f77d006d330 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.020+0000 7f77f51f5640 1 --2- 172.21.15.76:0/480369896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77f015daa0 0x7f77f01f1d70 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.020+0000 7f77f51f5640 1 --2- 172.21.15.76:0/480369896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77f00ce240 0x7f77f0152460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.020+0000 7f77f51f5640 1 --2- 172.21.15.76:0/480369896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77f00cde60 0x7f77f00cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.020+0000 7f77f51f5640 1 -- 172.21.15.76:0/480369896 >> 172.21.15.76:0/480369896 conn(0x7f77f0071f30 msgr2=0x7f77f00c9120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:41.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.020+0000 7f77f51f5640 1 -- 172.21.15.76:0/480369896 shutdown_connections 2023-11-09T03:48:41.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.020+0000 7f77f51f5640 1 -- 172.21.15.76:0/480369896 wait complete. 2023-11-09T03:48:41.030 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:48:41.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.131+0000 7f0a62776640 1 Processor -- start 2023-11-09T03:48:41.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.131+0000 7f0a62776640 1 -- start start 2023-11-09T03:48:41.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.132+0000 7f0a62776640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a5c0cda60 0x7f0a5c0ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:41.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.132+0000 7f0a62776640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0a5c0b0cd0 0x7f0a5c070630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:41.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.132+0000 7f0a62776640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0a5c0cace0 0x7f0a5c070b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:41.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.132+0000 7f0a62776640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a5c0cc990 con 0x7f0a5c0b0cd0 2023-11-09T03:48:41.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.132+0000 7f0a62776640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a5c0ccc70 con 0x7f0a5c0cace0 2023-11-09T03:48:41.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.132+0000 7f0a62776640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c0c9ce0 con 0x7f0a5c0cda60 2023-11-09T03:48:41.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.132+0000 7f0a5b7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0a5c0b0cd0 0x7f0a5c070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:41.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.132+0000 7f0a5b7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0a5c0b0cd0 0x7f0a5c070630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49938/0 (socket says 172.21.15.76:49938) 2023-11-09T03:48:41.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.132+0000 7f0a5b7fe640 1 -- 172.21.15.76:0/3166451022 learned_addr learned my addr 172.21.15.76:0/3166451022 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:41.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.132+0000 7f0a60d13640 1 --2- 172.21.15.76:0/3166451022 >> v2:172.21.15.113:3300/0 conn(0x7f0a5c0cace0 0x7f0a5c070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:41.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.132+0000 7f0a5bfff640 1 --2- 172.21.15.76:0/3166451022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a5c0cda60 0x7f0a5c0ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:41.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.133+0000 7f0a5b7fe640 1 -- 172.21.15.76:0/3166451022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a5c0cda60 msgr2=0x7f0a5c0ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:41.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.133+0000 7f0a5b7fe640 1 --2- 172.21.15.76:0/3166451022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a5c0cda60 0x7f0a5c0ca7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.133+0000 7f0a5b7fe640 1 -- 172.21.15.76:0/3166451022 >> v2:172.21.15.113:3300/0 conn(0x7f0a5c0cace0 msgr2=0x7f0a5c070b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:41.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.133+0000 7f0a5b7fe640 1 --2- 172.21.15.76:0/3166451022 >> v2:172.21.15.113:3300/0 conn(0x7f0a5c0cace0 0x7f0a5c070b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.133+0000 7f0a5b7fe640 1 -- 172.21.15.76:0/3166451022 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5c0c9fc0 con 0x7f0a5c0b0cd0 2023-11-09T03:48:41.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.133+0000 7f0a5b7fe640 1 --2- 172.21.15.76:0/3166451022 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a5c0b0cd0 0x7f0a5c070630 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c071ba0 tx=0x7f0a5002cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dedcbcfb6096f03 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:41.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.133+0000 7f0a5affd640 1 -- 172.21.15.76:0/3166451022 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5002d9e0 con 0x7f0a5c0b0cd0 2023-11-09T03:48:41.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.134+0000 7f0a5affd640 1 -- 172.21.15.76:0/3166451022 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a5002db40 con 0x7f0a5c0b0cd0 2023-11-09T03:48:41.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.134+0000 7f0a5affd640 1 -- 172.21.15.76:0/3166451022 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a50035440 con 0x7f0a5c0b0cd0 2023-11-09T03:48:41.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.134+0000 7f0a62776640 1 -- 172.21.15.76:0/3166451022 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a5c0b0cd0 msgr2=0x7f0a5c070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:41.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.134+0000 7f0a62776640 1 --2- 172.21.15.76:0/3166451022 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a5c0b0cd0 0x7f0a5c070630 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c071ba0 tx=0x7f0a5002cc30 comp rx=0 tx=0).stop 2023-11-09T03:48:41.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.134+0000 7f0a62776640 1 -- 172.21.15.76:0/3166451022 shutdown_connections 2023-11-09T03:48:41.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.134+0000 7f0a62776640 1 --2- 172.21.15.76:0/3166451022 >> v2:172.21.15.113:3300/0 conn(0x7f0a5c0cace0 0x7f0a5c070b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.134+0000 7f0a62776640 1 --2- 172.21.15.76:0/3166451022 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a5c0b0cd0 0x7f0a5c070630 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.134+0000 7f0a62776640 1 --2- 172.21.15.76:0/3166451022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a5c0cda60 0x7f0a5c0ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.134+0000 7f0a62776640 1 -- 172.21.15.76:0/3166451022 >> 172.21.15.76:0/3166451022 conn(0x7f0a5c0ce950 msgr2=0x7f0a5c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:41.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.134+0000 7f0a62776640 1 -- 172.21.15.76:0/3166451022 shutdown_connections 2023-11-09T03:48:41.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.134+0000 7f0a62776640 1 -- 172.21.15.76:0/3166451022 wait complete. 2023-11-09T03:48:41.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.135+0000 7f0a62776640 1 Processor -- start 2023-11-09T03:48:41.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.135+0000 7f0a62776640 1 -- start start 2023-11-09T03:48:41.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.136+0000 7f0a62776640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a5c0b0cd0 0x7f0a5c0c4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:41.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.136+0000 7f0a62776640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a5c0cace0 0x7f0a5c1588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:41.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.136+0000 7f0a62776640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a5c0cda60 0x7f0a5c158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:41.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.136+0000 7f0a62776640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c0c4980 con 0x7f0a5c0cace0 2023-11-09T03:48:41.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.136+0000 7f0a62776640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c0b23f0 con 0x7f0a5c0cda60 2023-11-09T03:48:41.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.136+0000 7f0a62776640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c0b2c90 con 0x7f0a5c0b0cd0 2023-11-09T03:48:41.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.136+0000 7f0a5b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a5c0cace0 0x7f0a5c1588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:41.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.136+0000 7f0a5b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a5c0cace0 0x7f0a5c1588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49946/0 (socket says 172.21.15.76:49946) 2023-11-09T03:48:41.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.136+0000 7f0a5b7fe640 1 -- 172.21.15.76:0/3539610687 learned_addr learned my addr 172.21.15.76:0/3539610687 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:41.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.136+0000 7f0a5b7fe640 1 -- 172.21.15.76:0/3539610687 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a5c0b0cd0 msgr2=0x7f0a5c0c4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:41.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.136+0000 7f0a5b7fe640 1 --2- 172.21.15.76:0/3539610687 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a5c0b0cd0 0x7f0a5c0c4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.136+0000 7f0a5b7fe640 1 -- 172.21.15.76:0/3539610687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a5c0cda60 msgr2=0x7f0a5c158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:41.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.136+0000 7f0a5b7fe640 1 --2- 172.21.15.76:0/3539610687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a5c0cda60 0x7f0a5c158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.137+0000 7f0a5b7fe640 1 -- 172.21.15.76:0/3539610687 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a500094d0 con 0x7f0a5c0cace0 2023-11-09T03:48:41.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.137+0000 7f0a5b7fe640 1 --2- 172.21.15.76:0/3539610687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a5c0cace0 0x7f0a5c1588a0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f0a5002cc70 tx=0x7f0a500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:41.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.137+0000 7f0a58ff9640 1 -- 172.21.15.76:0/3539610687 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5003f600 con 0x7f0a5c0cace0 2023-11-09T03:48:41.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.137+0000 7f0a58ff9640 1 -- 172.21.15.76:0/3539610687 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a5000c040 con 0x7f0a5c0cace0 2023-11-09T03:48:41.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.137+0000 7f0a62776640 1 -- 172.21.15.76:0/3539610687 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a5c0c87a0 con 0x7f0a5c0cace0 2023-11-09T03:48:41.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.137+0000 7f0a58ff9640 1 -- 172.21.15.76:0/3539610687 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5003f9c0 con 0x7f0a5c0cace0 2023-11-09T03:48:41.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.137+0000 7f0a62776640 1 -- 172.21.15.76:0/3539610687 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a5c0b2920 con 0x7f0a5c0cace0 2023-11-09T03:48:41.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.138+0000 7f0a62776640 1 -- 172.21.15.76:0/3539610687 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a5c074290 con 0x7f0a5c0cace0 2023-11-09T03:48:41.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.138+0000 7f0a58ff9640 1 -- 172.21.15.76:0/3539610687 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0a5003fb20 con 0x7f0a5c0cace0 2023-11-09T03:48:41.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.139+0000 7f0a58ff9640 1 --2- 172.21.15.76:0/3539610687 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a4806ae70 0x7f0a4806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:41.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.139+0000 7f0a58ff9640 1 -- 172.21.15.76:0/3539610687 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f0a500ac180 con 0x7f0a5c0cace0 2023-11-09T03:48:41.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.139+0000 7f0a5bfff640 1 --2- 172.21.15.76:0/3539610687 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a4806ae70 0x7f0a4806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:41.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.140+0000 7f0a5bfff640 1 --2- 172.21.15.76:0/3539610687 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a4806ae70 0x7f0a4806d330 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c006a90 tx=0x7f0a4c006a20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:41.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.143+0000 7f0a58ff9640 1 -- 172.21.15.76:0/3539610687 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0a50076d50 con 0x7f0a5c0cace0 2023-11-09T03:48:41.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.310+0000 7f0a62776640 1 -- 172.21.15.76:0/3539610687 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a5c0b2dd0 con 0x7f0a5c0cace0 2023-11-09T03:48:41.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.311+0000 7f0a58ff9640 1 -- 172.21.15.76:0/3539610687 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+21941 (secure 0 0 0) 0x7f0a5007a370 con 0x7f0a5c0cace0 2023-11-09T03:48:41.311 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:41.312 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":83,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:48:24.549937+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-09T03:47:59.280601+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-09T03:47:59.742039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:48:20.992533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-11-09T03:48:21.862178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:48:41.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.313+0000 7f0a62776640 1 -- 172.21.15.76:0/3539610687 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a4806ae70 msgr2=0x7f0a4806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:41.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.313+0000 7f0a62776640 1 --2- 172.21.15.76:0/3539610687 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a4806ae70 0x7f0a4806d330 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c006a90 tx=0x7f0a4c006a20 comp rx=0 tx=0).stop 2023-11-09T03:48:41.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.313+0000 7f0a62776640 1 -- 172.21.15.76:0/3539610687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a5c0cace0 msgr2=0x7f0a5c1588a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:41.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.313+0000 7f0a62776640 1 --2- 172.21.15.76:0/3539610687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a5c0cace0 0x7f0a5c1588a0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f0a5002cc70 tx=0x7f0a500046c0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.313+0000 7f0a62776640 1 -- 172.21.15.76:0/3539610687 shutdown_connections 2023-11-09T03:48:41.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.313+0000 7f0a62776640 1 --2- 172.21.15.76:0/3539610687 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a4806ae70 0x7f0a4806d330 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.313+0000 7f0a62776640 1 --2- 172.21.15.76:0/3539610687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a5c0cda60 0x7f0a5c158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.313+0000 7f0a62776640 1 --2- 172.21.15.76:0/3539610687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a5c0cace0 0x7f0a5c1588a0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.313+0000 7f0a62776640 1 --2- 172.21.15.76:0/3539610687 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a5c0b0cd0 0x7f0a5c0c4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.313+0000 7f0a62776640 1 -- 172.21.15.76:0/3539610687 >> 172.21.15.76:0/3539610687 conn(0x7f0a5c0ce950 msgr2=0x7f0a5c06d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:41.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.314+0000 7f0a62776640 1 -- 172.21.15.76:0/3539610687 shutdown_connections 2023-11-09T03:48:41.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.314+0000 7f0a62776640 1 -- 172.21.15.76:0/3539610687 wait complete. 2023-11-09T03:48:41.324 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-09T03:47:00.036540+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_data_pool', 'create_time': '2023-11-09T03:47:59.280601+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 70, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-09T03:47:59.742039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:48:20.992533+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:48:21.862178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-09T03:48:41.325 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:48:41.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.429+0000 7f1be6861640 1 Processor -- start 2023-11-09T03:48:41.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.429+0000 7f1be6861640 1 -- start start 2023-11-09T03:48:41.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.430+0000 7f1be6861640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1be00cae30 0x7f1be00cb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:41.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.430+0000 7f1be6861640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1be0065df0 0x7f1be006b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:41.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.430+0000 7f1be6861640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1be006abf0 con 0x7f1be0159e30 2023-11-09T03:48:41.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.430+0000 7f1be6861640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1be00cd180 con 0x7f1be00cae30 2023-11-09T03:48:41.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.430+0000 7f1be6861640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be00cba80 con 0x7f1be0065df0 2023-11-09T03:48:41.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.430+0000 7f1be4dfe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1be0065df0 0x7f1be006b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:41.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.430+0000 7f1bdffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1be00cae30 0x7f1be00cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:41.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.430+0000 7f1bdf7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1be0159e30 0x7f1be00c2d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:48370/0 (socket says 172.21.15.76:48370) 2023-11-09T03:48:41.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.430+0000 7f1bdf7fe640 1 -- 172.21.15.76:0/264499245 learned_addr learned my addr 172.21.15.76:0/264499245 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:41.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.431+0000 7f1bdffff640 1 -- 172.21.15.76:0/264499245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1be0065df0 msgr2=0x7f1be006b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:41.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.431+0000 7f1bdffff640 1 --2- 172.21.15.76:0/264499245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1be0065df0 0x7f1be006b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.431+0000 7f1bdffff640 1 -- 172.21.15.76:0/264499245 >> v1:172.21.15.113:6789/0 conn(0x7f1be0159e30 legacy=0x7f1be00c2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:41.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.431+0000 7f1bdffff640 1 -- 172.21.15.76:0/264499245 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be00c3470 con 0x7f1be00cae30 2023-11-09T03:48:41.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.431+0000 7f1be4dfe640 1 --2- 172.21.15.76:0/264499245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1be0065df0 0x7f1be006b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:48:41.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.431+0000 7f1bdffff640 1 --2- 172.21.15.76:0/264499245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1be00cae30 0x7f1be00cb210 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00097a0 tx=0x7f1bd002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e4ea1297f3439d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:41.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.432+0000 7f1bdeffd640 1 -- 172.21.15.76:0/264499245 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0031830 con 0x7f1be00cae30 2023-11-09T03:48:41.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.432+0000 7f1bdeffd640 1 -- 172.21.15.76:0/264499245 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd0031990 con 0x7f1be00cae30 2023-11-09T03:48:41.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.432+0000 7f1bdeffd640 1 -- 172.21.15.76:0/264499245 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0031d20 con 0x7f1be00cae30 2023-11-09T03:48:41.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.432+0000 7f1be6861640 1 -- 172.21.15.76:0/264499245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1be00cae30 msgr2=0x7f1be00cb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:41.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.432+0000 7f1be6861640 1 --2- 172.21.15.76:0/264499245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1be00cae30 0x7f1be00cb210 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00097a0 tx=0x7f1bd002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.432+0000 7f1be6861640 1 -- 172.21.15.76:0/264499245 shutdown_connections 2023-11-09T03:48:41.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.432+0000 7f1be6861640 1 --2- 172.21.15.76:0/264499245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1be0065df0 0x7f1be006b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.432+0000 7f1be6861640 1 --2- 172.21.15.76:0/264499245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1be00cae30 0x7f1be00cb210 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.432+0000 7f1be6861640 1 -- 172.21.15.76:0/264499245 >> 172.21.15.76:0/264499245 conn(0x7f1be00cbc90 msgr2=0x7f1be00cc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:41.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.432+0000 7f1be6861640 1 -- 172.21.15.76:0/264499245 shutdown_connections 2023-11-09T03:48:41.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.432+0000 7f1be6861640 1 -- 172.21.15.76:0/264499245 wait complete. 2023-11-09T03:48:41.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.433+0000 7f1be6861640 1 Processor -- start 2023-11-09T03:48:41.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.434+0000 7f1be6861640 1 -- start start 2023-11-09T03:48:41.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.434+0000 7f1be6861640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1be0065df0 0x7f1be01485d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:41.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.434+0000 7f1be6861640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1be00cae30 0x7f1be0158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:41.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.434+0000 7f1be6861640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1be0159e30 0x7f1be0158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:41.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.434+0000 7f1be6861640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be00c8c00 con 0x7f1be0065df0 2023-11-09T03:48:41.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.434+0000 7f1be6861640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be00b3220 con 0x7f1be0159e30 2023-11-09T03:48:41.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.434+0000 7f1be6861640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be00c7f80 con 0x7f1be00cae30 2023-11-09T03:48:41.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.434+0000 7f1bdffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1be0065df0 0x7f1be01485d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:41.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.434+0000 7f1bdffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1be0065df0 0x7f1be01485d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49958/0 (socket says 172.21.15.76:49958) 2023-11-09T03:48:41.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.434+0000 7f1bdffff640 1 -- 172.21.15.76:0/940855833 learned_addr learned my addr 172.21.15.76:0/940855833 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:41.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.435+0000 7f1bdf7fe640 1 --2- 172.21.15.76:0/940855833 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1be00cae30 0x7f1be0158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:41.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.435+0000 7f1bdffff640 1 -- 172.21.15.76:0/940855833 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1be00cae30 msgr2=0x7f1be0158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:41.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.435+0000 7f1bdffff640 1 --2- 172.21.15.76:0/940855833 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1be00cae30 0x7f1be0158920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.435+0000 7f1be4dfe640 1 --2- 172.21.15.76:0/940855833 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1be0159e30 0x7f1be0158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:41.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.435+0000 7f1bdffff640 1 -- 172.21.15.76:0/940855833 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1be0159e30 msgr2=0x7f1be0158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:41.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.435+0000 7f1bdffff640 1 --2- 172.21.15.76:0/940855833 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1be0159e30 0x7f1be0158e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.435+0000 7f1bdffff640 1 -- 172.21.15.76:0/940855833 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd8009a70 con 0x7f1be0065df0 2023-11-09T03:48:41.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.435+0000 7f1bdffff640 1 --2- 172.21.15.76:0/940855833 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1be0065df0 0x7f1be01485d0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f1bd002bee0 tx=0x7f1bd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:41.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.435+0000 7f1bdcff9640 1 -- 172.21.15.76:0/940855833 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0040070 con 0x7f1be0065df0 2023-11-09T03:48:41.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.435+0000 7f1bdcff9640 1 -- 172.21.15.76:0/940855833 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd003b890 con 0x7f1be0065df0 2023-11-09T03:48:41.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.435+0000 7f1bdcff9640 1 -- 172.21.15.76:0/940855833 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd003bc20 con 0x7f1be0065df0 2023-11-09T03:48:41.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.435+0000 7f1be6861640 1 -- 172.21.15.76:0/940855833 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bd0009400 con 0x7f1be0065df0 2023-11-09T03:48:41.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.435+0000 7f1be6861640 1 -- 172.21.15.76:0/940855833 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be00c8540 con 0x7f1be0065df0 2023-11-09T03:48:41.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.436+0000 7f1be6861640 1 -- 172.21.15.76:0/940855833 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1be00c8d40 con 0x7f1be0065df0 2023-11-09T03:48:41.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.436+0000 7f1bdcff9640 1 -- 172.21.15.76:0/940855833 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1bd004d400 con 0x7f1be0065df0 2023-11-09T03:48:41.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.437+0000 7f1bdcff9640 1 --2- 172.21.15.76:0/940855833 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1bcc06ae40 0x7f1bcc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:41.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.437+0000 7f1bdcff9640 1 -- 172.21.15.76:0/940855833 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f1bd00adad0 con 0x7f1be0065df0 2023-11-09T03:48:41.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.437+0000 7f1bdf7fe640 1 --2- 172.21.15.76:0/940855833 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1bcc06ae40 0x7f1bcc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:41.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.438+0000 7f1bdf7fe640 1 --2- 172.21.15.76:0/940855833 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1bcc06ae40 0x7f1bcc06d300 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1be00ca820 tx=0x7f1bd4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:41.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.441+0000 7f1bdcff9640 1 -- 172.21.15.76:0/940855833 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1be00c8d40 con 0x7f1be0065df0 2023-11-09T03:48:41.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.609+0000 7f1be6861640 1 -- 172.21.15.76:0/940855833 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1be00c8d40 con 0x7f1be0065df0 2023-11-09T03:48:41.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.609+0000 7f1bdcff9640 1 -- 172.21.15.76:0/940855833 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1be00c8d40 con 0x7f1be0065df0 2023-11-09T03:48:41.609 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:48:41.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.611+0000 7f1be6861640 1 -- 172.21.15.76:0/940855833 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1bcc06ae40 msgr2=0x7f1bcc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:41.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.611+0000 7f1be6861640 1 --2- 172.21.15.76:0/940855833 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1bcc06ae40 0x7f1bcc06d300 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1be00ca820 tx=0x7f1bd4009040 comp rx=0 tx=0).stop 2023-11-09T03:48:41.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.611+0000 7f1be6861640 1 -- 172.21.15.76:0/940855833 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1be0065df0 msgr2=0x7f1be01485d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:41.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.611+0000 7f1be6861640 1 --2- 172.21.15.76:0/940855833 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1be0065df0 0x7f1be01485d0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f1bd002bee0 tx=0x7f1bd00046c0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.611+0000 7f1be6861640 1 -- 172.21.15.76:0/940855833 shutdown_connections 2023-11-09T03:48:41.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.611+0000 7f1be6861640 1 --2- 172.21.15.76:0/940855833 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1bcc06ae40 0x7f1bcc06d300 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.611+0000 7f1be6861640 1 --2- 172.21.15.76:0/940855833 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1be0159e30 0x7f1be0158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.611+0000 7f1be6861640 1 --2- 172.21.15.76:0/940855833 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1be00cae30 0x7f1be0158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.611+0000 7f1be6861640 1 --2- 172.21.15.76:0/940855833 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1be0065df0 0x7f1be01485d0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.611+0000 7f1be6861640 1 -- 172.21.15.76:0/940855833 >> 172.21.15.76:0/940855833 conn(0x7f1be00cbc90 msgr2=0x7f1be006b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:41.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.611+0000 7f1be6861640 1 -- 172.21.15.76:0/940855833 shutdown_connections 2023-11-09T03:48:41.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.611+0000 7f1be6861640 1 -- 172.21.15.76:0/940855833 wait complete. 2023-11-09T03:48:41.621 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-09T03:48:41.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.724+0000 7f71237cc640 1 Processor -- start 2023-11-09T03:48:41.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.725+0000 7f71237cc640 1 -- start start 2023-11-09T03:48:41.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.725+0000 7f71237cc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f711c0cc520 0x7f711c070fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:41.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.725+0000 7f71237cc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f711c0cc9d0 0x7f711c0714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:41.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.725+0000 7f71237cc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f711c0c3860 con 0x7f711c153850 2023-11-09T03:48:41.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.725+0000 7f71237cc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f711c0c9250 con 0x7f711c0cc520 2023-11-09T03:48:41.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.725+0000 7f71237cc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f711c153c30 con 0x7f711c0cc9d0 2023-11-09T03:48:41.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.725+0000 7f7120d67640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f711c0cc520 0x7f711c070fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:41.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.726+0000 7f7120d67640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f711c0cc520 0x7f711c070fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49972/0 (socket says 172.21.15.76:49972) 2023-11-09T03:48:41.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.726+0000 7f7120d67640 1 -- 172.21.15.76:0/1370582429 learned_addr learned my addr 172.21.15.76:0/1370582429 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:41.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.726+0000 7f7121d69640 1 --2- 172.21.15.76:0/1370582429 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f711c0cc9d0 0x7f711c0714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:41.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.726+0000 7f7120d67640 1 -- 172.21.15.76:0/1370582429 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f711c0cc9d0 msgr2=0x7f711c0714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:41.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.726+0000 7f7120d67640 1 --2- 172.21.15.76:0/1370582429 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f711c0cc9d0 0x7f711c0714e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.726+0000 7f7120d67640 1 -- 172.21.15.76:0/1370582429 >> v1:172.21.15.113:6789/0 conn(0x7f711c153850 legacy=0x7f711c15a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:41.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.726+0000 7f7120d67640 1 -- 172.21.15.76:0/1370582429 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711c0ce590 con 0x7f711c0cc520 2023-11-09T03:48:41.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.726+0000 7f7121d69640 1 --2- 172.21.15.76:0/1370582429 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f711c0cc9d0 0x7f711c0714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:41.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.727+0000 7f7120d67640 1 --2- 172.21.15.76:0/1370582429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f711c0cc520 0x7f711c070fa0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f7118009870 tx=0x7f711802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d840da8c3298806d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:41.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.727+0000 7f7113fff640 1 -- 172.21.15.76:0/1370582429 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7118030840 con 0x7f711c0cc520 2023-11-09T03:48:41.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.727+0000 7f7113fff640 1 -- 172.21.15.76:0/1370582429 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71180309a0 con 0x7f711c0cc520 2023-11-09T03:48:41.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.727+0000 7f7113fff640 1 -- 172.21.15.76:0/1370582429 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7118030d30 con 0x7f711c0cc520 2023-11-09T03:48:41.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.727+0000 7f71237cc640 1 -- 172.21.15.76:0/1370582429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f711c0cc520 msgr2=0x7f711c070fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:41.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.727+0000 7f71237cc640 1 --2- 172.21.15.76:0/1370582429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f711c0cc520 0x7f711c070fa0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f7118009870 tx=0x7f711802b980 comp rx=0 tx=0).stop 2023-11-09T03:48:41.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.727+0000 7f71237cc640 1 -- 172.21.15.76:0/1370582429 shutdown_connections 2023-11-09T03:48:41.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.727+0000 7f71237cc640 1 --2- 172.21.15.76:0/1370582429 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f711c0cc9d0 0x7f711c0714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.727+0000 7f71237cc640 1 --2- 172.21.15.76:0/1370582429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f711c0cc520 0x7f711c070fa0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.727+0000 7f71237cc640 1 -- 172.21.15.76:0/1370582429 >> 172.21.15.76:0/1370582429 conn(0x7f711c0ce950 msgr2=0x7f711c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:41.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.727+0000 7f71237cc640 1 -- 172.21.15.76:0/1370582429 shutdown_connections 2023-11-09T03:48:41.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.727+0000 7f71237cc640 1 -- 172.21.15.76:0/1370582429 wait complete. 2023-11-09T03:48:41.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.729+0000 7f71237cc640 1 Processor -- start 2023-11-09T03:48:41.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.729+0000 7f71237cc640 1 -- start start 2023-11-09T03:48:41.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.729+0000 7f71237cc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f711c0cc520 0x7f711c155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:41.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.729+0000 7f71237cc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f711c0cc9d0 0x7f711c1479f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:41.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.729+0000 7f71237cc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f711c153850 0x7f711c144dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:41.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.729+0000 7f71237cc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711c156410 con 0x7f711c0cc9d0 2023-11-09T03:48:41.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.729+0000 7f71237cc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711c147f30 con 0x7f711c0cc520 2023-11-09T03:48:41.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.729+0000 7f71237cc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f711c156130 con 0x7f711c153850 2023-11-09T03:48:41.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.729+0000 7f7120d67640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f711c0cc9d0 0x7f711c1479f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:41.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.729+0000 7f7120d67640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f711c0cc9d0 0x7f711c1479f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49988/0 (socket says 172.21.15.76:49988) 2023-11-09T03:48:41.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.729+0000 7f7120d67640 1 -- 172.21.15.76:0/1901058275 learned_addr learned my addr 172.21.15.76:0/1901058275 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:41.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.730+0000 7f7120d67640 1 -- 172.21.15.76:0/1901058275 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f711c153850 msgr2=0x7f711c144dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:41.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.730+0000 7f7120d67640 1 --2- 172.21.15.76:0/1901058275 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f711c153850 0x7f711c144dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.730+0000 7f7120d67640 1 -- 172.21.15.76:0/1901058275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f711c0cc520 msgr2=0x7f711c155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:41.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.730+0000 7f7120d67640 1 --2- 172.21.15.76:0/1901058275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f711c0cc520 0x7f711c155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.730+0000 7f7120d67640 1 -- 172.21.15.76:0/1901058275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71180094d0 con 0x7f711c0cc9d0 2023-11-09T03:48:41.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.730+0000 7f7120d67640 1 --2- 172.21.15.76:0/1901058275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f711c0cc9d0 0x7f711c1479f0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f7118000c00 tx=0x7f7118004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:41.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.730+0000 7f7111ffb640 1 -- 172.21.15.76:0/1901058275 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7118030810 con 0x7f711c0cc9d0 2023-11-09T03:48:41.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.730+0000 7f7111ffb640 1 -- 172.21.15.76:0/1901058275 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711802f5a0 con 0x7f711c0cc9d0 2023-11-09T03:48:41.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.730+0000 7f71237cc640 1 -- 172.21.15.76:0/1901058275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f711c1493d0 con 0x7f711c0cc9d0 2023-11-09T03:48:41.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.730+0000 7f7111ffb640 1 -- 172.21.15.76:0/1901058275 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711802f970 con 0x7f711c0cc9d0 2023-11-09T03:48:41.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.730+0000 7f71237cc640 1 -- 172.21.15.76:0/1901058275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f711c149990 con 0x7f711c0cc9d0 2023-11-09T03:48:41.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.731+0000 7f71237cc640 1 -- 172.21.15.76:0/1901058275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f711c149bb0 con 0x7f711c0cc9d0 2023-11-09T03:48:41.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.732+0000 7f7111ffb640 1 -- 172.21.15.76:0/1901058275 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f711802fad0 con 0x7f711c0cc9d0 2023-11-09T03:48:41.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.732+0000 7f7111ffb640 1 --2- 172.21.15.76:0/1901058275 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f710006aea0 0x7f710006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:41.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.732+0000 7f7111ffb640 1 -- 172.21.15.76:0/1901058275 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f71180ae500 con 0x7f711c0cc9d0 2023-11-09T03:48:41.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.732+0000 7f7121568640 1 --2- 172.21.15.76:0/1901058275 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f710006aea0 0x7f710006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:41.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.733+0000 7f7121568640 1 --2- 172.21.15.76:0/1901058275 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f710006aea0 0x7f710006d360 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f710c000f60 tx=0x7f710c0079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:41.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.736+0000 7f7111ffb640 1 -- 172.21.15.76:0/1901058275 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f71180791f0 con 0x7f711c0cc9d0 2023-11-09T03:48:41.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.900+0000 7f71237cc640 1 -- 172.21.15.76:0/1901058275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f711c074290 con 0x7f711c0cc9d0 2023-11-09T03:48:41.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.901+0000 7f7111ffb640 1 -- 172.21.15.76:0/1901058275 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v83) v1 ==== 103+0+11 (secure 0 0 0) 0x7f711807c810 con 0x7f711c0cc9d0 2023-11-09T03:48:41.900 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:48:41.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.903+0000 7f71237cc640 1 -- 172.21.15.76:0/1901058275 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f710006aea0 msgr2=0x7f710006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:41.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.903+0000 7f71237cc640 1 --2- 172.21.15.76:0/1901058275 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f710006aea0 0x7f710006d360 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f710c000f60 tx=0x7f710c0079b0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.903+0000 7f71237cc640 1 -- 172.21.15.76:0/1901058275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f711c0cc9d0 msgr2=0x7f711c1479f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:41.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.903+0000 7f71237cc640 1 --2- 172.21.15.76:0/1901058275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f711c0cc9d0 0x7f711c1479f0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f7118000c00 tx=0x7f7118004730 comp rx=0 tx=0).stop 2023-11-09T03:48:41.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.903+0000 7f71237cc640 1 -- 172.21.15.76:0/1901058275 shutdown_connections 2023-11-09T03:48:41.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.903+0000 7f71237cc640 1 --2- 172.21.15.76:0/1901058275 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f710006aea0 0x7f710006d360 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.903+0000 7f71237cc640 1 --2- 172.21.15.76:0/1901058275 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f711c153850 0x7f711c144dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.903+0000 7f71237cc640 1 --2- 172.21.15.76:0/1901058275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f711c0cc9d0 0x7f711c1479f0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.903+0000 7f71237cc640 1 --2- 172.21.15.76:0/1901058275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f711c0cc520 0x7f711c155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:41.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.903+0000 7f71237cc640 1 -- 172.21.15.76:0/1901058275 >> 172.21.15.76:0/1901058275 conn(0x7f711c0ce950 msgr2=0x7f711c06d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:41.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.903+0000 7f71237cc640 1 -- 172.21.15.76:0/1901058275 shutdown_connections 2023-11-09T03:48:41.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:41.903+0000 7f71237cc640 1 -- 172.21.15.76:0/1901058275 wait complete. 2023-11-09T03:48:41.913 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-09T03:48:42.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.017+0000 7fa2362b4640 1 Processor -- start 2023-11-09T03:48:42.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.017+0000 7fa2362b4640 1 -- start start 2023-11-09T03:48:42.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.018+0000 7fa2362b4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa230159c00 0x7fa2300cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.018+0000 7fa2362b4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa2300c48e0 0x7fa2300c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.018+0000 7fa2362b4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2300adac0 0x7fa2300adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.018+0000 7fa2362b4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2300c52d0 con 0x7fa2300c48e0 2023-11-09T03:48:42.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.018+0000 7fa2362b4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa23015aff0 con 0x7fa230159c00 2023-11-09T03:48:42.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.018+0000 7fa2362b4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23015a3e0 con 0x7fa2300adac0 2023-11-09T03:48:42.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.018+0000 7fa22f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa2300c48e0 0x7fa2300c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:42.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.018+0000 7fa22f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa2300c48e0 0x7fa2300c4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50000/0 (socket says 172.21.15.76:50000) 2023-11-09T03:48:42.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.018+0000 7fa22f7fe640 1 -- 172.21.15.76:0/367806612 learned_addr learned my addr 172.21.15.76:0/367806612 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:42.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.018+0000 7fa22ffff640 1 --2- 172.21.15.76:0/367806612 >> v2:172.21.15.113:3300/0 conn(0x7fa230159c00 0x7fa2300cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:42.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.018+0000 7fa234851640 1 --2- 172.21.15.76:0/367806612 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2300adac0 0x7fa2300adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:42.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.018+0000 7fa22f7fe640 1 -- 172.21.15.76:0/367806612 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2300adac0 msgr2=0x7fa2300adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:42.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.018+0000 7fa22f7fe640 1 --2- 172.21.15.76:0/367806612 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2300adac0 0x7fa2300adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.018+0000 7fa22f7fe640 1 -- 172.21.15.76:0/367806612 >> v2:172.21.15.113:3300/0 conn(0x7fa230159c00 msgr2=0x7fa2300cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:42.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.018+0000 7fa22f7fe640 1 --2- 172.21.15.76:0/367806612 >> v2:172.21.15.113:3300/0 conn(0x7fa230159c00 0x7fa2300cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.018+0000 7fa22f7fe640 1 -- 172.21.15.76:0/367806612 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23015a6c0 con 0x7fa2300c48e0 2023-11-09T03:48:42.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.019+0000 7fa22f7fe640 1 --2- 172.21.15.76:0/367806612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2300c48e0 0x7fa2300c4d90 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fa2300cc4d0 tx=0x7fa22402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7155cc6dabd702 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:42.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.019+0000 7fa22effd640 1 -- 172.21.15.76:0/367806612 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224031870 con 0x7fa2300c48e0 2023-11-09T03:48:42.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.019+0000 7fa22effd640 1 -- 172.21.15.76:0/367806612 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2240319d0 con 0x7fa2300c48e0 2023-11-09T03:48:42.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.019+0000 7fa22effd640 1 -- 172.21.15.76:0/367806612 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224031d60 con 0x7fa2300c48e0 2023-11-09T03:48:42.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.020+0000 7fa2362b4640 1 -- 172.21.15.76:0/367806612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2300c48e0 msgr2=0x7fa2300c4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:42.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.020+0000 7fa2362b4640 1 --2- 172.21.15.76:0/367806612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2300c48e0 0x7fa2300c4d90 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fa2300cc4d0 tx=0x7fa22402b910 comp rx=0 tx=0).stop 2023-11-09T03:48:42.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.020+0000 7fa2362b4640 1 -- 172.21.15.76:0/367806612 shutdown_connections 2023-11-09T03:48:42.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.020+0000 7fa2362b4640 1 --2- 172.21.15.76:0/367806612 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2300adac0 0x7fa2300adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.020+0000 7fa2362b4640 1 --2- 172.21.15.76:0/367806612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2300c48e0 0x7fa2300c4d90 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.020+0000 7fa2362b4640 1 --2- 172.21.15.76:0/367806612 >> v2:172.21.15.113:3300/0 conn(0x7fa230159c00 0x7fa2300cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.020+0000 7fa2362b4640 1 -- 172.21.15.76:0/367806612 >> 172.21.15.76:0/367806612 conn(0x7fa2300ce950 msgr2=0x7fa2300ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:42.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.020+0000 7fa2362b4640 1 -- 172.21.15.76:0/367806612 shutdown_connections 2023-11-09T03:48:42.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.020+0000 7fa2362b4640 1 -- 172.21.15.76:0/367806612 wait complete. 2023-11-09T03:48:42.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.021+0000 7fa2362b4640 1 Processor -- start 2023-11-09T03:48:42.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.021+0000 7fa2362b4640 1 -- start start 2023-11-09T03:48:42.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.022+0000 7fa2362b4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2300adac0 0x7fa23013b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.022+0000 7fa2362b4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2300c48e0 0x7fa23014e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.022+0000 7fa2362b4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa230159c00 0x7fa23014eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.022+0000 7fa2362b4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23014d6c0 con 0x7fa2300adac0 2023-11-09T03:48:42.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.022+0000 7fa2362b4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23014ca40 con 0x7fa2300c48e0 2023-11-09T03:48:42.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.022+0000 7fa2362b4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23014cd20 con 0x7fa230159c00 2023-11-09T03:48:42.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.022+0000 7fa22ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2300adac0 0x7fa23013b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:42.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.022+0000 7fa22ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2300adac0 0x7fa23013b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50008/0 (socket says 172.21.15.76:50008) 2023-11-09T03:48:42.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.022+0000 7fa22ffff640 1 -- 172.21.15.76:0/1098728092 learned_addr learned my addr 172.21.15.76:0/1098728092 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:42.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.022+0000 7fa22f7fe640 1 --2- 172.21.15.76:0/1098728092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2300c48e0 0x7fa23014e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:42.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.022+0000 7fa22ffff640 1 -- 172.21.15.76:0/1098728092 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa230159c00 msgr2=0x7fa23014eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:42.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.022+0000 7fa234851640 1 --2- 172.21.15.76:0/1098728092 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa230159c00 0x7fa23014eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:42.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.022+0000 7fa22ffff640 1 --2- 172.21.15.76:0/1098728092 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa230159c00 0x7fa23014eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.022+0000 7fa22ffff640 1 -- 172.21.15.76:0/1098728092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2300c48e0 msgr2=0x7fa23014e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:42.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.022+0000 7fa22ffff640 1 --2- 172.21.15.76:0/1098728092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2300c48e0 0x7fa23014e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.022+0000 7fa22ffff640 1 -- 172.21.15.76:0/1098728092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2240094d0 con 0x7fa2300adac0 2023-11-09T03:48:42.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.022+0000 7fa22f7fe640 1 --2- 172.21.15.76:0/1098728092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2300c48e0 0x7fa23014e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:42.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.022+0000 7fa234851640 1 --2- 172.21.15.76:0/1098728092 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa230159c00 0x7fa23014eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:42.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.022+0000 7fa22ffff640 1 --2- 172.21.15.76:0/1098728092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2300adac0 0x7fa23013b210 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fa22000d5f0 tx=0x7fa22000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:42.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.022+0000 7fa22cff9640 1 -- 172.21.15.76:0/1098728092 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2200125f0 con 0x7fa2300adac0 2023-11-09T03:48:42.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.022+0000 7fa22cff9640 1 -- 172.21.15.76:0/1098728092 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa220012750 con 0x7fa2300adac0 2023-11-09T03:48:42.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.022+0000 7fa2362b4640 1 -- 172.21.15.76:0/1098728092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa23014d000 con 0x7fa2300adac0 2023-11-09T03:48:42.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.022+0000 7fa22cff9640 1 -- 172.21.15.76:0/1098728092 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa220012b00 con 0x7fa2300adac0 2023-11-09T03:48:42.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.022+0000 7fa2362b4640 1 -- 172.21.15.76:0/1098728092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa23013b750 con 0x7fa2300adac0 2023-11-09T03:48:42.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.023+0000 7fa2362b4640 1 -- 172.21.15.76:0/1098728092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2300719c0 con 0x7fa2300adac0 2023-11-09T03:48:42.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.024+0000 7fa22cff9640 1 -- 172.21.15.76:0/1098728092 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa220012c60 con 0x7fa2300adac0 2023-11-09T03:48:42.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.024+0000 7fa22cff9640 1 --2- 172.21.15.76:0/1098728092 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa21c06ae70 0x7fa21c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.024+0000 7fa22cff9640 1 -- 172.21.15.76:0/1098728092 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7fa22008a050 con 0x7fa2300adac0 2023-11-09T03:48:42.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.024+0000 7fa22f7fe640 1 --2- 172.21.15.76:0/1098728092 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa21c06ae70 0x7fa21c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:42.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.025+0000 7fa22f7fe640 1 --2- 172.21.15.76:0/1098728092 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa21c06ae70 0x7fa21c06d330 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa224036000 tx=0x7fa22403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:42.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.028+0000 7fa22cff9640 1 -- 172.21.15.76:0/1098728092 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa220054b90 con 0x7fa2300adac0 2023-11-09T03:48:42.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.192+0000 7fa2362b4640 1 -- 172.21.15.76:0/1098728092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7fa230074290 con 0x7fa2300adac0 2023-11-09T03:48:42.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.193+0000 7fa22cff9640 1 -- 172.21.15.76:0/1098728092 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v83) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa2200581b0 con 0x7fa2300adac0 2023-11-09T03:48:42.192 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:48:42.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.195+0000 7fa2362b4640 1 -- 172.21.15.76:0/1098728092 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa21c06ae70 msgr2=0x7fa21c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:42.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.195+0000 7fa2362b4640 1 --2- 172.21.15.76:0/1098728092 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa21c06ae70 0x7fa21c06d330 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa224036000 tx=0x7fa22403c040 comp rx=0 tx=0).stop 2023-11-09T03:48:42.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.195+0000 7fa2362b4640 1 -- 172.21.15.76:0/1098728092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2300adac0 msgr2=0x7fa23013b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:42.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.195+0000 7fa2362b4640 1 --2- 172.21.15.76:0/1098728092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2300adac0 0x7fa23013b210 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fa22000d5f0 tx=0x7fa22000dab0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.195+0000 7fa2362b4640 1 -- 172.21.15.76:0/1098728092 shutdown_connections 2023-11-09T03:48:42.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.195+0000 7fa2362b4640 1 --2- 172.21.15.76:0/1098728092 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa21c06ae70 0x7fa21c06d330 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.195+0000 7fa2362b4640 1 --2- 172.21.15.76:0/1098728092 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa230159c00 0x7fa23014eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.195+0000 7fa2362b4640 1 --2- 172.21.15.76:0/1098728092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2300c48e0 0x7fa23014e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.195+0000 7fa2362b4640 1 --2- 172.21.15.76:0/1098728092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2300adac0 0x7fa23013b210 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.195+0000 7fa2362b4640 1 -- 172.21.15.76:0/1098728092 >> 172.21.15.76:0/1098728092 conn(0x7fa2300ce950 msgr2=0x7fa2300cb340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:42.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.195+0000 7fa2362b4640 1 -- 172.21.15.76:0/1098728092 shutdown_connections 2023-11-09T03:48:42.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.195+0000 7fa2362b4640 1 -- 172.21.15.76:0/1098728092 wait complete. 2023-11-09T03:48:42.205 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-09T03:48:42.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.309+0000 7f986dad4640 1 Processor -- start 2023-11-09T03:48:42.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.309+0000 7f986dad4640 1 -- start start 2023-11-09T03:48:42.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.310+0000 7f986dad4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9868153e60 0x7f986815dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.310+0000 7f986dad4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98680cde60 con 0x7f98680b4880 2023-11-09T03:48:42.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.310+0000 7f986dad4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9868160f90 con 0x7f98680cd6d0 2023-11-09T03:48:42.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.310+0000 7f986dad4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f986815e400 con 0x7f9868153e60 2023-11-09T03:48:42.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.310+0000 7f9867fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f98680b4880 0x7f98680d0fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:49360/0 (socket says 172.21.15.76:49360) 2023-11-09T03:48:42.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.310+0000 7f9867fff640 1 -- 172.21.15.76:0/2257810979 learned_addr learned my addr 172.21.15.76:0/2257810979 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:42.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.311+0000 7f98677fe640 1 --2- 172.21.15.76:0/2257810979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9868153e60 0x7f986815dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:42.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.311+0000 7f9866ffd640 1 -- 172.21.15.76:0/2257810979 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2648507646 0 0) 0x7f98680cde60 con 0x7f98680b4880 2023-11-09T03:48:42.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.311+0000 7f9866ffd640 1 -- 172.21.15.76:0/2257810979 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9854003520 con 0x7f98680b4880 2023-11-09T03:48:42.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.311+0000 7f9866ffd640 1 -- 172.21.15.76:0/2257810979 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2925974054 0 0) 0x7f9854003520 con 0x7f98680b4880 2023-11-09T03:48:42.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.311+0000 7f9866ffd640 1 -- 172.21.15.76:0/2257810979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9868153e60 msgr2=0x7f986815dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:42.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.311+0000 7f9866ffd640 1 --2- 172.21.15.76:0/2257810979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9868153e60 0x7f986815dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.311+0000 7f9866ffd640 1 -- 172.21.15.76:0/2257810979 >> v1:172.21.15.113:6789/0 conn(0x7f98680cd6d0 legacy=0x7f9868161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:42.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.311+0000 7f9866ffd640 1 -- 172.21.15.76:0/2257810979 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98680ce090 con 0x7f98680b4880 2023-11-09T03:48:42.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.311+0000 7f9866ffd640 1 -- 172.21.15.76:0/2257810979 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f98600019f0 con 0x7f98680b4880 2023-11-09T03:48:42.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.311+0000 7f9866ffd640 1 -- 172.21.15.76:0/2257810979 >> v1:172.21.15.76:6789/0 conn(0x7f98680b4880 legacy=0x7f98680d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:42.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.311+0000 7f9866ffd640 1 --2- 172.21.15.76:0/2257810979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98540039a0 0x7f9854003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.311+0000 7f9866ffd640 1 --2- 172.21.15.76:0/2257810979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9868153e60 0x7f98540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.311+0000 7f9866ffd640 1 --2- 172.21.15.76:0/2257810979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9854004d00 0x7f9854005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.311+0000 7f9866ffd640 1 -- 172.21.15.76:0/2257810979 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986815e400 con 0x7f9854004d00 2023-11-09T03:48:42.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.311+0000 7f9866ffd640 1 -- 172.21.15.76:0/2257810979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985400b980 con 0x7f98540039a0 2023-11-09T03:48:42.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.311+0000 7f9866ffd640 1 -- 172.21.15.76:0/2257810979 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985400bb40 con 0x7f9868153e60 2023-11-09T03:48:42.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.311+0000 7f9867fff640 1 --2- 172.21.15.76:0/2257810979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9854004d00 0x7f9854005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:42.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.311+0000 7f98677fe640 1 --2- 172.21.15.76:0/2257810979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98540039a0 0x7f9854003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:42.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.311+0000 7f985effd640 1 --2- 172.21.15.76:0/2257810979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9868153e60 0x7f98540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:42.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.311+0000 7f9867fff640 1 -- 172.21.15.76:0/2257810979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9868153e60 msgr2=0x7f98540047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:42.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.311+0000 7f9867fff640 1 --2- 172.21.15.76:0/2257810979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9868153e60 0x7f98540047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.311+0000 7f9867fff640 1 -- 172.21.15.76:0/2257810979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98540039a0 msgr2=0x7f9854003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:42.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.311+0000 7f9867fff640 1 --2- 172.21.15.76:0/2257810979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98540039a0 0x7f9854003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.311+0000 7f9867fff640 1 -- 172.21.15.76:0/2257810979 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9854003520 con 0x7f9854004d00 2023-11-09T03:48:42.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.312+0000 7f9867fff640 1 --2- 172.21.15.76:0/2257810979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9854004d00 0x7f9854005180 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f986000a840 tx=0x7f98540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79000905feec0c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:42.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.312+0000 7f9866ffd640 1 -- 172.21.15.76:0/2257810979 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986002cea0 con 0x7f9854004d00 2023-11-09T03:48:42.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.312+0000 7f9866ffd640 1 -- 172.21.15.76:0/2257810979 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986002f070 con 0x7f9854004d00 2023-11-09T03:48:42.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.312+0000 7f9866ffd640 1 -- 172.21.15.76:0/2257810979 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9860034870 con 0x7f9854004d00 2023-11-09T03:48:42.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.313+0000 7f986dad4640 1 -- 172.21.15.76:0/2257810979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9854004d00 msgr2=0x7f9854005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:42.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.313+0000 7f986dad4640 1 --2- 172.21.15.76:0/2257810979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9854004d00 0x7f9854005180 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f986000a840 tx=0x7f98540057f0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.313+0000 7f9867fff640 1 -- 172.21.15.76:0/2257810979 reap_dead start 2023-11-09T03:48:42.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.313+0000 7f986dad4640 1 -- 172.21.15.76:0/2257810979 shutdown_connections 2023-11-09T03:48:42.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.313+0000 7f986dad4640 1 -- 172.21.15.76:0/2257810979 >> 172.21.15.76:0/2257810979 conn(0x7f98680cf880 msgr2=0x7f98680cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:42.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.313+0000 7f986dad4640 1 -- 172.21.15.76:0/2257810979 shutdown_connections 2023-11-09T03:48:42.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.313+0000 7f986dad4640 1 -- 172.21.15.76:0/2257810979 wait complete. 2023-11-09T03:48:42.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.314+0000 7f986dad4640 1 Processor -- start 2023-11-09T03:48:42.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.314+0000 7f986dad4640 1 -- start start 2023-11-09T03:48:42.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.315+0000 7f986dad4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9868154810 0x7f98680cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.315+0000 7f986dad4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98680ccb70 0x7f986814ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.315+0000 7f986dad4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9868154bf0 0x7f98681504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.315+0000 7f986dad4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9868150a10 con 0x7f9868154bf0 2023-11-09T03:48:42.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.315+0000 7f986dad4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9868155fa0 con 0x7f98680ccb70 2023-11-09T03:48:42.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.315+0000 7f986dad4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9868156fb0 con 0x7f9868154810 2023-11-09T03:48:42.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.315+0000 7f9867fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9868154bf0 0x7f98681504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:42.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.315+0000 7f9867fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9868154bf0 0x7f98681504d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50030/0 (socket says 172.21.15.76:50030) 2023-11-09T03:48:42.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.315+0000 7f9867fff640 1 -- 172.21.15.76:0/2095863367 learned_addr learned my addr 172.21.15.76:0/2095863367 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:42.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.315+0000 7f9867fff640 1 -- 172.21.15.76:0/2095863367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9868154810 msgr2=0x7f98680cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:42.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.315+0000 7f9867fff640 1 --2- 172.21.15.76:0/2095863367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9868154810 0x7f98680cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.315+0000 7f9867fff640 1 -- 172.21.15.76:0/2095863367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98680ccb70 msgr2=0x7f986814ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:42.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.315+0000 7f9867fff640 1 --2- 172.21.15.76:0/2095863367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98680ccb70 0x7f986814ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.315+0000 7f9867fff640 1 -- 172.21.15.76:0/2095863367 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98681561f0 con 0x7f9868154bf0 2023-11-09T03:48:42.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.315+0000 7f9867fff640 1 --2- 172.21.15.76:0/2095863367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9868154bf0 0x7f98681504d0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f986002cb60 tx=0x7f98680d0690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:42.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.316+0000 7f9864ff9640 1 -- 172.21.15.76:0/2095863367 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9860034840 con 0x7f9868154bf0 2023-11-09T03:48:42.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.316+0000 7f9864ff9640 1 -- 172.21.15.76:0/2095863367 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98600349a0 con 0x7f9868154bf0 2023-11-09T03:48:42.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.316+0000 7f9864ff9640 1 -- 172.21.15.76:0/2095863367 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9860034d10 con 0x7f9868154bf0 2023-11-09T03:48:42.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.316+0000 7f986dad4640 1 -- 172.21.15.76:0/2095863367 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98681564d0 con 0x7f9868154bf0 2023-11-09T03:48:42.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.316+0000 7f986dad4640 1 -- 172.21.15.76:0/2095863367 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9868156a90 con 0x7f9868154bf0 2023-11-09T03:48:42.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.317+0000 7f986dad4640 1 -- 172.21.15.76:0/2095863367 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98680cb050 con 0x7f9868154bf0 2023-11-09T03:48:42.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.317+0000 7f9864ff9640 1 -- 172.21.15.76:0/2095863367 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9860031030 con 0x7f9868154bf0 2023-11-09T03:48:42.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.317+0000 7f9864ff9640 1 --2- 172.21.15.76:0/2095863367 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f984806aea0 0x7f984806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.317+0000 7f9864ff9640 1 -- 172.21.15.76:0/2095863367 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f98600abe60 con 0x7f9868154bf0 2023-11-09T03:48:42.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.318+0000 7f98677fe640 1 --2- 172.21.15.76:0/2095863367 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f984806aea0 0x7f984806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:42.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.318+0000 7f98677fe640 1 --2- 172.21.15.76:0/2095863367 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f984806aea0 0x7f984806d360 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9858009ab0 tx=0x7f98580095d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:42.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.322+0000 7f9864ff9640 1 -- 172.21.15.76:0/2095863367 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9860076ac0 con 0x7f9868154bf0 2023-11-09T03:48:42.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.487+0000 7f986dad4640 1 -- 172.21.15.76:0/2095863367 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7f98680cb050 con 0x7f9868154bf0 2023-11-09T03:48:42.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.488+0000 7f9864ff9640 1 -- 172.21.15.76:0/2095863367 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7f98600011c0 con 0x7f9868154bf0 2023-11-09T03:48:42.487 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:48:42.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.490+0000 7f986dad4640 1 -- 172.21.15.76:0/2095863367 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f984806aea0 msgr2=0x7f984806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:42.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.490+0000 7f986dad4640 1 --2- 172.21.15.76:0/2095863367 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f984806aea0 0x7f984806d360 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9858009ab0 tx=0x7f98580095d0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.490+0000 7f986dad4640 1 -- 172.21.15.76:0/2095863367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9868154bf0 msgr2=0x7f98681504d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:42.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.490+0000 7f986dad4640 1 --2- 172.21.15.76:0/2095863367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9868154bf0 0x7f98681504d0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f986002cb60 tx=0x7f98680d0690 comp rx=0 tx=0).stop 2023-11-09T03:48:42.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.490+0000 7f986dad4640 1 -- 172.21.15.76:0/2095863367 shutdown_connections 2023-11-09T03:48:42.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.490+0000 7f986dad4640 1 --2- 172.21.15.76:0/2095863367 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f984806aea0 0x7f984806d360 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.490+0000 7f986dad4640 1 --2- 172.21.15.76:0/2095863367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9868154bf0 0x7f98681504d0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.490+0000 7f986dad4640 1 --2- 172.21.15.76:0/2095863367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98680ccb70 0x7f986814ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.490+0000 7f986dad4640 1 --2- 172.21.15.76:0/2095863367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9868154810 0x7f98680cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.490+0000 7f986dad4640 1 -- 172.21.15.76:0/2095863367 >> 172.21.15.76:0/2095863367 conn(0x7f98680cf880 msgr2=0x7f98680cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:42.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.490+0000 7f986dad4640 1 -- 172.21.15.76:0/2095863367 shutdown_connections 2023-11-09T03:48:42.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.490+0000 7f986dad4640 1 -- 172.21.15.76:0/2095863367 wait complete. 2023-11-09T03:48:42.500 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:48:42.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.604+0000 7f165347c640 1 Processor -- start 2023-11-09T03:48:42.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.604+0000 7f165347c640 1 -- start start 2023-11-09T03:48:42.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.605+0000 7f165347c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f164c0ae1d0 0x7f164c0cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.605+0000 7f165347c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f164c15aff0 con 0x7f164c159c00 2023-11-09T03:48:42.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.605+0000 7f165347c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f164c15a3e0 con 0x7f164c0c4ff0 2023-11-09T03:48:42.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.605+0000 7f165347c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f164c15a6c0 con 0x7f164c0ae1d0 2023-11-09T03:48:42.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.605+0000 7f1651218640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f164c159c00 0x7f164c0c48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48596/0 (socket says 172.21.15.76:48596) 2023-11-09T03:48:42.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.605+0000 7f1651218640 1 -- 172.21.15.76:0/3975639724 learned_addr learned my addr 172.21.15.76:0/3975639724 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:42.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.605+0000 7f1643fff640 1 -- 172.21.15.76:0/3975639724 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1132648106 0 0) 0x7f164c15aff0 con 0x7f164c159c00 2023-11-09T03:48:42.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.605+0000 7f1643fff640 1 -- 172.21.15.76:0/3975639724 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1638003520 con 0x7f164c159c00 2023-11-09T03:48:42.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.605+0000 7f1651a19640 1 --2- 172.21.15.76:0/3975639724 >> v2:172.21.15.113:3300/0 conn(0x7f164c0ae1d0 0x7f164c0cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:42.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.606+0000 7f1643fff640 1 -- 172.21.15.76:0/3975639724 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4168125992 0 0) 0x7f1638003520 con 0x7f164c159c00 2023-11-09T03:48:42.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.606+0000 7f1643fff640 1 -- 172.21.15.76:0/3975639724 >> v2:172.21.15.113:3300/0 conn(0x7f164c0ae1d0 msgr2=0x7f164c0cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:42.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.606+0000 7f1643fff640 1 --2- 172.21.15.76:0/3975639724 >> v2:172.21.15.113:3300/0 conn(0x7f164c0ae1d0 0x7f164c0cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.606+0000 7f1643fff640 1 -- 172.21.15.76:0/3975639724 >> v1:172.21.15.113:6789/0 conn(0x7f164c0c4ff0 legacy=0x7f164c0adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:42.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.606+0000 7f1643fff640 1 -- 172.21.15.76:0/3975639724 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164c15a9a0 con 0x7f164c159c00 2023-11-09T03:48:42.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.606+0000 7f1643fff640 1 -- 172.21.15.76:0/3975639724 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f16480019f0 con 0x7f164c159c00 2023-11-09T03:48:42.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.606+0000 7f1643fff640 1 -- 172.21.15.76:0/3975639724 >> v1:172.21.15.76:6789/0 conn(0x7f164c159c00 legacy=0x7f164c0c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:42.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.606+0000 7f1643fff640 1 --2- 172.21.15.76:0/3975639724 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1638003970 0x7f1638003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.606+0000 7f1643fff640 1 --2- 172.21.15.76:0/3975639724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16380047c0 0x7f1638004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.606+0000 7f1643fff640 1 --2- 172.21.15.76:0/3975639724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1638005180 0x7f163800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.606+0000 7f1643fff640 1 -- 172.21.15.76:0/3975639724 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164c15a6c0 con 0x7f16380047c0 2023-11-09T03:48:42.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.606+0000 7f1643fff640 1 -- 172.21.15.76:0/3975639724 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163800c020 con 0x7f1638005180 2023-11-09T03:48:42.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.606+0000 7f1643fff640 1 -- 172.21.15.76:0/3975639724 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163800c1e0 con 0x7f1638003970 2023-11-09T03:48:42.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.606+0000 7f1650a17640 1 --2- 172.21.15.76:0/3975639724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16380047c0 0x7f1638004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:42.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.606+0000 7f1651218640 1 --2- 172.21.15.76:0/3975639724 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1638003970 0x7f1638003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:42.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.606+0000 7f1651a19640 1 --2- 172.21.15.76:0/3975639724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1638005180 0x7f163800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:42.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.606+0000 7f1650a17640 1 -- 172.21.15.76:0/3975639724 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1638003970 msgr2=0x7f1638003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:42.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.606+0000 7f1650a17640 1 --2- 172.21.15.76:0/3975639724 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1638003970 0x7f1638003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.606+0000 7f1650a17640 1 -- 172.21.15.76:0/3975639724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1638005180 msgr2=0x7f163800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:42.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.606+0000 7f1650a17640 1 --2- 172.21.15.76:0/3975639724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1638005180 0x7f163800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.606+0000 7f1650a17640 1 -- 172.21.15.76:0/3975639724 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1638003520 con 0x7f16380047c0 2023-11-09T03:48:42.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.606+0000 7f1651a19640 1 -- 172.21.15.76:0/3975639724 reap_dead start 2023-11-09T03:48:42.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.607+0000 7f1650a17640 1 --2- 172.21.15.76:0/3975639724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16380047c0 0x7f1638004c40 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f164c0cfa50 tx=0x7f16380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba02c1792e0891f server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:42.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.607+0000 7f1643fff640 1 -- 172.21.15.76:0/3975639724 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c0052d0 con 0x7f16380047c0 2023-11-09T03:48:42.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.607+0000 7f1643fff640 1 -- 172.21.15.76:0/3975639724 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163c02e050 con 0x7f16380047c0 2023-11-09T03:48:42.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.607+0000 7f1643fff640 1 -- 172.21.15.76:0/3975639724 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c003830 con 0x7f16380047c0 2023-11-09T03:48:42.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.608+0000 7f165347c640 1 -- 172.21.15.76:0/3975639724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16380047c0 msgr2=0x7f1638004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:42.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.608+0000 7f165347c640 1 --2- 172.21.15.76:0/3975639724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16380047c0 0x7f1638004c40 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f164c0cfa50 tx=0x7f16380057f0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.608+0000 7f165347c640 1 -- 172.21.15.76:0/3975639724 shutdown_connections 2023-11-09T03:48:42.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.608+0000 7f165347c640 1 --2- 172.21.15.76:0/3975639724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16380047c0 0x7f1638004c40 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.608+0000 7f165347c640 1 -- 172.21.15.76:0/3975639724 >> 172.21.15.76:0/3975639724 conn(0x7f164c0ce950 msgr2=0x7f164c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:42.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.608+0000 7f165347c640 1 -- 172.21.15.76:0/3975639724 shutdown_connections 2023-11-09T03:48:42.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.608+0000 7f165347c640 1 -- 172.21.15.76:0/3975639724 wait complete. 2023-11-09T03:48:42.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.609+0000 7f165347c640 1 Processor -- start 2023-11-09T03:48:42.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.609+0000 7f165347c640 1 -- start start 2023-11-09T03:48:42.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.610+0000 7f165347c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16380047c0 0x7f164c152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.610+0000 7f165347c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f164c0c7d30 0x7f164c0ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.610+0000 7f165347c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f164c157700 0x7f164c06d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.610+0000 7f165347c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164c0b3f50 con 0x7f16380047c0 2023-11-09T03:48:42.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.610+0000 7f165347c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164c0b41c0 con 0x7f164c0c7d30 2023-11-09T03:48:42.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.610+0000 7f165347c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f164c1571e0 con 0x7f164c157700 2023-11-09T03:48:42.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.610+0000 7f1651218640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16380047c0 0x7f164c152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:42.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.610+0000 7f1651218640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16380047c0 0x7f164c152640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51936/0 (socket says 172.21.15.76:51936) 2023-11-09T03:48:42.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.610+0000 7f1651218640 1 -- 172.21.15.76:0/1110526307 learned_addr learned my addr 172.21.15.76:0/1110526307 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:42.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.610+0000 7f1651a19640 1 --2- 172.21.15.76:0/1110526307 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f164c157700 0x7f164c06d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:42.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.610+0000 7f1650a17640 1 --2- 172.21.15.76:0/1110526307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f164c0c7d30 0x7f164c0ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:42.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.610+0000 7f1651218640 1 -- 172.21.15.76:0/1110526307 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f164c157700 msgr2=0x7f164c06d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:42.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.610+0000 7f1651218640 1 --2- 172.21.15.76:0/1110526307 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f164c157700 0x7f164c06d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.610+0000 7f1651218640 1 -- 172.21.15.76:0/1110526307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f164c0c7d30 msgr2=0x7f164c0ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:42.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.610+0000 7f1651218640 1 --2- 172.21.15.76:0/1110526307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f164c0c7d30 0x7f164c0ae5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.610+0000 7f1651218640 1 -- 172.21.15.76:0/1110526307 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164c0b44a0 con 0x7f16380047c0 2023-11-09T03:48:42.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.610+0000 7f1651218640 1 --2- 172.21.15.76:0/1110526307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16380047c0 0x7f164c152640 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f1648007ae0 tx=0x7f164800a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:42.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.610+0000 7f1641ffb640 1 -- 172.21.15.76:0/1110526307 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1648015410 con 0x7f16380047c0 2023-11-09T03:48:42.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.610+0000 7f1641ffb640 1 -- 172.21.15.76:0/1110526307 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1648015570 con 0x7f16380047c0 2023-11-09T03:48:42.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.610+0000 7f165347c640 1 -- 172.21.15.76:0/1110526307 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f164c0c82a0 con 0x7f16380047c0 2023-11-09T03:48:42.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.610+0000 7f1641ffb640 1 -- 172.21.15.76:0/1110526307 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1648015920 con 0x7f16380047c0 2023-11-09T03:48:42.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.610+0000 7f165347c640 1 -- 172.21.15.76:0/1110526307 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f164c157b80 con 0x7f16380047c0 2023-11-09T03:48:42.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.611+0000 7f165347c640 1 -- 172.21.15.76:0/1110526307 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f164c074290 con 0x7f16380047c0 2023-11-09T03:48:42.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.612+0000 7f1641ffb640 1 -- 172.21.15.76:0/1110526307 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1648015a80 con 0x7f16380047c0 2023-11-09T03:48:42.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.612+0000 7f1641ffb640 1 --2- 172.21.15.76:0/1110526307 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f163006ae70 0x7f163006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.612+0000 7f1641ffb640 1 -- 172.21.15.76:0/1110526307 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f164808c990 con 0x7f16380047c0 2023-11-09T03:48:42.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.612+0000 7f1650a17640 1 --2- 172.21.15.76:0/1110526307 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f163006ae70 0x7f163006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:42.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.613+0000 7f1650a17640 1 --2- 172.21.15.76:0/1110526307 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f163006ae70 0x7f163006d330 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f163c033000 tx=0x7f163c009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:42.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.616+0000 7f1641ffb640 1 -- 172.21.15.76:0/1110526307 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f16480574d0 con 0x7f16380047c0 2023-11-09T03:48:42.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.786+0000 7f165347c640 1 -- 172.21.15.76:0/1110526307 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f164c15c600 con 0x7f16380047c0 2023-11-09T03:48:42.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.786+0000 7f1641ffb640 1 -- 172.21.15.76:0/1110526307 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f164805aaf0 con 0x7f16380047c0 2023-11-09T03:48:42.786 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:48:42.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.788+0000 7f165347c640 1 -- 172.21.15.76:0/1110526307 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f163006ae70 msgr2=0x7f163006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:42.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.788+0000 7f165347c640 1 --2- 172.21.15.76:0/1110526307 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f163006ae70 0x7f163006d330 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f163c033000 tx=0x7f163c009030 comp rx=0 tx=0).stop 2023-11-09T03:48:42.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.788+0000 7f165347c640 1 -- 172.21.15.76:0/1110526307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16380047c0 msgr2=0x7f164c152640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:42.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.788+0000 7f165347c640 1 --2- 172.21.15.76:0/1110526307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16380047c0 0x7f164c152640 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f1648007ae0 tx=0x7f164800a6e0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.788+0000 7f165347c640 1 -- 172.21.15.76:0/1110526307 shutdown_connections 2023-11-09T03:48:42.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.788+0000 7f165347c640 1 --2- 172.21.15.76:0/1110526307 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f163006ae70 0x7f163006d330 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.788+0000 7f165347c640 1 --2- 172.21.15.76:0/1110526307 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f164c157700 0x7f164c06d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.788+0000 7f165347c640 1 --2- 172.21.15.76:0/1110526307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f164c0c7d30 0x7f164c0ae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.788+0000 7f165347c640 1 --2- 172.21.15.76:0/1110526307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16380047c0 0x7f164c152640 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.788+0000 7f165347c640 1 -- 172.21.15.76:0/1110526307 >> 172.21.15.76:0/1110526307 conn(0x7f164c0ce950 msgr2=0x7f164c15cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:42.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.789+0000 7f165347c640 1 -- 172.21.15.76:0/1110526307 shutdown_connections 2023-11-09T03:48:42.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.789+0000 7f165347c640 1 -- 172.21.15.76:0/1110526307 wait complete. 2023-11-09T03:48:42.799 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:48:42.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.902+0000 7fb402e32640 1 Processor -- start 2023-11-09T03:48:42.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.902+0000 7fb402e32640 1 -- start start 2023-11-09T03:48:42.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.902+0000 7fb402e32640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb3fc0ad730 0x7fb3fc0adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.902+0000 7fb402e32640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3fc0c3900 con 0x7fb3fc070e20 2023-11-09T03:48:42.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.902+0000 7fb402e32640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3fc0c3be0 con 0x7fb3fc15ad30 2023-11-09T03:48:42.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.902+0000 7fb402e32640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3fc0ae260 con 0x7fb3fc0ad730 2023-11-09T03:48:42.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.902+0000 7fb3fbfff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb3fc070e20 0x7fb3fc15e740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48602/0 (socket says 172.21.15.76:48602) 2023-11-09T03:48:42.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.902+0000 7fb3fbfff640 1 -- 172.21.15.76:0/4281963347 learned_addr learned my addr 172.21.15.76:0/4281963347 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:42.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.903+0000 7fb4013cf640 1 --2- 172.21.15.76:0/4281963347 >> v2:172.21.15.113:3300/0 conn(0x7fb3fc0ad730 0x7fb3fc0adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:42.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.903+0000 7fb3fb7fe640 1 -- 172.21.15.76:0/4281963347 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1316837192 0 0) 0x7fb3fc0c3900 con 0x7fb3fc070e20 2023-11-09T03:48:42.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.903+0000 7fb3fb7fe640 1 -- 172.21.15.76:0/4281963347 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3ec003520 con 0x7fb3fc070e20 2023-11-09T03:48:42.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.903+0000 7fb3fb7fe640 1 -- 172.21.15.76:0/4281963347 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 109241448 0 0) 0x7fb3ec003520 con 0x7fb3fc070e20 2023-11-09T03:48:42.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.903+0000 7fb3fb7fe640 1 -- 172.21.15.76:0/4281963347 >> v2:172.21.15.113:3300/0 conn(0x7fb3fc0ad730 msgr2=0x7fb3fc0adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:42.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.903+0000 7fb3fb7fe640 1 --2- 172.21.15.76:0/4281963347 >> v2:172.21.15.113:3300/0 conn(0x7fb3fc0ad730 0x7fb3fc0adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.903+0000 7fb3fb7fe640 1 -- 172.21.15.76:0/4281963347 >> v1:172.21.15.113:6789/0 conn(0x7fb3fc15ad30 legacy=0x7fb3fc070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:42.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.903+0000 7fb3fb7fe640 1 -- 172.21.15.76:0/4281963347 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3fc0c3ec0 con 0x7fb3fc070e20 2023-11-09T03:48:42.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.903+0000 7fb3fb7fe640 1 -- 172.21.15.76:0/4281963347 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fb3e8001670 con 0x7fb3fc070e20 2023-11-09T03:48:42.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.903+0000 7fb3fb7fe640 1 -- 172.21.15.76:0/4281963347 >> v1:172.21.15.76:6789/0 conn(0x7fb3fc070e20 legacy=0x7fb3fc15e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:42.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.903+0000 7fb3fb7fe640 1 --2- 172.21.15.76:0/4281963347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3ec003970 0x7fb3ec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.903+0000 7fb3fb7fe640 1 --2- 172.21.15.76:0/4281963347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3ec0047c0 0x7fb3ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.903+0000 7fb3fb7fe640 1 --2- 172.21.15.76:0/4281963347 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3ec005180 0x7fb3ec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.903+0000 7fb3fb7fe640 1 -- 172.21.15.76:0/4281963347 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc0ae260 con 0x7fb3ec0047c0 2023-11-09T03:48:42.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.903+0000 7fb3fb7fe640 1 -- 172.21.15.76:0/4281963347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ec00c120 con 0x7fb3ec003970 2023-11-09T03:48:42.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.903+0000 7fb3fb7fe640 1 -- 172.21.15.76:0/4281963347 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ec00c2e0 con 0x7fb3ec005180 2023-11-09T03:48:42.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.903+0000 7fb3fbfff640 1 --2- 172.21.15.76:0/4281963347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3ec0047c0 0x7fb3ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:42.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.903+0000 7fb400bce640 1 --2- 172.21.15.76:0/4281963347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3ec003970 0x7fb3ec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:42.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.903+0000 7fb4013cf640 1 --2- 172.21.15.76:0/4281963347 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3ec005180 0x7fb3ec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:42.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.903+0000 7fb3fbfff640 1 -- 172.21.15.76:0/4281963347 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3ec005180 msgr2=0x7fb3ec00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:42.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.903+0000 7fb3fbfff640 1 --2- 172.21.15.76:0/4281963347 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3ec005180 0x7fb3ec00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.903+0000 7fb3fbfff640 1 -- 172.21.15.76:0/4281963347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3ec003970 msgr2=0x7fb3ec003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:42.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.903+0000 7fb3fbfff640 1 --2- 172.21.15.76:0/4281963347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3ec003970 0x7fb3ec003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.904+0000 7fb3fbfff640 1 -- 172.21.15.76:0/4281963347 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ec003520 con 0x7fb3ec0047c0 2023-11-09T03:48:42.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.904+0000 7fb4013cf640 1 -- 172.21.15.76:0/4281963347 reap_dead start 2023-11-09T03:48:42.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.904+0000 7fb3fbfff640 1 --2- 172.21.15.76:0/4281963347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3ec0047c0 0x7fb3ec004c40 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fb3e800c190 tx=0x7fb3ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d624e143c5f255ce server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:42.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.904+0000 7fb3fb7fe640 1 -- 172.21.15.76:0/4281963347 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8005ad0 con 0x7fb3ec0047c0 2023-11-09T03:48:42.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.904+0000 7fb3fb7fe640 1 -- 172.21.15.76:0/4281963347 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e8005c30 con 0x7fb3ec0047c0 2023-11-09T03:48:42.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.904+0000 7fb3fb7fe640 1 -- 172.21.15.76:0/4281963347 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e802ca70 con 0x7fb3ec0047c0 2023-11-09T03:48:42.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.905+0000 7fb402e32640 1 -- 172.21.15.76:0/4281963347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3ec0047c0 msgr2=0x7fb3ec004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:42.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.905+0000 7fb402e32640 1 --2- 172.21.15.76:0/4281963347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3ec0047c0 0x7fb3ec004c40 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fb3e800c190 tx=0x7fb3ec0057f0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.905+0000 7fb402e32640 1 -- 172.21.15.76:0/4281963347 shutdown_connections 2023-11-09T03:48:42.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.905+0000 7fb402e32640 1 --2- 172.21.15.76:0/4281963347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3ec0047c0 0x7fb3ec004c40 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.905+0000 7fb402e32640 1 -- 172.21.15.76:0/4281963347 >> 172.21.15.76:0/4281963347 conn(0x7fb3fc0ce950 msgr2=0x7fb3fc0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:42.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.905+0000 7fb402e32640 1 -- 172.21.15.76:0/4281963347 shutdown_connections 2023-11-09T03:48:42.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.905+0000 7fb402e32640 1 -- 172.21.15.76:0/4281963347 wait complete. 2023-11-09T03:48:42.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.906+0000 7fb402e32640 1 Processor -- start 2023-11-09T03:48:42.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.906+0000 7fb402e32640 1 -- start start 2023-11-09T03:48:42.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.907+0000 7fb402e32640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3ec0047c0 0x7fb3fc0ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.907+0000 7fb402e32640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3fc14b540 0x7fb3fc0cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.907+0000 7fb402e32640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3fc0ccb80 0x7fb3fc0b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.907+0000 7fb402e32640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc0cdd80 con 0x7fb3fc14b540 2023-11-09T03:48:42.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.907+0000 7fb402e32640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc0cd100 con 0x7fb3ec0047c0 2023-11-09T03:48:42.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.907+0000 7fb402e32640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc0cd3e0 con 0x7fb3fc0ccb80 2023-11-09T03:48:42.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.907+0000 7fb3fbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3fc14b540 0x7fb3fc0cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:42.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.907+0000 7fb3fbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3fc14b540 0x7fb3fc0cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51964/0 (socket says 172.21.15.76:51964) 2023-11-09T03:48:42.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.907+0000 7fb3fbfff640 1 -- 172.21.15.76:0/2369056785 learned_addr learned my addr 172.21.15.76:0/2369056785 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:42.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.907+0000 7fb400bce640 1 --2- 172.21.15.76:0/2369056785 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3ec0047c0 0x7fb3fc0ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:42.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.907+0000 7fb3fbfff640 1 -- 172.21.15.76:0/2369056785 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3fc0ccb80 msgr2=0x7fb3fc0b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:42.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.907+0000 7fb3fbfff640 1 --2- 172.21.15.76:0/2369056785 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3fc0ccb80 0x7fb3fc0b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.907+0000 7fb3fbfff640 1 -- 172.21.15.76:0/2369056785 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3ec0047c0 msgr2=0x7fb3fc0ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:42.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.907+0000 7fb3fbfff640 1 --2- 172.21.15.76:0/2369056785 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3ec0047c0 0x7fb3fc0ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:42.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.907+0000 7fb3fbfff640 1 -- 172.21.15.76:0/2369056785 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3fc0cd6c0 con 0x7fb3fc14b540 2023-11-09T03:48:42.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.907+0000 7fb400bce640 1 --2- 172.21.15.76:0/2369056785 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3ec0047c0 0x7fb3fc0ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:42.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.908+0000 7fb3fbfff640 1 --2- 172.21.15.76:0/2369056785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3fc14b540 0x7fb3fc0cc640 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8001930 tx=0x7fb3e802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:42.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.908+0000 7fb3f97fa640 1 -- 172.21.15.76:0/2369056785 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e803c040 con 0x7fb3fc14b540 2023-11-09T03:48:42.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.908+0000 7fb3f97fa640 1 -- 172.21.15.76:0/2369056785 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e802cd00 con 0x7fb3fc14b540 2023-11-09T03:48:42.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.908+0000 7fb402e32640 1 -- 172.21.15.76:0/2369056785 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3fc0cd9a0 con 0x7fb3fc14b540 2023-11-09T03:48:42.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.908+0000 7fb3f97fa640 1 -- 172.21.15.76:0/2369056785 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8007290 con 0x7fb3fc14b540 2023-11-09T03:48:42.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.908+0000 7fb402e32640 1 -- 172.21.15.76:0/2369056785 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3fc0b39c0 con 0x7fb3fc14b540 2023-11-09T03:48:42.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.909+0000 7fb402e32640 1 -- 172.21.15.76:0/2369056785 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3fc0cdec0 con 0x7fb3fc14b540 2023-11-09T03:48:42.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.909+0000 7fb3f97fa640 1 -- 172.21.15.76:0/2369056785 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb3e8032070 con 0x7fb3fc14b540 2023-11-09T03:48:42.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.909+0000 7fb3f97fa640 1 --2- 172.21.15.76:0/2369056785 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb3e006aea0 0x7fb3e006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:42.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.910+0000 7fb3f97fa640 1 -- 172.21.15.76:0/2369056785 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7148+0+0 (secure 0 0 0) 0x7fb3e80aaec0 con 0x7fb3fc14b540 2023-11-09T03:48:42.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.910+0000 7fb400bce640 1 --2- 172.21.15.76:0/2369056785 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb3e006aea0 0x7fb3e006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:42.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.910+0000 7fb400bce640 1 --2- 172.21.15.76:0/2369056785 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb3e006aea0 0x7fb3e006d360 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0687c0 tx=0x7fb3f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:42.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:42.914+0000 7fb3f97fa640 1 -- 172.21.15.76:0/2369056785 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb3fc0cdec0 con 0x7fb3fc14b540 2023-11-09T03:48:43.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.082+0000 7fb402e32640 1 -- 172.21.15.76:0/2369056785 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb3fc0cdec0 con 0x7fb3fc14b540 2023-11-09T03:48:43.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.082+0000 7fb3f97fa640 1 -- 172.21.15.76:0/2369056785 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb3fc0cdec0 con 0x7fb3fc14b540 2023-11-09T03:48:43.082 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:48:43.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.084+0000 7fb402e32640 1 -- 172.21.15.76:0/2369056785 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb3e006aea0 msgr2=0x7fb3e006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:43.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.084+0000 7fb402e32640 1 --2- 172.21.15.76:0/2369056785 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb3e006aea0 0x7fb3e006d360 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0687c0 tx=0x7fb3f0009040 comp rx=0 tx=0).stop 2023-11-09T03:48:43.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.084+0000 7fb402e32640 1 -- 172.21.15.76:0/2369056785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3fc14b540 msgr2=0x7fb3fc0cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:43.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.084+0000 7fb402e32640 1 --2- 172.21.15.76:0/2369056785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3fc14b540 0x7fb3fc0cc640 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8001930 tx=0x7fb3e802e070 comp rx=0 tx=0).stop 2023-11-09T03:48:43.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.084+0000 7fb402e32640 1 -- 172.21.15.76:0/2369056785 shutdown_connections 2023-11-09T03:48:43.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.084+0000 7fb402e32640 1 --2- 172.21.15.76:0/2369056785 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb3e006aea0 0x7fb3e006d360 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:43.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.084+0000 7fb402e32640 1 --2- 172.21.15.76:0/2369056785 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3fc0ccb80 0x7fb3fc0b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:43.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.084+0000 7fb402e32640 1 --2- 172.21.15.76:0/2369056785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3fc14b540 0x7fb3fc0cc640 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:43.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.084+0000 7fb402e32640 1 --2- 172.21.15.76:0/2369056785 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3ec0047c0 0x7fb3fc0ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:43.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.084+0000 7fb402e32640 1 -- 172.21.15.76:0/2369056785 >> 172.21.15.76:0/2369056785 conn(0x7fb3fc0ce950 msgr2=0x7fb3fc0afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:43.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.084+0000 7fb402e32640 1 -- 172.21.15.76:0/2369056785 shutdown_connections 2023-11-09T03:48:43.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.084+0000 7fb402e32640 1 -- 172.21.15.76:0/2369056785 wait complete. 2023-11-09T03:48:43.094 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:48:43.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.197+0000 7f88d7234640 1 Processor -- start 2023-11-09T03:48:43.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.197+0000 7f88d7234640 1 -- start start 2023-11-09T03:48:43.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.197+0000 7f88d7234640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f88d0159c70 0x7f88d00ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:43.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.197+0000 7f88d7234640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f88d00b0ae0 0x7f88d015d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:43.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.197+0000 7f88d7234640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d00b4110 con 0x7f88d006db20 2023-11-09T03:48:43.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.197+0000 7f88d7234640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88d00b49b0 con 0x7f88d00b0ae0 2023-11-09T03:48:43.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.197+0000 7f88d7234640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88d00b43f0 con 0x7f88d0159c70 2023-11-09T03:48:43.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.197+0000 7f88d57d1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f88d00b0ae0 0x7f88d015d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:43.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.198+0000 7f88d57d1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f88d00b0ae0 0x7f88d015d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51976/0 (socket says 172.21.15.76:51976) 2023-11-09T03:48:43.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.198+0000 7f88d57d1640 1 -- 172.21.15.76:0/3081746314 learned_addr learned my addr 172.21.15.76:0/3081746314 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:43.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.198+0000 7f88d57d1640 1 -- 172.21.15.76:0/3081746314 >> v2:172.21.15.113:3300/0 conn(0x7f88d0159c70 msgr2=0x7f88d00ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:43.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.198+0000 7f88d57d1640 1 --2- 172.21.15.76:0/3081746314 >> v2:172.21.15.113:3300/0 conn(0x7f88d0159c70 0x7f88d00ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:43.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.198+0000 7f88d57d1640 1 -- 172.21.15.76:0/3081746314 >> v1:172.21.15.113:6789/0 conn(0x7f88d006db20 legacy=0x7f88d00b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:43.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.198+0000 7f88d57d1640 1 -- 172.21.15.76:0/3081746314 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d00b46d0 con 0x7f88d00b0ae0 2023-11-09T03:48:43.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.198+0000 7f88d57d1640 1 --2- 172.21.15.76:0/3081746314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88d00b0ae0 0x7f88d015d7d0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f88d00cc440 tx=0x7f88c802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94e928125b11725 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:43.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.199+0000 7f88cf7fe640 1 -- 172.21.15.76:0/3081746314 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c8031870 con 0x7f88d00b0ae0 2023-11-09T03:48:43.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.199+0000 7f88cf7fe640 1 -- 172.21.15.76:0/3081746314 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c80319d0 con 0x7f88d00b0ae0 2023-11-09T03:48:43.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.199+0000 7f88cf7fe640 1 -- 172.21.15.76:0/3081746314 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c8031d60 con 0x7f88d00b0ae0 2023-11-09T03:48:43.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.199+0000 7f88d7234640 1 -- 172.21.15.76:0/3081746314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88d00b0ae0 msgr2=0x7f88d015d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:43.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.199+0000 7f88d7234640 1 --2- 172.21.15.76:0/3081746314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88d00b0ae0 0x7f88d015d7d0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f88d00cc440 tx=0x7f88c802b910 comp rx=0 tx=0).stop 2023-11-09T03:48:43.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.199+0000 7f88d7234640 1 -- 172.21.15.76:0/3081746314 shutdown_connections 2023-11-09T03:48:43.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.199+0000 7f88d7234640 1 --2- 172.21.15.76:0/3081746314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88d00b0ae0 0x7f88d015d7d0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:43.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.199+0000 7f88d7234640 1 --2- 172.21.15.76:0/3081746314 >> v2:172.21.15.113:3300/0 conn(0x7f88d0159c70 0x7f88d00ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:43.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.199+0000 7f88d7234640 1 -- 172.21.15.76:0/3081746314 >> 172.21.15.76:0/3081746314 conn(0x7f88d00ce950 msgr2=0x7f88d00ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:43.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.199+0000 7f88d7234640 1 -- 172.21.15.76:0/3081746314 shutdown_connections 2023-11-09T03:48:43.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.199+0000 7f88d7234640 1 -- 172.21.15.76:0/3081746314 wait complete. 2023-11-09T03:48:43.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.200+0000 7f88d7234640 1 Processor -- start 2023-11-09T03:48:43.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.200+0000 7f88d7234640 1 -- start start 2023-11-09T03:48:43.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.201+0000 7f88d7234640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88d006db20 0x7f88d014ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:43.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.201+0000 7f88d7234640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88d00b0ae0 0x7f88d015c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:43.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.201+0000 7f88d7234640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88d0159c70 0x7f88d01e8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:43.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.201+0000 7f88d7234640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d00c7680 con 0x7f88d0159c70 2023-11-09T03:48:43.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.201+0000 7f88d7234640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d00c67d0 con 0x7f88d006db20 2023-11-09T03:48:43.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.201+0000 7f88d7234640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d0158500 con 0x7f88d00b0ae0 2023-11-09T03:48:43.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.201+0000 7f88d57d1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88d0159c70 0x7f88d01e8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:43.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.201+0000 7f88d57d1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88d0159c70 0x7f88d01e8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51982/0 (socket says 172.21.15.76:51982) 2023-11-09T03:48:43.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.201+0000 7f88d57d1640 1 -- 172.21.15.76:0/433077653 learned_addr learned my addr 172.21.15.76:0/433077653 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:43.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.202+0000 7f88d4fd0640 1 --2- 172.21.15.76:0/433077653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88d006db20 0x7f88d014ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:43.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.202+0000 7f88cffff640 1 --2- 172.21.15.76:0/433077653 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88d00b0ae0 0x7f88d015c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:43.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.202+0000 7f88d57d1640 1 -- 172.21.15.76:0/433077653 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88d00b0ae0 msgr2=0x7f88d015c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:43.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.202+0000 7f88d57d1640 1 --2- 172.21.15.76:0/433077653 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88d00b0ae0 0x7f88d015c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:43.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.202+0000 7f88d57d1640 1 -- 172.21.15.76:0/433077653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88d006db20 msgr2=0x7f88d014ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:43.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.202+0000 7f88d57d1640 1 --2- 172.21.15.76:0/433077653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88d006db20 0x7f88d014ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:43.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.202+0000 7f88d57d1640 1 -- 172.21.15.76:0/433077653 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c80094d0 con 0x7f88d0159c70 2023-11-09T03:48:43.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.202+0000 7f88d4fd0640 1 --2- 172.21.15.76:0/433077653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88d006db20 0x7f88d014ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:43.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.202+0000 7f88d57d1640 1 --2- 172.21.15.76:0/433077653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88d0159c70 0x7f88d01e8100 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f88c802b950 tx=0x7f88c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:43.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.202+0000 7f88cd7fa640 1 -- 172.21.15.76:0/433077653 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c803adf0 con 0x7f88d0159c70 2023-11-09T03:48:43.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.202+0000 7f88cd7fa640 1 -- 172.21.15.76:0/433077653 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c80443f0 con 0x7f88d0159c70 2023-11-09T03:48:43.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.202+0000 7f88cd7fa640 1 -- 172.21.15.76:0/433077653 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c80447a0 con 0x7f88d0159c70 2023-11-09T03:48:43.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.202+0000 7f88d7234640 1 -- 172.21.15.76:0/433077653 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d00c6a00 con 0x7f88d0159c70 2023-11-09T03:48:43.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.202+0000 7f88d7234640 1 -- 172.21.15.76:0/433077653 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d00c6fc0 con 0x7f88d0159c70 2023-11-09T03:48:43.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.203+0000 7f88d7234640 1 -- 172.21.15.76:0/433077653 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d0074290 con 0x7f88d0159c70 2023-11-09T03:48:43.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.203+0000 7f88cd7fa640 1 -- 172.21.15.76:0/433077653 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f88c8044900 con 0x7f88d0159c70 2023-11-09T03:48:43.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.204+0000 7f88cd7fa640 1 --2- 172.21.15.76:0/433077653 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f88bc06ae70 0x7f88bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:43.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.204+0000 7f88cd7fa640 1 -- 172.21.15.76:0/433077653 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7148+0+0 (secure 0 0 0) 0x7f88c802d040 con 0x7f88d0159c70 2023-11-09T03:48:43.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.204+0000 7f88d4fd0640 1 --2- 172.21.15.76:0/433077653 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f88bc06ae70 0x7f88bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:43.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.205+0000 7f88d4fd0640 1 --2- 172.21.15.76:0/433077653 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f88bc06ae70 0x7f88bc06d330 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f88d015c430 tx=0x7f88c0009570 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:43.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.208+0000 7f88cd7fa640 1 -- 172.21.15.76:0/433077653 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f88c8076f20 con 0x7f88d0159c70 2023-11-09T03:48:43.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.409+0000 7f88d7234640 1 -- 172.21.15.76:0/433077653 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88d00c8ef0 con 0x7f88d0159c70 2023-11-09T03:48:43.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.409+0000 7f88cd7fa640 1 -- 172.21.15.76:0/433077653 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f88c807a540 con 0x7f88d0159c70 2023-11-09T03:48:43.410 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:43.410 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":37,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6097,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2408659865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2408659865},{"type":"v1","addr":"172.21.15.113:6836","nonce":2408659865}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6100,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1873078383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1873078383},{"type":"v1","addr":"172.21.15.113:6837","nonce":1873078383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/3549714130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3549714130},{"type":"v1","addr":"172.21.15.76:6834","nonce":3549714130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:22.895004+0000","modified":"2023-11-09T03:48:37.005363+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6984":{"gid":6984,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/2165197913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":2165197913},{"type":"v1","addr":"172.21.15.76:6835","nonce":2165197913}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-09T03:48:43.411 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 37 2023-11-09T03:48:43.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.411+0000 7f88d7234640 1 -- 172.21.15.76:0/433077653 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f88bc06ae70 msgr2=0x7f88bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:43.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.411+0000 7f88d7234640 1 --2- 172.21.15.76:0/433077653 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f88bc06ae70 0x7f88bc06d330 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f88d015c430 tx=0x7f88c0009570 comp rx=0 tx=0).stop 2023-11-09T03:48:43.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.411+0000 7f88d7234640 1 -- 172.21.15.76:0/433077653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88d0159c70 msgr2=0x7f88d01e8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:43.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.411+0000 7f88d7234640 1 --2- 172.21.15.76:0/433077653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88d0159c70 0x7f88d01e8100 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f88c802b950 tx=0x7f88c80046c0 comp rx=0 tx=0).stop 2023-11-09T03:48:43.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.411+0000 7f88d7234640 1 -- 172.21.15.76:0/433077653 shutdown_connections 2023-11-09T03:48:43.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.411+0000 7f88d7234640 1 --2- 172.21.15.76:0/433077653 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f88bc06ae70 0x7f88bc06d330 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:43.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.411+0000 7f88d7234640 1 --2- 172.21.15.76:0/433077653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88d0159c70 0x7f88d01e8100 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:43.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.411+0000 7f88d7234640 1 --2- 172.21.15.76:0/433077653 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88d00b0ae0 0x7f88d015c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:43.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.411+0000 7f88d7234640 1 --2- 172.21.15.76:0/433077653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88d006db20 0x7f88d014ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:43.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.411+0000 7f88d7234640 1 -- 172.21.15.76:0/433077653 >> 172.21.15.76:0/433077653 conn(0x7f88d00ce950 msgr2=0x7f88d00b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:43.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.411+0000 7f88d7234640 1 -- 172.21.15.76:0/433077653 shutdown_connections 2023-11-09T03:48:43.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.411+0000 7f88d7234640 1 -- 172.21.15.76:0/433077653 wait complete. 2023-11-09T03:48:43.423 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:48:43.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.522+0000 7fa30470f640 1 Processor -- start 2023-11-09T03:48:43.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.522+0000 7fa30470f640 1 -- start start 2023-11-09T03:48:43.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.523+0000 7fa30470f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa2fc0b0ae0 0x7fa2fc0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:43.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.523+0000 7fa30470f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2fc0b4110 con 0x7fa2fc159c70 2023-11-09T03:48:43.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.523+0000 7fa30470f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2fc0b43f0 con 0x7fa2fc06db20 2023-11-09T03:48:43.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.523+0000 7fa30470f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2fc0b49b0 con 0x7fa2fc0b0ae0 2023-11-09T03:48:43.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.523+0000 7fa301caa640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa2fc0b0ae0 0x7fa2fc0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:43.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.523+0000 7fa3024ab640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa2fc159c70 0x7fa2fc0b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48614/0 (socket says 172.21.15.76:48614) 2023-11-09T03:48:43.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.523+0000 7fa3024ab640 1 -- 172.21.15.76:0/788348223 learned_addr learned my addr 172.21.15.76:0/788348223 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:43.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.523+0000 7fa301caa640 1 -- 172.21.15.76:0/788348223 >> v1:172.21.15.113:6789/0 conn(0x7fa2fc06db20 legacy=0x7fa2fc15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:43.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.523+0000 7fa3014a9640 1 -- 172.21.15.76:0/788348223 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2434853525 0 0) 0x7fa2fc0b4110 con 0x7fa2fc159c70 2023-11-09T03:48:43.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.523+0000 7fa301caa640 1 -- 172.21.15.76:0/788348223 >> v1:172.21.15.76:6789/0 conn(0x7fa2fc159c70 legacy=0x7fa2fc0b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:43.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.523+0000 7fa301caa640 1 -- 172.21.15.76:0/788348223 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2fc0b46d0 con 0x7fa2fc0b0ae0 2023-11-09T03:48:43.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.524+0000 7fa301caa640 1 --2- 172.21.15.76:0/788348223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2fc0b0ae0 0x7fa2fc0ce370 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0097a0 tx=0x7fa2ec02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b008db7aeae22450 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:43.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.524+0000 7fa3014a9640 1 -- 172.21.15.76:0/788348223 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec030610 con 0x7fa2fc0b0ae0 2023-11-09T03:48:43.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.524+0000 7fa3014a9640 1 -- 172.21.15.76:0/788348223 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2ec030770 con 0x7fa2fc0b0ae0 2023-11-09T03:48:43.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.524+0000 7fa3014a9640 1 -- 172.21.15.76:0/788348223 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec030b00 con 0x7fa2fc0b0ae0 2023-11-09T03:48:43.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.524+0000 7fa30470f640 1 -- 172.21.15.76:0/788348223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2fc0b0ae0 msgr2=0x7fa2fc0ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:43.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.524+0000 7fa30470f640 1 --2- 172.21.15.76:0/788348223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2fc0b0ae0 0x7fa2fc0ce370 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0097a0 tx=0x7fa2ec02b960 comp rx=0 tx=0).stop 2023-11-09T03:48:43.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.525+0000 7fa30470f640 1 -- 172.21.15.76:0/788348223 shutdown_connections 2023-11-09T03:48:43.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.525+0000 7fa30470f640 1 --2- 172.21.15.76:0/788348223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2fc0b0ae0 0x7fa2fc0ce370 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:43.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.525+0000 7fa30470f640 1 -- 172.21.15.76:0/788348223 >> 172.21.15.76:0/788348223 conn(0x7fa2fc0ce950 msgr2=0x7fa2fc0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:43.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.525+0000 7fa30470f640 1 -- 172.21.15.76:0/788348223 shutdown_connections 2023-11-09T03:48:43.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.525+0000 7fa30470f640 1 -- 172.21.15.76:0/788348223 wait complete. 2023-11-09T03:48:43.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.526+0000 7fa30470f640 1 Processor -- start 2023-11-09T03:48:43.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.526+0000 7fa30470f640 1 -- start start 2023-11-09T03:48:43.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.526+0000 7fa30470f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2fc06db20 0x7fa2fc0afa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:43.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.526+0000 7fa30470f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2fc0b0ae0 0x7fa2fc154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:43.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.526+0000 7fa30470f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2fc159c70 0x7fa2fc1546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:43.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.526+0000 7fa30470f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc0af5c0 con 0x7fa2fc06db20 2023-11-09T03:48:43.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.526+0000 7fa30470f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc0ae9b0 con 0x7fa2fc0b0ae0 2023-11-09T03:48:43.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.526+0000 7fa30470f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc0aec90 con 0x7fa2fc159c70 2023-11-09T03:48:43.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.527+0000 7fa3024ab640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2fc06db20 0x7fa2fc0afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:43.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.527+0000 7fa3024ab640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2fc06db20 0x7fa2fc0afa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52002/0 (socket says 172.21.15.76:52002) 2023-11-09T03:48:43.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.527+0000 7fa3024ab640 1 -- 172.21.15.76:0/3286908978 learned_addr learned my addr 172.21.15.76:0/3286908978 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:43.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.527+0000 7fa301caa640 1 --2- 172.21.15.76:0/3286908978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2fc0b0ae0 0x7fa2fc154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:43.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.527+0000 7fa302cac640 1 --2- 172.21.15.76:0/3286908978 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2fc159c70 0x7fa2fc1546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:43.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.527+0000 7fa3024ab640 1 -- 172.21.15.76:0/3286908978 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2fc159c70 msgr2=0x7fa2fc1546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:43.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.527+0000 7fa3024ab640 1 --2- 172.21.15.76:0/3286908978 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2fc159c70 0x7fa2fc1546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:43.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.527+0000 7fa3024ab640 1 -- 172.21.15.76:0/3286908978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2fc0b0ae0 msgr2=0x7fa2fc154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:43.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.527+0000 7fa3024ab640 1 --2- 172.21.15.76:0/3286908978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2fc0b0ae0 0x7fa2fc154180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:43.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.527+0000 7fa3024ab640 1 -- 172.21.15.76:0/3286908978 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ec009400 con 0x7fa2fc06db20 2023-11-09T03:48:43.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.527+0000 7fa301caa640 1 --2- 172.21.15.76:0/3286908978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2fc0b0ae0 0x7fa2fc154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:43.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.527+0000 7fa3024ab640 1 --2- 172.21.15.76:0/3286908978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2fc06db20 0x7fa2fc0afa50 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fa2f800c3f0 tx=0x7fa2f800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:43.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.527+0000 7fa2f2ffd640 1 -- 172.21.15.76:0/3286908978 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f80074e0 con 0x7fa2fc06db20 2023-11-09T03:48:43.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.527+0000 7fa2f2ffd640 1 -- 172.21.15.76:0/3286908978 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f8007640 con 0x7fa2fc06db20 2023-11-09T03:48:43.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.527+0000 7fa30470f640 1 -- 172.21.15.76:0/3286908978 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2fc0aef70 con 0x7fa2fc06db20 2023-11-09T03:48:43.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.527+0000 7fa2f2ffd640 1 -- 172.21.15.76:0/3286908978 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8004a00 con 0x7fa2fc06db20 2023-11-09T03:48:43.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.527+0000 7fa30470f640 1 -- 172.21.15.76:0/3286908978 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2fc0aff90 con 0x7fa2fc06db20 2023-11-09T03:48:43.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.528+0000 7fa30470f640 1 -- 172.21.15.76:0/3286908978 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2fc0b0180 con 0x7fa2fc06db20 2023-11-09T03:48:43.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.528+0000 7fa2f2ffd640 1 -- 172.21.15.76:0/3286908978 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa2f8004b60 con 0x7fa2fc06db20 2023-11-09T03:48:43.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.529+0000 7fa2f2ffd640 1 --2- 172.21.15.76:0/3286908978 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa2e006ae70 0x7fa2e006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:43.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.529+0000 7fa2f2ffd640 1 -- 172.21.15.76:0/3286908978 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7148+0+0 (secure 0 0 0) 0x7fa2f80899b0 con 0x7fa2fc06db20 2023-11-09T03:48:43.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.529+0000 7fa301caa640 1 --2- 172.21.15.76:0/3286908978 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa2e006ae70 0x7fa2e006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:43.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.530+0000 7fa301caa640 1 --2- 172.21.15.76:0/3286908978 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa2e006ae70 0x7fa2e006d330 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec036000 tx=0x7fa2ec008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:43.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.533+0000 7fa2f2ffd640 1 -- 172.21.15.76:0/3286908978 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa2f80544c0 con 0x7fa2fc06db20 2023-11-09T03:48:43.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.700+0000 7fa30470f640 1 -- 172.21.15.76:0/3286908978 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2fc074290 con 0x7fa2fc06db20 2023-11-09T03:48:43.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.702+0000 7fa2f2ffd640 1 -- 172.21.15.76:0/3286908978 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22051 (secure 0 0 0) 0x7fa2f8057ae0 con 0x7fa2fc06db20 2023-11-09T03:48:43.702 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:43.703 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":84,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:48:42.631236+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-09T03:47:59.280601+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-09T03:47:59.742039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:48:20.992533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-11-09T03:48:21.862178+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":84,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:48:43.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.703+0000 7fa30470f640 1 -- 172.21.15.76:0/3286908978 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa2e006ae70 msgr2=0x7fa2e006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:43.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.703+0000 7fa30470f640 1 --2- 172.21.15.76:0/3286908978 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa2e006ae70 0x7fa2e006d330 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec036000 tx=0x7fa2ec008030 comp rx=0 tx=0).stop 2023-11-09T03:48:43.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.704+0000 7fa30470f640 1 -- 172.21.15.76:0/3286908978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2fc06db20 msgr2=0x7fa2fc0afa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:43.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.704+0000 7fa30470f640 1 --2- 172.21.15.76:0/3286908978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2fc06db20 0x7fa2fc0afa50 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fa2f800c3f0 tx=0x7fa2f800c8b0 comp rx=0 tx=0).stop 2023-11-09T03:48:43.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.704+0000 7fa30470f640 1 -- 172.21.15.76:0/3286908978 shutdown_connections 2023-11-09T03:48:43.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.704+0000 7fa30470f640 1 --2- 172.21.15.76:0/3286908978 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa2e006ae70 0x7fa2e006d330 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:43.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.704+0000 7fa30470f640 1 --2- 172.21.15.76:0/3286908978 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2fc159c70 0x7fa2fc1546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:43.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.704+0000 7fa30470f640 1 --2- 172.21.15.76:0/3286908978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2fc0b0ae0 0x7fa2fc154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:43.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.704+0000 7fa30470f640 1 --2- 172.21.15.76:0/3286908978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2fc06db20 0x7fa2fc0afa50 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:43.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.704+0000 7fa30470f640 1 -- 172.21.15.76:0/3286908978 >> 172.21.15.76:0/3286908978 conn(0x7fa2fc0ce950 msgr2=0x7fa2fc0cb060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:43.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.704+0000 7fa30470f640 1 -- 172.21.15.76:0/3286908978 shutdown_connections 2023-11-09T03:48:43.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.704+0000 7fa30470f640 1 -- 172.21.15.76:0/3286908978 wait complete. 2023-11-09T03:48:43.715 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-09T03:48:43.715 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:48:43.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.815+0000 7ff8403fb640 1 Processor -- start 2023-11-09T03:48:43.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.815+0000 7ff8403fb640 1 -- start start 2023-11-09T03:48:43.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.816+0000 7ff8403fb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8380aec80 0x7ff8380b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:43.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.816+0000 7ff8403fb640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff8380af060 0x7ff83806a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:43.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.816+0000 7ff8403fb640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8380b1590 con 0x7ff83806ef40 2023-11-09T03:48:43.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.816+0000 7ff8403fb640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff838155a40 con 0x7ff8380af060 2023-11-09T03:48:43.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.816+0000 7ff8403fb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff83806a660 con 0x7ff8380aec80 2023-11-09T03:48:43.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.816+0000 7ff83d996640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff8380af060 0x7ff83806a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:43.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.816+0000 7ff83d996640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff8380af060 0x7ff83806a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52018/0 (socket says 172.21.15.76:52018) 2023-11-09T03:48:43.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.816+0000 7ff83e998640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff83806ef40 0x7ff8380cb7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48624/0 (socket says 172.21.15.76:48624) 2023-11-09T03:48:43.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.816+0000 7ff83d996640 1 -- 172.21.15.76:0/2412200164 learned_addr learned my addr 172.21.15.76:0/2412200164 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:43.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.817+0000 7ff83e197640 1 --2- 172.21.15.76:0/2412200164 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8380aec80 0x7ff8380b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:43.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.817+0000 7ff83d996640 1 -- 172.21.15.76:0/2412200164 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8380aec80 msgr2=0x7ff8380b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:43.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.817+0000 7ff83d996640 1 --2- 172.21.15.76:0/2412200164 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8380aec80 0x7ff8380b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:43.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.817+0000 7ff83d996640 1 -- 172.21.15.76:0/2412200164 >> v1:172.21.15.76:6789/0 conn(0x7ff83806ef40 legacy=0x7ff8380cb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:43.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.817+0000 7ff83d996640 1 -- 172.21.15.76:0/2412200164 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff83806a8e0 con 0x7ff8380af060 2023-11-09T03:48:43.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.817+0000 7ff83d195640 1 -- 172.21.15.76:0/2412200164 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1272524085 0 0) 0x7ff8380b1590 con 0x7ff83806ef40 2023-11-09T03:48:43.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.817+0000 7ff83e197640 1 --2- 172.21.15.76:0/2412200164 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8380aec80 0x7ff8380b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:43.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.817+0000 7ff83d996640 1 --2- 172.21.15.76:0/2412200164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8380af060 0x7ff83806a120 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7ff828009850 tx=0x7ff82802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b6047948e8d937 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:43.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.817+0000 7ff83d195640 1 -- 172.21.15.76:0/2412200164 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff828004230 con 0x7ff8380af060 2023-11-09T03:48:43.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.818+0000 7ff83d195640 1 -- 172.21.15.76:0/2412200164 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff828004390 con 0x7ff8380af060 2023-11-09T03:48:43.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.818+0000 7ff83d195640 1 -- 172.21.15.76:0/2412200164 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff828003810 con 0x7ff8380af060 2023-11-09T03:48:43.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.818+0000 7ff8403fb640 1 -- 172.21.15.76:0/2412200164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8380af060 msgr2=0x7ff83806a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:43.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.818+0000 7ff8403fb640 1 --2- 172.21.15.76:0/2412200164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8380af060 0x7ff83806a120 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7ff828009850 tx=0x7ff82802b960 comp rx=0 tx=0).stop 2023-11-09T03:48:43.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.818+0000 7ff8403fb640 1 -- 172.21.15.76:0/2412200164 shutdown_connections 2023-11-09T03:48:43.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.818+0000 7ff8403fb640 1 --2- 172.21.15.76:0/2412200164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8380af060 0x7ff83806a120 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:43.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.818+0000 7ff8403fb640 1 --2- 172.21.15.76:0/2412200164 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8380aec80 0x7ff8380b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:43.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.818+0000 7ff8403fb640 1 -- 172.21.15.76:0/2412200164 >> 172.21.15.76:0/2412200164 conn(0x7ff838098d40 msgr2=0x7ff8380cfa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:43.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.818+0000 7ff8403fb640 1 -- 172.21.15.76:0/2412200164 shutdown_connections 2023-11-09T03:48:43.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.818+0000 7ff8403fb640 1 -- 172.21.15.76:0/2412200164 wait complete. 2023-11-09T03:48:43.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.819+0000 7ff8403fb640 1 Processor -- start 2023-11-09T03:48:43.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.819+0000 7ff8403fb640 1 -- start start 2023-11-09T03:48:43.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.820+0000 7ff8403fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff83806ef40 0x7ff8380cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:43.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.820+0000 7ff8403fb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8380aec80 0x7ff8380c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:43.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.820+0000 7ff8403fb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8380af060 0x7ff8380c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:43.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.820+0000 7ff8403fb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8380c6ff0 con 0x7ff83806ef40 2023-11-09T03:48:43.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.820+0000 7ff8403fb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8380c98b0 con 0x7ff8380af060 2023-11-09T03:48:43.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.820+0000 7ff8403fb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8380c7290 con 0x7ff8380aec80 2023-11-09T03:48:43.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.820+0000 7ff83e197640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff83806ef40 0x7ff8380cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:43.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.820+0000 7ff83e197640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff83806ef40 0x7ff8380cf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52026/0 (socket says 172.21.15.76:52026) 2023-11-09T03:48:43.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.820+0000 7ff83e197640 1 -- 172.21.15.76:0/3831222837 learned_addr learned my addr 172.21.15.76:0/3831222837 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:43.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.820+0000 7ff83d996640 1 --2- 172.21.15.76:0/3831222837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8380aec80 0x7ff8380c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:43.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.820+0000 7ff83e998640 1 --2- 172.21.15.76:0/3831222837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8380af060 0x7ff8380c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:43.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.820+0000 7ff83e197640 1 -- 172.21.15.76:0/3831222837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8380aec80 msgr2=0x7ff8380c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:43.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.820+0000 7ff83e197640 1 --2- 172.21.15.76:0/3831222837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8380aec80 0x7ff8380c6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:43.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.820+0000 7ff83e197640 1 -- 172.21.15.76:0/3831222837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8380af060 msgr2=0x7ff8380c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:43.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.820+0000 7ff83e197640 1 --2- 172.21.15.76:0/3831222837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8380af060 0x7ff8380c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:43.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.820+0000 7ff83e197640 1 -- 172.21.15.76:0/3831222837 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff828009460 con 0x7ff83806ef40 2023-11-09T03:48:43.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.821+0000 7ff83d996640 1 --2- 172.21.15.76:0/3831222837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8380aec80 0x7ff8380c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:43.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.821+0000 7ff83e998640 1 --2- 172.21.15.76:0/3831222837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8380af060 0x7ff8380c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:43.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.821+0000 7ff83e197640 1 --2- 172.21.15.76:0/3831222837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff83806ef40 0x7ff8380cf1e0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7ff83400cec0 tx=0x7ff83400a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:43.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.821+0000 7ff82effd640 1 -- 172.21.15.76:0/3831222837 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83400e860 con 0x7ff83806ef40 2023-11-09T03:48:43.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.821+0000 7ff82effd640 1 -- 172.21.15.76:0/3831222837 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff83400e9c0 con 0x7ff83806ef40 2023-11-09T03:48:43.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.821+0000 7ff8403fb640 1 -- 172.21.15.76:0/3831222837 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8380c7570 con 0x7ff83806ef40 2023-11-09T03:48:43.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.821+0000 7ff82effd640 1 -- 172.21.15.76:0/3831222837 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83400ed50 con 0x7ff83806ef40 2023-11-09T03:48:43.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.821+0000 7ff8403fb640 1 -- 172.21.15.76:0/3831222837 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8381f5eb0 con 0x7ff83806ef40 2023-11-09T03:48:43.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.822+0000 7ff8403fb640 1 -- 172.21.15.76:0/3831222837 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff838074290 con 0x7ff83806ef40 2023-11-09T03:48:43.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.822+0000 7ff82effd640 1 -- 172.21.15.76:0/3831222837 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff834016420 con 0x7ff83806ef40 2023-11-09T03:48:43.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.823+0000 7ff82effd640 1 --2- 172.21.15.76:0/3831222837 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff81c06ae70 0x7ff81c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:43.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.823+0000 7ff82effd640 1 -- 172.21.15.76:0/3831222837 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7148+0+0 (secure 0 0 0) 0x7ff834012070 con 0x7ff83806ef40 2023-11-09T03:48:43.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.823+0000 7ff83d996640 1 --2- 172.21.15.76:0/3831222837 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff81c06ae70 0x7ff81c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:43.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.823+0000 7ff83d996640 1 --2- 172.21.15.76:0/3831222837 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff81c06ae70 0x7ff81c06d330 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff828035000 tx=0x7ff828008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:43.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:43.827+0000 7ff82effd640 1 -- 172.21.15.76:0/3831222837 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff83404fca0 con 0x7ff83806ef40 2023-11-09T03:48:44.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.031+0000 7ff8403fb640 1 -- 172.21.15.76:0/3831222837 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8380af6b0 con 0x7ff83806ef40 2023-11-09T03:48:44.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.032+0000 7ff82effd640 1 -- 172.21.15.76:0/3831222837 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4921 (secure 0 0 0) 0x7ff834055370 con 0x7ff83806ef40 2023-11-09T03:48:44.032 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:44.033 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":37,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6097,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2408659865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2408659865},{"type":"v1","addr":"172.21.15.113:6836","nonce":2408659865}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6100,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1873078383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1873078383},{"type":"v1","addr":"172.21.15.113:6837","nonce":1873078383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/3549714130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3549714130},{"type":"v1","addr":"172.21.15.76:6834","nonce":3549714130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:22.895004+0000","modified":"2023-11-09T03:48:37.005363+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6984":{"gid":6984,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/2165197913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":2165197913},{"type":"v1","addr":"172.21.15.76:6835","nonce":2165197913}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-09T03:48:44.033 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 37 2023-11-09T03:48:44.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.034+0000 7ff8403fb640 1 -- 172.21.15.76:0/3831222837 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff81c06ae70 msgr2=0x7ff81c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:44.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.034+0000 7ff8403fb640 1 --2- 172.21.15.76:0/3831222837 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff81c06ae70 0x7ff81c06d330 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff828035000 tx=0x7ff828008030 comp rx=0 tx=0).stop 2023-11-09T03:48:44.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.034+0000 7ff8403fb640 1 -- 172.21.15.76:0/3831222837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff83806ef40 msgr2=0x7ff8380cf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:44.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.034+0000 7ff8403fb640 1 --2- 172.21.15.76:0/3831222837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff83806ef40 0x7ff8380cf1e0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7ff83400cec0 tx=0x7ff83400a6c0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.034+0000 7ff8403fb640 1 -- 172.21.15.76:0/3831222837 shutdown_connections 2023-11-09T03:48:44.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.034+0000 7ff8403fb640 1 --2- 172.21.15.76:0/3831222837 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff81c06ae70 0x7ff81c06d330 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.034+0000 7ff8403fb640 1 --2- 172.21.15.76:0/3831222837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8380af060 0x7ff8380c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.034+0000 7ff8403fb640 1 --2- 172.21.15.76:0/3831222837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8380aec80 0x7ff8380c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.034+0000 7ff8403fb640 1 --2- 172.21.15.76:0/3831222837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff83806ef40 0x7ff8380cf1e0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.034+0000 7ff8403fb640 1 -- 172.21.15.76:0/3831222837 >> 172.21.15.76:0/3831222837 conn(0x7ff838098d40 msgr2=0x7ff8380afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:44.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.034+0000 7ff8403fb640 1 -- 172.21.15.76:0/3831222837 shutdown_connections 2023-11-09T03:48:44.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.034+0000 7ff8403fb640 1 -- 172.21.15.76:0/3831222837 wait complete. 2023-11-09T03:48:44.045 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:48:44.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.143+0000 7fd5defee640 1 Processor -- start 2023-11-09T03:48:44.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.143+0000 7fd5defee640 1 -- start start 2023-11-09T03:48:44.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.144+0000 7fd5defee640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5d815ad80 0x7fd5d806c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:44.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.144+0000 7fd5defee640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5d806cd00 con 0x7fd5d80b42c0 2023-11-09T03:48:44.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.144+0000 7fd5defee640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5d806cfa0 con 0x7fd5d80c9e10 2023-11-09T03:48:44.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.144+0000 7fd5defee640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d806d280 con 0x7fd5d815ad80 2023-11-09T03:48:44.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.144+0000 7fd5d7fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd5d80b42c0 0x7fd5d80c9700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48628/0 (socket says 172.21.15.76:48628) 2023-11-09T03:48:44.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.144+0000 7fd5d7fff640 1 -- 172.21.15.76:0/709352624 learned_addr learned my addr 172.21.15.76:0/709352624 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:44.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.144+0000 7fd5dcd8a640 1 --2- 172.21.15.76:0/709352624 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5d815ad80 0x7fd5d806c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:44.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.144+0000 7fd5d77fe640 1 -- 172.21.15.76:0/709352624 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3673836001 0 0) 0x7fd5d806cd00 con 0x7fd5d80b42c0 2023-11-09T03:48:44.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.144+0000 7fd5d77fe640 1 -- 172.21.15.76:0/709352624 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5c0003520 con 0x7fd5d80b42c0 2023-11-09T03:48:44.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.145+0000 7fd5d77fe640 1 -- 172.21.15.76:0/709352624 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2545862671 0 0) 0x7fd5c0003520 con 0x7fd5d80b42c0 2023-11-09T03:48:44.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.145+0000 7fd5d77fe640 1 -- 172.21.15.76:0/709352624 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5d815ad80 msgr2=0x7fd5d806c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:44.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.145+0000 7fd5d77fe640 1 --2- 172.21.15.76:0/709352624 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5d815ad80 0x7fd5d806c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.145+0000 7fd5d77fe640 1 -- 172.21.15.76:0/709352624 >> v1:172.21.15.113:6789/0 conn(0x7fd5d80c9e10 legacy=0x7fd5d80b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:44.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.145+0000 7fd5d77fe640 1 -- 172.21.15.76:0/709352624 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d806d560 con 0x7fd5d80b42c0 2023-11-09T03:48:44.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.145+0000 7fd5d77fe640 1 -- 172.21.15.76:0/709352624 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fd5cc001690 con 0x7fd5d80b42c0 2023-11-09T03:48:44.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.145+0000 7fd5d77fe640 1 -- 172.21.15.76:0/709352624 >> v1:172.21.15.76:6789/0 conn(0x7fd5d80b42c0 legacy=0x7fd5d80c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:44.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.145+0000 7fd5d77fe640 1 --2- 172.21.15.76:0/709352624 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5c00039a0 0x7fd5c0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:44.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.145+0000 7fd5d77fe640 1 --2- 172.21.15.76:0/709352624 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5c00047c0 0x7fd5c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:44.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.145+0000 7fd5d77fe640 1 --2- 172.21.15.76:0/709352624 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5d815ad80 0x7fd5c0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:44.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.145+0000 7fd5d77fe640 1 -- 172.21.15.76:0/709352624 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d806d280 con 0x7fd5c00039a0 2023-11-09T03:48:44.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.145+0000 7fd5d77fe640 1 -- 172.21.15.76:0/709352624 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c000bac0 con 0x7fd5c00047c0 2023-11-09T03:48:44.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.145+0000 7fd5d77fe640 1 -- 172.21.15.76:0/709352624 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c000bc80 con 0x7fd5d815ad80 2023-11-09T03:48:44.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.145+0000 7fd5dcd8a640 1 --2- 172.21.15.76:0/709352624 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5c00039a0 0x7fd5c0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:44.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.145+0000 7fd5d7fff640 1 --2- 172.21.15.76:0/709352624 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5c00047c0 0x7fd5c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:44.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.145+0000 7fd5dd58b640 1 --2- 172.21.15.76:0/709352624 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5d815ad80 0x7fd5c0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:44.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.145+0000 7fd5dcd8a640 1 -- 172.21.15.76:0/709352624 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5d815ad80 msgr2=0x7fd5c0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:44.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.145+0000 7fd5dcd8a640 1 --2- 172.21.15.76:0/709352624 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5d815ad80 0x7fd5c0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.145+0000 7fd5dcd8a640 1 -- 172.21.15.76:0/709352624 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5c00047c0 msgr2=0x7fd5c0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:44.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.145+0000 7fd5dcd8a640 1 --2- 172.21.15.76:0/709352624 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5c00047c0 0x7fd5c0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.145+0000 7fd5dcd8a640 1 -- 172.21.15.76:0/709352624 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c0003520 con 0x7fd5c00039a0 2023-11-09T03:48:44.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.146+0000 7fd5dcd8a640 1 --2- 172.21.15.76:0/709352624 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5c00039a0 0x7fd5c0003dc0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8009a90 tx=0x7fd5c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d7b0337d850561 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:44.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.146+0000 7fd5d77fe640 1 -- 172.21.15.76:0/709352624 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c802c680 con 0x7fd5c00039a0 2023-11-09T03:48:44.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.146+0000 7fd5d77fe640 1 -- 172.21.15.76:0/709352624 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c802c7e0 con 0x7fd5c00039a0 2023-11-09T03:48:44.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.146+0000 7fd5d77fe640 1 -- 172.21.15.76:0/709352624 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c802cb70 con 0x7fd5c00039a0 2023-11-09T03:48:44.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.146+0000 7fd5defee640 1 -- 172.21.15.76:0/709352624 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5c00039a0 msgr2=0x7fd5c0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:44.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.146+0000 7fd5defee640 1 --2- 172.21.15.76:0/709352624 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5c00039a0 0x7fd5c0003dc0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8009a90 tx=0x7fd5c00057f0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.146+0000 7fd5dd58b640 1 -- 172.21.15.76:0/709352624 reap_dead start 2023-11-09T03:48:44.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.146+0000 7fd5defee640 1 -- 172.21.15.76:0/709352624 shutdown_connections 2023-11-09T03:48:44.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.146+0000 7fd5defee640 1 -- 172.21.15.76:0/709352624 >> 172.21.15.76:0/709352624 conn(0x7fd5d80ce950 msgr2=0x7fd5d80ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:44.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.146+0000 7fd5defee640 1 -- 172.21.15.76:0/709352624 shutdown_connections 2023-11-09T03:48:44.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.146+0000 7fd5defee640 1 -- 172.21.15.76:0/709352624 wait complete. 2023-11-09T03:48:44.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.148+0000 7fd5defee640 1 Processor -- start 2023-11-09T03:48:44.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.148+0000 7fd5defee640 1 -- start start 2023-11-09T03:48:44.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.148+0000 7fd5defee640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5c00039a0 0x7fd5d814dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:44.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.148+0000 7fd5defee640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5d815d1c0 0x7fd5d8155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:44.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.148+0000 7fd5defee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5d80c7b60 0x7fd5d80c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:44.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.149+0000 7fd5defee640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d80c8520 con 0x7fd5c00039a0 2023-11-09T03:48:44.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.149+0000 7fd5defee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d80a7670 con 0x7fd5d80c7b60 2023-11-09T03:48:44.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.149+0000 7fd5defee640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d80a7950 con 0x7fd5d815d1c0 2023-11-09T03:48:44.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.149+0000 7fd5dcd8a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5c00039a0 0x7fd5d814dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:44.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.149+0000 7fd5dcd8a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5c00039a0 0x7fd5d814dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52044/0 (socket says 172.21.15.76:52044) 2023-11-09T03:48:44.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.149+0000 7fd5dcd8a640 1 -- 172.21.15.76:0/2543786882 learned_addr learned my addr 172.21.15.76:0/2543786882 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:44.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.149+0000 7fd5dcd8a640 1 -- 172.21.15.76:0/2543786882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5d815d1c0 msgr2=0x7fd5d8155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:44.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.149+0000 7fd5dcd8a640 1 --2- 172.21.15.76:0/2543786882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5d815d1c0 0x7fd5d8155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.149+0000 7fd5dcd8a640 1 -- 172.21.15.76:0/2543786882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5d80c7b60 msgr2=0x7fd5d80c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:44.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.149+0000 7fd5dcd8a640 1 --2- 172.21.15.76:0/2543786882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5d80c7b60 0x7fd5d80c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.149+0000 7fd5dcd8a640 1 -- 172.21.15.76:0/2543786882 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d80a7c30 con 0x7fd5c00039a0 2023-11-09T03:48:44.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.149+0000 7fd5dcd8a640 1 --2- 172.21.15.76:0/2543786882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5c00039a0 0x7fd5d814dd50 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fd5c802c220 tx=0x7fd5c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:44.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.149+0000 7fd5d57fa640 1 -- 172.21.15.76:0/2543786882 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8003810 con 0x7fd5c00039a0 2023-11-09T03:48:44.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.149+0000 7fd5d57fa640 1 -- 172.21.15.76:0/2543786882 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c8003970 con 0x7fd5c00039a0 2023-11-09T03:48:44.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.149+0000 7fd5defee640 1 -- 172.21.15.76:0/2543786882 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5d80a7f10 con 0x7fd5c00039a0 2023-11-09T03:48:44.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.149+0000 7fd5d57fa640 1 -- 172.21.15.76:0/2543786882 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8003ce0 con 0x7fd5c00039a0 2023-11-09T03:48:44.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.149+0000 7fd5defee640 1 -- 172.21.15.76:0/2543786882 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d814e290 con 0x7fd5c00039a0 2023-11-09T03:48:44.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.150+0000 7fd5defee640 1 -- 172.21.15.76:0/2543786882 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5d8074290 con 0x7fd5c00039a0 2023-11-09T03:48:44.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.151+0000 7fd5d57fa640 1 -- 172.21.15.76:0/2543786882 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd5c8034440 con 0x7fd5c00039a0 2023-11-09T03:48:44.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.151+0000 7fd5d57fa640 1 --2- 172.21.15.76:0/2543786882 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd5c406aea0 0x7fd5c406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:44.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.151+0000 7fd5d57fa640 1 -- 172.21.15.76:0/2543786882 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7148+0+0 (secure 0 0 0) 0x7fd5c80ab9b0 con 0x7fd5c00039a0 2023-11-09T03:48:44.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.152+0000 7fd5d7fff640 1 --2- 172.21.15.76:0/2543786882 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd5c406aea0 0x7fd5c406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:44.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.152+0000 7fd5d7fff640 1 --2- 172.21.15.76:0/2543786882 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd5c406aea0 0x7fd5c406d360 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc001220 tx=0x7fd5cc001150 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:44.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.156+0000 7fd5d57fa640 1 -- 172.21.15.76:0/2543786882 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd5c8076570 con 0x7fd5c00039a0 2023-11-09T03:48:44.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.356+0000 7fd5defee640 1 -- 172.21.15.76:0/2543786882 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5d80c8520 con 0x7fd5c00039a0 2023-11-09T03:48:44.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.357+0000 7fd5d57fa640 1 -- 172.21.15.76:0/2543786882 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fd5d80c8520 con 0x7fd5c00039a0 2023-11-09T03:48:44.358 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:44.358 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":37,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6097,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2408659865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2408659865},{"type":"v1","addr":"172.21.15.113:6836","nonce":2408659865}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6100,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1873078383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1873078383},{"type":"v1","addr":"172.21.15.113:6837","nonce":1873078383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/3549714130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3549714130},{"type":"v1","addr":"172.21.15.76:6834","nonce":3549714130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:22.895004+0000","modified":"2023-11-09T03:48:37.005363+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6984":{"gid":6984,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/2165197913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":2165197913},{"type":"v1","addr":"172.21.15.76:6835","nonce":2165197913}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-09T03:48:44.358 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 37 2023-11-09T03:48:44.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.359+0000 7fd5defee640 1 -- 172.21.15.76:0/2543786882 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd5c406aea0 msgr2=0x7fd5c406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:44.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.359+0000 7fd5defee640 1 --2- 172.21.15.76:0/2543786882 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd5c406aea0 0x7fd5c406d360 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc001220 tx=0x7fd5cc001150 comp rx=0 tx=0).stop 2023-11-09T03:48:44.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.359+0000 7fd5defee640 1 -- 172.21.15.76:0/2543786882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5c00039a0 msgr2=0x7fd5d814dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:44.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.359+0000 7fd5defee640 1 --2- 172.21.15.76:0/2543786882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5c00039a0 0x7fd5d814dd50 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fd5c802c220 tx=0x7fd5c8004730 comp rx=0 tx=0).stop 2023-11-09T03:48:44.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.359+0000 7fd5defee640 1 -- 172.21.15.76:0/2543786882 shutdown_connections 2023-11-09T03:48:44.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.359+0000 7fd5defee640 1 --2- 172.21.15.76:0/2543786882 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd5c406aea0 0x7fd5c406d360 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.359+0000 7fd5defee640 1 --2- 172.21.15.76:0/2543786882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5d80c7b60 0x7fd5d80c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.359+0000 7fd5defee640 1 --2- 172.21.15.76:0/2543786882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5d815d1c0 0x7fd5d8155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.359+0000 7fd5defee640 1 --2- 172.21.15.76:0/2543786882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5c00039a0 0x7fd5d814dd50 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.359+0000 7fd5defee640 1 -- 172.21.15.76:0/2543786882 >> 172.21.15.76:0/2543786882 conn(0x7fd5d80ce950 msgr2=0x7fd5d80aea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:44.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.359+0000 7fd5defee640 1 -- 172.21.15.76:0/2543786882 shutdown_connections 2023-11-09T03:48:44.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.359+0000 7fd5defee640 1 -- 172.21.15.76:0/2543786882 wait complete. 2023-11-09T03:48:44.370 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:48:44.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.470+0000 7fd6e50a2640 1 Processor -- start 2023-11-09T03:48:44.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.470+0000 7fd6e50a2640 1 -- start start 2023-11-09T03:48:44.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.471+0000 7fd6e50a2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd6e015ad80 0x7fd6e006c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:44.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.471+0000 7fd6e50a2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd6e00b42c0 0x7fd6e00c9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:44.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.471+0000 7fd6e50a2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6e00c9c40 0x7fd6e0070630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:44.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.471+0000 7fd6e50a2640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e00ca050 con 0x7fd6e015ad80 2023-11-09T03:48:44.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.471+0000 7fd6e50a2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e006cd00 con 0x7fd6e00b42c0 2023-11-09T03:48:44.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.471+0000 7fd6e50a2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e006cfa0 con 0x7fd6e00c9c40 2023-11-09T03:48:44.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.471+0000 7fd6ded9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd6e015ad80 0x7fd6e006c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:44.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.471+0000 7fd6ded9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd6e015ad80 0x7fd6e006c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52060/0 (socket says 172.21.15.76:52060) 2023-11-09T03:48:44.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.471+0000 7fd6ded9d640 1 -- 172.21.15.76:0/828379477 learned_addr learned my addr 172.21.15.76:0/828379477 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:44.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.471+0000 7fd6df59e640 1 --2- 172.21.15.76:0/828379477 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6e00c9c40 0x7fd6e0070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:44.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.471+0000 7fd6d7fff640 1 --2- 172.21.15.76:0/828379477 >> v2:172.21.15.113:3300/0 conn(0x7fd6e00b42c0 0x7fd6e00c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:44.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.471+0000 7fd6ded9d640 1 -- 172.21.15.76:0/828379477 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6e00c9c40 msgr2=0x7fd6e0070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:44.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.471+0000 7fd6ded9d640 1 --2- 172.21.15.76:0/828379477 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6e00c9c40 0x7fd6e0070630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.471+0000 7fd6ded9d640 1 -- 172.21.15.76:0/828379477 >> v2:172.21.15.113:3300/0 conn(0x7fd6e00b42c0 msgr2=0x7fd6e00c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:44.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.471+0000 7fd6ded9d640 1 --2- 172.21.15.76:0/828379477 >> v2:172.21.15.113:3300/0 conn(0x7fd6e00b42c0 0x7fd6e00c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.471+0000 7fd6ded9d640 1 -- 172.21.15.76:0/828379477 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e006d280 con 0x7fd6e015ad80 2023-11-09T03:48:44.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.471+0000 7fd6df59e640 1 --2- 172.21.15.76:0/828379477 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6e00c9c40 0x7fd6e0070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:44.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.471+0000 7fd6d7fff640 1 --2- 172.21.15.76:0/828379477 >> v2:172.21.15.113:3300/0 conn(0x7fd6e00b42c0 0x7fd6e00c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:44.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.472+0000 7fd6ded9d640 1 --2- 172.21.15.76:0/828379477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6e015ad80 0x7fd6e006c7c0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00c8e50 tx=0x7fd6d002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb239eddf2be4a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:44.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.472+0000 7fd6de59c640 1 -- 172.21.15.76:0/828379477 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0030860 con 0x7fd6e015ad80 2023-11-09T03:48:44.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.472+0000 7fd6de59c640 1 -- 172.21.15.76:0/828379477 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d00309c0 con 0x7fd6e015ad80 2023-11-09T03:48:44.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.472+0000 7fd6de59c640 1 -- 172.21.15.76:0/828379477 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0030d50 con 0x7fd6e015ad80 2023-11-09T03:48:44.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.472+0000 7fd6e50a2640 1 -- 172.21.15.76:0/828379477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6e015ad80 msgr2=0x7fd6e006c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:44.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.472+0000 7fd6e50a2640 1 --2- 172.21.15.76:0/828379477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6e015ad80 0x7fd6e006c7c0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00c8e50 tx=0x7fd6d002b9a0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.472+0000 7fd6e50a2640 1 -- 172.21.15.76:0/828379477 shutdown_connections 2023-11-09T03:48:44.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.472+0000 7fd6e50a2640 1 --2- 172.21.15.76:0/828379477 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6e00c9c40 0x7fd6e0070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.472+0000 7fd6e50a2640 1 --2- 172.21.15.76:0/828379477 >> v2:172.21.15.113:3300/0 conn(0x7fd6e00b42c0 0x7fd6e00c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.472+0000 7fd6e50a2640 1 --2- 172.21.15.76:0/828379477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6e015ad80 0x7fd6e006c7c0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.472+0000 7fd6e50a2640 1 -- 172.21.15.76:0/828379477 >> 172.21.15.76:0/828379477 conn(0x7fd6e00ce950 msgr2=0x7fd6e00ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:44.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.472+0000 7fd6e50a2640 1 -- 172.21.15.76:0/828379477 shutdown_connections 2023-11-09T03:48:44.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.472+0000 7fd6e50a2640 1 -- 172.21.15.76:0/828379477 wait complete. 2023-11-09T03:48:44.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.474+0000 7fd6e50a2640 1 Processor -- start 2023-11-09T03:48:44.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.474+0000 7fd6e50a2640 1 -- start start 2023-11-09T03:48:44.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.474+0000 7fd6e50a2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6e00b42c0 0x7fd6e014be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:44.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.474+0000 7fd6e50a2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6e00c9c40 0x7fd6e015b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:44.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.474+0000 7fd6e50a2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6e015ad80 0x7fd6e015cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:44.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.475+0000 7fd6e50a2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e015c350 con 0x7fd6e00c9c40 2023-11-09T03:48:44.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.475+0000 7fd6e50a2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e014c370 con 0x7fd6e00b42c0 2023-11-09T03:48:44.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.475+0000 7fd6e50a2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e015bab0 con 0x7fd6e015ad80 2023-11-09T03:48:44.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.475+0000 7fd6d7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6e00c9c40 0x7fd6e015b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:44.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.475+0000 7fd6d7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6e00c9c40 0x7fd6e015b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52066/0 (socket says 172.21.15.76:52066) 2023-11-09T03:48:44.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.475+0000 7fd6d7fff640 1 -- 172.21.15.76:0/133755799 learned_addr learned my addr 172.21.15.76:0/133755799 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:44.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.475+0000 7fd6df59e640 1 --2- 172.21.15.76:0/133755799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6e015ad80 0x7fd6e015cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:44.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.475+0000 7fd6ded9d640 1 --2- 172.21.15.76:0/133755799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6e00b42c0 0x7fd6e014be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:44.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.475+0000 7fd6d7fff640 1 -- 172.21.15.76:0/133755799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6e015ad80 msgr2=0x7fd6e015cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:44.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.475+0000 7fd6d7fff640 1 --2- 172.21.15.76:0/133755799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6e015ad80 0x7fd6e015cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.475+0000 7fd6d7fff640 1 -- 172.21.15.76:0/133755799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6e00b42c0 msgr2=0x7fd6e014be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:44.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.475+0000 7fd6d7fff640 1 --2- 172.21.15.76:0/133755799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6e00b42c0 0x7fd6e014be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.475+0000 7fd6d7fff640 1 -- 172.21.15.76:0/133755799 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d00094d0 con 0x7fd6e00c9c40 2023-11-09T03:48:44.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.475+0000 7fd6d7fff640 1 --2- 172.21.15.76:0/133755799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6e00c9c40 0x7fd6e015b160 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800c7f0 tx=0x7fd6c800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:44.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.475+0000 7fd6d77fe640 1 -- 172.21.15.76:0/133755799 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8007820 con 0x7fd6e00c9c40 2023-11-09T03:48:44.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.475+0000 7fd6d77fe640 1 -- 172.21.15.76:0/133755799 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c8007980 con 0x7fd6e00c9c40 2023-11-09T03:48:44.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.475+0000 7fd6d77fe640 1 -- 172.21.15.76:0/133755799 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8007d30 con 0x7fd6e00c9c40 2023-11-09T03:48:44.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.475+0000 7fd6e50a2640 1 -- 172.21.15.76:0/133755799 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e015bd90 con 0x7fd6e00c9c40 2023-11-09T03:48:44.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.475+0000 7fd6e50a2640 1 -- 172.21.15.76:0/133755799 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e00c5b30 con 0x7fd6e00c9c40 2023-11-09T03:48:44.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.476+0000 7fd6e50a2640 1 -- 172.21.15.76:0/133755799 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e00c5990 con 0x7fd6e00c9c40 2023-11-09T03:48:44.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.477+0000 7fd6d77fe640 1 -- 172.21.15.76:0/133755799 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd6c80113f0 con 0x7fd6e00c9c40 2023-11-09T03:48:44.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.477+0000 7fd6d77fe640 1 --2- 172.21.15.76:0/133755799 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd6c006ae70 0x7fd6c006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:44.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.477+0000 7fd6d77fe640 1 -- 172.21.15.76:0/133755799 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7148+0+0 (secure 0 0 0) 0x7fd6c8089f80 con 0x7fd6e00c9c40 2023-11-09T03:48:44.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.477+0000 7fd6ded9d640 1 --2- 172.21.15.76:0/133755799 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd6c006ae70 0x7fd6c006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:44.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.478+0000 7fd6ded9d640 1 --2- 172.21.15.76:0/133755799 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd6c006ae70 0x7fd6c006d330 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0036000 tx=0x7fd6d0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:44.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.481+0000 7fd6d77fe640 1 -- 172.21.15.76:0/133755799 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd6c80bcca0 con 0x7fd6e00c9c40 2023-11-09T03:48:44.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.650+0000 7fd6e50a2640 1 -- 172.21.15.76:0/133755799 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6e0074290 con 0x7fd6e00c9c40 2023-11-09T03:48:44.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.651+0000 7fd6d77fe640 1 -- 172.21.15.76:0/133755799 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22051 (secure 0 0 0) 0x7fd6e00c5990 con 0x7fd6e00c9c40 2023-11-09T03:48:44.652 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:44.652 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":84,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:48:42.631236+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-09T03:47:59.280601+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-09T03:47:59.742039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:48:20.992533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-11-09T03:48:21.862178+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":84,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:48:44.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.653+0000 7fd6e50a2640 1 -- 172.21.15.76:0/133755799 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd6c006ae70 msgr2=0x7fd6c006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:44.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.653+0000 7fd6e50a2640 1 --2- 172.21.15.76:0/133755799 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd6c006ae70 0x7fd6c006d330 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0036000 tx=0x7fd6d0008030 comp rx=0 tx=0).stop 2023-11-09T03:48:44.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.653+0000 7fd6e50a2640 1 -- 172.21.15.76:0/133755799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6e00c9c40 msgr2=0x7fd6e015b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:44.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.653+0000 7fd6e50a2640 1 --2- 172.21.15.76:0/133755799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6e00c9c40 0x7fd6e015b160 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800c7f0 tx=0x7fd6c800ccb0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.653+0000 7fd6e50a2640 1 -- 172.21.15.76:0/133755799 shutdown_connections 2023-11-09T03:48:44.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.653+0000 7fd6e50a2640 1 --2- 172.21.15.76:0/133755799 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd6c006ae70 0x7fd6c006d330 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.653+0000 7fd6e50a2640 1 --2- 172.21.15.76:0/133755799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6e015ad80 0x7fd6e015cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.653+0000 7fd6e50a2640 1 --2- 172.21.15.76:0/133755799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6e00c9c40 0x7fd6e015b160 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.653+0000 7fd6e50a2640 1 --2- 172.21.15.76:0/133755799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6e00b42c0 0x7fd6e014be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.653+0000 7fd6e50a2640 1 -- 172.21.15.76:0/133755799 >> 172.21.15.76:0/133755799 conn(0x7fd6e00ce950 msgr2=0x7fd6e00ccf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:44.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.653+0000 7fd6e50a2640 1 -- 172.21.15.76:0/133755799 shutdown_connections 2023-11-09T03:48:44.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.653+0000 7fd6e50a2640 1 -- 172.21.15.76:0/133755799 wait complete. 2023-11-09T03:48:44.665 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-09T03:48:44.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.765+0000 7fe5a326f640 1 Processor -- start 2023-11-09T03:48:44.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.765+0000 7fe5a326f640 1 -- start start 2023-11-09T03:48:44.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.766+0000 7fe5a326f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe59c159c00 0x7fe59c0cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:44.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.766+0000 7fe5a326f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe59c0c48e0 0x7fe59c0c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:44.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.766+0000 7fe5a326f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe59c15aff0 con 0x7fe59c0adac0 2023-11-09T03:48:44.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.766+0000 7fe5a326f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe59c0c52a0 con 0x7fe59c0c48e0 2023-11-09T03:48:44.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.766+0000 7fe5a326f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c15a3e0 con 0x7fe59c159c00 2023-11-09T03:48:44.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.766+0000 7fe5a080a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe59c0c48e0 0x7fe59c0c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:44.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.766+0000 7fe5a080a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe59c0c48e0 0x7fe59c0c4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52076/0 (socket says 172.21.15.76:52076) 2023-11-09T03:48:44.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.766+0000 7fe5a080a640 1 -- 172.21.15.76:0/3809204779 learned_addr learned my addr 172.21.15.76:0/3809204779 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:44.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.766+0000 7fe5a080a640 1 -- 172.21.15.76:0/3809204779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe59c159c00 msgr2=0x7fe59c0cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:44.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.766+0000 7fe5a100b640 1 --2- 172.21.15.76:0/3809204779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe59c159c00 0x7fe59c0cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:44.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.766+0000 7fe5a080a640 1 --2- 172.21.15.76:0/3809204779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe59c159c00 0x7fe59c0cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.766+0000 7fe5a080a640 1 -- 172.21.15.76:0/3809204779 >> v1:172.21.15.113:6789/0 conn(0x7fe59c0adac0 legacy=0x7fe59c0adef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:44.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.766+0000 7fe5a080a640 1 -- 172.21.15.76:0/3809204779 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59c15a6c0 con 0x7fe59c0c48e0 2023-11-09T03:48:44.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.766+0000 7fe5a100b640 1 --2- 172.21.15.76:0/3809204779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe59c159c00 0x7fe59c0cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:44.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.767+0000 7fe5a080a640 1 --2- 172.21.15.76:0/3809204779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe59c0c48e0 0x7fe59c0c4d60 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fe58c00bb10 tx=0x7fe58c02e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0fe2e46fb2b1c63 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:44.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.767+0000 7fe593fff640 1 -- 172.21.15.76:0/3809204779 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c02ee70 con 0x7fe59c0c48e0 2023-11-09T03:48:44.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.767+0000 7fe593fff640 1 -- 172.21.15.76:0/3809204779 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58c036440 con 0x7fe59c0c48e0 2023-11-09T03:48:44.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.767+0000 7fe593fff640 1 -- 172.21.15.76:0/3809204779 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c0367d0 con 0x7fe59c0c48e0 2023-11-09T03:48:44.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.767+0000 7fe5a326f640 1 -- 172.21.15.76:0/3809204779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe59c0c48e0 msgr2=0x7fe59c0c4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:44.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.767+0000 7fe5a326f640 1 --2- 172.21.15.76:0/3809204779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe59c0c48e0 0x7fe59c0c4d60 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fe58c00bb10 tx=0x7fe58c02e0c0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.768+0000 7fe5a326f640 1 -- 172.21.15.76:0/3809204779 shutdown_connections 2023-11-09T03:48:44.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.768+0000 7fe5a326f640 1 --2- 172.21.15.76:0/3809204779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe59c0c48e0 0x7fe59c0c4d60 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.768+0000 7fe5a326f640 1 --2- 172.21.15.76:0/3809204779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe59c159c00 0x7fe59c0cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.768+0000 7fe5a326f640 1 -- 172.21.15.76:0/3809204779 >> 172.21.15.76:0/3809204779 conn(0x7fe59c0ce950 msgr2=0x7fe59c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:44.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.768+0000 7fe5a326f640 1 -- 172.21.15.76:0/3809204779 shutdown_connections 2023-11-09T03:48:44.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.768+0000 7fe5a326f640 1 -- 172.21.15.76:0/3809204779 wait complete. 2023-11-09T03:48:44.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.769+0000 7fe5a326f640 1 Processor -- start 2023-11-09T03:48:44.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.769+0000 7fe5a326f640 1 -- start start 2023-11-09T03:48:44.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.769+0000 7fe5a326f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe59c0adac0 0x7fe59c158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:44.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.769+0000 7fe5a326f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe59c0c48e0 0x7fe59c0aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:44.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.769+0000 7fe5a326f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe59c159c00 0x7fe59c149260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:44.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.769+0000 7fe5a326f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c1579e0 con 0x7fe59c0c48e0 2023-11-09T03:48:44.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.769+0000 7fe5a326f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c157c80 con 0x7fe59c159c00 2023-11-09T03:48:44.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.769+0000 7fe5a326f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c157f60 con 0x7fe59c0adac0 2023-11-09T03:48:44.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.770+0000 7fe5a080a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe59c0c48e0 0x7fe59c0aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:44.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.770+0000 7fe5a080a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe59c0c48e0 0x7fe59c0aef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52082/0 (socket says 172.21.15.76:52082) 2023-11-09T03:48:44.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.770+0000 7fe5a080a640 1 -- 172.21.15.76:0/2311421212 learned_addr learned my addr 172.21.15.76:0/2311421212 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:44.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.770+0000 7fe5a180c640 1 --2- 172.21.15.76:0/2311421212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe59c159c00 0x7fe59c149260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:44.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.770+0000 7fe5a100b640 1 --2- 172.21.15.76:0/2311421212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe59c0adac0 0x7fe59c158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:44.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.770+0000 7fe5a080a640 1 -- 172.21.15.76:0/2311421212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe59c0adac0 msgr2=0x7fe59c158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:44.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.770+0000 7fe5a080a640 1 --2- 172.21.15.76:0/2311421212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe59c0adac0 0x7fe59c158a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.770+0000 7fe5a080a640 1 -- 172.21.15.76:0/2311421212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe59c159c00 msgr2=0x7fe59c149260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:44.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.770+0000 7fe5a080a640 1 --2- 172.21.15.76:0/2311421212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe59c159c00 0x7fe59c149260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.770+0000 7fe5a080a640 1 -- 172.21.15.76:0/2311421212 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe58c00b770 con 0x7fe59c0c48e0 2023-11-09T03:48:44.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.770+0000 7fe5a080a640 1 --2- 172.21.15.76:0/2311421212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe59c0c48e0 0x7fe59c0aef50 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fe58c02e100 tx=0x7fe58c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:44.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.770+0000 7fe591ffb640 1 -- 172.21.15.76:0/2311421212 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c034b80 con 0x7fe59c0c48e0 2023-11-09T03:48:44.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.770+0000 7fe591ffb640 1 -- 172.21.15.76:0/2311421212 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58c034ce0 con 0x7fe59c0c48e0 2023-11-09T03:48:44.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.770+0000 7fe5a326f640 1 -- 172.21.15.76:0/2311421212 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe59c158240 con 0x7fe59c0c48e0 2023-11-09T03:48:44.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.770+0000 7fe591ffb640 1 -- 172.21.15.76:0/2311421212 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c041680 con 0x7fe59c0c48e0 2023-11-09T03:48:44.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.770+0000 7fe5a326f640 1 -- 172.21.15.76:0/2311421212 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe59c158f80 con 0x7fe59c0c48e0 2023-11-09T03:48:44.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.771+0000 7fe5a326f640 1 -- 172.21.15.76:0/2311421212 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe59c158760 con 0x7fe59c0c48e0 2023-11-09T03:48:44.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.772+0000 7fe591ffb640 1 -- 172.21.15.76:0/2311421212 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe58c03d040 con 0x7fe59c0c48e0 2023-11-09T03:48:44.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.772+0000 7fe591ffb640 1 --2- 172.21.15.76:0/2311421212 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe58006ae70 0x7fe58006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:44.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.772+0000 7fe591ffb640 1 -- 172.21.15.76:0/2311421212 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7148+0+0 (secure 0 0 0) 0x7fe58c0aeba0 con 0x7fe59c0c48e0 2023-11-09T03:48:44.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.772+0000 7fe5a100b640 1 --2- 172.21.15.76:0/2311421212 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe58006ae70 0x7fe58006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:44.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.773+0000 7fe5a100b640 1 --2- 172.21.15.76:0/2311421212 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe58006ae70 0x7fe58006d330 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe59800b840 tx=0x7fe598008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:44.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.776+0000 7fe591ffb640 1 -- 172.21.15.76:0/2311421212 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe59c158760 con 0x7fe59c0c48e0 2023-11-09T03:48:44.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.980+0000 7fe5a326f640 1 -- 172.21.15.76:0/2311421212 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe59c158760 con 0x7fe59c0c48e0 2023-11-09T03:48:44.981 INFO:tasks.ceph.mon.a.smithi076.stderr:2023-11-09T03:48:44.981+0000 7fe4a393d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-09T03:48:44.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.984+0000 7fe591ffb640 1 -- 172.21.15.76:0/2311421212 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe59c158760 con 0x7fe59c0c48e0 2023-11-09T03:48:44.985 INFO:teuthology.orchestra.run.smithi076.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-09T03:48:44.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.987+0000 7fe5a326f640 1 -- 172.21.15.76:0/2311421212 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe58006ae70 msgr2=0x7fe58006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:44.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.987+0000 7fe5a326f640 1 --2- 172.21.15.76:0/2311421212 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe58006ae70 0x7fe58006d330 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe59800b840 tx=0x7fe598008040 comp rx=0 tx=0).stop 2023-11-09T03:48:44.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.987+0000 7fe5a326f640 1 -- 172.21.15.76:0/2311421212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe59c0c48e0 msgr2=0x7fe59c0aef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:44.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.987+0000 7fe5a326f640 1 --2- 172.21.15.76:0/2311421212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe59c0c48e0 0x7fe59c0aef50 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fe58c02e100 tx=0x7fe58c0046c0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.987+0000 7fe5a326f640 1 -- 172.21.15.76:0/2311421212 shutdown_connections 2023-11-09T03:48:44.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.987+0000 7fe5a326f640 1 --2- 172.21.15.76:0/2311421212 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe58006ae70 0x7fe58006d330 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.987+0000 7fe5a326f640 1 --2- 172.21.15.76:0/2311421212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe59c159c00 0x7fe59c149260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.987+0000 7fe5a326f640 1 --2- 172.21.15.76:0/2311421212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe59c0c48e0 0x7fe59c0aef50 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.987+0000 7fe5a326f640 1 --2- 172.21.15.76:0/2311421212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe59c0adac0 0x7fe59c158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:44.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.987+0000 7fe5a326f640 1 -- 172.21.15.76:0/2311421212 >> 172.21.15.76:0/2311421212 conn(0x7fe59c0ce950 msgr2=0x7fe59c0cb340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:44.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.987+0000 7fe5a326f640 1 -- 172.21.15.76:0/2311421212 shutdown_connections 2023-11-09T03:48:44.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:44.987+0000 7fe5a326f640 1 -- 172.21.15.76:0/2311421212 wait complete. 2023-11-09T03:48:44.990 INFO:tasks.ceph.mds.c.smithi076.stderr: -2790> 2023-11-09T03:48:27.804+0000 7f719d522640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:48:45.000 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-09T03:48:45.019 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2023-11-09T03:48:45.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.102+0000 7f4d540f3640 1 Processor -- start 2023-11-09T03:48:45.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.102+0000 7f4d540f3640 1 -- start start 2023-11-09T03:48:45.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.103+0000 7f4d540f3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4d4c159c00 0x7f4d4c0cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:45.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.103+0000 7f4d540f3640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d4c15aff0 con 0x7f4d4c0c48e0 2023-11-09T03:48:45.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.103+0000 7f4d540f3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d4c15a3e0 con 0x7f4d4c0ae1d0 2023-11-09T03:48:45.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.103+0000 7f4d540f3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d4c15a6c0 con 0x7f4d4c159c00 2023-11-09T03:48:45.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.103+0000 7f4d5168e640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f4d4c0c48e0 0x7f4d4c0adac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48642/0 (socket says 172.21.15.76:48642) 2023-11-09T03:48:45.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.103+0000 7f4d5168e640 1 -- 172.21.15.76:0/4150972359 learned_addr learned my addr 172.21.15.76:0/4150972359 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:45.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.103+0000 7f4d51e8f640 1 --2- 172.21.15.76:0/4150972359 >> v2:172.21.15.113:3300/0 conn(0x7f4d4c159c00 0x7f4d4c0cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:45.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.103+0000 7f4d50e8d640 1 -- 172.21.15.76:0/4150972359 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1638759677 0 0) 0x7f4d4c15aff0 con 0x7f4d4c0c48e0 2023-11-09T03:48:45.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.103+0000 7f4d50e8d640 1 -- 172.21.15.76:0/4150972359 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d34003520 con 0x7f4d4c0c48e0 2023-11-09T03:48:45.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.103+0000 7f4d50e8d640 1 -- 172.21.15.76:0/4150972359 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2195564796 0 0) 0x7f4d4c15a3e0 con 0x7f4d4c0ae1d0 2023-11-09T03:48:45.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.103+0000 7f4d50e8d640 1 -- 172.21.15.76:0/4150972359 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d4c15aff0 con 0x7f4d4c0ae1d0 2023-11-09T03:48:45.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.103+0000 7f4d51e8f640 1 -- 172.21.15.76:0/4150972359 >> v1:172.21.15.113:6789/0 conn(0x7f4d4c0ae1d0 legacy=0x7f4d4c0709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:45.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.103+0000 7f4d51e8f640 1 -- 172.21.15.76:0/4150972359 >> v1:172.21.15.76:6789/0 conn(0x7f4d4c0c48e0 legacy=0x7f4d4c0adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:45.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.103+0000 7f4d51e8f640 1 -- 172.21.15.76:0/4150972359 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4c15a9a0 con 0x7f4d4c159c00 2023-11-09T03:48:45.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.104+0000 7f4d51e8f640 1 --2- 172.21.15.76:0/4150972359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d4c159c00 0x7f4d4c0cc890 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4d400097a0 tx=0x7f4d4002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df046a7d39c87607 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:45.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.104+0000 7f4d50e8d640 1 -- 172.21.15.76:0/4150972359 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d40031830 con 0x7f4d4c159c00 2023-11-09T03:48:45.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.104+0000 7f4d50e8d640 1 -- 172.21.15.76:0/4150972359 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d40031990 con 0x7f4d4c159c00 2023-11-09T03:48:45.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.104+0000 7f4d50e8d640 1 -- 172.21.15.76:0/4150972359 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d40031d00 con 0x7f4d4c159c00 2023-11-09T03:48:45.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.105+0000 7f4d540f3640 1 -- 172.21.15.76:0/4150972359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d4c159c00 msgr2=0x7f4d4c0cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:45.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.105+0000 7f4d540f3640 1 --2- 172.21.15.76:0/4150972359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d4c159c00 0x7f4d4c0cc890 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4d400097a0 tx=0x7f4d4002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:48:45.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.105+0000 7f4d540f3640 1 -- 172.21.15.76:0/4150972359 shutdown_connections 2023-11-09T03:48:45.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.105+0000 7f4d540f3640 1 --2- 172.21.15.76:0/4150972359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d4c159c00 0x7f4d4c0cc890 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:45.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.105+0000 7f4d540f3640 1 -- 172.21.15.76:0/4150972359 >> 172.21.15.76:0/4150972359 conn(0x7f4d4c0ce950 msgr2=0x7f4d4c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:45.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.105+0000 7f4d540f3640 1 -- 172.21.15.76:0/4150972359 shutdown_connections 2023-11-09T03:48:45.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.105+0000 7f4d540f3640 1 -- 172.21.15.76:0/4150972359 wait complete. 2023-11-09T03:48:45.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.106+0000 7f4d540f3640 1 Processor -- start 2023-11-09T03:48:45.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.106+0000 7f4d540f3640 1 -- start start 2023-11-09T03:48:45.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.107+0000 7f4d540f3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d4c0ae1d0 0x7f4d4c14e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:45.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.107+0000 7f4d540f3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d4c0c48e0 0x7f4d4c152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:45.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.107+0000 7f4d540f3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d4c159c00 0x7f4d4c1578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:45.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.107+0000 7f4d540f3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c14e660 con 0x7f4d4c0c48e0 2023-11-09T03:48:45.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.107+0000 7f4d540f3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c152650 con 0x7f4d4c159c00 2023-11-09T03:48:45.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.107+0000 7f4d540f3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c14e9e0 con 0x7f4d4c0ae1d0 2023-11-09T03:48:45.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.107+0000 7f4d5168e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d4c0c48e0 0x7f4d4c152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:45.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.107+0000 7f4d5168e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d4c0c48e0 0x7f4d4c152110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52108/0 (socket says 172.21.15.76:52108) 2023-11-09T03:48:45.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.107+0000 7f4d5168e640 1 -- 172.21.15.76:0/3029653398 learned_addr learned my addr 172.21.15.76:0/3029653398 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:45.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.107+0000 7f4d5168e640 1 -- 172.21.15.76:0/3029653398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d4c0ae1d0 msgr2=0x7f4d4c14e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:45.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.107+0000 7f4d5168e640 1 --2- 172.21.15.76:0/3029653398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d4c0ae1d0 0x7f4d4c14e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:45.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.107+0000 7f4d5168e640 1 -- 172.21.15.76:0/3029653398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d4c159c00 msgr2=0x7f4d4c1578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:45.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.107+0000 7f4d5168e640 1 --2- 172.21.15.76:0/3029653398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d4c159c00 0x7f4d4c1578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:45.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.107+0000 7f4d5168e640 1 -- 172.21.15.76:0/3029653398 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d40009400 con 0x7f4d4c0c48e0 2023-11-09T03:48:45.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.107+0000 7f4d5168e640 1 --2- 172.21.15.76:0/3029653398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d4c0c48e0 0x7f4d4c152110 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c00cbe0 tx=0x7f4d3c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:45.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.108+0000 7f4d4a7fc640 1 -- 172.21.15.76:0/3029653398 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c00ad40 con 0x7f4d4c0c48e0 2023-11-09T03:48:45.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.108+0000 7f4d4a7fc640 1 -- 172.21.15.76:0/3029653398 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d3c012400 con 0x7f4d4c0c48e0 2023-11-09T03:48:45.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.108+0000 7f4d4a7fc640 1 -- 172.21.15.76:0/3029653398 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c00aea0 con 0x7f4d4c0c48e0 2023-11-09T03:48:45.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.108+0000 7f4d540f3640 1 -- 172.21.15.76:0/3029653398 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d4c1528a0 con 0x7f4d4c0c48e0 2023-11-09T03:48:45.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.108+0000 7f4d540f3640 1 -- 172.21.15.76:0/3029653398 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d4c1f0f00 con 0x7f4d4c0c48e0 2023-11-09T03:48:45.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.109+0000 7f4d540f3640 1 -- 172.21.15.76:0/3029653398 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d4c074290 con 0x7f4d4c0c48e0 2023-11-09T03:48:45.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.109+0000 7f4d4a7fc640 1 -- 172.21.15.76:0/3029653398 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4d3c007070 con 0x7f4d4c0c48e0 2023-11-09T03:48:45.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.109+0000 7f4d4a7fc640 1 --2- 172.21.15.76:0/3029653398 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d3806ae70 0x7f4d3806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:45.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.109+0000 7f4d4a7fc640 1 -- 172.21.15.76:0/3029653398 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7206+0+0 (secure 0 0 0) 0x7f4d3c089ae0 con 0x7f4d4c0c48e0 2023-11-09T03:48:45.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.110+0000 7f4d51e8f640 1 --2- 172.21.15.76:0/3029653398 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d3806ae70 0x7f4d3806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:45.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.110+0000 7f4d51e8f640 1 --2- 172.21.15.76:0/3029653398 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d3806ae70 0x7f4d3806d330 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4d40036000 tx=0x7f4d40030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:45.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.114+0000 7f4d4a7fc640 1 -- 172.21.15.76:0/3029653398 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4d3c0546f0 con 0x7f4d4c0c48e0 2023-11-09T03:48:45.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.313+0000 7f4d540f3640 1 -- 172.21.15.76:0/3029653398 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4d4c0b4850 con 0x7f4d4c0c48e0 2023-11-09T03:48:45.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.986+0000 7f4d4a7fc640 1 -- 172.21.15.76:0/3029653398 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4d3c057d10 con 0x7f4d4c0c48e0 2023-11-09T03:48:45.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.988+0000 7f4d540f3640 1 -- 172.21.15.76:0/3029653398 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d3806ae70 msgr2=0x7f4d3806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:45.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.988+0000 7f4d540f3640 1 --2- 172.21.15.76:0/3029653398 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d3806ae70 0x7f4d3806d330 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4d40036000 tx=0x7f4d40030000 comp rx=0 tx=0).stop 2023-11-09T03:48:45.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.988+0000 7f4d540f3640 1 -- 172.21.15.76:0/3029653398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d4c0c48e0 msgr2=0x7f4d4c152110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:45.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.988+0000 7f4d540f3640 1 --2- 172.21.15.76:0/3029653398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d4c0c48e0 0x7f4d4c152110 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c00cbe0 tx=0x7f4d3c00a430 comp rx=0 tx=0).stop 2023-11-09T03:48:45.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.988+0000 7f4d540f3640 1 -- 172.21.15.76:0/3029653398 shutdown_connections 2023-11-09T03:48:45.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.988+0000 7f4d540f3640 1 --2- 172.21.15.76:0/3029653398 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d3806ae70 0x7f4d3806d330 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:45.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.988+0000 7f4d540f3640 1 --2- 172.21.15.76:0/3029653398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d4c159c00 0x7f4d4c1578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:45.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.988+0000 7f4d540f3640 1 --2- 172.21.15.76:0/3029653398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d4c0c48e0 0x7f4d4c152110 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:45.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.988+0000 7f4d540f3640 1 --2- 172.21.15.76:0/3029653398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d4c0ae1d0 0x7f4d4c14e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:45.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.988+0000 7f4d540f3640 1 -- 172.21.15.76:0/3029653398 >> 172.21.15.76:0/3029653398 conn(0x7f4d4c0ce950 msgr2=0x7f4d4c06deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:45.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.988+0000 7f4d540f3640 1 -- 172.21.15.76:0/3029653398 shutdown_connections 2023-11-09T03:48:45.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:45.988+0000 7f4d540f3640 1 -- 172.21.15.76:0/3029653398 wait complete. 2023-11-09T03:48:46.001 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-09T03:48:46.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.146+0000 7f593e432640 1 Processor -- start 2023-11-09T03:48:46.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.146+0000 7f593e432640 1 -- start start 2023-11-09T03:48:46.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.147+0000 7f593e432640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f593815d1b0 0x7f59380ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:46.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.147+0000 7f593e432640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f59380cecf0 0x7f59380cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:46.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.147+0000 7f593e432640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5938161870 con 0x7f59380cb680 2023-11-09T03:48:46.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.147+0000 7f593e432640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5938160c60 con 0x7f593815d1b0 2023-11-09T03:48:46.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.147+0000 7f593e432640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5938160f40 con 0x7f59380cecf0 2023-11-09T03:48:46.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.147+0000 7f5937fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f593815d1b0 0x7f59380ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:46.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.147+0000 7f593c9cf640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f59380cb680 0x7f59380b2660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48658/0 (socket says 172.21.15.76:48658) 2023-11-09T03:48:46.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.147+0000 7f593c9cf640 1 -- 172.21.15.76:0/3196825620 learned_addr learned my addr 172.21.15.76:0/3196825620 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:46.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.147+0000 7f59377fe640 1 --2- 172.21.15.76:0/3196825620 >> v2:172.21.15.113:3300/0 conn(0x7f59380cecf0 0x7f59380cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:46.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.147+0000 7f5937fff640 1 -- 172.21.15.76:0/3196825620 >> v2:172.21.15.113:3300/0 conn(0x7f59380cecf0 msgr2=0x7f59380cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:46.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.147+0000 7f5937fff640 1 --2- 172.21.15.76:0/3196825620 >> v2:172.21.15.113:3300/0 conn(0x7f59380cecf0 0x7f59380cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:46.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.147+0000 7f5937fff640 1 -- 172.21.15.76:0/3196825620 >> v1:172.21.15.76:6789/0 conn(0x7f59380cb680 legacy=0x7f59380b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:46.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.147+0000 7f5937fff640 1 -- 172.21.15.76:0/3196825620 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5938161220 con 0x7f593815d1b0 2023-11-09T03:48:46.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.148+0000 7f5937fff640 1 --2- 172.21.15.76:0/3196825620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f593815d1b0 0x7f59380ce7b0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f593806fb10 tx=0x7f592802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3efd7ae76b2ac154 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:46.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.148+0000 7f5936ffd640 1 -- 172.21.15.76:0/3196825620 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5928031830 con 0x7f593815d1b0 2023-11-09T03:48:46.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.148+0000 7f5936ffd640 1 -- 172.21.15.76:0/3196825620 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5928031990 con 0x7f593815d1b0 2023-11-09T03:48:46.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.148+0000 7f5936ffd640 1 -- 172.21.15.76:0/3196825620 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5928031d20 con 0x7f593815d1b0 2023-11-09T03:48:46.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.149+0000 7f593e432640 1 -- 172.21.15.76:0/3196825620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f593815d1b0 msgr2=0x7f59380ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:46.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.149+0000 7f593e432640 1 --2- 172.21.15.76:0/3196825620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f593815d1b0 0x7f59380ce7b0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f593806fb10 tx=0x7f592802b8d0 comp rx=0 tx=0).stop 2023-11-09T03:48:46.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.149+0000 7f593e432640 1 -- 172.21.15.76:0/3196825620 shutdown_connections 2023-11-09T03:48:46.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.149+0000 7f593e432640 1 --2- 172.21.15.76:0/3196825620 >> v2:172.21.15.113:3300/0 conn(0x7f59380cecf0 0x7f59380cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:46.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.149+0000 7f593e432640 1 --2- 172.21.15.76:0/3196825620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f593815d1b0 0x7f59380ce7b0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:46.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.149+0000 7f593e432640 1 -- 172.21.15.76:0/3196825620 >> 172.21.15.76:0/3196825620 conn(0x7f593806ea20 msgr2=0x7f593806ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:46.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.149+0000 7f593e432640 1 -- 172.21.15.76:0/3196825620 shutdown_connections 2023-11-09T03:48:46.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.149+0000 7f593e432640 1 -- 172.21.15.76:0/3196825620 wait complete. 2023-11-09T03:48:46.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.150+0000 7f593e432640 1 Processor -- start 2023-11-09T03:48:46.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.150+0000 7f593e432640 1 -- start start 2023-11-09T03:48:46.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.151+0000 7f593e432640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59380cb680 0x7f59380ca690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:46.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.151+0000 7f593e432640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59380cecf0 0x7f59380b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:46.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.151+0000 7f593e432640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f593815d1b0 0x7f59380c6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:46.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.151+0000 7f593e432640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59380b2990 con 0x7f593815d1b0 2023-11-09T03:48:46.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.151+0000 7f593e432640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59380c4710 con 0x7f59380cb680 2023-11-09T03:48:46.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.151+0000 7f593e432640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59380b2ba0 con 0x7f59380cecf0 2023-11-09T03:48:46.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.151+0000 7f593c9cf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f593815d1b0 0x7f59380c6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:46.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.151+0000 7f593c9cf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f593815d1b0 0x7f59380c6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52130/0 (socket says 172.21.15.76:52130) 2023-11-09T03:48:46.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.151+0000 7f593c9cf640 1 -- 172.21.15.76:0/193759318 learned_addr learned my addr 172.21.15.76:0/193759318 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:46.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.151+0000 7f5937fff640 1 --2- 172.21.15.76:0/193759318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59380cb680 0x7f59380ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:46.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.151+0000 7f59377fe640 1 --2- 172.21.15.76:0/193759318 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59380cecf0 0x7f59380b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:46.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.151+0000 7f593c9cf640 1 -- 172.21.15.76:0/193759318 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59380cecf0 msgr2=0x7f59380b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:46.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.151+0000 7f593c9cf640 1 --2- 172.21.15.76:0/193759318 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59380cecf0 0x7f59380b2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:46.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.151+0000 7f593c9cf640 1 -- 172.21.15.76:0/193759318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59380cb680 msgr2=0x7f59380ca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:46.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.151+0000 7f593c9cf640 1 --2- 172.21.15.76:0/193759318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59380cb680 0x7f59380ca690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:46.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.151+0000 7f593c9cf640 1 -- 172.21.15.76:0/193759318 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5928009400 con 0x7f593815d1b0 2023-11-09T03:48:46.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.151+0000 7f5937fff640 1 --2- 172.21.15.76:0/193759318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59380cb680 0x7f59380ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:46.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.152+0000 7f593c9cf640 1 --2- 172.21.15.76:0/193759318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f593815d1b0 0x7f59380c6ed0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f593000cbc0 tx=0x7f593000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:46.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.152+0000 7f5934ff9640 1 -- 172.21.15.76:0/193759318 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593000ad40 con 0x7f593815d1b0 2023-11-09T03:48:46.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.152+0000 7f5934ff9640 1 -- 172.21.15.76:0/193759318 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5930012400 con 0x7f593815d1b0 2023-11-09T03:48:46.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.152+0000 7f593e432640 1 -- 172.21.15.76:0/193759318 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59380c4960 con 0x7f593815d1b0 2023-11-09T03:48:46.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.152+0000 7f5934ff9640 1 -- 172.21.15.76:0/193759318 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593000aea0 con 0x7f593815d1b0 2023-11-09T03:48:46.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.152+0000 7f593e432640 1 -- 172.21.15.76:0/193759318 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59380c4f20 con 0x7f593815d1b0 2023-11-09T03:48:46.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.153+0000 7f593e432640 1 -- 172.21.15.76:0/193759318 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59380c5110 con 0x7f593815d1b0 2023-11-09T03:48:46.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.153+0000 7f5934ff9640 1 -- 172.21.15.76:0/193759318 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5930007070 con 0x7f593815d1b0 2023-11-09T03:48:46.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.153+0000 7f5934ff9640 1 --2- 172.21.15.76:0/193759318 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f592406ae70 0x7f592406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:46.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.154+0000 7f5934ff9640 1 -- 172.21.15.76:0/193759318 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7206+0+0 (secure 0 0 0) 0x7f5930089aa0 con 0x7f593815d1b0 2023-11-09T03:48:46.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.154+0000 7f5937fff640 1 --2- 172.21.15.76:0/193759318 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f592406ae70 0x7f592406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:46.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.154+0000 7f5937fff640 1 --2- 172.21.15.76:0/193759318 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f592406ae70 0x7f592406d330 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5928036000 tx=0x7f592803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:46.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.158+0000 7f5934ff9640 1 -- 172.21.15.76:0/193759318 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5930054630 con 0x7f593815d1b0 2023-11-09T03:48:46.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.325+0000 7f593e432640 1 -- 172.21.15.76:0/193759318 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f593804e490 con 0x7f593815d1b0 2023-11-09T03:48:46.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.989+0000 7f5934ff9640 1 -- 172.21.15.76:0/193759318 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v86) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5930057c50 con 0x7f593815d1b0 2023-11-09T03:48:46.990 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' removed 2023-11-09T03:48:46.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.992+0000 7f593e432640 1 -- 172.21.15.76:0/193759318 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f592406ae70 msgr2=0x7f592406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:46.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.992+0000 7f593e432640 1 --2- 172.21.15.76:0/193759318 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f592406ae70 0x7f592406d330 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5928036000 tx=0x7f592803c040 comp rx=0 tx=0).stop 2023-11-09T03:48:46.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.992+0000 7f593e432640 1 -- 172.21.15.76:0/193759318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f593815d1b0 msgr2=0x7f59380c6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:46.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.992+0000 7f593e432640 1 --2- 172.21.15.76:0/193759318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f593815d1b0 0x7f59380c6ed0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f593000cbc0 tx=0x7f593000a430 comp rx=0 tx=0).stop 2023-11-09T03:48:46.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.992+0000 7f593e432640 1 -- 172.21.15.76:0/193759318 shutdown_connections 2023-11-09T03:48:46.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.992+0000 7f593e432640 1 --2- 172.21.15.76:0/193759318 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f592406ae70 0x7f592406d330 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:46.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.992+0000 7f593e432640 1 --2- 172.21.15.76:0/193759318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f593815d1b0 0x7f59380c6ed0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:46.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.992+0000 7f593e432640 1 --2- 172.21.15.76:0/193759318 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59380cecf0 0x7f59380b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:46.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.992+0000 7f593e432640 1 --2- 172.21.15.76:0/193759318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59380cb680 0x7f59380ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:46.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.992+0000 7f593e432640 1 -- 172.21.15.76:0/193759318 >> 172.21.15.76:0/193759318 conn(0x7f593806ea20 msgr2=0x7f59380d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:46.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.992+0000 7f593e432640 1 -- 172.21.15.76:0/193759318 shutdown_connections 2023-11-09T03:48:46.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:46.992+0000 7f593e432640 1 -- 172.21.15.76:0/193759318 wait complete. 2023-11-09T03:48:47.005 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-09T03:48:47.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.109+0000 7fc33cee1640 1 Processor -- start 2023-11-09T03:48:47.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.109+0000 7fc33cee1640 1 -- start start 2023-11-09T03:48:47.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.110+0000 7fc33cee1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc338159be0 0x7fc3380c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:47.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.110+0000 7fc33cee1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc33815e420 0x7fc3380ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:47.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.110+0000 7fc33cee1640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3380c4850 con 0x7fc3380adc70 2023-11-09T03:48:47.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.110+0000 7fc33cee1640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3380ae100 con 0x7fc338159be0 2023-11-09T03:48:47.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.110+0000 7fc33cee1640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3380c5890 con 0x7fc33815e420 2023-11-09T03:48:47.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.110+0000 7fc33659c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc338159be0 0x7fc3380c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:47.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.110+0000 7fc336d9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc3380adc70 0x7fc3380cc590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48672/0 (socket says 172.21.15.76:48672) 2023-11-09T03:48:47.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.110+0000 7fc336d9d640 1 -- 172.21.15.76:0/1102650693 learned_addr learned my addr 172.21.15.76:0/1102650693 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:47.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.110+0000 7fc335d9b640 1 --2- 172.21.15.76:0/1102650693 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc33815e420 0x7fc3380ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:47.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.111+0000 7fc33659c640 1 -- 172.21.15.76:0/1102650693 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc33815e420 msgr2=0x7fc3380ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:47.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.111+0000 7fc33659c640 1 --2- 172.21.15.76:0/1102650693 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc33815e420 0x7fc3380ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:47.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.111+0000 7fc33659c640 1 -- 172.21.15.76:0/1102650693 >> v1:172.21.15.76:6789/0 conn(0x7fc3380adc70 legacy=0x7fc3380cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:47.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.111+0000 7fc33659c640 1 -- 172.21.15.76:0/1102650693 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3380cd8e0 con 0x7fc338159be0 2023-11-09T03:48:47.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.111+0000 7fc335d9b640 1 --2- 172.21.15.76:0/1102650693 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc33815e420 0x7fc3380ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:47.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.111+0000 7fc33659c640 1 --2- 172.21.15.76:0/1102650693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc338159be0 0x7fc3380c4f60 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fc328009d90 tx=0x7fc32802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451c7fbb0f9aea8b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:47.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.111+0000 7fc33559a640 1 -- 172.21.15.76:0/1102650693 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32802cbc0 con 0x7fc338159be0 2023-11-09T03:48:47.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.111+0000 7fc33559a640 1 -- 172.21.15.76:0/1102650693 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32802cd20 con 0x7fc338159be0 2023-11-09T03:48:47.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.111+0000 7fc33559a640 1 -- 172.21.15.76:0/1102650693 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc328034660 con 0x7fc338159be0 2023-11-09T03:48:47.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.112+0000 7fc33cee1640 1 -- 172.21.15.76:0/1102650693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc338159be0 msgr2=0x7fc3380c4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:47.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.112+0000 7fc33cee1640 1 --2- 172.21.15.76:0/1102650693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc338159be0 0x7fc3380c4f60 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fc328009d90 tx=0x7fc32802be10 comp rx=0 tx=0).stop 2023-11-09T03:48:47.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.112+0000 7fc33cee1640 1 -- 172.21.15.76:0/1102650693 shutdown_connections 2023-11-09T03:48:47.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.112+0000 7fc33cee1640 1 --2- 172.21.15.76:0/1102650693 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc33815e420 0x7fc3380ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:47.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.112+0000 7fc33cee1640 1 --2- 172.21.15.76:0/1102650693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc338159be0 0x7fc3380c4f60 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:47.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.112+0000 7fc33cee1640 1 -- 172.21.15.76:0/1102650693 >> 172.21.15.76:0/1102650693 conn(0x7fc3380b05d0 msgr2=0x7fc3380b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:47.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.112+0000 7fc33cee1640 1 -- 172.21.15.76:0/1102650693 shutdown_connections 2023-11-09T03:48:47.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.112+0000 7fc33cee1640 1 -- 172.21.15.76:0/1102650693 wait complete. 2023-11-09T03:48:47.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.113+0000 7fc33cee1640 1 Processor -- start 2023-11-09T03:48:47.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.113+0000 7fc33cee1640 1 -- start start 2023-11-09T03:48:47.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.114+0000 7fc33cee1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3380adc70 0x7fc3380ac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:47.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.114+0000 7fc33cee1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc338159be0 0x7fc338154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:47.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.114+0000 7fc33cee1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc33815e420 0x7fc3380b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:47.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.114+0000 7fc33cee1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3380ad2b0 con 0x7fc338159be0 2023-11-09T03:48:47.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.114+0000 7fc33cee1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3380b4160 con 0x7fc33815e420 2023-11-09T03:48:47.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.114+0000 7fc33cee1640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3380b4440 con 0x7fc3380adc70 2023-11-09T03:48:47.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.114+0000 7fc335d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc338159be0 0x7fc338154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:47.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.114+0000 7fc335d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc338159be0 0x7fc338154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52142/0 (socket says 172.21.15.76:52142) 2023-11-09T03:48:47.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.114+0000 7fc335d9b640 1 -- 172.21.15.76:0/804909223 learned_addr learned my addr 172.21.15.76:0/804909223 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:47.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.115+0000 7fc335d9b640 1 -- 172.21.15.76:0/804909223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3380adc70 msgr2=0x7fc3380ac2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:47.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.115+0000 7fc33659c640 1 --2- 172.21.15.76:0/804909223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3380adc70 0x7fc3380ac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:47.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.115+0000 7fc335d9b640 1 --2- 172.21.15.76:0/804909223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3380adc70 0x7fc3380ac2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:47.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.115+0000 7fc336d9d640 1 --2- 172.21.15.76:0/804909223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc33815e420 0x7fc3380b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:47.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.115+0000 7fc335d9b640 1 -- 172.21.15.76:0/804909223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc33815e420 msgr2=0x7fc3380b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:47.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.115+0000 7fc335d9b640 1 --2- 172.21.15.76:0/804909223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc33815e420 0x7fc3380b2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:47.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.115+0000 7fc335d9b640 1 -- 172.21.15.76:0/804909223 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc328009a70 con 0x7fc338159be0 2023-11-09T03:48:47.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.115+0000 7fc335d9b640 1 --2- 172.21.15.76:0/804909223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc338159be0 0x7fc338154180 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fc32c00c7f0 tx=0x7fc32c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:47.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.115+0000 7fc326ffd640 1 -- 172.21.15.76:0/804909223 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c00cea0 con 0x7fc338159be0 2023-11-09T03:48:47.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.115+0000 7fc326ffd640 1 -- 172.21.15.76:0/804909223 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32c007a30 con 0x7fc338159be0 2023-11-09T03:48:47.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.115+0000 7fc33cee1640 1 -- 172.21.15.76:0/804909223 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3380c6eb0 con 0x7fc338159be0 2023-11-09T03:48:47.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.115+0000 7fc326ffd640 1 -- 172.21.15.76:0/804909223 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c007dc0 con 0x7fc338159be0 2023-11-09T03:48:47.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.115+0000 7fc33cee1640 1 -- 172.21.15.76:0/804909223 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3380c7470 con 0x7fc338159be0 2023-11-09T03:48:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.116+0000 7fc33cee1640 1 -- 172.21.15.76:0/804909223 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc33804e490 con 0x7fc338159be0 2023-11-09T03:48:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.116+0000 7fc326ffd640 1 -- 172.21.15.76:0/804909223 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc32c0113f0 con 0x7fc338159be0 2023-11-09T03:48:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.117+0000 7fc326ffd640 1 --2- 172.21.15.76:0/804909223 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc31806aea0 0x7fc31806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.117+0000 7fc326ffd640 1 -- 172.21.15.76:0/804909223 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6768+0+0 (secure 0 0 0) 0x7fc32c089b10 con 0x7fc338159be0 2023-11-09T03:48:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.117+0000 7fc33659c640 1 --2- 172.21.15.76:0/804909223 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc31806aea0 0x7fc31806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:47.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.118+0000 7fc33659c640 1 --2- 172.21.15.76:0/804909223 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc31806aea0 0x7fc31806d360 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc328036000 tx=0x7fc328030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:47.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.121+0000 7fc326ffd640 1 -- 172.21.15.76:0/804909223 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc32c0547a0 con 0x7fc338159be0 2023-11-09T03:48:47.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.286+0000 7fc33cee1640 1 -- 172.21.15.76:0/804909223 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc3380c4d40 con 0x7fc338159be0 2023-11-09T03:48:47.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.994+0000 7fc326ffd640 1 -- 172.21.15.76:0/804909223 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v87) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc32c01f0d0 con 0x7fc338159be0 2023-11-09T03:48:47.995 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' removed 2023-11-09T03:48:47.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.997+0000 7fc33cee1640 1 -- 172.21.15.76:0/804909223 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc31806aea0 msgr2=0x7fc31806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:47.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.997+0000 7fc33cee1640 1 --2- 172.21.15.76:0/804909223 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc31806aea0 0x7fc31806d360 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc328036000 tx=0x7fc328030000 comp rx=0 tx=0).stop 2023-11-09T03:48:47.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.997+0000 7fc33cee1640 1 -- 172.21.15.76:0/804909223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc338159be0 msgr2=0x7fc338154180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:47.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.997+0000 7fc33cee1640 1 --2- 172.21.15.76:0/804909223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc338159be0 0x7fc338154180 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fc32c00c7f0 tx=0x7fc32c00ccb0 comp rx=0 tx=0).stop 2023-11-09T03:48:47.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.997+0000 7fc33cee1640 1 -- 172.21.15.76:0/804909223 shutdown_connections 2023-11-09T03:48:47.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.997+0000 7fc33cee1640 1 --2- 172.21.15.76:0/804909223 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc31806aea0 0x7fc31806d360 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:47.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.997+0000 7fc33cee1640 1 --2- 172.21.15.76:0/804909223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc33815e420 0x7fc3380b2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:47.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.997+0000 7fc33cee1640 1 --2- 172.21.15.76:0/804909223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc338159be0 0x7fc338154180 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:48.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.997+0000 7fc33cee1640 1 --2- 172.21.15.76:0/804909223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3380adc70 0x7fc3380ac2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:48.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.997+0000 7fc33cee1640 1 -- 172.21.15.76:0/804909223 >> 172.21.15.76:0/804909223 conn(0x7fc3380b05d0 msgr2=0x7fc3380c3840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:48.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.998+0000 7fc33cee1640 1 -- 172.21.15.76:0/804909223 shutdown_connections 2023-11-09T03:48:48.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:47.998+0000 7fc33cee1640 1 -- 172.21.15.76:0/804909223 wait complete. 2023-11-09T03:48:48.010 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-09T03:48:48.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.112+0000 7fb4868c9640 1 Processor -- start 2023-11-09T03:48:48.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.112+0000 7fb4868c9640 1 -- start start 2023-11-09T03:48:48.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.113+0000 7fb4868c9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb4800cd6d0 0x7fb480161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:48.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.113+0000 7fb4868c9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4800b4880 0x7fb4800b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:48.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.113+0000 7fb4868c9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4800cde60 con 0x7fb480153e60 2023-11-09T03:48:48.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.113+0000 7fb4868c9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb480160f90 con 0x7fb4800cd6d0 2023-11-09T03:48:48.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.113+0000 7fb4868c9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb480161960 con 0x7fb4800b4880 2023-11-09T03:48:48.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.113+0000 7fb47f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb4800cd6d0 0x7fb480161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:48.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.113+0000 7fb47f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb4800cd6d0 0x7fb480161420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52152/0 (socket says 172.21.15.76:52152) 2023-11-09T03:48:48.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.113+0000 7fb47f7fe640 1 -- 172.21.15.76:0/3888450186 learned_addr learned my addr 172.21.15.76:0/3888450186 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:48.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.114+0000 7fb484e66640 1 --2- 172.21.15.76:0/3888450186 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4800b4880 0x7fb4800b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:48.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.114+0000 7fb47f7fe640 1 -- 172.21.15.76:0/3888450186 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4800b4880 msgr2=0x7fb4800b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:48.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.114+0000 7fb47f7fe640 1 --2- 172.21.15.76:0/3888450186 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4800b4880 0x7fb4800b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:48.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.114+0000 7fb47f7fe640 1 -- 172.21.15.76:0/3888450186 >> v1:172.21.15.113:6789/0 conn(0x7fb480153e60 legacy=0x7fb48015dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:48.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.114+0000 7fb47f7fe640 1 -- 172.21.15.76:0/3888450186 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4800b5220 con 0x7fb4800cd6d0 2023-11-09T03:48:48.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.114+0000 7fb484e66640 1 --2- 172.21.15.76:0/3888450186 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4800b4880 0x7fb4800b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:48.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.114+0000 7fb47f7fe640 1 --2- 172.21.15.76:0/3888450186 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4800cd6d0 0x7fb480161420 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fb474009870 tx=0x7fb47402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b86bdb988097411 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:48.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.114+0000 7fb47effd640 1 -- 172.21.15.76:0/3888450186 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb474030840 con 0x7fb4800cd6d0 2023-11-09T03:48:48.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.115+0000 7fb47effd640 1 -- 172.21.15.76:0/3888450186 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4740309a0 con 0x7fb4800cd6d0 2023-11-09T03:48:48.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.115+0000 7fb47effd640 1 -- 172.21.15.76:0/3888450186 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb474030d30 con 0x7fb4800cd6d0 2023-11-09T03:48:48.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.115+0000 7fb4868c9640 1 -- 172.21.15.76:0/3888450186 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4800cd6d0 msgr2=0x7fb480161420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:48.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.115+0000 7fb4868c9640 1 --2- 172.21.15.76:0/3888450186 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4800cd6d0 0x7fb480161420 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fb474009870 tx=0x7fb47402b980 comp rx=0 tx=0).stop 2023-11-09T03:48:48.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.115+0000 7fb4868c9640 1 -- 172.21.15.76:0/3888450186 shutdown_connections 2023-11-09T03:48:48.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.115+0000 7fb4868c9640 1 --2- 172.21.15.76:0/3888450186 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4800b4880 0x7fb4800b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:48.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.115+0000 7fb4868c9640 1 --2- 172.21.15.76:0/3888450186 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4800cd6d0 0x7fb480161420 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:48.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.115+0000 7fb4868c9640 1 -- 172.21.15.76:0/3888450186 >> 172.21.15.76:0/3888450186 conn(0x7fb4800cf880 msgr2=0x7fb4800cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:48.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.115+0000 7fb4868c9640 1 -- 172.21.15.76:0/3888450186 shutdown_connections 2023-11-09T03:48:48.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.115+0000 7fb4868c9640 1 -- 172.21.15.76:0/3888450186 wait complete. 2023-11-09T03:48:48.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.116+0000 7fb4868c9640 1 Processor -- start 2023-11-09T03:48:48.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.116+0000 7fb4868c9640 1 -- start start 2023-11-09T03:48:48.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.117+0000 7fb4868c9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4800b4880 0x7fb480158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:48.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.117+0000 7fb4868c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4800cd6d0 0x7fb480153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:48.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.117+0000 7fb4868c9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb480153e60 0x7fb48015c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:48.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.117+0000 7fb4868c9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb480156fe0 con 0x7fb4800cd6d0 2023-11-09T03:48:48.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.117+0000 7fb4868c9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb480158ac0 con 0x7fb4800b4880 2023-11-09T03:48:48.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.117+0000 7fb4868c9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48015bb70 con 0x7fb480153e60 2023-11-09T03:48:48.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.117+0000 7fb47f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4800cd6d0 0x7fb480153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:48.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.117+0000 7fb47f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4800cd6d0 0x7fb480153210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52158/0 (socket says 172.21.15.76:52158) 2023-11-09T03:48:48.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.117+0000 7fb47f7fe640 1 -- 172.21.15.76:0/3359959390 learned_addr learned my addr 172.21.15.76:0/3359959390 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:48.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.118+0000 7fb484e66640 1 --2- 172.21.15.76:0/3359959390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb480153e60 0x7fb48015c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:48.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.118+0000 7fb47ffff640 1 --2- 172.21.15.76:0/3359959390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4800b4880 0x7fb480158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:48.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.118+0000 7fb47f7fe640 1 -- 172.21.15.76:0/3359959390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb480153e60 msgr2=0x7fb48015c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:48.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.118+0000 7fb47f7fe640 1 --2- 172.21.15.76:0/3359959390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb480153e60 0x7fb48015c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:48.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.118+0000 7fb47f7fe640 1 -- 172.21.15.76:0/3359959390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4800b4880 msgr2=0x7fb480158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:48.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.118+0000 7fb47f7fe640 1 --2- 172.21.15.76:0/3359959390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4800b4880 0x7fb480158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:48.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.118+0000 7fb47f7fe640 1 -- 172.21.15.76:0/3359959390 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4740094d0 con 0x7fb4800cd6d0 2023-11-09T03:48:48.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.118+0000 7fb484e66640 1 --2- 172.21.15.76:0/3359959390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb480153e60 0x7fb48015c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:48.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.118+0000 7fb47ffff640 1 --2- 172.21.15.76:0/3359959390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4800b4880 0x7fb480158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:48.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.118+0000 7fb47f7fe640 1 --2- 172.21.15.76:0/3359959390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4800cd6d0 0x7fb480153210 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fb474000c00 tx=0x7fb474004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:48.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.118+0000 7fb47cff9640 1 -- 172.21.15.76:0/3359959390 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb474030810 con 0x7fb4800cd6d0 2023-11-09T03:48:48.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.118+0000 7fb47cff9640 1 -- 172.21.15.76:0/3359959390 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb474043400 con 0x7fb4800cd6d0 2023-11-09T03:48:48.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.118+0000 7fb47cff9640 1 -- 172.21.15.76:0/3359959390 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4740437d0 con 0x7fb4800cd6d0 2023-11-09T03:48:48.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.118+0000 7fb4868c9640 1 -- 172.21.15.76:0/3359959390 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb48015b240 con 0x7fb4800cd6d0 2023-11-09T03:48:48.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.118+0000 7fb4868c9640 1 -- 172.21.15.76:0/3359959390 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb48015b800 con 0x7fb4800cd6d0 2023-11-09T03:48:48.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.119+0000 7fb4868c9640 1 -- 172.21.15.76:0/3359959390 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb48015b990 con 0x7fb4800cd6d0 2023-11-09T03:48:48.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.120+0000 7fb47cff9640 1 -- 172.21.15.76:0/3359959390 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb474043930 con 0x7fb4800cd6d0 2023-11-09T03:48:48.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.120+0000 7fb47cff9640 1 --2- 172.21.15.76:0/3359959390 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb46c06ae70 0x7fb46c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:48.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.120+0000 7fb47cff9640 1 -- 172.21.15.76:0/3359959390 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6354+0+0 (secure 0 0 0) 0x7fb47403e070 con 0x7fb4800cd6d0 2023-11-09T03:48:48.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.120+0000 7fb47ffff640 1 --2- 172.21.15.76:0/3359959390 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb46c06ae70 0x7fb46c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:48.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.121+0000 7fb47ffff640 1 --2- 172.21.15.76:0/3359959390 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb46c06ae70 0x7fb46c06d330 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb4801f5f70 tx=0x7fb470009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:48.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.124+0000 7fb47cff9640 1 -- 172.21.15.76:0/3359959390 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb4740779b0 con 0x7fb4800cd6d0 2023-11-09T03:48:48.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.289+0000 7fb4868c9640 1 -- 172.21.15.76:0/3359959390 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb480074290 con 0x7fb4800cd6d0 2023-11-09T03:48:48.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.292+0000 7fb47cff9640 1 -- 172.21.15.76:0/3359959390 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb47407afd0 con 0x7fb4800cd6d0 2023-11-09T03:48:48.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.294+0000 7fb4868c9640 1 -- 172.21.15.76:0/3359959390 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb46c06ae70 msgr2=0x7fb46c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:48.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.294+0000 7fb4868c9640 1 --2- 172.21.15.76:0/3359959390 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb46c06ae70 0x7fb46c06d330 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb4801f5f70 tx=0x7fb470009040 comp rx=0 tx=0).stop 2023-11-09T03:48:48.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.294+0000 7fb4868c9640 1 -- 172.21.15.76:0/3359959390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4800cd6d0 msgr2=0x7fb480153210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:48.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.294+0000 7fb4868c9640 1 --2- 172.21.15.76:0/3359959390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4800cd6d0 0x7fb480153210 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fb474000c00 tx=0x7fb474004730 comp rx=0 tx=0).stop 2023-11-09T03:48:48.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.294+0000 7fb4868c9640 1 -- 172.21.15.76:0/3359959390 shutdown_connections 2023-11-09T03:48:48.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.294+0000 7fb4868c9640 1 --2- 172.21.15.76:0/3359959390 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb46c06ae70 0x7fb46c06d330 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:48.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.294+0000 7fb4868c9640 1 --2- 172.21.15.76:0/3359959390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb480153e60 0x7fb48015c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:48.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.294+0000 7fb4868c9640 1 --2- 172.21.15.76:0/3359959390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4800cd6d0 0x7fb480153210 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:48.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.294+0000 7fb4868c9640 1 --2- 172.21.15.76:0/3359959390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4800b4880 0x7fb480158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:48.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.294+0000 7fb4868c9640 1 -- 172.21.15.76:0/3359959390 >> 172.21.15.76:0/3359959390 conn(0x7fb4800cf880 msgr2=0x7fb4800d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:48.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.294+0000 7fb4868c9640 1 -- 172.21.15.76:0/3359959390 shutdown_connections 2023-11-09T03:48:48.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.294+0000 7fb4868c9640 1 -- 172.21.15.76:0/3359959390 wait complete. 2023-11-09T03:48:48.306 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_allow_new_snaps_config' 2023-11-09T03:48:48.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.406+0000 7f9550c32640 1 Processor -- start 2023-11-09T03:48:48.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.406+0000 7f9550c32640 1 -- start start 2023-11-09T03:48:48.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.406+0000 7f9550c32640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f954c15ad80 0x7f954c06c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:48.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.406+0000 7f9550c32640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f954c0b42c0 0x7f954c0c9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:48.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.406+0000 7f9550c32640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f954c0c9c40 0x7f954c070630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:48.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.407+0000 7f9550c32640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f954c0ca070 con 0x7f954c15ad80 2023-11-09T03:48:48.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.407+0000 7f9550c32640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f954c06cd00 con 0x7f954c0c9c40 2023-11-09T03:48:48.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.407+0000 7f9550c32640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954c06cfa0 con 0x7f954c0b42c0 2023-11-09T03:48:48.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.407+0000 7f954a59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f954c15ad80 0x7f954c06c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:48.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.407+0000 7f954a59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f954c15ad80 0x7f954c06c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52168/0 (socket says 172.21.15.76:52168) 2023-11-09T03:48:48.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.407+0000 7f954a59c640 1 -- 172.21.15.76:0/868343176 learned_addr learned my addr 172.21.15.76:0/868343176 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:48.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.407+0000 7f954ad9d640 1 --2- 172.21.15.76:0/868343176 >> v2:172.21.15.113:3300/0 conn(0x7f954c0c9c40 0x7f954c070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:48.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.407+0000 7f9541d9b640 1 --2- 172.21.15.76:0/868343176 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f954c0b42c0 0x7f954c0c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:48.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.407+0000 7f954a59c640 1 -- 172.21.15.76:0/868343176 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f954c0b42c0 msgr2=0x7f954c0c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:48.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.407+0000 7f954a59c640 1 --2- 172.21.15.76:0/868343176 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f954c0b42c0 0x7f954c0c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:48.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.407+0000 7f954a59c640 1 -- 172.21.15.76:0/868343176 >> v2:172.21.15.113:3300/0 conn(0x7f954c0c9c40 msgr2=0x7f954c070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:48.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.407+0000 7f954a59c640 1 --2- 172.21.15.76:0/868343176 >> v2:172.21.15.113:3300/0 conn(0x7f954c0c9c40 0x7f954c070630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:48.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.407+0000 7f954a59c640 1 -- 172.21.15.76:0/868343176 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954c06d280 con 0x7f954c15ad80 2023-11-09T03:48:48.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.408+0000 7f954a59c640 1 --2- 172.21.15.76:0/868343176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f954c15ad80 0x7f954c06c7c0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f954c0ae320 tx=0x7f953c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7223dfb55e7a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:48.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.408+0000 7f9549d9b640 1 -- 172.21.15.76:0/868343176 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c030860 con 0x7f954c15ad80 2023-11-09T03:48:48.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.408+0000 7f9549d9b640 1 -- 172.21.15.76:0/868343176 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953c0309c0 con 0x7f954c15ad80 2023-11-09T03:48:48.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.408+0000 7f9549d9b640 1 -- 172.21.15.76:0/868343176 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c030d50 con 0x7f954c15ad80 2023-11-09T03:48:48.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.408+0000 7f9550c32640 1 -- 172.21.15.76:0/868343176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f954c15ad80 msgr2=0x7f954c06c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:48.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.408+0000 7f9550c32640 1 --2- 172.21.15.76:0/868343176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f954c15ad80 0x7f954c06c7c0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f954c0ae320 tx=0x7f953c02b9a0 comp rx=0 tx=0).stop 2023-11-09T03:48:48.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.408+0000 7f9550c32640 1 -- 172.21.15.76:0/868343176 shutdown_connections 2023-11-09T03:48:48.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.408+0000 7f9550c32640 1 --2- 172.21.15.76:0/868343176 >> v2:172.21.15.113:3300/0 conn(0x7f954c0c9c40 0x7f954c070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:48.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.408+0000 7f9550c32640 1 --2- 172.21.15.76:0/868343176 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f954c0b42c0 0x7f954c0c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:48.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.408+0000 7f9550c32640 1 --2- 172.21.15.76:0/868343176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f954c15ad80 0x7f954c06c7c0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:48.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.408+0000 7f9550c32640 1 -- 172.21.15.76:0/868343176 >> 172.21.15.76:0/868343176 conn(0x7f954c0ce950 msgr2=0x7f954c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:48.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.408+0000 7f9550c32640 1 -- 172.21.15.76:0/868343176 shutdown_connections 2023-11-09T03:48:48.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.408+0000 7f9550c32640 1 -- 172.21.15.76:0/868343176 wait complete. 2023-11-09T03:48:48.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.410+0000 7f9550c32640 1 Processor -- start 2023-11-09T03:48:48.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.410+0000 7f9550c32640 1 -- start start 2023-11-09T03:48:48.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.411+0000 7f9550c32640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f954c0b42c0 0x7f954c1f3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:48.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.411+0000 7f9550c32640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f954c0c9c40 0x7f954c1eca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:48.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.411+0000 7f9550c32640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f954c15ad80 0x7f954c1f2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:48.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.411+0000 7f9550c32640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c15cd70 con 0x7f954c0c9c40 2023-11-09T03:48:48.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.411+0000 7f9550c32640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c1f3530 con 0x7f954c0b42c0 2023-11-09T03:48:48.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.411+0000 7f9550c32640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954c1f3810 con 0x7f954c15ad80 2023-11-09T03:48:48.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.411+0000 7f9541d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f954c0c9c40 0x7f954c1eca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:48.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.411+0000 7f9541d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f954c0c9c40 0x7f954c1eca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52170/0 (socket says 172.21.15.76:52170) 2023-11-09T03:48:48.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.411+0000 7f9541d9b640 1 -- 172.21.15.76:0/3123448943 learned_addr learned my addr 172.21.15.76:0/3123448943 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:48.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.411+0000 7f954ad9d640 1 --2- 172.21.15.76:0/3123448943 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f954c15ad80 0x7f954c1f2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:48.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.411+0000 7f954a59c640 1 --2- 172.21.15.76:0/3123448943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f954c0b42c0 0x7f954c1f3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:48.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.411+0000 7f9541d9b640 1 -- 172.21.15.76:0/3123448943 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f954c15ad80 msgr2=0x7f954c1f2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:48.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.411+0000 7f9541d9b640 1 --2- 172.21.15.76:0/3123448943 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f954c15ad80 0x7f954c1f2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:48.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.411+0000 7f9541d9b640 1 -- 172.21.15.76:0/3123448943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f954c0b42c0 msgr2=0x7f954c1f3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:48.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.411+0000 7f9541d9b640 1 --2- 172.21.15.76:0/3123448943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f954c0b42c0 0x7f954c1f3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:48.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.411+0000 7f9541d9b640 1 -- 172.21.15.76:0/3123448943 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953c0094d0 con 0x7f954c0c9c40 2023-11-09T03:48:48.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.411+0000 7f9541d9b640 1 --2- 172.21.15.76:0/3123448943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f954c0c9c40 0x7f954c1eca40 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f953400d800 tx=0x7f953400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:48.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.411+0000 7f95437fe640 1 -- 172.21.15.76:0/3123448943 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95340096a0 con 0x7f954c0c9c40 2023-11-09T03:48:48.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.411+0000 7f95437fe640 1 -- 172.21.15.76:0/3123448943 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9534009800 con 0x7f954c0c9c40 2023-11-09T03:48:48.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.411+0000 7f9550c32640 1 -- 172.21.15.76:0/3123448943 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f954c15d010 con 0x7f954c0c9c40 2023-11-09T03:48:48.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.411+0000 7f95437fe640 1 -- 172.21.15.76:0/3123448943 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9534009990 con 0x7f954c0c9c40 2023-11-09T03:48:48.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.411+0000 7f9550c32640 1 -- 172.21.15.76:0/3123448943 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f954c1f42b0 con 0x7f954c0c9c40 2023-11-09T03:48:48.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.413+0000 7f9550c32640 1 -- 172.21.15.76:0/3123448943 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f954c074290 con 0x7f954c0c9c40 2023-11-09T03:48:48.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.413+0000 7f95437fe640 1 -- 172.21.15.76:0/3123448943 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9534009af0 con 0x7f954c0c9c40 2023-11-09T03:48:48.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.413+0000 7f95437fe640 1 --2- 172.21.15.76:0/3123448943 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f952c06ae70 0x7f952c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:48.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.413+0000 7f95437fe640 1 -- 172.21.15.76:0/3123448943 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6354+0+0 (secure 0 0 0) 0x7f9534089d10 con 0x7f954c0c9c40 2023-11-09T03:48:48.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.413+0000 7f954a59c640 1 --2- 172.21.15.76:0/3123448943 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f952c06ae70 0x7f952c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:48.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.414+0000 7f954a59c640 1 --2- 172.21.15.76:0/3123448943 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f952c06ae70 0x7f952c06d330 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f953c036000 tx=0x7f953c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:48.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.417+0000 7f95437fe640 1 -- 172.21.15.76:0/3123448943 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9534054b30 con 0x7f954c0c9c40 2023-11-09T03:48:48.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.653+0000 7f9550c32640 1 -- 172.21.15.76:0/3123448943 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_allow_new_snaps_config"]} v 0) v1 -- 0x7f954c0b4880 con 0x7f954c0c9c40 2023-11-09T03:48:48.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.997+0000 7f95437fe640 1 -- 172.21.15.76:0/3123448943 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_allow_new_snaps_config"]}]=0 v209) v1 ==== 148+0+0 (secure 0 0 0) 0x7f9534058150 con 0x7f954c0c9c40 2023-11-09T03:48:49.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.999+0000 7f9550c32640 1 -- 172.21.15.76:0/3123448943 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f952c06ae70 msgr2=0x7f952c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:49.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.999+0000 7f9550c32640 1 --2- 172.21.15.76:0/3123448943 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f952c06ae70 0x7f952c06d330 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f953c036000 tx=0x7f953c008030 comp rx=0 tx=0).stop 2023-11-09T03:48:49.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.999+0000 7f9550c32640 1 -- 172.21.15.76:0/3123448943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f954c0c9c40 msgr2=0x7f954c1eca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:49.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.999+0000 7f9550c32640 1 --2- 172.21.15.76:0/3123448943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f954c0c9c40 0x7f954c1eca40 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f953400d800 tx=0x7f953400dcc0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.999+0000 7f9550c32640 1 -- 172.21.15.76:0/3123448943 shutdown_connections 2023-11-09T03:48:49.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.999+0000 7f9550c32640 1 --2- 172.21.15.76:0/3123448943 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f952c06ae70 0x7f952c06d330 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.999+0000 7f9550c32640 1 --2- 172.21.15.76:0/3123448943 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f954c15ad80 0x7f954c1f2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.999+0000 7f9550c32640 1 --2- 172.21.15.76:0/3123448943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f954c0c9c40 0x7f954c1eca40 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.999+0000 7f9550c32640 1 --2- 172.21.15.76:0/3123448943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f954c0b42c0 0x7f954c1f3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.999+0000 7f9550c32640 1 -- 172.21.15.76:0/3123448943 >> 172.21.15.76:0/3123448943 conn(0x7f954c0ce950 msgr2=0x7f954c070630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:49.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.999+0000 7f9550c32640 1 -- 172.21.15.76:0/3123448943 shutdown_connections 2023-11-09T03:48:49.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:48.999+0000 7f9550c32640 1 -- 172.21.15.76:0/3123448943 wait complete. 2023-11-09T03:48:49.011 INFO:tasks.cephfs_test_runner:Check whether 'allow_new_snaps' setting works ... ok 2023-11-09T03:48:49.011 INFO:tasks.cephfs_test_runner:Starting test: test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots) check snaptable transcation 2023-11-09T03:48:49.012 INFO:tasks.cephfs_test_runner:test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots) 2023-11-09T03:48:49.012 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:48:49.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.112+0000 7f850f694640 1 Processor -- start 2023-11-09T03:48:49.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.112+0000 7f850f694640 1 -- start start 2023-11-09T03:48:49.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.113+0000 7f850f694640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85080cda60 0x7f85080ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:49.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.113+0000 7f850f694640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8508070d40 0x7f850806c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:49.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.113+0000 7f850f694640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85080cc990 con 0x7f85080b0cd0 2023-11-09T03:48:49.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.113+0000 7f850f694640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85080ccc70 con 0x7f8508070d40 2023-11-09T03:48:49.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.113+0000 7f850f694640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85080c9ce0 con 0x7f85080cda60 2023-11-09T03:48:49.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.114+0000 7f850cc2f640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f85080b0cd0 0x7f8508070630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:47072/0 (socket says 172.21.15.76:47072) 2023-11-09T03:48:49.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.114+0000 7f850d430640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85080cda60 0x7f85080ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:49.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.114+0000 7f850dc31640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8508070d40 0x7f850806c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:49.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.114+0000 7f850cc2f640 1 -- 172.21.15.76:0/2106005209 learned_addr learned my addr 172.21.15.76:0/2106005209 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:49.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.114+0000 7f84fffff640 1 -- 172.21.15.76:0/2106005209 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3394202932 0 0) 0x7f85080cc990 con 0x7f85080b0cd0 2023-11-09T03:48:49.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.114+0000 7f84fffff640 1 -- 172.21.15.76:0/2106005209 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84f4003520 con 0x7f85080b0cd0 2023-11-09T03:48:49.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.114+0000 7f850d430640 1 -- 172.21.15.76:0/2106005209 >> v2:172.21.15.113:3300/0 conn(0x7f8508070d40 msgr2=0x7f850806c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:49.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.114+0000 7f850d430640 1 --2- 172.21.15.76:0/2106005209 >> v2:172.21.15.113:3300/0 conn(0x7f8508070d40 0x7f850806c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.114+0000 7f850d430640 1 -- 172.21.15.76:0/2106005209 >> v1:172.21.15.113:6789/0 conn(0x7f85080b0cd0 legacy=0x7f8508070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:49.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.114+0000 7f850d430640 1 -- 172.21.15.76:0/2106005209 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85080c9fc0 con 0x7f85080cda60 2023-11-09T03:48:49.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.114+0000 7f850dc31640 1 --2- 172.21.15.76:0/2106005209 >> v2:172.21.15.113:3300/0 conn(0x7f8508070d40 0x7f850806c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:48:49.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.115+0000 7f850d430640 1 --2- 172.21.15.76:0/2106005209 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85080cda60 0x7f85080ca7a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f85000097a0 tx=0x7f850002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f825f4a52751a32 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:49.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.115+0000 7f84fffff640 1 -- 172.21.15.76:0/2106005209 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8500031830 con 0x7f85080cda60 2023-11-09T03:48:49.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.115+0000 7f84fffff640 1 -- 172.21.15.76:0/2106005209 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8500031990 con 0x7f85080cda60 2023-11-09T03:48:49.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.115+0000 7f84fffff640 1 -- 172.21.15.76:0/2106005209 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8500031d00 con 0x7f85080cda60 2023-11-09T03:48:49.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.115+0000 7f850f694640 1 -- 172.21.15.76:0/2106005209 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85080cda60 msgr2=0x7f85080ca7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:49.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.115+0000 7f850f694640 1 --2- 172.21.15.76:0/2106005209 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85080cda60 0x7f85080ca7a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f85000097a0 tx=0x7f850002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.115+0000 7f850f694640 1 -- 172.21.15.76:0/2106005209 shutdown_connections 2023-11-09T03:48:49.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.115+0000 7f850f694640 1 --2- 172.21.15.76:0/2106005209 >> v2:172.21.15.113:3300/0 conn(0x7f8508070d40 0x7f850806c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.115+0000 7f850f694640 1 --2- 172.21.15.76:0/2106005209 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85080cda60 0x7f85080ca7a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.115+0000 7f850f694640 1 -- 172.21.15.76:0/2106005209 >> 172.21.15.76:0/2106005209 conn(0x7f85080ce950 msgr2=0x7f85080ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:49.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.116+0000 7f850f694640 1 -- 172.21.15.76:0/2106005209 shutdown_connections 2023-11-09T03:48:49.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.116+0000 7f850f694640 1 -- 172.21.15.76:0/2106005209 wait complete. 2023-11-09T03:48:49.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.117+0000 7f850f694640 1 Processor -- start 2023-11-09T03:48:49.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.117+0000 7f850f694640 1 -- start start 2023-11-09T03:48:49.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.117+0000 7f850f694640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8508070d40 0x7f85081f0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:49.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.117+0000 7f850f694640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85080b0cd0 0x7f85080c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:49.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.117+0000 7f850f694640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85080cda60 0x7f85081e1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:49.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.117+0000 7f850f694640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85081485a0 con 0x7f85080cda60 2023-11-09T03:48:49.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.117+0000 7f850f694640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85081f4240 con 0x7f85080b0cd0 2023-11-09T03:48:49.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.117+0000 7f850f694640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85081f4520 con 0x7f8508070d40 2023-11-09T03:48:49.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.117+0000 7f850dc31640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85080cda60 0x7f85081e1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:49.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.118+0000 7f850dc31640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85080cda60 0x7f85081e1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52182/0 (socket says 172.21.15.76:52182) 2023-11-09T03:48:49.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.118+0000 7f850dc31640 1 -- 172.21.15.76:0/1237522568 learned_addr learned my addr 172.21.15.76:0/1237522568 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:49.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.118+0000 7f850d430640 1 --2- 172.21.15.76:0/1237522568 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8508070d40 0x7f85081f0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:49.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.118+0000 7f850cc2f640 1 --2- 172.21.15.76:0/1237522568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85080b0cd0 0x7f85080c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:49.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.118+0000 7f850dc31640 1 -- 172.21.15.76:0/1237522568 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8508070d40 msgr2=0x7f85081f0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:49.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.118+0000 7f850dc31640 1 --2- 172.21.15.76:0/1237522568 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8508070d40 0x7f85081f0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.118+0000 7f850dc31640 1 -- 172.21.15.76:0/1237522568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85080b0cd0 msgr2=0x7f85080c82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:49.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.118+0000 7f850dc31640 1 --2- 172.21.15.76:0/1237522568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85080b0cd0 0x7f85080c82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.118+0000 7f850dc31640 1 -- 172.21.15.76:0/1237522568 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8504009a70 con 0x7f85080cda60 2023-11-09T03:48:49.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.118+0000 7f850dc31640 1 --2- 172.21.15.76:0/1237522568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85080cda60 0x7f85081e1ab0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f850400efc0 tx=0x7f8504009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:49.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.118+0000 7f84fdffb640 1 -- 172.21.15.76:0/1237522568 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8504009d70 con 0x7f85080cda60 2023-11-09T03:48:49.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.118+0000 7f84fdffb640 1 -- 172.21.15.76:0/1237522568 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850400f040 con 0x7f85080cda60 2023-11-09T03:48:49.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.118+0000 7f84fdffb640 1 -- 172.21.15.76:0/1237522568 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8504003990 con 0x7f85080cda60 2023-11-09T03:48:49.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.118+0000 7f850f694640 1 -- 172.21.15.76:0/1237522568 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8500009400 con 0x7f85080cda60 2023-11-09T03:48:49.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.118+0000 7f850f694640 1 -- 172.21.15.76:0/1237522568 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85081f4ae0 con 0x7f85080cda60 2023-11-09T03:48:49.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.120+0000 7f850f694640 1 -- 172.21.15.76:0/1237522568 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8508148710 con 0x7f85080cda60 2023-11-09T03:48:49.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.120+0000 7f84fdffb640 1 -- 172.21.15.76:0/1237522568 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f85040144a0 con 0x7f85080cda60 2023-11-09T03:48:49.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.120+0000 7f84fdffb640 1 --2- 172.21.15.76:0/1237522568 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f84ec06ae70 0x7f84ec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:49.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.120+0000 7f84fdffb640 1 -- 172.21.15.76:0/1237522568 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6354+0+0 (secure 0 0 0) 0x7f850408a420 con 0x7f85080cda60 2023-11-09T03:48:49.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.120+0000 7f850d430640 1 --2- 172.21.15.76:0/1237522568 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f84ec06ae70 0x7f84ec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:49.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.121+0000 7f850d430640 1 --2- 172.21.15.76:0/1237522568 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f84ec06ae70 0x7f84ec06d330 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8500036000 tx=0x7f850003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:49.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.124+0000 7f84fdffb640 1 -- 172.21.15.76:0/1237522568 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f85040552f0 con 0x7f85080cda60 2023-11-09T03:48:49.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.291+0000 7f850f694640 1 -- 172.21.15.76:0/1237522568 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f850806b100 con 0x7f85080cda60 2023-11-09T03:48:49.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.292+0000 7f84fdffb640 1 -- 172.21.15.76:0/1237522568 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+18464 (secure 0 0 0) 0x7f8504058910 con 0x7f85080cda60 2023-11-09T03:48:49.293 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:49.294 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":87,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:48:47.991145+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-09T03:47:59.280601+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-09T03:47:59.742039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:6835/2165197913":"2023-11-10T03:48:44.982465+0000","172.21.15.76:6833/2165197913":"2023-11-10T03:48:44.982465+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:48:49.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.295+0000 7f850f694640 1 -- 172.21.15.76:0/1237522568 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f84ec06ae70 msgr2=0x7f84ec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:49.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.295+0000 7f850f694640 1 --2- 172.21.15.76:0/1237522568 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f84ec06ae70 0x7f84ec06d330 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8500036000 tx=0x7f850003c040 comp rx=0 tx=0).stop 2023-11-09T03:48:49.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.295+0000 7f850f694640 1 -- 172.21.15.76:0/1237522568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85080cda60 msgr2=0x7f85081e1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:49.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.295+0000 7f850f694640 1 --2- 172.21.15.76:0/1237522568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85080cda60 0x7f85081e1ab0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f850400efc0 tx=0x7f8504009f90 comp rx=0 tx=0).stop 2023-11-09T03:48:49.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.295+0000 7f850f694640 1 -- 172.21.15.76:0/1237522568 shutdown_connections 2023-11-09T03:48:49.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.295+0000 7f850f694640 1 --2- 172.21.15.76:0/1237522568 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f84ec06ae70 0x7f84ec06d330 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.295+0000 7f850f694640 1 --2- 172.21.15.76:0/1237522568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85080cda60 0x7f85081e1ab0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.295+0000 7f850f694640 1 --2- 172.21.15.76:0/1237522568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85080b0cd0 0x7f85080c82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.295+0000 7f850f694640 1 --2- 172.21.15.76:0/1237522568 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8508070d40 0x7f85081f0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.295+0000 7f850f694640 1 -- 172.21.15.76:0/1237522568 >> 172.21.15.76:0/1237522568 conn(0x7f85080ce950 msgr2=0x7f850806d900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:49.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.295+0000 7f850f694640 1 -- 172.21.15.76:0/1237522568 shutdown_connections 2023-11-09T03:48:49.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.295+0000 7f850f694640 1 -- 172.21.15.76:0/1237522568 wait complete. 2023-11-09T03:48:49.306 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-09T03:47:00.036540+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_data_pool', 'create_time': '2023-11-09T03:47:59.280601+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 70, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-09T03:47:59.742039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-09T03:48:49.307 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:48:49.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.408+0000 7f0af2ea9640 1 Processor -- start 2023-11-09T03:48:49.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.408+0000 7f0af2ea9640 1 -- start start 2023-11-09T03:48:49.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.409+0000 7f0af2ea9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0aec159c00 0x7f0aec0cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:49.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.409+0000 7f0af2ea9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0aec0c48e0 0x7f0aec0c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:49.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.409+0000 7f0af2ea9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aec15aff0 con 0x7f0aec0adac0 2023-11-09T03:48:49.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.409+0000 7f0af2ea9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aec0c52a0 con 0x7f0aec0c48e0 2023-11-09T03:48:49.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.409+0000 7f0af2ea9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aec15a3e0 con 0x7f0aec159c00 2023-11-09T03:48:49.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.409+0000 7f0aebfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0aec0c48e0 0x7f0aec0c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:49.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.409+0000 7f0af1446640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0aec0adac0 0x7f0aec0adef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48674/0 (socket says 172.21.15.76:48674) 2023-11-09T03:48:49.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.409+0000 7f0af1446640 1 -- 172.21.15.76:0/2042437251 learned_addr learned my addr 172.21.15.76:0/2042437251 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:49.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.409+0000 7f0af0c45640 1 --2- 172.21.15.76:0/2042437251 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0aec159c00 0x7f0aec0cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:49.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.410+0000 7f0aebfff640 1 -- 172.21.15.76:0/2042437251 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0aec159c00 msgr2=0x7f0aec0cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:49.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.410+0000 7f0aebfff640 1 --2- 172.21.15.76:0/2042437251 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0aec159c00 0x7f0aec0cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.410+0000 7f0aebfff640 1 -- 172.21.15.76:0/2042437251 >> v1:172.21.15.76:6789/0 conn(0x7f0aec0adac0 legacy=0x7f0aec0adef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:49.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.410+0000 7f0aebfff640 1 -- 172.21.15.76:0/2042437251 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aec15a6c0 con 0x7f0aec0c48e0 2023-11-09T03:48:49.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.410+0000 7f0aeb7fe640 1 -- 172.21.15.76:0/2042437251 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 302897213 0 0) 0x7f0aec15aff0 con 0x7f0aec0adac0 2023-11-09T03:48:49.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.410+0000 7f0af0c45640 1 --2- 172.21.15.76:0/2042437251 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0aec159c00 0x7f0aec0cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:49.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.410+0000 7f0aebfff640 1 --2- 172.21.15.76:0/2042437251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0aec0c48e0 0x7f0aec0c4d60 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00097a0 tx=0x7f0ae002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8947c3356826e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:49.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.410+0000 7f0aeb7fe640 1 -- 172.21.15.76:0/2042437251 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0031830 con 0x7f0aec0c48e0 2023-11-09T03:48:49.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.410+0000 7f0aeb7fe640 1 -- 172.21.15.76:0/2042437251 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae0031990 con 0x7f0aec0c48e0 2023-11-09T03:48:49.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.410+0000 7f0aeb7fe640 1 -- 172.21.15.76:0/2042437251 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0031d20 con 0x7f0aec0c48e0 2023-11-09T03:48:49.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.411+0000 7f0af2ea9640 1 -- 172.21.15.76:0/2042437251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0aec0c48e0 msgr2=0x7f0aec0c4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:49.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.411+0000 7f0af2ea9640 1 --2- 172.21.15.76:0/2042437251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0aec0c48e0 0x7f0aec0c4d60 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00097a0 tx=0x7f0ae002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.411+0000 7f0af2ea9640 1 -- 172.21.15.76:0/2042437251 shutdown_connections 2023-11-09T03:48:49.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.411+0000 7f0af2ea9640 1 --2- 172.21.15.76:0/2042437251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0aec0c48e0 0x7f0aec0c4d60 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.411+0000 7f0af2ea9640 1 --2- 172.21.15.76:0/2042437251 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0aec159c00 0x7f0aec0cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.411+0000 7f0af2ea9640 1 -- 172.21.15.76:0/2042437251 >> 172.21.15.76:0/2042437251 conn(0x7f0aec0ce950 msgr2=0x7f0aec0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:49.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.411+0000 7f0af2ea9640 1 -- 172.21.15.76:0/2042437251 shutdown_connections 2023-11-09T03:48:49.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.411+0000 7f0af2ea9640 1 -- 172.21.15.76:0/2042437251 wait complete. 2023-11-09T03:48:49.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.412+0000 7f0af2ea9640 1 Processor -- start 2023-11-09T03:48:49.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.412+0000 7f0af2ea9640 1 -- start start 2023-11-09T03:48:49.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.413+0000 7f0af2ea9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0aec0adac0 0x7f0aec145df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:49.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.413+0000 7f0af2ea9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0aec0c48e0 0x7f0aec15c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:49.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.413+0000 7f0af2ea9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0aec159c00 0x7f0aec15c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:49.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.413+0000 7f0af2ea9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aec146330 con 0x7f0aec0adac0 2023-11-09T03:48:49.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.413+0000 7f0af2ea9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aec1ef3e0 con 0x7f0aec159c00 2023-11-09T03:48:49.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.413+0000 7f0af2ea9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aec1ee620 con 0x7f0aec0c48e0 2023-11-09T03:48:49.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.413+0000 7f0af0c45640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0aec0adac0 0x7f0aec145df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:49.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.413+0000 7f0af0c45640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0aec0adac0 0x7f0aec145df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52210/0 (socket says 172.21.15.76:52210) 2023-11-09T03:48:49.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.413+0000 7f0af0c45640 1 -- 172.21.15.76:0/3899305678 learned_addr learned my addr 172.21.15.76:0/3899305678 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:49.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.413+0000 7f0aebfff640 1 --2- 172.21.15.76:0/3899305678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0aec0c48e0 0x7f0aec15c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:49.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.413+0000 7f0af1446640 1 --2- 172.21.15.76:0/3899305678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0aec159c00 0x7f0aec15c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:49.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.413+0000 7f0af0c45640 1 -- 172.21.15.76:0/3899305678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0aec0c48e0 msgr2=0x7f0aec15c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:49.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.413+0000 7f0af0c45640 1 --2- 172.21.15.76:0/3899305678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0aec0c48e0 0x7f0aec15c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.413+0000 7f0af0c45640 1 -- 172.21.15.76:0/3899305678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0aec159c00 msgr2=0x7f0aec15c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:49.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.413+0000 7f0af0c45640 1 --2- 172.21.15.76:0/3899305678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0aec159c00 0x7f0aec15c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.413+0000 7f0af0c45640 1 -- 172.21.15.76:0/3899305678 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae0009400 con 0x7f0aec0adac0 2023-11-09T03:48:49.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.413+0000 7f0af1446640 1 --2- 172.21.15.76:0/3899305678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0aec159c00 0x7f0aec15c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:49.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.414+0000 7f0af0c45640 1 --2- 172.21.15.76:0/3899305678 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0aec0adac0 0x7f0aec145df0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f0adc00c7f0 tx=0x7f0adc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:49.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.414+0000 7f0ae97fa640 1 -- 172.21.15.76:0/3899305678 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0adc007a30 con 0x7f0aec0adac0 2023-11-09T03:48:49.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.414+0000 7f0ae97fa640 1 -- 172.21.15.76:0/3899305678 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0adc007b90 con 0x7f0aec0adac0 2023-11-09T03:48:49.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.414+0000 7f0ae97fa640 1 -- 172.21.15.76:0/3899305678 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0adc00ce40 con 0x7f0aec0adac0 2023-11-09T03:48:49.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.414+0000 7f0af2ea9640 1 -- 172.21.15.76:0/3899305678 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aec1ee900 con 0x7f0aec0adac0 2023-11-09T03:48:49.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.414+0000 7f0af2ea9640 1 -- 172.21.15.76:0/3899305678 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aec1eeec0 con 0x7f0aec0adac0 2023-11-09T03:48:49.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.415+0000 7f0af2ea9640 1 -- 172.21.15.76:0/3899305678 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aec074290 con 0x7f0aec0adac0 2023-11-09T03:48:49.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.415+0000 7f0ae97fa640 1 -- 172.21.15.76:0/3899305678 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0adc0113f0 con 0x7f0aec0adac0 2023-11-09T03:48:49.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.415+0000 7f0ae97fa640 1 --2- 172.21.15.76:0/3899305678 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0ad806ae70 0x7f0ad806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:49.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.416+0000 7f0ae97fa640 1 -- 172.21.15.76:0/3899305678 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6354+0+0 (secure 0 0 0) 0x7f0adc089a40 con 0x7f0aec0adac0 2023-11-09T03:48:49.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.416+0000 7f0aebfff640 1 --2- 172.21.15.76:0/3899305678 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0ad806ae70 0x7f0ad806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:49.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.416+0000 7f0aebfff640 1 --2- 172.21.15.76:0/3899305678 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0ad806ae70 0x7f0ad806d330 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0036000 tx=0x7f0ae003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:49.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.420+0000 7f0ae97fa640 1 -- 172.21.15.76:0/3899305678 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0adc08be60 con 0x7f0aec0adac0 2023-11-09T03:48:49.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.587+0000 7f0af2ea9640 1 -- 172.21.15.76:0/3899305678 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0aec0c3f30 con 0x7f0aec0adac0 2023-11-09T03:48:49.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.587+0000 7f0ae97fa640 1 -- 172.21.15.76:0/3899305678 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0adc01f0a0 con 0x7f0aec0adac0 2023-11-09T03:48:49.588 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:48:49.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.589+0000 7f0af2ea9640 1 -- 172.21.15.76:0/3899305678 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0ad806ae70 msgr2=0x7f0ad806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:49.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.589+0000 7f0af2ea9640 1 --2- 172.21.15.76:0/3899305678 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0ad806ae70 0x7f0ad806d330 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0036000 tx=0x7f0ae003c040 comp rx=0 tx=0).stop 2023-11-09T03:48:49.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.589+0000 7f0af2ea9640 1 -- 172.21.15.76:0/3899305678 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0aec0adac0 msgr2=0x7f0aec145df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:49.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.589+0000 7f0af2ea9640 1 --2- 172.21.15.76:0/3899305678 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0aec0adac0 0x7f0aec145df0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f0adc00c7f0 tx=0x7f0adc00cbf0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.589+0000 7f0af2ea9640 1 -- 172.21.15.76:0/3899305678 shutdown_connections 2023-11-09T03:48:49.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.589+0000 7f0af2ea9640 1 --2- 172.21.15.76:0/3899305678 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0ad806ae70 0x7f0ad806d330 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.589+0000 7f0af2ea9640 1 --2- 172.21.15.76:0/3899305678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0aec159c00 0x7f0aec15c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.589+0000 7f0af2ea9640 1 --2- 172.21.15.76:0/3899305678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0aec0c48e0 0x7f0aec15c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.589+0000 7f0af2ea9640 1 --2- 172.21.15.76:0/3899305678 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0aec0adac0 0x7f0aec145df0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.589+0000 7f0af2ea9640 1 -- 172.21.15.76:0/3899305678 >> 172.21.15.76:0/3899305678 conn(0x7f0aec0ce950 msgr2=0x7f0aec06dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:49.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.589+0000 7f0af2ea9640 1 -- 172.21.15.76:0/3899305678 shutdown_connections 2023-11-09T03:48:49.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.589+0000 7f0af2ea9640 1 -- 172.21.15.76:0/3899305678 wait complete. 2023-11-09T03:48:49.600 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-09T03:48:49.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.703+0000 7f723ac0e640 1 Processor -- start 2023-11-09T03:48:49.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.703+0000 7f723ac0e640 1 -- start start 2023-11-09T03:48:49.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.704+0000 7f723ac0e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7234159c00 0x7f72340cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:49.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.704+0000 7f723ac0e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72340c48e0 0x7f72340c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:49.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.704+0000 7f723ac0e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f723415aff0 con 0x7f72340adac0 2023-11-09T03:48:49.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.704+0000 7f723ac0e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72340c52d0 con 0x7f7234159c00 2023-11-09T03:48:49.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.704+0000 7f723ac0e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723415a3e0 con 0x7f72340c48e0 2023-11-09T03:48:49.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.704+0000 7f72389aa640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7234159c00 0x7f72340cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:49.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.704+0000 7f72391ab640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f72340adac0 0x7f72340adf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:47078/0 (socket says 172.21.15.76:47078) 2023-11-09T03:48:49.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.704+0000 7f72391ab640 1 -- 172.21.15.76:0/211210767 learned_addr learned my addr 172.21.15.76:0/211210767 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:49.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.704+0000 7f7233fff640 1 --2- 172.21.15.76:0/211210767 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72340c48e0 0x7f72340c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:49.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.704+0000 7f72389aa640 1 -- 172.21.15.76:0/211210767 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72340c48e0 msgr2=0x7f72340c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:49.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.704+0000 7f72389aa640 1 --2- 172.21.15.76:0/211210767 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72340c48e0 0x7f72340c4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.704+0000 7f72389aa640 1 -- 172.21.15.76:0/211210767 >> v1:172.21.15.113:6789/0 conn(0x7f72340adac0 legacy=0x7f72340adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:49.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.704+0000 7f72389aa640 1 -- 172.21.15.76:0/211210767 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723415a6c0 con 0x7f7234159c00 2023-11-09T03:48:49.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.704+0000 7f7233fff640 1 --2- 172.21.15.76:0/211210767 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72340c48e0 0x7f72340c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:48:49.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.705+0000 7f72389aa640 1 --2- 172.21.15.76:0/211210767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7234159c00 0x7f72340cc890 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f72280097a0 tx=0x7f722802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42534a3bcb113254 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:49.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.705+0000 7f72337fe640 1 -- 172.21.15.76:0/211210767 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7228031830 con 0x7f7234159c00 2023-11-09T03:48:49.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.705+0000 7f72337fe640 1 -- 172.21.15.76:0/211210767 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7228031990 con 0x7f7234159c00 2023-11-09T03:48:49.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.705+0000 7f72337fe640 1 -- 172.21.15.76:0/211210767 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7228031d20 con 0x7f7234159c00 2023-11-09T03:48:49.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.706+0000 7f723ac0e640 1 -- 172.21.15.76:0/211210767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7234159c00 msgr2=0x7f72340cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:49.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.706+0000 7f723ac0e640 1 --2- 172.21.15.76:0/211210767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7234159c00 0x7f72340cc890 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f72280097a0 tx=0x7f722802b8d0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.706+0000 7f723ac0e640 1 -- 172.21.15.76:0/211210767 shutdown_connections 2023-11-09T03:48:49.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.706+0000 7f723ac0e640 1 --2- 172.21.15.76:0/211210767 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72340c48e0 0x7f72340c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.706+0000 7f723ac0e640 1 --2- 172.21.15.76:0/211210767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7234159c00 0x7f72340cc890 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.706+0000 7f723ac0e640 1 -- 172.21.15.76:0/211210767 >> 172.21.15.76:0/211210767 conn(0x7f72340ce950 msgr2=0x7f72340ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:49.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.706+0000 7f723ac0e640 1 -- 172.21.15.76:0/211210767 shutdown_connections 2023-11-09T03:48:49.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.706+0000 7f723ac0e640 1 -- 172.21.15.76:0/211210767 wait complete. 2023-11-09T03:48:49.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.707+0000 7f723ac0e640 1 Processor -- start 2023-11-09T03:48:49.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.707+0000 7f723ac0e640 1 -- start start 2023-11-09T03:48:49.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.708+0000 7f723ac0e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72340adac0 0x7f7234143220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:49.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.708+0000 7f723ac0e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72340c48e0 0x7f723414f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:49.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.708+0000 7f723ac0e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7234159c00 0x7f72341f1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:49.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.708+0000 7f723ac0e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72341f2b50 con 0x7f72340c48e0 2023-11-09T03:48:49.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.708+0000 7f723ac0e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72340c7770 con 0x7f7234159c00 2023-11-09T03:48:49.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.708+0000 7f723ac0e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7234143760 con 0x7f72340adac0 2023-11-09T03:48:49.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.708+0000 7f7233fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72340c48e0 0x7f723414f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:49.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.708+0000 7f7233fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72340c48e0 0x7f723414f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52224/0 (socket says 172.21.15.76:52224) 2023-11-09T03:48:49.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.708+0000 7f7233fff640 1 -- 172.21.15.76:0/3999263614 learned_addr learned my addr 172.21.15.76:0/3999263614 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:49.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.708+0000 7f72391ab640 1 --2- 172.21.15.76:0/3999263614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7234159c00 0x7f72341f1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:49.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.708+0000 7f72389aa640 1 --2- 172.21.15.76:0/3999263614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72340adac0 0x7f7234143220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:49.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.708+0000 7f7233fff640 1 -- 172.21.15.76:0/3999263614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72340adac0 msgr2=0x7f7234143220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:49.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.708+0000 7f7233fff640 1 --2- 172.21.15.76:0/3999263614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72340adac0 0x7f7234143220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.708+0000 7f7233fff640 1 -- 172.21.15.76:0/3999263614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7234159c00 msgr2=0x7f72341f1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:49.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.708+0000 7f7233fff640 1 --2- 172.21.15.76:0/3999263614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7234159c00 0x7f72341f1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.709+0000 7f7233fff640 1 -- 172.21.15.76:0/3999263614 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7220009400 con 0x7f72340c48e0 2023-11-09T03:48:49.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.709+0000 7f72391ab640 1 --2- 172.21.15.76:0/3999263614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7234159c00 0x7f72341f1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:49.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.709+0000 7f72389aa640 1 --2- 172.21.15.76:0/3999263614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72340adac0 0x7f7234143220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:49.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.709+0000 7f7233fff640 1 --2- 172.21.15.76:0/3999263614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72340c48e0 0x7f723414f950 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f722000ebe0 tx=0x7f722000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:49.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.709+0000 7f72317fa640 1 -- 172.21.15.76:0/3999263614 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7220009760 con 0x7f72340c48e0 2023-11-09T03:48:49.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.709+0000 7f72317fa640 1 -- 172.21.15.76:0/3999263614 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f722000cc30 con 0x7f72340c48e0 2023-11-09T03:48:49.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.709+0000 7f723ac0e640 1 -- 172.21.15.76:0/3999263614 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7228009400 con 0x7f72340c48e0 2023-11-09T03:48:49.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.709+0000 7f72317fa640 1 -- 172.21.15.76:0/3999263614 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722000eee0 con 0x7f72340c48e0 2023-11-09T03:48:49.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.709+0000 7f723ac0e640 1 -- 172.21.15.76:0/3999263614 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72341f1f40 con 0x7f72340c48e0 2023-11-09T03:48:49.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.710+0000 7f723ac0e640 1 -- 172.21.15.76:0/3999263614 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72341f2d00 con 0x7f72340c48e0 2023-11-09T03:48:49.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.710+0000 7f72317fa640 1 -- 172.21.15.76:0/3999263614 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7220013470 con 0x7f72340c48e0 2023-11-09T03:48:49.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.711+0000 7f72317fa640 1 --2- 172.21.15.76:0/3999263614 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f721806ae70 0x7f721806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:49.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.711+0000 7f72317fa640 1 -- 172.21.15.76:0/3999263614 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6354+0+0 (secure 0 0 0) 0x7f7220089bf0 con 0x7f72340c48e0 2023-11-09T03:48:49.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.711+0000 7f72389aa640 1 --2- 172.21.15.76:0/3999263614 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f721806ae70 0x7f721806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:49.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.712+0000 7f72389aa640 1 --2- 172.21.15.76:0/3999263614 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f721806ae70 0x7f721806d330 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7228036000 tx=0x7f722803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:49.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.715+0000 7f72317fa640 1 -- 172.21.15.76:0/3999263614 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f72341f2d00 con 0x7f72340c48e0 2023-11-09T03:48:49.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.881+0000 7f723ac0e640 1 -- 172.21.15.76:0/3999263614 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f7234074290 con 0x7f72340c48e0 2023-11-09T03:48:49.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.881+0000 7f72317fa640 1 -- 172.21.15.76:0/3999263614 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f72200170a0 con 0x7f72340c48e0 2023-11-09T03:48:49.882 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:48:49.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.883+0000 7f723ac0e640 1 -- 172.21.15.76:0/3999263614 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f721806ae70 msgr2=0x7f721806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:49.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.883+0000 7f723ac0e640 1 --2- 172.21.15.76:0/3999263614 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f721806ae70 0x7f721806d330 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7228036000 tx=0x7f722803c040 comp rx=0 tx=0).stop 2023-11-09T03:48:49.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.883+0000 7f723ac0e640 1 -- 172.21.15.76:0/3999263614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72340c48e0 msgr2=0x7f723414f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:49.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.883+0000 7f723ac0e640 1 --2- 172.21.15.76:0/3999263614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72340c48e0 0x7f723414f950 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f722000ebe0 tx=0x7f722000c520 comp rx=0 tx=0).stop 2023-11-09T03:48:49.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.883+0000 7f723ac0e640 1 -- 172.21.15.76:0/3999263614 shutdown_connections 2023-11-09T03:48:49.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.883+0000 7f723ac0e640 1 --2- 172.21.15.76:0/3999263614 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f721806ae70 0x7f721806d330 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.883+0000 7f723ac0e640 1 --2- 172.21.15.76:0/3999263614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7234159c00 0x7f72341f1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.883+0000 7f723ac0e640 1 --2- 172.21.15.76:0/3999263614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72340c48e0 0x7f723414f950 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.883+0000 7f723ac0e640 1 --2- 172.21.15.76:0/3999263614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72340adac0 0x7f7234143220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:49.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.883+0000 7f723ac0e640 1 -- 172.21.15.76:0/3999263614 >> 172.21.15.76:0/3999263614 conn(0x7f72340ce950 msgr2=0x7f723415b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:49.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.883+0000 7f723ac0e640 1 -- 172.21.15.76:0/3999263614 shutdown_connections 2023-11-09T03:48:49.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.883+0000 7f723ac0e640 1 -- 172.21.15.76:0/3999263614 wait complete. 2023-11-09T03:48:49.894 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-09T03:48:49.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.997+0000 7f10cd27c640 1 Processor -- start 2023-11-09T03:48:49.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.997+0000 7f10cd27c640 1 -- start start 2023-11-09T03:48:50.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.998+0000 7f10cd27c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f10c8159c00 0x7f10c80cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:50.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.998+0000 7f10cd27c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10c80ae1d0 0x7f10c80c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:50.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.998+0000 7f10cd27c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10c815aff0 con 0x7f10c80c48e0 2023-11-09T03:48:50.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.998+0000 7f10cd27c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10c80c52d0 con 0x7f10c8159c00 2023-11-09T03:48:50.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.998+0000 7f10cd27c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c815a3e0 con 0x7f10c80ae1d0 2023-11-09T03:48:50.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.998+0000 7f10c67fc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f10c80c48e0 0x7f10c80adac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48688/0 (socket says 172.21.15.76:48688) 2023-11-09T03:48:50.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.998+0000 7f10c67fc640 1 -- 172.21.15.76:0/4108658795 learned_addr learned my addr 172.21.15.76:0/4108658795 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:50.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.998+0000 7f10c77fe640 1 --2- 172.21.15.76:0/4108658795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10c80ae1d0 0x7f10c80c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:50.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.998+0000 7f10c6ffd640 1 --2- 172.21.15.76:0/4108658795 >> v2:172.21.15.113:3300/0 conn(0x7f10c8159c00 0x7f10c80cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:50.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.998+0000 7f10c77fe640 1 -- 172.21.15.76:0/4108658795 >> v2:172.21.15.113:3300/0 conn(0x7f10c8159c00 msgr2=0x7f10c80cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:50.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.998+0000 7f10c77fe640 1 --2- 172.21.15.76:0/4108658795 >> v2:172.21.15.113:3300/0 conn(0x7f10c8159c00 0x7f10c80cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:50.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.998+0000 7f10c77fe640 1 -- 172.21.15.76:0/4108658795 >> v1:172.21.15.76:6789/0 conn(0x7f10c80c48e0 legacy=0x7f10c80adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:50.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.998+0000 7f10c77fe640 1 -- 172.21.15.76:0/4108658795 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c815a6c0 con 0x7f10c80ae1d0 2023-11-09T03:48:50.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:49.999+0000 7f10c77fe640 1 --2- 172.21.15.76:0/4108658795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10c80ae1d0 0x7f10c80c4d90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f10c0009d90 tx=0x7f10c002be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a8461336803dd6d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:50.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.001+0000 7f10c5ffb640 1 -- 172.21.15.76:0/4108658795 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c002cbc0 con 0x7f10c80ae1d0 2023-11-09T03:48:50.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.001+0000 7f10c5ffb640 1 -- 172.21.15.76:0/4108658795 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c002cd20 con 0x7f10c80ae1d0 2023-11-09T03:48:50.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.001+0000 7f10c5ffb640 1 -- 172.21.15.76:0/4108658795 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c0034660 con 0x7f10c80ae1d0 2023-11-09T03:48:50.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.002+0000 7f10cd27c640 1 -- 172.21.15.76:0/4108658795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10c80ae1d0 msgr2=0x7f10c80c4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:50.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.002+0000 7f10cd27c640 1 --2- 172.21.15.76:0/4108658795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10c80ae1d0 0x7f10c80c4d90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f10c0009d90 tx=0x7f10c002be10 comp rx=0 tx=0).stop 2023-11-09T03:48:50.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.002+0000 7f10cd27c640 1 -- 172.21.15.76:0/4108658795 shutdown_connections 2023-11-09T03:48:50.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.002+0000 7f10cd27c640 1 --2- 172.21.15.76:0/4108658795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10c80ae1d0 0x7f10c80c4d90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:50.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.002+0000 7f10cd27c640 1 --2- 172.21.15.76:0/4108658795 >> v2:172.21.15.113:3300/0 conn(0x7f10c8159c00 0x7f10c80cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:50.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.002+0000 7f10cd27c640 1 -- 172.21.15.76:0/4108658795 >> 172.21.15.76:0/4108658795 conn(0x7f10c80ce950 msgr2=0x7f10c80ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:50.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.002+0000 7f10cd27c640 1 -- 172.21.15.76:0/4108658795 shutdown_connections 2023-11-09T03:48:50.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.002+0000 7f10cd27c640 1 -- 172.21.15.76:0/4108658795 wait complete. 2023-11-09T03:48:50.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.003+0000 7f10cd27c640 1 Processor -- start 2023-11-09T03:48:50.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.003+0000 7f10cd27c640 1 -- start start 2023-11-09T03:48:50.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.004+0000 7f10cd27c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10c80ae1d0 0x7f10c814f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:50.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.004+0000 7f10cd27c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10c80c48e0 0x7f10c81f15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:50.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.004+0000 7f10cd27c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10c8159c00 0x7f10c81f1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:50.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.004+0000 7f10cd27c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c814e600 con 0x7f10c80c48e0 2023-11-09T03:48:50.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.004+0000 7f10cd27c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c814e8e0 con 0x7f10c80ae1d0 2023-11-09T03:48:50.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.004+0000 7f10cd27c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c814ebc0 con 0x7f10c8159c00 2023-11-09T03:48:50.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.004+0000 7f10c67fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10c80c48e0 0x7f10c81f15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:50.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.004+0000 7f10c67fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10c80c48e0 0x7f10c81f15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52238/0 (socket says 172.21.15.76:52238) 2023-11-09T03:48:50.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.004+0000 7f10c67fc640 1 -- 172.21.15.76:0/3260241191 learned_addr learned my addr 172.21.15.76:0/3260241191 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:50.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.004+0000 7f10c77fe640 1 --2- 172.21.15.76:0/3260241191 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10c8159c00 0x7f10c81f1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:50.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.004+0000 7f10c6ffd640 1 --2- 172.21.15.76:0/3260241191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10c80ae1d0 0x7f10c814f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:50.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.004+0000 7f10c67fc640 1 -- 172.21.15.76:0/3260241191 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10c8159c00 msgr2=0x7f10c81f1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:50.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.004+0000 7f10c67fc640 1 --2- 172.21.15.76:0/3260241191 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10c8159c00 0x7f10c81f1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:50.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.004+0000 7f10c67fc640 1 -- 172.21.15.76:0/3260241191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10c80ae1d0 msgr2=0x7f10c814f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:50.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.004+0000 7f10c67fc640 1 --2- 172.21.15.76:0/3260241191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10c80ae1d0 0x7f10c814f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:50.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.004+0000 7f10c67fc640 1 -- 172.21.15.76:0/3260241191 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c0009a70 con 0x7f10c80c48e0 2023-11-09T03:48:50.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.004+0000 7f10c67fc640 1 --2- 172.21.15.76:0/3260241191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10c80c48e0 0x7f10c81f15e0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f10b000c4a0 tx=0x7f10b000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:50.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.004+0000 7f10bf7fe640 1 -- 172.21.15.76:0/3260241191 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0005ce0 con 0x7f10c80c48e0 2023-11-09T03:48:50.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.005+0000 7f10bf7fe640 1 -- 172.21.15.76:0/3260241191 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b0005e40 con 0x7f10c80c48e0 2023-11-09T03:48:50.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.005+0000 7f10bf7fe640 1 -- 172.21.15.76:0/3260241191 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0007270 con 0x7f10c80c48e0 2023-11-09T03:48:50.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.005+0000 7f10cd27c640 1 -- 172.21.15.76:0/3260241191 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c814eea0 con 0x7f10c80c48e0 2023-11-09T03:48:50.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.005+0000 7f10cd27c640 1 -- 172.21.15.76:0/3260241191 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c814fbe0 con 0x7f10c80c48e0 2023-11-09T03:48:50.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.006+0000 7f10cd27c640 1 -- 172.21.15.76:0/3260241191 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10c814f3c0 con 0x7f10c80c48e0 2023-11-09T03:48:50.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.006+0000 7f10bf7fe640 1 -- 172.21.15.76:0/3260241191 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f10b0007490 con 0x7f10c80c48e0 2023-11-09T03:48:50.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.006+0000 7f10bf7fe640 1 --2- 172.21.15.76:0/3260241191 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f10a806ae70 0x7f10a806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:50.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.006+0000 7f10bf7fe640 1 -- 172.21.15.76:0/3260241191 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6354+0+0 (secure 0 0 0) 0x7f10b0088ce0 con 0x7f10c80c48e0 2023-11-09T03:48:50.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.006+0000 7f10c6ffd640 1 --2- 172.21.15.76:0/3260241191 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f10a806ae70 0x7f10a806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:50.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.007+0000 7f10c6ffd640 1 --2- 172.21.15.76:0/3260241191 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f10a806ae70 0x7f10a806d330 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f10c80cf360 tx=0x7f10b8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:50.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.010+0000 7f10bf7fe640 1 -- 172.21.15.76:0/3260241191 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f10c814f3c0 con 0x7f10c80c48e0 2023-11-09T03:48:50.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.175+0000 7f10cd27c640 1 -- 172.21.15.76:0/3260241191 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7f10c814f3c0 con 0x7f10c80c48e0 2023-11-09T03:48:50.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.175+0000 7f10bf7fe640 1 -- 172.21.15.76:0/3260241191 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f10c814f3c0 con 0x7f10c80c48e0 2023-11-09T03:48:50.176 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:48:50.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.177+0000 7f10cd27c640 1 -- 172.21.15.76:0/3260241191 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f10a806ae70 msgr2=0x7f10a806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:50.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.177+0000 7f10cd27c640 1 --2- 172.21.15.76:0/3260241191 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f10a806ae70 0x7f10a806d330 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f10c80cf360 tx=0x7f10b8009190 comp rx=0 tx=0).stop 2023-11-09T03:48:50.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.177+0000 7f10cd27c640 1 -- 172.21.15.76:0/3260241191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10c80c48e0 msgr2=0x7f10c81f15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:50.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.177+0000 7f10cd27c640 1 --2- 172.21.15.76:0/3260241191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10c80c48e0 0x7f10c81f15e0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f10b000c4a0 tx=0x7f10b000c960 comp rx=0 tx=0).stop 2023-11-09T03:48:50.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.177+0000 7f10cd27c640 1 -- 172.21.15.76:0/3260241191 shutdown_connections 2023-11-09T03:48:50.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.177+0000 7f10cd27c640 1 --2- 172.21.15.76:0/3260241191 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f10a806ae70 0x7f10a806d330 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:50.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.177+0000 7f10cd27c640 1 --2- 172.21.15.76:0/3260241191 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10c8159c00 0x7f10c81f1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:50.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.177+0000 7f10cd27c640 1 --2- 172.21.15.76:0/3260241191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10c80c48e0 0x7f10c81f15e0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:50.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.177+0000 7f10cd27c640 1 --2- 172.21.15.76:0/3260241191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10c80ae1d0 0x7f10c814f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:50.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.177+0000 7f10cd27c640 1 -- 172.21.15.76:0/3260241191 >> 172.21.15.76:0/3260241191 conn(0x7f10c80ce950 msgr2=0x7f10c806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:50.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.177+0000 7f10cd27c640 1 -- 172.21.15.76:0/3260241191 shutdown_connections 2023-11-09T03:48:50.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.177+0000 7f10cd27c640 1 -- 172.21.15.76:0/3260241191 wait complete. 2023-11-09T03:48:50.189 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-09T03:48:50.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.292+0000 7f297207c640 1 Processor -- start 2023-11-09T03:48:50.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.292+0000 7f297207c640 1 -- start start 2023-11-09T03:48:50.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.292+0000 7f297207c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f296c153e60 0x7f296c15dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:50.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.292+0000 7f297207c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f296c0b4880 0x7f296c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:50.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.292+0000 7f297207c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f296c0cde60 con 0x7f296c0cd6d0 2023-11-09T03:48:50.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.292+0000 7f297207c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f296c0b5230 con 0x7f296c0b4880 2023-11-09T03:48:50.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.292+0000 7f297207c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296c160f90 con 0x7f296c153e60 2023-11-09T03:48:50.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.293+0000 7f296bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f296c0b4880 0x7f296c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:50.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.293+0000 7f296affd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f296c0cd6d0 0x7f296c161420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48696/0 (socket says 172.21.15.76:48696) 2023-11-09T03:48:50.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.293+0000 7f296affd640 1 -- 172.21.15.76:0/2665829680 learned_addr learned my addr 172.21.15.76:0/2665829680 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:50.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.293+0000 7f296b7fe640 1 --2- 172.21.15.76:0/2665829680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f296c153e60 0x7f296c15dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:50.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.293+0000 7f296bfff640 1 -- 172.21.15.76:0/2665829680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f296c153e60 msgr2=0x7f296c15dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:50.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.293+0000 7f296bfff640 1 --2- 172.21.15.76:0/2665829680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f296c153e60 0x7f296c15dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:50.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.293+0000 7f296bfff640 1 -- 172.21.15.76:0/2665829680 >> v1:172.21.15.76:6789/0 conn(0x7f296c0cd6d0 legacy=0x7f296c161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:50.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.293+0000 7f296a7fc640 1 -- 172.21.15.76:0/2665829680 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2515651544 0 0) 0x7f296c0cde60 con 0x7f296c0cd6d0 2023-11-09T03:48:50.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.293+0000 7f296bfff640 1 -- 172.21.15.76:0/2665829680 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296c15e400 con 0x7f296c0b4880 2023-11-09T03:48:50.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.293+0000 7f296b7fe640 1 --2- 172.21.15.76:0/2665829680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f296c153e60 0x7f296c15dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:50.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.294+0000 7f296bfff640 1 --2- 172.21.15.76:0/2665829680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f296c0b4880 0x7f296c0b4cf0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f2964009d90 tx=0x7f296402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f2aa0f192d3cbd server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:50.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.294+0000 7f296a7fc640 1 -- 172.21.15.76:0/2665829680 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296402cc50 con 0x7f296c0b4880 2023-11-09T03:48:50.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.294+0000 7f296a7fc640 1 -- 172.21.15.76:0/2665829680 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296402cdb0 con 0x7f296c0b4880 2023-11-09T03:48:50.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.294+0000 7f296a7fc640 1 -- 172.21.15.76:0/2665829680 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2964034660 con 0x7f296c0b4880 2023-11-09T03:48:50.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.294+0000 7f297207c640 1 -- 172.21.15.76:0/2665829680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f296c0b4880 msgr2=0x7f296c0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:50.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.294+0000 7f297207c640 1 --2- 172.21.15.76:0/2665829680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f296c0b4880 0x7f296c0b4cf0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f2964009d90 tx=0x7f296402bea0 comp rx=0 tx=0).stop 2023-11-09T03:48:50.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.294+0000 7f297207c640 1 -- 172.21.15.76:0/2665829680 shutdown_connections 2023-11-09T03:48:50.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.294+0000 7f297207c640 1 --2- 172.21.15.76:0/2665829680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f296c0b4880 0x7f296c0b4cf0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:50.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.294+0000 7f297207c640 1 --2- 172.21.15.76:0/2665829680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f296c153e60 0x7f296c15dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:50.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.294+0000 7f297207c640 1 -- 172.21.15.76:0/2665829680 >> 172.21.15.76:0/2665829680 conn(0x7f296c0cf880 msgr2=0x7f296c0cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:50.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.294+0000 7f297207c640 1 -- 172.21.15.76:0/2665829680 shutdown_connections 2023-11-09T03:48:50.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.294+0000 7f297207c640 1 -- 172.21.15.76:0/2665829680 wait complete. 2023-11-09T03:48:50.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.295+0000 7f297207c640 1 Processor -- start 2023-11-09T03:48:50.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.296+0000 7f297207c640 1 -- start start 2023-11-09T03:48:50.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.296+0000 7f297207c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f296c0b4880 0x7f296c152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:50.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.296+0000 7f297207c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f296c0cd6d0 0x7f296c15e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:50.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.296+0000 7f297207c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f296c153e60 0x7f296c15eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:50.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.296+0000 7f297207c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c15aa20 con 0x7f296c0b4880 2023-11-09T03:48:50.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.296+0000 7f297207c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c152dc0 con 0x7f296c153e60 2023-11-09T03:48:50.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.296+0000 7f297207c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296c159e10 con 0x7f296c0cd6d0 2023-11-09T03:48:50.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.296+0000 7f296b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f296c0b4880 0x7f296c152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:50.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.296+0000 7f296b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f296c0b4880 0x7f296c152880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52254/0 (socket says 172.21.15.76:52254) 2023-11-09T03:48:50.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.297+0000 7f296b7fe640 1 -- 172.21.15.76:0/1588816630 learned_addr learned my addr 172.21.15.76:0/1588816630 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:50.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.297+0000 7f296affd640 1 --2- 172.21.15.76:0/1588816630 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f296c0cd6d0 0x7f296c15e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:50.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.297+0000 7f296bfff640 1 --2- 172.21.15.76:0/1588816630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f296c153e60 0x7f296c15eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:50.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.297+0000 7f296b7fe640 1 -- 172.21.15.76:0/1588816630 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f296c0cd6d0 msgr2=0x7f296c15e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:50.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.297+0000 7f296b7fe640 1 --2- 172.21.15.76:0/1588816630 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f296c0cd6d0 0x7f296c15e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:50.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.297+0000 7f296b7fe640 1 -- 172.21.15.76:0/1588816630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f296c153e60 msgr2=0x7f296c15eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:50.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.297+0000 7f296b7fe640 1 --2- 172.21.15.76:0/1588816630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f296c153e60 0x7f296c15eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:50.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.297+0000 7f296b7fe640 1 -- 172.21.15.76:0/1588816630 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2964009a70 con 0x7f296c0b4880 2023-11-09T03:48:50.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.297+0000 7f296b7fe640 1 --2- 172.21.15.76:0/1588816630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f296c0b4880 0x7f296c152880 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f295c00c7f0 tx=0x7f295c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:50.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.297+0000 7f2953fff640 1 -- 172.21.15.76:0/1588816630 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c007820 con 0x7f296c0b4880 2023-11-09T03:48:50.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.297+0000 7f2953fff640 1 -- 172.21.15.76:0/1588816630 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295c007980 con 0x7f296c0b4880 2023-11-09T03:48:50.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.297+0000 7f297207c640 1 -- 172.21.15.76:0/1588816630 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f296c15a0f0 con 0x7f296c0b4880 2023-11-09T03:48:50.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.297+0000 7f2953fff640 1 -- 172.21.15.76:0/1588816630 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c007cf0 con 0x7f296c0b4880 2023-11-09T03:48:50.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.297+0000 7f297207c640 1 -- 172.21.15.76:0/1588816630 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296c15a6b0 con 0x7f296c0b4880 2023-11-09T03:48:50.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.298+0000 7f297207c640 1 -- 172.21.15.76:0/1588816630 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f296c0d2020 con 0x7f296c0b4880 2023-11-09T03:48:50.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.299+0000 7f2953fff640 1 -- 172.21.15.76:0/1588816630 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f295c0113f0 con 0x7f296c0b4880 2023-11-09T03:48:50.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.299+0000 7f2953fff640 1 --2- 172.21.15.76:0/1588816630 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f295806ae70 0x7f295806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:50.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.299+0000 7f2953fff640 1 -- 172.21.15.76:0/1588816630 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6354+0+0 (secure 0 0 0) 0x7f295c089aa0 con 0x7f296c0b4880 2023-11-09T03:48:50.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.299+0000 7f296affd640 1 --2- 172.21.15.76:0/1588816630 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f295806ae70 0x7f295806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:50.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.300+0000 7f296affd640 1 --2- 172.21.15.76:0/1588816630 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f295806ae70 0x7f295806d330 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f296c153400 tx=0x7f2960007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:50.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.303+0000 7f2953fff640 1 -- 172.21.15.76:0/1588816630 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f295c054a00 con 0x7f296c0b4880 2023-11-09T03:48:50.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.469+0000 7f297207c640 1 -- 172.21.15.76:0/1588816630 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7f296c074290 con 0x7f296c0b4880 2023-11-09T03:48:50.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.469+0000 7f2953fff640 1 -- 172.21.15.76:0/1588816630 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v87) v1 ==== 107+0+11 (secure 0 0 0) 0x7f295c01f0b0 con 0x7f296c0b4880 2023-11-09T03:48:50.470 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:48:50.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.471+0000 7f297207c640 1 -- 172.21.15.76:0/1588816630 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f295806ae70 msgr2=0x7f295806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:50.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.471+0000 7f297207c640 1 --2- 172.21.15.76:0/1588816630 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f295806ae70 0x7f295806d330 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f296c153400 tx=0x7f2960007d50 comp rx=0 tx=0).stop 2023-11-09T03:48:50.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.471+0000 7f297207c640 1 -- 172.21.15.76:0/1588816630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f296c0b4880 msgr2=0x7f296c152880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:50.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.471+0000 7f297207c640 1 --2- 172.21.15.76:0/1588816630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f296c0b4880 0x7f296c152880 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f295c00c7f0 tx=0x7f295c00ccb0 comp rx=0 tx=0).stop 2023-11-09T03:48:50.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.471+0000 7f297207c640 1 -- 172.21.15.76:0/1588816630 shutdown_connections 2023-11-09T03:48:50.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.471+0000 7f297207c640 1 --2- 172.21.15.76:0/1588816630 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f295806ae70 0x7f295806d330 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:50.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.471+0000 7f297207c640 1 --2- 172.21.15.76:0/1588816630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f296c153e60 0x7f296c15eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:50.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.471+0000 7f297207c640 1 --2- 172.21.15.76:0/1588816630 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f296c0cd6d0 0x7f296c15e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:50.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.471+0000 7f297207c640 1 --2- 172.21.15.76:0/1588816630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f296c0b4880 0x7f296c152880 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:50.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.471+0000 7f297207c640 1 -- 172.21.15.76:0/1588816630 >> 172.21.15.76:0/1588816630 conn(0x7f296c0cf880 msgr2=0x7f296c15f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:50.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.471+0000 7f297207c640 1 -- 172.21.15.76:0/1588816630 shutdown_connections 2023-11-09T03:48:50.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.471+0000 7f297207c640 1 -- 172.21.15.76:0/1588816630 wait complete. 2023-11-09T03:48:50.482 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_kill_mdstable' 2023-11-09T03:48:50.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.582+0000 7f447d31d640 1 Processor -- start 2023-11-09T03:48:50.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.582+0000 7f447d31d640 1 -- start start 2023-11-09T03:48:50.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.583+0000 7f447d31d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f447815e520 0x7f44780cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:50.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.583+0000 7f447d31d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f44780cfe30 0x7f44780cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:50.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.583+0000 7f447d31d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44780cd760 con 0x7f4478159c80 2023-11-09T03:48:50.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.583+0000 7f447d31d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44780d02e0 con 0x7f44780cfe30 2023-11-09T03:48:50.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.583+0000 7f447d31d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44780cb0f0 con 0x7f447815e520 2023-11-09T03:48:50.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.583+0000 7f44777fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f44780cfe30 0x7f44780cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:50.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.583+0000 7f44777fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f44780cfe30 0x7f44780cdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52270/0 (socket says 172.21.15.76:52270) 2023-11-09T03:48:50.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.583+0000 7f44777fe640 1 -- 172.21.15.76:0/3553263714 learned_addr learned my addr 172.21.15.76:0/3553263714 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:50.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.583+0000 7f44767fc640 1 --2- 172.21.15.76:0/3553263714 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f447815e520 0x7f44780cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:50.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.583+0000 7f44777fe640 1 -- 172.21.15.76:0/3553263714 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f447815e520 msgr2=0x7f44780cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:50.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.583+0000 7f44777fe640 1 --2- 172.21.15.76:0/3553263714 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f447815e520 0x7f44780cf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:50.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.583+0000 7f44777fe640 1 -- 172.21.15.76:0/3553263714 >> v1:172.21.15.113:6789/0 conn(0x7f4478159c80 legacy=0x7f44780ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:50.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.583+0000 7f44777fe640 1 -- 172.21.15.76:0/3553263714 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44780cb3d0 con 0x7f44780cfe30 2023-11-09T03:48:50.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.584+0000 7f44777fe640 1 --2- 172.21.15.76:0/3553263714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44780cfe30 0x7f44780cdbf0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f4470009e50 tx=0x7f447002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e29b86c31704c76 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:50.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.584+0000 7f4475ffb640 1 -- 172.21.15.76:0/3553263714 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447002cb00 con 0x7f44780cfe30 2023-11-09T03:48:50.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.584+0000 7f4475ffb640 1 -- 172.21.15.76:0/3553263714 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447002cc60 con 0x7f44780cfe30 2023-11-09T03:48:50.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.584+0000 7f4475ffb640 1 -- 172.21.15.76:0/3553263714 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44700343f0 con 0x7f44780cfe30 2023-11-09T03:48:50.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.585+0000 7f447d31d640 1 -- 172.21.15.76:0/3553263714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44780cfe30 msgr2=0x7f44780cdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:50.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.585+0000 7f447d31d640 1 --2- 172.21.15.76:0/3553263714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44780cfe30 0x7f44780cdbf0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f4470009e50 tx=0x7f447002bf60 comp rx=0 tx=0).stop 2023-11-09T03:48:50.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.585+0000 7f447d31d640 1 -- 172.21.15.76:0/3553263714 shutdown_connections 2023-11-09T03:48:50.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.585+0000 7f447d31d640 1 --2- 172.21.15.76:0/3553263714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44780cfe30 0x7f44780cdbf0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:50.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.585+0000 7f447d31d640 1 --2- 172.21.15.76:0/3553263714 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f447815e520 0x7f44780cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:50.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.585+0000 7f447d31d640 1 -- 172.21.15.76:0/3553263714 >> 172.21.15.76:0/3553263714 conn(0x7f4478098a60 msgr2=0x7f4478098e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:50.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.585+0000 7f447d31d640 1 -- 172.21.15.76:0/3553263714 shutdown_connections 2023-11-09T03:48:50.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.585+0000 7f447d31d640 1 -- 172.21.15.76:0/3553263714 wait complete. 2023-11-09T03:48:50.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.586+0000 7f447d31d640 1 Processor -- start 2023-11-09T03:48:50.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.586+0000 7f447d31d640 1 -- start start 2023-11-09T03:48:50.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.587+0000 7f447d31d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44780cfe30 0x7f447806a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:50.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.587+0000 7f447d31d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4478159c80 0x7f447806a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:50.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.587+0000 7f447d31d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f447815e520 0x7f44780c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:50.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.587+0000 7f447d31d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44780c9cf0 con 0x7f447815e520 2023-11-09T03:48:50.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.587+0000 7f447d31d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44780c9450 con 0x7f44780cfe30 2023-11-09T03:48:50.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.587+0000 7f447d31d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44780c9730 con 0x7f4478159c80 2023-11-09T03:48:50.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.587+0000 7f44777fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f447815e520 0x7f44780c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:50.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.587+0000 7f44777fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f447815e520 0x7f44780c86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52274/0 (socket says 172.21.15.76:52274) 2023-11-09T03:48:50.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.587+0000 7f44777fe640 1 -- 172.21.15.76:0/2107273605 learned_addr learned my addr 172.21.15.76:0/2107273605 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:50.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.587+0000 7f4476ffd640 1 --2- 172.21.15.76:0/2107273605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44780cfe30 0x7f447806a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:50.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.587+0000 7f44767fc640 1 --2- 172.21.15.76:0/2107273605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4478159c80 0x7f447806a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:50.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.587+0000 7f44777fe640 1 -- 172.21.15.76:0/2107273605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4478159c80 msgr2=0x7f447806a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:50.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.587+0000 7f44777fe640 1 --2- 172.21.15.76:0/2107273605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4478159c80 0x7f447806a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:50.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.587+0000 7f44777fe640 1 -- 172.21.15.76:0/2107273605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44780cfe30 msgr2=0x7f447806a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:50.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.587+0000 7f44777fe640 1 --2- 172.21.15.76:0/2107273605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44780cfe30 0x7f447806a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:50.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.587+0000 7f44777fe640 1 -- 172.21.15.76:0/2107273605 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4470009b30 con 0x7f447815e520 2023-11-09T03:48:50.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.587+0000 7f44777fe640 1 --2- 172.21.15.76:0/2107273605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f447815e520 0x7f44780c86e0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f4470009e50 tx=0x7f4470004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:50.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.588+0000 7f445f7fe640 1 -- 172.21.15.76:0/2107273605 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470009510 con 0x7f447815e520 2023-11-09T03:48:50.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.588+0000 7f445f7fe640 1 -- 172.21.15.76:0/2107273605 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447003e450 con 0x7f447815e520 2023-11-09T03:48:50.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.588+0000 7f447d31d640 1 -- 172.21.15.76:0/2107273605 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44780c9a10 con 0x7f447815e520 2023-11-09T03:48:50.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.588+0000 7f445f7fe640 1 -- 172.21.15.76:0/2107273605 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447003e800 con 0x7f447815e520 2023-11-09T03:48:50.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.588+0000 7f447d31d640 1 -- 172.21.15.76:0/2107273605 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4478158420 con 0x7f447815e520 2023-11-09T03:48:50.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.589+0000 7f447d31d640 1 -- 172.21.15.76:0/2107273605 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44781585e0 con 0x7f447815e520 2023-11-09T03:48:50.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.589+0000 7f445f7fe640 1 -- 172.21.15.76:0/2107273605 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f447003e960 con 0x7f447815e520 2023-11-09T03:48:50.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.589+0000 7f445f7fe640 1 --2- 172.21.15.76:0/2107273605 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f446406ae70 0x7f446406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:50.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.590+0000 7f445f7fe640 1 -- 172.21.15.76:0/2107273605 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6354+0+0 (secure 0 0 0) 0x7f44700ac3c0 con 0x7f447815e520 2023-11-09T03:48:50.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.590+0000 7f4476ffd640 1 --2- 172.21.15.76:0/2107273605 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f446406ae70 0x7f446406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:50.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.590+0000 7f4476ffd640 1 --2- 172.21.15.76:0/2107273605 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f446406ae70 0x7f446406d330 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4468007fd0 tx=0x7f4468009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:50.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.594+0000 7f445f7fe640 1 -- 172.21.15.76:0/2107273605 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4470077320 con 0x7f447815e520 2023-11-09T03:48:50.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:50.827+0000 7f447d31d640 1 -- 172.21.15.76:0/2107273605 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_kill_mdstable"]} v 0) v1 -- 0x7f44780c3940 con 0x7f447815e520 2023-11-09T03:48:51.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.005+0000 7f445f7fe640 1 -- 172.21.15.76:0/2107273605 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_kill_mdstable"]}]=0 v211) v1 ==== 142+0+0 (secure 0 0 0) 0x7f447007a940 con 0x7f447815e520 2023-11-09T03:48:51.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.007+0000 7f447d31d640 1 -- 172.21.15.76:0/2107273605 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f446406ae70 msgr2=0x7f446406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:51.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.007+0000 7f447d31d640 1 --2- 172.21.15.76:0/2107273605 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f446406ae70 0x7f446406d330 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4468007fd0 tx=0x7f4468009040 comp rx=0 tx=0).stop 2023-11-09T03:48:51.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.007+0000 7f447d31d640 1 -- 172.21.15.76:0/2107273605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f447815e520 msgr2=0x7f44780c86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:51.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.007+0000 7f447d31d640 1 --2- 172.21.15.76:0/2107273605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f447815e520 0x7f44780c86e0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f4470009e50 tx=0x7f4470004a40 comp rx=0 tx=0).stop 2023-11-09T03:48:51.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.007+0000 7f447d31d640 1 -- 172.21.15.76:0/2107273605 shutdown_connections 2023-11-09T03:48:51.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.007+0000 7f447d31d640 1 --2- 172.21.15.76:0/2107273605 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f446406ae70 0x7f446406d330 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:51.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.007+0000 7f447d31d640 1 --2- 172.21.15.76:0/2107273605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f447815e520 0x7f44780c86e0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:51.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.007+0000 7f447d31d640 1 --2- 172.21.15.76:0/2107273605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4478159c80 0x7f447806a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:51.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.007+0000 7f447d31d640 1 --2- 172.21.15.76:0/2107273605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44780cfe30 0x7f447806a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:51.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.007+0000 7f447d31d640 1 -- 172.21.15.76:0/2107273605 >> 172.21.15.76:0/2107273605 conn(0x7f4478098a60 msgr2=0x7f44780cc190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:51.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.007+0000 7f447d31d640 1 -- 172.21.15.76:0/2107273605 shutdown_connections 2023-11-09T03:48:51.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.007+0000 7f447d31d640 1 -- 172.21.15.76:0/2107273605 wait complete. 2023-11-09T03:48:51.019 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-09T03:48:51.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.120+0000 7fb043821640 1 Processor -- start 2023-11-09T03:48:51.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.121+0000 7fb043821640 1 -- start start 2023-11-09T03:48:51.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.121+0000 7fb043821640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb03c070e20 0x7fb03c0adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:51.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.121+0000 7fb043821640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb03c0ad730 0x7fb03c15e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:51.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.121+0000 7fb043821640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb03c0c3900 con 0x7fb03c15ad30 2023-11-09T03:48:51.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.121+0000 7fb043821640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb03c0ae260 con 0x7fb03c0ad730 2023-11-09T03:48:51.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.121+0000 7fb043821640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c0c3be0 con 0x7fb03c070e20 2023-11-09T03:48:51.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.122+0000 7fb041dbe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb03c0ad730 0x7fb03c15e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:51.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.122+0000 7fb041dbe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb03c0ad730 0x7fb03c15e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52286/0 (socket says 172.21.15.76:52286) 2023-11-09T03:48:51.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.122+0000 7fb041dbe640 1 -- 172.21.15.76:0/1034924076 learned_addr learned my addr 172.21.15.76:0/1034924076 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:51.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.122+0000 7fb041dbe640 1 -- 172.21.15.76:0/1034924076 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb03c070e20 msgr2=0x7fb03c0adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:51.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.122+0000 7fb041dbe640 1 --2- 172.21.15.76:0/1034924076 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb03c070e20 0x7fb03c0adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:51.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.122+0000 7fb041dbe640 1 -- 172.21.15.76:0/1034924076 >> v1:172.21.15.113:6789/0 conn(0x7fb03c15ad30 legacy=0x7fb03c070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:51.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.122+0000 7fb041dbe640 1 -- 172.21.15.76:0/1034924076 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c0c3ec0 con 0x7fb03c0ad730 2023-11-09T03:48:51.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.123+0000 7fb041dbe640 1 --2- 172.21.15.76:0/1034924076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb03c0ad730 0x7fb03c15e740 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fb034009e50 tx=0x7fb03402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20aeded16e92003c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:51.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.123+0000 7fb033fff640 1 -- 172.21.15.76:0/1034924076 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03402c9e0 con 0x7fb03c0ad730 2023-11-09T03:48:51.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.123+0000 7fb033fff640 1 -- 172.21.15.76:0/1034924076 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03402cb40 con 0x7fb03c0ad730 2023-11-09T03:48:51.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.123+0000 7fb033fff640 1 -- 172.21.15.76:0/1034924076 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0340343f0 con 0x7fb03c0ad730 2023-11-09T03:48:51.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.123+0000 7fb043821640 1 -- 172.21.15.76:0/1034924076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb03c0ad730 msgr2=0x7fb03c15e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:51.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.123+0000 7fb043821640 1 --2- 172.21.15.76:0/1034924076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb03c0ad730 0x7fb03c15e740 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fb034009e50 tx=0x7fb03402be40 comp rx=0 tx=0).stop 2023-11-09T03:48:51.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.123+0000 7fb043821640 1 -- 172.21.15.76:0/1034924076 shutdown_connections 2023-11-09T03:48:51.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.123+0000 7fb043821640 1 --2- 172.21.15.76:0/1034924076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb03c0ad730 0x7fb03c15e740 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:51.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.123+0000 7fb043821640 1 --2- 172.21.15.76:0/1034924076 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb03c070e20 0x7fb03c0adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:51.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.123+0000 7fb043821640 1 -- 172.21.15.76:0/1034924076 >> 172.21.15.76:0/1034924076 conn(0x7fb03c0ce950 msgr2=0x7fb03c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:51.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.123+0000 7fb043821640 1 -- 172.21.15.76:0/1034924076 shutdown_connections 2023-11-09T03:48:51.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.123+0000 7fb043821640 1 -- 172.21.15.76:0/1034924076 wait complete. 2023-11-09T03:48:51.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.125+0000 7fb043821640 1 Processor -- start 2023-11-09T03:48:51.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.125+0000 7fb043821640 1 -- start start 2023-11-09T03:48:51.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.125+0000 7fb043821640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb03c070e20 0x7fb03c1528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:51.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.125+0000 7fb043821640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb03c0ad730 0x7fb03c152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:51.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.125+0000 7fb043821640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb03c15ad30 0x7fb03c153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:51.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.125+0000 7fb043821640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c15dbb0 con 0x7fb03c15ad30 2023-11-09T03:48:51.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.125+0000 7fb043821640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c15de90 con 0x7fb03c070e20 2023-11-09T03:48:51.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.125+0000 7fb043821640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c0c8440 con 0x7fb03c0ad730 2023-11-09T03:48:51.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.125+0000 7fb041dbe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb03c15ad30 0x7fb03c153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:51.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.125+0000 7fb041dbe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb03c15ad30 0x7fb03c153370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52296/0 (socket says 172.21.15.76:52296) 2023-11-09T03:48:51.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.125+0000 7fb041dbe640 1 -- 172.21.15.76:0/2823731725 learned_addr learned my addr 172.21.15.76:0/2823731725 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:51.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.126+0000 7fb041dbe640 1 -- 172.21.15.76:0/2823731725 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb03c0ad730 msgr2=0x7fb03c152e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:51.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.126+0000 7fb0415bd640 1 --2- 172.21.15.76:0/2823731725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb03c070e20 0x7fb03c1528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:51.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.126+0000 7fb040dbc640 1 --2- 172.21.15.76:0/2823731725 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb03c0ad730 0x7fb03c152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:51.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.126+0000 7fb041dbe640 1 --2- 172.21.15.76:0/2823731725 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb03c0ad730 0x7fb03c152e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:51.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.126+0000 7fb041dbe640 1 -- 172.21.15.76:0/2823731725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb03c070e20 msgr2=0x7fb03c1528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:51.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.126+0000 7fb041dbe640 1 --2- 172.21.15.76:0/2823731725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb03c070e20 0x7fb03c1528f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:51.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.126+0000 7fb041dbe640 1 -- 172.21.15.76:0/2823731725 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb034009b30 con 0x7fb03c15ad30 2023-11-09T03:48:51.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.126+0000 7fb041dbe640 1 --2- 172.21.15.76:0/2823731725 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb03c15ad30 0x7fb03c153370 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fb034036040 tx=0x7fb034004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:51.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.126+0000 7fb031ffb640 1 -- 172.21.15.76:0/2823731725 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03403e450 con 0x7fb03c15ad30 2023-11-09T03:48:51.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.126+0000 7fb031ffb640 1 -- 172.21.15.76:0/2823731725 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03403e5d0 con 0x7fb03c15ad30 2023-11-09T03:48:51.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.126+0000 7fb043821640 1 -- 172.21.15.76:0/2823731725 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb03c0c8720 con 0x7fb03c15ad30 2023-11-09T03:48:51.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.126+0000 7fb031ffb640 1 -- 172.21.15.76:0/2823731725 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03403e980 con 0x7fb03c15ad30 2023-11-09T03:48:51.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.126+0000 7fb043821640 1 -- 172.21.15.76:0/2823731725 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03c15d860 con 0x7fb03c15ad30 2023-11-09T03:48:51.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.127+0000 7fb043821640 1 -- 172.21.15.76:0/2823731725 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb03c15e050 con 0x7fb03c15ad30 2023-11-09T03:48:51.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.128+0000 7fb031ffb640 1 -- 172.21.15.76:0/2823731725 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb03403eae0 con 0x7fb03c15ad30 2023-11-09T03:48:51.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.128+0000 7fb031ffb640 1 --2- 172.21.15.76:0/2823731725 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb02006ae70 0x7fb02006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:51.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.128+0000 7fb031ffb640 1 -- 172.21.15.76:0/2823731725 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6354+0+0 (secure 0 0 0) 0x7fb0340ac730 con 0x7fb03c15ad30 2023-11-09T03:48:51.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.128+0000 7fb0415bd640 1 --2- 172.21.15.76:0/2823731725 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb02006ae70 0x7fb02006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:51.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.129+0000 7fb0415bd640 1 --2- 172.21.15.76:0/2823731725 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb02006ae70 0x7fb02006d330 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00a750 tx=0x7fb02c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:51.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.132+0000 7fb031ffb640 1 -- 172.21.15.76:0/2823731725 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb034077670 con 0x7fb03c15ad30 2023-11-09T03:48:51.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.297+0000 7fb043821640 1 -- 172.21.15.76:0/2823731725 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb03c074290 con 0x7fb03c15ad30 2023-11-09T03:48:51.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.300+0000 7fb031ffb640 1 -- 172.21.15.76:0/2823731725 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v19)=0 v19) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb03407ac90 con 0x7fb03c15ad30 2023-11-09T03:48:51.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.302+0000 7fb043821640 1 -- 172.21.15.76:0/2823731725 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb02006ae70 msgr2=0x7fb02006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:51.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.302+0000 7fb043821640 1 --2- 172.21.15.76:0/2823731725 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb02006ae70 0x7fb02006d330 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00a750 tx=0x7fb02c008040 comp rx=0 tx=0).stop 2023-11-09T03:48:51.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.302+0000 7fb043821640 1 -- 172.21.15.76:0/2823731725 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb03c15ad30 msgr2=0x7fb03c153370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:51.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.302+0000 7fb043821640 1 --2- 172.21.15.76:0/2823731725 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb03c15ad30 0x7fb03c153370 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fb034036040 tx=0x7fb034004a40 comp rx=0 tx=0).stop 2023-11-09T03:48:51.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.302+0000 7fb043821640 1 -- 172.21.15.76:0/2823731725 shutdown_connections 2023-11-09T03:48:51.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.302+0000 7fb043821640 1 --2- 172.21.15.76:0/2823731725 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb02006ae70 0x7fb02006d330 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:51.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.302+0000 7fb043821640 1 --2- 172.21.15.76:0/2823731725 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb03c15ad30 0x7fb03c153370 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:51.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.302+0000 7fb043821640 1 --2- 172.21.15.76:0/2823731725 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb03c0ad730 0x7fb03c152e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:51.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.302+0000 7fb043821640 1 --2- 172.21.15.76:0/2823731725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb03c070e20 0x7fb03c1528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:51.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.302+0000 7fb043821640 1 -- 172.21.15.76:0/2823731725 >> 172.21.15.76:0/2823731725 conn(0x7fb03c0ce950 msgr2=0x7fb03c06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:51.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.302+0000 7fb043821640 1 -- 172.21.15.76:0/2823731725 shutdown_connections 2023-11-09T03:48:51.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.302+0000 7fb043821640 1 -- 172.21.15.76:0/2823731725 wait complete. 2023-11-09T03:48:51.313 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-09T03:48:51.314 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-09T03:48:51.363 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-09T03:48:51.428 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-09T03:48:51.428 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:48:51.428 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:48:51.443 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056381,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14761 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248868k,nr_inodes=812217,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-09T03:48:51.444 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:48:51.444 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:48:51.462 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2440 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-09T03:48:51.462 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:48:51.463 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:48:51.498 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056381,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14761 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248868k,nr_inodes=812217,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-09T03:48:51.498 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T03:48:51.499 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:48:51.519 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2440 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-09T03:48:51.519 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-09T03:48:51.520 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:48:51.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.637+0000 7f6bcdad4640 1 Processor -- start 2023-11-09T03:48:51.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.637+0000 7f6bcdad4640 1 -- start start 2023-11-09T03:48:51.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.638+0000 7f6bcdad4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6bc8159c70 0x7f6bc80ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:51.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.638+0000 7f6bcdad4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6bc80b0ae0 0x7f6bc815d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:51.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.638+0000 7f6bcdad4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bc80b4110 con 0x7f6bc806db20 2023-11-09T03:48:51.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.638+0000 7f6bcdad4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bc80b49b0 con 0x7f6bc80b0ae0 2023-11-09T03:48:51.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.638+0000 7f6bcdad4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bc80b43f0 con 0x7f6bc8159c70 2023-11-09T03:48:51.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.638+0000 7f6bc7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6bc80b0ae0 0x7f6bc815d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:51.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.638+0000 7f6bc7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6bc80b0ae0 0x7f6bc815d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52298/0 (socket says 172.21.15.76:52298) 2023-11-09T03:48:51.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.638+0000 7f6bc7fff640 1 -- 172.21.15.76:0/433791641 learned_addr learned my addr 172.21.15.76:0/433791641 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:51.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.638+0000 7f6bc7fff640 1 -- 172.21.15.76:0/433791641 >> v2:172.21.15.113:3300/0 conn(0x7f6bc8159c70 msgr2=0x7f6bc80ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:51.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.638+0000 7f6bc7fff640 1 --2- 172.21.15.76:0/433791641 >> v2:172.21.15.113:3300/0 conn(0x7f6bc8159c70 0x7f6bc80ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:51.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.638+0000 7f6bc7fff640 1 -- 172.21.15.76:0/433791641 >> v1:172.21.15.113:6789/0 conn(0x7f6bc806db20 legacy=0x7f6bc80b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:51.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.638+0000 7f6bc7fff640 1 -- 172.21.15.76:0/433791641 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc80b46d0 con 0x7f6bc80b0ae0 2023-11-09T03:48:51.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.639+0000 7f6bc7fff640 1 --2- 172.21.15.76:0/433791641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bc80b0ae0 0x7f6bc815d7d0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f6bc80cc440 tx=0x7f6bc002e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617ca35d75a17c1f server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:51.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.639+0000 7f6bc67fc640 1 -- 172.21.15.76:0/433791641 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc002ea90 con 0x7f6bc80b0ae0 2023-11-09T03:48:51.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.639+0000 7f6bc67fc640 1 -- 172.21.15.76:0/433791641 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc002ebf0 con 0x7f6bc80b0ae0 2023-11-09T03:48:51.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.639+0000 7f6bc67fc640 1 -- 172.21.15.76:0/433791641 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0009600 con 0x7f6bc80b0ae0 2023-11-09T03:48:51.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.640+0000 7f6bcdad4640 1 -- 172.21.15.76:0/433791641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bc80b0ae0 msgr2=0x7f6bc815d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:51.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.640+0000 7f6bcdad4640 1 --2- 172.21.15.76:0/433791641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bc80b0ae0 0x7f6bc815d7d0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f6bc80cc440 tx=0x7f6bc002e0f0 comp rx=0 tx=0).stop 2023-11-09T03:48:51.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.640+0000 7f6bcdad4640 1 -- 172.21.15.76:0/433791641 shutdown_connections 2023-11-09T03:48:51.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.640+0000 7f6bcdad4640 1 --2- 172.21.15.76:0/433791641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bc80b0ae0 0x7f6bc815d7d0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:51.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.640+0000 7f6bcdad4640 1 --2- 172.21.15.76:0/433791641 >> v2:172.21.15.113:3300/0 conn(0x7f6bc8159c70 0x7f6bc80ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:51.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.640+0000 7f6bcdad4640 1 -- 172.21.15.76:0/433791641 >> 172.21.15.76:0/433791641 conn(0x7f6bc80ce950 msgr2=0x7f6bc80ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:51.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.640+0000 7f6bcdad4640 1 -- 172.21.15.76:0/433791641 shutdown_connections 2023-11-09T03:48:51.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.640+0000 7f6bcdad4640 1 -- 172.21.15.76:0/433791641 wait complete. 2023-11-09T03:48:51.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.641+0000 7f6bcdad4640 1 Processor -- start 2023-11-09T03:48:51.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.641+0000 7f6bcdad4640 1 -- start start 2023-11-09T03:48:51.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.642+0000 7f6bcdad4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bc806db20 0x7f6bc814dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:51.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.642+0000 7f6bcdad4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bc80b0ae0 0x7f6bc814e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:51.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.642+0000 7f6bcdad4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bc8159c70 0x7f6bc814eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:51.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.642+0000 7f6bcdad4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc80c5e20 con 0x7f6bc806db20 2023-11-09T03:48:51.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.642+0000 7f6bcdad4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc80c6100 con 0x7f6bc8159c70 2023-11-09T03:48:51.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.642+0000 7f6bcdad4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc80c63e0 con 0x7f6bc80b0ae0 2023-11-09T03:48:51.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.642+0000 7f6bc77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bc806db20 0x7f6bc814dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:51.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.642+0000 7f6bc77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bc806db20 0x7f6bc814dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52308/0 (socket says 172.21.15.76:52308) 2023-11-09T03:48:51.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.642+0000 7f6bc77fe640 1 -- 172.21.15.76:0/851479886 learned_addr learned my addr 172.21.15.76:0/851479886 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:51.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.642+0000 7f6bc77fe640 1 -- 172.21.15.76:0/851479886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bc80b0ae0 msgr2=0x7f6bc814e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:51.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.642+0000 7f6bc77fe640 1 --2- 172.21.15.76:0/851479886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bc80b0ae0 0x7f6bc814e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:51.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.642+0000 7f6bc77fe640 1 -- 172.21.15.76:0/851479886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bc8159c70 msgr2=0x7f6bc814eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:51.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.642+0000 7f6bc77fe640 1 --2- 172.21.15.76:0/851479886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bc8159c70 0x7f6bc814eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:51.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.642+0000 7f6bc77fe640 1 -- 172.21.15.76:0/851479886 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc000bd40 con 0x7f6bc806db20 2023-11-09T03:48:51.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.642+0000 7f6bc77fe640 1 --2- 172.21.15.76:0/851479886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bc806db20 0x7f6bc814dd50 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f6bb800b4a0 tx=0x7f6bb800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:51.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.642+0000 7f6baffff640 1 -- 172.21.15.76:0/851479886 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8010610 con 0x7f6bc806db20 2023-11-09T03:48:51.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.643+0000 7f6baffff640 1 -- 172.21.15.76:0/851479886 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb8010770 con 0x7f6bc806db20 2023-11-09T03:48:51.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.643+0000 7f6baffff640 1 -- 172.21.15.76:0/851479886 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8010b20 con 0x7f6bc806db20 2023-11-09T03:48:51.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.643+0000 7f6bcdad4640 1 -- 172.21.15.76:0/851479886 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bc80c66c0 con 0x7f6bc806db20 2023-11-09T03:48:51.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.643+0000 7f6bcdad4640 1 -- 172.21.15.76:0/851479886 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc814e290 con 0x7f6bc806db20 2023-11-09T03:48:51.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.643+0000 7f6bcdad4640 1 -- 172.21.15.76:0/851479886 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bc80c6be0 con 0x7f6bc806db20 2023-11-09T03:48:51.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.644+0000 7f6baffff640 1 -- 172.21.15.76:0/851479886 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6bb8010c80 con 0x7f6bc806db20 2023-11-09T03:48:51.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.644+0000 7f6baffff640 1 --2- 172.21.15.76:0/851479886 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6bb406ae70 0x7f6bb406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:51.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.644+0000 7f6baffff640 1 -- 172.21.15.76:0/851479886 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6354+0+0 (secure 0 0 0) 0x7f6bb80894a0 con 0x7f6bc806db20 2023-11-09T03:48:51.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.644+0000 7f6bc6ffd640 1 --2- 172.21.15.76:0/851479886 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6bb406ae70 0x7f6bb406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:51.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.645+0000 7f6bc6ffd640 1 --2- 172.21.15.76:0/851479886 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6bb406ae70 0x7f6bb406d330 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6bc80ad740 tx=0x7f6bbc006ba0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:51.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.648+0000 7f6baffff640 1 -- 172.21.15.76:0/851479886 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6bc80c6be0 con 0x7f6bc806db20 2023-11-09T03:48:51.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.851+0000 7f6bcdad4640 1 -- 172.21.15.76:0/851479886 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6bc80c6be0 con 0x7f6bc806db20 2023-11-09T03:48:51.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.851+0000 7f6baffff640 1 -- 172.21.15.76:0/851479886 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f6bb801ee90 con 0x7f6bc806db20 2023-11-09T03:48:51.853 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:51.853 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":39,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6097,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2408659865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2408659865},{"type":"v1","addr":"172.21.15.113:6836","nonce":2408659865}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6100,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1873078383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1873078383},{"type":"v1","addr":"172.21.15.113:6837","nonce":1873078383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6834/3549714130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3549714130},{"type":"v1","addr":"172.21.15.76:6834","nonce":3549714130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":7356,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3109327841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6833","nonce":3109327841},{"type":"v1","addr":"172.21.15.76:6835","nonce":3109327841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[]} 2023-11-09T03:48:51.855 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 39 2023-11-09T03:48:51.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.853+0000 7f6bcdad4640 1 -- 172.21.15.76:0/851479886 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6bb406ae70 msgr2=0x7f6bb406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:51.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.853+0000 7f6bcdad4640 1 --2- 172.21.15.76:0/851479886 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6bb406ae70 0x7f6bb406d330 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6bc80ad740 tx=0x7f6bbc006ba0 comp rx=0 tx=0).stop 2023-11-09T03:48:51.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.853+0000 7f6bcdad4640 1 -- 172.21.15.76:0/851479886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bc806db20 msgr2=0x7f6bc814dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:51.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.853+0000 7f6bcdad4640 1 --2- 172.21.15.76:0/851479886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bc806db20 0x7f6bc814dd50 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f6bb800b4a0 tx=0x7f6bb800b960 comp rx=0 tx=0).stop 2023-11-09T03:48:51.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.853+0000 7f6bcdad4640 1 -- 172.21.15.76:0/851479886 shutdown_connections 2023-11-09T03:48:51.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.853+0000 7f6bcdad4640 1 --2- 172.21.15.76:0/851479886 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6bb406ae70 0x7f6bb406d330 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:51.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.853+0000 7f6bcdad4640 1 --2- 172.21.15.76:0/851479886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bc8159c70 0x7f6bc814eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:51.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.853+0000 7f6bcdad4640 1 --2- 172.21.15.76:0/851479886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bc80b0ae0 0x7f6bc814e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:51.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.853+0000 7f6bcdad4640 1 --2- 172.21.15.76:0/851479886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bc806db20 0x7f6bc814dd50 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:51.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.853+0000 7f6bcdad4640 1 -- 172.21.15.76:0/851479886 >> 172.21.15.76:0/851479886 conn(0x7f6bc80ce950 msgr2=0x7f6bc8070630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:51.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.854+0000 7f6bcdad4640 1 -- 172.21.15.76:0/851479886 shutdown_connections 2023-11-09T03:48:51.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:51.854+0000 7f6bcdad4640 1 -- 172.21.15.76:0/851479886 wait complete. 2023-11-09T03:48:51.865 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-09T03:48:51.866 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-09T03:48:51.866 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-09T03:48:51.867 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:48:51.867 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-09T03:48:51.868 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-09T03:48:51.868 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-09T03:48:51.869 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:48:51.869 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-09T03:48:51.869 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-09T03:48:51.870 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-09T03:48:51.870 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:48:51.871 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-09T03:48:51.871 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-09T03:48:51.872 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-09T03:48:51.872 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:48:51.876 INFO:tasks.ceph.mds.a:Stopped 2023-11-09T03:48:51.876 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-09T03:48:51.878 INFO:tasks.ceph.mds.c:Stopped 2023-11-09T03:48:51.879 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-09T03:48:51.883 INFO:tasks.ceph.mds.b:Stopped 2023-11-09T03:48:51.883 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-09T03:48:51.884 INFO:tasks.ceph.mds.d:Stopped 2023-11-09T03:48:51.885 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-09T03:48:51.920 INFO:tasks.ceph.mds.a:Started 2023-11-09T03:48:51.920 DEBUG:teuthology.parallel:result is None 2023-11-09T03:48:51.921 INFO:tasks.ceph.mds.c:Started 2023-11-09T03:48:51.922 DEBUG:teuthology.parallel:result is None 2023-11-09T03:48:51.927 INFO:tasks.ceph.mds.b:Started 2023-11-09T03:48:51.927 DEBUG:teuthology.parallel:result is None 2023-11-09T03:48:51.928 INFO:tasks.ceph.mds.d:Started 2023-11-09T03:48:51.929 DEBUG:teuthology.parallel:result is None 2023-11-09T03:48:51.929 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-09T03:48:51.995 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2023-11-09T03:48:51.996 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2023-11-09T03:48:52.011 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2023-11-09T03:48:52.011 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2023-11-09T03:48:52.012 INFO:tasks.ceph.mds.b.smithi113.stderr:2023-11-09T03:48:52.012+0000 7f402272e640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-11-09T03:48:52.012 INFO:tasks.ceph.mds.d.smithi113.stderr:2023-11-09T03:48:52.012+0000 7f1d51dab640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-11-09T03:48:52.012 INFO:tasks.ceph.mds.b.smithi113.stderr:2023-11-09T03:48:52.012+0000 7f402272e640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6835/0: (98) Address already in use 2023-11-09T03:48:52.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.034+0000 7fdfdf933640 1 Processor -- start 2023-11-09T03:48:52.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.034+0000 7fdfdf933640 1 -- start start 2023-11-09T03:48:52.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.035+0000 7fdfdf933640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfd8153e60 0x7fdfd815dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:52.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.035+0000 7fdfdf933640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdfd80cd6d0 0x7fdfd8161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:52.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.035+0000 7fdfdf933640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfd80cde60 con 0x7fdfd80b4880 2023-11-09T03:48:52.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.035+0000 7fdfdf933640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfd8160f90 con 0x7fdfd80cd6d0 2023-11-09T03:48:52.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.035+0000 7fdfdf933640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd815e400 con 0x7fdfd8153e60 2023-11-09T03:48:52.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.035+0000 7fdfdcece640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdfd80cd6d0 0x7fdfd8161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:52.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.035+0000 7fdfdcece640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdfd80cd6d0 0x7fdfd8161420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52350/0 (socket says 172.21.15.76:52350) 2023-11-09T03:48:52.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.035+0000 7fdfdcece640 1 -- 172.21.15.76:0/1789411013 learned_addr learned my addr 172.21.15.76:0/1789411013 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:52.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.036+0000 7fdfdd6cf640 1 --2- 172.21.15.76:0/1789411013 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfd8153e60 0x7fdfd815dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:52.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.036+0000 7fdfdcece640 1 -- 172.21.15.76:0/1789411013 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfd8153e60 msgr2=0x7fdfd815dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:52.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.036+0000 7fdfdcece640 1 --2- 172.21.15.76:0/1789411013 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfd8153e60 0x7fdfd815dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:52.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.036+0000 7fdfdcece640 1 -- 172.21.15.76:0/1789411013 >> v1:172.21.15.113:6789/0 conn(0x7fdfd80b4880 legacy=0x7fdfd806b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:52.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.036+0000 7fdfdcece640 1 -- 172.21.15.76:0/1789411013 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd8161960 con 0x7fdfd80cd6d0 2023-11-09T03:48:52.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.036+0000 7fdfdd6cf640 1 --2- 172.21.15.76:0/1789411013 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfd8153e60 0x7fdfd815dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:52.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.036+0000 7fdfdcece640 1 --2- 172.21.15.76:0/1789411013 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfd80cd6d0 0x7fdfd8161420 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8009870 tx=0x7fdfc802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46eba58b15fdcf0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:52.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.037+0000 7fdfcffff640 1 -- 172.21.15.76:0/1789411013 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8031640 con 0x7fdfd80cd6d0 2023-11-09T03:48:52.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.037+0000 7fdfcffff640 1 -- 172.21.15.76:0/1789411013 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc80317a0 con 0x7fdfd80cd6d0 2023-11-09T03:48:52.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.037+0000 7fdfcffff640 1 -- 172.21.15.76:0/1789411013 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8031b30 con 0x7fdfd80cd6d0 2023-11-09T03:48:52.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.037+0000 7fdfdf933640 1 -- 172.21.15.76:0/1789411013 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfd80cd6d0 msgr2=0x7fdfd8161420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:52.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.037+0000 7fdfdf933640 1 --2- 172.21.15.76:0/1789411013 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfd80cd6d0 0x7fdfd8161420 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8009870 tx=0x7fdfc802b8f0 comp rx=0 tx=0).stop 2023-11-09T03:48:52.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.037+0000 7fdfdf933640 1 -- 172.21.15.76:0/1789411013 shutdown_connections 2023-11-09T03:48:52.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.037+0000 7fdfdf933640 1 --2- 172.21.15.76:0/1789411013 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfd80cd6d0 0x7fdfd8161420 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:52.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.037+0000 7fdfdf933640 1 --2- 172.21.15.76:0/1789411013 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfd8153e60 0x7fdfd815dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:52.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.037+0000 7fdfdf933640 1 -- 172.21.15.76:0/1789411013 >> 172.21.15.76:0/1789411013 conn(0x7fdfd80cf880 msgr2=0x7fdfd80cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:52.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.037+0000 7fdfdf933640 1 -- 172.21.15.76:0/1789411013 shutdown_connections 2023-11-09T03:48:52.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.037+0000 7fdfdf933640 1 -- 172.21.15.76:0/1789411013 wait complete. 2023-11-09T03:48:52.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.039+0000 7fdfdf933640 1 Processor -- start 2023-11-09T03:48:52.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.039+0000 7fdfdf933640 1 -- start start 2023-11-09T03:48:52.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.039+0000 7fdfdf933640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfd80b4880 0x7fdfd8152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:52.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.039+0000 7fdfdf933640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfd80cd6d0 0x7fdfd8154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:52.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.040+0000 7fdfdf933640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfd8153e60 0x7fdfd8154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:52.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.040+0000 7fdfdf933640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd80c6990 con 0x7fdfd8153e60 2023-11-09T03:48:52.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.040+0000 7fdfdf933640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd80c6c70 con 0x7fdfd80b4880 2023-11-09T03:48:52.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.040+0000 7fdfdf933640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd80c6f50 con 0x7fdfd80cd6d0 2023-11-09T03:48:52.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.040+0000 7fdfdded0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfd8153e60 0x7fdfd8154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:52.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.040+0000 7fdfdcece640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfd80cd6d0 0x7fdfd8154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:52.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.040+0000 7fdfdded0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfd8153e60 0x7fdfd8154780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52366/0 (socket says 172.21.15.76:52366) 2023-11-09T03:48:52.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.040+0000 7fdfdded0640 1 -- 172.21.15.76:0/843143986 learned_addr learned my addr 172.21.15.76:0/843143986 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:52.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.040+0000 7fdfdd6cf640 1 --2- 172.21.15.76:0/843143986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfd80b4880 0x7fdfd8152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:52.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.040+0000 7fdfdded0640 1 -- 172.21.15.76:0/843143986 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfd80cd6d0 msgr2=0x7fdfd8154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:52.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.040+0000 7fdfdded0640 1 --2- 172.21.15.76:0/843143986 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfd80cd6d0 0x7fdfd8154240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:52.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.040+0000 7fdfdded0640 1 -- 172.21.15.76:0/843143986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfd80b4880 msgr2=0x7fdfd8152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:52.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.040+0000 7fdfdded0640 1 --2- 172.21.15.76:0/843143986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfd80b4880 0x7fdfd8152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:52.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.040+0000 7fdfdded0640 1 -- 172.21.15.76:0/843143986 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc80094d0 con 0x7fdfd8153e60 2023-11-09T03:48:52.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.040+0000 7fdfdded0640 1 --2- 172.21.15.76:0/843143986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfd8153e60 0x7fdfd8154780 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fdfd000c780 tx=0x7fdfd000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:52.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.040+0000 7fdfcdffb640 1 -- 172.21.15.76:0/843143986 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0004b60 con 0x7fdfd8153e60 2023-11-09T03:48:52.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.040+0000 7fdfcdffb640 1 -- 172.21.15.76:0/843143986 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfd000ce90 con 0x7fdfd8153e60 2023-11-09T03:48:52.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.040+0000 7fdfcdffb640 1 -- 172.21.15.76:0/843143986 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd000a850 con 0x7fdfd8153e60 2023-11-09T03:48:52.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.040+0000 7fdfdf933640 1 -- 172.21.15.76:0/843143986 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd80c7230 con 0x7fdfd8153e60 2023-11-09T03:48:52.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.041+0000 7fdfdf933640 1 -- 172.21.15.76:0/843143986 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd8153120 con 0x7fdfd8153e60 2023-11-09T03:48:52.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.042+0000 7fdfdf933640 1 -- 172.21.15.76:0/843143986 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfd80c7750 con 0x7fdfd8153e60 2023-11-09T03:48:52.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.042+0000 7fdfcdffb640 1 -- 172.21.15.76:0/843143986 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdfd000a9b0 con 0x7fdfd8153e60 2023-11-09T03:48:52.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.042+0000 7fdfcdffb640 1 --2- 172.21.15.76:0/843143986 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdfbc06ae70 0x7fdfbc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:52.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.043+0000 7fdfcdffb640 1 -- 172.21.15.76:0/843143986 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6354+0+0 (secure 0 0 0) 0x7fdfd00898b0 con 0x7fdfd8153e60 2023-11-09T03:48:52.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.043+0000 7fdfdd6cf640 1 --2- 172.21.15.76:0/843143986 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdfbc06ae70 0x7fdfbc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:52.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.043+0000 7fdfdd6cf640 1 --2- 172.21.15.76:0/843143986 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdfbc06ae70 0x7fdfbc06d330 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdfd80b8b30 tx=0x7fdfd4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:52.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.047+0000 7fdfcdffb640 1 -- 172.21.15.76:0/843143986 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdfd0054810 con 0x7fdfd8153e60 2023-11-09T03:48:52.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:52.225+0000 7fdfdf933640 1 -- 172.21.15.76:0/843143986 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fdfd81532b0 con 0x7fdfd8153e60 2023-11-09T03:48:53.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.017+0000 7fdfcdffb640 1 -- 172.21.15.76:0/843143986 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v89) v1 ==== 97+0+0 (secure 0 0 0) 0x7fdfd0057e30 con 0x7fdfd8153e60 2023-11-09T03:48:53.018 INFO:teuthology.orchestra.run.smithi076.stderr: removed all blocklist entries 2023-11-09T03:48:53.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.021+0000 7fdfdf933640 1 -- 172.21.15.76:0/843143986 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdfbc06ae70 msgr2=0x7fdfbc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:53.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.021+0000 7fdfdf933640 1 --2- 172.21.15.76:0/843143986 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdfbc06ae70 0x7fdfbc06d330 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdfd80b8b30 tx=0x7fdfd4008040 comp rx=0 tx=0).stop 2023-11-09T03:48:53.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.021+0000 7fdfdf933640 1 -- 172.21.15.76:0/843143986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfd8153e60 msgr2=0x7fdfd8154780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:53.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.021+0000 7fdfdf933640 1 --2- 172.21.15.76:0/843143986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfd8153e60 0x7fdfd8154780 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fdfd000c780 tx=0x7fdfd000cc40 comp rx=0 tx=0).stop 2023-11-09T03:48:53.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.021+0000 7fdfdf933640 1 -- 172.21.15.76:0/843143986 shutdown_connections 2023-11-09T03:48:53.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.021+0000 7fdfdf933640 1 --2- 172.21.15.76:0/843143986 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdfbc06ae70 0x7fdfbc06d330 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.021+0000 7fdfdf933640 1 --2- 172.21.15.76:0/843143986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfd8153e60 0x7fdfd8154780 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.021+0000 7fdfdf933640 1 --2- 172.21.15.76:0/843143986 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfd80cd6d0 0x7fdfd8154240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.021+0000 7fdfdf933640 1 --2- 172.21.15.76:0/843143986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfd80b4880 0x7fdfd8152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.021+0000 7fdfdf933640 1 -- 172.21.15.76:0/843143986 >> 172.21.15.76:0/843143986 conn(0x7fdfd80cf880 msgr2=0x7fdfd815fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:53.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.021+0000 7fdfdf933640 1 -- 172.21.15.76:0/843143986 shutdown_connections 2023-11-09T03:48:53.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.021+0000 7fdfdf933640 1 -- 172.21.15.76:0/843143986 wait complete. 2023-11-09T03:48:53.037 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-09T03:48:53.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.143+0000 7ffa17fd0640 1 Processor -- start 2023-11-09T03:48:53.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.144+0000 7ffa17fd0640 1 -- start start 2023-11-09T03:48:53.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.144+0000 7ffa17fd0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa100cd910 0x7ffa100d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:53.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.144+0000 7ffa17fd0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa1006c4a0 con 0x7ffa10161980 2023-11-09T03:48:53.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.144+0000 7ffa17fd0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa1006bc00 con 0x7ffa1006b6a0 2023-11-09T03:48:53.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.144+0000 7ffa17fd0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa100c6810 con 0x7ffa100cd910 2023-11-09T03:48:53.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.144+0000 7ffa1656d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ffa10161980 0x7ffa100b48a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35850/0 (socket says 172.21.15.76:35850) 2023-11-09T03:48:53.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.144+0000 7ffa1656d640 1 -- 172.21.15.76:0/3019944447 learned_addr learned my addr 172.21.15.76:0/3019944447 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:53.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.145+0000 7ffa15d6c640 1 --2- 172.21.15.76:0/3019944447 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa100cd910 0x7ffa100d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:53.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.145+0000 7ffa14d6a640 1 -- 172.21.15.76:0/3019944447 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4234619256 0 0) 0x7ffa1006c4a0 con 0x7ffa10161980 2023-11-09T03:48:53.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.145+0000 7ffa14d6a640 1 -- 172.21.15.76:0/3019944447 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9fc003520 con 0x7ffa10161980 2023-11-09T03:48:53.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.145+0000 7ffa14d6a640 1 -- 172.21.15.76:0/3019944447 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 193472437 0 0) 0x7ff9fc003520 con 0x7ffa10161980 2023-11-09T03:48:53.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.145+0000 7ffa14d6a640 1 -- 172.21.15.76:0/3019944447 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa100cd910 msgr2=0x7ffa100d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:53.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.145+0000 7ffa14d6a640 1 --2- 172.21.15.76:0/3019944447 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa100cd910 0x7ffa100d2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.145+0000 7ffa14d6a640 1 -- 172.21.15.76:0/3019944447 >> v1:172.21.15.113:6789/0 conn(0x7ffa1006b6a0 legacy=0x7ffa101655c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:53.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.145+0000 7ffa14d6a640 1 -- 172.21.15.76:0/3019944447 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1006bee0 con 0x7ffa10161980 2023-11-09T03:48:53.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.145+0000 7ffa14d6a640 1 -- 172.21.15.76:0/3019944447 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7ffa08001670 con 0x7ffa10161980 2023-11-09T03:48:53.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.145+0000 7ffa14d6a640 1 -- 172.21.15.76:0/3019944447 >> v1:172.21.15.76:6789/0 conn(0x7ffa10161980 legacy=0x7ffa100b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:53.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.145+0000 7ffa14d6a640 1 --2- 172.21.15.76:0/3019944447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9fc0039a0 0x7ff9fc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:53.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.145+0000 7ffa14d6a640 1 --2- 172.21.15.76:0/3019944447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9fc0047c0 0x7ff9fc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:53.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.145+0000 7ffa14d6a640 1 --2- 172.21.15.76:0/3019944447 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa100cd910 0x7ff9fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:53.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.145+0000 7ffa14d6a640 1 -- 172.21.15.76:0/3019944447 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa100c6810 con 0x7ff9fc0039a0 2023-11-09T03:48:53.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.145+0000 7ffa14d6a640 1 -- 172.21.15.76:0/3019944447 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc00bab0 con 0x7ff9fc0047c0 2023-11-09T03:48:53.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.145+0000 7ffa14d6a640 1 -- 172.21.15.76:0/3019944447 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc00bc70 con 0x7ffa100cd910 2023-11-09T03:48:53.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.145+0000 7ffa15d6c640 1 --2- 172.21.15.76:0/3019944447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9fc0039a0 0x7ff9fc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:53.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.145+0000 7ffa1656d640 1 --2- 172.21.15.76:0/3019944447 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa100cd910 0x7ff9fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:53.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.145+0000 7ffa1556b640 1 --2- 172.21.15.76:0/3019944447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9fc0047c0 0x7ff9fc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:53.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.145+0000 7ffa15d6c640 1 -- 172.21.15.76:0/3019944447 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa100cd910 msgr2=0x7ff9fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:53.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.145+0000 7ffa15d6c640 1 --2- 172.21.15.76:0/3019944447 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa100cd910 0x7ff9fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.145+0000 7ffa15d6c640 1 -- 172.21.15.76:0/3019944447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9fc0047c0 msgr2=0x7ff9fc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:53.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.145+0000 7ffa15d6c640 1 --2- 172.21.15.76:0/3019944447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9fc0047c0 0x7ff9fc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.145+0000 7ffa15d6c640 1 -- 172.21.15.76:0/3019944447 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9fc003520 con 0x7ff9fc0039a0 2023-11-09T03:48:53.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.145+0000 7ffa1656d640 1 --2- 172.21.15.76:0/3019944447 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa100cd910 0x7ff9fc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:53.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.146+0000 7ffa15d6c640 1 --2- 172.21.15.76:0/3019944447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9fc0039a0 0x7ff9fc003dc0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009fc0 tx=0x7ff9fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86048c79115ea7b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:53.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.146+0000 7ffa14d6a640 1 -- 172.21.15.76:0/3019944447 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c02cc20 con 0x7ff9fc0039a0 2023-11-09T03:48:53.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.146+0000 7ffa14d6a640 1 -- 172.21.15.76:0/3019944447 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa0c00f040 con 0x7ff9fc0039a0 2023-11-09T03:48:53.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.146+0000 7ffa14d6a640 1 -- 172.21.15.76:0/3019944447 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c034440 con 0x7ff9fc0039a0 2023-11-09T03:48:53.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.147+0000 7ffa17fd0640 1 -- 172.21.15.76:0/3019944447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9fc0039a0 msgr2=0x7ff9fc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:53.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.147+0000 7ffa17fd0640 1 --2- 172.21.15.76:0/3019944447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9fc0039a0 0x7ff9fc003dc0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009fc0 tx=0x7ff9fc0057f0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.147+0000 7ffa1656d640 1 -- 172.21.15.76:0/3019944447 reap_dead start 2023-11-09T03:48:53.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.147+0000 7ffa17fd0640 1 -- 172.21.15.76:0/3019944447 shutdown_connections 2023-11-09T03:48:53.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.147+0000 7ffa17fd0640 1 -- 172.21.15.76:0/3019944447 >> 172.21.15.76:0/3019944447 conn(0x7ffa10071f30 msgr2=0x7ffa100cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:53.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.147+0000 7ffa17fd0640 1 -- 172.21.15.76:0/3019944447 shutdown_connections 2023-11-09T03:48:53.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.147+0000 7ffa17fd0640 1 -- 172.21.15.76:0/3019944447 wait complete. 2023-11-09T03:48:53.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.148+0000 7ffa17fd0640 1 Processor -- start 2023-11-09T03:48:53.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.148+0000 7ffa17fd0640 1 -- start start 2023-11-09T03:48:53.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.149+0000 7ffa17fd0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9fc0039a0 0x7ffa100c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:53.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.149+0000 7ffa17fd0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa1014f520 0x7ffa100c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:53.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.149+0000 7ffa17fd0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa1014f9a0 0x7ffa100c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:53.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.149+0000 7ffa17fd0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa100d09d0 con 0x7ff9fc0039a0 2023-11-09T03:48:53.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.149+0000 7ffa17fd0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa100c84d0 con 0x7ffa1014f520 2023-11-09T03:48:53.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.149+0000 7ffa17fd0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1014ff70 con 0x7ffa1014f9a0 2023-11-09T03:48:53.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.149+0000 7ffa15d6c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9fc0039a0 0x7ffa100c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:53.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.149+0000 7ffa15d6c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9fc0039a0 0x7ffa100c7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43936/0 (socket says 172.21.15.76:43936) 2023-11-09T03:48:53.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.149+0000 7ffa15d6c640 1 -- 172.21.15.76:0/1220005604 learned_addr learned my addr 172.21.15.76:0/1220005604 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:53.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.149+0000 7ffa1656d640 1 --2- 172.21.15.76:0/1220005604 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa1014f9a0 0x7ffa100c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:53.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.149+0000 7ffa1556b640 1 --2- 172.21.15.76:0/1220005604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa1014f520 0x7ffa100c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:53.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.149+0000 7ffa15d6c640 1 -- 172.21.15.76:0/1220005604 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa1014f9a0 msgr2=0x7ffa100c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:53.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.149+0000 7ffa15d6c640 1 --2- 172.21.15.76:0/1220005604 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa1014f9a0 0x7ffa100c90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.149+0000 7ffa15d6c640 1 -- 172.21.15.76:0/1220005604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa1014f520 msgr2=0x7ffa100c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:53.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.149+0000 7ffa15d6c640 1 --2- 172.21.15.76:0/1220005604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa1014f520 0x7ffa100c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.149+0000 7ffa15d6c640 1 -- 172.21.15.76:0/1220005604 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa10150250 con 0x7ff9fc0039a0 2023-11-09T03:48:53.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.149+0000 7ffa1556b640 1 --2- 172.21.15.76:0/1220005604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa1014f520 0x7ffa100c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:53.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.149+0000 7ffa15d6c640 1 --2- 172.21.15.76:0/1220005604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9fc0039a0 0x7ffa100c7f90 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c02c690 tx=0x7ffa0c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:53.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.149+0000 7ffa067fc640 1 -- 172.21.15.76:0/1220005604 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c034bd0 con 0x7ff9fc0039a0 2023-11-09T03:48:53.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.150+0000 7ffa067fc640 1 -- 172.21.15.76:0/1220005604 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa0c034d30 con 0x7ff9fc0039a0 2023-11-09T03:48:53.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.150+0000 7ffa17fd0640 1 -- 172.21.15.76:0/1220005604 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa10150530 con 0x7ff9fc0039a0 2023-11-09T03:48:53.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.150+0000 7ffa067fc640 1 -- 172.21.15.76:0/1220005604 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c034ea0 con 0x7ff9fc0039a0 2023-11-09T03:48:53.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.150+0000 7ffa17fd0640 1 -- 172.21.15.76:0/1220005604 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa100c8830 con 0x7ff9fc0039a0 2023-11-09T03:48:53.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.151+0000 7ffa17fd0640 1 -- 172.21.15.76:0/1220005604 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa10074290 con 0x7ff9fc0039a0 2023-11-09T03:48:53.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.151+0000 7ffa067fc640 1 -- 172.21.15.76:0/1220005604 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffa0c040450 con 0x7ff9fc0039a0 2023-11-09T03:48:53.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.151+0000 7ffa067fc640 1 --2- 172.21.15.76:0/1220005604 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff9f406aea0 0x7ff9f406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:53.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.151+0000 7ffa067fc640 1 -- 172.21.15.76:0/1220005604 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6268+0+0 (secure 0 0 0) 0x7ffa0c0ac0d0 con 0x7ff9fc0039a0 2023-11-09T03:48:53.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.152+0000 7ffa1556b640 1 --2- 172.21.15.76:0/1220005604 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff9f406aea0 0x7ff9f406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:53.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.152+0000 7ffa1556b640 1 --2- 172.21.15.76:0/1220005604 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff9f406aea0 0x7ff9f406d360 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ffa101646a0 tx=0x7ffa00009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:53.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.155+0000 7ffa067fc640 1 -- 172.21.15.76:0/1220005604 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffa0c077070 con 0x7ff9fc0039a0 2023-11-09T03:48:53.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.381+0000 7ffa17fd0640 1 -- 172.21.15.76:0/1220005604 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ffa10154e90 con 0x7ff9fc0039a0 2023-11-09T03:48:53.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.382+0000 7ffa067fc640 1 -- 172.21.15.76:0/1220005604 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ffa0c07a690 con 0x7ff9fc0039a0 2023-11-09T03:48:53.382 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:53.383 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:48:53.383 INFO:teuthology.orchestra.run.smithi076.stdout: "auth_dump": [ 2023-11-09T03:48:53.383 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:53.383 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.a", 2023-11-09T03:48:53.384 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCgVUxlydtlLxAAi0bnTEOD+fpGL1cYic3caA==", 2023-11-09T03:48:53.384 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:53.384 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:48:53.384 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2023-11-09T03:48:53.384 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2023-11-09T03:48:53.385 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:53.385 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:53.385 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:53.385 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:53.385 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.b", 2023-11-09T03:48:53.386 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxl2+QnABAA7CTWWEbPPXkollRFDlLxPA==", 2023-11-09T03:48:53.386 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:53.386 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:48:53.386 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2023-11-09T03:48:53.387 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2023-11-09T03:48:53.387 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:53.387 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:53.387 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:53.387 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:53.388 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.c", 2023-11-09T03:48:53.388 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCgVUxlrTu9NhAAhg4GpZWZTmVPCP3ffjPo/Q==", 2023-11-09T03:48:53.388 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:53.388 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:48:53.388 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2023-11-09T03:48:53.389 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2023-11-09T03:48:53.389 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:53.390 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:53.390 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:53.391 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:53.391 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.d", 2023-11-09T03:48:53.391 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxl4IZ8BxAA8zGFE+1tlHkT5UyzL0nWxQ==", 2023-11-09T03:48:53.392 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:53.392 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:48:53.392 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2023-11-09T03:48:53.393 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2023-11-09T03:48:53.393 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:53.393 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:53.393 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:53.393 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:53.394 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.0", 2023-11-09T03:48:53.394 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCkVUxlXM5LKhAAM253+XEj5YSwmQywwDieTg==", 2023-11-09T03:48:53.394 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:53.394 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:48:53.394 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:48:53.395 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:53.395 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:53.395 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:53.395 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:53.396 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.1", 2023-11-09T03:48:53.396 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCnVUxlgoDVKBAAkmNFo4E0bAQ1UxkCfQAbjw==", 2023-11-09T03:48:53.396 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:53.396 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:48:53.396 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:48:53.396 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:53.397 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:53.397 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:53.397 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:53.397 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.2", 2023-11-09T03:48:53.397 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCqVUxlzQjVJhAAOoU5IdN86BmcPz71mTq7ng==", 2023-11-09T03:48:53.398 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:53.398 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:48:53.398 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:48:53.398 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:53.398 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:53.399 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:53.399 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:53.399 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.3", 2023-11-09T03:48:53.399 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCtVUxlhY6VJBAAmCO/ZS8vBa7enyuXX5Y+hw==", 2023-11-09T03:48:53.399 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:53.400 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:48:53.400 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:48:53.400 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:53.400 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:53.400 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:53.401 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:53.401 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.4", 2023-11-09T03:48:53.401 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCzVUxl79egLxAAnKBOQb/9EQyaHaQt7SawIg==", 2023-11-09T03:48:53.401 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:53.401 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:48:53.402 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:48:53.402 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:53.402 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:53.402 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:53.402 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:53.403 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.5", 2023-11-09T03:48:53.403 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQC2VUxlIfUgLhAAVPurhJCc8cmpt11pQjzL9w==", 2023-11-09T03:48:53.403 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:53.403 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:48:53.403 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:48:53.403 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:53.404 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:53.404 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:53.404 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:53.404 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.6", 2023-11-09T03:48:53.404 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQC5VUxlrI58LBAAWxZotpACR4e8vFj4IiaEKA==", 2023-11-09T03:48:53.405 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:53.405 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:48:53.405 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:48:53.405 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:53.406 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:53.406 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:53.406 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:53.406 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.7", 2023-11-09T03:48:53.406 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQC8VUxl0bA3KxAAwnKNJKqYIcGRLLhQHUaU9w==", 2023-11-09T03:48:53.407 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:53.407 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2023-11-09T03:48:53.407 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2023-11-09T03:48:53.407 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:53.407 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:53.408 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:53.408 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:53.408 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.0", 2023-11-09T03:48:53.408 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxlHj22CxAACNj5X24hdME+GNTLw4RahQ==", 2023-11-09T03:48:53.408 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:53.408 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:48:53.409 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow r", 2023-11-09T03:48:53.409 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-09T03:48:53.409 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:53.409 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:53.409 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:53.410 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.1", 2023-11-09T03:48:53.410 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxljXv4FBAAiTOhfIReQWTy6fbujbBrrw==", 2023-11-09T03:48:53.410 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:53.410 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:48:53.410 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2023-11-09T03:48:53.411 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2023-11-09T03:48:53.411 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2023-11-09T03:48:53.411 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:53.411 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:53.411 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:53.411 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.2", 2023-11-09T03:48:53.412 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxlGVSSERAAQts5odubZ5paZwg8US3QrQ==", 2023-11-09T03:48:53.412 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:53.412 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:48:53.412 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2023-11-09T03:48:53.412 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2023-11-09T03:48:53.413 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2023-11-09T03:48:53.413 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:53.413 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:53.413 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:53.413 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.3", 2023-11-09T03:48:53.414 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQChVUxlgmDHGhAA7r98HNQEMnjbqW6RjE5aig==", 2023-11-09T03:48:53.414 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:53.414 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2023-11-09T03:48:53.414 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2023-11-09T03:48:53.414 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2023-11-09T03:48:53.415 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2023-11-09T03:48:53.415 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:53.415 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:53.415 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:53.415 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.admin", 2023-11-09T03:48:53.415 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCgVUxlV4l8EhAAnrgeLaTaR99vNYGCJjI1TA==", 2023-11-09T03:48:53.416 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:53.416 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2023-11-09T03:48:53.416 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2023-11-09T03:48:53.416 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2023-11-09T03:48:53.416 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:53.417 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:53.417 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:53.417 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:53.417 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mgr.x", 2023-11-09T03:48:53.417 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCgVUxl7JryKxAABFx9CRqSbbJu8SIyi3Va8w==", 2023-11-09T03:48:53.418 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:53.418 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2023-11-09T03:48:53.418 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile mgr", 2023-11-09T03:48:53.418 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:53.418 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:53.418 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:53.419 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:53.419 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mgr.y", 2023-11-09T03:48:53.419 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCgVUxlb9L/JxAAu3IOXTn8TGvbzxYVta3yew==", 2023-11-09T03:48:53.419 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2023-11-09T03:48:53.419 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2023-11-09T03:48:53.420 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile mgr", 2023-11-09T03:48:53.420 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2023-11-09T03:48:53.420 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:53.420 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:53.420 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2023-11-09T03:48:53.420 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:48:53.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.384+0000 7ffa17fd0640 1 -- 172.21.15.76:0/1220005604 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff9f406aea0 msgr2=0x7ff9f406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:53.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.384+0000 7ffa17fd0640 1 --2- 172.21.15.76:0/1220005604 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff9f406aea0 0x7ff9f406d360 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ffa101646a0 tx=0x7ffa00009040 comp rx=0 tx=0).stop 2023-11-09T03:48:53.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.384+0000 7ffa17fd0640 1 -- 172.21.15.76:0/1220005604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9fc0039a0 msgr2=0x7ffa100c7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:53.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.384+0000 7ffa17fd0640 1 --2- 172.21.15.76:0/1220005604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9fc0039a0 0x7ffa100c7f90 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c02c690 tx=0x7ffa0c009750 comp rx=0 tx=0).stop 2023-11-09T03:48:53.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.384+0000 7ffa17fd0640 1 -- 172.21.15.76:0/1220005604 shutdown_connections 2023-11-09T03:48:53.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.384+0000 7ffa17fd0640 1 --2- 172.21.15.76:0/1220005604 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff9f406aea0 0x7ff9f406d360 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.384+0000 7ffa17fd0640 1 --2- 172.21.15.76:0/1220005604 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa1014f9a0 0x7ffa100c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.384+0000 7ffa17fd0640 1 --2- 172.21.15.76:0/1220005604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa1014f520 0x7ffa100c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.384+0000 7ffa17fd0640 1 --2- 172.21.15.76:0/1220005604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9fc0039a0 0x7ffa100c7f90 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.384+0000 7ffa17fd0640 1 -- 172.21.15.76:0/1220005604 >> 172.21.15.76:0/1220005604 conn(0x7ffa10071f30 msgr2=0x7ffa100b4900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:53.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.384+0000 7ffa17fd0640 1 -- 172.21.15.76:0/1220005604 shutdown_connections 2023-11-09T03:48:53.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.384+0000 7ffa17fd0640 1 -- 172.21.15.76:0/1220005604 wait complete. 2023-11-09T03:48:53.425 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:48:53.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.526+0000 7f2061bf5640 1 Processor -- start 2023-11-09T03:48:53.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.526+0000 7f2061bf5640 1 -- start start 2023-11-09T03:48:53.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.527+0000 7f2061bf5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c0b0ae0 0x7f205c0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:53.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.527+0000 7f2061bf5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f205c06db20 0x7f205c15d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:53.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.527+0000 7f2061bf5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f205c0b4110 con 0x7f205c159c70 2023-11-09T03:48:53.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.527+0000 7f2061bf5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f205c0b49b0 con 0x7f205c06db20 2023-11-09T03:48:53.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.527+0000 7f2061bf5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205c0b43f0 con 0x7f205c0b0ae0 2023-11-09T03:48:53.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.527+0000 7f205affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c0b0ae0 0x7f205c0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:53.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.527+0000 7f205affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c0b0ae0 0x7f205c0ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:56160/0 (socket says 172.21.15.76:56160) 2023-11-09T03:48:53.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.527+0000 7f205affd640 1 -- 172.21.15.76:0/1311448288 learned_addr learned my addr 172.21.15.76:0/1311448288 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:53.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.527+0000 7f205bfff640 1 --2- 172.21.15.76:0/1311448288 >> v2:172.21.15.113:3300/0 conn(0x7f205c06db20 0x7f205c15d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:53.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.527+0000 7f205affd640 1 -- 172.21.15.76:0/1311448288 >> v2:172.21.15.113:3300/0 conn(0x7f205c06db20 msgr2=0x7f205c15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:53.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.527+0000 7f205affd640 1 --2- 172.21.15.76:0/1311448288 >> v2:172.21.15.113:3300/0 conn(0x7f205c06db20 0x7f205c15d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.527+0000 7f205affd640 1 -- 172.21.15.76:0/1311448288 >> v1:172.21.15.113:6789/0 conn(0x7f205c159c70 legacy=0x7f205c0b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:53.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.527+0000 7f205affd640 1 -- 172.21.15.76:0/1311448288 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205c0b46d0 con 0x7f205c0b0ae0 2023-11-09T03:48:53.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.528+0000 7f205affd640 1 --2- 172.21.15.76:0/1311448288 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c0b0ae0 0x7f205c0ce370 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f204c00bb10 tx=0x7f204c02e150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2207f33067be8214 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:53.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.528+0000 7f205a7fc640 1 -- 172.21.15.76:0/1311448288 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c036440 con 0x7f205c0b0ae0 2023-11-09T03:48:53.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.528+0000 7f205a7fc640 1 -- 172.21.15.76:0/1311448288 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204c0365a0 con 0x7f205c0b0ae0 2023-11-09T03:48:53.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.528+0000 7f205a7fc640 1 -- 172.21.15.76:0/1311448288 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c036930 con 0x7f205c0b0ae0 2023-11-09T03:48:53.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.529+0000 7f2061bf5640 1 -- 172.21.15.76:0/1311448288 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c0b0ae0 msgr2=0x7f205c0ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:53.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.529+0000 7f2061bf5640 1 --2- 172.21.15.76:0/1311448288 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c0b0ae0 0x7f205c0ce370 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f204c00bb10 tx=0x7f204c02e150 comp rx=0 tx=0).stop 2023-11-09T03:48:53.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.529+0000 7f2061bf5640 1 -- 172.21.15.76:0/1311448288 shutdown_connections 2023-11-09T03:48:53.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.529+0000 7f2061bf5640 1 --2- 172.21.15.76:0/1311448288 >> v2:172.21.15.113:3300/0 conn(0x7f205c06db20 0x7f205c15d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.529+0000 7f2061bf5640 1 --2- 172.21.15.76:0/1311448288 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c0b0ae0 0x7f205c0ce370 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.529+0000 7f2061bf5640 1 -- 172.21.15.76:0/1311448288 >> 172.21.15.76:0/1311448288 conn(0x7f205c0ce950 msgr2=0x7f205c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:53.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.529+0000 7f2061bf5640 1 -- 172.21.15.76:0/1311448288 shutdown_connections 2023-11-09T03:48:53.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.529+0000 7f2061bf5640 1 -- 172.21.15.76:0/1311448288 wait complete. 2023-11-09T03:48:53.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.530+0000 7f2061bf5640 1 Processor -- start 2023-11-09T03:48:53.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.530+0000 7f2061bf5640 1 -- start start 2023-11-09T03:48:53.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.531+0000 7f2061bf5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f205c06db20 0x7f205c15c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:53.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.531+0000 7f2061bf5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f205c0b0ae0 0x7f205c152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:53.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.531+0000 7f2061bf5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c159c70 0x7f205c152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:53.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.531+0000 7f2061bf5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c0b0410 con 0x7f205c0b0ae0 2023-11-09T03:48:53.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.531+0000 7f2061bf5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c153510 con 0x7f205c06db20 2023-11-09T03:48:53.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.531+0000 7f2061bf5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205c0afee0 con 0x7f205c159c70 2023-11-09T03:48:53.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.531+0000 7f205affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f205c0b0ae0 0x7f205c152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:53.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.531+0000 7f205affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f205c0b0ae0 0x7f205c152a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43948/0 (socket says 172.21.15.76:43948) 2023-11-09T03:48:53.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.531+0000 7f205affd640 1 -- 172.21.15.76:0/1454508325 learned_addr learned my addr 172.21.15.76:0/1454508325 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:53.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.531+0000 7f205bfff640 1 --2- 172.21.15.76:0/1454508325 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c159c70 0x7f205c152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:53.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.531+0000 7f205b7fe640 1 --2- 172.21.15.76:0/1454508325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f205c06db20 0x7f205c15c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:53.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.531+0000 7f205affd640 1 -- 172.21.15.76:0/1454508325 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c159c70 msgr2=0x7f205c152fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:53.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.531+0000 7f205affd640 1 --2- 172.21.15.76:0/1454508325 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c159c70 0x7f205c152fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.531+0000 7f205affd640 1 -- 172.21.15.76:0/1454508325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f205c06db20 msgr2=0x7f205c15c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:53.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.531+0000 7f205affd640 1 --2- 172.21.15.76:0/1454508325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f205c06db20 0x7f205c15c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.531+0000 7f205affd640 1 -- 172.21.15.76:0/1454508325 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204c00b770 con 0x7f205c0b0ae0 2023-11-09T03:48:53.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.532+0000 7f205affd640 1 --2- 172.21.15.76:0/1454508325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f205c0b0ae0 0x7f205c152a90 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f204c00b2c0 tx=0x7f204c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:53.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.532+0000 7f2053fff640 1 -- 172.21.15.76:0/1454508325 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c036440 con 0x7f205c0b0ae0 2023-11-09T03:48:53.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.532+0000 7f2053fff640 1 -- 172.21.15.76:0/1454508325 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204c034bc0 con 0x7f205c0b0ae0 2023-11-09T03:48:53.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.532+0000 7f2053fff640 1 -- 172.21.15.76:0/1454508325 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c036e20 con 0x7f205c0b0ae0 2023-11-09T03:48:53.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.532+0000 7f2061bf5640 1 -- 172.21.15.76:0/1454508325 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f205c0c6bd0 con 0x7f205c0b0ae0 2023-11-09T03:48:53.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.532+0000 7f2061bf5640 1 -- 172.21.15.76:0/1454508325 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f205c15acb0 con 0x7f205c0b0ae0 2023-11-09T03:48:53.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.533+0000 7f2061bf5640 1 -- 172.21.15.76:0/1454508325 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f205c0c6d60 con 0x7f205c0b0ae0 2023-11-09T03:48:53.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.533+0000 7f2053fff640 1 -- 172.21.15.76:0/1454508325 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f204c03d040 con 0x7f205c0b0ae0 2023-11-09T03:48:53.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.534+0000 7f2053fff640 1 --2- 172.21.15.76:0/1454508325 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f203c06ae70 0x7f203c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:53.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.534+0000 7f2053fff640 1 -- 172.21.15.76:0/1454508325 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f204c0ae2b0 con 0x7f205c0b0ae0 2023-11-09T03:48:53.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.534+0000 7f205b7fe640 1 --2- 172.21.15.76:0/1454508325 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f203c06ae70 0x7f203c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:53.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.534+0000 7f205b7fe640 1 --2- 172.21.15.76:0/1454508325 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f203c06ae70 0x7f203c06d330 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2044000f60 tx=0x7f2044009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:53.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.538+0000 7f2053fff640 1 -- 172.21.15.76:0/1454508325 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f204c079240 con 0x7f205c0b0ae0 2023-11-09T03:48:53.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.707+0000 7f2061bf5640 1 -- 172.21.15.76:0/1454508325 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f205c074290 con 0x7f205c0b0ae0 2023-11-09T03:48:53.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.708+0000 7f2053fff640 1 -- 172.21.15.76:0/1454508325 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+18335 (secure 0 0 0) 0x7f204c07c860 con 0x7f205c0b0ae0 2023-11-09T03:48:53.709 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:53.709 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":91,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:48:53.029300+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-09T03:47:59.280601+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-09T03:47:59.742039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:48:53.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.710+0000 7f2061bf5640 1 -- 172.21.15.76:0/1454508325 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f203c06ae70 msgr2=0x7f203c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:53.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.710+0000 7f2061bf5640 1 --2- 172.21.15.76:0/1454508325 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f203c06ae70 0x7f203c06d330 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2044000f60 tx=0x7f2044009040 comp rx=0 tx=0).stop 2023-11-09T03:48:53.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.710+0000 7f2061bf5640 1 -- 172.21.15.76:0/1454508325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f205c0b0ae0 msgr2=0x7f205c152a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:53.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.710+0000 7f2061bf5640 1 --2- 172.21.15.76:0/1454508325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f205c0b0ae0 0x7f205c152a90 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f204c00b2c0 tx=0x7f204c0046c0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.710+0000 7f2061bf5640 1 -- 172.21.15.76:0/1454508325 shutdown_connections 2023-11-09T03:48:53.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.710+0000 7f2061bf5640 1 --2- 172.21.15.76:0/1454508325 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f203c06ae70 0x7f203c06d330 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.710+0000 7f2061bf5640 1 --2- 172.21.15.76:0/1454508325 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f205c159c70 0x7f205c152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.710+0000 7f2061bf5640 1 --2- 172.21.15.76:0/1454508325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f205c0b0ae0 0x7f205c152a90 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.710+0000 7f2061bf5640 1 --2- 172.21.15.76:0/1454508325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f205c06db20 0x7f205c15c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.710+0000 7f2061bf5640 1 -- 172.21.15.76:0/1454508325 >> 172.21.15.76:0/1454508325 conn(0x7f205c0ce950 msgr2=0x7f205c0b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:53.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.710+0000 7f2061bf5640 1 -- 172.21.15.76:0/1454508325 shutdown_connections 2023-11-09T03:48:53.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.710+0000 7f2061bf5640 1 -- 172.21.15.76:0/1454508325 wait complete. 2023-11-09T03:48:53.721 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-09T03:47:00.036540+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_data_pool', 'create_time': '2023-11-09T03:47:59.280601+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 70, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-09T03:47:59.742039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-09T03:48:53.722 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:48:53.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.825+0000 7ff312955640 1 Processor -- start 2023-11-09T03:48:53.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.825+0000 7ff312955640 1 -- start start 2023-11-09T03:48:53.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.826+0000 7ff312955640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff30c0c4ff0 0x7ff30c0cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:53.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.826+0000 7ff312955640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff30c0adac0 0x7ff30c0adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:53.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.826+0000 7ff312955640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff30c15aff0 con 0x7ff30c159c00 2023-11-09T03:48:53.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.826+0000 7ff312955640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff30c0ae450 con 0x7ff30c0adac0 2023-11-09T03:48:53.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.826+0000 7ff312955640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c15a3e0 con 0x7ff30c0c4ff0 2023-11-09T03:48:53.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.826+0000 7ff30b7fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff30c0c4ff0 0x7ff30c0cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:53.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.826+0000 7ff30b7fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff30c0c4ff0 0x7ff30c0cc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:56178/0 (socket says 172.21.15.76:56178) 2023-11-09T03:48:53.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.826+0000 7ff30b7fe640 1 -- 172.21.15.76:0/4121772161 learned_addr learned my addr 172.21.15.76:0/4121772161 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:53.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.826+0000 7ff310ef2640 1 --2- 172.21.15.76:0/4121772161 >> v2:172.21.15.113:3300/0 conn(0x7ff30c0adac0 0x7ff30c0adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:53.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.826+0000 7ff30b7fe640 1 -- 172.21.15.76:0/4121772161 >> v2:172.21.15.113:3300/0 conn(0x7ff30c0adac0 msgr2=0x7ff30c0adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:53.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.826+0000 7ff30b7fe640 1 --2- 172.21.15.76:0/4121772161 >> v2:172.21.15.113:3300/0 conn(0x7ff30c0adac0 0x7ff30c0adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.826+0000 7ff30b7fe640 1 -- 172.21.15.76:0/4121772161 >> v1:172.21.15.113:6789/0 conn(0x7ff30c159c00 legacy=0x7ff30c0c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:53.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.826+0000 7ff30b7fe640 1 -- 172.21.15.76:0/4121772161 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30c15a6c0 con 0x7ff30c0c4ff0 2023-11-09T03:48:53.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.827+0000 7ff30b7fe640 1 --2- 172.21.15.76:0/4121772161 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff30c0c4ff0 0x7ff30c0cc890 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff300009870 tx=0x7ff30002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c245a7f3cd3d310 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:53.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.827+0000 7ff30affd640 1 -- 172.21.15.76:0/4121772161 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff300030840 con 0x7ff30c0c4ff0 2023-11-09T03:48:53.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.827+0000 7ff30affd640 1 -- 172.21.15.76:0/4121772161 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3000309a0 con 0x7ff30c0c4ff0 2023-11-09T03:48:53.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.827+0000 7ff30affd640 1 -- 172.21.15.76:0/4121772161 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff300030d30 con 0x7ff30c0c4ff0 2023-11-09T03:48:53.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.828+0000 7ff312955640 1 -- 172.21.15.76:0/4121772161 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff30c0c4ff0 msgr2=0x7ff30c0cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:53.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.828+0000 7ff312955640 1 --2- 172.21.15.76:0/4121772161 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff30c0c4ff0 0x7ff30c0cc890 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff300009870 tx=0x7ff30002b980 comp rx=0 tx=0).stop 2023-11-09T03:48:53.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.828+0000 7ff312955640 1 -- 172.21.15.76:0/4121772161 shutdown_connections 2023-11-09T03:48:53.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.828+0000 7ff312955640 1 --2- 172.21.15.76:0/4121772161 >> v2:172.21.15.113:3300/0 conn(0x7ff30c0adac0 0x7ff30c0adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.828+0000 7ff312955640 1 --2- 172.21.15.76:0/4121772161 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff30c0c4ff0 0x7ff30c0cc890 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.828+0000 7ff312955640 1 -- 172.21.15.76:0/4121772161 >> 172.21.15.76:0/4121772161 conn(0x7ff30c0ce950 msgr2=0x7ff30c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:53.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.828+0000 7ff312955640 1 -- 172.21.15.76:0/4121772161 shutdown_connections 2023-11-09T03:48:53.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.828+0000 7ff312955640 1 -- 172.21.15.76:0/4121772161 wait complete. 2023-11-09T03:48:53.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.829+0000 7ff312955640 1 Processor -- start 2023-11-09T03:48:53.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.829+0000 7ff312955640 1 -- start start 2023-11-09T03:48:53.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.830+0000 7ff312955640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff30c0adac0 0x7ff30c15bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:53.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.830+0000 7ff312955640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff30c0c4ff0 0x7ff30c140060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:53.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.830+0000 7ff312955640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff30c159c00 0x7ff30c14ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:53.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.830+0000 7ff312955640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c1405a0 con 0x7ff30c0adac0 2023-11-09T03:48:53.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.830+0000 7ff312955640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c0c7240 con 0x7ff30c0c4ff0 2023-11-09T03:48:53.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.830+0000 7ff312955640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c0c7520 con 0x7ff30c159c00 2023-11-09T03:48:53.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.830+0000 7ff30bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff30c0adac0 0x7ff30c15bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:53.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.830+0000 7ff30bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff30c0adac0 0x7ff30c15bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43962/0 (socket says 172.21.15.76:43962) 2023-11-09T03:48:53.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.830+0000 7ff30bfff640 1 -- 172.21.15.76:0/4153407286 learned_addr learned my addr 172.21.15.76:0/4153407286 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:53.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.830+0000 7ff30b7fe640 1 --2- 172.21.15.76:0/4153407286 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff30c0c4ff0 0x7ff30c140060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:53.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.830+0000 7ff310ef2640 1 --2- 172.21.15.76:0/4153407286 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff30c159c00 0x7ff30c14ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:53.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.830+0000 7ff30bfff640 1 -- 172.21.15.76:0/4153407286 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff30c159c00 msgr2=0x7ff30c14ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:53.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.830+0000 7ff30bfff640 1 --2- 172.21.15.76:0/4153407286 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff30c159c00 0x7ff30c14ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.830+0000 7ff30bfff640 1 -- 172.21.15.76:0/4153407286 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff30c0c4ff0 msgr2=0x7ff30c140060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:53.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.830+0000 7ff30bfff640 1 --2- 172.21.15.76:0/4153407286 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff30c0c4ff0 0x7ff30c140060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:53.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.831+0000 7ff30bfff640 1 -- 172.21.15.76:0/4153407286 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3000094d0 con 0x7ff30c0adac0 2023-11-09T03:48:53.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.831+0000 7ff30bfff640 1 --2- 172.21.15.76:0/4153407286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff30c0adac0 0x7ff30c15bbb0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00c3c0 tx=0x7ff2fc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:53.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.831+0000 7ff308ff9640 1 -- 172.21.15.76:0/4153407286 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc0052d0 con 0x7ff30c0adac0 2023-11-09T03:48:53.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.831+0000 7ff308ff9640 1 -- 172.21.15.76:0/4153407286 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2fc005450 con 0x7ff30c0adac0 2023-11-09T03:48:53.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.831+0000 7ff312955640 1 -- 172.21.15.76:0/4153407286 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff30c0c7800 con 0x7ff30c0adac0 2023-11-09T03:48:53.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.831+0000 7ff308ff9640 1 -- 172.21.15.76:0/4153407286 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc007260 con 0x7ff30c0adac0 2023-11-09T03:48:53.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.831+0000 7ff312955640 1 -- 172.21.15.76:0/4153407286 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff30c0c7dc0 con 0x7ff30c0adac0 2023-11-09T03:48:53.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.832+0000 7ff312955640 1 -- 172.21.15.76:0/4153407286 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff30c074290 con 0x7ff30c0adac0 2023-11-09T03:48:53.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.832+0000 7ff308ff9640 1 -- 172.21.15.76:0/4153407286 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff2fc0073c0 con 0x7ff30c0adac0 2023-11-09T03:48:53.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.833+0000 7ff308ff9640 1 --2- 172.21.15.76:0/4153407286 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff2f806ae70 0x7ff2f806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:53.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.833+0000 7ff308ff9640 1 -- 172.21.15.76:0/4153407286 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6268+0+0 (secure 0 0 0) 0x7ff2fc088c60 con 0x7ff30c0adac0 2023-11-09T03:48:53.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.833+0000 7ff30b7fe640 1 --2- 172.21.15.76:0/4153407286 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff2f806ae70 0x7ff2f806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:53.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.833+0000 7ff30b7fe640 1 --2- 172.21.15.76:0/4153407286 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff2f806ae70 0x7ff2f806d330 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff300036000 tx=0x7ff300008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:53.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:53.837+0000 7ff308ff9640 1 -- 172.21.15.76:0/4153407286 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff2fc053a60 con 0x7ff30c0adac0 2023-11-09T03:48:54.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.002+0000 7ff312955640 1 -- 172.21.15.76:0/4153407286 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff30c14d300 con 0x7ff30c0adac0 2023-11-09T03:48:54.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.002+0000 7ff308ff9640 1 -- 172.21.15.76:0/4153407286 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff2fc057080 con 0x7ff30c0adac0 2023-11-09T03:48:54.003 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:48:54.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.004+0000 7ff312955640 1 -- 172.21.15.76:0/4153407286 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff2f806ae70 msgr2=0x7ff2f806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:54.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.004+0000 7ff312955640 1 --2- 172.21.15.76:0/4153407286 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff2f806ae70 0x7ff2f806d330 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff300036000 tx=0x7ff300008030 comp rx=0 tx=0).stop 2023-11-09T03:48:54.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.004+0000 7ff312955640 1 -- 172.21.15.76:0/4153407286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff30c0adac0 msgr2=0x7ff30c15bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:54.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.004+0000 7ff312955640 1 --2- 172.21.15.76:0/4153407286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff30c0adac0 0x7ff30c15bbb0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00c3c0 tx=0x7ff2fc00c880 comp rx=0 tx=0).stop 2023-11-09T03:48:54.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.004+0000 7ff312955640 1 -- 172.21.15.76:0/4153407286 shutdown_connections 2023-11-09T03:48:54.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.004+0000 7ff312955640 1 --2- 172.21.15.76:0/4153407286 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff2f806ae70 0x7ff2f806d330 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.004+0000 7ff312955640 1 --2- 172.21.15.76:0/4153407286 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff30c159c00 0x7ff30c14ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.004+0000 7ff312955640 1 --2- 172.21.15.76:0/4153407286 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff30c0c4ff0 0x7ff30c140060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.004+0000 7ff312955640 1 --2- 172.21.15.76:0/4153407286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff30c0adac0 0x7ff30c15bbb0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.004+0000 7ff312955640 1 -- 172.21.15.76:0/4153407286 >> 172.21.15.76:0/4153407286 conn(0x7ff30c0ce950 msgr2=0x7ff30c150c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:54.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.004+0000 7ff312955640 1 -- 172.21.15.76:0/4153407286 shutdown_connections 2023-11-09T03:48:54.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.004+0000 7ff312955640 1 -- 172.21.15.76:0/4153407286 wait complete. 2023-11-09T03:48:54.015 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-09T03:48:54.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.118+0000 7fbd084b0640 1 Processor -- start 2023-11-09T03:48:54.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.118+0000 7fbd084b0640 1 -- start start 2023-11-09T03:48:54.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.119+0000 7fbd084b0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd00160430 0x7fbd000b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:54.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.119+0000 7fbd084b0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd000b2fc0 con 0x7fbd0006c3a0 2023-11-09T03:48:54.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.119+0000 7fbd084b0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd0006c810 con 0x7fbd000b2b60 2023-11-09T03:48:54.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.119+0000 7fbd084b0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd000cb580 con 0x7fbd00160430 2023-11-09T03:48:54.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.119+0000 7fbd06a4d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fbd0006c3a0 0x7fbd000cbcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35852/0 (socket says 172.21.15.76:35852) 2023-11-09T03:48:54.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.119+0000 7fbd06a4d640 1 -- 172.21.15.76:0/2845751330 learned_addr learned my addr 172.21.15.76:0/2845751330 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:54.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.120+0000 7fbd0624c640 1 --2- 172.21.15.76:0/2845751330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd00160430 0x7fbd000b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:54.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.120+0000 7fbd0524a640 1 -- 172.21.15.76:0/2845751330 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3358096039 0 0) 0x7fbd000b2fc0 con 0x7fbd0006c3a0 2023-11-09T03:48:54.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.120+0000 7fbd0524a640 1 -- 172.21.15.76:0/2845751330 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcec003520 con 0x7fbd0006c3a0 2023-11-09T03:48:54.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.120+0000 7fbd0524a640 1 -- 172.21.15.76:0/2845751330 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4103898776 0 0) 0x7fbcec003520 con 0x7fbd0006c3a0 2023-11-09T03:48:54.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.120+0000 7fbd0524a640 1 -- 172.21.15.76:0/2845751330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd00160430 msgr2=0x7fbd000b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:54.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.120+0000 7fbd0524a640 1 --2- 172.21.15.76:0/2845751330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd00160430 0x7fbd000b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.120+0000 7fbd0524a640 1 -- 172.21.15.76:0/2845751330 >> v1:172.21.15.113:6789/0 conn(0x7fbd000b2b60 legacy=0x7fbd0006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:54.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.120+0000 7fbd0524a640 1 -- 172.21.15.76:0/2845751330 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0006ca40 con 0x7fbd0006c3a0 2023-11-09T03:48:54.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.120+0000 7fbd0524a640 1 -- 172.21.15.76:0/2845751330 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fbcfc001670 con 0x7fbd0006c3a0 2023-11-09T03:48:54.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.120+0000 7fbd0524a640 1 -- 172.21.15.76:0/2845751330 >> v1:172.21.15.76:6789/0 conn(0x7fbd0006c3a0 legacy=0x7fbd000cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:54.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.120+0000 7fbd0524a640 1 --2- 172.21.15.76:0/2845751330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcec0039a0 0x7fbcec003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:54.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.120+0000 7fbd0524a640 1 --2- 172.21.15.76:0/2845751330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd00160430 0x7fbcec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:54.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.120+0000 7fbd0524a640 1 --2- 172.21.15.76:0/2845751330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcec004d00 0x7fbcec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:54.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.120+0000 7fbd0524a640 1 -- 172.21.15.76:0/2845751330 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd000cb580 con 0x7fbcec0039a0 2023-11-09T03:48:54.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.120+0000 7fbd0524a640 1 -- 172.21.15.76:0/2845751330 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec00ba30 con 0x7fbcec004d00 2023-11-09T03:48:54.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.120+0000 7fbd0524a640 1 -- 172.21.15.76:0/2845751330 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec00bbf0 con 0x7fbd00160430 2023-11-09T03:48:54.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.120+0000 7fbd0624c640 1 --2- 172.21.15.76:0/2845751330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcec0039a0 0x7fbcec003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:54.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.120+0000 7fbd06a4d640 1 --2- 172.21.15.76:0/2845751330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcec004d00 0x7fbcec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:54.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.120+0000 7fbd05a4b640 1 --2- 172.21.15.76:0/2845751330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd00160430 0x7fbcec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:54.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.120+0000 7fbd0624c640 1 -- 172.21.15.76:0/2845751330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd00160430 msgr2=0x7fbcec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:54.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.120+0000 7fbd0624c640 1 --2- 172.21.15.76:0/2845751330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd00160430 0x7fbcec0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.120+0000 7fbd0624c640 1 -- 172.21.15.76:0/2845751330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcec004d00 msgr2=0x7fbcec005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:54.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.120+0000 7fbd0624c640 1 --2- 172.21.15.76:0/2845751330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcec004d00 0x7fbcec005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.120+0000 7fbd0624c640 1 -- 172.21.15.76:0/2845751330 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec003520 con 0x7fbcec0039a0 2023-11-09T03:48:54.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.121+0000 7fbd0624c640 1 --2- 172.21.15.76:0/2845751330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcec0039a0 0x7fbcec003dc0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8009f90 tx=0x7fbcec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a0416803cbc6e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:54.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.121+0000 7fbd0524a640 1 -- 172.21.15.76:0/2845751330 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf802cbd0 con 0x7fbcec0039a0 2023-11-09T03:48:54.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.121+0000 7fbd0524a640 1 -- 172.21.15.76:0/2845751330 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf800f040 con 0x7fbcec0039a0 2023-11-09T03:48:54.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.121+0000 7fbd0524a640 1 -- 172.21.15.76:0/2845751330 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8034440 con 0x7fbcec0039a0 2023-11-09T03:48:54.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.122+0000 7fbd084b0640 1 -- 172.21.15.76:0/2845751330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcec0039a0 msgr2=0x7fbcec003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:54.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.122+0000 7fbd084b0640 1 --2- 172.21.15.76:0/2845751330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcec0039a0 0x7fbcec003dc0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8009f90 tx=0x7fbcec0057f0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.122+0000 7fbd06a4d640 1 -- 172.21.15.76:0/2845751330 reap_dead start 2023-11-09T03:48:54.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.122+0000 7fbd084b0640 1 -- 172.21.15.76:0/2845751330 shutdown_connections 2023-11-09T03:48:54.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.122+0000 7fbd084b0640 1 -- 172.21.15.76:0/2845751330 >> 172.21.15.76:0/2845751330 conn(0x7fbd0006ea20 msgr2=0x7fbd0006ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:54.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.122+0000 7fbd084b0640 1 -- 172.21.15.76:0/2845751330 shutdown_connections 2023-11-09T03:48:54.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.122+0000 7fbd084b0640 1 -- 172.21.15.76:0/2845751330 wait complete. 2023-11-09T03:48:54.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.123+0000 7fbd084b0640 1 Processor -- start 2023-11-09T03:48:54.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.123+0000 7fbd084b0640 1 -- start start 2023-11-09T03:48:54.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.124+0000 7fbd084b0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbcec0039a0 0x7fbd000b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:54.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.124+0000 7fbd084b0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd000b5bb0 0x7fbd000c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:54.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.124+0000 7fbd084b0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd00151c40 0x7fbd001520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:54.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.124+0000 7fbd084b0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd00152df0 con 0x7fbd00151c40 2023-11-09T03:48:54.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.124+0000 7fbd084b0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd00152600 con 0x7fbd000b5bb0 2023-11-09T03:48:54.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.124+0000 7fbd084b0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd000b3d00 con 0x7fbcec0039a0 2023-11-09T03:48:54.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.124+0000 7fbd06a4d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd00151c40 0x7fbd001520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:54.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.124+0000 7fbd06a4d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd00151c40 0x7fbd001520c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43982/0 (socket says 172.21.15.76:43982) 2023-11-09T03:48:54.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.124+0000 7fbd06a4d640 1 -- 172.21.15.76:0/3273474746 learned_addr learned my addr 172.21.15.76:0/3273474746 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:54.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.124+0000 7fbd0624c640 1 --2- 172.21.15.76:0/3273474746 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbcec0039a0 0x7fbd000b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:54.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.124+0000 7fbd05a4b640 1 --2- 172.21.15.76:0/3273474746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd000b5bb0 0x7fbd000c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:54.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.124+0000 7fbd06a4d640 1 -- 172.21.15.76:0/3273474746 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbcec0039a0 msgr2=0x7fbd000b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:54.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.124+0000 7fbd06a4d640 1 --2- 172.21.15.76:0/3273474746 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbcec0039a0 0x7fbd000b37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.124+0000 7fbd06a4d640 1 -- 172.21.15.76:0/3273474746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd000b5bb0 msgr2=0x7fbd000c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:54.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.124+0000 7fbd06a4d640 1 --2- 172.21.15.76:0/3273474746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd000b5bb0 0x7fbd000c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.124+0000 7fbd06a4d640 1 -- 172.21.15.76:0/3273474746 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd000b4840 con 0x7fbd00151c40 2023-11-09T03:48:54.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.124+0000 7fbd05a4b640 1 --2- 172.21.15.76:0/3273474746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd000b5bb0 0x7fbd000c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:54.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.124+0000 7fbd06a4d640 1 --2- 172.21.15.76:0/3273474746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd00151c40 0x7fbd001520c0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fbd000c4600 tx=0x7fbcfc00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:54.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.124+0000 7fbcf6ffd640 1 -- 172.21.15.76:0/3273474746 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc00eed0 con 0x7fbd00151c40 2023-11-09T03:48:54.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.124+0000 7fbcf6ffd640 1 -- 172.21.15.76:0/3273474746 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcfc00c690 con 0x7fbd00151c40 2023-11-09T03:48:54.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.124+0000 7fbcf6ffd640 1 -- 172.21.15.76:0/3273474746 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc00ca20 con 0x7fbd00151c40 2023-11-09T03:48:54.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.124+0000 7fbd084b0640 1 -- 172.21.15.76:0/3273474746 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd000b3fa0 con 0x7fbd00151c40 2023-11-09T03:48:54.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.124+0000 7fbd084b0640 1 -- 172.21.15.76:0/3273474746 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd000b4560 con 0x7fbd00151c40 2023-11-09T03:48:54.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.125+0000 7fbd084b0640 1 -- 172.21.15.76:0/3273474746 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd00152f30 con 0x7fbd00151c40 2023-11-09T03:48:54.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.126+0000 7fbcf6ffd640 1 -- 172.21.15.76:0/3273474746 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbcfc00cb80 con 0x7fbd00151c40 2023-11-09T03:48:54.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.126+0000 7fbcf6ffd640 1 --2- 172.21.15.76:0/3273474746 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbce406ae70 0x7fbce406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:54.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.126+0000 7fbcf6ffd640 1 -- 172.21.15.76:0/3273474746 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6268+0+0 (secure 0 0 0) 0x7fbcfc089470 con 0x7fbd00151c40 2023-11-09T03:48:54.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.126+0000 7fbd0624c640 1 --2- 172.21.15.76:0/3273474746 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbce406ae70 0x7fbce406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:54.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.127+0000 7fbd0624c640 1 --2- 172.21.15.76:0/3273474746 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbce406ae70 0x7fbce406d330 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8036000 tx=0x7fbcf802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:54.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.130+0000 7fbcf6ffd640 1 -- 172.21.15.76:0/3273474746 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbd00152f30 con 0x7fbd00151c40 2023-11-09T03:48:54.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.295+0000 7fbd084b0640 1 -- 172.21.15.76:0/3273474746 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7fbd00152f30 con 0x7fbd00151c40 2023-11-09T03:48:54.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.296+0000 7fbcf6ffd640 1 -- 172.21.15.76:0/3273474746 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbd00152f30 con 0x7fbd00151c40 2023-11-09T03:48:54.296 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:48:54.298 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.297+0000 7fbd084b0640 1 -- 172.21.15.76:0/3273474746 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbce406ae70 msgr2=0x7fbce406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:54.298 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.297+0000 7fbd084b0640 1 --2- 172.21.15.76:0/3273474746 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbce406ae70 0x7fbce406d330 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8036000 tx=0x7fbcf802e000 comp rx=0 tx=0).stop 2023-11-09T03:48:54.298 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.297+0000 7fbd084b0640 1 -- 172.21.15.76:0/3273474746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd00151c40 msgr2=0x7fbd001520c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:54.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.297+0000 7fbd084b0640 1 --2- 172.21.15.76:0/3273474746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd00151c40 0x7fbd001520c0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fbd000c4600 tx=0x7fbcfc00eb90 comp rx=0 tx=0).stop 2023-11-09T03:48:54.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.297+0000 7fbd084b0640 1 -- 172.21.15.76:0/3273474746 shutdown_connections 2023-11-09T03:48:54.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.297+0000 7fbd084b0640 1 --2- 172.21.15.76:0/3273474746 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbce406ae70 0x7fbce406d330 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.297+0000 7fbd084b0640 1 --2- 172.21.15.76:0/3273474746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd00151c40 0x7fbd001520c0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.297+0000 7fbd084b0640 1 --2- 172.21.15.76:0/3273474746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd000b5bb0 0x7fbd000c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.297+0000 7fbd084b0640 1 --2- 172.21.15.76:0/3273474746 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbcec0039a0 0x7fbd000b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.297+0000 7fbd084b0640 1 -- 172.21.15.76:0/3273474746 >> 172.21.15.76:0/3273474746 conn(0x7fbd0006ea20 msgr2=0x7fbd00072be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:54.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.298+0000 7fbd084b0640 1 -- 172.21.15.76:0/3273474746 shutdown_connections 2023-11-09T03:48:54.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.298+0000 7fbd084b0640 1 -- 172.21.15.76:0/3273474746 wait complete. 2023-11-09T03:48:54.309 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-09T03:48:54.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.411+0000 7f8160456640 1 Processor -- start 2023-11-09T03:48:54.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.411+0000 7f8160456640 1 -- start start 2023-11-09T03:48:54.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.412+0000 7f8160456640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f815815ad30 0x7f81580adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:54.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.412+0000 7f8160456640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f81580ad730 0x7f8158070710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:54.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.412+0000 7f8160456640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81580c3900 con 0x7f8158070c50 2023-11-09T03:48:54.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.412+0000 7f8160456640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81580ae260 con 0x7f81580ad730 2023-11-09T03:48:54.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.412+0000 7f8160456640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f815815f750 con 0x7f815815ad30 2023-11-09T03:48:54.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.412+0000 7f815e1f2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f815815ad30 0x7f81580adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:54.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.412+0000 7f815e1f2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f815815ad30 0x7f81580adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:56248/0 (socket says 172.21.15.76:56248) 2023-11-09T03:48:54.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.412+0000 7f815e1f2640 1 -- 172.21.15.76:0/3157343743 learned_addr learned my addr 172.21.15.76:0/3157343743 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:54.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.412+0000 7f815d9f1640 1 --2- 172.21.15.76:0/3157343743 >> v2:172.21.15.113:3300/0 conn(0x7f81580ad730 0x7f8158070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:54.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.412+0000 7f815e1f2640 1 -- 172.21.15.76:0/3157343743 >> v2:172.21.15.113:3300/0 conn(0x7f81580ad730 msgr2=0x7f8158070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:54.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.412+0000 7f815e1f2640 1 --2- 172.21.15.76:0/3157343743 >> v2:172.21.15.113:3300/0 conn(0x7f81580ad730 0x7f8158070710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.412+0000 7f815e1f2640 1 -- 172.21.15.76:0/3157343743 >> v1:172.21.15.113:6789/0 conn(0x7f8158070c50 legacy=0x7f81580c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:54.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.412+0000 7f815e1f2640 1 -- 172.21.15.76:0/3157343743 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81580c3be0 con 0x7f815815ad30 2023-11-09T03:48:54.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.413+0000 7f815e1f2640 1 --2- 172.21.15.76:0/3157343743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f815815ad30 0x7f81580adc80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f81500098a0 tx=0x7f815002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca60fa21200a909e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:54.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.413+0000 7f815d1f0640 1 -- 172.21.15.76:0/3157343743 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8150031640 con 0x7f815815ad30 2023-11-09T03:48:54.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.413+0000 7f815d1f0640 1 -- 172.21.15.76:0/3157343743 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81500317a0 con 0x7f815815ad30 2023-11-09T03:48:54.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.413+0000 7f815d1f0640 1 -- 172.21.15.76:0/3157343743 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8150031b30 con 0x7f815815ad30 2023-11-09T03:48:54.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.414+0000 7f8160456640 1 -- 172.21.15.76:0/3157343743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f815815ad30 msgr2=0x7f81580adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:54.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.414+0000 7f8160456640 1 --2- 172.21.15.76:0/3157343743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f815815ad30 0x7f81580adc80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f81500098a0 tx=0x7f815002b8f0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.414+0000 7f8160456640 1 -- 172.21.15.76:0/3157343743 shutdown_connections 2023-11-09T03:48:54.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.414+0000 7f8160456640 1 --2- 172.21.15.76:0/3157343743 >> v2:172.21.15.113:3300/0 conn(0x7f81580ad730 0x7f8158070710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.414+0000 7f8160456640 1 --2- 172.21.15.76:0/3157343743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f815815ad30 0x7f81580adc80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.414+0000 7f8160456640 1 -- 172.21.15.76:0/3157343743 >> 172.21.15.76:0/3157343743 conn(0x7f81580ce950 msgr2=0x7f81580ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:54.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.414+0000 7f8160456640 1 -- 172.21.15.76:0/3157343743 shutdown_connections 2023-11-09T03:48:54.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.414+0000 7f8160456640 1 -- 172.21.15.76:0/3157343743 wait complete. 2023-11-09T03:48:54.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.415+0000 7f8160456640 1 Processor -- start 2023-11-09T03:48:54.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.415+0000 7f8160456640 1 -- start start 2023-11-09T03:48:54.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.416+0000 7f8160456640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8158070c50 0x7f815814f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:54.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.416+0000 7f8160456640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81580ad730 0x7f81580cce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:54.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.416+0000 7f8160456640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f815815ad30 0x7f81580cd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:54.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.416+0000 7f8160456640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81581564f0 con 0x7f8158070c50 2023-11-09T03:48:54.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.416+0000 7f8160456640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8158155690 con 0x7f815815ad30 2023-11-09T03:48:54.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.416+0000 7f8160456640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8158155970 con 0x7f81580ad730 2023-11-09T03:48:54.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.416+0000 7f815e1f2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8158070c50 0x7f815814f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:54.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.416+0000 7f815e1f2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8158070c50 0x7f815814f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43992/0 (socket says 172.21.15.76:43992) 2023-11-09T03:48:54.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.416+0000 7f815e1f2640 1 -- 172.21.15.76:0/1977295428 learned_addr learned my addr 172.21.15.76:0/1977295428 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:54.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.416+0000 7f815e9f3640 1 --2- 172.21.15.76:0/1977295428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f815815ad30 0x7f81580cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:54.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.416+0000 7f815d9f1640 1 --2- 172.21.15.76:0/1977295428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81580ad730 0x7f81580cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:54.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.416+0000 7f815e1f2640 1 -- 172.21.15.76:0/1977295428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81580ad730 msgr2=0x7f81580cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:54.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.416+0000 7f815e1f2640 1 --2- 172.21.15.76:0/1977295428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81580ad730 0x7f81580cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.416+0000 7f815e1f2640 1 -- 172.21.15.76:0/1977295428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f815815ad30 msgr2=0x7f81580cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:54.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.416+0000 7f815e1f2640 1 --2- 172.21.15.76:0/1977295428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f815815ad30 0x7f81580cd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.416+0000 7f815e1f2640 1 -- 172.21.15.76:0/1977295428 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81500094d0 con 0x7f8158070c50 2023-11-09T03:48:54.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.417+0000 7f815e1f2640 1 --2- 172.21.15.76:0/1977295428 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8158070c50 0x7f815814f680 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f815002b930 tx=0x7f81500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:54.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.417+0000 7f8156ffd640 1 -- 172.21.15.76:0/1977295428 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815003a930 con 0x7f8158070c50 2023-11-09T03:48:54.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.417+0000 7f8156ffd640 1 -- 172.21.15.76:0/1977295428 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f815003aa90 con 0x7f8158070c50 2023-11-09T03:48:54.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.417+0000 7f8160456640 1 -- 172.21.15.76:0/1977295428 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8158155c50 con 0x7f8158070c50 2023-11-09T03:48:54.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.417+0000 7f8156ffd640 1 -- 172.21.15.76:0/1977295428 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815003ae40 con 0x7f8158070c50 2023-11-09T03:48:54.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.417+0000 7f8160456640 1 -- 172.21.15.76:0/1977295428 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8158156210 con 0x7f8158070c50 2023-11-09T03:48:54.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.418+0000 7f8160456640 1 -- 172.21.15.76:0/1977295428 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8158074290 con 0x7f8158070c50 2023-11-09T03:48:54.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.418+0000 7f8156ffd640 1 -- 172.21.15.76:0/1977295428 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f815002f050 con 0x7f8158070c50 2023-11-09T03:48:54.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.419+0000 7f8156ffd640 1 --2- 172.21.15.76:0/1977295428 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f814406ae70 0x7f814406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:54.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.419+0000 7f8156ffd640 1 -- 172.21.15.76:0/1977295428 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f81500ab690 con 0x7f8158070c50 2023-11-09T03:48:54.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.419+0000 7f815d9f1640 1 --2- 172.21.15.76:0/1977295428 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f814406ae70 0x7f814406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:54.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.419+0000 7f815d9f1640 1 --2- 172.21.15.76:0/1977295428 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f814406ae70 0x7f814406d330 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f81580c8820 tx=0x7f814c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:54.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.423+0000 7f8156ffd640 1 -- 172.21.15.76:0/1977295428 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8150076520 con 0x7f8158070c50 2023-11-09T03:48:54.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.588+0000 7f8160456640 1 -- 172.21.15.76:0/1977295428 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7f8158071970 con 0x7f8158070c50 2023-11-09T03:48:54.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.589+0000 7f8156ffd640 1 -- 172.21.15.76:0/1977295428 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f815002d040 con 0x7f8158070c50 2023-11-09T03:48:54.590 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:48:54.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.590+0000 7f8160456640 1 -- 172.21.15.76:0/1977295428 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f814406ae70 msgr2=0x7f814406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:54.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.590+0000 7f8160456640 1 --2- 172.21.15.76:0/1977295428 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f814406ae70 0x7f814406d330 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f81580c8820 tx=0x7f814c008040 comp rx=0 tx=0).stop 2023-11-09T03:48:54.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.590+0000 7f8160456640 1 -- 172.21.15.76:0/1977295428 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8158070c50 msgr2=0x7f815814f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:54.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.590+0000 7f8160456640 1 --2- 172.21.15.76:0/1977295428 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8158070c50 0x7f815814f680 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f815002b930 tx=0x7f81500046c0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.591+0000 7f8160456640 1 -- 172.21.15.76:0/1977295428 shutdown_connections 2023-11-09T03:48:54.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.591+0000 7f8160456640 1 --2- 172.21.15.76:0/1977295428 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f814406ae70 0x7f814406d330 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.591+0000 7f8160456640 1 --2- 172.21.15.76:0/1977295428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f815815ad30 0x7f81580cd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.591+0000 7f8160456640 1 --2- 172.21.15.76:0/1977295428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81580ad730 0x7f81580cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.591+0000 7f8160456640 1 --2- 172.21.15.76:0/1977295428 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8158070c50 0x7f815814f680 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.591+0000 7f8160456640 1 -- 172.21.15.76:0/1977295428 >> 172.21.15.76:0/1977295428 conn(0x7f81580ce950 msgr2=0x7f815806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:54.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.591+0000 7f8160456640 1 -- 172.21.15.76:0/1977295428 shutdown_connections 2023-11-09T03:48:54.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.591+0000 7f8160456640 1 -- 172.21.15.76:0/1977295428 wait complete. 2023-11-09T03:48:54.602 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-09T03:48:54.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.704+0000 7fd328221640 1 Processor -- start 2023-11-09T03:48:54.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.704+0000 7fd328221640 1 -- start start 2023-11-09T03:48:54.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.705+0000 7fd328221640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd320070e20 0x7fd3200adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:54.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.705+0000 7fd328221640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3200ad730 0x7fd32015e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:54.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.705+0000 7fd328221640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3200c3900 con 0x7fd32015ad30 2023-11-09T03:48:54.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.705+0000 7fd328221640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3200ae260 con 0x7fd320070e20 2023-11-09T03:48:54.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.705+0000 7fd328221640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3200c3be0 con 0x7fd3200ad730 2023-11-09T03:48:54.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.705+0000 7fd3257bc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd320070e20 0x7fd3200adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:54.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.705+0000 7fd325fbd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd32015ad30 0x7fd320070710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35862/0 (socket says 172.21.15.76:35862) 2023-11-09T03:48:54.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.705+0000 7fd325fbd640 1 -- 172.21.15.76:0/3857628396 learned_addr learned my addr 172.21.15.76:0/3857628396 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:54.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.705+0000 7fd3267be640 1 --2- 172.21.15.76:0/3857628396 >> v2:172.21.15.113:3300/0 conn(0x7fd3200ad730 0x7fd32015e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:54.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.705+0000 7fd3257bc640 1 -- 172.21.15.76:0/3857628396 >> v2:172.21.15.113:3300/0 conn(0x7fd3200ad730 msgr2=0x7fd32015e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:54.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.705+0000 7fd3257bc640 1 --2- 172.21.15.76:0/3857628396 >> v2:172.21.15.113:3300/0 conn(0x7fd3200ad730 0x7fd32015e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.705+0000 7fd3257bc640 1 -- 172.21.15.76:0/3857628396 >> v1:172.21.15.76:6789/0 conn(0x7fd32015ad30 legacy=0x7fd320070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:54.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.705+0000 7fd3257bc640 1 -- 172.21.15.76:0/3857628396 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3200c3ec0 con 0x7fd320070e20 2023-11-09T03:48:54.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.705+0000 7fd324fbb640 1 -- 172.21.15.76:0/3857628396 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2602771361 0 0) 0x7fd3200c3900 con 0x7fd32015ad30 2023-11-09T03:48:54.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.705+0000 7fd3267be640 1 --2- 172.21.15.76:0/3857628396 >> v2:172.21.15.113:3300/0 conn(0x7fd3200ad730 0x7fd32015e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:54.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.706+0000 7fd3257bc640 1 --2- 172.21.15.76:0/3857628396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd320070e20 0x7fd3200adc80 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fd3200cf9e0 tx=0x7fd31002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865ef98d7129e0c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:54.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.706+0000 7fd324fbb640 1 -- 172.21.15.76:0/3857628396 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31002e070 con 0x7fd320070e20 2023-11-09T03:48:54.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.706+0000 7fd324fbb640 1 -- 172.21.15.76:0/3857628396 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3100315a0 con 0x7fd320070e20 2023-11-09T03:48:54.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.706+0000 7fd324fbb640 1 -- 172.21.15.76:0/3857628396 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd310031930 con 0x7fd320070e20 2023-11-09T03:48:54.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.707+0000 7fd328221640 1 -- 172.21.15.76:0/3857628396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd320070e20 msgr2=0x7fd3200adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:54.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.707+0000 7fd328221640 1 --2- 172.21.15.76:0/3857628396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd320070e20 0x7fd3200adc80 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fd3200cf9e0 tx=0x7fd31002b840 comp rx=0 tx=0).stop 2023-11-09T03:48:54.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.707+0000 7fd328221640 1 -- 172.21.15.76:0/3857628396 shutdown_connections 2023-11-09T03:48:54.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.707+0000 7fd328221640 1 --2- 172.21.15.76:0/3857628396 >> v2:172.21.15.113:3300/0 conn(0x7fd3200ad730 0x7fd32015e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.707+0000 7fd328221640 1 --2- 172.21.15.76:0/3857628396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd320070e20 0x7fd3200adc80 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.707+0000 7fd328221640 1 -- 172.21.15.76:0/3857628396 >> 172.21.15.76:0/3857628396 conn(0x7fd3200ce950 msgr2=0x7fd3200ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:54.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.707+0000 7fd328221640 1 -- 172.21.15.76:0/3857628396 shutdown_connections 2023-11-09T03:48:54.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.707+0000 7fd328221640 1 -- 172.21.15.76:0/3857628396 wait complete. 2023-11-09T03:48:54.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.708+0000 7fd328221640 1 Processor -- start 2023-11-09T03:48:54.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.708+0000 7fd328221640 1 -- start start 2023-11-09T03:48:54.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.709+0000 7fd328221640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd320070e20 0x7fd32014e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:54.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.709+0000 7fd328221640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3200ad730 0x7fd3200b3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:54.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.709+0000 7fd328221640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd32015ad30 0x7fd3200b3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:54.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.709+0000 7fd328221640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3200b28e0 con 0x7fd32015ad30 2023-11-09T03:48:54.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.709+0000 7fd328221640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32014e560 con 0x7fd3200ad730 2023-11-09T03:48:54.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.709+0000 7fd328221640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd32015b680 con 0x7fd320070e20 2023-11-09T03:48:54.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.709+0000 7fd3267be640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd32015ad30 0x7fd3200b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:54.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.709+0000 7fd3267be640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd32015ad30 0x7fd3200b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44024/0 (socket says 172.21.15.76:44024) 2023-11-09T03:48:54.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.709+0000 7fd3267be640 1 -- 172.21.15.76:0/98500661 learned_addr learned my addr 172.21.15.76:0/98500661 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:54.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.709+0000 7fd325fbd640 1 --2- 172.21.15.76:0/98500661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd320070e20 0x7fd32014e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:54.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.709+0000 7fd3267be640 1 -- 172.21.15.76:0/98500661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd320070e20 msgr2=0x7fd32014e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:54.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.709+0000 7fd3267be640 1 --2- 172.21.15.76:0/98500661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd320070e20 0x7fd32014e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.709+0000 7fd3267be640 1 -- 172.21.15.76:0/98500661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3200ad730 msgr2=0x7fd3200b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:54.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.709+0000 7fd3267be640 1 --2- 172.21.15.76:0/98500661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3200ad730 0x7fd3200b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.709+0000 7fd3267be640 1 -- 172.21.15.76:0/98500661 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd310009400 con 0x7fd32015ad30 2023-11-09T03:48:54.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.709+0000 7fd3267be640 1 --2- 172.21.15.76:0/98500661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd32015ad30 0x7fd3200b3880 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fd31800cef0 tx=0x7fd31800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:54.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.709+0000 7fd3167fc640 1 -- 172.21.15.76:0/98500661 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31800d840 con 0x7fd32015ad30 2023-11-09T03:48:54.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.709+0000 7fd3167fc640 1 -- 172.21.15.76:0/98500661 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd31800d9c0 con 0x7fd32015ad30 2023-11-09T03:48:54.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.709+0000 7fd328221640 1 -- 172.21.15.76:0/98500661 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3200b2290 con 0x7fd32015ad30 2023-11-09T03:48:54.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.709+0000 7fd3167fc640 1 -- 172.21.15.76:0/98500661 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31800dd70 con 0x7fd32015ad30 2023-11-09T03:48:54.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.709+0000 7fd328221640 1 -- 172.21.15.76:0/98500661 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3200b1eb0 con 0x7fd32015ad30 2023-11-09T03:48:54.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.710+0000 7fd328221640 1 -- 172.21.15.76:0/98500661 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3200b2a50 con 0x7fd32015ad30 2023-11-09T03:48:54.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.711+0000 7fd3167fc640 1 -- 172.21.15.76:0/98500661 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd31801e3b0 con 0x7fd32015ad30 2023-11-09T03:48:54.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.711+0000 7fd3167fc640 1 --2- 172.21.15.76:0/98500661 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd30406aea0 0x7fd30406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:54.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.711+0000 7fd3167fc640 1 -- 172.21.15.76:0/98500661 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6268+0+0 (secure 0 0 0) 0x7fd31808a260 con 0x7fd32015ad30 2023-11-09T03:48:54.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.711+0000 7fd325fbd640 1 --2- 172.21.15.76:0/98500661 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd30406aea0 0x7fd30406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:54.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.712+0000 7fd325fbd640 1 --2- 172.21.15.76:0/98500661 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd30406aea0 0x7fd30406d360 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd31c000f60 tx=0x7fd31c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:54.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.715+0000 7fd3167fc640 1 -- 172.21.15.76:0/98500661 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd318055170 con 0x7fd32015ad30 2023-11-09T03:48:54.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.880+0000 7fd328221640 1 -- 172.21.15.76:0/98500661 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7fd3200b20a0 con 0x7fd32015ad30 2023-11-09T03:48:54.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.881+0000 7fd3167fc640 1 -- 172.21.15.76:0/98500661 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v91) v1 ==== 107+0+11 (secure 0 0 0) 0x7fd318058790 con 0x7fd32015ad30 2023-11-09T03:48:54.882 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:48:54.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.882+0000 7fd328221640 1 -- 172.21.15.76:0/98500661 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd30406aea0 msgr2=0x7fd30406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:54.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.882+0000 7fd328221640 1 --2- 172.21.15.76:0/98500661 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd30406aea0 0x7fd30406d360 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd31c000f60 tx=0x7fd31c009040 comp rx=0 tx=0).stop 2023-11-09T03:48:54.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.883+0000 7fd328221640 1 -- 172.21.15.76:0/98500661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd32015ad30 msgr2=0x7fd3200b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:54.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.883+0000 7fd328221640 1 --2- 172.21.15.76:0/98500661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd32015ad30 0x7fd3200b3880 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fd31800cef0 tx=0x7fd31800a710 comp rx=0 tx=0).stop 2023-11-09T03:48:54.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.883+0000 7fd328221640 1 -- 172.21.15.76:0/98500661 shutdown_connections 2023-11-09T03:48:54.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.883+0000 7fd328221640 1 --2- 172.21.15.76:0/98500661 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd30406aea0 0x7fd30406d360 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.883+0000 7fd328221640 1 --2- 172.21.15.76:0/98500661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd32015ad30 0x7fd3200b3880 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.883+0000 7fd328221640 1 --2- 172.21.15.76:0/98500661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3200ad730 0x7fd3200b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.883+0000 7fd328221640 1 --2- 172.21.15.76:0/98500661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd320070e20 0x7fd32014e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.883+0000 7fd328221640 1 -- 172.21.15.76:0/98500661 >> 172.21.15.76:0/98500661 conn(0x7fd3200ce950 msgr2=0x7fd3200af590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:54.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.883+0000 7fd328221640 1 -- 172.21.15.76:0/98500661 shutdown_connections 2023-11-09T03:48:54.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.883+0000 7fd328221640 1 -- 172.21.15.76:0/98500661 wait complete. 2023-11-09T03:48:54.894 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-09T03:48:54.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.995+0000 7fa4cb5fb640 1 Processor -- start 2023-11-09T03:48:54.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.995+0000 7fa4cb5fb640 1 -- start start 2023-11-09T03:48:54.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.996+0000 7fa4cb5fb640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa4c40b0ae0 0x7fa4c40ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:54.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.996+0000 7fa4cb5fb640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa4c406db20 0x7fa4c415d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:54.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.996+0000 7fa4cb5fb640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4c40b4110 con 0x7fa4c4159c70 2023-11-09T03:48:54.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.996+0000 7fa4cb5fb640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4c40b49b0 con 0x7fa4c40b0ae0 2023-11-09T03:48:54.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.996+0000 7fa4cb5fb640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4c40b43f0 con 0x7fa4c406db20 2023-11-09T03:48:54.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.996+0000 7fa4c8b96640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa4c40b0ae0 0x7fa4c40ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:54.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.996+0000 7fa4c8b96640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa4c40b0ae0 0x7fa4c40ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44032/0 (socket says 172.21.15.76:44032) 2023-11-09T03:48:54.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.996+0000 7fa4c8b96640 1 -- 172.21.15.76:0/3812615732 learned_addr learned my addr 172.21.15.76:0/3812615732 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:54.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.996+0000 7fa4c9b98640 1 --2- 172.21.15.76:0/3812615732 >> v2:172.21.15.113:3300/0 conn(0x7fa4c406db20 0x7fa4c415d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:54.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.996+0000 7fa4c8b96640 1 -- 172.21.15.76:0/3812615732 >> v2:172.21.15.113:3300/0 conn(0x7fa4c406db20 msgr2=0x7fa4c415d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:54.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.996+0000 7fa4c8b96640 1 --2- 172.21.15.76:0/3812615732 >> v2:172.21.15.113:3300/0 conn(0x7fa4c406db20 0x7fa4c415d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:54.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.996+0000 7fa4c8b96640 1 -- 172.21.15.76:0/3812615732 >> v1:172.21.15.113:6789/0 conn(0x7fa4c4159c70 legacy=0x7fa4c40b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:54.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.996+0000 7fa4c8b96640 1 -- 172.21.15.76:0/3812615732 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c40b46d0 con 0x7fa4c40b0ae0 2023-11-09T03:48:54.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.996+0000 7fa4c9b98640 1 --2- 172.21.15.76:0/3812615732 >> v2:172.21.15.113:3300/0 conn(0x7fa4c406db20 0x7fa4c415d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:55.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.997+0000 7fa4c8b96640 1 --2- 172.21.15.76:0/3812615732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4c40b0ae0 0x7fa4c40ce370 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4070580 tx=0x7fa4b402e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956578ac1c174a81 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:55.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.997+0000 7fa4bbfff640 1 -- 172.21.15.76:0/3812615732 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4036440 con 0x7fa4c40b0ae0 2023-11-09T03:48:55.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.997+0000 7fa4bbfff640 1 -- 172.21.15.76:0/3812615732 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b40365a0 con 0x7fa4c40b0ae0 2023-11-09T03:48:55.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.997+0000 7fa4bbfff640 1 -- 172.21.15.76:0/3812615732 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4036930 con 0x7fa4c40b0ae0 2023-11-09T03:48:55.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.998+0000 7fa4cb5fb640 1 -- 172.21.15.76:0/3812615732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4c40b0ae0 msgr2=0x7fa4c40ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:55.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.998+0000 7fa4cb5fb640 1 --2- 172.21.15.76:0/3812615732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4c40b0ae0 0x7fa4c40ce370 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4070580 tx=0x7fa4b402e170 comp rx=0 tx=0).stop 2023-11-09T03:48:55.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.998+0000 7fa4cb5fb640 1 -- 172.21.15.76:0/3812615732 shutdown_connections 2023-11-09T03:48:55.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.998+0000 7fa4cb5fb640 1 --2- 172.21.15.76:0/3812615732 >> v2:172.21.15.113:3300/0 conn(0x7fa4c406db20 0x7fa4c415d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:55.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.998+0000 7fa4cb5fb640 1 --2- 172.21.15.76:0/3812615732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4c40b0ae0 0x7fa4c40ce370 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:55.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.998+0000 7fa4cb5fb640 1 -- 172.21.15.76:0/3812615732 >> 172.21.15.76:0/3812615732 conn(0x7fa4c40ce950 msgr2=0x7fa4c40ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:55.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.998+0000 7fa4cb5fb640 1 -- 172.21.15.76:0/3812615732 shutdown_connections 2023-11-09T03:48:55.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.998+0000 7fa4cb5fb640 1 -- 172.21.15.76:0/3812615732 wait complete. 2023-11-09T03:48:55.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.999+0000 7fa4cb5fb640 1 Processor -- start 2023-11-09T03:48:55.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:54.999+0000 7fa4cb5fb640 1 -- start start 2023-11-09T03:48:55.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.000+0000 7fa4cb5fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4c406db20 0x7fa4c4156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:55.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.000+0000 7fa4cb5fb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4c40b0ae0 0x7fa4c41e2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:55.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.000+0000 7fa4cb5fb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c4159c70 0x7fa4c41ee600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:55.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.000+0000 7fa4cb5fb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c40ae6d0 con 0x7fa4c406db20 2023-11-09T03:48:55.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.000+0000 7fa4cb5fb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c40ae990 con 0x7fa4c4159c70 2023-11-09T03:48:55.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.000+0000 7fa4cb5fb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c40aec70 con 0x7fa4c40b0ae0 2023-11-09T03:48:55.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.000+0000 7fa4c9397640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4c406db20 0x7fa4c4156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:55.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.000+0000 7fa4c9397640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4c406db20 0x7fa4c4156990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44040/0 (socket says 172.21.15.76:44040) 2023-11-09T03:48:55.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.000+0000 7fa4c9397640 1 -- 172.21.15.76:0/1205363605 learned_addr learned my addr 172.21.15.76:0/1205363605 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:55.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.000+0000 7fa4c9b98640 1 --2- 172.21.15.76:0/1205363605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c4159c70 0x7fa4c41ee600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:55.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.000+0000 7fa4c8b96640 1 --2- 172.21.15.76:0/1205363605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4c40b0ae0 0x7fa4c41e2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:55.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.000+0000 7fa4c9397640 1 -- 172.21.15.76:0/1205363605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4c40b0ae0 msgr2=0x7fa4c41e2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:55.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.000+0000 7fa4c9397640 1 --2- 172.21.15.76:0/1205363605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4c40b0ae0 0x7fa4c41e2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:55.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.000+0000 7fa4c9397640 1 -- 172.21.15.76:0/1205363605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c4159c70 msgr2=0x7fa4c41ee600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:55.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.000+0000 7fa4c9397640 1 --2- 172.21.15.76:0/1205363605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c4159c70 0x7fa4c41ee600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:55.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.000+0000 7fa4c9397640 1 -- 172.21.15.76:0/1205363605 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b400b770 con 0x7fa4c406db20 2023-11-09T03:48:55.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.000+0000 7fa4c9b98640 1 --2- 172.21.15.76:0/1205363605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c4159c70 0x7fa4c41ee600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:55.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.000+0000 7fa4c9397640 1 --2- 172.21.15.76:0/1205363605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4c406db20 0x7fa4c4156990 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fa4c000c450 tx=0x7fa4c000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:55.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.000+0000 7fa4b9ffb640 1 -- 172.21.15.76:0/1205363605 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c00052d0 con 0x7fa4c406db20 2023-11-09T03:48:55.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.000+0000 7fa4b9ffb640 1 -- 172.21.15.76:0/1205363605 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c0005430 con 0x7fa4c406db20 2023-11-09T03:48:55.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.000+0000 7fa4cb5fb640 1 -- 172.21.15.76:0/1205363605 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4c40aef50 con 0x7fa4c406db20 2023-11-09T03:48:55.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.000+0000 7fa4b9ffb640 1 -- 172.21.15.76:0/1205363605 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0012870 con 0x7fa4c406db20 2023-11-09T03:48:55.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.001+0000 7fa4cb5fb640 1 -- 172.21.15.76:0/1205363605 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c4156ed0 con 0x7fa4c406db20 2023-11-09T03:48:55.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.002+0000 7fa4cb5fb640 1 -- 172.21.15.76:0/1205363605 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4c415d120 con 0x7fa4c406db20 2023-11-09T03:48:55.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.002+0000 7fa4b9ffb640 1 -- 172.21.15.76:0/1205363605 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa4c00129d0 con 0x7fa4c406db20 2023-11-09T03:48:55.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.002+0000 7fa4b9ffb640 1 --2- 172.21.15.76:0/1205363605 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa4a806aea0 0x7fa4a806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:55.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.003+0000 7fa4b9ffb640 1 -- 172.21.15.76:0/1205363605 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6268+0+0 (secure 0 0 0) 0x7fa4c0089340 con 0x7fa4c406db20 2023-11-09T03:48:55.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.003+0000 7fa4c8b96640 1 --2- 172.21.15.76:0/1205363605 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa4a806aea0 0x7fa4a806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:55.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.003+0000 7fa4c8b96640 1 --2- 172.21.15.76:0/1205363605 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa4a806aea0 0x7fa4a806d360 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4038000 tx=0x7fa4b4032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:55.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.007+0000 7fa4b9ffb640 1 -- 172.21.15.76:0/1205363605 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa4c00541c0 con 0x7fa4c406db20 2023-11-09T03:48:55.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.176+0000 7fa4cb5fb640 1 -- 172.21.15.76:0/1205363605 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa4c406d3f0 con 0x7fa4c406db20 2023-11-09T03:48:55.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.176+0000 7fa4b9ffb640 1 -- 172.21.15.76:0/1205363605 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v91) v1 ==== 84+0+287 (secure 0 0 0) 0x7fa4c00577e0 con 0x7fa4c406db20 2023-11-09T03:48:55.177 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:55.177 INFO:teuthology.orchestra.run.smithi076.stdout:[ 2023-11-09T03:48:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: "poolnum": 1, 2023-11-09T03:48:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: "poolname": ".mgr" 2023-11-09T03:48:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: "poolnum": 7, 2023-11-09T03:48:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: "poolname": "snap-test-pool" 2023-11-09T03:48:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: "poolnum": 12, 2023-11-09T03:48:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: "poolname": "test_data_pool" 2023-11-09T03:48:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:48:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: { 2023-11-09T03:48:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: "poolnum": 13, 2023-11-09T03:48:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: "poolname": "test_metadata_pool" 2023-11-09T03:48:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:48:55.181 INFO:teuthology.orchestra.run.smithi076.stdout:] 2023-11-09T03:48:55.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.178+0000 7fa4cb5fb640 1 -- 172.21.15.76:0/1205363605 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa4a806aea0 msgr2=0x7fa4a806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:55.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.178+0000 7fa4cb5fb640 1 --2- 172.21.15.76:0/1205363605 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa4a806aea0 0x7fa4a806d360 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4038000 tx=0x7fa4b4032000 comp rx=0 tx=0).stop 2023-11-09T03:48:55.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.178+0000 7fa4cb5fb640 1 -- 172.21.15.76:0/1205363605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4c406db20 msgr2=0x7fa4c4156990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:55.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.178+0000 7fa4cb5fb640 1 --2- 172.21.15.76:0/1205363605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4c406db20 0x7fa4c4156990 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fa4c000c450 tx=0x7fa4c000c910 comp rx=0 tx=0).stop 2023-11-09T03:48:55.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.178+0000 7fa4cb5fb640 1 -- 172.21.15.76:0/1205363605 shutdown_connections 2023-11-09T03:48:55.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.178+0000 7fa4cb5fb640 1 --2- 172.21.15.76:0/1205363605 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa4a806aea0 0x7fa4a806d360 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:55.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.178+0000 7fa4cb5fb640 1 --2- 172.21.15.76:0/1205363605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c4159c70 0x7fa4c41ee600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:55.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.178+0000 7fa4cb5fb640 1 --2- 172.21.15.76:0/1205363605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4c40b0ae0 0x7fa4c41e2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:55.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.178+0000 7fa4cb5fb640 1 --2- 172.21.15.76:0/1205363605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4c406db20 0x7fa4c4156990 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:55.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.178+0000 7fa4cb5fb640 1 -- 172.21.15.76:0/1205363605 >> 172.21.15.76:0/1205363605 conn(0x7fa4c40ce950 msgr2=0x7fa4c40b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:55.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.178+0000 7fa4cb5fb640 1 -- 172.21.15.76:0/1205363605 shutdown_connections 2023-11-09T03:48:55.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.178+0000 7fa4cb5fb640 1 -- 172.21.15.76:0/1205363605 wait complete. 2023-11-09T03:48:55.189 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-09T03:48:55.190 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-09T03:48:55.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.293+0000 7f1d22e1a640 1 Processor -- start 2023-11-09T03:48:55.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.293+0000 7f1d22e1a640 1 -- start start 2023-11-09T03:48:55.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.293+0000 7f1d22e1a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1d1c0c4f90 0x7f1d1c15d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:55.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.293+0000 7f1d22e1a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d1c0caf80 0x7f1d1c15d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:55.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.293+0000 7f1d22e1a640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d1c15de50 con 0x7f1d1c0c5370 2023-11-09T03:48:55.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.293+0000 7f1d22e1a640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d1c15db80 con 0x7f1d1c0c4f90 2023-11-09T03:48:55.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.293+0000 7f1d22e1a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c0ca1c0 con 0x7f1d1c0caf80 2023-11-09T03:48:55.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.294+0000 7f1d20bb6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1d1c0c4f90 0x7f1d1c15d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:55.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.294+0000 7f1d213b7640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f1d1c0c5370 0x7f1d1c0c3840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35868/0 (socket says 172.21.15.76:35868) 2023-11-09T03:48:55.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.294+0000 7f1d213b7640 1 -- 172.21.15.76:0/4029256709 learned_addr learned my addr 172.21.15.76:0/4029256709 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:55.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.294+0000 7f1d1bfff640 1 --2- 172.21.15.76:0/4029256709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d1c0caf80 0x7f1d1c15d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:55.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.294+0000 7f1d20bb6640 1 -- 172.21.15.76:0/4029256709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d1c0caf80 msgr2=0x7f1d1c15d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:55.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.294+0000 7f1d20bb6640 1 --2- 172.21.15.76:0/4029256709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d1c0caf80 0x7f1d1c15d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:55.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.294+0000 7f1d20bb6640 1 -- 172.21.15.76:0/4029256709 >> v1:172.21.15.76:6789/0 conn(0x7f1d1c0c5370 legacy=0x7f1d1c0c3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:55.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.294+0000 7f1d20bb6640 1 -- 172.21.15.76:0/4029256709 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1c0ca4a0 con 0x7f1d1c0c4f90 2023-11-09T03:48:55.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.294+0000 7f1d1b7fe640 1 -- 172.21.15.76:0/4029256709 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 592358140 0 0) 0x7f1d1c15de50 con 0x7f1d1c0c5370 2023-11-09T03:48:55.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.295+0000 7f1d20bb6640 1 --2- 172.21.15.76:0/4029256709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d1c0c4f90 0x7f1d1c15d100 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c0097a0 tx=0x7f1d0c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67dd0d60a3bff22d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:55.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.295+0000 7f1d1b7fe640 1 -- 172.21.15.76:0/4029256709 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c02e070 con 0x7f1d1c0c4f90 2023-11-09T03:48:55.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.295+0000 7f1d1b7fe640 1 -- 172.21.15.76:0/4029256709 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0c0315a0 con 0x7f1d1c0c4f90 2023-11-09T03:48:55.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.295+0000 7f1d1b7fe640 1 -- 172.21.15.76:0/4029256709 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c031930 con 0x7f1d1c0c4f90 2023-11-09T03:48:55.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.295+0000 7f1d22e1a640 1 -- 172.21.15.76:0/4029256709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d1c0c4f90 msgr2=0x7f1d1c15d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:55.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.295+0000 7f1d22e1a640 1 --2- 172.21.15.76:0/4029256709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d1c0c4f90 0x7f1d1c15d100 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c0097a0 tx=0x7f1d0c02b840 comp rx=0 tx=0).stop 2023-11-09T03:48:55.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.295+0000 7f1d22e1a640 1 -- 172.21.15.76:0/4029256709 shutdown_connections 2023-11-09T03:48:55.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.295+0000 7f1d22e1a640 1 --2- 172.21.15.76:0/4029256709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d1c0caf80 0x7f1d1c15d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:55.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.295+0000 7f1d22e1a640 1 --2- 172.21.15.76:0/4029256709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d1c0c4f90 0x7f1d1c15d100 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:55.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.295+0000 7f1d22e1a640 1 -- 172.21.15.76:0/4029256709 >> 172.21.15.76:0/4029256709 conn(0x7f1d1c0b05d0 msgr2=0x7f1d1c0b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:55.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.295+0000 7f1d22e1a640 1 -- 172.21.15.76:0/4029256709 shutdown_connections 2023-11-09T03:48:55.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.295+0000 7f1d22e1a640 1 -- 172.21.15.76:0/4029256709 wait complete. 2023-11-09T03:48:55.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.296+0000 7f1d22e1a640 1 Processor -- start 2023-11-09T03:48:55.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.297+0000 7f1d22e1a640 1 -- start start 2023-11-09T03:48:55.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.297+0000 7f1d22e1a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d1c0c4f90 0x7f1d1c13d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:55.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.297+0000 7f1d22e1a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d1c0c5370 0x7f1d1c0b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:55.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.297+0000 7f1d22e1a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d1c0caf80 0x7f1d1c0b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:55.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.297+0000 7f1d22e1a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c157560 con 0x7f1d1c0c5370 2023-11-09T03:48:55.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.297+0000 7f1d22e1a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c0b1b10 con 0x7f1d1c0c4f90 2023-11-09T03:48:55.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.297+0000 7f1d22e1a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c156fa0 con 0x7f1d1c0caf80 2023-11-09T03:48:55.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.297+0000 7f1d1bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d1c0c5370 0x7f1d1c0b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:55.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.297+0000 7f1d1bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d1c0c5370 0x7f1d1c0b1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44060/0 (socket says 172.21.15.76:44060) 2023-11-09T03:48:55.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.297+0000 7f1d1bfff640 1 -- 172.21.15.76:0/1547818344 learned_addr learned my addr 172.21.15.76:0/1547818344 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:55.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.298+0000 7f1d213b7640 1 --2- 172.21.15.76:0/1547818344 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d1c0caf80 0x7f1d1c0b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:55.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.298+0000 7f1d1bfff640 1 -- 172.21.15.76:0/1547818344 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d1c0caf80 msgr2=0x7f1d1c0b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:55.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.298+0000 7f1d20bb6640 1 --2- 172.21.15.76:0/1547818344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d1c0c4f90 0x7f1d1c13d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:55.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.298+0000 7f1d1bfff640 1 --2- 172.21.15.76:0/1547818344 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d1c0caf80 0x7f1d1c0b2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:55.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.298+0000 7f1d1bfff640 1 -- 172.21.15.76:0/1547818344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d1c0c4f90 msgr2=0x7f1d1c13d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:55.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.298+0000 7f1d1bfff640 1 --2- 172.21.15.76:0/1547818344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d1c0c4f90 0x7f1d1c13d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:55.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.298+0000 7f1d1bfff640 1 -- 172.21.15.76:0/1547818344 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0c009400 con 0x7f1d1c0c5370 2023-11-09T03:48:55.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.298+0000 7f1d213b7640 1 --2- 172.21.15.76:0/1547818344 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d1c0caf80 0x7f1d1c0b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:55.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.298+0000 7f1d1bfff640 1 --2- 172.21.15.76:0/1547818344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d1c0c5370 0x7f1d1c0b1ef0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f1d0400d5f0 tx=0x7f1d0400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:55.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.298+0000 7f1d197fa640 1 -- 172.21.15.76:0/1547818344 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d040125f0 con 0x7f1d1c0c5370 2023-11-09T03:48:55.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.298+0000 7f1d197fa640 1 -- 172.21.15.76:0/1547818344 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d04012750 con 0x7f1d1c0c5370 2023-11-09T03:48:55.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.298+0000 7f1d197fa640 1 -- 172.21.15.76:0/1547818344 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d04012b00 con 0x7f1d1c0c5370 2023-11-09T03:48:55.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.298+0000 7f1d22e1a640 1 -- 172.21.15.76:0/1547818344 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d1c157280 con 0x7f1d1c0c5370 2023-11-09T03:48:55.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.298+0000 7f1d22e1a640 1 -- 172.21.15.76:0/1547818344 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d1c13d8a0 con 0x7f1d1c0c5370 2023-11-09T03:48:55.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.299+0000 7f1d22e1a640 1 -- 172.21.15.76:0/1547818344 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d1c0b4050 con 0x7f1d1c0c5370 2023-11-09T03:48:55.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.300+0000 7f1d197fa640 1 -- 172.21.15.76:0/1547818344 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1d04012c60 con 0x7f1d1c0c5370 2023-11-09T03:48:55.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.300+0000 7f1d197fa640 1 --2- 172.21.15.76:0/1547818344 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1cfc06aea0 0x7f1cfc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:55.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.300+0000 7f1d197fa640 1 -- 172.21.15.76:0/1547818344 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f1d04089c40 con 0x7f1d1c0c5370 2023-11-09T03:48:55.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.300+0000 7f1d20bb6640 1 --2- 172.21.15.76:0/1547818344 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1cfc06aea0 0x7f1cfc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:55.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.301+0000 7f1d20bb6640 1 --2- 172.21.15.76:0/1547818344 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1cfc06aea0 0x7f1cfc06d360 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c036000 tx=0x7f1d0c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:55.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.304+0000 7f1d197fa640 1 -- 172.21.15.76:0/1547818344 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1d1c0b4050 con 0x7f1d1c0c5370 2023-11-09T03:48:55.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:55.471+0000 7f1d22e1a640 1 -- 172.21.15.76:0/1547818344 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1d1c0b4050 con 0x7f1d1c0c5370 2023-11-09T03:48:56.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.027+0000 7f1d197fa640 1 -- 172.21.15.76:0/1547818344 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v92) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1d1c0b4050 con 0x7f1d1c0c5370 2023-11-09T03:48:56.028 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' created 2023-11-09T03:48:56.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.030+0000 7f1d22e1a640 1 -- 172.21.15.76:0/1547818344 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1cfc06aea0 msgr2=0x7f1cfc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:56.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.030+0000 7f1d22e1a640 1 --2- 172.21.15.76:0/1547818344 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1cfc06aea0 0x7f1cfc06d360 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c036000 tx=0x7f1d0c008030 comp rx=0 tx=0).stop 2023-11-09T03:48:56.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.030+0000 7f1d22e1a640 1 -- 172.21.15.76:0/1547818344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d1c0c5370 msgr2=0x7f1d1c0b1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:56.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.030+0000 7f1d22e1a640 1 --2- 172.21.15.76:0/1547818344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d1c0c5370 0x7f1d1c0b1ef0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f1d0400d5f0 tx=0x7f1d0400dab0 comp rx=0 tx=0).stop 2023-11-09T03:48:56.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.031+0000 7f1d22e1a640 1 -- 172.21.15.76:0/1547818344 shutdown_connections 2023-11-09T03:48:56.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.031+0000 7f1d22e1a640 1 --2- 172.21.15.76:0/1547818344 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1cfc06aea0 0x7f1cfc06d360 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:56.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.031+0000 7f1d22e1a640 1 --2- 172.21.15.76:0/1547818344 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d1c0caf80 0x7f1d1c0b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:56.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.031+0000 7f1d22e1a640 1 --2- 172.21.15.76:0/1547818344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d1c0c5370 0x7f1d1c0b1ef0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:56.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.031+0000 7f1d22e1a640 1 --2- 172.21.15.76:0/1547818344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d1c0c4f90 0x7f1d1c13d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:56.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.031+0000 7f1d22e1a640 1 -- 172.21.15.76:0/1547818344 >> 172.21.15.76:0/1547818344 conn(0x7f1d1c0b05d0 msgr2=0x7f1d1c0cc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:56.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.031+0000 7f1d22e1a640 1 -- 172.21.15.76:0/1547818344 shutdown_connections 2023-11-09T03:48:56.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.031+0000 7f1d22e1a640 1 -- 172.21.15.76:0/1547818344 wait complete. 2023-11-09T03:48:56.047 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-09T03:48:56.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.155+0000 7f2dc65b6640 1 Processor -- start 2023-11-09T03:48:56.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.155+0000 7f2dc65b6640 1 -- start start 2023-11-09T03:48:56.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.156+0000 7f2dc65b6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dc0153850 0x7f2dc015a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:56.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.156+0000 7f2dc65b6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2dc00cc9a0 0x7f2dc00c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:56.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.156+0000 7f2dc65b6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc00c3860 con 0x7f2dc00cc520 2023-11-09T03:48:56.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.156+0000 7f2dc65b6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dc00c9250 con 0x7f2dc00cc9a0 2023-11-09T03:48:56.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.156+0000 7f2dc65b6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc0153c30 con 0x7f2dc0153850 2023-11-09T03:48:56.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.157+0000 7f2dbffff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dc0153850 0x7f2dc015a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:56.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.157+0000 7f2dc4b53640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2dc00cc9a0 0x7f2dc00c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:56.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.157+0000 7f2dbffff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dc0153850 0x7f2dc015a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:56314/0 (socket says 172.21.15.76:56314) 2023-11-09T03:48:56.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.157+0000 7f2dbffff640 1 -- 172.21.15.76:0/2745859187 learned_addr learned my addr 172.21.15.76:0/2745859187 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:56.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.157+0000 7f2dbffff640 1 -- 172.21.15.76:0/2745859187 >> v2:172.21.15.113:3300/0 conn(0x7f2dc00cc9a0 msgr2=0x7f2dc00c3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:56.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.157+0000 7f2dbffff640 1 --2- 172.21.15.76:0/2745859187 >> v2:172.21.15.113:3300/0 conn(0x7f2dc00cc9a0 0x7f2dc00c3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:56.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.157+0000 7f2dbffff640 1 -- 172.21.15.76:0/2745859187 >> v1:172.21.15.113:6789/0 conn(0x7f2dc00cc520 legacy=0x7f2dc0070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:56.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.157+0000 7f2dbffff640 1 -- 172.21.15.76:0/2745859187 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc015a700 con 0x7f2dc0153850 2023-11-09T03:48:56.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.157+0000 7f2dc4b53640 1 --2- 172.21.15.76:0/2745859187 >> v2:172.21.15.113:3300/0 conn(0x7f2dc00cc9a0 0x7f2dc00c3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:48:56.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.158+0000 7f2dbffff640 1 --2- 172.21.15.76:0/2745859187 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dc0153850 0x7f2dc015a1c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2db0009870 tx=0x7f2db002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b23afaac33553dba server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:56.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.158+0000 7f2dbeffd640 1 -- 172.21.15.76:0/2745859187 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db0030840 con 0x7f2dc0153850 2023-11-09T03:48:56.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.158+0000 7f2dbeffd640 1 -- 172.21.15.76:0/2745859187 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db00309a0 con 0x7f2dc0153850 2023-11-09T03:48:56.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.158+0000 7f2dbeffd640 1 -- 172.21.15.76:0/2745859187 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db0030d30 con 0x7f2dc0153850 2023-11-09T03:48:56.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.158+0000 7f2dc65b6640 1 -- 172.21.15.76:0/2745859187 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dc0153850 msgr2=0x7f2dc015a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:56.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.158+0000 7f2dc65b6640 1 --2- 172.21.15.76:0/2745859187 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dc0153850 0x7f2dc015a1c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2db0009870 tx=0x7f2db002b980 comp rx=0 tx=0).stop 2023-11-09T03:48:56.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.159+0000 7f2dc65b6640 1 -- 172.21.15.76:0/2745859187 shutdown_connections 2023-11-09T03:48:56.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.159+0000 7f2dc65b6640 1 --2- 172.21.15.76:0/2745859187 >> v2:172.21.15.113:3300/0 conn(0x7f2dc00cc9a0 0x7f2dc00c3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:56.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.159+0000 7f2dc65b6640 1 --2- 172.21.15.76:0/2745859187 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dc0153850 0x7f2dc015a1c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:56.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.159+0000 7f2dc65b6640 1 -- 172.21.15.76:0/2745859187 >> 172.21.15.76:0/2745859187 conn(0x7f2dc00ce950 msgr2=0x7f2dc00ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:56.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.159+0000 7f2dc65b6640 1 -- 172.21.15.76:0/2745859187 shutdown_connections 2023-11-09T03:48:56.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.159+0000 7f2dc65b6640 1 -- 172.21.15.76:0/2745859187 wait complete. 2023-11-09T03:48:56.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.160+0000 7f2dc65b6640 1 Processor -- start 2023-11-09T03:48:56.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.160+0000 7f2dc65b6640 1 -- start start 2023-11-09T03:48:56.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.161+0000 7f2dc65b6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dc00cc520 0x7f2dc01f0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:56.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.161+0000 7f2dc65b6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dc00cc9a0 0x7f2dc0155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:56.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.161+0000 7f2dc65b6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2dc0153850 0x7f2dc0151250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:56.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.161+0000 7f2dc65b6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc0156520 con 0x7f2dc0153850 2023-11-09T03:48:56.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.161+0000 7f2dc65b6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc0156800 con 0x7f2dc00cc9a0 2023-11-09T03:48:56.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.161+0000 7f2dc65b6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc00b15d0 con 0x7f2dc00cc520 2023-11-09T03:48:56.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.161+0000 7f2dc4b53640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2dc0153850 0x7f2dc0151250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:56.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.161+0000 7f2dc4b53640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2dc0153850 0x7f2dc0151250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44064/0 (socket says 172.21.15.76:44064) 2023-11-09T03:48:56.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.161+0000 7f2dc4b53640 1 -- 172.21.15.76:0/1348755303 learned_addr learned my addr 172.21.15.76:0/1348755303 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:56.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.161+0000 7f2dbffff640 1 --2- 172.21.15.76:0/1348755303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dc00cc520 0x7f2dc01f0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:56.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.161+0000 7f2dbf7fe640 1 --2- 172.21.15.76:0/1348755303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dc00cc9a0 0x7f2dc0155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:56.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.161+0000 7f2dc4b53640 1 -- 172.21.15.76:0/1348755303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dc00cc520 msgr2=0x7f2dc01f0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:56.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.161+0000 7f2dc4b53640 1 --2- 172.21.15.76:0/1348755303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dc00cc520 0x7f2dc01f0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:56.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.161+0000 7f2dc4b53640 1 -- 172.21.15.76:0/1348755303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dc00cc9a0 msgr2=0x7f2dc0155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:56.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.161+0000 7f2dc4b53640 1 --2- 172.21.15.76:0/1348755303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dc00cc9a0 0x7f2dc0155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:56.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.161+0000 7f2dc4b53640 1 -- 172.21.15.76:0/1348755303 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db800ab80 con 0x7f2dc0153850 2023-11-09T03:48:56.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.161+0000 7f2dc4b53640 1 --2- 172.21.15.76:0/1348755303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2dc0153850 0x7f2dc0151250 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f2db8009f00 tx=0x7f2db80075c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:56.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.161+0000 7f2dbcff9640 1 -- 172.21.15.76:0/1348755303 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db80145f0 con 0x7f2dc0153850 2023-11-09T03:48:56.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.161+0000 7f2dbcff9640 1 -- 172.21.15.76:0/1348755303 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db8010040 con 0x7f2dc0153850 2023-11-09T03:48:56.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.161+0000 7f2dc65b6640 1 -- 172.21.15.76:0/1348755303 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db00094d0 con 0x7f2dc0153850 2023-11-09T03:48:56.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.161+0000 7f2dbcff9640 1 -- 172.21.15.76:0/1348755303 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db8014990 con 0x7f2dc0153850 2023-11-09T03:48:56.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.161+0000 7f2dc65b6640 1 -- 172.21.15.76:0/1348755303 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc0156230 con 0x7f2dc0153850 2023-11-09T03:48:56.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.162+0000 7f2dc65b6640 1 -- 172.21.15.76:0/1348755303 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dc0074290 con 0x7f2dc0153850 2023-11-09T03:48:56.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.163+0000 7f2dbcff9640 1 -- 172.21.15.76:0/1348755303 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2db8014af0 con 0x7f2dc0153850 2023-11-09T03:48:56.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.163+0000 7f2dbcff9640 1 --2- 172.21.15.76:0/1348755303 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2da006ae70 0x7f2da006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:56.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.163+0000 7f2dbcff9640 1 -- 172.21.15.76:0/1348755303 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6638+0+0 (secure 0 0 0) 0x7f2db8089f90 con 0x7f2dc0153850 2023-11-09T03:48:56.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.163+0000 7f2dbffff640 1 --2- 172.21.15.76:0/1348755303 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2da006ae70 0x7f2da006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:56.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.164+0000 7f2dbffff640 1 --2- 172.21.15.76:0/1348755303 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2da006ae70 0x7f2da006d330 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2db0036000 tx=0x7f2db0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:56.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.168+0000 7f2dbcff9640 1 -- 172.21.15.76:0/1348755303 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2db80bcca0 con 0x7f2dc0153850 2023-11-09T03:48:56.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:56.333+0000 7f2dc65b6640 1 -- 172.21.15.76:0/1348755303 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2dc00c5010 con 0x7f2dc0153850 2023-11-09T03:48:57.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.029+0000 7f2dbcff9640 1 -- 172.21.15.76:0/1348755303 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v93) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2db8058870 con 0x7f2dc0153850 2023-11-09T03:48:57.030 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' created 2023-11-09T03:48:57.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.032+0000 7f2dc65b6640 1 -- 172.21.15.76:0/1348755303 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2da006ae70 msgr2=0x7f2da006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:57.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.032+0000 7f2dc65b6640 1 --2- 172.21.15.76:0/1348755303 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2da006ae70 0x7f2da006d330 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2db0036000 tx=0x7f2db0008030 comp rx=0 tx=0).stop 2023-11-09T03:48:57.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.032+0000 7f2dc65b6640 1 -- 172.21.15.76:0/1348755303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2dc0153850 msgr2=0x7f2dc0151250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:57.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.032+0000 7f2dc65b6640 1 --2- 172.21.15.76:0/1348755303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2dc0153850 0x7f2dc0151250 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f2db8009f00 tx=0x7f2db80075c0 comp rx=0 tx=0).stop 2023-11-09T03:48:57.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.033+0000 7f2dc65b6640 1 -- 172.21.15.76:0/1348755303 shutdown_connections 2023-11-09T03:48:57.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.033+0000 7f2dc65b6640 1 --2- 172.21.15.76:0/1348755303 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2da006ae70 0x7f2da006d330 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:57.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.033+0000 7f2dc65b6640 1 --2- 172.21.15.76:0/1348755303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2dc0153850 0x7f2dc0151250 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:57.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.033+0000 7f2dc65b6640 1 --2- 172.21.15.76:0/1348755303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dc00cc9a0 0x7f2dc0155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:57.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.033+0000 7f2dc65b6640 1 --2- 172.21.15.76:0/1348755303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2dc00cc520 0x7f2dc01f0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:57.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.033+0000 7f2dc65b6640 1 -- 172.21.15.76:0/1348755303 >> 172.21.15.76:0/1348755303 conn(0x7f2dc00ce950 msgr2=0x7f2dc006d920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:57.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.036+0000 7f2dc65b6640 1 -- 172.21.15.76:0/1348755303 shutdown_connections 2023-11-09T03:48:57.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.040+0000 7f2dc65b6640 1 -- 172.21.15.76:0/1348755303 wait complete. 2023-11-09T03:48:57.064 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-09T03:48:57.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.169+0000 7f2fe601a640 1 Processor -- start 2023-11-09T03:48:57.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.169+0000 7f2fe601a640 1 -- start start 2023-11-09T03:48:57.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.170+0000 7f2fe601a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2fe0159c00 0x7f2fe00cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:57.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.170+0000 7f2fe601a640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe015aff0 con 0x7f2fe00c48e0 2023-11-09T03:48:57.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.170+0000 7f2fe601a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe015a3e0 con 0x7f2fe00ae1d0 2023-11-09T03:48:57.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.170+0000 7f2fe601a640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe015a6c0 con 0x7f2fe0159c00 2023-11-09T03:48:57.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.170+0000 7f2fdf7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2fe0159c00 0x7f2fe00cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:57.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.170+0000 7f2fdeffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2fe00c48e0 0x7f2fe00adac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35872/0 (socket says 172.21.15.76:35872) 2023-11-09T03:48:57.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.170+0000 7f2fdeffd640 1 -- 172.21.15.76:0/4127285808 learned_addr learned my addr 172.21.15.76:0/4127285808 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:57.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.170+0000 7f2fdf7fe640 1 -- 172.21.15.76:0/4127285808 >> v1:172.21.15.113:6789/0 conn(0x7f2fe00ae1d0 legacy=0x7f2fe00709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:57.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.170+0000 7f2fdf7fe640 1 -- 172.21.15.76:0/4127285808 >> v1:172.21.15.76:6789/0 conn(0x7f2fe00c48e0 legacy=0x7f2fe00adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:57.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.170+0000 7f2fdf7fe640 1 -- 172.21.15.76:0/4127285808 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe015a9a0 con 0x7f2fe0159c00 2023-11-09T03:48:57.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.170+0000 7f2fde7fc640 1 -- 172.21.15.76:0/4127285808 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1163793019 0 0) 0x7f2fe015aff0 con 0x7f2fe00c48e0 2023-11-09T03:48:57.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.171+0000 7f2fdf7fe640 1 --2- 172.21.15.76:0/4127285808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fe0159c00 0x7f2fe00cc890 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f2fd00097a0 tx=0x7f2fd002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a6678df71aff8f server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:57.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.171+0000 7f2fde7fc640 1 -- 172.21.15.76:0/4127285808 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0031830 con 0x7f2fe0159c00 2023-11-09T03:48:57.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.171+0000 7f2fde7fc640 1 -- 172.21.15.76:0/4127285808 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd0031990 con 0x7f2fe0159c00 2023-11-09T03:48:57.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.171+0000 7f2fde7fc640 1 -- 172.21.15.76:0/4127285808 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0031d20 con 0x7f2fe0159c00 2023-11-09T03:48:57.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.172+0000 7f2fe601a640 1 -- 172.21.15.76:0/4127285808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fe0159c00 msgr2=0x7f2fe00cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:57.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.172+0000 7f2fe601a640 1 --2- 172.21.15.76:0/4127285808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fe0159c00 0x7f2fe00cc890 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f2fd00097a0 tx=0x7f2fd002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:48:57.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.172+0000 7f2fe601a640 1 -- 172.21.15.76:0/4127285808 shutdown_connections 2023-11-09T03:48:57.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.172+0000 7f2fe601a640 1 --2- 172.21.15.76:0/4127285808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fe0159c00 0x7f2fe00cc890 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:57.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.172+0000 7f2fe601a640 1 -- 172.21.15.76:0/4127285808 >> 172.21.15.76:0/4127285808 conn(0x7f2fe00ce950 msgr2=0x7f2fe00ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:57.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.172+0000 7f2fe601a640 1 -- 172.21.15.76:0/4127285808 shutdown_connections 2023-11-09T03:48:57.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.172+0000 7f2fe601a640 1 -- 172.21.15.76:0/4127285808 wait complete. 2023-11-09T03:48:57.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.173+0000 7f2fe601a640 1 Processor -- start 2023-11-09T03:48:57.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.173+0000 7f2fe601a640 1 -- start start 2023-11-09T03:48:57.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.174+0000 7f2fe601a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fe00ae1d0 0x7f2fe0142340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:57.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.174+0000 7f2fe601a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fe00c48e0 0x7f2fe00c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:57.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.174+0000 7f2fe601a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2fe0159c00 0x7f2fe0151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:57.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.174+0000 7f2fe601a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe00c82b0 con 0x7f2fe00c48e0 2023-11-09T03:48:57.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.174+0000 7f2fe601a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe00c8740 con 0x7f2fe00ae1d0 2023-11-09T03:48:57.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.174+0000 7f2fe601a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe00c76a0 con 0x7f2fe0159c00 2023-11-09T03:48:57.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.174+0000 7f2fdeffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fe00c48e0 0x7f2fe00c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:57.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.174+0000 7f2fdeffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fe00c48e0 0x7f2fe00c6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44084/0 (socket says 172.21.15.76:44084) 2023-11-09T03:48:57.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.174+0000 7f2fdeffd640 1 -- 172.21.15.76:0/978373776 learned_addr learned my addr 172.21.15.76:0/978373776 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:57.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.174+0000 7f2fdf7fe640 1 --2- 172.21.15.76:0/978373776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fe00ae1d0 0x7f2fe0142340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:57.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.174+0000 7f2fdeffd640 1 -- 172.21.15.76:0/978373776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2fe0159c00 msgr2=0x7f2fe0151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:57.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.174+0000 7f2fdffff640 1 --2- 172.21.15.76:0/978373776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2fe0159c00 0x7f2fe0151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:57.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.174+0000 7f2fdeffd640 1 --2- 172.21.15.76:0/978373776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2fe0159c00 0x7f2fe0151cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:57.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.174+0000 7f2fdeffd640 1 -- 172.21.15.76:0/978373776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fe00ae1d0 msgr2=0x7f2fe0142340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:57.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.174+0000 7f2fdeffd640 1 --2- 172.21.15.76:0/978373776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fe00ae1d0 0x7f2fe0142340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:57.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.174+0000 7f2fdeffd640 1 -- 172.21.15.76:0/978373776 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd0009400 con 0x7f2fe00c48e0 2023-11-09T03:48:57.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.174+0000 7f2fdf7fe640 1 --2- 172.21.15.76:0/978373776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fe00ae1d0 0x7f2fe0142340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:48:57.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.174+0000 7f2fdeffd640 1 --2- 172.21.15.76:0/978373776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fe00c48e0 0x7f2fe00c6d50 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400c4a0 tx=0x7f2fd400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:57.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.174+0000 7f2fc7fff640 1 -- 172.21.15.76:0/978373776 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd40074e0 con 0x7f2fe00c48e0 2023-11-09T03:48:57.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.174+0000 7f2fc7fff640 1 -- 172.21.15.76:0/978373776 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd4007640 con 0x7f2fe00c48e0 2023-11-09T03:48:57.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.174+0000 7f2fc7fff640 1 -- 172.21.15.76:0/978373776 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4004a00 con 0x7f2fe00c48e0 2023-11-09T03:48:57.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.174+0000 7f2fe601a640 1 -- 172.21.15.76:0/978373776 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe00c7980 con 0x7f2fe00c48e0 2023-11-09T03:48:57.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.174+0000 7f2fe601a640 1 -- 172.21.15.76:0/978373776 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe00c7f40 con 0x7f2fe00c48e0 2023-11-09T03:48:57.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.175+0000 7f2fe601a640 1 -- 172.21.15.76:0/978373776 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe00c8420 con 0x7f2fe00c48e0 2023-11-09T03:48:57.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.176+0000 7f2fc7fff640 1 -- 172.21.15.76:0/978373776 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2fd4004b60 con 0x7f2fe00c48e0 2023-11-09T03:48:57.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.176+0000 7f2fc7fff640 1 --2- 172.21.15.76:0/978373776 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2fcc06ae70 0x7f2fcc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:57.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.177+0000 7f2fc7fff640 1 -- 172.21.15.76:0/978373776 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 7020+0+0 (secure 0 0 0) 0x7f2fd4088ca0 con 0x7f2fe00c48e0 2023-11-09T03:48:57.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.177+0000 7f2fdf7fe640 1 --2- 172.21.15.76:0/978373776 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2fcc06ae70 0x7f2fcc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:57.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.177+0000 7f2fdf7fe640 1 --2- 172.21.15.76:0/978373776 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2fcc06ae70 0x7f2fcc06d330 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0036000 tx=0x7f2fd0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:57.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.181+0000 7f2fc7fff640 1 -- 172.21.15.76:0/978373776 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2fd40538c0 con 0x7f2fe00c48e0 2023-11-09T03:48:57.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:57.381+0000 7f2fe601a640 1 -- 172.21.15.76:0/978373776 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2fe0074290 con 0x7f2fe00c48e0 2023-11-09T03:48:58.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.040+0000 7f2fc7fff640 1 -- 172.21.15.76:0/978373776 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v42) v1 ==== 174+0+0 (secure 0 0 0) 0x7f2fd4018020 con 0x7f2fe00c48e0 2023-11-09T03:48:58.041 INFO:teuthology.orchestra.run.smithi076.stderr:new fs with metadata pool 16 and data pool 17 2023-11-09T03:48:58.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.042+0000 7f2fe601a640 1 -- 172.21.15.76:0/978373776 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2fcc06ae70 msgr2=0x7f2fcc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:58.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.042+0000 7f2fe601a640 1 --2- 172.21.15.76:0/978373776 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2fcc06ae70 0x7f2fcc06d330 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0036000 tx=0x7f2fd0008030 comp rx=0 tx=0).stop 2023-11-09T03:48:58.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.042+0000 7f2fe601a640 1 -- 172.21.15.76:0/978373776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fe00c48e0 msgr2=0x7f2fe00c6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:58.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.042+0000 7f2fe601a640 1 --2- 172.21.15.76:0/978373776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fe00c48e0 0x7f2fe00c6d50 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400c4a0 tx=0x7f2fd400c960 comp rx=0 tx=0).stop 2023-11-09T03:48:58.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.043+0000 7f2fe601a640 1 -- 172.21.15.76:0/978373776 shutdown_connections 2023-11-09T03:48:58.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.043+0000 7f2fe601a640 1 --2- 172.21.15.76:0/978373776 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2fcc06ae70 0x7f2fcc06d330 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.043+0000 7f2fe601a640 1 --2- 172.21.15.76:0/978373776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2fe0159c00 0x7f2fe0151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.043+0000 7f2fe601a640 1 --2- 172.21.15.76:0/978373776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fe00c48e0 0x7f2fe00c6d50 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.043+0000 7f2fe601a640 1 --2- 172.21.15.76:0/978373776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fe00ae1d0 0x7f2fe0142340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.043+0000 7f2fe601a640 1 -- 172.21.15.76:0/978373776 >> 172.21.15.76:0/978373776 conn(0x7f2fe00ce950 msgr2=0x7f2fe006de80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:58.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.043+0000 7f2fe601a640 1 -- 172.21.15.76:0/978373776 shutdown_connections 2023-11-09T03:48:58.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.043+0000 7f2fe601a640 1 -- 172.21.15.76:0/978373776 wait complete. 2023-11-09T03:48:58.061 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:48:58.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.165+0000 7f7324d61640 1 Processor -- start 2023-11-09T03:48:58.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.165+0000 7f7324d61640 1 -- start start 2023-11-09T03:48:58.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.166+0000 7f7324d61640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f73200b0ae0 0x7f73200ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:58.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.166+0000 7f7324d61640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f732006db20 0x7f732015d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:58.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.166+0000 7f7324d61640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73200b4110 con 0x7f7320159c70 2023-11-09T03:48:58.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.166+0000 7f7324d61640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73200b49b0 con 0x7f73200b0ae0 2023-11-09T03:48:58.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.166+0000 7f7324d61640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73200b43f0 con 0x7f732006db20 2023-11-09T03:48:58.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.166+0000 7f731dd9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f73200b0ae0 0x7f73200ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:58.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.166+0000 7f731dd9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f73200b0ae0 0x7f73200ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44094/0 (socket says 172.21.15.76:44094) 2023-11-09T03:48:58.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.166+0000 7f731dd9b640 1 -- 172.21.15.76:0/3034260346 learned_addr learned my addr 172.21.15.76:0/3034260346 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:58.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.166+0000 7f731ed9d640 1 --2- 172.21.15.76:0/3034260346 >> v2:172.21.15.113:3300/0 conn(0x7f732006db20 0x7f732015d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:58.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.166+0000 7f731dd9b640 1 -- 172.21.15.76:0/3034260346 >> v2:172.21.15.113:3300/0 conn(0x7f732006db20 msgr2=0x7f732015d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:58.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.166+0000 7f731dd9b640 1 --2- 172.21.15.76:0/3034260346 >> v2:172.21.15.113:3300/0 conn(0x7f732006db20 0x7f732015d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.166+0000 7f731dd9b640 1 -- 172.21.15.76:0/3034260346 >> v1:172.21.15.113:6789/0 conn(0x7f7320159c70 legacy=0x7f73200b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:58.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.166+0000 7f731dd9b640 1 -- 172.21.15.76:0/3034260346 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73200b46d0 con 0x7f73200b0ae0 2023-11-09T03:48:58.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.167+0000 7f731dd9b640 1 --2- 172.21.15.76:0/3034260346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73200b0ae0 0x7f73200ce370 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f7320070580 tx=0x7f730802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b71467cc5d27010 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:58.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.167+0000 7f731d59a640 1 -- 172.21.15.76:0/3034260346 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7308031870 con 0x7f73200b0ae0 2023-11-09T03:48:58.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.167+0000 7f731d59a640 1 -- 172.21.15.76:0/3034260346 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73080319d0 con 0x7f73200b0ae0 2023-11-09T03:48:58.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.167+0000 7f731d59a640 1 -- 172.21.15.76:0/3034260346 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7308031d60 con 0x7f73200b0ae0 2023-11-09T03:48:58.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.167+0000 7f7324d61640 1 -- 172.21.15.76:0/3034260346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73200b0ae0 msgr2=0x7f73200ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:58.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.167+0000 7f7324d61640 1 --2- 172.21.15.76:0/3034260346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73200b0ae0 0x7f73200ce370 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f7320070580 tx=0x7f730802b910 comp rx=0 tx=0).stop 2023-11-09T03:48:58.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.167+0000 7f7324d61640 1 -- 172.21.15.76:0/3034260346 shutdown_connections 2023-11-09T03:48:58.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.168+0000 7f7324d61640 1 --2- 172.21.15.76:0/3034260346 >> v2:172.21.15.113:3300/0 conn(0x7f732006db20 0x7f732015d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.168+0000 7f7324d61640 1 --2- 172.21.15.76:0/3034260346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73200b0ae0 0x7f73200ce370 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.168+0000 7f7324d61640 1 -- 172.21.15.76:0/3034260346 >> 172.21.15.76:0/3034260346 conn(0x7f73200ce950 msgr2=0x7f73200ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:58.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.168+0000 7f7324d61640 1 -- 172.21.15.76:0/3034260346 shutdown_connections 2023-11-09T03:48:58.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.168+0000 7f7324d61640 1 -- 172.21.15.76:0/3034260346 wait complete. 2023-11-09T03:48:58.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.169+0000 7f7324d61640 1 Processor -- start 2023-11-09T03:48:58.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.169+0000 7f7324d61640 1 -- start start 2023-11-09T03:48:58.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.170+0000 7f7324d61640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f732006db20 0x7f732014f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:58.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.170+0000 7f7324d61640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73200b0ae0 0x7f732014f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:58.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.170+0000 7f7324d61640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7320159c70 0x7f7320152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:58.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.170+0000 7f7324d61640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7320151c50 con 0x7f73200b0ae0 2023-11-09T03:48:58.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.170+0000 7f7324d61640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7320156370 con 0x7f732006db20 2023-11-09T03:48:58.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.170+0000 7f7324d61640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73201512f0 con 0x7f7320159c70 2023-11-09T03:48:58.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.170+0000 7f731dd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73200b0ae0 0x7f732014f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:58.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.170+0000 7f731dd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73200b0ae0 0x7f732014f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44108/0 (socket says 172.21.15.76:44108) 2023-11-09T03:48:58.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.170+0000 7f731dd9b640 1 -- 172.21.15.76:0/1829081306 learned_addr learned my addr 172.21.15.76:0/1829081306 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:58.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.170+0000 7f731e59c640 1 --2- 172.21.15.76:0/1829081306 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f732006db20 0x7f732014f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:58.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.170+0000 7f731ed9d640 1 --2- 172.21.15.76:0/1829081306 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7320159c70 0x7f7320152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:58.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.170+0000 7f731dd9b640 1 -- 172.21.15.76:0/1829081306 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7320159c70 msgr2=0x7f7320152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:58.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.170+0000 7f731dd9b640 1 --2- 172.21.15.76:0/1829081306 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7320159c70 0x7f7320152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.170+0000 7f731dd9b640 1 -- 172.21.15.76:0/1829081306 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f732006db20 msgr2=0x7f732014f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:58.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.170+0000 7f731dd9b640 1 --2- 172.21.15.76:0/1829081306 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f732006db20 0x7f732014f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.170+0000 7f731dd9b640 1 -- 172.21.15.76:0/1829081306 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73080094d0 con 0x7f73200b0ae0 2023-11-09T03:48:58.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.170+0000 7f731dd9b640 1 --2- 172.21.15.76:0/1829081306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73200b0ae0 0x7f732014f2d0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f730802b950 tx=0x7f73080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:58.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.170+0000 7f7316ffd640 1 -- 172.21.15.76:0/1829081306 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730803abe0 con 0x7f73200b0ae0 2023-11-09T03:48:58.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.170+0000 7f7316ffd640 1 -- 172.21.15.76:0/1829081306 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730803ad40 con 0x7f73200b0ae0 2023-11-09T03:48:58.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.170+0000 7f7316ffd640 1 -- 172.21.15.76:0/1829081306 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7308044610 con 0x7f73200b0ae0 2023-11-09T03:48:58.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.170+0000 7f7324d61640 1 -- 172.21.15.76:0/1829081306 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7320151590 con 0x7f73200b0ae0 2023-11-09T03:48:58.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.171+0000 7f7324d61640 1 -- 172.21.15.76:0/1829081306 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7320155b20 con 0x7f73200b0ae0 2023-11-09T03:48:58.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.171+0000 7f7324d61640 1 -- 172.21.15.76:0/1829081306 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7320074290 con 0x7f73200b0ae0 2023-11-09T03:48:58.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.172+0000 7f7316ffd640 1 -- 172.21.15.76:0/1829081306 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7308044770 con 0x7f73200b0ae0 2023-11-09T03:48:58.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.172+0000 7f7316ffd640 1 --2- 172.21.15.76:0/1829081306 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f730006ae70 0x7f730006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:58.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.172+0000 7f7316ffd640 1 -- 172.21.15.76:0/1829081306 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f730802d040 con 0x7f73200b0ae0 2023-11-09T03:48:58.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.173+0000 7f731e59c640 1 --2- 172.21.15.76:0/1829081306 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f730006ae70 0x7f730006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:58.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.173+0000 7f731e59c640 1 --2- 172.21.15.76:0/1829081306 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f730006ae70 0x7f730006d330 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f73201537d0 tx=0x7f7310007e30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:58.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.177+0000 7f7316ffd640 1 -- 172.21.15.76:0/1829081306 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7308076d80 con 0x7f73200b0ae0 2023-11-09T03:48:58.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.344+0000 7f7324d61640 1 -- 172.21.15.76:0/1829081306 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f732006d3f0 con 0x7f73200b0ae0 2023-11-09T03:48:58.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.346+0000 7f7316ffd640 1 -- 172.21.15.76:0/1829081306 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+22006 (secure 0 0 0) 0x7f730807a3a0 con 0x7f73200b0ae0 2023-11-09T03:48:58.346 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:58.347 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":94,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:48:57.383531+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-09T03:47:59.280601+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-09T03:47:59.742039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:48:55.490832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-09T03:48:56.352624+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:48:58.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.347+0000 7f7324d61640 1 -- 172.21.15.76:0/1829081306 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f730006ae70 msgr2=0x7f730006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:58.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.347+0000 7f7324d61640 1 --2- 172.21.15.76:0/1829081306 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f730006ae70 0x7f730006d330 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f73201537d0 tx=0x7f7310007e30 comp rx=0 tx=0).stop 2023-11-09T03:48:58.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.348+0000 7f7324d61640 1 -- 172.21.15.76:0/1829081306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73200b0ae0 msgr2=0x7f732014f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:58.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.348+0000 7f7324d61640 1 --2- 172.21.15.76:0/1829081306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73200b0ae0 0x7f732014f2d0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f730802b950 tx=0x7f73080046c0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.348+0000 7f7324d61640 1 -- 172.21.15.76:0/1829081306 shutdown_connections 2023-11-09T03:48:58.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.348+0000 7f7324d61640 1 --2- 172.21.15.76:0/1829081306 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f730006ae70 0x7f730006d330 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.348+0000 7f7324d61640 1 --2- 172.21.15.76:0/1829081306 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7320159c70 0x7f7320152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.348+0000 7f7324d61640 1 --2- 172.21.15.76:0/1829081306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73200b0ae0 0x7f732014f2d0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.348+0000 7f7324d61640 1 --2- 172.21.15.76:0/1829081306 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f732006db20 0x7f732014f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.348+0000 7f7324d61640 1 -- 172.21.15.76:0/1829081306 >> 172.21.15.76:0/1829081306 conn(0x7f73200ce950 msgr2=0x7f73200cb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:58.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.348+0000 7f7324d61640 1 -- 172.21.15.76:0/1829081306 shutdown_connections 2023-11-09T03:48:58.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.348+0000 7f7324d61640 1 -- 172.21.15.76:0/1829081306 wait complete. 2023-11-09T03:48:58.359 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:48:58.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.459+0000 7f7289516640 1 Processor -- start 2023-11-09T03:48:58.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.459+0000 7f7289516640 1 -- start start 2023-11-09T03:48:58.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.460+0000 7f7289516640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72840ce240 0x7f7284157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:58.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.460+0000 7f7289516640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7284157540 0x7f72840d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:58.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.460+0000 7f7289516640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f728415e000 con 0x7f72840cde60 2023-11-09T03:48:58.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.460+0000 7f7289516640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f728415e5c0 con 0x7f7284157540 2023-11-09T03:48:58.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.460+0000 7f7289516640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728415e2e0 con 0x7f72840ce240 2023-11-09T03:48:58.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.460+0000 7f7282ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f72840cde60 0x7f728415d560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35888/0 (socket says 172.21.15.76:35888) 2023-11-09T03:48:58.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.460+0000 7f7282ffd640 1 -- 172.21.15.76:0/3297178788 learned_addr learned my addr 172.21.15.76:0/3297178788 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:48:58.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.460+0000 7f727bfff640 1 --2- 172.21.15.76:0/3297178788 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72840ce240 0x7f7284157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:58.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.460+0000 7f72837fe640 1 --2- 172.21.15.76:0/3297178788 >> v2:172.21.15.113:3300/0 conn(0x7f7284157540 0x7f72840d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:58.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.460+0000 7f72827fc640 1 -- 172.21.15.76:0/3297178788 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 301173827 0 0) 0x7f728415e000 con 0x7f72840cde60 2023-11-09T03:48:58.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.460+0000 7f72827fc640 1 -- 172.21.15.76:0/3297178788 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7270003520 con 0x7f72840cde60 2023-11-09T03:48:58.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.460+0000 7f72827fc640 1 -- 172.21.15.76:0/3297178788 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 34662300 0 0) 0x7f7270003520 con 0x7f72840cde60 2023-11-09T03:48:58.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.460+0000 7f727bfff640 1 -- 172.21.15.76:0/3297178788 >> v2:172.21.15.113:3300/0 conn(0x7f7284157540 msgr2=0x7f72840d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:58.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.460+0000 7f727bfff640 1 --2- 172.21.15.76:0/3297178788 >> v2:172.21.15.113:3300/0 conn(0x7f7284157540 0x7f72840d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.460+0000 7f727bfff640 1 -- 172.21.15.76:0/3297178788 >> v1:172.21.15.76:6789/0 conn(0x7f72840cde60 legacy=0x7f728415d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:58.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.460+0000 7f727bfff640 1 -- 172.21.15.76:0/3297178788 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72840d1120 con 0x7f72840ce240 2023-11-09T03:48:58.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.460+0000 7f72837fe640 1 --2- 172.21.15.76:0/3297178788 >> v2:172.21.15.113:3300/0 conn(0x7f7284157540 0x7f72840d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:48:58.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.461+0000 7f727bfff640 1 --2- 172.21.15.76:0/3297178788 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72840ce240 0x7f7284157000 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f726c0097a0 tx=0x7f726c02b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adaeecfa37d72d9c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:58.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.462+0000 7f72827fc640 1 -- 172.21.15.76:0/3297178788 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7274001670 con 0x7f72840ce240 2023-11-09T03:48:58.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.462+0000 7f72827fc640 1 -- 172.21.15.76:0/3297178788 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726c02e050 con 0x7f72840ce240 2023-11-09T03:48:58.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.462+0000 7f72827fc640 1 -- 172.21.15.76:0/3297178788 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c030730 con 0x7f72840ce240 2023-11-09T03:48:58.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.462+0000 7f7289516640 1 -- 172.21.15.76:0/3297178788 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72840ce240 msgr2=0x7f7284157000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:58.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.462+0000 7f7289516640 1 --2- 172.21.15.76:0/3297178788 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72840ce240 0x7f7284157000 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f726c0097a0 tx=0x7f726c02b790 comp rx=0 tx=0).stop 2023-11-09T03:48:58.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.462+0000 7f7289516640 1 -- 172.21.15.76:0/3297178788 shutdown_connections 2023-11-09T03:48:58.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.462+0000 7f7289516640 1 --2- 172.21.15.76:0/3297178788 >> v2:172.21.15.113:3300/0 conn(0x7f7284157540 0x7f72840d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.462+0000 7f7289516640 1 --2- 172.21.15.76:0/3297178788 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72840ce240 0x7f7284157000 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.462+0000 7f7289516640 1 -- 172.21.15.76:0/3297178788 >> 172.21.15.76:0/3297178788 conn(0x7f7284071f30 msgr2=0x7f72840cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:58.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.462+0000 7f7289516640 1 -- 172.21.15.76:0/3297178788 shutdown_connections 2023-11-09T03:48:58.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.462+0000 7f7289516640 1 -- 172.21.15.76:0/3297178788 wait complete. 2023-11-09T03:48:58.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.463+0000 7f7289516640 1 Processor -- start 2023-11-09T03:48:58.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.463+0000 7f7289516640 1 -- start start 2023-11-09T03:48:58.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.464+0000 7f7289516640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72840cde60 0x7f728414a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:58.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.464+0000 7f7289516640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72840ce240 0x7f72840d1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:58.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.464+0000 7f7289516640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7284157540 0x7f72840d1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:58.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.464+0000 7f7289516640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72841f3630 con 0x7f72840cde60 2023-11-09T03:48:58.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.464+0000 7f7289516640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72840cc260 con 0x7f72840ce240 2023-11-09T03:48:58.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.464+0000 7f7289516640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72840cc540 con 0x7f7284157540 2023-11-09T03:48:58.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.464+0000 7f7282ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72840cde60 0x7f728414a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:58.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.464+0000 7f7282ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72840cde60 0x7f728414a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44110/0 (socket says 172.21.15.76:44110) 2023-11-09T03:48:58.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.464+0000 7f7282ffd640 1 -- 172.21.15.76:0/2377885715 learned_addr learned my addr 172.21.15.76:0/2377885715 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:58.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.464+0000 7f72837fe640 1 --2- 172.21.15.76:0/2377885715 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7284157540 0x7f72840d1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:58.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.464+0000 7f727bfff640 1 --2- 172.21.15.76:0/2377885715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72840ce240 0x7f72840d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:58.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.464+0000 7f7282ffd640 1 -- 172.21.15.76:0/2377885715 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7284157540 msgr2=0x7f72840d1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:58.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.464+0000 7f7282ffd640 1 --2- 172.21.15.76:0/2377885715 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7284157540 0x7f72840d1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.464+0000 7f7282ffd640 1 -- 172.21.15.76:0/2377885715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72840ce240 msgr2=0x7f72840d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:58.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.464+0000 7f7282ffd640 1 --2- 172.21.15.76:0/2377885715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72840ce240 0x7f72840d1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.464+0000 7f7282ffd640 1 -- 172.21.15.76:0/2377885715 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727c009a70 con 0x7f72840cde60 2023-11-09T03:48:58.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.465+0000 7f7282ffd640 1 --2- 172.21.15.76:0/2377885715 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72840cde60 0x7f728414a0e0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f727400c710 tx=0x7f727400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:58.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.465+0000 7f727b7fe640 1 -- 172.21.15.76:0/2377885715 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7274005ad0 con 0x7f72840cde60 2023-11-09T03:48:58.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.465+0000 7f727b7fe640 1 -- 172.21.15.76:0/2377885715 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7274005c30 con 0x7f72840cde60 2023-11-09T03:48:58.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.465+0000 7f7289516640 1 -- 172.21.15.76:0/2377885715 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f726c009400 con 0x7f72840cde60 2023-11-09T03:48:58.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.465+0000 7f727b7fe640 1 -- 172.21.15.76:0/2377885715 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7274007070 con 0x7f72840cde60 2023-11-09T03:48:58.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.465+0000 7f7289516640 1 -- 172.21.15.76:0/2377885715 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72840ccb00 con 0x7f72840cde60 2023-11-09T03:48:58.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.466+0000 7f7289516640 1 -- 172.21.15.76:0/2377885715 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72840d13e0 con 0x7f72840cde60 2023-11-09T03:48:58.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.466+0000 7f727b7fe640 1 -- 172.21.15.76:0/2377885715 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f72740071f0 con 0x7f72840cde60 2023-11-09T03:48:58.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.466+0000 7f727b7fe640 1 --2- 172.21.15.76:0/2377885715 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f726406ae70 0x7f726406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:58.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.467+0000 7f727b7fe640 1 -- 172.21.15.76:0/2377885715 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f72740899c0 con 0x7f72840cde60 2023-11-09T03:48:58.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.467+0000 7f727bfff640 1 --2- 172.21.15.76:0/2377885715 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f726406ae70 0x7f726406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:58.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.467+0000 7f727bfff640 1 --2- 172.21.15.76:0/2377885715 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f726406ae70 0x7f726406d330 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f726c034000 tx=0x7f726c02f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:58.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.472+0000 7f727b7fe640 1 -- 172.21.15.76:0/2377885715 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f72840d13e0 con 0x7f72840cde60 2023-11-09T03:48:58.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.638+0000 7f7289516640 1 -- 172.21.15.76:0/2377885715 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72840d13e0 con 0x7f72840cde60 2023-11-09T03:48:58.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.640+0000 7f727b7fe640 1 -- 172.21.15.76:0/2377885715 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+21993 (secure 0 0 0) 0x7f72840d13e0 con 0x7f72840cde60 2023-11-09T03:48:58.641 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:48:58.641 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":95,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:48:58.423703+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-09T03:47:59.280601+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-09T03:47:59.742039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:48:55.490832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-09T03:48:56.352624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:48:58.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.642+0000 7f7289516640 1 -- 172.21.15.76:0/2377885715 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f726406ae70 msgr2=0x7f726406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:58.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.642+0000 7f7289516640 1 --2- 172.21.15.76:0/2377885715 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f726406ae70 0x7f726406d330 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f726c034000 tx=0x7f726c02f000 comp rx=0 tx=0).stop 2023-11-09T03:48:58.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.642+0000 7f7289516640 1 -- 172.21.15.76:0/2377885715 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72840cde60 msgr2=0x7f728414a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:58.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.642+0000 7f7289516640 1 --2- 172.21.15.76:0/2377885715 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72840cde60 0x7f728414a0e0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f727400c710 tx=0x7f727400cbd0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.642+0000 7f7289516640 1 -- 172.21.15.76:0/2377885715 shutdown_connections 2023-11-09T03:48:58.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.642+0000 7f7289516640 1 --2- 172.21.15.76:0/2377885715 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f726406ae70 0x7f726406d330 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.642+0000 7f7289516640 1 --2- 172.21.15.76:0/2377885715 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7284157540 0x7f72840d1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.642+0000 7f7289516640 1 --2- 172.21.15.76:0/2377885715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72840ce240 0x7f72840d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.642+0000 7f7289516640 1 --2- 172.21.15.76:0/2377885715 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72840cde60 0x7f728414a0e0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.642+0000 7f7289516640 1 -- 172.21.15.76:0/2377885715 >> 172.21.15.76:0/2377885715 conn(0x7f7284071f30 msgr2=0x7f72840b7210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:58.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.642+0000 7f7289516640 1 -- 172.21.15.76:0/2377885715 shutdown_connections 2023-11-09T03:48:58.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.642+0000 7f7289516640 1 -- 172.21.15.76:0/2377885715 wait complete. 2023-11-09T03:48:58.654 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-09T03:48:58.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.756+0000 7f0bd52f2640 1 Processor -- start 2023-11-09T03:48:58.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.756+0000 7f0bd52f2640 1 -- start start 2023-11-09T03:48:58.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.757+0000 7f0bd52f2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0bd015ad30 0x7f0bd00adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:58.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.757+0000 7f0bd52f2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0bd00ad730 0x7f0bd0070710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:58.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.757+0000 7f0bd52f2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0bd0070c50 0x7f0bd00c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:58.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.757+0000 7f0bd52f2640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bd00ae260 con 0x7f0bd0070c50 2023-11-09T03:48:58.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.757+0000 7f0bd52f2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bd00710e0 con 0x7f0bd015ad30 2023-11-09T03:48:58.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.757+0000 7f0bd52f2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd00c3900 con 0x7f0bd00ad730 2023-11-09T03:48:58.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.757+0000 7f0bcf7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0bd0070c50 0x7f0bd00c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:58.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.757+0000 7f0bcf7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0bd0070c50 0x7f0bd00c8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44116/0 (socket says 172.21.15.76:44116) 2023-11-09T03:48:58.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.757+0000 7f0bcf7fe640 1 -- 172.21.15.76:0/372222760 learned_addr learned my addr 172.21.15.76:0/372222760 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:58.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.757+0000 7f0bceffd640 1 --2- 172.21.15.76:0/372222760 >> v2:172.21.15.113:3300/0 conn(0x7f0bd015ad30 0x7f0bd00adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:58.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.757+0000 7f0bce7fc640 1 --2- 172.21.15.76:0/372222760 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0bd00ad730 0x7f0bd0070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:58.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.757+0000 7f0bcf7fe640 1 -- 172.21.15.76:0/372222760 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0bd00ad730 msgr2=0x7f0bd0070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:58.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.757+0000 7f0bcf7fe640 1 --2- 172.21.15.76:0/372222760 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0bd00ad730 0x7f0bd0070710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.757+0000 7f0bcf7fe640 1 -- 172.21.15.76:0/372222760 >> v2:172.21.15.113:3300/0 conn(0x7f0bd015ad30 msgr2=0x7f0bd00adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:58.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.757+0000 7f0bcf7fe640 1 --2- 172.21.15.76:0/372222760 >> v2:172.21.15.113:3300/0 conn(0x7f0bd015ad30 0x7f0bd00adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.757+0000 7f0bcf7fe640 1 -- 172.21.15.76:0/372222760 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd00c3be0 con 0x7f0bd0070c50 2023-11-09T03:48:58.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.757+0000 7f0bce7fc640 1 --2- 172.21.15.76:0/372222760 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0bd00ad730 0x7f0bd0070710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:58.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.757+0000 7f0bceffd640 1 --2- 172.21.15.76:0/372222760 >> v2:172.21.15.113:3300/0 conn(0x7f0bd015ad30 0x7f0bd00adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:48:58.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.758+0000 7f0bcf7fe640 1 --2- 172.21.15.76:0/372222760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bd0070c50 0x7f0bd00c8fa0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009870 tx=0x7f0bc802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ce74cf27766a27 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:58.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.758+0000 7f0bcdffb640 1 -- 172.21.15.76:0/372222760 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc80317c0 con 0x7f0bd0070c50 2023-11-09T03:48:58.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.758+0000 7f0bcdffb640 1 -- 172.21.15.76:0/372222760 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bc8031920 con 0x7f0bd0070c50 2023-11-09T03:48:58.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.758+0000 7f0bcdffb640 1 -- 172.21.15.76:0/372222760 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8031cb0 con 0x7f0bd0070c50 2023-11-09T03:48:58.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.759+0000 7f0bd52f2640 1 -- 172.21.15.76:0/372222760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bd0070c50 msgr2=0x7f0bd00c8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:58.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.759+0000 7f0bd52f2640 1 --2- 172.21.15.76:0/372222760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bd0070c50 0x7f0bd00c8fa0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009870 tx=0x7f0bc802b860 comp rx=0 tx=0).stop 2023-11-09T03:48:58.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.759+0000 7f0bd52f2640 1 -- 172.21.15.76:0/372222760 shutdown_connections 2023-11-09T03:48:58.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.759+0000 7f0bd52f2640 1 --2- 172.21.15.76:0/372222760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bd0070c50 0x7f0bd00c8fa0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.759+0000 7f0bd52f2640 1 --2- 172.21.15.76:0/372222760 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0bd00ad730 0x7f0bd0070710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.759+0000 7f0bd52f2640 1 --2- 172.21.15.76:0/372222760 >> v2:172.21.15.113:3300/0 conn(0x7f0bd015ad30 0x7f0bd00adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.759+0000 7f0bd52f2640 1 -- 172.21.15.76:0/372222760 >> 172.21.15.76:0/372222760 conn(0x7f0bd00ce950 msgr2=0x7f0bd00ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:58.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.759+0000 7f0bd52f2640 1 -- 172.21.15.76:0/372222760 shutdown_connections 2023-11-09T03:48:58.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.759+0000 7f0bd52f2640 1 -- 172.21.15.76:0/372222760 wait complete. 2023-11-09T03:48:58.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.760+0000 7f0bd52f2640 1 Processor -- start 2023-11-09T03:48:58.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.760+0000 7f0bd52f2640 1 -- start start 2023-11-09T03:48:58.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.761+0000 7f0bd52f2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bd0070c50 0x7f0bd0142c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:58.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.761+0000 7f0bd52f2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0bd00ad730 0x7f0bd0140cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:58.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.761+0000 7f0bd52f2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bd015ad30 0x7f0bd00c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:58.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.761+0000 7f0bd52f2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd01431a0 con 0x7f0bd015ad30 2023-11-09T03:48:58.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.761+0000 7f0bd52f2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd015cf70 con 0x7f0bd0070c50 2023-11-09T03:48:58.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.761+0000 7f0bd52f2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd0141200 con 0x7f0bd00ad730 2023-11-09T03:48:58.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.761+0000 7f0bcf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bd015ad30 0x7f0bd00c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:58.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.761+0000 7f0bcf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bd015ad30 0x7f0bd00c6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44130/0 (socket says 172.21.15.76:44130) 2023-11-09T03:48:58.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.761+0000 7f0bcf7fe640 1 -- 172.21.15.76:0/2071232084 learned_addr learned my addr 172.21.15.76:0/2071232084 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:58.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.761+0000 7f0bcf7fe640 1 -- 172.21.15.76:0/2071232084 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0bd00ad730 msgr2=0x7f0bd0140cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:58.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.761+0000 7f0bce7fc640 1 --2- 172.21.15.76:0/2071232084 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0bd00ad730 0x7f0bd0140cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:58.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.761+0000 7f0bcf7fe640 1 --2- 172.21.15.76:0/2071232084 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0bd00ad730 0x7f0bd0140cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.761+0000 7f0bcf7fe640 1 -- 172.21.15.76:0/2071232084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bd0070c50 msgr2=0x7f0bd0142c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:58.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.761+0000 7f0bceffd640 1 --2- 172.21.15.76:0/2071232084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bd0070c50 0x7f0bd0142c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:58.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.761+0000 7f0bcf7fe640 1 --2- 172.21.15.76:0/2071232084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bd0070c50 0x7f0bd0142c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:58.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.761+0000 7f0bcf7fe640 1 -- 172.21.15.76:0/2071232084 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc80094d0 con 0x7f0bd015ad30 2023-11-09T03:48:58.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.761+0000 7f0bcf7fe640 1 --2- 172.21.15.76:0/2071232084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bd015ad30 0x7f0bd00c6870 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80090b0 tx=0x7f0bc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:58.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.762+0000 7f0bb77fe640 1 -- 172.21.15.76:0/2071232084 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8032860 con 0x7f0bd015ad30 2023-11-09T03:48:58.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.762+0000 7f0bb77fe640 1 -- 172.21.15.76:0/2071232084 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bc80329e0 con 0x7f0bd015ad30 2023-11-09T03:48:58.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.762+0000 7f0bd52f2640 1 -- 172.21.15.76:0/2071232084 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd015c680 con 0x7f0bd015ad30 2023-11-09T03:48:58.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.762+0000 7f0bb77fe640 1 -- 172.21.15.76:0/2071232084 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8032d90 con 0x7f0bd015ad30 2023-11-09T03:48:58.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.762+0000 7f0bd52f2640 1 -- 172.21.15.76:0/2071232084 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd015c920 con 0x7f0bd015ad30 2023-11-09T03:48:58.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.762+0000 7f0bd52f2640 1 -- 172.21.15.76:0/2071232084 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bd0074290 con 0x7f0bd015ad30 2023-11-09T03:48:58.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.763+0000 7f0bb77fe640 1 -- 172.21.15.76:0/2071232084 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0bc804e450 con 0x7f0bd015ad30 2023-11-09T03:48:58.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.763+0000 7f0bb77fe640 1 --2- 172.21.15.76:0/2071232084 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0bbc06ae70 0x7f0bbc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:58.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.763+0000 7f0bb77fe640 1 -- 172.21.15.76:0/2071232084 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f0bc80abca0 con 0x7f0bd015ad30 2023-11-09T03:48:58.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.764+0000 7f0bceffd640 1 --2- 172.21.15.76:0/2071232084 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0bbc06ae70 0x7f0bbc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:58.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.764+0000 7f0bceffd640 1 --2- 172.21.15.76:0/2071232084 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0bbc06ae70 0x7f0bbc06d330 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0009ed0 tx=0x7f0bc000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:58.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.769+0000 7f0bb77fe640 1 -- 172.21.15.76:0/2071232084 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0bc8076980 con 0x7f0bd015ad30 2023-11-09T03:48:58.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:58.974+0000 7f0bd52f2640 1 -- 172.21.15.76:0/2071232084 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0bd015d0b0 con 0x7f0bd015ad30 2023-11-09T03:48:59.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.047+0000 7f0bb77fe640 1 -- 172.21.15.76:0/2071232084 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v43) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0bc8079fa0 con 0x7f0bd015ad30 2023-11-09T03:48:59.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.048+0000 7f0bd52f2640 1 -- 172.21.15.76:0/2071232084 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0bbc06ae70 msgr2=0x7f0bbc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:59.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.048+0000 7f0bd52f2640 1 --2- 172.21.15.76:0/2071232084 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0bbc06ae70 0x7f0bbc06d330 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0009ed0 tx=0x7f0bc000b040 comp rx=0 tx=0).stop 2023-11-09T03:48:59.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.048+0000 7f0bd52f2640 1 -- 172.21.15.76:0/2071232084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bd015ad30 msgr2=0x7f0bd00c6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:59.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.048+0000 7f0bd52f2640 1 --2- 172.21.15.76:0/2071232084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bd015ad30 0x7f0bd00c6870 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80090b0 tx=0x7f0bc80046c0 comp rx=0 tx=0).stop 2023-11-09T03:48:59.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.049+0000 7f0bd52f2640 1 -- 172.21.15.76:0/2071232084 shutdown_connections 2023-11-09T03:48:59.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.049+0000 7f0bd52f2640 1 --2- 172.21.15.76:0/2071232084 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0bbc06ae70 0x7f0bbc06d330 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:59.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.049+0000 7f0bd52f2640 1 --2- 172.21.15.76:0/2071232084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bd015ad30 0x7f0bd00c6870 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:59.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.049+0000 7f0bd52f2640 1 --2- 172.21.15.76:0/2071232084 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0bd00ad730 0x7f0bd0140cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:59.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.049+0000 7f0bd52f2640 1 --2- 172.21.15.76:0/2071232084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bd0070c50 0x7f0bd0142c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:59.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.049+0000 7f0bd52f2640 1 -- 172.21.15.76:0/2071232084 >> 172.21.15.76:0/2071232084 conn(0x7f0bd00ce950 msgr2=0x7f0bd006d680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:59.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.049+0000 7f0bd52f2640 1 -- 172.21.15.76:0/2071232084 shutdown_connections 2023-11-09T03:48:59.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.049+0000 7f0bd52f2640 1 -- 172.21.15.76:0/2071232084 wait complete. 2023-11-09T03:48:59.059 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-09T03:48:59.059 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-09T03:48:59.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.162+0000 7fd333f24640 1 Processor -- start 2023-11-09T03:48:59.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.163+0000 7fd333f24640 1 -- start start 2023-11-09T03:48:59.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.163+0000 7fd333f24640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd32c153e60 0x7fd32c15dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:59.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.163+0000 7fd333f24640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd32c0b4880 0x7fd32c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:59.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.163+0000 7fd333f24640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd32c0cde60 con 0x7fd32c0cd6d0 2023-11-09T03:48:59.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.163+0000 7fd333f24640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd32c0b5230 con 0x7fd32c0b4880 2023-11-09T03:48:59.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.163+0000 7fd333f24640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c160f90 con 0x7fd32c153e60 2023-11-09T03:48:59.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.164+0000 7fd331cc0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd32c153e60 0x7fd32c15dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:59.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.164+0000 7fd331cc0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd32c153e60 0x7fd32c15dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:56400/0 (socket says 172.21.15.76:56400) 2023-11-09T03:48:59.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.164+0000 7fd3324c1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd32c0b4880 0x7fd32c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:59.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.164+0000 7fd331cc0640 1 -- 172.21.15.76:0/1291961077 learned_addr learned my addr 172.21.15.76:0/1291961077 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:59.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.164+0000 7fd331cc0640 1 -- 172.21.15.76:0/1291961077 >> v2:172.21.15.113:3300/0 conn(0x7fd32c0b4880 msgr2=0x7fd32c0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:48:59.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.164+0000 7fd331cc0640 1 --2- 172.21.15.76:0/1291961077 >> v2:172.21.15.113:3300/0 conn(0x7fd32c0b4880 0x7fd32c0b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:59.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.164+0000 7fd331cc0640 1 -- 172.21.15.76:0/1291961077 >> v1:172.21.15.113:6789/0 conn(0x7fd32c0cd6d0 legacy=0x7fd32c161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:59.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.164+0000 7fd331cc0640 1 -- 172.21.15.76:0/1291961077 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd32c15e400 con 0x7fd32c153e60 2023-11-09T03:48:59.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.164+0000 7fd3324c1640 1 --2- 172.21.15.76:0/1291961077 >> v2:172.21.15.113:3300/0 conn(0x7fd32c0b4880 0x7fd32c0b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:48:59.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.165+0000 7fd331cc0640 1 --2- 172.21.15.76:0/1291961077 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd32c153e60 0x7fd32c15dec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd328009870 tx=0x7fd32802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c580052319a5c031 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:59.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.165+0000 7fd330cbe640 1 -- 172.21.15.76:0/1291961077 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd328030840 con 0x7fd32c153e60 2023-11-09T03:48:59.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.165+0000 7fd330cbe640 1 -- 172.21.15.76:0/1291961077 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3280309a0 con 0x7fd32c153e60 2023-11-09T03:48:59.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.165+0000 7fd330cbe640 1 -- 172.21.15.76:0/1291961077 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd328030d30 con 0x7fd32c153e60 2023-11-09T03:48:59.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.166+0000 7fd333f24640 1 -- 172.21.15.76:0/1291961077 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd32c153e60 msgr2=0x7fd32c15dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:59.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.166+0000 7fd333f24640 1 --2- 172.21.15.76:0/1291961077 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd32c153e60 0x7fd32c15dec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd328009870 tx=0x7fd32802b980 comp rx=0 tx=0).stop 2023-11-09T03:48:59.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.166+0000 7fd333f24640 1 -- 172.21.15.76:0/1291961077 shutdown_connections 2023-11-09T03:48:59.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.166+0000 7fd333f24640 1 --2- 172.21.15.76:0/1291961077 >> v2:172.21.15.113:3300/0 conn(0x7fd32c0b4880 0x7fd32c0b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:59.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.166+0000 7fd333f24640 1 --2- 172.21.15.76:0/1291961077 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd32c153e60 0x7fd32c15dec0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:59.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.166+0000 7fd333f24640 1 -- 172.21.15.76:0/1291961077 >> 172.21.15.76:0/1291961077 conn(0x7fd32c0cf880 msgr2=0x7fd32c0cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:48:59.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.166+0000 7fd333f24640 1 -- 172.21.15.76:0/1291961077 shutdown_connections 2023-11-09T03:48:59.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.166+0000 7fd333f24640 1 -- 172.21.15.76:0/1291961077 wait complete. 2023-11-09T03:48:59.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.167+0000 7fd333f24640 1 Processor -- start 2023-11-09T03:48:59.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.167+0000 7fd333f24640 1 -- start start 2023-11-09T03:48:59.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.168+0000 7fd333f24640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd32c0b4880 0x7fd32c1f3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:59.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.168+0000 7fd333f24640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd32c0cd6d0 0x7fd32c0c6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:59.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.168+0000 7fd333f24640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd32c153e60 0x7fd32c1562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:59.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.168+0000 7fd333f24640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c1f4110 con 0x7fd32c0b4880 2023-11-09T03:48:59.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.168+0000 7fd333f24640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c0c7a20 con 0x7fd32c0cd6d0 2023-11-09T03:48:59.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.168+0000 7fd333f24640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c0c7d00 con 0x7fd32c153e60 2023-11-09T03:48:59.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.168+0000 7fd331cc0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd32c0b4880 0x7fd32c1f3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:59.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.168+0000 7fd331cc0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd32c0b4880 0x7fd32c1f3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44146/0 (socket says 172.21.15.76:44146) 2023-11-09T03:48:59.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.168+0000 7fd331cc0640 1 -- 172.21.15.76:0/583154820 learned_addr learned my addr 172.21.15.76:0/583154820 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:48:59.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.168+0000 7fd3314bf640 1 --2- 172.21.15.76:0/583154820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd32c0cd6d0 0x7fd32c0c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:59.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.168+0000 7fd3324c1640 1 --2- 172.21.15.76:0/583154820 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd32c153e60 0x7fd32c1562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:59.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.168+0000 7fd331cc0640 1 -- 172.21.15.76:0/583154820 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd32c153e60 msgr2=0x7fd32c1562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:59.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.168+0000 7fd331cc0640 1 --2- 172.21.15.76:0/583154820 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd32c153e60 0x7fd32c1562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:59.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.168+0000 7fd331cc0640 1 -- 172.21.15.76:0/583154820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd32c0cd6d0 msgr2=0x7fd32c0c6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:48:59.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.168+0000 7fd331cc0640 1 --2- 172.21.15.76:0/583154820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd32c0cd6d0 0x7fd32c0c6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:48:59.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.168+0000 7fd331cc0640 1 -- 172.21.15.76:0/583154820 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd324009a70 con 0x7fd32c0b4880 2023-11-09T03:48:59.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.168+0000 7fd331cc0640 1 --2- 172.21.15.76:0/583154820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd32c0b4880 0x7fd32c1f3bd0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fd32802b9c0 tx=0x7fd3280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:59.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.168+0000 7fd3227fc640 1 -- 172.21.15.76:0/583154820 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32803e070 con 0x7fd32c0b4880 2023-11-09T03:48:59.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.168+0000 7fd3227fc640 1 -- 172.21.15.76:0/583154820 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd328039a10 con 0x7fd32c0b4880 2023-11-09T03:48:59.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.168+0000 7fd333f24640 1 -- 172.21.15.76:0/583154820 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3280094d0 con 0x7fd32c0b4880 2023-11-09T03:48:59.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.168+0000 7fd3227fc640 1 -- 172.21.15.76:0/583154820 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd328039de0 con 0x7fd32c0b4880 2023-11-09T03:48:59.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.168+0000 7fd333f24640 1 -- 172.21.15.76:0/583154820 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd32c0c82c0 con 0x7fd32c0b4880 2023-11-09T03:48:59.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.169+0000 7fd333f24640 1 -- 172.21.15.76:0/583154820 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd32c074290 con 0x7fd32c0b4880 2023-11-09T03:48:59.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.170+0000 7fd3227fc640 1 -- 172.21.15.76:0/583154820 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd32804b400 con 0x7fd32c0b4880 2023-11-09T03:48:59.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.170+0000 7fd3227fc640 1 --2- 172.21.15.76:0/583154820 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd31006ae70 0x7fd31006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:48:59.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.170+0000 7fd3227fc640 1 -- 172.21.15.76:0/583154820 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7120+0+0 (secure 0 0 0) 0x7fd3280ac060 con 0x7fd32c0b4880 2023-11-09T03:48:59.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.170+0000 7fd3314bf640 1 --2- 172.21.15.76:0/583154820 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd31006ae70 0x7fd31006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:48:59.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.171+0000 7fd3314bf640 1 --2- 172.21.15.76:0/583154820 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd31006ae70 0x7fd31006d330 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0d08c0 tx=0x7fd31c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:48:59.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.174+0000 7fd3227fc640 1 -- 172.21.15.76:0/583154820 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd328076b90 con 0x7fd32c0b4880 2023-11-09T03:48:59.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:48:59.377+0000 7fd333f24640 1 -- 172.21.15.76:0/583154820 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd32c153810 con 0x7fd32c0b4880 2023-11-09T03:49:00.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.057+0000 7fd3227fc640 1 -- 172.21.15.76:0/583154820 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v44) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd3280340c0 con 0x7fd32c0b4880 2023-11-09T03:49:00.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.060+0000 7fd333f24640 1 -- 172.21.15.76:0/583154820 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd31006ae70 msgr2=0x7fd31006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:00.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.060+0000 7fd333f24640 1 --2- 172.21.15.76:0/583154820 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd31006ae70 0x7fd31006d330 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0d08c0 tx=0x7fd31c009040 comp rx=0 tx=0).stop 2023-11-09T03:49:00.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.060+0000 7fd333f24640 1 -- 172.21.15.76:0/583154820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd32c0b4880 msgr2=0x7fd32c1f3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:00.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.060+0000 7fd333f24640 1 --2- 172.21.15.76:0/583154820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd32c0b4880 0x7fd32c1f3bd0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fd32802b9c0 tx=0x7fd3280046c0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.060+0000 7fd333f24640 1 -- 172.21.15.76:0/583154820 shutdown_connections 2023-11-09T03:49:00.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.060+0000 7fd333f24640 1 --2- 172.21.15.76:0/583154820 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd31006ae70 0x7fd31006d330 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.060+0000 7fd333f24640 1 --2- 172.21.15.76:0/583154820 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd32c153e60 0x7fd32c1562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.060+0000 7fd333f24640 1 --2- 172.21.15.76:0/583154820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd32c0cd6d0 0x7fd32c0c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.060+0000 7fd333f24640 1 --2- 172.21.15.76:0/583154820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd32c0b4880 0x7fd32c1f3bd0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.060+0000 7fd333f24640 1 -- 172.21.15.76:0/583154820 >> 172.21.15.76:0/583154820 conn(0x7fd32c0cf880 msgr2=0x7fd32c0cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:00.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.060+0000 7fd333f24640 1 -- 172.21.15.76:0/583154820 shutdown_connections 2023-11-09T03:49:00.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.060+0000 7fd333f24640 1 -- 172.21.15.76:0/583154820 wait complete. 2023-11-09T03:49:00.071 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-09T03:49:00.071 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:49:00.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.174+0000 7ff9a15b6640 1 Processor -- start 2023-11-09T03:49:00.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.174+0000 7ff9a15b6640 1 -- start start 2023-11-09T03:49:00.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.174+0000 7ff9a15b6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff99c0cde60 0x7ff99c15d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:00.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.174+0000 7ff9a15b6640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff99c15e000 con 0x7ff99c15daa0 2023-11-09T03:49:00.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.174+0000 7ff9a15b6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff99c15e2e0 con 0x7ff99c0ce240 2023-11-09T03:49:00.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.174+0000 7ff9a15b6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c15e5c0 con 0x7ff99c0cde60 2023-11-09T03:49:00.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.175+0000 7ff99a7fc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff99c15daa0 0x7ff99c157000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35896/0 (socket says 172.21.15.76:35896) 2023-11-09T03:49:00.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.175+0000 7ff99a7fc640 1 -- 172.21.15.76:0/846086738 learned_addr learned my addr 172.21.15.76:0/846086738 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:49:00.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.175+0000 7ff99affd640 1 --2- 172.21.15.76:0/846086738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff99c0cde60 0x7ff99c15d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:00.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.175+0000 7ff999ffb640 1 -- 172.21.15.76:0/846086738 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 748477395 0 0) 0x7ff99c15e000 con 0x7ff99c15daa0 2023-11-09T03:49:00.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.175+0000 7ff999ffb640 1 -- 172.21.15.76:0/846086738 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff988003520 con 0x7ff99c15daa0 2023-11-09T03:49:00.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.175+0000 7ff999ffb640 1 -- 172.21.15.76:0/846086738 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3967303222 0 0) 0x7ff99c15e2e0 con 0x7ff99c0ce240 2023-11-09T03:49:00.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.175+0000 7ff999ffb640 1 -- 172.21.15.76:0/846086738 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff99c15e000 con 0x7ff99c0ce240 2023-11-09T03:49:00.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.175+0000 7ff99affd640 1 -- 172.21.15.76:0/846086738 >> v1:172.21.15.113:6789/0 conn(0x7ff99c0ce240 legacy=0x7ff99c0d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:00.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.175+0000 7ff99affd640 1 -- 172.21.15.76:0/846086738 >> v1:172.21.15.76:6789/0 conn(0x7ff99c15daa0 legacy=0x7ff99c157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:00.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.175+0000 7ff99affd640 1 -- 172.21.15.76:0/846086738 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99c0d1120 con 0x7ff99c0cde60 2023-11-09T03:49:00.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.176+0000 7ff99affd640 1 --2- 172.21.15.76:0/846086738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff99c0cde60 0x7ff99c15d560 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009dc0 tx=0x7ff98c02bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=218cc990bdd7dafb server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:00.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.176+0000 7ff999ffb640 1 -- 172.21.15.76:0/846086738 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c02ccd0 con 0x7ff99c0cde60 2023-11-09T03:49:00.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.176+0000 7ff999ffb640 1 -- 172.21.15.76:0/846086738 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98c02ce30 con 0x7ff99c0cde60 2023-11-09T03:49:00.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.176+0000 7ff999ffb640 1 -- 172.21.15.76:0/846086738 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c034640 con 0x7ff99c0cde60 2023-11-09T03:49:00.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.177+0000 7ff9a15b6640 1 -- 172.21.15.76:0/846086738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff99c0cde60 msgr2=0x7ff99c15d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:00.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.177+0000 7ff9a15b6640 1 --2- 172.21.15.76:0/846086738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff99c0cde60 0x7ff99c15d560 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009dc0 tx=0x7ff98c02bf20 comp rx=0 tx=0).stop 2023-11-09T03:49:00.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.177+0000 7ff9a15b6640 1 -- 172.21.15.76:0/846086738 shutdown_connections 2023-11-09T03:49:00.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.177+0000 7ff9a15b6640 1 --2- 172.21.15.76:0/846086738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff99c0cde60 0x7ff99c15d560 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.177+0000 7ff9a15b6640 1 -- 172.21.15.76:0/846086738 >> 172.21.15.76:0/846086738 conn(0x7ff99c071f30 msgr2=0x7ff99c0cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:00.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.177+0000 7ff9a15b6640 1 -- 172.21.15.76:0/846086738 shutdown_connections 2023-11-09T03:49:00.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.177+0000 7ff9a15b6640 1 -- 172.21.15.76:0/846086738 wait complete. 2023-11-09T03:49:00.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.178+0000 7ff9a15b6640 1 Processor -- start 2023-11-09T03:49:00.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.178+0000 7ff9a15b6640 1 -- start start 2023-11-09T03:49:00.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.179+0000 7ff9a15b6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff99c0cde60 0x7ff99c1f2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:00.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.179+0000 7ff9a15b6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff99c0ce240 0x7ff99c14d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:00.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.179+0000 7ff9a15b6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff99c15daa0 0x7ff99c15c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:00.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.179+0000 7ff9a15b6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c157b40 con 0x7ff99c0ce240 2023-11-09T03:49:00.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.179+0000 7ff9a15b6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c157e20 con 0x7ff99c15daa0 2023-11-09T03:49:00.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.179+0000 7ff9a15b6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c158100 con 0x7ff99c0cde60 2023-11-09T03:49:00.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.179+0000 7ff99a7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff99c0ce240 0x7ff99c14d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:00.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.179+0000 7ff99a7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff99c0ce240 0x7ff99c14d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44158/0 (socket says 172.21.15.76:44158) 2023-11-09T03:49:00.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.179+0000 7ff99a7fc640 1 -- 172.21.15.76:0/2902319063 learned_addr learned my addr 172.21.15.76:0/2902319063 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:00.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.179+0000 7ff99affd640 1 --2- 172.21.15.76:0/2902319063 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff99c0cde60 0x7ff99c1f2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:00.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.179+0000 7ff99b7fe640 1 --2- 172.21.15.76:0/2902319063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff99c15daa0 0x7ff99c15c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:00.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.179+0000 7ff99a7fc640 1 -- 172.21.15.76:0/2902319063 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff99c0cde60 msgr2=0x7ff99c1f2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:00.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.179+0000 7ff99a7fc640 1 --2- 172.21.15.76:0/2902319063 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff99c0cde60 0x7ff99c1f2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.179+0000 7ff99a7fc640 1 -- 172.21.15.76:0/2902319063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff99c15daa0 msgr2=0x7ff99c15c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:00.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.179+0000 7ff99a7fc640 1 --2- 172.21.15.76:0/2902319063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff99c15daa0 0x7ff99c15c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.179+0000 7ff99a7fc640 1 -- 172.21.15.76:0/2902319063 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98c009a70 con 0x7ff99c0ce240 2023-11-09T03:49:00.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.179+0000 7ff99a7fc640 1 --2- 172.21.15.76:0/2902319063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff99c0ce240 0x7ff99c14d360 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7ff98400c410 tx=0x7ff98400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:00.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.179+0000 7ff9937fe640 1 -- 172.21.15.76:0/2902319063 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff984005ce0 con 0x7ff99c0ce240 2023-11-09T03:49:00.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.179+0000 7ff9937fe640 1 -- 172.21.15.76:0/2902319063 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff984005e40 con 0x7ff99c0ce240 2023-11-09T03:49:00.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.179+0000 7ff9937fe640 1 -- 172.21.15.76:0/2902319063 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9840072d0 con 0x7ff99c0ce240 2023-11-09T03:49:00.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.179+0000 7ff9a15b6640 1 -- 172.21.15.76:0/2902319063 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff99c1583e0 con 0x7ff99c0ce240 2023-11-09T03:49:00.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.179+0000 7ff9a15b6640 1 -- 172.21.15.76:0/2902319063 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff99c15c9b0 con 0x7ff99c0ce240 2023-11-09T03:49:00.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.180+0000 7ff9a15b6640 1 -- 172.21.15.76:0/2902319063 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff99c158900 con 0x7ff99c0ce240 2023-11-09T03:49:00.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.181+0000 7ff9937fe640 1 -- 172.21.15.76:0/2902319063 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff984007430 con 0x7ff99c0ce240 2023-11-09T03:49:00.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.181+0000 7ff9937fe640 1 --2- 172.21.15.76:0/2902319063 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff97c06aea0 0x7ff97c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:00.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.181+0000 7ff9937fe640 1 -- 172.21.15.76:0/2902319063 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7120+0+0 (secure 0 0 0) 0x7ff984089140 con 0x7ff99c0ce240 2023-11-09T03:49:00.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.181+0000 7ff99affd640 1 --2- 172.21.15.76:0/2902319063 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff97c06aea0 0x7ff97c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:00.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.182+0000 7ff99affd640 1 --2- 172.21.15.76:0/2902319063 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff97c06aea0 0x7ff97c06d360 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff98c036000 tx=0x7ff98c030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:00.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.185+0000 7ff9937fe640 1 -- 172.21.15.76:0/2902319063 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff99c158900 con 0x7ff99c0ce240 2023-11-09T03:49:00.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.386+0000 7ff9a15b6640 1 -- 172.21.15.76:0/2902319063 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff99c158900 con 0x7ff99c0ce240 2023-11-09T03:49:00.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.386+0000 7ff9937fe640 1 -- 172.21.15.76:0/2902319063 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4918 (secure 0 0 0) 0x7ff99c158900 con 0x7ff99c0ce240 2023-11-09T03:49:00.386 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:49:00.386 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":44,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5744,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3280098776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3280098776},{"type":"v1","addr":"172.21.15.113:6837","nonce":3280098776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":7449,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:48:59.377862+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7452":{"gid":7452,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/4011275120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4011275120},{"type":"v1","addr":"172.21.15.76:6835","nonce":4011275120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:49:00.387 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 44 2023-11-09T03:49:00.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.388+0000 7ff9a15b6640 1 -- 172.21.15.76:0/2902319063 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff97c06aea0 msgr2=0x7ff97c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:00.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.388+0000 7ff9a15b6640 1 --2- 172.21.15.76:0/2902319063 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff97c06aea0 0x7ff97c06d360 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff98c036000 tx=0x7ff98c030000 comp rx=0 tx=0).stop 2023-11-09T03:49:00.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.388+0000 7ff9a15b6640 1 -- 172.21.15.76:0/2902319063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff99c0ce240 msgr2=0x7ff99c14d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:00.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.388+0000 7ff9a15b6640 1 --2- 172.21.15.76:0/2902319063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff99c0ce240 0x7ff99c14d360 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7ff98400c410 tx=0x7ff98400c8d0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.388+0000 7ff9a15b6640 1 -- 172.21.15.76:0/2902319063 shutdown_connections 2023-11-09T03:49:00.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.388+0000 7ff9a15b6640 1 --2- 172.21.15.76:0/2902319063 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff97c06aea0 0x7ff97c06d360 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.388+0000 7ff9a15b6640 1 --2- 172.21.15.76:0/2902319063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff99c15daa0 0x7ff99c15c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.388+0000 7ff9a15b6640 1 --2- 172.21.15.76:0/2902319063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff99c0ce240 0x7ff99c14d360 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.388+0000 7ff9a15b6640 1 --2- 172.21.15.76:0/2902319063 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff99c0cde60 0x7ff99c1f2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.388+0000 7ff9a15b6640 1 -- 172.21.15.76:0/2902319063 >> 172.21.15.76:0/2902319063 conn(0x7ff99c071f30 msgr2=0x7ff99c156530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:00.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.388+0000 7ff9a15b6640 1 -- 172.21.15.76:0/2902319063 shutdown_connections 2023-11-09T03:49:00.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.388+0000 7ff9a15b6640 1 -- 172.21.15.76:0/2902319063 wait complete. 2023-11-09T03:49:00.399 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:49:00.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.499+0000 7f86ffdc0640 1 Processor -- start 2023-11-09T03:49:00.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.499+0000 7f86ffdc0640 1 -- start start 2023-11-09T03:49:00.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.500+0000 7f86ffdc0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f86f8159c70 0x7f86f80ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:00.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.500+0000 7f86ffdc0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f80b4110 con 0x7f86f806db20 2023-11-09T03:49:00.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.500+0000 7f86ffdc0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f80b43f0 con 0x7f86f80b0ae0 2023-11-09T03:49:00.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.500+0000 7f86ffdc0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f80b49b0 con 0x7f86f8159c70 2023-11-09T03:49:00.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.500+0000 7f86fdb5c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f86f8159c70 0x7f86f80ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:00.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.500+0000 7f86fd35b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f86f806db20 0x7f86f80b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35904/0 (socket says 172.21.15.76:35904) 2023-11-09T03:49:00.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.500+0000 7f86fd35b640 1 -- 172.21.15.76:0/1512386491 learned_addr learned my addr 172.21.15.76:0/1512386491 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:49:00.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.501+0000 7f86fdb5c640 1 -- 172.21.15.76:0/1512386491 >> v1:172.21.15.113:6789/0 conn(0x7f86f80b0ae0 legacy=0x7f86f815d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:00.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.501+0000 7f86fdb5c640 1 -- 172.21.15.76:0/1512386491 >> v1:172.21.15.76:6789/0 conn(0x7f86f806db20 legacy=0x7f86f80b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:00.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.501+0000 7f86fdb5c640 1 -- 172.21.15.76:0/1512386491 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f80b46d0 con 0x7f86f8159c70 2023-11-09T03:49:00.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.501+0000 7f86fdb5c640 1 --2- 172.21.15.76:0/1512386491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86f8159c70 0x7f86f80ce370 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009d90 tx=0x7f86e802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6674de20eb74c9e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:00.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.501+0000 7f86fcb5a640 1 -- 172.21.15.76:0/1512386491 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e802cc60 con 0x7f86f8159c70 2023-11-09T03:49:00.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.501+0000 7f86fcb5a640 1 -- 172.21.15.76:0/1512386491 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e802cdc0 con 0x7f86f8159c70 2023-11-09T03:49:00.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.501+0000 7f86fcb5a640 1 -- 172.21.15.76:0/1512386491 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e8034660 con 0x7f86f8159c70 2023-11-09T03:49:00.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.502+0000 7f86ffdc0640 1 -- 172.21.15.76:0/1512386491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86f8159c70 msgr2=0x7f86f80ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:00.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.502+0000 7f86ffdc0640 1 --2- 172.21.15.76:0/1512386491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86f8159c70 0x7f86f80ce370 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009d90 tx=0x7f86e802beb0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.502+0000 7f86ffdc0640 1 -- 172.21.15.76:0/1512386491 shutdown_connections 2023-11-09T03:49:00.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.502+0000 7f86ffdc0640 1 --2- 172.21.15.76:0/1512386491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86f8159c70 0x7f86f80ce370 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.502+0000 7f86ffdc0640 1 -- 172.21.15.76:0/1512386491 >> 172.21.15.76:0/1512386491 conn(0x7f86f80ce950 msgr2=0x7f86f80ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:00.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.502+0000 7f86ffdc0640 1 -- 172.21.15.76:0/1512386491 shutdown_connections 2023-11-09T03:49:00.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.502+0000 7f86ffdc0640 1 -- 172.21.15.76:0/1512386491 wait complete. 2023-11-09T03:49:00.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.503+0000 7f86ffdc0640 1 Processor -- start 2023-11-09T03:49:00.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.503+0000 7f86ffdc0640 1 -- start start 2023-11-09T03:49:00.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.504+0000 7f86ffdc0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86f806db20 0x7f86f81f24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:00.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.504+0000 7f86ffdc0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86f80b0ae0 0x7f86f80c5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:00.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.504+0000 7f86ffdc0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f86f8159c70 0x7f86f81521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:00.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.504+0000 7f86ffdc0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f8151140 con 0x7f86f80b0ae0 2023-11-09T03:49:00.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.504+0000 7f86ffdc0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f8151420 con 0x7f86f806db20 2023-11-09T03:49:00.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.504+0000 7f86ffdc0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f8151700 con 0x7f86f8159c70 2023-11-09T03:49:00.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.504+0000 7f86fd35b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86f80b0ae0 0x7f86f80c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:00.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.504+0000 7f86fd35b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86f80b0ae0 0x7f86f80c5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44176/0 (socket says 172.21.15.76:44176) 2023-11-09T03:49:00.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.504+0000 7f86fd35b640 1 -- 172.21.15.76:0/653393542 learned_addr learned my addr 172.21.15.76:0/653393542 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:00.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.504+0000 7f86fdb5c640 1 --2- 172.21.15.76:0/653393542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86f806db20 0x7f86f81f24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:00.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.504+0000 7f86fd35b640 1 -- 172.21.15.76:0/653393542 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f86f8159c70 msgr2=0x7f86f81521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:00.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.504+0000 7f86fe35d640 1 --2- 172.21.15.76:0/653393542 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f86f8159c70 0x7f86f81521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:00.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.504+0000 7f86fd35b640 1 --2- 172.21.15.76:0/653393542 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f86f8159c70 0x7f86f81521e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.504+0000 7f86fd35b640 1 -- 172.21.15.76:0/653393542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86f806db20 msgr2=0x7f86f81f24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:00.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.504+0000 7f86fd35b640 1 --2- 172.21.15.76:0/653393542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86f806db20 0x7f86f81f24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.504+0000 7f86fd35b640 1 -- 172.21.15.76:0/653393542 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e8009a70 con 0x7f86f80b0ae0 2023-11-09T03:49:00.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.504+0000 7f86fdb5c640 1 --2- 172.21.15.76:0/653393542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86f806db20 0x7f86f81f24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:49:00.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.505+0000 7f86fd35b640 1 --2- 172.21.15.76:0/653393542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86f80b0ae0 0x7f86f80c5990 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00c4a0 tx=0x7f86ec00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:00.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.505+0000 7f86f67fc640 1 -- 172.21.15.76:0/653393542 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec005ce0 con 0x7f86f80b0ae0 2023-11-09T03:49:00.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.505+0000 7f86f67fc640 1 -- 172.21.15.76:0/653393542 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86ec005e40 con 0x7f86f80b0ae0 2023-11-09T03:49:00.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.505+0000 7f86ffdc0640 1 -- 172.21.15.76:0/653393542 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f81519e0 con 0x7f86f80b0ae0 2023-11-09T03:49:00.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.505+0000 7f86f67fc640 1 -- 172.21.15.76:0/653393542 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec007270 con 0x7f86f80b0ae0 2023-11-09T03:49:00.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.505+0000 7f86ffdc0640 1 -- 172.21.15.76:0/653393542 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f81f2a20 con 0x7f86f80b0ae0 2023-11-09T03:49:00.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.506+0000 7f86ffdc0640 1 -- 172.21.15.76:0/653393542 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f8151f00 con 0x7f86f80b0ae0 2023-11-09T03:49:00.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.506+0000 7f86f67fc640 1 -- 172.21.15.76:0/653393542 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f86ec007490 con 0x7f86f80b0ae0 2023-11-09T03:49:00.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.507+0000 7f86f67fc640 1 --2- 172.21.15.76:0/653393542 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f86e406ae70 0x7f86e406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:00.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.507+0000 7f86f67fc640 1 -- 172.21.15.76:0/653393542 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f86ec088fa0 con 0x7f86f80b0ae0 2023-11-09T03:49:00.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.507+0000 7f86fdb5c640 1 --2- 172.21.15.76:0/653393542 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f86e406ae70 0x7f86e406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:00.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.507+0000 7f86fdb5c640 1 --2- 172.21.15.76:0/653393542 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f86e406ae70 0x7f86e406d330 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f86e8036000 tx=0x7f86e8030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:00.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.511+0000 7f86f67fc640 1 -- 172.21.15.76:0/653393542 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f86ec0bbca0 con 0x7f86f80b0ae0 2023-11-09T03:49:00.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.677+0000 7f86ffdc0640 1 -- 172.21.15.76:0/653393542 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86f8074290 con 0x7f86f80b0ae0 2023-11-09T03:49:00.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.678+0000 7f86f67fc640 1 -- 172.21.15.76:0/653393542 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+21993 (secure 0 0 0) 0x7f86ec017020 con 0x7f86f80b0ae0 2023-11-09T03:49:00.678 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:49:00.678 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":95,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:48:58.423703+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-09T03:47:59.280601+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-09T03:47:59.742039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:48:55.490832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-09T03:48:56.352624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:49:00.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.680+0000 7f86ffdc0640 1 -- 172.21.15.76:0/653393542 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f86e406ae70 msgr2=0x7f86e406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:00.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.680+0000 7f86ffdc0640 1 --2- 172.21.15.76:0/653393542 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f86e406ae70 0x7f86e406d330 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f86e8036000 tx=0x7f86e8030000 comp rx=0 tx=0).stop 2023-11-09T03:49:00.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.680+0000 7f86ffdc0640 1 -- 172.21.15.76:0/653393542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86f80b0ae0 msgr2=0x7f86f80c5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:00.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.680+0000 7f86ffdc0640 1 --2- 172.21.15.76:0/653393542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86f80b0ae0 0x7f86f80c5990 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00c4a0 tx=0x7f86ec00c960 comp rx=0 tx=0).stop 2023-11-09T03:49:00.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.680+0000 7f86ffdc0640 1 -- 172.21.15.76:0/653393542 shutdown_connections 2023-11-09T03:49:00.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.680+0000 7f86ffdc0640 1 --2- 172.21.15.76:0/653393542 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f86e406ae70 0x7f86e406d330 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.680+0000 7f86ffdc0640 1 --2- 172.21.15.76:0/653393542 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f86f8159c70 0x7f86f81521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.680+0000 7f86ffdc0640 1 --2- 172.21.15.76:0/653393542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86f80b0ae0 0x7f86f80c5990 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.680+0000 7f86ffdc0640 1 --2- 172.21.15.76:0/653393542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86f806db20 0x7f86f81f24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.680+0000 7f86ffdc0640 1 -- 172.21.15.76:0/653393542 >> 172.21.15.76:0/653393542 conn(0x7f86f80ce950 msgr2=0x7f86f80c8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:00.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.680+0000 7f86ffdc0640 1 -- 172.21.15.76:0/653393542 shutdown_connections 2023-11-09T03:49:00.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.680+0000 7f86ffdc0640 1 -- 172.21.15.76:0/653393542 wait complete. 2023-11-09T03:49:00.691 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-09T03:49:00.691 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-09T03:49:00.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.791+0000 7f8b9064f640 1 Processor -- start 2023-11-09T03:49:00.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.791+0000 7f8b9064f640 1 -- start start 2023-11-09T03:49:00.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.792+0000 7f8b9064f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8b88159c70 0x7f8b880ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:00.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.792+0000 7f8b9064f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b8806db20 0x7f8b880b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:00.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.792+0000 7f8b9064f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b880b4110 con 0x7f8b880b0910 2023-11-09T03:49:00.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.792+0000 7f8b9064f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b880b49b0 con 0x7f8b88159c70 2023-11-09T03:49:00.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.792+0000 7f8b9064f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b880b0da0 con 0x7f8b8806db20 2023-11-09T03:49:00.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.792+0000 7f8b8e3eb640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8b88159c70 0x7f8b880ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:00.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.792+0000 7f8b8e3eb640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8b88159c70 0x7f8b880ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44178/0 (socket says 172.21.15.76:44178) 2023-11-09T03:49:00.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.792+0000 7f8b8e3eb640 1 -- 172.21.15.76:0/3200914718 learned_addr learned my addr 172.21.15.76:0/3200914718 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:00.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.792+0000 7f8b8dbea640 1 --2- 172.21.15.76:0/3200914718 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b8806db20 0x7f8b880b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:00.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.792+0000 7f8b8e3eb640 1 -- 172.21.15.76:0/3200914718 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b8806db20 msgr2=0x7f8b880b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:00.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.792+0000 7f8b8e3eb640 1 --2- 172.21.15.76:0/3200914718 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b8806db20 0x7f8b880b03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.792+0000 7f8b8e3eb640 1 -- 172.21.15.76:0/3200914718 >> v1:172.21.15.113:6789/0 conn(0x7f8b880b0910 legacy=0x7f8b8815d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:00.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.792+0000 7f8b8e3eb640 1 -- 172.21.15.76:0/3200914718 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b880b43f0 con 0x7f8b88159c70 2023-11-09T03:49:00.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.792+0000 7f8b8dbea640 1 --2- 172.21.15.76:0/3200914718 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b8806db20 0x7f8b880b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:49:00.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.793+0000 7f8b8e3eb640 1 --2- 172.21.15.76:0/3200914718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b88159c70 0x7f8b880ce370 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f8b84009870 tx=0x7f8b8402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a74fd7cc3aff70 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:00.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.793+0000 7f8b8d3e9640 1 -- 172.21.15.76:0/3200914718 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b84030840 con 0x7f8b88159c70 2023-11-09T03:49:00.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.793+0000 7f8b8d3e9640 1 -- 172.21.15.76:0/3200914718 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b840309a0 con 0x7f8b88159c70 2023-11-09T03:49:00.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.793+0000 7f8b8d3e9640 1 -- 172.21.15.76:0/3200914718 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b84030d30 con 0x7f8b88159c70 2023-11-09T03:49:00.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.793+0000 7f8b9064f640 1 -- 172.21.15.76:0/3200914718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b88159c70 msgr2=0x7f8b880ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:00.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.793+0000 7f8b9064f640 1 --2- 172.21.15.76:0/3200914718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b88159c70 0x7f8b880ce370 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f8b84009870 tx=0x7f8b8402b980 comp rx=0 tx=0).stop 2023-11-09T03:49:00.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.794+0000 7f8b9064f640 1 -- 172.21.15.76:0/3200914718 shutdown_connections 2023-11-09T03:49:00.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.794+0000 7f8b9064f640 1 --2- 172.21.15.76:0/3200914718 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b8806db20 0x7f8b880b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.794+0000 7f8b9064f640 1 --2- 172.21.15.76:0/3200914718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b88159c70 0x7f8b880ce370 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.794+0000 7f8b9064f640 1 -- 172.21.15.76:0/3200914718 >> 172.21.15.76:0/3200914718 conn(0x7f8b880ce950 msgr2=0x7f8b880ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:00.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.794+0000 7f8b9064f640 1 -- 172.21.15.76:0/3200914718 shutdown_connections 2023-11-09T03:49:00.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.794+0000 7f8b9064f640 1 -- 172.21.15.76:0/3200914718 wait complete. 2023-11-09T03:49:00.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.795+0000 7f8b9064f640 1 Processor -- start 2023-11-09T03:49:00.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.795+0000 7f8b9064f640 1 -- start start 2023-11-09T03:49:00.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.795+0000 7f8b9064f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b8806db20 0x7f8b880b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:00.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.795+0000 7f8b9064f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b880b0910 0x7f8b880adae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:00.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.795+0000 7f8b9064f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b88159c70 0x7f8b880b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:00.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.795+0000 7f8b9064f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b880b1cd0 con 0x7f8b88159c70 2023-11-09T03:49:00.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.795+0000 7f8b9064f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b880ae020 con 0x7f8b880b0910 2023-11-09T03:49:00.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.795+0000 7f8b9064f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b880b3230 con 0x7f8b8806db20 2023-11-09T03:49:00.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.796+0000 7f8b8ebec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b88159c70 0x7f8b880b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:00.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.796+0000 7f8b8ebec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b88159c70 0x7f8b880b2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44186/0 (socket says 172.21.15.76:44186) 2023-11-09T03:49:00.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.796+0000 7f8b8ebec640 1 -- 172.21.15.76:0/2651582705 learned_addr learned my addr 172.21.15.76:0/2651582705 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:00.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.796+0000 7f8b8dbea640 1 --2- 172.21.15.76:0/2651582705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b880b0910 0x7f8b880adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:00.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.796+0000 7f8b8e3eb640 1 --2- 172.21.15.76:0/2651582705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b8806db20 0x7f8b880b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:00.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.796+0000 7f8b8ebec640 1 -- 172.21.15.76:0/2651582705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b8806db20 msgr2=0x7f8b880b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:00.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.796+0000 7f8b8ebec640 1 --2- 172.21.15.76:0/2651582705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b8806db20 0x7f8b880b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.796+0000 7f8b8ebec640 1 -- 172.21.15.76:0/2651582705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b880b0910 msgr2=0x7f8b880adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:00.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.796+0000 7f8b8ebec640 1 --2- 172.21.15.76:0/2651582705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b880b0910 0x7f8b880adae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.796+0000 7f8b8ebec640 1 -- 172.21.15.76:0/2651582705 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b840094d0 con 0x7f8b88159c70 2023-11-09T03:49:00.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.796+0000 7f8b8ebec640 1 --2- 172.21.15.76:0/2651582705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b88159c70 0x7f8b880b2950 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000c7a0 tx=0x7f8b8000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:00.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.796+0000 7f8b7effd640 1 -- 172.21.15.76:0/2651582705 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b80003b70 con 0x7f8b88159c70 2023-11-09T03:49:00.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.796+0000 7f8b7effd640 1 -- 172.21.15.76:0/2651582705 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b80003cf0 con 0x7f8b88159c70 2023-11-09T03:49:00.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.796+0000 7f8b9064f640 1 -- 172.21.15.76:0/2651582705 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b880b3510 con 0x7f8b88159c70 2023-11-09T03:49:00.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.796+0000 7f8b7effd640 1 -- 172.21.15.76:0/2651582705 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b80003ea0 con 0x7f8b88159c70 2023-11-09T03:49:00.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.796+0000 7f8b9064f640 1 -- 172.21.15.76:0/2651582705 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b880b18f0 con 0x7f8b88159c70 2023-11-09T03:49:00.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.797+0000 7f8b9064f640 1 -- 172.21.15.76:0/2651582705 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b880b1e10 con 0x7f8b88159c70 2023-11-09T03:49:00.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.798+0000 7f8b7effd640 1 -- 172.21.15.76:0/2651582705 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8b8000aa90 con 0x7f8b88159c70 2023-11-09T03:49:00.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.798+0000 7f8b7effd640 1 --2- 172.21.15.76:0/2651582705 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8b6c06ae70 0x7f8b6c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:00.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.798+0000 7f8b7effd640 1 -- 172.21.15.76:0/2651582705 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f8b800897d0 con 0x7f8b88159c70 2023-11-09T03:49:00.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.798+0000 7f8b8e3eb640 1 --2- 172.21.15.76:0/2651582705 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8b6c06ae70 0x7f8b6c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:00.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.799+0000 7f8b8e3eb640 1 --2- 172.21.15.76:0/2651582705 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8b6c06ae70 0x7f8b6c06d330 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8b84036000 tx=0x7f8b84008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:00.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.802+0000 7f8b7effd640 1 -- 172.21.15.76:0/2651582705 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8b880b1e10 con 0x7f8b88159c70 2023-11-09T03:49:00.963 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.963+0000 7f8b9064f640 1 -- 172.21.15.76:0/2651582705 --> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8b880cdb70 con 0x7f8b6c06ae70 2023-11-09T03:49:00.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.977+0000 7f8b7effd640 1 -- 172.21.15.76:0/2651582705 <== mgr.4105 v2:172.21.15.113:6832/1264193042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+531943 (secure 0 0 0) 0x7f8b84039450 con 0x7f8b6c06ae70 2023-11-09T03:49:00.981 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:49:00.985 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2023-11-09T03:49:00.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.981+0000 7f8b9064f640 1 -- 172.21.15.76:0/2651582705 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8b6c06ae70 msgr2=0x7f8b6c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:00.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.981+0000 7f8b9064f640 1 --2- 172.21.15.76:0/2651582705 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8b6c06ae70 0x7f8b6c06d330 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8b84036000 tx=0x7f8b84008030 comp rx=0 tx=0).stop 2023-11-09T03:49:00.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.981+0000 7f8b9064f640 1 -- 172.21.15.76:0/2651582705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b88159c70 msgr2=0x7f8b880b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:00.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.981+0000 7f8b9064f640 1 --2- 172.21.15.76:0/2651582705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b88159c70 0x7f8b880b2950 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000c7a0 tx=0x7f8b8000cc60 comp rx=0 tx=0).stop 2023-11-09T03:49:00.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.981+0000 7f8b9064f640 1 -- 172.21.15.76:0/2651582705 shutdown_connections 2023-11-09T03:49:00.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.981+0000 7f8b9064f640 1 --2- 172.21.15.76:0/2651582705 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8b6c06ae70 0x7f8b6c06d330 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.981+0000 7f8b9064f640 1 --2- 172.21.15.76:0/2651582705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b88159c70 0x7f8b880b2950 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.981+0000 7f8b9064f640 1 --2- 172.21.15.76:0/2651582705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b880b0910 0x7f8b880adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.981+0000 7f8b9064f640 1 --2- 172.21.15.76:0/2651582705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8b8806db20 0x7f8b880b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:00.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.981+0000 7f8b9064f640 1 -- 172.21.15.76:0/2651582705 >> 172.21.15.76:0/2651582705 conn(0x7f8b880ce950 msgr2=0x7f8b880ce210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:00.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.981+0000 7f8b9064f640 1 -- 172.21.15.76:0/2651582705 shutdown_connections 2023-11-09T03:49:00.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:00.981+0000 7f8b9064f640 1 -- 172.21.15.76:0/2651582705 wait complete. 2023-11-09T03:49:00.991 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":200,"stamp":"2023-11-09T03:49:00.804277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":268,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":251236,"kb_used_data":10092,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754723484,"statfs":{"total":773094113280,"available":772836847616,"internally_reserved":0,"allocated":10334208,"data_stored":5666273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.774536"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.447264+0000","last_change":"2023-11-09T03:48:57.445674+0000","last_active":"2023-11-09T03:48:57.447264+0000","last_peered":"2023-11-09T03:48:57.447264+0000","last_clean":"2023-11-09T03:48:57.447264+0000","last_became_active":"2023-11-09T03:48:57.445271+0000","last_became_peered":"2023-11-09T03:48:57.445271+0000","last_unstale":"2023-11-09T03:48:57.447264+0000","last_undegraded":"2023-11-09T03:48:57.447264+0000","last_fullsized":"2023-11-09T03:48:57.447264+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:28.967017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.441747+0000","last_change":"2023-11-09T03:48:57.439578+0000","last_active":"2023-11-09T03:48:57.441747+0000","last_peered":"2023-11-09T03:48:57.441747+0000","last_clean":"2023-11-09T03:48:57.441747+0000","last_became_active":"2023-11-09T03:48:57.438979+0000","last_became_peered":"2023-11-09T03:48:57.438979+0000","last_unstale":"2023-11-09T03:48:57.441747+0000","last_undegraded":"2023-11-09T03:48:57.441747+0000","last_fullsized":"2023-11-09T03:48:57.441747+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:21:31.419099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.453559+0000","last_change":"2023-11-09T03:48:57.447498+0000","last_active":"2023-11-09T03:48:57.453559+0000","last_peered":"2023-11-09T03:48:57.453559+0000","last_clean":"2023-11-09T03:48:57.453559+0000","last_became_active":"2023-11-09T03:48:57.447223+0000","last_became_peered":"2023-11-09T03:48:57.447223+0000","last_unstale":"2023-11-09T03:48:57.453559+0000","last_undegraded":"2023-11-09T03:48:57.453559+0000","last_fullsized":"2023-11-09T03:48:57.453559+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:02.054568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.437439+0000","last_change":"2023-11-09T03:48:57.434039+0000","last_active":"2023-11-09T03:48:57.437439+0000","last_peered":"2023-11-09T03:48:57.437439+0000","last_clean":"2023-11-09T03:48:57.437439+0000","last_became_active":"2023-11-09T03:48:57.433770+0000","last_became_peered":"2023-11-09T03:48:57.433770+0000","last_unstale":"2023-11-09T03:48:57.437439+0000","last_undegraded":"2023-11-09T03:48:57.437439+0000","last_fullsized":"2023-11-09T03:48:57.437439+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:51:42.409861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.444702+0000","last_change":"2023-11-09T03:48:57.443545+0000","last_active":"2023-11-09T03:48:57.444702+0000","last_peered":"2023-11-09T03:48:57.444702+0000","last_clean":"2023-11-09T03:48:57.444702+0000","last_became_active":"2023-11-09T03:48:57.443290+0000","last_became_peered":"2023-11-09T03:48:57.443290+0000","last_unstale":"2023-11-09T03:48:57.444702+0000","last_undegraded":"2023-11-09T03:48:57.444702+0000","last_fullsized":"2023-11-09T03:48:57.444702+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:00:56.307904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.446704+0000","last_change":"2023-11-09T03:48:57.442256+0000","last_active":"2023-11-09T03:48:57.446704+0000","last_peered":"2023-11-09T03:48:57.446704+0000","last_clean":"2023-11-09T03:48:57.446704+0000","last_became_active":"2023-11-09T03:48:57.442016+0000","last_became_peered":"2023-11-09T03:48:57.442016+0000","last_unstale":"2023-11-09T03:48:57.446704+0000","last_undegraded":"2023-11-09T03:48:57.446704+0000","last_fullsized":"2023-11-09T03:48:57.446704+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:08.671141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.448714+0000","last_change":"2023-11-09T03:48:57.445253+0000","last_active":"2023-11-09T03:48:57.448714+0000","last_peered":"2023-11-09T03:48:57.448714+0000","last_clean":"2023-11-09T03:48:57.448714+0000","last_became_active":"2023-11-09T03:48:57.444960+0000","last_became_peered":"2023-11-09T03:48:57.444960+0000","last_unstale":"2023-11-09T03:48:57.448714+0000","last_undegraded":"2023-11-09T03:48:57.448714+0000","last_fullsized":"2023-11-09T03:48:57.448714+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:46:17.736233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.451322+0000","last_change":"2023-11-09T03:48:57.446239+0000","last_active":"2023-11-09T03:48:57.451322+0000","last_peered":"2023-11-09T03:48:57.451322+0000","last_clean":"2023-11-09T03:48:57.451322+0000","last_became_active":"2023-11-09T03:48:57.445925+0000","last_became_peered":"2023-11-09T03:48:57.445925+0000","last_unstale":"2023-11-09T03:48:57.451322+0000","last_undegraded":"2023-11-09T03:48:57.451322+0000","last_fullsized":"2023-11-09T03:48:57.451322+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:56:57.961469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.437491+0000","last_change":"2023-11-09T03:48:57.433879+0000","last_active":"2023-11-09T03:48:57.437491+0000","last_peered":"2023-11-09T03:48:57.437491+0000","last_clean":"2023-11-09T03:48:57.437491+0000","last_became_active":"2023-11-09T03:48:57.433017+0000","last_became_peered":"2023-11-09T03:48:57.433017+0000","last_unstale":"2023-11-09T03:48:57.437491+0000","last_undegraded":"2023-11-09T03:48:57.437491+0000","last_fullsized":"2023-11-09T03:48:57.437491+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:15:04.864360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.440254+0000","last_change":"2023-11-09T03:48:57.438289+0000","last_active":"2023-11-09T03:48:57.440254+0000","last_peered":"2023-11-09T03:48:57.440254+0000","last_clean":"2023-11-09T03:48:57.440254+0000","last_became_active":"2023-11-09T03:48:57.438043+0000","last_became_peered":"2023-11-09T03:48:57.438043+0000","last_unstale":"2023-11-09T03:48:57.440254+0000","last_undegraded":"2023-11-09T03:48:57.440254+0000","last_fullsized":"2023-11-09T03:48:57.440254+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:13:49.228169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.444154+0000","last_change":"2023-11-09T03:48:57.441291+0000","last_active":"2023-11-09T03:48:57.444154+0000","last_peered":"2023-11-09T03:48:57.444154+0000","last_clean":"2023-11-09T03:48:57.444154+0000","last_became_active":"2023-11-09T03:48:57.441042+0000","last_became_peered":"2023-11-09T03:48:57.441042+0000","last_unstale":"2023-11-09T03:48:57.444154+0000","last_undegraded":"2023-11-09T03:48:57.444154+0000","last_fullsized":"2023-11-09T03:48:57.444154+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:04:31.603691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.441376+0000","last_change":"2023-11-09T03:48:57.438303+0000","last_active":"2023-11-09T03:48:57.441376+0000","last_peered":"2023-11-09T03:48:57.441376+0000","last_clean":"2023-11-09T03:48:57.441376+0000","last_became_active":"2023-11-09T03:48:57.438058+0000","last_became_peered":"2023-11-09T03:48:57.438058+0000","last_unstale":"2023-11-09T03:48:57.441376+0000","last_undegraded":"2023-11-09T03:48:57.441376+0000","last_fullsized":"2023-11-09T03:48:57.441376+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:58:45.939038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.447198+0000","last_change":"2023-11-09T03:48:57.439889+0000","last_active":"2023-11-09T03:48:57.447198+0000","last_peered":"2023-11-09T03:48:57.447198+0000","last_clean":"2023-11-09T03:48:57.447198+0000","last_became_active":"2023-11-09T03:48:57.439646+0000","last_became_peered":"2023-11-09T03:48:57.439646+0000","last_unstale":"2023-11-09T03:48:57.447198+0000","last_undegraded":"2023-11-09T03:48:57.447198+0000","last_fullsized":"2023-11-09T03:48:57.447198+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:55:52.899135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.451338+0000","last_change":"2023-11-09T03:48:57.446694+0000","last_active":"2023-11-09T03:48:57.451338+0000","last_peered":"2023-11-09T03:48:57.451338+0000","last_clean":"2023-11-09T03:48:57.451338+0000","last_became_active":"2023-11-09T03:48:57.446361+0000","last_became_peered":"2023-11-09T03:48:57.446361+0000","last_unstale":"2023-11-09T03:48:57.451338+0000","last_undegraded":"2023-11-09T03:48:57.451338+0000","last_fullsized":"2023-11-09T03:48:57.451338+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:03.847302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.449453+0000","last_change":"2023-11-09T03:48:57.444296+0000","last_active":"2023-11-09T03:48:57.449453+0000","last_peered":"2023-11-09T03:48:57.449453+0000","last_clean":"2023-11-09T03:48:57.449453+0000","last_became_active":"2023-11-09T03:48:57.444032+0000","last_became_peered":"2023-11-09T03:48:57.444032+0000","last_unstale":"2023-11-09T03:48:57.449453+0000","last_undegraded":"2023-11-09T03:48:57.449453+0000","last_fullsized":"2023-11-09T03:48:57.449453+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:18:20.971683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.444183+0000","last_change":"2023-11-09T03:48:57.440405+0000","last_active":"2023-11-09T03:48:57.444183+0000","last_peered":"2023-11-09T03:48:57.444183+0000","last_clean":"2023-11-09T03:48:57.444183+0000","last_became_active":"2023-11-09T03:48:57.440158+0000","last_became_peered":"2023-11-09T03:48:57.440158+0000","last_unstale":"2023-11-09T03:48:57.444183+0000","last_undegraded":"2023-11-09T03:48:57.444183+0000","last_fullsized":"2023-11-09T03:48:57.444183+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:09:28.936062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.448792+0000","last_change":"2023-11-09T03:48:57.446115+0000","last_active":"2023-11-09T03:48:57.448792+0000","last_peered":"2023-11-09T03:48:57.448792+0000","last_clean":"2023-11-09T03:48:57.448792+0000","last_became_active":"2023-11-09T03:48:57.445824+0000","last_became_peered":"2023-11-09T03:48:57.445824+0000","last_unstale":"2023-11-09T03:48:57.448792+0000","last_undegraded":"2023-11-09T03:48:57.448792+0000","last_fullsized":"2023-11-09T03:48:57.448792+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:01:16.444216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.444553+0000","last_change":"2023-11-09T03:48:57.439366+0000","last_active":"2023-11-09T03:48:57.444553+0000","last_peered":"2023-11-09T03:48:57.444553+0000","last_clean":"2023-11-09T03:48:57.444553+0000","last_became_active":"2023-11-09T03:48:57.439114+0000","last_became_peered":"2023-11-09T03:48:57.439114+0000","last_unstale":"2023-11-09T03:48:57.444553+0000","last_undegraded":"2023-11-09T03:48:57.444553+0000","last_fullsized":"2023-11-09T03:48:57.444553+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:01:16.444216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.447696+0000","last_change":"2023-11-09T03:48:57.441391+0000","last_active":"2023-11-09T03:48:57.447696+0000","last_peered":"2023-11-09T03:48:57.447696+0000","last_clean":"2023-11-09T03:48:57.447696+0000","last_became_active":"2023-11-09T03:48:57.441094+0000","last_became_peered":"2023-11-09T03:48:57.441094+0000","last_unstale":"2023-11-09T03:48:57.447696+0000","last_undegraded":"2023-11-09T03:48:57.447696+0000","last_fullsized":"2023-11-09T03:48:57.447696+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:20:58.173238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.440932+0000","last_change":"2023-11-09T03:48:57.438369+0000","last_active":"2023-11-09T03:48:57.440932+0000","last_peered":"2023-11-09T03:48:57.440932+0000","last_clean":"2023-11-09T03:48:57.440932+0000","last_became_active":"2023-11-09T03:48:57.438089+0000","last_became_peered":"2023-11-09T03:48:57.438089+0000","last_unstale":"2023-11-09T03:48:57.440932+0000","last_undegraded":"2023-11-09T03:48:57.440932+0000","last_fullsized":"2023-11-09T03:48:57.440932+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:49:52.205154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.436973+0000","last_change":"2023-11-09T03:48:57.432344+0000","last_active":"2023-11-09T03:48:57.436973+0000","last_peered":"2023-11-09T03:48:57.436973+0000","last_clean":"2023-11-09T03:48:57.436973+0000","last_became_active":"2023-11-09T03:48:57.432077+0000","last_became_peered":"2023-11-09T03:48:57.432077+0000","last_unstale":"2023-11-09T03:48:57.436973+0000","last_undegraded":"2023-11-09T03:48:57.436973+0000","last_fullsized":"2023-11-09T03:48:57.436973+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:51:18.482982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.443737+0000","last_change":"2023-11-09T03:48:57.438613+0000","last_active":"2023-11-09T03:48:57.443737+0000","last_peered":"2023-11-09T03:48:57.443737+0000","last_clean":"2023-11-09T03:48:57.443737+0000","last_became_active":"2023-11-09T03:48:57.438367+0000","last_became_peered":"2023-11-09T03:48:57.438367+0000","last_unstale":"2023-11-09T03:48:57.443737+0000","last_undegraded":"2023-11-09T03:48:57.443737+0000","last_fullsized":"2023-11-09T03:48:57.443737+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:02.054568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.439194+0000","last_change":"2023-11-09T03:48:57.436456+0000","last_active":"2023-11-09T03:48:57.439194+0000","last_peered":"2023-11-09T03:48:57.439194+0000","last_clean":"2023-11-09T03:48:57.439194+0000","last_became_active":"2023-11-09T03:48:57.436172+0000","last_became_peered":"2023-11-09T03:48:57.436172+0000","last_unstale":"2023-11-09T03:48:57.439194+0000","last_undegraded":"2023-11-09T03:48:57.439194+0000","last_fullsized":"2023-11-09T03:48:57.439194+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:28:01.062126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.443860+0000","last_change":"2023-11-09T03:48:57.440743+0000","last_active":"2023-11-09T03:48:57.443860+0000","last_peered":"2023-11-09T03:48:57.443860+0000","last_clean":"2023-11-09T03:48:57.443860+0000","last_became_active":"2023-11-09T03:48:57.440480+0000","last_became_peered":"2023-11-09T03:48:57.440480+0000","last_unstale":"2023-11-09T03:48:57.443860+0000","last_undegraded":"2023-11-09T03:48:57.443860+0000","last_fullsized":"2023-11-09T03:48:57.443860+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:00:55.763662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.446738+0000","last_change":"2023-11-09T03:48:57.442454+0000","last_active":"2023-11-09T03:48:57.446738+0000","last_peered":"2023-11-09T03:48:57.446738+0000","last_clean":"2023-11-09T03:48:57.446738+0000","last_became_active":"2023-11-09T03:48:57.442179+0000","last_became_peered":"2023-11-09T03:48:57.442179+0000","last_unstale":"2023-11-09T03:48:57.446738+0000","last_undegraded":"2023-11-09T03:48:57.446738+0000","last_fullsized":"2023-11-09T03:48:57.446738+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:59:11.986476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.444226+0000","last_change":"2023-11-09T03:48:57.441769+0000","last_active":"2023-11-09T03:48:57.444226+0000","last_peered":"2023-11-09T03:48:57.444226+0000","last_clean":"2023-11-09T03:48:57.444226+0000","last_became_active":"2023-11-09T03:48:57.441539+0000","last_became_peered":"2023-11-09T03:48:57.441539+0000","last_unstale":"2023-11-09T03:48:57.444226+0000","last_undegraded":"2023-11-09T03:48:57.444226+0000","last_fullsized":"2023-11-09T03:48:57.444226+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:03:06.129711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.442032+0000","last_change":"2023-11-09T03:48:57.439869+0000","last_active":"2023-11-09T03:48:57.442032+0000","last_peered":"2023-11-09T03:48:57.442032+0000","last_clean":"2023-11-09T03:48:57.442032+0000","last_became_active":"2023-11-09T03:48:57.439618+0000","last_became_peered":"2023-11-09T03:48:57.439618+0000","last_unstale":"2023-11-09T03:48:57.442032+0000","last_undegraded":"2023-11-09T03:48:57.442032+0000","last_fullsized":"2023-11-09T03:48:57.442032+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:33:11.940885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.445044+0000","last_change":"2023-11-09T03:48:57.443612+0000","last_active":"2023-11-09T03:48:57.445044+0000","last_peered":"2023-11-09T03:48:57.445044+0000","last_clean":"2023-11-09T03:48:57.445044+0000","last_became_active":"2023-11-09T03:48:57.443351+0000","last_became_peered":"2023-11-09T03:48:57.443351+0000","last_unstale":"2023-11-09T03:48:57.445044+0000","last_undegraded":"2023-11-09T03:48:57.445044+0000","last_fullsized":"2023-11-09T03:48:57.445044+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:25:37.963645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.439477+0000","last_change":"2023-11-09T03:48:57.437084+0000","last_active":"2023-11-09T03:48:57.439477+0000","last_peered":"2023-11-09T03:48:57.439477+0000","last_clean":"2023-11-09T03:48:57.439477+0000","last_became_active":"2023-11-09T03:48:57.436804+0000","last_became_peered":"2023-11-09T03:48:57.436804+0000","last_unstale":"2023-11-09T03:48:57.439477+0000","last_undegraded":"2023-11-09T03:48:57.439477+0000","last_fullsized":"2023-11-09T03:48:57.439477+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:25:46.922398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.446598+0000","last_change":"2023-11-09T03:48:57.443908+0000","last_active":"2023-11-09T03:48:57.446598+0000","last_peered":"2023-11-09T03:48:57.446598+0000","last_clean":"2023-11-09T03:48:57.446598+0000","last_became_active":"2023-11-09T03:48:57.443414+0000","last_became_peered":"2023-11-09T03:48:57.443414+0000","last_unstale":"2023-11-09T03:48:57.446598+0000","last_undegraded":"2023-11-09T03:48:57.446598+0000","last_fullsized":"2023-11-09T03:48:57.446598+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:29:29.623406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.444154+0000","last_change":"2023-11-09T03:48:57.442341+0000","last_active":"2023-11-09T03:48:57.444154+0000","last_peered":"2023-11-09T03:48:57.444154+0000","last_clean":"2023-11-09T03:48:57.444154+0000","last_became_active":"2023-11-09T03:48:57.442083+0000","last_became_peered":"2023-11-09T03:48:57.442083+0000","last_unstale":"2023-11-09T03:48:57.444154+0000","last_undegraded":"2023-11-09T03:48:57.444154+0000","last_fullsized":"2023-11-09T03:48:57.444154+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:38:27.818497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.446730+0000","last_change":"2023-11-09T03:48:57.442871+0000","last_active":"2023-11-09T03:48:57.446730+0000","last_peered":"2023-11-09T03:48:57.446730+0000","last_clean":"2023-11-09T03:48:57.446730+0000","last_became_active":"2023-11-09T03:48:57.442618+0000","last_became_peered":"2023-11-09T03:48:57.442618+0000","last_unstale":"2023-11-09T03:48:57.446730+0000","last_undegraded":"2023-11-09T03:48:57.446730+0000","last_fullsized":"2023-11-09T03:48:57.446730+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:54:30.889690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.442426+0000","last_change":"2023-11-09T03:48:57.439545+0000","last_active":"2023-11-09T03:48:57.442426+0000","last_peered":"2023-11-09T03:48:57.442426+0000","last_clean":"2023-11-09T03:48:57.442426+0000","last_became_active":"2023-11-09T03:48:57.438639+0000","last_became_peered":"2023-11-09T03:48:57.438639+0000","last_unstale":"2023-11-09T03:48:57.442426+0000","last_undegraded":"2023-11-09T03:48:57.442426+0000","last_fullsized":"2023-11-09T03:48:57.442426+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:21:56.045654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.435259+0000","last_change":"2023-11-09T03:48:57.051318+0000","last_active":"2023-11-09T03:48:57.435259+0000","last_peered":"2023-11-09T03:48:57.435259+0000","last_clean":"2023-11-09T03:48:57.435259+0000","last_became_active":"2023-11-09T03:48:57.051078+0000","last_became_peered":"2023-11-09T03:48:57.051078+0000","last_unstale":"2023-11-09T03:48:57.435259+0000","last_undegraded":"2023-11-09T03:48:57.435259+0000","last_fullsized":"2023-11-09T03:48:57.435259+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:55.963904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.042938+0000","last_change":"2023-11-09T03:48:01.449077+0000","last_active":"2023-11-09T03:48:57.042938+0000","last_peered":"2023-11-09T03:48:57.042938+0000","last_clean":"2023-11-09T03:48:57.042938+0000","last_became_active":"2023-11-09T03:48:01.448806+0000","last_became_peered":"2023-11-09T03:48:01.448806+0000","last_unstale":"2023-11-09T03:48:57.042938+0000","last_undegraded":"2023-11-09T03:48:57.042938+0000","last_fullsized":"2023-11-09T03:48:57.042938+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:16:00.220961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.047787+0000","last_change":"2023-11-09T03:48:02.611756+0000","last_active":"2023-11-09T03:48:57.047787+0000","last_peered":"2023-11-09T03:48:57.047787+0000","last_clean":"2023-11-09T03:48:57.047787+0000","last_became_active":"2023-11-09T03:48:00.445880+0000","last_became_peered":"2023-11-09T03:48:00.445880+0000","last_unstale":"2023-11-09T03:48:57.047787+0000","last_undegraded":"2023-11-09T03:48:57.047787+0000","last_fullsized":"2023-11-09T03:48:57.047787+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:15:35.281296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085514900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.037336+0000","last_change":"2023-11-09T03:47:02.581917+0000","last_active":"2023-11-09T03:48:57.037336+0000","last_peered":"2023-11-09T03:48:57.037336+0000","last_clean":"2023-11-09T03:48:57.037336+0000","last_became_active":"2023-11-09T03:47:01.588082+0000","last_became_peered":"2023-11-09T03:47:01.588082+0000","last_unstale":"2023-11-09T03:48:57.037336+0000","last_undegraded":"2023-11-09T03:48:57.037336+0000","last_fullsized":"2023-11-09T03:48:57.037336+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:44:07.653557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034538899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.447275+0000","last_change":"2023-11-09T03:48:57.441364+0000","last_active":"2023-11-09T03:48:57.447275+0000","last_peered":"2023-11-09T03:48:57.447275+0000","last_clean":"2023-11-09T03:48:57.447275+0000","last_became_active":"2023-11-09T03:48:57.441090+0000","last_became_peered":"2023-11-09T03:48:57.441090+0000","last_unstale":"2023-11-09T03:48:57.447275+0000","last_undegraded":"2023-11-09T03:48:57.447275+0000","last_fullsized":"2023-11-09T03:48:57.447275+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:01:29.460175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.431107+0000","last_change":"2023-11-09T03:48:57.052926+0000","last_active":"2023-11-09T03:48:57.431107+0000","last_peered":"2023-11-09T03:48:57.431107+0000","last_clean":"2023-11-09T03:48:57.431107+0000","last_became_active":"2023-11-09T03:48:57.052660+0000","last_became_peered":"2023-11-09T03:48:57.052660+0000","last_unstale":"2023-11-09T03:48:57.431107+0000","last_undegraded":"2023-11-09T03:48:57.431107+0000","last_fullsized":"2023-11-09T03:48:57.431107+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:54:32.585076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036800+0000","last_change":"2023-11-09T03:48:01.456245+0000","last_active":"2023-11-09T03:48:57.036800+0000","last_peered":"2023-11-09T03:48:57.036800+0000","last_clean":"2023-11-09T03:48:57.036800+0000","last_became_active":"2023-11-09T03:48:01.455944+0000","last_became_peered":"2023-11-09T03:48:01.455944+0000","last_unstale":"2023-11-09T03:48:57.036800+0000","last_undegraded":"2023-11-09T03:48:57.036800+0000","last_fullsized":"2023-11-09T03:48:57.036800+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:06:14.849196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.047935+0000","last_change":"2023-11-09T03:48:02.612341+0000","last_active":"2023-11-09T03:48:57.047935+0000","last_peered":"2023-11-09T03:48:57.047935+0000","last_clean":"2023-11-09T03:48:57.047935+0000","last_became_active":"2023-11-09T03:48:00.444426+0000","last_became_peered":"2023-11-09T03:48:00.444426+0000","last_unstale":"2023-11-09T03:48:57.047935+0000","last_undegraded":"2023-11-09T03:48:57.047935+0000","last_fullsized":"2023-11-09T03:48:57.047935+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:19:53.969861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010812420000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":57,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.042396+0000","last_change":"2023-11-09T03:47:03.029063+0000","last_active":"2023-11-09T03:48:57.042396+0000","last_peered":"2023-11-09T03:48:57.042396+0000","last_clean":"2023-11-09T03:48:57.042396+0000","last_became_active":"2023-11-09T03:47:01.593582+0000","last_became_peered":"2023-11-09T03:47:01.593582+0000","last_unstale":"2023-11-09T03:48:57.042396+0000","last_undegraded":"2023-11-09T03:48:57.042396+0000","last_fullsized":"2023-11-09T03:48:57.042396+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:56:20.649937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041855100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.443351+0000","last_change":"2023-11-09T03:48:57.436261+0000","last_active":"2023-11-09T03:48:57.443351+0000","last_peered":"2023-11-09T03:48:57.443351+0000","last_clean":"2023-11-09T03:48:57.443351+0000","last_became_active":"2023-11-09T03:48:57.435990+0000","last_became_peered":"2023-11-09T03:48:57.435990+0000","last_unstale":"2023-11-09T03:48:57.443351+0000","last_undegraded":"2023-11-09T03:48:57.443351+0000","last_fullsized":"2023-11-09T03:48:57.443351+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:58:16.470882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.426528+0000","last_change":"2023-11-09T03:48:57.053135+0000","last_active":"2023-11-09T03:48:57.426528+0000","last_peered":"2023-11-09T03:48:57.426528+0000","last_clean":"2023-11-09T03:48:57.426528+0000","last_became_active":"2023-11-09T03:48:57.052901+0000","last_became_peered":"2023-11-09T03:48:57.052901+0000","last_unstale":"2023-11-09T03:48:57.426528+0000","last_undegraded":"2023-11-09T03:48:57.426528+0000","last_fullsized":"2023-11-09T03:48:57.426528+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:33:16.524514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.037144+0000","last_change":"2023-11-09T03:48:01.450271+0000","last_active":"2023-11-09T03:48:57.037144+0000","last_peered":"2023-11-09T03:48:57.037144+0000","last_clean":"2023-11-09T03:48:57.037144+0000","last_became_active":"2023-11-09T03:48:01.449998+0000","last_became_peered":"2023-11-09T03:48:01.449998+0000","last_unstale":"2023-11-09T03:48:57.037144+0000","last_undegraded":"2023-11-09T03:48:57.037144+0000","last_fullsized":"2023-11-09T03:48:57.037144+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:12:27.170650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.042158+0000","last_change":"2023-11-09T03:48:02.852114+0000","last_active":"2023-11-09T03:48:57.042158+0000","last_peered":"2023-11-09T03:48:57.042158+0000","last_clean":"2023-11-09T03:48:57.042158+0000","last_became_active":"2023-11-09T03:48:00.447203+0000","last_became_peered":"2023-11-09T03:48:00.447203+0000","last_unstale":"2023-11-09T03:48:57.042158+0000","last_undegraded":"2023-11-09T03:48:57.042158+0000","last_fullsized":"2023-11-09T03:48:57.042158+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:11:36.826430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049189500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.048164+0000","last_change":"2023-11-09T03:47:02.619336+0000","last_active":"2023-11-09T03:48:57.048164+0000","last_peered":"2023-11-09T03:48:57.048164+0000","last_clean":"2023-11-09T03:48:57.048164+0000","last_became_active":"2023-11-09T03:47:01.587749+0000","last_became_peered":"2023-11-09T03:47:01.587749+0000","last_unstale":"2023-11-09T03:48:57.048164+0000","last_undegraded":"2023-11-09T03:48:57.048164+0000","last_fullsized":"2023-11-09T03:48:57.048164+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063032099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.451279+0000","last_change":"2023-11-09T03:48:57.441977+0000","last_active":"2023-11-09T03:48:57.451279+0000","last_peered":"2023-11-09T03:48:57.451279+0000","last_clean":"2023-11-09T03:48:57.451279+0000","last_became_active":"2023-11-09T03:48:57.441727+0000","last_became_peered":"2023-11-09T03:48:57.441727+0000","last_unstale":"2023-11-09T03:48:57.451279+0000","last_undegraded":"2023-11-09T03:48:57.451279+0000","last_fullsized":"2023-11-09T03:48:57.451279+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:38:58.229222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.431566+0000","last_change":"2023-11-09T03:48:57.053278+0000","last_active":"2023-11-09T03:48:57.431566+0000","last_peered":"2023-11-09T03:48:57.431566+0000","last_clean":"2023-11-09T03:48:57.431566+0000","last_became_active":"2023-11-09T03:48:57.053028+0000","last_became_peered":"2023-11-09T03:48:57.053028+0000","last_unstale":"2023-11-09T03:48:57.431566+0000","last_undegraded":"2023-11-09T03:48:57.431566+0000","last_fullsized":"2023-11-09T03:48:57.431566+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:52:53.826209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.033557+0000","last_change":"2023-11-09T03:48:01.454147+0000","last_active":"2023-11-09T03:48:57.033557+0000","last_peered":"2023-11-09T03:48:57.033557+0000","last_clean":"2023-11-09T03:48:57.033557+0000","last_became_active":"2023-11-09T03:48:01.453868+0000","last_became_peered":"2023-11-09T03:48:01.453868+0000","last_unstale":"2023-11-09T03:48:57.033557+0000","last_undegraded":"2023-11-09T03:48:57.033557+0000","last_fullsized":"2023-11-09T03:48:57.033557+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:05:09.655318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.045943+0000","last_change":"2023-11-09T03:48:02.612000+0000","last_active":"2023-11-09T03:48:57.045943+0000","last_peered":"2023-11-09T03:48:57.045943+0000","last_clean":"2023-11-09T03:48:57.045943+0000","last_became_active":"2023-11-09T03:48:00.441667+0000","last_became_peered":"2023-11-09T03:48:00.441667+0000","last_unstale":"2023-11-09T03:48:57.045943+0000","last_undegraded":"2023-11-09T03:48:57.045943+0000","last_fullsized":"2023-11-09T03:48:57.045943+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:12:26.164644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000966781,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036868+0000","last_change":"2023-11-09T03:47:02.581658+0000","last_active":"2023-11-09T03:48:57.036868+0000","last_peered":"2023-11-09T03:48:57.036868+0000","last_clean":"2023-11-09T03:48:57.036868+0000","last_became_active":"2023-11-09T03:47:01.588612+0000","last_became_peered":"2023-11-09T03:47:01.588612+0000","last_unstale":"2023-11-09T03:48:57.036868+0000","last_undegraded":"2023-11-09T03:48:57.036868+0000","last_fullsized":"2023-11-09T03:48:57.036868+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045238500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.440573+0000","last_change":"2023-11-09T03:48:57.439162+0000","last_active":"2023-11-09T03:48:57.440573+0000","last_peered":"2023-11-09T03:48:57.440573+0000","last_clean":"2023-11-09T03:48:57.440573+0000","last_became_active":"2023-11-09T03:48:57.438911+0000","last_became_peered":"2023-11-09T03:48:57.438911+0000","last_unstale":"2023-11-09T03:48:57.440573+0000","last_undegraded":"2023-11-09T03:48:57.440573+0000","last_fullsized":"2023-11-09T03:48:57.440573+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:33:46.080846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.434733+0000","last_change":"2023-11-09T03:48:57.049187+0000","last_active":"2023-11-09T03:48:57.434733+0000","last_peered":"2023-11-09T03:48:57.434733+0000","last_clean":"2023-11-09T03:48:57.434733+0000","last_became_active":"2023-11-09T03:48:57.048829+0000","last_became_peered":"2023-11-09T03:48:57.048829+0000","last_unstale":"2023-11-09T03:48:57.434733+0000","last_undegraded":"2023-11-09T03:48:57.434733+0000","last_fullsized":"2023-11-09T03:48:57.434733+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:00.081394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.043113+0000","last_change":"2023-11-09T03:48:01.449834+0000","last_active":"2023-11-09T03:48:57.043113+0000","last_peered":"2023-11-09T03:48:57.043113+0000","last_clean":"2023-11-09T03:48:57.043113+0000","last_became_active":"2023-11-09T03:48:01.449552+0000","last_became_peered":"2023-11-09T03:48:01.449552+0000","last_unstale":"2023-11-09T03:48:57.043113+0000","last_undegraded":"2023-11-09T03:48:57.043113+0000","last_fullsized":"2023-11-09T03:48:57.043113+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:11:37.832436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036293+0000","last_change":"2023-11-09T03:48:02.682197+0000","last_active":"2023-11-09T03:48:57.036293+0000","last_peered":"2023-11-09T03:48:57.036293+0000","last_clean":"2023-11-09T03:48:57.036293+0000","last_became_active":"2023-11-09T03:48:00.445616+0000","last_became_peered":"2023-11-09T03:48:00.445616+0000","last_unstale":"2023-11-09T03:48:57.036293+0000","last_undegraded":"2023-11-09T03:48:57.036293+0000","last_fullsized":"2023-11-09T03:48:57.036293+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:57:03.474084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000416494,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.042714+0000","last_change":"2023-11-09T03:47:02.618828+0000","last_active":"2023-11-09T03:48:57.042714+0000","last_peered":"2023-11-09T03:48:57.042714+0000","last_clean":"2023-11-09T03:48:57.042714+0000","last_became_active":"2023-11-09T03:47:01.590679+0000","last_became_peered":"2023-11-09T03:47:01.590679+0000","last_unstale":"2023-11-09T03:48:57.042714+0000","last_undegraded":"2023-11-09T03:48:57.042714+0000","last_fullsized":"2023-11-09T03:48:57.042714+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:06:30.982987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070668,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.442589+0000","last_change":"2023-11-09T03:48:57.439536+0000","last_active":"2023-11-09T03:48:57.442589+0000","last_peered":"2023-11-09T03:48:57.442589+0000","last_clean":"2023-11-09T03:48:57.442589+0000","last_became_active":"2023-11-09T03:48:57.439282+0000","last_became_peered":"2023-11-09T03:48:57.439282+0000","last_unstale":"2023-11-09T03:48:57.442589+0000","last_undegraded":"2023-11-09T03:48:57.442589+0000","last_fullsized":"2023-11-09T03:48:57.442589+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:36:03.301496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.427325+0000","last_change":"2023-11-09T03:48:57.047878+0000","last_active":"2023-11-09T03:48:57.427325+0000","last_peered":"2023-11-09T03:48:57.427325+0000","last_clean":"2023-11-09T03:48:57.427325+0000","last_became_active":"2023-11-09T03:48:57.047632+0000","last_became_peered":"2023-11-09T03:48:57.047632+0000","last_unstale":"2023-11-09T03:48:57.427325+0000","last_undegraded":"2023-11-09T03:48:57.427325+0000","last_fullsized":"2023-11-09T03:48:57.427325+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:26:21.550088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.033218+0000","last_change":"2023-11-09T03:48:01.447647+0000","last_active":"2023-11-09T03:48:57.033218+0000","last_peered":"2023-11-09T03:48:57.033218+0000","last_clean":"2023-11-09T03:48:57.033218+0000","last_became_active":"2023-11-09T03:48:01.447302+0000","last_became_peered":"2023-11-09T03:48:01.447302+0000","last_unstale":"2023-11-09T03:48:57.033218+0000","last_undegraded":"2023-11-09T03:48:57.033218+0000","last_fullsized":"2023-11-09T03:48:57.033218+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:08:47.931939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036646+0000","last_change":"2023-11-09T03:48:03.327731+0000","last_active":"2023-11-09T03:48:57.036646+0000","last_peered":"2023-11-09T03:48:57.036646+0000","last_clean":"2023-11-09T03:48:57.036646+0000","last_became_active":"2023-11-09T03:48:00.449660+0000","last_became_peered":"2023-11-09T03:48:00.449660+0000","last_unstale":"2023-11-09T03:48:57.036646+0000","last_undegraded":"2023-11-09T03:48:57.036646+0000","last_fullsized":"2023-11-09T03:48:57.036646+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:02:07.940908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037816599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.033938+0000","last_change":"2023-11-09T03:47:03.146528+0000","last_active":"2023-11-09T03:48:57.033938+0000","last_peered":"2023-11-09T03:48:57.033938+0000","last_clean":"2023-11-09T03:48:57.033938+0000","last_became_active":"2023-11-09T03:47:01.587060+0000","last_became_peered":"2023-11-09T03:47:01.587060+0000","last_unstale":"2023-11-09T03:48:57.033938+0000","last_undegraded":"2023-11-09T03:48:57.033938+0000","last_fullsized":"2023-11-09T03:48:57.033938+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:39:15.238496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079064999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.451252+0000","last_change":"2023-11-09T03:48:57.443732+0000","last_active":"2023-11-09T03:48:57.451252+0000","last_peered":"2023-11-09T03:48:57.451252+0000","last_clean":"2023-11-09T03:48:57.451252+0000","last_became_active":"2023-11-09T03:48:57.443458+0000","last_became_peered":"2023-11-09T03:48:57.443458+0000","last_unstale":"2023-11-09T03:48:57.451252+0000","last_undegraded":"2023-11-09T03:48:57.451252+0000","last_fullsized":"2023-11-09T03:48:57.451252+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:50:34.258596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.435028+0000","last_change":"2023-11-09T03:48:57.049776+0000","last_active":"2023-11-09T03:48:57.435028+0000","last_peered":"2023-11-09T03:48:57.435028+0000","last_clean":"2023-11-09T03:48:57.435028+0000","last_became_active":"2023-11-09T03:48:57.049533+0000","last_became_peered":"2023-11-09T03:48:57.049533+0000","last_unstale":"2023-11-09T03:48:57.435028+0000","last_undegraded":"2023-11-09T03:48:57.435028+0000","last_fullsized":"2023-11-09T03:48:57.435028+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:33:03.051538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036452+0000","last_change":"2023-11-09T03:48:01.453939+0000","last_active":"2023-11-09T03:48:57.036452+0000","last_peered":"2023-11-09T03:48:57.036452+0000","last_clean":"2023-11-09T03:48:57.036452+0000","last_became_active":"2023-11-09T03:48:01.453600+0000","last_became_peered":"2023-11-09T03:48:01.453600+0000","last_unstale":"2023-11-09T03:48:57.036452+0000","last_undegraded":"2023-11-09T03:48:57.036452+0000","last_fullsized":"2023-11-09T03:48:57.036452+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:45:10.522254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.033988+0000","last_change":"2023-11-09T03:48:02.440910+0000","last_active":"2023-11-09T03:48:57.033988+0000","last_peered":"2023-11-09T03:48:57.033988+0000","last_clean":"2023-11-09T03:48:57.033988+0000","last_became_active":"2023-11-09T03:48:00.443791+0000","last_became_peered":"2023-11-09T03:48:00.443791+0000","last_unstale":"2023-11-09T03:48:57.033988+0000","last_undegraded":"2023-11-09T03:48:57.033988+0000","last_fullsized":"2023-11-09T03:48:57.033988+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:11:36.826430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069223400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.033390+0000","last_change":"2023-11-09T03:47:03.148246+0000","last_active":"2023-11-09T03:48:57.033390+0000","last_peered":"2023-11-09T03:48:57.033390+0000","last_clean":"2023-11-09T03:48:57.033390+0000","last_became_active":"2023-11-09T03:47:01.590988+0000","last_became_peered":"2023-11-09T03:47:01.590988+0000","last_unstale":"2023-11-09T03:48:57.033390+0000","last_undegraded":"2023-11-09T03:48:57.033390+0000","last_fullsized":"2023-11-09T03:48:57.033390+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:07:20.785445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055651099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.451283+0000","last_change":"2023-11-09T03:48:57.443662+0000","last_active":"2023-11-09T03:48:57.451283+0000","last_peered":"2023-11-09T03:48:57.451283+0000","last_clean":"2023-11-09T03:48:57.451283+0000","last_became_active":"2023-11-09T03:48:57.443352+0000","last_became_peered":"2023-11-09T03:48:57.443352+0000","last_unstale":"2023-11-09T03:48:57.451283+0000","last_undegraded":"2023-11-09T03:48:57.451283+0000","last_fullsized":"2023-11-09T03:48:57.451283+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:00:55.763662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.437037+0000","last_change":"2023-11-09T03:48:57.049287+0000","last_active":"2023-11-09T03:48:57.437037+0000","last_peered":"2023-11-09T03:48:57.437037+0000","last_clean":"2023-11-09T03:48:57.437037+0000","last_became_active":"2023-11-09T03:48:57.048990+0000","last_became_peered":"2023-11-09T03:48:57.048990+0000","last_unstale":"2023-11-09T03:48:57.437037+0000","last_undegraded":"2023-11-09T03:48:57.437037+0000","last_fullsized":"2023-11-09T03:48:57.437037+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:49:51.200196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.037483+0000","last_change":"2023-11-09T03:48:01.450557+0000","last_active":"2023-11-09T03:48:57.037483+0000","last_peered":"2023-11-09T03:48:57.037483+0000","last_clean":"2023-11-09T03:48:57.037483+0000","last_became_active":"2023-11-09T03:48:01.450291+0000","last_became_peered":"2023-11-09T03:48:01.450291+0000","last_unstale":"2023-11-09T03:48:57.037483+0000","last_undegraded":"2023-11-09T03:48:57.037483+0000","last_fullsized":"2023-11-09T03:48:57.037483+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:21:29.699594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.040557+0000","last_change":"2023-11-09T03:48:02.852010+0000","last_active":"2023-11-09T03:48:57.040557+0000","last_peered":"2023-11-09T03:48:57.040557+0000","last_clean":"2023-11-09T03:48:57.040557+0000","last_became_active":"2023-11-09T03:48:00.445267+0000","last_became_peered":"2023-11-09T03:48:00.445267+0000","last_unstale":"2023-11-09T03:48:57.040557+0000","last_undegraded":"2023-11-09T03:48:57.040557+0000","last_fullsized":"2023-11-09T03:48:57.040557+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:17.989628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064559199999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":63,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036817+0000","last_change":"2023-11-09T03:47:02.582754+0000","last_active":"2023-11-09T03:48:57.036817+0000","last_peered":"2023-11-09T03:48:57.036817+0000","last_clean":"2023-11-09T03:48:57.036817+0000","last_became_active":"2023-11-09T03:47:01.589715+0000","last_became_peered":"2023-11-09T03:47:01.589715+0000","last_unstale":"2023-11-09T03:48:57.036817+0000","last_undegraded":"2023-11-09T03:48:57.036817+0000","last_fullsized":"2023-11-09T03:48:57.036817+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099342799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.441775+0000","last_change":"2023-11-09T03:48:57.440148+0000","last_active":"2023-11-09T03:48:57.441775+0000","last_peered":"2023-11-09T03:48:57.441775+0000","last_clean":"2023-11-09T03:48:57.441775+0000","last_became_active":"2023-11-09T03:48:57.439835+0000","last_became_peered":"2023-11-09T03:48:57.439835+0000","last_unstale":"2023-11-09T03:48:57.441775+0000","last_undegraded":"2023-11-09T03:48:57.441775+0000","last_fullsized":"2023-11-09T03:48:57.441775+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:44:32.408986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.428519+0000","last_change":"2023-11-09T03:48:57.038706+0000","last_active":"2023-11-09T03:48:57.428519+0000","last_peered":"2023-11-09T03:48:57.428519+0000","last_clean":"2023-11-09T03:48:57.428519+0000","last_became_active":"2023-11-09T03:48:57.038433+0000","last_became_peered":"2023-11-09T03:48:57.038433+0000","last_unstale":"2023-11-09T03:48:57.428519+0000","last_undegraded":"2023-11-09T03:48:57.428519+0000","last_fullsized":"2023-11-09T03:48:57.428519+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:36:42.256387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.048135+0000","last_change":"2023-11-09T03:48:01.452456+0000","last_active":"2023-11-09T03:48:57.048135+0000","last_peered":"2023-11-09T03:48:57.048135+0000","last_clean":"2023-11-09T03:48:57.048135+0000","last_became_active":"2023-11-09T03:48:01.452059+0000","last_became_peered":"2023-11-09T03:48:01.452059+0000","last_unstale":"2023-11-09T03:48:57.048135+0000","last_undegraded":"2023-11-09T03:48:57.048135+0000","last_fullsized":"2023-11-09T03:48:57.048135+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:57:56.855869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.037342+0000","last_change":"2023-11-09T03:48:02.442557+0000","last_active":"2023-11-09T03:48:57.037342+0000","last_peered":"2023-11-09T03:48:57.037342+0000","last_clean":"2023-11-09T03:48:57.037342+0000","last_became_active":"2023-11-09T03:48:00.444823+0000","last_became_peered":"2023-11-09T03:48:00.444823+0000","last_unstale":"2023-11-09T03:48:57.037342+0000","last_undegraded":"2023-11-09T03:48:57.037342+0000","last_fullsized":"2023-11-09T03:48:57.037342+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:19:53.969861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075076299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.043872+0000","last_change":"2023-11-09T03:47:02.724896+0000","last_active":"2023-11-09T03:48:57.043872+0000","last_peered":"2023-11-09T03:48:57.043872+0000","last_clean":"2023-11-09T03:48:57.043872+0000","last_became_active":"2023-11-09T03:47:01.592784+0000","last_became_peered":"2023-11-09T03:47:01.592784+0000","last_unstale":"2023-11-09T03:48:57.043872+0000","last_undegraded":"2023-11-09T03:48:57.043872+0000","last_fullsized":"2023-11-09T03:48:57.043872+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:51:32.016204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056683500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.439394+0000","last_change":"2023-11-09T03:48:57.437705+0000","last_active":"2023-11-09T03:48:57.439394+0000","last_peered":"2023-11-09T03:48:57.439394+0000","last_clean":"2023-11-09T03:48:57.439394+0000","last_became_active":"2023-11-09T03:48:57.437436+0000","last_became_peered":"2023-11-09T03:48:57.437436+0000","last_unstale":"2023-11-09T03:48:57.439394+0000","last_undegraded":"2023-11-09T03:48:57.439394+0000","last_fullsized":"2023-11-09T03:48:57.439394+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:37:27.085476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.435502+0000","last_change":"2023-11-09T03:48:57.050307+0000","last_active":"2023-11-09T03:48:57.435502+0000","last_peered":"2023-11-09T03:48:57.435502+0000","last_clean":"2023-11-09T03:48:57.435502+0000","last_became_active":"2023-11-09T03:48:57.050048+0000","last_became_peered":"2023-11-09T03:48:57.050048+0000","last_unstale":"2023-11-09T03:48:57.435502+0000","last_undegraded":"2023-11-09T03:48:57.435502+0000","last_fullsized":"2023-11-09T03:48:57.435502+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:51:17.478024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.037238+0000","last_change":"2023-11-09T03:48:01.452167+0000","last_active":"2023-11-09T03:48:57.037238+0000","last_peered":"2023-11-09T03:48:57.037238+0000","last_clean":"2023-11-09T03:48:57.037238+0000","last_became_active":"2023-11-09T03:48:01.451780+0000","last_became_peered":"2023-11-09T03:48:01.451780+0000","last_unstale":"2023-11-09T03:48:57.037238+0000","last_undegraded":"2023-11-09T03:48:57.037238+0000","last_fullsized":"2023-11-09T03:48:57.037238+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:15:36.287302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.047850+0000","last_change":"2023-11-09T03:48:02.612246+0000","last_active":"2023-11-09T03:48:57.047850+0000","last_peered":"2023-11-09T03:48:57.047850+0000","last_clean":"2023-11-09T03:48:57.047850+0000","last_became_active":"2023-11-09T03:48:00.444629+0000","last_became_peered":"2023-11-09T03:48:00.444629+0000","last_unstale":"2023-11-09T03:48:57.047850+0000","last_undegraded":"2023-11-09T03:48:57.047850+0000","last_fullsized":"2023-11-09T03:48:57.047850+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:51:17.754054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098167299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.037571+0000","last_change":"2023-11-09T03:47:03.146527+0000","last_active":"2023-11-09T03:48:57.037571+0000","last_peered":"2023-11-09T03:48:57.037571+0000","last_clean":"2023-11-09T03:48:57.037571+0000","last_became_active":"2023-11-09T03:47:01.587858+0000","last_became_peered":"2023-11-09T03:47:01.587858+0000","last_unstale":"2023-11-09T03:48:57.037571+0000","last_undegraded":"2023-11-09T03:48:57.037571+0000","last_fullsized":"2023-11-09T03:48:57.037571+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:56:04.201126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084439599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.440300+0000","last_change":"2023-11-09T03:48:57.437592+0000","last_active":"2023-11-09T03:48:57.440300+0000","last_peered":"2023-11-09T03:48:57.440300+0000","last_clean":"2023-11-09T03:48:57.440300+0000","last_became_active":"2023-11-09T03:48:57.437311+0000","last_became_peered":"2023-11-09T03:48:57.437311+0000","last_unstale":"2023-11-09T03:48:57.440300+0000","last_undegraded":"2023-11-09T03:48:57.440300+0000","last_fullsized":"2023-11-09T03:48:57.440300+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:50:47.233700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.425939+0000","last_change":"2023-11-09T03:48:57.052503+0000","last_active":"2023-11-09T03:48:57.425939+0000","last_peered":"2023-11-09T03:48:57.425939+0000","last_clean":"2023-11-09T03:48:57.425939+0000","last_became_active":"2023-11-09T03:48:57.052219+0000","last_became_peered":"2023-11-09T03:48:57.052219+0000","last_unstale":"2023-11-09T03:48:57.425939+0000","last_undegraded":"2023-11-09T03:48:57.425939+0000","last_fullsized":"2023-11-09T03:48:57.425939+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:46:31.166452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.043687+0000","last_change":"2023-11-09T03:48:01.450685+0000","last_active":"2023-11-09T03:48:57.043687+0000","last_peered":"2023-11-09T03:48:57.043687+0000","last_clean":"2023-11-09T03:48:57.043687+0000","last_became_active":"2023-11-09T03:48:01.450445+0000","last_became_peered":"2023-11-09T03:48:01.450445+0000","last_unstale":"2023-11-09T03:48:57.043687+0000","last_undegraded":"2023-11-09T03:48:57.043687+0000","last_fullsized":"2023-11-09T03:48:57.043687+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:16:00.220961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.033727+0000","last_change":"2023-11-09T03:48:02.441928+0000","last_active":"2023-11-09T03:48:57.033727+0000","last_peered":"2023-11-09T03:48:57.033727+0000","last_clean":"2023-11-09T03:48:57.033727+0000","last_became_active":"2023-11-09T03:48:00.444417+0000","last_became_peered":"2023-11-09T03:48:00.444417+0000","last_unstale":"2023-11-09T03:48:57.033727+0000","last_undegraded":"2023-11-09T03:48:57.033727+0000","last_fullsized":"2023-11-09T03:48:57.033727+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:55.450096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072428900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.044814+0000","last_change":"2023-11-09T03:47:02.725223+0000","last_active":"2023-11-09T03:48:57.044814+0000","last_peered":"2023-11-09T03:48:57.044814+0000","last_clean":"2023-11-09T03:48:57.044814+0000","last_became_active":"2023-11-09T03:47:01.590635+0000","last_became_peered":"2023-11-09T03:47:01.590635+0000","last_unstale":"2023-11-09T03:48:57.044814+0000","last_undegraded":"2023-11-09T03:48:57.044814+0000","last_fullsized":"2023-11-09T03:48:57.044814+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:05.754357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051685100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.451304+0000","last_change":"2023-11-09T03:48:57.445874+0000","last_active":"2023-11-09T03:48:57.451304+0000","last_peered":"2023-11-09T03:48:57.451304+0000","last_clean":"2023-11-09T03:48:57.451304+0000","last_became_active":"2023-11-09T03:48:57.445519+0000","last_became_peered":"2023-11-09T03:48:57.445519+0000","last_unstale":"2023-11-09T03:48:57.451304+0000","last_undegraded":"2023-11-09T03:48:57.451304+0000","last_fullsized":"2023-11-09T03:48:57.451304+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:33:12.883340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.434564+0000","last_change":"2023-11-09T03:48:57.051180+0000","last_active":"2023-11-09T03:48:57.434564+0000","last_peered":"2023-11-09T03:48:57.434564+0000","last_clean":"2023-11-09T03:48:57.434564+0000","last_became_active":"2023-11-09T03:48:57.050907+0000","last_became_peered":"2023-11-09T03:48:57.050907+0000","last_unstale":"2023-11-09T03:48:57.434564+0000","last_undegraded":"2023-11-09T03:48:57.434564+0000","last_fullsized":"2023-11-09T03:48:57.434564+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:44:34.138203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.042673+0000","last_change":"2023-11-09T03:48:01.453143+0000","last_active":"2023-11-09T03:48:57.042673+0000","last_peered":"2023-11-09T03:48:57.042673+0000","last_clean":"2023-11-09T03:48:57.042673+0000","last_became_active":"2023-11-09T03:48:01.452873+0000","last_became_peered":"2023-11-09T03:48:01.452873+0000","last_unstale":"2023-11-09T03:48:57.042673+0000","last_undegraded":"2023-11-09T03:48:57.042673+0000","last_fullsized":"2023-11-09T03:48:57.042673+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:06:28.575429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.034059+0000","last_change":"2023-11-09T03:48:02.442416+0000","last_active":"2023-11-09T03:48:57.034059+0000","last_peered":"2023-11-09T03:48:57.034059+0000","last_clean":"2023-11-09T03:48:57.034059+0000","last_became_active":"2023-11-09T03:48:00.443765+0000","last_became_peered":"2023-11-09T03:48:00.443765+0000","last_unstale":"2023-11-09T03:48:57.034059+0000","last_undegraded":"2023-11-09T03:48:57.034059+0000","last_fullsized":"2023-11-09T03:48:57.034059+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:41:22.321811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012468850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":63,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.037371+0000","last_change":"2023-11-09T03:47:02.582739+0000","last_active":"2023-11-09T03:48:57.037371+0000","last_peered":"2023-11-09T03:48:57.037371+0000","last_clean":"2023-11-09T03:48:57.037371+0000","last_became_active":"2023-11-09T03:47:01.592743+0000","last_became_peered":"2023-11-09T03:47:01.592743+0000","last_unstale":"2023-11-09T03:48:57.037371+0000","last_undegraded":"2023-11-09T03:48:57.037371+0000","last_fullsized":"2023-11-09T03:48:57.037371+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051802700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.438429+0000","last_change":"2023-11-09T03:48:57.436905+0000","last_active":"2023-11-09T03:48:57.438429+0000","last_peered":"2023-11-09T03:48:57.438429+0000","last_clean":"2023-11-09T03:48:57.438429+0000","last_became_active":"2023-11-09T03:48:57.436190+0000","last_became_peered":"2023-11-09T03:48:57.436190+0000","last_unstale":"2023-11-09T03:48:57.438429+0000","last_undegraded":"2023-11-09T03:48:57.438429+0000","last_fullsized":"2023-11-09T03:48:57.438429+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:18:06.196599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.436575+0000","last_change":"2023-11-09T03:48:57.052108+0000","last_active":"2023-11-09T03:48:57.436575+0000","last_peered":"2023-11-09T03:48:57.436575+0000","last_clean":"2023-11-09T03:48:57.436575+0000","last_became_active":"2023-11-09T03:48:57.051813+0000","last_became_peered":"2023-11-09T03:48:57.051813+0000","last_unstale":"2023-11-09T03:48:57.436575+0000","last_undegraded":"2023-11-09T03:48:57.436575+0000","last_fullsized":"2023-11-09T03:48:57.436575+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:06:46.097576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.037044+0000","last_change":"2023-11-09T03:48:01.442195+0000","last_active":"2023-11-09T03:48:57.037044+0000","last_peered":"2023-11-09T03:48:57.037044+0000","last_clean":"2023-11-09T03:48:57.037044+0000","last_became_active":"2023-11-09T03:48:01.441887+0000","last_became_peered":"2023-11-09T03:48:01.441887+0000","last_unstale":"2023-11-09T03:48:57.037044+0000","last_undegraded":"2023-11-09T03:48:57.037044+0000","last_fullsized":"2023-11-09T03:48:57.037044+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:06:28.575429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036749+0000","last_change":"2023-11-09T03:48:02.442061+0000","last_active":"2023-11-09T03:48:57.036749+0000","last_peered":"2023-11-09T03:48:57.036749+0000","last_clean":"2023-11-09T03:48:57.036749+0000","last_became_active":"2023-11-09T03:48:00.445921+0000","last_became_peered":"2023-11-09T03:48:00.445921+0000","last_unstale":"2023-11-09T03:48:57.036749+0000","last_undegraded":"2023-11-09T03:48:57.036749+0000","last_fullsized":"2023-11-09T03:48:57.036749+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:50:55.299235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012719770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.042931+0000","last_change":"2023-11-09T03:47:02.724953+0000","last_active":"2023-11-09T03:48:57.042931+0000","last_peered":"2023-11-09T03:48:57.042931+0000","last_clean":"2023-11-09T03:48:57.042931+0000","last_became_active":"2023-11-09T03:47:01.587133+0000","last_became_peered":"2023-11-09T03:47:01.587133+0000","last_unstale":"2023-11-09T03:48:57.042931+0000","last_undegraded":"2023-11-09T03:48:57.042931+0000","last_fullsized":"2023-11-09T03:48:57.042931+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:23:48.521161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066497599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.446994+0000","last_change":"2023-11-09T03:48:57.442902+0000","last_active":"2023-11-09T03:48:57.446994+0000","last_peered":"2023-11-09T03:48:57.446994+0000","last_clean":"2023-11-09T03:48:57.446994+0000","last_became_active":"2023-11-09T03:48:57.442644+0000","last_became_peered":"2023-11-09T03:48:57.442644+0000","last_unstale":"2023-11-09T03:48:57.446994+0000","last_undegraded":"2023-11-09T03:48:57.446994+0000","last_fullsized":"2023-11-09T03:48:57.446994+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:59:11.986476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.430585+0000","last_change":"2023-11-09T03:48:57.045458+0000","last_active":"2023-11-09T03:48:57.430585+0000","last_peered":"2023-11-09T03:48:57.430585+0000","last_clean":"2023-11-09T03:48:57.430585+0000","last_became_active":"2023-11-09T03:48:57.045192+0000","last_became_peered":"2023-11-09T03:48:57.045192+0000","last_unstale":"2023-11-09T03:48:57.430585+0000","last_undegraded":"2023-11-09T03:48:57.430585+0000","last_fullsized":"2023-11-09T03:48:57.430585+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:09:44.721115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036589+0000","last_change":"2023-11-09T03:48:01.450507+0000","last_active":"2023-11-09T03:48:57.036589+0000","last_peered":"2023-11-09T03:48:57.036589+0000","last_clean":"2023-11-09T03:48:57.036589+0000","last_became_active":"2023-11-09T03:48:01.450191+0000","last_became_peered":"2023-11-09T03:48:01.450191+0000","last_unstale":"2023-11-09T03:48:57.036589+0000","last_undegraded":"2023-11-09T03:48:57.036589+0000","last_fullsized":"2023-11-09T03:48:57.036589+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:32:40.342502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.037053+0000","last_change":"2023-11-09T03:48:02.442120+0000","last_active":"2023-11-09T03:48:57.037053+0000","last_peered":"2023-11-09T03:48:57.037053+0000","last_clean":"2023-11-09T03:48:57.037053+0000","last_became_active":"2023-11-09T03:48:00.445504+0000","last_became_peered":"2023-11-09T03:48:00.445504+0000","last_unstale":"2023-11-09T03:48:57.037053+0000","last_undegraded":"2023-11-09T03:48:57.037053+0000","last_fullsized":"2023-11-09T03:48:57.037053+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:35:53.864627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001095766,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036480+0000","last_change":"2023-11-09T03:47:03.146217+0000","last_active":"2023-11-09T03:48:57.036480+0000","last_peered":"2023-11-09T03:48:57.036480+0000","last_clean":"2023-11-09T03:48:57.036480+0000","last_became_active":"2023-11-09T03:47:01.587598+0000","last_became_peered":"2023-11-09T03:47:01.587598+0000","last_unstale":"2023-11-09T03:48:57.036480+0000","last_undegraded":"2023-11-09T03:48:57.036480+0000","last_fullsized":"2023-11-09T03:48:57.036480+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:15:22.723455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090126400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.446917+0000","last_change":"2023-11-09T03:48:57.442288+0000","last_active":"2023-11-09T03:48:57.446917+0000","last_peered":"2023-11-09T03:48:57.446917+0000","last_clean":"2023-11-09T03:48:57.446917+0000","last_became_active":"2023-11-09T03:48:57.442008+0000","last_became_peered":"2023-11-09T03:48:57.442008+0000","last_unstale":"2023-11-09T03:48:57.446917+0000","last_undegraded":"2023-11-09T03:48:57.446917+0000","last_fullsized":"2023-11-09T03:48:57.446917+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:28:01.062126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.430864+0000","last_change":"2023-11-09T03:48:57.045639+0000","last_active":"2023-11-09T03:48:57.430864+0000","last_peered":"2023-11-09T03:48:57.430864+0000","last_clean":"2023-11-09T03:48:57.430864+0000","last_became_active":"2023-11-09T03:48:57.044947+0000","last_became_peered":"2023-11-09T03:48:57.044947+0000","last_unstale":"2023-11-09T03:48:57.430864+0000","last_undegraded":"2023-11-09T03:48:57.430864+0000","last_fullsized":"2023-11-09T03:48:57.430864+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:56:05.508536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.037061+0000","last_change":"2023-11-09T03:48:01.454878+0000","last_active":"2023-11-09T03:48:57.037061+0000","last_peered":"2023-11-09T03:48:57.037061+0000","last_clean":"2023-11-09T03:48:57.037061+0000","last_became_active":"2023-11-09T03:48:01.454591+0000","last_became_peered":"2023-11-09T03:48:01.454591+0000","last_unstale":"2023-11-09T03:48:57.037061+0000","last_undegraded":"2023-11-09T03:48:57.037061+0000","last_fullsized":"2023-11-09T03:48:57.037061+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:38.536898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.037060+0000","last_change":"2023-11-09T03:48:03.327800+0000","last_active":"2023-11-09T03:48:57.037060+0000","last_peered":"2023-11-09T03:48:57.037060+0000","last_clean":"2023-11-09T03:48:57.037060+0000","last_became_active":"2023-11-09T03:48:00.446576+0000","last_became_peered":"2023-11-09T03:48:00.446576+0000","last_unstale":"2023-11-09T03:48:57.037060+0000","last_undegraded":"2023-11-09T03:48:57.037060+0000","last_fullsized":"2023-11-09T03:48:57.037060+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:37:08.851102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054679700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.033451+0000","last_change":"2023-11-09T03:47:03.148283+0000","last_active":"2023-11-09T03:48:57.033451+0000","last_peered":"2023-11-09T03:48:57.033451+0000","last_clean":"2023-11-09T03:48:57.033451+0000","last_became_active":"2023-11-09T03:47:01.588841+0000","last_became_peered":"2023-11-09T03:47:01.588841+0000","last_unstale":"2023-11-09T03:48:57.033451+0000","last_undegraded":"2023-11-09T03:48:57.033451+0000","last_fullsized":"2023-11-09T03:48:57.033451+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:47:44.880741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000436081,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.437458+0000","last_change":"2023-11-09T03:48:57.434924+0000","last_active":"2023-11-09T03:48:57.437458+0000","last_peered":"2023-11-09T03:48:57.437458+0000","last_clean":"2023-11-09T03:48:57.437458+0000","last_became_active":"2023-11-09T03:48:57.434658+0000","last_became_peered":"2023-11-09T03:48:57.434658+0000","last_unstale":"2023-11-09T03:48:57.437458+0000","last_undegraded":"2023-11-09T03:48:57.437458+0000","last_fullsized":"2023-11-09T03:48:57.437458+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:18:33.315635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.425940+0000","last_change":"2023-11-09T03:48:57.054349+0000","last_active":"2023-11-09T03:48:57.425940+0000","last_peered":"2023-11-09T03:48:57.425940+0000","last_clean":"2023-11-09T03:48:57.425940+0000","last_became_active":"2023-11-09T03:48:57.054099+0000","last_became_peered":"2023-11-09T03:48:57.054099+0000","last_unstale":"2023-11-09T03:48:57.425940+0000","last_undegraded":"2023-11-09T03:48:57.425940+0000","last_fullsized":"2023-11-09T03:48:57.425940+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:59:29.131332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.043458+0000","last_change":"2023-11-09T03:48:01.450581+0000","last_active":"2023-11-09T03:48:57.043458+0000","last_peered":"2023-11-09T03:48:57.043458+0000","last_clean":"2023-11-09T03:48:57.043458+0000","last_became_active":"2023-11-09T03:48:01.450280+0000","last_became_peered":"2023-11-09T03:48:01.450280+0000","last_unstale":"2023-11-09T03:48:57.043458+0000","last_undegraded":"2023-11-09T03:48:57.043458+0000","last_fullsized":"2023-11-09T03:48:57.043458+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:18.995634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.033062+0000","last_change":"2023-11-09T03:48:02.440934+0000","last_active":"2023-11-09T03:48:57.033062+0000","last_peered":"2023-11-09T03:48:57.033062+0000","last_clean":"2023-11-09T03:48:57.033062+0000","last_became_active":"2023-11-09T03:48:00.444394+0000","last_became_peered":"2023-11-09T03:48:00.444394+0000","last_unstale":"2023-11-09T03:48:57.033062+0000","last_undegraded":"2023-11-09T03:48:57.033062+0000","last_fullsized":"2023-11-09T03:48:57.033062+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:15:59.214955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063908300000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.042931+0000","last_change":"2023-11-09T03:47:02.618856+0000","last_active":"2023-11-09T03:48:57.042931+0000","last_peered":"2023-11-09T03:48:57.042931+0000","last_clean":"2023-11-09T03:48:57.042931+0000","last_became_active":"2023-11-09T03:47:01.590430+0000","last_became_peered":"2023-11-09T03:47:01.590430+0000","last_unstale":"2023-11-09T03:48:57.042931+0000","last_undegraded":"2023-11-09T03:48:57.042931+0000","last_fullsized":"2023-11-09T03:48:57.042931+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:57:18.470305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055030200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.441790+0000","last_change":"2023-11-09T03:48:57.440624+0000","last_active":"2023-11-09T03:48:57.441790+0000","last_peered":"2023-11-09T03:48:57.441790+0000","last_clean":"2023-11-09T03:48:57.441790+0000","last_became_active":"2023-11-09T03:48:57.440385+0000","last_became_peered":"2023-11-09T03:48:57.440385+0000","last_unstale":"2023-11-09T03:48:57.441790+0000","last_undegraded":"2023-11-09T03:48:57.441790+0000","last_fullsized":"2023-11-09T03:48:57.441790+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:45:04.731311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.429447+0000","last_change":"2023-11-09T03:48:57.050494+0000","last_active":"2023-11-09T03:48:57.429447+0000","last_peered":"2023-11-09T03:48:57.429447+0000","last_clean":"2023-11-09T03:48:57.429447+0000","last_became_active":"2023-11-09T03:48:57.050181+0000","last_became_peered":"2023-11-09T03:48:57.050181+0000","last_unstale":"2023-11-09T03:48:57.429447+0000","last_undegraded":"2023-11-09T03:48:57.429447+0000","last_fullsized":"2023-11-09T03:48:57.429447+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:07:35.452073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.044641+0000","last_change":"2023-11-09T03:48:01.453600+0000","last_active":"2023-11-09T03:48:57.044641+0000","last_peered":"2023-11-09T03:48:57.044641+0000","last_clean":"2023-11-09T03:48:57.044641+0000","last_became_active":"2023-11-09T03:48:01.453330+0000","last_became_peered":"2023-11-09T03:48:01.453330+0000","last_unstale":"2023-11-09T03:48:57.044641+0000","last_undegraded":"2023-11-09T03:48:57.044641+0000","last_fullsized":"2023-11-09T03:48:57.044641+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:08:47.931939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036796+0000","last_change":"2023-11-09T03:48:03.328210+0000","last_active":"2023-11-09T03:48:57.036796+0000","last_peered":"2023-11-09T03:48:57.036796+0000","last_clean":"2023-11-09T03:48:57.036796+0000","last_became_active":"2023-11-09T03:48:00.450002+0000","last_became_peered":"2023-11-09T03:48:00.450002+0000","last_unstale":"2023-11-09T03:48:57.036796+0000","last_undegraded":"2023-11-09T03:48:57.036796+0000","last_fullsized":"2023-11-09T03:48:57.036796+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:50.391187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036521900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.046169+0000","last_change":"2023-11-09T03:47:02.619144+0000","last_active":"2023-11-09T03:48:57.046169+0000","last_peered":"2023-11-09T03:48:57.046169+0000","last_clean":"2023-11-09T03:48:57.046169+0000","last_became_active":"2023-11-09T03:47:01.590672+0000","last_became_peered":"2023-11-09T03:47:01.590672+0000","last_unstale":"2023-11-09T03:48:57.046169+0000","last_undegraded":"2023-11-09T03:48:57.046169+0000","last_fullsized":"2023-11-09T03:48:57.046169+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:03.025119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060997500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.437411+0000","last_change":"2023-11-09T03:48:57.434355+0000","last_active":"2023-11-09T03:48:57.437411+0000","last_peered":"2023-11-09T03:48:57.437411+0000","last_clean":"2023-11-09T03:48:57.437411+0000","last_became_active":"2023-11-09T03:48:57.434117+0000","last_became_peered":"2023-11-09T03:48:57.434117+0000","last_unstale":"2023-11-09T03:48:57.437411+0000","last_undegraded":"2023-11-09T03:48:57.437411+0000","last_fullsized":"2023-11-09T03:48:57.437411+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:02:49.169981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.435063+0000","last_change":"2023-11-09T03:48:57.055386+0000","last_active":"2023-11-09T03:48:57.435063+0000","last_peered":"2023-11-09T03:48:57.435063+0000","last_clean":"2023-11-09T03:48:57.435063+0000","last_became_active":"2023-11-09T03:48:57.055104+0000","last_became_peered":"2023-11-09T03:48:57.055104+0000","last_unstale":"2023-11-09T03:48:57.435063+0000","last_undegraded":"2023-11-09T03:48:57.435063+0000","last_fullsized":"2023-11-09T03:48:57.435063+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:33:03.786483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.037237+0000","last_change":"2023-11-09T03:48:01.447278+0000","last_active":"2023-11-09T03:48:57.037237+0000","last_peered":"2023-11-09T03:48:57.037237+0000","last_clean":"2023-11-09T03:48:57.037237+0000","last_became_active":"2023-11-09T03:48:01.446991+0000","last_became_peered":"2023-11-09T03:48:01.446991+0000","last_unstale":"2023-11-09T03:48:57.037237+0000","last_undegraded":"2023-11-09T03:48:57.037237+0000","last_fullsized":"2023-11-09T03:48:57.037237+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:27:49.190426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036648+0000","last_change":"2023-11-09T03:48:03.328278+0000","last_active":"2023-11-09T03:48:57.036648+0000","last_peered":"2023-11-09T03:48:57.036648+0000","last_clean":"2023-11-09T03:48:57.036648+0000","last_became_active":"2023-11-09T03:48:00.443213+0000","last_became_peered":"2023-11-09T03:48:00.443213+0000","last_unstale":"2023-11-09T03:48:57.036648+0000","last_undegraded":"2023-11-09T03:48:57.036648+0000","last_fullsized":"2023-11-09T03:48:57.036648+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:20:59.702297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030886100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.045843+0000","last_change":"2023-11-09T03:47:02.619079+0000","last_active":"2023-11-09T03:48:57.045843+0000","last_peered":"2023-11-09T03:48:57.045843+0000","last_clean":"2023-11-09T03:48:57.045843+0000","last_became_active":"2023-11-09T03:47:01.591421+0000","last_became_peered":"2023-11-09T03:47:01.591421+0000","last_unstale":"2023-11-09T03:48:57.045843+0000","last_undegraded":"2023-11-09T03:48:57.045843+0000","last_fullsized":"2023-11-09T03:48:57.045843+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:18:58.230606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068736399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.448693+0000","last_change":"2023-11-09T03:48:57.445015+0000","last_active":"2023-11-09T03:48:57.448693+0000","last_peered":"2023-11-09T03:48:57.448693+0000","last_clean":"2023-11-09T03:48:57.448693+0000","last_became_active":"2023-11-09T03:48:57.444759+0000","last_became_peered":"2023-11-09T03:48:57.444759+0000","last_unstale":"2023-11-09T03:48:57.448693+0000","last_undegraded":"2023-11-09T03:48:57.448693+0000","last_fullsized":"2023-11-09T03:48:57.448693+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:36:54.461560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.426066+0000","last_change":"2023-11-09T03:48:57.047168+0000","last_active":"2023-11-09T03:48:57.426066+0000","last_peered":"2023-11-09T03:48:57.426066+0000","last_clean":"2023-11-09T03:48:57.426066+0000","last_became_active":"2023-11-09T03:48:57.046914+0000","last_became_peered":"2023-11-09T03:48:57.046914+0000","last_unstale":"2023-11-09T03:48:57.426066+0000","last_undegraded":"2023-11-09T03:48:57.426066+0000","last_fullsized":"2023-11-09T03:48:57.426066+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:55:04.192277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.037212+0000","last_change":"2023-11-09T03:48:01.449069+0000","last_active":"2023-11-09T03:48:57.037212+0000","last_peered":"2023-11-09T03:48:57.037212+0000","last_clean":"2023-11-09T03:48:57.037212+0000","last_became_active":"2023-11-09T03:48:01.448810+0000","last_became_peered":"2023-11-09T03:48:01.448810+0000","last_unstale":"2023-11-09T03:48:57.037212+0000","last_undegraded":"2023-11-09T03:48:57.037212+0000","last_fullsized":"2023-11-09T03:48:57.037212+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:56:51.158572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.042858+0000","last_change":"2023-11-09T03:48:02.612067+0000","last_active":"2023-11-09T03:48:57.042858+0000","last_peered":"2023-11-09T03:48:57.042858+0000","last_clean":"2023-11-09T03:48:57.042858+0000","last_became_active":"2023-11-09T03:48:00.443296+0000","last_became_peered":"2023-11-09T03:48:00.443296+0000","last_unstale":"2023-11-09T03:48:57.042858+0000","last_undegraded":"2023-11-09T03:48:57.042858+0000","last_fullsized":"2023-11-09T03:48:57.042858+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:27:48.184420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010423330000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.033997+0000","last_change":"2023-11-09T03:47:03.148529+0000","last_active":"2023-11-09T03:48:57.033997+0000","last_peered":"2023-11-09T03:48:57.033997+0000","last_clean":"2023-11-09T03:48:57.033997+0000","last_became_active":"2023-11-09T03:47:01.584148+0000","last_became_peered":"2023-11-09T03:47:01.584148+0000","last_unstale":"2023-11-09T03:48:57.033997+0000","last_undegraded":"2023-11-09T03:48:57.033997+0000","last_fullsized":"2023-11-09T03:48:57.033997+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:40:00.237248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061475600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.444200+0000","last_change":"2023-11-09T03:48:57.440696+0000","last_active":"2023-11-09T03:48:57.444200+0000","last_peered":"2023-11-09T03:48:57.444200+0000","last_clean":"2023-11-09T03:48:57.444200+0000","last_became_active":"2023-11-09T03:48:57.440448+0000","last_became_peered":"2023-11-09T03:48:57.440448+0000","last_unstale":"2023-11-09T03:48:57.444200+0000","last_undegraded":"2023-11-09T03:48:57.444200+0000","last_fullsized":"2023-11-09T03:48:57.444200+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:56:03.482548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.427955+0000","last_change":"2023-11-09T03:48:57.047017+0000","last_active":"2023-11-09T03:48:57.427955+0000","last_peered":"2023-11-09T03:48:57.427955+0000","last_clean":"2023-11-09T03:48:57.427955+0000","last_became_active":"2023-11-09T03:48:57.046736+0000","last_became_peered":"2023-11-09T03:48:57.046736+0000","last_unstale":"2023-11-09T03:48:57.427955+0000","last_undegraded":"2023-11-09T03:48:57.427955+0000","last_fullsized":"2023-11-09T03:48:57.427955+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:06:46.097576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.042936+0000","last_change":"2023-11-09T03:48:01.451750+0000","last_active":"2023-11-09T03:48:57.042936+0000","last_peered":"2023-11-09T03:48:57.042936+0000","last_clean":"2023-11-09T03:48:57.042936+0000","last_became_active":"2023-11-09T03:48:01.451393+0000","last_became_peered":"2023-11-09T03:48:01.451393+0000","last_unstale":"2023-11-09T03:48:57.042936+0000","last_undegraded":"2023-11-09T03:48:57.042936+0000","last_fullsized":"2023-11-09T03:48:57.042936+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:56:51.158572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.046045+0000","last_change":"2023-11-09T03:48:02.611721+0000","last_active":"2023-11-09T03:48:57.046045+0000","last_peered":"2023-11-09T03:48:57.046045+0000","last_clean":"2023-11-09T03:48:57.046045+0000","last_became_active":"2023-11-09T03:48:00.439557+0000","last_became_peered":"2023-11-09T03:48:00.439557+0000","last_unstale":"2023-11-09T03:48:57.046045+0000","last_undegraded":"2023-11-09T03:48:57.046045+0000","last_fullsized":"2023-11-09T03:48:57.046045+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:35:53.864627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001037187,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":57,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.041573+0000","last_change":"2023-11-09T03:47:03.148217+0000","last_active":"2023-11-09T03:48:57.041573+0000","last_peered":"2023-11-09T03:48:57.041573+0000","last_clean":"2023-11-09T03:48:57.041573+0000","last_became_active":"2023-11-09T03:47:01.593424+0000","last_became_peered":"2023-11-09T03:47:01.593424+0000","last_unstale":"2023-11-09T03:48:57.041573+0000","last_undegraded":"2023-11-09T03:48:57.041573+0000","last_fullsized":"2023-11-09T03:48:57.041573+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:19:32.693691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001769865,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.447017+0000","last_change":"2023-11-09T03:48:57.442650+0000","last_active":"2023-11-09T03:48:57.447017+0000","last_peered":"2023-11-09T03:48:57.447017+0000","last_clean":"2023-11-09T03:48:57.447017+0000","last_became_active":"2023-11-09T03:48:57.442373+0000","last_became_peered":"2023-11-09T03:48:57.442373+0000","last_unstale":"2023-11-09T03:48:57.447017+0000","last_undegraded":"2023-11-09T03:48:57.447017+0000","last_fullsized":"2023-11-09T03:48:57.447017+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:21:31.419099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.435532+0000","last_change":"2023-11-09T03:48:57.050569+0000","last_active":"2023-11-09T03:48:57.435532+0000","last_peered":"2023-11-09T03:48:57.435532+0000","last_clean":"2023-11-09T03:48:57.435532+0000","last_became_active":"2023-11-09T03:48:57.050291+0000","last_became_peered":"2023-11-09T03:48:57.050291+0000","last_unstale":"2023-11-09T03:48:57.435532+0000","last_undegraded":"2023-11-09T03:48:57.435532+0000","last_fullsized":"2023-11-09T03:48:57.435532+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:02:35.821071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.033236+0000","last_change":"2023-11-09T03:48:01.447617+0000","last_active":"2023-11-09T03:48:57.033236+0000","last_peered":"2023-11-09T03:48:57.033236+0000","last_clean":"2023-11-09T03:48:57.033236+0000","last_became_active":"2023-11-09T03:48:01.447329+0000","last_became_peered":"2023-11-09T03:48:01.447329+0000","last_unstale":"2023-11-09T03:48:57.033236+0000","last_undegraded":"2023-11-09T03:48:57.033236+0000","last_fullsized":"2023-11-09T03:48:57.033236+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:54:52.433604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036831+0000","last_change":"2023-11-09T03:48:02.442427+0000","last_active":"2023-11-09T03:48:57.036831+0000","last_peered":"2023-11-09T03:48:57.036831+0000","last_clean":"2023-11-09T03:48:57.036831+0000","last_became_active":"2023-11-09T03:48:00.444856+0000","last_became_peered":"2023-11-09T03:48:00.444856+0000","last_unstale":"2023-11-09T03:48:57.036831+0000","last_undegraded":"2023-11-09T03:48:57.036831+0000","last_fullsized":"2023-11-09T03:48:57.036831+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:53:42.763434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011927730000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.033114+0000","last_change":"2023-11-09T03:47:03.146470+0000","last_active":"2023-11-09T03:48:57.033114+0000","last_peered":"2023-11-09T03:48:57.033114+0000","last_clean":"2023-11-09T03:48:57.033114+0000","last_became_active":"2023-11-09T03:47:01.587304+0000","last_became_peered":"2023-11-09T03:47:01.587304+0000","last_unstale":"2023-11-09T03:48:57.033114+0000","last_undegraded":"2023-11-09T03:48:57.033114+0000","last_fullsized":"2023-11-09T03:48:57.033114+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:58:23.339287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00110395,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.443966+0000","last_change":"2023-11-09T03:48:57.440684+0000","last_active":"2023-11-09T03:48:57.443966+0000","last_peered":"2023-11-09T03:48:57.443966+0000","last_clean":"2023-11-09T03:48:57.443966+0000","last_became_active":"2023-11-09T03:48:57.439352+0000","last_became_peered":"2023-11-09T03:48:57.439352+0000","last_unstale":"2023-11-09T03:48:57.443966+0000","last_undegraded":"2023-11-09T03:48:57.443966+0000","last_fullsized":"2023-11-09T03:48:57.443966+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:22.900543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.434897+0000","last_change":"2023-11-09T03:48:57.048247+0000","last_active":"2023-11-09T03:48:57.434897+0000","last_peered":"2023-11-09T03:48:57.434897+0000","last_clean":"2023-11-09T03:48:57.434897+0000","last_became_active":"2023-11-09T03:48:57.047624+0000","last_became_peered":"2023-11-09T03:48:57.047624+0000","last_unstale":"2023-11-09T03:48:57.434897+0000","last_undegraded":"2023-11-09T03:48:57.434897+0000","last_fullsized":"2023-11-09T03:48:57.434897+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:58:16.010165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.040582+0000","last_change":"2023-11-09T03:48:01.449406+0000","last_active":"2023-11-09T03:48:57.040582+0000","last_peered":"2023-11-09T03:48:57.040582+0000","last_clean":"2023-11-09T03:48:57.040582+0000","last_became_active":"2023-11-09T03:48:01.449130+0000","last_became_peered":"2023-11-09T03:48:01.449130+0000","last_unstale":"2023-11-09T03:48:57.040582+0000","last_undegraded":"2023-11-09T03:48:57.040582+0000","last_fullsized":"2023-11-09T03:48:57.040582+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:20:36.303940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.033343+0000","last_change":"2023-11-09T03:48:02.441801+0000","last_active":"2023-11-09T03:48:57.033343+0000","last_peered":"2023-11-09T03:48:57.033343+0000","last_clean":"2023-11-09T03:48:57.033343+0000","last_became_active":"2023-11-09T03:48:00.444975+0000","last_became_peered":"2023-11-09T03:48:00.444975+0000","last_unstale":"2023-11-09T03:48:57.033343+0000","last_undegraded":"2023-11-09T03:48:57.033343+0000","last_fullsized":"2023-11-09T03:48:57.033343+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:39:31.347349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061165099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":57,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.049538+0000","last_change":"2023-11-09T03:47:03.147367+0000","last_active":"2023-11-09T03:48:57.049538+0000","last_peered":"2023-11-09T03:48:57.049538+0000","last_clean":"2023-11-09T03:48:57.049538+0000","last_became_active":"2023-11-09T03:47:01.588359+0000","last_became_peered":"2023-11-09T03:47:01.588359+0000","last_unstale":"2023-11-09T03:48:57.049538+0000","last_undegraded":"2023-11-09T03:48:57.049538+0000","last_fullsized":"2023-11-09T03:48:57.049538+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:21:34.303068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014106660000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.447677+0000","last_change":"2023-11-09T03:48:57.440700+0000","last_active":"2023-11-09T03:48:57.447677+0000","last_peered":"2023-11-09T03:48:57.447677+0000","last_clean":"2023-11-09T03:48:57.447677+0000","last_became_active":"2023-11-09T03:48:57.440436+0000","last_became_peered":"2023-11-09T03:48:57.440436+0000","last_unstale":"2023-11-09T03:48:57.447677+0000","last_undegraded":"2023-11-09T03:48:57.447677+0000","last_fullsized":"2023-11-09T03:48:57.447677+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:44:32.089939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.434741+0000","last_change":"2023-11-09T03:48:57.051186+0000","last_active":"2023-11-09T03:48:57.434741+0000","last_peered":"2023-11-09T03:48:57.434741+0000","last_clean":"2023-11-09T03:48:57.434741+0000","last_became_active":"2023-11-09T03:48:57.050935+0000","last_became_peered":"2023-11-09T03:48:57.050935+0000","last_unstale":"2023-11-09T03:48:57.434741+0000","last_undegraded":"2023-11-09T03:48:57.434741+0000","last_fullsized":"2023-11-09T03:48:57.434741+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:28:06.094368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.042287+0000","last_change":"2023-11-09T03:48:01.451866+0000","last_active":"2023-11-09T03:48:57.042287+0000","last_peered":"2023-11-09T03:48:57.042287+0000","last_clean":"2023-11-09T03:48:57.042287+0000","last_became_active":"2023-11-09T03:48:01.451350+0000","last_became_peered":"2023-11-09T03:48:01.451350+0000","last_unstale":"2023-11-09T03:48:57.042287+0000","last_undegraded":"2023-11-09T03:48:57.042287+0000","last_fullsized":"2023-11-09T03:48:57.042287+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:36.735029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036657+0000","last_change":"2023-11-09T03:48:02.441648+0000","last_active":"2023-11-09T03:48:57.036657+0000","last_peered":"2023-11-09T03:48:57.036657+0000","last_clean":"2023-11-09T03:48:57.036657+0000","last_became_active":"2023-11-09T03:48:00.448541+0000","last_became_peered":"2023-11-09T03:48:00.448541+0000","last_unstale":"2023-11-09T03:48:57.036657+0000","last_undegraded":"2023-11-09T03:48:57.036657+0000","last_fullsized":"2023-11-09T03:48:57.036657+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:35.729023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036727+0000","last_change":"2023-11-09T03:47:02.581682+0000","last_active":"2023-11-09T03:48:57.036727+0000","last_peered":"2023-11-09T03:48:57.036727+0000","last_clean":"2023-11-09T03:48:57.036727+0000","last_became_active":"2023-11-09T03:47:01.593051+0000","last_became_peered":"2023-11-09T03:47:01.593051+0000","last_unstale":"2023-11-09T03:48:57.036727+0000","last_undegraded":"2023-11-09T03:48:57.036727+0000","last_fullsized":"2023-11-09T03:48:57.036727+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047277799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.446756+0000","last_change":"2023-11-09T03:48:57.443729+0000","last_active":"2023-11-09T03:48:57.446756+0000","last_peered":"2023-11-09T03:48:57.446756+0000","last_clean":"2023-11-09T03:48:57.446756+0000","last_became_active":"2023-11-09T03:48:57.443483+0000","last_became_peered":"2023-11-09T03:48:57.443483+0000","last_unstale":"2023-11-09T03:48:57.446756+0000","last_undegraded":"2023-11-09T03:48:57.446756+0000","last_fullsized":"2023-11-09T03:48:57.446756+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:21:08.155680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.433446+0000","last_change":"2023-11-09T03:48:57.055649+0000","last_active":"2023-11-09T03:48:57.433446+0000","last_peered":"2023-11-09T03:48:57.433446+0000","last_clean":"2023-11-09T03:48:57.433446+0000","last_became_active":"2023-11-09T03:48:57.055366+0000","last_became_peered":"2023-11-09T03:48:57.055366+0000","last_unstale":"2023-11-09T03:48:57.433446+0000","last_undegraded":"2023-11-09T03:48:57.433446+0000","last_fullsized":"2023-11-09T03:48:57.433446+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:56:52.342978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.043544+0000","last_change":"2023-11-09T03:48:01.455784+0000","last_active":"2023-11-09T03:48:57.043544+0000","last_peered":"2023-11-09T03:48:57.043544+0000","last_clean":"2023-11-09T03:48:57.043544+0000","last_became_active":"2023-11-09T03:48:01.455532+0000","last_became_peered":"2023-11-09T03:48:01.455532+0000","last_unstale":"2023-11-09T03:48:57.043544+0000","last_undegraded":"2023-11-09T03:48:57.043544+0000","last_fullsized":"2023-11-09T03:48:57.043544+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:21:23.156761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.042520+0000","last_change":"2023-11-09T03:48:02.611298+0000","last_active":"2023-11-09T03:48:57.042520+0000","last_peered":"2023-11-09T03:48:57.042520+0000","last_clean":"2023-11-09T03:48:57.042520+0000","last_became_active":"2023-11-09T03:48:00.441261+0000","last_became_peered":"2023-11-09T03:48:00.441261+0000","last_unstale":"2023-11-09T03:48:57.042520+0000","last_undegraded":"2023-11-09T03:48:57.042520+0000","last_fullsized":"2023-11-09T03:48:57.042520+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:50:55.299235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081460199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036362+0000","last_change":"2023-11-09T03:47:03.146309+0000","last_active":"2023-11-09T03:48:57.036362+0000","last_peered":"2023-11-09T03:48:57.036362+0000","last_clean":"2023-11-09T03:48:57.036362+0000","last_became_active":"2023-11-09T03:47:01.587560+0000","last_became_peered":"2023-11-09T03:47:01.587560+0000","last_unstale":"2023-11-09T03:48:57.036362+0000","last_undegraded":"2023-11-09T03:48:57.036362+0000","last_fullsized":"2023-11-09T03:48:57.036362+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:27:55.271945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070988400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.442204+0000","last_change":"2023-11-09T03:48:57.435551+0000","last_active":"2023-11-09T03:48:57.442204+0000","last_peered":"2023-11-09T03:48:57.442204+0000","last_clean":"2023-11-09T03:48:57.442204+0000","last_became_active":"2023-11-09T03:48:57.435303+0000","last_became_peered":"2023-11-09T03:48:57.435303+0000","last_unstale":"2023-11-09T03:48:57.442204+0000","last_undegraded":"2023-11-09T03:48:57.442204+0000","last_fullsized":"2023-11-09T03:48:57.442204+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:01:29.460175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.436518+0000","last_change":"2023-11-09T03:48:57.050746+0000","last_active":"2023-11-09T03:48:57.436518+0000","last_peered":"2023-11-09T03:48:57.436518+0000","last_clean":"2023-11-09T03:48:57.436518+0000","last_became_active":"2023-11-09T03:48:57.050448+0000","last_became_peered":"2023-11-09T03:48:57.050448+0000","last_unstale":"2023-11-09T03:48:57.436518+0000","last_undegraded":"2023-11-09T03:48:57.436518+0000","last_fullsized":"2023-11-09T03:48:57.436518+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:25:45.917439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.048037+0000","last_change":"2023-11-09T03:48:01.452477+0000","last_active":"2023-11-09T03:48:57.048037+0000","last_peered":"2023-11-09T03:48:57.048037+0000","last_clean":"2023-11-09T03:48:57.048037+0000","last_became_active":"2023-11-09T03:48:01.452186+0000","last_became_peered":"2023-11-09T03:48:01.452186+0000","last_unstale":"2023-11-09T03:48:57.048037+0000","last_undegraded":"2023-11-09T03:48:57.048037+0000","last_fullsized":"2023-11-09T03:48:57.048037+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:32:21.825778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.034003+0000","last_change":"2023-11-09T03:48:02.442101+0000","last_active":"2023-11-09T03:48:57.034003+0000","last_peered":"2023-11-09T03:48:57.034003+0000","last_clean":"2023-11-09T03:48:57.034003+0000","last_became_active":"2023-11-09T03:48:00.449354+0000","last_became_peered":"2023-11-09T03:48:00.449354+0000","last_unstale":"2023-11-09T03:48:57.034003+0000","last_undegraded":"2023-11-09T03:48:57.034003+0000","last_fullsized":"2023-11-09T03:48:57.034003+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:06:32.906582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093455000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.043974+0000","last_change":"2023-11-09T03:47:02.725125+0000","last_active":"2023-11-09T03:48:57.043974+0000","last_peered":"2023-11-09T03:48:57.043974+0000","last_clean":"2023-11-09T03:48:57.043974+0000","last_became_active":"2023-11-09T03:47:01.591521+0000","last_became_peered":"2023-11-09T03:47:01.591521+0000","last_unstale":"2023-11-09T03:48:57.043974+0000","last_undegraded":"2023-11-09T03:48:57.043974+0000","last_fullsized":"2023-11-09T03:48:57.043974+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:52:40.076542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000758675,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.443753+0000","last_change":"2023-11-09T03:48:57.441295+0000","last_active":"2023-11-09T03:48:57.443753+0000","last_peered":"2023-11-09T03:48:57.443753+0000","last_clean":"2023-11-09T03:48:57.443753+0000","last_became_active":"2023-11-09T03:48:57.441049+0000","last_became_peered":"2023-11-09T03:48:57.441049+0000","last_unstale":"2023-11-09T03:48:57.443753+0000","last_undegraded":"2023-11-09T03:48:57.443753+0000","last_fullsized":"2023-11-09T03:48:57.443753+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:02:36.826029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.430975+0000","last_change":"2023-11-09T03:48:57.045457+0000","last_active":"2023-11-09T03:48:57.430975+0000","last_peered":"2023-11-09T03:48:57.430975+0000","last_clean":"2023-11-09T03:48:57.430975+0000","last_became_active":"2023-11-09T03:48:57.045213+0000","last_became_peered":"2023-11-09T03:48:57.045213+0000","last_unstale":"2023-11-09T03:48:57.430975+0000","last_undegraded":"2023-11-09T03:48:57.430975+0000","last_fullsized":"2023-11-09T03:48:57.430975+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:01:10.734137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.043341+0000","last_change":"2023-11-09T03:48:01.448129+0000","last_active":"2023-11-09T03:48:57.043341+0000","last_peered":"2023-11-09T03:48:57.043341+0000","last_clean":"2023-11-09T03:48:57.043341+0000","last_became_active":"2023-11-09T03:48:01.447844+0000","last_became_peered":"2023-11-09T03:48:01.447844+0000","last_unstale":"2023-11-09T03:48:57.043341+0000","last_undegraded":"2023-11-09T03:48:57.043341+0000","last_fullsized":"2023-11-09T03:48:57.043341+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:54:52.433604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.048068+0000","last_change":"2023-11-09T03:48:02.612450+0000","last_active":"2023-11-09T03:48:57.048068+0000","last_peered":"2023-11-09T03:48:57.048068+0000","last_clean":"2023-11-09T03:48:57.048068+0000","last_became_active":"2023-11-09T03:48:00.441855+0000","last_became_peered":"2023-11-09T03:48:00.441855+0000","last_unstale":"2023-11-09T03:48:57.048068+0000","last_undegraded":"2023-11-09T03:48:57.048068+0000","last_fullsized":"2023-11-09T03:48:57.048068+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:53:42.763434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095368899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.033862+0000","last_change":"2023-11-09T03:47:03.147289+0000","last_active":"2023-11-09T03:48:57.033862+0000","last_peered":"2023-11-09T03:48:57.033862+0000","last_clean":"2023-11-09T03:48:57.033862+0000","last_became_active":"2023-11-09T03:47:01.587784+0000","last_became_peered":"2023-11-09T03:47:01.587784+0000","last_unstale":"2023-11-09T03:48:57.033862+0000","last_undegraded":"2023-11-09T03:48:57.033862+0000","last_fullsized":"2023-11-09T03:48:57.033862+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:11:23.824112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038781299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.436337+0000","last_change":"2023-11-09T03:48:57.431983+0000","last_active":"2023-11-09T03:48:57.436337+0000","last_peered":"2023-11-09T03:48:57.436337+0000","last_clean":"2023-11-09T03:48:57.436337+0000","last_became_active":"2023-11-09T03:48:57.431716+0000","last_became_peered":"2023-11-09T03:48:57.431716+0000","last_unstale":"2023-11-09T03:48:57.436337+0000","last_undegraded":"2023-11-09T03:48:57.436337+0000","last_fullsized":"2023-11-09T03:48:57.436337+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:50:34.258596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.434937+0000","last_change":"2023-11-09T03:48:57.053092+0000","last_active":"2023-11-09T03:48:57.434937+0000","last_peered":"2023-11-09T03:48:57.434937+0000","last_clean":"2023-11-09T03:48:57.434937+0000","last_became_active":"2023-11-09T03:48:57.052820+0000","last_became_peered":"2023-11-09T03:48:57.052820+0000","last_unstale":"2023-11-09T03:48:57.434937+0000","last_undegraded":"2023-11-09T03:48:57.434937+0000","last_fullsized":"2023-11-09T03:48:57.434937+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:07:10.450500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.037169+0000","last_change":"2023-11-09T03:48:01.449617+0000","last_active":"2023-11-09T03:48:57.037169+0000","last_peered":"2023-11-09T03:48:57.037169+0000","last_clean":"2023-11-09T03:48:57.037169+0000","last_became_active":"2023-11-09T03:48:01.449328+0000","last_became_peered":"2023-11-09T03:48:01.449328+0000","last_unstale":"2023-11-09T03:48:57.037169+0000","last_undegraded":"2023-11-09T03:48:57.037169+0000","last_fullsized":"2023-11-09T03:48:57.037169+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:51:18.760059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.033640+0000","last_change":"2023-11-09T03:48:02.442215+0000","last_active":"2023-11-09T03:48:57.033640+0000","last_peered":"2023-11-09T03:48:57.033640+0000","last_clean":"2023-11-09T03:48:57.033640+0000","last_became_active":"2023-11-09T03:48:00.444720+0000","last_became_peered":"2023-11-09T03:48:00.444720+0000","last_unstale":"2023-11-09T03:48:57.033640+0000","last_undegraded":"2023-11-09T03:48:57.033640+0000","last_fullsized":"2023-11-09T03:48:57.033640+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:55:55.735668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001407176,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.048221+0000","last_change":"2023-11-09T03:47:02.619346+0000","last_active":"2023-11-09T03:48:57.048221+0000","last_peered":"2023-11-09T03:48:57.048221+0000","last_clean":"2023-11-09T03:48:57.048221+0000","last_became_active":"2023-11-09T03:47:01.590389+0000","last_became_peered":"2023-11-09T03:47:01.590389+0000","last_unstale":"2023-11-09T03:48:57.048221+0000","last_undegraded":"2023-11-09T03:48:57.048221+0000","last_fullsized":"2023-11-09T03:48:57.048221+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000384072,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.451137+0000","last_change":"2023-11-09T03:48:57.445676+0000","last_active":"2023-11-09T03:48:57.451137+0000","last_peered":"2023-11-09T03:48:57.451137+0000","last_clean":"2023-11-09T03:48:57.451137+0000","last_became_active":"2023-11-09T03:48:57.445349+0000","last_became_peered":"2023-11-09T03:48:57.445349+0000","last_unstale":"2023-11-09T03:48:57.451137+0000","last_undegraded":"2023-11-09T03:48:57.451137+0000","last_fullsized":"2023-11-09T03:48:57.451137+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:44:32.408986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.430067+0000","last_change":"2023-11-09T03:48:57.052325+0000","last_active":"2023-11-09T03:48:57.430067+0000","last_peered":"2023-11-09T03:48:57.430067+0000","last_clean":"2023-11-09T03:48:57.430067+0000","last_became_active":"2023-11-09T03:48:57.051915+0000","last_became_peered":"2023-11-09T03:48:57.051915+0000","last_unstale":"2023-11-09T03:48:57.430067+0000","last_undegraded":"2023-11-09T03:48:57.430067+0000","last_fullsized":"2023-11-09T03:48:57.430067+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:14:36.990468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.041871+0000","last_change":"2023-11-09T03:48:01.450018+0000","last_active":"2023-11-09T03:48:57.041871+0000","last_peered":"2023-11-09T03:48:57.041871+0000","last_clean":"2023-11-09T03:48:57.041871+0000","last_became_active":"2023-11-09T03:48:01.449749+0000","last_became_peered":"2023-11-09T03:48:01.449749+0000","last_unstale":"2023-11-09T03:48:57.041871+0000","last_undegraded":"2023-11-09T03:48:57.041871+0000","last_fullsized":"2023-11-09T03:48:57.041871+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:21:23.156761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.041011+0000","last_change":"2023-11-09T03:48:02.524216+0000","last_active":"2023-11-09T03:48:57.041011+0000","last_peered":"2023-11-09T03:48:57.041011+0000","last_clean":"2023-11-09T03:48:57.041011+0000","last_became_active":"2023-11-09T03:48:00.446972+0000","last_became_peered":"2023-11-09T03:48:00.446972+0000","last_unstale":"2023-11-09T03:48:57.041011+0000","last_undegraded":"2023-11-09T03:48:57.041011+0000","last_fullsized":"2023-11-09T03:48:57.041011+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:54:07.987773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000825255,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":63,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036699+0000","last_change":"2023-11-09T03:47:02.582393+0000","last_active":"2023-11-09T03:48:57.036699+0000","last_peered":"2023-11-09T03:48:57.036699+0000","last_clean":"2023-11-09T03:48:57.036699+0000","last_became_active":"2023-11-09T03:47:01.591752+0000","last_became_peered":"2023-11-09T03:47:01.591752+0000","last_unstale":"2023-11-09T03:48:57.036699+0000","last_undegraded":"2023-11-09T03:48:57.036699+0000","last_fullsized":"2023-11-09T03:48:57.036699+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:03.025119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067021599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.445270+0000","last_change":"2023-11-09T03:48:57.440690+0000","last_active":"2023-11-09T03:48:57.445270+0000","last_peered":"2023-11-09T03:48:57.445270+0000","last_clean":"2023-11-09T03:48:57.445270+0000","last_became_active":"2023-11-09T03:48:57.440425+0000","last_became_peered":"2023-11-09T03:48:57.440425+0000","last_unstale":"2023-11-09T03:48:57.445270+0000","last_undegraded":"2023-11-09T03:48:57.445270+0000","last_fullsized":"2023-11-09T03:48:57.445270+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:37:25.942349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.433424+0000","last_change":"2023-11-09T03:48:57.055564+0000","last_active":"2023-11-09T03:48:57.433424+0000","last_peered":"2023-11-09T03:48:57.433424+0000","last_clean":"2023-11-09T03:48:57.433424+0000","last_became_active":"2023-11-09T03:48:57.055276+0000","last_became_peered":"2023-11-09T03:48:57.055276+0000","last_unstale":"2023-11-09T03:48:57.433424+0000","last_undegraded":"2023-11-09T03:48:57.433424+0000","last_fullsized":"2023-11-09T03:48:57.433424+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:42:18.929121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.041083+0000","last_change":"2023-11-09T03:48:01.451462+0000","last_active":"2023-11-09T03:48:57.041083+0000","last_peered":"2023-11-09T03:48:57.041083+0000","last_clean":"2023-11-09T03:48:57.041083+0000","last_became_active":"2023-11-09T03:48:01.451188+0000","last_became_peered":"2023-11-09T03:48:01.451188+0000","last_unstale":"2023-11-09T03:48:57.041083+0000","last_undegraded":"2023-11-09T03:48:57.041083+0000","last_fullsized":"2023-11-09T03:48:57.041083+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:28.631801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036536+0000","last_change":"2023-11-09T03:48:02.682290+0000","last_active":"2023-11-09T03:48:57.036536+0000","last_peered":"2023-11-09T03:48:57.036536+0000","last_clean":"2023-11-09T03:48:57.036536+0000","last_became_active":"2023-11-09T03:48:00.444246+0000","last_became_peered":"2023-11-09T03:48:00.444246+0000","last_unstale":"2023-11-09T03:48:57.036536+0000","last_undegraded":"2023-11-09T03:48:57.036536+0000","last_fullsized":"2023-11-09T03:48:57.036536+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:09.487058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000423359,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":57,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.040154+0000","last_change":"2023-11-09T03:47:03.028744+0000","last_active":"2023-11-09T03:48:57.040154+0000","last_peered":"2023-11-09T03:48:57.040154+0000","last_clean":"2023-11-09T03:48:57.040154+0000","last_became_active":"2023-11-09T03:47:01.593890+0000","last_became_peered":"2023-11-09T03:47:01.593890+0000","last_unstale":"2023-11-09T03:48:57.040154+0000","last_undegraded":"2023-11-09T03:48:57.040154+0000","last_fullsized":"2023-11-09T03:48:57.040154+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:05.754357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053008299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.451206+0000","last_change":"2023-11-09T03:48:57.448257+0000","last_active":"2023-11-09T03:48:57.451206+0000","last_peered":"2023-11-09T03:48:57.451206+0000","last_clean":"2023-11-09T03:48:57.451206+0000","last_became_active":"2023-11-09T03:48:57.448007+0000","last_became_peered":"2023-11-09T03:48:57.448007+0000","last_unstale":"2023-11-09T03:48:57.451206+0000","last_undegraded":"2023-11-09T03:48:57.451206+0000","last_fullsized":"2023-11-09T03:48:57.451206+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:37:27.085476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.426070+0000","last_change":"2023-11-09T03:48:57.046917+0000","last_active":"2023-11-09T03:48:57.426070+0000","last_peered":"2023-11-09T03:48:57.426070+0000","last_clean":"2023-11-09T03:48:57.426070+0000","last_became_active":"2023-11-09T03:48:57.046465+0000","last_became_peered":"2023-11-09T03:48:57.046465+0000","last_unstale":"2023-11-09T03:48:57.426070+0000","last_undegraded":"2023-11-09T03:48:57.426070+0000","last_fullsized":"2023-11-09T03:48:57.426070+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:20:44.093352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.040104+0000","last_change":"2023-11-09T03:48:01.450144+0000","last_active":"2023-11-09T03:48:57.040104+0000","last_peered":"2023-11-09T03:48:57.040104+0000","last_clean":"2023-11-09T03:48:57.040104+0000","last_became_active":"2023-11-09T03:48:01.449859+0000","last_became_peered":"2023-11-09T03:48:01.449859+0000","last_unstale":"2023-11-09T03:48:57.040104+0000","last_undegraded":"2023-11-09T03:48:57.040104+0000","last_fullsized":"2023-11-09T03:48:57.040104+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:57:56.855869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036399+0000","last_change":"2023-11-09T03:48:02.681869+0000","last_active":"2023-11-09T03:48:57.036399+0000","last_peered":"2023-11-09T03:48:57.036399+0000","last_clean":"2023-11-09T03:48:57.036399+0000","last_became_active":"2023-11-09T03:48:00.444928+0000","last_became_peered":"2023-11-09T03:48:00.444928+0000","last_unstale":"2023-11-09T03:48:57.036399+0000","last_undegraded":"2023-11-09T03:48:57.036399+0000","last_fullsized":"2023-11-09T03:48:57.036399+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:59:58.695636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044822599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036609+0000","last_change":"2023-11-09T03:47:03.146783+0000","last_active":"2023-11-09T03:48:57.036609+0000","last_peered":"2023-11-09T03:48:57.036609+0000","last_clean":"2023-11-09T03:48:57.036609+0000","last_became_active":"2023-11-09T03:47:01.586923+0000","last_became_peered":"2023-11-09T03:47:01.586923+0000","last_unstale":"2023-11-09T03:48:57.036609+0000","last_undegraded":"2023-11-09T03:48:57.036609+0000","last_fullsized":"2023-11-09T03:48:57.036609+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:37:50.326442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060384700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.441714+0000","last_change":"2023-11-09T03:48:57.432183+0000","last_active":"2023-11-09T03:48:57.441714+0000","last_peered":"2023-11-09T03:48:57.441714+0000","last_clean":"2023-11-09T03:48:57.441714+0000","last_became_active":"2023-11-09T03:48:57.431684+0000","last_became_peered":"2023-11-09T03:48:57.431684+0000","last_unstale":"2023-11-09T03:48:57.441714+0000","last_undegraded":"2023-11-09T03:48:57.441714+0000","last_fullsized":"2023-11-09T03:48:57.441714+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:30:30.967923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.425579+0000","last_change":"2023-11-09T03:48:57.040734+0000","last_active":"2023-11-09T03:48:57.425579+0000","last_peered":"2023-11-09T03:48:57.425579+0000","last_clean":"2023-11-09T03:48:57.425579+0000","last_became_active":"2023-11-09T03:48:57.040161+0000","last_became_peered":"2023-11-09T03:48:57.040161+0000","last_unstale":"2023-11-09T03:48:57.425579+0000","last_undegraded":"2023-11-09T03:48:57.425579+0000","last_fullsized":"2023-11-09T03:48:57.425579+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:37.857562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.049439+0000","last_change":"2023-11-09T03:48:01.448383+0000","last_active":"2023-11-09T03:48:57.049439+0000","last_peered":"2023-11-09T03:48:57.049439+0000","last_clean":"2023-11-09T03:48:57.049439+0000","last_became_active":"2023-11-09T03:48:01.448050+0000","last_became_peered":"2023-11-09T03:48:01.448050+0000","last_unstale":"2023-11-09T03:48:57.049439+0000","last_undegraded":"2023-11-09T03:48:57.049439+0000","last_fullsized":"2023-11-09T03:48:57.049439+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:53:44.478182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.040656+0000","last_change":"2023-11-09T03:48:02.523794+0000","last_active":"2023-11-09T03:48:57.040656+0000","last_peered":"2023-11-09T03:48:57.040656+0000","last_clean":"2023-11-09T03:48:57.040656+0000","last_became_active":"2023-11-09T03:48:00.440255+0000","last_became_peered":"2023-11-09T03:48:00.440255+0000","last_unstale":"2023-11-09T03:48:57.040656+0000","last_undegraded":"2023-11-09T03:48:57.040656+0000","last_fullsized":"2023-11-09T03:48:57.040656+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:58:02.392696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064572299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036384+0000","last_change":"2023-11-09T03:47:03.146580+0000","last_active":"2023-11-09T03:48:57.036384+0000","last_peered":"2023-11-09T03:48:57.036384+0000","last_clean":"2023-11-09T03:48:57.036384+0000","last_became_active":"2023-11-09T03:47:01.589201+0000","last_became_peered":"2023-11-09T03:47:01.589201+0000","last_unstale":"2023-11-09T03:48:57.036384+0000","last_undegraded":"2023-11-09T03:48:57.036384+0000","last_fullsized":"2023-11-09T03:48:57.036384+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:14.300421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049706999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.443842+0000","last_change":"2023-11-09T03:48:57.436623+0000","last_active":"2023-11-09T03:48:57.443842+0000","last_peered":"2023-11-09T03:48:57.443842+0000","last_clean":"2023-11-09T03:48:57.443842+0000","last_became_active":"2023-11-09T03:48:57.436370+0000","last_became_peered":"2023-11-09T03:48:57.436370+0000","last_unstale":"2023-11-09T03:48:57.443842+0000","last_undegraded":"2023-11-09T03:48:57.443842+0000","last_fullsized":"2023-11-09T03:48:57.443842+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:56:57.961469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.434494+0000","last_change":"2023-11-09T03:48:57.050983+0000","last_active":"2023-11-09T03:48:57.434494+0000","last_peered":"2023-11-09T03:48:57.434494+0000","last_clean":"2023-11-09T03:48:57.434494+0000","last_became_active":"2023-11-09T03:48:57.050070+0000","last_became_peered":"2023-11-09T03:48:57.050070+0000","last_unstale":"2023-11-09T03:48:57.434494+0000","last_undegraded":"2023-11-09T03:48:57.434494+0000","last_fullsized":"2023-11-09T03:48:57.434494+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:23:59.729104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.035953+0000","last_change":"2023-11-09T03:48:01.451284+0000","last_active":"2023-11-09T03:48:57.035953+0000","last_peered":"2023-11-09T03:48:57.035953+0000","last_clean":"2023-11-09T03:48:57.035953+0000","last_became_active":"2023-11-09T03:48:01.450954+0000","last_became_peered":"2023-11-09T03:48:01.450954+0000","last_unstale":"2023-11-09T03:48:57.035953+0000","last_undegraded":"2023-11-09T03:48:57.035953+0000","last_fullsized":"2023-11-09T03:48:57.035953+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:49:24.680020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.043771+0000","last_change":"2023-11-09T03:48:02.733038+0000","last_active":"2023-11-09T03:48:57.043771+0000","last_peered":"2023-11-09T03:48:57.043771+0000","last_clean":"2023-11-09T03:48:57.043771+0000","last_became_active":"2023-11-09T03:48:00.446661+0000","last_became_peered":"2023-11-09T03:48:00.446661+0000","last_unstale":"2023-11-09T03:48:57.043771+0000","last_undegraded":"2023-11-09T03:48:57.043771+0000","last_fullsized":"2023-11-09T03:48:57.043771+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:05:08.649312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040102300000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.034130+0000","last_change":"2023-11-09T03:47:03.147851+0000","last_active":"2023-11-09T03:48:57.034130+0000","last_peered":"2023-11-09T03:48:57.034130+0000","last_clean":"2023-11-09T03:48:57.034130+0000","last_became_active":"2023-11-09T03:47:01.589893+0000","last_became_peered":"2023-11-09T03:47:01.589893+0000","last_unstale":"2023-11-09T03:48:57.034130+0000","last_undegraded":"2023-11-09T03:48:57.034130+0000","last_fullsized":"2023-11-09T03:48:57.034130+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:08:17.183462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029151399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":226,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.041182+0000","last_change":"2023-11-09T03:45:21.030926+0000","last_active":"2023-11-09T03:48:57.041182+0000","last_peered":"2023-11-09T03:48:57.041182+0000","last_clean":"2023-11-09T03:48:57.041182+0000","last_became_active":"2023-11-09T03:45:21.029801+0000","last_became_peered":"2023-11-09T03:45:21.029801+0000","last_unstale":"2023-11-09T03:48:57.041182+0000","last_undegraded":"2023-11-09T03:48:57.041182+0000","last_fullsized":"2023-11-09T03:48:57.041182+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_clean_scrub_stamp":"2023-11-09T03:45:19.768433+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:12:59.153713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574895,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32872,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338968,"statfs":{"total":96636764160,"available":96603103232,"internally_reserved":0,"allocated":1728512,"data_stored":1145248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.503,"15min":0.499},"min":{"1min":0.36599999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53200000000000003,"15min":0.52100000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.66700000000000004,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.624}]},{"osd":1,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.38600000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70599999999999996,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53000000000000003,"15min":0.51900000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.65600000000000003}]},{"osd":2,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52700000000000002,"15min":0.51500000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72099999999999997,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.755},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.65400000000000003,"15min":0.69699999999999995},"min":{"1min":0.41599999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.73299999999999998,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.72499999999999998}]},{"osd":3,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.52200000000000002,"15min":0.51300000000000001},"min":{"1min":0.42599999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.67300000000000004,"5min":0.754,"15min":0.754},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56399999999999995,"15min":0.54500000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81499999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.70999999999999996}]},{"osd":4,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.45500000000000002,"15min":0.44400000000000001},"min":{"1min":0.32000000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.5,"15min":0.48199999999999998},"min":{"1min":0.32300000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.73199999999999998,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.57699999999999996}]},{"osd":5,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.48499999999999999,"15min":0.46000000000000002},"min":{"1min":0.30599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76300000000000001,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51900000000000002,"15min":0.499},"min":{"1min":0.32700000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.79700000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.55700000000000005}]},{"osd":6,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55700000000000005,"15min":0.54300000000000004},"min":{"1min":0.38900000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.74299999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54000000000000004,"15min":0.52200000000000002},"min":{"1min":0.40400000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.64100000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1159168,"data_stored":570752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.504,"15min":0.503},"min":{"1min":0.34100000000000003,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.65700000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51400000000000001,"15min":0.498},"min":{"1min":0.36699999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73199999999999998,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.755}]},{"osd":1,"last update":"Thu Nov 9 03:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52200000000000002,"15min":0.51200000000000001},"min":{"1min":0.35899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.497,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.39300000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.65300000000000002,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.71499999999999997}]},{"osd":2,"last update":"Thu Nov 9 03:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.42199999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.65400000000000003,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52900000000000003,"15min":0.52100000000000002},"min":{"1min":0.39900000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.70399999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.72699999999999998}]},{"osd":3,"last update":"Thu Nov 9 03:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52500000000000002,"15min":0.52400000000000002},"min":{"1min":0.39100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.42799999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.68400000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.59899999999999998}]},{"osd":4,"last update":"Thu Nov 9 03:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.497,"15min":0.501},"min":{"1min":0.32000000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.66700000000000004,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53200000000000003,"15min":0.52900000000000003},"min":{"1min":0.34599999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.76900000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.44400000000000001}]},{"osd":5,"last update":"Thu Nov 9 03:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57999999999999996,"15min":0.57699999999999996},"min":{"1min":0.39500000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80800000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55000000000000004,"15min":0.55600000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.78800000000000003,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46600000000000003}]},{"osd":7,"last update":"Thu Nov 9 03:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.45200000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83199999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.498},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56699999999999995,"15min":0.55800000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71499999999999997,"5min":0.877,"15min":0.877},"last":0.48599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574895,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1134592,"data_stored":554880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.504,"15min":0.48899999999999999},"min":{"1min":0.36199999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.376,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.51400000000000001}]},{"osd":1,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88300000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.36599999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.84199999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.46800000000000003}]},{"osd":2,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.45800000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.55600000000000005},"min":{"1min":0.40300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.52800000000000002}]},{"osd":3,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57099999999999995,"15min":0.56499999999999995},"min":{"1min":0.39400000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.436},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55200000000000005,"15min":0.55100000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.53900000000000003}]},{"osd":4,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.49199999999999999,"15min":0.48399999999999999},"min":{"1min":0.32400000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.48999999999999999,"15min":0.48299999999999998},"min":{"1min":0.29699999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.32300000000000001}]},{"osd":6,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.50600000000000001,"15min":0.51000000000000001},"min":{"1min":0.33700000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.755,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.53300000000000003,"15min":0.51700000000000002},"min":{"1min":0.38300000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.42099999999999999}]},{"osd":7,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.34699999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.81200000000000006,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.498},{"interface":"front","average":{"1min":0.621,"5min":0.61899999999999999,"15min":0.62},"min":{"1min":0.432,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84899999999999998,"5min":1.01,"15min":1.01},"last":0.45300000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1134592,"data_stored":554880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69399999999999995,"5min":1.034,"15min":1.034},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.38800000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78300000000000003,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.54900000000000004}]},{"osd":1,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.34000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.74299999999999999,"5min":1.093,"15min":1.093},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54200000000000004,"15min":0.54100000000000004},"min":{"1min":0.36399999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71899999999999997,"5min":0.871,"15min":0.871},"last":0.629}]},{"osd":2,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54100000000000004,"15min":0.55400000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.69199999999999995,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.442},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54800000000000004,"15min":0.54500000000000004},"min":{"1min":0.39700000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.53600000000000003}]},{"osd":3,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.433,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.70199999999999996,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.376,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72499999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.57099999999999995}]},{"osd":5,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.33300000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.78700000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.318,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81100000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.41399999999999998}]},{"osd":6,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.75800000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57899999999999996,"15min":0.58399999999999996},"min":{"1min":0.39600000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.75900000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.47799999999999998}]},{"osd":7,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61599999999999999,"15min":0.629},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83499999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61099999999999999,"15min":0.61899999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79300000000000004,"5min":1.113,"15min":1.113},"last":0.51400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1146880,"data_stored":562566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.32100000000000001,"15min":0.32300000000000001},"min":{"1min":0.22,"5min":0.215,"15min":0.215},"max":{"1min":0.40200000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.313,"5min":0.32300000000000001,"15min":0.32400000000000001},"min":{"1min":0.23999999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.47799999999999998,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.26800000000000002}]},{"osd":1,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.32600000000000001,"15min":0.33000000000000002},"min":{"1min":0.23599999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.42899999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.36699999999999999,"15min":0.373},"min":{"1min":0.28100000000000003,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.47899999999999998,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.31}]},{"osd":2,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.37,"15min":0.372},"min":{"1min":0.27200000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.439,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.379,"15min":0.38300000000000001},"min":{"1min":0.29799999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.45700000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.33000000000000002}]},{"osd":4,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58899999999999997,"15min":0.59499999999999997},"min":{"1min":0.42299999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79000000000000004,"5min":0.87,"15min":0.87},"last":0.432},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.59699999999999998,"15min":0.61099999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.54600000000000004}]},{"osd":5,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.60499999999999998,"15min":0.62},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86599999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60699999999999998,"15min":0.61899999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.82099999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.46600000000000003}]},{"osd":6,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60999999999999999,"15min":0.623},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78200000000000003,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.45400000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.80300000000000005,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.61099999999999999,"15min":0.62},"min":{"1min":0.41099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.70899999999999996,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.621,"15min":0.63700000000000001},"min":{"1min":0.46100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81299999999999994,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.52500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28160,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343680,"statfs":{"total":96636764160,"available":96607928320,"internally_reserved":0,"allocated":1159168,"data_stored":570752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:25 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32900000000000001,"15min":0.33100000000000002},"min":{"1min":0.20999999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.42599999999999999,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.255},{"interface":"front","average":{"1min":0.31,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.23300000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.51400000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.35699999999999998}]},{"osd":1,"last update":"Thu Nov 9 03:48:25 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.32200000000000001,"15min":0.32500000000000001},"min":{"1min":0.249,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.46600000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.36399999999999999,"15min":0.36499999999999999},"min":{"1min":0.252,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.372}]},{"osd":3,"last update":"Thu Nov 9 03:48:25 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.35699999999999998,"15min":0.36299999999999999},"min":{"1min":0.25,"5min":0.22,"15min":0.22},"max":{"1min":0.54100000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.39600000000000002,"15min":0.40200000000000002},"min":{"1min":0.30299999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.495,"5min":0.754,"15min":0.754},"last":0.33900000000000002}]},{"osd":4,"last update":"Thu Nov 9 03:48:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.44800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.501}]},{"osd":5,"last update":"Thu Nov 9 03:48:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56599999999999995,"15min":0.57399999999999995},"min":{"1min":0.38,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.69799999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.434,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.49099999999999999}]},{"osd":6,"last update":"Thu Nov 9 03:48:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60599999999999998,"15min":0.60999999999999999},"min":{"1min":0.438,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59099999999999997,"15min":0.59199999999999997},"min":{"1min":0.436,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.51200000000000001}]},{"osd":7,"last update":"Thu Nov 9 03:48:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.61499999999999999,"15min":0.623},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.58699999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28128,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343712,"statfs":{"total":96636764160,"available":96607961088,"internally_reserved":0,"allocated":1134592,"data_stored":554880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.313,"15min":0.315},"min":{"1min":0.221,"5min":0.19,"15min":0.19},"max":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.29899999999999999,"5min":0.313,"15min":0.311},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.35899999999999999}]},{"osd":2,"last update":"Thu Nov 9 03:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.314,"15min":0.31},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.34399999999999997,"15min":0.34200000000000003},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.53300000000000003,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.38500000000000001}]},{"osd":3,"last update":"Thu Nov 9 03:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36699999999999999,"15min":0.371},"min":{"1min":0.26000000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.36399999999999999,"15min":0.36899999999999999},"min":{"1min":0.27100000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Nov 9 03:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.55400000000000005},"min":{"1min":0.40799999999999997,"5min":0.37,"15min":0.37},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56200000000000006,"15min":0.56100000000000005},"min":{"1min":0.41499999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.64700000000000002}]},{"osd":5,"last update":"Thu Nov 9 03:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.44400000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80500000000000005,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.55600000000000005}]},{"osd":6,"last update":"Thu Nov 9 03:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.41199999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.61699999999999999}]},{"osd":7,"last update":"Thu Nov 9 03:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59099999999999997,"15min":0.60399999999999998},"min":{"1min":0.41399999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.83099999999999996,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.59799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1736704,"data_stored":1152315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32000000000000001,"15min":0.32400000000000001},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.34599999999999997,"15min":0.36199999999999999},"min":{"1min":0.253,"5min":0.219,"15min":0.219},"max":{"1min":0.58299999999999996,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.435}]},{"osd":2,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35799999999999998,"15min":0.36399999999999999},"min":{"1min":0.24299999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.35999999999999999,"15min":0.36899999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.40799999999999997}]},{"osd":3,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.379,"15min":0.38900000000000001},"min":{"1min":0.27300000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.40699999999999997,"15min":0.41799999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.56599999999999995,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.49299999999999999}]},{"osd":4,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58999999999999997,"15min":0.59399999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.71899999999999997}]},{"osd":5,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.59199999999999997},"min":{"1min":0.378,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61499999999999999,"15min":0.63100000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80800000000000005,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.77600000000000002}]},{"osd":6,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63400000000000001,"15min":0.64500000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88300000000000001,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.624,"15min":0.63100000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85799999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.68799999999999994}]},{"osd":7,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61299999999999999,"15min":0.621},"min":{"1min":0.39800000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.42599999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.78800000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.75700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-09T03:49:01.000 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-09T03:49:01.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.132+0000 7fc2e6090640 1 Processor -- start 2023-11-09T03:49:01.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.133+0000 7fc2e6090640 1 -- start start 2023-11-09T03:49:01.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.133+0000 7fc2e6090640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc2e00cde60 0x7fc2e015d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:01.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.133+0000 7fc2e6090640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc2e015daa0 0x7fc2e0157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:01.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.133+0000 7fc2e6090640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2e015e000 con 0x7fc2e00ce240 2023-11-09T03:49:01.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.133+0000 7fc2e6090640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e015e5c0 con 0x7fc2e015daa0 2023-11-09T03:49:01.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.133+0000 7fc2e6090640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e015e2e0 con 0x7fc2e00cde60 2023-11-09T03:49:01.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.133+0000 7fc2deffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc2e015daa0 0x7fc2e0157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:01.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.133+0000 7fc2dffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc2e00ce240 0x7fc2e006b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35914/0 (socket says 172.21.15.76:35914) 2023-11-09T03:49:01.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.133+0000 7fc2dffff640 1 -- 172.21.15.76:0/1022733086 learned_addr learned my addr 172.21.15.76:0/1022733086 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:49:01.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.134+0000 7fc2df7fe640 1 --2- 172.21.15.76:0/1022733086 >> v2:172.21.15.113:3300/0 conn(0x7fc2e00cde60 0x7fc2e015d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:01.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.134+0000 7fc2deffd640 1 -- 172.21.15.76:0/1022733086 >> v2:172.21.15.113:3300/0 conn(0x7fc2e00cde60 msgr2=0x7fc2e015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:01.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.134+0000 7fc2deffd640 1 --2- 172.21.15.76:0/1022733086 >> v2:172.21.15.113:3300/0 conn(0x7fc2e00cde60 0x7fc2e015d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:01.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.134+0000 7fc2deffd640 1 -- 172.21.15.76:0/1022733086 >> v1:172.21.15.76:6789/0 conn(0x7fc2e00ce240 legacy=0x7fc2e006b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:01.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.134+0000 7fc2deffd640 1 -- 172.21.15.76:0/1022733086 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e00d1120 con 0x7fc2e015daa0 2023-11-09T03:49:01.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.134+0000 7fc2de7fc640 1 -- 172.21.15.76:0/1022733086 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 207106272 0 0) 0x7fc2e015e000 con 0x7fc2e00ce240 2023-11-09T03:49:01.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.134+0000 7fc2deffd640 1 --2- 172.21.15.76:0/1022733086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2e015daa0 0x7fc2e0157000 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00b5550 tx=0x7fc2d402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144f46366fd885bd server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:01.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.135+0000 7fc2de7fc640 1 -- 172.21.15.76:0/1022733086 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4031830 con 0x7fc2e015daa0 2023-11-09T03:49:01.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.135+0000 7fc2de7fc640 1 -- 172.21.15.76:0/1022733086 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d4031990 con 0x7fc2e015daa0 2023-11-09T03:49:01.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.135+0000 7fc2de7fc640 1 -- 172.21.15.76:0/1022733086 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4031d20 con 0x7fc2e015daa0 2023-11-09T03:49:01.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.135+0000 7fc2e6090640 1 -- 172.21.15.76:0/1022733086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2e015daa0 msgr2=0x7fc2e0157000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:01.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.135+0000 7fc2e6090640 1 --2- 172.21.15.76:0/1022733086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2e015daa0 0x7fc2e0157000 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00b5550 tx=0x7fc2d402b8d0 comp rx=0 tx=0).stop 2023-11-09T03:49:01.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.135+0000 7fc2e6090640 1 -- 172.21.15.76:0/1022733086 shutdown_connections 2023-11-09T03:49:01.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.135+0000 7fc2e6090640 1 --2- 172.21.15.76:0/1022733086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2e015daa0 0x7fc2e0157000 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:01.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.135+0000 7fc2e6090640 1 --2- 172.21.15.76:0/1022733086 >> v2:172.21.15.113:3300/0 conn(0x7fc2e00cde60 0x7fc2e015d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:01.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.135+0000 7fc2e6090640 1 -- 172.21.15.76:0/1022733086 >> 172.21.15.76:0/1022733086 conn(0x7fc2e0071f30 msgr2=0x7fc2e00cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:01.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.135+0000 7fc2e6090640 1 -- 172.21.15.76:0/1022733086 shutdown_connections 2023-11-09T03:49:01.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.135+0000 7fc2e6090640 1 -- 172.21.15.76:0/1022733086 wait complete. 2023-11-09T03:49:01.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.137+0000 7fc2e6090640 1 Processor -- start 2023-11-09T03:49:01.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.137+0000 7fc2e6090640 1 -- start start 2023-11-09T03:49:01.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.137+0000 7fc2e6090640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2e00cde60 0x7fc2e014d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:01.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.137+0000 7fc2e6090640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2e00ce240 0x7fc2e00c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:01.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.137+0000 7fc2e6090640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2e015daa0 0x7fc2e00ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:01.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.137+0000 7fc2e6090640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e0153b30 con 0x7fc2e015daa0 2023-11-09T03:49:01.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.137+0000 7fc2e6090640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e00caa40 con 0x7fc2e00cde60 2023-11-09T03:49:01.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.137+0000 7fc2e6090640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e00cad20 con 0x7fc2e00ce240 2023-11-09T03:49:01.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.138+0000 7fc2dffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2e015daa0 0x7fc2e00ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:01.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.138+0000 7fc2dffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2e015daa0 0x7fc2e00ca430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44202/0 (socket says 172.21.15.76:44202) 2023-11-09T03:49:01.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.138+0000 7fc2dffff640 1 -- 172.21.15.76:0/3650152390 learned_addr learned my addr 172.21.15.76:0/3650152390 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:01.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.138+0000 7fc2df7fe640 1 --2- 172.21.15.76:0/3650152390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2e00cde60 0x7fc2e014d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:01.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.138+0000 7fc2deffd640 1 --2- 172.21.15.76:0/3650152390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2e00ce240 0x7fc2e00c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:01.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.138+0000 7fc2dffff640 1 -- 172.21.15.76:0/3650152390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2e00ce240 msgr2=0x7fc2e00c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:01.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.138+0000 7fc2dffff640 1 --2- 172.21.15.76:0/3650152390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2e00ce240 0x7fc2e00c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:01.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.138+0000 7fc2dffff640 1 -- 172.21.15.76:0/3650152390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2e00cde60 msgr2=0x7fc2e014d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:01.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.138+0000 7fc2dffff640 1 --2- 172.21.15.76:0/3650152390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2e00cde60 0x7fc2e014d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:01.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.138+0000 7fc2dffff640 1 -- 172.21.15.76:0/3650152390 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d4009400 con 0x7fc2e015daa0 2023-11-09T03:49:01.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.138+0000 7fc2dffff640 1 --2- 172.21.15.76:0/3650152390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2e015daa0 0x7fc2e00ca430 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8001560 tx=0x7fc2d800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:01.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.138+0000 7fc2c7fff640 1 -- 172.21.15.76:0/3650152390 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d800a640 con 0x7fc2e015daa0 2023-11-09T03:49:01.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.138+0000 7fc2c7fff640 1 -- 172.21.15.76:0/3650152390 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d800a7a0 con 0x7fc2e015daa0 2023-11-09T03:49:01.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.138+0000 7fc2c7fff640 1 -- 172.21.15.76:0/3650152390 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d800ab30 con 0x7fc2e015daa0 2023-11-09T03:49:01.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.138+0000 7fc2e6090640 1 -- 172.21.15.76:0/3650152390 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e00cb000 con 0x7fc2e015daa0 2023-11-09T03:49:01.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.138+0000 7fc2e6090640 1 -- 172.21.15.76:0/3650152390 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e00cb2e0 con 0x7fc2e015daa0 2023-11-09T03:49:01.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.139+0000 7fc2e6090640 1 -- 172.21.15.76:0/3650152390 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2e0153c70 con 0x7fc2e015daa0 2023-11-09T03:49:01.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.140+0000 7fc2c7fff640 1 -- 172.21.15.76:0/3650152390 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc2d800ac90 con 0x7fc2e015daa0 2023-11-09T03:49:01.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.140+0000 7fc2c7fff640 1 --2- 172.21.15.76:0/3650152390 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc2cc06ae70 0x7fc2cc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:01.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.140+0000 7fc2c7fff640 1 -- 172.21.15.76:0/3650152390 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7120+0+0 (secure 0 0 0) 0x7fc2d8088ff0 con 0x7fc2e015daa0 2023-11-09T03:49:01.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.140+0000 7fc2df7fe640 1 --2- 172.21.15.76:0/3650152390 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc2cc06ae70 0x7fc2cc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:01.147 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:49:01.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.141+0000 7fc2df7fe640 1 --2- 172.21.15.76:0/3650152390 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc2cc06ae70 0x7fc2cc06d330 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0006fd0 tx=0x7fc2d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:01.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.144+0000 7fc2c7fff640 1 -- 172.21.15.76:0/3650152390 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc2e0153c70 con 0x7fc2e015daa0 2023-11-09T03:49:01.149 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:49:01.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.312+0000 7fc2e6090640 1 -- 172.21.15.76:0/3650152390 --> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc2e00d1400 con 0x7fc2cc06ae70 2023-11-09T03:49:01.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.325+0000 7fc2c7fff640 1 -- 172.21.15.76:0/3650152390 <== mgr.4105 v2:172.21.15.113:6832/1264193042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+531943 (secure 0 0 0) 0x7fc2d0010990 con 0x7fc2cc06ae70 2023-11-09T03:49:01.330 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:49:01.333 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2023-11-09T03:49:01.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.329+0000 7fc2e6090640 1 -- 172.21.15.76:0/3650152390 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc2cc06ae70 msgr2=0x7fc2cc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:01.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.329+0000 7fc2e6090640 1 --2- 172.21.15.76:0/3650152390 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc2cc06ae70 0x7fc2cc06d330 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0006fd0 tx=0x7fc2d0008040 comp rx=0 tx=0).stop 2023-11-09T03:49:01.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.329+0000 7fc2e6090640 1 -- 172.21.15.76:0/3650152390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2e015daa0 msgr2=0x7fc2e00ca430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:01.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.329+0000 7fc2e6090640 1 --2- 172.21.15.76:0/3650152390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2e015daa0 0x7fc2e00ca430 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8001560 tx=0x7fc2d800cc70 comp rx=0 tx=0).stop 2023-11-09T03:49:01.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.329+0000 7fc2e6090640 1 -- 172.21.15.76:0/3650152390 shutdown_connections 2023-11-09T03:49:01.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.329+0000 7fc2e6090640 1 --2- 172.21.15.76:0/3650152390 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc2cc06ae70 0x7fc2cc06d330 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:01.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.329+0000 7fc2e6090640 1 --2- 172.21.15.76:0/3650152390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2e015daa0 0x7fc2e00ca430 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:01.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.329+0000 7fc2e6090640 1 --2- 172.21.15.76:0/3650152390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2e00ce240 0x7fc2e00c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:01.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.329+0000 7fc2e6090640 1 --2- 172.21.15.76:0/3650152390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2e00cde60 0x7fc2e014d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:01.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.329+0000 7fc2e6090640 1 -- 172.21.15.76:0/3650152390 >> 172.21.15.76:0/3650152390 conn(0x7fc2e0071f30 msgr2=0x7fc2e00c90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:01.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.329+0000 7fc2e6090640 1 -- 172.21.15.76:0/3650152390 shutdown_connections 2023-11-09T03:49:01.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.329+0000 7fc2e6090640 1 -- 172.21.15.76:0/3650152390 wait complete. 2023-11-09T03:49:01.340 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":200,"stamp":"2023-11-09T03:49:00.804277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":268,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":251236,"kb_used_data":10092,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754723484,"statfs":{"total":773094113280,"available":772836847616,"internally_reserved":0,"allocated":10334208,"data_stored":5666273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.774536"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.447264+0000","last_change":"2023-11-09T03:48:57.445674+0000","last_active":"2023-11-09T03:48:57.447264+0000","last_peered":"2023-11-09T03:48:57.447264+0000","last_clean":"2023-11-09T03:48:57.447264+0000","last_became_active":"2023-11-09T03:48:57.445271+0000","last_became_peered":"2023-11-09T03:48:57.445271+0000","last_unstale":"2023-11-09T03:48:57.447264+0000","last_undegraded":"2023-11-09T03:48:57.447264+0000","last_fullsized":"2023-11-09T03:48:57.447264+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:28.967017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.441747+0000","last_change":"2023-11-09T03:48:57.439578+0000","last_active":"2023-11-09T03:48:57.441747+0000","last_peered":"2023-11-09T03:48:57.441747+0000","last_clean":"2023-11-09T03:48:57.441747+0000","last_became_active":"2023-11-09T03:48:57.438979+0000","last_became_peered":"2023-11-09T03:48:57.438979+0000","last_unstale":"2023-11-09T03:48:57.441747+0000","last_undegraded":"2023-11-09T03:48:57.441747+0000","last_fullsized":"2023-11-09T03:48:57.441747+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:21:31.419099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.453559+0000","last_change":"2023-11-09T03:48:57.447498+0000","last_active":"2023-11-09T03:48:57.453559+0000","last_peered":"2023-11-09T03:48:57.453559+0000","last_clean":"2023-11-09T03:48:57.453559+0000","last_became_active":"2023-11-09T03:48:57.447223+0000","last_became_peered":"2023-11-09T03:48:57.447223+0000","last_unstale":"2023-11-09T03:48:57.453559+0000","last_undegraded":"2023-11-09T03:48:57.453559+0000","last_fullsized":"2023-11-09T03:48:57.453559+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:02.054568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.437439+0000","last_change":"2023-11-09T03:48:57.434039+0000","last_active":"2023-11-09T03:48:57.437439+0000","last_peered":"2023-11-09T03:48:57.437439+0000","last_clean":"2023-11-09T03:48:57.437439+0000","last_became_active":"2023-11-09T03:48:57.433770+0000","last_became_peered":"2023-11-09T03:48:57.433770+0000","last_unstale":"2023-11-09T03:48:57.437439+0000","last_undegraded":"2023-11-09T03:48:57.437439+0000","last_fullsized":"2023-11-09T03:48:57.437439+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:51:42.409861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.444702+0000","last_change":"2023-11-09T03:48:57.443545+0000","last_active":"2023-11-09T03:48:57.444702+0000","last_peered":"2023-11-09T03:48:57.444702+0000","last_clean":"2023-11-09T03:48:57.444702+0000","last_became_active":"2023-11-09T03:48:57.443290+0000","last_became_peered":"2023-11-09T03:48:57.443290+0000","last_unstale":"2023-11-09T03:48:57.444702+0000","last_undegraded":"2023-11-09T03:48:57.444702+0000","last_fullsized":"2023-11-09T03:48:57.444702+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:00:56.307904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.446704+0000","last_change":"2023-11-09T03:48:57.442256+0000","last_active":"2023-11-09T03:48:57.446704+0000","last_peered":"2023-11-09T03:48:57.446704+0000","last_clean":"2023-11-09T03:48:57.446704+0000","last_became_active":"2023-11-09T03:48:57.442016+0000","last_became_peered":"2023-11-09T03:48:57.442016+0000","last_unstale":"2023-11-09T03:48:57.446704+0000","last_undegraded":"2023-11-09T03:48:57.446704+0000","last_fullsized":"2023-11-09T03:48:57.446704+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:08.671141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.448714+0000","last_change":"2023-11-09T03:48:57.445253+0000","last_active":"2023-11-09T03:48:57.448714+0000","last_peered":"2023-11-09T03:48:57.448714+0000","last_clean":"2023-11-09T03:48:57.448714+0000","last_became_active":"2023-11-09T03:48:57.444960+0000","last_became_peered":"2023-11-09T03:48:57.444960+0000","last_unstale":"2023-11-09T03:48:57.448714+0000","last_undegraded":"2023-11-09T03:48:57.448714+0000","last_fullsized":"2023-11-09T03:48:57.448714+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:46:17.736233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.451322+0000","last_change":"2023-11-09T03:48:57.446239+0000","last_active":"2023-11-09T03:48:57.451322+0000","last_peered":"2023-11-09T03:48:57.451322+0000","last_clean":"2023-11-09T03:48:57.451322+0000","last_became_active":"2023-11-09T03:48:57.445925+0000","last_became_peered":"2023-11-09T03:48:57.445925+0000","last_unstale":"2023-11-09T03:48:57.451322+0000","last_undegraded":"2023-11-09T03:48:57.451322+0000","last_fullsized":"2023-11-09T03:48:57.451322+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:56:57.961469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.437491+0000","last_change":"2023-11-09T03:48:57.433879+0000","last_active":"2023-11-09T03:48:57.437491+0000","last_peered":"2023-11-09T03:48:57.437491+0000","last_clean":"2023-11-09T03:48:57.437491+0000","last_became_active":"2023-11-09T03:48:57.433017+0000","last_became_peered":"2023-11-09T03:48:57.433017+0000","last_unstale":"2023-11-09T03:48:57.437491+0000","last_undegraded":"2023-11-09T03:48:57.437491+0000","last_fullsized":"2023-11-09T03:48:57.437491+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:15:04.864360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.440254+0000","last_change":"2023-11-09T03:48:57.438289+0000","last_active":"2023-11-09T03:48:57.440254+0000","last_peered":"2023-11-09T03:48:57.440254+0000","last_clean":"2023-11-09T03:48:57.440254+0000","last_became_active":"2023-11-09T03:48:57.438043+0000","last_became_peered":"2023-11-09T03:48:57.438043+0000","last_unstale":"2023-11-09T03:48:57.440254+0000","last_undegraded":"2023-11-09T03:48:57.440254+0000","last_fullsized":"2023-11-09T03:48:57.440254+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:13:49.228169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.444154+0000","last_change":"2023-11-09T03:48:57.441291+0000","last_active":"2023-11-09T03:48:57.444154+0000","last_peered":"2023-11-09T03:48:57.444154+0000","last_clean":"2023-11-09T03:48:57.444154+0000","last_became_active":"2023-11-09T03:48:57.441042+0000","last_became_peered":"2023-11-09T03:48:57.441042+0000","last_unstale":"2023-11-09T03:48:57.444154+0000","last_undegraded":"2023-11-09T03:48:57.444154+0000","last_fullsized":"2023-11-09T03:48:57.444154+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:04:31.603691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.441376+0000","last_change":"2023-11-09T03:48:57.438303+0000","last_active":"2023-11-09T03:48:57.441376+0000","last_peered":"2023-11-09T03:48:57.441376+0000","last_clean":"2023-11-09T03:48:57.441376+0000","last_became_active":"2023-11-09T03:48:57.438058+0000","last_became_peered":"2023-11-09T03:48:57.438058+0000","last_unstale":"2023-11-09T03:48:57.441376+0000","last_undegraded":"2023-11-09T03:48:57.441376+0000","last_fullsized":"2023-11-09T03:48:57.441376+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:58:45.939038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.447198+0000","last_change":"2023-11-09T03:48:57.439889+0000","last_active":"2023-11-09T03:48:57.447198+0000","last_peered":"2023-11-09T03:48:57.447198+0000","last_clean":"2023-11-09T03:48:57.447198+0000","last_became_active":"2023-11-09T03:48:57.439646+0000","last_became_peered":"2023-11-09T03:48:57.439646+0000","last_unstale":"2023-11-09T03:48:57.447198+0000","last_undegraded":"2023-11-09T03:48:57.447198+0000","last_fullsized":"2023-11-09T03:48:57.447198+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:55:52.899135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.451338+0000","last_change":"2023-11-09T03:48:57.446694+0000","last_active":"2023-11-09T03:48:57.451338+0000","last_peered":"2023-11-09T03:48:57.451338+0000","last_clean":"2023-11-09T03:48:57.451338+0000","last_became_active":"2023-11-09T03:48:57.446361+0000","last_became_peered":"2023-11-09T03:48:57.446361+0000","last_unstale":"2023-11-09T03:48:57.451338+0000","last_undegraded":"2023-11-09T03:48:57.451338+0000","last_fullsized":"2023-11-09T03:48:57.451338+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:03.847302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.449453+0000","last_change":"2023-11-09T03:48:57.444296+0000","last_active":"2023-11-09T03:48:57.449453+0000","last_peered":"2023-11-09T03:48:57.449453+0000","last_clean":"2023-11-09T03:48:57.449453+0000","last_became_active":"2023-11-09T03:48:57.444032+0000","last_became_peered":"2023-11-09T03:48:57.444032+0000","last_unstale":"2023-11-09T03:48:57.449453+0000","last_undegraded":"2023-11-09T03:48:57.449453+0000","last_fullsized":"2023-11-09T03:48:57.449453+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:18:20.971683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.444183+0000","last_change":"2023-11-09T03:48:57.440405+0000","last_active":"2023-11-09T03:48:57.444183+0000","last_peered":"2023-11-09T03:48:57.444183+0000","last_clean":"2023-11-09T03:48:57.444183+0000","last_became_active":"2023-11-09T03:48:57.440158+0000","last_became_peered":"2023-11-09T03:48:57.440158+0000","last_unstale":"2023-11-09T03:48:57.444183+0000","last_undegraded":"2023-11-09T03:48:57.444183+0000","last_fullsized":"2023-11-09T03:48:57.444183+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:09:28.936062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.448792+0000","last_change":"2023-11-09T03:48:57.446115+0000","last_active":"2023-11-09T03:48:57.448792+0000","last_peered":"2023-11-09T03:48:57.448792+0000","last_clean":"2023-11-09T03:48:57.448792+0000","last_became_active":"2023-11-09T03:48:57.445824+0000","last_became_peered":"2023-11-09T03:48:57.445824+0000","last_unstale":"2023-11-09T03:48:57.448792+0000","last_undegraded":"2023-11-09T03:48:57.448792+0000","last_fullsized":"2023-11-09T03:48:57.448792+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:01:16.444216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.444553+0000","last_change":"2023-11-09T03:48:57.439366+0000","last_active":"2023-11-09T03:48:57.444553+0000","last_peered":"2023-11-09T03:48:57.444553+0000","last_clean":"2023-11-09T03:48:57.444553+0000","last_became_active":"2023-11-09T03:48:57.439114+0000","last_became_peered":"2023-11-09T03:48:57.439114+0000","last_unstale":"2023-11-09T03:48:57.444553+0000","last_undegraded":"2023-11-09T03:48:57.444553+0000","last_fullsized":"2023-11-09T03:48:57.444553+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:01:16.444216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.447696+0000","last_change":"2023-11-09T03:48:57.441391+0000","last_active":"2023-11-09T03:48:57.447696+0000","last_peered":"2023-11-09T03:48:57.447696+0000","last_clean":"2023-11-09T03:48:57.447696+0000","last_became_active":"2023-11-09T03:48:57.441094+0000","last_became_peered":"2023-11-09T03:48:57.441094+0000","last_unstale":"2023-11-09T03:48:57.447696+0000","last_undegraded":"2023-11-09T03:48:57.447696+0000","last_fullsized":"2023-11-09T03:48:57.447696+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:20:58.173238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.440932+0000","last_change":"2023-11-09T03:48:57.438369+0000","last_active":"2023-11-09T03:48:57.440932+0000","last_peered":"2023-11-09T03:48:57.440932+0000","last_clean":"2023-11-09T03:48:57.440932+0000","last_became_active":"2023-11-09T03:48:57.438089+0000","last_became_peered":"2023-11-09T03:48:57.438089+0000","last_unstale":"2023-11-09T03:48:57.440932+0000","last_undegraded":"2023-11-09T03:48:57.440932+0000","last_fullsized":"2023-11-09T03:48:57.440932+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:49:52.205154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.436973+0000","last_change":"2023-11-09T03:48:57.432344+0000","last_active":"2023-11-09T03:48:57.436973+0000","last_peered":"2023-11-09T03:48:57.436973+0000","last_clean":"2023-11-09T03:48:57.436973+0000","last_became_active":"2023-11-09T03:48:57.432077+0000","last_became_peered":"2023-11-09T03:48:57.432077+0000","last_unstale":"2023-11-09T03:48:57.436973+0000","last_undegraded":"2023-11-09T03:48:57.436973+0000","last_fullsized":"2023-11-09T03:48:57.436973+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:51:18.482982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.443737+0000","last_change":"2023-11-09T03:48:57.438613+0000","last_active":"2023-11-09T03:48:57.443737+0000","last_peered":"2023-11-09T03:48:57.443737+0000","last_clean":"2023-11-09T03:48:57.443737+0000","last_became_active":"2023-11-09T03:48:57.438367+0000","last_became_peered":"2023-11-09T03:48:57.438367+0000","last_unstale":"2023-11-09T03:48:57.443737+0000","last_undegraded":"2023-11-09T03:48:57.443737+0000","last_fullsized":"2023-11-09T03:48:57.443737+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:02.054568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.439194+0000","last_change":"2023-11-09T03:48:57.436456+0000","last_active":"2023-11-09T03:48:57.439194+0000","last_peered":"2023-11-09T03:48:57.439194+0000","last_clean":"2023-11-09T03:48:57.439194+0000","last_became_active":"2023-11-09T03:48:57.436172+0000","last_became_peered":"2023-11-09T03:48:57.436172+0000","last_unstale":"2023-11-09T03:48:57.439194+0000","last_undegraded":"2023-11-09T03:48:57.439194+0000","last_fullsized":"2023-11-09T03:48:57.439194+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:28:01.062126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.443860+0000","last_change":"2023-11-09T03:48:57.440743+0000","last_active":"2023-11-09T03:48:57.443860+0000","last_peered":"2023-11-09T03:48:57.443860+0000","last_clean":"2023-11-09T03:48:57.443860+0000","last_became_active":"2023-11-09T03:48:57.440480+0000","last_became_peered":"2023-11-09T03:48:57.440480+0000","last_unstale":"2023-11-09T03:48:57.443860+0000","last_undegraded":"2023-11-09T03:48:57.443860+0000","last_fullsized":"2023-11-09T03:48:57.443860+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:00:55.763662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.446738+0000","last_change":"2023-11-09T03:48:57.442454+0000","last_active":"2023-11-09T03:48:57.446738+0000","last_peered":"2023-11-09T03:48:57.446738+0000","last_clean":"2023-11-09T03:48:57.446738+0000","last_became_active":"2023-11-09T03:48:57.442179+0000","last_became_peered":"2023-11-09T03:48:57.442179+0000","last_unstale":"2023-11-09T03:48:57.446738+0000","last_undegraded":"2023-11-09T03:48:57.446738+0000","last_fullsized":"2023-11-09T03:48:57.446738+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:59:11.986476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.444226+0000","last_change":"2023-11-09T03:48:57.441769+0000","last_active":"2023-11-09T03:48:57.444226+0000","last_peered":"2023-11-09T03:48:57.444226+0000","last_clean":"2023-11-09T03:48:57.444226+0000","last_became_active":"2023-11-09T03:48:57.441539+0000","last_became_peered":"2023-11-09T03:48:57.441539+0000","last_unstale":"2023-11-09T03:48:57.444226+0000","last_undegraded":"2023-11-09T03:48:57.444226+0000","last_fullsized":"2023-11-09T03:48:57.444226+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:03:06.129711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.442032+0000","last_change":"2023-11-09T03:48:57.439869+0000","last_active":"2023-11-09T03:48:57.442032+0000","last_peered":"2023-11-09T03:48:57.442032+0000","last_clean":"2023-11-09T03:48:57.442032+0000","last_became_active":"2023-11-09T03:48:57.439618+0000","last_became_peered":"2023-11-09T03:48:57.439618+0000","last_unstale":"2023-11-09T03:48:57.442032+0000","last_undegraded":"2023-11-09T03:48:57.442032+0000","last_fullsized":"2023-11-09T03:48:57.442032+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:33:11.940885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.445044+0000","last_change":"2023-11-09T03:48:57.443612+0000","last_active":"2023-11-09T03:48:57.445044+0000","last_peered":"2023-11-09T03:48:57.445044+0000","last_clean":"2023-11-09T03:48:57.445044+0000","last_became_active":"2023-11-09T03:48:57.443351+0000","last_became_peered":"2023-11-09T03:48:57.443351+0000","last_unstale":"2023-11-09T03:48:57.445044+0000","last_undegraded":"2023-11-09T03:48:57.445044+0000","last_fullsized":"2023-11-09T03:48:57.445044+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:25:37.963645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.439477+0000","last_change":"2023-11-09T03:48:57.437084+0000","last_active":"2023-11-09T03:48:57.439477+0000","last_peered":"2023-11-09T03:48:57.439477+0000","last_clean":"2023-11-09T03:48:57.439477+0000","last_became_active":"2023-11-09T03:48:57.436804+0000","last_became_peered":"2023-11-09T03:48:57.436804+0000","last_unstale":"2023-11-09T03:48:57.439477+0000","last_undegraded":"2023-11-09T03:48:57.439477+0000","last_fullsized":"2023-11-09T03:48:57.439477+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:25:46.922398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.446598+0000","last_change":"2023-11-09T03:48:57.443908+0000","last_active":"2023-11-09T03:48:57.446598+0000","last_peered":"2023-11-09T03:48:57.446598+0000","last_clean":"2023-11-09T03:48:57.446598+0000","last_became_active":"2023-11-09T03:48:57.443414+0000","last_became_peered":"2023-11-09T03:48:57.443414+0000","last_unstale":"2023-11-09T03:48:57.446598+0000","last_undegraded":"2023-11-09T03:48:57.446598+0000","last_fullsized":"2023-11-09T03:48:57.446598+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:29:29.623406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.444154+0000","last_change":"2023-11-09T03:48:57.442341+0000","last_active":"2023-11-09T03:48:57.444154+0000","last_peered":"2023-11-09T03:48:57.444154+0000","last_clean":"2023-11-09T03:48:57.444154+0000","last_became_active":"2023-11-09T03:48:57.442083+0000","last_became_peered":"2023-11-09T03:48:57.442083+0000","last_unstale":"2023-11-09T03:48:57.444154+0000","last_undegraded":"2023-11-09T03:48:57.444154+0000","last_fullsized":"2023-11-09T03:48:57.444154+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:38:27.818497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.446730+0000","last_change":"2023-11-09T03:48:57.442871+0000","last_active":"2023-11-09T03:48:57.446730+0000","last_peered":"2023-11-09T03:48:57.446730+0000","last_clean":"2023-11-09T03:48:57.446730+0000","last_became_active":"2023-11-09T03:48:57.442618+0000","last_became_peered":"2023-11-09T03:48:57.442618+0000","last_unstale":"2023-11-09T03:48:57.446730+0000","last_undegraded":"2023-11-09T03:48:57.446730+0000","last_fullsized":"2023-11-09T03:48:57.446730+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:54:30.889690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.442426+0000","last_change":"2023-11-09T03:48:57.439545+0000","last_active":"2023-11-09T03:48:57.442426+0000","last_peered":"2023-11-09T03:48:57.442426+0000","last_clean":"2023-11-09T03:48:57.442426+0000","last_became_active":"2023-11-09T03:48:57.438639+0000","last_became_peered":"2023-11-09T03:48:57.438639+0000","last_unstale":"2023-11-09T03:48:57.442426+0000","last_undegraded":"2023-11-09T03:48:57.442426+0000","last_fullsized":"2023-11-09T03:48:57.442426+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:21:56.045654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.435259+0000","last_change":"2023-11-09T03:48:57.051318+0000","last_active":"2023-11-09T03:48:57.435259+0000","last_peered":"2023-11-09T03:48:57.435259+0000","last_clean":"2023-11-09T03:48:57.435259+0000","last_became_active":"2023-11-09T03:48:57.051078+0000","last_became_peered":"2023-11-09T03:48:57.051078+0000","last_unstale":"2023-11-09T03:48:57.435259+0000","last_undegraded":"2023-11-09T03:48:57.435259+0000","last_fullsized":"2023-11-09T03:48:57.435259+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:55.963904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.042938+0000","last_change":"2023-11-09T03:48:01.449077+0000","last_active":"2023-11-09T03:48:57.042938+0000","last_peered":"2023-11-09T03:48:57.042938+0000","last_clean":"2023-11-09T03:48:57.042938+0000","last_became_active":"2023-11-09T03:48:01.448806+0000","last_became_peered":"2023-11-09T03:48:01.448806+0000","last_unstale":"2023-11-09T03:48:57.042938+0000","last_undegraded":"2023-11-09T03:48:57.042938+0000","last_fullsized":"2023-11-09T03:48:57.042938+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:16:00.220961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.047787+0000","last_change":"2023-11-09T03:48:02.611756+0000","last_active":"2023-11-09T03:48:57.047787+0000","last_peered":"2023-11-09T03:48:57.047787+0000","last_clean":"2023-11-09T03:48:57.047787+0000","last_became_active":"2023-11-09T03:48:00.445880+0000","last_became_peered":"2023-11-09T03:48:00.445880+0000","last_unstale":"2023-11-09T03:48:57.047787+0000","last_undegraded":"2023-11-09T03:48:57.047787+0000","last_fullsized":"2023-11-09T03:48:57.047787+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:15:35.281296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085514900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.037336+0000","last_change":"2023-11-09T03:47:02.581917+0000","last_active":"2023-11-09T03:48:57.037336+0000","last_peered":"2023-11-09T03:48:57.037336+0000","last_clean":"2023-11-09T03:48:57.037336+0000","last_became_active":"2023-11-09T03:47:01.588082+0000","last_became_peered":"2023-11-09T03:47:01.588082+0000","last_unstale":"2023-11-09T03:48:57.037336+0000","last_undegraded":"2023-11-09T03:48:57.037336+0000","last_fullsized":"2023-11-09T03:48:57.037336+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:44:07.653557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034538899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.447275+0000","last_change":"2023-11-09T03:48:57.441364+0000","last_active":"2023-11-09T03:48:57.447275+0000","last_peered":"2023-11-09T03:48:57.447275+0000","last_clean":"2023-11-09T03:48:57.447275+0000","last_became_active":"2023-11-09T03:48:57.441090+0000","last_became_peered":"2023-11-09T03:48:57.441090+0000","last_unstale":"2023-11-09T03:48:57.447275+0000","last_undegraded":"2023-11-09T03:48:57.447275+0000","last_fullsized":"2023-11-09T03:48:57.447275+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:01:29.460175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.431107+0000","last_change":"2023-11-09T03:48:57.052926+0000","last_active":"2023-11-09T03:48:57.431107+0000","last_peered":"2023-11-09T03:48:57.431107+0000","last_clean":"2023-11-09T03:48:57.431107+0000","last_became_active":"2023-11-09T03:48:57.052660+0000","last_became_peered":"2023-11-09T03:48:57.052660+0000","last_unstale":"2023-11-09T03:48:57.431107+0000","last_undegraded":"2023-11-09T03:48:57.431107+0000","last_fullsized":"2023-11-09T03:48:57.431107+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:54:32.585076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036800+0000","last_change":"2023-11-09T03:48:01.456245+0000","last_active":"2023-11-09T03:48:57.036800+0000","last_peered":"2023-11-09T03:48:57.036800+0000","last_clean":"2023-11-09T03:48:57.036800+0000","last_became_active":"2023-11-09T03:48:01.455944+0000","last_became_peered":"2023-11-09T03:48:01.455944+0000","last_unstale":"2023-11-09T03:48:57.036800+0000","last_undegraded":"2023-11-09T03:48:57.036800+0000","last_fullsized":"2023-11-09T03:48:57.036800+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:06:14.849196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.047935+0000","last_change":"2023-11-09T03:48:02.612341+0000","last_active":"2023-11-09T03:48:57.047935+0000","last_peered":"2023-11-09T03:48:57.047935+0000","last_clean":"2023-11-09T03:48:57.047935+0000","last_became_active":"2023-11-09T03:48:00.444426+0000","last_became_peered":"2023-11-09T03:48:00.444426+0000","last_unstale":"2023-11-09T03:48:57.047935+0000","last_undegraded":"2023-11-09T03:48:57.047935+0000","last_fullsized":"2023-11-09T03:48:57.047935+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:19:53.969861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010812420000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":57,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.042396+0000","last_change":"2023-11-09T03:47:03.029063+0000","last_active":"2023-11-09T03:48:57.042396+0000","last_peered":"2023-11-09T03:48:57.042396+0000","last_clean":"2023-11-09T03:48:57.042396+0000","last_became_active":"2023-11-09T03:47:01.593582+0000","last_became_peered":"2023-11-09T03:47:01.593582+0000","last_unstale":"2023-11-09T03:48:57.042396+0000","last_undegraded":"2023-11-09T03:48:57.042396+0000","last_fullsized":"2023-11-09T03:48:57.042396+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:56:20.649937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041855100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.443351+0000","last_change":"2023-11-09T03:48:57.436261+0000","last_active":"2023-11-09T03:48:57.443351+0000","last_peered":"2023-11-09T03:48:57.443351+0000","last_clean":"2023-11-09T03:48:57.443351+0000","last_became_active":"2023-11-09T03:48:57.435990+0000","last_became_peered":"2023-11-09T03:48:57.435990+0000","last_unstale":"2023-11-09T03:48:57.443351+0000","last_undegraded":"2023-11-09T03:48:57.443351+0000","last_fullsized":"2023-11-09T03:48:57.443351+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:58:16.470882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.426528+0000","last_change":"2023-11-09T03:48:57.053135+0000","last_active":"2023-11-09T03:48:57.426528+0000","last_peered":"2023-11-09T03:48:57.426528+0000","last_clean":"2023-11-09T03:48:57.426528+0000","last_became_active":"2023-11-09T03:48:57.052901+0000","last_became_peered":"2023-11-09T03:48:57.052901+0000","last_unstale":"2023-11-09T03:48:57.426528+0000","last_undegraded":"2023-11-09T03:48:57.426528+0000","last_fullsized":"2023-11-09T03:48:57.426528+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:33:16.524514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.037144+0000","last_change":"2023-11-09T03:48:01.450271+0000","last_active":"2023-11-09T03:48:57.037144+0000","last_peered":"2023-11-09T03:48:57.037144+0000","last_clean":"2023-11-09T03:48:57.037144+0000","last_became_active":"2023-11-09T03:48:01.449998+0000","last_became_peered":"2023-11-09T03:48:01.449998+0000","last_unstale":"2023-11-09T03:48:57.037144+0000","last_undegraded":"2023-11-09T03:48:57.037144+0000","last_fullsized":"2023-11-09T03:48:57.037144+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:12:27.170650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.042158+0000","last_change":"2023-11-09T03:48:02.852114+0000","last_active":"2023-11-09T03:48:57.042158+0000","last_peered":"2023-11-09T03:48:57.042158+0000","last_clean":"2023-11-09T03:48:57.042158+0000","last_became_active":"2023-11-09T03:48:00.447203+0000","last_became_peered":"2023-11-09T03:48:00.447203+0000","last_unstale":"2023-11-09T03:48:57.042158+0000","last_undegraded":"2023-11-09T03:48:57.042158+0000","last_fullsized":"2023-11-09T03:48:57.042158+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:11:36.826430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049189500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.048164+0000","last_change":"2023-11-09T03:47:02.619336+0000","last_active":"2023-11-09T03:48:57.048164+0000","last_peered":"2023-11-09T03:48:57.048164+0000","last_clean":"2023-11-09T03:48:57.048164+0000","last_became_active":"2023-11-09T03:47:01.587749+0000","last_became_peered":"2023-11-09T03:47:01.587749+0000","last_unstale":"2023-11-09T03:48:57.048164+0000","last_undegraded":"2023-11-09T03:48:57.048164+0000","last_fullsized":"2023-11-09T03:48:57.048164+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063032099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.451279+0000","last_change":"2023-11-09T03:48:57.441977+0000","last_active":"2023-11-09T03:48:57.451279+0000","last_peered":"2023-11-09T03:48:57.451279+0000","last_clean":"2023-11-09T03:48:57.451279+0000","last_became_active":"2023-11-09T03:48:57.441727+0000","last_became_peered":"2023-11-09T03:48:57.441727+0000","last_unstale":"2023-11-09T03:48:57.451279+0000","last_undegraded":"2023-11-09T03:48:57.451279+0000","last_fullsized":"2023-11-09T03:48:57.451279+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:38:58.229222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.431566+0000","last_change":"2023-11-09T03:48:57.053278+0000","last_active":"2023-11-09T03:48:57.431566+0000","last_peered":"2023-11-09T03:48:57.431566+0000","last_clean":"2023-11-09T03:48:57.431566+0000","last_became_active":"2023-11-09T03:48:57.053028+0000","last_became_peered":"2023-11-09T03:48:57.053028+0000","last_unstale":"2023-11-09T03:48:57.431566+0000","last_undegraded":"2023-11-09T03:48:57.431566+0000","last_fullsized":"2023-11-09T03:48:57.431566+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:52:53.826209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.033557+0000","last_change":"2023-11-09T03:48:01.454147+0000","last_active":"2023-11-09T03:48:57.033557+0000","last_peered":"2023-11-09T03:48:57.033557+0000","last_clean":"2023-11-09T03:48:57.033557+0000","last_became_active":"2023-11-09T03:48:01.453868+0000","last_became_peered":"2023-11-09T03:48:01.453868+0000","last_unstale":"2023-11-09T03:48:57.033557+0000","last_undegraded":"2023-11-09T03:48:57.033557+0000","last_fullsized":"2023-11-09T03:48:57.033557+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:05:09.655318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.045943+0000","last_change":"2023-11-09T03:48:02.612000+0000","last_active":"2023-11-09T03:48:57.045943+0000","last_peered":"2023-11-09T03:48:57.045943+0000","last_clean":"2023-11-09T03:48:57.045943+0000","last_became_active":"2023-11-09T03:48:00.441667+0000","last_became_peered":"2023-11-09T03:48:00.441667+0000","last_unstale":"2023-11-09T03:48:57.045943+0000","last_undegraded":"2023-11-09T03:48:57.045943+0000","last_fullsized":"2023-11-09T03:48:57.045943+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:12:26.164644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000966781,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036868+0000","last_change":"2023-11-09T03:47:02.581658+0000","last_active":"2023-11-09T03:48:57.036868+0000","last_peered":"2023-11-09T03:48:57.036868+0000","last_clean":"2023-11-09T03:48:57.036868+0000","last_became_active":"2023-11-09T03:47:01.588612+0000","last_became_peered":"2023-11-09T03:47:01.588612+0000","last_unstale":"2023-11-09T03:48:57.036868+0000","last_undegraded":"2023-11-09T03:48:57.036868+0000","last_fullsized":"2023-11-09T03:48:57.036868+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045238500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.440573+0000","last_change":"2023-11-09T03:48:57.439162+0000","last_active":"2023-11-09T03:48:57.440573+0000","last_peered":"2023-11-09T03:48:57.440573+0000","last_clean":"2023-11-09T03:48:57.440573+0000","last_became_active":"2023-11-09T03:48:57.438911+0000","last_became_peered":"2023-11-09T03:48:57.438911+0000","last_unstale":"2023-11-09T03:48:57.440573+0000","last_undegraded":"2023-11-09T03:48:57.440573+0000","last_fullsized":"2023-11-09T03:48:57.440573+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:33:46.080846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.434733+0000","last_change":"2023-11-09T03:48:57.049187+0000","last_active":"2023-11-09T03:48:57.434733+0000","last_peered":"2023-11-09T03:48:57.434733+0000","last_clean":"2023-11-09T03:48:57.434733+0000","last_became_active":"2023-11-09T03:48:57.048829+0000","last_became_peered":"2023-11-09T03:48:57.048829+0000","last_unstale":"2023-11-09T03:48:57.434733+0000","last_undegraded":"2023-11-09T03:48:57.434733+0000","last_fullsized":"2023-11-09T03:48:57.434733+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:00.081394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.043113+0000","last_change":"2023-11-09T03:48:01.449834+0000","last_active":"2023-11-09T03:48:57.043113+0000","last_peered":"2023-11-09T03:48:57.043113+0000","last_clean":"2023-11-09T03:48:57.043113+0000","last_became_active":"2023-11-09T03:48:01.449552+0000","last_became_peered":"2023-11-09T03:48:01.449552+0000","last_unstale":"2023-11-09T03:48:57.043113+0000","last_undegraded":"2023-11-09T03:48:57.043113+0000","last_fullsized":"2023-11-09T03:48:57.043113+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:11:37.832436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036293+0000","last_change":"2023-11-09T03:48:02.682197+0000","last_active":"2023-11-09T03:48:57.036293+0000","last_peered":"2023-11-09T03:48:57.036293+0000","last_clean":"2023-11-09T03:48:57.036293+0000","last_became_active":"2023-11-09T03:48:00.445616+0000","last_became_peered":"2023-11-09T03:48:00.445616+0000","last_unstale":"2023-11-09T03:48:57.036293+0000","last_undegraded":"2023-11-09T03:48:57.036293+0000","last_fullsized":"2023-11-09T03:48:57.036293+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:57:03.474084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000416494,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.042714+0000","last_change":"2023-11-09T03:47:02.618828+0000","last_active":"2023-11-09T03:48:57.042714+0000","last_peered":"2023-11-09T03:48:57.042714+0000","last_clean":"2023-11-09T03:48:57.042714+0000","last_became_active":"2023-11-09T03:47:01.590679+0000","last_became_peered":"2023-11-09T03:47:01.590679+0000","last_unstale":"2023-11-09T03:48:57.042714+0000","last_undegraded":"2023-11-09T03:48:57.042714+0000","last_fullsized":"2023-11-09T03:48:57.042714+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:06:30.982987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070668,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.442589+0000","last_change":"2023-11-09T03:48:57.439536+0000","last_active":"2023-11-09T03:48:57.442589+0000","last_peered":"2023-11-09T03:48:57.442589+0000","last_clean":"2023-11-09T03:48:57.442589+0000","last_became_active":"2023-11-09T03:48:57.439282+0000","last_became_peered":"2023-11-09T03:48:57.439282+0000","last_unstale":"2023-11-09T03:48:57.442589+0000","last_undegraded":"2023-11-09T03:48:57.442589+0000","last_fullsized":"2023-11-09T03:48:57.442589+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:36:03.301496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.427325+0000","last_change":"2023-11-09T03:48:57.047878+0000","last_active":"2023-11-09T03:48:57.427325+0000","last_peered":"2023-11-09T03:48:57.427325+0000","last_clean":"2023-11-09T03:48:57.427325+0000","last_became_active":"2023-11-09T03:48:57.047632+0000","last_became_peered":"2023-11-09T03:48:57.047632+0000","last_unstale":"2023-11-09T03:48:57.427325+0000","last_undegraded":"2023-11-09T03:48:57.427325+0000","last_fullsized":"2023-11-09T03:48:57.427325+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:26:21.550088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.033218+0000","last_change":"2023-11-09T03:48:01.447647+0000","last_active":"2023-11-09T03:48:57.033218+0000","last_peered":"2023-11-09T03:48:57.033218+0000","last_clean":"2023-11-09T03:48:57.033218+0000","last_became_active":"2023-11-09T03:48:01.447302+0000","last_became_peered":"2023-11-09T03:48:01.447302+0000","last_unstale":"2023-11-09T03:48:57.033218+0000","last_undegraded":"2023-11-09T03:48:57.033218+0000","last_fullsized":"2023-11-09T03:48:57.033218+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:08:47.931939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036646+0000","last_change":"2023-11-09T03:48:03.327731+0000","last_active":"2023-11-09T03:48:57.036646+0000","last_peered":"2023-11-09T03:48:57.036646+0000","last_clean":"2023-11-09T03:48:57.036646+0000","last_became_active":"2023-11-09T03:48:00.449660+0000","last_became_peered":"2023-11-09T03:48:00.449660+0000","last_unstale":"2023-11-09T03:48:57.036646+0000","last_undegraded":"2023-11-09T03:48:57.036646+0000","last_fullsized":"2023-11-09T03:48:57.036646+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:02:07.940908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037816599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.033938+0000","last_change":"2023-11-09T03:47:03.146528+0000","last_active":"2023-11-09T03:48:57.033938+0000","last_peered":"2023-11-09T03:48:57.033938+0000","last_clean":"2023-11-09T03:48:57.033938+0000","last_became_active":"2023-11-09T03:47:01.587060+0000","last_became_peered":"2023-11-09T03:47:01.587060+0000","last_unstale":"2023-11-09T03:48:57.033938+0000","last_undegraded":"2023-11-09T03:48:57.033938+0000","last_fullsized":"2023-11-09T03:48:57.033938+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:39:15.238496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079064999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.451252+0000","last_change":"2023-11-09T03:48:57.443732+0000","last_active":"2023-11-09T03:48:57.451252+0000","last_peered":"2023-11-09T03:48:57.451252+0000","last_clean":"2023-11-09T03:48:57.451252+0000","last_became_active":"2023-11-09T03:48:57.443458+0000","last_became_peered":"2023-11-09T03:48:57.443458+0000","last_unstale":"2023-11-09T03:48:57.451252+0000","last_undegraded":"2023-11-09T03:48:57.451252+0000","last_fullsized":"2023-11-09T03:48:57.451252+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:50:34.258596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.435028+0000","last_change":"2023-11-09T03:48:57.049776+0000","last_active":"2023-11-09T03:48:57.435028+0000","last_peered":"2023-11-09T03:48:57.435028+0000","last_clean":"2023-11-09T03:48:57.435028+0000","last_became_active":"2023-11-09T03:48:57.049533+0000","last_became_peered":"2023-11-09T03:48:57.049533+0000","last_unstale":"2023-11-09T03:48:57.435028+0000","last_undegraded":"2023-11-09T03:48:57.435028+0000","last_fullsized":"2023-11-09T03:48:57.435028+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:33:03.051538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036452+0000","last_change":"2023-11-09T03:48:01.453939+0000","last_active":"2023-11-09T03:48:57.036452+0000","last_peered":"2023-11-09T03:48:57.036452+0000","last_clean":"2023-11-09T03:48:57.036452+0000","last_became_active":"2023-11-09T03:48:01.453600+0000","last_became_peered":"2023-11-09T03:48:01.453600+0000","last_unstale":"2023-11-09T03:48:57.036452+0000","last_undegraded":"2023-11-09T03:48:57.036452+0000","last_fullsized":"2023-11-09T03:48:57.036452+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:45:10.522254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.033988+0000","last_change":"2023-11-09T03:48:02.440910+0000","last_active":"2023-11-09T03:48:57.033988+0000","last_peered":"2023-11-09T03:48:57.033988+0000","last_clean":"2023-11-09T03:48:57.033988+0000","last_became_active":"2023-11-09T03:48:00.443791+0000","last_became_peered":"2023-11-09T03:48:00.443791+0000","last_unstale":"2023-11-09T03:48:57.033988+0000","last_undegraded":"2023-11-09T03:48:57.033988+0000","last_fullsized":"2023-11-09T03:48:57.033988+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:11:36.826430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069223400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.033390+0000","last_change":"2023-11-09T03:47:03.148246+0000","last_active":"2023-11-09T03:48:57.033390+0000","last_peered":"2023-11-09T03:48:57.033390+0000","last_clean":"2023-11-09T03:48:57.033390+0000","last_became_active":"2023-11-09T03:47:01.590988+0000","last_became_peered":"2023-11-09T03:47:01.590988+0000","last_unstale":"2023-11-09T03:48:57.033390+0000","last_undegraded":"2023-11-09T03:48:57.033390+0000","last_fullsized":"2023-11-09T03:48:57.033390+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:07:20.785445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055651099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.451283+0000","last_change":"2023-11-09T03:48:57.443662+0000","last_active":"2023-11-09T03:48:57.451283+0000","last_peered":"2023-11-09T03:48:57.451283+0000","last_clean":"2023-11-09T03:48:57.451283+0000","last_became_active":"2023-11-09T03:48:57.443352+0000","last_became_peered":"2023-11-09T03:48:57.443352+0000","last_unstale":"2023-11-09T03:48:57.451283+0000","last_undegraded":"2023-11-09T03:48:57.451283+0000","last_fullsized":"2023-11-09T03:48:57.451283+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:00:55.763662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.437037+0000","last_change":"2023-11-09T03:48:57.049287+0000","last_active":"2023-11-09T03:48:57.437037+0000","last_peered":"2023-11-09T03:48:57.437037+0000","last_clean":"2023-11-09T03:48:57.437037+0000","last_became_active":"2023-11-09T03:48:57.048990+0000","last_became_peered":"2023-11-09T03:48:57.048990+0000","last_unstale":"2023-11-09T03:48:57.437037+0000","last_undegraded":"2023-11-09T03:48:57.437037+0000","last_fullsized":"2023-11-09T03:48:57.437037+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:49:51.200196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.037483+0000","last_change":"2023-11-09T03:48:01.450557+0000","last_active":"2023-11-09T03:48:57.037483+0000","last_peered":"2023-11-09T03:48:57.037483+0000","last_clean":"2023-11-09T03:48:57.037483+0000","last_became_active":"2023-11-09T03:48:01.450291+0000","last_became_peered":"2023-11-09T03:48:01.450291+0000","last_unstale":"2023-11-09T03:48:57.037483+0000","last_undegraded":"2023-11-09T03:48:57.037483+0000","last_fullsized":"2023-11-09T03:48:57.037483+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:21:29.699594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.040557+0000","last_change":"2023-11-09T03:48:02.852010+0000","last_active":"2023-11-09T03:48:57.040557+0000","last_peered":"2023-11-09T03:48:57.040557+0000","last_clean":"2023-11-09T03:48:57.040557+0000","last_became_active":"2023-11-09T03:48:00.445267+0000","last_became_peered":"2023-11-09T03:48:00.445267+0000","last_unstale":"2023-11-09T03:48:57.040557+0000","last_undegraded":"2023-11-09T03:48:57.040557+0000","last_fullsized":"2023-11-09T03:48:57.040557+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:17.989628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064559199999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":63,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036817+0000","last_change":"2023-11-09T03:47:02.582754+0000","last_active":"2023-11-09T03:48:57.036817+0000","last_peered":"2023-11-09T03:48:57.036817+0000","last_clean":"2023-11-09T03:48:57.036817+0000","last_became_active":"2023-11-09T03:47:01.589715+0000","last_became_peered":"2023-11-09T03:47:01.589715+0000","last_unstale":"2023-11-09T03:48:57.036817+0000","last_undegraded":"2023-11-09T03:48:57.036817+0000","last_fullsized":"2023-11-09T03:48:57.036817+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:53.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099342799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.441775+0000","last_change":"2023-11-09T03:48:57.440148+0000","last_active":"2023-11-09T03:48:57.441775+0000","last_peered":"2023-11-09T03:48:57.441775+0000","last_clean":"2023-11-09T03:48:57.441775+0000","last_became_active":"2023-11-09T03:48:57.439835+0000","last_became_peered":"2023-11-09T03:48:57.439835+0000","last_unstale":"2023-11-09T03:48:57.441775+0000","last_undegraded":"2023-11-09T03:48:57.441775+0000","last_fullsized":"2023-11-09T03:48:57.441775+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:44:32.408986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.428519+0000","last_change":"2023-11-09T03:48:57.038706+0000","last_active":"2023-11-09T03:48:57.428519+0000","last_peered":"2023-11-09T03:48:57.428519+0000","last_clean":"2023-11-09T03:48:57.428519+0000","last_became_active":"2023-11-09T03:48:57.038433+0000","last_became_peered":"2023-11-09T03:48:57.038433+0000","last_unstale":"2023-11-09T03:48:57.428519+0000","last_undegraded":"2023-11-09T03:48:57.428519+0000","last_fullsized":"2023-11-09T03:48:57.428519+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:36:42.256387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.048135+0000","last_change":"2023-11-09T03:48:01.452456+0000","last_active":"2023-11-09T03:48:57.048135+0000","last_peered":"2023-11-09T03:48:57.048135+0000","last_clean":"2023-11-09T03:48:57.048135+0000","last_became_active":"2023-11-09T03:48:01.452059+0000","last_became_peered":"2023-11-09T03:48:01.452059+0000","last_unstale":"2023-11-09T03:48:57.048135+0000","last_undegraded":"2023-11-09T03:48:57.048135+0000","last_fullsized":"2023-11-09T03:48:57.048135+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:57:56.855869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.037342+0000","last_change":"2023-11-09T03:48:02.442557+0000","last_active":"2023-11-09T03:48:57.037342+0000","last_peered":"2023-11-09T03:48:57.037342+0000","last_clean":"2023-11-09T03:48:57.037342+0000","last_became_active":"2023-11-09T03:48:00.444823+0000","last_became_peered":"2023-11-09T03:48:00.444823+0000","last_unstale":"2023-11-09T03:48:57.037342+0000","last_undegraded":"2023-11-09T03:48:57.037342+0000","last_fullsized":"2023-11-09T03:48:57.037342+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:19:53.969861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075076299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.043872+0000","last_change":"2023-11-09T03:47:02.724896+0000","last_active":"2023-11-09T03:48:57.043872+0000","last_peered":"2023-11-09T03:48:57.043872+0000","last_clean":"2023-11-09T03:48:57.043872+0000","last_became_active":"2023-11-09T03:47:01.592784+0000","last_became_peered":"2023-11-09T03:47:01.592784+0000","last_unstale":"2023-11-09T03:48:57.043872+0000","last_undegraded":"2023-11-09T03:48:57.043872+0000","last_fullsized":"2023-11-09T03:48:57.043872+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:51:32.016204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056683500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.439394+0000","last_change":"2023-11-09T03:48:57.437705+0000","last_active":"2023-11-09T03:48:57.439394+0000","last_peered":"2023-11-09T03:48:57.439394+0000","last_clean":"2023-11-09T03:48:57.439394+0000","last_became_active":"2023-11-09T03:48:57.437436+0000","last_became_peered":"2023-11-09T03:48:57.437436+0000","last_unstale":"2023-11-09T03:48:57.439394+0000","last_undegraded":"2023-11-09T03:48:57.439394+0000","last_fullsized":"2023-11-09T03:48:57.439394+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:37:27.085476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.435502+0000","last_change":"2023-11-09T03:48:57.050307+0000","last_active":"2023-11-09T03:48:57.435502+0000","last_peered":"2023-11-09T03:48:57.435502+0000","last_clean":"2023-11-09T03:48:57.435502+0000","last_became_active":"2023-11-09T03:48:57.050048+0000","last_became_peered":"2023-11-09T03:48:57.050048+0000","last_unstale":"2023-11-09T03:48:57.435502+0000","last_undegraded":"2023-11-09T03:48:57.435502+0000","last_fullsized":"2023-11-09T03:48:57.435502+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:51:17.478024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.037238+0000","last_change":"2023-11-09T03:48:01.452167+0000","last_active":"2023-11-09T03:48:57.037238+0000","last_peered":"2023-11-09T03:48:57.037238+0000","last_clean":"2023-11-09T03:48:57.037238+0000","last_became_active":"2023-11-09T03:48:01.451780+0000","last_became_peered":"2023-11-09T03:48:01.451780+0000","last_unstale":"2023-11-09T03:48:57.037238+0000","last_undegraded":"2023-11-09T03:48:57.037238+0000","last_fullsized":"2023-11-09T03:48:57.037238+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:15:36.287302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.047850+0000","last_change":"2023-11-09T03:48:02.612246+0000","last_active":"2023-11-09T03:48:57.047850+0000","last_peered":"2023-11-09T03:48:57.047850+0000","last_clean":"2023-11-09T03:48:57.047850+0000","last_became_active":"2023-11-09T03:48:00.444629+0000","last_became_peered":"2023-11-09T03:48:00.444629+0000","last_unstale":"2023-11-09T03:48:57.047850+0000","last_undegraded":"2023-11-09T03:48:57.047850+0000","last_fullsized":"2023-11-09T03:48:57.047850+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:51:17.754054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098167299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.037571+0000","last_change":"2023-11-09T03:47:03.146527+0000","last_active":"2023-11-09T03:48:57.037571+0000","last_peered":"2023-11-09T03:48:57.037571+0000","last_clean":"2023-11-09T03:48:57.037571+0000","last_became_active":"2023-11-09T03:47:01.587858+0000","last_became_peered":"2023-11-09T03:47:01.587858+0000","last_unstale":"2023-11-09T03:48:57.037571+0000","last_undegraded":"2023-11-09T03:48:57.037571+0000","last_fullsized":"2023-11-09T03:48:57.037571+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:56:04.201126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084439599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.440300+0000","last_change":"2023-11-09T03:48:57.437592+0000","last_active":"2023-11-09T03:48:57.440300+0000","last_peered":"2023-11-09T03:48:57.440300+0000","last_clean":"2023-11-09T03:48:57.440300+0000","last_became_active":"2023-11-09T03:48:57.437311+0000","last_became_peered":"2023-11-09T03:48:57.437311+0000","last_unstale":"2023-11-09T03:48:57.440300+0000","last_undegraded":"2023-11-09T03:48:57.440300+0000","last_fullsized":"2023-11-09T03:48:57.440300+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:50:47.233700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.425939+0000","last_change":"2023-11-09T03:48:57.052503+0000","last_active":"2023-11-09T03:48:57.425939+0000","last_peered":"2023-11-09T03:48:57.425939+0000","last_clean":"2023-11-09T03:48:57.425939+0000","last_became_active":"2023-11-09T03:48:57.052219+0000","last_became_peered":"2023-11-09T03:48:57.052219+0000","last_unstale":"2023-11-09T03:48:57.425939+0000","last_undegraded":"2023-11-09T03:48:57.425939+0000","last_fullsized":"2023-11-09T03:48:57.425939+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:46:31.166452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.043687+0000","last_change":"2023-11-09T03:48:01.450685+0000","last_active":"2023-11-09T03:48:57.043687+0000","last_peered":"2023-11-09T03:48:57.043687+0000","last_clean":"2023-11-09T03:48:57.043687+0000","last_became_active":"2023-11-09T03:48:01.450445+0000","last_became_peered":"2023-11-09T03:48:01.450445+0000","last_unstale":"2023-11-09T03:48:57.043687+0000","last_undegraded":"2023-11-09T03:48:57.043687+0000","last_fullsized":"2023-11-09T03:48:57.043687+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:16:00.220961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.033727+0000","last_change":"2023-11-09T03:48:02.441928+0000","last_active":"2023-11-09T03:48:57.033727+0000","last_peered":"2023-11-09T03:48:57.033727+0000","last_clean":"2023-11-09T03:48:57.033727+0000","last_became_active":"2023-11-09T03:48:00.444417+0000","last_became_peered":"2023-11-09T03:48:00.444417+0000","last_unstale":"2023-11-09T03:48:57.033727+0000","last_undegraded":"2023-11-09T03:48:57.033727+0000","last_fullsized":"2023-11-09T03:48:57.033727+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:55.450096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072428900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.044814+0000","last_change":"2023-11-09T03:47:02.725223+0000","last_active":"2023-11-09T03:48:57.044814+0000","last_peered":"2023-11-09T03:48:57.044814+0000","last_clean":"2023-11-09T03:48:57.044814+0000","last_became_active":"2023-11-09T03:47:01.590635+0000","last_became_peered":"2023-11-09T03:47:01.590635+0000","last_unstale":"2023-11-09T03:48:57.044814+0000","last_undegraded":"2023-11-09T03:48:57.044814+0000","last_fullsized":"2023-11-09T03:48:57.044814+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:05.754357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051685100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.451304+0000","last_change":"2023-11-09T03:48:57.445874+0000","last_active":"2023-11-09T03:48:57.451304+0000","last_peered":"2023-11-09T03:48:57.451304+0000","last_clean":"2023-11-09T03:48:57.451304+0000","last_became_active":"2023-11-09T03:48:57.445519+0000","last_became_peered":"2023-11-09T03:48:57.445519+0000","last_unstale":"2023-11-09T03:48:57.451304+0000","last_undegraded":"2023-11-09T03:48:57.451304+0000","last_fullsized":"2023-11-09T03:48:57.451304+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:33:12.883340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.434564+0000","last_change":"2023-11-09T03:48:57.051180+0000","last_active":"2023-11-09T03:48:57.434564+0000","last_peered":"2023-11-09T03:48:57.434564+0000","last_clean":"2023-11-09T03:48:57.434564+0000","last_became_active":"2023-11-09T03:48:57.050907+0000","last_became_peered":"2023-11-09T03:48:57.050907+0000","last_unstale":"2023-11-09T03:48:57.434564+0000","last_undegraded":"2023-11-09T03:48:57.434564+0000","last_fullsized":"2023-11-09T03:48:57.434564+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:44:34.138203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.042673+0000","last_change":"2023-11-09T03:48:01.453143+0000","last_active":"2023-11-09T03:48:57.042673+0000","last_peered":"2023-11-09T03:48:57.042673+0000","last_clean":"2023-11-09T03:48:57.042673+0000","last_became_active":"2023-11-09T03:48:01.452873+0000","last_became_peered":"2023-11-09T03:48:01.452873+0000","last_unstale":"2023-11-09T03:48:57.042673+0000","last_undegraded":"2023-11-09T03:48:57.042673+0000","last_fullsized":"2023-11-09T03:48:57.042673+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:06:28.575429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.034059+0000","last_change":"2023-11-09T03:48:02.442416+0000","last_active":"2023-11-09T03:48:57.034059+0000","last_peered":"2023-11-09T03:48:57.034059+0000","last_clean":"2023-11-09T03:48:57.034059+0000","last_became_active":"2023-11-09T03:48:00.443765+0000","last_became_peered":"2023-11-09T03:48:00.443765+0000","last_unstale":"2023-11-09T03:48:57.034059+0000","last_undegraded":"2023-11-09T03:48:57.034059+0000","last_fullsized":"2023-11-09T03:48:57.034059+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:41:22.321811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012468850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":63,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.037371+0000","last_change":"2023-11-09T03:47:02.582739+0000","last_active":"2023-11-09T03:48:57.037371+0000","last_peered":"2023-11-09T03:48:57.037371+0000","last_clean":"2023-11-09T03:48:57.037371+0000","last_became_active":"2023-11-09T03:47:01.592743+0000","last_became_peered":"2023-11-09T03:47:01.592743+0000","last_unstale":"2023-11-09T03:48:57.037371+0000","last_undegraded":"2023-11-09T03:48:57.037371+0000","last_fullsized":"2023-11-09T03:48:57.037371+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051802700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.438429+0000","last_change":"2023-11-09T03:48:57.436905+0000","last_active":"2023-11-09T03:48:57.438429+0000","last_peered":"2023-11-09T03:48:57.438429+0000","last_clean":"2023-11-09T03:48:57.438429+0000","last_became_active":"2023-11-09T03:48:57.436190+0000","last_became_peered":"2023-11-09T03:48:57.436190+0000","last_unstale":"2023-11-09T03:48:57.438429+0000","last_undegraded":"2023-11-09T03:48:57.438429+0000","last_fullsized":"2023-11-09T03:48:57.438429+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:18:06.196599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.436575+0000","last_change":"2023-11-09T03:48:57.052108+0000","last_active":"2023-11-09T03:48:57.436575+0000","last_peered":"2023-11-09T03:48:57.436575+0000","last_clean":"2023-11-09T03:48:57.436575+0000","last_became_active":"2023-11-09T03:48:57.051813+0000","last_became_peered":"2023-11-09T03:48:57.051813+0000","last_unstale":"2023-11-09T03:48:57.436575+0000","last_undegraded":"2023-11-09T03:48:57.436575+0000","last_fullsized":"2023-11-09T03:48:57.436575+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:06:46.097576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.037044+0000","last_change":"2023-11-09T03:48:01.442195+0000","last_active":"2023-11-09T03:48:57.037044+0000","last_peered":"2023-11-09T03:48:57.037044+0000","last_clean":"2023-11-09T03:48:57.037044+0000","last_became_active":"2023-11-09T03:48:01.441887+0000","last_became_peered":"2023-11-09T03:48:01.441887+0000","last_unstale":"2023-11-09T03:48:57.037044+0000","last_undegraded":"2023-11-09T03:48:57.037044+0000","last_fullsized":"2023-11-09T03:48:57.037044+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:06:28.575429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036749+0000","last_change":"2023-11-09T03:48:02.442061+0000","last_active":"2023-11-09T03:48:57.036749+0000","last_peered":"2023-11-09T03:48:57.036749+0000","last_clean":"2023-11-09T03:48:57.036749+0000","last_became_active":"2023-11-09T03:48:00.445921+0000","last_became_peered":"2023-11-09T03:48:00.445921+0000","last_unstale":"2023-11-09T03:48:57.036749+0000","last_undegraded":"2023-11-09T03:48:57.036749+0000","last_fullsized":"2023-11-09T03:48:57.036749+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:50:55.299235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012719770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.042931+0000","last_change":"2023-11-09T03:47:02.724953+0000","last_active":"2023-11-09T03:48:57.042931+0000","last_peered":"2023-11-09T03:48:57.042931+0000","last_clean":"2023-11-09T03:48:57.042931+0000","last_became_active":"2023-11-09T03:47:01.587133+0000","last_became_peered":"2023-11-09T03:47:01.587133+0000","last_unstale":"2023-11-09T03:48:57.042931+0000","last_undegraded":"2023-11-09T03:48:57.042931+0000","last_fullsized":"2023-11-09T03:48:57.042931+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:23:48.521161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066497599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.446994+0000","last_change":"2023-11-09T03:48:57.442902+0000","last_active":"2023-11-09T03:48:57.446994+0000","last_peered":"2023-11-09T03:48:57.446994+0000","last_clean":"2023-11-09T03:48:57.446994+0000","last_became_active":"2023-11-09T03:48:57.442644+0000","last_became_peered":"2023-11-09T03:48:57.442644+0000","last_unstale":"2023-11-09T03:48:57.446994+0000","last_undegraded":"2023-11-09T03:48:57.446994+0000","last_fullsized":"2023-11-09T03:48:57.446994+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:59:11.986476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.430585+0000","last_change":"2023-11-09T03:48:57.045458+0000","last_active":"2023-11-09T03:48:57.430585+0000","last_peered":"2023-11-09T03:48:57.430585+0000","last_clean":"2023-11-09T03:48:57.430585+0000","last_became_active":"2023-11-09T03:48:57.045192+0000","last_became_peered":"2023-11-09T03:48:57.045192+0000","last_unstale":"2023-11-09T03:48:57.430585+0000","last_undegraded":"2023-11-09T03:48:57.430585+0000","last_fullsized":"2023-11-09T03:48:57.430585+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:09:44.721115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036589+0000","last_change":"2023-11-09T03:48:01.450507+0000","last_active":"2023-11-09T03:48:57.036589+0000","last_peered":"2023-11-09T03:48:57.036589+0000","last_clean":"2023-11-09T03:48:57.036589+0000","last_became_active":"2023-11-09T03:48:01.450191+0000","last_became_peered":"2023-11-09T03:48:01.450191+0000","last_unstale":"2023-11-09T03:48:57.036589+0000","last_undegraded":"2023-11-09T03:48:57.036589+0000","last_fullsized":"2023-11-09T03:48:57.036589+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:32:40.342502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.037053+0000","last_change":"2023-11-09T03:48:02.442120+0000","last_active":"2023-11-09T03:48:57.037053+0000","last_peered":"2023-11-09T03:48:57.037053+0000","last_clean":"2023-11-09T03:48:57.037053+0000","last_became_active":"2023-11-09T03:48:00.445504+0000","last_became_peered":"2023-11-09T03:48:00.445504+0000","last_unstale":"2023-11-09T03:48:57.037053+0000","last_undegraded":"2023-11-09T03:48:57.037053+0000","last_fullsized":"2023-11-09T03:48:57.037053+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:35:53.864627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001095766,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036480+0000","last_change":"2023-11-09T03:47:03.146217+0000","last_active":"2023-11-09T03:48:57.036480+0000","last_peered":"2023-11-09T03:48:57.036480+0000","last_clean":"2023-11-09T03:48:57.036480+0000","last_became_active":"2023-11-09T03:47:01.587598+0000","last_became_peered":"2023-11-09T03:47:01.587598+0000","last_unstale":"2023-11-09T03:48:57.036480+0000","last_undegraded":"2023-11-09T03:48:57.036480+0000","last_fullsized":"2023-11-09T03:48:57.036480+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:15:22.723455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090126400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.446917+0000","last_change":"2023-11-09T03:48:57.442288+0000","last_active":"2023-11-09T03:48:57.446917+0000","last_peered":"2023-11-09T03:48:57.446917+0000","last_clean":"2023-11-09T03:48:57.446917+0000","last_became_active":"2023-11-09T03:48:57.442008+0000","last_became_peered":"2023-11-09T03:48:57.442008+0000","last_unstale":"2023-11-09T03:48:57.446917+0000","last_undegraded":"2023-11-09T03:48:57.446917+0000","last_fullsized":"2023-11-09T03:48:57.446917+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:28:01.062126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.430864+0000","last_change":"2023-11-09T03:48:57.045639+0000","last_active":"2023-11-09T03:48:57.430864+0000","last_peered":"2023-11-09T03:48:57.430864+0000","last_clean":"2023-11-09T03:48:57.430864+0000","last_became_active":"2023-11-09T03:48:57.044947+0000","last_became_peered":"2023-11-09T03:48:57.044947+0000","last_unstale":"2023-11-09T03:48:57.430864+0000","last_undegraded":"2023-11-09T03:48:57.430864+0000","last_fullsized":"2023-11-09T03:48:57.430864+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:56:05.508536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.037061+0000","last_change":"2023-11-09T03:48:01.454878+0000","last_active":"2023-11-09T03:48:57.037061+0000","last_peered":"2023-11-09T03:48:57.037061+0000","last_clean":"2023-11-09T03:48:57.037061+0000","last_became_active":"2023-11-09T03:48:01.454591+0000","last_became_peered":"2023-11-09T03:48:01.454591+0000","last_unstale":"2023-11-09T03:48:57.037061+0000","last_undegraded":"2023-11-09T03:48:57.037061+0000","last_fullsized":"2023-11-09T03:48:57.037061+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:43:38.536898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.037060+0000","last_change":"2023-11-09T03:48:03.327800+0000","last_active":"2023-11-09T03:48:57.037060+0000","last_peered":"2023-11-09T03:48:57.037060+0000","last_clean":"2023-11-09T03:48:57.037060+0000","last_became_active":"2023-11-09T03:48:00.446576+0000","last_became_peered":"2023-11-09T03:48:00.446576+0000","last_unstale":"2023-11-09T03:48:57.037060+0000","last_undegraded":"2023-11-09T03:48:57.037060+0000","last_fullsized":"2023-11-09T03:48:57.037060+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:37:08.851102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054679700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.033451+0000","last_change":"2023-11-09T03:47:03.148283+0000","last_active":"2023-11-09T03:48:57.033451+0000","last_peered":"2023-11-09T03:48:57.033451+0000","last_clean":"2023-11-09T03:48:57.033451+0000","last_became_active":"2023-11-09T03:47:01.588841+0000","last_became_peered":"2023-11-09T03:47:01.588841+0000","last_unstale":"2023-11-09T03:48:57.033451+0000","last_undegraded":"2023-11-09T03:48:57.033451+0000","last_fullsized":"2023-11-09T03:48:57.033451+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:47:44.880741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000436081,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.437458+0000","last_change":"2023-11-09T03:48:57.434924+0000","last_active":"2023-11-09T03:48:57.437458+0000","last_peered":"2023-11-09T03:48:57.437458+0000","last_clean":"2023-11-09T03:48:57.437458+0000","last_became_active":"2023-11-09T03:48:57.434658+0000","last_became_peered":"2023-11-09T03:48:57.434658+0000","last_unstale":"2023-11-09T03:48:57.437458+0000","last_undegraded":"2023-11-09T03:48:57.437458+0000","last_fullsized":"2023-11-09T03:48:57.437458+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:18:33.315635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.425940+0000","last_change":"2023-11-09T03:48:57.054349+0000","last_active":"2023-11-09T03:48:57.425940+0000","last_peered":"2023-11-09T03:48:57.425940+0000","last_clean":"2023-11-09T03:48:57.425940+0000","last_became_active":"2023-11-09T03:48:57.054099+0000","last_became_peered":"2023-11-09T03:48:57.054099+0000","last_unstale":"2023-11-09T03:48:57.425940+0000","last_undegraded":"2023-11-09T03:48:57.425940+0000","last_fullsized":"2023-11-09T03:48:57.425940+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:59:29.131332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.043458+0000","last_change":"2023-11-09T03:48:01.450581+0000","last_active":"2023-11-09T03:48:57.043458+0000","last_peered":"2023-11-09T03:48:57.043458+0000","last_clean":"2023-11-09T03:48:57.043458+0000","last_became_active":"2023-11-09T03:48:01.450280+0000","last_became_peered":"2023-11-09T03:48:01.450280+0000","last_unstale":"2023-11-09T03:48:57.043458+0000","last_undegraded":"2023-11-09T03:48:57.043458+0000","last_fullsized":"2023-11-09T03:48:57.043458+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:18.995634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.033062+0000","last_change":"2023-11-09T03:48:02.440934+0000","last_active":"2023-11-09T03:48:57.033062+0000","last_peered":"2023-11-09T03:48:57.033062+0000","last_clean":"2023-11-09T03:48:57.033062+0000","last_became_active":"2023-11-09T03:48:00.444394+0000","last_became_peered":"2023-11-09T03:48:00.444394+0000","last_unstale":"2023-11-09T03:48:57.033062+0000","last_undegraded":"2023-11-09T03:48:57.033062+0000","last_fullsized":"2023-11-09T03:48:57.033062+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:15:59.214955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063908300000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.042931+0000","last_change":"2023-11-09T03:47:02.618856+0000","last_active":"2023-11-09T03:48:57.042931+0000","last_peered":"2023-11-09T03:48:57.042931+0000","last_clean":"2023-11-09T03:48:57.042931+0000","last_became_active":"2023-11-09T03:47:01.590430+0000","last_became_peered":"2023-11-09T03:47:01.590430+0000","last_unstale":"2023-11-09T03:48:57.042931+0000","last_undegraded":"2023-11-09T03:48:57.042931+0000","last_fullsized":"2023-11-09T03:48:57.042931+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:57:18.470305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055030200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.441790+0000","last_change":"2023-11-09T03:48:57.440624+0000","last_active":"2023-11-09T03:48:57.441790+0000","last_peered":"2023-11-09T03:48:57.441790+0000","last_clean":"2023-11-09T03:48:57.441790+0000","last_became_active":"2023-11-09T03:48:57.440385+0000","last_became_peered":"2023-11-09T03:48:57.440385+0000","last_unstale":"2023-11-09T03:48:57.441790+0000","last_undegraded":"2023-11-09T03:48:57.441790+0000","last_fullsized":"2023-11-09T03:48:57.441790+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:45:04.731311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.429447+0000","last_change":"2023-11-09T03:48:57.050494+0000","last_active":"2023-11-09T03:48:57.429447+0000","last_peered":"2023-11-09T03:48:57.429447+0000","last_clean":"2023-11-09T03:48:57.429447+0000","last_became_active":"2023-11-09T03:48:57.050181+0000","last_became_peered":"2023-11-09T03:48:57.050181+0000","last_unstale":"2023-11-09T03:48:57.429447+0000","last_undegraded":"2023-11-09T03:48:57.429447+0000","last_fullsized":"2023-11-09T03:48:57.429447+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:07:35.452073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.044641+0000","last_change":"2023-11-09T03:48:01.453600+0000","last_active":"2023-11-09T03:48:57.044641+0000","last_peered":"2023-11-09T03:48:57.044641+0000","last_clean":"2023-11-09T03:48:57.044641+0000","last_became_active":"2023-11-09T03:48:01.453330+0000","last_became_peered":"2023-11-09T03:48:01.453330+0000","last_unstale":"2023-11-09T03:48:57.044641+0000","last_undegraded":"2023-11-09T03:48:57.044641+0000","last_fullsized":"2023-11-09T03:48:57.044641+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:08:47.931939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036796+0000","last_change":"2023-11-09T03:48:03.328210+0000","last_active":"2023-11-09T03:48:57.036796+0000","last_peered":"2023-11-09T03:48:57.036796+0000","last_clean":"2023-11-09T03:48:57.036796+0000","last_became_active":"2023-11-09T03:48:00.450002+0000","last_became_peered":"2023-11-09T03:48:00.450002+0000","last_unstale":"2023-11-09T03:48:57.036796+0000","last_undegraded":"2023-11-09T03:48:57.036796+0000","last_fullsized":"2023-11-09T03:48:57.036796+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:50.391187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036521900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.046169+0000","last_change":"2023-11-09T03:47:02.619144+0000","last_active":"2023-11-09T03:48:57.046169+0000","last_peered":"2023-11-09T03:48:57.046169+0000","last_clean":"2023-11-09T03:48:57.046169+0000","last_became_active":"2023-11-09T03:47:01.590672+0000","last_became_peered":"2023-11-09T03:47:01.590672+0000","last_unstale":"2023-11-09T03:48:57.046169+0000","last_undegraded":"2023-11-09T03:48:57.046169+0000","last_fullsized":"2023-11-09T03:48:57.046169+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:03.025119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060997500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.437411+0000","last_change":"2023-11-09T03:48:57.434355+0000","last_active":"2023-11-09T03:48:57.437411+0000","last_peered":"2023-11-09T03:48:57.437411+0000","last_clean":"2023-11-09T03:48:57.437411+0000","last_became_active":"2023-11-09T03:48:57.434117+0000","last_became_peered":"2023-11-09T03:48:57.434117+0000","last_unstale":"2023-11-09T03:48:57.437411+0000","last_undegraded":"2023-11-09T03:48:57.437411+0000","last_fullsized":"2023-11-09T03:48:57.437411+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:02:49.169981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.435063+0000","last_change":"2023-11-09T03:48:57.055386+0000","last_active":"2023-11-09T03:48:57.435063+0000","last_peered":"2023-11-09T03:48:57.435063+0000","last_clean":"2023-11-09T03:48:57.435063+0000","last_became_active":"2023-11-09T03:48:57.055104+0000","last_became_peered":"2023-11-09T03:48:57.055104+0000","last_unstale":"2023-11-09T03:48:57.435063+0000","last_undegraded":"2023-11-09T03:48:57.435063+0000","last_fullsized":"2023-11-09T03:48:57.435063+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:33:03.786483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.037237+0000","last_change":"2023-11-09T03:48:01.447278+0000","last_active":"2023-11-09T03:48:57.037237+0000","last_peered":"2023-11-09T03:48:57.037237+0000","last_clean":"2023-11-09T03:48:57.037237+0000","last_became_active":"2023-11-09T03:48:01.446991+0000","last_became_peered":"2023-11-09T03:48:01.446991+0000","last_unstale":"2023-11-09T03:48:57.037237+0000","last_undegraded":"2023-11-09T03:48:57.037237+0000","last_fullsized":"2023-11-09T03:48:57.037237+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:27:49.190426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036648+0000","last_change":"2023-11-09T03:48:03.328278+0000","last_active":"2023-11-09T03:48:57.036648+0000","last_peered":"2023-11-09T03:48:57.036648+0000","last_clean":"2023-11-09T03:48:57.036648+0000","last_became_active":"2023-11-09T03:48:00.443213+0000","last_became_peered":"2023-11-09T03:48:00.443213+0000","last_unstale":"2023-11-09T03:48:57.036648+0000","last_undegraded":"2023-11-09T03:48:57.036648+0000","last_fullsized":"2023-11-09T03:48:57.036648+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:20:59.702297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030886100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.045843+0000","last_change":"2023-11-09T03:47:02.619079+0000","last_active":"2023-11-09T03:48:57.045843+0000","last_peered":"2023-11-09T03:48:57.045843+0000","last_clean":"2023-11-09T03:48:57.045843+0000","last_became_active":"2023-11-09T03:47:01.591421+0000","last_became_peered":"2023-11-09T03:47:01.591421+0000","last_unstale":"2023-11-09T03:48:57.045843+0000","last_undegraded":"2023-11-09T03:48:57.045843+0000","last_fullsized":"2023-11-09T03:48:57.045843+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:18:58.230606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068736399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.448693+0000","last_change":"2023-11-09T03:48:57.445015+0000","last_active":"2023-11-09T03:48:57.448693+0000","last_peered":"2023-11-09T03:48:57.448693+0000","last_clean":"2023-11-09T03:48:57.448693+0000","last_became_active":"2023-11-09T03:48:57.444759+0000","last_became_peered":"2023-11-09T03:48:57.444759+0000","last_unstale":"2023-11-09T03:48:57.448693+0000","last_undegraded":"2023-11-09T03:48:57.448693+0000","last_fullsized":"2023-11-09T03:48:57.448693+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:36:54.461560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.426066+0000","last_change":"2023-11-09T03:48:57.047168+0000","last_active":"2023-11-09T03:48:57.426066+0000","last_peered":"2023-11-09T03:48:57.426066+0000","last_clean":"2023-11-09T03:48:57.426066+0000","last_became_active":"2023-11-09T03:48:57.046914+0000","last_became_peered":"2023-11-09T03:48:57.046914+0000","last_unstale":"2023-11-09T03:48:57.426066+0000","last_undegraded":"2023-11-09T03:48:57.426066+0000","last_fullsized":"2023-11-09T03:48:57.426066+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:55:04.192277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.037212+0000","last_change":"2023-11-09T03:48:01.449069+0000","last_active":"2023-11-09T03:48:57.037212+0000","last_peered":"2023-11-09T03:48:57.037212+0000","last_clean":"2023-11-09T03:48:57.037212+0000","last_became_active":"2023-11-09T03:48:01.448810+0000","last_became_peered":"2023-11-09T03:48:01.448810+0000","last_unstale":"2023-11-09T03:48:57.037212+0000","last_undegraded":"2023-11-09T03:48:57.037212+0000","last_fullsized":"2023-11-09T03:48:57.037212+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:56:51.158572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.042858+0000","last_change":"2023-11-09T03:48:02.612067+0000","last_active":"2023-11-09T03:48:57.042858+0000","last_peered":"2023-11-09T03:48:57.042858+0000","last_clean":"2023-11-09T03:48:57.042858+0000","last_became_active":"2023-11-09T03:48:00.443296+0000","last_became_peered":"2023-11-09T03:48:00.443296+0000","last_unstale":"2023-11-09T03:48:57.042858+0000","last_undegraded":"2023-11-09T03:48:57.042858+0000","last_fullsized":"2023-11-09T03:48:57.042858+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:27:48.184420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010423330000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.033997+0000","last_change":"2023-11-09T03:47:03.148529+0000","last_active":"2023-11-09T03:48:57.033997+0000","last_peered":"2023-11-09T03:48:57.033997+0000","last_clean":"2023-11-09T03:48:57.033997+0000","last_became_active":"2023-11-09T03:47:01.584148+0000","last_became_peered":"2023-11-09T03:47:01.584148+0000","last_unstale":"2023-11-09T03:48:57.033997+0000","last_undegraded":"2023-11-09T03:48:57.033997+0000","last_fullsized":"2023-11-09T03:48:57.033997+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:40:00.237248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061475600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.444200+0000","last_change":"2023-11-09T03:48:57.440696+0000","last_active":"2023-11-09T03:48:57.444200+0000","last_peered":"2023-11-09T03:48:57.444200+0000","last_clean":"2023-11-09T03:48:57.444200+0000","last_became_active":"2023-11-09T03:48:57.440448+0000","last_became_peered":"2023-11-09T03:48:57.440448+0000","last_unstale":"2023-11-09T03:48:57.444200+0000","last_undegraded":"2023-11-09T03:48:57.444200+0000","last_fullsized":"2023-11-09T03:48:57.444200+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:56:03.482548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.427955+0000","last_change":"2023-11-09T03:48:57.047017+0000","last_active":"2023-11-09T03:48:57.427955+0000","last_peered":"2023-11-09T03:48:57.427955+0000","last_clean":"2023-11-09T03:48:57.427955+0000","last_became_active":"2023-11-09T03:48:57.046736+0000","last_became_peered":"2023-11-09T03:48:57.046736+0000","last_unstale":"2023-11-09T03:48:57.427955+0000","last_undegraded":"2023-11-09T03:48:57.427955+0000","last_fullsized":"2023-11-09T03:48:57.427955+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:06:46.097576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.042936+0000","last_change":"2023-11-09T03:48:01.451750+0000","last_active":"2023-11-09T03:48:57.042936+0000","last_peered":"2023-11-09T03:48:57.042936+0000","last_clean":"2023-11-09T03:48:57.042936+0000","last_became_active":"2023-11-09T03:48:01.451393+0000","last_became_peered":"2023-11-09T03:48:01.451393+0000","last_unstale":"2023-11-09T03:48:57.042936+0000","last_undegraded":"2023-11-09T03:48:57.042936+0000","last_fullsized":"2023-11-09T03:48:57.042936+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:56:51.158572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.046045+0000","last_change":"2023-11-09T03:48:02.611721+0000","last_active":"2023-11-09T03:48:57.046045+0000","last_peered":"2023-11-09T03:48:57.046045+0000","last_clean":"2023-11-09T03:48:57.046045+0000","last_became_active":"2023-11-09T03:48:00.439557+0000","last_became_peered":"2023-11-09T03:48:00.439557+0000","last_unstale":"2023-11-09T03:48:57.046045+0000","last_undegraded":"2023-11-09T03:48:57.046045+0000","last_fullsized":"2023-11-09T03:48:57.046045+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:35:53.864627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001037187,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":57,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.041573+0000","last_change":"2023-11-09T03:47:03.148217+0000","last_active":"2023-11-09T03:48:57.041573+0000","last_peered":"2023-11-09T03:48:57.041573+0000","last_clean":"2023-11-09T03:48:57.041573+0000","last_became_active":"2023-11-09T03:47:01.593424+0000","last_became_peered":"2023-11-09T03:47:01.593424+0000","last_unstale":"2023-11-09T03:48:57.041573+0000","last_undegraded":"2023-11-09T03:48:57.041573+0000","last_fullsized":"2023-11-09T03:48:57.041573+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:19:32.693691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001769865,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.447017+0000","last_change":"2023-11-09T03:48:57.442650+0000","last_active":"2023-11-09T03:48:57.447017+0000","last_peered":"2023-11-09T03:48:57.447017+0000","last_clean":"2023-11-09T03:48:57.447017+0000","last_became_active":"2023-11-09T03:48:57.442373+0000","last_became_peered":"2023-11-09T03:48:57.442373+0000","last_unstale":"2023-11-09T03:48:57.447017+0000","last_undegraded":"2023-11-09T03:48:57.447017+0000","last_fullsized":"2023-11-09T03:48:57.447017+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:21:31.419099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.435532+0000","last_change":"2023-11-09T03:48:57.050569+0000","last_active":"2023-11-09T03:48:57.435532+0000","last_peered":"2023-11-09T03:48:57.435532+0000","last_clean":"2023-11-09T03:48:57.435532+0000","last_became_active":"2023-11-09T03:48:57.050291+0000","last_became_peered":"2023-11-09T03:48:57.050291+0000","last_unstale":"2023-11-09T03:48:57.435532+0000","last_undegraded":"2023-11-09T03:48:57.435532+0000","last_fullsized":"2023-11-09T03:48:57.435532+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:02:35.821071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.033236+0000","last_change":"2023-11-09T03:48:01.447617+0000","last_active":"2023-11-09T03:48:57.033236+0000","last_peered":"2023-11-09T03:48:57.033236+0000","last_clean":"2023-11-09T03:48:57.033236+0000","last_became_active":"2023-11-09T03:48:01.447329+0000","last_became_peered":"2023-11-09T03:48:01.447329+0000","last_unstale":"2023-11-09T03:48:57.033236+0000","last_undegraded":"2023-11-09T03:48:57.033236+0000","last_fullsized":"2023-11-09T03:48:57.033236+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:54:52.433604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036831+0000","last_change":"2023-11-09T03:48:02.442427+0000","last_active":"2023-11-09T03:48:57.036831+0000","last_peered":"2023-11-09T03:48:57.036831+0000","last_clean":"2023-11-09T03:48:57.036831+0000","last_became_active":"2023-11-09T03:48:00.444856+0000","last_became_peered":"2023-11-09T03:48:00.444856+0000","last_unstale":"2023-11-09T03:48:57.036831+0000","last_undegraded":"2023-11-09T03:48:57.036831+0000","last_fullsized":"2023-11-09T03:48:57.036831+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:53:42.763434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011927730000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.033114+0000","last_change":"2023-11-09T03:47:03.146470+0000","last_active":"2023-11-09T03:48:57.033114+0000","last_peered":"2023-11-09T03:48:57.033114+0000","last_clean":"2023-11-09T03:48:57.033114+0000","last_became_active":"2023-11-09T03:47:01.587304+0000","last_became_peered":"2023-11-09T03:47:01.587304+0000","last_unstale":"2023-11-09T03:48:57.033114+0000","last_undegraded":"2023-11-09T03:48:57.033114+0000","last_fullsized":"2023-11-09T03:48:57.033114+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:58:23.339287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00110395,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.443966+0000","last_change":"2023-11-09T03:48:57.440684+0000","last_active":"2023-11-09T03:48:57.443966+0000","last_peered":"2023-11-09T03:48:57.443966+0000","last_clean":"2023-11-09T03:48:57.443966+0000","last_became_active":"2023-11-09T03:48:57.439352+0000","last_became_peered":"2023-11-09T03:48:57.439352+0000","last_unstale":"2023-11-09T03:48:57.443966+0000","last_undegraded":"2023-11-09T03:48:57.443966+0000","last_fullsized":"2023-11-09T03:48:57.443966+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:22.900543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.434897+0000","last_change":"2023-11-09T03:48:57.048247+0000","last_active":"2023-11-09T03:48:57.434897+0000","last_peered":"2023-11-09T03:48:57.434897+0000","last_clean":"2023-11-09T03:48:57.434897+0000","last_became_active":"2023-11-09T03:48:57.047624+0000","last_became_peered":"2023-11-09T03:48:57.047624+0000","last_unstale":"2023-11-09T03:48:57.434897+0000","last_undegraded":"2023-11-09T03:48:57.434897+0000","last_fullsized":"2023-11-09T03:48:57.434897+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:58:16.010165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.040582+0000","last_change":"2023-11-09T03:48:01.449406+0000","last_active":"2023-11-09T03:48:57.040582+0000","last_peered":"2023-11-09T03:48:57.040582+0000","last_clean":"2023-11-09T03:48:57.040582+0000","last_became_active":"2023-11-09T03:48:01.449130+0000","last_became_peered":"2023-11-09T03:48:01.449130+0000","last_unstale":"2023-11-09T03:48:57.040582+0000","last_undegraded":"2023-11-09T03:48:57.040582+0000","last_fullsized":"2023-11-09T03:48:57.040582+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:20:36.303940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.033343+0000","last_change":"2023-11-09T03:48:02.441801+0000","last_active":"2023-11-09T03:48:57.033343+0000","last_peered":"2023-11-09T03:48:57.033343+0000","last_clean":"2023-11-09T03:48:57.033343+0000","last_became_active":"2023-11-09T03:48:00.444975+0000","last_became_peered":"2023-11-09T03:48:00.444975+0000","last_unstale":"2023-11-09T03:48:57.033343+0000","last_undegraded":"2023-11-09T03:48:57.033343+0000","last_fullsized":"2023-11-09T03:48:57.033343+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:39:31.347349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061165099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":57,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.049538+0000","last_change":"2023-11-09T03:47:03.147367+0000","last_active":"2023-11-09T03:48:57.049538+0000","last_peered":"2023-11-09T03:48:57.049538+0000","last_clean":"2023-11-09T03:48:57.049538+0000","last_became_active":"2023-11-09T03:47:01.588359+0000","last_became_peered":"2023-11-09T03:47:01.588359+0000","last_unstale":"2023-11-09T03:48:57.049538+0000","last_undegraded":"2023-11-09T03:48:57.049538+0000","last_fullsized":"2023-11-09T03:48:57.049538+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:21:34.303068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014106660000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.447677+0000","last_change":"2023-11-09T03:48:57.440700+0000","last_active":"2023-11-09T03:48:57.447677+0000","last_peered":"2023-11-09T03:48:57.447677+0000","last_clean":"2023-11-09T03:48:57.447677+0000","last_became_active":"2023-11-09T03:48:57.440436+0000","last_became_peered":"2023-11-09T03:48:57.440436+0000","last_unstale":"2023-11-09T03:48:57.447677+0000","last_undegraded":"2023-11-09T03:48:57.447677+0000","last_fullsized":"2023-11-09T03:48:57.447677+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:44:32.089939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.434741+0000","last_change":"2023-11-09T03:48:57.051186+0000","last_active":"2023-11-09T03:48:57.434741+0000","last_peered":"2023-11-09T03:48:57.434741+0000","last_clean":"2023-11-09T03:48:57.434741+0000","last_became_active":"2023-11-09T03:48:57.050935+0000","last_became_peered":"2023-11-09T03:48:57.050935+0000","last_unstale":"2023-11-09T03:48:57.434741+0000","last_undegraded":"2023-11-09T03:48:57.434741+0000","last_fullsized":"2023-11-09T03:48:57.434741+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:28:06.094368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.042287+0000","last_change":"2023-11-09T03:48:01.451866+0000","last_active":"2023-11-09T03:48:57.042287+0000","last_peered":"2023-11-09T03:48:57.042287+0000","last_clean":"2023-11-09T03:48:57.042287+0000","last_became_active":"2023-11-09T03:48:01.451350+0000","last_became_peered":"2023-11-09T03:48:01.451350+0000","last_unstale":"2023-11-09T03:48:57.042287+0000","last_undegraded":"2023-11-09T03:48:57.042287+0000","last_fullsized":"2023-11-09T03:48:57.042287+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:36.735029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036657+0000","last_change":"2023-11-09T03:48:02.441648+0000","last_active":"2023-11-09T03:48:57.036657+0000","last_peered":"2023-11-09T03:48:57.036657+0000","last_clean":"2023-11-09T03:48:57.036657+0000","last_became_active":"2023-11-09T03:48:00.448541+0000","last_became_peered":"2023-11-09T03:48:00.448541+0000","last_unstale":"2023-11-09T03:48:57.036657+0000","last_undegraded":"2023-11-09T03:48:57.036657+0000","last_fullsized":"2023-11-09T03:48:57.036657+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:35.729023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036727+0000","last_change":"2023-11-09T03:47:02.581682+0000","last_active":"2023-11-09T03:48:57.036727+0000","last_peered":"2023-11-09T03:48:57.036727+0000","last_clean":"2023-11-09T03:48:57.036727+0000","last_became_active":"2023-11-09T03:47:01.593051+0000","last_became_peered":"2023-11-09T03:47:01.593051+0000","last_unstale":"2023-11-09T03:48:57.036727+0000","last_undegraded":"2023-11-09T03:48:57.036727+0000","last_fullsized":"2023-11-09T03:48:57.036727+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047277799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.446756+0000","last_change":"2023-11-09T03:48:57.443729+0000","last_active":"2023-11-09T03:48:57.446756+0000","last_peered":"2023-11-09T03:48:57.446756+0000","last_clean":"2023-11-09T03:48:57.446756+0000","last_became_active":"2023-11-09T03:48:57.443483+0000","last_became_peered":"2023-11-09T03:48:57.443483+0000","last_unstale":"2023-11-09T03:48:57.446756+0000","last_undegraded":"2023-11-09T03:48:57.446756+0000","last_fullsized":"2023-11-09T03:48:57.446756+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:21:08.155680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.433446+0000","last_change":"2023-11-09T03:48:57.055649+0000","last_active":"2023-11-09T03:48:57.433446+0000","last_peered":"2023-11-09T03:48:57.433446+0000","last_clean":"2023-11-09T03:48:57.433446+0000","last_became_active":"2023-11-09T03:48:57.055366+0000","last_became_peered":"2023-11-09T03:48:57.055366+0000","last_unstale":"2023-11-09T03:48:57.433446+0000","last_undegraded":"2023-11-09T03:48:57.433446+0000","last_fullsized":"2023-11-09T03:48:57.433446+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:56:52.342978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.043544+0000","last_change":"2023-11-09T03:48:01.455784+0000","last_active":"2023-11-09T03:48:57.043544+0000","last_peered":"2023-11-09T03:48:57.043544+0000","last_clean":"2023-11-09T03:48:57.043544+0000","last_became_active":"2023-11-09T03:48:01.455532+0000","last_became_peered":"2023-11-09T03:48:01.455532+0000","last_unstale":"2023-11-09T03:48:57.043544+0000","last_undegraded":"2023-11-09T03:48:57.043544+0000","last_fullsized":"2023-11-09T03:48:57.043544+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:21:23.156761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.042520+0000","last_change":"2023-11-09T03:48:02.611298+0000","last_active":"2023-11-09T03:48:57.042520+0000","last_peered":"2023-11-09T03:48:57.042520+0000","last_clean":"2023-11-09T03:48:57.042520+0000","last_became_active":"2023-11-09T03:48:00.441261+0000","last_became_peered":"2023-11-09T03:48:00.441261+0000","last_unstale":"2023-11-09T03:48:57.042520+0000","last_undegraded":"2023-11-09T03:48:57.042520+0000","last_fullsized":"2023-11-09T03:48:57.042520+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:50:55.299235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081460199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036362+0000","last_change":"2023-11-09T03:47:03.146309+0000","last_active":"2023-11-09T03:48:57.036362+0000","last_peered":"2023-11-09T03:48:57.036362+0000","last_clean":"2023-11-09T03:48:57.036362+0000","last_became_active":"2023-11-09T03:47:01.587560+0000","last_became_peered":"2023-11-09T03:47:01.587560+0000","last_unstale":"2023-11-09T03:48:57.036362+0000","last_undegraded":"2023-11-09T03:48:57.036362+0000","last_fullsized":"2023-11-09T03:48:57.036362+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:27:55.271945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070988400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.442204+0000","last_change":"2023-11-09T03:48:57.435551+0000","last_active":"2023-11-09T03:48:57.442204+0000","last_peered":"2023-11-09T03:48:57.442204+0000","last_clean":"2023-11-09T03:48:57.442204+0000","last_became_active":"2023-11-09T03:48:57.435303+0000","last_became_peered":"2023-11-09T03:48:57.435303+0000","last_unstale":"2023-11-09T03:48:57.442204+0000","last_undegraded":"2023-11-09T03:48:57.442204+0000","last_fullsized":"2023-11-09T03:48:57.442204+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:01:29.460175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.436518+0000","last_change":"2023-11-09T03:48:57.050746+0000","last_active":"2023-11-09T03:48:57.436518+0000","last_peered":"2023-11-09T03:48:57.436518+0000","last_clean":"2023-11-09T03:48:57.436518+0000","last_became_active":"2023-11-09T03:48:57.050448+0000","last_became_peered":"2023-11-09T03:48:57.050448+0000","last_unstale":"2023-11-09T03:48:57.436518+0000","last_undegraded":"2023-11-09T03:48:57.436518+0000","last_fullsized":"2023-11-09T03:48:57.436518+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T11:25:45.917439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.048037+0000","last_change":"2023-11-09T03:48:01.452477+0000","last_active":"2023-11-09T03:48:57.048037+0000","last_peered":"2023-11-09T03:48:57.048037+0000","last_clean":"2023-11-09T03:48:57.048037+0000","last_became_active":"2023-11-09T03:48:01.452186+0000","last_became_peered":"2023-11-09T03:48:01.452186+0000","last_unstale":"2023-11-09T03:48:57.048037+0000","last_undegraded":"2023-11-09T03:48:57.048037+0000","last_fullsized":"2023-11-09T03:48:57.048037+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:32:21.825778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.034003+0000","last_change":"2023-11-09T03:48:02.442101+0000","last_active":"2023-11-09T03:48:57.034003+0000","last_peered":"2023-11-09T03:48:57.034003+0000","last_clean":"2023-11-09T03:48:57.034003+0000","last_became_active":"2023-11-09T03:48:00.449354+0000","last_became_peered":"2023-11-09T03:48:00.449354+0000","last_unstale":"2023-11-09T03:48:57.034003+0000","last_undegraded":"2023-11-09T03:48:57.034003+0000","last_fullsized":"2023-11-09T03:48:57.034003+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:06:32.906582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093455000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.043974+0000","last_change":"2023-11-09T03:47:02.725125+0000","last_active":"2023-11-09T03:48:57.043974+0000","last_peered":"2023-11-09T03:48:57.043974+0000","last_clean":"2023-11-09T03:48:57.043974+0000","last_became_active":"2023-11-09T03:47:01.591521+0000","last_became_peered":"2023-11-09T03:47:01.591521+0000","last_unstale":"2023-11-09T03:48:57.043974+0000","last_undegraded":"2023-11-09T03:48:57.043974+0000","last_fullsized":"2023-11-09T03:48:57.043974+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:52:40.076542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000758675,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.443753+0000","last_change":"2023-11-09T03:48:57.441295+0000","last_active":"2023-11-09T03:48:57.443753+0000","last_peered":"2023-11-09T03:48:57.443753+0000","last_clean":"2023-11-09T03:48:57.443753+0000","last_became_active":"2023-11-09T03:48:57.441049+0000","last_became_peered":"2023-11-09T03:48:57.441049+0000","last_unstale":"2023-11-09T03:48:57.443753+0000","last_undegraded":"2023-11-09T03:48:57.443753+0000","last_fullsized":"2023-11-09T03:48:57.443753+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:02:36.826029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.430975+0000","last_change":"2023-11-09T03:48:57.045457+0000","last_active":"2023-11-09T03:48:57.430975+0000","last_peered":"2023-11-09T03:48:57.430975+0000","last_clean":"2023-11-09T03:48:57.430975+0000","last_became_active":"2023-11-09T03:48:57.045213+0000","last_became_peered":"2023-11-09T03:48:57.045213+0000","last_unstale":"2023-11-09T03:48:57.430975+0000","last_undegraded":"2023-11-09T03:48:57.430975+0000","last_fullsized":"2023-11-09T03:48:57.430975+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:01:10.734137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.043341+0000","last_change":"2023-11-09T03:48:01.448129+0000","last_active":"2023-11-09T03:48:57.043341+0000","last_peered":"2023-11-09T03:48:57.043341+0000","last_clean":"2023-11-09T03:48:57.043341+0000","last_became_active":"2023-11-09T03:48:01.447844+0000","last_became_peered":"2023-11-09T03:48:01.447844+0000","last_unstale":"2023-11-09T03:48:57.043341+0000","last_undegraded":"2023-11-09T03:48:57.043341+0000","last_fullsized":"2023-11-09T03:48:57.043341+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:54:52.433604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.048068+0000","last_change":"2023-11-09T03:48:02.612450+0000","last_active":"2023-11-09T03:48:57.048068+0000","last_peered":"2023-11-09T03:48:57.048068+0000","last_clean":"2023-11-09T03:48:57.048068+0000","last_became_active":"2023-11-09T03:48:00.441855+0000","last_became_peered":"2023-11-09T03:48:00.441855+0000","last_unstale":"2023-11-09T03:48:57.048068+0000","last_undegraded":"2023-11-09T03:48:57.048068+0000","last_fullsized":"2023-11-09T03:48:57.048068+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T09:53:42.763434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095368899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.033862+0000","last_change":"2023-11-09T03:47:03.147289+0000","last_active":"2023-11-09T03:48:57.033862+0000","last_peered":"2023-11-09T03:48:57.033862+0000","last_clean":"2023-11-09T03:48:57.033862+0000","last_became_active":"2023-11-09T03:47:01.587784+0000","last_became_peered":"2023-11-09T03:47:01.587784+0000","last_unstale":"2023-11-09T03:48:57.033862+0000","last_undegraded":"2023-11-09T03:48:57.033862+0000","last_fullsized":"2023-11-09T03:48:57.033862+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:11:23.824112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038781299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.436337+0000","last_change":"2023-11-09T03:48:57.431983+0000","last_active":"2023-11-09T03:48:57.436337+0000","last_peered":"2023-11-09T03:48:57.436337+0000","last_clean":"2023-11-09T03:48:57.436337+0000","last_became_active":"2023-11-09T03:48:57.431716+0000","last_became_peered":"2023-11-09T03:48:57.431716+0000","last_unstale":"2023-11-09T03:48:57.436337+0000","last_undegraded":"2023-11-09T03:48:57.436337+0000","last_fullsized":"2023-11-09T03:48:57.436337+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:50:34.258596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.434937+0000","last_change":"2023-11-09T03:48:57.053092+0000","last_active":"2023-11-09T03:48:57.434937+0000","last_peered":"2023-11-09T03:48:57.434937+0000","last_clean":"2023-11-09T03:48:57.434937+0000","last_became_active":"2023-11-09T03:48:57.052820+0000","last_became_peered":"2023-11-09T03:48:57.052820+0000","last_unstale":"2023-11-09T03:48:57.434937+0000","last_undegraded":"2023-11-09T03:48:57.434937+0000","last_fullsized":"2023-11-09T03:48:57.434937+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:07:10.450500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.037169+0000","last_change":"2023-11-09T03:48:01.449617+0000","last_active":"2023-11-09T03:48:57.037169+0000","last_peered":"2023-11-09T03:48:57.037169+0000","last_clean":"2023-11-09T03:48:57.037169+0000","last_became_active":"2023-11-09T03:48:01.449328+0000","last_became_peered":"2023-11-09T03:48:01.449328+0000","last_unstale":"2023-11-09T03:48:57.037169+0000","last_undegraded":"2023-11-09T03:48:57.037169+0000","last_fullsized":"2023-11-09T03:48:57.037169+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:51:18.760059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.033640+0000","last_change":"2023-11-09T03:48:02.442215+0000","last_active":"2023-11-09T03:48:57.033640+0000","last_peered":"2023-11-09T03:48:57.033640+0000","last_clean":"2023-11-09T03:48:57.033640+0000","last_became_active":"2023-11-09T03:48:00.444720+0000","last_became_peered":"2023-11-09T03:48:00.444720+0000","last_unstale":"2023-11-09T03:48:57.033640+0000","last_undegraded":"2023-11-09T03:48:57.033640+0000","last_fullsized":"2023-11-09T03:48:57.033640+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:55:55.735668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001407176,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.048221+0000","last_change":"2023-11-09T03:47:02.619346+0000","last_active":"2023-11-09T03:48:57.048221+0000","last_peered":"2023-11-09T03:48:57.048221+0000","last_clean":"2023-11-09T03:48:57.048221+0000","last_became_active":"2023-11-09T03:47:01.590389+0000","last_became_peered":"2023-11-09T03:47:01.590389+0000","last_unstale":"2023-11-09T03:48:57.048221+0000","last_undegraded":"2023-11-09T03:48:57.048221+0000","last_fullsized":"2023-11-09T03:48:57.048221+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:38:04.048012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000384072,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.451137+0000","last_change":"2023-11-09T03:48:57.445676+0000","last_active":"2023-11-09T03:48:57.451137+0000","last_peered":"2023-11-09T03:48:57.451137+0000","last_clean":"2023-11-09T03:48:57.451137+0000","last_became_active":"2023-11-09T03:48:57.445349+0000","last_became_peered":"2023-11-09T03:48:57.445349+0000","last_unstale":"2023-11-09T03:48:57.451137+0000","last_undegraded":"2023-11-09T03:48:57.451137+0000","last_fullsized":"2023-11-09T03:48:57.451137+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:44:32.408986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.430067+0000","last_change":"2023-11-09T03:48:57.052325+0000","last_active":"2023-11-09T03:48:57.430067+0000","last_peered":"2023-11-09T03:48:57.430067+0000","last_clean":"2023-11-09T03:48:57.430067+0000","last_became_active":"2023-11-09T03:48:57.051915+0000","last_became_peered":"2023-11-09T03:48:57.051915+0000","last_unstale":"2023-11-09T03:48:57.430067+0000","last_undegraded":"2023-11-09T03:48:57.430067+0000","last_fullsized":"2023-11-09T03:48:57.430067+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:14:36.990468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.041871+0000","last_change":"2023-11-09T03:48:01.450018+0000","last_active":"2023-11-09T03:48:57.041871+0000","last_peered":"2023-11-09T03:48:57.041871+0000","last_clean":"2023-11-09T03:48:57.041871+0000","last_became_active":"2023-11-09T03:48:01.449749+0000","last_became_peered":"2023-11-09T03:48:01.449749+0000","last_unstale":"2023-11-09T03:48:57.041871+0000","last_undegraded":"2023-11-09T03:48:57.041871+0000","last_fullsized":"2023-11-09T03:48:57.041871+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:21:23.156761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.041011+0000","last_change":"2023-11-09T03:48:02.524216+0000","last_active":"2023-11-09T03:48:57.041011+0000","last_peered":"2023-11-09T03:48:57.041011+0000","last_clean":"2023-11-09T03:48:57.041011+0000","last_became_active":"2023-11-09T03:48:00.446972+0000","last_became_peered":"2023-11-09T03:48:00.446972+0000","last_unstale":"2023-11-09T03:48:57.041011+0000","last_undegraded":"2023-11-09T03:48:57.041011+0000","last_fullsized":"2023-11-09T03:48:57.041011+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:54:07.987773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000825255,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":63,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036699+0000","last_change":"2023-11-09T03:47:02.582393+0000","last_active":"2023-11-09T03:48:57.036699+0000","last_peered":"2023-11-09T03:48:57.036699+0000","last_clean":"2023-11-09T03:48:57.036699+0000","last_became_active":"2023-11-09T03:47:01.591752+0000","last_became_peered":"2023-11-09T03:47:01.591752+0000","last_unstale":"2023-11-09T03:48:57.036699+0000","last_undegraded":"2023-11-09T03:48:57.036699+0000","last_fullsized":"2023-11-09T03:48:57.036699+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T10:16:03.025119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067021599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.445270+0000","last_change":"2023-11-09T03:48:57.440690+0000","last_active":"2023-11-09T03:48:57.445270+0000","last_peered":"2023-11-09T03:48:57.445270+0000","last_clean":"2023-11-09T03:48:57.445270+0000","last_became_active":"2023-11-09T03:48:57.440425+0000","last_became_peered":"2023-11-09T03:48:57.440425+0000","last_unstale":"2023-11-09T03:48:57.445270+0000","last_undegraded":"2023-11-09T03:48:57.445270+0000","last_fullsized":"2023-11-09T03:48:57.445270+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:37:25.942349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.433424+0000","last_change":"2023-11-09T03:48:57.055564+0000","last_active":"2023-11-09T03:48:57.433424+0000","last_peered":"2023-11-09T03:48:57.433424+0000","last_clean":"2023-11-09T03:48:57.433424+0000","last_became_active":"2023-11-09T03:48:57.055276+0000","last_became_peered":"2023-11-09T03:48:57.055276+0000","last_unstale":"2023-11-09T03:48:57.433424+0000","last_undegraded":"2023-11-09T03:48:57.433424+0000","last_fullsized":"2023-11-09T03:48:57.433424+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:42:18.929121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.041083+0000","last_change":"2023-11-09T03:48:01.451462+0000","last_active":"2023-11-09T03:48:57.041083+0000","last_peered":"2023-11-09T03:48:57.041083+0000","last_clean":"2023-11-09T03:48:57.041083+0000","last_became_active":"2023-11-09T03:48:01.451188+0000","last_became_peered":"2023-11-09T03:48:01.451188+0000","last_unstale":"2023-11-09T03:48:57.041083+0000","last_undegraded":"2023-11-09T03:48:57.041083+0000","last_fullsized":"2023-11-09T03:48:57.041083+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:28.631801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036536+0000","last_change":"2023-11-09T03:48:02.682290+0000","last_active":"2023-11-09T03:48:57.036536+0000","last_peered":"2023-11-09T03:48:57.036536+0000","last_clean":"2023-11-09T03:48:57.036536+0000","last_became_active":"2023-11-09T03:48:00.444246+0000","last_became_peered":"2023-11-09T03:48:00.444246+0000","last_unstale":"2023-11-09T03:48:57.036536+0000","last_undegraded":"2023-11-09T03:48:57.036536+0000","last_fullsized":"2023-11-09T03:48:57.036536+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:09.487058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000423359,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":57,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.040154+0000","last_change":"2023-11-09T03:47:03.028744+0000","last_active":"2023-11-09T03:48:57.040154+0000","last_peered":"2023-11-09T03:48:57.040154+0000","last_clean":"2023-11-09T03:48:57.040154+0000","last_became_active":"2023-11-09T03:47:01.593890+0000","last_became_peered":"2023-11-09T03:47:01.593890+0000","last_unstale":"2023-11-09T03:48:57.040154+0000","last_undegraded":"2023-11-09T03:48:57.040154+0000","last_fullsized":"2023-11-09T03:48:57.040154+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:05.754357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053008299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.451206+0000","last_change":"2023-11-09T03:48:57.448257+0000","last_active":"2023-11-09T03:48:57.451206+0000","last_peered":"2023-11-09T03:48:57.451206+0000","last_clean":"2023-11-09T03:48:57.451206+0000","last_became_active":"2023-11-09T03:48:57.448007+0000","last_became_peered":"2023-11-09T03:48:57.448007+0000","last_unstale":"2023-11-09T03:48:57.451206+0000","last_undegraded":"2023-11-09T03:48:57.451206+0000","last_fullsized":"2023-11-09T03:48:57.451206+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T12:37:27.085476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.426070+0000","last_change":"2023-11-09T03:48:57.046917+0000","last_active":"2023-11-09T03:48:57.426070+0000","last_peered":"2023-11-09T03:48:57.426070+0000","last_clean":"2023-11-09T03:48:57.426070+0000","last_became_active":"2023-11-09T03:48:57.046465+0000","last_became_peered":"2023-11-09T03:48:57.046465+0000","last_unstale":"2023-11-09T03:48:57.426070+0000","last_undegraded":"2023-11-09T03:48:57.426070+0000","last_fullsized":"2023-11-09T03:48:57.426070+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:20:44.093352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.040104+0000","last_change":"2023-11-09T03:48:01.450144+0000","last_active":"2023-11-09T03:48:57.040104+0000","last_peered":"2023-11-09T03:48:57.040104+0000","last_clean":"2023-11-09T03:48:57.040104+0000","last_became_active":"2023-11-09T03:48:01.449859+0000","last_became_peered":"2023-11-09T03:48:01.449859+0000","last_unstale":"2023-11-09T03:48:57.040104+0000","last_undegraded":"2023-11-09T03:48:57.040104+0000","last_fullsized":"2023-11-09T03:48:57.040104+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:57:56.855869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036399+0000","last_change":"2023-11-09T03:48:02.681869+0000","last_active":"2023-11-09T03:48:57.036399+0000","last_peered":"2023-11-09T03:48:57.036399+0000","last_clean":"2023-11-09T03:48:57.036399+0000","last_became_active":"2023-11-09T03:48:00.444928+0000","last_became_peered":"2023-11-09T03:48:00.444928+0000","last_unstale":"2023-11-09T03:48:57.036399+0000","last_undegraded":"2023-11-09T03:48:57.036399+0000","last_fullsized":"2023-11-09T03:48:57.036399+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:59:58.695636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044822599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036609+0000","last_change":"2023-11-09T03:47:03.146783+0000","last_active":"2023-11-09T03:48:57.036609+0000","last_peered":"2023-11-09T03:48:57.036609+0000","last_clean":"2023-11-09T03:48:57.036609+0000","last_became_active":"2023-11-09T03:47:01.586923+0000","last_became_peered":"2023-11-09T03:47:01.586923+0000","last_unstale":"2023-11-09T03:48:57.036609+0000","last_undegraded":"2023-11-09T03:48:57.036609+0000","last_fullsized":"2023-11-09T03:48:57.036609+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T13:37:50.326442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060384700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.441714+0000","last_change":"2023-11-09T03:48:57.432183+0000","last_active":"2023-11-09T03:48:57.441714+0000","last_peered":"2023-11-09T03:48:57.441714+0000","last_clean":"2023-11-09T03:48:57.441714+0000","last_became_active":"2023-11-09T03:48:57.431684+0000","last_became_peered":"2023-11-09T03:48:57.431684+0000","last_unstale":"2023-11-09T03:48:57.441714+0000","last_undegraded":"2023-11-09T03:48:57.441714+0000","last_fullsized":"2023-11-09T03:48:57.441714+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T15:30:30.967923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.425579+0000","last_change":"2023-11-09T03:48:57.040734+0000","last_active":"2023-11-09T03:48:57.425579+0000","last_peered":"2023-11-09T03:48:57.425579+0000","last_clean":"2023-11-09T03:48:57.425579+0000","last_became_active":"2023-11-09T03:48:57.040161+0000","last_became_peered":"2023-11-09T03:48:57.040161+0000","last_unstale":"2023-11-09T03:48:57.425579+0000","last_undegraded":"2023-11-09T03:48:57.425579+0000","last_fullsized":"2023-11-09T03:48:57.425579+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:37.857562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.049439+0000","last_change":"2023-11-09T03:48:01.448383+0000","last_active":"2023-11-09T03:48:57.049439+0000","last_peered":"2023-11-09T03:48:57.049439+0000","last_clean":"2023-11-09T03:48:57.049439+0000","last_became_active":"2023-11-09T03:48:01.448050+0000","last_became_peered":"2023-11-09T03:48:01.448050+0000","last_unstale":"2023-11-09T03:48:57.049439+0000","last_undegraded":"2023-11-09T03:48:57.049439+0000","last_fullsized":"2023-11-09T03:48:57.049439+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:53:44.478182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.040656+0000","last_change":"2023-11-09T03:48:02.523794+0000","last_active":"2023-11-09T03:48:57.040656+0000","last_peered":"2023-11-09T03:48:57.040656+0000","last_clean":"2023-11-09T03:48:57.040656+0000","last_became_active":"2023-11-09T03:48:00.440255+0000","last_became_peered":"2023-11-09T03:48:00.440255+0000","last_unstale":"2023-11-09T03:48:57.040656+0000","last_undegraded":"2023-11-09T03:48:57.040656+0000","last_fullsized":"2023-11-09T03:48:57.040656+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:58:02.392696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064572299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.036384+0000","last_change":"2023-11-09T03:47:03.146580+0000","last_active":"2023-11-09T03:48:57.036384+0000","last_peered":"2023-11-09T03:48:57.036384+0000","last_clean":"2023-11-09T03:48:57.036384+0000","last_became_active":"2023-11-09T03:47:01.589201+0000","last_became_peered":"2023-11-09T03:47:01.589201+0000","last_unstale":"2023-11-09T03:48:57.036384+0000","last_undegraded":"2023-11-09T03:48:57.036384+0000","last_fullsized":"2023-11-09T03:48:57.036384+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:14.300421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049706999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.443842+0000","last_change":"2023-11-09T03:48:57.436623+0000","last_active":"2023-11-09T03:48:57.443842+0000","last_peered":"2023-11-09T03:48:57.443842+0000","last_clean":"2023-11-09T03:48:57.443842+0000","last_became_active":"2023-11-09T03:48:57.436370+0000","last_became_peered":"2023-11-09T03:48:57.436370+0000","last_unstale":"2023-11-09T03:48:57.443842+0000","last_undegraded":"2023-11-09T03:48:57.443842+0000","last_fullsized":"2023-11-09T03:48:57.443842+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:57.028994+0000","last_clean_scrub_stamp":"2023-11-09T03:48:57.028994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:56:57.961469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.434494+0000","last_change":"2023-11-09T03:48:57.050983+0000","last_active":"2023-11-09T03:48:57.434494+0000","last_peered":"2023-11-09T03:48:57.434494+0000","last_clean":"2023-11-09T03:48:57.434494+0000","last_became_active":"2023-11-09T03:48:57.050070+0000","last_became_peered":"2023-11-09T03:48:57.050070+0000","last_unstale":"2023-11-09T03:48:57.434494+0000","last_undegraded":"2023-11-09T03:48:57.434494+0000","last_fullsized":"2023-11-09T03:48:57.434494+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:56.024035+0000","last_clean_scrub_stamp":"2023-11-09T03:48:56.024035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:23:59.729104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.035953+0000","last_change":"2023-11-09T03:48:01.451284+0000","last_active":"2023-11-09T03:48:57.035953+0000","last_peered":"2023-11-09T03:48:57.035953+0000","last_clean":"2023-11-09T03:48:57.035953+0000","last_became_active":"2023-11-09T03:48:01.450954+0000","last_became_peered":"2023-11-09T03:48:01.450954+0000","last_unstale":"2023-11-09T03:48:57.035953+0000","last_undegraded":"2023-11-09T03:48:57.035953+0000","last_fullsized":"2023-11-09T03:48:57.035953+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:48:00.422731+0000","last_clean_scrub_stamp":"2023-11-09T03:48:00.422731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:49:24.680020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.043771+0000","last_change":"2023-11-09T03:48:02.733038+0000","last_active":"2023-11-09T03:48:57.043771+0000","last_peered":"2023-11-09T03:48:57.043771+0000","last_clean":"2023-11-09T03:48:57.043771+0000","last_became_active":"2023-11-09T03:48:00.446661+0000","last_became_peered":"2023-11-09T03:48:00.446661+0000","last_unstale":"2023-11-09T03:48:57.043771+0000","last_undegraded":"2023-11-09T03:48:57.043771+0000","last_fullsized":"2023-11-09T03:48:57.043771+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:59.416725+0000","last_clean_scrub_stamp":"2023-11-09T03:47:59.416725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:05:08.649312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040102300000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.034130+0000","last_change":"2023-11-09T03:47:03.147851+0000","last_active":"2023-11-09T03:48:57.034130+0000","last_peered":"2023-11-09T03:48:57.034130+0000","last_clean":"2023-11-09T03:48:57.034130+0000","last_became_active":"2023-11-09T03:47:01.589893+0000","last_became_peered":"2023-11-09T03:47:01.589893+0000","last_unstale":"2023-11-09T03:48:57.034130+0000","last_undegraded":"2023-11-09T03:48:57.034130+0000","last_fullsized":"2023-11-09T03:48:57.034130+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:47:00.564895+0000","last_clean_scrub_stamp":"2023-11-09T03:47:00.564895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T14:08:17.183462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029151399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":226,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-09T03:48:57.041182+0000","last_change":"2023-11-09T03:45:21.030926+0000","last_active":"2023-11-09T03:48:57.041182+0000","last_peered":"2023-11-09T03:48:57.041182+0000","last_clean":"2023-11-09T03:48:57.041182+0000","last_became_active":"2023-11-09T03:45:21.029801+0000","last_became_peered":"2023-11-09T03:45:21.029801+0000","last_unstale":"2023-11-09T03:48:57.041182+0000","last_undegraded":"2023-11-09T03:48:57.041182+0000","last_fullsized":"2023-11-09T03:48:57.041182+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-09T03:45:19.768433+0000","last_clean_scrub_stamp":"2023-11-09T03:45:19.768433+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T08:12:59.153713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574895,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32872,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338968,"statfs":{"total":96636764160,"available":96603103232,"internally_reserved":0,"allocated":1728512,"data_stored":1145248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.503,"15min":0.499},"min":{"1min":0.36599999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53200000000000003,"15min":0.52100000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.66700000000000004,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.624}]},{"osd":1,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.38600000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70599999999999996,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53000000000000003,"15min":0.51900000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.65600000000000003}]},{"osd":2,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52700000000000002,"15min":0.51500000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72099999999999997,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.755},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.65400000000000003,"15min":0.69699999999999995},"min":{"1min":0.41599999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.73299999999999998,"5min":4.9139999999999997,"15min":4.9139999999999997},"last":0.72499999999999998}]},{"osd":3,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.52200000000000002,"15min":0.51300000000000001},"min":{"1min":0.42599999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.67300000000000004,"5min":0.754,"15min":0.754},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56399999999999995,"15min":0.54500000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81499999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.70999999999999996}]},{"osd":4,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.45500000000000002,"15min":0.44400000000000001},"min":{"1min":0.32000000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.5,"15min":0.48199999999999998},"min":{"1min":0.32300000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.73199999999999998,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.57699999999999996}]},{"osd":5,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.48499999999999999,"15min":0.46000000000000002},"min":{"1min":0.30599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76300000000000001,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51900000000000002,"15min":0.499},"min":{"1min":0.32700000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.79700000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.55700000000000005}]},{"osd":6,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55700000000000005,"15min":0.54300000000000004},"min":{"1min":0.38900000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.74299999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54000000000000004,"15min":0.52200000000000002},"min":{"1min":0.40400000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.64100000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1159168,"data_stored":570752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.504,"15min":0.503},"min":{"1min":0.34100000000000003,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.65700000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51400000000000001,"15min":0.498},"min":{"1min":0.36699999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73199999999999998,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.755}]},{"osd":1,"last update":"Thu Nov 9 03:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52200000000000002,"15min":0.51200000000000001},"min":{"1min":0.35899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.497,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.39300000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.65300000000000002,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.71499999999999997}]},{"osd":2,"last update":"Thu Nov 9 03:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.42199999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.65400000000000003,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52900000000000003,"15min":0.52100000000000002},"min":{"1min":0.39900000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.70399999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.72699999999999998}]},{"osd":3,"last update":"Thu Nov 9 03:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52500000000000002,"15min":0.52400000000000002},"min":{"1min":0.39100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.42799999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.68400000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.59899999999999998}]},{"osd":4,"last update":"Thu Nov 9 03:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.497,"15min":0.501},"min":{"1min":0.32000000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.66700000000000004,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53200000000000003,"15min":0.52900000000000003},"min":{"1min":0.34599999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.76900000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.44400000000000001}]},{"osd":5,"last update":"Thu Nov 9 03:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57999999999999996,"15min":0.57699999999999996},"min":{"1min":0.39500000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80800000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55000000000000004,"15min":0.55600000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.78800000000000003,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46600000000000003}]},{"osd":7,"last update":"Thu Nov 9 03:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.45200000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83199999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.498},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56699999999999995,"15min":0.55800000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71499999999999997,"5min":0.877,"15min":0.877},"last":0.48599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574895,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1134592,"data_stored":554880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.504,"15min":0.48899999999999999},"min":{"1min":0.36199999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.376,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.51400000000000001}]},{"osd":1,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88300000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.36599999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.84199999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.46800000000000003}]},{"osd":2,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.45800000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.55600000000000005},"min":{"1min":0.40300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.52800000000000002}]},{"osd":3,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57099999999999995,"15min":0.56499999999999995},"min":{"1min":0.39400000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.436},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55200000000000005,"15min":0.55100000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.53900000000000003}]},{"osd":4,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.49199999999999999,"15min":0.48399999999999999},"min":{"1min":0.32400000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.48999999999999999,"15min":0.48299999999999998},"min":{"1min":0.29699999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.32300000000000001}]},{"osd":6,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.50600000000000001,"15min":0.51000000000000001},"min":{"1min":0.33700000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.755,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.53300000000000003,"15min":0.51700000000000002},"min":{"1min":0.38300000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.42099999999999999}]},{"osd":7,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.34699999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.81200000000000006,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.498},{"interface":"front","average":{"1min":0.621,"5min":0.61899999999999999,"15min":0.62},"min":{"1min":0.432,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84899999999999998,"5min":1.01,"15min":1.01},"last":0.45300000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1134592,"data_stored":554880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69399999999999995,"5min":1.034,"15min":1.034},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.38800000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78300000000000003,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.54900000000000004}]},{"osd":1,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.34000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.74299999999999999,"5min":1.093,"15min":1.093},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54200000000000004,"15min":0.54100000000000004},"min":{"1min":0.36399999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71899999999999997,"5min":0.871,"15min":0.871},"last":0.629}]},{"osd":2,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54100000000000004,"15min":0.55400000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.69199999999999995,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.442},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54800000000000004,"15min":0.54500000000000004},"min":{"1min":0.39700000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.53600000000000003}]},{"osd":3,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.433,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.70199999999999996,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.376,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72499999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.57099999999999995}]},{"osd":5,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.33300000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.78700000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.318,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81100000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.41399999999999998}]},{"osd":6,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.75800000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57899999999999996,"15min":0.58399999999999996},"min":{"1min":0.39600000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.75900000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.47799999999999998}]},{"osd":7,"last update":"Thu Nov 9 03:48:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61599999999999999,"15min":0.629},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83499999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61099999999999999,"15min":0.61899999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79300000000000004,"5min":1.113,"15min":1.113},"last":0.51400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1146880,"data_stored":562566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.32100000000000001,"15min":0.32300000000000001},"min":{"1min":0.22,"5min":0.215,"15min":0.215},"max":{"1min":0.40200000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.313,"5min":0.32300000000000001,"15min":0.32400000000000001},"min":{"1min":0.23999999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.47799999999999998,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.26800000000000002}]},{"osd":1,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.32600000000000001,"15min":0.33000000000000002},"min":{"1min":0.23599999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.42899999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.36699999999999999,"15min":0.373},"min":{"1min":0.28100000000000003,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.47899999999999998,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.31}]},{"osd":2,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.37,"15min":0.372},"min":{"1min":0.27200000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.439,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.379,"15min":0.38300000000000001},"min":{"1min":0.29799999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.45700000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.33000000000000002}]},{"osd":4,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58899999999999997,"15min":0.59499999999999997},"min":{"1min":0.42299999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79000000000000004,"5min":0.87,"15min":0.87},"last":0.432},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.59699999999999998,"15min":0.61099999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.54600000000000004}]},{"osd":5,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.60499999999999998,"15min":0.62},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86599999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60699999999999998,"15min":0.61899999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.82099999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.46600000000000003}]},{"osd":6,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60999999999999999,"15min":0.623},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78200000000000003,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.45400000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.80300000000000005,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Nov 9 03:48:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.61099999999999999,"15min":0.62},"min":{"1min":0.41099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.70899999999999996,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.621,"15min":0.63700000000000001},"min":{"1min":0.46100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81299999999999994,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.52500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28160,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343680,"statfs":{"total":96636764160,"available":96607928320,"internally_reserved":0,"allocated":1159168,"data_stored":570752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:25 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.32900000000000001,"15min":0.33100000000000002},"min":{"1min":0.20999999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.42599999999999999,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.255},{"interface":"front","average":{"1min":0.31,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.23300000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.51400000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.35699999999999998}]},{"osd":1,"last update":"Thu Nov 9 03:48:25 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.32200000000000001,"15min":0.32500000000000001},"min":{"1min":0.249,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.46600000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.36399999999999999,"15min":0.36499999999999999},"min":{"1min":0.252,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.372}]},{"osd":3,"last update":"Thu Nov 9 03:48:25 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.35699999999999998,"15min":0.36299999999999999},"min":{"1min":0.25,"5min":0.22,"15min":0.22},"max":{"1min":0.54100000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.39600000000000002,"15min":0.40200000000000002},"min":{"1min":0.30299999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.495,"5min":0.754,"15min":0.754},"last":0.33900000000000002}]},{"osd":4,"last update":"Thu Nov 9 03:48:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.44800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.501}]},{"osd":5,"last update":"Thu Nov 9 03:48:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56599999999999995,"15min":0.57399999999999995},"min":{"1min":0.38,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.69799999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.434,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.49099999999999999}]},{"osd":6,"last update":"Thu Nov 9 03:48:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60599999999999998,"15min":0.60999999999999999},"min":{"1min":0.438,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59099999999999997,"15min":0.59199999999999997},"min":{"1min":0.436,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.51200000000000001}]},{"osd":7,"last update":"Thu Nov 9 03:48:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.61499999999999999,"15min":0.623},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.58699999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28128,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343712,"statfs":{"total":96636764160,"available":96607961088,"internally_reserved":0,"allocated":1134592,"data_stored":554880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 9 03:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.313,"15min":0.315},"min":{"1min":0.221,"5min":0.19,"15min":0.19},"max":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.29899999999999999,"5min":0.313,"15min":0.311},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.35899999999999999}]},{"osd":2,"last update":"Thu Nov 9 03:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.314,"15min":0.31},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.34399999999999997,"15min":0.34200000000000003},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.53300000000000003,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.38500000000000001}]},{"osd":3,"last update":"Thu Nov 9 03:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36699999999999999,"15min":0.371},"min":{"1min":0.26000000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.36399999999999999,"15min":0.36899999999999999},"min":{"1min":0.27100000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Nov 9 03:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.55400000000000005},"min":{"1min":0.40799999999999997,"5min":0.37,"15min":0.37},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56200000000000006,"15min":0.56100000000000005},"min":{"1min":0.41499999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.64700000000000002}]},{"osd":5,"last update":"Thu Nov 9 03:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.44400000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80500000000000005,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.55600000000000005}]},{"osd":6,"last update":"Thu Nov 9 03:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.41199999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.61699999999999999}]},{"osd":7,"last update":"Thu Nov 9 03:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59099999999999997,"15min":0.60399999999999998},"min":{"1min":0.41399999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.83099999999999996,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.59799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1736704,"data_stored":1152315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32000000000000001,"15min":0.32400000000000001},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.34599999999999997,"15min":0.36199999999999999},"min":{"1min":0.253,"5min":0.219,"15min":0.219},"max":{"1min":0.58299999999999996,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.435}]},{"osd":2,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35799999999999998,"15min":0.36399999999999999},"min":{"1min":0.24299999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.35999999999999999,"15min":0.36899999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.40799999999999997}]},{"osd":3,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.379,"15min":0.38900000000000001},"min":{"1min":0.27300000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.40699999999999997,"15min":0.41799999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.56599999999999995,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.49299999999999999}]},{"osd":4,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58999999999999997,"15min":0.59399999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.71899999999999997}]},{"osd":5,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.59199999999999997},"min":{"1min":0.378,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61499999999999999,"15min":0.63100000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80800000000000005,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.77600000000000002}]},{"osd":6,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63400000000000001,"15min":0.64500000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88300000000000001,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.624,"15min":0.63100000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85799999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.68799999999999994}]},{"osd":7,"last update":"Thu Nov 9 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61299999999999999,"15min":0.621},"min":{"1min":0.39800000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.42599999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.78800000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.75700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-09T03:49:01.349 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-09T03:49:01.349 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-09T03:49:01.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.488+0000 7f48d6244640 1 Processor -- start 2023-11-09T03:49:01.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.488+0000 7f48d6244640 1 -- start start 2023-11-09T03:49:01.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.489+0000 7f48d6244640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f48d015ad30 0x7f48d00adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:01.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.489+0000 7f48d6244640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d00ad730 0x7f48d0070710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:01.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.489+0000 7f48d6244640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d00c3900 con 0x7f48d0070c50 2023-11-09T03:49:01.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.489+0000 7f48d6244640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d00ae260 con 0x7f48d015ad30 2023-11-09T03:49:01.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.489+0000 7f48d6244640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d015f750 con 0x7f48d00ad730 2023-11-09T03:49:01.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.490+0000 7f48c6ffd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d00ad730 0x7f48d0070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:01.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.490+0000 7f48cf7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f48d015ad30 0x7f48d00adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:01.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.490+0000 7f48cf7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f48d015ad30 0x7f48d00adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:36772/0 (socket says 172.21.15.76:36772) 2023-11-09T03:49:01.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.490+0000 7f48c6ffd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d00ad730 0x7f48d0070710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:56444/0 (socket says 172.21.15.76:56444) 2023-11-09T03:49:01.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.490+0000 7f48cf7fe640 1 -- 172.21.15.76:0/3948520647 learned_addr learned my addr 172.21.15.76:0/3948520647 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:01.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.490+0000 7f48ceffd640 1 -- 172.21.15.76:0/3948520647 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 399584335 0 0) 0x7f48d00c3900 con 0x7f48d0070c50 2023-11-09T03:49:01.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.490+0000 7f48ceffd640 1 -- 172.21.15.76:0/3948520647 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48bc003520 con 0x7f48d0070c50 2023-11-09T03:49:01.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.490+0000 7f48c6ffd640 1 -- 172.21.15.76:0/3948520647 >> v2:172.21.15.113:3300/0 conn(0x7f48d015ad30 msgr2=0x7f48d00adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:01.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.490+0000 7f48c6ffd640 1 --2- 172.21.15.76:0/3948520647 >> v2:172.21.15.113:3300/0 conn(0x7f48d015ad30 0x7f48d00adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:01.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.490+0000 7f48c6ffd640 1 -- 172.21.15.76:0/3948520647 >> v1:172.21.15.113:6789/0 conn(0x7f48d0070c50 legacy=0x7f48d00c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:01.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.490+0000 7f48c6ffd640 1 -- 172.21.15.76:0/3948520647 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d00c3be0 con 0x7f48d00ad730 2023-11-09T03:49:01.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.490+0000 7f48cf7fe640 1 --2- 172.21.15.76:0/3948520647 >> v2:172.21.15.113:3300/0 conn(0x7f48d015ad30 0x7f48d00adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:49:01.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.491+0000 7f48c6ffd640 1 --2- 172.21.15.76:0/3948520647 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d00ad730 0x7f48d0070710 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f48b80097d0 tx=0x7f48b802b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a73218c5f72b33ed server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:01.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.491+0000 7f48ceffd640 1 -- 172.21.15.76:0/3948520647 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b8031770 con 0x7f48d00ad730 2023-11-09T03:49:01.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.491+0000 7f48ceffd640 1 -- 172.21.15.76:0/3948520647 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b80318d0 con 0x7f48d00ad730 2023-11-09T03:49:01.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.491+0000 7f48ceffd640 1 -- 172.21.15.76:0/3948520647 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b8031c40 con 0x7f48d00ad730 2023-11-09T03:49:01.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.492+0000 7f48d6244640 1 -- 172.21.15.76:0/3948520647 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d00ad730 msgr2=0x7f48d0070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:01.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.492+0000 7f48d6244640 1 --2- 172.21.15.76:0/3948520647 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d00ad730 0x7f48d0070710 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f48b80097d0 tx=0x7f48b802b810 comp rx=0 tx=0).stop 2023-11-09T03:49:01.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.492+0000 7f48d6244640 1 -- 172.21.15.76:0/3948520647 shutdown_connections 2023-11-09T03:49:01.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.492+0000 7f48d6244640 1 --2- 172.21.15.76:0/3948520647 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d00ad730 0x7f48d0070710 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:01.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.492+0000 7f48d6244640 1 --2- 172.21.15.76:0/3948520647 >> v2:172.21.15.113:3300/0 conn(0x7f48d015ad30 0x7f48d00adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:01.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.492+0000 7f48d6244640 1 -- 172.21.15.76:0/3948520647 >> 172.21.15.76:0/3948520647 conn(0x7f48d00ce950 msgr2=0x7f48d00ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:01.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.492+0000 7f48d6244640 1 -- 172.21.15.76:0/3948520647 shutdown_connections 2023-11-09T03:49:01.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.492+0000 7f48d6244640 1 -- 172.21.15.76:0/3948520647 wait complete. 2023-11-09T03:49:01.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.493+0000 7f48d6244640 1 Processor -- start 2023-11-09T03:49:01.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.493+0000 7f48d6244640 1 -- start start 2023-11-09T03:49:01.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.494+0000 7f48d6244640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48d0070c50 0x7f48d0155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:01.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.494+0000 7f48d6244640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48d00ad730 0x7f48d00ac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:01.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.494+0000 7f48d6244640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d015ad30 0x7f48d00b0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:01.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.494+0000 7f48d6244640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d00ac820 con 0x7f48d0070c50 2023-11-09T03:49:01.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.494+0000 7f48d6244640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d01563d0 con 0x7f48d00ad730 2023-11-09T03:49:01.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.494+0000 7f48d6244640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d01566b0 con 0x7f48d015ad30 2023-11-09T03:49:01.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.494+0000 7f48cf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48d0070c50 0x7f48d0155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:01.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.494+0000 7f48cf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48d0070c50 0x7f48d0155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44212/0 (socket says 172.21.15.76:44212) 2023-11-09T03:49:01.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.494+0000 7f48cf7fe640 1 -- 172.21.15.76:0/1294744083 learned_addr learned my addr 172.21.15.76:0/1294744083 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:01.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.494+0000 7f48c6ffd640 1 --2- 172.21.15.76:0/1294744083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48d00ad730 0x7f48d00ac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:01.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.494+0000 7f48cffff640 1 --2- 172.21.15.76:0/1294744083 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d015ad30 0x7f48d00b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:01.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.494+0000 7f48cf7fe640 1 -- 172.21.15.76:0/1294744083 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d015ad30 msgr2=0x7f48d00b0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:01.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.494+0000 7f48cf7fe640 1 --2- 172.21.15.76:0/1294744083 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d015ad30 0x7f48d00b0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:01.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.494+0000 7f48cf7fe640 1 -- 172.21.15.76:0/1294744083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48d00ad730 msgr2=0x7f48d00ac2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:01.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.494+0000 7f48cf7fe640 1 --2- 172.21.15.76:0/1294744083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48d00ad730 0x7f48d00ac2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:01.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.494+0000 7f48cf7fe640 1 -- 172.21.15.76:0/1294744083 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c00094d0 con 0x7f48d0070c50 2023-11-09T03:49:01.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.494+0000 7f48c6ffd640 1 --2- 172.21.15.76:0/1294744083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48d00ad730 0x7f48d00ac2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:49:01.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.494+0000 7f48cffff640 1 --2- 172.21.15.76:0/1294744083 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d015ad30 0x7f48d00b0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:49:01.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.494+0000 7f48cf7fe640 1 --2- 172.21.15.76:0/1294744083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48d0070c50 0x7f48d0155df0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f48c000e9b0 tx=0x7f48c000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:01.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.495+0000 7f48ccff9640 1 -- 172.21.15.76:0/1294744083 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c000c9a0 con 0x7f48d0070c50 2023-11-09T03:49:01.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.495+0000 7f48d6244640 1 -- 172.21.15.76:0/1294744083 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b8009400 con 0x7f48d0070c50 2023-11-09T03:49:01.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.495+0000 7f48ccff9640 1 -- 172.21.15.76:0/1294744083 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c000cb00 con 0x7f48d0070c50 2023-11-09T03:49:01.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.495+0000 7f48ccff9640 1 -- 172.21.15.76:0/1294744083 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c000ce70 con 0x7f48d0070c50 2023-11-09T03:49:01.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.495+0000 7f48d6244640 1 -- 172.21.15.76:0/1294744083 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d0156c70 con 0x7f48d0070c50 2023-11-09T03:49:01.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.495+0000 7f48d6244640 1 -- 172.21.15.76:0/1294744083 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d0074290 con 0x7f48d0070c50 2023-11-09T03:49:01.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.496+0000 7f48ccff9640 1 -- 172.21.15.76:0/1294744083 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f48c0013470 con 0x7f48d0070c50 2023-11-09T03:49:01.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.496+0000 7f48ccff9640 1 --2- 172.21.15.76:0/1294744083 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f48b006ae70 0x7f48b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:01.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.496+0000 7f48ccff9640 1 -- 172.21.15.76:0/1294744083 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f48c008a190 con 0x7f48d0070c50 2023-11-09T03:49:01.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.497+0000 7f48c6ffd640 1 --2- 172.21.15.76:0/1294744083 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f48b006ae70 0x7f48b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:01.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.497+0000 7f48c6ffd640 1 --2- 172.21.15.76:0/1294744083 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f48b006ae70 0x7f48b006d330 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f48b8036000 tx=0x7f48b803d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:01.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.501+0000 7f48ccff9640 1 -- 172.21.15.76:0/1294744083 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f48c0054c40 con 0x7f48d0070c50 2023-11-09T03:49:01.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.735+0000 7f48d6244640 1 -- 172.21.15.76:0/1294744083 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f48d0071970 con 0x7f48d0070c50 2023-11-09T03:49:01.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.737+0000 7f48ccff9640 1 -- 172.21.15.76:0/1294744083 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7f48c0075e60 con 0x7f48d0070c50 2023-11-09T03:49:01.736 INFO:teuthology.orchestra.run.smithi076.stdout:[client.0] 2023-11-09T03:49:01.736 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQChVUxlHj22CxAACNj5X24hdME+GNTLw4RahQ== 2023-11-09T03:49:01.737 INFO:teuthology.orchestra.run.smithi076.stdout: caps mds = "allow" 2023-11-09T03:49:01.737 INFO:teuthology.orchestra.run.smithi076.stdout: caps mon = "allow r" 2023-11-09T03:49:01.737 INFO:teuthology.orchestra.run.smithi076.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-09T03:49:01.737 INFO:teuthology.orchestra.run.smithi076.stderr:updated caps for client.0 2023-11-09T03:49:01.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.739+0000 7f48d6244640 1 -- 172.21.15.76:0/1294744083 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f48b006ae70 msgr2=0x7f48b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:01.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.739+0000 7f48d6244640 1 --2- 172.21.15.76:0/1294744083 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f48b006ae70 0x7f48b006d330 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f48b8036000 tx=0x7f48b803d040 comp rx=0 tx=0).stop 2023-11-09T03:49:01.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.739+0000 7f48d6244640 1 -- 172.21.15.76:0/1294744083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48d0070c50 msgr2=0x7f48d0155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:01.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.739+0000 7f48d6244640 1 --2- 172.21.15.76:0/1294744083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48d0070c50 0x7f48d0155df0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f48c000e9b0 tx=0x7f48c000ee70 comp rx=0 tx=0).stop 2023-11-09T03:49:01.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.739+0000 7f48d6244640 1 -- 172.21.15.76:0/1294744083 shutdown_connections 2023-11-09T03:49:01.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.739+0000 7f48d6244640 1 --2- 172.21.15.76:0/1294744083 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f48b006ae70 0x7f48b006d330 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:01.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.739+0000 7f48d6244640 1 --2- 172.21.15.76:0/1294744083 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d015ad30 0x7f48d00b0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:01.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.739+0000 7f48d6244640 1 --2- 172.21.15.76:0/1294744083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48d00ad730 0x7f48d00ac2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:01.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.739+0000 7f48d6244640 1 --2- 172.21.15.76:0/1294744083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48d0070c50 0x7f48d0155df0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:01.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.739+0000 7f48d6244640 1 -- 172.21.15.76:0/1294744083 >> 172.21.15.76:0/1294744083 conn(0x7f48d00ce950 msgr2=0x7f48d00ca4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:01.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.739+0000 7f48d6244640 1 -- 172.21.15.76:0/1294744083 shutdown_connections 2023-11-09T03:49:01.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.739+0000 7f48d6244640 1 -- 172.21.15.76:0/1294744083 wait complete. 2023-11-09T03:49:01.749 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:49:01.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.850+0000 7fcd1d9c5640 1 Processor -- start 2023-11-09T03:49:01.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.850+0000 7fcd1d9c5640 1 -- start start 2023-11-09T03:49:01.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.851+0000 7fcd1d9c5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd180cde60 0x7fcd1815d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:01.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.851+0000 7fcd1d9c5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcd1815daa0 0x7fcd18157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:01.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.851+0000 7fcd1d9c5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1815e000 con 0x7fcd180ce240 2023-11-09T03:49:01.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.851+0000 7fcd1d9c5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd1815e5c0 con 0x7fcd1815daa0 2023-11-09T03:49:01.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.851+0000 7fcd1d9c5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1815e2e0 con 0x7fcd180cde60 2023-11-09T03:49:01.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.851+0000 7fcd167fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcd1815daa0 0x7fcd18157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:01.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.851+0000 7fcd177fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fcd180ce240 0x7fcd1806b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35916/0 (socket says 172.21.15.76:35916) 2023-11-09T03:49:01.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.851+0000 7fcd177fe640 1 -- 172.21.15.76:0/3775410704 learned_addr learned my addr 172.21.15.76:0/3775410704 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:49:01.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.851+0000 7fcd16ffd640 1 --2- 172.21.15.76:0/3775410704 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd180cde60 0x7fcd1815d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:01.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.851+0000 7fcd167fc640 1 -- 172.21.15.76:0/3775410704 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd180cde60 msgr2=0x7fcd1815d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:01.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.851+0000 7fcd167fc640 1 --2- 172.21.15.76:0/3775410704 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd180cde60 0x7fcd1815d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:01.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.851+0000 7fcd167fc640 1 -- 172.21.15.76:0/3775410704 >> v1:172.21.15.76:6789/0 conn(0x7fcd180ce240 legacy=0x7fcd1806b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:01.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.851+0000 7fcd167fc640 1 -- 172.21.15.76:0/3775410704 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd180d1120 con 0x7fcd1815daa0 2023-11-09T03:49:01.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.851+0000 7fcd15ffb640 1 -- 172.21.15.76:0/3775410704 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2860949043 0 0) 0x7fcd1815e000 con 0x7fcd180ce240 2023-11-09T03:49:01.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.852+0000 7fcd167fc640 1 --2- 172.21.15.76:0/3775410704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd1815daa0 0x7fcd18157000 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c0097a0 tx=0x7fcd0c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b426cf066f2af15c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:01.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.852+0000 7fcd15ffb640 1 -- 172.21.15.76:0/3775410704 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c031830 con 0x7fcd1815daa0 2023-11-09T03:49:01.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.852+0000 7fcd15ffb640 1 -- 172.21.15.76:0/3775410704 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0c031990 con 0x7fcd1815daa0 2023-11-09T03:49:01.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.852+0000 7fcd15ffb640 1 -- 172.21.15.76:0/3775410704 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c031d20 con 0x7fcd1815daa0 2023-11-09T03:49:01.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.853+0000 7fcd1d9c5640 1 -- 172.21.15.76:0/3775410704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd1815daa0 msgr2=0x7fcd18157000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:01.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.853+0000 7fcd1d9c5640 1 --2- 172.21.15.76:0/3775410704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd1815daa0 0x7fcd18157000 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c0097a0 tx=0x7fcd0c02b8d0 comp rx=0 tx=0).stop 2023-11-09T03:49:01.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.853+0000 7fcd1d9c5640 1 -- 172.21.15.76:0/3775410704 shutdown_connections 2023-11-09T03:49:01.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.853+0000 7fcd1d9c5640 1 --2- 172.21.15.76:0/3775410704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd1815daa0 0x7fcd18157000 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:01.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.853+0000 7fcd1d9c5640 1 --2- 172.21.15.76:0/3775410704 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd180cde60 0x7fcd1815d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:01.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.853+0000 7fcd1d9c5640 1 -- 172.21.15.76:0/3775410704 >> 172.21.15.76:0/3775410704 conn(0x7fcd18071f30 msgr2=0x7fcd180cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:01.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.853+0000 7fcd1d9c5640 1 -- 172.21.15.76:0/3775410704 shutdown_connections 2023-11-09T03:49:01.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.853+0000 7fcd1d9c5640 1 -- 172.21.15.76:0/3775410704 wait complete. 2023-11-09T03:49:01.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.854+0000 7fcd1d9c5640 1 Processor -- start 2023-11-09T03:49:01.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.854+0000 7fcd1d9c5640 1 -- start start 2023-11-09T03:49:01.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.855+0000 7fcd1d9c5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd180cde60 0x7fcd18153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:01.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.855+0000 7fcd1d9c5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd180ce240 0x7fcd18151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:01.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.855+0000 7fcd1d9c5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd1815daa0 0x7fcd18151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:01.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.855+0000 7fcd1d9c5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd18152ef0 con 0x7fcd180ce240 2023-11-09T03:49:01.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.855+0000 7fcd1d9c5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd18152550 con 0x7fcd1815daa0 2023-11-09T03:49:01.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.855+0000 7fcd1d9c5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd18152830 con 0x7fcd180cde60 2023-11-09T03:49:01.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.855+0000 7fcd167fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd180ce240 0x7fcd18151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:01.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.855+0000 7fcd167fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd180ce240 0x7fcd18151020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44234/0 (socket says 172.21.15.76:44234) 2023-11-09T03:49:01.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.855+0000 7fcd167fc640 1 -- 172.21.15.76:0/1843494460 learned_addr learned my addr 172.21.15.76:0/1843494460 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:01.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.855+0000 7fcd16ffd640 1 --2- 172.21.15.76:0/1843494460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd180cde60 0x7fcd18153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:01.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.855+0000 7fcd177fe640 1 --2- 172.21.15.76:0/1843494460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd1815daa0 0x7fcd18151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:01.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.855+0000 7fcd167fc640 1 -- 172.21.15.76:0/1843494460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd180cde60 msgr2=0x7fcd18153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:01.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.855+0000 7fcd167fc640 1 --2- 172.21.15.76:0/1843494460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd180cde60 0x7fcd18153310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:01.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.855+0000 7fcd167fc640 1 -- 172.21.15.76:0/1843494460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd1815daa0 msgr2=0x7fcd18151560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:01.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.855+0000 7fcd167fc640 1 --2- 172.21.15.76:0/1843494460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd1815daa0 0x7fcd18151560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:01.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.855+0000 7fcd167fc640 1 -- 172.21.15.76:0/1843494460 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0c009400 con 0x7fcd180ce240 2023-11-09T03:49:01.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.855+0000 7fcd16ffd640 1 --2- 172.21.15.76:0/1843494460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd180cde60 0x7fcd18153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:01.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.855+0000 7fcd177fe640 1 --2- 172.21.15.76:0/1843494460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd1815daa0 0x7fcd18151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:01.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.855+0000 7fcd167fc640 1 --2- 172.21.15.76:0/1843494460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd180ce240 0x7fcd18151020 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c000c00 tx=0x7fcd0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:01.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.855+0000 7fccfffff640 1 -- 172.21.15.76:0/1843494460 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c03aa60 con 0x7fcd180ce240 2023-11-09T03:49:01.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.855+0000 7fccfffff640 1 -- 172.21.15.76:0/1843494460 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0c03abe0 con 0x7fcd180ce240 2023-11-09T03:49:01.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.855+0000 7fcd1d9c5640 1 -- 172.21.15.76:0/1843494460 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd18152b10 con 0x7fcd180ce240 2023-11-09T03:49:01.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.855+0000 7fccfffff640 1 -- 172.21.15.76:0/1843494460 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c0443f0 con 0x7fcd180ce240 2023-11-09T03:49:01.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.856+0000 7fcd1d9c5640 1 -- 172.21.15.76:0/1843494460 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd18154540 con 0x7fcd180ce240 2023-11-09T03:49:01.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.857+0000 7fcd1d9c5640 1 -- 172.21.15.76:0/1843494460 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd18074290 con 0x7fcd180ce240 2023-11-09T03:49:01.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.857+0000 7fccfffff640 1 -- 172.21.15.76:0/1843494460 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcd0c04d450 con 0x7fcd180ce240 2023-11-09T03:49:01.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.857+0000 7fccfffff640 1 --2- 172.21.15.76:0/1843494460 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcd0406aea0 0x7fcd0406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:01.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.857+0000 7fccfffff640 1 -- 172.21.15.76:0/1843494460 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7120+0+0 (secure 0 0 0) 0x7fcd0c0abe00 con 0x7fcd180ce240 2023-11-09T03:49:01.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.858+0000 7fcd16ffd640 1 --2- 172.21.15.76:0/1843494460 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcd0406aea0 0x7fcd0406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:01.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.858+0000 7fcd16ffd640 1 --2- 172.21.15.76:0/1843494460 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcd0406aea0 0x7fcd0406d360 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800a9d0 tx=0x7fcd08008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:01.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:01.862+0000 7fccfffff640 1 -- 172.21.15.76:0/1843494460 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcd0c0768a0 con 0x7fcd180ce240 2023-11-09T03:49:02.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.062+0000 7fcd1d9c5640 1 -- 172.21.15.76:0/1843494460 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd180b6840 con 0x7fcd180ce240 2023-11-09T03:49:02.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.063+0000 7fccfffff640 1 -- 172.21.15.76:0/1843494460 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fcd0c035030 con 0x7fcd180ce240 2023-11-09T03:49:02.062 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:49:02.063 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":44,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5744,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3280098776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3280098776},{"type":"v1","addr":"172.21.15.113:6837","nonce":3280098776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":7449,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:48:59.377862+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7452":{"gid":7452,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/4011275120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4011275120},{"type":"v1","addr":"172.21.15.76:6835","nonce":4011275120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:49:02.063 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 44 2023-11-09T03:49:02.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.064+0000 7fcd1d9c5640 1 -- 172.21.15.76:0/1843494460 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcd0406aea0 msgr2=0x7fcd0406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:02.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.064+0000 7fcd1d9c5640 1 --2- 172.21.15.76:0/1843494460 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcd0406aea0 0x7fcd0406d360 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800a9d0 tx=0x7fcd08008040 comp rx=0 tx=0).stop 2023-11-09T03:49:02.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.065+0000 7fcd1d9c5640 1 -- 172.21.15.76:0/1843494460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd180ce240 msgr2=0x7fcd18151020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:02.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.065+0000 7fcd1d9c5640 1 --2- 172.21.15.76:0/1843494460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd180ce240 0x7fcd18151020 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c000c00 tx=0x7fcd0c0046c0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.065+0000 7fcd1d9c5640 1 -- 172.21.15.76:0/1843494460 shutdown_connections 2023-11-09T03:49:02.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.065+0000 7fcd1d9c5640 1 --2- 172.21.15.76:0/1843494460 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fcd0406aea0 0x7fcd0406d360 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.065+0000 7fcd1d9c5640 1 --2- 172.21.15.76:0/1843494460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd1815daa0 0x7fcd18151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.065+0000 7fcd1d9c5640 1 --2- 172.21.15.76:0/1843494460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd180ce240 0x7fcd18151020 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.065+0000 7fcd1d9c5640 1 --2- 172.21.15.76:0/1843494460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd180cde60 0x7fcd18153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.065+0000 7fcd1d9c5640 1 -- 172.21.15.76:0/1843494460 >> 172.21.15.76:0/1843494460 conn(0x7fcd18071f30 msgr2=0x7fcd180b6050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:02.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.065+0000 7fcd1d9c5640 1 -- 172.21.15.76:0/1843494460 shutdown_connections 2023-11-09T03:49:02.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.065+0000 7fcd1d9c5640 1 -- 172.21.15.76:0/1843494460 wait complete. 2023-11-09T03:49:02.477 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-09T03:48:57.383848+0000', 'modified': '2023-11-09T03:48:59.377862+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7452}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7452': {'gid': 7452, 'name': 'c', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6835/4011275120', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 4011275120}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 4011275120}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-09T03:49:02.477 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-09T03:49:02.477 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-09T03:49:02.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.579+0000 7f9cfc5bb640 1 Processor -- start 2023-11-09T03:49:02.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.579+0000 7f9cfc5bb640 1 -- start start 2023-11-09T03:49:02.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.580+0000 7f9cfc5bb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cf4159c00 0x7f9cf40cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:02.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.580+0000 7f9cfc5bb640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9cf40c48e0 0x7f9cf40c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:02.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.580+0000 7f9cfc5bb640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9cf40adac0 0x7f9cf40adef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:02.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.580+0000 7f9cfc5bb640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf40c52a0 con 0x7f9cf40c48e0 2023-11-09T03:49:02.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.580+0000 7f9cfc5bb640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf415aff0 con 0x7f9cf40adac0 2023-11-09T03:49:02.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.580+0000 7f9cfc5bb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf40ae430 con 0x7f9cf4159c00 2023-11-09T03:49:02.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.580+0000 7f9cf9b56640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9cf40c48e0 0x7f9cf40c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:02.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.581+0000 7f9cf9b56640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9cf40c48e0 0x7f9cf40c4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38186/0 (socket says 172.21.15.76:38186) 2023-11-09T03:49:02.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.581+0000 7f9cf9b56640 1 -- 172.21.15.76:0/593563691 learned_addr learned my addr 172.21.15.76:0/593563691 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:02.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.581+0000 7f9cfab58640 1 --2- 172.21.15.76:0/593563691 >> v2:172.21.15.113:3300/0 conn(0x7f9cf40adac0 0x7f9cf40adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:02.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.581+0000 7f9cfa357640 1 --2- 172.21.15.76:0/593563691 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cf4159c00 0x7f9cf40cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:02.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.581+0000 7f9cf9b56640 1 -- 172.21.15.76:0/593563691 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cf4159c00 msgr2=0x7f9cf40cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:02.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.581+0000 7f9cf9b56640 1 --2- 172.21.15.76:0/593563691 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cf4159c00 0x7f9cf40cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.581+0000 7f9cf9b56640 1 -- 172.21.15.76:0/593563691 >> v2:172.21.15.113:3300/0 conn(0x7f9cf40adac0 msgr2=0x7f9cf40adef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:02.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.581+0000 7f9cf9b56640 1 --2- 172.21.15.76:0/593563691 >> v2:172.21.15.113:3300/0 conn(0x7f9cf40adac0 0x7f9cf40adef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.581+0000 7f9cf9b56640 1 -- 172.21.15.76:0/593563691 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf415a3e0 con 0x7f9cf40c48e0 2023-11-09T03:49:02.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.581+0000 7f9cfa357640 1 --2- 172.21.15.76:0/593563691 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cf4159c00 0x7f9cf40cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:49:02.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.581+0000 7f9cf9b56640 1 --2- 172.21.15.76:0/593563691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cf40c48e0 0x7f9cf40c4d60 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f9cf415d5c0 tx=0x7f9cf002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9933edcc5bb4cb4e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:02.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.582+0000 7f9cf9355640 1 -- 172.21.15.76:0/593563691 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0031860 con 0x7f9cf40c48e0 2023-11-09T03:49:02.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.582+0000 7f9cf9355640 1 -- 172.21.15.76:0/593563691 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf00319c0 con 0x7f9cf40c48e0 2023-11-09T03:49:02.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.582+0000 7f9cf9355640 1 -- 172.21.15.76:0/593563691 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0031d50 con 0x7f9cf40c48e0 2023-11-09T03:49:02.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.582+0000 7f9cfc5bb640 1 -- 172.21.15.76:0/593563691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cf40c48e0 msgr2=0x7f9cf40c4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:02.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.582+0000 7f9cfc5bb640 1 --2- 172.21.15.76:0/593563691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cf40c48e0 0x7f9cf40c4d60 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f9cf415d5c0 tx=0x7f9cf002b900 comp rx=0 tx=0).stop 2023-11-09T03:49:02.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.582+0000 7f9cfc5bb640 1 -- 172.21.15.76:0/593563691 shutdown_connections 2023-11-09T03:49:02.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.582+0000 7f9cfc5bb640 1 --2- 172.21.15.76:0/593563691 >> v2:172.21.15.113:3300/0 conn(0x7f9cf40adac0 0x7f9cf40adef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.582+0000 7f9cfc5bb640 1 --2- 172.21.15.76:0/593563691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cf40c48e0 0x7f9cf40c4d60 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.582+0000 7f9cfc5bb640 1 --2- 172.21.15.76:0/593563691 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cf4159c00 0x7f9cf40cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.582+0000 7f9cfc5bb640 1 -- 172.21.15.76:0/593563691 >> 172.21.15.76:0/593563691 conn(0x7f9cf40ce950 msgr2=0x7f9cf40ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:02.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.582+0000 7f9cfc5bb640 1 -- 172.21.15.76:0/593563691 shutdown_connections 2023-11-09T03:49:02.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.582+0000 7f9cfc5bb640 1 -- 172.21.15.76:0/593563691 wait complete. 2023-11-09T03:49:02.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.583+0000 7f9cfc5bb640 1 Processor -- start 2023-11-09T03:49:02.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.584+0000 7f9cfc5bb640 1 -- start start 2023-11-09T03:49:02.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.584+0000 7f9cfc5bb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cf40adac0 0x7f9cf40c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:02.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.584+0000 7f9cfc5bb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cf40c48e0 0x7f9cf4142540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:02.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.584+0000 7f9cfc5bb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cf4159c00 0x7f9cf4156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:02.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.584+0000 7f9cfc5bb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf40aca80 con 0x7f9cf40adac0 2023-11-09T03:49:02.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.584+0000 7f9cfc5bb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf40c7400 con 0x7f9cf4159c00 2023-11-09T03:49:02.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.584+0000 7f9cfc5bb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf40abe70 con 0x7f9cf40c48e0 2023-11-09T03:49:02.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.584+0000 7f9cfa357640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cf40adac0 0x7f9cf40c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:02.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.584+0000 7f9cfa357640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cf40adac0 0x7f9cf40c6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38202/0 (socket says 172.21.15.76:38202) 2023-11-09T03:49:02.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.584+0000 7f9cfa357640 1 -- 172.21.15.76:0/609753808 learned_addr learned my addr 172.21.15.76:0/609753808 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:02.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.585+0000 7f9cfab58640 1 --2- 172.21.15.76:0/609753808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cf4159c00 0x7f9cf4156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:02.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.585+0000 7f9cfa357640 1 -- 172.21.15.76:0/609753808 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cf40c48e0 msgr2=0x7f9cf4142540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:02.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.585+0000 7f9cf9b56640 1 --2- 172.21.15.76:0/609753808 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cf40c48e0 0x7f9cf4142540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:02.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.585+0000 7f9cfa357640 1 --2- 172.21.15.76:0/609753808 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cf40c48e0 0x7f9cf4142540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.585+0000 7f9cfa357640 1 -- 172.21.15.76:0/609753808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cf4159c00 msgr2=0x7f9cf4156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:02.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.585+0000 7f9cfa357640 1 --2- 172.21.15.76:0/609753808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cf4159c00 0x7f9cf4156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.585+0000 7f9cfa357640 1 -- 172.21.15.76:0/609753808 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf00094d0 con 0x7f9cf40adac0 2023-11-09T03:49:02.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.585+0000 7f9cfab58640 1 --2- 172.21.15.76:0/609753808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cf4159c00 0x7f9cf4156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:49:02.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.585+0000 7f9cfa357640 1 --2- 172.21.15.76:0/609753808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cf40adac0 0x7f9cf40c6ec0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f9ce800c7f0 tx=0x7f9ce800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:02.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.585+0000 7f9ce6ffd640 1 -- 172.21.15.76:0/609753808 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8007a30 con 0x7f9cf40adac0 2023-11-09T03:49:02.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.585+0000 7f9ce6ffd640 1 -- 172.21.15.76:0/609753808 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce8007b90 con 0x7f9cf40adac0 2023-11-09T03:49:02.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.585+0000 7f9cfc5bb640 1 -- 172.21.15.76:0/609753808 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf40ac150 con 0x7f9cf40adac0 2023-11-09T03:49:02.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.585+0000 7f9ce6ffd640 1 -- 172.21.15.76:0/609753808 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce800ce40 con 0x7f9cf40adac0 2023-11-09T03:49:02.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.585+0000 7f9cfc5bb640 1 -- 172.21.15.76:0/609753808 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf40ac710 con 0x7f9cf40adac0 2023-11-09T03:49:02.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.586+0000 7f9ce6ffd640 1 -- 172.21.15.76:0/609753808 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9ce80113f0 con 0x7f9cf40adac0 2023-11-09T03:49:02.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.587+0000 7f9ce6ffd640 1 --2- 172.21.15.76:0/609753808 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9cd806ae70 0x7f9cd806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:02.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.587+0000 7f9ce6ffd640 1 -- 172.21.15.76:0/609753808 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f9ce8089d30 con 0x7f9cf40adac0 2023-11-09T03:49:02.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.587+0000 7f9cf9b56640 1 --2- 172.21.15.76:0/609753808 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9cd806ae70 0x7f9cd806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:02.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.587+0000 7f9cf9b56640 1 --2- 172.21.15.76:0/609753808 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9cd806ae70 0x7f9cd806d330 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0036000 tx=0x7f9cf003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:02.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.588+0000 7f9cff22a740 1 Processor -- start 2023-11-09T03:49:02.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.588+0000 7f9cff22a740 1 -- start start 2023-11-09T03:49:02.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.589+0000 7f9cff22a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56412afe4850 0x56412afb7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:02.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.589+0000 7f9cff22a740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56412afe4c30 0x56412afe75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:02.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.589+0000 7f9cff22a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56412afb7c90 0x56412afe7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:02.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.589+0000 7f9cff22a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412aec8740 con 0x56412afe4850 2023-11-09T03:49:02.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.589+0000 7f9cff22a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412af2a8d0 con 0x56412afb7c90 2023-11-09T03:49:02.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.589+0000 7f9cff22a740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x56412af27fc0 con 0x56412afe4c30 2023-11-09T03:49:02.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.589+0000 7f9cfa357640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56412afe4850 0x56412afb7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:02.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.589+0000 7f9cfa357640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56412afe4850 0x56412afb7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38210/0 (socket says 172.21.15.76:38210) 2023-11-09T03:49:02.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.589+0000 7f9cfa357640 1 -- 172.21.15.76:0/146594773 learned_addr learned my addr 172.21.15.76:0/146594773 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:02.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.589+0000 7f9cfab58640 1 --2- 172.21.15.76:0/146594773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56412afb7c90 0x56412afe7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:02.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.589+0000 7f9cf9b56640 1 --2- 172.21.15.76:0/146594773 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56412afe4c30 0x56412afe75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:02.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.589+0000 7f9cfa357640 1 -- 172.21.15.76:0/146594773 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56412afe4c30 msgr2=0x56412afe75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:02.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.589+0000 7f9cfa357640 1 --2- 172.21.15.76:0/146594773 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56412afe4c30 0x56412afe75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.589+0000 7f9cfa357640 1 -- 172.21.15.76:0/146594773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56412afb7c90 msgr2=0x56412afe7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:02.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.589+0000 7f9cfa357640 1 --2- 172.21.15.76:0/146594773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56412afb7c90 0x56412afe7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.589+0000 7f9cfa357640 1 -- 172.21.15.76:0/146594773 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56412aee6ca0 con 0x56412afe4850 2023-11-09T03:49:02.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.589+0000 7f9cfa357640 1 --2- 172.21.15.76:0/146594773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56412afe4850 0x56412afb7750 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f9ce800a120 tx=0x7f9ce8058520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd33f99ec9ef20f server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:02.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.589+0000 7f9cd7fff640 1 -- 172.21.15.76:0/146594773 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce80568c0 con 0x56412afe4850 2023-11-09T03:49:02.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.589+0000 7f9cd7fff640 1 -- 172.21.15.76:0/146594773 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce8054c30 con 0x56412afe4850 2023-11-09T03:49:02.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.589+0000 7f9cd7fff640 1 -- 172.21.15.76:0/146594773 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce80605b0 con 0x56412afe4850 2023-11-09T03:49:02.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.590+0000 7f9cff22a740 1 -- 172.21.15.76:0/146594773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56412afe4850 msgr2=0x56412afb7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:02.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.590+0000 7f9cff22a740 1 --2- 172.21.15.76:0/146594773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56412afe4850 0x56412afb7750 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f9ce800a120 tx=0x7f9ce8058520 comp rx=0 tx=0).stop 2023-11-09T03:49:02.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.590+0000 7f9cff22a740 1 -- 172.21.15.76:0/146594773 shutdown_connections 2023-11-09T03:49:02.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.590+0000 7f9cff22a740 1 --2- 172.21.15.76:0/146594773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56412afb7c90 0x56412afe7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.590+0000 7f9cff22a740 1 --2- 172.21.15.76:0/146594773 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56412afe4c30 0x56412afe75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.590+0000 7f9cff22a740 1 --2- 172.21.15.76:0/146594773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56412afe4850 0x56412afb7750 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.590+0000 7f9cff22a740 1 -- 172.21.15.76:0/146594773 >> 172.21.15.76:0/146594773 conn(0x56412afb9a60 msgr2=0x56412b0025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:02.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.590+0000 7f9cff22a740 1 -- 172.21.15.76:0/146594773 shutdown_connections 2023-11-09T03:49:02.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.590+0000 7f9cff22a740 1 -- 172.21.15.76:0/146594773 wait complete. 2023-11-09T03:49:02.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.591+0000 7f9cff22a740 1 -- start start 2023-11-09T03:49:02.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.591+0000 7f9cff22a740 1 Processor -- start 2023-11-09T03:49:02.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.591+0000 7f9cff22a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56412afe4850 0x56412afdd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:02.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.591+0000 7f9cff22a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56412afe4c30 0x56412afe8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:02.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.591+0000 7f9cff22a740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56412afe6f50 0x56412afe01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:02.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.592+0000 7f9cff22a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412af238b0 con 0x56412afe4c30 2023-11-09T03:49:02.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.592+0000 7f9cff22a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412aeddc80 con 0x56412afe4850 2023-11-09T03:49:02.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.592+0000 7f9cff22a740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x56412aedd920 con 0x56412afe6f50 2023-11-09T03:49:02.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.592+0000 7f9cff22a740 10 client.0 _ll_register_callbacks cb 0x56412ac35ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:49:02.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.592+0000 7f9cf9b56640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56412afe4c30 0x56412afe8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:02.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.592+0000 7f9cf9b56640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56412afe4c30 0x56412afe8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38226/0 (socket says 172.21.15.76:38226) 2023-11-09T03:49:02.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.592+0000 7f9cf9b56640 1 -- 172.21.15.76:0/4046556828 learned_addr learned my addr 172.21.15.76:0/4046556828 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:02.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.592+0000 7f9cfa357640 1 --2- 172.21.15.76:0/4046556828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56412afe4850 0x56412afdd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:02.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.592+0000 7f9cf9b56640 1 -- 172.21.15.76:0/4046556828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56412afe6f50 msgr2=0x56412afe01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:02.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.592+0000 7f9cfab58640 1 --2- 172.21.15.76:0/4046556828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56412afe6f50 0x56412afe01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:02.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.592+0000 7f9cf9b56640 1 --2- 172.21.15.76:0/4046556828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56412afe6f50 0x56412afe01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.592+0000 7f9cf9b56640 1 -- 172.21.15.76:0/4046556828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56412afe4850 msgr2=0x56412afdd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:02.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.592+0000 7f9cf9b56640 1 --2- 172.21.15.76:0/4046556828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56412afe4850 0x56412afdd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.592+0000 7f9cf9b56640 1 -- 172.21.15.76:0/4046556828 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf40ac710 con 0x56412afe4c30 2023-11-09T03:49:02.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.592+0000 7f9cf9b56640 1 --2- 172.21.15.76:0/4046556828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56412afe4c30 0x56412afe8fc0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0006da0 tx=0x7f9cf0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c2625c62873337 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:02.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.592+0000 7f9cd57fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:49:02.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.592+0000 7f9cd57fa640 1 -- 172.21.15.76:0/4046556828 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf004b070 con 0x56412afe4c30 2023-11-09T03:49:02.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.592+0000 7f9cff22a740 1 -- 172.21.15.76:0/4046556828 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56412aecfbf0 con 0x56412afe4c30 2023-11-09T03:49:02.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.592+0000 7f9cd57fa640 1 -- 172.21.15.76:0/4046556828 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f9cf0003cf0 con 0x56412afe4c30 2023-11-09T03:49:02.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.592+0000 7f9cd57fa640 1 -- 172.21.15.76:0/4046556828 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cf003a420 con 0x56412afe4c30 2023-11-09T03:49:02.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.592+0000 7f9cff22a740 10 client.7602 fetch_fsmap learned FSMap version 44 2023-11-09T03:49:02.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.592+0000 7f9cff22a740 1 -- 172.21.15.76:0/4046556828 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x56412af21f30 con 0x56412afe4c30 2023-11-09T03:49:02.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.593+0000 7f9cd57fa640 1 -- 172.21.15.76:0/4046556828 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf0058070 con 0x56412afe4c30 2023-11-09T03:49:02.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.593+0000 7f9cd57fa640 1 -- 172.21.15.76:0/4046556828 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9cf003a580 con 0x56412afe4c30 2023-11-09T03:49:02.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.593+0000 7f9cff22a740 10 client.7602 fetch_fsmap finished waiting for FSMap version 44 2023-11-09T03:49:02.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.593+0000 7f9cff22a740 10 client.7602 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-09T03:49:02.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.593+0000 7f9cff22a740 20 client.7602 populate_metadata read hostname 'smithi076' 2023-11-09T03:49:02.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.593+0000 7f9cff22a740 1 --2- 172.21.15.76:0/4046556828 >> [v2:172.21.15.76:6834/4011275120,v1:172.21.15.76:6835/4011275120] conn(0x56412b002190 0x56412b002570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:02.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.593+0000 7f9cff22a740 4 client.7602 mds_command: new command op to 7452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:49:02.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.593+0000 7f9cff22a740 1 -- 172.21.15.76:0/4046556828 --> [v2:172.21.15.76:6834/4011275120,v1:172.21.15.76:6835/4011275120] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56412af1c890 con 0x56412b002190 2023-11-09T03:49:02.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.593+0000 7f9cfa357640 1 --2- 172.21.15.76:0/4046556828 >> [v2:172.21.15.76:6834/4011275120,v1:172.21.15.76:6835/4011275120] conn(0x56412b002190 0x56412b002570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:02.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.593+0000 7f9cd57fa640 10 client.7602 ms_handle_connect on v2:172.21.15.76:6834/4011275120 2023-11-09T03:49:02.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.593+0000 7f9cfa357640 1 --2- 172.21.15.76:0/4046556828 >> [v2:172.21.15.76:6834/4011275120,v1:172.21.15.76:6835/4011275120] conn(0x56412b002190 0x56412b002570 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=379bd216cf8d3bfd server_cookie=f2bf623f9a9caffd in_seq=0 out_seq=0 2023-11-09T03:49:02.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.594+0000 7f9cd57fa640 1 -- 172.21.15.76:0/4046556828 <== mds.0 v2:172.21.15.76:6834/4011275120 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56412af1c890 con 0x56412b002190 2023-11-09T03:49:02.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.594+0000 7f9cd57fa640 10 client.7602 handle_command_reply: tid=0 2023-11-09T03:49:02.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.594+0000 7f9cff22a740 1 client.7602 shutdown 2023-11-09T03:49:02.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.594+0000 7f9cff22a740 1 -- 172.21.15.76:0/4046556828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56412afe4c30 msgr2=0x56412afe8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:02.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.594+0000 7f9cff22a740 1 --2- 172.21.15.76:0/4046556828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56412afe4c30 0x56412afe8fc0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0006da0 tx=0x7f9cf0004730 comp rx=0 tx=0).stop 2023-11-09T03:49:02.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.594+0000 7f9cff22a740 1 -- 172.21.15.76:0/4046556828 shutdown_connections 2023-11-09T03:49:02.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.594+0000 7f9cff22a740 1 --2- 172.21.15.76:0/4046556828 >> [v2:172.21.15.76:6834/4011275120,v1:172.21.15.76:6835/4011275120] conn(0x56412b002190 0x56412b002570 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.594+0000 7f9cff22a740 1 --2- 172.21.15.76:0/4046556828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56412afe6f50 0x56412afe01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.594+0000 7f9cff22a740 1 --2- 172.21.15.76:0/4046556828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56412afe4c30 0x56412afe8fc0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.594+0000 7f9cd57fa640 0 client.7602 ms_handle_reset on v2:172.21.15.76:6834/4011275120 2023-11-09T03:49:02.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.594+0000 7f9cff22a740 1 --2- 172.21.15.76:0/4046556828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56412afe4850 0x56412afdd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.594+0000 7f9cff22a740 1 -- 172.21.15.76:0/4046556828 >> 172.21.15.76:0/4046556828 conn(0x56412afb7c90 msgr2=0x56412afbb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:02.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.594+0000 7f9cff22a740 1 -- 172.21.15.76:0/4046556828 shutdown_connections 2023-11-09T03:49:02.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.594+0000 7f9cff22a740 1 -- 172.21.15.76:0/4046556828 wait complete. 2023-11-09T03:49:02.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.594+0000 7f9cff22a740 20 client.7602 trim_cache size 0 max 16384 2023-11-09T03:49:02.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.609+0000 7f9cff22a740 1 Processor -- start 2023-11-09T03:49:02.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.610+0000 7f9cff22a740 1 -- start start 2023-11-09T03:49:02.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.610+0000 7f9cff22a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56412afe4850 0x56412afe01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:02.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.610+0000 7f9cff22a740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56412afe4c30 0x56412b011f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:02.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.610+0000 7f9cff22a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56412afe6f50 0x56412afe8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:02.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.610+0000 7f9cff22a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412aea3280 con 0x56412afe4850 2023-11-09T03:49:02.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.610+0000 7f9cff22a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412afe2ca0 con 0x56412afe6f50 2023-11-09T03:49:02.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.610+0000 7f9cff22a740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x56412aeef7b0 con 0x56412afe4c30 2023-11-09T03:49:02.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.610+0000 7f9cfa357640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56412afe4850 0x56412afe01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:02.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.610+0000 7f9cfa357640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56412afe4850 0x56412afe01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38232/0 (socket says 172.21.15.76:38232) 2023-11-09T03:49:02.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.610+0000 7f9cfa357640 1 -- 172.21.15.76:0/2674472781 learned_addr learned my addr 172.21.15.76:0/2674472781 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:02.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.611+0000 7f9cfab58640 1 --2- 172.21.15.76:0/2674472781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56412afe6f50 0x56412afe8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:02.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.611+0000 7f9cf9b56640 1 --2- 172.21.15.76:0/2674472781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56412afe4c30 0x56412b011f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:02.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.611+0000 7f9cfa357640 1 -- 172.21.15.76:0/2674472781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56412afe4c30 msgr2=0x56412b011f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:02.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.611+0000 7f9cfa357640 1 --2- 172.21.15.76:0/2674472781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56412afe4c30 0x56412b011f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.611+0000 7f9cfa357640 1 -- 172.21.15.76:0/2674472781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56412afe6f50 msgr2=0x56412afe8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:02.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.611+0000 7f9cfa357640 1 --2- 172.21.15.76:0/2674472781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56412afe6f50 0x56412afe8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.611+0000 7f9cfa357640 1 -- 172.21.15.76:0/2674472781 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56412aebfd10 con 0x56412afe4850 2023-11-09T03:49:02.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.611+0000 7f9cf9b56640 1 --2- 172.21.15.76:0/2674472781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56412afe4c30 0x56412b011f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:49:02.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.611+0000 7f9cfa357640 1 --2- 172.21.15.76:0/2674472781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56412afe4850 0x56412afe01f0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8058220 tx=0x7f9ce801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b61a30ba0aa0c46 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:02.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.611+0000 7f9ce4ff9640 1 -- 172.21.15.76:0/2674472781 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8054da0 con 0x56412afe4850 2023-11-09T03:49:02.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.611+0000 7f9ce4ff9640 1 -- 172.21.15.76:0/2674472781 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce8056660 con 0x56412afe4850 2023-11-09T03:49:02.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.611+0000 7f9ce4ff9640 1 -- 172.21.15.76:0/2674472781 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8060390 con 0x56412afe4850 2023-11-09T03:49:02.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.611+0000 7f9cff22a740 1 -- 172.21.15.76:0/2674472781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56412afe4850 msgr2=0x56412afe01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:02.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.611+0000 7f9cff22a740 1 --2- 172.21.15.76:0/2674472781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56412afe4850 0x56412afe01f0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8058220 tx=0x7f9ce801c000 comp rx=0 tx=0).stop 2023-11-09T03:49:02.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.611+0000 7f9cff22a740 1 -- 172.21.15.76:0/2674472781 shutdown_connections 2023-11-09T03:49:02.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.611+0000 7f9cff22a740 1 --2- 172.21.15.76:0/2674472781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56412afe6f50 0x56412afe8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.611+0000 7f9cff22a740 1 --2- 172.21.15.76:0/2674472781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56412afe4c30 0x56412b011f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.611+0000 7f9cff22a740 1 --2- 172.21.15.76:0/2674472781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56412afe4850 0x56412afe01f0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.611+0000 7f9cff22a740 1 -- 172.21.15.76:0/2674472781 >> 172.21.15.76:0/2674472781 conn(0x56412afb7c90 msgr2=0x56412afb2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:02.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.611+0000 7f9cff22a740 1 -- 172.21.15.76:0/2674472781 shutdown_connections 2023-11-09T03:49:02.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.611+0000 7f9cff22a740 1 -- 172.21.15.76:0/2674472781 wait complete. 2023-11-09T03:49:02.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.612+0000 7f9cff22a740 1 -- start start 2023-11-09T03:49:02.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.612+0000 7f9cff22a740 1 Processor -- start 2023-11-09T03:49:02.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.613+0000 7f9cff22a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56412afe4850 0x56412afd0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:02.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.613+0000 7f9cff22a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56412afe4c30 0x56412afe5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:02.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.613+0000 7f9cff22a740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56412afe6f50 0x56412afe5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:02.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.613+0000 7f9cff22a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412b06d060 con 0x56412afe4c30 2023-11-09T03:49:02.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.613+0000 7f9cff22a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x56412afe2e90 con 0x56412afe4850 2023-11-09T03:49:02.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.613+0000 7f9cff22a740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x56412afd11a0 con 0x56412afe6f50 2023-11-09T03:49:02.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.613+0000 7f9cff22a740 10 client.0 _ll_register_callbacks cb 0x7f9ce80548e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:49:02.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.613+0000 7f9cf9b56640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56412afe4c30 0x56412afe5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:02.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.613+0000 7f9cf9b56640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56412afe4c30 0x56412afe5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38248/0 (socket says 172.21.15.76:38248) 2023-11-09T03:49:02.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.613+0000 7f9cf9b56640 1 -- 172.21.15.76:0/643528674 learned_addr learned my addr 172.21.15.76:0/643528674 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:02.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.613+0000 7f9cfa357640 1 --2- 172.21.15.76:0/643528674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56412afe4850 0x56412afd0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:02.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.613+0000 7f9cfab58640 1 --2- 172.21.15.76:0/643528674 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56412afe6f50 0x56412afe5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:02.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.613+0000 7f9cf9b56640 1 -- 172.21.15.76:0/643528674 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56412afe6f50 msgr2=0x56412afe5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:02.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.613+0000 7f9cf9b56640 1 --2- 172.21.15.76:0/643528674 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56412afe6f50 0x56412afe5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.613+0000 7f9cf9b56640 1 -- 172.21.15.76:0/643528674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56412afe4850 msgr2=0x56412afd0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:02.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.613+0000 7f9cf9b56640 1 --2- 172.21.15.76:0/643528674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56412afe4850 0x56412afd0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.613+0000 7f9cf9b56640 1 -- 172.21.15.76:0/643528674 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56412aee6ca0 con 0x56412afe4c30 2023-11-09T03:49:02.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.614+0000 7f9cf9b56640 1 --2- 172.21.15.76:0/643528674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56412afe4c30 0x56412afe5010 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x56412afb8190 tx=0x7f9cf0031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2204b6323e9b773 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:02.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.614+0000 7f9cd57fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:49:02.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.614+0000 7f9cd57fa640 1 -- 172.21.15.76:0/643528674 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0003cf0 con 0x56412afe4c30 2023-11-09T03:49:02.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.614+0000 7f9cd57fa640 1 -- 172.21.15.76:0/643528674 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f9cf0003e50 con 0x56412afe4c30 2023-11-09T03:49:02.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.614+0000 7f9cff22a740 1 -- 172.21.15.76:0/643528674 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ce8054c30 con 0x56412afe4c30 2023-11-09T03:49:02.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.614+0000 7f9cd57fa640 1 -- 172.21.15.76:0/643528674 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cf0004010 con 0x56412afe4c30 2023-11-09T03:49:02.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.614+0000 7f9cff22a740 10 client.7608 fetch_fsmap learned FSMap version 44 2023-11-09T03:49:02.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.614+0000 7f9cff22a740 1 -- 172.21.15.76:0/643528674 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x56412aee5500 con 0x56412afe4c30 2023-11-09T03:49:02.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.614+0000 7f9cd57fa640 1 -- 172.21.15.76:0/643528674 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce8054c30 con 0x56412afe4c30 2023-11-09T03:49:02.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.614+0000 7f9cd57fa640 1 -- 172.21.15.76:0/643528674 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9cf00541f0 con 0x56412afe4c30 2023-11-09T03:49:02.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.614+0000 7f9cff22a740 10 client.7608 fetch_fsmap finished waiting for FSMap version 44 2023-11-09T03:49:02.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.614+0000 7f9cff22a740 10 client.7608 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-09T03:49:02.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.614+0000 7f9cff22a740 20 client.7608 populate_metadata read hostname 'smithi076' 2023-11-09T03:49:02.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.614+0000 7f9cff22a740 1 --2- 172.21.15.76:0/643528674 >> [v2:172.21.15.76:6834/4011275120,v1:172.21.15.76:6835/4011275120] conn(0x56412b002ab0 0x56412b002e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:02.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.614+0000 7f9cff22a740 4 client.7608 mds_command: new command op to 7452 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-09T03:49:02.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.614+0000 7f9cff22a740 1 -- 172.21.15.76:0/643528674 --> [v2:172.21.15.76:6834/4011275120,v1:172.21.15.76:6835/4011275120] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56412af1c5b0 con 0x56412b002ab0 2023-11-09T03:49:02.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.614+0000 7f9cfa357640 1 --2- 172.21.15.76:0/643528674 >> [v2:172.21.15.76:6834/4011275120,v1:172.21.15.76:6835/4011275120] conn(0x56412b002ab0 0x56412b002e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:02.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.614+0000 7f9cd57fa640 10 client.7608 ms_handle_connect on v2:172.21.15.76:6834/4011275120 2023-11-09T03:49:02.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.614+0000 7f9cfa357640 1 --2- 172.21.15.76:0/643528674 >> [v2:172.21.15.76:6834/4011275120,v1:172.21.15.76:6835/4011275120] conn(0x56412b002ab0 0x56412b002e90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7cb3756be14eed server_cookie=78f82f83f8100c7c in_seq=0 out_seq=0 2023-11-09T03:49:02.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.615+0000 7f9cd57fa640 1 -- 172.21.15.76:0/643528674 <== mds.0 v2:172.21.15.76:6834/4011275120 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x56412af1c5b0 con 0x56412b002ab0 2023-11-09T03:49:02.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.615+0000 7f9cd57fa640 10 client.7608 handle_command_reply: tid=0 2023-11-09T03:49:02.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.615+0000 7f9cff22a740 1 client.7608 shutdown 2023-11-09T03:49:02.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.615+0000 7f9cff22a740 1 -- 172.21.15.76:0/643528674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56412afe4c30 msgr2=0x56412afe5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:02.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.615+0000 7f9cff22a740 1 --2- 172.21.15.76:0/643528674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56412afe4c30 0x56412afe5010 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x56412afb8190 tx=0x7f9cf0031000 comp rx=0 tx=0).stop 2023-11-09T03:49:02.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.615+0000 7f9cff22a740 1 -- 172.21.15.76:0/643528674 shutdown_connections 2023-11-09T03:49:02.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.615+0000 7f9cff22a740 1 --2- 172.21.15.76:0/643528674 >> [v2:172.21.15.76:6834/4011275120,v1:172.21.15.76:6835/4011275120] conn(0x56412b002ab0 0x56412b002e90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.615+0000 7f9cff22a740 1 --2- 172.21.15.76:0/643528674 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56412afe6f50 0x56412afe5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.615+0000 7f9cd57fa640 0 client.7608 ms_handle_reset on v2:172.21.15.76:6834/4011275120 2023-11-09T03:49:02.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.615+0000 7f9cff22a740 1 --2- 172.21.15.76:0/643528674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56412afe4c30 0x56412afe5010 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.615+0000 7f9cff22a740 1 --2- 172.21.15.76:0/643528674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56412afe4850 0x56412afd0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.615+0000 7f9cff22a740 1 -- 172.21.15.76:0/643528674 >> 172.21.15.76:0/643528674 conn(0x56412afb7c90 msgr2=0x56412afddc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:02.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.615+0000 7f9cff22a740 1 -- 172.21.15.76:0/643528674 shutdown_connections 2023-11-09T03:49:02.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.615+0000 7f9cff22a740 1 -- 172.21.15.76:0/643528674 wait complete. 2023-11-09T03:49:02.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.615+0000 7f9cff22a740 20 client.7608 trim_cache size 0 max 16384 2023-11-09T03:49:02.635 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:49:02.635 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "c213b26f-d0da-45d5-8e34-bb75c096d4f2", 2023-11-09T03:49:02.635 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2023-11-09T03:49:02.635 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 7452, 2023-11-09T03:49:02.635 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2023-11-09T03:49:02.636 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2023-11-09T03:49:02.636 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2023-11-09T03:49:02.636 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 4.5733920430000001, 2023-11-09T03:49:02.636 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 44, 2023-11-09T03:49:02.636 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 95, 2023-11-09T03:49:02.636 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 95, 2023-11-09T03:49:02.637 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 10.616578884999999 2023-11-09T03:49:02.637 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:49:02.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.615+0000 7f9cfc5bb640 1 -- 172.21.15.76:0/609753808 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9cd806ae70 msgr2=0x7f9cd806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:02.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.615+0000 7f9cfc5bb640 1 --2- 172.21.15.76:0/609753808 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9cd806ae70 0x7f9cd806d330 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0036000 tx=0x7f9cf003c040 comp rx=0 tx=0).stop 2023-11-09T03:49:02.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.615+0000 7f9cfc5bb640 1 -- 172.21.15.76:0/609753808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cf40adac0 msgr2=0x7f9cf40c6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:02.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.615+0000 7f9cfc5bb640 1 --2- 172.21.15.76:0/609753808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cf40adac0 0x7f9cf40c6ec0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f9ce800c7f0 tx=0x7f9ce800cbf0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.615+0000 7f9cfc5bb640 1 -- 172.21.15.76:0/609753808 shutdown_connections 2023-11-09T03:49:02.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.615+0000 7f9cfc5bb640 1 --2- 172.21.15.76:0/609753808 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9cd806ae70 0x7f9cd806d330 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.615+0000 7f9cfc5bb640 1 --2- 172.21.15.76:0/609753808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cf4159c00 0x7f9cf4156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.616+0000 7f9cfc5bb640 1 --2- 172.21.15.76:0/609753808 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cf40c48e0 0x7f9cf4142540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.616+0000 7f9cfc5bb640 1 --2- 172.21.15.76:0/609753808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cf40adac0 0x7f9cf40c6ec0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.616+0000 7f9cfc5bb640 1 -- 172.21.15.76:0/609753808 >> 172.21.15.76:0/609753808 conn(0x7f9cf40ce950 msgr2=0x7f9cf40c7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:02.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.616+0000 7f9cfc5bb640 1 -- 172.21.15.76:0/609753808 shutdown_connections 2023-11-09T03:49:02.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.616+0000 7f9cfc5bb640 1 -- 172.21.15.76:0/609753808 wait complete. 2023-11-09T03:49:02.640 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-09T03:49:02.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.740+0000 7fad2cbf5640 1 Processor -- start 2023-11-09T03:49:02.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.740+0000 7fad2cbf5640 1 -- start start 2023-11-09T03:49:02.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.741+0000 7fad2cbf5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fad280cda60 0x7fad280ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:02.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.741+0000 7fad2cbf5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad280b0cd0 0x7fad28070630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:02.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.741+0000 7fad2cbf5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad280cc990 con 0x7fad280cace0 2023-11-09T03:49:02.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.741+0000 7fad2cbf5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad280ccc70 con 0x7fad280cda60 2023-11-09T03:49:02.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.741+0000 7fad2cbf5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad280c9ce0 con 0x7fad280b0cd0 2023-11-09T03:49:02.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.742+0000 7fad2659c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fad280cda60 0x7fad280ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:02.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.742+0000 7fad26d9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fad280cace0 0x7fad2806c200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53252/0 (socket says 172.21.15.76:53252) 2023-11-09T03:49:02.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.742+0000 7fad26d9d640 1 -- 172.21.15.76:0/2808263551 learned_addr learned my addr 172.21.15.76:0/2808263551 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:49:02.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.742+0000 7fad25d9b640 1 --2- 172.21.15.76:0/2808263551 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad280b0cd0 0x7fad28070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:02.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.742+0000 7fad2659c640 1 -- 172.21.15.76:0/2808263551 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad280b0cd0 msgr2=0x7fad28070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:02.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.742+0000 7fad2659c640 1 --2- 172.21.15.76:0/2808263551 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad280b0cd0 0x7fad28070630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.742+0000 7fad2659c640 1 -- 172.21.15.76:0/2808263551 >> v1:172.21.15.76:6789/0 conn(0x7fad280cace0 legacy=0x7fad2806c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:02.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.742+0000 7fad2659c640 1 -- 172.21.15.76:0/2808263551 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad280c9fc0 con 0x7fad280cda60 2023-11-09T03:49:02.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.742+0000 7fad2559a640 1 -- 172.21.15.76:0/2808263551 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3057839998 0 0) 0x7fad280cc990 con 0x7fad280cace0 2023-11-09T03:49:02.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.743+0000 7fad2659c640 1 --2- 172.21.15.76:0/2808263551 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad280cda60 0x7fad280ca7a0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fad1c0097a0 tx=0x7fad1c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcae2521275ab582 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:02.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.743+0000 7fad2559a640 1 -- 172.21.15.76:0/2808263551 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c031830 con 0x7fad280cda60 2023-11-09T03:49:02.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.743+0000 7fad2559a640 1 -- 172.21.15.76:0/2808263551 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1c031990 con 0x7fad280cda60 2023-11-09T03:49:02.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.743+0000 7fad2559a640 1 -- 172.21.15.76:0/2808263551 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c031d20 con 0x7fad280cda60 2023-11-09T03:49:02.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.743+0000 7fad2cbf5640 1 -- 172.21.15.76:0/2808263551 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad280cda60 msgr2=0x7fad280ca7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:02.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.743+0000 7fad2cbf5640 1 --2- 172.21.15.76:0/2808263551 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad280cda60 0x7fad280ca7a0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fad1c0097a0 tx=0x7fad1c02b8d0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.743+0000 7fad2cbf5640 1 -- 172.21.15.76:0/2808263551 shutdown_connections 2023-11-09T03:49:02.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.743+0000 7fad2cbf5640 1 --2- 172.21.15.76:0/2808263551 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad280b0cd0 0x7fad28070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.743+0000 7fad2cbf5640 1 --2- 172.21.15.76:0/2808263551 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad280cda60 0x7fad280ca7a0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.743+0000 7fad2cbf5640 1 -- 172.21.15.76:0/2808263551 >> 172.21.15.76:0/2808263551 conn(0x7fad280ce950 msgr2=0x7fad280ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:02.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.743+0000 7fad2cbf5640 1 -- 172.21.15.76:0/2808263551 shutdown_connections 2023-11-09T03:49:02.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.743+0000 7fad2cbf5640 1 -- 172.21.15.76:0/2808263551 wait complete. 2023-11-09T03:49:02.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.745+0000 7fad2cbf5640 1 Processor -- start 2023-11-09T03:49:02.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.745+0000 7fad2cbf5640 1 -- start start 2023-11-09T03:49:02.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.745+0000 7fad2cbf5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad280b0cd0 0x7fad280c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:02.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.745+0000 7fad2cbf5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad280cace0 0x7fad281f4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:02.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.745+0000 7fad2cbf5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad280cda60 0x7fad281f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:02.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.745+0000 7fad2cbf5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2814dde0 con 0x7fad280cace0 2023-11-09T03:49:02.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.745+0000 7fad2cbf5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2806d630 con 0x7fad280cda60 2023-11-09T03:49:02.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.745+0000 7fad2cbf5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2814fae0 con 0x7fad280b0cd0 2023-11-09T03:49:02.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.745+0000 7fad25d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad280cace0 0x7fad281f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:02.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.745+0000 7fad25d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad280cace0 0x7fad281f4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38272/0 (socket says 172.21.15.76:38272) 2023-11-09T03:49:02.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.745+0000 7fad25d9b640 1 -- 172.21.15.76:0/4284460009 learned_addr learned my addr 172.21.15.76:0/4284460009 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:02.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.746+0000 7fad2659c640 1 --2- 172.21.15.76:0/4284460009 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad280b0cd0 0x7fad280c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:02.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.746+0000 7fad26d9d640 1 --2- 172.21.15.76:0/4284460009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad280cda60 0x7fad281f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:02.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.746+0000 7fad25d9b640 1 -- 172.21.15.76:0/4284460009 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad280b0cd0 msgr2=0x7fad280c7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:02.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.746+0000 7fad25d9b640 1 --2- 172.21.15.76:0/4284460009 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad280b0cd0 0x7fad280c7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.746+0000 7fad25d9b640 1 -- 172.21.15.76:0/4284460009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad280cda60 msgr2=0x7fad281f49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:02.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.746+0000 7fad25d9b640 1 --2- 172.21.15.76:0/4284460009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad280cda60 0x7fad281f49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.746+0000 7fad25d9b640 1 -- 172.21.15.76:0/4284460009 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1c009400 con 0x7fad280cace0 2023-11-09T03:49:02.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.746+0000 7fad2659c640 1 --2- 172.21.15.76:0/4284460009 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad280b0cd0 0x7fad280c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:02.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.746+0000 7fad26d9d640 1 --2- 172.21.15.76:0/4284460009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad280cda60 0x7fad281f49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:02.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.746+0000 7fad25d9b640 1 --2- 172.21.15.76:0/4284460009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad280cace0 0x7fad281f4480 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fad1800d800 tx=0x7fad1800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:02.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.746+0000 7fad16ffd640 1 -- 172.21.15.76:0/4284460009 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1800b6c0 con 0x7fad280cace0 2023-11-09T03:49:02.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.746+0000 7fad16ffd640 1 -- 172.21.15.76:0/4284460009 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1800b820 con 0x7fad280cace0 2023-11-09T03:49:02.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.746+0000 7fad2cbf5640 1 -- 172.21.15.76:0/4284460009 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad281507b0 con 0x7fad280cace0 2023-11-09T03:49:02.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.746+0000 7fad16ffd640 1 -- 172.21.15.76:0/4284460009 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1800bbb0 con 0x7fad280cace0 2023-11-09T03:49:02.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.746+0000 7fad2cbf5640 1 -- 172.21.15.76:0/4284460009 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad280c83f0 con 0x7fad280cace0 2023-11-09T03:49:02.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.747+0000 7fad2cbf5640 1 -- 172.21.15.76:0/4284460009 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad28074290 con 0x7fad280cace0 2023-11-09T03:49:02.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.747+0000 7fad16ffd640 1 -- 172.21.15.76:0/4284460009 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fad1800bd10 con 0x7fad280cace0 2023-11-09T03:49:02.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.748+0000 7fad16ffd640 1 --2- 172.21.15.76:0/4284460009 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fad0806ae70 0x7fad0806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:02.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.748+0000 7fad16ffd640 1 -- 172.21.15.76:0/4284460009 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7120+0+0 (secure 0 0 0) 0x7fad1808a0f0 con 0x7fad280cace0 2023-11-09T03:49:02.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.748+0000 7fad2659c640 1 --2- 172.21.15.76:0/4284460009 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fad0806ae70 0x7fad0806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:02.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.749+0000 7fad2659c640 1 --2- 172.21.15.76:0/4284460009 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fad0806ae70 0x7fad0806d330 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fad1c036000 tx=0x7fad1c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:02.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.752+0000 7fad16ffd640 1 -- 172.21.15.76:0/4284460009 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fad18054c20 con 0x7fad280cace0 2023-11-09T03:49:02.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.954+0000 7fad2cbf5640 1 -- 172.21.15.76:0/4284460009 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fad280b48d0 con 0x7fad280cace0 2023-11-09T03:49:02.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.954+0000 7fad16ffd640 1 -- 172.21.15.76:0/4284460009 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) v1 ==== 53+0+73 (secure 0 0 0) 0x7fad18058240 con 0x7fad280cace0 2023-11-09T03:49:02.954 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-09T03:49:02.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.956+0000 7fad2cbf5640 1 -- 172.21.15.76:0/4284460009 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fad0806ae70 msgr2=0x7fad0806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:02.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.956+0000 7fad2cbf5640 1 --2- 172.21.15.76:0/4284460009 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fad0806ae70 0x7fad0806d330 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fad1c036000 tx=0x7fad1c008030 comp rx=0 tx=0).stop 2023-11-09T03:49:02.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.956+0000 7fad2cbf5640 1 -- 172.21.15.76:0/4284460009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad280cace0 msgr2=0x7fad281f4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:02.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.956+0000 7fad2cbf5640 1 --2- 172.21.15.76:0/4284460009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad280cace0 0x7fad281f4480 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fad1800d800 tx=0x7fad1800dcc0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.956+0000 7fad2cbf5640 1 -- 172.21.15.76:0/4284460009 shutdown_connections 2023-11-09T03:49:02.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.956+0000 7fad2cbf5640 1 --2- 172.21.15.76:0/4284460009 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fad0806ae70 0x7fad0806d330 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.956+0000 7fad2cbf5640 1 --2- 172.21.15.76:0/4284460009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad280cda60 0x7fad281f49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.956+0000 7fad2cbf5640 1 --2- 172.21.15.76:0/4284460009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad280cace0 0x7fad281f4480 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.956+0000 7fad2cbf5640 1 --2- 172.21.15.76:0/4284460009 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad280b0cd0 0x7fad280c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:02.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.956+0000 7fad2cbf5640 1 -- 172.21.15.76:0/4284460009 >> 172.21.15.76:0/4284460009 conn(0x7fad280ce950 msgr2=0x7fad2815bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:02.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.956+0000 7fad2cbf5640 1 -- 172.21.15.76:0/4284460009 shutdown_connections 2023-11-09T03:49:02.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:02.956+0000 7fad2cbf5640 1 -- 172.21.15.76:0/4284460009 wait complete. 2023-11-09T03:49:02.966 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-09T03:49:02.966 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-09T03:49:02.966 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2023-11-09T03:49:02.967 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-09T03:49:02.967 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-09T03:49:02.967 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-09T03:49:02.967 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-09T03:49:02.968 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-09T03:49:02.968 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:49:02.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:49:02.969 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2023-11-09T03:49:02.984 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-09T03:49:02.984 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-09T03:49:02.984 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2023-11-09T03:49:02.985 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:49:02.985 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2023-11-09T03:49:02.985 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:49:02.985 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:49:02.985 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:49:02.986 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2023-11-09T03:49:02.986 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:49:02.986 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:49:02.986 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2023-11-09T03:49:02.986 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:49:02.986 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:49:02.987 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-09T03:49:02.987 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:49:02.987 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-09T03:49:02.987 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42406sec preferred_lft 42406sec 2023-11-09T03:49:02.987 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2023-11-09T03:49:02.988 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:49:02.988 INFO:teuthology.orchestra.run.smithi076.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-09T03:49:02.988 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2e:15:bd:03:a8:2e brd ff:ff:ff:ff:ff:ff 2023-11-09T03:49:02.988 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-09T03:49:02.988 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:49:02.988 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::bcda:38ff:fed4:fdb0/64 scope link 2023-11-09T03:49:02.989 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:49:02.989 INFO:teuthology.orchestra.run.smithi076.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-09T03:49:02.989 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 6a:8d:94:d7:e5:00 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:49:02.989 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::688d:94ff:fed7:e500/64 scope link 2023-11-09T03:49:02.989 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:49:02.989 INFO:teuthology.orchestra.run.smithi076.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-09T03:49:02.990 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2e:15:bd:03:a8:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:49:02.990 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::2c15:bdff:fe03:a82e/64 scope link 2023-11-09T03:49:02.990 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:49:02.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:49:02.991 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-09T03:49:03.050 INFO:teuthology.orchestra.run.smithi076.stdout:1 2023-11-09T03:49:03.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:49:03.052 DEBUG:teuthology.orchestra.run.smithi076:> route 2023-11-09T03:49:03.107 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2023-11-09T03:49:03.107 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-09T03:49:03.107 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-09T03:49:03.107 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-09T03:49:03.107 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-09T03:49:03.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:49:03.108 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-09T03:49:03.109 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-09T03:49:03.109 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-09T03:49:03.109 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-09T03:49:03.110 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-09T03:49:03.110 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-09T03:49:03.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:49:03.202 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2023-11-09T03:49:03.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-09T03:49:03.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-09T03:49:03.217 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:49:03.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:49:03.218 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2023-11-09T03:49:03.284 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-09T03:49:03.284 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:49:03.285 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-09T03:49:03.339 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:49:03.339 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:49:03.339 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-09T03:49:03.393 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:49:03.448 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:49:03.449 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:49:03.449 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:49:03.449 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:49:03.449 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:49:03.449 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:49:03.449 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:49:03.450 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:49:03.450 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:49:03.450 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:49:03.450 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:49:03.450 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:49:03.451 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:49:03.451 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:49:03.451 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:49:03.451 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:49:03.451 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:49:03.451 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:49:03.452 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:49:03.452 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:49:03.452 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:49:03.452 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:49:03.452 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:49:03.452 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:49:03.453 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:49:03.453 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:49:03.453 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:49:03.453 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:49:03.453 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:49:03.453 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:49:03.454 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:49:03.454 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:49:03.454 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:49:03.454 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:49:03.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:49:03.455 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:49:03.503 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-09T03:49:03.504 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-09T03:49:03.546 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:49:03.561 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:49:03.562 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:49:03.562 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:49:03.562 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:49:03.562 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:49:03.562 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:49:03.562 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:49:03.563 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:49:03.563 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:49:03.563 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:49:03.563 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:49:03.563 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:49:03.564 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:49:03.564 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:49:03.564 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:49:03.564 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:49:03.564 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:49:03.564 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:49:03.565 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:49:03.565 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:49:03.565 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:49:03.565 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:49:03.565 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:49:03.566 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:49:03.566 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:49:03.566 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:49:03.566 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:49:03.566 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:49:03.566 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:49:03.567 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:49:03.567 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:49:03.567 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:49:03.567 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:49:03.567 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:49:03.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:49:03.568 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:49:03.636 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:49:03.637+0000 7fe10a733480 -1 init, newargv = 0x5652827ec8e0 newargc=15 2023-11-09T03:49:03.636 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[117777]: starting ceph client 2023-11-09T03:49:03.640 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[117777]: starting fuse 2023-11-09T03:49:04.021 INFO:tasks.ceph.mds.c.smithi076.stderr:2023-11-09T03:49:04.022+0000 7f65da07c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:49:04.718 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:49:04.737 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:49:04.738 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:49:04.738 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:49:04.738 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:49:04.738 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:49:04.738 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:49:04.739 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:49:04.739 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:49:04.739 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:49:04.739 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:49:04.739 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:49:04.739 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:49:04.740 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:49:04.740 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:49:04.740 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:49:04.740 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:49:04.740 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:49:04.741 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:49:04.741 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:49:04.741 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:49:04.741 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:49:04.741 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:49:04.742 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:49:04.742 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:49:04.742 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:49:04.742 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:49:04.742 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:49:04.742 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:49:04.743 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:49:04.743 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:49:04.743 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:49:04.743 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:49:04.743 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:49:04.743 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:49:04.744 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-09T03:49:04.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:49:04.745 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:49:04.802 INFO:teuthology.orchestra.run.smithi076.stdout:43 2023-11-09T03:49:04.803 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-09T03:49:04.803 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer -- python3 -c ' 2023-11-09T03:49:04.803 DEBUG:teuthology.orchestra.run.smithi076:> import glob 2023-11-09T03:49:04.804 DEBUG:teuthology.orchestra.run.smithi076:> import re 2023-11-09T03:49:04.804 DEBUG:teuthology.orchestra.run.smithi076:> import os 2023-11-09T03:49:04.804 DEBUG:teuthology.orchestra.run.smithi076:> import subprocess 2023-11-09T03:49:04.805 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:49:04.805 DEBUG:teuthology.orchestra.run.smithi076:> def _find_admin_socket(client_name): 2023-11-09T03:49:04.805 DEBUG:teuthology.orchestra.run.smithi076:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-09T03:49:04.806 DEBUG:teuthology.orchestra.run.smithi076:> files = glob.glob(asok_path) 2023-11-09T03:49:04.806 DEBUG:teuthology.orchestra.run.smithi076:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-09T03:49:04.806 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:49:04.807 DEBUG:teuthology.orchestra.run.smithi076:> # Given a non-glob path, it better be there 2023-11-09T03:49:04.807 DEBUG:teuthology.orchestra.run.smithi076:> if "*" not in asok_path: 2023-11-09T03:49:04.807 DEBUG:teuthology.orchestra.run.smithi076:> assert(len(files) == 1) 2023-11-09T03:49:04.808 DEBUG:teuthology.orchestra.run.smithi076:> return files[0] 2023-11-09T03:49:04.808 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:49:04.808 DEBUG:teuthology.orchestra.run.smithi076:> for f in files: 2023-11-09T03:49:04.809 DEBUG:teuthology.orchestra.run.smithi076:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-09T03:49:04.809 DEBUG:teuthology.orchestra.run.smithi076:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-09T03:49:04.810 DEBUG:teuthology.orchestra.run.smithi076:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-09T03:49:04.810 DEBUG:teuthology.orchestra.run.smithi076:> contents = proc_f.read() 2023-11-09T03:49:04.811 DEBUG:teuthology.orchestra.run.smithi076:> if mountpoint in contents: 2023-11-09T03:49:04.811 DEBUG:teuthology.orchestra.run.smithi076:> return f 2023-11-09T03:49:04.812 DEBUG:teuthology.orchestra.run.smithi076:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-09T03:49:04.812 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:49:04.813 DEBUG:teuthology.orchestra.run.smithi076:> print(_find_admin_socket("client.0")) 2023-11-09T03:49:04.813 DEBUG:teuthology.orchestra.run.smithi076:> ' 2023-11-09T03:49:04.935 INFO:teuthology.orchestra.run.smithi076.stdout:/var/run/ceph/ceph-client.0.117777.asok 2023-11-09T03:49:04.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:49:04.947 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.117777.asok 2023-11-09T03:49:04.947 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:49:04.948 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.117777.asok status 2023-11-09T03:49:05.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:49:05.120 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:49:05.120 INFO:teuthology.orchestra.run.smithi076.stdout: "metadata": { 2023-11-09T03:49:05.120 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_sha1": "7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4", 2023-11-09T03:49:05.120 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev)", 2023-11-09T03:49:05.121 INFO:teuthology.orchestra.run.smithi076.stdout: "entity_id": "0", 2023-11-09T03:49:05.121 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": "smithi076", 2023-11-09T03:49:05.121 INFO:teuthology.orchestra.run.smithi076.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-09T03:49:05.121 INFO:teuthology.orchestra.run.smithi076.stdout: "pid": "117777", 2023-11-09T03:49:05.121 INFO:teuthology.orchestra.run.smithi076.stdout: "root": "/" 2023-11-09T03:49:05.121 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:49:05.122 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_count": 0, 2023-11-09T03:49:05.122 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_pinned_count": 0, 2023-11-09T03:49:05.122 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 7626, 2023-11-09T03:49:05.122 INFO:teuthology.orchestra.run.smithi076.stdout: "inst": { 2023-11-09T03:49:05.122 INFO:teuthology.orchestra.run.smithi076.stdout: "name": { 2023-11-09T03:49:05.122 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "client", 2023-11-09T03:49:05.123 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 7626 2023-11-09T03:49:05.123 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:49:05.123 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2023-11-09T03:49:05.123 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2023-11-09T03:49:05.123 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2023-11-09T03:49:05.124 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 503424885 2023-11-09T03:49:05.124 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:49:05.124 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:49:05.124 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2023-11-09T03:49:05.124 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2023-11-09T03:49:05.124 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2023-11-09T03:49:05.125 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 503424885 2023-11-09T03:49:05.125 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:49:05.125 INFO:teuthology.orchestra.run.smithi076.stdout: "inst_str": "client.7626 192.168.0.1:0/503424885", 2023-11-09T03:49:05.125 INFO:teuthology.orchestra.run.smithi076.stdout: "addr_str": "192.168.0.1:0/503424885", 2023-11-09T03:49:05.125 INFO:teuthology.orchestra.run.smithi076.stdout: "inode_count": 1, 2023-11-09T03:49:05.125 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_epoch": 44, 2023-11-09T03:49:05.126 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch": 95, 2023-11-09T03:49:05.126 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch_barrier": 0, 2023-11-09T03:49:05.126 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklisted": false, 2023-11-09T03:49:05.126 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs" 2023-11-09T03:49:05.126 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:49:05.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:49:05.134 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:49:05.135 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-11-09T03:49:05.135 DEBUG:teuthology.orchestra.run.smithi076:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-09T03:49:05.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:49:05.193 INFO:teuthology.orchestra.run.smithi076.stdout:fuseblk 2023-11-09T03:49:05.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:49:05.200 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-09T03:49:05.200 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:49:05.201 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-09T03:49:05.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:49:05.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:49:05.267 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_max_snaps_per_dir 2023-11-09T03:49:05.378 INFO:teuthology.orchestra.run.smithi076.stdout:{"mds_max_snaps_per_dir":"100"} 2023-11-09T03:49:05.379 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_max_snaps_per_dir": "100" } 2023-11-09T03:49:05.379 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2023-11-09T03:49:05.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.486+0000 7f6144e07640 1 Processor -- start 2023-11-09T03:49:05.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.486+0000 7f6144e07640 1 -- start start 2023-11-09T03:49:05.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.487+0000 7f6144e07640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61400cd9f0 0x7f61400cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:05.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.487+0000 7f6144e07640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61400d1320 con 0x7f614015d4b0 2023-11-09T03:49:05.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.487+0000 7f6144e07640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61400b5600 con 0x7f61400d27c0 2023-11-09T03:49:05.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.487+0000 7f6144e07640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f614015e4d0 con 0x7f61400cd9f0 2023-11-09T03:49:05.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.487+0000 7f613dd9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f614015d4b0 0x7f61400b47e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53266/0 (socket says 172.21.15.76:53266) 2023-11-09T03:49:05.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.487+0000 7f613dd9b640 1 -- 172.21.15.76:0/3419578447 learned_addr learned my addr 172.21.15.76:0/3419578447 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:49:05.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.488+0000 7f613d59a640 1 -- 172.21.15.76:0/3419578447 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 281450596 0 0) 0x7f61400d1320 con 0x7f614015d4b0 2023-11-09T03:49:05.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.488+0000 7f613d59a640 1 -- 172.21.15.76:0/3419578447 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6128003520 con 0x7f614015d4b0 2023-11-09T03:49:05.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.488+0000 7f613e59c640 1 --2- 172.21.15.76:0/3419578447 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61400cd9f0 0x7f61400cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:05.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.488+0000 7f613d59a640 1 -- 172.21.15.76:0/3419578447 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1013364234 0 0) 0x7f6128003520 con 0x7f614015d4b0 2023-11-09T03:49:05.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.488+0000 7f613d59a640 1 -- 172.21.15.76:0/3419578447 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61400cd9f0 msgr2=0x7f61400cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:05.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.488+0000 7f613d59a640 1 --2- 172.21.15.76:0/3419578447 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61400cd9f0 0x7f61400cddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:05.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.488+0000 7f613d59a640 1 -- 172.21.15.76:0/3419578447 >> v1:172.21.15.113:6789/0 conn(0x7f61400d27c0 legacy=0x7f61400b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:05.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.488+0000 7f613d59a640 1 -- 172.21.15.76:0/3419578447 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61400ce310 con 0x7f614015d4b0 2023-11-09T03:49:05.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.488+0000 7f613d59a640 1 -- 172.21.15.76:0/3419578447 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f6134001670 con 0x7f614015d4b0 2023-11-09T03:49:05.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.488+0000 7f613d59a640 1 -- 172.21.15.76:0/3419578447 >> v1:172.21.15.76:6789/0 conn(0x7f614015d4b0 legacy=0x7f61400b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:05.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.488+0000 7f613d59a640 1 --2- 172.21.15.76:0/3419578447 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61400cd9f0 0x7f61280039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:05.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.488+0000 7f613d59a640 1 --2- 172.21.15.76:0/3419578447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6128003f20 0x7f61280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:05.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.488+0000 7f613d59a640 1 --2- 172.21.15.76:0/3419578447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6128004d00 0x7f6128005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:05.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.488+0000 7f613d59a640 1 -- 172.21.15.76:0/3419578447 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614015e4d0 con 0x7f6128003f20 2023-11-09T03:49:05.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.488+0000 7f613d59a640 1 -- 172.21.15.76:0/3419578447 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612800baf0 con 0x7f6128004d00 2023-11-09T03:49:05.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.488+0000 7f613d59a640 1 -- 172.21.15.76:0/3419578447 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612800bcb0 con 0x7f61400cd9f0 2023-11-09T03:49:05.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.488+0000 7f613dd9b640 1 --2- 172.21.15.76:0/3419578447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6128003f20 0x7f61280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:05.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.488+0000 7f613ed9d640 1 --2- 172.21.15.76:0/3419578447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6128004d00 0x7f6128005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:05.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.488+0000 7f613e59c640 1 --2- 172.21.15.76:0/3419578447 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61400cd9f0 0x7f61280039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:05.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.488+0000 7f613dd9b640 1 -- 172.21.15.76:0/3419578447 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61400cd9f0 msgr2=0x7f61280039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:05.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.488+0000 7f613dd9b640 1 --2- 172.21.15.76:0/3419578447 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61400cd9f0 0x7f61280039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:05.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.488+0000 7f613dd9b640 1 -- 172.21.15.76:0/3419578447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6128004d00 msgr2=0x7f6128005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:05.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.488+0000 7f613dd9b640 1 --2- 172.21.15.76:0/3419578447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6128004d00 0x7f6128005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:05.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.488+0000 7f613dd9b640 1 -- 172.21.15.76:0/3419578447 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6128003520 con 0x7f6128003f20 2023-11-09T03:49:05.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.489+0000 7f613dd9b640 1 --2- 172.21.15.76:0/3419578447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6128003f20 0x7f61280047c0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f613400c240 tx=0x7f61280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f669dcf966b15b84 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:05.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.489+0000 7f613d59a640 1 -- 172.21.15.76:0/3419578447 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6134005ad0 con 0x7f6128003f20 2023-11-09T03:49:05.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.489+0000 7f613d59a640 1 -- 172.21.15.76:0/3419578447 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6134005c30 con 0x7f6128003f20 2023-11-09T03:49:05.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.489+0000 7f613d59a640 1 -- 172.21.15.76:0/3419578447 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613402caf0 con 0x7f6128003f20 2023-11-09T03:49:05.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.490+0000 7f6144e07640 1 -- 172.21.15.76:0/3419578447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6128003f20 msgr2=0x7f61280047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:05.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.490+0000 7f6144e07640 1 --2- 172.21.15.76:0/3419578447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6128003f20 0x7f61280047c0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f613400c240 tx=0x7f61280057f0 comp rx=0 tx=0).stop 2023-11-09T03:49:05.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.490+0000 7f613ed9d640 1 -- 172.21.15.76:0/3419578447 reap_dead start 2023-11-09T03:49:05.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.490+0000 7f6144e07640 1 -- 172.21.15.76:0/3419578447 shutdown_connections 2023-11-09T03:49:05.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.490+0000 7f6144e07640 1 -- 172.21.15.76:0/3419578447 >> 172.21.15.76:0/3419578447 conn(0x7f6140071c40 msgr2=0x7f6140072050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:05.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.490+0000 7f6144e07640 1 -- 172.21.15.76:0/3419578447 shutdown_connections 2023-11-09T03:49:05.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.490+0000 7f6144e07640 1 -- 172.21.15.76:0/3419578447 wait complete. 2023-11-09T03:49:05.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.491+0000 7f6144e07640 1 Processor -- start 2023-11-09T03:49:05.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.491+0000 7f6144e07640 1 -- start start 2023-11-09T03:49:05.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.492+0000 7f6144e07640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6128003f20 0x7f6140151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:05.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.492+0000 7f6144e07640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6140152070 0x7f61400b3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:05.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.492+0000 7f6144e07640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61400b3d50 0x7f6140156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:05.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.492+0000 7f6144e07640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6140151e90 con 0x7f61400b3d50 2023-11-09T03:49:05.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.492+0000 7f6144e07640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61401558d0 con 0x7f6140152070 2023-11-09T03:49:05.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.492+0000 7f6144e07640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61400cb4c0 con 0x7f6128003f20 2023-11-09T03:49:05.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.492+0000 7f613ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61400b3d50 0x7f6140156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:05.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.492+0000 7f613ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61400b3d50 0x7f6140156000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38296/0 (socket says 172.21.15.76:38296) 2023-11-09T03:49:05.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.492+0000 7f613ed9d640 1 -- 172.21.15.76:0/710553723 learned_addr learned my addr 172.21.15.76:0/710553723 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:05.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.492+0000 7f613dd9b640 1 --2- 172.21.15.76:0/710553723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6140152070 0x7f61400b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:05.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.492+0000 7f613e59c640 1 --2- 172.21.15.76:0/710553723 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6128003f20 0x7f6140151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:05.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.492+0000 7f613ed9d640 1 -- 172.21.15.76:0/710553723 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6128003f20 msgr2=0x7f6140151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:05.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.492+0000 7f613ed9d640 1 --2- 172.21.15.76:0/710553723 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6128003f20 0x7f6140151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:05.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.492+0000 7f613ed9d640 1 -- 172.21.15.76:0/710553723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6140152070 msgr2=0x7f61400b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:05.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.492+0000 7f613ed9d640 1 --2- 172.21.15.76:0/710553723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6140152070 0x7f61400b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:05.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.492+0000 7f613ed9d640 1 -- 172.21.15.76:0/710553723 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61400cb760 con 0x7f61400b3d50 2023-11-09T03:49:05.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.493+0000 7f613dd9b640 1 --2- 172.21.15.76:0/710553723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6140152070 0x7f61400b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:05.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.493+0000 7f613e59c640 1 --2- 172.21.15.76:0/710553723 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6128003f20 0x7f6140151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:05.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.493+0000 7f613ed9d640 1 --2- 172.21.15.76:0/710553723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61400b3d50 0x7f6140156000 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f61400d3be0 tx=0x7f613800f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:05.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.493+0000 7f612effd640 1 -- 172.21.15.76:0/710553723 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61380053a0 con 0x7f61400b3d50 2023-11-09T03:49:05.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.493+0000 7f612effd640 1 -- 172.21.15.76:0/710553723 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61380035f0 con 0x7f61400b3d50 2023-11-09T03:49:05.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.493+0000 7f6144e07640 1 -- 172.21.15.76:0/710553723 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61400cba40 con 0x7f61400b3d50 2023-11-09T03:49:05.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.493+0000 7f612effd640 1 -- 172.21.15.76:0/710553723 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61380039a0 con 0x7f61400b3d50 2023-11-09T03:49:05.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.493+0000 7f6144e07640 1 -- 172.21.15.76:0/710553723 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61400b4360 con 0x7f61400b3d50 2023-11-09T03:49:05.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.494+0000 7f6144e07640 1 -- 172.21.15.76:0/710553723 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61400cbf60 con 0x7f61400b3d50 2023-11-09T03:49:05.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.494+0000 7f612effd640 1 -- 172.21.15.76:0/710553723 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6138003b00 con 0x7f61400b3d50 2023-11-09T03:49:05.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.495+0000 7f612effd640 1 --2- 172.21.15.76:0/710553723 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f612006aea0 0x7f612006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:05.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.495+0000 7f612effd640 1 -- 172.21.15.76:0/710553723 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f61380897c0 con 0x7f61400b3d50 2023-11-09T03:49:05.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.495+0000 7f613e59c640 1 --2- 172.21.15.76:0/710553723 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f612006aea0 0x7f612006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:05.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.495+0000 7f613e59c640 1 --2- 172.21.15.76:0/710553723 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f612006aea0 0x7f612006d360 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f61300074d0 tx=0x7f613000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:05.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.499+0000 7f612effd640 1 -- 172.21.15.76:0/710553723 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f61400cbf60 con 0x7f61400b3d50 2023-11-09T03:49:05.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:05.701+0000 7f6144e07640 1 -- 172.21.15.76:0/710553723 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f61400cbf60 con 0x7f61400b3d50 2023-11-09T03:49:06.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.432+0000 7f612effd640 1 -- 172.21.15.76:0/710553723 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v45) v1 ==== 167+0+0 (secure 0 0 0) 0x7f61400cbf60 con 0x7f61400b3d50 2023-11-09T03:49:06.432 INFO:teuthology.orchestra.run.smithi076.stderr:enabled new snapshots 2023-11-09T03:49:06.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.436+0000 7f6144e07640 1 -- 172.21.15.76:0/710553723 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f612006aea0 msgr2=0x7f612006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:06.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.436+0000 7f6144e07640 1 --2- 172.21.15.76:0/710553723 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f612006aea0 0x7f612006d360 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f61300074d0 tx=0x7f613000b040 comp rx=0 tx=0).stop 2023-11-09T03:49:06.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.436+0000 7f6144e07640 1 -- 172.21.15.76:0/710553723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61400b3d50 msgr2=0x7f6140156000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:06.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.436+0000 7f6144e07640 1 --2- 172.21.15.76:0/710553723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61400b3d50 0x7f6140156000 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f61400d3be0 tx=0x7f613800f920 comp rx=0 tx=0).stop 2023-11-09T03:49:06.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.436+0000 7f6144e07640 1 -- 172.21.15.76:0/710553723 shutdown_connections 2023-11-09T03:49:06.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.436+0000 7f6144e07640 1 --2- 172.21.15.76:0/710553723 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f612006aea0 0x7f612006d360 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:06.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.436+0000 7f6144e07640 1 --2- 172.21.15.76:0/710553723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61400b3d50 0x7f6140156000 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:06.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.436+0000 7f6144e07640 1 --2- 172.21.15.76:0/710553723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6140152070 0x7f61400b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:06.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.436+0000 7f6144e07640 1 --2- 172.21.15.76:0/710553723 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6128003f20 0x7f6140151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:06.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.436+0000 7f6144e07640 1 -- 172.21.15.76:0/710553723 >> 172.21.15.76:0/710553723 conn(0x7f6140071c40 msgr2=0x7f61400d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:06.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.436+0000 7f6144e07640 1 -- 172.21.15.76:0/710553723 shutdown_connections 2023-11-09T03:49:06.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.436+0000 7f6144e07640 1 -- 172.21.15.76:0/710553723 wait complete. 2023-11-09T03:49:06.448 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2023-11-09T03:49:06.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.557+0000 7f2f3fba9640 1 Processor -- start 2023-11-09T03:49:06.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.557+0000 7f2f3fba9640 1 -- start start 2023-11-09T03:49:06.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.558+0000 7f2f3fba9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2f38159c00 0x7f2f380cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:06.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.558+0000 7f2f3fba9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f3815aff0 con 0x7f2f380ae1d0 2023-11-09T03:49:06.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.558+0000 7f2f3fba9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f3815a3e0 con 0x7f2f380c48e0 2023-11-09T03:49:06.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.558+0000 7f2f3fba9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f3815a6c0 con 0x7f2f38159c00 2023-11-09T03:49:06.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.558+0000 7f2f3e146640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2f380ae1d0 0x7f2f380709f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53278/0 (socket says 172.21.15.76:53278) 2023-11-09T03:49:06.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.558+0000 7f2f3e146640 1 -- 172.21.15.76:0/926806218 learned_addr learned my addr 172.21.15.76:0/926806218 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:49:06.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.558+0000 7f2f3d945640 1 --2- 172.21.15.76:0/926806218 >> v2:172.21.15.113:3300/0 conn(0x7f2f38159c00 0x7f2f380cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:06.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.558+0000 7f2f3c943640 1 -- 172.21.15.76:0/926806218 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3219633968 0 0) 0x7f2f3815aff0 con 0x7f2f380ae1d0 2023-11-09T03:49:06.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.558+0000 7f2f3c943640 1 -- 172.21.15.76:0/926806218 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f20003520 con 0x7f2f380ae1d0 2023-11-09T03:49:06.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.558+0000 7f2f3c943640 1 -- 172.21.15.76:0/926806218 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2977765384 0 0) 0x7f2f20003520 con 0x7f2f380ae1d0 2023-11-09T03:49:06.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.558+0000 7f2f3c943640 1 -- 172.21.15.76:0/926806218 >> v2:172.21.15.113:3300/0 conn(0x7f2f38159c00 msgr2=0x7f2f380cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:06.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.558+0000 7f2f3c943640 1 --2- 172.21.15.76:0/926806218 >> v2:172.21.15.113:3300/0 conn(0x7f2f38159c00 0x7f2f380cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:06.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.558+0000 7f2f3c943640 1 -- 172.21.15.76:0/926806218 >> v1:172.21.15.113:6789/0 conn(0x7f2f380c48e0 legacy=0x7f2f380adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:06.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.558+0000 7f2f3c943640 1 -- 172.21.15.76:0/926806218 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3815a9a0 con 0x7f2f380ae1d0 2023-11-09T03:49:06.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.558+0000 7f2f3d945640 1 --2- 172.21.15.76:0/926806218 >> v2:172.21.15.113:3300/0 conn(0x7f2f38159c00 0x7f2f380cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:49:06.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.558+0000 7f2f3c943640 1 -- 172.21.15.76:0/926806218 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f2f280019f0 con 0x7f2f380ae1d0 2023-11-09T03:49:06.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.558+0000 7f2f3c943640 1 -- 172.21.15.76:0/926806218 >> v1:172.21.15.76:6789/0 conn(0x7f2f380ae1d0 legacy=0x7f2f380709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:06.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.558+0000 7f2f3c943640 1 --2- 172.21.15.76:0/926806218 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f20003970 0x7f2f20003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:06.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.558+0000 7f2f3c943640 1 --2- 172.21.15.76:0/926806218 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f200047c0 0x7f2f20004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:06.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.558+0000 7f2f3c943640 1 --2- 172.21.15.76:0/926806218 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f20005180 0x7f2f2000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:06.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.558+0000 7f2f3c943640 1 -- 172.21.15.76:0/926806218 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3815a6c0 con 0x7f2f20003970 2023-11-09T03:49:06.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.558+0000 7f2f3c943640 1 -- 172.21.15.76:0/926806218 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2000c120 con 0x7f2f200047c0 2023-11-09T03:49:06.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.558+0000 7f2f3c943640 1 -- 172.21.15.76:0/926806218 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2000c2e0 con 0x7f2f20005180 2023-11-09T03:49:06.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.558+0000 7f2f3d945640 1 --2- 172.21.15.76:0/926806218 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f20003970 0x7f2f20003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:06.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.559+0000 7f2f3d144640 1 --2- 172.21.15.76:0/926806218 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f200047c0 0x7f2f20004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:06.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.559+0000 7f2f3e146640 1 --2- 172.21.15.76:0/926806218 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f20005180 0x7f2f2000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:06.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.559+0000 7f2f3d945640 1 -- 172.21.15.76:0/926806218 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f20005180 msgr2=0x7f2f2000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:06.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.559+0000 7f2f3d945640 1 --2- 172.21.15.76:0/926806218 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f20005180 0x7f2f2000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:06.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.559+0000 7f2f3d945640 1 -- 172.21.15.76:0/926806218 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f200047c0 msgr2=0x7f2f20004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:06.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.559+0000 7f2f3d945640 1 --2- 172.21.15.76:0/926806218 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f200047c0 0x7f2f20004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:06.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.559+0000 7f2f3d945640 1 -- 172.21.15.76:0/926806218 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f20003520 con 0x7f2f20003970 2023-11-09T03:49:06.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.559+0000 7f2f3e146640 1 -- 172.21.15.76:0/926806218 reap_dead start 2023-11-09T03:49:06.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.559+0000 7f2f3d945640 1 --2- 172.21.15.76:0/926806218 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f20003970 0x7f2f20003d90 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009820 tx=0x7f2f200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52fdba6fd3c3a3f server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:06.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.560+0000 7f2f3c943640 1 -- 172.21.15.76:0/926806218 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3002eb20 con 0x7f2f20003970 2023-11-09T03:49:06.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.560+0000 7f2f3c943640 1 -- 172.21.15.76:0/926806218 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3002ec80 con 0x7f2f20003970 2023-11-09T03:49:06.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.560+0000 7f2f3c943640 1 -- 172.21.15.76:0/926806218 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f30036640 con 0x7f2f20003970 2023-11-09T03:49:06.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.560+0000 7f2f3fba9640 1 -- 172.21.15.76:0/926806218 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f20003970 msgr2=0x7f2f20003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:06.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.560+0000 7f2f3fba9640 1 --2- 172.21.15.76:0/926806218 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f20003970 0x7f2f20003d90 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009820 tx=0x7f2f200057f0 comp rx=0 tx=0).stop 2023-11-09T03:49:06.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.560+0000 7f2f3fba9640 1 -- 172.21.15.76:0/926806218 shutdown_connections 2023-11-09T03:49:06.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.560+0000 7f2f3fba9640 1 --2- 172.21.15.76:0/926806218 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f20003970 0x7f2f20003d90 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:06.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.560+0000 7f2f3fba9640 1 -- 172.21.15.76:0/926806218 >> 172.21.15.76:0/926806218 conn(0x7f2f380ce950 msgr2=0x7f2f380ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:06.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.560+0000 7f2f3fba9640 1 -- 172.21.15.76:0/926806218 shutdown_connections 2023-11-09T03:49:06.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.560+0000 7f2f3fba9640 1 -- 172.21.15.76:0/926806218 wait complete. 2023-11-09T03:49:06.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.561+0000 7f2f3fba9640 1 Processor -- start 2023-11-09T03:49:06.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.561+0000 7f2f3fba9640 1 -- start start 2023-11-09T03:49:06.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.562+0000 7f2f3fba9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f20003970 0x7f2f380c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:06.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.562+0000 7f2f3fba9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f3814a080 0x7f2f38152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:06.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.562+0000 7f2f3fba9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f38148100 0x7f2f38148580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:06.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.562+0000 7f2f3fba9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f380c8910 con 0x7f2f20003970 2023-11-09T03:49:06.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.562+0000 7f2f3fba9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f380c8070 con 0x7f2f38148100 2023-11-09T03:49:06.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.562+0000 7f2f3fba9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f380c8350 con 0x7f2f3814a080 2023-11-09T03:49:06.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.562+0000 7f2f3d945640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f20003970 0x7f2f380c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:06.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.562+0000 7f2f3d945640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f20003970 0x7f2f380c6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38318/0 (socket says 172.21.15.76:38318) 2023-11-09T03:49:06.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.562+0000 7f2f3d945640 1 -- 172.21.15.76:0/1257084375 learned_addr learned my addr 172.21.15.76:0/1257084375 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:06.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.562+0000 7f2f3d945640 1 -- 172.21.15.76:0/1257084375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f3814a080 msgr2=0x7f2f38152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:06.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.562+0000 7f2f3d945640 1 --2- 172.21.15.76:0/1257084375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f3814a080 0x7f2f38152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:06.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.562+0000 7f2f3d945640 1 -- 172.21.15.76:0/1257084375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f38148100 msgr2=0x7f2f38148580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:06.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.562+0000 7f2f3d945640 1 --2- 172.21.15.76:0/1257084375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f38148100 0x7f2f38148580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:06.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.562+0000 7f2f3d945640 1 -- 172.21.15.76:0/1257084375 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f380c8630 con 0x7f2f20003970 2023-11-09T03:49:06.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.563+0000 7f2f3d945640 1 --2- 172.21.15.76:0/1257084375 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f20003970 0x7f2f380c6430 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009d80 tx=0x7f2f30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:06.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.563+0000 7f2f367fc640 1 -- 172.21.15.76:0/1257084375 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f30036be0 con 0x7f2f20003970 2023-11-09T03:49:06.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.563+0000 7f2f367fc640 1 -- 172.21.15.76:0/1257084375 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f30036d40 con 0x7f2f20003970 2023-11-09T03:49:06.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.563+0000 7f2f367fc640 1 -- 172.21.15.76:0/1257084375 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3003f620 con 0x7f2f20003970 2023-11-09T03:49:06.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.563+0000 7f2f3fba9640 1 -- 172.21.15.76:0/1257084375 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f380b2bf0 con 0x7f2f20003970 2023-11-09T03:49:06.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.563+0000 7f2f3fba9640 1 -- 172.21.15.76:0/1257084375 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f38153310 con 0x7f2f20003970 2023-11-09T03:49:06.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.564+0000 7f2f3fba9640 1 -- 172.21.15.76:0/1257084375 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f380b2a50 con 0x7f2f20003970 2023-11-09T03:49:06.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.564+0000 7f2f367fc640 1 -- 172.21.15.76:0/1257084375 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2f3003d070 con 0x7f2f20003970 2023-11-09T03:49:06.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.565+0000 7f2f367fc640 1 --2- 172.21.15.76:0/1257084375 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2f2406aea0 0x7f2f2406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:06.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.565+0000 7f2f367fc640 1 -- 172.21.15.76:0/1257084375 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f2f300ab550 con 0x7f2f20003970 2023-11-09T03:49:06.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.565+0000 7f2f3d144640 1 --2- 172.21.15.76:0/1257084375 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2f2406aea0 0x7f2f2406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:06.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.566+0000 7f2f3d144640 1 --2- 172.21.15.76:0/1257084375 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2f2406aea0 0x7f2f2406d360 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c000c00 tx=0x7f2f2c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:06.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.569+0000 7f2f367fc640 1 -- 172.21.15.76:0/1257084375 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2f380b2a50 con 0x7f2f20003970 2023-11-09T03:49:06.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:06.772+0000 7f2f3fba9640 1 -- 172.21.15.76:0/1257084375 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f2f380b2a50 con 0x7f2f20003970 2023-11-09T03:49:07.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.441+0000 7f2f367fc640 1 -- 172.21.15.76:0/1257084375 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v46) v1 ==== 105+0+0 (secure 0 0 0) 0x7f2f380b2a50 con 0x7f2f20003970 2023-11-09T03:49:07.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.443+0000 7f2f3fba9640 1 -- 172.21.15.76:0/1257084375 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2f2406aea0 msgr2=0x7f2f2406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:07.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.443+0000 7f2f3fba9640 1 --2- 172.21.15.76:0/1257084375 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2f2406aea0 0x7f2f2406d360 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c000c00 tx=0x7f2f2c009040 comp rx=0 tx=0).stop 2023-11-09T03:49:07.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.444+0000 7f2f3fba9640 1 -- 172.21.15.76:0/1257084375 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f20003970 msgr2=0x7f2f380c6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:07.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.444+0000 7f2f3fba9640 1 --2- 172.21.15.76:0/1257084375 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f20003970 0x7f2f380c6430 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009d80 tx=0x7f2f30004730 comp rx=0 tx=0).stop 2023-11-09T03:49:07.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.444+0000 7f2f3fba9640 1 -- 172.21.15.76:0/1257084375 shutdown_connections 2023-11-09T03:49:07.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.444+0000 7f2f3fba9640 1 --2- 172.21.15.76:0/1257084375 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2f2406aea0 0x7f2f2406d360 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:07.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.444+0000 7f2f3fba9640 1 --2- 172.21.15.76:0/1257084375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f38148100 0x7f2f38148580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:07.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.444+0000 7f2f3fba9640 1 --2- 172.21.15.76:0/1257084375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2f3814a080 0x7f2f38152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:07.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.444+0000 7f2f3fba9640 1 --2- 172.21.15.76:0/1257084375 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f20003970 0x7f2f380c6430 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:07.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.444+0000 7f2f3fba9640 1 -- 172.21.15.76:0/1257084375 >> 172.21.15.76:0/1257084375 conn(0x7f2f380ce950 msgr2=0x7f2f3806d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:07.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.444+0000 7f2f3fba9640 1 -- 172.21.15.76:0/1257084375 shutdown_connections 2023-11-09T03:49:07.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.444+0000 7f2f3fba9640 1 -- 172.21.15.76:0/1257084375 wait complete. 2023-11-09T03:49:07.462 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:49:07.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.566+0000 7f7582644640 1 Processor -- start 2023-11-09T03:49:07.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.566+0000 7f7582644640 1 -- start start 2023-11-09T03:49:07.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.567+0000 7f7582644640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f757c0cd520 0x7f757c06c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:07.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.567+0000 7f7582644640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f757c0ae200 con 0x7f757c0c1e80 2023-11-09T03:49:07.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.567+0000 7f7582644640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f757c0c9d10 con 0x7f757c0ade20 2023-11-09T03:49:07.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.567+0000 7f7582644640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f757c1617c0 con 0x7f757c0cd520 2023-11-09T03:49:07.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.567+0000 7f757b7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f757c0c1e80 0x7f757c0cce10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53282/0 (socket says 172.21.15.76:53282) 2023-11-09T03:49:07.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.567+0000 7f757b7fe640 1 -- 172.21.15.76:0/1590522169 learned_addr learned my addr 172.21.15.76:0/1590522169 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:49:07.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.567+0000 7f7580be1640 1 --2- 172.21.15.76:0/1590522169 >> v2:172.21.15.113:3300/0 conn(0x7f757c0cd520 0x7f757c06c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:07.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.567+0000 7f757affd640 1 -- 172.21.15.76:0/1590522169 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 953764954 0 0) 0x7f757c0ae200 con 0x7f757c0c1e80 2023-11-09T03:49:07.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.567+0000 7f757affd640 1 -- 172.21.15.76:0/1590522169 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7564003520 con 0x7f757c0c1e80 2023-11-09T03:49:07.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.567+0000 7f757affd640 1 -- 172.21.15.76:0/1590522169 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1491402950 0 0) 0x7f7564003520 con 0x7f757c0c1e80 2023-11-09T03:49:07.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.567+0000 7f757affd640 1 -- 172.21.15.76:0/1590522169 >> v2:172.21.15.113:3300/0 conn(0x7f757c0cd520 msgr2=0x7f757c06c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:07.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.567+0000 7f757affd640 1 --2- 172.21.15.76:0/1590522169 >> v2:172.21.15.113:3300/0 conn(0x7f757c0cd520 0x7f757c06c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:07.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.567+0000 7f757affd640 1 -- 172.21.15.76:0/1590522169 >> v1:172.21.15.113:6789/0 conn(0x7f757c0ade20 legacy=0x7f757c0c92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:07.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.567+0000 7f757affd640 1 -- 172.21.15.76:0/1590522169 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c0c9ff0 con 0x7f757c0c1e80 2023-11-09T03:49:07.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.567+0000 7f757affd640 1 -- 172.21.15.76:0/1590522169 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f7570001670 con 0x7f757c0c1e80 2023-11-09T03:49:07.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.567+0000 7f757affd640 1 -- 172.21.15.76:0/1590522169 >> v1:172.21.15.76:6789/0 conn(0x7f757c0c1e80 legacy=0x7f757c0cce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:07.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.567+0000 7f757affd640 1 --2- 172.21.15.76:0/1590522169 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7564003970 0x7f7564003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:07.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.567+0000 7f757affd640 1 --2- 172.21.15.76:0/1590522169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7564004370 0x7f7564004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:07.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.567+0000 7f757affd640 1 --2- 172.21.15.76:0/1590522169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7564005040 0x7f756400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:07.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.567+0000 7f757affd640 1 -- 172.21.15.76:0/1590522169 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c1617c0 con 0x7f7564003970 2023-11-09T03:49:07.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.567+0000 7f757affd640 1 -- 172.21.15.76:0/1590522169 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756400bfa0 con 0x7f7564005040 2023-11-09T03:49:07.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.568+0000 7f757affd640 1 -- 172.21.15.76:0/1590522169 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756400c160 con 0x7f7564004370 2023-11-09T03:49:07.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.568+0000 7f757bfff640 1 --2- 172.21.15.76:0/1590522169 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7564003970 0x7f7564003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:07.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.568+0000 7f757b7fe640 1 --2- 172.21.15.76:0/1590522169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7564004370 0x7f7564004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:07.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.568+0000 7f7580be1640 1 --2- 172.21.15.76:0/1590522169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7564005040 0x7f756400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:07.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.568+0000 7f757bfff640 1 -- 172.21.15.76:0/1590522169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7564004370 msgr2=0x7f7564004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:07.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.568+0000 7f757bfff640 1 --2- 172.21.15.76:0/1590522169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7564004370 0x7f7564004840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:07.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.568+0000 7f757bfff640 1 -- 172.21.15.76:0/1590522169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7564005040 msgr2=0x7f756400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:07.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.568+0000 7f757bfff640 1 --2- 172.21.15.76:0/1590522169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7564005040 0x7f756400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:07.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.568+0000 7f757bfff640 1 -- 172.21.15.76:0/1590522169 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7564003520 con 0x7f7564003970 2023-11-09T03:49:07.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.568+0000 7f7580be1640 1 --2- 172.21.15.76:0/1590522169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7564005040 0x7f756400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:07.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.568+0000 7f757b7fe640 1 --2- 172.21.15.76:0/1590522169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7564004370 0x7f7564004840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:07.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.568+0000 7f7580be1640 1 -- 172.21.15.76:0/1590522169 reap_dead start 2023-11-09T03:49:07.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.569+0000 7f757bfff640 1 --2- 172.21.15.76:0/1590522169 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7564003970 0x7f7564003d50 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f756c001860 tx=0x7f75640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b1a41d20559785 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:07.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.569+0000 7f757affd640 1 -- 172.21.15.76:0/1590522169 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c033410 con 0x7f7564003970 2023-11-09T03:49:07.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.569+0000 7f757affd640 1 -- 172.21.15.76:0/1590522169 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756c033590 con 0x7f7564003970 2023-11-09T03:49:07.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.569+0000 7f757affd640 1 -- 172.21.15.76:0/1590522169 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c033940 con 0x7f7564003970 2023-11-09T03:49:07.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.569+0000 7f7582644640 1 -- 172.21.15.76:0/1590522169 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7564003970 msgr2=0x7f7564003d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:07.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.569+0000 7f7582644640 1 --2- 172.21.15.76:0/1590522169 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7564003970 0x7f7564003d50 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f756c001860 tx=0x7f75640057f0 comp rx=0 tx=0).stop 2023-11-09T03:49:07.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.569+0000 7f7582644640 1 -- 172.21.15.76:0/1590522169 shutdown_connections 2023-11-09T03:49:07.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.569+0000 7f7582644640 1 --2- 172.21.15.76:0/1590522169 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7564003970 0x7f7564003d50 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:07.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.569+0000 7f7582644640 1 -- 172.21.15.76:0/1590522169 >> 172.21.15.76:0/1590522169 conn(0x7f757c0ce950 msgr2=0x7f757c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:07.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.569+0000 7f7582644640 1 -- 172.21.15.76:0/1590522169 shutdown_connections 2023-11-09T03:49:07.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.569+0000 7f7582644640 1 -- 172.21.15.76:0/1590522169 wait complete. 2023-11-09T03:49:07.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.570+0000 7f7582644640 1 Processor -- start 2023-11-09T03:49:07.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.571+0000 7f7582644640 1 -- start start 2023-11-09T03:49:07.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.571+0000 7f7582644640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7564003970 0x7f757c0b2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:07.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.571+0000 7f7582644640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f757c0b3b70 0x7f757c0c4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:07.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.571+0000 7f7582644640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f757c1f7d50 0x7f757c06d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:07.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.571+0000 7f7582644640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c0b2950 con 0x7f757c0b3b70 2023-11-09T03:49:07.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.571+0000 7f7582644640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c0b2c30 con 0x7f757c1f7d50 2023-11-09T03:49:07.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.571+0000 7f7582644640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757c0b2f10 con 0x7f7564003970 2023-11-09T03:49:07.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.571+0000 7f757b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f757c0b3b70 0x7f757c0c4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:07.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.571+0000 7f757b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f757c0b3b70 0x7f757c0c4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38346/0 (socket says 172.21.15.76:38346) 2023-11-09T03:49:07.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.571+0000 7f757b7fe640 1 -- 172.21.15.76:0/3026953090 learned_addr learned my addr 172.21.15.76:0/3026953090 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:07.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.572+0000 7f757b7fe640 1 -- 172.21.15.76:0/3026953090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7564003970 msgr2=0x7f757c0b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:07.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.572+0000 7f757bfff640 1 --2- 172.21.15.76:0/3026953090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7564003970 0x7f757c0b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:07.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.572+0000 7f7580be1640 1 --2- 172.21.15.76:0/3026953090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f757c1f7d50 0x7f757c06d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:07.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.572+0000 7f757b7fe640 1 --2- 172.21.15.76:0/3026953090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7564003970 0x7f757c0b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:07.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.572+0000 7f757b7fe640 1 -- 172.21.15.76:0/3026953090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f757c1f7d50 msgr2=0x7f757c06d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:07.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.572+0000 7f757b7fe640 1 --2- 172.21.15.76:0/3026953090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f757c1f7d50 0x7f757c06d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:07.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.572+0000 7f757b7fe640 1 -- 172.21.15.76:0/3026953090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c0b31f0 con 0x7f757c0b3b70 2023-11-09T03:49:07.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.572+0000 7f757b7fe640 1 --2- 172.21.15.76:0/3026953090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f757c0b3b70 0x7f757c0c4250 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f7570001cb0 tx=0x7f757000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:07.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.572+0000 7f7578ff9640 1 -- 172.21.15.76:0/3026953090 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757000c6c0 con 0x7f757c0b3b70 2023-11-09T03:49:07.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.572+0000 7f7578ff9640 1 -- 172.21.15.76:0/3026953090 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757000c820 con 0x7f757c0b3b70 2023-11-09T03:49:07.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.572+0000 7f7578ff9640 1 -- 172.21.15.76:0/3026953090 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757000cb90 con 0x7f757c0b3b70 2023-11-09T03:49:07.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.572+0000 7f7582644640 1 -- 172.21.15.76:0/3026953090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757c1f6ca0 con 0x7f757c0b3b70 2023-11-09T03:49:07.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.572+0000 7f7582644640 1 -- 172.21.15.76:0/3026953090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757c0b3ff0 con 0x7f757c0b3b70 2023-11-09T03:49:07.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.573+0000 7f7582644640 1 -- 172.21.15.76:0/3026953090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f757c1f6e90 con 0x7f757c0b3b70 2023-11-09T03:49:07.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.573+0000 7f7578ff9640 1 -- 172.21.15.76:0/3026953090 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f757000ccf0 con 0x7f757c0b3b70 2023-11-09T03:49:07.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.574+0000 7f7578ff9640 1 --2- 172.21.15.76:0/3026953090 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f756806aea0 0x7f756806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:07.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.574+0000 7f7578ff9640 1 -- 172.21.15.76:0/3026953090 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f7570089a50 con 0x7f757c0b3b70 2023-11-09T03:49:07.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.574+0000 7f757bfff640 1 --2- 172.21.15.76:0/3026953090 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f756806aea0 0x7f756806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:07.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.575+0000 7f757bfff640 1 --2- 172.21.15.76:0/3026953090 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f756806aea0 0x7f756806d360 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f756c035000 tx=0x7f756c009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:07.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.578+0000 7f7578ff9640 1 -- 172.21.15.76:0/3026953090 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7570054580 con 0x7f757c0b3b70 2023-11-09T03:49:07.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.778+0000 7f7582644640 1 -- 172.21.15.76:0/3026953090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f757c1f6e90 con 0x7f757c0b3b70 2023-11-09T03:49:07.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.779+0000 7f7578ff9640 1 -- 172.21.15.76:0/3026953090 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f7570057ba0 con 0x7f757c0b3b70 2023-11-09T03:49:07.779 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:49:07.779 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":47,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5744,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3280098776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3280098776},{"type":"v1","addr":"172.21.15.113:6837","nonce":3280098776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:49:07.444487+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7452,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:creating","state_seq":1,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7452":{"gid":7452,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/4011275120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4011275120},{"type":"v1","addr":"172.21.15.76:6835","nonce":4011275120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:49:07.779 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 47 2023-11-09T03:49:07.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.781+0000 7f7582644640 1 -- 172.21.15.76:0/3026953090 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f756806aea0 msgr2=0x7f756806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:07.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.781+0000 7f7582644640 1 --2- 172.21.15.76:0/3026953090 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f756806aea0 0x7f756806d360 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f756c035000 tx=0x7f756c009030 comp rx=0 tx=0).stop 2023-11-09T03:49:07.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.781+0000 7f7582644640 1 -- 172.21.15.76:0/3026953090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f757c0b3b70 msgr2=0x7f757c0c4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:07.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.781+0000 7f7582644640 1 --2- 172.21.15.76:0/3026953090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f757c0b3b70 0x7f757c0c4250 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f7570001cb0 tx=0x7f757000eb90 comp rx=0 tx=0).stop 2023-11-09T03:49:07.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.781+0000 7f7582644640 1 -- 172.21.15.76:0/3026953090 shutdown_connections 2023-11-09T03:49:07.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.781+0000 7f7582644640 1 --2- 172.21.15.76:0/3026953090 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f756806aea0 0x7f756806d360 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:07.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.781+0000 7f7582644640 1 --2- 172.21.15.76:0/3026953090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f757c1f7d50 0x7f757c06d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:07.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.781+0000 7f7582644640 1 --2- 172.21.15.76:0/3026953090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f757c0b3b70 0x7f757c0c4250 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:07.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.781+0000 7f7582644640 1 --2- 172.21.15.76:0/3026953090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7564003970 0x7f757c0b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:07.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.781+0000 7f7582644640 1 -- 172.21.15.76:0/3026953090 >> 172.21.15.76:0/3026953090 conn(0x7f757c0ce950 msgr2=0x7f757c160f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:07.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.781+0000 7f7582644640 1 -- 172.21.15.76:0/3026953090 shutdown_connections 2023-11-09T03:49:07.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:07.781+0000 7f7582644640 1 -- 172.21.15.76:0/3026953090 wait complete. 2023-11-09T03:49:08.193 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-09T03:48:57.383848+0000', 'modified': '2023-11-09T03:49:07.444487+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 7452, 'mds_1': 7449}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7449': {'gid': 7449, 'name': 'a', 'rank': 1, 'incarnation': 47, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.76:6833/3697850101', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6832', 'nonce': 3697850101}, {'type': 'v1', 'addr': '172.21.15.76:6833', 'nonce': 3697850101}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_7452': {'gid': 7452, 'name': 'c', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6835/4011275120', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 4011275120}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 4011275120}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-09T03:49:08.193 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7449:up:creating 2023-11-09T03:49:09.021 INFO:tasks.ceph.mds.c.smithi076.stderr:2023-11-09T03:49:09.022+0000 7f65da07c640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-09T03:49:09.195 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:49:09.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.304+0000 7f98d1db6640 1 Processor -- start 2023-11-09T03:49:09.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.304+0000 7f98d1db6640 1 -- start start 2023-11-09T03:49:09.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.304+0000 7f98d1db6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f98cc0b4190 0x7f98cc0ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:09.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.304+0000 7f98d1db6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98cc0cb230 0x7f98cc0cb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:09.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.304+0000 7f98d1db6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98cc0cdd30 con 0x7f98cc15abf0 2023-11-09T03:49:09.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.304+0000 7f98d1db6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98cc0ce700 con 0x7f98cc0b4190 2023-11-09T03:49:09.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.304+0000 7f98d1db6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc0cbba0 con 0x7f98cc0cb230 2023-11-09T03:49:09.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.305+0000 7f98cbfff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98cc0cb230 0x7f98cc0cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:09.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.305+0000 7f98caffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f98cc0b4190 0x7f98cc0ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:09.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.305+0000 7f98cb7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f98cc15abf0 0x7f98cc0cc8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:54874/0 (socket says 172.21.15.76:54874) 2023-11-09T03:49:09.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.305+0000 7f98cb7fe640 1 -- 172.21.15.76:0/3821177041 learned_addr learned my addr 172.21.15.76:0/3821177041 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:49:09.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.305+0000 7f98cbfff640 1 -- 172.21.15.76:0/3821177041 >> v2:172.21.15.113:3300/0 conn(0x7f98cc0b4190 msgr2=0x7f98cc0ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:09.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.305+0000 7f98cbfff640 1 --2- 172.21.15.76:0/3821177041 >> v2:172.21.15.113:3300/0 conn(0x7f98cc0b4190 0x7f98cc0ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:09.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.305+0000 7f98cbfff640 1 -- 172.21.15.76:0/3821177041 >> v1:172.21.15.113:6789/0 conn(0x7f98cc15abf0 legacy=0x7f98cc0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:09.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.305+0000 7f98cbfff640 1 -- 172.21.15.76:0/3821177041 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc0cd120 con 0x7f98cc0cb230 2023-11-09T03:49:09.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.305+0000 7f98caffd640 1 --2- 172.21.15.76:0/3821177041 >> v2:172.21.15.113:3300/0 conn(0x7f98cc0b4190 0x7f98cc0ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:49:09.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.306+0000 7f98cbfff640 1 --2- 172.21.15.76:0/3821177041 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98cc0cb230 0x7f98cc0cb660 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009d90 tx=0x7f98c402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=498136425e1f08b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:09.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.306+0000 7f98ca7fc640 1 -- 172.21.15.76:0/3821177041 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c402cc50 con 0x7f98cc0cb230 2023-11-09T03:49:09.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.306+0000 7f98ca7fc640 1 -- 172.21.15.76:0/3821177041 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c402cdb0 con 0x7f98cc0cb230 2023-11-09T03:49:09.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.306+0000 7f98ca7fc640 1 -- 172.21.15.76:0/3821177041 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c4034660 con 0x7f98cc0cb230 2023-11-09T03:49:09.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.307+0000 7f98d1db6640 1 -- 172.21.15.76:0/3821177041 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98cc0cb230 msgr2=0x7f98cc0cb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:09.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.307+0000 7f98d1db6640 1 --2- 172.21.15.76:0/3821177041 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98cc0cb230 0x7f98cc0cb660 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009d90 tx=0x7f98c402bea0 comp rx=0 tx=0).stop 2023-11-09T03:49:09.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.307+0000 7f98d1db6640 1 -- 172.21.15.76:0/3821177041 shutdown_connections 2023-11-09T03:49:09.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.307+0000 7f98d1db6640 1 --2- 172.21.15.76:0/3821177041 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98cc0cb230 0x7f98cc0cb660 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:09.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.307+0000 7f98d1db6640 1 --2- 172.21.15.76:0/3821177041 >> v2:172.21.15.113:3300/0 conn(0x7f98cc0b4190 0x7f98cc0ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:09.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.307+0000 7f98d1db6640 1 -- 172.21.15.76:0/3821177041 >> 172.21.15.76:0/3821177041 conn(0x7f98cc0b03c0 msgr2=0x7f98cc0b07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:09.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.307+0000 7f98d1db6640 1 -- 172.21.15.76:0/3821177041 shutdown_connections 2023-11-09T03:49:09.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.307+0000 7f98d1db6640 1 -- 172.21.15.76:0/3821177041 wait complete. 2023-11-09T03:49:09.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.308+0000 7f98d1db6640 1 Processor -- start 2023-11-09T03:49:09.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.308+0000 7f98d1db6640 1 -- start start 2023-11-09T03:49:09.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.309+0000 7f98d1db6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98cc0b4190 0x7f98cc0c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:09.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.309+0000 7f98d1db6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98cc0cb230 0x7f98cc0a76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:09.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.309+0000 7f98d1db6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98cc15abf0 0x7f98cc0ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:09.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.309+0000 7f98d1db6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc0c79d0 con 0x7f98cc0cb230 2023-11-09T03:49:09.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.309+0000 7f98d1db6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc0c7cb0 con 0x7f98cc15abf0 2023-11-09T03:49:09.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.309+0000 7f98d1db6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc0c7f90 con 0x7f98cc0b4190 2023-11-09T03:49:09.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.309+0000 7f98caffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98cc0cb230 0x7f98cc0a76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:09.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.309+0000 7f98caffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98cc0cb230 0x7f98cc0a76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38356/0 (socket says 172.21.15.76:38356) 2023-11-09T03:49:09.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.309+0000 7f98caffd640 1 -- 172.21.15.76:0/2066421995 learned_addr learned my addr 172.21.15.76:0/2066421995 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:09.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.309+0000 7f98cbfff640 1 --2- 172.21.15.76:0/2066421995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98cc15abf0 0x7f98cc0ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:09.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.309+0000 7f98cb7fe640 1 --2- 172.21.15.76:0/2066421995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98cc0b4190 0x7f98cc0c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:09.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.309+0000 7f98caffd640 1 -- 172.21.15.76:0/2066421995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98cc0b4190 msgr2=0x7f98cc0c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:09.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.309+0000 7f98caffd640 1 --2- 172.21.15.76:0/2066421995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98cc0b4190 0x7f98cc0c7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:09.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.309+0000 7f98caffd640 1 -- 172.21.15.76:0/2066421995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98cc15abf0 msgr2=0x7f98cc0ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:09.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.309+0000 7f98caffd640 1 --2- 172.21.15.76:0/2066421995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98cc15abf0 0x7f98cc0ad4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:09.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.309+0000 7f98caffd640 1 -- 172.21.15.76:0/2066421995 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c00093a0 con 0x7f98cc0cb230 2023-11-09T03:49:09.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.310+0000 7f98cbfff640 1 --2- 172.21.15.76:0/2066421995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98cc15abf0 0x7f98cc0ad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:09.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.310+0000 7f98cb7fe640 1 --2- 172.21.15.76:0/2066421995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98cc0b4190 0x7f98cc0c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:09.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.310+0000 7f98caffd640 1 --2- 172.21.15.76:0/2066421995 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98cc0cb230 0x7f98cc0a76a0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f98c000e9b0 tx=0x7f98c000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:09.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.310+0000 7f98b3fff640 1 -- 172.21.15.76:0/2066421995 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0005990 con 0x7f98cc0cb230 2023-11-09T03:49:09.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.310+0000 7f98d1db6640 1 -- 172.21.15.76:0/2066421995 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c4009a70 con 0x7f98cc0cb230 2023-11-09T03:49:09.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.310+0000 7f98b3fff640 1 -- 172.21.15.76:0/2066421995 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c0005af0 con 0x7f98cc0cb230 2023-11-09T03:49:09.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.310+0000 7f98b3fff640 1 -- 172.21.15.76:0/2066421995 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0005e80 con 0x7f98cc0cb230 2023-11-09T03:49:09.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.310+0000 7f98d1db6640 1 -- 172.21.15.76:0/2066421995 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98cc149c60 con 0x7f98cc0cb230 2023-11-09T03:49:09.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.311+0000 7f98d1db6640 1 -- 172.21.15.76:0/2066421995 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98cc0b1000 con 0x7f98cc0cb230 2023-11-09T03:49:09.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.311+0000 7f98b3fff640 1 -- 172.21.15.76:0/2066421995 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f98c000c9a0 con 0x7f98cc0cb230 2023-11-09T03:49:09.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.312+0000 7f98b3fff640 1 --2- 172.21.15.76:0/2066421995 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f98b806ae70 0x7f98b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:09.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.312+0000 7f98b3fff640 1 -- 172.21.15.76:0/2066421995 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f98c008a050 con 0x7f98cc0cb230 2023-11-09T03:49:09.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.312+0000 7f98cb7fe640 1 --2- 172.21.15.76:0/2066421995 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f98b806ae70 0x7f98b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:09.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.312+0000 7f98cb7fe640 1 --2- 172.21.15.76:0/2066421995 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f98b806ae70 0x7f98b806d330 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f98bc005880 tx=0x7f98bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:09.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.316+0000 7f98b3fff640 1 -- 172.21.15.76:0/2066421995 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f98c0054b00 con 0x7f98cc0cb230 2023-11-09T03:49:09.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.516+0000 7f98d1db6640 1 -- 172.21.15.76:0/2066421995 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98cc074290 con 0x7f98cc0cb230 2023-11-09T03:49:09.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.516+0000 7f98b3fff640 1 -- 172.21.15.76:0/2066421995 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4932 (secure 0 0 0) 0x7f98c0058120 con 0x7f98cc0cb230 2023-11-09T03:49:09.516 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:49:09.516 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5744,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3280098776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3280098776},{"type":"v1","addr":"172.21.15.113:6837","nonce":3280098776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:49:07.461923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7452,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7452":{"gid":7452,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/4011275120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4011275120},{"type":"v1","addr":"172.21.15.76:6835","nonce":4011275120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:49:09.517 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 48 2023-11-09T03:49:09.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.518+0000 7f98d1db6640 1 -- 172.21.15.76:0/2066421995 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f98b806ae70 msgr2=0x7f98b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:09.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.518+0000 7f98d1db6640 1 --2- 172.21.15.76:0/2066421995 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f98b806ae70 0x7f98b806d330 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f98bc005880 tx=0x7f98bc009040 comp rx=0 tx=0).stop 2023-11-09T03:49:09.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.518+0000 7f98d1db6640 1 -- 172.21.15.76:0/2066421995 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98cc0cb230 msgr2=0x7f98cc0a76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:09.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.518+0000 7f98d1db6640 1 --2- 172.21.15.76:0/2066421995 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98cc0cb230 0x7f98cc0a76a0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f98c000e9b0 tx=0x7f98c000ee70 comp rx=0 tx=0).stop 2023-11-09T03:49:09.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.518+0000 7f98d1db6640 1 -- 172.21.15.76:0/2066421995 shutdown_connections 2023-11-09T03:49:09.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.518+0000 7f98d1db6640 1 --2- 172.21.15.76:0/2066421995 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f98b806ae70 0x7f98b806d330 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:09.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.518+0000 7f98d1db6640 1 --2- 172.21.15.76:0/2066421995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98cc15abf0 0x7f98cc0ad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:09.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.518+0000 7f98d1db6640 1 --2- 172.21.15.76:0/2066421995 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98cc0cb230 0x7f98cc0a76a0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:09.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.518+0000 7f98d1db6640 1 --2- 172.21.15.76:0/2066421995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98cc0b4190 0x7f98cc0c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:09.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.518+0000 7f98d1db6640 1 -- 172.21.15.76:0/2066421995 >> 172.21.15.76:0/2066421995 conn(0x7f98cc0b03c0 msgr2=0x7f98cc0c4390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:09.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.518+0000 7f98d1db6640 1 -- 172.21.15.76:0/2066421995 shutdown_connections 2023-11-09T03:49:09.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:09.518+0000 7f98d1db6640 1 -- 172.21.15.76:0/2066421995 wait complete. 2023-11-09T03:49:09.930 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-09T03:48:57.383848+0000', 'modified': '2023-11-09T03:49:07.461923+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 7452, 'mds_1': 7449}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7449': {'gid': 7449, 'name': 'a', 'rank': 1, 'incarnation': 47, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.76:6833/3697850101', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6832', 'nonce': 3697850101}, {'type': 'v1', 'addr': '172.21.15.76:6833', 'nonce': 3697850101}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_7452': {'gid': 7452, 'name': 'c', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6835/4011275120', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 4011275120}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 4011275120}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-09T03:49:09.931 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2023-11-09T03:49:09.931 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-09T03:49:10.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.035+0000 7f702615e640 1 Processor -- start 2023-11-09T03:49:10.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.035+0000 7f702615e640 1 -- start start 2023-11-09T03:49:10.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.035+0000 7f702615e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7020159c00 0x7f70200cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.035+0000 7f702615e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f70200c48e0 0x7f70200c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.035+0000 7f702615e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f702015aff0 con 0x7f70200adac0 2023-11-09T03:49:10.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.035+0000 7f702615e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70200c52d0 con 0x7f70200c48e0 2023-11-09T03:49:10.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.035+0000 7f702615e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f702015a3e0 con 0x7f7020159c00 2023-11-09T03:49:10.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.035+0000 7f701effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f70200c48e0 0x7f70200c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.036+0000 7f701effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f70200c48e0 0x7f70200c4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38368/0 (socket says 172.21.15.76:38368) 2023-11-09T03:49:10.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.036+0000 7f701effd640 1 -- 172.21.15.76:0/1703753514 learned_addr learned my addr 172.21.15.76:0/1703753514 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:10.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.036+0000 7f701effd640 1 -- 172.21.15.76:0/1703753514 >> v2:172.21.15.113:3300/0 conn(0x7f7020159c00 msgr2=0x7f70200cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:10.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.036+0000 7f701effd640 1 --2- 172.21.15.76:0/1703753514 >> v2:172.21.15.113:3300/0 conn(0x7f7020159c00 0x7f70200cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.036+0000 7f701effd640 1 -- 172.21.15.76:0/1703753514 >> v1:172.21.15.113:6789/0 conn(0x7f70200adac0 legacy=0x7f70200adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:10.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.036+0000 7f701effd640 1 -- 172.21.15.76:0/1703753514 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f702015a6c0 con 0x7f70200c48e0 2023-11-09T03:49:10.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.037+0000 7f701effd640 1 --2- 172.21.15.76:0/1703753514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70200c48e0 0x7f70200c4d90 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f70200cc4d0 tx=0x7f701402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd74dff11265266e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:10.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.037+0000 7f701e7fc640 1 -- 172.21.15.76:0/1703753514 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7014031870 con 0x7f70200c48e0 2023-11-09T03:49:10.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.037+0000 7f701e7fc640 1 -- 172.21.15.76:0/1703753514 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70140319d0 con 0x7f70200c48e0 2023-11-09T03:49:10.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.037+0000 7f701e7fc640 1 -- 172.21.15.76:0/1703753514 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7014031d60 con 0x7f70200c48e0 2023-11-09T03:49:10.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.037+0000 7f702615e640 1 -- 172.21.15.76:0/1703753514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70200c48e0 msgr2=0x7f70200c4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:10.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.037+0000 7f702615e640 1 --2- 172.21.15.76:0/1703753514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70200c48e0 0x7f70200c4d90 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f70200cc4d0 tx=0x7f701402b910 comp rx=0 tx=0).stop 2023-11-09T03:49:10.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.037+0000 7f702615e640 1 -- 172.21.15.76:0/1703753514 shutdown_connections 2023-11-09T03:49:10.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.037+0000 7f702615e640 1 --2- 172.21.15.76:0/1703753514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70200c48e0 0x7f70200c4d90 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.037+0000 7f702615e640 1 --2- 172.21.15.76:0/1703753514 >> v2:172.21.15.113:3300/0 conn(0x7f7020159c00 0x7f70200cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.037+0000 7f702615e640 1 -- 172.21.15.76:0/1703753514 >> 172.21.15.76:0/1703753514 conn(0x7f70200ce950 msgr2=0x7f70200ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:10.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.037+0000 7f702615e640 1 -- 172.21.15.76:0/1703753514 shutdown_connections 2023-11-09T03:49:10.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.037+0000 7f702615e640 1 -- 172.21.15.76:0/1703753514 wait complete. 2023-11-09T03:49:10.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.038+0000 7f702615e640 1 Processor -- start 2023-11-09T03:49:10.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.039+0000 7f702615e640 1 -- start start 2023-11-09T03:49:10.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.039+0000 7f702615e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70200adac0 0x7f7020154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.039+0000 7f702615e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70200c48e0 0x7f7020155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.039+0000 7f702615e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7020159c00 0x7f7020155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.039+0000 7f702615e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70200b2b80 con 0x7f70200c48e0 2023-11-09T03:49:10.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.039+0000 7f702615e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70200b2e60 con 0x7f70200adac0 2023-11-09T03:49:10.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.039+0000 7f702615e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70200b3140 con 0x7f7020159c00 2023-11-09T03:49:10.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.039+0000 7f701effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70200c48e0 0x7f7020155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.039+0000 7f701effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70200c48e0 0x7f7020155950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38378/0 (socket says 172.21.15.76:38378) 2023-11-09T03:49:10.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.039+0000 7f701effd640 1 -- 172.21.15.76:0/662317337 learned_addr learned my addr 172.21.15.76:0/662317337 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:10.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.040+0000 7f701f7fe640 1 --2- 172.21.15.76:0/662317337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70200adac0 0x7f7020154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.040+0000 7f701ffff640 1 --2- 172.21.15.76:0/662317337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7020159c00 0x7f7020155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.040+0000 7f701effd640 1 -- 172.21.15.76:0/662317337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7020159c00 msgr2=0x7f7020155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:10.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.040+0000 7f701effd640 1 --2- 172.21.15.76:0/662317337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7020159c00 0x7f7020155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.040+0000 7f701effd640 1 -- 172.21.15.76:0/662317337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70200adac0 msgr2=0x7f7020154600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:10.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.040+0000 7f701effd640 1 --2- 172.21.15.76:0/662317337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70200adac0 0x7f7020154600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.040+0000 7f701effd640 1 -- 172.21.15.76:0/662317337 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70140094d0 con 0x7f70200c48e0 2023-11-09T03:49:10.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.040+0000 7f701ffff640 1 --2- 172.21.15.76:0/662317337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7020159c00 0x7f7020155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:10.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.040+0000 7f701effd640 1 --2- 172.21.15.76:0/662317337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70200c48e0 0x7f7020155950 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f7014000c00 tx=0x7f70140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:10.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.040+0000 7f7007fff640 1 -- 172.21.15.76:0/662317337 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7014031840 con 0x7f70200c48e0 2023-11-09T03:49:10.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.040+0000 7f7007fff640 1 -- 172.21.15.76:0/662317337 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70140443f0 con 0x7f70200c48e0 2023-11-09T03:49:10.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.040+0000 7f702615e640 1 -- 172.21.15.76:0/662317337 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70200b3420 con 0x7f70200c48e0 2023-11-09T03:49:10.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.040+0000 7f7007fff640 1 -- 172.21.15.76:0/662317337 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70140447a0 con 0x7f70200c48e0 2023-11-09T03:49:10.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.040+0000 7f702615e640 1 -- 172.21.15.76:0/662317337 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7020154b40 con 0x7f70200c48e0 2023-11-09T03:49:10.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.042+0000 7f7007fff640 1 -- 172.21.15.76:0/662317337 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f701404d450 con 0x7f70200c48e0 2023-11-09T03:49:10.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.042+0000 7f7007fff640 1 --2- 172.21.15.76:0/662317337 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f700c06aea0 0x7f700c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.042+0000 7f7007fff640 1 -- 172.21.15.76:0/662317337 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f701402d040 con 0x7f70200c48e0 2023-11-09T03:49:10.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.043+0000 7f701f7fe640 1 --2- 172.21.15.76:0/662317337 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f700c06aea0 0x7f700c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.043+0000 7f7028dc6740 1 Processor -- start 2023-11-09T03:49:10.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.043+0000 7f701f7fe640 1 --2- 172.21.15.76:0/662317337 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f700c06aea0 0x7f700c06d360 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f701000a400 tx=0x7f701000d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:10.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.043+0000 7f7028dc6740 1 -- start start 2023-11-09T03:49:10.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.044+0000 7f7028dc6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a9514db7d0 0x55a9514f7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.044+0000 7f7028dc6740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a9514dbbb0 0x55a9514dc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.044+0000 7f7028dc6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9514f5a90 0x55a9514f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.044+0000 7f7028dc6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9513bc740 con 0x55a9514f5a90 2023-11-09T03:49:10.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.044+0000 7f7028dc6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a95141e8d0 con 0x55a9514db7d0 2023-11-09T03:49:10.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.044+0000 7f7028dc6740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a95141bfc0 con 0x55a9514dbbb0 2023-11-09T03:49:10.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.044+0000 7f701ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9514f5a90 0x55a9514f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.044+0000 7f701ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9514f5a90 0x55a9514f5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38388/0 (socket says 172.21.15.76:38388) 2023-11-09T03:49:10.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.044+0000 7f701ffff640 1 -- 172.21.15.76:0/3911391797 learned_addr learned my addr 172.21.15.76:0/3911391797 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:10.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.044+0000 7f701f7fe640 1 --2- 172.21.15.76:0/3911391797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a9514db7d0 0x55a9514f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.044+0000 7f701effd640 1 --2- 172.21.15.76:0/3911391797 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a9514dbbb0 0x55a9514dc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.044+0000 7f701ffff640 1 -- 172.21.15.76:0/3911391797 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a9514dbbb0 msgr2=0x55a9514dc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:10.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.044+0000 7f701ffff640 1 --2- 172.21.15.76:0/3911391797 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a9514dbbb0 0x55a9514dc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.044+0000 7f701ffff640 1 -- 172.21.15.76:0/3911391797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a9514db7d0 msgr2=0x55a9514f7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:10.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.044+0000 7f701ffff640 1 --2- 172.21.15.76:0/3911391797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a9514db7d0 0x55a9514f7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.044+0000 7f701ffff640 1 -- 172.21.15.76:0/3911391797 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9513daca0 con 0x55a9514f5a90 2023-11-09T03:49:10.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.045+0000 7f701ffff640 1 --2- 172.21.15.76:0/3911391797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9514f5a90 0x55a9514f5f10 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f7018001640 tx=0x7f7018001b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd64c079cdce448 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:10.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.045+0000 7f70057fa640 1 -- 172.21.15.76:0/3911391797 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7018001d50 con 0x55a9514f5a90 2023-11-09T03:49:10.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.045+0000 7f70057fa640 1 -- 172.21.15.76:0/3911391797 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70180052e0 con 0x55a9514f5a90 2023-11-09T03:49:10.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.045+0000 7f70057fa640 1 -- 172.21.15.76:0/3911391797 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7018007230 con 0x55a9514f5a90 2023-11-09T03:49:10.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.045+0000 7f7028dc6740 1 -- 172.21.15.76:0/3911391797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9514f5a90 msgr2=0x55a9514f5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:10.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.045+0000 7f7028dc6740 1 --2- 172.21.15.76:0/3911391797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9514f5a90 0x55a9514f5f10 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f7018001640 tx=0x7f7018001b00 comp rx=0 tx=0).stop 2023-11-09T03:49:10.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.045+0000 7f7028dc6740 1 -- 172.21.15.76:0/3911391797 shutdown_connections 2023-11-09T03:49:10.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.045+0000 7f7028dc6740 1 --2- 172.21.15.76:0/3911391797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9514f5a90 0x55a9514f5f10 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.045+0000 7f7028dc6740 1 --2- 172.21.15.76:0/3911391797 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a9514dbbb0 0x55a9514dc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.045+0000 7f7028dc6740 1 --2- 172.21.15.76:0/3911391797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a9514db7d0 0x55a9514f7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.045+0000 7f7028dc6740 1 -- 172.21.15.76:0/3911391797 >> 172.21.15.76:0/3911391797 conn(0x55a9514ada60 msgr2=0x55a9514f65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:10.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.045+0000 7f7028dc6740 1 -- 172.21.15.76:0/3911391797 shutdown_connections 2023-11-09T03:49:10.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.045+0000 7f7028dc6740 1 -- 172.21.15.76:0/3911391797 wait complete. 2023-11-09T03:49:10.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.046+0000 7f7028dc6740 1 -- start start 2023-11-09T03:49:10.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.046+0000 7f7028dc6740 1 Processor -- start 2023-11-09T03:49:10.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.047+0000 7f7028dc6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a9514dbbb0 0x55a9514ddce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.047+0000 7f7028dc6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9514f5a90 0x55a9514f2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.047+0000 7f7028dc6740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a9514d29f0 0x55a9514dc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.047+0000 7f7028dc6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9514178b0 con 0x55a9514f5a90 2023-11-09T03:49:10.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.047+0000 7f7028dc6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9513d1c80 con 0x55a9514dbbb0 2023-11-09T03:49:10.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.047+0000 7f7028dc6740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a9513d1920 con 0x55a9514d29f0 2023-11-09T03:49:10.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.047+0000 7f7028dc6740 10 client.0 _ll_register_callbacks cb 0x55a951129ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:49:10.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.047+0000 7f701effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9514f5a90 0x55a9514f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.047+0000 7f701effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9514f5a90 0x55a9514f2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38402/0 (socket says 172.21.15.76:38402) 2023-11-09T03:49:10.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.047+0000 7f701effd640 1 -- 172.21.15.76:0/3680696233 learned_addr learned my addr 172.21.15.76:0/3680696233 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:10.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.047+0000 7f701f7fe640 1 --2- 172.21.15.76:0/3680696233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a9514dbbb0 0x55a9514ddce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.047+0000 7f701ffff640 1 --2- 172.21.15.76:0/3680696233 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a9514d29f0 0x55a9514dc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.047+0000 7f701effd640 1 -- 172.21.15.76:0/3680696233 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a9514d29f0 msgr2=0x55a9514dc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:10.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.047+0000 7f701effd640 1 --2- 172.21.15.76:0/3680696233 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a9514d29f0 0x55a9514dc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.047+0000 7f701effd640 1 -- 172.21.15.76:0/3680696233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a9514dbbb0 msgr2=0x55a9514ddce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:10.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.047+0000 7f701effd640 1 --2- 172.21.15.76:0/3680696233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a9514dbbb0 0x55a9514ddce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.047+0000 7f701effd640 1 -- 172.21.15.76:0/3680696233 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f701800f040 con 0x55a9514f5a90 2023-11-09T03:49:10.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.047+0000 7f701effd640 1 --2- 172.21.15.76:0/3680696233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9514f5a90 0x55a9514f2040 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f701402b950 tx=0x7f701407aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3996852daba9c7ab server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:10.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.047+0000 7f6ffe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:49:10.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.047+0000 7f6ffe7fc640 1 -- 172.21.15.76:0/3680696233 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701407a180 con 0x55a9514f5a90 2023-11-09T03:49:10.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.047+0000 7f6ffe7fc640 1 -- 172.21.15.76:0/3680696233 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f7014082900 con 0x55a9514f5a90 2023-11-09T03:49:10.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.048+0000 7f7028dc6740 1 -- 172.21.15.76:0/3680696233 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9513c3bf0 con 0x55a9514f5a90 2023-11-09T03:49:10.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.048+0000 7f6ffe7fc640 1 -- 172.21.15.76:0/3680696233 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7014099400 con 0x55a9514f5a90 2023-11-09T03:49:10.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.048+0000 7f7028dc6740 10 client.7659 fetch_fsmap learned FSMap version 48 2023-11-09T03:49:10.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.048+0000 7f7028dc6740 1 -- 172.21.15.76:0/3680696233 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55a951415f30 con 0x55a9514f5a90 2023-11-09T03:49:10.068 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.048+0000 7f6ffe7fc640 1 -- 172.21.15.76:0/3680696233 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a9513c3bf0 con 0x55a9514f5a90 2023-11-09T03:49:10.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.048+0000 7f6ffe7fc640 1 -- 172.21.15.76:0/3680696233 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 48) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f70140920d0 con 0x55a9514f5a90 2023-11-09T03:49:10.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.048+0000 7f7028dc6740 10 client.7659 fetch_fsmap finished waiting for FSMap version 48 2023-11-09T03:49:10.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.048+0000 7f7028dc6740 10 client.7659 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-09T03:49:10.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.048+0000 7f7028dc6740 20 client.7659 populate_metadata read hostname 'smithi076' 2023-11-09T03:49:10.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.048+0000 7f7028dc6740 1 --2- 172.21.15.76:0/3680696233 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x55a95150bd70 0x55a95150c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.048+0000 7f7028dc6740 4 client.7659 mds_command: new command op to 7449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:49:10.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.048+0000 7f7028dc6740 1 -- 172.21.15.76:0/3680696233 --> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a951410890 con 0x55a95150bd70 2023-11-09T03:49:10.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.048+0000 7f701f7fe640 1 --2- 172.21.15.76:0/3680696233 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x55a95150bd70 0x55a95150c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.048+0000 7f6ffe7fc640 10 client.7659 ms_handle_connect on v2:172.21.15.76:6832/3697850101 2023-11-09T03:49:10.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.048+0000 7f701f7fe640 1 --2- 172.21.15.76:0/3680696233 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x55a95150bd70 0x55a95150c150 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3776e09d5d4e4238 server_cookie=99fd3bdf18d9169b in_seq=0 out_seq=0 2023-11-09T03:49:10.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.049+0000 7f6ffe7fc640 1 -- 172.21.15.76:0/3680696233 <== mds.1 v2:172.21.15.76:6832/3697850101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a951410890 con 0x55a95150bd70 2023-11-09T03:49:10.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.049+0000 7f6ffe7fc640 10 client.7659 handle_command_reply: tid=0 2023-11-09T03:49:10.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.049+0000 7f7028dc6740 1 client.7659 shutdown 2023-11-09T03:49:10.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.049+0000 7f7028dc6740 1 -- 172.21.15.76:0/3680696233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9514f5a90 msgr2=0x55a9514f2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:10.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.049+0000 7f7028dc6740 1 --2- 172.21.15.76:0/3680696233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9514f5a90 0x55a9514f2040 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f701402b950 tx=0x7f701407aa90 comp rx=0 tx=0).stop 2023-11-09T03:49:10.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.049+0000 7f7028dc6740 1 -- 172.21.15.76:0/3680696233 shutdown_connections 2023-11-09T03:49:10.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.049+0000 7f7028dc6740 1 --2- 172.21.15.76:0/3680696233 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x55a95150bd70 0x55a95150c150 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.049+0000 7f7028dc6740 1 --2- 172.21.15.76:0/3680696233 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a9514d29f0 0x55a9514dc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.049+0000 7f6ffe7fc640 0 client.7659 ms_handle_reset on v2:172.21.15.76:6832/3697850101 2023-11-09T03:49:10.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.049+0000 7f7028dc6740 1 --2- 172.21.15.76:0/3680696233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9514f5a90 0x55a9514f2040 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.049+0000 7f7028dc6740 1 --2- 172.21.15.76:0/3680696233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a9514dbbb0 0x55a9514ddce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.049+0000 7f7028dc6740 1 -- 172.21.15.76:0/3680696233 >> 172.21.15.76:0/3680696233 conn(0x55a9514db7d0 msgr2=0x55a9514f5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:10.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.049+0000 7f7028dc6740 1 -- 172.21.15.76:0/3680696233 shutdown_connections 2023-11-09T03:49:10.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.049+0000 7f7028dc6740 1 -- 172.21.15.76:0/3680696233 wait complete. 2023-11-09T03:49:10.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.049+0000 7f7028dc6740 20 client.7659 trim_cache size 0 max 16384 2023-11-09T03:49:10.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.065+0000 7f7028dc6740 1 Processor -- start 2023-11-09T03:49:10.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.065+0000 7f7028dc6740 1 -- start start 2023-11-09T03:49:10.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.065+0000 7f7028dc6740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a9514d29f0 0x55a9514f6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.066+0000 7f7028dc6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a9514dbbb0 0x55a951512180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.066+0000 7f7028dc6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9514f5a90 0x55a9515126c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.066+0000 7f7028dc6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a951397280 con 0x55a9514f5a90 2023-11-09T03:49:10.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.066+0000 7f7028dc6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9514d8e00 con 0x55a9514dbbb0 2023-11-09T03:49:10.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.066+0000 7f7028dc6740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a9514b0840 con 0x55a9514d29f0 2023-11-09T03:49:10.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.066+0000 7f701ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9514f5a90 0x55a9515126c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.066+0000 7f701ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9514f5a90 0x55a9515126c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38410/0 (socket says 172.21.15.76:38410) 2023-11-09T03:49:10.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.066+0000 7f701ffff640 1 -- 172.21.15.76:0/3755461074 learned_addr learned my addr 172.21.15.76:0/3755461074 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:10.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.066+0000 7f701effd640 1 --2- 172.21.15.76:0/3755461074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a9514dbbb0 0x55a951512180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.066+0000 7f701f7fe640 1 --2- 172.21.15.76:0/3755461074 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a9514d29f0 0x55a9514f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.066+0000 7f701ffff640 1 -- 172.21.15.76:0/3755461074 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a9514d29f0 msgr2=0x55a9514f6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:10.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.066+0000 7f701ffff640 1 --2- 172.21.15.76:0/3755461074 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a9514d29f0 0x55a9514f6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.066+0000 7f701ffff640 1 -- 172.21.15.76:0/3755461074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a9514dbbb0 msgr2=0x55a951512180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:10.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.066+0000 7f701ffff640 1 --2- 172.21.15.76:0/3755461074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a9514dbbb0 0x55a951512180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.066+0000 7f701ffff640 1 -- 172.21.15.76:0/3755461074 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9513b3d10 con 0x55a9514f5a90 2023-11-09T03:49:10.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.066+0000 7f701effd640 1 --2- 172.21.15.76:0/3755461074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a9514dbbb0 0x55a951512180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:10.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.066+0000 7f701ffff640 1 --2- 172.21.15.76:0/3755461074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9514f5a90 0x55a9515126c0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f7018001610 tx=0x7f7018005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe2dc46ee4cb60 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:10.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.066+0000 7f7005ffb640 1 -- 172.21.15.76:0/3755461074 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701800b030 con 0x55a9514f5a90 2023-11-09T03:49:10.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.066+0000 7f7005ffb640 1 -- 172.21.15.76:0/3755461074 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7018001d50 con 0x55a9514f5a90 2023-11-09T03:49:10.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.066+0000 7f7005ffb640 1 -- 172.21.15.76:0/3755461074 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7018007230 con 0x55a9514f5a90 2023-11-09T03:49:10.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.067+0000 7f7028dc6740 1 -- 172.21.15.76:0/3755461074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9514f5a90 msgr2=0x55a9515126c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:10.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.067+0000 7f7028dc6740 1 --2- 172.21.15.76:0/3755461074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9514f5a90 0x55a9515126c0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f7018001610 tx=0x7f7018005000 comp rx=0 tx=0).stop 2023-11-09T03:49:10.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.067+0000 7f7028dc6740 1 -- 172.21.15.76:0/3755461074 shutdown_connections 2023-11-09T03:49:10.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.067+0000 7f7028dc6740 1 --2- 172.21.15.76:0/3755461074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9514f5a90 0x55a9515126c0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.067+0000 7f7028dc6740 1 --2- 172.21.15.76:0/3755461074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a9514dbbb0 0x55a951512180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.067+0000 7f7028dc6740 1 --2- 172.21.15.76:0/3755461074 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a9514d29f0 0x55a9514f6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.067+0000 7f7028dc6740 1 -- 172.21.15.76:0/3755461074 >> 172.21.15.76:0/3755461074 conn(0x55a9514db7d0 msgr2=0x55a9514f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:10.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.067+0000 7f7028dc6740 1 -- 172.21.15.76:0/3755461074 shutdown_connections 2023-11-09T03:49:10.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.067+0000 7f7028dc6740 1 -- 172.21.15.76:0/3755461074 wait complete. 2023-11-09T03:49:10.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.067+0000 7f7028dc6740 1 -- start start 2023-11-09T03:49:10.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.068+0000 7f7028dc6740 1 Processor -- start 2023-11-09T03:49:10.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.068+0000 7f7028dc6740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a9514d29f0 0x55a9514afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.068+0000 7f7028dc6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a9514dbbb0 0x55a95154be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.068+0000 7f7028dc6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9514f5a90 0x55a9514b1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.068+0000 7f7028dc6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a95154c3a0 con 0x55a9514f5a90 2023-11-09T03:49:10.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.068+0000 7f7028dc6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a95154c7d0 con 0x55a9514dbbb0 2023-11-09T03:49:10.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.068+0000 7f7028dc6740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a9514d2150 con 0x55a9514d29f0 2023-11-09T03:49:10.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.068+0000 7f7028dc6740 10 client.0 _ll_register_callbacks cb 0x7f7018007ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:49:10.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.068+0000 7f701ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9514f5a90 0x55a9514b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.068+0000 7f701ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9514f5a90 0x55a9514b1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38420/0 (socket says 172.21.15.76:38420) 2023-11-09T03:49:10.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.068+0000 7f701ffff640 1 -- 172.21.15.76:0/2881450844 learned_addr learned my addr 172.21.15.76:0/2881450844 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:10.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.068+0000 7f701f7fe640 1 --2- 172.21.15.76:0/2881450844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a9514d29f0 0x55a9514afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.068+0000 7f701effd640 1 --2- 172.21.15.76:0/2881450844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a9514dbbb0 0x55a95154be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.069+0000 7f701ffff640 1 -- 172.21.15.76:0/2881450844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a9514d29f0 msgr2=0x55a9514afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:10.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.069+0000 7f701ffff640 1 --2- 172.21.15.76:0/2881450844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a9514d29f0 0x55a9514afc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.069+0000 7f701ffff640 1 -- 172.21.15.76:0/2881450844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a9514dbbb0 msgr2=0x55a95154be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:10.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.069+0000 7f701ffff640 1 --2- 172.21.15.76:0/2881450844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a9514dbbb0 0x55a95154be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.069+0000 7f701ffff640 1 -- 172.21.15.76:0/2881450844 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9513daca0 con 0x55a9514f5a90 2023-11-09T03:49:10.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.069+0000 7f701effd640 1 --2- 172.21.15.76:0/2881450844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a9514dbbb0 0x55a95154be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:10.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.069+0000 7f701ffff640 1 --2- 172.21.15.76:0/2881450844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9514f5a90 0x55a9514b1530 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f70180015e0 tx=0x7f7018000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dacfd9879c68ea40 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:10.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.069+0000 7f6ffe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:49:10.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.069+0000 7f6ffe7fc640 1 -- 172.21.15.76:0/2881450844 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7018007570 con 0x55a9514f5a90 2023-11-09T03:49:10.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.069+0000 7f6ffe7fc640 1 -- 172.21.15.76:0/2881450844 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f70180076d0 con 0x55a9514f5a90 2023-11-09T03:49:10.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.069+0000 7f7028dc6740 1 -- 172.21.15.76:0/2881450844 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9514f27f0 con 0x55a9514f5a90 2023-11-09T03:49:10.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.069+0000 7f6ffe7fc640 1 -- 172.21.15.76:0/2881450844 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70180067d0 con 0x55a9514f5a90 2023-11-09T03:49:10.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.069+0000 7f7028dc6740 10 client.7665 fetch_fsmap learned FSMap version 48 2023-11-09T03:49:10.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.069+0000 7f7028dc6740 1 -- 172.21.15.76:0/2881450844 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55a9513d9500 con 0x55a9514f5a90 2023-11-09T03:49:10.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.069+0000 7f6ffe7fc640 1 -- 172.21.15.76:0/2881450844 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a9514f27f0 con 0x55a9514f5a90 2023-11-09T03:49:10.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.069+0000 7f6ffe7fc640 1 -- 172.21.15.76:0/2881450844 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 48) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f7018027070 con 0x55a9514f5a90 2023-11-09T03:49:10.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.069+0000 7f7028dc6740 10 client.7665 fetch_fsmap finished waiting for FSMap version 48 2023-11-09T03:49:10.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.069+0000 7f7028dc6740 10 client.7665 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-09T03:49:10.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.069+0000 7f7028dc6740 20 client.7665 populate_metadata read hostname 'smithi076' 2023-11-09T03:49:10.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.069+0000 7f7028dc6740 1 --2- 172.21.15.76:0/2881450844 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x55a9514f9740 0x55a9514f9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.069+0000 7f7028dc6740 4 client.7665 mds_command: new command op to 7449 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-09T03:49:10.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.069+0000 7f7028dc6740 1 -- 172.21.15.76:0/2881450844 --> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f70180052e0 con 0x55a9514f9740 2023-11-09T03:49:10.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.069+0000 7f701f7fe640 1 --2- 172.21.15.76:0/2881450844 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x55a9514f9740 0x55a9514f9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.070+0000 7f6ffe7fc640 10 client.7665 ms_handle_connect on v2:172.21.15.76:6832/3697850101 2023-11-09T03:49:10.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.070+0000 7f701f7fe640 1 --2- 172.21.15.76:0/2881450844 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x55a9514f9740 0x55a9514f9b20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6026e50db2a1e4d server_cookie=cf842c4c9ee358f6 in_seq=0 out_seq=0 2023-11-09T03:49:10.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.070+0000 7f6ffe7fc640 1 -- 172.21.15.76:0/2881450844 <== mds.1 v2:172.21.15.76:6832/3697850101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x7f70180052e0 con 0x55a9514f9740 2023-11-09T03:49:10.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.070+0000 7f6ffe7fc640 10 client.7665 handle_command_reply: tid=0 2023-11-09T03:49:10.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.070+0000 7f7028dc6740 1 client.7665 shutdown 2023-11-09T03:49:10.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.070+0000 7f7028dc6740 1 -- 172.21.15.76:0/2881450844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9514f5a90 msgr2=0x55a9514b1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:10.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.070+0000 7f7028dc6740 1 --2- 172.21.15.76:0/2881450844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9514f5a90 0x55a9514b1530 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f70180015e0 tx=0x7f7018000ed0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.070+0000 7f7028dc6740 1 -- 172.21.15.76:0/2881450844 shutdown_connections 2023-11-09T03:49:10.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.070+0000 7f7028dc6740 1 --2- 172.21.15.76:0/2881450844 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x55a9514f9740 0x55a9514f9b20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.070+0000 7f7028dc6740 1 --2- 172.21.15.76:0/2881450844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9514f5a90 0x55a9514b1530 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.070+0000 7f7028dc6740 1 --2- 172.21.15.76:0/2881450844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a9514dbbb0 0x55a95154be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.070+0000 7f6ffe7fc640 0 client.7665 ms_handle_reset on v2:172.21.15.76:6832/3697850101 2023-11-09T03:49:10.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.070+0000 7f7028dc6740 1 --2- 172.21.15.76:0/2881450844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a9514d29f0 0x55a9514afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.070+0000 7f7028dc6740 1 -- 172.21.15.76:0/2881450844 >> 172.21.15.76:0/2881450844 conn(0x55a9514db7d0 msgr2=0x55a9514df0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:10.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.070+0000 7f7028dc6740 1 -- 172.21.15.76:0/2881450844 shutdown_connections 2023-11-09T03:49:10.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.070+0000 7f7028dc6740 1 -- 172.21.15.76:0/2881450844 wait complete. 2023-11-09T03:49:10.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.070+0000 7f7028dc6740 20 client.7665 trim_cache size 0 max 16384 2023-11-09T03:49:10.095 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:49:10.095 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "c213b26f-d0da-45d5-8e34-bb75c096d4f2", 2023-11-09T03:49:10.096 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 1, 2023-11-09T03:49:10.096 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 7449, 2023-11-09T03:49:10.096 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2023-11-09T03:49:10.096 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2023-11-09T03:49:10.096 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2023-11-09T03:49:10.097 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 2.622604071, 2023-11-09T03:49:10.097 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 48, 2023-11-09T03:49:10.097 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 95, 2023-11-09T03:49:10.097 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 95, 2023-11-09T03:49:10.097 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 18.072896592999999 2023-11-09T03:49:10.097 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:49:10.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.071+0000 7f702615e640 1 -- 172.21.15.76:0/662317337 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f700c06aea0 msgr2=0x7f700c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:10.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.071+0000 7f702615e640 1 --2- 172.21.15.76:0/662317337 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f700c06aea0 0x7f700c06d360 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f701000a400 tx=0x7f701000d040 comp rx=0 tx=0).stop 2023-11-09T03:49:10.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.071+0000 7f702615e640 1 -- 172.21.15.76:0/662317337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70200c48e0 msgr2=0x7f7020155950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:10.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.071+0000 7f702615e640 1 --2- 172.21.15.76:0/662317337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70200c48e0 0x7f7020155950 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f7014000c00 tx=0x7f70140046c0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.071+0000 7f702615e640 1 -- 172.21.15.76:0/662317337 shutdown_connections 2023-11-09T03:49:10.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.071+0000 7f702615e640 1 --2- 172.21.15.76:0/662317337 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f700c06aea0 0x7f700c06d360 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.071+0000 7f702615e640 1 --2- 172.21.15.76:0/662317337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7020159c00 0x7f7020155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.071+0000 7f702615e640 1 --2- 172.21.15.76:0/662317337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70200c48e0 0x7f7020155950 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.071+0000 7f702615e640 1 --2- 172.21.15.76:0/662317337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70200adac0 0x7f7020154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.071+0000 7f702615e640 1 -- 172.21.15.76:0/662317337 >> 172.21.15.76:0/662317337 conn(0x7f70200ce950 msgr2=0x7f702015b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:10.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.071+0000 7f702615e640 1 -- 172.21.15.76:0/662317337 shutdown_connections 2023-11-09T03:49:10.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.071+0000 7f702615e640 1 -- 172.21.15.76:0/662317337 wait complete. 2023-11-09T03:49:10.101 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-09T03:49:10.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.202+0000 7f0f94634640 1 Processor -- start 2023-11-09T03:49:10.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.202+0000 7f0f94634640 1 -- start start 2023-11-09T03:49:10.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.203+0000 7f0f94634640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0f8c153e60 0x7f0f8c15dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.203+0000 7f0f94634640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f8c0b4880 0x7f0f8c0b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.203+0000 7f0f94634640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f8c0cde60 con 0x7f0f8c0cd6d0 2023-11-09T03:49:10.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.203+0000 7f0f94634640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f8c160f90 con 0x7f0f8c153e60 2023-11-09T03:49:10.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.203+0000 7f0f94634640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c0b5200 con 0x7f0f8c0b4880 2023-11-09T03:49:10.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f91bcf640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0f8c0cd6d0 0x7f0f8c161420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53284/0 (socket says 172.21.15.76:53284) 2023-11-09T03:49:10.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f91bcf640 1 -- 172.21.15.76:0/2806197519 learned_addr learned my addr 172.21.15.76:0/2806197519 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:49:10.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f923d0640 1 --2- 172.21.15.76:0/2806197519 >> v2:172.21.15.113:3300/0 conn(0x7f0f8c153e60 0x7f0f8c15dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f92bd1640 1 --2- 172.21.15.76:0/2806197519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f8c0b4880 0x7f0f8c0b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f913ce640 1 -- 172.21.15.76:0/2806197519 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1878179408 0 0) 0x7f0f8c0cde60 con 0x7f0f8c0cd6d0 2023-11-09T03:49:10.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f913ce640 1 -- 172.21.15.76:0/2806197519 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f78003520 con 0x7f0f8c0cd6d0 2023-11-09T03:49:10.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f913ce640 1 -- 172.21.15.76:0/2806197519 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2272584119 0 0) 0x7f0f78003520 con 0x7f0f8c0cd6d0 2023-11-09T03:49:10.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f913ce640 1 -- 172.21.15.76:0/2806197519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f8c0b4880 msgr2=0x7f0f8c0b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:10.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f913ce640 1 --2- 172.21.15.76:0/2806197519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f8c0b4880 0x7f0f8c0b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f913ce640 1 -- 172.21.15.76:0/2806197519 >> v2:172.21.15.113:3300/0 conn(0x7f0f8c153e60 msgr2=0x7f0f8c15dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:10.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f913ce640 1 --2- 172.21.15.76:0/2806197519 >> v2:172.21.15.113:3300/0 conn(0x7f0f8c153e60 0x7f0f8c15dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f913ce640 1 -- 172.21.15.76:0/2806197519 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8c15e400 con 0x7f0f8c0cd6d0 2023-11-09T03:49:10.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f913ce640 1 -- 172.21.15.76:0/2806197519 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f0f7c001670 con 0x7f0f8c0cd6d0 2023-11-09T03:49:10.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f913ce640 1 -- 172.21.15.76:0/2806197519 >> v1:172.21.15.76:6789/0 conn(0x7f0f8c0cd6d0 legacy=0x7f0f8c161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:10.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f913ce640 1 --2- 172.21.15.76:0/2806197519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f8c0b4880 0x7f0f78003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f913ce640 1 --2- 172.21.15.76:0/2806197519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f78003ef0 0x7f0f780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f913ce640 1 --2- 172.21.15.76:0/2806197519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f78004d00 0x7f0f78005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f913ce640 1 -- 172.21.15.76:0/2806197519 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c160f90 con 0x7f0f78003ef0 2023-11-09T03:49:10.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f913ce640 1 -- 172.21.15.76:0/2806197519 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c0b5200 con 0x7f0f78004d00 2023-11-09T03:49:10.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f913ce640 1 -- 172.21.15.76:0/2806197519 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7800b980 con 0x7f0f8c0b4880 2023-11-09T03:49:10.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f91bcf640 1 --2- 172.21.15.76:0/2806197519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f78003ef0 0x7f0f780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f92bd1640 1 --2- 172.21.15.76:0/2806197519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f78004d00 0x7f0f78005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f923d0640 1 --2- 172.21.15.76:0/2806197519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f8c0b4880 0x7f0f78003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f91bcf640 1 -- 172.21.15.76:0/2806197519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f8c0b4880 msgr2=0x7f0f78003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:10.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f91bcf640 1 --2- 172.21.15.76:0/2806197519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f8c0b4880 0x7f0f78003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f91bcf640 1 -- 172.21.15.76:0/2806197519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f78004d00 msgr2=0x7f0f78005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:10.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f91bcf640 1 --2- 172.21.15.76:0/2806197519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f78004d00 0x7f0f78005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f91bcf640 1 -- 172.21.15.76:0/2806197519 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f78003520 con 0x7f0f78003ef0 2023-11-09T03:49:10.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f92bd1640 1 --2- 172.21.15.76:0/2806197519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f78004d00 0x7f0f78005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:10.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.204+0000 7f0f923d0640 1 --2- 172.21.15.76:0/2806197519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f8c0b4880 0x7f0f78003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:10.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.205+0000 7f0f91bcf640 1 --2- 172.21.15.76:0/2806197519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f78003ef0 0x7f0f780047c0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00c210 tx=0x7f0f780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a48861535b63aa server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:10.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.205+0000 7f0f913ce640 1 -- 172.21.15.76:0/2806197519 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c005ad0 con 0x7f0f78003ef0 2023-11-09T03:49:10.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.205+0000 7f0f913ce640 1 -- 172.21.15.76:0/2806197519 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f7c005c50 con 0x7f0f78003ef0 2023-11-09T03:49:10.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.205+0000 7f0f913ce640 1 -- 172.21.15.76:0/2806197519 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c02c880 con 0x7f0f78003ef0 2023-11-09T03:49:10.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.206+0000 7f0f94634640 1 -- 172.21.15.76:0/2806197519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f78003ef0 msgr2=0x7f0f780047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:10.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.206+0000 7f0f94634640 1 --2- 172.21.15.76:0/2806197519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f78003ef0 0x7f0f780047c0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00c210 tx=0x7f0f780057f0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.206+0000 7f0f92bd1640 1 -- 172.21.15.76:0/2806197519 reap_dead start 2023-11-09T03:49:10.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.206+0000 7f0f94634640 1 -- 172.21.15.76:0/2806197519 shutdown_connections 2023-11-09T03:49:10.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.206+0000 7f0f94634640 1 -- 172.21.15.76:0/2806197519 >> 172.21.15.76:0/2806197519 conn(0x7f0f8c0cf880 msgr2=0x7f0f8c0cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:10.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.206+0000 7f0f94634640 1 -- 172.21.15.76:0/2806197519 shutdown_connections 2023-11-09T03:49:10.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.206+0000 7f0f94634640 1 -- 172.21.15.76:0/2806197519 wait complete. 2023-11-09T03:49:10.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.207+0000 7f0f94634640 1 Processor -- start 2023-11-09T03:49:10.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.207+0000 7f0f94634640 1 -- start start 2023-11-09T03:49:10.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.208+0000 7f0f94634640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f78003ef0 0x7f0f8c150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.208+0000 7f0f94634640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f8c0ca8e0 0x7f0f8c0c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.208+0000 7f0f94634640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f8c0c80b0 0x7f0f8c155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.208+0000 7f0f94634640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c0c76e0 con 0x7f0f78003ef0 2023-11-09T03:49:10.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.208+0000 7f0f94634640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c0c6ad0 con 0x7f0f8c0ca8e0 2023-11-09T03:49:10.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.208+0000 7f0f94634640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c0c6db0 con 0x7f0f8c0c80b0 2023-11-09T03:49:10.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.208+0000 7f0f923d0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f78003ef0 0x7f0f8c150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.208+0000 7f0f923d0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f78003ef0 0x7f0f8c150370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38450/0 (socket says 172.21.15.76:38450) 2023-11-09T03:49:10.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.208+0000 7f0f923d0640 1 -- 172.21.15.76:0/939108340 learned_addr learned my addr 172.21.15.76:0/939108340 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:10.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.208+0000 7f0f923d0640 1 -- 172.21.15.76:0/939108340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f8c0c80b0 msgr2=0x7f0f8c155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:10.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.208+0000 7f0f92bd1640 1 --2- 172.21.15.76:0/939108340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f8c0c80b0 0x7f0f8c155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.208+0000 7f0f91bcf640 1 --2- 172.21.15.76:0/939108340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f8c0ca8e0 0x7f0f8c0c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.208+0000 7f0f923d0640 1 --2- 172.21.15.76:0/939108340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f8c0c80b0 0x7f0f8c155160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.208+0000 7f0f923d0640 1 -- 172.21.15.76:0/939108340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f8c0ca8e0 msgr2=0x7f0f8c0c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:10.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.208+0000 7f0f923d0640 1 --2- 172.21.15.76:0/939108340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f8c0ca8e0 0x7f0f8c0c7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.208+0000 7f0f923d0640 1 -- 172.21.15.76:0/939108340 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8c0c7090 con 0x7f0f78003ef0 2023-11-09T03:49:10.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.208+0000 7f0f923d0640 1 --2- 172.21.15.76:0/939108340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f78003ef0 0x7f0f8c150370 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f0f880116d0 tx=0x7f0f88011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:10.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.209+0000 7f0f82ffd640 1 -- 172.21.15.76:0/939108340 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8800f9c0 con 0x7f0f78003ef0 2023-11-09T03:49:10.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.209+0000 7f0f82ffd640 1 -- 172.21.15.76:0/939108340 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8800fb20 con 0x7f0f78003ef0 2023-11-09T03:49:10.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.209+0000 7f0f94634640 1 -- 172.21.15.76:0/939108340 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f8c0c7370 con 0x7f0f78003ef0 2023-11-09T03:49:10.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.209+0000 7f0f82ffd640 1 -- 172.21.15.76:0/939108340 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8800fed0 con 0x7f0f78003ef0 2023-11-09T03:49:10.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.209+0000 7f0f94634640 1 -- 172.21.15.76:0/939108340 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8c0cad60 con 0x7f0f78003ef0 2023-11-09T03:49:10.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.210+0000 7f0f82ffd640 1 -- 172.21.15.76:0/939108340 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0f88015460 con 0x7f0f78003ef0 2023-11-09T03:49:10.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.210+0000 7f0f82ffd640 1 --2- 172.21.15.76:0/939108340 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0f7006ae70 0x7f0f7006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.210+0000 7f0f82ffd640 1 -- 172.21.15.76:0/939108340 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f0f8808a420 con 0x7f0f78003ef0 2023-11-09T03:49:10.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.211+0000 7f0f91bcf640 1 --2- 172.21.15.76:0/939108340 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0f7006ae70 0x7f0f7006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.211+0000 7f0f91bcf640 1 --2- 172.21.15.76:0/939108340 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0f7006ae70 0x7f0f7006d330 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c034000 tx=0x7f0f7c007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:10.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.211+0000 7f0f971c7740 1 Processor -- start 2023-11-09T03:49:10.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.212+0000 7f0f971c7740 1 -- start start 2023-11-09T03:49:10.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.212+0000 7f0f971c7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bc1a5d850 0x555bc1a30750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.212+0000 7f0f971c7740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555bc1a5dc30 0x555bc1a605e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.212+0000 7f0f971c7740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555bc1a30c90 0x555bc1a60b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.212+0000 7f0f971c7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bc1941740 con 0x555bc1a5d850 2023-11-09T03:49:10.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.212+0000 7f0f971c7740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bc19a38d0 con 0x555bc1a5dc30 2023-11-09T03:49:10.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.212+0000 7f0f971c7740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x555bc19a0fc0 con 0x555bc1a30c90 2023-11-09T03:49:10.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.212+0000 7f0f923d0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bc1a5d850 0x555bc1a30750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.212+0000 7f0f923d0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bc1a5d850 0x555bc1a30750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38460/0 (socket says 172.21.15.76:38460) 2023-11-09T03:49:10.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.212+0000 7f0f923d0640 1 -- 172.21.15.76:0/2256850141 learned_addr learned my addr 172.21.15.76:0/2256850141 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:10.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.213+0000 7f0f91bcf640 1 --2- 172.21.15.76:0/2256850141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555bc1a5dc30 0x555bc1a605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.213+0000 7f0f92bd1640 1 --2- 172.21.15.76:0/2256850141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555bc1a30c90 0x555bc1a60b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.213+0000 7f0f923d0640 1 -- 172.21.15.76:0/2256850141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555bc1a30c90 msgr2=0x555bc1a60b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:10.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.213+0000 7f0f923d0640 1 --2- 172.21.15.76:0/2256850141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555bc1a30c90 0x555bc1a60b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.213+0000 7f0f923d0640 1 -- 172.21.15.76:0/2256850141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555bc1a5dc30 msgr2=0x555bc1a605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:10.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.213+0000 7f0f923d0640 1 --2- 172.21.15.76:0/2256850141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555bc1a5dc30 0x555bc1a605e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.213+0000 7f0f923d0640 1 -- 172.21.15.76:0/2256850141 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bc195fca0 con 0x555bc1a5d850 2023-11-09T03:49:10.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.213+0000 7f0f91bcf640 1 --2- 172.21.15.76:0/2256850141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555bc1a5dc30 0x555bc1a605e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:10.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.213+0000 7f0f923d0640 1 --2- 172.21.15.76:0/2256850141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bc1a5d850 0x555bc1a30750 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f0f88088400 tx=0x7f0f8805cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b20939ac327058 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:10.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.213+0000 7f0f6ffff640 1 -- 172.21.15.76:0/2256850141 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f880570f0 con 0x555bc1a5d850 2023-11-09T03:49:10.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.213+0000 7f0f6ffff640 1 -- 172.21.15.76:0/2256850141 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f88055460 con 0x555bc1a5d850 2023-11-09T03:49:10.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.213+0000 7f0f6ffff640 1 -- 172.21.15.76:0/2256850141 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f880555d0 con 0x555bc1a5d850 2023-11-09T03:49:10.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.214+0000 7f0f971c7740 1 -- 172.21.15.76:0/2256850141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bc1a5d850 msgr2=0x555bc1a30750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:10.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.214+0000 7f0f971c7740 1 --2- 172.21.15.76:0/2256850141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bc1a5d850 0x555bc1a30750 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f0f88088400 tx=0x7f0f8805cdf0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.214+0000 7f0f971c7740 1 -- 172.21.15.76:0/2256850141 shutdown_connections 2023-11-09T03:49:10.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.214+0000 7f0f971c7740 1 --2- 172.21.15.76:0/2256850141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555bc1a30c90 0x555bc1a60b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.214+0000 7f0f971c7740 1 --2- 172.21.15.76:0/2256850141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555bc1a5dc30 0x555bc1a605e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.214+0000 7f0f971c7740 1 --2- 172.21.15.76:0/2256850141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bc1a5d850 0x555bc1a30750 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.214+0000 7f0f971c7740 1 -- 172.21.15.76:0/2256850141 >> 172.21.15.76:0/2256850141 conn(0x555bc1a32a60 msgr2=0x555bc1a7b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:10.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.214+0000 7f0f971c7740 1 -- 172.21.15.76:0/2256850141 shutdown_connections 2023-11-09T03:49:10.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.214+0000 7f0f971c7740 1 -- 172.21.15.76:0/2256850141 wait complete. 2023-11-09T03:49:10.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.214+0000 7f0f971c7740 1 -- start start 2023-11-09T03:49:10.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.214+0000 7f0f971c7740 1 Processor -- start 2023-11-09T03:49:10.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.215+0000 7f0f971c7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bc1a5d850 0x555bc1a75de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.215+0000 7f0f971c7740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555bc1a5dc30 0x555bc1a64140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.215+0000 7f0f971c7740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555bc1a51c40 0x555bc1a63320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.215+0000 7f0f971c7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bc199c8b0 con 0x555bc1a5d850 2023-11-09T03:49:10.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.215+0000 7f0f971c7740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bc1956c80 con 0x555bc1a5dc30 2023-11-09T03:49:10.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.215+0000 7f0f971c7740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x555bc1956920 con 0x555bc1a51c40 2023-11-09T03:49:10.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.215+0000 7f0f971c7740 10 client.0 _ll_register_callbacks cb 0x555bc16aeac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:49:10.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.215+0000 7f0f923d0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bc1a5d850 0x555bc1a75de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.215+0000 7f0f923d0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bc1a5d850 0x555bc1a75de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38472/0 (socket says 172.21.15.76:38472) 2023-11-09T03:49:10.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.215+0000 7f0f923d0640 1 -- 172.21.15.76:0/3601826491 learned_addr learned my addr 172.21.15.76:0/3601826491 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:10.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.215+0000 7f0f91bcf640 1 --2- 172.21.15.76:0/3601826491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555bc1a5dc30 0x555bc1a64140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.215+0000 7f0f92bd1640 1 --2- 172.21.15.76:0/3601826491 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555bc1a51c40 0x555bc1a63320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.216+0000 7f0f923d0640 1 -- 172.21.15.76:0/3601826491 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555bc1a51c40 msgr2=0x555bc1a63320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:10.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.216+0000 7f0f923d0640 1 --2- 172.21.15.76:0/3601826491 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555bc1a51c40 0x555bc1a63320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.216+0000 7f0f923d0640 1 -- 172.21.15.76:0/3601826491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555bc1a5dc30 msgr2=0x555bc1a64140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:10.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.216+0000 7f0f923d0640 1 --2- 172.21.15.76:0/3601826491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555bc1a5dc30 0x555bc1a64140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.216+0000 7f0f923d0640 1 -- 172.21.15.76:0/3601826491 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8c0cad60 con 0x555bc1a5d850 2023-11-09T03:49:10.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.216+0000 7f0f923d0640 1 --2- 172.21.15.76:0/3601826491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bc1a5d850 0x555bc1a75de0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f0f88058a00 tx=0x7f0f88000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f746dea7be702458 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:10.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.216+0000 7f0f6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:49:10.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.216+0000 7f0f6d7fa640 1 -- 172.21.15.76:0/3601826491 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f88058440 con 0x555bc1a5d850 2023-11-09T03:49:10.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.216+0000 7f0f6d7fa640 1 -- 172.21.15.76:0/3601826491 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f0f88060b10 con 0x555bc1a5d850 2023-11-09T03:49:10.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.216+0000 7f0f971c7740 1 -- 172.21.15.76:0/3601826491 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555bc1948bf0 con 0x555bc1a5d850 2023-11-09T03:49:10.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.216+0000 7f0f6d7fa640 1 -- 172.21.15.76:0/3601826491 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f88060cd0 con 0x555bc1a5d850 2023-11-09T03:49:10.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.216+0000 7f0f971c7740 10 client.7677 fetch_fsmap learned FSMap version 48 2023-11-09T03:49:10.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.216+0000 7f0f971c7740 1 -- 172.21.15.76:0/3601826491 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x555bc199af30 con 0x555bc1a5d850 2023-11-09T03:49:10.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.216+0000 7f0f6d7fa640 1 -- 172.21.15.76:0/3601826491 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8807f020 con 0x555bc1a5d850 2023-11-09T03:49:10.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.216+0000 7f0f6d7fa640 1 -- 172.21.15.76:0/3601826491 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 48) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f0f8801b020 con 0x555bc1a5d850 2023-11-09T03:49:10.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.216+0000 7f0f971c7740 10 client.7677 fetch_fsmap finished waiting for FSMap version 48 2023-11-09T03:49:10.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.216+0000 7f0f971c7740 10 client.7677 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-09T03:49:10.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.216+0000 7f0f971c7740 20 client.7677 populate_metadata read hostname 'smithi076' 2023-11-09T03:49:10.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.216+0000 7f0f971c7740 1 --2- 172.21.15.76:0/3601826491 >> [v2:172.21.15.76:6834/4011275120,v1:172.21.15.76:6835/4011275120] conn(0x555bc1a747a0 0x555bc1a98a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.216+0000 7f0f971c7740 4 client.7677 mds_command: new command op to 7452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:49:10.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.216+0000 7f0f971c7740 1 -- 172.21.15.76:0/3601826491 --> [v2:172.21.15.76:6834/4011275120,v1:172.21.15.76:6835/4011275120] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555bc1995890 con 0x555bc1a747a0 2023-11-09T03:49:10.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.216+0000 7f0f91bcf640 1 --2- 172.21.15.76:0/3601826491 >> [v2:172.21.15.76:6834/4011275120,v1:172.21.15.76:6835/4011275120] conn(0x555bc1a747a0 0x555bc1a98a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.217+0000 7f0f91bcf640 1 --2- 172.21.15.76:0/3601826491 >> [v2:172.21.15.76:6834/4011275120,v1:172.21.15.76:6835/4011275120] conn(0x555bc1a747a0 0x555bc1a98a70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f7ceb9166a8e4b1 server_cookie=f658dbbf2e2652af in_seq=0 out_seq=0 2023-11-09T03:49:10.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.217+0000 7f0f6d7fa640 10 client.7677 ms_handle_connect on v2:172.21.15.76:6834/4011275120 2023-11-09T03:49:10.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.217+0000 7f0f6d7fa640 1 -- 172.21.15.76:0/3601826491 <== mds.0 v2:172.21.15.76:6834/4011275120 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555bc1995890 con 0x555bc1a747a0 2023-11-09T03:49:10.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.217+0000 7f0f6d7fa640 10 client.7677 handle_command_reply: tid=0 2023-11-09T03:49:10.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.217+0000 7f0f971c7740 1 client.7677 shutdown 2023-11-09T03:49:10.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.217+0000 7f0f971c7740 1 -- 172.21.15.76:0/3601826491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bc1a5d850 msgr2=0x555bc1a75de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:10.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.217+0000 7f0f971c7740 1 --2- 172.21.15.76:0/3601826491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bc1a5d850 0x555bc1a75de0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f0f88058a00 tx=0x7f0f88000bc0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.217+0000 7f0f971c7740 1 -- 172.21.15.76:0/3601826491 shutdown_connections 2023-11-09T03:49:10.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.217+0000 7f0f971c7740 1 --2- 172.21.15.76:0/3601826491 >> [v2:172.21.15.76:6834/4011275120,v1:172.21.15.76:6835/4011275120] conn(0x555bc1a747a0 0x555bc1a98a70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.217+0000 7f0f971c7740 1 --2- 172.21.15.76:0/3601826491 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555bc1a51c40 0x555bc1a63320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.217+0000 7f0f971c7740 1 --2- 172.21.15.76:0/3601826491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555bc1a5dc30 0x555bc1a64140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.217+0000 7f0f6d7fa640 0 client.7677 ms_handle_reset on v2:172.21.15.76:6834/4011275120 2023-11-09T03:49:10.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.217+0000 7f0f971c7740 1 --2- 172.21.15.76:0/3601826491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bc1a5d850 0x555bc1a75de0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.217+0000 7f0f971c7740 1 -- 172.21.15.76:0/3601826491 >> 172.21.15.76:0/3601826491 conn(0x555bc1a30c90 msgr2=0x555bc1a75440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:10.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.218+0000 7f0f971c7740 1 -- 172.21.15.76:0/3601826491 shutdown_connections 2023-11-09T03:49:10.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.218+0000 7f0f971c7740 1 -- 172.21.15.76:0/3601826491 wait complete. 2023-11-09T03:49:10.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.218+0000 7f0f971c7740 20 client.7677 trim_cache size 0 max 16384 2023-11-09T03:49:10.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.233+0000 7f0f971c7740 1 Processor -- start 2023-11-09T03:49:10.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.233+0000 7f0f971c7740 1 -- start start 2023-11-09T03:49:10.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.234+0000 7f0f971c7740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555bc1a51c40 0x555bc1a349a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.234+0000 7f0f971c7740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555bc1a5d850 0x555bc1a8b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.234+0000 7f0f971c7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bc1a5dc30 0x555bc1a63d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.234+0000 7f0f971c7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bc191c280 con 0x555bc1a5dc30 2023-11-09T03:49:10.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.234+0000 7f0f971c7740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bc1a954f0 con 0x555bc1a51c40 2023-11-09T03:49:10.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.234+0000 7f0f971c7740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x555bc1a9b210 con 0x555bc1a5d850 2023-11-09T03:49:10.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.234+0000 7f0f92bd1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bc1a5dc30 0x555bc1a63d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.234+0000 7f0f92bd1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bc1a5dc30 0x555bc1a63d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38488/0 (socket says 172.21.15.76:38488) 2023-11-09T03:49:10.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.234+0000 7f0f92bd1640 1 -- 172.21.15.76:0/4103846419 learned_addr learned my addr 172.21.15.76:0/4103846419 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:10.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.234+0000 7f0f923d0640 1 --2- 172.21.15.76:0/4103846419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555bc1a51c40 0x555bc1a349a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.234+0000 7f0f92bd1640 1 -- 172.21.15.76:0/4103846419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555bc1a5d850 msgr2=0x555bc1a8b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:10.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.234+0000 7f0f91bcf640 1 --2- 172.21.15.76:0/4103846419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555bc1a5d850 0x555bc1a8b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.234+0000 7f0f92bd1640 1 --2- 172.21.15.76:0/4103846419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555bc1a5d850 0x555bc1a8b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.234+0000 7f0f92bd1640 1 -- 172.21.15.76:0/4103846419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555bc1a51c40 msgr2=0x555bc1a349a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:10.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.234+0000 7f0f92bd1640 1 --2- 172.21.15.76:0/4103846419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555bc1a51c40 0x555bc1a349a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.234+0000 7f0f92bd1640 1 -- 172.21.15.76:0/4103846419 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bc1938d10 con 0x555bc1a5dc30 2023-11-09T03:49:10.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.234+0000 7f0f923d0640 1 --2- 172.21.15.76:0/4103846419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555bc1a51c40 0x555bc1a349a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:49:10.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.234+0000 7f0f91bcf640 1 --2- 172.21.15.76:0/4103846419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555bc1a5d850 0x555bc1a8b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:49:10.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.234+0000 7f0f92bd1640 1 --2- 172.21.15.76:0/4103846419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bc1a5dc30 0x555bc1a63d20 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f0f840153e0 tx=0x7f0f84009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bffe0d9f806ba261 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:10.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.235+0000 7f0f80ff9640 1 -- 172.21.15.76:0/4103846419 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8400e040 con 0x555bc1a5dc30 2023-11-09T03:49:10.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.235+0000 7f0f80ff9640 1 -- 172.21.15.76:0/4103846419 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f84017820 con 0x555bc1a5dc30 2023-11-09T03:49:10.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.235+0000 7f0f80ff9640 1 -- 172.21.15.76:0/4103846419 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f84017bb0 con 0x555bc1a5dc30 2023-11-09T03:49:10.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.235+0000 7f0f971c7740 1 -- 172.21.15.76:0/4103846419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bc1a5dc30 msgr2=0x555bc1a63d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:10.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.235+0000 7f0f971c7740 1 --2- 172.21.15.76:0/4103846419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bc1a5dc30 0x555bc1a63d20 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f0f840153e0 tx=0x7f0f84009750 comp rx=0 tx=0).stop 2023-11-09T03:49:10.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.235+0000 7f0f971c7740 1 -- 172.21.15.76:0/4103846419 shutdown_connections 2023-11-09T03:49:10.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.235+0000 7f0f971c7740 1 --2- 172.21.15.76:0/4103846419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bc1a5dc30 0x555bc1a63d20 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.235+0000 7f0f971c7740 1 --2- 172.21.15.76:0/4103846419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555bc1a5d850 0x555bc1a8b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.235+0000 7f0f971c7740 1 --2- 172.21.15.76:0/4103846419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555bc1a51c40 0x555bc1a349a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.235+0000 7f0f971c7740 1 -- 172.21.15.76:0/4103846419 >> 172.21.15.76:0/4103846419 conn(0x555bc1a30c90 msgr2=0x555bc1a7dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:10.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.235+0000 7f0f971c7740 1 -- 172.21.15.76:0/4103846419 shutdown_connections 2023-11-09T03:49:10.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.235+0000 7f0f971c7740 1 -- 172.21.15.76:0/4103846419 wait complete. 2023-11-09T03:49:10.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.236+0000 7f0f971c7740 1 -- start start 2023-11-09T03:49:10.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.236+0000 7f0f971c7740 1 Processor -- start 2023-11-09T03:49:10.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.237+0000 7f0f971c7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bc1a51c40 0x555bc1a34350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.237+0000 7f0f971c7740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555bc1a5d850 0x555bc1a34890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.237+0000 7f0f971c7740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555bc1a5dc30 0x555bc1a60650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.237+0000 7f0f971c7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bc1a516d0 con 0x555bc1a51c40 2023-11-09T03:49:10.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.237+0000 7f0f971c7740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bc1a99170 con 0x555bc1a5d850 2023-11-09T03:49:10.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.237+0000 7f0f971c7740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x555bc1a36760 con 0x555bc1a5dc30 2023-11-09T03:49:10.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.237+0000 7f0f971c7740 10 client.0 _ll_register_callbacks cb 0x7f0f88058b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:49:10.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.237+0000 7f0f923d0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bc1a51c40 0x555bc1a34350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.237+0000 7f0f923d0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bc1a51c40 0x555bc1a34350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38492/0 (socket says 172.21.15.76:38492) 2023-11-09T03:49:10.253 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.237+0000 7f0f923d0640 1 -- 172.21.15.76:0/2911493445 learned_addr learned my addr 172.21.15.76:0/2911493445 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:10.253 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.237+0000 7f0f91bcf640 1 --2- 172.21.15.76:0/2911493445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555bc1a5d850 0x555bc1a34890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.253 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.237+0000 7f0f92bd1640 1 --2- 172.21.15.76:0/2911493445 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555bc1a5dc30 0x555bc1a60650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.253 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.237+0000 7f0f923d0640 1 -- 172.21.15.76:0/2911493445 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555bc1a5dc30 msgr2=0x555bc1a60650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:10.253 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.237+0000 7f0f923d0640 1 --2- 172.21.15.76:0/2911493445 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555bc1a5dc30 0x555bc1a60650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.253 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.237+0000 7f0f923d0640 1 -- 172.21.15.76:0/2911493445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555bc1a5d850 msgr2=0x555bc1a34890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:10.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.237+0000 7f0f923d0640 1 --2- 172.21.15.76:0/2911493445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555bc1a5d850 0x555bc1a34890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.237+0000 7f0f923d0640 1 -- 172.21.15.76:0/2911493445 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f84009da0 con 0x555bc1a51c40 2023-11-09T03:49:10.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.237+0000 7f0f92bd1640 1 --2- 172.21.15.76:0/2911493445 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555bc1a5dc30 0x555bc1a60650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:10.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.237+0000 7f0f91bcf640 1 --2- 172.21.15.76:0/2911493445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555bc1a5d850 0x555bc1a34890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:10.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.237+0000 7f0f923d0640 1 --2- 172.21.15.76:0/2911493445 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bc1a51c40 0x555bc1a34350 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f0f8808a700 tx=0x7f0f88058c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8369f5333559961 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:10.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.237+0000 7f0f6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:49:10.255 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.237+0000 7f0f6d7fa640 1 -- 172.21.15.76:0/2911493445 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f88054b40 con 0x555bc1a51c40 2023-11-09T03:49:10.255 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.237+0000 7f0f6d7fa640 1 -- 172.21.15.76:0/2911493445 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f0f88080080 con 0x555bc1a51c40 2023-11-09T03:49:10.255 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.237+0000 7f0f971c7740 1 -- 172.21.15.76:0/2911493445 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f88055460 con 0x555bc1a51c40 2023-11-09T03:49:10.255 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.237+0000 7f0f6d7fa640 1 -- 172.21.15.76:0/2911493445 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f880555d0 con 0x555bc1a51c40 2023-11-09T03:49:10.255 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.237+0000 7f0f971c7740 10 client.7683 fetch_fsmap learned FSMap version 48 2023-11-09T03:49:10.255 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.238+0000 7f0f971c7740 1 -- 172.21.15.76:0/2911493445 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x555bc195e500 con 0x555bc1a51c40 2023-11-09T03:49:10.256 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.238+0000 7f0f6d7fa640 1 -- 172.21.15.76:0/2911493445 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f88055460 con 0x555bc1a51c40 2023-11-09T03:49:10.256 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.238+0000 7f0f6d7fa640 1 -- 172.21.15.76:0/2911493445 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 48) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f0f8805db30 con 0x555bc1a51c40 2023-11-09T03:49:10.256 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.238+0000 7f0f971c7740 10 client.7683 fetch_fsmap finished waiting for FSMap version 48 2023-11-09T03:49:10.256 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.238+0000 7f0f971c7740 10 client.7683 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-09T03:49:10.256 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.238+0000 7f0f971c7740 20 client.7683 populate_metadata read hostname 'smithi076' 2023-11-09T03:49:10.256 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.238+0000 7f0f971c7740 1 --2- 172.21.15.76:0/2911493445 >> [v2:172.21.15.76:6834/4011275120,v1:172.21.15.76:6835/4011275120] conn(0x555bc1ae5770 0x555bc1a60b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.257 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.238+0000 7f0f971c7740 4 client.7683 mds_command: new command op to 7452 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-09T03:49:10.257 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.238+0000 7f0f971c7740 1 -- 172.21.15.76:0/2911493445 --> [v2:172.21.15.76:6834/4011275120,v1:172.21.15.76:6835/4011275120] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555bc19955b0 con 0x555bc1ae5770 2023-11-09T03:49:10.257 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.238+0000 7f0f91bcf640 1 --2- 172.21.15.76:0/2911493445 >> [v2:172.21.15.76:6834/4011275120,v1:172.21.15.76:6835/4011275120] conn(0x555bc1ae5770 0x555bc1a60b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.257 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.238+0000 7f0f6d7fa640 10 client.7683 ms_handle_connect on v2:172.21.15.76:6834/4011275120 2023-11-09T03:49:10.257 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.238+0000 7f0f91bcf640 1 --2- 172.21.15.76:0/2911493445 >> [v2:172.21.15.76:6834/4011275120,v1:172.21.15.76:6835/4011275120] conn(0x555bc1ae5770 0x555bc1a60b90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41ab0aa381e407b8 server_cookie=f8f1ca7db086d3ea in_seq=0 out_seq=0 2023-11-09T03:49:10.258 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.238+0000 7f0f6d7fa640 1 -- 172.21.15.76:0/2911493445 <== mds.0 v2:172.21.15.76:6834/4011275120 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x555bc19955b0 con 0x555bc1ae5770 2023-11-09T03:49:10.258 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.238+0000 7f0f6d7fa640 10 client.7683 handle_command_reply: tid=0 2023-11-09T03:49:10.258 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.238+0000 7f0f971c7740 1 client.7683 shutdown 2023-11-09T03:49:10.258 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.238+0000 7f0f971c7740 1 -- 172.21.15.76:0/2911493445 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bc1a51c40 msgr2=0x555bc1a34350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:10.258 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.238+0000 7f0f971c7740 1 --2- 172.21.15.76:0/2911493445 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bc1a51c40 0x555bc1a34350 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f0f8808a700 tx=0x7f0f88058c50 comp rx=0 tx=0).stop 2023-11-09T03:49:10.258 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.239+0000 7f0f971c7740 1 -- 172.21.15.76:0/2911493445 shutdown_connections 2023-11-09T03:49:10.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.239+0000 7f0f971c7740 1 --2- 172.21.15.76:0/2911493445 >> [v2:172.21.15.76:6834/4011275120,v1:172.21.15.76:6835/4011275120] conn(0x555bc1ae5770 0x555bc1a60b90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.239+0000 7f0f971c7740 1 --2- 172.21.15.76:0/2911493445 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555bc1a5dc30 0x555bc1a60650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.239+0000 7f0f971c7740 1 --2- 172.21.15.76:0/2911493445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555bc1a5d850 0x555bc1a34890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.239+0000 7f0f971c7740 1 --2- 172.21.15.76:0/2911493445 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555bc1a51c40 0x555bc1a34350 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.239+0000 7f0f6d7fa640 0 client.7683 ms_handle_reset on v2:172.21.15.76:6834/4011275120 2023-11-09T03:49:10.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.239+0000 7f0f971c7740 1 -- 172.21.15.76:0/2911493445 >> 172.21.15.76:0/2911493445 conn(0x555bc1a30c90 msgr2=0x555bc1a63fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:10.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.239+0000 7f0f971c7740 1 -- 172.21.15.76:0/2911493445 shutdown_connections 2023-11-09T03:49:10.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.239+0000 7f0f971c7740 1 -- 172.21.15.76:0/2911493445 wait complete. 2023-11-09T03:49:10.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.239+0000 7f0f971c7740 20 client.7683 trim_cache size 0 max 16384 2023-11-09T03:49:10.260 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:49:10.260 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "c213b26f-d0da-45d5-8e34-bb75c096d4f2", 2023-11-09T03:49:10.261 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2023-11-09T03:49:10.261 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 7452, 2023-11-09T03:49:10.261 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2023-11-09T03:49:10.261 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2023-11-09T03:49:10.261 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2023-11-09T03:49:10.261 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 12.197059992, 2023-11-09T03:49:10.262 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 48, 2023-11-09T03:49:10.262 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 95, 2023-11-09T03:49:10.262 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 95, 2023-11-09T03:49:10.262 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 18.240247507999999 2023-11-09T03:49:10.262 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:49:10.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.239+0000 7f0f94634640 1 -- 172.21.15.76:0/939108340 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0f7006ae70 msgr2=0x7f0f7006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:10.264 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.239+0000 7f0f94634640 1 --2- 172.21.15.76:0/939108340 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0f7006ae70 0x7f0f7006d330 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c034000 tx=0x7f0f7c007000 comp rx=0 tx=0).stop 2023-11-09T03:49:10.264 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.239+0000 7f0f94634640 1 -- 172.21.15.76:0/939108340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f78003ef0 msgr2=0x7f0f8c150370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:10.264 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.239+0000 7f0f94634640 1 --2- 172.21.15.76:0/939108340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f78003ef0 0x7f0f8c150370 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f0f880116d0 tx=0x7f0f88011b90 comp rx=0 tx=0).stop 2023-11-09T03:49:10.264 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.239+0000 7f0f94634640 1 -- 172.21.15.76:0/939108340 shutdown_connections 2023-11-09T03:49:10.264 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.239+0000 7f0f94634640 1 --2- 172.21.15.76:0/939108340 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0f7006ae70 0x7f0f7006d330 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.239+0000 7f0f94634640 1 --2- 172.21.15.76:0/939108340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f8c0c80b0 0x7f0f8c155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.239+0000 7f0f94634640 1 --2- 172.21.15.76:0/939108340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f8c0ca8e0 0x7f0f8c0c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.239+0000 7f0f94634640 1 --2- 172.21.15.76:0/939108340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f78003ef0 0x7f0f8c150370 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.239+0000 7f0f94634640 1 -- 172.21.15.76:0/939108340 >> 172.21.15.76:0/939108340 conn(0x7f0f8c0cf880 msgr2=0x7f0f8c0cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:10.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.239+0000 7f0f94634640 1 -- 172.21.15.76:0/939108340 shutdown_connections 2023-11-09T03:49:10.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.239+0000 7f0f94634640 1 -- 172.21.15.76:0/939108340 wait complete. 2023-11-09T03:49:10.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:49:10.266 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/dir) 2023-11-09T03:49:10.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:49:10.286 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1) 2023-11-09T03:49:10.343 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:49:10.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.484+0000 7fa99920e640 1 Processor -- start 2023-11-09T03:49:10.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.484+0000 7fa99920e640 1 -- start start 2023-11-09T03:49:10.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.485+0000 7fa99920e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa994159c70 0x7fa9940ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.485+0000 7fa99920e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9940b4110 con 0x7fa99406db20 2023-11-09T03:49:10.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.485+0000 7fa99920e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9940b43f0 con 0x7fa9940b0ae0 2023-11-09T03:49:10.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.485+0000 7fa99920e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9940b49b0 con 0x7fa994159c70 2023-11-09T03:49:10.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.485+0000 7fa99259c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa99406db20 0x7fa9940b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53300/0 (socket says 172.21.15.76:53300) 2023-11-09T03:49:10.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.485+0000 7fa99259c640 1 -- 172.21.15.76:0/3305815755 learned_addr learned my addr 172.21.15.76:0/3305815755 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:49:10.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.485+0000 7fa991d9b640 1 -- 172.21.15.76:0/3305815755 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1906189055 0 0) 0x7fa9940b4110 con 0x7fa99406db20 2023-11-09T03:49:10.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.485+0000 7fa992d9d640 1 --2- 172.21.15.76:0/3305815755 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa994159c70 0x7fa9940ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.485+0000 7fa991d9b640 1 -- 172.21.15.76:0/3305815755 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa97c003520 con 0x7fa99406db20 2023-11-09T03:49:10.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.485+0000 7fa991d9b640 1 -- 172.21.15.76:0/3305815755 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 865783313 0 0) 0x7fa97c003520 con 0x7fa99406db20 2023-11-09T03:49:10.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.485+0000 7fa991d9b640 1 -- 172.21.15.76:0/3305815755 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa994159c70 msgr2=0x7fa9940ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:10.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.485+0000 7fa991d9b640 1 --2- 172.21.15.76:0/3305815755 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa994159c70 0x7fa9940ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.485+0000 7fa991d9b640 1 -- 172.21.15.76:0/3305815755 >> v1:172.21.15.113:6789/0 conn(0x7fa9940b0ae0 legacy=0x7fa99415d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:10.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.485+0000 7fa991d9b640 1 -- 172.21.15.76:0/3305815755 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9940b46d0 con 0x7fa99406db20 2023-11-09T03:49:10.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.485+0000 7fa991d9b640 1 -- 172.21.15.76:0/3305815755 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fa988001670 con 0x7fa99406db20 2023-11-09T03:49:10.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.485+0000 7fa991d9b640 1 -- 172.21.15.76:0/3305815755 >> v1:172.21.15.76:6789/0 conn(0x7fa99406db20 legacy=0x7fa9940b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:10.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.485+0000 7fa991d9b640 1 --2- 172.21.15.76:0/3305815755 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa994159c70 0x7fa97c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.485+0000 7fa991d9b640 1 --2- 172.21.15.76:0/3305815755 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa97c003f20 0x7fa97c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.485+0000 7fa991d9b640 1 --2- 172.21.15.76:0/3305815755 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa97c004d00 0x7fa97c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.486+0000 7fa991d9b640 1 -- 172.21.15.76:0/3305815755 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9940b49b0 con 0x7fa97c004d00 2023-11-09T03:49:10.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.486+0000 7fa991d9b640 1 -- 172.21.15.76:0/3305815755 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97c00ba60 con 0x7fa97c003f20 2023-11-09T03:49:10.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.486+0000 7fa991d9b640 1 -- 172.21.15.76:0/3305815755 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa97c00bc20 con 0x7fa994159c70 2023-11-09T03:49:10.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.486+0000 7fa99359e640 1 --2- 172.21.15.76:0/3305815755 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa97c004d00 0x7fa97c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.486+0000 7fa99259c640 1 --2- 172.21.15.76:0/3305815755 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa97c003f20 0x7fa97c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.486+0000 7fa992d9d640 1 --2- 172.21.15.76:0/3305815755 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa994159c70 0x7fa97c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.486+0000 7fa99359e640 1 -- 172.21.15.76:0/3305815755 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa994159c70 msgr2=0x7fa97c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:10.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.486+0000 7fa99359e640 1 --2- 172.21.15.76:0/3305815755 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa994159c70 0x7fa97c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.486+0000 7fa99359e640 1 -- 172.21.15.76:0/3305815755 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa97c003f20 msgr2=0x7fa97c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:10.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.486+0000 7fa99359e640 1 --2- 172.21.15.76:0/3305815755 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa97c003f20 0x7fa97c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.486+0000 7fa99359e640 1 -- 172.21.15.76:0/3305815755 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa97c003520 con 0x7fa97c004d00 2023-11-09T03:49:10.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.486+0000 7fa99259c640 1 --2- 172.21.15.76:0/3305815755 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa97c003f20 0x7fa97c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:10.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.486+0000 7fa992d9d640 1 --2- 172.21.15.76:0/3305815755 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa994159c70 0x7fa97c0039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:10.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.487+0000 7fa99359e640 1 --2- 172.21.15.76:0/3305815755 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa97c004d00 0x7fa97c005180 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fa98c001e50 tx=0x7fa97c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41fc12df10bbaf6 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:10.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.487+0000 7fa991d9b640 1 -- 172.21.15.76:0/3305815755 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c003b70 con 0x7fa97c004d00 2023-11-09T03:49:10.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.487+0000 7fa991d9b640 1 -- 172.21.15.76:0/3305815755 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98c003cd0 con 0x7fa97c004d00 2023-11-09T03:49:10.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.487+0000 7fa991d9b640 1 -- 172.21.15.76:0/3305815755 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c005310 con 0x7fa97c004d00 2023-11-09T03:49:10.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.487+0000 7fa99920e640 1 -- 172.21.15.76:0/3305815755 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa97c004d00 msgr2=0x7fa97c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:10.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.487+0000 7fa99920e640 1 --2- 172.21.15.76:0/3305815755 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa97c004d00 0x7fa97c005180 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fa98c001e50 tx=0x7fa97c0057f0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.487+0000 7fa99359e640 1 -- 172.21.15.76:0/3305815755 reap_dead start 2023-11-09T03:49:10.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.487+0000 7fa99920e640 1 -- 172.21.15.76:0/3305815755 shutdown_connections 2023-11-09T03:49:10.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.487+0000 7fa99920e640 1 -- 172.21.15.76:0/3305815755 >> 172.21.15.76:0/3305815755 conn(0x7fa9940ce950 msgr2=0x7fa9940ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:10.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.487+0000 7fa99920e640 1 -- 172.21.15.76:0/3305815755 shutdown_connections 2023-11-09T03:49:10.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.487+0000 7fa99920e640 1 -- 172.21.15.76:0/3305815755 wait complete. 2023-11-09T03:49:10.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.489+0000 7fa99920e640 1 Processor -- start 2023-11-09T03:49:10.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.489+0000 7fa99920e640 1 -- start start 2023-11-09T03:49:10.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.489+0000 7fa99920e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa994146ce0 0x7fa9940b3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.489+0000 7fa99920e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa994150e20 0x7fa99415c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.489+0000 7fa99920e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa99415c940 0x7fa9940c5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.489+0000 7fa99920e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99415cdc0 con 0x7fa99415c940 2023-11-09T03:49:10.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.489+0000 7fa99920e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa994154600 con 0x7fa994150e20 2023-11-09T03:49:10.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.489+0000 7fa99920e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9941539f0 con 0x7fa994146ce0 2023-11-09T03:49:10.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.490+0000 7fa99359e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa99415c940 0x7fa9940c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.490+0000 7fa99359e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa99415c940 0x7fa9940c5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38504/0 (socket says 172.21.15.76:38504) 2023-11-09T03:49:10.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.490+0000 7fa99359e640 1 -- 172.21.15.76:0/3407353736 learned_addr learned my addr 172.21.15.76:0/3407353736 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:10.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.490+0000 7fa992d9d640 1 --2- 172.21.15.76:0/3407353736 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa994146ce0 0x7fa9940b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.490+0000 7fa99259c640 1 --2- 172.21.15.76:0/3407353736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa994150e20 0x7fa99415c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.490+0000 7fa99359e640 1 -- 172.21.15.76:0/3407353736 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa994146ce0 msgr2=0x7fa9940b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:10.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.490+0000 7fa99359e640 1 --2- 172.21.15.76:0/3407353736 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa994146ce0 0x7fa9940b3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.490+0000 7fa99359e640 1 -- 172.21.15.76:0/3407353736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa994150e20 msgr2=0x7fa99415c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:10.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.490+0000 7fa99359e640 1 --2- 172.21.15.76:0/3407353736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa994150e20 0x7fa99415c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.490+0000 7fa99359e640 1 -- 172.21.15.76:0/3407353736 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa994153cd0 con 0x7fa99415c940 2023-11-09T03:49:10.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.490+0000 7fa99359e640 1 --2- 172.21.15.76:0/3407353736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa99415c940 0x7fa9940c5990 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fa98c02c060 tx=0x7fa98c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:10.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.490+0000 7fa97b7fe640 1 -- 172.21.15.76:0/3407353736 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c03e070 con 0x7fa99415c940 2023-11-09T03:49:10.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.490+0000 7fa97b7fe640 1 -- 172.21.15.76:0/3407353736 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98c007db0 con 0x7fa99415c940 2023-11-09T03:49:10.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.490+0000 7fa99920e640 1 -- 172.21.15.76:0/3407353736 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa994153fb0 con 0x7fa99415c940 2023-11-09T03:49:10.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.490+0000 7fa97b7fe640 1 -- 172.21.15.76:0/3407353736 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c006d90 con 0x7fa99415c940 2023-11-09T03:49:10.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.490+0000 7fa99920e640 1 -- 172.21.15.76:0/3407353736 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9940b35a0 con 0x7fa99415c940 2023-11-09T03:49:10.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.491+0000 7fa99920e640 1 -- 172.21.15.76:0/3407353736 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa994074290 con 0x7fa99415c940 2023-11-09T03:49:10.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.491+0000 7fa97b7fe640 1 -- 172.21.15.76:0/3407353736 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa98c041070 con 0x7fa99415c940 2023-11-09T03:49:10.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.492+0000 7fa97b7fe640 1 --2- 172.21.15.76:0/3407353736 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa98006ae70 0x7fa98006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:10.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.492+0000 7fa97b7fe640 1 -- 172.21.15.76:0/3407353736 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7120+0+0 (secure 0 0 0) 0x7fa98c0aa9c0 con 0x7fa99415c940 2023-11-09T03:49:10.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.492+0000 7fa992d9d640 1 --2- 172.21.15.76:0/3407353736 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa98006ae70 0x7fa98006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:10.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.493+0000 7fa992d9d640 1 --2- 172.21.15.76:0/3407353736 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa98006ae70 0x7fa98006d330 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa9840099b0 tx=0x7fa984008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:10.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.496+0000 7fa97b7fe640 1 -- 172.21.15.76:0/3407353736 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa98c075460 con 0x7fa99415c940 2023-11-09T03:49:10.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.697+0000 7fa99920e640 1 -- 172.21.15.76:0/3407353736 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa99415cdc0 con 0x7fa99415c940 2023-11-09T03:49:10.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.697+0000 7fa97b7fe640 1 -- 172.21.15.76:0/3407353736 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4932 (secure 0 0 0) 0x7fa99415cdc0 con 0x7fa99415c940 2023-11-09T03:49:10.697 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:49:10.697 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5744,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3280098776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3280098776},{"type":"v1","addr":"172.21.15.113:6837","nonce":3280098776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:49:07.461923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7452,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7452":{"gid":7452,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/4011275120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4011275120},{"type":"v1","addr":"172.21.15.76:6835","nonce":4011275120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:49:10.698 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 48 2023-11-09T03:49:10.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.699+0000 7fa99920e640 1 -- 172.21.15.76:0/3407353736 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa98006ae70 msgr2=0x7fa98006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:10.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.699+0000 7fa99920e640 1 --2- 172.21.15.76:0/3407353736 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa98006ae70 0x7fa98006d330 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa9840099b0 tx=0x7fa984008040 comp rx=0 tx=0).stop 2023-11-09T03:49:10.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.699+0000 7fa99920e640 1 -- 172.21.15.76:0/3407353736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa99415c940 msgr2=0x7fa9940c5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:10.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.699+0000 7fa99920e640 1 --2- 172.21.15.76:0/3407353736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa99415c940 0x7fa9940c5990 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fa98c02c060 tx=0x7fa98c0014a0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.699+0000 7fa99920e640 1 -- 172.21.15.76:0/3407353736 shutdown_connections 2023-11-09T03:49:10.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.699+0000 7fa99920e640 1 --2- 172.21.15.76:0/3407353736 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa98006ae70 0x7fa98006d330 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.699+0000 7fa99920e640 1 --2- 172.21.15.76:0/3407353736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa99415c940 0x7fa9940c5990 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.699+0000 7fa99920e640 1 --2- 172.21.15.76:0/3407353736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa994150e20 0x7fa99415c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.699+0000 7fa99920e640 1 --2- 172.21.15.76:0/3407353736 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa994146ce0 0x7fa9940b3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:10.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.699+0000 7fa99920e640 1 -- 172.21.15.76:0/3407353736 >> 172.21.15.76:0/3407353736 conn(0x7fa9940ce950 msgr2=0x7fa9940b3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:10.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.699+0000 7fa99920e640 1 -- 172.21.15.76:0/3407353736 shutdown_connections 2023-11-09T03:49:10.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:10.699+0000 7fa99920e640 1 -- 172.21.15.76:0/3407353736 wait complete. 2023-11-09T03:49:10.709 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-11-09T03:49:10.823 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-09T03:49:10.823 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-09T03:49:10.823 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1', 1)] 2023-11-09T03:49:12.825 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:49:12.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.935+0000 7fa68fdcc640 1 Processor -- start 2023-11-09T03:49:12.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.935+0000 7fa68fdcc640 1 -- start start 2023-11-09T03:49:12.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.936+0000 7fa68fdcc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6880b0ae0 0x7fa6880ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:12.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.936+0000 7fa68fdcc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa68806db20 0x7fa68815d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:12.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.936+0000 7fa68fdcc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6880b4110 con 0x7fa688159c70 2023-11-09T03:49:12.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.936+0000 7fa68fdcc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6880b49b0 con 0x7fa68806db20 2023-11-09T03:49:12.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.936+0000 7fa68fdcc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6880b43f0 con 0x7fa6880b0ae0 2023-11-09T03:49:12.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.936+0000 7fa68e369640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa68806db20 0x7fa68815d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:12.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.936+0000 7fa68e369640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa68806db20 0x7fa68815d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34650/0 (socket says 172.21.15.76:34650) 2023-11-09T03:49:12.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.936+0000 7fa68e369640 1 -- 172.21.15.76:0/990349889 learned_addr learned my addr 172.21.15.76:0/990349889 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:12.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.936+0000 7fa68d367640 1 --2- 172.21.15.76:0/990349889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6880b0ae0 0x7fa6880ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:12.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.936+0000 7fa68e369640 1 -- 172.21.15.76:0/990349889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6880b0ae0 msgr2=0x7fa6880ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:12.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.936+0000 7fa68e369640 1 --2- 172.21.15.76:0/990349889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6880b0ae0 0x7fa6880ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:12.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.936+0000 7fa68e369640 1 -- 172.21.15.76:0/990349889 >> v1:172.21.15.113:6789/0 conn(0x7fa688159c70 legacy=0x7fa6880b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:12.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.936+0000 7fa68e369640 1 -- 172.21.15.76:0/990349889 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6880b46d0 con 0x7fa68806db20 2023-11-09T03:49:12.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.937+0000 7fa68e369640 1 --2- 172.21.15.76:0/990349889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa68806db20 0x7fa68815d7d0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fa680009e50 tx=0x7fa68002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8180dc60607735 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:12.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.937+0000 7fa68cb66640 1 -- 172.21.15.76:0/990349889 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68002cb00 con 0x7fa68806db20 2023-11-09T03:49:12.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.937+0000 7fa68cb66640 1 -- 172.21.15.76:0/990349889 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68002cc60 con 0x7fa68806db20 2023-11-09T03:49:12.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.937+0000 7fa68cb66640 1 -- 172.21.15.76:0/990349889 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6800343f0 con 0x7fa68806db20 2023-11-09T03:49:12.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.938+0000 7fa68fdcc640 1 -- 172.21.15.76:0/990349889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa68806db20 msgr2=0x7fa68815d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:12.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.938+0000 7fa68fdcc640 1 --2- 172.21.15.76:0/990349889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa68806db20 0x7fa68815d7d0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fa680009e50 tx=0x7fa68002bf60 comp rx=0 tx=0).stop 2023-11-09T03:49:12.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.938+0000 7fa68fdcc640 1 -- 172.21.15.76:0/990349889 shutdown_connections 2023-11-09T03:49:12.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.938+0000 7fa68fdcc640 1 --2- 172.21.15.76:0/990349889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa68806db20 0x7fa68815d7d0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:12.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.938+0000 7fa68fdcc640 1 --2- 172.21.15.76:0/990349889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6880b0ae0 0x7fa6880ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:12.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.938+0000 7fa68fdcc640 1 -- 172.21.15.76:0/990349889 >> 172.21.15.76:0/990349889 conn(0x7fa6880ce950 msgr2=0x7fa6880ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:12.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.938+0000 7fa68fdcc640 1 -- 172.21.15.76:0/990349889 shutdown_connections 2023-11-09T03:49:12.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.938+0000 7fa68fdcc640 1 -- 172.21.15.76:0/990349889 wait complete. 2023-11-09T03:49:12.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.939+0000 7fa68fdcc640 1 Processor -- start 2023-11-09T03:49:12.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.939+0000 7fa68fdcc640 1 -- start start 2023-11-09T03:49:12.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.940+0000 7fa68fdcc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa68806db20 0x7fa68814ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:12.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.940+0000 7fa68fdcc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6880b0ae0 0x7fa688151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:12.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.940+0000 7fa68fdcc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa688159c70 0x7fa6880c6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:12.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.940+0000 7fa68fdcc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6881526b0 con 0x7fa68806db20 2023-11-09T03:49:12.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.940+0000 7fa68fdcc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68814f170 con 0x7fa688159c70 2023-11-09T03:49:12.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.940+0000 7fa68fdcc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa68814f960 con 0x7fa6880b0ae0 2023-11-09T03:49:12.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.940+0000 7fa68db68640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa68806db20 0x7fa68814ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:12.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.940+0000 7fa68db68640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa68806db20 0x7fa68814ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34658/0 (socket says 172.21.15.76:34658) 2023-11-09T03:49:12.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.940+0000 7fa68db68640 1 -- 172.21.15.76:0/398645518 learned_addr learned my addr 172.21.15.76:0/398645518 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:12.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.940+0000 7fa68e369640 1 --2- 172.21.15.76:0/398645518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa688159c70 0x7fa6880c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:12.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.940+0000 7fa68d367640 1 --2- 172.21.15.76:0/398645518 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6880b0ae0 0x7fa688151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:12.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.940+0000 7fa68db68640 1 -- 172.21.15.76:0/398645518 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6880b0ae0 msgr2=0x7fa688151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:12.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.940+0000 7fa68db68640 1 --2- 172.21.15.76:0/398645518 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6880b0ae0 0x7fa688151dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:12.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.940+0000 7fa68db68640 1 -- 172.21.15.76:0/398645518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa688159c70 msgr2=0x7fa6880c6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:12.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.940+0000 7fa68db68640 1 --2- 172.21.15.76:0/398645518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa688159c70 0x7fa6880c6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:12.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.940+0000 7fa68db68640 1 -- 172.21.15.76:0/398645518 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa680009b30 con 0x7fa68806db20 2023-11-09T03:49:12.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.940+0000 7fa68db68640 1 --2- 172.21.15.76:0/398645518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa68806db20 0x7fa68814ec30 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fa68400c400 tx=0x7fa68400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:12.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.940+0000 7fa67e7fc640 1 -- 172.21.15.76:0/398645518 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6840052d0 con 0x7fa68806db20 2023-11-09T03:49:12.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.941+0000 7fa67e7fc640 1 -- 172.21.15.76:0/398645518 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa684005430 con 0x7fa68806db20 2023-11-09T03:49:12.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.941+0000 7fa68fdcc640 1 -- 172.21.15.76:0/398645518 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa68814fc40 con 0x7fa68806db20 2023-11-09T03:49:12.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.941+0000 7fa67e7fc640 1 -- 172.21.15.76:0/398645518 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa684003830 con 0x7fa68806db20 2023-11-09T03:49:12.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.941+0000 7fa68fdcc640 1 -- 172.21.15.76:0/398645518 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa688150200 con 0x7fa68806db20 2023-11-09T03:49:12.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.942+0000 7fa68fdcc640 1 -- 172.21.15.76:0/398645518 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa688150420 con 0x7fa68806db20 2023-11-09T03:49:12.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.942+0000 7fa67e7fc640 1 -- 172.21.15.76:0/398645518 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa684003990 con 0x7fa68806db20 2023-11-09T03:49:12.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.942+0000 7fa67e7fc640 1 --2- 172.21.15.76:0/398645518 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa66c06ae70 0x7fa66c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:12.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.942+0000 7fa67e7fc640 1 -- 172.21.15.76:0/398645518 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7120+0+0 (secure 0 0 0) 0x7fa684089080 con 0x7fa68806db20 2023-11-09T03:49:12.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.943+0000 7fa68d367640 1 --2- 172.21.15.76:0/398645518 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa66c06ae70 0x7fa66c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:12.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.943+0000 7fa68d367640 1 --2- 172.21.15.76:0/398645518 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa66c06ae70 0x7fa66c06d330 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa6881f4cc0 tx=0x7fa678008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:12.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:12.947+0000 7fa67e7fc640 1 -- 172.21.15.76:0/398645518 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa684053c40 con 0x7fa68806db20 2023-11-09T03:49:13.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:13.153+0000 7fa68fdcc640 1 -- 172.21.15.76:0/398645518 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6880ced30 con 0x7fa68806db20 2023-11-09T03:49:13.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:13.154+0000 7fa67e7fc640 1 -- 172.21.15.76:0/398645518 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4932 (secure 0 0 0) 0x7fa684057260 con 0x7fa68806db20 2023-11-09T03:49:13.154 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:49:13.154 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5744,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3280098776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3280098776},{"type":"v1","addr":"172.21.15.113:6837","nonce":3280098776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:49:07.461923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7452,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7452":{"gid":7452,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/4011275120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4011275120},{"type":"v1","addr":"172.21.15.76:6835","nonce":4011275120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:49:13.155 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 48 2023-11-09T03:49:13.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:13.156+0000 7fa68fdcc640 1 -- 172.21.15.76:0/398645518 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa66c06ae70 msgr2=0x7fa66c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:13.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:13.156+0000 7fa68fdcc640 1 --2- 172.21.15.76:0/398645518 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa66c06ae70 0x7fa66c06d330 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa6881f4cc0 tx=0x7fa678008040 comp rx=0 tx=0).stop 2023-11-09T03:49:13.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:13.156+0000 7fa68fdcc640 1 -- 172.21.15.76:0/398645518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa68806db20 msgr2=0x7fa68814ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:13.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:13.156+0000 7fa68fdcc640 1 --2- 172.21.15.76:0/398645518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa68806db20 0x7fa68814ec30 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fa68400c400 tx=0x7fa68400c8c0 comp rx=0 tx=0).stop 2023-11-09T03:49:13.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:13.156+0000 7fa68fdcc640 1 -- 172.21.15.76:0/398645518 shutdown_connections 2023-11-09T03:49:13.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:13.156+0000 7fa68fdcc640 1 --2- 172.21.15.76:0/398645518 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa66c06ae70 0x7fa66c06d330 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:13.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:13.156+0000 7fa68fdcc640 1 --2- 172.21.15.76:0/398645518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa688159c70 0x7fa6880c6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:13.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:13.156+0000 7fa68fdcc640 1 --2- 172.21.15.76:0/398645518 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6880b0ae0 0x7fa688151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:13.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:13.156+0000 7fa68fdcc640 1 --2- 172.21.15.76:0/398645518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa68806db20 0x7fa68814ec30 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:13.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:13.156+0000 7fa68fdcc640 1 -- 172.21.15.76:0/398645518 >> 172.21.15.76:0/398645518 conn(0x7fa6880ce950 msgr2=0x7fa6880ccff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:13.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:13.156+0000 7fa68fdcc640 1 -- 172.21.15.76:0/398645518 shutdown_connections 2023-11-09T03:49:13.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:13.156+0000 7fa68fdcc640 1 -- 172.21.15.76:0/398645518 wait complete. 2023-11-09T03:49:13.166 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-11-09T03:49:13.280 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-09T03:49:13.280 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-09T03:49:13.281 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1', 1)] 2023-11-09T03:49:15.282 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:49:15.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.393+0000 7fd20319e640 1 Processor -- start 2023-11-09T03:49:15.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.393+0000 7fd20319e640 1 -- start start 2023-11-09T03:49:15.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.394+0000 7fd20319e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd1fc0cf760 0x7fd1fc15f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:15.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.394+0000 7fd20319e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd1fc0cc750 0x7fd1fc0d1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:15.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.394+0000 7fd20319e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1fc0cb990 con 0x7fd1fc15af20 2023-11-09T03:49:15.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.394+0000 7fd20319e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1fc15b300 con 0x7fd1fc0cf760 2023-11-09T03:49:15.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.394+0000 7fd20319e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1fc0d18d0 con 0x7fd1fc0cc750 2023-11-09T03:49:15.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.394+0000 7fd20199b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd1fc0cf760 0x7fd1fc15f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:15.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.394+0000 7fd20199b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd1fc0cf760 0x7fd1fc15f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34662/0 (socket says 172.21.15.76:34662) 2023-11-09T03:49:15.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.394+0000 7fd20199b640 1 -- 172.21.15.76:0/3079644635 learned_addr learned my addr 172.21.15.76:0/3079644635 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:15.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.394+0000 7fd20299d640 1 --2- 172.21.15.76:0/3079644635 >> v2:172.21.15.113:3300/0 conn(0x7fd1fc0cc750 0x7fd1fc0d1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:15.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.394+0000 7fd20199b640 1 -- 172.21.15.76:0/3079644635 >> v2:172.21.15.113:3300/0 conn(0x7fd1fc0cc750 msgr2=0x7fd1fc0d1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:15.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.394+0000 7fd20199b640 1 --2- 172.21.15.76:0/3079644635 >> v2:172.21.15.113:3300/0 conn(0x7fd1fc0cc750 0x7fd1fc0d1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:15.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.394+0000 7fd20199b640 1 -- 172.21.15.76:0/3079644635 >> v1:172.21.15.113:6789/0 conn(0x7fd1fc15af20 legacy=0x7fd1fc0cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:15.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.394+0000 7fd20199b640 1 -- 172.21.15.76:0/3079644635 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1fc0cbc70 con 0x7fd1fc0cf760 2023-11-09T03:49:15.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.395+0000 7fd20199b640 1 --2- 172.21.15.76:0/3079644635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1fc0cf760 0x7fd1fc15f3f0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0b6af0 tx=0x7fd1ec02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef1e2a438e61272 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:15.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.395+0000 7fd20119a640 1 -- 172.21.15.76:0/3079644635 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec031750 con 0x7fd1fc0cf760 2023-11-09T03:49:15.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.395+0000 7fd20119a640 1 -- 172.21.15.76:0/3079644635 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1ec0318b0 con 0x7fd1fc0cf760 2023-11-09T03:49:15.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.395+0000 7fd20119a640 1 -- 172.21.15.76:0/3079644635 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec031c40 con 0x7fd1fc0cf760 2023-11-09T03:49:15.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.396+0000 7fd20319e640 1 -- 172.21.15.76:0/3079644635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1fc0cf760 msgr2=0x7fd1fc15f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:15.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.396+0000 7fd20319e640 1 --2- 172.21.15.76:0/3079644635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1fc0cf760 0x7fd1fc15f3f0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0b6af0 tx=0x7fd1ec02b7f0 comp rx=0 tx=0).stop 2023-11-09T03:49:15.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.396+0000 7fd20319e640 1 -- 172.21.15.76:0/3079644635 shutdown_connections 2023-11-09T03:49:15.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.396+0000 7fd20319e640 1 --2- 172.21.15.76:0/3079644635 >> v2:172.21.15.113:3300/0 conn(0x7fd1fc0cc750 0x7fd1fc0d1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:15.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.396+0000 7fd20319e640 1 --2- 172.21.15.76:0/3079644635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1fc0cf760 0x7fd1fc15f3f0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:15.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.396+0000 7fd20319e640 1 -- 172.21.15.76:0/3079644635 >> 172.21.15.76:0/3079644635 conn(0x7fd1fc099d80 msgr2=0x7fd1fc0d0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:15.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.396+0000 7fd20319e640 1 -- 172.21.15.76:0/3079644635 shutdown_connections 2023-11-09T03:49:15.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.396+0000 7fd20319e640 1 -- 172.21.15.76:0/3079644635 wait complete. 2023-11-09T03:49:15.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.397+0000 7fd20319e640 1 Processor -- start 2023-11-09T03:49:15.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.397+0000 7fd20319e640 1 -- start start 2023-11-09T03:49:15.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.398+0000 7fd20319e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1fc0cc750 0x7fd1fc0b5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:15.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.398+0000 7fd20319e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1fc0cf760 0x7fd1fc15dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:15.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.398+0000 7fd20319e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1fc15af20 0x7fd1fc151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:15.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.398+0000 7fd20319e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1fc15b940 con 0x7fd1fc15af20 2023-11-09T03:49:15.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.398+0000 7fd20319e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1fc0c5470 con 0x7fd1fc0cc750 2023-11-09T03:49:15.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.398+0000 7fd20319e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1fc0b5b50 con 0x7fd1fc0cf760 2023-11-09T03:49:15.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.398+0000 7fd20299d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1fc15af20 0x7fd1fc151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:15.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.398+0000 7fd20299d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1fc15af20 0x7fd1fc151ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34664/0 (socket says 172.21.15.76:34664) 2023-11-09T03:49:15.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.398+0000 7fd20299d640 1 -- 172.21.15.76:0/352552090 learned_addr learned my addr 172.21.15.76:0/352552090 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:15.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.398+0000 7fd20219c640 1 --2- 172.21.15.76:0/352552090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1fc0cc750 0x7fd1fc0b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:15.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.398+0000 7fd20199b640 1 --2- 172.21.15.76:0/352552090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1fc0cf760 0x7fd1fc15dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:15.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.398+0000 7fd20299d640 1 -- 172.21.15.76:0/352552090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1fc0cf760 msgr2=0x7fd1fc15dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:15.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.398+0000 7fd20299d640 1 --2- 172.21.15.76:0/352552090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1fc0cf760 0x7fd1fc15dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:15.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.398+0000 7fd20299d640 1 -- 172.21.15.76:0/352552090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1fc0cc750 msgr2=0x7fd1fc0b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:15.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.398+0000 7fd20299d640 1 --2- 172.21.15.76:0/352552090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1fc0cc750 0x7fd1fc0b5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:15.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.398+0000 7fd20299d640 1 -- 172.21.15.76:0/352552090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec0094d0 con 0x7fd1fc15af20 2023-11-09T03:49:15.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.398+0000 7fd20219c640 1 --2- 172.21.15.76:0/352552090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1fc0cc750 0x7fd1fc0b5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:49:15.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.399+0000 7fd20299d640 1 --2- 172.21.15.76:0/352552090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1fc15af20 0x7fd1fc151ad0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400cc30 tx=0x7fd1f400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:15.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.399+0000 7fd1f2ffd640 1 -- 172.21.15.76:0/352552090 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f400ce40 con 0x7fd1fc15af20 2023-11-09T03:49:15.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.399+0000 7fd1f2ffd640 1 -- 172.21.15.76:0/352552090 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f4003990 con 0x7fd1fc15af20 2023-11-09T03:49:15.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.399+0000 7fd20319e640 1 -- 172.21.15.76:0/352552090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1fc0b5e30 con 0x7fd1fc15af20 2023-11-09T03:49:15.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.399+0000 7fd1f2ffd640 1 -- 172.21.15.76:0/352552090 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4003d20 con 0x7fd1fc15af20 2023-11-09T03:49:15.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.399+0000 7fd20319e640 1 -- 172.21.15.76:0/352552090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1fc15e2b0 con 0x7fd1fc15af20 2023-11-09T03:49:15.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.400+0000 7fd20319e640 1 -- 172.21.15.76:0/352552090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1fc0b6070 con 0x7fd1fc15af20 2023-11-09T03:49:15.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.400+0000 7fd1f2ffd640 1 -- 172.21.15.76:0/352552090 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd1f4003e80 con 0x7fd1fc15af20 2023-11-09T03:49:15.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.400+0000 7fd1f2ffd640 1 --2- 172.21.15.76:0/352552090 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd1e006ae70 0x7fd1e006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:15.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.401+0000 7fd1f2ffd640 1 -- 172.21.15.76:0/352552090 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7178+0+0 (secure 0 0 0) 0x7fd1f4089f70 con 0x7fd1fc15af20 2023-11-09T03:49:15.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.401+0000 7fd20219c640 1 --2- 172.21.15.76:0/352552090 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd1e006ae70 0x7fd1e006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:15.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.401+0000 7fd20219c640 1 --2- 172.21.15.76:0/352552090 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd1e006ae70 0x7fd1e006d330 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0d0350 tx=0x7fd1f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:15.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.405+0000 7fd1f2ffd640 1 -- 172.21.15.76:0/352552090 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd1f40bcca0 con 0x7fd1fc15af20 2023-11-09T03:49:15.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.606+0000 7fd20319e640 1 -- 172.21.15.76:0/352552090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1fc04ece0 con 0x7fd1fc15af20 2023-11-09T03:49:15.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.606+0000 7fd1f2ffd640 1 -- 172.21.15.76:0/352552090 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4932 (secure 0 0 0) 0x7fd1f401d070 con 0x7fd1fc15af20 2023-11-09T03:49:15.606 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:49:15.607 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5744,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3280098776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3280098776},{"type":"v1","addr":"172.21.15.113:6837","nonce":3280098776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:49:07.461923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7452,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7452":{"gid":7452,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/4011275120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4011275120},{"type":"v1","addr":"172.21.15.76:6835","nonce":4011275120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:49:15.608 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 48 2023-11-09T03:49:15.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.608+0000 7fd20319e640 1 -- 172.21.15.76:0/352552090 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd1e006ae70 msgr2=0x7fd1e006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:15.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.608+0000 7fd20319e640 1 --2- 172.21.15.76:0/352552090 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd1e006ae70 0x7fd1e006d330 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0d0350 tx=0x7fd1f8008040 comp rx=0 tx=0).stop 2023-11-09T03:49:15.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.608+0000 7fd20319e640 1 -- 172.21.15.76:0/352552090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1fc15af20 msgr2=0x7fd1fc151ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:15.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.608+0000 7fd20319e640 1 --2- 172.21.15.76:0/352552090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1fc15af20 0x7fd1fc151ad0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400cc30 tx=0x7fd1f400a5c0 comp rx=0 tx=0).stop 2023-11-09T03:49:15.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.608+0000 7fd20319e640 1 -- 172.21.15.76:0/352552090 shutdown_connections 2023-11-09T03:49:15.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.608+0000 7fd20319e640 1 --2- 172.21.15.76:0/352552090 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd1e006ae70 0x7fd1e006d330 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:15.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.608+0000 7fd20319e640 1 --2- 172.21.15.76:0/352552090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1fc15af20 0x7fd1fc151ad0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:15.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.608+0000 7fd20319e640 1 --2- 172.21.15.76:0/352552090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1fc0cf760 0x7fd1fc15dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:15.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.608+0000 7fd20319e640 1 --2- 172.21.15.76:0/352552090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1fc0cc750 0x7fd1fc0b5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:15.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.608+0000 7fd20319e640 1 -- 172.21.15.76:0/352552090 >> 172.21.15.76:0/352552090 conn(0x7fd1fc099d80 msgr2=0x7fd1fc0cce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:15.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.608+0000 7fd20319e640 1 -- 172.21.15.76:0/352552090 shutdown_connections 2023-11-09T03:49:15.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:15.608+0000 7fd20319e640 1 -- 172.21.15.76:0/352552090 wait complete. 2023-11-09T03:49:15.619 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-11-09T03:49:15.731 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-09T03:49:15.732 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-09T03:49:15.732 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1', 1)] 2023-11-09T03:49:17.733 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:49:17.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.843+0000 7fd521d18640 1 Processor -- start 2023-11-09T03:49:17.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.843+0000 7fd521d18640 1 -- start start 2023-11-09T03:49:17.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.844+0000 7fd521d18640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd51c0ce240 0x7fd51c157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:17.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.844+0000 7fd521d18640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd51c15e000 con 0x7fd51c157540 2023-11-09T03:49:17.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.844+0000 7fd521d18640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd51c15e2e0 con 0x7fd51c0cde60 2023-11-09T03:49:17.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.844+0000 7fd521d18640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd51c15e5c0 con 0x7fd51c0ce240 2023-11-09T03:49:17.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.844+0000 7fd51bfff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd51c157540 0x7fd51c0d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:55938/0 (socket says 172.21.15.76:55938) 2023-11-09T03:49:17.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.844+0000 7fd51bfff640 1 -- 172.21.15.76:0/148119281 learned_addr learned my addr 172.21.15.76:0/148119281 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:49:17.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.845+0000 7fd51a7fc640 1 -- 172.21.15.76:0/148119281 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 365516242 0 0) 0x7fd51c15e000 con 0x7fd51c157540 2023-11-09T03:49:17.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.845+0000 7fd51a7fc640 1 -- 172.21.15.76:0/148119281 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd504003520 con 0x7fd51c157540 2023-11-09T03:49:17.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.845+0000 7fd51affd640 1 --2- 172.21.15.76:0/148119281 >> v2:172.21.15.113:3300/0 conn(0x7fd51c0ce240 0x7fd51c157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:17.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.845+0000 7fd51a7fc640 1 -- 172.21.15.76:0/148119281 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 506556467 0 0) 0x7fd504003520 con 0x7fd51c157540 2023-11-09T03:49:17.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.845+0000 7fd51a7fc640 1 -- 172.21.15.76:0/148119281 >> v2:172.21.15.113:3300/0 conn(0x7fd51c0ce240 msgr2=0x7fd51c157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:17.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.845+0000 7fd51a7fc640 1 --2- 172.21.15.76:0/148119281 >> v2:172.21.15.113:3300/0 conn(0x7fd51c0ce240 0x7fd51c157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:17.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.845+0000 7fd51a7fc640 1 -- 172.21.15.76:0/148119281 >> v1:172.21.15.113:6789/0 conn(0x7fd51c0cde60 legacy=0x7fd51c15d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:17.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.845+0000 7fd51a7fc640 1 -- 172.21.15.76:0/148119281 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51c0d1120 con 0x7fd51c157540 2023-11-09T03:49:17.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.845+0000 7fd51a7fc640 1 -- 172.21.15.76:0/148119281 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fd5140019f0 con 0x7fd51c157540 2023-11-09T03:49:17.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.845+0000 7fd51affd640 1 --2- 172.21.15.76:0/148119281 >> v2:172.21.15.113:3300/0 conn(0x7fd51c0ce240 0x7fd51c157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:17.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.845+0000 7fd51a7fc640 1 -- 172.21.15.76:0/148119281 >> v1:172.21.15.76:6789/0 conn(0x7fd51c157540 legacy=0x7fd51c0d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:17.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.845+0000 7fd51a7fc640 1 --2- 172.21.15.76:0/148119281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5040039c0 0x7fd504003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:17.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.845+0000 7fd51a7fc640 1 --2- 172.21.15.76:0/148119281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5040047c0 0x7fd504004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:17.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.845+0000 7fd51a7fc640 1 --2- 172.21.15.76:0/148119281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5040051d0 0x7fd50400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:17.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.845+0000 7fd51a7fc640 1 -- 172.21.15.76:0/148119281 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c15e5c0 con 0x7fd5040047c0 2023-11-09T03:49:17.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.845+0000 7fd51a7fc640 1 -- 172.21.15.76:0/148119281 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50400c120 con 0x7fd5040051d0 2023-11-09T03:49:17.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.845+0000 7fd51a7fc640 1 -- 172.21.15.76:0/148119281 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd50400c2e0 con 0x7fd5040039c0 2023-11-09T03:49:17.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.845+0000 7fd51affd640 1 --2- 172.21.15.76:0/148119281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5040047c0 0x7fd504004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:17.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.845+0000 7fd51b7fe640 1 --2- 172.21.15.76:0/148119281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5040039c0 0x7fd504003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:17.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.845+0000 7fd51bfff640 1 --2- 172.21.15.76:0/148119281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5040051d0 0x7fd50400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:17.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.845+0000 7fd51affd640 1 -- 172.21.15.76:0/148119281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5040039c0 msgr2=0x7fd504003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:17.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.845+0000 7fd51affd640 1 --2- 172.21.15.76:0/148119281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5040039c0 0x7fd504003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:17.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.845+0000 7fd51affd640 1 -- 172.21.15.76:0/148119281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5040051d0 msgr2=0x7fd50400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:17.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.845+0000 7fd51affd640 1 --2- 172.21.15.76:0/148119281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5040051d0 0x7fd50400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:17.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.846+0000 7fd51bfff640 1 -- 172.21.15.76:0/148119281 reap_dead start 2023-11-09T03:49:17.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.845+0000 7fd51affd640 1 -- 172.21.15.76:0/148119281 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd504003520 con 0x7fd5040047c0 2023-11-09T03:49:17.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.846+0000 7fd51affd640 1 --2- 172.21.15.76:0/148119281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5040047c0 0x7fd504004c40 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fd5100098a0 tx=0x7fd5040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce579df27b2a657 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:17.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.846+0000 7fd51a7fc640 1 -- 172.21.15.76:0/148119281 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd510004230 con 0x7fd5040047c0 2023-11-09T03:49:17.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.846+0000 7fd51a7fc640 1 -- 172.21.15.76:0/148119281 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd510004390 con 0x7fd5040047c0 2023-11-09T03:49:17.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.846+0000 7fd51a7fc640 1 -- 172.21.15.76:0/148119281 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd510003830 con 0x7fd5040047c0 2023-11-09T03:49:17.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.847+0000 7fd521d18640 1 -- 172.21.15.76:0/148119281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5040047c0 msgr2=0x7fd504004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:17.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.847+0000 7fd521d18640 1 --2- 172.21.15.76:0/148119281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5040047c0 0x7fd504004c40 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fd5100098a0 tx=0x7fd5040057f0 comp rx=0 tx=0).stop 2023-11-09T03:49:17.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.847+0000 7fd521d18640 1 -- 172.21.15.76:0/148119281 shutdown_connections 2023-11-09T03:49:17.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.847+0000 7fd521d18640 1 --2- 172.21.15.76:0/148119281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5040047c0 0x7fd504004c40 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:17.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.847+0000 7fd521d18640 1 -- 172.21.15.76:0/148119281 >> 172.21.15.76:0/148119281 conn(0x7fd51c071f30 msgr2=0x7fd51c0cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:17.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.847+0000 7fd521d18640 1 -- 172.21.15.76:0/148119281 shutdown_connections 2023-11-09T03:49:17.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.847+0000 7fd521d18640 1 -- 172.21.15.76:0/148119281 wait complete. 2023-11-09T03:49:17.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.848+0000 7fd521d18640 1 Processor -- start 2023-11-09T03:49:17.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.848+0000 7fd521d18640 1 -- start start 2023-11-09T03:49:17.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.849+0000 7fd521d18640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5040047c0 0x7fd51c0c8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:17.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.849+0000 7fd521d18640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd51c1f1fe0 0x7fd51c0c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:17.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.849+0000 7fd521d18640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd51c1594e0 0x7fd51c159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:17.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.849+0000 7fd521d18640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c0cad50 con 0x7fd5040047c0 2023-11-09T03:49:17.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.849+0000 7fd521d18640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c14fbc0 con 0x7fd51c1f1fe0 2023-11-09T03:49:17.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.849+0000 7fd521d18640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c0cbef0 con 0x7fd51c1594e0 2023-11-09T03:49:17.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.849+0000 7fd51b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5040047c0 0x7fd51c0c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:17.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.849+0000 7fd51b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5040047c0 0x7fd51c0c8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34674/0 (socket says 172.21.15.76:34674) 2023-11-09T03:49:17.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.849+0000 7fd51b7fe640 1 -- 172.21.15.76:0/3241991068 learned_addr learned my addr 172.21.15.76:0/3241991068 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:17.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.849+0000 7fd51bfff640 1 --2- 172.21.15.76:0/3241991068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd51c1594e0 0x7fd51c159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:17.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.849+0000 7fd51affd640 1 --2- 172.21.15.76:0/3241991068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd51c1f1fe0 0x7fd51c0c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:17.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.849+0000 7fd51b7fe640 1 -- 172.21.15.76:0/3241991068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd51c1594e0 msgr2=0x7fd51c159960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:17.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.849+0000 7fd51b7fe640 1 --2- 172.21.15.76:0/3241991068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd51c1594e0 0x7fd51c159960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:17.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.849+0000 7fd51b7fe640 1 -- 172.21.15.76:0/3241991068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd51c1f1fe0 msgr2=0x7fd51c0c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:17.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.849+0000 7fd51b7fe640 1 --2- 172.21.15.76:0/3241991068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd51c1f1fe0 0x7fd51c0c8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:17.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.849+0000 7fd51b7fe640 1 -- 172.21.15.76:0/3241991068 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51c0cca70 con 0x7fd5040047c0 2023-11-09T03:49:17.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.850+0000 7fd51b7fe640 1 --2- 172.21.15.76:0/3241991068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5040047c0 0x7fd51c0c8970 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fd50c00f5f0 tx=0x7fd50c00fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:17.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.850+0000 7fd503fff640 1 -- 172.21.15.76:0/3241991068 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50c0052e0 con 0x7fd5040047c0 2023-11-09T03:49:17.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.850+0000 7fd503fff640 1 -- 172.21.15.76:0/3241991068 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50c005440 con 0x7fd5040047c0 2023-11-09T03:49:17.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.850+0000 7fd521d18640 1 -- 172.21.15.76:0/3241991068 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd51c159ea0 con 0x7fd5040047c0 2023-11-09T03:49:17.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.850+0000 7fd503fff640 1 -- 172.21.15.76:0/3241991068 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50c003810 con 0x7fd5040047c0 2023-11-09T03:49:17.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.850+0000 7fd521d18640 1 -- 172.21.15.76:0/3241991068 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd51c0cc420 con 0x7fd5040047c0 2023-11-09T03:49:17.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.851+0000 7fd521d18640 1 -- 172.21.15.76:0/3241991068 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd51c074290 con 0x7fd5040047c0 2023-11-09T03:49:17.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.851+0000 7fd503fff640 1 -- 172.21.15.76:0/3241991068 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd50c003970 con 0x7fd5040047c0 2023-11-09T03:49:17.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.851+0000 7fd503fff640 1 --2- 172.21.15.76:0/3241991068 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd50806ae70 0x7fd50806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:17.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.852+0000 7fd503fff640 1 -- 172.21.15.76:0/3241991068 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7178+0+0 (secure 0 0 0) 0x7fd50c089ad0 con 0x7fd5040047c0 2023-11-09T03:49:17.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.852+0000 7fd51affd640 1 --2- 172.21.15.76:0/3241991068 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd50806ae70 0x7fd50806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:17.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.852+0000 7fd51affd640 1 --2- 172.21.15.76:0/3241991068 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd50806ae70 0x7fd50806d330 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd510035000 tx=0x7fd510031000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:17.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:17.856+0000 7fd503fff640 1 -- 172.21.15.76:0/3241991068 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd50c0545c0 con 0x7fd5040047c0 2023-11-09T03:49:18.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.055+0000 7fd521d18640 1 -- 172.21.15.76:0/3241991068 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd51c0d20c0 con 0x7fd5040047c0 2023-11-09T03:49:18.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.056+0000 7fd503fff640 1 -- 172.21.15.76:0/3241991068 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4933 (secure 0 0 0) 0x7fd50c087e60 con 0x7fd5040047c0 2023-11-09T03:49:18.057 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:49:18.057 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5744,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3280098776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3280098776},{"type":"v1","addr":"172.21.15.113:6837","nonce":3280098776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:49:17.001700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7452,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7452":{"gid":7452,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/4011275120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4011275120},{"type":"v1","addr":"172.21.15.76:6835","nonce":4011275120}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:49:18.057 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 49 2023-11-09T03:49:18.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.058+0000 7fd521d18640 1 -- 172.21.15.76:0/3241991068 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd50806ae70 msgr2=0x7fd50806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:18.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.058+0000 7fd521d18640 1 --2- 172.21.15.76:0/3241991068 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd50806ae70 0x7fd50806d330 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd510035000 tx=0x7fd510031000 comp rx=0 tx=0).stop 2023-11-09T03:49:18.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.058+0000 7fd521d18640 1 -- 172.21.15.76:0/3241991068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5040047c0 msgr2=0x7fd51c0c8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:18.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.058+0000 7fd521d18640 1 --2- 172.21.15.76:0/3241991068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5040047c0 0x7fd51c0c8970 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fd50c00f5f0 tx=0x7fd50c00fab0 comp rx=0 tx=0).stop 2023-11-09T03:49:18.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.058+0000 7fd521d18640 1 -- 172.21.15.76:0/3241991068 shutdown_connections 2023-11-09T03:49:18.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.058+0000 7fd521d18640 1 --2- 172.21.15.76:0/3241991068 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd50806ae70 0x7fd50806d330 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:18.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.058+0000 7fd521d18640 1 --2- 172.21.15.76:0/3241991068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd51c1594e0 0x7fd51c159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:18.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.058+0000 7fd521d18640 1 --2- 172.21.15.76:0/3241991068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd51c1f1fe0 0x7fd51c0c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:18.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.058+0000 7fd521d18640 1 --2- 172.21.15.76:0/3241991068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5040047c0 0x7fd51c0c8970 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:18.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.058+0000 7fd521d18640 1 -- 172.21.15.76:0/3241991068 >> 172.21.15.76:0/3241991068 conn(0x7fd51c071f30 msgr2=0x7fd51c0cd130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:18.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.058+0000 7fd521d18640 1 -- 172.21.15.76:0/3241991068 shutdown_connections 2023-11-09T03:49:18.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.058+0000 7fd521d18640 1 -- 172.21.15.76:0/3241991068 wait complete. 2023-11-09T03:49:18.069 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2023-11-09T03:49:18.183 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"8","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"3","version":"3","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2023-11-09T03:49:18.183 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "8" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "3", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "3" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-09T03:49:18.184 INFO:tasks.cephfs.cephfs_test_case:[('/d1', 1)] =?= [('/d1', 1)] 2023-11-09T03:49:18.184 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2023-11-09T03:49:18.298 INFO:teuthology.orchestra.run.smithi076.stdout:{"last_snap":1,"last_created":1,"last_destroyed":1,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-09T03:49:18.298 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 1, "last_destroyed": 1, "last_snap": 1, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [] } 2023-11-09T03:49:18.298 INFO:tasks.cephfs.test_snapshots:testing snapserver mds_kill_mdstable_at=1 2023-11-09T03:49:18.299 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:49:18.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.398+0000 7f3185e34640 1 Processor -- start 2023-11-09T03:49:18.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.398+0000 7f3185e34640 1 -- start start 2023-11-09T03:49:18.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.399+0000 7f3185e34640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3180159c70 0x7f31800ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:18.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.399+0000 7f3185e34640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f31800b0ae0 0x7f318015d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:18.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.399+0000 7f3185e34640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31800b4110 con 0x7f318006db20 2023-11-09T03:49:18.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.399+0000 7f3185e34640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31800b49b0 con 0x7f31800b0ae0 2023-11-09T03:49:18.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.399+0000 7f3185e34640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31800b43f0 con 0x7f3180159c70 2023-11-09T03:49:18.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.399+0000 7f317ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f31800b0ae0 0x7f318015d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:18.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.399+0000 7f317effd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f318006db20 0x7f31800b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:55948/0 (socket says 172.21.15.76:55948) 2023-11-09T03:49:18.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.399+0000 7f317ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f31800b0ae0 0x7f318015d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34678/0 (socket says 172.21.15.76:34678) 2023-11-09T03:49:18.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.399+0000 7f317effd640 1 -- 172.21.15.76:0/2479243322 learned_addr learned my addr 172.21.15.76:0/2479243322 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:49:18.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.400+0000 7f317f7fe640 1 --2- 172.21.15.76:0/2479243322 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3180159c70 0x7f31800ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:18.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.400+0000 7f317ffff640 1 -- 172.21.15.76:0/2479243322 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3180159c70 msgr2=0x7f31800ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:18.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.400+0000 7f317ffff640 1 --2- 172.21.15.76:0/2479243322 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3180159c70 0x7f31800ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:18.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.400+0000 7f317ffff640 1 -- 172.21.15.76:0/2479243322 >> v1:172.21.15.76:6789/0 conn(0x7f318006db20 legacy=0x7f31800b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:18.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.400+0000 7f317ffff640 1 -- 172.21.15.76:0/2479243322 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31800b46d0 con 0x7f31800b0ae0 2023-11-09T03:49:18.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.400+0000 7f317ffff640 1 --2- 172.21.15.76:0/2479243322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31800b0ae0 0x7f318015d7d0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f3178009d90 tx=0x7f317802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9288e2246778897 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:18.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.401+0000 7f317e7fc640 1 -- 172.21.15.76:0/2479243322 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317802cc50 con 0x7f31800b0ae0 2023-11-09T03:49:18.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.401+0000 7f317e7fc640 1 -- 172.21.15.76:0/2479243322 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317802cdb0 con 0x7f31800b0ae0 2023-11-09T03:49:18.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.401+0000 7f317e7fc640 1 -- 172.21.15.76:0/2479243322 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3178034660 con 0x7f31800b0ae0 2023-11-09T03:49:18.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.401+0000 7f3185e34640 1 -- 172.21.15.76:0/2479243322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31800b0ae0 msgr2=0x7f318015d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:18.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.401+0000 7f3185e34640 1 --2- 172.21.15.76:0/2479243322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31800b0ae0 0x7f318015d7d0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f3178009d90 tx=0x7f317802bea0 comp rx=0 tx=0).stop 2023-11-09T03:49:18.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.401+0000 7f3185e34640 1 -- 172.21.15.76:0/2479243322 shutdown_connections 2023-11-09T03:49:18.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.401+0000 7f3185e34640 1 --2- 172.21.15.76:0/2479243322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31800b0ae0 0x7f318015d7d0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:18.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.401+0000 7f3185e34640 1 --2- 172.21.15.76:0/2479243322 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3180159c70 0x7f31800ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:18.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.401+0000 7f3185e34640 1 -- 172.21.15.76:0/2479243322 >> 172.21.15.76:0/2479243322 conn(0x7f31800ce950 msgr2=0x7f31800ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:18.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.401+0000 7f3185e34640 1 -- 172.21.15.76:0/2479243322 shutdown_connections 2023-11-09T03:49:18.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.401+0000 7f3185e34640 1 -- 172.21.15.76:0/2479243322 wait complete. 2023-11-09T03:49:18.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.402+0000 7f3185e34640 1 Processor -- start 2023-11-09T03:49:18.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.403+0000 7f3185e34640 1 -- start start 2023-11-09T03:49:18.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.403+0000 7f3185e34640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f318006db20 0x7f318014ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:18.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.403+0000 7f3185e34640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31800b0ae0 0x7f31801f18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:18.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.403+0000 7f3185e34640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3180159c70 0x7f3180070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:18.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.403+0000 7f3185e34640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318015ca40 con 0x7f318006db20 2023-11-09T03:49:18.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.403+0000 7f3185e34640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318014b360 con 0x7f31800b0ae0 2023-11-09T03:49:18.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.403+0000 7f3185e34640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31800c8040 con 0x7f3180159c70 2023-11-09T03:49:18.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.403+0000 7f317f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f318006db20 0x7f318014ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:18.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.403+0000 7f317f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f318006db20 0x7f318014ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34692/0 (socket says 172.21.15.76:34692) 2023-11-09T03:49:18.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.403+0000 7f317f7fe640 1 -- 172.21.15.76:0/2211928261 learned_addr learned my addr 172.21.15.76:0/2211928261 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:18.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.404+0000 7f317f7fe640 1 -- 172.21.15.76:0/2211928261 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3180159c70 msgr2=0x7f3180070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:18.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.404+0000 7f317f7fe640 1 --2- 172.21.15.76:0/2211928261 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3180159c70 0x7f3180070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:18.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.404+0000 7f317f7fe640 1 -- 172.21.15.76:0/2211928261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31800b0ae0 msgr2=0x7f31801f18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:18.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.404+0000 7f317f7fe640 1 --2- 172.21.15.76:0/2211928261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31800b0ae0 0x7f31801f18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:18.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.404+0000 7f317f7fe640 1 -- 172.21.15.76:0/2211928261 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3178009a70 con 0x7f318006db20 2023-11-09T03:49:18.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.404+0000 7f317f7fe640 1 --2- 172.21.15.76:0/2211928261 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f318006db20 0x7f318014ae20 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f317000d620 tx=0x7f317000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:18.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.404+0000 7f3167fff640 1 -- 172.21.15.76:0/2211928261 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317000b620 con 0x7f318006db20 2023-11-09T03:49:18.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.404+0000 7f3167fff640 1 -- 172.21.15.76:0/2211928261 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317000b780 con 0x7f318006db20 2023-11-09T03:49:18.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.404+0000 7f3167fff640 1 -- 172.21.15.76:0/2211928261 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317000bb30 con 0x7f318006db20 2023-11-09T03:49:18.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.404+0000 7f3185e34640 1 -- 172.21.15.76:0/2211928261 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31800c7a80 con 0x7f318006db20 2023-11-09T03:49:18.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.404+0000 7f3185e34640 1 -- 172.21.15.76:0/2211928261 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31800c5aa0 con 0x7f318006db20 2023-11-09T03:49:18.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.405+0000 7f3185e34640 1 -- 172.21.15.76:0/2211928261 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3180074290 con 0x7f318006db20 2023-11-09T03:49:18.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.405+0000 7f3167fff640 1 -- 172.21.15.76:0/2211928261 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f317000bc90 con 0x7f318006db20 2023-11-09T03:49:18.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.406+0000 7f3167fff640 1 --2- 172.21.15.76:0/2211928261 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f316c06aea0 0x7f316c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:18.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.406+0000 7f3167fff640 1 -- 172.21.15.76:0/2211928261 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7178+0+0 (secure 0 0 0) 0x7f317008a060 con 0x7f318006db20 2023-11-09T03:49:18.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.406+0000 7f317effd640 1 --2- 172.21.15.76:0/2211928261 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f316c06aea0 0x7f316c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:18.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.407+0000 7f317effd640 1 --2- 172.21.15.76:0/2211928261 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f316c06aea0 0x7f316c06d360 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f318015d960 tx=0x7f3174007a40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:18.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.410+0000 7f3167fff640 1 -- 172.21.15.76:0/2211928261 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3170054b50 con 0x7f318006db20 2023-11-09T03:49:18.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.613+0000 7f3185e34640 1 -- 172.21.15.76:0/2211928261 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f318006d3f0 con 0x7f318006db20 2023-11-09T03:49:18.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.614+0000 7f3167fff640 1 -- 172.21.15.76:0/2211928261 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4933 (secure 0 0 0) 0x7f3170058170 con 0x7f318006db20 2023-11-09T03:49:18.614 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:49:18.615 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5744,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3280098776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3280098776},{"type":"v1","addr":"172.21.15.113:6837","nonce":3280098776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:49:17.001700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7452,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7452":{"gid":7452,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/4011275120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4011275120},{"type":"v1","addr":"172.21.15.76:6835","nonce":4011275120}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:49:18.615 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 49 2023-11-09T03:49:18.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.616+0000 7f3185e34640 1 -- 172.21.15.76:0/2211928261 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f316c06aea0 msgr2=0x7f316c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:18.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.616+0000 7f3185e34640 1 --2- 172.21.15.76:0/2211928261 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f316c06aea0 0x7f316c06d360 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f318015d960 tx=0x7f3174007a40 comp rx=0 tx=0).stop 2023-11-09T03:49:18.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.616+0000 7f3185e34640 1 -- 172.21.15.76:0/2211928261 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f318006db20 msgr2=0x7f318014ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:18.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.616+0000 7f3185e34640 1 --2- 172.21.15.76:0/2211928261 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f318006db20 0x7f318014ae20 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f317000d620 tx=0x7f317000dae0 comp rx=0 tx=0).stop 2023-11-09T03:49:18.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.616+0000 7f3185e34640 1 -- 172.21.15.76:0/2211928261 shutdown_connections 2023-11-09T03:49:18.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.616+0000 7f3185e34640 1 --2- 172.21.15.76:0/2211928261 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f316c06aea0 0x7f316c06d360 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:18.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.616+0000 7f3185e34640 1 --2- 172.21.15.76:0/2211928261 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3180159c70 0x7f3180070710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:18.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.616+0000 7f3185e34640 1 --2- 172.21.15.76:0/2211928261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31800b0ae0 0x7f31801f18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:18.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.616+0000 7f3185e34640 1 --2- 172.21.15.76:0/2211928261 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f318006db20 0x7f318014ae20 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:18.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.616+0000 7f3185e34640 1 -- 172.21.15.76:0/2211928261 >> 172.21.15.76:0/2211928261 conn(0x7f31800ce950 msgr2=0x7f31800ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:18.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.616+0000 7f3185e34640 1 -- 172.21.15.76:0/2211928261 shutdown_connections 2023-11-09T03:49:18.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.616+0000 7f3185e34640 1 -- 172.21.15.76:0/2211928261 wait complete. 2023-11-09T03:49:18.627 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:0 true 2023-11-09T03:49:18.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.728+0000 7fa1f514c640 1 Processor -- start 2023-11-09T03:49:18.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.728+0000 7fa1f514c640 1 -- start start 2023-11-09T03:49:18.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.729+0000 7fa1f514c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1f00b4880 0x7fa1f00b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:18.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.729+0000 7fa1f514c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f00cde60 con 0x7fa1f0153e60 2023-11-09T03:49:18.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.729+0000 7fa1f514c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f0160f90 con 0x7fa1f00cd6d0 2023-11-09T03:49:18.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.729+0000 7fa1f514c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f00b5200 con 0x7fa1f00b4880 2023-11-09T03:49:18.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.729+0000 7fa1eed9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa1f0153e60 0x7fa1f015dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:55954/0 (socket says 172.21.15.76:55954) 2023-11-09T03:49:18.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.729+0000 7fa1eed9d640 1 -- 172.21.15.76:0/4242456822 learned_addr learned my addr 172.21.15.76:0/4242456822 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:49:18.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.729+0000 7fa1edd9b640 1 -- 172.21.15.76:0/4242456822 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1493459060 0 0) 0x7fa1f00cde60 con 0x7fa1f0153e60 2023-11-09T03:49:18.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.729+0000 7fa1edd9b640 1 -- 172.21.15.76:0/4242456822 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1d8003520 con 0x7fa1f0153e60 2023-11-09T03:49:18.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.729+0000 7fa1ef59e640 1 --2- 172.21.15.76:0/4242456822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1f00b4880 0x7fa1f00b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:18.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.730+0000 7fa1edd9b640 1 -- 172.21.15.76:0/4242456822 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2390623701 0 0) 0x7fa1d8003520 con 0x7fa1f0153e60 2023-11-09T03:49:18.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.730+0000 7fa1edd9b640 1 -- 172.21.15.76:0/4242456822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1f00b4880 msgr2=0x7fa1f00b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:18.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.730+0000 7fa1edd9b640 1 --2- 172.21.15.76:0/4242456822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1f00b4880 0x7fa1f00b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:18.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.730+0000 7fa1edd9b640 1 -- 172.21.15.76:0/4242456822 >> v1:172.21.15.113:6789/0 conn(0x7fa1f00cd6d0 legacy=0x7fa1f0161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:18.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.730+0000 7fa1edd9b640 1 -- 172.21.15.76:0/4242456822 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f00ce090 con 0x7fa1f0153e60 2023-11-09T03:49:18.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.730+0000 7fa1edd9b640 1 -- 172.21.15.76:0/4242456822 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fa1e00019f0 con 0x7fa1f0153e60 2023-11-09T03:49:18.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.730+0000 7fa1edd9b640 1 -- 172.21.15.76:0/4242456822 >> v1:172.21.15.76:6789/0 conn(0x7fa1f0153e60 legacy=0x7fa1f015dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:18.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.730+0000 7fa1edd9b640 1 --2- 172.21.15.76:0/4242456822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1f00b4880 0x7fa1d80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:18.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.730+0000 7fa1edd9b640 1 --2- 172.21.15.76:0/4242456822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d8003fd0 0x7fa1d80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:18.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.730+0000 7fa1edd9b640 1 --2- 172.21.15.76:0/4242456822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d8004d00 0x7fa1d8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:18.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.730+0000 7fa1edd9b640 1 -- 172.21.15.76:0/4242456822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f00b5200 con 0x7fa1d8003fd0 2023-11-09T03:49:18.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.730+0000 7fa1edd9b640 1 -- 172.21.15.76:0/4242456822 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d800bb40 con 0x7fa1d8004d00 2023-11-09T03:49:18.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.730+0000 7fa1edd9b640 1 -- 172.21.15.76:0/4242456822 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d800bd00 con 0x7fa1f00b4880 2023-11-09T03:49:18.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.730+0000 7fa1ee59c640 1 --2- 172.21.15.76:0/4242456822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d8003fd0 0x7fa1d80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:18.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.730+0000 7fa1eed9d640 1 --2- 172.21.15.76:0/4242456822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1f00b4880 0x7fa1d80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:18.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.730+0000 7fa1ef59e640 1 --2- 172.21.15.76:0/4242456822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d8004d00 0x7fa1d8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:18.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.730+0000 7fa1ee59c640 1 -- 172.21.15.76:0/4242456822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1f00b4880 msgr2=0x7fa1d80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:18.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.730+0000 7fa1ee59c640 1 --2- 172.21.15.76:0/4242456822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1f00b4880 0x7fa1d80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:18.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.730+0000 7fa1ee59c640 1 -- 172.21.15.76:0/4242456822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d8004d00 msgr2=0x7fa1d8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:18.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.730+0000 7fa1ee59c640 1 --2- 172.21.15.76:0/4242456822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d8004d00 0x7fa1d8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:18.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.730+0000 7fa1ee59c640 1 -- 172.21.15.76:0/4242456822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d8003520 con 0x7fa1d8003fd0 2023-11-09T03:49:18.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.731+0000 7fa1ee59c640 1 --2- 172.21.15.76:0/4242456822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d8003fd0 0x7fa1d80047c0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4001720 tx=0x7fa1d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad010dbd8c72967 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:18.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.731+0000 7fa1edd9b640 1 -- 172.21.15.76:0/4242456822 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e40052d0 con 0x7fa1d8003fd0 2023-11-09T03:49:18.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.731+0000 7fa1edd9b640 1 -- 172.21.15.76:0/4242456822 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e402e050 con 0x7fa1d8003fd0 2023-11-09T03:49:18.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.731+0000 7fa1edd9b640 1 -- 172.21.15.76:0/4242456822 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4003830 con 0x7fa1d8003fd0 2023-11-09T03:49:18.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.731+0000 7fa1f514c640 1 -- 172.21.15.76:0/4242456822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d8003fd0 msgr2=0x7fa1d80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:18.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.731+0000 7fa1f514c640 1 --2- 172.21.15.76:0/4242456822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d8003fd0 0x7fa1d80047c0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4001720 tx=0x7fa1d80057f0 comp rx=0 tx=0).stop 2023-11-09T03:49:18.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.731+0000 7fa1ef59e640 1 -- 172.21.15.76:0/4242456822 reap_dead start 2023-11-09T03:49:18.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.731+0000 7fa1f514c640 1 -- 172.21.15.76:0/4242456822 shutdown_connections 2023-11-09T03:49:18.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.731+0000 7fa1f514c640 1 -- 172.21.15.76:0/4242456822 >> 172.21.15.76:0/4242456822 conn(0x7fa1f00cf880 msgr2=0x7fa1f00cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:18.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.732+0000 7fa1f514c640 1 -- 172.21.15.76:0/4242456822 shutdown_connections 2023-11-09T03:49:18.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.732+0000 7fa1f514c640 1 -- 172.21.15.76:0/4242456822 wait complete. 2023-11-09T03:49:18.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.733+0000 7fa1f514c640 1 Processor -- start 2023-11-09T03:49:18.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.733+0000 7fa1f514c640 1 -- start start 2023-11-09T03:49:18.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.734+0000 7fa1f514c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d8003fd0 0x7fa1f01f2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:18.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.734+0000 7fa1f514c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1f0153630 0x7fa1f015ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:18.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.734+0000 7fa1f514c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1f0158040 0x7fa1f015f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:18.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.734+0000 7fa1f514c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f01593d0 con 0x7fa1f0158040 2023-11-09T03:49:18.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.734+0000 7fa1f514c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f01f2ad0 con 0x7fa1d8003fd0 2023-11-09T03:49:18.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.734+0000 7fa1f514c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f01f2db0 con 0x7fa1f0153630 2023-11-09T03:49:18.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.734+0000 7fa1ef59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1f0158040 0x7fa1f015f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:18.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.734+0000 7fa1ef59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1f0158040 0x7fa1f015f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34716/0 (socket says 172.21.15.76:34716) 2023-11-09T03:49:18.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.734+0000 7fa1ef59e640 1 -- 172.21.15.76:0/1709701026 learned_addr learned my addr 172.21.15.76:0/1709701026 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:18.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.734+0000 7fa1ee59c640 1 --2- 172.21.15.76:0/1709701026 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1f0153630 0x7fa1f015ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:18.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.734+0000 7fa1eed9d640 1 --2- 172.21.15.76:0/1709701026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d8003fd0 0x7fa1f01f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:18.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.734+0000 7fa1ef59e640 1 -- 172.21.15.76:0/1709701026 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1f0153630 msgr2=0x7fa1f015ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:18.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.734+0000 7fa1ef59e640 1 --2- 172.21.15.76:0/1709701026 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1f0153630 0x7fa1f015ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:18.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.734+0000 7fa1ef59e640 1 -- 172.21.15.76:0/1709701026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d8003fd0 msgr2=0x7fa1f01f2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:18.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.734+0000 7fa1ef59e640 1 --2- 172.21.15.76:0/1709701026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d8003fd0 0x7fa1f01f2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:18.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.734+0000 7fa1ef59e640 1 -- 172.21.15.76:0/1709701026 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f01f3090 con 0x7fa1f0158040 2023-11-09T03:49:18.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.734+0000 7fa1ef59e640 1 --2- 172.21.15.76:0/1709701026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1f0158040 0x7fa1f015f2b0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00c9e50 tx=0x7fa1e800bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:18.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.734+0000 7fa1d77fe640 1 -- 172.21.15.76:0/1709701026 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8010780 con 0x7fa1f0158040 2023-11-09T03:49:18.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.734+0000 7fa1d77fe640 1 -- 172.21.15.76:0/1709701026 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e8010920 con 0x7fa1f0158040 2023-11-09T03:49:18.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.734+0000 7fa1f514c640 1 -- 172.21.15.76:0/1709701026 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f00c96b0 con 0x7fa1f0158040 2023-11-09T03:49:18.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.734+0000 7fa1d77fe640 1 -- 172.21.15.76:0/1709701026 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8010cd0 con 0x7fa1f0158040 2023-11-09T03:49:18.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.734+0000 7fa1f514c640 1 -- 172.21.15.76:0/1709701026 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f01555b0 con 0x7fa1f0158040 2023-11-09T03:49:18.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.735+0000 7fa1f514c640 1 -- 172.21.15.76:0/1709701026 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f0074290 con 0x7fa1f0158040 2023-11-09T03:49:18.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.736+0000 7fa1d77fe640 1 -- 172.21.15.76:0/1709701026 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa1e8020460 con 0x7fa1f0158040 2023-11-09T03:49:18.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.736+0000 7fa1d77fe640 1 --2- 172.21.15.76:0/1709701026 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa1dc06ae70 0x7fa1dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:18.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.736+0000 7fa1d77fe640 1 -- 172.21.15.76:0/1709701026 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7178+0+0 (secure 0 0 0) 0x7fa1e8089d40 con 0x7fa1f0158040 2023-11-09T03:49:18.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.737+0000 7fa1eed9d640 1 --2- 172.21.15.76:0/1709701026 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa1dc06ae70 0x7fa1dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:18.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.737+0000 7fa1eed9d640 1 --2- 172.21.15.76:0/1709701026 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa1dc06ae70 0x7fa1dc06d330 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00013d0 tx=0x7fa1e0006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:18.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.741+0000 7fa1d77fe640 1 -- 172.21.15.76:0/1709701026 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa1e8054830 con 0x7fa1f0158040 2023-11-09T03:49:18.971 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:18.970+0000 7fa1f514c640 1 -- 172.21.15.76:0/1709701026 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"} v 0) v1 -- 0x7fa1f0157e20 con 0x7fa1f0158040 2023-11-09T03:49:19.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.493+0000 7fa1d77fe640 1 -- 172.21.15.76:0/1709701026 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"}]=0 freezing mds.7452 v50) v1 ==== 112+0+0 (secure 0 0 0) 0x7fa1e8057e50 con 0x7fa1f0158040 2023-11-09T03:49:19.494 INFO:teuthology.orchestra.run.smithi076.stderr:freezing mds.7452 2023-11-09T03:49:19.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.496+0000 7fa1f514c640 1 -- 172.21.15.76:0/1709701026 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa1dc06ae70 msgr2=0x7fa1dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:19.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.496+0000 7fa1f514c640 1 --2- 172.21.15.76:0/1709701026 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa1dc06ae70 0x7fa1dc06d330 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00013d0 tx=0x7fa1e0006040 comp rx=0 tx=0).stop 2023-11-09T03:49:19.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.496+0000 7fa1f514c640 1 -- 172.21.15.76:0/1709701026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1f0158040 msgr2=0x7fa1f015f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:19.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.496+0000 7fa1f514c640 1 --2- 172.21.15.76:0/1709701026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1f0158040 0x7fa1f015f2b0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00c9e50 tx=0x7fa1e800bb80 comp rx=0 tx=0).stop 2023-11-09T03:49:19.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.496+0000 7fa1f514c640 1 -- 172.21.15.76:0/1709701026 shutdown_connections 2023-11-09T03:49:19.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.496+0000 7fa1f514c640 1 --2- 172.21.15.76:0/1709701026 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa1dc06ae70 0x7fa1dc06d330 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:19.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.496+0000 7fa1f514c640 1 --2- 172.21.15.76:0/1709701026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1f0158040 0x7fa1f015f2b0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:19.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.496+0000 7fa1f514c640 1 --2- 172.21.15.76:0/1709701026 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1f0153630 0x7fa1f015ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:19.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.496+0000 7fa1f514c640 1 --2- 172.21.15.76:0/1709701026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d8003fd0 0x7fa1f01f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:19.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.496+0000 7fa1f514c640 1 -- 172.21.15.76:0/1709701026 >> 172.21.15.76:0/1709701026 conn(0x7fa1f00cf880 msgr2=0x7fa1f00b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:19.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.496+0000 7fa1f514c640 1 -- 172.21.15.76:0/1709701026 shutdown_connections 2023-11-09T03:49:19.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.496+0000 7fa1f514c640 1 -- 172.21.15.76:0/1709701026 wait complete. 2023-11-09T03:49:19.509 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_kill_mdstable_at 1 2023-11-09T03:49:19.623 INFO:teuthology.orchestra.run.smithi076.stdout:{"success":"mds_kill_mdstable_at = '1' (not observed, change may require restart) "} 2023-11-09T03:49:19.623 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '1' (not observed, change may require restart) " } 2023-11-09T03:49:19.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:49:19.624 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s11) 2023-11-09T03:49:19.626 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2023-11-09T03:49:19.642 INFO:tasks.ceph.mds.c.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_prepare(ceph::cref_t&)' thread 7f65dc881640 time 2023-11-09T03:49:19.643719+0000 2023-11-09T03:49:19.642 INFO:tasks.ceph.mds.c.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 61: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 1) 2023-11-09T03:49:19.644 INFO:tasks.ceph.mds.c.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:49:19.644 INFO:tasks.ceph.mds.c.smithi076.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f65e236efaf] 2023-11-09T03:49:19.644 INFO:tasks.ceph.mds.c.smithi076.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f65e236f16d] 2023-11-09T03:49:19.644 INFO:tasks.ceph.mds.c.smithi076.stderr: 3: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x354) [0x557b47fa8254] 2023-11-09T03:49:19.644 INFO:tasks.ceph.mds.c.smithi076.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x557b47d3b0a9] 2023-11-09T03:49:19.645 INFO:tasks.ceph.mds.c.smithi076.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x557b47d3db2b] 2023-11-09T03:49:19.645 INFO:tasks.ceph.mds.c.smithi076.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557b47d3e01c] 2023-11-09T03:49:19.645 INFO:tasks.ceph.mds.c.smithi076.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x557b47d27325] 2023-11-09T03:49:19.645 INFO:tasks.ceph.mds.c.smithi076.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7f65e255ff9a] 2023-11-09T03:49:19.645 INFO:tasks.ceph.mds.c.smithi076.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f65e25f1741] 2023-11-09T03:49:19.646 INFO:tasks.ceph.mds.c.smithi076.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7f65e1a9f802] 2023-11-09T03:49:19.646 INFO:tasks.ceph.mds.c.smithi076.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7f65e1a3f450] 2023-11-09T03:49:19.646 INFO:tasks.ceph.mds.c.smithi076.stderr:2023-11-09T03:49:19.642+0000 7f65dc881640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_prepare(ceph::cref_t&)' thread 7f65dc881640 time 2023-11-09T03:49:19.643719+0000 2023-11-09T03:49:19.646 INFO:tasks.ceph.mds.c.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 61: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 1) 2023-11-09T03:49:19.646 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:49:19.647 INFO:tasks.ceph.mds.c.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:49:19.647 INFO:tasks.ceph.mds.c.smithi076.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f65e236efaf] 2023-11-09T03:49:19.647 INFO:tasks.ceph.mds.c.smithi076.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f65e236f16d] 2023-11-09T03:49:19.647 INFO:tasks.ceph.mds.c.smithi076.stderr: 3: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x354) [0x557b47fa8254] 2023-11-09T03:49:19.647 INFO:tasks.ceph.mds.c.smithi076.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x557b47d3b0a9] 2023-11-09T03:49:19.648 INFO:tasks.ceph.mds.c.smithi076.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x557b47d3db2b] 2023-11-09T03:49:19.648 INFO:tasks.ceph.mds.c.smithi076.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557b47d3e01c] 2023-11-09T03:49:19.648 INFO:tasks.ceph.mds.c.smithi076.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x557b47d27325] 2023-11-09T03:49:19.648 INFO:tasks.ceph.mds.c.smithi076.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7f65e255ff9a] 2023-11-09T03:49:19.648 INFO:tasks.ceph.mds.c.smithi076.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f65e25f1741] 2023-11-09T03:49:19.649 INFO:tasks.ceph.mds.c.smithi076.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7f65e1a9f802] 2023-11-09T03:49:19.649 INFO:tasks.ceph.mds.c.smithi076.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7f65e1a3f450] 2023-11-09T03:49:19.649 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:49:19.649 INFO:tasks.ceph.mds.c.smithi076.stderr:*** Caught signal (Aborted) ** 2023-11-09T03:49:19.649 INFO:tasks.ceph.mds.c.smithi076.stderr: in thread 7f65dc881640 thread_name:ms_dispatch 2023-11-09T03:49:19.649 INFO:tasks.ceph.mds.c.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:49:19.650 INFO:tasks.ceph.mds.c.smithi076.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f65e1a54db0] 2023-11-09T03:49:19.650 INFO:tasks.ceph.mds.c.smithi076.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f65e1aa154c] 2023-11-09T03:49:19.650 INFO:tasks.ceph.mds.c.smithi076.stderr: 3: raise() 2023-11-09T03:49:19.650 INFO:tasks.ceph.mds.c.smithi076.stderr: 4: abort() 2023-11-09T03:49:19.650 INFO:tasks.ceph.mds.c.smithi076.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f65e236f009] 2023-11-09T03:49:19.651 INFO:tasks.ceph.mds.c.smithi076.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f65e236f16d] 2023-11-09T03:49:19.651 INFO:tasks.ceph.mds.c.smithi076.stderr: 7: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x354) [0x557b47fa8254] 2023-11-09T03:49:19.651 INFO:tasks.ceph.mds.c.smithi076.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x557b47d3b0a9] 2023-11-09T03:49:19.651 INFO:tasks.ceph.mds.c.smithi076.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x557b47d3db2b] 2023-11-09T03:49:19.651 INFO:tasks.ceph.mds.c.smithi076.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557b47d3e01c] 2023-11-09T03:49:19.652 INFO:tasks.ceph.mds.c.smithi076.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x557b47d27325] 2023-11-09T03:49:19.652 INFO:tasks.ceph.mds.c.smithi076.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7f65e255ff9a] 2023-11-09T03:49:19.652 INFO:tasks.ceph.mds.c.smithi076.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f65e25f1741] 2023-11-09T03:49:19.652 INFO:tasks.ceph.mds.c.smithi076.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7f65e1a9f802] 2023-11-09T03:49:19.652 INFO:tasks.ceph.mds.c.smithi076.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7f65e1a3f450] 2023-11-09T03:49:19.652 INFO:tasks.ceph.mds.c.smithi076.stderr:2023-11-09T03:49:19.643+0000 7f65dc881640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:49:19.653 INFO:tasks.ceph.mds.c.smithi076.stderr: in thread 7f65dc881640 thread_name:ms_dispatch 2023-11-09T03:49:19.653 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:49:19.653 INFO:tasks.ceph.mds.c.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:49:19.653 INFO:tasks.ceph.mds.c.smithi076.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f65e1a54db0] 2023-11-09T03:49:19.653 INFO:tasks.ceph.mds.c.smithi076.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f65e1aa154c] 2023-11-09T03:49:19.653 INFO:tasks.ceph.mds.c.smithi076.stderr: 3: raise() 2023-11-09T03:49:19.654 INFO:tasks.ceph.mds.c.smithi076.stderr: 4: abort() 2023-11-09T03:49:19.654 INFO:tasks.ceph.mds.c.smithi076.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f65e236f009] 2023-11-09T03:49:19.654 INFO:tasks.ceph.mds.c.smithi076.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f65e236f16d] 2023-11-09T03:49:19.654 INFO:tasks.ceph.mds.c.smithi076.stderr: 7: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x354) [0x557b47fa8254] 2023-11-09T03:49:19.654 INFO:tasks.ceph.mds.c.smithi076.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x557b47d3b0a9] 2023-11-09T03:49:19.655 INFO:tasks.ceph.mds.c.smithi076.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x557b47d3db2b] 2023-11-09T03:49:19.655 INFO:tasks.ceph.mds.c.smithi076.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557b47d3e01c] 2023-11-09T03:49:19.655 INFO:tasks.ceph.mds.c.smithi076.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x557b47d27325] 2023-11-09T03:49:19.655 INFO:tasks.ceph.mds.c.smithi076.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7f65e255ff9a] 2023-11-09T03:49:19.655 INFO:tasks.ceph.mds.c.smithi076.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f65e25f1741] 2023-11-09T03:49:19.655 INFO:tasks.ceph.mds.c.smithi076.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7f65e1a9f802] 2023-11-09T03:49:19.656 INFO:tasks.ceph.mds.c.smithi076.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7f65e1a3f450] 2023-11-09T03:49:19.656 INFO:tasks.ceph.mds.c.smithi076.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:49:19.656 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:49:19.657 INFO:tasks.ceph.mds.c.smithi076.stderr: -2398> 2023-11-09T03:49:04.022+0000 7f65da07c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:49:19.657 INFO:tasks.ceph.mds.c.smithi076.stderr: -2038> 2023-11-09T03:49:09.022+0000 7f65da07c640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-09T03:49:19.657 INFO:tasks.ceph.mds.c.smithi076.stderr: -1> 2023-11-09T03:49:19.642+0000 7f65dc881640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_prepare(ceph::cref_t&)' thread 7f65dc881640 time 2023-11-09T03:49:19.643719+0000 2023-11-09T03:49:19.658 INFO:tasks.ceph.mds.c.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 61: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 1) 2023-11-09T03:49:19.658 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:49:19.658 INFO:tasks.ceph.mds.c.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:49:19.658 INFO:tasks.ceph.mds.c.smithi076.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f65e236efaf] 2023-11-09T03:49:19.658 INFO:tasks.ceph.mds.c.smithi076.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f65e236f16d] 2023-11-09T03:49:19.659 INFO:tasks.ceph.mds.c.smithi076.stderr: 3: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x354) [0x557b47fa8254] 2023-11-09T03:49:19.659 INFO:tasks.ceph.mds.c.smithi076.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x557b47d3b0a9] 2023-11-09T03:49:19.659 INFO:tasks.ceph.mds.c.smithi076.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x557b47d3db2b] 2023-11-09T03:49:19.659 INFO:tasks.ceph.mds.c.smithi076.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557b47d3e01c] 2023-11-09T03:49:19.659 INFO:tasks.ceph.mds.c.smithi076.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x557b47d27325] 2023-11-09T03:49:19.659 INFO:tasks.ceph.mds.c.smithi076.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7f65e255ff9a] 2023-11-09T03:49:19.660 INFO:tasks.ceph.mds.c.smithi076.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f65e25f1741] 2023-11-09T03:49:19.660 INFO:tasks.ceph.mds.c.smithi076.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7f65e1a9f802] 2023-11-09T03:49:19.660 INFO:tasks.ceph.mds.c.smithi076.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7f65e1a3f450] 2023-11-09T03:49:19.660 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:49:19.660 INFO:tasks.ceph.mds.c.smithi076.stderr: 0> 2023-11-09T03:49:19.643+0000 7f65dc881640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:49:19.661 INFO:tasks.ceph.mds.c.smithi076.stderr: in thread 7f65dc881640 thread_name:ms_dispatch 2023-11-09T03:49:19.661 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:49:19.661 INFO:tasks.ceph.mds.c.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:49:19.661 INFO:tasks.ceph.mds.c.smithi076.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f65e1a54db0] 2023-11-09T03:49:19.661 INFO:tasks.ceph.mds.c.smithi076.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f65e1aa154c] 2023-11-09T03:49:19.661 INFO:tasks.ceph.mds.c.smithi076.stderr: 3: raise() 2023-11-09T03:49:19.662 INFO:tasks.ceph.mds.c.smithi076.stderr: 4: abort() 2023-11-09T03:49:19.662 INFO:tasks.ceph.mds.c.smithi076.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f65e236f009] 2023-11-09T03:49:19.662 INFO:tasks.ceph.mds.c.smithi076.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f65e236f16d] 2023-11-09T03:49:19.662 INFO:tasks.ceph.mds.c.smithi076.stderr: 7: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x354) [0x557b47fa8254] 2023-11-09T03:49:19.662 INFO:tasks.ceph.mds.c.smithi076.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x557b47d3b0a9] 2023-11-09T03:49:19.663 INFO:tasks.ceph.mds.c.smithi076.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x557b47d3db2b] 2023-11-09T03:49:19.663 INFO:tasks.ceph.mds.c.smithi076.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557b47d3e01c] 2023-11-09T03:49:19.663 INFO:tasks.ceph.mds.c.smithi076.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x557b47d27325] 2023-11-09T03:49:19.663 INFO:tasks.ceph.mds.c.smithi076.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7f65e255ff9a] 2023-11-09T03:49:19.663 INFO:tasks.ceph.mds.c.smithi076.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f65e25f1741] 2023-11-09T03:49:19.663 INFO:tasks.ceph.mds.c.smithi076.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7f65e1a9f802] 2023-11-09T03:49:19.664 INFO:tasks.ceph.mds.c.smithi076.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7f65e1a3f450] 2023-11-09T03:49:19.664 INFO:tasks.ceph.mds.c.smithi076.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:49:19.664 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:49:19.664 INFO:tasks.ceph.mds.c.smithi076.stderr: -2398> 2023-11-09T03:49:04.022+0000 7f65da07c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:49:19.664 INFO:tasks.ceph.mds.c.smithi076.stderr: -2038> 2023-11-09T03:49:09.022+0000 7f65da07c640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-09T03:49:19.664 INFO:tasks.ceph.mds.c.smithi076.stderr: -1> 2023-11-09T03:49:19.642+0000 7f65dc881640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_prepare(ceph::cref_t&)' thread 7f65dc881640 time 2023-11-09T03:49:19.643719+0000 2023-11-09T03:49:19.665 INFO:tasks.ceph.mds.c.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 61: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 1) 2023-11-09T03:49:19.665 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:49:19.665 INFO:tasks.ceph.mds.c.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:49:19.665 INFO:tasks.ceph.mds.c.smithi076.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f65e236efaf] 2023-11-09T03:49:19.665 INFO:tasks.ceph.mds.c.smithi076.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f65e236f16d] 2023-11-09T03:49:19.666 INFO:tasks.ceph.mds.c.smithi076.stderr: 3: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x354) [0x557b47fa8254] 2023-11-09T03:49:19.666 INFO:tasks.ceph.mds.c.smithi076.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x557b47d3b0a9] 2023-11-09T03:49:19.666 INFO:tasks.ceph.mds.c.smithi076.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x557b47d3db2b] 2023-11-09T03:49:19.666 INFO:tasks.ceph.mds.c.smithi076.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557b47d3e01c] 2023-11-09T03:49:19.666 INFO:tasks.ceph.mds.c.smithi076.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x557b47d27325] 2023-11-09T03:49:19.666 INFO:tasks.ceph.mds.c.smithi076.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7f65e255ff9a] 2023-11-09T03:49:19.667 INFO:tasks.ceph.mds.c.smithi076.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f65e25f1741] 2023-11-09T03:49:19.667 INFO:tasks.ceph.mds.c.smithi076.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7f65e1a9f802] 2023-11-09T03:49:19.667 INFO:tasks.ceph.mds.c.smithi076.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7f65e1a3f450] 2023-11-09T03:49:19.667 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:49:19.667 INFO:tasks.ceph.mds.c.smithi076.stderr: 0> 2023-11-09T03:49:19.643+0000 7f65dc881640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:49:19.668 INFO:tasks.ceph.mds.c.smithi076.stderr: in thread 7f65dc881640 thread_name:ms_dispatch 2023-11-09T03:49:19.668 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:49:19.668 INFO:tasks.ceph.mds.c.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:49:19.668 INFO:tasks.ceph.mds.c.smithi076.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f65e1a54db0] 2023-11-09T03:49:19.668 INFO:tasks.ceph.mds.c.smithi076.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f65e1aa154c] 2023-11-09T03:49:19.668 INFO:tasks.ceph.mds.c.smithi076.stderr: 3: raise() 2023-11-09T03:49:19.669 INFO:tasks.ceph.mds.c.smithi076.stderr: 4: abort() 2023-11-09T03:49:19.669 INFO:tasks.ceph.mds.c.smithi076.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f65e236f009] 2023-11-09T03:49:19.669 INFO:tasks.ceph.mds.c.smithi076.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f65e236f16d] 2023-11-09T03:49:19.669 INFO:tasks.ceph.mds.c.smithi076.stderr: 7: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x354) [0x557b47fa8254] 2023-11-09T03:49:19.669 INFO:tasks.ceph.mds.c.smithi076.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x557b47d3b0a9] 2023-11-09T03:49:19.670 INFO:tasks.ceph.mds.c.smithi076.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x557b47d3db2b] 2023-11-09T03:49:19.670 INFO:tasks.ceph.mds.c.smithi076.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557b47d3e01c] 2023-11-09T03:49:19.670 INFO:tasks.ceph.mds.c.smithi076.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x557b47d27325] 2023-11-09T03:49:19.670 INFO:tasks.ceph.mds.c.smithi076.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7f65e255ff9a] 2023-11-09T03:49:19.670 INFO:tasks.ceph.mds.c.smithi076.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f65e25f1741] 2023-11-09T03:49:19.670 INFO:tasks.ceph.mds.c.smithi076.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7f65e1a9f802] 2023-11-09T03:49:19.671 INFO:tasks.ceph.mds.c.smithi076.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7f65e1a3f450] 2023-11-09T03:49:19.671 INFO:tasks.ceph.mds.c.smithi076.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:49:19.671 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:49:19.749 INFO:teuthology.orchestra.run.smithi076.stdout:{"mds_beacon_grace":"15.000000"} 2023-11-09T03:49:19.749 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2023-11-09T03:49:19.750 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:49:19.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.888+0000 7fd637ad7640 1 Processor -- start 2023-11-09T03:49:19.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.888+0000 7fd637ad7640 1 -- start start 2023-11-09T03:49:19.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.889+0000 7fd637ad7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd630159c70 0x7fd6300ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:19.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.889+0000 7fd637ad7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd63006db20 0x7fd6300b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:19.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.889+0000 7fd637ad7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6300b4110 con 0x7fd6300b0910 2023-11-09T03:49:19.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.889+0000 7fd637ad7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6300b49b0 con 0x7fd630159c70 2023-11-09T03:49:19.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.889+0000 7fd637ad7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6300b0da0 con 0x7fd63006db20 2023-11-09T03:49:19.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.889+0000 7fd636074640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd6300b0910 0x7fd63015d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:55960/0 (socket says 172.21.15.76:55960) 2023-11-09T03:49:19.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.889+0000 7fd636074640 1 -- 172.21.15.76:0/50421992 learned_addr learned my addr 172.21.15.76:0/50421992 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:49:19.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.889+0000 7fd634871640 1 -- 172.21.15.76:0/50421992 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2151576264 0 0) 0x7fd6300b4110 con 0x7fd6300b0910 2023-11-09T03:49:19.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.889+0000 7fd635072640 1 --2- 172.21.15.76:0/50421992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd63006db20 0x7fd6300b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:19.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.889+0000 7fd634871640 1 -- 172.21.15.76:0/50421992 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd61c003520 con 0x7fd6300b0910 2023-11-09T03:49:19.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.890+0000 7fd635873640 1 --2- 172.21.15.76:0/50421992 >> v2:172.21.15.113:3300/0 conn(0x7fd630159c70 0x7fd6300ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:19.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.890+0000 7fd634871640 1 -- 172.21.15.76:0/50421992 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3303756640 0 0) 0x7fd61c003520 con 0x7fd6300b0910 2023-11-09T03:49:19.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.890+0000 7fd634871640 1 -- 172.21.15.76:0/50421992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd63006db20 msgr2=0x7fd6300b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:19.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.890+0000 7fd634871640 1 --2- 172.21.15.76:0/50421992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd63006db20 0x7fd6300b03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:19.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.890+0000 7fd634871640 1 -- 172.21.15.76:0/50421992 >> v2:172.21.15.113:3300/0 conn(0x7fd630159c70 msgr2=0x7fd6300ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:19.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.890+0000 7fd634871640 1 --2- 172.21.15.76:0/50421992 >> v2:172.21.15.113:3300/0 conn(0x7fd630159c70 0x7fd6300ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:19.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.890+0000 7fd634871640 1 -- 172.21.15.76:0/50421992 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6300b43f0 con 0x7fd6300b0910 2023-11-09T03:49:19.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.890+0000 7fd634871640 1 -- 172.21.15.76:0/50421992 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fd6280019f0 con 0x7fd6300b0910 2023-11-09T03:49:19.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.890+0000 7fd635873640 1 --2- 172.21.15.76:0/50421992 >> v2:172.21.15.113:3300/0 conn(0x7fd630159c70 0x7fd6300ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:49:19.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.890+0000 7fd634871640 1 -- 172.21.15.76:0/50421992 >> v1:172.21.15.76:6789/0 conn(0x7fd6300b0910 legacy=0x7fd63015d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:19.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.890+0000 7fd634871640 1 --2- 172.21.15.76:0/50421992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd63006db20 0x7fd61c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:19.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.890+0000 7fd634871640 1 --2- 172.21.15.76:0/50421992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd61c003ef0 0x7fd61c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:19.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.890+0000 7fd634871640 1 --2- 172.21.15.76:0/50421992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd61c004d00 0x7fd61c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:19.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.890+0000 7fd634871640 1 -- 172.21.15.76:0/50421992 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6300b49b0 con 0x7fd61c004d00 2023-11-09T03:49:19.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.890+0000 7fd634871640 1 -- 172.21.15.76:0/50421992 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6300b0da0 con 0x7fd61c003ef0 2023-11-09T03:49:19.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.890+0000 7fd634871640 1 -- 172.21.15.76:0/50421992 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c00bab0 con 0x7fd63006db20 2023-11-09T03:49:19.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.890+0000 7fd636074640 1 --2- 172.21.15.76:0/50421992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd61c004d00 0x7fd61c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:19.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.890+0000 7fd635072640 1 --2- 172.21.15.76:0/50421992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd61c003ef0 0x7fd61c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:19.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.890+0000 7fd635873640 1 --2- 172.21.15.76:0/50421992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd63006db20 0x7fd61c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:19.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.890+0000 7fd636074640 1 -- 172.21.15.76:0/50421992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd63006db20 msgr2=0x7fd61c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:19.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.890+0000 7fd636074640 1 --2- 172.21.15.76:0/50421992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd63006db20 0x7fd61c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:19.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.890+0000 7fd636074640 1 -- 172.21.15.76:0/50421992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd61c003ef0 msgr2=0x7fd61c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:19.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.890+0000 7fd636074640 1 --2- 172.21.15.76:0/50421992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd61c003ef0 0x7fd61c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:19.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.890+0000 7fd636074640 1 -- 172.21.15.76:0/50421992 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61c003520 con 0x7fd61c004d00 2023-11-09T03:49:19.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.890+0000 7fd635873640 1 --2- 172.21.15.76:0/50421992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd63006db20 0x7fd61c003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:19.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.891+0000 7fd636074640 1 --2- 172.21.15.76:0/50421992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd61c004d00 0x7fd61c005180 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fd62800a8d0 tx=0x7fd61c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59942da2b830fc9a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:19.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.891+0000 7fd634871640 1 -- 172.21.15.76:0/50421992 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd628034630 con 0x7fd61c004d00 2023-11-09T03:49:19.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.891+0000 7fd634871640 1 -- 172.21.15.76:0/50421992 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62802f070 con 0x7fd61c004d00 2023-11-09T03:49:19.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.891+0000 7fd634871640 1 -- 172.21.15.76:0/50421992 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6280349d0 con 0x7fd61c004d00 2023-11-09T03:49:19.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.891+0000 7fd637ad7640 1 -- 172.21.15.76:0/50421992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd61c004d00 msgr2=0x7fd61c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:19.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.891+0000 7fd637ad7640 1 --2- 172.21.15.76:0/50421992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd61c004d00 0x7fd61c005180 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fd62800a8d0 tx=0x7fd61c0057f0 comp rx=0 tx=0).stop 2023-11-09T03:49:19.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.891+0000 7fd636074640 1 -- 172.21.15.76:0/50421992 reap_dead start 2023-11-09T03:49:19.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.892+0000 7fd637ad7640 1 -- 172.21.15.76:0/50421992 shutdown_connections 2023-11-09T03:49:19.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.892+0000 7fd637ad7640 1 -- 172.21.15.76:0/50421992 >> 172.21.15.76:0/50421992 conn(0x7fd6300ce950 msgr2=0x7fd6300ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:19.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.892+0000 7fd637ad7640 1 -- 172.21.15.76:0/50421992 shutdown_connections 2023-11-09T03:49:19.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.892+0000 7fd637ad7640 1 -- 172.21.15.76:0/50421992 wait complete. 2023-11-09T03:49:19.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.893+0000 7fd637ad7640 1 Processor -- start 2023-11-09T03:49:19.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.893+0000 7fd637ad7640 1 -- start start 2023-11-09T03:49:19.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.893+0000 7fd637ad7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd61c004d00 0x7fd630153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:19.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.893+0000 7fd637ad7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd630154310 0x7fd6301f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:19.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.893+0000 7fd637ad7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6301f3370 0x7fd6301e3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:19.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.893+0000 7fd637ad7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd630153580 con 0x7fd61c004d00 2023-11-09T03:49:19.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.893+0000 7fd637ad7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd630154790 con 0x7fd6301f3370 2023-11-09T03:49:19.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.893+0000 7fd637ad7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6301549e0 con 0x7fd630154310 2023-11-09T03:49:19.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.894+0000 7fd635873640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd61c004d00 0x7fd630153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:19.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.894+0000 7fd635873640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd61c004d00 0x7fd630153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34734/0 (socket says 172.21.15.76:34734) 2023-11-09T03:49:19.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.894+0000 7fd635873640 1 -- 172.21.15.76:0/1573780774 learned_addr learned my addr 172.21.15.76:0/1573780774 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:19.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.894+0000 7fd636074640 1 --2- 172.21.15.76:0/1573780774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6301f3370 0x7fd6301e3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:19.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.894+0000 7fd635072640 1 --2- 172.21.15.76:0/1573780774 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd630154310 0x7fd6301f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:19.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.894+0000 7fd635873640 1 -- 172.21.15.76:0/1573780774 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd630154310 msgr2=0x7fd6301f2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:19.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.894+0000 7fd635873640 1 --2- 172.21.15.76:0/1573780774 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd630154310 0x7fd6301f2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:19.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.894+0000 7fd635873640 1 -- 172.21.15.76:0/1573780774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6301f3370 msgr2=0x7fd6301e3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:19.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.894+0000 7fd635873640 1 --2- 172.21.15.76:0/1573780774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6301f3370 0x7fd6301e3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:19.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.894+0000 7fd635873640 1 -- 172.21.15.76:0/1573780774 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd630154cc0 con 0x7fd61c004d00 2023-11-09T03:49:19.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.894+0000 7fd635873640 1 --2- 172.21.15.76:0/1573780774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd61c004d00 0x7fd630153dd0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00dfd0 tx=0x7fd62c00f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:19.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.894+0000 7fd6267fc640 1 -- 172.21.15.76:0/1573780774 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c014890 con 0x7fd61c004d00 2023-11-09T03:49:19.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.894+0000 7fd6267fc640 1 -- 172.21.15.76:0/1573780774 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62c00c030 con 0x7fd61c004d00 2023-11-09T03:49:19.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.894+0000 7fd637ad7640 1 -- 172.21.15.76:0/1573780774 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd630154fa0 con 0x7fd61c004d00 2023-11-09T03:49:19.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.894+0000 7fd6267fc640 1 -- 172.21.15.76:0/1573780774 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c014bf0 con 0x7fd61c004d00 2023-11-09T03:49:19.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.894+0000 7fd637ad7640 1 -- 172.21.15.76:0/1573780774 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd630155560 con 0x7fd61c004d00 2023-11-09T03:49:19.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.895+0000 7fd637ad7640 1 -- 172.21.15.76:0/1573780774 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd630074290 con 0x7fd61c004d00 2023-11-09T03:49:19.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.896+0000 7fd6267fc640 1 -- 172.21.15.76:0/1573780774 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd62c014d50 con 0x7fd61c004d00 2023-11-09T03:49:19.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.896+0000 7fd6267fc640 1 --2- 172.21.15.76:0/1573780774 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd61406aea0 0x7fd61406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:19.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.896+0000 7fd6267fc640 1 -- 172.21.15.76:0/1573780774 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7178+0+0 (secure 0 0 0) 0x7fd62c089d00 con 0x7fd61c004d00 2023-11-09T03:49:19.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.896+0000 7fd635072640 1 --2- 172.21.15.76:0/1573780774 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd61406aea0 0x7fd61406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:19.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.897+0000 7fd635072640 1 --2- 172.21.15.76:0/1573780774 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd61406aea0 0x7fd61406d360 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd6301539e0 tx=0x7fd620009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:19.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:19.900+0000 7fd6267fc640 1 -- 172.21.15.76:0/1573780774 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd62c054930 con 0x7fd61c004d00 2023-11-09T03:49:20.011 INFO:tasks.ceph.mds.c.smithi076.stderr:daemon-helper: command crashed with signal 6 2023-11-09T03:49:20.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:20.103+0000 7fd637ad7640 1 -- 172.21.15.76:0/1573780774 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd630159030 con 0x7fd61c004d00 2023-11-09T03:49:20.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:20.103+0000 7fd6267fc640 1 -- 172.21.15.76:0/1573780774 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4933 (secure 0 0 0) 0x7fd62c057f50 con 0x7fd61c004d00 2023-11-09T03:49:20.104 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:49:20.104 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5744,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3280098776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3280098776},{"type":"v1","addr":"172.21.15.113:6837","nonce":3280098776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:49:18.972774+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7452,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7452":{"gid":7452,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/4011275120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4011275120},{"type":"v1","addr":"172.21.15.76:6835","nonce":4011275120}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:49:20.105 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 50 2023-11-09T03:49:20.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:20.105+0000 7fd637ad7640 1 -- 172.21.15.76:0/1573780774 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd61406aea0 msgr2=0x7fd61406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:20.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:20.105+0000 7fd637ad7640 1 --2- 172.21.15.76:0/1573780774 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd61406aea0 0x7fd61406d360 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd6301539e0 tx=0x7fd620009190 comp rx=0 tx=0).stop 2023-11-09T03:49:20.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:20.105+0000 7fd637ad7640 1 -- 172.21.15.76:0/1573780774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd61c004d00 msgr2=0x7fd630153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:20.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:20.105+0000 7fd637ad7640 1 --2- 172.21.15.76:0/1573780774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd61c004d00 0x7fd630153dd0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00dfd0 tx=0x7fd62c00f920 comp rx=0 tx=0).stop 2023-11-09T03:49:20.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:20.105+0000 7fd637ad7640 1 -- 172.21.15.76:0/1573780774 shutdown_connections 2023-11-09T03:49:20.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:20.105+0000 7fd637ad7640 1 --2- 172.21.15.76:0/1573780774 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd61406aea0 0x7fd61406d360 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:20.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:20.105+0000 7fd637ad7640 1 --2- 172.21.15.76:0/1573780774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6301f3370 0x7fd6301e3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:20.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:20.105+0000 7fd637ad7640 1 --2- 172.21.15.76:0/1573780774 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd630154310 0x7fd6301f2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:20.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:20.105+0000 7fd637ad7640 1 --2- 172.21.15.76:0/1573780774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd61c004d00 0x7fd630153dd0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:20.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:20.105+0000 7fd637ad7640 1 -- 172.21.15.76:0/1573780774 >> 172.21.15.76:0/1573780774 conn(0x7fd6300ce950 msgr2=0x7fd6300cc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:20.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:20.105+0000 7fd637ad7640 1 -- 172.21.15.76:0/1573780774 shutdown_connections 2023-11-09T03:49:20.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:20.105+0000 7fd637ad7640 1 -- 172.21.15.76:0/1573780774 wait complete. 2023-11-09T03:49:20.117 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:49:25.118 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:49:25.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.230+0000 7f53f035e640 1 Processor -- start 2023-11-09T03:49:25.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.230+0000 7f53f035e640 1 -- start start 2023-11-09T03:49:25.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.231+0000 7f53f035e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f53e8159c70 0x7f53e80ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:25.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.231+0000 7f53f035e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53e80b0ae0 0x7f53e815d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:25.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.231+0000 7f53f035e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e80b4110 con 0x7f53e806db20 2023-11-09T03:49:25.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.231+0000 7f53f035e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e80b49b0 con 0x7f53e8159c70 2023-11-09T03:49:25.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.231+0000 7f53f035e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e80b43f0 con 0x7f53e80b0ae0 2023-11-09T03:49:25.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.231+0000 7f53ee0fa640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f53e8159c70 0x7f53e80ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:25.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.231+0000 7f53ee0fa640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f53e8159c70 0x7f53e80ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47798/0 (socket says 172.21.15.76:47798) 2023-11-09T03:49:25.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.231+0000 7f53ee0fa640 1 -- 172.21.15.76:0/1052654464 learned_addr learned my addr 172.21.15.76:0/1052654464 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:25.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.231+0000 7f53ee8fb640 1 --2- 172.21.15.76:0/1052654464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53e80b0ae0 0x7f53e815d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:25.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.231+0000 7f53ee0fa640 1 -- 172.21.15.76:0/1052654464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53e80b0ae0 msgr2=0x7f53e815d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:25.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.231+0000 7f53ee0fa640 1 --2- 172.21.15.76:0/1052654464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53e80b0ae0 0x7f53e815d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:25.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.231+0000 7f53ee0fa640 1 -- 172.21.15.76:0/1052654464 >> v1:172.21.15.113:6789/0 conn(0x7f53e806db20 legacy=0x7f53e80b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:25.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.231+0000 7f53ee0fa640 1 -- 172.21.15.76:0/1052654464 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e80b46d0 con 0x7f53e8159c70 2023-11-09T03:49:25.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.231+0000 7f53ee8fb640 1 --2- 172.21.15.76:0/1052654464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53e80b0ae0 0x7f53e815d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:25.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.232+0000 7f53ee0fa640 1 --2- 172.21.15.76:0/1052654464 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53e8159c70 0x7f53e80ce370 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009870 tx=0x7f53e402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfacba95cb6b8329 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:25.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.232+0000 7f53ed0f8640 1 -- 172.21.15.76:0/1052654464 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e4030840 con 0x7f53e8159c70 2023-11-09T03:49:25.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.232+0000 7f53ed0f8640 1 -- 172.21.15.76:0/1052654464 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e40309a0 con 0x7f53e8159c70 2023-11-09T03:49:25.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.232+0000 7f53ed0f8640 1 -- 172.21.15.76:0/1052654464 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e4030d30 con 0x7f53e8159c70 2023-11-09T03:49:25.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.233+0000 7f53f035e640 1 -- 172.21.15.76:0/1052654464 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53e8159c70 msgr2=0x7f53e80ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:25.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.233+0000 7f53f035e640 1 --2- 172.21.15.76:0/1052654464 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53e8159c70 0x7f53e80ce370 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009870 tx=0x7f53e402b980 comp rx=0 tx=0).stop 2023-11-09T03:49:25.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.233+0000 7f53f035e640 1 -- 172.21.15.76:0/1052654464 shutdown_connections 2023-11-09T03:49:25.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.233+0000 7f53f035e640 1 --2- 172.21.15.76:0/1052654464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53e80b0ae0 0x7f53e815d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:25.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.233+0000 7f53f035e640 1 --2- 172.21.15.76:0/1052654464 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53e8159c70 0x7f53e80ce370 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:25.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.233+0000 7f53f035e640 1 -- 172.21.15.76:0/1052654464 >> 172.21.15.76:0/1052654464 conn(0x7f53e80ce950 msgr2=0x7f53e80ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:25.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.233+0000 7f53f035e640 1 -- 172.21.15.76:0/1052654464 shutdown_connections 2023-11-09T03:49:25.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.233+0000 7f53f035e640 1 -- 172.21.15.76:0/1052654464 wait complete. 2023-11-09T03:49:25.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.234+0000 7f53f035e640 1 Processor -- start 2023-11-09T03:49:25.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.234+0000 7f53f035e640 1 -- start start 2023-11-09T03:49:25.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.235+0000 7f53f035e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53e806db20 0x7f53e80c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:25.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.235+0000 7f53f035e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53e80b0ae0 0x7f53e814e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:25.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.235+0000 7f53f035e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53e8159c70 0x7f53e814e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:25.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.235+0000 7f53f035e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e8149e00 con 0x7f53e8159c70 2023-11-09T03:49:25.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.235+0000 7f53f035e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e80c6ab0 con 0x7f53e80b0ae0 2023-11-09T03:49:25.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.235+0000 7f53f035e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e814b520 con 0x7f53e806db20 2023-11-09T03:49:25.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.235+0000 7f53ee8fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53e8159c70 0x7f53e814e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:25.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.235+0000 7f53ee8fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53e8159c70 0x7f53e814e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47806/0 (socket says 172.21.15.76:47806) 2023-11-09T03:49:25.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.235+0000 7f53ee8fb640 1 -- 172.21.15.76:0/2515253915 learned_addr learned my addr 172.21.15.76:0/2515253915 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:25.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.235+0000 7f53ed8f9640 1 --2- 172.21.15.76:0/2515253915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53e80b0ae0 0x7f53e814e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:25.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.235+0000 7f53ee0fa640 1 --2- 172.21.15.76:0/2515253915 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53e806db20 0x7f53e80c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:25.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.235+0000 7f53ee8fb640 1 -- 172.21.15.76:0/2515253915 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53e806db20 msgr2=0x7f53e80c4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:25.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.235+0000 7f53ee8fb640 1 --2- 172.21.15.76:0/2515253915 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53e806db20 0x7f53e80c4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:25.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.235+0000 7f53ee8fb640 1 -- 172.21.15.76:0/2515253915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53e80b0ae0 msgr2=0x7f53e814e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:25.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.235+0000 7f53ee8fb640 1 --2- 172.21.15.76:0/2515253915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53e80b0ae0 0x7f53e814e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:25.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.235+0000 7f53ee8fb640 1 -- 172.21.15.76:0/2515253915 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e40094d0 con 0x7f53e8159c70 2023-11-09T03:49:25.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.235+0000 7f53ed8f9640 1 --2- 172.21.15.76:0/2515253915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53e80b0ae0 0x7f53e814e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:25.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.236+0000 7f53ee8fb640 1 --2- 172.21.15.76:0/2515253915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53e8159c70 0x7f53e814e700 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f53e0007ef0 tx=0x7f53e000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:25.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.236+0000 7f53deffd640 1 -- 172.21.15.76:0/2515253915 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0003b90 con 0x7f53e8159c70 2023-11-09T03:49:25.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.236+0000 7f53deffd640 1 -- 172.21.15.76:0/2515253915 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e000f040 con 0x7f53e8159c70 2023-11-09T03:49:25.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.236+0000 7f53f035e640 1 -- 172.21.15.76:0/2515253915 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e80c5f30 con 0x7f53e8159c70 2023-11-09T03:49:25.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.236+0000 7f53deffd640 1 -- 172.21.15.76:0/2515253915 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e000de80 con 0x7f53e8159c70 2023-11-09T03:49:25.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.236+0000 7f53f035e640 1 -- 172.21.15.76:0/2515253915 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e80c64f0 con 0x7f53e8159c70 2023-11-09T03:49:25.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.237+0000 7f53f035e640 1 -- 172.21.15.76:0/2515253915 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e8149f40 con 0x7f53e8159c70 2023-11-09T03:49:25.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.237+0000 7f53deffd640 1 -- 172.21.15.76:0/2515253915 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f53e0004030 con 0x7f53e8159c70 2023-11-09T03:49:25.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.238+0000 7f53deffd640 1 --2- 172.21.15.76:0/2515253915 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f53cc06aea0 0x7f53cc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:25.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.238+0000 7f53deffd640 1 -- 172.21.15.76:0/2515253915 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7178+0+0 (secure 0 0 0) 0x7f53e0089d10 con 0x7f53e8159c70 2023-11-09T03:49:25.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.238+0000 7f53ee0fa640 1 --2- 172.21.15.76:0/2515253915 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f53cc06aea0 0x7f53cc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:25.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.238+0000 7f53ee0fa640 1 --2- 172.21.15.76:0/2515253915 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f53cc06aea0 0x7f53cc06d360 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f53e4036000 tx=0x7f53e403b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:25.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.242+0000 7f53deffd640 1 -- 172.21.15.76:0/2515253915 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f53e0054940 con 0x7f53e8159c70 2023-11-09T03:49:25.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.441+0000 7f53f035e640 1 -- 172.21.15.76:0/2515253915 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53e8074290 con 0x7f53e8159c70 2023-11-09T03:49:25.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.442+0000 7f53deffd640 1 -- 172.21.15.76:0/2515253915 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4933 (secure 0 0 0) 0x7f53e001a090 con 0x7f53e8159c70 2023-11-09T03:49:25.443 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:49:25.443 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5744,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3280098776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3280098776},{"type":"v1","addr":"172.21.15.113:6837","nonce":3280098776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:49:18.972774+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7452,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7452":{"gid":7452,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/4011275120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4011275120},{"type":"v1","addr":"172.21.15.76:6835","nonce":4011275120}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:49:25.443 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 50 2023-11-09T03:49:25.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.444+0000 7f53f035e640 1 -- 172.21.15.76:0/2515253915 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f53cc06aea0 msgr2=0x7f53cc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:25.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.444+0000 7f53f035e640 1 --2- 172.21.15.76:0/2515253915 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f53cc06aea0 0x7f53cc06d360 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f53e4036000 tx=0x7f53e403b040 comp rx=0 tx=0).stop 2023-11-09T03:49:25.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.444+0000 7f53f035e640 1 -- 172.21.15.76:0/2515253915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53e8159c70 msgr2=0x7f53e814e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:25.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.444+0000 7f53f035e640 1 --2- 172.21.15.76:0/2515253915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53e8159c70 0x7f53e814e700 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f53e0007ef0 tx=0x7f53e000d4a0 comp rx=0 tx=0).stop 2023-11-09T03:49:25.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.444+0000 7f53f035e640 1 -- 172.21.15.76:0/2515253915 shutdown_connections 2023-11-09T03:49:25.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.444+0000 7f53f035e640 1 --2- 172.21.15.76:0/2515253915 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f53cc06aea0 0x7f53cc06d360 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:25.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.444+0000 7f53f035e640 1 --2- 172.21.15.76:0/2515253915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53e8159c70 0x7f53e814e700 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:25.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.444+0000 7f53f035e640 1 --2- 172.21.15.76:0/2515253915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53e80b0ae0 0x7f53e814e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:25.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.444+0000 7f53f035e640 1 --2- 172.21.15.76:0/2515253915 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53e806db20 0x7f53e80c4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:25.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.444+0000 7f53f035e640 1 -- 172.21.15.76:0/2515253915 >> 172.21.15.76:0/2515253915 conn(0x7f53e80ce950 msgr2=0x7f53e814fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:25.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.444+0000 7f53f035e640 1 -- 172.21.15.76:0/2515253915 shutdown_connections 2023-11-09T03:49:25.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:25.444+0000 7f53f035e640 1 -- 172.21.15.76:0/2515253915 wait complete. 2023-11-09T03:49:25.455 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:49:26.214 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-11-09T03:49:30.456 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:49:30.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.565+0000 7f73cd875640 1 Processor -- start 2023-11-09T03:49:30.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.565+0000 7f73cd875640 1 -- start start 2023-11-09T03:49:30.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.566+0000 7f73cd875640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f73c80cbea0 0x7f73c806eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:30.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.566+0000 7f73cd875640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73c80cc2b0 con 0x7f73c815ab70 2023-11-09T03:49:30.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.566+0000 7f73cd875640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73c806d310 con 0x7f73c806a830 2023-11-09T03:49:30.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.566+0000 7f73cd875640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73c80cba70 con 0x7f73c80cbea0 2023-11-09T03:49:30.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.566+0000 7f73c77fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f73c80cbea0 0x7f73c806eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:30.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.566+0000 7f73c6ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f73c815ab70 0x7f73c806a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:56598/0 (socket says 172.21.15.76:56598) 2023-11-09T03:49:30.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.566+0000 7f73c77fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f73c80cbea0 0x7f73c806eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47814/0 (socket says 172.21.15.76:47814) 2023-11-09T03:49:30.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.566+0000 7f73c6ffd640 1 -- 172.21.15.76:0/3784927600 learned_addr learned my addr 172.21.15.76:0/3784927600 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:49:30.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.567+0000 7f73c77fe640 1 -- 172.21.15.76:0/3784927600 >> v1:172.21.15.113:6789/0 conn(0x7f73c806a830 legacy=0x7f73c80ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:30.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.567+0000 7f73c77fe640 1 -- 172.21.15.76:0/3784927600 >> v1:172.21.15.76:6789/0 conn(0x7f73c815ab70 legacy=0x7f73c806a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:30.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.567+0000 7f73c77fe640 1 -- 172.21.15.76:0/3784927600 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c815d860 con 0x7f73c80cbea0 2023-11-09T03:49:30.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.567+0000 7f73c67fc640 1 -- 172.21.15.76:0/3784927600 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2461692107 0 0) 0x7f73c80cc2b0 con 0x7f73c815ab70 2023-11-09T03:49:30.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.567+0000 7f73c77fe640 1 --2- 172.21.15.76:0/3784927600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73c80cbea0 0x7f73c806eba0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009d90 tx=0x7f73c002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbfae37df8f2ae8e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:30.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.568+0000 7f73c67fc640 1 -- 172.21.15.76:0/3784927600 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c002cc60 con 0x7f73c80cbea0 2023-11-09T03:49:30.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.568+0000 7f73c67fc640 1 -- 172.21.15.76:0/3784927600 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c002cdc0 con 0x7f73c80cbea0 2023-11-09T03:49:30.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.568+0000 7f73c67fc640 1 -- 172.21.15.76:0/3784927600 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c0034660 con 0x7f73c80cbea0 2023-11-09T03:49:30.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.568+0000 7f73cd875640 1 -- 172.21.15.76:0/3784927600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73c80cbea0 msgr2=0x7f73c806eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:30.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.568+0000 7f73cd875640 1 --2- 172.21.15.76:0/3784927600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73c80cbea0 0x7f73c806eba0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009d90 tx=0x7f73c002beb0 comp rx=0 tx=0).stop 2023-11-09T03:49:30.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.568+0000 7f73cd875640 1 -- 172.21.15.76:0/3784927600 shutdown_connections 2023-11-09T03:49:30.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.568+0000 7f73cd875640 1 --2- 172.21.15.76:0/3784927600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73c80cbea0 0x7f73c806eba0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:30.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.568+0000 7f73cd875640 1 -- 172.21.15.76:0/3784927600 >> 172.21.15.76:0/3784927600 conn(0x7f73c8098d40 msgr2=0x7f73c80cfa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:30.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.568+0000 7f73cd875640 1 -- 172.21.15.76:0/3784927600 shutdown_connections 2023-11-09T03:49:30.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.568+0000 7f73cd875640 1 -- 172.21.15.76:0/3784927600 wait complete. 2023-11-09T03:49:30.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.569+0000 7f73cd875640 1 Processor -- start 2023-11-09T03:49:30.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.569+0000 7f73cd875640 1 -- start start 2023-11-09T03:49:30.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.570+0000 7f73cd875640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73c806a830 0x7f73c81f3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:30.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.570+0000 7f73cd875640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73c80cbea0 0x7f73c81573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:30.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.570+0000 7f73cd875640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73c815ab70 0x7f73c809a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:30.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.570+0000 7f73cd875640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c8161c50 con 0x7f73c815ab70 2023-11-09T03:49:30.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.570+0000 7f73cd875640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c809abc0 con 0x7f73c80cbea0 2023-11-09T03:49:30.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.570+0000 7f73cd875640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73c81f4200 con 0x7f73c806a830 2023-11-09T03:49:30.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.570+0000 7f73c77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73c815ab70 0x7f73c809a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:30.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.570+0000 7f73c77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73c815ab70 0x7f73c809a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47818/0 (socket says 172.21.15.76:47818) 2023-11-09T03:49:30.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.570+0000 7f73c77fe640 1 -- 172.21.15.76:0/2717776501 learned_addr learned my addr 172.21.15.76:0/2717776501 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:30.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.570+0000 7f73bffff640 1 --2- 172.21.15.76:0/2717776501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73c80cbea0 0x7f73c81573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:30.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.571+0000 7f73c6ffd640 1 --2- 172.21.15.76:0/2717776501 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73c806a830 0x7f73c81f3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:30.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.571+0000 7f73c77fe640 1 -- 172.21.15.76:0/2717776501 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73c806a830 msgr2=0x7f73c81f3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:30.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.571+0000 7f73c77fe640 1 --2- 172.21.15.76:0/2717776501 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73c806a830 0x7f73c81f3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:30.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.571+0000 7f73c77fe640 1 -- 172.21.15.76:0/2717776501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73c80cbea0 msgr2=0x7f73c81573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:30.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.571+0000 7f73c77fe640 1 --2- 172.21.15.76:0/2717776501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73c80cbea0 0x7f73c81573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:30.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.571+0000 7f73c77fe640 1 -- 172.21.15.76:0/2717776501 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c0009a70 con 0x7f73c815ab70 2023-11-09T03:49:30.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.571+0000 7f73bffff640 1 --2- 172.21.15.76:0/2717776501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73c80cbea0 0x7f73c81573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:49:30.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.571+0000 7f73c77fe640 1 --2- 172.21.15.76:0/2717776501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73c815ab70 0x7f73c809a680 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009d90 tx=0x7f73c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:30.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.571+0000 7f73c4ff9640 1 -- 172.21.15.76:0/2717776501 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c0040450 con 0x7f73c815ab70 2023-11-09T03:49:30.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.571+0000 7f73c4ff9640 1 -- 172.21.15.76:0/2717776501 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c00405d0 con 0x7f73c815ab70 2023-11-09T03:49:30.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.571+0000 7f73cd875640 1 -- 172.21.15.76:0/2717776501 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c81f44e0 con 0x7f73c815ab70 2023-11-09T03:49:30.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.571+0000 7f73c4ff9640 1 -- 172.21.15.76:0/2717776501 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c0040980 con 0x7f73c815ab70 2023-11-09T03:49:30.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.571+0000 7f73cd875640 1 -- 172.21.15.76:0/2717776501 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c81f4aa0 con 0x7f73c815ab70 2023-11-09T03:49:30.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.572+0000 7f73cd875640 1 -- 172.21.15.76:0/2717776501 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73c8074290 con 0x7f73c815ab70 2023-11-09T03:49:30.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.572+0000 7f73c4ff9640 1 -- 172.21.15.76:0/2717776501 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f73c0040ae0 con 0x7f73c815ab70 2023-11-09T03:49:30.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.573+0000 7f73c4ff9640 1 --2- 172.21.15.76:0/2717776501 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f73a806ae70 0x7f73a806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:30.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.573+0000 7f73c4ff9640 1 -- 172.21.15.76:0/2717776501 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7178+0+0 (secure 0 0 0) 0x7f73c00ac700 con 0x7f73c815ab70 2023-11-09T03:49:30.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.573+0000 7f73c6ffd640 1 --2- 172.21.15.76:0/2717776501 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f73a806ae70 0x7f73a806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:30.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.573+0000 7f73c6ffd640 1 --2- 172.21.15.76:0/2717776501 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f73a806ae70 0x7f73a806d330 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f73b80010e0 tx=0x7f73b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:30.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.577+0000 7f73c4ff9640 1 -- 172.21.15.76:0/2717776501 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f73c00772b0 con 0x7f73c815ab70 2023-11-09T03:49:30.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.779+0000 7f73cd875640 1 -- 172.21.15.76:0/2717776501 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73c80cefe0 con 0x7f73c815ab70 2023-11-09T03:49:30.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.779+0000 7f73c4ff9640 1 -- 172.21.15.76:0/2717776501 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4933 (secure 0 0 0) 0x7f73c007a8d0 con 0x7f73c815ab70 2023-11-09T03:49:30.780 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:49:30.780 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5744,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3280098776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3280098776},{"type":"v1","addr":"172.21.15.113:6837","nonce":3280098776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:49:18.972774+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7452,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7452":{"gid":7452,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/4011275120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4011275120},{"type":"v1","addr":"172.21.15.76:6835","nonce":4011275120}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:49:30.781 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 50 2023-11-09T03:49:30.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.781+0000 7f73cd875640 1 -- 172.21.15.76:0/2717776501 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f73a806ae70 msgr2=0x7f73a806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:30.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.781+0000 7f73cd875640 1 --2- 172.21.15.76:0/2717776501 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f73a806ae70 0x7f73a806d330 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f73b80010e0 tx=0x7f73b8009040 comp rx=0 tx=0).stop 2023-11-09T03:49:30.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.781+0000 7f73cd875640 1 -- 172.21.15.76:0/2717776501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73c815ab70 msgr2=0x7f73c809a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:30.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.781+0000 7f73cd875640 1 --2- 172.21.15.76:0/2717776501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73c815ab70 0x7f73c809a680 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009d90 tx=0x7f73c0004a40 comp rx=0 tx=0).stop 2023-11-09T03:49:30.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.781+0000 7f73cd875640 1 -- 172.21.15.76:0/2717776501 shutdown_connections 2023-11-09T03:49:30.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.781+0000 7f73cd875640 1 --2- 172.21.15.76:0/2717776501 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f73a806ae70 0x7f73a806d330 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:30.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.781+0000 7f73cd875640 1 --2- 172.21.15.76:0/2717776501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73c815ab70 0x7f73c809a680 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:30.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.781+0000 7f73cd875640 1 --2- 172.21.15.76:0/2717776501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73c80cbea0 0x7f73c81573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:30.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.781+0000 7f73cd875640 1 --2- 172.21.15.76:0/2717776501 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73c806a830 0x7f73c81f3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:30.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.781+0000 7f73cd875640 1 -- 172.21.15.76:0/2717776501 >> 172.21.15.76:0/2717776501 conn(0x7f73c8098d40 msgr2=0x7f73c806a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:30.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.781+0000 7f73cd875640 1 -- 172.21.15.76:0/2717776501 shutdown_connections 2023-11-09T03:49:30.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:30.781+0000 7f73cd875640 1 -- 172.21.15.76:0/2717776501 wait complete. 2023-11-09T03:49:30.792 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:49:31.300 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:49:31.302 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:49:32.825 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~7s 2023-11-09T03:49:35.794 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:49:35.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.911+0000 7f88ab7a9640 1 Processor -- start 2023-11-09T03:49:35.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.912+0000 7f88ab7a9640 1 -- start start 2023-11-09T03:49:35.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.912+0000 7f88ab7a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88a415ce60 0x7f88a406b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:35.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.912+0000 7f88ab7a9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f88a406bd30 0x7f88a4160b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:35.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.912+0000 7f88ab7a9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a41615d0 con 0x7f88a4161050 2023-11-09T03:49:35.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.912+0000 7f88ab7a9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88a40cbde0 con 0x7f88a406bd30 2023-11-09T03:49:35.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.912+0000 7f88ab7a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a41618b0 con 0x7f88a415ce60 2023-11-09T03:49:35.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.913+0000 7f88a8d44640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f88a406bd30 0x7f88a4160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:35.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.913+0000 7f88a8d44640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f88a406bd30 0x7f88a4160b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57300/0 (socket says 172.21.15.76:57300) 2023-11-09T03:49:35.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.913+0000 7f88a8d44640 1 -- 172.21.15.76:0/2618952707 learned_addr learned my addr 172.21.15.76:0/2618952707 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:35.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.913+0000 7f88a9545640 1 --2- 172.21.15.76:0/2618952707 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88a415ce60 0x7f88a406b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:35.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.913+0000 7f88a8d44640 1 -- 172.21.15.76:0/2618952707 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88a415ce60 msgr2=0x7f88a406b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:35.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.913+0000 7f88a8d44640 1 --2- 172.21.15.76:0/2618952707 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88a415ce60 0x7f88a406b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:35.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.913+0000 7f88a8d44640 1 -- 172.21.15.76:0/2618952707 >> v1:172.21.15.113:6789/0 conn(0x7f88a4161050 legacy=0x7f88a40cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:35.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.913+0000 7f88a8d44640 1 -- 172.21.15.76:0/2618952707 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a40cba00 con 0x7f88a406bd30 2023-11-09T03:49:35.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.914+0000 7f88a8d44640 1 --2- 172.21.15.76:0/2618952707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88a406bd30 0x7f88a4160b10 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f8894009870 tx=0x7f889402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fbd3080fc826a7f server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:35.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.914+0000 7f889bfff640 1 -- 172.21.15.76:0/2618952707 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8894031640 con 0x7f88a406bd30 2023-11-09T03:49:35.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.914+0000 7f889bfff640 1 -- 172.21.15.76:0/2618952707 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88940317a0 con 0x7f88a406bd30 2023-11-09T03:49:35.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.914+0000 7f889bfff640 1 -- 172.21.15.76:0/2618952707 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8894031b30 con 0x7f88a406bd30 2023-11-09T03:49:35.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.914+0000 7f88ab7a9640 1 -- 172.21.15.76:0/2618952707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88a406bd30 msgr2=0x7f88a4160b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:35.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.914+0000 7f88ab7a9640 1 --2- 172.21.15.76:0/2618952707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88a406bd30 0x7f88a4160b10 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f8894009870 tx=0x7f889402b8f0 comp rx=0 tx=0).stop 2023-11-09T03:49:35.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.914+0000 7f88ab7a9640 1 -- 172.21.15.76:0/2618952707 shutdown_connections 2023-11-09T03:49:35.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.914+0000 7f88ab7a9640 1 --2- 172.21.15.76:0/2618952707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88a406bd30 0x7f88a4160b10 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:35.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.914+0000 7f88ab7a9640 1 --2- 172.21.15.76:0/2618952707 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88a415ce60 0x7f88a406b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:35.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.915+0000 7f88ab7a9640 1 -- 172.21.15.76:0/2618952707 >> 172.21.15.76:0/2618952707 conn(0x7f88a406ea90 msgr2=0x7f88a406ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:35.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.915+0000 7f88ab7a9640 1 -- 172.21.15.76:0/2618952707 shutdown_connections 2023-11-09T03:49:35.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.915+0000 7f88ab7a9640 1 -- 172.21.15.76:0/2618952707 wait complete. 2023-11-09T03:49:35.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.916+0000 7f88ab7a9640 1 Processor -- start 2023-11-09T03:49:35.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.916+0000 7f88ab7a9640 1 -- start start 2023-11-09T03:49:35.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.917+0000 7f88ab7a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88a415ce60 0x7f88a41540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:35.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.917+0000 7f88ab7a9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88a4161050 0x7f88a40b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:35.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.917+0000 7f88ab7a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88a40b6260 0x7f88a415e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:35.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.917+0000 7f88ab7a9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a40b6710 con 0x7f88a4161050 2023-11-09T03:49:35.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.917+0000 7f88ab7a9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a415f3a0 con 0x7f88a40b6260 2023-11-09T03:49:35.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.917+0000 7f88ab7a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a415f680 con 0x7f88a415ce60 2023-11-09T03:49:35.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.917+0000 7f88a8d44640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88a4161050 0x7f88a40b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:35.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.917+0000 7f88a8d44640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88a4161050 0x7f88a40b5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57310/0 (socket says 172.21.15.76:57310) 2023-11-09T03:49:35.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.917+0000 7f88a8d44640 1 -- 172.21.15.76:0/3692541204 learned_addr learned my addr 172.21.15.76:0/3692541204 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:35.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.917+0000 7f88a9545640 1 --2- 172.21.15.76:0/3692541204 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88a415ce60 0x7f88a41540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:35.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.917+0000 7f88a9d46640 1 --2- 172.21.15.76:0/3692541204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88a40b6260 0x7f88a415e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:35.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.917+0000 7f88a8d44640 1 -- 172.21.15.76:0/3692541204 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88a415ce60 msgr2=0x7f88a41540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:35.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.917+0000 7f88a8d44640 1 --2- 172.21.15.76:0/3692541204 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88a415ce60 0x7f88a41540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:35.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.917+0000 7f88a8d44640 1 -- 172.21.15.76:0/3692541204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88a40b6260 msgr2=0x7f88a415e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:35.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.917+0000 7f88a8d44640 1 --2- 172.21.15.76:0/3692541204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88a40b6260 0x7f88a415e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:35.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.917+0000 7f88a8d44640 1 -- 172.21.15.76:0/3692541204 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88940094d0 con 0x7f88a4161050 2023-11-09T03:49:35.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.917+0000 7f88a8d44640 1 --2- 172.21.15.76:0/3692541204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88a4161050 0x7f88a40b5d20 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f8894009870 tx=0x7f88940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:35.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.917+0000 7f8899ffb640 1 -- 172.21.15.76:0/3692541204 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889403ac60 con 0x7f88a4161050 2023-11-09T03:49:35.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.917+0000 7f8899ffb640 1 -- 172.21.15.76:0/3692541204 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f889403adc0 con 0x7f88a4161050 2023-11-09T03:49:35.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.917+0000 7f8899ffb640 1 -- 172.21.15.76:0/3692541204 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8894044630 con 0x7f88a4161050 2023-11-09T03:49:35.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.917+0000 7f88ab7a9640 1 -- 172.21.15.76:0/3692541204 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88a40b37c0 con 0x7f88a4161050 2023-11-09T03:49:35.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.918+0000 7f88ab7a9640 1 -- 172.21.15.76:0/3692541204 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a40b3d80 con 0x7f88a4161050 2023-11-09T03:49:35.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.919+0000 7f88ab7a9640 1 -- 172.21.15.76:0/3692541204 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88a40b3f70 con 0x7f88a4161050 2023-11-09T03:49:35.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.919+0000 7f8899ffb640 1 -- 172.21.15.76:0/3692541204 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f889402f050 con 0x7f88a4161050 2023-11-09T03:49:35.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.920+0000 7f8899ffb640 1 --2- 172.21.15.76:0/3692541204 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f888806ae70 0x7f888806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:35.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.920+0000 7f8899ffb640 1 -- 172.21.15.76:0/3692541204 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7178+0+0 (secure 0 0 0) 0x7f88940ab8b0 con 0x7f88a4161050 2023-11-09T03:49:35.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.920+0000 7f88a9545640 1 --2- 172.21.15.76:0/3692541204 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f888806ae70 0x7f888806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:35.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.920+0000 7f88a9545640 1 --2- 172.21.15.76:0/3692541204 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f888806ae70 0x7f888806d330 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f88a0003b80 tx=0x7f88a00098d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:35.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:35.924+0000 7f8899ffb640 1 -- 172.21.15.76:0/3692541204 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8894076450 con 0x7f88a4161050 2023-11-09T03:49:36.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:36.124+0000 7f88ab7a9640 1 -- 172.21.15.76:0/3692541204 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88a4074290 con 0x7f88a4161050 2023-11-09T03:49:36.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:36.125+0000 7f8899ffb640 1 -- 172.21.15.76:0/3692541204 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4933 (secure 0 0 0) 0x7f8894031e60 con 0x7f88a4161050 2023-11-09T03:49:36.125 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:49:36.125 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5744,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3280098776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3280098776},{"type":"v1","addr":"172.21.15.113:6837","nonce":3280098776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:49:18.972774+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7452,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7452":{"gid":7452,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/4011275120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4011275120},{"type":"v1","addr":"172.21.15.76:6835","nonce":4011275120}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:49:36.126 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 50 2023-11-09T03:49:36.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:36.127+0000 7f88ab7a9640 1 -- 172.21.15.76:0/3692541204 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f888806ae70 msgr2=0x7f888806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:36.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:36.127+0000 7f88ab7a9640 1 --2- 172.21.15.76:0/3692541204 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f888806ae70 0x7f888806d330 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f88a0003b80 tx=0x7f88a00098d0 comp rx=0 tx=0).stop 2023-11-09T03:49:36.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:36.127+0000 7f88ab7a9640 1 -- 172.21.15.76:0/3692541204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88a4161050 msgr2=0x7f88a40b5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:36.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:36.127+0000 7f88ab7a9640 1 --2- 172.21.15.76:0/3692541204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88a4161050 0x7f88a40b5d20 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f8894009870 tx=0x7f88940046c0 comp rx=0 tx=0).stop 2023-11-09T03:49:36.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:36.127+0000 7f88ab7a9640 1 -- 172.21.15.76:0/3692541204 shutdown_connections 2023-11-09T03:49:36.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:36.127+0000 7f88ab7a9640 1 --2- 172.21.15.76:0/3692541204 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f888806ae70 0x7f888806d330 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:36.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:36.127+0000 7f88ab7a9640 1 --2- 172.21.15.76:0/3692541204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88a40b6260 0x7f88a415e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:36.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:36.127+0000 7f88ab7a9640 1 --2- 172.21.15.76:0/3692541204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88a4161050 0x7f88a40b5d20 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:36.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:36.127+0000 7f88ab7a9640 1 --2- 172.21.15.76:0/3692541204 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88a415ce60 0x7f88a41540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:36.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:36.127+0000 7f88ab7a9640 1 -- 172.21.15.76:0/3692541204 >> 172.21.15.76:0/3692541204 conn(0x7f88a406ea90 msgr2=0x7f88a40c46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:36.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:36.127+0000 7f88ab7a9640 1 -- 172.21.15.76:0/3692541204 shutdown_connections 2023-11-09T03:49:36.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:36.127+0000 7f88ab7a9640 1 -- 172.21.15.76:0/3692541204 wait complete. 2023-11-09T03:49:36.138 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:49:39.435 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~13s 2023-11-09T03:49:41.138 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:49:41.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.248+0000 7f8e94221640 1 Processor -- start 2023-11-09T03:49:41.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.249+0000 7f8e94221640 1 -- start start 2023-11-09T03:49:41.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.249+0000 7f8e94221640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8e8c0cde60 0x7f8e8c15d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:41.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.249+0000 7f8e94221640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e8c15daa0 0x7f8e8c157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:41.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.249+0000 7f8e94221640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e8c15e000 con 0x7f8e8c0ce240 2023-11-09T03:49:41.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.249+0000 7f8e94221640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e8c15e5c0 con 0x7f8e8c0cde60 2023-11-09T03:49:41.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.249+0000 7f8e94221640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c15e2e0 con 0x7f8e8c15daa0 2023-11-09T03:49:41.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.249+0000 7f8e91fbd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8e8c0cde60 0x7f8e8c15d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:41.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.249+0000 7f8e91fbd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8e8c0cde60 0x7f8e8c15d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57314/0 (socket says 172.21.15.76:57314) 2023-11-09T03:49:41.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.249+0000 7f8e91fbd640 1 -- 172.21.15.76:0/591449952 learned_addr learned my addr 172.21.15.76:0/591449952 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:41.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.250+0000 7f8e917bc640 1 --2- 172.21.15.76:0/591449952 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e8c15daa0 0x7f8e8c157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:41.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.250+0000 7f8e91fbd640 1 -- 172.21.15.76:0/591449952 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e8c15daa0 msgr2=0x7f8e8c157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:41.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.250+0000 7f8e91fbd640 1 --2- 172.21.15.76:0/591449952 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e8c15daa0 0x7f8e8c157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:41.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.250+0000 7f8e91fbd640 1 -- 172.21.15.76:0/591449952 >> v1:172.21.15.113:6789/0 conn(0x7f8e8c0ce240 legacy=0x7f8e8c06b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:41.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.250+0000 7f8e91fbd640 1 -- 172.21.15.76:0/591449952 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8c0d1120 con 0x7f8e8c0cde60 2023-11-09T03:49:41.255 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.251+0000 7f8e91fbd640 1 --2- 172.21.15.76:0/591449952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e8c0cde60 0x7f8e8c15d560 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f8e88009e50 tx=0x7f8e8802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4656618289124e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:41.255 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.251+0000 7f8e90fbb640 1 -- 172.21.15.76:0/591449952 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8802cb00 con 0x7f8e8c0cde60 2023-11-09T03:49:41.255 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.251+0000 7f8e90fbb640 1 -- 172.21.15.76:0/591449952 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e8802cc60 con 0x7f8e8c0cde60 2023-11-09T03:49:41.255 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.251+0000 7f8e90fbb640 1 -- 172.21.15.76:0/591449952 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e880343f0 con 0x7f8e8c0cde60 2023-11-09T03:49:41.256 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.251+0000 7f8e94221640 1 -- 172.21.15.76:0/591449952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e8c0cde60 msgr2=0x7f8e8c15d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:41.256 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.251+0000 7f8e94221640 1 --2- 172.21.15.76:0/591449952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e8c0cde60 0x7f8e8c15d560 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f8e88009e50 tx=0x7f8e8802bf60 comp rx=0 tx=0).stop 2023-11-09T03:49:41.256 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.251+0000 7f8e94221640 1 -- 172.21.15.76:0/591449952 shutdown_connections 2023-11-09T03:49:41.256 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.251+0000 7f8e94221640 1 --2- 172.21.15.76:0/591449952 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e8c15daa0 0x7f8e8c157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:41.256 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.251+0000 7f8e94221640 1 --2- 172.21.15.76:0/591449952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e8c0cde60 0x7f8e8c15d560 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:41.257 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.251+0000 7f8e94221640 1 -- 172.21.15.76:0/591449952 >> 172.21.15.76:0/591449952 conn(0x7f8e8c071f30 msgr2=0x7f8e8c0cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:41.257 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.251+0000 7f8e94221640 1 -- 172.21.15.76:0/591449952 shutdown_connections 2023-11-09T03:49:41.257 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.251+0000 7f8e94221640 1 -- 172.21.15.76:0/591449952 wait complete. 2023-11-09T03:49:41.257 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.253+0000 7f8e94221640 1 Processor -- start 2023-11-09T03:49:41.257 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.253+0000 7f8e94221640 1 -- start start 2023-11-09T03:49:41.258 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.253+0000 7f8e94221640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e8c0cde60 0x7f8e8c15acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:41.258 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.253+0000 7f8e94221640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e8c0ce240 0x7f8e8c1f24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:41.258 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.253+0000 7f8e94221640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e8c15daa0 0x7f8e8c15cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:41.258 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.253+0000 7f8e94221640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c15b230 con 0x7f8e8c15daa0 2023-11-09T03:49:41.258 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.253+0000 7f8e94221640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c1f2a30 con 0x7f8e8c0ce240 2023-11-09T03:49:41.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.253+0000 7f8e94221640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c15d040 con 0x7f8e8c0cde60 2023-11-09T03:49:41.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.253+0000 7f8e927be640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e8c15daa0 0x7f8e8c15cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:41.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.253+0000 7f8e927be640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e8c15daa0 0x7f8e8c15cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57320/0 (socket says 172.21.15.76:57320) 2023-11-09T03:49:41.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.253+0000 7f8e927be640 1 -- 172.21.15.76:0/20540702 learned_addr learned my addr 172.21.15.76:0/20540702 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:41.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.254+0000 7f8e917bc640 1 --2- 172.21.15.76:0/20540702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e8c0ce240 0x7f8e8c1f24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:41.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.254+0000 7f8e91fbd640 1 --2- 172.21.15.76:0/20540702 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e8c0cde60 0x7f8e8c15acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:41.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.254+0000 7f8e927be640 1 -- 172.21.15.76:0/20540702 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e8c0cde60 msgr2=0x7f8e8c15acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:41.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.254+0000 7f8e927be640 1 --2- 172.21.15.76:0/20540702 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e8c0cde60 0x7f8e8c15acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:41.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.254+0000 7f8e927be640 1 -- 172.21.15.76:0/20540702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e8c0ce240 msgr2=0x7f8e8c1f24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:41.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.254+0000 7f8e927be640 1 --2- 172.21.15.76:0/20540702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e8c0ce240 0x7f8e8c1f24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:41.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.254+0000 7f8e927be640 1 -- 172.21.15.76:0/20540702 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e88009b30 con 0x7f8e8c15daa0 2023-11-09T03:49:41.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.254+0000 7f8e927be640 1 --2- 172.21.15.76:0/20540702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e8c15daa0 0x7f8e8c15cb00 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400c400 tx=0x7f8e8400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:41.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.254+0000 7f8e827fc640 1 -- 172.21.15.76:0/20540702 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e840052d0 con 0x7f8e8c15daa0 2023-11-09T03:49:41.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.254+0000 7f8e827fc640 1 -- 172.21.15.76:0/20540702 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e84005430 con 0x7f8e8c15daa0 2023-11-09T03:49:41.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.254+0000 7f8e94221640 1 -- 172.21.15.76:0/20540702 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e8c1f34e0 con 0x7f8e8c15daa0 2023-11-09T03:49:41.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.254+0000 7f8e827fc640 1 -- 172.21.15.76:0/20540702 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e84003810 con 0x7f8e8c15daa0 2023-11-09T03:49:41.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.254+0000 7f8e94221640 1 -- 172.21.15.76:0/20540702 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e8c1f37f0 con 0x7f8e8c15daa0 2023-11-09T03:49:41.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.255+0000 7f8e94221640 1 -- 172.21.15.76:0/20540702 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e8c1f3980 con 0x7f8e8c15daa0 2023-11-09T03:49:41.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.255+0000 7f8e827fc640 1 -- 172.21.15.76:0/20540702 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8e84003970 con 0x7f8e8c15daa0 2023-11-09T03:49:41.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.256+0000 7f8e827fc640 1 --2- 172.21.15.76:0/20540702 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8e7006ae70 0x7f8e7006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:41.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.256+0000 7f8e827fc640 1 -- 172.21.15.76:0/20540702 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7178+0+0 (secure 0 0 0) 0x7f8e84089050 con 0x7f8e8c15daa0 2023-11-09T03:49:41.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.256+0000 7f8e91fbd640 1 --2- 172.21.15.76:0/20540702 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8e7006ae70 0x7f8e7006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:41.264 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.257+0000 7f8e91fbd640 1 --2- 172.21.15.76:0/20540702 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8e7006ae70 0x7f8e7006d330 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8e88035000 tx=0x7f8e88030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:41.264 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.260+0000 7f8e827fc640 1 -- 172.21.15.76:0/20540702 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8e84053bf0 con 0x7f8e8c15daa0 2023-11-09T03:49:41.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.460+0000 7f8e94221640 1 -- 172.21.15.76:0/20540702 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e8c074290 con 0x7f8e8c15daa0 2023-11-09T03:49:41.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.460+0000 7f8e827fc640 1 -- 172.21.15.76:0/20540702 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4981 (secure 0 0 0) 0x7f8e84057210 con 0x7f8e8c15daa0 2023-11-09T03:49:41.460 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:49:41.461 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":51,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5744,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3280098776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3280098776},{"type":"v1","addr":"172.21.15.113:6837","nonce":3280098776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:49:36.428265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7452,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7452":{"gid":7452,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6835/4011275120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":4011275120},{"type":"v1","addr":"172.21.15.76:6835","nonce":4011275120}]},"join_fscid":-1,"laggy_since":"2023-11-09T03:49:36.428265+0000","export_targets":[1],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:49:41.461 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 51 2023-11-09T03:49:41.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.462+0000 7f8e94221640 1 -- 172.21.15.76:0/20540702 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8e7006ae70 msgr2=0x7f8e7006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:41.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.462+0000 7f8e94221640 1 --2- 172.21.15.76:0/20540702 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8e7006ae70 0x7f8e7006d330 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8e88035000 tx=0x7f8e88030000 comp rx=0 tx=0).stop 2023-11-09T03:49:41.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.462+0000 7f8e94221640 1 -- 172.21.15.76:0/20540702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e8c15daa0 msgr2=0x7f8e8c15cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:41.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.462+0000 7f8e94221640 1 --2- 172.21.15.76:0/20540702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e8c15daa0 0x7f8e8c15cb00 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400c400 tx=0x7f8e8400c8c0 comp rx=0 tx=0).stop 2023-11-09T03:49:41.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.462+0000 7f8e94221640 1 -- 172.21.15.76:0/20540702 shutdown_connections 2023-11-09T03:49:41.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.462+0000 7f8e94221640 1 --2- 172.21.15.76:0/20540702 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8e7006ae70 0x7f8e7006d330 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:41.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.462+0000 7f8e94221640 1 --2- 172.21.15.76:0/20540702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e8c15daa0 0x7f8e8c15cb00 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:41.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.462+0000 7f8e94221640 1 --2- 172.21.15.76:0/20540702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e8c0ce240 0x7f8e8c1f24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:41.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.462+0000 7f8e94221640 1 --2- 172.21.15.76:0/20540702 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e8c0cde60 0x7f8e8c15acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:41.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.462+0000 7f8e94221640 1 -- 172.21.15.76:0/20540702 >> 172.21.15.76:0/20540702 conn(0x7f8e8c071f30 msgr2=0x7f8e8c0c69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:41.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.463+0000 7f8e94221640 1 -- 172.21.15.76:0/20540702 shutdown_connections 2023-11-09T03:49:41.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.463+0000 7f8e94221640 1 -- 172.21.15.76:0/20540702 wait complete. 2023-11-09T03:49:41.473 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2023-11-09T03:49:41.473 DEBUG:teuthology.orchestra.run.smithi076:> sudo sysctl -n kernel.core_pattern 2023-11-09T03:49:41.499 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-09T03:49:41.500 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2023-11-09T03:49:41.500 DEBUG:teuthology.orchestra.run.smithi076:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2023-11-09T03:49:41.570 INFO:teuthology.orchestra.run.smithi076.stdout:1699501759.116502.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i c', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2023-11-09T03:49:41.571 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1699501759.116502.core'] 2023-11-09T03:49:41.572 INFO:tasks.cephfs.cephfs_test_case:Found core file 1699501759.116502.core, deleting it 2023-11-09T03:49:41.572 DEBUG:teuthology.orchestra.run.smithi076:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1699501759.116502.core 2023-11-09T03:49:41.641 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:0 2023-11-09T03:49:41.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.783+0000 7ff3b66ec640 1 Processor -- start 2023-11-09T03:49:41.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.783+0000 7ff3b66ec640 1 -- start start 2023-11-09T03:49:41.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.783+0000 7ff3b66ec640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b0159c00 0x7ff3b00cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:41.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.783+0000 7ff3b66ec640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff3b00c48e0 0x7ff3b00c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:41.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.783+0000 7ff3b66ec640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b015aff0 con 0x7ff3b00adac0 2023-11-09T03:49:41.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.783+0000 7ff3b66ec640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b00c52d0 con 0x7ff3b00c48e0 2023-11-09T03:49:41.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.783+0000 7ff3b66ec640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b015a3e0 con 0x7ff3b0159c00 2023-11-09T03:49:41.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.784+0000 7ff3af7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff3b00c48e0 0x7ff3b00c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:41.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.784+0000 7ff3b4c89640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff3b00adac0 0x7ff3b00adea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:42024/0 (socket says 172.21.15.76:42024) 2023-11-09T03:49:41.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.784+0000 7ff3b4c89640 1 -- 172.21.15.76:0/1059179023 learned_addr learned my addr 172.21.15.76:0/1059179023 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:49:41.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.784+0000 7ff3affff640 1 --2- 172.21.15.76:0/1059179023 >> v2:172.21.15.113:3300/0 conn(0x7ff3b0159c00 0x7ff3b00cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:41.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.784+0000 7ff3af7fe640 1 -- 172.21.15.76:0/1059179023 >> v2:172.21.15.113:3300/0 conn(0x7ff3b0159c00 msgr2=0x7ff3b00cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:41.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.784+0000 7ff3af7fe640 1 --2- 172.21.15.76:0/1059179023 >> v2:172.21.15.113:3300/0 conn(0x7ff3b0159c00 0x7ff3b00cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:41.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.784+0000 7ff3af7fe640 1 -- 172.21.15.76:0/1059179023 >> v1:172.21.15.76:6789/0 conn(0x7ff3b00adac0 legacy=0x7ff3b00adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:41.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.784+0000 7ff3aeffd640 1 -- 172.21.15.76:0/1059179023 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1686782245 0 0) 0x7ff3b015aff0 con 0x7ff3b00adac0 2023-11-09T03:49:41.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.784+0000 7ff3af7fe640 1 -- 172.21.15.76:0/1059179023 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b015a6c0 con 0x7ff3b00c48e0 2023-11-09T03:49:41.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.785+0000 7ff3af7fe640 1 --2- 172.21.15.76:0/1059179023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3b00c48e0 0x7ff3b00c4d90 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00cc4d0 tx=0x7ff3a002e010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f443b36040600816 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:41.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.785+0000 7ff3aeffd640 1 -- 172.21.15.76:0/1059179023 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a002edc0 con 0x7ff3b00c48e0 2023-11-09T03:49:41.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.785+0000 7ff3aeffd640 1 -- 172.21.15.76:0/1059179023 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a0036440 con 0x7ff3b00c48e0 2023-11-09T03:49:41.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.785+0000 7ff3aeffd640 1 -- 172.21.15.76:0/1059179023 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a00367d0 con 0x7ff3b00c48e0 2023-11-09T03:49:41.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.785+0000 7ff3b66ec640 1 -- 172.21.15.76:0/1059179023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3b00c48e0 msgr2=0x7ff3b00c4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:41.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.785+0000 7ff3b66ec640 1 --2- 172.21.15.76:0/1059179023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3b00c48e0 0x7ff3b00c4d90 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00cc4d0 tx=0x7ff3a002e010 comp rx=0 tx=0).stop 2023-11-09T03:49:41.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.785+0000 7ff3b66ec640 1 -- 172.21.15.76:0/1059179023 shutdown_connections 2023-11-09T03:49:41.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.785+0000 7ff3b66ec640 1 --2- 172.21.15.76:0/1059179023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3b00c48e0 0x7ff3b00c4d90 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:41.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.785+0000 7ff3b66ec640 1 --2- 172.21.15.76:0/1059179023 >> v2:172.21.15.113:3300/0 conn(0x7ff3b0159c00 0x7ff3b00cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:41.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.785+0000 7ff3b66ec640 1 -- 172.21.15.76:0/1059179023 >> 172.21.15.76:0/1059179023 conn(0x7ff3b00ce950 msgr2=0x7ff3b00ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:41.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.785+0000 7ff3b66ec640 1 -- 172.21.15.76:0/1059179023 shutdown_connections 2023-11-09T03:49:41.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.785+0000 7ff3b66ec640 1 -- 172.21.15.76:0/1059179023 wait complete. 2023-11-09T03:49:41.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.786+0000 7ff3b66ec640 1 Processor -- start 2023-11-09T03:49:41.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.787+0000 7ff3b66ec640 1 -- start start 2023-11-09T03:49:41.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.787+0000 7ff3b66ec640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3b00adac0 0x7ff3b01f17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:41.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.787+0000 7ff3b66ec640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3b00c48e0 0x7ff3b0150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:41.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.787+0000 7ff3b66ec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3b0159c00 0x7ff3b006c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:41.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.787+0000 7ff3b66ec640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b0150a60 con 0x7ff3b0159c00 2023-11-09T03:49:41.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.787+0000 7ff3b66ec640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b0156470 con 0x7ff3b00c48e0 2023-11-09T03:49:41.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.787+0000 7ff3b66ec640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b0156750 con 0x7ff3b00adac0 2023-11-09T03:49:41.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.788+0000 7ff3b4c89640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3b0159c00 0x7ff3b006c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:41.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.788+0000 7ff3b4c89640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3b0159c00 0x7ff3b006c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57348/0 (socket says 172.21.15.76:57348) 2023-11-09T03:49:41.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.788+0000 7ff3b4c89640 1 -- 172.21.15.76:0/1180877894 learned_addr learned my addr 172.21.15.76:0/1180877894 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:41.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.788+0000 7ff3af7fe640 1 --2- 172.21.15.76:0/1180877894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3b00c48e0 0x7ff3b0150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:41.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.788+0000 7ff3affff640 1 --2- 172.21.15.76:0/1180877894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3b00adac0 0x7ff3b01f17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:41.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.788+0000 7ff3b4c89640 1 -- 172.21.15.76:0/1180877894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3b00adac0 msgr2=0x7ff3b01f17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:41.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.788+0000 7ff3b4c89640 1 --2- 172.21.15.76:0/1180877894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3b00adac0 0x7ff3b01f17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:41.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.788+0000 7ff3b4c89640 1 -- 172.21.15.76:0/1180877894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3b00c48e0 msgr2=0x7ff3b0150520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:41.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.788+0000 7ff3b4c89640 1 --2- 172.21.15.76:0/1180877894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3b00c48e0 0x7ff3b0150520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:41.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.788+0000 7ff3b4c89640 1 -- 172.21.15.76:0/1180877894 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a000b6a0 con 0x7ff3b0159c00 2023-11-09T03:49:41.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.788+0000 7ff3b4c89640 1 --2- 172.21.15.76:0/1180877894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3b0159c00 0x7ff3b006c710 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800c7a0 tx=0x7ff3a800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:41.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.788+0000 7ff3acff9640 1 -- 172.21.15.76:0/1180877894 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a800a640 con 0x7ff3b0159c00 2023-11-09T03:49:41.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.788+0000 7ff3acff9640 1 -- 172.21.15.76:0/1180877894 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a800a7a0 con 0x7ff3b0159c00 2023-11-09T03:49:41.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.788+0000 7ff3b66ec640 1 -- 172.21.15.76:0/1180877894 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b0156a30 con 0x7ff3b0159c00 2023-11-09T03:49:41.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.788+0000 7ff3acff9640 1 -- 172.21.15.76:0/1180877894 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a800abd0 con 0x7ff3b0159c00 2023-11-09T03:49:41.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.788+0000 7ff3b66ec640 1 -- 172.21.15.76:0/1180877894 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b0154760 con 0x7ff3b0159c00 2023-11-09T03:49:41.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.789+0000 7ff3b66ec640 1 -- 172.21.15.76:0/1180877894 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b0074290 con 0x7ff3b0159c00 2023-11-09T03:49:41.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.789+0000 7ff3acff9640 1 -- 172.21.15.76:0/1180877894 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff3a800ad30 con 0x7ff3b0159c00 2023-11-09T03:49:41.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.790+0000 7ff3acff9640 1 --2- 172.21.15.76:0/1180877894 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff39c06ae70 0x7ff39c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:41.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.790+0000 7ff3acff9640 1 -- 172.21.15.76:0/1180877894 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7178+0+0 (secure 0 0 0) 0x7ff3a80892a0 con 0x7ff3b0159c00 2023-11-09T03:49:41.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.790+0000 7ff3affff640 1 --2- 172.21.15.76:0/1180877894 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff39c06ae70 0x7ff39c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:41.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.791+0000 7ff3affff640 1 --2- 172.21.15.76:0/1180877894 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff39c06ae70 0x7ff39c06d330 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40079a0 tx=0x7ff3a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:41.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:41.794+0000 7ff3acff9640 1 -- 172.21.15.76:0/1180877894 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff3a8053ed0 con 0x7ff3b0159c00 2023-11-09T03:49:42.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.028+0000 7ff3b66ec640 1 -- 172.21.15.76:0/1180877894 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:0"} v 0) v1 -- 0x7ff3b0153f10 con 0x7ff3b0159c00 2023-11-09T03:49:42.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.030+0000 7ff3acff9640 1 -- 172.21.15.76:0/1180877894 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:0"}]=0 failed mds gid 7452 v52) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff3a80574f0 con 0x7ff3b0159c00 2023-11-09T03:49:42.029 INFO:teuthology.orchestra.run.smithi076.stderr:failed mds gid 7452 2023-11-09T03:49:42.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.033+0000 7ff3b66ec640 1 -- 172.21.15.76:0/1180877894 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff39c06ae70 msgr2=0x7ff39c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:42.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.033+0000 7ff3b66ec640 1 --2- 172.21.15.76:0/1180877894 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff39c06ae70 0x7ff39c06d330 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40079a0 tx=0x7ff3a4008040 comp rx=0 tx=0).stop 2023-11-09T03:49:42.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.033+0000 7ff3b66ec640 1 -- 172.21.15.76:0/1180877894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3b0159c00 msgr2=0x7ff3b006c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:42.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.033+0000 7ff3b66ec640 1 --2- 172.21.15.76:0/1180877894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3b0159c00 0x7ff3b006c710 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800c7a0 tx=0x7ff3a800cc60 comp rx=0 tx=0).stop 2023-11-09T03:49:42.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.033+0000 7ff3b66ec640 1 -- 172.21.15.76:0/1180877894 shutdown_connections 2023-11-09T03:49:42.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.033+0000 7ff3b66ec640 1 --2- 172.21.15.76:0/1180877894 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff39c06ae70 0x7ff39c06d330 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:42.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.033+0000 7ff3b66ec640 1 --2- 172.21.15.76:0/1180877894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3b0159c00 0x7ff3b006c710 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:42.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.033+0000 7ff3b66ec640 1 --2- 172.21.15.76:0/1180877894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3b00c48e0 0x7ff3b0150520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:42.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.033+0000 7ff3b66ec640 1 --2- 172.21.15.76:0/1180877894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3b00adac0 0x7ff3b01f17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:42.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.033+0000 7ff3b66ec640 1 -- 172.21.15.76:0/1180877894 >> 172.21.15.76:0/1180877894 conn(0x7ff3b00ce950 msgr2=0x7ff3b00cb340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:42.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.033+0000 7ff3b66ec640 1 -- 172.21.15.76:0/1180877894 shutdown_connections 2023-11-09T03:49:42.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.033+0000 7ff3b66ec640 1 -- 172.21.15.76:0/1180877894 wait complete. 2023-11-09T03:49:42.043 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-09T03:49:42.044 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-09T03:49:42.044 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-09T03:49:42.044 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:49:42.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:49:42.045 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi076 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c' 2023-11-09T03:49:42.046 INFO:tasks.ceph.mds.c:Stopped 2023-11-09T03:49:42.046 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-09T03:49:42.049 INFO:tasks.ceph.mds.c:Started 2023-11-09T03:49:42.049 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:49:42.124 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2023-11-09T03:49:42.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.151+0000 7fb8854a9640 1 Processor -- start 2023-11-09T03:49:42.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.151+0000 7fb8854a9640 1 -- start start 2023-11-09T03:49:42.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.152+0000 7fb8854a9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb8800ce370 0x7fb88006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:42.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.152+0000 7fb8854a9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb88006e060 con 0x7fb880159c70 2023-11-09T03:49:42.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.152+0000 7fb8854a9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8800b4110 con 0x7fb8800b0ae0 2023-11-09T03:49:42.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.152+0000 7fb8854a9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8800b49b0 con 0x7fb8800ce370 2023-11-09T03:49:42.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.152+0000 7fb87f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb8800ce370 0x7fb88006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:42.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.152+0000 7fb87effd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb880159c70 0x7fb8800b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:42044/0 (socket says 172.21.15.76:42044) 2023-11-09T03:49:42.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.152+0000 7fb87effd640 1 -- 172.21.15.76:0/658907468 learned_addr learned my addr 172.21.15.76:0/658907468 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:49:42.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.152+0000 7fb87f7fe640 1 -- 172.21.15.76:0/658907468 >> v1:172.21.15.113:6789/0 conn(0x7fb8800b0ae0 legacy=0x7fb88015d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:42.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.152+0000 7fb87f7fe640 1 -- 172.21.15.76:0/658907468 >> v1:172.21.15.76:6789/0 conn(0x7fb880159c70 legacy=0x7fb8800b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:42.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.152+0000 7fb87e7fc640 1 -- 172.21.15.76:0/658907468 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2127442443 0 0) 0x7fb88006e060 con 0x7fb880159c70 2023-11-09T03:49:42.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.152+0000 7fb87f7fe640 1 -- 172.21.15.76:0/658907468 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8800b43f0 con 0x7fb8800ce370 2023-11-09T03:49:42.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.153+0000 7fb87f7fe640 1 --2- 172.21.15.76:0/658907468 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8800ce370 0x7fb88006db20 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fb88006e1f0 tx=0x7fb87802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3b1179a3074989 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:42.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.153+0000 7fb87e7fc640 1 -- 172.21.15.76:0/658907468 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87802cc50 con 0x7fb8800ce370 2023-11-09T03:49:42.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.153+0000 7fb87e7fc640 1 -- 172.21.15.76:0/658907468 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb87802cdb0 con 0x7fb8800ce370 2023-11-09T03:49:42.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.153+0000 7fb87e7fc640 1 -- 172.21.15.76:0/658907468 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb878034660 con 0x7fb8800ce370 2023-11-09T03:49:42.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.153+0000 7fb8854a9640 1 -- 172.21.15.76:0/658907468 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8800ce370 msgr2=0x7fb88006db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:42.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.153+0000 7fb8854a9640 1 --2- 172.21.15.76:0/658907468 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8800ce370 0x7fb88006db20 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fb88006e1f0 tx=0x7fb87802bea0 comp rx=0 tx=0).stop 2023-11-09T03:49:42.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.153+0000 7fb8854a9640 1 -- 172.21.15.76:0/658907468 shutdown_connections 2023-11-09T03:49:42.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.153+0000 7fb8854a9640 1 --2- 172.21.15.76:0/658907468 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8800ce370 0x7fb88006db20 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:42.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.153+0000 7fb8854a9640 1 -- 172.21.15.76:0/658907468 >> 172.21.15.76:0/658907468 conn(0x7fb8800ce950 msgr2=0x7fb8800ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:42.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.154+0000 7fb8854a9640 1 -- 172.21.15.76:0/658907468 shutdown_connections 2023-11-09T03:49:42.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.154+0000 7fb8854a9640 1 -- 172.21.15.76:0/658907468 wait complete. 2023-11-09T03:49:42.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.155+0000 7fb8854a9640 1 Processor -- start 2023-11-09T03:49:42.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.155+0000 7fb8854a9640 1 -- start start 2023-11-09T03:49:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.156+0000 7fb8854a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8800b0ae0 0x7fb8800c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.156+0000 7fb8854a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8800ce370 0x7fb8800af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.156+0000 7fb8854a9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb880159c70 0x7fb880070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.156+0000 7fb8854a9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8800ce7e0 con 0x7fb880159c70 2023-11-09T03:49:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.156+0000 7fb8854a9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8800afcf0 con 0x7fb8800b0ae0 2023-11-09T03:49:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.156+0000 7fb8854a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8800affd0 con 0x7fb8800ce370 2023-11-09T03:49:42.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.156+0000 7fb87f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb880159c70 0x7fb880070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:42.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.156+0000 7fb87f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb880159c70 0x7fb880070710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57392/0 (socket says 172.21.15.76:57392) 2023-11-09T03:49:42.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.156+0000 7fb87f7fe640 1 -- 172.21.15.76:0/737005656 learned_addr learned my addr 172.21.15.76:0/737005656 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:42.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.156+0000 7fb87effd640 1 --2- 172.21.15.76:0/737005656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8800b0ae0 0x7fb8800c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:42.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.156+0000 7fb8767fc640 1 --2- 172.21.15.76:0/737005656 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8800ce370 0x7fb8800af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:42.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.156+0000 7fb87f7fe640 1 -- 172.21.15.76:0/737005656 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8800ce370 msgr2=0x7fb8800af2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:42.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.156+0000 7fb87f7fe640 1 --2- 172.21.15.76:0/737005656 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8800ce370 0x7fb8800af2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:42.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.156+0000 7fb87f7fe640 1 -- 172.21.15.76:0/737005656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8800b0ae0 msgr2=0x7fb8800c46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:42.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.156+0000 7fb87f7fe640 1 --2- 172.21.15.76:0/737005656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8800b0ae0 0x7fb8800c46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:42.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.156+0000 7fb87f7fe640 1 -- 172.21.15.76:0/737005656 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb878009a70 con 0x7fb880159c70 2023-11-09T03:49:42.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.156+0000 7fb8767fc640 1 --2- 172.21.15.76:0/737005656 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8800ce370 0x7fb8800af2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:42.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.156+0000 7fb87f7fe640 1 --2- 172.21.15.76:0/737005656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb880159c70 0x7fb880070710 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fb87802c4b0 tx=0x7fb878004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:42.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.156+0000 7fb877fff640 1 -- 172.21.15.76:0/737005656 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb878003990 con 0x7fb880159c70 2023-11-09T03:49:42.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.156+0000 7fb877fff640 1 -- 172.21.15.76:0/737005656 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb878003af0 con 0x7fb880159c70 2023-11-09T03:49:42.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.156+0000 7fb8854a9640 1 -- 172.21.15.76:0/737005656 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8800b02b0 con 0x7fb880159c70 2023-11-09T03:49:42.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.156+0000 7fb877fff640 1 -- 172.21.15.76:0/737005656 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb878003e80 con 0x7fb880159c70 2023-11-09T03:49:42.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.156+0000 7fb8854a9640 1 -- 172.21.15.76:0/737005656 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb88014e0f0 con 0x7fb880159c70 2023-11-09T03:49:42.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.157+0000 7fb8854a9640 1 -- 172.21.15.76:0/737005656 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb88014e950 con 0x7fb880159c70 2023-11-09T03:49:42.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.158+0000 7fb877fff640 1 -- 172.21.15.76:0/737005656 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb878003fe0 con 0x7fb880159c70 2023-11-09T03:49:42.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.158+0000 7fb877fff640 1 --2- 172.21.15.76:0/737005656 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb86006ae70 0x7fb86006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:42.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.158+0000 7fb877fff640 1 -- 172.21.15.76:0/737005656 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7fb8780ac740 con 0x7fb880159c70 2023-11-09T03:49:42.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.158+0000 7fb87effd640 1 --2- 172.21.15.76:0/737005656 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb86006ae70 0x7fb86006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:42.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.159+0000 7fb87effd640 1 --2- 172.21.15.76:0/737005656 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb86006ae70 0x7fb86006d330 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb870007fd0 tx=0x7fb870007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:42.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.163+0000 7fb877fff640 1 -- 172.21.15.76:0/737005656 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb88014e950 con 0x7fb880159c70 2023-11-09T03:49:42.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.363+0000 7fb8854a9640 1 -- 172.21.15.76:0/737005656 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb88014e950 con 0x7fb880159c70 2023-11-09T03:49:42.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.363+0000 7fb877fff640 1 -- 172.21.15.76:0/737005656 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4142 (secure 0 0 0) 0x7fb88014e950 con 0x7fb880159c70 2023-11-09T03:49:42.363 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:49:42.363 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:49:42.033699+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5744,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5744":{"gid":5744,"name":"b","rank":0,"incarnation":53,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/3280098776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3280098776},{"type":"v1","addr":"172.21.15.113:6837","nonce":3280098776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:49:42.364 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 53 2023-11-09T03:49:42.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.366+0000 7fb8854a9640 1 -- 172.21.15.76:0/737005656 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb86006ae70 msgr2=0x7fb86006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:42.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.366+0000 7fb8854a9640 1 --2- 172.21.15.76:0/737005656 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb86006ae70 0x7fb86006d330 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb870007fd0 tx=0x7fb870007de0 comp rx=0 tx=0).stop 2023-11-09T03:49:42.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.366+0000 7fb8854a9640 1 -- 172.21.15.76:0/737005656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb880159c70 msgr2=0x7fb880070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:42.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.366+0000 7fb8854a9640 1 --2- 172.21.15.76:0/737005656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb880159c70 0x7fb880070710 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fb87802c4b0 tx=0x7fb878004a40 comp rx=0 tx=0).stop 2023-11-09T03:49:42.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.366+0000 7fb8854a9640 1 -- 172.21.15.76:0/737005656 shutdown_connections 2023-11-09T03:49:42.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.366+0000 7fb8854a9640 1 --2- 172.21.15.76:0/737005656 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb86006ae70 0x7fb86006d330 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:42.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.366+0000 7fb8854a9640 1 --2- 172.21.15.76:0/737005656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb880159c70 0x7fb880070710 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:42.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.366+0000 7fb8854a9640 1 --2- 172.21.15.76:0/737005656 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8800ce370 0x7fb8800af2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:42.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.366+0000 7fb8854a9640 1 --2- 172.21.15.76:0/737005656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8800b0ae0 0x7fb8800c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:42.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.366+0000 7fb8854a9640 1 -- 172.21.15.76:0/737005656 >> 172.21.15.76:0/737005656 conn(0x7fb8800ce950 msgr2=0x7fb88006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:42.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.366+0000 7fb8854a9640 1 -- 172.21.15.76:0/737005656 shutdown_connections 2023-11-09T03:49:42.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:42.366+0000 7fb8854a9640 1 -- 172.21.15.76:0/737005656 wait complete. 2023-11-09T03:49:42.376 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-11-09T03:49:46.146 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2023-11-09T03:49:47.377 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:49:47.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.486+0000 7f9d33839640 1 Processor -- start 2023-11-09T03:49:47.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.486+0000 7f9d33839640 1 -- start start 2023-11-09T03:49:47.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.487+0000 7f9d33839640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d2c159c70 0x7f9d2c0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:47.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.487+0000 7f9d33839640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9d2c06db20 0x7f9d2c0b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:47.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.487+0000 7f9d33839640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d2c0b4110 con 0x7f9d2c0b0910 2023-11-09T03:49:47.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.487+0000 7f9d33839640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d2c0b49b0 con 0x7f9d2c06db20 2023-11-09T03:49:47.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.487+0000 7f9d33839640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c0b0da0 con 0x7f9d2c159c70 2023-11-09T03:49:47.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.487+0000 7f9d30dd4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9d2c06db20 0x7f9d2c0b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:47.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.487+0000 7f9d31dd6640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f9d2c0b0910 0x7f9d2c15d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46200/0 (socket says 172.21.15.76:46200) 2023-11-09T03:49:47.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.487+0000 7f9d31dd6640 1 -- 172.21.15.76:0/1982534836 learned_addr learned my addr 172.21.15.76:0/1982534836 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:49:47.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.487+0000 7f9d315d5640 1 --2- 172.21.15.76:0/1982534836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d2c159c70 0x7f9d2c0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:47.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.487+0000 7f9d30dd4640 1 -- 172.21.15.76:0/1982534836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d2c159c70 msgr2=0x7f9d2c0ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:47.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.487+0000 7f9d30dd4640 1 --2- 172.21.15.76:0/1982534836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d2c159c70 0x7f9d2c0ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:47.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.487+0000 7f9d30dd4640 1 -- 172.21.15.76:0/1982534836 >> v1:172.21.15.76:6789/0 conn(0x7f9d2c0b0910 legacy=0x7f9d2c15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:47.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.487+0000 7f9d30dd4640 1 -- 172.21.15.76:0/1982534836 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c0b43f0 con 0x7f9d2c06db20 2023-11-09T03:49:47.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.487+0000 7f9d315d5640 1 --2- 172.21.15.76:0/1982534836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d2c159c70 0x7f9d2c0ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:47.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.488+0000 7f9d30dd4640 1 --2- 172.21.15.76:0/1982534836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d2c06db20 0x7f9d2c0b03d0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c0097a0 tx=0x7f9d1c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a333f93f4593fdce server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:47.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.488+0000 7f9d23fff640 1 -- 172.21.15.76:0/1982534836 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c031830 con 0x7f9d2c06db20 2023-11-09T03:49:47.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.488+0000 7f9d23fff640 1 -- 172.21.15.76:0/1982534836 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1c031990 con 0x7f9d2c06db20 2023-11-09T03:49:47.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.488+0000 7f9d23fff640 1 -- 172.21.15.76:0/1982534836 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c031d20 con 0x7f9d2c06db20 2023-11-09T03:49:47.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.489+0000 7f9d33839640 1 -- 172.21.15.76:0/1982534836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d2c06db20 msgr2=0x7f9d2c0b03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:47.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.489+0000 7f9d33839640 1 --2- 172.21.15.76:0/1982534836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d2c06db20 0x7f9d2c0b03d0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c0097a0 tx=0x7f9d1c02b8d0 comp rx=0 tx=0).stop 2023-11-09T03:49:47.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.489+0000 7f9d33839640 1 -- 172.21.15.76:0/1982534836 shutdown_connections 2023-11-09T03:49:47.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.489+0000 7f9d33839640 1 --2- 172.21.15.76:0/1982534836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d2c06db20 0x7f9d2c0b03d0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:47.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.489+0000 7f9d33839640 1 --2- 172.21.15.76:0/1982534836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d2c159c70 0x7f9d2c0ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:47.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.489+0000 7f9d33839640 1 -- 172.21.15.76:0/1982534836 >> 172.21.15.76:0/1982534836 conn(0x7f9d2c0ce950 msgr2=0x7f9d2c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:47.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.489+0000 7f9d33839640 1 -- 172.21.15.76:0/1982534836 shutdown_connections 2023-11-09T03:49:47.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.489+0000 7f9d33839640 1 -- 172.21.15.76:0/1982534836 wait complete. 2023-11-09T03:49:47.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.490+0000 7f9d33839640 1 Processor -- start 2023-11-09T03:49:47.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.490+0000 7f9d33839640 1 -- start start 2023-11-09T03:49:47.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.491+0000 7f9d33839640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d2c06db20 0x7f9d2c1eee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:47.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.491+0000 7f9d33839640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d2c0b0910 0x7f9d2c0b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:47.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.491+0000 7f9d33839640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d2c159c70 0x7f9d2c0b2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:47.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.491+0000 7f9d33839640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c0c6d60 con 0x7f9d2c159c70 2023-11-09T03:49:47.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.491+0000 7f9d33839640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c0a7670 con 0x7f9d2c06db20 2023-11-09T03:49:47.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.491+0000 7f9d33839640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c0c67a0 con 0x7f9d2c0b0910 2023-11-09T03:49:47.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.491+0000 7f9d31dd6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d2c159c70 0x7f9d2c0b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:47.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.491+0000 7f9d31dd6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d2c159c70 0x7f9d2c0b2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47506/0 (socket says 172.21.15.76:47506) 2023-11-09T03:49:47.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.491+0000 7f9d31dd6640 1 -- 172.21.15.76:0/1253214552 learned_addr learned my addr 172.21.15.76:0/1253214552 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:47.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.491+0000 7f9d30dd4640 1 --2- 172.21.15.76:0/1253214552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d2c0b0910 0x7f9d2c0b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:47.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.491+0000 7f9d315d5640 1 --2- 172.21.15.76:0/1253214552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d2c06db20 0x7f9d2c1eee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:47.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.491+0000 7f9d31dd6640 1 -- 172.21.15.76:0/1253214552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d2c0b0910 msgr2=0x7f9d2c0b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:47.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.491+0000 7f9d31dd6640 1 --2- 172.21.15.76:0/1253214552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d2c0b0910 0x7f9d2c0b2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:47.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.491+0000 7f9d31dd6640 1 -- 172.21.15.76:0/1253214552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d2c06db20 msgr2=0x7f9d2c1eee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:47.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.491+0000 7f9d31dd6640 1 --2- 172.21.15.76:0/1253214552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d2c06db20 0x7f9d2c1eee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:47.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.491+0000 7f9d31dd6640 1 -- 172.21.15.76:0/1253214552 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1c009400 con 0x7f9d2c159c70 2023-11-09T03:49:47.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.491+0000 7f9d315d5640 1 --2- 172.21.15.76:0/1253214552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d2c06db20 0x7f9d2c1eee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:47.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.491+0000 7f9d31dd6640 1 --2- 172.21.15.76:0/1253214552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d2c159c70 0x7f9d2c0b2960 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400cbc0 tx=0x7f9d2400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:47.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.491+0000 7f9d21ffb640 1 -- 172.21.15.76:0/1253214552 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2400ad40 con 0x7f9d2c159c70 2023-11-09T03:49:47.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.491+0000 7f9d21ffb640 1 -- 172.21.15.76:0/1253214552 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d24012400 con 0x7f9d2c159c70 2023-11-09T03:49:47.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.491+0000 7f9d21ffb640 1 -- 172.21.15.76:0/1253214552 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2400aea0 con 0x7f9d2c159c70 2023-11-09T03:49:47.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.491+0000 7f9d33839640 1 -- 172.21.15.76:0/1253214552 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2c0c6a80 con 0x7f9d2c159c70 2023-11-09T03:49:47.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.492+0000 7f9d33839640 1 -- 172.21.15.76:0/1253214552 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2c0c5a70 con 0x7f9d2c159c70 2023-11-09T03:49:47.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.493+0000 7f9d33839640 1 -- 172.21.15.76:0/1253214552 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d2c0a74d0 con 0x7f9d2c159c70 2023-11-09T03:49:47.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.493+0000 7f9d21ffb640 1 -- 172.21.15.76:0/1253214552 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9d24007070 con 0x7f9d2c159c70 2023-11-09T03:49:47.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.493+0000 7f9d21ffb640 1 --2- 172.21.15.76:0/1253214552 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9d1006ae70 0x7f9d1006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:47.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.494+0000 7f9d21ffb640 1 -- 172.21.15.76:0/1253214552 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7f9d24089a40 con 0x7f9d2c159c70 2023-11-09T03:49:47.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.494+0000 7f9d315d5640 1 --2- 172.21.15.76:0/1253214552 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9d1006ae70 0x7f9d1006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:47.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.494+0000 7f9d315d5640 1 --2- 172.21.15.76:0/1253214552 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9d1006ae70 0x7f9d1006d330 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9d28006fd0 tx=0x7f9d28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:47.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.498+0000 7f9d21ffb640 1 -- 172.21.15.76:0/1253214552 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9d2c0a74d0 con 0x7f9d2c159c70 2023-11-09T03:49:47.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.699+0000 7f9d33839640 1 -- 172.21.15.76:0/1253214552 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d2c0a7b50 con 0x7f9d2c159c70 2023-11-09T03:49:47.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.699+0000 7f9d21ffb640 1 -- 172.21.15.76:0/1253214552 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f9d2c0a7b50 con 0x7f9d2c159c70 2023-11-09T03:49:47.699 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:49:47.700 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":7773,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1663988060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1663988060},{"type":"v1","addr":"172.21.15.76:6835","nonce":1663988060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:49:45.047290+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5744,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5744":{"gid":5744,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6837/3280098776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3280098776},{"type":"v1","addr":"172.21.15.113:6837","nonce":3280098776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:49:47.700 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 57 2023-11-09T03:49:47.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.701+0000 7f9d33839640 1 -- 172.21.15.76:0/1253214552 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9d1006ae70 msgr2=0x7f9d1006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:47.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.701+0000 7f9d33839640 1 --2- 172.21.15.76:0/1253214552 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9d1006ae70 0x7f9d1006d330 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9d28006fd0 tx=0x7f9d28008040 comp rx=0 tx=0).stop 2023-11-09T03:49:47.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.701+0000 7f9d33839640 1 -- 172.21.15.76:0/1253214552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d2c159c70 msgr2=0x7f9d2c0b2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:47.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.701+0000 7f9d33839640 1 --2- 172.21.15.76:0/1253214552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d2c159c70 0x7f9d2c0b2960 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400cbc0 tx=0x7f9d2400a430 comp rx=0 tx=0).stop 2023-11-09T03:49:47.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.701+0000 7f9d33839640 1 -- 172.21.15.76:0/1253214552 shutdown_connections 2023-11-09T03:49:47.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.701+0000 7f9d33839640 1 --2- 172.21.15.76:0/1253214552 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9d1006ae70 0x7f9d1006d330 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:47.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.701+0000 7f9d33839640 1 --2- 172.21.15.76:0/1253214552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d2c159c70 0x7f9d2c0b2960 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:47.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.701+0000 7f9d33839640 1 --2- 172.21.15.76:0/1253214552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d2c0b0910 0x7f9d2c0b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:47.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.701+0000 7f9d33839640 1 --2- 172.21.15.76:0/1253214552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d2c06db20 0x7f9d2c1eee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:47.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.701+0000 7f9d33839640 1 -- 172.21.15.76:0/1253214552 >> 172.21.15.76:0/1253214552 conn(0x7f9d2c0ce950 msgr2=0x7f9d2c0b33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:47.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.701+0000 7f9d33839640 1 -- 172.21.15.76:0/1253214552 shutdown_connections 2023-11-09T03:49:47.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.701+0000 7f9d33839640 1 -- 172.21.15.76:0/1253214552 wait complete. 2023-11-09T03:49:47.711 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-09T03:49:47.712 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:49:47.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.812+0000 7f39e5610640 1 Processor -- start 2023-11-09T03:49:47.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.812+0000 7f39e5610640 1 -- start start 2023-11-09T03:49:47.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.813+0000 7f39e5610640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39e0159c70 0x7f39e00ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:47.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.813+0000 7f39e5610640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f39e006db20 0x7f39e00b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:47.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.813+0000 7f39e5610640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39e00b4110 con 0x7f39e00b0910 2023-11-09T03:49:47.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.813+0000 7f39e5610640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39e00b49b0 con 0x7f39e006db20 2023-11-09T03:49:47.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.813+0000 7f39e5610640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e00b0da0 con 0x7f39e0159c70 2023-11-09T03:49:47.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.813+0000 7f39de7fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f39e006db20 0x7f39e00b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:47.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.814+0000 7f39de7fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f39e006db20 0x7f39e00b03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47514/0 (socket says 172.21.15.76:47514) 2023-11-09T03:49:47.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.814+0000 7f39de7fc640 1 -- 172.21.15.76:0/669915807 learned_addr learned my addr 172.21.15.76:0/669915807 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:47.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.814+0000 7f39deffd640 1 --2- 172.21.15.76:0/669915807 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39e0159c70 0x7f39e00ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:47.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.814+0000 7f39de7fc640 1 -- 172.21.15.76:0/669915807 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39e0159c70 msgr2=0x7f39e00ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:47.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.814+0000 7f39de7fc640 1 --2- 172.21.15.76:0/669915807 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39e0159c70 0x7f39e00ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:47.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.814+0000 7f39de7fc640 1 -- 172.21.15.76:0/669915807 >> v1:172.21.15.113:6789/0 conn(0x7f39e00b0910 legacy=0x7f39e015d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:47.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.814+0000 7f39de7fc640 1 -- 172.21.15.76:0/669915807 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e00b43f0 con 0x7f39e006db20 2023-11-09T03:49:47.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.815+0000 7f39de7fc640 1 --2- 172.21.15.76:0/669915807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39e006db20 0x7f39e00b03d0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f39d400bb10 tx=0x7f39d402e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5321758526d06350 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:47.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.815+0000 7f39ddffb640 1 -- 172.21.15.76:0/669915807 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d4036440 con 0x7f39e006db20 2023-11-09T03:49:47.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.815+0000 7f39ddffb640 1 -- 172.21.15.76:0/669915807 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d40365a0 con 0x7f39e006db20 2023-11-09T03:49:47.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.815+0000 7f39ddffb640 1 -- 172.21.15.76:0/669915807 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d4036930 con 0x7f39e006db20 2023-11-09T03:49:47.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.815+0000 7f39e5610640 1 -- 172.21.15.76:0/669915807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39e006db20 msgr2=0x7f39e00b03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:47.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.815+0000 7f39e5610640 1 --2- 172.21.15.76:0/669915807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39e006db20 0x7f39e00b03d0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f39d400bb10 tx=0x7f39d402e150 comp rx=0 tx=0).stop 2023-11-09T03:49:47.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.815+0000 7f39e5610640 1 -- 172.21.15.76:0/669915807 shutdown_connections 2023-11-09T03:49:47.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.815+0000 7f39e5610640 1 --2- 172.21.15.76:0/669915807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39e006db20 0x7f39e00b03d0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:47.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.815+0000 7f39e5610640 1 --2- 172.21.15.76:0/669915807 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39e0159c70 0x7f39e00ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:47.820 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.815+0000 7f39e5610640 1 -- 172.21.15.76:0/669915807 >> 172.21.15.76:0/669915807 conn(0x7f39e00ce950 msgr2=0x7f39e00ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:47.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.815+0000 7f39e5610640 1 -- 172.21.15.76:0/669915807 shutdown_connections 2023-11-09T03:49:47.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.815+0000 7f39e5610640 1 -- 172.21.15.76:0/669915807 wait complete. 2023-11-09T03:49:47.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.817+0000 7f39e5610640 1 Processor -- start 2023-11-09T03:49:47.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.817+0000 7f39e5610640 1 -- start start 2023-11-09T03:49:47.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.817+0000 7f39e5610640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39e006db20 0x7f39e01557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:47.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.817+0000 7f39e5610640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39e00b0910 0x7f39e015c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:47.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.817+0000 7f39e5610640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39e0159c70 0x7f39e01ee9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:47.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.817+0000 7f39e5610640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e015b7b0 con 0x7f39e0159c70 2023-11-09T03:49:47.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.817+0000 7f39e5610640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e015ba90 con 0x7f39e006db20 2023-11-09T03:49:47.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.817+0000 7f39e5610640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e015bd70 con 0x7f39e00b0910 2023-11-09T03:49:47.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.817+0000 7f39df7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39e0159c70 0x7f39e01ee9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:47.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.818+0000 7f39df7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39e0159c70 0x7f39e01ee9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47526/0 (socket says 172.21.15.76:47526) 2023-11-09T03:49:47.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.818+0000 7f39df7fe640 1 -- 172.21.15.76:0/3512698004 learned_addr learned my addr 172.21.15.76:0/3512698004 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:47.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.818+0000 7f39de7fc640 1 --2- 172.21.15.76:0/3512698004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39e00b0910 0x7f39e015c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:47.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.818+0000 7f39deffd640 1 --2- 172.21.15.76:0/3512698004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39e006db20 0x7f39e01557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:47.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.818+0000 7f39df7fe640 1 -- 172.21.15.76:0/3512698004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39e00b0910 msgr2=0x7f39e015c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:47.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.818+0000 7f39df7fe640 1 --2- 172.21.15.76:0/3512698004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39e00b0910 0x7f39e015c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:47.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.818+0000 7f39df7fe640 1 -- 172.21.15.76:0/3512698004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39e006db20 msgr2=0x7f39e01557c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:47.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.818+0000 7f39df7fe640 1 --2- 172.21.15.76:0/3512698004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39e006db20 0x7f39e01557c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:47.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.818+0000 7f39df7fe640 1 -- 172.21.15.76:0/3512698004 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d400b770 con 0x7f39e0159c70 2023-11-09T03:49:47.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.818+0000 7f39df7fe640 1 --2- 172.21.15.76:0/3512698004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39e0159c70 0x7f39e01ee9d0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f39d8001eb0 tx=0x7f39d800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:47.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.818+0000 7f39c77fe640 1 -- 172.21.15.76:0/3512698004 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d80052d0 con 0x7f39e0159c70 2023-11-09T03:49:47.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.818+0000 7f39c77fe640 1 -- 172.21.15.76:0/3512698004 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d8005430 con 0x7f39e0159c70 2023-11-09T03:49:47.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.818+0000 7f39c77fe640 1 -- 172.21.15.76:0/3512698004 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d8003810 con 0x7f39e0159c70 2023-11-09T03:49:47.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.818+0000 7f39e5610640 1 -- 172.21.15.76:0/3512698004 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e015c050 con 0x7f39e0159c70 2023-11-09T03:49:47.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.818+0000 7f39e5610640 1 -- 172.21.15.76:0/3512698004 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e0155d00 con 0x7f39e0159c70 2023-11-09T03:49:47.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.819+0000 7f39e5610640 1 -- 172.21.15.76:0/3512698004 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39e015c570 con 0x7f39e0159c70 2023-11-09T03:49:47.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.820+0000 7f39c77fe640 1 -- 172.21.15.76:0/3512698004 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f39d8003970 con 0x7f39e0159c70 2023-11-09T03:49:47.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.820+0000 7f39c77fe640 1 --2- 172.21.15.76:0/3512698004 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f39cc06ae70 0x7f39cc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:47.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.820+0000 7f39c77fe640 1 -- 172.21.15.76:0/3512698004 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7f39d8089110 con 0x7f39e0159c70 2023-11-09T03:49:47.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.820+0000 7f39deffd640 1 --2- 172.21.15.76:0/3512698004 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f39cc06ae70 0x7f39cc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:47.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.821+0000 7f39deffd640 1 --2- 172.21.15.76:0/3512698004 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f39cc06ae70 0x7f39cc06d330 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f39e0070650 tx=0x7f39d00098d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:47.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:47.824+0000 7f39c77fe640 1 -- 172.21.15.76:0/3512698004 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f39e015c570 con 0x7f39e0159c70 2023-11-09T03:49:48.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.025+0000 7f39e5610640 1 -- 172.21.15.76:0/3512698004 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39e015c570 con 0x7f39e0159c70 2023-11-09T03:49:48.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.025+0000 7f39c77fe640 1 -- 172.21.15.76:0/3512698004 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f39e015c570 con 0x7f39e0159c70 2023-11-09T03:49:48.025 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:49:48.025 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":7773,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1663988060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1663988060},{"type":"v1","addr":"172.21.15.76:6835","nonce":1663988060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:49:45.047290+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5744,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5744":{"gid":5744,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6837/3280098776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3280098776},{"type":"v1","addr":"172.21.15.113:6837","nonce":3280098776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:49:48.025 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 57 2023-11-09T03:49:48.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.027+0000 7f39e5610640 1 -- 172.21.15.76:0/3512698004 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f39cc06ae70 msgr2=0x7f39cc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:48.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.027+0000 7f39e5610640 1 --2- 172.21.15.76:0/3512698004 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f39cc06ae70 0x7f39cc06d330 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f39e0070650 tx=0x7f39d00098d0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.027+0000 7f39e5610640 1 -- 172.21.15.76:0/3512698004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39e0159c70 msgr2=0x7f39e01ee9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:48.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.027+0000 7f39e5610640 1 --2- 172.21.15.76:0/3512698004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39e0159c70 0x7f39e01ee9d0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f39d8001eb0 tx=0x7f39d800c890 comp rx=0 tx=0).stop 2023-11-09T03:49:48.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.027+0000 7f39e5610640 1 -- 172.21.15.76:0/3512698004 shutdown_connections 2023-11-09T03:49:48.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.027+0000 7f39e5610640 1 --2- 172.21.15.76:0/3512698004 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f39cc06ae70 0x7f39cc06d330 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.027+0000 7f39e5610640 1 --2- 172.21.15.76:0/3512698004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39e0159c70 0x7f39e01ee9d0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.027+0000 7f39e5610640 1 --2- 172.21.15.76:0/3512698004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39e00b0910 0x7f39e015c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.027+0000 7f39e5610640 1 --2- 172.21.15.76:0/3512698004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39e006db20 0x7f39e01557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.027+0000 7f39e5610640 1 -- 172.21.15.76:0/3512698004 >> 172.21.15.76:0/3512698004 conn(0x7f39e00ce950 msgr2=0x7f39e00b1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:48.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.027+0000 7f39e5610640 1 -- 172.21.15.76:0/3512698004 shutdown_connections 2023-11-09T03:49:48.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.027+0000 7f39e5610640 1 -- 172.21.15.76:0/3512698004 wait complete. 2023-11-09T03:49:48.439 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-09T03:48:57.383848+0000', 'modified': '2023-11-09T03:49:45.047290+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 98, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 5744, 'mds_1': 7449}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5744': {'gid': 5744, 'name': 'b', 'rank': 0, 'incarnation': 53, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.113:6837/3280098776', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6836', 'nonce': 3280098776}, {'type': 'v1', 'addr': '172.21.15.113:6837', 'nonce': 3280098776}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_7449': {'gid': 7449, 'name': 'a', 'rank': 1, 'incarnation': 47, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.76:6833/3697850101', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6832', 'nonce': 3697850101}, {'type': 'v1', 'addr': '172.21.15.76:6833', 'nonce': 3697850101}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-09T03:49:48.440 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2023-11-09T03:49:48.440 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-09T03:49:48.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.542+0000 7fab6731d640 1 Processor -- start 2023-11-09T03:49:48.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.542+0000 7fab6731d640 1 -- start start 2023-11-09T03:49:48.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.543+0000 7fab6731d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab60159c00 0x7fab600cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.543+0000 7fab6731d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fab600c48e0 0x7fab600c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.543+0000 7fab6731d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab6015aff0 con 0x7fab600adac0 2023-11-09T03:49:48.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.543+0000 7fab6731d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab600c52a0 con 0x7fab600c48e0 2023-11-09T03:49:48.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.543+0000 7fab6731d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6015a3e0 con 0x7fab60159c00 2023-11-09T03:49:48.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.543+0000 7fab648b8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fab600c48e0 0x7fab600c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.543+0000 7fab648b8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fab600c48e0 0x7fab600c4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47542/0 (socket says 172.21.15.76:47542) 2023-11-09T03:49:48.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.543+0000 7fab648b8640 1 -- 172.21.15.76:0/4086030905 learned_addr learned my addr 172.21.15.76:0/4086030905 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:48.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.543+0000 7fab658ba640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fab600adac0 0x7fab600adef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46202/0 (socket says 172.21.15.76:46202) 2023-11-09T03:49:48.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.543+0000 7fab650b9640 1 --2- 172.21.15.76:0/4086030905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab60159c00 0x7fab600cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.543+0000 7fab648b8640 1 -- 172.21.15.76:0/4086030905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab60159c00 msgr2=0x7fab600cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:48.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.543+0000 7fab648b8640 1 --2- 172.21.15.76:0/4086030905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab60159c00 0x7fab600cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.543+0000 7fab648b8640 1 -- 172.21.15.76:0/4086030905 >> v1:172.21.15.76:6789/0 conn(0x7fab600adac0 legacy=0x7fab600adef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:48.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.543+0000 7fab648b8640 1 -- 172.21.15.76:0/4086030905 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6015a6c0 con 0x7fab600c48e0 2023-11-09T03:49:48.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.543+0000 7fab57fff640 1 -- 172.21.15.76:0/4086030905 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3751391417 0 0) 0x7fab6015aff0 con 0x7fab600adac0 2023-11-09T03:49:48.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.543+0000 7fab650b9640 1 --2- 172.21.15.76:0/4086030905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab60159c00 0x7fab600cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:49:48.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.544+0000 7fab648b8640 1 --2- 172.21.15.76:0/4086030905 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab600c48e0 0x7fab600c4d60 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fab50009870 tx=0x7fab5002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d586d64bb8cbb23 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:48.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.544+0000 7fab57fff640 1 -- 172.21.15.76:0/4086030905 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab50030840 con 0x7fab600c48e0 2023-11-09T03:49:48.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.544+0000 7fab57fff640 1 -- 172.21.15.76:0/4086030905 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab500309a0 con 0x7fab600c48e0 2023-11-09T03:49:48.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.544+0000 7fab57fff640 1 -- 172.21.15.76:0/4086030905 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab50030d30 con 0x7fab600c48e0 2023-11-09T03:49:48.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.545+0000 7fab6731d640 1 -- 172.21.15.76:0/4086030905 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab600c48e0 msgr2=0x7fab600c4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:48.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.545+0000 7fab6731d640 1 --2- 172.21.15.76:0/4086030905 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab600c48e0 0x7fab600c4d60 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fab50009870 tx=0x7fab5002b980 comp rx=0 tx=0).stop 2023-11-09T03:49:48.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.545+0000 7fab6731d640 1 -- 172.21.15.76:0/4086030905 shutdown_connections 2023-11-09T03:49:48.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.545+0000 7fab6731d640 1 --2- 172.21.15.76:0/4086030905 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab600c48e0 0x7fab600c4d60 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.545+0000 7fab6731d640 1 --2- 172.21.15.76:0/4086030905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab60159c00 0x7fab600cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.545+0000 7fab6731d640 1 -- 172.21.15.76:0/4086030905 >> 172.21.15.76:0/4086030905 conn(0x7fab600ce950 msgr2=0x7fab600ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:48.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.545+0000 7fab6731d640 1 -- 172.21.15.76:0/4086030905 shutdown_connections 2023-11-09T03:49:48.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.545+0000 7fab6731d640 1 -- 172.21.15.76:0/4086030905 wait complete. 2023-11-09T03:49:48.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.546+0000 7fab6731d640 1 Processor -- start 2023-11-09T03:49:48.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.546+0000 7fab6731d640 1 -- start start 2023-11-09T03:49:48.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.547+0000 7fab6731d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab600adac0 0x7fab6014fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.547+0000 7fab6731d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab600c48e0 0x7fab601414a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.547+0000 7fab6731d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab60159c00 0x7fab6014dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.547+0000 7fab6731d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6014f570 con 0x7fab600adac0 2023-11-09T03:49:48.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.547+0000 7fab6731d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6014e960 con 0x7fab600c48e0 2023-11-09T03:49:48.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.547+0000 7fab6731d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6014ec40 con 0x7fab60159c00 2023-11-09T03:49:48.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.547+0000 7fab650b9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab600adac0 0x7fab6014fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.547+0000 7fab650b9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab600adac0 0x7fab6014fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47544/0 (socket says 172.21.15.76:47544) 2023-11-09T03:49:48.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.547+0000 7fab650b9640 1 -- 172.21.15.76:0/3015334063 learned_addr learned my addr 172.21.15.76:0/3015334063 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:48.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.547+0000 7fab658ba640 1 --2- 172.21.15.76:0/3015334063 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab60159c00 0x7fab6014dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.547+0000 7fab648b8640 1 --2- 172.21.15.76:0/3015334063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab600c48e0 0x7fab601414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.547+0000 7fab650b9640 1 -- 172.21.15.76:0/3015334063 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab60159c00 msgr2=0x7fab6014dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:48.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.547+0000 7fab650b9640 1 --2- 172.21.15.76:0/3015334063 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab60159c00 0x7fab6014dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.547+0000 7fab650b9640 1 -- 172.21.15.76:0/3015334063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab600c48e0 msgr2=0x7fab601414a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:48.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.547+0000 7fab650b9640 1 --2- 172.21.15.76:0/3015334063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab600c48e0 0x7fab601414a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.547+0000 7fab650b9640 1 -- 172.21.15.76:0/3015334063 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab500094d0 con 0x7fab600adac0 2023-11-09T03:49:48.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.547+0000 7fab650b9640 1 --2- 172.21.15.76:0/3015334063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab600adac0 0x7fab6014fa00 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00c7f0 tx=0x7fab5c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:48.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.547+0000 7fab55ffb640 1 -- 172.21.15.76:0/3015334063 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c007a30 con 0x7fab600adac0 2023-11-09T03:49:48.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.547+0000 7fab55ffb640 1 -- 172.21.15.76:0/3015334063 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5c007b90 con 0x7fab600adac0 2023-11-09T03:49:48.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.547+0000 7fab6731d640 1 -- 172.21.15.76:0/3015334063 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab6014ef20 con 0x7fab600adac0 2023-11-09T03:49:48.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.547+0000 7fab55ffb640 1 -- 172.21.15.76:0/3015334063 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c00ce40 con 0x7fab600adac0 2023-11-09T03:49:48.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.548+0000 7fab6731d640 1 -- 172.21.15.76:0/3015334063 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6014d8d0 con 0x7fab600adac0 2023-11-09T03:49:48.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.549+0000 7fab55ffb640 1 -- 172.21.15.76:0/3015334063 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fab5c0113f0 con 0x7fab600adac0 2023-11-09T03:49:48.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.549+0000 7fab55ffb640 1 --2- 172.21.15.76:0/3015334063 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fab4406ae70 0x7fab4406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.549+0000 7fab55ffb640 1 -- 172.21.15.76:0/3015334063 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7fab5c089ed0 con 0x7fab600adac0 2023-11-09T03:49:48.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.550+0000 7fab648b8640 1 --2- 172.21.15.76:0/3015334063 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fab4406ae70 0x7fab4406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.550+0000 7fab648b8640 1 --2- 172.21.15.76:0/3015334063 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fab4406ae70 0x7fab4406d330 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fab50036000 tx=0x7fab50008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:48.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.551+0000 7fab69f86740 1 Processor -- start 2023-11-09T03:49:48.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.551+0000 7fab69f86740 1 -- start start 2023-11-09T03:49:48.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.551+0000 7fab69f86740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565788e57d0 0x556578901090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.551+0000 7fab69f86740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5565788e5bb0 0x5565788e6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.551+0000 7fab69f86740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5565788ffa90 0x5565788fff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.551+0000 7fab69f86740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565787c6740 con 0x5565788e57d0 2023-11-09T03:49:48.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.551+0000 7fab69f86740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565788288d0 con 0x5565788ffa90 2023-11-09T03:49:48.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.551+0000 7fab69f86740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x556578825fc0 con 0x5565788e5bb0 2023-11-09T03:49:48.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.551+0000 7fab650b9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565788e57d0 0x556578901090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.551+0000 7fab650b9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565788e57d0 0x556578901090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47560/0 (socket says 172.21.15.76:47560) 2023-11-09T03:49:48.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.551+0000 7fab650b9640 1 -- 172.21.15.76:0/3327779843 learned_addr learned my addr 172.21.15.76:0/3327779843 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:48.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.552+0000 7fab658ba640 1 --2- 172.21.15.76:0/3327779843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5565788ffa90 0x5565788fff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.552+0000 7fab648b8640 1 --2- 172.21.15.76:0/3327779843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5565788e5bb0 0x5565788e6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.552+0000 7fab650b9640 1 -- 172.21.15.76:0/3327779843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5565788e5bb0 msgr2=0x5565788e6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:48.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.552+0000 7fab650b9640 1 --2- 172.21.15.76:0/3327779843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5565788e5bb0 0x5565788e6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.552+0000 7fab650b9640 1 -- 172.21.15.76:0/3327779843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5565788ffa90 msgr2=0x5565788fff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:48.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.552+0000 7fab650b9640 1 --2- 172.21.15.76:0/3327779843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5565788ffa90 0x5565788fff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.552+0000 7fab650b9640 1 -- 172.21.15.76:0/3327779843 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565787e4ca0 con 0x5565788e57d0 2023-11-09T03:49:48.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.552+0000 7fab650b9640 1 --2- 172.21.15.76:0/3327779843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565788e57d0 0x556578901090 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fab5c087cb0 tx=0x7fab5c058d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b5f79dc6512b95 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:48.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.552+0000 7fab4affd640 1 -- 172.21.15.76:0/3327779843 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c0608b0 con 0x5565788e57d0 2023-11-09T03:49:48.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.552+0000 7fab4affd640 1 -- 172.21.15.76:0/3327779843 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5c056660 con 0x5565788e57d0 2023-11-09T03:49:48.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.552+0000 7fab4affd640 1 -- 172.21.15.76:0/3327779843 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c054c30 con 0x5565788e57d0 2023-11-09T03:49:48.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.553+0000 7fab69f86740 1 -- 172.21.15.76:0/3327779843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565788e57d0 msgr2=0x556578901090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:48.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.553+0000 7fab69f86740 1 --2- 172.21.15.76:0/3327779843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565788e57d0 0x556578901090 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fab5c087cb0 tx=0x7fab5c058d60 comp rx=0 tx=0).stop 2023-11-09T03:49:48.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.553+0000 7fab69f86740 1 -- 172.21.15.76:0/3327779843 shutdown_connections 2023-11-09T03:49:48.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.553+0000 7fab69f86740 1 --2- 172.21.15.76:0/3327779843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5565788ffa90 0x5565788fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.553+0000 7fab69f86740 1 --2- 172.21.15.76:0/3327779843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5565788e5bb0 0x5565788e6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.553+0000 7fab69f86740 1 --2- 172.21.15.76:0/3327779843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565788e57d0 0x556578901090 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.553+0000 7fab69f86740 1 -- 172.21.15.76:0/3327779843 >> 172.21.15.76:0/3327779843 conn(0x5565788b7a60 msgr2=0x5565789005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:48.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.553+0000 7fab69f86740 1 -- 172.21.15.76:0/3327779843 shutdown_connections 2023-11-09T03:49:48.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.553+0000 7fab69f86740 1 -- 172.21.15.76:0/3327779843 wait complete. 2023-11-09T03:49:48.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.553+0000 7fab69f86740 1 -- start start 2023-11-09T03:49:48.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.553+0000 7fab69f86740 1 Processor -- start 2023-11-09T03:49:48.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.554+0000 7fab69f86740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5565788e5bb0 0x5565788df0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.554+0000 7fab69f86740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5565788ffa90 0x55657890a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.554+0000 7fab69f86740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565788e65e0 0x5565788ddcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.554+0000 7fab69f86740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565788218b0 con 0x5565788e65e0 2023-11-09T03:49:48.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.554+0000 7fab69f86740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565787dbc80 con 0x5565788e5bb0 2023-11-09T03:49:48.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.554+0000 7fab69f86740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565787db920 con 0x5565788ffa90 2023-11-09T03:49:48.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.554+0000 7fab69f86740 10 client.0 _ll_register_callbacks cb 0x556578533ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:49:48.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.554+0000 7fab658ba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565788e65e0 0x5565788ddcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.554+0000 7fab658ba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565788e65e0 0x5565788ddcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47570/0 (socket says 172.21.15.76:47570) 2023-11-09T03:49:48.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.554+0000 7fab658ba640 1 -- 172.21.15.76:0/279328051 learned_addr learned my addr 172.21.15.76:0/279328051 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:48.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.554+0000 7fab648b8640 1 --2- 172.21.15.76:0/279328051 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5565788ffa90 0x55657890a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.554+0000 7fab650b9640 1 --2- 172.21.15.76:0/279328051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5565788e5bb0 0x5565788df0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.555+0000 7fab658ba640 1 -- 172.21.15.76:0/279328051 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5565788ffa90 msgr2=0x55657890a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:48.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.555+0000 7fab658ba640 1 --2- 172.21.15.76:0/279328051 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5565788ffa90 0x55657890a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.555+0000 7fab658ba640 1 -- 172.21.15.76:0/279328051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5565788e5bb0 msgr2=0x5565788df0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:48.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.555+0000 7fab658ba640 1 --2- 172.21.15.76:0/279328051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5565788e5bb0 0x5565788df0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.555+0000 7fab658ba640 1 -- 172.21.15.76:0/279328051 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6014d8d0 con 0x5565788e65e0 2023-11-09T03:49:48.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.555+0000 7fab650b9640 1 --2- 172.21.15.76:0/279328051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5565788e5bb0 0x5565788df0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:48.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.555+0000 7fab658ba640 1 --2- 172.21.15.76:0/279328051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565788e65e0 0x5565788ddcf0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fab58011530 tx=0x7fab580119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a59bc7b0965d66c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:48.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.555+0000 7fab3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:49:48.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.555+0000 7fab3ffff640 1 -- 172.21.15.76:0/279328051 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab58015680 con 0x5565788e65e0 2023-11-09T03:49:48.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.555+0000 7fab69f86740 1 -- 172.21.15.76:0/279328051 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565787cdbf0 con 0x5565788e65e0 2023-11-09T03:49:48.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.555+0000 7fab3ffff640 1 -- 172.21.15.76:0/279328051 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7fab580157e0 con 0x5565788e65e0 2023-11-09T03:49:48.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.555+0000 7fab3ffff640 1 -- 172.21.15.76:0/279328051 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab580159a0 con 0x5565788e65e0 2023-11-09T03:49:48.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.555+0000 7fab69f86740 10 client.7812 fetch_fsmap learned FSMap version 57 2023-11-09T03:49:48.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.555+0000 7fab69f86740 1 -- 172.21.15.76:0/279328051 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55657881ff30 con 0x5565788e65e0 2023-11-09T03:49:48.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.555+0000 7fab3ffff640 1 -- 172.21.15.76:0/279328051 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5565787cdbf0 con 0x5565788e65e0 2023-11-09T03:49:48.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.555+0000 7fab3ffff640 1 -- 172.21.15.76:0/279328051 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 57) v1 ==== 3306+0+0 (secure 0 0 0) 0x7fab58015b00 con 0x5565788e65e0 2023-11-09T03:49:48.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.555+0000 7fab69f86740 10 client.7812 fetch_fsmap finished waiting for FSMap version 57 2023-11-09T03:49:48.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.555+0000 7fab69f86740 10 client.7812 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-09T03:49:48.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.555+0000 7fab69f86740 20 client.7812 populate_metadata read hostname 'smithi076' 2023-11-09T03:49:48.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.555+0000 7fab69f86740 1 --2- 172.21.15.76:0/279328051 >> [v2:172.21.15.113:6836/3280098776,v1:172.21.15.113:6837/3280098776] conn(0x5565788e9000 0x5565789163e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.555+0000 7fab69f86740 4 client.7812 mds_command: new command op to 5744 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:49:48.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.555+0000 7fab69f86740 1 -- 172.21.15.76:0/279328051 --> [v2:172.21.15.113:6836/3280098776,v1:172.21.15.113:6837/3280098776] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55657881a890 con 0x5565788e9000 2023-11-09T03:49:48.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.556+0000 7fab650b9640 1 --2- 172.21.15.76:0/279328051 >> [v2:172.21.15.113:6836/3280098776,v1:172.21.15.113:6837/3280098776] conn(0x5565788e9000 0x5565789163e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.556+0000 7fab650b9640 1 --2- 172.21.15.76:0/279328051 >> [v2:172.21.15.113:6836/3280098776,v1:172.21.15.113:6837/3280098776] conn(0x5565788e9000 0x5565789163e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=490751d25afd6e35 server_cookie=4e2a35cc31bc47be in_seq=0 out_seq=0 2023-11-09T03:49:48.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.556+0000 7fab3ffff640 10 client.7812 ms_handle_connect on v2:172.21.15.113:6836/3280098776 2023-11-09T03:49:48.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.557+0000 7fab3ffff640 1 -- 172.21.15.76:0/279328051 <== mds.0 v2:172.21.15.113:6836/3280098776 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55657881a890 con 0x5565788e9000 2023-11-09T03:49:48.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.557+0000 7fab3ffff640 10 client.7812 handle_command_reply: tid=0 2023-11-09T03:49:48.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.557+0000 7fab69f86740 1 client.7812 shutdown 2023-11-09T03:49:48.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.557+0000 7fab69f86740 1 -- 172.21.15.76:0/279328051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565788e65e0 msgr2=0x5565788ddcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:48.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.557+0000 7fab69f86740 1 --2- 172.21.15.76:0/279328051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565788e65e0 0x5565788ddcf0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fab58011530 tx=0x7fab580119f0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.557+0000 7fab69f86740 1 -- 172.21.15.76:0/279328051 shutdown_connections 2023-11-09T03:49:48.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.557+0000 7fab69f86740 1 --2- 172.21.15.76:0/279328051 >> [v2:172.21.15.113:6836/3280098776,v1:172.21.15.113:6837/3280098776] conn(0x5565788e9000 0x5565789163e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.557+0000 7fab69f86740 1 --2- 172.21.15.76:0/279328051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565788e65e0 0x5565788ddcf0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.557+0000 7fab69f86740 1 --2- 172.21.15.76:0/279328051 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5565788ffa90 0x55657890a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.557+0000 7fab3ffff640 0 client.7812 ms_handle_reset on v2:172.21.15.113:6836/3280098776 2023-11-09T03:49:48.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.557+0000 7fab69f86740 1 --2- 172.21.15.76:0/279328051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5565788e5bb0 0x5565788df0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.557+0000 7fab69f86740 1 -- 172.21.15.76:0/279328051 >> 172.21.15.76:0/279328051 conn(0x5565788e57d0 msgr2=0x556578903940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:48.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.557+0000 7fab69f86740 1 -- 172.21.15.76:0/279328051 shutdown_connections 2023-11-09T03:49:48.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.557+0000 7fab69f86740 1 -- 172.21.15.76:0/279328051 wait complete. 2023-11-09T03:49:48.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.557+0000 7fab69f86740 20 client.7812 trim_cache size 0 max 16384 2023-11-09T03:49:48.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.572+0000 7fab69f86740 1 Processor -- start 2023-11-09T03:49:48.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.572+0000 7fab69f86740 1 -- start start 2023-11-09T03:49:48.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.573+0000 7fab69f86740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5565788e5bb0 0x5565788fda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.573+0000 7fab69f86740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565788e65e0 0x556578909d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.573+0000 7fab69f86740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5565788ffa90 0x55657890a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.573+0000 7fab69f86740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565787a1280 con 0x5565788e65e0 2023-11-09T03:49:48.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.573+0000 7fab69f86740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565788fd580 con 0x5565788e5bb0 2023-11-09T03:49:48.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.573+0000 7fab69f86740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565788dc830 con 0x5565788ffa90 2023-11-09T03:49:48.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.573+0000 7fab648b8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565788e65e0 0x556578909d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.573+0000 7fab648b8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565788e65e0 0x556578909d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47586/0 (socket says 172.21.15.76:47586) 2023-11-09T03:49:48.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.573+0000 7fab648b8640 1 -- 172.21.15.76:0/3726578065 learned_addr learned my addr 172.21.15.76:0/3726578065 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:48.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.574+0000 7fab648b8640 1 -- 172.21.15.76:0/3726578065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5565788ffa90 msgr2=0x55657890a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:48.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.574+0000 7fab648b8640 1 --2- 172.21.15.76:0/3726578065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5565788ffa90 0x55657890a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.574+0000 7fab648b8640 1 -- 172.21.15.76:0/3726578065 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5565788e5bb0 msgr2=0x5565788fda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:48.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.574+0000 7fab648b8640 1 --2- 172.21.15.76:0/3726578065 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5565788e5bb0 0x5565788fda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.574+0000 7fab648b8640 1 -- 172.21.15.76:0/3726578065 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565787bdd10 con 0x5565788e65e0 2023-11-09T03:49:48.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.574+0000 7fab648b8640 1 --2- 172.21.15.76:0/3726578065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565788e65e0 0x556578909d80 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fab500308d0 tx=0x7fab50009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2f9012c679be15 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:48.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.574+0000 7fab4b7fe640 1 -- 172.21.15.76:0/3726578065 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5003e040 con 0x5565788e65e0 2023-11-09T03:49:48.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.574+0000 7fab4b7fe640 1 -- 172.21.15.76:0/3726578065 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5004b040 con 0x5565788e65e0 2023-11-09T03:49:48.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.574+0000 7fab4b7fe640 1 -- 172.21.15.76:0/3726578065 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab50003ea0 con 0x5565788e65e0 2023-11-09T03:49:48.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.574+0000 7fab69f86740 1 -- 172.21.15.76:0/3726578065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565788e65e0 msgr2=0x556578909d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:48.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.574+0000 7fab69f86740 1 --2- 172.21.15.76:0/3726578065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565788e65e0 0x556578909d80 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fab500308d0 tx=0x7fab50009240 comp rx=0 tx=0).stop 2023-11-09T03:49:48.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.574+0000 7fab69f86740 1 -- 172.21.15.76:0/3726578065 shutdown_connections 2023-11-09T03:49:48.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.574+0000 7fab69f86740 1 --2- 172.21.15.76:0/3726578065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5565788ffa90 0x55657890a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.574+0000 7fab69f86740 1 --2- 172.21.15.76:0/3726578065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565788e65e0 0x556578909d80 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.574+0000 7fab69f86740 1 --2- 172.21.15.76:0/3726578065 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5565788e5bb0 0x5565788fda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.574+0000 7fab69f86740 1 -- 172.21.15.76:0/3726578065 >> 172.21.15.76:0/3726578065 conn(0x5565788e57d0 msgr2=0x5565788e0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:48.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.574+0000 7fab69f86740 1 -- 172.21.15.76:0/3726578065 shutdown_connections 2023-11-09T03:49:48.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.574+0000 7fab69f86740 1 -- 172.21.15.76:0/3726578065 wait complete. 2023-11-09T03:49:48.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.575+0000 7fab69f86740 1 -- start start 2023-11-09T03:49:48.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.575+0000 7fab69f86740 1 Processor -- start 2023-11-09T03:49:48.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.576+0000 7fab69f86740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565788e5bb0 0x556578961990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.576+0000 7fab69f86740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5565788e65e0 0x556578961ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.576+0000 7fab69f86740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5565788ffa90 0x5565788b9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.576+0000 7fab69f86740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565788fda10 con 0x5565788e5bb0 2023-11-09T03:49:48.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.576+0000 7fab69f86740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55657896edc0 con 0x5565788e65e0 2023-11-09T03:49:48.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.576+0000 7fab69f86740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565788e9a10 con 0x5565788ffa90 2023-11-09T03:49:48.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.576+0000 7fab69f86740 10 client.0 _ll_register_callbacks cb 0x7fab5c0548e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:49:48.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.576+0000 7fab650b9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565788e5bb0 0x556578961990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.576+0000 7fab650b9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565788e5bb0 0x556578961990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47590/0 (socket says 172.21.15.76:47590) 2023-11-09T03:49:48.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.576+0000 7fab650b9640 1 -- 172.21.15.76:0/1202093131 learned_addr learned my addr 172.21.15.76:0/1202093131 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:48.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.576+0000 7fab648b8640 1 --2- 172.21.15.76:0/1202093131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5565788e65e0 0x556578961ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.576+0000 7fab658ba640 1 --2- 172.21.15.76:0/1202093131 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5565788ffa90 0x5565788b9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.576+0000 7fab650b9640 1 -- 172.21.15.76:0/1202093131 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5565788ffa90 msgr2=0x5565788b9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:48.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.576+0000 7fab650b9640 1 --2- 172.21.15.76:0/1202093131 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5565788ffa90 0x5565788b9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.576+0000 7fab650b9640 1 -- 172.21.15.76:0/1202093131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5565788e65e0 msgr2=0x556578961ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:48.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.576+0000 7fab650b9640 1 --2- 172.21.15.76:0/1202093131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5565788e65e0 0x556578961ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.576+0000 7fab650b9640 1 -- 172.21.15.76:0/1202093131 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6015a6c0 con 0x5565788e5bb0 2023-11-09T03:49:48.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.576+0000 7fab650b9640 1 --2- 172.21.15.76:0/1202093131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565788e5bb0 0x556578961990 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fab5c0586e0 tx=0x7fab5c087c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45bf8a932d86a49 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:48.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.576+0000 7fab3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:49:48.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.577+0000 7fab3ffff640 1 -- 172.21.15.76:0/1202093131 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c060620 con 0x5565788e5bb0 2023-11-09T03:49:48.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.577+0000 7fab3ffff640 1 -- 172.21.15.76:0/1202093131 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7fab5c089040 con 0x5565788e5bb0 2023-11-09T03:49:48.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.577+0000 7fab69f86740 1 -- 172.21.15.76:0/1202093131 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab5c056660 con 0x5565788e5bb0 2023-11-09T03:49:48.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.577+0000 7fab3ffff640 1 -- 172.21.15.76:0/1202093131 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab5c058060 con 0x5565788e5bb0 2023-11-09T03:49:48.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.577+0000 7fab69f86740 10 client.7818 fetch_fsmap learned FSMap version 57 2023-11-09T03:49:48.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.577+0000 7fab69f86740 1 -- 172.21.15.76:0/1202093131 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x5565787e3500 con 0x5565788e5bb0 2023-11-09T03:49:48.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.577+0000 7fab3ffff640 1 -- 172.21.15.76:0/1202093131 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5c056660 con 0x5565788e5bb0 2023-11-09T03:49:48.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.577+0000 7fab3ffff640 1 -- 172.21.15.76:0/1202093131 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 57) v1 ==== 3306+0+0 (secure 0 0 0) 0x7fab5c079020 con 0x5565788e5bb0 2023-11-09T03:49:48.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.577+0000 7fab69f86740 10 client.7818 fetch_fsmap finished waiting for FSMap version 57 2023-11-09T03:49:48.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.577+0000 7fab69f86740 10 client.7818 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-09T03:49:48.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.577+0000 7fab69f86740 20 client.7818 populate_metadata read hostname 'smithi076' 2023-11-09T03:49:48.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.577+0000 7fab69f86740 1 --2- 172.21.15.76:0/1202093131 >> [v2:172.21.15.113:6836/3280098776,v1:172.21.15.113:6837/3280098776] conn(0x5565788bb470 0x5565788bb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.577+0000 7fab69f86740 4 client.7818 mds_command: new command op to 5744 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-09T03:49:48.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.577+0000 7fab69f86740 1 -- 172.21.15.76:0/1202093131 --> [v2:172.21.15.113:6836/3280098776,v1:172.21.15.113:6837/3280098776] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55657881a5b0 con 0x5565788bb470 2023-11-09T03:49:48.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.577+0000 7fab648b8640 1 --2- 172.21.15.76:0/1202093131 >> [v2:172.21.15.113:6836/3280098776,v1:172.21.15.113:6837/3280098776] conn(0x5565788bb470 0x5565788bb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.578+0000 7fab648b8640 1 --2- 172.21.15.76:0/1202093131 >> [v2:172.21.15.113:6836/3280098776,v1:172.21.15.113:6837/3280098776] conn(0x5565788bb470 0x5565788bb850 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d61c47f8dd04a517 server_cookie=f3fd5de92e4ff1c in_seq=0 out_seq=0 2023-11-09T03:49:48.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.578+0000 7fab3ffff640 10 client.7818 ms_handle_connect on v2:172.21.15.113:6836/3280098776 2023-11-09T03:49:48.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.578+0000 7fab3ffff640 1 -- 172.21.15.76:0/1202093131 <== mds.0 v2:172.21.15.113:6836/3280098776 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55657881a5b0 con 0x5565788bb470 2023-11-09T03:49:48.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.578+0000 7fab3ffff640 10 client.7818 handle_command_reply: tid=0 2023-11-09T03:49:48.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.578+0000 7fab69f86740 1 client.7818 shutdown 2023-11-09T03:49:48.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.578+0000 7fab69f86740 1 -- 172.21.15.76:0/1202093131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565788e5bb0 msgr2=0x556578961990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:48.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.578+0000 7fab69f86740 1 --2- 172.21.15.76:0/1202093131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565788e5bb0 0x556578961990 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fab5c0586e0 tx=0x7fab5c087c40 comp rx=0 tx=0).stop 2023-11-09T03:49:48.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.578+0000 7fab69f86740 1 -- 172.21.15.76:0/1202093131 shutdown_connections 2023-11-09T03:49:48.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.578+0000 7fab69f86740 1 --2- 172.21.15.76:0/1202093131 >> [v2:172.21.15.113:6836/3280098776,v1:172.21.15.113:6837/3280098776] conn(0x5565788bb470 0x5565788bb850 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.578+0000 7fab69f86740 1 --2- 172.21.15.76:0/1202093131 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5565788ffa90 0x5565788b9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.578+0000 7fab3ffff640 0 client.7818 ms_handle_reset on v2:172.21.15.113:6836/3280098776 2023-11-09T03:49:48.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.578+0000 7fab69f86740 1 --2- 172.21.15.76:0/1202093131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5565788e65e0 0x556578961ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.578+0000 7fab69f86740 1 --2- 172.21.15.76:0/1202093131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5565788e5bb0 0x556578961990 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.578+0000 7fab69f86740 1 -- 172.21.15.76:0/1202093131 >> 172.21.15.76:0/1202093131 conn(0x5565788e57d0 msgr2=0x5565788e23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:48.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.578+0000 7fab69f86740 1 -- 172.21.15.76:0/1202093131 shutdown_connections 2023-11-09T03:49:48.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.578+0000 7fab69f86740 1 -- 172.21.15.76:0/1202093131 wait complete. 2023-11-09T03:49:48.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.578+0000 7fab69f86740 20 client.7818 trim_cache size 0 max 16384 2023-11-09T03:49:48.598 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:49:48.598 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "c213b26f-d0da-45d5-8e34-bb75c096d4f2", 2023-11-09T03:49:48.598 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2023-11-09T03:49:48.598 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 5744, 2023-11-09T03:49:48.598 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2023-11-09T03:49:48.598 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2023-11-09T03:49:48.599 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2023-11-09T03:49:48.599 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 6.5382653309999998, 2023-11-09T03:49:48.599 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 57, 2023-11-09T03:49:48.599 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 98, 2023-11-09T03:49:48.599 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 98, 2023-11-09T03:49:48.600 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 56.565172914999998 2023-11-09T03:49:48.600 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:49:48.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.579+0000 7fab6731d640 1 -- 172.21.15.76:0/3015334063 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fab4406ae70 msgr2=0x7fab4406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:48.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.579+0000 7fab6731d640 1 --2- 172.21.15.76:0/3015334063 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fab4406ae70 0x7fab4406d330 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fab50036000 tx=0x7fab50008030 comp rx=0 tx=0).stop 2023-11-09T03:49:48.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.579+0000 7fab6731d640 1 -- 172.21.15.76:0/3015334063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab600adac0 msgr2=0x7fab6014fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:48.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.579+0000 7fab6731d640 1 --2- 172.21.15.76:0/3015334063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab600adac0 0x7fab6014fa00 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00c7f0 tx=0x7fab5c00cbf0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.579+0000 7fab6731d640 1 -- 172.21.15.76:0/3015334063 shutdown_connections 2023-11-09T03:49:48.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.579+0000 7fab6731d640 1 --2- 172.21.15.76:0/3015334063 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fab4406ae70 0x7fab4406d330 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.579+0000 7fab6731d640 1 --2- 172.21.15.76:0/3015334063 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab60159c00 0x7fab6014dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.579+0000 7fab6731d640 1 --2- 172.21.15.76:0/3015334063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab600c48e0 0x7fab601414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.579+0000 7fab6731d640 1 --2- 172.21.15.76:0/3015334063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab600adac0 0x7fab6014fa00 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.579+0000 7fab6731d640 1 -- 172.21.15.76:0/3015334063 >> 172.21.15.76:0/3015334063 conn(0x7fab600ce950 msgr2=0x7fab600709f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:48.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.579+0000 7fab6731d640 1 -- 172.21.15.76:0/3015334063 shutdown_connections 2023-11-09T03:49:48.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.579+0000 7fab6731d640 1 -- 172.21.15.76:0/3015334063 wait complete. 2023-11-09T03:49:48.603 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-09T03:49:48.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.704+0000 7f8005b33640 1 Processor -- start 2023-11-09T03:49:48.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.705+0000 7f8005b33640 1 -- start start 2023-11-09T03:49:48.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.705+0000 7f8005b33640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f80000cd6d0 0x7f8000161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.705+0000 7f8005b33640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80000cde60 con 0x7f80000b4880 2023-11-09T03:49:48.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.705+0000 7f8005b33640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8000160f90 con 0x7f8000153e60 2023-11-09T03:49:48.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.705+0000 7f8005b33640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8000161960 con 0x7f80000cd6d0 2023-11-09T03:49:48.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.705+0000 7f7ffffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f80000b4880 0x7f80000d0fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46212/0 (socket says 172.21.15.76:46212) 2023-11-09T03:49:48.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.705+0000 7f7ffffff640 1 -- 172.21.15.76:0/3412296935 learned_addr learned my addr 172.21.15.76:0/3412296935 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:49:48.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.706+0000 7f7ffeffd640 1 --2- 172.21.15.76:0/3412296935 >> v2:172.21.15.113:3300/0 conn(0x7f80000cd6d0 0x7f8000161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.706+0000 7f7ffe7fc640 1 -- 172.21.15.76:0/3412296935 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1691613910 0 0) 0x7f80000cde60 con 0x7f80000b4880 2023-11-09T03:49:48.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.706+0000 7f7ffe7fc640 1 -- 172.21.15.76:0/3412296935 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fe8003520 con 0x7f80000b4880 2023-11-09T03:49:48.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.706+0000 7f7ffe7fc640 1 -- 172.21.15.76:0/3412296935 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2015230512 0 0) 0x7f8000160f90 con 0x7f8000153e60 2023-11-09T03:49:48.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.706+0000 7f7ffe7fc640 1 -- 172.21.15.76:0/3412296935 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80000cde60 con 0x7f8000153e60 2023-11-09T03:49:48.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.706+0000 7f7ffe7fc640 1 -- 172.21.15.76:0/3412296935 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2013977721 0 0) 0x7f7fe8003520 con 0x7f80000b4880 2023-11-09T03:49:48.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.706+0000 7f7ffe7fc640 1 -- 172.21.15.76:0/3412296935 >> v2:172.21.15.113:3300/0 conn(0x7f80000cd6d0 msgr2=0x7f8000161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:48.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.706+0000 7f7ffe7fc640 1 --2- 172.21.15.76:0/3412296935 >> v2:172.21.15.113:3300/0 conn(0x7f80000cd6d0 0x7f8000161420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.706+0000 7f7ffe7fc640 1 -- 172.21.15.76:0/3412296935 >> v1:172.21.15.113:6789/0 conn(0x7f8000153e60 legacy=0x7f800015dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:48.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.706+0000 7f7ffe7fc640 1 -- 172.21.15.76:0/3412296935 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80000ce090 con 0x7f80000b4880 2023-11-09T03:49:48.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.706+0000 7f7ffe7fc640 1 -- 172.21.15.76:0/3412296935 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f7ff80019f0 con 0x7f80000b4880 2023-11-09T03:49:48.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.706+0000 7f7ffe7fc640 1 -- 172.21.15.76:0/3412296935 >> v1:172.21.15.76:6789/0 conn(0x7f80000b4880 legacy=0x7f80000d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:48.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.706+0000 7f7ffe7fc640 1 --2- 172.21.15.76:0/3412296935 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fe8003b00 0x7f7fe8003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.706+0000 7f7ffe7fc640 1 --2- 172.21.15.76:0/3412296935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fe80047c0 0x7f7fe8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.706+0000 7f7ffe7fc640 1 --2- 172.21.15.76:0/3412296935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fe8005180 0x7f7fe800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.706+0000 7f7ffe7fc640 1 -- 172.21.15.76:0/3412296935 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8000161960 con 0x7f7fe8005180 2023-11-09T03:49:48.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.706+0000 7f7ffe7fc640 1 -- 172.21.15.76:0/3412296935 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe800c000 con 0x7f7fe80047c0 2023-11-09T03:49:48.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.706+0000 7f7ffe7fc640 1 -- 172.21.15.76:0/3412296935 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe800c1c0 con 0x7f7fe8003b00 2023-11-09T03:49:48.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.706+0000 7f7ffffff640 1 --2- 172.21.15.76:0/3412296935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fe8005180 0x7f7fe800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.706+0000 7f7fff7fe640 1 --2- 172.21.15.76:0/3412296935 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fe8003b00 0x7f7fe8003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.706+0000 7f7ffeffd640 1 --2- 172.21.15.76:0/3412296935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fe80047c0 0x7f7fe8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.706+0000 7f7ffffff640 1 -- 172.21.15.76:0/3412296935 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fe8003b00 msgr2=0x7f7fe8003f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:48.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.706+0000 7f7ffffff640 1 --2- 172.21.15.76:0/3412296935 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fe8003b00 0x7f7fe8003f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.706+0000 7f7ffffff640 1 -- 172.21.15.76:0/3412296935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fe80047c0 msgr2=0x7f7fe8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:48.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.706+0000 7f7ffffff640 1 --2- 172.21.15.76:0/3412296935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fe80047c0 0x7f7fe8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.706+0000 7f7ffffff640 1 -- 172.21.15.76:0/3412296935 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe8003520 con 0x7f7fe8005180 2023-11-09T03:49:48.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.707+0000 7f7ffffff640 1 -- 172.21.15.76:0/3412296935 reap_dead start 2023-11-09T03:49:48.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.707+0000 7f7ffffff640 1 --2- 172.21.15.76:0/3412296935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fe8005180 0x7f7fe800b980 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800a7a0 tx=0x7f7fe80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241b0304c25ad522 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:48.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.707+0000 7f7ffe7fc640 1 -- 172.21.15.76:0/3412296935 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8034630 con 0x7f7fe8005180 2023-11-09T03:49:48.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.707+0000 7f7ffe7fc640 1 -- 172.21.15.76:0/3412296935 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff802f070 con 0x7f7fe8005180 2023-11-09T03:49:48.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.707+0000 7f7ffe7fc640 1 -- 172.21.15.76:0/3412296935 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff80349b0 con 0x7f7fe8005180 2023-11-09T03:49:48.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.708+0000 7f8005b33640 1 -- 172.21.15.76:0/3412296935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fe8005180 msgr2=0x7f7fe800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:48.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.708+0000 7f8005b33640 1 --2- 172.21.15.76:0/3412296935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fe8005180 0x7f7fe800b980 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800a7a0 tx=0x7f7fe80057f0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.708+0000 7f8005b33640 1 -- 172.21.15.76:0/3412296935 shutdown_connections 2023-11-09T03:49:48.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.708+0000 7f8005b33640 1 --2- 172.21.15.76:0/3412296935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fe8005180 0x7f7fe800b980 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.708+0000 7f8005b33640 1 -- 172.21.15.76:0/3412296935 >> 172.21.15.76:0/3412296935 conn(0x7f80000cf880 msgr2=0x7f80000cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:48.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.708+0000 7f8005b33640 1 -- 172.21.15.76:0/3412296935 shutdown_connections 2023-11-09T03:49:48.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.708+0000 7f8005b33640 1 -- 172.21.15.76:0/3412296935 wait complete. 2023-11-09T03:49:48.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.709+0000 7f8005b33640 1 Processor -- start 2023-11-09T03:49:48.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.709+0000 7f8005b33640 1 -- start start 2023-11-09T03:49:48.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.710+0000 7f8005b33640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fe8005180 0x7f80000c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.710+0000 7f8005b33640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8000152150 0x7f800015bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.710+0000 7f8005b33640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80001525d0 0x7f800015c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.710+0000 7f8005b33640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80000c71d0 con 0x7f7fe8005180 2023-11-09T03:49:48.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.710+0000 7f8005b33640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80000c6900 con 0x7f8000152150 2023-11-09T03:49:48.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.710+0000 7f8005b33640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80001f38a0 con 0x7f80001525d0 2023-11-09T03:49:48.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.710+0000 7f7fff7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fe8005180 0x7f80000c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.710+0000 7f7fff7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fe8005180 0x7f80000c75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47610/0 (socket says 172.21.15.76:47610) 2023-11-09T03:49:48.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.710+0000 7f7fff7fe640 1 -- 172.21.15.76:0/1421031859 learned_addr learned my addr 172.21.15.76:0/1421031859 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:48.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.710+0000 7f7ffeffd640 1 --2- 172.21.15.76:0/1421031859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8000152150 0x7f800015bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.710+0000 7f7ffffff640 1 --2- 172.21.15.76:0/1421031859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80001525d0 0x7f800015c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.710+0000 7f7fff7fe640 1 -- 172.21.15.76:0/1421031859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80001525d0 msgr2=0x7f800015c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:48.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.710+0000 7f7fff7fe640 1 --2- 172.21.15.76:0/1421031859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80001525d0 0x7f800015c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.710+0000 7f7fff7fe640 1 -- 172.21.15.76:0/1421031859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8000152150 msgr2=0x7f800015bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:48.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.710+0000 7f7fff7fe640 1 --2- 172.21.15.76:0/1421031859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8000152150 0x7f800015bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.710+0000 7f7fff7fe640 1 -- 172.21.15.76:0/1421031859 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80000c6b10 con 0x7f7fe8005180 2023-11-09T03:49:48.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.710+0000 7f7ffeffd640 1 --2- 172.21.15.76:0/1421031859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8000152150 0x7f800015bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:48.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.710+0000 7f7ffffff640 1 --2- 172.21.15.76:0/1421031859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80001525d0 0x7f800015c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:48.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.710+0000 7f7fff7fe640 1 --2- 172.21.15.76:0/1421031859 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fe8005180 0x7f80000c75f0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000f7a0 tx=0x7f7ff000fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:48.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.711+0000 7f7fe7fff640 1 -- 172.21.15.76:0/1421031859 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff000fe50 con 0x7f7fe8005180 2023-11-09T03:49:48.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.711+0000 7f7fe7fff640 1 -- 172.21.15.76:0/1421031859 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff0005320 con 0x7f7fe8005180 2023-11-09T03:49:48.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.711+0000 7f8005b33640 1 -- 172.21.15.76:0/1421031859 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80000c6df0 con 0x7f7fe8005180 2023-11-09T03:49:48.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.711+0000 7f7fe7fff640 1 -- 172.21.15.76:0/1421031859 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0007290 con 0x7f7fe8005180 2023-11-09T03:49:48.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.711+0000 7f8005b33640 1 -- 172.21.15.76:0/1421031859 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80001f2d20 con 0x7f7fe8005180 2023-11-09T03:49:48.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.712+0000 7f7fe7fff640 1 -- 172.21.15.76:0/1421031859 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7ff00073f0 con 0x7f7fe8005180 2023-11-09T03:49:48.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.712+0000 7f7fe7fff640 1 --2- 172.21.15.76:0/1421031859 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7fec06ae70 0x7f7fec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.712+0000 7f7fe7fff640 1 -- 172.21.15.76:0/1421031859 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7f7ff0089560 con 0x7f7fe8005180 2023-11-09T03:49:48.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.713+0000 7f7ffeffd640 1 --2- 172.21.15.76:0/1421031859 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7fec06ae70 0x7f7fec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.713+0000 7f7ffeffd640 1 --2- 172.21.15.76:0/1421031859 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7fec06ae70 0x7f7fec06d330 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f80000c84d0 tx=0x7f7ff4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:48.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.713+0000 7f800879d740 1 Processor -- start 2023-11-09T03:49:48.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.713+0000 7f800879d740 1 -- start start 2023-11-09T03:49:48.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.714+0000 7f800879d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623e13a2850 0x5623e1375750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.714+0000 7f800879d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623e13a2c30 0x5623e13a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.714+0000 7f800879d740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5623e1375c90 0x5623e13a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.714+0000 7f800879d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e1286740 con 0x5623e13a2850 2023-11-09T03:49:48.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.714+0000 7f800879d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e12e88d0 con 0x5623e13a2c30 2023-11-09T03:49:48.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.714+0000 7f800879d740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623e12e5fc0 con 0x5623e1375c90 2023-11-09T03:49:48.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.714+0000 7f7fff7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623e13a2850 0x5623e1375750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.714+0000 7f7fff7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623e13a2850 0x5623e1375750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47626/0 (socket says 172.21.15.76:47626) 2023-11-09T03:49:48.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.714+0000 7f7fff7fe640 1 -- 172.21.15.76:0/1074586097 learned_addr learned my addr 172.21.15.76:0/1074586097 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:48.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.714+0000 7f7ffeffd640 1 --2- 172.21.15.76:0/1074586097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623e13a2c30 0x5623e13a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.714+0000 7f7ffffff640 1 --2- 172.21.15.76:0/1074586097 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5623e1375c90 0x5623e13a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.715+0000 7f7fff7fe640 1 -- 172.21.15.76:0/1074586097 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5623e1375c90 msgr2=0x5623e13a5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:48.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.715+0000 7f7fff7fe640 1 --2- 172.21.15.76:0/1074586097 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5623e1375c90 0x5623e13a5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.715+0000 7f7fff7fe640 1 -- 172.21.15.76:0/1074586097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623e13a2c30 msgr2=0x5623e13a55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:48.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.715+0000 7f7fff7fe640 1 --2- 172.21.15.76:0/1074586097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623e13a2c30 0x5623e13a55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.715+0000 7f7fff7fe640 1 -- 172.21.15.76:0/1074586097 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623e12a4ca0 con 0x5623e13a2850 2023-11-09T03:49:48.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.715+0000 7f7ffeffd640 1 --2- 172.21.15.76:0/1074586097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623e13a2c30 0x5623e13a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:48.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.715+0000 7f7fff7fe640 1 --2- 172.21.15.76:0/1074586097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623e13a2850 0x5623e1375750 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0010010 tx=0x7f7ff00577e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744e021909a125c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:48.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.715+0000 7f7fe57fa640 1 -- 172.21.15.76:0/1074586097 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0057d50 con 0x5623e13a2850 2023-11-09T03:49:48.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.715+0000 7f7fe57fa640 1 -- 172.21.15.76:0/1074586097 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff005bea0 con 0x5623e13a2850 2023-11-09T03:49:48.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.715+0000 7f7fe57fa640 1 -- 172.21.15.76:0/1074586097 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0060050 con 0x5623e13a2850 2023-11-09T03:49:48.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.716+0000 7f800879d740 1 -- 172.21.15.76:0/1074586097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623e13a2850 msgr2=0x5623e1375750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:48.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.716+0000 7f800879d740 1 --2- 172.21.15.76:0/1074586097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623e13a2850 0x5623e1375750 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0010010 tx=0x7f7ff00577e0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.716+0000 7f800879d740 1 -- 172.21.15.76:0/1074586097 shutdown_connections 2023-11-09T03:49:48.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.716+0000 7f800879d740 1 --2- 172.21.15.76:0/1074586097 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5623e1375c90 0x5623e13a5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.716+0000 7f800879d740 1 --2- 172.21.15.76:0/1074586097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623e13a2c30 0x5623e13a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.716+0000 7f800879d740 1 --2- 172.21.15.76:0/1074586097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623e13a2850 0x5623e1375750 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.716+0000 7f800879d740 1 -- 172.21.15.76:0/1074586097 >> 172.21.15.76:0/1074586097 conn(0x5623e1377a60 msgr2=0x5623e13c05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:48.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.716+0000 7f800879d740 1 -- 172.21.15.76:0/1074586097 shutdown_connections 2023-11-09T03:49:48.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.716+0000 7f800879d740 1 -- 172.21.15.76:0/1074586097 wait complete. 2023-11-09T03:49:48.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.716+0000 7f800879d740 1 -- start start 2023-11-09T03:49:48.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.716+0000 7f800879d740 1 Processor -- start 2023-11-09T03:49:48.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.717+0000 7f800879d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623e13a2850 0x5623e13a7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.717+0000 7f800879d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623e13a2c30 0x5623e1396bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.717+0000 7f800879d740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5623e13a1e00 0x5623e13bdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.717+0000 7f800879d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e12e18b0 con 0x5623e13a2850 2023-11-09T03:49:48.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.717+0000 7f800879d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e129bc80 con 0x5623e13a2c30 2023-11-09T03:49:48.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.717+0000 7f800879d740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623e129b920 con 0x5623e13a1e00 2023-11-09T03:49:48.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.717+0000 7f800879d740 10 client.0 _ll_register_callbacks cb 0x5623e0ff3ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:49:48.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.717+0000 7f7fff7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623e13a2850 0x5623e13a7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.717+0000 7f7fff7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623e13a2850 0x5623e13a7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47632/0 (socket says 172.21.15.76:47632) 2023-11-09T03:49:48.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.717+0000 7f7fff7fe640 1 -- 172.21.15.76:0/112402312 learned_addr learned my addr 172.21.15.76:0/112402312 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:48.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.717+0000 7f7ffffff640 1 --2- 172.21.15.76:0/112402312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5623e13a1e00 0x5623e13bdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.717+0000 7f7ffeffd640 1 --2- 172.21.15.76:0/112402312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623e13a2c30 0x5623e1396bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.718+0000 7f7fff7fe640 1 -- 172.21.15.76:0/112402312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5623e13a1e00 msgr2=0x5623e13bdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:48.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.718+0000 7f7fff7fe640 1 --2- 172.21.15.76:0/112402312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5623e13a1e00 0x5623e13bdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.718+0000 7f7fff7fe640 1 -- 172.21.15.76:0/112402312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623e13a2c30 msgr2=0x5623e1396bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:48.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.718+0000 7f7fff7fe640 1 --2- 172.21.15.76:0/112402312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623e13a2c30 0x5623e1396bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.718+0000 7f7fff7fe640 1 -- 172.21.15.76:0/112402312 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80001f2d20 con 0x5623e13a2850 2023-11-09T03:49:48.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.718+0000 7f7fff7fe640 1 --2- 172.21.15.76:0/112402312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623e13a2850 0x5623e13a7620 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0016040 tx=0x7f7ff00012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=920714d355b6a0f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:48.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.718+0000 7f7fde7fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:49:48.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.718+0000 7f7fde7fc640 1 -- 172.21.15.76:0/112402312 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0057d50 con 0x5623e13a2850 2023-11-09T03:49:48.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.718+0000 7f7fde7fc640 1 -- 172.21.15.76:0/112402312 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7f7ff0060050 con 0x5623e13a2850 2023-11-09T03:49:48.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.718+0000 7f800879d740 1 -- 172.21.15.76:0/112402312 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5623e128dbf0 con 0x5623e13a2850 2023-11-09T03:49:48.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.718+0000 7f7fde7fc640 1 -- 172.21.15.76:0/112402312 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ff005cbe0 con 0x5623e13a2850 2023-11-09T03:49:48.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.718+0000 7f800879d740 10 client.7830 fetch_fsmap learned FSMap version 57 2023-11-09T03:49:48.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.718+0000 7f800879d740 1 -- 172.21.15.76:0/112402312 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x5623e12dff30 con 0x5623e13a2850 2023-11-09T03:49:48.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.718+0000 7f7fde7fc640 1 -- 172.21.15.76:0/112402312 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5623e128dbf0 con 0x5623e13a2850 2023-11-09T03:49:48.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.718+0000 7f7fde7fc640 1 -- 172.21.15.76:0/112402312 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 57) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f7ff0012070 con 0x5623e13a2850 2023-11-09T03:49:48.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.718+0000 7f800879d740 10 client.7830 fetch_fsmap finished waiting for FSMap version 57 2023-11-09T03:49:48.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.718+0000 7f800879d740 10 client.7830 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-09T03:49:48.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.718+0000 7f800879d740 20 client.7830 populate_metadata read hostname 'smithi076' 2023-11-09T03:49:48.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.718+0000 7f800879d740 1 --2- 172.21.15.76:0/112402312 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x5623e13d4530 0x5623e13ddcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.718+0000 7f800879d740 4 client.7830 mds_command: new command op to 7449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:49:48.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.718+0000 7f800879d740 1 -- 172.21.15.76:0/112402312 --> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623e12da890 con 0x5623e13d4530 2023-11-09T03:49:48.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.718+0000 7f7ffeffd640 1 --2- 172.21.15.76:0/112402312 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x5623e13d4530 0x5623e13ddcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.719+0000 7f7fde7fc640 10 client.7830 ms_handle_connect on v2:172.21.15.76:6832/3697850101 2023-11-09T03:49:48.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.719+0000 7f7ffeffd640 1 --2- 172.21.15.76:0/112402312 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x5623e13d4530 0x5623e13ddcf0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9ad0713e6a3621e9 server_cookie=e0d5aed3563e3ac0 in_seq=0 out_seq=0 2023-11-09T03:49:48.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.719+0000 7f7fde7fc640 1 -- 172.21.15.76:0/112402312 <== mds.1 v2:172.21.15.76:6832/3697850101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5623e12da890 con 0x5623e13d4530 2023-11-09T03:49:48.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.719+0000 7f7fde7fc640 10 client.7830 handle_command_reply: tid=0 2023-11-09T03:49:48.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.719+0000 7f800879d740 1 client.7830 shutdown 2023-11-09T03:49:48.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.719+0000 7f800879d740 1 -- 172.21.15.76:0/112402312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623e13a2850 msgr2=0x5623e13a7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:48.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.719+0000 7f800879d740 1 --2- 172.21.15.76:0/112402312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623e13a2850 0x5623e13a7620 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0016040 tx=0x7f7ff00012d0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.719+0000 7f800879d740 1 -- 172.21.15.76:0/112402312 shutdown_connections 2023-11-09T03:49:48.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.719+0000 7f800879d740 1 --2- 172.21.15.76:0/112402312 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x5623e13d4530 0x5623e13ddcf0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.719+0000 7f800879d740 1 --2- 172.21.15.76:0/112402312 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5623e13a1e00 0x5623e13bdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.719+0000 7f800879d740 1 --2- 172.21.15.76:0/112402312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623e13a2c30 0x5623e1396bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.719+0000 7f7fde7fc640 0 client.7830 ms_handle_reset on v2:172.21.15.76:6832/3697850101 2023-11-09T03:49:48.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.719+0000 7f800879d740 1 --2- 172.21.15.76:0/112402312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623e13a2850 0x5623e13a7620 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.719+0000 7f800879d740 1 -- 172.21.15.76:0/112402312 >> 172.21.15.76:0/112402312 conn(0x5623e1375c90 msgr2=0x5623e13744f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:48.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.719+0000 7f800879d740 1 -- 172.21.15.76:0/112402312 shutdown_connections 2023-11-09T03:49:48.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.720+0000 7f800879d740 1 -- 172.21.15.76:0/112402312 wait complete. 2023-11-09T03:49:48.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.720+0000 7f800879d740 20 client.7830 trim_cache size 0 max 16384 2023-11-09T03:49:48.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.735+0000 7f800879d740 1 Processor -- start 2023-11-09T03:49:48.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.735+0000 7f800879d740 1 -- start start 2023-11-09T03:49:48.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.736+0000 7f800879d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623e13a1e00 0x5623e13a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.736+0000 7f800879d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623e13a2850 0x5623e13969e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.736+0000 7f800879d740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5623e13a2c30 0x5623e13d2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.736+0000 7f800879d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e1261280 con 0x5623e13a2850 2023-11-09T03:49:48.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.736+0000 7f800879d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e13bf0a0 con 0x5623e13a1e00 2023-11-09T03:49:48.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.736+0000 7f800879d740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623e13a55d0 con 0x5623e13a2c30 2023-11-09T03:49:48.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.736+0000 7f7ffeffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623e13a2850 0x5623e13969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.736+0000 7f7ffeffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623e13a2850 0x5623e13969e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47640/0 (socket says 172.21.15.76:47640) 2023-11-09T03:49:48.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.736+0000 7f7ffeffd640 1 -- 172.21.15.76:0/1279367979 learned_addr learned my addr 172.21.15.76:0/1279367979 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:48.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.736+0000 7f7fff7fe640 1 --2- 172.21.15.76:0/1279367979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623e13a1e00 0x5623e13a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.736+0000 7f7ffffff640 1 --2- 172.21.15.76:0/1279367979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5623e13a2c30 0x5623e13d2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.736+0000 7f7ffeffd640 1 -- 172.21.15.76:0/1279367979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5623e13a2c30 msgr2=0x5623e13d2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:48.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.736+0000 7f7ffeffd640 1 --2- 172.21.15.76:0/1279367979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5623e13a2c30 0x5623e13d2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.736+0000 7f7ffeffd640 1 -- 172.21.15.76:0/1279367979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623e13a1e00 msgr2=0x5623e13a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:48.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.736+0000 7f7ffeffd640 1 --2- 172.21.15.76:0/1279367979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623e13a1e00 0x5623e13a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.736+0000 7f7ffeffd640 1 -- 172.21.15.76:0/1279367979 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623e127dd10 con 0x5623e13a2850 2023-11-09T03:49:48.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.736+0000 7f7ffffff640 1 --2- 172.21.15.76:0/1279367979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5623e13a2c30 0x5623e13d2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:49:48.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.736+0000 7f7fff7fe640 1 --2- 172.21.15.76:0/1279367979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623e13a1e00 0x5623e13a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:49:48.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.736+0000 7f7ffeffd640 1 --2- 172.21.15.76:0/1279367979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623e13a2850 0x5623e13969e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f7ff402cdd0 tx=0x7f7ff4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4bc92b13d9708fc server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:48.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.736+0000 7f7fe5ffb640 1 -- 172.21.15.76:0/1279367979 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff402b070 con 0x5623e13a2850 2023-11-09T03:49:48.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.736+0000 7f7fe5ffb640 1 -- 172.21.15.76:0/1279367979 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff402ce00 con 0x5623e13a2850 2023-11-09T03:49:48.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.737+0000 7f7fe5ffb640 1 -- 172.21.15.76:0/1279367979 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4026110 con 0x5623e13a2850 2023-11-09T03:49:48.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.737+0000 7f800879d740 1 -- 172.21.15.76:0/1279367979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623e13a2850 msgr2=0x5623e13969e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:48.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.737+0000 7f800879d740 1 --2- 172.21.15.76:0/1279367979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623e13a2850 0x5623e13969e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f7ff402cdd0 tx=0x7f7ff4017000 comp rx=0 tx=0).stop 2023-11-09T03:49:48.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.737+0000 7f800879d740 1 -- 172.21.15.76:0/1279367979 shutdown_connections 2023-11-09T03:49:48.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.737+0000 7f800879d740 1 --2- 172.21.15.76:0/1279367979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5623e13a2c30 0x5623e13d2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.737+0000 7f800879d740 1 --2- 172.21.15.76:0/1279367979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623e13a2850 0x5623e13969e0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.737+0000 7f800879d740 1 --2- 172.21.15.76:0/1279367979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623e13a1e00 0x5623e13a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.737+0000 7f800879d740 1 -- 172.21.15.76:0/1279367979 >> 172.21.15.76:0/1279367979 conn(0x5623e1375c90 msgr2=0x5623e1374b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:48.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.737+0000 7f800879d740 1 -- 172.21.15.76:0/1279367979 shutdown_connections 2023-11-09T03:49:48.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.737+0000 7f800879d740 1 -- 172.21.15.76:0/1279367979 wait complete. 2023-11-09T03:49:48.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.738+0000 7f800879d740 1 -- start start 2023-11-09T03:49:48.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.738+0000 7f800879d740 1 Processor -- start 2023-11-09T03:49:48.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.738+0000 7f800879d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623e13a1e00 0x5623e142aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.738+0000 7f800879d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623e13a2850 0x5623e142afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.738+0000 7f800879d740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5623e13a2c30 0x5623e1379220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.738+0000 7f800879d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e1396630 con 0x5623e13a2850 2023-11-09T03:49:48.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.738+0000 7f800879d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e1394bb0 con 0x5623e13a1e00 2023-11-09T03:49:48.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.738+0000 7f800879d740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623e13c4610 con 0x5623e13a2c30 2023-11-09T03:49:48.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.738+0000 7f800879d740 10 client.0 _ll_register_callbacks cb 0x7f7ff0054090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:49:48.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.739+0000 7f7ffeffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623e13a2850 0x5623e142afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.739+0000 7f7ffeffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623e13a2850 0x5623e142afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47646/0 (socket says 172.21.15.76:47646) 2023-11-09T03:49:48.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.739+0000 7f7ffeffd640 1 -- 172.21.15.76:0/1527907854 learned_addr learned my addr 172.21.15.76:0/1527907854 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:48.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.739+0000 7f7fff7fe640 1 --2- 172.21.15.76:0/1527907854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623e13a1e00 0x5623e142aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.739+0000 7f7ffffff640 1 --2- 172.21.15.76:0/1527907854 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5623e13a2c30 0x5623e1379220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.739+0000 7f7ffeffd640 1 -- 172.21.15.76:0/1527907854 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5623e13a2c30 msgr2=0x5623e1379220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:48.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.739+0000 7f7ffeffd640 1 --2- 172.21.15.76:0/1527907854 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5623e13a2c30 0x5623e1379220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.739+0000 7f7ffeffd640 1 -- 172.21.15.76:0/1527907854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623e13a1e00 msgr2=0x5623e142aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:48.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.739+0000 7f7ffeffd640 1 --2- 172.21.15.76:0/1527907854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623e13a1e00 0x5623e142aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.739+0000 7f7ffeffd640 1 -- 172.21.15.76:0/1527907854 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff402cb00 con 0x5623e13a2850 2023-11-09T03:49:48.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.739+0000 7f7ffeffd640 1 --2- 172.21.15.76:0/1527907854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623e13a2850 0x5623e142afe0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4030040 tx=0x7f7ff4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3222d0354e9c46b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:48.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.739+0000 7f7fde7fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:49:48.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.739+0000 7f7fde7fc640 1 -- 172.21.15.76:0/1527907854 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff402c620 con 0x5623e13a2850 2023-11-09T03:49:48.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.739+0000 7f7fde7fc640 1 -- 172.21.15.76:0/1527907854 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7f7ff402c780 con 0x5623e13a2850 2023-11-09T03:49:48.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.739+0000 7f800879d740 1 -- 172.21.15.76:0/1527907854 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ff005bea0 con 0x5623e13a2850 2023-11-09T03:49:48.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.739+0000 7f7fde7fc640 1 -- 172.21.15.76:0/1527907854 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ff4038460 con 0x5623e13a2850 2023-11-09T03:49:48.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.739+0000 7f800879d740 10 client.7836 fetch_fsmap learned FSMap version 57 2023-11-09T03:49:48.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.739+0000 7f800879d740 1 -- 172.21.15.76:0/1527907854 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x5623e12a3500 con 0x5623e13a2850 2023-11-09T03:49:48.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.740+0000 7f7fde7fc640 1 -- 172.21.15.76:0/1527907854 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff005bea0 con 0x5623e13a2850 2023-11-09T03:49:48.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.740+0000 7f7fde7fc640 1 -- 172.21.15.76:0/1527907854 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 57) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f7ff40385c0 con 0x5623e13a2850 2023-11-09T03:49:48.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.740+0000 7f800879d740 10 client.7836 fetch_fsmap finished waiting for FSMap version 57 2023-11-09T03:49:48.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.740+0000 7f800879d740 10 client.7836 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-09T03:49:48.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.740+0000 7f800879d740 20 client.7836 populate_metadata read hostname 'smithi076' 2023-11-09T03:49:48.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.740+0000 7f800879d740 1 --2- 172.21.15.76:0/1527907854 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x5623e13a7260 0x5623e13a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.740+0000 7f800879d740 4 client.7836 mds_command: new command op to 7449 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-09T03:49:48.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.740+0000 7f800879d740 1 -- 172.21.15.76:0/1527907854 --> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5623e12da5b0 con 0x5623e13a7260 2023-11-09T03:49:48.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.740+0000 7f7fff7fe640 1 --2- 172.21.15.76:0/1527907854 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x5623e13a7260 0x5623e13a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.740+0000 7f7fff7fe640 1 --2- 172.21.15.76:0/1527907854 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x5623e13a7260 0x5623e13a68a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3e08f930861c9c73 server_cookie=bd410fbeffe01987 in_seq=0 out_seq=0 2023-11-09T03:49:48.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.740+0000 7f7fde7fc640 10 client.7836 ms_handle_connect on v2:172.21.15.76:6832/3697850101 2023-11-09T03:49:48.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.740+0000 7f7fde7fc640 1 -- 172.21.15.76:0/1527907854 <== mds.1 v2:172.21.15.76:6832/3697850101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5623e12da5b0 con 0x5623e13a7260 2023-11-09T03:49:48.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.740+0000 7f7fde7fc640 10 client.7836 handle_command_reply: tid=0 2023-11-09T03:49:48.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.740+0000 7f800879d740 1 client.7836 shutdown 2023-11-09T03:49:48.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.740+0000 7f800879d740 1 -- 172.21.15.76:0/1527907854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623e13a2850 msgr2=0x5623e142afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:48.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.740+0000 7f800879d740 1 --2- 172.21.15.76:0/1527907854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623e13a2850 0x5623e142afe0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4030040 tx=0x7f7ff4004730 comp rx=0 tx=0).stop 2023-11-09T03:49:48.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.740+0000 7f800879d740 1 -- 172.21.15.76:0/1527907854 shutdown_connections 2023-11-09T03:49:48.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.740+0000 7f800879d740 1 --2- 172.21.15.76:0/1527907854 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x5623e13a7260 0x5623e13a68a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.740+0000 7f800879d740 1 --2- 172.21.15.76:0/1527907854 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5623e13a2c30 0x5623e1379220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.740+0000 7f7fde7fc640 0 client.7836 ms_handle_reset on v2:172.21.15.76:6832/3697850101 2023-11-09T03:49:48.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.740+0000 7f800879d740 1 --2- 172.21.15.76:0/1527907854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623e13a2850 0x5623e142afe0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.740+0000 7f800879d740 1 --2- 172.21.15.76:0/1527907854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623e13a1e00 0x5623e142aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.740+0000 7f800879d740 1 -- 172.21.15.76:0/1527907854 >> 172.21.15.76:0/1527907854 conn(0x5623e1375c90 msgr2=0x5623e13d2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:48.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.740+0000 7f800879d740 1 -- 172.21.15.76:0/1527907854 shutdown_connections 2023-11-09T03:49:48.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.740+0000 7f800879d740 1 -- 172.21.15.76:0/1527907854 wait complete. 2023-11-09T03:49:48.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.741+0000 7f800879d740 20 client.7836 trim_cache size 0 max 16384 2023-11-09T03:49:48.768 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:49:48.768 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "c213b26f-d0da-45d5-8e34-bb75c096d4f2", 2023-11-09T03:49:48.768 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 1, 2023-11-09T03:49:48.768 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 7449, 2023-11-09T03:49:48.769 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2023-11-09T03:49:48.769 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2023-11-09T03:49:48.769 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2023-11-09T03:49:48.769 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 41.293896703000001, 2023-11-09T03:49:48.769 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 57, 2023-11-09T03:49:48.769 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 98, 2023-11-09T03:49:48.770 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 98, 2023-11-09T03:49:48.770 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 56.744191012000002 2023-11-09T03:49:48.770 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:49:48.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.741+0000 7f8005b33640 1 -- 172.21.15.76:0/1421031859 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7fec06ae70 msgr2=0x7f7fec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:48.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.741+0000 7f8005b33640 1 --2- 172.21.15.76:0/1421031859 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7fec06ae70 0x7f7fec06d330 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f80000c84d0 tx=0x7f7ff4008040 comp rx=0 tx=0).stop 2023-11-09T03:49:48.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.741+0000 7f8005b33640 1 -- 172.21.15.76:0/1421031859 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fe8005180 msgr2=0x7f80000c75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:48.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.741+0000 7f8005b33640 1 --2- 172.21.15.76:0/1421031859 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fe8005180 0x7f80000c75f0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000f7a0 tx=0x7f7ff000fc60 comp rx=0 tx=0).stop 2023-11-09T03:49:48.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.741+0000 7f8005b33640 1 -- 172.21.15.76:0/1421031859 shutdown_connections 2023-11-09T03:49:48.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.741+0000 7f8005b33640 1 --2- 172.21.15.76:0/1421031859 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7fec06ae70 0x7f7fec06d330 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.741+0000 7f8005b33640 1 --2- 172.21.15.76:0/1421031859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80001525d0 0x7f800015c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.741+0000 7f8005b33640 1 --2- 172.21.15.76:0/1421031859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8000152150 0x7f800015bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.741+0000 7f8005b33640 1 --2- 172.21.15.76:0/1421031859 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fe8005180 0x7f80000c75f0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.741+0000 7f8005b33640 1 -- 172.21.15.76:0/1421031859 >> 172.21.15.76:0/1421031859 conn(0x7f80000cf880 msgr2=0x7f80000cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:48.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.741+0000 7f8005b33640 1 -- 172.21.15.76:0/1421031859 shutdown_connections 2023-11-09T03:49:48.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.741+0000 7f8005b33640 1 -- 172.21.15.76:0/1421031859 wait complete. 2023-11-09T03:49:48.774 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:49:48.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.874+0000 7fa1e1034640 1 Processor -- start 2023-11-09T03:49:48.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.874+0000 7fa1e1034640 1 -- start start 2023-11-09T03:49:48.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.874+0000 7fa1e1034640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa1dc0b0ae0 0x7fa1dc0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.874+0000 7fa1e1034640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa1dc06db20 0x7fa1dc15d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.874+0000 7fa1e1034640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1dc0b4110 con 0x7fa1dc159c70 2023-11-09T03:49:48.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.874+0000 7fa1e1034640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1dc0b49b0 con 0x7fa1dc06db20 2023-11-09T03:49:48.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.874+0000 7fa1e1034640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1dc0b43f0 con 0x7fa1dc0b0ae0 2023-11-09T03:49:48.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.875+0000 7fa1db59e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa1dc06db20 0x7fa1dc15d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.875+0000 7fa1dad9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa1dc159c70 0x7fa1dc0b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46226/0 (socket says 172.21.15.76:46226) 2023-11-09T03:49:48.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.875+0000 7fa1dad9d640 1 -- 172.21.15.76:0/3622944962 learned_addr learned my addr 172.21.15.76:0/3622944962 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:49:48.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.875+0000 7fa1db59e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa1dc06db20 0x7fa1dc15d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47656/0 (socket says 172.21.15.76:47656) 2023-11-09T03:49:48.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.875+0000 7fa1da59c640 1 --2- 172.21.15.76:0/3622944962 >> v2:172.21.15.113:3300/0 conn(0x7fa1dc0b0ae0 0x7fa1dc0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.875+0000 7fa1db59e640 1 -- 172.21.15.76:0/3622944962 >> v2:172.21.15.113:3300/0 conn(0x7fa1dc0b0ae0 msgr2=0x7fa1dc0ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:48.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.875+0000 7fa1db59e640 1 --2- 172.21.15.76:0/3622944962 >> v2:172.21.15.113:3300/0 conn(0x7fa1dc0b0ae0 0x7fa1dc0ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.875+0000 7fa1db59e640 1 -- 172.21.15.76:0/3622944962 >> v1:172.21.15.76:6789/0 conn(0x7fa1dc159c70 legacy=0x7fa1dc0b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:48.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.875+0000 7fa1db59e640 1 -- 172.21.15.76:0/3622944962 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc0b46d0 con 0x7fa1dc06db20 2023-11-09T03:49:48.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.876+0000 7fa1db59e640 1 --2- 172.21.15.76:0/3622944962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1dc06db20 0x7fa1dc15d7d0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc0cf800 tx=0x7fa1d402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac423da5b0bec3d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:48.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.876+0000 7fa1d9d9b640 1 -- 172.21.15.76:0/3622944962 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d402cc80 con 0x7fa1dc06db20 2023-11-09T03:49:48.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.876+0000 7fa1d9d9b640 1 -- 172.21.15.76:0/3622944962 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d402cde0 con 0x7fa1dc06db20 2023-11-09T03:49:48.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.876+0000 7fa1d9d9b640 1 -- 172.21.15.76:0/3622944962 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4034660 con 0x7fa1dc06db20 2023-11-09T03:49:48.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.876+0000 7fa1e1034640 1 -- 172.21.15.76:0/3622944962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1dc06db20 msgr2=0x7fa1dc15d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:48.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.876+0000 7fa1e1034640 1 --2- 172.21.15.76:0/3622944962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1dc06db20 0x7fa1dc15d7d0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc0cf800 tx=0x7fa1d402bed0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.876+0000 7fa1e1034640 1 -- 172.21.15.76:0/3622944962 shutdown_connections 2023-11-09T03:49:48.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.876+0000 7fa1e1034640 1 --2- 172.21.15.76:0/3622944962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1dc06db20 0x7fa1dc15d7d0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.876+0000 7fa1e1034640 1 --2- 172.21.15.76:0/3622944962 >> v2:172.21.15.113:3300/0 conn(0x7fa1dc0b0ae0 0x7fa1dc0ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.876+0000 7fa1e1034640 1 -- 172.21.15.76:0/3622944962 >> 172.21.15.76:0/3622944962 conn(0x7fa1dc0ce950 msgr2=0x7fa1dc0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:48.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.876+0000 7fa1e1034640 1 -- 172.21.15.76:0/3622944962 shutdown_connections 2023-11-09T03:49:48.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.876+0000 7fa1e1034640 1 -- 172.21.15.76:0/3622944962 wait complete. 2023-11-09T03:49:48.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.878+0000 7fa1e1034640 1 Processor -- start 2023-11-09T03:49:48.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.878+0000 7fa1e1034640 1 -- start start 2023-11-09T03:49:48.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.878+0000 7fa1e1034640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1dc06db20 0x7fa1dc0ac290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.878+0000 7fa1e1034640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1dc0b0ae0 0x7fa1dc155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.878+0000 7fa1e1034640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1dc159c70 0x7fa1dc157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.878+0000 7fa1e1034640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc0a87e0 con 0x7fa1dc0b0ae0 2023-11-09T03:49:48.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.878+0000 7fa1e1034640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc0a91d0 con 0x7fa1dc159c70 2023-11-09T03:49:48.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.878+0000 7fa1e1034640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc0a94b0 con 0x7fa1dc06db20 2023-11-09T03:49:48.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.879+0000 7fa1da59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1dc0b0ae0 0x7fa1dc155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.879+0000 7fa1da59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1dc0b0ae0 0x7fa1dc155ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47662/0 (socket says 172.21.15.76:47662) 2023-11-09T03:49:48.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.879+0000 7fa1da59c640 1 -- 172.21.15.76:0/2137226996 learned_addr learned my addr 172.21.15.76:0/2137226996 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:48.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.879+0000 7fa1dad9d640 1 --2- 172.21.15.76:0/2137226996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1dc06db20 0x7fa1dc0ac290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.879+0000 7fa1db59e640 1 --2- 172.21.15.76:0/2137226996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1dc159c70 0x7fa1dc157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.879+0000 7fa1da59c640 1 -- 172.21.15.76:0/2137226996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1dc06db20 msgr2=0x7fa1dc0ac290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:48.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.879+0000 7fa1da59c640 1 --2- 172.21.15.76:0/2137226996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1dc06db20 0x7fa1dc0ac290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.879+0000 7fa1da59c640 1 -- 172.21.15.76:0/2137226996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1dc159c70 msgr2=0x7fa1dc157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:48.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.879+0000 7fa1da59c640 1 --2- 172.21.15.76:0/2137226996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1dc159c70 0x7fa1dc157ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:48.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.879+0000 7fa1da59c640 1 -- 172.21.15.76:0/2137226996 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d4009a70 con 0x7fa1dc0b0ae0 2023-11-09T03:49:48.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.879+0000 7fa1dad9d640 1 --2- 172.21.15.76:0/2137226996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1dc06db20 0x7fa1dc0ac290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:49:48.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.879+0000 7fa1da59c640 1 --2- 172.21.15.76:0/2137226996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1dc0b0ae0 0x7fa1dc155ee0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc00d800 tx=0x7fa1cc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:48.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.879+0000 7fa1c37fe640 1 -- 172.21.15.76:0/2137226996 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc00b6c0 con 0x7fa1dc0b0ae0 2023-11-09T03:49:48.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.879+0000 7fa1c37fe640 1 -- 172.21.15.76:0/2137226996 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1cc00b820 con 0x7fa1dc0b0ae0 2023-11-09T03:49:48.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.879+0000 7fa1e1034640 1 -- 172.21.15.76:0/2137226996 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1dc0a9790 con 0x7fa1dc0b0ae0 2023-11-09T03:49:48.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.879+0000 7fa1c37fe640 1 -- 172.21.15.76:0/2137226996 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc00bbb0 con 0x7fa1dc0b0ae0 2023-11-09T03:49:48.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.879+0000 7fa1e1034640 1 -- 172.21.15.76:0/2137226996 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1dc1581e0 con 0x7fa1dc0b0ae0 2023-11-09T03:49:48.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.880+0000 7fa1e1034640 1 -- 172.21.15.76:0/2137226996 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1dc074290 con 0x7fa1dc0b0ae0 2023-11-09T03:49:48.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.881+0000 7fa1c37fe640 1 -- 172.21.15.76:0/2137226996 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa1cc00bd10 con 0x7fa1dc0b0ae0 2023-11-09T03:49:48.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.881+0000 7fa1c37fe640 1 --2- 172.21.15.76:0/2137226996 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa1c806ae70 0x7fa1c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:48.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.881+0000 7fa1c37fe640 1 -- 172.21.15.76:0/2137226996 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7fa1cc08a130 con 0x7fa1dc0b0ae0 2023-11-09T03:49:48.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.881+0000 7fa1dad9d640 1 --2- 172.21.15.76:0/2137226996 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa1c806ae70 0x7fa1c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:48.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.882+0000 7fa1dad9d640 1 --2- 172.21.15.76:0/2137226996 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa1c806ae70 0x7fa1c806d330 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0005880 tx=0x7fa1d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:48.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:48.885+0000 7fa1c37fe640 1 -- 172.21.15.76:0/2137226996 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa1cc054c60 con 0x7fa1dc0b0ae0 2023-11-09T03:49:49.024 INFO:tasks.ceph.mds.b.smithi113.stderr:2023-11-09T03:49:49.024+0000 7f401c722640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:49:49.024 INFO:tasks.ceph.mds.b.smithi113.stderr:2023-11-09T03:49:49.024+0000 7f401c722640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-09T03:49:49.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.085+0000 7fa1e1034640 1 -- 172.21.15.76:0/2137226996 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1dc0b3c40 con 0x7fa1dc0b0ae0 2023-11-09T03:49:49.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.085+0000 7fa1c37fe640 1 -- 172.21.15.76:0/2137226996 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4934 (secure 0 0 0) 0x7fa1cc058280 con 0x7fa1dc0b0ae0 2023-11-09T03:49:49.086 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:49:49.086 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":7773,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1663988060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1663988060},{"type":"v1","addr":"172.21.15.76:6835","nonce":1663988060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:49:45.047290+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5744,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5744":{"gid":5744,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6837/3280098776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3280098776},{"type":"v1","addr":"172.21.15.113:6837","nonce":3280098776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:49:49.087 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 57 2023-11-09T03:49:49.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.087+0000 7fa1e1034640 1 -- 172.21.15.76:0/2137226996 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa1c806ae70 msgr2=0x7fa1c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:49.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.087+0000 7fa1e1034640 1 --2- 172.21.15.76:0/2137226996 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa1c806ae70 0x7fa1c806d330 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0005880 tx=0x7fa1d0009040 comp rx=0 tx=0).stop 2023-11-09T03:49:49.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.087+0000 7fa1e1034640 1 -- 172.21.15.76:0/2137226996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1dc0b0ae0 msgr2=0x7fa1dc155ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:49.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.087+0000 7fa1e1034640 1 --2- 172.21.15.76:0/2137226996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1dc0b0ae0 0x7fa1dc155ee0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc00d800 tx=0x7fa1cc00dcc0 comp rx=0 tx=0).stop 2023-11-09T03:49:49.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.087+0000 7fa1e1034640 1 -- 172.21.15.76:0/2137226996 shutdown_connections 2023-11-09T03:49:49.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.087+0000 7fa1e1034640 1 --2- 172.21.15.76:0/2137226996 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa1c806ae70 0x7fa1c806d330 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:49.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.087+0000 7fa1e1034640 1 --2- 172.21.15.76:0/2137226996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1dc159c70 0x7fa1dc157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:49.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.087+0000 7fa1e1034640 1 --2- 172.21.15.76:0/2137226996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1dc0b0ae0 0x7fa1dc155ee0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:49.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.087+0000 7fa1e1034640 1 --2- 172.21.15.76:0/2137226996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa1dc06db20 0x7fa1dc0ac290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:49.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.087+0000 7fa1e1034640 1 -- 172.21.15.76:0/2137226996 >> 172.21.15.76:0/2137226996 conn(0x7fa1dc0ce950 msgr2=0x7fa1dc15a920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:49.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.087+0000 7fa1e1034640 1 -- 172.21.15.76:0/2137226996 shutdown_connections 2023-11-09T03:49:49.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.087+0000 7fa1e1034640 1 -- 172.21.15.76:0/2137226996 wait complete. 2023-11-09T03:49:49.098 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump snaps --server 2023-11-09T03:49:49.221 INFO:teuthology.orchestra.run.smithi113.stdout:{"last_snap":2,"last_created":2,"last_destroyed":1,"pending_noop":[],"snaps":[{"snapid":2,"ino":1099511627777,"stamp":"2023-11-09T03:49:19.643623+0000","name":"s11","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-09T03:49:49.221 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 2, "last_destroyed": 1, "last_snap": 2, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s11", "snapid": 2, "stamp": "2023-11-09T03:49:19.643623+0000" } ] } 2023-11-09T03:49:49.222 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-09T03:49:49.222 INFO:tasks.cephfs.test_snapshots:testing snapserver mds_kill_mdstable_at=2 2023-11-09T03:49:49.223 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:49:49.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.322+0000 7f23bbc64640 1 Processor -- start 2023-11-09T03:49:49.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.323+0000 7f23bbc64640 1 -- start start 2023-11-09T03:49:49.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.323+0000 7f23bbc64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23b40b0ae0 0x7f23b40ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:49.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.323+0000 7f23bbc64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f23b406db20 0x7f23b415d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:49.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.323+0000 7f23bbc64640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b40b4110 con 0x7f23b4159c70 2023-11-09T03:49:49.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.323+0000 7f23bbc64640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b40b49b0 con 0x7f23b406db20 2023-11-09T03:49:49.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.323+0000 7f23bbc64640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b40b43f0 con 0x7f23b40b0ae0 2023-11-09T03:49:49.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.323+0000 7f23b9a00640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f23b4159c70 0x7f23b40b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46236/0 (socket says 172.21.15.76:46236) 2023-11-09T03:49:49.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.324+0000 7f23b9a00640 1 -- 172.21.15.76:0/509707440 learned_addr learned my addr 172.21.15.76:0/509707440 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:49:49.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.324+0000 7f23b91ff640 1 --2- 172.21.15.76:0/509707440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23b40b0ae0 0x7f23b40ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:49.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.324+0000 7f23ba201640 1 --2- 172.21.15.76:0/509707440 >> v2:172.21.15.113:3300/0 conn(0x7f23b406db20 0x7f23b415d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:49.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.324+0000 7f23b89fe640 1 -- 172.21.15.76:0/509707440 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1729708572 0 0) 0x7f23b40b4110 con 0x7f23b4159c70 2023-11-09T03:49:49.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.324+0000 7f23b89fe640 1 -- 172.21.15.76:0/509707440 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23a0003520 con 0x7f23b4159c70 2023-11-09T03:49:49.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.324+0000 7f23b91ff640 1 -- 172.21.15.76:0/509707440 >> v2:172.21.15.113:3300/0 conn(0x7f23b406db20 msgr2=0x7f23b415d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:49.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.324+0000 7f23b91ff640 1 --2- 172.21.15.76:0/509707440 >> v2:172.21.15.113:3300/0 conn(0x7f23b406db20 0x7f23b415d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:49.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.324+0000 7f23b91ff640 1 -- 172.21.15.76:0/509707440 >> v1:172.21.15.76:6789/0 conn(0x7f23b4159c70 legacy=0x7f23b40b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:49.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.324+0000 7f23b91ff640 1 -- 172.21.15.76:0/509707440 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b40b46d0 con 0x7f23b40b0ae0 2023-11-09T03:49:49.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.324+0000 7f23ba201640 1 --2- 172.21.15.76:0/509707440 >> v2:172.21.15.113:3300/0 conn(0x7f23b406db20 0x7f23b415d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:49:49.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.325+0000 7f23b91ff640 1 --2- 172.21.15.76:0/509707440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23b40b0ae0 0x7f23b40ce370 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f23a80097a0 tx=0x7f23a802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dd62506f4db1258 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:49.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.325+0000 7f23b89fe640 1 -- 172.21.15.76:0/509707440 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a8031830 con 0x7f23b40b0ae0 2023-11-09T03:49:49.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.325+0000 7f23b89fe640 1 -- 172.21.15.76:0/509707440 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a8031990 con 0x7f23b40b0ae0 2023-11-09T03:49:49.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.325+0000 7f23b89fe640 1 -- 172.21.15.76:0/509707440 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a8031d00 con 0x7f23b40b0ae0 2023-11-09T03:49:49.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.326+0000 7f23bbc64640 1 -- 172.21.15.76:0/509707440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23b40b0ae0 msgr2=0x7f23b40ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:49.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.326+0000 7f23bbc64640 1 --2- 172.21.15.76:0/509707440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23b40b0ae0 0x7f23b40ce370 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f23a80097a0 tx=0x7f23a802b8d0 comp rx=0 tx=0).stop 2023-11-09T03:49:49.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.326+0000 7f23bbc64640 1 -- 172.21.15.76:0/509707440 shutdown_connections 2023-11-09T03:49:49.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.326+0000 7f23bbc64640 1 --2- 172.21.15.76:0/509707440 >> v2:172.21.15.113:3300/0 conn(0x7f23b406db20 0x7f23b415d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:49.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.326+0000 7f23bbc64640 1 --2- 172.21.15.76:0/509707440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23b40b0ae0 0x7f23b40ce370 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:49.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.326+0000 7f23bbc64640 1 -- 172.21.15.76:0/509707440 >> 172.21.15.76:0/509707440 conn(0x7f23b40ce950 msgr2=0x7f23b40ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:49.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.326+0000 7f23bbc64640 1 -- 172.21.15.76:0/509707440 shutdown_connections 2023-11-09T03:49:49.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.326+0000 7f23bbc64640 1 -- 172.21.15.76:0/509707440 wait complete. 2023-11-09T03:49:49.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.327+0000 7f23bbc64640 1 Processor -- start 2023-11-09T03:49:49.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.327+0000 7f23bbc64640 1 -- start start 2023-11-09T03:49:49.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.328+0000 7f23bbc64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23b406db20 0x7f23b414dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:49.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.328+0000 7f23bbc64640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23b40b0ae0 0x7f23b414ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:49.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.328+0000 7f23bbc64640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23b4159c70 0x7f23b414f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:49.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.328+0000 7f23bbc64640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b415b270 con 0x7f23b40b0ae0 2023-11-09T03:49:49.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.328+0000 7f23bbc64640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b415b550 con 0x7f23b4159c70 2023-11-09T03:49:49.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.328+0000 7f23bbc64640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b415b830 con 0x7f23b406db20 2023-11-09T03:49:49.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.328+0000 7f23b91ff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23b40b0ae0 0x7f23b414ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:49.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.328+0000 7f23b91ff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23b40b0ae0 0x7f23b414ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47674/0 (socket says 172.21.15.76:47674) 2023-11-09T03:49:49.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.328+0000 7f23b91ff640 1 -- 172.21.15.76:0/4112849784 learned_addr learned my addr 172.21.15.76:0/4112849784 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:49.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.328+0000 7f23ba201640 1 --2- 172.21.15.76:0/4112849784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23b4159c70 0x7f23b414f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:49.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.328+0000 7f23b9a00640 1 --2- 172.21.15.76:0/4112849784 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23b406db20 0x7f23b414dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:49.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.328+0000 7f23b91ff640 1 -- 172.21.15.76:0/4112849784 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23b406db20 msgr2=0x7f23b414dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:49.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.328+0000 7f23b91ff640 1 --2- 172.21.15.76:0/4112849784 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23b406db20 0x7f23b414dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:49.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.328+0000 7f23b91ff640 1 -- 172.21.15.76:0/4112849784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23b4159c70 msgr2=0x7f23b414f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:49.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.328+0000 7f23b91ff640 1 --2- 172.21.15.76:0/4112849784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23b4159c70 0x7f23b414f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:49.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.328+0000 7f23b91ff640 1 -- 172.21.15.76:0/4112849784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ac009400 con 0x7f23b40b0ae0 2023-11-09T03:49:49.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.328+0000 7f23b9a00640 1 --2- 172.21.15.76:0/4112849784 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23b406db20 0x7f23b414dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:49.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.328+0000 7f23b91ff640 1 --2- 172.21.15.76:0/4112849784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23b40b0ae0 0x7f23b414ee10 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f23a8000c00 tx=0x7f23a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:49.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.328+0000 7f23a67fc640 1 -- 172.21.15.76:0/4112849784 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a803a630 con 0x7f23b40b0ae0 2023-11-09T03:49:49.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.328+0000 7f23a67fc640 1 -- 172.21.15.76:0/4112849784 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a803a790 con 0x7f23b40b0ae0 2023-11-09T03:49:49.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.328+0000 7f23bbc64640 1 -- 172.21.15.76:0/4112849784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a8009400 con 0x7f23b40b0ae0 2023-11-09T03:49:49.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.328+0000 7f23a67fc640 1 -- 172.21.15.76:0/4112849784 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a803ab40 con 0x7f23b40b0ae0 2023-11-09T03:49:49.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.328+0000 7f23bbc64640 1 -- 172.21.15.76:0/4112849784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b414e320 con 0x7f23b40b0ae0 2023-11-09T03:49:49.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.329+0000 7f23bbc64640 1 -- 172.21.15.76:0/4112849784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b415b690 con 0x7f23b40b0ae0 2023-11-09T03:49:49.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.330+0000 7f23a67fc640 1 -- 172.21.15.76:0/4112849784 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f23a803aca0 con 0x7f23b40b0ae0 2023-11-09T03:49:49.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.330+0000 7f23a67fc640 1 --2- 172.21.15.76:0/4112849784 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f239806ae70 0x7f239806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:49.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.330+0000 7f23a67fc640 1 -- 172.21.15.76:0/4112849784 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7f23a80ab6b0 con 0x7f23b40b0ae0 2023-11-09T03:49:49.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.330+0000 7f23b9a00640 1 --2- 172.21.15.76:0/4112849784 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f239806ae70 0x7f239806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:49.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.331+0000 7f23b9a00640 1 --2- 172.21.15.76:0/4112849784 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f239806ae70 0x7f239806d330 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f23b0005c00 tx=0x7f23b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:49.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.334+0000 7f23a67fc640 1 -- 172.21.15.76:0/4112849784 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f23b415b690 con 0x7f23b40b0ae0 2023-11-09T03:49:49.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.536+0000 7f23bbc64640 1 -- 172.21.15.76:0/4112849784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23b415b690 con 0x7f23b40b0ae0 2023-11-09T03:49:49.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.537+0000 7f23a67fc640 1 -- 172.21.15.76:0/4112849784 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f23b415b690 con 0x7f23b40b0ae0 2023-11-09T03:49:49.538 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:49:49.538 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":7773,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1663988060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1663988060},{"type":"v1","addr":"172.21.15.76:6835","nonce":1663988060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:49:45.047290+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5744,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5744":{"gid":5744,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6837/3280098776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3280098776},{"type":"v1","addr":"172.21.15.113:6837","nonce":3280098776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:49:49.540 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 57 2023-11-09T03:49:49.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.539+0000 7f23bbc64640 1 -- 172.21.15.76:0/4112849784 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f239806ae70 msgr2=0x7f239806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:49.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.539+0000 7f23bbc64640 1 --2- 172.21.15.76:0/4112849784 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f239806ae70 0x7f239806d330 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f23b0005c00 tx=0x7f23b0009040 comp rx=0 tx=0).stop 2023-11-09T03:49:49.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.539+0000 7f23bbc64640 1 -- 172.21.15.76:0/4112849784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23b40b0ae0 msgr2=0x7f23b414ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:49.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.539+0000 7f23bbc64640 1 --2- 172.21.15.76:0/4112849784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23b40b0ae0 0x7f23b414ee10 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f23a8000c00 tx=0x7f23a80046c0 comp rx=0 tx=0).stop 2023-11-09T03:49:49.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.539+0000 7f23bbc64640 1 -- 172.21.15.76:0/4112849784 shutdown_connections 2023-11-09T03:49:49.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.539+0000 7f23bbc64640 1 --2- 172.21.15.76:0/4112849784 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f239806ae70 0x7f239806d330 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:49.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.539+0000 7f23bbc64640 1 --2- 172.21.15.76:0/4112849784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23b4159c70 0x7f23b414f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:49.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.539+0000 7f23bbc64640 1 --2- 172.21.15.76:0/4112849784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23b40b0ae0 0x7f23b414ee10 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:49.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.539+0000 7f23bbc64640 1 --2- 172.21.15.76:0/4112849784 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f23b406db20 0x7f23b414dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:49.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.539+0000 7f23bbc64640 1 -- 172.21.15.76:0/4112849784 >> 172.21.15.76:0/4112849784 conn(0x7f23b40ce950 msgr2=0x7f23b40ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:49.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.539+0000 7f23bbc64640 1 -- 172.21.15.76:0/4112849784 shutdown_connections 2023-11-09T03:49:49.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.539+0000 7f23bbc64640 1 -- 172.21.15.76:0/4112849784 wait complete. 2023-11-09T03:49:49.550 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:0 true 2023-11-09T03:49:49.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.652+0000 7f4b3be44640 1 Processor -- start 2023-11-09T03:49:49.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.652+0000 7f4b3be44640 1 -- start start 2023-11-09T03:49:49.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.653+0000 7f4b3be44640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4b34153e60 0x7f4b3415dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:49.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.653+0000 7f4b3be44640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b340cde60 con 0x7f4b340cd6d0 2023-11-09T03:49:49.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.653+0000 7f4b3be44640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b34160f90 con 0x7f4b340b4880 2023-11-09T03:49:49.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.653+0000 7f4b3be44640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b3415e400 con 0x7f4b34153e60 2023-11-09T03:49:49.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.653+0000 7f4b393df640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f4b340cd6d0 0x7f4b34161420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46244/0 (socket says 172.21.15.76:46244) 2023-11-09T03:49:49.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.653+0000 7f4b393df640 1 -- 172.21.15.76:0/2714464028 learned_addr learned my addr 172.21.15.76:0/2714464028 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:49:49.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.654+0000 7f4b39be0640 1 --2- 172.21.15.76:0/2714464028 >> v2:172.21.15.113:3300/0 conn(0x7f4b34153e60 0x7f4b3415dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:49.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.654+0000 7f4b38bde640 1 -- 172.21.15.76:0/2714464028 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2260247183 0 0) 0x7f4b340cde60 con 0x7f4b340cd6d0 2023-11-09T03:49:49.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.654+0000 7f4b38bde640 1 -- 172.21.15.76:0/2714464028 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b20003520 con 0x7f4b340cd6d0 2023-11-09T03:49:49.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.654+0000 7f4b38bde640 1 -- 172.21.15.76:0/2714464028 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 225427256 0 0) 0x7f4b34160f90 con 0x7f4b340b4880 2023-11-09T03:49:49.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.654+0000 7f4b38bde640 1 -- 172.21.15.76:0/2714464028 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b340cde60 con 0x7f4b340b4880 2023-11-09T03:49:49.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.654+0000 7f4b38bde640 1 -- 172.21.15.76:0/2714464028 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 48258178 0 0) 0x7f4b20003520 con 0x7f4b340cd6d0 2023-11-09T03:49:49.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.654+0000 7f4b38bde640 1 -- 172.21.15.76:0/2714464028 >> v2:172.21.15.113:3300/0 conn(0x7f4b34153e60 msgr2=0x7f4b3415dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:49.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.654+0000 7f4b38bde640 1 --2- 172.21.15.76:0/2714464028 >> v2:172.21.15.113:3300/0 conn(0x7f4b34153e60 0x7f4b3415dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:49.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.654+0000 7f4b38bde640 1 -- 172.21.15.76:0/2714464028 >> v1:172.21.15.113:6789/0 conn(0x7f4b340b4880 legacy=0x7f4b340d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:49.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.654+0000 7f4b38bde640 1 -- 172.21.15.76:0/2714464028 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b340ce090 con 0x7f4b340cd6d0 2023-11-09T03:49:49.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.654+0000 7f4b38bde640 1 -- 172.21.15.76:0/2714464028 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f4b24001670 con 0x7f4b340cd6d0 2023-11-09T03:49:49.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.654+0000 7f4b38bde640 1 -- 172.21.15.76:0/2714464028 >> v1:172.21.15.76:6789/0 conn(0x7f4b340cd6d0 legacy=0x7f4b34161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:49.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.654+0000 7f4b38bde640 1 --2- 172.21.15.76:0/2714464028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b20003b00 0x7f4b20003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:49.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.654+0000 7f4b38bde640 1 --2- 172.21.15.76:0/2714464028 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b200047c0 0x7f4b20004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:49.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.654+0000 7f4b38bde640 1 --2- 172.21.15.76:0/2714464028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b20005180 0x7f4b2000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:49.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.654+0000 7f4b38bde640 1 -- 172.21.15.76:0/2714464028 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3415e400 con 0x7f4b20003b00 2023-11-09T03:49:49.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.654+0000 7f4b38bde640 1 -- 172.21.15.76:0/2714464028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2000c000 con 0x7f4b20005180 2023-11-09T03:49:49.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.654+0000 7f4b38bde640 1 -- 172.21.15.76:0/2714464028 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2000c1c0 con 0x7f4b200047c0 2023-11-09T03:49:49.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.654+0000 7f4b39be0640 1 --2- 172.21.15.76:0/2714464028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b20003b00 0x7f4b20003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:49.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.654+0000 7f4b393df640 1 --2- 172.21.15.76:0/2714464028 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b200047c0 0x7f4b20004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:49.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.654+0000 7f4b3a3e1640 1 --2- 172.21.15.76:0/2714464028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b20005180 0x7f4b2000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:49.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.654+0000 7f4b39be0640 1 -- 172.21.15.76:0/2714464028 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b200047c0 msgr2=0x7f4b20004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:49.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.654+0000 7f4b39be0640 1 --2- 172.21.15.76:0/2714464028 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b200047c0 0x7f4b20004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:49.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.654+0000 7f4b39be0640 1 -- 172.21.15.76:0/2714464028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b20005180 msgr2=0x7f4b2000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:49.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.654+0000 7f4b39be0640 1 --2- 172.21.15.76:0/2714464028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b20005180 0x7f4b2000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:49.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.654+0000 7f4b39be0640 1 -- 172.21.15.76:0/2714464028 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b20003520 con 0x7f4b20003b00 2023-11-09T03:49:49.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.654+0000 7f4b3a3e1640 1 -- 172.21.15.76:0/2714464028 reap_dead start 2023-11-09T03:49:49.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.655+0000 7f4b39be0640 1 --2- 172.21.15.76:0/2714464028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b20003b00 0x7f4b20003f00 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f4b30009fc0 tx=0x7f4b200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4028134d0db44c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:49.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.655+0000 7f4b38bde640 1 -- 172.21.15.76:0/2714464028 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3002cc20 con 0x7f4b20003b00 2023-11-09T03:49:49.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.655+0000 7f4b38bde640 1 -- 172.21.15.76:0/2714464028 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3000f040 con 0x7f4b20003b00 2023-11-09T03:49:49.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.655+0000 7f4b38bde640 1 -- 172.21.15.76:0/2714464028 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b30034440 con 0x7f4b20003b00 2023-11-09T03:49:49.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.656+0000 7f4b3be44640 1 -- 172.21.15.76:0/2714464028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b20003b00 msgr2=0x7f4b20003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:49.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.656+0000 7f4b3be44640 1 --2- 172.21.15.76:0/2714464028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b20003b00 0x7f4b20003f00 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f4b30009fc0 tx=0x7f4b200057f0 comp rx=0 tx=0).stop 2023-11-09T03:49:49.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.656+0000 7f4b3be44640 1 -- 172.21.15.76:0/2714464028 shutdown_connections 2023-11-09T03:49:49.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.656+0000 7f4b3be44640 1 --2- 172.21.15.76:0/2714464028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b20003b00 0x7f4b20003f00 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:49.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.656+0000 7f4b3be44640 1 -- 172.21.15.76:0/2714464028 >> 172.21.15.76:0/2714464028 conn(0x7f4b340cf880 msgr2=0x7f4b340cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:49.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.656+0000 7f4b3be44640 1 -- 172.21.15.76:0/2714464028 shutdown_connections 2023-11-09T03:49:49.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.656+0000 7f4b3be44640 1 -- 172.21.15.76:0/2714464028 wait complete. 2023-11-09T03:49:49.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.657+0000 7f4b3be44640 1 Processor -- start 2023-11-09T03:49:49.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.657+0000 7f4b3be44640 1 -- start start 2023-11-09T03:49:49.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.658+0000 7f4b3be44640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b20003b00 0x7f4b3414a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:49.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.658+0000 7f4b3be44640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b3414bbf0 0x7f4b341483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:49.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.658+0000 7f4b3be44640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b341460e0 0x7f4b34146590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:49.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.658+0000 7f4b3be44640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3414b170 con 0x7f4b20003b00 2023-11-09T03:49:49.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.658+0000 7f4b3be44640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b34147d20 con 0x7f4b341460e0 2023-11-09T03:49:49.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.658+0000 7f4b3be44640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3414b350 con 0x7f4b3414bbf0 2023-11-09T03:49:49.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.658+0000 7f4b39be0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b20003b00 0x7f4b3414a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:49.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.658+0000 7f4b39be0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b20003b00 0x7f4b3414a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47684/0 (socket says 172.21.15.76:47684) 2023-11-09T03:49:49.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.658+0000 7f4b39be0640 1 -- 172.21.15.76:0/1949301998 learned_addr learned my addr 172.21.15.76:0/1949301998 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:49.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.658+0000 7f4b3a3e1640 1 --2- 172.21.15.76:0/1949301998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b341460e0 0x7f4b34146590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:49.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.658+0000 7f4b393df640 1 --2- 172.21.15.76:0/1949301998 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b3414bbf0 0x7f4b341483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:49.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.658+0000 7f4b39be0640 1 -- 172.21.15.76:0/1949301998 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b3414bbf0 msgr2=0x7f4b341483b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:49.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.658+0000 7f4b39be0640 1 --2- 172.21.15.76:0/1949301998 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b3414bbf0 0x7f4b341483b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:49.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.658+0000 7f4b39be0640 1 -- 172.21.15.76:0/1949301998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b341460e0 msgr2=0x7f4b34146590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:49.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.658+0000 7f4b39be0640 1 --2- 172.21.15.76:0/1949301998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b341460e0 0x7f4b34146590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:49.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.658+0000 7f4b39be0640 1 -- 172.21.15.76:0/1949301998 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3414b630 con 0x7f4b20003b00 2023-11-09T03:49:49.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.658+0000 7f4b3a3e1640 1 --2- 172.21.15.76:0/1949301998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b341460e0 0x7f4b34146590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:49:49.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.658+0000 7f4b393df640 1 --2- 172.21.15.76:0/1949301998 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b3414bbf0 0x7f4b341483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:49:49.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.658+0000 7f4b39be0640 1 --2- 172.21.15.76:0/1949301998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b20003b00 0x7f4b3414a150 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f4b3000e230 tx=0x7f4b30009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:49.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.659+0000 7f4b2a7fc640 1 -- 172.21.15.76:0/1949301998 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b30034c50 con 0x7f4b20003b00 2023-11-09T03:49:49.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.659+0000 7f4b2a7fc640 1 -- 172.21.15.76:0/1949301998 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b30034dd0 con 0x7f4b20003b00 2023-11-09T03:49:49.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.659+0000 7f4b2a7fc640 1 -- 172.21.15.76:0/1949301998 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b30003bd0 con 0x7f4b20003b00 2023-11-09T03:49:49.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.659+0000 7f4b3be44640 1 -- 172.21.15.76:0/1949301998 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b3414b910 con 0x7f4b20003b00 2023-11-09T03:49:49.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.659+0000 7f4b3be44640 1 -- 172.21.15.76:0/1949301998 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b3414c070 con 0x7f4b20003b00 2023-11-09T03:49:49.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.660+0000 7f4b3be44640 1 -- 172.21.15.76:0/1949301998 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b34160c00 con 0x7f4b20003b00 2023-11-09T03:49:49.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.660+0000 7f4b2a7fc640 1 -- 172.21.15.76:0/1949301998 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4b30040450 con 0x7f4b20003b00 2023-11-09T03:49:49.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.660+0000 7f4b2a7fc640 1 --2- 172.21.15.76:0/1949301998 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4b1806ae70 0x7f4b1806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:49.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.661+0000 7f4b2a7fc640 1 -- 172.21.15.76:0/1949301998 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7f4b300ac450 con 0x7f4b20003b00 2023-11-09T03:49:49.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.661+0000 7f4b393df640 1 --2- 172.21.15.76:0/1949301998 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4b1806ae70 0x7f4b1806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:49.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.661+0000 7f4b393df640 1 --2- 172.21.15.76:0/1949301998 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4b1806ae70 0x7f4b1806d330 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4b24000f60 tx=0x7f4b24000ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:49.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.665+0000 7f4b2a7fc640 1 -- 172.21.15.76:0/1949301998 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4b34160c00 con 0x7f4b20003b00 2023-11-09T03:49:49.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:49.899+0000 7f4b3be44640 1 -- 172.21.15.76:0/1949301998 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"} v 0) v1 -- 0x7f4b34160c00 con 0x7f4b20003b00 2023-11-09T03:49:50.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.443+0000 7f4b2a7fc640 1 -- 172.21.15.76:0/1949301998 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"}]=0 freezing mds.5744 v58) v1 ==== 112+0+0 (secure 0 0 0) 0x7f4b34160c00 con 0x7f4b20003b00 2023-11-09T03:49:50.444 INFO:teuthology.orchestra.run.smithi076.stderr:freezing mds.5744 2023-11-09T03:49:50.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.446+0000 7f4b3be44640 1 -- 172.21.15.76:0/1949301998 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4b1806ae70 msgr2=0x7f4b1806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:50.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.446+0000 7f4b3be44640 1 --2- 172.21.15.76:0/1949301998 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4b1806ae70 0x7f4b1806d330 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4b24000f60 tx=0x7f4b24000ed0 comp rx=0 tx=0).stop 2023-11-09T03:49:50.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.446+0000 7f4b3be44640 1 -- 172.21.15.76:0/1949301998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b20003b00 msgr2=0x7f4b3414a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:50.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.446+0000 7f4b3be44640 1 --2- 172.21.15.76:0/1949301998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b20003b00 0x7f4b3414a150 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f4b3000e230 tx=0x7f4b30009750 comp rx=0 tx=0).stop 2023-11-09T03:49:50.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.446+0000 7f4b3be44640 1 -- 172.21.15.76:0/1949301998 shutdown_connections 2023-11-09T03:49:50.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.446+0000 7f4b3be44640 1 --2- 172.21.15.76:0/1949301998 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4b1806ae70 0x7f4b1806d330 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:50.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.446+0000 7f4b3be44640 1 --2- 172.21.15.76:0/1949301998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b341460e0 0x7f4b34146590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:50.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.446+0000 7f4b3be44640 1 --2- 172.21.15.76:0/1949301998 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b3414bbf0 0x7f4b341483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:50.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.446+0000 7f4b3be44640 1 --2- 172.21.15.76:0/1949301998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b20003b00 0x7f4b3414a150 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:50.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.446+0000 7f4b3be44640 1 -- 172.21.15.76:0/1949301998 >> 172.21.15.76:0/1949301998 conn(0x7f4b340cf880 msgr2=0x7f4b340cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:50.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.446+0000 7f4b3be44640 1 -- 172.21.15.76:0/1949301998 shutdown_connections 2023-11-09T03:49:50.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.446+0000 7f4b3be44640 1 -- 172.21.15.76:0/1949301998 wait complete. 2023-11-09T03:49:50.458 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_kill_mdstable_at 2 2023-11-09T03:49:50.582 INFO:teuthology.orchestra.run.smithi113.stdout:{"success":"mds_kill_mdstable_at = '2' (not observed, change may require restart) "} 2023-11-09T03:49:50.582 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '2' (not observed, change may require restart) " } 2023-11-09T03:49:50.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:49:50.583 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s12) 2023-11-09T03:49:50.586 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2023-11-09T03:49:50.606 INFO:tasks.ceph.mds.b.smithi113.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7f4018f1b640 time 2023-11-09T03:49:50.607520+0000 2023-11-09T03:49:50.606 INFO:tasks.ceph.mds.b.smithi113.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2023-11-09T03:49:50.606 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:49:50.607 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f4024b6efaf] 2023-11-09T03:49:50.607 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f4024b6f16d] 2023-11-09T03:49:50.607 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x55c54213f9ce] 2023-11-09T03:49:50.607 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: (MDSContext::complete(int)+0x5c) [0x55c54216fbcc] 2023-11-09T03:49:50.608 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (MDSIOContextBase::complete(int)+0x31c) [0x55c542178b0c] 2023-11-09T03:49:50.608 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: (MDSLogContextBase::complete(int)+0x53) [0x55c542178c63] 2023-11-09T03:49:50.608 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (Finisher::finisher_thread_entry()+0x175) [0x7f4024c164a5] 2023-11-09T03:49:50.608 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7f402429f802] 2023-11-09T03:49:50.608 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7f402423f450] 2023-11-09T03:49:50.608 INFO:tasks.ceph.mds.b.smithi113.stderr:2023-11-09T03:49:50.606+0000 7f4018f1b640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7f4018f1b640 time 2023-11-09T03:49:50.607520+0000 2023-11-09T03:49:50.609 INFO:tasks.ceph.mds.b.smithi113.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2023-11-09T03:49:50.609 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:49:50.609 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:49:50.609 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f4024b6efaf] 2023-11-09T03:49:50.609 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f4024b6f16d] 2023-11-09T03:49:50.610 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x55c54213f9ce] 2023-11-09T03:49:50.610 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: (MDSContext::complete(int)+0x5c) [0x55c54216fbcc] 2023-11-09T03:49:50.610 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (MDSIOContextBase::complete(int)+0x31c) [0x55c542178b0c] 2023-11-09T03:49:50.610 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: (MDSLogContextBase::complete(int)+0x53) [0x55c542178c63] 2023-11-09T03:49:50.611 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (Finisher::finisher_thread_entry()+0x175) [0x7f4024c164a5] 2023-11-09T03:49:50.611 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7f402429f802] 2023-11-09T03:49:50.611 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7f402423f450] 2023-11-09T03:49:50.611 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:49:50.611 INFO:tasks.ceph.mds.b.smithi113.stderr:*** Caught signal (Aborted) ** 2023-11-09T03:49:50.612 INFO:tasks.ceph.mds.b.smithi113.stderr: in thread 7f4018f1b640 thread_name:MR_Finisher 2023-11-09T03:49:50.612 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:49:50.612 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f4024254db0] 2023-11-09T03:49:50.612 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f40242a154c] 2023-11-09T03:49:50.612 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: raise() 2023-11-09T03:49:50.612 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: abort() 2023-11-09T03:49:50.613 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f4024b6f009] 2023-11-09T03:49:50.613 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f4024b6f16d] 2023-11-09T03:49:50.613 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x55c54213f9ce] 2023-11-09T03:49:50.613 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (MDSContext::complete(int)+0x5c) [0x55c54216fbcc] 2023-11-09T03:49:50.614 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: (MDSIOContextBase::complete(int)+0x31c) [0x55c542178b0c] 2023-11-09T03:49:50.614 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: (MDSLogContextBase::complete(int)+0x53) [0x55c542178c63] 2023-11-09T03:49:50.614 INFO:tasks.ceph.mds.b.smithi113.stderr: 11: (Finisher::finisher_thread_entry()+0x175) [0x7f4024c164a5] 2023-11-09T03:49:50.614 INFO:tasks.ceph.mds.b.smithi113.stderr: 12: /lib64/libc.so.6(+0x9f802) [0x7f402429f802] 2023-11-09T03:49:50.614 INFO:tasks.ceph.mds.b.smithi113.stderr: 13: /lib64/libc.so.6(+0x3f450) [0x7f402423f450] 2023-11-09T03:49:50.615 INFO:tasks.ceph.mds.b.smithi113.stderr:2023-11-09T03:49:50.607+0000 7f4018f1b640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:49:50.616 INFO:tasks.ceph.mds.b.smithi113.stderr: in thread 7f4018f1b640 thread_name:MR_Finisher 2023-11-09T03:49:50.616 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:49:50.616 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:49:50.616 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f4024254db0] 2023-11-09T03:49:50.616 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f40242a154c] 2023-11-09T03:49:50.617 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: raise() 2023-11-09T03:49:50.617 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: abort() 2023-11-09T03:49:50.617 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f4024b6f009] 2023-11-09T03:49:50.617 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f4024b6f16d] 2023-11-09T03:49:50.617 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x55c54213f9ce] 2023-11-09T03:49:50.618 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (MDSContext::complete(int)+0x5c) [0x55c54216fbcc] 2023-11-09T03:49:50.618 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: (MDSIOContextBase::complete(int)+0x31c) [0x55c542178b0c] 2023-11-09T03:49:50.618 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: (MDSLogContextBase::complete(int)+0x53) [0x55c542178c63] 2023-11-09T03:49:50.618 INFO:tasks.ceph.mds.b.smithi113.stderr: 11: (Finisher::finisher_thread_entry()+0x175) [0x7f4024c164a5] 2023-11-09T03:49:50.618 INFO:tasks.ceph.mds.b.smithi113.stderr: 12: /lib64/libc.so.6(+0x9f802) [0x7f402429f802] 2023-11-09T03:49:50.618 INFO:tasks.ceph.mds.b.smithi113.stderr: 13: /lib64/libc.so.6(+0x3f450) [0x7f402423f450] 2023-11-09T03:49:50.619 INFO:tasks.ceph.mds.b.smithi113.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:49:50.619 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:49:50.619 INFO:tasks.ceph.mds.b.smithi113.stderr: -2394> 2023-11-09T03:48:52.012+0000 7f402272e640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-11-09T03:49:50.619 INFO:tasks.ceph.mds.b.smithi113.stderr: -2393> 2023-11-09T03:48:52.012+0000 7f402272e640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6835/0: (98) Address already in use 2023-11-09T03:49:50.619 INFO:tasks.ceph.mds.b.smithi113.stderr: -154> 2023-11-09T03:49:49.024+0000 7f401c722640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:49:50.620 INFO:tasks.ceph.mds.b.smithi113.stderr: -152> 2023-11-09T03:49:49.024+0000 7f401c722640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-09T03:49:50.620 INFO:tasks.ceph.mds.b.smithi113.stderr: -1> 2023-11-09T03:49:50.606+0000 7f4018f1b640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7f4018f1b640 time 2023-11-09T03:49:50.607520+0000 2023-11-09T03:49:50.620 INFO:tasks.ceph.mds.b.smithi113.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2023-11-09T03:49:50.620 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:49:50.620 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:49:50.620 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f4024b6efaf] 2023-11-09T03:49:50.621 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f4024b6f16d] 2023-11-09T03:49:50.621 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x55c54213f9ce] 2023-11-09T03:49:50.621 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: (MDSContext::complete(int)+0x5c) [0x55c54216fbcc] 2023-11-09T03:49:50.621 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (MDSIOContextBase::complete(int)+0x31c) [0x55c542178b0c] 2023-11-09T03:49:50.621 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: (MDSLogContextBase::complete(int)+0x53) [0x55c542178c63] 2023-11-09T03:49:50.622 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (Finisher::finisher_thread_entry()+0x175) [0x7f4024c164a5] 2023-11-09T03:49:50.622 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7f402429f802] 2023-11-09T03:49:50.622 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7f402423f450] 2023-11-09T03:49:50.622 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:49:50.622 INFO:tasks.ceph.mds.b.smithi113.stderr: 0> 2023-11-09T03:49:50.607+0000 7f4018f1b640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:49:50.623 INFO:tasks.ceph.mds.b.smithi113.stderr: in thread 7f4018f1b640 thread_name:MR_Finisher 2023-11-09T03:49:50.623 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:49:50.623 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:49:50.623 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f4024254db0] 2023-11-09T03:49:50.623 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f40242a154c] 2023-11-09T03:49:50.623 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: raise() 2023-11-09T03:49:50.624 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: abort() 2023-11-09T03:49:50.624 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f4024b6f009] 2023-11-09T03:49:50.624 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f4024b6f16d] 2023-11-09T03:49:50.624 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x55c54213f9ce] 2023-11-09T03:49:50.624 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (MDSContext::complete(int)+0x5c) [0x55c54216fbcc] 2023-11-09T03:49:50.625 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: (MDSIOContextBase::complete(int)+0x31c) [0x55c542178b0c] 2023-11-09T03:49:50.625 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: (MDSLogContextBase::complete(int)+0x53) [0x55c542178c63] 2023-11-09T03:49:50.625 INFO:tasks.ceph.mds.b.smithi113.stderr: 11: (Finisher::finisher_thread_entry()+0x175) [0x7f4024c164a5] 2023-11-09T03:49:50.625 INFO:tasks.ceph.mds.b.smithi113.stderr: 12: /lib64/libc.so.6(+0x9f802) [0x7f402429f802] 2023-11-09T03:49:50.625 INFO:tasks.ceph.mds.b.smithi113.stderr: 13: /lib64/libc.so.6(+0x3f450) [0x7f402423f450] 2023-11-09T03:49:50.626 INFO:tasks.ceph.mds.b.smithi113.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:49:50.626 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:49:50.626 INFO:tasks.ceph.mds.b.smithi113.stderr: -2394> 2023-11-09T03:48:52.012+0000 7f402272e640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-11-09T03:49:50.626 INFO:tasks.ceph.mds.b.smithi113.stderr: -2393> 2023-11-09T03:48:52.012+0000 7f402272e640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6835/0: (98) Address already in use 2023-11-09T03:49:50.626 INFO:tasks.ceph.mds.b.smithi113.stderr: -154> 2023-11-09T03:49:49.024+0000 7f401c722640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:49:50.627 INFO:tasks.ceph.mds.b.smithi113.stderr: -152> 2023-11-09T03:49:49.024+0000 7f401c722640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-09T03:49:50.627 INFO:tasks.ceph.mds.b.smithi113.stderr: -1> 2023-11-09T03:49:50.606+0000 7f4018f1b640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7f4018f1b640 time 2023-11-09T03:49:50.607520+0000 2023-11-09T03:49:50.627 INFO:tasks.ceph.mds.b.smithi113.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2023-11-09T03:49:50.627 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:49:50.627 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:49:50.628 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f4024b6efaf] 2023-11-09T03:49:50.628 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f4024b6f16d] 2023-11-09T03:49:50.628 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x55c54213f9ce] 2023-11-09T03:49:50.628 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: (MDSContext::complete(int)+0x5c) [0x55c54216fbcc] 2023-11-09T03:49:50.628 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (MDSIOContextBase::complete(int)+0x31c) [0x55c542178b0c] 2023-11-09T03:49:50.629 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: (MDSLogContextBase::complete(int)+0x53) [0x55c542178c63] 2023-11-09T03:49:50.629 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (Finisher::finisher_thread_entry()+0x175) [0x7f4024c164a5] 2023-11-09T03:49:50.629 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7f402429f802] 2023-11-09T03:49:50.629 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7f402423f450] 2023-11-09T03:49:50.629 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:49:50.629 INFO:tasks.ceph.mds.b.smithi113.stderr: 0> 2023-11-09T03:49:50.607+0000 7f4018f1b640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:49:50.630 INFO:tasks.ceph.mds.b.smithi113.stderr: in thread 7f4018f1b640 thread_name:MR_Finisher 2023-11-09T03:49:50.630 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:49:50.630 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:49:50.630 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f4024254db0] 2023-11-09T03:49:50.630 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f40242a154c] 2023-11-09T03:49:50.631 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: raise() 2023-11-09T03:49:50.631 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: abort() 2023-11-09T03:49:50.631 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f4024b6f009] 2023-11-09T03:49:50.631 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f4024b6f16d] 2023-11-09T03:49:50.631 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x55c54213f9ce] 2023-11-09T03:49:50.631 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (MDSContext::complete(int)+0x5c) [0x55c54216fbcc] 2023-11-09T03:49:50.632 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: (MDSIOContextBase::complete(int)+0x31c) [0x55c542178b0c] 2023-11-09T03:49:50.632 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: (MDSLogContextBase::complete(int)+0x53) [0x55c542178c63] 2023-11-09T03:49:50.632 INFO:tasks.ceph.mds.b.smithi113.stderr: 11: (Finisher::finisher_thread_entry()+0x175) [0x7f4024c164a5] 2023-11-09T03:49:50.632 INFO:tasks.ceph.mds.b.smithi113.stderr: 12: /lib64/libc.so.6(+0x9f802) [0x7f402429f802] 2023-11-09T03:49:50.632 INFO:tasks.ceph.mds.b.smithi113.stderr: 13: /lib64/libc.so.6(+0x3f450) [0x7f402423f450] 2023-11-09T03:49:50.632 INFO:tasks.ceph.mds.b.smithi113.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:49:50.633 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:49:50.736 INFO:teuthology.orchestra.run.smithi076.stdout:{"mds_beacon_grace":"15.000000"} 2023-11-09T03:49:50.737 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2023-11-09T03:49:50.737 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:49:50.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.837+0000 7f5cd020e640 1 Processor -- start 2023-11-09T03:49:50.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.837+0000 7f5cd020e640 1 -- start start 2023-11-09T03:49:50.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.838+0000 7f5cd020e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5cc815ab70 0x7f5cc80cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:50.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.838+0000 7f5cd020e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5cc806a830 0x7f5cc80ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:50.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.838+0000 7f5cd020e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc806d310 con 0x7f5cc806eba0 2023-11-09T03:49:50.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.838+0000 7f5cd020e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc80cba70 con 0x7f5cc815ab70 2023-11-09T03:49:50.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.838+0000 7f5cd020e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc8161c50 con 0x7f5cc806a830 2023-11-09T03:49:50.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.838+0000 7f5cce7ab640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5cc806a830 0x7f5cc80ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:50.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.838+0000 7f5ccdfaa640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5cc815ab70 0x7f5cc80cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:50.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.838+0000 7f5ccd7a9640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f5cc806eba0 0x7f5cc806a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:42100/0 (socket says 172.21.15.76:42100) 2023-11-09T03:49:50.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.838+0000 7f5cce7ab640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5cc806a830 0x7f5cc80ae5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:58350/0 (socket says 172.21.15.76:58350) 2023-11-09T03:49:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.838+0000 7f5ccdfaa640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5cc815ab70 0x7f5cc80cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:42050/0 (socket says 172.21.15.76:42050) 2023-11-09T03:49:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.838+0000 7f5ccd7a9640 1 -- 172.21.15.76:0/1266209488 learned_addr learned my addr 172.21.15.76:0/1266209488 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:49:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.838+0000 7f5ccdfaa640 1 -- 172.21.15.76:0/1266209488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5cc806a830 msgr2=0x7f5cc80ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.838+0000 7f5ccdfaa640 1 --2- 172.21.15.76:0/1266209488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5cc806a830 0x7f5cc80ae5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.838+0000 7f5ccdfaa640 1 -- 172.21.15.76:0/1266209488 >> v1:172.21.15.113:6789/0 conn(0x7f5cc806eba0 legacy=0x7f5cc806a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.838+0000 7f5ccdfaa640 1 -- 172.21.15.76:0/1266209488 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc815d860 con 0x7f5cc815ab70 2023-11-09T03:49:50.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.838+0000 7f5cce7ab640 1 --2- 172.21.15.76:0/1266209488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5cc806a830 0x7f5cc80ae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:49:50.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.839+0000 7f5ccdfaa640 1 --2- 172.21.15.76:0/1266209488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cc815ab70 0x7f5cc80cbea0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4009790 tx=0x7f5cc402b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=476ff56ff0f69240 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:50.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.839+0000 7f5cccfa8640 1 -- 172.21.15.76:0/1266209488 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4031770 con 0x7f5cc815ab70 2023-11-09T03:49:50.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.839+0000 7f5cccfa8640 1 -- 172.21.15.76:0/1266209488 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc40318d0 con 0x7f5cc815ab70 2023-11-09T03:49:50.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.839+0000 7f5cccfa8640 1 -- 172.21.15.76:0/1266209488 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4031c60 con 0x7f5cc815ab70 2023-11-09T03:49:50.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.840+0000 7f5cd020e640 1 -- 172.21.15.76:0/1266209488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cc815ab70 msgr2=0x7f5cc80cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:50.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.840+0000 7f5cd020e640 1 --2- 172.21.15.76:0/1266209488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cc815ab70 0x7f5cc80cbea0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4009790 tx=0x7f5cc402b810 comp rx=0 tx=0).stop 2023-11-09T03:49:50.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.840+0000 7f5cd020e640 1 -- 172.21.15.76:0/1266209488 shutdown_connections 2023-11-09T03:49:50.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.840+0000 7f5cd020e640 1 --2- 172.21.15.76:0/1266209488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5cc806a830 0x7f5cc80ae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:50.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.840+0000 7f5cd020e640 1 --2- 172.21.15.76:0/1266209488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cc815ab70 0x7f5cc80cbea0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:50.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.840+0000 7f5cd020e640 1 -- 172.21.15.76:0/1266209488 >> 172.21.15.76:0/1266209488 conn(0x7f5cc8098d40 msgr2=0x7f5cc80cfa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:50.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.840+0000 7f5cd020e640 1 -- 172.21.15.76:0/1266209488 shutdown_connections 2023-11-09T03:49:50.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.840+0000 7f5cd020e640 1 -- 172.21.15.76:0/1266209488 wait complete. 2023-11-09T03:49:50.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.841+0000 7f5cd020e640 1 Processor -- start 2023-11-09T03:49:50.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.841+0000 7f5cd020e640 1 -- start start 2023-11-09T03:49:50.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.842+0000 7f5cd020e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5cc806a830 0x7f5cc81e8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:50.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.842+0000 7f5cd020e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cc806eba0 0x7f5cc8155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:50.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.842+0000 7f5cd020e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cc815ab70 0x7f5cc81e6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:50.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.842+0000 7f5cd020e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc81e74b0 con 0x7f5cc815ab70 2023-11-09T03:49:50.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.842+0000 7f5cd020e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc81e7790 con 0x7f5cc806eba0 2023-11-09T03:49:50.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.842+0000 7f5cd020e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc81e7a70 con 0x7f5cc806a830 2023-11-09T03:49:50.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.842+0000 7f5cce7ab640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cc815ab70 0x7f5cc81e6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:50.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.842+0000 7f5cce7ab640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cc815ab70 0x7f5cc81e6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47686/0 (socket says 172.21.15.76:47686) 2023-11-09T03:49:50.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.842+0000 7f5cce7ab640 1 -- 172.21.15.76:0/1841275682 learned_addr learned my addr 172.21.15.76:0/1841275682 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:50.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.842+0000 7f5ccdfaa640 1 --2- 172.21.15.76:0/1841275682 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5cc806a830 0x7f5cc81e8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:50.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.842+0000 7f5ccd7a9640 1 --2- 172.21.15.76:0/1841275682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cc806eba0 0x7f5cc8155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:50.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.842+0000 7f5cce7ab640 1 -- 172.21.15.76:0/1841275682 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5cc806a830 msgr2=0x7f5cc81e8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:50.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.842+0000 7f5cce7ab640 1 --2- 172.21.15.76:0/1841275682 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5cc806a830 0x7f5cc81e8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:50.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.842+0000 7f5cce7ab640 1 -- 172.21.15.76:0/1841275682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cc806eba0 msgr2=0x7f5cc8155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:50.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.842+0000 7f5cce7ab640 1 --2- 172.21.15.76:0/1841275682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cc806eba0 0x7f5cc8155ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:50.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.842+0000 7f5cce7ab640 1 -- 172.21.15.76:0/1841275682 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc0009a70 con 0x7f5cc815ab70 2023-11-09T03:49:50.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.843+0000 7f5cce7ab640 1 --2- 172.21.15.76:0/1841275682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cc815ab70 0x7f5cc81e6aa0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000ef90 tx=0x7f5cc000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:50.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.843+0000 7f5cbe7fc640 1 -- 172.21.15.76:0/1841275682 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0009cd0 con 0x7f5cc815ab70 2023-11-09T03:49:50.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.843+0000 7f5cbe7fc640 1 -- 172.21.15.76:0/1841275682 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc000f040 con 0x7f5cc815ab70 2023-11-09T03:49:50.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.843+0000 7f5cd020e640 1 -- 172.21.15.76:0/1841275682 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc40093a0 con 0x7f5cc815ab70 2023-11-09T03:49:50.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.843+0000 7f5cbe7fc640 1 -- 172.21.15.76:0/1841275682 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc00146a0 con 0x7f5cc815ab70 2023-11-09T03:49:50.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.843+0000 7f5cd020e640 1 -- 172.21.15.76:0/1841275682 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc81e8030 con 0x7f5cc815ab70 2023-11-09T03:49:50.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.844+0000 7f5cd020e640 1 -- 172.21.15.76:0/1841275682 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc8066b00 con 0x7f5cc815ab70 2023-11-09T03:49:50.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.844+0000 7f5cbe7fc640 1 -- 172.21.15.76:0/1841275682 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5cc0014800 con 0x7f5cc815ab70 2023-11-09T03:49:50.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.844+0000 7f5cbe7fc640 1 --2- 172.21.15.76:0/1841275682 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5cac06aea0 0x7f5cac06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:50.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.845+0000 7f5cbe7fc640 1 -- 172.21.15.76:0/1841275682 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7f5cc008a910 con 0x7f5cc815ab70 2023-11-09T03:49:50.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.845+0000 7f5ccdfaa640 1 --2- 172.21.15.76:0/1841275682 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5cac06aea0 0x7f5cac06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:50.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.845+0000 7f5ccdfaa640 1 --2- 172.21.15.76:0/1841275682 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5cac06aea0 0x7f5cac06d360 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4036000 tx=0x7f5cc4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:50.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:50.849+0000 7f5cbe7fc640 1 -- 172.21.15.76:0/1841275682 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5cc00554f0 con 0x7f5cc815ab70 2023-11-09T03:49:50.869 INFO:tasks.ceph.mds.b.smithi113.stderr:daemon-helper: command crashed with signal 6 2023-11-09T03:49:51.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:51.047+0000 7f5cd020e640 1 -- 172.21.15.76:0/1841275682 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5cc8074290 con 0x7f5cc815ab70 2023-11-09T03:49:51.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:51.048+0000 7f5cbe7fc640 1 -- 172.21.15.76:0/1841275682 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f5cc0058b10 con 0x7f5cc815ab70 2023-11-09T03:49:51.049 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:49:51.049 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":7773,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1663988060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1663988060},{"type":"v1","addr":"172.21.15.76:6835","nonce":1663988060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:49:49.901803+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5744,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5744":{"gid":5744,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6837/3280098776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3280098776},{"type":"v1","addr":"172.21.15.113:6837","nonce":3280098776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:49:51.049 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 58 2023-11-09T03:49:51.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:51.050+0000 7f5cd020e640 1 -- 172.21.15.76:0/1841275682 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5cac06aea0 msgr2=0x7f5cac06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:51.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:51.050+0000 7f5cd020e640 1 --2- 172.21.15.76:0/1841275682 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5cac06aea0 0x7f5cac06d360 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4036000 tx=0x7f5cc4008030 comp rx=0 tx=0).stop 2023-11-09T03:49:51.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:51.050+0000 7f5cd020e640 1 -- 172.21.15.76:0/1841275682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cc815ab70 msgr2=0x7f5cc81e6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:51.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:51.050+0000 7f5cd020e640 1 --2- 172.21.15.76:0/1841275682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cc815ab70 0x7f5cc81e6aa0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000ef90 tx=0x7f5cc000c490 comp rx=0 tx=0).stop 2023-11-09T03:49:51.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:51.050+0000 7f5cd020e640 1 -- 172.21.15.76:0/1841275682 shutdown_connections 2023-11-09T03:49:51.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:51.050+0000 7f5cd020e640 1 --2- 172.21.15.76:0/1841275682 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5cac06aea0 0x7f5cac06d360 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:51.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:51.050+0000 7f5cd020e640 1 --2- 172.21.15.76:0/1841275682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cc815ab70 0x7f5cc81e6aa0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:51.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:51.050+0000 7f5cd020e640 1 --2- 172.21.15.76:0/1841275682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cc806eba0 0x7f5cc8155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:51.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:51.050+0000 7f5cd020e640 1 --2- 172.21.15.76:0/1841275682 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5cc806a830 0x7f5cc81e8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:51.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:51.050+0000 7f5cd020e640 1 -- 172.21.15.76:0/1841275682 >> 172.21.15.76:0/1841275682 conn(0x7f5cc8098d40 msgr2=0x7f5cc806a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:51.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:51.050+0000 7f5cd020e640 1 -- 172.21.15.76:0/1841275682 shutdown_connections 2023-11-09T03:49:51.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:51.050+0000 7f5cd020e640 1 -- 172.21.15.76:0/1841275682 wait complete. 2023-11-09T03:49:51.061 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:49:52.757 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-11-09T03:49:56.062 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:49:56.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.171+0000 7f24ff0e0640 1 Processor -- start 2023-11-09T03:49:56.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.171+0000 7f24ff0e0640 1 -- start start 2023-11-09T03:49:56.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.172+0000 7f24ff0e0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f24f8159c70 0x7f24f80ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:56.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.172+0000 7f24ff0e0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24f806db20 0x7f24f80b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:56.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.172+0000 7f24ff0e0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f80b4110 con 0x7f24f80b0910 2023-11-09T03:49:56.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.172+0000 7f24ff0e0640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f80b49b0 con 0x7f24f8159c70 2023-11-09T03:49:56.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.172+0000 7f24ff0e0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f80b0da0 con 0x7f24f806db20 2023-11-09T03:49:56.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.173+0000 7f24f7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24f806db20 0x7f24f80b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:56.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.173+0000 7f24fce7c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f24f8159c70 0x7f24f80ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:56.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.173+0000 7f24fce7c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f24f8159c70 0x7f24f80ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:49174/0 (socket says 172.21.15.76:49174) 2023-11-09T03:49:56.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.173+0000 7f24f7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24f806db20 0x7f24f80b03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:52272/0 (socket says 172.21.15.76:52272) 2023-11-09T03:49:56.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.173+0000 7f24fce7c640 1 -- 172.21.15.76:0/4039829686 learned_addr learned my addr 172.21.15.76:0/4039829686 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:56.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.173+0000 7f24f77fe640 1 -- 172.21.15.76:0/4039829686 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2329499675 0 0) 0x7f24f80b4110 con 0x7f24f80b0910 2023-11-09T03:49:56.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.173+0000 7f24f77fe640 1 -- 172.21.15.76:0/4039829686 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24e4003520 con 0x7f24f80b0910 2023-11-09T03:49:56.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.173+0000 7f24f7fff640 1 -- 172.21.15.76:0/4039829686 >> v2:172.21.15.113:3300/0 conn(0x7f24f8159c70 msgr2=0x7f24f80ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:49:56.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.173+0000 7f24f7fff640 1 --2- 172.21.15.76:0/4039829686 >> v2:172.21.15.113:3300/0 conn(0x7f24f8159c70 0x7f24f80ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:56.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.173+0000 7f24f7fff640 1 -- 172.21.15.76:0/4039829686 >> v1:172.21.15.113:6789/0 conn(0x7f24f80b0910 legacy=0x7f24f815d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:56.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.173+0000 7f24f7fff640 1 -- 172.21.15.76:0/4039829686 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f80b43f0 con 0x7f24f806db20 2023-11-09T03:49:56.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.173+0000 7f24fce7c640 1 --2- 172.21.15.76:0/4039829686 >> v2:172.21.15.113:3300/0 conn(0x7f24f8159c70 0x7f24f80ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:49:56.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.174+0000 7f24f7fff640 1 --2- 172.21.15.76:0/4039829686 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24f806db20 0x7f24f80b03d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f24e00097a0 tx=0x7f24e002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6dd3f9c93844180 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:56.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.174+0000 7f24f77fe640 1 -- 172.21.15.76:0/4039829686 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e002d040 con 0x7f24f806db20 2023-11-09T03:49:56.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.174+0000 7f24f77fe640 1 -- 172.21.15.76:0/4039829686 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e0031810 con 0x7f24f806db20 2023-11-09T03:49:56.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.174+0000 7f24f77fe640 1 -- 172.21.15.76:0/4039829686 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e0031b80 con 0x7f24f806db20 2023-11-09T03:49:56.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.174+0000 7f24ff0e0640 1 -- 172.21.15.76:0/4039829686 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24f806db20 msgr2=0x7f24f80b03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:56.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.174+0000 7f24ff0e0640 1 --2- 172.21.15.76:0/4039829686 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24f806db20 0x7f24f80b03d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f24e00097a0 tx=0x7f24e002b8b0 comp rx=0 tx=0).stop 2023-11-09T03:49:56.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.174+0000 7f24ff0e0640 1 -- 172.21.15.76:0/4039829686 shutdown_connections 2023-11-09T03:49:56.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.174+0000 7f24ff0e0640 1 --2- 172.21.15.76:0/4039829686 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24f806db20 0x7f24f80b03d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:56.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.174+0000 7f24ff0e0640 1 --2- 172.21.15.76:0/4039829686 >> v2:172.21.15.113:3300/0 conn(0x7f24f8159c70 0x7f24f80ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:56.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.174+0000 7f24ff0e0640 1 -- 172.21.15.76:0/4039829686 >> 172.21.15.76:0/4039829686 conn(0x7f24f80ce950 msgr2=0x7f24f80ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:56.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.175+0000 7f24ff0e0640 1 -- 172.21.15.76:0/4039829686 shutdown_connections 2023-11-09T03:49:56.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.175+0000 7f24ff0e0640 1 -- 172.21.15.76:0/4039829686 wait complete. 2023-11-09T03:49:56.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.176+0000 7f24ff0e0640 1 Processor -- start 2023-11-09T03:49:56.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.176+0000 7f24ff0e0640 1 -- start start 2023-11-09T03:49:56.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.177+0000 7f24ff0e0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24f806db20 0x7f24f81f5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:56.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.177+0000 7f24ff0e0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f80b0910 0x7f24f80a8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:56.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.177+0000 7f24ff0e0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24f8159c70 0x7f24f81f2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:56.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.177+0000 7f24ff0e0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f81f62f0 con 0x7f24f806db20 2023-11-09T03:49:56.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.177+0000 7f24ff0e0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f81f65c0 con 0x7f24f80b0910 2023-11-09T03:49:56.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.177+0000 7f24ff0e0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f81f68a0 con 0x7f24f8159c70 2023-11-09T03:49:56.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.177+0000 7f24fce7c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24f806db20 0x7f24f81f5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:56.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.177+0000 7f24fce7c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24f806db20 0x7f24f81f5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43156/0 (socket says 172.21.15.76:43156) 2023-11-09T03:49:56.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.177+0000 7f24fce7c640 1 -- 172.21.15.76:0/3131562321 learned_addr learned my addr 172.21.15.76:0/3131562321 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:49:56.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.177+0000 7f24f7fff640 1 --2- 172.21.15.76:0/3131562321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f80b0910 0x7f24f80a8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:56.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.177+0000 7f24fd67d640 1 --2- 172.21.15.76:0/3131562321 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24f8159c70 0x7f24f81f2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:56.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.177+0000 7f24fce7c640 1 -- 172.21.15.76:0/3131562321 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24f8159c70 msgr2=0x7f24f81f2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:56.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.177+0000 7f24fce7c640 1 --2- 172.21.15.76:0/3131562321 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24f8159c70 0x7f24f81f2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:56.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.177+0000 7f24fce7c640 1 -- 172.21.15.76:0/3131562321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f80b0910 msgr2=0x7f24f80a8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:56.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.177+0000 7f24fce7c640 1 --2- 172.21.15.76:0/3131562321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f80b0910 0x7f24f80a8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:56.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.177+0000 7f24fce7c640 1 -- 172.21.15.76:0/3131562321 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e80094d0 con 0x7f24f806db20 2023-11-09T03:49:56.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.177+0000 7f24f7fff640 1 --2- 172.21.15.76:0/3131562321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f80b0910 0x7f24f80a8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:56.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.177+0000 7f24fd67d640 1 --2- 172.21.15.76:0/3131562321 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24f8159c70 0x7f24f81f2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:49:56.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.177+0000 7f24fce7c640 1 --2- 172.21.15.76:0/3131562321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24f806db20 0x7f24f81f5db0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f24e800e9d0 tx=0x7f24e800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:56.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.177+0000 7f24f57fa640 1 -- 172.21.15.76:0/3131562321 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e800ca60 con 0x7f24f806db20 2023-11-09T03:49:56.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.178+0000 7f24f57fa640 1 -- 172.21.15.76:0/3131562321 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e800cbc0 con 0x7f24f806db20 2023-11-09T03:49:56.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.178+0000 7f24f57fa640 1 -- 172.21.15.76:0/3131562321 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e8013470 con 0x7f24f806db20 2023-11-09T03:49:56.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.178+0000 7f24ff0e0640 1 -- 172.21.15.76:0/3131562321 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e0009400 con 0x7f24f806db20 2023-11-09T03:49:56.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.178+0000 7f24ff0e0640 1 -- 172.21.15.76:0/3131562321 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f81f2a80 con 0x7f24f806db20 2023-11-09T03:49:56.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.179+0000 7f24ff0e0640 1 -- 172.21.15.76:0/3131562321 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f8074290 con 0x7f24f806db20 2023-11-09T03:49:56.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.179+0000 7f24f57fa640 1 -- 172.21.15.76:0/3131562321 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f24e80135d0 con 0x7f24f806db20 2023-11-09T03:49:56.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.179+0000 7f24f57fa640 1 --2- 172.21.15.76:0/3131562321 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f24d806ae70 0x7f24d806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:49:56.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.179+0000 7f24f57fa640 1 -- 172.21.15.76:0/3131562321 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7f24e8089ed0 con 0x7f24f806db20 2023-11-09T03:49:56.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.180+0000 7f24f7fff640 1 --2- 172.21.15.76:0/3131562321 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f24d806ae70 0x7f24d806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:49:56.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.180+0000 7f24f7fff640 1 --2- 172.21.15.76:0/3131562321 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f24d806ae70 0x7f24d806d330 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f24e0036000 tx=0x7f24e003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:49:56.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.184+0000 7f24f57fa640 1 -- 172.21.15.76:0/3131562321 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f24e8054a00 con 0x7f24f806db20 2023-11-09T03:49:56.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.384+0000 7f24ff0e0640 1 -- 172.21.15.76:0/3131562321 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24f80b06f0 con 0x7f24f806db20 2023-11-09T03:49:56.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.385+0000 7f24f57fa640 1 -- 172.21.15.76:0/3131562321 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f24e8017090 con 0x7f24f806db20 2023-11-09T03:49:56.385 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:49:56.385 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":7773,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1663988060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1663988060},{"type":"v1","addr":"172.21.15.76:6835","nonce":1663988060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:49:49.901803+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5744,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5744":{"gid":5744,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6837/3280098776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3280098776},{"type":"v1","addr":"172.21.15.113:6837","nonce":3280098776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:49:56.386 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 58 2023-11-09T03:49:56.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.386+0000 7f24ff0e0640 1 -- 172.21.15.76:0/3131562321 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f24d806ae70 msgr2=0x7f24d806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:56.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.386+0000 7f24ff0e0640 1 --2- 172.21.15.76:0/3131562321 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f24d806ae70 0x7f24d806d330 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f24e0036000 tx=0x7f24e003c040 comp rx=0 tx=0).stop 2023-11-09T03:49:56.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.386+0000 7f24ff0e0640 1 -- 172.21.15.76:0/3131562321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24f806db20 msgr2=0x7f24f81f5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:49:56.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.386+0000 7f24ff0e0640 1 --2- 172.21.15.76:0/3131562321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24f806db20 0x7f24f81f5db0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f24e800e9d0 tx=0x7f24e800ee90 comp rx=0 tx=0).stop 2023-11-09T03:49:56.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.386+0000 7f24ff0e0640 1 -- 172.21.15.76:0/3131562321 shutdown_connections 2023-11-09T03:49:56.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.387+0000 7f24ff0e0640 1 --2- 172.21.15.76:0/3131562321 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f24d806ae70 0x7f24d806d330 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:56.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.387+0000 7f24ff0e0640 1 --2- 172.21.15.76:0/3131562321 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24f8159c70 0x7f24f81f2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:56.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.387+0000 7f24ff0e0640 1 --2- 172.21.15.76:0/3131562321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f80b0910 0x7f24f80a8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:56.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.387+0000 7f24ff0e0640 1 --2- 172.21.15.76:0/3131562321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24f806db20 0x7f24f81f5db0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:49:56.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.387+0000 7f24ff0e0640 1 -- 172.21.15.76:0/3131562321 >> 172.21.15.76:0/3131562321 conn(0x7f24f80ce950 msgr2=0x7f24f815cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:49:56.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.387+0000 7f24ff0e0640 1 -- 172.21.15.76:0/3131562321 shutdown_connections 2023-11-09T03:49:56.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:49:56.387+0000 7f24ff0e0640 1 -- 172.21.15.76:0/3131562321 wait complete. 2023-11-09T03:49:56.398 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:49:59.369 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~7s 2023-11-09T03:50:01.400 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:50:01.420 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:50:01.446 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:50:01.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.520+0000 7fc8d4bc9640 1 Processor -- start 2023-11-09T03:50:01.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.520+0000 7fc8d4bc9640 1 -- start start 2023-11-09T03:50:01.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.521+0000 7fc8d4bc9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc8d00cb5c0 0x7fc8d00ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:01.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.521+0000 7fc8d4bc9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d006a120 con 0x7fc8d015ce70 2023-11-09T03:50:01.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.521+0000 7fc8d4bc9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d00c9f40 con 0x7fc8d015d250 2023-11-09T03:50:01.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.521+0000 7fc8d4bc9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8d00cdc10 con 0x7fc8d00cb5c0 2023-11-09T03:50:01.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.521+0000 7fc8ce59c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc8d015ce70 0x7fc8d006d9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:42976/0 (socket says 172.21.15.76:42976) 2023-11-09T03:50:01.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.521+0000 7fc8ced9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc8d00cb5c0 0x7fc8d00ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:01.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.521+0000 7fc8ce59c640 1 -- 172.21.15.76:0/2121533275 learned_addr learned my addr 172.21.15.76:0/2121533275 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:50:01.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.521+0000 7fc8ced9d640 1 -- 172.21.15.76:0/2121533275 >> v1:172.21.15.113:6789/0 conn(0x7fc8d015d250 legacy=0x7fc8d00c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:01.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.521+0000 7fc8ced9d640 1 -- 172.21.15.76:0/2121533275 >> v1:172.21.15.76:6789/0 conn(0x7fc8d015ce70 legacy=0x7fc8d006d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:01.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.521+0000 7fc8cdd9b640 1 -- 172.21.15.76:0/2121533275 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4179531491 0 0) 0x7fc8d006a120 con 0x7fc8d015ce70 2023-11-09T03:50:01.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.521+0000 7fc8ced9d640 1 -- 172.21.15.76:0/2121533275 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d006a370 con 0x7fc8d00cb5c0 2023-11-09T03:50:01.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.522+0000 7fc8ced9d640 1 --2- 172.21.15.76:0/2121533275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8d00cb5c0 0x7fc8d00ccdb0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8009d90 tx=0x7fc8c802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c50ee3619572b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:01.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.522+0000 7fc8cdd9b640 1 -- 172.21.15.76:0/2121533275 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c802cd00 con 0x7fc8d00cb5c0 2023-11-09T03:50:01.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.522+0000 7fc8cdd9b640 1 -- 172.21.15.76:0/2121533275 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c802ce60 con 0x7fc8d00cb5c0 2023-11-09T03:50:01.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.522+0000 7fc8cdd9b640 1 -- 172.21.15.76:0/2121533275 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8034660 con 0x7fc8d00cb5c0 2023-11-09T03:50:01.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.523+0000 7fc8d4bc9640 1 -- 172.21.15.76:0/2121533275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8d00cb5c0 msgr2=0x7fc8d00ccdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:01.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.523+0000 7fc8d4bc9640 1 --2- 172.21.15.76:0/2121533275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8d00cb5c0 0x7fc8d00ccdb0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8009d90 tx=0x7fc8c802bf50 comp rx=0 tx=0).stop 2023-11-09T03:50:01.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.523+0000 7fc8d4bc9640 1 -- 172.21.15.76:0/2121533275 shutdown_connections 2023-11-09T03:50:01.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.523+0000 7fc8d4bc9640 1 --2- 172.21.15.76:0/2121533275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8d00cb5c0 0x7fc8d00ccdb0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:01.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.523+0000 7fc8d4bc9640 1 -- 172.21.15.76:0/2121533275 >> 172.21.15.76:0/2121533275 conn(0x7fc8d00b0cd0 msgr2=0x7fc8d00b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:01.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.523+0000 7fc8d4bc9640 1 -- 172.21.15.76:0/2121533275 shutdown_connections 2023-11-09T03:50:01.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.523+0000 7fc8d4bc9640 1 -- 172.21.15.76:0/2121533275 wait complete. 2023-11-09T03:50:01.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.524+0000 7fc8d4bc9640 1 Processor -- start 2023-11-09T03:50:01.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.524+0000 7fc8d4bc9640 1 -- start start 2023-11-09T03:50:01.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.525+0000 7fc8d4bc9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8d00cb5c0 0x7fc8d01587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:01.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.525+0000 7fc8d4bc9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8d015ce70 0x7fc8d015aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:01.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.525+0000 7fc8d4bc9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8d015d250 0x7fc8d0159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:01.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.525+0000 7fc8d4bc9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d00c73c0 con 0x7fc8d00cb5c0 2023-11-09T03:50:01.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.525+0000 7fc8d4bc9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d0158ce0 con 0x7fc8d015ce70 2023-11-09T03:50:01.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.525+0000 7fc8d4bc9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d00c67b0 con 0x7fc8d015d250 2023-11-09T03:50:01.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.525+0000 7fc8ce59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8d00cb5c0 0x7fc8d01587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:01.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.525+0000 7fc8ce59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8d00cb5c0 0x7fc8d01587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43186/0 (socket says 172.21.15.76:43186) 2023-11-09T03:50:01.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.525+0000 7fc8ce59c640 1 -- 172.21.15.76:0/580308289 learned_addr learned my addr 172.21.15.76:0/580308289 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:01.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.525+0000 7fc8c7fff640 1 --2- 172.21.15.76:0/580308289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8d015ce70 0x7fc8d015aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:01.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.525+0000 7fc8ced9d640 1 --2- 172.21.15.76:0/580308289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8d015d250 0x7fc8d0159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:01.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.526+0000 7fc8ce59c640 1 -- 172.21.15.76:0/580308289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8d015d250 msgr2=0x7fc8d0159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:01.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.526+0000 7fc8ce59c640 1 --2- 172.21.15.76:0/580308289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8d015d250 0x7fc8d0159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:01.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.526+0000 7fc8ce59c640 1 -- 172.21.15.76:0/580308289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8d015ce70 msgr2=0x7fc8d015aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:01.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.526+0000 7fc8ce59c640 1 --2- 172.21.15.76:0/580308289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8d015ce70 0x7fc8d015aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:01.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.526+0000 7fc8ce59c640 1 -- 172.21.15.76:0/580308289 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c8009a70 con 0x7fc8d00cb5c0 2023-11-09T03:50:01.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.526+0000 7fc8ce59c640 1 --2- 172.21.15.76:0/580308289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8d00cb5c0 0x7fc8d01587a0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fc8c000c4a0 tx=0x7fc8c000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:01.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.526+0000 7fc8c6ffd640 1 -- 172.21.15.76:0/580308289 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c00074e0 con 0x7fc8d00cb5c0 2023-11-09T03:50:01.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.526+0000 7fc8c6ffd640 1 -- 172.21.15.76:0/580308289 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c0007640 con 0x7fc8d00cb5c0 2023-11-09T03:50:01.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.526+0000 7fc8d4bc9640 1 -- 172.21.15.76:0/580308289 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8d00c6a90 con 0x7fc8d00cb5c0 2023-11-09T03:50:01.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.526+0000 7fc8c6ffd640 1 -- 172.21.15.76:0/580308289 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0004a00 con 0x7fc8d00cb5c0 2023-11-09T03:50:01.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.526+0000 7fc8d4bc9640 1 -- 172.21.15.76:0/580308289 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d00c7050 con 0x7fc8d00cb5c0 2023-11-09T03:50:01.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.527+0000 7fc8d4bc9640 1 -- 172.21.15.76:0/580308289 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8d0074290 con 0x7fc8d00cb5c0 2023-11-09T03:50:01.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.527+0000 7fc8c6ffd640 1 -- 172.21.15.76:0/580308289 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc8c0004b60 con 0x7fc8d00cb5c0 2023-11-09T03:50:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.528+0000 7fc8c6ffd640 1 --2- 172.21.15.76:0/580308289 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc8b006ae70 0x7fc8b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.528+0000 7fc8c6ffd640 1 -- 172.21.15.76:0/580308289 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7fc8c0088c30 con 0x7fc8d00cb5c0 2023-11-09T03:50:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.528+0000 7fc8c7fff640 1 --2- 172.21.15.76:0/580308289 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc8b006ae70 0x7fc8b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.529+0000 7fc8c7fff640 1 --2- 172.21.15.76:0/580308289 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc8b006ae70 0x7fc8b006d330 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00cb4b0 tx=0x7fc8b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:01.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.532+0000 7fc8c6ffd640 1 -- 172.21.15.76:0/580308289 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc8c0053810 con 0x7fc8d00cb5c0 2023-11-09T03:50:01.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.735+0000 7fc8d4bc9640 1 -- 172.21.15.76:0/580308289 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8d0066170 con 0x7fc8d00cb5c0 2023-11-09T03:50:01.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.736+0000 7fc8c6ffd640 1 -- 172.21.15.76:0/580308289 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4934 (secure 0 0 0) 0x7fc8c0018090 con 0x7fc8d00cb5c0 2023-11-09T03:50:01.737 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:50:01.737 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":7773,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1663988060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1663988060},{"type":"v1","addr":"172.21.15.76:6835","nonce":1663988060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:49:49.901803+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5744,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5744":{"gid":5744,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6837/3280098776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3280098776},{"type":"v1","addr":"172.21.15.113:6837","nonce":3280098776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:50:01.737 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 58 2023-11-09T03:50:01.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.738+0000 7fc8d4bc9640 1 -- 172.21.15.76:0/580308289 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc8b006ae70 msgr2=0x7fc8b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:01.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.738+0000 7fc8d4bc9640 1 --2- 172.21.15.76:0/580308289 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc8b006ae70 0x7fc8b006d330 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00cb4b0 tx=0x7fc8b8009040 comp rx=0 tx=0).stop 2023-11-09T03:50:01.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.738+0000 7fc8d4bc9640 1 -- 172.21.15.76:0/580308289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8d00cb5c0 msgr2=0x7fc8d01587a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:01.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.738+0000 7fc8d4bc9640 1 --2- 172.21.15.76:0/580308289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8d00cb5c0 0x7fc8d01587a0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fc8c000c4a0 tx=0x7fc8c000c960 comp rx=0 tx=0).stop 2023-11-09T03:50:01.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.738+0000 7fc8d4bc9640 1 -- 172.21.15.76:0/580308289 shutdown_connections 2023-11-09T03:50:01.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.738+0000 7fc8d4bc9640 1 --2- 172.21.15.76:0/580308289 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc8b006ae70 0x7fc8b006d330 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:01.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.738+0000 7fc8d4bc9640 1 --2- 172.21.15.76:0/580308289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8d015d250 0x7fc8d0159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:01.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.738+0000 7fc8d4bc9640 1 --2- 172.21.15.76:0/580308289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8d015ce70 0x7fc8d015aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:01.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.738+0000 7fc8d4bc9640 1 --2- 172.21.15.76:0/580308289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8d00cb5c0 0x7fc8d01587a0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:01.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.738+0000 7fc8d4bc9640 1 -- 172.21.15.76:0/580308289 >> 172.21.15.76:0/580308289 conn(0x7fc8d00b0cd0 msgr2=0x7fc8d00ccd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:01.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.738+0000 7fc8d4bc9640 1 -- 172.21.15.76:0/580308289 shutdown_connections 2023-11-09T03:50:01.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:01.738+0000 7fc8d4bc9640 1 -- 172.21.15.76:0/580308289 wait complete. 2023-11-09T03:50:01.749 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:50:05.980 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~13s 2023-11-09T03:50:06.750 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:50:06.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.862+0000 7f6fd2207640 1 Processor -- start 2023-11-09T03:50:06.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.862+0000 7f6fd2207640 1 -- start start 2023-11-09T03:50:06.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.863+0000 7f6fd2207640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6fcc159c70 0x7f6fcc0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:06.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.863+0000 7f6fd2207640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fcc0b0ae0 0x7f6fcc15d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:06.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.863+0000 7f6fd2207640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fcc0b4110 con 0x7f6fcc06db20 2023-11-09T03:50:06.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.863+0000 7f6fd2207640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fcc0b49b0 con 0x7f6fcc159c70 2023-11-09T03:50:06.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.863+0000 7f6fd2207640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc0b43f0 con 0x7f6fcc0b0ae0 2023-11-09T03:50:06.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.863+0000 7f6fcb7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6fcc159c70 0x7f6fcc0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:06.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.863+0000 7f6fcaffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f6fcc06db20 0x7f6fcc0b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48918/0 (socket says 172.21.15.76:48918) 2023-11-09T03:50:06.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.863+0000 7f6fcaffd640 1 -- 172.21.15.76:0/1694359256 learned_addr learned my addr 172.21.15.76:0/1694359256 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:50:06.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.864+0000 7f6fcbfff640 1 --2- 172.21.15.76:0/1694359256 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fcc0b0ae0 0x7f6fcc15d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:06.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.864+0000 7f6fcb7fe640 1 -- 172.21.15.76:0/1694359256 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fcc0b0ae0 msgr2=0x7f6fcc15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:06.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.864+0000 7f6fca7fc640 1 -- 172.21.15.76:0/1694359256 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1077973386 0 0) 0x7f6fcc0b4110 con 0x7f6fcc06db20 2023-11-09T03:50:06.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.864+0000 7f6fcb7fe640 1 --2- 172.21.15.76:0/1694359256 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fcc0b0ae0 0x7f6fcc15d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:06.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.864+0000 7f6fcb7fe640 1 -- 172.21.15.76:0/1694359256 >> v1:172.21.15.76:6789/0 conn(0x7f6fcc06db20 legacy=0x7f6fcc0b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:06.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.864+0000 7f6fcb7fe640 1 -- 172.21.15.76:0/1694359256 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fcc0b46d0 con 0x7f6fcc159c70 2023-11-09T03:50:06.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.864+0000 7f6fcbfff640 1 --2- 172.21.15.76:0/1694359256 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fcc0b0ae0 0x7f6fcc15d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:50:06.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.864+0000 7f6fcb7fe640 1 --2- 172.21.15.76:0/1694359256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fcc159c70 0x7f6fcc0ce370 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0097a0 tx=0x7f6fbc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be35331234a2822 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:06.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.864+0000 7f6fca7fc640 1 -- 172.21.15.76:0/1694359256 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc031830 con 0x7f6fcc159c70 2023-11-09T03:50:06.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.865+0000 7f6fca7fc640 1 -- 172.21.15.76:0/1694359256 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fbc031990 con 0x7f6fcc159c70 2023-11-09T03:50:06.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.865+0000 7f6fca7fc640 1 -- 172.21.15.76:0/1694359256 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc031d20 con 0x7f6fcc159c70 2023-11-09T03:50:06.871 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.865+0000 7f6fd2207640 1 -- 172.21.15.76:0/1694359256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fcc159c70 msgr2=0x7f6fcc0ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:06.871 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.865+0000 7f6fd2207640 1 --2- 172.21.15.76:0/1694359256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fcc159c70 0x7f6fcc0ce370 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0097a0 tx=0x7f6fbc02b8d0 comp rx=0 tx=0).stop 2023-11-09T03:50:06.871 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.865+0000 7f6fd2207640 1 -- 172.21.15.76:0/1694359256 shutdown_connections 2023-11-09T03:50:06.871 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.865+0000 7f6fd2207640 1 --2- 172.21.15.76:0/1694359256 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fcc0b0ae0 0x7f6fcc15d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:06.871 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.865+0000 7f6fd2207640 1 --2- 172.21.15.76:0/1694359256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fcc159c70 0x7f6fcc0ce370 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:06.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.865+0000 7f6fd2207640 1 -- 172.21.15.76:0/1694359256 >> 172.21.15.76:0/1694359256 conn(0x7f6fcc0ce950 msgr2=0x7f6fcc0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:06.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.865+0000 7f6fd2207640 1 -- 172.21.15.76:0/1694359256 shutdown_connections 2023-11-09T03:50:06.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.865+0000 7f6fd2207640 1 -- 172.21.15.76:0/1694359256 wait complete. 2023-11-09T03:50:06.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.866+0000 7f6fd2207640 1 Processor -- start 2023-11-09T03:50:06.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.866+0000 7f6fd2207640 1 -- start start 2023-11-09T03:50:06.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.867+0000 7f6fd2207640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fcc06db20 0x7f6fcc148d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:06.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.867+0000 7f6fd2207640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fcc0b0ae0 0x7f6fcc14c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:06.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.867+0000 7f6fd2207640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fcc159c70 0x7f6fcc0b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:06.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.867+0000 7f6fd2207640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc15d1c0 con 0x7f6fcc06db20 2023-11-09T03:50:06.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.867+0000 7f6fd2207640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc1466a0 con 0x7f6fcc159c70 2023-11-09T03:50:06.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.867+0000 7f6fd2207640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc15c920 con 0x7f6fcc0b0ae0 2023-11-09T03:50:06.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.867+0000 7f6fcb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fcc06db20 0x7f6fcc148d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:06.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.867+0000 7f6fcb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fcc06db20 0x7f6fcc148d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50608/0 (socket says 172.21.15.76:50608) 2023-11-09T03:50:06.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.867+0000 7f6fcb7fe640 1 -- 172.21.15.76:0/2742767876 learned_addr learned my addr 172.21.15.76:0/2742767876 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:06.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.868+0000 7f6fcb7fe640 1 -- 172.21.15.76:0/2742767876 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fcc0b0ae0 msgr2=0x7f6fcc14c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:06.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.868+0000 7f6fcaffd640 1 --2- 172.21.15.76:0/2742767876 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fcc0b0ae0 0x7f6fcc14c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:06.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.868+0000 7f6fcbfff640 1 --2- 172.21.15.76:0/2742767876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fcc159c70 0x7f6fcc0b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:06.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.868+0000 7f6fcb7fe640 1 --2- 172.21.15.76:0/2742767876 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fcc0b0ae0 0x7f6fcc14c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:06.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.868+0000 7f6fcb7fe640 1 -- 172.21.15.76:0/2742767876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fcc159c70 msgr2=0x7f6fcc0b21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:06.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.868+0000 7f6fcb7fe640 1 --2- 172.21.15.76:0/2742767876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fcc159c70 0x7f6fcc0b21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:06.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.868+0000 7f6fcb7fe640 1 -- 172.21.15.76:0/2742767876 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc009400 con 0x7f6fcc06db20 2023-11-09T03:50:06.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.868+0000 7f6fcb7fe640 1 --2- 172.21.15.76:0/2742767876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fcc06db20 0x7f6fcc148d20 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0097a0 tx=0x7f6fbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:06.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.868+0000 7f6fb3fff640 1 -- 172.21.15.76:0/2742767876 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc004230 con 0x7f6fcc06db20 2023-11-09T03:50:06.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.868+0000 7f6fb3fff640 1 -- 172.21.15.76:0/2742767876 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fbc004390 con 0x7f6fcc06db20 2023-11-09T03:50:06.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.868+0000 7f6fd2207640 1 -- 172.21.15.76:0/2742767876 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fcc15cc00 con 0x7f6fcc06db20 2023-11-09T03:50:06.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.868+0000 7f6fb3fff640 1 -- 172.21.15.76:0/2742767876 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc03a930 con 0x7f6fcc06db20 2023-11-09T03:50:06.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.868+0000 7f6fd2207640 1 -- 172.21.15.76:0/2742767876 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fcc1469b0 con 0x7f6fcc06db20 2023-11-09T03:50:06.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.869+0000 7f6fd2207640 1 -- 172.21.15.76:0/2742767876 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fcc146810 con 0x7f6fcc06db20 2023-11-09T03:50:06.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.869+0000 7f6fb3fff640 1 -- 172.21.15.76:0/2742767876 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6fbc03aa90 con 0x7f6fcc06db20 2023-11-09T03:50:06.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.870+0000 7f6fb3fff640 1 --2- 172.21.15.76:0/2742767876 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6fb806ae70 0x7f6fb806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:06.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.870+0000 7f6fb3fff640 1 -- 172.21.15.76:0/2742767876 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7f6fbc0abea0 con 0x7f6fcc06db20 2023-11-09T03:50:06.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.870+0000 7f6fcaffd640 1 --2- 172.21.15.76:0/2742767876 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6fb806ae70 0x7f6fb806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:06.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.871+0000 7f6fcaffd640 1 --2- 172.21.15.76:0/2742767876 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6fb806ae70 0x7f6fb806d330 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0007fd0 tx=0x7f6fc0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:06.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:06.874+0000 7f6fb3fff640 1 -- 172.21.15.76:0/2742767876 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6fcc146810 con 0x7f6fcc06db20 2023-11-09T03:50:07.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.075+0000 7f6fd2207640 1 -- 172.21.15.76:0/2742767876 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6fcc146810 con 0x7f6fcc06db20 2023-11-09T03:50:07.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.075+0000 7f6fb3fff640 1 -- 172.21.15.76:0/2742767876 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4982 (secure 0 0 0) 0x7f6fcc146810 con 0x7f6fcc06db20 2023-11-09T03:50:07.075 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:50:07.075 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":59,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":7773,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1663988060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1663988060},{"type":"v1","addr":"172.21.15.76:6835","nonce":1663988060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:50:06.433886+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5744,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5744":{"gid":5744,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6837/3280098776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3280098776},{"type":"v1","addr":"172.21.15.113:6837","nonce":3280098776}]},"join_fscid":-1,"laggy_since":"2023-11-09T03:50:06.433886+0000","export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:50:07.076 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 59 2023-11-09T03:50:07.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.077+0000 7f6fd2207640 1 -- 172.21.15.76:0/2742767876 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6fb806ae70 msgr2=0x7f6fb806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:07.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.077+0000 7f6fd2207640 1 --2- 172.21.15.76:0/2742767876 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6fb806ae70 0x7f6fb806d330 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0007fd0 tx=0x7f6fc0009040 comp rx=0 tx=0).stop 2023-11-09T03:50:07.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.077+0000 7f6fd2207640 1 -- 172.21.15.76:0/2742767876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fcc06db20 msgr2=0x7f6fcc148d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:07.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.077+0000 7f6fd2207640 1 --2- 172.21.15.76:0/2742767876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fcc06db20 0x7f6fcc148d20 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0097a0 tx=0x7f6fbc0046c0 comp rx=0 tx=0).stop 2023-11-09T03:50:07.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.077+0000 7f6fd2207640 1 -- 172.21.15.76:0/2742767876 shutdown_connections 2023-11-09T03:50:07.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.077+0000 7f6fd2207640 1 --2- 172.21.15.76:0/2742767876 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6fb806ae70 0x7f6fb806d330 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:07.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.077+0000 7f6fd2207640 1 --2- 172.21.15.76:0/2742767876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fcc159c70 0x7f6fcc0b21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:07.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.077+0000 7f6fd2207640 1 --2- 172.21.15.76:0/2742767876 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fcc0b0ae0 0x7f6fcc14c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:07.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.077+0000 7f6fd2207640 1 --2- 172.21.15.76:0/2742767876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fcc06db20 0x7f6fcc148d20 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:07.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.077+0000 7f6fd2207640 1 -- 172.21.15.76:0/2742767876 >> 172.21.15.76:0/2742767876 conn(0x7f6fcc0ce950 msgr2=0x7f6fcc0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:07.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.077+0000 7f6fd2207640 1 -- 172.21.15.76:0/2742767876 shutdown_connections 2023-11-09T03:50:07.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.077+0000 7f6fd2207640 1 -- 172.21.15.76:0/2742767876 wait complete. 2023-11-09T03:50:07.087 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2023-11-09T03:50:07.088 DEBUG:teuthology.orchestra.run.smithi113:> sudo sysctl -n kernel.core_pattern 2023-11-09T03:50:07.120 INFO:teuthology.orchestra.run.smithi113.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-09T03:50:07.122 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2023-11-09T03:50:07.122 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2023-11-09T03:50:07.195 INFO:teuthology.orchestra.run.smithi113.stdout:1699501790.95542.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i b', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2023-11-09T03:50:07.197 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1699501790.95542.core'] 2023-11-09T03:50:07.197 INFO:tasks.cephfs.cephfs_test_case:Found core file 1699501790.95542.core, deleting it 2023-11-09T03:50:07.198 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1699501790.95542.core 2023-11-09T03:50:07.269 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:0 2023-11-09T03:50:07.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.369+0000 7f5a0c89b640 1 Processor -- start 2023-11-09T03:50:07.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.369+0000 7f5a0c89b640 1 -- start start 2023-11-09T03:50:07.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.370+0000 7f5a0c89b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5a080c4ff0 0x7f5a080cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:07.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.370+0000 7f5a0c89b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a080adac0 0x7f5a080adf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:07.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.370+0000 7f5a0c89b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a0815aff0 con 0x7f5a08159c00 2023-11-09T03:50:07.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.370+0000 7f5a0c89b640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a080ae460 con 0x7f5a080c4ff0 2023-11-09T03:50:07.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.370+0000 7f5a0c89b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0815a3e0 con 0x7f5a080adac0 2023-11-09T03:50:07.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.371+0000 7f5a05d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5a080c4ff0 0x7f5a080cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:07.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.371+0000 7f5a05d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5a080c4ff0 0x7f5a080cc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50610/0 (socket says 172.21.15.76:50610) 2023-11-09T03:50:07.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.371+0000 7f5a05d9b640 1 -- 172.21.15.76:0/3829148405 learned_addr learned my addr 172.21.15.76:0/3829148405 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:07.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.371+0000 7f5a06d9d640 1 --2- 172.21.15.76:0/3829148405 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a080adac0 0x7f5a080adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:07.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.371+0000 7f5a05d9b640 1 -- 172.21.15.76:0/3829148405 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a080adac0 msgr2=0x7f5a080adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:07.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.371+0000 7f5a05d9b640 1 --2- 172.21.15.76:0/3829148405 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a080adac0 0x7f5a080adf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:07.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.371+0000 7f5a05d9b640 1 -- 172.21.15.76:0/3829148405 >> v1:172.21.15.113:6789/0 conn(0x7f5a08159c00 legacy=0x7f5a080c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:07.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.371+0000 7f5a05d9b640 1 -- 172.21.15.76:0/3829148405 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0815a6c0 con 0x7f5a080c4ff0 2023-11-09T03:50:07.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.372+0000 7f5a05d9b640 1 --2- 172.21.15.76:0/3829148405 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a080c4ff0 0x7f5a080cc890 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00bb10 tx=0x7f59fc02e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686865be1a3f5643 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:07.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.372+0000 7f5a0559a640 1 -- 172.21.15.76:0/3829148405 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc036440 con 0x7f5a080c4ff0 2023-11-09T03:50:07.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.372+0000 7f5a0559a640 1 -- 172.21.15.76:0/3829148405 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59fc0365a0 con 0x7f5a080c4ff0 2023-11-09T03:50:07.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.372+0000 7f5a0559a640 1 -- 172.21.15.76:0/3829148405 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc036930 con 0x7f5a080c4ff0 2023-11-09T03:50:07.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.372+0000 7f5a0c89b640 1 -- 172.21.15.76:0/3829148405 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a080c4ff0 msgr2=0x7f5a080cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:07.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.372+0000 7f5a0c89b640 1 --2- 172.21.15.76:0/3829148405 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a080c4ff0 0x7f5a080cc890 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00bb10 tx=0x7f59fc02e150 comp rx=0 tx=0).stop 2023-11-09T03:50:07.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.372+0000 7f5a0c89b640 1 -- 172.21.15.76:0/3829148405 shutdown_connections 2023-11-09T03:50:07.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.372+0000 7f5a0c89b640 1 --2- 172.21.15.76:0/3829148405 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a080adac0 0x7f5a080adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:07.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.372+0000 7f5a0c89b640 1 --2- 172.21.15.76:0/3829148405 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a080c4ff0 0x7f5a080cc890 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:07.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.372+0000 7f5a0c89b640 1 -- 172.21.15.76:0/3829148405 >> 172.21.15.76:0/3829148405 conn(0x7f5a080ce950 msgr2=0x7f5a080ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:07.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.372+0000 7f5a0c89b640 1 -- 172.21.15.76:0/3829148405 shutdown_connections 2023-11-09T03:50:07.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.372+0000 7f5a0c89b640 1 -- 172.21.15.76:0/3829148405 wait complete. 2023-11-09T03:50:07.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.374+0000 7f5a0c89b640 1 Processor -- start 2023-11-09T03:50:07.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.374+0000 7f5a0c89b640 1 -- start start 2023-11-09T03:50:07.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.374+0000 7f5a0c89b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a080adac0 0x7f5a081589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:07.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.374+0000 7f5a0c89b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a080c4ff0 0x7f5a08147c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:07.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.374+0000 7f5a0c89b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a08159c00 0x7f5a08154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:07.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.374+0000 7f5a0c89b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a080ac150 con 0x7f5a08159c00 2023-11-09T03:50:07.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.374+0000 7f5a0c89b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a08144eb0 con 0x7f5a080c4ff0 2023-11-09T03:50:07.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.374+0000 7f5a0c89b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a08158f20 con 0x7f5a080adac0 2023-11-09T03:50:07.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.375+0000 7f5a06d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a08159c00 0x7f5a08154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:07.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.375+0000 7f5a06d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a08159c00 0x7f5a08154610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50614/0 (socket says 172.21.15.76:50614) 2023-11-09T03:50:07.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.375+0000 7f5a06d9d640 1 -- 172.21.15.76:0/532103508 learned_addr learned my addr 172.21.15.76:0/532103508 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:07.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.375+0000 7f5a05d9b640 1 --2- 172.21.15.76:0/532103508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a080c4ff0 0x7f5a08147c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:07.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.375+0000 7f5a0659c640 1 --2- 172.21.15.76:0/532103508 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a080adac0 0x7f5a081589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:07.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.375+0000 7f5a06d9d640 1 -- 172.21.15.76:0/532103508 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a080adac0 msgr2=0x7f5a081589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:07.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.375+0000 7f5a06d9d640 1 --2- 172.21.15.76:0/532103508 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a080adac0 0x7f5a081589e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:07.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.375+0000 7f5a06d9d640 1 -- 172.21.15.76:0/532103508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a080c4ff0 msgr2=0x7f5a08147c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:07.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.375+0000 7f5a06d9d640 1 --2- 172.21.15.76:0/532103508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a080c4ff0 0x7f5a08147c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:07.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.375+0000 7f5a06d9d640 1 -- 172.21.15.76:0/532103508 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59fc00b770 con 0x7f5a08159c00 2023-11-09T03:50:07.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.375+0000 7f5a05d9b640 1 --2- 172.21.15.76:0/532103508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a080c4ff0 0x7f5a08147c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:50:07.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.375+0000 7f5a06d9d640 1 --2- 172.21.15.76:0/532103508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a08159c00 0x7f5a08154610 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f59f800d800 tx=0x7f59f800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:07.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.375+0000 7f59f6ffd640 1 -- 172.21.15.76:0/532103508 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f800b6c0 con 0x7f5a08159c00 2023-11-09T03:50:07.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.375+0000 7f59f6ffd640 1 -- 172.21.15.76:0/532103508 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f800b840 con 0x7f5a08159c00 2023-11-09T03:50:07.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.375+0000 7f5a0c89b640 1 -- 172.21.15.76:0/532103508 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a08148140 con 0x7f5a08159c00 2023-11-09T03:50:07.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.375+0000 7f59f6ffd640 1 -- 172.21.15.76:0/532103508 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f800bbf0 con 0x7f5a08159c00 2023-11-09T03:50:07.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.375+0000 7f5a0c89b640 1 -- 172.21.15.76:0/532103508 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a080aba90 con 0x7f5a08159c00 2023-11-09T03:50:07.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.376+0000 7f5a0c89b640 1 -- 172.21.15.76:0/532103508 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a080ab8d0 con 0x7f5a08159c00 2023-11-09T03:50:07.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.376+0000 7f59f6ffd640 1 -- 172.21.15.76:0/532103508 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f59f800bd50 con 0x7f5a08159c00 2023-11-09T03:50:07.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.377+0000 7f59f6ffd640 1 --2- 172.21.15.76:0/532103508 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f59e806ae70 0x7f59e806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:07.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.377+0000 7f59f6ffd640 1 -- 172.21.15.76:0/532103508 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7f59f808a090 con 0x7f5a08159c00 2023-11-09T03:50:07.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.377+0000 7f5a0659c640 1 --2- 172.21.15.76:0/532103508 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f59e806ae70 0x7f59e806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:07.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.378+0000 7f5a0659c640 1 --2- 172.21.15.76:0/532103508 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f59e806ae70 0x7f59e806d330 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5a080ad4f0 tx=0x7f5a00009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:07.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.381+0000 7f59f6ffd640 1 -- 172.21.15.76:0/532103508 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f59f8054bc0 con 0x7f5a08159c00 2023-11-09T03:50:07.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.615+0000 7f5a0c89b640 1 -- 172.21.15.76:0/532103508 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:0"} v 0) v1 -- 0x7f5a08074290 con 0x7f5a08159c00 2023-11-09T03:50:07.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.617+0000 7f59f6ffd640 1 -- 172.21.15.76:0/532103508 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:0"}]=0 failed mds gid 5744 v60) v1 ==== 97+0+0 (secure 0 0 0) 0x7f59f80581e0 con 0x7f5a08159c00 2023-11-09T03:50:07.617 INFO:teuthology.orchestra.run.smithi076.stderr:failed mds gid 5744 2023-11-09T03:50:07.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.620+0000 7f5a0c89b640 1 -- 172.21.15.76:0/532103508 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f59e806ae70 msgr2=0x7f59e806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:07.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.620+0000 7f5a0c89b640 1 --2- 172.21.15.76:0/532103508 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f59e806ae70 0x7f59e806d330 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5a080ad4f0 tx=0x7f5a00009040 comp rx=0 tx=0).stop 2023-11-09T03:50:07.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.620+0000 7f5a0c89b640 1 -- 172.21.15.76:0/532103508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a08159c00 msgr2=0x7f5a08154610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:07.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.620+0000 7f5a0c89b640 1 --2- 172.21.15.76:0/532103508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a08159c00 0x7f5a08154610 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f59f800d800 tx=0x7f59f800dcc0 comp rx=0 tx=0).stop 2023-11-09T03:50:07.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.620+0000 7f5a0c89b640 1 -- 172.21.15.76:0/532103508 shutdown_connections 2023-11-09T03:50:07.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.620+0000 7f5a0c89b640 1 --2- 172.21.15.76:0/532103508 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f59e806ae70 0x7f59e806d330 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:07.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.620+0000 7f5a0c89b640 1 --2- 172.21.15.76:0/532103508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a08159c00 0x7f5a08154610 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:07.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.620+0000 7f5a0c89b640 1 --2- 172.21.15.76:0/532103508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a080c4ff0 0x7f5a08147c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:07.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.620+0000 7f5a0c89b640 1 --2- 172.21.15.76:0/532103508 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a080adac0 0x7f5a081589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:07.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.620+0000 7f5a0c89b640 1 -- 172.21.15.76:0/532103508 >> 172.21.15.76:0/532103508 conn(0x7f5a080ce950 msgr2=0x7f5a080c4490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:07.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.620+0000 7f5a0c89b640 1 -- 172.21.15.76:0/532103508 shutdown_connections 2023-11-09T03:50:07.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.620+0000 7f5a0c89b640 1 -- 172.21.15.76:0/532103508 wait complete. 2023-11-09T03:50:07.633 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-09T03:50:07.633 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-09T03:50:07.634 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-09T03:50:07.634 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:50:07.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:50:07.635 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi113 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b' 2023-11-09T03:50:07.635 INFO:tasks.ceph.mds.b:Stopped 2023-11-09T03:50:07.636 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-09T03:50:07.638 INFO:tasks.ceph.mds.b:Started 2023-11-09T03:50:07.639 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:50:07.718 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2023-11-09T03:50:07.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.740+0000 7f3f733a9640 1 Processor -- start 2023-11-09T03:50:07.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.741+0000 7f3f733a9640 1 -- start start 2023-11-09T03:50:07.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.741+0000 7f3f733a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f6c0cde60 0x7f3f6c15d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:07.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.741+0000 7f3f733a9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f6c15e000 con 0x7f3f6c0ce240 2023-11-09T03:50:07.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.741+0000 7f3f733a9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f6c15e2e0 con 0x7f3f6c15daa0 2023-11-09T03:50:07.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.741+0000 7f3f733a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c15e5c0 con 0x7f3f6c0cde60 2023-11-09T03:50:07.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.741+0000 7f3f71946640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f3f6c0ce240 0x7f3f6c0d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48920/0 (socket says 172.21.15.76:48920) 2023-11-09T03:50:07.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.741+0000 7f3f71946640 1 -- 172.21.15.76:0/2770470773 learned_addr learned my addr 172.21.15.76:0/2770470773 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:50:07.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.742+0000 7f3f63fff640 1 -- 172.21.15.76:0/2770470773 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2734860696 0 0) 0x7f3f6c15e000 con 0x7f3f6c0ce240 2023-11-09T03:50:07.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.742+0000 7f3f71145640 1 --2- 172.21.15.76:0/2770470773 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f6c0cde60 0x7f3f6c15d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:07.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.742+0000 7f3f63fff640 1 -- 172.21.15.76:0/2770470773 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f58003520 con 0x7f3f6c0ce240 2023-11-09T03:50:07.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.742+0000 7f3f63fff640 1 -- 172.21.15.76:0/2770470773 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2639557760 0 0) 0x7f3f58003520 con 0x7f3f6c0ce240 2023-11-09T03:50:07.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.742+0000 7f3f63fff640 1 -- 172.21.15.76:0/2770470773 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f6c0cde60 msgr2=0x7f3f6c15d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:07.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.742+0000 7f3f63fff640 1 --2- 172.21.15.76:0/2770470773 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f6c0cde60 0x7f3f6c15d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:07.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.742+0000 7f3f63fff640 1 -- 172.21.15.76:0/2770470773 >> v1:172.21.15.113:6789/0 conn(0x7f3f6c15daa0 legacy=0x7f3f6c157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:07.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.742+0000 7f3f63fff640 1 -- 172.21.15.76:0/2770470773 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6c0d1120 con 0x7f3f6c0ce240 2023-11-09T03:50:07.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.742+0000 7f3f63fff640 1 -- 172.21.15.76:0/2770470773 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f3f64001670 con 0x7f3f6c0ce240 2023-11-09T03:50:07.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.742+0000 7f3f63fff640 1 -- 172.21.15.76:0/2770470773 >> v1:172.21.15.76:6789/0 conn(0x7f3f6c0ce240 legacy=0x7f3f6c0d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:07.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.742+0000 7f3f63fff640 1 --2- 172.21.15.76:0/2770470773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f580039a0 0x7f3f58003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:07.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.742+0000 7f3f63fff640 1 --2- 172.21.15.76:0/2770470773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f58004350 0x7f3f58004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:07.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.742+0000 7f3f63fff640 1 --2- 172.21.15.76:0/2770470773 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f6c0cde60 0x7f3f58005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:07.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.742+0000 7f3f63fff640 1 -- 172.21.15.76:0/2770470773 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c15e5c0 con 0x7f3f580039a0 2023-11-09T03:50:07.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.742+0000 7f3f63fff640 1 -- 172.21.15.76:0/2770470773 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5800b980 con 0x7f3f58004350 2023-11-09T03:50:07.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.742+0000 7f3f63fff640 1 -- 172.21.15.76:0/2770470773 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5800bb40 con 0x7f3f6c0cde60 2023-11-09T03:50:07.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.742+0000 7f3f71145640 1 --2- 172.21.15.76:0/2770470773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f580039a0 0x7f3f58003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:07.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.742+0000 7f3f71946640 1 --2- 172.21.15.76:0/2770470773 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f6c0cde60 0x7f3f58005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:07.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.742+0000 7f3f70944640 1 --2- 172.21.15.76:0/2770470773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f58004350 0x7f3f58004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:07.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.742+0000 7f3f71145640 1 -- 172.21.15.76:0/2770470773 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f6c0cde60 msgr2=0x7f3f58005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:07.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.742+0000 7f3f71145640 1 --2- 172.21.15.76:0/2770470773 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f6c0cde60 0x7f3f58005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:07.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.742+0000 7f3f71145640 1 -- 172.21.15.76:0/2770470773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f58004350 msgr2=0x7f3f58004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:07.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.742+0000 7f3f71145640 1 --2- 172.21.15.76:0/2770470773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f58004350 0x7f3f58004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:07.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.742+0000 7f3f71145640 1 -- 172.21.15.76:0/2770470773 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f58003520 con 0x7f3f580039a0 2023-11-09T03:50:07.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.743+0000 7f3f71145640 1 --2- 172.21.15.76:0/2770470773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f580039a0 0x7f3f58003d80 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f3f68009fc0 tx=0x7f3f580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3d44bb57e45f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:07.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.743+0000 7f3f63fff640 1 -- 172.21.15.76:0/2770470773 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6802cc20 con 0x7f3f580039a0 2023-11-09T03:50:07.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.743+0000 7f3f63fff640 1 -- 172.21.15.76:0/2770470773 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6800f040 con 0x7f3f580039a0 2023-11-09T03:50:07.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.743+0000 7f3f63fff640 1 -- 172.21.15.76:0/2770470773 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f68034440 con 0x7f3f580039a0 2023-11-09T03:50:07.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.744+0000 7f3f733a9640 1 -- 172.21.15.76:0/2770470773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f580039a0 msgr2=0x7f3f58003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:07.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.744+0000 7f3f733a9640 1 --2- 172.21.15.76:0/2770470773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f580039a0 0x7f3f58003d80 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f3f68009fc0 tx=0x7f3f580057f0 comp rx=0 tx=0).stop 2023-11-09T03:50:07.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.744+0000 7f3f71946640 1 -- 172.21.15.76:0/2770470773 reap_dead start 2023-11-09T03:50:07.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.744+0000 7f3f733a9640 1 -- 172.21.15.76:0/2770470773 shutdown_connections 2023-11-09T03:50:07.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.744+0000 7f3f733a9640 1 -- 172.21.15.76:0/2770470773 >> 172.21.15.76:0/2770470773 conn(0x7f3f6c071f30 msgr2=0x7f3f6c0cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:07.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.744+0000 7f3f733a9640 1 -- 172.21.15.76:0/2770470773 shutdown_connections 2023-11-09T03:50:07.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.744+0000 7f3f733a9640 1 -- 172.21.15.76:0/2770470773 wait complete. 2023-11-09T03:50:07.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.745+0000 7f3f733a9640 1 Processor -- start 2023-11-09T03:50:07.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.745+0000 7f3f733a9640 1 -- start start 2023-11-09T03:50:07.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.746+0000 7f3f733a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f580039a0 0x7f3f6c1f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:07.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.746+0000 7f3f733a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f6c1f5b60 0x7f3f6c1f5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:07.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.746+0000 7f3f733a9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f6c14b960 0x7f3f6c14bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:07.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.746+0000 7f3f733a9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c1f6520 con 0x7f3f6c14b960 2023-11-09T03:50:07.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.746+0000 7f3f733a9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c151620 con 0x7f3f580039a0 2023-11-09T03:50:07.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.746+0000 7f3f733a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c150a10 con 0x7f3f6c1f5b60 2023-11-09T03:50:07.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.746+0000 7f3f71946640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f6c14b960 0x7f3f6c14bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:07.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.746+0000 7f3f71946640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f6c14b960 0x7f3f6c14bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50626/0 (socket says 172.21.15.76:50626) 2023-11-09T03:50:07.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.746+0000 7f3f71946640 1 -- 172.21.15.76:0/3287702618 learned_addr learned my addr 172.21.15.76:0/3287702618 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:07.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.746+0000 7f3f70944640 1 --2- 172.21.15.76:0/3287702618 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f6c1f5b60 0x7f3f6c1f5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:07.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.746+0000 7f3f71145640 1 --2- 172.21.15.76:0/3287702618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f580039a0 0x7f3f6c1f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:07.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.746+0000 7f3f71946640 1 -- 172.21.15.76:0/3287702618 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f6c1f5b60 msgr2=0x7f3f6c1f5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:07.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.746+0000 7f3f71946640 1 --2- 172.21.15.76:0/3287702618 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f6c1f5b60 0x7f3f6c1f5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:07.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.746+0000 7f3f71946640 1 -- 172.21.15.76:0/3287702618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f580039a0 msgr2=0x7f3f6c1f2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:07.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.746+0000 7f3f71946640 1 --2- 172.21.15.76:0/3287702618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f580039a0 0x7f3f6c1f2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:07.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.746+0000 7f3f71946640 1 -- 172.21.15.76:0/3287702618 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6c150cf0 con 0x7f3f6c14b960 2023-11-09T03:50:07.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.746+0000 7f3f70944640 1 --2- 172.21.15.76:0/3287702618 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f6c1f5b60 0x7f3f6c1f5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:50:07.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.746+0000 7f3f71946640 1 --2- 172.21.15.76:0/3287702618 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f6c14b960 0x7f3f6c14bde0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c06c790 tx=0x7f3f64007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:07.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.746+0000 7f3f61ffb640 1 -- 172.21.15.76:0/3287702618 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6400f6a0 con 0x7f3f6c14b960 2023-11-09T03:50:07.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.746+0000 7f3f61ffb640 1 -- 172.21.15.76:0/3287702618 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6400f800 con 0x7f3f6c14b960 2023-11-09T03:50:07.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.746+0000 7f3f733a9640 1 -- 172.21.15.76:0/3287702618 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f6c150fd0 con 0x7f3f6c14b960 2023-11-09T03:50:07.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.746+0000 7f3f61ffb640 1 -- 172.21.15.76:0/3287702618 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6400fbb0 con 0x7f3f6c14b960 2023-11-09T03:50:07.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.747+0000 7f3f733a9640 1 -- 172.21.15.76:0/3287702618 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f6c1e6e60 con 0x7f3f6c14b960 2023-11-09T03:50:07.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.748+0000 7f3f733a9640 1 -- 172.21.15.76:0/3287702618 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f6c074290 con 0x7f3f6c14b960 2023-11-09T03:50:07.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.748+0000 7f3f61ffb640 1 -- 172.21.15.76:0/3287702618 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3f6400fd10 con 0x7f3f6c14b960 2023-11-09T03:50:07.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.748+0000 7f3f61ffb640 1 --2- 172.21.15.76:0/3287702618 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3f5006aea0 0x7f3f5006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:07.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.748+0000 7f3f61ffb640 1 -- 172.21.15.76:0/3287702618 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7f3f64089320 con 0x7f3f6c14b960 2023-11-09T03:50:07.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.748+0000 7f3f71145640 1 --2- 172.21.15.76:0/3287702618 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3f5006aea0 0x7f3f5006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:07.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.749+0000 7f3f71145640 1 --2- 172.21.15.76:0/3287702618 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3f5006aea0 0x7f3f5006d360 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3f68037040 tx=0x7f3f6802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:07.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.752+0000 7f3f61ffb640 1 -- 172.21.15.76:0/3287702618 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3f64053d60 con 0x7f3f6c14b960 2023-11-09T03:50:07.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.953+0000 7f3f733a9640 1 -- 172.21.15.76:0/3287702618 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f6c0b72e0 con 0x7f3f6c14b960 2023-11-09T03:50:07.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.954+0000 7f3f61ffb640 1 -- 172.21.15.76:0/3287702618 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4139 (secure 0 0 0) 0x7f3f6401a090 con 0x7f3f6c14b960 2023-11-09T03:50:07.953 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:50:07.954 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:50:07.621083+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7773,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7773":{"gid":7773,"name":"c","rank":0,"incarnation":61,"state":"up:replay","state_seq":1,"addr":"172.21.15.76:6835/1663988060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1663988060},{"type":"v1","addr":"172.21.15.76:6835","nonce":1663988060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:50:07.954 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 61 2023-11-09T03:50:07.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.956+0000 7f3f733a9640 1 -- 172.21.15.76:0/3287702618 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3f5006aea0 msgr2=0x7f3f5006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:07.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.956+0000 7f3f733a9640 1 --2- 172.21.15.76:0/3287702618 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3f5006aea0 0x7f3f5006d360 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3f68037040 tx=0x7f3f6802e000 comp rx=0 tx=0).stop 2023-11-09T03:50:07.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.956+0000 7f3f733a9640 1 -- 172.21.15.76:0/3287702618 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f6c14b960 msgr2=0x7f3f6c14bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:07.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.956+0000 7f3f733a9640 1 --2- 172.21.15.76:0/3287702618 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f6c14b960 0x7f3f6c14bde0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c06c790 tx=0x7f3f64007430 comp rx=0 tx=0).stop 2023-11-09T03:50:07.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.956+0000 7f3f733a9640 1 -- 172.21.15.76:0/3287702618 shutdown_connections 2023-11-09T03:50:07.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.956+0000 7f3f733a9640 1 --2- 172.21.15.76:0/3287702618 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3f5006aea0 0x7f3f5006d360 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:07.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.956+0000 7f3f733a9640 1 --2- 172.21.15.76:0/3287702618 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f6c14b960 0x7f3f6c14bde0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:07.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.956+0000 7f3f733a9640 1 --2- 172.21.15.76:0/3287702618 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f6c1f5b60 0x7f3f6c1f5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:07.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.956+0000 7f3f733a9640 1 --2- 172.21.15.76:0/3287702618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f580039a0 0x7f3f6c1f2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:07.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.956+0000 7f3f733a9640 1 -- 172.21.15.76:0/3287702618 >> 172.21.15.76:0/3287702618 conn(0x7f3f6c071f30 msgr2=0x7f3f6c0cd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:07.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.956+0000 7f3f733a9640 1 -- 172.21.15.76:0/3287702618 shutdown_connections 2023-11-09T03:50:07.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:07.956+0000 7f3f733a9640 1 -- 172.21.15.76:0/3287702618 wait complete. 2023-11-09T03:50:07.966 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-11-09T03:50:12.692 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-11-09T03:50:12.967 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:50:13.047 INFO:tasks.ceph.mds.c.smithi076.stderr:2023-11-09T03:50:13.048+0000 7fa5f446b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:50:13.048 INFO:tasks.ceph.mds.c.smithi076.stderr:2023-11-09T03:50:13.048+0000 7fa5f446b640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-09T03:50:13.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.083+0000 7f08270da640 1 Processor -- start 2023-11-09T03:50:13.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.083+0000 7f08270da640 1 -- start start 2023-11-09T03:50:13.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.083+0000 7f08270da640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f08200cda60 0x7f08200ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:13.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.083+0000 7f08270da640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08200cc990 con 0x7f0820070d40 2023-11-09T03:50:13.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.083+0000 7f08270da640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08200ccc70 con 0x7f08200b0cd0 2023-11-09T03:50:13.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.083+0000 7f08270da640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08200c9ce0 con 0x7f08200cda60 2023-11-09T03:50:13.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.084+0000 7f0824e76640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f08200cda60 0x7f08200ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:13.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.084+0000 7f0825677640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0820070d40 0x7f082006c200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:54114/0 (socket says 172.21.15.76:54114) 2023-11-09T03:50:13.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.084+0000 7f0824e76640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f08200cda60 0x7f08200ca7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41152/0 (socket says 172.21.15.76:41152) 2023-11-09T03:50:13.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.084+0000 7f0825677640 1 -- 172.21.15.76:0/4148670122 learned_addr learned my addr 172.21.15.76:0/4148670122 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:50:13.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.084+0000 7f0824e76640 1 -- 172.21.15.76:0/4148670122 >> v1:172.21.15.113:6789/0 conn(0x7f08200b0cd0 legacy=0x7f0820070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:13.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.084+0000 7f0824e76640 1 -- 172.21.15.76:0/4148670122 >> v1:172.21.15.76:6789/0 conn(0x7f0820070d40 legacy=0x7f082006c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:13.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.084+0000 7f0824e76640 1 -- 172.21.15.76:0/4148670122 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08200c9fc0 con 0x7f08200cda60 2023-11-09T03:50:13.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.084+0000 7f081f7fe640 1 -- 172.21.15.76:0/4148670122 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1670188380 0 0) 0x7f08200cc990 con 0x7f0820070d40 2023-11-09T03:50:13.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.085+0000 7f0824e76640 1 --2- 172.21.15.76:0/4148670122 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08200cda60 0x7f08200ca7a0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f08140097a0 tx=0x7f081402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fdad68cd10de6bd server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:13.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.085+0000 7f081f7fe640 1 -- 172.21.15.76:0/4148670122 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0814030610 con 0x7f08200cda60 2023-11-09T03:50:13.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.085+0000 7f081f7fe640 1 -- 172.21.15.76:0/4148670122 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0814030770 con 0x7f08200cda60 2023-11-09T03:50:13.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.085+0000 7f081f7fe640 1 -- 172.21.15.76:0/4148670122 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0814030b00 con 0x7f08200cda60 2023-11-09T03:50:13.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.086+0000 7f08270da640 1 -- 172.21.15.76:0/4148670122 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08200cda60 msgr2=0x7f08200ca7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:13.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.086+0000 7f08270da640 1 --2- 172.21.15.76:0/4148670122 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08200cda60 0x7f08200ca7a0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f08140097a0 tx=0x7f081402b960 comp rx=0 tx=0).stop 2023-11-09T03:50:13.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.086+0000 7f08270da640 1 -- 172.21.15.76:0/4148670122 shutdown_connections 2023-11-09T03:50:13.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.086+0000 7f08270da640 1 --2- 172.21.15.76:0/4148670122 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08200cda60 0x7f08200ca7a0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:13.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.086+0000 7f08270da640 1 -- 172.21.15.76:0/4148670122 >> 172.21.15.76:0/4148670122 conn(0x7f08200ce950 msgr2=0x7f08200ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:13.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.086+0000 7f08270da640 1 -- 172.21.15.76:0/4148670122 shutdown_connections 2023-11-09T03:50:13.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.086+0000 7f08270da640 1 -- 172.21.15.76:0/4148670122 wait complete. 2023-11-09T03:50:13.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.087+0000 7f08270da640 1 Processor -- start 2023-11-09T03:50:13.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.087+0000 7f08270da640 1 -- start start 2023-11-09T03:50:13.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.088+0000 7f08270da640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0820070d40 0x7f08200c5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:13.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.088+0000 7f08270da640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08200b0cd0 0x7f0820158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:13.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.088+0000 7f08270da640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08200cda60 0x7f082014cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:13.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.088+0000 7f08270da640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08201593b0 con 0x7f0820070d40 2023-11-09T03:50:13.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.088+0000 7f08270da640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0820148580 con 0x7f08200cda60 2023-11-09T03:50:13.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.088+0000 7f08270da640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0820147c30 con 0x7f08200b0cd0 2023-11-09T03:50:13.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.088+0000 7f0824e76640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0820070d40 0x7f08200c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:13.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.088+0000 7f0824e76640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0820070d40 0x7f08200c5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41164/0 (socket says 172.21.15.76:41164) 2023-11-09T03:50:13.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.088+0000 7f0824e76640 1 -- 172.21.15.76:0/2512287934 learned_addr learned my addr 172.21.15.76:0/2512287934 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:13.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.088+0000 7f0825677640 1 --2- 172.21.15.76:0/2512287934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08200cda60 0x7f082014cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:13.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.088+0000 7f081ffff640 1 --2- 172.21.15.76:0/2512287934 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08200b0cd0 0x7f0820158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:13.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.088+0000 7f0824e76640 1 -- 172.21.15.76:0/2512287934 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08200b0cd0 msgr2=0x7f0820158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:13.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.088+0000 7f0824e76640 1 --2- 172.21.15.76:0/2512287934 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08200b0cd0 0x7f0820158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:13.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.088+0000 7f0824e76640 1 -- 172.21.15.76:0/2512287934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08200cda60 msgr2=0x7f082014cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:13.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.088+0000 7f0824e76640 1 --2- 172.21.15.76:0/2512287934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08200cda60 0x7f082014cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:13.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.088+0000 7f0824e76640 1 -- 172.21.15.76:0/2512287934 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0814009400 con 0x7f0820070d40 2023-11-09T03:50:13.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.089+0000 7f0824e76640 1 --2- 172.21.15.76:0/2512287934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0820070d40 0x7f08200c5990 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f08140097a0 tx=0x7f08140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:13.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.089+0000 7f081d7fa640 1 -- 172.21.15.76:0/2512287934 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0814039c90 con 0x7f0820070d40 2023-11-09T03:50:13.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.089+0000 7f081d7fa640 1 -- 172.21.15.76:0/2512287934 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081403f040 con 0x7f0820070d40 2023-11-09T03:50:13.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.089+0000 7f081d7fa640 1 -- 172.21.15.76:0/2512287934 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08140433f0 con 0x7f0820070d40 2023-11-09T03:50:13.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.089+0000 7f08270da640 1 -- 172.21.15.76:0/2512287934 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0820146b90 con 0x7f0820070d40 2023-11-09T03:50:13.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.089+0000 7f08270da640 1 -- 172.21.15.76:0/2512287934 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0820147150 con 0x7f0820070d40 2023-11-09T03:50:13.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.090+0000 7f08270da640 1 -- 172.21.15.76:0/2512287934 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0820074290 con 0x7f0820070d40 2023-11-09T03:50:13.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.090+0000 7f081d7fa640 1 -- 172.21.15.76:0/2512287934 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0814043550 con 0x7f0820070d40 2023-11-09T03:50:13.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.091+0000 7f081d7fa640 1 --2- 172.21.15.76:0/2512287934 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f080406ae70 0x7f080406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:13.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.091+0000 7f081d7fa640 1 -- 172.21.15.76:0/2512287934 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7f08140ac6f0 con 0x7f0820070d40 2023-11-09T03:50:13.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.091+0000 7f081ffff640 1 --2- 172.21.15.76:0/2512287934 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f080406ae70 0x7f080406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:13.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.091+0000 7f081ffff640 1 --2- 172.21.15.76:0/2512287934 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f080406ae70 0x7f080406d330 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f082014e270 tx=0x7f080c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:13.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.095+0000 7f081d7fa640 1 -- 172.21.15.76:0/2512287934 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f08140772e0 con 0x7f0820070d40 2023-11-09T03:50:13.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.300+0000 7f08270da640 1 -- 172.21.15.76:0/2512287934 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08200b3de0 con 0x7f0820070d40 2023-11-09T03:50:13.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.300+0000 7f081d7fa640 1 -- 172.21.15.76:0/2512287934 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f081407a900 con 0x7f0820070d40 2023-11-09T03:50:13.300 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:50:13.300 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3449910842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3449910842},{"type":"v1","addr":"172.21.15.113:6837","nonce":3449910842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:50:10.633468+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7773,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7773":{"gid":7773,"name":"c","rank":0,"incarnation":61,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/1663988060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1663988060},{"type":"v1","addr":"172.21.15.76:6835","nonce":1663988060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:50:13.301 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 65 2023-11-09T03:50:13.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.302+0000 7f08270da640 1 -- 172.21.15.76:0/2512287934 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f080406ae70 msgr2=0x7f080406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:13.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.302+0000 7f08270da640 1 --2- 172.21.15.76:0/2512287934 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f080406ae70 0x7f080406d330 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f082014e270 tx=0x7f080c009040 comp rx=0 tx=0).stop 2023-11-09T03:50:13.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.302+0000 7f08270da640 1 -- 172.21.15.76:0/2512287934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0820070d40 msgr2=0x7f08200c5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.302+0000 7f08270da640 1 --2- 172.21.15.76:0/2512287934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0820070d40 0x7f08200c5990 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f08140097a0 tx=0x7f08140046c0 comp rx=0 tx=0).stop 2023-11-09T03:50:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.302+0000 7f08270da640 1 -- 172.21.15.76:0/2512287934 shutdown_connections 2023-11-09T03:50:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.302+0000 7f08270da640 1 --2- 172.21.15.76:0/2512287934 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f080406ae70 0x7f080406d330 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.302+0000 7f08270da640 1 --2- 172.21.15.76:0/2512287934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08200cda60 0x7f082014cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.302+0000 7f08270da640 1 --2- 172.21.15.76:0/2512287934 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08200b0cd0 0x7f0820158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:13.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.302+0000 7f08270da640 1 --2- 172.21.15.76:0/2512287934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0820070d40 0x7f08200c5990 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:13.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.302+0000 7f08270da640 1 -- 172.21.15.76:0/2512287934 >> 172.21.15.76:0/2512287934 conn(0x7f08200ce950 msgr2=0x7f0820070630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:13.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.302+0000 7f08270da640 1 -- 172.21.15.76:0/2512287934 shutdown_connections 2023-11-09T03:50:13.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.302+0000 7f08270da640 1 -- 172.21.15.76:0/2512287934 wait complete. 2023-11-09T03:50:13.312 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-09T03:50:13.313 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:50:13.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.413+0000 7fd2be390640 1 Processor -- start 2023-11-09T03:50:13.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.413+0000 7fd2be390640 1 -- start start 2023-11-09T03:50:13.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.414+0000 7fd2be390640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd2b815ad80 0x7fd2b806c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:13.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.414+0000 7fd2be390640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd2b80b42c0 0x7fd2b80c9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:13.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.414+0000 7fd2be390640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2b806cd00 con 0x7fd2b80c9c40 2023-11-09T03:50:13.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.414+0000 7fd2be390640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b80ca020 con 0x7fd2b80b42c0 2023-11-09T03:50:13.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.414+0000 7fd2be390640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b806cfa0 con 0x7fd2b815ad80 2023-11-09T03:50:13.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.414+0000 7fd2b77fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd2b80b42c0 0x7fd2b80c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:13.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.414+0000 7fd2b77fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd2b80b42c0 0x7fd2b80c9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41166/0 (socket says 172.21.15.76:41166) 2023-11-09T03:50:13.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.414+0000 7fd2b77fe640 1 -- 172.21.15.76:0/765002109 learned_addr learned my addr 172.21.15.76:0/765002109 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:13.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.414+0000 7fd2b7fff640 1 --2- 172.21.15.76:0/765002109 >> v2:172.21.15.113:3300/0 conn(0x7fd2b815ad80 0x7fd2b806c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:13.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.414+0000 7fd2b77fe640 1 -- 172.21.15.76:0/765002109 >> v2:172.21.15.113:3300/0 conn(0x7fd2b815ad80 msgr2=0x7fd2b806c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:13.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.414+0000 7fd2b77fe640 1 --2- 172.21.15.76:0/765002109 >> v2:172.21.15.113:3300/0 conn(0x7fd2b815ad80 0x7fd2b806c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:13.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.414+0000 7fd2b77fe640 1 -- 172.21.15.76:0/765002109 >> v1:172.21.15.113:6789/0 conn(0x7fd2b80c9c40 legacy=0x7fd2b8070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:13.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.414+0000 7fd2b77fe640 1 -- 172.21.15.76:0/765002109 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b806d280 con 0x7fd2b80b42c0 2023-11-09T03:50:13.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.415+0000 7fd2b77fe640 1 --2- 172.21.15.76:0/765002109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2b80b42c0 0x7fd2b80c9700 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fd2b80b4ac0 tx=0x7fd2ac02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f60294fd6f009842 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:13.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.415+0000 7fd2b6ffd640 1 -- 172.21.15.76:0/765002109 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac02f850 con 0x7fd2b80b42c0 2023-11-09T03:50:13.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.415+0000 7fd2b6ffd640 1 -- 172.21.15.76:0/765002109 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ac02f9b0 con 0x7fd2b80b42c0 2023-11-09T03:50:13.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.415+0000 7fd2b6ffd640 1 -- 172.21.15.76:0/765002109 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac02fd40 con 0x7fd2b80b42c0 2023-11-09T03:50:13.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.416+0000 7fd2be390640 1 -- 172.21.15.76:0/765002109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2b80b42c0 msgr2=0x7fd2b80c9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:13.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.416+0000 7fd2be390640 1 --2- 172.21.15.76:0/765002109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2b80b42c0 0x7fd2b80c9700 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fd2b80b4ac0 tx=0x7fd2ac02ba30 comp rx=0 tx=0).stop 2023-11-09T03:50:13.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.416+0000 7fd2be390640 1 -- 172.21.15.76:0/765002109 shutdown_connections 2023-11-09T03:50:13.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.416+0000 7fd2be390640 1 --2- 172.21.15.76:0/765002109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2b80b42c0 0x7fd2b80c9700 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:13.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.416+0000 7fd2be390640 1 --2- 172.21.15.76:0/765002109 >> v2:172.21.15.113:3300/0 conn(0x7fd2b815ad80 0x7fd2b806c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:13.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.416+0000 7fd2be390640 1 -- 172.21.15.76:0/765002109 >> 172.21.15.76:0/765002109 conn(0x7fd2b80ce950 msgr2=0x7fd2b80ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:13.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.416+0000 7fd2be390640 1 -- 172.21.15.76:0/765002109 shutdown_connections 2023-11-09T03:50:13.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.416+0000 7fd2be390640 1 -- 172.21.15.76:0/765002109 wait complete. 2023-11-09T03:50:13.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.417+0000 7fd2be390640 1 Processor -- start 2023-11-09T03:50:13.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.417+0000 7fd2be390640 1 -- start start 2023-11-09T03:50:13.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.418+0000 7fd2be390640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2b80b42c0 0x7fd2b81f0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:13.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.418+0000 7fd2be390640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2b80c9c40 0x7fd2b815c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:13.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.418+0000 7fd2be390640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2b815ad80 0x7fd2b80af890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:13.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.418+0000 7fd2be390640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b815cf10 con 0x7fd2b80c9c40 2023-11-09T03:50:13.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.418+0000 7fd2be390640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b815bb30 con 0x7fd2b80b42c0 2023-11-09T03:50:13.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.418+0000 7fd2be390640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b815d100 con 0x7fd2b815ad80 2023-11-09T03:50:13.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.418+0000 7fd2b77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2b80c9c40 0x7fd2b815c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:13.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.418+0000 7fd2b77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2b80c9c40 0x7fd2b815c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41180/0 (socket says 172.21.15.76:41180) 2023-11-09T03:50:13.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.418+0000 7fd2b77fe640 1 -- 172.21.15.76:0/537680580 learned_addr learned my addr 172.21.15.76:0/537680580 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:13.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.418+0000 7fd2b77fe640 1 -- 172.21.15.76:0/537680580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2b815ad80 msgr2=0x7fd2b80af890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:13.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.418+0000 7fd2bc92d640 1 --2- 172.21.15.76:0/537680580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2b815ad80 0x7fd2b80af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:13.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.418+0000 7fd2b7fff640 1 --2- 172.21.15.76:0/537680580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2b80b42c0 0x7fd2b81f0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:13.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.418+0000 7fd2b77fe640 1 --2- 172.21.15.76:0/537680580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2b815ad80 0x7fd2b80af890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:13.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.418+0000 7fd2b77fe640 1 -- 172.21.15.76:0/537680580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2b80b42c0 msgr2=0x7fd2b81f0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:13.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.418+0000 7fd2b77fe640 1 --2- 172.21.15.76:0/537680580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2b80b42c0 0x7fd2b81f0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:13.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.418+0000 7fd2b77fe640 1 -- 172.21.15.76:0/537680580 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ac0094d0 con 0x7fd2b80c9c40 2023-11-09T03:50:13.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.418+0000 7fd2bc92d640 1 --2- 172.21.15.76:0/537680580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2b815ad80 0x7fd2b80af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:50:13.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.418+0000 7fd2b7fff640 1 --2- 172.21.15.76:0/537680580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2b80b42c0 0x7fd2b81f0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:50:13.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.418+0000 7fd2b77fe640 1 --2- 172.21.15.76:0/537680580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2b80c9c40 0x7fd2b815c9d0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac02ba70 tx=0x7fd2ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:13.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.419+0000 7fd2b4ff9640 1 -- 172.21.15.76:0/537680580 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac037d10 con 0x7fd2b80c9c40 2023-11-09T03:50:13.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.419+0000 7fd2b4ff9640 1 -- 172.21.15.76:0/537680580 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ac037e90 con 0x7fd2b80c9c40 2023-11-09T03:50:13.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.419+0000 7fd2be390640 1 -- 172.21.15.76:0/537680580 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b815b200 con 0x7fd2b80c9c40 2023-11-09T03:50:13.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.419+0000 7fd2b4ff9640 1 -- 172.21.15.76:0/537680580 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac041640 con 0x7fd2b80c9c40 2023-11-09T03:50:13.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.419+0000 7fd2be390640 1 -- 172.21.15.76:0/537680580 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b815b7c0 con 0x7fd2b80c9c40 2023-11-09T03:50:13.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.420+0000 7fd2be390640 1 -- 172.21.15.76:0/537680580 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b815bce0 con 0x7fd2b80c9c40 2023-11-09T03:50:13.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.420+0000 7fd2b4ff9640 1 -- 172.21.15.76:0/537680580 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd2ac0417a0 con 0x7fd2b80c9c40 2023-11-09T03:50:13.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.421+0000 7fd2b4ff9640 1 --2- 172.21.15.76:0/537680580 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd2a406ae70 0x7fd2a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:13.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.421+0000 7fd2b4ff9640 1 -- 172.21.15.76:0/537680580 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7fd2ac03c070 con 0x7fd2b80c9c40 2023-11-09T03:50:13.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.421+0000 7fd2b7fff640 1 --2- 172.21.15.76:0/537680580 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd2a406ae70 0x7fd2a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:13.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.421+0000 7fd2b7fff640 1 --2- 172.21.15.76:0/537680580 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd2a406ae70 0x7fd2a406d330 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8006fd0 tx=0x7fd2a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:13.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.425+0000 7fd2b4ff9640 1 -- 172.21.15.76:0/537680580 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd2b815bce0 con 0x7fd2b80c9c40 2023-11-09T03:50:13.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.626+0000 7fd2be390640 1 -- 172.21.15.76:0/537680580 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2b815bca0 con 0x7fd2b80c9c40 2023-11-09T03:50:13.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.627+0000 7fd2b4ff9640 1 -- 172.21.15.76:0/537680580 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4934 (secure 0 0 0) 0x7fd2ac077500 con 0x7fd2b80c9c40 2023-11-09T03:50:13.627 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:50:13.627 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3449910842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3449910842},{"type":"v1","addr":"172.21.15.113:6837","nonce":3449910842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:50:10.633468+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7773,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7773":{"gid":7773,"name":"c","rank":0,"incarnation":61,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/1663988060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1663988060},{"type":"v1","addr":"172.21.15.76:6835","nonce":1663988060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:50:13.627 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 65 2023-11-09T03:50:13.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.629+0000 7fd2be390640 1 -- 172.21.15.76:0/537680580 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd2a406ae70 msgr2=0x7fd2a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:13.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.629+0000 7fd2be390640 1 --2- 172.21.15.76:0/537680580 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd2a406ae70 0x7fd2a406d330 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8006fd0 tx=0x7fd2a8008040 comp rx=0 tx=0).stop 2023-11-09T03:50:13.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.629+0000 7fd2be390640 1 -- 172.21.15.76:0/537680580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2b80c9c40 msgr2=0x7fd2b815c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:13.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.629+0000 7fd2be390640 1 --2- 172.21.15.76:0/537680580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2b80c9c40 0x7fd2b815c9d0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac02ba70 tx=0x7fd2ac004730 comp rx=0 tx=0).stop 2023-11-09T03:50:13.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.629+0000 7fd2be390640 1 -- 172.21.15.76:0/537680580 shutdown_connections 2023-11-09T03:50:13.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.629+0000 7fd2be390640 1 --2- 172.21.15.76:0/537680580 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd2a406ae70 0x7fd2a406d330 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:13.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.629+0000 7fd2be390640 1 --2- 172.21.15.76:0/537680580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2b815ad80 0x7fd2b80af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:13.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.629+0000 7fd2be390640 1 --2- 172.21.15.76:0/537680580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2b80c9c40 0x7fd2b815c9d0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:13.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.629+0000 7fd2be390640 1 --2- 172.21.15.76:0/537680580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2b80b42c0 0x7fd2b81f0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:13.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.629+0000 7fd2be390640 1 -- 172.21.15.76:0/537680580 >> 172.21.15.76:0/537680580 conn(0x7fd2b80ce950 msgr2=0x7fd2b815a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:13.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.629+0000 7fd2be390640 1 -- 172.21.15.76:0/537680580 shutdown_connections 2023-11-09T03:50:13.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:13.629+0000 7fd2be390640 1 -- 172.21.15.76:0/537680580 wait complete. 2023-11-09T03:50:14.042 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-09T03:48:57.383848+0000', 'modified': '2023-11-09T03:50:10.633468+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 99, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 7773, 'mds_1': 7449}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7449': {'gid': 7449, 'name': 'a', 'rank': 1, 'incarnation': 47, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.76:6833/3697850101', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6832', 'nonce': 3697850101}, {'type': 'v1', 'addr': '172.21.15.76:6833', 'nonce': 3697850101}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_7773': {'gid': 7773, 'name': 'c', 'rank': 0, 'incarnation': 61, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.76:6835/1663988060', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 1663988060}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 1663988060}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-09T03:50:14.042 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2023-11-09T03:50:14.043 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-09T03:50:14.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.145+0000 7f87a3578640 1 Processor -- start 2023-11-09T03:50:14.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.145+0000 7f87a3578640 1 -- start start 2023-11-09T03:50:14.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.146+0000 7f87a3578640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f879c159c00 0x7f879c0cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.146+0000 7f87a3578640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f879c15aff0 con 0x7f879c0c48e0 2023-11-09T03:50:14.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.146+0000 7f87a3578640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f879c15a3e0 con 0x7f879c0ae1d0 2023-11-09T03:50:14.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.146+0000 7f87a3578640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f879c15a6c0 con 0x7f879c159c00 2023-11-09T03:50:14.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.146+0000 7f87a0b13640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f879c0c48e0 0x7f879c0adac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:54124/0 (socket says 172.21.15.76:54124) 2023-11-09T03:50:14.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.146+0000 7f87a0b13640 1 -- 172.21.15.76:0/3976111840 learned_addr learned my addr 172.21.15.76:0/3976111840 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:50:14.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.146+0000 7f87a1314640 1 --2- 172.21.15.76:0/3976111840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f879c159c00 0x7f879c0cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.147+0000 7f8793fff640 1 -- 172.21.15.76:0/3976111840 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1734652012 0 0) 0x7f879c15aff0 con 0x7f879c0c48e0 2023-11-09T03:50:14.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.147+0000 7f8793fff640 1 -- 172.21.15.76:0/3976111840 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8788003520 con 0x7f879c0c48e0 2023-11-09T03:50:14.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.147+0000 7f8793fff640 1 -- 172.21.15.76:0/3976111840 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4079596539 0 0) 0x7f8788003520 con 0x7f879c0c48e0 2023-11-09T03:50:14.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.147+0000 7f8793fff640 1 -- 172.21.15.76:0/3976111840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f879c159c00 msgr2=0x7f879c0cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:14.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.147+0000 7f8793fff640 1 --2- 172.21.15.76:0/3976111840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f879c159c00 0x7f879c0cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.147+0000 7f8793fff640 1 -- 172.21.15.76:0/3976111840 >> v1:172.21.15.113:6789/0 conn(0x7f879c0ae1d0 legacy=0x7f879c0709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.147+0000 7f8793fff640 1 -- 172.21.15.76:0/3976111840 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f879c15a9a0 con 0x7f879c0c48e0 2023-11-09T03:50:14.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.147+0000 7f8793fff640 1 -- 172.21.15.76:0/3976111840 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f878c001670 con 0x7f879c0c48e0 2023-11-09T03:50:14.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.147+0000 7f8793fff640 1 -- 172.21.15.76:0/3976111840 >> v1:172.21.15.76:6789/0 conn(0x7f879c0c48e0 legacy=0x7f879c0adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.147+0000 7f8793fff640 1 --2- 172.21.15.76:0/3976111840 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87880039a0 0x7f8788003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.147+0000 7f8793fff640 1 --2- 172.21.15.76:0/3976111840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8788004350 0x7f8788004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.147+0000 7f8793fff640 1 --2- 172.21.15.76:0/3976111840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f879c159c00 0x7f8788005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.147+0000 7f8793fff640 1 -- 172.21.15.76:0/3976111840 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c15a6c0 con 0x7f87880039a0 2023-11-09T03:50:14.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.147+0000 7f8793fff640 1 -- 172.21.15.76:0/3976111840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878800b980 con 0x7f8788004350 2023-11-09T03:50:14.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.147+0000 7f8793fff640 1 -- 172.21.15.76:0/3976111840 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878800bb40 con 0x7f879c159c00 2023-11-09T03:50:14.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.147+0000 7f87a1314640 1 --2- 172.21.15.76:0/3976111840 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87880039a0 0x7f8788003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.147+0000 7f87a0b13640 1 --2- 172.21.15.76:0/3976111840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8788004350 0x7f8788004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.147+0000 7f87a1b15640 1 --2- 172.21.15.76:0/3976111840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f879c159c00 0x7f8788005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.147+0000 7f87a1314640 1 -- 172.21.15.76:0/3976111840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f879c159c00 msgr2=0x7f8788005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:14.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.147+0000 7f87a1314640 1 --2- 172.21.15.76:0/3976111840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f879c159c00 0x7f8788005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.147+0000 7f87a1314640 1 -- 172.21.15.76:0/3976111840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8788004350 msgr2=0x7f8788004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:14.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.147+0000 7f87a1314640 1 --2- 172.21.15.76:0/3976111840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8788004350 0x7f8788004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.147+0000 7f87a1314640 1 -- 172.21.15.76:0/3976111840 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8788003520 con 0x7f87880039a0 2023-11-09T03:50:14.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.148+0000 7f87a1314640 1 --2- 172.21.15.76:0/3976111840 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87880039a0 0x7f8788003d80 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f8798009fc0 tx=0x7f87880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=babf34cdd34d7a4b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:14.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.148+0000 7f8793fff640 1 -- 172.21.15.76:0/3976111840 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879802cc20 con 0x7f87880039a0 2023-11-09T03:50:14.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.148+0000 7f8793fff640 1 -- 172.21.15.76:0/3976111840 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f879800f040 con 0x7f87880039a0 2023-11-09T03:50:14.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.148+0000 7f8793fff640 1 -- 172.21.15.76:0/3976111840 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8798034440 con 0x7f87880039a0 2023-11-09T03:50:14.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.149+0000 7f87a3578640 1 -- 172.21.15.76:0/3976111840 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87880039a0 msgr2=0x7f8788003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.149+0000 7f87a3578640 1 --2- 172.21.15.76:0/3976111840 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87880039a0 0x7f8788003d80 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f8798009fc0 tx=0x7f87880057f0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.149+0000 7f87a1b15640 1 -- 172.21.15.76:0/3976111840 reap_dead start 2023-11-09T03:50:14.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.149+0000 7f87a3578640 1 -- 172.21.15.76:0/3976111840 shutdown_connections 2023-11-09T03:50:14.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.149+0000 7f87a3578640 1 -- 172.21.15.76:0/3976111840 >> 172.21.15.76:0/3976111840 conn(0x7f879c0ce950 msgr2=0x7f879c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:14.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.149+0000 7f87a3578640 1 -- 172.21.15.76:0/3976111840 shutdown_connections 2023-11-09T03:50:14.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.149+0000 7f87a3578640 1 -- 172.21.15.76:0/3976111840 wait complete. 2023-11-09T03:50:14.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.150+0000 7f87a3578640 1 Processor -- start 2023-11-09T03:50:14.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.150+0000 7f87a3578640 1 -- start start 2023-11-09T03:50:14.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.151+0000 7f87a3578640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87880039a0 0x7f879c0b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.151+0000 7f87a3578640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f879c158030 0x7f879c0b0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.151+0000 7f87a3578640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f879c152ee0 0x7f879c06cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.151+0000 7f87a3578640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c158f60 con 0x7f87880039a0 2023-11-09T03:50:14.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.151+0000 7f87a3578640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c1584b0 con 0x7f879c158030 2023-11-09T03:50:14.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.151+0000 7f87a3578640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f879c153800 con 0x7f879c152ee0 2023-11-09T03:50:14.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.151+0000 7f87a1314640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87880039a0 0x7f879c0b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.151+0000 7f87a1314640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87880039a0 0x7f879c0b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41204/0 (socket says 172.21.15.76:41204) 2023-11-09T03:50:14.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.151+0000 7f87a1314640 1 -- 172.21.15.76:0/1888545525 learned_addr learned my addr 172.21.15.76:0/1888545525 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:14.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.151+0000 7f87a1b15640 1 --2- 172.21.15.76:0/1888545525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f879c152ee0 0x7f879c06cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.151+0000 7f87a0b13640 1 --2- 172.21.15.76:0/1888545525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f879c158030 0x7f879c0b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.151+0000 7f87a1314640 1 -- 172.21.15.76:0/1888545525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f879c152ee0 msgr2=0x7f879c06cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.151+0000 7f87a1314640 1 --2- 172.21.15.76:0/1888545525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f879c152ee0 0x7f879c06cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.151+0000 7f87a1314640 1 -- 172.21.15.76:0/1888545525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f879c158030 msgr2=0x7f879c0b0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.151+0000 7f87a1314640 1 --2- 172.21.15.76:0/1888545525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f879c158030 0x7f879c0b0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.151+0000 7f87a1314640 1 -- 172.21.15.76:0/1888545525 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f879c1586c0 con 0x7f87880039a0 2023-11-09T03:50:14.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.151+0000 7f87a1314640 1 --2- 172.21.15.76:0/1888545525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87880039a0 0x7f879c0b4140 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f879802c690 tx=0x7f8798009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:14.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.151+0000 7f8791ffb640 1 -- 172.21.15.76:0/1888545525 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8798034dd0 con 0x7f87880039a0 2023-11-09T03:50:14.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.151+0000 7f8791ffb640 1 -- 172.21.15.76:0/1888545525 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8798003990 con 0x7f87880039a0 2023-11-09T03:50:14.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.151+0000 7f87a3578640 1 -- 172.21.15.76:0/1888545525 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f879c1589a0 con 0x7f87880039a0 2023-11-09T03:50:14.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.151+0000 7f8791ffb640 1 -- 172.21.15.76:0/1888545525 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8798003d40 con 0x7f87880039a0 2023-11-09T03:50:14.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.152+0000 7f87a3578640 1 -- 172.21.15.76:0/1888545525 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879c153360 con 0x7f87880039a0 2023-11-09T03:50:14.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.153+0000 7f8791ffb640 1 -- 172.21.15.76:0/1888545525 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f879803b050 con 0x7f87880039a0 2023-11-09T03:50:14.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.154+0000 7f8791ffb640 1 --2- 172.21.15.76:0/1888545525 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f878006aea0 0x7f878006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.154+0000 7f8791ffb640 1 -- 172.21.15.76:0/1888545525 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7f87980ac060 con 0x7f87880039a0 2023-11-09T03:50:14.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.154+0000 7f87a0b13640 1 --2- 172.21.15.76:0/1888545525 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f878006aea0 0x7f878006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.154+0000 7f87a61e6740 1 Processor -- start 2023-11-09T03:50:14.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.154+0000 7f87a0b13640 1 --2- 172.21.15.76:0/1888545525 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f878006aea0 0x7f878006d360 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f878c001cb0 tx=0x7f878c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:14.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.154+0000 7f87a61e6740 1 -- start start 2023-11-09T03:50:14.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.155+0000 7f87a61e6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557bba050850 0x557bba023750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.155+0000 7f87a61e6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bba050c30 0x557bba0535e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.155+0000 7f87a61e6740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557bba023c90 0x557bba053b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.155+0000 7f87a61e6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bb9f34740 con 0x557bba050850 2023-11-09T03:50:14.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.155+0000 7f87a61e6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bb9f968d0 con 0x557bba050c30 2023-11-09T03:50:14.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.155+0000 7f87a61e6740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x557bb9f93fc0 con 0x557bba023c90 2023-11-09T03:50:14.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.155+0000 7f87a1314640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557bba050850 0x557bba023750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.155+0000 7f87a1314640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557bba050850 0x557bba023750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41214/0 (socket says 172.21.15.76:41214) 2023-11-09T03:50:14.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.155+0000 7f87a1314640 1 -- 172.21.15.76:0/3029456309 learned_addr learned my addr 172.21.15.76:0/3029456309 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:14.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.155+0000 7f87a0b13640 1 --2- 172.21.15.76:0/3029456309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bba050c30 0x557bba0535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.155+0000 7f87a1b15640 1 --2- 172.21.15.76:0/3029456309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557bba023c90 0x557bba053b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.155+0000 7f87a1314640 1 -- 172.21.15.76:0/3029456309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557bba023c90 msgr2=0x557bba053b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:14.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.155+0000 7f87a1314640 1 --2- 172.21.15.76:0/3029456309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557bba023c90 0x557bba053b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.155+0000 7f87a1314640 1 -- 172.21.15.76:0/3029456309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bba050c30 msgr2=0x557bba0535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:14.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.156+0000 7f87a1314640 1 --2- 172.21.15.76:0/3029456309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bba050c30 0x557bba0535e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.156+0000 7f87a1314640 1 -- 172.21.15.76:0/3029456309 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bb9f52ca0 con 0x557bba050850 2023-11-09T03:50:14.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.156+0000 7f87a0b13640 1 --2- 172.21.15.76:0/3029456309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bba050c30 0x557bba0535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:50:14.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.156+0000 7f87a1314640 1 --2- 172.21.15.76:0/3029456309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557bba050850 0x557bba023750 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f8798037040 tx=0x7f879807afa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68eea462f31be17f server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:14.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.156+0000 7f8786ffd640 1 -- 172.21.15.76:0/3029456309 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8798082a60 con 0x557bba050850 2023-11-09T03:50:14.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.156+0000 7f8786ffd640 1 -- 172.21.15.76:0/3029456309 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8798078b00 con 0x557bba050850 2023-11-09T03:50:14.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.156+0000 7f8786ffd640 1 -- 172.21.15.76:0/3029456309 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8798076e70 con 0x557bba050850 2023-11-09T03:50:14.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.156+0000 7f87a61e6740 1 -- 172.21.15.76:0/3029456309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557bba050850 msgr2=0x557bba023750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.156+0000 7f87a61e6740 1 --2- 172.21.15.76:0/3029456309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557bba050850 0x557bba023750 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f8798037040 tx=0x7f879807afa0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.156+0000 7f87a61e6740 1 -- 172.21.15.76:0/3029456309 shutdown_connections 2023-11-09T03:50:14.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.156+0000 7f87a61e6740 1 --2- 172.21.15.76:0/3029456309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557bba023c90 0x557bba053b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.156+0000 7f87a61e6740 1 --2- 172.21.15.76:0/3029456309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bba050c30 0x557bba0535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.156+0000 7f87a61e6740 1 --2- 172.21.15.76:0/3029456309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557bba050850 0x557bba023750 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.156+0000 7f87a61e6740 1 -- 172.21.15.76:0/3029456309 >> 172.21.15.76:0/3029456309 conn(0x557bba025a60 msgr2=0x557bba06e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.156+0000 7f87a61e6740 1 -- 172.21.15.76:0/3029456309 shutdown_connections 2023-11-09T03:50:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.156+0000 7f87a61e6740 1 -- 172.21.15.76:0/3029456309 wait complete. 2023-11-09T03:50:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.157+0000 7f87a61e6740 1 -- start start 2023-11-09T03:50:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.157+0000 7f87a61e6740 1 Processor -- start 2023-11-09T03:50:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.158+0000 7f87a61e6740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557bba050850 0x557bba0568f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.158+0000 7f87a61e6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557bba050c30 0x557bba071d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.158+0000 7f87a61e6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bba057220 0x557bba0445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.158+0000 7f87a61e6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bb9f8f8b0 con 0x557bba050c30 2023-11-09T03:50:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.158+0000 7f87a61e6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bb9f49c80 con 0x557bba057220 2023-11-09T03:50:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.158+0000 7f87a61e6740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x557bb9f49920 con 0x557bba050850 2023-11-09T03:50:14.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.158+0000 7f87a61e6740 10 client.0 _ll_register_callbacks cb 0x557bb9ca1ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:50:14.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.158+0000 7f87a0b13640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557bba050c30 0x557bba071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.158+0000 7f87a0b13640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557bba050c30 0x557bba071d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41222/0 (socket says 172.21.15.76:41222) 2023-11-09T03:50:14.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.158+0000 7f87a0b13640 1 -- 172.21.15.76:0/4137149789 learned_addr learned my addr 172.21.15.76:0/4137149789 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:14.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.158+0000 7f87a1b15640 1 --2- 172.21.15.76:0/4137149789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bba057220 0x557bba0445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.158+0000 7f87a1314640 1 --2- 172.21.15.76:0/4137149789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557bba050850 0x557bba0568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.158+0000 7f87a0b13640 1 -- 172.21.15.76:0/4137149789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557bba050850 msgr2=0x557bba0568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.158+0000 7f87a0b13640 1 --2- 172.21.15.76:0/4137149789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557bba050850 0x557bba0568f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.158+0000 7f87a0b13640 1 -- 172.21.15.76:0/4137149789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bba057220 msgr2=0x557bba0445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.158+0000 7f87a0b13640 1 --2- 172.21.15.76:0/4137149789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bba057220 0x557bba0445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.158+0000 7f87a0b13640 1 -- 172.21.15.76:0/4137149789 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879c153360 con 0x557bba050c30 2023-11-09T03:50:14.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.158+0000 7f87a1b15640 1 --2- 172.21.15.76:0/4137149789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bba057220 0x557bba0445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:50:14.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.158+0000 7f87a0b13640 1 --2- 172.21.15.76:0/4137149789 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557bba050c30 0x557bba071d40 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f878c0144e0 tx=0x7f878c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d6b6e097a7839d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:14.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.158+0000 7f877bfff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:50:14.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.159+0000 7f877bfff640 1 -- 172.21.15.76:0/4137149789 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c013770 con 0x557bba050c30 2023-11-09T03:50:14.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.159+0000 7f877bfff640 1 -- 172.21.15.76:0/4137149789 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7f878c01c070 con 0x557bba050c30 2023-11-09T03:50:14.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.159+0000 7f87a61e6740 1 -- 172.21.15.76:0/4137149789 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557bb9f3bbf0 con 0x557bba050c30 2023-11-09T03:50:14.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.159+0000 7f877bfff640 1 -- 172.21.15.76:0/4137149789 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f878c0138d0 con 0x557bba050c30 2023-11-09T03:50:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.159+0000 7f87a61e6740 10 client.7926 fetch_fsmap learned FSMap version 65 2023-11-09T03:50:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.159+0000 7f87a61e6740 1 -- 172.21.15.76:0/4137149789 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x557bb9f8df30 con 0x557bba050c30 2023-11-09T03:50:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.159+0000 7f877bfff640 1 -- 172.21.15.76:0/4137149789 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557bb9f3bbf0 con 0x557bba050c30 2023-11-09T03:50:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.159+0000 7f877bfff640 1 -- 172.21.15.76:0/4137149789 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 65) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f878c013a30 con 0x557bba050c30 2023-11-09T03:50:14.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.159+0000 7f87a61e6740 10 client.7926 fetch_fsmap finished waiting for FSMap version 65 2023-11-09T03:50:14.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.159+0000 7f87a61e6740 10 client.7926 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-09T03:50:14.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.159+0000 7f87a61e6740 20 client.7926 populate_metadata read hostname 'smithi076' 2023-11-09T03:50:14.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.159+0000 7f87a61e6740 1 --2- 172.21.15.76:0/4137149789 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x557bba065100 0x557bba0654e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.159+0000 7f87a61e6740 4 client.7926 mds_command: new command op to 7449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:50:14.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.159+0000 7f87a61e6740 1 -- 172.21.15.76:0/4137149789 --> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557bb9f88890 con 0x557bba065100 2023-11-09T03:50:14.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.159+0000 7f87a1314640 1 --2- 172.21.15.76:0/4137149789 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x557bba065100 0x557bba0654e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.159+0000 7f877bfff640 10 client.7926 ms_handle_connect on v2:172.21.15.76:6832/3697850101 2023-11-09T03:50:14.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.159+0000 7f87a1314640 1 --2- 172.21.15.76:0/4137149789 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x557bba065100 0x557bba0654e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec959a3071292855 server_cookie=9622beae2da0cf7d in_seq=0 out_seq=0 2023-11-09T03:50:14.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.160+0000 7f877bfff640 1 -- 172.21.15.76:0/4137149789 <== mds.1 v2:172.21.15.76:6832/3697850101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557bb9f88890 con 0x557bba065100 2023-11-09T03:50:14.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.160+0000 7f877bfff640 10 client.7926 handle_command_reply: tid=0 2023-11-09T03:50:14.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.160+0000 7f87a61e6740 1 client.7926 shutdown 2023-11-09T03:50:14.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.160+0000 7f87a61e6740 1 -- 172.21.15.76:0/4137149789 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557bba050c30 msgr2=0x557bba071d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.160+0000 7f87a61e6740 1 --2- 172.21.15.76:0/4137149789 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557bba050c30 0x557bba071d40 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f878c0144e0 tx=0x7f878c001a00 comp rx=0 tx=0).stop 2023-11-09T03:50:14.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.160+0000 7f87a61e6740 1 -- 172.21.15.76:0/4137149789 shutdown_connections 2023-11-09T03:50:14.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.160+0000 7f87a61e6740 1 --2- 172.21.15.76:0/4137149789 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x557bba065100 0x557bba0654e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.160+0000 7f87a61e6740 1 --2- 172.21.15.76:0/4137149789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bba057220 0x557bba0445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.160+0000 7f877bfff640 0 client.7926 ms_handle_reset on v2:172.21.15.76:6832/3697850101 2023-11-09T03:50:14.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.160+0000 7f87a61e6740 1 --2- 172.21.15.76:0/4137149789 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557bba050c30 0x557bba071d40 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.160+0000 7f87a61e6740 1 --2- 172.21.15.76:0/4137149789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557bba050850 0x557bba0568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.160+0000 7f87a61e6740 1 -- 172.21.15.76:0/4137149789 >> 172.21.15.76:0/4137149789 conn(0x557bba023c90 msgr2=0x557bba06f830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.160+0000 7f87a61e6740 1 -- 172.21.15.76:0/4137149789 shutdown_connections 2023-11-09T03:50:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.160+0000 7f87a61e6740 1 -- 172.21.15.76:0/4137149789 wait complete. 2023-11-09T03:50:14.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.160+0000 7f87a61e6740 20 client.7926 trim_cache size 0 max 16384 2023-11-09T03:50:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.176+0000 7f87a61e6740 1 Processor -- start 2023-11-09T03:50:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.176+0000 7f87a61e6740 1 -- start start 2023-11-09T03:50:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.176+0000 7f87a61e6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bba050850 0x557bba053530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.176+0000 7f87a61e6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557bba050c30 0x557bba053a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.176+0000 7f87a61e6740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557bba057220 0x557bba070c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.176+0000 7f87a61e6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bb9f0f280 con 0x557bba050c30 2023-11-09T03:50:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.176+0000 7f87a61e6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bb9f8ea50 con 0x557bba050850 2023-11-09T03:50:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.176+0000 7f87a61e6740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x557bba029200 con 0x557bba057220 2023-11-09T03:50:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.176+0000 7f87a0b13640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557bba050c30 0x557bba053a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.177+0000 7f87a0b13640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557bba050c30 0x557bba053a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41238/0 (socket says 172.21.15.76:41238) 2023-11-09T03:50:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.177+0000 7f87a0b13640 1 -- 172.21.15.76:0/2141516821 learned_addr learned my addr 172.21.15.76:0/2141516821 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.177+0000 7f87a1314640 1 --2- 172.21.15.76:0/2141516821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bba050850 0x557bba053530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.177+0000 7f87a0b13640 1 -- 172.21.15.76:0/2141516821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557bba057220 msgr2=0x557bba070c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:14.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.177+0000 7f87a1b15640 1 --2- 172.21.15.76:0/2141516821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557bba057220 0x557bba070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.177+0000 7f87a0b13640 1 --2- 172.21.15.76:0/2141516821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557bba057220 0x557bba070c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.177+0000 7f87a0b13640 1 -- 172.21.15.76:0/2141516821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bba050850 msgr2=0x557bba053530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:14.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.177+0000 7f87a0b13640 1 --2- 172.21.15.76:0/2141516821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bba050850 0x557bba053530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.177+0000 7f87a0b13640 1 -- 172.21.15.76:0/2141516821 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bb9f2bd10 con 0x557bba050c30 2023-11-09T03:50:14.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.177+0000 7f87a1314640 1 --2- 172.21.15.76:0/2141516821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bba050850 0x557bba053530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:50:14.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.177+0000 7f87a1b15640 1 --2- 172.21.15.76:0/2141516821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557bba057220 0x557bba070c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:50:14.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.177+0000 7f87a0b13640 1 --2- 172.21.15.76:0/2141516821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557bba050c30 0x557bba053a70 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f878c001730 tx=0x7f878c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3cb6be8511b4cd server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:14.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.177+0000 7f87877fe640 1 -- 172.21.15.76:0/2141516821 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c013770 con 0x557bba050c30 2023-11-09T03:50:14.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.177+0000 7f87877fe640 1 -- 172.21.15.76:0/2141516821 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878c0138f0 con 0x557bba050c30 2023-11-09T03:50:14.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.177+0000 7f87877fe640 1 -- 172.21.15.76:0/2141516821 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c010bd0 con 0x557bba050c30 2023-11-09T03:50:14.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.177+0000 7f87a61e6740 1 -- 172.21.15.76:0/2141516821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557bba050c30 msgr2=0x557bba053a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.177+0000 7f87a61e6740 1 --2- 172.21.15.76:0/2141516821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557bba050c30 0x557bba053a70 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f878c001730 tx=0x7f878c021000 comp rx=0 tx=0).stop 2023-11-09T03:50:14.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.178+0000 7f87a61e6740 1 -- 172.21.15.76:0/2141516821 shutdown_connections 2023-11-09T03:50:14.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.178+0000 7f87a61e6740 1 --2- 172.21.15.76:0/2141516821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557bba057220 0x557bba070c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.178+0000 7f87a61e6740 1 --2- 172.21.15.76:0/2141516821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557bba050c30 0x557bba053a70 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.178+0000 7f87a61e6740 1 --2- 172.21.15.76:0/2141516821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bba050850 0x557bba053530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.178+0000 7f87a61e6740 1 -- 172.21.15.76:0/2141516821 >> 172.21.15.76:0/2141516821 conn(0x557bba023c90 msgr2=0x557bba071b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:14.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.178+0000 7f87a61e6740 1 -- 172.21.15.76:0/2141516821 shutdown_connections 2023-11-09T03:50:14.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.178+0000 7f87a61e6740 1 -- 172.21.15.76:0/2141516821 wait complete. 2023-11-09T03:50:14.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.178+0000 7f87a61e6740 1 -- start start 2023-11-09T03:50:14.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.178+0000 7f87a61e6740 1 Processor -- start 2023-11-09T03:50:14.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.179+0000 7f87a61e6740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557bba050850 0x557bba056830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.179+0000 7f87a61e6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bba050c30 0x557bba053530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.179+0000 7f87a61e6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557bba057220 0x557bba053a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.179+0000 7f87a61e6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bba071210 con 0x557bba057220 2023-11-09T03:50:14.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.179+0000 7f87a61e6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bba071630 con 0x557bba050c30 2023-11-09T03:50:14.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.179+0000 7f87a61e6740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x557bba07d080 con 0x557bba050850 2023-11-09T03:50:14.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.179+0000 7f87a61e6740 10 client.0 _ll_register_callbacks cb 0x7f8798076b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:50:14.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.179+0000 7f87a1b15640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557bba057220 0x557bba053a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.179+0000 7f87a1b15640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557bba057220 0x557bba053a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41250/0 (socket says 172.21.15.76:41250) 2023-11-09T03:50:14.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.179+0000 7f87a1b15640 1 -- 172.21.15.76:0/4260943359 learned_addr learned my addr 172.21.15.76:0/4260943359 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:14.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.179+0000 7f87a0b13640 1 --2- 172.21.15.76:0/4260943359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bba050c30 0x557bba053530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.179+0000 7f87a1314640 1 --2- 172.21.15.76:0/4260943359 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557bba050850 0x557bba056830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.179+0000 7f87a1b15640 1 -- 172.21.15.76:0/4260943359 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557bba050850 msgr2=0x557bba056830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:14.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.179+0000 7f87a1b15640 1 --2- 172.21.15.76:0/4260943359 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557bba050850 0x557bba056830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.179+0000 7f87a1b15640 1 -- 172.21.15.76:0/4260943359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bba050c30 msgr2=0x557bba053530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:14.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.179+0000 7f87a1b15640 1 --2- 172.21.15.76:0/4260943359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bba050c30 0x557bba053530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.179+0000 7f87a1b15640 1 -- 172.21.15.76:0/4260943359 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557bb9f8df30 con 0x557bba057220 2023-11-09T03:50:14.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.179+0000 7f87a0b13640 1 --2- 172.21.15.76:0/4260943359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bba050c30 0x557bba053530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:50:14.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.180+0000 7f87a1b15640 1 --2- 172.21.15.76:0/4260943359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557bba057220 0x557bba053a70 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f879c0cc4d0 tx=0x7f8794001700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40f4e59e18779e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:14.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.180+0000 7f877bfff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:50:14.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.180+0000 7f877bfff640 1 -- 172.21.15.76:0/4260943359 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87940163f0 con 0x557bba057220 2023-11-09T03:50:14.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.180+0000 7f877bfff640 1 -- 172.21.15.76:0/4260943359 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7f8794016550 con 0x557bba057220 2023-11-09T03:50:14.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.180+0000 7f87a61e6740 1 -- 172.21.15.76:0/4260943359 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8798078b00 con 0x557bba057220 2023-11-09T03:50:14.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.180+0000 7f877bfff640 1 -- 172.21.15.76:0/4260943359 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8794016710 con 0x557bba057220 2023-11-09T03:50:14.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.180+0000 7f87a61e6740 10 client.7932 fetch_fsmap learned FSMap version 65 2023-11-09T03:50:14.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.180+0000 7f87a61e6740 1 -- 172.21.15.76:0/4260943359 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x557bb9f51500 con 0x557bba057220 2023-11-09T03:50:14.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.180+0000 7f877bfff640 1 -- 172.21.15.76:0/4260943359 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8798078b00 con 0x557bba057220 2023-11-09T03:50:14.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.180+0000 7f877bfff640 1 -- 172.21.15.76:0/4260943359 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 65) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f879401d060 con 0x557bba057220 2023-11-09T03:50:14.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.180+0000 7f87a61e6740 10 client.7932 fetch_fsmap finished waiting for FSMap version 65 2023-11-09T03:50:14.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.180+0000 7f87a61e6740 10 client.7932 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-09T03:50:14.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.180+0000 7f87a61e6740 20 client.7932 populate_metadata read hostname 'smithi076' 2023-11-09T03:50:14.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.180+0000 7f87a61e6740 1 --2- 172.21.15.76:0/4260943359 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x557bba0dc220 0x557bba063f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.180+0000 7f87a61e6740 4 client.7932 mds_command: new command op to 7449 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-09T03:50:14.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.180+0000 7f87a61e6740 1 -- 172.21.15.76:0/4260943359 --> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557bb9f885b0 con 0x557bba0dc220 2023-11-09T03:50:14.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.180+0000 7f87a1314640 1 --2- 172.21.15.76:0/4260943359 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x557bba0dc220 0x557bba063f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.181+0000 7f877bfff640 10 client.7932 ms_handle_connect on v2:172.21.15.76:6832/3697850101 2023-11-09T03:50:14.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.181+0000 7f87a1314640 1 --2- 172.21.15.76:0/4260943359 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x557bba0dc220 0x557bba063f70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3dc9b0ca91c5be5c server_cookie=5e42bc4f65a884c4 in_seq=0 out_seq=0 2023-11-09T03:50:14.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.181+0000 7f877bfff640 1 -- 172.21.15.76:0/4260943359 <== mds.1 v2:172.21.15.76:6832/3697850101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x557bb9f885b0 con 0x557bba0dc220 2023-11-09T03:50:14.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.181+0000 7f877bfff640 10 client.7932 handle_command_reply: tid=0 2023-11-09T03:50:14.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.181+0000 7f87a61e6740 1 client.7932 shutdown 2023-11-09T03:50:14.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.181+0000 7f87a61e6740 1 -- 172.21.15.76:0/4260943359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557bba057220 msgr2=0x557bba053a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.181+0000 7f87a61e6740 1 --2- 172.21.15.76:0/4260943359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557bba057220 0x557bba053a70 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f879c0cc4d0 tx=0x7f8794001700 comp rx=0 tx=0).stop 2023-11-09T03:50:14.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.181+0000 7f87a61e6740 1 -- 172.21.15.76:0/4260943359 shutdown_connections 2023-11-09T03:50:14.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.181+0000 7f87a61e6740 1 --2- 172.21.15.76:0/4260943359 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x557bba0dc220 0x557bba063f70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.181+0000 7f87a61e6740 1 --2- 172.21.15.76:0/4260943359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557bba057220 0x557bba053a70 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.181+0000 7f87a61e6740 1 --2- 172.21.15.76:0/4260943359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bba050c30 0x557bba053530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.181+0000 7f877bfff640 0 client.7932 ms_handle_reset on v2:172.21.15.76:6832/3697850101 2023-11-09T03:50:14.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.181+0000 7f87a61e6740 1 --2- 172.21.15.76:0/4260943359 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557bba050850 0x557bba056830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.181+0000 7f87a61e6740 1 -- 172.21.15.76:0/4260943359 >> 172.21.15.76:0/4260943359 conn(0x557bba023c90 msgr2=0x557bba01eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:14.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.181+0000 7f87a61e6740 1 -- 172.21.15.76:0/4260943359 shutdown_connections 2023-11-09T03:50:14.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.181+0000 7f87a61e6740 1 -- 172.21.15.76:0/4260943359 wait complete. 2023-11-09T03:50:14.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.181+0000 7f87a61e6740 20 client.7932 trim_cache size 0 max 16384 2023-11-09T03:50:14.211 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:50:14.212 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "c213b26f-d0da-45d5-8e34-bb75c096d4f2", 2023-11-09T03:50:14.212 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 1, 2023-11-09T03:50:14.212 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 7449, 2023-11-09T03:50:14.212 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2023-11-09T03:50:14.212 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2023-11-09T03:50:14.213 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2023-11-09T03:50:14.213 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 66.733418423000003, 2023-11-09T03:50:14.213 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 65, 2023-11-09T03:50:14.213 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 99, 2023-11-09T03:50:14.213 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 99, 2023-11-09T03:50:14.213 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 82.183713350000005 2023-11-09T03:50:14.214 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:50:14.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.182+0000 7f87a3578640 1 -- 172.21.15.76:0/1888545525 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f878006aea0 msgr2=0x7f878006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.182+0000 7f87a3578640 1 --2- 172.21.15.76:0/1888545525 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f878006aea0 0x7f878006d360 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f878c001cb0 tx=0x7f878c009040 comp rx=0 tx=0).stop 2023-11-09T03:50:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.182+0000 7f87a3578640 1 -- 172.21.15.76:0/1888545525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87880039a0 msgr2=0x7f879c0b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.182+0000 7f87a3578640 1 --2- 172.21.15.76:0/1888545525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87880039a0 0x7f879c0b4140 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f879802c690 tx=0x7f8798009750 comp rx=0 tx=0).stop 2023-11-09T03:50:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.182+0000 7f87a3578640 1 -- 172.21.15.76:0/1888545525 shutdown_connections 2023-11-09T03:50:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.182+0000 7f87a3578640 1 --2- 172.21.15.76:0/1888545525 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f878006aea0 0x7f878006d360 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.182+0000 7f87a3578640 1 --2- 172.21.15.76:0/1888545525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f879c152ee0 0x7f879c06cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.182+0000 7f87a3578640 1 --2- 172.21.15.76:0/1888545525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f879c158030 0x7f879c0b0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.182+0000 7f87a3578640 1 --2- 172.21.15.76:0/1888545525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87880039a0 0x7f879c0b4140 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.182+0000 7f87a3578640 1 -- 172.21.15.76:0/1888545525 >> 172.21.15.76:0/1888545525 conn(0x7f879c0ce950 msgr2=0x7f879c15b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:14.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.182+0000 7f87a3578640 1 -- 172.21.15.76:0/1888545525 shutdown_connections 2023-11-09T03:50:14.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.182+0000 7f87a3578640 1 -- 172.21.15.76:0/1888545525 wait complete. 2023-11-09T03:50:14.218 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-09T03:50:14.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.319+0000 7f40c9fe2640 1 Processor -- start 2023-11-09T03:50:14.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.319+0000 7f40c9fe2640 1 -- start start 2023-11-09T03:50:14.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.319+0000 7f40c9fe2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f40c4153e60 0x7f40c415dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.319+0000 7f40c9fe2640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c40cde60 con 0x7f40c40b4880 2023-11-09T03:50:14.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.319+0000 7f40c9fe2640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c4160f90 con 0x7f40c40cd6d0 2023-11-09T03:50:14.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.319+0000 7f40c9fe2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c415e400 con 0x7f40c4153e60 2023-11-09T03:50:14.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.320+0000 7f40c3fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f40c40b4880 0x7f40c40d0fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:54134/0 (socket says 172.21.15.76:54134) 2023-11-09T03:50:14.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.320+0000 7f40c3fff640 1 -- 172.21.15.76:0/4214050093 learned_addr learned my addr 172.21.15.76:0/4214050093 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:50:14.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.320+0000 7f40c27fc640 1 -- 172.21.15.76:0/4214050093 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 55358784 0 0) 0x7f40c40cde60 con 0x7f40c40b4880 2023-11-09T03:50:14.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.320+0000 7f40c27fc640 1 -- 172.21.15.76:0/4214050093 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40ac003520 con 0x7f40c40b4880 2023-11-09T03:50:14.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.320+0000 7f40c37fe640 1 --2- 172.21.15.76:0/4214050093 >> v2:172.21.15.113:3300/0 conn(0x7f40c4153e60 0x7f40c415dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.320+0000 7f40c27fc640 1 -- 172.21.15.76:0/4214050093 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 107977359 0 0) 0x7f40ac003520 con 0x7f40c40b4880 2023-11-09T03:50:14.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.320+0000 7f40c27fc640 1 -- 172.21.15.76:0/4214050093 >> v2:172.21.15.113:3300/0 conn(0x7f40c4153e60 msgr2=0x7f40c415dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:14.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.320+0000 7f40c27fc640 1 --2- 172.21.15.76:0/4214050093 >> v2:172.21.15.113:3300/0 conn(0x7f40c4153e60 0x7f40c415dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.320+0000 7f40c27fc640 1 -- 172.21.15.76:0/4214050093 >> v1:172.21.15.113:6789/0 conn(0x7f40c40cd6d0 legacy=0x7f40c4161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.320+0000 7f40c27fc640 1 -- 172.21.15.76:0/4214050093 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c40ce090 con 0x7f40c40b4880 2023-11-09T03:50:14.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.320+0000 7f40c27fc640 1 -- 172.21.15.76:0/4214050093 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f40bc0019f0 con 0x7f40c40b4880 2023-11-09T03:50:14.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.320+0000 7f40c27fc640 1 -- 172.21.15.76:0/4214050093 >> v1:172.21.15.76:6789/0 conn(0x7f40c40b4880 legacy=0x7f40c40d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.320+0000 7f40c27fc640 1 --2- 172.21.15.76:0/4214050093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40ac003970 0x7f40ac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.320+0000 7f40c27fc640 1 --2- 172.21.15.76:0/4214050093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40ac0047c0 0x7f40ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.320+0000 7f40c27fc640 1 --2- 172.21.15.76:0/4214050093 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40ac005180 0x7f40ac00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.320+0000 7f40c27fc640 1 -- 172.21.15.76:0/4214050093 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c415e400 con 0x7f40ac003970 2023-11-09T03:50:14.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.320+0000 7f40c27fc640 1 -- 172.21.15.76:0/4214050093 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac00c020 con 0x7f40ac0047c0 2023-11-09T03:50:14.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.320+0000 7f40c27fc640 1 -- 172.21.15.76:0/4214050093 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac00c1e0 con 0x7f40ac005180 2023-11-09T03:50:14.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.320+0000 7f40c37fe640 1 --2- 172.21.15.76:0/4214050093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40ac003970 0x7f40ac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.320+0000 7f40c2ffd640 1 --2- 172.21.15.76:0/4214050093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40ac0047c0 0x7f40ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.320+0000 7f40c3fff640 1 --2- 172.21.15.76:0/4214050093 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40ac005180 0x7f40ac00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.320+0000 7f40c37fe640 1 -- 172.21.15.76:0/4214050093 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40ac005180 msgr2=0x7f40ac00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:14.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.320+0000 7f40c37fe640 1 --2- 172.21.15.76:0/4214050093 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40ac005180 0x7f40ac00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.321+0000 7f40c37fe640 1 -- 172.21.15.76:0/4214050093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40ac0047c0 msgr2=0x7f40ac004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:14.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.321+0000 7f40c37fe640 1 --2- 172.21.15.76:0/4214050093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40ac0047c0 0x7f40ac004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.321+0000 7f40c37fe640 1 -- 172.21.15.76:0/4214050093 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ac003520 con 0x7f40ac003970 2023-11-09T03:50:14.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.321+0000 7f40c3fff640 1 -- 172.21.15.76:0/4214050093 reap_dead start 2023-11-09T03:50:14.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.321+0000 7f40c2ffd640 1 --2- 172.21.15.76:0/4214050093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40ac0047c0 0x7f40ac004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:50:14.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.321+0000 7f40c37fe640 1 --2- 172.21.15.76:0/4214050093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40ac003970 0x7f40ac003d90 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f40b4009ac0 tx=0x7f40ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf47c3c823f154eb server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:14.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.321+0000 7f40c27fc640 1 -- 172.21.15.76:0/4214050093 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b402c6b0 con 0x7f40ac003970 2023-11-09T03:50:14.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.321+0000 7f40c27fc640 1 -- 172.21.15.76:0/4214050093 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b402c810 con 0x7f40ac003970 2023-11-09T03:50:14.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.321+0000 7f40c27fc640 1 -- 172.21.15.76:0/4214050093 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b402cba0 con 0x7f40ac003970 2023-11-09T03:50:14.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.322+0000 7f40c9fe2640 1 -- 172.21.15.76:0/4214050093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40ac003970 msgr2=0x7f40ac003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.322+0000 7f40c9fe2640 1 --2- 172.21.15.76:0/4214050093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40ac003970 0x7f40ac003d90 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f40b4009ac0 tx=0x7f40ac0057f0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.322+0000 7f40c9fe2640 1 -- 172.21.15.76:0/4214050093 shutdown_connections 2023-11-09T03:50:14.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.322+0000 7f40c9fe2640 1 --2- 172.21.15.76:0/4214050093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40ac003970 0x7f40ac003d90 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.322+0000 7f40c9fe2640 1 -- 172.21.15.76:0/4214050093 >> 172.21.15.76:0/4214050093 conn(0x7f40c40cf880 msgr2=0x7f40c40cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:14.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.322+0000 7f40c9fe2640 1 -- 172.21.15.76:0/4214050093 shutdown_connections 2023-11-09T03:50:14.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.322+0000 7f40c9fe2640 1 -- 172.21.15.76:0/4214050093 wait complete. 2023-11-09T03:50:14.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.323+0000 7f40c9fe2640 1 Processor -- start 2023-11-09T03:50:14.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.323+0000 7f40c9fe2640 1 -- start start 2023-11-09T03:50:14.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.324+0000 7f40c9fe2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40ac003970 0x7f40c4147cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.324+0000 7f40c9fe2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c40cca60 0x7f40c41f21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.324+0000 7f40c9fe2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c40d2de0 0x7f40c40d3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.324+0000 7f40c9fe2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c40d37a0 con 0x7f40ac003970 2023-11-09T03:50:14.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.324+0000 7f40c9fe2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c40ccee0 con 0x7f40c40cca60 2023-11-09T03:50:14.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.324+0000 7f40c9fe2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c40c6900 con 0x7f40c40d2de0 2023-11-09T03:50:14.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.324+0000 7f40c37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40ac003970 0x7f40c4147cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.324+0000 7f40c37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40ac003970 0x7f40c4147cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41262/0 (socket says 172.21.15.76:41262) 2023-11-09T03:50:14.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.324+0000 7f40c37fe640 1 -- 172.21.15.76:0/477669423 learned_addr learned my addr 172.21.15.76:0/477669423 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:14.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.324+0000 7f40c2ffd640 1 --2- 172.21.15.76:0/477669423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c40cca60 0x7f40c41f21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.324+0000 7f40c3fff640 1 --2- 172.21.15.76:0/477669423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c40d2de0 0x7f40c40d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.324+0000 7f40c37fe640 1 -- 172.21.15.76:0/477669423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c40d2de0 msgr2=0x7f40c40d3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.324+0000 7f40c37fe640 1 --2- 172.21.15.76:0/477669423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c40d2de0 0x7f40c40d3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.324+0000 7f40c37fe640 1 -- 172.21.15.76:0/477669423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c40cca60 msgr2=0x7f40c41f21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.324+0000 7f40c37fe640 1 --2- 172.21.15.76:0/477669423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c40cca60 0x7f40c41f21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.324+0000 7f40c37fe640 1 -- 172.21.15.76:0/477669423 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c41f26e0 con 0x7f40ac003970 2023-11-09T03:50:14.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.324+0000 7f40c3fff640 1 --2- 172.21.15.76:0/477669423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c40d2de0 0x7f40c40d3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:50:14.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.325+0000 7f40c37fe640 1 --2- 172.21.15.76:0/477669423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40ac003970 0x7f40c4147cc0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f40b402c250 tx=0x7f40b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:14.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.325+0000 7f40abfff640 1 -- 172.21.15.76:0/477669423 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4003810 con 0x7f40ac003970 2023-11-09T03:50:14.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.325+0000 7f40abfff640 1 -- 172.21.15.76:0/477669423 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b4003970 con 0x7f40ac003970 2023-11-09T03:50:14.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.325+0000 7f40c9fe2640 1 -- 172.21.15.76:0/477669423 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c40c6ba0 con 0x7f40ac003970 2023-11-09T03:50:14.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.325+0000 7f40abfff640 1 -- 172.21.15.76:0/477669423 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4003d00 con 0x7f40ac003970 2023-11-09T03:50:14.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.325+0000 7f40c9fe2640 1 -- 172.21.15.76:0/477669423 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c40c7160 con 0x7f40ac003970 2023-11-09T03:50:14.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.327+0000 7f40abfff640 1 -- 172.21.15.76:0/477669423 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f40b4047450 con 0x7f40ac003970 2023-11-09T03:50:14.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.327+0000 7f40abfff640 1 --2- 172.21.15.76:0/477669423 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f40b006ae70 0x7f40b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.327+0000 7f40abfff640 1 -- 172.21.15.76:0/477669423 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7f40b40abed0 con 0x7f40ac003970 2023-11-09T03:50:14.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.327+0000 7f40c2ffd640 1 --2- 172.21.15.76:0/477669423 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f40b006ae70 0x7f40b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.328+0000 7f40ccb74740 1 Processor -- start 2023-11-09T03:50:14.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.328+0000 7f40ccb74740 1 -- start start 2023-11-09T03:50:14.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.328+0000 7f40c2ffd640 1 --2- 172.21.15.76:0/477669423 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f40b006ae70 0x7f40b006d330 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f40b8000c00 tx=0x7f40b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:14.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.329+0000 7f40ccb74740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564841890850 0x564841863750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.329+0000 7f40ccb74740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564841890c30 0x5648418935e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.329+0000 7f40ccb74740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564841863c90 0x564841893b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.329+0000 7f40ccb74740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x564841774740 con 0x564841890c30 2023-11-09T03:50:14.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.329+0000 7f40ccb74740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648417d68d0 con 0x564841890850 2023-11-09T03:50:14.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.329+0000 7f40ccb74740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5648417d3fc0 con 0x564841863c90 2023-11-09T03:50:14.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.329+0000 7f40c2ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564841890c30 0x5648418935e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.329+0000 7f40c2ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564841890c30 0x5648418935e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41278/0 (socket says 172.21.15.76:41278) 2023-11-09T03:50:14.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.329+0000 7f40c2ffd640 1 -- 172.21.15.76:0/4061878776 learned_addr learned my addr 172.21.15.76:0/4061878776 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:14.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.329+0000 7f40c37fe640 1 --2- 172.21.15.76:0/4061878776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564841890850 0x564841863750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.329+0000 7f40c3fff640 1 --2- 172.21.15.76:0/4061878776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564841863c90 0x564841893b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.329+0000 7f40c2ffd640 1 -- 172.21.15.76:0/4061878776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564841863c90 msgr2=0x564841893b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:14.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.329+0000 7f40c2ffd640 1 --2- 172.21.15.76:0/4061878776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564841863c90 0x564841893b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.329+0000 7f40c2ffd640 1 -- 172.21.15.76:0/4061878776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564841890850 msgr2=0x564841863750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:14.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.329+0000 7f40c2ffd640 1 --2- 172.21.15.76:0/4061878776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564841890850 0x564841863750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.329+0000 7f40c2ffd640 1 -- 172.21.15.76:0/4061878776 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564841792ca0 con 0x564841890c30 2023-11-09T03:50:14.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.329+0000 7f40c2ffd640 1 --2- 172.21.15.76:0/4061878776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564841890c30 0x5648418935e0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f40b8011730 tx=0x7f40b8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9111f508225f75 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:14.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.329+0000 7f40a97fa640 1 -- 172.21.15.76:0/4061878776 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8015880 con 0x564841890c30 2023-11-09T03:50:14.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.329+0000 7f40a97fa640 1 -- 172.21.15.76:0/4061878776 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b80159e0 con 0x564841890c30 2023-11-09T03:50:14.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.329+0000 7f40a97fa640 1 -- 172.21.15.76:0/4061878776 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8015d50 con 0x564841890c30 2023-11-09T03:50:14.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.330+0000 7f40ccb74740 1 -- 172.21.15.76:0/4061878776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564841890c30 msgr2=0x5648418935e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.330+0000 7f40ccb74740 1 --2- 172.21.15.76:0/4061878776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564841890c30 0x5648418935e0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f40b8011730 tx=0x7f40b8014000 comp rx=0 tx=0).stop 2023-11-09T03:50:14.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.330+0000 7f40ccb74740 1 -- 172.21.15.76:0/4061878776 shutdown_connections 2023-11-09T03:50:14.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.330+0000 7f40ccb74740 1 --2- 172.21.15.76:0/4061878776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564841863c90 0x564841893b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.330+0000 7f40ccb74740 1 --2- 172.21.15.76:0/4061878776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564841890c30 0x5648418935e0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.330+0000 7f40ccb74740 1 --2- 172.21.15.76:0/4061878776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564841890850 0x564841863750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.330+0000 7f40ccb74740 1 -- 172.21.15.76:0/4061878776 >> 172.21.15.76:0/4061878776 conn(0x564841865a60 msgr2=0x5648418ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:14.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.330+0000 7f40ccb74740 1 -- 172.21.15.76:0/4061878776 shutdown_connections 2023-11-09T03:50:14.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.330+0000 7f40ccb74740 1 -- 172.21.15.76:0/4061878776 wait complete. 2023-11-09T03:50:14.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.331+0000 7f40ccb74740 1 -- start start 2023-11-09T03:50:14.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.331+0000 7f40ccb74740 1 Processor -- start 2023-11-09T03:50:14.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.331+0000 7f40ccb74740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564841890850 0x564841884aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.331+0000 7f40ccb74740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564841890c30 0x564841895620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.331+0000 7f40ccb74740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56484188fc90 0x5648418624f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.331+0000 7f40ccb74740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648417cf8b0 con 0x56484188fc90 2023-11-09T03:50:14.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.331+0000 7f40ccb74740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x564841789c80 con 0x564841890c30 2023-11-09T03:50:14.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.331+0000 7f40ccb74740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x564841789920 con 0x564841890850 2023-11-09T03:50:14.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.331+0000 7f40ccb74740 10 client.0 _ll_register_callbacks cb 0x5648414e1ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:50:14.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.331+0000 7f40c3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56484188fc90 0x5648418624f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.332+0000 7f40c3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56484188fc90 0x5648418624f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41288/0 (socket says 172.21.15.76:41288) 2023-11-09T03:50:14.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.332+0000 7f40c3fff640 1 -- 172.21.15.76:0/4273130037 learned_addr learned my addr 172.21.15.76:0/4273130037 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:14.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.332+0000 7f40c37fe640 1 --2- 172.21.15.76:0/4273130037 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564841890850 0x564841884aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.332+0000 7f40c2ffd640 1 --2- 172.21.15.76:0/4273130037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564841890c30 0x564841895620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.332+0000 7f40c3fff640 1 -- 172.21.15.76:0/4273130037 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564841890850 msgr2=0x564841884aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:14.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.332+0000 7f40c3fff640 1 --2- 172.21.15.76:0/4273130037 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564841890850 0x564841884aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.332+0000 7f40c3fff640 1 -- 172.21.15.76:0/4273130037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564841890c30 msgr2=0x564841895620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:14.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.332+0000 7f40c3fff640 1 --2- 172.21.15.76:0/4273130037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564841890c30 0x564841895620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.332+0000 7f40c3fff640 1 -- 172.21.15.76:0/4273130037 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b800f040 con 0x56484188fc90 2023-11-09T03:50:14.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.332+0000 7f40c3fff640 1 --2- 172.21.15.76:0/4273130037 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56484188fc90 0x5648418624f0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f40c40c9e50 tx=0x7f40bc007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c02a35e525eeffa server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:14.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.332+0000 7f40a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:50:14.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.332+0000 7f40a27fc640 1 -- 172.21.15.76:0/4273130037 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc01a070 con 0x56484188fc90 2023-11-09T03:50:14.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.332+0000 7f40a27fc640 1 -- 172.21.15.76:0/4273130037 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7f40bc017d90 con 0x56484188fc90 2023-11-09T03:50:14.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.332+0000 7f40ccb74740 1 -- 172.21.15.76:0/4273130037 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56484177bbf0 con 0x56484188fc90 2023-11-09T03:50:14.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.332+0000 7f40a27fc640 1 -- 172.21.15.76:0/4273130037 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40bc011a40 con 0x56484188fc90 2023-11-09T03:50:14.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.332+0000 7f40ccb74740 10 client.7944 fetch_fsmap learned FSMap version 65 2023-11-09T03:50:14.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.332+0000 7f40ccb74740 1 -- 172.21.15.76:0/4273130037 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x5648417cdf30 con 0x56484188fc90 2023-11-09T03:50:14.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.332+0000 7f40a27fc640 1 -- 172.21.15.76:0/4273130037 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56484177bbf0 con 0x56484188fc90 2023-11-09T03:50:14.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.332+0000 7f40a27fc640 1 -- 172.21.15.76:0/4273130037 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 65) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f40bc01d020 con 0x56484188fc90 2023-11-09T03:50:14.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.333+0000 7f40ccb74740 10 client.7944 fetch_fsmap finished waiting for FSMap version 65 2023-11-09T03:50:14.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.333+0000 7f40ccb74740 10 client.7944 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-09T03:50:14.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.333+0000 7f40ccb74740 20 client.7944 populate_metadata read hostname 'smithi076' 2023-11-09T03:50:14.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.333+0000 7f40ccb74740 1 --2- 172.21.15.76:0/4273130037 >> [v2:172.21.15.76:6834/1663988060,v1:172.21.15.76:6835/1663988060] conn(0x5648418c9b40 0x5648418cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.333+0000 7f40ccb74740 4 client.7944 mds_command: new command op to 7773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:50:14.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.333+0000 7f40ccb74740 1 -- 172.21.15.76:0/4273130037 --> [v2:172.21.15.76:6834/1663988060,v1:172.21.15.76:6835/1663988060] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5648417c8890 con 0x5648418c9b40 2023-11-09T03:50:14.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.333+0000 7f40c37fe640 1 --2- 172.21.15.76:0/4273130037 >> [v2:172.21.15.76:6834/1663988060,v1:172.21.15.76:6835/1663988060] conn(0x5648418c9b40 0x5648418cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.333+0000 7f40a27fc640 10 client.7944 ms_handle_connect on v2:172.21.15.76:6834/1663988060 2023-11-09T03:50:14.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.333+0000 7f40c37fe640 1 --2- 172.21.15.76:0/4273130037 >> [v2:172.21.15.76:6834/1663988060,v1:172.21.15.76:6835/1663988060] conn(0x5648418c9b40 0x5648418cbf30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7ed3bc6302cf6cc server_cookie=e8089680778302ca in_seq=0 out_seq=0 2023-11-09T03:50:14.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.333+0000 7f40a27fc640 1 -- 172.21.15.76:0/4273130037 <== mds.0 v2:172.21.15.76:6834/1663988060 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5648417c8890 con 0x5648418c9b40 2023-11-09T03:50:14.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.333+0000 7f40a27fc640 10 client.7944 handle_command_reply: tid=0 2023-11-09T03:50:14.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.333+0000 7f40ccb74740 1 client.7944 shutdown 2023-11-09T03:50:14.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.334+0000 7f40ccb74740 1 -- 172.21.15.76:0/4273130037 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56484188fc90 msgr2=0x5648418624f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.334+0000 7f40ccb74740 1 --2- 172.21.15.76:0/4273130037 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56484188fc90 0x5648418624f0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f40c40c9e50 tx=0x7f40bc007a90 comp rx=0 tx=0).stop 2023-11-09T03:50:14.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.334+0000 7f40ccb74740 1 -- 172.21.15.76:0/4273130037 shutdown_connections 2023-11-09T03:50:14.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.334+0000 7f40ccb74740 1 --2- 172.21.15.76:0/4273130037 >> [v2:172.21.15.76:6834/1663988060,v1:172.21.15.76:6835/1663988060] conn(0x5648418c9b40 0x5648418cbf30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.334+0000 7f40ccb74740 1 --2- 172.21.15.76:0/4273130037 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56484188fc90 0x5648418624f0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.334+0000 7f40ccb74740 1 --2- 172.21.15.76:0/4273130037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564841890c30 0x564841895620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.334+0000 7f40a27fc640 0 client.7944 ms_handle_reset on v2:172.21.15.76:6834/1663988060 2023-11-09T03:50:14.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.334+0000 7f40ccb74740 1 --2- 172.21.15.76:0/4273130037 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564841890850 0x564841884aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.334+0000 7f40ccb74740 1 -- 172.21.15.76:0/4273130037 >> 172.21.15.76:0/4273130037 conn(0x564841863c90 msgr2=0x5648418b2380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:14.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.334+0000 7f40ccb74740 1 -- 172.21.15.76:0/4273130037 shutdown_connections 2023-11-09T03:50:14.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.334+0000 7f40ccb74740 1 -- 172.21.15.76:0/4273130037 wait complete. 2023-11-09T03:50:14.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.334+0000 7f40ccb74740 20 client.7944 trim_cache size 0 max 16384 2023-11-09T03:50:14.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.349+0000 7f40ccb74740 1 Processor -- start 2023-11-09T03:50:14.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.349+0000 7f40ccb74740 1 -- start start 2023-11-09T03:50:14.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.350+0000 7f40ccb74740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56484188fc90 0x5648418937c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.350+0000 7f40ccb74740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564841890850 0x5648418b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.350+0000 7f40ccb74740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564841890c30 0x5648418b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.350+0000 7f40ccb74740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56484174f280 con 0x564841890c30 2023-11-09T03:50:14.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.350+0000 7f40ccb74740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648418c16e0 con 0x56484188fc90 2023-11-09T03:50:14.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.350+0000 7f40ccb74740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x56484186a880 con 0x564841890850 2023-11-09T03:50:14.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.350+0000 7f40c3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564841890c30 0x5648418b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.350+0000 7f40c3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564841890c30 0x5648418b2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41296/0 (socket says 172.21.15.76:41296) 2023-11-09T03:50:14.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.350+0000 7f40c3fff640 1 -- 172.21.15.76:0/213507124 learned_addr learned my addr 172.21.15.76:0/213507124 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:14.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.350+0000 7f40c3fff640 1 -- 172.21.15.76:0/213507124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564841890850 msgr2=0x5648418b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:14.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.350+0000 7f40c3fff640 1 --2- 172.21.15.76:0/213507124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564841890850 0x5648418b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.350+0000 7f40c3fff640 1 -- 172.21.15.76:0/213507124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56484188fc90 msgr2=0x5648418937c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.350+0000 7f40c37fe640 1 --2- 172.21.15.76:0/213507124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56484188fc90 0x5648418937c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.350+0000 7f40c3fff640 1 --2- 172.21.15.76:0/213507124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56484188fc90 0x5648418937c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.350+0000 7f40c3fff640 1 -- 172.21.15.76:0/213507124 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564841896b80 con 0x564841890c30 2023-11-09T03:50:14.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.350+0000 7f40c37fe640 1 --2- 172.21.15.76:0/213507124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56484188fc90 0x5648418937c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:50:14.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.351+0000 7f40c3fff640 1 --2- 172.21.15.76:0/213507124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564841890c30 0x5648418b2030 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f40bc007c40 tx=0x7f40bc00a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c252bba08202678 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:14.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.351+0000 7f40a9ffb640 1 -- 172.21.15.76:0/213507124 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc01a050 con 0x564841890c30 2023-11-09T03:50:14.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.351+0000 7f40a9ffb640 1 -- 172.21.15.76:0/213507124 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40bc01b040 con 0x564841890c30 2023-11-09T03:50:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.351+0000 7f40a9ffb640 1 -- 172.21.15.76:0/213507124 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc011a40 con 0x564841890c30 2023-11-09T03:50:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.351+0000 7f40ccb74740 1 -- 172.21.15.76:0/213507124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564841890c30 msgr2=0x5648418b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.351+0000 7f40ccb74740 1 --2- 172.21.15.76:0/213507124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564841890c30 0x5648418b2030 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f40bc007c40 tx=0x7f40bc00a000 comp rx=0 tx=0).stop 2023-11-09T03:50:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.351+0000 7f40ccb74740 1 -- 172.21.15.76:0/213507124 shutdown_connections 2023-11-09T03:50:14.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.351+0000 7f40ccb74740 1 --2- 172.21.15.76:0/213507124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564841890c30 0x5648418b2030 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.351+0000 7f40ccb74740 1 --2- 172.21.15.76:0/213507124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564841890850 0x5648418b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.351+0000 7f40ccb74740 1 --2- 172.21.15.76:0/213507124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56484188fc90 0x5648418937c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.351+0000 7f40ccb74740 1 -- 172.21.15.76:0/213507124 >> 172.21.15.76:0/213507124 conn(0x564841863c90 msgr2=0x56484185ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:14.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.351+0000 7f40ccb74740 1 -- 172.21.15.76:0/213507124 shutdown_connections 2023-11-09T03:50:14.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.351+0000 7f40ccb74740 1 -- 172.21.15.76:0/213507124 wait complete. 2023-11-09T03:50:14.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.352+0000 7f40ccb74740 1 -- start start 2023-11-09T03:50:14.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.352+0000 7f40ccb74740 1 Processor -- start 2023-11-09T03:50:14.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.353+0000 7f40ccb74740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56484188fc90 0x5648418b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.353+0000 7f40ccb74740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564841890850 0x5648418b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.353+0000 7f40ccb74740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564841890c30 0x56484188e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.353+0000 7f40ccb74740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x564841869d70 con 0x56484188fc90 2023-11-09T03:50:14.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.353+0000 7f40ccb74740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x56484188f9a0 con 0x564841890c30 2023-11-09T03:50:14.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.353+0000 7f40ccb74740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x564841869ac0 con 0x564841890850 2023-11-09T03:50:14.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.353+0000 7f40ccb74740 10 client.0 _ll_register_callbacks cb 0x7f40b80013f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:50:14.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.353+0000 7f40c37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56484188fc90 0x5648418b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.353+0000 7f40c37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56484188fc90 0x5648418b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41308/0 (socket says 172.21.15.76:41308) 2023-11-09T03:50:14.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.353+0000 7f40c37fe640 1 -- 172.21.15.76:0/3098664932 learned_addr learned my addr 172.21.15.76:0/3098664932 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:14.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.353+0000 7f40c3fff640 1 --2- 172.21.15.76:0/3098664932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564841890c30 0x56484188e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.353+0000 7f40c2ffd640 1 --2- 172.21.15.76:0/3098664932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564841890850 0x5648418b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.353+0000 7f40c37fe640 1 -- 172.21.15.76:0/3098664932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564841890850 msgr2=0x5648418b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:14.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.353+0000 7f40c37fe640 1 --2- 172.21.15.76:0/3098664932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564841890850 0x5648418b2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.353+0000 7f40c37fe640 1 -- 172.21.15.76:0/3098664932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564841890c30 msgr2=0x56484188e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:14.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.353+0000 7f40c37fe640 1 --2- 172.21.15.76:0/3098664932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564841890c30 0x56484188e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.353+0000 7f40c37fe640 1 -- 172.21.15.76:0/3098664932 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648417cdf30 con 0x56484188fc90 2023-11-09T03:50:14.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.353+0000 7f40c2ffd640 1 --2- 172.21.15.76:0/3098664932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564841890850 0x5648418b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:50:14.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.353+0000 7f40c37fe640 1 --2- 172.21.15.76:0/3098664932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56484188fc90 0x5648418b1af0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f40b402dfd0 tx=0x7f40b4006c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd34e44dc220b7d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:14.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.353+0000 7f40a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:50:14.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.353+0000 7f40a27fc640 1 -- 172.21.15.76:0/3098664932 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4009690 con 0x56484188fc90 2023-11-09T03:50:14.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.353+0000 7f40a27fc640 1 -- 172.21.15.76:0/3098664932 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7f40b40a5d90 con 0x56484188fc90 2023-11-09T03:50:14.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.353+0000 7f40ccb74740 1 -- 172.21.15.76:0/3098664932 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40b80159e0 con 0x56484188fc90 2023-11-09T03:50:14.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.353+0000 7f40a27fc640 1 -- 172.21.15.76:0/3098664932 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40b4076d90 con 0x56484188fc90 2023-11-09T03:50:14.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.353+0000 7f40ccb74740 10 client.7950 fetch_fsmap learned FSMap version 65 2023-11-09T03:50:14.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.353+0000 7f40ccb74740 1 -- 172.21.15.76:0/3098664932 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x56484176bd10 con 0x56484188fc90 2023-11-09T03:50:14.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.354+0000 7f40a27fc640 1 -- 172.21.15.76:0/3098664932 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b80159e0 con 0x56484188fc90 2023-11-09T03:50:14.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.354+0000 7f40a27fc640 1 -- 172.21.15.76:0/3098664932 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 65) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f40b404f070 con 0x56484188fc90 2023-11-09T03:50:14.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.354+0000 7f40ccb74740 10 client.7950 fetch_fsmap finished waiting for FSMap version 65 2023-11-09T03:50:14.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.354+0000 7f40ccb74740 10 client.7950 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-09T03:50:14.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.354+0000 7f40ccb74740 20 client.7950 populate_metadata read hostname 'smithi076' 2023-11-09T03:50:14.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.354+0000 7f40ccb74740 1 --2- 172.21.15.76:0/3098664932 >> [v2:172.21.15.76:6834/1663988060,v1:172.21.15.76:6835/1663988060] conn(0x564841895670 0x56484188ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.354+0000 7f40ccb74740 4 client.7950 mds_command: new command op to 7773 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-09T03:50:14.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.354+0000 7f40ccb74740 1 -- 172.21.15.76:0/3098664932 --> [v2:172.21.15.76:6834/1663988060,v1:172.21.15.76:6835/1663988060] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5648417c85b0 con 0x564841895670 2023-11-09T03:50:14.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.354+0000 7f40c2ffd640 1 --2- 172.21.15.76:0/3098664932 >> [v2:172.21.15.76:6834/1663988060,v1:172.21.15.76:6835/1663988060] conn(0x564841895670 0x56484188ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.354+0000 7f40a27fc640 10 client.7950 ms_handle_connect on v2:172.21.15.76:6834/1663988060 2023-11-09T03:50:14.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.354+0000 7f40c2ffd640 1 --2- 172.21.15.76:0/3098664932 >> [v2:172.21.15.76:6834/1663988060,v1:172.21.15.76:6835/1663988060] conn(0x564841895670 0x56484188ee40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3fbf73f765ec342 server_cookie=db3ae9ef04c05d73 in_seq=0 out_seq=0 2023-11-09T03:50:14.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.354+0000 7f40a27fc640 1 -- 172.21.15.76:0/3098664932 <== mds.0 v2:172.21.15.76:6834/1663988060 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5648417c85b0 con 0x564841895670 2023-11-09T03:50:14.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.354+0000 7f40a27fc640 10 client.7950 handle_command_reply: tid=0 2023-11-09T03:50:14.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.354+0000 7f40ccb74740 1 client.7950 shutdown 2023-11-09T03:50:14.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.354+0000 7f40ccb74740 1 -- 172.21.15.76:0/3098664932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56484188fc90 msgr2=0x5648418b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.354+0000 7f40ccb74740 1 --2- 172.21.15.76:0/3098664932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56484188fc90 0x5648418b1af0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f40b402dfd0 tx=0x7f40b4006c30 comp rx=0 tx=0).stop 2023-11-09T03:50:14.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.354+0000 7f40ccb74740 1 -- 172.21.15.76:0/3098664932 shutdown_connections 2023-11-09T03:50:14.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.354+0000 7f40ccb74740 1 --2- 172.21.15.76:0/3098664932 >> [v2:172.21.15.76:6834/1663988060,v1:172.21.15.76:6835/1663988060] conn(0x564841895670 0x56484188ee40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.354+0000 7f40ccb74740 1 --2- 172.21.15.76:0/3098664932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564841890c30 0x56484188e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.354+0000 7f40ccb74740 1 --2- 172.21.15.76:0/3098664932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564841890850 0x5648418b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.354+0000 7f40a27fc640 0 client.7950 ms_handle_reset on v2:172.21.15.76:6834/1663988060 2023-11-09T03:50:14.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.354+0000 7f40ccb74740 1 --2- 172.21.15.76:0/3098664932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56484188fc90 0x5648418b1af0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.354+0000 7f40ccb74740 1 -- 172.21.15.76:0/3098664932 >> 172.21.15.76:0/3098664932 conn(0x564841863c90 msgr2=0x5648418a73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:14.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.355+0000 7f40ccb74740 1 -- 172.21.15.76:0/3098664932 shutdown_connections 2023-11-09T03:50:14.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.355+0000 7f40ccb74740 1 -- 172.21.15.76:0/3098664932 wait complete. 2023-11-09T03:50:14.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.355+0000 7f40ccb74740 20 client.7950 trim_cache size 0 max 16384 2023-11-09T03:50:14.384 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:50:14.385 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "c213b26f-d0da-45d5-8e34-bb75c096d4f2", 2023-11-09T03:50:14.385 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2023-11-09T03:50:14.385 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 7773, 2023-11-09T03:50:14.385 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2023-11-09T03:50:14.386 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2023-11-09T03:50:14.386 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2023-11-09T03:50:14.386 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 6.7295342570000001, 2023-11-09T03:50:14.386 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 65, 2023-11-09T03:50:14.386 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 99, 2023-11-09T03:50:14.387 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 99, 2023-11-09T03:50:14.387 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 32.228243446 2023-11-09T03:50:14.387 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:50:14.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.355+0000 7f40c9fe2640 1 -- 172.21.15.76:0/477669423 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f40b006ae70 msgr2=0x7f40b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.355+0000 7f40c9fe2640 1 --2- 172.21.15.76:0/477669423 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f40b006ae70 0x7f40b006d330 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f40b8000c00 tx=0x7f40b8009040 comp rx=0 tx=0).stop 2023-11-09T03:50:14.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.355+0000 7f40c9fe2640 1 -- 172.21.15.76:0/477669423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40ac003970 msgr2=0x7f40c4147cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.355+0000 7f40c9fe2640 1 --2- 172.21.15.76:0/477669423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40ac003970 0x7f40c4147cc0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f40b402c250 tx=0x7f40b4004730 comp rx=0 tx=0).stop 2023-11-09T03:50:14.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.355+0000 7f40c9fe2640 1 -- 172.21.15.76:0/477669423 shutdown_connections 2023-11-09T03:50:14.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.355+0000 7f40c9fe2640 1 --2- 172.21.15.76:0/477669423 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f40b006ae70 0x7f40b006d330 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.355+0000 7f40c9fe2640 1 --2- 172.21.15.76:0/477669423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c40d2de0 0x7f40c40d3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.355+0000 7f40c9fe2640 1 --2- 172.21.15.76:0/477669423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c40cca60 0x7f40c41f21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.355+0000 7f40c9fe2640 1 --2- 172.21.15.76:0/477669423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40ac003970 0x7f40c4147cc0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.355+0000 7f40c9fe2640 1 -- 172.21.15.76:0/477669423 >> 172.21.15.76:0/477669423 conn(0x7f40c40cf880 msgr2=0x7f40c415e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:14.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.355+0000 7f40c9fe2640 1 -- 172.21.15.76:0/477669423 shutdown_connections 2023-11-09T03:50:14.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.355+0000 7f40c9fe2640 1 -- 172.21.15.76:0/477669423 wait complete. 2023-11-09T03:50:14.391 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:50:14.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.492+0000 7f7d62d1d640 1 Processor -- start 2023-11-09T03:50:14.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.492+0000 7f7d62d1d640 1 -- start start 2023-11-09T03:50:14.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.492+0000 7f7d62d1d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7d5c159c70 0x7f7d5c0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.492+0000 7f7d62d1d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7d5c0b0ae0 0x7f7d5c15d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.492+0000 7f7d62d1d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d5c0b4110 con 0x7f7d5c06db20 2023-11-09T03:50:14.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.492+0000 7f7d62d1d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d5c0b49b0 con 0x7f7d5c159c70 2023-11-09T03:50:14.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.492+0000 7f7d62d1d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d5c0b43f0 con 0x7f7d5c0b0ae0 2023-11-09T03:50:14.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.493+0000 7f7d60ab9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7d5c159c70 0x7f7d5c0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.493+0000 7f7d60ab9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7d5c159c70 0x7f7d5c0ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41312/0 (socket says 172.21.15.76:41312) 2023-11-09T03:50:14.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.493+0000 7f7d5bfff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f7d5c06db20 0x7f7d5c0b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:54138/0 (socket says 172.21.15.76:54138) 2023-11-09T03:50:14.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.493+0000 7f7d60ab9640 1 -- 172.21.15.76:0/3778713232 learned_addr learned my addr 172.21.15.76:0/3778713232 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:14.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.493+0000 7f7d612ba640 1 --2- 172.21.15.76:0/3778713232 >> v2:172.21.15.113:3300/0 conn(0x7f7d5c0b0ae0 0x7f7d5c15d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.493+0000 7f7d60ab9640 1 -- 172.21.15.76:0/3778713232 >> v2:172.21.15.113:3300/0 conn(0x7f7d5c0b0ae0 msgr2=0x7f7d5c15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:14.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.493+0000 7f7d60ab9640 1 --2- 172.21.15.76:0/3778713232 >> v2:172.21.15.113:3300/0 conn(0x7f7d5c0b0ae0 0x7f7d5c15d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.493+0000 7f7d60ab9640 1 -- 172.21.15.76:0/3778713232 >> v1:172.21.15.76:6789/0 conn(0x7f7d5c06db20 legacy=0x7f7d5c0b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.493+0000 7f7d5b7fe640 1 -- 172.21.15.76:0/3778713232 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592544420 0 0) 0x7f7d5c0b4110 con 0x7f7d5c06db20 2023-11-09T03:50:14.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.493+0000 7f7d60ab9640 1 -- 172.21.15.76:0/3778713232 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5c0b46d0 con 0x7f7d5c159c70 2023-11-09T03:50:14.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.494+0000 7f7d60ab9640 1 --2- 172.21.15.76:0/3778713232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d5c159c70 0x7f7d5c0ce370 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0c5910 tx=0x7f7d5002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411424971e8d2200 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:14.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.494+0000 7f7d5b7fe640 1 -- 172.21.15.76:0/3778713232 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50030860 con 0x7f7d5c159c70 2023-11-09T03:50:14.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.494+0000 7f7d5b7fe640 1 -- 172.21.15.76:0/3778713232 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d500309c0 con 0x7f7d5c159c70 2023-11-09T03:50:14.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.494+0000 7f7d5b7fe640 1 -- 172.21.15.76:0/3778713232 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50030d50 con 0x7f7d5c159c70 2023-11-09T03:50:14.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.494+0000 7f7d62d1d640 1 -- 172.21.15.76:0/3778713232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d5c159c70 msgr2=0x7f7d5c0ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.494+0000 7f7d62d1d640 1 --2- 172.21.15.76:0/3778713232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d5c159c70 0x7f7d5c0ce370 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0c5910 tx=0x7f7d5002b9a0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.494+0000 7f7d62d1d640 1 -- 172.21.15.76:0/3778713232 shutdown_connections 2023-11-09T03:50:14.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.494+0000 7f7d62d1d640 1 --2- 172.21.15.76:0/3778713232 >> v2:172.21.15.113:3300/0 conn(0x7f7d5c0b0ae0 0x7f7d5c15d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.494+0000 7f7d62d1d640 1 --2- 172.21.15.76:0/3778713232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d5c159c70 0x7f7d5c0ce370 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.494+0000 7f7d62d1d640 1 -- 172.21.15.76:0/3778713232 >> 172.21.15.76:0/3778713232 conn(0x7f7d5c0ce950 msgr2=0x7f7d5c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:14.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.494+0000 7f7d62d1d640 1 -- 172.21.15.76:0/3778713232 shutdown_connections 2023-11-09T03:50:14.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.494+0000 7f7d62d1d640 1 -- 172.21.15.76:0/3778713232 wait complete. 2023-11-09T03:50:14.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.496+0000 7f7d62d1d640 1 Processor -- start 2023-11-09T03:50:14.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.496+0000 7f7d62d1d640 1 -- start start 2023-11-09T03:50:14.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.496+0000 7f7d62d1d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d5c06db20 0x7f7d5c155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.496+0000 7f7d62d1d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d5c0b0ae0 0x7f7d5c1504b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.496+0000 7f7d62d1d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d5c159c70 0x7f7d5c151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.496+0000 7f7d62d1d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c14f6a0 con 0x7f7d5c0b0ae0 2023-11-09T03:50:14.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.496+0000 7f7d62d1d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c1f0060 con 0x7f7d5c06db20 2023-11-09T03:50:14.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.496+0000 7f7d62d1d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c1f0340 con 0x7f7d5c159c70 2023-11-09T03:50:14.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.497+0000 7f7d5bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d5c0b0ae0 0x7f7d5c1504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.497+0000 7f7d5bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d5c0b0ae0 0x7f7d5c1504b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41326/0 (socket says 172.21.15.76:41326) 2023-11-09T03:50:14.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.497+0000 7f7d5bfff640 1 -- 172.21.15.76:0/1892318302 learned_addr learned my addr 172.21.15.76:0/1892318302 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:14.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.497+0000 7f7d612ba640 1 --2- 172.21.15.76:0/1892318302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d5c159c70 0x7f7d5c151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.497+0000 7f7d60ab9640 1 --2- 172.21.15.76:0/1892318302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d5c06db20 0x7f7d5c155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.497+0000 7f7d5bfff640 1 -- 172.21.15.76:0/1892318302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d5c159c70 msgr2=0x7f7d5c151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.497+0000 7f7d5bfff640 1 --2- 172.21.15.76:0/1892318302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d5c159c70 0x7f7d5c151c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.497+0000 7f7d5bfff640 1 -- 172.21.15.76:0/1892318302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d5c06db20 msgr2=0x7f7d5c155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.497+0000 7f7d5bfff640 1 --2- 172.21.15.76:0/1892318302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d5c06db20 0x7f7d5c155540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.497+0000 7f7d5bfff640 1 -- 172.21.15.76:0/1892318302 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d500094d0 con 0x7f7d5c0b0ae0 2023-11-09T03:50:14.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.497+0000 7f7d60ab9640 1 --2- 172.21.15.76:0/1892318302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d5c06db20 0x7f7d5c155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:50:14.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.497+0000 7f7d5bfff640 1 --2- 172.21.15.76:0/1892318302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d5c0b0ae0 0x7f7d5c1504b0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800e730 tx=0x7f7d4800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:14.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.497+0000 7f7d597fa640 1 -- 172.21.15.76:0/1892318302 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4800ee40 con 0x7f7d5c0b0ae0 2023-11-09T03:50:14.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.497+0000 7f7d597fa640 1 -- 172.21.15.76:0/1892318302 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d480072e0 con 0x7f7d5c0b0ae0 2023-11-09T03:50:14.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.497+0000 7f7d62d1d640 1 -- 172.21.15.76:0/1892318302 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d5c14f9d0 con 0x7f7d5c0b0ae0 2023-11-09T03:50:14.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.497+0000 7f7d597fa640 1 -- 172.21.15.76:0/1892318302 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d48007690 con 0x7f7d5c0b0ae0 2023-11-09T03:50:14.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.497+0000 7f7d62d1d640 1 -- 172.21.15.76:0/1892318302 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d5c14ff90 con 0x7f7d5c0b0ae0 2023-11-09T03:50:14.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.498+0000 7f7d62d1d640 1 -- 172.21.15.76:0/1892318302 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d5c1501b0 con 0x7f7d5c0b0ae0 2023-11-09T03:50:14.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.499+0000 7f7d597fa640 1 -- 172.21.15.76:0/1892318302 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7d480016d0 con 0x7f7d5c0b0ae0 2023-11-09T03:50:14.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.499+0000 7f7d597fa640 1 --2- 172.21.15.76:0/1892318302 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7d4006ae70 0x7f7d4006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.499+0000 7f7d597fa640 1 -- 172.21.15.76:0/1892318302 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7f7d4808bed0 con 0x7f7d5c0b0ae0 2023-11-09T03:50:14.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.499+0000 7f7d60ab9640 1 --2- 172.21.15.76:0/1892318302 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7d4006ae70 0x7f7d4006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.500+0000 7f7d60ab9640 1 --2- 172.21.15.76:0/1892318302 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7d4006ae70 0x7f7d4006d330 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7d50036000 tx=0x7f7d50008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:14.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.503+0000 7f7d597fa640 1 -- 172.21.15.76:0/1892318302 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7d48056910 con 0x7f7d5c0b0ae0 2023-11-09T03:50:14.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.706+0000 7f7d62d1d640 1 -- 172.21.15.76:0/1892318302 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d5c074290 con 0x7f7d5c0b0ae0 2023-11-09T03:50:14.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.706+0000 7f7d597fa640 1 -- 172.21.15.76:0/1892318302 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f7d48015070 con 0x7f7d5c0b0ae0 2023-11-09T03:50:14.706 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:50:14.706 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3449910842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3449910842},{"type":"v1","addr":"172.21.15.113:6837","nonce":3449910842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:50:10.633468+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7773,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7773":{"gid":7773,"name":"c","rank":0,"incarnation":61,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/1663988060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1663988060},{"type":"v1","addr":"172.21.15.76:6835","nonce":1663988060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:50:14.707 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 65 2023-11-09T03:50:14.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.708+0000 7f7d62d1d640 1 -- 172.21.15.76:0/1892318302 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7d4006ae70 msgr2=0x7f7d4006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.708+0000 7f7d62d1d640 1 --2- 172.21.15.76:0/1892318302 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7d4006ae70 0x7f7d4006d330 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7d50036000 tx=0x7f7d50008030 comp rx=0 tx=0).stop 2023-11-09T03:50:14.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.708+0000 7f7d62d1d640 1 -- 172.21.15.76:0/1892318302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d5c0b0ae0 msgr2=0x7f7d5c1504b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.708+0000 7f7d62d1d640 1 --2- 172.21.15.76:0/1892318302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d5c0b0ae0 0x7f7d5c1504b0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800e730 tx=0x7f7d4800ebf0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.708+0000 7f7d62d1d640 1 -- 172.21.15.76:0/1892318302 shutdown_connections 2023-11-09T03:50:14.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.708+0000 7f7d62d1d640 1 --2- 172.21.15.76:0/1892318302 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7d4006ae70 0x7f7d4006d330 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.708+0000 7f7d62d1d640 1 --2- 172.21.15.76:0/1892318302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d5c159c70 0x7f7d5c151c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.708+0000 7f7d62d1d640 1 --2- 172.21.15.76:0/1892318302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d5c0b0ae0 0x7f7d5c1504b0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.708+0000 7f7d62d1d640 1 --2- 172.21.15.76:0/1892318302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d5c06db20 0x7f7d5c155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.708+0000 7f7d62d1d640 1 -- 172.21.15.76:0/1892318302 >> 172.21.15.76:0/1892318302 conn(0x7f7d5c0ce950 msgr2=0x7f7d5c158f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:14.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.708+0000 7f7d62d1d640 1 -- 172.21.15.76:0/1892318302 shutdown_connections 2023-11-09T03:50:14.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.708+0000 7f7d62d1d640 1 -- 172.21.15.76:0/1892318302 wait complete. 2023-11-09T03:50:14.719 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2023-11-09T03:50:14.832 INFO:teuthology.orchestra.run.smithi076.stdout:{"last_snap":3,"last_created":3,"last_destroyed":1,"pending_noop":[],"snaps":[{"snapid":2,"ino":1099511627777,"stamp":"2023-11-09T03:49:19.643623+0000","name":"s11","metadata":{}},{"snapid":3,"ino":1099511627777,"stamp":"2023-11-09T03:49:50.605488+0000","name":"s12","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-09T03:50:14.833 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 3, "last_destroyed": 1, "last_snap": 3, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s11", "snapid": 2, "stamp": "2023-11-09T03:49:19.643623+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s12", "snapid": 3, "stamp": "2023-11-09T03:49:50.605488+0000" } ] } 2023-11-09T03:50:14.833 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-09T03:50:14.833 INFO:tasks.cephfs.test_snapshots:testing snapserver mds_kill_mdstable_at=5 2023-11-09T03:50:14.834 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:50:14.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.934+0000 7fd996583640 1 Processor -- start 2023-11-09T03:50:14.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.934+0000 7fd996583640 1 -- start start 2023-11-09T03:50:14.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.935+0000 7fd996583640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd9900ce370 0x7fd99006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.935+0000 7fd996583640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd99006e060 con 0x7fd990159c70 2023-11-09T03:50:14.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.935+0000 7fd996583640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9900b4110 con 0x7fd9900b0ae0 2023-11-09T03:50:14.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.935+0000 7fd996583640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9900b49b0 con 0x7fd9900ce370 2023-11-09T03:50:14.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.935+0000 7fd98ffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd990159c70 0x7fd9900b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:54152/0 (socket says 172.21.15.76:54152) 2023-11-09T03:50:14.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.935+0000 7fd98ffff640 1 -- 172.21.15.76:0/4141222513 learned_addr learned my addr 172.21.15.76:0/4141222513 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:50:14.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.936+0000 7fd98effd640 1 -- 172.21.15.76:0/4141222513 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1567410255 0 0) 0x7fd99006e060 con 0x7fd990159c70 2023-11-09T03:50:14.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.936+0000 7fd98effd640 1 -- 172.21.15.76:0/4141222513 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd978003520 con 0x7fd990159c70 2023-11-09T03:50:14.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.936+0000 7fd994b20640 1 --2- 172.21.15.76:0/4141222513 >> v2:172.21.15.113:3300/0 conn(0x7fd9900ce370 0x7fd99006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.936+0000 7fd98effd640 1 -- 172.21.15.76:0/4141222513 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3664746068 0 0) 0x7fd978003520 con 0x7fd990159c70 2023-11-09T03:50:14.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.936+0000 7fd98effd640 1 -- 172.21.15.76:0/4141222513 >> v2:172.21.15.113:3300/0 conn(0x7fd9900ce370 msgr2=0x7fd99006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:14.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.936+0000 7fd98effd640 1 --2- 172.21.15.76:0/4141222513 >> v2:172.21.15.113:3300/0 conn(0x7fd9900ce370 0x7fd99006db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.936+0000 7fd98effd640 1 -- 172.21.15.76:0/4141222513 >> v1:172.21.15.113:6789/0 conn(0x7fd9900b0ae0 legacy=0x7fd99015d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.936+0000 7fd98effd640 1 -- 172.21.15.76:0/4141222513 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9900b43f0 con 0x7fd990159c70 2023-11-09T03:50:14.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.936+0000 7fd98effd640 1 -- 172.21.15.76:0/4141222513 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fd984001670 con 0x7fd990159c70 2023-11-09T03:50:14.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.936+0000 7fd98effd640 1 -- 172.21.15.76:0/4141222513 >> v1:172.21.15.76:6789/0 conn(0x7fd990159c70 legacy=0x7fd9900b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.936+0000 7fd98effd640 1 --2- 172.21.15.76:0/4141222513 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd978003970 0x7fd978003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.936+0000 7fd98effd640 1 --2- 172.21.15.76:0/4141222513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9780047c0 0x7fd978004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.936+0000 7fd98effd640 1 --2- 172.21.15.76:0/4141222513 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9780051d0 0x7fd97800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.936+0000 7fd98effd640 1 -- 172.21.15.76:0/4141222513 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9900b49b0 con 0x7fd9780051d0 2023-11-09T03:50:14.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.936+0000 7fd98effd640 1 -- 172.21.15.76:0/4141222513 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97800c100 con 0x7fd9780047c0 2023-11-09T03:50:14.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.936+0000 7fd98effd640 1 -- 172.21.15.76:0/4141222513 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97800c2c0 con 0x7fd978003970 2023-11-09T03:50:14.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.936+0000 7fd994b20640 1 --2- 172.21.15.76:0/4141222513 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9780051d0 0x7fd97800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.936+0000 7fd98f7fe640 1 --2- 172.21.15.76:0/4141222513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9780047c0 0x7fd978004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.936+0000 7fd98ffff640 1 --2- 172.21.15.76:0/4141222513 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd978003970 0x7fd978003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.936+0000 7fd994b20640 1 -- 172.21.15.76:0/4141222513 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd978003970 msgr2=0x7fd978003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:14.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.936+0000 7fd994b20640 1 --2- 172.21.15.76:0/4141222513 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd978003970 0x7fd978003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.936+0000 7fd994b20640 1 -- 172.21.15.76:0/4141222513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9780047c0 msgr2=0x7fd978004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:14.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.936+0000 7fd994b20640 1 --2- 172.21.15.76:0/4141222513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9780047c0 0x7fd978004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.936+0000 7fd994b20640 1 -- 172.21.15.76:0/4141222513 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd978003520 con 0x7fd9780051d0 2023-11-09T03:50:14.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.937+0000 7fd994b20640 1 -- 172.21.15.76:0/4141222513 reap_dead start 2023-11-09T03:50:14.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.937+0000 7fd994b20640 1 --2- 172.21.15.76:0/4141222513 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9780051d0 0x7fd97800b980 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fd988007360 tx=0x7fd9780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac3b32c01440b12 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:14.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.937+0000 7fd98effd640 1 -- 172.21.15.76:0/4141222513 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98802ccf0 con 0x7fd9780051d0 2023-11-09T03:50:14.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.937+0000 7fd98effd640 1 -- 172.21.15.76:0/4141222513 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98802ce90 con 0x7fd9780051d0 2023-11-09T03:50:14.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.937+0000 7fd98effd640 1 -- 172.21.15.76:0/4141222513 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd988003bd0 con 0x7fd9780051d0 2023-11-09T03:50:14.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.938+0000 7fd996583640 1 -- 172.21.15.76:0/4141222513 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9780051d0 msgr2=0x7fd97800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.938+0000 7fd996583640 1 --2- 172.21.15.76:0/4141222513 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9780051d0 0x7fd97800b980 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fd988007360 tx=0x7fd9780057f0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.938+0000 7fd996583640 1 -- 172.21.15.76:0/4141222513 shutdown_connections 2023-11-09T03:50:14.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.938+0000 7fd996583640 1 --2- 172.21.15.76:0/4141222513 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9780051d0 0x7fd97800b980 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.938+0000 7fd996583640 1 -- 172.21.15.76:0/4141222513 >> 172.21.15.76:0/4141222513 conn(0x7fd9900ce950 msgr2=0x7fd9900ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:14.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.938+0000 7fd996583640 1 -- 172.21.15.76:0/4141222513 shutdown_connections 2023-11-09T03:50:14.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.938+0000 7fd996583640 1 -- 172.21.15.76:0/4141222513 wait complete. 2023-11-09T03:50:14.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.939+0000 7fd996583640 1 Processor -- start 2023-11-09T03:50:14.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.939+0000 7fd996583640 1 -- start start 2023-11-09T03:50:14.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.940+0000 7fd996583640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9780051d0 0x7fd990156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.940+0000 7fd996583640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd990154cd0 0x7fd9901504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.940+0000 7fd996583640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9901e2920 0x7fd9901e2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.940+0000 7fd996583640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9900ce7e0 con 0x7fd9780051d0 2023-11-09T03:50:14.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.940+0000 7fd996583640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9901e2420 con 0x7fd9901e2920 2023-11-09T03:50:14.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.940+0000 7fd996583640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd990155150 con 0x7fd990154cd0 2023-11-09T03:50:14.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.940+0000 7fd98ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9780051d0 0x7fd990156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.940+0000 7fd98ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9780051d0 0x7fd990156450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41340/0 (socket says 172.21.15.76:41340) 2023-11-09T03:50:14.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.940+0000 7fd98ffff640 1 -- 172.21.15.76:0/2039839341 learned_addr learned my addr 172.21.15.76:0/2039839341 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:14.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.940+0000 7fd98f7fe640 1 --2- 172.21.15.76:0/2039839341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd990154cd0 0x7fd9901504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.940+0000 7fd994b20640 1 --2- 172.21.15.76:0/2039839341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9901e2920 0x7fd9901e2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.940+0000 7fd98ffff640 1 -- 172.21.15.76:0/2039839341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd990154cd0 msgr2=0x7fd9901504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.940+0000 7fd98ffff640 1 --2- 172.21.15.76:0/2039839341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd990154cd0 0x7fd9901504e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.940+0000 7fd98ffff640 1 -- 172.21.15.76:0/2039839341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9901e2920 msgr2=0x7fd9901e2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:14.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.940+0000 7fd98ffff640 1 --2- 172.21.15.76:0/2039839341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9901e2920 0x7fd9901e2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:14.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.940+0000 7fd98ffff640 1 -- 172.21.15.76:0/2039839341 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9901553b0 con 0x7fd9780051d0 2023-11-09T03:50:14.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.940+0000 7fd98ffff640 1 --2- 172.21.15.76:0/2039839341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9780051d0 0x7fd990156450 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fd9840072c0 tx=0x7fd984007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:14.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.940+0000 7fd98cff9640 1 -- 172.21.15.76:0/2039839341 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98400f960 con 0x7fd9780051d0 2023-11-09T03:50:14.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.940+0000 7fd98cff9640 1 -- 172.21.15.76:0/2039839341 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98400fac0 con 0x7fd9780051d0 2023-11-09T03:50:14.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.941+0000 7fd996583640 1 -- 172.21.15.76:0/2039839341 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd990155690 con 0x7fd9780051d0 2023-11-09T03:50:14.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.941+0000 7fd98cff9640 1 -- 172.21.15.76:0/2039839341 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98400fe70 con 0x7fd9780051d0 2023-11-09T03:50:14.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.941+0000 7fd996583640 1 -- 172.21.15.76:0/2039839341 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd990155c50 con 0x7fd9780051d0 2023-11-09T03:50:14.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.942+0000 7fd996583640 1 -- 172.21.15.76:0/2039839341 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd990074290 con 0x7fd9780051d0 2023-11-09T03:50:14.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.942+0000 7fd98cff9640 1 -- 172.21.15.76:0/2039839341 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd98401d460 con 0x7fd9780051d0 2023-11-09T03:50:14.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.942+0000 7fd98cff9640 1 --2- 172.21.15.76:0/2039839341 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd97c06aea0 0x7fd97c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:14.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.942+0000 7fd98cff9640 1 -- 172.21.15.76:0/2039839341 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7fd984089e20 con 0x7fd9780051d0 2023-11-09T03:50:14.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.943+0000 7fd98f7fe640 1 --2- 172.21.15.76:0/2039839341 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd97c06aea0 0x7fd97c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:14.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.943+0000 7fd98f7fe640 1 --2- 172.21.15.76:0/2039839341 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd97c06aea0 0x7fd97c06d360 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd980000c00 tx=0x7fd980009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:14.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:14.947+0000 7fd98cff9640 1 -- 172.21.15.76:0/2039839341 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd984054860 con 0x7fd9780051d0 2023-11-09T03:50:15.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.148+0000 7fd996583640 1 -- 172.21.15.76:0/2039839341 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9901e7260 con 0x7fd9780051d0 2023-11-09T03:50:15.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.148+0000 7fd98cff9640 1 -- 172.21.15.76:0/2039839341 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4934 (secure 0 0 0) 0x7fd984057e80 con 0x7fd9780051d0 2023-11-09T03:50:15.148 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:50:15.148 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3449910842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3449910842},{"type":"v1","addr":"172.21.15.113:6837","nonce":3449910842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:50:10.633468+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7773,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7773":{"gid":7773,"name":"c","rank":0,"incarnation":61,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/1663988060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1663988060},{"type":"v1","addr":"172.21.15.76:6835","nonce":1663988060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:50:15.149 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 65 2023-11-09T03:50:15.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.150+0000 7fd996583640 1 -- 172.21.15.76:0/2039839341 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd97c06aea0 msgr2=0x7fd97c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:15.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.150+0000 7fd996583640 1 --2- 172.21.15.76:0/2039839341 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd97c06aea0 0x7fd97c06d360 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd980000c00 tx=0x7fd980009040 comp rx=0 tx=0).stop 2023-11-09T03:50:15.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.150+0000 7fd996583640 1 -- 172.21.15.76:0/2039839341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9780051d0 msgr2=0x7fd990156450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:15.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.150+0000 7fd996583640 1 --2- 172.21.15.76:0/2039839341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9780051d0 0x7fd990156450 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fd9840072c0 tx=0x7fd984007780 comp rx=0 tx=0).stop 2023-11-09T03:50:15.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.150+0000 7fd996583640 1 -- 172.21.15.76:0/2039839341 shutdown_connections 2023-11-09T03:50:15.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.150+0000 7fd996583640 1 --2- 172.21.15.76:0/2039839341 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd97c06aea0 0x7fd97c06d360 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:15.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.150+0000 7fd996583640 1 --2- 172.21.15.76:0/2039839341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9901e2920 0x7fd9901e2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:15.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.150+0000 7fd996583640 1 --2- 172.21.15.76:0/2039839341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd990154cd0 0x7fd9901504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:15.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.150+0000 7fd996583640 1 --2- 172.21.15.76:0/2039839341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9780051d0 0x7fd990156450 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:15.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.150+0000 7fd996583640 1 -- 172.21.15.76:0/2039839341 >> 172.21.15.76:0/2039839341 conn(0x7fd9900ce950 msgr2=0x7fd990070710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:15.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.150+0000 7fd996583640 1 -- 172.21.15.76:0/2039839341 shutdown_connections 2023-11-09T03:50:15.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.150+0000 7fd996583640 1 -- 172.21.15.76:0/2039839341 wait complete. 2023-11-09T03:50:15.160 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:0 true 2023-11-09T03:50:15.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.262+0000 7fdcce8ec640 1 Processor -- start 2023-11-09T03:50:15.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.262+0000 7fdcce8ec640 1 -- start start 2023-11-09T03:50:15.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.263+0000 7fdcce8ec640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdcc8153e60 0x7fdcc815dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:15.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.263+0000 7fdcce8ec640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcc80b4880 0x7fdcc80b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:15.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.263+0000 7fdcce8ec640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc80cde60 con 0x7fdcc80cd6d0 2023-11-09T03:50:15.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.263+0000 7fdcce8ec640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcc8160f90 con 0x7fdcc8153e60 2023-11-09T03:50:15.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.263+0000 7fdcce8ec640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc80b5200 con 0x7fdcc80b4880 2023-11-09T03:50:15.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.263+0000 7fdcc7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdcc8153e60 0x7fdcc815dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:15.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.263+0000 7fdcc77fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fdcc80cd6d0 0x7fdcc8161420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:54160/0 (socket says 172.21.15.76:54160) 2023-11-09T03:50:15.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.263+0000 7fdcc77fe640 1 -- 172.21.15.76:0/3345887393 learned_addr learned my addr 172.21.15.76:0/3345887393 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:50:15.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.264+0000 7fdccce89640 1 --2- 172.21.15.76:0/3345887393 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcc80b4880 0x7fdcc80b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:15.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.264+0000 7fdcc7fff640 1 -- 172.21.15.76:0/3345887393 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcc80b4880 msgr2=0x7fdcc80b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:15.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.264+0000 7fdcc7fff640 1 --2- 172.21.15.76:0/3345887393 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcc80b4880 0x7fdcc80b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:15.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.264+0000 7fdcc7fff640 1 -- 172.21.15.76:0/3345887393 >> v1:172.21.15.76:6789/0 conn(0x7fdcc80cd6d0 legacy=0x7fdcc8161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:15.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.264+0000 7fdcc7fff640 1 -- 172.21.15.76:0/3345887393 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc815e400 con 0x7fdcc8153e60 2023-11-09T03:50:15.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.264+0000 7fdcc6ffd640 1 -- 172.21.15.76:0/3345887393 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2086363390 0 0) 0x7fdcc80cde60 con 0x7fdcc80cd6d0 2023-11-09T03:50:15.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.264+0000 7fdcc7fff640 1 --2- 172.21.15.76:0/3345887393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcc8153e60 0x7fdcc815dec0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80097a0 tx=0x7fdcb802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8703e2320419fad server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:15.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.265+0000 7fdcc6ffd640 1 -- 172.21.15.76:0/3345887393 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8031830 con 0x7fdcc8153e60 2023-11-09T03:50:15.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.265+0000 7fdcc6ffd640 1 -- 172.21.15.76:0/3345887393 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb8031990 con 0x7fdcc8153e60 2023-11-09T03:50:15.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.265+0000 7fdcc6ffd640 1 -- 172.21.15.76:0/3345887393 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8031d20 con 0x7fdcc8153e60 2023-11-09T03:50:15.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.265+0000 7fdcce8ec640 1 -- 172.21.15.76:0/3345887393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcc8153e60 msgr2=0x7fdcc815dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:15.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.265+0000 7fdcce8ec640 1 --2- 172.21.15.76:0/3345887393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcc8153e60 0x7fdcc815dec0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80097a0 tx=0x7fdcb802b8d0 comp rx=0 tx=0).stop 2023-11-09T03:50:15.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.265+0000 7fdcce8ec640 1 -- 172.21.15.76:0/3345887393 shutdown_connections 2023-11-09T03:50:15.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.265+0000 7fdcce8ec640 1 --2- 172.21.15.76:0/3345887393 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcc80b4880 0x7fdcc80b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:15.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.265+0000 7fdcce8ec640 1 --2- 172.21.15.76:0/3345887393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcc8153e60 0x7fdcc815dec0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:15.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.265+0000 7fdcce8ec640 1 -- 172.21.15.76:0/3345887393 >> 172.21.15.76:0/3345887393 conn(0x7fdcc80cf880 msgr2=0x7fdcc80cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:15.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.265+0000 7fdcce8ec640 1 -- 172.21.15.76:0/3345887393 shutdown_connections 2023-11-09T03:50:15.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.265+0000 7fdcce8ec640 1 -- 172.21.15.76:0/3345887393 wait complete. 2023-11-09T03:50:15.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.266+0000 7fdcce8ec640 1 Processor -- start 2023-11-09T03:50:15.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.267+0000 7fdcce8ec640 1 -- start start 2023-11-09T03:50:15.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.267+0000 7fdcce8ec640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcc80b4880 0x7fdcc80cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:15.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.267+0000 7fdcce8ec640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcc80cd6d0 0x7fdcc80ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:15.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.267+0000 7fdcce8ec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcc8153e60 0x7fdcc8152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:15.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.267+0000 7fdcce8ec640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc80caeb0 con 0x7fdcc8153e60 2023-11-09T03:50:15.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.267+0000 7fdcce8ec640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc80cbac0 con 0x7fdcc80b4880 2023-11-09T03:50:15.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.267+0000 7fdcce8ec640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc80cbda0 con 0x7fdcc80cd6d0 2023-11-09T03:50:15.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.267+0000 7fdccce89640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcc8153e60 0x7fdcc8152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:15.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.267+0000 7fdccce89640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcc8153e60 0x7fdcc8152f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41368/0 (socket says 172.21.15.76:41368) 2023-11-09T03:50:15.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.267+0000 7fdccce89640 1 -- 172.21.15.76:0/3326024607 learned_addr learned my addr 172.21.15.76:0/3326024607 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:15.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.268+0000 7fdcc77fe640 1 --2- 172.21.15.76:0/3326024607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcc80cd6d0 0x7fdcc80ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:15.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.268+0000 7fdcc7fff640 1 --2- 172.21.15.76:0/3326024607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcc80b4880 0x7fdcc80cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:15.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.268+0000 7fdccce89640 1 -- 172.21.15.76:0/3326024607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcc80cd6d0 msgr2=0x7fdcc80ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:15.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.268+0000 7fdccce89640 1 --2- 172.21.15.76:0/3326024607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcc80cd6d0 0x7fdcc80ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:15.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.268+0000 7fdccce89640 1 -- 172.21.15.76:0/3326024607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcc80b4880 msgr2=0x7fdcc80cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:15.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.268+0000 7fdccce89640 1 --2- 172.21.15.76:0/3326024607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcc80b4880 0x7fdcc80cb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:15.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.268+0000 7fdccce89640 1 -- 172.21.15.76:0/3326024607 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb8009400 con 0x7fdcc8153e60 2023-11-09T03:50:15.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.268+0000 7fdcc7fff640 1 --2- 172.21.15.76:0/3326024607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcc80b4880 0x7fdcc80cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:50:15.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.268+0000 7fdcc77fe640 1 --2- 172.21.15.76:0/3326024607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcc80cd6d0 0x7fdcc80ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:50:15.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.268+0000 7fdccce89640 1 --2- 172.21.15.76:0/3326024607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcc8153e60 0x7fdcc8152f40 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0007e40 tx=0x7fdcc000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:15.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.268+0000 7fdcc4ff9640 1 -- 172.21.15.76:0/3326024607 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0014620 con 0x7fdcc8153e60 2023-11-09T03:50:15.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.268+0000 7fdcc4ff9640 1 -- 172.21.15.76:0/3326024607 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc000ee90 con 0x7fdcc8153e60 2023-11-09T03:50:15.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.268+0000 7fdcce8ec640 1 -- 172.21.15.76:0/3326024607 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc80cc080 con 0x7fdcc8153e60 2023-11-09T03:50:15.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.268+0000 7fdcc4ff9640 1 -- 172.21.15.76:0/3326024607 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc00149e0 con 0x7fdcc8153e60 2023-11-09T03:50:15.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.268+0000 7fdcce8ec640 1 -- 172.21.15.76:0/3326024607 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc80cc640 con 0x7fdcc8153e60 2023-11-09T03:50:15.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.269+0000 7fdcce8ec640 1 -- 172.21.15.76:0/3326024607 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc8074290 con 0x7fdcc8153e60 2023-11-09T03:50:15.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.269+0000 7fdcc4ff9640 1 -- 172.21.15.76:0/3326024607 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdcc0014b40 con 0x7fdcc8153e60 2023-11-09T03:50:15.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.270+0000 7fdcc4ff9640 1 --2- 172.21.15.76:0/3326024607 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdcb406ae70 0x7fdcb406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:15.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.270+0000 7fdcc4ff9640 1 -- 172.21.15.76:0/3326024607 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7fdcc008a610 con 0x7fdcc8153e60 2023-11-09T03:50:15.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.270+0000 7fdcc7fff640 1 --2- 172.21.15.76:0/3326024607 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdcb406ae70 0x7fdcb406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:15.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.270+0000 7fdcc7fff640 1 --2- 172.21.15.76:0/3326024607 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdcb406ae70 0x7fdcb406d330 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8036000 tx=0x7fdcb803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:15.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.274+0000 7fdcc4ff9640 1 -- 172.21.15.76:0/3326024607 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdcc00550e0 con 0x7fdcc8153e60 2023-11-09T03:50:15.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:15.511+0000 7fdcce8ec640 1 -- 172.21.15.76:0/3326024607 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"} v 0) v1 -- 0x7fdcc80d1d00 con 0x7fdcc8153e60 2023-11-09T03:50:16.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.463+0000 7fdcc4ff9640 1 -- 172.21.15.76:0/3326024607 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"}]=0 freezing mds.7773 v66) v1 ==== 112+0+0 (secure 0 0 0) 0x7fdcc0058700 con 0x7fdcc8153e60 2023-11-09T03:50:16.463 INFO:teuthology.orchestra.run.smithi076.stderr:freezing mds.7773 2023-11-09T03:50:16.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.466+0000 7fdcce8ec640 1 -- 172.21.15.76:0/3326024607 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdcb406ae70 msgr2=0x7fdcb406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:16.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.466+0000 7fdcce8ec640 1 --2- 172.21.15.76:0/3326024607 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdcb406ae70 0x7fdcb406d330 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8036000 tx=0x7fdcb803c040 comp rx=0 tx=0).stop 2023-11-09T03:50:16.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.466+0000 7fdcce8ec640 1 -- 172.21.15.76:0/3326024607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcc8153e60 msgr2=0x7fdcc8152f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:16.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.466+0000 7fdcce8ec640 1 --2- 172.21.15.76:0/3326024607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcc8153e60 0x7fdcc8152f40 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0007e40 tx=0x7fdcc000e4b0 comp rx=0 tx=0).stop 2023-11-09T03:50:16.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.466+0000 7fdcce8ec640 1 -- 172.21.15.76:0/3326024607 shutdown_connections 2023-11-09T03:50:16.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.466+0000 7fdcce8ec640 1 --2- 172.21.15.76:0/3326024607 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdcb406ae70 0x7fdcb406d330 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:16.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.466+0000 7fdcce8ec640 1 --2- 172.21.15.76:0/3326024607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcc8153e60 0x7fdcc8152f40 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:16.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.466+0000 7fdcce8ec640 1 --2- 172.21.15.76:0/3326024607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcc80cd6d0 0x7fdcc80ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:16.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.466+0000 7fdcce8ec640 1 --2- 172.21.15.76:0/3326024607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcc80b4880 0x7fdcc80cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:16.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.466+0000 7fdcce8ec640 1 -- 172.21.15.76:0/3326024607 >> 172.21.15.76:0/3326024607 conn(0x7fdcc80cf880 msgr2=0x7fdcc80b8220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:16.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.466+0000 7fdcce8ec640 1 -- 172.21.15.76:0/3326024607 shutdown_connections 2023-11-09T03:50:16.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.466+0000 7fdcce8ec640 1 -- 172.21.15.76:0/3326024607 wait complete. 2023-11-09T03:50:16.478 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_kill_mdstable_at 5 2023-11-09T03:50:16.596 INFO:teuthology.orchestra.run.smithi076.stdout:{"success":"mds_kill_mdstable_at = '5' (not observed, change may require restart) "} 2023-11-09T03:50:16.596 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '5' (not observed, change may require restart) " } 2023-11-09T03:50:16.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:50:16.597 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s15) 2023-11-09T03:50:16.599 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2023-11-09T03:50:16.617 INFO:tasks.ceph.mds.c.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_commit(ceph::cref_t&)' thread 7fa5f6c70640 time 2023-11-09T03:50:16.618910+0000 2023-11-09T03:50:16.617 INFO:tasks.ceph.mds.c.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 145: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 5) 2023-11-09T03:50:16.618 INFO:tasks.ceph.mds.c.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:50:16.619 INFO:tasks.ceph.mds.c.smithi076.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fa5fc76efaf] 2023-11-09T03:50:16.619 INFO:tasks.ceph.mds.c.smithi076.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fa5fc76f16d] 2023-11-09T03:50:16.619 INFO:tasks.ceph.mds.c.smithi076.stderr: 3: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x7e7) [0x559e845bd187] 2023-11-09T03:50:16.619 INFO:tasks.ceph.mds.c.smithi076.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x559e8434c0a9] 2023-11-09T03:50:16.619 INFO:tasks.ceph.mds.c.smithi076.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x559e8434eb2b] 2023-11-09T03:50:16.620 INFO:tasks.ceph.mds.c.smithi076.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559e8434f01c] 2023-11-09T03:50:16.620 INFO:tasks.ceph.mds.c.smithi076.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x559e84338325] 2023-11-09T03:50:16.620 INFO:tasks.ceph.mds.c.smithi076.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7fa5fc95ff9a] 2023-11-09T03:50:16.620 INFO:tasks.ceph.mds.c.smithi076.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7fa5fc9f1741] 2023-11-09T03:50:16.620 INFO:tasks.ceph.mds.c.smithi076.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7fa5fbe9f802] 2023-11-09T03:50:16.621 INFO:tasks.ceph.mds.c.smithi076.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7fa5fbe3f450] 2023-11-09T03:50:16.621 INFO:tasks.ceph.mds.c.smithi076.stderr:2023-11-09T03:50:16.618+0000 7fa5f6c70640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_commit(ceph::cref_t&)' thread 7fa5f6c70640 time 2023-11-09T03:50:16.618910+0000 2023-11-09T03:50:16.621 INFO:tasks.ceph.mds.c.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 145: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 5) 2023-11-09T03:50:16.621 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:50:16.621 INFO:tasks.ceph.mds.c.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:50:16.622 INFO:tasks.ceph.mds.c.smithi076.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fa5fc76efaf] 2023-11-09T03:50:16.622 INFO:tasks.ceph.mds.c.smithi076.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fa5fc76f16d] 2023-11-09T03:50:16.622 INFO:tasks.ceph.mds.c.smithi076.stderr: 3: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x7e7) [0x559e845bd187] 2023-11-09T03:50:16.622 INFO:tasks.ceph.mds.c.smithi076.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x559e8434c0a9] 2023-11-09T03:50:16.622 INFO:tasks.ceph.mds.c.smithi076.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x559e8434eb2b] 2023-11-09T03:50:16.623 INFO:tasks.ceph.mds.c.smithi076.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559e8434f01c] 2023-11-09T03:50:16.623 INFO:tasks.ceph.mds.c.smithi076.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x559e84338325] 2023-11-09T03:50:16.623 INFO:tasks.ceph.mds.c.smithi076.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7fa5fc95ff9a] 2023-11-09T03:50:16.623 INFO:tasks.ceph.mds.c.smithi076.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7fa5fc9f1741] 2023-11-09T03:50:16.623 INFO:tasks.ceph.mds.c.smithi076.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7fa5fbe9f802] 2023-11-09T03:50:16.624 INFO:tasks.ceph.mds.c.smithi076.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7fa5fbe3f450] 2023-11-09T03:50:16.624 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:50:16.624 INFO:tasks.ceph.mds.c.smithi076.stderr:*** Caught signal (Aborted) ** 2023-11-09T03:50:16.624 INFO:tasks.ceph.mds.c.smithi076.stderr: in thread 7fa5f6c70640 thread_name:ms_dispatch 2023-11-09T03:50:16.625 INFO:tasks.ceph.mds.c.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:50:16.625 INFO:tasks.ceph.mds.c.smithi076.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fa5fbe54db0] 2023-11-09T03:50:16.626 INFO:tasks.ceph.mds.c.smithi076.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fa5fbea154c] 2023-11-09T03:50:16.626 INFO:tasks.ceph.mds.c.smithi076.stderr: 3: raise() 2023-11-09T03:50:16.626 INFO:tasks.ceph.mds.c.smithi076.stderr: 4: abort() 2023-11-09T03:50:16.626 INFO:tasks.ceph.mds.c.smithi076.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fa5fc76f009] 2023-11-09T03:50:16.626 INFO:tasks.ceph.mds.c.smithi076.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fa5fc76f16d] 2023-11-09T03:50:16.627 INFO:tasks.ceph.mds.c.smithi076.stderr: 7: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x7e7) [0x559e845bd187] 2023-11-09T03:50:16.627 INFO:tasks.ceph.mds.c.smithi076.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x559e8434c0a9] 2023-11-09T03:50:16.627 INFO:tasks.ceph.mds.c.smithi076.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x559e8434eb2b] 2023-11-09T03:50:16.627 INFO:tasks.ceph.mds.c.smithi076.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559e8434f01c] 2023-11-09T03:50:16.627 INFO:tasks.ceph.mds.c.smithi076.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x559e84338325] 2023-11-09T03:50:16.628 INFO:tasks.ceph.mds.c.smithi076.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7fa5fc95ff9a] 2023-11-09T03:50:16.628 INFO:tasks.ceph.mds.c.smithi076.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7fa5fc9f1741] 2023-11-09T03:50:16.628 INFO:tasks.ceph.mds.c.smithi076.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7fa5fbe9f802] 2023-11-09T03:50:16.628 INFO:tasks.ceph.mds.c.smithi076.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7fa5fbe3f450] 2023-11-09T03:50:16.628 INFO:tasks.ceph.mds.c.smithi076.stderr:2023-11-09T03:50:16.619+0000 7fa5f6c70640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:50:16.629 INFO:tasks.ceph.mds.c.smithi076.stderr: in thread 7fa5f6c70640 thread_name:ms_dispatch 2023-11-09T03:50:16.629 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:50:16.629 INFO:tasks.ceph.mds.c.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:50:16.629 INFO:tasks.ceph.mds.c.smithi076.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fa5fbe54db0] 2023-11-09T03:50:16.629 INFO:tasks.ceph.mds.c.smithi076.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fa5fbea154c] 2023-11-09T03:50:16.630 INFO:tasks.ceph.mds.c.smithi076.stderr: 3: raise() 2023-11-09T03:50:16.630 INFO:tasks.ceph.mds.c.smithi076.stderr: 4: abort() 2023-11-09T03:50:16.630 INFO:tasks.ceph.mds.c.smithi076.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fa5fc76f009] 2023-11-09T03:50:16.630 INFO:tasks.ceph.mds.c.smithi076.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fa5fc76f16d] 2023-11-09T03:50:16.630 INFO:tasks.ceph.mds.c.smithi076.stderr: 7: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x7e7) [0x559e845bd187] 2023-11-09T03:50:16.631 INFO:tasks.ceph.mds.c.smithi076.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x559e8434c0a9] 2023-11-09T03:50:16.631 INFO:tasks.ceph.mds.c.smithi076.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x559e8434eb2b] 2023-11-09T03:50:16.631 INFO:tasks.ceph.mds.c.smithi076.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559e8434f01c] 2023-11-09T03:50:16.631 INFO:tasks.ceph.mds.c.smithi076.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x559e84338325] 2023-11-09T03:50:16.631 INFO:tasks.ceph.mds.c.smithi076.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7fa5fc95ff9a] 2023-11-09T03:50:16.631 INFO:tasks.ceph.mds.c.smithi076.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7fa5fc9f1741] 2023-11-09T03:50:16.632 INFO:tasks.ceph.mds.c.smithi076.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7fa5fbe9f802] 2023-11-09T03:50:16.632 INFO:tasks.ceph.mds.c.smithi076.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7fa5fbe3f450] 2023-11-09T03:50:16.632 INFO:tasks.ceph.mds.c.smithi076.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:50:16.632 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:50:16.632 INFO:tasks.ceph.mds.c.smithi076.stderr: -203> 2023-11-09T03:50:13.048+0000 7fa5f446b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:50:16.633 INFO:tasks.ceph.mds.c.smithi076.stderr: -201> 2023-11-09T03:50:13.048+0000 7fa5f446b640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-09T03:50:16.633 INFO:tasks.ceph.mds.c.smithi076.stderr: -1> 2023-11-09T03:50:16.618+0000 7fa5f6c70640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_commit(ceph::cref_t&)' thread 7fa5f6c70640 time 2023-11-09T03:50:16.618910+0000 2023-11-09T03:50:16.633 INFO:tasks.ceph.mds.c.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 145: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 5) 2023-11-09T03:50:16.633 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:50:16.633 INFO:tasks.ceph.mds.c.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:50:16.634 INFO:tasks.ceph.mds.c.smithi076.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fa5fc76efaf] 2023-11-09T03:50:16.634 INFO:tasks.ceph.mds.c.smithi076.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fa5fc76f16d] 2023-11-09T03:50:16.634 INFO:tasks.ceph.mds.c.smithi076.stderr: 3: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x7e7) [0x559e845bd187] 2023-11-09T03:50:16.634 INFO:tasks.ceph.mds.c.smithi076.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x559e8434c0a9] 2023-11-09T03:50:16.634 INFO:tasks.ceph.mds.c.smithi076.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x559e8434eb2b] 2023-11-09T03:50:16.635 INFO:tasks.ceph.mds.c.smithi076.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559e8434f01c] 2023-11-09T03:50:16.635 INFO:tasks.ceph.mds.c.smithi076.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x559e84338325] 2023-11-09T03:50:16.635 INFO:tasks.ceph.mds.c.smithi076.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7fa5fc95ff9a] 2023-11-09T03:50:16.635 INFO:tasks.ceph.mds.c.smithi076.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7fa5fc9f1741] 2023-11-09T03:50:16.635 INFO:tasks.ceph.mds.c.smithi076.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7fa5fbe9f802] 2023-11-09T03:50:16.636 INFO:tasks.ceph.mds.c.smithi076.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7fa5fbe3f450] 2023-11-09T03:50:16.636 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:50:16.636 INFO:tasks.ceph.mds.c.smithi076.stderr: 0> 2023-11-09T03:50:16.619+0000 7fa5f6c70640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:50:16.636 INFO:tasks.ceph.mds.c.smithi076.stderr: in thread 7fa5f6c70640 thread_name:ms_dispatch 2023-11-09T03:50:16.636 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:50:16.637 INFO:tasks.ceph.mds.c.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:50:16.637 INFO:tasks.ceph.mds.c.smithi076.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fa5fbe54db0] 2023-11-09T03:50:16.637 INFO:tasks.ceph.mds.c.smithi076.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fa5fbea154c] 2023-11-09T03:50:16.637 INFO:tasks.ceph.mds.c.smithi076.stderr: 3: raise() 2023-11-09T03:50:16.637 INFO:tasks.ceph.mds.c.smithi076.stderr: 4: abort() 2023-11-09T03:50:16.637 INFO:tasks.ceph.mds.c.smithi076.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fa5fc76f009] 2023-11-09T03:50:16.638 INFO:tasks.ceph.mds.c.smithi076.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fa5fc76f16d] 2023-11-09T03:50:16.638 INFO:tasks.ceph.mds.c.smithi076.stderr: 7: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x7e7) [0x559e845bd187] 2023-11-09T03:50:16.638 INFO:tasks.ceph.mds.c.smithi076.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x559e8434c0a9] 2023-11-09T03:50:16.638 INFO:tasks.ceph.mds.c.smithi076.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x559e8434eb2b] 2023-11-09T03:50:16.638 INFO:tasks.ceph.mds.c.smithi076.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559e8434f01c] 2023-11-09T03:50:16.639 INFO:tasks.ceph.mds.c.smithi076.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x559e84338325] 2023-11-09T03:50:16.639 INFO:tasks.ceph.mds.c.smithi076.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7fa5fc95ff9a] 2023-11-09T03:50:16.639 INFO:tasks.ceph.mds.c.smithi076.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7fa5fc9f1741] 2023-11-09T03:50:16.639 INFO:tasks.ceph.mds.c.smithi076.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7fa5fbe9f802] 2023-11-09T03:50:16.639 INFO:tasks.ceph.mds.c.smithi076.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7fa5fbe3f450] 2023-11-09T03:50:16.640 INFO:tasks.ceph.mds.c.smithi076.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:50:16.640 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:50:16.640 INFO:tasks.ceph.mds.c.smithi076.stderr: -203> 2023-11-09T03:50:13.048+0000 7fa5f446b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:50:16.640 INFO:tasks.ceph.mds.c.smithi076.stderr: -201> 2023-11-09T03:50:13.048+0000 7fa5f446b640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-09T03:50:16.641 INFO:tasks.ceph.mds.c.smithi076.stderr: -1> 2023-11-09T03:50:16.618+0000 7fa5f6c70640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_commit(ceph::cref_t&)' thread 7fa5f6c70640 time 2023-11-09T03:50:16.618910+0000 2023-11-09T03:50:16.641 INFO:tasks.ceph.mds.c.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 145: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 5) 2023-11-09T03:50:16.641 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:50:16.641 INFO:tasks.ceph.mds.c.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:50:16.641 INFO:tasks.ceph.mds.c.smithi076.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fa5fc76efaf] 2023-11-09T03:50:16.642 INFO:tasks.ceph.mds.c.smithi076.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fa5fc76f16d] 2023-11-09T03:50:16.642 INFO:tasks.ceph.mds.c.smithi076.stderr: 3: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x7e7) [0x559e845bd187] 2023-11-09T03:50:16.642 INFO:tasks.ceph.mds.c.smithi076.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x559e8434c0a9] 2023-11-09T03:50:16.642 INFO:tasks.ceph.mds.c.smithi076.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x559e8434eb2b] 2023-11-09T03:50:16.642 INFO:tasks.ceph.mds.c.smithi076.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559e8434f01c] 2023-11-09T03:50:16.643 INFO:tasks.ceph.mds.c.smithi076.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x559e84338325] 2023-11-09T03:50:16.643 INFO:tasks.ceph.mds.c.smithi076.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7fa5fc95ff9a] 2023-11-09T03:50:16.643 INFO:tasks.ceph.mds.c.smithi076.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7fa5fc9f1741] 2023-11-09T03:50:16.643 INFO:tasks.ceph.mds.c.smithi076.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7fa5fbe9f802] 2023-11-09T03:50:16.643 INFO:tasks.ceph.mds.c.smithi076.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7fa5fbe3f450] 2023-11-09T03:50:16.644 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:50:16.644 INFO:tasks.ceph.mds.c.smithi076.stderr: 0> 2023-11-09T03:50:16.619+0000 7fa5f6c70640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:50:16.644 INFO:tasks.ceph.mds.c.smithi076.stderr: in thread 7fa5f6c70640 thread_name:ms_dispatch 2023-11-09T03:50:16.644 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:50:16.644 INFO:tasks.ceph.mds.c.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:50:16.645 INFO:tasks.ceph.mds.c.smithi076.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fa5fbe54db0] 2023-11-09T03:50:16.645 INFO:tasks.ceph.mds.c.smithi076.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fa5fbea154c] 2023-11-09T03:50:16.645 INFO:tasks.ceph.mds.c.smithi076.stderr: 3: raise() 2023-11-09T03:50:16.645 INFO:tasks.ceph.mds.c.smithi076.stderr: 4: abort() 2023-11-09T03:50:16.645 INFO:tasks.ceph.mds.c.smithi076.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fa5fc76f009] 2023-11-09T03:50:16.646 INFO:tasks.ceph.mds.c.smithi076.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fa5fc76f16d] 2023-11-09T03:50:16.646 INFO:tasks.ceph.mds.c.smithi076.stderr: 7: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x7e7) [0x559e845bd187] 2023-11-09T03:50:16.646 INFO:tasks.ceph.mds.c.smithi076.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x559e8434c0a9] 2023-11-09T03:50:16.646 INFO:tasks.ceph.mds.c.smithi076.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x559e8434eb2b] 2023-11-09T03:50:16.646 INFO:tasks.ceph.mds.c.smithi076.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559e8434f01c] 2023-11-09T03:50:16.646 INFO:tasks.ceph.mds.c.smithi076.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x559e84338325] 2023-11-09T03:50:16.647 INFO:tasks.ceph.mds.c.smithi076.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7fa5fc95ff9a] 2023-11-09T03:50:16.647 INFO:tasks.ceph.mds.c.smithi076.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7fa5fc9f1741] 2023-11-09T03:50:16.647 INFO:tasks.ceph.mds.c.smithi076.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7fa5fbe9f802] 2023-11-09T03:50:16.647 INFO:tasks.ceph.mds.c.smithi076.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7fa5fbe3f450] 2023-11-09T03:50:16.647 INFO:tasks.ceph.mds.c.smithi076.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:50:16.648 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:50:16.724 INFO:teuthology.orchestra.run.smithi076.stdout:{"mds_beacon_grace":"15.000000"} 2023-11-09T03:50:16.724 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2023-11-09T03:50:16.725 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:50:16.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.865+0000 7f7488333640 1 Processor -- start 2023-11-09T03:50:16.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.865+0000 7f7488333640 1 -- start start 2023-11-09T03:50:16.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.866+0000 7f7488333640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7480159da0 0x7f74800b5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:16.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.866+0000 7f7488333640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74800b5780 0x7f74800cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:16.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.866+0000 7f7488333640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74800cf740 con 0x7f74800cc970 2023-11-09T03:50:16.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.866+0000 7f7488333640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74800cce00 con 0x7f7480159da0 2023-11-09T03:50:16.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.866+0000 7f7488333640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74800cd690 con 0x7f74800b5780 2023-11-09T03:50:16.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.866+0000 7f74860cf640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7480159da0 0x7f74800b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:16.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.866+0000 7f74860cf640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7480159da0 0x7f74800b5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41382/0 (socket says 172.21.15.76:41382) 2023-11-09T03:50:16.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.866+0000 7f74860cf640 1 -- 172.21.15.76:0/1495411513 learned_addr learned my addr 172.21.15.76:0/1495411513 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:16.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.866+0000 7f74858ce640 1 --2- 172.21.15.76:0/1495411513 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74800b5780 0x7f74800cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:16.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.867+0000 7f74860cf640 1 -- 172.21.15.76:0/1495411513 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74800b5780 msgr2=0x7f74800cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:16.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.867+0000 7f74860cf640 1 --2- 172.21.15.76:0/1495411513 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74800b5780 0x7f74800cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:16.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.867+0000 7f74860cf640 1 -- 172.21.15.76:0/1495411513 >> v1:172.21.15.113:6789/0 conn(0x7f74800cc970 legacy=0x7f74800b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:16.869 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.867+0000 7f74860cf640 1 -- 172.21.15.76:0/1495411513 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74800cd970 con 0x7f7480159da0 2023-11-09T03:50:16.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.867+0000 7f74860cf640 1 --2- 172.21.15.76:0/1495411513 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7480159da0 0x7f74800b5240 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f7478009e50 tx=0x7f747802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d896c1fc40dbb725 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:16.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.868+0000 7f74850cd640 1 -- 172.21.15.76:0/1495411513 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747802cb00 con 0x7f7480159da0 2023-11-09T03:50:16.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.868+0000 7f74850cd640 1 -- 172.21.15.76:0/1495411513 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747802cc60 con 0x7f7480159da0 2023-11-09T03:50:16.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.868+0000 7f74850cd640 1 -- 172.21.15.76:0/1495411513 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74780343f0 con 0x7f7480159da0 2023-11-09T03:50:16.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.868+0000 7f7488333640 1 -- 172.21.15.76:0/1495411513 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7480159da0 msgr2=0x7f74800b5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:16.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.868+0000 7f7488333640 1 --2- 172.21.15.76:0/1495411513 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7480159da0 0x7f74800b5240 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f7478009e50 tx=0x7f747802bf60 comp rx=0 tx=0).stop 2023-11-09T03:50:16.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.868+0000 7f7488333640 1 -- 172.21.15.76:0/1495411513 shutdown_connections 2023-11-09T03:50:16.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.868+0000 7f7488333640 1 --2- 172.21.15.76:0/1495411513 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f74800b5780 0x7f74800cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:16.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.868+0000 7f7488333640 1 --2- 172.21.15.76:0/1495411513 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7480159da0 0x7f74800b5240 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:16.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.868+0000 7f7488333640 1 -- 172.21.15.76:0/1495411513 >> 172.21.15.76:0/1495411513 conn(0x7f748006d8b0 msgr2=0x7f748006dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:16.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.868+0000 7f7488333640 1 -- 172.21.15.76:0/1495411513 shutdown_connections 2023-11-09T03:50:16.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.868+0000 7f7488333640 1 -- 172.21.15.76:0/1495411513 wait complete. 2023-11-09T03:50:16.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.869+0000 7f7488333640 1 Processor -- start 2023-11-09T03:50:16.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.869+0000 7f7488333640 1 -- start start 2023-11-09T03:50:16.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.870+0000 7f7488333640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74800b5780 0x7f74800adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:16.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.870+0000 7f7488333640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74800cc970 0x7f748014a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:16.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.870+0000 7f7488333640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7480159da0 0x7f748014ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:16.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.870+0000 7f7488333640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74800ae3f0 con 0x7f74800b5780 2023-11-09T03:50:16.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.870+0000 7f7488333640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74800c8980 con 0x7f74800cc970 2023-11-09T03:50:16.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.870+0000 7f7488333640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74800c83c0 con 0x7f7480159da0 2023-11-09T03:50:16.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.870+0000 7f74860cf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74800b5780 0x7f74800adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:16.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.870+0000 7f74860cf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74800b5780 0x7f74800adeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41388/0 (socket says 172.21.15.76:41388) 2023-11-09T03:50:16.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.870+0000 7f74860cf640 1 -- 172.21.15.76:0/3303847543 learned_addr learned my addr 172.21.15.76:0/3303847543 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:16.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.870+0000 7f74860cf640 1 -- 172.21.15.76:0/3303847543 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7480159da0 msgr2=0x7f748014ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:16.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.870+0000 7f74860cf640 1 --2- 172.21.15.76:0/3303847543 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7480159da0 0x7f748014ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:16.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.870+0000 7f74860cf640 1 -- 172.21.15.76:0/3303847543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74800cc970 msgr2=0x7f748014a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:16.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.870+0000 7f74860cf640 1 --2- 172.21.15.76:0/3303847543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74800cc970 0x7f748014a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:16.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.870+0000 7f74860cf640 1 -- 172.21.15.76:0/3303847543 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7478009b30 con 0x7f74800b5780 2023-11-09T03:50:16.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.871+0000 7f74860cf640 1 --2- 172.21.15.76:0/3303847543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74800b5780 0x7f74800adeb0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f7478009e50 tx=0x7f7478004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:16.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.871+0000 7f7476ffd640 1 -- 172.21.15.76:0/3303847543 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7478009510 con 0x7f74800b5780 2023-11-09T03:50:16.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.871+0000 7f7476ffd640 1 -- 172.21.15.76:0/3303847543 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747803e450 con 0x7f74800b5780 2023-11-09T03:50:16.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.871+0000 7f7476ffd640 1 -- 172.21.15.76:0/3303847543 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747803e800 con 0x7f74800b5780 2023-11-09T03:50:16.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.871+0000 7f7488333640 1 -- 172.21.15.76:0/3303847543 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74800c86a0 con 0x7f74800b5780 2023-11-09T03:50:16.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.871+0000 7f7488333640 1 -- 172.21.15.76:0/3303847543 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74800af890 con 0x7f74800b5780 2023-11-09T03:50:16.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.872+0000 7f7488333640 1 -- 172.21.15.76:0/3303847543 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74800afa80 con 0x7f74800b5780 2023-11-09T03:50:16.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.872+0000 7f7476ffd640 1 -- 172.21.15.76:0/3303847543 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f747803e960 con 0x7f74800b5780 2023-11-09T03:50:16.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.872+0000 7f7476ffd640 1 --2- 172.21.15.76:0/3303847543 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f746406ae70 0x7f746406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:16.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.872+0000 7f7476ffd640 1 -- 172.21.15.76:0/3303847543 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7f74780aca10 con 0x7f74800b5780 2023-11-09T03:50:16.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.873+0000 7f74858ce640 1 --2- 172.21.15.76:0/3303847543 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f746406ae70 0x7f746406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:16.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.873+0000 7f74858ce640 1 --2- 172.21.15.76:0/3303847543 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f746406ae70 0x7f746406d330 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7470006fd0 tx=0x7f7470008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:16.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:16.877+0000 7f7476ffd640 1 -- 172.21.15.76:0/3303847543 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f74780774e0 con 0x7f74800b5780 2023-11-09T03:50:16.950 INFO:tasks.ceph.mds.c.smithi076.stderr:daemon-helper: command crashed with signal 6 2023-11-09T03:50:17.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:17.082+0000 7f7488333640 1 -- 172.21.15.76:0/3303847543 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74800674e0 con 0x7f74800b5780 2023-11-09T03:50:17.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:17.083+0000 7f7476ffd640 1 -- 172.21.15.76:0/3303847543 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f747807ab00 con 0x7f74800b5780 2023-11-09T03:50:17.083 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:50:17.083 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3449910842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3449910842},{"type":"v1","addr":"172.21.15.113:6837","nonce":3449910842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:50:15.512400+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7773,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7773":{"gid":7773,"name":"c","rank":0,"incarnation":61,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/1663988060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1663988060},{"type":"v1","addr":"172.21.15.76:6835","nonce":1663988060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:50:17.083 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 66 2023-11-09T03:50:17.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:17.085+0000 7f7488333640 1 -- 172.21.15.76:0/3303847543 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f746406ae70 msgr2=0x7f746406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:17.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:17.085+0000 7f7488333640 1 --2- 172.21.15.76:0/3303847543 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f746406ae70 0x7f746406d330 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7470006fd0 tx=0x7f7470008040 comp rx=0 tx=0).stop 2023-11-09T03:50:17.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:17.085+0000 7f7488333640 1 -- 172.21.15.76:0/3303847543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74800b5780 msgr2=0x7f74800adeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:17.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:17.085+0000 7f7488333640 1 --2- 172.21.15.76:0/3303847543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74800b5780 0x7f74800adeb0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f7478009e50 tx=0x7f7478004a40 comp rx=0 tx=0).stop 2023-11-09T03:50:17.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:17.085+0000 7f7488333640 1 -- 172.21.15.76:0/3303847543 shutdown_connections 2023-11-09T03:50:17.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:17.085+0000 7f7488333640 1 --2- 172.21.15.76:0/3303847543 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f746406ae70 0x7f746406d330 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:17.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:17.085+0000 7f7488333640 1 --2- 172.21.15.76:0/3303847543 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7480159da0 0x7f748014ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:17.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:17.085+0000 7f7488333640 1 --2- 172.21.15.76:0/3303847543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74800cc970 0x7f748014a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:17.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:17.085+0000 7f7488333640 1 --2- 172.21.15.76:0/3303847543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74800b5780 0x7f74800adeb0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:17.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:17.085+0000 7f7488333640 1 -- 172.21.15.76:0/3303847543 >> 172.21.15.76:0/3303847543 conn(0x7f748006d8b0 msgr2=0x7f748015a750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:17.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:17.085+0000 7f7488333640 1 -- 172.21.15.76:0/3303847543 shutdown_connections 2023-11-09T03:50:17.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:17.085+0000 7f7488333640 1 -- 172.21.15.76:0/3303847543 wait complete. 2023-11-09T03:50:17.095 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:50:19.303 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-11-09T03:50:22.096 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:50:22.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.205+0000 7fd2b7644640 1 Processor -- start 2023-11-09T03:50:22.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.205+0000 7fd2b7644640 1 -- start start 2023-11-09T03:50:22.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.206+0000 7fd2b7644640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd2b0159c70 0x7fd2b00ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:22.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.206+0000 7fd2b7644640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2b006db20 0x7fd2b00b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:22.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.206+0000 7fd2b7644640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2b00b4110 con 0x7fd2b00b0910 2023-11-09T03:50:22.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.206+0000 7fd2b7644640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b00b49b0 con 0x7fd2b0159c70 2023-11-09T03:50:22.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.206+0000 7fd2b7644640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b00b0da0 con 0x7fd2b006db20 2023-11-09T03:50:22.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.206+0000 7fd2b53e0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd2b0159c70 0x7fd2b00ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:22.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.206+0000 7fd2b53e0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd2b0159c70 0x7fd2b00ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41400/0 (socket says 172.21.15.76:41400) 2023-11-09T03:50:22.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.206+0000 7fd2b53e0640 1 -- 172.21.15.76:0/2978486889 learned_addr learned my addr 172.21.15.76:0/2978486889 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:22.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.206+0000 7fd2b4bdf640 1 --2- 172.21.15.76:0/2978486889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2b006db20 0x7fd2b00b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:22.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.206+0000 7fd2b53e0640 1 -- 172.21.15.76:0/2978486889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2b006db20 msgr2=0x7fd2b00b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:22.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.206+0000 7fd2b53e0640 1 --2- 172.21.15.76:0/2978486889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2b006db20 0x7fd2b00b03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:22.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.206+0000 7fd2b53e0640 1 -- 172.21.15.76:0/2978486889 >> v1:172.21.15.113:6789/0 conn(0x7fd2b00b0910 legacy=0x7fd2b015d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:22.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.206+0000 7fd2b53e0640 1 -- 172.21.15.76:0/2978486889 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b00b43f0 con 0x7fd2b0159c70 2023-11-09T03:50:22.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.207+0000 7fd2b53e0640 1 --2- 172.21.15.76:0/2978486889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2b0159c70 0x7fd2b00ce370 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009870 tx=0x7fd2a802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162fafdb50c6805c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:22.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.207+0000 7fd2a7fff640 1 -- 172.21.15.76:0/2978486889 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8030840 con 0x7fd2b0159c70 2023-11-09T03:50:22.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.207+0000 7fd2a7fff640 1 -- 172.21.15.76:0/2978486889 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a80309a0 con 0x7fd2b0159c70 2023-11-09T03:50:22.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.207+0000 7fd2a7fff640 1 -- 172.21.15.76:0/2978486889 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8030d30 con 0x7fd2b0159c70 2023-11-09T03:50:22.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.208+0000 7fd2b7644640 1 -- 172.21.15.76:0/2978486889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2b0159c70 msgr2=0x7fd2b00ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:22.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.208+0000 7fd2b7644640 1 --2- 172.21.15.76:0/2978486889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2b0159c70 0x7fd2b00ce370 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009870 tx=0x7fd2a802b980 comp rx=0 tx=0).stop 2023-11-09T03:50:22.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.208+0000 7fd2b7644640 1 -- 172.21.15.76:0/2978486889 shutdown_connections 2023-11-09T03:50:22.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.208+0000 7fd2b7644640 1 --2- 172.21.15.76:0/2978486889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2b006db20 0x7fd2b00b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:22.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.208+0000 7fd2b7644640 1 --2- 172.21.15.76:0/2978486889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2b0159c70 0x7fd2b00ce370 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:22.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.208+0000 7fd2b7644640 1 -- 172.21.15.76:0/2978486889 >> 172.21.15.76:0/2978486889 conn(0x7fd2b00ce950 msgr2=0x7fd2b00ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:22.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.208+0000 7fd2b7644640 1 -- 172.21.15.76:0/2978486889 shutdown_connections 2023-11-09T03:50:22.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.208+0000 7fd2b7644640 1 -- 172.21.15.76:0/2978486889 wait complete. 2023-11-09T03:50:22.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.209+0000 7fd2b7644640 1 Processor -- start 2023-11-09T03:50:22.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.209+0000 7fd2b7644640 1 -- start start 2023-11-09T03:50:22.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.210+0000 7fd2b7644640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2b006db20 0x7fd2b01f0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:22.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.210+0000 7fd2b7644640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2b00b0910 0x7fd2b01f10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:22.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.210+0000 7fd2b7644640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2b0159c70 0x7fd2b01f17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:22.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.210+0000 7fd2b7644640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b01f15e0 con 0x7fd2b0159c70 2023-11-09T03:50:22.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.210+0000 7fd2b7644640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b0153d30 con 0x7fd2b00b0910 2023-11-09T03:50:22.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.210+0000 7fd2b7644640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b0154010 con 0x7fd2b006db20 2023-11-09T03:50:22.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.210+0000 7fd2b5be1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2b0159c70 0x7fd2b01f17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:22.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.210+0000 7fd2b5be1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2b0159c70 0x7fd2b01f17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41406/0 (socket says 172.21.15.76:41406) 2023-11-09T03:50:22.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.210+0000 7fd2b5be1640 1 -- 172.21.15.76:0/1847853742 learned_addr learned my addr 172.21.15.76:0/1847853742 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:22.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.210+0000 7fd2b53e0640 1 --2- 172.21.15.76:0/1847853742 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2b006db20 0x7fd2b01f0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:22.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.210+0000 7fd2b4bdf640 1 --2- 172.21.15.76:0/1847853742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2b00b0910 0x7fd2b01f10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:22.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.210+0000 7fd2b5be1640 1 -- 172.21.15.76:0/1847853742 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2b006db20 msgr2=0x7fd2b01f0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:22.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.210+0000 7fd2b5be1640 1 --2- 172.21.15.76:0/1847853742 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2b006db20 0x7fd2b01f0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:22.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.210+0000 7fd2b5be1640 1 -- 172.21.15.76:0/1847853742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2b00b0910 msgr2=0x7fd2b01f10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:22.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.210+0000 7fd2b5be1640 1 --2- 172.21.15.76:0/1847853742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2b00b0910 0x7fd2b01f10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:22.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.210+0000 7fd2b5be1640 1 -- 172.21.15.76:0/1847853742 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a80094d0 con 0x7fd2b0159c70 2023-11-09T03:50:22.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.210+0000 7fd2b5be1640 1 --2- 172.21.15.76:0/1847853742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2b0159c70 0x7fd2b01f17e0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac00c7a0 tx=0x7fd2ac00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:22.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.210+0000 7fd2a5ffb640 1 -- 172.21.15.76:0/1847853742 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac003b70 con 0x7fd2b0159c70 2023-11-09T03:50:22.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.210+0000 7fd2a5ffb640 1 -- 172.21.15.76:0/1847853742 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ac003cf0 con 0x7fd2b0159c70 2023-11-09T03:50:22.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.210+0000 7fd2a5ffb640 1 -- 172.21.15.76:0/1847853742 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac003ea0 con 0x7fd2b0159c70 2023-11-09T03:50:22.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.210+0000 7fd2b7644640 1 -- 172.21.15.76:0/1847853742 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b01542f0 con 0x7fd2b0159c70 2023-11-09T03:50:22.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.211+0000 7fd2b7644640 1 -- 172.21.15.76:0/1847853742 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b01548b0 con 0x7fd2b0159c70 2023-11-09T03:50:22.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.212+0000 7fd2b7644640 1 -- 172.21.15.76:0/1847853742 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b0074290 con 0x7fd2b0159c70 2023-11-09T03:50:22.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.212+0000 7fd2a5ffb640 1 -- 172.21.15.76:0/1847853742 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd2ac00aa90 con 0x7fd2b0159c70 2023-11-09T03:50:22.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.212+0000 7fd2a5ffb640 1 --2- 172.21.15.76:0/1847853742 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd29406ae70 0x7fd29406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:22.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.212+0000 7fd2a5ffb640 1 -- 172.21.15.76:0/1847853742 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7fd2ac0898c0 con 0x7fd2b0159c70 2023-11-09T03:50:22.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.213+0000 7fd2b53e0640 1 --2- 172.21.15.76:0/1847853742 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd29406ae70 0x7fd29406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:22.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.213+0000 7fd2b53e0640 1 --2- 172.21.15.76:0/1847853742 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd29406ae70 0x7fd29406d330 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8036000 tx=0x7fd2a8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:22.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.217+0000 7fd2a5ffb640 1 -- 172.21.15.76:0/1847853742 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd2ac054440 con 0x7fd2b0159c70 2023-11-09T03:50:22.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.415+0000 7fd2b7644640 1 -- 172.21.15.76:0/1847853742 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2b006d3f0 con 0x7fd2b0159c70 2023-11-09T03:50:22.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.415+0000 7fd2a5ffb640 1 -- 172.21.15.76:0/1847853742 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4934 (secure 0 0 0) 0x7fd2ac057a60 con 0x7fd2b0159c70 2023-11-09T03:50:22.416 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:50:22.417 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3449910842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3449910842},{"type":"v1","addr":"172.21.15.113:6837","nonce":3449910842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:50:15.512400+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7773,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7773":{"gid":7773,"name":"c","rank":0,"incarnation":61,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/1663988060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1663988060},{"type":"v1","addr":"172.21.15.76:6835","nonce":1663988060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:50:22.417 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 66 2023-11-09T03:50:22.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.417+0000 7fd2b7644640 1 -- 172.21.15.76:0/1847853742 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd29406ae70 msgr2=0x7fd29406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:22.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.417+0000 7fd2b7644640 1 --2- 172.21.15.76:0/1847853742 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd29406ae70 0x7fd29406d330 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8036000 tx=0x7fd2a8008030 comp rx=0 tx=0).stop 2023-11-09T03:50:22.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.417+0000 7fd2b7644640 1 -- 172.21.15.76:0/1847853742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2b0159c70 msgr2=0x7fd2b01f17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:22.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.417+0000 7fd2b7644640 1 --2- 172.21.15.76:0/1847853742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2b0159c70 0x7fd2b01f17e0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac00c7a0 tx=0x7fd2ac00cc60 comp rx=0 tx=0).stop 2023-11-09T03:50:22.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.417+0000 7fd2b7644640 1 -- 172.21.15.76:0/1847853742 shutdown_connections 2023-11-09T03:50:22.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.417+0000 7fd2b7644640 1 --2- 172.21.15.76:0/1847853742 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd29406ae70 0x7fd29406d330 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:22.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.417+0000 7fd2b7644640 1 --2- 172.21.15.76:0/1847853742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2b0159c70 0x7fd2b01f17e0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:22.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.417+0000 7fd2b7644640 1 --2- 172.21.15.76:0/1847853742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2b00b0910 0x7fd2b01f10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:22.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.417+0000 7fd2b7644640 1 --2- 172.21.15.76:0/1847853742 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2b006db20 0x7fd2b01f0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:22.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.417+0000 7fd2b7644640 1 -- 172.21.15.76:0/1847853742 >> 172.21.15.76:0/1847853742 conn(0x7fd2b00ce950 msgr2=0x7fd2b00ce210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:22.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.417+0000 7fd2b7644640 1 -- 172.21.15.76:0/1847853742 shutdown_connections 2023-11-09T03:50:22.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:22.417+0000 7fd2b7644640 1 -- 172.21.15.76:0/1847853742 wait complete. 2023-11-09T03:50:22.429 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:50:25.914 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~7s 2023-11-09T03:50:27.430 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:50:27.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.540+0000 7fb21ab6c640 1 Processor -- start 2023-11-09T03:50:27.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.540+0000 7fb21ab6c640 1 -- start start 2023-11-09T03:50:27.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.541+0000 7fb21ab6c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb21415ad80 0x7fb21406c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:27.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.541+0000 7fb21ab6c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb2140b42c0 0x7fb2140c9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:27.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.541+0000 7fb21ab6c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2140ca070 con 0x7fb2140c9c40 2023-11-09T03:50:27.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.541+0000 7fb21ab6c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb21406cd00 con 0x7fb2140b42c0 2023-11-09T03:50:27.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.541+0000 7fb21ab6c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb21406cfa0 con 0x7fb21415ad80 2023-11-09T03:50:27.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.541+0000 7fb20bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb2140b42c0 0x7fb2140c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:27.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.541+0000 7fb219109640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb2140c9c40 0x7fb214070630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:49292/0 (socket says 172.21.15.76:49292) 2023-11-09T03:50:27.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.541+0000 7fb219109640 1 -- 172.21.15.76:0/1771265589 learned_addr learned my addr 172.21.15.76:0/1771265589 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:50:27.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.541+0000 7fb218908640 1 --2- 172.21.15.76:0/1771265589 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb21415ad80 0x7fb21406c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:27.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.541+0000 7fb20bfff640 1 -- 172.21.15.76:0/1771265589 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb21415ad80 msgr2=0x7fb21406c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:27.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.541+0000 7fb20bfff640 1 --2- 172.21.15.76:0/1771265589 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb21415ad80 0x7fb21406c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:27.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.541+0000 7fb20bfff640 1 -- 172.21.15.76:0/1771265589 >> v1:172.21.15.76:6789/0 conn(0x7fb2140c9c40 legacy=0x7fb214070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:27.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.541+0000 7fb20bfff640 1 -- 172.21.15.76:0/1771265589 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21406d280 con 0x7fb2140b42c0 2023-11-09T03:50:27.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.541+0000 7fb20b7fe640 1 -- 172.21.15.76:0/1771265589 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 428950431 0 0) 0x7fb2140ca070 con 0x7fb2140c9c40 2023-11-09T03:50:27.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.541+0000 7fb218908640 1 --2- 172.21.15.76:0/1771265589 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb21415ad80 0x7fb21406c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:50:27.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.542+0000 7fb20bfff640 1 --2- 172.21.15.76:0/1771265589 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2140b42c0 0x7fb2140c9700 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fb2040097a0 tx=0x7fb20402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764672ab3d0ca9a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:27.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.542+0000 7fb20b7fe640 1 -- 172.21.15.76:0/1771265589 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb204030610 con 0x7fb2140b42c0 2023-11-09T03:50:27.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.542+0000 7fb20b7fe640 1 -- 172.21.15.76:0/1771265589 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb204030770 con 0x7fb2140b42c0 2023-11-09T03:50:27.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.542+0000 7fb20b7fe640 1 -- 172.21.15.76:0/1771265589 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb204030b00 con 0x7fb2140b42c0 2023-11-09T03:50:27.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.543+0000 7fb21ab6c640 1 -- 172.21.15.76:0/1771265589 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2140b42c0 msgr2=0x7fb2140c9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:27.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.543+0000 7fb21ab6c640 1 --2- 172.21.15.76:0/1771265589 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2140b42c0 0x7fb2140c9700 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fb2040097a0 tx=0x7fb20402b960 comp rx=0 tx=0).stop 2023-11-09T03:50:27.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.543+0000 7fb21ab6c640 1 -- 172.21.15.76:0/1771265589 shutdown_connections 2023-11-09T03:50:27.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.543+0000 7fb21ab6c640 1 --2- 172.21.15.76:0/1771265589 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2140b42c0 0x7fb2140c9700 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:27.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.543+0000 7fb21ab6c640 1 --2- 172.21.15.76:0/1771265589 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb21415ad80 0x7fb21406c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:27.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.543+0000 7fb21ab6c640 1 -- 172.21.15.76:0/1771265589 >> 172.21.15.76:0/1771265589 conn(0x7fb2140ce950 msgr2=0x7fb2140ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:27.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.543+0000 7fb21ab6c640 1 -- 172.21.15.76:0/1771265589 shutdown_connections 2023-11-09T03:50:27.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.543+0000 7fb21ab6c640 1 -- 172.21.15.76:0/1771265589 wait complete. 2023-11-09T03:50:27.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.544+0000 7fb21ab6c640 1 Processor -- start 2023-11-09T03:50:27.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.544+0000 7fb21ab6c640 1 -- start start 2023-11-09T03:50:27.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.545+0000 7fb21ab6c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2140b42c0 0x7fb214153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:27.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.545+0000 7fb21ab6c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2140c9c40 0x7fb2141f3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:27.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.545+0000 7fb21ab6c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb21415ad80 0x7fb2140b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:27.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.545+0000 7fb21ab6c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2141f4610 con 0x7fb2140c9c40 2023-11-09T03:50:27.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.545+0000 7fb21ab6c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2141f48f0 con 0x7fb21415ad80 2023-11-09T03:50:27.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.545+0000 7fb21ab6c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2141f4bd0 con 0x7fb2140b42c0 2023-11-09T03:50:27.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.545+0000 7fb20bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2140c9c40 0x7fb2141f3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:27.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.545+0000 7fb20bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2140c9c40 0x7fb2141f3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39870/0 (socket says 172.21.15.76:39870) 2023-11-09T03:50:27.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.545+0000 7fb20bfff640 1 -- 172.21.15.76:0/530635440 learned_addr learned my addr 172.21.15.76:0/530635440 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:27.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.545+0000 7fb219109640 1 --2- 172.21.15.76:0/530635440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb21415ad80 0x7fb2140b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:27.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.545+0000 7fb218908640 1 --2- 172.21.15.76:0/530635440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2140b42c0 0x7fb214153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:27.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.545+0000 7fb20bfff640 1 -- 172.21.15.76:0/530635440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2140b42c0 msgr2=0x7fb214153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:27.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.545+0000 7fb20bfff640 1 --2- 172.21.15.76:0/530635440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2140b42c0 0x7fb214153fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:27.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.545+0000 7fb20bfff640 1 -- 172.21.15.76:0/530635440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb21415ad80 msgr2=0x7fb2140b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:27.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.545+0000 7fb20bfff640 1 --2- 172.21.15.76:0/530635440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb21415ad80 0x7fb2140b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:27.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.545+0000 7fb20bfff640 1 -- 172.21.15.76:0/530635440 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb204009400 con 0x7fb2140c9c40 2023-11-09T03:50:27.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.545+0000 7fb20bfff640 1 --2- 172.21.15.76:0/530635440 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2140c9c40 0x7fb2141f3d10 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fb20402bf70 tx=0x7fb2040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:27.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.545+0000 7fb2097fa640 1 -- 172.21.15.76:0/530635440 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20403e070 con 0x7fb2140c9c40 2023-11-09T03:50:27.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.546+0000 7fb2097fa640 1 -- 172.21.15.76:0/530635440 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb204039a10 con 0x7fb2140c9c40 2023-11-09T03:50:27.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.546+0000 7fb2097fa640 1 -- 172.21.15.76:0/530635440 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb204039da0 con 0x7fb2140c9c40 2023-11-09T03:50:27.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.546+0000 7fb21ab6c640 1 -- 172.21.15.76:0/530635440 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb21415c920 con 0x7fb2140c9c40 2023-11-09T03:50:27.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.546+0000 7fb21ab6c640 1 -- 172.21.15.76:0/530635440 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb214154500 con 0x7fb2140c9c40 2023-11-09T03:50:27.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.546+0000 7fb21ab6c640 1 -- 172.21.15.76:0/530635440 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb21415ce40 con 0x7fb2140c9c40 2023-11-09T03:50:27.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.547+0000 7fb2097fa640 1 -- 172.21.15.76:0/530635440 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb20404b400 con 0x7fb2140c9c40 2023-11-09T03:50:27.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.547+0000 7fb2097fa640 1 --2- 172.21.15.76:0/530635440 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb20006ae70 0x7fb20006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:27.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.547+0000 7fb2097fa640 1 -- 172.21.15.76:0/530635440 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7fb2040ac0e0 con 0x7fb2140c9c40 2023-11-09T03:50:27.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.548+0000 7fb218908640 1 --2- 172.21.15.76:0/530635440 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb20006ae70 0x7fb20006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:27.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.548+0000 7fb218908640 1 --2- 172.21.15.76:0/530635440 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb20006ae70 0x7fb20006d330 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb21000a990 tx=0x7fb210008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:27.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.552+0000 7fb2097fa640 1 -- 172.21.15.76:0/530635440 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb21415ce40 con 0x7fb2140c9c40 2023-11-09T03:50:27.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.753+0000 7fb21ab6c640 1 -- 172.21.15.76:0/530635440 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb21415ce40 con 0x7fb2140c9c40 2023-11-09T03:50:27.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.754+0000 7fb2097fa640 1 -- 172.21.15.76:0/530635440 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4934 (secure 0 0 0) 0x7fb21415ce40 con 0x7fb2140c9c40 2023-11-09T03:50:27.754 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:50:27.754 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3449910842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3449910842},{"type":"v1","addr":"172.21.15.113:6837","nonce":3449910842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:50:15.512400+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7773,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7773":{"gid":7773,"name":"c","rank":0,"incarnation":61,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/1663988060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1663988060},{"type":"v1","addr":"172.21.15.76:6835","nonce":1663988060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:50:27.755 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 66 2023-11-09T03:50:27.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.755+0000 7fb21ab6c640 1 -- 172.21.15.76:0/530635440 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb20006ae70 msgr2=0x7fb20006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:27.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.755+0000 7fb21ab6c640 1 --2- 172.21.15.76:0/530635440 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb20006ae70 0x7fb20006d330 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb21000a990 tx=0x7fb210008040 comp rx=0 tx=0).stop 2023-11-09T03:50:27.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.755+0000 7fb21ab6c640 1 -- 172.21.15.76:0/530635440 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2140c9c40 msgr2=0x7fb2141f3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:27.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.755+0000 7fb21ab6c640 1 --2- 172.21.15.76:0/530635440 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2140c9c40 0x7fb2141f3d10 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fb20402bf70 tx=0x7fb2040046c0 comp rx=0 tx=0).stop 2023-11-09T03:50:27.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.755+0000 7fb21ab6c640 1 -- 172.21.15.76:0/530635440 shutdown_connections 2023-11-09T03:50:27.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.756+0000 7fb21ab6c640 1 --2- 172.21.15.76:0/530635440 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb20006ae70 0x7fb20006d330 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:27.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.756+0000 7fb21ab6c640 1 --2- 172.21.15.76:0/530635440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb21415ad80 0x7fb2140b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:27.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.756+0000 7fb21ab6c640 1 --2- 172.21.15.76:0/530635440 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2140c9c40 0x7fb2141f3d10 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:27.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.756+0000 7fb21ab6c640 1 --2- 172.21.15.76:0/530635440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2140b42c0 0x7fb214153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:27.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.756+0000 7fb21ab6c640 1 -- 172.21.15.76:0/530635440 >> 172.21.15.76:0/530635440 conn(0x7fb2140ce950 msgr2=0x7fb2140b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:27.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.756+0000 7fb21ab6c640 1 -- 172.21.15.76:0/530635440 shutdown_connections 2023-11-09T03:50:27.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:27.756+0000 7fb21ab6c640 1 -- 172.21.15.76:0/530635440 wait complete. 2023-11-09T03:50:27.767 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:50:31.548 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:50:31.551 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:50:32.527 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~13s 2023-11-09T03:50:32.768 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:50:32.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.879+0000 7fddebc90640 1 Processor -- start 2023-11-09T03:50:32.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.879+0000 7fddebc90640 1 -- start start 2023-11-09T03:50:32.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.880+0000 7fddebc90640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdde4159c70 0x7fdde40ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:32.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.880+0000 7fddebc90640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdde40b0ae0 0x7fdde415d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:32.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.880+0000 7fddebc90640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde40b4110 con 0x7fdde406db20 2023-11-09T03:50:32.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.880+0000 7fddebc90640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdde40b49b0 con 0x7fdde4159c70 2023-11-09T03:50:32.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.880+0000 7fddebc90640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdde40b43f0 con 0x7fdde40b0ae0 2023-11-09T03:50:32.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.880+0000 7fdde9a2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdde4159c70 0x7fdde40ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:32.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.880+0000 7fdde922b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fdde406db20 0x7fdde40b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:55046/0 (socket says 172.21.15.76:55046) 2023-11-09T03:50:32.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.880+0000 7fdde922b640 1 -- 172.21.15.76:0/225785260 learned_addr learned my addr 172.21.15.76:0/225785260 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:50:32.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.880+0000 7fddea22d640 1 --2- 172.21.15.76:0/225785260 >> v2:172.21.15.113:3300/0 conn(0x7fdde40b0ae0 0x7fdde415d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:32.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.880+0000 7fdde9a2c640 1 -- 172.21.15.76:0/225785260 >> v2:172.21.15.113:3300/0 conn(0x7fdde40b0ae0 msgr2=0x7fdde415d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:32.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.880+0000 7fdde9a2c640 1 --2- 172.21.15.76:0/225785260 >> v2:172.21.15.113:3300/0 conn(0x7fdde40b0ae0 0x7fdde415d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:32.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.880+0000 7fdde9a2c640 1 -- 172.21.15.76:0/225785260 >> v1:172.21.15.76:6789/0 conn(0x7fdde406db20 legacy=0x7fdde40b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:32.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.880+0000 7fdde8a2a640 1 -- 172.21.15.76:0/225785260 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2964164632 0 0) 0x7fdde40b4110 con 0x7fdde406db20 2023-11-09T03:50:32.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.880+0000 7fdde9a2c640 1 -- 172.21.15.76:0/225785260 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde40b46d0 con 0x7fdde4159c70 2023-11-09T03:50:32.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.881+0000 7fdde9a2c640 1 --2- 172.21.15.76:0/225785260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdde4159c70 0x7fdde40ce370 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fdde40c5910 tx=0x7fddd402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05c39430bf98ea4 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:32.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.881+0000 7fdde8a2a640 1 -- 172.21.15.76:0/225785260 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd4031830 con 0x7fdde4159c70 2023-11-09T03:50:32.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.881+0000 7fdde8a2a640 1 -- 172.21.15.76:0/225785260 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd4031990 con 0x7fdde4159c70 2023-11-09T03:50:32.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.881+0000 7fdde8a2a640 1 -- 172.21.15.76:0/225785260 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd4031d20 con 0x7fdde4159c70 2023-11-09T03:50:32.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.882+0000 7fddebc90640 1 -- 172.21.15.76:0/225785260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdde4159c70 msgr2=0x7fdde40ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:32.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.882+0000 7fddebc90640 1 --2- 172.21.15.76:0/225785260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdde4159c70 0x7fdde40ce370 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fdde40c5910 tx=0x7fddd402b8d0 comp rx=0 tx=0).stop 2023-11-09T03:50:32.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.882+0000 7fddebc90640 1 -- 172.21.15.76:0/225785260 shutdown_connections 2023-11-09T03:50:32.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.882+0000 7fddebc90640 1 --2- 172.21.15.76:0/225785260 >> v2:172.21.15.113:3300/0 conn(0x7fdde40b0ae0 0x7fdde415d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:32.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.882+0000 7fddebc90640 1 --2- 172.21.15.76:0/225785260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdde4159c70 0x7fdde40ce370 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:32.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.882+0000 7fddebc90640 1 -- 172.21.15.76:0/225785260 >> 172.21.15.76:0/225785260 conn(0x7fdde40ce950 msgr2=0x7fdde40ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:32.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.882+0000 7fddebc90640 1 -- 172.21.15.76:0/225785260 shutdown_connections 2023-11-09T03:50:32.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.882+0000 7fddebc90640 1 -- 172.21.15.76:0/225785260 wait complete. 2023-11-09T03:50:32.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.883+0000 7fddebc90640 1 Processor -- start 2023-11-09T03:50:32.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.883+0000 7fddebc90640 1 -- start start 2023-11-09T03:50:32.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.884+0000 7fddebc90640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdde406db20 0x7fdde41ef920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:32.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.884+0000 7fddebc90640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdde40b0ae0 0x7fdde4155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:32.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.884+0000 7fddebc90640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdde4159c70 0x7fdde40caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:32.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.884+0000 7fddebc90640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde41efe60 con 0x7fdde4159c70 2023-11-09T03:50:32.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.884+0000 7fddebc90640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde41f5f80 con 0x7fdde406db20 2023-11-09T03:50:32.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.884+0000 7fddebc90640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde41f6260 con 0x7fdde40b0ae0 2023-11-09T03:50:32.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.884+0000 7fddea22d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdde4159c70 0x7fdde40caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:32.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.884+0000 7fddea22d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdde4159c70 0x7fdde40caed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36604/0 (socket says 172.21.15.76:36604) 2023-11-09T03:50:32.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.884+0000 7fddea22d640 1 -- 172.21.15.76:0/1230800460 learned_addr learned my addr 172.21.15.76:0/1230800460 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:32.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.884+0000 7fdde922b640 1 --2- 172.21.15.76:0/1230800460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdde40b0ae0 0x7fdde4155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:32.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.884+0000 7fdde9a2c640 1 --2- 172.21.15.76:0/1230800460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdde406db20 0x7fdde41ef920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:32.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.884+0000 7fddea22d640 1 -- 172.21.15.76:0/1230800460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdde40b0ae0 msgr2=0x7fdde4155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:32.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.884+0000 7fddea22d640 1 --2- 172.21.15.76:0/1230800460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdde40b0ae0 0x7fdde4155cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:32.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.884+0000 7fddea22d640 1 -- 172.21.15.76:0/1230800460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdde406db20 msgr2=0x7fdde41ef920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:32.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.884+0000 7fddea22d640 1 --2- 172.21.15.76:0/1230800460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdde406db20 0x7fdde41ef920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:32.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.884+0000 7fddea22d640 1 -- 172.21.15.76:0/1230800460 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd4009400 con 0x7fdde4159c70 2023-11-09T03:50:32.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.884+0000 7fdde922b640 1 --2- 172.21.15.76:0/1230800460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdde40b0ae0 0x7fdde4155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:50:32.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.885+0000 7fddea22d640 1 --2- 172.21.15.76:0/1230800460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdde4159c70 0x7fdde40caed0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00efc0 tx=0x7fdddc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:32.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.885+0000 7fddda7fc640 1 -- 172.21.15.76:0/1230800460 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdddc009ca0 con 0x7fdde4159c70 2023-11-09T03:50:32.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.885+0000 7fddda7fc640 1 -- 172.21.15.76:0/1230800460 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdddc00f040 con 0x7fdde4159c70 2023-11-09T03:50:32.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.885+0000 7fddebc90640 1 -- 172.21.15.76:0/1230800460 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdde41f6540 con 0x7fdde4159c70 2023-11-09T03:50:32.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.885+0000 7fddda7fc640 1 -- 172.21.15.76:0/1230800460 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdddc0146c0 con 0x7fdde4159c70 2023-11-09T03:50:32.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.885+0000 7fddebc90640 1 -- 172.21.15.76:0/1230800460 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde41f2440 con 0x7fdde4159c70 2023-11-09T03:50:32.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.886+0000 7fddebc90640 1 -- 172.21.15.76:0/1230800460 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdde4074290 con 0x7fdde4159c70 2023-11-09T03:50:32.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.886+0000 7fddda7fc640 1 -- 172.21.15.76:0/1230800460 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdddc014820 con 0x7fdde4159c70 2023-11-09T03:50:32.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.887+0000 7fddda7fc640 1 --2- 172.21.15.76:0/1230800460 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fddc806ae70 0x7fddc806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:32.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.887+0000 7fddda7fc640 1 -- 172.21.15.76:0/1230800460 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7fdddc08abe0 con 0x7fdde4159c70 2023-11-09T03:50:32.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.887+0000 7fdde9a2c640 1 --2- 172.21.15.76:0/1230800460 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fddc806ae70 0x7fddc806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:32.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.887+0000 7fdde9a2c640 1 --2- 172.21.15.76:0/1230800460 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fddc806ae70 0x7fddc806d330 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fddd4036000 tx=0x7fddd403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:32.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:32.891+0000 7fddda7fc640 1 -- 172.21.15.76:0/1230800460 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdddc055620 con 0x7fdde4159c70 2023-11-09T03:50:33.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.094+0000 7fddebc90640 1 -- 172.21.15.76:0/1230800460 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdde406d3f0 con 0x7fdde4159c70 2023-11-09T03:50:33.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.095+0000 7fddda7fc640 1 -- 172.21.15.76:0/1230800460 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4982 (secure 0 0 0) 0x7fdddc018090 con 0x7fdde4159c70 2023-11-09T03:50:33.096 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:50:33.096 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3449910842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3449910842},{"type":"v1","addr":"172.21.15.113:6837","nonce":3449910842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:50:31.438686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7773,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7773":{"gid":7773,"name":"c","rank":0,"incarnation":61,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/1663988060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1663988060},{"type":"v1","addr":"172.21.15.76:6835","nonce":1663988060}]},"join_fscid":-1,"laggy_since":"2023-11-09T03:50:31.438686+0000","export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:50:33.096 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 67 2023-11-09T03:50:33.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.097+0000 7fddebc90640 1 -- 172.21.15.76:0/1230800460 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fddc806ae70 msgr2=0x7fddc806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:33.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.097+0000 7fddebc90640 1 --2- 172.21.15.76:0/1230800460 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fddc806ae70 0x7fddc806d330 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fddd4036000 tx=0x7fddd403c040 comp rx=0 tx=0).stop 2023-11-09T03:50:33.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.097+0000 7fddebc90640 1 -- 172.21.15.76:0/1230800460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdde4159c70 msgr2=0x7fdde40caed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:33.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.097+0000 7fddebc90640 1 --2- 172.21.15.76:0/1230800460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdde4159c70 0x7fdde40caed0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00efc0 tx=0x7fdddc00c490 comp rx=0 tx=0).stop 2023-11-09T03:50:33.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.097+0000 7fddebc90640 1 -- 172.21.15.76:0/1230800460 shutdown_connections 2023-11-09T03:50:33.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.097+0000 7fddebc90640 1 --2- 172.21.15.76:0/1230800460 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fddc806ae70 0x7fddc806d330 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:33.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.097+0000 7fddebc90640 1 --2- 172.21.15.76:0/1230800460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdde4159c70 0x7fdde40caed0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:33.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.097+0000 7fddebc90640 1 --2- 172.21.15.76:0/1230800460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdde40b0ae0 0x7fdde4155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:33.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.097+0000 7fddebc90640 1 --2- 172.21.15.76:0/1230800460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdde406db20 0x7fdde41ef920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:33.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.097+0000 7fddebc90640 1 -- 172.21.15.76:0/1230800460 >> 172.21.15.76:0/1230800460 conn(0x7fdde40ce950 msgr2=0x7fdde40ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:33.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.097+0000 7fddebc90640 1 -- 172.21.15.76:0/1230800460 shutdown_connections 2023-11-09T03:50:33.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.097+0000 7fddebc90640 1 -- 172.21.15.76:0/1230800460 wait complete. 2023-11-09T03:50:33.108 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2023-11-09T03:50:33.109 DEBUG:teuthology.orchestra.run.smithi076:> sudo sysctl -n kernel.core_pattern 2023-11-09T03:50:33.134 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-09T03:50:33.135 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2023-11-09T03:50:33.136 DEBUG:teuthology.orchestra.run.smithi076:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2023-11-09T03:50:33.205 INFO:teuthology.orchestra.run.smithi076.stdout:1699501816.119062.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i c', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2023-11-09T03:50:33.207 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1699501816.119062.core'] 2023-11-09T03:50:33.207 INFO:tasks.cephfs.cephfs_test_case:Found core file 1699501816.119062.core, deleting it 2023-11-09T03:50:33.208 DEBUG:teuthology.orchestra.run.smithi076:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1699501816.119062.core 2023-11-09T03:50:33.276 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:0 2023-11-09T03:50:33.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.415+0000 7fe61fb90640 1 Processor -- start 2023-11-09T03:50:33.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.416+0000 7fe61fb90640 1 -- start start 2023-11-09T03:50:33.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.416+0000 7fe61fb90640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe618153e60 0x7fe61815dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:33.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.416+0000 7fe61fb90640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe6180cd6d0 0x7fe618161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:33.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.416+0000 7fe61fb90640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6180cde60 con 0x7fe6180b4880 2023-11-09T03:50:33.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.416+0000 7fe61fb90640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe618160f90 con 0x7fe6180cd6d0 2023-11-09T03:50:33.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.416+0000 7fe61fb90640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe61815e400 con 0x7fe618153e60 2023-11-09T03:50:33.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.416+0000 7fe61d12b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe6180cd6d0 0x7fe618161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:33.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.416+0000 7fe61d12b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe6180cd6d0 0x7fe618161420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36618/0 (socket says 172.21.15.76:36618) 2023-11-09T03:50:33.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.416+0000 7fe61d12b640 1 -- 172.21.15.76:0/1664866369 learned_addr learned my addr 172.21.15.76:0/1664866369 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:33.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.417+0000 7fe61d92c640 1 --2- 172.21.15.76:0/1664866369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe618153e60 0x7fe61815dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:33.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.417+0000 7fe61d12b640 1 -- 172.21.15.76:0/1664866369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe618153e60 msgr2=0x7fe61815dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:33.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.417+0000 7fe61d12b640 1 --2- 172.21.15.76:0/1664866369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe618153e60 0x7fe61815dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:33.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.417+0000 7fe61d12b640 1 -- 172.21.15.76:0/1664866369 >> v1:172.21.15.113:6789/0 conn(0x7fe6180b4880 legacy=0x7fe61806b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:33.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.417+0000 7fe61d12b640 1 -- 172.21.15.76:0/1664866369 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe618161960 con 0x7fe6180cd6d0 2023-11-09T03:50:33.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.417+0000 7fe61d12b640 1 --2- 172.21.15.76:0/1664866369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6180cd6d0 0x7fe618161420 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fe608009870 tx=0x7fe60802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c767c8be3fbc9df3 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:33.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.418+0000 7fe61c92a640 1 -- 172.21.15.76:0/1664866369 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe608030840 con 0x7fe6180cd6d0 2023-11-09T03:50:33.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.418+0000 7fe61c92a640 1 -- 172.21.15.76:0/1664866369 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6080309a0 con 0x7fe6180cd6d0 2023-11-09T03:50:33.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.418+0000 7fe61c92a640 1 -- 172.21.15.76:0/1664866369 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe608030d30 con 0x7fe6180cd6d0 2023-11-09T03:50:33.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.418+0000 7fe61fb90640 1 -- 172.21.15.76:0/1664866369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6180cd6d0 msgr2=0x7fe618161420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:33.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.418+0000 7fe61fb90640 1 --2- 172.21.15.76:0/1664866369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6180cd6d0 0x7fe618161420 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fe608009870 tx=0x7fe60802b980 comp rx=0 tx=0).stop 2023-11-09T03:50:33.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.418+0000 7fe61fb90640 1 -- 172.21.15.76:0/1664866369 shutdown_connections 2023-11-09T03:50:33.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.418+0000 7fe61fb90640 1 --2- 172.21.15.76:0/1664866369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6180cd6d0 0x7fe618161420 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:33.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.418+0000 7fe61fb90640 1 --2- 172.21.15.76:0/1664866369 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe618153e60 0x7fe61815dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:33.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.418+0000 7fe61fb90640 1 -- 172.21.15.76:0/1664866369 >> 172.21.15.76:0/1664866369 conn(0x7fe6180cf880 msgr2=0x7fe6180cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:33.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.418+0000 7fe61fb90640 1 -- 172.21.15.76:0/1664866369 shutdown_connections 2023-11-09T03:50:33.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.418+0000 7fe61fb90640 1 -- 172.21.15.76:0/1664866369 wait complete. 2023-11-09T03:50:33.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.419+0000 7fe61fb90640 1 Processor -- start 2023-11-09T03:50:33.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.419+0000 7fe61fb90640 1 -- start start 2023-11-09T03:50:33.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.420+0000 7fe61fb90640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6180b4880 0x7fe6180c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:33.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.420+0000 7fe61fb90640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6180cd6d0 0x7fe618155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:33.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.420+0000 7fe61fb90640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe618153e60 0x7fe6180cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:33.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.420+0000 7fe61fb90640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe618157980 con 0x7fe6180cd6d0 2023-11-09T03:50:33.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.420+0000 7fe61fb90640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6181574f0 con 0x7fe6180b4880 2023-11-09T03:50:33.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.420+0000 7fe61fb90640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6180c8d00 con 0x7fe618153e60 2023-11-09T03:50:33.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.420+0000 7fe61d12b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6180cd6d0 0x7fe618155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:33.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.420+0000 7fe61d12b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6180cd6d0 0x7fe618155e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36630/0 (socket says 172.21.15.76:36630) 2023-11-09T03:50:33.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.420+0000 7fe61d12b640 1 -- 172.21.15.76:0/4017260776 learned_addr learned my addr 172.21.15.76:0/4017260776 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:33.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.420+0000 7fe61d92c640 1 --2- 172.21.15.76:0/4017260776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6180b4880 0x7fe6180c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:33.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.420+0000 7fe61e12d640 1 --2- 172.21.15.76:0/4017260776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe618153e60 0x7fe6180cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:33.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.420+0000 7fe61d12b640 1 -- 172.21.15.76:0/4017260776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe618153e60 msgr2=0x7fe6180cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:33.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.420+0000 7fe61d12b640 1 --2- 172.21.15.76:0/4017260776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe618153e60 0x7fe6180cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.420+0000 7fe61d12b640 1 -- 172.21.15.76:0/4017260776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6180b4880 msgr2=0x7fe6180c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.420+0000 7fe61d12b640 1 --2- 172.21.15.76:0/4017260776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6180b4880 0x7fe6180c87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.420+0000 7fe61d12b640 1 -- 172.21.15.76:0/4017260776 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6080094d0 con 0x7fe6180cd6d0 2023-11-09T03:50:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.421+0000 7fe61d92c640 1 --2- 172.21.15.76:0/4017260776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6180b4880 0x7fe6180c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:50:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.421+0000 7fe61e12d640 1 --2- 172.21.15.76:0/4017260776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe618153e60 0x7fe6180cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:50:33.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.421+0000 7fe61d12b640 1 --2- 172.21.15.76:0/4017260776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6180cd6d0 0x7fe618155e90 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fe60802b9c0 tx=0x7fe608004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:33.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.421+0000 7fe60e7fc640 1 -- 172.21.15.76:0/4017260776 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe608039c90 con 0x7fe6180cd6d0 2023-11-09T03:50:33.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.421+0000 7fe60e7fc640 1 -- 172.21.15.76:0/4017260776 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe608039df0 con 0x7fe6180cd6d0 2023-11-09T03:50:33.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.421+0000 7fe61fb90640 1 -- 172.21.15.76:0/4017260776 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6181568e0 con 0x7fe6180cd6d0 2023-11-09T03:50:33.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.421+0000 7fe60e7fc640 1 -- 172.21.15.76:0/4017260776 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6080446c0 con 0x7fe6180cd6d0 2023-11-09T03:50:33.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.421+0000 7fe61fb90640 1 -- 172.21.15.76:0/4017260776 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe618156ea0 con 0x7fe6180cd6d0 2023-11-09T03:50:33.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.422+0000 7fe61fb90640 1 -- 172.21.15.76:0/4017260776 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe618074290 con 0x7fe6180cd6d0 2023-11-09T03:50:33.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.422+0000 7fe60e7fc640 1 -- 172.21.15.76:0/4017260776 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe60804d3b0 con 0x7fe6180cd6d0 2023-11-09T03:50:33.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.423+0000 7fe60e7fc640 1 --2- 172.21.15.76:0/4017260776 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe5fc06ae70 0x7fe5fc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:33.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.423+0000 7fe60e7fc640 1 -- 172.21.15.76:0/4017260776 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7fe60803e070 con 0x7fe6180cd6d0 2023-11-09T03:50:33.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.423+0000 7fe61d92c640 1 --2- 172.21.15.76:0/4017260776 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe5fc06ae70 0x7fe5fc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:33.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.423+0000 7fe61d92c640 1 --2- 172.21.15.76:0/4017260776 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe5fc06ae70 0x7fe5fc06d330 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe614007440 tx=0x7fe61400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:33.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.427+0000 7fe60e7fc640 1 -- 172.21.15.76:0/4017260776 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe608077180 con 0x7fe6180cd6d0 2023-11-09T03:50:33.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.663+0000 7fe61fb90640 1 -- 172.21.15.76:0/4017260776 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:0"} v 0) v1 -- 0x7fe618157630 con 0x7fe6180cd6d0 2023-11-09T03:50:33.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.665+0000 7fe60e7fc640 1 -- 172.21.15.76:0/4017260776 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:0"}]=0 failed mds gid 7773 v68) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe60807a7a0 con 0x7fe6180cd6d0 2023-11-09T03:50:33.665 INFO:teuthology.orchestra.run.smithi076.stderr:failed mds gid 7773 2023-11-09T03:50:33.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.667+0000 7fe61fb90640 1 -- 172.21.15.76:0/4017260776 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe5fc06ae70 msgr2=0x7fe5fc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:33.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.667+0000 7fe61fb90640 1 --2- 172.21.15.76:0/4017260776 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe5fc06ae70 0x7fe5fc06d330 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe614007440 tx=0x7fe61400b040 comp rx=0 tx=0).stop 2023-11-09T03:50:33.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.667+0000 7fe61fb90640 1 -- 172.21.15.76:0/4017260776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6180cd6d0 msgr2=0x7fe618155e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:33.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.667+0000 7fe61fb90640 1 --2- 172.21.15.76:0/4017260776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6180cd6d0 0x7fe618155e90 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fe60802b9c0 tx=0x7fe608004730 comp rx=0 tx=0).stop 2023-11-09T03:50:33.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.667+0000 7fe61fb90640 1 -- 172.21.15.76:0/4017260776 shutdown_connections 2023-11-09T03:50:33.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.667+0000 7fe61fb90640 1 --2- 172.21.15.76:0/4017260776 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe5fc06ae70 0x7fe5fc06d330 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:33.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.667+0000 7fe61fb90640 1 --2- 172.21.15.76:0/4017260776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe618153e60 0x7fe6180cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:33.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.667+0000 7fe61fb90640 1 --2- 172.21.15.76:0/4017260776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6180cd6d0 0x7fe618155e90 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:33.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.667+0000 7fe61fb90640 1 --2- 172.21.15.76:0/4017260776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6180b4880 0x7fe6180c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:33.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.667+0000 7fe61fb90640 1 -- 172.21.15.76:0/4017260776 >> 172.21.15.76:0/4017260776 conn(0x7fe6180cf880 msgr2=0x7fe61815efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:33.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.667+0000 7fe61fb90640 1 -- 172.21.15.76:0/4017260776 shutdown_connections 2023-11-09T03:50:33.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.667+0000 7fe61fb90640 1 -- 172.21.15.76:0/4017260776 wait complete. 2023-11-09T03:50:33.681 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-09T03:50:33.681 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-09T03:50:33.681 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-09T03:50:33.682 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:50:33.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:50:33.682 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi076 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c' 2023-11-09T03:50:33.683 INFO:tasks.ceph.mds.c:Stopped 2023-11-09T03:50:33.683 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-09T03:50:33.686 INFO:tasks.ceph.mds.c:Started 2023-11-09T03:50:33.686 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:50:33.762 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2023-11-09T03:50:33.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.789+0000 7f1f1a00e640 1 Processor -- start 2023-11-09T03:50:33.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.789+0000 7f1f1a00e640 1 -- start start 2023-11-09T03:50:33.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.789+0000 7f1f1a00e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1f140cde60 0x7f1f1415d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:33.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.789+0000 7f1f1a00e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1f1415daa0 0x7f1f14157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:33.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.789+0000 7f1f1a00e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f1415e000 con 0x7f1f140ce240 2023-11-09T03:50:33.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.789+0000 7f1f1a00e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f1415e5c0 con 0x7f1f1415daa0 2023-11-09T03:50:33.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.789+0000 7f1f1a00e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f1415e2e0 con 0x7f1f140cde60 2023-11-09T03:50:33.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.789+0000 7f1f12ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1f1415daa0 0x7f1f14157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:33.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.790+0000 7f1f12ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1f1415daa0 0x7f1f14157000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36650/0 (socket says 172.21.15.76:36650) 2023-11-09T03:50:33.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.790+0000 7f1f12ffd640 1 -- 172.21.15.76:0/248147225 learned_addr learned my addr 172.21.15.76:0/248147225 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:33.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.790+0000 7f1f137fe640 1 --2- 172.21.15.76:0/248147225 >> v2:172.21.15.113:3300/0 conn(0x7f1f140cde60 0x7f1f1415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:33.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.790+0000 7f1f12ffd640 1 -- 172.21.15.76:0/248147225 >> v2:172.21.15.113:3300/0 conn(0x7f1f140cde60 msgr2=0x7f1f1415d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:33.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.790+0000 7f1f12ffd640 1 --2- 172.21.15.76:0/248147225 >> v2:172.21.15.113:3300/0 conn(0x7f1f140cde60 0x7f1f1415d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:33.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.790+0000 7f1f12ffd640 1 -- 172.21.15.76:0/248147225 >> v1:172.21.15.76:6789/0 conn(0x7f1f140ce240 legacy=0x7f1f1406b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:33.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.790+0000 7f1f12ffd640 1 -- 172.21.15.76:0/248147225 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f140d1120 con 0x7f1f1415daa0 2023-11-09T03:50:33.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.790+0000 7f1f127fc640 1 -- 172.21.15.76:0/248147225 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 488576718 0 0) 0x7f1f1415e000 con 0x7f1f140ce240 2023-11-09T03:50:33.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.791+0000 7f1f12ffd640 1 --2- 172.21.15.76:0/248147225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f1415daa0 0x7f1f14157000 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f1f140b5550 tx=0x7f1f0802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602ecb71bfa38e36 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:33.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.791+0000 7f1f127fc640 1 -- 172.21.15.76:0/248147225 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f08030860 con 0x7f1f1415daa0 2023-11-09T03:50:33.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.791+0000 7f1f127fc640 1 -- 172.21.15.76:0/248147225 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f080309c0 con 0x7f1f1415daa0 2023-11-09T03:50:33.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.791+0000 7f1f127fc640 1 -- 172.21.15.76:0/248147225 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f08030d50 con 0x7f1f1415daa0 2023-11-09T03:50:33.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.791+0000 7f1f1a00e640 1 -- 172.21.15.76:0/248147225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f1415daa0 msgr2=0x7f1f14157000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:33.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.791+0000 7f1f1a00e640 1 --2- 172.21.15.76:0/248147225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f1415daa0 0x7f1f14157000 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f1f140b5550 tx=0x7f1f0802b9a0 comp rx=0 tx=0).stop 2023-11-09T03:50:33.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.791+0000 7f1f1a00e640 1 -- 172.21.15.76:0/248147225 shutdown_connections 2023-11-09T03:50:33.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.791+0000 7f1f1a00e640 1 --2- 172.21.15.76:0/248147225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f1415daa0 0x7f1f14157000 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:33.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.791+0000 7f1f1a00e640 1 --2- 172.21.15.76:0/248147225 >> v2:172.21.15.113:3300/0 conn(0x7f1f140cde60 0x7f1f1415d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:33.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.791+0000 7f1f1a00e640 1 -- 172.21.15.76:0/248147225 >> 172.21.15.76:0/248147225 conn(0x7f1f14071f30 msgr2=0x7f1f140cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:33.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.791+0000 7f1f1a00e640 1 -- 172.21.15.76:0/248147225 shutdown_connections 2023-11-09T03:50:33.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.791+0000 7f1f1a00e640 1 -- 172.21.15.76:0/248147225 wait complete. 2023-11-09T03:50:33.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.792+0000 7f1f1a00e640 1 Processor -- start 2023-11-09T03:50:33.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.793+0000 7f1f1a00e640 1 -- start start 2023-11-09T03:50:33.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.793+0000 7f1f1a00e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f140cde60 0x7f1f1414d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:33.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.793+0000 7f1f1a00e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f140ce240 0x7f1f140c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:33.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.793+0000 7f1f1a00e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f1415daa0 0x7f1f140ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:33.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.793+0000 7f1f1a00e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f14153b30 con 0x7f1f140ce240 2023-11-09T03:50:33.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.793+0000 7f1f1a00e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f140caa40 con 0x7f1f1415daa0 2023-11-09T03:50:33.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.793+0000 7f1f1a00e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f140cad20 con 0x7f1f140cde60 2023-11-09T03:50:33.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.793+0000 7f1f12ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f140ce240 0x7f1f140c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:33.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.793+0000 7f1f12ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f140ce240 0x7f1f140c9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36656/0 (socket says 172.21.15.76:36656) 2023-11-09T03:50:33.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.793+0000 7f1f12ffd640 1 -- 172.21.15.76:0/3502787290 learned_addr learned my addr 172.21.15.76:0/3502787290 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:33.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.793+0000 7f1f13fff640 1 --2- 172.21.15.76:0/3502787290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f1415daa0 0x7f1f140ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:33.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.793+0000 7f1f137fe640 1 --2- 172.21.15.76:0/3502787290 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f140cde60 0x7f1f1414d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:33.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.794+0000 7f1f12ffd640 1 -- 172.21.15.76:0/3502787290 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f140cde60 msgr2=0x7f1f1414d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:33.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.794+0000 7f1f12ffd640 1 --2- 172.21.15.76:0/3502787290 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f140cde60 0x7f1f1414d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:33.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.794+0000 7f1f12ffd640 1 -- 172.21.15.76:0/3502787290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f1415daa0 msgr2=0x7f1f140ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:33.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.794+0000 7f1f12ffd640 1 --2- 172.21.15.76:0/3502787290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f1415daa0 0x7f1f140ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:33.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.794+0000 7f1f12ffd640 1 -- 172.21.15.76:0/3502787290 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f080094d0 con 0x7f1f140ce240 2023-11-09T03:50:33.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.794+0000 7f1f137fe640 1 --2- 172.21.15.76:0/3502787290 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f140cde60 0x7f1f1414d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:50:33.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.794+0000 7f1f12ffd640 1 --2- 172.21.15.76:0/3502787290 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f140ce240 0x7f1f140c9ef0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f1f0802b9e0 tx=0x7f1f080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:33.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.794+0000 7f1efbfff640 1 -- 172.21.15.76:0/3502787290 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0803e070 con 0x7f1f140ce240 2023-11-09T03:50:33.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.794+0000 7f1efbfff640 1 -- 172.21.15.76:0/3502787290 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f08004030 con 0x7f1f140ce240 2023-11-09T03:50:33.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.794+0000 7f1f1a00e640 1 -- 172.21.15.76:0/3502787290 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f140cb000 con 0x7f1f140ce240 2023-11-09T03:50:33.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.794+0000 7f1efbfff640 1 -- 172.21.15.76:0/3502787290 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f080043a0 con 0x7f1f140ce240 2023-11-09T03:50:33.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.794+0000 7f1f1a00e640 1 -- 172.21.15.76:0/3502787290 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f140cb2e0 con 0x7f1f140ce240 2023-11-09T03:50:33.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.795+0000 7f1f1a00e640 1 -- 172.21.15.76:0/3502787290 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f14074290 con 0x7f1f140ce240 2023-11-09T03:50:33.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.795+0000 7f1efbfff640 1 -- 172.21.15.76:0/3502787290 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1f08042070 con 0x7f1f140ce240 2023-11-09T03:50:33.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.796+0000 7f1efbfff640 1 --2- 172.21.15.76:0/3502787290 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1f0006ae70 0x7f1f0006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:33.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.796+0000 7f1efbfff640 1 -- 172.21.15.76:0/3502787290 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7f1f080abab0 con 0x7f1f140ce240 2023-11-09T03:50:33.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.796+0000 7f1f137fe640 1 --2- 172.21.15.76:0/3502787290 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1f0006ae70 0x7f1f0006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:33.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.796+0000 7f1f137fe640 1 --2- 172.21.15.76:0/3502787290 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1f0006ae70 0x7f1f0006d330 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1f04006fd0 tx=0x7f1f04008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:33.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:33.800+0000 7f1efbfff640 1 -- 172.21.15.76:0/3502787290 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1f08076530 con 0x7f1f140ce240 2023-11-09T03:50:34.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:34.002+0000 7f1f1a00e640 1 -- 172.21.15.76:0/3502787290 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f140d05d0 con 0x7f1f140ce240 2023-11-09T03:50:34.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:34.002+0000 7f1efbfff640 1 -- 172.21.15.76:0/3502787290 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4143 (secure 0 0 0) 0x7f1f08079b50 con 0x7f1f140ce240 2023-11-09T03:50:34.003 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:50:34.003 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":69,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:50:33.669946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5993,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5993":{"gid":5993,"name":"b","rank":0,"incarnation":69,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/3449910842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3449910842},{"type":"v1","addr":"172.21.15.113:6837","nonce":3449910842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:50:34.004 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 69 2023-11-09T03:50:34.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:34.004+0000 7f1f1a00e640 1 -- 172.21.15.76:0/3502787290 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1f0006ae70 msgr2=0x7f1f0006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:34.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:34.004+0000 7f1f1a00e640 1 --2- 172.21.15.76:0/3502787290 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1f0006ae70 0x7f1f0006d330 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1f04006fd0 tx=0x7f1f04008040 comp rx=0 tx=0).stop 2023-11-09T03:50:34.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:34.004+0000 7f1f1a00e640 1 -- 172.21.15.76:0/3502787290 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f140ce240 msgr2=0x7f1f140c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:34.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:34.004+0000 7f1f1a00e640 1 --2- 172.21.15.76:0/3502787290 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f140ce240 0x7f1f140c9ef0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f1f0802b9e0 tx=0x7f1f080046c0 comp rx=0 tx=0).stop 2023-11-09T03:50:34.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:34.004+0000 7f1f1a00e640 1 -- 172.21.15.76:0/3502787290 shutdown_connections 2023-11-09T03:50:34.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:34.004+0000 7f1f1a00e640 1 --2- 172.21.15.76:0/3502787290 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1f0006ae70 0x7f1f0006d330 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:34.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:34.004+0000 7f1f1a00e640 1 --2- 172.21.15.76:0/3502787290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f1415daa0 0x7f1f140ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:34.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:34.004+0000 7f1f1a00e640 1 --2- 172.21.15.76:0/3502787290 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f140ce240 0x7f1f140c9ef0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:34.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:34.004+0000 7f1f1a00e640 1 --2- 172.21.15.76:0/3502787290 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f140cde60 0x7f1f1414d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:34.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:34.004+0000 7f1f1a00e640 1 -- 172.21.15.76:0/3502787290 >> 172.21.15.76:0/3502787290 conn(0x7f1f14071f30 msgr2=0x7f1f140c90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:34.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:34.004+0000 7f1f1a00e640 1 -- 172.21.15.76:0/3502787290 shutdown_connections 2023-11-09T03:50:34.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:34.004+0000 7f1f1a00e640 1 -- 172.21.15.76:0/3502787290 wait complete. 2023-11-09T03:50:34.015 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-11-09T03:50:38.633 INFO:tasks.ceph.mds.b.smithi113.stderr:2023-11-09T03:50:38.633+0000 7fd09156f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:50:38.633 INFO:tasks.ceph.mds.b.smithi113.stderr:2023-11-09T03:50:38.633+0000 7fd09156f640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-09T03:50:39.016 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:50:39.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.126+0000 7f4a3ea4b640 1 Processor -- start 2023-11-09T03:50:39.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.126+0000 7f4a3ea4b640 1 -- start start 2023-11-09T03:50:39.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.127+0000 7f4a3ea4b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a38159c70 0x7f4a380ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:39.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.127+0000 7f4a3ea4b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4a3806db20 0x7f4a380b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:39.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.127+0000 7f4a3ea4b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4a380b0910 0x7f4a3815d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:39.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.127+0000 7f4a3ea4b640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a380b49b0 con 0x7f4a3806db20 2023-11-09T03:50:39.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.127+0000 7f4a3ea4b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a380b0da0 con 0x7f4a380b0910 2023-11-09T03:50:39.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.127+0000 7f4a3ea4b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a380b4110 con 0x7f4a38159c70 2023-11-09T03:50:39.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.127+0000 7f4a377fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4a3806db20 0x7f4a380b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:39.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.127+0000 7f4a377fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4a3806db20 0x7f4a380b03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36664/0 (socket says 172.21.15.76:36664) 2023-11-09T03:50:39.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.127+0000 7f4a377fe640 1 -- 172.21.15.76:0/1978394917 learned_addr learned my addr 172.21.15.76:0/1978394917 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:39.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.127+0000 7f4a3cfe8640 1 --2- 172.21.15.76:0/1978394917 >> v2:172.21.15.113:3300/0 conn(0x7f4a380b0910 0x7f4a3815d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:39.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.127+0000 7f4a37fff640 1 --2- 172.21.15.76:0/1978394917 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a38159c70 0x7f4a380ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:39.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.127+0000 7f4a377fe640 1 -- 172.21.15.76:0/1978394917 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a38159c70 msgr2=0x7f4a380ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:39.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.127+0000 7f4a377fe640 1 --2- 172.21.15.76:0/1978394917 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a38159c70 0x7f4a380ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:39.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.127+0000 7f4a377fe640 1 -- 172.21.15.76:0/1978394917 >> v2:172.21.15.113:3300/0 conn(0x7f4a380b0910 msgr2=0x7f4a3815d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:39.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.127+0000 7f4a377fe640 1 --2- 172.21.15.76:0/1978394917 >> v2:172.21.15.113:3300/0 conn(0x7f4a380b0910 0x7f4a3815d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:39.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.127+0000 7f4a377fe640 1 -- 172.21.15.76:0/1978394917 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a380b43f0 con 0x7f4a3806db20 2023-11-09T03:50:39.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.127+0000 7f4a3cfe8640 1 --2- 172.21.15.76:0/1978394917 >> v2:172.21.15.113:3300/0 conn(0x7f4a380b0910 0x7f4a3815d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:50:39.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.128+0000 7f4a377fe640 1 --2- 172.21.15.76:0/1978394917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a3806db20 0x7f4a380b03d0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f4a380ca130 tx=0x7f4a2c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826ca9ff4e68f209 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:39.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.128+0000 7f4a36ffd640 1 -- 172.21.15.76:0/1978394917 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c031870 con 0x7f4a3806db20 2023-11-09T03:50:39.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.128+0000 7f4a36ffd640 1 -- 172.21.15.76:0/1978394917 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a2c0319d0 con 0x7f4a3806db20 2023-11-09T03:50:39.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.128+0000 7f4a36ffd640 1 -- 172.21.15.76:0/1978394917 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c031d60 con 0x7f4a3806db20 2023-11-09T03:50:39.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.129+0000 7f4a3ea4b640 1 -- 172.21.15.76:0/1978394917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a3806db20 msgr2=0x7f4a380b03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:39.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.129+0000 7f4a3ea4b640 1 --2- 172.21.15.76:0/1978394917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a3806db20 0x7f4a380b03d0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f4a380ca130 tx=0x7f4a2c02b910 comp rx=0 tx=0).stop 2023-11-09T03:50:39.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.129+0000 7f4a3ea4b640 1 -- 172.21.15.76:0/1978394917 shutdown_connections 2023-11-09T03:50:39.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.129+0000 7f4a3ea4b640 1 --2- 172.21.15.76:0/1978394917 >> v2:172.21.15.113:3300/0 conn(0x7f4a380b0910 0x7f4a3815d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:39.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.129+0000 7f4a3ea4b640 1 --2- 172.21.15.76:0/1978394917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a3806db20 0x7f4a380b03d0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:39.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.129+0000 7f4a3ea4b640 1 --2- 172.21.15.76:0/1978394917 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a38159c70 0x7f4a380ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:39.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.129+0000 7f4a3ea4b640 1 -- 172.21.15.76:0/1978394917 >> 172.21.15.76:0/1978394917 conn(0x7f4a380ce950 msgr2=0x7f4a380ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:39.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.129+0000 7f4a3ea4b640 1 -- 172.21.15.76:0/1978394917 shutdown_connections 2023-11-09T03:50:39.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.129+0000 7f4a3ea4b640 1 -- 172.21.15.76:0/1978394917 wait complete. 2023-11-09T03:50:39.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.130+0000 7f4a3ea4b640 1 Processor -- start 2023-11-09T03:50:39.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.130+0000 7f4a3ea4b640 1 -- start start 2023-11-09T03:50:39.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.131+0000 7f4a3ea4b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a3806db20 0x7f4a38153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:39.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.131+0000 7f4a3ea4b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a380b0910 0x7f4a380c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:39.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.131+0000 7f4a3ea4b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a38159c70 0x7f4a380706f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:39.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.131+0000 7f4a3ea4b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a380b1af0 con 0x7f4a380b0910 2023-11-09T03:50:39.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.131+0000 7f4a3ea4b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a38154c00 con 0x7f4a38159c70 2023-11-09T03:50:39.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.131+0000 7f4a3ea4b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a38153590 con 0x7f4a3806db20 2023-11-09T03:50:39.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.131+0000 7f4a377fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a380b0910 0x7f4a380c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:39.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.131+0000 7f4a377fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a380b0910 0x7f4a380c82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36676/0 (socket says 172.21.15.76:36676) 2023-11-09T03:50:39.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.131+0000 7f4a377fe640 1 -- 172.21.15.76:0/809907304 learned_addr learned my addr 172.21.15.76:0/809907304 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:39.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.131+0000 7f4a3cfe8640 1 --2- 172.21.15.76:0/809907304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a38159c70 0x7f4a380706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:39.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.131+0000 7f4a37fff640 1 --2- 172.21.15.76:0/809907304 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a3806db20 0x7f4a38153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:39.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.131+0000 7f4a377fe640 1 -- 172.21.15.76:0/809907304 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a3806db20 msgr2=0x7f4a38153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:39.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.131+0000 7f4a377fe640 1 --2- 172.21.15.76:0/809907304 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a3806db20 0x7f4a38153050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:39.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.132+0000 7f4a377fe640 1 -- 172.21.15.76:0/809907304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a38159c70 msgr2=0x7f4a380706f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:39.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.132+0000 7f4a377fe640 1 --2- 172.21.15.76:0/809907304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a38159c70 0x7f4a380706f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:39.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.132+0000 7f4a377fe640 1 -- 172.21.15.76:0/809907304 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a2c0094d0 con 0x7f4a380b0910 2023-11-09T03:50:39.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.132+0000 7f4a37fff640 1 --2- 172.21.15.76:0/809907304 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a3806db20 0x7f4a38153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:50:39.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.132+0000 7f4a3cfe8640 1 --2- 172.21.15.76:0/809907304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a38159c70 0x7f4a380706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:50:39.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.132+0000 7f4a377fe640 1 --2- 172.21.15.76:0/809907304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a380b0910 0x7f4a380c82a0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c000c00 tx=0x7f4a2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:39.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.132+0000 7f4a34ff9640 1 -- 172.21.15.76:0/809907304 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c03abe0 con 0x7f4a380b0910 2023-11-09T03:50:39.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.132+0000 7f4a34ff9640 1 -- 172.21.15.76:0/809907304 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a2c03ad40 con 0x7f4a380b0910 2023-11-09T03:50:39.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.132+0000 7f4a3ea4b640 1 -- 172.21.15.76:0/809907304 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a38154640 con 0x7f4a380b0910 2023-11-09T03:50:39.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.132+0000 7f4a34ff9640 1 -- 172.21.15.76:0/809907304 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c044610 con 0x7f4a380b0910 2023-11-09T03:50:39.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.132+0000 7f4a3ea4b640 1 -- 172.21.15.76:0/809907304 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a380b1710 con 0x7f4a380b0910 2023-11-09T03:50:39.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.133+0000 7f4a3ea4b640 1 -- 172.21.15.76:0/809907304 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a380b1570 con 0x7f4a380b0910 2023-11-09T03:50:39.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.133+0000 7f4a34ff9640 1 -- 172.21.15.76:0/809907304 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4a2c04d450 con 0x7f4a380b0910 2023-11-09T03:50:39.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.134+0000 7f4a34ff9640 1 --2- 172.21.15.76:0/809907304 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4a2406ae70 0x7f4a2406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:39.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.134+0000 7f4a34ff9640 1 -- 172.21.15.76:0/809907304 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7f4a2c02d040 con 0x7f4a380b0910 2023-11-09T03:50:39.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.134+0000 7f4a37fff640 1 --2- 172.21.15.76:0/809907304 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4a2406ae70 0x7f4a2406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:39.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.134+0000 7f4a37fff640 1 --2- 172.21.15.76:0/809907304 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4a2406ae70 0x7f4a2406d330 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4a28006fd0 tx=0x7f4a28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:39.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.138+0000 7f4a34ff9640 1 -- 172.21.15.76:0/809907304 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4a380b1570 con 0x7f4a380b0910 2023-11-09T03:50:39.246 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2023-11-09T03:50:39.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.337+0000 7f4a3ea4b640 1 -- 172.21.15.76:0/809907304 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a380b1c30 con 0x7f4a380b0910 2023-11-09T03:50:39.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.338+0000 7f4a34ff9640 1 -- 172.21.15.76:0/809907304 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4935 (secure 0 0 0) 0x7f4a380b1c30 con 0x7f4a380b0910 2023-11-09T03:50:39.338 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:50:39.338 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8016,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3229239133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3229239133},{"type":"v1","addr":"172.21.15.76:6835","nonce":3229239133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:50:36.685855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5993,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5993":{"gid":5993,"name":"b","rank":0,"incarnation":69,"state":"up:active","state_seq":11,"addr":"172.21.15.113:6837/3449910842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3449910842},{"type":"v1","addr":"172.21.15.113:6837","nonce":3449910842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:50:39.338 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 73 2023-11-09T03:50:39.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.339+0000 7f4a3ea4b640 1 -- 172.21.15.76:0/809907304 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4a2406ae70 msgr2=0x7f4a2406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:39.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.339+0000 7f4a3ea4b640 1 --2- 172.21.15.76:0/809907304 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4a2406ae70 0x7f4a2406d330 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4a28006fd0 tx=0x7f4a28008040 comp rx=0 tx=0).stop 2023-11-09T03:50:39.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.339+0000 7f4a3ea4b640 1 -- 172.21.15.76:0/809907304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a380b0910 msgr2=0x7f4a380c82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:39.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.339+0000 7f4a3ea4b640 1 --2- 172.21.15.76:0/809907304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a380b0910 0x7f4a380c82a0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c000c00 tx=0x7f4a2c0046c0 comp rx=0 tx=0).stop 2023-11-09T03:50:39.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.339+0000 7f4a3ea4b640 1 -- 172.21.15.76:0/809907304 shutdown_connections 2023-11-09T03:50:39.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.339+0000 7f4a3ea4b640 1 --2- 172.21.15.76:0/809907304 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4a2406ae70 0x7f4a2406d330 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:39.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.339+0000 7f4a3ea4b640 1 --2- 172.21.15.76:0/809907304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a38159c70 0x7f4a380706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:39.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.339+0000 7f4a3ea4b640 1 --2- 172.21.15.76:0/809907304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a380b0910 0x7f4a380c82a0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:39.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.339+0000 7f4a3ea4b640 1 --2- 172.21.15.76:0/809907304 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a3806db20 0x7f4a38153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:39.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.339+0000 7f4a3ea4b640 1 -- 172.21.15.76:0/809907304 >> 172.21.15.76:0/809907304 conn(0x7f4a380ce950 msgr2=0x7f4a380b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:39.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.340+0000 7f4a3ea4b640 1 -- 172.21.15.76:0/809907304 shutdown_connections 2023-11-09T03:50:39.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.340+0000 7f4a3ea4b640 1 -- 172.21.15.76:0/809907304 wait complete. 2023-11-09T03:50:39.350 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-09T03:50:39.351 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:50:39.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.451+0000 7ff08d80e640 1 Processor -- start 2023-11-09T03:50:39.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.451+0000 7ff08d80e640 1 -- start start 2023-11-09T03:50:39.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.452+0000 7ff08d80e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff0880cde60 0x7ff08815d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:39.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.452+0000 7ff08d80e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0880ce240 0x7ff0880d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:39.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.452+0000 7ff08d80e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff08815e000 con 0x7ff08815daa0 2023-11-09T03:50:39.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.452+0000 7ff08d80e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff08815e5c0 con 0x7ff0880cde60 2023-11-09T03:50:39.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.452+0000 7ff08d80e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff08815e2e0 con 0x7ff0880ce240 2023-11-09T03:50:39.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.452+0000 7ff0877fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0880ce240 0x7ff0880d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:39.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.452+0000 7ff086ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff0880cde60 0x7ff08815d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:39.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.452+0000 7ff0867fc640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff08815daa0 0x7ff088157000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:44876/0 (socket says 172.21.15.76:44876) 2023-11-09T03:50:39.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.452+0000 7ff0867fc640 1 -- 172.21.15.76:0/53451991 learned_addr learned my addr 172.21.15.76:0/53451991 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:50:39.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.452+0000 7ff086ffd640 1 -- 172.21.15.76:0/53451991 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0880ce240 msgr2=0x7ff0880d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:39.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.452+0000 7ff086ffd640 1 --2- 172.21.15.76:0/53451991 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0880ce240 0x7ff0880d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:39.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.452+0000 7ff086ffd640 1 -- 172.21.15.76:0/53451991 >> v1:172.21.15.113:6789/0 conn(0x7ff08815daa0 legacy=0x7ff088157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:39.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.452+0000 7ff086ffd640 1 -- 172.21.15.76:0/53451991 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0880d1120 con 0x7ff0880cde60 2023-11-09T03:50:39.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.452+0000 7ff0877fe640 1 --2- 172.21.15.76:0/53451991 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0880ce240 0x7ff0880d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:50:39.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.453+0000 7ff086ffd640 1 --2- 172.21.15.76:0/53451991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0880cde60 0x7ff08815d560 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff0780097a0 tx=0x7ff07802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3f0295d489a47d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:39.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.453+0000 7ff085ffb640 1 -- 172.21.15.76:0/53451991 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff078031830 con 0x7ff0880cde60 2023-11-09T03:50:39.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.453+0000 7ff085ffb640 1 -- 172.21.15.76:0/53451991 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff078031990 con 0x7ff0880cde60 2023-11-09T03:50:39.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.453+0000 7ff085ffb640 1 -- 172.21.15.76:0/53451991 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff078031d20 con 0x7ff0880cde60 2023-11-09T03:50:39.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.454+0000 7ff08d80e640 1 -- 172.21.15.76:0/53451991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0880cde60 msgr2=0x7ff08815d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:39.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.454+0000 7ff08d80e640 1 --2- 172.21.15.76:0/53451991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0880cde60 0x7ff08815d560 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff0780097a0 tx=0x7ff07802b8d0 comp rx=0 tx=0).stop 2023-11-09T03:50:39.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.454+0000 7ff08d80e640 1 -- 172.21.15.76:0/53451991 shutdown_connections 2023-11-09T03:50:39.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.454+0000 7ff08d80e640 1 --2- 172.21.15.76:0/53451991 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0880ce240 0x7ff0880d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:39.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.454+0000 7ff08d80e640 1 --2- 172.21.15.76:0/53451991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0880cde60 0x7ff08815d560 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:39.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.454+0000 7ff08d80e640 1 -- 172.21.15.76:0/53451991 >> 172.21.15.76:0/53451991 conn(0x7ff088071f30 msgr2=0x7ff0880cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:39.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.454+0000 7ff08d80e640 1 -- 172.21.15.76:0/53451991 shutdown_connections 2023-11-09T03:50:39.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.454+0000 7ff08d80e640 1 -- 172.21.15.76:0/53451991 wait complete. 2023-11-09T03:50:39.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.455+0000 7ff08d80e640 1 Processor -- start 2023-11-09T03:50:39.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.455+0000 7ff08d80e640 1 -- start start 2023-11-09T03:50:39.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.456+0000 7ff08d80e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0880cde60 0x7ff0880cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:39.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.456+0000 7ff08d80e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0880ce240 0x7ff0881f3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:39.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.456+0000 7ff08d80e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08815daa0 0x7ff088158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:39.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.456+0000 7ff08d80e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0880cadc0 con 0x7ff0880cde60 2023-11-09T03:50:39.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.456+0000 7ff08d80e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0880c9ef0 con 0x7ff0880ce240 2023-11-09T03:50:39.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.456+0000 7ff08d80e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0880ca140 con 0x7ff08815daa0 2023-11-09T03:50:39.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.456+0000 7ff086ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0880cde60 0x7ff0880cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:39.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.456+0000 7ff086ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0880cde60 0x7ff0880cc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36690/0 (socket says 172.21.15.76:36690) 2023-11-09T03:50:39.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.456+0000 7ff086ffd640 1 -- 172.21.15.76:0/650787425 learned_addr learned my addr 172.21.15.76:0/650787425 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:39.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.456+0000 7ff0877fe640 1 --2- 172.21.15.76:0/650787425 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08815daa0 0x7ff088158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:39.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.456+0000 7ff0867fc640 1 --2- 172.21.15.76:0/650787425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0880ce240 0x7ff0881f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:39.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.456+0000 7ff086ffd640 1 -- 172.21.15.76:0/650787425 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08815daa0 msgr2=0x7ff088158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:39.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.456+0000 7ff086ffd640 1 --2- 172.21.15.76:0/650787425 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08815daa0 0x7ff088158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:39.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.456+0000 7ff086ffd640 1 -- 172.21.15.76:0/650787425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0880ce240 msgr2=0x7ff0881f3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:39.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.456+0000 7ff086ffd640 1 --2- 172.21.15.76:0/650787425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0880ce240 0x7ff0881f3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:39.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.456+0000 7ff086ffd640 1 -- 172.21.15.76:0/650787425 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff080009a70 con 0x7ff0880cde60 2023-11-09T03:50:39.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.456+0000 7ff0877fe640 1 --2- 172.21.15.76:0/650787425 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08815daa0 0x7ff088158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:50:39.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.456+0000 7ff086ffd640 1 --2- 172.21.15.76:0/650787425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0880cde60 0x7ff0880cc7d0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7ff07802bee0 tx=0x7ff0780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:39.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.457+0000 7ff06ffff640 1 -- 172.21.15.76:0/650787425 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07803a6f0 con 0x7ff0880cde60 2023-11-09T03:50:39.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.457+0000 7ff06ffff640 1 -- 172.21.15.76:0/650787425 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07803a870 con 0x7ff0880cde60 2023-11-09T03:50:39.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.457+0000 7ff08d80e640 1 -- 172.21.15.76:0/650787425 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff078009400 con 0x7ff0880cde60 2023-11-09T03:50:39.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.457+0000 7ff06ffff640 1 -- 172.21.15.76:0/650787425 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07803ac20 con 0x7ff0880cde60 2023-11-09T03:50:39.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.457+0000 7ff08d80e640 1 -- 172.21.15.76:0/650787425 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0880ca700 con 0x7ff0880cde60 2023-11-09T03:50:39.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.458+0000 7ff08d80e640 1 -- 172.21.15.76:0/650787425 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0880cda10 con 0x7ff0880cde60 2023-11-09T03:50:39.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.458+0000 7ff06ffff640 1 -- 172.21.15.76:0/650787425 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff07804c3b0 con 0x7ff0880cde60 2023-11-09T03:50:39.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.458+0000 7ff06ffff640 1 --2- 172.21.15.76:0/650787425 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff07406aea0 0x7ff07406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:39.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.458+0000 7ff06ffff640 1 -- 172.21.15.76:0/650787425 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7ff0780ab8c0 con 0x7ff0880cde60 2023-11-09T03:50:39.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.458+0000 7ff0867fc640 1 --2- 172.21.15.76:0/650787425 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff07406aea0 0x7ff07406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:39.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.459+0000 7ff0867fc640 1 --2- 172.21.15.76:0/650787425 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff07406aea0 0x7ff07406d360 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff0880c96a0 tx=0x7ff07c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:39.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.462+0000 7ff06ffff640 1 -- 172.21.15.76:0/650787425 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff0780762b0 con 0x7ff0880cde60 2023-11-09T03:50:39.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.663+0000 7ff08d80e640 1 -- 172.21.15.76:0/650787425 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff088074290 con 0x7ff0880cde60 2023-11-09T03:50:39.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.664+0000 7ff06ffff640 1 -- 172.21.15.76:0/650787425 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4935 (secure 0 0 0) 0x7ff0780798d0 con 0x7ff0880cde60 2023-11-09T03:50:39.664 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:50:39.664 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8016,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3229239133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3229239133},{"type":"v1","addr":"172.21.15.76:6835","nonce":3229239133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:50:36.685855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5993,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5993":{"gid":5993,"name":"b","rank":0,"incarnation":69,"state":"up:active","state_seq":11,"addr":"172.21.15.113:6837/3449910842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3449910842},{"type":"v1","addr":"172.21.15.113:6837","nonce":3449910842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:50:39.665 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 73 2023-11-09T03:50:39.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.666+0000 7ff08d80e640 1 -- 172.21.15.76:0/650787425 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff07406aea0 msgr2=0x7ff07406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:39.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.666+0000 7ff08d80e640 1 --2- 172.21.15.76:0/650787425 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff07406aea0 0x7ff07406d360 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff0880c96a0 tx=0x7ff07c009040 comp rx=0 tx=0).stop 2023-11-09T03:50:39.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.666+0000 7ff08d80e640 1 -- 172.21.15.76:0/650787425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0880cde60 msgr2=0x7ff0880cc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:39.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.666+0000 7ff08d80e640 1 --2- 172.21.15.76:0/650787425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0880cde60 0x7ff0880cc7d0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7ff07802bee0 tx=0x7ff0780046c0 comp rx=0 tx=0).stop 2023-11-09T03:50:39.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.666+0000 7ff08d80e640 1 -- 172.21.15.76:0/650787425 shutdown_connections 2023-11-09T03:50:39.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.666+0000 7ff08d80e640 1 --2- 172.21.15.76:0/650787425 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff07406aea0 0x7ff07406d360 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:39.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.666+0000 7ff08d80e640 1 --2- 172.21.15.76:0/650787425 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08815daa0 0x7ff088158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:39.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.666+0000 7ff08d80e640 1 --2- 172.21.15.76:0/650787425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0880ce240 0x7ff0881f3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:39.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.666+0000 7ff08d80e640 1 --2- 172.21.15.76:0/650787425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0880cde60 0x7ff0880cc7d0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:39.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.666+0000 7ff08d80e640 1 -- 172.21.15.76:0/650787425 >> 172.21.15.76:0/650787425 conn(0x7ff088071f30 msgr2=0x7ff0880b78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:39.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.666+0000 7ff08d80e640 1 -- 172.21.15.76:0/650787425 shutdown_connections 2023-11-09T03:50:39.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:39.666+0000 7ff08d80e640 1 -- 172.21.15.76:0/650787425 wait complete. 2023-11-09T03:50:40.079 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-09T03:48:57.383848+0000', 'modified': '2023-11-09T03:50:36.685855+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 100, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 5993, 'mds_1': 7449}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5993': {'gid': 5993, 'name': 'b', 'rank': 0, 'incarnation': 69, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.113:6837/3449910842', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6836', 'nonce': 3449910842}, {'type': 'v1', 'addr': '172.21.15.113:6837', 'nonce': 3449910842}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_7449': {'gid': 7449, 'name': 'a', 'rank': 1, 'incarnation': 47, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.76:6833/3697850101', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6832', 'nonce': 3697850101}, {'type': 'v1', 'addr': '172.21.15.76:6833', 'nonce': 3697850101}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-09T03:50:40.079 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2023-11-09T03:50:40.080 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-09T03:50:40.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.183+0000 7f28cd5fb640 1 Processor -- start 2023-11-09T03:50:40.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.183+0000 7f28cd5fb640 1 -- start start 2023-11-09T03:50:40.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.184+0000 7f28cd5fb640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f28c8153e60 0x7f28c815dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.184+0000 7f28cd5fb640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f28c80b4880 0x7f28c80b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.184+0000 7f28cd5fb640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c80cde60 con 0x7f28c80cd6d0 2023-11-09T03:50:40.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.184+0000 7f28cd5fb640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c8160f90 con 0x7f28c8153e60 2023-11-09T03:50:40.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.184+0000 7f28cd5fb640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c80b5200 con 0x7f28c80b4880 2023-11-09T03:50:40.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.184+0000 7f28c6ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f28c8153e60 0x7f28c815dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.184+0000 7f28c6ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f28c8153e60 0x7f28c815dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36700/0 (socket says 172.21.15.76:36700) 2023-11-09T03:50:40.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.184+0000 7f28c6ffd640 1 -- 172.21.15.76:0/1447068443 learned_addr learned my addr 172.21.15.76:0/1447068443 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:40.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.184+0000 7f28c6ffd640 1 -- 172.21.15.76:0/1447068443 >> v2:172.21.15.113:3300/0 conn(0x7f28c80b4880 msgr2=0x7f28c80b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:40.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.184+0000 7f28c77fe640 1 --2- 172.21.15.76:0/1447068443 >> v2:172.21.15.113:3300/0 conn(0x7f28c80b4880 0x7f28c80b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.184+0000 7f28c6ffd640 1 --2- 172.21.15.76:0/1447068443 >> v2:172.21.15.113:3300/0 conn(0x7f28c80b4880 0x7f28c80b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.184+0000 7f28c6ffd640 1 -- 172.21.15.76:0/1447068443 >> v1:172.21.15.113:6789/0 conn(0x7f28c80cd6d0 legacy=0x7f28c8161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:40.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.184+0000 7f28c6ffd640 1 -- 172.21.15.76:0/1447068443 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c815e400 con 0x7f28c8153e60 2023-11-09T03:50:40.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.184+0000 7f28c77fe640 1 --2- 172.21.15.76:0/1447068443 >> v2:172.21.15.113:3300/0 conn(0x7f28c80b4880 0x7f28c80b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:50:40.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.185+0000 7f28c6ffd640 1 --2- 172.21.15.76:0/1447068443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28c8153e60 0x7f28c815dec0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f28c80b8f40 tx=0x7f28b802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a6a6dd5d9364b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:40.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.185+0000 7f28c5ffb640 1 -- 172.21.15.76:0/1447068443 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b8031870 con 0x7f28c8153e60 2023-11-09T03:50:40.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.185+0000 7f28c5ffb640 1 -- 172.21.15.76:0/1447068443 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b80319d0 con 0x7f28c8153e60 2023-11-09T03:50:40.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.185+0000 7f28c5ffb640 1 -- 172.21.15.76:0/1447068443 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b8031d60 con 0x7f28c8153e60 2023-11-09T03:50:40.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.185+0000 7f28cd5fb640 1 -- 172.21.15.76:0/1447068443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28c8153e60 msgr2=0x7f28c815dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.185+0000 7f28cd5fb640 1 --2- 172.21.15.76:0/1447068443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28c8153e60 0x7f28c815dec0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f28c80b8f40 tx=0x7f28b802b910 comp rx=0 tx=0).stop 2023-11-09T03:50:40.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.185+0000 7f28cd5fb640 1 -- 172.21.15.76:0/1447068443 shutdown_connections 2023-11-09T03:50:40.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.185+0000 7f28cd5fb640 1 --2- 172.21.15.76:0/1447068443 >> v2:172.21.15.113:3300/0 conn(0x7f28c80b4880 0x7f28c80b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.185+0000 7f28cd5fb640 1 --2- 172.21.15.76:0/1447068443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28c8153e60 0x7f28c815dec0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.185+0000 7f28cd5fb640 1 -- 172.21.15.76:0/1447068443 >> 172.21.15.76:0/1447068443 conn(0x7f28c80cf880 msgr2=0x7f28c80cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:40.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.185+0000 7f28cd5fb640 1 -- 172.21.15.76:0/1447068443 shutdown_connections 2023-11-09T03:50:40.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.185+0000 7f28cd5fb640 1 -- 172.21.15.76:0/1447068443 wait complete. 2023-11-09T03:50:40.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.187+0000 7f28cd5fb640 1 Processor -- start 2023-11-09T03:50:40.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.187+0000 7f28cd5fb640 1 -- start start 2023-11-09T03:50:40.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.187+0000 7f28cd5fb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f28c80b4880 0x7f28c8150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.187+0000 7f28cd5fb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28c80cd6d0 0x7f28c80c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.187+0000 7f28cd5fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28c8153e60 0x7f28c80ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.187+0000 7f28cd5fb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c8150560 con 0x7f28c8153e60 2023-11-09T03:50:40.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.187+0000 7f28cd5fb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c8150a70 con 0x7f28c80cd6d0 2023-11-09T03:50:40.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.187+0000 7f28cd5fb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c80caab0 con 0x7f28c80b4880 2023-11-09T03:50:40.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.188+0000 7f28c77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28c8153e60 0x7f28c80ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.188+0000 7f28c77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28c8153e60 0x7f28c80ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36704/0 (socket says 172.21.15.76:36704) 2023-11-09T03:50:40.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.188+0000 7f28c77fe640 1 -- 172.21.15.76:0/3816508464 learned_addr learned my addr 172.21.15.76:0/3816508464 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:40.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.188+0000 7f28c67fc640 1 --2- 172.21.15.76:0/3816508464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28c80cd6d0 0x7f28c80c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.188+0000 7f28c6ffd640 1 --2- 172.21.15.76:0/3816508464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f28c80b4880 0x7f28c8150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.188+0000 7f28c77fe640 1 -- 172.21.15.76:0/3816508464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f28c80b4880 msgr2=0x7f28c8150020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.188+0000 7f28c77fe640 1 --2- 172.21.15.76:0/3816508464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f28c80b4880 0x7f28c8150020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.188+0000 7f28c77fe640 1 -- 172.21.15.76:0/3816508464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28c80cd6d0 msgr2=0x7f28c80c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.188+0000 7f28c77fe640 1 --2- 172.21.15.76:0/3816508464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28c80cd6d0 0x7f28c80c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.188+0000 7f28c77fe640 1 -- 172.21.15.76:0/3816508464 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b80094d0 con 0x7f28c8153e60 2023-11-09T03:50:40.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.188+0000 7f28c77fe640 1 --2- 172.21.15.76:0/3816508464 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28c8153e60 0x7f28c80ca410 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f28c000cef0 tx=0x7f28c000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:40.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.188+0000 7f28af7fe640 1 -- 172.21.15.76:0/3816508464 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c000d840 con 0x7f28c8153e60 2023-11-09T03:50:40.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.188+0000 7f28af7fe640 1 -- 172.21.15.76:0/3816508464 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c000d9c0 con 0x7f28c8153e60 2023-11-09T03:50:40.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.188+0000 7f28cd5fb640 1 -- 172.21.15.76:0/3816508464 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c80cad90 con 0x7f28c8153e60 2023-11-09T03:50:40.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.188+0000 7f28af7fe640 1 -- 172.21.15.76:0/3816508464 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c000dd70 con 0x7f28c8153e60 2023-11-09T03:50:40.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.188+0000 7f28cd5fb640 1 -- 172.21.15.76:0/3816508464 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c80cb350 con 0x7f28c8153e60 2023-11-09T03:50:40.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.190+0000 7f28af7fe640 1 -- 172.21.15.76:0/3816508464 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f28c001e3b0 con 0x7f28c8153e60 2023-11-09T03:50:40.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.190+0000 7f28af7fe640 1 --2- 172.21.15.76:0/3816508464 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f28b406ae70 0x7f28b406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.190+0000 7f28af7fe640 1 -- 172.21.15.76:0/3816508464 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7f28c008a7c0 con 0x7f28c8153e60 2023-11-09T03:50:40.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.190+0000 7f28c6ffd640 1 --2- 172.21.15.76:0/3816508464 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f28b406ae70 0x7f28b406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.191+0000 7f28c6ffd640 1 --2- 172.21.15.76:0/3816508464 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f28b406ae70 0x7f28b406d330 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f28b8036000 tx=0x7f28b803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:40.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.191+0000 7f28d018c740 1 Processor -- start 2023-11-09T03:50:40.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.191+0000 7f28d018c740 1 -- start start 2023-11-09T03:50:40.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.192+0000 7f28d018c740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e46e188850 0x55e46e15b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.192+0000 7f28d018c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e46e188c30 0x55e46e18b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.192+0000 7f28d018c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e46e15bc90 0x55e46e18bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.192+0000 7f28d018c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46e06c740 con 0x55e46e15bc90 2023-11-09T03:50:40.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.192+0000 7f28d018c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46e0ce8d0 con 0x55e46e188c30 2023-11-09T03:50:40.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.192+0000 7f28d018c740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e46e0cbfc0 con 0x55e46e188850 2023-11-09T03:50:40.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.192+0000 7f28c77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e46e15bc90 0x55e46e18bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.192+0000 7f28c77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e46e15bc90 0x55e46e18bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36706/0 (socket says 172.21.15.76:36706) 2023-11-09T03:50:40.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.192+0000 7f28c77fe640 1 -- 172.21.15.76:0/537463521 learned_addr learned my addr 172.21.15.76:0/537463521 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:40.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.192+0000 7f28c6ffd640 1 --2- 172.21.15.76:0/537463521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e46e188850 0x55e46e15b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.192+0000 7f28c67fc640 1 --2- 172.21.15.76:0/537463521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e46e188c30 0x55e46e18b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.192+0000 7f28c77fe640 1 -- 172.21.15.76:0/537463521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e46e188850 msgr2=0x55e46e15b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:40.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.192+0000 7f28c77fe640 1 --2- 172.21.15.76:0/537463521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e46e188850 0x55e46e15b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.192+0000 7f28c77fe640 1 -- 172.21.15.76:0/537463521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e46e188c30 msgr2=0x55e46e18b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:40.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.192+0000 7f28c77fe640 1 --2- 172.21.15.76:0/537463521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e46e188c30 0x55e46e18b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.192+0000 7f28c77fe640 1 -- 172.21.15.76:0/537463521 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e46e08aca0 con 0x55e46e15bc90 2023-11-09T03:50:40.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.193+0000 7f28c77fe640 1 --2- 172.21.15.76:0/537463521 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e46e15bc90 0x55e46e18bb20 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f28c0055420 tx=0x7f28c00596e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4572ce5fd3add server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:40.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.193+0000 7f28acff9640 1 -- 172.21.15.76:0/537463521 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0061220 con 0x55e46e15bc90 2023-11-09T03:50:40.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.193+0000 7f28acff9640 1 -- 172.21.15.76:0/537463521 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c00572c0 con 0x55e46e15bc90 2023-11-09T03:50:40.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.193+0000 7f28acff9640 1 -- 172.21.15.76:0/537463521 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0055630 con 0x55e46e15bc90 2023-11-09T03:50:40.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.193+0000 7f28d018c740 1 -- 172.21.15.76:0/537463521 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e46e15bc90 msgr2=0x55e46e18bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.193+0000 7f28d018c740 1 --2- 172.21.15.76:0/537463521 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e46e15bc90 0x55e46e18bb20 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f28c0055420 tx=0x7f28c00596e0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.193+0000 7f28d018c740 1 -- 172.21.15.76:0/537463521 shutdown_connections 2023-11-09T03:50:40.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.193+0000 7f28d018c740 1 --2- 172.21.15.76:0/537463521 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e46e15bc90 0x55e46e18bb20 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.193+0000 7f28d018c740 1 --2- 172.21.15.76:0/537463521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e46e188c30 0x55e46e18b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.193+0000 7f28d018c740 1 --2- 172.21.15.76:0/537463521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e46e188850 0x55e46e15b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.193+0000 7f28d018c740 1 -- 172.21.15.76:0/537463521 >> 172.21.15.76:0/537463521 conn(0x55e46e15da60 msgr2=0x55e46e1a65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:40.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.193+0000 7f28d018c740 1 -- 172.21.15.76:0/537463521 shutdown_connections 2023-11-09T03:50:40.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.194+0000 7f28d018c740 1 -- 172.21.15.76:0/537463521 wait complete. 2023-11-09T03:50:40.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.194+0000 7f28d018c740 1 -- start start 2023-11-09T03:50:40.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.194+0000 7f28d018c740 1 Processor -- start 2023-11-09T03:50:40.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.195+0000 7f28d018c740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e46e188850 0x55e46e19f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.195+0000 7f28d018c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e46e188c30 0x55e46e19f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.195+0000 7f28d018c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e46e182850 0x55e46e1b9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.195+0000 7f28d018c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46e0c78b0 con 0x55e46e188c30 2023-11-09T03:50:40.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.195+0000 7f28d018c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46e081c80 con 0x55e46e182850 2023-11-09T03:50:40.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.195+0000 7f28d018c740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e46e081920 con 0x55e46e188850 2023-11-09T03:50:40.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.195+0000 7f28d018c740 10 client.0 _ll_register_callbacks cb 0x55e46ddd9ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:50:40.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.195+0000 7f28c67fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e46e188c30 0x55e46e19f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.195+0000 7f28c67fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e46e188c30 0x55e46e19f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36714/0 (socket says 172.21.15.76:36714) 2023-11-09T03:50:40.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.195+0000 7f28c67fc640 1 -- 172.21.15.76:0/949341029 learned_addr learned my addr 172.21.15.76:0/949341029 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:40.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.195+0000 7f28c6ffd640 1 --2- 172.21.15.76:0/949341029 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e46e188850 0x55e46e19f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.195+0000 7f28c77fe640 1 --2- 172.21.15.76:0/949341029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e46e182850 0x55e46e1b9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.195+0000 7f28c67fc640 1 -- 172.21.15.76:0/949341029 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e46e188850 msgr2=0x55e46e19f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:40.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.195+0000 7f28c67fc640 1 --2- 172.21.15.76:0/949341029 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e46e188850 0x55e46e19f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.195+0000 7f28c67fc640 1 -- 172.21.15.76:0/949341029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e46e182850 msgr2=0x55e46e1b9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:40.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.195+0000 7f28c67fc640 1 --2- 172.21.15.76:0/949341029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e46e182850 0x55e46e1b9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.195+0000 7f28c67fc640 1 -- 172.21.15.76:0/949341029 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c80cb350 con 0x55e46e188c30 2023-11-09T03:50:40.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.195+0000 7f28c67fc640 1 --2- 172.21.15.76:0/949341029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e46e188c30 0x55e46e19f970 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f28bc005410 tx=0x7f28bc0105b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d6fd38b5bd28b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:40.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.195+0000 7f28a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:50:40.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.196+0000 7f28a5ffb640 1 -- 172.21.15.76:0/949341029 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc015660 con 0x55e46e188c30 2023-11-09T03:50:40.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.196+0000 7f28a5ffb640 1 -- 172.21.15.76:0/949341029 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7f28bc0157c0 con 0x55e46e188c30 2023-11-09T03:50:40.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.196+0000 7f28d018c740 1 -- 172.21.15.76:0/949341029 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e46e073bf0 con 0x55e46e188c30 2023-11-09T03:50:40.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.196+0000 7f28a5ffb640 1 -- 172.21.15.76:0/949341029 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28bc010ed0 con 0x55e46e188c30 2023-11-09T03:50:40.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.196+0000 7f28d018c740 10 client.8046 fetch_fsmap learned FSMap version 73 2023-11-09T03:50:40.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.196+0000 7f28d018c740 1 -- 172.21.15.76:0/949341029 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x55e46e0c5f30 con 0x55e46e188c30 2023-11-09T03:50:40.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.196+0000 7f28a5ffb640 1 -- 172.21.15.76:0/949341029 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e46e073bf0 con 0x55e46e188c30 2023-11-09T03:50:40.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.196+0000 7f28a5ffb640 1 -- 172.21.15.76:0/949341029 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 73) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f28bc024050 con 0x55e46e188c30 2023-11-09T03:50:40.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.196+0000 7f28d018c740 10 client.8046 fetch_fsmap finished waiting for FSMap version 73 2023-11-09T03:50:40.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.196+0000 7f28d018c740 10 client.8046 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-09T03:50:40.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.196+0000 7f28d018c740 20 client.8046 populate_metadata read hostname 'smithi076' 2023-11-09T03:50:40.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.196+0000 7f28d018c740 1 --2- 172.21.15.76:0/949341029 >> [v2:172.21.15.113:6836/3449910842,v1:172.21.15.113:6837/3449910842] conn(0x55e46e1ba290 0x55e46e18ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.196+0000 7f28d018c740 4 client.8046 mds_command: new command op to 5993 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:50:40.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.196+0000 7f28d018c740 1 -- 172.21.15.76:0/949341029 --> [v2:172.21.15.113:6836/3449910842,v1:172.21.15.113:6837/3449910842] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e46e0c0890 con 0x55e46e1ba290 2023-11-09T03:50:40.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.196+0000 7f28c6ffd640 1 --2- 172.21.15.76:0/949341029 >> [v2:172.21.15.113:6836/3449910842,v1:172.21.15.113:6837/3449910842] conn(0x55e46e1ba290 0x55e46e18ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.197+0000 7f28a5ffb640 10 client.8046 ms_handle_connect on v2:172.21.15.113:6836/3449910842 2023-11-09T03:50:40.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.197+0000 7f28c6ffd640 1 --2- 172.21.15.76:0/949341029 >> [v2:172.21.15.113:6836/3449910842,v1:172.21.15.113:6837/3449910842] conn(0x55e46e1ba290 0x55e46e18ca60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e78562852dfc4910 server_cookie=6910c1e1f21230f in_seq=0 out_seq=0 2023-11-09T03:50:40.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.197+0000 7f28a5ffb640 1 -- 172.21.15.76:0/949341029 <== mds.0 v2:172.21.15.113:6836/3449910842 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e46e0c0890 con 0x55e46e1ba290 2023-11-09T03:50:40.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.197+0000 7f28a5ffb640 10 client.8046 handle_command_reply: tid=0 2023-11-09T03:50:40.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.198+0000 7f28d018c740 1 client.8046 shutdown 2023-11-09T03:50:40.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.198+0000 7f28d018c740 1 -- 172.21.15.76:0/949341029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e46e188c30 msgr2=0x55e46e19f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.198+0000 7f28d018c740 1 --2- 172.21.15.76:0/949341029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e46e188c30 0x55e46e19f970 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f28bc005410 tx=0x7f28bc0105b0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.198+0000 7f28d018c740 1 -- 172.21.15.76:0/949341029 shutdown_connections 2023-11-09T03:50:40.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.198+0000 7f28d018c740 1 --2- 172.21.15.76:0/949341029 >> [v2:172.21.15.113:6836/3449910842,v1:172.21.15.113:6837/3449910842] conn(0x55e46e1ba290 0x55e46e18ca60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.198+0000 7f28d018c740 1 --2- 172.21.15.76:0/949341029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e46e182850 0x55e46e1b9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.198+0000 7f28a5ffb640 0 client.8046 ms_handle_reset on v2:172.21.15.113:6836/3449910842 2023-11-09T03:50:40.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.198+0000 7f28d018c740 1 --2- 172.21.15.76:0/949341029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e46e188c30 0x55e46e19f970 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.198+0000 7f28d018c740 1 --2- 172.21.15.76:0/949341029 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e46e188850 0x55e46e19f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.198+0000 7f28d018c740 1 -- 172.21.15.76:0/949341029 >> 172.21.15.76:0/949341029 conn(0x55e46e15bc90 msgr2=0x55e46e15f830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:40.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.198+0000 7f28d018c740 1 -- 172.21.15.76:0/949341029 shutdown_connections 2023-11-09T03:50:40.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.198+0000 7f28d018c740 1 -- 172.21.15.76:0/949341029 wait complete. 2023-11-09T03:50:40.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.198+0000 7f28d018c740 20 client.8046 trim_cache size 0 max 16384 2023-11-09T03:50:40.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.213+0000 7f28d018c740 1 Processor -- start 2023-11-09T03:50:40.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.213+0000 7f28d018c740 1 -- start start 2023-11-09T03:50:40.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.214+0000 7f28d018c740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e46e182850 0x55e46e15b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.214+0000 7f28d018c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e46e188850 0x55e46e19d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.214+0000 7f28d018c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e46e188c30 0x55e46e15a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.214+0000 7f28d018c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46e047280 con 0x55e46e188850 2023-11-09T03:50:40.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.214+0000 7f28d018c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46e1ba9f0 con 0x55e46e188c30 2023-11-09T03:50:40.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.214+0000 7f28d018c740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e46e1a8ec0 con 0x55e46e182850 2023-11-09T03:50:40.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.214+0000 7f28c67fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e46e188850 0x55e46e19d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.214+0000 7f28c67fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e46e188850 0x55e46e19d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36718/0 (socket says 172.21.15.76:36718) 2023-11-09T03:50:40.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.214+0000 7f28c67fc640 1 -- 172.21.15.76:0/3868202448 learned_addr learned my addr 172.21.15.76:0/3868202448 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:40.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.214+0000 7f28c77fe640 1 --2- 172.21.15.76:0/3868202448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e46e188c30 0x55e46e15a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.214+0000 7f28c67fc640 1 -- 172.21.15.76:0/3868202448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e46e182850 msgr2=0x55e46e15b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:40.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.214+0000 7f28c67fc640 1 --2- 172.21.15.76:0/3868202448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e46e182850 0x55e46e15b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.214+0000 7f28c67fc640 1 -- 172.21.15.76:0/3868202448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e46e188c30 msgr2=0x55e46e15a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:40.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.214+0000 7f28c67fc640 1 --2- 172.21.15.76:0/3868202448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e46e188c30 0x55e46e15a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.214+0000 7f28c67fc640 1 -- 172.21.15.76:0/3868202448 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e46e063d10 con 0x55e46e188850 2023-11-09T03:50:40.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.214+0000 7f28c77fe640 1 --2- 172.21.15.76:0/3868202448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e46e188c30 0x55e46e15a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:50:40.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.214+0000 7f28c67fc640 1 --2- 172.21.15.76:0/3868202448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e46e188850 0x55e46e19d3a0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f28bc008fa0 tx=0x7f28bc00a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46401a5d1c06848a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:40.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.215+0000 7f28ad7fa640 1 -- 172.21.15.76:0/3868202448 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc015a70 con 0x55e46e188850 2023-11-09T03:50:40.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.215+0000 7f28ad7fa640 1 -- 172.21.15.76:0/3868202448 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28bc015bf0 con 0x55e46e188850 2023-11-09T03:50:40.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.215+0000 7f28ad7fa640 1 -- 172.21.15.76:0/3868202448 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc0156a0 con 0x55e46e188850 2023-11-09T03:50:40.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.215+0000 7f28d018c740 1 -- 172.21.15.76:0/3868202448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e46e188850 msgr2=0x55e46e19d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.215+0000 7f28d018c740 1 --2- 172.21.15.76:0/3868202448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e46e188850 0x55e46e19d3a0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f28bc008fa0 tx=0x7f28bc00a000 comp rx=0 tx=0).stop 2023-11-09T03:50:40.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.215+0000 7f28d018c740 1 -- 172.21.15.76:0/3868202448 shutdown_connections 2023-11-09T03:50:40.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.215+0000 7f28d018c740 1 --2- 172.21.15.76:0/3868202448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e46e188c30 0x55e46e15a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.215+0000 7f28d018c740 1 --2- 172.21.15.76:0/3868202448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e46e188850 0x55e46e19d3a0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.215+0000 7f28d018c740 1 --2- 172.21.15.76:0/3868202448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e46e182850 0x55e46e15b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.215+0000 7f28d018c740 1 -- 172.21.15.76:0/3868202448 >> 172.21.15.76:0/3868202448 conn(0x55e46e15bc90 msgr2=0x55e46e181c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:40.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.215+0000 7f28d018c740 1 -- 172.21.15.76:0/3868202448 shutdown_connections 2023-11-09T03:50:40.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.215+0000 7f28d018c740 1 -- 172.21.15.76:0/3868202448 wait complete. 2023-11-09T03:50:40.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.216+0000 7f28d018c740 1 -- start start 2023-11-09T03:50:40.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.216+0000 7f28d018c740 1 Processor -- start 2023-11-09T03:50:40.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.216+0000 7f28d018c740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e46e182850 0x55e46e210cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.216+0000 7f28d018c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e46e188850 0x55e46e18c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.216+0000 7f28d018c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e46e188c30 0x55e46e19c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.216+0000 7f28d018c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46e18d2b0 con 0x55e46e188850 2023-11-09T03:50:40.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.216+0000 7f28d018c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46e187c80 con 0x55e46e188c30 2023-11-09T03:50:40.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.216+0000 7f28d018c740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e46e211210 con 0x55e46e182850 2023-11-09T03:50:40.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.216+0000 7f28d018c740 10 client.0 _ll_register_callbacks cb 0x7f28c0055260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:50:40.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.216+0000 7f28c67fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e46e188850 0x55e46e18c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.217+0000 7f28c67fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e46e188850 0x55e46e18c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36732/0 (socket says 172.21.15.76:36732) 2023-11-09T03:50:40.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.217+0000 7f28c67fc640 1 -- 172.21.15.76:0/3704404408 learned_addr learned my addr 172.21.15.76:0/3704404408 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:40.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.217+0000 7f28c6ffd640 1 --2- 172.21.15.76:0/3704404408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e46e182850 0x55e46e210cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.217+0000 7f28c77fe640 1 --2- 172.21.15.76:0/3704404408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e46e188c30 0x55e46e19c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.217+0000 7f28c67fc640 1 -- 172.21.15.76:0/3704404408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e46e182850 msgr2=0x55e46e210cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:40.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.217+0000 7f28c67fc640 1 --2- 172.21.15.76:0/3704404408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e46e182850 0x55e46e210cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.217+0000 7f28c67fc640 1 -- 172.21.15.76:0/3704404408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e46e188c30 msgr2=0x55e46e19c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:40.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.217+0000 7f28c67fc640 1 --2- 172.21.15.76:0/3704404408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e46e188c30 0x55e46e19c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.217+0000 7f28c67fc640 1 -- 172.21.15.76:0/3704404408 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e46e0c5f30 con 0x55e46e188850 2023-11-09T03:50:40.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.217+0000 7f28c77fe640 1 --2- 172.21.15.76:0/3704404408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e46e188c30 0x55e46e19c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:50:40.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.217+0000 7f28c67fc640 1 --2- 172.21.15.76:0/3704404408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e46e188850 0x55e46e18c9d0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f28bc005410 tx=0x7f28bc003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca25490a832adc60 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:40.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.217+0000 7f28a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:50:40.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.217+0000 7f28a5ffb640 1 -- 172.21.15.76:0/3704404408 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc034070 con 0x55e46e188850 2023-11-09T03:50:40.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.217+0000 7f28d018c740 1 -- 172.21.15.76:0/3704404408 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28c00572c0 con 0x55e46e188850 2023-11-09T03:50:40.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.217+0000 7f28a5ffb640 1 -- 172.21.15.76:0/3704404408 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7f28bc000e40 con 0x55e46e188850 2023-11-09T03:50:40.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.217+0000 7f28a5ffb640 1 -- 172.21.15.76:0/3704404408 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28bc010b30 con 0x55e46e188850 2023-11-09T03:50:40.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.217+0000 7f28d018c740 10 client.8052 fetch_fsmap learned FSMap version 73 2023-11-09T03:50:40.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.217+0000 7f28d018c740 1 -- 172.21.15.76:0/3704404408 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x55e46e089500 con 0x55e46e188850 2023-11-09T03:50:40.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.217+0000 7f28a5ffb640 1 -- 172.21.15.76:0/3704404408 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c00572c0 con 0x55e46e188850 2023-11-09T03:50:40.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.217+0000 7f28a5ffb640 1 -- 172.21.15.76:0/3704404408 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 73) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f28bc024040 con 0x55e46e188850 2023-11-09T03:50:40.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.217+0000 7f28d018c740 10 client.8052 fetch_fsmap finished waiting for FSMap version 73 2023-11-09T03:50:40.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.217+0000 7f28d018c740 10 client.8052 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-09T03:50:40.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.217+0000 7f28d018c740 20 client.8052 populate_metadata read hostname 'smithi076' 2023-11-09T03:50:40.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.218+0000 7f28d018c740 1 --2- 172.21.15.76:0/3704404408 >> [v2:172.21.15.113:6836/3449910842,v1:172.21.15.113:6837/3449910842] conn(0x55e46e19cd00 0x55e46e18b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.218+0000 7f28d018c740 4 client.8052 mds_command: new command op to 5993 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-09T03:50:40.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.218+0000 7f28d018c740 1 -- 172.21.15.76:0/3704404408 --> [v2:172.21.15.113:6836/3449910842,v1:172.21.15.113:6837/3449910842] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e46e0c05b0 con 0x55e46e19cd00 2023-11-09T03:50:40.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.218+0000 7f28c6ffd640 1 --2- 172.21.15.76:0/3704404408 >> [v2:172.21.15.113:6836/3449910842,v1:172.21.15.113:6837/3449910842] conn(0x55e46e19cd00 0x55e46e18b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.218+0000 7f28a5ffb640 10 client.8052 ms_handle_connect on v2:172.21.15.113:6836/3449910842 2023-11-09T03:50:40.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.218+0000 7f28c6ffd640 1 --2- 172.21.15.76:0/3704404408 >> [v2:172.21.15.113:6836/3449910842,v1:172.21.15.113:6837/3449910842] conn(0x55e46e19cd00 0x55e46e18b020 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e0f80b91f5b8934 server_cookie=63db792bac73f7bf in_seq=0 out_seq=0 2023-11-09T03:50:40.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.218+0000 7f28a5ffb640 1 -- 172.21.15.76:0/3704404408 <== mds.0 v2:172.21.15.113:6836/3449910842 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55e46e0c05b0 con 0x55e46e19cd00 2023-11-09T03:50:40.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.218+0000 7f28a5ffb640 10 client.8052 handle_command_reply: tid=0 2023-11-09T03:50:40.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.218+0000 7f28d018c740 1 client.8052 shutdown 2023-11-09T03:50:40.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.218+0000 7f28d018c740 1 -- 172.21.15.76:0/3704404408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e46e188850 msgr2=0x55e46e18c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.218+0000 7f28d018c740 1 --2- 172.21.15.76:0/3704404408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e46e188850 0x55e46e18c9d0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f28bc005410 tx=0x7f28bc003760 comp rx=0 tx=0).stop 2023-11-09T03:50:40.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.218+0000 7f28d018c740 1 -- 172.21.15.76:0/3704404408 shutdown_connections 2023-11-09T03:50:40.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.218+0000 7f28d018c740 1 --2- 172.21.15.76:0/3704404408 >> [v2:172.21.15.113:6836/3449910842,v1:172.21.15.113:6837/3449910842] conn(0x55e46e19cd00 0x55e46e18b020 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.218+0000 7f28d018c740 1 --2- 172.21.15.76:0/3704404408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e46e188c30 0x55e46e19c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.218+0000 7f28d018c740 1 --2- 172.21.15.76:0/3704404408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e46e188850 0x55e46e18c9d0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.218+0000 7f28d018c740 1 --2- 172.21.15.76:0/3704404408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e46e182850 0x55e46e210cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.218+0000 7f28a5ffb640 0 client.8052 ms_handle_reset on v2:172.21.15.113:6836/3449910842 2023-11-09T03:50:40.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.218+0000 7f28d018c740 1 -- 172.21.15.76:0/3704404408 >> 172.21.15.76:0/3704404408 conn(0x55e46e15bc90 msgr2=0x55e46e19d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:40.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.218+0000 7f28d018c740 1 -- 172.21.15.76:0/3704404408 shutdown_connections 2023-11-09T03:50:40.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.219+0000 7f28d018c740 1 -- 172.21.15.76:0/3704404408 wait complete. 2023-11-09T03:50:40.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.219+0000 7f28d018c740 20 client.8052 trim_cache size 0 max 16384 2023-11-09T03:50:40.241 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:50:40.241 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "c213b26f-d0da-45d5-8e34-bb75c096d4f2", 2023-11-09T03:50:40.242 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2023-11-09T03:50:40.242 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 5993, 2023-11-09T03:50:40.242 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2023-11-09T03:50:40.242 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2023-11-09T03:50:40.242 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2023-11-09T03:50:40.243 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 6.5469299870000004, 2023-11-09T03:50:40.243 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 73, 2023-11-09T03:50:40.243 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 100, 2023-11-09T03:50:40.243 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 100, 2023-11-09T03:50:40.243 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 32.498044767000003 2023-11-09T03:50:40.244 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:50:40.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.219+0000 7f28cd5fb640 1 -- 172.21.15.76:0/3816508464 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f28b406ae70 msgr2=0x7f28b406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.219+0000 7f28cd5fb640 1 --2- 172.21.15.76:0/3816508464 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f28b406ae70 0x7f28b406d330 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f28b8036000 tx=0x7f28b803c040 comp rx=0 tx=0).stop 2023-11-09T03:50:40.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.219+0000 7f28cd5fb640 1 -- 172.21.15.76:0/3816508464 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28c8153e60 msgr2=0x7f28c80ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.219+0000 7f28cd5fb640 1 --2- 172.21.15.76:0/3816508464 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28c8153e60 0x7f28c80ca410 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f28c000cef0 tx=0x7f28c000a710 comp rx=0 tx=0).stop 2023-11-09T03:50:40.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.219+0000 7f28cd5fb640 1 -- 172.21.15.76:0/3816508464 shutdown_connections 2023-11-09T03:50:40.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.219+0000 7f28cd5fb640 1 --2- 172.21.15.76:0/3816508464 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f28b406ae70 0x7f28b406d330 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.219+0000 7f28cd5fb640 1 --2- 172.21.15.76:0/3816508464 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28c8153e60 0x7f28c80ca410 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.219+0000 7f28cd5fb640 1 --2- 172.21.15.76:0/3816508464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28c80cd6d0 0x7f28c80c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.219+0000 7f28cd5fb640 1 --2- 172.21.15.76:0/3816508464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f28c80b4880 0x7f28c8150020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.219+0000 7f28cd5fb640 1 -- 172.21.15.76:0/3816508464 >> 172.21.15.76:0/3816508464 conn(0x7f28c80cf880 msgr2=0x7f28c815ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:40.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.219+0000 7f28cd5fb640 1 -- 172.21.15.76:0/3816508464 shutdown_connections 2023-11-09T03:50:40.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.219+0000 7f28cd5fb640 1 -- 172.21.15.76:0/3816508464 wait complete. 2023-11-09T03:50:40.247 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-09T03:50:40.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.348+0000 7f9256ff5640 1 Processor -- start 2023-11-09T03:50:40.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.348+0000 7f9256ff5640 1 -- start start 2023-11-09T03:50:40.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.349+0000 7f9256ff5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f92500c37d0 0x7f92500c3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.349+0000 7f9256ff5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9250070710 0x7f9250070b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.349+0000 7f9256ff5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92500710a0 con 0x7f92501460c0 2023-11-09T03:50:40.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.349+0000 7f9256ff5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92501464a0 con 0x7f92500c37d0 2023-11-09T03:50:40.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.349+0000 7f9256ff5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92500c41a0 con 0x7f9250070710 2023-11-09T03:50:40.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.349+0000 7f9247fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f92500c37d0 0x7f92500c3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.349+0000 7f9247fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f92500c37d0 0x7f92500c3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36744/0 (socket says 172.21.15.76:36744) 2023-11-09T03:50:40.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.349+0000 7f9247fff640 1 -- 172.21.15.76:0/3699242707 learned_addr learned my addr 172.21.15.76:0/3699242707 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:40.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.350+0000 7f9255592640 1 --2- 172.21.15.76:0/3699242707 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9250070710 0x7f9250070b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.350+0000 7f9247fff640 1 -- 172.21.15.76:0/3699242707 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9250070710 msgr2=0x7f9250070b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:40.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.350+0000 7f9247fff640 1 --2- 172.21.15.76:0/3699242707 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9250070710 0x7f9250070b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.350+0000 7f9247fff640 1 -- 172.21.15.76:0/3699242707 >> v1:172.21.15.113:6789/0 conn(0x7f92501460c0 legacy=0x7f92500b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:40.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.350+0000 7f9247fff640 1 -- 172.21.15.76:0/3699242707 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9250146950 con 0x7f92500c37d0 2023-11-09T03:50:40.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.350+0000 7f9255592640 1 --2- 172.21.15.76:0/3699242707 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9250070710 0x7f9250070b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:50:40.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.350+0000 7f9247fff640 1 --2- 172.21.15.76:0/3699242707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92500c37d0 0x7f92500c3c60 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f92500cfa50 tx=0x7f924002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fca21071f55cb86 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:40.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.350+0000 7f92477fe640 1 -- 172.21.15.76:0/3699242707 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9240031830 con 0x7f92500c37d0 2023-11-09T03:50:40.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.350+0000 7f92477fe640 1 -- 172.21.15.76:0/3699242707 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9240031990 con 0x7f92500c37d0 2023-11-09T03:50:40.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.350+0000 7f92477fe640 1 -- 172.21.15.76:0/3699242707 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9240031d20 con 0x7f92500c37d0 2023-11-09T03:50:40.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.351+0000 7f9256ff5640 1 -- 172.21.15.76:0/3699242707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92500c37d0 msgr2=0x7f92500c3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.351+0000 7f9256ff5640 1 --2- 172.21.15.76:0/3699242707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92500c37d0 0x7f92500c3c60 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f92500cfa50 tx=0x7f924002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.351+0000 7f9256ff5640 1 -- 172.21.15.76:0/3699242707 shutdown_connections 2023-11-09T03:50:40.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.351+0000 7f9256ff5640 1 --2- 172.21.15.76:0/3699242707 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9250070710 0x7f9250070b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.351+0000 7f9256ff5640 1 --2- 172.21.15.76:0/3699242707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92500c37d0 0x7f92500c3c60 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.351+0000 7f9256ff5640 1 -- 172.21.15.76:0/3699242707 >> 172.21.15.76:0/3699242707 conn(0x7f92500ce950 msgr2=0x7f92500ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:40.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.351+0000 7f9256ff5640 1 -- 172.21.15.76:0/3699242707 shutdown_connections 2023-11-09T03:50:40.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.351+0000 7f9256ff5640 1 -- 172.21.15.76:0/3699242707 wait complete. 2023-11-09T03:50:40.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.352+0000 7f9256ff5640 1 Processor -- start 2023-11-09T03:50:40.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.352+0000 7f9256ff5640 1 -- start start 2023-11-09T03:50:40.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.353+0000 7f9256ff5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9250070710 0x7f92500acb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.353+0000 7f9256ff5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92500c37d0 0x7f92501f6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.353+0000 7f9256ff5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92501460c0 0x7f925015f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.353+0000 7f9256ff5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925015cf50 con 0x7f9250070710 2023-11-09T03:50:40.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.353+0000 7f9256ff5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92500aba70 con 0x7f92500c37d0 2023-11-09T03:50:40.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.353+0000 7f9256ff5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92500abd50 con 0x7f92501460c0 2023-11-09T03:50:40.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.353+0000 7f9254d91640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9250070710 0x7f92500acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.353+0000 7f9254d91640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9250070710 0x7f92500acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36760/0 (socket says 172.21.15.76:36760) 2023-11-09T03:50:40.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.353+0000 7f9254d91640 1 -- 172.21.15.76:0/4223520448 learned_addr learned my addr 172.21.15.76:0/4223520448 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:40.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.353+0000 7f9247fff640 1 --2- 172.21.15.76:0/4223520448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92500c37d0 0x7f92501f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.353+0000 7f9255592640 1 --2- 172.21.15.76:0/4223520448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92501460c0 0x7f925015f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.354+0000 7f9254d91640 1 -- 172.21.15.76:0/4223520448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92501460c0 msgr2=0x7f925015f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.354+0000 7f9254d91640 1 --2- 172.21.15.76:0/4223520448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92501460c0 0x7f925015f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.354+0000 7f9254d91640 1 -- 172.21.15.76:0/4223520448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92500c37d0 msgr2=0x7f92501f6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.354+0000 7f9254d91640 1 --2- 172.21.15.76:0/4223520448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92500c37d0 0x7f92501f6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.354+0000 7f9254d91640 1 -- 172.21.15.76:0/4223520448 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92400094d0 con 0x7f9250070710 2023-11-09T03:50:40.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.354+0000 7f9247fff640 1 --2- 172.21.15.76:0/4223520448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92500c37d0 0x7f92501f6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:50:40.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.354+0000 7f9255592640 1 --2- 172.21.15.76:0/4223520448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92501460c0 0x7f925015f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:50:40.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.354+0000 7f9254d91640 1 --2- 172.21.15.76:0/4223520448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9250070710 0x7f92500acb10 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f924c00b3f0 tx=0x7f924c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:40.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.354+0000 7f92457fa640 1 -- 172.21.15.76:0/4223520448 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c011830 con 0x7f9250070710 2023-11-09T03:50:40.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.354+0000 7f92457fa640 1 -- 172.21.15.76:0/4223520448 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f924c011990 con 0x7f9250070710 2023-11-09T03:50:40.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.354+0000 7f9256ff5640 1 -- 172.21.15.76:0/4223520448 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92500ac030 con 0x7f9250070710 2023-11-09T03:50:40.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.354+0000 7f92457fa640 1 -- 172.21.15.76:0/4223520448 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c011d40 con 0x7f9250070710 2023-11-09T03:50:40.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.354+0000 7f9256ff5640 1 -- 172.21.15.76:0/4223520448 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92500ac5f0 con 0x7f9250070710 2023-11-09T03:50:40.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.355+0000 7f92457fa640 1 -- 172.21.15.76:0/4223520448 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f924c01a400 con 0x7f9250070710 2023-11-09T03:50:40.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.356+0000 7f92457fa640 1 --2- 172.21.15.76:0/4223520448 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f923c06ae70 0x7f923c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.356+0000 7f92457fa640 1 -- 172.21.15.76:0/4223520448 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7f924c089ae0 con 0x7f9250070710 2023-11-09T03:50:40.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.356+0000 7f9247fff640 1 --2- 172.21.15.76:0/4223520448 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f923c06ae70 0x7f923c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.356+0000 7f9247fff640 1 --2- 172.21.15.76:0/4223520448 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f923c06ae70 0x7f923c06d330 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9240036000 tx=0x7f924003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:40.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.357+0000 7f9259b84740 1 Processor -- start 2023-11-09T03:50:40.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.357+0000 7f9259b84740 1 -- start start 2023-11-09T03:50:40.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.357+0000 7f9259b84740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5629e52e2850 0x5629e52b5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.357+0000 7f9259b84740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e52e2c30 0x5629e52e55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.357+0000 7f9259b84740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629e52b5c90 0x5629e52e5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.357+0000 7f9259b84740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e51c6740 con 0x5629e52b5c90 2023-11-09T03:50:40.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.357+0000 7f9259b84740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e52288d0 con 0x5629e52e2c30 2023-11-09T03:50:40.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.357+0000 7f9259b84740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629e5225fc0 con 0x5629e52e2850 2023-11-09T03:50:40.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.357+0000 7f9255592640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629e52b5c90 0x5629e52e5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.357+0000 7f9255592640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629e52b5c90 0x5629e52e5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36766/0 (socket says 172.21.15.76:36766) 2023-11-09T03:50:40.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.357+0000 7f9255592640 1 -- 172.21.15.76:0/1089751512 learned_addr learned my addr 172.21.15.76:0/1089751512 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:40.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.357+0000 7f9254d91640 1 --2- 172.21.15.76:0/1089751512 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5629e52e2850 0x5629e52b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.357+0000 7f9247fff640 1 --2- 172.21.15.76:0/1089751512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e52e2c30 0x5629e52e55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.358+0000 7f9255592640 1 -- 172.21.15.76:0/1089751512 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5629e52e2850 msgr2=0x5629e52b5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:40.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.358+0000 7f9255592640 1 --2- 172.21.15.76:0/1089751512 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5629e52e2850 0x5629e52b5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.358+0000 7f9255592640 1 -- 172.21.15.76:0/1089751512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e52e2c30 msgr2=0x5629e52e55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:40.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.358+0000 7f9255592640 1 --2- 172.21.15.76:0/1089751512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e52e2c30 0x5629e52e55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.358+0000 7f9255592640 1 -- 172.21.15.76:0/1089751512 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629e51e4ca0 con 0x5629e52b5c90 2023-11-09T03:50:40.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.358+0000 7f9255592640 1 --2- 172.21.15.76:0/1089751512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629e52b5c90 0x5629e52e5b20 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f924800d510 tx=0x7f924800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1eee61b6ed1fe47 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:40.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.358+0000 7f92367fc640 1 -- 172.21.15.76:0/1089751512 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924800b760 con 0x5629e52b5c90 2023-11-09T03:50:40.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.358+0000 7f92367fc640 1 -- 172.21.15.76:0/1089751512 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f924800b8c0 con 0x5629e52b5c90 2023-11-09T03:50:40.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.358+0000 7f92367fc640 1 -- 172.21.15.76:0/1089751512 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924800bc50 con 0x5629e52b5c90 2023-11-09T03:50:40.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.359+0000 7f9259b84740 1 -- 172.21.15.76:0/1089751512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629e52b5c90 msgr2=0x5629e52e5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.359+0000 7f9259b84740 1 --2- 172.21.15.76:0/1089751512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629e52b5c90 0x5629e52e5b20 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f924800d510 tx=0x7f924800d9d0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.359+0000 7f9259b84740 1 -- 172.21.15.76:0/1089751512 shutdown_connections 2023-11-09T03:50:40.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.359+0000 7f9259b84740 1 --2- 172.21.15.76:0/1089751512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629e52b5c90 0x5629e52e5b20 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.359+0000 7f9259b84740 1 --2- 172.21.15.76:0/1089751512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e52e2c30 0x5629e52e55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.359+0000 7f9259b84740 1 --2- 172.21.15.76:0/1089751512 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5629e52e2850 0x5629e52b5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.359+0000 7f9259b84740 1 -- 172.21.15.76:0/1089751512 >> 172.21.15.76:0/1089751512 conn(0x5629e52b7a60 msgr2=0x5629e53005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:40.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.359+0000 7f9259b84740 1 -- 172.21.15.76:0/1089751512 shutdown_connections 2023-11-09T03:50:40.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.359+0000 7f9259b84740 1 -- 172.21.15.76:0/1089751512 wait complete. 2023-11-09T03:50:40.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.359+0000 7f9259b84740 1 -- start start 2023-11-09T03:50:40.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.359+0000 7f9259b84740 1 Processor -- start 2023-11-09T03:50:40.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.360+0000 7f9259b84740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5629e52e2850 0x5629e52e90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.360+0000 7f9259b84740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e52e2c30 0x5629e52de390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.360+0000 7f9259b84740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629e52e35c0 0x5629e52c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.360+0000 7f9259b84740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e52218b0 con 0x5629e52e35c0 2023-11-09T03:50:40.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.360+0000 7f9259b84740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e51dbc80 con 0x5629e52e2c30 2023-11-09T03:50:40.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.360+0000 7f9259b84740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629e51db920 con 0x5629e52e2850 2023-11-09T03:50:40.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.360+0000 7f9259b84740 10 client.0 _ll_register_callbacks cb 0x5629e4f33ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:50:40.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.360+0000 7f9255592640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629e52e35c0 0x5629e52c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.360+0000 7f9255592640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629e52e35c0 0x5629e52c9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36770/0 (socket says 172.21.15.76:36770) 2023-11-09T03:50:40.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.360+0000 7f9255592640 1 -- 172.21.15.76:0/3348481074 learned_addr learned my addr 172.21.15.76:0/3348481074 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:40.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.360+0000 7f9247fff640 1 --2- 172.21.15.76:0/3348481074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e52e2c30 0x5629e52de390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.360+0000 7f9254d91640 1 --2- 172.21.15.76:0/3348481074 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5629e52e2850 0x5629e52e90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.361+0000 7f9255592640 1 -- 172.21.15.76:0/3348481074 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5629e52e2850 msgr2=0x5629e52e90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:40.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.361+0000 7f9255592640 1 --2- 172.21.15.76:0/3348481074 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5629e52e2850 0x5629e52e90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.361+0000 7f9255592640 1 -- 172.21.15.76:0/3348481074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e52e2c30 msgr2=0x5629e52de390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:40.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.361+0000 7f9255592640 1 --2- 172.21.15.76:0/3348481074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e52e2c30 0x5629e52de390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.361+0000 7f9255592640 1 -- 172.21.15.76:0/3348481074 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9248007c80 con 0x5629e52e35c0 2023-11-09T03:50:40.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.361+0000 7f9254d91640 1 --2- 172.21.15.76:0/3348481074 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5629e52e2850 0x5629e52e90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:50:40.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.361+0000 7f9255592640 1 --2- 172.21.15.76:0/3348481074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629e52e35c0 0x5629e52c9e70 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f924800ded0 tx=0x7f9248004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b62122d99d96c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:40.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.361+0000 7f922f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:50:40.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.361+0000 7f922f7fe640 1 -- 172.21.15.76:0/3348481074 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9248015820 con 0x5629e52e35c0 2023-11-09T03:50:40.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.361+0000 7f922f7fe640 1 -- 172.21.15.76:0/3348481074 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7f9248015980 con 0x5629e52e35c0 2023-11-09T03:50:40.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.361+0000 7f9259b84740 1 -- 172.21.15.76:0/3348481074 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5629e51cdbf0 con 0x5629e52e35c0 2023-11-09T03:50:40.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.361+0000 7f922f7fe640 1 -- 172.21.15.76:0/3348481074 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9248015b40 con 0x5629e52e35c0 2023-11-09T03:50:40.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.361+0000 7f9259b84740 10 client.8064 fetch_fsmap learned FSMap version 73 2023-11-09T03:50:40.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.361+0000 7f9259b84740 1 -- 172.21.15.76:0/3348481074 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x5629e521ff30 con 0x5629e52e35c0 2023-11-09T03:50:40.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.361+0000 7f922f7fe640 1 -- 172.21.15.76:0/3348481074 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5629e51cdbf0 con 0x5629e52e35c0 2023-11-09T03:50:40.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.361+0000 7f922f7fe640 1 -- 172.21.15.76:0/3348481074 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 73) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f9248008040 con 0x5629e52e35c0 2023-11-09T03:50:40.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.361+0000 7f9259b84740 10 client.8064 fetch_fsmap finished waiting for FSMap version 73 2023-11-09T03:50:40.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.361+0000 7f9259b84740 10 client.8064 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-09T03:50:40.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.361+0000 7f9259b84740 20 client.8064 populate_metadata read hostname 'smithi076' 2023-11-09T03:50:40.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.361+0000 7f9259b84740 1 --2- 172.21.15.76:0/3348481074 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x5629e52cc3d0 0x5629e52dd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.361+0000 7f9259b84740 4 client.8064 mds_command: new command op to 7449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:50:40.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.361+0000 7f9259b84740 1 -- 172.21.15.76:0/3348481074 --> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629e521a890 con 0x5629e52cc3d0 2023-11-09T03:50:40.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.362+0000 7f9254d91640 1 --2- 172.21.15.76:0/3348481074 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x5629e52cc3d0 0x5629e52dd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.362+0000 7f9254d91640 1 --2- 172.21.15.76:0/3348481074 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x5629e52cc3d0 0x5629e52dd7e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ff36ff67ea9be3fc server_cookie=f29432fe74f2c407 in_seq=0 out_seq=0 2023-11-09T03:50:40.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.362+0000 7f922f7fe640 10 client.8064 ms_handle_connect on v2:172.21.15.76:6832/3697850101 2023-11-09T03:50:40.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.362+0000 7f922f7fe640 1 -- 172.21.15.76:0/3348481074 <== mds.1 v2:172.21.15.76:6832/3697850101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5629e521a890 con 0x5629e52cc3d0 2023-11-09T03:50:40.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.362+0000 7f922f7fe640 10 client.8064 handle_command_reply: tid=0 2023-11-09T03:50:40.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.362+0000 7f9259b84740 1 client.8064 shutdown 2023-11-09T03:50:40.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.362+0000 7f9259b84740 1 -- 172.21.15.76:0/3348481074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629e52e35c0 msgr2=0x5629e52c9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.362+0000 7f9259b84740 1 --2- 172.21.15.76:0/3348481074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629e52e35c0 0x5629e52c9e70 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f924800ded0 tx=0x7f9248004ab0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.362+0000 7f9259b84740 1 -- 172.21.15.76:0/3348481074 shutdown_connections 2023-11-09T03:50:40.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.362+0000 7f9259b84740 1 --2- 172.21.15.76:0/3348481074 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x5629e52cc3d0 0x5629e52dd7e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.362+0000 7f9259b84740 1 --2- 172.21.15.76:0/3348481074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629e52e35c0 0x5629e52c9e70 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.362+0000 7f9259b84740 1 --2- 172.21.15.76:0/3348481074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e52e2c30 0x5629e52de390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.362+0000 7f922f7fe640 0 client.8064 ms_handle_reset on v2:172.21.15.76:6832/3697850101 2023-11-09T03:50:40.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.362+0000 7f9259b84740 1 --2- 172.21.15.76:0/3348481074 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5629e52e2850 0x5629e52e90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.362+0000 7f9259b84740 1 -- 172.21.15.76:0/3348481074 >> 172.21.15.76:0/3348481074 conn(0x5629e52b5c90 msgr2=0x5629e52b9600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:40.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.363+0000 7f9259b84740 1 -- 172.21.15.76:0/3348481074 shutdown_connections 2023-11-09T03:50:40.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.363+0000 7f9259b84740 1 -- 172.21.15.76:0/3348481074 wait complete. 2023-11-09T03:50:40.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.363+0000 7f9259b84740 20 client.8064 trim_cache size 0 max 16384 2023-11-09T03:50:40.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.378+0000 7f9259b84740 1 Processor -- start 2023-11-09T03:50:40.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.378+0000 7f9259b84740 1 -- start start 2023-11-09T03:50:40.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.379+0000 7f9259b84740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e52e2850 0x5629e52b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.379+0000 7f9259b84740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5629e52e2c30 0x5629e52b9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.379+0000 7f9259b84740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629e52e35c0 0x5629e52e8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.379+0000 7f9259b84740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e51a1280 con 0x5629e52e35c0 2023-11-09T03:50:40.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.379+0000 7f9259b84740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e5301670 con 0x5629e52e2850 2023-11-09T03:50:40.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.379+0000 7f9259b84740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629e52e22f0 con 0x5629e52e2c30 2023-11-09T03:50:40.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.379+0000 7f9255592640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629e52e35c0 0x5629e52e8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.379+0000 7f9255592640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629e52e35c0 0x5629e52e8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36776/0 (socket says 172.21.15.76:36776) 2023-11-09T03:50:40.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.379+0000 7f9255592640 1 -- 172.21.15.76:0/550569504 learned_addr learned my addr 172.21.15.76:0/550569504 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:40.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.379+0000 7f9254d91640 1 --2- 172.21.15.76:0/550569504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e52e2850 0x5629e52b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.379+0000 7f9247fff640 1 --2- 172.21.15.76:0/550569504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5629e52e2c30 0x5629e52b9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.379+0000 7f9255592640 1 -- 172.21.15.76:0/550569504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5629e52e2c30 msgr2=0x5629e52b9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:40.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.379+0000 7f9255592640 1 --2- 172.21.15.76:0/550569504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5629e52e2c30 0x5629e52b9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.379+0000 7f9255592640 1 -- 172.21.15.76:0/550569504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e52e2850 msgr2=0x5629e52b44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:40.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.379+0000 7f9255592640 1 --2- 172.21.15.76:0/550569504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e52e2850 0x5629e52b44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.379+0000 7f9255592640 1 -- 172.21.15.76:0/550569504 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629e51bdd10 con 0x5629e52e35c0 2023-11-09T03:50:40.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.379+0000 7f9255592640 1 --2- 172.21.15.76:0/550569504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629e52e35c0 0x5629e52e8ed0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f9248007f70 tx=0x7f924801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52bcf7931d5124f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:40.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.379+0000 7f9236ffd640 1 -- 172.21.15.76:0/550569504 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9248015620 con 0x5629e52e35c0 2023-11-09T03:50:40.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.379+0000 7f9236ffd640 1 -- 172.21.15.76:0/550569504 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92480157c0 con 0x5629e52e35c0 2023-11-09T03:50:40.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.379+0000 7f9236ffd640 1 -- 172.21.15.76:0/550569504 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9248015b70 con 0x5629e52e35c0 2023-11-09T03:50:40.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.380+0000 7f9259b84740 1 -- 172.21.15.76:0/550569504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629e52e35c0 msgr2=0x5629e52e8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.380+0000 7f9259b84740 1 --2- 172.21.15.76:0/550569504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629e52e35c0 0x5629e52e8ed0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f9248007f70 tx=0x7f924801e000 comp rx=0 tx=0).stop 2023-11-09T03:50:40.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.380+0000 7f9259b84740 1 -- 172.21.15.76:0/550569504 shutdown_connections 2023-11-09T03:50:40.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.380+0000 7f9259b84740 1 --2- 172.21.15.76:0/550569504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629e52e35c0 0x5629e52e8ed0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.380+0000 7f9259b84740 1 --2- 172.21.15.76:0/550569504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5629e52e2c30 0x5629e52b9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.380+0000 7f9259b84740 1 --2- 172.21.15.76:0/550569504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e52e2850 0x5629e52b44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.380+0000 7f9259b84740 1 -- 172.21.15.76:0/550569504 >> 172.21.15.76:0/550569504 conn(0x5629e52b5c90 msgr2=0x5629e52db5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:40.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.380+0000 7f9259b84740 1 -- 172.21.15.76:0/550569504 shutdown_connections 2023-11-09T03:50:40.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.380+0000 7f9259b84740 1 -- 172.21.15.76:0/550569504 wait complete. 2023-11-09T03:50:40.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.380+0000 7f9259b84740 1 -- start start 2023-11-09T03:50:40.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.381+0000 7f9259b84740 1 Processor -- start 2023-11-09T03:50:40.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.381+0000 7f9259b84740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e52e2850 0x5629e52e17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.381+0000 7f9259b84740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5629e52e2c30 0x5629e534d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.381+0000 7f9259b84740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629e52e35c0 0x5629e52e9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.381+0000 7f9259b84740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e52da310 con 0x5629e52e35c0 2023-11-09T03:50:40.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.381+0000 7f9259b84740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e52ba500 con 0x5629e52e2850 2023-11-09T03:50:40.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.381+0000 7f9259b84740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629e534db20 con 0x5629e52e2c30 2023-11-09T03:50:40.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.381+0000 7f9259b84740 10 client.0 _ll_register_callbacks cb 0x7f92480077f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:50:40.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.382+0000 7f9255592640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629e52e35c0 0x5629e52e9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.382+0000 7f9255592640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629e52e35c0 0x5629e52e9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36790/0 (socket says 172.21.15.76:36790) 2023-11-09T03:50:40.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.382+0000 7f9255592640 1 -- 172.21.15.76:0/1894521089 learned_addr learned my addr 172.21.15.76:0/1894521089 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:40.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.382+0000 7f9254d91640 1 --2- 172.21.15.76:0/1894521089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e52e2850 0x5629e52e17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.382+0000 7f9247fff640 1 --2- 172.21.15.76:0/1894521089 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5629e52e2c30 0x5629e534d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.382+0000 7f9255592640 1 -- 172.21.15.76:0/1894521089 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5629e52e2c30 msgr2=0x5629e534d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:40.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.382+0000 7f9255592640 1 --2- 172.21.15.76:0/1894521089 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5629e52e2c30 0x5629e534d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.382+0000 7f9255592640 1 -- 172.21.15.76:0/1894521089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e52e2850 msgr2=0x5629e52e17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:40.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.382+0000 7f9255592640 1 --2- 172.21.15.76:0/1894521089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e52e2850 0x5629e52e17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.382+0000 7f9255592640 1 -- 172.21.15.76:0/1894521089 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629e521ff30 con 0x5629e52e35c0 2023-11-09T03:50:40.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.382+0000 7f9255592640 1 --2- 172.21.15.76:0/1894521089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629e52e35c0 0x5629e52e9330 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f9248007f70 tx=0x7f9248013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827e7edef37ed936 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:40.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.382+0000 7f922f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:50:40.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.382+0000 7f922f7fe640 1 -- 172.21.15.76:0/1894521089 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9248029850 con 0x5629e52e35c0 2023-11-09T03:50:40.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.382+0000 7f922f7fe640 1 -- 172.21.15.76:0/1894521089 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7f92480299b0 con 0x5629e52e35c0 2023-11-09T03:50:40.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.382+0000 7f9259b84740 1 -- 172.21.15.76:0/1894521089 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f924800b8c0 con 0x5629e52e35c0 2023-11-09T03:50:40.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.382+0000 7f922f7fe640 1 -- 172.21.15.76:0/1894521089 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9248031470 con 0x5629e52e35c0 2023-11-09T03:50:40.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.382+0000 7f9259b84740 10 client.8070 fetch_fsmap learned FSMap version 73 2023-11-09T03:50:40.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.382+0000 7f9259b84740 1 -- 172.21.15.76:0/1894521089 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x5629e51e3500 con 0x5629e52e35c0 2023-11-09T03:50:40.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.382+0000 7f922f7fe640 1 -- 172.21.15.76:0/1894521089 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f924800b8c0 con 0x5629e52e35c0 2023-11-09T03:50:40.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.383+0000 7f922f7fe640 1 -- 172.21.15.76:0/1894521089 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 73) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f9248043070 con 0x5629e52e35c0 2023-11-09T03:50:40.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.383+0000 7f9259b84740 10 client.8070 fetch_fsmap finished waiting for FSMap version 73 2023-11-09T03:50:40.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.383+0000 7f9259b84740 10 client.8070 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-09T03:50:40.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.383+0000 7f9259b84740 20 client.8070 populate_metadata read hostname 'smithi076' 2023-11-09T03:50:40.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.383+0000 7f9259b84740 1 --2- 172.21.15.76:0/1894521089 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x5629e5300100 0x5629e53004e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.383+0000 7f9259b84740 4 client.8070 mds_command: new command op to 7449 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-09T03:50:40.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.383+0000 7f9259b84740 1 -- 172.21.15.76:0/1894521089 --> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5629e521a5b0 con 0x5629e5300100 2023-11-09T03:50:40.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.383+0000 7f9254d91640 1 --2- 172.21.15.76:0/1894521089 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x5629e5300100 0x5629e53004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.383+0000 7f922f7fe640 10 client.8070 ms_handle_connect on v2:172.21.15.76:6832/3697850101 2023-11-09T03:50:40.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.383+0000 7f9254d91640 1 --2- 172.21.15.76:0/1894521089 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x5629e5300100 0x5629e53004e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b19c87d470df34b server_cookie=f4bab79d49f27932 in_seq=0 out_seq=0 2023-11-09T03:50:40.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.383+0000 7f922f7fe640 1 -- 172.21.15.76:0/1894521089 <== mds.1 v2:172.21.15.76:6832/3697850101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x5629e521a5b0 con 0x5629e5300100 2023-11-09T03:50:40.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.383+0000 7f922f7fe640 10 client.8070 handle_command_reply: tid=0 2023-11-09T03:50:40.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.383+0000 7f9259b84740 1 client.8070 shutdown 2023-11-09T03:50:40.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.383+0000 7f9259b84740 1 -- 172.21.15.76:0/1894521089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629e52e35c0 msgr2=0x5629e52e9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.383+0000 7f9259b84740 1 --2- 172.21.15.76:0/1894521089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629e52e35c0 0x5629e52e9330 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f9248007f70 tx=0x7f9248013000 comp rx=0 tx=0).stop 2023-11-09T03:50:40.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.383+0000 7f9259b84740 1 -- 172.21.15.76:0/1894521089 shutdown_connections 2023-11-09T03:50:40.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.383+0000 7f9259b84740 1 --2- 172.21.15.76:0/1894521089 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x5629e5300100 0x5629e53004e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.383+0000 7f9259b84740 1 --2- 172.21.15.76:0/1894521089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629e52e35c0 0x5629e52e9330 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.383+0000 7f9259b84740 1 --2- 172.21.15.76:0/1894521089 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5629e52e2c30 0x5629e534d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.383+0000 7f922f7fe640 0 client.8070 ms_handle_reset on v2:172.21.15.76:6832/3697850101 2023-11-09T03:50:40.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.383+0000 7f9259b84740 1 --2- 172.21.15.76:0/1894521089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629e52e2850 0x5629e52e17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.383+0000 7f9259b84740 1 -- 172.21.15.76:0/1894521089 >> 172.21.15.76:0/1894521089 conn(0x5629e52b5c90 msgr2=0x5629e5322d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:40.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.383+0000 7f9259b84740 1 -- 172.21.15.76:0/1894521089 shutdown_connections 2023-11-09T03:50:40.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.383+0000 7f9259b84740 1 -- 172.21.15.76:0/1894521089 wait complete. 2023-11-09T03:50:40.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.384+0000 7f9259b84740 20 client.8070 trim_cache size 0 max 16384 2023-11-09T03:50:40.406 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:50:40.406 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "c213b26f-d0da-45d5-8e34-bb75c096d4f2", 2023-11-09T03:50:40.406 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 1, 2023-11-09T03:50:40.407 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 7449, 2023-11-09T03:50:40.407 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2023-11-09T03:50:40.407 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2023-11-09T03:50:40.407 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2023-11-09T03:50:40.407 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 92.935838953000001, 2023-11-09T03:50:40.407 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 73, 2023-11-09T03:50:40.408 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 100, 2023-11-09T03:50:40.408 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 100, 2023-11-09T03:50:40.408 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 108.38613223900001 2023-11-09T03:50:40.408 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:50:40.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.384+0000 7f9256ff5640 1 -- 172.21.15.76:0/4223520448 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f923c06ae70 msgr2=0x7f923c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.384+0000 7f9256ff5640 1 --2- 172.21.15.76:0/4223520448 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f923c06ae70 0x7f923c06d330 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9240036000 tx=0x7f924003c040 comp rx=0 tx=0).stop 2023-11-09T03:50:40.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.384+0000 7f9256ff5640 1 -- 172.21.15.76:0/4223520448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9250070710 msgr2=0x7f92500acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.384+0000 7f9256ff5640 1 --2- 172.21.15.76:0/4223520448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9250070710 0x7f92500acb10 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f924c00b3f0 tx=0x7f924c00b8b0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.384+0000 7f9256ff5640 1 -- 172.21.15.76:0/4223520448 shutdown_connections 2023-11-09T03:50:40.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.384+0000 7f9256ff5640 1 --2- 172.21.15.76:0/4223520448 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f923c06ae70 0x7f923c06d330 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.384+0000 7f9256ff5640 1 --2- 172.21.15.76:0/4223520448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92501460c0 0x7f925015f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.384+0000 7f9256ff5640 1 --2- 172.21.15.76:0/4223520448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92500c37d0 0x7f92501f6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.384+0000 7f9256ff5640 1 --2- 172.21.15.76:0/4223520448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9250070710 0x7f92500acb10 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.384+0000 7f9256ff5640 1 -- 172.21.15.76:0/4223520448 >> 172.21.15.76:0/4223520448 conn(0x7f92500ce950 msgr2=0x7f92500ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:40.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.384+0000 7f9256ff5640 1 -- 172.21.15.76:0/4223520448 shutdown_connections 2023-11-09T03:50:40.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.384+0000 7f9256ff5640 1 -- 172.21.15.76:0/4223520448 wait complete. 2023-11-09T03:50:40.412 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:50:40.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.513+0000 7f31bb776640 1 Processor -- start 2023-11-09T03:50:40.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.514+0000 7f31bb776640 1 -- start start 2023-11-09T03:50:40.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.514+0000 7f31bb776640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f31b40cde60 0x7f31b415d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.514+0000 7f31bb776640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31b415daa0 0x7f31b4157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.514+0000 7f31bb776640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b415e000 con 0x7f31b40ce240 2023-11-09T03:50:40.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.514+0000 7f31bb776640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31b415e5c0 con 0x7f31b40cde60 2023-11-09T03:50:40.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.514+0000 7f31bb776640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b415e2e0 con 0x7f31b415daa0 2023-11-09T03:50:40.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.515+0000 7f31b9d13640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f31b40ce240 0x7f31b406b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:55078/0 (socket says 172.21.15.76:55078) 2023-11-09T03:50:40.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.515+0000 7f31b9d13640 1 -- 172.21.15.76:0/2791367996 learned_addr learned my addr 172.21.15.76:0/2791367996 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:50:40.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.515+0000 7f31b8d11640 1 --2- 172.21.15.76:0/2791367996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31b415daa0 0x7f31b4157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.515+0000 7f31abfff640 1 -- 172.21.15.76:0/2791367996 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 112719544 0 0) 0x7f31b415e000 con 0x7f31b40ce240 2023-11-09T03:50:40.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.515+0000 7f31b9512640 1 --2- 172.21.15.76:0/2791367996 >> v2:172.21.15.113:3300/0 conn(0x7f31b40cde60 0x7f31b415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.515+0000 7f31abfff640 1 -- 172.21.15.76:0/2791367996 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31a0003520 con 0x7f31b40ce240 2023-11-09T03:50:40.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.515+0000 7f31abfff640 1 -- 172.21.15.76:0/2791367996 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3826816459 0 0) 0x7f31a0003520 con 0x7f31b40ce240 2023-11-09T03:50:40.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.515+0000 7f31abfff640 1 -- 172.21.15.76:0/2791367996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31b415daa0 msgr2=0x7f31b4157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:40.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.515+0000 7f31abfff640 1 --2- 172.21.15.76:0/2791367996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31b415daa0 0x7f31b4157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.515+0000 7f31abfff640 1 -- 172.21.15.76:0/2791367996 >> v2:172.21.15.113:3300/0 conn(0x7f31b40cde60 msgr2=0x7f31b415d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:40.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.515+0000 7f31abfff640 1 --2- 172.21.15.76:0/2791367996 >> v2:172.21.15.113:3300/0 conn(0x7f31b40cde60 0x7f31b415d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.515+0000 7f31abfff640 1 -- 172.21.15.76:0/2791367996 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b40d1120 con 0x7f31b40ce240 2023-11-09T03:50:40.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.515+0000 7f31abfff640 1 -- 172.21.15.76:0/2791367996 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f31ac0019f0 con 0x7f31b40ce240 2023-11-09T03:50:40.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.515+0000 7f31abfff640 1 -- 172.21.15.76:0/2791367996 >> v1:172.21.15.76:6789/0 conn(0x7f31b40ce240 legacy=0x7f31b406b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.515+0000 7f31abfff640 1 --2- 172.21.15.76:0/2791367996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31b415daa0 0x7f31a0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.515+0000 7f31abfff640 1 --2- 172.21.15.76:0/2791367996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31a0003f40 0x7f31a00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.515+0000 7f31abfff640 1 --2- 172.21.15.76:0/2791367996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31a0004d00 0x7f31a0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.515+0000 7f31abfff640 1 -- 172.21.15.76:0/2791367996 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b415e5c0 con 0x7f31a0003f40 2023-11-09T03:50:40.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.515+0000 7f31abfff640 1 -- 172.21.15.76:0/2791367996 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b415e2e0 con 0x7f31a0004d00 2023-11-09T03:50:40.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.515+0000 7f31abfff640 1 -- 172.21.15.76:0/2791367996 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a000b980 con 0x7f31b415daa0 2023-11-09T03:50:40.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.515+0000 7f31b8d11640 1 --2- 172.21.15.76:0/2791367996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31a0003f40 0x7f31a00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.515+0000 7f31b9512640 1 --2- 172.21.15.76:0/2791367996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31b415daa0 0x7f31a0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.515+0000 7f31b9d13640 1 --2- 172.21.15.76:0/2791367996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31a0004d00 0x7f31a0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.516+0000 7f31b8d11640 1 -- 172.21.15.76:0/2791367996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31b415daa0 msgr2=0x7f31a0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:40.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.516+0000 7f31b8d11640 1 --2- 172.21.15.76:0/2791367996 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31b415daa0 0x7f31a0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.516+0000 7f31b8d11640 1 -- 172.21.15.76:0/2791367996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31a0004d00 msgr2=0x7f31a0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:40.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.516+0000 7f31b8d11640 1 --2- 172.21.15.76:0/2791367996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31a0004d00 0x7f31a0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.516+0000 7f31b8d11640 1 -- 172.21.15.76:0/2791367996 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a0003520 con 0x7f31a0003f40 2023-11-09T03:50:40.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.516+0000 7f31b8d11640 1 --2- 172.21.15.76:0/2791367996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31a0003f40 0x7f31a00047c0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f31a4009af0 tx=0x7f31a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e3800e5c4b2d58 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:40.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.516+0000 7f31abfff640 1 -- 172.21.15.76:0/2791367996 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a402c6b0 con 0x7f31a0003f40 2023-11-09T03:50:40.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.516+0000 7f31abfff640 1 -- 172.21.15.76:0/2791367996 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a402c810 con 0x7f31a0003f40 2023-11-09T03:50:40.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.516+0000 7f31abfff640 1 -- 172.21.15.76:0/2791367996 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a402cba0 con 0x7f31a0003f40 2023-11-09T03:50:40.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.517+0000 7f31bb776640 1 -- 172.21.15.76:0/2791367996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31a0003f40 msgr2=0x7f31a00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.517+0000 7f31bb776640 1 --2- 172.21.15.76:0/2791367996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31a0003f40 0x7f31a00047c0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f31a4009af0 tx=0x7f31a00057f0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.517+0000 7f31b9d13640 1 -- 172.21.15.76:0/2791367996 reap_dead start 2023-11-09T03:50:40.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.517+0000 7f31bb776640 1 -- 172.21.15.76:0/2791367996 shutdown_connections 2023-11-09T03:50:40.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.517+0000 7f31bb776640 1 -- 172.21.15.76:0/2791367996 >> 172.21.15.76:0/2791367996 conn(0x7f31b4071f30 msgr2=0x7f31b40cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:40.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.517+0000 7f31bb776640 1 -- 172.21.15.76:0/2791367996 shutdown_connections 2023-11-09T03:50:40.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.517+0000 7f31bb776640 1 -- 172.21.15.76:0/2791367996 wait complete. 2023-11-09T03:50:40.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.518+0000 7f31bb776640 1 Processor -- start 2023-11-09T03:50:40.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.518+0000 7f31bb776640 1 -- start start 2023-11-09T03:50:40.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.519+0000 7f31bb776640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31a0003f40 0x7f31b40c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.519+0000 7f31bb776640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b41f6650 0x7f31b4156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.519+0000 7f31bb776640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31b41f3990 0x7f31b41f5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.519+0000 7f31bb776640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b41509c0 con 0x7f31b41f6650 2023-11-09T03:50:40.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.519+0000 7f31bb776640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b40c8e70 con 0x7f31b41f3990 2023-11-09T03:50:40.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.519+0000 7f31bb776640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b4157740 con 0x7f31a0003f40 2023-11-09T03:50:40.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.519+0000 7f31b8d11640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b41f6650 0x7f31b4156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.519+0000 7f31b8d11640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b41f6650 0x7f31b4156e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36804/0 (socket says 172.21.15.76:36804) 2023-11-09T03:50:40.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.519+0000 7f31b8d11640 1 -- 172.21.15.76:0/3159535947 learned_addr learned my addr 172.21.15.76:0/3159535947 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:40.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.519+0000 7f31b9d13640 1 --2- 172.21.15.76:0/3159535947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31b41f3990 0x7f31b41f5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.519+0000 7f31b9512640 1 --2- 172.21.15.76:0/3159535947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31a0003f40 0x7f31b40c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.519+0000 7f31b8d11640 1 -- 172.21.15.76:0/3159535947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31a0003f40 msgr2=0x7f31b40c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.519+0000 7f31b8d11640 1 --2- 172.21.15.76:0/3159535947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31a0003f40 0x7f31b40c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.519+0000 7f31b8d11640 1 -- 172.21.15.76:0/3159535947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31b41f3990 msgr2=0x7f31b41f5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.519+0000 7f31b8d11640 1 --2- 172.21.15.76:0/3159535947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31b41f3990 0x7f31b41f5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.519+0000 7f31b8d11640 1 -- 172.21.15.76:0/3159535947 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b4150020 con 0x7f31b41f6650 2023-11-09T03:50:40.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.519+0000 7f31b9d13640 1 --2- 172.21.15.76:0/3159535947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31b41f3990 0x7f31b41f5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:50:40.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.520+0000 7f31b8d11640 1 --2- 172.21.15.76:0/3159535947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b41f6650 0x7f31b4156e40 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f31a4037040 tx=0x7f31a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:40.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.520+0000 7f31a9ffb640 1 -- 172.21.15.76:0/3159535947 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a4003810 con 0x7f31b41f6650 2023-11-09T03:50:40.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.520+0000 7f31a9ffb640 1 -- 172.21.15.76:0/3159535947 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a4003970 con 0x7f31b41f6650 2023-11-09T03:50:40.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.520+0000 7f31a9ffb640 1 -- 172.21.15.76:0/3159535947 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a4003d00 con 0x7f31b41f6650 2023-11-09T03:50:40.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.520+0000 7f31bb776640 1 -- 172.21.15.76:0/3159535947 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b4150300 con 0x7f31b41f6650 2023-11-09T03:50:40.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.520+0000 7f31bb776640 1 -- 172.21.15.76:0/3159535947 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b4157380 con 0x7f31b41f6650 2023-11-09T03:50:40.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.521+0000 7f31bb776640 1 -- 172.21.15.76:0/3159535947 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b4157510 con 0x7f31b41f6650 2023-11-09T03:50:40.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.521+0000 7f31a9ffb640 1 -- 172.21.15.76:0/3159535947 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f31a4040450 con 0x7f31b41f6650 2023-11-09T03:50:40.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.522+0000 7f31a9ffb640 1 --2- 172.21.15.76:0/3159535947 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f319806ae70 0x7f319806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.522+0000 7f31a9ffb640 1 -- 172.21.15.76:0/3159535947 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7f31a40abfb0 con 0x7f31b41f6650 2023-11-09T03:50:40.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.522+0000 7f31b9512640 1 --2- 172.21.15.76:0/3159535947 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f319806ae70 0x7f319806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.523+0000 7f31b9512640 1 --2- 172.21.15.76:0/3159535947 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f319806ae70 0x7f319806d330 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f31b0009b80 tx=0x7f31b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:40.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.526+0000 7f31a9ffb640 1 -- 172.21.15.76:0/3159535947 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f31a40deca0 con 0x7f31b41f6650 2023-11-09T03:50:40.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.727+0000 7f31bb776640 1 -- 172.21.15.76:0/3159535947 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31b4150b00 con 0x7f31b41f6650 2023-11-09T03:50:40.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.728+0000 7f31a9ffb640 1 -- 172.21.15.76:0/3159535947 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4935 (secure 0 0 0) 0x7f31b4150b00 con 0x7f31b41f6650 2023-11-09T03:50:40.728 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:50:40.728 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8016,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3229239133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3229239133},{"type":"v1","addr":"172.21.15.76:6835","nonce":3229239133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:50:36.685855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5993,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5993":{"gid":5993,"name":"b","rank":0,"incarnation":69,"state":"up:active","state_seq":11,"addr":"172.21.15.113:6837/3449910842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3449910842},{"type":"v1","addr":"172.21.15.113:6837","nonce":3449910842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:50:40.728 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 73 2023-11-09T03:50:40.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.730+0000 7f31bb776640 1 -- 172.21.15.76:0/3159535947 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f319806ae70 msgr2=0x7f319806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.730+0000 7f31bb776640 1 --2- 172.21.15.76:0/3159535947 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f319806ae70 0x7f319806d330 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f31b0009b80 tx=0x7f31b0008040 comp rx=0 tx=0).stop 2023-11-09T03:50:40.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.730+0000 7f31bb776640 1 -- 172.21.15.76:0/3159535947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b41f6650 msgr2=0x7f31b4156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.730+0000 7f31bb776640 1 --2- 172.21.15.76:0/3159535947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b41f6650 0x7f31b4156e40 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f31a4037040 tx=0x7f31a4004730 comp rx=0 tx=0).stop 2023-11-09T03:50:40.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.730+0000 7f31bb776640 1 -- 172.21.15.76:0/3159535947 shutdown_connections 2023-11-09T03:50:40.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.730+0000 7f31bb776640 1 --2- 172.21.15.76:0/3159535947 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f319806ae70 0x7f319806d330 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.730+0000 7f31bb776640 1 --2- 172.21.15.76:0/3159535947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f31b41f3990 0x7f31b41f5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.730+0000 7f31bb776640 1 --2- 172.21.15.76:0/3159535947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b41f6650 0x7f31b4156e40 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.730+0000 7f31bb776640 1 --2- 172.21.15.76:0/3159535947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f31a0003f40 0x7f31b40c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.730+0000 7f31bb776640 1 -- 172.21.15.76:0/3159535947 >> 172.21.15.76:0/3159535947 conn(0x7f31b4071f30 msgr2=0x7f31b40d09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:40.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.730+0000 7f31bb776640 1 -- 172.21.15.76:0/3159535947 shutdown_connections 2023-11-09T03:50:40.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.730+0000 7f31bb776640 1 -- 172.21.15.76:0/3159535947 wait complete. 2023-11-09T03:50:40.740 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump snaps --server 2023-11-09T03:50:40.865 INFO:teuthology.orchestra.run.smithi113.stdout:{"last_snap":4,"last_created":4,"last_destroyed":1,"pending_noop":[],"snaps":[{"snapid":2,"ino":1099511627777,"stamp":"2023-11-09T03:49:19.643623+0000","name":"s11","metadata":{}},{"snapid":3,"ino":1099511627777,"stamp":"2023-11-09T03:49:50.605488+0000","name":"s12","metadata":{}},{"snapid":4,"ino":1099511627777,"stamp":"2023-11-09T03:50:16.615058+0000","name":"s15","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-09T03:50:40.865 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 4, "last_destroyed": 1, "last_snap": 4, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s11", "snapid": 2, "stamp": "2023-11-09T03:49:19.643623+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s12", "snapid": 3, "stamp": "2023-11-09T03:49:50.605488+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s15", "snapid": 4, "stamp": "2023-11-09T03:50:16.615058+0000" } ] } 2023-11-09T03:50:40.866 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-09T03:50:40.866 INFO:tasks.cephfs.test_snapshots:testing snapserver mds_kill_mdstable_at=6 2023-11-09T03:50:40.866 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:50:40.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.968+0000 7f55b064b640 1 Processor -- start 2023-11-09T03:50:40.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.968+0000 7f55b064b640 1 -- start start 2023-11-09T03:50:40.970 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.969+0000 7f55b064b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f55a806a830 0x7f55a80cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.971 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.969+0000 7f55b064b640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55a806d310 con 0x7f55a815ab70 2023-11-09T03:50:40.971 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.969+0000 7f55b064b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55a815d860 con 0x7f55a806eba0 2023-11-09T03:50:40.971 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.969+0000 7f55b064b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55a80cba70 con 0x7f55a806a830 2023-11-09T03:50:40.971 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.969+0000 7f55ae3e7640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f55a815ab70 0x7f55a806a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:55080/0 (socket says 172.21.15.76:55080) 2023-11-09T03:50:40.971 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.969+0000 7f55ae3e7640 1 -- 172.21.15.76:0/2604252231 learned_addr learned my addr 172.21.15.76:0/2604252231 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:50:40.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.969+0000 7f55ad3e5640 1 -- 172.21.15.76:0/2604252231 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568461201 0 0) 0x7f55a806d310 con 0x7f55a815ab70 2023-11-09T03:50:40.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.969+0000 7f55ad3e5640 1 -- 172.21.15.76:0/2604252231 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5594003520 con 0x7f55a815ab70 2023-11-09T03:50:40.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.969+0000 7f55adbe6640 1 --2- 172.21.15.76:0/2604252231 >> v2:172.21.15.113:3300/0 conn(0x7f55a806a830 0x7f55a80cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.969+0000 7f55ad3e5640 1 -- 172.21.15.76:0/2604252231 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1671566049 0 0) 0x7f5594003520 con 0x7f55a815ab70 2023-11-09T03:50:40.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.969+0000 7f55ad3e5640 1 -- 172.21.15.76:0/2604252231 >> v2:172.21.15.113:3300/0 conn(0x7f55a806a830 msgr2=0x7f55a80cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:40.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.969+0000 7f55ad3e5640 1 --2- 172.21.15.76:0/2604252231 >> v2:172.21.15.113:3300/0 conn(0x7f55a806a830 0x7f55a80cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.969+0000 7f55ad3e5640 1 -- 172.21.15.76:0/2604252231 >> v1:172.21.15.113:6789/0 conn(0x7f55a806eba0 legacy=0x7f55a80ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.969+0000 7f55ad3e5640 1 -- 172.21.15.76:0/2604252231 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a815db40 con 0x7f55a815ab70 2023-11-09T03:50:40.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.969+0000 7f55ad3e5640 1 -- 172.21.15.76:0/2604252231 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f55a40017f0 con 0x7f55a815ab70 2023-11-09T03:50:40.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.970+0000 7f55ad3e5640 1 -- 172.21.15.76:0/2604252231 >> v1:172.21.15.76:6789/0 conn(0x7f55a815ab70 legacy=0x7f55a806a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.970+0000 7f55ad3e5640 1 --2- 172.21.15.76:0/2604252231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5594003970 0x7f5594003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.970+0000 7f55ad3e5640 1 --2- 172.21.15.76:0/2604252231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55940047c0 0x7f5594004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.970+0000 7f55ad3e5640 1 --2- 172.21.15.76:0/2604252231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5594005180 0x7f559400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.970+0000 7f55ad3e5640 1 -- 172.21.15.76:0/2604252231 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a80cba70 con 0x7f55940047c0 2023-11-09T03:50:40.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.970+0000 7f55ad3e5640 1 -- 172.21.15.76:0/2604252231 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559400c120 con 0x7f5594005180 2023-11-09T03:50:40.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.970+0000 7f55ad3e5640 1 -- 172.21.15.76:0/2604252231 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f559400c2e0 con 0x7f5594003970 2023-11-09T03:50:40.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.970+0000 7f55adbe6640 1 --2- 172.21.15.76:0/2604252231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55940047c0 0x7f5594004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.970+0000 7f55ae3e7640 1 --2- 172.21.15.76:0/2604252231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5594003970 0x7f5594003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.970+0000 7f55aebe8640 1 --2- 172.21.15.76:0/2604252231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5594005180 0x7f559400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.970+0000 7f55adbe6640 1 -- 172.21.15.76:0/2604252231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5594003970 msgr2=0x7f5594003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:40.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.970+0000 7f55adbe6640 1 --2- 172.21.15.76:0/2604252231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5594003970 0x7f5594003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.970+0000 7f55adbe6640 1 -- 172.21.15.76:0/2604252231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5594005180 msgr2=0x7f559400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:40.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.970+0000 7f55adbe6640 1 --2- 172.21.15.76:0/2604252231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5594005180 0x7f559400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.970+0000 7f55aebe8640 1 -- 172.21.15.76:0/2604252231 reap_dead start 2023-11-09T03:50:40.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.970+0000 7f55adbe6640 1 -- 172.21.15.76:0/2604252231 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5594003520 con 0x7f55940047c0 2023-11-09T03:50:40.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.970+0000 7f55ae3e7640 1 --2- 172.21.15.76:0/2604252231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5594003970 0x7f5594003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:50:40.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.971+0000 7f55adbe6640 1 --2- 172.21.15.76:0/2604252231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55940047c0 0x7f5594004c40 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f5598009a80 tx=0x7f55940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fefb6ca65d4a7350 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:40.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.971+0000 7f55ad3e5640 1 -- 172.21.15.76:0/2604252231 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5598005990 con 0x7f55940047c0 2023-11-09T03:50:40.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.971+0000 7f55ad3e5640 1 -- 172.21.15.76:0/2604252231 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5598005af0 con 0x7f55940047c0 2023-11-09T03:50:40.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.971+0000 7f55ad3e5640 1 -- 172.21.15.76:0/2604252231 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5598005ea0 con 0x7f55940047c0 2023-11-09T03:50:40.981 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.971+0000 7f55b064b640 1 -- 172.21.15.76:0/2604252231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55940047c0 msgr2=0x7f5594004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.981 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.971+0000 7f55b064b640 1 --2- 172.21.15.76:0/2604252231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55940047c0 0x7f5594004c40 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f5598009a80 tx=0x7f55940057f0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.971+0000 7f55b064b640 1 -- 172.21.15.76:0/2604252231 shutdown_connections 2023-11-09T03:50:40.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.971+0000 7f55b064b640 1 --2- 172.21.15.76:0/2604252231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55940047c0 0x7f5594004c40 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.972+0000 7f55b064b640 1 -- 172.21.15.76:0/2604252231 >> 172.21.15.76:0/2604252231 conn(0x7f55a8098d40 msgr2=0x7f55a80cfa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:40.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.972+0000 7f55b064b640 1 -- 172.21.15.76:0/2604252231 shutdown_connections 2023-11-09T03:50:40.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.972+0000 7f55b064b640 1 -- 172.21.15.76:0/2604252231 wait complete. 2023-11-09T03:50:40.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.973+0000 7f55b064b640 1 Processor -- start 2023-11-09T03:50:40.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.973+0000 7f55b064b640 1 -- start start 2023-11-09T03:50:40.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.973+0000 7f55b064b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55940047c0 0x7f55a809bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.973+0000 7f55b064b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55a809c060 0x7f55a80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.973+0000 7f55b064b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55a80b52d0 0x7f55a806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.973+0000 7f55b064b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a8161c50 con 0x7f55a809c060 2023-11-09T03:50:40.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.973+0000 7f55b064b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a81f2990 con 0x7f55940047c0 2023-11-09T03:50:40.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.973+0000 7f55b064b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a81f2b80 con 0x7f55a80b52d0 2023-11-09T03:50:40.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.974+0000 7f55adbe6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55a809c060 0x7f55a80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.974+0000 7f55adbe6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55a809c060 0x7f55a80b4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36810/0 (socket says 172.21.15.76:36810) 2023-11-09T03:50:40.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.974+0000 7f55adbe6640 1 -- 172.21.15.76:0/1225522660 learned_addr learned my addr 172.21.15.76:0/1225522660 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:40.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.974+0000 7f55ae3e7640 1 --2- 172.21.15.76:0/1225522660 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55940047c0 0x7f55a809bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.974+0000 7f55aebe8640 1 --2- 172.21.15.76:0/1225522660 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55a80b52d0 0x7f55a806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.974+0000 7f55adbe6640 1 -- 172.21.15.76:0/1225522660 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55a80b52d0 msgr2=0x7f55a806d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.974+0000 7f55adbe6640 1 --2- 172.21.15.76:0/1225522660 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55a80b52d0 0x7f55a806d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.974+0000 7f55adbe6640 1 -- 172.21.15.76:0/1225522660 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55940047c0 msgr2=0x7f55a809bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:40.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.974+0000 7f55adbe6640 1 --2- 172.21.15.76:0/1225522660 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55940047c0 0x7f55a809bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:40.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.974+0000 7f55adbe6640 1 -- 172.21.15.76:0/1225522660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a80b3a70 con 0x7f55a809c060 2023-11-09T03:50:40.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.974+0000 7f55ae3e7640 1 --2- 172.21.15.76:0/1225522660 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55940047c0 0x7f55a809bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:50:40.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.974+0000 7f55adbe6640 1 --2- 172.21.15.76:0/1225522660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55a809c060 0x7f55a80b4d90 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f5598036040 tx=0x7f5598004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:40.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.974+0000 7f559effd640 1 -- 172.21.15.76:0/1225522660 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5598006030 con 0x7f55a809c060 2023-11-09T03:50:40.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.974+0000 7f559effd640 1 -- 172.21.15.76:0/1225522660 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5598006190 con 0x7f55a809c060 2023-11-09T03:50:40.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.974+0000 7f55b064b640 1 -- 172.21.15.76:0/1225522660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a80b3d50 con 0x7f55a809c060 2023-11-09T03:50:40.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.974+0000 7f559effd640 1 -- 172.21.15.76:0/1225522660 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5598006520 con 0x7f55a809c060 2023-11-09T03:50:40.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.974+0000 7f55b064b640 1 -- 172.21.15.76:0/1225522660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a80b4310 con 0x7f55a809c060 2023-11-09T03:50:40.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.975+0000 7f55b064b640 1 -- 172.21.15.76:0/1225522660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a8074290 con 0x7f55a809c060 2023-11-09T03:50:40.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.976+0000 7f559effd640 1 -- 172.21.15.76:0/1225522660 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5598006680 con 0x7f55a809c060 2023-11-09T03:50:40.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.976+0000 7f559effd640 1 --2- 172.21.15.76:0/1225522660 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f558c06ae70 0x7f558c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:40.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.976+0000 7f559effd640 1 -- 172.21.15.76:0/1225522660 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7f55980ac1e0 con 0x7f55a809c060 2023-11-09T03:50:40.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.976+0000 7f55ae3e7640 1 --2- 172.21.15.76:0/1225522660 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f558c06ae70 0x7f558c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:40.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.977+0000 7f55ae3e7640 1 --2- 172.21.15.76:0/1225522660 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f558c06ae70 0x7f558c06d330 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f55a4000ef0 tx=0x7f55a4000f70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:40.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:40.980+0000 7f559effd640 1 -- 172.21.15.76:0/1225522660 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5598076d80 con 0x7f55a809c060 2023-11-09T03:50:41.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.182+0000 7f55b064b640 1 -- 172.21.15.76:0/1225522660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55a806ae70 con 0x7f55a809c060 2023-11-09T03:50:41.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.183+0000 7f559effd640 1 -- 172.21.15.76:0/1225522660 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4935 (secure 0 0 0) 0x7f559807a3a0 con 0x7f55a809c060 2023-11-09T03:50:41.183 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:50:41.183 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8016,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3229239133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3229239133},{"type":"v1","addr":"172.21.15.76:6835","nonce":3229239133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:50:36.685855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5993,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5993":{"gid":5993,"name":"b","rank":0,"incarnation":69,"state":"up:active","state_seq":11,"addr":"172.21.15.113:6837/3449910842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3449910842},{"type":"v1","addr":"172.21.15.113:6837","nonce":3449910842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:50:41.184 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 73 2023-11-09T03:50:41.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.185+0000 7f55b064b640 1 -- 172.21.15.76:0/1225522660 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f558c06ae70 msgr2=0x7f558c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:41.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.185+0000 7f55b064b640 1 --2- 172.21.15.76:0/1225522660 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f558c06ae70 0x7f558c06d330 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f55a4000ef0 tx=0x7f55a4000f70 comp rx=0 tx=0).stop 2023-11-09T03:50:41.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.185+0000 7f55b064b640 1 -- 172.21.15.76:0/1225522660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55a809c060 msgr2=0x7f55a80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:41.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.185+0000 7f55b064b640 1 --2- 172.21.15.76:0/1225522660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55a809c060 0x7f55a80b4d90 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f5598036040 tx=0x7f5598004730 comp rx=0 tx=0).stop 2023-11-09T03:50:41.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.185+0000 7f55b064b640 1 -- 172.21.15.76:0/1225522660 shutdown_connections 2023-11-09T03:50:41.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.185+0000 7f55b064b640 1 --2- 172.21.15.76:0/1225522660 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f558c06ae70 0x7f558c06d330 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:41.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.185+0000 7f55b064b640 1 --2- 172.21.15.76:0/1225522660 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55a80b52d0 0x7f55a806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:41.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.185+0000 7f55b064b640 1 --2- 172.21.15.76:0/1225522660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55a809c060 0x7f55a80b4d90 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:41.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.185+0000 7f55b064b640 1 --2- 172.21.15.76:0/1225522660 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55940047c0 0x7f55a809bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:41.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.185+0000 7f55b064b640 1 -- 172.21.15.76:0/1225522660 >> 172.21.15.76:0/1225522660 conn(0x7f55a8098d40 msgr2=0x7f55a80b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:41.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.185+0000 7f55b064b640 1 -- 172.21.15.76:0/1225522660 shutdown_connections 2023-11-09T03:50:41.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.185+0000 7f55b064b640 1 -- 172.21.15.76:0/1225522660 wait complete. 2023-11-09T03:50:41.195 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:0 true 2023-11-09T03:50:41.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.297+0000 7f2bedd61640 1 Processor -- start 2023-11-09T03:50:41.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.297+0000 7f2bedd61640 1 -- start start 2023-11-09T03:50:41.298 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.298+0000 7f2bedd61640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2be8159c00 0x7f2be80cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:41.298 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.298+0000 7f2bedd61640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2be80ae1d0 0x7f2be80c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:41.298 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.298+0000 7f2bedd61640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be815aff0 con 0x7f2be80c48e0 2023-11-09T03:50:41.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.298+0000 7f2bedd61640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be80c52d0 con 0x7f2be8159c00 2023-11-09T03:50:41.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.298+0000 7f2bedd61640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be815a3e0 con 0x7f2be80ae1d0 2023-11-09T03:50:41.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.298+0000 7f2be77fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2be8159c00 0x7f2be80cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:41.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.298+0000 7f2be6ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2be80c48e0 0x7f2be80adac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:55094/0 (socket says 172.21.15.76:55094) 2023-11-09T03:50:41.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.298+0000 7f2be6ffd640 1 -- 172.21.15.76:0/303308476 learned_addr learned my addr 172.21.15.76:0/303308476 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:50:41.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.298+0000 7f2be7fff640 1 --2- 172.21.15.76:0/303308476 >> v2:172.21.15.113:3300/0 conn(0x7f2be80ae1d0 0x7f2be80c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:41.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.298+0000 7f2be77fe640 1 -- 172.21.15.76:0/303308476 >> v2:172.21.15.113:3300/0 conn(0x7f2be80ae1d0 msgr2=0x7f2be80c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:41.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.298+0000 7f2be77fe640 1 --2- 172.21.15.76:0/303308476 >> v2:172.21.15.113:3300/0 conn(0x7f2be80ae1d0 0x7f2be80c4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:41.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.298+0000 7f2be77fe640 1 -- 172.21.15.76:0/303308476 >> v1:172.21.15.76:6789/0 conn(0x7f2be80c48e0 legacy=0x7f2be80adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:41.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.298+0000 7f2be77fe640 1 -- 172.21.15.76:0/303308476 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be815a6c0 con 0x7f2be8159c00 2023-11-09T03:50:41.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.298+0000 7f2be67fc640 1 -- 172.21.15.76:0/303308476 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2994136009 0 0) 0x7f2be815aff0 con 0x7f2be80c48e0 2023-11-09T03:50:41.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.299+0000 7f2be77fe640 1 --2- 172.21.15.76:0/303308476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2be8159c00 0x7f2be80cc890 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f2be80ce800 tx=0x7f2bd802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c28e162ef292aee server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:41.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.299+0000 7f2be67fc640 1 -- 172.21.15.76:0/303308476 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd802e070 con 0x7f2be8159c00 2023-11-09T03:50:41.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.299+0000 7f2be67fc640 1 -- 172.21.15.76:0/303308476 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd80315a0 con 0x7f2be8159c00 2023-11-09T03:50:41.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.299+0000 7f2be67fc640 1 -- 172.21.15.76:0/303308476 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8031930 con 0x7f2be8159c00 2023-11-09T03:50:41.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.300+0000 7f2bedd61640 1 -- 172.21.15.76:0/303308476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2be8159c00 msgr2=0x7f2be80cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:41.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.300+0000 7f2bedd61640 1 --2- 172.21.15.76:0/303308476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2be8159c00 0x7f2be80cc890 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f2be80ce800 tx=0x7f2bd802b840 comp rx=0 tx=0).stop 2023-11-09T03:50:41.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.300+0000 7f2bedd61640 1 -- 172.21.15.76:0/303308476 shutdown_connections 2023-11-09T03:50:41.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.300+0000 7f2bedd61640 1 --2- 172.21.15.76:0/303308476 >> v2:172.21.15.113:3300/0 conn(0x7f2be80ae1d0 0x7f2be80c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:41.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.300+0000 7f2bedd61640 1 --2- 172.21.15.76:0/303308476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2be8159c00 0x7f2be80cc890 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:41.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.300+0000 7f2bedd61640 1 -- 172.21.15.76:0/303308476 >> 172.21.15.76:0/303308476 conn(0x7f2be80ce950 msgr2=0x7f2be80ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:41.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.300+0000 7f2bedd61640 1 -- 172.21.15.76:0/303308476 shutdown_connections 2023-11-09T03:50:41.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.300+0000 7f2bedd61640 1 -- 172.21.15.76:0/303308476 wait complete. 2023-11-09T03:50:41.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.301+0000 7f2bedd61640 1 Processor -- start 2023-11-09T03:50:41.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.301+0000 7f2bedd61640 1 -- start start 2023-11-09T03:50:41.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.302+0000 7f2bedd61640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2be80ae1d0 0x7f2be8156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:41.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.302+0000 7f2bedd61640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2be80c48e0 0x7f2be80b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:41.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.302+0000 7f2bedd61640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2be8159c00 0x7f2be81e4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:41.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.302+0000 7f2bedd61640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be80b1050 con 0x7f2be80c48e0 2023-11-09T03:50:41.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.302+0000 7f2bedd61640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be81e2390 con 0x7f2be8159c00 2023-11-09T03:50:41.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.302+0000 7f2bedd61640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be80b0a90 con 0x7f2be80ae1d0 2023-11-09T03:50:41.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.302+0000 7f2be6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2be80c48e0 0x7f2be80b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:41.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.302+0000 7f2be6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2be80c48e0 0x7f2be80b0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36830/0 (socket says 172.21.15.76:36830) 2023-11-09T03:50:41.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.302+0000 7f2be6ffd640 1 -- 172.21.15.76:0/1667909227 learned_addr learned my addr 172.21.15.76:0/1667909227 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:41.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.302+0000 7f2be77fe640 1 --2- 172.21.15.76:0/1667909227 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2be80ae1d0 0x7f2be8156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:41.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.302+0000 7f2be7fff640 1 --2- 172.21.15.76:0/1667909227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2be8159c00 0x7f2be81e4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:41.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.302+0000 7f2be6ffd640 1 -- 172.21.15.76:0/1667909227 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2be80ae1d0 msgr2=0x7f2be8156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:41.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.302+0000 7f2be6ffd640 1 --2- 172.21.15.76:0/1667909227 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2be80ae1d0 0x7f2be8156660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:41.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.302+0000 7f2be6ffd640 1 -- 172.21.15.76:0/1667909227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2be8159c00 msgr2=0x7f2be81e4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:41.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.302+0000 7f2be6ffd640 1 --2- 172.21.15.76:0/1667909227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2be8159c00 0x7f2be81e4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:41.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.302+0000 7f2be6ffd640 1 -- 172.21.15.76:0/1667909227 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd8009400 con 0x7f2be80c48e0 2023-11-09T03:50:41.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.302+0000 7f2be7fff640 1 --2- 172.21.15.76:0/1667909227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2be8159c00 0x7f2be81e4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:50:41.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.302+0000 7f2be6ffd640 1 --2- 172.21.15.76:0/1667909227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2be80c48e0 0x7f2be80b0040 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00c4d0 tx=0x7f2bdc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:41.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.303+0000 7f2bcffff640 1 -- 172.21.15.76:0/1667909227 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc0074e0 con 0x7f2be80c48e0 2023-11-09T03:50:41.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.303+0000 7f2bcffff640 1 -- 172.21.15.76:0/1667909227 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bdc007660 con 0x7f2be80c48e0 2023-11-09T03:50:41.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.303+0000 7f2bedd61640 1 -- 172.21.15.76:0/1667909227 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be80b0d70 con 0x7f2be80c48e0 2023-11-09T03:50:41.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.303+0000 7f2bcffff640 1 -- 172.21.15.76:0/1667909227 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc004a20 con 0x7f2be80c48e0 2023-11-09T03:50:41.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.303+0000 7f2bedd61640 1 -- 172.21.15.76:0/1667909227 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be81e28c0 con 0x7f2be80c48e0 2023-11-09T03:50:41.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.304+0000 7f2bedd61640 1 -- 172.21.15.76:0/1667909227 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be8074290 con 0x7f2be80c48e0 2023-11-09T03:50:41.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.304+0000 7f2bcffff640 1 -- 172.21.15.76:0/1667909227 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2bdc004b80 con 0x7f2be80c48e0 2023-11-09T03:50:41.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.304+0000 7f2bcffff640 1 --2- 172.21.15.76:0/1667909227 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2bd406ae70 0x7f2bd406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:41.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.305+0000 7f2bcffff640 1 -- 172.21.15.76:0/1667909227 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7f2bdc089180 con 0x7f2be80c48e0 2023-11-09T03:50:41.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.305+0000 7f2be77fe640 1 --2- 172.21.15.76:0/1667909227 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2bd406ae70 0x7f2bd406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:41.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.305+0000 7f2be77fe640 1 --2- 172.21.15.76:0/1667909227 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2bd406ae70 0x7f2bd406d330 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8036000 tx=0x7f2bd803d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:41.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.309+0000 7f2bcffff640 1 -- 172.21.15.76:0/1667909227 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2bdc053c00 con 0x7f2be80c48e0 2023-11-09T03:50:41.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:41.544+0000 7f2bedd61640 1 -- 172.21.15.76:0/1667909227 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"} v 0) v1 -- 0x7f2be80ca360 con 0x7f2be80c48e0 2023-11-09T03:50:42.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.443+0000 7f2bcffff640 1 -- 172.21.15.76:0/1667909227 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"}]=0 freezing mds.5993 v74) v1 ==== 112+0+0 (secure 0 0 0) 0x7f2bdc057220 con 0x7f2be80c48e0 2023-11-09T03:50:42.443 INFO:teuthology.orchestra.run.smithi076.stderr:freezing mds.5993 2023-11-09T03:50:42.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.446+0000 7f2bedd61640 1 -- 172.21.15.76:0/1667909227 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2bd406ae70 msgr2=0x7f2bd406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:42.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.446+0000 7f2bedd61640 1 --2- 172.21.15.76:0/1667909227 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2bd406ae70 0x7f2bd406d330 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8036000 tx=0x7f2bd803d040 comp rx=0 tx=0).stop 2023-11-09T03:50:42.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.446+0000 7f2bedd61640 1 -- 172.21.15.76:0/1667909227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2be80c48e0 msgr2=0x7f2be80b0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:42.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.446+0000 7f2bedd61640 1 --2- 172.21.15.76:0/1667909227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2be80c48e0 0x7f2be80b0040 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00c4d0 tx=0x7f2bdc00c990 comp rx=0 tx=0).stop 2023-11-09T03:50:42.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.446+0000 7f2bedd61640 1 -- 172.21.15.76:0/1667909227 shutdown_connections 2023-11-09T03:50:42.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.446+0000 7f2bedd61640 1 --2- 172.21.15.76:0/1667909227 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2bd406ae70 0x7f2bd406d330 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:42.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.446+0000 7f2bedd61640 1 --2- 172.21.15.76:0/1667909227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2be8159c00 0x7f2be81e4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:42.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.446+0000 7f2bedd61640 1 --2- 172.21.15.76:0/1667909227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2be80c48e0 0x7f2be80b0040 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:42.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.446+0000 7f2bedd61640 1 --2- 172.21.15.76:0/1667909227 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2be80ae1d0 0x7f2be8156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:42.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.446+0000 7f2bedd61640 1 -- 172.21.15.76:0/1667909227 >> 172.21.15.76:0/1667909227 conn(0x7f2be80ce950 msgr2=0x7f2be8158d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:42.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.446+0000 7f2bedd61640 1 -- 172.21.15.76:0/1667909227 shutdown_connections 2023-11-09T03:50:42.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.446+0000 7f2bedd61640 1 -- 172.21.15.76:0/1667909227 wait complete. 2023-11-09T03:50:42.459 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_kill_mdstable_at 6 2023-11-09T03:50:42.583 INFO:teuthology.orchestra.run.smithi113.stdout:{"success":"mds_kill_mdstable_at = '6' (not observed, change may require restart) "} 2023-11-09T03:50:42.584 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '6' (not observed, change may require restart) " } 2023-11-09T03:50:42.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:50:42.584 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s16) 2023-11-09T03:50:42.587 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2023-11-09T03:50:42.720 INFO:teuthology.orchestra.run.smithi076.stdout:{"mds_beacon_grace":"15.000000"} 2023-11-09T03:50:42.720 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2023-11-09T03:50:42.721 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:50:42.726 INFO:tasks.ceph.mds.b.smithi113.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_commit_logged(ceph::cref_t&)' thread 7fd08dd68640 time 2023-11-09T03:50:42.728095+0000 2023-11-09T03:50:42.726 INFO:tasks.ceph.mds.b.smithi113.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 171: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 6) 2023-11-09T03:50:42.727 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:50:42.728 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fd09996efaf] 2023-11-09T03:50:42.728 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fd09996f16d] 2023-11-09T03:50:42.728 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x41b) [0x55adb6daa21b] 2023-11-09T03:50:42.728 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: (MDSContext::complete(int)+0x5c) [0x55adb6dd9bcc] 2023-11-09T03:50:42.729 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (MDSIOContextBase::complete(int)+0x31c) [0x55adb6de2b0c] 2023-11-09T03:50:42.729 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: (MDSLogContextBase::complete(int)+0x53) [0x55adb6de2c63] 2023-11-09T03:50:42.729 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (Finisher::finisher_thread_entry()+0x175) [0x7fd099a164a5] 2023-11-09T03:50:42.729 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7fd09909f802] 2023-11-09T03:50:42.729 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7fd09903f450] 2023-11-09T03:50:42.729 INFO:tasks.ceph.mds.b.smithi113.stderr:*** Caught signal (Aborted) ** 2023-11-09T03:50:42.730 INFO:tasks.ceph.mds.b.smithi113.stderr: in thread 7fd08dd68640 thread_name:MR_Finisher 2023-11-09T03:50:42.730 INFO:tasks.ceph.mds.b.smithi113.stderr:2023-11-09T03:50:42.727+0000 7fd08dd68640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_commit_logged(ceph::cref_t&)' thread 7fd08dd68640 time 2023-11-09T03:50:42.728095+0000 2023-11-09T03:50:42.730 INFO:tasks.ceph.mds.b.smithi113.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 171: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 6) 2023-11-09T03:50:42.730 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:50:42.730 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:50:42.731 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fd09996efaf] 2023-11-09T03:50:42.731 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fd09996f16d] 2023-11-09T03:50:42.731 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x41b) [0x55adb6daa21b] 2023-11-09T03:50:42.731 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: (MDSContext::complete(int)+0x5c) [0x55adb6dd9bcc] 2023-11-09T03:50:42.731 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (MDSIOContextBase::complete(int)+0x31c) [0x55adb6de2b0c] 2023-11-09T03:50:42.732 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: (MDSLogContextBase::complete(int)+0x53) [0x55adb6de2c63] 2023-11-09T03:50:42.732 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (Finisher::finisher_thread_entry()+0x175) [0x7fd099a164a5] 2023-11-09T03:50:42.732 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7fd09909f802] 2023-11-09T03:50:42.732 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7fd09903f450] 2023-11-09T03:50:42.732 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:50:42.733 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:50:42.733 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fd099054db0] 2023-11-09T03:50:42.733 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fd0990a154c] 2023-11-09T03:50:42.733 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: raise() 2023-11-09T03:50:42.734 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: abort() 2023-11-09T03:50:42.734 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fd09996f009] 2023-11-09T03:50:42.734 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fd09996f16d] 2023-11-09T03:50:42.734 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x41b) [0x55adb6daa21b] 2023-11-09T03:50:42.734 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (MDSContext::complete(int)+0x5c) [0x55adb6dd9bcc] 2023-11-09T03:50:42.735 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: (MDSIOContextBase::complete(int)+0x31c) [0x55adb6de2b0c] 2023-11-09T03:50:42.735 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: (MDSLogContextBase::complete(int)+0x53) [0x55adb6de2c63] 2023-11-09T03:50:42.735 INFO:tasks.ceph.mds.b.smithi113.stderr: 11: (Finisher::finisher_thread_entry()+0x175) [0x7fd099a164a5] 2023-11-09T03:50:42.735 INFO:tasks.ceph.mds.b.smithi113.stderr: 12: /lib64/libc.so.6(+0x9f802) [0x7fd09909f802] 2023-11-09T03:50:42.735 INFO:tasks.ceph.mds.b.smithi113.stderr: 13: /lib64/libc.so.6(+0x3f450) [0x7fd09903f450] 2023-11-09T03:50:42.736 INFO:tasks.ceph.mds.b.smithi113.stderr:2023-11-09T03:50:42.727+0000 7fd08dd68640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:50:42.736 INFO:tasks.ceph.mds.b.smithi113.stderr: in thread 7fd08dd68640 thread_name:MR_Finisher 2023-11-09T03:50:42.736 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:50:42.736 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:50:42.736 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fd099054db0] 2023-11-09T03:50:42.737 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fd0990a154c] 2023-11-09T03:50:42.737 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: raise() 2023-11-09T03:50:42.737 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: abort() 2023-11-09T03:50:42.737 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fd09996f009] 2023-11-09T03:50:42.737 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fd09996f16d] 2023-11-09T03:50:42.737 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x41b) [0x55adb6daa21b] 2023-11-09T03:50:42.738 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (MDSContext::complete(int)+0x5c) [0x55adb6dd9bcc] 2023-11-09T03:50:42.738 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: (MDSIOContextBase::complete(int)+0x31c) [0x55adb6de2b0c] 2023-11-09T03:50:42.738 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: (MDSLogContextBase::complete(int)+0x53) [0x55adb6de2c63] 2023-11-09T03:50:42.738 INFO:tasks.ceph.mds.b.smithi113.stderr: 11: (Finisher::finisher_thread_entry()+0x175) [0x7fd099a164a5] 2023-11-09T03:50:42.738 INFO:tasks.ceph.mds.b.smithi113.stderr: 12: /lib64/libc.so.6(+0x9f802) [0x7fd09909f802] 2023-11-09T03:50:42.739 INFO:tasks.ceph.mds.b.smithi113.stderr: 13: /lib64/libc.so.6(+0x3f450) [0x7fd09903f450] 2023-11-09T03:50:42.739 INFO:tasks.ceph.mds.b.smithi113.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:50:42.739 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:50:42.740 INFO:tasks.ceph.mds.b.smithi113.stderr: -499> 2023-11-09T03:50:38.633+0000 7fd09156f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:50:42.740 INFO:tasks.ceph.mds.b.smithi113.stderr: -497> 2023-11-09T03:50:38.633+0000 7fd09156f640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-09T03:50:42.740 INFO:tasks.ceph.mds.b.smithi113.stderr: -1> 2023-11-09T03:50:42.727+0000 7fd08dd68640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_commit_logged(ceph::cref_t&)' thread 7fd08dd68640 time 2023-11-09T03:50:42.728095+0000 2023-11-09T03:50:42.740 INFO:tasks.ceph.mds.b.smithi113.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 171: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 6) 2023-11-09T03:50:42.741 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:50:42.741 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:50:42.741 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fd09996efaf] 2023-11-09T03:50:42.741 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fd09996f16d] 2023-11-09T03:50:42.741 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x41b) [0x55adb6daa21b] 2023-11-09T03:50:42.742 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: (MDSContext::complete(int)+0x5c) [0x55adb6dd9bcc] 2023-11-09T03:50:42.742 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (MDSIOContextBase::complete(int)+0x31c) [0x55adb6de2b0c] 2023-11-09T03:50:42.742 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: (MDSLogContextBase::complete(int)+0x53) [0x55adb6de2c63] 2023-11-09T03:50:42.742 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (Finisher::finisher_thread_entry()+0x175) [0x7fd099a164a5] 2023-11-09T03:50:42.742 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7fd09909f802] 2023-11-09T03:50:42.742 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7fd09903f450] 2023-11-09T03:50:42.743 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:50:42.743 INFO:tasks.ceph.mds.b.smithi113.stderr: 0> 2023-11-09T03:50:42.727+0000 7fd08dd68640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:50:42.743 INFO:tasks.ceph.mds.b.smithi113.stderr: in thread 7fd08dd68640 thread_name:MR_Finisher 2023-11-09T03:50:42.743 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:50:42.743 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:50:42.744 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fd099054db0] 2023-11-09T03:50:42.744 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fd0990a154c] 2023-11-09T03:50:42.744 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: raise() 2023-11-09T03:50:42.744 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: abort() 2023-11-09T03:50:42.744 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fd09996f009] 2023-11-09T03:50:42.744 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fd09996f16d] 2023-11-09T03:50:42.745 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x41b) [0x55adb6daa21b] 2023-11-09T03:50:42.745 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (MDSContext::complete(int)+0x5c) [0x55adb6dd9bcc] 2023-11-09T03:50:42.745 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: (MDSIOContextBase::complete(int)+0x31c) [0x55adb6de2b0c] 2023-11-09T03:50:42.745 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: (MDSLogContextBase::complete(int)+0x53) [0x55adb6de2c63] 2023-11-09T03:50:42.745 INFO:tasks.ceph.mds.b.smithi113.stderr: 11: (Finisher::finisher_thread_entry()+0x175) [0x7fd099a164a5] 2023-11-09T03:50:42.745 INFO:tasks.ceph.mds.b.smithi113.stderr: 12: /lib64/libc.so.6(+0x9f802) [0x7fd09909f802] 2023-11-09T03:50:42.746 INFO:tasks.ceph.mds.b.smithi113.stderr: 13: /lib64/libc.so.6(+0x3f450) [0x7fd09903f450] 2023-11-09T03:50:42.746 INFO:tasks.ceph.mds.b.smithi113.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:50:42.746 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:50:42.746 INFO:tasks.ceph.mds.b.smithi113.stderr: -499> 2023-11-09T03:50:38.633+0000 7fd09156f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:50:42.746 INFO:tasks.ceph.mds.b.smithi113.stderr: -497> 2023-11-09T03:50:38.633+0000 7fd09156f640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-09T03:50:42.747 INFO:tasks.ceph.mds.b.smithi113.stderr: -1> 2023-11-09T03:50:42.727+0000 7fd08dd68640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_commit_logged(ceph::cref_t&)' thread 7fd08dd68640 time 2023-11-09T03:50:42.728095+0000 2023-11-09T03:50:42.747 INFO:tasks.ceph.mds.b.smithi113.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 171: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 6) 2023-11-09T03:50:42.747 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:50:42.747 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:50:42.747 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fd09996efaf] 2023-11-09T03:50:42.747 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fd09996f16d] 2023-11-09T03:50:42.748 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x41b) [0x55adb6daa21b] 2023-11-09T03:50:42.748 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: (MDSContext::complete(int)+0x5c) [0x55adb6dd9bcc] 2023-11-09T03:50:42.748 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (MDSIOContextBase::complete(int)+0x31c) [0x55adb6de2b0c] 2023-11-09T03:50:42.748 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: (MDSLogContextBase::complete(int)+0x53) [0x55adb6de2c63] 2023-11-09T03:50:42.748 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (Finisher::finisher_thread_entry()+0x175) [0x7fd099a164a5] 2023-11-09T03:50:42.749 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7fd09909f802] 2023-11-09T03:50:42.749 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7fd09903f450] 2023-11-09T03:50:42.749 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:50:42.749 INFO:tasks.ceph.mds.b.smithi113.stderr: 0> 2023-11-09T03:50:42.727+0000 7fd08dd68640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:50:42.749 INFO:tasks.ceph.mds.b.smithi113.stderr: in thread 7fd08dd68640 thread_name:MR_Finisher 2023-11-09T03:50:42.750 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:50:42.750 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:50:42.750 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fd099054db0] 2023-11-09T03:50:42.750 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fd0990a154c] 2023-11-09T03:50:42.750 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: raise() 2023-11-09T03:50:42.750 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: abort() 2023-11-09T03:50:42.751 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fd09996f009] 2023-11-09T03:50:42.751 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fd09996f16d] 2023-11-09T03:50:42.751 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x41b) [0x55adb6daa21b] 2023-11-09T03:50:42.751 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (MDSContext::complete(int)+0x5c) [0x55adb6dd9bcc] 2023-11-09T03:50:42.751 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: (MDSIOContextBase::complete(int)+0x31c) [0x55adb6de2b0c] 2023-11-09T03:50:42.751 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: (MDSLogContextBase::complete(int)+0x53) [0x55adb6de2c63] 2023-11-09T03:50:42.752 INFO:tasks.ceph.mds.b.smithi113.stderr: 11: (Finisher::finisher_thread_entry()+0x175) [0x7fd099a164a5] 2023-11-09T03:50:42.752 INFO:tasks.ceph.mds.b.smithi113.stderr: 12: /lib64/libc.so.6(+0x9f802) [0x7fd09909f802] 2023-11-09T03:50:42.752 INFO:tasks.ceph.mds.b.smithi113.stderr: 13: /lib64/libc.so.6(+0x3f450) [0x7fd09903f450] 2023-11-09T03:50:42.752 INFO:tasks.ceph.mds.b.smithi113.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:50:42.752 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:50:42.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.821+0000 7f803dbfb640 1 Processor -- start 2023-11-09T03:50:42.821 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.821+0000 7f803dbfb640 1 -- start start 2023-11-09T03:50:42.822 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.822+0000 7f803dbfb640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8038159c70 0x7f80380ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:42.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.822+0000 7f803dbfb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80380b0ae0 0x7f803815d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:42.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.822+0000 7f803dbfb640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80380b4110 con 0x7f803806db20 2023-11-09T03:50:42.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.822+0000 7f803dbfb640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80380b49b0 con 0x7f8038159c70 2023-11-09T03:50:42.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.822+0000 7f803dbfb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80380b43f0 con 0x7f80380b0ae0 2023-11-09T03:50:42.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.822+0000 7f80377fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8038159c70 0x7f80380ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:42.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.822+0000 7f8037fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80380b0ae0 0x7f803815d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:42.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.823+0000 7f8036ffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f803806db20 0x7f80380b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:36792/0 (socket says 172.21.15.76:36792) 2023-11-09T03:50:42.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.823+0000 7f8036ffd640 1 -- 172.21.15.76:0/299616173 learned_addr learned my addr 172.21.15.76:0/299616173 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:50:42.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.823+0000 7f8037fff640 1 -- 172.21.15.76:0/299616173 >> v2:172.21.15.113:3300/0 conn(0x7f8038159c70 msgr2=0x7f80380ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:42.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.823+0000 7f8037fff640 1 --2- 172.21.15.76:0/299616173 >> v2:172.21.15.113:3300/0 conn(0x7f8038159c70 0x7f80380ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:42.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.823+0000 7f8037fff640 1 -- 172.21.15.76:0/299616173 >> v1:172.21.15.113:6789/0 conn(0x7f803806db20 legacy=0x7f80380b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:42.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.823+0000 7f8037fff640 1 -- 172.21.15.76:0/299616173 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80380b46d0 con 0x7f80380b0ae0 2023-11-09T03:50:42.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.824+0000 7f8037fff640 1 --2- 172.21.15.76:0/299616173 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80380b0ae0 0x7f803815d7d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8030009d90 tx=0x7f803002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aee3bb5116b158ff server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:42.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.824+0000 7f80367fc640 1 -- 172.21.15.76:0/299616173 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803002cc50 con 0x7f80380b0ae0 2023-11-09T03:50:42.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.824+0000 7f80367fc640 1 -- 172.21.15.76:0/299616173 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803002cdb0 con 0x7f80380b0ae0 2023-11-09T03:50:42.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.824+0000 7f80367fc640 1 -- 172.21.15.76:0/299616173 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030034660 con 0x7f80380b0ae0 2023-11-09T03:50:42.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.824+0000 7f803dbfb640 1 -- 172.21.15.76:0/299616173 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80380b0ae0 msgr2=0x7f803815d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:42.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.824+0000 7f803dbfb640 1 --2- 172.21.15.76:0/299616173 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80380b0ae0 0x7f803815d7d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8030009d90 tx=0x7f803002bea0 comp rx=0 tx=0).stop 2023-11-09T03:50:42.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.824+0000 7f803dbfb640 1 -- 172.21.15.76:0/299616173 shutdown_connections 2023-11-09T03:50:42.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.824+0000 7f803dbfb640 1 --2- 172.21.15.76:0/299616173 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80380b0ae0 0x7f803815d7d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:42.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.824+0000 7f803dbfb640 1 --2- 172.21.15.76:0/299616173 >> v2:172.21.15.113:3300/0 conn(0x7f8038159c70 0x7f80380ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:42.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.824+0000 7f803dbfb640 1 -- 172.21.15.76:0/299616173 >> 172.21.15.76:0/299616173 conn(0x7f80380ce950 msgr2=0x7f80380ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:42.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.824+0000 7f803dbfb640 1 -- 172.21.15.76:0/299616173 shutdown_connections 2023-11-09T03:50:42.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.824+0000 7f803dbfb640 1 -- 172.21.15.76:0/299616173 wait complete. 2023-11-09T03:50:42.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.825+0000 7f803dbfb640 1 Processor -- start 2023-11-09T03:50:42.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.826+0000 7f803dbfb640 1 -- start start 2023-11-09T03:50:42.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.826+0000 7f803dbfb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f803806db20 0x7f8038156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:42.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.826+0000 7f803dbfb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80380b0ae0 0x7f80381584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:42.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.826+0000 7f803dbfb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8038159c70 0x7f8038155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:42.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.826+0000 7f803dbfb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803815bbd0 con 0x7f80380b0ae0 2023-11-09T03:50:42.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.826+0000 7f803dbfb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80381589f0 con 0x7f8038159c70 2023-11-09T03:50:42.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.826+0000 7f803dbfb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803815afc0 con 0x7f803806db20 2023-11-09T03:50:42.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.826+0000 7f8036ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80380b0ae0 0x7f80381584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:42.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.826+0000 7f8036ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80380b0ae0 0x7f80381584b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34926/0 (socket says 172.21.15.76:34926) 2023-11-09T03:50:42.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.826+0000 7f8036ffd640 1 -- 172.21.15.76:0/3104282519 learned_addr learned my addr 172.21.15.76:0/3104282519 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:42.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.827+0000 7f80377fe640 1 --2- 172.21.15.76:0/3104282519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f803806db20 0x7f8038156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:42.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.827+0000 7f8037fff640 1 --2- 172.21.15.76:0/3104282519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8038159c70 0x7f8038155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:42.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.827+0000 7f8036ffd640 1 -- 172.21.15.76:0/3104282519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f803806db20 msgr2=0x7f8038156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:42.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.827+0000 7f8036ffd640 1 --2- 172.21.15.76:0/3104282519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f803806db20 0x7f8038156b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:42.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.827+0000 7f8036ffd640 1 -- 172.21.15.76:0/3104282519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8038159c70 msgr2=0x7f8038155f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:42.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.827+0000 7f8036ffd640 1 --2- 172.21.15.76:0/3104282519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8038159c70 0x7f8038155f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:42.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.827+0000 7f8036ffd640 1 -- 172.21.15.76:0/3104282519 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8030009a70 con 0x7f80380b0ae0 2023-11-09T03:50:42.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.827+0000 7f8037fff640 1 --2- 172.21.15.76:0/3104282519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8038159c70 0x7f8038155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:50:42.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.827+0000 7f8036ffd640 1 --2- 172.21.15.76:0/3104282519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80380b0ae0 0x7f80381584b0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f802c00c4a0 tx=0x7f802c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:42.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.827+0000 7f801ffff640 1 -- 172.21.15.76:0/3104282519 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c005ce0 con 0x7f80380b0ae0 2023-11-09T03:50:42.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.827+0000 7f801ffff640 1 -- 172.21.15.76:0/3104282519 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802c005e40 con 0x7f80380b0ae0 2023-11-09T03:50:42.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.827+0000 7f803dbfb640 1 -- 172.21.15.76:0/3104282519 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803815b2a0 con 0x7f80380b0ae0 2023-11-09T03:50:42.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.827+0000 7f801ffff640 1 -- 172.21.15.76:0/3104282519 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c007270 con 0x7f80380b0ae0 2023-11-09T03:50:42.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.827+0000 7f803dbfb640 1 -- 172.21.15.76:0/3104282519 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f803815b860 con 0x7f80380b0ae0 2023-11-09T03:50:42.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.828+0000 7f803dbfb640 1 -- 172.21.15.76:0/3104282519 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8038074290 con 0x7f80380b0ae0 2023-11-09T03:50:42.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.828+0000 7f801ffff640 1 -- 172.21.15.76:0/3104282519 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f802c007490 con 0x7f80380b0ae0 2023-11-09T03:50:42.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.829+0000 7f801ffff640 1 --2- 172.21.15.76:0/3104282519 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f802406ae70 0x7f802406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:42.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.829+0000 7f801ffff640 1 -- 172.21.15.76:0/3104282519 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7f802c089080 con 0x7f80380b0ae0 2023-11-09T03:50:42.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.829+0000 7f80377fe640 1 --2- 172.21.15.76:0/3104282519 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f802406ae70 0x7f802406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:42.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.829+0000 7f80377fe640 1 --2- 172.21.15.76:0/3104282519 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f802406ae70 0x7f802406d330 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f803815cd50 tx=0x7f8028009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:42.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:42.833+0000 7f801ffff640 1 -- 172.21.15.76:0/3104282519 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f802c053bb0 con 0x7f80380b0ae0 2023-11-09T03:50:42.946 INFO:tasks.ceph.mds.b.smithi113.stderr:daemon-helper: command crashed with signal 6 2023-11-09T03:50:43.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:43.035+0000 7f803dbfb640 1 -- 172.21.15.76:0/3104282519 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f803806d3f0 con 0x7f80380b0ae0 2023-11-09T03:50:43.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:43.036+0000 7f801ffff640 1 -- 172.21.15.76:0/3104282519 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4935 (secure 0 0 0) 0x7f802c017090 con 0x7f80380b0ae0 2023-11-09T03:50:43.036 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:50:43.036 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8016,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3229239133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3229239133},{"type":"v1","addr":"172.21.15.76:6835","nonce":3229239133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:50:41.545080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5993,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5993":{"gid":5993,"name":"b","rank":0,"incarnation":69,"state":"up:active","state_seq":11,"addr":"172.21.15.113:6837/3449910842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3449910842},{"type":"v1","addr":"172.21.15.113:6837","nonce":3449910842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:50:43.036 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 74 2023-11-09T03:50:43.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:43.038+0000 7f803dbfb640 1 -- 172.21.15.76:0/3104282519 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f802406ae70 msgr2=0x7f802406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:43.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:43.038+0000 7f803dbfb640 1 --2- 172.21.15.76:0/3104282519 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f802406ae70 0x7f802406d330 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f803815cd50 tx=0x7f8028009190 comp rx=0 tx=0).stop 2023-11-09T03:50:43.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:43.038+0000 7f803dbfb640 1 -- 172.21.15.76:0/3104282519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80380b0ae0 msgr2=0x7f80381584b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:43.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:43.038+0000 7f803dbfb640 1 --2- 172.21.15.76:0/3104282519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80380b0ae0 0x7f80381584b0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f802c00c4a0 tx=0x7f802c00c960 comp rx=0 tx=0).stop 2023-11-09T03:50:43.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:43.038+0000 7f803dbfb640 1 -- 172.21.15.76:0/3104282519 shutdown_connections 2023-11-09T03:50:43.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:43.038+0000 7f803dbfb640 1 --2- 172.21.15.76:0/3104282519 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f802406ae70 0x7f802406d330 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:43.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:43.038+0000 7f803dbfb640 1 --2- 172.21.15.76:0/3104282519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8038159c70 0x7f8038155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:43.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:43.038+0000 7f803dbfb640 1 --2- 172.21.15.76:0/3104282519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80380b0ae0 0x7f80381584b0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:43.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:43.038+0000 7f803dbfb640 1 --2- 172.21.15.76:0/3104282519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f803806db20 0x7f8038156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:43.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:43.038+0000 7f803dbfb640 1 -- 172.21.15.76:0/3104282519 >> 172.21.15.76:0/3104282519 conn(0x7f80380ce950 msgr2=0x7f80380b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:43.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:43.038+0000 7f803dbfb640 1 -- 172.21.15.76:0/3104282519 shutdown_connections 2023-11-09T03:50:43.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:43.038+0000 7f803dbfb640 1 -- 172.21.15.76:0/3104282519 wait complete. 2023-11-09T03:50:43.048 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:50:45.860 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-11-09T03:50:48.049 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:50:48.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.162+0000 7f67581c9640 1 Processor -- start 2023-11-09T03:50:48.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.163+0000 7f67581c9640 1 -- start start 2023-11-09T03:50:48.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.164+0000 7f67581c9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67500ce370 0x7f675006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:48.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.164+0000 7f67581c9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f675006e060 con 0x7f6750159c70 2023-11-09T03:50:48.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.164+0000 7f67581c9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67500b4110 con 0x7f67500b0ae0 2023-11-09T03:50:48.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.164+0000 7f67581c9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67500b49b0 con 0x7f67500ce370 2023-11-09T03:50:48.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.164+0000 7f6755f65640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f6750159c70 0x7f67500b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:58204/0 (socket says 172.21.15.76:58204) 2023-11-09T03:50:48.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.164+0000 7f6755f65640 1 -- 172.21.15.76:0/3782830913 learned_addr learned my addr 172.21.15.76:0/3782830913 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:50:48.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.164+0000 7f6756766640 1 --2- 172.21.15.76:0/3782830913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67500ce370 0x7f675006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:48.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.164+0000 7f6754f63640 1 -- 172.21.15.76:0/3782830913 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3651888345 0 0) 0x7f675006e060 con 0x7f6750159c70 2023-11-09T03:50:48.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.164+0000 7f6754f63640 1 -- 172.21.15.76:0/3782830913 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6738005730 con 0x7f6750159c70 2023-11-09T03:50:48.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.164+0000 7f6754f63640 1 -- 172.21.15.76:0/3782830913 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1625914086 0 0) 0x7f67500b4110 con 0x7f67500b0ae0 2023-11-09T03:50:48.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.164+0000 7f6754f63640 1 -- 172.21.15.76:0/3782830913 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f675006e060 con 0x7f67500b0ae0 2023-11-09T03:50:48.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.164+0000 7f6754f63640 1 -- 172.21.15.76:0/3782830913 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2303937673 0 0) 0x7f6738005730 con 0x7f6750159c70 2023-11-09T03:50:48.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.164+0000 7f6754f63640 1 -- 172.21.15.76:0/3782830913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67500ce370 msgr2=0x7f675006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:48.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.164+0000 7f6754f63640 1 --2- 172.21.15.76:0/3782830913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67500ce370 0x7f675006db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:48.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.164+0000 7f6754f63640 1 -- 172.21.15.76:0/3782830913 >> v1:172.21.15.113:6789/0 conn(0x7f67500b0ae0 legacy=0x7f675015d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:48.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.164+0000 7f6754f63640 1 -- 172.21.15.76:0/3782830913 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67500b43f0 con 0x7f6750159c70 2023-11-09T03:50:48.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.164+0000 7f6754f63640 1 -- 172.21.15.76:0/3782830913 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f6748001670 con 0x7f6750159c70 2023-11-09T03:50:48.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.164+0000 7f6756766640 1 --2- 172.21.15.76:0/3782830913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67500ce370 0x7f675006db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:50:48.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.164+0000 7f6754f63640 1 -- 172.21.15.76:0/3782830913 >> v1:172.21.15.76:6789/0 conn(0x7f6750159c70 legacy=0x7f67500b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:48.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.164+0000 7f6754f63640 1 --2- 172.21.15.76:0/3782830913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6738005d40 0x7f6738006140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:48.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.164+0000 7f6754f63640 1 --2- 172.21.15.76:0/3782830913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67380069d0 0x7f6738006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:48.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.164+0000 7f6754f63640 1 --2- 172.21.15.76:0/3782830913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67500ce370 0x7f6738007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:48.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.164+0000 7f6754f63640 1 -- 172.21.15.76:0/3782830913 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67500b49b0 con 0x7f67380069d0 2023-11-09T03:50:48.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.164+0000 7f6754f63640 1 -- 172.21.15.76:0/3782830913 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673800dbc0 con 0x7f6738005d40 2023-11-09T03:50:48.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.164+0000 7f6754f63640 1 -- 172.21.15.76:0/3782830913 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f673800dd80 con 0x7f67500ce370 2023-11-09T03:50:48.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.165+0000 7f6755764640 1 --2- 172.21.15.76:0/3782830913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67380069d0 0x7f6738006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:48.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.165+0000 7f6756766640 1 --2- 172.21.15.76:0/3782830913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67500ce370 0x7f6738007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:48.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.165+0000 7f6755f65640 1 --2- 172.21.15.76:0/3782830913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6738005d40 0x7f6738006140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:48.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.165+0000 7f6755764640 1 -- 172.21.15.76:0/3782830913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67500ce370 msgr2=0x7f6738007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:48.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.165+0000 7f6755764640 1 --2- 172.21.15.76:0/3782830913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67500ce370 0x7f6738007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:48.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.165+0000 7f6755764640 1 -- 172.21.15.76:0/3782830913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6738005d40 msgr2=0x7f6738006140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:48.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.165+0000 7f6755764640 1 --2- 172.21.15.76:0/3782830913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6738005d40 0x7f6738006140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:48.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.165+0000 7f6755764640 1 -- 172.21.15.76:0/3782830913 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6740009a70 con 0x7f67380069d0 2023-11-09T03:50:48.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.166+0000 7f6755764640 1 --2- 172.21.15.76:0/3782830913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67380069d0 0x7f6738006e50 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f67440018a0 tx=0x7f6738007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e80e03e0c00cc08f server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:48.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.166+0000 7f6754f63640 1 -- 172.21.15.76:0/3782830913 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67440052d0 con 0x7f67380069d0 2023-11-09T03:50:48.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.166+0000 7f6754f63640 1 -- 172.21.15.76:0/3782830913 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6744005430 con 0x7f67380069d0 2023-11-09T03:50:48.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.166+0000 7f6754f63640 1 -- 172.21.15.76:0/3782830913 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6744003810 con 0x7f67380069d0 2023-11-09T03:50:48.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.166+0000 7f67581c9640 1 -- 172.21.15.76:0/3782830913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67380069d0 msgr2=0x7f6738006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:48.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.166+0000 7f67581c9640 1 --2- 172.21.15.76:0/3782830913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67380069d0 0x7f6738006e50 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f67440018a0 tx=0x7f6738007a00 comp rx=0 tx=0).stop 2023-11-09T03:50:48.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.166+0000 7f6756766640 1 -- 172.21.15.76:0/3782830913 reap_dead start 2023-11-09T03:50:48.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.166+0000 7f67581c9640 1 -- 172.21.15.76:0/3782830913 shutdown_connections 2023-11-09T03:50:48.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.166+0000 7f67581c9640 1 -- 172.21.15.76:0/3782830913 >> 172.21.15.76:0/3782830913 conn(0x7f67500ce950 msgr2=0x7f67500ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:48.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.166+0000 7f67581c9640 1 -- 172.21.15.76:0/3782830913 shutdown_connections 2023-11-09T03:50:48.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.166+0000 7f67581c9640 1 -- 172.21.15.76:0/3782830913 wait complete. 2023-11-09T03:50:48.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.167+0000 7f67581c9640 1 Processor -- start 2023-11-09T03:50:48.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.168+0000 7f67581c9640 1 -- start start 2023-11-09T03:50:48.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.168+0000 7f67581c9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67380069d0 0x7f67500c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:48.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.168+0000 7f67581c9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f675013d740 0x7f67500c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:48.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.168+0000 7f67581c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f675014d4b0 0x7f675014d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:48.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.168+0000 7f67581c9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675015b0a0 con 0x7f675014d4b0 2023-11-09T03:50:48.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.168+0000 7f67581c9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675015b380 con 0x7f675013d740 2023-11-09T03:50:48.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.168+0000 7f67581c9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67500c8540 con 0x7f67380069d0 2023-11-09T03:50:48.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.168+0000 7f6756766640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f675014d4b0 0x7f675014d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:48.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.168+0000 7f6756766640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f675014d4b0 0x7f675014d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34938/0 (socket says 172.21.15.76:34938) 2023-11-09T03:50:48.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.168+0000 7f6756766640 1 -- 172.21.15.76:0/3730203913 learned_addr learned my addr 172.21.15.76:0/3730203913 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:48.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.169+0000 7f6756766640 1 -- 172.21.15.76:0/3730203913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67380069d0 msgr2=0x7f67500c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:48.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.169+0000 7f6755f65640 1 --2- 172.21.15.76:0/3730203913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67380069d0 0x7f67500c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:48.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.169+0000 7f6755764640 1 --2- 172.21.15.76:0/3730203913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f675013d740 0x7f67500c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:48.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.169+0000 7f6756766640 1 --2- 172.21.15.76:0/3730203913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67380069d0 0x7f67500c7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:48.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.169+0000 7f6756766640 1 -- 172.21.15.76:0/3730203913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f675013d740 msgr2=0x7f67500c5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:48.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.169+0000 7f6756766640 1 --2- 172.21.15.76:0/3730203913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f675013d740 0x7f67500c5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:48.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.169+0000 7f6756766640 1 -- 172.21.15.76:0/3730203913 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67500c8820 con 0x7f675014d4b0 2023-11-09T03:50:48.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.169+0000 7f6755764640 1 --2- 172.21.15.76:0/3730203913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f675013d740 0x7f67500c5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:50:48.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.169+0000 7f6756766640 1 --2- 172.21.15.76:0/3730203913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f675014d4b0 0x7f675014d920 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f67500cf6e0 tx=0x7f674000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:48.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.169+0000 7f674e7fc640 1 -- 172.21.15.76:0/3730203913 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674000e040 con 0x7f675014d4b0 2023-11-09T03:50:48.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.169+0000 7f674e7fc640 1 -- 172.21.15.76:0/3730203913 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f674000f040 con 0x7f675014d4b0 2023-11-09T03:50:48.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.169+0000 7f67581c9640 1 -- 172.21.15.76:0/3730203913 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67500c8b00 con 0x7f675014d4b0 2023-11-09T03:50:48.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.169+0000 7f674e7fc640 1 -- 172.21.15.76:0/3730203913 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6740003dd0 con 0x7f675014d4b0 2023-11-09T03:50:48.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.169+0000 7f67581c9640 1 -- 172.21.15.76:0/3730203913 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67500c8130 con 0x7f675014d4b0 2023-11-09T03:50:48.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.170+0000 7f67581c9640 1 -- 172.21.15.76:0/3730203913 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f675015b4f0 con 0x7f675014d4b0 2023-11-09T03:50:48.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.170+0000 7f674e7fc640 1 -- 172.21.15.76:0/3730203913 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6740003f30 con 0x7f675014d4b0 2023-11-09T03:50:48.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.171+0000 7f674e7fc640 1 --2- 172.21.15.76:0/3730203913 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f673c06ae70 0x7f673c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:48.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.171+0000 7f674e7fc640 1 -- 172.21.15.76:0/3730203913 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7f674008a540 con 0x7f675014d4b0 2023-11-09T03:50:48.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.171+0000 7f6755f65640 1 --2- 172.21.15.76:0/3730203913 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f673c06ae70 0x7f673c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:48.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.172+0000 7f6755f65640 1 --2- 172.21.15.76:0/3730203913 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f673c06ae70 0x7f673c06d330 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6748001eb0 tx=0x7f6748009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:48.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.175+0000 7f674e7fc640 1 -- 172.21.15.76:0/3730203913 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6740054fc0 con 0x7f675014d4b0 2023-11-09T03:50:48.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.380+0000 7f67581c9640 1 -- 172.21.15.76:0/3730203913 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6750074290 con 0x7f675014d4b0 2023-11-09T03:50:48.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.380+0000 7f674e7fc640 1 -- 172.21.15.76:0/3730203913 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4935 (secure 0 0 0) 0x7f67400585e0 con 0x7f675014d4b0 2023-11-09T03:50:48.380 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:50:48.380 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8016,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3229239133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3229239133},{"type":"v1","addr":"172.21.15.76:6835","nonce":3229239133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:50:41.545080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5993,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5993":{"gid":5993,"name":"b","rank":0,"incarnation":69,"state":"up:active","state_seq":11,"addr":"172.21.15.113:6837/3449910842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3449910842},{"type":"v1","addr":"172.21.15.113:6837","nonce":3449910842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:50:48.380 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 74 2023-11-09T03:50:48.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.382+0000 7f67581c9640 1 -- 172.21.15.76:0/3730203913 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f673c06ae70 msgr2=0x7f673c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:48.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.382+0000 7f67581c9640 1 --2- 172.21.15.76:0/3730203913 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f673c06ae70 0x7f673c06d330 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6748001eb0 tx=0x7f6748009040 comp rx=0 tx=0).stop 2023-11-09T03:50:48.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.382+0000 7f67581c9640 1 -- 172.21.15.76:0/3730203913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f675014d4b0 msgr2=0x7f675014d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:48.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.382+0000 7f67581c9640 1 --2- 172.21.15.76:0/3730203913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f675014d4b0 0x7f675014d920 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f67500cf6e0 tx=0x7f674000f580 comp rx=0 tx=0).stop 2023-11-09T03:50:48.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.382+0000 7f67581c9640 1 -- 172.21.15.76:0/3730203913 shutdown_connections 2023-11-09T03:50:48.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.382+0000 7f67581c9640 1 --2- 172.21.15.76:0/3730203913 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f673c06ae70 0x7f673c06d330 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:48.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.382+0000 7f67581c9640 1 --2- 172.21.15.76:0/3730203913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f675014d4b0 0x7f675014d920 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:48.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.382+0000 7f67581c9640 1 --2- 172.21.15.76:0/3730203913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f675013d740 0x7f67500c5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:48.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.382+0000 7f67581c9640 1 --2- 172.21.15.76:0/3730203913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67380069d0 0x7f67500c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:48.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.382+0000 7f67581c9640 1 -- 172.21.15.76:0/3730203913 >> 172.21.15.76:0/3730203913 conn(0x7f67500ce950 msgr2=0x7f67500cc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:48.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.382+0000 7f67581c9640 1 -- 172.21.15.76:0/3730203913 shutdown_connections 2023-11-09T03:50:48.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:48.382+0000 7f67581c9640 1 -- 172.21.15.76:0/3730203913 wait complete. 2023-11-09T03:50:48.392 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:50:52.470 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~7s 2023-11-09T03:50:53.394 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:50:53.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.504+0000 7fdfa35a9640 1 Processor -- start 2023-11-09T03:50:53.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.504+0000 7fdfa35a9640 1 -- start start 2023-11-09T03:50:53.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.504+0000 7fdfa35a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf9c0b0ae0 0x7fdf9c070780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:53.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.504+0000 7fdfa35a9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdf9c070cc0 0x7fdf9c06ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:53.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.504+0000 7fdfa35a9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf9c0714b0 con 0x7fdf9c15cfd0 2023-11-09T03:50:53.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.504+0000 7fdfa35a9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf9c0ab8e0 con 0x7fdf9c070cc0 2023-11-09T03:50:53.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.504+0000 7fdfa35a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c071790 con 0x7fdf9c0b0ae0 2023-11-09T03:50:53.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.504+0000 7fdfa1b46640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdf9c070cc0 0x7fdf9c06ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:53.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.505+0000 7fdfa1b46640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdf9c070cc0 0x7fdf9c06ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37236/0 (socket says 172.21.15.76:37236) 2023-11-09T03:50:53.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.505+0000 7fdfa1345640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fdf9c15cfd0 0x7fdf9c0b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36084/0 (socket says 172.21.15.76:36084) 2023-11-09T03:50:53.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.505+0000 7fdfa1b46640 1 -- 172.21.15.76:0/2615214154 learned_addr learned my addr 172.21.15.76:0/2615214154 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:53.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.505+0000 7fdfa0b44640 1 --2- 172.21.15.76:0/2615214154 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf9c0b0ae0 0x7fdf9c070780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:53.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.505+0000 7fdfa1b46640 1 -- 172.21.15.76:0/2615214154 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf9c0b0ae0 msgr2=0x7fdf9c070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:53.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.505+0000 7fdfa1b46640 1 --2- 172.21.15.76:0/2615214154 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf9c0b0ae0 0x7fdf9c070780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:53.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.505+0000 7fdfa1b46640 1 -- 172.21.15.76:0/2615214154 >> v1:172.21.15.76:6789/0 conn(0x7fdf9c15cfd0 legacy=0x7fdf9c0b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:53.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.505+0000 7fdfa1b46640 1 -- 172.21.15.76:0/2615214154 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9c06c690 con 0x7fdf9c070cc0 2023-11-09T03:50:53.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.505+0000 7fdf9bfff640 1 -- 172.21.15.76:0/2615214154 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 562311889 0 0) 0x7fdf9c0714b0 con 0x7fdf9c15cfd0 2023-11-09T03:50:53.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.506+0000 7fdfa1b46640 1 --2- 172.21.15.76:0/2615214154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf9c070cc0 0x7fdf9c06ce90 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c009870 tx=0x7fdf8c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae8cf658970a6ef server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:53.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.506+0000 7fdf9bfff640 1 -- 172.21.15.76:0/2615214154 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c030840 con 0x7fdf9c070cc0 2023-11-09T03:50:53.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.506+0000 7fdf9bfff640 1 -- 172.21.15.76:0/2615214154 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8c0309a0 con 0x7fdf9c070cc0 2023-11-09T03:50:53.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.506+0000 7fdf9bfff640 1 -- 172.21.15.76:0/2615214154 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c030d30 con 0x7fdf9c070cc0 2023-11-09T03:50:53.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.506+0000 7fdfa35a9640 1 -- 172.21.15.76:0/2615214154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf9c070cc0 msgr2=0x7fdf9c06ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:53.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.506+0000 7fdfa35a9640 1 --2- 172.21.15.76:0/2615214154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf9c070cc0 0x7fdf9c06ce90 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c009870 tx=0x7fdf8c02b980 comp rx=0 tx=0).stop 2023-11-09T03:50:53.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.506+0000 7fdfa35a9640 1 -- 172.21.15.76:0/2615214154 shutdown_connections 2023-11-09T03:50:53.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.506+0000 7fdfa35a9640 1 --2- 172.21.15.76:0/2615214154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf9c070cc0 0x7fdf9c06ce90 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:53.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.506+0000 7fdfa35a9640 1 --2- 172.21.15.76:0/2615214154 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf9c0b0ae0 0x7fdf9c070780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:53.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.506+0000 7fdfa35a9640 1 -- 172.21.15.76:0/2615214154 >> 172.21.15.76:0/2615214154 conn(0x7fdf9c0ce950 msgr2=0x7fdf9c0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:53.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.506+0000 7fdfa35a9640 1 -- 172.21.15.76:0/2615214154 shutdown_connections 2023-11-09T03:50:53.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.506+0000 7fdfa35a9640 1 -- 172.21.15.76:0/2615214154 wait complete. 2023-11-09T03:50:53.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.507+0000 7fdfa35a9640 1 Processor -- start 2023-11-09T03:50:53.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.508+0000 7fdfa35a9640 1 -- start start 2023-11-09T03:50:53.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.508+0000 7fdfa35a9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf9c070cc0 0x7fdf9c153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:53.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.508+0000 7fdfa35a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf9c0b0ae0 0x7fdf9c0c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:53.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.508+0000 7fdfa35a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf9c15cfd0 0x7fdf9c0cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:53.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.508+0000 7fdfa35a9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c0cab80 con 0x7fdf9c070cc0 2023-11-09T03:50:53.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.508+0000 7fdfa35a9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c153a90 con 0x7fdf9c0b0ae0 2023-11-09T03:50:53.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.508+0000 7fdfa35a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c0ca620 con 0x7fdf9c15cfd0 2023-11-09T03:50:53.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.508+0000 7fdfa1345640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf9c070cc0 0x7fdf9c153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:53.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.508+0000 7fdfa1345640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf9c070cc0 0x7fdf9c153550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37252/0 (socket says 172.21.15.76:37252) 2023-11-09T03:50:53.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.508+0000 7fdfa1345640 1 -- 172.21.15.76:0/4018328146 learned_addr learned my addr 172.21.15.76:0/4018328146 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:53.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.509+0000 7fdfa1b46640 1 --2- 172.21.15.76:0/4018328146 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf9c15cfd0 0x7fdf9c0cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:53.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.509+0000 7fdfa0b44640 1 --2- 172.21.15.76:0/4018328146 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf9c0b0ae0 0x7fdf9c0c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:53.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.509+0000 7fdfa1345640 1 -- 172.21.15.76:0/4018328146 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf9c15cfd0 msgr2=0x7fdf9c0cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:53.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.509+0000 7fdfa1345640 1 --2- 172.21.15.76:0/4018328146 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf9c15cfd0 0x7fdf9c0cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:53.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.509+0000 7fdfa1345640 1 -- 172.21.15.76:0/4018328146 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf9c0b0ae0 msgr2=0x7fdf9c0c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:53.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.509+0000 7fdfa1345640 1 --2- 172.21.15.76:0/4018328146 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf9c0b0ae0 0x7fdf9c0c9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:53.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.509+0000 7fdfa1345640 1 -- 172.21.15.76:0/4018328146 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf8c0094d0 con 0x7fdf9c070cc0 2023-11-09T03:50:53.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.509+0000 7fdfa0b44640 1 --2- 172.21.15.76:0/4018328146 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf9c0b0ae0 0x7fdf9c0c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:50:53.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.509+0000 7fdfa1345640 1 --2- 172.21.15.76:0/4018328146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf9c070cc0 0x7fdf9c153550 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fdf9400c7c0 tx=0x7fdf9400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:53.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.509+0000 7fdf99ffb640 1 -- 172.21.15.76:0/4018328146 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9400a640 con 0x7fdf9c070cc0 2023-11-09T03:50:53.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.509+0000 7fdf99ffb640 1 -- 172.21.15.76:0/4018328146 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9400a7c0 con 0x7fdf9c070cc0 2023-11-09T03:50:53.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.509+0000 7fdfa35a9640 1 -- 172.21.15.76:0/4018328146 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf9c0cadd0 con 0x7fdf9c070cc0 2023-11-09T03:50:53.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.509+0000 7fdf99ffb640 1 -- 172.21.15.76:0/4018328146 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9400ab70 con 0x7fdf9c070cc0 2023-11-09T03:50:53.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.509+0000 7fdfa35a9640 1 -- 172.21.15.76:0/4018328146 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf9c0cb390 con 0x7fdf9c070cc0 2023-11-09T03:50:53.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.510+0000 7fdfa35a9640 1 -- 172.21.15.76:0/4018328146 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf9c074290 con 0x7fdf9c070cc0 2023-11-09T03:50:53.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.510+0000 7fdf99ffb640 1 -- 172.21.15.76:0/4018328146 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdf9400acd0 con 0x7fdf9c070cc0 2023-11-09T03:50:53.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.511+0000 7fdf99ffb640 1 --2- 172.21.15.76:0/4018328146 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdf8806ae70 0x7fdf8806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:53.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.511+0000 7fdf99ffb640 1 -- 172.21.15.76:0/4018328146 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7fdf94089540 con 0x7fdf9c070cc0 2023-11-09T03:50:53.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.511+0000 7fdfa0b44640 1 --2- 172.21.15.76:0/4018328146 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdf8806ae70 0x7fdf8806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:53.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.511+0000 7fdfa0b44640 1 --2- 172.21.15.76:0/4018328146 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdf8806ae70 0x7fdf8806d330 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c0cc830 tx=0x7fdf90008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:53.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.515+0000 7fdf99ffb640 1 -- 172.21.15.76:0/4018328146 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdf94053f30 con 0x7fdf9c070cc0 2023-11-09T03:50:53.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.717+0000 7fdfa35a9640 1 -- 172.21.15.76:0/4018328146 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf9c06c2d0 con 0x7fdf9c070cc0 2023-11-09T03:50:53.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.718+0000 7fdf99ffb640 1 -- 172.21.15.76:0/4018328146 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4935 (secure 0 0 0) 0x7fdf94057550 con 0x7fdf9c070cc0 2023-11-09T03:50:53.719 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:50:53.720 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8016,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3229239133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3229239133},{"type":"v1","addr":"172.21.15.76:6835","nonce":3229239133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:50:41.545080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5993,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5993":{"gid":5993,"name":"b","rank":0,"incarnation":69,"state":"up:active","state_seq":11,"addr":"172.21.15.113:6837/3449910842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3449910842},{"type":"v1","addr":"172.21.15.113:6837","nonce":3449910842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:50:53.721 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 74 2023-11-09T03:50:53.721 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.720+0000 7fdfa35a9640 1 -- 172.21.15.76:0/4018328146 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdf8806ae70 msgr2=0x7fdf8806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:53.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.720+0000 7fdfa35a9640 1 --2- 172.21.15.76:0/4018328146 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdf8806ae70 0x7fdf8806d330 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c0cc830 tx=0x7fdf90008040 comp rx=0 tx=0).stop 2023-11-09T03:50:53.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.720+0000 7fdfa35a9640 1 -- 172.21.15.76:0/4018328146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf9c070cc0 msgr2=0x7fdf9c153550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:53.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.720+0000 7fdfa35a9640 1 --2- 172.21.15.76:0/4018328146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf9c070cc0 0x7fdf9c153550 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fdf9400c7c0 tx=0x7fdf9400cc80 comp rx=0 tx=0).stop 2023-11-09T03:50:53.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.720+0000 7fdfa35a9640 1 -- 172.21.15.76:0/4018328146 shutdown_connections 2023-11-09T03:50:53.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.720+0000 7fdfa35a9640 1 --2- 172.21.15.76:0/4018328146 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdf8806ae70 0x7fdf8806d330 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:53.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.720+0000 7fdfa35a9640 1 --2- 172.21.15.76:0/4018328146 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf9c15cfd0 0x7fdf9c0cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:53.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.720+0000 7fdfa35a9640 1 --2- 172.21.15.76:0/4018328146 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf9c0b0ae0 0x7fdf9c0c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:53.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.720+0000 7fdfa35a9640 1 --2- 172.21.15.76:0/4018328146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf9c070cc0 0x7fdf9c153550 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:53.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.720+0000 7fdfa35a9640 1 -- 172.21.15.76:0/4018328146 >> 172.21.15.76:0/4018328146 conn(0x7fdf9c0ce950 msgr2=0x7fdf9c1612f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:53.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.720+0000 7fdfa35a9640 1 -- 172.21.15.76:0/4018328146 shutdown_connections 2023-11-09T03:50:53.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:53.720+0000 7fdfa35a9640 1 -- 172.21.15.76:0/4018328146 wait complete. 2023-11-09T03:50:53.731 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:50:58.732 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:50:58.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.840+0000 7fe05f1a5640 1 Processor -- start 2023-11-09T03:50:58.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.841+0000 7fe05f1a5640 1 -- start start 2023-11-09T03:50:58.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.841+0000 7fe05f1a5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0580cde60 0x7fe05815d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:58.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.841+0000 7fe05f1a5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe05815daa0 0x7fe058157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:58.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.841+0000 7fe05f1a5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe05815e000 con 0x7fe0580ce240 2023-11-09T03:50:58.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.841+0000 7fe05f1a5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe05815e5c0 con 0x7fe05815daa0 2023-11-09T03:50:58.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.841+0000 7fe05f1a5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe05815e2e0 con 0x7fe0580cde60 2023-11-09T03:50:58.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.841+0000 7fe057fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe05815daa0 0x7fe058157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:58.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.841+0000 7fe057fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe05815daa0 0x7fe058157000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37264/0 (socket says 172.21.15.76:37264) 2023-11-09T03:50:58.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.842+0000 7fe057fff640 1 -- 172.21.15.76:0/859953567 learned_addr learned my addr 172.21.15.76:0/859953567 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:58.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.842+0000 7fe05cf41640 1 --2- 172.21.15.76:0/859953567 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0580cde60 0x7fe05815d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:58.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.842+0000 7fe057fff640 1 -- 172.21.15.76:0/859953567 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0580cde60 msgr2=0x7fe05815d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:58.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.842+0000 7fe057fff640 1 --2- 172.21.15.76:0/859953567 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0580cde60 0x7fe05815d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:58.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.842+0000 7fe057fff640 1 -- 172.21.15.76:0/859953567 >> v1:172.21.15.113:6789/0 conn(0x7fe0580ce240 legacy=0x7fe05806b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:58.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.842+0000 7fe057fff640 1 -- 172.21.15.76:0/859953567 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0580d1120 con 0x7fe05815daa0 2023-11-09T03:50:58.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.843+0000 7fe057fff640 1 --2- 172.21.15.76:0/859953567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe05815daa0 0x7fe058157000 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009870 tx=0x7fe04c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84db4e66a0af7f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:58.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.843+0000 7fe0577fe640 1 -- 172.21.15.76:0/859953567 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c030840 con 0x7fe05815daa0 2023-11-09T03:50:58.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.843+0000 7fe0577fe640 1 -- 172.21.15.76:0/859953567 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04c0309a0 con 0x7fe05815daa0 2023-11-09T03:50:58.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.843+0000 7fe0577fe640 1 -- 172.21.15.76:0/859953567 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c030d30 con 0x7fe05815daa0 2023-11-09T03:50:58.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.843+0000 7fe05f1a5640 1 -- 172.21.15.76:0/859953567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe05815daa0 msgr2=0x7fe058157000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:58.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.843+0000 7fe05f1a5640 1 --2- 172.21.15.76:0/859953567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe05815daa0 0x7fe058157000 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009870 tx=0x7fe04c02b980 comp rx=0 tx=0).stop 2023-11-09T03:50:58.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.843+0000 7fe05f1a5640 1 -- 172.21.15.76:0/859953567 shutdown_connections 2023-11-09T03:50:58.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.843+0000 7fe05f1a5640 1 --2- 172.21.15.76:0/859953567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe05815daa0 0x7fe058157000 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:58.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.843+0000 7fe05f1a5640 1 --2- 172.21.15.76:0/859953567 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0580cde60 0x7fe05815d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:58.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.843+0000 7fe05f1a5640 1 -- 172.21.15.76:0/859953567 >> 172.21.15.76:0/859953567 conn(0x7fe058071f30 msgr2=0x7fe0580cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:58.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.843+0000 7fe05f1a5640 1 -- 172.21.15.76:0/859953567 shutdown_connections 2023-11-09T03:50:58.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.843+0000 7fe05f1a5640 1 -- 172.21.15.76:0/859953567 wait complete. 2023-11-09T03:50:58.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.844+0000 7fe05f1a5640 1 Processor -- start 2023-11-09T03:50:58.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.844+0000 7fe05f1a5640 1 -- start start 2023-11-09T03:50:58.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.845+0000 7fe05f1a5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0580cde60 0x7fe0580cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:58.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.845+0000 7fe05f1a5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0580ce240 0x7fe0581575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:58.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.845+0000 7fe05f1a5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe05815daa0 0x7fe0581548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:58.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.845+0000 7fe05f1a5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0580cbfd0 con 0x7fe0580cde60 2023-11-09T03:50:58.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.845+0000 7fe05f1a5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0580cc2b0 con 0x7fe05815daa0 2023-11-09T03:50:58.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.845+0000 7fe05f1a5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0580cc590 con 0x7fe0580ce240 2023-11-09T03:50:58.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.845+0000 7fe05cf41640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0580cde60 0x7fe0580cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:58.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.845+0000 7fe05cf41640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0580cde60 0x7fe0580cd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37280/0 (socket says 172.21.15.76:37280) 2023-11-09T03:50:58.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.845+0000 7fe05cf41640 1 -- 172.21.15.76:0/1190978226 learned_addr learned my addr 172.21.15.76:0/1190978226 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:58.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.846+0000 7fe05d742640 1 --2- 172.21.15.76:0/1190978226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe05815daa0 0x7fe0581548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:58.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.846+0000 7fe057fff640 1 --2- 172.21.15.76:0/1190978226 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0580ce240 0x7fe0581575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:58.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.846+0000 7fe05cf41640 1 -- 172.21.15.76:0/1190978226 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0580ce240 msgr2=0x7fe0581575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:58.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.846+0000 7fe05cf41640 1 --2- 172.21.15.76:0/1190978226 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0580ce240 0x7fe0581575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:58.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.846+0000 7fe05cf41640 1 -- 172.21.15.76:0/1190978226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe05815daa0 msgr2=0x7fe0581548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:58.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.846+0000 7fe05cf41640 1 --2- 172.21.15.76:0/1190978226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe05815daa0 0x7fe0581548d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:58.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.846+0000 7fe05cf41640 1 -- 172.21.15.76:0/1190978226 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04c0094d0 con 0x7fe0580cde60 2023-11-09T03:50:58.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.846+0000 7fe057fff640 1 --2- 172.21.15.76:0/1190978226 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0580ce240 0x7fe0581575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:50:58.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.846+0000 7fe05d742640 1 --2- 172.21.15.76:0/1190978226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe05815daa0 0x7fe0581548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:50:58.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.846+0000 7fe05cf41640 1 --2- 172.21.15.76:0/1190978226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0580cde60 0x7fe0580cd070 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fe04800d800 tx=0x7fe04800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:58.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.846+0000 7fe0557fa640 1 -- 172.21.15.76:0/1190978226 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04800b6c0 con 0x7fe0580cde60 2023-11-09T03:50:58.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.846+0000 7fe0557fa640 1 -- 172.21.15.76:0/1190978226 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04800b820 con 0x7fe0580cde60 2023-11-09T03:50:58.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.846+0000 7fe05f1a5640 1 -- 172.21.15.76:0/1190978226 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0580cc870 con 0x7fe0580cde60 2023-11-09T03:50:58.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.846+0000 7fe0557fa640 1 -- 172.21.15.76:0/1190978226 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04800bbb0 con 0x7fe0580cde60 2023-11-09T03:50:58.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.846+0000 7fe05f1a5640 1 -- 172.21.15.76:0/1190978226 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0580cd5b0 con 0x7fe0580cde60 2023-11-09T03:50:58.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.847+0000 7fe05f1a5640 1 -- 172.21.15.76:0/1190978226 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe058074290 con 0x7fe0580cde60 2023-11-09T03:50:58.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.848+0000 7fe0557fa640 1 -- 172.21.15.76:0/1190978226 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe04800bd10 con 0x7fe0580cde60 2023-11-09T03:50:58.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.848+0000 7fe0557fa640 1 --2- 172.21.15.76:0/1190978226 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe04406ae70 0x7fe04406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:58.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.848+0000 7fe0557fa640 1 -- 172.21.15.76:0/1190978226 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7fe04808a1d0 con 0x7fe0580cde60 2023-11-09T03:50:58.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.848+0000 7fe057fff640 1 --2- 172.21.15.76:0/1190978226 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe04406ae70 0x7fe04406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:58.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.849+0000 7fe057fff640 1 --2- 172.21.15.76:0/1190978226 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe04406ae70 0x7fe04406d330 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe04c036000 tx=0x7fe04c03b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:58.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:58.853+0000 7fe0557fa640 1 -- 172.21.15.76:0/1190978226 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe048054c50 con 0x7fe0580cde60 2023-11-09T03:50:59.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.053+0000 7fe05f1a5640 1 -- 172.21.15.76:0/1190978226 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0580c66f0 con 0x7fe0580cde60 2023-11-09T03:50:59.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.054+0000 7fe0557fa640 1 -- 172.21.15.76:0/1190978226 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4983 (secure 0 0 0) 0x7fe048058270 con 0x7fe0580cde60 2023-11-09T03:50:59.055 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:50:59.055 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":75,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8016,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3229239133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3229239133},{"type":"v1","addr":"172.21.15.76:6835","nonce":3229239133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:50:56.443642+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5993,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5993":{"gid":5993,"name":"b","rank":0,"incarnation":69,"state":"up:active","state_seq":11,"addr":"172.21.15.113:6837/3449910842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3449910842},{"type":"v1","addr":"172.21.15.113:6837","nonce":3449910842}]},"join_fscid":-1,"laggy_since":"2023-11-09T03:50:56.443642+0000","export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:50:59.055 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 75 2023-11-09T03:50:59.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.056+0000 7fe05f1a5640 1 -- 172.21.15.76:0/1190978226 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe04406ae70 msgr2=0x7fe04406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:59.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.056+0000 7fe05f1a5640 1 --2- 172.21.15.76:0/1190978226 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe04406ae70 0x7fe04406d330 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe04c036000 tx=0x7fe04c03b040 comp rx=0 tx=0).stop 2023-11-09T03:50:59.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.056+0000 7fe05f1a5640 1 -- 172.21.15.76:0/1190978226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0580cde60 msgr2=0x7fe0580cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:59.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.056+0000 7fe05f1a5640 1 --2- 172.21.15.76:0/1190978226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0580cde60 0x7fe0580cd070 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fe04800d800 tx=0x7fe04800dcc0 comp rx=0 tx=0).stop 2023-11-09T03:50:59.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.056+0000 7fe05f1a5640 1 -- 172.21.15.76:0/1190978226 shutdown_connections 2023-11-09T03:50:59.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.056+0000 7fe05f1a5640 1 --2- 172.21.15.76:0/1190978226 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe04406ae70 0x7fe04406d330 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:59.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.056+0000 7fe05f1a5640 1 --2- 172.21.15.76:0/1190978226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe05815daa0 0x7fe0581548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:59.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.056+0000 7fe05f1a5640 1 --2- 172.21.15.76:0/1190978226 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0580ce240 0x7fe0581575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:59.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.056+0000 7fe05f1a5640 1 --2- 172.21.15.76:0/1190978226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0580cde60 0x7fe0580cd070 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:59.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.056+0000 7fe05f1a5640 1 -- 172.21.15.76:0/1190978226 >> 172.21.15.76:0/1190978226 conn(0x7fe058071f30 msgr2=0x7fe0580b6050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:59.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.056+0000 7fe05f1a5640 1 -- 172.21.15.76:0/1190978226 shutdown_connections 2023-11-09T03:50:59.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.056+0000 7fe05f1a5640 1 -- 172.21.15.76:0/1190978226 wait complete. 2023-11-09T03:50:59.067 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2023-11-09T03:50:59.068 DEBUG:teuthology.orchestra.run.smithi113:> sudo sysctl -n kernel.core_pattern 2023-11-09T03:50:59.081 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~13s 2023-11-09T03:50:59.099 INFO:teuthology.orchestra.run.smithi113.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-09T03:50:59.101 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2023-11-09T03:50:59.102 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2023-11-09T03:50:59.174 INFO:teuthology.orchestra.run.smithi113.stdout:1699501842.95779.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i b', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2023-11-09T03:50:59.176 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1699501842.95779.core'] 2023-11-09T03:50:59.177 INFO:tasks.cephfs.cephfs_test_case:Found core file 1699501842.95779.core, deleting it 2023-11-09T03:50:59.177 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1699501842.95779.core 2023-11-09T03:50:59.247 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:0 2023-11-09T03:50:59.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.352+0000 7f3980ff5640 1 Processor -- start 2023-11-09T03:50:59.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.352+0000 7f3980ff5640 1 -- start start 2023-11-09T03:50:59.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.352+0000 7f3980ff5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f397c0d0710 0x7f397c0cd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:59.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.352+0000 7f3980ff5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f397c0d0280 con 0x7f397c15e430 2023-11-09T03:50:59.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.352+0000 7f3980ff5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f397c0cf670 con 0x7f397c0cda50 2023-11-09T03:50:59.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.352+0000 7f3980ff5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f397c0cf950 con 0x7f397c0d0710 2023-11-09T03:50:59.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.353+0000 7f3973fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f397c0d0710 0x7f397c0cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:59.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.353+0000 7f397a59c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f397c15e430 0x7f397c0cec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36086/0 (socket says 172.21.15.76:36086) 2023-11-09T03:50:59.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.353+0000 7f3973fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f397c0d0710 0x7f397c0cd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37292/0 (socket says 172.21.15.76:37292) 2023-11-09T03:50:59.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.353+0000 7f397a59c640 1 -- 172.21.15.76:0/1544549689 learned_addr learned my addr 172.21.15.76:0/1544549689 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:50:59.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.353+0000 7f3973fff640 1 -- 172.21.15.76:0/1544549689 >> v1:172.21.15.113:6789/0 conn(0x7f397c0cda50 legacy=0x7f397c0b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:59.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.353+0000 7f3973fff640 1 -- 172.21.15.76:0/1544549689 >> v1:172.21.15.76:6789/0 conn(0x7f397c15e430 legacy=0x7f397c0cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:59.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.353+0000 7f3979d9b640 1 -- 172.21.15.76:0/1544549689 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1505181491 0 0) 0x7f397c0d0280 con 0x7f397c15e430 2023-11-09T03:50:59.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.353+0000 7f3973fff640 1 -- 172.21.15.76:0/1544549689 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397c0cfc30 con 0x7f397c0d0710 2023-11-09T03:50:59.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.354+0000 7f3973fff640 1 --2- 172.21.15.76:0/1544549689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f397c0d0710 0x7f397c0cd510 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f39640097a0 tx=0x7f396402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ab764f231bfb3b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:59.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.354+0000 7f3979d9b640 1 -- 172.21.15.76:0/1544549689 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3964031830 con 0x7f397c0d0710 2023-11-09T03:50:59.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.354+0000 7f3979d9b640 1 -- 172.21.15.76:0/1544549689 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3964031990 con 0x7f397c0d0710 2023-11-09T03:50:59.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.354+0000 7f3979d9b640 1 -- 172.21.15.76:0/1544549689 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3964031d20 con 0x7f397c0d0710 2023-11-09T03:50:59.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.354+0000 7f3980ff5640 1 -- 172.21.15.76:0/1544549689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f397c0d0710 msgr2=0x7f397c0cd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:59.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.354+0000 7f3980ff5640 1 --2- 172.21.15.76:0/1544549689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f397c0d0710 0x7f397c0cd510 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f39640097a0 tx=0x7f396402b8d0 comp rx=0 tx=0).stop 2023-11-09T03:50:59.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.354+0000 7f3980ff5640 1 -- 172.21.15.76:0/1544549689 shutdown_connections 2023-11-09T03:50:59.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.354+0000 7f3980ff5640 1 --2- 172.21.15.76:0/1544549689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f397c0d0710 0x7f397c0cd510 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:59.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.354+0000 7f3980ff5640 1 -- 172.21.15.76:0/1544549689 >> 172.21.15.76:0/1544549689 conn(0x7f397c06ea20 msgr2=0x7f397c06ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:59.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.354+0000 7f3980ff5640 1 -- 172.21.15.76:0/1544549689 shutdown_connections 2023-11-09T03:50:59.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.354+0000 7f3980ff5640 1 -- 172.21.15.76:0/1544549689 wait complete. 2023-11-09T03:50:59.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.356+0000 7f3980ff5640 1 Processor -- start 2023-11-09T03:50:59.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.356+0000 7f3980ff5640 1 -- start start 2023-11-09T03:50:59.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.356+0000 7f3980ff5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f397c0cda50 0x7f397c1524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:59.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.356+0000 7f3980ff5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f397c0d0710 0x7f397c15bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:59.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.356+0000 7f3980ff5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f397c15e430 0x7f397c15c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:59.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.356+0000 7f3980ff5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397c1f3da0 con 0x7f397c0cda50 2023-11-09T03:50:59.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.356+0000 7f3980ff5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397c1f4080 con 0x7f397c15e430 2023-11-09T03:50:59.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.356+0000 7f3980ff5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f397c1f4360 con 0x7f397c0d0710 2023-11-09T03:50:59.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.357+0000 7f397a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f397c0cda50 0x7f397c1524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:59.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.357+0000 7f397a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f397c0cda50 0x7f397c1524a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37294/0 (socket says 172.21.15.76:37294) 2023-11-09T03:50:59.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.357+0000 7f397a59c640 1 -- 172.21.15.76:0/1867132754 learned_addr learned my addr 172.21.15.76:0/1867132754 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:59.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.357+0000 7f3973fff640 1 --2- 172.21.15.76:0/1867132754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f397c0d0710 0x7f397c15bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:59.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.357+0000 7f397ad9d640 1 --2- 172.21.15.76:0/1867132754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f397c15e430 0x7f397c15c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:59.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.357+0000 7f397a59c640 1 -- 172.21.15.76:0/1867132754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f397c0d0710 msgr2=0x7f397c15bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:59.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.357+0000 7f397a59c640 1 --2- 172.21.15.76:0/1867132754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f397c0d0710 0x7f397c15bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:59.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.357+0000 7f397a59c640 1 -- 172.21.15.76:0/1867132754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f397c15e430 msgr2=0x7f397c15c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:59.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.357+0000 7f397a59c640 1 --2- 172.21.15.76:0/1867132754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f397c15e430 0x7f397c15c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:59.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.357+0000 7f397a59c640 1 -- 172.21.15.76:0/1867132754 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3964009400 con 0x7f397c0cda50 2023-11-09T03:50:59.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.357+0000 7f397a59c640 1 --2- 172.21.15.76:0/1867132754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f397c0cda50 0x7f397c1524a0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f396c00c4d0 tx=0x7f396c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:59.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.357+0000 7f3972ffd640 1 -- 172.21.15.76:0/1867132754 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c0074e0 con 0x7f397c0cda50 2023-11-09T03:50:59.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.357+0000 7f3972ffd640 1 -- 172.21.15.76:0/1867132754 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f396c007660 con 0x7f397c0cda50 2023-11-09T03:50:59.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.357+0000 7f3972ffd640 1 -- 172.21.15.76:0/1867132754 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c004a20 con 0x7f397c0cda50 2023-11-09T03:50:59.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.357+0000 7f3980ff5640 1 -- 172.21.15.76:0/1867132754 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f397c1f4640 con 0x7f397c0cda50 2023-11-09T03:50:59.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.357+0000 7f3980ff5640 1 -- 172.21.15.76:0/1867132754 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f397c156290 con 0x7f397c0cda50 2023-11-09T03:50:59.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.358+0000 7f3980ff5640 1 -- 172.21.15.76:0/1867132754 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f397c074290 con 0x7f397c0cda50 2023-11-09T03:50:59.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.359+0000 7f3972ffd640 1 -- 172.21.15.76:0/1867132754 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f396c004b80 con 0x7f397c0cda50 2023-11-09T03:50:59.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.359+0000 7f3972ffd640 1 --2- 172.21.15.76:0/1867132754 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f395c06ae70 0x7f395c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:59.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.359+0000 7f3972ffd640 1 -- 172.21.15.76:0/1867132754 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7f396c088de0 con 0x7f397c0cda50 2023-11-09T03:50:59.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.359+0000 7f3973fff640 1 --2- 172.21.15.76:0/1867132754 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f395c06ae70 0x7f395c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:59.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.360+0000 7f3973fff640 1 --2- 172.21.15.76:0/1867132754 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f395c06ae70 0x7f395c06d330 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3964036000 tx=0x7f396403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:59.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.364+0000 7f3972ffd640 1 -- 172.21.15.76:0/1867132754 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f396c053860 con 0x7f397c0cda50 2023-11-09T03:50:59.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.609+0000 7f3980ff5640 1 -- 172.21.15.76:0/1867132754 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:0"} v 0) v1 -- 0x7f397c1f4c90 con 0x7f397c0cda50 2023-11-09T03:50:59.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.611+0000 7f3972ffd640 1 -- 172.21.15.76:0/1867132754 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:0"}]=0 failed mds gid 5993 v76) v1 ==== 97+0+0 (secure 0 0 0) 0x7f397c1f4c90 con 0x7f397c0cda50 2023-11-09T03:50:59.612 INFO:teuthology.orchestra.run.smithi076.stderr:failed mds gid 5993 2023-11-09T03:50:59.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.614+0000 7f3980ff5640 1 -- 172.21.15.76:0/1867132754 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f395c06ae70 msgr2=0x7f395c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:59.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.614+0000 7f3980ff5640 1 --2- 172.21.15.76:0/1867132754 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f395c06ae70 0x7f395c06d330 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3964036000 tx=0x7f396403c040 comp rx=0 tx=0).stop 2023-11-09T03:50:59.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.614+0000 7f3980ff5640 1 -- 172.21.15.76:0/1867132754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f397c0cda50 msgr2=0x7f397c1524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:59.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.614+0000 7f3980ff5640 1 --2- 172.21.15.76:0/1867132754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f397c0cda50 0x7f397c1524a0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f396c00c4d0 tx=0x7f396c00c990 comp rx=0 tx=0).stop 2023-11-09T03:50:59.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.614+0000 7f3980ff5640 1 -- 172.21.15.76:0/1867132754 shutdown_connections 2023-11-09T03:50:59.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.614+0000 7f3980ff5640 1 --2- 172.21.15.76:0/1867132754 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f395c06ae70 0x7f395c06d330 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:59.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.614+0000 7f3980ff5640 1 --2- 172.21.15.76:0/1867132754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f397c15e430 0x7f397c15c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:59.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.614+0000 7f3980ff5640 1 --2- 172.21.15.76:0/1867132754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f397c0d0710 0x7f397c15bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:59.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.614+0000 7f3980ff5640 1 --2- 172.21.15.76:0/1867132754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f397c0cda50 0x7f397c1524a0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:59.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.614+0000 7f3980ff5640 1 -- 172.21.15.76:0/1867132754 >> 172.21.15.76:0/1867132754 conn(0x7f397c06ea20 msgr2=0x7f397c0c6680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:59.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.614+0000 7f3980ff5640 1 -- 172.21.15.76:0/1867132754 shutdown_connections 2023-11-09T03:50:59.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.614+0000 7f3980ff5640 1 -- 172.21.15.76:0/1867132754 wait complete. 2023-11-09T03:50:59.630 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-09T03:50:59.631 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-09T03:50:59.631 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-09T03:50:59.631 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:50:59.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:50:59.632 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi113 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b' 2023-11-09T03:50:59.633 INFO:tasks.ceph.mds.b:Stopped 2023-11-09T03:50:59.633 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-09T03:50:59.637 INFO:tasks.ceph.mds.b:Started 2023-11-09T03:50:59.637 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:50:59.717 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2023-11-09T03:50:59.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.751+0000 7f441332c640 1 Processor -- start 2023-11-09T03:50:59.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.751+0000 7f441332c640 1 -- start start 2023-11-09T03:50:59.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.752+0000 7f441332c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f440c0cde60 0x7f440c15d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:59.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.752+0000 7f441332c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f440c15daa0 0x7f440c157000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:59.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.752+0000 7f441332c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f440c0ce240 0x7f440c157540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:59.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.752+0000 7f441332c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f440c15e5c0 con 0x7f440c15daa0 2023-11-09T03:50:59.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.752+0000 7f441332c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f440c157a80 con 0x7f440c0cde60 2023-11-09T03:50:59.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.752+0000 7f441332c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440c15e000 con 0x7f440c0ce240 2023-11-09T03:50:59.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.752+0000 7f44108c7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f440c15daa0 0x7f440c157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:59.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.752+0000 7f44108c7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f440c15daa0 0x7f440c157000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37300/0 (socket says 172.21.15.76:37300) 2023-11-09T03:50:59.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.752+0000 7f44108c7640 1 -- 172.21.15.76:0/738435979 learned_addr learned my addr 172.21.15.76:0/738435979 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:59.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.752+0000 7f44110c8640 1 --2- 172.21.15.76:0/738435979 >> v2:172.21.15.113:3300/0 conn(0x7f440c0cde60 0x7f440c15d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:59.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.752+0000 7f44108c7640 1 -- 172.21.15.76:0/738435979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f440c0ce240 msgr2=0x7f440c157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:59.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.752+0000 7f44118c9640 1 --2- 172.21.15.76:0/738435979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f440c0ce240 0x7f440c157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:59.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.752+0000 7f44108c7640 1 --2- 172.21.15.76:0/738435979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f440c0ce240 0x7f440c157540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:59.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.752+0000 7f44108c7640 1 -- 172.21.15.76:0/738435979 >> v2:172.21.15.113:3300/0 conn(0x7f440c0cde60 msgr2=0x7f440c15d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:50:59.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.752+0000 7f44108c7640 1 --2- 172.21.15.76:0/738435979 >> v2:172.21.15.113:3300/0 conn(0x7f440c0cde60 0x7f440c15d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:59.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.752+0000 7f44108c7640 1 -- 172.21.15.76:0/738435979 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440c15e2e0 con 0x7f440c15daa0 2023-11-09T03:50:59.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.752+0000 7f44110c8640 1 --2- 172.21.15.76:0/738435979 >> v2:172.21.15.113:3300/0 conn(0x7f440c0cde60 0x7f440c15d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:50:59.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.753+0000 7f44108c7640 1 --2- 172.21.15.76:0/738435979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f440c15daa0 0x7f440c157000 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f440c0b5550 tx=0x7f43fc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22316627a98e644a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:59.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.753+0000 7f4403fff640 1 -- 172.21.15.76:0/738435979 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc031870 con 0x7f440c15daa0 2023-11-09T03:50:59.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.753+0000 7f4403fff640 1 -- 172.21.15.76:0/738435979 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43fc0319d0 con 0x7f440c15daa0 2023-11-09T03:50:59.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.753+0000 7f4403fff640 1 -- 172.21.15.76:0/738435979 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc031d60 con 0x7f440c15daa0 2023-11-09T03:50:59.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.754+0000 7f441332c640 1 -- 172.21.15.76:0/738435979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f440c15daa0 msgr2=0x7f440c157000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:59.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.754+0000 7f441332c640 1 --2- 172.21.15.76:0/738435979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f440c15daa0 0x7f440c157000 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f440c0b5550 tx=0x7f43fc02b910 comp rx=0 tx=0).stop 2023-11-09T03:50:59.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.754+0000 7f441332c640 1 -- 172.21.15.76:0/738435979 shutdown_connections 2023-11-09T03:50:59.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.754+0000 7f441332c640 1 --2- 172.21.15.76:0/738435979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f440c0ce240 0x7f440c157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:59.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.754+0000 7f441332c640 1 --2- 172.21.15.76:0/738435979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f440c15daa0 0x7f440c157000 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:59.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.754+0000 7f441332c640 1 --2- 172.21.15.76:0/738435979 >> v2:172.21.15.113:3300/0 conn(0x7f440c0cde60 0x7f440c15d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:59.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.754+0000 7f441332c640 1 -- 172.21.15.76:0/738435979 >> 172.21.15.76:0/738435979 conn(0x7f440c071f30 msgr2=0x7f440c0cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:59.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.754+0000 7f441332c640 1 -- 172.21.15.76:0/738435979 shutdown_connections 2023-11-09T03:50:59.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.754+0000 7f441332c640 1 -- 172.21.15.76:0/738435979 wait complete. 2023-11-09T03:50:59.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.756+0000 7f441332c640 1 Processor -- start 2023-11-09T03:50:59.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.756+0000 7f441332c640 1 -- start start 2023-11-09T03:50:59.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.757+0000 7f441332c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f440c0cde60 0x7f440c152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:59.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.757+0000 7f441332c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f440c0ce240 0x7f440c15a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:59.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.757+0000 7f441332c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f440c15daa0 0x7f440c15ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:59.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.757+0000 7f441332c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c0cc900 con 0x7f440c15daa0 2023-11-09T03:50:59.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.757+0000 7f441332c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c0cbcf0 con 0x7f440c0ce240 2023-11-09T03:50:59.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.757+0000 7f441332c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440c0cbfd0 con 0x7f440c0cde60 2023-11-09T03:50:59.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.757+0000 7f44118c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f440c15daa0 0x7f440c15ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:59.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.757+0000 7f44118c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f440c15daa0 0x7f440c15ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37310/0 (socket says 172.21.15.76:37310) 2023-11-09T03:50:59.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.757+0000 7f44118c9640 1 -- 172.21.15.76:0/2837876332 learned_addr learned my addr 172.21.15.76:0/2837876332 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:50:59.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.757+0000 7f44110c8640 1 --2- 172.21.15.76:0/2837876332 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f440c0cde60 0x7f440c152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:59.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.757+0000 7f44108c7640 1 --2- 172.21.15.76:0/2837876332 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f440c0ce240 0x7f440c15a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:59.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.757+0000 7f44118c9640 1 -- 172.21.15.76:0/2837876332 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f440c0cde60 msgr2=0x7f440c152770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:59.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.757+0000 7f44118c9640 1 --2- 172.21.15.76:0/2837876332 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f440c0cde60 0x7f440c152770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:59.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.757+0000 7f44118c9640 1 -- 172.21.15.76:0/2837876332 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f440c0ce240 msgr2=0x7f440c15a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:59.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.757+0000 7f44118c9640 1 --2- 172.21.15.76:0/2837876332 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f440c0ce240 0x7f440c15a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:59.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.757+0000 7f44118c9640 1 -- 172.21.15.76:0/2837876332 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43fc0094d0 con 0x7f440c15daa0 2023-11-09T03:50:59.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.757+0000 7f44108c7640 1 --2- 172.21.15.76:0/2837876332 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f440c0ce240 0x7f440c15a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:50:59.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.758+0000 7f44118c9640 1 --2- 172.21.15.76:0/2837876332 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f440c15daa0 0x7f440c15ae80 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f440400cd60 tx=0x7f440400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:59.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.758+0000 7f4401ffb640 1 -- 172.21.15.76:0/2837876332 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4404003b90 con 0x7f440c15daa0 2023-11-09T03:50:59.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.758+0000 7f4401ffb640 1 -- 172.21.15.76:0/2837876332 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4404003d10 con 0x7f440c15daa0 2023-11-09T03:50:59.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.758+0000 7f4401ffb640 1 -- 172.21.15.76:0/2837876332 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4404004260 con 0x7f440c15daa0 2023-11-09T03:50:59.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.758+0000 7f441332c640 1 -- 172.21.15.76:0/2837876332 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f440c0cc2b0 con 0x7f440c15daa0 2023-11-09T03:50:59.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.758+0000 7f441332c640 1 -- 172.21.15.76:0/2837876332 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f440c152cb0 con 0x7f440c15daa0 2023-11-09T03:50:59.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.759+0000 7f441332c640 1 -- 172.21.15.76:0/2837876332 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f440c0cca70 con 0x7f440c15daa0 2023-11-09T03:50:59.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.759+0000 7f4401ffb640 1 -- 172.21.15.76:0/2837876332 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f44040043c0 con 0x7f440c15daa0 2023-11-09T03:50:59.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.760+0000 7f4401ffb640 1 --2- 172.21.15.76:0/2837876332 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f43f006ae70 0x7f43f006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:50:59.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.760+0000 7f4401ffb640 1 -- 172.21.15.76:0/2837876332 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7f4404012070 con 0x7f440c15daa0 2023-11-09T03:50:59.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.760+0000 7f44110c8640 1 --2- 172.21.15.76:0/2837876332 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f43f006ae70 0x7f43f006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:50:59.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.760+0000 7f44110c8640 1 --2- 172.21.15.76:0/2837876332 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f43f006ae70 0x7f43f006d330 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4408006b10 tx=0x7f4408008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:50:59.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.764+0000 7f4401ffb640 1 -- 172.21.15.76:0/2837876332 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f44040550c0 con 0x7f440c15daa0 2023-11-09T03:50:59.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.973+0000 7f441332c640 1 -- 172.21.15.76:0/2837876332 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f440c074290 con 0x7f440c15daa0 2023-11-09T03:50:59.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.973+0000 7f4401ffb640 1 -- 172.21.15.76:0/2837876332 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4140 (secure 0 0 0) 0x7f44040586e0 con 0x7f440c15daa0 2023-11-09T03:50:59.974 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:50:59.975 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":77,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:50:59.615830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8016,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8016":{"gid":8016,"name":"c","rank":0,"incarnation":77,"state":"up:replay","state_seq":1,"addr":"172.21.15.76:6835/3229239133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3229239133},{"type":"v1","addr":"172.21.15.76:6835","nonce":3229239133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:50:59.975 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 77 2023-11-09T03:50:59.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.975+0000 7f441332c640 1 -- 172.21.15.76:0/2837876332 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f43f006ae70 msgr2=0x7f43f006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:59.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.975+0000 7f441332c640 1 --2- 172.21.15.76:0/2837876332 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f43f006ae70 0x7f43f006d330 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4408006b10 tx=0x7f4408008040 comp rx=0 tx=0).stop 2023-11-09T03:50:59.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.975+0000 7f441332c640 1 -- 172.21.15.76:0/2837876332 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f440c15daa0 msgr2=0x7f440c15ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:50:59.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.975+0000 7f441332c640 1 --2- 172.21.15.76:0/2837876332 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f440c15daa0 0x7f440c15ae80 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f440400cd60 tx=0x7f440400a660 comp rx=0 tx=0).stop 2023-11-09T03:50:59.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.975+0000 7f441332c640 1 -- 172.21.15.76:0/2837876332 shutdown_connections 2023-11-09T03:50:59.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.975+0000 7f441332c640 1 --2- 172.21.15.76:0/2837876332 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f43f006ae70 0x7f43f006d330 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:59.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.975+0000 7f441332c640 1 --2- 172.21.15.76:0/2837876332 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f440c15daa0 0x7f440c15ae80 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:59.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.975+0000 7f441332c640 1 --2- 172.21.15.76:0/2837876332 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f440c0ce240 0x7f440c15a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:59.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.975+0000 7f441332c640 1 --2- 172.21.15.76:0/2837876332 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f440c0cde60 0x7f440c152770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:50:59.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.975+0000 7f441332c640 1 -- 172.21.15.76:0/2837876332 >> 172.21.15.76:0/2837876332 conn(0x7f440c071f30 msgr2=0x7f440c0d2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:50:59.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.975+0000 7f441332c640 1 -- 172.21.15.76:0/2837876332 shutdown_connections 2023-11-09T03:50:59.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:50:59.975+0000 7f441332c640 1 -- 172.21.15.76:0/2837876332 wait complete. 2023-11-09T03:50:59.986 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-11-09T03:51:01.701 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:51:01.705 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:51:04.678 INFO:tasks.ceph.mds.c.smithi076.stderr:2023-11-09T03:51:04.677+0000 7fc93bfd6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:51:04.678 INFO:tasks.ceph.mds.c.smithi076.stderr:2023-11-09T03:51:04.677+0000 7fc93bfd6640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-09T03:51:04.987 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:51:05.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.101+0000 7f7e11675640 1 Processor -- start 2023-11-09T03:51:05.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.101+0000 7f7e11675640 1 -- start start 2023-11-09T03:51:05.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.102+0000 7f7e11675640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7e0c160b10 0x7f7e0c160f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:05.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.102+0000 7f7e11675640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7e0c0cf160 0x7f7e0c0cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:05.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.102+0000 7f7e11675640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e0c06bf00 con 0x7f7e0c15ce60 2023-11-09T03:51:05.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.102+0000 7f7e11675640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e0c0cbde0 con 0x7f7e0c160b10 2023-11-09T03:51:05.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.102+0000 7f7e11675640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e0c0cfb30 con 0x7f7e0c0cf160 2023-11-09T03:51:05.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.102+0000 7f7e0a7fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7e0c160b10 0x7f7e0c160f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:05.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.102+0000 7f7e0a7fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7e0c160b10 0x7f7e0c160f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45034/0 (socket says 172.21.15.76:45034) 2023-11-09T03:51:05.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.102+0000 7f7e0a7fc640 1 -- 172.21.15.76:0/2688496479 learned_addr learned my addr 172.21.15.76:0/2688496479 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:05.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.102+0000 7f7e0b7fe640 1 --2- 172.21.15.76:0/2688496479 >> v2:172.21.15.113:3300/0 conn(0x7f7e0c0cf160 0x7f7e0c0cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:05.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.103+0000 7f7e0a7fc640 1 -- 172.21.15.76:0/2688496479 >> v2:172.21.15.113:3300/0 conn(0x7f7e0c0cf160 msgr2=0x7f7e0c0cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:05.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.103+0000 7f7e0a7fc640 1 --2- 172.21.15.76:0/2688496479 >> v2:172.21.15.113:3300/0 conn(0x7f7e0c0cf160 0x7f7e0c0cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:05.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.103+0000 7f7e0a7fc640 1 -- 172.21.15.76:0/2688496479 >> v1:172.21.15.113:6789/0 conn(0x7f7e0c15ce60 legacy=0x7f7e0c06b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:05.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.103+0000 7f7e0a7fc640 1 -- 172.21.15.76:0/2688496479 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e0c1615d0 con 0x7f7e0c160b10 2023-11-09T03:51:05.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.103+0000 7f7e0b7fe640 1 --2- 172.21.15.76:0/2688496479 >> v2:172.21.15.113:3300/0 conn(0x7f7e0c0cf160 0x7f7e0c0cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:51:05.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.103+0000 7f7e0a7fc640 1 --2- 172.21.15.76:0/2688496479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e0c160b10 0x7f7e0c160f90 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c1614d0 tx=0x7f7e0002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c403b2508d541652 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:05.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.103+0000 7f7e09ffb640 1 -- 172.21.15.76:0/2688496479 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e00031870 con 0x7f7e0c160b10 2023-11-09T03:51:05.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.103+0000 7f7e09ffb640 1 -- 172.21.15.76:0/2688496479 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e000319d0 con 0x7f7e0c160b10 2023-11-09T03:51:05.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.104+0000 7f7e09ffb640 1 -- 172.21.15.76:0/2688496479 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e00031d60 con 0x7f7e0c160b10 2023-11-09T03:51:05.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.104+0000 7f7e11675640 1 -- 172.21.15.76:0/2688496479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e0c160b10 msgr2=0x7f7e0c160f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:05.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.104+0000 7f7e11675640 1 --2- 172.21.15.76:0/2688496479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e0c160b10 0x7f7e0c160f90 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c1614d0 tx=0x7f7e0002b910 comp rx=0 tx=0).stop 2023-11-09T03:51:05.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.104+0000 7f7e11675640 1 -- 172.21.15.76:0/2688496479 shutdown_connections 2023-11-09T03:51:05.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.104+0000 7f7e11675640 1 --2- 172.21.15.76:0/2688496479 >> v2:172.21.15.113:3300/0 conn(0x7f7e0c0cf160 0x7f7e0c0cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:05.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.104+0000 7f7e11675640 1 --2- 172.21.15.76:0/2688496479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e0c160b10 0x7f7e0c160f90 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:05.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.104+0000 7f7e11675640 1 -- 172.21.15.76:0/2688496479 >> 172.21.15.76:0/2688496479 conn(0x7f7e0c06ea90 msgr2=0x7f7e0c06ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:05.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.104+0000 7f7e11675640 1 -- 172.21.15.76:0/2688496479 shutdown_connections 2023-11-09T03:51:05.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.104+0000 7f7e11675640 1 -- 172.21.15.76:0/2688496479 wait complete. 2023-11-09T03:51:05.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.105+0000 7f7e11675640 1 Processor -- start 2023-11-09T03:51:05.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.105+0000 7f7e11675640 1 -- start start 2023-11-09T03:51:05.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.106+0000 7f7e11675640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e0c0cf160 0x7f7e0c159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:05.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.106+0000 7f7e11675640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e0c15ce60 0x7f7e0c0c5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:05.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.106+0000 7f7e11675640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e0c160b10 0x7f7e0c0c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:05.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.106+0000 7f7e11675640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c15eb40 con 0x7f7e0c160b10 2023-11-09T03:51:05.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.106+0000 7f7e11675640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c15df30 con 0x7f7e0c0cf160 2023-11-09T03:51:05.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.106+0000 7f7e11675640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c15e210 con 0x7f7e0c15ce60 2023-11-09T03:51:05.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.106+0000 7f7e0b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e0c160b10 0x7f7e0c0c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:05.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.106+0000 7f7e0b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e0c160b10 0x7f7e0c0c5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45048/0 (socket says 172.21.15.76:45048) 2023-11-09T03:51:05.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.106+0000 7f7e0b7fe640 1 -- 172.21.15.76:0/1982876542 learned_addr learned my addr 172.21.15.76:0/1982876542 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:05.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.107+0000 7f7e0a7fc640 1 --2- 172.21.15.76:0/1982876542 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e0c15ce60 0x7f7e0c0c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:05.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.107+0000 7f7e0affd640 1 --2- 172.21.15.76:0/1982876542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e0c0cf160 0x7f7e0c159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:05.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.107+0000 7f7e0b7fe640 1 -- 172.21.15.76:0/1982876542 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e0c15ce60 msgr2=0x7f7e0c0c5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:05.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.107+0000 7f7e0b7fe640 1 --2- 172.21.15.76:0/1982876542 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e0c15ce60 0x7f7e0c0c5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:05.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.107+0000 7f7e0b7fe640 1 -- 172.21.15.76:0/1982876542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e0c0cf160 msgr2=0x7f7e0c159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:05.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.107+0000 7f7e0b7fe640 1 --2- 172.21.15.76:0/1982876542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e0c0cf160 0x7f7e0c159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:05.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.107+0000 7f7e0b7fe640 1 -- 172.21.15.76:0/1982876542 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e000094d0 con 0x7f7e0c160b10 2023-11-09T03:51:05.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.107+0000 7f7e0a7fc640 1 --2- 172.21.15.76:0/1982876542 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e0c15ce60 0x7f7e0c0c5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:51:05.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.107+0000 7f7e0affd640 1 --2- 172.21.15.76:0/1982876542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e0c0cf160 0x7f7e0c159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:51:05.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.107+0000 7f7e0b7fe640 1 --2- 172.21.15.76:0/1982876542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e0c160b10 0x7f7e0c0c5bc0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400c7c0 tx=0x7f7e0400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:05.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.107+0000 7f7df37fe640 1 -- 172.21.15.76:0/1982876542 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e04004230 con 0x7f7e0c160b10 2023-11-09T03:51:05.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.107+0000 7f7df37fe640 1 -- 172.21.15.76:0/1982876542 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e04004390 con 0x7f7e0c160b10 2023-11-09T03:51:05.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.107+0000 7f7e11675640 1 -- 172.21.15.76:0/1982876542 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e0c15e4f0 con 0x7f7e0c160b10 2023-11-09T03:51:05.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.107+0000 7f7df37fe640 1 -- 172.21.15.76:0/1982876542 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e04007860 con 0x7f7e0c160b10 2023-11-09T03:51:05.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.107+0000 7f7e11675640 1 -- 172.21.15.76:0/1982876542 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e0c1f2930 con 0x7f7e0c160b10 2023-11-09T03:51:05.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.108+0000 7f7e11675640 1 -- 172.21.15.76:0/1982876542 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e0c0ce950 con 0x7f7e0c160b10 2023-11-09T03:51:05.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.108+0000 7f7df37fe640 1 -- 172.21.15.76:0/1982876542 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7e040079c0 con 0x7f7e0c160b10 2023-11-09T03:51:05.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.109+0000 7f7df37fe640 1 --2- 172.21.15.76:0/1982876542 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7df806ae70 0x7f7df806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:05.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.109+0000 7f7df37fe640 1 -- 172.21.15.76:0/1982876542 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7f7e04089f80 con 0x7f7e0c160b10 2023-11-09T03:51:05.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.109+0000 7f7e0affd640 1 --2- 172.21.15.76:0/1982876542 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7df806ae70 0x7f7df806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:05.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.110+0000 7f7e0affd640 1 --2- 172.21.15.76:0/1982876542 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7df806ae70 0x7f7df806d330 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc007eb0 tx=0x7f7dfc0073c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:05.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.113+0000 7f7df37fe640 1 -- 172.21.15.76:0/1982876542 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7e040bcca0 con 0x7f7e0c160b10 2023-11-09T03:51:05.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.315+0000 7f7e11675640 1 -- 172.21.15.76:0/1982876542 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e0c1f2b20 con 0x7f7e0c160b10 2023-11-09T03:51:05.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.315+0000 7f7df37fe640 1 -- 172.21.15.76:0/1982876542 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4932 (secure 0 0 0) 0x7f7e04015090 con 0x7f7e0c160b10 2023-11-09T03:51:05.317 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:05.317 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":81,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6904,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:51:02.634881+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8016,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8016":{"gid":8016,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3229239133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3229239133},{"type":"v1","addr":"172.21.15.76:6835","nonce":3229239133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:51:05.317 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 81 2023-11-09T03:51:05.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.317+0000 7f7e11675640 1 -- 172.21.15.76:0/1982876542 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7df806ae70 msgr2=0x7f7df806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:05.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.317+0000 7f7e11675640 1 --2- 172.21.15.76:0/1982876542 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7df806ae70 0x7f7df806d330 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc007eb0 tx=0x7f7dfc0073c0 comp rx=0 tx=0).stop 2023-11-09T03:51:05.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.317+0000 7f7e11675640 1 -- 172.21.15.76:0/1982876542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e0c160b10 msgr2=0x7f7e0c0c5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:05.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.317+0000 7f7e11675640 1 --2- 172.21.15.76:0/1982876542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e0c160b10 0x7f7e0c0c5bc0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400c7c0 tx=0x7f7e0400cc80 comp rx=0 tx=0).stop 2023-11-09T03:51:05.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.317+0000 7f7e11675640 1 -- 172.21.15.76:0/1982876542 shutdown_connections 2023-11-09T03:51:05.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.317+0000 7f7e11675640 1 --2- 172.21.15.76:0/1982876542 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7df806ae70 0x7f7df806d330 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:05.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.317+0000 7f7e11675640 1 --2- 172.21.15.76:0/1982876542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e0c160b10 0x7f7e0c0c5bc0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:05.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.317+0000 7f7e11675640 1 --2- 172.21.15.76:0/1982876542 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e0c15ce60 0x7f7e0c0c5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:05.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.317+0000 7f7e11675640 1 --2- 172.21.15.76:0/1982876542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e0c0cf160 0x7f7e0c159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:05.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.317+0000 7f7e11675640 1 -- 172.21.15.76:0/1982876542 >> 172.21.15.76:0/1982876542 conn(0x7f7e0c06ea90 msgr2=0x7f7e0c0afe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:05.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.317+0000 7f7e11675640 1 -- 172.21.15.76:0/1982876542 shutdown_connections 2023-11-09T03:51:05.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.318+0000 7f7e11675640 1 -- 172.21.15.76:0/1982876542 wait complete. 2023-11-09T03:51:05.329 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-09T03:51:05.329 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:51:05.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.428+0000 7f6f1faa9640 1 Processor -- start 2023-11-09T03:51:05.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.429+0000 7f6f1faa9640 1 -- start start 2023-11-09T03:51:05.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.429+0000 7f6f1faa9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6f180ade20 0x7f6f1806c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:05.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.429+0000 7f6f1faa9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6f180c1e80 0x7f6f180c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:05.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.429+0000 7f6f1faa9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f180ae200 con 0x7f6f180c97e0 2023-11-09T03:51:05.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.429+0000 7f6f1faa9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f181617c0 con 0x7f6f180c1e80 2023-11-09T03:51:05.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.429+0000 7f6f1faa9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f180c9d10 con 0x7f6f180ade20 2023-11-09T03:51:05.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.430+0000 7f6f1d044640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6f180c1e80 0x7f6f180c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:05.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.430+0000 7f6f1d044640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6f180c1e80 0x7f6f180c92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45058/0 (socket says 172.21.15.76:45058) 2023-11-09T03:51:05.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.430+0000 7f6f1d044640 1 -- 172.21.15.76:0/171811225 learned_addr learned my addr 172.21.15.76:0/171811225 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:05.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.430+0000 7f6f1d845640 1 --2- 172.21.15.76:0/171811225 >> v2:172.21.15.113:3300/0 conn(0x7f6f180ade20 0x7f6f1806c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:05.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.430+0000 7f6f1d044640 1 -- 172.21.15.76:0/171811225 >> v2:172.21.15.113:3300/0 conn(0x7f6f180ade20 msgr2=0x7f6f1806c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:05.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.430+0000 7f6f1d044640 1 --2- 172.21.15.76:0/171811225 >> v2:172.21.15.113:3300/0 conn(0x7f6f180ade20 0x7f6f1806c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:05.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.430+0000 7f6f1d044640 1 -- 172.21.15.76:0/171811225 >> v1:172.21.15.113:6789/0 conn(0x7f6f180c97e0 legacy=0x7f6f180cce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:05.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.430+0000 7f6f1d044640 1 -- 172.21.15.76:0/171811225 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f180c9ff0 con 0x7f6f180c1e80 2023-11-09T03:51:05.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.430+0000 7f6f1d845640 1 --2- 172.21.15.76:0/171811225 >> v2:172.21.15.113:3300/0 conn(0x7f6f180ade20 0x7f6f1806c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:51:05.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.431+0000 7f6f1d044640 1 --2- 172.21.15.76:0/171811225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f180c1e80 0x7f6f180c92a0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f6f180cc440 tx=0x7f6f0802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac7c25d76356b3e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:05.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.431+0000 7f6f1c843640 1 -- 172.21.15.76:0/171811225 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f08031870 con 0x7f6f180c1e80 2023-11-09T03:51:05.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.431+0000 7f6f1c843640 1 -- 172.21.15.76:0/171811225 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f080319d0 con 0x7f6f180c1e80 2023-11-09T03:51:05.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.431+0000 7f6f1c843640 1 -- 172.21.15.76:0/171811225 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f08031d60 con 0x7f6f180c1e80 2023-11-09T03:51:05.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.431+0000 7f6f1faa9640 1 -- 172.21.15.76:0/171811225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f180c1e80 msgr2=0x7f6f180c92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:05.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.431+0000 7f6f1faa9640 1 --2- 172.21.15.76:0/171811225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f180c1e80 0x7f6f180c92a0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f6f180cc440 tx=0x7f6f0802b910 comp rx=0 tx=0).stop 2023-11-09T03:51:05.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.431+0000 7f6f1faa9640 1 -- 172.21.15.76:0/171811225 shutdown_connections 2023-11-09T03:51:05.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.431+0000 7f6f1faa9640 1 --2- 172.21.15.76:0/171811225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f180c1e80 0x7f6f180c92a0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:05.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.431+0000 7f6f1faa9640 1 --2- 172.21.15.76:0/171811225 >> v2:172.21.15.113:3300/0 conn(0x7f6f180ade20 0x7f6f1806c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:05.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.431+0000 7f6f1faa9640 1 -- 172.21.15.76:0/171811225 >> 172.21.15.76:0/171811225 conn(0x7f6f180ce950 msgr2=0x7f6f180ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:05.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.431+0000 7f6f1faa9640 1 -- 172.21.15.76:0/171811225 shutdown_connections 2023-11-09T03:51:05.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.431+0000 7f6f1faa9640 1 -- 172.21.15.76:0/171811225 wait complete. 2023-11-09T03:51:05.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.433+0000 7f6f1faa9640 1 Processor -- start 2023-11-09T03:51:05.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.433+0000 7f6f1faa9640 1 -- start start 2023-11-09T03:51:05.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.433+0000 7f6f1faa9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f180ade20 0x7f6f1815fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:05.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.433+0000 7f6f1faa9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f180c1e80 0x7f6f180c2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:05.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.433+0000 7f6f1faa9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f180c97e0 0x7f6f180c2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:05.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.433+0000 7f6f1faa9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f181636c0 con 0x7f6f180ade20 2023-11-09T03:51:05.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.433+0000 7f6f1faa9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f180c4830 con 0x7f6f180c1e80 2023-11-09T03:51:05.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.433+0000 7f6f1faa9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f180c3f90 con 0x7f6f180c97e0 2023-11-09T03:51:05.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.433+0000 7f6f1d845640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f180ade20 0x7f6f1815fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:05.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.433+0000 7f6f1d845640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f180ade20 0x7f6f1815fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45072/0 (socket says 172.21.15.76:45072) 2023-11-09T03:51:05.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.434+0000 7f6f1d845640 1 -- 172.21.15.76:0/2224394575 learned_addr learned my addr 172.21.15.76:0/2224394575 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:05.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.434+0000 7f6f1d044640 1 --2- 172.21.15.76:0/2224394575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f180c1e80 0x7f6f180c2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:05.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.434+0000 7f6f1e046640 1 --2- 172.21.15.76:0/2224394575 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f180c97e0 0x7f6f180c2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:05.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.434+0000 7f6f1d845640 1 -- 172.21.15.76:0/2224394575 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f180c97e0 msgr2=0x7f6f180c2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:05.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.434+0000 7f6f1d845640 1 --2- 172.21.15.76:0/2224394575 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f180c97e0 0x7f6f180c2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:05.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.434+0000 7f6f1d845640 1 -- 172.21.15.76:0/2224394575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f180c1e80 msgr2=0x7f6f180c2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:05.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.434+0000 7f6f1d845640 1 --2- 172.21.15.76:0/2224394575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f180c1e80 0x7f6f180c2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:05.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.434+0000 7f6f1d845640 1 -- 172.21.15.76:0/2224394575 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f080094d0 con 0x7f6f180ade20 2023-11-09T03:51:05.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.434+0000 7f6f1e046640 1 --2- 172.21.15.76:0/2224394575 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f180c97e0 0x7f6f180c2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:51:05.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.434+0000 7f6f1d845640 1 --2- 172.21.15.76:0/2224394575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f180ade20 0x7f6f1815fac0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f6f1400cf60 tx=0x7f6f1400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:05.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.434+0000 7f6f0e7fc640 1 -- 172.21.15.76:0/2224394575 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1400d840 con 0x7f6f180ade20 2023-11-09T03:51:05.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.434+0000 7f6f0e7fc640 1 -- 172.21.15.76:0/2224394575 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1400d9a0 con 0x7f6f180ade20 2023-11-09T03:51:05.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.434+0000 7f6f1faa9640 1 -- 172.21.15.76:0/2224394575 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f180c4270 con 0x7f6f180ade20 2023-11-09T03:51:05.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.434+0000 7f6f0e7fc640 1 -- 172.21.15.76:0/2224394575 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1400dd30 con 0x7f6f180ade20 2023-11-09T03:51:05.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.434+0000 7f6f1faa9640 1 -- 172.21.15.76:0/2224394575 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f181619a0 con 0x7f6f180ade20 2023-11-09T03:51:05.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.435+0000 7f6f1faa9640 1 -- 172.21.15.76:0/2224394575 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f18161b90 con 0x7f6f180ade20 2023-11-09T03:51:05.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.436+0000 7f6f0e7fc640 1 -- 172.21.15.76:0/2224394575 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6f14015440 con 0x7f6f180ade20 2023-11-09T03:51:05.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.436+0000 7f6f0e7fc640 1 --2- 172.21.15.76:0/2224394575 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6efc06ae70 0x7f6efc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:05.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.436+0000 7f6f0e7fc640 1 -- 172.21.15.76:0/2224394575 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7f6f1408a6f0 con 0x7f6f180ade20 2023-11-09T03:51:05.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.436+0000 7f6f1d044640 1 --2- 172.21.15.76:0/2224394575 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6efc06ae70 0x7f6efc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:05.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.437+0000 7f6f1d044640 1 --2- 172.21.15.76:0/2224394575 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6efc06ae70 0x7f6efc06d330 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6f08036000 tx=0x7f6f0803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:05.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.441+0000 7f6f0e7fc640 1 -- 172.21.15.76:0/2224394575 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6f14055130 con 0x7f6f180ade20 2023-11-09T03:51:05.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.644+0000 7f6f1faa9640 1 -- 172.21.15.76:0/2224394575 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f18074290 con 0x7f6f180ade20 2023-11-09T03:51:05.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.646+0000 7f6f0e7fc640 1 -- 172.21.15.76:0/2224394575 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4932 (secure 0 0 0) 0x7f6f14058750 con 0x7f6f180ade20 2023-11-09T03:51:05.647 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:05.647 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":81,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6904,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:51:02.634881+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8016,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8016":{"gid":8016,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3229239133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3229239133},{"type":"v1","addr":"172.21.15.76:6835","nonce":3229239133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:51:05.647 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 81 2023-11-09T03:51:05.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.648+0000 7f6f1faa9640 1 -- 172.21.15.76:0/2224394575 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6efc06ae70 msgr2=0x7f6efc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:05.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.648+0000 7f6f1faa9640 1 --2- 172.21.15.76:0/2224394575 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6efc06ae70 0x7f6efc06d330 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6f08036000 tx=0x7f6f0803c040 comp rx=0 tx=0).stop 2023-11-09T03:51:05.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.648+0000 7f6f1faa9640 1 -- 172.21.15.76:0/2224394575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f180ade20 msgr2=0x7f6f1815fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:05.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.648+0000 7f6f1faa9640 1 --2- 172.21.15.76:0/2224394575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f180ade20 0x7f6f1815fac0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f6f1400cf60 tx=0x7f6f1400cf90 comp rx=0 tx=0).stop 2023-11-09T03:51:05.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.648+0000 7f6f1faa9640 1 -- 172.21.15.76:0/2224394575 shutdown_connections 2023-11-09T03:51:05.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.648+0000 7f6f1faa9640 1 --2- 172.21.15.76:0/2224394575 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6efc06ae70 0x7f6efc06d330 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:05.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.648+0000 7f6f1faa9640 1 --2- 172.21.15.76:0/2224394575 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f180c97e0 0x7f6f180c2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:05.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.648+0000 7f6f1faa9640 1 --2- 172.21.15.76:0/2224394575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f180c1e80 0x7f6f180c2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:05.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.648+0000 7f6f1faa9640 1 --2- 172.21.15.76:0/2224394575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f180ade20 0x7f6f1815fac0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:05.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.648+0000 7f6f1faa9640 1 -- 172.21.15.76:0/2224394575 >> 172.21.15.76:0/2224394575 conn(0x7f6f180ce950 msgr2=0x7f6f1806d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:05.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.648+0000 7f6f1faa9640 1 -- 172.21.15.76:0/2224394575 shutdown_connections 2023-11-09T03:51:05.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:05.648+0000 7f6f1faa9640 1 -- 172.21.15.76:0/2224394575 wait complete. 2023-11-09T03:51:05.794 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-11-09T03:51:06.062 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 81, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-09T03:48:57.383848+0000', 'modified': '2023-11-09T03:51:02.634881+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 101, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 8016, 'mds_1': 7449}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7449': {'gid': 7449, 'name': 'a', 'rank': 1, 'incarnation': 47, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.76:6833/3697850101', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6832', 'nonce': 3697850101}, {'type': 'v1', 'addr': '172.21.15.76:6833', 'nonce': 3697850101}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_8016': {'gid': 8016, 'name': 'c', 'rank': 0, 'incarnation': 77, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.76:6835/3229239133', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 3229239133}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 3229239133}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-09T03:51:06.062 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2023-11-09T03:51:06.063 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-09T03:51:06.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.164+0000 7f5003597640 1 Processor -- start 2023-11-09T03:51:06.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.164+0000 7f5003597640 1 -- start start 2023-11-09T03:51:06.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.165+0000 7f5003597640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ffc159c00 0x7f4ffc0cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.165+0000 7f5003597640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4ffc0ae1d0 0x7f4ffc0c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.165+0000 7f5003597640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ffc15aff0 con 0x7f4ffc0c48e0 2023-11-09T03:51:06.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.165+0000 7f5003597640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ffc0c52a0 con 0x7f4ffc0ae1d0 2023-11-09T03:51:06.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.165+0000 7f5003597640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc15a3e0 con 0x7f4ffc159c00 2023-11-09T03:51:06.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.165+0000 7f5000b32640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f4ffc0c48e0 0x7f4ffc0adac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39534/0 (socket says 172.21.15.76:39534) 2023-11-09T03:51:06.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.165+0000 7f5000b32640 1 -- 172.21.15.76:0/3643712705 learned_addr learned my addr 172.21.15.76:0/3643712705 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:51:06.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.165+0000 7f5001b34640 1 --2- 172.21.15.76:0/3643712705 >> v2:172.21.15.113:3300/0 conn(0x7f4ffc0ae1d0 0x7f4ffc0c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.165+0000 7f5001333640 1 --2- 172.21.15.76:0/3643712705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ffc159c00 0x7f4ffc0cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.165+0000 7f4ff3fff640 1 -- 172.21.15.76:0/3643712705 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3586261397 0 0) 0x7f4ffc15aff0 con 0x7f4ffc0c48e0 2023-11-09T03:51:06.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.165+0000 7f4ff3fff640 1 -- 172.21.15.76:0/3643712705 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fe8003520 con 0x7f4ffc0c48e0 2023-11-09T03:51:06.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.166+0000 7f4ff3fff640 1 -- 172.21.15.76:0/3643712705 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1786456100 0 0) 0x7f4fe8003520 con 0x7f4ffc0c48e0 2023-11-09T03:51:06.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.166+0000 7f4ff3fff640 1 -- 172.21.15.76:0/3643712705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ffc159c00 msgr2=0x7f4ffc0cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:06.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.166+0000 7f4ff3fff640 1 --2- 172.21.15.76:0/3643712705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ffc159c00 0x7f4ffc0cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.166+0000 7f4ff3fff640 1 -- 172.21.15.76:0/3643712705 >> v2:172.21.15.113:3300/0 conn(0x7f4ffc0ae1d0 msgr2=0x7f4ffc0c4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:06.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.166+0000 7f4ff3fff640 1 --2- 172.21.15.76:0/3643712705 >> v2:172.21.15.113:3300/0 conn(0x7f4ffc0ae1d0 0x7f4ffc0c4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.166+0000 7f4ff3fff640 1 -- 172.21.15.76:0/3643712705 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ffc15a6c0 con 0x7f4ffc0c48e0 2023-11-09T03:51:06.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.166+0000 7f4ff3fff640 1 -- 172.21.15.76:0/3643712705 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f4fec001670 con 0x7f4ffc0c48e0 2023-11-09T03:51:06.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.166+0000 7f4ff3fff640 1 -- 172.21.15.76:0/3643712705 >> v1:172.21.15.76:6789/0 conn(0x7f4ffc0c48e0 legacy=0x7f4ffc0adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:06.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.166+0000 7f4ff3fff640 1 --2- 172.21.15.76:0/3643712705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fe8003970 0x7f4fe8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.166+0000 7f4ff3fff640 1 --2- 172.21.15.76:0/3643712705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ffc159c00 0x7f4fe80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.166+0000 7f4ff3fff640 1 --2- 172.21.15.76:0/3643712705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fe8004d00 0x7f4fe8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.166+0000 7f4ff3fff640 1 -- 172.21.15.76:0/3643712705 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc0c52a0 con 0x7f4fe8003970 2023-11-09T03:51:06.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.166+0000 7f4ff3fff640 1 -- 172.21.15.76:0/3643712705 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc15a3e0 con 0x7f4fe8004d00 2023-11-09T03:51:06.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.166+0000 7f4ff3fff640 1 -- 172.21.15.76:0/3643712705 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe800db90 con 0x7f4ffc159c00 2023-11-09T03:51:06.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.166+0000 7f5001333640 1 --2- 172.21.15.76:0/3643712705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fe8003970 0x7f4fe8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.166+0000 7f5000b32640 1 --2- 172.21.15.76:0/3643712705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ffc159c00 0x7f4fe80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.166+0000 7f5001b34640 1 --2- 172.21.15.76:0/3643712705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fe8004d00 0x7f4fe8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.166+0000 7f5001333640 1 -- 172.21.15.76:0/3643712705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ffc159c00 msgr2=0x7f4fe80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:06.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.166+0000 7f5001333640 1 --2- 172.21.15.76:0/3643712705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ffc159c00 0x7f4fe80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.166+0000 7f5001333640 1 -- 172.21.15.76:0/3643712705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fe8004d00 msgr2=0x7f4fe8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:06.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.166+0000 7f5001333640 1 --2- 172.21.15.76:0/3643712705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fe8004d00 0x7f4fe8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.166+0000 7f5001333640 1 -- 172.21.15.76:0/3643712705 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe8003520 con 0x7f4fe8003970 2023-11-09T03:51:06.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.167+0000 7f5001333640 1 --2- 172.21.15.76:0/3643712705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fe8003970 0x7f4fe8003d90 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8009ac0 tx=0x7f4fe80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f7352defae3d3b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:06.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.167+0000 7f4ff3fff640 1 -- 172.21.15.76:0/3643712705 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff802c6b0 con 0x7f4fe8003970 2023-11-09T03:51:06.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.167+0000 7f4ff3fff640 1 -- 172.21.15.76:0/3643712705 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff802c810 con 0x7f4fe8003970 2023-11-09T03:51:06.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.167+0000 7f4ff3fff640 1 -- 172.21.15.76:0/3643712705 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff802cba0 con 0x7f4fe8003970 2023-11-09T03:51:06.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.167+0000 7f5003597640 1 -- 172.21.15.76:0/3643712705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fe8003970 msgr2=0x7f4fe8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:06.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.167+0000 7f5003597640 1 --2- 172.21.15.76:0/3643712705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fe8003970 0x7f4fe8003d90 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8009ac0 tx=0x7f4fe80057f0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.167+0000 7f5001b34640 1 -- 172.21.15.76:0/3643712705 reap_dead start 2023-11-09T03:51:06.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.167+0000 7f5003597640 1 -- 172.21.15.76:0/3643712705 shutdown_connections 2023-11-09T03:51:06.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.167+0000 7f5003597640 1 -- 172.21.15.76:0/3643712705 >> 172.21.15.76:0/3643712705 conn(0x7f4ffc0ce950 msgr2=0x7f4ffc0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:06.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.168+0000 7f5003597640 1 -- 172.21.15.76:0/3643712705 shutdown_connections 2023-11-09T03:51:06.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.168+0000 7f5003597640 1 -- 172.21.15.76:0/3643712705 wait complete. 2023-11-09T03:51:06.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.169+0000 7f5003597640 1 Processor -- start 2023-11-09T03:51:06.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.169+0000 7f5003597640 1 -- start start 2023-11-09T03:51:06.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.169+0000 7f5003597640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fe8003970 0x7f4ffc0ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.169+0000 7f5003597640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ffc1e1eb0 0x7f4ffc1530e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.169+0000 7f5003597640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ffc1e2310 0x7f4ffc153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.169+0000 7f5003597640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc1e0e10 con 0x7f4ffc1e1eb0 2023-11-09T03:51:06.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.169+0000 7f5003597640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc1e10f0 con 0x7f4ffc1e2310 2023-11-09T03:51:06.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.169+0000 7f5003597640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc1e13d0 con 0x7f4fe8003970 2023-11-09T03:51:06.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.170+0000 7f5000b32640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ffc1e1eb0 0x7f4ffc1530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.170+0000 7f5000b32640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ffc1e1eb0 0x7f4ffc1530e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45094/0 (socket says 172.21.15.76:45094) 2023-11-09T03:51:06.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.170+0000 7f5000b32640 1 -- 172.21.15.76:0/1962862600 learned_addr learned my addr 172.21.15.76:0/1962862600 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:06.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.170+0000 7f5001b34640 1 --2- 172.21.15.76:0/1962862600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ffc1e2310 0x7f4ffc153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.170+0000 7f5001333640 1 --2- 172.21.15.76:0/1962862600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fe8003970 0x7f4ffc0ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.170+0000 7f5000b32640 1 -- 172.21.15.76:0/1962862600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fe8003970 msgr2=0x7f4ffc0ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:06.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.170+0000 7f5000b32640 1 --2- 172.21.15.76:0/1962862600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fe8003970 0x7f4ffc0ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.170+0000 7f5000b32640 1 -- 172.21.15.76:0/1962862600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ffc1e2310 msgr2=0x7f4ffc153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:06.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.170+0000 7f5000b32640 1 --2- 172.21.15.76:0/1962862600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ffc1e2310 0x7f4ffc153620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.170+0000 7f5000b32640 1 -- 172.21.15.76:0/1962862600 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ffc1e16b0 con 0x7f4ffc1e1eb0 2023-11-09T03:51:06.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.170+0000 7f5001b34640 1 --2- 172.21.15.76:0/1962862600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ffc1e2310 0x7f4ffc153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:51:06.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.170+0000 7f5000b32640 1 --2- 172.21.15.76:0/1962862600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ffc1e1eb0 0x7f4ffc1530e0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f4fec000c00 tx=0x7f4fec00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:06.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.170+0000 7f4ff1ffb640 1 -- 172.21.15.76:0/1962862600 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec00ee60 con 0x7f4ffc1e1eb0 2023-11-09T03:51:06.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.170+0000 7f4ff1ffb640 1 -- 172.21.15.76:0/1962862600 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fec00c8d0 con 0x7f4ffc1e1eb0 2023-11-09T03:51:06.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.170+0000 7f4ff1ffb640 1 -- 172.21.15.76:0/1962862600 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec00cc80 con 0x7f4ffc1e1eb0 2023-11-09T03:51:06.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.170+0000 7f5003597640 1 -- 172.21.15.76:0/1962862600 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ffc1e1990 con 0x7f4ffc1e1eb0 2023-11-09T03:51:06.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.170+0000 7f5003597640 1 -- 172.21.15.76:0/1962862600 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ffc0cd220 con 0x7f4ffc1e1eb0 2023-11-09T03:51:06.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.172+0000 7f4ff1ffb640 1 -- 172.21.15.76:0/1962862600 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4fec01d460 con 0x7f4ffc1e1eb0 2023-11-09T03:51:06.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.172+0000 7f4ff1ffb640 1 --2- 172.21.15.76:0/1962862600 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4fe006aea0 0x7f4fe006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.172+0000 7f4ff1ffb640 1 -- 172.21.15.76:0/1962862600 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7f4fec089b30 con 0x7f4ffc1e1eb0 2023-11-09T03:51:06.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.172+0000 7f5001333640 1 --2- 172.21.15.76:0/1962862600 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4fe006aea0 0x7f4fe006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.173+0000 7f5001333640 1 --2- 172.21.15.76:0/1962862600 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4fe006aea0 0x7f4fe006d360 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8036000 tx=0x7f4ff8030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:06.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.173+0000 7f5006206740 1 Processor -- start 2023-11-09T03:51:06.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.173+0000 7f5006206740 1 -- start start 2023-11-09T03:51:06.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.174+0000 7f5006206740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cd5e4d3d0 0x557cd5e4af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.174+0000 7f5006206740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557cd5e2cc10 0x557cd5e04900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.174+0000 7f5006206740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557cd5e04e40 0x557cd5e2f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.174+0000 7f5006206740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cd5d10740 con 0x557cd5e4d3d0 2023-11-09T03:51:06.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.174+0000 7f5006206740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cd5d728d0 con 0x557cd5e2cc10 2023-11-09T03:51:06.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.174+0000 7f5006206740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x557cd5d6ffc0 con 0x557cd5e04e40 2023-11-09T03:51:06.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.174+0000 7f5001333640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cd5e4d3d0 0x557cd5e4af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.174+0000 7f5001333640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cd5e4d3d0 0x557cd5e4af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45110/0 (socket says 172.21.15.76:45110) 2023-11-09T03:51:06.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.174+0000 7f5001333640 1 -- 172.21.15.76:0/2381404711 learned_addr learned my addr 172.21.15.76:0/2381404711 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:06.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.174+0000 7f5000b32640 1 --2- 172.21.15.76:0/2381404711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557cd5e2cc10 0x557cd5e04900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.174+0000 7f5001b34640 1 --2- 172.21.15.76:0/2381404711 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557cd5e04e40 0x557cd5e2f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.174+0000 7f5001333640 1 -- 172.21.15.76:0/2381404711 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557cd5e04e40 msgr2=0x557cd5e2f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:06.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.174+0000 7f5001333640 1 --2- 172.21.15.76:0/2381404711 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557cd5e04e40 0x557cd5e2f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.174+0000 7f5001333640 1 -- 172.21.15.76:0/2381404711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557cd5e2cc10 msgr2=0x557cd5e04900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:06.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.174+0000 7f5001333640 1 --2- 172.21.15.76:0/2381404711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557cd5e2cc10 0x557cd5e04900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.174+0000 7f5001333640 1 -- 172.21.15.76:0/2381404711 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cd5d2eca0 con 0x557cd5e4d3d0 2023-11-09T03:51:06.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.174+0000 7f5001b34640 1 --2- 172.21.15.76:0/2381404711 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557cd5e04e40 0x557cd5e2f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:51:06.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.174+0000 7f5001333640 1 --2- 172.21.15.76:0/2381404711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cd5e4d3d0 0x557cd5e4af90 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8009bb0 tx=0x7f4ff802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab3e826e7cec5f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:06.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.175+0000 7f4fdeffd640 1 -- 172.21.15.76:0/2381404711 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8003610 con 0x557cd5e4d3d0 2023-11-09T03:51:06.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.175+0000 7f4fdeffd640 1 -- 172.21.15.76:0/2381404711 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff8003770 con 0x557cd5e4d3d0 2023-11-09T03:51:06.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.175+0000 7f4fdeffd640 1 -- 172.21.15.76:0/2381404711 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8003ae0 con 0x557cd5e4d3d0 2023-11-09T03:51:06.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.175+0000 7f5006206740 1 -- 172.21.15.76:0/2381404711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cd5e4d3d0 msgr2=0x557cd5e4af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:06.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.175+0000 7f5006206740 1 --2- 172.21.15.76:0/2381404711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cd5e4d3d0 0x557cd5e4af90 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8009bb0 tx=0x7f4ff802e000 comp rx=0 tx=0).stop 2023-11-09T03:51:06.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.175+0000 7f5006206740 1 -- 172.21.15.76:0/2381404711 shutdown_connections 2023-11-09T03:51:06.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.175+0000 7f5006206740 1 --2- 172.21.15.76:0/2381404711 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557cd5e04e40 0x557cd5e2f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.175+0000 7f5006206740 1 --2- 172.21.15.76:0/2381404711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557cd5e2cc10 0x557cd5e04900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.175+0000 7f5006206740 1 --2- 172.21.15.76:0/2381404711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cd5e4d3d0 0x557cd5e4af90 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.175+0000 7f5006206740 1 -- 172.21.15.76:0/2381404711 >> 172.21.15.76:0/2381404711 conn(0x557cd5e01a60 msgr2=0x557cd5e4a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:06.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.175+0000 7f5006206740 1 -- 172.21.15.76:0/2381404711 shutdown_connections 2023-11-09T03:51:06.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.175+0000 7f5006206740 1 -- 172.21.15.76:0/2381404711 wait complete. 2023-11-09T03:51:06.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.176+0000 7f5006206740 1 -- start start 2023-11-09T03:51:06.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.176+0000 7f5006206740 1 Processor -- start 2023-11-09T03:51:06.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.177+0000 7f5006206740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557cd5e2cc10 0x557cd5e278f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.177+0000 7f5006206740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557cd5e4d3d0 0x557cd5e2f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.177+0000 7f5006206740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cd5e2fe30 0x557cd5e31fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.177+0000 7f5006206740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cd5d6b8b0 con 0x557cd5e2fe30 2023-11-09T03:51:06.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.177+0000 7f5006206740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cd5d25c80 con 0x557cd5e4d3d0 2023-11-09T03:51:06.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.177+0000 7f5006206740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x557cd5d25920 con 0x557cd5e2cc10 2023-11-09T03:51:06.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.177+0000 7f5001b34640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cd5e2fe30 0x557cd5e31fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.177+0000 7f5006206740 10 client.0 _ll_register_callbacks cb 0x557cd5a7dac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:51:06.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.177+0000 7f5001b34640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cd5e2fe30 0x557cd5e31fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45118/0 (socket says 172.21.15.76:45118) 2023-11-09T03:51:06.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.177+0000 7f5001b34640 1 -- 172.21.15.76:0/1380161399 learned_addr learned my addr 172.21.15.76:0/1380161399 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:06.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.177+0000 7f5000b32640 1 --2- 172.21.15.76:0/1380161399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557cd5e4d3d0 0x557cd5e2f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.177+0000 7f5001333640 1 --2- 172.21.15.76:0/1380161399 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557cd5e2cc10 0x557cd5e278f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.177+0000 7f5001b34640 1 -- 172.21.15.76:0/1380161399 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557cd5e2cc10 msgr2=0x557cd5e278f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:06.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.177+0000 7f5001b34640 1 --2- 172.21.15.76:0/1380161399 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557cd5e2cc10 0x557cd5e278f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.177+0000 7f5001b34640 1 -- 172.21.15.76:0/1380161399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557cd5e4d3d0 msgr2=0x557cd5e2f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:06.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.177+0000 7f5001b34640 1 --2- 172.21.15.76:0/1380161399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557cd5e4d3d0 0x557cd5e2f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.177+0000 7f5001b34640 1 -- 172.21.15.76:0/1380161399 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe8003520 con 0x557cd5e2fe30 2023-11-09T03:51:06.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.177+0000 7f5001b34640 1 --2- 172.21.15.76:0/1380161399 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cd5e2fe30 0x557cd5e31fe0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0cde50 tx=0x7f4ff4012610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=924b8e6a73ba0be4 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:06.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.177+0000 7f4fd7fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:51:06.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.177+0000 7f4fd7fff640 1 -- 172.21.15.76:0/1380161399 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff400e040 con 0x557cd5e2fe30 2023-11-09T03:51:06.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.177+0000 7f5006206740 1 -- 172.21.15.76:0/1380161399 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557cd5d17bf0 con 0x557cd5e2fe30 2023-11-09T03:51:06.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.177+0000 7f4fd7fff640 1 -- 172.21.15.76:0/1380161399 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7f4ff4003990 con 0x557cd5e2fe30 2023-11-09T03:51:06.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.178+0000 7f4fd7fff640 1 -- 172.21.15.76:0/1380161399 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ff4009e80 con 0x557cd5e2fe30 2023-11-09T03:51:06.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.178+0000 7f5006206740 10 client.8163 fetch_fsmap learned FSMap version 81 2023-11-09T03:51:06.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.178+0000 7f5006206740 1 -- 172.21.15.76:0/1380161399 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x557cd5d69f30 con 0x557cd5e2fe30 2023-11-09T03:51:06.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.178+0000 7f4fd7fff640 1 -- 172.21.15.76:0/1380161399 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557cd5d17bf0 con 0x557cd5e2fe30 2023-11-09T03:51:06.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.178+0000 7f4fd7fff640 1 -- 172.21.15.76:0/1380161399 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 81) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f4ff4003b50 con 0x557cd5e2fe30 2023-11-09T03:51:06.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.178+0000 7f5006206740 10 client.8163 fetch_fsmap finished waiting for FSMap version 81 2023-11-09T03:51:06.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.178+0000 7f5006206740 10 client.8163 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-09T03:51:06.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.178+0000 7f5006206740 20 client.8163 populate_metadata read hostname 'smithi076' 2023-11-09T03:51:06.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.178+0000 7f5006206740 1 --2- 172.21.15.76:0/1380161399 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x557cd5e6a260 0x557cd5e6c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.178+0000 7f5006206740 4 client.8163 mds_command: new command op to 7449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:51:06.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.178+0000 7f5006206740 1 -- 172.21.15.76:0/1380161399 --> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557cd5d64890 con 0x557cd5e6a260 2023-11-09T03:51:06.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.178+0000 7f5001333640 1 --2- 172.21.15.76:0/1380161399 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x557cd5e6a260 0x557cd5e6c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.178+0000 7f4fd7fff640 10 client.8163 ms_handle_connect on v2:172.21.15.76:6832/3697850101 2023-11-09T03:51:06.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.178+0000 7f5001333640 1 --2- 172.21.15.76:0/1380161399 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x557cd5e6a260 0x557cd5e6c650 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=58391cd83a46812d server_cookie=4176452a7197d32 in_seq=0 out_seq=0 2023-11-09T03:51:06.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.179+0000 7f4fd7fff640 1 -- 172.21.15.76:0/1380161399 <== mds.1 v2:172.21.15.76:6832/3697850101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557cd5d64890 con 0x557cd5e6a260 2023-11-09T03:51:06.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.179+0000 7f4fd7fff640 10 client.8163 handle_command_reply: tid=0 2023-11-09T03:51:06.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.179+0000 7f5006206740 1 client.8163 shutdown 2023-11-09T03:51:06.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.179+0000 7f5006206740 1 -- 172.21.15.76:0/1380161399 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cd5e2fe30 msgr2=0x557cd5e31fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:06.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.179+0000 7f5006206740 1 --2- 172.21.15.76:0/1380161399 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cd5e2fe30 0x557cd5e31fe0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0cde50 tx=0x7f4ff4012610 comp rx=0 tx=0).stop 2023-11-09T03:51:06.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.179+0000 7f5006206740 1 -- 172.21.15.76:0/1380161399 shutdown_connections 2023-11-09T03:51:06.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.179+0000 7f5006206740 1 --2- 172.21.15.76:0/1380161399 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x557cd5e6a260 0x557cd5e6c650 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.179+0000 7f5006206740 1 --2- 172.21.15.76:0/1380161399 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cd5e2fe30 0x557cd5e31fe0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.179+0000 7f5006206740 1 --2- 172.21.15.76:0/1380161399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557cd5e4d3d0 0x557cd5e2f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.179+0000 7f4fd7fff640 0 client.8163 ms_handle_reset on v2:172.21.15.76:6832/3697850101 2023-11-09T03:51:06.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.179+0000 7f5006206740 1 --2- 172.21.15.76:0/1380161399 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557cd5e2cc10 0x557cd5e278f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.179+0000 7f5006206740 1 -- 172.21.15.76:0/1380161399 >> 172.21.15.76:0/1380161399 conn(0x557cd5e04e40 msgr2=0x557cd5e4d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:06.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.179+0000 7f5006206740 1 -- 172.21.15.76:0/1380161399 shutdown_connections 2023-11-09T03:51:06.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.179+0000 7f5006206740 1 -- 172.21.15.76:0/1380161399 wait complete. 2023-11-09T03:51:06.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.179+0000 7f5006206740 20 client.8163 trim_cache size 0 max 16384 2023-11-09T03:51:06.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.195+0000 7f5006206740 1 Processor -- start 2023-11-09T03:51:06.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.195+0000 7f5006206740 1 -- start start 2023-11-09T03:51:06.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.195+0000 7f5006206740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cd5e2cc10 0x557cd5dffc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.195+0000 7f5006206740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557cd5e2fe30 0x557cd5e4d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.195+0000 7f5006206740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557cd5e4d3d0 0x557cd5e4dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.195+0000 7f5006206740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cd5ceb280 con 0x557cd5e2cc10 2023-11-09T03:51:06.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.195+0000 7f5006206740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cd5e27110 con 0x557cd5e2fe30 2023-11-09T03:51:06.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.195+0000 7f5006206740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x557cd5e2e980 con 0x557cd5e4d3d0 2023-11-09T03:51:06.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.195+0000 7f5001333640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cd5e2cc10 0x557cd5dffc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.196+0000 7f5001333640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cd5e2cc10 0x557cd5dffc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45132/0 (socket says 172.21.15.76:45132) 2023-11-09T03:51:06.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.196+0000 7f5001333640 1 -- 172.21.15.76:0/656543410 learned_addr learned my addr 172.21.15.76:0/656543410 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:06.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.196+0000 7f5000b32640 1 --2- 172.21.15.76:0/656543410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557cd5e2fe30 0x557cd5e4d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.196+0000 7f5001b34640 1 --2- 172.21.15.76:0/656543410 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557cd5e4d3d0 0x557cd5e4dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.196+0000 7f5001333640 1 -- 172.21.15.76:0/656543410 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557cd5e4d3d0 msgr2=0x557cd5e4dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:06.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.196+0000 7f5001333640 1 --2- 172.21.15.76:0/656543410 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557cd5e4d3d0 0x557cd5e4dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.196+0000 7f5001333640 1 -- 172.21.15.76:0/656543410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557cd5e2fe30 msgr2=0x557cd5e4d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:06.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.196+0000 7f5001333640 1 --2- 172.21.15.76:0/656543410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557cd5e2fe30 0x557cd5e4d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.196+0000 7f5001333640 1 -- 172.21.15.76:0/656543410 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cd5e32c20 con 0x557cd5e2cc10 2023-11-09T03:51:06.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.196+0000 7f5001333640 1 --2- 172.21.15.76:0/656543410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cd5e2cc10 0x557cd5dffc20 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8037040 tx=0x7f4ff803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a0d376ccb5cce5 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:06.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.196+0000 7f4fdf7fe640 1 -- 172.21.15.76:0/656543410 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff80038e0 con 0x557cd5e2cc10 2023-11-09T03:51:06.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.196+0000 7f4fdf7fe640 1 -- 172.21.15.76:0/656543410 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff8003a40 con 0x557cd5e2cc10 2023-11-09T03:51:06.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.196+0000 7f4fdf7fe640 1 -- 172.21.15.76:0/656543410 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8003db0 con 0x557cd5e2cc10 2023-11-09T03:51:06.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.196+0000 7f5006206740 1 -- 172.21.15.76:0/656543410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cd5e2cc10 msgr2=0x557cd5dffc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:06.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.196+0000 7f5006206740 1 --2- 172.21.15.76:0/656543410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cd5e2cc10 0x557cd5dffc20 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8037040 tx=0x7f4ff803e000 comp rx=0 tx=0).stop 2023-11-09T03:51:06.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.197+0000 7f5006206740 1 -- 172.21.15.76:0/656543410 shutdown_connections 2023-11-09T03:51:06.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.197+0000 7f5006206740 1 --2- 172.21.15.76:0/656543410 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557cd5e4d3d0 0x557cd5e4dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.197+0000 7f5006206740 1 --2- 172.21.15.76:0/656543410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557cd5e2fe30 0x557cd5e4d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.197+0000 7f5006206740 1 --2- 172.21.15.76:0/656543410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cd5e2cc10 0x557cd5dffc20 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.197+0000 7f5006206740 1 -- 172.21.15.76:0/656543410 >> 172.21.15.76:0/656543410 conn(0x557cd5e04e40 msgr2=0x557cd5e27700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:06.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.197+0000 7f5006206740 1 -- 172.21.15.76:0/656543410 shutdown_connections 2023-11-09T03:51:06.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.197+0000 7f5006206740 1 -- 172.21.15.76:0/656543410 wait complete. 2023-11-09T03:51:06.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.197+0000 7f5006206740 1 -- start start 2023-11-09T03:51:06.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.197+0000 7f5006206740 1 Processor -- start 2023-11-09T03:51:06.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.198+0000 7f5006206740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cd5e2cc10 0x557cd5e4a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.198+0000 7f5006206740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557cd5e2fe30 0x557cd5e4a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.198+0000 7f5006206740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557cd5e4d3d0 0x557cd5e55c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.198+0000 7f5006206740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cd5e5e240 con 0x557cd5e2cc10 2023-11-09T03:51:06.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.198+0000 7f5006206740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cd5e40750 con 0x557cd5e2fe30 2023-11-09T03:51:06.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.198+0000 7f5006206740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x557cd5e05680 con 0x557cd5e4d3d0 2023-11-09T03:51:06.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.198+0000 7f5006206740 10 client.0 _ll_register_callbacks cb 0x7f4ff8037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:51:06.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.198+0000 7f5001333640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cd5e2cc10 0x557cd5e4a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.198+0000 7f5001333640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cd5e2cc10 0x557cd5e4a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45134/0 (socket says 172.21.15.76:45134) 2023-11-09T03:51:06.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.198+0000 7f5001333640 1 -- 172.21.15.76:0/578964366 learned_addr learned my addr 172.21.15.76:0/578964366 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:06.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.198+0000 7f5000b32640 1 --2- 172.21.15.76:0/578964366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557cd5e2fe30 0x557cd5e4a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.198+0000 7f5001b34640 1 --2- 172.21.15.76:0/578964366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557cd5e4d3d0 0x557cd5e55c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.198+0000 7f5001333640 1 -- 172.21.15.76:0/578964366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557cd5e4d3d0 msgr2=0x557cd5e55c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:06.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.198+0000 7f5001333640 1 --2- 172.21.15.76:0/578964366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557cd5e4d3d0 0x557cd5e55c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.198+0000 7f5001333640 1 -- 172.21.15.76:0/578964366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557cd5e2fe30 msgr2=0x557cd5e4a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:06.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.198+0000 7f5001333640 1 --2- 172.21.15.76:0/578964366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557cd5e2fe30 0x557cd5e4a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.198+0000 7f5001333640 1 -- 172.21.15.76:0/578964366 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557cd5d2eca0 con 0x557cd5e2cc10 2023-11-09T03:51:06.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.198+0000 7f5001b34640 1 --2- 172.21.15.76:0/578964366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557cd5e4d3d0 0x557cd5e55c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:51:06.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.199+0000 7f5001333640 1 --2- 172.21.15.76:0/578964366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cd5e2cc10 0x557cd5e4a2c0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f4ff802c320 tx=0x7f4ff80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17553508d8c54930 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:06.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.199+0000 7f4fd7fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:51:06.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.199+0000 7f4fd7fff640 1 -- 172.21.15.76:0/578964366 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff80038e0 con 0x557cd5e2cc10 2023-11-09T03:51:06.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.199+0000 7f4fd7fff640 1 -- 172.21.15.76:0/578964366 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7f4ff8056b00 con 0x557cd5e2cc10 2023-11-09T03:51:06.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.199+0000 7f5006206740 1 -- 172.21.15.76:0/578964366 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ff8003770 con 0x557cd5e2cc10 2023-11-09T03:51:06.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.199+0000 7f4fd7fff640 1 -- 172.21.15.76:0/578964366 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ff8056cc0 con 0x557cd5e2cc10 2023-11-09T03:51:06.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.199+0000 7f5006206740 10 client.8169 fetch_fsmap learned FSMap version 81 2023-11-09T03:51:06.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.199+0000 7f5006206740 1 -- 172.21.15.76:0/578964366 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x557cd5d07d10 con 0x557cd5e2cc10 2023-11-09T03:51:06.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.199+0000 7f4fd7fff640 1 -- 172.21.15.76:0/578964366 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff8003770 con 0x557cd5e2cc10 2023-11-09T03:51:06.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.199+0000 7f4fd7fff640 1 -- 172.21.15.76:0/578964366 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 81) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f4ff805c020 con 0x557cd5e2cc10 2023-11-09T03:51:06.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.199+0000 7f5006206740 10 client.8169 fetch_fsmap finished waiting for FSMap version 81 2023-11-09T03:51:06.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.199+0000 7f5006206740 10 client.8169 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-09T03:51:06.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.199+0000 7f5006206740 20 client.8169 populate_metadata read hostname 'smithi076' 2023-11-09T03:51:06.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.199+0000 7f5006206740 1 --2- 172.21.15.76:0/578964366 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x557cd5ebbc90 0x557cd5e56140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.199+0000 7f5006206740 4 client.8169 mds_command: new command op to 7449 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-09T03:51:06.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.199+0000 7f5006206740 1 -- 172.21.15.76:0/578964366 --> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557cd5d645b0 con 0x557cd5ebbc90 2023-11-09T03:51:06.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.199+0000 7f5000b32640 1 --2- 172.21.15.76:0/578964366 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x557cd5ebbc90 0x557cd5e56140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.200+0000 7f5000b32640 1 --2- 172.21.15.76:0/578964366 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x557cd5ebbc90 0x557cd5e56140 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=84c7e80aad88c647 server_cookie=ea94b435e739cb4 in_seq=0 out_seq=0 2023-11-09T03:51:06.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.200+0000 7f4fd7fff640 10 client.8169 ms_handle_connect on v2:172.21.15.76:6832/3697850101 2023-11-09T03:51:06.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.200+0000 7f4fd7fff640 1 -- 172.21.15.76:0/578964366 <== mds.1 v2:172.21.15.76:6832/3697850101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x557cd5d645b0 con 0x557cd5ebbc90 2023-11-09T03:51:06.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.200+0000 7f4fd7fff640 10 client.8169 handle_command_reply: tid=0 2023-11-09T03:51:06.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.200+0000 7f5006206740 1 client.8169 shutdown 2023-11-09T03:51:06.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.200+0000 7f5006206740 1 -- 172.21.15.76:0/578964366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cd5e2cc10 msgr2=0x557cd5e4a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:06.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.200+0000 7f5006206740 1 --2- 172.21.15.76:0/578964366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cd5e2cc10 0x557cd5e4a2c0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f4ff802c320 tx=0x7f4ff80046c0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.200+0000 7f5006206740 1 -- 172.21.15.76:0/578964366 shutdown_connections 2023-11-09T03:51:06.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.200+0000 7f5006206740 1 --2- 172.21.15.76:0/578964366 >> [v2:172.21.15.76:6832/3697850101,v1:172.21.15.76:6833/3697850101] conn(0x557cd5ebbc90 0x557cd5e56140 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.200+0000 7f5006206740 1 --2- 172.21.15.76:0/578964366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557cd5e4d3d0 0x557cd5e55c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.200+0000 7f4fd7fff640 0 client.8169 ms_handle_reset on v2:172.21.15.76:6832/3697850101 2023-11-09T03:51:06.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.200+0000 7f5006206740 1 --2- 172.21.15.76:0/578964366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557cd5e2fe30 0x557cd5e4a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.200+0000 7f5006206740 1 --2- 172.21.15.76:0/578964366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557cd5e2cc10 0x557cd5e4a2c0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.200+0000 7f5006206740 1 -- 172.21.15.76:0/578964366 >> 172.21.15.76:0/578964366 conn(0x557cd5e04e40 msgr2=0x557cd5e27700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:06.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.200+0000 7f5006206740 1 -- 172.21.15.76:0/578964366 shutdown_connections 2023-11-09T03:51:06.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.200+0000 7f5006206740 1 -- 172.21.15.76:0/578964366 wait complete. 2023-11-09T03:51:06.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.200+0000 7f5006206740 20 client.8169 trim_cache size 0 max 16384 2023-11-09T03:51:06.226 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:51:06.226 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "c213b26f-d0da-45d5-8e34-bb75c096d4f2", 2023-11-09T03:51:06.226 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 1, 2023-11-09T03:51:06.227 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 7449, 2023-11-09T03:51:06.227 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2023-11-09T03:51:06.227 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2023-11-09T03:51:06.227 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2023-11-09T03:51:06.227 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 118.75348782899999, 2023-11-09T03:51:06.227 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 81, 2023-11-09T03:51:06.228 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 101, 2023-11-09T03:51:06.228 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 101, 2023-11-09T03:51:06.228 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 134.20378150799999 2023-11-09T03:51:06.228 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:51:06.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.201+0000 7f5003597640 1 -- 172.21.15.76:0/1962862600 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4fe006aea0 msgr2=0x7f4fe006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:06.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.201+0000 7f5003597640 1 --2- 172.21.15.76:0/1962862600 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4fe006aea0 0x7f4fe006d360 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8036000 tx=0x7f4ff8030000 comp rx=0 tx=0).stop 2023-11-09T03:51:06.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.201+0000 7f5003597640 1 -- 172.21.15.76:0/1962862600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ffc1e1eb0 msgr2=0x7f4ffc1530e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:06.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.201+0000 7f5003597640 1 --2- 172.21.15.76:0/1962862600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ffc1e1eb0 0x7f4ffc1530e0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f4fec000c00 tx=0x7f4fec00ec10 comp rx=0 tx=0).stop 2023-11-09T03:51:06.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.201+0000 7f5003597640 1 -- 172.21.15.76:0/1962862600 shutdown_connections 2023-11-09T03:51:06.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.201+0000 7f5003597640 1 --2- 172.21.15.76:0/1962862600 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4fe006aea0 0x7f4fe006d360 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.201+0000 7f5003597640 1 --2- 172.21.15.76:0/1962862600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ffc1e2310 0x7f4ffc153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.201+0000 7f5003597640 1 --2- 172.21.15.76:0/1962862600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ffc1e1eb0 0x7f4ffc1530e0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.201+0000 7f5003597640 1 --2- 172.21.15.76:0/1962862600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fe8003970 0x7f4ffc0ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.201+0000 7f5003597640 1 -- 172.21.15.76:0/1962862600 >> 172.21.15.76:0/1962862600 conn(0x7f4ffc0ce950 msgr2=0x7f4ffc0c7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:06.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.201+0000 7f5003597640 1 -- 172.21.15.76:0/1962862600 shutdown_connections 2023-11-09T03:51:06.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.201+0000 7f5003597640 1 -- 172.21.15.76:0/1962862600 wait complete. 2023-11-09T03:51:06.232 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-09T03:51:06.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.332+0000 7f171a1e7640 1 Processor -- start 2023-11-09T03:51:06.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.332+0000 7f171a1e7640 1 -- start start 2023-11-09T03:51:06.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.332+0000 7f171a1e7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17140c4ff0 0x7f17140cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.332+0000 7f171a1e7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f17140adac0 0x7f17140adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.332+0000 7f171a1e7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f171415aff0 con 0x7f1714159c00 2023-11-09T03:51:06.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.333+0000 7f171a1e7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17140ae450 con 0x7f17140adac0 2023-11-09T03:51:06.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.333+0000 7f171a1e7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171415a3e0 con 0x7f17140c4ff0 2023-11-09T03:51:06.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.333+0000 7f1713fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f17140adac0 0x7f17140adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.333+0000 7f17137fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f1714159c00 0x7f17140c48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39544/0 (socket says 172.21.15.76:39544) 2023-11-09T03:51:06.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.333+0000 7f1713fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f17140adac0 0x7f17140adf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45144/0 (socket says 172.21.15.76:45144) 2023-11-09T03:51:06.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.333+0000 7f17137fe640 1 -- 172.21.15.76:0/3828932317 learned_addr learned my addr 172.21.15.76:0/3828932317 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:51:06.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.333+0000 7f1712ffd640 1 --2- 172.21.15.76:0/3828932317 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17140c4ff0 0x7f17140cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.333+0000 7f1713fff640 1 -- 172.21.15.76:0/3828932317 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17140c4ff0 msgr2=0x7f17140cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:06.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.333+0000 7f1713fff640 1 --2- 172.21.15.76:0/3828932317 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17140c4ff0 0x7f17140cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.333+0000 7f1713fff640 1 -- 172.21.15.76:0/3828932317 >> v1:172.21.15.76:6789/0 conn(0x7f1714159c00 legacy=0x7f17140c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:06.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.333+0000 7f17127fc640 1 -- 172.21.15.76:0/3828932317 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 425134129 0 0) 0x7f171415aff0 con 0x7f1714159c00 2023-11-09T03:51:06.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.333+0000 7f1713fff640 1 -- 172.21.15.76:0/3828932317 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171415a6c0 con 0x7f17140adac0 2023-11-09T03:51:06.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.334+0000 7f1713fff640 1 --2- 172.21.15.76:0/3828932317 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17140adac0 0x7f17140adf10 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f170c009d90 tx=0x7f170c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e11b848628d20 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:06.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.334+0000 7f17127fc640 1 -- 172.21.15.76:0/3828932317 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c02cc50 con 0x7f17140adac0 2023-11-09T03:51:06.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.334+0000 7f17127fc640 1 -- 172.21.15.76:0/3828932317 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170c02cdb0 con 0x7f17140adac0 2023-11-09T03:51:06.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.334+0000 7f17127fc640 1 -- 172.21.15.76:0/3828932317 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c034660 con 0x7f17140adac0 2023-11-09T03:51:06.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.334+0000 7f171a1e7640 1 -- 172.21.15.76:0/3828932317 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17140adac0 msgr2=0x7f17140adf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:06.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.334+0000 7f171a1e7640 1 --2- 172.21.15.76:0/3828932317 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17140adac0 0x7f17140adf10 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f170c009d90 tx=0x7f170c02bea0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.334+0000 7f171a1e7640 1 -- 172.21.15.76:0/3828932317 shutdown_connections 2023-11-09T03:51:06.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.334+0000 7f171a1e7640 1 --2- 172.21.15.76:0/3828932317 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17140adac0 0x7f17140adf10 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.334+0000 7f171a1e7640 1 --2- 172.21.15.76:0/3828932317 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17140c4ff0 0x7f17140cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.334+0000 7f171a1e7640 1 -- 172.21.15.76:0/3828932317 >> 172.21.15.76:0/3828932317 conn(0x7f17140ce950 msgr2=0x7f17140ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:06.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.334+0000 7f171a1e7640 1 -- 172.21.15.76:0/3828932317 shutdown_connections 2023-11-09T03:51:06.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.334+0000 7f171a1e7640 1 -- 172.21.15.76:0/3828932317 wait complete. 2023-11-09T03:51:06.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.336+0000 7f171a1e7640 1 Processor -- start 2023-11-09T03:51:06.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.336+0000 7f171a1e7640 1 -- start start 2023-11-09T03:51:06.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.336+0000 7f171a1e7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17140adac0 0x7f1714147400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.336+0000 7f171a1e7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17140c4ff0 0x7f17140b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.336+0000 7f171a1e7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1714159c00 0x7f17140c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.336+0000 7f171a1e7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17141575a0 con 0x7f17140c4ff0 2023-11-09T03:51:06.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.336+0000 7f171a1e7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17140ae6e0 con 0x7f1714159c00 2023-11-09T03:51:06.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.336+0000 7f171a1e7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17140ae9c0 con 0x7f17140adac0 2023-11-09T03:51:06.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.337+0000 7f1712ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17140c4ff0 0x7f17140b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.337+0000 7f1712ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17140c4ff0 0x7f17140b1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45156/0 (socket says 172.21.15.76:45156) 2023-11-09T03:51:06.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.337+0000 7f1712ffd640 1 -- 172.21.15.76:0/4209228895 learned_addr learned my addr 172.21.15.76:0/4209228895 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:06.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.337+0000 7f17137fe640 1 --2- 172.21.15.76:0/4209228895 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17140adac0 0x7f1714147400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.337+0000 7f1713fff640 1 --2- 172.21.15.76:0/4209228895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1714159c00 0x7f17140c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.337+0000 7f1712ffd640 1 -- 172.21.15.76:0/4209228895 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17140adac0 msgr2=0x7f1714147400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:06.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.337+0000 7f1712ffd640 1 --2- 172.21.15.76:0/4209228895 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17140adac0 0x7f1714147400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.337+0000 7f1712ffd640 1 -- 172.21.15.76:0/4209228895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1714159c00 msgr2=0x7f17140c7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:06.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.337+0000 7f1712ffd640 1 --2- 172.21.15.76:0/4209228895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1714159c00 0x7f17140c7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.337+0000 7f1712ffd640 1 -- 172.21.15.76:0/4209228895 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170c009a70 con 0x7f17140c4ff0 2023-11-09T03:51:06.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.337+0000 7f17137fe640 1 --2- 172.21.15.76:0/4209228895 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17140adac0 0x7f1714147400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:51:06.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.337+0000 7f1713fff640 1 --2- 172.21.15.76:0/4209228895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1714159c00 0x7f17140c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:51:06.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.337+0000 7f1712ffd640 1 --2- 172.21.15.76:0/4209228895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17140c4ff0 0x7f17140b1ee0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f170800d7d0 tx=0x7f170800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:06.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.337+0000 7f16fbfff640 1 -- 172.21.15.76:0/4209228895 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1708018070 con 0x7f17140c4ff0 2023-11-09T03:51:06.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.337+0000 7f16fbfff640 1 -- 172.21.15.76:0/4209228895 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1708004020 con 0x7f17140c4ff0 2023-11-09T03:51:06.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.337+0000 7f171a1e7640 1 -- 172.21.15.76:0/4209228895 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17140aeca0 con 0x7f17140c4ff0 2023-11-09T03:51:06.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.337+0000 7f16fbfff640 1 -- 172.21.15.76:0/4209228895 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17080041b0 con 0x7f17140c4ff0 2023-11-09T03:51:06.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.337+0000 7f171a1e7640 1 -- 172.21.15.76:0/4209228895 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1714147940 con 0x7f17140c4ff0 2023-11-09T03:51:06.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.338+0000 7f16fbfff640 1 -- 172.21.15.76:0/4209228895 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1708004310 con 0x7f17140c4ff0 2023-11-09T03:51:06.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.339+0000 7f16fbfff640 1 --2- 172.21.15.76:0/4209228895 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f170006ae70 0x7f170006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.339+0000 7f16fbfff640 1 -- 172.21.15.76:0/4209228895 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7f1708089bb0 con 0x7f17140c4ff0 2023-11-09T03:51:06.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.339+0000 7f17137fe640 1 --2- 172.21.15.76:0/4209228895 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f170006ae70 0x7f170006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.340+0000 7f17137fe640 1 --2- 172.21.15.76:0/4209228895 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f170006ae70 0x7f170006d330 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1704000f60 tx=0x7f1704007ca0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:06.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.340+0000 7f171cd7b740 1 Processor -- start 2023-11-09T03:51:06.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.340+0000 7f171cd7b740 1 -- start start 2023-11-09T03:51:06.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.341+0000 7f171cd7b740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fdedc2850 0x555fded95750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.341+0000 7f171cd7b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555fdedc2c30 0x555fdedc55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.341+0000 7f171cd7b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fded95c90 0x555fdedc5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.341+0000 7f171cd7b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdeca6740 con 0x555fdedc2c30 2023-11-09T03:51:06.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.341+0000 7f171cd7b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fded088d0 con 0x555fded95c90 2023-11-09T03:51:06.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.341+0000 7f171cd7b740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x555fded05fc0 con 0x555fdedc2850 2023-11-09T03:51:06.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.341+0000 7f1712ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555fdedc2c30 0x555fdedc55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.341+0000 7f1712ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555fdedc2c30 0x555fdedc55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45160/0 (socket says 172.21.15.76:45160) 2023-11-09T03:51:06.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.341+0000 7f1712ffd640 1 -- 172.21.15.76:0/1106986771 learned_addr learned my addr 172.21.15.76:0/1106986771 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:06.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.341+0000 7f17137fe640 1 --2- 172.21.15.76:0/1106986771 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fdedc2850 0x555fded95750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.341+0000 7f1713fff640 1 --2- 172.21.15.76:0/1106986771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fded95c90 0x555fdedc5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.341+0000 7f1712ffd640 1 -- 172.21.15.76:0/1106986771 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fdedc2850 msgr2=0x555fded95750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:06.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.341+0000 7f1712ffd640 1 --2- 172.21.15.76:0/1106986771 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fdedc2850 0x555fded95750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.341+0000 7f1712ffd640 1 -- 172.21.15.76:0/1106986771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fded95c90 msgr2=0x555fdedc5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:06.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.341+0000 7f1712ffd640 1 --2- 172.21.15.76:0/1106986771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fded95c90 0x555fdedc5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.341+0000 7f1712ffd640 1 -- 172.21.15.76:0/1106986771 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fdecc4ca0 con 0x555fdedc2c30 2023-11-09T03:51:06.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.341+0000 7f1713fff640 1 --2- 172.21.15.76:0/1106986771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fded95c90 0x555fdedc5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:51:06.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.341+0000 7f1712ffd640 1 --2- 172.21.15.76:0/1106986771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555fdedc2c30 0x555fdedc55e0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f1708060060 tx=0x7f1708058a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82307c9171c485ff server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:06.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.342+0000 7f16f97fa640 1 -- 172.21.15.76:0/1106986771 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170805c1b0 con 0x555fdedc2c30 2023-11-09T03:51:06.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.342+0000 7f16f97fa640 1 -- 172.21.15.76:0/1106986771 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1708060590 con 0x555fdedc2c30 2023-11-09T03:51:06.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.342+0000 7f16f97fa640 1 -- 172.21.15.76:0/1106986771 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17080549a0 con 0x555fdedc2c30 2023-11-09T03:51:06.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.342+0000 7f171cd7b740 1 -- 172.21.15.76:0/1106986771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555fdedc2c30 msgr2=0x555fdedc55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:06.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.342+0000 7f171cd7b740 1 --2- 172.21.15.76:0/1106986771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555fdedc2c30 0x555fdedc55e0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f1708060060 tx=0x7f1708058a50 comp rx=0 tx=0).stop 2023-11-09T03:51:06.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.342+0000 7f171cd7b740 1 -- 172.21.15.76:0/1106986771 shutdown_connections 2023-11-09T03:51:06.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.342+0000 7f171cd7b740 1 --2- 172.21.15.76:0/1106986771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fded95c90 0x555fdedc5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.342+0000 7f171cd7b740 1 --2- 172.21.15.76:0/1106986771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555fdedc2c30 0x555fdedc55e0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.342+0000 7f171cd7b740 1 --2- 172.21.15.76:0/1106986771 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fdedc2850 0x555fded95750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.342+0000 7f171cd7b740 1 -- 172.21.15.76:0/1106986771 >> 172.21.15.76:0/1106986771 conn(0x555fded97a60 msgr2=0x555fdede05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:06.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.342+0000 7f171cd7b740 1 -- 172.21.15.76:0/1106986771 shutdown_connections 2023-11-09T03:51:06.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.342+0000 7f171cd7b740 1 -- 172.21.15.76:0/1106986771 wait complete. 2023-11-09T03:51:06.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.343+0000 7f171cd7b740 1 -- start start 2023-11-09T03:51:06.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.343+0000 7f171cd7b740 1 Processor -- start 2023-11-09T03:51:06.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.344+0000 7f171cd7b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555fdedc2850 0x555fdedb5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.344+0000 7f171cd7b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fdedc2c30 0x555fdedd8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.344+0000 7f171cd7b740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fdedc8d30 0x555fdedd9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.344+0000 7f171cd7b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fded018b0 con 0x555fdedc2850 2023-11-09T03:51:06.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.344+0000 7f171cd7b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdecbbc80 con 0x555fdedc2c30 2023-11-09T03:51:06.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.344+0000 7f171cd7b740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x555fdecbb920 con 0x555fdedc8d30 2023-11-09T03:51:06.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.344+0000 7f171cd7b740 10 client.0 _ll_register_callbacks cb 0x555fdea13ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:51:06.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.344+0000 7f17137fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555fdedc2850 0x555fdedb5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.344+0000 7f17137fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555fdedc2850 0x555fdedb5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45170/0 (socket says 172.21.15.76:45170) 2023-11-09T03:51:06.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.344+0000 7f17137fe640 1 -- 172.21.15.76:0/2648840618 learned_addr learned my addr 172.21.15.76:0/2648840618 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:06.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.344+0000 7f1713fff640 1 --2- 172.21.15.76:0/2648840618 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fdedc8d30 0x555fdedd9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.344+0000 7f1712ffd640 1 --2- 172.21.15.76:0/2648840618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fdedc2c30 0x555fdedd8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.344+0000 7f17137fe640 1 -- 172.21.15.76:0/2648840618 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fdedc8d30 msgr2=0x555fdedd9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:06.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.344+0000 7f17137fe640 1 --2- 172.21.15.76:0/2648840618 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fdedc8d30 0x555fdedd9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.344+0000 7f17137fe640 1 -- 172.21.15.76:0/2648840618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fdedc2c30 msgr2=0x555fdedd8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:06.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.344+0000 7f17137fe640 1 --2- 172.21.15.76:0/2648840618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fdedc2c30 0x555fdedd8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.344+0000 7f17137fe640 1 -- 172.21.15.76:0/2648840618 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1714147940 con 0x555fdedc2850 2023-11-09T03:51:06.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.344+0000 7f17137fe640 1 --2- 172.21.15.76:0/2648840618 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555fdedc2850 0x555fdedb5b00 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f1704001d80 tx=0x7f1704011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba35b62acfc175a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:06.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.344+0000 7f16f27fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:51:06.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.344+0000 7f16f27fc640 1 -- 172.21.15.76:0/2648840618 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1704015e50 con 0x555fdedc2850 2023-11-09T03:51:06.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.344+0000 7f16f27fc640 1 -- 172.21.15.76:0/2648840618 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7f1704013420 con 0x555fdedc2850 2023-11-09T03:51:06.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.344+0000 7f171cd7b740 1 -- 172.21.15.76:0/2648840618 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555fdecadbf0 con 0x555fdedc2850 2023-11-09T03:51:06.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.344+0000 7f16f27fc640 1 -- 172.21.15.76:0/2648840618 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17040135e0 con 0x555fdedc2850 2023-11-09T03:51:06.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.344+0000 7f171cd7b740 10 client.8184 fetch_fsmap learned FSMap version 81 2023-11-09T03:51:06.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.344+0000 7f171cd7b740 1 -- 172.21.15.76:0/2648840618 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x555fdecfff30 con 0x555fdedc2850 2023-11-09T03:51:06.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.345+0000 7f16f27fc640 1 -- 172.21.15.76:0/2648840618 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555fdecadbf0 con 0x555fdedc2850 2023-11-09T03:51:06.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.345+0000 7f16f27fc640 1 -- 172.21.15.76:0/2648840618 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 81) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f1704013740 con 0x555fdedc2850 2023-11-09T03:51:06.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.345+0000 7f171cd7b740 10 client.8184 fetch_fsmap finished waiting for FSMap version 81 2023-11-09T03:51:06.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.345+0000 7f171cd7b740 10 client.8184 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-09T03:51:06.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.345+0000 7f171cd7b740 20 client.8184 populate_metadata read hostname 'smithi076' 2023-11-09T03:51:06.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.345+0000 7f171cd7b740 1 --2- 172.21.15.76:0/2648840618 >> [v2:172.21.15.76:6834/3229239133,v1:172.21.15.76:6835/3229239133] conn(0x555fdedc7150 0x555fdedc7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.345+0000 7f171cd7b740 4 client.8184 mds_command: new command op to 8016 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:51:06.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.345+0000 7f171cd7b740 1 -- 172.21.15.76:0/2648840618 --> [v2:172.21.15.76:6834/3229239133,v1:172.21.15.76:6835/3229239133] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555fdecfa890 con 0x555fdedc7150 2023-11-09T03:51:06.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.345+0000 7f1712ffd640 1 --2- 172.21.15.76:0/2648840618 >> [v2:172.21.15.76:6834/3229239133,v1:172.21.15.76:6835/3229239133] conn(0x555fdedc7150 0x555fdedc7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.345+0000 7f1712ffd640 1 --2- 172.21.15.76:0/2648840618 >> [v2:172.21.15.76:6834/3229239133,v1:172.21.15.76:6835/3229239133] conn(0x555fdedc7150 0x555fdedc7530 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f540e6ff1afcdc0 server_cookie=e343c2bade3f74a9 in_seq=0 out_seq=0 2023-11-09T03:51:06.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.345+0000 7f16f27fc640 10 client.8184 ms_handle_connect on v2:172.21.15.76:6834/3229239133 2023-11-09T03:51:06.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.346+0000 7f16f27fc640 1 -- 172.21.15.76:0/2648840618 <== mds.0 v2:172.21.15.76:6834/3229239133 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555fdecfa890 con 0x555fdedc7150 2023-11-09T03:51:06.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.346+0000 7f16f27fc640 10 client.8184 handle_command_reply: tid=0 2023-11-09T03:51:06.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.346+0000 7f171cd7b740 1 client.8184 shutdown 2023-11-09T03:51:06.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.346+0000 7f171cd7b740 1 -- 172.21.15.76:0/2648840618 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555fdedc2850 msgr2=0x555fdedb5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:06.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.346+0000 7f171cd7b740 1 --2- 172.21.15.76:0/2648840618 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555fdedc2850 0x555fdedb5b00 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f1704001d80 tx=0x7f1704011000 comp rx=0 tx=0).stop 2023-11-09T03:51:06.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.346+0000 7f171cd7b740 1 -- 172.21.15.76:0/2648840618 shutdown_connections 2023-11-09T03:51:06.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.346+0000 7f171cd7b740 1 --2- 172.21.15.76:0/2648840618 >> [v2:172.21.15.76:6834/3229239133,v1:172.21.15.76:6835/3229239133] conn(0x555fdedc7150 0x555fdedc7530 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.346+0000 7f171cd7b740 1 --2- 172.21.15.76:0/2648840618 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fdedc8d30 0x555fdedd9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.346+0000 7f171cd7b740 1 --2- 172.21.15.76:0/2648840618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fdedc2c30 0x555fdedd8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.346+0000 7f16f27fc640 0 client.8184 ms_handle_reset on v2:172.21.15.76:6834/3229239133 2023-11-09T03:51:06.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.346+0000 7f171cd7b740 1 --2- 172.21.15.76:0/2648840618 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555fdedc2850 0x555fdedb5b00 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.346+0000 7f171cd7b740 1 -- 172.21.15.76:0/2648840618 >> 172.21.15.76:0/2648840618 conn(0x555fded95c90 msgr2=0x555fded944f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:06.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.346+0000 7f171cd7b740 1 -- 172.21.15.76:0/2648840618 shutdown_connections 2023-11-09T03:51:06.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.346+0000 7f171cd7b740 1 -- 172.21.15.76:0/2648840618 wait complete. 2023-11-09T03:51:06.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.346+0000 7f171cd7b740 20 client.8184 trim_cache size 0 max 16384 2023-11-09T03:51:06.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.361+0000 7f171cd7b740 1 Processor -- start 2023-11-09T03:51:06.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.361+0000 7f171cd7b740 1 -- start start 2023-11-09T03:51:06.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.362+0000 7f171cd7b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fdedc2850 0x555fded99770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.362+0000 7f171cd7b740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fdedc2c30 0x555fded944f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.362+0000 7f171cd7b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555fdedc8d30 0x555fded94a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.362+0000 7f171cd7b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdec81280 con 0x555fdedc8d30 2023-11-09T03:51:06.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.362+0000 7f171cd7b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdedb52b0 con 0x555fdedc2850 2023-11-09T03:51:06.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.362+0000 7f171cd7b740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x555fded975a0 con 0x555fdedc2c30 2023-11-09T03:51:06.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.362+0000 7f1713fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555fdedc8d30 0x555fded94a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.362+0000 7f1713fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555fdedc8d30 0x555fded94a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45174/0 (socket says 172.21.15.76:45174) 2023-11-09T03:51:06.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.362+0000 7f1713fff640 1 -- 172.21.15.76:0/730928208 learned_addr learned my addr 172.21.15.76:0/730928208 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:06.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.363+0000 7f1712ffd640 1 --2- 172.21.15.76:0/730928208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fdedc2c30 0x555fded944f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.363+0000 7f17137fe640 1 --2- 172.21.15.76:0/730928208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fdedc2850 0x555fded99770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.363+0000 7f1713fff640 1 -- 172.21.15.76:0/730928208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fdedc2c30 msgr2=0x555fded944f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:06.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.363+0000 7f1713fff640 1 --2- 172.21.15.76:0/730928208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fdedc2c30 0x555fded944f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.363+0000 7f1713fff640 1 -- 172.21.15.76:0/730928208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fdedc2850 msgr2=0x555fded99770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:06.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.363+0000 7f1713fff640 1 --2- 172.21.15.76:0/730928208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fdedc2850 0x555fded99770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.363+0000 7f1713fff640 1 -- 172.21.15.76:0/730928208 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fdec9dd10 con 0x555fdedc8d30 2023-11-09T03:51:06.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.363+0000 7f1713fff640 1 --2- 172.21.15.76:0/730928208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555fdedc8d30 0x555fded94a30 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f170c009eb0 tx=0x7f170c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c36003549b7ce8e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:06.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.363+0000 7f16f9ffb640 1 -- 172.21.15.76:0/730928208 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c034640 con 0x555fdedc8d30 2023-11-09T03:51:06.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.363+0000 7f16f9ffb640 1 -- 172.21.15.76:0/730928208 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170c0347a0 con 0x555fdedc8d30 2023-11-09T03:51:06.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.363+0000 7f16f9ffb640 1 -- 172.21.15.76:0/730928208 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c034b10 con 0x555fdedc8d30 2023-11-09T03:51:06.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.364+0000 7f171cd7b740 1 -- 172.21.15.76:0/730928208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555fdedc8d30 msgr2=0x555fded94a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:06.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.364+0000 7f171cd7b740 1 --2- 172.21.15.76:0/730928208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555fdedc8d30 0x555fded94a30 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f170c009eb0 tx=0x7f170c02e000 comp rx=0 tx=0).stop 2023-11-09T03:51:06.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.364+0000 7f171cd7b740 1 -- 172.21.15.76:0/730928208 shutdown_connections 2023-11-09T03:51:06.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.364+0000 7f171cd7b740 1 --2- 172.21.15.76:0/730928208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555fdedc8d30 0x555fded94a30 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.364+0000 7f171cd7b740 1 --2- 172.21.15.76:0/730928208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fdedc2c30 0x555fded944f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.364+0000 7f171cd7b740 1 --2- 172.21.15.76:0/730928208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fdedc2850 0x555fded99770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.364+0000 7f171cd7b740 1 -- 172.21.15.76:0/730928208 >> 172.21.15.76:0/730928208 conn(0x555fded95c90 msgr2=0x555fdedd8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:06.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.364+0000 7f171cd7b740 1 -- 172.21.15.76:0/730928208 shutdown_connections 2023-11-09T03:51:06.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.364+0000 7f171cd7b740 1 -- 172.21.15.76:0/730928208 wait complete. 2023-11-09T03:51:06.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.364+0000 7f171cd7b740 1 -- start start 2023-11-09T03:51:06.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.364+0000 7f171cd7b740 1 Processor -- start 2023-11-09T03:51:06.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.365+0000 7f171cd7b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fdedc2850 0x555fdedd8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.365+0000 7f171cd7b740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fdedc2c30 0x555fdedeefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.365+0000 7f171cd7b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555fdedc8d30 0x555fdee4a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.365+0000 7f171cd7b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdedc50f0 con 0x555fdedc8d30 2023-11-09T03:51:06.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.365+0000 7f171cd7b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdedc56b0 con 0x555fdedc2850 2023-11-09T03:51:06.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.365+0000 7f171cd7b740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x555fdedd93b0 con 0x555fdedc2c30 2023-11-09T03:51:06.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.365+0000 7f171cd7b740 10 client.0 _ll_register_callbacks cb 0x7f17080545d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:51:06.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.365+0000 7f1713fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555fdedc8d30 0x555fdee4a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.365+0000 7f1713fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555fdedc8d30 0x555fdee4a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45176/0 (socket says 172.21.15.76:45176) 2023-11-09T03:51:06.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.365+0000 7f1713fff640 1 -- 172.21.15.76:0/1660430986 learned_addr learned my addr 172.21.15.76:0/1660430986 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:06.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.365+0000 7f17137fe640 1 --2- 172.21.15.76:0/1660430986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fdedc2850 0x555fdedd8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.365+0000 7f1712ffd640 1 --2- 172.21.15.76:0/1660430986 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fdedc2c30 0x555fdedeefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.365+0000 7f1713fff640 1 -- 172.21.15.76:0/1660430986 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fdedc2c30 msgr2=0x555fdedeefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:06.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.365+0000 7f1713fff640 1 --2- 172.21.15.76:0/1660430986 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fdedc2c30 0x555fdedeefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.365+0000 7f1713fff640 1 -- 172.21.15.76:0/1660430986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fdedc2850 msgr2=0x555fdedd8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:06.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.365+0000 7f1713fff640 1 --2- 172.21.15.76:0/1660430986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fdedc2850 0x555fdedd8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.365+0000 7f1713fff640 1 -- 172.21.15.76:0/1660430986 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f171415a6c0 con 0x555fdedc8d30 2023-11-09T03:51:06.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.366+0000 7f1713fff640 1 --2- 172.21.15.76:0/1660430986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555fdedc8d30 0x555fdee4a420 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f170c02c4b0 tx=0x7f170c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5cf78f638e1e270 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:06.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.366+0000 7f16f27fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:51:06.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.366+0000 7f16f27fc640 1 -- 172.21.15.76:0/1660430986 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c052630 con 0x555fdedc8d30 2023-11-09T03:51:06.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.366+0000 7f16f27fc640 1 -- 172.21.15.76:0/1660430986 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7f170c052790 con 0x555fdedc8d30 2023-11-09T03:51:06.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.366+0000 7f171cd7b740 1 -- 172.21.15.76:0/1660430986 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1708060590 con 0x555fdedc8d30 2023-11-09T03:51:06.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.366+0000 7f16f27fc640 1 -- 172.21.15.76:0/1660430986 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f170c052950 con 0x555fdedc8d30 2023-11-09T03:51:06.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.366+0000 7f171cd7b740 10 client.8190 fetch_fsmap learned FSMap version 81 2023-11-09T03:51:06.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.366+0000 7f171cd7b740 1 -- 172.21.15.76:0/1660430986 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x555fdecc3500 con 0x555fdedc8d30 2023-11-09T03:51:06.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.366+0000 7f16f27fc640 1 -- 172.21.15.76:0/1660430986 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1708060590 con 0x555fdedc8d30 2023-11-09T03:51:06.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.366+0000 7f16f27fc640 1 -- 172.21.15.76:0/1660430986 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 81) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f170c052ab0 con 0x555fdedc8d30 2023-11-09T03:51:06.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.366+0000 7f171cd7b740 10 client.8190 fetch_fsmap finished waiting for FSMap version 81 2023-11-09T03:51:06.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.366+0000 7f171cd7b740 10 client.8190 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-09T03:51:06.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.366+0000 7f171cd7b740 20 client.8190 populate_metadata read hostname 'smithi076' 2023-11-09T03:51:06.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.366+0000 7f171cd7b740 1 --2- 172.21.15.76:0/1660430986 >> [v2:172.21.15.76:6834/3229239133,v1:172.21.15.76:6835/3229239133] conn(0x555fdedc7670 0x555fdedc7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.366+0000 7f171cd7b740 4 client.8190 mds_command: new command op to 8016 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-09T03:51:06.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.366+0000 7f171cd7b740 1 -- 172.21.15.76:0/1660430986 --> [v2:172.21.15.76:6834/3229239133,v1:172.21.15.76:6835/3229239133] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555fdecfa5b0 con 0x555fdedc7670 2023-11-09T03:51:06.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.366+0000 7f17137fe640 1 --2- 172.21.15.76:0/1660430986 >> [v2:172.21.15.76:6834/3229239133,v1:172.21.15.76:6835/3229239133] conn(0x555fdedc7670 0x555fdedc7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.367+0000 7f16f27fc640 10 client.8190 ms_handle_connect on v2:172.21.15.76:6834/3229239133 2023-11-09T03:51:06.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.367+0000 7f17137fe640 1 --2- 172.21.15.76:0/1660430986 >> [v2:172.21.15.76:6834/3229239133,v1:172.21.15.76:6835/3229239133] conn(0x555fdedc7670 0x555fdedc7a50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8fa135710327aca server_cookie=58746ab624439d96 in_seq=0 out_seq=0 2023-11-09T03:51:06.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.367+0000 7f16f27fc640 1 -- 172.21.15.76:0/1660430986 <== mds.0 v2:172.21.15.76:6834/3229239133 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x555fdecfa5b0 con 0x555fdedc7670 2023-11-09T03:51:06.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.367+0000 7f16f27fc640 10 client.8190 handle_command_reply: tid=0 2023-11-09T03:51:06.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.367+0000 7f171cd7b740 1 client.8190 shutdown 2023-11-09T03:51:06.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.367+0000 7f171cd7b740 1 -- 172.21.15.76:0/1660430986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555fdedc8d30 msgr2=0x555fdee4a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:06.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.367+0000 7f171cd7b740 1 --2- 172.21.15.76:0/1660430986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555fdedc8d30 0x555fdee4a420 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f170c02c4b0 tx=0x7f170c004ab0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.367+0000 7f171cd7b740 1 -- 172.21.15.76:0/1660430986 shutdown_connections 2023-11-09T03:51:06.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.367+0000 7f171cd7b740 1 --2- 172.21.15.76:0/1660430986 >> [v2:172.21.15.76:6834/3229239133,v1:172.21.15.76:6835/3229239133] conn(0x555fdedc7670 0x555fdedc7a50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.367+0000 7f171cd7b740 1 --2- 172.21.15.76:0/1660430986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555fdedc8d30 0x555fdee4a420 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.367+0000 7f171cd7b740 1 --2- 172.21.15.76:0/1660430986 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fdedc2c30 0x555fdedeefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.367+0000 7f16f27fc640 0 client.8190 ms_handle_reset on v2:172.21.15.76:6834/3229239133 2023-11-09T03:51:06.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.367+0000 7f171cd7b740 1 --2- 172.21.15.76:0/1660430986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fdedc2850 0x555fdedd8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.367+0000 7f171cd7b740 1 -- 172.21.15.76:0/1660430986 >> 172.21.15.76:0/1660430986 conn(0x555fded95c90 msgr2=0x555fded99770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:06.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.367+0000 7f171cd7b740 1 -- 172.21.15.76:0/1660430986 shutdown_connections 2023-11-09T03:51:06.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.367+0000 7f171cd7b740 1 -- 172.21.15.76:0/1660430986 wait complete. 2023-11-09T03:51:06.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.367+0000 7f171cd7b740 20 client.8190 trim_cache size 0 max 16384 2023-11-09T03:51:06.390 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:51:06.390 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "c213b26f-d0da-45d5-8e34-bb75c096d4f2", 2023-11-09T03:51:06.390 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2023-11-09T03:51:06.390 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 8016, 2023-11-09T03:51:06.390 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2023-11-09T03:51:06.391 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2023-11-09T03:51:06.391 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2023-11-09T03:51:06.391 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 6.7513188199999998, 2023-11-09T03:51:06.391 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 81, 2023-11-09T03:51:06.391 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 101, 2023-11-09T03:51:06.391 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 101, 2023-11-09T03:51:06.392 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 32.603519294999998 2023-11-09T03:51:06.392 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:51:06.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.367+0000 7f171a1e7640 1 -- 172.21.15.76:0/4209228895 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f170006ae70 msgr2=0x7f170006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:06.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.368+0000 7f171a1e7640 1 --2- 172.21.15.76:0/4209228895 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f170006ae70 0x7f170006d330 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1704000f60 tx=0x7f1704007ca0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.368+0000 7f171a1e7640 1 -- 172.21.15.76:0/4209228895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17140c4ff0 msgr2=0x7f17140b1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:06.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.368+0000 7f171a1e7640 1 --2- 172.21.15.76:0/4209228895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17140c4ff0 0x7f17140b1ee0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f170800d7d0 tx=0x7f170800dc90 comp rx=0 tx=0).stop 2023-11-09T03:51:06.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.368+0000 7f171a1e7640 1 -- 172.21.15.76:0/4209228895 shutdown_connections 2023-11-09T03:51:06.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.368+0000 7f171a1e7640 1 --2- 172.21.15.76:0/4209228895 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f170006ae70 0x7f170006d330 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.368+0000 7f171a1e7640 1 --2- 172.21.15.76:0/4209228895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1714159c00 0x7f17140c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.368+0000 7f171a1e7640 1 --2- 172.21.15.76:0/4209228895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17140c4ff0 0x7f17140b1ee0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.368+0000 7f171a1e7640 1 --2- 172.21.15.76:0/4209228895 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17140adac0 0x7f1714147400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.368+0000 7f171a1e7640 1 -- 172.21.15.76:0/4209228895 >> 172.21.15.76:0/4209228895 conn(0x7f17140ce950 msgr2=0x7f171406ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:06.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.368+0000 7f171a1e7640 1 -- 172.21.15.76:0/4209228895 shutdown_connections 2023-11-09T03:51:06.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.368+0000 7f171a1e7640 1 -- 172.21.15.76:0/4209228895 wait complete. 2023-11-09T03:51:06.395 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:51:06.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.495+0000 7f4a93755640 1 Processor -- start 2023-11-09T03:51:06.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.496+0000 7f4a93755640 1 -- start start 2023-11-09T03:51:06.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.496+0000 7f4a93755640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4a8c0cde60 0x7f4a8c15d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.496+0000 7f4a93755640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4a8c0ce240 0x7f4a8c0d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.496+0000 7f4a93755640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a8c15e000 con 0x7f4a8c15daa0 2023-11-09T03:51:06.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.496+0000 7f4a93755640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a8c15e5c0 con 0x7f4a8c0cde60 2023-11-09T03:51:06.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.496+0000 7f4a93755640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a8c15e2e0 con 0x7f4a8c0ce240 2023-11-09T03:51:06.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.497+0000 7f4a914f1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4a8c0cde60 0x7f4a8c15d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.497+0000 7f4a914f1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4a8c0cde60 0x7f4a8c15d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45192/0 (socket says 172.21.15.76:45192) 2023-11-09T03:51:06.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.497+0000 7f4a914f1640 1 -- 172.21.15.76:0/1617363917 learned_addr learned my addr 172.21.15.76:0/1617363917 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:06.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.497+0000 7f4a91cf2640 1 --2- 172.21.15.76:0/1617363917 >> v2:172.21.15.113:3300/0 conn(0x7f4a8c0ce240 0x7f4a8c0d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.497+0000 7f4a914f1640 1 -- 172.21.15.76:0/1617363917 >> v2:172.21.15.113:3300/0 conn(0x7f4a8c0ce240 msgr2=0x7f4a8c0d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:06.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.497+0000 7f4a914f1640 1 --2- 172.21.15.76:0/1617363917 >> v2:172.21.15.113:3300/0 conn(0x7f4a8c0ce240 0x7f4a8c0d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.497+0000 7f4a914f1640 1 -- 172.21.15.76:0/1617363917 >> v1:172.21.15.113:6789/0 conn(0x7f4a8c15daa0 legacy=0x7f4a8c157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:06.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.497+0000 7f4a914f1640 1 -- 172.21.15.76:0/1617363917 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8c0d1120 con 0x7f4a8c0cde60 2023-11-09T03:51:06.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.498+0000 7f4a914f1640 1 --2- 172.21.15.76:0/1617363917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a8c0cde60 0x7f4a8c15d560 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0cf2e0 tx=0x7f4a8802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2f01ba517cc74d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:06.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.498+0000 7f4a83fff640 1 -- 172.21.15.76:0/1617363917 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8802ca80 con 0x7f4a8c0cde60 2023-11-09T03:51:06.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.498+0000 7f4a83fff640 1 -- 172.21.15.76:0/1617363917 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a8802cbe0 con 0x7f4a8c0cde60 2023-11-09T03:51:06.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.498+0000 7f4a83fff640 1 -- 172.21.15.76:0/1617363917 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a880343f0 con 0x7f4a8c0cde60 2023-11-09T03:51:06.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.498+0000 7f4a93755640 1 -- 172.21.15.76:0/1617363917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a8c0cde60 msgr2=0x7f4a8c15d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:06.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.498+0000 7f4a93755640 1 --2- 172.21.15.76:0/1617363917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a8c0cde60 0x7f4a8c15d560 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0cf2e0 tx=0x7f4a8802bee0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.498+0000 7f4a93755640 1 -- 172.21.15.76:0/1617363917 shutdown_connections 2023-11-09T03:51:06.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.498+0000 7f4a93755640 1 --2- 172.21.15.76:0/1617363917 >> v2:172.21.15.113:3300/0 conn(0x7f4a8c0ce240 0x7f4a8c0d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.498+0000 7f4a93755640 1 --2- 172.21.15.76:0/1617363917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a8c0cde60 0x7f4a8c15d560 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.498+0000 7f4a93755640 1 -- 172.21.15.76:0/1617363917 >> 172.21.15.76:0/1617363917 conn(0x7f4a8c071f30 msgr2=0x7f4a8c0cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:06.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.498+0000 7f4a93755640 1 -- 172.21.15.76:0/1617363917 shutdown_connections 2023-11-09T03:51:06.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.498+0000 7f4a93755640 1 -- 172.21.15.76:0/1617363917 wait complete. 2023-11-09T03:51:06.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.499+0000 7f4a93755640 1 Processor -- start 2023-11-09T03:51:06.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.499+0000 7f4a93755640 1 -- start start 2023-11-09T03:51:06.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.500+0000 7f4a93755640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a8c0cde60 0x7f4a8c154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.500+0000 7f4a93755640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a8c0ce240 0x7f4a8c0b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.500+0000 7f4a93755640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a8c15daa0 0x7f4a8c0b7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.500+0000 7f4a93755640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c1e6c90 con 0x7f4a8c0ce240 2023-11-09T03:51:06.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.500+0000 7f4a93755640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c1e5e40 con 0x7f4a8c0cde60 2023-11-09T03:51:06.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.500+0000 7f4a93755640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c1e6120 con 0x7f4a8c15daa0 2023-11-09T03:51:06.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.500+0000 7f4a90cf0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a8c0ce240 0x7f4a8c0b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.500+0000 7f4a90cf0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a8c0ce240 0x7f4a8c0b77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45208/0 (socket says 172.21.15.76:45208) 2023-11-09T03:51:06.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.500+0000 7f4a90cf0640 1 -- 172.21.15.76:0/1055431587 learned_addr learned my addr 172.21.15.76:0/1055431587 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:06.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.500+0000 7f4a914f1640 1 --2- 172.21.15.76:0/1055431587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a8c0cde60 0x7f4a8c154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.500+0000 7f4a91cf2640 1 --2- 172.21.15.76:0/1055431587 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a8c15daa0 0x7f4a8c0b7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.501+0000 7f4a90cf0640 1 -- 172.21.15.76:0/1055431587 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a8c15daa0 msgr2=0x7f4a8c0b7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:06.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.501+0000 7f4a90cf0640 1 --2- 172.21.15.76:0/1055431587 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a8c15daa0 0x7f4a8c0b7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.501+0000 7f4a90cf0640 1 -- 172.21.15.76:0/1055431587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a8c0cde60 msgr2=0x7f4a8c154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:06.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.501+0000 7f4a90cf0640 1 --2- 172.21.15.76:0/1055431587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a8c0cde60 0x7f4a8c154c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.501+0000 7f4a90cf0640 1 -- 172.21.15.76:0/1055431587 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a88009b30 con 0x7f4a8c0ce240 2023-11-09T03:51:06.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.501+0000 7f4a90cf0640 1 --2- 172.21.15.76:0/1055431587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a8c0ce240 0x7f4a8c0b77b0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00b3c0 tx=0x7f4a7c00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:06.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.501+0000 7f4a81ffb640 1 -- 172.21.15.76:0/1055431587 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c011830 con 0x7f4a8c0ce240 2023-11-09T03:51:06.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.501+0000 7f4a81ffb640 1 -- 172.21.15.76:0/1055431587 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7c011990 con 0x7f4a8c0ce240 2023-11-09T03:51:06.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.501+0000 7f4a93755640 1 -- 172.21.15.76:0/1055431587 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a8c1e6400 con 0x7f4a8c0ce240 2023-11-09T03:51:06.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.501+0000 7f4a81ffb640 1 -- 172.21.15.76:0/1055431587 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c011d80 con 0x7f4a8c0ce240 2023-11-09T03:51:06.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.501+0000 7f4a93755640 1 -- 172.21.15.76:0/1055431587 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a8c1551b0 con 0x7f4a8c0ce240 2023-11-09T03:51:06.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.502+0000 7f4a93755640 1 -- 172.21.15.76:0/1055431587 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a8c1e6e40 con 0x7f4a8c0ce240 2023-11-09T03:51:06.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.502+0000 7f4a81ffb640 1 -- 172.21.15.76:0/1055431587 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4a7c01b460 con 0x7f4a8c0ce240 2023-11-09T03:51:06.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.503+0000 7f4a81ffb640 1 --2- 172.21.15.76:0/1055431587 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4a7006ae70 0x7f4a7006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:06.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.503+0000 7f4a81ffb640 1 -- 172.21.15.76:0/1055431587 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7f4a7c0896e0 con 0x7f4a8c0ce240 2023-11-09T03:51:06.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.503+0000 7f4a914f1640 1 --2- 172.21.15.76:0/1055431587 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4a7006ae70 0x7f4a7006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:06.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.503+0000 7f4a914f1640 1 --2- 172.21.15.76:0/1055431587 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4a7006ae70 0x7f4a7006d330 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4a88035000 tx=0x7f4a88030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:06.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.507+0000 7f4a81ffb640 1 -- 172.21.15.76:0/1055431587 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4a8c1e6e40 con 0x7f4a8c0ce240 2023-11-09T03:51:06.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.708+0000 7f4a93755640 1 -- 172.21.15.76:0/1055431587 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a8c074290 con 0x7f4a8c0ce240 2023-11-09T03:51:06.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.708+0000 7f4a81ffb640 1 -- 172.21.15.76:0/1055431587 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4932 (secure 0 0 0) 0x7f4a7c054120 con 0x7f4a8c0ce240 2023-11-09T03:51:06.709 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.710 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":81,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6904,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:51:02.634881+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8016,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8016":{"gid":8016,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3229239133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3229239133},{"type":"v1","addr":"172.21.15.76:6835","nonce":3229239133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:51:06.710 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 81 2023-11-09T03:51:06.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.710+0000 7f4a93755640 1 -- 172.21.15.76:0/1055431587 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4a7006ae70 msgr2=0x7f4a7006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:06.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.710+0000 7f4a93755640 1 --2- 172.21.15.76:0/1055431587 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4a7006ae70 0x7f4a7006d330 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4a88035000 tx=0x7f4a88030000 comp rx=0 tx=0).stop 2023-11-09T03:51:06.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.710+0000 7f4a93755640 1 -- 172.21.15.76:0/1055431587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a8c0ce240 msgr2=0x7f4a8c0b77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:06.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.710+0000 7f4a93755640 1 --2- 172.21.15.76:0/1055431587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a8c0ce240 0x7f4a8c0b77b0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00b3c0 tx=0x7f4a7c00b880 comp rx=0 tx=0).stop 2023-11-09T03:51:06.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.710+0000 7f4a93755640 1 -- 172.21.15.76:0/1055431587 shutdown_connections 2023-11-09T03:51:06.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.710+0000 7f4a93755640 1 --2- 172.21.15.76:0/1055431587 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4a7006ae70 0x7f4a7006d330 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.710+0000 7f4a93755640 1 --2- 172.21.15.76:0/1055431587 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4a8c15daa0 0x7f4a8c0b7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.710+0000 7f4a93755640 1 --2- 172.21.15.76:0/1055431587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a8c0ce240 0x7f4a8c0b77b0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.710+0000 7f4a93755640 1 --2- 172.21.15.76:0/1055431587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a8c0cde60 0x7f4a8c154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:06.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.710+0000 7f4a93755640 1 -- 172.21.15.76:0/1055431587 >> 172.21.15.76:0/1055431587 conn(0x7f4a8c071f30 msgr2=0x7f4a8c0b6500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:06.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.710+0000 7f4a93755640 1 -- 172.21.15.76:0/1055431587 shutdown_connections 2023-11-09T03:51:06.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:06.710+0000 7f4a93755640 1 -- 172.21.15.76:0/1055431587 wait complete. 2023-11-09T03:51:06.722 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2023-11-09T03:51:06.835 INFO:teuthology.orchestra.run.smithi076.stdout:{"last_snap":5,"last_created":5,"last_destroyed":1,"pending_noop":[],"snaps":[{"snapid":2,"ino":1099511627777,"stamp":"2023-11-09T03:49:19.643623+0000","name":"s11","metadata":{}},{"snapid":3,"ino":1099511627777,"stamp":"2023-11-09T03:49:50.605488+0000","name":"s12","metadata":{}},{"snapid":4,"ino":1099511627777,"stamp":"2023-11-09T03:50:16.615058+0000","name":"s15","metadata":{}},{"snapid":5,"ino":1099511627777,"stamp":"2023-11-09T03:50:42.606497+0000","name":"s16","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-09T03:51:06.836 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 5, "last_destroyed": 1, "last_snap": 5, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s11", "snapid": 2, "stamp": "2023-11-09T03:49:19.643623+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s12", "snapid": 3, "stamp": "2023-11-09T03:49:50.605488+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s15", "snapid": 4, "stamp": "2023-11-09T03:50:16.615058+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s16", "snapid": 5, "stamp": "2023-11-09T03:50:42.606497+0000" } ] } 2023-11-09T03:51:06.836 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-09T03:51:06.837 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-09T03:51:06.840 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-09T03:51:06.889 INFO:teuthology.orchestra.run.smithi076.stdout:[global] 2023-11-09T03:51:06.889 INFO:teuthology.orchestra.run.smithi076.stdout: chdir = "" 2023-11-09T03:51:06.889 INFO:teuthology.orchestra.run.smithi076.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-09T03:51:06.889 INFO:teuthology.orchestra.run.smithi076.stdout: auth supported = cephx 2023-11-09T03:51:06.889 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.890 INFO:teuthology.orchestra.run.smithi076.stdout: filestore xattr use omap = true 2023-11-09T03:51:06.890 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.890 INFO:teuthology.orchestra.run.smithi076.stdout: mon clock drift allowed = 1.000 2023-11-09T03:51:06.890 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.891 INFO:teuthology.orchestra.run.smithi076.stdout: osd crush chooseleaf type = 0 2023-11-09T03:51:06.891 INFO:teuthology.orchestra.run.smithi076.stdout: auth debug = true 2023-11-09T03:51:06.891 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.891 INFO:teuthology.orchestra.run.smithi076.stdout: ms die on old message = true 2023-11-09T03:51:06.891 INFO:teuthology.orchestra.run.smithi076.stdout: ms die on bug = true 2023-11-09T03:51:06.891 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.892 INFO:teuthology.orchestra.run.smithi076.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-09T03:51:06.892 INFO:teuthology.orchestra.run.smithi076.stdout: mon pg warn max object skew = 0 2023-11-09T03:51:06.892 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.892 INFO:teuthology.orchestra.run.smithi076.stdout: # disable pg_autoscaler by default for new pools 2023-11-09T03:51:06.893 INFO:teuthology.orchestra.run.smithi076.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-09T03:51:06.893 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.893 INFO:teuthology.orchestra.run.smithi076.stdout: osd pool default size = 2 2023-11-09T03:51:06.893 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.893 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd allow primary affinity = true 2023-11-09T03:51:06.894 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd allow pg remap = true 2023-11-09T03:51:06.894 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on legacy crush tunables = false 2023-11-09T03:51:06.894 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on crush straw calc version zero = false 2023-11-09T03:51:06.894 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on no sortbitwise = false 2023-11-09T03:51:06.894 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on osd down out interval zero = false 2023-11-09T03:51:06.895 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on too few osds = false 2023-11-09T03:51:06.895 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-09T03:51:06.895 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-09T03:51:06.895 INFO:teuthology.orchestra.run.smithi076.stdout: mon_allow_pool_size_one = true 2023-11-09T03:51:06.896 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.896 INFO:teuthology.orchestra.run.smithi076.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-09T03:51:06.896 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.896 INFO:teuthology.orchestra.run.smithi076.stdout: osd default data pool replay window = 5 2023-11-09T03:51:06.896 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.897 INFO:teuthology.orchestra.run.smithi076.stdout: mon allow pool delete = true 2023-11-09T03:51:06.897 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.897 INFO:teuthology.orchestra.run.smithi076.stdout: mon cluster log file level = debug 2023-11-09T03:51:06.897 INFO:teuthology.orchestra.run.smithi076.stdout: debug asserts on shutdown = true 2023-11-09T03:51:06.897 INFO:teuthology.orchestra.run.smithi076.stdout: mon health detail to clog = false 2023-11-09T03:51:06.898 INFO:teuthology.orchestra.run.smithi076.stdout: mon host = "172.21.15.76,172.21.15.113,[v2:172.21.15.113:3301,v1:172.21.15.113:6790]" 2023-11-09T03:51:06.898 INFO:teuthology.orchestra.run.smithi076.stdout: fsid = c213b26f-d0da-45d5-8e34-bb75c096d4f2 2023-11-09T03:51:06.898 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.898 INFO:teuthology.orchestra.run.smithi076.stdout:[osd] 2023-11-09T03:51:06.899 INFO:teuthology.orchestra.run.smithi076.stdout: osd journal size = 100 2023-11-09T03:51:06.899 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.899 INFO:teuthology.orchestra.run.smithi076.stdout: osd scrub load threshold = 5.0 2023-11-09T03:51:06.899 INFO:teuthology.orchestra.run.smithi076.stdout: osd scrub max interval = 600 2023-11-09T03:51:06.899 INFO:teuthology.orchestra.run.smithi076.stdout: osd mclock profile = high_recovery_ops 2023-11-09T03:51:06.900 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.900 INFO:teuthology.orchestra.run.smithi076.stdout: osd recover clone overlap = true 2023-11-09T03:51:06.900 INFO:teuthology.orchestra.run.smithi076.stdout: osd recovery max chunk = 1048576 2023-11-09T03:51:06.900 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.901 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug shutdown = true 2023-11-09T03:51:06.901 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug op order = true 2023-11-09T03:51:06.901 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug verify stray on activate = true 2023-11-09T03:51:06.901 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug trim objects = true 2023-11-09T03:51:06.901 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.902 INFO:teuthology.orchestra.run.smithi076.stdout: osd open classes on start = true 2023-11-09T03:51:06.902 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug pg log writeout = true 2023-11-09T03:51:06.902 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.902 INFO:teuthology.orchestra.run.smithi076.stdout: osd deep scrub update digest min age = 30 2023-11-09T03:51:06.902 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.903 INFO:teuthology.orchestra.run.smithi076.stdout: osd map max advance = 10 2023-11-09T03:51:06.903 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.903 INFO:teuthology.orchestra.run.smithi076.stdout: journal zero on create = true 2023-11-09T03:51:06.903 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.903 INFO:teuthology.orchestra.run.smithi076.stdout: filestore ondisk finisher threads = 3 2023-11-09T03:51:06.904 INFO:teuthology.orchestra.run.smithi076.stdout: filestore apply finisher threads = 3 2023-11-09T03:51:06.904 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.904 INFO:teuthology.orchestra.run.smithi076.stdout: bdev debug aio = true 2023-11-09T03:51:06.904 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug misdirected ops = true 2023-11-09T03:51:06.904 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore block size = 96636764160 2023-11-09T03:51:06.905 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore fsck on mount = True 2023-11-09T03:51:06.905 INFO:teuthology.orchestra.run.smithi076.stdout: debug bluefs = 20 2023-11-09T03:51:06.905 INFO:teuthology.orchestra.run.smithi076.stdout: debug bluestore = 20 2023-11-09T03:51:06.905 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2023-11-09T03:51:06.905 INFO:teuthology.orchestra.run.smithi076.stdout: debug osd = 20 2023-11-09T03:51:06.906 INFO:teuthology.orchestra.run.smithi076.stdout: debug rocksdb = 10 2023-11-09T03:51:06.906 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd backfillfull_ratio = 0.85 2023-11-09T03:51:06.906 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd full ratio = 0.9 2023-11-09T03:51:06.906 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd nearfull ratio = 0.8 2023-11-09T03:51:06.906 INFO:teuthology.orchestra.run.smithi076.stdout: osd failsafe full ratio = 0.95 2023-11-09T03:51:06.907 INFO:teuthology.orchestra.run.smithi076.stdout: osd objectstore = bluestore 2023-11-09T03:51:06.907 INFO:teuthology.orchestra.run.smithi076.stdout: osd op complaint time = 180 2023-11-09T03:51:06.907 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.907 INFO:teuthology.orchestra.run.smithi076.stdout:[mgr] 2023-11-09T03:51:06.907 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2023-11-09T03:51:06.907 INFO:teuthology.orchestra.run.smithi076.stdout: debug mgr = 20 2023-11-09T03:51:06.908 INFO:teuthology.orchestra.run.smithi076.stdout: debug mon = 20 2023-11-09T03:51:06.908 INFO:teuthology.orchestra.run.smithi076.stdout: debug auth = 20 2023-11-09T03:51:06.908 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min pgs per osd = 4 2023-11-09T03:51:06.908 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min bytes per osd = 10 2023-11-09T03:51:06.908 INFO:teuthology.orchestra.run.smithi076.stdout: mgr/telemetry/nag = false 2023-11-09T03:51:06.909 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.909 INFO:teuthology.orchestra.run.smithi076.stdout:[mon] 2023-11-09T03:51:06.909 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2023-11-09T03:51:06.909 INFO:teuthology.orchestra.run.smithi076.stdout: debug mon = 20 2023-11-09T03:51:06.909 INFO:teuthology.orchestra.run.smithi076.stdout: debug paxos = 20 2023-11-09T03:51:06.910 INFO:teuthology.orchestra.run.smithi076.stdout: debug auth = 20 2023-11-09T03:51:06.910 INFO:teuthology.orchestra.run.smithi076.stdout: mon data avail warn = 5 2023-11-09T03:51:06.910 INFO:teuthology.orchestra.run.smithi076.stdout: mon mgr mkfs grace = 240 2023-11-09T03:51:06.910 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min pgs per osd = 4 2023-11-09T03:51:06.910 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd reporter subtree level = osd 2023-11-09T03:51:06.910 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd prime pg temp = true 2023-11-09T03:51:06.911 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min bytes per osd = 10 2023-11-09T03:51:06.911 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.911 INFO:teuthology.orchestra.run.smithi076.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-09T03:51:06.911 INFO:teuthology.orchestra.run.smithi076.stdout: auth mon ticket ttl = 660 # 11m 2023-11-09T03:51:06.911 INFO:teuthology.orchestra.run.smithi076.stdout: auth service ticket ttl = 240 # 4m 2023-11-09T03:51:06.912 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.912 INFO:teuthology.orchestra.run.smithi076.stdout: # don't complain about insecure global_id in the test suite 2023-11-09T03:51:06.912 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-09T03:51:06.912 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-09T03:51:06.912 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.913 INFO:teuthology.orchestra.run.smithi076.stdout: # 1m isn't quite enough 2023-11-09T03:51:06.913 INFO:teuthology.orchestra.run.smithi076.stdout: mon_down_mkfs_grace = 2m 2023-11-09T03:51:06.913 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.913 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_filestore_osds = false 2023-11-09T03:51:06.913 INFO:teuthology.orchestra.run.smithi076.stdout: mon op complaint time = 120 2023-11-09T03:51:06.913 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:06.914 INFO:teuthology.orchestra.run.smithi076.stdout:[client] 2023-11-09T03:51:06.914 INFO:teuthology.orchestra.run.smithi076.stdout: rgw cache enabled = true 2023-11-09T03:51:06.914 INFO:teuthology.orchestra.run.smithi076.stdout: rgw enable ops log = true 2023-11-09T03:51:06.914 INFO:teuthology.orchestra.run.smithi076.stdout: rgw enable usage log = true 2023-11-09T03:51:06.914 INFO:teuthology.orchestra.run.smithi076.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-09T03:51:06.915 INFO:teuthology.orchestra.run.smithi076.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-09T03:51:06.915 INFO:teuthology.orchestra.run.smithi076.stdout: client mount timeout = 600 2023-11-09T03:51:06.915 INFO:teuthology.orchestra.run.smithi076.stdout: debug client = 20 2023-11-09T03:51:06.915 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2023-11-09T03:51:06.915 INFO:teuthology.orchestra.run.smithi076.stdout: pid file = "" 2023-11-09T03:51:06.915 INFO:teuthology.orchestra.run.smithi076.stdout: rados mon op timeout = 900 2023-11-09T03:51:06.916 INFO:teuthology.orchestra.run.smithi076.stdout: rados osd op timeout = 900 2023-11-09T03:51:06.916 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.a] 2023-11-09T03:51:06.916 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.b] 2023-11-09T03:51:06.916 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.c] 2023-11-09T03:51:06.916 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.a] 2023-11-09T03:51:06.916 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.c] 2023-11-09T03:51:06.917 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.b] 2023-11-09T03:51:06.917 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.d] 2023-11-09T03:51:06.917 INFO:teuthology.orchestra.run.smithi076.stdout:[mds] 2023-11-09T03:51:06.917 INFO:teuthology.orchestra.run.smithi076.stdout: debug mds = 20 2023-11-09T03:51:06.917 INFO:teuthology.orchestra.run.smithi076.stdout: debug mds balancer = 20 2023-11-09T03:51:06.918 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2023-11-09T03:51:06.918 INFO:teuthology.orchestra.run.smithi076.stdout: mds debug frag = True 2023-11-09T03:51:06.918 INFO:teuthology.orchestra.run.smithi076.stdout: mds debug scatterstat = True 2023-11-09T03:51:06.918 INFO:teuthology.orchestra.run.smithi076.stdout: mds op complaint time = 180 2023-11-09T03:51:06.918 INFO:teuthology.orchestra.run.smithi076.stdout: mds verify scatter = True 2023-11-09T03:51:06.918 INFO:teuthology.orchestra.run.smithi076.stdout: osd op complaint time = 180 2023-11-09T03:51:06.919 INFO:teuthology.orchestra.run.smithi076.stdout: rados mon op timeout = 900 2023-11-09T03:51:06.919 INFO:teuthology.orchestra.run.smithi076.stdout: rados osd op timeout = 900 2023-11-09T03:51:06.919 INFO:teuthology.orchestra.run.smithi076.stdout: mds_reconnect_timeout = 5 2023-11-09T03:51:06.920 INFO:teuthology.orchestra.run.smithi113.stdout:[global] 2023-11-09T03:51:06.921 INFO:teuthology.orchestra.run.smithi113.stdout: chdir = "" 2023-11-09T03:51:06.921 INFO:teuthology.orchestra.run.smithi113.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-09T03:51:06.921 INFO:teuthology.orchestra.run.smithi113.stdout: auth supported = cephx 2023-11-09T03:51:06.921 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.921 INFO:teuthology.orchestra.run.smithi113.stdout: filestore xattr use omap = true 2023-11-09T03:51:06.921 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.922 INFO:teuthology.orchestra.run.smithi113.stdout: mon clock drift allowed = 1.000 2023-11-09T03:51:06.922 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.922 INFO:teuthology.orchestra.run.smithi113.stdout: osd crush chooseleaf type = 0 2023-11-09T03:51:06.922 INFO:teuthology.orchestra.run.smithi113.stdout: auth debug = true 2023-11-09T03:51:06.922 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.923 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on old message = true 2023-11-09T03:51:06.923 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on bug = true 2023-11-09T03:51:06.923 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.923 INFO:teuthology.orchestra.run.smithi113.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-09T03:51:06.923 INFO:teuthology.orchestra.run.smithi113.stdout: mon pg warn max object skew = 0 2023-11-09T03:51:06.924 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.924 INFO:teuthology.orchestra.run.smithi113.stdout: # disable pg_autoscaler by default for new pools 2023-11-09T03:51:06.924 INFO:teuthology.orchestra.run.smithi113.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-09T03:51:06.924 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.924 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default size = 2 2023-11-09T03:51:06.925 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.925 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow primary affinity = true 2023-11-09T03:51:06.925 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow pg remap = true 2023-11-09T03:51:06.925 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on legacy crush tunables = false 2023-11-09T03:51:06.925 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on crush straw calc version zero = false 2023-11-09T03:51:06.925 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on no sortbitwise = false 2023-11-09T03:51:06.926 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on osd down out interval zero = false 2023-11-09T03:51:06.926 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on too few osds = false 2023-11-09T03:51:06.926 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-09T03:51:06.926 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-09T03:51:06.926 INFO:teuthology.orchestra.run.smithi113.stdout: mon_allow_pool_size_one = true 2023-11-09T03:51:06.927 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.927 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-09T03:51:06.927 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.927 INFO:teuthology.orchestra.run.smithi113.stdout: osd default data pool replay window = 5 2023-11-09T03:51:06.927 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.928 INFO:teuthology.orchestra.run.smithi113.stdout: mon allow pool delete = true 2023-11-09T03:51:06.928 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.928 INFO:teuthology.orchestra.run.smithi113.stdout: mon cluster log file level = debug 2023-11-09T03:51:06.928 INFO:teuthology.orchestra.run.smithi113.stdout: debug asserts on shutdown = true 2023-11-09T03:51:06.928 INFO:teuthology.orchestra.run.smithi113.stdout: mon health detail to clog = false 2023-11-09T03:51:06.928 INFO:teuthology.orchestra.run.smithi113.stdout: mon host = "172.21.15.76,172.21.15.113,[v2:172.21.15.113:3301,v1:172.21.15.113:6790]" 2023-11-09T03:51:06.929 INFO:teuthology.orchestra.run.smithi113.stdout: fsid = c213b26f-d0da-45d5-8e34-bb75c096d4f2 2023-11-09T03:51:06.929 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.929 INFO:teuthology.orchestra.run.smithi113.stdout:[osd] 2023-11-09T03:51:06.929 INFO:teuthology.orchestra.run.smithi113.stdout: osd journal size = 100 2023-11-09T03:51:06.929 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.929 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub load threshold = 5.0 2023-11-09T03:51:06.930 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub max interval = 600 2023-11-09T03:51:06.930 INFO:teuthology.orchestra.run.smithi113.stdout: osd mclock profile = high_recovery_ops 2023-11-09T03:51:06.930 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.930 INFO:teuthology.orchestra.run.smithi113.stdout: osd recover clone overlap = true 2023-11-09T03:51:06.930 INFO:teuthology.orchestra.run.smithi113.stdout: osd recovery max chunk = 1048576 2023-11-09T03:51:06.931 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.931 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug shutdown = true 2023-11-09T03:51:06.931 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug op order = true 2023-11-09T03:51:06.931 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug verify stray on activate = true 2023-11-09T03:51:06.931 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug trim objects = true 2023-11-09T03:51:06.931 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.932 INFO:teuthology.orchestra.run.smithi113.stdout: osd open classes on start = true 2023-11-09T03:51:06.932 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug pg log writeout = true 2023-11-09T03:51:06.932 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.932 INFO:teuthology.orchestra.run.smithi113.stdout: osd deep scrub update digest min age = 30 2023-11-09T03:51:06.932 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.932 INFO:teuthology.orchestra.run.smithi113.stdout: osd map max advance = 10 2023-11-09T03:51:06.933 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.933 INFO:teuthology.orchestra.run.smithi113.stdout: journal zero on create = true 2023-11-09T03:51:06.933 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.933 INFO:teuthology.orchestra.run.smithi113.stdout: filestore ondisk finisher threads = 3 2023-11-09T03:51:06.933 INFO:teuthology.orchestra.run.smithi113.stdout: filestore apply finisher threads = 3 2023-11-09T03:51:06.934 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.934 INFO:teuthology.orchestra.run.smithi113.stdout: bdev debug aio = true 2023-11-09T03:51:06.934 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug misdirected ops = true 2023-11-09T03:51:06.934 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore block size = 96636764160 2023-11-09T03:51:06.934 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore fsck on mount = True 2023-11-09T03:51:06.935 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluefs = 20 2023-11-09T03:51:06.935 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluestore = 20 2023-11-09T03:51:06.935 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2023-11-09T03:51:06.935 INFO:teuthology.orchestra.run.smithi113.stdout: debug osd = 20 2023-11-09T03:51:06.935 INFO:teuthology.orchestra.run.smithi113.stdout: debug rocksdb = 10 2023-11-09T03:51:06.936 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd backfillfull_ratio = 0.85 2023-11-09T03:51:06.936 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd full ratio = 0.9 2023-11-09T03:51:06.936 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd nearfull ratio = 0.8 2023-11-09T03:51:06.936 INFO:teuthology.orchestra.run.smithi113.stdout: osd failsafe full ratio = 0.95 2023-11-09T03:51:06.936 INFO:teuthology.orchestra.run.smithi113.stdout: osd objectstore = bluestore 2023-11-09T03:51:06.936 INFO:teuthology.orchestra.run.smithi113.stdout: osd op complaint time = 180 2023-11-09T03:51:06.937 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.937 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr] 2023-11-09T03:51:06.937 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2023-11-09T03:51:06.937 INFO:teuthology.orchestra.run.smithi113.stdout: debug mgr = 20 2023-11-09T03:51:06.937 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2023-11-09T03:51:06.937 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2023-11-09T03:51:06.938 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2023-11-09T03:51:06.938 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2023-11-09T03:51:06.938 INFO:teuthology.orchestra.run.smithi113.stdout: mgr/telemetry/nag = false 2023-11-09T03:51:06.938 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.938 INFO:teuthology.orchestra.run.smithi113.stdout:[mon] 2023-11-09T03:51:06.939 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2023-11-09T03:51:06.939 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2023-11-09T03:51:06.939 INFO:teuthology.orchestra.run.smithi113.stdout: debug paxos = 20 2023-11-09T03:51:06.939 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2023-11-09T03:51:06.939 INFO:teuthology.orchestra.run.smithi113.stdout: mon data avail warn = 5 2023-11-09T03:51:06.939 INFO:teuthology.orchestra.run.smithi113.stdout: mon mgr mkfs grace = 240 2023-11-09T03:51:06.940 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2023-11-09T03:51:06.940 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd reporter subtree level = osd 2023-11-09T03:51:06.940 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd prime pg temp = true 2023-11-09T03:51:06.940 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2023-11-09T03:51:06.940 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.940 INFO:teuthology.orchestra.run.smithi113.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-09T03:51:06.941 INFO:teuthology.orchestra.run.smithi113.stdout: auth mon ticket ttl = 660 # 11m 2023-11-09T03:51:06.941 INFO:teuthology.orchestra.run.smithi113.stdout: auth service ticket ttl = 240 # 4m 2023-11-09T03:51:06.941 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.941 INFO:teuthology.orchestra.run.smithi113.stdout: # don't complain about insecure global_id in the test suite 2023-11-09T03:51:06.941 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-09T03:51:06.942 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-09T03:51:06.942 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.942 INFO:teuthology.orchestra.run.smithi113.stdout: # 1m isn't quite enough 2023-11-09T03:51:06.942 INFO:teuthology.orchestra.run.smithi113.stdout: mon_down_mkfs_grace = 2m 2023-11-09T03:51:06.943 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.943 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_filestore_osds = false 2023-11-09T03:51:06.943 INFO:teuthology.orchestra.run.smithi113.stdout: mon op complaint time = 120 2023-11-09T03:51:06.943 INFO:teuthology.orchestra.run.smithi113.stdout: 2023-11-09T03:51:06.944 INFO:teuthology.orchestra.run.smithi113.stdout:[client] 2023-11-09T03:51:06.944 INFO:teuthology.orchestra.run.smithi113.stdout: rgw cache enabled = true 2023-11-09T03:51:06.944 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable ops log = true 2023-11-09T03:51:06.944 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable usage log = true 2023-11-09T03:51:06.944 INFO:teuthology.orchestra.run.smithi113.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-09T03:51:06.945 INFO:teuthology.orchestra.run.smithi113.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-09T03:51:06.945 INFO:teuthology.orchestra.run.smithi113.stdout: client mount timeout = 600 2023-11-09T03:51:06.945 INFO:teuthology.orchestra.run.smithi113.stdout: debug client = 20 2023-11-09T03:51:06.945 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2023-11-09T03:51:06.946 INFO:teuthology.orchestra.run.smithi113.stdout: pid file = "" 2023-11-09T03:51:06.946 INFO:teuthology.orchestra.run.smithi113.stdout: rados mon op timeout = 900 2023-11-09T03:51:06.946 INFO:teuthology.orchestra.run.smithi113.stdout: rados osd op timeout = 900 2023-11-09T03:51:06.946 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.a] 2023-11-09T03:51:06.946 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.b] 2023-11-09T03:51:06.946 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.c] 2023-11-09T03:51:06.947 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.a] 2023-11-09T03:51:06.947 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.c] 2023-11-09T03:51:06.947 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.b] 2023-11-09T03:51:06.947 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.d] 2023-11-09T03:51:06.947 INFO:teuthology.orchestra.run.smithi113.stdout:[mds] 2023-11-09T03:51:06.948 INFO:teuthology.orchestra.run.smithi113.stdout: debug mds = 20 2023-11-09T03:51:06.948 INFO:teuthology.orchestra.run.smithi113.stdout: debug mds balancer = 20 2023-11-09T03:51:06.948 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2023-11-09T03:51:06.948 INFO:teuthology.orchestra.run.smithi113.stdout: mds debug frag = True 2023-11-09T03:51:06.948 INFO:teuthology.orchestra.run.smithi113.stdout: mds debug scatterstat = True 2023-11-09T03:51:06.949 INFO:teuthology.orchestra.run.smithi113.stdout: mds op complaint time = 180 2023-11-09T03:51:06.949 INFO:teuthology.orchestra.run.smithi113.stdout: mds verify scatter = True 2023-11-09T03:51:06.949 INFO:teuthology.orchestra.run.smithi113.stdout: osd op complaint time = 180 2023-11-09T03:51:06.949 INFO:teuthology.orchestra.run.smithi113.stdout: rados mon op timeout = 900 2023-11-09T03:51:06.949 INFO:teuthology.orchestra.run.smithi113.stdout: rados osd op timeout = 900 2023-11-09T03:51:06.949 INFO:teuthology.orchestra.run.smithi113.stdout: mds_reconnect_timeout = 5 2023-11-09T03:51:06.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:51:06.950 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir d1/dir/.snap/*) 2023-11-09T03:51:06.995 INFO:tasks.cephfs.test_snapshots:testing snapserver mds_kill_mdstable_at=2, also kill snapclient 2023-11-09T03:51:06.996 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:51:07.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.095+0000 7f68ea155640 1 Processor -- start 2023-11-09T03:51:07.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.095+0000 7f68ea155640 1 -- start start 2023-11-09T03:51:07.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.096+0000 7f68ea155640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f68e40b0c40 0x7f68e406fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:07.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.096+0000 7f68ea155640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f68e406bb60 0x7f68e4070530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:07.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.096+0000 7f68ea155640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e4071500 con 0x7f68e4159e40 2023-11-09T03:51:07.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.096+0000 7f68ea155640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e4070a70 con 0x7f68e406bb60 2023-11-09T03:51:07.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.096+0000 7f68ea155640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e415dae0 con 0x7f68e40b0c40 2023-11-09T03:51:07.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.096+0000 7f68e3fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f68e406bb60 0x7f68e4070530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:07.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.096+0000 7f68e3fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f68e406bb60 0x7f68e4070530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45220/0 (socket says 172.21.15.76:45220) 2023-11-09T03:51:07.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.096+0000 7f68e3fff640 1 -- 172.21.15.76:0/2036105486 learned_addr learned my addr 172.21.15.76:0/2036105486 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:07.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.096+0000 7f68e2ffd640 1 --2- 172.21.15.76:0/2036105486 >> v2:172.21.15.113:3300/0 conn(0x7f68e40b0c40 0x7f68e406fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:07.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.096+0000 7f68e3fff640 1 -- 172.21.15.76:0/2036105486 >> v2:172.21.15.113:3300/0 conn(0x7f68e40b0c40 msgr2=0x7f68e406fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:07.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.096+0000 7f68e3fff640 1 --2- 172.21.15.76:0/2036105486 >> v2:172.21.15.113:3300/0 conn(0x7f68e40b0c40 0x7f68e406fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:07.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.096+0000 7f68e3fff640 1 -- 172.21.15.76:0/2036105486 >> v1:172.21.15.113:6789/0 conn(0x7f68e4159e40 legacy=0x7f68e40b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:07.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.096+0000 7f68e3fff640 1 -- 172.21.15.76:0/2036105486 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e4070c60 con 0x7f68e406bb60 2023-11-09T03:51:07.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.097+0000 7f68e3fff640 1 --2- 172.21.15.76:0/2036105486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68e406bb60 0x7f68e4070530 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f68e406fdc0 tx=0x7f68dc02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ad30470fc8159c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:07.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.097+0000 7f68e27fc640 1 -- 172.21.15.76:0/2036105486 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc02c9f0 con 0x7f68e406bb60 2023-11-09T03:51:07.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.097+0000 7f68e27fc640 1 -- 172.21.15.76:0/2036105486 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68dc02cb50 con 0x7f68e406bb60 2023-11-09T03:51:07.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.097+0000 7f68e27fc640 1 -- 172.21.15.76:0/2036105486 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc0343f0 con 0x7f68e406bb60 2023-11-09T03:51:07.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.098+0000 7f68ea155640 1 -- 172.21.15.76:0/2036105486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68e406bb60 msgr2=0x7f68e4070530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:07.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.098+0000 7f68ea155640 1 --2- 172.21.15.76:0/2036105486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68e406bb60 0x7f68e4070530 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f68e406fdc0 tx=0x7f68dc02be50 comp rx=0 tx=0).stop 2023-11-09T03:51:07.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.098+0000 7f68ea155640 1 -- 172.21.15.76:0/2036105486 shutdown_connections 2023-11-09T03:51:07.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.098+0000 7f68ea155640 1 --2- 172.21.15.76:0/2036105486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68e406bb60 0x7f68e4070530 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:07.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.098+0000 7f68ea155640 1 --2- 172.21.15.76:0/2036105486 >> v2:172.21.15.113:3300/0 conn(0x7f68e40b0c40 0x7f68e406fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:07.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.098+0000 7f68ea155640 1 -- 172.21.15.76:0/2036105486 >> 172.21.15.76:0/2036105486 conn(0x7f68e40cb530 msgr2=0x7f68e40cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:07.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.098+0000 7f68ea155640 1 -- 172.21.15.76:0/2036105486 shutdown_connections 2023-11-09T03:51:07.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.098+0000 7f68ea155640 1 -- 172.21.15.76:0/2036105486 wait complete. 2023-11-09T03:51:07.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.099+0000 7f68ea155640 1 Processor -- start 2023-11-09T03:51:07.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.099+0000 7f68ea155640 1 -- start start 2023-11-09T03:51:07.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.100+0000 7f68ea155640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68e406bb60 0x7f68e41503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:07.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.100+0000 7f68ea155640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68e40b0c40 0x7f68e414b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:07.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.100+0000 7f68ea155640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68e4159e40 0x7f68e40c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:07.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.100+0000 7f68ea155640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e40c3e50 con 0x7f68e406bb60 2023-11-09T03:51:07.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.100+0000 7f68ea155640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e41508e0 con 0x7f68e40b0c40 2023-11-09T03:51:07.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.100+0000 7f68ea155640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e4151760 con 0x7f68e4159e40 2023-11-09T03:51:07.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.100+0000 7f68e37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68e406bb60 0x7f68e41503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:07.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.100+0000 7f68e37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68e406bb60 0x7f68e41503a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45236/0 (socket says 172.21.15.76:45236) 2023-11-09T03:51:07.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.100+0000 7f68e37fe640 1 -- 172.21.15.76:0/560265762 learned_addr learned my addr 172.21.15.76:0/560265762 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:07.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.100+0000 7f68e2ffd640 1 --2- 172.21.15.76:0/560265762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68e40b0c40 0x7f68e414b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:07.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.100+0000 7f68e3fff640 1 --2- 172.21.15.76:0/560265762 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68e4159e40 0x7f68e40c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:07.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.100+0000 7f68e37fe640 1 -- 172.21.15.76:0/560265762 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68e4159e40 msgr2=0x7f68e40c48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:07.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.100+0000 7f68e37fe640 1 --2- 172.21.15.76:0/560265762 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68e4159e40 0x7f68e40c48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:07.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.100+0000 7f68e37fe640 1 -- 172.21.15.76:0/560265762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68e40b0c40 msgr2=0x7f68e414b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:07.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.100+0000 7f68e37fe640 1 --2- 172.21.15.76:0/560265762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68e40b0c40 0x7f68e414b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:07.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.100+0000 7f68e37fe640 1 -- 172.21.15.76:0/560265762 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68dc009b30 con 0x7f68e406bb60 2023-11-09T03:51:07.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.100+0000 7f68e2ffd640 1 --2- 172.21.15.76:0/560265762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68e40b0c40 0x7f68e414b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:51:07.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.101+0000 7f68e37fe640 1 --2- 172.21.15.76:0/560265762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68e406bb60 0x7f68e41503a0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f68d400b3f0 tx=0x7f68d400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:07.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.101+0000 7f68cbfff640 1 -- 172.21.15.76:0/560265762 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d4011830 con 0x7f68e406bb60 2023-11-09T03:51:07.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.101+0000 7f68cbfff640 1 -- 172.21.15.76:0/560265762 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d4011990 con 0x7f68e406bb60 2023-11-09T03:51:07.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.101+0000 7f68ea155640 1 -- 172.21.15.76:0/560265762 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e4151a40 con 0x7f68e406bb60 2023-11-09T03:51:07.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.101+0000 7f68cbfff640 1 -- 172.21.15.76:0/560265762 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d4011d40 con 0x7f68e406bb60 2023-11-09T03:51:07.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.101+0000 7f68ea155640 1 -- 172.21.15.76:0/560265762 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e40c4380 con 0x7f68e406bb60 2023-11-09T03:51:07.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.102+0000 7f68ea155640 1 -- 172.21.15.76:0/560265762 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e40c45a0 con 0x7f68e406bb60 2023-11-09T03:51:07.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.102+0000 7f68cbfff640 1 -- 172.21.15.76:0/560265762 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f68d401a400 con 0x7f68e406bb60 2023-11-09T03:51:07.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.103+0000 7f68cbfff640 1 --2- 172.21.15.76:0/560265762 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f68d006ae70 0x7f68d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:07.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.103+0000 7f68cbfff640 1 -- 172.21.15.76:0/560265762 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7f68d4089e10 con 0x7f68e406bb60 2023-11-09T03:51:07.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.103+0000 7f68e2ffd640 1 --2- 172.21.15.76:0/560265762 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f68d006ae70 0x7f68d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:07.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.103+0000 7f68e2ffd640 1 --2- 172.21.15.76:0/560265762 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f68d006ae70 0x7f68d006d330 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f68e4150d00 tx=0x7f68d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:07.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.107+0000 7f68cbfff640 1 -- 172.21.15.76:0/560265762 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f68d4054830 con 0x7f68e406bb60 2023-11-09T03:51:07.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.307+0000 7f68ea155640 1 -- 172.21.15.76:0/560265762 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68e40684b0 con 0x7f68e406bb60 2023-11-09T03:51:07.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.308+0000 7f68cbfff640 1 -- 172.21.15.76:0/560265762 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4932 (secure 0 0 0) 0x7f68d4015090 con 0x7f68e406bb60 2023-11-09T03:51:07.309 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:07.309 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":81,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6904,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:51:02.634881+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8016,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8016":{"gid":8016,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3229239133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3229239133},{"type":"v1","addr":"172.21.15.76:6835","nonce":3229239133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:51:07.309 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 81 2023-11-09T03:51:07.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.310+0000 7f68ea155640 1 -- 172.21.15.76:0/560265762 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f68d006ae70 msgr2=0x7f68d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:07.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.310+0000 7f68ea155640 1 --2- 172.21.15.76:0/560265762 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f68d006ae70 0x7f68d006d330 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f68e4150d00 tx=0x7f68d8008040 comp rx=0 tx=0).stop 2023-11-09T03:51:07.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.310+0000 7f68ea155640 1 -- 172.21.15.76:0/560265762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68e406bb60 msgr2=0x7f68e41503a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:07.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.310+0000 7f68ea155640 1 --2- 172.21.15.76:0/560265762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68e406bb60 0x7f68e41503a0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f68d400b3f0 tx=0x7f68d400b8b0 comp rx=0 tx=0).stop 2023-11-09T03:51:07.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.310+0000 7f68ea155640 1 -- 172.21.15.76:0/560265762 shutdown_connections 2023-11-09T03:51:07.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.310+0000 7f68ea155640 1 --2- 172.21.15.76:0/560265762 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f68d006ae70 0x7f68d006d330 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:07.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.310+0000 7f68ea155640 1 --2- 172.21.15.76:0/560265762 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68e4159e40 0x7f68e40c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:07.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.310+0000 7f68ea155640 1 --2- 172.21.15.76:0/560265762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68e40b0c40 0x7f68e414b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:07.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.310+0000 7f68ea155640 1 --2- 172.21.15.76:0/560265762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68e406bb60 0x7f68e41503a0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:07.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.310+0000 7f68ea155640 1 -- 172.21.15.76:0/560265762 >> 172.21.15.76:0/560265762 conn(0x7f68e40cb530 msgr2=0x7f68e40cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:07.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.310+0000 7f68ea155640 1 -- 172.21.15.76:0/560265762 shutdown_connections 2023-11-09T03:51:07.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.310+0000 7f68ea155640 1 -- 172.21.15.76:0/560265762 wait complete. 2023-11-09T03:51:07.321 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2023-11-09T03:51:07.434 INFO:teuthology.orchestra.run.smithi076.stdout:{"last_snap":9,"last_created":5,"last_destroyed":9,"pending_noop":[],"snaps":[],"need_to_purge":{"17":[2,3,4,5,6,7,8,9]},"pending_update":[],"pending_destroy":[]} 2023-11-09T03:51:07.435 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 5, "last_destroyed": 9, "last_snap": 9, "need_to_purge": { "17": [ 2, 3, 4, 5, 6, 7, 8, 9 ] }, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [] } 2023-11-09T03:51:07.435 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:0 true 2023-11-09T03:51:07.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.536+0000 7f511ecaf640 1 Processor -- start 2023-11-09T03:51:07.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.536+0000 7f511ecaf640 1 -- start start 2023-11-09T03:51:07.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.537+0000 7f511ecaf640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f511806c220 0x7f51180cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:07.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.537+0000 7f511ecaf640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51180cd720 con 0x7f51180cc680 2023-11-09T03:51:07.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.537+0000 7f511ecaf640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51180ae1b0 con 0x7f51180addd0 2023-11-09T03:51:07.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.537+0000 7f511ecaf640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51180ccb10 con 0x7f511806c220 2023-11-09T03:51:07.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.537+0000 7f5117fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f51180cc680 0x7f511806f8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39550/0 (socket says 172.21.15.76:39550) 2023-11-09T03:51:07.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.537+0000 7f5117fff640 1 -- 172.21.15.76:0/129281 learned_addr learned my addr 172.21.15.76:0/129281 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:51:07.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.537+0000 7f511d24c640 1 --2- 172.21.15.76:0/129281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f511806c220 0x7f51180cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:07.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.538+0000 7f51177fe640 1 -- 172.21.15.76:0/129281 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2743843938 0 0) 0x7f51180cd720 con 0x7f51180cc680 2023-11-09T03:51:07.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.538+0000 7f51177fe640 1 -- 172.21.15.76:0/129281 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5100003520 con 0x7f51180cc680 2023-11-09T03:51:07.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.538+0000 7f51177fe640 1 -- 172.21.15.76:0/129281 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2157676307 0 0) 0x7f5100003520 con 0x7f51180cc680 2023-11-09T03:51:07.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.538+0000 7f51177fe640 1 -- 172.21.15.76:0/129281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f511806c220 msgr2=0x7f51180cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:07.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.538+0000 7f51177fe640 1 --2- 172.21.15.76:0/129281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f511806c220 0x7f51180cdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:07.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.538+0000 7f51177fe640 1 -- 172.21.15.76:0/129281 >> v1:172.21.15.113:6789/0 conn(0x7f51180addd0 legacy=0x7f511806bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:07.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.538+0000 7f51177fe640 1 -- 172.21.15.76:0/129281 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51180ccdf0 con 0x7f51180cc680 2023-11-09T03:51:07.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.538+0000 7f51177fe640 1 -- 172.21.15.76:0/129281 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f510c001670 con 0x7f51180cc680 2023-11-09T03:51:07.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.538+0000 7f51177fe640 1 -- 172.21.15.76:0/129281 >> v1:172.21.15.76:6789/0 conn(0x7f51180cc680 legacy=0x7f511806f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:07.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.538+0000 7f51177fe640 1 --2- 172.21.15.76:0/129281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51000039a0 0x7f5100003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:07.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.538+0000 7f51177fe640 1 --2- 172.21.15.76:0/129281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5100004350 0x7f5100004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:07.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.538+0000 7f51177fe640 1 --2- 172.21.15.76:0/129281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f511806c220 0x7f5100005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:07.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.538+0000 7f51177fe640 1 -- 172.21.15.76:0/129281 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51180ccb10 con 0x7f51000039a0 2023-11-09T03:51:07.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.538+0000 7f51177fe640 1 -- 172.21.15.76:0/129281 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510000b9e0 con 0x7f5100004350 2023-11-09T03:51:07.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.538+0000 7f51177fe640 1 -- 172.21.15.76:0/129281 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f510000bba0 con 0x7f511806c220 2023-11-09T03:51:07.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.538+0000 7f511ca4b640 1 --2- 172.21.15.76:0/129281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51000039a0 0x7f5100003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:07.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.538+0000 7f511d24c640 1 --2- 172.21.15.76:0/129281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f511806c220 0x7f5100005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:07.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.538+0000 7f5117fff640 1 --2- 172.21.15.76:0/129281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5100004350 0x7f5100004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:07.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.538+0000 7f511ca4b640 1 -- 172.21.15.76:0/129281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f511806c220 msgr2=0x7f5100005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:07.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.538+0000 7f511ca4b640 1 --2- 172.21.15.76:0/129281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f511806c220 0x7f5100005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:07.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.538+0000 7f511ca4b640 1 -- 172.21.15.76:0/129281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5100004350 msgr2=0x7f5100004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:07.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.538+0000 7f511ca4b640 1 --2- 172.21.15.76:0/129281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5100004350 0x7f5100004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:07.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.538+0000 7f511ca4b640 1 -- 172.21.15.76:0/129281 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5100003520 con 0x7f51000039a0 2023-11-09T03:51:07.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.538+0000 7f511d24c640 1 --2- 172.21.15.76:0/129281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f511806c220 0x7f5100005080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:51:07.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.538+0000 7f5117fff640 1 --2- 172.21.15.76:0/129281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5100004350 0x7f5100004820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:51:07.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.539+0000 7f511ca4b640 1 --2- 172.21.15.76:0/129281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51000039a0 0x7f5100003d80 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f5108001830 tx=0x7f51000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b668d27261d493 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:07.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.539+0000 7f51177fe640 1 -- 172.21.15.76:0/129281 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510802f050 con 0x7f51000039a0 2023-11-09T03:51:07.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.539+0000 7f51177fe640 1 -- 172.21.15.76:0/129281 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5108033610 con 0x7f51000039a0 2023-11-09T03:51:07.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.539+0000 7f51177fe640 1 -- 172.21.15.76:0/129281 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51080339c0 con 0x7f51000039a0 2023-11-09T03:51:07.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.540+0000 7f511ecaf640 1 -- 172.21.15.76:0/129281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51000039a0 msgr2=0x7f5100003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:07.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.540+0000 7f511ecaf640 1 --2- 172.21.15.76:0/129281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51000039a0 0x7f5100003d80 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f5108001830 tx=0x7f51000057f0 comp rx=0 tx=0).stop 2023-11-09T03:51:07.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.540+0000 7f511d24c640 1 -- 172.21.15.76:0/129281 reap_dead start 2023-11-09T03:51:07.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.540+0000 7f511ecaf640 1 -- 172.21.15.76:0/129281 shutdown_connections 2023-11-09T03:51:07.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.540+0000 7f511ecaf640 1 -- 172.21.15.76:0/129281 >> 172.21.15.76:0/129281 conn(0x7f51180cb530 msgr2=0x7f51180cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:07.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.540+0000 7f511ecaf640 1 -- 172.21.15.76:0/129281 shutdown_connections 2023-11-09T03:51:07.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.540+0000 7f511ecaf640 1 -- 172.21.15.76:0/129281 wait complete. 2023-11-09T03:51:07.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.541+0000 7f511ecaf640 1 Processor -- start 2023-11-09T03:51:07.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.541+0000 7f511ecaf640 1 -- start start 2023-11-09T03:51:07.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.542+0000 7f511ecaf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51000039a0 0x7f51180c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:07.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.542+0000 7f511ecaf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5118147fc0 0x7f511815a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:07.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.542+0000 7f511ecaf640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f511815aa20 0x7f511814e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:07.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.542+0000 7f511ecaf640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51180c4a40 con 0x7f5118147fc0 2023-11-09T03:51:07.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.542+0000 7f511ecaf640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51180c41a0 con 0x7f51000039a0 2023-11-09T03:51:07.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.542+0000 7f511ecaf640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51180c4480 con 0x7f511815aa20 2023-11-09T03:51:07.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.542+0000 7f5117fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5118147fc0 0x7f511815a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:07.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.542+0000 7f5117fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5118147fc0 0x7f511815a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45254/0 (socket says 172.21.15.76:45254) 2023-11-09T03:51:07.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.542+0000 7f5117fff640 1 -- 172.21.15.76:0/1885548125 learned_addr learned my addr 172.21.15.76:0/1885548125 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:07.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.542+0000 7f511d24c640 1 --2- 172.21.15.76:0/1885548125 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f511815aa20 0x7f511814e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:07.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.542+0000 7f511ca4b640 1 --2- 172.21.15.76:0/1885548125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51000039a0 0x7f51180c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:07.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.542+0000 7f5117fff640 1 -- 172.21.15.76:0/1885548125 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f511815aa20 msgr2=0x7f511814e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:07.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.542+0000 7f5117fff640 1 --2- 172.21.15.76:0/1885548125 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f511815aa20 0x7f511814e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:07.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.542+0000 7f5117fff640 1 -- 172.21.15.76:0/1885548125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51000039a0 msgr2=0x7f51180c3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:07.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.542+0000 7f5117fff640 1 --2- 172.21.15.76:0/1885548125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51000039a0 0x7f51180c3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:07.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.542+0000 7f5117fff640 1 -- 172.21.15.76:0/1885548125 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51180c4760 con 0x7f5118147fc0 2023-11-09T03:51:07.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.542+0000 7f5117fff640 1 --2- 172.21.15.76:0/1885548125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5118147fc0 0x7f511815a4e0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f510c001200 tx=0x7f510c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:07.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.542+0000 7f51157fa640 1 -- 172.21.15.76:0/1885548125 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c00c6c0 con 0x7f5118147fc0 2023-11-09T03:51:07.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.542+0000 7f51157fa640 1 -- 172.21.15.76:0/1885548125 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510c00c820 con 0x7f5118147fc0 2023-11-09T03:51:07.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.542+0000 7f51157fa640 1 -- 172.21.15.76:0/1885548125 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c00cb90 con 0x7f5118147fc0 2023-11-09T03:51:07.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.542+0000 7f511ecaf640 1 -- 172.21.15.76:0/1885548125 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f511814e120 con 0x7f5118147fc0 2023-11-09T03:51:07.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.543+0000 7f511ecaf640 1 -- 172.21.15.76:0/1885548125 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f511815c880 con 0x7f5118147fc0 2023-11-09T03:51:07.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.543+0000 7f511ecaf640 1 -- 172.21.15.76:0/1885548125 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f511814df80 con 0x7f5118147fc0 2023-11-09T03:51:07.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.544+0000 7f51157fa640 1 -- 172.21.15.76:0/1885548125 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f510c00ccf0 con 0x7f5118147fc0 2023-11-09T03:51:07.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.544+0000 7f51157fa640 1 --2- 172.21.15.76:0/1885548125 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f510406aea0 0x7f510406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:07.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.544+0000 7f51157fa640 1 -- 172.21.15.76:0/1885548125 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7f510c089b40 con 0x7f5118147fc0 2023-11-09T03:51:07.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.544+0000 7f511ca4b640 1 --2- 172.21.15.76:0/1885548125 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f510406aea0 0x7f510406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:07.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.545+0000 7f511ca4b640 1 --2- 172.21.15.76:0/1885548125 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f510406aea0 0x7f510406d360 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5108036000 tx=0x7f5108035040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:07.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.549+0000 7f51157fa640 1 -- 172.21.15.76:0/1885548125 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f511814df80 con 0x7f5118147fc0 2023-11-09T03:51:07.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:07.787+0000 7f511ecaf640 1 -- 172.21.15.76:0/1885548125 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"} v 0) v1 -- 0x7f511814df80 con 0x7f5118147fc0 2023-11-09T03:51:08.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.653+0000 7f51157fa640 1 -- 172.21.15.76:0/1885548125 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"}]=0 freezing mds.8016 v82) v1 ==== 112+0+0 (secure 0 0 0) 0x7f511814df80 con 0x7f5118147fc0 2023-11-09T03:51:08.654 INFO:teuthology.orchestra.run.smithi076.stderr:freezing mds.8016 2023-11-09T03:51:08.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.656+0000 7f511ecaf640 1 -- 172.21.15.76:0/1885548125 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f510406aea0 msgr2=0x7f510406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:08.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.656+0000 7f511ecaf640 1 --2- 172.21.15.76:0/1885548125 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f510406aea0 0x7f510406d360 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5108036000 tx=0x7f5108035040 comp rx=0 tx=0).stop 2023-11-09T03:51:08.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.656+0000 7f511ecaf640 1 -- 172.21.15.76:0/1885548125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5118147fc0 msgr2=0x7f511815a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:08.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.656+0000 7f511ecaf640 1 --2- 172.21.15.76:0/1885548125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5118147fc0 0x7f511815a4e0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f510c001200 tx=0x7f510c00ec10 comp rx=0 tx=0).stop 2023-11-09T03:51:08.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.656+0000 7f511ecaf640 1 -- 172.21.15.76:0/1885548125 shutdown_connections 2023-11-09T03:51:08.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.656+0000 7f511ecaf640 1 --2- 172.21.15.76:0/1885548125 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f510406aea0 0x7f510406d360 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:08.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.656+0000 7f511ecaf640 1 --2- 172.21.15.76:0/1885548125 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f511815aa20 0x7f511814e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:08.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.656+0000 7f511ecaf640 1 --2- 172.21.15.76:0/1885548125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5118147fc0 0x7f511815a4e0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:08.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.656+0000 7f511ecaf640 1 --2- 172.21.15.76:0/1885548125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51000039a0 0x7f51180c3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:08.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.656+0000 7f511ecaf640 1 -- 172.21.15.76:0/1885548125 >> 172.21.15.76:0/1885548125 conn(0x7f51180cb530 msgr2=0x7f511806bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:08.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.656+0000 7f511ecaf640 1 -- 172.21.15.76:0/1885548125 shutdown_connections 2023-11-09T03:51:08.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.656+0000 7f511ecaf640 1 -- 172.21.15.76:0/1885548125 wait complete. 2023-11-09T03:51:08.670 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:1 true 2023-11-09T03:51:08.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.777+0000 7effd66ec640 1 Processor -- start 2023-11-09T03:51:08.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.777+0000 7effd66ec640 1 -- start start 2023-11-09T03:51:08.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.778+0000 7effd66ec640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7effd015d100 0x7effd00ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:08.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.778+0000 7effd66ec640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7effd015d4e0 0x7effd00d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:08.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.778+0000 7effd66ec640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effd00b32b0 con 0x7effd00d3b60 2023-11-09T03:51:08.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.778+0000 7effd66ec640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd00cee10 con 0x7effd015d4e0 2023-11-09T03:51:08.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.778+0000 7effd66ec640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd00b4750 con 0x7effd015d100 2023-11-09T03:51:08.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.778+0000 7effcf7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7effd015d4e0 0x7effd00d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:08.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.778+0000 7effcf7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7effd015d4e0 0x7effd00d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45262/0 (socket says 172.21.15.76:45262) 2023-11-09T03:51:08.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.778+0000 7effd4c89640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7effd00d3b60 0x7effd00cf520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39552/0 (socket says 172.21.15.76:39552) 2023-11-09T03:51:08.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.778+0000 7effcf7fe640 1 -- 172.21.15.76:0/3381292592 learned_addr learned my addr 172.21.15.76:0/3381292592 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:08.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.779+0000 7effcffff640 1 --2- 172.21.15.76:0/3381292592 >> v2:172.21.15.113:3300/0 conn(0x7effd015d100 0x7effd00ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:08.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.779+0000 7effcf7fe640 1 -- 172.21.15.76:0/3381292592 >> v2:172.21.15.113:3300/0 conn(0x7effd015d100 msgr2=0x7effd00ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:08.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.779+0000 7effcf7fe640 1 --2- 172.21.15.76:0/3381292592 >> v2:172.21.15.113:3300/0 conn(0x7effd015d100 0x7effd00ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:08.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.779+0000 7effcf7fe640 1 -- 172.21.15.76:0/3381292592 >> v1:172.21.15.76:6789/0 conn(0x7effd00d3b60 legacy=0x7effd00cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:08.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.779+0000 7effcf7fe640 1 -- 172.21.15.76:0/3381292592 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd00b2980 con 0x7effd015d4e0 2023-11-09T03:51:08.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.779+0000 7effceffd640 1 -- 172.21.15.76:0/3381292592 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695279461 0 0) 0x7effd00b32b0 con 0x7effd00d3b60 2023-11-09T03:51:08.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.779+0000 7effcf7fe640 1 --2- 172.21.15.76:0/3381292592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effd015d4e0 0x7effd00d14d0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7effd00d09f0 tx=0x7effc402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2955c714d7abb71f server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:08.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.780+0000 7effceffd640 1 -- 172.21.15.76:0/3381292592 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc4030860 con 0x7effd015d4e0 2023-11-09T03:51:08.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.780+0000 7effceffd640 1 -- 172.21.15.76:0/3381292592 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc40309c0 con 0x7effd015d4e0 2023-11-09T03:51:08.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.780+0000 7effceffd640 1 -- 172.21.15.76:0/3381292592 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc4030d50 con 0x7effd015d4e0 2023-11-09T03:51:08.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.780+0000 7effd66ec640 1 -- 172.21.15.76:0/3381292592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effd015d4e0 msgr2=0x7effd00d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:08.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.780+0000 7effd66ec640 1 --2- 172.21.15.76:0/3381292592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effd015d4e0 0x7effd00d14d0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7effd00d09f0 tx=0x7effc402b9a0 comp rx=0 tx=0).stop 2023-11-09T03:51:08.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.780+0000 7effd66ec640 1 -- 172.21.15.76:0/3381292592 shutdown_connections 2023-11-09T03:51:08.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.780+0000 7effd66ec640 1 --2- 172.21.15.76:0/3381292592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effd015d4e0 0x7effd00d14d0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:08.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.780+0000 7effd66ec640 1 --2- 172.21.15.76:0/3381292592 >> v2:172.21.15.113:3300/0 conn(0x7effd015d100 0x7effd00ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:08.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.780+0000 7effd66ec640 1 -- 172.21.15.76:0/3381292592 >> 172.21.15.76:0/3381292592 conn(0x7effd00d2ef0 msgr2=0x7effd00d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:08.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.780+0000 7effd66ec640 1 -- 172.21.15.76:0/3381292592 shutdown_connections 2023-11-09T03:51:08.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.780+0000 7effd66ec640 1 -- 172.21.15.76:0/3381292592 wait complete. 2023-11-09T03:51:08.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.781+0000 7effd66ec640 1 Processor -- start 2023-11-09T03:51:08.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.781+0000 7effd66ec640 1 -- start start 2023-11-09T03:51:08.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.782+0000 7effd66ec640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effd00d3b60 0x7effd0151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:08.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.782+0000 7effd66ec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effd015d100 0x7effd00cb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:08.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.782+0000 7effd66ec640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effd015d4e0 0x7effd00b7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:08.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.782+0000 7effd66ec640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd01519b0 con 0x7effd015d100 2023-11-09T03:51:08.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.782+0000 7effd66ec640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd00cce10 con 0x7effd015d4e0 2023-11-09T03:51:08.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.782+0000 7effd66ec640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd00cb5a0 con 0x7effd00d3b60 2023-11-09T03:51:08.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.782+0000 7effcf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effd015d100 0x7effd00cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:08.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.782+0000 7effcf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effd015d100 0x7effd00cb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45272/0 (socket says 172.21.15.76:45272) 2023-11-09T03:51:08.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.782+0000 7effcf7fe640 1 -- 172.21.15.76:0/561853503 learned_addr learned my addr 172.21.15.76:0/561853503 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:08.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.783+0000 7effd4c89640 1 --2- 172.21.15.76:0/561853503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effd015d4e0 0x7effd00b7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:08.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.783+0000 7effcf7fe640 1 -- 172.21.15.76:0/561853503 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effd00d3b60 msgr2=0x7effd0151470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:08.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.783+0000 7effcf7fe640 1 --2- 172.21.15.76:0/561853503 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effd00d3b60 0x7effd0151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:08.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.783+0000 7effcf7fe640 1 -- 172.21.15.76:0/561853503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effd015d4e0 msgr2=0x7effd00b7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:08.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.783+0000 7effcf7fe640 1 --2- 172.21.15.76:0/561853503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effd015d4e0 0x7effd00b7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:08.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.783+0000 7effcf7fe640 1 -- 172.21.15.76:0/561853503 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc40094d0 con 0x7effd015d100 2023-11-09T03:51:08.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.783+0000 7effcf7fe640 1 --2- 172.21.15.76:0/561853503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effd015d100 0x7effd00cb060 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7effc402b9e0 tx=0x7effc4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:08.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.783+0000 7effccff9640 1 -- 172.21.15.76:0/561853503 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc403e040 con 0x7effd015d100 2023-11-09T03:51:08.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.783+0000 7effccff9640 1 -- 172.21.15.76:0/561853503 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc4039d00 con 0x7effd015d100 2023-11-09T03:51:08.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.783+0000 7effccff9640 1 -- 172.21.15.76:0/561853503 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc4044630 con 0x7effd015d100 2023-11-09T03:51:08.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.783+0000 7effd66ec640 1 -- 172.21.15.76:0/561853503 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effd00c9fc0 con 0x7effd015d100 2023-11-09T03:51:08.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.783+0000 7effd66ec640 1 -- 172.21.15.76:0/561853503 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd00ca580 con 0x7effd015d100 2023-11-09T03:51:08.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.784+0000 7effd66ec640 1 -- 172.21.15.76:0/561853503 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effd00684b0 con 0x7effd015d100 2023-11-09T03:51:08.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.784+0000 7effccff9640 1 -- 172.21.15.76:0/561853503 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7effc402f3b0 con 0x7effd015d100 2023-11-09T03:51:08.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.785+0000 7effccff9640 1 --2- 172.21.15.76:0/561853503 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7effbc06ae70 0x7effbc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:08.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.785+0000 7effccff9640 1 -- 172.21.15.76:0/561853503 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7effc40ac2c0 con 0x7effd015d100 2023-11-09T03:51:08.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.785+0000 7effcffff640 1 --2- 172.21.15.76:0/561853503 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7effbc06ae70 0x7effbc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:08.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.785+0000 7effcffff640 1 --2- 172.21.15.76:0/561853503 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7effbc06ae70 0x7effbc06d330 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7effc0006e20 tx=0x7effc0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:08.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:08.789+0000 7effccff9640 1 -- 172.21.15.76:0/561853503 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7effc4076c50 con 0x7effd015d100 2023-11-09T03:51:09.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:09.022+0000 7effd66ec640 1 -- 172.21.15.76:0/561853503 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"} v 0) v1 -- 0x7effd00d2cd0 con 0x7effd015d100 2023-11-09T03:51:09.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:09.668+0000 7effccff9640 1 -- 172.21.15.76:0/561853503 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"}]=0 freezing mds.7449 v83) v1 ==== 112+0+0 (secure 0 0 0) 0x7effc407a270 con 0x7effd015d100 2023-11-09T03:51:09.667 INFO:teuthology.orchestra.run.smithi076.stderr:freezing mds.7449 2023-11-09T03:51:09.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:09.670+0000 7effd66ec640 1 -- 172.21.15.76:0/561853503 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7effbc06ae70 msgr2=0x7effbc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:09.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:09.670+0000 7effd66ec640 1 --2- 172.21.15.76:0/561853503 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7effbc06ae70 0x7effbc06d330 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7effc0006e20 tx=0x7effc0008040 comp rx=0 tx=0).stop 2023-11-09T03:51:09.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:09.670+0000 7effd66ec640 1 -- 172.21.15.76:0/561853503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effd015d100 msgr2=0x7effd00cb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:09.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:09.670+0000 7effd66ec640 1 --2- 172.21.15.76:0/561853503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effd015d100 0x7effd00cb060 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7effc402b9e0 tx=0x7effc4004730 comp rx=0 tx=0).stop 2023-11-09T03:51:09.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:09.670+0000 7effd66ec640 1 -- 172.21.15.76:0/561853503 shutdown_connections 2023-11-09T03:51:09.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:09.670+0000 7effd66ec640 1 --2- 172.21.15.76:0/561853503 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7effbc06ae70 0x7effbc06d330 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:09.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:09.670+0000 7effd66ec640 1 --2- 172.21.15.76:0/561853503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effd015d4e0 0x7effd00b7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:09.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:09.670+0000 7effd66ec640 1 --2- 172.21.15.76:0/561853503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effd015d100 0x7effd00cb060 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:09.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:09.670+0000 7effd66ec640 1 --2- 172.21.15.76:0/561853503 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7effd00d3b60 0x7effd0151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:09.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:09.670+0000 7effd66ec640 1 -- 172.21.15.76:0/561853503 >> 172.21.15.76:0/561853503 conn(0x7effd00d2ef0 msgr2=0x7effd00d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:09.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:09.670+0000 7effd66ec640 1 -- 172.21.15.76:0/561853503 shutdown_connections 2023-11-09T03:51:09.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:09.670+0000 7effd66ec640 1 -- 172.21.15.76:0/561853503 wait complete. 2023-11-09T03:51:09.687 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_kill_mdstable_at 2 2023-11-09T03:51:09.806 INFO:teuthology.orchestra.run.smithi076.stdout:{"success":"mds_kill_mdstable_at = '2' (not observed, change may require restart) "} 2023-11-09T03:51:09.806 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '2' (not observed, change may require restart) " } 2023-11-09T03:51:09.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:51:09.807 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s22) 2023-11-09T03:51:09.809 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2023-11-09T03:51:09.825 INFO:tasks.ceph.mds.c.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7fc9387cf640 time 2023-11-09T03:51:09.826667+0000 2023-11-09T03:51:09.825 INFO:tasks.ceph.mds.c.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2023-11-09T03:51:09.828 INFO:tasks.ceph.mds.c.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:51:09.829 INFO:tasks.ceph.mds.c.smithi076.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fc94436efaf] 2023-11-09T03:51:09.829 INFO:tasks.ceph.mds.c.smithi076.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fc94436f16d] 2023-11-09T03:51:09.829 INFO:tasks.ceph.mds.c.smithi076.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x564c07d629ce] 2023-11-09T03:51:09.829 INFO:tasks.ceph.mds.c.smithi076.stderr: 4: (MDSContext::complete(int)+0x5c) [0x564c07d92bcc] 2023-11-09T03:51:09.830 INFO:tasks.ceph.mds.c.smithi076.stderr: 5: (MDSIOContextBase::complete(int)+0x31c) [0x564c07d9bb0c] 2023-11-09T03:51:09.830 INFO:tasks.ceph.mds.c.smithi076.stderr: 6: (MDSLogContextBase::complete(int)+0x53) [0x564c07d9bc63] 2023-11-09T03:51:09.830 INFO:tasks.ceph.mds.c.smithi076.stderr: 7: (Finisher::finisher_thread_entry()+0x175) [0x7fc9444164a5] 2023-11-09T03:51:09.830 INFO:tasks.ceph.mds.c.smithi076.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7fc943a9f802] 2023-11-09T03:51:09.831 INFO:tasks.ceph.mds.c.smithi076.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7fc943a3f450] 2023-11-09T03:51:09.831 INFO:tasks.ceph.mds.c.smithi076.stderr:*** Caught signal (Aborted) ** 2023-11-09T03:51:09.831 INFO:tasks.ceph.mds.c.smithi076.stderr: in thread 7fc9387cf640 thread_name:MR_Finisher 2023-11-09T03:51:09.831 INFO:tasks.ceph.mds.c.smithi076.stderr:2023-11-09T03:51:09.826+0000 7fc9387cf640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7fc9387cf640 time 2023-11-09T03:51:09.826667+0000 2023-11-09T03:51:09.831 INFO:tasks.ceph.mds.c.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2023-11-09T03:51:09.831 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:51:09.832 INFO:tasks.ceph.mds.c.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:51:09.832 INFO:tasks.ceph.mds.c.smithi076.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fc94436efaf] 2023-11-09T03:51:09.832 INFO:tasks.ceph.mds.c.smithi076.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fc94436f16d] 2023-11-09T03:51:09.832 INFO:tasks.ceph.mds.c.smithi076.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x564c07d629ce] 2023-11-09T03:51:09.832 INFO:tasks.ceph.mds.c.smithi076.stderr: 4: (MDSContext::complete(int)+0x5c) [0x564c07d92bcc] 2023-11-09T03:51:09.833 INFO:tasks.ceph.mds.c.smithi076.stderr: 5: (MDSIOContextBase::complete(int)+0x31c) [0x564c07d9bb0c] 2023-11-09T03:51:09.833 INFO:tasks.ceph.mds.c.smithi076.stderr: 6: (MDSLogContextBase::complete(int)+0x53) [0x564c07d9bc63] 2023-11-09T03:51:09.833 INFO:tasks.ceph.mds.c.smithi076.stderr: 7: (Finisher::finisher_thread_entry()+0x175) [0x7fc9444164a5] 2023-11-09T03:51:09.833 INFO:tasks.ceph.mds.c.smithi076.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7fc943a9f802] 2023-11-09T03:51:09.833 INFO:tasks.ceph.mds.c.smithi076.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7fc943a3f450] 2023-11-09T03:51:09.834 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:51:09.834 INFO:tasks.ceph.mds.c.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:51:09.834 INFO:tasks.ceph.mds.c.smithi076.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fc943a54db0] 2023-11-09T03:51:09.834 INFO:tasks.ceph.mds.c.smithi076.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fc943aa154c] 2023-11-09T03:51:09.834 INFO:tasks.ceph.mds.c.smithi076.stderr: 3: raise() 2023-11-09T03:51:09.835 INFO:tasks.ceph.mds.c.smithi076.stderr: 4: abort() 2023-11-09T03:51:09.835 INFO:tasks.ceph.mds.c.smithi076.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fc94436f009] 2023-11-09T03:51:09.835 INFO:tasks.ceph.mds.c.smithi076.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fc94436f16d] 2023-11-09T03:51:09.835 INFO:tasks.ceph.mds.c.smithi076.stderr: 7: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x564c07d629ce] 2023-11-09T03:51:09.835 INFO:tasks.ceph.mds.c.smithi076.stderr: 8: (MDSContext::complete(int)+0x5c) [0x564c07d92bcc] 2023-11-09T03:51:09.836 INFO:tasks.ceph.mds.c.smithi076.stderr: 9: (MDSIOContextBase::complete(int)+0x31c) [0x564c07d9bb0c] 2023-11-09T03:51:09.836 INFO:tasks.ceph.mds.c.smithi076.stderr: 10: (MDSLogContextBase::complete(int)+0x53) [0x564c07d9bc63] 2023-11-09T03:51:09.836 INFO:tasks.ceph.mds.c.smithi076.stderr: 11: (Finisher::finisher_thread_entry()+0x175) [0x7fc9444164a5] 2023-11-09T03:51:09.836 INFO:tasks.ceph.mds.c.smithi076.stderr: 12: /lib64/libc.so.6(+0x9f802) [0x7fc943a9f802] 2023-11-09T03:51:09.836 INFO:tasks.ceph.mds.c.smithi076.stderr: 13: /lib64/libc.so.6(+0x3f450) [0x7fc943a3f450] 2023-11-09T03:51:09.836 INFO:tasks.ceph.mds.c.smithi076.stderr:2023-11-09T03:51:09.827+0000 7fc9387cf640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:51:09.837 INFO:tasks.ceph.mds.c.smithi076.stderr: in thread 7fc9387cf640 thread_name:MR_Finisher 2023-11-09T03:51:09.837 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:51:09.837 INFO:tasks.ceph.mds.c.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:51:09.837 INFO:tasks.ceph.mds.c.smithi076.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fc943a54db0] 2023-11-09T03:51:09.837 INFO:tasks.ceph.mds.c.smithi076.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fc943aa154c] 2023-11-09T03:51:09.837 INFO:tasks.ceph.mds.c.smithi076.stderr: 3: raise() 2023-11-09T03:51:09.838 INFO:tasks.ceph.mds.c.smithi076.stderr: 4: abort() 2023-11-09T03:51:09.838 INFO:tasks.ceph.mds.c.smithi076.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fc94436f009] 2023-11-09T03:51:09.838 INFO:tasks.ceph.mds.c.smithi076.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fc94436f16d] 2023-11-09T03:51:09.838 INFO:tasks.ceph.mds.c.smithi076.stderr: 7: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x564c07d629ce] 2023-11-09T03:51:09.839 INFO:tasks.ceph.mds.c.smithi076.stderr: 8: (MDSContext::complete(int)+0x5c) [0x564c07d92bcc] 2023-11-09T03:51:09.839 INFO:tasks.ceph.mds.c.smithi076.stderr: 9: (MDSIOContextBase::complete(int)+0x31c) [0x564c07d9bb0c] 2023-11-09T03:51:09.839 INFO:tasks.ceph.mds.c.smithi076.stderr: 10: (MDSLogContextBase::complete(int)+0x53) [0x564c07d9bc63] 2023-11-09T03:51:09.839 INFO:tasks.ceph.mds.c.smithi076.stderr: 11: (Finisher::finisher_thread_entry()+0x175) [0x7fc9444164a5] 2023-11-09T03:51:09.839 INFO:tasks.ceph.mds.c.smithi076.stderr: 12: /lib64/libc.so.6(+0x9f802) [0x7fc943a9f802] 2023-11-09T03:51:09.840 INFO:tasks.ceph.mds.c.smithi076.stderr: 13: /lib64/libc.so.6(+0x3f450) [0x7fc943a3f450] 2023-11-09T03:51:09.840 INFO:tasks.ceph.mds.c.smithi076.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:51:09.840 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:51:09.841 INFO:tasks.ceph.mds.c.smithi076.stderr: -746> 2023-11-09T03:51:04.677+0000 7fc93bfd6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:51:09.841 INFO:tasks.ceph.mds.c.smithi076.stderr: -744> 2023-11-09T03:51:04.677+0000 7fc93bfd6640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-09T03:51:09.841 INFO:tasks.ceph.mds.c.smithi076.stderr: -1> 2023-11-09T03:51:09.826+0000 7fc9387cf640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7fc9387cf640 time 2023-11-09T03:51:09.826667+0000 2023-11-09T03:51:09.841 INFO:tasks.ceph.mds.c.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2023-11-09T03:51:09.842 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:51:09.842 INFO:tasks.ceph.mds.c.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:51:09.842 INFO:tasks.ceph.mds.c.smithi076.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fc94436efaf] 2023-11-09T03:51:09.842 INFO:tasks.ceph.mds.c.smithi076.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fc94436f16d] 2023-11-09T03:51:09.842 INFO:tasks.ceph.mds.c.smithi076.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x564c07d629ce] 2023-11-09T03:51:09.843 INFO:tasks.ceph.mds.c.smithi076.stderr: 4: (MDSContext::complete(int)+0x5c) [0x564c07d92bcc] 2023-11-09T03:51:09.843 INFO:tasks.ceph.mds.c.smithi076.stderr: 5: (MDSIOContextBase::complete(int)+0x31c) [0x564c07d9bb0c] 2023-11-09T03:51:09.843 INFO:tasks.ceph.mds.c.smithi076.stderr: 6: (MDSLogContextBase::complete(int)+0x53) [0x564c07d9bc63] 2023-11-09T03:51:09.843 INFO:tasks.ceph.mds.c.smithi076.stderr: 7: (Finisher::finisher_thread_entry()+0x175) [0x7fc9444164a5] 2023-11-09T03:51:09.843 INFO:tasks.ceph.mds.c.smithi076.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7fc943a9f802] 2023-11-09T03:51:09.843 INFO:tasks.ceph.mds.c.smithi076.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7fc943a3f450] 2023-11-09T03:51:09.844 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:51:09.844 INFO:tasks.ceph.mds.c.smithi076.stderr: 0> 2023-11-09T03:51:09.827+0000 7fc9387cf640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:51:09.844 INFO:tasks.ceph.mds.c.smithi076.stderr: in thread 7fc9387cf640 thread_name:MR_Finisher 2023-11-09T03:51:09.844 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:51:09.844 INFO:tasks.ceph.mds.c.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:51:09.845 INFO:tasks.ceph.mds.c.smithi076.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fc943a54db0] 2023-11-09T03:51:09.845 INFO:tasks.ceph.mds.c.smithi076.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fc943aa154c] 2023-11-09T03:51:09.845 INFO:tasks.ceph.mds.c.smithi076.stderr: 3: raise() 2023-11-09T03:51:09.845 INFO:tasks.ceph.mds.c.smithi076.stderr: 4: abort() 2023-11-09T03:51:09.845 INFO:tasks.ceph.mds.c.smithi076.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fc94436f009] 2023-11-09T03:51:09.845 INFO:tasks.ceph.mds.c.smithi076.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fc94436f16d] 2023-11-09T03:51:09.846 INFO:tasks.ceph.mds.c.smithi076.stderr: 7: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x564c07d629ce] 2023-11-09T03:51:09.846 INFO:tasks.ceph.mds.c.smithi076.stderr: 8: (MDSContext::complete(int)+0x5c) [0x564c07d92bcc] 2023-11-09T03:51:09.846 INFO:tasks.ceph.mds.c.smithi076.stderr: 9: (MDSIOContextBase::complete(int)+0x31c) [0x564c07d9bb0c] 2023-11-09T03:51:09.846 INFO:tasks.ceph.mds.c.smithi076.stderr: 10: (MDSLogContextBase::complete(int)+0x53) [0x564c07d9bc63] 2023-11-09T03:51:09.846 INFO:tasks.ceph.mds.c.smithi076.stderr: 11: (Finisher::finisher_thread_entry()+0x175) [0x7fc9444164a5] 2023-11-09T03:51:09.847 INFO:tasks.ceph.mds.c.smithi076.stderr: 12: /lib64/libc.so.6(+0x9f802) [0x7fc943a9f802] 2023-11-09T03:51:09.847 INFO:tasks.ceph.mds.c.smithi076.stderr: 13: /lib64/libc.so.6(+0x3f450) [0x7fc943a3f450] 2023-11-09T03:51:09.847 INFO:tasks.ceph.mds.c.smithi076.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:51:09.847 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:51:09.847 INFO:tasks.ceph.mds.c.smithi076.stderr: -746> 2023-11-09T03:51:04.677+0000 7fc93bfd6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:51:09.847 INFO:tasks.ceph.mds.c.smithi076.stderr: -744> 2023-11-09T03:51:04.677+0000 7fc93bfd6640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-09T03:51:09.848 INFO:tasks.ceph.mds.c.smithi076.stderr: -1> 2023-11-09T03:51:09.826+0000 7fc9387cf640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7fc9387cf640 time 2023-11-09T03:51:09.826667+0000 2023-11-09T03:51:09.848 INFO:tasks.ceph.mds.c.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2023-11-09T03:51:09.848 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:51:09.848 INFO:tasks.ceph.mds.c.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:51:09.848 INFO:tasks.ceph.mds.c.smithi076.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fc94436efaf] 2023-11-09T03:51:09.848 INFO:tasks.ceph.mds.c.smithi076.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fc94436f16d] 2023-11-09T03:51:09.849 INFO:tasks.ceph.mds.c.smithi076.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x564c07d629ce] 2023-11-09T03:51:09.849 INFO:tasks.ceph.mds.c.smithi076.stderr: 4: (MDSContext::complete(int)+0x5c) [0x564c07d92bcc] 2023-11-09T03:51:09.849 INFO:tasks.ceph.mds.c.smithi076.stderr: 5: (MDSIOContextBase::complete(int)+0x31c) [0x564c07d9bb0c] 2023-11-09T03:51:09.849 INFO:tasks.ceph.mds.c.smithi076.stderr: 6: (MDSLogContextBase::complete(int)+0x53) [0x564c07d9bc63] 2023-11-09T03:51:09.849 INFO:tasks.ceph.mds.c.smithi076.stderr: 7: (Finisher::finisher_thread_entry()+0x175) [0x7fc9444164a5] 2023-11-09T03:51:09.849 INFO:tasks.ceph.mds.c.smithi076.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7fc943a9f802] 2023-11-09T03:51:09.850 INFO:tasks.ceph.mds.c.smithi076.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7fc943a3f450] 2023-11-09T03:51:09.850 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:51:09.850 INFO:tasks.ceph.mds.c.smithi076.stderr: 0> 2023-11-09T03:51:09.827+0000 7fc9387cf640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:51:09.850 INFO:tasks.ceph.mds.c.smithi076.stderr: in thread 7fc9387cf640 thread_name:MR_Finisher 2023-11-09T03:51:09.850 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:51:09.850 INFO:tasks.ceph.mds.c.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:51:09.851 INFO:tasks.ceph.mds.c.smithi076.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fc943a54db0] 2023-11-09T03:51:09.851 INFO:tasks.ceph.mds.c.smithi076.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fc943aa154c] 2023-11-09T03:51:09.851 INFO:tasks.ceph.mds.c.smithi076.stderr: 3: raise() 2023-11-09T03:51:09.851 INFO:tasks.ceph.mds.c.smithi076.stderr: 4: abort() 2023-11-09T03:51:09.851 INFO:tasks.ceph.mds.c.smithi076.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fc94436f009] 2023-11-09T03:51:09.851 INFO:tasks.ceph.mds.c.smithi076.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fc94436f16d] 2023-11-09T03:51:09.852 INFO:tasks.ceph.mds.c.smithi076.stderr: 7: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x564c07d629ce] 2023-11-09T03:51:09.852 INFO:tasks.ceph.mds.c.smithi076.stderr: 8: (MDSContext::complete(int)+0x5c) [0x564c07d92bcc] 2023-11-09T03:51:09.852 INFO:tasks.ceph.mds.c.smithi076.stderr: 9: (MDSIOContextBase::complete(int)+0x31c) [0x564c07d9bb0c] 2023-11-09T03:51:09.852 INFO:tasks.ceph.mds.c.smithi076.stderr: 10: (MDSLogContextBase::complete(int)+0x53) [0x564c07d9bc63] 2023-11-09T03:51:09.852 INFO:tasks.ceph.mds.c.smithi076.stderr: 11: (Finisher::finisher_thread_entry()+0x175) [0x7fc9444164a5] 2023-11-09T03:51:09.853 INFO:tasks.ceph.mds.c.smithi076.stderr: 12: /lib64/libc.so.6(+0x9f802) [0x7fc943a9f802] 2023-11-09T03:51:09.853 INFO:tasks.ceph.mds.c.smithi076.stderr: 13: /lib64/libc.so.6(+0x3f450) [0x7fc943a3f450] 2023-11-09T03:51:09.853 INFO:tasks.ceph.mds.c.smithi076.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:51:09.853 INFO:tasks.ceph.mds.c.smithi076.stderr: 2023-11-09T03:51:09.941 INFO:teuthology.orchestra.run.smithi076.stdout:{"mds_beacon_grace":"15.000000"} 2023-11-09T03:51:09.941 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2023-11-09T03:51:09.942 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:51:10.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.081+0000 7f97391c0640 1 Processor -- start 2023-11-09T03:51:10.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.081+0000 7f97391c0640 1 -- start start 2023-11-09T03:51:10.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.082+0000 7f97391c0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f973415d4e0 0x7f97340d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:10.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.082+0000 7f97391c0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f97340d3b60 0x7f97340d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:10.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.082+0000 7f97391c0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97340b32b0 con 0x7f973415d100 2023-11-09T03:51:10.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.082+0000 7f97391c0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97340cec20 con 0x7f97340d3b60 2023-11-09T03:51:10.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.082+0000 7f97391c0640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97340cee10 con 0x7f973415d4e0 2023-11-09T03:51:10.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.082+0000 7f973359e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f97340d3b60 0x7f97340d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:10.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.082+0000 7f973359e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f97340d3b60 0x7f97340d1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45278/0 (socket says 172.21.15.76:45278) 2023-11-09T03:51:10.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.082+0000 7f9732d9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f973415d100 0x7f97340ce510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:39566/0 (socket says 172.21.15.76:39566) 2023-11-09T03:51:10.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.082+0000 7f973359e640 1 -- 172.21.15.76:0/3278810681 learned_addr learned my addr 172.21.15.76:0/3278810681 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:10.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.082+0000 7f973259c640 1 --2- 172.21.15.76:0/3278810681 >> v2:172.21.15.113:3300/0 conn(0x7f973415d4e0 0x7f97340d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:10.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.082+0000 7f973359e640 1 -- 172.21.15.76:0/3278810681 >> v2:172.21.15.113:3300/0 conn(0x7f973415d4e0 msgr2=0x7f97340d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:10.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.082+0000 7f973359e640 1 --2- 172.21.15.76:0/3278810681 >> v2:172.21.15.113:3300/0 conn(0x7f973415d4e0 0x7f97340d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:10.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.082+0000 7f973359e640 1 -- 172.21.15.76:0/3278810681 >> v1:172.21.15.76:6789/0 conn(0x7f973415d100 legacy=0x7f97340ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:10.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.082+0000 7f973359e640 1 -- 172.21.15.76:0/3278810681 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97340b4750 con 0x7f97340d3b60 2023-11-09T03:51:10.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.083+0000 7f973359e640 1 --2- 172.21.15.76:0/3278810681 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97340d3b60 0x7f97340d1a10 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f9734160da0 tx=0x7f972402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293d2c6cb96b0a76 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:10.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.083+0000 7f9731d9b640 1 -- 172.21.15.76:0/3278810681 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9724030860 con 0x7f97340d3b60 2023-11-09T03:51:10.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.083+0000 7f9731d9b640 1 -- 172.21.15.76:0/3278810681 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97240309c0 con 0x7f97340d3b60 2023-11-09T03:51:10.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.083+0000 7f9731d9b640 1 -- 172.21.15.76:0/3278810681 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9724030d50 con 0x7f97340d3b60 2023-11-09T03:51:10.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.084+0000 7f97391c0640 1 -- 172.21.15.76:0/3278810681 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97340d3b60 msgr2=0x7f97340d1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:10.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.084+0000 7f97391c0640 1 --2- 172.21.15.76:0/3278810681 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97340d3b60 0x7f97340d1a10 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f9734160da0 tx=0x7f972402b9a0 comp rx=0 tx=0).stop 2023-11-09T03:51:10.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.084+0000 7f97391c0640 1 -- 172.21.15.76:0/3278810681 shutdown_connections 2023-11-09T03:51:10.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.084+0000 7f97391c0640 1 --2- 172.21.15.76:0/3278810681 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97340d3b60 0x7f97340d1a10 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:10.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.084+0000 7f97391c0640 1 --2- 172.21.15.76:0/3278810681 >> v2:172.21.15.113:3300/0 conn(0x7f973415d4e0 0x7f97340d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:10.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.084+0000 7f97391c0640 1 -- 172.21.15.76:0/3278810681 >> 172.21.15.76:0/3278810681 conn(0x7f97340d2ef0 msgr2=0x7f97340d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:10.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.084+0000 7f97391c0640 1 -- 172.21.15.76:0/3278810681 shutdown_connections 2023-11-09T03:51:10.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.084+0000 7f97391c0640 1 -- 172.21.15.76:0/3278810681 wait complete. 2023-11-09T03:51:10.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.085+0000 7f97391c0640 1 Processor -- start 2023-11-09T03:51:10.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.085+0000 7f97391c0640 1 -- start start 2023-11-09T03:51:10.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.086+0000 7f97391c0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97340d3b60 0x7f97341513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:10.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.086+0000 7f97391c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f973415d100 0x7f97341f6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:10.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.086+0000 7f97391c0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f973415d4e0 0x7f973415f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:10.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.086+0000 7f97391c0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9734151920 con 0x7f973415d100 2023-11-09T03:51:10.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.086+0000 7f97391c0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97340b6a60 con 0x7f97340d3b60 2023-11-09T03:51:10.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.086+0000 7f97391c0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97341f6fb0 con 0x7f973415d4e0 2023-11-09T03:51:10.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.086+0000 7f973259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f973415d100 0x7f97341f6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:10.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.086+0000 7f973259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f973415d100 0x7f97341f6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45288/0 (socket says 172.21.15.76:45288) 2023-11-09T03:51:10.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.086+0000 7f973259c640 1 -- 172.21.15.76:0/3793970843 learned_addr learned my addr 172.21.15.76:0/3793970843 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:10.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.087+0000 7f9732d9d640 1 --2- 172.21.15.76:0/3793970843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97340d3b60 0x7f97341513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:10.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.087+0000 7f973359e640 1 --2- 172.21.15.76:0/3793970843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f973415d4e0 0x7f973415f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:10.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.087+0000 7f973259c640 1 -- 172.21.15.76:0/3793970843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f973415d4e0 msgr2=0x7f973415f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:10.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.087+0000 7f973259c640 1 --2- 172.21.15.76:0/3793970843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f973415d4e0 0x7f973415f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:10.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.087+0000 7f973259c640 1 -- 172.21.15.76:0/3793970843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97340d3b60 msgr2=0x7f97341513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:10.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.087+0000 7f973259c640 1 --2- 172.21.15.76:0/3793970843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97340d3b60 0x7f97341513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:10.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.087+0000 7f973259c640 1 -- 172.21.15.76:0/3793970843 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97240094d0 con 0x7f973415d100 2023-11-09T03:51:10.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.087+0000 7f973259c640 1 --2- 172.21.15.76:0/3793970843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f973415d100 0x7f97341f6a70 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f9728007e60 tx=0x7f972800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:10.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.087+0000 7f971b7fe640 1 -- 172.21.15.76:0/3793970843 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9728014620 con 0x7f973415d100 2023-11-09T03:51:10.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.087+0000 7f971b7fe640 1 -- 172.21.15.76:0/3793970843 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9728014780 con 0x7f973415d100 2023-11-09T03:51:10.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.087+0000 7f971b7fe640 1 -- 172.21.15.76:0/3793970843 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9728014b30 con 0x7f973415d100 2023-11-09T03:51:10.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.087+0000 7f97391c0640 1 -- 172.21.15.76:0/3793970843 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97340b6ef0 con 0x7f973415d100 2023-11-09T03:51:10.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.087+0000 7f97391c0640 1 -- 172.21.15.76:0/3793970843 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97340b6130 con 0x7f973415d100 2023-11-09T03:51:10.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.088+0000 7f97391c0640 1 -- 172.21.15.76:0/3793970843 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97340684b0 con 0x7f973415d100 2023-11-09T03:51:10.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.088+0000 7f971b7fe640 1 -- 172.21.15.76:0/3793970843 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9728014c90 con 0x7f973415d100 2023-11-09T03:51:10.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.089+0000 7f971b7fe640 1 --2- 172.21.15.76:0/3793970843 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f972006ae70 0x7f972006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:10.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.089+0000 7f971b7fe640 1 -- 172.21.15.76:0/3793970843 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 7578+0+0 (secure 0 0 0) 0x7f972808a5d0 con 0x7f973415d100 2023-11-09T03:51:10.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.089+0000 7f9732d9d640 1 --2- 172.21.15.76:0/3793970843 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f972006ae70 0x7f972006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:10.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.090+0000 7f9732d9d640 1 --2- 172.21.15.76:0/3793970843 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f972006ae70 0x7f972006d330 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f972c001020 tx=0x7f972c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:10.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.093+0000 7f971b7fe640 1 -- 172.21.15.76:0/3793970843 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9728054fc0 con 0x7f973415d100 2023-11-09T03:51:10.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.297+0000 7f97391c0640 1 -- 172.21.15.76:0/3793970843 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97340b6ba0 con 0x7f973415d100 2023-11-09T03:51:10.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.298+0000 7f971b7fe640 1 -- 172.21.15.76:0/3793970843 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4932 (secure 0 0 0) 0x7f97280585e0 con 0x7f973415d100 2023-11-09T03:51:10.298 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:10.298 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":83,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6904,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:51:09.024454+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8016,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8016":{"gid":8016,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3229239133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3229239133},{"type":"v1","addr":"172.21.15.76:6835","nonce":3229239133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:51:10.298 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 83 2023-11-09T03:51:10.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.300+0000 7f97391c0640 1 -- 172.21.15.76:0/3793970843 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f972006ae70 msgr2=0x7f972006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:10.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.300+0000 7f97391c0640 1 --2- 172.21.15.76:0/3793970843 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f972006ae70 0x7f972006d330 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f972c001020 tx=0x7f972c009040 comp rx=0 tx=0).stop 2023-11-09T03:51:10.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.300+0000 7f97391c0640 1 -- 172.21.15.76:0/3793970843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f973415d100 msgr2=0x7f97341f6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:10.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.300+0000 7f97391c0640 1 --2- 172.21.15.76:0/3793970843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f973415d100 0x7f97341f6a70 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f9728007e60 tx=0x7f972800e4b0 comp rx=0 tx=0).stop 2023-11-09T03:51:10.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.300+0000 7f97391c0640 1 -- 172.21.15.76:0/3793970843 shutdown_connections 2023-11-09T03:51:10.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.300+0000 7f97391c0640 1 --2- 172.21.15.76:0/3793970843 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f972006ae70 0x7f972006d330 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:10.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.300+0000 7f97391c0640 1 --2- 172.21.15.76:0/3793970843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f973415d4e0 0x7f973415f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:10.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.300+0000 7f97391c0640 1 --2- 172.21.15.76:0/3793970843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f973415d100 0x7f97341f6a70 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:10.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.300+0000 7f97391c0640 1 --2- 172.21.15.76:0/3793970843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97340d3b60 0x7f97341513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:10.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.300+0000 7f97391c0640 1 -- 172.21.15.76:0/3793970843 >> 172.21.15.76:0/3793970843 conn(0x7f97340d2ef0 msgr2=0x7f97340cd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:10.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.300+0000 7f97391c0640 1 -- 172.21.15.76:0/3793970843 shutdown_connections 2023-11-09T03:51:10.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:10.300+0000 7f97391c0640 1 -- 172.21.15.76:0/3793970843 wait complete. 2023-11-09T03:51:10.311 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:51:10.389 INFO:tasks.ceph.mds.c.smithi076.stderr:daemon-helper: command crashed with signal 6 2023-11-09T03:51:12.405 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-11-09T03:51:15.313 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:51:15.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.423+0000 7f4bcaf5e640 1 Processor -- start 2023-11-09T03:51:15.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.423+0000 7f4bcaf5e640 1 -- start start 2023-11-09T03:51:15.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.424+0000 7f4bcaf5e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4bc4159e40 0x7f4bc40b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:15.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.424+0000 7f4bcaf5e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4bc406bb60 0x7f4bc40b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:15.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.424+0000 7f4bcaf5e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc4071500 con 0x7f4bc406fff0 2023-11-09T03:51:15.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.424+0000 7f4bcaf5e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bc415dae0 con 0x7f4bc4159e40 2023-11-09T03:51:15.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.424+0000 7f4bcaf5e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc4070c60 con 0x7f4bc406bb60 2023-11-09T03:51:15.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.424+0000 7f4bc3fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4bc406bb60 0x7f4bc40b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:15.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.424+0000 7f4bc8cfa640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4bc4159e40 0x7f4bc40b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:15.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.424+0000 7f4bc94fb640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f4bc406fff0 0x7f4bc4070480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:52972/0 (socket says 172.21.15.76:52972) 2023-11-09T03:51:15.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.424+0000 7f4bc94fb640 1 -- 172.21.15.76:0/4282129507 learned_addr learned my addr 172.21.15.76:0/4282129507 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:51:15.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.424+0000 7f4bc8cfa640 1 --2- 172.21.15.76:0/4282129507 >> v2:172.21.15.113:3300/0 conn(0x7f4bc4159e40 0x7f4bc40b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:33634/0 (socket says 172.21.15.76:33634) 2023-11-09T03:51:15.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.424+0000 7f4bc3fff640 1 --2- 172.21.15.76:0/4282129507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4bc406bb60 0x7f4bc40b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:43652/0 (socket says 172.21.15.76:43652) 2023-11-09T03:51:15.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.425+0000 7f4bc37fe640 1 -- 172.21.15.76:0/4282129507 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1844438814 0 0) 0x7f4bc4071500 con 0x7f4bc406fff0 2023-11-09T03:51:15.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.425+0000 7f4bc37fe640 1 -- 172.21.15.76:0/4282129507 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bac003520 con 0x7f4bc406fff0 2023-11-09T03:51:15.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.425+0000 7f4bc8cfa640 1 -- 172.21.15.76:0/4282129507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4bc406bb60 msgr2=0x7f4bc40b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:15.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.425+0000 7f4bc8cfa640 1 --2- 172.21.15.76:0/4282129507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4bc406bb60 0x7f4bc40b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:15.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.425+0000 7f4bc8cfa640 1 -- 172.21.15.76:0/4282129507 >> v1:172.21.15.113:6789/0 conn(0x7f4bc406fff0 legacy=0x7f4bc4070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:15.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.425+0000 7f4bc8cfa640 1 -- 172.21.15.76:0/4282129507 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc4070f40 con 0x7f4bc4159e40 2023-11-09T03:51:15.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.425+0000 7f4bc3fff640 1 --2- 172.21.15.76:0/4282129507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4bc406bb60 0x7f4bc40b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:51:15.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.426+0000 7f4bc8cfa640 1 --2- 172.21.15.76:0/4282129507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bc4159e40 0x7f4bc40b0530 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4bb40097a0 tx=0x7f4bb402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3af9a224ca228d7d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:15.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.426+0000 7f4bc37fe640 1 -- 172.21.15.76:0/4282129507 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4031780 con 0x7f4bc4159e40 2023-11-09T03:51:15.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.426+0000 7f4bc37fe640 1 -- 172.21.15.76:0/4282129507 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb40318e0 con 0x7f4bc4159e40 2023-11-09T03:51:15.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.426+0000 7f4bc37fe640 1 -- 172.21.15.76:0/4282129507 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4031c50 con 0x7f4bc4159e40 2023-11-09T03:51:15.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.426+0000 7f4bcaf5e640 1 -- 172.21.15.76:0/4282129507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bc4159e40 msgr2=0x7f4bc40b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:15.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.426+0000 7f4bcaf5e640 1 --2- 172.21.15.76:0/4282129507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bc4159e40 0x7f4bc40b0530 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4bb40097a0 tx=0x7f4bb402b820 comp rx=0 tx=0).stop 2023-11-09T03:51:15.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.426+0000 7f4bcaf5e640 1 -- 172.21.15.76:0/4282129507 shutdown_connections 2023-11-09T03:51:15.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.426+0000 7f4bcaf5e640 1 --2- 172.21.15.76:0/4282129507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4bc406bb60 0x7f4bc40b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:15.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.426+0000 7f4bcaf5e640 1 --2- 172.21.15.76:0/4282129507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bc4159e40 0x7f4bc40b0530 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:15.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.426+0000 7f4bcaf5e640 1 -- 172.21.15.76:0/4282129507 >> 172.21.15.76:0/4282129507 conn(0x7f4bc40cb530 msgr2=0x7f4bc40cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:15.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.426+0000 7f4bcaf5e640 1 -- 172.21.15.76:0/4282129507 shutdown_connections 2023-11-09T03:51:15.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.426+0000 7f4bcaf5e640 1 -- 172.21.15.76:0/4282129507 wait complete. 2023-11-09T03:51:15.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.428+0000 7f4bcaf5e640 1 Processor -- start 2023-11-09T03:51:15.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.428+0000 7f4bcaf5e640 1 -- start start 2023-11-09T03:51:15.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.428+0000 7f4bcaf5e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bc406bb60 0x7f4bc40c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:15.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.428+0000 7f4bcaf5e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4bc406fff0 0x7f4bc40c2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:15.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.428+0000 7f4bcaf5e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bc4159e40 0x7f4bc4150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:15.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.428+0000 7f4bcaf5e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc40c3800 con 0x7f4bc4159e40 2023-11-09T03:51:15.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.428+0000 7f4bcaf5e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc41f39a0 con 0x7f4bc406bb60 2023-11-09T03:51:15.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.428+0000 7f4bcaf5e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc41f3c80 con 0x7f4bc406fff0 2023-11-09T03:51:15.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.429+0000 7f4bc94fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bc4159e40 0x7f4bc4150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:15.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.429+0000 7f4bc94fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bc4159e40 0x7f4bc4150e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34522/0 (socket says 172.21.15.76:34522) 2023-11-09T03:51:15.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.429+0000 7f4bc8cfa640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bc406bb60 0x7f4bc40c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:15.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.429+0000 7f4bc94fb640 1 -- 172.21.15.76:0/3185821977 learned_addr learned my addr 172.21.15.76:0/3185821977 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:15.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.429+0000 7f4bc3fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4bc406fff0 0x7f4bc40c2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:15.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.429+0000 7f4bc94fb640 1 -- 172.21.15.76:0/3185821977 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4bc406fff0 msgr2=0x7f4bc40c2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:15.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.429+0000 7f4bc94fb640 1 --2- 172.21.15.76:0/3185821977 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4bc406fff0 0x7f4bc40c2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:15.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.429+0000 7f4bc94fb640 1 -- 172.21.15.76:0/3185821977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bc406bb60 msgr2=0x7f4bc40c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:15.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.429+0000 7f4bc94fb640 1 --2- 172.21.15.76:0/3185821977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bc406bb60 0x7f4bc40c55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:15.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.429+0000 7f4bc94fb640 1 -- 172.21.15.76:0/3185821977 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb8009400 con 0x7f4bc4159e40 2023-11-09T03:51:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.429+0000 7f4bc94fb640 1 --2- 172.21.15.76:0/3185821977 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bc4159e40 0x7f4bc4150e40 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00cbb0 tx=0x7f4bbc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.429+0000 7f4bc17fa640 1 -- 172.21.15.76:0/3185821977 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc00ad40 con 0x7f4bc4159e40 2023-11-09T03:51:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.429+0000 7f4bc17fa640 1 -- 172.21.15.76:0/3185821977 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bbc012400 con 0x7f4bc4159e40 2023-11-09T03:51:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.429+0000 7f4bcaf5e640 1 -- 172.21.15.76:0/3185821977 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bb4009400 con 0x7f4bc4159e40 2023-11-09T03:51:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.429+0000 7f4bc17fa640 1 -- 172.21.15.76:0/3185821977 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc00aea0 con 0x7f4bc4159e40 2023-11-09T03:51:15.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.429+0000 7f4bcaf5e640 1 -- 172.21.15.76:0/3185821977 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc41f4240 con 0x7f4bc4159e40 2023-11-09T03:51:15.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.430+0000 7f4bcaf5e640 1 -- 172.21.15.76:0/3185821977 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc41f4460 con 0x7f4bc4159e40 2023-11-09T03:51:15.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.431+0000 7f4bc17fa640 1 -- 172.21.15.76:0/3185821977 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4bbc007070 con 0x7f4bc4159e40 2023-11-09T03:51:15.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.431+0000 7f4bc17fa640 1 --2- 172.21.15.76:0/3185821977 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4bb006ae70 0x7f4bb006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:15.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.431+0000 7f4bc17fa640 1 -- 172.21.15.76:0/3185821977 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 7578+0+0 (secure 0 0 0) 0x7f4bbc089c10 con 0x7f4bc4159e40 2023-11-09T03:51:15.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.431+0000 7f4bc8cfa640 1 --2- 172.21.15.76:0/3185821977 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4bb006ae70 0x7f4bb006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:15.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.432+0000 7f4bc8cfa640 1 --2- 172.21.15.76:0/3185821977 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4bb006ae70 0x7f4bb006d330 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4036000 tx=0x7f4bb403d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:15.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.436+0000 7f4bc17fa640 1 -- 172.21.15.76:0/3185821977 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4bbc054630 con 0x7f4bc4159e40 2023-11-09T03:51:15.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.635+0000 7f4bcaf5e640 1 -- 172.21.15.76:0/3185821977 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4bc40684b0 con 0x7f4bc4159e40 2023-11-09T03:51:15.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.636+0000 7f4bc17fa640 1 -- 172.21.15.76:0/3185821977 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4932 (secure 0 0 0) 0x7f4bbc057c50 con 0x7f4bc4159e40 2023-11-09T03:51:15.636 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:15.636 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":83,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6904,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:51:09.024454+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8016,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8016":{"gid":8016,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3229239133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3229239133},{"type":"v1","addr":"172.21.15.76:6835","nonce":3229239133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:51:15.637 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 83 2023-11-09T03:51:15.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.637+0000 7f4bcaf5e640 1 -- 172.21.15.76:0/3185821977 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4bb006ae70 msgr2=0x7f4bb006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:15.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.637+0000 7f4bcaf5e640 1 --2- 172.21.15.76:0/3185821977 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4bb006ae70 0x7f4bb006d330 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4036000 tx=0x7f4bb403d040 comp rx=0 tx=0).stop 2023-11-09T03:51:15.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.637+0000 7f4bcaf5e640 1 -- 172.21.15.76:0/3185821977 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bc4159e40 msgr2=0x7f4bc4150e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:15.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.637+0000 7f4bcaf5e640 1 --2- 172.21.15.76:0/3185821977 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bc4159e40 0x7f4bc4150e40 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00cbb0 tx=0x7f4bbc00a430 comp rx=0 tx=0).stop 2023-11-09T03:51:15.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.638+0000 7f4bcaf5e640 1 -- 172.21.15.76:0/3185821977 shutdown_connections 2023-11-09T03:51:15.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.638+0000 7f4bcaf5e640 1 --2- 172.21.15.76:0/3185821977 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4bb006ae70 0x7f4bb006d330 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:15.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.638+0000 7f4bcaf5e640 1 --2- 172.21.15.76:0/3185821977 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bc4159e40 0x7f4bc4150e40 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:15.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.638+0000 7f4bcaf5e640 1 --2- 172.21.15.76:0/3185821977 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4bc406fff0 0x7f4bc40c2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:15.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.638+0000 7f4bcaf5e640 1 --2- 172.21.15.76:0/3185821977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bc406bb60 0x7f4bc40c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:15.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.638+0000 7f4bcaf5e640 1 -- 172.21.15.76:0/3185821977 >> 172.21.15.76:0/3185821977 conn(0x7f4bc40cb530 msgr2=0x7f4bc40cf4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:15.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.638+0000 7f4bcaf5e640 1 -- 172.21.15.76:0/3185821977 shutdown_connections 2023-11-09T03:51:15.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:15.638+0000 7f4bcaf5e640 1 -- 172.21.15.76:0/3185821977 wait complete. 2023-11-09T03:51:15.648 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:51:19.016 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~7s 2023-11-09T03:51:20.649 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:51:20.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.759+0000 7f834712c640 1 Processor -- start 2023-11-09T03:51:20.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.759+0000 7f834712c640 1 -- start start 2023-11-09T03:51:20.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.760+0000 7f834712c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f834015b200 0x7f83400ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:20.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.760+0000 7f834712c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83400cd9c0 0x7f83400ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:20.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.760+0000 7f834712c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83400cde80 con 0x7f83400afda0 2023-11-09T03:51:20.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.760+0000 7f834712c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83400cf670 con 0x7f83400cd9c0 2023-11-09T03:51:20.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.760+0000 7f834712c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83400cf950 con 0x7f834015b200 2023-11-09T03:51:20.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.760+0000 7f833ffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f83400afda0 0x7f83400cd2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50374/0 (socket says 172.21.15.76:50374) 2023-11-09T03:51:20.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.760+0000 7f833ffff640 1 -- 172.21.15.76:0/75358722 learned_addr learned my addr 172.21.15.76:0/75358722 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:51:20.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.760+0000 7f83456c9640 1 --2- 172.21.15.76:0/75358722 >> v2:172.21.15.113:3300/0 conn(0x7f83400cd9c0 0x7f83400ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:20.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.760+0000 7f8344ec8640 1 --2- 172.21.15.76:0/75358722 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f834015b200 0x7f83400ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:20.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.760+0000 7f833f7fe640 1 -- 172.21.15.76:0/75358722 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3699693740 0 0) 0x7f83400cde80 con 0x7f83400afda0 2023-11-09T03:51:20.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.760+0000 7f833f7fe640 1 -- 172.21.15.76:0/75358722 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8330003520 con 0x7f83400afda0 2023-11-09T03:51:20.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.761+0000 7f83456c9640 1 -- 172.21.15.76:0/75358722 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f834015b200 msgr2=0x7f83400ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:20.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.761+0000 7f833f7fe640 1 -- 172.21.15.76:0/75358722 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2692017802 0 0) 0x7f8330003520 con 0x7f83400afda0 2023-11-09T03:51:20.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.761+0000 7f83456c9640 1 --2- 172.21.15.76:0/75358722 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f834015b200 0x7f83400ceb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:20.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.761+0000 7f83456c9640 1 -- 172.21.15.76:0/75358722 >> v1:172.21.15.76:6789/0 conn(0x7f83400afda0 legacy=0x7f83400cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:20.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.761+0000 7f83456c9640 1 -- 172.21.15.76:0/75358722 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83400ce120 con 0x7f83400cd9c0 2023-11-09T03:51:20.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.761+0000 7f8344ec8640 1 --2- 172.21.15.76:0/75358722 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f834015b200 0x7f83400ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:51:20.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.761+0000 7f83456c9640 1 --2- 172.21.15.76:0/75358722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83400cd9c0 0x7f83400ae2b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8338009d90 tx=0x7f833802bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=489c418af2b66442 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:20.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.762+0000 7f833f7fe640 1 -- 172.21.15.76:0/75358722 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c001670 con 0x7f83400cd9c0 2023-11-09T03:51:20.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.762+0000 7f833f7fe640 1 -- 172.21.15.76:0/75358722 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833802c810 con 0x7f83400cd9c0 2023-11-09T03:51:20.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.762+0000 7f833f7fe640 1 -- 172.21.15.76:0/75358722 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833802cba0 con 0x7f83400cd9c0 2023-11-09T03:51:20.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.762+0000 7f834712c640 1 -- 172.21.15.76:0/75358722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83400cd9c0 msgr2=0x7f83400ae2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:20.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.762+0000 7f834712c640 1 --2- 172.21.15.76:0/75358722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83400cd9c0 0x7f83400ae2b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8338009d90 tx=0x7f833802bce0 comp rx=0 tx=0).stop 2023-11-09T03:51:20.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.762+0000 7f834712c640 1 -- 172.21.15.76:0/75358722 shutdown_connections 2023-11-09T03:51:20.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.762+0000 7f834712c640 1 --2- 172.21.15.76:0/75358722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83400cd9c0 0x7f83400ae2b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:20.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.762+0000 7f834712c640 1 --2- 172.21.15.76:0/75358722 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f834015b200 0x7f83400ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:20.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.762+0000 7f834712c640 1 -- 172.21.15.76:0/75358722 >> 172.21.15.76:0/75358722 conn(0x7f83400d0c30 msgr2=0x7f83400d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:20.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.762+0000 7f834712c640 1 -- 172.21.15.76:0/75358722 shutdown_connections 2023-11-09T03:51:20.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.762+0000 7f834712c640 1 -- 172.21.15.76:0/75358722 wait complete. 2023-11-09T03:51:20.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.763+0000 7f834712c640 1 Processor -- start 2023-11-09T03:51:20.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.763+0000 7f834712c640 1 -- start start 2023-11-09T03:51:20.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.764+0000 7f834712c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83400afda0 0x7f83401530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:20.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.764+0000 7f834712c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83400cd9c0 0x7f83400b5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:20.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.764+0000 7f834712c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f834015b200 0x7f83400b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:20.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.764+0000 7f834712c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83400c9630 con 0x7f834015b200 2023-11-09T03:51:20.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.764+0000 7f834712c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83400b4380 con 0x7f83400cd9c0 2023-11-09T03:51:20.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.764+0000 7f834712c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83400b4660 con 0x7f83400afda0 2023-11-09T03:51:20.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.764+0000 7f83456c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f834015b200 0x7f83400b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:20.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.764+0000 7f83456c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f834015b200 0x7f83400b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34532/0 (socket says 172.21.15.76:34532) 2023-11-09T03:51:20.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.764+0000 7f83456c9640 1 -- 172.21.15.76:0/3519702794 learned_addr learned my addr 172.21.15.76:0/3519702794 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:20.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.765+0000 7f8344ec8640 1 --2- 172.21.15.76:0/3519702794 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83400afda0 0x7f83401530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:20.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.765+0000 7f833ffff640 1 --2- 172.21.15.76:0/3519702794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83400cd9c0 0x7f83400b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:20.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.765+0000 7f83456c9640 1 -- 172.21.15.76:0/3519702794 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83400afda0 msgr2=0x7f83401530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:20.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.765+0000 7f83456c9640 1 --2- 172.21.15.76:0/3519702794 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83400afda0 0x7f83401530d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:20.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.765+0000 7f83456c9640 1 -- 172.21.15.76:0/3519702794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83400cd9c0 msgr2=0x7f83400b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:20.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.765+0000 7f83456c9640 1 --2- 172.21.15.76:0/3519702794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83400cd9c0 0x7f83400b5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:20.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.765+0000 7f83456c9640 1 -- 172.21.15.76:0/3519702794 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83340093a0 con 0x7f834015b200 2023-11-09T03:51:20.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.765+0000 7f83456c9640 1 --2- 172.21.15.76:0/3519702794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f834015b200 0x7f83400b5da0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f8338009d90 tx=0x7f8338004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:20.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.765+0000 7f833d7fa640 1 -- 172.21.15.76:0/3519702794 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8338009450 con 0x7f834015b200 2023-11-09T03:51:20.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.765+0000 7f833d7fa640 1 -- 172.21.15.76:0/3519702794 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8338034c50 con 0x7f834015b200 2023-11-09T03:51:20.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.765+0000 7f834712c640 1 -- 172.21.15.76:0/3519702794 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8338009a70 con 0x7f834015b200 2023-11-09T03:51:20.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.765+0000 7f833d7fa640 1 -- 172.21.15.76:0/3519702794 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8338040450 con 0x7f834015b200 2023-11-09T03:51:20.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.765+0000 7f834712c640 1 -- 172.21.15.76:0/3519702794 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83400b4c20 con 0x7f834015b200 2023-11-09T03:51:20.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.766+0000 7f834712c640 1 -- 172.21.15.76:0/3519702794 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83400684b0 con 0x7f834015b200 2023-11-09T03:51:20.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.767+0000 7f833d7fa640 1 -- 172.21.15.76:0/3519702794 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f83380405b0 con 0x7f834015b200 2023-11-09T03:51:20.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.767+0000 7f833d7fa640 1 --2- 172.21.15.76:0/3519702794 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f832406ae70 0x7f832406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:20.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.767+0000 7f833d7fa640 1 -- 172.21.15.76:0/3519702794 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7550+0+0 (secure 0 0 0) 0x7f83380ac680 con 0x7f834015b200 2023-11-09T03:51:20.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.767+0000 7f8344ec8640 1 --2- 172.21.15.76:0/3519702794 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f832406ae70 0x7f832406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:20.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.768+0000 7f8344ec8640 1 --2- 172.21.15.76:0/3519702794 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f832406ae70 0x7f832406d330 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8334009ad0 tx=0x7f8334008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:20.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.771+0000 7f833d7fa640 1 -- 172.21.15.76:0/3519702794 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8338077090 con 0x7f834015b200 2023-11-09T03:51:20.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.972+0000 7f834712c640 1 -- 172.21.15.76:0/3519702794 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83401e5980 con 0x7f834015b200 2023-11-09T03:51:20.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.973+0000 7f833d7fa640 1 -- 172.21.15.76:0/3519702794 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4932 (secure 0 0 0) 0x7f833807a6b0 con 0x7f834015b200 2023-11-09T03:51:20.973 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:20.973 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":83,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6904,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:51:09.024454+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8016,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8016":{"gid":8016,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3229239133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3229239133},{"type":"v1","addr":"172.21.15.76:6835","nonce":3229239133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:51:20.973 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 83 2023-11-09T03:51:20.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.974+0000 7f834712c640 1 -- 172.21.15.76:0/3519702794 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f832406ae70 msgr2=0x7f832406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:20.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.974+0000 7f834712c640 1 --2- 172.21.15.76:0/3519702794 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f832406ae70 0x7f832406d330 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8334009ad0 tx=0x7f8334008040 comp rx=0 tx=0).stop 2023-11-09T03:51:20.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.974+0000 7f834712c640 1 -- 172.21.15.76:0/3519702794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f834015b200 msgr2=0x7f83400b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:20.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.974+0000 7f834712c640 1 --2- 172.21.15.76:0/3519702794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f834015b200 0x7f83400b5da0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f8338009d90 tx=0x7f8338004a40 comp rx=0 tx=0).stop 2023-11-09T03:51:20.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.975+0000 7f834712c640 1 -- 172.21.15.76:0/3519702794 shutdown_connections 2023-11-09T03:51:20.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.975+0000 7f834712c640 1 --2- 172.21.15.76:0/3519702794 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f832406ae70 0x7f832406d330 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:20.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.975+0000 7f834712c640 1 --2- 172.21.15.76:0/3519702794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f834015b200 0x7f83400b5da0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:20.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.975+0000 7f834712c640 1 --2- 172.21.15.76:0/3519702794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83400cd9c0 0x7f83400b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:20.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.975+0000 7f834712c640 1 --2- 172.21.15.76:0/3519702794 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83400afda0 0x7f83401530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:20.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.975+0000 7f834712c640 1 -- 172.21.15.76:0/3519702794 >> 172.21.15.76:0/3519702794 conn(0x7f83400d0c30 msgr2=0x7f83400b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:20.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.975+0000 7f834712c640 1 -- 172.21.15.76:0/3519702794 shutdown_connections 2023-11-09T03:51:20.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:20.975+0000 7f834712c640 1 -- 172.21.15.76:0/3519702794 wait complete. 2023-11-09T03:51:20.985 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:51:25.627 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~13s 2023-11-09T03:51:25.986 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:51:26.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.100+0000 7f016081a640 1 Processor -- start 2023-11-09T03:51:26.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.101+0000 7f016081a640 1 -- start start 2023-11-09T03:51:26.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.102+0000 7f016081a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f015c15d4e0 0x7f015c0d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:26.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.102+0000 7f016081a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f015c0d3b60 0x7f015c0d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:26.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.102+0000 7f016081a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f015c0b32b0 con 0x7f015c15d100 2023-11-09T03:51:26.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.102+0000 7f016081a640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f015c0cec20 con 0x7f015c15d4e0 2023-11-09T03:51:26.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.102+0000 7f016081a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f015c0cee10 con 0x7f015c0d3b60 2023-11-09T03:51:26.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.102+0000 7f0159d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f015c15d4e0 0x7f015c0d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:26.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.102+0000 7f0159d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f015c15d4e0 0x7f015c0d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38908/0 (socket says 172.21.15.76:38908) 2023-11-09T03:51:26.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.102+0000 7f0159d9b640 1 -- 172.21.15.76:0/3293400489 learned_addr learned my addr 172.21.15.76:0/3293400489 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:26.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.102+0000 7f015ad9d640 1 --2- 172.21.15.76:0/3293400489 >> v2:172.21.15.113:3300/0 conn(0x7f015c0d3b60 0x7f015c0d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:26.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.102+0000 7f0159d9b640 1 -- 172.21.15.76:0/3293400489 >> v2:172.21.15.113:3300/0 conn(0x7f015c0d3b60 msgr2=0x7f015c0d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:26.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.102+0000 7f0159d9b640 1 --2- 172.21.15.76:0/3293400489 >> v2:172.21.15.113:3300/0 conn(0x7f015c0d3b60 0x7f015c0d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:26.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.102+0000 7f0159d9b640 1 -- 172.21.15.76:0/3293400489 >> v1:172.21.15.113:6789/0 conn(0x7f015c15d100 legacy=0x7f015c0ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:26.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.102+0000 7f0159d9b640 1 -- 172.21.15.76:0/3293400489 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015c0b4750 con 0x7f015c15d4e0 2023-11-09T03:51:26.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.103+0000 7f0159d9b640 1 --2- 172.21.15.76:0/3293400489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f015c15d4e0 0x7f015c0d14d0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f015c08d6a0 tx=0x7f014c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290c087b85d2539a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:26.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.103+0000 7f015959a640 1 -- 172.21.15.76:0/3293400489 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c031870 con 0x7f015c15d4e0 2023-11-09T03:51:26.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.103+0000 7f015959a640 1 -- 172.21.15.76:0/3293400489 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014c0319d0 con 0x7f015c15d4e0 2023-11-09T03:51:26.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.103+0000 7f015959a640 1 -- 172.21.15.76:0/3293400489 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c031d60 con 0x7f015c15d4e0 2023-11-09T03:51:26.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.104+0000 7f016081a640 1 -- 172.21.15.76:0/3293400489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f015c15d4e0 msgr2=0x7f015c0d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:26.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.104+0000 7f016081a640 1 --2- 172.21.15.76:0/3293400489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f015c15d4e0 0x7f015c0d14d0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f015c08d6a0 tx=0x7f014c02b910 comp rx=0 tx=0).stop 2023-11-09T03:51:26.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.104+0000 7f016081a640 1 -- 172.21.15.76:0/3293400489 shutdown_connections 2023-11-09T03:51:26.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.104+0000 7f016081a640 1 --2- 172.21.15.76:0/3293400489 >> v2:172.21.15.113:3300/0 conn(0x7f015c0d3b60 0x7f015c0d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:26.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.104+0000 7f016081a640 1 --2- 172.21.15.76:0/3293400489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f015c15d4e0 0x7f015c0d14d0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:26.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.104+0000 7f016081a640 1 -- 172.21.15.76:0/3293400489 >> 172.21.15.76:0/3293400489 conn(0x7f015c0d2ef0 msgr2=0x7f015c0d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:26.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.104+0000 7f016081a640 1 -- 172.21.15.76:0/3293400489 shutdown_connections 2023-11-09T03:51:26.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.104+0000 7f016081a640 1 -- 172.21.15.76:0/3293400489 wait complete. 2023-11-09T03:51:26.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.105+0000 7f016081a640 1 Processor -- start 2023-11-09T03:51:26.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.105+0000 7f016081a640 1 -- start start 2023-11-09T03:51:26.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.106+0000 7f016081a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f015c0d3b60 0x7f015c1571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:26.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.106+0000 7f016081a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f015c15d100 0x7f015c156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:26.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.106+0000 7f016081a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f015c15d4e0 0x7f015c0b8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:26.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.106+0000 7f016081a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c0cc300 con 0x7f015c15d100 2023-11-09T03:51:26.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.106+0000 7f016081a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c0b7350 con 0x7f015c0d3b60 2023-11-09T03:51:26.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.106+0000 7f016081a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f015c0cb680 con 0x7f015c15d4e0 2023-11-09T03:51:26.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.106+0000 7f0159d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f015c15d100 0x7f015c156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:26.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.106+0000 7f0159d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f015c15d100 0x7f015c156b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38916/0 (socket says 172.21.15.76:38916) 2023-11-09T03:51:26.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.106+0000 7f0159d9b640 1 -- 172.21.15.76:0/362044174 learned_addr learned my addr 172.21.15.76:0/362044174 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:26.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.106+0000 7f015a59c640 1 --2- 172.21.15.76:0/362044174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f015c0d3b60 0x7f015c1571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:26.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.106+0000 7f015ad9d640 1 --2- 172.21.15.76:0/362044174 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f015c15d4e0 0x7f015c0b8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:26.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.106+0000 7f0159d9b640 1 -- 172.21.15.76:0/362044174 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f015c15d4e0 msgr2=0x7f015c0b8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:26.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.106+0000 7f0159d9b640 1 --2- 172.21.15.76:0/362044174 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f015c15d4e0 0x7f015c0b8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:26.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.106+0000 7f0159d9b640 1 -- 172.21.15.76:0/362044174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f015c0d3b60 msgr2=0x7f015c1571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:26.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.106+0000 7f0159d9b640 1 --2- 172.21.15.76:0/362044174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f015c0d3b60 0x7f015c1571b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:26.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.106+0000 7f0159d9b640 1 -- 172.21.15.76:0/362044174 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014c0094d0 con 0x7f015c15d100 2023-11-09T03:51:26.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.107+0000 7f0159d9b640 1 --2- 172.21.15.76:0/362044174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f015c15d100 0x7f015c156b30 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f014c0090b0 tx=0x7f014c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:26.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.107+0000 7f014affd640 1 -- 172.21.15.76:0/362044174 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c03ac60 con 0x7f015c15d100 2023-11-09T03:51:26.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.107+0000 7f014affd640 1 -- 172.21.15.76:0/362044174 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014c03adc0 con 0x7f015c15d100 2023-11-09T03:51:26.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.107+0000 7f016081a640 1 -- 172.21.15.76:0/362044174 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f015c0cb960 con 0x7f015c15d100 2023-11-09T03:51:26.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.107+0000 7f014affd640 1 -- 172.21.15.76:0/362044174 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c044630 con 0x7f015c15d100 2023-11-09T03:51:26.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.107+0000 7f016081a640 1 -- 172.21.15.76:0/362044174 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f015c0cbf20 con 0x7f015c15d100 2023-11-09T03:51:26.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.108+0000 7f016081a640 1 -- 172.21.15.76:0/362044174 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f015c0d0470 con 0x7f015c15d100 2023-11-09T03:51:26.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.108+0000 7f014affd640 1 -- 172.21.15.76:0/362044174 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f014c044790 con 0x7f015c15d100 2023-11-09T03:51:26.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.109+0000 7f014affd640 1 --2- 172.21.15.76:0/362044174 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f013c06ae70 0x7f013c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:26.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.109+0000 7f014affd640 1 -- 172.21.15.76:0/362044174 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7550+0+0 (secure 0 0 0) 0x7f014c02d040 con 0x7f015c15d100 2023-11-09T03:51:26.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.109+0000 7f015a59c640 1 --2- 172.21.15.76:0/362044174 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f013c06ae70 0x7f013c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:26.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.109+0000 7f015a59c640 1 --2- 172.21.15.76:0/362044174 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f013c06ae70 0x7f013c06d330 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f015c1f6340 tx=0x7f0150009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:26.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.113+0000 7f014affd640 1 -- 172.21.15.76:0/362044174 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f014c076cc0 con 0x7f015c15d100 2023-11-09T03:51:26.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.315+0000 7f016081a640 1 -- 172.21.15.76:0/362044174 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f015c0684b0 con 0x7f015c15d100 2023-11-09T03:51:26.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.315+0000 7f014affd640 1 -- 172.21.15.76:0/362044174 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4932 (secure 0 0 0) 0x7f014c07a2e0 con 0x7f015c15d100 2023-11-09T03:51:26.316 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:26.316 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":83,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6904,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:51:09.024454+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8016,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8016":{"gid":8016,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3229239133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3229239133},{"type":"v1","addr":"172.21.15.76:6835","nonce":3229239133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:51:26.317 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 83 2023-11-09T03:51:26.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.317+0000 7f016081a640 1 -- 172.21.15.76:0/362044174 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f013c06ae70 msgr2=0x7f013c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:26.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.317+0000 7f016081a640 1 --2- 172.21.15.76:0/362044174 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f013c06ae70 0x7f013c06d330 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f015c1f6340 tx=0x7f0150009040 comp rx=0 tx=0).stop 2023-11-09T03:51:26.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.317+0000 7f016081a640 1 -- 172.21.15.76:0/362044174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f015c15d100 msgr2=0x7f015c156b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:26.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.317+0000 7f016081a640 1 --2- 172.21.15.76:0/362044174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f015c15d100 0x7f015c156b30 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f014c0090b0 tx=0x7f014c0046c0 comp rx=0 tx=0).stop 2023-11-09T03:51:26.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.317+0000 7f016081a640 1 -- 172.21.15.76:0/362044174 shutdown_connections 2023-11-09T03:51:26.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.317+0000 7f016081a640 1 --2- 172.21.15.76:0/362044174 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f013c06ae70 0x7f013c06d330 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:26.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.317+0000 7f016081a640 1 --2- 172.21.15.76:0/362044174 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f015c15d4e0 0x7f015c0b8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:26.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.317+0000 7f016081a640 1 --2- 172.21.15.76:0/362044174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f015c15d100 0x7f015c156b30 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:26.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.317+0000 7f016081a640 1 --2- 172.21.15.76:0/362044174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f015c0d3b60 0x7f015c1571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:26.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.317+0000 7f016081a640 1 -- 172.21.15.76:0/362044174 >> 172.21.15.76:0/362044174 conn(0x7f015c0d2ef0 msgr2=0x7f015c0b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:26.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.317+0000 7f016081a640 1 -- 172.21.15.76:0/362044174 shutdown_connections 2023-11-09T03:51:26.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:26.317+0000 7f016081a640 1 -- 172.21.15.76:0/362044174 wait complete. 2023-11-09T03:51:26.329 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:51:31.329 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:51:31.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.443+0000 7fc451efc640 1 Processor -- start 2023-11-09T03:51:31.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.443+0000 7fc451efc640 1 -- start start 2023-11-09T03:51:31.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.443+0000 7fc451efc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc44c0ade00 0x7fc44c0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:31.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.443+0000 7fc451efc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc44c06cc90 con 0x7fc44c06d9c0 2023-11-09T03:51:31.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.443+0000 7fc451efc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc44c0b0e60 con 0x7fc44c06c5f0 2023-11-09T03:51:31.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.443+0000 7fc451efc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc44c06cee0 con 0x7fc44c0ade00 2023-11-09T03:51:31.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.444+0000 7fc44b7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc44c0ade00 0x7fc44c0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:31.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.444+0000 7fc44affd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc44c06d9c0 0x7fc44c0710d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:38598/0 (socket says 172.21.15.76:38598) 2023-11-09T03:51:31.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.444+0000 7fc44affd640 1 -- 172.21.15.76:0/609188378 learned_addr learned my addr 172.21.15.76:0/609188378 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:51:31.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.444+0000 7fc44b7fe640 1 -- 172.21.15.76:0/609188378 >> v1:172.21.15.113:6789/0 conn(0x7fc44c06c5f0 legacy=0x7fc44c0cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:31.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.444+0000 7fc44b7fe640 1 -- 172.21.15.76:0/609188378 >> v1:172.21.15.76:6789/0 conn(0x7fc44c06d9c0 legacy=0x7fc44c0710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:31.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.444+0000 7fc44b7fe640 1 -- 172.21.15.76:0/609188378 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44c06d1c0 con 0x7fc44c0ade00 2023-11-09T03:51:31.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.444+0000 7fc44a7fc640 1 -- 172.21.15.76:0/609188378 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2786252149 0 0) 0x7fc44c06cc90 con 0x7fc44c06d9c0 2023-11-09T03:51:31.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.445+0000 7fc44b7fe640 1 --2- 172.21.15.76:0/609188378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc44c0ade00 0x7fc44c0b0530 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009d90 tx=0x7fc43c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214ee24cfe9c7d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:31.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.445+0000 7fc44a7fc640 1 -- 172.21.15.76:0/609188378 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c02cbf0 con 0x7fc44c0ade00 2023-11-09T03:51:31.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.445+0000 7fc44a7fc640 1 -- 172.21.15.76:0/609188378 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43c02cd50 con 0x7fc44c0ade00 2023-11-09T03:51:31.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.445+0000 7fc44a7fc640 1 -- 172.21.15.76:0/609188378 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c034660 con 0x7fc44c0ade00 2023-11-09T03:51:31.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.445+0000 7fc451efc640 1 -- 172.21.15.76:0/609188378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc44c0ade00 msgr2=0x7fc44c0b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:31.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.445+0000 7fc451efc640 1 --2- 172.21.15.76:0/609188378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc44c0ade00 0x7fc44c0b0530 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009d90 tx=0x7fc43c02be40 comp rx=0 tx=0).stop 2023-11-09T03:51:31.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.446+0000 7fc451efc640 1 -- 172.21.15.76:0/609188378 shutdown_connections 2023-11-09T03:51:31.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.446+0000 7fc451efc640 1 --2- 172.21.15.76:0/609188378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc44c0ade00 0x7fc44c0b0530 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:31.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.446+0000 7fc451efc640 1 -- 172.21.15.76:0/609188378 >> 172.21.15.76:0/609188378 conn(0x7fc44c0cb530 msgr2=0x7fc44c0cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:31.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.446+0000 7fc451efc640 1 -- 172.21.15.76:0/609188378 shutdown_connections 2023-11-09T03:51:31.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.446+0000 7fc451efc640 1 -- 172.21.15.76:0/609188378 wait complete. 2023-11-09T03:51:31.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.447+0000 7fc451efc640 1 Processor -- start 2023-11-09T03:51:31.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.447+0000 7fc451efc640 1 -- start start 2023-11-09T03:51:31.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.448+0000 7fc451efc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc44c06c5f0 0x7fc44c1f4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:31.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.448+0000 7fc451efc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc44c06d9c0 0x7fc44c1570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:31.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.448+0000 7fc451efc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc44c0ade00 0x7fc44c155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:31.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.448+0000 7fc451efc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c1f4ad0 con 0x7fc44c06c5f0 2023-11-09T03:51:31.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.448+0000 7fc451efc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c1f3af0 con 0x7fc44c0ade00 2023-11-09T03:51:31.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.448+0000 7fc451efc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c157920 con 0x7fc44c06d9c0 2023-11-09T03:51:31.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.448+0000 7fc44b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc44c06c5f0 0x7fc44c1f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:31.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.448+0000 7fc44b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc44c06c5f0 0x7fc44c1f4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38930/0 (socket says 172.21.15.76:38930) 2023-11-09T03:51:31.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.448+0000 7fc44b7fe640 1 -- 172.21.15.76:0/2271361631 learned_addr learned my addr 172.21.15.76:0/2271361631 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:31.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.448+0000 7fc44b7fe640 1 -- 172.21.15.76:0/2271361631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc44c06d9c0 msgr2=0x7fc44c1570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:31.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.448+0000 7fc44b7fe640 1 --2- 172.21.15.76:0/2271361631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc44c06d9c0 0x7fc44c1570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:31.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.448+0000 7fc44b7fe640 1 -- 172.21.15.76:0/2271361631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc44c0ade00 msgr2=0x7fc44c155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:31.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.448+0000 7fc44bfff640 1 --2- 172.21.15.76:0/2271361631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc44c0ade00 0x7fc44c155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:31.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.448+0000 7fc44b7fe640 1 --2- 172.21.15.76:0/2271361631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc44c0ade00 0x7fc44c155390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:31.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.448+0000 7fc44b7fe640 1 -- 172.21.15.76:0/2271361631 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc43c009a70 con 0x7fc44c06c5f0 2023-11-09T03:51:31.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.448+0000 7fc44b7fe640 1 --2- 172.21.15.76:0/2271361631 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc44c06c5f0 0x7fc44c1f4590 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fc43c000f80 tx=0x7fc43c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:31.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.449+0000 7fc433fff640 1 -- 172.21.15.76:0/2271361631 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c003b90 con 0x7fc44c06c5f0 2023-11-09T03:51:31.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.449+0000 7fc433fff640 1 -- 172.21.15.76:0/2271361631 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43c003cf0 con 0x7fc44c06c5f0 2023-11-09T03:51:31.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.449+0000 7fc451efc640 1 -- 172.21.15.76:0/2271361631 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc44c157c00 con 0x7fc44c06c5f0 2023-11-09T03:51:31.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.449+0000 7fc433fff640 1 -- 172.21.15.76:0/2271361631 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c004230 con 0x7fc44c06c5f0 2023-11-09T03:51:31.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.449+0000 7fc451efc640 1 -- 172.21.15.76:0/2271361631 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44c1f4070 con 0x7fc44c06c5f0 2023-11-09T03:51:31.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.450+0000 7fc451efc640 1 -- 172.21.15.76:0/2271361631 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44c0684b0 con 0x7fc44c06c5f0 2023-11-09T03:51:31.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.451+0000 7fc433fff640 1 -- 172.21.15.76:0/2271361631 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc43c004390 con 0x7fc44c06c5f0 2023-11-09T03:51:31.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.451+0000 7fc433fff640 1 --2- 172.21.15.76:0/2271361631 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc43806ae70 0x7fc43806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:31.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.451+0000 7fc433fff640 1 -- 172.21.15.76:0/2271361631 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7550+0+0 (secure 0 0 0) 0x7fc43c0ace20 con 0x7fc44c06c5f0 2023-11-09T03:51:31.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.452+0000 7fc44affd640 1 --2- 172.21.15.76:0/2271361631 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc43806ae70 0x7fc43806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:31.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.452+0000 7fc44affd640 1 --2- 172.21.15.76:0/2271361631 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc43806ae70 0x7fc43806d330 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc440000c00 tx=0x7fc4400079c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:31.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.457+0000 7fc433fff640 1 -- 172.21.15.76:0/2271361631 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc43c077930 con 0x7fc44c06c5f0 2023-11-09T03:51:31.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.655+0000 7fc451efc640 1 -- 172.21.15.76:0/2271361631 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc44c0cddc0 con 0x7fc44c06c5f0 2023-11-09T03:51:31.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.656+0000 7fc433fff640 1 -- 172.21.15.76:0/2271361631 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4980 (secure 0 0 0) 0x7fc43c039070 con 0x7fc44c06c5f0 2023-11-09T03:51:31.657 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:31.657 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":84,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6904,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:51:26.449284+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8016,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8016":{"gid":8016,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3229239133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3229239133},{"type":"v1","addr":"172.21.15.76:6835","nonce":3229239133}]},"join_fscid":-1,"laggy_since":"2023-11-09T03:51:26.449283+0000","export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:51:31.657 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 84 2023-11-09T03:51:31.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.658+0000 7fc451efc640 1 -- 172.21.15.76:0/2271361631 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc43806ae70 msgr2=0x7fc43806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:31.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.658+0000 7fc451efc640 1 --2- 172.21.15.76:0/2271361631 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc43806ae70 0x7fc43806d330 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc440000c00 tx=0x7fc4400079c0 comp rx=0 tx=0).stop 2023-11-09T03:51:31.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.658+0000 7fc451efc640 1 -- 172.21.15.76:0/2271361631 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc44c06c5f0 msgr2=0x7fc44c1f4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:31.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.658+0000 7fc451efc640 1 --2- 172.21.15.76:0/2271361631 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc44c06c5f0 0x7fc44c1f4590 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fc43c000f80 tx=0x7fc43c004a40 comp rx=0 tx=0).stop 2023-11-09T03:51:31.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.658+0000 7fc451efc640 1 -- 172.21.15.76:0/2271361631 shutdown_connections 2023-11-09T03:51:31.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.658+0000 7fc451efc640 1 --2- 172.21.15.76:0/2271361631 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc43806ae70 0x7fc43806d330 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:31.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.658+0000 7fc451efc640 1 --2- 172.21.15.76:0/2271361631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc44c0ade00 0x7fc44c155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:31.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.658+0000 7fc451efc640 1 --2- 172.21.15.76:0/2271361631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc44c06d9c0 0x7fc44c1570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:31.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.658+0000 7fc451efc640 1 --2- 172.21.15.76:0/2271361631 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc44c06c5f0 0x7fc44c1f4590 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:31.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.658+0000 7fc451efc640 1 -- 172.21.15.76:0/2271361631 >> 172.21.15.76:0/2271361631 conn(0x7fc44c0cb530 msgr2=0x7fc44c0cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:31.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.658+0000 7fc451efc640 1 -- 172.21.15.76:0/2271361631 shutdown_connections 2023-11-09T03:51:31.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.658+0000 7fc451efc640 1 -- 172.21.15.76:0/2271361631 wait complete. 2023-11-09T03:51:31.669 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2023-11-09T03:51:31.670 DEBUG:teuthology.orchestra.run.smithi076:> sudo sysctl -n kernel.core_pattern 2023-11-09T03:51:31.695 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-09T03:51:31.696 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2023-11-09T03:51:31.697 DEBUG:teuthology.orchestra.run.smithi076:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2023-11-09T03:51:31.766 INFO:teuthology.orchestra.run.smithi076.stdout:1699501869.120519.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i c', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2023-11-09T03:51:31.768 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1699501869.120519.core'] 2023-11-09T03:51:31.768 INFO:tasks.cephfs.cephfs_test_case:Found core file 1699501869.120519.core, deleting it 2023-11-09T03:51:31.769 DEBUG:teuthology.orchestra.run.smithi076:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1699501869.120519.core 2023-11-09T03:51:31.838 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:51:31.882 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:51:31.923 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:51:31.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.979+0000 7f1d3f061640 1 Processor -- start 2023-11-09T03:51:31.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.980+0000 7f1d3f061640 1 -- start start 2023-11-09T03:51:31.981 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.980+0000 7f1d3f061640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1d3815d4e0 0x7f1d380ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:31.981 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.980+0000 7f1d3f061640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d380ced90 0x7f1d380ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:31.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.980+0000 7f1d3f061640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d380b25a0 con 0x7f1d380ca6e0 2023-11-09T03:51:31.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.980+0000 7f1d3f061640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d380cab70 con 0x7f1d3815d4e0 2023-11-09T03:51:31.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.980+0000 7f1d3f061640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d380b2300 con 0x7f1d380ced90 2023-11-09T03:51:31.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.980+0000 7f1d3cdfd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1d3815d4e0 0x7f1d380ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:31.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.980+0000 7f1d3cdfd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1d3815d4e0 0x7f1d380ce850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38940/0 (socket says 172.21.15.76:38940) 2023-11-09T03:51:31.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.980+0000 7f1d3cdfd640 1 -- 172.21.15.76:0/3096994195 learned_addr learned my addr 172.21.15.76:0/3096994195 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:31.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.981+0000 7f1d37fff640 1 --2- 172.21.15.76:0/3096994195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d380ced90 0x7f1d380ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:31.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.981+0000 7f1d3cdfd640 1 -- 172.21.15.76:0/3096994195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d380ced90 msgr2=0x7f1d380ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:31.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.981+0000 7f1d3cdfd640 1 --2- 172.21.15.76:0/3096994195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d380ced90 0x7f1d380ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:31.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.981+0000 7f1d3cdfd640 1 -- 172.21.15.76:0/3096994195 >> v1:172.21.15.113:6789/0 conn(0x7f1d380ca6e0 legacy=0x7f1d38160fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:31.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.981+0000 7f1d3cdfd640 1 -- 172.21.15.76:0/3096994195 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3815d8c0 con 0x7f1d3815d4e0 2023-11-09T03:51:31.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.982+0000 7f1d3cdfd640 1 --2- 172.21.15.76:0/3096994195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d3815d4e0 0x7f1d380ce850 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009870 tx=0x7f1d2802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4820f1b53536cde7 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:31.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.982+0000 7f1d377fe640 1 -- 172.21.15.76:0/3096994195 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d28031640 con 0x7f1d3815d4e0 2023-11-09T03:51:31.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.982+0000 7f1d377fe640 1 -- 172.21.15.76:0/3096994195 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d280317a0 con 0x7f1d3815d4e0 2023-11-09T03:51:31.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.982+0000 7f1d377fe640 1 -- 172.21.15.76:0/3096994195 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d28031b30 con 0x7f1d3815d4e0 2023-11-09T03:51:31.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.982+0000 7f1d3f061640 1 -- 172.21.15.76:0/3096994195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d3815d4e0 msgr2=0x7f1d380ce850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:31.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.982+0000 7f1d3f061640 1 --2- 172.21.15.76:0/3096994195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d3815d4e0 0x7f1d380ce850 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009870 tx=0x7f1d2802b8f0 comp rx=0 tx=0).stop 2023-11-09T03:51:31.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.982+0000 7f1d3f061640 1 -- 172.21.15.76:0/3096994195 shutdown_connections 2023-11-09T03:51:31.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.982+0000 7f1d3f061640 1 --2- 172.21.15.76:0/3096994195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d380ced90 0x7f1d380ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:31.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.982+0000 7f1d3f061640 1 --2- 172.21.15.76:0/3096994195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d3815d4e0 0x7f1d380ce850 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:31.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.982+0000 7f1d3f061640 1 -- 172.21.15.76:0/3096994195 >> 172.21.15.76:0/3096994195 conn(0x7f1d380b1640 msgr2=0x7f1d380b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:31.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.982+0000 7f1d3f061640 1 -- 172.21.15.76:0/3096994195 shutdown_connections 2023-11-09T03:51:31.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.982+0000 7f1d3f061640 1 -- 172.21.15.76:0/3096994195 wait complete. 2023-11-09T03:51:31.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.984+0000 7f1d3f061640 1 Processor -- start 2023-11-09T03:51:31.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.984+0000 7f1d3f061640 1 -- start start 2023-11-09T03:51:31.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.984+0000 7f1d3f061640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d380ca6e0 0x7f1d38158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:31.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.984+0000 7f1d3f061640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d380ced90 0x7f1d380c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:31.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.984+0000 7f1d3f061640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d3815d4e0 0x7f1d380c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:31.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.984+0000 7f1d3f061640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d381e4600 con 0x7f1d3815d4e0 2023-11-09T03:51:31.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.984+0000 7f1d3f061640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d381e48e0 con 0x7f1d380ca6e0 2023-11-09T03:51:31.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.984+0000 7f1d3f061640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d38150b60 con 0x7f1d380ced90 2023-11-09T03:51:31.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.985+0000 7f1d3d5fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d3815d4e0 0x7f1d380c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:31.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.985+0000 7f1d3d5fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d3815d4e0 0x7f1d380c53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38952/0 (socket says 172.21.15.76:38952) 2023-11-09T03:51:31.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.985+0000 7f1d3d5fe640 1 -- 172.21.15.76:0/461800747 learned_addr learned my addr 172.21.15.76:0/461800747 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:31.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.985+0000 7f1d3cdfd640 1 --2- 172.21.15.76:0/461800747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d380ca6e0 0x7f1d38158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:31.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.985+0000 7f1d37fff640 1 --2- 172.21.15.76:0/461800747 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d380ced90 0x7f1d380c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:31.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.985+0000 7f1d3d5fe640 1 -- 172.21.15.76:0/461800747 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d380ced90 msgr2=0x7f1d380c4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:31.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.985+0000 7f1d3d5fe640 1 --2- 172.21.15.76:0/461800747 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d380ced90 0x7f1d380c4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:31.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.985+0000 7f1d3d5fe640 1 -- 172.21.15.76:0/461800747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d380ca6e0 msgr2=0x7f1d38158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:31.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.985+0000 7f1d3d5fe640 1 --2- 172.21.15.76:0/461800747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d380ca6e0 0x7f1d38158470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:31.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.985+0000 7f1d3d5fe640 1 -- 172.21.15.76:0/461800747 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d280094d0 con 0x7f1d3815d4e0 2023-11-09T03:51:31.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.985+0000 7f1d3d5fe640 1 --2- 172.21.15.76:0/461800747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d3815d4e0 0x7f1d380c53c0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000c780 tx=0x7f1d3000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:31.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.985+0000 7f1d357fa640 1 -- 172.21.15.76:0/461800747 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d30004b60 con 0x7f1d3815d4e0 2023-11-09T03:51:31.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.985+0000 7f1d357fa640 1 -- 172.21.15.76:0/461800747 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3000ce90 con 0x7f1d3815d4e0 2023-11-09T03:51:31.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.985+0000 7f1d3f061640 1 -- 172.21.15.76:0/461800747 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d38150e40 con 0x7f1d3815d4e0 2023-11-09T03:51:31.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.985+0000 7f1d357fa640 1 -- 172.21.15.76:0/461800747 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3000a850 con 0x7f1d3815d4e0 2023-11-09T03:51:31.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.985+0000 7f1d3f061640 1 -- 172.21.15.76:0/461800747 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d381589b0 con 0x7f1d3815d4e0 2023-11-09T03:51:31.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.986+0000 7f1d3f061640 1 -- 172.21.15.76:0/461800747 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d38151360 con 0x7f1d3815d4e0 2023-11-09T03:51:31.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.987+0000 7f1d357fa640 1 -- 172.21.15.76:0/461800747 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1d3000a9b0 con 0x7f1d3815d4e0 2023-11-09T03:51:31.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.987+0000 7f1d357fa640 1 --2- 172.21.15.76:0/461800747 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1d1806ae70 0x7f1d1806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:31.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.987+0000 7f1d357fa640 1 -- 172.21.15.76:0/461800747 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7550+0+0 (secure 0 0 0) 0x7f1d30089e60 con 0x7f1d3815d4e0 2023-11-09T03:51:31.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.987+0000 7f1d3cdfd640 1 --2- 172.21.15.76:0/461800747 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1d1806ae70 0x7f1d1806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:31.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.988+0000 7f1d3cdfd640 1 --2- 172.21.15.76:0/461800747 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1d1806ae70 0x7f1d1806d330 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1d28036000 tx=0x7f1d2803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:31.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:31.992+0000 7f1d357fa640 1 -- 172.21.15.76:0/461800747 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1d38151360 con 0x7f1d3815d4e0 2023-11-09T03:51:32.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.194+0000 7f1d3f061640 1 -- 172.21.15.76:0/461800747 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d38151360 con 0x7f1d3815d4e0 2023-11-09T03:51:32.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.195+0000 7f1d357fa640 1 -- 172.21.15.76:0/461800747 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4980 (secure 0 0 0) 0x7f1d38151360 con 0x7f1d3815d4e0 2023-11-09T03:51:32.196 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:32.196 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":84,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6904,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:51:26.449284+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8016,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8016":{"gid":8016,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3229239133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3229239133},{"type":"v1","addr":"172.21.15.76:6835","nonce":3229239133}]},"join_fscid":-1,"laggy_since":"2023-11-09T03:51:26.449283+0000","export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:51:32.196 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 84 2023-11-09T03:51:32.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.197+0000 7f1d3f061640 1 -- 172.21.15.76:0/461800747 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1d1806ae70 msgr2=0x7f1d1806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:32.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.197+0000 7f1d3f061640 1 --2- 172.21.15.76:0/461800747 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1d1806ae70 0x7f1d1806d330 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1d28036000 tx=0x7f1d2803c040 comp rx=0 tx=0).stop 2023-11-09T03:51:32.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.197+0000 7f1d3f061640 1 -- 172.21.15.76:0/461800747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d3815d4e0 msgr2=0x7f1d380c53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:32.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.197+0000 7f1d3f061640 1 --2- 172.21.15.76:0/461800747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d3815d4e0 0x7f1d380c53c0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000c780 tx=0x7f1d3000cc40 comp rx=0 tx=0).stop 2023-11-09T03:51:32.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.197+0000 7f1d3f061640 1 -- 172.21.15.76:0/461800747 shutdown_connections 2023-11-09T03:51:32.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.197+0000 7f1d3f061640 1 --2- 172.21.15.76:0/461800747 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1d1806ae70 0x7f1d1806d330 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:32.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.197+0000 7f1d3f061640 1 --2- 172.21.15.76:0/461800747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d3815d4e0 0x7f1d380c53c0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:32.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.197+0000 7f1d3f061640 1 --2- 172.21.15.76:0/461800747 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d380ced90 0x7f1d380c4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:32.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.197+0000 7f1d3f061640 1 --2- 172.21.15.76:0/461800747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d380ca6e0 0x7f1d38158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:32.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.197+0000 7f1d3f061640 1 -- 172.21.15.76:0/461800747 >> 172.21.15.76:0/461800747 conn(0x7f1d380b1640 msgr2=0x7f1d38072ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:32.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.197+0000 7f1d3f061640 1 -- 172.21.15.76:0/461800747 shutdown_connections 2023-11-09T03:51:32.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.197+0000 7f1d3f061640 1 -- 172.21.15.76:0/461800747 wait complete. 2023-11-09T03:51:32.208 INFO:tasks.ceph.mds.a:Sent signal 9 2023-11-09T03:51:32.209 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:51:32.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:51:32.210 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 down 2023-11-09T03:51:32.237 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:0 2023-11-09T03:51:32.238 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~20s 2023-11-09T03:51:32.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.377+0000 7f1e37407640 1 Processor -- start 2023-11-09T03:51:32.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.377+0000 7f1e37407640 1 -- start start 2023-11-09T03:51:32.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.377+0000 7f1e37407640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1e30070700 0x7f1e30164b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:32.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.378+0000 7f1e37407640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e30071500 con 0x7f1e30159e40 2023-11-09T03:51:32.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.378+0000 7f1e37407640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e3015dae0 con 0x7f1e300b0c40 2023-11-09T03:51:32.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.378+0000 7f1e37407640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e30070c60 con 0x7f1e30070700 2023-11-09T03:51:32.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.378+0000 7f1e359a4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1e30070700 0x7f1e30164b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:32.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.378+0000 7f1e359a4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1e30070700 0x7f1e30164b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38956/0 (socket says 172.21.15.76:38956) 2023-11-09T03:51:32.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.378+0000 7f1e359a4640 1 -- 172.21.15.76:0/973315509 learned_addr learned my addr 172.21.15.76:0/973315509 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:32.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.378+0000 7f1e359a4640 1 -- 172.21.15.76:0/973315509 >> v1:172.21.15.113:6789/0 conn(0x7f1e300b0c40 legacy=0x7f1e3006fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:32.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.378+0000 7f1e359a4640 1 -- 172.21.15.76:0/973315509 >> v1:172.21.15.76:6789/0 conn(0x7f1e30159e40 legacy=0x7f1e300b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:32.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.378+0000 7f1e359a4640 1 -- 172.21.15.76:0/973315509 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e30070f40 con 0x7f1e30070700 2023-11-09T03:51:32.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.378+0000 7f1e27fff640 1 -- 172.21.15.76:0/973315509 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1358797068 0 0) 0x7f1e30071500 con 0x7f1e30159e40 2023-11-09T03:51:32.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.379+0000 7f1e359a4640 1 --2- 172.21.15.76:0/973315509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e30070700 0x7f1e30164b60 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009870 tx=0x7f1e2802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e4ae86616a98c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:32.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.379+0000 7f1e27fff640 1 -- 172.21.15.76:0/973315509 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e28031860 con 0x7f1e30070700 2023-11-09T03:51:32.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.379+0000 7f1e27fff640 1 -- 172.21.15.76:0/973315509 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e280319c0 con 0x7f1e30070700 2023-11-09T03:51:32.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.379+0000 7f1e27fff640 1 -- 172.21.15.76:0/973315509 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e28031d50 con 0x7f1e30070700 2023-11-09T03:51:32.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.379+0000 7f1e37407640 1 -- 172.21.15.76:0/973315509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e30070700 msgr2=0x7f1e30164b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:32.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.379+0000 7f1e37407640 1 --2- 172.21.15.76:0/973315509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e30070700 0x7f1e30164b60 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009870 tx=0x7f1e2802b900 comp rx=0 tx=0).stop 2023-11-09T03:51:32.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.379+0000 7f1e37407640 1 -- 172.21.15.76:0/973315509 shutdown_connections 2023-11-09T03:51:32.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.379+0000 7f1e37407640 1 --2- 172.21.15.76:0/973315509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e30070700 0x7f1e30164b60 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:32.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.379+0000 7f1e37407640 1 -- 172.21.15.76:0/973315509 >> 172.21.15.76:0/973315509 conn(0x7f1e300cb530 msgr2=0x7f1e300cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:32.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.380+0000 7f1e37407640 1 -- 172.21.15.76:0/973315509 shutdown_connections 2023-11-09T03:51:32.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.380+0000 7f1e37407640 1 -- 172.21.15.76:0/973315509 wait complete. 2023-11-09T03:51:32.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.381+0000 7f1e37407640 1 Processor -- start 2023-11-09T03:51:32.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.381+0000 7f1e37407640 1 -- start start 2023-11-09T03:51:32.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.381+0000 7f1e37407640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e30070700 0x7f1e300c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:32.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.381+0000 7f1e37407640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e300b0c40 0x7f1e300cec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:32.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.381+0000 7f1e37407640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e30159e40 0x7f1e30150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:32.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.381+0000 7f1e37407640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e301e39c0 con 0x7f1e30159e40 2023-11-09T03:51:32.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.381+0000 7f1e37407640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e300c75e0 con 0x7f1e30070700 2023-11-09T03:51:32.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.381+0000 7f1e37407640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e300c8890 con 0x7f1e300b0c40 2023-11-09T03:51:32.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.382+0000 7f1e359a4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e30159e40 0x7f1e30150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:32.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.382+0000 7f1e359a4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e30159e40 0x7f1e30150e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38972/0 (socket says 172.21.15.76:38972) 2023-11-09T03:51:32.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.382+0000 7f1e359a4640 1 -- 172.21.15.76:0/2666607391 learned_addr learned my addr 172.21.15.76:0/2666607391 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:32.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.382+0000 7f1e351a3640 1 --2- 172.21.15.76:0/2666607391 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e30070700 0x7f1e300c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:32.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.382+0000 7f1e349a2640 1 --2- 172.21.15.76:0/2666607391 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e300b0c40 0x7f1e300cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:32.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.382+0000 7f1e359a4640 1 -- 172.21.15.76:0/2666607391 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e300b0c40 msgr2=0x7f1e300cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:32.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.382+0000 7f1e359a4640 1 --2- 172.21.15.76:0/2666607391 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e300b0c40 0x7f1e300cec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:32.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.382+0000 7f1e359a4640 1 -- 172.21.15.76:0/2666607391 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e30070700 msgr2=0x7f1e300c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:32.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.382+0000 7f1e359a4640 1 --2- 172.21.15.76:0/2666607391 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e30070700 0x7f1e300c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:32.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.382+0000 7f1e359a4640 1 -- 172.21.15.76:0/2666607391 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e280094d0 con 0x7f1e30159e40 2023-11-09T03:51:32.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.382+0000 7f1e351a3640 1 --2- 172.21.15.76:0/2666607391 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e30070700 0x7f1e300c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:51:32.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.382+0000 7f1e359a4640 1 --2- 172.21.15.76:0/2666607391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e30159e40 0x7f1e30150e00 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009870 tx=0x7f1e280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:32.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.382+0000 7f1e25ffb640 1 -- 172.21.15.76:0/2666607391 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2803adf0 con 0x7f1e30159e40 2023-11-09T03:51:32.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.382+0000 7f1e25ffb640 1 -- 172.21.15.76:0/2666607391 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e280443f0 con 0x7f1e30159e40 2023-11-09T03:51:32.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.382+0000 7f1e37407640 1 -- 172.21.15.76:0/2666607391 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e300c7830 con 0x7f1e30159e40 2023-11-09T03:51:32.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.382+0000 7f1e25ffb640 1 -- 172.21.15.76:0/2666607391 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e280447a0 con 0x7f1e30159e40 2023-11-09T03:51:32.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.382+0000 7f1e37407640 1 -- 172.21.15.76:0/2666607391 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e301e4f90 con 0x7f1e30159e40 2023-11-09T03:51:32.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.383+0000 7f1e37407640 1 -- 172.21.15.76:0/2666607391 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e301e4df0 con 0x7f1e30159e40 2023-11-09T03:51:32.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.384+0000 7f1e25ffb640 1 -- 172.21.15.76:0/2666607391 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1e28044900 con 0x7f1e30159e40 2023-11-09T03:51:32.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.384+0000 7f1e25ffb640 1 --2- 172.21.15.76:0/2666607391 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1e1406ae70 0x7f1e1406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:32.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.384+0000 7f1e25ffb640 1 -- 172.21.15.76:0/2666607391 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7550+0+0 (secure 0 0 0) 0x7f1e280ac440 con 0x7f1e30159e40 2023-11-09T03:51:32.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.385+0000 7f1e351a3640 1 --2- 172.21.15.76:0/2666607391 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1e1406ae70 0x7f1e1406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:32.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.385+0000 7f1e351a3640 1 --2- 172.21.15.76:0/2666607391 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1e1406ae70 0x7f1e1406d330 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c0012e0 tx=0x7f1e2c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:32.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.389+0000 7f1e25ffb640 1 -- 172.21.15.76:0/2666607391 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1e301e4df0 con 0x7f1e30159e40 2023-11-09T03:51:32.410 INFO:tasks.ceph.mds.a.smithi076.stderr:daemon-helper: command crashed with signal 9 2023-11-09T03:51:32.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.623+0000 7f1e37407640 1 -- 172.21.15.76:0/2666607391 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:0"} v 0) v1 -- 0x7f1e301e4df0 con 0x7f1e30159e40 2023-11-09T03:51:32.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.624+0000 7f1e25ffb640 1 -- 172.21.15.76:0/2666607391 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:0"}]=0 failed mds gid 8016 v85) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1e301e4df0 con 0x7f1e30159e40 2023-11-09T03:51:32.625 INFO:teuthology.orchestra.run.smithi076.stderr:failed mds gid 8016 2023-11-09T03:51:32.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.627+0000 7f1e37407640 1 -- 172.21.15.76:0/2666607391 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1e1406ae70 msgr2=0x7f1e1406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:32.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.627+0000 7f1e37407640 1 --2- 172.21.15.76:0/2666607391 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1e1406ae70 0x7f1e1406d330 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c0012e0 tx=0x7f1e2c009040 comp rx=0 tx=0).stop 2023-11-09T03:51:32.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.627+0000 7f1e37407640 1 -- 172.21.15.76:0/2666607391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e30159e40 msgr2=0x7f1e30150e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:32.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.627+0000 7f1e37407640 1 --2- 172.21.15.76:0/2666607391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e30159e40 0x7f1e30150e00 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009870 tx=0x7f1e280046c0 comp rx=0 tx=0).stop 2023-11-09T03:51:32.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.627+0000 7f1e37407640 1 -- 172.21.15.76:0/2666607391 shutdown_connections 2023-11-09T03:51:32.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.627+0000 7f1e37407640 1 --2- 172.21.15.76:0/2666607391 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1e1406ae70 0x7f1e1406d330 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:32.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.627+0000 7f1e37407640 1 --2- 172.21.15.76:0/2666607391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e30159e40 0x7f1e30150e00 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:32.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.627+0000 7f1e37407640 1 --2- 172.21.15.76:0/2666607391 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e300b0c40 0x7f1e300cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:32.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.628+0000 7f1e37407640 1 --2- 172.21.15.76:0/2666607391 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e30070700 0x7f1e300c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:32.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.628+0000 7f1e37407640 1 -- 172.21.15.76:0/2666607391 >> 172.21.15.76:0/2666607391 conn(0x7f1e300cb530 msgr2=0x7f1e301582a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:32.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.628+0000 7f1e37407640 1 -- 172.21.15.76:0/2666607391 shutdown_connections 2023-11-09T03:51:32.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.628+0000 7f1e37407640 1 -- 172.21.15.76:0/2666607391 wait complete. 2023-11-09T03:51:32.642 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-09T03:51:32.642 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-09T03:51:32.642 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-09T03:51:32.643 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:51:32.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:51:32.644 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi076 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c' 2023-11-09T03:51:32.644 INFO:tasks.ceph.mds.c:Stopped 2023-11-09T03:51:32.644 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-09T03:51:32.646 INFO:tasks.ceph.mds.c:Started 2023-11-09T03:51:32.647 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:51:32.722 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2023-11-09T03:51:32.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.748+0000 7f829640e640 1 Processor -- start 2023-11-09T03:51:32.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.748+0000 7f829640e640 1 -- start start 2023-11-09T03:51:32.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.749+0000 7f829640e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f829015d4e0 0x7f82900ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:32.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.749+0000 7f829640e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82900b25a0 con 0x7f82900ca8b0 2023-11-09T03:51:32.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.749+0000 7f829640e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82900b2300 con 0x7f82900ced90 2023-11-09T03:51:32.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.749+0000 7f829640e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829015d8c0 con 0x7f829015d4e0 2023-11-09T03:51:32.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.749+0000 7f82949ab640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f82900ca8b0 0x7f829006a0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:55912/0 (socket says 172.21.15.76:55912) 2023-11-09T03:51:32.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.749+0000 7f82949ab640 1 -- 172.21.15.76:0/3882360170 learned_addr learned my addr 172.21.15.76:0/3882360170 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:51:32.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.749+0000 7f828effd640 1 -- 172.21.15.76:0/3882360170 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2830090779 0 0) 0x7f82900b25a0 con 0x7f82900ca8b0 2023-11-09T03:51:32.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.749+0000 7f828ffff640 1 --2- 172.21.15.76:0/3882360170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f829015d4e0 0x7f82900ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:32.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.749+0000 7f828effd640 1 -- 172.21.15.76:0/3882360170 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8278003520 con 0x7f82900ca8b0 2023-11-09T03:51:32.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.750+0000 7f828effd640 1 -- 172.21.15.76:0/3882360170 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3408291265 0 0) 0x7f8278003520 con 0x7f82900ca8b0 2023-11-09T03:51:32.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.750+0000 7f828effd640 1 -- 172.21.15.76:0/3882360170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f829015d4e0 msgr2=0x7f82900ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:32.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.750+0000 7f828effd640 1 --2- 172.21.15.76:0/3882360170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f829015d4e0 0x7f82900ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:32.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.750+0000 7f828effd640 1 -- 172.21.15.76:0/3882360170 >> v1:172.21.15.113:6789/0 conn(0x7f82900ced90 legacy=0x7f82900ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:32.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.750+0000 7f828effd640 1 -- 172.21.15.76:0/3882360170 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f829015e5f0 con 0x7f82900ca8b0 2023-11-09T03:51:32.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.750+0000 7f828effd640 1 -- 172.21.15.76:0/3882360170 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f82880019f0 con 0x7f82900ca8b0 2023-11-09T03:51:32.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.750+0000 7f828effd640 1 -- 172.21.15.76:0/3882360170 >> v1:172.21.15.76:6789/0 conn(0x7f82900ca8b0 legacy=0x7f829006a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:32.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.750+0000 7f828effd640 1 --2- 172.21.15.76:0/3882360170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82780039a0 0x7f8278003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:32.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.750+0000 7f828effd640 1 --2- 172.21.15.76:0/3882360170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f829015d4e0 0x7f82780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:32.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.750+0000 7f828effd640 1 --2- 172.21.15.76:0/3882360170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8278004d00 0x7f8278005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:32.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.750+0000 7f828effd640 1 -- 172.21.15.76:0/3882360170 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829015d8c0 con 0x7f82780039a0 2023-11-09T03:51:32.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.750+0000 7f828effd640 1 -- 172.21.15.76:0/3882360170 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827800ba30 con 0x7f8278004d00 2023-11-09T03:51:32.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.750+0000 7f828effd640 1 -- 172.21.15.76:0/3882360170 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827800bbf0 con 0x7f829015d4e0 2023-11-09T03:51:32.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.750+0000 7f828ffff640 1 --2- 172.21.15.76:0/3882360170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82780039a0 0x7f8278003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:32.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.750+0000 7f828f7fe640 1 --2- 172.21.15.76:0/3882360170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f829015d4e0 0x7f82780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:32.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.750+0000 7f82949ab640 1 --2- 172.21.15.76:0/3882360170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8278004d00 0x7f8278005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:32.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.750+0000 7f828ffff640 1 -- 172.21.15.76:0/3882360170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f829015d4e0 msgr2=0x7f82780047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:32.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.750+0000 7f828ffff640 1 --2- 172.21.15.76:0/3882360170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f829015d4e0 0x7f82780047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:32.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.750+0000 7f828ffff640 1 -- 172.21.15.76:0/3882360170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8278004d00 msgr2=0x7f8278005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:32.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.750+0000 7f828ffff640 1 --2- 172.21.15.76:0/3882360170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8278004d00 0x7f8278005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:32.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.750+0000 7f828ffff640 1 -- 172.21.15.76:0/3882360170 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8278003520 con 0x7f82780039a0 2023-11-09T03:51:32.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.751+0000 7f828ffff640 1 --2- 172.21.15.76:0/3882360170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82780039a0 0x7f8278003dc0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f8284009a90 tx=0x7f82780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d878693230c2a25 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:32.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.751+0000 7f828effd640 1 -- 172.21.15.76:0/3882360170 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828402c680 con 0x7f82780039a0 2023-11-09T03:51:32.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.751+0000 7f828effd640 1 -- 172.21.15.76:0/3882360170 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828402c7e0 con 0x7f82780039a0 2023-11-09T03:51:32.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.751+0000 7f828effd640 1 -- 172.21.15.76:0/3882360170 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828402cb70 con 0x7f82780039a0 2023-11-09T03:51:32.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.752+0000 7f829640e640 1 -- 172.21.15.76:0/3882360170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82780039a0 msgr2=0x7f8278003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:32.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.752+0000 7f829640e640 1 --2- 172.21.15.76:0/3882360170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82780039a0 0x7f8278003dc0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f8284009a90 tx=0x7f82780057f0 comp rx=0 tx=0).stop 2023-11-09T03:51:32.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.752+0000 7f82949ab640 1 -- 172.21.15.76:0/3882360170 reap_dead start 2023-11-09T03:51:32.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.752+0000 7f829640e640 1 -- 172.21.15.76:0/3882360170 shutdown_connections 2023-11-09T03:51:32.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.752+0000 7f829640e640 1 -- 172.21.15.76:0/3882360170 >> 172.21.15.76:0/3882360170 conn(0x7f82900b1640 msgr2=0x7f82900b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:32.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.752+0000 7f829640e640 1 -- 172.21.15.76:0/3882360170 shutdown_connections 2023-11-09T03:51:32.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.752+0000 7f829640e640 1 -- 172.21.15.76:0/3882360170 wait complete. 2023-11-09T03:51:32.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.753+0000 7f829640e640 1 Processor -- start 2023-11-09T03:51:32.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.753+0000 7f829640e640 1 -- start start 2023-11-09T03:51:32.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.754+0000 7f829640e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82780039a0 0x7f829015b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:32.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.754+0000 7f829640e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82901f2dd0 0x7f829015b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:32.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.754+0000 7f829640e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82901f67f0 0x7f82901f6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:32.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.754+0000 7f829640e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82900b5990 con 0x7f82901f2dd0 2023-11-09T03:51:32.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.754+0000 7f829640e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82901f71b0 con 0x7f82780039a0 2023-11-09T03:51:32.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.754+0000 7f829640e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82900c4760 con 0x7f82901f67f0 2023-11-09T03:51:32.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.754+0000 7f828f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82901f2dd0 0x7f829015b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:32.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.754+0000 7f828f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82901f2dd0 0x7f829015b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59362/0 (socket says 172.21.15.76:59362) 2023-11-09T03:51:32.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.754+0000 7f828f7fe640 1 -- 172.21.15.76:0/86605491 learned_addr learned my addr 172.21.15.76:0/86605491 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:32.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.754+0000 7f828ffff640 1 --2- 172.21.15.76:0/86605491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82780039a0 0x7f829015b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:32.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.754+0000 7f82949ab640 1 --2- 172.21.15.76:0/86605491 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82901f67f0 0x7f82901f6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:32.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.754+0000 7f828f7fe640 1 -- 172.21.15.76:0/86605491 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82901f67f0 msgr2=0x7f82901f6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:32.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.754+0000 7f828f7fe640 1 --2- 172.21.15.76:0/86605491 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82901f67f0 0x7f82901f6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:32.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.754+0000 7f828f7fe640 1 -- 172.21.15.76:0/86605491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82780039a0 msgr2=0x7f829015b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:32.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.754+0000 7f828f7fe640 1 --2- 172.21.15.76:0/86605491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82780039a0 0x7f829015b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:32.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.754+0000 7f828f7fe640 1 -- 172.21.15.76:0/86605491 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f829015bf60 con 0x7f82901f2dd0 2023-11-09T03:51:32.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.754+0000 7f828ffff640 1 --2- 172.21.15.76:0/86605491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82780039a0 0x7f829015b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:51:32.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.754+0000 7f828f7fe640 1 --2- 172.21.15.76:0/86605491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82901f2dd0 0x7f829015b950 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f828000f820 tx=0x7f828000fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:32.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.754+0000 7f828cff9640 1 -- 172.21.15.76:0/86605491 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82800050d0 con 0x7f82901f2dd0 2023-11-09T03:51:32.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.754+0000 7f828cff9640 1 -- 172.21.15.76:0/86605491 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8280005230 con 0x7f82901f2dd0 2023-11-09T03:51:32.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.754+0000 7f829640e640 1 -- 172.21.15.76:0/86605491 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f829015c240 con 0x7f82901f2dd0 2023-11-09T03:51:32.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.755+0000 7f828cff9640 1 -- 172.21.15.76:0/86605491 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828000d6d0 con 0x7f82901f2dd0 2023-11-09T03:51:32.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.755+0000 7f829640e640 1 -- 172.21.15.76:0/86605491 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f829015c800 con 0x7f82901f2dd0 2023-11-09T03:51:32.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.756+0000 7f829640e640 1 -- 172.21.15.76:0/86605491 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82900c3420 con 0x7f82901f2dd0 2023-11-09T03:51:32.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.756+0000 7f828cff9640 1 -- 172.21.15.76:0/86605491 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f828000d830 con 0x7f82901f2dd0 2023-11-09T03:51:32.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.756+0000 7f828cff9640 1 --2- 172.21.15.76:0/86605491 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f827c06ae70 0x7f827c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:32.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.756+0000 7f828cff9640 1 -- 172.21.15.76:0/86605491 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7608+0+0 (secure 0 0 0) 0x7f8280089ce0 con 0x7f82901f2dd0 2023-11-09T03:51:32.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.757+0000 7f828ffff640 1 --2- 172.21.15.76:0/86605491 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f827c06ae70 0x7f827c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:32.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.757+0000 7f828ffff640 1 --2- 172.21.15.76:0/86605491 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f827c06ae70 0x7f827c06d330 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8284036000 tx=0x7f8284030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:32.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.761+0000 7f828cff9640 1 -- 172.21.15.76:0/86605491 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f82800545a0 con 0x7f82901f2dd0 2023-11-09T03:51:32.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.963+0000 7f829640e640 1 -- 172.21.15.76:0/86605491 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82900684b0 con 0x7f82901f2dd0 2023-11-09T03:51:32.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.964+0000 7f828cff9640 1 -- 172.21.15.76:0/86605491 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4140 (secure 0 0 0) 0x7f8280057bc0 con 0x7f82901f2dd0 2023-11-09T03:51:32.965 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:32.965 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":86,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:51:32.629126+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6904,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6904":{"gid":6904,"name":"b","rank":0,"incarnation":86,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:51:32.965 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 86 2023-11-09T03:51:32.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.966+0000 7f829640e640 1 -- 172.21.15.76:0/86605491 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f827c06ae70 msgr2=0x7f827c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:32.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.966+0000 7f829640e640 1 --2- 172.21.15.76:0/86605491 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f827c06ae70 0x7f827c06d330 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8284036000 tx=0x7f8284030000 comp rx=0 tx=0).stop 2023-11-09T03:51:32.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.966+0000 7f829640e640 1 -- 172.21.15.76:0/86605491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82901f2dd0 msgr2=0x7f829015b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:32.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.966+0000 7f829640e640 1 --2- 172.21.15.76:0/86605491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82901f2dd0 0x7f829015b950 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f828000f820 tx=0x7f828000fce0 comp rx=0 tx=0).stop 2023-11-09T03:51:32.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.966+0000 7f829640e640 1 -- 172.21.15.76:0/86605491 shutdown_connections 2023-11-09T03:51:32.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.966+0000 7f829640e640 1 --2- 172.21.15.76:0/86605491 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f827c06ae70 0x7f827c06d330 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:32.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.966+0000 7f829640e640 1 --2- 172.21.15.76:0/86605491 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82901f67f0 0x7f82901f6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:32.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.966+0000 7f829640e640 1 --2- 172.21.15.76:0/86605491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82901f2dd0 0x7f829015b950 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:32.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.966+0000 7f829640e640 1 --2- 172.21.15.76:0/86605491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82780039a0 0x7f829015b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:32.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.966+0000 7f829640e640 1 -- 172.21.15.76:0/86605491 >> 172.21.15.76:0/86605491 conn(0x7f82900b1640 msgr2=0x7f829006ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:32.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.966+0000 7f829640e640 1 -- 172.21.15.76:0/86605491 shutdown_connections 2023-11-09T03:51:32.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:32.966+0000 7f829640e640 1 -- 172.21.15.76:0/86605491 wait complete. 2023-11-09T03:51:32.977 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-11-09T03:51:37.978 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:51:38.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.089+0000 7fddd064b640 1 Processor -- start 2023-11-09T03:51:38.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.089+0000 7fddd064b640 1 -- start start 2023-11-09T03:51:38.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.090+0000 7fddd064b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fddc8159e40 0x7fddc80b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:38.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.090+0000 7fddd064b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fddc806bb60 0x7fddc80b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:38.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.090+0000 7fddd064b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fddc806fff0 0x7fddc8070480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:38.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.090+0000 7fddd064b640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc8071500 con 0x7fddc806fff0 2023-11-09T03:51:38.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.090+0000 7fddd064b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc815dae0 con 0x7fddc806bb60 2023-11-09T03:51:38.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.090+0000 7fddd064b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc80709c0 con 0x7fddc8159e40 2023-11-09T03:51:38.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.090+0000 7fddcebe8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fddc806fff0 0x7fddc8070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:38.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.090+0000 7fddcebe8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fddc806fff0 0x7fddc8070480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59378/0 (socket says 172.21.15.76:59378) 2023-11-09T03:51:38.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.090+0000 7fddcebe8640 1 -- 172.21.15.76:0/636208956 learned_addr learned my addr 172.21.15.76:0/636208956 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:38.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.090+0000 7fddcebe8640 1 -- 172.21.15.76:0/636208956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fddc8159e40 msgr2=0x7fddc80b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:38.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.090+0000 7fddce3e7640 1 --2- 172.21.15.76:0/636208956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fddc8159e40 0x7fddc80b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:38.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.090+0000 7fddcdbe6640 1 --2- 172.21.15.76:0/636208956 >> v2:172.21.15.113:3300/0 conn(0x7fddc806bb60 0x7fddc80b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:38.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.090+0000 7fddcebe8640 1 --2- 172.21.15.76:0/636208956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fddc8159e40 0x7fddc80b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.090+0000 7fddcebe8640 1 -- 172.21.15.76:0/636208956 >> v2:172.21.15.113:3300/0 conn(0x7fddc806bb60 msgr2=0x7fddc80b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:38.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.090+0000 7fddcebe8640 1 --2- 172.21.15.76:0/636208956 >> v2:172.21.15.113:3300/0 conn(0x7fddc806bb60 0x7fddc80b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.090+0000 7fddcebe8640 1 -- 172.21.15.76:0/636208956 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc8070c60 con 0x7fddc806fff0 2023-11-09T03:51:38.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.090+0000 7fddcdbe6640 1 --2- 172.21.15.76:0/636208956 >> v2:172.21.15.113:3300/0 conn(0x7fddc806bb60 0x7fddc80b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:51:38.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.091+0000 7fddce3e7640 1 --2- 172.21.15.76:0/636208956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fddc8159e40 0x7fddc80b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:51:38.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.091+0000 7fddcebe8640 1 --2- 172.21.15.76:0/636208956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddc806fff0 0x7fddc8070480 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fddc80ce820 tx=0x7fddc002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec477542dde9364 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:38.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.091+0000 7fddcd3e5640 1 -- 172.21.15.76:0/636208956 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc002c9e0 con 0x7fddc806fff0 2023-11-09T03:51:38.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.091+0000 7fddcd3e5640 1 -- 172.21.15.76:0/636208956 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc002cb40 con 0x7fddc806fff0 2023-11-09T03:51:38.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.091+0000 7fddcd3e5640 1 -- 172.21.15.76:0/636208956 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc00343f0 con 0x7fddc806fff0 2023-11-09T03:51:38.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.092+0000 7fddd064b640 1 -- 172.21.15.76:0/636208956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddc806fff0 msgr2=0x7fddc8070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:38.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.092+0000 7fddd064b640 1 --2- 172.21.15.76:0/636208956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddc806fff0 0x7fddc8070480 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fddc80ce820 tx=0x7fddc002be40 comp rx=0 tx=0).stop 2023-11-09T03:51:38.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.092+0000 7fddd064b640 1 -- 172.21.15.76:0/636208956 shutdown_connections 2023-11-09T03:51:38.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.092+0000 7fddd064b640 1 --2- 172.21.15.76:0/636208956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddc806fff0 0x7fddc8070480 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.092+0000 7fddd064b640 1 --2- 172.21.15.76:0/636208956 >> v2:172.21.15.113:3300/0 conn(0x7fddc806bb60 0x7fddc80b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.092+0000 7fddd064b640 1 --2- 172.21.15.76:0/636208956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fddc8159e40 0x7fddc80b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.092+0000 7fddd064b640 1 -- 172.21.15.76:0/636208956 >> 172.21.15.76:0/636208956 conn(0x7fddc80cb530 msgr2=0x7fddc80cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:38.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.092+0000 7fddd064b640 1 -- 172.21.15.76:0/636208956 shutdown_connections 2023-11-09T03:51:38.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.092+0000 7fddd064b640 1 -- 172.21.15.76:0/636208956 wait complete. 2023-11-09T03:51:38.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.093+0000 7fddd064b640 1 Processor -- start 2023-11-09T03:51:38.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.093+0000 7fddd064b640 1 -- start start 2023-11-09T03:51:38.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.094+0000 7fddd064b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddc806bb60 0x7fddc80c5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:38.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.094+0000 7fddd064b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fddc806fff0 0x7fddc80c7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:38.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.094+0000 7fddd064b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddc8159e40 0x7fddc80c8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:38.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.094+0000 7fddd064b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc80c76a0 con 0x7fddc8159e40 2023-11-09T03:51:38.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.094+0000 7fddd064b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc80c7980 con 0x7fddc806bb60 2023-11-09T03:51:38.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.094+0000 7fddd064b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc80c7c60 con 0x7fddc806fff0 2023-11-09T03:51:38.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.094+0000 7fddcebe8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddc8159e40 0x7fddc80c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:38.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.094+0000 7fddcebe8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddc8159e40 0x7fddc80c8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59380/0 (socket says 172.21.15.76:59380) 2023-11-09T03:51:38.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.094+0000 7fddcebe8640 1 -- 172.21.15.76:0/1391813763 learned_addr learned my addr 172.21.15.76:0/1391813763 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:38.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.094+0000 7fddcebe8640 1 -- 172.21.15.76:0/1391813763 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fddc806fff0 msgr2=0x7fddc80c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:38.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.094+0000 7fddcebe8640 1 --2- 172.21.15.76:0/1391813763 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fddc806fff0 0x7fddc80c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.094+0000 7fddce3e7640 1 --2- 172.21.15.76:0/1391813763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddc806bb60 0x7fddc80c5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:38.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.094+0000 7fddcebe8640 1 -- 172.21.15.76:0/1391813763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddc806bb60 msgr2=0x7fddc80c5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:38.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.094+0000 7fddcebe8640 1 --2- 172.21.15.76:0/1391813763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddc806bb60 0x7fddc80c5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.095+0000 7fddcebe8640 1 -- 172.21.15.76:0/1391813763 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc0009b30 con 0x7fddc8159e40 2023-11-09T03:51:38.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.095+0000 7fddce3e7640 1 --2- 172.21.15.76:0/1391813763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddc806bb60 0x7fddc80c5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:51:38.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.095+0000 7fddcebe8640 1 --2- 172.21.15.76:0/1391813763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddc8159e40 0x7fddc80c8720 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fddc0036040 tx=0x7fddc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:38.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.095+0000 7fddbeffd640 1 -- 172.21.15.76:0/1391813763 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc003e450 con 0x7fddc8159e40 2023-11-09T03:51:38.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.095+0000 7fddbeffd640 1 -- 172.21.15.76:0/1391813763 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc003e5d0 con 0x7fddc8159e40 2023-11-09T03:51:38.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.095+0000 7fddd064b640 1 -- 172.21.15.76:0/1391813763 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc80c7f40 con 0x7fddc8159e40 2023-11-09T03:51:38.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.095+0000 7fddbeffd640 1 -- 172.21.15.76:0/1391813763 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc003e980 con 0x7fddc8159e40 2023-11-09T03:51:38.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.095+0000 7fddd064b640 1 -- 172.21.15.76:0/1391813763 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc8151240 con 0x7fddc8159e40 2023-11-09T03:51:38.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.096+0000 7fddd064b640 1 -- 172.21.15.76:0/1391813763 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc80684b0 con 0x7fddc8159e40 2023-11-09T03:51:38.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.096+0000 7fddbeffd640 1 -- 172.21.15.76:0/1391813763 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fddc003eae0 con 0x7fddc8159e40 2023-11-09T03:51:38.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.097+0000 7fddbeffd640 1 --2- 172.21.15.76:0/1391813763 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fddac06ae70 0x7fddac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:38.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.097+0000 7fddbeffd640 1 -- 172.21.15.76:0/1391813763 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7608+0+0 (secure 0 0 0) 0x7fddc00acca0 con 0x7fddc8159e40 2023-11-09T03:51:38.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.097+0000 7fddce3e7640 1 --2- 172.21.15.76:0/1391813763 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fddac06ae70 0x7fddac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:38.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.098+0000 7fddce3e7640 1 --2- 172.21.15.76:0/1391813763 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fddac06ae70 0x7fddac06d330 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fddc4006fd0 tx=0x7fddc4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:38.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.101+0000 7fddbeffd640 1 -- 172.21.15.76:0/1391813763 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fddc00775e0 con 0x7fddc8159e40 2023-11-09T03:51:38.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.302+0000 7fddd064b640 1 -- 172.21.15.76:0/1391813763 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fddc80c5940 con 0x7fddc8159e40 2023-11-09T03:51:38.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.303+0000 7fddbeffd640 1 -- 172.21.15.76:0/1391813763 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4933 (secure 0 0 0) 0x7fddc007ac00 con 0x7fddc8159e40 2023-11-09T03:51:38.304 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:38.304 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":87,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8274,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/1536527256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1536527256},{"type":"v1","addr":"172.21.15.76:6833","nonce":1536527256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:51:32.641425+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6904,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6904":{"gid":6904,"name":"b","rank":0,"incarnation":86,"state":"up:resolve","state_seq":10,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:51:38.304 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 87 2023-11-09T03:51:38.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.305+0000 7fddd064b640 1 -- 172.21.15.76:0/1391813763 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fddac06ae70 msgr2=0x7fddac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:38.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.305+0000 7fddd064b640 1 --2- 172.21.15.76:0/1391813763 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fddac06ae70 0x7fddac06d330 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fddc4006fd0 tx=0x7fddc4008040 comp rx=0 tx=0).stop 2023-11-09T03:51:38.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.305+0000 7fddd064b640 1 -- 172.21.15.76:0/1391813763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddc8159e40 msgr2=0x7fddc80c8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:38.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.305+0000 7fddd064b640 1 --2- 172.21.15.76:0/1391813763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddc8159e40 0x7fddc80c8720 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fddc0036040 tx=0x7fddc0004a40 comp rx=0 tx=0).stop 2023-11-09T03:51:38.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.305+0000 7fddd064b640 1 -- 172.21.15.76:0/1391813763 shutdown_connections 2023-11-09T03:51:38.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.305+0000 7fddd064b640 1 --2- 172.21.15.76:0/1391813763 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fddac06ae70 0x7fddac06d330 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.305+0000 7fddd064b640 1 --2- 172.21.15.76:0/1391813763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddc8159e40 0x7fddc80c8720 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.305+0000 7fddd064b640 1 --2- 172.21.15.76:0/1391813763 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fddc806fff0 0x7fddc80c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.305+0000 7fddd064b640 1 --2- 172.21.15.76:0/1391813763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddc806bb60 0x7fddc80c5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.305+0000 7fddd064b640 1 -- 172.21.15.76:0/1391813763 >> 172.21.15.76:0/1391813763 conn(0x7fddc80cb530 msgr2=0x7fddc8153c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:38.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.305+0000 7fddd064b640 1 -- 172.21.15.76:0/1391813763 shutdown_connections 2023-11-09T03:51:38.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.305+0000 7fddd064b640 1 -- 172.21.15.76:0/1391813763 wait complete. 2023-11-09T03:51:38.317 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-09T03:51:38.317 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:51:38.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.416+0000 7f2a4e133640 1 Processor -- start 2023-11-09T03:51:38.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.416+0000 7f2a4e133640 1 -- start start 2023-11-09T03:51:38.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.417+0000 7f2a4e133640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a48159e40 0x7f2a480b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:38.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.417+0000 7f2a4e133640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2a4806bb60 0x7f2a480b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:38.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.417+0000 7f2a4e133640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2a4806fff0 0x7f2a48070480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:38.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.417+0000 7f2a4e133640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a48071500 con 0x7f2a4806fff0 2023-11-09T03:51:38.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.417+0000 7f2a4e133640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a4815dae0 con 0x7f2a4806bb60 2023-11-09T03:51:38.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.417+0000 7f2a4e133640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a480709c0 con 0x7f2a48159e40 2023-11-09T03:51:38.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.417+0000 7f2a47fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2a4806fff0 0x7f2a48070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:38.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.417+0000 7f2a47fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2a4806fff0 0x7f2a48070480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59390/0 (socket says 172.21.15.76:59390) 2023-11-09T03:51:38.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.417+0000 7f2a47fff640 1 -- 172.21.15.76:0/1181669839 learned_addr learned my addr 172.21.15.76:0/1181669839 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:38.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.417+0000 7f2a46ffd640 1 --2- 172.21.15.76:0/1181669839 >> v2:172.21.15.113:3300/0 conn(0x7f2a4806bb60 0x7f2a480b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:38.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.417+0000 7f2a477fe640 1 --2- 172.21.15.76:0/1181669839 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a48159e40 0x7f2a480b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:38.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.417+0000 7f2a47fff640 1 -- 172.21.15.76:0/1181669839 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a48159e40 msgr2=0x7f2a480b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:38.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.417+0000 7f2a47fff640 1 --2- 172.21.15.76:0/1181669839 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a48159e40 0x7f2a480b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.417+0000 7f2a47fff640 1 -- 172.21.15.76:0/1181669839 >> v2:172.21.15.113:3300/0 conn(0x7f2a4806bb60 msgr2=0x7f2a480b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:38.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.417+0000 7f2a47fff640 1 --2- 172.21.15.76:0/1181669839 >> v2:172.21.15.113:3300/0 conn(0x7f2a4806bb60 0x7f2a480b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.417+0000 7f2a47fff640 1 -- 172.21.15.76:0/1181669839 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a48070c60 con 0x7f2a4806fff0 2023-11-09T03:51:38.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.418+0000 7f2a47fff640 1 --2- 172.21.15.76:0/1181669839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a4806fff0 0x7f2a48070480 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f2a480ce820 tx=0x7f2a4002e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d18871b3658600 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:38.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.418+0000 7f2a467fc640 1 -- 172.21.15.76:0/1181669839 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4002ebf0 con 0x7f2a4806fff0 2023-11-09T03:51:38.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.418+0000 7f2a467fc640 1 -- 172.21.15.76:0/1181669839 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4002ed50 con 0x7f2a4806fff0 2023-11-09T03:51:38.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.418+0000 7f2a467fc640 1 -- 172.21.15.76:0/1181669839 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a40036610 con 0x7f2a4806fff0 2023-11-09T03:51:38.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.419+0000 7f2a4e133640 1 -- 172.21.15.76:0/1181669839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a4806fff0 msgr2=0x7f2a48070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:38.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.419+0000 7f2a4e133640 1 --2- 172.21.15.76:0/1181669839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a4806fff0 0x7f2a48070480 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f2a480ce820 tx=0x7f2a4002e050 comp rx=0 tx=0).stop 2023-11-09T03:51:38.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.419+0000 7f2a4e133640 1 -- 172.21.15.76:0/1181669839 shutdown_connections 2023-11-09T03:51:38.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.419+0000 7f2a4e133640 1 --2- 172.21.15.76:0/1181669839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a4806fff0 0x7f2a48070480 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.419+0000 7f2a4e133640 1 --2- 172.21.15.76:0/1181669839 >> v2:172.21.15.113:3300/0 conn(0x7f2a4806bb60 0x7f2a480b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.419+0000 7f2a4e133640 1 --2- 172.21.15.76:0/1181669839 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a48159e40 0x7f2a480b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.419+0000 7f2a4e133640 1 -- 172.21.15.76:0/1181669839 >> 172.21.15.76:0/1181669839 conn(0x7f2a480cb530 msgr2=0x7f2a480cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:38.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.419+0000 7f2a4e133640 1 -- 172.21.15.76:0/1181669839 shutdown_connections 2023-11-09T03:51:38.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.419+0000 7f2a4e133640 1 -- 172.21.15.76:0/1181669839 wait complete. 2023-11-09T03:51:38.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.420+0000 7f2a4e133640 1 Processor -- start 2023-11-09T03:51:38.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.420+0000 7f2a4e133640 1 -- start start 2023-11-09T03:51:38.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.421+0000 7f2a4e133640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a4806bb60 0x7f2a481f3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:38.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.421+0000 7f2a4e133640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a4806fff0 0x7f2a4814cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:38.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.421+0000 7f2a4e133640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a48159e40 0x7f2a4814e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:38.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.421+0000 7f2a4e133640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a48154230 con 0x7f2a48159e40 2023-11-09T03:51:38.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.421+0000 7f2a4e133640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a48153c70 con 0x7f2a4806bb60 2023-11-09T03:51:38.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.421+0000 7f2a4e133640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a48153f50 con 0x7f2a4806fff0 2023-11-09T03:51:38.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.421+0000 7f2a47fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a48159e40 0x7f2a4814e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:38.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.421+0000 7f2a47fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a48159e40 0x7f2a4814e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59394/0 (socket says 172.21.15.76:59394) 2023-11-09T03:51:38.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.421+0000 7f2a47fff640 1 -- 172.21.15.76:0/2344339630 learned_addr learned my addr 172.21.15.76:0/2344339630 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:38.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.421+0000 7f2a47fff640 1 -- 172.21.15.76:0/2344339630 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a4806fff0 msgr2=0x7f2a4814cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:38.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.421+0000 7f2a47fff640 1 --2- 172.21.15.76:0/2344339630 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a4806fff0 0x7f2a4814cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.421+0000 7f2a47fff640 1 -- 172.21.15.76:0/2344339630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a4806bb60 msgr2=0x7f2a481f3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:38.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.421+0000 7f2a47fff640 1 --2- 172.21.15.76:0/2344339630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a4806bb60 0x7f2a481f3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.421+0000 7f2a47fff640 1 -- 172.21.15.76:0/2344339630 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a40009b30 con 0x7f2a48159e40 2023-11-09T03:51:38.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.421+0000 7f2a47fff640 1 --2- 172.21.15.76:0/2344339630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a48159e40 0x7f2a4814e420 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f2a40038040 tx=0x7f2a40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:38.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.421+0000 7f2a3ffff640 1 -- 172.21.15.76:0/2344339630 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4002ebf0 con 0x7f2a48159e40 2023-11-09T03:51:38.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.421+0000 7f2a3ffff640 1 -- 172.21.15.76:0/2344339630 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4003f470 con 0x7f2a48159e40 2023-11-09T03:51:38.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.421+0000 7f2a4e133640 1 -- 172.21.15.76:0/2344339630 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a481f32d0 con 0x7f2a48159e40 2023-11-09T03:51:38.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.421+0000 7f2a3ffff640 1 -- 172.21.15.76:0/2344339630 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4003f820 con 0x7f2a48159e40 2023-11-09T03:51:38.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.421+0000 7f2a4e133640 1 -- 172.21.15.76:0/2344339630 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a481f4010 con 0x7f2a48159e40 2023-11-09T03:51:38.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.422+0000 7f2a4e133640 1 -- 172.21.15.76:0/2344339630 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a481f37f0 con 0x7f2a48159e40 2023-11-09T03:51:38.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.423+0000 7f2a3ffff640 1 -- 172.21.15.76:0/2344339630 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2a4003f980 con 0x7f2a48159e40 2023-11-09T03:51:38.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.423+0000 7f2a3ffff640 1 --2- 172.21.15.76:0/2344339630 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2a3006ae70 0x7f2a3006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:38.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.423+0000 7f2a3ffff640 1 -- 172.21.15.76:0/2344339630 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7608+0+0 (secure 0 0 0) 0x7f2a400ac5c0 con 0x7f2a48159e40 2023-11-09T03:51:38.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.424+0000 7f2a477fe640 1 --2- 172.21.15.76:0/2344339630 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2a3006ae70 0x7f2a3006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:38.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.424+0000 7f2a477fe640 1 --2- 172.21.15.76:0/2344339630 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2a3006ae70 0x7f2a3006d330 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2a380098f0 tx=0x7f2a38008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:38.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.428+0000 7f2a3ffff640 1 -- 172.21.15.76:0/2344339630 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2a481f37f0 con 0x7f2a48159e40 2023-11-09T03:51:38.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.629+0000 7f2a4e133640 1 -- 172.21.15.76:0/2344339630 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a481f37f0 con 0x7f2a48159e40 2023-11-09T03:51:38.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.629+0000 7f2a3ffff640 1 -- 172.21.15.76:0/2344339630 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4933 (secure 0 0 0) 0x7f2a481f37f0 con 0x7f2a48159e40 2023-11-09T03:51:38.630 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:38.631 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":87,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8274,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/1536527256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1536527256},{"type":"v1","addr":"172.21.15.76:6833","nonce":1536527256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:51:32.641425+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6904,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6904":{"gid":6904,"name":"b","rank":0,"incarnation":86,"state":"up:resolve","state_seq":10,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:51:38.631 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 87 2023-11-09T03:51:38.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.631+0000 7f2a4e133640 1 -- 172.21.15.76:0/2344339630 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2a3006ae70 msgr2=0x7f2a3006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:38.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.631+0000 7f2a4e133640 1 --2- 172.21.15.76:0/2344339630 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2a3006ae70 0x7f2a3006d330 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2a380098f0 tx=0x7f2a38008040 comp rx=0 tx=0).stop 2023-11-09T03:51:38.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.631+0000 7f2a4e133640 1 -- 172.21.15.76:0/2344339630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a48159e40 msgr2=0x7f2a4814e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:38.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.631+0000 7f2a4e133640 1 --2- 172.21.15.76:0/2344339630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a48159e40 0x7f2a4814e420 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f2a40038040 tx=0x7f2a40004a40 comp rx=0 tx=0).stop 2023-11-09T03:51:38.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.632+0000 7f2a4e133640 1 -- 172.21.15.76:0/2344339630 shutdown_connections 2023-11-09T03:51:38.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.632+0000 7f2a4e133640 1 --2- 172.21.15.76:0/2344339630 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2a3006ae70 0x7f2a3006d330 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.632+0000 7f2a4e133640 1 --2- 172.21.15.76:0/2344339630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a48159e40 0x7f2a4814e420 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.632+0000 7f2a4e133640 1 --2- 172.21.15.76:0/2344339630 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a4806fff0 0x7f2a4814cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.632+0000 7f2a4e133640 1 --2- 172.21.15.76:0/2344339630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a4806bb60 0x7f2a481f3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.632+0000 7f2a4e133640 1 -- 172.21.15.76:0/2344339630 >> 172.21.15.76:0/2344339630 conn(0x7f2a480cb530 msgr2=0x7f2a480cea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:38.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.632+0000 7f2a4e133640 1 -- 172.21.15.76:0/2344339630 shutdown_connections 2023-11-09T03:51:38.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.632+0000 7f2a4e133640 1 -- 172.21.15.76:0/2344339630 wait complete. 2023-11-09T03:51:38.643 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:resolve 2023-11-09T03:51:38.644 DEBUG:tasks.cephfs.filesystem:reached state 'up:resolve' in 0.32689595222473145s 2023-11-09T03:51:38.644 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:51:38.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.744+0000 7f85f0555640 1 Processor -- start 2023-11-09T03:51:38.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.744+0000 7f85f0555640 1 -- start start 2023-11-09T03:51:38.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.745+0000 7f85f0555640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85e80b0c40 0x7f85e806fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:38.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.745+0000 7f85f0555640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f85e806bb60 0x7f85e8070530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:38.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.745+0000 7f85f0555640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85e8071500 con 0x7f85e8159e40 2023-11-09T03:51:38.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.745+0000 7f85f0555640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e8070a70 con 0x7f85e806bb60 2023-11-09T03:51:38.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.745+0000 7f85f0555640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e815dae0 con 0x7f85e80b0c40 2023-11-09T03:51:38.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.745+0000 7f85ee2f1640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f85e8159e40 0x7f85e80b0530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:55926/0 (socket says 172.21.15.76:55926) 2023-11-09T03:51:38.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.745+0000 7f85ee2f1640 1 -- 172.21.15.76:0/4276790690 learned_addr learned my addr 172.21.15.76:0/4276790690 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:51:38.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.745+0000 7f85eeaf2640 1 --2- 172.21.15.76:0/4276790690 >> v2:172.21.15.113:3300/0 conn(0x7f85e806bb60 0x7f85e8070530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:38.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.745+0000 7f85edaf0640 1 --2- 172.21.15.76:0/4276790690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85e80b0c40 0x7f85e806fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:38.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.745+0000 7f85ed2ef640 1 -- 172.21.15.76:0/4276790690 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 765768927 0 0) 0x7f85e8071500 con 0x7f85e8159e40 2023-11-09T03:51:38.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.745+0000 7f85ed2ef640 1 -- 172.21.15.76:0/4276790690 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85d4003520 con 0x7f85e8159e40 2023-11-09T03:51:38.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.746+0000 7f85eeaf2640 1 -- 172.21.15.76:0/4276790690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85e80b0c40 msgr2=0x7f85e806fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:38.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.746+0000 7f85eeaf2640 1 --2- 172.21.15.76:0/4276790690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85e80b0c40 0x7f85e806fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.746+0000 7f85eeaf2640 1 -- 172.21.15.76:0/4276790690 >> v1:172.21.15.76:6789/0 conn(0x7f85e8159e40 legacy=0x7f85e80b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:38.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.746+0000 7f85ed2ef640 1 -- 172.21.15.76:0/4276790690 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3360738457 0 0) 0x7f85d4003520 con 0x7f85e8159e40 2023-11-09T03:51:38.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.746+0000 7f85eeaf2640 1 -- 172.21.15.76:0/4276790690 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e8070c60 con 0x7f85e806bb60 2023-11-09T03:51:38.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.746+0000 7f85edaf0640 1 --2- 172.21.15.76:0/4276790690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85e80b0c40 0x7f85e806fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:51:38.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.746+0000 7f85eeaf2640 1 --2- 172.21.15.76:0/4276790690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85e806bb60 0x7f85e8070530 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009d90 tx=0x7f85e002bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=648975a4e6f0de6b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:38.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.747+0000 7f85ed2ef640 1 -- 172.21.15.76:0/4276790690 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e4001670 con 0x7f85e806bb60 2023-11-09T03:51:38.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.747+0000 7f85ed2ef640 1 -- 172.21.15.76:0/4276790690 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e002c810 con 0x7f85e806bb60 2023-11-09T03:51:38.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.747+0000 7f85ed2ef640 1 -- 172.21.15.76:0/4276790690 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e002cb80 con 0x7f85e806bb60 2023-11-09T03:51:38.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.747+0000 7f85f0555640 1 -- 172.21.15.76:0/4276790690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85e806bb60 msgr2=0x7f85e8070530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:38.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.747+0000 7f85f0555640 1 --2- 172.21.15.76:0/4276790690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85e806bb60 0x7f85e8070530 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009d90 tx=0x7f85e002bce0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.747+0000 7f85f0555640 1 -- 172.21.15.76:0/4276790690 shutdown_connections 2023-11-09T03:51:38.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.747+0000 7f85f0555640 1 --2- 172.21.15.76:0/4276790690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85e806bb60 0x7f85e8070530 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.747+0000 7f85f0555640 1 --2- 172.21.15.76:0/4276790690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85e80b0c40 0x7f85e806fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.747+0000 7f85f0555640 1 -- 172.21.15.76:0/4276790690 >> 172.21.15.76:0/4276790690 conn(0x7f85e80cb530 msgr2=0x7f85e80cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:38.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.747+0000 7f85f0555640 1 -- 172.21.15.76:0/4276790690 shutdown_connections 2023-11-09T03:51:38.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.747+0000 7f85f0555640 1 -- 172.21.15.76:0/4276790690 wait complete. 2023-11-09T03:51:38.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.748+0000 7f85f0555640 1 Processor -- start 2023-11-09T03:51:38.754 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.748+0000 7f85f0555640 1 -- start start 2023-11-09T03:51:38.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.749+0000 7f85f0555640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85e806bb60 0x7f85e80c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:38.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.749+0000 7f85f0555640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85e80b0c40 0x7f85e80c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:38.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.749+0000 7f85f0555640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85e8159e40 0x7f85e8150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:38.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.749+0000 7f85f0555640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e815adf0 con 0x7f85e806bb60 2023-11-09T03:51:38.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.749+0000 7f85f0555640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e815a220 con 0x7f85e8159e40 2023-11-09T03:51:38.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.749+0000 7f85f0555640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e815a4c0 con 0x7f85e80b0c40 2023-11-09T03:51:38.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.749+0000 7f85ee2f1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85e806bb60 0x7f85e80c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:38.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.749+0000 7f85ee2f1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85e806bb60 0x7f85e80c82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59406/0 (socket says 172.21.15.76:59406) 2023-11-09T03:51:38.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.749+0000 7f85ee2f1640 1 -- 172.21.15.76:0/2957319530 learned_addr learned my addr 172.21.15.76:0/2957319530 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:38.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.749+0000 7f85ee2f1640 1 -- 172.21.15.76:0/2957319530 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85e80b0c40 msgr2=0x7f85e80c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:38.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.749+0000 7f85ee2f1640 1 --2- 172.21.15.76:0/2957319530 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85e80b0c40 0x7f85e80c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.749+0000 7f85ee2f1640 1 -- 172.21.15.76:0/2957319530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85e8159e40 msgr2=0x7f85e8150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:38.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.749+0000 7f85ee2f1640 1 --2- 172.21.15.76:0/2957319530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85e8159e40 0x7f85e8150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.749+0000 7f85ee2f1640 1 -- 172.21.15.76:0/2957319530 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d8009400 con 0x7f85e806bb60 2023-11-09T03:51:38.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.750+0000 7f85ee2f1640 1 --2- 172.21.15.76:0/2957319530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85e806bb60 0x7f85e80c82a0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f85e400c750 tx=0x7f85e400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:38.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.750+0000 7f85deffd640 1 -- 172.21.15.76:0/2957319530 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e4005ad0 con 0x7f85e806bb60 2023-11-09T03:51:38.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.750+0000 7f85deffd640 1 -- 172.21.15.76:0/2957319530 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e4005c30 con 0x7f85e806bb60 2023-11-09T03:51:38.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.750+0000 7f85f0555640 1 -- 172.21.15.76:0/2957319530 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e0009a70 con 0x7f85e806bb60 2023-11-09T03:51:38.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.750+0000 7f85deffd640 1 -- 172.21.15.76:0/2957319530 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e4007070 con 0x7f85e806bb60 2023-11-09T03:51:38.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.750+0000 7f85f0555640 1 -- 172.21.15.76:0/2957319530 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e815aa80 con 0x7f85e806bb60 2023-11-09T03:51:38.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.751+0000 7f85f0555640 1 -- 172.21.15.76:0/2957319530 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e80684b0 con 0x7f85e806bb60 2023-11-09T03:51:38.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.751+0000 7f85deffd640 1 -- 172.21.15.76:0/2957319530 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f85e40071d0 con 0x7f85e806bb60 2023-11-09T03:51:38.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.751+0000 7f85deffd640 1 --2- 172.21.15.76:0/2957319530 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f85cc06ae70 0x7f85cc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:38.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.752+0000 7f85deffd640 1 -- 172.21.15.76:0/2957319530 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7608+0+0 (secure 0 0 0) 0x7f85e4089540 con 0x7f85e806bb60 2023-11-09T03:51:38.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.752+0000 7f85edaf0640 1 --2- 172.21.15.76:0/2957319530 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f85cc06ae70 0x7f85cc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:38.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.752+0000 7f85edaf0640 1 --2- 172.21.15.76:0/2957319530 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f85cc06ae70 0x7f85cc06d330 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f85d8006e80 tx=0x7f85d8006dd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:38.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.756+0000 7f85deffd640 1 -- 172.21.15.76:0/2957319530 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f85e4053e00 con 0x7f85e806bb60 2023-11-09T03:51:38.850 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-11-09T03:51:38.851 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2023-11-09T03:51:38.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.957+0000 7f85f0555640 1 -- 172.21.15.76:0/2957319530 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85e80c9750 con 0x7f85e806bb60 2023-11-09T03:51:38.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.958+0000 7f85deffd640 1 -- 172.21.15.76:0/2957319530 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4933 (secure 0 0 0) 0x7f85e4057420 con 0x7f85e806bb60 2023-11-09T03:51:38.958 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:38.959 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":87,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8274,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/1536527256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1536527256},{"type":"v1","addr":"172.21.15.76:6833","nonce":1536527256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:51:32.641425+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6904,"mds_1":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6904":{"gid":6904,"name":"b","rank":0,"incarnation":86,"state":"up:resolve","state_seq":10,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7449":{"gid":7449,"name":"a","rank":1,"incarnation":47,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6833/3697850101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3697850101},{"type":"v1","addr":"172.21.15.76:6833","nonce":3697850101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:51:38.959 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 87 2023-11-09T03:51:38.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.960+0000 7f85f0555640 1 -- 172.21.15.76:0/2957319530 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f85cc06ae70 msgr2=0x7f85cc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:38.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.960+0000 7f85f0555640 1 --2- 172.21.15.76:0/2957319530 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f85cc06ae70 0x7f85cc06d330 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f85d8006e80 tx=0x7f85d8006dd0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.960+0000 7f85f0555640 1 -- 172.21.15.76:0/2957319530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85e806bb60 msgr2=0x7f85e80c82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:38.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.960+0000 7f85f0555640 1 --2- 172.21.15.76:0/2957319530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85e806bb60 0x7f85e80c82a0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f85e400c750 tx=0x7f85e400cc10 comp rx=0 tx=0).stop 2023-11-09T03:51:38.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.960+0000 7f85f0555640 1 -- 172.21.15.76:0/2957319530 shutdown_connections 2023-11-09T03:51:38.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.960+0000 7f85f0555640 1 --2- 172.21.15.76:0/2957319530 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f85cc06ae70 0x7f85cc06d330 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.960+0000 7f85f0555640 1 --2- 172.21.15.76:0/2957319530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85e8159e40 0x7f85e8150bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.960+0000 7f85f0555640 1 --2- 172.21.15.76:0/2957319530 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85e80b0c40 0x7f85e80c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.960+0000 7f85f0555640 1 --2- 172.21.15.76:0/2957319530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85e806bb60 0x7f85e80c82a0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:38.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.960+0000 7f85f0555640 1 -- 172.21.15.76:0/2957319530 >> 172.21.15.76:0/2957319530 conn(0x7f85e80cb530 msgr2=0x7f85e80ae5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:38.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.960+0000 7f85f0555640 1 -- 172.21.15.76:0/2957319530 shutdown_connections 2023-11-09T03:51:38.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:38.960+0000 7f85f0555640 1 -- 172.21.15.76:0/2957319530 wait complete. 2023-11-09T03:51:38.971 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump snaps --server 2023-11-09T03:51:39.097 INFO:teuthology.orchestra.run.smithi113.stdout:{"last_snap":10,"last_created":5,"last_destroyed":9,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[{"version":20,"snapinfo":{"snapid":10,"ino":1099511627777,"stamp":"2023-11-09T03:51:09.824977+0000","name":"s22","metadata":{}}}],"pending_destroy":[]} 2023-11-09T03:51:39.098 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 5, "last_destroyed": 9, "last_snap": 10, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [ { "snapInfinityo": { "ino": 1099511627777, "metadata": {}, "name": "s22", "snapid": 10, "stamp": "2023-11-09T03:51:09.824977+0000" }, "version": 20 } ], "snaps": [] } 2023-11-09T03:51:39.098 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:1 2023-11-09T03:51:39.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.198+0000 7f1a60dc5640 1 Processor -- start 2023-11-09T03:51:39.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.198+0000 7f1a60dc5640 1 -- start start 2023-11-09T03:51:39.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.199+0000 7f1a60dc5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1a5c15d4e0 0x7f1a5c0d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:39.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.199+0000 7f1a60dc5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a5c0d3b60 0x7f1a5c0d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:39.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.199+0000 7f1a60dc5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a5c0b32b0 con 0x7f1a5c15d100 2023-11-09T03:51:39.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.199+0000 7f1a60dc5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a5c0cec20 con 0x7f1a5c15d4e0 2023-11-09T03:51:39.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.199+0000 7f1a60dc5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c0cee10 con 0x7f1a5c0d3b60 2023-11-09T03:51:39.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.199+0000 7f1a5a59c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f1a5c15d100 0x7f1a5c0ce510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:55940/0 (socket says 172.21.15.76:55940) 2023-11-09T03:51:39.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.199+0000 7f1a5a59c640 1 -- 172.21.15.76:0/3641584391 learned_addr learned my addr 172.21.15.76:0/3641584391 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:51:39.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.200+0000 7f1a5959a640 1 -- 172.21.15.76:0/3641584391 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3248919059 0 0) 0x7f1a5c0b32b0 con 0x7f1a5c15d100 2023-11-09T03:51:39.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.200+0000 7f1a59d9b640 1 --2- 172.21.15.76:0/3641584391 >> v2:172.21.15.113:3300/0 conn(0x7f1a5c15d4e0 0x7f1a5c0d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:39.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.200+0000 7f1a5ad9d640 1 --2- 172.21.15.76:0/3641584391 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a5c0d3b60 0x7f1a5c0d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:39.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.200+0000 7f1a5959a640 1 -- 172.21.15.76:0/3641584391 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a44003520 con 0x7f1a5c15d100 2023-11-09T03:51:39.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.200+0000 7f1a5959a640 1 -- 172.21.15.76:0/3641584391 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1368998173 0 0) 0x7f1a44003520 con 0x7f1a5c15d100 2023-11-09T03:51:39.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.200+0000 7f1a5959a640 1 -- 172.21.15.76:0/3641584391 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a5c0d3b60 msgr2=0x7f1a5c0d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:39.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.200+0000 7f1a5959a640 1 --2- 172.21.15.76:0/3641584391 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a5c0d3b60 0x7f1a5c0d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:39.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.200+0000 7f1a5959a640 1 -- 172.21.15.76:0/3641584391 >> v2:172.21.15.113:3300/0 conn(0x7f1a5c15d4e0 msgr2=0x7f1a5c0d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:39.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.200+0000 7f1a5959a640 1 --2- 172.21.15.76:0/3641584391 >> v2:172.21.15.113:3300/0 conn(0x7f1a5c15d4e0 0x7f1a5c0d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:39.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.200+0000 7f1a5959a640 1 -- 172.21.15.76:0/3641584391 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5c0b4750 con 0x7f1a5c15d100 2023-11-09T03:51:39.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.200+0000 7f1a5959a640 1 -- 172.21.15.76:0/3641584391 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f1a4c001b70 con 0x7f1a5c15d100 2023-11-09T03:51:39.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.200+0000 7f1a5959a640 1 -- 172.21.15.76:0/3641584391 >> v1:172.21.15.76:6789/0 conn(0x7f1a5c15d100 legacy=0x7f1a5c0ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:39.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.200+0000 7f1a5959a640 1 --2- 172.21.15.76:0/3641584391 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a44003970 0x7f1a44003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:39.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.200+0000 7f1a5959a640 1 --2- 172.21.15.76:0/3641584391 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a5c0d3b60 0x7f1a440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:39.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.200+0000 7f1a5959a640 1 --2- 172.21.15.76:0/3641584391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a44004d00 0x7f1a44005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:39.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.200+0000 7f1a5959a640 1 -- 172.21.15.76:0/3641584391 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c0cee10 con 0x7f1a44004d00 2023-11-09T03:51:39.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.200+0000 7f1a5959a640 1 -- 172.21.15.76:0/3641584391 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4400b980 con 0x7f1a44003970 2023-11-09T03:51:39.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.200+0000 7f1a5959a640 1 -- 172.21.15.76:0/3641584391 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4400bac0 con 0x7f1a5c0d3b60 2023-11-09T03:51:39.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.200+0000 7f1a5ad9d640 1 --2- 172.21.15.76:0/3641584391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a44004d00 0x7f1a44005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:39.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.200+0000 7f1a5a59c640 1 --2- 172.21.15.76:0/3641584391 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a44003970 0x7f1a44003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:39.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.200+0000 7f1a59d9b640 1 --2- 172.21.15.76:0/3641584391 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a5c0d3b60 0x7f1a440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:39.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.200+0000 7f1a5ad9d640 1 -- 172.21.15.76:0/3641584391 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a5c0d3b60 msgr2=0x7f1a440047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:39.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.200+0000 7f1a5ad9d640 1 --2- 172.21.15.76:0/3641584391 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a5c0d3b60 0x7f1a440047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:39.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.200+0000 7f1a5ad9d640 1 -- 172.21.15.76:0/3641584391 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a44003970 msgr2=0x7f1a44003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:39.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.200+0000 7f1a5ad9d640 1 --2- 172.21.15.76:0/3641584391 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a44003970 0x7f1a44003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:39.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.200+0000 7f1a5ad9d640 1 -- 172.21.15.76:0/3641584391 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a44003520 con 0x7f1a44004d00 2023-11-09T03:51:39.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.201+0000 7f1a5ad9d640 1 --2- 172.21.15.76:0/3641584391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a44004d00 0x7f1a44005180 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f1a54009a90 tx=0x7f1a440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88ae8bdad46801 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:39.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.201+0000 7f1a5959a640 1 -- 172.21.15.76:0/3641584391 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5402c680 con 0x7f1a44004d00 2023-11-09T03:51:39.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.201+0000 7f1a5959a640 1 -- 172.21.15.76:0/3641584391 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5402c7e0 con 0x7f1a44004d00 2023-11-09T03:51:39.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.201+0000 7f1a5959a640 1 -- 172.21.15.76:0/3641584391 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5402cb70 con 0x7f1a44004d00 2023-11-09T03:51:39.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.202+0000 7f1a60dc5640 1 -- 172.21.15.76:0/3641584391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a44004d00 msgr2=0x7f1a44005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:39.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.202+0000 7f1a60dc5640 1 --2- 172.21.15.76:0/3641584391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a44004d00 0x7f1a44005180 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f1a54009a90 tx=0x7f1a440057f0 comp rx=0 tx=0).stop 2023-11-09T03:51:39.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.202+0000 7f1a5ad9d640 1 -- 172.21.15.76:0/3641584391 reap_dead start 2023-11-09T03:51:39.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.202+0000 7f1a60dc5640 1 -- 172.21.15.76:0/3641584391 shutdown_connections 2023-11-09T03:51:39.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.202+0000 7f1a60dc5640 1 -- 172.21.15.76:0/3641584391 >> 172.21.15.76:0/3641584391 conn(0x7f1a5c0d2ef0 msgr2=0x7f1a5c0d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:39.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.202+0000 7f1a60dc5640 1 -- 172.21.15.76:0/3641584391 shutdown_connections 2023-11-09T03:51:39.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.202+0000 7f1a60dc5640 1 -- 172.21.15.76:0/3641584391 wait complete. 2023-11-09T03:51:39.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.203+0000 7f1a60dc5640 1 Processor -- start 2023-11-09T03:51:39.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.203+0000 7f1a60dc5640 1 -- start start 2023-11-09T03:51:39.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.204+0000 7f1a60dc5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a44004d00 0x7f1a5c0b8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:39.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.204+0000 7f1a60dc5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a5c0b6ac0 0x7f1a5c15b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:39.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.204+0000 7f1a60dc5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a5c15f950 0x7f1a5c15fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:39.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.204+0000 7f1a60dc5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c160310 con 0x7f1a44004d00 2023-11-09T03:51:39.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.204+0000 7f1a60dc5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c0b8870 con 0x7f1a5c15f950 2023-11-09T03:51:39.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.204+0000 7f1a60dc5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c0b7080 con 0x7f1a5c0b6ac0 2023-11-09T03:51:39.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.204+0000 7f1a5a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a44004d00 0x7f1a5c0b8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:39.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.204+0000 7f1a5a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a44004d00 0x7f1a5c0b8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59438/0 (socket says 172.21.15.76:59438) 2023-11-09T03:51:39.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.204+0000 7f1a5a59c640 1 -- 172.21.15.76:0/1038274736 learned_addr learned my addr 172.21.15.76:0/1038274736 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:39.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.204+0000 7f1a59d9b640 1 --2- 172.21.15.76:0/1038274736 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a5c0b6ac0 0x7f1a5c15b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:39.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.204+0000 7f1a5ad9d640 1 --2- 172.21.15.76:0/1038274736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a5c15f950 0x7f1a5c15fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:39.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.204+0000 7f1a5a59c640 1 -- 172.21.15.76:0/1038274736 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a5c0b6ac0 msgr2=0x7f1a5c15b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:39.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.204+0000 7f1a5a59c640 1 --2- 172.21.15.76:0/1038274736 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a5c0b6ac0 0x7f1a5c15b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:39.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.204+0000 7f1a5a59c640 1 -- 172.21.15.76:0/1038274736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a5c15f950 msgr2=0x7f1a5c15fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:39.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.204+0000 7f1a5a59c640 1 --2- 172.21.15.76:0/1038274736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a5c15f950 0x7f1a5c15fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:39.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.204+0000 7f1a5a59c640 1 -- 172.21.15.76:0/1038274736 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5c0b7360 con 0x7f1a44004d00 2023-11-09T03:51:39.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.204+0000 7f1a5ad9d640 1 --2- 172.21.15.76:0/1038274736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a5c15f950 0x7f1a5c15fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:51:39.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.204+0000 7f1a5a59c640 1 --2- 172.21.15.76:0/1038274736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a44004d00 0x7f1a5c0b8330 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c00ffd0 tx=0x7f1a4c00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:39.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.204+0000 7f1a4affd640 1 -- 172.21.15.76:0/1038274736 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c00dde0 con 0x7f1a44004d00 2023-11-09T03:51:39.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.205+0000 7f1a4affd640 1 -- 172.21.15.76:0/1038274736 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4c005b20 con 0x7f1a44004d00 2023-11-09T03:51:39.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.205+0000 7f1a60dc5640 1 -- 172.21.15.76:0/1038274736 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a5c0b7640 con 0x7f1a44004d00 2023-11-09T03:51:39.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.205+0000 7f1a4affd640 1 -- 172.21.15.76:0/1038274736 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c0070d0 con 0x7f1a44004d00 2023-11-09T03:51:39.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.205+0000 7f1a60dc5640 1 -- 172.21.15.76:0/1038274736 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a5c0ca060 con 0x7f1a44004d00 2023-11-09T03:51:39.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.205+0000 7f1a60dc5640 1 -- 172.21.15.76:0/1038274736 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a5c0ca280 con 0x7f1a44004d00 2023-11-09T03:51:39.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.206+0000 7f1a4affd640 1 -- 172.21.15.76:0/1038274736 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1a4c007230 con 0x7f1a44004d00 2023-11-09T03:51:39.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.206+0000 7f1a4affd640 1 --2- 172.21.15.76:0/1038274736 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1a3c06ae70 0x7f1a3c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:39.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.206+0000 7f1a4affd640 1 -- 172.21.15.76:0/1038274736 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7608+0+0 (secure 0 0 0) 0x7f1a4c08a3e0 con 0x7f1a44004d00 2023-11-09T03:51:39.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.206+0000 7f1a59d9b640 1 --2- 172.21.15.76:0/1038274736 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1a3c06ae70 0x7f1a3c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:39.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.207+0000 7f1a59d9b640 1 --2- 172.21.15.76:0/1038274736 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1a3c06ae70 0x7f1a3c06d330 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c160560 tx=0x7f1a50008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:39.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.211+0000 7f1a4affd640 1 -- 172.21.15.76:0/1038274736 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1a4c054d20 con 0x7f1a44004d00 2023-11-09T03:51:39.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.441+0000 7f1a60dc5640 1 -- 172.21.15.76:0/1038274736 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:1"} v 0) v1 -- 0x7f1a5c0684b0 con 0x7f1a44004d00 2023-11-09T03:51:39.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.443+0000 7f1a4affd640 1 -- 172.21.15.76:0/1038274736 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:1"}]=0 failed mds gid 7449 v88) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1a4c0012f0 con 0x7f1a44004d00 2023-11-09T03:51:39.443 INFO:teuthology.orchestra.run.smithi076.stderr:failed mds gid 7449 2023-11-09T03:51:39.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.445+0000 7f1a60dc5640 1 -- 172.21.15.76:0/1038274736 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1a3c06ae70 msgr2=0x7f1a3c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:39.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.445+0000 7f1a60dc5640 1 --2- 172.21.15.76:0/1038274736 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1a3c06ae70 0x7f1a3c06d330 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c160560 tx=0x7f1a50008040 comp rx=0 tx=0).stop 2023-11-09T03:51:39.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.445+0000 7f1a60dc5640 1 -- 172.21.15.76:0/1038274736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a44004d00 msgr2=0x7f1a5c0b8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:39.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.445+0000 7f1a60dc5640 1 --2- 172.21.15.76:0/1038274736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a44004d00 0x7f1a5c0b8330 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c00ffd0 tx=0x7f1a4c00d520 comp rx=0 tx=0).stop 2023-11-09T03:51:39.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.445+0000 7f1a60dc5640 1 -- 172.21.15.76:0/1038274736 shutdown_connections 2023-11-09T03:51:39.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.445+0000 7f1a60dc5640 1 --2- 172.21.15.76:0/1038274736 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1a3c06ae70 0x7f1a3c06d330 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:39.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.445+0000 7f1a60dc5640 1 --2- 172.21.15.76:0/1038274736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a5c15f950 0x7f1a5c15fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:39.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.445+0000 7f1a60dc5640 1 --2- 172.21.15.76:0/1038274736 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a5c0b6ac0 0x7f1a5c15b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:39.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.445+0000 7f1a60dc5640 1 --2- 172.21.15.76:0/1038274736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a44004d00 0x7f1a5c0b8330 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:39.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.445+0000 7f1a60dc5640 1 -- 172.21.15.76:0/1038274736 >> 172.21.15.76:0/1038274736 conn(0x7f1a5c0d2ef0 msgr2=0x7f1a5c15c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:39.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.445+0000 7f1a60dc5640 1 -- 172.21.15.76:0/1038274736 shutdown_connections 2023-11-09T03:51:39.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.445+0000 7f1a60dc5640 1 -- 172.21.15.76:0/1038274736 wait complete. 2023-11-09T03:51:39.459 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-09T03:51:39.460 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-09T03:51:39.460 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-09T03:51:39.460 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:51:39.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:51:39.461 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi076 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2023-11-09T03:51:39.461 INFO:tasks.ceph.mds.a:Stopped 2023-11-09T03:51:39.462 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-09T03:51:39.464 INFO:tasks.ceph.mds.a:Started 2023-11-09T03:51:39.464 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:51:39.540 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2023-11-09T03:51:39.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.565+0000 7fb65c533640 1 Processor -- start 2023-11-09T03:51:39.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.565+0000 7fb65c533640 1 -- start start 2023-11-09T03:51:39.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.566+0000 7fb65c533640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb65415d100 0x7fb6540ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:39.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.566+0000 7fb65c533640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb65415d4e0 0x7fb6540d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:39.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.566+0000 7fb65c533640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6540b32b0 con 0x7fb6540d3b60 2023-11-09T03:51:39.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.566+0000 7fb65c533640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6540cee10 con 0x7fb65415d100 2023-11-09T03:51:39.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.566+0000 7fb65c533640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6540b4750 con 0x7fb65415d4e0 2023-11-09T03:51:39.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.566+0000 7fb65a2cf640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb65415d100 0x7fb6540ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:39.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.566+0000 7fb65a2cf640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb65415d100 0x7fb6540ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59470/0 (socket says 172.21.15.76:59470) 2023-11-09T03:51:39.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.566+0000 7fb65a2cf640 1 -- 172.21.15.76:0/1259078161 learned_addr learned my addr 172.21.15.76:0/1259078161 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:39.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.567+0000 7fb659ace640 1 --2- 172.21.15.76:0/1259078161 >> v2:172.21.15.113:3300/0 conn(0x7fb65415d4e0 0x7fb6540d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:39.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.567+0000 7fb65a2cf640 1 -- 172.21.15.76:0/1259078161 >> v2:172.21.15.113:3300/0 conn(0x7fb65415d4e0 msgr2=0x7fb6540d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:39.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.567+0000 7fb65a2cf640 1 --2- 172.21.15.76:0/1259078161 >> v2:172.21.15.113:3300/0 conn(0x7fb65415d4e0 0x7fb6540d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:39.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.567+0000 7fb65a2cf640 1 -- 172.21.15.76:0/1259078161 >> v1:172.21.15.113:6789/0 conn(0x7fb6540d3b60 legacy=0x7fb6540cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:39.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.567+0000 7fb65a2cf640 1 -- 172.21.15.76:0/1259078161 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6540b2980 con 0x7fb65415d100 2023-11-09T03:51:39.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.567+0000 7fb659ace640 1 --2- 172.21.15.76:0/1259078161 >> v2:172.21.15.113:3300/0 conn(0x7fb65415d4e0 0x7fb6540d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:51:39.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.567+0000 7fb65a2cf640 1 --2- 172.21.15.76:0/1259078161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb65415d100 0x7fb6540ce510 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fb654160da0 tx=0x7fb64402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc191ddf598f62f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:39.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.568+0000 7fb6592cd640 1 -- 172.21.15.76:0/1259078161 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb644004230 con 0x7fb65415d100 2023-11-09T03:51:39.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.568+0000 7fb6592cd640 1 -- 172.21.15.76:0/1259078161 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb644004390 con 0x7fb65415d100 2023-11-09T03:51:39.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.568+0000 7fb6592cd640 1 -- 172.21.15.76:0/1259078161 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb644030840 con 0x7fb65415d100 2023-11-09T03:51:39.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.568+0000 7fb65c533640 1 -- 172.21.15.76:0/1259078161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb65415d100 msgr2=0x7fb6540ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:39.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.568+0000 7fb65c533640 1 --2- 172.21.15.76:0/1259078161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb65415d100 0x7fb6540ce510 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fb654160da0 tx=0x7fb64402b980 comp rx=0 tx=0).stop 2023-11-09T03:51:39.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.568+0000 7fb65c533640 1 -- 172.21.15.76:0/1259078161 shutdown_connections 2023-11-09T03:51:39.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.568+0000 7fb65c533640 1 --2- 172.21.15.76:0/1259078161 >> v2:172.21.15.113:3300/0 conn(0x7fb65415d4e0 0x7fb6540d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:39.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.568+0000 7fb65c533640 1 --2- 172.21.15.76:0/1259078161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb65415d100 0x7fb6540ce510 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:39.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.568+0000 7fb65c533640 1 -- 172.21.15.76:0/1259078161 >> 172.21.15.76:0/1259078161 conn(0x7fb6540d2ef0 msgr2=0x7fb6540d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:39.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.568+0000 7fb65c533640 1 -- 172.21.15.76:0/1259078161 shutdown_connections 2023-11-09T03:51:39.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.568+0000 7fb65c533640 1 -- 172.21.15.76:0/1259078161 wait complete. 2023-11-09T03:51:39.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.569+0000 7fb65c533640 1 Processor -- start 2023-11-09T03:51:39.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.569+0000 7fb65c533640 1 -- start start 2023-11-09T03:51:39.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.570+0000 7fb65c533640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6540d3b60 0x7fb6541f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:39.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.570+0000 7fb65c533640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65415d100 0x7fb65415ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:39.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.570+0000 7fb65c533640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb65415d4e0 0x7fb6540b67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:39.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.570+0000 7fb65c533640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65415f6c0 con 0x7fb6540d3b60 2023-11-09T03:51:39.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.570+0000 7fb65c533640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6540b7550 con 0x7fb65415d100 2023-11-09T03:51:39.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.570+0000 7fb65c533640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6540b7270 con 0x7fb65415d4e0 2023-11-09T03:51:39.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.570+0000 7fb65a2cf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6540d3b60 0x7fb6541f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:39.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.570+0000 7fb65a2cf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6540d3b60 0x7fb6541f2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59480/0 (socket says 172.21.15.76:59480) 2023-11-09T03:51:39.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.570+0000 7fb65a2cf640 1 -- 172.21.15.76:0/3990300005 learned_addr learned my addr 172.21.15.76:0/3990300005 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:39.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.570+0000 7fb659ace640 1 --2- 172.21.15.76:0/3990300005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65415d100 0x7fb65415ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:39.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.570+0000 7fb65a2cf640 1 -- 172.21.15.76:0/3990300005 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb65415d4e0 msgr2=0x7fb6540b67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:39.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.570+0000 7fb65a2cf640 1 --2- 172.21.15.76:0/3990300005 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb65415d4e0 0x7fb6540b67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:39.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.570+0000 7fb65a2cf640 1 -- 172.21.15.76:0/3990300005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65415d100 msgr2=0x7fb65415ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:39.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.570+0000 7fb65a2cf640 1 --2- 172.21.15.76:0/3990300005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65415d100 0x7fb65415ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:39.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.570+0000 7fb65a2cf640 1 -- 172.21.15.76:0/3990300005 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb644009460 con 0x7fb6540d3b60 2023-11-09T03:51:39.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.571+0000 7fb65a2cf640 1 --2- 172.21.15.76:0/3990300005 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6540d3b60 0x7fb6541f2ba0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fb644009940 tx=0x7fb6440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:39.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.571+0000 7fb64affd640 1 -- 172.21.15.76:0/3990300005 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64403d070 con 0x7fb6540d3b60 2023-11-09T03:51:39.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.571+0000 7fb64affd640 1 -- 172.21.15.76:0/3990300005 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb644004500 con 0x7fb6540d3b60 2023-11-09T03:51:39.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.571+0000 7fb65c533640 1 -- 172.21.15.76:0/3990300005 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb65415f070 con 0x7fb6540d3b60 2023-11-09T03:51:39.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.571+0000 7fb64affd640 1 -- 172.21.15.76:0/3990300005 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb644030e40 con 0x7fb6540d3b60 2023-11-09T03:51:39.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.571+0000 7fb65c533640 1 -- 172.21.15.76:0/3990300005 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6541f30e0 con 0x7fb6540d3b60 2023-11-09T03:51:39.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.572+0000 7fb65c533640 1 -- 172.21.15.76:0/3990300005 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb65415eed0 con 0x7fb6540d3b60 2023-11-09T03:51:39.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.572+0000 7fb64affd640 1 -- 172.21.15.76:0/3990300005 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb644003610 con 0x7fb6540d3b60 2023-11-09T03:51:39.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.572+0000 7fb64affd640 1 --2- 172.21.15.76:0/3990300005 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb63806ae70 0x7fb63806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:39.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.573+0000 7fb64affd640 1 -- 172.21.15.76:0/3990300005 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7694+0+0 (secure 0 0 0) 0x7fb6440ac2e0 con 0x7fb6540d3b60 2023-11-09T03:51:39.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.573+0000 7fb659ace640 1 --2- 172.21.15.76:0/3990300005 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb63806ae70 0x7fb63806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:39.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.573+0000 7fb659ace640 1 --2- 172.21.15.76:0/3990300005 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb63806ae70 0x7fb63806d330 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb6540d1ab0 tx=0x7fb650008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:39.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.577+0000 7fb64affd640 1 -- 172.21.15.76:0/3990300005 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb65415eed0 con 0x7fb6540d3b60 2023-11-09T03:51:39.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.779+0000 7fb65c533640 1 -- 172.21.15.76:0/3990300005 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb65415eed0 con 0x7fb6540d3b60 2023-11-09T03:51:39.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.779+0000 7fb64affd640 1 -- 172.21.15.76:0/3990300005 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4142 (secure 0 0 0) 0x7fb65415eed0 con 0x7fb6540d3b60 2023-11-09T03:51:39.780 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:39.780 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":89,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:51:39.447420+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6904,"mds_1":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6904":{"gid":6904,"name":"b","rank":0,"incarnation":86,"state":"up:resolve","state_seq":10,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8274":{"gid":8274,"name":"c","rank":1,"incarnation":89,"state":"up:replay","state_seq":1,"addr":"172.21.15.76:6833/1536527256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1536527256},{"type":"v1","addr":"172.21.15.76:6833","nonce":1536527256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:51:39.781 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 89 2023-11-09T03:51:39.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.781+0000 7fb65c533640 1 -- 172.21.15.76:0/3990300005 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb63806ae70 msgr2=0x7fb63806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:39.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.781+0000 7fb65c533640 1 --2- 172.21.15.76:0/3990300005 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb63806ae70 0x7fb63806d330 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb6540d1ab0 tx=0x7fb650008040 comp rx=0 tx=0).stop 2023-11-09T03:51:39.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.781+0000 7fb65c533640 1 -- 172.21.15.76:0/3990300005 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6540d3b60 msgr2=0x7fb6541f2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:39.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.781+0000 7fb65c533640 1 --2- 172.21.15.76:0/3990300005 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6540d3b60 0x7fb6541f2ba0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fb644009940 tx=0x7fb6440046c0 comp rx=0 tx=0).stop 2023-11-09T03:51:39.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.781+0000 7fb65c533640 1 -- 172.21.15.76:0/3990300005 shutdown_connections 2023-11-09T03:51:39.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.781+0000 7fb65c533640 1 --2- 172.21.15.76:0/3990300005 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb63806ae70 0x7fb63806d330 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:39.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.781+0000 7fb65c533640 1 --2- 172.21.15.76:0/3990300005 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb65415d4e0 0x7fb6540b67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:39.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.781+0000 7fb65c533640 1 --2- 172.21.15.76:0/3990300005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65415d100 0x7fb65415ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:39.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.781+0000 7fb65c533640 1 --2- 172.21.15.76:0/3990300005 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6540d3b60 0x7fb6541f2ba0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:39.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.781+0000 7fb65c533640 1 -- 172.21.15.76:0/3990300005 >> 172.21.15.76:0/3990300005 conn(0x7fb6540d2ef0 msgr2=0x7fb6540cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:39.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.781+0000 7fb65c533640 1 -- 172.21.15.76:0/3990300005 shutdown_connections 2023-11-09T03:51:39.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:39.781+0000 7fb65c533640 1 -- 172.21.15.76:0/3990300005 wait complete. 2023-11-09T03:51:39.792 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-11-09T03:51:44.793 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:51:44.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.903+0000 7fad3eada640 1 Processor -- start 2023-11-09T03:51:44.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.903+0000 7fad3eada640 1 -- start start 2023-11-09T03:51:44.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.904+0000 7fad3eada640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fad3815d100 0x7fad380ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:44.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.904+0000 7fad3eada640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fad380d1be0 0x7fad380b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:44.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.904+0000 7fad3eada640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad380b32b0 con 0x7fad3815d4e0 2023-11-09T03:51:44.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.904+0000 7fad3eada640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad380cee10 con 0x7fad380d1be0 2023-11-09T03:51:44.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.904+0000 7fad3eada640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad380b2980 con 0x7fad3815d100 2023-11-09T03:51:44.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.904+0000 7fad3d077640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fad380d1be0 0x7fad380b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:44.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.904+0000 7fad37fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fad3815d4e0 0x7fad380d14d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46628/0 (socket says 172.21.15.76:46628) 2023-11-09T03:51:44.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.904+0000 7fad37fff640 1 -- 172.21.15.76:0/1938254270 learned_addr learned my addr 172.21.15.76:0/1938254270 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:51:44.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.905+0000 7fad3c876640 1 --2- 172.21.15.76:0/1938254270 >> v2:172.21.15.113:3300/0 conn(0x7fad3815d100 0x7fad380ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:44.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.905+0000 7fad3d077640 1 -- 172.21.15.76:0/1938254270 >> v2:172.21.15.113:3300/0 conn(0x7fad3815d100 msgr2=0x7fad380ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:44.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.905+0000 7fad3d077640 1 --2- 172.21.15.76:0/1938254270 >> v2:172.21.15.113:3300/0 conn(0x7fad3815d100 0x7fad380ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:44.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.905+0000 7fad3d077640 1 -- 172.21.15.76:0/1938254270 >> v1:172.21.15.76:6789/0 conn(0x7fad3815d4e0 legacy=0x7fad380d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:44.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.905+0000 7fad3d077640 1 -- 172.21.15.76:0/1938254270 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad380b2c60 con 0x7fad380d1be0 2023-11-09T03:51:44.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.905+0000 7fad3c876640 1 --2- 172.21.15.76:0/1938254270 >> v2:172.21.15.113:3300/0 conn(0x7fad3815d100 0x7fad380ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:51:44.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.905+0000 7fad3d077640 1 --2- 172.21.15.76:0/1938254270 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad380d1be0 0x7fad380b3740 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fad380d09f0 tx=0x7fad3002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca1350ee4c0b46a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:44.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.906+0000 7fad377fe640 1 -- 172.21.15.76:0/1938254270 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3002cc80 con 0x7fad380d1be0 2023-11-09T03:51:44.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.906+0000 7fad377fe640 1 -- 172.21.15.76:0/1938254270 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad3002cde0 con 0x7fad380d1be0 2023-11-09T03:51:44.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.906+0000 7fad377fe640 1 -- 172.21.15.76:0/1938254270 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad30034660 con 0x7fad380d1be0 2023-11-09T03:51:44.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.906+0000 7fad3eada640 1 -- 172.21.15.76:0/1938254270 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad380d1be0 msgr2=0x7fad380b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:44.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.906+0000 7fad3eada640 1 --2- 172.21.15.76:0/1938254270 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad380d1be0 0x7fad380b3740 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fad380d09f0 tx=0x7fad3002bed0 comp rx=0 tx=0).stop 2023-11-09T03:51:44.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.906+0000 7fad3eada640 1 -- 172.21.15.76:0/1938254270 shutdown_connections 2023-11-09T03:51:44.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.906+0000 7fad3eada640 1 --2- 172.21.15.76:0/1938254270 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad380d1be0 0x7fad380b3740 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:44.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.906+0000 7fad3eada640 1 --2- 172.21.15.76:0/1938254270 >> v2:172.21.15.113:3300/0 conn(0x7fad3815d100 0x7fad380ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:44.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.906+0000 7fad3eada640 1 -- 172.21.15.76:0/1938254270 >> 172.21.15.76:0/1938254270 conn(0x7fad380d2ef0 msgr2=0x7fad380d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:44.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.906+0000 7fad3eada640 1 -- 172.21.15.76:0/1938254270 shutdown_connections 2023-11-09T03:51:44.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.906+0000 7fad3eada640 1 -- 172.21.15.76:0/1938254270 wait complete. 2023-11-09T03:51:44.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.908+0000 7fad3eada640 1 Processor -- start 2023-11-09T03:51:44.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.908+0000 7fad3eada640 1 -- start start 2023-11-09T03:51:44.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.908+0000 7fad3eada640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad380d1be0 0x7fad381f2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:44.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.908+0000 7fad3eada640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad3815d100 0x7fad3815e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:44.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.908+0000 7fad3eada640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad3815d4e0 0x7fad381577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:44.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.908+0000 7fad3eada640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad38160380 con 0x7fad3815d100 2023-11-09T03:51:44.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.908+0000 7fad3eada640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3815fef0 con 0x7fad3815d4e0 2023-11-09T03:51:44.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.909+0000 7fad3eada640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad3815f2e0 con 0x7fad380d1be0 2023-11-09T03:51:44.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.909+0000 7fad37fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad3815d100 0x7fad3815e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:44.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.909+0000 7fad37fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad3815d100 0x7fad3815e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49114/0 (socket says 172.21.15.76:49114) 2023-11-09T03:51:44.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.909+0000 7fad37fff640 1 -- 172.21.15.76:0/1863136170 learned_addr learned my addr 172.21.15.76:0/1863136170 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:44.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.909+0000 7fad3d077640 1 --2- 172.21.15.76:0/1863136170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad3815d4e0 0x7fad381577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:44.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.909+0000 7fad3c876640 1 --2- 172.21.15.76:0/1863136170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad380d1be0 0x7fad381f2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:44.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.909+0000 7fad37fff640 1 -- 172.21.15.76:0/1863136170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad380d1be0 msgr2=0x7fad381f2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:44.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.909+0000 7fad37fff640 1 --2- 172.21.15.76:0/1863136170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad380d1be0 0x7fad381f2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:44.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.909+0000 7fad37fff640 1 -- 172.21.15.76:0/1863136170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad3815d4e0 msgr2=0x7fad381577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:44.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.909+0000 7fad37fff640 1 --2- 172.21.15.76:0/1863136170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad3815d4e0 0x7fad381577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:44.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.909+0000 7fad37fff640 1 -- 172.21.15.76:0/1863136170 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad30009a70 con 0x7fad3815d100 2023-11-09T03:51:44.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.909+0000 7fad37fff640 1 --2- 172.21.15.76:0/1863136170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad3815d100 0x7fad3815e990 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00c500 tx=0x7fad2c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:44.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.909+0000 7fad357fa640 1 -- 172.21.15.76:0/1863136170 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c005ce0 con 0x7fad3815d100 2023-11-09T03:51:44.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.909+0000 7fad357fa640 1 -- 172.21.15.76:0/1863136170 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2c005e40 con 0x7fad3815d100 2023-11-09T03:51:44.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.909+0000 7fad3eada640 1 -- 172.21.15.76:0/1863136170 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad3815f5c0 con 0x7fad3815d100 2023-11-09T03:51:44.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.909+0000 7fad357fa640 1 -- 172.21.15.76:0/1863136170 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c0072f0 con 0x7fad3815d100 2023-11-09T03:51:44.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.909+0000 7fad3eada640 1 -- 172.21.15.76:0/1863136170 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad3815fb80 con 0x7fad3815d100 2023-11-09T03:51:44.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.910+0000 7fad3eada640 1 -- 172.21.15.76:0/1863136170 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad380684b0 con 0x7fad3815d100 2023-11-09T03:51:44.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.911+0000 7fad357fa640 1 -- 172.21.15.76:0/1863136170 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fad2c007450 con 0x7fad3815d100 2023-11-09T03:51:44.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.911+0000 7fad357fa640 1 --2- 172.21.15.76:0/1863136170 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fad2406aea0 0x7fad2406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:44.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.911+0000 7fad357fa640 1 -- 172.21.15.76:0/1863136170 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7694+0+0 (secure 0 0 0) 0x7fad2c0891e0 con 0x7fad3815d100 2023-11-09T03:51:44.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.911+0000 7fad3c876640 1 --2- 172.21.15.76:0/1863136170 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fad2406aea0 0x7fad2406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:44.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.912+0000 7fad3c876640 1 --2- 172.21.15.76:0/1863136170 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fad2406aea0 0x7fad2406d360 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fad28006fd0 tx=0x7fad28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:44.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:44.916+0000 7fad357fa640 1 -- 172.21.15.76:0/1863136170 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fad2c053c10 con 0x7fad3815d100 2023-11-09T03:51:45.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.119+0000 7fad3eada640 1 -- 172.21.15.76:0/1863136170 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad38160030 con 0x7fad3815d100 2023-11-09T03:51:45.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.119+0000 7fad357fa640 1 -- 172.21.15.76:0/1863136170 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4935 (secure 0 0 0) 0x7fad2c017090 con 0x7fad3815d100 2023-11-09T03:51:45.119 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:45.120 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":91,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8310,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:51:40.454195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6904,"mds_1":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6904":{"gid":6904,"name":"b","rank":0,"incarnation":86,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8274":{"gid":8274,"name":"c","rank":1,"incarnation":89,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.76:6833/1536527256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1536527256},{"type":"v1","addr":"172.21.15.76:6833","nonce":1536527256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:51:45.120 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 91 2023-11-09T03:51:45.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.121+0000 7fad3eada640 1 -- 172.21.15.76:0/1863136170 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fad2406aea0 msgr2=0x7fad2406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:45.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.121+0000 7fad3eada640 1 --2- 172.21.15.76:0/1863136170 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fad2406aea0 0x7fad2406d360 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fad28006fd0 tx=0x7fad28008040 comp rx=0 tx=0).stop 2023-11-09T03:51:45.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.121+0000 7fad3eada640 1 -- 172.21.15.76:0/1863136170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad3815d100 msgr2=0x7fad3815e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:45.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.121+0000 7fad3eada640 1 --2- 172.21.15.76:0/1863136170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad3815d100 0x7fad3815e990 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00c500 tx=0x7fad2c00c9c0 comp rx=0 tx=0).stop 2023-11-09T03:51:45.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.121+0000 7fad3eada640 1 -- 172.21.15.76:0/1863136170 shutdown_connections 2023-11-09T03:51:45.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.121+0000 7fad3eada640 1 --2- 172.21.15.76:0/1863136170 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fad2406aea0 0x7fad2406d360 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:45.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.121+0000 7fad3eada640 1 --2- 172.21.15.76:0/1863136170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad3815d4e0 0x7fad381577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:45.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.121+0000 7fad3eada640 1 --2- 172.21.15.76:0/1863136170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad3815d100 0x7fad3815e990 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:45.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.121+0000 7fad3eada640 1 --2- 172.21.15.76:0/1863136170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad380d1be0 0x7fad381f2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:45.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.121+0000 7fad3eada640 1 -- 172.21.15.76:0/1863136170 >> 172.21.15.76:0/1863136170 conn(0x7fad380d2ef0 msgr2=0x7fad380d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:45.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.122+0000 7fad3eada640 1 -- 172.21.15.76:0/1863136170 shutdown_connections 2023-11-09T03:51:45.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.122+0000 7fad3eada640 1 -- 172.21.15.76:0/1863136170 wait complete. 2023-11-09T03:51:45.132 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-09T03:51:45.132 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:51:45.231 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.232+0000 7f40cfb1d640 1 Processor -- start 2023-11-09T03:51:45.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.232+0000 7f40cfb1d640 1 -- start start 2023-11-09T03:51:45.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.233+0000 7f40cfb1d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f40c8159e40 0x7f40c80b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:45.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.233+0000 7f40cfb1d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f40c806bb60 0x7f40c80b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:45.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.233+0000 7f40cfb1d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c806fff0 0x7f40c8070460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:45.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.233+0000 7f40cfb1d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c8071500 con 0x7f40c8159e40 2023-11-09T03:51:45.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.233+0000 7f40cfb1d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c815dae0 con 0x7f40c806bb60 2023-11-09T03:51:45.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.233+0000 7f40cfb1d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c80709a0 con 0x7f40c806fff0 2023-11-09T03:51:45.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.233+0000 7f40cd8b9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f40c8159e40 0x7f40c80b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:45.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.233+0000 7f40cd8b9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f40c8159e40 0x7f40c80b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49124/0 (socket says 172.21.15.76:49124) 2023-11-09T03:51:45.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.233+0000 7f40cd8b9640 1 -- 172.21.15.76:0/2140586705 learned_addr learned my addr 172.21.15.76:0/2140586705 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:45.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.233+0000 7f40ce0ba640 1 --2- 172.21.15.76:0/2140586705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c806fff0 0x7f40c8070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:45.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.233+0000 7f40cd0b8640 1 --2- 172.21.15.76:0/2140586705 >> v2:172.21.15.113:3300/0 conn(0x7f40c806bb60 0x7f40c80b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:45.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.234+0000 7f40cd8b9640 1 -- 172.21.15.76:0/2140586705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c806fff0 msgr2=0x7f40c8070460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:45.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.234+0000 7f40cd8b9640 1 --2- 172.21.15.76:0/2140586705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c806fff0 0x7f40c8070460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:45.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.234+0000 7f40cd8b9640 1 -- 172.21.15.76:0/2140586705 >> v2:172.21.15.113:3300/0 conn(0x7f40c806bb60 msgr2=0x7f40c80b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:45.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.234+0000 7f40cd8b9640 1 --2- 172.21.15.76:0/2140586705 >> v2:172.21.15.113:3300/0 conn(0x7f40c806bb60 0x7f40c80b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:45.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.234+0000 7f40cd8b9640 1 -- 172.21.15.76:0/2140586705 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c8070c60 con 0x7f40c8159e40 2023-11-09T03:51:45.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.234+0000 7f40cd8b9640 1 --2- 172.21.15.76:0/2140586705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40c8159e40 0x7f40c80b0530 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f40c806fdc0 tx=0x7f40c402e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4eda19066606c0c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:45.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.234+0000 7f40cc8b7640 1 -- 172.21.15.76:0/2140586705 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c402ee00 con 0x7f40c8159e40 2023-11-09T03:51:45.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.234+0000 7f40cc8b7640 1 -- 172.21.15.76:0/2140586705 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c4036440 con 0x7f40c8159e40 2023-11-09T03:51:45.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.234+0000 7f40cc8b7640 1 -- 172.21.15.76:0/2140586705 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c40367d0 con 0x7f40c8159e40 2023-11-09T03:51:45.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.235+0000 7f40cfb1d640 1 -- 172.21.15.76:0/2140586705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40c8159e40 msgr2=0x7f40c80b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:45.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.235+0000 7f40cfb1d640 1 --2- 172.21.15.76:0/2140586705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40c8159e40 0x7f40c80b0530 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f40c806fdc0 tx=0x7f40c402e050 comp rx=0 tx=0).stop 2023-11-09T03:51:45.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.235+0000 7f40cfb1d640 1 -- 172.21.15.76:0/2140586705 shutdown_connections 2023-11-09T03:51:45.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.235+0000 7f40cfb1d640 1 --2- 172.21.15.76:0/2140586705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c806fff0 0x7f40c8070460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:45.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.235+0000 7f40cfb1d640 1 --2- 172.21.15.76:0/2140586705 >> v2:172.21.15.113:3300/0 conn(0x7f40c806bb60 0x7f40c80b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:45.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.235+0000 7f40cfb1d640 1 --2- 172.21.15.76:0/2140586705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40c8159e40 0x7f40c80b0530 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:45.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.235+0000 7f40cfb1d640 1 -- 172.21.15.76:0/2140586705 >> 172.21.15.76:0/2140586705 conn(0x7f40c80cb530 msgr2=0x7f40c80cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:45.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.235+0000 7f40cfb1d640 1 -- 172.21.15.76:0/2140586705 shutdown_connections 2023-11-09T03:51:45.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.235+0000 7f40cfb1d640 1 -- 172.21.15.76:0/2140586705 wait complete. 2023-11-09T03:51:45.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.236+0000 7f40cfb1d640 1 Processor -- start 2023-11-09T03:51:45.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.236+0000 7f40cfb1d640 1 -- start start 2023-11-09T03:51:45.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.237+0000 7f40cfb1d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c806bb60 0x7f40c8153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:45.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.237+0000 7f40cfb1d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c806fff0 0x7f40c80c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:45.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.237+0000 7f40cfb1d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40c8159e40 0x7f40c8153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:45.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.237+0000 7f40cfb1d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c80c8b40 con 0x7f40c8159e40 2023-11-09T03:51:45.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.237+0000 7f40cfb1d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c80c8630 con 0x7f40c806bb60 2023-11-09T03:51:45.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.237+0000 7f40cfb1d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c8153990 con 0x7f40c806fff0 2023-11-09T03:51:45.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.237+0000 7f40ce0ba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40c8159e40 0x7f40c8153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:45.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.237+0000 7f40ce0ba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40c8159e40 0x7f40c8153450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49132/0 (socket says 172.21.15.76:49132) 2023-11-09T03:51:45.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.237+0000 7f40ce0ba640 1 -- 172.21.15.76:0/4158378036 learned_addr learned my addr 172.21.15.76:0/4158378036 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:45.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.237+0000 7f40cd8b9640 1 --2- 172.21.15.76:0/4158378036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c806bb60 0x7f40c8153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:45.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.237+0000 7f40cd0b8640 1 --2- 172.21.15.76:0/4158378036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c806fff0 0x7f40c80c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:45.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.237+0000 7f40ce0ba640 1 -- 172.21.15.76:0/4158378036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c806fff0 msgr2=0x7f40c80c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:45.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.237+0000 7f40ce0ba640 1 --2- 172.21.15.76:0/4158378036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c806fff0 0x7f40c80c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:45.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.237+0000 7f40ce0ba640 1 -- 172.21.15.76:0/4158378036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c806bb60 msgr2=0x7f40c8153b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:45.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.237+0000 7f40ce0ba640 1 --2- 172.21.15.76:0/4158378036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c806bb60 0x7f40c8153b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:45.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.237+0000 7f40ce0ba640 1 -- 172.21.15.76:0/4158378036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c400b770 con 0x7f40c8159e40 2023-11-09T03:51:45.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.237+0000 7f40cd0b8640 1 --2- 172.21.15.76:0/4158378036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c806fff0 0x7f40c80c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:51:45.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.238+0000 7f40ce0ba640 1 --2- 172.21.15.76:0/4158378036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40c8159e40 0x7f40c8153450 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f40c0007e70 tx=0x7f40c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:45.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.238+0000 7f40be7fc640 1 -- 172.21.15.76:0/4158378036 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c0014420 con 0x7f40c8159e40 2023-11-09T03:51:45.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.238+0000 7f40be7fc640 1 -- 172.21.15.76:0/4158378036 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c00145a0 con 0x7f40c8159e40 2023-11-09T03:51:45.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.238+0000 7f40cfb1d640 1 -- 172.21.15.76:0/4158378036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c80c3e70 con 0x7f40c8159e40 2023-11-09T03:51:45.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.238+0000 7f40be7fc640 1 -- 172.21.15.76:0/4158378036 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c0014950 con 0x7f40c8159e40 2023-11-09T03:51:45.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.238+0000 7f40cfb1d640 1 -- 172.21.15.76:0/4158378036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c80c8860 con 0x7f40c8159e40 2023-11-09T03:51:45.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.239+0000 7f40cfb1d640 1 -- 172.21.15.76:0/4158378036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c80c4000 con 0x7f40c8159e40 2023-11-09T03:51:45.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.239+0000 7f40be7fc640 1 -- 172.21.15.76:0/4158378036 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f40c0014ab0 con 0x7f40c8159e40 2023-11-09T03:51:45.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.240+0000 7f40be7fc640 1 --2- 172.21.15.76:0/4158378036 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f40ac06ae70 0x7f40ac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:45.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.240+0000 7f40be7fc640 1 -- 172.21.15.76:0/4158378036 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7694+0+0 (secure 0 0 0) 0x7f40c008a6d0 con 0x7f40c8159e40 2023-11-09T03:51:45.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.240+0000 7f40cd8b9640 1 --2- 172.21.15.76:0/4158378036 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f40ac06ae70 0x7f40ac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:45.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.240+0000 7f40cd8b9640 1 --2- 172.21.15.76:0/4158378036 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f40ac06ae70 0x7f40ac06d330 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f40c4038000 tx=0x7f40c4032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:45.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.244+0000 7f40be7fc640 1 -- 172.21.15.76:0/4158378036 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f40c0054fc0 con 0x7f40c8159e40 2023-11-09T03:51:45.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.446+0000 7f40cfb1d640 1 -- 172.21.15.76:0/4158378036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40c80684b0 con 0x7f40c8159e40 2023-11-09T03:51:45.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.446+0000 7f40be7fc640 1 -- 172.21.15.76:0/4158378036 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4935 (secure 0 0 0) 0x7f40c00585e0 con 0x7f40c8159e40 2023-11-09T03:51:45.446 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:45.447 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":91,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8310,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:51:40.454195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6904,"mds_1":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6904":{"gid":6904,"name":"b","rank":0,"incarnation":86,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8274":{"gid":8274,"name":"c","rank":1,"incarnation":89,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.76:6833/1536527256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1536527256},{"type":"v1","addr":"172.21.15.76:6833","nonce":1536527256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:51:45.447 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 91 2023-11-09T03:51:45.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.448+0000 7f40cfb1d640 1 -- 172.21.15.76:0/4158378036 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f40ac06ae70 msgr2=0x7f40ac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:45.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.448+0000 7f40cfb1d640 1 --2- 172.21.15.76:0/4158378036 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f40ac06ae70 0x7f40ac06d330 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f40c4038000 tx=0x7f40c4032000 comp rx=0 tx=0).stop 2023-11-09T03:51:45.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.448+0000 7f40cfb1d640 1 -- 172.21.15.76:0/4158378036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40c8159e40 msgr2=0x7f40c8153450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:45.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.448+0000 7f40cfb1d640 1 --2- 172.21.15.76:0/4158378036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40c8159e40 0x7f40c8153450 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f40c0007e70 tx=0x7f40c000e4b0 comp rx=0 tx=0).stop 2023-11-09T03:51:45.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.448+0000 7f40cfb1d640 1 -- 172.21.15.76:0/4158378036 shutdown_connections 2023-11-09T03:51:45.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.448+0000 7f40cfb1d640 1 --2- 172.21.15.76:0/4158378036 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f40ac06ae70 0x7f40ac06d330 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:45.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.448+0000 7f40cfb1d640 1 --2- 172.21.15.76:0/4158378036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40c8159e40 0x7f40c8153450 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:45.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.448+0000 7f40cfb1d640 1 --2- 172.21.15.76:0/4158378036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c806fff0 0x7f40c80c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:45.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.448+0000 7f40cfb1d640 1 --2- 172.21.15.76:0/4158378036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c806bb60 0x7f40c8153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:45.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.448+0000 7f40cfb1d640 1 -- 172.21.15.76:0/4158378036 >> 172.21.15.76:0/4158378036 conn(0x7f40c80cb530 msgr2=0x7f40c80cea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:45.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.448+0000 7f40cfb1d640 1 -- 172.21.15.76:0/4158378036 shutdown_connections 2023-11-09T03:51:45.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:45.448+0000 7f40cfb1d640 1 -- 172.21.15.76:0/4158378036 wait complete. 2023-11-09T03:51:45.460 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:reconnect 2023-11-09T03:51:45.563 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-11-09T03:51:46.461 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:51:46.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.576+0000 7f8e1035e640 1 Processor -- start 2023-11-09T03:51:46.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.576+0000 7f8e1035e640 1 -- start start 2023-11-09T03:51:46.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.577+0000 7f8e1035e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8e0815d100 0x7f8e080ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:46.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.577+0000 7f8e1035e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e080d1be0 0x7f8e080b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:46.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.577+0000 7f8e1035e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e080b32b0 con 0x7f8e0815d4e0 2023-11-09T03:51:46.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.577+0000 7f8e1035e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e080cee10 con 0x7f8e0815d100 2023-11-09T03:51:46.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.577+0000 7f8e1035e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e080b2980 con 0x7f8e080d1be0 2023-11-09T03:51:46.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.577+0000 7f8e0e0fa640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8e0815d100 0x7f8e080ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:46.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.578+0000 7f8e0d8f9640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8e0815d4e0 0x7f8e080d14d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46638/0 (socket says 172.21.15.76:46638) 2023-11-09T03:51:46.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.578+0000 7f8e0d8f9640 1 -- 172.21.15.76:0/2920288633 learned_addr learned my addr 172.21.15.76:0/2920288633 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:51:46.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.578+0000 7f8e0e8fb640 1 --2- 172.21.15.76:0/2920288633 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e080d1be0 0x7f8e080b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:46.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.578+0000 7f8e0e0fa640 1 -- 172.21.15.76:0/2920288633 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e080d1be0 msgr2=0x7f8e080b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:46.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.578+0000 7f8e0e0fa640 1 --2- 172.21.15.76:0/2920288633 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e080d1be0 0x7f8e080b3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:46.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.578+0000 7f8e0e0fa640 1 -- 172.21.15.76:0/2920288633 >> v1:172.21.15.76:6789/0 conn(0x7f8e0815d4e0 legacy=0x7f8e080d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:46.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.578+0000 7f8e0d0f8640 1 -- 172.21.15.76:0/2920288633 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 115084072 0 0) 0x7f8e080b32b0 con 0x7f8e0815d4e0 2023-11-09T03:51:46.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.578+0000 7f8e0e0fa640 1 -- 172.21.15.76:0/2920288633 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e080b2c60 con 0x7f8e0815d100 2023-11-09T03:51:46.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.578+0000 7f8e0e8fb640 1 --2- 172.21.15.76:0/2920288633 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e080d1be0 0x7f8e080b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:51:46.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.579+0000 7f8e0e0fa640 1 --2- 172.21.15.76:0/2920288633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e0815d100 0x7f8e080ce510 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f8e04009de0 tx=0x7f8e0402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab47c7953e0f453 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:46.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.579+0000 7f8e0d0f8640 1 -- 172.21.15.76:0/2920288633 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0402cc90 con 0x7f8e0815d100 2023-11-09T03:51:46.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.579+0000 7f8e0d0f8640 1 -- 172.21.15.76:0/2920288633 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0402cdf0 con 0x7f8e0815d100 2023-11-09T03:51:46.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.579+0000 7f8e0d0f8640 1 -- 172.21.15.76:0/2920288633 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e04034660 con 0x7f8e0815d100 2023-11-09T03:51:46.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.579+0000 7f8e1035e640 1 -- 172.21.15.76:0/2920288633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e0815d100 msgr2=0x7f8e080ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:46.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.579+0000 7f8e1035e640 1 --2- 172.21.15.76:0/2920288633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e0815d100 0x7f8e080ce510 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f8e04009de0 tx=0x7f8e0402bee0 comp rx=0 tx=0).stop 2023-11-09T03:51:46.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.579+0000 7f8e1035e640 1 -- 172.21.15.76:0/2920288633 shutdown_connections 2023-11-09T03:51:46.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.579+0000 7f8e1035e640 1 --2- 172.21.15.76:0/2920288633 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e080d1be0 0x7f8e080b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:46.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.580+0000 7f8e1035e640 1 --2- 172.21.15.76:0/2920288633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e0815d100 0x7f8e080ce510 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:46.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.580+0000 7f8e1035e640 1 -- 172.21.15.76:0/2920288633 >> 172.21.15.76:0/2920288633 conn(0x7f8e080d2ef0 msgr2=0x7f8e080d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:46.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.580+0000 7f8e1035e640 1 -- 172.21.15.76:0/2920288633 shutdown_connections 2023-11-09T03:51:46.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.580+0000 7f8e1035e640 1 -- 172.21.15.76:0/2920288633 wait complete. 2023-11-09T03:51:46.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.581+0000 7f8e1035e640 1 Processor -- start 2023-11-09T03:51:46.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.581+0000 7f8e1035e640 1 -- start start 2023-11-09T03:51:46.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.582+0000 7f8e1035e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e080d1be0 0x7f8e081f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:46.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.582+0000 7f8e1035e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e0815d100 0x7f8e0815e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:46.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.582+0000 7f8e1035e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e0815d4e0 0x7f8e0806b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:46.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.582+0000 7f8e1035e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0815f980 con 0x7f8e080d1be0 2023-11-09T03:51:46.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.582+0000 7f8e1035e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e080cb170 con 0x7f8e0815d100 2023-11-09T03:51:46.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.582+0000 7f8e1035e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0815f3c0 con 0x7f8e0815d4e0 2023-11-09T03:51:46.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.582+0000 7f8e0e0fa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e080d1be0 0x7f8e081f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:46.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.582+0000 7f8e0e0fa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e080d1be0 0x7f8e081f3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49146/0 (socket says 172.21.15.76:49146) 2023-11-09T03:51:46.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.582+0000 7f8e0e0fa640 1 -- 172.21.15.76:0/4143027464 learned_addr learned my addr 172.21.15.76:0/4143027464 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:46.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.582+0000 7f8e0e8fb640 1 --2- 172.21.15.76:0/4143027464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e0815d4e0 0x7f8e0806b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:46.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.582+0000 7f8e0d8f9640 1 --2- 172.21.15.76:0/4143027464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e0815d100 0x7f8e0815e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:46.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.582+0000 7f8e0e0fa640 1 -- 172.21.15.76:0/4143027464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e0815d4e0 msgr2=0x7f8e0806b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:46.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.582+0000 7f8e0e0fa640 1 --2- 172.21.15.76:0/4143027464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e0815d4e0 0x7f8e0806b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:46.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.582+0000 7f8e0e0fa640 1 -- 172.21.15.76:0/4143027464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e0815d100 msgr2=0x7f8e0815e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:46.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.582+0000 7f8e0e0fa640 1 --2- 172.21.15.76:0/4143027464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e0815d100 0x7f8e0815e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:46.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.582+0000 7f8e0e0fa640 1 -- 172.21.15.76:0/4143027464 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e04009a70 con 0x7f8e080d1be0 2023-11-09T03:51:46.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.582+0000 7f8e0e8fb640 1 --2- 172.21.15.76:0/4143027464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e0815d4e0 0x7f8e0806b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:51:46.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.582+0000 7f8e0e0fa640 1 --2- 172.21.15.76:0/4143027464 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e080d1be0 0x7f8e081f3c00 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f8e04037040 tx=0x7f8e04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:46.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.582+0000 7f8dfeffd640 1 -- 172.21.15.76:0/4143027464 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0403e610 con 0x7f8e080d1be0 2023-11-09T03:51:46.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.582+0000 7f8dfeffd640 1 -- 172.21.15.76:0/4143027464 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0403e770 con 0x7f8e080d1be0 2023-11-09T03:51:46.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.582+0000 7f8dfeffd640 1 -- 172.21.15.76:0/4143027464 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0403eb60 con 0x7f8e080d1be0 2023-11-09T03:51:46.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.582+0000 7f8e1035e640 1 -- 172.21.15.76:0/4143027464 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e0815f6a0 con 0x7f8e080d1be0 2023-11-09T03:51:46.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.583+0000 7f8e1035e640 1 -- 172.21.15.76:0/4143027464 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e080cb6d0 con 0x7f8e080d1be0 2023-11-09T03:51:46.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.584+0000 7f8e1035e640 1 -- 172.21.15.76:0/4143027464 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e080684b0 con 0x7f8e080d1be0 2023-11-09T03:51:46.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.584+0000 7f8dfeffd640 1 -- 172.21.15.76:0/4143027464 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8e0403b050 con 0x7f8e080d1be0 2023-11-09T03:51:46.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.584+0000 7f8dfeffd640 1 --2- 172.21.15.76:0/4143027464 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8dec06ae70 0x7f8dec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:46.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.585+0000 7f8dfeffd640 1 -- 172.21.15.76:0/4143027464 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7694+0+0 (secure 0 0 0) 0x7f8e040ac740 con 0x7f8e080d1be0 2023-11-09T03:51:46.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.585+0000 7f8e0d8f9640 1 --2- 172.21.15.76:0/4143027464 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8dec06ae70 0x7f8dec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:46.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.585+0000 7f8e0d8f9640 1 --2- 172.21.15.76:0/4143027464 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8dec06ae70 0x7f8dec06d330 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8e080a0ed0 tx=0x7f8df8007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:46.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.589+0000 7f8dfeffd640 1 -- 172.21.15.76:0/4143027464 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8e040770c0 con 0x7f8e080d1be0 2023-11-09T03:51:46.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.793+0000 7f8e1035e640 1 -- 172.21.15.76:0/4143027464 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e080c9d30 con 0x7f8e080d1be0 2023-11-09T03:51:46.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.793+0000 7f8dfeffd640 1 -- 172.21.15.76:0/4143027464 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4935 (secure 0 0 0) 0x7f8e0407a6e0 con 0x7f8e080d1be0 2023-11-09T03:51:46.793 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:46.793 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":91,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8310,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:51:40.454195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6904,"mds_1":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6904":{"gid":6904,"name":"b","rank":0,"incarnation":86,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8274":{"gid":8274,"name":"c","rank":1,"incarnation":89,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.76:6833/1536527256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1536527256},{"type":"v1","addr":"172.21.15.76:6833","nonce":1536527256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:51:46.793 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 91 2023-11-09T03:51:46.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.795+0000 7f8e1035e640 1 -- 172.21.15.76:0/4143027464 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8dec06ae70 msgr2=0x7f8dec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:46.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.795+0000 7f8e1035e640 1 --2- 172.21.15.76:0/4143027464 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8dec06ae70 0x7f8dec06d330 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8e080a0ed0 tx=0x7f8df8007c00 comp rx=0 tx=0).stop 2023-11-09T03:51:46.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.795+0000 7f8e1035e640 1 -- 172.21.15.76:0/4143027464 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e080d1be0 msgr2=0x7f8e081f3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:46.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.795+0000 7f8e1035e640 1 --2- 172.21.15.76:0/4143027464 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e080d1be0 0x7f8e081f3c00 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f8e04037040 tx=0x7f8e04004a40 comp rx=0 tx=0).stop 2023-11-09T03:51:46.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.795+0000 7f8e1035e640 1 -- 172.21.15.76:0/4143027464 shutdown_connections 2023-11-09T03:51:46.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.795+0000 7f8e1035e640 1 --2- 172.21.15.76:0/4143027464 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8dec06ae70 0x7f8dec06d330 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:46.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.795+0000 7f8e1035e640 1 --2- 172.21.15.76:0/4143027464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e0815d4e0 0x7f8e0806b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:46.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.795+0000 7f8e1035e640 1 --2- 172.21.15.76:0/4143027464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e0815d100 0x7f8e0815e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:46.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.795+0000 7f8e1035e640 1 --2- 172.21.15.76:0/4143027464 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e080d1be0 0x7f8e081f3c00 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:46.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.795+0000 7f8e1035e640 1 -- 172.21.15.76:0/4143027464 >> 172.21.15.76:0/4143027464 conn(0x7f8e080d2ef0 msgr2=0x7f8e080d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:46.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.795+0000 7f8e1035e640 1 -- 172.21.15.76:0/4143027464 shutdown_connections 2023-11-09T03:51:46.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:46.795+0000 7f8e1035e640 1 -- 172.21.15.76:0/4143027464 wait complete. 2023-11-09T03:51:46.805 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:reconnect 2023-11-09T03:51:47.807 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:51:47.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.918+0000 7f7d50e5d640 1 Processor -- start 2023-11-09T03:51:47.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.918+0000 7f7d50e5d640 1 -- start start 2023-11-09T03:51:47.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.919+0000 7f7d50e5d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7d4c15d4e0 0x7f7d4c0d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:47.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.919+0000 7f7d50e5d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7d4c0d3b60 0x7f7d4c0d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:47.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.919+0000 7f7d50e5d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d4c0b32b0 con 0x7f7d4c15d100 2023-11-09T03:51:47.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.919+0000 7f7d50e5d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d4c0cec20 con 0x7f7d4c15d4e0 2023-11-09T03:51:47.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.919+0000 7f7d50e5d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d4c0cee10 con 0x7f7d4c0d3b60 2023-11-09T03:51:47.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.919+0000 7f7d43fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7d4c15d4e0 0x7f7d4c0d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:47.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.919+0000 7f7d4a59c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f7d4c15d100 0x7f7d4c0ce510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46642/0 (socket says 172.21.15.76:46642) 2023-11-09T03:51:47.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.919+0000 7f7d4a59c640 1 -- 172.21.15.76:0/3901690655 learned_addr learned my addr 172.21.15.76:0/3901690655 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:51:47.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.919+0000 7f7d4ad9d640 1 --2- 172.21.15.76:0/3901690655 >> v2:172.21.15.113:3300/0 conn(0x7f7d4c0d3b60 0x7f7d4c0d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:47.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.919+0000 7f7d43fff640 1 -- 172.21.15.76:0/3901690655 >> v2:172.21.15.113:3300/0 conn(0x7f7d4c0d3b60 msgr2=0x7f7d4c0d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:47.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.919+0000 7f7d43fff640 1 --2- 172.21.15.76:0/3901690655 >> v2:172.21.15.113:3300/0 conn(0x7f7d4c0d3b60 0x7f7d4c0d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:47.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.919+0000 7f7d43fff640 1 -- 172.21.15.76:0/3901690655 >> v1:172.21.15.76:6789/0 conn(0x7f7d4c15d100 legacy=0x7f7d4c0ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:47.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.919+0000 7f7d49d9b640 1 -- 172.21.15.76:0/3901690655 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2840147964 0 0) 0x7f7d4c0b32b0 con 0x7f7d4c15d100 2023-11-09T03:51:47.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.919+0000 7f7d43fff640 1 -- 172.21.15.76:0/3901690655 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4c0b4750 con 0x7f7d4c15d4e0 2023-11-09T03:51:47.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.920+0000 7f7d43fff640 1 --2- 172.21.15.76:0/3901690655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d4c15d4e0 0x7f7d4c0d14d0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c08d6a0 tx=0x7f7d3402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f3188445d1f49b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:47.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.920+0000 7f7d49d9b640 1 -- 172.21.15.76:0/3901690655 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d34031830 con 0x7f7d4c15d4e0 2023-11-09T03:51:47.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.920+0000 7f7d49d9b640 1 -- 172.21.15.76:0/3901690655 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d34031990 con 0x7f7d4c15d4e0 2023-11-09T03:51:47.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.920+0000 7f7d49d9b640 1 -- 172.21.15.76:0/3901690655 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d34031d20 con 0x7f7d4c15d4e0 2023-11-09T03:51:47.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.921+0000 7f7d50e5d640 1 -- 172.21.15.76:0/3901690655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d4c15d4e0 msgr2=0x7f7d4c0d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:47.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.921+0000 7f7d50e5d640 1 --2- 172.21.15.76:0/3901690655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d4c15d4e0 0x7f7d4c0d14d0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c08d6a0 tx=0x7f7d3402b8d0 comp rx=0 tx=0).stop 2023-11-09T03:51:47.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.921+0000 7f7d50e5d640 1 -- 172.21.15.76:0/3901690655 shutdown_connections 2023-11-09T03:51:47.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.921+0000 7f7d50e5d640 1 --2- 172.21.15.76:0/3901690655 >> v2:172.21.15.113:3300/0 conn(0x7f7d4c0d3b60 0x7f7d4c0d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:47.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.921+0000 7f7d50e5d640 1 --2- 172.21.15.76:0/3901690655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d4c15d4e0 0x7f7d4c0d14d0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:47.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.921+0000 7f7d50e5d640 1 -- 172.21.15.76:0/3901690655 >> 172.21.15.76:0/3901690655 conn(0x7f7d4c0d2ef0 msgr2=0x7f7d4c0d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:47.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.921+0000 7f7d50e5d640 1 -- 172.21.15.76:0/3901690655 shutdown_connections 2023-11-09T03:51:47.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.921+0000 7f7d50e5d640 1 -- 172.21.15.76:0/3901690655 wait complete. 2023-11-09T03:51:47.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.922+0000 7f7d50e5d640 1 Processor -- start 2023-11-09T03:51:47.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.922+0000 7f7d50e5d640 1 -- start start 2023-11-09T03:51:47.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.923+0000 7f7d50e5d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d4c0d3b60 0x7f7d4c0ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:47.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.923+0000 7f7d50e5d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d4c15d100 0x7f7d4c0b6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:47.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.923+0000 7f7d50e5d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d4c15d4e0 0x7f7d4c15a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:47.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.923+0000 7f7d50e5d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4c0b7440 con 0x7f7d4c0d3b60 2023-11-09T03:51:47.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.923+0000 7f7d50e5d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4c0b7720 con 0x7f7d4c15d4e0 2023-11-09T03:51:47.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.923+0000 7f7d50e5d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4c0b7a00 con 0x7f7d4c15d100 2023-11-09T03:51:47.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.923+0000 7f7d4a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d4c0d3b60 0x7f7d4c0ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:47.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.923+0000 7f7d4a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d4c0d3b60 0x7f7d4c0ccbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49164/0 (socket says 172.21.15.76:49164) 2023-11-09T03:51:47.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.923+0000 7f7d4a59c640 1 -- 172.21.15.76:0/2978431142 learned_addr learned my addr 172.21.15.76:0/2978431142 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:47.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.923+0000 7f7d4ad9d640 1 --2- 172.21.15.76:0/2978431142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d4c15d4e0 0x7f7d4c15a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:47.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.923+0000 7f7d43fff640 1 --2- 172.21.15.76:0/2978431142 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d4c15d100 0x7f7d4c0b6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:47.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.923+0000 7f7d4a59c640 1 -- 172.21.15.76:0/2978431142 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d4c15d100 msgr2=0x7f7d4c0b6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:47.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.923+0000 7f7d4a59c640 1 --2- 172.21.15.76:0/2978431142 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d4c15d100 0x7f7d4c0b6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:47.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.923+0000 7f7d4a59c640 1 -- 172.21.15.76:0/2978431142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d4c15d4e0 msgr2=0x7f7d4c15a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:47.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.923+0000 7f7d4a59c640 1 --2- 172.21.15.76:0/2978431142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d4c15d4e0 0x7f7d4c15a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:47.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.923+0000 7f7d4a59c640 1 -- 172.21.15.76:0/2978431142 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d34009400 con 0x7f7d4c0d3b60 2023-11-09T03:51:47.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.923+0000 7f7d4a59c640 1 --2- 172.21.15.76:0/2978431142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d4c0d3b60 0x7f7d4c0ccbb0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c0010e0 tx=0x7f7d3c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:47.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.924+0000 7f7d42ffd640 1 -- 172.21.15.76:0/2978431142 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c007480 con 0x7f7d4c0d3b60 2023-11-09T03:51:47.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.924+0000 7f7d42ffd640 1 -- 172.21.15.76:0/2978431142 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3c0075e0 con 0x7f7d4c0d3b60 2023-11-09T03:51:47.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.924+0000 7f7d50e5d640 1 -- 172.21.15.76:0/2978431142 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d4c0b7ce0 con 0x7f7d4c0d3b60 2023-11-09T03:51:47.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.924+0000 7f7d42ffd640 1 -- 172.21.15.76:0/2978431142 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c0059c0 con 0x7f7d4c0d3b60 2023-11-09T03:51:47.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.924+0000 7f7d50e5d640 1 -- 172.21.15.76:0/2978431142 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d4c0b7130 con 0x7f7d4c0d3b60 2023-11-09T03:51:47.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.925+0000 7f7d50e5d640 1 -- 172.21.15.76:0/2978431142 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d4c0b8200 con 0x7f7d4c0d3b60 2023-11-09T03:51:47.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.925+0000 7f7d42ffd640 1 -- 172.21.15.76:0/2978431142 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7d3c005b20 con 0x7f7d4c0d3b60 2023-11-09T03:51:47.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.925+0000 7f7d42ffd640 1 --2- 172.21.15.76:0/2978431142 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7d2c06ae70 0x7f7d2c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:47.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.926+0000 7f7d42ffd640 1 -- 172.21.15.76:0/2978431142 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7694+0+0 (secure 0 0 0) 0x7f7d3c089390 con 0x7f7d4c0d3b60 2023-11-09T03:51:47.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.926+0000 7f7d43fff640 1 --2- 172.21.15.76:0/2978431142 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7d2c06ae70 0x7f7d2c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:47.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.926+0000 7f7d43fff640 1 --2- 172.21.15.76:0/2978431142 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7d2c06ae70 0x7f7d2c06d330 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7d34036000 tx=0x7f7d3403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:47.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:47.930+0000 7f7d42ffd640 1 -- 172.21.15.76:0/2978431142 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7d3c053c80 con 0x7f7d4c0d3b60 2023-11-09T03:51:48.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:48.131+0000 7f7d50e5d640 1 -- 172.21.15.76:0/2978431142 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d4c0c9a30 con 0x7f7d4c0d3b60 2023-11-09T03:51:48.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:48.132+0000 7f7d42ffd640 1 -- 172.21.15.76:0/2978431142 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4935 (secure 0 0 0) 0x7f7d3c0572a0 con 0x7f7d4c0d3b60 2023-11-09T03:51:48.132 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:48.132 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":91,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8310,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:51:40.454195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6904,"mds_1":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6904":{"gid":6904,"name":"b","rank":0,"incarnation":86,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8274":{"gid":8274,"name":"c","rank":1,"incarnation":89,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.76:6833/1536527256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1536527256},{"type":"v1","addr":"172.21.15.76:6833","nonce":1536527256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:51:48.132 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 91 2023-11-09T03:51:48.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:48.134+0000 7f7d50e5d640 1 -- 172.21.15.76:0/2978431142 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7d2c06ae70 msgr2=0x7f7d2c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:48.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:48.134+0000 7f7d50e5d640 1 --2- 172.21.15.76:0/2978431142 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7d2c06ae70 0x7f7d2c06d330 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7d34036000 tx=0x7f7d3403c040 comp rx=0 tx=0).stop 2023-11-09T03:51:48.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:48.134+0000 7f7d50e5d640 1 -- 172.21.15.76:0/2978431142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d4c0d3b60 msgr2=0x7f7d4c0ccbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:48.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:48.134+0000 7f7d50e5d640 1 --2- 172.21.15.76:0/2978431142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d4c0d3b60 0x7f7d4c0ccbb0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c0010e0 tx=0x7f7d3c00c9b0 comp rx=0 tx=0).stop 2023-11-09T03:51:48.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:48.134+0000 7f7d50e5d640 1 -- 172.21.15.76:0/2978431142 shutdown_connections 2023-11-09T03:51:48.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:48.134+0000 7f7d50e5d640 1 --2- 172.21.15.76:0/2978431142 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7d2c06ae70 0x7f7d2c06d330 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:48.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:48.134+0000 7f7d50e5d640 1 --2- 172.21.15.76:0/2978431142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d4c15d4e0 0x7f7d4c15a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:48.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:48.134+0000 7f7d50e5d640 1 --2- 172.21.15.76:0/2978431142 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7d4c15d100 0x7f7d4c0b6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:48.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:48.134+0000 7f7d50e5d640 1 --2- 172.21.15.76:0/2978431142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d4c0d3b60 0x7f7d4c0ccbb0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:48.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:48.134+0000 7f7d50e5d640 1 -- 172.21.15.76:0/2978431142 >> 172.21.15.76:0/2978431142 conn(0x7f7d4c0d2ef0 msgr2=0x7f7d4c15c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:48.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:48.134+0000 7f7d50e5d640 1 -- 172.21.15.76:0/2978431142 shutdown_connections 2023-11-09T03:51:48.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:48.134+0000 7f7d50e5d640 1 -- 172.21.15.76:0/2978431142 wait complete. 2023-11-09T03:51:48.144 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:reconnect 2023-11-09T03:51:49.145 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:51:49.258 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.258+0000 7f3012c5d640 1 Processor -- start 2023-11-09T03:51:49.258 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.258+0000 7f3012c5d640 1 -- start start 2023-11-09T03:51:49.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.259+0000 7f3012c5d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f300c156ef0 0x7f300c15a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:49.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.259+0000 7f3012c5d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f300c070210 0x7f300c070660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:49.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.259+0000 7f3012c5d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f300c070ba0 con 0x7f300c0ce1c0 2023-11-09T03:51:49.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.259+0000 7f3012c5d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f300c071ad0 con 0x7f300c070210 2023-11-09T03:51:49.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.259+0000 7f3012c5d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f300c071510 con 0x7f300c156ef0 2023-11-09T03:51:49.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.259+0000 7f30111fa640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f300c070210 0x7f300c070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:49.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.259+0000 7f300bfff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f300c0ce1c0 0x7f300c0ae500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46652/0 (socket says 172.21.15.76:46652) 2023-11-09T03:51:49.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.259+0000 7f300bfff640 1 -- 172.21.15.76:0/573807343 learned_addr learned my addr 172.21.15.76:0/573807343 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:51:49.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.260+0000 7f30109f9640 1 --2- 172.21.15.76:0/573807343 >> v2:172.21.15.113:3300/0 conn(0x7f300c156ef0 0x7f300c15a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:49.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.260+0000 7f30111fa640 1 -- 172.21.15.76:0/573807343 >> v2:172.21.15.113:3300/0 conn(0x7f300c156ef0 msgr2=0x7f300c15a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:49.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.260+0000 7f30111fa640 1 --2- 172.21.15.76:0/573807343 >> v2:172.21.15.113:3300/0 conn(0x7f300c156ef0 0x7f300c15a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:49.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.260+0000 7f30111fa640 1 -- 172.21.15.76:0/573807343 >> v1:172.21.15.76:6789/0 conn(0x7f300c0ce1c0 legacy=0x7f300c0ae500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:49.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.260+0000 7f30111fa640 1 -- 172.21.15.76:0/573807343 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f300c0717f0 con 0x7f300c070210 2023-11-09T03:51:49.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.260+0000 7f30111fa640 1 --2- 172.21.15.76:0/573807343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f300c070210 0x7f300c070660 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f300c0b0560 tx=0x7f300402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e6101404bb9b57 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:49.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.260+0000 7f300b7fe640 1 -- 172.21.15.76:0/573807343 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300402cbf0 con 0x7f300c070210 2023-11-09T03:51:49.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.261+0000 7f300b7fe640 1 -- 172.21.15.76:0/573807343 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f300402cd50 con 0x7f300c070210 2023-11-09T03:51:49.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.261+0000 7f300b7fe640 1 -- 172.21.15.76:0/573807343 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3004034660 con 0x7f300c070210 2023-11-09T03:51:49.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.261+0000 7f3012c5d640 1 -- 172.21.15.76:0/573807343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f300c070210 msgr2=0x7f300c070660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:49.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.261+0000 7f3012c5d640 1 --2- 172.21.15.76:0/573807343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f300c070210 0x7f300c070660 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f300c0b0560 tx=0x7f300402be40 comp rx=0 tx=0).stop 2023-11-09T03:51:49.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.261+0000 7f3012c5d640 1 -- 172.21.15.76:0/573807343 shutdown_connections 2023-11-09T03:51:49.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.261+0000 7f3012c5d640 1 --2- 172.21.15.76:0/573807343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f300c070210 0x7f300c070660 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:49.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.261+0000 7f3012c5d640 1 --2- 172.21.15.76:0/573807343 >> v2:172.21.15.113:3300/0 conn(0x7f300c156ef0 0x7f300c15a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:49.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.261+0000 7f3012c5d640 1 -- 172.21.15.76:0/573807343 >> 172.21.15.76:0/573807343 conn(0x7f300c0b0870 msgr2=0x7f300c0b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:49.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.261+0000 7f3012c5d640 1 -- 172.21.15.76:0/573807343 shutdown_connections 2023-11-09T03:51:49.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.261+0000 7f3012c5d640 1 -- 172.21.15.76:0/573807343 wait complete. 2023-11-09T03:51:49.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.262+0000 7f3012c5d640 1 Processor -- start 2023-11-09T03:51:49.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.262+0000 7f3012c5d640 1 -- start start 2023-11-09T03:51:49.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.263+0000 7f3012c5d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f300c070210 0x7f300c14de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:49.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.263+0000 7f3012c5d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f300c0ce1c0 0x7f300c06cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:49.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.263+0000 7f3012c5d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f300c156ef0 0x7f300c0cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:49.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.263+0000 7f3012c5d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c159180 con 0x7f300c070210 2023-11-09T03:51:49.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.263+0000 7f3012c5d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c14a250 con 0x7f300c156ef0 2023-11-09T03:51:49.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.263+0000 7f3012c5d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f300c0ad4e0 con 0x7f300c0ce1c0 2023-11-09T03:51:49.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.263+0000 7f30109f9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f300c070210 0x7f300c14de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:49.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.263+0000 7f30109f9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f300c070210 0x7f300c14de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49184/0 (socket says 172.21.15.76:49184) 2023-11-09T03:51:49.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.263+0000 7f30109f9640 1 -- 172.21.15.76:0/2187373115 learned_addr learned my addr 172.21.15.76:0/2187373115 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:49.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.264+0000 7f30109f9640 1 -- 172.21.15.76:0/2187373115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f300c0ce1c0 msgr2=0x7f300c06cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:49.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.264+0000 7f30109f9640 1 --2- 172.21.15.76:0/2187373115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f300c0ce1c0 0x7f300c06cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:49.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.264+0000 7f30109f9640 1 -- 172.21.15.76:0/2187373115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f300c156ef0 msgr2=0x7f300c0cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:49.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.264+0000 7f30109f9640 1 --2- 172.21.15.76:0/2187373115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f300c156ef0 0x7f300c0cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:49.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.264+0000 7f30109f9640 1 -- 172.21.15.76:0/2187373115 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3004009a70 con 0x7f300c070210 2023-11-09T03:51:49.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.264+0000 7f30109f9640 1 --2- 172.21.15.76:0/2187373115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f300c070210 0x7f300c14de40 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc00d5f0 tx=0x7f2ffc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:49.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.264+0000 7f30097fa640 1 -- 172.21.15.76:0/2187373115 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc0125f0 con 0x7f300c070210 2023-11-09T03:51:49.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.264+0000 7f30097fa640 1 -- 172.21.15.76:0/2187373115 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ffc012750 con 0x7f300c070210 2023-11-09T03:51:49.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.264+0000 7f30097fa640 1 -- 172.21.15.76:0/2187373115 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc012b00 con 0x7f300c070210 2023-11-09T03:51:49.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.264+0000 7f3012c5d640 1 -- 172.21.15.76:0/2187373115 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f300c158bc0 con 0x7f300c070210 2023-11-09T03:51:49.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.264+0000 7f3012c5d640 1 -- 172.21.15.76:0/2187373115 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f300c14d940 con 0x7f300c070210 2023-11-09T03:51:49.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.265+0000 7f3012c5d640 1 -- 172.21.15.76:0/2187373115 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f300c14dad0 con 0x7f300c070210 2023-11-09T03:51:49.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.265+0000 7f30097fa640 1 -- 172.21.15.76:0/2187373115 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2ffc012c60 con 0x7f300c070210 2023-11-09T03:51:49.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.266+0000 7f30097fa640 1 --2- 172.21.15.76:0/2187373115 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2ff806ae70 0x7f2ff806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:49.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.266+0000 7f30097fa640 1 -- 172.21.15.76:0/2187373115 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7737+0+0 (secure 0 0 0) 0x7f2ffc08a210 con 0x7f300c070210 2023-11-09T03:51:49.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.266+0000 7f300bfff640 1 --2- 172.21.15.76:0/2187373115 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2ff806ae70 0x7f2ff806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:49.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.266+0000 7f300bfff640 1 --2- 172.21.15.76:0/2187373115 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2ff806ae70 0x7f2ff806d330 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f300c0cd6f0 tx=0x7f3000007a40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:49.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.270+0000 7f30097fa640 1 -- 172.21.15.76:0/2187373115 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2ffc054b60 con 0x7f300c070210 2023-11-09T03:51:49.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.471+0000 7f3012c5d640 1 -- 172.21.15.76:0/2187373115 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f300c0aebe0 con 0x7f300c070210 2023-11-09T03:51:49.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.472+0000 7f30097fa640 1 -- 172.21.15.76:0/2187373115 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4935 (secure 0 0 0) 0x7f2ffc058180 con 0x7f300c070210 2023-11-09T03:51:49.471 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:49.472 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":91,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8310,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:51:40.454195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6904,"mds_1":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6904":{"gid":6904,"name":"b","rank":0,"incarnation":86,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8274":{"gid":8274,"name":"c","rank":1,"incarnation":89,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.76:6833/1536527256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1536527256},{"type":"v1","addr":"172.21.15.76:6833","nonce":1536527256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:51:49.472 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 91 2023-11-09T03:51:49.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.474+0000 7f3012c5d640 1 -- 172.21.15.76:0/2187373115 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2ff806ae70 msgr2=0x7f2ff806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:49.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.474+0000 7f3012c5d640 1 --2- 172.21.15.76:0/2187373115 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2ff806ae70 0x7f2ff806d330 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f300c0cd6f0 tx=0x7f3000007a40 comp rx=0 tx=0).stop 2023-11-09T03:51:49.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.474+0000 7f3012c5d640 1 -- 172.21.15.76:0/2187373115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f300c070210 msgr2=0x7f300c14de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:49.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.474+0000 7f3012c5d640 1 --2- 172.21.15.76:0/2187373115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f300c070210 0x7f300c14de40 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc00d5f0 tx=0x7f2ffc00dab0 comp rx=0 tx=0).stop 2023-11-09T03:51:49.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.474+0000 7f3012c5d640 1 -- 172.21.15.76:0/2187373115 shutdown_connections 2023-11-09T03:51:49.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.474+0000 7f3012c5d640 1 --2- 172.21.15.76:0/2187373115 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2ff806ae70 0x7f2ff806d330 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:49.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.474+0000 7f3012c5d640 1 --2- 172.21.15.76:0/2187373115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f300c156ef0 0x7f300c0cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:49.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.474+0000 7f3012c5d640 1 --2- 172.21.15.76:0/2187373115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f300c0ce1c0 0x7f300c06cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:49.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.474+0000 7f3012c5d640 1 --2- 172.21.15.76:0/2187373115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f300c070210 0x7f300c14de40 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:49.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.474+0000 7f3012c5d640 1 -- 172.21.15.76:0/2187373115 >> 172.21.15.76:0/2187373115 conn(0x7f300c0b0870 msgr2=0x7f300c0ce600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:49.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.474+0000 7f3012c5d640 1 -- 172.21.15.76:0/2187373115 shutdown_connections 2023-11-09T03:51:49.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:49.474+0000 7f3012c5d640 1 -- 172.21.15.76:0/2187373115 wait complete. 2023-11-09T03:51:49.485 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:reconnect 2023-11-09T03:51:50.486 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:51:50.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.595+0000 7f1b5cc44640 1 Processor -- start 2023-11-09T03:51:50.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.595+0000 7f1b5cc44640 1 -- start start 2023-11-09T03:51:50.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.596+0000 7f1b5cc44640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1b5815d4e0 0x7f1b580d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:50.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.596+0000 7f1b5cc44640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b580d3b60 0x7f1b580d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:50.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.596+0000 7f1b5cc44640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b580b32b0 con 0x7f1b5815d100 2023-11-09T03:51:50.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.596+0000 7f1b5cc44640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b580cec20 con 0x7f1b5815d4e0 2023-11-09T03:51:50.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.596+0000 7f1b5cc44640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b580cee10 con 0x7f1b580d3b60 2023-11-09T03:51:50.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.596+0000 7f1b5659c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f1b5815d100 0x7f1b580ce510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46664/0 (socket says 172.21.15.76:46664) 2023-11-09T03:51:50.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.596+0000 7f1b5659c640 1 -- 172.21.15.76:0/3726209315 learned_addr learned my addr 172.21.15.76:0/3726209315 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:51:50.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.597+0000 7f1b55d9b640 1 --2- 172.21.15.76:0/3726209315 >> v2:172.21.15.113:3300/0 conn(0x7f1b5815d4e0 0x7f1b580d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:50.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.597+0000 7f1b56d9d640 1 --2- 172.21.15.76:0/3726209315 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b580d3b60 0x7f1b580d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:50.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.597+0000 7f1b5559a640 1 -- 172.21.15.76:0/3726209315 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3858934604 0 0) 0x7f1b580b32b0 con 0x7f1b5815d100 2023-11-09T03:51:50.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.597+0000 7f1b5559a640 1 -- 172.21.15.76:0/3726209315 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b40003520 con 0x7f1b5815d100 2023-11-09T03:51:50.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.597+0000 7f1b5559a640 1 -- 172.21.15.76:0/3726209315 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 524982229 0 0) 0x7f1b40003520 con 0x7f1b5815d100 2023-11-09T03:51:50.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.597+0000 7f1b5559a640 1 -- 172.21.15.76:0/3726209315 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b580d3b60 msgr2=0x7f1b580d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:50.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.597+0000 7f1b5559a640 1 --2- 172.21.15.76:0/3726209315 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b580d3b60 0x7f1b580d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:50.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.597+0000 7f1b5559a640 1 -- 172.21.15.76:0/3726209315 >> v2:172.21.15.113:3300/0 conn(0x7f1b5815d4e0 msgr2=0x7f1b580d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:50.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.597+0000 7f1b5559a640 1 --2- 172.21.15.76:0/3726209315 >> v2:172.21.15.113:3300/0 conn(0x7f1b5815d4e0 0x7f1b580d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:50.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.597+0000 7f1b5559a640 1 -- 172.21.15.76:0/3726209315 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b580b4750 con 0x7f1b5815d100 2023-11-09T03:51:50.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.597+0000 7f1b5559a640 1 -- 172.21.15.76:0/3726209315 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f1b48001b70 con 0x7f1b5815d100 2023-11-09T03:51:50.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.597+0000 7f1b5559a640 1 -- 172.21.15.76:0/3726209315 >> v1:172.21.15.76:6789/0 conn(0x7f1b5815d100 legacy=0x7f1b580ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:50.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.597+0000 7f1b5559a640 1 --2- 172.21.15.76:0/3726209315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b40003970 0x7f1b40003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:50.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.597+0000 7f1b5559a640 1 --2- 172.21.15.76:0/3726209315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b400047c0 0x7f1b40004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:50.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.597+0000 7f1b5559a640 1 --2- 172.21.15.76:0/3726209315 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b580d3b60 0x7f1b40005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:50.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.597+0000 7f1b5559a640 1 -- 172.21.15.76:0/3726209315 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b580cee10 con 0x7f1b400047c0 2023-11-09T03:51:50.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.597+0000 7f1b5559a640 1 -- 172.21.15.76:0/3726209315 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4000b980 con 0x7f1b40003970 2023-11-09T03:51:50.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.597+0000 7f1b5559a640 1 -- 172.21.15.76:0/3726209315 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4000bac0 con 0x7f1b580d3b60 2023-11-09T03:51:50.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.597+0000 7f1b55d9b640 1 --2- 172.21.15.76:0/3726209315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b400047c0 0x7f1b40004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:50.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.597+0000 7f1b56d9d640 1 --2- 172.21.15.76:0/3726209315 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b580d3b60 0x7f1b40005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:50.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.597+0000 7f1b5659c640 1 --2- 172.21.15.76:0/3726209315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b40003970 0x7f1b40003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:50.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.597+0000 7f1b55d9b640 1 -- 172.21.15.76:0/3726209315 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b580d3b60 msgr2=0x7f1b40005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:50.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.597+0000 7f1b55d9b640 1 --2- 172.21.15.76:0/3726209315 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b580d3b60 0x7f1b40005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:50.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.597+0000 7f1b55d9b640 1 -- 172.21.15.76:0/3726209315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b40003970 msgr2=0x7f1b40003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:50.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.597+0000 7f1b55d9b640 1 --2- 172.21.15.76:0/3726209315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b40003970 0x7f1b40003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:50.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.597+0000 7f1b55d9b640 1 -- 172.21.15.76:0/3726209315 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b40003520 con 0x7f1b400047c0 2023-11-09T03:51:50.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.597+0000 7f1b5659c640 1 --2- 172.21.15.76:0/3726209315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b40003970 0x7f1b40003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:51:50.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.598+0000 7f1b55d9b640 1 --2- 172.21.15.76:0/3726209315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b400047c0 0x7f1b40004c40 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009a90 tx=0x7f1b400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1512c3ecb1dca49 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:50.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.598+0000 7f1b5559a640 1 -- 172.21.15.76:0/3726209315 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c02c6b0 con 0x7f1b400047c0 2023-11-09T03:51:50.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.598+0000 7f1b5559a640 1 -- 172.21.15.76:0/3726209315 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4c02c810 con 0x7f1b400047c0 2023-11-09T03:51:50.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.598+0000 7f1b5559a640 1 -- 172.21.15.76:0/3726209315 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c02cba0 con 0x7f1b400047c0 2023-11-09T03:51:50.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.599+0000 7f1b5cc44640 1 -- 172.21.15.76:0/3726209315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b400047c0 msgr2=0x7f1b40004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:50.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.599+0000 7f1b5cc44640 1 --2- 172.21.15.76:0/3726209315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b400047c0 0x7f1b40004c40 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009a90 tx=0x7f1b400057f0 comp rx=0 tx=0).stop 2023-11-09T03:51:50.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.599+0000 7f1b56d9d640 1 -- 172.21.15.76:0/3726209315 reap_dead start 2023-11-09T03:51:50.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.599+0000 7f1b5cc44640 1 -- 172.21.15.76:0/3726209315 shutdown_connections 2023-11-09T03:51:50.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.599+0000 7f1b5cc44640 1 -- 172.21.15.76:0/3726209315 >> 172.21.15.76:0/3726209315 conn(0x7f1b580d2ef0 msgr2=0x7f1b580d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:50.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.599+0000 7f1b5cc44640 1 -- 172.21.15.76:0/3726209315 shutdown_connections 2023-11-09T03:51:50.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.599+0000 7f1b5cc44640 1 -- 172.21.15.76:0/3726209315 wait complete. 2023-11-09T03:51:50.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.600+0000 7f1b5cc44640 1 Processor -- start 2023-11-09T03:51:50.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.600+0000 7f1b5cc44640 1 -- start start 2023-11-09T03:51:50.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.601+0000 7f1b5cc44640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b400047c0 0x7f1b580b8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:50.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.601+0000 7f1b5cc44640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b5815d940 0x7f1b5815ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:50.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.601+0000 7f1b5cc44640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b5815e300 0x7f1b580b6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:50.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.601+0000 7f1b5cc44640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b580c9320 con 0x7f1b5815e300 2023-11-09T03:51:50.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.601+0000 7f1b5cc44640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5815e780 con 0x7f1b5815d940 2023-11-09T03:51:50.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.601+0000 7f1b5cc44640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b580b5bb0 con 0x7f1b400047c0 2023-11-09T03:51:50.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.601+0000 7f1b56d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b5815e300 0x7f1b580b6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:50.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.601+0000 7f1b56d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b5815e300 0x7f1b580b6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49194/0 (socket says 172.21.15.76:49194) 2023-11-09T03:51:50.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.601+0000 7f1b56d9d640 1 -- 172.21.15.76:0/3683042944 learned_addr learned my addr 172.21.15.76:0/3683042944 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:50.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.601+0000 7f1b5659c640 1 --2- 172.21.15.76:0/3683042944 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b400047c0 0x7f1b580b8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:50.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.601+0000 7f1b55d9b640 1 --2- 172.21.15.76:0/3683042944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b5815d940 0x7f1b5815ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:50.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.601+0000 7f1b56d9d640 1 -- 172.21.15.76:0/3683042944 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b400047c0 msgr2=0x7f1b580b8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:50.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.601+0000 7f1b56d9d640 1 --2- 172.21.15.76:0/3683042944 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b400047c0 0x7f1b580b8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:50.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.601+0000 7f1b56d9d640 1 -- 172.21.15.76:0/3683042944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b5815d940 msgr2=0x7f1b5815ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:50.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.601+0000 7f1b56d9d640 1 --2- 172.21.15.76:0/3683042944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b5815d940 0x7f1b5815ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:50.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.601+0000 7f1b56d9d640 1 -- 172.21.15.76:0/3683042944 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b580b5e90 con 0x7f1b5815e300 2023-11-09T03:51:50.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.601+0000 7f1b55d9b640 1 --2- 172.21.15.76:0/3683042944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b5815d940 0x7f1b5815ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:51:50.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.602+0000 7f1b56d9d640 1 --2- 172.21.15.76:0/3683042944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b5815e300 0x7f1b580b6970 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f1b5808c730 tx=0x7f1b50011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:50.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.602+0000 7f1b46ffd640 1 -- 172.21.15.76:0/3683042944 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b50011ee0 con 0x7f1b5815e300 2023-11-09T03:51:50.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.602+0000 7f1b46ffd640 1 -- 172.21.15.76:0/3683042944 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b50004240 con 0x7f1b5815e300 2023-11-09T03:51:50.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.602+0000 7f1b5cc44640 1 -- 172.21.15.76:0/3683042944 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b580b6170 con 0x7f1b5815e300 2023-11-09T03:51:50.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.602+0000 7f1b46ffd640 1 -- 172.21.15.76:0/3683042944 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5000f7a0 con 0x7f1b5815e300 2023-11-09T03:51:50.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.602+0000 7f1b5cc44640 1 -- 172.21.15.76:0/3683042944 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b58152770 con 0x7f1b5815e300 2023-11-09T03:51:50.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.603+0000 7f1b5cc44640 1 -- 172.21.15.76:0/3683042944 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b580b6690 con 0x7f1b5815e300 2023-11-09T03:51:50.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.603+0000 7f1b46ffd640 1 -- 172.21.15.76:0/3683042944 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1b5000f900 con 0x7f1b5815e300 2023-11-09T03:51:50.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.603+0000 7f1b46ffd640 1 --2- 172.21.15.76:0/3683042944 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1b3806ae70 0x7f1b3806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:50.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.603+0000 7f1b46ffd640 1 -- 172.21.15.76:0/3683042944 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7737+0+0 (secure 0 0 0) 0x7f1b50089f30 con 0x7f1b5815e300 2023-11-09T03:51:50.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.604+0000 7f1b5659c640 1 --2- 172.21.15.76:0/3683042944 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1b3806ae70 0x7f1b3806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:50.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.604+0000 7f1b5659c640 1 --2- 172.21.15.76:0/3683042944 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1b3806ae70 0x7f1b3806d330 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1b48002230 tx=0x7f1b48009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:50.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.608+0000 7f1b46ffd640 1 -- 172.21.15.76:0/3683042944 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1b580b6690 con 0x7f1b5815e300 2023-11-09T03:51:50.625 INFO:tasks.ceph.mds.b.smithi113.stderr:2023-11-09T03:51:50.626+0000 7f0f54993640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:51:50.625 INFO:tasks.ceph.mds.b.smithi113.stderr:2023-11-09T03:51:50.626+0000 7f0f54993640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-09T03:51:50.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.808+0000 7f1b5cc44640 1 -- 172.21.15.76:0/3683042944 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b580b6690 con 0x7f1b5815e300 2023-11-09T03:51:50.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.809+0000 7f1b46ffd640 1 -- 172.21.15.76:0/3683042944 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4929 (secure 0 0 0) 0x7f1b580b6690 con 0x7f1b5815e300 2023-11-09T03:51:50.808 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:50.809 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":93,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8310,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:51:49.482369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6904,"mds_1":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6904":{"gid":6904,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":16,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8274":{"gid":8274,"name":"c","rank":1,"incarnation":89,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6833/1536527256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1536527256},{"type":"v1","addr":"172.21.15.76:6833","nonce":1536527256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:51:50.809 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 93 2023-11-09T03:51:50.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.811+0000 7f1b5cc44640 1 -- 172.21.15.76:0/3683042944 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1b3806ae70 msgr2=0x7f1b3806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:50.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.811+0000 7f1b5cc44640 1 --2- 172.21.15.76:0/3683042944 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1b3806ae70 0x7f1b3806d330 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1b48002230 tx=0x7f1b48009040 comp rx=0 tx=0).stop 2023-11-09T03:51:50.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.811+0000 7f1b5cc44640 1 -- 172.21.15.76:0/3683042944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b5815e300 msgr2=0x7f1b580b6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:50.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.811+0000 7f1b5cc44640 1 --2- 172.21.15.76:0/3683042944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b5815e300 0x7f1b580b6970 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f1b5808c730 tx=0x7f1b50011cf0 comp rx=0 tx=0).stop 2023-11-09T03:51:50.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.811+0000 7f1b5cc44640 1 -- 172.21.15.76:0/3683042944 shutdown_connections 2023-11-09T03:51:50.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.811+0000 7f1b5cc44640 1 --2- 172.21.15.76:0/3683042944 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1b3806ae70 0x7f1b3806d330 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:50.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.811+0000 7f1b5cc44640 1 --2- 172.21.15.76:0/3683042944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b5815e300 0x7f1b580b6970 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:50.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.811+0000 7f1b5cc44640 1 --2- 172.21.15.76:0/3683042944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b5815d940 0x7f1b5815ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:50.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.811+0000 7f1b5cc44640 1 --2- 172.21.15.76:0/3683042944 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b400047c0 0x7f1b580b8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:50.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.811+0000 7f1b5cc44640 1 -- 172.21.15.76:0/3683042944 >> 172.21.15.76:0/3683042944 conn(0x7f1b580d2ef0 msgr2=0x7f1b580d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:50.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.811+0000 7f1b5cc44640 1 -- 172.21.15.76:0/3683042944 shutdown_connections 2023-11-09T03:51:50.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.811+0000 7f1b5cc44640 1 -- 172.21.15.76:0/3683042944 wait complete. 2023-11-09T03:51:50.821 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:active 2023-11-09T03:51:50.821 DEBUG:tasks.cephfs.filesystem:reached state 'up:active' in 5.689468860626221s 2023-11-09T03:51:50.822 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:51:50.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.923+0000 7f5dcb9b0640 1 Processor -- start 2023-11-09T03:51:50.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.923+0000 7f5dcb9b0640 1 -- start start 2023-11-09T03:51:50.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.924+0000 7f5dcb9b0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5dc40ade00 0x7f5dc40b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:50.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.924+0000 7f5dcb9b0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5dc406c5f0 0x7f5dc40cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:50.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.924+0000 7f5dcb9b0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc406cc90 con 0x7f5dc406d9c0 2023-11-09T03:51:50.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.924+0000 7f5dcb9b0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc40b0e60 con 0x7f5dc40ade00 2023-11-09T03:51:50.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.924+0000 7f5dcb9b0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc406cee0 con 0x7f5dc406c5f0 2023-11-09T03:51:50.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.924+0000 7f5dc974c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5dc40ade00 0x7f5dc40b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:50.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.924+0000 7f5dc8f4b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f5dc406d9c0 0x7f5dc40710d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46668/0 (socket says 172.21.15.76:46668) 2023-11-09T03:51:50.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.924+0000 7f5dc8f4b640 1 -- 172.21.15.76:0/1191766957 learned_addr learned my addr 172.21.15.76:0/1191766957 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:51:50.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.924+0000 7f5dc9f4d640 1 --2- 172.21.15.76:0/1191766957 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5dc406c5f0 0x7f5dc40cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:50.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.924+0000 7f5dc974c640 1 -- 172.21.15.76:0/1191766957 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5dc406c5f0 msgr2=0x7f5dc40cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:50.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.924+0000 7f5dc974c640 1 --2- 172.21.15.76:0/1191766957 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5dc406c5f0 0x7f5dc40cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:50.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.924+0000 7f5dc974c640 1 -- 172.21.15.76:0/1191766957 >> v1:172.21.15.76:6789/0 conn(0x7f5dc406d9c0 legacy=0x7f5dc40710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:50.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.924+0000 7f5dbbfff640 1 -- 172.21.15.76:0/1191766957 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1896976257 0 0) 0x7f5dc406cc90 con 0x7f5dc406d9c0 2023-11-09T03:51:50.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.924+0000 7f5dc974c640 1 -- 172.21.15.76:0/1191766957 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc406d1c0 con 0x7f5dc40ade00 2023-11-09T03:51:50.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.924+0000 7f5dc9f4d640 1 --2- 172.21.15.76:0/1191766957 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5dc406c5f0 0x7f5dc40cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:51:50.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.925+0000 7f5dc974c640 1 --2- 172.21.15.76:0/1191766957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dc40ade00 0x7f5dc40b0530 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00097a0 tx=0x7f5dc002b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94af5f16b897e28 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:50.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.925+0000 7f5dbbfff640 1 -- 172.21.15.76:0/1191766957 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0031710 con 0x7f5dc40ade00 2023-11-09T03:51:50.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.925+0000 7f5dbbfff640 1 -- 172.21.15.76:0/1191766957 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc002f050 con 0x7f5dc40ade00 2023-11-09T03:51:50.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.925+0000 7f5dbbfff640 1 -- 172.21.15.76:0/1191766957 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0031a90 con 0x7f5dc40ade00 2023-11-09T03:51:50.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.925+0000 7f5dcb9b0640 1 -- 172.21.15.76:0/1191766957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dc40ade00 msgr2=0x7f5dc40b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:50.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.925+0000 7f5dcb9b0640 1 --2- 172.21.15.76:0/1191766957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dc40ade00 0x7f5dc40b0530 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00097a0 tx=0x7f5dc002b7b0 comp rx=0 tx=0).stop 2023-11-09T03:51:50.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.925+0000 7f5dcb9b0640 1 -- 172.21.15.76:0/1191766957 shutdown_connections 2023-11-09T03:51:50.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.925+0000 7f5dcb9b0640 1 --2- 172.21.15.76:0/1191766957 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5dc406c5f0 0x7f5dc40cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:50.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.925+0000 7f5dcb9b0640 1 --2- 172.21.15.76:0/1191766957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dc40ade00 0x7f5dc40b0530 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:50.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.925+0000 7f5dcb9b0640 1 -- 172.21.15.76:0/1191766957 >> 172.21.15.76:0/1191766957 conn(0x7f5dc40cb530 msgr2=0x7f5dc40cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:50.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.926+0000 7f5dcb9b0640 1 -- 172.21.15.76:0/1191766957 shutdown_connections 2023-11-09T03:51:50.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.926+0000 7f5dcb9b0640 1 -- 172.21.15.76:0/1191766957 wait complete. 2023-11-09T03:51:50.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.927+0000 7f5dcb9b0640 1 Processor -- start 2023-11-09T03:51:50.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.927+0000 7f5dcb9b0640 1 -- start start 2023-11-09T03:51:50.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.927+0000 7f5dcb9b0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dc406c5f0 0x7f5dc40a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:50.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.927+0000 7f5dcb9b0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5dc406d9c0 0x7f5dc40aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:50.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.927+0000 7f5dcb9b0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5dc40ade00 0x7f5dc40c32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:50.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.927+0000 7f5dcb9b0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40c39c0 con 0x7f5dc406c5f0 2023-11-09T03:51:50.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.927+0000 7f5dcb9b0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40c37f0 con 0x7f5dc40ade00 2023-11-09T03:51:50.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.927+0000 7f5dcb9b0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40a94d0 con 0x7f5dc406d9c0 2023-11-09T03:51:50.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.927+0000 7f5dc974c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dc406c5f0 0x7f5dc40a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:50.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.928+0000 7f5dc974c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dc406c5f0 0x7f5dc40a8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49210/0 (socket says 172.21.15.76:49210) 2023-11-09T03:51:50.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.928+0000 7f5dc974c640 1 -- 172.21.15.76:0/507111202 learned_addr learned my addr 172.21.15.76:0/507111202 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:50.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.928+0000 7f5dc8f4b640 1 --2- 172.21.15.76:0/507111202 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5dc406d9c0 0x7f5dc40aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:50.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.928+0000 7f5dc9f4d640 1 --2- 172.21.15.76:0/507111202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5dc40ade00 0x7f5dc40c32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:50.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.928+0000 7f5dc974c640 1 -- 172.21.15.76:0/507111202 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5dc406d9c0 msgr2=0x7f5dc40aabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:50.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.928+0000 7f5dc974c640 1 --2- 172.21.15.76:0/507111202 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5dc406d9c0 0x7f5dc40aabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:50.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.928+0000 7f5dc974c640 1 -- 172.21.15.76:0/507111202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5dc40ade00 msgr2=0x7f5dc40c32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:50.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.928+0000 7f5dc974c640 1 --2- 172.21.15.76:0/507111202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5dc40ade00 0x7f5dc40c32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:50.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.928+0000 7f5dc974c640 1 -- 172.21.15.76:0/507111202 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc0009400 con 0x7f5dc406c5f0 2023-11-09T03:51:50.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.928+0000 7f5dc9f4d640 1 --2- 172.21.15.76:0/507111202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5dc40ade00 0x7f5dc40c32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:51:50.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.928+0000 7f5dc974c640 1 --2- 172.21.15.76:0/507111202 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dc406c5f0 0x7f5dc40a8e40 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00098c0 tx=0x7f5dc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:50.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.928+0000 7f5db9ffb640 1 -- 172.21.15.76:0/507111202 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0033a50 con 0x7f5dc406c5f0 2023-11-09T03:51:50.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.928+0000 7f5db9ffb640 1 -- 172.21.15.76:0/507111202 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc0033bb0 con 0x7f5dc406c5f0 2023-11-09T03:51:50.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.928+0000 7f5db9ffb640 1 -- 172.21.15.76:0/507111202 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc003c400 con 0x7f5dc406c5f0 2023-11-09T03:51:50.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.928+0000 7f5dcb9b0640 1 -- 172.21.15.76:0/507111202 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc40c3c10 con 0x7f5dc406c5f0 2023-11-09T03:51:50.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.928+0000 7f5dcb9b0640 1 -- 172.21.15.76:0/507111202 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc40c41d0 con 0x7f5dc406c5f0 2023-11-09T03:51:50.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.929+0000 7f5dcb9b0640 1 -- 172.21.15.76:0/507111202 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc40c43c0 con 0x7f5dc406c5f0 2023-11-09T03:51:50.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.930+0000 7f5db9ffb640 1 -- 172.21.15.76:0/507111202 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5dc003a040 con 0x7f5dc406c5f0 2023-11-09T03:51:50.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.930+0000 7f5db9ffb640 1 --2- 172.21.15.76:0/507111202 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5da806ae70 0x7f5da806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:50.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.930+0000 7f5db9ffb640 1 -- 172.21.15.76:0/507111202 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7737+0+0 (secure 0 0 0) 0x7f5dc00ab3a0 con 0x7f5dc406c5f0 2023-11-09T03:51:50.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.930+0000 7f5dc8f4b640 1 --2- 172.21.15.76:0/507111202 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5da806ae70 0x7f5da806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:50.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.931+0000 7f5dc8f4b640 1 --2- 172.21.15.76:0/507111202 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5da806ae70 0x7f5da806d330 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40c22b0 tx=0x7f5db4007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:50.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:50.934+0000 7f5db9ffb640 1 -- 172.21.15.76:0/507111202 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5dc0075cf0 con 0x7f5dc406c5f0 2023-11-09T03:51:51.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.133+0000 7f5dcb9b0640 1 -- 172.21.15.76:0/507111202 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5dc40684b0 con 0x7f5dc406c5f0 2023-11-09T03:51:51.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.134+0000 7f5db9ffb640 1 -- 172.21.15.76:0/507111202 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4929 (secure 0 0 0) 0x7f5dc0079310 con 0x7f5dc406c5f0 2023-11-09T03:51:51.134 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:51.134 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":93,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8310,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:51:49.482369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6904,"mds_1":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6904":{"gid":6904,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":16,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8274":{"gid":8274,"name":"c","rank":1,"incarnation":89,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6833/1536527256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1536527256},{"type":"v1","addr":"172.21.15.76:6833","nonce":1536527256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:51:51.134 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 93 2023-11-09T03:51:51.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.136+0000 7f5dcb9b0640 1 -- 172.21.15.76:0/507111202 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5da806ae70 msgr2=0x7f5da806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:51.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.136+0000 7f5dcb9b0640 1 --2- 172.21.15.76:0/507111202 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5da806ae70 0x7f5da806d330 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40c22b0 tx=0x7f5db4007de0 comp rx=0 tx=0).stop 2023-11-09T03:51:51.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.136+0000 7f5dcb9b0640 1 -- 172.21.15.76:0/507111202 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dc406c5f0 msgr2=0x7f5dc40a8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:51.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.136+0000 7f5dcb9b0640 1 --2- 172.21.15.76:0/507111202 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dc406c5f0 0x7f5dc40a8e40 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00098c0 tx=0x7f5dc00046c0 comp rx=0 tx=0).stop 2023-11-09T03:51:51.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.136+0000 7f5dcb9b0640 1 -- 172.21.15.76:0/507111202 shutdown_connections 2023-11-09T03:51:51.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.136+0000 7f5dcb9b0640 1 --2- 172.21.15.76:0/507111202 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5da806ae70 0x7f5da806d330 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:51.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.136+0000 7f5dcb9b0640 1 --2- 172.21.15.76:0/507111202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5dc40ade00 0x7f5dc40c32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:51.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.136+0000 7f5dcb9b0640 1 --2- 172.21.15.76:0/507111202 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5dc406d9c0 0x7f5dc40aabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:51.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.136+0000 7f5dcb9b0640 1 --2- 172.21.15.76:0/507111202 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dc406c5f0 0x7f5dc40a8e40 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:51.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.136+0000 7f5dcb9b0640 1 -- 172.21.15.76:0/507111202 >> 172.21.15.76:0/507111202 conn(0x7f5dc40cb530 msgr2=0x7f5dc40b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:51.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.136+0000 7f5dcb9b0640 1 -- 172.21.15.76:0/507111202 shutdown_connections 2023-11-09T03:51:51.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.136+0000 7f5dcb9b0640 1 -- 172.21.15.76:0/507111202 wait complete. 2023-11-09T03:51:51.146 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump snaps --server 2023-11-09T03:51:51.270 INFO:teuthology.orchestra.run.smithi113.stdout:{"last_snap":10,"last_created":5,"last_destroyed":9,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-09T03:51:51.271 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 5, "last_destroyed": 9, "last_snap": 10, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [] } 2023-11-09T03:51:51.271 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-09T03:51:51.271 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:51:51.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.371+0000 7f3f78290640 1 Processor -- start 2023-11-09T03:51:51.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.371+0000 7f3f78290640 1 -- start start 2023-11-09T03:51:51.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.372+0000 7f3f78290640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3f70159e40 0x7f3f700b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:51.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.372+0000 7f3f78290640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f7006bb60 0x7f3f700b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:51.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.372+0000 7f3f78290640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3f7006fff0 0x7f3f70070480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:51.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.372+0000 7f3f78290640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f70071500 con 0x7f3f7006fff0 2023-11-09T03:51:51.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.372+0000 7f3f78290640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f7015dae0 con 0x7f3f70159e40 2023-11-09T03:51:51.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.372+0000 7f3f78290640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f700709c0 con 0x7f3f7006bb60 2023-11-09T03:51:51.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.372+0000 7f3f7682d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3f7006fff0 0x7f3f70070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:51.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.372+0000 7f3f7682d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3f7006fff0 0x7f3f70070480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49214/0 (socket says 172.21.15.76:49214) 2023-11-09T03:51:51.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.372+0000 7f3f7682d640 1 -- 172.21.15.76:0/162240889 learned_addr learned my addr 172.21.15.76:0/162240889 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:51.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.373+0000 7f3f7602c640 1 --2- 172.21.15.76:0/162240889 >> v2:172.21.15.113:3300/0 conn(0x7f3f70159e40 0x7f3f700b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:51.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.373+0000 7f3f7582b640 1 --2- 172.21.15.76:0/162240889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f7006bb60 0x7f3f700b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:51.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.373+0000 7f3f7682d640 1 -- 172.21.15.76:0/162240889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f7006bb60 msgr2=0x7f3f700b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:51.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.373+0000 7f3f7682d640 1 --2- 172.21.15.76:0/162240889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f7006bb60 0x7f3f700b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:51.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.373+0000 7f3f7682d640 1 -- 172.21.15.76:0/162240889 >> v2:172.21.15.113:3300/0 conn(0x7f3f70159e40 msgr2=0x7f3f700b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:51.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.373+0000 7f3f7682d640 1 --2- 172.21.15.76:0/162240889 >> v2:172.21.15.113:3300/0 conn(0x7f3f70159e40 0x7f3f700b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:51.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.373+0000 7f3f7682d640 1 -- 172.21.15.76:0/162240889 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f70070c60 con 0x7f3f7006fff0 2023-11-09T03:51:51.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.373+0000 7f3f7602c640 1 --2- 172.21.15.76:0/162240889 >> v2:172.21.15.113:3300/0 conn(0x7f3f70159e40 0x7f3f700b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:51:51.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.373+0000 7f3f7582b640 1 --2- 172.21.15.76:0/162240889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f7006bb60 0x7f3f700b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:51:51.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.373+0000 7f3f7682d640 1 --2- 172.21.15.76:0/162240889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f7006fff0 0x7f3f70070480 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f3f700ce7f0 tx=0x7f3f6802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0267a629bfab2e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:51.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.374+0000 7f3f7502a640 1 -- 172.21.15.76:0/162240889 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f680317e0 con 0x7f3f7006fff0 2023-11-09T03:51:51.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.374+0000 7f3f7502a640 1 -- 172.21.15.76:0/162240889 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f68031940 con 0x7f3f7006fff0 2023-11-09T03:51:51.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.374+0000 7f3f7502a640 1 -- 172.21.15.76:0/162240889 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f68031cd0 con 0x7f3f7006fff0 2023-11-09T03:51:51.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.374+0000 7f3f78290640 1 -- 172.21.15.76:0/162240889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f7006fff0 msgr2=0x7f3f70070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:51.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.374+0000 7f3f78290640 1 --2- 172.21.15.76:0/162240889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f7006fff0 0x7f3f70070480 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f3f700ce7f0 tx=0x7f3f6802b880 comp rx=0 tx=0).stop 2023-11-09T03:51:51.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.374+0000 7f3f78290640 1 -- 172.21.15.76:0/162240889 shutdown_connections 2023-11-09T03:51:51.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.374+0000 7f3f78290640 1 --2- 172.21.15.76:0/162240889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f7006fff0 0x7f3f70070480 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:51.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.374+0000 7f3f78290640 1 --2- 172.21.15.76:0/162240889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f7006bb60 0x7f3f700b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:51.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.374+0000 7f3f78290640 1 --2- 172.21.15.76:0/162240889 >> v2:172.21.15.113:3300/0 conn(0x7f3f70159e40 0x7f3f700b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:51.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.374+0000 7f3f78290640 1 -- 172.21.15.76:0/162240889 >> 172.21.15.76:0/162240889 conn(0x7f3f700cb530 msgr2=0x7f3f700cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:51.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.374+0000 7f3f78290640 1 -- 172.21.15.76:0/162240889 shutdown_connections 2023-11-09T03:51:51.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.374+0000 7f3f78290640 1 -- 172.21.15.76:0/162240889 wait complete. 2023-11-09T03:51:51.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.375+0000 7f3f78290640 1 Processor -- start 2023-11-09T03:51:51.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.375+0000 7f3f78290640 1 -- start start 2023-11-09T03:51:51.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.376+0000 7f3f78290640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f7006bb60 0x7f3f7014e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:51.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.376+0000 7f3f78290640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f7006fff0 0x7f3f701f1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:51.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.376+0000 7f3f78290640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f70159e40 0x7f3f701f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:51.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.376+0000 7f3f78290640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f701f4e50 con 0x7f3f7006bb60 2023-11-09T03:51:51.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.376+0000 7f3f78290640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f701f4890 con 0x7f3f70159e40 2023-11-09T03:51:51.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.376+0000 7f3f78290640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f701f4b70 con 0x7f3f7006fff0 2023-11-09T03:51:51.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.376+0000 7f3f7602c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f7006bb60 0x7f3f7014e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:51.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.376+0000 7f3f7602c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f7006bb60 0x7f3f7014e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49216/0 (socket says 172.21.15.76:49216) 2023-11-09T03:51:51.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.376+0000 7f3f7602c640 1 -- 172.21.15.76:0/67046523 learned_addr learned my addr 172.21.15.76:0/67046523 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:51.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.376+0000 7f3f7582b640 1 --2- 172.21.15.76:0/67046523 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f7006fff0 0x7f3f701f1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:51.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.376+0000 7f3f7682d640 1 --2- 172.21.15.76:0/67046523 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f70159e40 0x7f3f701f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:51.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.376+0000 7f3f7602c640 1 -- 172.21.15.76:0/67046523 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f7006fff0 msgr2=0x7f3f701f1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:51.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.376+0000 7f3f7602c640 1 --2- 172.21.15.76:0/67046523 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f7006fff0 0x7f3f701f1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:51.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.376+0000 7f3f7602c640 1 -- 172.21.15.76:0/67046523 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f70159e40 msgr2=0x7f3f701f3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:51.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.376+0000 7f3f7602c640 1 --2- 172.21.15.76:0/67046523 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f70159e40 0x7f3f701f3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:51.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.376+0000 7f3f7602c640 1 -- 172.21.15.76:0/67046523 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f680094d0 con 0x7f3f7006bb60 2023-11-09T03:51:51.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.376+0000 7f3f7582b640 1 --2- 172.21.15.76:0/67046523 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f7006fff0 0x7f3f701f1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:51:51.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.376+0000 7f3f7682d640 1 --2- 172.21.15.76:0/67046523 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f70159e40 0x7f3f701f3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:51:51.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.377+0000 7f3f7602c640 1 --2- 172.21.15.76:0/67046523 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f7006bb60 0x7f3f7014e1d0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00cf60 tx=0x7f3f6c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:51.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.377+0000 7f3f66ffd640 1 -- 172.21.15.76:0/67046523 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c00d840 con 0x7f3f7006bb60 2023-11-09T03:51:51.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.377+0000 7f3f66ffd640 1 -- 172.21.15.76:0/67046523 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6c00d9a0 con 0x7f3f7006bb60 2023-11-09T03:51:51.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.377+0000 7f3f66ffd640 1 -- 172.21.15.76:0/67046523 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c00dd30 con 0x7f3f7006bb60 2023-11-09T03:51:51.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.377+0000 7f3f78290640 1 -- 172.21.15.76:0/67046523 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f7014ec50 con 0x7f3f7006bb60 2023-11-09T03:51:51.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.377+0000 7f3f78290640 1 -- 172.21.15.76:0/67046523 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f701f23d0 con 0x7f3f7006bb60 2023-11-09T03:51:51.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.378+0000 7f3f78290640 1 -- 172.21.15.76:0/67046523 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f700684b0 con 0x7f3f7006bb60 2023-11-09T03:51:51.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.378+0000 7f3f66ffd640 1 -- 172.21.15.76:0/67046523 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3f6c015440 con 0x7f3f7006bb60 2023-11-09T03:51:51.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.378+0000 7f3f66ffd640 1 --2- 172.21.15.76:0/67046523 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3f5406ae70 0x7f3f5406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:51.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.379+0000 7f3f66ffd640 1 -- 172.21.15.76:0/67046523 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7737+0+0 (secure 0 0 0) 0x7f3f6c08a730 con 0x7f3f7006bb60 2023-11-09T03:51:51.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.379+0000 7f3f7582b640 1 --2- 172.21.15.76:0/67046523 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3f5406ae70 0x7f3f5406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:51.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.379+0000 7f3f7582b640 1 --2- 172.21.15.76:0/67046523 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3f5406ae70 0x7f3f5406d330 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3f7015c6b0 tx=0x7f3f60008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:51.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.383+0000 7f3f66ffd640 1 -- 172.21.15.76:0/67046523 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3f6c055130 con 0x7f3f7006bb60 2023-11-09T03:51:51.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.585+0000 7f3f78290640 1 -- 172.21.15.76:0/67046523 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f700c9750 con 0x7f3f7006bb60 2023-11-09T03:51:51.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.586+0000 7f3f66ffd640 1 -- 172.21.15.76:0/67046523 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4929 (secure 0 0 0) 0x7f3f6c058750 con 0x7f3f7006bb60 2023-11-09T03:51:51.586 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:51.586 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":93,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8310,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:51:49.482369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6904,"mds_1":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6904":{"gid":6904,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":16,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8274":{"gid":8274,"name":"c","rank":1,"incarnation":89,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6833/1536527256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1536527256},{"type":"v1","addr":"172.21.15.76:6833","nonce":1536527256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:51:51.586 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 93 2023-11-09T03:51:51.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.588+0000 7f3f78290640 1 -- 172.21.15.76:0/67046523 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3f5406ae70 msgr2=0x7f3f5406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:51.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.588+0000 7f3f78290640 1 --2- 172.21.15.76:0/67046523 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3f5406ae70 0x7f3f5406d330 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3f7015c6b0 tx=0x7f3f60008040 comp rx=0 tx=0).stop 2023-11-09T03:51:51.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.588+0000 7f3f78290640 1 -- 172.21.15.76:0/67046523 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f7006bb60 msgr2=0x7f3f7014e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:51.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.588+0000 7f3f78290640 1 --2- 172.21.15.76:0/67046523 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f7006bb60 0x7f3f7014e1d0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00cf60 tx=0x7f3f6c00cf90 comp rx=0 tx=0).stop 2023-11-09T03:51:51.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.588+0000 7f3f78290640 1 -- 172.21.15.76:0/67046523 shutdown_connections 2023-11-09T03:51:51.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.588+0000 7f3f78290640 1 --2- 172.21.15.76:0/67046523 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3f5406ae70 0x7f3f5406d330 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:51.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.588+0000 7f3f78290640 1 --2- 172.21.15.76:0/67046523 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f70159e40 0x7f3f701f3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:51.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.588+0000 7f3f78290640 1 --2- 172.21.15.76:0/67046523 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f7006fff0 0x7f3f701f1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:51.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.588+0000 7f3f78290640 1 --2- 172.21.15.76:0/67046523 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f7006bb60 0x7f3f7014e1d0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:51.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.588+0000 7f3f78290640 1 -- 172.21.15.76:0/67046523 >> 172.21.15.76:0/67046523 conn(0x7f3f700cb530 msgr2=0x7f3f70158270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:51.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.588+0000 7f3f78290640 1 -- 172.21.15.76:0/67046523 shutdown_connections 2023-11-09T03:51:51.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:51.588+0000 7f3f78290640 1 -- 172.21.15.76:0/67046523 wait complete. 2023-11-09T03:51:51.598 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump snaps --server 2023-11-09T03:51:51.714 INFO:teuthology.orchestra.run.smithi113.stdout:{"last_snap":10,"last_created":5,"last_destroyed":9,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-09T03:51:51.714 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 5, "last_destroyed": 9, "last_snap": 10, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [] } 2023-11-09T03:51:51.715 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:51:51.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:51:51.716 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2023-11-09T03:51:56.659 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:51:56.659+0000 7fe0effff640 -1 client.7626 mds.1 rejected us (blocklisted (blacklisted)) 2023-11-09T03:51:56.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:51:56.660 INFO:teuthology.orchestra.run.smithi076.stderr:mkdir: cannot create directory ‘d1/dir/.snap/s22’: Operation not permitted 2023-11-09T03:51:56.661 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:51:56.661 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:51:56.679 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:51:56.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.833+0000 7f844ed4c640 1 Processor -- start 2023-11-09T03:51:56.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.833+0000 7f844ed4c640 1 -- start start 2023-11-09T03:51:56.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.834+0000 7f844ed4c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8448159e40 0x7f84480b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:56.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.834+0000 7f844ed4c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f844806bb60 0x7f84480b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:56.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.834+0000 7f844ed4c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8448071500 con 0x7f844806fff0 2023-11-09T03:51:56.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.834+0000 7f844ed4c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f844815dae0 con 0x7f8448159e40 2023-11-09T03:51:56.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.834+0000 7f844ed4c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8448070c60 con 0x7f844806bb60 2023-11-09T03:51:56.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.834+0000 7f844cae8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8448159e40 0x7f84480b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:56.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.834+0000 7f844cae8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8448159e40 0x7f84480b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41978/0 (socket says 172.21.15.76:41978) 2023-11-09T03:51:56.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.834+0000 7f844cae8640 1 -- 172.21.15.76:0/2527845468 learned_addr learned my addr 172.21.15.76:0/2527845468 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:56.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.834+0000 7f8447fff640 1 --2- 172.21.15.76:0/2527845468 >> v2:172.21.15.113:3300/0 conn(0x7f844806bb60 0x7f84480b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:56.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.834+0000 7f844cae8640 1 -- 172.21.15.76:0/2527845468 >> v2:172.21.15.113:3300/0 conn(0x7f844806bb60 msgr2=0x7f84480b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:56.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.834+0000 7f844cae8640 1 --2- 172.21.15.76:0/2527845468 >> v2:172.21.15.113:3300/0 conn(0x7f844806bb60 0x7f84480b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:56.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.834+0000 7f844cae8640 1 -- 172.21.15.76:0/2527845468 >> v1:172.21.15.113:6789/0 conn(0x7f844806fff0 legacy=0x7f8448070420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:56.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.834+0000 7f844cae8640 1 -- 172.21.15.76:0/2527845468 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8448070f40 con 0x7f8448159e40 2023-11-09T03:51:56.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.835+0000 7f844cae8640 1 --2- 172.21.15.76:0/2527845468 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8448159e40 0x7f84480b0530 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f844806fdc0 tx=0x7f843c02e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3709adc5dcb948fd server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:56.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.835+0000 7f84477fe640 1 -- 172.21.15.76:0/2527845468 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c02ee00 con 0x7f8448159e40 2023-11-09T03:51:56.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.835+0000 7f84477fe640 1 -- 172.21.15.76:0/2527845468 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843c036440 con 0x7f8448159e40 2023-11-09T03:51:56.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.835+0000 7f84477fe640 1 -- 172.21.15.76:0/2527845468 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c0367d0 con 0x7f8448159e40 2023-11-09T03:51:56.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.836+0000 7f844ed4c640 1 -- 172.21.15.76:0/2527845468 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8448159e40 msgr2=0x7f84480b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:56.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.836+0000 7f844ed4c640 1 --2- 172.21.15.76:0/2527845468 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8448159e40 0x7f84480b0530 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f844806fdc0 tx=0x7f843c02e050 comp rx=0 tx=0).stop 2023-11-09T03:51:56.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.836+0000 7f844ed4c640 1 -- 172.21.15.76:0/2527845468 shutdown_connections 2023-11-09T03:51:56.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.836+0000 7f844ed4c640 1 --2- 172.21.15.76:0/2527845468 >> v2:172.21.15.113:3300/0 conn(0x7f844806bb60 0x7f84480b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:56.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.836+0000 7f844ed4c640 1 --2- 172.21.15.76:0/2527845468 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8448159e40 0x7f84480b0530 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:56.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.836+0000 7f844ed4c640 1 -- 172.21.15.76:0/2527845468 >> 172.21.15.76:0/2527845468 conn(0x7f84480cb530 msgr2=0x7f84480cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:56.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.836+0000 7f844ed4c640 1 -- 172.21.15.76:0/2527845468 shutdown_connections 2023-11-09T03:51:56.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.836+0000 7f844ed4c640 1 -- 172.21.15.76:0/2527845468 wait complete. 2023-11-09T03:51:56.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.837+0000 7f844ed4c640 1 Processor -- start 2023-11-09T03:51:56.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.837+0000 7f844ed4c640 1 -- start start 2023-11-09T03:51:56.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.838+0000 7f844ed4c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f844806bb60 0x7f84480c1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:56.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.838+0000 7f844ed4c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f844806fff0 0x7f844815bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:56.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.838+0000 7f844ed4c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8448159e40 0x7f84480ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:56.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.838+0000 7f844ed4c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84480c6170 con 0x7f844806fff0 2023-11-09T03:51:56.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.838+0000 7f844ed4c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84480c5b80 con 0x7f8448159e40 2023-11-09T03:51:56.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.838+0000 7f844ed4c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f844815c950 con 0x7f844806bb60 2023-11-09T03:51:56.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.838+0000 7f8447fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f844806fff0 0x7f844815bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:56.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.838+0000 7f8447fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f844806fff0 0x7f844815bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41988/0 (socket says 172.21.15.76:41988) 2023-11-09T03:51:56.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.838+0000 7f8447fff640 1 -- 172.21.15.76:0/3435379955 learned_addr learned my addr 172.21.15.76:0/3435379955 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:56.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.838+0000 7f844cae8640 1 --2- 172.21.15.76:0/3435379955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f844806bb60 0x7f84480c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:56.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.838+0000 7f844d2e9640 1 --2- 172.21.15.76:0/3435379955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8448159e40 0x7f84480ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:56.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.838+0000 7f8447fff640 1 -- 172.21.15.76:0/3435379955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f844806bb60 msgr2=0x7f84480c1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:56.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.838+0000 7f8447fff640 1 --2- 172.21.15.76:0/3435379955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f844806bb60 0x7f84480c1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:56.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.838+0000 7f8447fff640 1 -- 172.21.15.76:0/3435379955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8448159e40 msgr2=0x7f84480ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:56.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.838+0000 7f8447fff640 1 --2- 172.21.15.76:0/3435379955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8448159e40 0x7f84480ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:56.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.838+0000 7f8447fff640 1 -- 172.21.15.76:0/3435379955 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c00b770 con 0x7f844806fff0 2023-11-09T03:51:56.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.838+0000 7f844cae8640 1 --2- 172.21.15.76:0/3435379955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f844806bb60 0x7f84480c1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:51:56.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.838+0000 7f844d2e9640 1 --2- 172.21.15.76:0/3435379955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8448159e40 0x7f84480ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:51:56.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.839+0000 7f8447fff640 1 --2- 172.21.15.76:0/3435379955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f844806fff0 0x7f844815bfb0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f843400e9b0 tx=0x7f843400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:56.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.839+0000 7f84457fa640 1 -- 172.21.15.76:0/3435379955 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8434009670 con 0x7f844806fff0 2023-11-09T03:51:56.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.839+0000 7f84457fa640 1 -- 172.21.15.76:0/3435379955 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843400c9a0 con 0x7f844806fff0 2023-11-09T03:51:56.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.839+0000 7f84457fa640 1 -- 172.21.15.76:0/3435379955 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843400cd50 con 0x7f844806fff0 2023-11-09T03:51:56.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.839+0000 7f844ed4c640 1 -- 172.21.15.76:0/3435379955 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f844815cc30 con 0x7f844806fff0 2023-11-09T03:51:56.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.839+0000 7f844ed4c640 1 -- 172.21.15.76:0/3435379955 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84480c16d0 con 0x7f844806fff0 2023-11-09T03:51:56.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.840+0000 7f844ed4c640 1 -- 172.21.15.76:0/3435379955 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84480684b0 con 0x7f844806fff0 2023-11-09T03:51:56.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.840+0000 7f84457fa640 1 -- 172.21.15.76:0/3435379955 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8434013470 con 0x7f844806fff0 2023-11-09T03:51:56.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.841+0000 7f84457fa640 1 --2- 172.21.15.76:0/3435379955 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f842c06ae70 0x7f842c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:56.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.841+0000 7f84457fa640 1 -- 172.21.15.76:0/3435379955 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7737+0+0 (secure 0 0 0) 0x7f843408a010 con 0x7f844806fff0 2023-11-09T03:51:56.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.841+0000 7f844cae8640 1 --2- 172.21.15.76:0/3435379955 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f842c06ae70 0x7f842c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:56.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.841+0000 7f844cae8640 1 --2- 172.21.15.76:0/3435379955 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f842c06ae70 0x7f842c06d330 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f843c038000 tx=0x7f843c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:56.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:56.845+0000 7f84457fa640 1 -- 172.21.15.76:0/3435379955 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f84340548d0 con 0x7f844806fff0 2023-11-09T03:51:57.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.011+0000 7f844ed4c640 1 -- 172.21.15.76:0/3435379955 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84480c9750 con 0x7f844806fff0 2023-11-09T03:51:57.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.013+0000 7f84457fa640 1 -- 172.21.15.76:0/3435379955 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+23015 (secure 0 0 0) 0x7f8434017090 con 0x7f844806fff0 2023-11-09T03:51:57.013 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:51:57.013 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":106,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:51:48.465322+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-09T03:47:59.280601+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-09T03:47:59.742039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:48:55.490832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-09T03:48:56.352624+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":9,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.17","mappings":[{"from":0,"to":2}]},{"pgid":"17.36","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:6832/3697850101":"2023-11-10T03:51:39.442929+0000","172.21.15.113:6837/3449910842":"2023-11-10T03:50:59.611256+0000","172.21.15.76:6835/3229239133":"2023-11-10T03:51:32.624856+0000","172.21.15.113:6836/3449910842":"2023-11-10T03:50:59.611256+0000","172.21.15.76:6835/1663988060":"2023-11-10T03:50:33.665092+0000","172.21.15.76:6834/1663988060":"2023-11-10T03:50:33.665092+0000","172.21.15.113:6837/3280098776":"2023-11-10T03:50:07.616780+0000","172.21.15.113:6836/3280098776":"2023-11-10T03:50:07.616780+0000","172.21.15.76:6835/4011275120":"2023-11-10T03:49:42.029057+0000","192.168.0.1:0/503424885":"2023-11-09T04:51:47.727422+0000","172.21.15.76:6833/3697850101":"2023-11-10T03:51:39.442929+0000","172.21.15.76:6834/3229239133":"2023-11-10T03:51:32.624856+0000","172.21.15.76:6834/4011275120":"2023-11-10T03:49:42.029057+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:51:57.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.015+0000 7f844ed4c640 1 -- 172.21.15.76:0/3435379955 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f842c06ae70 msgr2=0x7f842c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:57.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.015+0000 7f844ed4c640 1 --2- 172.21.15.76:0/3435379955 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f842c06ae70 0x7f842c06d330 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f843c038000 tx=0x7f843c008040 comp rx=0 tx=0).stop 2023-11-09T03:51:57.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.015+0000 7f844ed4c640 1 -- 172.21.15.76:0/3435379955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f844806fff0 msgr2=0x7f844815bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:57.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.015+0000 7f844ed4c640 1 --2- 172.21.15.76:0/3435379955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f844806fff0 0x7f844815bfb0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f843400e9b0 tx=0x7f843400ee70 comp rx=0 tx=0).stop 2023-11-09T03:51:57.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.015+0000 7f844ed4c640 1 -- 172.21.15.76:0/3435379955 shutdown_connections 2023-11-09T03:51:57.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.015+0000 7f844ed4c640 1 --2- 172.21.15.76:0/3435379955 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f842c06ae70 0x7f842c06d330 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.015+0000 7f844ed4c640 1 --2- 172.21.15.76:0/3435379955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8448159e40 0x7f84480ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.015+0000 7f844ed4c640 1 --2- 172.21.15.76:0/3435379955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f844806fff0 0x7f844815bfb0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.015+0000 7f844ed4c640 1 --2- 172.21.15.76:0/3435379955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f844806bb60 0x7f84480c1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.015+0000 7f844ed4c640 1 -- 172.21.15.76:0/3435379955 >> 172.21.15.76:0/3435379955 conn(0x7f84480cb530 msgr2=0x7f8448159030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:57.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.015+0000 7f844ed4c640 1 -- 172.21.15.76:0/3435379955 shutdown_connections 2023-11-09T03:51:57.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.015+0000 7f844ed4c640 1 -- 172.21.15.76:0/3435379955 wait complete. 2023-11-09T03:51:57.025 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-09T03:47:00.036540+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_data_pool', 'create_time': '2023-11-09T03:47:59.280601+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 70, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-09T03:47:59.742039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:48:55.490832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:48:56.352624+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 9, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-09T03:51:57.026 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:51:57.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.129+0000 7fec77fb7640 1 Processor -- start 2023-11-09T03:51:57.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.129+0000 7fec77fb7640 1 -- start start 2023-11-09T03:51:57.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.130+0000 7fec77fb7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec700addd0 0x7fec700cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:57.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.130+0000 7fec77fb7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fec700cc680 0x7fec700ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:57.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.130+0000 7fec77fb7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fec7006bb10 0x7fec7006bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:57.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.130+0000 7fec77fb7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec7006c4c0 con 0x7fec700cc680 2023-11-09T03:51:57.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.130+0000 7fec77fb7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec700cd720 con 0x7fec7006bb10 2023-11-09T03:51:57.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.130+0000 7fec77fb7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec700ae1b0 con 0x7fec700addd0 2023-11-09T03:51:57.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.130+0000 7fec75552640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fec700cc680 0x7fec700ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:57.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.130+0000 7fec75552640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fec700cc680 0x7fec700ca2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41992/0 (socket says 172.21.15.76:41992) 2023-11-09T03:51:57.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.130+0000 7fec75552640 1 -- 172.21.15.76:0/2097282036 learned_addr learned my addr 172.21.15.76:0/2097282036 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:57.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.130+0000 7fec75552640 1 -- 172.21.15.76:0/2097282036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec700addd0 msgr2=0x7fec700cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:57.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.130+0000 7fec75d53640 1 --2- 172.21.15.76:0/2097282036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec700addd0 0x7fec700cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:57.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.130+0000 7fec76554640 1 --2- 172.21.15.76:0/2097282036 >> v2:172.21.15.113:3300/0 conn(0x7fec7006bb10 0x7fec7006bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:57.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.130+0000 7fec75552640 1 --2- 172.21.15.76:0/2097282036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec700addd0 0x7fec700cdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.130+0000 7fec75552640 1 -- 172.21.15.76:0/2097282036 >> v2:172.21.15.113:3300/0 conn(0x7fec7006bb10 msgr2=0x7fec7006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:57.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.130+0000 7fec75552640 1 --2- 172.21.15.76:0/2097282036 >> v2:172.21.15.113:3300/0 conn(0x7fec7006bb10 0x7fec7006bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.130+0000 7fec75552640 1 -- 172.21.15.76:0/2097282036 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec700ccb10 con 0x7fec700cc680 2023-11-09T03:51:57.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.131+0000 7fec75552640 1 --2- 172.21.15.76:0/2097282036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec700cc680 0x7fec700ca2d0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fec7015e0c0 tx=0x7fec6c02cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d03bd2cace3c9f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:57.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.131+0000 7fec74d51640 1 -- 172.21.15.76:0/2097282036 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c02dba0 con 0x7fec700cc680 2023-11-09T03:51:57.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.131+0000 7fec74d51640 1 -- 172.21.15.76:0/2097282036 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec6c02dd00 con 0x7fec700cc680 2023-11-09T03:51:57.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.131+0000 7fec74d51640 1 -- 172.21.15.76:0/2097282036 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c035660 con 0x7fec700cc680 2023-11-09T03:51:57.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.131+0000 7fec77fb7640 1 -- 172.21.15.76:0/2097282036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec700cc680 msgr2=0x7fec700ca2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:57.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.131+0000 7fec77fb7640 1 --2- 172.21.15.76:0/2097282036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec700cc680 0x7fec700ca2d0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fec7015e0c0 tx=0x7fec6c02cdf0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.131+0000 7fec77fb7640 1 -- 172.21.15.76:0/2097282036 shutdown_connections 2023-11-09T03:51:57.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.131+0000 7fec77fb7640 1 --2- 172.21.15.76:0/2097282036 >> v2:172.21.15.113:3300/0 conn(0x7fec7006bb10 0x7fec7006bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.131+0000 7fec77fb7640 1 --2- 172.21.15.76:0/2097282036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec700cc680 0x7fec700ca2d0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.131+0000 7fec77fb7640 1 --2- 172.21.15.76:0/2097282036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec700addd0 0x7fec700cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.131+0000 7fec77fb7640 1 -- 172.21.15.76:0/2097282036 >> 172.21.15.76:0/2097282036 conn(0x7fec700cb530 msgr2=0x7fec700cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:57.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.132+0000 7fec77fb7640 1 -- 172.21.15.76:0/2097282036 shutdown_connections 2023-11-09T03:51:57.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.132+0000 7fec77fb7640 1 -- 172.21.15.76:0/2097282036 wait complete. 2023-11-09T03:51:57.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.133+0000 7fec77fb7640 1 Processor -- start 2023-11-09T03:51:57.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.133+0000 7fec77fb7640 1 -- start start 2023-11-09T03:51:57.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.133+0000 7fec77fb7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec7006bb10 0x7fec700c25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:57.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.133+0000 7fec77fb7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec700addd0 0x7fec701525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:57.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.133+0000 7fec77fb7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec70152d70 0x7fec700c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:57.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.133+0000 7fec77fb7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec70153680 con 0x7fec7006bb10 2023-11-09T03:51:57.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.133+0000 7fec77fb7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec700b1ad0 con 0x7fec70152d70 2023-11-09T03:51:57.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.133+0000 7fec77fb7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec700c2b00 con 0x7fec700addd0 2023-11-09T03:51:57.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.133+0000 7fec75d53640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec7006bb10 0x7fec700c25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:57.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.133+0000 7fec75d53640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec7006bb10 0x7fec700c25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42002/0 (socket says 172.21.15.76:42002) 2023-11-09T03:51:57.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.133+0000 7fec75d53640 1 -- 172.21.15.76:0/73926346 learned_addr learned my addr 172.21.15.76:0/73926346 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:57.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.134+0000 7fec76554640 1 --2- 172.21.15.76:0/73926346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec70152d70 0x7fec700c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:57.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.134+0000 7fec75552640 1 --2- 172.21.15.76:0/73926346 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec700addd0 0x7fec701525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:57.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.134+0000 7fec75d53640 1 -- 172.21.15.76:0/73926346 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec700addd0 msgr2=0x7fec701525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:57.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.134+0000 7fec75d53640 1 --2- 172.21.15.76:0/73926346 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec700addd0 0x7fec701525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.134+0000 7fec75d53640 1 -- 172.21.15.76:0/73926346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec70152d70 msgr2=0x7fec700c41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:57.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.134+0000 7fec75d53640 1 --2- 172.21.15.76:0/73926346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec70152d70 0x7fec700c41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.134+0000 7fec75d53640 1 -- 172.21.15.76:0/73926346 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6c00a670 con 0x7fec7006bb10 2023-11-09T03:51:57.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.134+0000 7fec76554640 1 --2- 172.21.15.76:0/73926346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec70152d70 0x7fec700c41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:51:57.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.134+0000 7fec75d53640 1 --2- 172.21.15.76:0/73926346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec7006bb10 0x7fec700c25c0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fec6000c7f0 tx=0x7fec6000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:57.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.134+0000 7fec667fc640 1 -- 172.21.15.76:0/73926346 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec60004220 con 0x7fec7006bb10 2023-11-09T03:51:57.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.134+0000 7fec667fc640 1 -- 172.21.15.76:0/73926346 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec600043a0 con 0x7fec7006bb10 2023-11-09T03:51:57.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.134+0000 7fec667fc640 1 -- 172.21.15.76:0/73926346 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec60007860 con 0x7fec7006bb10 2023-11-09T03:51:57.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.134+0000 7fec77fb7640 1 -- 172.21.15.76:0/73926346 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec700c2d10 con 0x7fec7006bb10 2023-11-09T03:51:57.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.134+0000 7fec77fb7640 1 -- 172.21.15.76:0/73926346 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec700c5270 con 0x7fec7006bb10 2023-11-09T03:51:57.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.135+0000 7fec77fb7640 1 -- 172.21.15.76:0/73926346 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec700684b0 con 0x7fec7006bb10 2023-11-09T03:51:57.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.136+0000 7fec667fc640 1 -- 172.21.15.76:0/73926346 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fec60007bd0 con 0x7fec7006bb10 2023-11-09T03:51:57.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.136+0000 7fec667fc640 1 --2- 172.21.15.76:0/73926346 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fec5406aea0 0x7fec5406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:57.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.136+0000 7fec667fc640 1 -- 172.21.15.76:0/73926346 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7737+0+0 (secure 0 0 0) 0x7fec60089e20 con 0x7fec7006bb10 2023-11-09T03:51:57.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.136+0000 7fec75552640 1 --2- 172.21.15.76:0/73926346 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fec5406aea0 0x7fec5406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:57.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.136+0000 7fec75552640 1 --2- 172.21.15.76:0/73926346 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fec5406aea0 0x7fec5406d360 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fec6c037000 tx=0x7fec6c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:57.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.140+0000 7fec667fc640 1 -- 172.21.15.76:0/73926346 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fec60054650 con 0x7fec7006bb10 2023-11-09T03:51:57.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.304+0000 7fec77fb7640 1 -- 172.21.15.76:0/73926346 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fec700b4a30 con 0x7fec7006bb10 2023-11-09T03:51:57.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.304+0000 7fec667fc640 1 -- 172.21.15.76:0/73926346 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7fec6001e0a0 con 0x7fec7006bb10 2023-11-09T03:51:57.304 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:51:57.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.306+0000 7fec77fb7640 1 -- 172.21.15.76:0/73926346 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fec5406aea0 msgr2=0x7fec5406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:57.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.306+0000 7fec77fb7640 1 --2- 172.21.15.76:0/73926346 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fec5406aea0 0x7fec5406d360 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fec6c037000 tx=0x7fec6c008040 comp rx=0 tx=0).stop 2023-11-09T03:51:57.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.306+0000 7fec77fb7640 1 -- 172.21.15.76:0/73926346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec7006bb10 msgr2=0x7fec700c25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:57.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.306+0000 7fec77fb7640 1 --2- 172.21.15.76:0/73926346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec7006bb10 0x7fec700c25c0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fec6000c7f0 tx=0x7fec6000ccb0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.306+0000 7fec77fb7640 1 -- 172.21.15.76:0/73926346 shutdown_connections 2023-11-09T03:51:57.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.306+0000 7fec77fb7640 1 --2- 172.21.15.76:0/73926346 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fec5406aea0 0x7fec5406d360 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.306+0000 7fec77fb7640 1 --2- 172.21.15.76:0/73926346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec70152d70 0x7fec700c41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.306+0000 7fec77fb7640 1 --2- 172.21.15.76:0/73926346 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec700addd0 0x7fec701525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.306+0000 7fec77fb7640 1 --2- 172.21.15.76:0/73926346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec7006bb10 0x7fec700c25c0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.306+0000 7fec77fb7640 1 -- 172.21.15.76:0/73926346 >> 172.21.15.76:0/73926346 conn(0x7fec700cb530 msgr2=0x7fec700afaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:57.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.306+0000 7fec77fb7640 1 -- 172.21.15.76:0/73926346 shutdown_connections 2023-11-09T03:51:57.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.306+0000 7fec77fb7640 1 -- 172.21.15.76:0/73926346 wait complete. 2023-11-09T03:51:57.316 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-09T03:51:57.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.419+0000 7fc4dbb41640 1 Processor -- start 2023-11-09T03:51:57.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.419+0000 7fc4dbb41640 1 -- start start 2023-11-09T03:51:57.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.420+0000 7fc4dbb41640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4d415d4e0 0x7fc4d40d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:57.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.420+0000 7fc4dbb41640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc4d40d3b60 0x7fc4d40d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:57.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.420+0000 7fc4dbb41640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4d40b32b0 con 0x7fc4d415d100 2023-11-09T03:51:57.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.420+0000 7fc4dbb41640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4d40cec20 con 0x7fc4d40d3b60 2023-11-09T03:51:57.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.420+0000 7fc4dbb41640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d40cee10 con 0x7fc4d415d4e0 2023-11-09T03:51:57.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.420+0000 7fc4da0de640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc4d40d3b60 0x7fc4d40d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:57.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.420+0000 7fc4d98dd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc4d415d100 0x7fc4d40ce510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50714/0 (socket says 172.21.15.76:50714) 2023-11-09T03:51:57.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.420+0000 7fc4da0de640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc4d40d3b60 0x7fc4d40d1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42012/0 (socket says 172.21.15.76:42012) 2023-11-09T03:51:57.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.420+0000 7fc4d98dd640 1 -- 172.21.15.76:0/1486151193 learned_addr learned my addr 172.21.15.76:0/1486151193 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:51:57.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.421+0000 7fc4da0de640 1 -- 172.21.15.76:0/1486151193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4d415d4e0 msgr2=0x7fc4d40d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:57.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.421+0000 7fc4d90dc640 1 --2- 172.21.15.76:0/1486151193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4d415d4e0 0x7fc4d40d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:57.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.421+0000 7fc4d88db640 1 -- 172.21.15.76:0/1486151193 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3810650559 0 0) 0x7fc4d40b32b0 con 0x7fc4d415d100 2023-11-09T03:51:57.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.421+0000 7fc4da0de640 1 --2- 172.21.15.76:0/1486151193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4d415d4e0 0x7fc4d40d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.421+0000 7fc4da0de640 1 -- 172.21.15.76:0/1486151193 >> v1:172.21.15.76:6789/0 conn(0x7fc4d415d100 legacy=0x7fc4d40ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:57.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.421+0000 7fc4da0de640 1 -- 172.21.15.76:0/1486151193 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d40b4750 con 0x7fc4d40d3b60 2023-11-09T03:51:57.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.421+0000 7fc4d90dc640 1 --2- 172.21.15.76:0/1486151193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4d415d4e0 0x7fc4d40d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:51:57.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.421+0000 7fc4da0de640 1 --2- 172.21.15.76:0/1486151193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4d40d3b60 0x7fc4d40d1a10 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc0097a0 tx=0x7fc4cc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe647b17740c725f server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:57.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.422+0000 7fc4d88db640 1 -- 172.21.15.76:0/1486151193 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc031830 con 0x7fc4d40d3b60 2023-11-09T03:51:57.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.422+0000 7fc4d88db640 1 -- 172.21.15.76:0/1486151193 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4cc031990 con 0x7fc4d40d3b60 2023-11-09T03:51:57.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.422+0000 7fc4d88db640 1 -- 172.21.15.76:0/1486151193 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc031d20 con 0x7fc4d40d3b60 2023-11-09T03:51:57.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.422+0000 7fc4dbb41640 1 -- 172.21.15.76:0/1486151193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4d40d3b60 msgr2=0x7fc4d40d1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:57.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.422+0000 7fc4dbb41640 1 --2- 172.21.15.76:0/1486151193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4d40d3b60 0x7fc4d40d1a10 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc0097a0 tx=0x7fc4cc02b8d0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.422+0000 7fc4dbb41640 1 -- 172.21.15.76:0/1486151193 shutdown_connections 2023-11-09T03:51:57.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.422+0000 7fc4dbb41640 1 --2- 172.21.15.76:0/1486151193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4d40d3b60 0x7fc4d40d1a10 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.422+0000 7fc4dbb41640 1 --2- 172.21.15.76:0/1486151193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4d415d4e0 0x7fc4d40d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.422+0000 7fc4dbb41640 1 -- 172.21.15.76:0/1486151193 >> 172.21.15.76:0/1486151193 conn(0x7fc4d40d2ef0 msgr2=0x7fc4d40d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:57.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.422+0000 7fc4dbb41640 1 -- 172.21.15.76:0/1486151193 shutdown_connections 2023-11-09T03:51:57.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.422+0000 7fc4dbb41640 1 -- 172.21.15.76:0/1486151193 wait complete. 2023-11-09T03:51:57.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.423+0000 7fc4dbb41640 1 Processor -- start 2023-11-09T03:51:57.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.423+0000 7fc4dbb41640 1 -- start start 2023-11-09T03:51:57.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.424+0000 7fc4dbb41640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4d40d3b60 0x7fc4d4158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:57.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.424+0000 7fc4dbb41640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4d415d100 0x7fc4d41553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:57.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.424+0000 7fc4dbb41640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4d415d4e0 0x7fc4d41f6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:57.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.424+0000 7fc4dbb41640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d4152610 con 0x7fc4d415d4e0 2023-11-09T03:51:57.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.424+0000 7fc4dbb41640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d41f6e60 con 0x7fc4d40d3b60 2023-11-09T03:51:57.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.424+0000 7fc4dbb41640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d4158cb0 con 0x7fc4d415d100 2023-11-09T03:51:57.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.424+0000 7fc4da0de640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4d415d4e0 0x7fc4d41f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:57.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.424+0000 7fc4da0de640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4d415d4e0 0x7fc4d41f6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42016/0 (socket says 172.21.15.76:42016) 2023-11-09T03:51:57.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.424+0000 7fc4da0de640 1 -- 172.21.15.76:0/942366293 learned_addr learned my addr 172.21.15.76:0/942366293 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:57.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.425+0000 7fc4d90dc640 1 --2- 172.21.15.76:0/942366293 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4d415d100 0x7fc4d41553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:57.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.425+0000 7fc4da0de640 1 -- 172.21.15.76:0/942366293 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4d415d100 msgr2=0x7fc4d41553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:57.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.425+0000 7fc4d98dd640 1 --2- 172.21.15.76:0/942366293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4d40d3b60 0x7fc4d4158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:57.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.425+0000 7fc4da0de640 1 --2- 172.21.15.76:0/942366293 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4d415d100 0x7fc4d41553d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.425+0000 7fc4da0de640 1 -- 172.21.15.76:0/942366293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4d40d3b60 msgr2=0x7fc4d4158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:57.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.425+0000 7fc4da0de640 1 --2- 172.21.15.76:0/942366293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4d40d3b60 0x7fc4d4158770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.425+0000 7fc4da0de640 1 -- 172.21.15.76:0/942366293 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4cc009400 con 0x7fc4d415d4e0 2023-11-09T03:51:57.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.425+0000 7fc4d90dc640 1 --2- 172.21.15.76:0/942366293 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4d415d100 0x7fc4d41553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:51:57.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.425+0000 7fc4d98dd640 1 --2- 172.21.15.76:0/942366293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4d40d3b60 0x7fc4d4158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:51:57.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.425+0000 7fc4da0de640 1 --2- 172.21.15.76:0/942366293 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4d415d4e0 0x7fc4d41f6000 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc0098c0 tx=0x7fc4cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:57.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.425+0000 7fc4ca7fc640 1 -- 172.21.15.76:0/942366293 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc004230 con 0x7fc4d415d4e0 2023-11-09T03:51:57.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.425+0000 7fc4ca7fc640 1 -- 172.21.15.76:0/942366293 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4cc004390 con 0x7fc4d415d4e0 2023-11-09T03:51:57.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.425+0000 7fc4dbb41640 1 -- 172.21.15.76:0/942366293 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d415e6f0 con 0x7fc4d415d4e0 2023-11-09T03:51:57.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.425+0000 7fc4ca7fc640 1 -- 172.21.15.76:0/942366293 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc03a920 con 0x7fc4d415d4e0 2023-11-09T03:51:57.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.425+0000 7fc4dbb41640 1 -- 172.21.15.76:0/942366293 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d4152330 con 0x7fc4d415d4e0 2023-11-09T03:51:57.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.426+0000 7fc4dbb41640 1 -- 172.21.15.76:0/942366293 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d40684b0 con 0x7fc4d415d4e0 2023-11-09T03:51:57.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.426+0000 7fc4ca7fc640 1 -- 172.21.15.76:0/942366293 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc4cc03aa80 con 0x7fc4d415d4e0 2023-11-09T03:51:57.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.427+0000 7fc4ca7fc640 1 --2- 172.21.15.76:0/942366293 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc4b806ae70 0x7fc4b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:57.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.427+0000 7fc4ca7fc640 1 -- 172.21.15.76:0/942366293 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7737+0+0 (secure 0 0 0) 0x7fc4cc0ac400 con 0x7fc4d415d4e0 2023-11-09T03:51:57.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.427+0000 7fc4d98dd640 1 --2- 172.21.15.76:0/942366293 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc4b806ae70 0x7fc4b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:57.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.427+0000 7fc4d98dd640 1 --2- 172.21.15.76:0/942366293 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc4b806ae70 0x7fc4b806d330 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0007fd0 tx=0x7fc4d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:57.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.431+0000 7fc4ca7fc640 1 -- 172.21.15.76:0/942366293 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc4cc076e70 con 0x7fc4d415d4e0 2023-11-09T03:51:57.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.601+0000 7fc4dbb41640 1 -- 172.21.15.76:0/942366293 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7fc4d40d2cd0 con 0x7fc4d415d4e0 2023-11-09T03:51:57.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.601+0000 7fc4ca7fc640 1 -- 172.21.15.76:0/942366293 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v106) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc4cc0350a0 con 0x7fc4d415d4e0 2023-11-09T03:51:57.601 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:51:57.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.603+0000 7fc4dbb41640 1 -- 172.21.15.76:0/942366293 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc4b806ae70 msgr2=0x7fc4b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:57.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.603+0000 7fc4dbb41640 1 --2- 172.21.15.76:0/942366293 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc4b806ae70 0x7fc4b806d330 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0007fd0 tx=0x7fc4d0009040 comp rx=0 tx=0).stop 2023-11-09T03:51:57.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.603+0000 7fc4dbb41640 1 -- 172.21.15.76:0/942366293 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4d415d4e0 msgr2=0x7fc4d41f6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:57.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.603+0000 7fc4dbb41640 1 --2- 172.21.15.76:0/942366293 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4d415d4e0 0x7fc4d41f6000 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc0098c0 tx=0x7fc4cc0046c0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.603+0000 7fc4dbb41640 1 -- 172.21.15.76:0/942366293 shutdown_connections 2023-11-09T03:51:57.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.603+0000 7fc4dbb41640 1 --2- 172.21.15.76:0/942366293 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc4b806ae70 0x7fc4b806d330 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.603+0000 7fc4dbb41640 1 --2- 172.21.15.76:0/942366293 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4d415d4e0 0x7fc4d41f6000 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.603+0000 7fc4dbb41640 1 --2- 172.21.15.76:0/942366293 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4d415d100 0x7fc4d41553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.603+0000 7fc4dbb41640 1 --2- 172.21.15.76:0/942366293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4d40d3b60 0x7fc4d4158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.603+0000 7fc4dbb41640 1 -- 172.21.15.76:0/942366293 >> 172.21.15.76:0/942366293 conn(0x7fc4d40d2ef0 msgr2=0x7fc4d40c9750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:57.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.603+0000 7fc4dbb41640 1 -- 172.21.15.76:0/942366293 shutdown_connections 2023-11-09T03:51:57.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.603+0000 7fc4dbb41640 1 -- 172.21.15.76:0/942366293 wait complete. 2023-11-09T03:51:57.613 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-09T03:51:57.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.716+0000 7f0f44c61640 1 Processor -- start 2023-11-09T03:51:57.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.716+0000 7f0f44c61640 1 -- start start 2023-11-09T03:51:57.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.717+0000 7f0f44c61640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0f4015d100 0x7f0f400ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:57.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.717+0000 7f0f44c61640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0f4015d4e0 0x7f0f400d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:57.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.717+0000 7f0f44c61640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f400b32b0 con 0x7f0f400d3b60 2023-11-09T03:51:57.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.717+0000 7f0f44c61640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f400cee10 con 0x7f0f4015d100 2023-11-09T03:51:57.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.717+0000 7f0f44c61640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f400b4750 con 0x7f0f4015d4e0 2023-11-09T03:51:57.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.717+0000 7f0f3e59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0f4015d100 0x7f0f400ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:57.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.718+0000 7f0f3e59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0f4015d100 0x7f0f400ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42032/0 (socket says 172.21.15.76:42032) 2023-11-09T03:51:57.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.718+0000 7f0f3e59c640 1 -- 172.21.15.76:0/1510660599 learned_addr learned my addr 172.21.15.76:0/1510660599 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:57.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.718+0000 7f0f3e59c640 1 -- 172.21.15.76:0/1510660599 >> v2:172.21.15.113:3300/0 conn(0x7f0f4015d4e0 msgr2=0x7f0f400d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:57.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.718+0000 7f0f3e59c640 1 --2- 172.21.15.76:0/1510660599 >> v2:172.21.15.113:3300/0 conn(0x7f0f4015d4e0 0x7f0f400d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.718+0000 7f0f3e59c640 1 -- 172.21.15.76:0/1510660599 >> v1:172.21.15.113:6789/0 conn(0x7f0f400d3b60 legacy=0x7f0f400cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:57.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.718+0000 7f0f3e59c640 1 -- 172.21.15.76:0/1510660599 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f400b2980 con 0x7f0f4015d100 2023-11-09T03:51:57.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.718+0000 7f0f3e59c640 1 --2- 172.21.15.76:0/1510660599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f4015d100 0x7f0f400ce510 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f0f40160da0 tx=0x7f0f3002bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142ad7124070f930 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:57.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.719+0000 7f0f3d59a640 1 -- 172.21.15.76:0/1510660599 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3002cad0 con 0x7f0f4015d100 2023-11-09T03:51:57.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.719+0000 7f0f3d59a640 1 -- 172.21.15.76:0/1510660599 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f3002cc30 con 0x7f0f4015d100 2023-11-09T03:51:57.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.719+0000 7f0f3d59a640 1 -- 172.21.15.76:0/1510660599 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f300343f0 con 0x7f0f4015d100 2023-11-09T03:51:57.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.719+0000 7f0f44c61640 1 -- 172.21.15.76:0/1510660599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f4015d100 msgr2=0x7f0f400ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:57.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.719+0000 7f0f44c61640 1 --2- 172.21.15.76:0/1510660599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f4015d100 0x7f0f400ce510 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f0f40160da0 tx=0x7f0f3002bf30 comp rx=0 tx=0).stop 2023-11-09T03:51:57.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.719+0000 7f0f44c61640 1 -- 172.21.15.76:0/1510660599 shutdown_connections 2023-11-09T03:51:57.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.719+0000 7f0f44c61640 1 --2- 172.21.15.76:0/1510660599 >> v2:172.21.15.113:3300/0 conn(0x7f0f4015d4e0 0x7f0f400d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.719+0000 7f0f44c61640 1 --2- 172.21.15.76:0/1510660599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f4015d100 0x7f0f400ce510 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.719+0000 7f0f44c61640 1 -- 172.21.15.76:0/1510660599 >> 172.21.15.76:0/1510660599 conn(0x7f0f400d2ef0 msgr2=0x7f0f400d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:57.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.719+0000 7f0f44c61640 1 -- 172.21.15.76:0/1510660599 shutdown_connections 2023-11-09T03:51:57.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.719+0000 7f0f44c61640 1 -- 172.21.15.76:0/1510660599 wait complete. 2023-11-09T03:51:57.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.721+0000 7f0f44c61640 1 Processor -- start 2023-11-09T03:51:57.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.721+0000 7f0f44c61640 1 -- start start 2023-11-09T03:51:57.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.721+0000 7f0f44c61640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f400d3b60 0x7f0f40159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:57.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.721+0000 7f0f44c61640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f4015d100 0x7f0f400cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:57.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.721+0000 7f0f44c61640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f4015d4e0 0x7f0f4006ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:57.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.721+0000 7f0f44c61640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f401f2b70 con 0x7f0f4015d4e0 2023-11-09T03:51:57.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.721+0000 7f0f44c61640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f401f2d60 con 0x7f0f400d3b60 2023-11-09T03:51:57.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.721+0000 7f0f44c61640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f401f3040 con 0x7f0f4015d100 2023-11-09T03:51:57.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.721+0000 7f0f3ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f4015d4e0 0x7f0f4006ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:57.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.721+0000 7f0f3ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f4015d4e0 0x7f0f4006ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42040/0 (socket says 172.21.15.76:42040) 2023-11-09T03:51:57.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.721+0000 7f0f3ed9d640 1 -- 172.21.15.76:0/1155134553 learned_addr learned my addr 172.21.15.76:0/1155134553 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:57.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.722+0000 7f0f3e59c640 1 --2- 172.21.15.76:0/1155134553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f400d3b60 0x7f0f40159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:57.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.722+0000 7f0f3ed9d640 1 -- 172.21.15.76:0/1155134553 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f4015d100 msgr2=0x7f0f400cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:57.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.722+0000 7f0f3ed9d640 1 --2- 172.21.15.76:0/1155134553 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f4015d100 0x7f0f400cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.722+0000 7f0f3ed9d640 1 -- 172.21.15.76:0/1155134553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f400d3b60 msgr2=0x7f0f40159360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:57.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.722+0000 7f0f3ed9d640 1 --2- 172.21.15.76:0/1155134553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f400d3b60 0x7f0f40159360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.722+0000 7f0f3ed9d640 1 -- 172.21.15.76:0/1155134553 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f30009b30 con 0x7f0f4015d4e0 2023-11-09T03:51:57.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.722+0000 7f0f3ed9d640 1 --2- 172.21.15.76:0/1155134553 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f4015d4e0 0x7f0f4006ad90 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800b490 tx=0x7f0f3800b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:57.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.722+0000 7f0f2effd640 1 -- 172.21.15.76:0/1155134553 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f38011830 con 0x7f0f4015d4e0 2023-11-09T03:51:57.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.722+0000 7f0f2effd640 1 -- 172.21.15.76:0/1155134553 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f380119b0 con 0x7f0f4015d4e0 2023-11-09T03:51:57.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.722+0000 7f0f44c61640 1 -- 172.21.15.76:0/1155134553 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f401f3320 con 0x7f0f4015d4e0 2023-11-09T03:51:57.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.722+0000 7f0f2effd640 1 -- 172.21.15.76:0/1155134553 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f38011d60 con 0x7f0f4015d4e0 2023-11-09T03:51:57.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.722+0000 7f0f44c61640 1 -- 172.21.15.76:0/1155134553 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f401f38e0 con 0x7f0f4015d4e0 2023-11-09T03:51:57.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.723+0000 7f0f44c61640 1 -- 172.21.15.76:0/1155134553 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f401f3b00 con 0x7f0f4015d4e0 2023-11-09T03:51:57.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.723+0000 7f0f2effd640 1 -- 172.21.15.76:0/1155134553 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0f3801a400 con 0x7f0f4015d4e0 2023-11-09T03:51:57.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.724+0000 7f0f2effd640 1 --2- 172.21.15.76:0/1155134553 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0f2006ae70 0x7f0f2006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:57.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.724+0000 7f0f2effd640 1 -- 172.21.15.76:0/1155134553 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7737+0+0 (secure 0 0 0) 0x7f0f38089f10 con 0x7f0f4015d4e0 2023-11-09T03:51:57.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.724+0000 7f0f3e59c640 1 --2- 172.21.15.76:0/1155134553 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0f2006ae70 0x7f0f2006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:57.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.725+0000 7f0f3e59c640 1 --2- 172.21.15.76:0/1155134553 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0f2006ae70 0x7f0f2006d330 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0f30035000 tx=0x7f0f30030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:57.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.728+0000 7f0f2effd640 1 -- 172.21.15.76:0/1155134553 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0f380547d0 con 0x7f0f4015d4e0 2023-11-09T03:51:57.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.894+0000 7f0f44c61640 1 -- 172.21.15.76:0/1155134553 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7f0f400684b0 con 0x7f0f4015d4e0 2023-11-09T03:51:57.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.895+0000 7f0f2effd640 1 -- 172.21.15.76:0/1155134553 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v106) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0f38015020 con 0x7f0f4015d4e0 2023-11-09T03:51:57.894 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:51:57.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.897+0000 7f0f44c61640 1 -- 172.21.15.76:0/1155134553 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0f2006ae70 msgr2=0x7f0f2006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:57.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.897+0000 7f0f44c61640 1 --2- 172.21.15.76:0/1155134553 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0f2006ae70 0x7f0f2006d330 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0f30035000 tx=0x7f0f30030000 comp rx=0 tx=0).stop 2023-11-09T03:51:57.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.897+0000 7f0f44c61640 1 -- 172.21.15.76:0/1155134553 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f4015d4e0 msgr2=0x7f0f4006ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:57.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.897+0000 7f0f44c61640 1 --2- 172.21.15.76:0/1155134553 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f4015d4e0 0x7f0f4006ad90 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800b490 tx=0x7f0f3800b950 comp rx=0 tx=0).stop 2023-11-09T03:51:57.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.897+0000 7f0f44c61640 1 -- 172.21.15.76:0/1155134553 shutdown_connections 2023-11-09T03:51:57.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.897+0000 7f0f44c61640 1 --2- 172.21.15.76:0/1155134553 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0f2006ae70 0x7f0f2006d330 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.897+0000 7f0f44c61640 1 --2- 172.21.15.76:0/1155134553 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f4015d4e0 0x7f0f4006ad90 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.897+0000 7f0f44c61640 1 --2- 172.21.15.76:0/1155134553 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f4015d100 0x7f0f400cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.897+0000 7f0f44c61640 1 --2- 172.21.15.76:0/1155134553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f400d3b60 0x7f0f40159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:57.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.897+0000 7f0f44c61640 1 -- 172.21.15.76:0/1155134553 >> 172.21.15.76:0/1155134553 conn(0x7f0f400d2ef0 msgr2=0x7f0f400cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:57.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.897+0000 7f0f44c61640 1 -- 172.21.15.76:0/1155134553 shutdown_connections 2023-11-09T03:51:57.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:57.897+0000 7f0f44c61640 1 -- 172.21.15.76:0/1155134553 wait complete. 2023-11-09T03:51:57.907 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-09T03:51:58.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.010+0000 7f393e7f5640 1 Processor -- start 2023-11-09T03:51:58.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.010+0000 7f393e7f5640 1 -- start start 2023-11-09T03:51:58.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.011+0000 7f393e7f5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f39380d1be0 0x7f3938167e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:58.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.011+0000 7f393e7f5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39380b32b0 con 0x7f393815d100 2023-11-09T03:51:58.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.011+0000 7f393e7f5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39380cee10 con 0x7f393815d4e0 2023-11-09T03:51:58.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.011+0000 7f393e7f5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39380cec20 con 0x7f39380d1be0 2023-11-09T03:51:58.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.011+0000 7f393cd92640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f39380d1be0 0x7f3938167e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:58.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.011+0000 7f3937fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f393815d100 0x7f39380ce510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50728/0 (socket says 172.21.15.76:50728) 2023-11-09T03:51:58.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.011+0000 7f393cd92640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f39380d1be0 0x7f3938167e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42048/0 (socket says 172.21.15.76:42048) 2023-11-09T03:51:58.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.011+0000 7f3937fff640 1 -- 172.21.15.76:0/2142355691 learned_addr learned my addr 172.21.15.76:0/2142355691 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:51:58.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.012+0000 7f393cd92640 1 -- 172.21.15.76:0/2142355691 >> v1:172.21.15.113:6789/0 conn(0x7f393815d4e0 legacy=0x7f39380d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:58.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.012+0000 7f393cd92640 1 -- 172.21.15.76:0/2142355691 >> v1:172.21.15.76:6789/0 conn(0x7f393815d100 legacy=0x7f39380ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:58.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.012+0000 7f393cd92640 1 -- 172.21.15.76:0/2142355691 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39380b4750 con 0x7f39380d1be0 2023-11-09T03:51:58.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.012+0000 7f3936ffd640 1 -- 172.21.15.76:0/2142355691 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4274500064 0 0) 0x7f39380b32b0 con 0x7f393815d100 2023-11-09T03:51:58.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.012+0000 7f393cd92640 1 --2- 172.21.15.76:0/2142355691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39380d1be0 0x7f3938167e80 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f3930009d90 tx=0x7f393002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bef93260b8f5611 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:58.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.013+0000 7f3936ffd640 1 -- 172.21.15.76:0/2142355691 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393002cd00 con 0x7f39380d1be0 2023-11-09T03:51:58.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.013+0000 7f3936ffd640 1 -- 172.21.15.76:0/2142355691 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393002ce60 con 0x7f39380d1be0 2023-11-09T03:51:58.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.013+0000 7f3936ffd640 1 -- 172.21.15.76:0/2142355691 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3930034660 con 0x7f39380d1be0 2023-11-09T03:51:58.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.013+0000 7f393e7f5640 1 -- 172.21.15.76:0/2142355691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39380d1be0 msgr2=0x7f3938167e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:58.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.013+0000 7f393e7f5640 1 --2- 172.21.15.76:0/2142355691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39380d1be0 0x7f3938167e80 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f3930009d90 tx=0x7f393002bf50 comp rx=0 tx=0).stop 2023-11-09T03:51:58.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.013+0000 7f393e7f5640 1 -- 172.21.15.76:0/2142355691 shutdown_connections 2023-11-09T03:51:58.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.013+0000 7f393e7f5640 1 --2- 172.21.15.76:0/2142355691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39380d1be0 0x7f3938167e80 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.013+0000 7f393e7f5640 1 -- 172.21.15.76:0/2142355691 >> 172.21.15.76:0/2142355691 conn(0x7f39380d2ef0 msgr2=0x7f39380d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:58.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.013+0000 7f393e7f5640 1 -- 172.21.15.76:0/2142355691 shutdown_connections 2023-11-09T03:51:58.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.013+0000 7f393e7f5640 1 -- 172.21.15.76:0/2142355691 wait complete. 2023-11-09T03:51:58.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.014+0000 7f393e7f5640 1 Processor -- start 2023-11-09T03:51:58.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.014+0000 7f393e7f5640 1 -- start start 2023-11-09T03:51:58.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.015+0000 7f393e7f5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39380d1be0 0x7f3938152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:58.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.015+0000 7f393e7f5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f393815d100 0x7f393806bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:58.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.015+0000 7f393e7f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f393815d4e0 0x7f393806c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:58.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.015+0000 7f393e7f5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39380caa00 con 0x7f393815d100 2023-11-09T03:51:58.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.015+0000 7f393e7f5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39380b86b0 con 0x7f393815d4e0 2023-11-09T03:51:58.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.015+0000 7f393e7f5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39380ca060 con 0x7f39380d1be0 2023-11-09T03:51:58.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.015+0000 7f39377fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f393815d100 0x7f393806bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:58.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.015+0000 7f39377fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f393815d100 0x7f393806bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42052/0 (socket says 172.21.15.76:42052) 2023-11-09T03:51:58.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.015+0000 7f39377fe640 1 -- 172.21.15.76:0/4053594605 learned_addr learned my addr 172.21.15.76:0/4053594605 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:58.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.016+0000 7f39377fe640 1 -- 172.21.15.76:0/4053594605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39380d1be0 msgr2=0x7f3938152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:58.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.016+0000 7f3937fff640 1 --2- 172.21.15.76:0/4053594605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39380d1be0 0x7f3938152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:58.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.016+0000 7f393cd92640 1 --2- 172.21.15.76:0/4053594605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f393815d4e0 0x7f393806c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:58.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.016+0000 7f39377fe640 1 --2- 172.21.15.76:0/4053594605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39380d1be0 0x7f3938152310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.016+0000 7f39377fe640 1 -- 172.21.15.76:0/4053594605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f393815d4e0 msgr2=0x7f393806c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:58.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.016+0000 7f39377fe640 1 --2- 172.21.15.76:0/4053594605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f393815d4e0 0x7f393806c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.016+0000 7f39377fe640 1 -- 172.21.15.76:0/4053594605 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3930009a70 con 0x7f393815d100 2023-11-09T03:51:58.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.016+0000 7f39377fe640 1 --2- 172.21.15.76:0/4053594605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f393815d100 0x7f393806bbd0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f392c00c3c0 tx=0x7f392c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:58.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.016+0000 7f3934ff9640 1 -- 172.21.15.76:0/4053594605 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392c0052d0 con 0x7f393815d100 2023-11-09T03:51:58.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.016+0000 7f3934ff9640 1 -- 172.21.15.76:0/4053594605 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f392c005450 con 0x7f393815d100 2023-11-09T03:51:58.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.016+0000 7f393e7f5640 1 -- 172.21.15.76:0/4053594605 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39380ca340 con 0x7f393815d100 2023-11-09T03:51:58.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.016+0000 7f3934ff9640 1 -- 172.21.15.76:0/4053594605 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392c003830 con 0x7f393815d100 2023-11-09T03:51:58.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.016+0000 7f393e7f5640 1 -- 172.21.15.76:0/4053594605 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3938152850 con 0x7f393815d100 2023-11-09T03:51:58.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.017+0000 7f393e7f5640 1 -- 172.21.15.76:0/4053594605 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39380b2960 con 0x7f393815d100 2023-11-09T03:51:58.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.017+0000 7f3934ff9640 1 -- 172.21.15.76:0/4053594605 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f392c003990 con 0x7f393815d100 2023-11-09T03:51:58.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.018+0000 7f3934ff9640 1 --2- 172.21.15.76:0/4053594605 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f392406ae70 0x7f392406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:58.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.018+0000 7f3934ff9640 1 -- 172.21.15.76:0/4053594605 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7737+0+0 (secure 0 0 0) 0x7f392c089270 con 0x7f393815d100 2023-11-09T03:51:58.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.018+0000 7f3937fff640 1 --2- 172.21.15.76:0/4053594605 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f392406ae70 0x7f392406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:58.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.018+0000 7f3937fff640 1 --2- 172.21.15.76:0/4053594605 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f392406ae70 0x7f392406d330 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f39280010e0 tx=0x7f3928009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:58.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.022+0000 7f3934ff9640 1 -- 172.21.15.76:0/4053594605 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f39380b2960 con 0x7f393815d100 2023-11-09T03:51:58.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.187+0000 7f393e7f5640 1 -- 172.21.15.76:0/4053594605 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7f39380cab40 con 0x7f393815d100 2023-11-09T03:51:58.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.187+0000 7f3934ff9640 1 -- 172.21.15.76:0/4053594605 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v106) v1 ==== 107+0+11 (secure 0 0 0) 0x7f392c017020 con 0x7f393815d100 2023-11-09T03:51:58.187 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:51:58.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.189+0000 7f393e7f5640 1 -- 172.21.15.76:0/4053594605 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f392406ae70 msgr2=0x7f392406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:58.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.189+0000 7f393e7f5640 1 --2- 172.21.15.76:0/4053594605 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f392406ae70 0x7f392406d330 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f39280010e0 tx=0x7f3928009040 comp rx=0 tx=0).stop 2023-11-09T03:51:58.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.189+0000 7f393e7f5640 1 -- 172.21.15.76:0/4053594605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f393815d100 msgr2=0x7f393806bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:58.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.189+0000 7f393e7f5640 1 --2- 172.21.15.76:0/4053594605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f393815d100 0x7f393806bbd0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f392c00c3c0 tx=0x7f392c00c880 comp rx=0 tx=0).stop 2023-11-09T03:51:58.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.189+0000 7f393e7f5640 1 -- 172.21.15.76:0/4053594605 shutdown_connections 2023-11-09T03:51:58.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.189+0000 7f393e7f5640 1 --2- 172.21.15.76:0/4053594605 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f392406ae70 0x7f392406d330 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.189+0000 7f393e7f5640 1 --2- 172.21.15.76:0/4053594605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f393815d4e0 0x7f393806c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.189+0000 7f393e7f5640 1 --2- 172.21.15.76:0/4053594605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f393815d100 0x7f393806bbd0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.189+0000 7f393e7f5640 1 --2- 172.21.15.76:0/4053594605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39380d1be0 0x7f3938152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.189+0000 7f393e7f5640 1 -- 172.21.15.76:0/4053594605 >> 172.21.15.76:0/4053594605 conn(0x7f39380d2ef0 msgr2=0x7f393815c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:58.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.189+0000 7f393e7f5640 1 -- 172.21.15.76:0/4053594605 shutdown_connections 2023-11-09T03:51:58.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.189+0000 7f393e7f5640 1 -- 172.21.15.76:0/4053594605 wait complete. 2023-11-09T03:51:58.199 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:51:58.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.302+0000 7f9c60f97640 1 Processor -- start 2023-11-09T03:51:58.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.302+0000 7f9c60f97640 1 -- start start 2023-11-09T03:51:58.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.303+0000 7f9c60f97640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9c5c0addd0 0x7f9c5c0cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:58.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.303+0000 7f9c60f97640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c5c0cc680 0x7f9c5c0ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:58.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.303+0000 7f9c60f97640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c5c0cd720 con 0x7f9c5c06bb10 2023-11-09T03:51:58.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.303+0000 7f9c60f97640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c5c0ae1b0 con 0x7f9c5c0addd0 2023-11-09T03:51:58.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.303+0000 7f9c60f97640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c0ccb10 con 0x7f9c5c0cc680 2023-11-09T03:51:58.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c5ad9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f9c5c06bb10 0x7f9c5c06f8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50744/0 (socket says 172.21.15.76:50744) 2023-11-09T03:51:58.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c5ad9d640 1 -- 172.21.15.76:0/957465048 learned_addr learned my addr 172.21.15.76:0/957465048 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:51:58.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c5959a640 1 -- 172.21.15.76:0/957465048 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 124118326 0 0) 0x7f9c5c0cd720 con 0x7f9c5c06bb10 2023-11-09T03:51:58.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c5959a640 1 -- 172.21.15.76:0/957465048 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c44003520 con 0x7f9c5c06bb10 2023-11-09T03:51:58.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c5a59c640 1 --2- 172.21.15.76:0/957465048 >> v2:172.21.15.113:3300/0 conn(0x7f9c5c0addd0 0x7f9c5c0cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:58.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c59d9b640 1 --2- 172.21.15.76:0/957465048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c5c0cc680 0x7f9c5c0ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:58.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c5959a640 1 -- 172.21.15.76:0/957465048 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 850887449 0 0) 0x7f9c44003520 con 0x7f9c5c06bb10 2023-11-09T03:51:58.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c5959a640 1 -- 172.21.15.76:0/957465048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c5c0cc680 msgr2=0x7f9c5c0ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:58.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c5959a640 1 --2- 172.21.15.76:0/957465048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c5c0cc680 0x7f9c5c0ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c5959a640 1 -- 172.21.15.76:0/957465048 >> v2:172.21.15.113:3300/0 conn(0x7f9c5c0addd0 msgr2=0x7f9c5c0cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:58.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c5959a640 1 --2- 172.21.15.76:0/957465048 >> v2:172.21.15.113:3300/0 conn(0x7f9c5c0addd0 0x7f9c5c0cdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c5959a640 1 -- 172.21.15.76:0/957465048 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5c0ccdf0 con 0x7f9c5c06bb10 2023-11-09T03:51:58.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c5959a640 1 -- 172.21.15.76:0/957465048 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f9c54001670 con 0x7f9c5c06bb10 2023-11-09T03:51:58.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c5959a640 1 -- 172.21.15.76:0/957465048 >> v1:172.21.15.76:6789/0 conn(0x7f9c5c06bb10 legacy=0x7f9c5c06f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:58.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c5959a640 1 --2- 172.21.15.76:0/957465048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c44003970 0x7f9c44003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:58.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c5959a640 1 --2- 172.21.15.76:0/957465048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c440042d0 0x7f9c440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:58.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c5959a640 1 --2- 172.21.15.76:0/957465048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c44004ec0 0x7f9c4400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:58.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c5959a640 1 -- 172.21.15.76:0/957465048 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c0ae1b0 con 0x7f9c440042d0 2023-11-09T03:51:58.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c5959a640 1 -- 172.21.15.76:0/957465048 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c0ccb10 con 0x7f9c44004ec0 2023-11-09T03:51:58.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c5959a640 1 -- 172.21.15.76:0/957465048 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4400bec0 con 0x7f9c44003970 2023-11-09T03:51:58.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c5a59c640 1 --2- 172.21.15.76:0/957465048 >> v2:172.21.15.113:3300/0 conn(0x7f9c5c0addd0 0x7f9c5c0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:51:58.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c59d9b640 1 --2- 172.21.15.76:0/957465048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c440042d0 0x7f9c440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:58.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c5ad9d640 1 --2- 172.21.15.76:0/957465048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c44004ec0 0x7f9c4400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:58.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c5a59c640 1 --2- 172.21.15.76:0/957465048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c44003970 0x7f9c44003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:58.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c59d9b640 1 -- 172.21.15.76:0/957465048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c44003970 msgr2=0x7f9c44003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:58.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c59d9b640 1 --2- 172.21.15.76:0/957465048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c44003970 0x7f9c44003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c59d9b640 1 -- 172.21.15.76:0/957465048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c44004ec0 msgr2=0x7f9c4400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:58.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c59d9b640 1 --2- 172.21.15.76:0/957465048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c44004ec0 0x7f9c4400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c59d9b640 1 -- 172.21.15.76:0/957465048 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c44003520 con 0x7f9c440042d0 2023-11-09T03:51:58.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.304+0000 7f9c5ad9d640 1 --2- 172.21.15.76:0/957465048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c44004ec0 0x7f9c4400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:51:58.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.305+0000 7f9c59d9b640 1 --2- 172.21.15.76:0/957465048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c440042d0 0x7f9c440047c0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009ac0 tx=0x7f9c440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb52424dbded7a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:58.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.305+0000 7f9c5959a640 1 -- 172.21.15.76:0/957465048 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5002c6b0 con 0x7f9c440042d0 2023-11-09T03:51:58.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.305+0000 7f9c5959a640 1 -- 172.21.15.76:0/957465048 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5002c810 con 0x7f9c440042d0 2023-11-09T03:51:58.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.305+0000 7f9c5959a640 1 -- 172.21.15.76:0/957465048 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5002cba0 con 0x7f9c440042d0 2023-11-09T03:51:58.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.306+0000 7f9c60f97640 1 -- 172.21.15.76:0/957465048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c440042d0 msgr2=0x7f9c440047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:58.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.306+0000 7f9c60f97640 1 --2- 172.21.15.76:0/957465048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c440042d0 0x7f9c440047c0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009ac0 tx=0x7f9c440057f0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.306+0000 7f9c5ad9d640 1 -- 172.21.15.76:0/957465048 reap_dead start 2023-11-09T03:51:58.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.306+0000 7f9c60f97640 1 -- 172.21.15.76:0/957465048 shutdown_connections 2023-11-09T03:51:58.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.306+0000 7f9c60f97640 1 -- 172.21.15.76:0/957465048 >> 172.21.15.76:0/957465048 conn(0x7f9c5c0cb530 msgr2=0x7f9c5c0cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:58.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.306+0000 7f9c60f97640 1 -- 172.21.15.76:0/957465048 shutdown_connections 2023-11-09T03:51:58.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.306+0000 7f9c60f97640 1 -- 172.21.15.76:0/957465048 wait complete. 2023-11-09T03:51:58.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.307+0000 7f9c60f97640 1 Processor -- start 2023-11-09T03:51:58.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.307+0000 7f9c60f97640 1 -- start start 2023-11-09T03:51:58.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.308+0000 7f9c60f97640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c440042d0 0x7f9c5c13ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:58.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.308+0000 7f9c60f97640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c5c0b16d0 0x7f9c5c15bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:58.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.308+0000 7f9c60f97640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c5c0b1b50 0x7f9c5c15c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:58.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.308+0000 7f9c60f97640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c0c6790 con 0x7f9c5c0b16d0 2023-11-09T03:51:58.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.308+0000 7f9c60f97640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c0c7030 con 0x7f9c5c0b1b50 2023-11-09T03:51:58.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.308+0000 7f9c60f97640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c0b20e0 con 0x7f9c440042d0 2023-11-09T03:51:58.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.308+0000 7f9c59d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c5c0b16d0 0x7f9c5c15bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:58.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.308+0000 7f9c59d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c5c0b16d0 0x7f9c5c15bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42082/0 (socket says 172.21.15.76:42082) 2023-11-09T03:51:58.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.308+0000 7f9c59d9b640 1 -- 172.21.15.76:0/3282363758 learned_addr learned my addr 172.21.15.76:0/3282363758 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:58.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.308+0000 7f9c5ad9d640 1 --2- 172.21.15.76:0/3282363758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c5c0b1b50 0x7f9c5c15c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:58.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.308+0000 7f9c5a59c640 1 --2- 172.21.15.76:0/3282363758 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c440042d0 0x7f9c5c13ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:58.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.308+0000 7f9c59d9b640 1 -- 172.21.15.76:0/3282363758 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c440042d0 msgr2=0x7f9c5c13ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:58.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.308+0000 7f9c59d9b640 1 --2- 172.21.15.76:0/3282363758 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c440042d0 0x7f9c5c13ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.308+0000 7f9c59d9b640 1 -- 172.21.15.76:0/3282363758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c5c0b1b50 msgr2=0x7f9c5c15c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:58.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.308+0000 7f9c59d9b640 1 --2- 172.21.15.76:0/3282363758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c5c0b1b50 0x7f9c5c15c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.308+0000 7f9c59d9b640 1 -- 172.21.15.76:0/3282363758 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5c0c69e0 con 0x7f9c5c0b16d0 2023-11-09T03:51:58.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.308+0000 7f9c5a59c640 1 --2- 172.21.15.76:0/3282363758 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c440042d0 0x7f9c5c13ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:51:58.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.309+0000 7f9c59d9b640 1 --2- 172.21.15.76:0/3282363758 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c5c0b16d0 0x7f9c5c15bcf0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f9c50037040 tx=0x7f9c50004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:58.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.309+0000 7f9c4affd640 1 -- 172.21.15.76:0/3282363758 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c50003810 con 0x7f9c5c0b16d0 2023-11-09T03:51:58.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.309+0000 7f9c4affd640 1 -- 172.21.15.76:0/3282363758 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c50003970 con 0x7f9c5c0b16d0 2023-11-09T03:51:58.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.309+0000 7f9c60f97640 1 -- 172.21.15.76:0/3282363758 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5c0c6cc0 con 0x7f9c5c0b16d0 2023-11-09T03:51:58.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.309+0000 7f9c4affd640 1 -- 172.21.15.76:0/3282363758 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c50003d00 con 0x7f9c5c0b16d0 2023-11-09T03:51:58.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.309+0000 7f9c60f97640 1 -- 172.21.15.76:0/3282363758 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5c1f06e0 con 0x7f9c5c0b16d0 2023-11-09T03:51:58.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.310+0000 7f9c60f97640 1 -- 172.21.15.76:0/3282363758 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c5c0684b0 con 0x7f9c5c0b16d0 2023-11-09T03:51:58.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.310+0000 7f9c4affd640 1 -- 172.21.15.76:0/3282363758 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9c50040450 con 0x7f9c5c0b16d0 2023-11-09T03:51:58.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.311+0000 7f9c4affd640 1 --2- 172.21.15.76:0/3282363758 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c3c06aea0 0x7f9c3c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:58.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.311+0000 7f9c4affd640 1 -- 172.21.15.76:0/3282363758 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7737+0+0 (secure 0 0 0) 0x7f9c500abbe0 con 0x7f9c5c0b16d0 2023-11-09T03:51:58.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.311+0000 7f9c5a59c640 1 --2- 172.21.15.76:0/3282363758 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c3c06aea0 0x7f9c3c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:58.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.311+0000 7f9c5a59c640 1 --2- 172.21.15.76:0/3282363758 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c3c06aea0 0x7f9c3c06d360 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0099e0 tx=0x7f9c4c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:58.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.315+0000 7f9c4affd640 1 -- 172.21.15.76:0/3282363758 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9c50076530 con 0x7f9c5c0b16d0 2023-11-09T03:51:58.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.479+0000 7f9c60f97640 1 -- 172.21.15.76:0/3282363758 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9c5c06d820 con 0x7f9c5c0b16d0 2023-11-09T03:51:58.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.480+0000 7f9c4affd640 1 -- 172.21.15.76:0/3282363758 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9c50039020 con 0x7f9c5c0b16d0 2023-11-09T03:51:58.479 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:51:58.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.482+0000 7f9c60f97640 1 -- 172.21.15.76:0/3282363758 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c3c06aea0 msgr2=0x7f9c3c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:58.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.482+0000 7f9c60f97640 1 --2- 172.21.15.76:0/3282363758 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c3c06aea0 0x7f9c3c06d360 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0099e0 tx=0x7f9c4c00b040 comp rx=0 tx=0).stop 2023-11-09T03:51:58.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.482+0000 7f9c60f97640 1 -- 172.21.15.76:0/3282363758 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c5c0b16d0 msgr2=0x7f9c5c15bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:58.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.482+0000 7f9c60f97640 1 --2- 172.21.15.76:0/3282363758 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c5c0b16d0 0x7f9c5c15bcf0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f9c50037040 tx=0x7f9c50004730 comp rx=0 tx=0).stop 2023-11-09T03:51:58.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.482+0000 7f9c60f97640 1 -- 172.21.15.76:0/3282363758 shutdown_connections 2023-11-09T03:51:58.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.482+0000 7f9c60f97640 1 --2- 172.21.15.76:0/3282363758 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c3c06aea0 0x7f9c3c06d360 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.482+0000 7f9c60f97640 1 --2- 172.21.15.76:0/3282363758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c5c0b1b50 0x7f9c5c15c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.482+0000 7f9c60f97640 1 --2- 172.21.15.76:0/3282363758 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c5c0b16d0 0x7f9c5c15bcf0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.482+0000 7f9c60f97640 1 --2- 172.21.15.76:0/3282363758 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c440042d0 0x7f9c5c13ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.482+0000 7f9c60f97640 1 -- 172.21.15.76:0/3282363758 >> 172.21.15.76:0/3282363758 conn(0x7f9c5c0cb530 msgr2=0x7f9c5c06bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:58.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.482+0000 7f9c60f97640 1 -- 172.21.15.76:0/3282363758 shutdown_connections 2023-11-09T03:51:58.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.482+0000 7f9c60f97640 1 -- 172.21.15.76:0/3282363758 wait complete. 2023-11-09T03:51:58.493 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:51:58.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.596+0000 7fa52d8d4640 1 Processor -- start 2023-11-09T03:51:58.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.596+0000 7fa52d8d4640 1 -- start start 2023-11-09T03:51:58.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.597+0000 7fa52d8d4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa52806ac50 0x7fa5280b6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:58.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.597+0000 7fa52d8d4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa5280d2ab0 0x7fa5280b6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:58.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.597+0000 7fa52d8d4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5280b6120 con 0x7fa5280b82c0 2023-11-09T03:51:58.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.597+0000 7fa52d8d4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5280d2750 con 0x7fa5280d2ab0 2023-11-09T03:51:58.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.597+0000 7fa52d8d4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5280d1d70 con 0x7fa52806ac50 2023-11-09T03:51:58.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.597+0000 7fa5277fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa5280d2ab0 0x7fa5280b6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:58.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.597+0000 7fa5267fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa52806ac50 0x7fa5280b6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:58.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.597+0000 7fa526ffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa5280b82c0 0x7fa52806a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:58376/0 (socket says 172.21.15.76:58376) 2023-11-09T03:51:58.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.597+0000 7fa5267fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa52806ac50 0x7fa5280b6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:42790/0 (socket says 172.21.15.76:42790) 2023-11-09T03:51:58.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.597+0000 7fa526ffd640 1 -- 172.21.15.76:0/2809321766 learned_addr learned my addr 172.21.15.76:0/2809321766 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:51:58.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.597+0000 7fa525ffb640 1 -- 172.21.15.76:0/2809321766 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1459375938 0 0) 0x7fa5280b6120 con 0x7fa5280b82c0 2023-11-09T03:51:58.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.598+0000 7fa525ffb640 1 -- 172.21.15.76:0/2809321766 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa510003520 con 0x7fa5280b82c0 2023-11-09T03:51:58.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.598+0000 7fa5267fc640 1 -- 172.21.15.76:0/2809321766 >> v2:172.21.15.113:3300/0 conn(0x7fa5280d2ab0 msgr2=0x7fa5280b6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:58.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.598+0000 7fa5267fc640 1 --2- 172.21.15.76:0/2809321766 >> v2:172.21.15.113:3300/0 conn(0x7fa5280d2ab0 0x7fa5280b6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.598+0000 7fa5267fc640 1 -- 172.21.15.76:0/2809321766 >> v1:172.21.15.113:6789/0 conn(0x7fa5280b82c0 legacy=0x7fa52806a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:58.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.598+0000 7fa5267fc640 1 -- 172.21.15.76:0/2809321766 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5280d3a10 con 0x7fa52806ac50 2023-11-09T03:51:58.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.598+0000 7fa5277fe640 1 --2- 172.21.15.76:0/2809321766 >> v2:172.21.15.113:3300/0 conn(0x7fa5280d2ab0 0x7fa5280b6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:51:58.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.598+0000 7fa5267fc640 1 --2- 172.21.15.76:0/2809321766 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa52806ac50 0x7fa5280b6590 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0097a0 tx=0x7fa51c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44ded123ed6f2da6 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:58.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.598+0000 7fa525ffb640 1 -- 172.21.15.76:0/2809321766 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c02d040 con 0x7fa52806ac50 2023-11-09T03:51:58.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.599+0000 7fa525ffb640 1 -- 172.21.15.76:0/2809321766 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51c031810 con 0x7fa52806ac50 2023-11-09T03:51:58.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.599+0000 7fa525ffb640 1 -- 172.21.15.76:0/2809321766 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c031b80 con 0x7fa52806ac50 2023-11-09T03:51:58.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.599+0000 7fa52d8d4640 1 -- 172.21.15.76:0/2809321766 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa52806ac50 msgr2=0x7fa5280b6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:58.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.599+0000 7fa52d8d4640 1 --2- 172.21.15.76:0/2809321766 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa52806ac50 0x7fa5280b6590 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0097a0 tx=0x7fa51c02b8b0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.599+0000 7fa52d8d4640 1 -- 172.21.15.76:0/2809321766 shutdown_connections 2023-11-09T03:51:58.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.599+0000 7fa52d8d4640 1 --2- 172.21.15.76:0/2809321766 >> v2:172.21.15.113:3300/0 conn(0x7fa5280d2ab0 0x7fa5280b6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.599+0000 7fa52d8d4640 1 --2- 172.21.15.76:0/2809321766 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa52806ac50 0x7fa5280b6590 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.599+0000 7fa52d8d4640 1 -- 172.21.15.76:0/2809321766 >> 172.21.15.76:0/2809321766 conn(0x7fa5280d2ef0 msgr2=0x7fa5280d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:58.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.599+0000 7fa52d8d4640 1 -- 172.21.15.76:0/2809321766 shutdown_connections 2023-11-09T03:51:58.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.599+0000 7fa52d8d4640 1 -- 172.21.15.76:0/2809321766 wait complete. 2023-11-09T03:51:58.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.600+0000 7fa52d8d4640 1 Processor -- start 2023-11-09T03:51:58.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.600+0000 7fa52d8d4640 1 -- start start 2023-11-09T03:51:58.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.601+0000 7fa52d8d4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa52806ac50 0x7fa5280a9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:58.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.601+0000 7fa52d8d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5280b82c0 0x7fa5281e7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:58.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.601+0000 7fa52d8d4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5280d2ab0 0x7fa5281e8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:58.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.601+0000 7fa52d8d4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5281f2bb0 con 0x7fa5280b82c0 2023-11-09T03:51:58.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.601+0000 7fa52d8d4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5280aab20 con 0x7fa52806ac50 2023-11-09T03:51:58.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.601+0000 7fa52d8d4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5280a9d40 con 0x7fa5280d2ab0 2023-11-09T03:51:58.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.601+0000 7fa5267fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5280b82c0 0x7fa5281e7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:58.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.601+0000 7fa5267fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5280b82c0 0x7fa5281e7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42090/0 (socket says 172.21.15.76:42090) 2023-11-09T03:51:58.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.601+0000 7fa5267fc640 1 -- 172.21.15.76:0/1271088100 learned_addr learned my addr 172.21.15.76:0/1271088100 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:58.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.601+0000 7fa526ffd640 1 --2- 172.21.15.76:0/1271088100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa52806ac50 0x7fa5280a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:58.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.601+0000 7fa5277fe640 1 --2- 172.21.15.76:0/1271088100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5280d2ab0 0x7fa5281e8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:58.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.601+0000 7fa5267fc640 1 -- 172.21.15.76:0/1271088100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5280d2ab0 msgr2=0x7fa5281e8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:58.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.601+0000 7fa5267fc640 1 --2- 172.21.15.76:0/1271088100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5280d2ab0 0x7fa5281e8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.601+0000 7fa5267fc640 1 -- 172.21.15.76:0/1271088100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa52806ac50 msgr2=0x7fa5280a9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:58.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.601+0000 7fa5267fc640 1 --2- 172.21.15.76:0/1271088100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa52806ac50 0x7fa5280a9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.601+0000 7fa5267fc640 1 -- 172.21.15.76:0/1271088100 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa520009400 con 0x7fa5280b82c0 2023-11-09T03:51:58.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.602+0000 7fa5267fc640 1 --2- 172.21.15.76:0/1271088100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5280b82c0 0x7fa5281e7f50 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fa51c02b8f0 tx=0x7fa51c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:58.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.602+0000 7fa50ffff640 1 -- 172.21.15.76:0/1271088100 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c031610 con 0x7fa5280b82c0 2023-11-09T03:51:58.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.602+0000 7fa50ffff640 1 -- 172.21.15.76:0/1271088100 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51c03ad60 con 0x7fa5280b82c0 2023-11-09T03:51:58.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.602+0000 7fa50ffff640 1 -- 172.21.15.76:0/1271088100 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c042610 con 0x7fa5280b82c0 2023-11-09T03:51:58.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.602+0000 7fa52d8d4640 1 -- 172.21.15.76:0/1271088100 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa51c009400 con 0x7fa5280b82c0 2023-11-09T03:51:58.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.602+0000 7fa52d8d4640 1 -- 172.21.15.76:0/1271088100 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5280aa280 con 0x7fa5280b82c0 2023-11-09T03:51:58.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.603+0000 7fa52d8d4640 1 -- 172.21.15.76:0/1271088100 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5280b1bb0 con 0x7fa5280b82c0 2023-11-09T03:51:58.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.603+0000 7fa50ffff640 1 -- 172.21.15.76:0/1271088100 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa51c04b450 con 0x7fa5280b82c0 2023-11-09T03:51:58.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.603+0000 7fa50ffff640 1 --2- 172.21.15.76:0/1271088100 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa51406ae70 0x7fa51406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:58.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.604+0000 7fa50ffff640 1 -- 172.21.15.76:0/1271088100 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7737+0+0 (secure 0 0 0) 0x7fa51c0aba10 con 0x7fa5280b82c0 2023-11-09T03:51:58.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.604+0000 7fa526ffd640 1 --2- 172.21.15.76:0/1271088100 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa51406ae70 0x7fa51406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:58.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.604+0000 7fa526ffd640 1 --2- 172.21.15.76:0/1271088100 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa51406ae70 0x7fa51406d330 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa5180012e0 tx=0x7fa518009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:58.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.608+0000 7fa50ffff640 1 -- 172.21.15.76:0/1271088100 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa51c076360 con 0x7fa5280b82c0 2023-11-09T03:51:58.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.772+0000 7fa52d8d4640 1 -- 172.21.15.76:0/1271088100 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa5281f2d00 con 0x7fa5280b82c0 2023-11-09T03:51:58.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.773+0000 7fa50ffff640 1 -- 172.21.15.76:0/1271088100 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa51c0350a0 con 0x7fa5280b82c0 2023-11-09T03:51:58.774 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:51:58.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.775+0000 7fa52d8d4640 1 -- 172.21.15.76:0/1271088100 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa51406ae70 msgr2=0x7fa51406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:58.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.775+0000 7fa52d8d4640 1 --2- 172.21.15.76:0/1271088100 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa51406ae70 0x7fa51406d330 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa5180012e0 tx=0x7fa518009040 comp rx=0 tx=0).stop 2023-11-09T03:51:58.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.775+0000 7fa52d8d4640 1 -- 172.21.15.76:0/1271088100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5280b82c0 msgr2=0x7fa5281e7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:58.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.775+0000 7fa52d8d4640 1 --2- 172.21.15.76:0/1271088100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5280b82c0 0x7fa5281e7f50 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fa51c02b8f0 tx=0x7fa51c0046c0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.775+0000 7fa52d8d4640 1 -- 172.21.15.76:0/1271088100 shutdown_connections 2023-11-09T03:51:58.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.775+0000 7fa52d8d4640 1 --2- 172.21.15.76:0/1271088100 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa51406ae70 0x7fa51406d330 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.775+0000 7fa52d8d4640 1 --2- 172.21.15.76:0/1271088100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5280d2ab0 0x7fa5281e8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.775+0000 7fa52d8d4640 1 --2- 172.21.15.76:0/1271088100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5280b82c0 0x7fa5281e7f50 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.775+0000 7fa52d8d4640 1 --2- 172.21.15.76:0/1271088100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa52806ac50 0x7fa5280a9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.775+0000 7fa52d8d4640 1 -- 172.21.15.76:0/1271088100 >> 172.21.15.76:0/1271088100 conn(0x7fa5280d2ef0 msgr2=0x7fa5280b6490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:58.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.775+0000 7fa52d8d4640 1 -- 172.21.15.76:0/1271088100 shutdown_connections 2023-11-09T03:51:58.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.775+0000 7fa52d8d4640 1 -- 172.21.15.76:0/1271088100 wait complete. 2023-11-09T03:51:58.786 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-09T03:51:58.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.886+0000 7f51b9264640 1 Processor -- start 2023-11-09T03:51:58.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.886+0000 7f51b9264640 1 -- start start 2023-11-09T03:51:58.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.887+0000 7f51b9264640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f51b415d100 0x7f51b40ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:58.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.887+0000 7f51b9264640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b40b32b0 con 0x7f51b40d1be0 2023-11-09T03:51:58.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.887+0000 7f51b9264640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b40cee10 con 0x7f51b415d4e0 2023-11-09T03:51:58.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.887+0000 7f51b9264640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51b40b2980 con 0x7f51b415d100 2023-11-09T03:51:58.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.887+0000 7f51b37fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f51b40d1be0 0x7f51b40b3740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50754/0 (socket says 172.21.15.76:50754) 2023-11-09T03:51:58.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.887+0000 7f51b37fe640 1 -- 172.21.15.76:0/1543612085 learned_addr learned my addr 172.21.15.76:0/1543612085 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:51:58.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.887+0000 7f51b2ffd640 1 --2- 172.21.15.76:0/1543612085 >> v2:172.21.15.113:3300/0 conn(0x7f51b415d100 0x7f51b40ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:58.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.888+0000 7f51b1ffb640 1 -- 172.21.15.76:0/1543612085 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3694298203 0 0) 0x7f51b40b32b0 con 0x7f51b40d1be0 2023-11-09T03:51:58.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.888+0000 7f51b1ffb640 1 -- 172.21.15.76:0/1543612085 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f519c003520 con 0x7f51b40d1be0 2023-11-09T03:51:58.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.888+0000 7f51b2ffd640 1 -- 172.21.15.76:0/1543612085 >> v1:172.21.15.113:6789/0 conn(0x7f51b415d4e0 legacy=0x7f51b40d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:58.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.888+0000 7f51b1ffb640 1 -- 172.21.15.76:0/1543612085 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3420367914 0 0) 0x7f519c003520 con 0x7f51b40d1be0 2023-11-09T03:51:58.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.888+0000 7f51b2ffd640 1 -- 172.21.15.76:0/1543612085 >> v1:172.21.15.76:6789/0 conn(0x7f51b40d1be0 legacy=0x7f51b40b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:58.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.888+0000 7f51b2ffd640 1 -- 172.21.15.76:0/1543612085 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b40b2c60 con 0x7f51b415d100 2023-11-09T03:51:58.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.889+0000 7f51b2ffd640 1 --2- 172.21.15.76:0/1543612085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51b415d100 0x7f51b40ce510 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f51a4009de0 tx=0x7f51a402be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c31a56dde013cd24 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:58.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.889+0000 7f51b1ffb640 1 -- 172.21.15.76:0/1543612085 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac001670 con 0x7f51b415d100 2023-11-09T03:51:58.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.889+0000 7f51b1ffb640 1 -- 172.21.15.76:0/1543612085 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a402c980 con 0x7f51b415d100 2023-11-09T03:51:58.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.889+0000 7f51b1ffb640 1 -- 172.21.15.76:0/1543612085 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a402ccf0 con 0x7f51b415d100 2023-11-09T03:51:58.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.889+0000 7f51b9264640 1 -- 172.21.15.76:0/1543612085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51b415d100 msgr2=0x7f51b40ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:58.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.889+0000 7f51b9264640 1 --2- 172.21.15.76:0/1543612085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51b415d100 0x7f51b40ce510 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f51a4009de0 tx=0x7f51a402be50 comp rx=0 tx=0).stop 2023-11-09T03:51:58.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.889+0000 7f51b9264640 1 -- 172.21.15.76:0/1543612085 shutdown_connections 2023-11-09T03:51:58.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.889+0000 7f51b9264640 1 --2- 172.21.15.76:0/1543612085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51b415d100 0x7f51b40ce510 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.889+0000 7f51b9264640 1 -- 172.21.15.76:0/1543612085 >> 172.21.15.76:0/1543612085 conn(0x7f51b40d2ef0 msgr2=0x7f51b40d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:58.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.889+0000 7f51b9264640 1 -- 172.21.15.76:0/1543612085 shutdown_connections 2023-11-09T03:51:58.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.889+0000 7f51b9264640 1 -- 172.21.15.76:0/1543612085 wait complete. 2023-11-09T03:51:58.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.891+0000 7f51b9264640 1 Processor -- start 2023-11-09T03:51:58.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.891+0000 7f51b9264640 1 -- start start 2023-11-09T03:51:58.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.891+0000 7f51b9264640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51b40d1be0 0x7f51b41f5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:58.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.891+0000 7f51b9264640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51b415d100 0x7f51b41f20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:58.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.891+0000 7f51b9264640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51b415d4e0 0x7f51b40b4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:58.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.891+0000 7f51b9264640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b41f69c0 con 0x7f51b415d4e0 2023-11-09T03:51:58.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.891+0000 7f51b9264640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b41f6ca0 con 0x7f51b40d1be0 2023-11-09T03:51:58.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.891+0000 7f51b9264640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b40b5c10 con 0x7f51b415d100 2023-11-09T03:51:58.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.892+0000 7f51b37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51b415d4e0 0x7f51b40b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:58.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.892+0000 7f51b37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51b415d4e0 0x7f51b40b4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42098/0 (socket says 172.21.15.76:42098) 2023-11-09T03:51:58.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.892+0000 7f51b37fe640 1 -- 172.21.15.76:0/1300037120 learned_addr learned my addr 172.21.15.76:0/1300037120 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:58.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.892+0000 7f51b2ffd640 1 --2- 172.21.15.76:0/1300037120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51b40d1be0 0x7f51b41f5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:58.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.892+0000 7f51b27fc640 1 --2- 172.21.15.76:0/1300037120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51b415d100 0x7f51b41f20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:58.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.892+0000 7f51b37fe640 1 -- 172.21.15.76:0/1300037120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51b415d100 msgr2=0x7f51b41f20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:58.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.892+0000 7f51b37fe640 1 --2- 172.21.15.76:0/1300037120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51b415d100 0x7f51b41f20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.892+0000 7f51b37fe640 1 -- 172.21.15.76:0/1300037120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51b40d1be0 msgr2=0x7f51b41f5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:58.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.892+0000 7f51b37fe640 1 --2- 172.21.15.76:0/1300037120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51b40d1be0 0x7f51b41f5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:58.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.892+0000 7f51b37fe640 1 -- 172.21.15.76:0/1300037120 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a4009a70 con 0x7f51b415d4e0 2023-11-09T03:51:58.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.892+0000 7f51b37fe640 1 --2- 172.21.15.76:0/1300037120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51b415d4e0 0x7f51b40b4090 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0017a0 tx=0x7f51ac00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:58.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.892+0000 7f519b7fe640 1 -- 172.21.15.76:0/1300037120 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac005ad0 con 0x7f51b415d4e0 2023-11-09T03:51:58.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.892+0000 7f519b7fe640 1 -- 172.21.15.76:0/1300037120 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ac005c30 con 0x7f51b415d4e0 2023-11-09T03:51:58.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.892+0000 7f51b9264640 1 -- 172.21.15.76:0/1300037120 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b40b5ef0 con 0x7f51b415d4e0 2023-11-09T03:51:58.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.892+0000 7f519b7fe640 1 -- 172.21.15.76:0/1300037120 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac007070 con 0x7f51b415d4e0 2023-11-09T03:51:58.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.892+0000 7f51b9264640 1 -- 172.21.15.76:0/1300037120 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b41f6520 con 0x7f51b415d4e0 2023-11-09T03:51:58.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.893+0000 7f51b9264640 1 -- 172.21.15.76:0/1300037120 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b40b5a70 con 0x7f51b415d4e0 2023-11-09T03:51:58.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.894+0000 7f519b7fe640 1 -- 172.21.15.76:0/1300037120 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f51ac0071d0 con 0x7f51b415d4e0 2023-11-09T03:51:58.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.894+0000 7f519b7fe640 1 --2- 172.21.15.76:0/1300037120 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f51a006ae70 0x7f51a006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:58.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.894+0000 7f519b7fe640 1 -- 172.21.15.76:0/1300037120 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7737+0+0 (secure 0 0 0) 0x7f51ac089980 con 0x7f51b415d4e0 2023-11-09T03:51:58.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.894+0000 7f51b2ffd640 1 --2- 172.21.15.76:0/1300037120 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f51a006ae70 0x7f51a006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:58.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.895+0000 7f51b2ffd640 1 --2- 172.21.15.76:0/1300037120 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f51a006ae70 0x7f51a006d330 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f51a4036000 tx=0x7f51a4030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:58.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:58.898+0000 7f519b7fe640 1 -- 172.21.15.76:0/1300037120 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f51b40b5a70 con 0x7f51b415d4e0 2023-11-09T03:51:59.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.075+0000 7f51b9264640 1 -- 172.21.15.76:0/1300037120 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f51b40b5a70 con 0x7f51b415d4e0 2023-11-09T03:51:59.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.075+0000 7f519b7fe640 1 -- 172.21.15.76:0/1300037120 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 13 entries v106) v1 ==== 81+0+792 (secure 0 0 0) 0x7f51b40b5a70 con 0x7f51b415d4e0 2023-11-09T03:51:59.076 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6832/3697850101 2023-11-10T03:51:39.442929+0000 2023-11-09T03:51:59.076 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/3449910842 2023-11-10T03:50:59.611256+0000 2023-11-09T03:51:59.077 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/3229239133 2023-11-10T03:51:32.624856+0000 2023-11-09T03:51:59.077 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/3449910842 2023-11-10T03:50:59.611256+0000 2023-11-09T03:51:59.077 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/1663988060 2023-11-10T03:50:33.665092+0000 2023-11-09T03:51:59.077 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/1663988060 2023-11-10T03:50:33.665092+0000 2023-11-09T03:51:59.077 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/3280098776 2023-11-10T03:50:07.616780+0000 2023-11-09T03:51:59.078 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/3280098776 2023-11-10T03:50:07.616780+0000 2023-11-09T03:51:59.078 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/4011275120 2023-11-10T03:49:42.029057+0000 2023-11-09T03:51:59.078 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/503424885 2023-11-09T04:51:47.727422+0000 2023-11-09T03:51:59.078 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6833/3697850101 2023-11-10T03:51:39.442929+0000 2023-11-09T03:51:59.078 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/3229239133 2023-11-10T03:51:32.624856+0000 2023-11-09T03:51:59.078 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/4011275120 2023-11-10T03:49:42.029057+0000 2023-11-09T03:51:59.079 INFO:teuthology.orchestra.run.smithi076.stderr:listed 13 entries 2023-11-09T03:51:59.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.077+0000 7f51b9264640 1 -- 172.21.15.76:0/1300037120 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f51a006ae70 msgr2=0x7f51a006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:59.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.077+0000 7f51b9264640 1 --2- 172.21.15.76:0/1300037120 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f51a006ae70 0x7f51a006d330 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f51a4036000 tx=0x7f51a4030000 comp rx=0 tx=0).stop 2023-11-09T03:51:59.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.077+0000 7f51b9264640 1 -- 172.21.15.76:0/1300037120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51b415d4e0 msgr2=0x7f51b40b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:59.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.077+0000 7f51b9264640 1 --2- 172.21.15.76:0/1300037120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51b415d4e0 0x7f51b40b4090 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0017a0 tx=0x7f51ac00cbe0 comp rx=0 tx=0).stop 2023-11-09T03:51:59.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.077+0000 7f51b9264640 1 -- 172.21.15.76:0/1300037120 shutdown_connections 2023-11-09T03:51:59.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.077+0000 7f51b9264640 1 --2- 172.21.15.76:0/1300037120 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f51a006ae70 0x7f51a006d330 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:59.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.077+0000 7f51b9264640 1 --2- 172.21.15.76:0/1300037120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51b415d4e0 0x7f51b40b4090 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:59.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.077+0000 7f51b9264640 1 --2- 172.21.15.76:0/1300037120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51b415d100 0x7f51b41f20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:59.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.077+0000 7f51b9264640 1 --2- 172.21.15.76:0/1300037120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51b40d1be0 0x7f51b41f5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:59.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.077+0000 7f51b9264640 1 -- 172.21.15.76:0/1300037120 >> 172.21.15.76:0/1300037120 conn(0x7f51b40d2ef0 msgr2=0x7f51b40d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:59.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.077+0000 7f51b9264640 1 -- 172.21.15.76:0/1300037120 shutdown_connections 2023-11-09T03:51:59.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.077+0000 7f51b9264640 1 -- 172.21.15.76:0/1300037120 wait complete. 2023-11-09T03:51:59.088 DEBUG:tasks.cephfs.mount:Force/lazy unmounting on client.0 2023-11-09T03:51:59.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:51:59.089 DEBUG:teuthology.orchestra.run.smithi076:> sudo umount --lazy --force /home/ubuntu/cephtest/mnt.0 2023-11-09T03:51:59.115 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[117777]: fuse finished with error 0 and tester_r 0 2023-11-09T03:51:59.116 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T03:51:59.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:51:59.117 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-09T03:51:59.171 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:51:59.171 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:51:59.226 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056381,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14761 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248868k,nr_inodes=812217,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-09T03:51:59.226 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-09T03:51:59.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.365+0000 7fd1aa256640 1 Processor -- start 2023-11-09T03:51:59.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.365+0000 7fd1aa256640 1 -- start start 2023-11-09T03:51:59.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.366+0000 7fd1aa256640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd1a4159e40 0x7fd1a40b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:59.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.366+0000 7fd1aa256640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a4071500 con 0x7fd1a4070700 2023-11-09T03:51:59.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.366+0000 7fd1aa256640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a415dae0 con 0x7fd1a406bb60 2023-11-09T03:51:59.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.366+0000 7fd1aa256640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a4070c60 con 0x7fd1a4159e40 2023-11-09T03:51:59.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.366+0000 7fd1a37fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd1a4159e40 0x7fd1a40b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:59.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.366+0000 7fd1a3fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd1a4070700 0x7fd1a40cd790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50760/0 (socket says 172.21.15.76:50760) 2023-11-09T03:51:59.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.366+0000 7fd1a3fff640 1 -- 172.21.15.76:0/2095026105 learned_addr learned my addr 172.21.15.76:0/2095026105 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:51:59.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.366+0000 7fd1a37fe640 1 -- 172.21.15.76:0/2095026105 >> v1:172.21.15.113:6789/0 conn(0x7fd1a406bb60 legacy=0x7fd1a406fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:51:59.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.366+0000 7fd1a37fe640 1 -- 172.21.15.76:0/2095026105 >> v1:172.21.15.76:6789/0 conn(0x7fd1a4070700 legacy=0x7fd1a40cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:59.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.366+0000 7fd1a37fe640 1 -- 172.21.15.76:0/2095026105 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a4070f40 con 0x7fd1a4159e40 2023-11-09T03:51:59.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.366+0000 7fd1a27fc640 1 -- 172.21.15.76:0/2095026105 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1434632607 0 0) 0x7fd1a4071500 con 0x7fd1a4070700 2023-11-09T03:51:59.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.367+0000 7fd1a37fe640 1 --2- 172.21.15.76:0/2095026105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1a4159e40 0x7fd1a40b0530 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fd1940097a0 tx=0x7fd19402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0cc8501a331efa8 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:59.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.367+0000 7fd1a27fc640 1 -- 172.21.15.76:0/2095026105 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19402f050 con 0x7fd1a4159e40 2023-11-09T03:51:59.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.367+0000 7fd1a27fc640 1 -- 172.21.15.76:0/2095026105 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd194031700 con 0x7fd1a4159e40 2023-11-09T03:51:59.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.367+0000 7fd1a27fc640 1 -- 172.21.15.76:0/2095026105 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd194031a90 con 0x7fd1a4159e40 2023-11-09T03:51:59.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.368+0000 7fd1aa256640 1 -- 172.21.15.76:0/2095026105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1a4159e40 msgr2=0x7fd1a40b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:59.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.368+0000 7fd1aa256640 1 --2- 172.21.15.76:0/2095026105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1a4159e40 0x7fd1a40b0530 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fd1940097a0 tx=0x7fd19402b7a0 comp rx=0 tx=0).stop 2023-11-09T03:51:59.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.368+0000 7fd1aa256640 1 -- 172.21.15.76:0/2095026105 shutdown_connections 2023-11-09T03:51:59.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.368+0000 7fd1aa256640 1 --2- 172.21.15.76:0/2095026105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1a4159e40 0x7fd1a40b0530 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:59.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.368+0000 7fd1aa256640 1 -- 172.21.15.76:0/2095026105 >> 172.21.15.76:0/2095026105 conn(0x7fd1a40cb530 msgr2=0x7fd1a40cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:59.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.368+0000 7fd1aa256640 1 -- 172.21.15.76:0/2095026105 shutdown_connections 2023-11-09T03:51:59.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.368+0000 7fd1aa256640 1 -- 172.21.15.76:0/2095026105 wait complete. 2023-11-09T03:51:59.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.369+0000 7fd1aa256640 1 Processor -- start 2023-11-09T03:51:59.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.369+0000 7fd1aa256640 1 -- start start 2023-11-09T03:51:59.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.370+0000 7fd1aa256640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1a406bb60 0x7fd1a40aaf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:59.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.370+0000 7fd1aa256640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1a4070700 0x7fd1a40cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:59.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.370+0000 7fd1aa256640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1a4159e40 0x7fd1a40a8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:59.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.370+0000 7fd1aa256640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a40a9b50 con 0x7fd1a4159e40 2023-11-09T03:51:59.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.370+0000 7fd1aa256640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a40ab4b0 con 0x7fd1a406bb60 2023-11-09T03:51:59.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.370+0000 7fd1aa256640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a40c3cd0 con 0x7fd1a4070700 2023-11-09T03:51:59.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.370+0000 7fd1a3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1a4159e40 0x7fd1a40a8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:59.376 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.370+0000 7fd1a3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1a4159e40 0x7fd1a40a8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42122/0 (socket says 172.21.15.76:42122) 2023-11-09T03:51:59.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.370+0000 7fd1a3fff640 1 -- 172.21.15.76:0/1211867702 learned_addr learned my addr 172.21.15.76:0/1211867702 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:51:59.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.370+0000 7fd1a37fe640 1 --2- 172.21.15.76:0/1211867702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1a406bb60 0x7fd1a40aaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:59.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.370+0000 7fd1a3fff640 1 -- 172.21.15.76:0/1211867702 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1a4070700 msgr2=0x7fd1a40cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:59.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.370+0000 7fd1a3fff640 1 --2- 172.21.15.76:0/1211867702 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1a4070700 0x7fd1a40cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:59.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.370+0000 7fd1a3fff640 1 -- 172.21.15.76:0/1211867702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1a406bb60 msgr2=0x7fd1a40aaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:59.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.370+0000 7fd1a3fff640 1 --2- 172.21.15.76:0/1211867702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1a406bb60 0x7fd1a40aaf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:59.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.370+0000 7fd1a3fff640 1 -- 172.21.15.76:0/1211867702 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd194009400 con 0x7fd1a4159e40 2023-11-09T03:51:59.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.370+0000 7fd1a37fe640 1 --2- 172.21.15.76:0/1211867702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1a406bb60 0x7fd1a40aaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:51:59.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.370+0000 7fd1a3fff640 1 --2- 172.21.15.76:0/1211867702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1a4159e40 0x7fd1a40a8ed0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fd19c00c7e0 tx=0x7fd19c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:59.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.371+0000 7fd18bfff640 1 -- 172.21.15.76:0/1211867702 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c00a640 con 0x7fd1a4159e40 2023-11-09T03:51:59.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.371+0000 7fd18bfff640 1 -- 172.21.15.76:0/1211867702 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19c00a7c0 con 0x7fd1a4159e40 2023-11-09T03:51:59.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.371+0000 7fd1aa256640 1 -- 172.21.15.76:0/1211867702 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a40c3fb0 con 0x7fd1a4159e40 2023-11-09T03:51:59.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.371+0000 7fd18bfff640 1 -- 172.21.15.76:0/1211867702 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c00ab70 con 0x7fd1a4159e40 2023-11-09T03:51:59.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.371+0000 7fd1aa256640 1 -- 172.21.15.76:0/1211867702 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a40c4570 con 0x7fd1a4159e40 2023-11-09T03:51:59.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.372+0000 7fd1aa256640 1 -- 172.21.15.76:0/1211867702 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a40c4790 con 0x7fd1a4159e40 2023-11-09T03:51:59.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.372+0000 7fd18bfff640 1 -- 172.21.15.76:0/1211867702 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd19c00acd0 con 0x7fd1a4159e40 2023-11-09T03:51:59.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.372+0000 7fd18bfff640 1 --2- 172.21.15.76:0/1211867702 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd19006ae70 0x7fd19006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:51:59.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.372+0000 7fd18bfff640 1 -- 172.21.15.76:0/1211867702 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7737+0+0 (secure 0 0 0) 0x7fd19c089670 con 0x7fd1a4159e40 2023-11-09T03:51:59.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.373+0000 7fd1a37fe640 1 --2- 172.21.15.76:0/1211867702 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd19006ae70 0x7fd19006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:51:59.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.373+0000 7fd1a37fe640 1 --2- 172.21.15.76:0/1211867702 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd19006ae70 0x7fd19006d330 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd194035000 tx=0x7fd194008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:51:59.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.377+0000 7fd18bfff640 1 -- 172.21.15.76:0/1211867702 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd19c053f30 con 0x7fd1a4159e40 2023-11-09T03:51:59.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.578+0000 7fd1aa256640 1 -- 172.21.15.76:0/1211867702 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd1a40684b0 con 0x7fd1a4159e40 2023-11-09T03:51:59.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.578+0000 7fd18bfff640 1 -- 172.21.15.76:0/1211867702 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v93) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd19c057550 con 0x7fd1a4159e40 2023-11-09T03:51:59.579 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-09T03:51:59.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.580+0000 7fd1aa256640 1 -- 172.21.15.76:0/1211867702 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd19006ae70 msgr2=0x7fd19006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:59.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.580+0000 7fd1aa256640 1 --2- 172.21.15.76:0/1211867702 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd19006ae70 0x7fd19006d330 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd194035000 tx=0x7fd194008030 comp rx=0 tx=0).stop 2023-11-09T03:51:59.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.580+0000 7fd1aa256640 1 -- 172.21.15.76:0/1211867702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1a4159e40 msgr2=0x7fd1a40a8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:51:59.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.580+0000 7fd1aa256640 1 --2- 172.21.15.76:0/1211867702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1a4159e40 0x7fd1a40a8ed0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fd19c00c7e0 tx=0x7fd19c00cca0 comp rx=0 tx=0).stop 2023-11-09T03:51:59.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.580+0000 7fd1aa256640 1 -- 172.21.15.76:0/1211867702 shutdown_connections 2023-11-09T03:51:59.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.580+0000 7fd1aa256640 1 --2- 172.21.15.76:0/1211867702 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd19006ae70 0x7fd19006d330 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:59.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.580+0000 7fd1aa256640 1 --2- 172.21.15.76:0/1211867702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1a4159e40 0x7fd1a40a8ed0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:59.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.580+0000 7fd1aa256640 1 --2- 172.21.15.76:0/1211867702 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1a4070700 0x7fd1a40cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:59.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.580+0000 7fd1aa256640 1 --2- 172.21.15.76:0/1211867702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1a406bb60 0x7fd1a40aaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:51:59.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.580+0000 7fd1aa256640 1 -- 172.21.15.76:0/1211867702 >> 172.21.15.76:0/1211867702 conn(0x7fd1a40cb530 msgr2=0x7fd1a406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:51:59.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.580+0000 7fd1aa256640 1 -- 172.21.15.76:0/1211867702 shutdown_connections 2023-11-09T03:51:59.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:51:59.580+0000 7fd1aa256640 1 -- 172.21.15.76:0/1211867702 wait complete. 2023-11-09T03:51:59.591 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-09T03:51:59.591 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-09T03:51:59.591 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2023-11-09T03:51:59.592 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-09T03:51:59.592 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-09T03:51:59.592 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-09T03:51:59.593 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-09T03:51:59.593 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-09T03:51:59.593 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:51:59.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:51:59.594 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2023-11-09T03:51:59.609 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-09T03:51:59.609 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-09T03:51:59.609 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2023-11-09T03:51:59.610 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:51:59.610 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2023-11-09T03:51:59.610 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:51:59.610 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:51:59.610 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:51:59.611 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2023-11-09T03:51:59.611 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:51:59.611 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:51:59.611 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2023-11-09T03:51:59.611 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:51:59.611 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:51:59.612 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-09T03:51:59.612 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:51:59.612 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-09T03:51:59.612 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42229sec preferred_lft 42229sec 2023-11-09T03:51:59.612 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2023-11-09T03:51:59.612 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:51:59.613 INFO:teuthology.orchestra.run.smithi076.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-09T03:51:59.613 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2e:15:bd:03:a8:2e brd ff:ff:ff:ff:ff:ff 2023-11-09T03:51:59.613 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-09T03:51:59.613 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:51:59.613 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::bcda:38ff:fed4:fdb0/64 scope link 2023-11-09T03:51:59.614 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:51:59.614 INFO:teuthology.orchestra.run.smithi076.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-09T03:51:59.614 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 6a:8d:94:d7:e5:00 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:51:59.614 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::688d:94ff:fed7:e500/64 scope link 2023-11-09T03:51:59.614 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:51:59.614 INFO:teuthology.orchestra.run.smithi076.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-09T03:51:59.615 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2e:15:bd:03:a8:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:51:59.615 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::2c15:bdff:fe03:a82e/64 scope link 2023-11-09T03:51:59.615 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:51:59.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:51:59.616 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-09T03:51:59.671 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:51:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:51:59.675 INFO:teuthology.orchestra.run.smithi076.stdout:1 2023-11-09T03:51:59.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:51:59.677 DEBUG:teuthology.orchestra.run.smithi076:> route 2023-11-09T03:51:59.734 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2023-11-09T03:51:59.734 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-09T03:51:59.735 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-09T03:51:59.735 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-09T03:51:59.735 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-09T03:51:59.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:51:59.736 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-09T03:51:59.736 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-09T03:51:59.736 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-09T03:51:59.737 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-09T03:51:59.737 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-09T03:51:59.737 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-09T03:51:59.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:51:59.831 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2023-11-09T03:51:59.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-09T03:51:59.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-09T03:51:59.846 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:51:59.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:51:59.847 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2023-11-09T03:51:59.912 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-09T03:51:59.912 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:51:59.913 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-09T03:51:59.967 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:51:59.967 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:51:59.967 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-09T03:52:00.022 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:52:00.077 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:52:00.077 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:52:00.077 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:52:00.078 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:52:00.078 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:52:00.078 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:52:00.078 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:52:00.078 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:52:00.078 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:52:00.079 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:52:00.079 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:52:00.079 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:52:00.079 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:52:00.079 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:52:00.080 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:52:00.080 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:52:00.080 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:52:00.080 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:52:00.080 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:52:00.080 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:52:00.081 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:52:00.081 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:52:00.081 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:52:00.081 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:52:00.081 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:52:00.081 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:52:00.082 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:52:00.082 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:52:00.082 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:52:00.082 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:52:00.082 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:52:00.083 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:52:00.083 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:52:00.083 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:52:00.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:52:00.084 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:52:00.139 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-09T03:52:00.139 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-09T03:52:00.181 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:52:00.196 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:52:00.196 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:52:00.196 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:52:00.197 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:52:00.197 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:52:00.197 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:52:00.197 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:52:00.197 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:52:00.198 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:52:00.198 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:52:00.198 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:52:00.198 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:52:00.198 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:52:00.199 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:52:00.199 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:52:00.199 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:52:00.199 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:52:00.199 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:52:00.199 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:52:00.200 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:52:00.200 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:52:00.200 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:52:00.200 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:52:00.200 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:52:00.200 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:52:00.201 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:52:00.201 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:52:00.201 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:52:00.201 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:52:00.201 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:52:00.201 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:52:00.202 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:52:00.202 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:52:00.202 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:52:00.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:52:00.203 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:52:00.270 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:52:00.270+0000 7fe439664480 -1 init, newargv = 0x557b5ee9ce20 newargc=15 2023-11-09T03:52:00.271 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[123533]: starting ceph client 2023-11-09T03:52:00.277 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[123533]: starting fuse 2023-11-09T03:52:01.353 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:52:01.373 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:52:01.373 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:52:01.374 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:52:01.374 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:52:01.374 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:52:01.374 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:52:01.374 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:52:01.375 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:52:01.375 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:52:01.375 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:52:01.375 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:52:01.375 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:52:01.376 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:52:01.376 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:52:01.376 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:52:01.376 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:52:01.376 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:52:01.376 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:52:01.377 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:52:01.377 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:52:01.377 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:52:01.377 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:52:01.377 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:52:01.378 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:52:01.378 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:52:01.378 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:52:01.378 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:52:01.378 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:52:01.378 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:52:01.379 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:52:01.379 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:52:01.379 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:52:01.379 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:52:01.379 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:52:01.379 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-09T03:52:01.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:52:01.380 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:52:01.430 INFO:teuthology.orchestra.run.smithi076.stdout:43 2023-11-09T03:52:01.431 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-09T03:52:01.431 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer -- python3 -c ' 2023-11-09T03:52:01.431 DEBUG:teuthology.orchestra.run.smithi076:> import glob 2023-11-09T03:52:01.432 DEBUG:teuthology.orchestra.run.smithi076:> import re 2023-11-09T03:52:01.432 DEBUG:teuthology.orchestra.run.smithi076:> import os 2023-11-09T03:52:01.433 DEBUG:teuthology.orchestra.run.smithi076:> import subprocess 2023-11-09T03:52:01.433 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:52:01.433 DEBUG:teuthology.orchestra.run.smithi076:> def _find_admin_socket(client_name): 2023-11-09T03:52:01.434 DEBUG:teuthology.orchestra.run.smithi076:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-09T03:52:01.434 DEBUG:teuthology.orchestra.run.smithi076:> files = glob.glob(asok_path) 2023-11-09T03:52:01.434 DEBUG:teuthology.orchestra.run.smithi076:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-09T03:52:01.435 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:52:01.435 DEBUG:teuthology.orchestra.run.smithi076:> # Given a non-glob path, it better be there 2023-11-09T03:52:01.435 DEBUG:teuthology.orchestra.run.smithi076:> if "*" not in asok_path: 2023-11-09T03:52:01.436 DEBUG:teuthology.orchestra.run.smithi076:> assert(len(files) == 1) 2023-11-09T03:52:01.436 DEBUG:teuthology.orchestra.run.smithi076:> return files[0] 2023-11-09T03:52:01.436 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:52:01.437 DEBUG:teuthology.orchestra.run.smithi076:> for f in files: 2023-11-09T03:52:01.437 DEBUG:teuthology.orchestra.run.smithi076:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-09T03:52:01.437 DEBUG:teuthology.orchestra.run.smithi076:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-09T03:52:01.438 DEBUG:teuthology.orchestra.run.smithi076:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-09T03:52:01.438 DEBUG:teuthology.orchestra.run.smithi076:> contents = proc_f.read() 2023-11-09T03:52:01.438 DEBUG:teuthology.orchestra.run.smithi076:> if mountpoint in contents: 2023-11-09T03:52:01.439 DEBUG:teuthology.orchestra.run.smithi076:> return f 2023-11-09T03:52:01.439 DEBUG:teuthology.orchestra.run.smithi076:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-09T03:52:01.439 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:52:01.440 DEBUG:teuthology.orchestra.run.smithi076:> print(_find_admin_socket("client.0")) 2023-11-09T03:52:01.440 DEBUG:teuthology.orchestra.run.smithi076:> ' 2023-11-09T03:52:01.559 INFO:teuthology.orchestra.run.smithi076.stdout:/var/run/ceph/ceph-client.0.123533.asok 2023-11-09T03:52:01.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:52:01.570 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.123533.asok 2023-11-09T03:52:01.570 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:52:01.570 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.123533.asok status 2023-11-09T03:52:01.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:52:01.740 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:52:01.740 INFO:teuthology.orchestra.run.smithi076.stdout: "metadata": { 2023-11-09T03:52:01.740 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_sha1": "7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4", 2023-11-09T03:52:01.740 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev)", 2023-11-09T03:52:01.740 INFO:teuthology.orchestra.run.smithi076.stdout: "entity_id": "0", 2023-11-09T03:52:01.741 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": "smithi076", 2023-11-09T03:52:01.741 INFO:teuthology.orchestra.run.smithi076.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-09T03:52:01.741 INFO:teuthology.orchestra.run.smithi076.stdout: "pid": "123533", 2023-11-09T03:52:01.741 INFO:teuthology.orchestra.run.smithi076.stdout: "root": "/" 2023-11-09T03:52:01.741 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:52:01.741 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_count": 0, 2023-11-09T03:52:01.742 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_pinned_count": 0, 2023-11-09T03:52:01.742 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 8445, 2023-11-09T03:52:01.742 INFO:teuthology.orchestra.run.smithi076.stdout: "inst": { 2023-11-09T03:52:01.742 INFO:teuthology.orchestra.run.smithi076.stdout: "name": { 2023-11-09T03:52:01.742 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "client", 2023-11-09T03:52:01.743 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 8445 2023-11-09T03:52:01.743 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:52:01.743 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2023-11-09T03:52:01.743 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2023-11-09T03:52:01.743 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2023-11-09T03:52:01.743 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3087804343 2023-11-09T03:52:01.744 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:52:01.744 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:52:01.744 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2023-11-09T03:52:01.744 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2023-11-09T03:52:01.744 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2023-11-09T03:52:01.745 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3087804343 2023-11-09T03:52:01.745 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:52:01.745 INFO:teuthology.orchestra.run.smithi076.stdout: "inst_str": "client.8445 192.168.0.1:0/3087804343", 2023-11-09T03:52:01.745 INFO:teuthology.orchestra.run.smithi076.stdout: "addr_str": "192.168.0.1:0/3087804343", 2023-11-09T03:52:01.745 INFO:teuthology.orchestra.run.smithi076.stdout: "inode_count": 1, 2023-11-09T03:52:01.745 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_epoch": 93, 2023-11-09T03:52:01.746 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch": 106, 2023-11-09T03:52:01.746 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch_barrier": 0, 2023-11-09T03:52:01.746 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklisted": false, 2023-11-09T03:52:01.746 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs" 2023-11-09T03:52:01.746 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:52:01.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:52:01.754 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:52:01.754 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-11-09T03:52:01.754 DEBUG:teuthology.orchestra.run.smithi076:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-09T03:52:01.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:52:01.814 INFO:teuthology.orchestra.run.smithi076.stdout:fuseblk 2023-11-09T03:52:01.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:52:01.820 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-09T03:52:01.821 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:52:01.821 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-09T03:52:01.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:52:01.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:52:01.886 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 flush journal 2023-11-09T03:52:01.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.987+0000 7f4dbdbb6640 1 Processor -- start 2023-11-09T03:52:01.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.987+0000 7f4dbdbb6640 1 -- start start 2023-11-09T03:52:01.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.988+0000 7f4dbdbb6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4db815d110 0x7f4db806b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:01.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.988+0000 7f4dbdbb6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4db80b3ca0 0x7f4db80ace70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:01.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.988+0000 7f4dbdbb6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4db815d4f0 con 0x7f4db80ad3b0 2023-11-09T03:52:01.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.988+0000 7f4dbdbb6640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4db80664f0 con 0x7f4db80b3ca0 2023-11-09T03:52:01.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.988+0000 7f4dbdbb6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db806b9f0 con 0x7f4db815d110 2023-11-09T03:52:01.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.988+0000 7f4db6ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4db80b3ca0 0x7f4db80ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:01.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.988+0000 7f4db6ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4db80b3ca0 0x7f4db80ace70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42128/0 (socket says 172.21.15.76:42128) 2023-11-09T03:52:01.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.988+0000 7f4db6ffd640 1 -- 172.21.15.76:0/947363697 learned_addr learned my addr 172.21.15.76:0/947363697 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:01.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.989+0000 7f4db6ffd640 1 -- 172.21.15.76:0/947363697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4db815d110 msgr2=0x7f4db806b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:01.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.989+0000 7f4db6ffd640 1 --2- 172.21.15.76:0/947363697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4db815d110 0x7f4db806b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:01.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.989+0000 7f4db6ffd640 1 -- 172.21.15.76:0/947363697 >> v1:172.21.15.113:6789/0 conn(0x7f4db80ad3b0 legacy=0x7f4db80b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:01.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.989+0000 7f4db6ffd640 1 -- 172.21.15.76:0/947363697 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db806c2e0 con 0x7f4db80b3ca0 2023-11-09T03:52:01.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.989+0000 7f4db6ffd640 1 --2- 172.21.15.76:0/947363697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4db80b3ca0 0x7f4db80ace70 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f4dac009870 tx=0x7f4dac02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b1351c1b674bbf server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:01.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.989+0000 7f4db67fc640 1 -- 172.21.15.76:0/947363697 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac031640 con 0x7f4db80b3ca0 2023-11-09T03:52:01.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.990+0000 7f4db67fc640 1 -- 172.21.15.76:0/947363697 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dac0317a0 con 0x7f4db80b3ca0 2023-11-09T03:52:01.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.990+0000 7f4db67fc640 1 -- 172.21.15.76:0/947363697 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac031b30 con 0x7f4db80b3ca0 2023-11-09T03:52:01.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.990+0000 7f4dbdbb6640 1 -- 172.21.15.76:0/947363697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4db80b3ca0 msgr2=0x7f4db80ace70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:01.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.990+0000 7f4dbdbb6640 1 --2- 172.21.15.76:0/947363697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4db80b3ca0 0x7f4db80ace70 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f4dac009870 tx=0x7f4dac02b8f0 comp rx=0 tx=0).stop 2023-11-09T03:52:01.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.990+0000 7f4dbdbb6640 1 -- 172.21.15.76:0/947363697 shutdown_connections 2023-11-09T03:52:01.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.990+0000 7f4dbdbb6640 1 --2- 172.21.15.76:0/947363697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4db80b3ca0 0x7f4db80ace70 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:01.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.990+0000 7f4dbdbb6640 1 --2- 172.21.15.76:0/947363697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4db815d110 0x7f4db806b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:01.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.990+0000 7f4dbdbb6640 1 -- 172.21.15.76:0/947363697 >> 172.21.15.76:0/947363697 conn(0x7f4db80cbd30 msgr2=0x7f4db80cc140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:01.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.990+0000 7f4dbdbb6640 1 -- 172.21.15.76:0/947363697 shutdown_connections 2023-11-09T03:52:01.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.990+0000 7f4dbdbb6640 1 -- 172.21.15.76:0/947363697 wait complete. 2023-11-09T03:52:01.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.991+0000 7f4dbdbb6640 1 Processor -- start 2023-11-09T03:52:01.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.991+0000 7f4dbdbb6640 1 -- start start 2023-11-09T03:52:01.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.992+0000 7f4dbdbb6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4db80ad3b0 0x7f4db81598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:01.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.992+0000 7f4dbdbb6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4db80b3ca0 0x7f4db80c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:01.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.992+0000 7f4dbdbb6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4db815d110 0x7f4db815ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:01.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.992+0000 7f4dbdbb6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db8159e30 con 0x7f4db80b3ca0 2023-11-09T03:52:01.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.992+0000 7f4dbdbb6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db80c6e80 con 0x7f4db80ad3b0 2023-11-09T03:52:01.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.992+0000 7f4dbdbb6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db815dae0 con 0x7f4db815d110 2023-11-09T03:52:01.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.992+0000 7f4db6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4db80b3ca0 0x7f4db80c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:01.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.992+0000 7f4db6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4db80b3ca0 0x7f4db80c6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42140/0 (socket says 172.21.15.76:42140) 2023-11-09T03:52:01.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.992+0000 7f4db6ffd640 1 -- 172.21.15.76:0/3762300112 learned_addr learned my addr 172.21.15.76:0/3762300112 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:01.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.992+0000 7f4db77fe640 1 --2- 172.21.15.76:0/3762300112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4db80ad3b0 0x7f4db81598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:01.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.992+0000 7f4db7fff640 1 --2- 172.21.15.76:0/3762300112 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4db815d110 0x7f4db815ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:01.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.993+0000 7f4db6ffd640 1 -- 172.21.15.76:0/3762300112 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4db815d110 msgr2=0x7f4db815ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:01.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.993+0000 7f4db6ffd640 1 --2- 172.21.15.76:0/3762300112 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4db815d110 0x7f4db815ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:01.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.993+0000 7f4db6ffd640 1 -- 172.21.15.76:0/3762300112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4db80ad3b0 msgr2=0x7f4db81598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:01.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.993+0000 7f4db6ffd640 1 --2- 172.21.15.76:0/3762300112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4db80ad3b0 0x7f4db81598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:01.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.993+0000 7f4db6ffd640 1 -- 172.21.15.76:0/3762300112 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dac0094d0 con 0x7f4db80b3ca0 2023-11-09T03:52:02.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.993+0000 7f4db77fe640 1 --2- 172.21.15.76:0/3762300112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4db80ad3b0 0x7f4db81598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:52:02.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.993+0000 7f4db6ffd640 1 --2- 172.21.15.76:0/3762300112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4db80b3ca0 0x7f4db80c6940 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f4dac009870 tx=0x7f4dac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:02.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.993+0000 7f4d9ffff640 1 -- 172.21.15.76:0/3762300112 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac03ac60 con 0x7f4db80b3ca0 2023-11-09T03:52:02.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.993+0000 7f4d9ffff640 1 -- 172.21.15.76:0/3762300112 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dac03ade0 con 0x7f4db80b3ca0 2023-11-09T03:52:02.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.993+0000 7f4d9ffff640 1 -- 172.21.15.76:0/3762300112 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac044630 con 0x7f4db80b3ca0 2023-11-09T03:52:02.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.993+0000 7f4dbdbb6640 1 -- 172.21.15.76:0/3762300112 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4db815ddc0 con 0x7f4db80b3ca0 2023-11-09T03:52:02.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.993+0000 7f4dbdbb6640 1 -- 172.21.15.76:0/3762300112 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db815e380 con 0x7f4db80b3ca0 2023-11-09T03:52:02.001 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:52:02.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.994+0000 7f4d9ffff640 1 -- 172.21.15.76:0/3762300112 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4dac02f050 con 0x7f4db80b3ca0 2023-11-09T03:52:02.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.995+0000 7f4d9ffff640 1 --2- 172.21.15.76:0/3762300112 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4da406ae70 0x7f4da406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:02.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.995+0000 7f4d9ffff640 1 -- 172.21.15.76:0/3762300112 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7737+0+0 (secure 0 0 0) 0x7f4dac0abf30 con 0x7f4db80b3ca0 2023-11-09T03:52:02.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.995+0000 7f4db77fe640 1 --2- 172.21.15.76:0/3762300112 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4da406ae70 0x7f4da406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:02.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.995+0000 7f4db77fe640 1 --2- 172.21.15.76:0/3762300112 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4da406ae70 0x7f4da406d330 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f4da8006fd0 tx=0x7f4da8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:02.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.996+0000 7f4dc0827740 1 Processor -- start 2023-11-09T03:52:02.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.996+0000 7f4dc0827740 1 -- start start 2023-11-09T03:52:02.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.997+0000 7f4dc0827740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d70cd4b4c0 0x55d70cd4b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:02.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.997+0000 7f4dc0827740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d70cd1b6d0 0x55d70cd48730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:02.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.997+0000 7f4dc0827740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d70cd1f040 0x55d70cd48c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:02.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.997+0000 7f4dc0827740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70cc8ed70 con 0x55d70cd4b4c0 2023-11-09T03:52:02.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.997+0000 7f4dc0827740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70cc8c460 con 0x55d70cd1f040 2023-11-09T03:52:02.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.997+0000 7f4dc0827740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d70cc87d50 con 0x55d70cd1b6d0 2023-11-09T03:52:02.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.997+0000 7f4db77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d70cd4b4c0 0x55d70cd4b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:02.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.997+0000 7f4db77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d70cd4b4c0 0x55d70cd4b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42150/0 (socket says 172.21.15.76:42150) 2023-11-09T03:52:02.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.997+0000 7f4db77fe640 1 -- 172.21.15.76:0/2940314193 learned_addr learned my addr 172.21.15.76:0/2940314193 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:02.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.997+0000 7f4db6ffd640 1 --2- 172.21.15.76:0/2940314193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d70cd1b6d0 0x55d70cd48730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:02.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.997+0000 7f4db7fff640 1 --2- 172.21.15.76:0/2940314193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d70cd1f040 0x55d70cd48c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:02.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.997+0000 7f4db77fe640 1 -- 172.21.15.76:0/2940314193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d70cd1b6d0 msgr2=0x55d70cd48730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:02.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.997+0000 7f4db77fe640 1 --2- 172.21.15.76:0/2940314193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d70cd1b6d0 0x55d70cd48730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.997+0000 7f4db77fe640 1 -- 172.21.15.76:0/2940314193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d70cd1f040 msgr2=0x55d70cd48c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:02.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.997+0000 7f4db77fe640 1 --2- 172.21.15.76:0/2940314193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d70cd1f040 0x55d70cd48c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.997+0000 7f4db77fe640 1 -- 172.21.15.76:0/2940314193 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d70cc4ae30 con 0x55d70cd4b4c0 2023-11-09T03:52:02.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.997+0000 7f4db77fe640 1 --2- 172.21.15.76:0/2940314193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d70cd4b4c0 0x55d70cd4b8a0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f4da8009750 tx=0x7f4da8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ffc0835570f7b14 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:02.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.997+0000 7f4d9d7fa640 1 -- 172.21.15.76:0/2940314193 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da8005a10 con 0x55d70cd4b4c0 2023-11-09T03:52:02.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.997+0000 7f4d9d7fa640 1 -- 172.21.15.76:0/2940314193 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da8005b70 con 0x55d70cd4b4c0 2023-11-09T03:52:02.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.997+0000 7f4d9d7fa640 1 -- 172.21.15.76:0/2940314193 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da80149b0 con 0x55d70cd4b4c0 2023-11-09T03:52:02.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.998+0000 7f4dc0827740 1 -- 172.21.15.76:0/2940314193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d70cd4b4c0 msgr2=0x55d70cd4b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:02.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.998+0000 7f4dc0827740 1 --2- 172.21.15.76:0/2940314193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d70cd4b4c0 0x55d70cd4b8a0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f4da8009750 tx=0x7f4da8010000 comp rx=0 tx=0).stop 2023-11-09T03:52:02.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.998+0000 7f4dc0827740 1 -- 172.21.15.76:0/2940314193 shutdown_connections 2023-11-09T03:52:02.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.998+0000 7f4dc0827740 1 --2- 172.21.15.76:0/2940314193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d70cd1f040 0x55d70cd48c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.998+0000 7f4dc0827740 1 --2- 172.21.15.76:0/2940314193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d70cd1b6d0 0x55d70cd48730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.998+0000 7f4dc0827740 1 --2- 172.21.15.76:0/2940314193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d70cd4b4c0 0x55d70cd4b8a0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.998+0000 7f4dc0827740 1 -- 172.21.15.76:0/2940314193 >> 172.21.15.76:0/2940314193 conn(0x55d70cd1d9e0 msgr2=0x55d70cd662d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:02.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.998+0000 7f4dc0827740 1 -- 172.21.15.76:0/2940314193 shutdown_connections 2023-11-09T03:52:02.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.998+0000 7f4dc0827740 1 -- 172.21.15.76:0/2940314193 wait complete. 2023-11-09T03:52:02.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.999+0000 7f4dc0827740 1 -- start start 2023-11-09T03:52:02.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.999+0000 7f4dc0827740 1 Processor -- start 2023-11-09T03:52:02.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.999+0000 7f4dc0827740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d70cd1f040 0x55d70cd79a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:02.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.999+0000 7f4dc0827740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d70cd4b4c0 0x55d70cd775d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:02.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.999+0000 7f4dc0827740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d70cd43650 0x55d70cd43ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:02.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.999+0000 7f4dc0827740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70cc41ee0 con 0x55d70cd1f040 2023-11-09T03:52:02.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.999+0000 7f4dc0827740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70cc41b80 con 0x55d70cd4b4c0 2023-11-09T03:52:02.015 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.999+0000 7f4dc0827740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d70cc073a0 con 0x55d70cd43650 2023-11-09T03:52:02.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.999+0000 7f4dc0827740 10 client.0 _ll_register_callbacks cb 0x55d70cc0b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:52:02.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:01.999+0000 7f4db77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d70cd1f040 0x55d70cd79a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:02.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.000+0000 7f4db77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d70cd1f040 0x55d70cd79a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42160/0 (socket says 172.21.15.76:42160) 2023-11-09T03:52:02.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.000+0000 7f4db77fe640 1 -- 172.21.15.76:0/2426871358 learned_addr learned my addr 172.21.15.76:0/2426871358 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:02.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.000+0000 7f4db7fff640 1 --2- 172.21.15.76:0/2426871358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d70cd43650 0x55d70cd43ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:02.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.000+0000 7f4db6ffd640 1 --2- 172.21.15.76:0/2426871358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d70cd4b4c0 0x55d70cd775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:02.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.000+0000 7f4db77fe640 1 -- 172.21.15.76:0/2426871358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d70cd43650 msgr2=0x55d70cd43ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:02.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.000+0000 7f4db77fe640 1 --2- 172.21.15.76:0/2426871358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d70cd43650 0x55d70cd43ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.000+0000 7f4db77fe640 1 -- 172.21.15.76:0/2426871358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d70cd4b4c0 msgr2=0x55d70cd775d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:02.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.000+0000 7f4db77fe640 1 --2- 172.21.15.76:0/2426871358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d70cd4b4c0 0x55d70cd775d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.000+0000 7f4db77fe640 1 -- 172.21.15.76:0/2426871358 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da800e040 con 0x55d70cd1f040 2023-11-09T03:52:02.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.000+0000 7f4db7fff640 1 --2- 172.21.15.76:0/2426871358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d70cd43650 0x55d70cd43ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:52:02.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.000+0000 7f4db77fe640 1 --2- 172.21.15.76:0/2426871358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d70cd1f040 0x55d70cd79a60 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f4da8014800 tx=0x7f4da80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e77d6b9c185b115 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:02.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.000+0000 7f4d967fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:52:02.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.000+0000 7f4d967fc640 1 -- 172.21.15.76:0/2426871358 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da801f690 con 0x55d70cd1f040 2023-11-09T03:52:02.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.000+0000 7f4dc0827740 1 -- 172.21.15.76:0/2426871358 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d70cc80d30 con 0x55d70cd1f040 2023-11-09T03:52:02.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.000+0000 7f4d967fc640 1 -- 172.21.15.76:0/2426871358 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 7737+0+0 (secure 0 0 0) 0x7f4da801f7f0 con 0x55d70cd1f040 2023-11-09T03:52:02.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.000+0000 7f4d967fc640 1 -- 172.21.15.76:0/2426871358 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4da801f9b0 con 0x55d70cd1f040 2023-11-09T03:52:02.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.000+0000 7f4dc0827740 10 client.8457 fetch_fsmap learned FSMap version 93 2023-11-09T03:52:02.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.000+0000 7f4dc0827740 1 -- 172.21.15.76:0/2426871358 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x55d70cc863d0 con 0x55d70cd1f040 2023-11-09T03:52:02.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.000+0000 7f4d967fc640 1 -- 172.21.15.76:0/2426871358 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d70cc80d30 con 0x55d70cd1f040 2023-11-09T03:52:02.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.000+0000 7f4d967fc640 1 -- 172.21.15.76:0/2426871358 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 93) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f4da8027040 con 0x55d70cd1f040 2023-11-09T03:52:02.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.001+0000 7f4dc0827740 10 client.8457 fetch_fsmap finished waiting for FSMap version 93 2023-11-09T03:52:02.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.001+0000 7f4dc0827740 10 client.8457 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2023-11-09T03:52:02.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.001+0000 7f4dc0827740 20 client.8457 populate_metadata read hostname 'smithi076' 2023-11-09T03:52:02.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.001+0000 7f4dc0827740 1 --2- 172.21.15.76:0/2426871358 >> [v2:172.21.15.113:6836/445306598,v1:172.21.15.113:6837/445306598] conn(0x55d70cd77d90 0x55d70cd7a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:02.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.001+0000 7f4dc0827740 4 client.8457 mds_command: new command op to 6904 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:52:02.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.001+0000 7f4dc0827740 1 -- 172.21.15.76:0/2426871358 --> [v2:172.21.15.113:6836/445306598,v1:172.21.15.113:6837/445306598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d70cc80a50 con 0x55d70cd77d90 2023-11-09T03:52:02.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.001+0000 7f4db6ffd640 1 --2- 172.21.15.76:0/2426871358 >> [v2:172.21.15.113:6836/445306598,v1:172.21.15.113:6837/445306598] conn(0x55d70cd77d90 0x55d70cd7a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:02.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.001+0000 7f4d967fc640 10 client.8457 ms_handle_connect on v2:172.21.15.113:6836/445306598 2023-11-09T03:52:02.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.001+0000 7f4db6ffd640 1 --2- 172.21.15.76:0/2426871358 >> [v2:172.21.15.113:6836/445306598,v1:172.21.15.113:6837/445306598] conn(0x55d70cd77d90 0x55d70cd7a610 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c1c192c4101cd52 server_cookie=a17a94bf678f6f09 in_seq=0 out_seq=0 2023-11-09T03:52:02.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.002+0000 7f4d967fc640 1 -- 172.21.15.76:0/2426871358 <== mds.0 v2:172.21.15.113:6836/445306598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d70cc80a50 con 0x55d70cd77d90 2023-11-09T03:52:02.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.002+0000 7f4d967fc640 10 client.8457 handle_command_reply: tid=0 2023-11-09T03:52:02.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.002+0000 7f4dc0827740 1 client.8457 shutdown 2023-11-09T03:52:02.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.002+0000 7f4dc0827740 1 -- 172.21.15.76:0/2426871358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d70cd1f040 msgr2=0x55d70cd79a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:02.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.002+0000 7f4dc0827740 1 --2- 172.21.15.76:0/2426871358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d70cd1f040 0x55d70cd79a60 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f4da8014800 tx=0x7f4da80046c0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.002+0000 7f4dc0827740 1 -- 172.21.15.76:0/2426871358 shutdown_connections 2023-11-09T03:52:02.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.002+0000 7f4dc0827740 1 --2- 172.21.15.76:0/2426871358 >> [v2:172.21.15.113:6836/445306598,v1:172.21.15.113:6837/445306598] conn(0x55d70cd77d90 0x55d70cd7a610 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.002+0000 7f4dc0827740 1 --2- 172.21.15.76:0/2426871358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d70cd43650 0x55d70cd43ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.002+0000 7f4d967fc640 0 client.8457 ms_handle_reset on v2:172.21.15.113:6836/445306598 2023-11-09T03:52:02.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.002+0000 7f4dc0827740 1 --2- 172.21.15.76:0/2426871358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d70cd4b4c0 0x55d70cd775d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.002+0000 7f4dc0827740 1 --2- 172.21.15.76:0/2426871358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d70cd1f040 0x55d70cd79a60 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.002+0000 7f4dc0827740 1 -- 172.21.15.76:0/2426871358 >> 172.21.15.76:0/2426871358 conn(0x55d70cd1b6d0 msgr2=0x55d70cd1f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:02.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.002+0000 7f4dc0827740 1 -- 172.21.15.76:0/2426871358 shutdown_connections 2023-11-09T03:52:02.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.002+0000 7f4dc0827740 1 -- 172.21.15.76:0/2426871358 wait complete. 2023-11-09T03:52:02.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.002+0000 7f4dc0827740 20 client.8457 trim_cache size 0 max 16384 2023-11-09T03:52:02.025 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:52:02.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.015+0000 7f4dc0827740 1 Processor -- start 2023-11-09T03:52:02.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.015+0000 7f4dc0827740 1 -- start start 2023-11-09T03:52:02.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.016+0000 7f4dc0827740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d70cd1f040 0x55d70cd40c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:02.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.016+0000 7f4dc0827740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d70cd43650 0x55d70cd48100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:02.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.016+0000 7f4dc0827740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d70cd4b4c0 0x55d70cd80640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:02.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.016+0000 7f4dc0827740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70cd79a10 con 0x55d70cd4b4c0 2023-11-09T03:52:02.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.016+0000 7f4dc0827740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70cd41150 con 0x55d70cd43650 2023-11-09T03:52:02.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.016+0000 7f4dc0827740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d70cd69ff0 con 0x55d70cd1f040 2023-11-09T03:52:02.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.016+0000 7f4db7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d70cd4b4c0 0x55d70cd80640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:02.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.016+0000 7f4db7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d70cd4b4c0 0x55d70cd80640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42162/0 (socket says 172.21.15.76:42162) 2023-11-09T03:52:02.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.016+0000 7f4db7fff640 1 -- 172.21.15.76:0/2358475158 learned_addr learned my addr 172.21.15.76:0/2358475158 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:02.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.016+0000 7f4db7fff640 1 -- 172.21.15.76:0/2358475158 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d70cd1f040 msgr2=0x55d70cd40c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:02.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.016+0000 7f4db7fff640 1 --2- 172.21.15.76:0/2358475158 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d70cd1f040 0x55d70cd40c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.016+0000 7f4db7fff640 1 -- 172.21.15.76:0/2358475158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d70cd43650 msgr2=0x55d70cd48100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:02.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.016+0000 7f4db7fff640 1 --2- 172.21.15.76:0/2358475158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d70cd43650 0x55d70cd48100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.016+0000 7f4db7fff640 1 -- 172.21.15.76:0/2358475158 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d70cc23ea0 con 0x55d70cd4b4c0 2023-11-09T03:52:02.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.017+0000 7f4db7fff640 1 --2- 172.21.15.76:0/2358475158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d70cd4b4c0 0x55d70cd80640 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f4db0009f00 tx=0x7f4db000e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd2fb8961b432e41 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:02.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.017+0000 7f4d9dffb640 1 -- 172.21.15.76:0/2358475158 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db0011050 con 0x55d70cd4b4c0 2023-11-09T03:52:02.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.017+0000 7f4d9dffb640 1 -- 172.21.15.76:0/2358475158 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db0012040 con 0x55d70cd4b4c0 2023-11-09T03:52:02.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.017+0000 7f4d9dffb640 1 -- 172.21.15.76:0/2358475158 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db0016a30 con 0x55d70cd4b4c0 2023-11-09T03:52:02.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.017+0000 7f4dc0827740 1 -- 172.21.15.76:0/2358475158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d70cd4b4c0 msgr2=0x55d70cd80640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:02.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.017+0000 7f4dc0827740 1 --2- 172.21.15.76:0/2358475158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d70cd4b4c0 0x55d70cd80640 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f4db0009f00 tx=0x7f4db000e040 comp rx=0 tx=0).stop 2023-11-09T03:52:02.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.018+0000 7f4dc0827740 1 -- 172.21.15.76:0/2358475158 shutdown_connections 2023-11-09T03:52:02.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.018+0000 7f4dc0827740 1 --2- 172.21.15.76:0/2358475158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d70cd4b4c0 0x55d70cd80640 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.018+0000 7f4dc0827740 1 --2- 172.21.15.76:0/2358475158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d70cd43650 0x55d70cd48100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.018+0000 7f4dc0827740 1 --2- 172.21.15.76:0/2358475158 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d70cd1f040 0x55d70cd40c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.018+0000 7f4dc0827740 1 -- 172.21.15.76:0/2358475158 >> 172.21.15.76:0/2358475158 conn(0x55d70cd1b6d0 msgr2=0x55d70cd697e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:02.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.018+0000 7f4dc0827740 1 -- 172.21.15.76:0/2358475158 shutdown_connections 2023-11-09T03:52:02.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.018+0000 7f4dc0827740 1 -- 172.21.15.76:0/2358475158 wait complete. 2023-11-09T03:52:02.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.018+0000 7f4dc0827740 1 -- start start 2023-11-09T03:52:02.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.018+0000 7f4dc0827740 1 Processor -- start 2023-11-09T03:52:02.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.019+0000 7f4dc0827740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d70cd1f040 0x55d70cd78170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:02.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.019+0000 7f4dc0827740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d70cd43650 0x55d70cdd2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:02.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.019+0000 7f4dc0827740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d70cd4b4c0 0x55d70cd867d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:02.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.019+0000 7f4dc0827740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70cd78c20 con 0x55d70cd1f040 2023-11-09T03:52:02.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.019+0000 7f4dc0827740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70cdd24e0 con 0x55d70cd4b4c0 2023-11-09T03:52:02.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.019+0000 7f4dc0827740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d70cd4ed50 con 0x55d70cd43650 2023-11-09T03:52:02.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.019+0000 7f4dc0827740 10 client.0 _ll_register_callbacks cb 0x7f4da8009180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:52:02.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.019+0000 7f4db77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d70cd1f040 0x55d70cd78170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:02.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.019+0000 7f4db77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d70cd1f040 0x55d70cd78170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42174/0 (socket says 172.21.15.76:42174) 2023-11-09T03:52:02.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.019+0000 7f4db77fe640 1 -- 172.21.15.76:0/725663414 learned_addr learned my addr 172.21.15.76:0/725663414 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:02.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.019+0000 7f4db6ffd640 1 --2- 172.21.15.76:0/725663414 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d70cd43650 0x55d70cdd2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:02.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.019+0000 7f4db7fff640 1 --2- 172.21.15.76:0/725663414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d70cd4b4c0 0x55d70cd867d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:02.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.019+0000 7f4db77fe640 1 -- 172.21.15.76:0/725663414 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d70cd43650 msgr2=0x55d70cdd2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:02.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.019+0000 7f4db77fe640 1 --2- 172.21.15.76:0/725663414 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d70cd43650 0x55d70cdd2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.019+0000 7f4db77fe640 1 -- 172.21.15.76:0/725663414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d70cd4b4c0 msgr2=0x55d70cd867d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:02.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.019+0000 7f4db77fe640 1 --2- 172.21.15.76:0/725663414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d70cd4b4c0 0x55d70cd867d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.019+0000 7f4db77fe640 1 -- 172.21.15.76:0/725663414 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db0009b60 con 0x55d70cd1f040 2023-11-09T03:52:02.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.019+0000 7f4db7fff640 1 --2- 172.21.15.76:0/725663414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d70cd4b4c0 0x55d70cd867d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:52:02.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.020+0000 7f4db77fe640 1 --2- 172.21.15.76:0/725663414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d70cd1f040 0x55d70cd78170 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f4da8014df0 tx=0x7f4da8004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd3c5d5d3746324 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:02.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.020+0000 7f4d967fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:52:02.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.020+0000 7f4d967fc640 1 -- 172.21.15.76:0/725663414 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da80095a0 con 0x55d70cd1f040 2023-11-09T03:52:02.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.020+0000 7f4d967fc640 1 -- 172.21.15.76:0/725663414 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 7737+0+0 (secure 0 0 0) 0x7f4da801f690 con 0x55d70cd1f040 2023-11-09T03:52:02.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.020+0000 7f4dc0827740 1 -- 172.21.15.76:0/725663414 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4da8005b70 con 0x55d70cd1f040 2023-11-09T03:52:02.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.020+0000 7f4d967fc640 1 -- 172.21.15.76:0/725663414 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4da8005a10 con 0x55d70cd1f040 2023-11-09T03:52:02.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.020+0000 7f4dc0827740 10 client.8463 fetch_fsmap learned FSMap version 93 2023-11-09T03:52:02.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.020+0000 7f4dc0827740 1 -- 172.21.15.76:0/725663414 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x55d70cc49690 con 0x55d70cd1f040 2023-11-09T03:52:02.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.020+0000 7f4d967fc640 1 -- 172.21.15.76:0/725663414 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da8005b70 con 0x55d70cd1f040 2023-11-09T03:52:02.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.020+0000 7f4d967fc640 1 -- 172.21.15.76:0/725663414 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 93) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f4da8027040 con 0x55d70cd1f040 2023-11-09T03:52:02.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.020+0000 7f4dc0827740 10 client.8463 fetch_fsmap finished waiting for FSMap version 93 2023-11-09T03:52:02.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.020+0000 7f4dc0827740 10 client.8463 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2023-11-09T03:52:02.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.020+0000 7f4dc0827740 20 client.8463 populate_metadata read hostname 'smithi076' 2023-11-09T03:52:02.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.020+0000 7f4dc0827740 1 --2- 172.21.15.76:0/725663414 >> [v2:172.21.15.113:6836/445306598,v1:172.21.15.113:6837/445306598] conn(0x55d70cd40c10 0x55d70cd86d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:02.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.020+0000 7f4dc0827740 4 client.8463 mds_command: new command op to 6904 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-11-09T03:52:02.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.020+0000 7f4dc0827740 1 -- 172.21.15.76:0/725663414 --> [v2:172.21.15.113:6836/445306598,v1:172.21.15.113:6837/445306598] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d70c9887e0 con 0x55d70cd40c10 2023-11-09T03:52:02.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.020+0000 7f4db6ffd640 1 --2- 172.21.15.76:0/725663414 >> [v2:172.21.15.113:6836/445306598,v1:172.21.15.113:6837/445306598] conn(0x55d70cd40c10 0x55d70cd86d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:02.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.021+0000 7f4d967fc640 10 client.8463 ms_handle_connect on v2:172.21.15.113:6836/445306598 2023-11-09T03:52:02.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.021+0000 7f4db6ffd640 1 --2- 172.21.15.76:0/725663414 >> [v2:172.21.15.113:6836/445306598,v1:172.21.15.113:6837/445306598] conn(0x55d70cd40c10 0x55d70cd86d10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=819ec818b84cb115 server_cookie=ee43c4260cf9eb85 in_seq=0 out_seq=0 2023-11-09T03:52:02.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.032+0000 7f4d967fc640 1 -- 172.21.15.76:0/725663414 <== mds.0 v2:172.21.15.113:6836/445306598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d70c9887e0 con 0x55d70cd40c10 2023-11-09T03:52:02.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.032+0000 7f4d967fc640 10 client.8463 handle_command_reply: tid=0 2023-11-09T03:52:02.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.032+0000 7f4dc0827740 1 client.8463 shutdown 2023-11-09T03:52:02.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.032+0000 7f4dc0827740 1 -- 172.21.15.76:0/725663414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d70cd1f040 msgr2=0x55d70cd78170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:02.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.032+0000 7f4dc0827740 1 --2- 172.21.15.76:0/725663414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d70cd1f040 0x55d70cd78170 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f4da8014df0 tx=0x7f4da8004820 comp rx=0 tx=0).stop 2023-11-09T03:52:02.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.032+0000 7f4dc0827740 1 -- 172.21.15.76:0/725663414 shutdown_connections 2023-11-09T03:52:02.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.032+0000 7f4dc0827740 1 --2- 172.21.15.76:0/725663414 >> [v2:172.21.15.113:6836/445306598,v1:172.21.15.113:6837/445306598] conn(0x55d70cd40c10 0x55d70cd86d10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.032+0000 7f4dc0827740 1 --2- 172.21.15.76:0/725663414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d70cd4b4c0 0x55d70cd867d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.032+0000 7f4dc0827740 1 --2- 172.21.15.76:0/725663414 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d70cd43650 0x55d70cdd2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.032+0000 7f4d967fc640 0 client.8463 ms_handle_reset on v2:172.21.15.113:6836/445306598 2023-11-09T03:52:02.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.032+0000 7f4dc0827740 1 --2- 172.21.15.76:0/725663414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d70cd1f040 0x55d70cd78170 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.032+0000 7f4dc0827740 1 -- 172.21.15.76:0/725663414 >> 172.21.15.76:0/725663414 conn(0x55d70cd1b6d0 msgr2=0x55d70cd697e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:02.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.033+0000 7f4dc0827740 1 -- 172.21.15.76:0/725663414 shutdown_connections 2023-11-09T03:52:02.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.033+0000 7f4dc0827740 1 -- 172.21.15.76:0/725663414 wait complete. 2023-11-09T03:52:02.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.033+0000 7f4dc0827740 20 client.8463 trim_cache size 0 max 16384 2023-11-09T03:52:02.046 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:52:02.046 INFO:teuthology.orchestra.run.smithi076.stdout: "message": "", 2023-11-09T03:52:02.046 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2023-11-09T03:52:02.046 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:52:02.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.033+0000 7f4dbdbb6640 1 -- 172.21.15.76:0/3762300112 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4da406ae70 msgr2=0x7f4da406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:02.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.033+0000 7f4dbdbb6640 1 --2- 172.21.15.76:0/3762300112 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4da406ae70 0x7f4da406d330 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f4da8006fd0 tx=0x7f4da8008040 comp rx=0 tx=0).stop 2023-11-09T03:52:02.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.033+0000 7f4dbdbb6640 1 -- 172.21.15.76:0/3762300112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4db80b3ca0 msgr2=0x7f4db80c6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:02.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.033+0000 7f4dbdbb6640 1 --2- 172.21.15.76:0/3762300112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4db80b3ca0 0x7f4db80c6940 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f4dac009870 tx=0x7f4dac0046c0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.033+0000 7f4dbdbb6640 1 -- 172.21.15.76:0/3762300112 shutdown_connections 2023-11-09T03:52:02.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.033+0000 7f4dbdbb6640 1 --2- 172.21.15.76:0/3762300112 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4da406ae70 0x7f4da406d330 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.033+0000 7f4dbdbb6640 1 --2- 172.21.15.76:0/3762300112 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4db815d110 0x7f4db815ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.033+0000 7f4dbdbb6640 1 --2- 172.21.15.76:0/3762300112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4db80b3ca0 0x7f4db80c6940 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.033+0000 7f4dbdbb6640 1 --2- 172.21.15.76:0/3762300112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4db80ad3b0 0x7f4db81598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.033+0000 7f4dbdbb6640 1 -- 172.21.15.76:0/3762300112 >> 172.21.15.76:0/3762300112 conn(0x7f4db80cbd30 msgr2=0x7f4db80b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:02.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.033+0000 7f4dbdbb6640 1 -- 172.21.15.76:0/3762300112 shutdown_connections 2023-11-09T03:52:02.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.033+0000 7f4dbdbb6640 1 -- 172.21.15.76:0/3762300112 wait complete. 2023-11-09T03:52:02.050 INFO:tasks.cephfs.test_snapshots:testing snapserver mds_kill_mdstable_at=5, also kill snapclient 2023-11-09T03:52:02.051 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:52:02.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.199+0000 7f2aa9b8a640 1 Processor -- start 2023-11-09T03:52:02.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.199+0000 7f2aa9b8a640 1 -- start start 2023-11-09T03:52:02.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.200+0000 7f2aa9b8a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2aa4159e40 0x7f2aa40b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:02.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.200+0000 7f2aa9b8a640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aa4071500 con 0x7f2aa4070700 2023-11-09T03:52:02.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.200+0000 7f2aa9b8a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aa415dae0 con 0x7f2aa406bb60 2023-11-09T03:52:02.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.200+0000 7f2aa9b8a640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aa4070c60 con 0x7f2aa4159e40 2023-11-09T03:52:02.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.200+0000 7f2aa37fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2aa4159e40 0x7f2aa40b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:02.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.200+0000 7f2aa3fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2aa4070700 0x7f2aa40cd790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50762/0 (socket says 172.21.15.76:50762) 2023-11-09T03:52:02.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.200+0000 7f2aa3fff640 1 -- 172.21.15.76:0/1766097162 learned_addr learned my addr 172.21.15.76:0/1766097162 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:52:02.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.200+0000 7f2aa37fe640 1 -- 172.21.15.76:0/1766097162 >> v1:172.21.15.113:6789/0 conn(0x7f2aa406bb60 legacy=0x7f2aa406fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:02.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.200+0000 7f2aa37fe640 1 -- 172.21.15.76:0/1766097162 >> v1:172.21.15.76:6789/0 conn(0x7f2aa4070700 legacy=0x7f2aa40cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:02.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.200+0000 7f2aa37fe640 1 -- 172.21.15.76:0/1766097162 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa4070f40 con 0x7f2aa4159e40 2023-11-09T03:52:02.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.200+0000 7f2aa27fc640 1 -- 172.21.15.76:0/1766097162 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3896464130 0 0) 0x7f2aa4071500 con 0x7f2aa4070700 2023-11-09T03:52:02.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.201+0000 7f2aa37fe640 1 --2- 172.21.15.76:0/1766097162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2aa4159e40 0x7f2aa40b0530 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f2a940097a0 tx=0x7f2a9402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfe3affe3b0cdaa server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:02.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.201+0000 7f2aa27fc640 1 -- 172.21.15.76:0/1766097162 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9402f050 con 0x7f2aa4159e40 2023-11-09T03:52:02.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.201+0000 7f2aa27fc640 1 -- 172.21.15.76:0/1766097162 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a94031700 con 0x7f2aa4159e40 2023-11-09T03:52:02.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.201+0000 7f2aa27fc640 1 -- 172.21.15.76:0/1766097162 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a94031a90 con 0x7f2aa4159e40 2023-11-09T03:52:02.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.202+0000 7f2aa9b8a640 1 -- 172.21.15.76:0/1766097162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2aa4159e40 msgr2=0x7f2aa40b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:02.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.202+0000 7f2aa9b8a640 1 --2- 172.21.15.76:0/1766097162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2aa4159e40 0x7f2aa40b0530 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f2a940097a0 tx=0x7f2a9402b7a0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.202+0000 7f2aa9b8a640 1 -- 172.21.15.76:0/1766097162 shutdown_connections 2023-11-09T03:52:02.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.202+0000 7f2aa9b8a640 1 --2- 172.21.15.76:0/1766097162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2aa4159e40 0x7f2aa40b0530 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.202+0000 7f2aa9b8a640 1 -- 172.21.15.76:0/1766097162 >> 172.21.15.76:0/1766097162 conn(0x7f2aa40cb530 msgr2=0x7f2aa40cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:02.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.202+0000 7f2aa9b8a640 1 -- 172.21.15.76:0/1766097162 shutdown_connections 2023-11-09T03:52:02.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.202+0000 7f2aa9b8a640 1 -- 172.21.15.76:0/1766097162 wait complete. 2023-11-09T03:52:02.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.203+0000 7f2aa9b8a640 1 Processor -- start 2023-11-09T03:52:02.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.203+0000 7f2aa9b8a640 1 -- start start 2023-11-09T03:52:02.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.204+0000 7f2aa9b8a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2aa406bb60 0x7f2aa40c1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:02.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.204+0000 7f2aa9b8a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2aa4070700 0x7f2aa41f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:02.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.204+0000 7f2aa9b8a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2aa4159e40 0x7f2aa41581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:02.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.204+0000 7f2aa9b8a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa41f04c0 con 0x7f2aa406bb60 2023-11-09T03:52:02.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.204+0000 7f2aa9b8a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa40c1940 con 0x7f2aa4070700 2023-11-09T03:52:02.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.204+0000 7f2aa9b8a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa41ef840 con 0x7f2aa4159e40 2023-11-09T03:52:02.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.204+0000 7f2aa37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2aa406bb60 0x7f2aa40c1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:02.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.204+0000 7f2aa37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2aa406bb60 0x7f2aa40c1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42192/0 (socket says 172.21.15.76:42192) 2023-11-09T03:52:02.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.204+0000 7f2aa37fe640 1 -- 172.21.15.76:0/3311998074 learned_addr learned my addr 172.21.15.76:0/3311998074 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:02.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.204+0000 7f2aa37fe640 1 -- 172.21.15.76:0/3311998074 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2aa4159e40 msgr2=0x7f2aa41581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:02.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.204+0000 7f2aa37fe640 1 --2- 172.21.15.76:0/3311998074 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2aa4159e40 0x7f2aa41581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.204+0000 7f2aa37fe640 1 -- 172.21.15.76:0/3311998074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2aa4070700 msgr2=0x7f2aa41f34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:02.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.204+0000 7f2aa37fe640 1 --2- 172.21.15.76:0/3311998074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2aa4070700 0x7f2aa41f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.204+0000 7f2aa37fe640 1 -- 172.21.15.76:0/3311998074 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a94009400 con 0x7f2aa406bb60 2023-11-09T03:52:02.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.204+0000 7f2aa37fe640 1 --2- 172.21.15.76:0/3311998074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2aa406bb60 0x7f2aa40c1400 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f2a940098c0 tx=0x7f2a940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:02.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.204+0000 7f2a8bfff640 1 -- 172.21.15.76:0/3311998074 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a94033c30 con 0x7f2aa406bb60 2023-11-09T03:52:02.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.204+0000 7f2a8bfff640 1 -- 172.21.15.76:0/3311998074 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a94033db0 con 0x7f2aa406bb60 2023-11-09T03:52:02.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.204+0000 7f2aa9b8a640 1 -- 172.21.15.76:0/3311998074 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aa41efb20 con 0x7f2aa406bb60 2023-11-09T03:52:02.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.204+0000 7f2a8bfff640 1 -- 172.21.15.76:0/3311998074 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9403d6b0 con 0x7f2aa406bb60 2023-11-09T03:52:02.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.205+0000 7f2aa9b8a640 1 -- 172.21.15.76:0/3311998074 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa41f00e0 con 0x7f2aa406bb60 2023-11-09T03:52:02.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.205+0000 7f2aa9b8a640 1 -- 172.21.15.76:0/3311998074 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aa41f0600 con 0x7f2aa406bb60 2023-11-09T03:52:02.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.206+0000 7f2a8bfff640 1 -- 172.21.15.76:0/3311998074 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2a9403a040 con 0x7f2aa406bb60 2023-11-09T03:52:02.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.206+0000 7f2a8bfff640 1 --2- 172.21.15.76:0/3311998074 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2a9006ae70 0x7f2a9006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:02.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.206+0000 7f2a8bfff640 1 -- 172.21.15.76:0/3311998074 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7737+0+0 (secure 0 0 0) 0x7f2a940abc30 con 0x7f2aa406bb60 2023-11-09T03:52:02.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.206+0000 7f2aa2ffd640 1 --2- 172.21.15.76:0/3311998074 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2a9006ae70 0x7f2a9006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:02.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.207+0000 7f2aa2ffd640 1 --2- 172.21.15.76:0/3311998074 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2a9006ae70 0x7f2a9006d330 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2aa40c3b30 tx=0x7f2a98007990 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:02.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.210+0000 7f2a8bfff640 1 -- 172.21.15.76:0/3311998074 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2aa41f0600 con 0x7f2aa406bb60 2023-11-09T03:52:02.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.411+0000 7f2aa9b8a640 1 -- 172.21.15.76:0/3311998074 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2aa41f0600 con 0x7f2aa406bb60 2023-11-09T03:52:02.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.412+0000 7f2a8bfff640 1 -- 172.21.15.76:0/3311998074 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4929 (secure 0 0 0) 0x7f2aa41f0600 con 0x7f2aa406bb60 2023-11-09T03:52:02.412 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:52:02.413 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":93,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8310,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:51:49.482369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6904,"mds_1":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6904":{"gid":6904,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":16,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8274":{"gid":8274,"name":"c","rank":1,"incarnation":89,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6833/1536527256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1536527256},{"type":"v1","addr":"172.21.15.76:6833","nonce":1536527256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:52:02.413 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 93 2023-11-09T03:52:02.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.413+0000 7f2aa9b8a640 1 -- 172.21.15.76:0/3311998074 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2a9006ae70 msgr2=0x7f2a9006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:02.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.413+0000 7f2aa9b8a640 1 --2- 172.21.15.76:0/3311998074 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2a9006ae70 0x7f2a9006d330 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2aa40c3b30 tx=0x7f2a98007990 comp rx=0 tx=0).stop 2023-11-09T03:52:02.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.413+0000 7f2aa9b8a640 1 -- 172.21.15.76:0/3311998074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2aa406bb60 msgr2=0x7f2aa40c1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:02.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.413+0000 7f2aa9b8a640 1 --2- 172.21.15.76:0/3311998074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2aa406bb60 0x7f2aa40c1400 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f2a940098c0 tx=0x7f2a940046c0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.413+0000 7f2aa9b8a640 1 -- 172.21.15.76:0/3311998074 shutdown_connections 2023-11-09T03:52:02.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.413+0000 7f2aa9b8a640 1 --2- 172.21.15.76:0/3311998074 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2a9006ae70 0x7f2a9006d330 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.413+0000 7f2aa9b8a640 1 --2- 172.21.15.76:0/3311998074 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2aa4159e40 0x7f2aa41581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.415 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.413+0000 7f2aa9b8a640 1 --2- 172.21.15.76:0/3311998074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2aa4070700 0x7f2aa41f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.413+0000 7f2aa9b8a640 1 --2- 172.21.15.76:0/3311998074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2aa406bb60 0x7f2aa40c1400 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.413+0000 7f2aa9b8a640 1 -- 172.21.15.76:0/3311998074 >> 172.21.15.76:0/3311998074 conn(0x7f2aa40cb530 msgr2=0x7f2aa40ae4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:02.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.414+0000 7f2aa9b8a640 1 -- 172.21.15.76:0/3311998074 shutdown_connections 2023-11-09T03:52:02.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.414+0000 7f2aa9b8a640 1 -- 172.21.15.76:0/3311998074 wait complete. 2023-11-09T03:52:02.425 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump snaps --server 2023-11-09T03:52:02.551 INFO:teuthology.orchestra.run.smithi113.stdout:{"last_snap":10,"last_created":5,"last_destroyed":9,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-09T03:52:02.551 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 5, "last_destroyed": 9, "last_snap": 10, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [] } 2023-11-09T03:52:02.551 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:0 true 2023-11-09T03:52:02.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.653+0000 7f8ffb9d7640 1 Processor -- start 2023-11-09T03:52:02.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.653+0000 7f8ffb9d7640 1 -- start start 2023-11-09T03:52:02.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.654+0000 7f8ffb9d7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8ff40b0c40 0x7f8ff406fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:02.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.654+0000 7f8ffb9d7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff4071500 con 0x7f8ff406bb60 2023-11-09T03:52:02.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.654+0000 7f8ffb9d7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff415dae0 con 0x7f8ff4159e40 2023-11-09T03:52:02.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.654+0000 7f8ffb9d7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff4070c60 con 0x7f8ff40b0c40 2023-11-09T03:52:02.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.654+0000 7f8ff9f74640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8ff406bb60 0x7f8ff40cd790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47330/0 (socket says 172.21.15.76:47330) 2023-11-09T03:52:02.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.654+0000 7f8ff9f74640 1 -- 172.21.15.76:0/1696090181 learned_addr learned my addr 172.21.15.76:0/1696090181 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:52:02.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.654+0000 7f8ff8f72640 1 --2- 172.21.15.76:0/1696090181 >> v2:172.21.15.113:3300/0 conn(0x7f8ff40b0c40 0x7f8ff406fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:02.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.654+0000 7f8febfff640 1 -- 172.21.15.76:0/1696090181 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2592697556 0 0) 0x7f8ff4071500 con 0x7f8ff406bb60 2023-11-09T03:52:02.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.654+0000 7f8febfff640 1 -- 172.21.15.76:0/1696090181 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fe0003520 con 0x7f8ff406bb60 2023-11-09T03:52:02.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.655+0000 7f8ff8f72640 1 -- 172.21.15.76:0/1696090181 >> v1:172.21.15.113:6789/0 conn(0x7f8ff4159e40 legacy=0x7f8ff40b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:02.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.655+0000 7f8ff8f72640 1 -- 172.21.15.76:0/1696090181 >> v1:172.21.15.76:6789/0 conn(0x7f8ff406bb60 legacy=0x7f8ff40cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:02.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.655+0000 7f8ff8f72640 1 -- 172.21.15.76:0/1696090181 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff4070f40 con 0x7f8ff40b0c40 2023-11-09T03:52:02.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.655+0000 7f8ff8f72640 1 --2- 172.21.15.76:0/1696090181 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ff40b0c40 0x7f8ff406fff0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40097a0 tx=0x7f8fe402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3aad2a7593066ff server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:02.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.656+0000 7f8febfff640 1 -- 172.21.15.76:0/1696090181 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4031830 con 0x7f8ff40b0c40 2023-11-09T03:52:02.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.656+0000 7f8febfff640 1 -- 172.21.15.76:0/1696090181 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe4031990 con 0x7f8ff40b0c40 2023-11-09T03:52:02.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.656+0000 7f8febfff640 1 -- 172.21.15.76:0/1696090181 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4031d20 con 0x7f8ff40b0c40 2023-11-09T03:52:02.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.656+0000 7f8ffb9d7640 1 -- 172.21.15.76:0/1696090181 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ff40b0c40 msgr2=0x7f8ff406fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:02.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.656+0000 7f8ffb9d7640 1 --2- 172.21.15.76:0/1696090181 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ff40b0c40 0x7f8ff406fff0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40097a0 tx=0x7f8fe402b8d0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.656+0000 7f8ffb9d7640 1 -- 172.21.15.76:0/1696090181 shutdown_connections 2023-11-09T03:52:02.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.656+0000 7f8ffb9d7640 1 --2- 172.21.15.76:0/1696090181 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ff40b0c40 0x7f8ff406fff0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.656+0000 7f8ffb9d7640 1 -- 172.21.15.76:0/1696090181 >> 172.21.15.76:0/1696090181 conn(0x7f8ff40cb530 msgr2=0x7f8ff40cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:02.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.656+0000 7f8ffb9d7640 1 -- 172.21.15.76:0/1696090181 shutdown_connections 2023-11-09T03:52:02.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.656+0000 7f8ffb9d7640 1 -- 172.21.15.76:0/1696090181 wait complete. 2023-11-09T03:52:02.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.657+0000 7f8ffb9d7640 1 Processor -- start 2023-11-09T03:52:02.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.657+0000 7f8ffb9d7640 1 -- start start 2023-11-09T03:52:02.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.658+0000 7f8ffb9d7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ff406bb60 0x7f8ff414dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:02.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.658+0000 7f8ffb9d7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ff40b0c40 0x7f8ff415a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:02.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.658+0000 7f8ffb9d7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ff4159e40 0x7f8ff40ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:02.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.658+0000 7f8ffb9d7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff414e230 con 0x7f8ff40b0c40 2023-11-09T03:52:02.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.658+0000 7f8ffb9d7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff40c25c0 con 0x7f8ff406bb60 2023-11-09T03:52:02.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.658+0000 7f8ffb9d7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff40c2810 con 0x7f8ff4159e40 2023-11-09T03:52:02.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.658+0000 7f8ff8f72640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ff40b0c40 0x7f8ff415a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:02.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.658+0000 7f8ff8f72640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ff40b0c40 0x7f8ff415a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55618/0 (socket says 172.21.15.76:55618) 2023-11-09T03:52:02.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.658+0000 7f8ff8f72640 1 -- 172.21.15.76:0/2903950640 learned_addr learned my addr 172.21.15.76:0/2903950640 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:02.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.658+0000 7f8ff9f74640 1 --2- 172.21.15.76:0/2903950640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ff4159e40 0x7f8ff40ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:02.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.658+0000 7f8ff9773640 1 --2- 172.21.15.76:0/2903950640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ff406bb60 0x7f8ff414dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:02.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.659+0000 7f8ff8f72640 1 -- 172.21.15.76:0/2903950640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ff4159e40 msgr2=0x7f8ff40ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:02.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.659+0000 7f8ff8f72640 1 --2- 172.21.15.76:0/2903950640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ff4159e40 0x7f8ff40ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.659+0000 7f8ff8f72640 1 -- 172.21.15.76:0/2903950640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ff406bb60 msgr2=0x7f8ff414dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:02.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.659+0000 7f8ff8f72640 1 --2- 172.21.15.76:0/2903950640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ff406bb60 0x7f8ff414dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:02.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.659+0000 7f8ff8f72640 1 -- 172.21.15.76:0/2903950640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe4009400 con 0x7f8ff40b0c40 2023-11-09T03:52:02.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.659+0000 7f8ff9f74640 1 --2- 172.21.15.76:0/2903950640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ff4159e40 0x7f8ff40ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:52:02.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.659+0000 7f8ff8f72640 1 --2- 172.21.15.76:0/2903950640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ff40b0c40 0x7f8ff415a7e0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40097a0 tx=0x7f8fe40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:02.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.659+0000 7f8fe9ffb640 1 -- 172.21.15.76:0/2903950640 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4004230 con 0x7f8ff40b0c40 2023-11-09T03:52:02.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.659+0000 7f8fe9ffb640 1 -- 172.21.15.76:0/2903950640 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe4004390 con 0x7f8ff40b0c40 2023-11-09T03:52:02.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.659+0000 7f8ffb9d7640 1 -- 172.21.15.76:0/2903950640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff40c2af0 con 0x7f8ff40b0c40 2023-11-09T03:52:02.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.659+0000 7f8fe9ffb640 1 -- 172.21.15.76:0/2903950640 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe403a880 con 0x7f8ff40b0c40 2023-11-09T03:52:02.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.659+0000 7f8ffb9d7640 1 -- 172.21.15.76:0/2903950640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff40c30b0 con 0x7f8ff40b0c40 2023-11-09T03:52:02.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.660+0000 7f8ffb9d7640 1 -- 172.21.15.76:0/2903950640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff40684b0 con 0x7f8ff40b0c40 2023-11-09T03:52:02.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.660+0000 7f8fe9ffb640 1 -- 172.21.15.76:0/2903950640 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8fe403a9e0 con 0x7f8ff40b0c40 2023-11-09T03:52:02.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.661+0000 7f8fe9ffb640 1 --2- 172.21.15.76:0/2903950640 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8fd806ae70 0x7f8fd806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:02.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.661+0000 7f8fe9ffb640 1 -- 172.21.15.76:0/2903950640 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7737+0+0 (secure 0 0 0) 0x7f8fe40ac4c0 con 0x7f8ff40b0c40 2023-11-09T03:52:02.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.661+0000 7f8ff9773640 1 --2- 172.21.15.76:0/2903950640 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8fd806ae70 0x7f8fd806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:02.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.661+0000 7f8ff9773640 1 --2- 172.21.15.76:0/2903950640 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8fd806ae70 0x7f8fd806d330 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8ff00016e0 tx=0x7f8ff0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:02.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.665+0000 7f8fe9ffb640 1 -- 172.21.15.76:0/2903950640 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8fe4076ea0 con 0x7f8ff40b0c40 2023-11-09T03:52:02.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:02.897+0000 7f8ffb9d7640 1 -- 172.21.15.76:0/2903950640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"} v 0) v1 -- 0x7f8ff40b0a20 con 0x7f8ff40b0c40 2023-11-09T03:52:03.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.522+0000 7f8fe9ffb640 1 -- 172.21.15.76:0/2903950640 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"}]=0 freezing mds.6904 v94) v1 ==== 112+0+0 (secure 0 0 0) 0x7f8fe4035020 con 0x7f8ff40b0c40 2023-11-09T03:52:03.523 INFO:teuthology.orchestra.run.smithi076.stderr:freezing mds.6904 2023-11-09T03:52:03.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.524+0000 7f8ffb9d7640 1 -- 172.21.15.76:0/2903950640 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8fd806ae70 msgr2=0x7f8fd806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:03.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.524+0000 7f8ffb9d7640 1 --2- 172.21.15.76:0/2903950640 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8fd806ae70 0x7f8fd806d330 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8ff00016e0 tx=0x7f8ff0009040 comp rx=0 tx=0).stop 2023-11-09T03:52:03.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.524+0000 7f8ffb9d7640 1 -- 172.21.15.76:0/2903950640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ff40b0c40 msgr2=0x7f8ff415a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:03.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.524+0000 7f8ffb9d7640 1 --2- 172.21.15.76:0/2903950640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ff40b0c40 0x7f8ff415a7e0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40097a0 tx=0x7f8fe40046c0 comp rx=0 tx=0).stop 2023-11-09T03:52:03.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.524+0000 7f8ffb9d7640 1 -- 172.21.15.76:0/2903950640 shutdown_connections 2023-11-09T03:52:03.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.524+0000 7f8ffb9d7640 1 --2- 172.21.15.76:0/2903950640 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8fd806ae70 0x7f8fd806d330 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:03.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.524+0000 7f8ffb9d7640 1 --2- 172.21.15.76:0/2903950640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ff4159e40 0x7f8ff40ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:03.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.524+0000 7f8ffb9d7640 1 --2- 172.21.15.76:0/2903950640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ff40b0c40 0x7f8ff415a7e0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:03.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.524+0000 7f8ffb9d7640 1 --2- 172.21.15.76:0/2903950640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ff406bb60 0x7f8ff414dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:03.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.524+0000 7f8ffb9d7640 1 -- 172.21.15.76:0/2903950640 >> 172.21.15.76:0/2903950640 conn(0x7f8ff40cb530 msgr2=0x7f8ff40cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:03.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.524+0000 7f8ffb9d7640 1 -- 172.21.15.76:0/2903950640 shutdown_connections 2023-11-09T03:52:03.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.524+0000 7f8ffb9d7640 1 -- 172.21.15.76:0/2903950640 wait complete. 2023-11-09T03:52:03.536 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:1 true 2023-11-09T03:52:03.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.637+0000 7f15f9f61640 1 Processor -- start 2023-11-09T03:52:03.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.638+0000 7f15f9f61640 1 -- start start 2023-11-09T03:52:03.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.638+0000 7f15f9f61640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15f415d100 0x7f15f40ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:03.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.638+0000 7f15f9f61640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f15f40d1be0 0x7f15f40b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:03.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.638+0000 7f15f9f61640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15f40b32b0 con 0x7f15f415d4e0 2023-11-09T03:52:03.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.638+0000 7f15f9f61640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15f40cee10 con 0x7f15f40d1be0 2023-11-09T03:52:03.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.638+0000 7f15f9f61640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f40b2980 con 0x7f15f415d100 2023-11-09T03:52:03.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.639+0000 7f15f37fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15f415d100 0x7f15f40ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:03.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.639+0000 7f15f3fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f15f40d1be0 0x7f15f40b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:03.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.639+0000 7f15f2ffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f15f415d4e0 0x7f15f40d14d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:57244/0 (socket says 172.21.15.76:57244) 2023-11-09T03:52:03.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.639+0000 7f15f2ffd640 1 -- 172.21.15.76:0/3380666612 learned_addr learned my addr 172.21.15.76:0/3380666612 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:52:03.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.639+0000 7f15f37fe640 1 -- 172.21.15.76:0/3380666612 >> v2:172.21.15.113:3300/0 conn(0x7f15f40d1be0 msgr2=0x7f15f40b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:03.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.639+0000 7f15f37fe640 1 --2- 172.21.15.76:0/3380666612 >> v2:172.21.15.113:3300/0 conn(0x7f15f40d1be0 0x7f15f40b3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:03.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.639+0000 7f15f37fe640 1 -- 172.21.15.76:0/3380666612 >> v1:172.21.15.113:6789/0 conn(0x7f15f415d4e0 legacy=0x7f15f40d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:03.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.639+0000 7f15f37fe640 1 -- 172.21.15.76:0/3380666612 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f40b2c60 con 0x7f15f415d100 2023-11-09T03:52:03.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.639+0000 7f15f3fff640 1 --2- 172.21.15.76:0/3380666612 >> v2:172.21.15.113:3300/0 conn(0x7f15f40d1be0 0x7f15f40b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:52:03.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.640+0000 7f15f37fe640 1 --2- 172.21.15.76:0/3380666612 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15f415d100 0x7f15f40ce510 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f15e40097c0 tx=0x7f15e402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad952771804f3a8d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:03.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.640+0000 7f15f27fc640 1 -- 172.21.15.76:0/3380666612 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e4031800 con 0x7f15f415d100 2023-11-09T03:52:03.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.640+0000 7f15f27fc640 1 -- 172.21.15.76:0/3380666612 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e4031960 con 0x7f15f415d100 2023-11-09T03:52:03.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.640+0000 7f15f27fc640 1 -- 172.21.15.76:0/3380666612 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e4031cf0 con 0x7f15f415d100 2023-11-09T03:52:03.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.641+0000 7f15f9f61640 1 -- 172.21.15.76:0/3380666612 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15f415d100 msgr2=0x7f15f40ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:03.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.641+0000 7f15f9f61640 1 --2- 172.21.15.76:0/3380666612 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15f415d100 0x7f15f40ce510 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f15e40097c0 tx=0x7f15e402b8a0 comp rx=0 tx=0).stop 2023-11-09T03:52:03.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.641+0000 7f15f9f61640 1 -- 172.21.15.76:0/3380666612 shutdown_connections 2023-11-09T03:52:03.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.641+0000 7f15f9f61640 1 --2- 172.21.15.76:0/3380666612 >> v2:172.21.15.113:3300/0 conn(0x7f15f40d1be0 0x7f15f40b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:03.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.641+0000 7f15f9f61640 1 --2- 172.21.15.76:0/3380666612 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15f415d100 0x7f15f40ce510 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:03.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.641+0000 7f15f9f61640 1 -- 172.21.15.76:0/3380666612 >> 172.21.15.76:0/3380666612 conn(0x7f15f40d2ef0 msgr2=0x7f15f40d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:03.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.641+0000 7f15f9f61640 1 -- 172.21.15.76:0/3380666612 shutdown_connections 2023-11-09T03:52:03.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.641+0000 7f15f9f61640 1 -- 172.21.15.76:0/3380666612 wait complete. 2023-11-09T03:52:03.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.642+0000 7f15f9f61640 1 Processor -- start 2023-11-09T03:52:03.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.642+0000 7f15f9f61640 1 -- start start 2023-11-09T03:52:03.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.643+0000 7f15f9f61640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15f40d1be0 0x7f15f40b83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:03.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.643+0000 7f15f9f61640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15f415d100 0x7f15f41f5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:03.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.643+0000 7f15f9f61640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15f415d4e0 0x7f15f41f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:03.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.643+0000 7f15f9f61640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f40b71a0 con 0x7f15f40d1be0 2023-11-09T03:52:03.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.643+0000 7f15f9f61640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f40b88e0 con 0x7f15f415d4e0 2023-11-09T03:52:03.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.643+0000 7f15f9f61640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f40b6520 con 0x7f15f415d100 2023-11-09T03:52:03.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.643+0000 7f15f37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15f40d1be0 0x7f15f40b83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:03.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.643+0000 7f15f37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15f40d1be0 0x7f15f40b83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55630/0 (socket says 172.21.15.76:55630) 2023-11-09T03:52:03.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.643+0000 7f15f37fe640 1 -- 172.21.15.76:0/703162204 learned_addr learned my addr 172.21.15.76:0/703162204 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:03.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.643+0000 7f15f3fff640 1 --2- 172.21.15.76:0/703162204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15f415d4e0 0x7f15f41f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:03.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.643+0000 7f15f2ffd640 1 --2- 172.21.15.76:0/703162204 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15f415d100 0x7f15f41f5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:03.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.643+0000 7f15f37fe640 1 -- 172.21.15.76:0/703162204 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15f415d100 msgr2=0x7f15f41f5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:03.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.643+0000 7f15f37fe640 1 --2- 172.21.15.76:0/703162204 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15f415d100 0x7f15f41f5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:03.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.643+0000 7f15f37fe640 1 -- 172.21.15.76:0/703162204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15f415d4e0 msgr2=0x7f15f41f2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:03.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.643+0000 7f15f37fe640 1 --2- 172.21.15.76:0/703162204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15f415d4e0 0x7f15f41f2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:03.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.643+0000 7f15f37fe640 1 -- 172.21.15.76:0/703162204 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ec009a70 con 0x7f15f40d1be0 2023-11-09T03:52:03.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.643+0000 7f15f37fe640 1 --2- 172.21.15.76:0/703162204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15f40d1be0 0x7f15f40b83a0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f15e40098e0 tx=0x7f15e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:03.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.643+0000 7f15dbfff640 1 -- 172.21.15.76:0/703162204 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e403bc70 con 0x7f15f40d1be0 2023-11-09T03:52:03.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.644+0000 7f15dbfff640 1 -- 172.21.15.76:0/703162204 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e403bdf0 con 0x7f15f40d1be0 2023-11-09T03:52:03.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.644+0000 7f15f9f61640 1 -- 172.21.15.76:0/703162204 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e40093a0 con 0x7f15f40d1be0 2023-11-09T03:52:03.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.644+0000 7f15dbfff640 1 -- 172.21.15.76:0/703162204 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e4045630 con 0x7f15f40d1be0 2023-11-09T03:52:03.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.644+0000 7f15f9f61640 1 -- 172.21.15.76:0/703162204 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f40b6ae0 con 0x7f15f40d1be0 2023-11-09T03:52:03.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.644+0000 7f15f9f61640 1 -- 172.21.15.76:0/703162204 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15f40684b0 con 0x7f15f40d1be0 2023-11-09T03:52:03.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.645+0000 7f15dbfff640 1 -- 172.21.15.76:0/703162204 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f15e4045790 con 0x7f15f40d1be0 2023-11-09T03:52:03.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.645+0000 7f15dbfff640 1 --2- 172.21.15.76:0/703162204 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f15e006ae70 0x7f15e006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:03.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.645+0000 7f15dbfff640 1 -- 172.21.15.76:0/703162204 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7737+0+0 (secure 0 0 0) 0x7f15e402e070 con 0x7f15f40d1be0 2023-11-09T03:52:03.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.645+0000 7f15f2ffd640 1 --2- 172.21.15.76:0/703162204 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f15e006ae70 0x7f15e006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:03.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.646+0000 7f15f2ffd640 1 --2- 172.21.15.76:0/703162204 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f15e006ae70 0x7f15e006d330 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f15f415ffe0 tx=0x7f15e8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:03.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.650+0000 7f15dbfff640 1 -- 172.21.15.76:0/703162204 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f15e40769f0 con 0x7f15f40d1be0 2023-11-09T03:52:03.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:03.884+0000 7f15f9f61640 1 -- 172.21.15.76:0/703162204 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"} v 0) v1 -- 0x7f15f40d2cd0 con 0x7f15f40d1be0 2023-11-09T03:52:04.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.528+0000 7f15dbfff640 1 -- 172.21.15.76:0/703162204 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"}]=0 freezing mds.8274 v95) v1 ==== 112+0+0 (secure 0 0 0) 0x7f15e4035020 con 0x7f15f40d1be0 2023-11-09T03:52:04.529 INFO:teuthology.orchestra.run.smithi076.stderr:freezing mds.8274 2023-11-09T03:52:04.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.530+0000 7f15f9f61640 1 -- 172.21.15.76:0/703162204 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f15e006ae70 msgr2=0x7f15e006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:04.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.530+0000 7f15f9f61640 1 --2- 172.21.15.76:0/703162204 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f15e006ae70 0x7f15e006d330 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f15f415ffe0 tx=0x7f15e8009040 comp rx=0 tx=0).stop 2023-11-09T03:52:04.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.530+0000 7f15f9f61640 1 -- 172.21.15.76:0/703162204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15f40d1be0 msgr2=0x7f15f40b83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:04.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.530+0000 7f15f9f61640 1 --2- 172.21.15.76:0/703162204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15f40d1be0 0x7f15f40b83a0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f15e40098e0 tx=0x7f15e40046c0 comp rx=0 tx=0).stop 2023-11-09T03:52:04.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.530+0000 7f15f9f61640 1 -- 172.21.15.76:0/703162204 shutdown_connections 2023-11-09T03:52:04.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.530+0000 7f15f9f61640 1 --2- 172.21.15.76:0/703162204 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f15e006ae70 0x7f15e006d330 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:04.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.530+0000 7f15f9f61640 1 --2- 172.21.15.76:0/703162204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15f415d4e0 0x7f15f41f2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:04.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.530+0000 7f15f9f61640 1 --2- 172.21.15.76:0/703162204 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15f415d100 0x7f15f41f5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:04.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.530+0000 7f15f9f61640 1 --2- 172.21.15.76:0/703162204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15f40d1be0 0x7f15f40b83a0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:04.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.530+0000 7f15f9f61640 1 -- 172.21.15.76:0/703162204 >> 172.21.15.76:0/703162204 conn(0x7f15f40d2ef0 msgr2=0x7f15f40cd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:04.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.530+0000 7f15f9f61640 1 -- 172.21.15.76:0/703162204 shutdown_connections 2023-11-09T03:52:04.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.530+0000 7f15f9f61640 1 -- 172.21.15.76:0/703162204 wait complete. 2023-11-09T03:52:04.543 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_kill_mdstable_at 5 2023-11-09T03:52:04.667 INFO:teuthology.orchestra.run.smithi113.stdout:{"success":"mds_kill_mdstable_at = '5' (not observed, change may require restart) "} 2023-11-09T03:52:04.667 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '5' (not observed, change may require restart) " } 2023-11-09T03:52:04.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:52:04.668 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s25) 2023-11-09T03:52:04.670 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2023-11-09T03:52:04.692 INFO:tasks.ceph.mds.b.smithi113.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_commit(ceph::cref_t&)' thread 7f0f57198640 time 2023-11-09T03:52:04.693306+0000 2023-11-09T03:52:04.692 INFO:tasks.ceph.mds.b.smithi113.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 145: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 5) 2023-11-09T03:52:04.694 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:52:04.694 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f0f5cd6efaf] 2023-11-09T03:52:04.694 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f0f5cd6f16d] 2023-11-09T03:52:04.694 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x7e7) [0x564a5d0f2187] 2023-11-09T03:52:04.695 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x564a5ce810a9] 2023-11-09T03:52:04.695 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x564a5ce83b2b] 2023-11-09T03:52:04.695 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564a5ce8401c] 2023-11-09T03:52:04.695 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x564a5ce6d325] 2023-11-09T03:52:04.695 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7f0f5cf5ff9a] 2023-11-09T03:52:04.696 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f0f5cff1741] 2023-11-09T03:52:04.696 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7f0f5c49f802] 2023-11-09T03:52:04.696 INFO:tasks.ceph.mds.b.smithi113.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7f0f5c43f450] 2023-11-09T03:52:04.696 INFO:tasks.ceph.mds.b.smithi113.stderr:*** Caught signal (Aborted) ** 2023-11-09T03:52:04.696 INFO:tasks.ceph.mds.b.smithi113.stderr: in thread 7f0f57198640 thread_name:ms_dispatch 2023-11-09T03:52:04.696 INFO:tasks.ceph.mds.b.smithi113.stderr:2023-11-09T03:52:04.692+0000 7f0f57198640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_commit(ceph::cref_t&)' thread 7f0f57198640 time 2023-11-09T03:52:04.693306+0000 2023-11-09T03:52:04.697 INFO:tasks.ceph.mds.b.smithi113.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 145: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 5) 2023-11-09T03:52:04.697 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:52:04.697 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:52:04.697 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f0f5cd6efaf] 2023-11-09T03:52:04.697 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f0f5cd6f16d] 2023-11-09T03:52:04.698 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x7e7) [0x564a5d0f2187] 2023-11-09T03:52:04.698 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x564a5ce810a9] 2023-11-09T03:52:04.698 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x564a5ce83b2b] 2023-11-09T03:52:04.698 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564a5ce8401c] 2023-11-09T03:52:04.698 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x564a5ce6d325] 2023-11-09T03:52:04.699 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7f0f5cf5ff9a] 2023-11-09T03:52:04.699 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f0f5cff1741] 2023-11-09T03:52:04.699 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7f0f5c49f802] 2023-11-09T03:52:04.699 INFO:tasks.ceph.mds.b.smithi113.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7f0f5c43f450] 2023-11-09T03:52:04.699 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:52:04.700 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:52:04.700 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f0f5c454db0] 2023-11-09T03:52:04.700 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f0f5c4a154c] 2023-11-09T03:52:04.700 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: raise() 2023-11-09T03:52:04.700 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: abort() 2023-11-09T03:52:04.700 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f0f5cd6f009] 2023-11-09T03:52:04.701 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f0f5cd6f16d] 2023-11-09T03:52:04.701 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x7e7) [0x564a5d0f2187] 2023-11-09T03:52:04.701 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x564a5ce810a9] 2023-11-09T03:52:04.701 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x564a5ce83b2b] 2023-11-09T03:52:04.701 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564a5ce8401c] 2023-11-09T03:52:04.702 INFO:tasks.ceph.mds.b.smithi113.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x564a5ce6d325] 2023-11-09T03:52:04.702 INFO:tasks.ceph.mds.b.smithi113.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7f0f5cf5ff9a] 2023-11-09T03:52:04.702 INFO:tasks.ceph.mds.b.smithi113.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f0f5cff1741] 2023-11-09T03:52:04.702 INFO:tasks.ceph.mds.b.smithi113.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7f0f5c49f802] 2023-11-09T03:52:04.702 INFO:tasks.ceph.mds.b.smithi113.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7f0f5c43f450] 2023-11-09T03:52:04.703 INFO:tasks.ceph.mds.b.smithi113.stderr:2023-11-09T03:52:04.693+0000 7f0f57198640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:52:04.703 INFO:tasks.ceph.mds.b.smithi113.stderr: in thread 7f0f57198640 thread_name:ms_dispatch 2023-11-09T03:52:04.703 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:52:04.703 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:52:04.703 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f0f5c454db0] 2023-11-09T03:52:04.703 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f0f5c4a154c] 2023-11-09T03:52:04.704 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: raise() 2023-11-09T03:52:04.704 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: abort() 2023-11-09T03:52:04.704 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f0f5cd6f009] 2023-11-09T03:52:04.704 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f0f5cd6f16d] 2023-11-09T03:52:04.704 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x7e7) [0x564a5d0f2187] 2023-11-09T03:52:04.705 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x564a5ce810a9] 2023-11-09T03:52:04.705 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x564a5ce83b2b] 2023-11-09T03:52:04.705 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564a5ce8401c] 2023-11-09T03:52:04.705 INFO:tasks.ceph.mds.b.smithi113.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x564a5ce6d325] 2023-11-09T03:52:04.705 INFO:tasks.ceph.mds.b.smithi113.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7f0f5cf5ff9a] 2023-11-09T03:52:04.705 INFO:tasks.ceph.mds.b.smithi113.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f0f5cff1741] 2023-11-09T03:52:04.706 INFO:tasks.ceph.mds.b.smithi113.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7f0f5c49f802] 2023-11-09T03:52:04.706 INFO:tasks.ceph.mds.b.smithi113.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7f0f5c43f450] 2023-11-09T03:52:04.706 INFO:tasks.ceph.mds.b.smithi113.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:52:04.706 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:52:04.707 INFO:tasks.ceph.mds.b.smithi113.stderr: -1231> 2023-11-09T03:51:50.626+0000 7f0f54993640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:52:04.708 INFO:tasks.ceph.mds.b.smithi113.stderr: -1229> 2023-11-09T03:51:50.626+0000 7f0f54993640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-09T03:52:04.708 INFO:tasks.ceph.mds.b.smithi113.stderr: -1> 2023-11-09T03:52:04.692+0000 7f0f57198640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_commit(ceph::cref_t&)' thread 7f0f57198640 time 2023-11-09T03:52:04.693306+0000 2023-11-09T03:52:04.708 INFO:tasks.ceph.mds.b.smithi113.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 145: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 5) 2023-11-09T03:52:04.708 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:52:04.708 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:52:04.709 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f0f5cd6efaf] 2023-11-09T03:52:04.709 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f0f5cd6f16d] 2023-11-09T03:52:04.709 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x7e7) [0x564a5d0f2187] 2023-11-09T03:52:04.709 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x564a5ce810a9] 2023-11-09T03:52:04.709 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x564a5ce83b2b] 2023-11-09T03:52:04.710 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564a5ce8401c] 2023-11-09T03:52:04.710 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x564a5ce6d325] 2023-11-09T03:52:04.710 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7f0f5cf5ff9a] 2023-11-09T03:52:04.710 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f0f5cff1741] 2023-11-09T03:52:04.710 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7f0f5c49f802] 2023-11-09T03:52:04.711 INFO:tasks.ceph.mds.b.smithi113.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7f0f5c43f450] 2023-11-09T03:52:04.711 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:52:04.711 INFO:tasks.ceph.mds.b.smithi113.stderr: 0> 2023-11-09T03:52:04.693+0000 7f0f57198640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:52:04.711 INFO:tasks.ceph.mds.b.smithi113.stderr: in thread 7f0f57198640 thread_name:ms_dispatch 2023-11-09T03:52:04.711 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:52:04.712 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:52:04.712 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f0f5c454db0] 2023-11-09T03:52:04.712 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f0f5c4a154c] 2023-11-09T03:52:04.712 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: raise() 2023-11-09T03:52:04.712 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: abort() 2023-11-09T03:52:04.713 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f0f5cd6f009] 2023-11-09T03:52:04.713 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f0f5cd6f16d] 2023-11-09T03:52:04.713 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x7e7) [0x564a5d0f2187] 2023-11-09T03:52:04.713 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x564a5ce810a9] 2023-11-09T03:52:04.713 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x564a5ce83b2b] 2023-11-09T03:52:04.713 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564a5ce8401c] 2023-11-09T03:52:04.714 INFO:tasks.ceph.mds.b.smithi113.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x564a5ce6d325] 2023-11-09T03:52:04.714 INFO:tasks.ceph.mds.b.smithi113.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7f0f5cf5ff9a] 2023-11-09T03:52:04.714 INFO:tasks.ceph.mds.b.smithi113.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f0f5cff1741] 2023-11-09T03:52:04.714 INFO:tasks.ceph.mds.b.smithi113.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7f0f5c49f802] 2023-11-09T03:52:04.714 INFO:tasks.ceph.mds.b.smithi113.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7f0f5c43f450] 2023-11-09T03:52:04.715 INFO:tasks.ceph.mds.b.smithi113.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:52:04.715 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:52:04.715 INFO:tasks.ceph.mds.b.smithi113.stderr: -1231> 2023-11-09T03:51:50.626+0000 7f0f54993640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:52:04.715 INFO:tasks.ceph.mds.b.smithi113.stderr: -1229> 2023-11-09T03:51:50.626+0000 7f0f54993640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-09T03:52:04.716 INFO:tasks.ceph.mds.b.smithi113.stderr: -1> 2023-11-09T03:52:04.692+0000 7f0f57198640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_commit(ceph::cref_t&)' thread 7f0f57198640 time 2023-11-09T03:52:04.693306+0000 2023-11-09T03:52:04.716 INFO:tasks.ceph.mds.b.smithi113.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 145: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 5) 2023-11-09T03:52:04.716 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:52:04.716 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:52:04.716 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f0f5cd6efaf] 2023-11-09T03:52:04.717 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f0f5cd6f16d] 2023-11-09T03:52:04.717 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x7e7) [0x564a5d0f2187] 2023-11-09T03:52:04.717 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x564a5ce810a9] 2023-11-09T03:52:04.717 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x564a5ce83b2b] 2023-11-09T03:52:04.717 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564a5ce8401c] 2023-11-09T03:52:04.717 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x564a5ce6d325] 2023-11-09T03:52:04.718 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7f0f5cf5ff9a] 2023-11-09T03:52:04.718 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f0f5cff1741] 2023-11-09T03:52:04.718 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7f0f5c49f802] 2023-11-09T03:52:04.718 INFO:tasks.ceph.mds.b.smithi113.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7f0f5c43f450] 2023-11-09T03:52:04.718 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:52:04.719 INFO:tasks.ceph.mds.b.smithi113.stderr: 0> 2023-11-09T03:52:04.693+0000 7f0f57198640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:52:04.719 INFO:tasks.ceph.mds.b.smithi113.stderr: in thread 7f0f57198640 thread_name:ms_dispatch 2023-11-09T03:52:04.719 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:52:04.719 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:52:04.719 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f0f5c454db0] 2023-11-09T03:52:04.720 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f0f5c4a154c] 2023-11-09T03:52:04.720 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: raise() 2023-11-09T03:52:04.720 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: abort() 2023-11-09T03:52:04.720 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f0f5cd6f009] 2023-11-09T03:52:04.720 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f0f5cd6f16d] 2023-11-09T03:52:04.721 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x7e7) [0x564a5d0f2187] 2023-11-09T03:52:04.721 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x564a5ce810a9] 2023-11-09T03:52:04.721 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x564a5ce83b2b] 2023-11-09T03:52:04.721 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564a5ce8401c] 2023-11-09T03:52:04.721 INFO:tasks.ceph.mds.b.smithi113.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x564a5ce6d325] 2023-11-09T03:52:04.721 INFO:tasks.ceph.mds.b.smithi113.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7f0f5cf5ff9a] 2023-11-09T03:52:04.722 INFO:tasks.ceph.mds.b.smithi113.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f0f5cff1741] 2023-11-09T03:52:04.722 INFO:tasks.ceph.mds.b.smithi113.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7f0f5c49f802] 2023-11-09T03:52:04.722 INFO:tasks.ceph.mds.b.smithi113.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7f0f5c43f450] 2023-11-09T03:52:04.722 INFO:tasks.ceph.mds.b.smithi113.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:52:04.722 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:52:04.797 INFO:teuthology.orchestra.run.smithi076.stdout:{"mds_beacon_grace":"15.000000"} 2023-11-09T03:52:04.797 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2023-11-09T03:52:04.798 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:52:04.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.897+0000 7fc25fc58640 1 Processor -- start 2023-11-09T03:52:04.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.897+0000 7fc25fc58640 1 -- start start 2023-11-09T03:52:04.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.898+0000 7fc25fc58640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc25815d100 0x7fc2580ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:04.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.898+0000 7fc25fc58640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2580d1be0 0x7fc2580b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:04.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.898+0000 7fc25fc58640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2580b32b0 con 0x7fc25815d4e0 2023-11-09T03:52:04.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.898+0000 7fc25fc58640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2580cee10 con 0x7fc25815d100 2023-11-09T03:52:04.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.898+0000 7fc25fc58640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2580b2980 con 0x7fc2580d1be0 2023-11-09T03:52:04.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.898+0000 7fc25d9f4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc25815d100 0x7fc2580ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:04.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.898+0000 7fc25d9f4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc25815d100 0x7fc2580ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55644/0 (socket says 172.21.15.76:55644) 2023-11-09T03:52:04.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.898+0000 7fc25d9f4640 1 -- 172.21.15.76:0/1162383824 learned_addr learned my addr 172.21.15.76:0/1162383824 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:04.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.898+0000 7fc25d9f4640 1 -- 172.21.15.76:0/1162383824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2580d1be0 msgr2=0x7fc2580b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:04.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.898+0000 7fc25d9f4640 1 --2- 172.21.15.76:0/1162383824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2580d1be0 0x7fc2580b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:04.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.898+0000 7fc25d9f4640 1 -- 172.21.15.76:0/1162383824 >> v1:172.21.15.113:6789/0 conn(0x7fc25815d4e0 legacy=0x7fc2580d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:04.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.898+0000 7fc25d9f4640 1 -- 172.21.15.76:0/1162383824 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2580b2c60 con 0x7fc25815d100 2023-11-09T03:52:04.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.899+0000 7fc25d9f4640 1 --2- 172.21.15.76:0/1162383824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc25815d100 0x7fc2580ce510 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fc254009850 tx=0x7fc25402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cff6dfb171ee331 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:04.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.899+0000 7fc25c9f2640 1 -- 172.21.15.76:0/1162383824 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc254004230 con 0x7fc25815d100 2023-11-09T03:52:04.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.899+0000 7fc25c9f2640 1 -- 172.21.15.76:0/1162383824 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc254004390 con 0x7fc25815d100 2023-11-09T03:52:04.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.899+0000 7fc25c9f2640 1 -- 172.21.15.76:0/1162383824 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc254003810 con 0x7fc25815d100 2023-11-09T03:52:04.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.899+0000 7fc25fc58640 1 -- 172.21.15.76:0/1162383824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc25815d100 msgr2=0x7fc2580ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:04.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.899+0000 7fc25fc58640 1 --2- 172.21.15.76:0/1162383824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc25815d100 0x7fc2580ce510 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fc254009850 tx=0x7fc25402b8f0 comp rx=0 tx=0).stop 2023-11-09T03:52:04.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.899+0000 7fc25fc58640 1 -- 172.21.15.76:0/1162383824 shutdown_connections 2023-11-09T03:52:04.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.900+0000 7fc25fc58640 1 --2- 172.21.15.76:0/1162383824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2580d1be0 0x7fc2580b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:04.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.900+0000 7fc25fc58640 1 --2- 172.21.15.76:0/1162383824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc25815d100 0x7fc2580ce510 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:04.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.900+0000 7fc25fc58640 1 -- 172.21.15.76:0/1162383824 >> 172.21.15.76:0/1162383824 conn(0x7fc2580d2ef0 msgr2=0x7fc2580d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:04.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.900+0000 7fc25fc58640 1 -- 172.21.15.76:0/1162383824 shutdown_connections 2023-11-09T03:52:04.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.900+0000 7fc25fc58640 1 -- 172.21.15.76:0/1162383824 wait complete. 2023-11-09T03:52:04.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.901+0000 7fc25fc58640 1 Processor -- start 2023-11-09T03:52:04.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.901+0000 7fc25fc58640 1 -- start start 2023-11-09T03:52:04.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.901+0000 7fc25fc58640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2580d1be0 0x7fc2580b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:04.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.901+0000 7fc25fc58640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc25815d100 0x7fc2580c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:04.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.901+0000 7fc25fc58640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc25815d4e0 0x7fc25815b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:04.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.901+0000 7fc25fc58640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc258157990 con 0x7fc25815d4e0 2023-11-09T03:52:04.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.901+0000 7fc25fc58640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2580b8170 con 0x7fc2580d1be0 2023-11-09T03:52:04.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.901+0000 7fc25fc58640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2580b8450 con 0x7fc25815d100 2023-11-09T03:52:04.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.902+0000 7fc25e1f5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc25815d4e0 0x7fc25815b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:04.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.902+0000 7fc25e1f5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc25815d4e0 0x7fc25815b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55652/0 (socket says 172.21.15.76:55652) 2023-11-09T03:52:04.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.902+0000 7fc25e1f5640 1 -- 172.21.15.76:0/674702563 learned_addr learned my addr 172.21.15.76:0/674702563 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:04.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.902+0000 7fc25d9f4640 1 --2- 172.21.15.76:0/674702563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2580d1be0 0x7fc2580b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:04.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.902+0000 7fc25d1f3640 1 --2- 172.21.15.76:0/674702563 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc25815d100 0x7fc2580c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:04.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.902+0000 7fc25e1f5640 1 -- 172.21.15.76:0/674702563 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc25815d100 msgr2=0x7fc2580c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:04.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.902+0000 7fc25e1f5640 1 --2- 172.21.15.76:0/674702563 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc25815d100 0x7fc2580c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:04.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.902+0000 7fc25e1f5640 1 -- 172.21.15.76:0/674702563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2580d1be0 msgr2=0x7fc2580b65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:04.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.902+0000 7fc25e1f5640 1 --2- 172.21.15.76:0/674702563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2580d1be0 0x7fc2580b65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:04.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.902+0000 7fc25e1f5640 1 -- 172.21.15.76:0/674702563 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc254009460 con 0x7fc25815d4e0 2023-11-09T03:52:04.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.902+0000 7fc25e1f5640 1 --2- 172.21.15.76:0/674702563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc25815d4e0 0x7fc25815b830 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fc25000b9e0 tx=0x7fc25000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:04.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.902+0000 7fc24e7fc640 1 -- 172.21.15.76:0/674702563 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25000cb20 con 0x7fc25815d4e0 2023-11-09T03:52:04.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.902+0000 7fc24e7fc640 1 -- 172.21.15.76:0/674702563 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25000cca0 con 0x7fc25815d4e0 2023-11-09T03:52:04.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.902+0000 7fc25fc58640 1 -- 172.21.15.76:0/674702563 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2580b5b10 con 0x7fc25815d4e0 2023-11-09T03:52:04.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.902+0000 7fc24e7fc640 1 -- 172.21.15.76:0/674702563 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc250014680 con 0x7fc25815d4e0 2023-11-09T03:52:04.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.902+0000 7fc25fc58640 1 -- 172.21.15.76:0/674702563 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2580b60d0 con 0x7fc25815d4e0 2023-11-09T03:52:04.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.903+0000 7fc25fc58640 1 -- 172.21.15.76:0/674702563 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2580684b0 con 0x7fc25815d4e0 2023-11-09T03:52:04.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.903+0000 7fc24e7fc640 1 -- 172.21.15.76:0/674702563 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc25001d3b0 con 0x7fc25815d4e0 2023-11-09T03:52:04.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.904+0000 7fc24e7fc640 1 --2- 172.21.15.76:0/674702563 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc23c06ae70 0x7fc23c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:04.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.904+0000 7fc24e7fc640 1 -- 172.21.15.76:0/674702563 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7737+0+0 (secure 0 0 0) 0x7fc250089c40 con 0x7fc25815d4e0 2023-11-09T03:52:04.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.904+0000 7fc25d9f4640 1 --2- 172.21.15.76:0/674702563 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc23c06ae70 0x7fc23c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:04.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.904+0000 7fc25d9f4640 1 --2- 172.21.15.76:0/674702563 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc23c06ae70 0x7fc23c06d330 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc254035000 tx=0x7fc254008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:04.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:04.908+0000 7fc24e7fc640 1 -- 172.21.15.76:0/674702563 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc2500545b0 con 0x7fc25815d4e0 2023-11-09T03:52:04.987 INFO:tasks.ceph.mds.b.smithi113.stderr:daemon-helper: command crashed with signal 6 2023-11-09T03:52:05.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:05.108+0000 7fc25fc58640 1 -- 172.21.15.76:0/674702563 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2580d0430 con 0x7fc25815d4e0 2023-11-09T03:52:05.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:05.109+0000 7fc24e7fc640 1 -- 172.21.15.76:0/674702563 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4929 (secure 0 0 0) 0x7fc250019090 con 0x7fc25815d4e0 2023-11-09T03:52:05.110 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:52:05.110 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":95,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8310,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:52:03.885950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6904,"mds_1":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6904":{"gid":6904,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":16,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8274":{"gid":8274,"name":"c","rank":1,"incarnation":89,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6833/1536527256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1536527256},{"type":"v1","addr":"172.21.15.76:6833","nonce":1536527256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:52:05.110 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 95 2023-11-09T03:52:05.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:05.111+0000 7fc25fc58640 1 -- 172.21.15.76:0/674702563 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc23c06ae70 msgr2=0x7fc23c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:05.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:05.111+0000 7fc25fc58640 1 --2- 172.21.15.76:0/674702563 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc23c06ae70 0x7fc23c06d330 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc254035000 tx=0x7fc254008030 comp rx=0 tx=0).stop 2023-11-09T03:52:05.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:05.111+0000 7fc25fc58640 1 -- 172.21.15.76:0/674702563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc25815d4e0 msgr2=0x7fc25815b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:05.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:05.111+0000 7fc25fc58640 1 --2- 172.21.15.76:0/674702563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc25815d4e0 0x7fc25815b830 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fc25000b9e0 tx=0x7fc25000bea0 comp rx=0 tx=0).stop 2023-11-09T03:52:05.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:05.111+0000 7fc25fc58640 1 -- 172.21.15.76:0/674702563 shutdown_connections 2023-11-09T03:52:05.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:05.111+0000 7fc25fc58640 1 --2- 172.21.15.76:0/674702563 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc23c06ae70 0x7fc23c06d330 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:05.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:05.111+0000 7fc25fc58640 1 --2- 172.21.15.76:0/674702563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc25815d4e0 0x7fc25815b830 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:05.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:05.111+0000 7fc25fc58640 1 --2- 172.21.15.76:0/674702563 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc25815d100 0x7fc2580c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:05.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:05.111+0000 7fc25fc58640 1 --2- 172.21.15.76:0/674702563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2580d1be0 0x7fc2580b65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:05.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:05.111+0000 7fc25fc58640 1 -- 172.21.15.76:0/674702563 >> 172.21.15.76:0/674702563 conn(0x7fc2580d2ef0 msgr2=0x7fc2580b5100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:05.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:05.111+0000 7fc25fc58640 1 -- 172.21.15.76:0/674702563 shutdown_connections 2023-11-09T03:52:05.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:05.111+0000 7fc25fc58640 1 -- 172.21.15.76:0/674702563 wait complete. 2023-11-09T03:52:05.122 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:52:05.599 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-11-09T03:52:10.124 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:52:10.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.238+0000 7f6bd7b16640 1 Processor -- start 2023-11-09T03:52:10.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.239+0000 7f6bd7b16640 1 -- start start 2023-11-09T03:52:10.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.239+0000 7f6bd7b16640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6bd0070700 0x7f6bd0164b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:10.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.240+0000 7f6bd7b16640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bd0071500 con 0x7f6bd00b0c40 2023-11-09T03:52:10.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.240+0000 7f6bd7b16640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bd015dae0 con 0x7f6bd0159e40 2023-11-09T03:52:10.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.240+0000 7f6bd7b16640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bd0070c60 con 0x7f6bd0070700 2023-11-09T03:52:10.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.240+0000 7f6bd60b3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6bd0070700 0x7f6bd0164b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:10.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.240+0000 7f6bd50b1640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f6bd00b0c40 0x7f6bd006fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47342/0 (socket says 172.21.15.76:47342) 2023-11-09T03:52:10.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.240+0000 7f6bd50b1640 1 -- 172.21.15.76:0/1505603325 learned_addr learned my addr 172.21.15.76:0/1505603325 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:52:10.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.240+0000 7f6bd60b3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6bd0070700 0x7f6bd0164b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55664/0 (socket says 172.21.15.76:55664) 2023-11-09T03:52:10.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.240+0000 7f6bd60b3640 1 -- 172.21.15.76:0/1505603325 >> v1:172.21.15.113:6789/0 conn(0x7f6bd0159e40 legacy=0x7f6bd00b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:10.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.240+0000 7f6bd60b3640 1 -- 172.21.15.76:0/1505603325 >> v1:172.21.15.76:6789/0 conn(0x7f6bd00b0c40 legacy=0x7f6bd006fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:10.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.240+0000 7f6bd60b3640 1 -- 172.21.15.76:0/1505603325 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd0070f40 con 0x7f6bd0070700 2023-11-09T03:52:10.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.240+0000 7f6bd48b0640 1 -- 172.21.15.76:0/1505603325 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1679160032 0 0) 0x7f6bd0071500 con 0x7f6bd00b0c40 2023-11-09T03:52:10.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.241+0000 7f6bd60b3640 1 --2- 172.21.15.76:0/1505603325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bd0070700 0x7f6bd0164b60 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8009d90 tx=0x7f6bc802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc2da4a460b0e71 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:10.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.241+0000 7f6bd48b0640 1 -- 172.21.15.76:0/1505603325 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc802cc60 con 0x7f6bd0070700 2023-11-09T03:52:10.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.241+0000 7f6bd48b0640 1 -- 172.21.15.76:0/1505603325 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc802cdc0 con 0x7f6bd0070700 2023-11-09T03:52:10.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.241+0000 7f6bd48b0640 1 -- 172.21.15.76:0/1505603325 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8034660 con 0x7f6bd0070700 2023-11-09T03:52:10.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.242+0000 7f6bd7b16640 1 -- 172.21.15.76:0/1505603325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bd0070700 msgr2=0x7f6bd0164b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:10.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.242+0000 7f6bd7b16640 1 --2- 172.21.15.76:0/1505603325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bd0070700 0x7f6bd0164b60 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8009d90 tx=0x7f6bc802beb0 comp rx=0 tx=0).stop 2023-11-09T03:52:10.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.242+0000 7f6bd7b16640 1 -- 172.21.15.76:0/1505603325 shutdown_connections 2023-11-09T03:52:10.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.242+0000 7f6bd7b16640 1 --2- 172.21.15.76:0/1505603325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bd0070700 0x7f6bd0164b60 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:10.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.242+0000 7f6bd7b16640 1 -- 172.21.15.76:0/1505603325 >> 172.21.15.76:0/1505603325 conn(0x7f6bd00cb530 msgr2=0x7f6bd00cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:10.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.242+0000 7f6bd7b16640 1 -- 172.21.15.76:0/1505603325 shutdown_connections 2023-11-09T03:52:10.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.242+0000 7f6bd7b16640 1 -- 172.21.15.76:0/1505603325 wait complete. 2023-11-09T03:52:10.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.243+0000 7f6bd7b16640 1 Processor -- start 2023-11-09T03:52:10.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.243+0000 7f6bd7b16640 1 -- start start 2023-11-09T03:52:10.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.244+0000 7f6bd7b16640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bd0070700 0x7f6bd015bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:10.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.244+0000 7f6bd7b16640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bd00b0c40 0x7f6bd00c4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:10.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.244+0000 7f6bd7b16640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bd0159e40 0x7f6bd00c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:10.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.244+0000 7f6bd7b16640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd015c300 con 0x7f6bd00b0c40 2023-11-09T03:52:10.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.244+0000 7f6bd7b16640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd015d140 con 0x7f6bd0159e40 2023-11-09T03:52:10.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.244+0000 7f6bd7b16640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd015c530 con 0x7f6bd0070700 2023-11-09T03:52:10.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.244+0000 7f6bd50b1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bd00b0c40 0x7f6bd00c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:10.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.244+0000 7f6bd50b1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bd00b0c40 0x7f6bd00c4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55672/0 (socket says 172.21.15.76:55672) 2023-11-09T03:52:10.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.244+0000 7f6bd50b1640 1 -- 172.21.15.76:0/2566889509 learned_addr learned my addr 172.21.15.76:0/2566889509 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:10.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.244+0000 7f6bd50b1640 1 -- 172.21.15.76:0/2566889509 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bd0070700 msgr2=0x7f6bd015bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:10.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.244+0000 7f6bd60b3640 1 --2- 172.21.15.76:0/2566889509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bd0159e40 0x7f6bd00c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:10.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.244+0000 7f6bd50b1640 1 --2- 172.21.15.76:0/2566889509 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bd0070700 0x7f6bd015bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:10.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.244+0000 7f6bd50b1640 1 -- 172.21.15.76:0/2566889509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bd0159e40 msgr2=0x7f6bd00c4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:10.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.244+0000 7f6bd50b1640 1 --2- 172.21.15.76:0/2566889509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bd0159e40 0x7f6bd00c4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:10.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.244+0000 7f6bd50b1640 1 -- 172.21.15.76:0/2566889509 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc8009a70 con 0x7f6bd00b0c40 2023-11-09T03:52:10.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.244+0000 7f6bd60b3640 1 --2- 172.21.15.76:0/2566889509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bd0159e40 0x7f6bd00c4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:52:10.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.244+0000 7f6bd50b1640 1 --2- 172.21.15.76:0/2566889509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bd00b0c40 0x7f6bd00c4790 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc00c4a0 tx=0x7f6bcc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:10.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.244+0000 7f6bc67fc640 1 -- 172.21.15.76:0/2566889509 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc0074e0 con 0x7f6bd00b0c40 2023-11-09T03:52:10.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.244+0000 7f6bc67fc640 1 -- 172.21.15.76:0/2566889509 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bcc007640 con 0x7f6bd00b0c40 2023-11-09T03:52:10.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.244+0000 7f6bd7b16640 1 -- 172.21.15.76:0/2566889509 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd015c810 con 0x7f6bd00b0c40 2023-11-09T03:52:10.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.245+0000 7f6bc67fc640 1 -- 172.21.15.76:0/2566889509 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc004a00 con 0x7f6bd00b0c40 2023-11-09T03:52:10.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.245+0000 7f6bd7b16640 1 -- 172.21.15.76:0/2566889509 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd015cdd0 con 0x7f6bd00b0c40 2023-11-09T03:52:10.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.245+0000 7f6bd7b16640 1 -- 172.21.15.76:0/2566889509 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bd00684b0 con 0x7f6bd00b0c40 2023-11-09T03:52:10.253 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.246+0000 7f6bc67fc640 1 -- 172.21.15.76:0/2566889509 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6bcc004b60 con 0x7f6bd00b0c40 2023-11-09T03:52:10.253 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.246+0000 7f6bc67fc640 1 --2- 172.21.15.76:0/2566889509 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6bb406aea0 0x7f6bb406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:10.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.246+0000 7f6bc67fc640 1 -- 172.21.15.76:0/2566889509 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7737+0+0 (secure 0 0 0) 0x7f6bcc088f90 con 0x7f6bd00b0c40 2023-11-09T03:52:10.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.247+0000 7f6bd58b2640 1 --2- 172.21.15.76:0/2566889509 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6bb406aea0 0x7f6bb406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:10.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.247+0000 7f6bd58b2640 1 --2- 172.21.15.76:0/2566889509 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6bb406aea0 0x7f6bb406d360 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0000f60 tx=0x7f6bc00079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:10.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.251+0000 7f6bc67fc640 1 -- 172.21.15.76:0/2566889509 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6bcc0bbca0 con 0x7f6bd00b0c40 2023-11-09T03:52:10.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.451+0000 7f6bd7b16640 1 -- 172.21.15.76:0/2566889509 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6bd00c56f0 con 0x7f6bd00b0c40 2023-11-09T03:52:10.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.452+0000 7f6bc67fc640 1 -- 172.21.15.76:0/2566889509 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4929 (secure 0 0 0) 0x7f6bcc018020 con 0x7f6bd00b0c40 2023-11-09T03:52:10.453 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:52:10.453 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":95,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8310,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:52:03.885950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6904,"mds_1":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6904":{"gid":6904,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":16,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8274":{"gid":8274,"name":"c","rank":1,"incarnation":89,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6833/1536527256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1536527256},{"type":"v1","addr":"172.21.15.76:6833","nonce":1536527256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:52:10.453 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 95 2023-11-09T03:52:10.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.454+0000 7f6bd7b16640 1 -- 172.21.15.76:0/2566889509 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6bb406aea0 msgr2=0x7f6bb406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:10.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.454+0000 7f6bd7b16640 1 --2- 172.21.15.76:0/2566889509 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6bb406aea0 0x7f6bb406d360 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0000f60 tx=0x7f6bc00079b0 comp rx=0 tx=0).stop 2023-11-09T03:52:10.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.454+0000 7f6bd7b16640 1 -- 172.21.15.76:0/2566889509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bd00b0c40 msgr2=0x7f6bd00c4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:10.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.454+0000 7f6bd7b16640 1 --2- 172.21.15.76:0/2566889509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bd00b0c40 0x7f6bd00c4790 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc00c4a0 tx=0x7f6bcc00c960 comp rx=0 tx=0).stop 2023-11-09T03:52:10.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.454+0000 7f6bd7b16640 1 -- 172.21.15.76:0/2566889509 shutdown_connections 2023-11-09T03:52:10.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.454+0000 7f6bd7b16640 1 --2- 172.21.15.76:0/2566889509 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6bb406aea0 0x7f6bb406d360 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:10.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.454+0000 7f6bd7b16640 1 --2- 172.21.15.76:0/2566889509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bd0159e40 0x7f6bd00c4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:10.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.454+0000 7f6bd7b16640 1 --2- 172.21.15.76:0/2566889509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bd00b0c40 0x7f6bd00c4790 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:10.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.454+0000 7f6bd7b16640 1 --2- 172.21.15.76:0/2566889509 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bd0070700 0x7f6bd015bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:10.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.454+0000 7f6bd7b16640 1 -- 172.21.15.76:0/2566889509 >> 172.21.15.76:0/2566889509 conn(0x7f6bd00cb530 msgr2=0x7f6bd006fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:10.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.454+0000 7f6bd7b16640 1 -- 172.21.15.76:0/2566889509 shutdown_connections 2023-11-09T03:52:10.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:10.454+0000 7f6bd7b16640 1 -- 172.21.15.76:0/2566889509 wait complete. 2023-11-09T03:52:10.465 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:52:12.208 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~7s 2023-11-09T03:52:15.467 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:52:15.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.579+0000 7f6b9b590640 1 Processor -- start 2023-11-09T03:52:15.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.580+0000 7f6b9b590640 1 -- start start 2023-11-09T03:52:15.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.580+0000 7f6b9b590640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b9415d100 0x7f6b940ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:15.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.580+0000 7f6b9b590640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b9415d4e0 0x7f6b940d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:15.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.580+0000 7f6b9b590640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b940d3b60 0x7f6b940d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:15.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.580+0000 7f6b9b590640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b940b32b0 con 0x7f6b940d3b60 2023-11-09T03:52:15.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.581+0000 7f6b9b590640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b940cee10 con 0x7f6b9415d100 2023-11-09T03:52:15.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.581+0000 7f6b9b590640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b940b2980 con 0x7f6b9415d4e0 2023-11-09T03:52:15.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.581+0000 7f6b99b2d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b940d3b60 0x7f6b940d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:15.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.581+0000 7f6b99b2d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b940d3b60 0x7f6b940d1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35920/0 (socket says 172.21.15.76:35920) 2023-11-09T03:52:15.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.581+0000 7f6b99b2d640 1 -- 172.21.15.76:0/3385345323 learned_addr learned my addr 172.21.15.76:0/3385345323 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:15.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.581+0000 7f6b9932c640 1 --2- 172.21.15.76:0/3385345323 >> v2:172.21.15.113:3300/0 conn(0x7f6b9415d100 0x7f6b940ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:15.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.581+0000 7f6b98b2b640 1 --2- 172.21.15.76:0/3385345323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b9415d4e0 0x7f6b940d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:15.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.581+0000 7f6b99b2d640 1 -- 172.21.15.76:0/3385345323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b9415d4e0 msgr2=0x7f6b940d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:15.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.581+0000 7f6b99b2d640 1 --2- 172.21.15.76:0/3385345323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b9415d4e0 0x7f6b940d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:15.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.581+0000 7f6b99b2d640 1 -- 172.21.15.76:0/3385345323 >> v2:172.21.15.113:3300/0 conn(0x7f6b9415d100 msgr2=0x7f6b940ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:15.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.581+0000 7f6b99b2d640 1 --2- 172.21.15.76:0/3385345323 >> v2:172.21.15.113:3300/0 conn(0x7f6b9415d100 0x7f6b940ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:15.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.581+0000 7f6b99b2d640 1 -- 172.21.15.76:0/3385345323 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b940b2c60 con 0x7f6b940d3b60 2023-11-09T03:52:15.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.582+0000 7f6b99b2d640 1 --2- 172.21.15.76:0/3385345323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b940d3b60 0x7f6b940d1a10 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f6b940d09f0 tx=0x7f6b8c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2fbb9448cba127 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:15.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.582+0000 7f6b8bfff640 1 -- 172.21.15.76:0/3385345323 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c02ca70 con 0x7f6b940d3b60 2023-11-09T03:52:15.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.582+0000 7f6b8bfff640 1 -- 172.21.15.76:0/3385345323 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8c02cbd0 con 0x7f6b940d3b60 2023-11-09T03:52:15.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.582+0000 7f6b8bfff640 1 -- 172.21.15.76:0/3385345323 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c0343f0 con 0x7f6b940d3b60 2023-11-09T03:52:15.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.582+0000 7f6b9b590640 1 -- 172.21.15.76:0/3385345323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b940d3b60 msgr2=0x7f6b940d1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:15.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.582+0000 7f6b9b590640 1 --2- 172.21.15.76:0/3385345323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b940d3b60 0x7f6b940d1a10 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f6b940d09f0 tx=0x7f6b8c02bed0 comp rx=0 tx=0).stop 2023-11-09T03:52:15.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.582+0000 7f6b9b590640 1 -- 172.21.15.76:0/3385345323 shutdown_connections 2023-11-09T03:52:15.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.582+0000 7f6b9b590640 1 --2- 172.21.15.76:0/3385345323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b940d3b60 0x7f6b940d1a10 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:15.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.582+0000 7f6b9b590640 1 --2- 172.21.15.76:0/3385345323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b9415d4e0 0x7f6b940d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:15.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.582+0000 7f6b9b590640 1 --2- 172.21.15.76:0/3385345323 >> v2:172.21.15.113:3300/0 conn(0x7f6b9415d100 0x7f6b940ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:15.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.582+0000 7f6b9b590640 1 -- 172.21.15.76:0/3385345323 >> 172.21.15.76:0/3385345323 conn(0x7f6b940d2ef0 msgr2=0x7f6b940d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:15.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.583+0000 7f6b9b590640 1 -- 172.21.15.76:0/3385345323 shutdown_connections 2023-11-09T03:52:15.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.583+0000 7f6b9b590640 1 -- 172.21.15.76:0/3385345323 wait complete. 2023-11-09T03:52:15.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.584+0000 7f6b9b590640 1 Processor -- start 2023-11-09T03:52:15.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.584+0000 7f6b9b590640 1 -- start start 2023-11-09T03:52:15.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.584+0000 7f6b9b590640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b940d3b60 0x7f6b9414a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:15.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.584+0000 7f6b9b590640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b9415d100 0x7f6b940b1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:15.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.584+0000 7f6b9b590640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b9415d4e0 0x7f6b940b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:15.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.584+0000 7f6b9b590640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9414ca90 con 0x7f6b9415d4e0 2023-11-09T03:52:15.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.584+0000 7f6b9b590640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9414ae90 con 0x7f6b9415d100 2023-11-09T03:52:15.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.584+0000 7f6b9b590640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9414be80 con 0x7f6b940d3b60 2023-11-09T03:52:15.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.585+0000 7f6b99b2d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b9415d4e0 0x7f6b940b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:15.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.585+0000 7f6b99b2d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b9415d4e0 0x7f6b940b19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35932/0 (socket says 172.21.15.76:35932) 2023-11-09T03:52:15.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.585+0000 7f6b99b2d640 1 -- 172.21.15.76:0/462149840 learned_addr learned my addr 172.21.15.76:0/462149840 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:15.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.585+0000 7f6b98b2b640 1 --2- 172.21.15.76:0/462149840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b9415d100 0x7f6b940b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:15.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.585+0000 7f6b9932c640 1 --2- 172.21.15.76:0/462149840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b940d3b60 0x7f6b9414a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:15.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.585+0000 7f6b99b2d640 1 -- 172.21.15.76:0/462149840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b940d3b60 msgr2=0x7f6b9414a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:15.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.585+0000 7f6b99b2d640 1 --2- 172.21.15.76:0/462149840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b940d3b60 0x7f6b9414a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:15.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.585+0000 7f6b99b2d640 1 -- 172.21.15.76:0/462149840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b9415d100 msgr2=0x7f6b940b1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:15.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.585+0000 7f6b99b2d640 1 --2- 172.21.15.76:0/462149840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b9415d100 0x7f6b940b1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:15.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.585+0000 7f6b99b2d640 1 -- 172.21.15.76:0/462149840 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b8c009b30 con 0x7f6b9415d4e0 2023-11-09T03:52:15.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.585+0000 7f6b99b2d640 1 --2- 172.21.15.76:0/462149840 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b9415d4e0 0x7f6b940b19b0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c036040 tx=0x7f6b8c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:15.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.585+0000 7f6b89ffb640 1 -- 172.21.15.76:0/462149840 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c009510 con 0x7f6b9415d4e0 2023-11-09T03:52:15.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.585+0000 7f6b89ffb640 1 -- 172.21.15.76:0/462149840 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8c03e450 con 0x7f6b9415d4e0 2023-11-09T03:52:15.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.585+0000 7f6b9b590640 1 -- 172.21.15.76:0/462149840 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b9414c160 con 0x7f6b9415d4e0 2023-11-09T03:52:15.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.585+0000 7f6b89ffb640 1 -- 172.21.15.76:0/462149840 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c03e800 con 0x7f6b9415d4e0 2023-11-09T03:52:15.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.585+0000 7f6b9b590640 1 -- 172.21.15.76:0/462149840 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b9414c720 con 0x7f6b9415d4e0 2023-11-09T03:52:15.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.586+0000 7f6b9b590640 1 -- 172.21.15.76:0/462149840 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b9414cc40 con 0x7f6b9415d4e0 2023-11-09T03:52:15.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.587+0000 7f6b89ffb640 1 -- 172.21.15.76:0/462149840 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6b8c03e960 con 0x7f6b9415d4e0 2023-11-09T03:52:15.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.587+0000 7f6b89ffb640 1 --2- 172.21.15.76:0/462149840 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6b7806ae70 0x7f6b7806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:15.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.587+0000 7f6b89ffb640 1 -- 172.21.15.76:0/462149840 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7737+0+0 (secure 0 0 0) 0x7f6b8c0acaf0 con 0x7f6b9415d4e0 2023-11-09T03:52:15.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.587+0000 7f6b9932c640 1 --2- 172.21.15.76:0/462149840 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6b7806ae70 0x7f6b7806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:15.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.588+0000 7f6b9932c640 1 --2- 172.21.15.76:0/462149840 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6b7806ae70 0x7f6b7806d330 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6b9000ba10 tx=0x7f6b90008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:15.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.592+0000 7f6b89ffb640 1 -- 172.21.15.76:0/462149840 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6b9414cc40 con 0x7f6b9415d4e0 2023-11-09T03:52:15.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.793+0000 7f6b9b590640 1 -- 172.21.15.76:0/462149840 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b940684b0 con 0x7f6b9415d4e0 2023-11-09T03:52:15.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.793+0000 7f6b89ffb640 1 -- 172.21.15.76:0/462149840 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4929 (secure 0 0 0) 0x7f6b8c0773b0 con 0x7f6b9415d4e0 2023-11-09T03:52:15.793 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:52:15.793 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":95,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8310,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:52:03.885950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6904,"mds_1":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6904":{"gid":6904,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":16,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8274":{"gid":8274,"name":"c","rank":1,"incarnation":89,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6833/1536527256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1536527256},{"type":"v1","addr":"172.21.15.76:6833","nonce":1536527256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:52:15.794 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 95 2023-11-09T03:52:15.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.795+0000 7f6b9b590640 1 -- 172.21.15.76:0/462149840 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6b7806ae70 msgr2=0x7f6b7806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:15.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.795+0000 7f6b9b590640 1 --2- 172.21.15.76:0/462149840 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6b7806ae70 0x7f6b7806d330 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6b9000ba10 tx=0x7f6b90008040 comp rx=0 tx=0).stop 2023-11-09T03:52:15.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.795+0000 7f6b9b590640 1 -- 172.21.15.76:0/462149840 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b9415d4e0 msgr2=0x7f6b940b19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:15.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.795+0000 7f6b9b590640 1 --2- 172.21.15.76:0/462149840 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b9415d4e0 0x7f6b940b19b0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c036040 tx=0x7f6b8c004a40 comp rx=0 tx=0).stop 2023-11-09T03:52:15.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.795+0000 7f6b9b590640 1 -- 172.21.15.76:0/462149840 shutdown_connections 2023-11-09T03:52:15.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.795+0000 7f6b9b590640 1 --2- 172.21.15.76:0/462149840 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6b7806ae70 0x7f6b7806d330 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:15.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.795+0000 7f6b9b590640 1 --2- 172.21.15.76:0/462149840 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b9415d4e0 0x7f6b940b19b0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:15.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.795+0000 7f6b9b590640 1 --2- 172.21.15.76:0/462149840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b9415d100 0x7f6b940b1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:15.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.795+0000 7f6b9b590640 1 --2- 172.21.15.76:0/462149840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b940d3b60 0x7f6b9414a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:15.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.795+0000 7f6b9b590640 1 -- 172.21.15.76:0/462149840 >> 172.21.15.76:0/462149840 conn(0x7f6b940d2ef0 msgr2=0x7f6b9406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:15.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.795+0000 7f6b9b590640 1 -- 172.21.15.76:0/462149840 shutdown_connections 2023-11-09T03:52:15.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:15.795+0000 7f6b9b590640 1 -- 172.21.15.76:0/462149840 wait complete. 2023-11-09T03:52:15.805 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:52:18.820 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~13s 2023-11-09T03:52:20.807 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:52:20.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.917+0000 7f3324976640 1 Processor -- start 2023-11-09T03:52:20.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.917+0000 7f3324976640 1 -- start start 2023-11-09T03:52:20.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.918+0000 7f3324976640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33200d1be0 0x7f3320167e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:20.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.918+0000 7f3324976640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33200b32b0 con 0x7f332015d100 2023-11-09T03:52:20.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.918+0000 7f3324976640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33200cee10 con 0x7f332015d4e0 2023-11-09T03:52:20.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.918+0000 7f3324976640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33200cec20 con 0x7f33200d1be0 2023-11-09T03:52:20.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.918+0000 7f331e59c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f332015d100 0x7f33200ce510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46178/0 (socket says 172.21.15.76:46178) 2023-11-09T03:52:20.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.918+0000 7f331e59c640 1 -- 172.21.15.76:0/5916643 learned_addr learned my addr 172.21.15.76:0/5916643 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:52:20.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.918+0000 7f331dd9b640 1 -- 172.21.15.76:0/5916643 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 163305948 0 0) 0x7f33200b32b0 con 0x7f332015d100 2023-11-09T03:52:20.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.918+0000 7f331ed9d640 1 --2- 172.21.15.76:0/5916643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33200d1be0 0x7f3320167e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:20.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.918+0000 7f331dd9b640 1 -- 172.21.15.76:0/5916643 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f330c003520 con 0x7f332015d100 2023-11-09T03:52:20.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.918+0000 7f331dd9b640 1 -- 172.21.15.76:0/5916643 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2920409708 0 0) 0x7f330c003520 con 0x7f332015d100 2023-11-09T03:52:20.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.918+0000 7f331dd9b640 1 -- 172.21.15.76:0/5916643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33200d1be0 msgr2=0x7f3320167e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:20.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.918+0000 7f331dd9b640 1 --2- 172.21.15.76:0/5916643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33200d1be0 0x7f3320167e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:20.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.918+0000 7f331dd9b640 1 -- 172.21.15.76:0/5916643 >> v1:172.21.15.113:6789/0 conn(0x7f332015d4e0 legacy=0x7f33200d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:20.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.918+0000 7f331dd9b640 1 -- 172.21.15.76:0/5916643 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33200b4750 con 0x7f332015d100 2023-11-09T03:52:20.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.918+0000 7f331dd9b640 1 -- 172.21.15.76:0/5916643 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f33100017f0 con 0x7f332015d100 2023-11-09T03:52:20.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.918+0000 7f331dd9b640 1 -- 172.21.15.76:0/5916643 >> v1:172.21.15.76:6789/0 conn(0x7f332015d100 legacy=0x7f33200ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:20.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.918+0000 7f331dd9b640 1 --2- 172.21.15.76:0/5916643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f330c0039a0 0x7f330c003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:20.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.918+0000 7f331dd9b640 1 --2- 172.21.15.76:0/5916643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f330c004350 0x7f330c004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:20.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.918+0000 7f331dd9b640 1 --2- 172.21.15.76:0/5916643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33200d1be0 0x7f330c005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:20.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.918+0000 7f331dd9b640 1 -- 172.21.15.76:0/5916643 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c00b980 con 0x7f330c004350 2023-11-09T03:52:20.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.918+0000 7f331dd9b640 1 -- 172.21.15.76:0/5916643 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c00baf0 con 0x7f330c0039a0 2023-11-09T03:52:20.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.918+0000 7f331dd9b640 1 -- 172.21.15.76:0/5916643 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f330c00bcb0 con 0x7f33200d1be0 2023-11-09T03:52:20.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.919+0000 7f3317fff640 1 --2- 172.21.15.76:0/5916643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f330c004350 0x7f330c004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:20.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.919+0000 7f331e59c640 1 --2- 172.21.15.76:0/5916643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f330c0039a0 0x7f330c003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:20.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.919+0000 7f331ed9d640 1 --2- 172.21.15.76:0/5916643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33200d1be0 0x7f330c005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:20.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.919+0000 7f3317fff640 1 -- 172.21.15.76:0/5916643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33200d1be0 msgr2=0x7f330c005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:20.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.919+0000 7f3317fff640 1 --2- 172.21.15.76:0/5916643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33200d1be0 0x7f330c005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:20.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.919+0000 7f3317fff640 1 -- 172.21.15.76:0/5916643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f330c0039a0 msgr2=0x7f330c003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:20.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.919+0000 7f3317fff640 1 --2- 172.21.15.76:0/5916643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f330c0039a0 0x7f330c003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:20.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.919+0000 7f3317fff640 1 -- 172.21.15.76:0/5916643 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330c003520 con 0x7f330c004350 2023-11-09T03:52:20.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.920+0000 7f3317fff640 1 --2- 172.21.15.76:0/5916643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f330c004350 0x7f330c004820 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f3308001830 tx=0x7f330c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff957a4b5eddbae server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:20.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.920+0000 7f331dd9b640 1 -- 172.21.15.76:0/5916643 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330802f050 con 0x7f330c004350 2023-11-09T03:52:20.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.920+0000 7f331dd9b640 1 -- 172.21.15.76:0/5916643 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3308033610 con 0x7f330c004350 2023-11-09T03:52:20.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.920+0000 7f331dd9b640 1 -- 172.21.15.76:0/5916643 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33080339c0 con 0x7f330c004350 2023-11-09T03:52:20.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.920+0000 7f3324976640 1 -- 172.21.15.76:0/5916643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f330c004350 msgr2=0x7f330c004820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:20.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.920+0000 7f3324976640 1 --2- 172.21.15.76:0/5916643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f330c004350 0x7f330c004820 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f3308001830 tx=0x7f330c0057f0 comp rx=0 tx=0).stop 2023-11-09T03:52:20.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.920+0000 7f331ed9d640 1 -- 172.21.15.76:0/5916643 reap_dead start 2023-11-09T03:52:20.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.920+0000 7f3324976640 1 -- 172.21.15.76:0/5916643 shutdown_connections 2023-11-09T03:52:20.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.920+0000 7f3324976640 1 -- 172.21.15.76:0/5916643 >> 172.21.15.76:0/5916643 conn(0x7f33200d2ef0 msgr2=0x7f33200d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:20.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.920+0000 7f3324976640 1 -- 172.21.15.76:0/5916643 shutdown_connections 2023-11-09T03:52:20.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.920+0000 7f3324976640 1 -- 172.21.15.76:0/5916643 wait complete. 2023-11-09T03:52:20.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.922+0000 7f3324976640 1 Processor -- start 2023-11-09T03:52:20.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.922+0000 7f3324976640 1 -- start start 2023-11-09T03:52:20.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.922+0000 7f3324976640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f330c004350 0x7f33201f2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:20.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.922+0000 7f3324976640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3320157e20 0x7f33201582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:20.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.922+0000 7f3324976640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33200cb280 0x7f33200cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:20.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.922+0000 7f3324976640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332015deb0 con 0x7f33200cb280 2023-11-09T03:52:20.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.922+0000 7f3324976640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33200cbfe0 con 0x7f330c004350 2023-11-09T03:52:20.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.922+0000 7f3324976640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33200cc2c0 con 0x7f3320157e20 2023-11-09T03:52:20.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.923+0000 7f331ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33200cb280 0x7f33200cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:20.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.923+0000 7f331ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33200cb280 0x7f33200cb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35946/0 (socket says 172.21.15.76:35946) 2023-11-09T03:52:20.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.923+0000 7f331ed9d640 1 -- 172.21.15.76:0/3069006139 learned_addr learned my addr 172.21.15.76:0/3069006139 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:20.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.923+0000 7f331e59c640 1 --2- 172.21.15.76:0/3069006139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f330c004350 0x7f33201f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:20.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.923+0000 7f3317fff640 1 --2- 172.21.15.76:0/3069006139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3320157e20 0x7f33201582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:20.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.923+0000 7f331ed9d640 1 -- 172.21.15.76:0/3069006139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3320157e20 msgr2=0x7f33201582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:20.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.923+0000 7f331ed9d640 1 --2- 172.21.15.76:0/3069006139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3320157e20 0x7f33201582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:20.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.923+0000 7f331ed9d640 1 -- 172.21.15.76:0/3069006139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f330c004350 msgr2=0x7f33201f2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:20.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.923+0000 7f331ed9d640 1 --2- 172.21.15.76:0/3069006139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f330c004350 0x7f33201f2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:20.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.923+0000 7f331ed9d640 1 -- 172.21.15.76:0/3069006139 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33200cc5a0 con 0x7f33200cb280 2023-11-09T03:52:20.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.923+0000 7f331ed9d640 1 --2- 172.21.15.76:0/3069006139 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33200cb280 0x7f33200cb700 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f332008d6c0 tx=0x7f331800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:20.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.923+0000 7f3316ffd640 1 -- 172.21.15.76:0/3069006139 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331800e040 con 0x7f33200cb280 2023-11-09T03:52:20.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.923+0000 7f3316ffd640 1 -- 172.21.15.76:0/3069006139 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331800f040 con 0x7f33200cb280 2023-11-09T03:52:20.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.923+0000 7f3324976640 1 -- 172.21.15.76:0/3069006139 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33200cc880 con 0x7f33200cb280 2023-11-09T03:52:20.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.923+0000 7f3316ffd640 1 -- 172.21.15.76:0/3069006139 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3318015670 con 0x7f33200cb280 2023-11-09T03:52:20.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.923+0000 7f3324976640 1 -- 172.21.15.76:0/3069006139 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33201f2680 con 0x7f33200cb280 2023-11-09T03:52:20.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.924+0000 7f3324976640 1 -- 172.21.15.76:0/3069006139 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33200684b0 con 0x7f33200cb280 2023-11-09T03:52:20.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.925+0000 7f3316ffd640 1 -- 172.21.15.76:0/3069006139 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f33180157d0 con 0x7f33200cb280 2023-11-09T03:52:20.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.925+0000 7f3316ffd640 1 --2- 172.21.15.76:0/3069006139 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f330006aea0 0x7f330006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:20.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.925+0000 7f3316ffd640 1 -- 172.21.15.76:0/3069006139 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7737+0+0 (secure 0 0 0) 0x7f331808a480 con 0x7f33200cb280 2023-11-09T03:52:20.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.925+0000 7f331e59c640 1 --2- 172.21.15.76:0/3069006139 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f330006aea0 0x7f330006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:20.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.926+0000 7f331e59c640 1 --2- 172.21.15.76:0/3069006139 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f330006aea0 0x7f330006d360 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f33100011d0 tx=0x7f3310009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:20.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:20.929+0000 7f3316ffd640 1 -- 172.21.15.76:0/3069006139 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3318054df0 con 0x7f33200cb280 2023-11-09T03:52:21.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:21.132+0000 7f3324976640 1 -- 172.21.15.76:0/3069006139 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33200d2cd0 con 0x7f33200cb280 2023-11-09T03:52:21.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:21.133+0000 7f3316ffd640 1 -- 172.21.15.76:0/3069006139 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4929 (secure 0 0 0) 0x7f3318058410 con 0x7f33200cb280 2023-11-09T03:52:21.133 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:52:21.133 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":95,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8310,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:52:03.885950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6904,"mds_1":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6904":{"gid":6904,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":16,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8274":{"gid":8274,"name":"c","rank":1,"incarnation":89,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6833/1536527256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1536527256},{"type":"v1","addr":"172.21.15.76:6833","nonce":1536527256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:52:21.133 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 95 2023-11-09T03:52:21.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:21.134+0000 7f3324976640 1 -- 172.21.15.76:0/3069006139 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f330006aea0 msgr2=0x7f330006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:21.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:21.134+0000 7f3324976640 1 --2- 172.21.15.76:0/3069006139 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f330006aea0 0x7f330006d360 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f33100011d0 tx=0x7f3310009040 comp rx=0 tx=0).stop 2023-11-09T03:52:21.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:21.134+0000 7f3324976640 1 -- 172.21.15.76:0/3069006139 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33200cb280 msgr2=0x7f33200cb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:21.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:21.134+0000 7f3324976640 1 --2- 172.21.15.76:0/3069006139 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33200cb280 0x7f33200cb700 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f332008d6c0 tx=0x7f331800f580 comp rx=0 tx=0).stop 2023-11-09T03:52:21.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:21.134+0000 7f3324976640 1 -- 172.21.15.76:0/3069006139 shutdown_connections 2023-11-09T03:52:21.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:21.134+0000 7f3324976640 1 --2- 172.21.15.76:0/3069006139 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f330006aea0 0x7f330006d360 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:21.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:21.134+0000 7f3324976640 1 --2- 172.21.15.76:0/3069006139 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33200cb280 0x7f33200cb700 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:21.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:21.134+0000 7f3324976640 1 --2- 172.21.15.76:0/3069006139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3320157e20 0x7f33201582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:21.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:21.134+0000 7f3324976640 1 --2- 172.21.15.76:0/3069006139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f330c004350 0x7f33201f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:21.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:21.134+0000 7f3324976640 1 -- 172.21.15.76:0/3069006139 >> 172.21.15.76:0/3069006139 conn(0x7f33200d2ef0 msgr2=0x7f33200d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:21.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:21.135+0000 7f3324976640 1 -- 172.21.15.76:0/3069006139 shutdown_connections 2023-11-09T03:52:21.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:21.135+0000 7f3324976640 1 -- 172.21.15.76:0/3069006139 wait complete. 2023-11-09T03:52:21.145 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:52:25.431 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~20s 2023-11-09T03:52:26.146 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:52:26.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.261+0000 7f06a8189640 1 Processor -- start 2023-11-09T03:52:26.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.261+0000 7f06a8189640 1 -- start start 2023-11-09T03:52:26.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.262+0000 7f06a8189640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f06a006d9c0 0x7f06a00710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:26.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.262+0000 7f06a8189640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f06a006c5f0 0x7f06a0071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:26.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.262+0000 7f06a8189640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06a006cc90 con 0x7f06a00ade00 2023-11-09T03:52:26.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.262+0000 7f06a8189640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a00b0c40 con 0x7f06a006d9c0 2023-11-09T03:52:26.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.262+0000 7f06a8189640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a0071b50 con 0x7f06a006c5f0 2023-11-09T03:52:26.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.262+0000 7f06a5724640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f06a006d9c0 0x7f06a00710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:26.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.262+0000 7f06a5724640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f06a006d9c0 0x7f06a00710d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42380/0 (socket says 172.21.15.76:42380) 2023-11-09T03:52:26.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.262+0000 7f06a5724640 1 -- 172.21.15.76:0/1715684122 learned_addr learned my addr 172.21.15.76:0/1715684122 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:26.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.262+0000 7f06a6726640 1 --2- 172.21.15.76:0/1715684122 >> v2:172.21.15.113:3300/0 conn(0x7f06a006c5f0 0x7f06a0071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:26.264 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.262+0000 7f06a5724640 1 -- 172.21.15.76:0/1715684122 >> v2:172.21.15.113:3300/0 conn(0x7f06a006c5f0 msgr2=0x7f06a0071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:26.264 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.262+0000 7f06a5724640 1 --2- 172.21.15.76:0/1715684122 >> v2:172.21.15.113:3300/0 conn(0x7f06a006c5f0 0x7f06a0071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:26.264 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.262+0000 7f06a5724640 1 -- 172.21.15.76:0/1715684122 >> v1:172.21.15.113:6789/0 conn(0x7f06a00ade00 legacy=0x7f06a00b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:26.264 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.262+0000 7f06a5724640 1 -- 172.21.15.76:0/1715684122 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a00b0e60 con 0x7f06a006d9c0 2023-11-09T03:52:26.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.263+0000 7f06a5724640 1 --2- 172.21.15.76:0/1715684122 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06a006d9c0 0x7f06a00710d0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f06a00b44f0 tx=0x7f069002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89657ce0263f113a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:26.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.263+0000 7f06a4f23640 1 -- 172.21.15.76:0/1715684122 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06900317e0 con 0x7f06a006d9c0 2023-11-09T03:52:26.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.263+0000 7f06a4f23640 1 -- 172.21.15.76:0/1715684122 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0690031940 con 0x7f06a006d9c0 2023-11-09T03:52:26.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.263+0000 7f06a4f23640 1 -- 172.21.15.76:0/1715684122 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0690031cd0 con 0x7f06a006d9c0 2023-11-09T03:52:26.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.264+0000 7f06a8189640 1 -- 172.21.15.76:0/1715684122 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06a006d9c0 msgr2=0x7f06a00710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:26.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.264+0000 7f06a8189640 1 --2- 172.21.15.76:0/1715684122 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06a006d9c0 0x7f06a00710d0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f06a00b44f0 tx=0x7f069002b880 comp rx=0 tx=0).stop 2023-11-09T03:52:26.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.264+0000 7f06a8189640 1 -- 172.21.15.76:0/1715684122 shutdown_connections 2023-11-09T03:52:26.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.264+0000 7f06a8189640 1 --2- 172.21.15.76:0/1715684122 >> v2:172.21.15.113:3300/0 conn(0x7f06a006c5f0 0x7f06a0071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:26.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.264+0000 7f06a8189640 1 --2- 172.21.15.76:0/1715684122 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06a006d9c0 0x7f06a00710d0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:26.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.264+0000 7f06a8189640 1 -- 172.21.15.76:0/1715684122 >> 172.21.15.76:0/1715684122 conn(0x7f06a00cb530 msgr2=0x7f06a00cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:26.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.264+0000 7f06a8189640 1 -- 172.21.15.76:0/1715684122 shutdown_connections 2023-11-09T03:52:26.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.264+0000 7f06a8189640 1 -- 172.21.15.76:0/1715684122 wait complete. 2023-11-09T03:52:26.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.265+0000 7f06a8189640 1 Processor -- start 2023-11-09T03:52:26.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.265+0000 7f06a8189640 1 -- start start 2023-11-09T03:52:26.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.266+0000 7f06a8189640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06a006c5f0 0x7f06a0159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:26.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.266+0000 7f06a8189640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06a006d9c0 0x7f06a00c6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:26.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.266+0000 7f06a8189640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06a00ade00 0x7f06a014f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:26.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.266+0000 7f06a8189640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a0159550 con 0x7f06a006d9c0 2023-11-09T03:52:26.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.266+0000 7f06a8189640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a00c6ad0 con 0x7f06a00ade00 2023-11-09T03:52:26.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.266+0000 7f06a8189640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a0154b10 con 0x7f06a006c5f0 2023-11-09T03:52:26.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.266+0000 7f06a5724640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06a006d9c0 0x7f06a00c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:26.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.266+0000 7f06a5724640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06a006d9c0 0x7f06a00c6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42394/0 (socket says 172.21.15.76:42394) 2023-11-09T03:52:26.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.266+0000 7f06a5724640 1 -- 172.21.15.76:0/1336144414 learned_addr learned my addr 172.21.15.76:0/1336144414 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:26.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.266+0000 7f06a6726640 1 --2- 172.21.15.76:0/1336144414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06a00ade00 0x7f06a014f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:26.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.266+0000 7f06a5f25640 1 --2- 172.21.15.76:0/1336144414 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06a006c5f0 0x7f06a0159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:26.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.266+0000 7f06a5724640 1 -- 172.21.15.76:0/1336144414 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06a006c5f0 msgr2=0x7f06a0159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:26.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.266+0000 7f06a5724640 1 --2- 172.21.15.76:0/1336144414 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06a006c5f0 0x7f06a0159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:26.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.266+0000 7f06a5724640 1 -- 172.21.15.76:0/1336144414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06a00ade00 msgr2=0x7f06a014f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:26.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.266+0000 7f06a5724640 1 --2- 172.21.15.76:0/1336144414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06a00ade00 0x7f06a014f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:26.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.266+0000 7f06a5724640 1 -- 172.21.15.76:0/1336144414 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06900094d0 con 0x7f06a006d9c0 2023-11-09T03:52:26.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.266+0000 7f06a5724640 1 --2- 172.21.15.76:0/1336144414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06a006d9c0 0x7f06a00c6590 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f0690000c00 tx=0x7f06900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:26.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.266+0000 7f06967fc640 1 -- 172.21.15.76:0/1336144414 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0690032880 con 0x7f06a006d9c0 2023-11-09T03:52:26.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.267+0000 7f06967fc640 1 -- 172.21.15.76:0/1336144414 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0690032a20 con 0x7f06a006d9c0 2023-11-09T03:52:26.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.267+0000 7f06a8189640 1 -- 172.21.15.76:0/1336144414 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a00c6d70 con 0x7f06a006d9c0 2023-11-09T03:52:26.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.267+0000 7f06967fc640 1 -- 172.21.15.76:0/1336144414 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0690032dd0 con 0x7f06a006d9c0 2023-11-09T03:52:26.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.267+0000 7f06a8189640 1 -- 172.21.15.76:0/1336144414 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a01550b0 con 0x7f06a006d9c0 2023-11-09T03:52:26.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.268+0000 7f06a8189640 1 -- 172.21.15.76:0/1336144414 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a00684b0 con 0x7f06a006d9c0 2023-11-09T03:52:26.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.268+0000 7f06967fc640 1 -- 172.21.15.76:0/1336144414 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f069004e450 con 0x7f06a006d9c0 2023-11-09T03:52:26.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.268+0000 7f06967fc640 1 --2- 172.21.15.76:0/1336144414 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f068406aea0 0x7f068406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:26.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.268+0000 7f06967fc640 1 -- 172.21.15.76:0/1336144414 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7737+0+0 (secure 0 0 0) 0x7f06900abdb0 con 0x7f06a006d9c0 2023-11-09T03:52:26.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.269+0000 7f06a5f25640 1 --2- 172.21.15.76:0/1336144414 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f068406aea0 0x7f068406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:26.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.269+0000 7f06a5f25640 1 --2- 172.21.15.76:0/1336144414 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f068406aea0 0x7f068406d360 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f069c0012e0 tx=0x7f069c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:26.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.273+0000 7f06967fc640 1 -- 172.21.15.76:0/1336144414 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0690076700 con 0x7f06a006d9c0 2023-11-09T03:52:26.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.475+0000 7f06a8189640 1 -- 172.21.15.76:0/1336144414 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06a00cddc0 con 0x7f06a006d9c0 2023-11-09T03:52:26.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.475+0000 7f06967fc640 1 -- 172.21.15.76:0/1336144414 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4977 (secure 0 0 0) 0x7f0690079d20 con 0x7f06a006d9c0 2023-11-09T03:52:26.475 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:52:26.475 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":96,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8310,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:52:21.461112+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6904,"mds_1":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6904":{"gid":6904,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":16,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"laggy_since":"2023-11-09T03:52:21.461112+0000","export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8274":{"gid":8274,"name":"c","rank":1,"incarnation":89,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6833/1536527256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1536527256},{"type":"v1","addr":"172.21.15.76:6833","nonce":1536527256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:52:26.476 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 96 2023-11-09T03:52:26.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.477+0000 7f06a8189640 1 -- 172.21.15.76:0/1336144414 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f068406aea0 msgr2=0x7f068406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:26.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.477+0000 7f06a8189640 1 --2- 172.21.15.76:0/1336144414 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f068406aea0 0x7f068406d360 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f069c0012e0 tx=0x7f069c009040 comp rx=0 tx=0).stop 2023-11-09T03:52:26.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.477+0000 7f06a8189640 1 -- 172.21.15.76:0/1336144414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06a006d9c0 msgr2=0x7f06a00c6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:26.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.477+0000 7f06a8189640 1 --2- 172.21.15.76:0/1336144414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06a006d9c0 0x7f06a00c6590 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f0690000c00 tx=0x7f06900046c0 comp rx=0 tx=0).stop 2023-11-09T03:52:26.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.477+0000 7f06a8189640 1 -- 172.21.15.76:0/1336144414 shutdown_connections 2023-11-09T03:52:26.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.477+0000 7f06a8189640 1 --2- 172.21.15.76:0/1336144414 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f068406aea0 0x7f068406d360 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:26.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.477+0000 7f06a8189640 1 --2- 172.21.15.76:0/1336144414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06a00ade00 0x7f06a014f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:26.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.477+0000 7f06a8189640 1 --2- 172.21.15.76:0/1336144414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06a006d9c0 0x7f06a00c6590 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:26.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.477+0000 7f06a8189640 1 --2- 172.21.15.76:0/1336144414 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06a006c5f0 0x7f06a0159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:26.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.477+0000 7f06a8189640 1 -- 172.21.15.76:0/1336144414 >> 172.21.15.76:0/1336144414 conn(0x7f06a00cb530 msgr2=0x7f06a00ae270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:26.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.477+0000 7f06a8189640 1 -- 172.21.15.76:0/1336144414 shutdown_connections 2023-11-09T03:52:26.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.477+0000 7f06a8189640 1 -- 172.21.15.76:0/1336144414 wait complete. 2023-11-09T03:52:26.487 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2023-11-09T03:52:26.488 DEBUG:teuthology.orchestra.run.smithi113:> sudo sysctl -n kernel.core_pattern 2023-11-09T03:52:26.519 INFO:teuthology.orchestra.run.smithi113.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-09T03:52:26.520 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2023-11-09T03:52:26.520 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2023-11-09T03:52:26.593 INFO:teuthology.orchestra.run.smithi113.stdout:1699501924.95952.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i b', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2023-11-09T03:52:26.595 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1699501924.95952.core'] 2023-11-09T03:52:26.595 INFO:tasks.cephfs.cephfs_test_case:Found core file 1699501924.95952.core, deleting it 2023-11-09T03:52:26.596 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1699501924.95952.core 2023-11-09T03:52:26.667 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:52:26.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.769+0000 7f9092f90640 1 Processor -- start 2023-11-09T03:52:26.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.769+0000 7f9092f90640 1 -- start start 2023-11-09T03:52:26.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.769+0000 7f9092f90640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f908c159e40 0x7f908c0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:26.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.769+0000 7f9092f90640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f908c071500 con 0x7f908c06bb60 2023-11-09T03:52:26.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.770+0000 7f9092f90640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f908c15dae0 con 0x7f908c070700 2023-11-09T03:52:26.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.770+0000 7f9092f90640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f908c070c60 con 0x7f908c159e40 2023-11-09T03:52:26.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.770+0000 7f908bfff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f908c06bb60 0x7f908c06fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:41654/0 (socket says 172.21.15.76:41654) 2023-11-09T03:52:26.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.770+0000 7f908bfff640 1 -- 172.21.15.76:0/2872756062 learned_addr learned my addr 172.21.15.76:0/2872756062 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:52:26.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.770+0000 7f908b7fe640 1 -- 172.21.15.76:0/2872756062 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4055444550 0 0) 0x7f908c071500 con 0x7f908c06bb60 2023-11-09T03:52:26.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.770+0000 7f908b7fe640 1 -- 172.21.15.76:0/2872756062 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9078003520 con 0x7f908c06bb60 2023-11-09T03:52:26.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.770+0000 7f9090d2c640 1 --2- 172.21.15.76:0/2872756062 >> v2:172.21.15.113:3300/0 conn(0x7f908c159e40 0x7f908c0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:26.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.770+0000 7f908b7fe640 1 -- 172.21.15.76:0/2872756062 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1591848512 0 0) 0x7f9078003520 con 0x7f908c06bb60 2023-11-09T03:52:26.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.770+0000 7f908b7fe640 1 -- 172.21.15.76:0/2872756062 >> v2:172.21.15.113:3300/0 conn(0x7f908c159e40 msgr2=0x7f908c0b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:26.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.770+0000 7f908b7fe640 1 --2- 172.21.15.76:0/2872756062 >> v2:172.21.15.113:3300/0 conn(0x7f908c159e40 0x7f908c0b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:26.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.770+0000 7f908b7fe640 1 -- 172.21.15.76:0/2872756062 >> v1:172.21.15.113:6789/0 conn(0x7f908c070700 legacy=0x7f908c0cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:26.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.770+0000 7f908b7fe640 1 -- 172.21.15.76:0/2872756062 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908c070f40 con 0x7f908c06bb60 2023-11-09T03:52:26.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.770+0000 7f908b7fe640 1 -- 172.21.15.76:0/2872756062 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f9074001670 con 0x7f908c06bb60 2023-11-09T03:52:26.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.770+0000 7f908b7fe640 1 -- 172.21.15.76:0/2872756062 >> v1:172.21.15.76:6789/0 conn(0x7f908c06bb60 legacy=0x7f908c06fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:26.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.770+0000 7f908b7fe640 1 --2- 172.21.15.76:0/2872756062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9078003970 0x7f9078003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:26.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.770+0000 7f908b7fe640 1 --2- 172.21.15.76:0/2872756062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90780047c0 0x7f9078004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:26.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.770+0000 7f908b7fe640 1 --2- 172.21.15.76:0/2872756062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90780052a0 0x7f907800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:26.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.770+0000 7f908b7fe640 1 -- 172.21.15.76:0/2872756062 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c070c60 con 0x7f90780047c0 2023-11-09T03:52:26.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.770+0000 7f908b7fe640 1 -- 172.21.15.76:0/2872756062 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907800c170 con 0x7f9078003970 2023-11-09T03:52:26.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.770+0000 7f908b7fe640 1 -- 172.21.15.76:0/2872756062 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f907800c330 con 0x7f90780052a0 2023-11-09T03:52:26.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.770+0000 7f908bfff640 1 --2- 172.21.15.76:0/2872756062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90780047c0 0x7f9078004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:26.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.770+0000 7f9090d2c640 1 --2- 172.21.15.76:0/2872756062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9078003970 0x7f9078003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:26.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.770+0000 7f909152d640 1 --2- 172.21.15.76:0/2872756062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90780052a0 0x7f907800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:26.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.771+0000 7f908bfff640 1 -- 172.21.15.76:0/2872756062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90780052a0 msgr2=0x7f907800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:26.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.771+0000 7f908bfff640 1 --2- 172.21.15.76:0/2872756062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90780052a0 0x7f907800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:26.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.771+0000 7f908bfff640 1 -- 172.21.15.76:0/2872756062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9078003970 msgr2=0x7f9078003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:26.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.771+0000 7f908bfff640 1 --2- 172.21.15.76:0/2872756062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9078003970 0x7f9078003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:26.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.771+0000 7f908bfff640 1 -- 172.21.15.76:0/2872756062 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9078003520 con 0x7f90780047c0 2023-11-09T03:52:26.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.771+0000 7f909152d640 1 --2- 172.21.15.76:0/2872756062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90780052a0 0x7f907800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:52:26.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.771+0000 7f909152d640 1 -- 172.21.15.76:0/2872756062 reap_dead start 2023-11-09T03:52:26.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.771+0000 7f908bfff640 1 --2- 172.21.15.76:0/2872756062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90780047c0 0x7f9078004c40 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f907400c210 tx=0x7f90780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21969dd3484d3dbe server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:26.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.771+0000 7f908b7fe640 1 -- 172.21.15.76:0/2872756062 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074005ad0 con 0x7f90780047c0 2023-11-09T03:52:26.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.772+0000 7f908b7fe640 1 -- 172.21.15.76:0/2872756062 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9074005c30 con 0x7f90780047c0 2023-11-09T03:52:26.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.772+0000 7f908b7fe640 1 -- 172.21.15.76:0/2872756062 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907402cab0 con 0x7f90780047c0 2023-11-09T03:52:26.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.772+0000 7f9092f90640 1 -- 172.21.15.76:0/2872756062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90780047c0 msgr2=0x7f9078004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:26.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.772+0000 7f9092f90640 1 --2- 172.21.15.76:0/2872756062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90780047c0 0x7f9078004c40 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f907400c210 tx=0x7f90780057f0 comp rx=0 tx=0).stop 2023-11-09T03:52:26.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.772+0000 7f9092f90640 1 -- 172.21.15.76:0/2872756062 shutdown_connections 2023-11-09T03:52:26.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.772+0000 7f9092f90640 1 --2- 172.21.15.76:0/2872756062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90780047c0 0x7f9078004c40 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:26.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.772+0000 7f9092f90640 1 -- 172.21.15.76:0/2872756062 >> 172.21.15.76:0/2872756062 conn(0x7f908c0cb530 msgr2=0x7f908c0cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:26.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.772+0000 7f9092f90640 1 -- 172.21.15.76:0/2872756062 shutdown_connections 2023-11-09T03:52:26.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.772+0000 7f9092f90640 1 -- 172.21.15.76:0/2872756062 wait complete. 2023-11-09T03:52:26.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.773+0000 7f9092f90640 1 Processor -- start 2023-11-09T03:52:26.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.773+0000 7f9092f90640 1 -- start start 2023-11-09T03:52:26.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.774+0000 7f9092f90640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90780047c0 0x7f908c151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:26.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.774+0000 7f9092f90640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f908c14e910 0x7f908c0c50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:26.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.774+0000 7f9092f90640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f908c0cc680 0x7f908c0ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:26.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.774+0000 7f9092f90640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c0cd070 con 0x7f90780047c0 2023-11-09T03:52:26.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.774+0000 7f9092f90640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c0ae190 con 0x7f908c0cc680 2023-11-09T03:52:26.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.774+0000 7f9092f90640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f908c1f3590 con 0x7f908c14e910 2023-11-09T03:52:26.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.774+0000 7f9090d2c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90780047c0 0x7f908c151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:26.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.774+0000 7f9090d2c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90780047c0 0x7f908c151350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42416/0 (socket says 172.21.15.76:42416) 2023-11-09T03:52:26.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.774+0000 7f9090d2c640 1 -- 172.21.15.76:0/4121651975 learned_addr learned my addr 172.21.15.76:0/4121651975 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:26.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.774+0000 7f908bfff640 1 --2- 172.21.15.76:0/4121651975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f908c14e910 0x7f908c0c50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:26.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.774+0000 7f909152d640 1 --2- 172.21.15.76:0/4121651975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f908c0cc680 0x7f908c0ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:26.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.775+0000 7f9090d2c640 1 -- 172.21.15.76:0/4121651975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f908c14e910 msgr2=0x7f908c0c50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:26.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.775+0000 7f9090d2c640 1 --2- 172.21.15.76:0/4121651975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f908c14e910 0x7f908c0c50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:26.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.775+0000 7f9090d2c640 1 -- 172.21.15.76:0/4121651975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f908c0cc680 msgr2=0x7f908c0ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:26.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.775+0000 7f9090d2c640 1 --2- 172.21.15.76:0/4121651975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f908c0cc680 0x7f908c0ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:26.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.775+0000 7f9090d2c640 1 -- 172.21.15.76:0/4121651975 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908c14f060 con 0x7f90780047c0 2023-11-09T03:52:26.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.775+0000 7f9090d2c640 1 --2- 172.21.15.76:0/4121651975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90780047c0 0x7f908c151350 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f9084011e60 tx=0x7f908400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:26.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.775+0000 7f90897fa640 1 -- 172.21.15.76:0/4121651975 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908400e040 con 0x7f90780047c0 2023-11-09T03:52:26.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.775+0000 7f90897fa640 1 -- 172.21.15.76:0/4121651975 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f908400f040 con 0x7f90780047c0 2023-11-09T03:52:26.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.775+0000 7f9092f90640 1 -- 172.21.15.76:0/4121651975 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f908c0c5610 con 0x7f90780047c0 2023-11-09T03:52:26.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.775+0000 7f90897fa640 1 -- 172.21.15.76:0/4121651975 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9084015850 con 0x7f90780047c0 2023-11-09T03:52:26.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.775+0000 7f9092f90640 1 -- 172.21.15.76:0/4121651975 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f908c14ed90 con 0x7f90780047c0 2023-11-09T03:52:26.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.776+0000 7f9092f90640 1 -- 172.21.15.76:0/4121651975 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f908c0684b0 con 0x7f90780047c0 2023-11-09T03:52:26.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.776+0000 7f90897fa640 1 -- 172.21.15.76:0/4121651975 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f90840159b0 con 0x7f90780047c0 2023-11-09T03:52:26.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.777+0000 7f90897fa640 1 --2- 172.21.15.76:0/4121651975 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f906c06aea0 0x7f906c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:26.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.777+0000 7f90897fa640 1 -- 172.21.15.76:0/4121651975 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7737+0+0 (secure 0 0 0) 0x7f908408aa90 con 0x7f90780047c0 2023-11-09T03:52:26.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.777+0000 7f908bfff640 1 --2- 172.21.15.76:0/4121651975 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f906c06aea0 0x7f906c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:26.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.777+0000 7f908bfff640 1 --2- 172.21.15.76:0/4121651975 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f906c06aea0 0x7f906c06d360 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9074034000 tx=0x7f9074009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:26.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.781+0000 7f90897fa640 1 -- 172.21.15.76:0/4121651975 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9084055400 con 0x7f90780047c0 2023-11-09T03:52:26.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.980+0000 7f9092f90640 1 -- 172.21.15.76:0/4121651975 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f908c0c9750 con 0x7f90780047c0 2023-11-09T03:52:26.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.981+0000 7f90897fa640 1 -- 172.21.15.76:0/4121651975 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4977 (secure 0 0 0) 0x7f9084058a20 con 0x7f90780047c0 2023-11-09T03:52:26.981 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:52:26.981 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":96,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8310,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:52:21.461112+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6904,"mds_1":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6904":{"gid":6904,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":16,"addr":"172.21.15.113:6837/445306598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":445306598},{"type":"v1","addr":"172.21.15.113:6837","nonce":445306598}]},"join_fscid":-1,"laggy_since":"2023-11-09T03:52:21.461112+0000","export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8274":{"gid":8274,"name":"c","rank":1,"incarnation":89,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6833/1536527256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1536527256},{"type":"v1","addr":"172.21.15.76:6833","nonce":1536527256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:52:26.981 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 96 2023-11-09T03:52:26.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.983+0000 7f9092f90640 1 -- 172.21.15.76:0/4121651975 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f906c06aea0 msgr2=0x7f906c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:26.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.983+0000 7f9092f90640 1 --2- 172.21.15.76:0/4121651975 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f906c06aea0 0x7f906c06d360 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9074034000 tx=0x7f9074009030 comp rx=0 tx=0).stop 2023-11-09T03:52:26.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.983+0000 7f9092f90640 1 -- 172.21.15.76:0/4121651975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90780047c0 msgr2=0x7f908c151350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:26.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.983+0000 7f9092f90640 1 --2- 172.21.15.76:0/4121651975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90780047c0 0x7f908c151350 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f9084011e60 tx=0x7f908400f580 comp rx=0 tx=0).stop 2023-11-09T03:52:26.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.983+0000 7f9092f90640 1 -- 172.21.15.76:0/4121651975 shutdown_connections 2023-11-09T03:52:26.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.983+0000 7f9092f90640 1 --2- 172.21.15.76:0/4121651975 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f906c06aea0 0x7f906c06d360 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:26.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.983+0000 7f9092f90640 1 --2- 172.21.15.76:0/4121651975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f908c0cc680 0x7f908c0ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:26.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.983+0000 7f9092f90640 1 --2- 172.21.15.76:0/4121651975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f908c14e910 0x7f908c0c50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:26.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.983+0000 7f9092f90640 1 --2- 172.21.15.76:0/4121651975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90780047c0 0x7f908c151350 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:26.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.983+0000 7f9092f90640 1 -- 172.21.15.76:0/4121651975 >> 172.21.15.76:0/4121651975 conn(0x7f908c0cb530 msgr2=0x7f908c0b08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:26.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.983+0000 7f9092f90640 1 -- 172.21.15.76:0/4121651975 shutdown_connections 2023-11-09T03:52:26.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:26.983+0000 7f9092f90640 1 -- 172.21.15.76:0/4121651975 wait complete. 2023-11-09T03:52:26.993 INFO:tasks.ceph.mds.c:Sent signal 9 2023-11-09T03:52:26.994 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:52:26.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:52:26.994 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 down 2023-11-09T03:52:27.022 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:0 2023-11-09T03:52:27.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.163+0000 7ff03f12c640 1 Processor -- start 2023-11-09T03:52:27.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.163+0000 7ff03f12c640 1 -- start start 2023-11-09T03:52:27.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.163+0000 7ff03f12c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff03815d100 0x7ff0380ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:27.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.163+0000 7ff03f12c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff0380d1be0 0x7ff0380b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:27.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.163+0000 7ff03f12c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0380b32b0 con 0x7ff03815d4e0 2023-11-09T03:52:27.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.163+0000 7ff03f12c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0380cee10 con 0x7ff0380d1be0 2023-11-09T03:52:27.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.163+0000 7ff03f12c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0380b2980 con 0x7ff03815d100 2023-11-09T03:52:27.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.164+0000 7ff03d6c9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff0380d1be0 0x7ff0380b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:27.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.164+0000 7ff03d6c9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff0380d1be0 0x7ff0380b3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42418/0 (socket says 172.21.15.76:42418) 2023-11-09T03:52:27.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.164+0000 7ff03d6c9640 1 -- 172.21.15.76:0/1667684319 learned_addr learned my addr 172.21.15.76:0/1667684319 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:27.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.164+0000 7ff03cec8640 1 --2- 172.21.15.76:0/1667684319 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff03815d100 0x7ff0380ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:27.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.164+0000 7ff03d6c9640 1 -- 172.21.15.76:0/1667684319 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff03815d100 msgr2=0x7ff0380ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:27.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.164+0000 7ff03d6c9640 1 --2- 172.21.15.76:0/1667684319 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff03815d100 0x7ff0380ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:27.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.164+0000 7ff03d6c9640 1 -- 172.21.15.76:0/1667684319 >> v1:172.21.15.113:6789/0 conn(0x7ff03815d4e0 legacy=0x7ff0380d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:27.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.164+0000 7ff03d6c9640 1 -- 172.21.15.76:0/1667684319 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0380b2c60 con 0x7ff0380d1be0 2023-11-09T03:52:27.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.165+0000 7ff03d6c9640 1 --2- 172.21.15.76:0/1667684319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0380d1be0 0x7ff0380b3740 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7ff030009e80 tx=0x7ff03002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4f4a4ddff32f61 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:27.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.165+0000 7ff0377fe640 1 -- 172.21.15.76:0/1667684319 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03002caf0 con 0x7ff0380d1be0 2023-11-09T03:52:27.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.165+0000 7ff0377fe640 1 -- 172.21.15.76:0/1667684319 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03002cc50 con 0x7ff0380d1be0 2023-11-09T03:52:27.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.165+0000 7ff0377fe640 1 -- 172.21.15.76:0/1667684319 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0300343f0 con 0x7ff0380d1be0 2023-11-09T03:52:27.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.165+0000 7ff03f12c640 1 -- 172.21.15.76:0/1667684319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0380d1be0 msgr2=0x7ff0380b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:27.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.165+0000 7ff03f12c640 1 --2- 172.21.15.76:0/1667684319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0380d1be0 0x7ff0380b3740 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7ff030009e80 tx=0x7ff03002bf50 comp rx=0 tx=0).stop 2023-11-09T03:52:27.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.165+0000 7ff03f12c640 1 -- 172.21.15.76:0/1667684319 shutdown_connections 2023-11-09T03:52:27.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.165+0000 7ff03f12c640 1 --2- 172.21.15.76:0/1667684319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0380d1be0 0x7ff0380b3740 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:27.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.165+0000 7ff03f12c640 1 --2- 172.21.15.76:0/1667684319 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff03815d100 0x7ff0380ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:27.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.165+0000 7ff03f12c640 1 -- 172.21.15.76:0/1667684319 >> 172.21.15.76:0/1667684319 conn(0x7ff0380d2ef0 msgr2=0x7ff0380d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:27.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.165+0000 7ff03f12c640 1 -- 172.21.15.76:0/1667684319 shutdown_connections 2023-11-09T03:52:27.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.165+0000 7ff03f12c640 1 -- 172.21.15.76:0/1667684319 wait complete. 2023-11-09T03:52:27.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.166+0000 7ff03f12c640 1 Processor -- start 2023-11-09T03:52:27.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.167+0000 7ff03f12c640 1 -- start start 2023-11-09T03:52:27.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.167+0000 7ff03f12c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0380d1be0 0x7ff0380b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:27.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.167+0000 7ff03f12c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff03815d100 0x7ff0380b8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:27.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.167+0000 7ff03f12c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff03815d4e0 0x7ff0381f2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:27.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.167+0000 7ff03f12c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0380b8e60 con 0x7ff03815d4e0 2023-11-09T03:52:27.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.167+0000 7ff03f12c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0380b8c50 con 0x7ff03815d100 2023-11-09T03:52:27.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.167+0000 7ff03f12c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff03815f6f0 con 0x7ff0380d1be0 2023-11-09T03:52:27.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.167+0000 7ff03d6c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff03815d4e0 0x7ff0381f2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:27.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.167+0000 7ff03d6c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff03815d4e0 0x7ff0381f2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42426/0 (socket says 172.21.15.76:42426) 2023-11-09T03:52:27.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.168+0000 7ff03d6c9640 1 -- 172.21.15.76:0/2909261360 learned_addr learned my addr 172.21.15.76:0/2909261360 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:27.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.168+0000 7ff037fff640 1 --2- 172.21.15.76:0/2909261360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff03815d100 0x7ff0380b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:27.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.168+0000 7ff03cec8640 1 --2- 172.21.15.76:0/2909261360 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0380d1be0 0x7ff0380b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:27.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.168+0000 7ff03d6c9640 1 -- 172.21.15.76:0/2909261360 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0380d1be0 msgr2=0x7ff0380b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:27.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.168+0000 7ff03d6c9640 1 --2- 172.21.15.76:0/2909261360 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0380d1be0 0x7ff0380b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:27.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.168+0000 7ff03d6c9640 1 -- 172.21.15.76:0/2909261360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff03815d100 msgr2=0x7ff0380b8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:27.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.168+0000 7ff03d6c9640 1 --2- 172.21.15.76:0/2909261360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff03815d100 0x7ff0380b8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:27.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.168+0000 7ff03d6c9640 1 -- 172.21.15.76:0/2909261360 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff030009b30 con 0x7ff03815d4e0 2023-11-09T03:52:27.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.168+0000 7ff037fff640 1 --2- 172.21.15.76:0/2909261360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff03815d100 0x7ff0380b8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:52:27.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.168+0000 7ff03d6c9640 1 --2- 172.21.15.76:0/2909261360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff03815d4e0 0x7ff0381f2fe0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7ff030036040 tx=0x7ff030004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:27.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.168+0000 7ff0357fa640 1 -- 172.21.15.76:0/2909261360 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030009510 con 0x7ff03815d4e0 2023-11-09T03:52:27.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.168+0000 7ff0357fa640 1 -- 172.21.15.76:0/2909261360 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03003e450 con 0x7ff03815d4e0 2023-11-09T03:52:27.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.168+0000 7ff0357fa640 1 -- 172.21.15.76:0/2909261360 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03003e800 con 0x7ff03815d4e0 2023-11-09T03:52:27.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.168+0000 7ff03f12c640 1 -- 172.21.15.76:0/2909261360 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff03815edc0 con 0x7ff03815d4e0 2023-11-09T03:52:27.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.168+0000 7ff03f12c640 1 -- 172.21.15.76:0/2909261360 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff03815f380 con 0x7ff03815d4e0 2023-11-09T03:52:27.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.169+0000 7ff03f12c640 1 -- 172.21.15.76:0/2909261360 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0380684b0 con 0x7ff03815d4e0 2023-11-09T03:52:27.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.170+0000 7ff0357fa640 1 -- 172.21.15.76:0/2909261360 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff03003e960 con 0x7ff03815d4e0 2023-11-09T03:52:27.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.170+0000 7ff0357fa640 1 --2- 172.21.15.76:0/2909261360 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff01c06ae70 0x7ff01c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:27.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.170+0000 7ff0357fa640 1 -- 172.21.15.76:0/2909261360 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7737+0+0 (secure 0 0 0) 0x7ff0300acbb0 con 0x7ff03815d4e0 2023-11-09T03:52:27.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.170+0000 7ff03cec8640 1 --2- 172.21.15.76:0/2909261360 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff01c06ae70 0x7ff01c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:27.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.171+0000 7ff03cec8640 1 --2- 172.21.15.76:0/2909261360 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff01c06ae70 0x7ff01c06d330 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00b9b0 tx=0x7ff02c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:27.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.174+0000 7ff0357fa640 1 -- 172.21.15.76:0/2909261360 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff030077470 con 0x7ff03815d4e0 2023-11-09T03:52:27.194 INFO:tasks.ceph.mds.c.smithi076.stderr:daemon-helper: command crashed with signal 9 2023-11-09T03:52:27.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.406+0000 7ff03f12c640 1 -- 172.21.15.76:0/2909261360 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:0"} v 0) v1 -- 0x7ff03815f830 con 0x7ff03815d4e0 2023-11-09T03:52:27.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.407+0000 7ff0357fa640 1 -- 172.21.15.76:0/2909261360 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:0"}]=0 failed mds gid 6904 v97) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff030043090 con 0x7ff03815d4e0 2023-11-09T03:52:27.407 INFO:teuthology.orchestra.run.smithi076.stderr:failed mds gid 6904 2023-11-09T03:52:27.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.410+0000 7ff03f12c640 1 -- 172.21.15.76:0/2909261360 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff01c06ae70 msgr2=0x7ff01c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:27.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.410+0000 7ff03f12c640 1 --2- 172.21.15.76:0/2909261360 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff01c06ae70 0x7ff01c06d330 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00b9b0 tx=0x7ff02c008040 comp rx=0 tx=0).stop 2023-11-09T03:52:27.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.410+0000 7ff03f12c640 1 -- 172.21.15.76:0/2909261360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff03815d4e0 msgr2=0x7ff0381f2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:27.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.410+0000 7ff03f12c640 1 --2- 172.21.15.76:0/2909261360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff03815d4e0 0x7ff0381f2fe0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7ff030036040 tx=0x7ff030004a40 comp rx=0 tx=0).stop 2023-11-09T03:52:27.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.410+0000 7ff03f12c640 1 -- 172.21.15.76:0/2909261360 shutdown_connections 2023-11-09T03:52:27.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.410+0000 7ff03f12c640 1 --2- 172.21.15.76:0/2909261360 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff01c06ae70 0x7ff01c06d330 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:27.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.410+0000 7ff03f12c640 1 --2- 172.21.15.76:0/2909261360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff03815d4e0 0x7ff0381f2fe0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:27.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.410+0000 7ff03f12c640 1 --2- 172.21.15.76:0/2909261360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff03815d100 0x7ff0380b8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:27.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.410+0000 7ff03f12c640 1 --2- 172.21.15.76:0/2909261360 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0380d1be0 0x7ff0380b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:27.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.410+0000 7ff03f12c640 1 -- 172.21.15.76:0/2909261360 >> 172.21.15.76:0/2909261360 conn(0x7ff0380d2ef0 msgr2=0x7ff0380b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:27.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.410+0000 7ff03f12c640 1 -- 172.21.15.76:0/2909261360 shutdown_connections 2023-11-09T03:52:27.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.410+0000 7ff03f12c640 1 -- 172.21.15.76:0/2909261360 wait complete. 2023-11-09T03:52:27.421 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-09T03:52:27.422 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-09T03:52:27.422 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-09T03:52:27.423 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:52:27.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:52:27.423 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi113 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b' 2023-11-09T03:52:27.424 INFO:tasks.ceph.mds.b:Stopped 2023-11-09T03:52:27.424 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-09T03:52:27.427 INFO:tasks.ceph.mds.b:Started 2023-11-09T03:52:27.427 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:52:27.507 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2023-11-09T03:52:27.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.527+0000 7f7b36c64640 1 Processor -- start 2023-11-09T03:52:27.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.527+0000 7f7b36c64640 1 -- start start 2023-11-09T03:52:27.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.528+0000 7f7b36c64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b3006e1d0 0x7f7b300ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:27.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.528+0000 7f7b36c64640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b3006e660 con 0x7f7b300ce150 2023-11-09T03:52:27.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.528+0000 7f7b36c64640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b3006d0e0 con 0x7f7b30160630 2023-11-09T03:52:27.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.528+0000 7f7b36c64640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3006d3c0 con 0x7f7b3006e1d0 2023-11-09T03:52:27.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.528+0000 7f7b35201640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f7b300ce150 0x7f7b300ace10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:41656/0 (socket says 172.21.15.76:41656) 2023-11-09T03:52:27.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.528+0000 7f7b35201640 1 -- 172.21.15.76:0/527668469 learned_addr learned my addr 172.21.15.76:0/527668469 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:52:27.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.529+0000 7f7b2ffff640 1 --2- 172.21.15.76:0/527668469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b3006e1d0 0x7f7b300ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:27.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.529+0000 7f7b2f7fe640 1 -- 172.21.15.76:0/527668469 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4222343809 0 0) 0x7f7b3006e660 con 0x7f7b300ce150 2023-11-09T03:52:27.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.529+0000 7f7b2f7fe640 1 -- 172.21.15.76:0/527668469 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b1c003520 con 0x7f7b300ce150 2023-11-09T03:52:27.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.529+0000 7f7b2f7fe640 1 -- 172.21.15.76:0/527668469 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3401598471 0 0) 0x7f7b1c003520 con 0x7f7b300ce150 2023-11-09T03:52:27.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.529+0000 7f7b2f7fe640 1 -- 172.21.15.76:0/527668469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b3006e1d0 msgr2=0x7f7b300ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:27.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.529+0000 7f7b2f7fe640 1 --2- 172.21.15.76:0/527668469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b3006e1d0 0x7f7b300ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:27.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.529+0000 7f7b2f7fe640 1 -- 172.21.15.76:0/527668469 >> v1:172.21.15.113:6789/0 conn(0x7f7b30160630 legacy=0x7f7b300cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:27.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.529+0000 7f7b2f7fe640 1 -- 172.21.15.76:0/527668469 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b3006d6a0 con 0x7f7b300ce150 2023-11-09T03:52:27.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.529+0000 7f7b2f7fe640 1 -- 172.21.15.76:0/527668469 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f7b28001670 con 0x7f7b300ce150 2023-11-09T03:52:27.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.529+0000 7f7b2f7fe640 1 -- 172.21.15.76:0/527668469 >> v1:172.21.15.76:6789/0 conn(0x7f7b300ce150 legacy=0x7f7b300ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:27.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.529+0000 7f7b2f7fe640 1 --2- 172.21.15.76:0/527668469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b1c0039a0 0x7f7b1c003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:27.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.529+0000 7f7b2f7fe640 1 --2- 172.21.15.76:0/527668469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b1c0042c0 0x7f7b1c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:27.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.529+0000 7f7b2f7fe640 1 --2- 172.21.15.76:0/527668469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b1c005010 0x7f7b1c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:27.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.529+0000 7f7b2f7fe640 1 -- 172.21.15.76:0/527668469 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3006d3c0 con 0x7f7b1c005010 2023-11-09T03:52:27.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.529+0000 7f7b2f7fe640 1 -- 172.21.15.76:0/527668469 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c00bef0 con 0x7f7b1c0042c0 2023-11-09T03:52:27.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.529+0000 7f7b2f7fe640 1 -- 172.21.15.76:0/527668469 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c00c0b0 con 0x7f7b1c0039a0 2023-11-09T03:52:27.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.529+0000 7f7b35201640 1 --2- 172.21.15.76:0/527668469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b1c005010 0x7f7b1c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:27.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.529+0000 7f7b34a00640 1 --2- 172.21.15.76:0/527668469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b1c0039a0 0x7f7b1c003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:27.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.529+0000 7f7b2ffff640 1 --2- 172.21.15.76:0/527668469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b1c0042c0 0x7f7b1c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:27.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.529+0000 7f7b35201640 1 -- 172.21.15.76:0/527668469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b1c0039a0 msgr2=0x7f7b1c003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:27.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.529+0000 7f7b35201640 1 --2- 172.21.15.76:0/527668469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b1c0039a0 0x7f7b1c003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:27.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.529+0000 7f7b35201640 1 -- 172.21.15.76:0/527668469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b1c0042c0 msgr2=0x7f7b1c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:27.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.529+0000 7f7b35201640 1 --2- 172.21.15.76:0/527668469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b1c0042c0 0x7f7b1c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:27.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.529+0000 7f7b35201640 1 -- 172.21.15.76:0/527668469 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1c003520 con 0x7f7b1c005010 2023-11-09T03:52:27.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.530+0000 7f7b35201640 1 --2- 172.21.15.76:0/527668469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b1c005010 0x7f7b1c00b980 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800c190 tx=0x7f7b1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b19d356acc4464 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:27.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.530+0000 7f7b2f7fe640 1 -- 172.21.15.76:0/527668469 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28005ad0 con 0x7f7b1c005010 2023-11-09T03:52:27.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.530+0000 7f7b2f7fe640 1 -- 172.21.15.76:0/527668469 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b28005c50 con 0x7f7b1c005010 2023-11-09T03:52:27.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.530+0000 7f7b2f7fe640 1 -- 172.21.15.76:0/527668469 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2802c850 con 0x7f7b1c005010 2023-11-09T03:52:27.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.531+0000 7f7b36c64640 1 -- 172.21.15.76:0/527668469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b1c005010 msgr2=0x7f7b1c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:27.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.531+0000 7f7b36c64640 1 --2- 172.21.15.76:0/527668469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b1c005010 0x7f7b1c00b980 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800c190 tx=0x7f7b1c0057f0 comp rx=0 tx=0).stop 2023-11-09T03:52:27.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.531+0000 7f7b35201640 1 -- 172.21.15.76:0/527668469 reap_dead start 2023-11-09T03:52:27.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.531+0000 7f7b36c64640 1 -- 172.21.15.76:0/527668469 shutdown_connections 2023-11-09T03:52:27.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.531+0000 7f7b36c64640 1 -- 172.21.15.76:0/527668469 >> 172.21.15.76:0/527668469 conn(0x7f7b300cb8e0 msgr2=0x7f7b300cbcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:27.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.531+0000 7f7b36c64640 1 -- 172.21.15.76:0/527668469 shutdown_connections 2023-11-09T03:52:27.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.531+0000 7f7b36c64640 1 -- 172.21.15.76:0/527668469 wait complete. 2023-11-09T03:52:27.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.532+0000 7f7b36c64640 1 Processor -- start 2023-11-09T03:52:27.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.532+0000 7f7b36c64640 1 -- start start 2023-11-09T03:52:27.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.533+0000 7f7b36c64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b1c005010 0x7f7b300cca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:27.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.533+0000 7f7b36c64640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b300b1ae0 0x7f7b300b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:27.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.533+0000 7f7b36c64640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b300b2f30 0x7f7b300b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:27.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.533+0000 7f7b36c64640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b300c6a10 con 0x7f7b300b2f30 2023-11-09T03:52:27.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.533+0000 7f7b36c64640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b300b2530 con 0x7f7b300b1ae0 2023-11-09T03:52:27.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.533+0000 7f7b36c64640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b300b2810 con 0x7f7b1c005010 2023-11-09T03:52:27.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.533+0000 7f7b35201640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b300b2f30 0x7f7b300b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:27.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.533+0000 7f7b35201640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b300b2f30 0x7f7b300b3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42442/0 (socket says 172.21.15.76:42442) 2023-11-09T03:52:27.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.533+0000 7f7b35201640 1 -- 172.21.15.76:0/2093478053 learned_addr learned my addr 172.21.15.76:0/2093478053 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:27.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.533+0000 7f7b34a00640 1 --2- 172.21.15.76:0/2093478053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b1c005010 0x7f7b300cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:27.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.533+0000 7f7b2ffff640 1 --2- 172.21.15.76:0/2093478053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b300b1ae0 0x7f7b300b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:27.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.533+0000 7f7b35201640 1 -- 172.21.15.76:0/2093478053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b1c005010 msgr2=0x7f7b300cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:27.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.533+0000 7f7b35201640 1 --2- 172.21.15.76:0/2093478053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b1c005010 0x7f7b300cca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:27.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.533+0000 7f7b35201640 1 -- 172.21.15.76:0/2093478053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b300b1ae0 msgr2=0x7f7b300b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:27.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.533+0000 7f7b35201640 1 --2- 172.21.15.76:0/2093478053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b300b1ae0 0x7f7b300b1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:27.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.533+0000 7f7b35201640 1 -- 172.21.15.76:0/2093478053 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b300c66a0 con 0x7f7b300b2f30 2023-11-09T03:52:27.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.534+0000 7f7b35201640 1 --2- 172.21.15.76:0/2093478053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b300b2f30 0x7f7b300b3390 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800c190 tx=0x7f7b2802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:27.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.534+0000 7f7b2d7fa640 1 -- 172.21.15.76:0/2093478053 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2803c040 con 0x7f7b300b2f30 2023-11-09T03:52:27.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.534+0000 7f7b2d7fa640 1 -- 172.21.15.76:0/2093478053 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2802cc60 con 0x7f7b300b2f30 2023-11-09T03:52:27.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.534+0000 7f7b36c64640 1 -- 172.21.15.76:0/2093478053 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b300b2af0 con 0x7f7b300b2f30 2023-11-09T03:52:27.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.534+0000 7f7b2d7fa640 1 -- 172.21.15.76:0/2093478053 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28007070 con 0x7f7b300b2f30 2023-11-09T03:52:27.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.534+0000 7f7b36c64640 1 -- 172.21.15.76:0/2093478053 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b300c90e0 con 0x7f7b300b2f30 2023-11-09T03:52:27.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.535+0000 7f7b36c64640 1 -- 172.21.15.76:0/2093478053 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b300cabd0 con 0x7f7b300b2f30 2023-11-09T03:52:27.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.535+0000 7f7b2d7fa640 1 -- 172.21.15.76:0/2093478053 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7b28032070 con 0x7f7b300b2f30 2023-11-09T03:52:27.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.536+0000 7f7b2d7fa640 1 --2- 172.21.15.76:0/2093478053 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7b1006aea0 0x7f7b1006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:27.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.536+0000 7f7b2d7fa640 1 -- 172.21.15.76:0/2093478053 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 7823+0+0 (secure 0 0 0) 0x7f7b280aae30 con 0x7f7b300b2f30 2023-11-09T03:52:27.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.536+0000 7f7b34a00640 1 --2- 172.21.15.76:0/2093478053 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7b1006aea0 0x7f7b1006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:27.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.536+0000 7f7b34a00640 1 --2- 172.21.15.76:0/2093478053 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7b1006aea0 0x7f7b1006d360 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7b20003ea0 tx=0x7f7b20009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:27.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.540+0000 7f7b2d7fa640 1 -- 172.21.15.76:0/2093478053 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7b28075730 con 0x7f7b300b2f30 2023-11-09T03:52:27.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.742+0000 7f7b36c64640 1 -- 172.21.15.76:0/2093478053 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b30073600 con 0x7f7b300b2f30 2023-11-09T03:52:27.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.742+0000 7f7b2d7fa640 1 -- 172.21.15.76:0/2093478053 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4137 (secure 0 0 0) 0x7f7b28078d50 con 0x7f7b300b2f30 2023-11-09T03:52:27.742 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:52:27.742 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":98,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:52:27.410925+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8310,"mds_1":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8274":{"gid":8274,"name":"c","rank":1,"incarnation":89,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6833/1536527256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1536527256},{"type":"v1","addr":"172.21.15.76:6833","nonce":1536527256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8310":{"gid":8310,"name":"a","rank":0,"incarnation":98,"state":"up:replay","state_seq":1,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:52:27.743 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 98 2023-11-09T03:52:27.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.744+0000 7f7b36c64640 1 -- 172.21.15.76:0/2093478053 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7b1006aea0 msgr2=0x7f7b1006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:27.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.744+0000 7f7b36c64640 1 --2- 172.21.15.76:0/2093478053 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7b1006aea0 0x7f7b1006d360 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7b20003ea0 tx=0x7f7b20009040 comp rx=0 tx=0).stop 2023-11-09T03:52:27.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.744+0000 7f7b36c64640 1 -- 172.21.15.76:0/2093478053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b300b2f30 msgr2=0x7f7b300b3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:27.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.744+0000 7f7b36c64640 1 --2- 172.21.15.76:0/2093478053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b300b2f30 0x7f7b300b3390 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800c190 tx=0x7f7b2802e070 comp rx=0 tx=0).stop 2023-11-09T03:52:27.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.744+0000 7f7b36c64640 1 -- 172.21.15.76:0/2093478053 shutdown_connections 2023-11-09T03:52:27.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.744+0000 7f7b36c64640 1 --2- 172.21.15.76:0/2093478053 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7b1006aea0 0x7f7b1006d360 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:27.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.744+0000 7f7b36c64640 1 --2- 172.21.15.76:0/2093478053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b300b2f30 0x7f7b300b3390 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:27.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.744+0000 7f7b36c64640 1 --2- 172.21.15.76:0/2093478053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b300b1ae0 0x7f7b300b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:27.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.744+0000 7f7b36c64640 1 --2- 172.21.15.76:0/2093478053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b1c005010 0x7f7b300cca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:27.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.744+0000 7f7b36c64640 1 -- 172.21.15.76:0/2093478053 >> 172.21.15.76:0/2093478053 conn(0x7f7b300cb8e0 msgr2=0x7f7b3015ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:27.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.744+0000 7f7b36c64640 1 -- 172.21.15.76:0/2093478053 shutdown_connections 2023-11-09T03:52:27.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:27.744+0000 7f7b36c64640 1 -- 172.21.15.76:0/2093478053 wait complete. 2023-11-09T03:52:27.755 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-11-09T03:52:32.041 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-11-09T03:52:32.042 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-11-09T03:52:32.104 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:52:32.107 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:52:32.756 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:52:32.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.865+0000 7fef7f8da640 1 Processor -- start 2023-11-09T03:52:32.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.866+0000 7fef7f8da640 1 -- start start 2023-11-09T03:52:32.870 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.866+0000 7fef7f8da640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fef7815d100 0x7fef780ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:32.870 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.866+0000 7fef7f8da640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fef780d1be0 0x7fef780b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:32.870 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.866+0000 7fef7f8da640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef780b32b0 con 0x7fef7815d4e0 2023-11-09T03:52:32.870 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.866+0000 7fef7f8da640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef780cee10 con 0x7fef780d1be0 2023-11-09T03:52:32.871 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.866+0000 7fef7f8da640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef780b2980 con 0x7fef7815d100 2023-11-09T03:52:32.871 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.866+0000 7fef7de77640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fef780d1be0 0x7fef780b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:32.871 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.866+0000 7fef7de77640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fef780d1be0 0x7fef780b3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59974/0 (socket says 172.21.15.76:59974) 2023-11-09T03:52:32.871 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.867+0000 7fef7de77640 1 -- 172.21.15.76:0/638250175 learned_addr learned my addr 172.21.15.76:0/638250175 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:32.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.867+0000 7fef7d676640 1 --2- 172.21.15.76:0/638250175 >> v2:172.21.15.113:3300/0 conn(0x7fef7815d100 0x7fef780ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:32.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.867+0000 7fef7de77640 1 -- 172.21.15.76:0/638250175 >> v2:172.21.15.113:3300/0 conn(0x7fef7815d100 msgr2=0x7fef780ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:32.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.867+0000 7fef7de77640 1 --2- 172.21.15.76:0/638250175 >> v2:172.21.15.113:3300/0 conn(0x7fef7815d100 0x7fef780ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:32.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.867+0000 7fef7de77640 1 -- 172.21.15.76:0/638250175 >> v1:172.21.15.76:6789/0 conn(0x7fef7815d4e0 legacy=0x7fef780d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:32.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.867+0000 7fef7de77640 1 -- 172.21.15.76:0/638250175 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef780b2c60 con 0x7fef780d1be0 2023-11-09T03:52:32.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.867+0000 7fef7d676640 1 --2- 172.21.15.76:0/638250175 >> v2:172.21.15.113:3300/0 conn(0x7fef7815d100 0x7fef780ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:52:32.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.868+0000 7fef7de77640 1 --2- 172.21.15.76:0/638250175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef780d1be0 0x7fef780b3740 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fef780d09f0 tx=0x7fef7002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16201de5d495aed5 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:32.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.868+0000 7fef6ffff640 1 -- 172.21.15.76:0/638250175 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef70030860 con 0x7fef780d1be0 2023-11-09T03:52:32.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.868+0000 7fef6ffff640 1 -- 172.21.15.76:0/638250175 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef700309c0 con 0x7fef780d1be0 2023-11-09T03:52:32.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.868+0000 7fef6ffff640 1 -- 172.21.15.76:0/638250175 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef70030d50 con 0x7fef780d1be0 2023-11-09T03:52:32.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.869+0000 7fef7f8da640 1 -- 172.21.15.76:0/638250175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef780d1be0 msgr2=0x7fef780b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:32.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.869+0000 7fef7f8da640 1 --2- 172.21.15.76:0/638250175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef780d1be0 0x7fef780b3740 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fef780d09f0 tx=0x7fef7002b9a0 comp rx=0 tx=0).stop 2023-11-09T03:52:32.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.869+0000 7fef7f8da640 1 -- 172.21.15.76:0/638250175 shutdown_connections 2023-11-09T03:52:32.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.869+0000 7fef7f8da640 1 --2- 172.21.15.76:0/638250175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef780d1be0 0x7fef780b3740 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:32.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.869+0000 7fef7f8da640 1 --2- 172.21.15.76:0/638250175 >> v2:172.21.15.113:3300/0 conn(0x7fef7815d100 0x7fef780ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:32.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.869+0000 7fef7f8da640 1 -- 172.21.15.76:0/638250175 >> 172.21.15.76:0/638250175 conn(0x7fef780d2ef0 msgr2=0x7fef780d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:32.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.869+0000 7fef7f8da640 1 -- 172.21.15.76:0/638250175 shutdown_connections 2023-11-09T03:52:32.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.869+0000 7fef7f8da640 1 -- 172.21.15.76:0/638250175 wait complete. 2023-11-09T03:52:32.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.870+0000 7fef7f8da640 1 Processor -- start 2023-11-09T03:52:32.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.870+0000 7fef7f8da640 1 -- start start 2023-11-09T03:52:32.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.871+0000 7fef7f8da640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef780d1be0 0x7fef7814f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:32.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.871+0000 7fef7f8da640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef7815d100 0x7fef780cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:32.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.871+0000 7fef7f8da640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef7815d4e0 0x7fef780cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:32.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.871+0000 7fef7f8da640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef780cada0 con 0x7fef7815d4e0 2023-11-09T03:52:32.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.871+0000 7fef7f8da640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef780cb080 con 0x7fef780d1be0 2023-11-09T03:52:32.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.871+0000 7fef7f8da640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef780cb360 con 0x7fef7815d100 2023-11-09T03:52:32.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.871+0000 7fef7de77640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef7815d4e0 0x7fef780cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:32.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.871+0000 7fef7de77640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef7815d4e0 0x7fef780cc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59978/0 (socket says 172.21.15.76:59978) 2023-11-09T03:52:32.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.871+0000 7fef7de77640 1 -- 172.21.15.76:0/397093740 learned_addr learned my addr 172.21.15.76:0/397093740 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:32.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.871+0000 7fef7d676640 1 --2- 172.21.15.76:0/397093740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef780d1be0 0x7fef7814f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:32.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.871+0000 7fef7ce75640 1 --2- 172.21.15.76:0/397093740 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef7815d100 0x7fef780cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:32.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.871+0000 7fef7de77640 1 -- 172.21.15.76:0/397093740 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef7815d100 msgr2=0x7fef780cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:32.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.871+0000 7fef7de77640 1 --2- 172.21.15.76:0/397093740 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef7815d100 0x7fef780cbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:32.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.871+0000 7fef7de77640 1 -- 172.21.15.76:0/397093740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef780d1be0 msgr2=0x7fef7814f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:32.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.871+0000 7fef7de77640 1 --2- 172.21.15.76:0/397093740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef780d1be0 0x7fef7814f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:32.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.871+0000 7fef7de77640 1 -- 172.21.15.76:0/397093740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef700094d0 con 0x7fef7815d4e0 2023-11-09T03:52:32.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.871+0000 7fef7de77640 1 --2- 172.21.15.76:0/397093740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef7815d4e0 0x7fef780cc060 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fef700090b0 tx=0x7fef70004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:32.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.871+0000 7fef6dffb640 1 -- 172.21.15.76:0/397093740 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef70039a90 con 0x7fef7815d4e0 2023-11-09T03:52:32.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.871+0000 7fef6dffb640 1 -- 172.21.15.76:0/397093740 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef70039bf0 con 0x7fef7815d4e0 2023-11-09T03:52:32.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.871+0000 7fef7f8da640 1 -- 172.21.15.76:0/397093740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef780b5a70 con 0x7fef7815d4e0 2023-11-09T03:52:32.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.871+0000 7fef6dffb640 1 -- 172.21.15.76:0/397093740 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef70043410 con 0x7fef7815d4e0 2023-11-09T03:52:32.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.872+0000 7fef7f8da640 1 -- 172.21.15.76:0/397093740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7814f820 con 0x7fef7815d4e0 2023-11-09T03:52:32.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.873+0000 7fef7f8da640 1 -- 172.21.15.76:0/397093740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef780b5f90 con 0x7fef7815d4e0 2023-11-09T03:52:32.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.873+0000 7fef6dffb640 1 -- 172.21.15.76:0/397093740 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fef7004c3b0 con 0x7fef7815d4e0 2023-11-09T03:52:32.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.873+0000 7fef6dffb640 1 --2- 172.21.15.76:0/397093740 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fef5c06ae70 0x7fef5c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:32.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.873+0000 7fef6dffb640 1 -- 172.21.15.76:0/397093740 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 7823+0+0 (secure 0 0 0) 0x7fef700ac360 con 0x7fef7815d4e0 2023-11-09T03:52:32.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.874+0000 7fef7d676640 1 --2- 172.21.15.76:0/397093740 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fef5c06ae70 0x7fef5c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:32.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.874+0000 7fef7d676640 1 --2- 172.21.15.76:0/397093740 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fef5c06ae70 0x7fef5c06d330 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fef74003fd0 tx=0x7fef740039f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:32.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:32.878+0000 7fef6dffb640 1 -- 172.21.15.76:0/397093740 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fef780b5f90 con 0x7fef7815d4e0 2023-11-09T03:52:33.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.079+0000 7fef7f8da640 1 -- 172.21.15.76:0/397093740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef780b5f90 con 0x7fef7815d4e0 2023-11-09T03:52:33.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.080+0000 7fef6dffb640 1 -- 172.21.15.76:0/397093740 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4924 (secure 0 0 0) 0x7fef780b5f90 con 0x7fef7815d4e0 2023-11-09T03:52:33.081 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:52:33.082 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":99,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":7144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/7360844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":7360844},{"type":"v1","addr":"172.21.15.113:6837","nonce":7360844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:52:27.427128+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8310,"mds_1":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8274":{"gid":8274,"name":"c","rank":1,"incarnation":89,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6833/1536527256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1536527256},{"type":"v1","addr":"172.21.15.76:6833","nonce":1536527256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8310":{"gid":8310,"name":"a","rank":0,"incarnation":98,"state":"up:resolve","state_seq":13,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:52:33.083 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 99 2023-11-09T03:52:33.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.082+0000 7fef7f8da640 1 -- 172.21.15.76:0/397093740 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fef5c06ae70 msgr2=0x7fef5c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:33.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.082+0000 7fef7f8da640 1 --2- 172.21.15.76:0/397093740 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fef5c06ae70 0x7fef5c06d330 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fef74003fd0 tx=0x7fef740039f0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.082+0000 7fef7f8da640 1 -- 172.21.15.76:0/397093740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef7815d4e0 msgr2=0x7fef780cc060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:33.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.082+0000 7fef7f8da640 1 --2- 172.21.15.76:0/397093740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef7815d4e0 0x7fef780cc060 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fef700090b0 tx=0x7fef70004730 comp rx=0 tx=0).stop 2023-11-09T03:52:33.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.082+0000 7fef7f8da640 1 -- 172.21.15.76:0/397093740 shutdown_connections 2023-11-09T03:52:33.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.082+0000 7fef7f8da640 1 --2- 172.21.15.76:0/397093740 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fef5c06ae70 0x7fef5c06d330 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.082+0000 7fef7f8da640 1 --2- 172.21.15.76:0/397093740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef7815d4e0 0x7fef780cc060 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.082+0000 7fef7f8da640 1 --2- 172.21.15.76:0/397093740 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef7815d100 0x7fef780cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.082+0000 7fef7f8da640 1 --2- 172.21.15.76:0/397093740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef780d1be0 0x7fef7814f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.082+0000 7fef7f8da640 1 -- 172.21.15.76:0/397093740 >> 172.21.15.76:0/397093740 conn(0x7fef780d2ef0 msgr2=0x7fef780b4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:33.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.082+0000 7fef7f8da640 1 -- 172.21.15.76:0/397093740 shutdown_connections 2023-11-09T03:52:33.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.082+0000 7fef7f8da640 1 -- 172.21.15.76:0/397093740 wait complete. 2023-11-09T03:52:33.094 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-09T03:52:33.094 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:52:33.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.194+0000 7f44896a2640 1 Processor -- start 2023-11-09T03:52:33.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.194+0000 7f44896a2640 1 -- start start 2023-11-09T03:52:33.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.195+0000 7f44896a2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f448415d100 0x7f44840ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:33.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.195+0000 7f44896a2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f448415d4e0 0x7f44840d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:33.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.195+0000 7f44896a2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f44840d3b60 0x7f44840d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:33.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.195+0000 7f44896a2640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44840b32b0 con 0x7f44840d3b60 2023-11-09T03:52:33.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.195+0000 7f44896a2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44840cee10 con 0x7f448415d100 2023-11-09T03:52:33.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.195+0000 7f44896a2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44840b2980 con 0x7f448415d4e0 2023-11-09T03:52:33.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.195+0000 7f44837fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f44840d3b60 0x7f44840d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:33.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.195+0000 7f44837fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f44840d3b60 0x7f44840d1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59986/0 (socket says 172.21.15.76:59986) 2023-11-09T03:52:33.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.195+0000 7f44837fe640 1 -- 172.21.15.76:0/4287595514 learned_addr learned my addr 172.21.15.76:0/4287595514 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:33.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.196+0000 7f4482ffd640 1 --2- 172.21.15.76:0/4287595514 >> v2:172.21.15.113:3300/0 conn(0x7f448415d100 0x7f44840ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:33.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.196+0000 7f44827fc640 1 --2- 172.21.15.76:0/4287595514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f448415d4e0 0x7f44840d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:33.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.196+0000 7f44837fe640 1 -- 172.21.15.76:0/4287595514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f448415d4e0 msgr2=0x7f44840d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:33.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.196+0000 7f44837fe640 1 --2- 172.21.15.76:0/4287595514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f448415d4e0 0x7f44840d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.196+0000 7f44837fe640 1 -- 172.21.15.76:0/4287595514 >> v2:172.21.15.113:3300/0 conn(0x7f448415d100 msgr2=0x7f44840ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:33.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.196+0000 7f44837fe640 1 --2- 172.21.15.76:0/4287595514 >> v2:172.21.15.113:3300/0 conn(0x7f448415d100 0x7f44840ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.196+0000 7f44837fe640 1 -- 172.21.15.76:0/4287595514 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44840b2c60 con 0x7f44840d3b60 2023-11-09T03:52:33.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.196+0000 7f44837fe640 1 --2- 172.21.15.76:0/4287595514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44840d3b60 0x7f44840d1a10 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f44840d09f0 tx=0x7f447c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f67bd664a0415d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:33.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.197+0000 7f4481ffb640 1 -- 172.21.15.76:0/4287595514 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c031870 con 0x7f44840d3b60 2023-11-09T03:52:33.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.197+0000 7f4481ffb640 1 -- 172.21.15.76:0/4287595514 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447c0319d0 con 0x7f44840d3b60 2023-11-09T03:52:33.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.197+0000 7f4481ffb640 1 -- 172.21.15.76:0/4287595514 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c031d60 con 0x7f44840d3b60 2023-11-09T03:52:33.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.197+0000 7f44896a2640 1 -- 172.21.15.76:0/4287595514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44840d3b60 msgr2=0x7f44840d1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:33.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.197+0000 7f44896a2640 1 --2- 172.21.15.76:0/4287595514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44840d3b60 0x7f44840d1a10 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f44840d09f0 tx=0x7f447c02b910 comp rx=0 tx=0).stop 2023-11-09T03:52:33.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.197+0000 7f44896a2640 1 -- 172.21.15.76:0/4287595514 shutdown_connections 2023-11-09T03:52:33.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.197+0000 7f44896a2640 1 --2- 172.21.15.76:0/4287595514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44840d3b60 0x7f44840d1a10 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.197+0000 7f44896a2640 1 --2- 172.21.15.76:0/4287595514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f448415d4e0 0x7f44840d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.197+0000 7f44896a2640 1 --2- 172.21.15.76:0/4287595514 >> v2:172.21.15.113:3300/0 conn(0x7f448415d100 0x7f44840ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.197+0000 7f44896a2640 1 -- 172.21.15.76:0/4287595514 >> 172.21.15.76:0/4287595514 conn(0x7f44840d2ef0 msgr2=0x7f44840d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:33.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.197+0000 7f44896a2640 1 -- 172.21.15.76:0/4287595514 shutdown_connections 2023-11-09T03:52:33.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.197+0000 7f44896a2640 1 -- 172.21.15.76:0/4287595514 wait complete. 2023-11-09T03:52:33.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.199+0000 7f44896a2640 1 Processor -- start 2023-11-09T03:52:33.208 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.199+0000 7f44896a2640 1 -- start start 2023-11-09T03:52:33.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.199+0000 7f44896a2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44840d3b60 0x7f448409e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:33.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.199+0000 7f44896a2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f448415d100 0x7f448414e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:33.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.199+0000 7f44896a2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f448415d4e0 0x7f448409f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:33.209 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.199+0000 7f44896a2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448409e5d0 con 0x7f44840d3b60 2023-11-09T03:52:33.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.199+0000 7f44896a2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448409ef30 con 0x7f448415d100 2023-11-09T03:52:33.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.199+0000 7f44896a2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f448414e880 con 0x7f448415d4e0 2023-11-09T03:52:33.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.199+0000 7f4482ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44840d3b60 0x7f448409e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:33.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.199+0000 7f4482ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44840d3b60 0x7f448409e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59998/0 (socket says 172.21.15.76:59998) 2023-11-09T03:52:33.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.199+0000 7f4482ffd640 1 -- 172.21.15.76:0/3812916947 learned_addr learned my addr 172.21.15.76:0/3812916947 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:33.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.200+0000 7f4482ffd640 1 -- 172.21.15.76:0/3812916947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f448415d4e0 msgr2=0x7f448409f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:33.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.200+0000 7f4482ffd640 1 --2- 172.21.15.76:0/3812916947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f448415d4e0 0x7f448409f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.200+0000 7f4482ffd640 1 -- 172.21.15.76:0/3812916947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f448415d100 msgr2=0x7f448414e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:33.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.200+0000 7f4482ffd640 1 --2- 172.21.15.76:0/3812916947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f448415d100 0x7f448414e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.200+0000 7f4482ffd640 1 -- 172.21.15.76:0/3812916947 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c0094d0 con 0x7f44840d3b60 2023-11-09T03:52:33.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.200+0000 7f4482ffd640 1 --2- 172.21.15.76:0/3812916947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44840d3b60 0x7f448409e9f0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f447400ef00 tx=0x7f447400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:33.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.200+0000 7f446b7fe640 1 -- 172.21.15.76:0/3812916947 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4474010040 con 0x7f44840d3b60 2023-11-09T03:52:33.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.200+0000 7f446b7fe640 1 -- 172.21.15.76:0/3812916947 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4474014400 con 0x7f44840d3b60 2023-11-09T03:52:33.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.200+0000 7f446b7fe640 1 -- 172.21.15.76:0/3812916947 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4474014790 con 0x7f44840d3b60 2023-11-09T03:52:33.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.200+0000 7f44896a2640 1 -- 172.21.15.76:0/3812916947 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f448409d950 con 0x7f44840d3b60 2023-11-09T03:52:33.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.200+0000 7f44896a2640 1 -- 172.21.15.76:0/3812916947 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f448409df10 con 0x7f44840d3b60 2023-11-09T03:52:33.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.201+0000 7f44896a2640 1 -- 172.21.15.76:0/3812916947 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4484067f00 con 0x7f44840d3b60 2023-11-09T03:52:33.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.201+0000 7f446b7fe640 1 -- 172.21.15.76:0/3812916947 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f44740148f0 con 0x7f44840d3b60 2023-11-09T03:52:33.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.202+0000 7f446b7fe640 1 --2- 172.21.15.76:0/3812916947 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f447006ae70 0x7f447006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:33.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.202+0000 7f446b7fe640 1 -- 172.21.15.76:0/3812916947 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 7823+0+0 (secure 0 0 0) 0x7f447408aa20 con 0x7f44840d3b60 2023-11-09T03:52:33.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.202+0000 7f44827fc640 1 --2- 172.21.15.76:0/3812916947 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f447006ae70 0x7f447006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:33.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.202+0000 7f44827fc640 1 --2- 172.21.15.76:0/3812916947 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f447006ae70 0x7f447006d330 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4478006e80 tx=0x7f4478006dd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:33.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.206+0000 7f446b7fe640 1 -- 172.21.15.76:0/3812916947 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4474055290 con 0x7f44840d3b60 2023-11-09T03:52:33.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.407+0000 7f44896a2640 1 -- 172.21.15.76:0/3812916947 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44840684b0 con 0x7f44840d3b60 2023-11-09T03:52:33.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.407+0000 7f446b7fe640 1 -- 172.21.15.76:0/3812916947 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f4474009c70 con 0x7f44840d3b60 2023-11-09T03:52:33.409 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:52:33.409 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":99,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":7144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/7360844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":7360844},{"type":"v1","addr":"172.21.15.113:6837","nonce":7360844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:52:27.427128+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8310,"mds_1":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8274":{"gid":8274,"name":"c","rank":1,"incarnation":89,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6833/1536527256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1536527256},{"type":"v1","addr":"172.21.15.76:6833","nonce":1536527256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8310":{"gid":8310,"name":"a","rank":0,"incarnation":98,"state":"up:resolve","state_seq":13,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:52:33.409 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 99 2023-11-09T03:52:33.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.409+0000 7f44896a2640 1 -- 172.21.15.76:0/3812916947 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f447006ae70 msgr2=0x7f447006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:33.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.409+0000 7f44896a2640 1 --2- 172.21.15.76:0/3812916947 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f447006ae70 0x7f447006d330 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4478006e80 tx=0x7f4478006dd0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.409+0000 7f44896a2640 1 -- 172.21.15.76:0/3812916947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44840d3b60 msgr2=0x7f448409e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:33.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.409+0000 7f44896a2640 1 --2- 172.21.15.76:0/3812916947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44840d3b60 0x7f448409e9f0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f447400ef00 tx=0x7f447400b480 comp rx=0 tx=0).stop 2023-11-09T03:52:33.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.409+0000 7f44896a2640 1 -- 172.21.15.76:0/3812916947 shutdown_connections 2023-11-09T03:52:33.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.409+0000 7f44896a2640 1 --2- 172.21.15.76:0/3812916947 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f447006ae70 0x7f447006d330 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.409+0000 7f44896a2640 1 --2- 172.21.15.76:0/3812916947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f448415d4e0 0x7f448409f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.410+0000 7f44896a2640 1 --2- 172.21.15.76:0/3812916947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f448415d100 0x7f448414e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.410+0000 7f44896a2640 1 --2- 172.21.15.76:0/3812916947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44840d3b60 0x7f448409e9f0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.410+0000 7f44896a2640 1 -- 172.21.15.76:0/3812916947 >> 172.21.15.76:0/3812916947 conn(0x7f44840d2ef0 msgr2=0x7f448415af50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:33.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.410+0000 7f44896a2640 1 -- 172.21.15.76:0/3812916947 shutdown_connections 2023-11-09T03:52:33.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.410+0000 7f44896a2640 1 -- 172.21.15.76:0/3812916947 wait complete. 2023-11-09T03:52:33.421 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:resolve 2023-11-09T03:52:33.422 DEBUG:tasks.cephfs.filesystem:reached state 'up:resolve' in 0.32750821113586426s 2023-11-09T03:52:33.422 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:52:33.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.523+0000 7f461b725640 1 Processor -- start 2023-11-09T03:52:33.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.524+0000 7f461b725640 1 -- start start 2023-11-09T03:52:33.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.524+0000 7f461b725640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4614159e40 0x7f46140b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:33.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.524+0000 7f461b725640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f461406bb60 0x7f46140b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:33.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.524+0000 7f461b725640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4614071500 con 0x7f461406fff0 2023-11-09T03:52:33.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.524+0000 7f461b725640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f461415dae0 con 0x7f4614159e40 2023-11-09T03:52:33.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.524+0000 7f461b725640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4614070c60 con 0x7f461406bb60 2023-11-09T03:52:33.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.524+0000 7f46194c1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4614159e40 0x7f46140b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:33.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.524+0000 7f46194c1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4614159e40 0x7f46140b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60006/0 (socket says 172.21.15.76:60006) 2023-11-09T03:52:33.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.525+0000 7f46194c1640 1 -- 172.21.15.76:0/2564684608 learned_addr learned my addr 172.21.15.76:0/2564684608 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:33.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.525+0000 7f4619cc2640 1 --1- 172.21.15.76:0/2564684608 >> v1:172.21.15.76:6789/0 conn(0x7f461406fff0 0x7f4614070420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52664/0 (socket says 172.21.15.76:52664) 2023-11-09T03:52:33.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.525+0000 7f4618cc0640 1 --2- 172.21.15.76:0/2564684608 >> v2:172.21.15.113:3300/0 conn(0x7f461406bb60 0x7f46140b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:33.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.525+0000 7f46194c1640 1 -- 172.21.15.76:0/2564684608 >> v2:172.21.15.113:3300/0 conn(0x7f461406bb60 msgr2=0x7f46140b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:33.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.525+0000 7f46194c1640 1 --2- 172.21.15.76:0/2564684608 >> v2:172.21.15.113:3300/0 conn(0x7f461406bb60 0x7f46140b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.525+0000 7f46194c1640 1 -- 172.21.15.76:0/2564684608 >> v1:172.21.15.76:6789/0 conn(0x7f461406fff0 legacy=0x7f4614070420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:33.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.525+0000 7f46194c1640 1 -- 172.21.15.76:0/2564684608 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4614070f40 con 0x7f4614159e40 2023-11-09T03:52:33.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.525+0000 7f46194c1640 1 --2- 172.21.15.76:0/2564684608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4614159e40 0x7f46140b0530 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f461406fdc0 tx=0x7f461002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47feb61a270fd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:33.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.526+0000 7f460bfff640 1 -- 172.21.15.76:0/2564684608 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461002c9f0 con 0x7f4614159e40 2023-11-09T03:52:33.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.526+0000 7f460bfff640 1 -- 172.21.15.76:0/2564684608 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461002cb50 con 0x7f4614159e40 2023-11-09T03:52:33.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.526+0000 7f460bfff640 1 -- 172.21.15.76:0/2564684608 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46100343f0 con 0x7f4614159e40 2023-11-09T03:52:33.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.526+0000 7f461b725640 1 -- 172.21.15.76:0/2564684608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4614159e40 msgr2=0x7f46140b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:33.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.526+0000 7f461b725640 1 --2- 172.21.15.76:0/2564684608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4614159e40 0x7f46140b0530 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f461406fdc0 tx=0x7f461002be50 comp rx=0 tx=0).stop 2023-11-09T03:52:33.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.526+0000 7f461b725640 1 -- 172.21.15.76:0/2564684608 shutdown_connections 2023-11-09T03:52:33.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.526+0000 7f461b725640 1 --2- 172.21.15.76:0/2564684608 >> v2:172.21.15.113:3300/0 conn(0x7f461406bb60 0x7f46140b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.526+0000 7f461b725640 1 --2- 172.21.15.76:0/2564684608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4614159e40 0x7f46140b0530 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.526+0000 7f461b725640 1 -- 172.21.15.76:0/2564684608 >> 172.21.15.76:0/2564684608 conn(0x7f46140cb530 msgr2=0x7f46140cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:33.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.526+0000 7f461b725640 1 -- 172.21.15.76:0/2564684608 shutdown_connections 2023-11-09T03:52:33.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.526+0000 7f461b725640 1 -- 172.21.15.76:0/2564684608 wait complete. 2023-11-09T03:52:33.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.527+0000 7f461b725640 1 Processor -- start 2023-11-09T03:52:33.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.527+0000 7f461b725640 1 -- start start 2023-11-09T03:52:33.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.528+0000 7f461b725640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f461406bb60 0x7f46141583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:33.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.528+0000 7f461b725640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f461406fff0 0x7f46140c7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:33.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.528+0000 7f461b725640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4614159e40 0x7f46140c4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:33.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.528+0000 7f461b725640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461415bc20 con 0x7f4614159e40 2023-11-09T03:52:33.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.528+0000 7f461b725640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461415b010 con 0x7f461406bb60 2023-11-09T03:52:33.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.528+0000 7f461b725640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461415b2f0 con 0x7f461406fff0 2023-11-09T03:52:33.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.528+0000 7f4619cc2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4614159e40 0x7f46140c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:33.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.528+0000 7f4619cc2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4614159e40 0x7f46140c4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60008/0 (socket says 172.21.15.76:60008) 2023-11-09T03:52:33.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.528+0000 7f4619cc2640 1 -- 172.21.15.76:0/622054133 learned_addr learned my addr 172.21.15.76:0/622054133 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:33.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.528+0000 7f4618cc0640 1 --2- 172.21.15.76:0/622054133 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f461406fff0 0x7f46140c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:33.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.528+0000 7f46194c1640 1 --2- 172.21.15.76:0/622054133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f461406bb60 0x7f46141583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:33.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.529+0000 7f4619cc2640 1 -- 172.21.15.76:0/622054133 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f461406fff0 msgr2=0x7f46140c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:33.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.529+0000 7f4619cc2640 1 --2- 172.21.15.76:0/622054133 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f461406fff0 0x7f46140c7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.529+0000 7f4619cc2640 1 -- 172.21.15.76:0/622054133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f461406bb60 msgr2=0x7f46141583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:33.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.529+0000 7f4619cc2640 1 --2- 172.21.15.76:0/622054133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f461406bb60 0x7f46141583e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.529+0000 7f4619cc2640 1 -- 172.21.15.76:0/622054133 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4610009b30 con 0x7f4614159e40 2023-11-09T03:52:33.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.529+0000 7f46194c1640 1 --2- 172.21.15.76:0/622054133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f461406bb60 0x7f46141583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:52:33.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.529+0000 7f4618cc0640 1 --2- 172.21.15.76:0/622054133 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f461406fff0 0x7f46140c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:52:33.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.529+0000 7f4619cc2640 1 --2- 172.21.15.76:0/622054133 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4614159e40 0x7f46140c4790 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f460c00c400 tx=0x7f460c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:33.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.529+0000 7f4609ffb640 1 -- 172.21.15.76:0/622054133 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c0074e0 con 0x7f4614159e40 2023-11-09T03:52:33.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.529+0000 7f4609ffb640 1 -- 172.21.15.76:0/622054133 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460c007640 con 0x7f4614159e40 2023-11-09T03:52:33.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.529+0000 7f461b725640 1 -- 172.21.15.76:0/622054133 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f461415b5d0 con 0x7f4614159e40 2023-11-09T03:52:33.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.529+0000 7f4609ffb640 1 -- 172.21.15.76:0/622054133 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c0049e0 con 0x7f4614159e40 2023-11-09T03:52:33.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.529+0000 7f461b725640 1 -- 172.21.15.76:0/622054133 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4614158920 con 0x7f4614159e40 2023-11-09T03:52:33.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.530+0000 7f461b725640 1 -- 172.21.15.76:0/622054133 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f461415bdd0 con 0x7f4614159e40 2023-11-09T03:52:33.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.530+0000 7f4609ffb640 1 -- 172.21.15.76:0/622054133 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f460c004b40 con 0x7f4614159e40 2023-11-09T03:52:33.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.531+0000 7f4609ffb640 1 --2- 172.21.15.76:0/622054133 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f45f806ae70 0x7f45f806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:33.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.531+0000 7f4609ffb640 1 -- 172.21.15.76:0/622054133 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 7823+0+0 (secure 0 0 0) 0x7f460c0897e0 con 0x7f4614159e40 2023-11-09T03:52:33.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.531+0000 7f46194c1640 1 --2- 172.21.15.76:0/622054133 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f45f806ae70 0x7f45f806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:33.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.531+0000 7f46194c1640 1 --2- 172.21.15.76:0/622054133 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f45f806ae70 0x7f45f806d330 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4610035000 tx=0x7f4610030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:33.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.535+0000 7f4609ffb640 1 -- 172.21.15.76:0/622054133 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f461415bdd0 con 0x7f4614159e40 2023-11-09T03:52:33.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.737+0000 7f461b725640 1 -- 172.21.15.76:0/622054133 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46140711f0 con 0x7f4614159e40 2023-11-09T03:52:33.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.737+0000 7f4609ffb640 1 -- 172.21.15.76:0/622054133 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f460c0540e0 con 0x7f4614159e40 2023-11-09T03:52:33.739 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:52:33.739 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":99,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":7144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/7360844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":7360844},{"type":"v1","addr":"172.21.15.113:6837","nonce":7360844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:52:27.427128+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8310,"mds_1":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8274":{"gid":8274,"name":"c","rank":1,"incarnation":89,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6833/1536527256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1536527256},{"type":"v1","addr":"172.21.15.76:6833","nonce":1536527256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8310":{"gid":8310,"name":"a","rank":0,"incarnation":98,"state":"up:resolve","state_seq":13,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:52:33.739 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 99 2023-11-09T03:52:33.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.740+0000 7f461b725640 1 -- 172.21.15.76:0/622054133 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f45f806ae70 msgr2=0x7f45f806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:33.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.740+0000 7f461b725640 1 --2- 172.21.15.76:0/622054133 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f45f806ae70 0x7f45f806d330 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4610035000 tx=0x7f4610030000 comp rx=0 tx=0).stop 2023-11-09T03:52:33.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.740+0000 7f461b725640 1 -- 172.21.15.76:0/622054133 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4614159e40 msgr2=0x7f46140c4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:33.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.740+0000 7f461b725640 1 --2- 172.21.15.76:0/622054133 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4614159e40 0x7f46140c4790 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f460c00c400 tx=0x7f460c00c8c0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.740+0000 7f461b725640 1 -- 172.21.15.76:0/622054133 shutdown_connections 2023-11-09T03:52:33.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.740+0000 7f461b725640 1 --2- 172.21.15.76:0/622054133 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f45f806ae70 0x7f45f806d330 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.740+0000 7f461b725640 1 --2- 172.21.15.76:0/622054133 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4614159e40 0x7f46140c4790 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.740+0000 7f461b725640 1 --2- 172.21.15.76:0/622054133 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f461406fff0 0x7f46140c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.740+0000 7f461b725640 1 --2- 172.21.15.76:0/622054133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f461406bb60 0x7f46141583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.740+0000 7f461b725640 1 -- 172.21.15.76:0/622054133 >> 172.21.15.76:0/622054133 conn(0x7f46140cb530 msgr2=0x7f461406d280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:33.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.740+0000 7f461b725640 1 -- 172.21.15.76:0/622054133 shutdown_connections 2023-11-09T03:52:33.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.740+0000 7f461b725640 1 -- 172.21.15.76:0/622054133 wait complete. 2023-11-09T03:52:33.751 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump snaps --server 2023-11-09T03:52:33.865 INFO:teuthology.orchestra.run.smithi076.stdout:{"last_snap":11,"last_created":5,"last_destroyed":9,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[{"version":22,"snapinfo":{"snapid":11,"ino":1099511627777,"stamp":"2023-11-09T03:52:04.689262+0000","name":"s25","metadata":{}}}],"pending_destroy":[]} 2023-11-09T03:52:33.866 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 5, "last_destroyed": 9, "last_snap": 11, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [ { "snapInfinityo": { "ino": 1099511627777, "metadata": {}, "name": "s25", "snapid": 11, "stamp": "2023-11-09T03:52:04.689262+0000" }, "version": 22 } ], "snaps": [] } 2023-11-09T03:52:33.866 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:1 2023-11-09T03:52:33.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.966+0000 7fc94f561640 1 Processor -- start 2023-11-09T03:52:33.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.966+0000 7fc94f561640 1 -- start start 2023-11-09T03:52:33.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.967+0000 7fc94f561640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc94815d100 0x7fc9480ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:33.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.967+0000 7fc94f561640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9480d1be0 0x7fc9480b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:33.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.967+0000 7fc94f561640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9480b32b0 con 0x7fc94815d4e0 2023-11-09T03:52:33.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.967+0000 7fc94f561640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9480cee10 con 0x7fc94815d100 2023-11-09T03:52:33.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.967+0000 7fc94f561640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9480b2980 con 0x7fc9480d1be0 2023-11-09T03:52:33.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.968+0000 7fc94d2fd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc94815d100 0x7fc9480ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:33.970 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.968+0000 7fc94dafe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9480d1be0 0x7fc9480b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:33.970 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.968+0000 7fc94cafc640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc94815d4e0 0x7fc9480d14d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:58594/0 (socket says 172.21.15.76:58594) 2023-11-09T03:52:33.970 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.968+0000 7fc94cafc640 1 -- 172.21.15.76:0/1639588729 learned_addr learned my addr 172.21.15.76:0/1639588729 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:52:33.970 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.968+0000 7fc94dafe640 1 -- 172.21.15.76:0/1639588729 >> v2:172.21.15.113:3300/0 conn(0x7fc94815d100 msgr2=0x7fc9480ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:33.970 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.968+0000 7fc94dafe640 1 --2- 172.21.15.76:0/1639588729 >> v2:172.21.15.113:3300/0 conn(0x7fc94815d100 0x7fc9480ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.971 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.968+0000 7fc94dafe640 1 -- 172.21.15.76:0/1639588729 >> v1:172.21.15.113:6789/0 conn(0x7fc94815d4e0 legacy=0x7fc9480d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:33.971 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.968+0000 7fc94dafe640 1 -- 172.21.15.76:0/1639588729 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9480b2c60 con 0x7fc9480d1be0 2023-11-09T03:52:33.971 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.968+0000 7fc94d2fd640 1 --2- 172.21.15.76:0/1639588729 >> v2:172.21.15.113:3300/0 conn(0x7fc94815d100 0x7fc9480ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:52:33.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.969+0000 7fc94dafe640 1 --2- 172.21.15.76:0/1639588729 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9480d1be0 0x7fc9480b3740 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc9400097a0 tx=0x7fc94002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9985ee69e9e6f57 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:33.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.969+0000 7fc93ffff640 1 -- 172.21.15.76:0/1639588729 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc940031830 con 0x7fc9480d1be0 2023-11-09T03:52:33.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.969+0000 7fc93ffff640 1 -- 172.21.15.76:0/1639588729 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc940031990 con 0x7fc9480d1be0 2023-11-09T03:52:33.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.969+0000 7fc93ffff640 1 -- 172.21.15.76:0/1639588729 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc940031d20 con 0x7fc9480d1be0 2023-11-09T03:52:33.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.970+0000 7fc94f561640 1 -- 172.21.15.76:0/1639588729 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9480d1be0 msgr2=0x7fc9480b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:33.975 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.970+0000 7fc94f561640 1 --2- 172.21.15.76:0/1639588729 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9480d1be0 0x7fc9480b3740 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc9400097a0 tx=0x7fc94002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.970+0000 7fc94f561640 1 -- 172.21.15.76:0/1639588729 shutdown_connections 2023-11-09T03:52:33.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.970+0000 7fc94f561640 1 --2- 172.21.15.76:0/1639588729 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9480d1be0 0x7fc9480b3740 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.970+0000 7fc94f561640 1 --2- 172.21.15.76:0/1639588729 >> v2:172.21.15.113:3300/0 conn(0x7fc94815d100 0x7fc9480ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.970+0000 7fc94f561640 1 -- 172.21.15.76:0/1639588729 >> 172.21.15.76:0/1639588729 conn(0x7fc9480d2ef0 msgr2=0x7fc9480d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:33.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.970+0000 7fc94f561640 1 -- 172.21.15.76:0/1639588729 shutdown_connections 2023-11-09T03:52:33.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.970+0000 7fc94f561640 1 -- 172.21.15.76:0/1639588729 wait complete. 2023-11-09T03:52:33.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.971+0000 7fc94f561640 1 Processor -- start 2023-11-09T03:52:33.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.971+0000 7fc94f561640 1 -- start start 2023-11-09T03:52:33.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.971+0000 7fc94f561640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9480d1be0 0x7fc94815a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:33.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.971+0000 7fc94f561640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc94815d100 0x7fc9480b5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:33.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.971+0000 7fc94f561640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc94815d4e0 0x7fc9480b6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:33.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.971+0000 7fc94f561640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9480cb010 con 0x7fc94815d100 2023-11-09T03:52:33.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.971+0000 7fc94f561640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9480cb2d0 con 0x7fc9480d1be0 2023-11-09T03:52:33.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.971+0000 7fc94f561640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9480cb5b0 con 0x7fc94815d4e0 2023-11-09T03:52:33.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.972+0000 7fc94cafc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc94815d100 0x7fc9480b5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:33.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.972+0000 7fc94cafc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc94815d100 0x7fc9480b5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60022/0 (socket says 172.21.15.76:60022) 2023-11-09T03:52:33.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.972+0000 7fc94cafc640 1 -- 172.21.15.76:0/2287850512 learned_addr learned my addr 172.21.15.76:0/2287850512 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:33.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.972+0000 7fc94d2fd640 1 --2- 172.21.15.76:0/2287850512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9480d1be0 0x7fc94815a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:33.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.972+0000 7fc94dafe640 1 --2- 172.21.15.76:0/2287850512 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc94815d4e0 0x7fc9480b6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:33.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.972+0000 7fc94cafc640 1 -- 172.21.15.76:0/2287850512 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc94815d4e0 msgr2=0x7fc9480b6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:33.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.972+0000 7fc94cafc640 1 --2- 172.21.15.76:0/2287850512 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc94815d4e0 0x7fc9480b6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.972+0000 7fc94cafc640 1 -- 172.21.15.76:0/2287850512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9480d1be0 msgr2=0x7fc94815a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:33.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.972+0000 7fc94cafc640 1 --2- 172.21.15.76:0/2287850512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9480d1be0 0x7fc94815a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:33.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.972+0000 7fc94cafc640 1 -- 172.21.15.76:0/2287850512 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc944009a70 con 0x7fc94815d100 2023-11-09T03:52:33.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.972+0000 7fc94cafc640 1 --2- 172.21.15.76:0/2287850512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc94815d100 0x7fc9480b5d10 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fc93800c3c0 tx=0x7fc93800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:33.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.972+0000 7fc93dffb640 1 -- 172.21.15.76:0/2287850512 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc938005ce0 con 0x7fc94815d100 2023-11-09T03:52:33.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.972+0000 7fc93dffb640 1 -- 172.21.15.76:0/2287850512 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc938005e40 con 0x7fc94815d100 2023-11-09T03:52:33.981 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.972+0000 7fc94f561640 1 -- 172.21.15.76:0/2287850512 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc940009400 con 0x7fc94815d100 2023-11-09T03:52:33.981 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.972+0000 7fc93dffb640 1 -- 172.21.15.76:0/2287850512 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9380072d0 con 0x7fc94815d100 2023-11-09T03:52:33.981 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.972+0000 7fc94f561640 1 -- 172.21.15.76:0/2287850512 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9480cbb70 con 0x7fc94815d100 2023-11-09T03:52:33.981 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.973+0000 7fc94f561640 1 -- 172.21.15.76:0/2287850512 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9480cea10 con 0x7fc94815d100 2023-11-09T03:52:33.981 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.974+0000 7fc93dffb640 1 -- 172.21.15.76:0/2287850512 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc938007430 con 0x7fc94815d100 2023-11-09T03:52:33.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.974+0000 7fc93dffb640 1 --2- 172.21.15.76:0/2287850512 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc92c06aea0 0x7fc92c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:33.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.974+0000 7fc93dffb640 1 -- 172.21.15.76:0/2287850512 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 7823+0+0 (secure 0 0 0) 0x7fc938089250 con 0x7fc94815d100 2023-11-09T03:52:33.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.974+0000 7fc94d2fd640 1 --2- 172.21.15.76:0/2287850512 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc92c06aea0 0x7fc92c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:33.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.975+0000 7fc94d2fd640 1 --2- 172.21.15.76:0/2287850512 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc92c06aea0 0x7fc92c06d360 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc9440115a0 tx=0x7fc94400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:33.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:33.978+0000 7fc93dffb640 1 -- 172.21.15.76:0/2287850512 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc938053b70 con 0x7fc94815d100 2023-11-09T03:52:34.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.211+0000 7fc94f561640 1 -- 172.21.15.76:0/2287850512 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:1"} v 0) v1 -- 0x7fc9480684b0 con 0x7fc94815d100 2023-11-09T03:52:34.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.213+0000 7fc93dffb640 1 -- 172.21.15.76:0/2287850512 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:1"}]=0 failed mds gid 8274 v100) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc938017020 con 0x7fc94815d100 2023-11-09T03:52:34.214 INFO:teuthology.orchestra.run.smithi076.stderr:failed mds gid 8274 2023-11-09T03:52:34.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.216+0000 7fc94f561640 1 -- 172.21.15.76:0/2287850512 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc92c06aea0 msgr2=0x7fc92c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:34.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.216+0000 7fc94f561640 1 --2- 172.21.15.76:0/2287850512 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc92c06aea0 0x7fc92c06d360 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc9440115a0 tx=0x7fc94400b040 comp rx=0 tx=0).stop 2023-11-09T03:52:34.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.216+0000 7fc94f561640 1 -- 172.21.15.76:0/2287850512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc94815d100 msgr2=0x7fc9480b5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:34.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.216+0000 7fc94f561640 1 --2- 172.21.15.76:0/2287850512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc94815d100 0x7fc9480b5d10 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fc93800c3c0 tx=0x7fc93800c880 comp rx=0 tx=0).stop 2023-11-09T03:52:34.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.216+0000 7fc94f561640 1 -- 172.21.15.76:0/2287850512 shutdown_connections 2023-11-09T03:52:34.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.216+0000 7fc94f561640 1 --2- 172.21.15.76:0/2287850512 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc92c06aea0 0x7fc92c06d360 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:34.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.216+0000 7fc94f561640 1 --2- 172.21.15.76:0/2287850512 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc94815d4e0 0x7fc9480b6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:34.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.216+0000 7fc94f561640 1 --2- 172.21.15.76:0/2287850512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc94815d100 0x7fc9480b5d10 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:34.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.216+0000 7fc94f561640 1 --2- 172.21.15.76:0/2287850512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9480d1be0 0x7fc94815a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:34.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.216+0000 7fc94f561640 1 -- 172.21.15.76:0/2287850512 >> 172.21.15.76:0/2287850512 conn(0x7fc9480d2ef0 msgr2=0x7fc9480b3740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:34.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.217+0000 7fc94f561640 1 -- 172.21.15.76:0/2287850512 shutdown_connections 2023-11-09T03:52:34.219 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.217+0000 7fc94f561640 1 -- 172.21.15.76:0/2287850512 wait complete. 2023-11-09T03:52:34.230 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-09T03:52:34.230 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-09T03:52:34.230 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-09T03:52:34.231 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:52:34.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:52:34.232 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi076 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c' 2023-11-09T03:52:34.232 INFO:tasks.ceph.mds.c:Stopped 2023-11-09T03:52:34.233 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-09T03:52:34.235 INFO:tasks.ceph.mds.c:Started 2023-11-09T03:52:34.236 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:52:34.310 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2023-11-09T03:52:34.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.336+0000 7fc304832640 1 Processor -- start 2023-11-09T03:52:34.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.336+0000 7fc304832640 1 -- start start 2023-11-09T03:52:34.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.337+0000 7fc304832640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc30015d4e0 0x7fc3000d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:34.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.337+0000 7fc304832640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3000d3b60 0x7fc3000d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:34.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.337+0000 7fc304832640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3000b32b0 con 0x7fc30015d100 2023-11-09T03:52:34.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.337+0000 7fc304832640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3000cec20 con 0x7fc30015d4e0 2023-11-09T03:52:34.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.337+0000 7fc304832640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3000cee10 con 0x7fc3000d3b60 2023-11-09T03:52:34.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.337+0000 7fc2fdd9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc30015d4e0 0x7fc3000d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:34.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.337+0000 7fc2fed9d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3000d3b60 0x7fc3000d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:34.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.337+0000 7fc2fed9d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3000d3b60 0x7fc3000d1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:34230/0 (socket says 172.21.15.76:34230) 2023-11-09T03:52:34.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.337+0000 7fc2fe59c640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc30015d100 0x7fc3000ce510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:58604/0 (socket says 172.21.15.76:58604) 2023-11-09T03:52:34.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.337+0000 7fc2fed9d640 1 -- 172.21.15.76:0/1301157952 learned_addr learned my addr 172.21.15.76:0/1301157952 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:34.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.337+0000 7fc2fdd9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc30015d4e0 0x7fc3000d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:42808/0 (socket says 172.21.15.76:42808) 2023-11-09T03:52:34.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.338+0000 7fc2fdd9b640 1 -- 172.21.15.76:0/1301157952 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3000d3b60 msgr2=0x7fc3000d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:34.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.338+0000 7fc2fdd9b640 1 --2- 172.21.15.76:0/1301157952 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3000d3b60 0x7fc3000d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:34.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.338+0000 7fc2fdd9b640 1 -- 172.21.15.76:0/1301157952 >> v1:172.21.15.113:6789/0 conn(0x7fc30015d100 legacy=0x7fc3000ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:34.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.338+0000 7fc2fdd9b640 1 -- 172.21.15.76:0/1301157952 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3000b4750 con 0x7fc30015d4e0 2023-11-09T03:52:34.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.338+0000 7fc2fd59a640 1 -- 172.21.15.76:0/1301157952 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1974575522 0 0) 0x7fc3000b32b0 con 0x7fc30015d100 2023-11-09T03:52:34.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.338+0000 7fc2fed9d640 1 --2- 172.21.15.76:0/1301157952 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3000d3b60 0x7fc3000d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:52:34.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.338+0000 7fc2fdd9b640 1 --2- 172.21.15.76:0/1301157952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc30015d4e0 0x7fc3000d14d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00097a0 tx=0x7fc2f002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afc70b182a393e6e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:34.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.339+0000 7fc2fd59a640 1 -- 172.21.15.76:0/1301157952 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0031830 con 0x7fc30015d4e0 2023-11-09T03:52:34.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.339+0000 7fc2fd59a640 1 -- 172.21.15.76:0/1301157952 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f0031990 con 0x7fc30015d4e0 2023-11-09T03:52:34.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.339+0000 7fc2fd59a640 1 -- 172.21.15.76:0/1301157952 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0031d20 con 0x7fc30015d4e0 2023-11-09T03:52:34.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.339+0000 7fc304832640 1 -- 172.21.15.76:0/1301157952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc30015d4e0 msgr2=0x7fc3000d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:34.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.339+0000 7fc304832640 1 --2- 172.21.15.76:0/1301157952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc30015d4e0 0x7fc3000d14d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00097a0 tx=0x7fc2f002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:52:34.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.339+0000 7fc304832640 1 -- 172.21.15.76:0/1301157952 shutdown_connections 2023-11-09T03:52:34.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.339+0000 7fc304832640 1 --2- 172.21.15.76:0/1301157952 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3000d3b60 0x7fc3000d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:34.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.339+0000 7fc304832640 1 --2- 172.21.15.76:0/1301157952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc30015d4e0 0x7fc3000d14d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:34.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.339+0000 7fc304832640 1 -- 172.21.15.76:0/1301157952 >> 172.21.15.76:0/1301157952 conn(0x7fc3000d2ef0 msgr2=0x7fc3000d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:34.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.339+0000 7fc304832640 1 -- 172.21.15.76:0/1301157952 shutdown_connections 2023-11-09T03:52:34.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.339+0000 7fc304832640 1 -- 172.21.15.76:0/1301157952 wait complete. 2023-11-09T03:52:34.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.341+0000 7fc304832640 1 Processor -- start 2023-11-09T03:52:34.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.341+0000 7fc304832640 1 -- start start 2023-11-09T03:52:34.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.341+0000 7fc304832640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3000d3b60 0x7fc30015a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:34.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.341+0000 7fc304832640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc30015d100 0x7fc3000b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:34.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.341+0000 7fc304832640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc30015d4e0 0x7fc3000b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:34.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.341+0000 7fc304832640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3000b6440 con 0x7fc30015d4e0 2023-11-09T03:52:34.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.341+0000 7fc304832640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30009cd00 con 0x7fc30015d100 2023-11-09T03:52:34.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.341+0000 7fc304832640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3000b5b10 con 0x7fc3000d3b60 2023-11-09T03:52:34.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.341+0000 7fc2fed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc30015d4e0 0x7fc3000b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:34.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.341+0000 7fc2fed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc30015d4e0 0x7fc3000b0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60036/0 (socket says 172.21.15.76:60036) 2023-11-09T03:52:34.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.341+0000 7fc2fed9d640 1 -- 172.21.15.76:0/4276791770 learned_addr learned my addr 172.21.15.76:0/4276791770 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:34.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.341+0000 7fc2fdd9b640 1 --2- 172.21.15.76:0/4276791770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc30015d100 0x7fc3000b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:34.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.341+0000 7fc2fe59c640 1 --2- 172.21.15.76:0/4276791770 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3000d3b60 0x7fc30015a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:34.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.342+0000 7fc2fed9d640 1 -- 172.21.15.76:0/4276791770 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3000d3b60 msgr2=0x7fc30015a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:34.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.342+0000 7fc2fed9d640 1 --2- 172.21.15.76:0/4276791770 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3000d3b60 0x7fc30015a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:34.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.342+0000 7fc2fed9d640 1 -- 172.21.15.76:0/4276791770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc30015d100 msgr2=0x7fc3000b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:34.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.342+0000 7fc2fed9d640 1 --2- 172.21.15.76:0/4276791770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc30015d100 0x7fc3000b0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:34.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.342+0000 7fc2fed9d640 1 -- 172.21.15.76:0/4276791770 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f8009b30 con 0x7fc30015d4e0 2023-11-09T03:52:34.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.342+0000 7fc2fed9d640 1 --2- 172.21.15.76:0/4276791770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc30015d4e0 0x7fc3000b0ad0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800efc0 tx=0x7fc2f800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:34.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.342+0000 7fc2eeffd640 1 -- 172.21.15.76:0/4276791770 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8018070 con 0x7fc30015d4e0 2023-11-09T03:52:34.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.342+0000 7fc2eeffd640 1 -- 172.21.15.76:0/4276791770 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f800f040 con 0x7fc30015d4e0 2023-11-09T03:52:34.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.342+0000 7fc304832640 1 -- 172.21.15.76:0/4276791770 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f0009400 con 0x7fc30015d4e0 2023-11-09T03:52:34.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.342+0000 7fc2eeffd640 1 -- 172.21.15.76:0/4276791770 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f800ce20 con 0x7fc30015d4e0 2023-11-09T03:52:34.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.342+0000 7fc304832640 1 -- 172.21.15.76:0/4276791770 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3000b6060 con 0x7fc30015d4e0 2023-11-09T03:52:34.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.343+0000 7fc304832640 1 -- 172.21.15.76:0/4276791770 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3000b6580 con 0x7fc30015d4e0 2023-11-09T03:52:34.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.343+0000 7fc2eeffd640 1 -- 172.21.15.76:0/4276791770 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc2f8014480 con 0x7fc30015d4e0 2023-11-09T03:52:34.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.344+0000 7fc2eeffd640 1 --2- 172.21.15.76:0/4276791770 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc2e006ae70 0x7fc2e006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:34.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.344+0000 7fc2eeffd640 1 -- 172.21.15.76:0/4276791770 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 7909+0+0 (secure 0 0 0) 0x7fc2f808a010 con 0x7fc30015d4e0 2023-11-09T03:52:34.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.344+0000 7fc2fe59c640 1 --2- 172.21.15.76:0/4276791770 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc2e006ae70 0x7fc2e006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:34.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.344+0000 7fc2fe59c640 1 --2- 172.21.15.76:0/4276791770 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc2e006ae70 0x7fc2e006d330 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4000fb0 tx=0x7fc2f4007d20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:34.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.348+0000 7fc2eeffd640 1 -- 172.21.15.76:0/4276791770 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc3000b6580 con 0x7fc30015d4e0 2023-11-09T03:52:34.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.548+0000 7fc304832640 1 -- 172.21.15.76:0/4276791770 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3000b6580 con 0x7fc30015d4e0 2023-11-09T03:52:34.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.549+0000 7fc2eeffd640 1 -- 172.21.15.76:0/4276791770 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4136 (secure 0 0 0) 0x7fc3000b6580 con 0x7fc30015d4e0 2023-11-09T03:52:34.550 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:52:34.550 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":101,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:52:34.218740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8310,"mds_1":7144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7144":{"gid":7144,"name":"b","rank":1,"incarnation":101,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/7360844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":7360844},{"type":"v1","addr":"172.21.15.113:6837","nonce":7360844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8310":{"gid":8310,"name":"a","rank":0,"incarnation":98,"state":"up:resolve","state_seq":13,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:52:34.550 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 101 2023-11-09T03:52:34.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.551+0000 7fc304832640 1 -- 172.21.15.76:0/4276791770 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc2e006ae70 msgr2=0x7fc2e006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:34.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.551+0000 7fc304832640 1 --2- 172.21.15.76:0/4276791770 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc2e006ae70 0x7fc2e006d330 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4000fb0 tx=0x7fc2f4007d20 comp rx=0 tx=0).stop 2023-11-09T03:52:34.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.551+0000 7fc304832640 1 -- 172.21.15.76:0/4276791770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc30015d4e0 msgr2=0x7fc3000b0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:34.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.551+0000 7fc304832640 1 --2- 172.21.15.76:0/4276791770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc30015d4e0 0x7fc3000b0ad0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800efc0 tx=0x7fc2f800c490 comp rx=0 tx=0).stop 2023-11-09T03:52:34.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.551+0000 7fc304832640 1 -- 172.21.15.76:0/4276791770 shutdown_connections 2023-11-09T03:52:34.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.551+0000 7fc304832640 1 --2- 172.21.15.76:0/4276791770 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc2e006ae70 0x7fc2e006d330 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:34.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.551+0000 7fc304832640 1 --2- 172.21.15.76:0/4276791770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc30015d4e0 0x7fc3000b0ad0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:34.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.551+0000 7fc304832640 1 --2- 172.21.15.76:0/4276791770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc30015d100 0x7fc3000b0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:34.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.551+0000 7fc304832640 1 --2- 172.21.15.76:0/4276791770 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3000d3b60 0x7fc30015a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:34.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.551+0000 7fc304832640 1 -- 172.21.15.76:0/4276791770 >> 172.21.15.76:0/4276791770 conn(0x7fc3000d2ef0 msgr2=0x7fc3000b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:34.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.551+0000 7fc304832640 1 -- 172.21.15.76:0/4276791770 shutdown_connections 2023-11-09T03:52:34.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:34.551+0000 7fc304832640 1 -- 172.21.15.76:0/4276791770 wait complete. 2023-11-09T03:52:34.562 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-11-09T03:52:38.754 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2023-11-09T03:52:39.563 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:52:39.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.674+0000 7f40fa15e640 1 Processor -- start 2023-11-09T03:52:39.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.674+0000 7f40fa15e640 1 -- start start 2023-11-09T03:52:39.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.674+0000 7f40fa15e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f40f415d100 0x7f40f40ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:39.676 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.674+0000 7f40fa15e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f40f415d4e0 0x7f40f40d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:39.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.674+0000 7f40fa15e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40f40b32b0 con 0x7f40f40d3b60 2023-11-09T03:52:39.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.674+0000 7f40fa15e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40f40cee10 con 0x7f40f415d4e0 2023-11-09T03:52:39.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.674+0000 7f40fa15e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40f40b4750 con 0x7f40f415d100 2023-11-09T03:52:39.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.675+0000 7f40f2ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f40f415d4e0 0x7f40f40d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:39.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.675+0000 7f40f2ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f40f415d4e0 0x7f40f40d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60050/0 (socket says 172.21.15.76:60050) 2023-11-09T03:52:39.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.675+0000 7f40f2ffd640 1 -- 172.21.15.76:0/148170710 learned_addr learned my addr 172.21.15.76:0/148170710 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:39.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.675+0000 7f40f37fe640 1 --2- 172.21.15.76:0/148170710 >> v2:172.21.15.113:3300/0 conn(0x7f40f415d100 0x7f40f40ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:39.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.675+0000 7f40f2ffd640 1 -- 172.21.15.76:0/148170710 >> v2:172.21.15.113:3300/0 conn(0x7f40f415d100 msgr2=0x7f40f40ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:39.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.675+0000 7f40f2ffd640 1 --2- 172.21.15.76:0/148170710 >> v2:172.21.15.113:3300/0 conn(0x7f40f415d100 0x7f40f40ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:39.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.675+0000 7f40f2ffd640 1 -- 172.21.15.76:0/148170710 >> v1:172.21.15.113:6789/0 conn(0x7f40f40d3b60 legacy=0x7f40f40cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:39.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.675+0000 7f40f2ffd640 1 -- 172.21.15.76:0/148170710 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f40b2980 con 0x7f40f415d4e0 2023-11-09T03:52:39.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.675+0000 7f40f37fe640 1 --2- 172.21.15.76:0/148170710 >> v2:172.21.15.113:3300/0 conn(0x7f40f415d100 0x7f40f40ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:52:39.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.676+0000 7f40f2ffd640 1 --2- 172.21.15.76:0/148170710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40f415d4e0 0x7f40f40d14d0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f40f40d09f0 tx=0x7f40e802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e03707cefccea8 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:39.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.676+0000 7f40f27fc640 1 -- 172.21.15.76:0/148170710 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e8031870 con 0x7f40f415d4e0 2023-11-09T03:52:39.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.676+0000 7f40f27fc640 1 -- 172.21.15.76:0/148170710 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e80319d0 con 0x7f40f415d4e0 2023-11-09T03:52:39.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.676+0000 7f40f27fc640 1 -- 172.21.15.76:0/148170710 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e8031d60 con 0x7f40f415d4e0 2023-11-09T03:52:39.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.676+0000 7f40fa15e640 1 -- 172.21.15.76:0/148170710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40f415d4e0 msgr2=0x7f40f40d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:39.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.676+0000 7f40fa15e640 1 --2- 172.21.15.76:0/148170710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40f415d4e0 0x7f40f40d14d0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f40f40d09f0 tx=0x7f40e802b910 comp rx=0 tx=0).stop 2023-11-09T03:52:39.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.676+0000 7f40fa15e640 1 -- 172.21.15.76:0/148170710 shutdown_connections 2023-11-09T03:52:39.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.676+0000 7f40fa15e640 1 --2- 172.21.15.76:0/148170710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40f415d4e0 0x7f40f40d14d0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:39.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.676+0000 7f40fa15e640 1 --2- 172.21.15.76:0/148170710 >> v2:172.21.15.113:3300/0 conn(0x7f40f415d100 0x7f40f40ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:39.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.676+0000 7f40fa15e640 1 -- 172.21.15.76:0/148170710 >> 172.21.15.76:0/148170710 conn(0x7f40f40d2ef0 msgr2=0x7f40f40d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:39.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.676+0000 7f40fa15e640 1 -- 172.21.15.76:0/148170710 shutdown_connections 2023-11-09T03:52:39.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.677+0000 7f40fa15e640 1 -- 172.21.15.76:0/148170710 wait complete. 2023-11-09T03:52:39.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.678+0000 7f40fa15e640 1 Processor -- start 2023-11-09T03:52:39.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.678+0000 7f40fa15e640 1 -- start start 2023-11-09T03:52:39.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.679+0000 7f40fa15e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40f40d3b60 0x7f40f415dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:39.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.679+0000 7f40fa15e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40f415d100 0x7f40f40b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:39.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.679+0000 7f40fa15e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40f415d4e0 0x7f40f40b6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:39.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.679+0000 7f40fa15e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f40cd070 con 0x7f40f40d3b60 2023-11-09T03:52:39.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.679+0000 7f40fa15e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f40cc7d0 con 0x7f40f415d4e0 2023-11-09T03:52:39.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.679+0000 7f40fa15e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f40ccab0 con 0x7f40f415d100 2023-11-09T03:52:39.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.679+0000 7f40f37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40f40d3b60 0x7f40f415dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:39.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.679+0000 7f40f37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40f40d3b60 0x7f40f415dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60052/0 (socket says 172.21.15.76:60052) 2023-11-09T03:52:39.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.679+0000 7f40f37fe640 1 -- 172.21.15.76:0/673905094 learned_addr learned my addr 172.21.15.76:0/673905094 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:39.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.679+0000 7f40f2ffd640 1 --2- 172.21.15.76:0/673905094 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40f415d100 0x7f40f40b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:39.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.679+0000 7f40f3fff640 1 --2- 172.21.15.76:0/673905094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40f415d4e0 0x7f40f40b6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:39.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.679+0000 7f40f37fe640 1 -- 172.21.15.76:0/673905094 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40f415d100 msgr2=0x7f40f40b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:39.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.679+0000 7f40f37fe640 1 --2- 172.21.15.76:0/673905094 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40f415d100 0x7f40f40b5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:39.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.679+0000 7f40f37fe640 1 -- 172.21.15.76:0/673905094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40f415d4e0 msgr2=0x7f40f40b6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:39.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.679+0000 7f40f37fe640 1 --2- 172.21.15.76:0/673905094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40f415d4e0 0x7f40f40b6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:39.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.679+0000 7f40f37fe640 1 -- 172.21.15.76:0/673905094 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e80094d0 con 0x7f40f40d3b60 2023-11-09T03:52:39.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.679+0000 7f40f37fe640 1 --2- 172.21.15.76:0/673905094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40f40d3b60 0x7f40f415dd90 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f40e400c7f0 tx=0x7f40e400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:39.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.679+0000 7f40dbfff640 1 -- 172.21.15.76:0/673905094 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e4004220 con 0x7f40f40d3b60 2023-11-09T03:52:39.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.679+0000 7f40dbfff640 1 -- 172.21.15.76:0/673905094 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e4004380 con 0x7f40f40d3b60 2023-11-09T03:52:39.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.679+0000 7f40fa15e640 1 -- 172.21.15.76:0/673905094 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f40ccd90 con 0x7f40f40d3b60 2023-11-09T03:52:39.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.679+0000 7f40dbfff640 1 -- 172.21.15.76:0/673905094 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e4005bb0 con 0x7f40f40d3b60 2023-11-09T03:52:39.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.679+0000 7f40fa15e640 1 -- 172.21.15.76:0/673905094 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f40ca180 con 0x7f40f40d3b60 2023-11-09T03:52:39.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.680+0000 7f40fa15e640 1 -- 172.21.15.76:0/673905094 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f40ca310 con 0x7f40f40d3b60 2023-11-09T03:52:39.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.681+0000 7f40dbfff640 1 -- 172.21.15.76:0/673905094 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f40e4005d10 con 0x7f40f40d3b60 2023-11-09T03:52:39.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.681+0000 7f40dbfff640 1 --2- 172.21.15.76:0/673905094 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f40e006aea0 0x7f40e006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:39.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.681+0000 7f40dbfff640 1 -- 172.21.15.76:0/673905094 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 7909+0+0 (secure 0 0 0) 0x7f40e408a0a0 con 0x7f40f40d3b60 2023-11-09T03:52:39.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.681+0000 7f40f2ffd640 1 --2- 172.21.15.76:0/673905094 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f40e006aea0 0x7f40e006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:39.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.682+0000 7f40f2ffd640 1 --2- 172.21.15.76:0/673905094 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f40e006aea0 0x7f40e006d360 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f40e8036000 tx=0x7f40e803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:39.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.685+0000 7f40dbfff640 1 -- 172.21.15.76:0/673905094 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f40e4054940 con 0x7f40f40d3b60 2023-11-09T03:52:39.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.885+0000 7f40fa15e640 1 -- 172.21.15.76:0/673905094 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40f40684b0 con 0x7f40f40d3b60 2023-11-09T03:52:39.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.886+0000 7f40dbfff640 1 -- 172.21.15.76:0/673905094 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4930 (secure 0 0 0) 0x7f40e4057f60 con 0x7f40f40d3b60 2023-11-09T03:52:39.887 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:52:39.887 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":103,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8568,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:52:35.229057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8310,"mds_1":7144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7144":{"gid":7144,"name":"b","rank":1,"incarnation":101,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.113:6837/7360844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":7360844},{"type":"v1","addr":"172.21.15.113:6837","nonce":7360844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8310":{"gid":8310,"name":"a","rank":0,"incarnation":98,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:52:39.887 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 103 2023-11-09T03:52:39.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.888+0000 7f40fa15e640 1 -- 172.21.15.76:0/673905094 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f40e006aea0 msgr2=0x7f40e006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:39.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.888+0000 7f40fa15e640 1 --2- 172.21.15.76:0/673905094 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f40e006aea0 0x7f40e006d360 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f40e8036000 tx=0x7f40e803c040 comp rx=0 tx=0).stop 2023-11-09T03:52:39.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.888+0000 7f40fa15e640 1 -- 172.21.15.76:0/673905094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40f40d3b60 msgr2=0x7f40f415dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:39.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.888+0000 7f40fa15e640 1 --2- 172.21.15.76:0/673905094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40f40d3b60 0x7f40f415dd90 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f40e400c7f0 tx=0x7f40e400ccb0 comp rx=0 tx=0).stop 2023-11-09T03:52:39.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.888+0000 7f40fa15e640 1 -- 172.21.15.76:0/673905094 shutdown_connections 2023-11-09T03:52:39.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.888+0000 7f40fa15e640 1 --2- 172.21.15.76:0/673905094 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f40e006aea0 0x7f40e006d360 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:39.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.888+0000 7f40fa15e640 1 --2- 172.21.15.76:0/673905094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40f415d4e0 0x7f40f40b6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:39.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.888+0000 7f40fa15e640 1 --2- 172.21.15.76:0/673905094 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40f415d100 0x7f40f40b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:39.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.888+0000 7f40fa15e640 1 --2- 172.21.15.76:0/673905094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40f40d3b60 0x7f40f415dd90 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:39.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.888+0000 7f40fa15e640 1 -- 172.21.15.76:0/673905094 >> 172.21.15.76:0/673905094 conn(0x7f40f40d2ef0 msgr2=0x7f40f406ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:39.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.888+0000 7f40fa15e640 1 -- 172.21.15.76:0/673905094 shutdown_connections 2023-11-09T03:52:39.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.888+0000 7f40fa15e640 1 -- 172.21.15.76:0/673905094 wait complete. 2023-11-09T03:52:39.899 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-09T03:52:39.899 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:52:39.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.998+0000 7f3af7fa5640 1 Processor -- start 2023-11-09T03:52:39.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.999+0000 7f3af7fa5640 1 -- start start 2023-11-09T03:52:40.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.999+0000 7f3af7fa5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3af00addd0 0x7f3af00cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:40.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.999+0000 7f3af7fa5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3af006c220 0x7f3af00ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:40.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.999+0000 7f3af7fa5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3af00cd720 con 0x7f3af00cc680 2023-11-09T03:52:40.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.999+0000 7f3af7fa5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3af00ae1b0 con 0x7f3af00addd0 2023-11-09T03:52:40.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:39.999+0000 7f3af7fa5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3af00ccb10 con 0x7f3af006c220 2023-11-09T03:52:40.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.000+0000 7f3af5d41640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3af00addd0 0x7f3af00cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:40.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.000+0000 7f3af5d41640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3af00addd0 0x7f3af00cdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60054/0 (socket says 172.21.15.76:60054) 2023-11-09T03:52:40.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.000+0000 7f3af5d41640 1 -- 172.21.15.76:0/1970148869 learned_addr learned my addr 172.21.15.76:0/1970148869 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:40.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.000+0000 7f3af6542640 1 --2- 172.21.15.76:0/1970148869 >> v2:172.21.15.113:3300/0 conn(0x7f3af006c220 0x7f3af00ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:40.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.000+0000 7f3af5d41640 1 -- 172.21.15.76:0/1970148869 >> v2:172.21.15.113:3300/0 conn(0x7f3af006c220 msgr2=0x7f3af00ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:40.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.000+0000 7f3af5d41640 1 --2- 172.21.15.76:0/1970148869 >> v2:172.21.15.113:3300/0 conn(0x7f3af006c220 0x7f3af00ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:40.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.000+0000 7f3af5d41640 1 -- 172.21.15.76:0/1970148869 >> v1:172.21.15.113:6789/0 conn(0x7f3af00cc680 legacy=0x7f3af006bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:40.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.000+0000 7f3af5d41640 1 -- 172.21.15.76:0/1970148869 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af00ccdf0 con 0x7f3af00addd0 2023-11-09T03:52:40.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.001+0000 7f3af5d41640 1 --2- 172.21.15.76:0/1970148869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3af00addd0 0x7f3af00cdbb0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f3af00ce6d0 tx=0x7f3aec02e190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b3db8ef157f92c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:40.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.001+0000 7f3af4d3f640 1 -- 172.21.15.76:0/1970148869 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec02ed30 con 0x7f3af00addd0 2023-11-09T03:52:40.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.001+0000 7f3af4d3f640 1 -- 172.21.15.76:0/1970148869 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aec02ee90 con 0x7f3af00addd0 2023-11-09T03:52:40.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.001+0000 7f3af4d3f640 1 -- 172.21.15.76:0/1970148869 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec036610 con 0x7f3af00addd0 2023-11-09T03:52:40.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.001+0000 7f3af7fa5640 1 -- 172.21.15.76:0/1970148869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3af00addd0 msgr2=0x7f3af00cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:40.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.001+0000 7f3af7fa5640 1 --2- 172.21.15.76:0/1970148869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3af00addd0 0x7f3af00cdbb0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f3af00ce6d0 tx=0x7f3aec02e190 comp rx=0 tx=0).stop 2023-11-09T03:52:40.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.001+0000 7f3af7fa5640 1 -- 172.21.15.76:0/1970148869 shutdown_connections 2023-11-09T03:52:40.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.001+0000 7f3af7fa5640 1 --2- 172.21.15.76:0/1970148869 >> v2:172.21.15.113:3300/0 conn(0x7f3af006c220 0x7f3af00ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:40.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.001+0000 7f3af7fa5640 1 --2- 172.21.15.76:0/1970148869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3af00addd0 0x7f3af00cdbb0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:40.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.001+0000 7f3af7fa5640 1 -- 172.21.15.76:0/1970148869 >> 172.21.15.76:0/1970148869 conn(0x7f3af00cb530 msgr2=0x7f3af00cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:40.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.001+0000 7f3af7fa5640 1 -- 172.21.15.76:0/1970148869 shutdown_connections 2023-11-09T03:52:40.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.001+0000 7f3af7fa5640 1 -- 172.21.15.76:0/1970148869 wait complete. 2023-11-09T03:52:40.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.002+0000 7f3af7fa5640 1 Processor -- start 2023-11-09T03:52:40.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.003+0000 7f3af7fa5640 1 -- start start 2023-11-09T03:52:40.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.003+0000 7f3af7fa5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3af006c220 0x7f3af0151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:40.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.003+0000 7f3af7fa5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3af00addd0 0x7f3af015a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:40.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.003+0000 7f3af7fa5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3af015c0b0 0x7f3af00704a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:40.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.003+0000 7f3af7fa5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af015a130 con 0x7f3af006c220 2023-11-09T03:52:40.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.003+0000 7f3af7fa5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af0159b50 con 0x7f3af00addd0 2023-11-09T03:52:40.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.003+0000 7f3af7fa5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af00c44f0 con 0x7f3af015c0b0 2023-11-09T03:52:40.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.003+0000 7f3af5d41640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3af006c220 0x7f3af0151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:40.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.003+0000 7f3af5d41640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3af006c220 0x7f3af0151e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60058/0 (socket says 172.21.15.76:60058) 2023-11-09T03:52:40.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.003+0000 7f3af5d41640 1 -- 172.21.15.76:0/3481643877 learned_addr learned my addr 172.21.15.76:0/3481643877 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:40.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.004+0000 7f3af5d41640 1 -- 172.21.15.76:0/3481643877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3af015c0b0 msgr2=0x7f3af00704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:40.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.004+0000 7f3af5d41640 1 --2- 172.21.15.76:0/3481643877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3af015c0b0 0x7f3af00704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:40.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.004+0000 7f3af5d41640 1 -- 172.21.15.76:0/3481643877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3af00addd0 msgr2=0x7f3af015a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:40.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.004+0000 7f3af5d41640 1 --2- 172.21.15.76:0/3481643877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3af00addd0 0x7f3af015a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:40.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.004+0000 7f3af5d41640 1 -- 172.21.15.76:0/3481643877 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aec009b30 con 0x7f3af006c220 2023-11-09T03:52:40.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.004+0000 7f3af5d41640 1 --2- 172.21.15.76:0/3481643877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3af006c220 0x7f3af0151e50 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f3aec009f40 tx=0x7f3aec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:40.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.004+0000 7f3ae67fc640 1 -- 172.21.15.76:0/3481643877 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec042670 con 0x7f3af006c220 2023-11-09T03:52:40.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.004+0000 7f3ae67fc640 1 -- 172.21.15.76:0/3481643877 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aec0427d0 con 0x7f3af006c220 2023-11-09T03:52:40.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.004+0000 7f3af7fa5640 1 -- 172.21.15.76:0/3481643877 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af00c4790 con 0x7f3af006c220 2023-11-09T03:52:40.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.004+0000 7f3ae67fc640 1 -- 172.21.15.76:0/3481643877 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec042b60 con 0x7f3af006c220 2023-11-09T03:52:40.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.004+0000 7f3af7fa5640 1 -- 172.21.15.76:0/3481643877 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af015ab00 con 0x7f3af006c220 2023-11-09T03:52:40.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.005+0000 7f3af7fa5640 1 -- 172.21.15.76:0/3481643877 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af00684b0 con 0x7f3af006c220 2023-11-09T03:52:40.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.005+0000 7f3ae67fc640 1 -- 172.21.15.76:0/3481643877 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3aec042cc0 con 0x7f3af006c220 2023-11-09T03:52:40.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.006+0000 7f3ae67fc640 1 --2- 172.21.15.76:0/3481643877 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3ad406ae70 0x7f3ad406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:40.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.006+0000 7f3ae67fc640 1 -- 172.21.15.76:0/3481643877 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 7909+0+0 (secure 0 0 0) 0x7f3aec0aeed0 con 0x7f3af006c220 2023-11-09T03:52:40.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.006+0000 7f3af5540640 1 --2- 172.21.15.76:0/3481643877 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3ad406ae70 0x7f3ad406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:40.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.006+0000 7f3af5540640 1 --2- 172.21.15.76:0/3481643877 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3ad406ae70 0x7f3ad406d330 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0000f60 tx=0x7f3ae00079d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:40.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.010+0000 7f3ae67fc640 1 -- 172.21.15.76:0/3481643877 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3aec079820 con 0x7f3af006c220 2023-11-09T03:52:40.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.209+0000 7f3af7fa5640 1 -- 172.21.15.76:0/3481643877 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3af00c38c0 con 0x7f3af006c220 2023-11-09T03:52:40.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.210+0000 7f3ae67fc640 1 -- 172.21.15.76:0/3481643877 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4930 (secure 0 0 0) 0x7f3aec009510 con 0x7f3af006c220 2023-11-09T03:52:40.211 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:52:40.211 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":103,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8568,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:52:35.229057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8310,"mds_1":7144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7144":{"gid":7144,"name":"b","rank":1,"incarnation":101,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.113:6837/7360844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":7360844},{"type":"v1","addr":"172.21.15.113:6837","nonce":7360844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8310":{"gid":8310,"name":"a","rank":0,"incarnation":98,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:52:40.211 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 103 2023-11-09T03:52:40.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.212+0000 7f3af7fa5640 1 -- 172.21.15.76:0/3481643877 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3ad406ae70 msgr2=0x7f3ad406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:40.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.212+0000 7f3af7fa5640 1 --2- 172.21.15.76:0/3481643877 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3ad406ae70 0x7f3ad406d330 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0000f60 tx=0x7f3ae00079d0 comp rx=0 tx=0).stop 2023-11-09T03:52:40.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.212+0000 7f3af7fa5640 1 -- 172.21.15.76:0/3481643877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3af006c220 msgr2=0x7f3af0151e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:40.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.212+0000 7f3af7fa5640 1 --2- 172.21.15.76:0/3481643877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3af006c220 0x7f3af0151e50 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f3aec009f40 tx=0x7f3aec004a40 comp rx=0 tx=0).stop 2023-11-09T03:52:40.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.212+0000 7f3af7fa5640 1 -- 172.21.15.76:0/3481643877 shutdown_connections 2023-11-09T03:52:40.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.212+0000 7f3af7fa5640 1 --2- 172.21.15.76:0/3481643877 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3ad406ae70 0x7f3ad406d330 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:40.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.212+0000 7f3af7fa5640 1 --2- 172.21.15.76:0/3481643877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3af015c0b0 0x7f3af00704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:40.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.212+0000 7f3af7fa5640 1 --2- 172.21.15.76:0/3481643877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3af00addd0 0x7f3af015a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:40.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.212+0000 7f3af7fa5640 1 --2- 172.21.15.76:0/3481643877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3af006c220 0x7f3af0151e50 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:40.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.212+0000 7f3af7fa5640 1 -- 172.21.15.76:0/3481643877 >> 172.21.15.76:0/3481643877 conn(0x7f3af00cb530 msgr2=0x7f3af00b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:40.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.212+0000 7f3af7fa5640 1 -- 172.21.15.76:0/3481643877 shutdown_connections 2023-11-09T03:52:40.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:40.212+0000 7f3af7fa5640 1 -- 172.21.15.76:0/3481643877 wait complete. 2023-11-09T03:52:40.223 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:reconnect 2023-11-09T03:52:41.223 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:52:41.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.330+0000 7ff24f464640 1 Processor -- start 2023-11-09T03:52:41.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.330+0000 7ff24f464640 1 -- start start 2023-11-09T03:52:41.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.331+0000 7ff24f464640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff24815d110 0x7ff24806b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:41.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.331+0000 7ff24f464640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2480b3ca0 0x7ff2480ace70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:41.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.331+0000 7ff24f464640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff2480ad3b0 0x7ff2480b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:41.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.331+0000 7ff24f464640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2480664f0 con 0x7ff2480ad3b0 2023-11-09T03:52:41.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.331+0000 7ff24f464640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff24815d4f0 con 0x7ff24815d110 2023-11-09T03:52:41.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.331+0000 7ff24f464640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff24806b9f0 con 0x7ff2480b3ca0 2023-11-09T03:52:41.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.331+0000 7ff24da01640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff2480ad3b0 0x7ff2480b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:41.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.331+0000 7ff24da01640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff2480ad3b0 0x7ff2480b2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60060/0 (socket says 172.21.15.76:60060) 2023-11-09T03:52:41.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.331+0000 7ff24da01640 1 -- 172.21.15.76:0/962763056 learned_addr learned my addr 172.21.15.76:0/962763056 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:41.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.331+0000 7ff24c9ff640 1 --2- 172.21.15.76:0/962763056 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2480b3ca0 0x7ff2480ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:41.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.331+0000 7ff24d200640 1 --2- 172.21.15.76:0/962763056 >> v2:172.21.15.113:3300/0 conn(0x7ff24815d110 0x7ff24806b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:41.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.332+0000 7ff24da01640 1 -- 172.21.15.76:0/962763056 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2480b3ca0 msgr2=0x7ff2480ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:41.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.332+0000 7ff24da01640 1 --2- 172.21.15.76:0/962763056 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2480b3ca0 0x7ff2480ace70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:41.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.332+0000 7ff24da01640 1 -- 172.21.15.76:0/962763056 >> v2:172.21.15.113:3300/0 conn(0x7ff24815d110 msgr2=0x7ff24806b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:41.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.332+0000 7ff24da01640 1 --2- 172.21.15.76:0/962763056 >> v2:172.21.15.113:3300/0 conn(0x7ff24815d110 0x7ff24806b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:41.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.332+0000 7ff24da01640 1 -- 172.21.15.76:0/962763056 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24806c2e0 con 0x7ff2480ad3b0 2023-11-09T03:52:41.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.332+0000 7ff24d200640 1 --2- 172.21.15.76:0/962763056 >> v2:172.21.15.113:3300/0 conn(0x7ff24815d110 0x7ff24806b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:52:41.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.332+0000 7ff24da01640 1 --2- 172.21.15.76:0/962763056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2480ad3b0 0x7ff2480b2100 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7ff2480ccd90 tx=0x7ff24002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd19e730aace5a73 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:41.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.333+0000 7ff23ffff640 1 -- 172.21.15.76:0/962763056 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff240031870 con 0x7ff2480ad3b0 2023-11-09T03:52:41.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.333+0000 7ff23ffff640 1 -- 172.21.15.76:0/962763056 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2400319d0 con 0x7ff2480ad3b0 2023-11-09T03:52:41.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.333+0000 7ff23ffff640 1 -- 172.21.15.76:0/962763056 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff240031d60 con 0x7ff2480ad3b0 2023-11-09T03:52:41.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.333+0000 7ff24f464640 1 -- 172.21.15.76:0/962763056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2480ad3b0 msgr2=0x7ff2480b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:41.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.333+0000 7ff24f464640 1 --2- 172.21.15.76:0/962763056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2480ad3b0 0x7ff2480b2100 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7ff2480ccd90 tx=0x7ff24002b910 comp rx=0 tx=0).stop 2023-11-09T03:52:41.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.333+0000 7ff24f464640 1 -- 172.21.15.76:0/962763056 shutdown_connections 2023-11-09T03:52:41.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.333+0000 7ff24f464640 1 --2- 172.21.15.76:0/962763056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2480ad3b0 0x7ff2480b2100 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:41.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.333+0000 7ff24f464640 1 --2- 172.21.15.76:0/962763056 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2480b3ca0 0x7ff2480ace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:41.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.333+0000 7ff24f464640 1 --2- 172.21.15.76:0/962763056 >> v2:172.21.15.113:3300/0 conn(0x7ff24815d110 0x7ff24806b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:41.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.333+0000 7ff24f464640 1 -- 172.21.15.76:0/962763056 >> 172.21.15.76:0/962763056 conn(0x7ff2480cbd30 msgr2=0x7ff2480cc140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:41.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.333+0000 7ff24f464640 1 -- 172.21.15.76:0/962763056 shutdown_connections 2023-11-09T03:52:41.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.333+0000 7ff24f464640 1 -- 172.21.15.76:0/962763056 wait complete. 2023-11-09T03:52:41.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.335+0000 7ff24f464640 1 Processor -- start 2023-11-09T03:52:41.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.335+0000 7ff24f464640 1 -- start start 2023-11-09T03:52:41.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.335+0000 7ff24f464640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2480ad3b0 0x7ff24814c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:41.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.335+0000 7ff24f464640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2480b3ca0 0x7ff24814cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:41.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.335+0000 7ff24f464640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff24815d110 0x7ff2480af660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:41.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.335+0000 7ff24f464640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2481f66a0 con 0x7ff2480ad3b0 2023-11-09T03:52:41.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.335+0000 7ff24f464640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2481f6980 con 0x7ff2480b3ca0 2023-11-09T03:52:41.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.335+0000 7ff24f464640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2481f6c60 con 0x7ff24815d110 2023-11-09T03:52:41.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.335+0000 7ff24d200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2480ad3b0 0x7ff24814c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:41.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.335+0000 7ff24d200640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2480ad3b0 0x7ff24814c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60064/0 (socket says 172.21.15.76:60064) 2023-11-09T03:52:41.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.335+0000 7ff24d200640 1 -- 172.21.15.76:0/1321012064 learned_addr learned my addr 172.21.15.76:0/1321012064 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:41.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.336+0000 7ff24c9ff640 1 --2- 172.21.15.76:0/1321012064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2480b3ca0 0x7ff24814cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:41.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.336+0000 7ff24da01640 1 --2- 172.21.15.76:0/1321012064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff24815d110 0x7ff2480af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:41.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.336+0000 7ff24d200640 1 -- 172.21.15.76:0/1321012064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff24815d110 msgr2=0x7ff2480af660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:41.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.336+0000 7ff24d200640 1 --2- 172.21.15.76:0/1321012064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff24815d110 0x7ff2480af660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:41.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.336+0000 7ff24d200640 1 -- 172.21.15.76:0/1321012064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2480b3ca0 msgr2=0x7ff24814cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:41.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.336+0000 7ff24d200640 1 --2- 172.21.15.76:0/1321012064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2480b3ca0 0x7ff24814cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:41.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.336+0000 7ff24d200640 1 -- 172.21.15.76:0/1321012064 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2400094d0 con 0x7ff2480ad3b0 2023-11-09T03:52:41.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.336+0000 7ff24d200640 1 --2- 172.21.15.76:0/1321012064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2480ad3b0 0x7ff24814c040 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7ff24400d100 tx=0x7ff24400d5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:41.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.336+0000 7ff23dffb640 1 -- 172.21.15.76:0/1321012064 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff244003b90 con 0x7ff2480ad3b0 2023-11-09T03:52:41.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.336+0000 7ff23dffb640 1 -- 172.21.15.76:0/1321012064 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff24400f040 con 0x7ff2480ad3b0 2023-11-09T03:52:41.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.336+0000 7ff24f464640 1 -- 172.21.15.76:0/1321012064 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2481f6f40 con 0x7ff2480ad3b0 2023-11-09T03:52:41.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.336+0000 7ff23dffb640 1 -- 172.21.15.76:0/1321012064 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff244004030 con 0x7ff2480ad3b0 2023-11-09T03:52:41.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.336+0000 7ff24f464640 1 -- 172.21.15.76:0/1321012064 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2481f6230 con 0x7ff2480ad3b0 2023-11-09T03:52:41.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.337+0000 7ff24f464640 1 -- 172.21.15.76:0/1321012064 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2480684b0 con 0x7ff2480ad3b0 2023-11-09T03:52:41.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.338+0000 7ff23dffb640 1 -- 172.21.15.76:0/1321012064 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff244004190 con 0x7ff2480ad3b0 2023-11-09T03:52:41.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.338+0000 7ff23dffb640 1 --2- 172.21.15.76:0/1321012064 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff22c06ae70 0x7ff22c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:41.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.338+0000 7ff23dffb640 1 -- 172.21.15.76:0/1321012064 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 7909+0+0 (secure 0 0 0) 0x7ff24408a160 con 0x7ff2480ad3b0 2023-11-09T03:52:41.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.338+0000 7ff24c9ff640 1 --2- 172.21.15.76:0/1321012064 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff22c06ae70 0x7ff22c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:41.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.339+0000 7ff24c9ff640 1 --2- 172.21.15.76:0/1321012064 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff22c06ae70 0x7ff22c06d330 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff248154f90 tx=0x7ff238008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:41.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.342+0000 7ff23dffb640 1 -- 172.21.15.76:0/1321012064 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff244054ab0 con 0x7ff2480ad3b0 2023-11-09T03:52:41.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.549+0000 7ff24f464640 1 -- 172.21.15.76:0/1321012064 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2480700e0 con 0x7ff2480ad3b0 2023-11-09T03:52:41.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.550+0000 7ff23dffb640 1 -- 172.21.15.76:0/1321012064 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4930 (secure 0 0 0) 0x7ff24401a020 con 0x7ff2480ad3b0 2023-11-09T03:52:41.551 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:52:41.551 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":103,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8568,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:52:35.229057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8310,"mds_1":7144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7144":{"gid":7144,"name":"b","rank":1,"incarnation":101,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.113:6837/7360844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":7360844},{"type":"v1","addr":"172.21.15.113:6837","nonce":7360844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8310":{"gid":8310,"name":"a","rank":0,"incarnation":98,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:52:41.552 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 103 2023-11-09T03:52:41.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.552+0000 7ff24f464640 1 -- 172.21.15.76:0/1321012064 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff22c06ae70 msgr2=0x7ff22c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:41.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.552+0000 7ff24f464640 1 --2- 172.21.15.76:0/1321012064 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff22c06ae70 0x7ff22c06d330 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff248154f90 tx=0x7ff238008040 comp rx=0 tx=0).stop 2023-11-09T03:52:41.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.552+0000 7ff24f464640 1 -- 172.21.15.76:0/1321012064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2480ad3b0 msgr2=0x7ff24814c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:41.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.552+0000 7ff24f464640 1 --2- 172.21.15.76:0/1321012064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2480ad3b0 0x7ff24814c040 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7ff24400d100 tx=0x7ff24400d5c0 comp rx=0 tx=0).stop 2023-11-09T03:52:41.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.552+0000 7ff24f464640 1 -- 172.21.15.76:0/1321012064 shutdown_connections 2023-11-09T03:52:41.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.552+0000 7ff24f464640 1 --2- 172.21.15.76:0/1321012064 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff22c06ae70 0x7ff22c06d330 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:41.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.552+0000 7ff24f464640 1 --2- 172.21.15.76:0/1321012064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff24815d110 0x7ff2480af660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:41.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.552+0000 7ff24f464640 1 --2- 172.21.15.76:0/1321012064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2480b3ca0 0x7ff24814cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:41.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.552+0000 7ff24f464640 1 --2- 172.21.15.76:0/1321012064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff2480ad3b0 0x7ff24814c040 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:41.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.552+0000 7ff24f464640 1 -- 172.21.15.76:0/1321012064 >> 172.21.15.76:0/1321012064 conn(0x7ff2480cbd30 msgr2=0x7ff24806a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:41.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.552+0000 7ff24f464640 1 -- 172.21.15.76:0/1321012064 shutdown_connections 2023-11-09T03:52:41.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:41.552+0000 7ff24f464640 1 -- 172.21.15.76:0/1321012064 wait complete. 2023-11-09T03:52:41.563 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:reconnect 2023-11-09T03:52:42.565 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:52:42.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.679+0000 7f39cded4640 1 Processor -- start 2023-11-09T03:52:42.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.679+0000 7f39cded4640 1 -- start start 2023-11-09T03:52:42.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.680+0000 7f39cded4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f39c80b0c40 0x7f39c806fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:42.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.680+0000 7f39cded4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39c806bb60 0x7f39c8070530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:42.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.680+0000 7f39cded4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39c8071500 con 0x7f39c8159e40 2023-11-09T03:52:42.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.680+0000 7f39cded4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39c8070a70 con 0x7f39c80b0c40 2023-11-09T03:52:42.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.680+0000 7f39cded4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c815dae0 con 0x7f39c806bb60 2023-11-09T03:52:42.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.680+0000 7f39c77fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f39c8159e40 0x7f39c80b0530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45148/0 (socket says 172.21.15.76:45148) 2023-11-09T03:52:42.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.680+0000 7f39c77fe640 1 -- 172.21.15.76:0/1353010731 learned_addr learned my addr 172.21.15.76:0/1353010731 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:52:42.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.680+0000 7f39c6ffd640 1 --2- 172.21.15.76:0/1353010731 >> v2:172.21.15.113:3300/0 conn(0x7f39c80b0c40 0x7f39c806fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:42.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.680+0000 7f39c7fff640 1 --2- 172.21.15.76:0/1353010731 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39c806bb60 0x7f39c8070530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:42.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.680+0000 7f39c67fc640 1 -- 172.21.15.76:0/1353010731 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1893843626 0 0) 0x7f39c8071500 con 0x7f39c8159e40 2023-11-09T03:52:42.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.680+0000 7f39c67fc640 1 -- 172.21.15.76:0/1353010731 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39b0003520 con 0x7f39c8159e40 2023-11-09T03:52:42.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.681+0000 7f39c6ffd640 1 -- 172.21.15.76:0/1353010731 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39c806bb60 msgr2=0x7f39c8070530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:42.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.681+0000 7f39c6ffd640 1 --2- 172.21.15.76:0/1353010731 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39c806bb60 0x7f39c8070530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:42.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.681+0000 7f39c6ffd640 1 -- 172.21.15.76:0/1353010731 >> v1:172.21.15.76:6789/0 conn(0x7f39c8159e40 legacy=0x7f39c80b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:42.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.681+0000 7f39c67fc640 1 -- 172.21.15.76:0/1353010731 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3705405858 0 0) 0x7f39b0003520 con 0x7f39c8159e40 2023-11-09T03:52:42.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.681+0000 7f39c6ffd640 1 -- 172.21.15.76:0/1353010731 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c8070c60 con 0x7f39c80b0c40 2023-11-09T03:52:42.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.681+0000 7f39c7fff640 1 --2- 172.21.15.76:0/1353010731 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39c806bb60 0x7f39c8070530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:52:42.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.681+0000 7f39c6ffd640 1 --2- 172.21.15.76:0/1353010731 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39c80b0c40 0x7f39c806fff0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0097a0 tx=0x7f39bc02b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ea158a1caffdc15 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:42.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.682+0000 7f39c67fc640 1 -- 172.21.15.76:0/1353010731 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b8001670 con 0x7f39c80b0c40 2023-11-09T03:52:42.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.682+0000 7f39c67fc640 1 -- 172.21.15.76:0/1353010731 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39bc02e050 con 0x7f39c80b0c40 2023-11-09T03:52:42.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.682+0000 7f39c67fc640 1 -- 172.21.15.76:0/1353010731 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc030710 con 0x7f39c80b0c40 2023-11-09T03:52:42.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.682+0000 7f39cded4640 1 -- 172.21.15.76:0/1353010731 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39c80b0c40 msgr2=0x7f39c806fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:42.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.682+0000 7f39cded4640 1 --2- 172.21.15.76:0/1353010731 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39c80b0c40 0x7f39c806fff0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0097a0 tx=0x7f39bc02b710 comp rx=0 tx=0).stop 2023-11-09T03:52:42.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.682+0000 7f39cded4640 1 -- 172.21.15.76:0/1353010731 shutdown_connections 2023-11-09T03:52:42.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.682+0000 7f39cded4640 1 --2- 172.21.15.76:0/1353010731 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39c806bb60 0x7f39c8070530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:42.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.682+0000 7f39cded4640 1 --2- 172.21.15.76:0/1353010731 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39c80b0c40 0x7f39c806fff0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:42.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.682+0000 7f39cded4640 1 -- 172.21.15.76:0/1353010731 >> 172.21.15.76:0/1353010731 conn(0x7f39c80cb530 msgr2=0x7f39c80cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:42.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.682+0000 7f39cded4640 1 -- 172.21.15.76:0/1353010731 shutdown_connections 2023-11-09T03:52:42.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.682+0000 7f39cded4640 1 -- 172.21.15.76:0/1353010731 wait complete. 2023-11-09T03:52:42.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.683+0000 7f39cded4640 1 Processor -- start 2023-11-09T03:52:42.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.684+0000 7f39cded4640 1 -- start start 2023-11-09T03:52:42.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.685+0000 7f39cded4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39c806bb60 0x7f39c81570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:42.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.685+0000 7f39cded4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39c80b0c40 0x7f39c80c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:42.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.685+0000 7f39cded4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39c8159e40 0x7f39c80cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:42.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.685+0000 7f39cded4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c80cd600 con 0x7f39c8159e40 2023-11-09T03:52:42.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.685+0000 7f39cded4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c80c49a0 con 0x7f39c806bb60 2023-11-09T03:52:42.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.685+0000 7f39cded4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c80cd040 con 0x7f39c80b0c40 2023-11-09T03:52:42.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.685+0000 7f39c7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39c8159e40 0x7f39c80cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:42.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.685+0000 7f39c7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39c8159e40 0x7f39c80cc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38094/0 (socket says 172.21.15.76:38094) 2023-11-09T03:52:42.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.685+0000 7f39c7fff640 1 -- 172.21.15.76:0/3762644590 learned_addr learned my addr 172.21.15.76:0/3762644590 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:42.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.685+0000 7f39c77fe640 1 --2- 172.21.15.76:0/3762644590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39c806bb60 0x7f39c81570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:42.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.685+0000 7f39c6ffd640 1 --2- 172.21.15.76:0/3762644590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39c80b0c40 0x7f39c80c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:42.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.685+0000 7f39c7fff640 1 -- 172.21.15.76:0/3762644590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39c80b0c40 msgr2=0x7f39c80c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:42.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.685+0000 7f39c7fff640 1 --2- 172.21.15.76:0/3762644590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39c80b0c40 0x7f39c80c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:42.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.685+0000 7f39c7fff640 1 -- 172.21.15.76:0/3762644590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39c806bb60 msgr2=0x7f39c81570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:42.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.685+0000 7f39c7fff640 1 --2- 172.21.15.76:0/3762644590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39c806bb60 0x7f39c81570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:42.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.685+0000 7f39c7fff640 1 -- 172.21.15.76:0/3762644590 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c0009a70 con 0x7f39c8159e40 2023-11-09T03:52:42.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.685+0000 7f39c7fff640 1 --2- 172.21.15.76:0/3762644590 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39c8159e40 0x7f39c80cc660 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f39c000efc0 tx=0x7f39c0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:42.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.685+0000 7f39affff640 1 -- 172.21.15.76:0/3762644590 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c0009d70 con 0x7f39c8159e40 2023-11-09T03:52:42.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.685+0000 7f39affff640 1 -- 172.21.15.76:0/3762644590 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c000f040 con 0x7f39c8159e40 2023-11-09T03:52:42.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.685+0000 7f39affff640 1 -- 172.21.15.76:0/3762644590 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c0003990 con 0x7f39c8159e40 2023-11-09T03:52:42.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.685+0000 7f39cded4640 1 -- 172.21.15.76:0/3762644590 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39bc009400 con 0x7f39c8159e40 2023-11-09T03:52:42.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.686+0000 7f39cded4640 1 -- 172.21.15.76:0/3762644590 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c80c4630 con 0x7f39c8159e40 2023-11-09T03:52:42.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.686+0000 7f39cded4640 1 -- 172.21.15.76:0/3762644590 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39c80c4490 con 0x7f39c8159e40 2023-11-09T03:52:42.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.687+0000 7f39affff640 1 -- 172.21.15.76:0/3762644590 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f39c00144a0 con 0x7f39c8159e40 2023-11-09T03:52:42.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.687+0000 7f39affff640 1 --2- 172.21.15.76:0/3762644590 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f39b406aea0 0x7f39b406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:42.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.687+0000 7f39affff640 1 -- 172.21.15.76:0/3762644590 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 7909+0+0 (secure 0 0 0) 0x7f39c008a980 con 0x7f39c8159e40 2023-11-09T03:52:42.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.688+0000 7f39c77fe640 1 --2- 172.21.15.76:0/3762644590 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f39b406aea0 0x7f39b406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:42.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.688+0000 7f39c77fe640 1 --2- 172.21.15.76:0/3762644590 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f39b406aea0 0x7f39b406d360 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f39b8006ad0 tx=0x7f39b8001d20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:42.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.692+0000 7f39affff640 1 -- 172.21.15.76:0/3762644590 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f39c80c4490 con 0x7f39c8159e40 2023-11-09T03:52:42.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.893+0000 7f39cded4640 1 -- 172.21.15.76:0/3762644590 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39c80c4490 con 0x7f39c8159e40 2023-11-09T03:52:42.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.894+0000 7f39affff640 1 -- 172.21.15.76:0/3762644590 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4930 (secure 0 0 0) 0x7f39c80c4490 con 0x7f39c8159e40 2023-11-09T03:52:42.895 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:52:42.895 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":103,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8568,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:52:35.229057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8310,"mds_1":7144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7144":{"gid":7144,"name":"b","rank":1,"incarnation":101,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.113:6837/7360844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":7360844},{"type":"v1","addr":"172.21.15.113:6837","nonce":7360844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8310":{"gid":8310,"name":"a","rank":0,"incarnation":98,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:52:42.895 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 103 2023-11-09T03:52:42.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.895+0000 7f39cded4640 1 -- 172.21.15.76:0/3762644590 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f39b406aea0 msgr2=0x7f39b406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:42.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.895+0000 7f39cded4640 1 --2- 172.21.15.76:0/3762644590 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f39b406aea0 0x7f39b406d360 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f39b8006ad0 tx=0x7f39b8001d20 comp rx=0 tx=0).stop 2023-11-09T03:52:42.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.895+0000 7f39cded4640 1 -- 172.21.15.76:0/3762644590 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39c8159e40 msgr2=0x7f39c80cc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:42.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.895+0000 7f39cded4640 1 --2- 172.21.15.76:0/3762644590 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39c8159e40 0x7f39c80cc660 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f39c000efc0 tx=0x7f39c0009f90 comp rx=0 tx=0).stop 2023-11-09T03:52:42.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.895+0000 7f39cded4640 1 -- 172.21.15.76:0/3762644590 shutdown_connections 2023-11-09T03:52:42.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.895+0000 7f39cded4640 1 --2- 172.21.15.76:0/3762644590 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f39b406aea0 0x7f39b406d360 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:42.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.895+0000 7f39cded4640 1 --2- 172.21.15.76:0/3762644590 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39c8159e40 0x7f39c80cc660 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:42.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.895+0000 7f39cded4640 1 --2- 172.21.15.76:0/3762644590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39c80b0c40 0x7f39c80c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:42.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.895+0000 7f39cded4640 1 --2- 172.21.15.76:0/3762644590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39c806bb60 0x7f39c81570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:42.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.895+0000 7f39cded4640 1 -- 172.21.15.76:0/3762644590 >> 172.21.15.76:0/3762644590 conn(0x7f39c80cb530 msgr2=0x7f39c80ae580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:42.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.896+0000 7f39cded4640 1 -- 172.21.15.76:0/3762644590 shutdown_connections 2023-11-09T03:52:42.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:42.896+0000 7f39cded4640 1 -- 172.21.15.76:0/3762644590 wait complete. 2023-11-09T03:52:42.907 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:reconnect 2023-11-09T03:52:43.909 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:52:44.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.019+0000 7f3f96290640 1 Processor -- start 2023-11-09T03:52:44.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.019+0000 7f3f96290640 1 -- start start 2023-11-09T03:52:44.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.020+0000 7f3f96290640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f900cc680 0x7f3f900cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:44.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.020+0000 7f3f96290640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f900cd720 con 0x7f3f900addd0 2023-11-09T03:52:44.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.020+0000 7f3f96290640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f900ae1b0 con 0x7f3f9006c220 2023-11-09T03:52:44.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.020+0000 7f3f96290640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f900ccb10 con 0x7f3f900cc680 2023-11-09T03:52:44.024 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.020+0000 7f3f8ffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f3f900addd0 0x7f3f9006bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45164/0 (socket says 172.21.15.76:45164) 2023-11-09T03:52:44.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.020+0000 7f3f8ffff640 1 -- 172.21.15.76:0/1367458075 learned_addr learned my addr 172.21.15.76:0/1367458075 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:52:44.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.021+0000 7f3f8f7fe640 1 --2- 172.21.15.76:0/1367458075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f900cc680 0x7f3f900cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:44.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.021+0000 7f3f8effd640 1 -- 172.21.15.76:0/1367458075 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808814028 0 0) 0x7f3f900cd720 con 0x7f3f900addd0 2023-11-09T03:52:44.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.021+0000 7f3f8effd640 1 -- 172.21.15.76:0/1367458075 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f78003440 con 0x7f3f900addd0 2023-11-09T03:52:44.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.021+0000 7f3f8effd640 1 -- 172.21.15.76:0/1367458075 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1276547388 0 0) 0x7f3f78003440 con 0x7f3f900addd0 2023-11-09T03:52:44.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.021+0000 7f3f8effd640 1 -- 172.21.15.76:0/1367458075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f900cc680 msgr2=0x7f3f900cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:44.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.021+0000 7f3f8effd640 1 --2- 172.21.15.76:0/1367458075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f900cc680 0x7f3f900cdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:44.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.021+0000 7f3f8effd640 1 -- 172.21.15.76:0/1367458075 >> v1:172.21.15.113:6789/0 conn(0x7f3f9006c220 legacy=0x7f3f9006f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:44.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.021+0000 7f3f8effd640 1 -- 172.21.15.76:0/1367458075 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f900ccdf0 con 0x7f3f900addd0 2023-11-09T03:52:44.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.021+0000 7f3f8effd640 1 -- 172.21.15.76:0/1367458075 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f3f80001670 con 0x7f3f900addd0 2023-11-09T03:52:44.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.021+0000 7f3f8effd640 1 -- 172.21.15.76:0/1367458075 >> v1:172.21.15.76:6789/0 conn(0x7f3f900addd0 legacy=0x7f3f9006bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:44.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.021+0000 7f3f8effd640 1 --2- 172.21.15.76:0/1367458075 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f78003a90 0x7f3f78003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:44.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.021+0000 7f3f8effd640 1 --2- 172.21.15.76:0/1367458075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f78008850 0x7f3f7800ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:44.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.021+0000 7f3f8effd640 1 --2- 172.21.15.76:0/1367458075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f7800b460 0x7f3f7800d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:44.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.021+0000 7f3f8effd640 1 -- 172.21.15.76:0/1367458075 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f900ccb10 con 0x7f3f78003a90 2023-11-09T03:52:44.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.021+0000 7f3f8effd640 1 -- 172.21.15.76:0/1367458075 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7800dff0 con 0x7f3f78008850 2023-11-09T03:52:44.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.021+0000 7f3f8effd640 1 -- 172.21.15.76:0/1367458075 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7800e1b0 con 0x7f3f7800b460 2023-11-09T03:52:44.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.021+0000 7f3f8ffff640 1 --2- 172.21.15.76:0/1367458075 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f78003a90 0x7f3f78003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:44.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.021+0000 7f3f9482d640 1 --2- 172.21.15.76:0/1367458075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f7800b460 0x7f3f7800d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:44.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.021+0000 7f3f8f7fe640 1 --2- 172.21.15.76:0/1367458075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f78008850 0x7f3f7800ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:44.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.021+0000 7f3f8ffff640 1 -- 172.21.15.76:0/1367458075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f7800b460 msgr2=0x7f3f7800d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:44.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.021+0000 7f3f8ffff640 1 --2- 172.21.15.76:0/1367458075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f7800b460 0x7f3f7800d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:44.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.021+0000 7f3f8ffff640 1 -- 172.21.15.76:0/1367458075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f78008850 msgr2=0x7f3f7800ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:44.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.021+0000 7f3f8ffff640 1 --2- 172.21.15.76:0/1367458075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f78008850 0x7f3f7800ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:44.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.021+0000 7f3f8ffff640 1 -- 172.21.15.76:0/1367458075 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f78003440 con 0x7f3f78003a90 2023-11-09T03:52:44.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.021+0000 7f3f8f7fe640 1 --2- 172.21.15.76:0/1367458075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f78008850 0x7f3f7800ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:52:44.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.022+0000 7f3f8ffff640 1 --2- 172.21.15.76:0/1367458075 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f78003a90 0x7f3f78003eb0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000c190 tx=0x7f3f780036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79cd4aa3c28b4497 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:44.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.022+0000 7f3f8effd640 1 -- 172.21.15.76:0/1367458075 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f80005ad0 con 0x7f3f78003a90 2023-11-09T03:52:44.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.022+0000 7f3f8effd640 1 -- 172.21.15.76:0/1367458075 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f80005c50 con 0x7f3f78003a90 2023-11-09T03:52:44.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.022+0000 7f3f8effd640 1 -- 172.21.15.76:0/1367458075 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8002c8a0 con 0x7f3f78003a90 2023-11-09T03:52:44.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.023+0000 7f3f96290640 1 -- 172.21.15.76:0/1367458075 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f78003a90 msgr2=0x7f3f78003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:44.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.023+0000 7f3f96290640 1 --2- 172.21.15.76:0/1367458075 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f78003a90 0x7f3f78003eb0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000c190 tx=0x7f3f780036f0 comp rx=0 tx=0).stop 2023-11-09T03:52:44.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.023+0000 7f3f9482d640 1 -- 172.21.15.76:0/1367458075 reap_dead start 2023-11-09T03:52:44.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.023+0000 7f3f96290640 1 -- 172.21.15.76:0/1367458075 shutdown_connections 2023-11-09T03:52:44.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.023+0000 7f3f96290640 1 -- 172.21.15.76:0/1367458075 >> 172.21.15.76:0/1367458075 conn(0x7f3f900cb530 msgr2=0x7f3f900cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:44.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.023+0000 7f3f96290640 1 -- 172.21.15.76:0/1367458075 shutdown_connections 2023-11-09T03:52:44.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.023+0000 7f3f96290640 1 -- 172.21.15.76:0/1367458075 wait complete. 2023-11-09T03:52:44.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.024+0000 7f3f96290640 1 Processor -- start 2023-11-09T03:52:44.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.024+0000 7f3f96290640 1 -- start start 2023-11-09T03:52:44.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.025+0000 7f3f96290640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f78003a90 0x7f3f90159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:44.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.025+0000 7f3f96290640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f900c7600 0x7f3f90070be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:44.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.025+0000 7f3f96290640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f90148ac0 0x7f3f90148f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:44.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.025+0000 7f3f96290640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f900b2140 con 0x7f3f78003a90 2023-11-09T03:52:44.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.025+0000 7f3f96290640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f90149480 con 0x7f3f900c7600 2023-11-09T03:52:44.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.025+0000 7f3f96290640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9015ad80 con 0x7f3f90148ac0 2023-11-09T03:52:44.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.025+0000 7f3f8ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f78003a90 0x7f3f90159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:44.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.025+0000 7f3f8ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f78003a90 0x7f3f90159b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38120/0 (socket says 172.21.15.76:38120) 2023-11-09T03:52:44.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.025+0000 7f3f8ffff640 1 -- 172.21.15.76:0/1474866854 learned_addr learned my addr 172.21.15.76:0/1474866854 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:44.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.025+0000 7f3f8ffff640 1 -- 172.21.15.76:0/1474866854 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f90148ac0 msgr2=0x7f3f90148f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:44.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.025+0000 7f3f8ffff640 1 --2- 172.21.15.76:0/1474866854 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f90148ac0 0x7f3f90148f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:44.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.025+0000 7f3f8ffff640 1 -- 172.21.15.76:0/1474866854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f900c7600 msgr2=0x7f3f90070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:44.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.025+0000 7f3f8ffff640 1 --2- 172.21.15.76:0/1474866854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f900c7600 0x7f3f90070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:44.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.025+0000 7f3f8ffff640 1 -- 172.21.15.76:0/1474866854 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f9015a200 con 0x7f3f78003a90 2023-11-09T03:52:44.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.025+0000 7f3f8ffff640 1 --2- 172.21.15.76:0/1474866854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f78003a90 0x7f3f90159b20 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000c2b0 tx=0x7f3f900cd690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:44.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.026+0000 7f3f8cff9640 1 -- 172.21.15.76:0/1474866854 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8002ccb0 con 0x7f3f78003a90 2023-11-09T03:52:44.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.026+0000 7f3f8cff9640 1 -- 172.21.15.76:0/1474866854 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f80000f10 con 0x7f3f78003a90 2023-11-09T03:52:44.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.026+0000 7f3f96290640 1 -- 172.21.15.76:0/1474866854 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f9015a4e0 con 0x7f3f78003a90 2023-11-09T03:52:44.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.026+0000 7f3f8cff9640 1 -- 172.21.15.76:0/1474866854 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f80007290 con 0x7f3f78003a90 2023-11-09T03:52:44.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.026+0000 7f3f96290640 1 -- 172.21.15.76:0/1474866854 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f9015aaa0 con 0x7f3f78003a90 2023-11-09T03:52:44.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.027+0000 7f3f96290640 1 -- 172.21.15.76:0/1474866854 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f900684b0 con 0x7f3f78003a90 2023-11-09T03:52:44.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.027+0000 7f3f8cff9640 1 -- 172.21.15.76:0/1474866854 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3f80032070 con 0x7f3f78003a90 2023-11-09T03:52:44.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.027+0000 7f3f8cff9640 1 --2- 172.21.15.76:0/1474866854 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3f7c06aea0 0x7f3f7c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:44.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.027+0000 7f3f8cff9640 1 -- 172.21.15.76:0/1474866854 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7952+0+0 (secure 0 0 0) 0x7f3f8003c040 con 0x7f3f78003a90 2023-11-09T03:52:44.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.028+0000 7f3f8f7fe640 1 --2- 172.21.15.76:0/1474866854 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3f7c06aea0 0x7f3f7c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:44.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.028+0000 7f3f8f7fe640 1 --2- 172.21.15.76:0/1474866854 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3f7c06aea0 0x7f3f7c06d360 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3f900b4a00 tx=0x7f3f84008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:44.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.032+0000 7f3f8cff9640 1 -- 172.21.15.76:0/1474866854 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3f80075b70 con 0x7f3f78003a90 2023-11-09T03:52:44.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.233+0000 7f3f96290640 1 -- 172.21.15.76:0/1474866854 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f900b4a30 con 0x7f3f78003a90 2023-11-09T03:52:44.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.233+0000 7f3f8cff9640 1 -- 172.21.15.76:0/1474866854 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4930 (secure 0 0 0) 0x7f3f80045070 con 0x7f3f78003a90 2023-11-09T03:52:44.235 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:52:44.235 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":103,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8568,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:52:35.229057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8310,"mds_1":7144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7144":{"gid":7144,"name":"b","rank":1,"incarnation":101,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.113:6837/7360844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":7360844},{"type":"v1","addr":"172.21.15.113:6837","nonce":7360844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8310":{"gid":8310,"name":"a","rank":0,"incarnation":98,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:52:44.235 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 103 2023-11-09T03:52:44.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.236+0000 7f3f96290640 1 -- 172.21.15.76:0/1474866854 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3f7c06aea0 msgr2=0x7f3f7c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:44.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.236+0000 7f3f96290640 1 --2- 172.21.15.76:0/1474866854 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3f7c06aea0 0x7f3f7c06d360 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3f900b4a00 tx=0x7f3f84008040 comp rx=0 tx=0).stop 2023-11-09T03:52:44.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.236+0000 7f3f96290640 1 -- 172.21.15.76:0/1474866854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f78003a90 msgr2=0x7f3f90159b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:44.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.236+0000 7f3f96290640 1 --2- 172.21.15.76:0/1474866854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f78003a90 0x7f3f90159b20 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000c2b0 tx=0x7f3f900cd690 comp rx=0 tx=0).stop 2023-11-09T03:52:44.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.236+0000 7f3f96290640 1 -- 172.21.15.76:0/1474866854 shutdown_connections 2023-11-09T03:52:44.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.236+0000 7f3f96290640 1 --2- 172.21.15.76:0/1474866854 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3f7c06aea0 0x7f3f7c06d360 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:44.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.236+0000 7f3f96290640 1 --2- 172.21.15.76:0/1474866854 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f90148ac0 0x7f3f90148f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:44.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.236+0000 7f3f96290640 1 --2- 172.21.15.76:0/1474866854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f900c7600 0x7f3f90070be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:44.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.236+0000 7f3f96290640 1 --2- 172.21.15.76:0/1474866854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f78003a90 0x7f3f90159b20 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:44.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.236+0000 7f3f96290640 1 -- 172.21.15.76:0/1474866854 >> 172.21.15.76:0/1474866854 conn(0x7f3f900cb530 msgr2=0x7f3f900c98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:44.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.236+0000 7f3f96290640 1 -- 172.21.15.76:0/1474866854 shutdown_connections 2023-11-09T03:52:44.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:44.236+0000 7f3f96290640 1 -- 172.21.15.76:0/1474866854 wait complete. 2023-11-09T03:52:44.247 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:reconnect 2023-11-09T03:52:45.248 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:52:45.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.356+0000 7ff861c56640 1 Processor -- start 2023-11-09T03:52:45.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.357+0000 7ff861c56640 1 -- start start 2023-11-09T03:52:45.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.357+0000 7ff861c56640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff85c15d100 0x7ff85c0ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:45.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.357+0000 7ff861c56640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff85c15d4e0 0x7ff85c0d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:45.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.357+0000 7ff861c56640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff85c0b32b0 con 0x7ff85c0d3b60 2023-11-09T03:52:45.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.357+0000 7ff861c56640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff85c0cee10 con 0x7ff85c15d100 2023-11-09T03:52:45.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.357+0000 7ff861c56640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c0b4750 con 0x7ff85c15d4e0 2023-11-09T03:52:45.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.357+0000 7ff85b7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff85c15d100 0x7ff85c0ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:45.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.358+0000 7ff85bfff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff85c0d3b60 0x7ff85c0cf520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45166/0 (socket says 172.21.15.76:45166) 2023-11-09T03:52:45.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.358+0000 7ff85b7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff85c15d100 0x7ff85c0ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38132/0 (socket says 172.21.15.76:38132) 2023-11-09T03:52:45.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.358+0000 7ff85bfff640 1 -- 172.21.15.76:0/3293339900 learned_addr learned my addr 172.21.15.76:0/3293339900 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:52:45.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.358+0000 7ff85b7fe640 1 -- 172.21.15.76:0/3293339900 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff85c15d4e0 msgr2=0x7ff85c0d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:45.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.358+0000 7ff85b7fe640 1 --2- 172.21.15.76:0/3293339900 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff85c15d4e0 0x7ff85c0d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:45.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.358+0000 7ff85b7fe640 1 -- 172.21.15.76:0/3293339900 >> v1:172.21.15.76:6789/0 conn(0x7ff85c0d3b60 legacy=0x7ff85c0cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:45.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.358+0000 7ff85affd640 1 -- 172.21.15.76:0/3293339900 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2785219950 0 0) 0x7ff85c0b32b0 con 0x7ff85c0d3b60 2023-11-09T03:52:45.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.358+0000 7ff85b7fe640 1 -- 172.21.15.76:0/3293339900 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff85c0b2980 con 0x7ff85c15d100 2023-11-09T03:52:45.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.359+0000 7ff85b7fe640 1 --2- 172.21.15.76:0/3293339900 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff85c15d100 0x7ff85c0ce510 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7ff84c009790 tx=0x7ff84c02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5e201713a2b19d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:45.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.359+0000 7ff85affd640 1 -- 172.21.15.76:0/3293339900 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c031800 con 0x7ff85c15d100 2023-11-09T03:52:45.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.359+0000 7ff85affd640 1 -- 172.21.15.76:0/3293339900 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff84c031960 con 0x7ff85c15d100 2023-11-09T03:52:45.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.359+0000 7ff85affd640 1 -- 172.21.15.76:0/3293339900 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c031cf0 con 0x7ff85c15d100 2023-11-09T03:52:45.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.359+0000 7ff861c56640 1 -- 172.21.15.76:0/3293339900 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff85c15d100 msgr2=0x7ff85c0ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:45.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.359+0000 7ff861c56640 1 --2- 172.21.15.76:0/3293339900 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff85c15d100 0x7ff85c0ce510 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7ff84c009790 tx=0x7ff84c02b8a0 comp rx=0 tx=0).stop 2023-11-09T03:52:45.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.359+0000 7ff861c56640 1 -- 172.21.15.76:0/3293339900 shutdown_connections 2023-11-09T03:52:45.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.359+0000 7ff861c56640 1 --2- 172.21.15.76:0/3293339900 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff85c15d4e0 0x7ff85c0d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:45.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.359+0000 7ff861c56640 1 --2- 172.21.15.76:0/3293339900 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff85c15d100 0x7ff85c0ce510 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:45.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.359+0000 7ff861c56640 1 -- 172.21.15.76:0/3293339900 >> 172.21.15.76:0/3293339900 conn(0x7ff85c0d2ef0 msgr2=0x7ff85c0d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:45.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.359+0000 7ff861c56640 1 -- 172.21.15.76:0/3293339900 shutdown_connections 2023-11-09T03:52:45.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.359+0000 7ff861c56640 1 -- 172.21.15.76:0/3293339900 wait complete. 2023-11-09T03:52:45.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.361+0000 7ff861c56640 1 Processor -- start 2023-11-09T03:52:45.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.361+0000 7ff861c56640 1 -- start start 2023-11-09T03:52:45.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.361+0000 7ff861c56640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff85c0d3b60 0x7ff85c1592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:45.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.361+0000 7ff861c56640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff85c15d100 0x7ff85c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:45.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.361+0000 7ff861c56640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff85c15d4e0 0x7ff85c0b8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:45.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.361+0000 7ff861c56640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c0b6db0 con 0x7ff85c0d3b60 2023-11-09T03:52:45.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.361+0000 7ff861c56640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c0cae70 con 0x7ff85c15d4e0 2023-11-09T03:52:45.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.361+0000 7ff861c56640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c0b67f0 con 0x7ff85c15d100 2023-11-09T03:52:45.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.362+0000 7ff85b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff85c0d3b60 0x7ff85c1592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:45.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.362+0000 7ff85b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff85c0d3b60 0x7ff85c1592a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38144/0 (socket says 172.21.15.76:38144) 2023-11-09T03:52:45.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.362+0000 7ff85b7fe640 1 -- 172.21.15.76:0/1939561444 learned_addr learned my addr 172.21.15.76:0/1939561444 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:45.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.362+0000 7ff85bfff640 1 --2- 172.21.15.76:0/1939561444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff85c15d4e0 0x7ff85c0b8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:45.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.362+0000 7ff852ffd640 1 --2- 172.21.15.76:0/1939561444 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff85c15d100 0x7ff85c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:45.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.362+0000 7ff85b7fe640 1 -- 172.21.15.76:0/1939561444 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff85c15d100 msgr2=0x7ff85c0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:45.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.362+0000 7ff85b7fe640 1 --2- 172.21.15.76:0/1939561444 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff85c15d100 0x7ff85c0b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:45.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.362+0000 7ff85b7fe640 1 -- 172.21.15.76:0/1939561444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff85c15d4e0 msgr2=0x7ff85c0b8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:45.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.362+0000 7ff85b7fe640 1 --2- 172.21.15.76:0/1939561444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff85c15d4e0 0x7ff85c0b8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:45.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.362+0000 7ff85b7fe640 1 -- 172.21.15.76:0/1939561444 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff84c0093a0 con 0x7ff85c0d3b60 2023-11-09T03:52:45.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.362+0000 7ff85b7fe640 1 --2- 172.21.15.76:0/1939561444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff85c0d3b60 0x7ff85c1592a0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7ff84c034040 tx=0x7ff84c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:45.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.362+0000 7ff858ff9640 1 -- 172.21.15.76:0/1939561444 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c005e60 con 0x7ff85c0d3b60 2023-11-09T03:52:45.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.362+0000 7ff858ff9640 1 -- 172.21.15.76:0/1939561444 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff84c03a8a0 con 0x7ff85c0d3b60 2023-11-09T03:52:45.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.362+0000 7ff858ff9640 1 -- 172.21.15.76:0/1939561444 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c03ac50 con 0x7ff85c0d3b60 2023-11-09T03:52:45.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.362+0000 7ff861c56640 1 -- 172.21.15.76:0/1939561444 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff85c0b6ad0 con 0x7ff85c0d3b60 2023-11-09T03:52:45.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.362+0000 7ff861c56640 1 -- 172.21.15.76:0/1939561444 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff85c15edd0 con 0x7ff85c0d3b60 2023-11-09T03:52:45.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.363+0000 7ff861c56640 1 -- 172.21.15.76:0/1939561444 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff85c15e950 con 0x7ff85c0d3b60 2023-11-09T03:52:45.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.364+0000 7ff858ff9640 1 -- 172.21.15.76:0/1939561444 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff84c006030 con 0x7ff85c0d3b60 2023-11-09T03:52:45.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.364+0000 7ff858ff9640 1 --2- 172.21.15.76:0/1939561444 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff83c06ae70 0x7ff83c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:45.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.364+0000 7ff858ff9640 1 -- 172.21.15.76:0/1939561444 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7952+0+0 (secure 0 0 0) 0x7ff84c02e070 con 0x7ff85c0d3b60 2023-11-09T03:52:45.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.364+0000 7ff852ffd640 1 --2- 172.21.15.76:0/1939561444 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff83c06ae70 0x7ff83c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:45.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.365+0000 7ff852ffd640 1 --2- 172.21.15.76:0/1939561444 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff83c06ae70 0x7ff83c06d330 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff85c15b9a0 tx=0x7ff844008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:45.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.368+0000 7ff858ff9640 1 -- 172.21.15.76:0/1939561444 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff85c15e950 con 0x7ff85c0d3b60 2023-11-09T03:52:45.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.576+0000 7ff861c56640 1 -- 172.21.15.76:0/1939561444 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff85c15e950 con 0x7ff85c0d3b60 2023-11-09T03:52:45.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.576+0000 7ff858ff9640 1 -- 172.21.15.76:0/1939561444 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4924 (secure 0 0 0) 0x7ff85c15e950 con 0x7ff85c0d3b60 2023-11-09T03:52:45.577 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:52:45.578 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":105,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8568,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:52:44.483938+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8310,"mds_1":7144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7144":{"gid":7144,"name":"b","rank":1,"incarnation":101,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/7360844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":7360844},{"type":"v1","addr":"172.21.15.113:6837","nonce":7360844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8310":{"gid":8310,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":19,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:52:45.579 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 105 2023-11-09T03:52:45.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.578+0000 7ff861c56640 1 -- 172.21.15.76:0/1939561444 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff83c06ae70 msgr2=0x7ff83c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:45.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.578+0000 7ff861c56640 1 --2- 172.21.15.76:0/1939561444 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff83c06ae70 0x7ff83c06d330 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff85c15b9a0 tx=0x7ff844008040 comp rx=0 tx=0).stop 2023-11-09T03:52:45.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.578+0000 7ff861c56640 1 -- 172.21.15.76:0/1939561444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff85c0d3b60 msgr2=0x7ff85c1592a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:45.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.578+0000 7ff861c56640 1 --2- 172.21.15.76:0/1939561444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff85c0d3b60 0x7ff85c1592a0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7ff84c034040 tx=0x7ff84c0046c0 comp rx=0 tx=0).stop 2023-11-09T03:52:45.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.578+0000 7ff861c56640 1 -- 172.21.15.76:0/1939561444 shutdown_connections 2023-11-09T03:52:45.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.578+0000 7ff861c56640 1 --2- 172.21.15.76:0/1939561444 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff83c06ae70 0x7ff83c06d330 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:45.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.578+0000 7ff861c56640 1 --2- 172.21.15.76:0/1939561444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff85c15d4e0 0x7ff85c0b8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:45.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.578+0000 7ff861c56640 1 --2- 172.21.15.76:0/1939561444 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff85c15d100 0x7ff85c0b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:45.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.578+0000 7ff861c56640 1 --2- 172.21.15.76:0/1939561444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff85c0d3b60 0x7ff85c1592a0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:45.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.578+0000 7ff861c56640 1 -- 172.21.15.76:0/1939561444 >> 172.21.15.76:0/1939561444 conn(0x7ff85c0d2ef0 msgr2=0x7ff85c06bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:45.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.578+0000 7ff861c56640 1 -- 172.21.15.76:0/1939561444 shutdown_connections 2023-11-09T03:52:45.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.578+0000 7ff861c56640 1 -- 172.21.15.76:0/1939561444 wait complete. 2023-11-09T03:52:45.590 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:active 2023-11-09T03:52:45.590 DEBUG:tasks.cephfs.filesystem:reached state 'up:active' in 5.690814018249512s 2023-11-09T03:52:45.591 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:52:45.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.691+0000 7f295dd4c640 1 Processor -- start 2023-11-09T03:52:45.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.691+0000 7f295dd4c640 1 -- start start 2023-11-09T03:52:45.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.692+0000 7f295dd4c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2958159e40 0x7f29580b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:45.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.692+0000 7f295dd4c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f295806bb60 0x7f29580b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:45.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.692+0000 7f295dd4c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2958071500 con 0x7f295806fff0 2023-11-09T03:52:45.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.692+0000 7f295dd4c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f295815dae0 con 0x7f295806bb60 2023-11-09T03:52:45.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.692+0000 7f295dd4c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2958070c60 con 0x7f2958159e40 2023-11-09T03:52:45.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.692+0000 7f2956ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f295806bb60 0x7f29580b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:45.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.692+0000 7f2956ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f295806bb60 0x7f29580b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38152/0 (socket says 172.21.15.76:38152) 2023-11-09T03:52:45.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.692+0000 7f2956ffd640 1 -- 172.21.15.76:0/4022838906 learned_addr learned my addr 172.21.15.76:0/4022838906 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:45.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.692+0000 7f29577fe640 1 --2- 172.21.15.76:0/4022838906 >> v2:172.21.15.113:3300/0 conn(0x7f2958159e40 0x7f29580b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:45.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.692+0000 7f2956ffd640 1 -- 172.21.15.76:0/4022838906 >> v2:172.21.15.113:3300/0 conn(0x7f2958159e40 msgr2=0x7f29580b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:45.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.692+0000 7f2956ffd640 1 --2- 172.21.15.76:0/4022838906 >> v2:172.21.15.113:3300/0 conn(0x7f2958159e40 0x7f29580b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:45.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.692+0000 7f2956ffd640 1 -- 172.21.15.76:0/4022838906 >> v1:172.21.15.113:6789/0 conn(0x7f295806fff0 legacy=0x7f2958070420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:45.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.692+0000 7f2956ffd640 1 -- 172.21.15.76:0/4022838906 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2958070f40 con 0x7f295806bb60 2023-11-09T03:52:45.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.693+0000 7f2956ffd640 1 --2- 172.21.15.76:0/4022838906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f295806bb60 0x7f29580b0a70 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f29580ce7f0 tx=0x7f294c02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59aad2afd5325973 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:45.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.693+0000 7f29567fc640 1 -- 172.21.15.76:0/4022838906 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c031750 con 0x7f295806bb60 2023-11-09T03:52:45.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.693+0000 7f29567fc640 1 -- 172.21.15.76:0/4022838906 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294c0318b0 con 0x7f295806bb60 2023-11-09T03:52:45.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.693+0000 7f29567fc640 1 -- 172.21.15.76:0/4022838906 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c031c40 con 0x7f295806bb60 2023-11-09T03:52:45.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.694+0000 7f295dd4c640 1 -- 172.21.15.76:0/4022838906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f295806bb60 msgr2=0x7f29580b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:45.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.694+0000 7f295dd4c640 1 --2- 172.21.15.76:0/4022838906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f295806bb60 0x7f29580b0a70 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f29580ce7f0 tx=0x7f294c02b7f0 comp rx=0 tx=0).stop 2023-11-09T03:52:45.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.694+0000 7f295dd4c640 1 -- 172.21.15.76:0/4022838906 shutdown_connections 2023-11-09T03:52:45.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.694+0000 7f295dd4c640 1 --2- 172.21.15.76:0/4022838906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f295806bb60 0x7f29580b0a70 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:45.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.694+0000 7f295dd4c640 1 --2- 172.21.15.76:0/4022838906 >> v2:172.21.15.113:3300/0 conn(0x7f2958159e40 0x7f29580b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:45.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.694+0000 7f295dd4c640 1 -- 172.21.15.76:0/4022838906 >> 172.21.15.76:0/4022838906 conn(0x7f29580cb530 msgr2=0x7f29580cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:45.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.694+0000 7f295dd4c640 1 -- 172.21.15.76:0/4022838906 shutdown_connections 2023-11-09T03:52:45.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.694+0000 7f295dd4c640 1 -- 172.21.15.76:0/4022838906 wait complete. 2023-11-09T03:52:45.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.695+0000 7f295dd4c640 1 Processor -- start 2023-11-09T03:52:45.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.695+0000 7f295dd4c640 1 -- start start 2023-11-09T03:52:45.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.696+0000 7f295dd4c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f295806bb60 0x7f295813fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:45.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.696+0000 7f295dd4c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f295806fff0 0x7f29580c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:45.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.696+0000 7f295dd4c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2958159e40 0x7f29580c4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:45.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.696+0000 7f295dd4c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29580c1840 con 0x7f295806bb60 2023-11-09T03:52:45.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.696+0000 7f295dd4c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29581579d0 con 0x7f2958159e40 2023-11-09T03:52:45.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.696+0000 7f295dd4c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29580c4140 con 0x7f295806fff0 2023-11-09T03:52:45.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.696+0000 7f29577fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f295806bb60 0x7f295813fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:45.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.696+0000 7f29577fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f295806bb60 0x7f295813fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38162/0 (socket says 172.21.15.76:38162) 2023-11-09T03:52:45.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.696+0000 7f29577fe640 1 -- 172.21.15.76:0/896805779 learned_addr learned my addr 172.21.15.76:0/896805779 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:45.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.696+0000 7f2957fff640 1 --2- 172.21.15.76:0/896805779 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2958159e40 0x7f29580c4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:45.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.696+0000 7f2956ffd640 1 --2- 172.21.15.76:0/896805779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f295806fff0 0x7f29580c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:45.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.696+0000 7f29577fe640 1 -- 172.21.15.76:0/896805779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f295806fff0 msgr2=0x7f29580c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:45.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.696+0000 7f29577fe640 1 --2- 172.21.15.76:0/896805779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f295806fff0 0x7f29580c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:45.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.696+0000 7f29577fe640 1 -- 172.21.15.76:0/896805779 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2958159e40 msgr2=0x7f29580c4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:45.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.696+0000 7f29577fe640 1 --2- 172.21.15.76:0/896805779 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2958159e40 0x7f29580c4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:45.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.696+0000 7f29577fe640 1 -- 172.21.15.76:0/896805779 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c0094d0 con 0x7f295806bb60 2023-11-09T03:52:45.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.696+0000 7f2957fff640 1 --2- 172.21.15.76:0/896805779 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2958159e40 0x7f29580c4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:52:45.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.696+0000 7f2956ffd640 1 --2- 172.21.15.76:0/896805779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f295806fff0 0x7f29580c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:52:45.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.696+0000 7f29577fe640 1 --2- 172.21.15.76:0/896805779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f295806bb60 0x7f295813fd80 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f294800d5f0 tx=0x7f294800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:45.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.696+0000 7f293ffff640 1 -- 172.21.15.76:0/896805779 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29480125f0 con 0x7f295806bb60 2023-11-09T03:52:45.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.696+0000 7f293ffff640 1 -- 172.21.15.76:0/896805779 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2948012750 con 0x7f295806bb60 2023-11-09T03:52:45.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.697+0000 7f295dd4c640 1 -- 172.21.15.76:0/896805779 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29580c4420 con 0x7f295806bb60 2023-11-09T03:52:45.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.697+0000 7f293ffff640 1 -- 172.21.15.76:0/896805779 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948012b00 con 0x7f295806bb60 2023-11-09T03:52:45.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.697+0000 7f295dd4c640 1 -- 172.21.15.76:0/896805779 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29580c85c0 con 0x7f295806bb60 2023-11-09T03:52:45.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.698+0000 7f295dd4c640 1 -- 172.21.15.76:0/896805779 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29580684b0 con 0x7f295806bb60 2023-11-09T03:52:45.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.698+0000 7f293ffff640 1 -- 172.21.15.76:0/896805779 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2948012c60 con 0x7f295806bb60 2023-11-09T03:52:45.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.698+0000 7f293ffff640 1 --2- 172.21.15.76:0/896805779 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f294406ae70 0x7f294406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:45.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.698+0000 7f293ffff640 1 -- 172.21.15.76:0/896805779 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7952+0+0 (secure 0 0 0) 0x7f294808a3a0 con 0x7f295806bb60 2023-11-09T03:52:45.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.698+0000 7f2956ffd640 1 --2- 172.21.15.76:0/896805779 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f294406ae70 0x7f294406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:45.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.699+0000 7f2956ffd640 1 --2- 172.21.15.76:0/896805779 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f294406ae70 0x7f294406d330 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f294c035000 tx=0x7f294c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:45.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.703+0000 7f293ffff640 1 -- 172.21.15.76:0/896805779 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2948054b90 con 0x7f295806bb60 2023-11-09T03:52:45.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.906+0000 7f295dd4c640 1 -- 172.21.15.76:0/896805779 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29580c9500 con 0x7f295806bb60 2023-11-09T03:52:45.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.906+0000 7f293ffff640 1 -- 172.21.15.76:0/896805779 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4924 (secure 0 0 0) 0x7f29480581b0 con 0x7f295806bb60 2023-11-09T03:52:45.908 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:52:45.908 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":105,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8568,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:52:44.483938+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8310,"mds_1":7144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7144":{"gid":7144,"name":"b","rank":1,"incarnation":101,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/7360844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":7360844},{"type":"v1","addr":"172.21.15.113:6837","nonce":7360844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8310":{"gid":8310,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":19,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:52:45.910 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 105 2023-11-09T03:52:45.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.908+0000 7f295dd4c640 1 -- 172.21.15.76:0/896805779 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f294406ae70 msgr2=0x7f294406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:45.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.908+0000 7f295dd4c640 1 --2- 172.21.15.76:0/896805779 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f294406ae70 0x7f294406d330 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f294c035000 tx=0x7f294c008030 comp rx=0 tx=0).stop 2023-11-09T03:52:45.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.908+0000 7f295dd4c640 1 -- 172.21.15.76:0/896805779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f295806bb60 msgr2=0x7f295813fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:45.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.908+0000 7f295dd4c640 1 --2- 172.21.15.76:0/896805779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f295806bb60 0x7f295813fd80 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f294800d5f0 tx=0x7f294800dab0 comp rx=0 tx=0).stop 2023-11-09T03:52:45.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.908+0000 7f295dd4c640 1 -- 172.21.15.76:0/896805779 shutdown_connections 2023-11-09T03:52:45.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.908+0000 7f295dd4c640 1 --2- 172.21.15.76:0/896805779 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f294406ae70 0x7f294406d330 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:45.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.908+0000 7f295dd4c640 1 --2- 172.21.15.76:0/896805779 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2958159e40 0x7f29580c4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:45.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.908+0000 7f295dd4c640 1 --2- 172.21.15.76:0/896805779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f295806fff0 0x7f29580c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:45.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.908+0000 7f295dd4c640 1 --2- 172.21.15.76:0/896805779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f295806bb60 0x7f295813fd80 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:45.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.908+0000 7f295dd4c640 1 -- 172.21.15.76:0/896805779 >> 172.21.15.76:0/896805779 conn(0x7f29580cb530 msgr2=0x7f29580cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:45.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.908+0000 7f295dd4c640 1 -- 172.21.15.76:0/896805779 shutdown_connections 2023-11-09T03:52:45.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:45.908+0000 7f295dd4c640 1 -- 172.21.15.76:0/896805779 wait complete. 2023-11-09T03:52:45.920 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump snaps --server 2023-11-09T03:52:46.035 INFO:teuthology.orchestra.run.smithi076.stdout:{"last_snap":11,"last_created":11,"last_destroyed":9,"pending_noop":[],"snaps":[{"snapid":11,"ino":1099511627777,"stamp":"2023-11-09T03:52:04.689262+0000","name":"s25","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-09T03:52:46.035 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 11, "last_destroyed": 9, "last_snap": 11, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s25", "snapid": 11, "stamp": "2023-11-09T03:52:04.689262+0000" } ] } 2023-11-09T03:52:46.036 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-09T03:52:46.036 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:52:46.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.136+0000 7fe316f68640 1 Processor -- start 2023-11-09T03:52:46.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.136+0000 7fe316f68640 1 -- start start 2023-11-09T03:52:46.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.137+0000 7fe316f68640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3100ade00 0x7fe3100b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:46.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.137+0000 7fe316f68640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe31006d9c0 0x7fe3100710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:46.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.137+0000 7fe316f68640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe31006c5f0 0x7fe310071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:46.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.137+0000 7fe316f68640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe31006cc90 con 0x7fe31006d9c0 2023-11-09T03:52:46.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.137+0000 7fe316f68640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe310071b50 con 0x7fe31006c5f0 2023-11-09T03:52:46.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.137+0000 7fe316f68640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3100b0e60 con 0x7fe3100ade00 2023-11-09T03:52:46.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.137+0000 7fe30ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe31006d9c0 0x7fe3100710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:46.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.137+0000 7fe30ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe31006d9c0 0x7fe3100710d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38172/0 (socket says 172.21.15.76:38172) 2023-11-09T03:52:46.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.137+0000 7fe30ffff640 1 -- 172.21.15.76:0/3012227970 learned_addr learned my addr 172.21.15.76:0/3012227970 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:46.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.137+0000 7fe315505640 1 --2- 172.21.15.76:0/3012227970 >> v2:172.21.15.113:3300/0 conn(0x7fe31006c5f0 0x7fe310071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:46.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.137+0000 7fe314d04640 1 --2- 172.21.15.76:0/3012227970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3100ade00 0x7fe3100b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:46.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.137+0000 7fe30ffff640 1 -- 172.21.15.76:0/3012227970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3100ade00 msgr2=0x7fe3100b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:46.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.137+0000 7fe30ffff640 1 --2- 172.21.15.76:0/3012227970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3100ade00 0x7fe3100b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:46.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.137+0000 7fe30ffff640 1 -- 172.21.15.76:0/3012227970 >> v2:172.21.15.113:3300/0 conn(0x7fe31006c5f0 msgr2=0x7fe310071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:46.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.137+0000 7fe30ffff640 1 --2- 172.21.15.76:0/3012227970 >> v2:172.21.15.113:3300/0 conn(0x7fe31006c5f0 0x7fe310071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:46.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.137+0000 7fe30ffff640 1 -- 172.21.15.76:0/3012227970 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31006cee0 con 0x7fe31006d9c0 2023-11-09T03:52:46.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.137+0000 7fe314d04640 1 --2- 172.21.15.76:0/3012227970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3100ade00 0x7fe3100b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:52:46.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.138+0000 7fe30ffff640 1 --2- 172.21.15.76:0/3012227970 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe31006d9c0 0x7fe3100710d0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fe3100c5b10 tx=0x7fe30402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276efba0b0d889a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:46.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.138+0000 7fe30f7fe640 1 -- 172.21.15.76:0/3012227970 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304031750 con 0x7fe31006d9c0 2023-11-09T03:52:46.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.138+0000 7fe30f7fe640 1 -- 172.21.15.76:0/3012227970 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3040318b0 con 0x7fe31006d9c0 2023-11-09T03:52:46.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.138+0000 7fe30f7fe640 1 -- 172.21.15.76:0/3012227970 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304031c40 con 0x7fe31006d9c0 2023-11-09T03:52:46.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.139+0000 7fe316f68640 1 -- 172.21.15.76:0/3012227970 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe31006d9c0 msgr2=0x7fe3100710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:46.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.139+0000 7fe316f68640 1 --2- 172.21.15.76:0/3012227970 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe31006d9c0 0x7fe3100710d0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fe3100c5b10 tx=0x7fe30402b7f0 comp rx=0 tx=0).stop 2023-11-09T03:52:46.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.139+0000 7fe316f68640 1 -- 172.21.15.76:0/3012227970 shutdown_connections 2023-11-09T03:52:46.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.139+0000 7fe316f68640 1 --2- 172.21.15.76:0/3012227970 >> v2:172.21.15.113:3300/0 conn(0x7fe31006c5f0 0x7fe310071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:46.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.139+0000 7fe316f68640 1 --2- 172.21.15.76:0/3012227970 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe31006d9c0 0x7fe3100710d0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:46.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.139+0000 7fe316f68640 1 --2- 172.21.15.76:0/3012227970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3100ade00 0x7fe3100b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:46.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.139+0000 7fe316f68640 1 -- 172.21.15.76:0/3012227970 >> 172.21.15.76:0/3012227970 conn(0x7fe3100cb530 msgr2=0x7fe3100cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:46.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.139+0000 7fe316f68640 1 -- 172.21.15.76:0/3012227970 shutdown_connections 2023-11-09T03:52:46.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.139+0000 7fe316f68640 1 -- 172.21.15.76:0/3012227970 wait complete. 2023-11-09T03:52:46.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.140+0000 7fe316f68640 1 Processor -- start 2023-11-09T03:52:46.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.140+0000 7fe316f68640 1 -- start start 2023-11-09T03:52:46.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.141+0000 7fe316f68640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe31006c5f0 0x7fe310151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:46.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.141+0000 7fe316f68640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe31006d9c0 0x7fe3100a93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:46.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.141+0000 7fe316f68640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3100ade00 0x7fe3100c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:46.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.141+0000 7fe316f68640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3100aafa0 con 0x7fe31006c5f0 2023-11-09T03:52:46.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.141+0000 7fe316f68640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3100b37c0 con 0x7fe3100ade00 2023-11-09T03:52:46.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.141+0000 7fe316f68640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3100b3aa0 con 0x7fe31006d9c0 2023-11-09T03:52:46.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.141+0000 7fe314d04640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe31006c5f0 0x7fe310151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:46.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.141+0000 7fe314d04640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe31006c5f0 0x7fe310151ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38176/0 (socket says 172.21.15.76:38176) 2023-11-09T03:52:46.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.141+0000 7fe314d04640 1 -- 172.21.15.76:0/565022178 learned_addr learned my addr 172.21.15.76:0/565022178 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:46.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.141+0000 7fe314d04640 1 -- 172.21.15.76:0/565022178 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe31006d9c0 msgr2=0x7fe3100a93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:46.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.141+0000 7fe314d04640 1 --2- 172.21.15.76:0/565022178 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe31006d9c0 0x7fe3100a93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:46.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.141+0000 7fe314d04640 1 -- 172.21.15.76:0/565022178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3100ade00 msgr2=0x7fe3100c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:46.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.141+0000 7fe314d04640 1 --2- 172.21.15.76:0/565022178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3100ade00 0x7fe3100c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:46.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.141+0000 7fe314d04640 1 -- 172.21.15.76:0/565022178 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3040094d0 con 0x7fe31006c5f0 2023-11-09T03:52:46.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.141+0000 7fe314d04640 1 --2- 172.21.15.76:0/565022178 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe31006c5f0 0x7fe310151ef0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fe30000c7f0 tx=0x7fe30000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:46.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.142+0000 7fe30d7fa640 1 -- 172.21.15.76:0/565022178 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe300007820 con 0x7fe31006c5f0 2023-11-09T03:52:46.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.142+0000 7fe30d7fa640 1 -- 172.21.15.76:0/565022178 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe300007980 con 0x7fe31006c5f0 2023-11-09T03:52:46.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.142+0000 7fe316f68640 1 -- 172.21.15.76:0/565022178 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3100b3d80 con 0x7fe31006c5f0 2023-11-09T03:52:46.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.142+0000 7fe30d7fa640 1 -- 172.21.15.76:0/565022178 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe300007cf0 con 0x7fe31006c5f0 2023-11-09T03:52:46.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.142+0000 7fe316f68640 1 -- 172.21.15.76:0/565022178 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe310152430 con 0x7fe31006c5f0 2023-11-09T03:52:46.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.143+0000 7fe316f68640 1 -- 172.21.15.76:0/565022178 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3100684b0 con 0x7fe31006c5f0 2023-11-09T03:52:46.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.143+0000 7fe30d7fa640 1 -- 172.21.15.76:0/565022178 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe3000113f0 con 0x7fe31006c5f0 2023-11-09T03:52:46.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.143+0000 7fe30d7fa640 1 --2- 172.21.15.76:0/565022178 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe2fc06ae70 0x7fe2fc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:46.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.143+0000 7fe30d7fa640 1 -- 172.21.15.76:0/565022178 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7952+0+0 (secure 0 0 0) 0x7fe30008a1b0 con 0x7fe31006c5f0 2023-11-09T03:52:46.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.144+0000 7fe30ffff640 1 --2- 172.21.15.76:0/565022178 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe2fc06ae70 0x7fe2fc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:46.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.144+0000 7fe30ffff640 1 --2- 172.21.15.76:0/565022178 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe2fc06ae70 0x7fe2fc06d330 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe304035000 tx=0x7fe304030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:46.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.148+0000 7fe30d7fa640 1 -- 172.21.15.76:0/565022178 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe3000549a0 con 0x7fe31006c5f0 2023-11-09T03:52:46.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.348+0000 7fe316f68640 1 -- 172.21.15.76:0/565022178 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3100cddc0 con 0x7fe31006c5f0 2023-11-09T03:52:46.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.348+0000 7fe30d7fa640 1 -- 172.21.15.76:0/565022178 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4924 (secure 0 0 0) 0x7fe30001f090 con 0x7fe31006c5f0 2023-11-09T03:52:46.349 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:52:46.349 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":105,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8568,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:52:44.483938+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8310,"mds_1":7144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7144":{"gid":7144,"name":"b","rank":1,"incarnation":101,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/7360844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":7360844},{"type":"v1","addr":"172.21.15.113:6837","nonce":7360844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8310":{"gid":8310,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":19,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:52:46.350 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 105 2023-11-09T03:52:46.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.350+0000 7fe316f68640 1 -- 172.21.15.76:0/565022178 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe2fc06ae70 msgr2=0x7fe2fc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:46.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.350+0000 7fe316f68640 1 --2- 172.21.15.76:0/565022178 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe2fc06ae70 0x7fe2fc06d330 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe304035000 tx=0x7fe304030000 comp rx=0 tx=0).stop 2023-11-09T03:52:46.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.350+0000 7fe316f68640 1 -- 172.21.15.76:0/565022178 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe31006c5f0 msgr2=0x7fe310151ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:46.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.350+0000 7fe316f68640 1 --2- 172.21.15.76:0/565022178 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe31006c5f0 0x7fe310151ef0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fe30000c7f0 tx=0x7fe30000ccb0 comp rx=0 tx=0).stop 2023-11-09T03:52:46.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.350+0000 7fe316f68640 1 -- 172.21.15.76:0/565022178 shutdown_connections 2023-11-09T03:52:46.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.350+0000 7fe316f68640 1 --2- 172.21.15.76:0/565022178 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe2fc06ae70 0x7fe2fc06d330 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:46.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.350+0000 7fe316f68640 1 --2- 172.21.15.76:0/565022178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3100ade00 0x7fe3100c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:46.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.350+0000 7fe316f68640 1 --2- 172.21.15.76:0/565022178 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe31006d9c0 0x7fe3100a93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:46.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.350+0000 7fe316f68640 1 --2- 172.21.15.76:0/565022178 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe31006c5f0 0x7fe310151ef0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:46.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.351+0000 7fe316f68640 1 -- 172.21.15.76:0/565022178 >> 172.21.15.76:0/565022178 conn(0x7fe3100cb530 msgr2=0x7fe3100c8410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:46.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.351+0000 7fe316f68640 1 -- 172.21.15.76:0/565022178 shutdown_connections 2023-11-09T03:52:46.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.351+0000 7fe316f68640 1 -- 172.21.15.76:0/565022178 wait complete. 2023-11-09T03:52:46.362 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump snaps --server 2023-11-09T03:52:46.475 INFO:teuthology.orchestra.run.smithi076.stdout:{"last_snap":11,"last_created":11,"last_destroyed":9,"pending_noop":[],"snaps":[{"snapid":11,"ino":1099511627777,"stamp":"2023-11-09T03:52:04.689262+0000","name":"s25","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-09T03:52:46.476 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 11, "last_destroyed": 9, "last_snap": 11, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s25", "snapid": 11, "stamp": "2023-11-09T03:52:04.689262+0000" } ] } 2023-11-09T03:52:46.476 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:52:46.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:52:46.477 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2023-11-09T03:52:46.505 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:52:46.505 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:52:46.561 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:52:46.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.701+0000 7f0338e07640 1 Processor -- start 2023-11-09T03:52:46.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.701+0000 7f0338e07640 1 -- start start 2023-11-09T03:52:46.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.702+0000 7f0338e07640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f03340ade00 0x7f03340b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:46.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.702+0000 7f0338e07640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f033406d9c0 0x7f03340710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:46.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.702+0000 7f0338e07640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f033406cc90 con 0x7f033406c5f0 2023-11-09T03:52:46.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.702+0000 7f0338e07640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03340b0e60 con 0x7f03340ade00 2023-11-09T03:52:46.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.702+0000 7f0338e07640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f033406cee0 con 0x7f033406d9c0 2023-11-09T03:52:46.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.702+0000 7f033259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f03340ade00 0x7f03340b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:46.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.702+0000 7f033259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f03340ade00 0x7f03340b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38178/0 (socket says 172.21.15.76:38178) 2023-11-09T03:52:46.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.702+0000 7f033259c640 1 -- 172.21.15.76:0/3095641851 learned_addr learned my addr 172.21.15.76:0/3095641851 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:46.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.702+0000 7f032bfff640 1 --2- 172.21.15.76:0/3095641851 >> v2:172.21.15.113:3300/0 conn(0x7f033406d9c0 0x7f03340710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:46.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.702+0000 7f033259c640 1 -- 172.21.15.76:0/3095641851 >> v2:172.21.15.113:3300/0 conn(0x7f033406d9c0 msgr2=0x7f03340710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:46.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.702+0000 7f033259c640 1 --2- 172.21.15.76:0/3095641851 >> v2:172.21.15.113:3300/0 conn(0x7f033406d9c0 0x7f03340710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:46.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.702+0000 7f033259c640 1 -- 172.21.15.76:0/3095641851 >> v1:172.21.15.113:6789/0 conn(0x7f033406c5f0 legacy=0x7f033415e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:46.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.702+0000 7f033259c640 1 -- 172.21.15.76:0/3095641851 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033406d1c0 con 0x7f03340ade00 2023-11-09T03:52:46.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.703+0000 7f033259c640 1 --2- 172.21.15.76:0/3095641851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03340ade00 0x7f03340b0530 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f03340c9080 tx=0x7f032402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da538b5fdc7bda30 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:46.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.703+0000 7f0331d9b640 1 -- 172.21.15.76:0/3095641851 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0324031750 con 0x7f03340ade00 2023-11-09T03:52:46.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.703+0000 7f0331d9b640 1 -- 172.21.15.76:0/3095641851 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03240318b0 con 0x7f03340ade00 2023-11-09T03:52:46.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.703+0000 7f0331d9b640 1 -- 172.21.15.76:0/3095641851 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0324031c40 con 0x7f03340ade00 2023-11-09T03:52:46.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.703+0000 7f0338e07640 1 -- 172.21.15.76:0/3095641851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03340ade00 msgr2=0x7f03340b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:46.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.703+0000 7f0338e07640 1 --2- 172.21.15.76:0/3095641851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03340ade00 0x7f03340b0530 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f03340c9080 tx=0x7f032402b7f0 comp rx=0 tx=0).stop 2023-11-09T03:52:46.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.703+0000 7f0338e07640 1 -- 172.21.15.76:0/3095641851 shutdown_connections 2023-11-09T03:52:46.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.703+0000 7f0338e07640 1 --2- 172.21.15.76:0/3095641851 >> v2:172.21.15.113:3300/0 conn(0x7f033406d9c0 0x7f03340710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:46.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.703+0000 7f0338e07640 1 --2- 172.21.15.76:0/3095641851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03340ade00 0x7f03340b0530 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:46.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.703+0000 7f0338e07640 1 -- 172.21.15.76:0/3095641851 >> 172.21.15.76:0/3095641851 conn(0x7f03340cb530 msgr2=0x7f03340cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:46.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.704+0000 7f0338e07640 1 -- 172.21.15.76:0/3095641851 shutdown_connections 2023-11-09T03:52:46.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.704+0000 7f0338e07640 1 -- 172.21.15.76:0/3095641851 wait complete. 2023-11-09T03:52:46.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.705+0000 7f0338e07640 1 Processor -- start 2023-11-09T03:52:46.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.705+0000 7f0338e07640 1 -- start start 2023-11-09T03:52:46.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.705+0000 7f0338e07640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f033406c5f0 0x7f033415d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:46.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.705+0000 7f0338e07640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f033406d9c0 0x7f03340ab240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:46.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.705+0000 7f0338e07640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03340ade00 0x7f03340ab780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:46.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.705+0000 7f0338e07640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03340a91d0 con 0x7f03340ade00 2023-11-09T03:52:46.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.705+0000 7f0338e07640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03340ad860 con 0x7f033406d9c0 2023-11-09T03:52:46.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.705+0000 7f0338e07640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033415d550 con 0x7f033406c5f0 2023-11-09T03:52:46.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.706+0000 7f0332d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03340ade00 0x7f03340ab780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:46.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.706+0000 7f0332d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03340ade00 0x7f03340ab780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38190/0 (socket says 172.21.15.76:38190) 2023-11-09T03:52:46.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.706+0000 7f0332d9d640 1 -- 172.21.15.76:0/3632795252 learned_addr learned my addr 172.21.15.76:0/3632795252 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:46.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.706+0000 7f032bfff640 1 --2- 172.21.15.76:0/3632795252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f033406d9c0 0x7f03340ab240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:46.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.706+0000 7f033259c640 1 --2- 172.21.15.76:0/3632795252 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f033406c5f0 0x7f033415d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:46.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.706+0000 7f0332d9d640 1 -- 172.21.15.76:0/3632795252 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f033406c5f0 msgr2=0x7f033415d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:46.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.706+0000 7f0332d9d640 1 --2- 172.21.15.76:0/3632795252 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f033406c5f0 0x7f033415d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:46.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.706+0000 7f0332d9d640 1 -- 172.21.15.76:0/3632795252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f033406d9c0 msgr2=0x7f03340ab240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:46.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.706+0000 7f0332d9d640 1 --2- 172.21.15.76:0/3632795252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f033406d9c0 0x7f03340ab240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:46.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.706+0000 7f0332d9d640 1 -- 172.21.15.76:0/3632795252 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03240094d0 con 0x7f03340ade00 2023-11-09T03:52:46.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.706+0000 7f0332d9d640 1 --2- 172.21.15.76:0/3632795252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03340ade00 0x7f03340ab780 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f032c00b970 tx=0x7f032c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:46.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.706+0000 7f032affd640 1 -- 172.21.15.76:0/3632795252 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c00cae0 con 0x7f03340ade00 2023-11-09T03:52:46.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.706+0000 7f032affd640 1 -- 172.21.15.76:0/3632795252 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032c00cc40 con 0x7f03340ade00 2023-11-09T03:52:46.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.706+0000 7f0338e07640 1 -- 172.21.15.76:0/3632795252 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03340c4bc0 con 0x7f03340ade00 2023-11-09T03:52:46.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.706+0000 7f032affd640 1 -- 172.21.15.76:0/3632795252 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c014440 con 0x7f03340ade00 2023-11-09T03:52:46.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.706+0000 7f0338e07640 1 -- 172.21.15.76:0/3632795252 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03340a8b10 con 0x7f03340ade00 2023-11-09T03:52:46.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.707+0000 7f0338e07640 1 -- 172.21.15.76:0/3632795252 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03340a9310 con 0x7f03340ade00 2023-11-09T03:52:46.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.708+0000 7f032affd640 1 -- 172.21.15.76:0/3632795252 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f032c01d3b0 con 0x7f03340ade00 2023-11-09T03:52:46.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.708+0000 7f032affd640 1 --2- 172.21.15.76:0/3632795252 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f031406ae70 0x7f031406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:46.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.708+0000 7f032affd640 1 -- 172.21.15.76:0/3632795252 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7952+0+0 (secure 0 0 0) 0x7f032c089c60 con 0x7f03340ade00 2023-11-09T03:52:46.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.708+0000 7f033259c640 1 --2- 172.21.15.76:0/3632795252 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f031406ae70 0x7f031406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:46.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.709+0000 7f033259c640 1 --2- 172.21.15.76:0/3632795252 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f031406ae70 0x7f031406d330 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0324035000 tx=0x7f0324008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:46.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.713+0000 7f032affd640 1 -- 172.21.15.76:0/3632795252 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f03340a9310 con 0x7f03340ade00 2023-11-09T03:52:46.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.882+0000 7f0338e07640 1 -- 172.21.15.76:0/3632795252 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03340a9310 con 0x7f03340ade00 2023-11-09T03:52:46.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.883+0000 7f032affd640 1 -- 172.21.15.76:0/3632795252 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+23336 (secure 0 0 0) 0x7f03340a9310 con 0x7f03340ade00 2023-11-09T03:52:46.884 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:52:46.884 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":109,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:52:43.475048+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-09T03:47:59.280601+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-09T03:47:59.742039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:48:55.490832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-09T03:48:56.352624+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":9,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.17","mappings":[{"from":0,"to":2}]},{"pgid":"17.36","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/3087804343":"2023-11-09T04:52:42.513556+0000","172.21.15.76:6833/1536527256":"2023-11-10T03:52:34.213804+0000","172.21.15.113:6837/445306598":"2023-11-10T03:52:27.406838+0000","172.21.15.76:6834/3229239133":"2023-11-10T03:51:32.624856+0000","192.168.0.1:0/503424885":"2023-11-09T04:51:47.727422+0000","172.21.15.76:6834/4011275120":"2023-11-10T03:49:42.029057+0000","172.21.15.76:6835/4011275120":"2023-11-10T03:49:42.029057+0000","172.21.15.113:6836/3280098776":"2023-11-10T03:50:07.616780+0000","172.21.15.113:6837/3280098776":"2023-11-10T03:50:07.616780+0000","172.21.15.76:6834/1663988060":"2023-11-10T03:50:33.665092+0000","172.21.15.76:6835/1663988060":"2023-11-10T03:50:33.665092+0000","172.21.15.113:6836/445306598":"2023-11-10T03:52:27.406838+0000","172.21.15.76:6833/3697850101":"2023-11-10T03:51:39.442929+0000","172.21.15.113:6836/3449910842":"2023-11-10T03:50:59.611256+0000","172.21.15.76:6835/3229239133":"2023-11-10T03:51:32.624856+0000","172.21.15.76:6832/1536527256":"2023-11-10T03:52:34.213804+0000","172.21.15.113:6837/3449910842":"2023-11-10T03:50:59.611256+0000","172.21.15.76:6832/3697850101":"2023-11-10T03:51:39.442929+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:52:46.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.885+0000 7f0338e07640 1 -- 172.21.15.76:0/3632795252 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f031406ae70 msgr2=0x7f031406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:46.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.885+0000 7f0338e07640 1 --2- 172.21.15.76:0/3632795252 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f031406ae70 0x7f031406d330 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0324035000 tx=0x7f0324008030 comp rx=0 tx=0).stop 2023-11-09T03:52:46.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.885+0000 7f0338e07640 1 -- 172.21.15.76:0/3632795252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03340ade00 msgr2=0x7f03340ab780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:46.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.885+0000 7f0338e07640 1 --2- 172.21.15.76:0/3632795252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03340ade00 0x7f03340ab780 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f032c00b970 tx=0x7f032c00be30 comp rx=0 tx=0).stop 2023-11-09T03:52:46.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.885+0000 7f0338e07640 1 -- 172.21.15.76:0/3632795252 shutdown_connections 2023-11-09T03:52:46.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.885+0000 7f0338e07640 1 --2- 172.21.15.76:0/3632795252 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f031406ae70 0x7f031406d330 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:46.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.885+0000 7f0338e07640 1 --2- 172.21.15.76:0/3632795252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03340ade00 0x7f03340ab780 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:46.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.885+0000 7f0338e07640 1 --2- 172.21.15.76:0/3632795252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f033406d9c0 0x7f03340ab240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:46.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.885+0000 7f0338e07640 1 --2- 172.21.15.76:0/3632795252 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f033406c5f0 0x7f033415d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:46.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.885+0000 7f0338e07640 1 -- 172.21.15.76:0/3632795252 >> 172.21.15.76:0/3632795252 conn(0x7f03340cb530 msgr2=0x7f0334071090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:46.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.885+0000 7f0338e07640 1 -- 172.21.15.76:0/3632795252 shutdown_connections 2023-11-09T03:52:46.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.885+0000 7f0338e07640 1 -- 172.21.15.76:0/3632795252 wait complete. 2023-11-09T03:52:46.896 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-09T03:47:00.036540+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_data_pool', 'create_time': '2023-11-09T03:47:59.280601+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 70, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-09T03:47:59.742039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:48:55.490832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:48:56.352624+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 9, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-09T03:52:46.897 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:52:46.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:46.999+0000 7fb51eccf640 1 Processor -- start 2023-11-09T03:52:47.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.000+0000 7fb51eccf640 1 -- start start 2023-11-09T03:52:47.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.000+0000 7fb51eccf640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5180d1be0 0x7fb518167e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:47.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.000+0000 7fb51eccf640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5180b32b0 con 0x7fb51815d4e0 2023-11-09T03:52:47.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.000+0000 7fb51eccf640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5180cee10 con 0x7fb51815d100 2023-11-09T03:52:47.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.000+0000 7fb51eccf640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5180cec20 con 0x7fb5180d1be0 2023-11-09T03:52:47.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.000+0000 7fb517fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb51815d4e0 0x7fb5180d14d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45174/0 (socket says 172.21.15.76:45174) 2023-11-09T03:52:47.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.000+0000 7fb517fff640 1 -- 172.21.15.76:0/1001452438 learned_addr learned my addr 172.21.15.76:0/1001452438 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:52:47.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.001+0000 7fb5177fe640 1 -- 172.21.15.76:0/1001452438 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2409853686 0 0) 0x7fb5180b32b0 con 0x7fb51815d4e0 2023-11-09T03:52:47.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.001+0000 7fb5177fe640 1 -- 172.21.15.76:0/1001452438 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb508003520 con 0x7fb51815d4e0 2023-11-09T03:52:47.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.001+0000 7fb51d26c640 1 --2- 172.21.15.76:0/1001452438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5180d1be0 0x7fb518167e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:47.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.001+0000 7fb5177fe640 1 -- 172.21.15.76:0/1001452438 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3978761464 0 0) 0x7fb508003520 con 0x7fb51815d4e0 2023-11-09T03:52:47.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.001+0000 7fb5177fe640 1 -- 172.21.15.76:0/1001452438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5180d1be0 msgr2=0x7fb518167e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:47.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.001+0000 7fb5177fe640 1 --2- 172.21.15.76:0/1001452438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5180d1be0 0x7fb518167e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.001+0000 7fb5177fe640 1 -- 172.21.15.76:0/1001452438 >> v1:172.21.15.113:6789/0 conn(0x7fb51815d100 legacy=0x7fb5180ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:47.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.001+0000 7fb5177fe640 1 -- 172.21.15.76:0/1001452438 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5180b4750 con 0x7fb51815d4e0 2023-11-09T03:52:47.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.001+0000 7fb5177fe640 1 -- 172.21.15.76:0/1001452438 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fb504001670 con 0x7fb51815d4e0 2023-11-09T03:52:47.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.001+0000 7fb5177fe640 1 -- 172.21.15.76:0/1001452438 >> v1:172.21.15.76:6789/0 conn(0x7fb51815d4e0 legacy=0x7fb5180d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:47.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.001+0000 7fb5177fe640 1 --2- 172.21.15.76:0/1001452438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5080039a0 0x7fb508003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:47.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.001+0000 7fb5177fe640 1 --2- 172.21.15.76:0/1001452438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5080047c0 0x7fb508004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:47.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.001+0000 7fb5177fe640 1 --2- 172.21.15.76:0/1001452438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5180d1be0 0x7fb508005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:47.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.001+0000 7fb5177fe640 1 -- 172.21.15.76:0/1001452438 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50800ba30 con 0x7fb5080039a0 2023-11-09T03:52:47.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.001+0000 7fb5177fe640 1 -- 172.21.15.76:0/1001452438 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50800bba0 con 0x7fb5080047c0 2023-11-09T03:52:47.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.001+0000 7fb5177fe640 1 -- 172.21.15.76:0/1001452438 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50800bd60 con 0x7fb5180d1be0 2023-11-09T03:52:47.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.001+0000 7fb51ca6b640 1 --2- 172.21.15.76:0/1001452438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5080039a0 0x7fb508003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:47.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.001+0000 7fb517fff640 1 --2- 172.21.15.76:0/1001452438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5080047c0 0x7fb508004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:47.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.001+0000 7fb51d26c640 1 --2- 172.21.15.76:0/1001452438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5180d1be0 0x7fb508005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:47.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.001+0000 7fb51ca6b640 1 -- 172.21.15.76:0/1001452438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5180d1be0 msgr2=0x7fb508005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:47.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.001+0000 7fb51ca6b640 1 --2- 172.21.15.76:0/1001452438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5180d1be0 0x7fb508005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.001+0000 7fb51ca6b640 1 -- 172.21.15.76:0/1001452438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5080047c0 msgr2=0x7fb508004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:47.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.001+0000 7fb51ca6b640 1 --2- 172.21.15.76:0/1001452438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5080047c0 0x7fb508004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.001+0000 7fb51ca6b640 1 -- 172.21.15.76:0/1001452438 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb508003520 con 0x7fb5080039a0 2023-11-09T03:52:47.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.001+0000 7fb51d26c640 1 --2- 172.21.15.76:0/1001452438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5180d1be0 0x7fb508005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:52:47.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.002+0000 7fb51ca6b640 1 --2- 172.21.15.76:0/1001452438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5080039a0 0x7fb508003dc0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fb50c007fd0 tx=0x7fb5080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4580f87a166f8f28 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:47.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.002+0000 7fb5177fe640 1 -- 172.21.15.76:0/1001452438 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c0052d0 con 0x7fb5080039a0 2023-11-09T03:52:47.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.002+0000 7fb5177fe640 1 -- 172.21.15.76:0/1001452438 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50c02e050 con 0x7fb5080039a0 2023-11-09T03:52:47.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.002+0000 7fb5177fe640 1 -- 172.21.15.76:0/1001452438 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c003640 con 0x7fb5080039a0 2023-11-09T03:52:47.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.003+0000 7fb51eccf640 1 -- 172.21.15.76:0/1001452438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5080039a0 msgr2=0x7fb508003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:47.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.003+0000 7fb51eccf640 1 --2- 172.21.15.76:0/1001452438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5080039a0 0x7fb508003dc0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fb50c007fd0 tx=0x7fb5080057f0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.003+0000 7fb51d26c640 1 -- 172.21.15.76:0/1001452438 reap_dead start 2023-11-09T03:52:47.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.003+0000 7fb51eccf640 1 -- 172.21.15.76:0/1001452438 shutdown_connections 2023-11-09T03:52:47.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.003+0000 7fb51eccf640 1 -- 172.21.15.76:0/1001452438 >> 172.21.15.76:0/1001452438 conn(0x7fb5180d2ef0 msgr2=0x7fb5180d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:47.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.003+0000 7fb51eccf640 1 -- 172.21.15.76:0/1001452438 shutdown_connections 2023-11-09T03:52:47.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.003+0000 7fb51eccf640 1 -- 172.21.15.76:0/1001452438 wait complete. 2023-11-09T03:52:47.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.004+0000 7fb51eccf640 1 Processor -- start 2023-11-09T03:52:47.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.004+0000 7fb51eccf640 1 -- start start 2023-11-09T03:52:47.016 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.005+0000 7fb51eccf640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5080039a0 0x7fb5181602e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:47.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.005+0000 7fb51eccf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5181f6f20 0x7fb518152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:47.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.005+0000 7fb51eccf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5181f6380 0x7fb5181f6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:47.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.005+0000 7fb51eccf640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5181f6d70 con 0x7fb5181f6380 2023-11-09T03:52:47.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.005+0000 7fb51eccf640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb518160820 con 0x7fb5181f6f20 2023-11-09T03:52:47.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.005+0000 7fb51eccf640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51815e320 con 0x7fb5080039a0 2023-11-09T03:52:47.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.005+0000 7fb51d26c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5181f6380 0x7fb5181f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:47.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.005+0000 7fb51d26c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5181f6380 0x7fb5181f6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38210/0 (socket says 172.21.15.76:38210) 2023-11-09T03:52:47.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.005+0000 7fb51d26c640 1 -- 172.21.15.76:0/3268337646 learned_addr learned my addr 172.21.15.76:0/3268337646 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:47.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.005+0000 7fb51ca6b640 1 --2- 172.21.15.76:0/3268337646 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5080039a0 0x7fb5181602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:47.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.005+0000 7fb517fff640 1 --2- 172.21.15.76:0/3268337646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5181f6f20 0x7fb518152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:47.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.005+0000 7fb51d26c640 1 -- 172.21.15.76:0/3268337646 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5080039a0 msgr2=0x7fb5181602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:47.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.005+0000 7fb51d26c640 1 --2- 172.21.15.76:0/3268337646 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5080039a0 0x7fb5181602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.019 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.005+0000 7fb51d26c640 1 -- 172.21.15.76:0/3268337646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5181f6f20 msgr2=0x7fb518152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:47.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.005+0000 7fb51d26c640 1 --2- 172.21.15.76:0/3268337646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5181f6f20 0x7fb518152310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.005+0000 7fb51d26c640 1 -- 172.21.15.76:0/3268337646 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51815e5a0 con 0x7fb5181f6380 2023-11-09T03:52:47.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.006+0000 7fb51d26c640 1 --2- 172.21.15.76:0/3268337646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5181f6380 0x7fb5181f6830 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fb51808d6c0 tx=0x7fb510009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:47.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.006+0000 7fb5157fa640 1 -- 172.21.15.76:0/3268337646 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51001b070 con 0x7fb5181f6380 2023-11-09T03:52:47.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.006+0000 7fb5157fa640 1 -- 172.21.15.76:0/3268337646 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51000f040 con 0x7fb5181f6380 2023-11-09T03:52:47.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.006+0000 7fb51eccf640 1 -- 172.21.15.76:0/3268337646 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51815e880 con 0x7fb5181f6380 2023-11-09T03:52:47.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.006+0000 7fb5157fa640 1 -- 172.21.15.76:0/3268337646 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51000e050 con 0x7fb5181f6380 2023-11-09T03:52:47.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.006+0000 7fb51eccf640 1 -- 172.21.15.76:0/3268337646 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51815ee40 con 0x7fb5181f6380 2023-11-09T03:52:47.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.007+0000 7fb51eccf640 1 -- 172.21.15.76:0/3268337646 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5180684b0 con 0x7fb5181f6380 2023-11-09T03:52:47.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.007+0000 7fb5157fa640 1 -- 172.21.15.76:0/3268337646 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb510015440 con 0x7fb5181f6380 2023-11-09T03:52:47.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.007+0000 7fb5157fa640 1 --2- 172.21.15.76:0/3268337646 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb4fc06ae70 0x7fb4fc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:47.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.008+0000 7fb5157fa640 1 -- 172.21.15.76:0/3268337646 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7952+0+0 (secure 0 0 0) 0x7fb51008a300 con 0x7fb5181f6380 2023-11-09T03:52:47.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.008+0000 7fb51ca6b640 1 --2- 172.21.15.76:0/3268337646 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb4fc06ae70 0x7fb4fc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:47.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.008+0000 7fb51ca6b640 1 --2- 172.21.15.76:0/3268337646 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb4fc06ae70 0x7fb4fc06d330 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb50c033000 tx=0x7fb50c009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:47.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.012+0000 7fb5157fa640 1 -- 172.21.15.76:0/3268337646 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb510054af0 con 0x7fb5181f6380 2023-11-09T03:52:47.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.177+0000 7fb51eccf640 1 -- 172.21.15.76:0/3268337646 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb5180d2cd0 con 0x7fb5181f6380 2023-11-09T03:52:47.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.178+0000 7fb5157fa640 1 -- 172.21.15.76:0/3268337646 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb510058110 con 0x7fb5181f6380 2023-11-09T03:52:47.178 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:52:47.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.180+0000 7fb51eccf640 1 -- 172.21.15.76:0/3268337646 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb4fc06ae70 msgr2=0x7fb4fc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:47.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.180+0000 7fb51eccf640 1 --2- 172.21.15.76:0/3268337646 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb4fc06ae70 0x7fb4fc06d330 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb50c033000 tx=0x7fb50c009030 comp rx=0 tx=0).stop 2023-11-09T03:52:47.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.180+0000 7fb51eccf640 1 -- 172.21.15.76:0/3268337646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5181f6380 msgr2=0x7fb5181f6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:47.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.180+0000 7fb51eccf640 1 --2- 172.21.15.76:0/3268337646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5181f6380 0x7fb5181f6830 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fb51808d6c0 tx=0x7fb510009f90 comp rx=0 tx=0).stop 2023-11-09T03:52:47.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.180+0000 7fb51eccf640 1 -- 172.21.15.76:0/3268337646 shutdown_connections 2023-11-09T03:52:47.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.180+0000 7fb51eccf640 1 --2- 172.21.15.76:0/3268337646 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb4fc06ae70 0x7fb4fc06d330 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.180+0000 7fb51eccf640 1 --2- 172.21.15.76:0/3268337646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5181f6380 0x7fb5181f6830 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.180+0000 7fb51eccf640 1 --2- 172.21.15.76:0/3268337646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5181f6f20 0x7fb518152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.180+0000 7fb51eccf640 1 --2- 172.21.15.76:0/3268337646 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5080039a0 0x7fb5181602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.180+0000 7fb51eccf640 1 -- 172.21.15.76:0/3268337646 >> 172.21.15.76:0/3268337646 conn(0x7fb5180d2ef0 msgr2=0x7fb5180d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:47.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.180+0000 7fb51eccf640 1 -- 172.21.15.76:0/3268337646 shutdown_connections 2023-11-09T03:52:47.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.180+0000 7fb51eccf640 1 -- 172.21.15.76:0/3268337646 wait complete. 2023-11-09T03:52:47.190 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-09T03:52:47.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.294+0000 7fa5f0bd7640 1 Processor -- start 2023-11-09T03:52:47.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.294+0000 7fa5f0bd7640 1 -- start start 2023-11-09T03:52:47.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.295+0000 7fa5f0bd7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa5ec06d9c0 0x7fa5ec0710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:47.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.295+0000 7fa5f0bd7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa5ec06c5f0 0x7fa5ec071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:47.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.295+0000 7fa5f0bd7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5ec06cc90 con 0x7fa5ec0ade00 2023-11-09T03:52:47.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.295+0000 7fa5f0bd7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5ec0b0c40 con 0x7fa5ec06d9c0 2023-11-09T03:52:47.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.295+0000 7fa5f0bd7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5ec071b50 con 0x7fa5ec06c5f0 2023-11-09T03:52:47.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.295+0000 7fa5e3fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa5ec06d9c0 0x7fa5ec0710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:47.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.295+0000 7fa5e3fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa5ec06d9c0 0x7fa5ec0710d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38220/0 (socket says 172.21.15.76:38220) 2023-11-09T03:52:47.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.295+0000 7fa5e3fff640 1 -- 172.21.15.76:0/73117581 learned_addr learned my addr 172.21.15.76:0/73117581 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:47.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.295+0000 7fa5e3fff640 1 -- 172.21.15.76:0/73117581 >> v2:172.21.15.113:3300/0 conn(0x7fa5ec06c5f0 msgr2=0x7fa5ec071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:47.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.295+0000 7fa5ead9d640 1 --2- 172.21.15.76:0/73117581 >> v2:172.21.15.113:3300/0 conn(0x7fa5ec06c5f0 0x7fa5ec071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:47.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.295+0000 7fa5e3fff640 1 --2- 172.21.15.76:0/73117581 >> v2:172.21.15.113:3300/0 conn(0x7fa5ec06c5f0 0x7fa5ec071610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.295+0000 7fa5e3fff640 1 -- 172.21.15.76:0/73117581 >> v1:172.21.15.113:6789/0 conn(0x7fa5ec0ade00 legacy=0x7fa5ec0b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:47.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.295+0000 7fa5e3fff640 1 -- 172.21.15.76:0/73117581 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ec0b0e60 con 0x7fa5ec06d9c0 2023-11-09T03:52:47.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.296+0000 7fa5e3fff640 1 --2- 172.21.15.76:0/73117581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5ec06d9c0 0x7fa5ec0710d0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec0b44f0 tx=0x7fa5d402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9164921e81de08 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:47.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.296+0000 7fa5e9d9b640 1 -- 172.21.15.76:0/73117581 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4031750 con 0x7fa5ec06d9c0 2023-11-09T03:52:47.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.296+0000 7fa5e9d9b640 1 -- 172.21.15.76:0/73117581 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d40318b0 con 0x7fa5ec06d9c0 2023-11-09T03:52:47.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.296+0000 7fa5e9d9b640 1 -- 172.21.15.76:0/73117581 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4031c40 con 0x7fa5ec06d9c0 2023-11-09T03:52:47.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.296+0000 7fa5f0bd7640 1 -- 172.21.15.76:0/73117581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5ec06d9c0 msgr2=0x7fa5ec0710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:47.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.296+0000 7fa5f0bd7640 1 --2- 172.21.15.76:0/73117581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5ec06d9c0 0x7fa5ec0710d0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec0b44f0 tx=0x7fa5d402b7f0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.297+0000 7fa5f0bd7640 1 -- 172.21.15.76:0/73117581 shutdown_connections 2023-11-09T03:52:47.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.297+0000 7fa5f0bd7640 1 --2- 172.21.15.76:0/73117581 >> v2:172.21.15.113:3300/0 conn(0x7fa5ec06c5f0 0x7fa5ec071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.297+0000 7fa5f0bd7640 1 --2- 172.21.15.76:0/73117581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5ec06d9c0 0x7fa5ec0710d0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.297+0000 7fa5f0bd7640 1 -- 172.21.15.76:0/73117581 >> 172.21.15.76:0/73117581 conn(0x7fa5ec0cb530 msgr2=0x7fa5ec0cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:47.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.297+0000 7fa5f0bd7640 1 -- 172.21.15.76:0/73117581 shutdown_connections 2023-11-09T03:52:47.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.297+0000 7fa5f0bd7640 1 -- 172.21.15.76:0/73117581 wait complete. 2023-11-09T03:52:47.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.298+0000 7fa5f0bd7640 1 Processor -- start 2023-11-09T03:52:47.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.298+0000 7fa5f0bd7640 1 -- start start 2023-11-09T03:52:47.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.299+0000 7fa5f0bd7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5ec06c5f0 0x7fa5ec1f04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:47.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.299+0000 7fa5f0bd7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5ec06d9c0 0x7fa5ec0c5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:47.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.299+0000 7fa5f0bd7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5ec0ade00 0x7fa5ec153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:47.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.299+0000 7fa5f0bd7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec0c59d0 con 0x7fa5ec06d9c0 2023-11-09T03:52:47.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.299+0000 7fa5f0bd7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec1f1490 con 0x7fa5ec06c5f0 2023-11-09T03:52:47.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.299+0000 7fa5f0bd7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec155de0 con 0x7fa5ec0ade00 2023-11-09T03:52:47.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.299+0000 7fa5e3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5ec06d9c0 0x7fa5ec0c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:47.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.299+0000 7fa5e3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5ec06d9c0 0x7fa5ec0c5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38234/0 (socket says 172.21.15.76:38234) 2023-11-09T03:52:47.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.299+0000 7fa5e3fff640 1 -- 172.21.15.76:0/1131202597 learned_addr learned my addr 172.21.15.76:0/1131202597 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:47.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.299+0000 7fa5ea59c640 1 --2- 172.21.15.76:0/1131202597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5ec06c5f0 0x7fa5ec1f04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:47.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.299+0000 7fa5ead9d640 1 --2- 172.21.15.76:0/1131202597 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5ec0ade00 0x7fa5ec153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:47.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.299+0000 7fa5e3fff640 1 -- 172.21.15.76:0/1131202597 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5ec0ade00 msgr2=0x7fa5ec153e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:47.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.299+0000 7fa5e3fff640 1 --2- 172.21.15.76:0/1131202597 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5ec0ade00 0x7fa5ec153e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.299+0000 7fa5e3fff640 1 -- 172.21.15.76:0/1131202597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5ec06c5f0 msgr2=0x7fa5ec1f04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:47.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.299+0000 7fa5e3fff640 1 --2- 172.21.15.76:0/1131202597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5ec06c5f0 0x7fa5ec1f04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.299+0000 7fa5e3fff640 1 -- 172.21.15.76:0/1131202597 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d40094d0 con 0x7fa5ec06d9c0 2023-11-09T03:52:47.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.299+0000 7fa5e3fff640 1 --2- 172.21.15.76:0/1131202597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5ec06d9c0 0x7fa5ec0c5490 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4000c00 tx=0x7fa5d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:47.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.299+0000 7fa5e2ffd640 1 -- 172.21.15.76:0/1131202597 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4033c80 con 0x7fa5ec06d9c0 2023-11-09T03:52:47.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.299+0000 7fa5e2ffd640 1 -- 172.21.15.76:0/1131202597 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d4033de0 con 0x7fa5ec06d9c0 2023-11-09T03:52:47.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.299+0000 7fa5e2ffd640 1 -- 172.21.15.76:0/1131202597 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d403c640 con 0x7fa5ec06d9c0 2023-11-09T03:52:47.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.299+0000 7fa5f0bd7640 1 -- 172.21.15.76:0/1131202597 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5ec1f0bf0 con 0x7fa5ec06d9c0 2023-11-09T03:52:47.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.299+0000 7fa5f0bd7640 1 -- 172.21.15.76:0/1131202597 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5ec1f11b0 con 0x7fa5ec06d9c0 2023-11-09T03:52:47.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.301+0000 7fa5f0bd7640 1 -- 172.21.15.76:0/1131202597 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5ec0684b0 con 0x7fa5ec06d9c0 2023-11-09T03:52:47.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.301+0000 7fa5e2ffd640 1 -- 172.21.15.76:0/1131202597 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa5d403a040 con 0x7fa5ec06d9c0 2023-11-09T03:52:47.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.301+0000 7fa5e2ffd640 1 --2- 172.21.15.76:0/1131202597 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa5cc06ae70 0x7fa5cc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:47.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.301+0000 7fa5e2ffd640 1 -- 172.21.15.76:0/1131202597 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7952+0+0 (secure 0 0 0) 0x7fa5d40abad0 con 0x7fa5ec06d9c0 2023-11-09T03:52:47.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.301+0000 7fa5ea59c640 1 --2- 172.21.15.76:0/1131202597 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa5cc06ae70 0x7fa5cc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:47.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.302+0000 7fa5ea59c640 1 --2- 172.21.15.76:0/1131202597 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa5cc06ae70 0x7fa5cc06d330 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc0017f0 tx=0x7fa5dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:47.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.305+0000 7fa5e2ffd640 1 -- 172.21.15.76:0/1131202597 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa5d4076400 con 0x7fa5ec06d9c0 2023-11-09T03:52:47.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.470+0000 7fa5f0bd7640 1 -- 172.21.15.76:0/1131202597 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7fa5ec0cddc0 con 0x7fa5ec06d9c0 2023-11-09T03:52:47.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.470+0000 7fa5e2ffd640 1 -- 172.21.15.76:0/1131202597 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v109) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa5d40380a0 con 0x7fa5ec06d9c0 2023-11-09T03:52:47.470 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:52:47.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.472+0000 7fa5f0bd7640 1 -- 172.21.15.76:0/1131202597 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa5cc06ae70 msgr2=0x7fa5cc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:47.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.472+0000 7fa5f0bd7640 1 --2- 172.21.15.76:0/1131202597 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa5cc06ae70 0x7fa5cc06d330 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc0017f0 tx=0x7fa5dc009040 comp rx=0 tx=0).stop 2023-11-09T03:52:47.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.472+0000 7fa5f0bd7640 1 -- 172.21.15.76:0/1131202597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5ec06d9c0 msgr2=0x7fa5ec0c5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:47.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.472+0000 7fa5f0bd7640 1 --2- 172.21.15.76:0/1131202597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5ec06d9c0 0x7fa5ec0c5490 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4000c00 tx=0x7fa5d40046c0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.472+0000 7fa5f0bd7640 1 -- 172.21.15.76:0/1131202597 shutdown_connections 2023-11-09T03:52:47.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.472+0000 7fa5f0bd7640 1 --2- 172.21.15.76:0/1131202597 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa5cc06ae70 0x7fa5cc06d330 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.472+0000 7fa5f0bd7640 1 --2- 172.21.15.76:0/1131202597 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5ec0ade00 0x7fa5ec153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.472+0000 7fa5f0bd7640 1 --2- 172.21.15.76:0/1131202597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5ec06d9c0 0x7fa5ec0c5490 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.472+0000 7fa5f0bd7640 1 --2- 172.21.15.76:0/1131202597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5ec06c5f0 0x7fa5ec1f04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.472+0000 7fa5f0bd7640 1 -- 172.21.15.76:0/1131202597 >> 172.21.15.76:0/1131202597 conn(0x7fa5ec0cb530 msgr2=0x7fa5ec15d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:47.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.472+0000 7fa5f0bd7640 1 -- 172.21.15.76:0/1131202597 shutdown_connections 2023-11-09T03:52:47.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.472+0000 7fa5f0bd7640 1 -- 172.21.15.76:0/1131202597 wait complete. 2023-11-09T03:52:47.482 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-09T03:52:47.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.586+0000 7f4fac456640 1 Processor -- start 2023-11-09T03:52:47.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.586+0000 7f4fac456640 1 -- start start 2023-11-09T03:52:47.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.587+0000 7f4fac456640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fa40b0c40 0x7f4fa406fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:47.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.587+0000 7f4fac456640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa4071500 con 0x7f4fa4159e40 2023-11-09T03:52:47.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.587+0000 7f4fac456640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa415dae0 con 0x7f4fa406bb60 2023-11-09T03:52:47.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.587+0000 7f4fac456640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa4070c60 con 0x7f4fa40b0c40 2023-11-09T03:52:47.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.587+0000 7f4faa1f2640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f4fa4159e40 0x7f4fa40b0530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45188/0 (socket says 172.21.15.76:45188) 2023-11-09T03:52:47.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.587+0000 7f4faa1f2640 1 -- 172.21.15.76:0/2745651455 learned_addr learned my addr 172.21.15.76:0/2745651455 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:52:47.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.588+0000 7f4fa91f0640 1 -- 172.21.15.76:0/2745651455 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2517701025 0 0) 0x7f4fa4071500 con 0x7f4fa4159e40 2023-11-09T03:52:47.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.588+0000 7f4fa99f1640 1 --2- 172.21.15.76:0/2745651455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fa40b0c40 0x7f4fa406fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:47.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.588+0000 7f4fa91f0640 1 -- 172.21.15.76:0/2745651455 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f90003520 con 0x7f4fa4159e40 2023-11-09T03:52:47.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.588+0000 7f4fa91f0640 1 -- 172.21.15.76:0/2745651455 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4043231719 0 0) 0x7f4f90003520 con 0x7f4fa4159e40 2023-11-09T03:52:47.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.588+0000 7f4fa91f0640 1 -- 172.21.15.76:0/2745651455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fa40b0c40 msgr2=0x7f4fa406fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:47.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.588+0000 7f4fa91f0640 1 --2- 172.21.15.76:0/2745651455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fa40b0c40 0x7f4fa406fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.588+0000 7f4fa91f0640 1 -- 172.21.15.76:0/2745651455 >> v1:172.21.15.113:6789/0 conn(0x7f4fa406bb60 legacy=0x7f4fa40cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:47.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.588+0000 7f4fa91f0640 1 -- 172.21.15.76:0/2745651455 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa4070f40 con 0x7f4fa4159e40 2023-11-09T03:52:47.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.588+0000 7f4fa91f0640 1 -- 172.21.15.76:0/2745651455 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f4fa0001670 con 0x7f4fa4159e40 2023-11-09T03:52:47.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.588+0000 7f4fa91f0640 1 -- 172.21.15.76:0/2745651455 >> v1:172.21.15.76:6789/0 conn(0x7f4fa4159e40 legacy=0x7f4fa40b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:47.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.588+0000 7f4fa91f0640 1 --2- 172.21.15.76:0/2745651455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f900039a0 0x7f4f90003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:47.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.588+0000 7f4fa91f0640 1 --2- 172.21.15.76:0/2745651455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fa40b0c40 0x7f4f900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:47.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.588+0000 7f4fa91f0640 1 --2- 172.21.15.76:0/2745651455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f90004d00 0x7f4f90005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:47.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.588+0000 7f4fa91f0640 1 -- 172.21.15.76:0/2745651455 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa4070c60 con 0x7f4f900039a0 2023-11-09T03:52:47.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.588+0000 7f4fa91f0640 1 -- 172.21.15.76:0/2745651455 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9000ba30 con 0x7f4f90004d00 2023-11-09T03:52:47.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.588+0000 7f4fa91f0640 1 -- 172.21.15.76:0/2745651455 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9000de00 con 0x7f4fa40b0c40 2023-11-09T03:52:47.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.588+0000 7f4faa1f2640 1 --2- 172.21.15.76:0/2745651455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f900039a0 0x7f4f90003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:47.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.588+0000 7f4fa99f1640 1 --2- 172.21.15.76:0/2745651455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fa40b0c40 0x7f4f900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:47.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.588+0000 7f4faa9f3640 1 --2- 172.21.15.76:0/2745651455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f90004d00 0x7f4f90005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:47.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.588+0000 7f4faa1f2640 1 -- 172.21.15.76:0/2745651455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fa40b0c40 msgr2=0x7f4f900047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:47.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.588+0000 7f4faa1f2640 1 --2- 172.21.15.76:0/2745651455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fa40b0c40 0x7f4f900047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.588+0000 7f4faa1f2640 1 -- 172.21.15.76:0/2745651455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f90004d00 msgr2=0x7f4f90005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:47.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.588+0000 7f4faa1f2640 1 --2- 172.21.15.76:0/2745651455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f90004d00 0x7f4f90005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.588+0000 7f4faa1f2640 1 -- 172.21.15.76:0/2745651455 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f90003520 con 0x7f4f900039a0 2023-11-09T03:52:47.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.589+0000 7f4faa1f2640 1 --2- 172.21.15.76:0/2745651455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f900039a0 0x7f4f90003d80 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f4fa000c210 tx=0x7f4f900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc9fa030d5429d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:47.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.589+0000 7f4fa91f0640 1 -- 172.21.15.76:0/2745651455 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa002cae0 con 0x7f4f900039a0 2023-11-09T03:52:47.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.589+0000 7f4fa91f0640 1 -- 172.21.15.76:0/2745651455 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa002cc60 con 0x7f4f900039a0 2023-11-09T03:52:47.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.589+0000 7f4fa91f0640 1 -- 172.21.15.76:0/2745651455 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa00058d0 con 0x7f4f900039a0 2023-11-09T03:52:47.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.590+0000 7f4fac456640 1 -- 172.21.15.76:0/2745651455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f900039a0 msgr2=0x7f4f90003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:47.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.590+0000 7f4fac456640 1 --2- 172.21.15.76:0/2745651455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f900039a0 0x7f4f90003d80 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f4fa000c210 tx=0x7f4f900057f0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.590+0000 7f4faa9f3640 1 -- 172.21.15.76:0/2745651455 reap_dead start 2023-11-09T03:52:47.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.590+0000 7f4fac456640 1 -- 172.21.15.76:0/2745651455 shutdown_connections 2023-11-09T03:52:47.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.590+0000 7f4fac456640 1 -- 172.21.15.76:0/2745651455 >> 172.21.15.76:0/2745651455 conn(0x7f4fa40cb530 msgr2=0x7f4fa40cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:47.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.590+0000 7f4fac456640 1 -- 172.21.15.76:0/2745651455 shutdown_connections 2023-11-09T03:52:47.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.590+0000 7f4fac456640 1 -- 172.21.15.76:0/2745651455 wait complete. 2023-11-09T03:52:47.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.591+0000 7f4fac456640 1 Processor -- start 2023-11-09T03:52:47.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.591+0000 7f4fac456640 1 -- start start 2023-11-09T03:52:47.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.592+0000 7f4fac456640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f900039a0 0x7f4fa415bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:47.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.592+0000 7f4fac456640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fa40ccfa0 0x7f4fa40cd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:47.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.592+0000 7f4fac456640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fa415a7c0 0x7f4fa415ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:47.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.592+0000 7f4fac456640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa40cd960 con 0x7f4fa415a7c0 2023-11-09T03:52:47.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.592+0000 7f4fac456640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa40c5980 con 0x7f4fa40ccfa0 2023-11-09T03:52:47.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.592+0000 7f4fac456640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa415b180 con 0x7f4f900039a0 2023-11-09T03:52:47.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.592+0000 7f4faa9f3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fa415a7c0 0x7f4fa415ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:47.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.592+0000 7f4faa9f3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fa415a7c0 0x7f4fa415ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38260/0 (socket says 172.21.15.76:38260) 2023-11-09T03:52:47.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.592+0000 7f4faa9f3640 1 -- 172.21.15.76:0/2198423704 learned_addr learned my addr 172.21.15.76:0/2198423704 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:47.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.592+0000 7f4fa99f1640 1 --2- 172.21.15.76:0/2198423704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fa40ccfa0 0x7f4fa40cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:47.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.592+0000 7f4faa1f2640 1 --2- 172.21.15.76:0/2198423704 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f900039a0 0x7f4fa415bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:47.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.592+0000 7f4faa9f3640 1 -- 172.21.15.76:0/2198423704 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f900039a0 msgr2=0x7f4fa415bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:47.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.592+0000 7f4faa9f3640 1 --2- 172.21.15.76:0/2198423704 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f900039a0 0x7f4fa415bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.592+0000 7f4faa9f3640 1 -- 172.21.15.76:0/2198423704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fa40ccfa0 msgr2=0x7f4fa40cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:47.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.592+0000 7f4faa9f3640 1 --2- 172.21.15.76:0/2198423704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fa40ccfa0 0x7f4fa40cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.592+0000 7f4faa9f3640 1 -- 172.21.15.76:0/2198423704 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa415c5a0 con 0x7f4fa415a7c0 2023-11-09T03:52:47.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.592+0000 7f4faa1f2640 1 --2- 172.21.15.76:0/2198423704 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f900039a0 0x7f4fa415bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:52:47.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.592+0000 7f4faa9f3640 1 --2- 172.21.15.76:0/2198423704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fa415a7c0 0x7f4fa415ac40 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f4fa406de00 tx=0x7f4f9c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:47.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.592+0000 7f4f9affd640 1 -- 172.21.15.76:0/2198423704 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c00ee50 con 0x7f4fa415a7c0 2023-11-09T03:52:47.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.592+0000 7f4f9affd640 1 -- 172.21.15.76:0/2198423704 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9c003b80 con 0x7f4fa415a7c0 2023-11-09T03:52:47.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.592+0000 7f4f9affd640 1 -- 172.21.15.76:0/2198423704 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c0050d0 con 0x7f4fa415a7c0 2023-11-09T03:52:47.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.592+0000 7f4fac456640 1 -- 172.21.15.76:0/2198423704 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fa415c880 con 0x7f4fa415a7c0 2023-11-09T03:52:47.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.593+0000 7f4fac456640 1 -- 172.21.15.76:0/2198423704 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fa415ce40 con 0x7f4fa415a7c0 2023-11-09T03:52:47.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.594+0000 7f4fac456640 1 -- 172.21.15.76:0/2198423704 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fa415d000 con 0x7f4fa415a7c0 2023-11-09T03:52:47.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.594+0000 7f4f9affd640 1 -- 172.21.15.76:0/2198423704 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4f9c005230 con 0x7f4fa415a7c0 2023-11-09T03:52:47.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.594+0000 7f4f9affd640 1 --2- 172.21.15.76:0/2198423704 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4f8806aea0 0x7f4f8806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:47.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.594+0000 7f4f9affd640 1 -- 172.21.15.76:0/2198423704 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7952+0+0 (secure 0 0 0) 0x7f4f9c089e20 con 0x7f4fa415a7c0 2023-11-09T03:52:47.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.595+0000 7f4faa1f2640 1 --2- 172.21.15.76:0/2198423704 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4f8806aea0 0x7f4f8806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:47.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.595+0000 7f4faa1f2640 1 --2- 172.21.15.76:0/2198423704 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4f8806aea0 0x7f4f8806d360 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0034000 tx=0x7f4fa0009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:47.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.599+0000 7f4f9affd640 1 -- 172.21.15.76:0/2198423704 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4f9c054610 con 0x7f4fa415a7c0 2023-11-09T03:52:47.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.765+0000 7f4fac456640 1 -- 172.21.15.76:0/2198423704 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7f4fa40684b0 con 0x7f4fa415a7c0 2023-11-09T03:52:47.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.765+0000 7f4f9affd640 1 -- 172.21.15.76:0/2198423704 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v109) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4f9c057c30 con 0x7f4fa415a7c0 2023-11-09T03:52:47.765 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:52:47.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.767+0000 7f4fac456640 1 -- 172.21.15.76:0/2198423704 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4f8806aea0 msgr2=0x7f4f8806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:47.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.767+0000 7f4fac456640 1 --2- 172.21.15.76:0/2198423704 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4f8806aea0 0x7f4f8806d360 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0034000 tx=0x7f4fa0009030 comp rx=0 tx=0).stop 2023-11-09T03:52:47.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.767+0000 7f4fac456640 1 -- 172.21.15.76:0/2198423704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fa415a7c0 msgr2=0x7f4fa415ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:47.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.767+0000 7f4fac456640 1 --2- 172.21.15.76:0/2198423704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fa415a7c0 0x7f4fa415ac40 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f4fa406de00 tx=0x7f4f9c00ec00 comp rx=0 tx=0).stop 2023-11-09T03:52:47.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.767+0000 7f4fac456640 1 -- 172.21.15.76:0/2198423704 shutdown_connections 2023-11-09T03:52:47.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.767+0000 7f4fac456640 1 --2- 172.21.15.76:0/2198423704 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4f8806aea0 0x7f4f8806d360 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.767+0000 7f4fac456640 1 --2- 172.21.15.76:0/2198423704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fa415a7c0 0x7f4fa415ac40 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.767+0000 7f4fac456640 1 --2- 172.21.15.76:0/2198423704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fa40ccfa0 0x7f4fa40cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.767+0000 7f4fac456640 1 --2- 172.21.15.76:0/2198423704 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f900039a0 0x7f4fa415bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.767+0000 7f4fac456640 1 -- 172.21.15.76:0/2198423704 >> 172.21.15.76:0/2198423704 conn(0x7f4fa40cb530 msgr2=0x7f4fa4157960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:47.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.767+0000 7f4fac456640 1 -- 172.21.15.76:0/2198423704 shutdown_connections 2023-11-09T03:52:47.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.767+0000 7f4fac456640 1 -- 172.21.15.76:0/2198423704 wait complete. 2023-11-09T03:52:47.778 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-09T03:52:47.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.881+0000 7f85cfbb6640 1 Processor -- start 2023-11-09T03:52:47.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.881+0000 7f85cfbb6640 1 -- start start 2023-11-09T03:52:47.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.882+0000 7f85cfbb6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85c80addd0 0x7f85c80cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:47.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.882+0000 7f85cfbb6640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c80cd720 con 0x7f85c80cc680 2023-11-09T03:52:47.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.882+0000 7f85cfbb6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c80ae1b0 con 0x7f85c806c220 2023-11-09T03:52:47.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.882+0000 7f85cfbb6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c80ccb10 con 0x7f85c80addd0 2023-11-09T03:52:47.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.882+0000 7f85cd151640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f85c80cc680 0x7f85c806bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45192/0 (socket says 172.21.15.76:45192) 2023-11-09T03:52:47.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.882+0000 7f85cd151640 1 -- 172.21.15.76:0/2032334126 learned_addr learned my addr 172.21.15.76:0/2032334126 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:52:47.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.882+0000 7f85cd952640 1 --2- 172.21.15.76:0/2032334126 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85c80addd0 0x7f85c80cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:47.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.882+0000 7f85cc950640 1 -- 172.21.15.76:0/2032334126 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1365816919 0 0) 0x7f85c80cd720 con 0x7f85c80cc680 2023-11-09T03:52:47.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.882+0000 7f85cc950640 1 -- 172.21.15.76:0/2032334126 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85b4003520 con 0x7f85c80cc680 2023-11-09T03:52:47.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.882+0000 7f85cc950640 1 -- 172.21.15.76:0/2032334126 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1822917415 0 0) 0x7f85c80ae1b0 con 0x7f85c806c220 2023-11-09T03:52:47.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.882+0000 7f85cc950640 1 -- 172.21.15.76:0/2032334126 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85c80cd720 con 0x7f85c806c220 2023-11-09T03:52:47.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.882+0000 7f85cc950640 1 -- 172.21.15.76:0/2032334126 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3915757134 0 0) 0x7f85b4003520 con 0x7f85c80cc680 2023-11-09T03:52:47.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.882+0000 7f85cd952640 1 -- 172.21.15.76:0/2032334126 >> v1:172.21.15.113:6789/0 conn(0x7f85c806c220 legacy=0x7f85c806f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:47.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.882+0000 7f85cd952640 1 -- 172.21.15.76:0/2032334126 >> v1:172.21.15.76:6789/0 conn(0x7f85c80cc680 legacy=0x7f85c806bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:47.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.882+0000 7f85cd952640 1 -- 172.21.15.76:0/2032334126 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c80ccdf0 con 0x7f85c80addd0 2023-11-09T03:52:47.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.883+0000 7f85cd952640 1 --2- 172.21.15.76:0/2032334126 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85c80addd0 0x7f85c80cdbb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f85c400a970 tx=0x7f85c402ccd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed0cbaef095a1b60 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:47.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.883+0000 7f85cc950640 1 -- 172.21.15.76:0/2032334126 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8001670 con 0x7f85c80addd0 2023-11-09T03:52:47.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.883+0000 7f85cc950640 1 -- 172.21.15.76:0/2032334126 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c402d800 con 0x7f85c80addd0 2023-11-09T03:52:47.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.883+0000 7f85cc950640 1 -- 172.21.15.76:0/2032334126 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c402db70 con 0x7f85c80addd0 2023-11-09T03:52:47.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.884+0000 7f85cfbb6640 1 -- 172.21.15.76:0/2032334126 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85c80addd0 msgr2=0x7f85c80cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:47.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.884+0000 7f85cfbb6640 1 --2- 172.21.15.76:0/2032334126 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85c80addd0 0x7f85c80cdbb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f85c400a970 tx=0x7f85c402ccd0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.884+0000 7f85cfbb6640 1 -- 172.21.15.76:0/2032334126 shutdown_connections 2023-11-09T03:52:47.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.884+0000 7f85cfbb6640 1 --2- 172.21.15.76:0/2032334126 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85c80addd0 0x7f85c80cdbb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.884+0000 7f85cfbb6640 1 -- 172.21.15.76:0/2032334126 >> 172.21.15.76:0/2032334126 conn(0x7f85c80cb530 msgr2=0x7f85c80cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:47.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.884+0000 7f85cfbb6640 1 -- 172.21.15.76:0/2032334126 shutdown_connections 2023-11-09T03:52:47.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.884+0000 7f85cfbb6640 1 -- 172.21.15.76:0/2032334126 wait complete. 2023-11-09T03:52:47.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.885+0000 7f85cfbb6640 1 Processor -- start 2023-11-09T03:52:47.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.885+0000 7f85cfbb6640 1 -- start start 2023-11-09T03:52:47.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.886+0000 7f85cfbb6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85c806c220 0x7f85c81f2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:47.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.886+0000 7f85cfbb6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85c80addd0 0x7f85c8070200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:47.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.886+0000 7f85cfbb6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85c81f56f0 0x7f85c81f39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:47.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.886+0000 7f85cfbb6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c815ae70 con 0x7f85c80addd0 2023-11-09T03:52:47.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.886+0000 7f85cfbb6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c81f5b80 con 0x7f85c806c220 2023-11-09T03:52:47.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.886+0000 7f85cfbb6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c81f2e50 con 0x7f85c81f56f0 2023-11-09T03:52:47.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.886+0000 7f85cd151640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85c80addd0 0x7f85c8070200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:47.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.886+0000 7f85cd151640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85c80addd0 0x7f85c8070200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38264/0 (socket says 172.21.15.76:38264) 2023-11-09T03:52:47.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.886+0000 7f85cd151640 1 -- 172.21.15.76:0/1752318469 learned_addr learned my addr 172.21.15.76:0/1752318469 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:47.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.886+0000 7f85cd952640 1 --2- 172.21.15.76:0/1752318469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85c806c220 0x7f85c81f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:47.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.886+0000 7f85cd151640 1 -- 172.21.15.76:0/1752318469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85c81f56f0 msgr2=0x7f85c81f39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:47.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.886+0000 7f85ce153640 1 --2- 172.21.15.76:0/1752318469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85c81f56f0 0x7f85c81f39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:47.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.886+0000 7f85cd151640 1 --2- 172.21.15.76:0/1752318469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85c81f56f0 0x7f85c81f39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.886+0000 7f85cd151640 1 -- 172.21.15.76:0/1752318469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85c806c220 msgr2=0x7f85c81f2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:47.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.886+0000 7f85cd151640 1 --2- 172.21.15.76:0/1752318469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85c806c220 0x7f85c81f2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:47.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.886+0000 7f85cd151640 1 -- 172.21.15.76:0/1752318469 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c400a5a0 con 0x7f85c80addd0 2023-11-09T03:52:47.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.886+0000 7f85cd952640 1 --2- 172.21.15.76:0/1752318469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85c806c220 0x7f85c81f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:52:47.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.886+0000 7f85ce153640 1 --2- 172.21.15.76:0/1752318469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85c81f56f0 0x7f85c81f39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:52:47.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.886+0000 7f85cd151640 1 --2- 172.21.15.76:0/1752318469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85c80addd0 0x7f85c8070200 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f85b800c700 tx=0x7f85b800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:47.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.886+0000 7f85be7fc640 1 -- 172.21.15.76:0/1752318469 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8005ad0 con 0x7f85c80addd0 2023-11-09T03:52:47.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.887+0000 7f85be7fc640 1 -- 172.21.15.76:0/1752318469 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b8005c30 con 0x7f85c80addd0 2023-11-09T03:52:47.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.887+0000 7f85be7fc640 1 -- 172.21.15.76:0/1752318469 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8007070 con 0x7f85c80addd0 2023-11-09T03:52:47.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.887+0000 7f85cfbb6640 1 -- 172.21.15.76:0/1752318469 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c81f3130 con 0x7f85c80addd0 2023-11-09T03:52:47.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.887+0000 7f85cfbb6640 1 -- 172.21.15.76:0/1752318469 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c81f3f00 con 0x7f85c80addd0 2023-11-09T03:52:47.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.888+0000 7f85cfbb6640 1 -- 172.21.15.76:0/1752318469 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c815afe0 con 0x7f85c80addd0 2023-11-09T03:52:47.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.888+0000 7f85be7fc640 1 -- 172.21.15.76:0/1752318469 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f85b80071d0 con 0x7f85c80addd0 2023-11-09T03:52:47.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.888+0000 7f85be7fc640 1 --2- 172.21.15.76:0/1752318469 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f85ac06ae70 0x7f85ac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:47.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.888+0000 7f85be7fc640 1 -- 172.21.15.76:0/1752318469 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7952+0+0 (secure 0 0 0) 0x7f85b80894c0 con 0x7f85c80addd0 2023-11-09T03:52:47.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.889+0000 7f85cd952640 1 --2- 172.21.15.76:0/1752318469 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f85ac06ae70 0x7f85ac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:47.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.889+0000 7f85cd952640 1 --2- 172.21.15.76:0/1752318469 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f85ac06ae70 0x7f85ac06d330 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f85c4037000 tx=0x7f85c4031000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:47.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:47.893+0000 7f85be7fc640 1 -- 172.21.15.76:0/1752318469 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f85b8053d60 con 0x7f85c80addd0 2023-11-09T03:52:48.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.057+0000 7f85cfbb6640 1 -- 172.21.15.76:0/1752318469 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7f85c80684b0 con 0x7f85c80addd0 2023-11-09T03:52:48.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.057+0000 7f85be7fc640 1 -- 172.21.15.76:0/1752318469 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v109) v1 ==== 107+0+11 (secure 0 0 0) 0x7f85b8017020 con 0x7f85c80addd0 2023-11-09T03:52:48.057 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:52:48.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.059+0000 7f85cfbb6640 1 -- 172.21.15.76:0/1752318469 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f85ac06ae70 msgr2=0x7f85ac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:48.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.059+0000 7f85cfbb6640 1 --2- 172.21.15.76:0/1752318469 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f85ac06ae70 0x7f85ac06d330 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f85c4037000 tx=0x7f85c4031000 comp rx=0 tx=0).stop 2023-11-09T03:52:48.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.059+0000 7f85cfbb6640 1 -- 172.21.15.76:0/1752318469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85c80addd0 msgr2=0x7f85c8070200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:48.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.059+0000 7f85cfbb6640 1 --2- 172.21.15.76:0/1752318469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85c80addd0 0x7f85c8070200 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f85b800c700 tx=0x7f85b800cbc0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.059+0000 7f85cfbb6640 1 -- 172.21.15.76:0/1752318469 shutdown_connections 2023-11-09T03:52:48.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.059+0000 7f85cfbb6640 1 --2- 172.21.15.76:0/1752318469 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f85ac06ae70 0x7f85ac06d330 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.059+0000 7f85cfbb6640 1 --2- 172.21.15.76:0/1752318469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85c81f56f0 0x7f85c81f39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.059+0000 7f85cfbb6640 1 --2- 172.21.15.76:0/1752318469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85c80addd0 0x7f85c8070200 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.059+0000 7f85cfbb6640 1 --2- 172.21.15.76:0/1752318469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85c806c220 0x7f85c81f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.059+0000 7f85cfbb6640 1 -- 172.21.15.76:0/1752318469 >> 172.21.15.76:0/1752318469 conn(0x7f85c80cb530 msgr2=0x7f85c80b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:48.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.059+0000 7f85cfbb6640 1 -- 172.21.15.76:0/1752318469 shutdown_connections 2023-11-09T03:52:48.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.059+0000 7f85cfbb6640 1 -- 172.21.15.76:0/1752318469 wait complete. 2023-11-09T03:52:48.069 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:52:48.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.172+0000 7f4ff1087640 1 Processor -- start 2023-11-09T03:52:48.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.172+0000 7f4ff1087640 1 -- start start 2023-11-09T03:52:48.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.173+0000 7f4ff1087640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4fec15d100 0x7f4fec0ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:48.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.173+0000 7f4ff1087640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fec15d4e0 0x7f4fec0d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:48.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.173+0000 7f4ff1087640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fec0b32b0 con 0x7f4fec0d3b60 2023-11-09T03:52:48.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.173+0000 7f4ff1087640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fec0cee10 con 0x7f4fec15d100 2023-11-09T03:52:48.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.173+0000 7f4ff1087640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec0b4750 con 0x7f4fec15d4e0 2023-11-09T03:52:48.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.173+0000 7f4fead9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4fec15d100 0x7f4fec0ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:48.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.173+0000 7f4fead9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4fec15d100 0x7f4fec0ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38280/0 (socket says 172.21.15.76:38280) 2023-11-09T03:52:48.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.173+0000 7f4fead9d640 1 -- 172.21.15.76:0/4192960822 learned_addr learned my addr 172.21.15.76:0/4192960822 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:48.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.173+0000 7f4feb59e640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f4fec0d3b60 0x7f4fec0cf520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45206/0 (socket says 172.21.15.76:45206) 2023-11-09T03:52:48.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.173+0000 7f4fea59c640 1 --2- 172.21.15.76:0/4192960822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fec15d4e0 0x7f4fec0d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:48.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.174+0000 7f4fead9d640 1 -- 172.21.15.76:0/4192960822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fec15d4e0 msgr2=0x7f4fec0d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:48.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.174+0000 7f4fead9d640 1 --2- 172.21.15.76:0/4192960822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fec15d4e0 0x7f4fec0d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.174+0000 7f4fead9d640 1 -- 172.21.15.76:0/4192960822 >> v1:172.21.15.76:6789/0 conn(0x7f4fec0d3b60 legacy=0x7f4fec0cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:48.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.174+0000 7f4fead9d640 1 -- 172.21.15.76:0/4192960822 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fec0b2980 con 0x7f4fec15d100 2023-11-09T03:52:48.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.174+0000 7f4fe9d9b640 1 -- 172.21.15.76:0/4192960822 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3741119234 0 0) 0x7f4fec0b32b0 con 0x7f4fec0d3b60 2023-11-09T03:52:48.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.174+0000 7f4fead9d640 1 --2- 172.21.15.76:0/4192960822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fec15d100 0x7f4fec0ce510 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009ea0 tx=0x7f4fdc02bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5807fd7809050d95 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:48.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.174+0000 7f4fe9d9b640 1 -- 172.21.15.76:0/4192960822 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc02cb40 con 0x7f4fec15d100 2023-11-09T03:52:48.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.174+0000 7f4fe9d9b640 1 -- 172.21.15.76:0/4192960822 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fdc02cca0 con 0x7f4fec15d100 2023-11-09T03:52:48.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.174+0000 7f4fe9d9b640 1 -- 172.21.15.76:0/4192960822 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc034610 con 0x7f4fec15d100 2023-11-09T03:52:48.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.175+0000 7f4ff1087640 1 -- 172.21.15.76:0/4192960822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fec15d100 msgr2=0x7f4fec0ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:48.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.175+0000 7f4ff1087640 1 --2- 172.21.15.76:0/4192960822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fec15d100 0x7f4fec0ce510 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009ea0 tx=0x7f4fdc02bfa0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.175+0000 7f4ff1087640 1 -- 172.21.15.76:0/4192960822 shutdown_connections 2023-11-09T03:52:48.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.175+0000 7f4ff1087640 1 --2- 172.21.15.76:0/4192960822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fec15d4e0 0x7f4fec0d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.175+0000 7f4ff1087640 1 --2- 172.21.15.76:0/4192960822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fec15d100 0x7f4fec0ce510 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.175+0000 7f4ff1087640 1 -- 172.21.15.76:0/4192960822 >> 172.21.15.76:0/4192960822 conn(0x7f4fec0d2ef0 msgr2=0x7f4fec0d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:48.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.175+0000 7f4ff1087640 1 -- 172.21.15.76:0/4192960822 shutdown_connections 2023-11-09T03:52:48.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.175+0000 7f4ff1087640 1 -- 172.21.15.76:0/4192960822 wait complete. 2023-11-09T03:52:48.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.176+0000 7f4ff1087640 1 Processor -- start 2023-11-09T03:52:48.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.176+0000 7f4ff1087640 1 -- start start 2023-11-09T03:52:48.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.177+0000 7f4ff1087640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fec0d3b60 0x7f4fec152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:48.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.177+0000 7f4ff1087640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fec15d100 0x7f4fec0b4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:48.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.177+0000 7f4ff1087640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fec15d4e0 0x7f4fec0b6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:48.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.177+0000 7f4ff1087640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec1528c0 con 0x7f4fec15d100 2023-11-09T03:52:48.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.177+0000 7f4ff1087640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec0b5330 con 0x7f4fec0d3b60 2023-11-09T03:52:48.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.177+0000 7f4ff1087640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec0b5610 con 0x7f4fec15d4e0 2023-11-09T03:52:48.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.177+0000 7f4fea59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fec15d100 0x7f4fec0b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:48.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.177+0000 7f4fea59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fec15d100 0x7f4fec0b4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38292/0 (socket says 172.21.15.76:38292) 2023-11-09T03:52:48.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.177+0000 7f4fea59c640 1 -- 172.21.15.76:0/1977086009 learned_addr learned my addr 172.21.15.76:0/1977086009 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:48.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.177+0000 7f4feb59e640 1 --2- 172.21.15.76:0/1977086009 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fec15d4e0 0x7f4fec0b6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:48.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.177+0000 7f4fead9d640 1 --2- 172.21.15.76:0/1977086009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fec0d3b60 0x7f4fec152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:48.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.178+0000 7f4fea59c640 1 -- 172.21.15.76:0/1977086009 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fec15d4e0 msgr2=0x7f4fec0b6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:48.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.178+0000 7f4fea59c640 1 --2- 172.21.15.76:0/1977086009 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fec15d4e0 0x7f4fec0b6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.178+0000 7f4fea59c640 1 -- 172.21.15.76:0/1977086009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fec0d3b60 msgr2=0x7f4fec152380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:48.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.178+0000 7f4fea59c640 1 --2- 172.21.15.76:0/1977086009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fec0d3b60 0x7f4fec152380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.178+0000 7f4fea59c640 1 -- 172.21.15.76:0/1977086009 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fdc009b30 con 0x7f4fec15d100 2023-11-09T03:52:48.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.178+0000 7f4feb59e640 1 --2- 172.21.15.76:0/1977086009 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fec15d4e0 0x7f4fec0b6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:52:48.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.178+0000 7f4fead9d640 1 --2- 172.21.15.76:0/1977086009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fec0d3b60 0x7f4fec152380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:52:48.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.178+0000 7f4fea59c640 1 --2- 172.21.15.76:0/1977086009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fec15d100 0x7f4fec0b4950 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000e9b0 tx=0x7f4fe000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:48.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.178+0000 7f4fd37fe640 1 -- 172.21.15.76:0/1977086009 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0009670 con 0x7f4fec15d100 2023-11-09T03:52:48.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.178+0000 7f4fd37fe640 1 -- 172.21.15.76:0/1977086009 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe000c9a0 con 0x7f4fec15d100 2023-11-09T03:52:48.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.178+0000 7f4fd37fe640 1 -- 172.21.15.76:0/1977086009 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe000cd50 con 0x7f4fec15d100 2023-11-09T03:52:48.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.178+0000 7f4ff1087640 1 -- 172.21.15.76:0/1977086009 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fec0b58f0 con 0x7f4fec15d100 2023-11-09T03:52:48.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.178+0000 7f4ff1087640 1 -- 172.21.15.76:0/1977086009 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fec0b5eb0 con 0x7f4fec15d100 2023-11-09T03:52:48.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.179+0000 7f4ff1087640 1 -- 172.21.15.76:0/1977086009 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fec0684b0 con 0x7f4fec15d100 2023-11-09T03:52:48.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.179+0000 7f4fd37fe640 1 -- 172.21.15.76:0/1977086009 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4fe0013470 con 0x7f4fec15d100 2023-11-09T03:52:48.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.180+0000 7f4fd37fe640 1 --2- 172.21.15.76:0/1977086009 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4fd806ae70 0x7f4fd806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:48.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.180+0000 7f4fd37fe640 1 -- 172.21.15.76:0/1977086009 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7952+0+0 (secure 0 0 0) 0x7f4fe008a1c0 con 0x7f4fec15d100 2023-11-09T03:52:48.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.180+0000 7f4fead9d640 1 --2- 172.21.15.76:0/1977086009 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4fd806ae70 0x7f4fd806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:48.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.181+0000 7f4fead9d640 1 --2- 172.21.15.76:0/1977086009 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4fd806ae70 0x7f4fd806d330 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc035000 tx=0x7f4fdc030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:48.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.184+0000 7f4fd37fe640 1 -- 172.21.15.76:0/1977086009 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4fe0054970 con 0x7f4fec15d100 2023-11-09T03:52:48.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.348+0000 7f4ff1087640 1 -- 172.21.15.76:0/1977086009 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4fec0d2cd0 con 0x7f4fec15d100 2023-11-09T03:52:48.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.349+0000 7f4fd37fe640 1 -- 172.21.15.76:0/1977086009 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4fe0017020 con 0x7f4fec15d100 2023-11-09T03:52:48.349 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:52:48.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.350+0000 7f4ff1087640 1 -- 172.21.15.76:0/1977086009 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4fd806ae70 msgr2=0x7f4fd806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:48.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.350+0000 7f4ff1087640 1 --2- 172.21.15.76:0/1977086009 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4fd806ae70 0x7f4fd806d330 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc035000 tx=0x7f4fdc030000 comp rx=0 tx=0).stop 2023-11-09T03:52:48.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.350+0000 7f4ff1087640 1 -- 172.21.15.76:0/1977086009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fec15d100 msgr2=0x7f4fec0b4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:48.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.350+0000 7f4ff1087640 1 --2- 172.21.15.76:0/1977086009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fec15d100 0x7f4fec0b4950 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000e9b0 tx=0x7f4fe000ee70 comp rx=0 tx=0).stop 2023-11-09T03:52:48.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.351+0000 7f4ff1087640 1 -- 172.21.15.76:0/1977086009 shutdown_connections 2023-11-09T03:52:48.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.351+0000 7f4ff1087640 1 --2- 172.21.15.76:0/1977086009 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4fd806ae70 0x7f4fd806d330 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.351+0000 7f4ff1087640 1 --2- 172.21.15.76:0/1977086009 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fec15d4e0 0x7f4fec0b6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.351+0000 7f4ff1087640 1 --2- 172.21.15.76:0/1977086009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fec15d100 0x7f4fec0b4950 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.351+0000 7f4ff1087640 1 --2- 172.21.15.76:0/1977086009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fec0d3b60 0x7f4fec152380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.351+0000 7f4ff1087640 1 -- 172.21.15.76:0/1977086009 >> 172.21.15.76:0/1977086009 conn(0x7f4fec0d2ef0 msgr2=0x7f4fec0b01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:48.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.351+0000 7f4ff1087640 1 -- 172.21.15.76:0/1977086009 shutdown_connections 2023-11-09T03:52:48.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.351+0000 7f4ff1087640 1 -- 172.21.15.76:0/1977086009 wait complete. 2023-11-09T03:52:48.361 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:52:48.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.465+0000 7fa8db3c9640 1 Processor -- start 2023-11-09T03:52:48.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.465+0000 7fa8db3c9640 1 -- start start 2023-11-09T03:52:48.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.466+0000 7fa8db3c9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa8d415d100 0x7fa8d40ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:48.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.466+0000 7fa8db3c9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa8d415d4e0 0x7fa8d40d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:48.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.466+0000 7fa8db3c9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8d40b32b0 con 0x7fa8d40d3b60 2023-11-09T03:52:48.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.466+0000 7fa8db3c9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8d40cee10 con 0x7fa8d415d4e0 2023-11-09T03:52:48.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.466+0000 7fa8db3c9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8d40b4750 con 0x7fa8d415d100 2023-11-09T03:52:48.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.466+0000 7fa8d8964640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa8d415d4e0 0x7fa8d40d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:48.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.466+0000 7fa8d9966640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa8d40d3b60 0x7fa8d40cf520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45208/0 (socket says 172.21.15.76:45208) 2023-11-09T03:52:48.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.466+0000 7fa8d9966640 1 -- 172.21.15.76:0/996487231 learned_addr learned my addr 172.21.15.76:0/996487231 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:52:48.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.466+0000 7fa8d8964640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa8d415d4e0 0x7fa8d40d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38302/0 (socket says 172.21.15.76:38302) 2023-11-09T03:52:48.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.467+0000 7fa8d9165640 1 --2- 172.21.15.76:0/996487231 >> v2:172.21.15.113:3300/0 conn(0x7fa8d415d100 0x7fa8d40ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:48.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.467+0000 7fa8d8964640 1 -- 172.21.15.76:0/996487231 >> v2:172.21.15.113:3300/0 conn(0x7fa8d415d100 msgr2=0x7fa8d40ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:48.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.467+0000 7fa8d8964640 1 --2- 172.21.15.76:0/996487231 >> v2:172.21.15.113:3300/0 conn(0x7fa8d415d100 0x7fa8d40ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.467+0000 7fa8cbfff640 1 -- 172.21.15.76:0/996487231 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1476539684 0 0) 0x7fa8d40b32b0 con 0x7fa8d40d3b60 2023-11-09T03:52:48.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.467+0000 7fa8d8964640 1 -- 172.21.15.76:0/996487231 >> v1:172.21.15.76:6789/0 conn(0x7fa8d40d3b60 legacy=0x7fa8d40cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:48.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.467+0000 7fa8d8964640 1 -- 172.21.15.76:0/996487231 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d40b2980 con 0x7fa8d415d4e0 2023-11-09T03:52:48.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.467+0000 7fa8d8964640 1 --2- 172.21.15.76:0/996487231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8d415d4e0 0x7fa8d40d14d0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fa8d40d09f0 tx=0x7fa8c402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eccdd4fa477f434d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:48.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.468+0000 7fa8cbfff640 1 -- 172.21.15.76:0/996487231 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4031830 con 0x7fa8d415d4e0 2023-11-09T03:52:48.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.468+0000 7fa8cbfff640 1 -- 172.21.15.76:0/996487231 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c4031990 con 0x7fa8d415d4e0 2023-11-09T03:52:48.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.468+0000 7fa8cbfff640 1 -- 172.21.15.76:0/996487231 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4031d20 con 0x7fa8d415d4e0 2023-11-09T03:52:48.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.468+0000 7fa8db3c9640 1 -- 172.21.15.76:0/996487231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8d415d4e0 msgr2=0x7fa8d40d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:48.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.468+0000 7fa8db3c9640 1 --2- 172.21.15.76:0/996487231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8d415d4e0 0x7fa8d40d14d0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fa8d40d09f0 tx=0x7fa8c402b8d0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.468+0000 7fa8db3c9640 1 -- 172.21.15.76:0/996487231 shutdown_connections 2023-11-09T03:52:48.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.468+0000 7fa8db3c9640 1 --2- 172.21.15.76:0/996487231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8d415d4e0 0x7fa8d40d14d0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.468+0000 7fa8db3c9640 1 --2- 172.21.15.76:0/996487231 >> v2:172.21.15.113:3300/0 conn(0x7fa8d415d100 0x7fa8d40ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.468+0000 7fa8db3c9640 1 -- 172.21.15.76:0/996487231 >> 172.21.15.76:0/996487231 conn(0x7fa8d40d2ef0 msgr2=0x7fa8d40d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:48.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.468+0000 7fa8db3c9640 1 -- 172.21.15.76:0/996487231 shutdown_connections 2023-11-09T03:52:48.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.468+0000 7fa8db3c9640 1 -- 172.21.15.76:0/996487231 wait complete. 2023-11-09T03:52:48.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.469+0000 7fa8db3c9640 1 Processor -- start 2023-11-09T03:52:48.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.469+0000 7fa8db3c9640 1 -- start start 2023-11-09T03:52:48.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.470+0000 7fa8db3c9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8d40d3b60 0x7fa8d4151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:48.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.470+0000 7fa8db3c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8d415d100 0x7fa8d40cb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:48.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.470+0000 7fa8db3c9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8d415d4e0 0x7fa8d40b7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:48.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.470+0000 7fa8db3c9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d41519b0 con 0x7fa8d415d100 2023-11-09T03:52:48.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.470+0000 7fa8db3c9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d40cce10 con 0x7fa8d40d3b60 2023-11-09T03:52:48.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.470+0000 7fa8db3c9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d40cb5a0 con 0x7fa8d415d4e0 2023-11-09T03:52:48.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.470+0000 7fa8d8964640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8d415d100 0x7fa8d40cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:48.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.470+0000 7fa8d8964640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8d415d100 0x7fa8d40cb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38318/0 (socket says 172.21.15.76:38318) 2023-11-09T03:52:48.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.470+0000 7fa8d8964640 1 -- 172.21.15.76:0/2048171659 learned_addr learned my addr 172.21.15.76:0/2048171659 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:48.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.470+0000 7fa8d8964640 1 -- 172.21.15.76:0/2048171659 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8d415d4e0 msgr2=0x7fa8d40b7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:48.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.470+0000 7fa8d8964640 1 --2- 172.21.15.76:0/2048171659 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8d415d4e0 0x7fa8d40b7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.470+0000 7fa8d9165640 1 --2- 172.21.15.76:0/2048171659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8d40d3b60 0x7fa8d4151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:48.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.470+0000 7fa8d8964640 1 -- 172.21.15.76:0/2048171659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8d40d3b60 msgr2=0x7fa8d4151470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:48.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.470+0000 7fa8d8964640 1 --2- 172.21.15.76:0/2048171659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8d40d3b60 0x7fa8d4151470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.470+0000 7fa8d8964640 1 -- 172.21.15.76:0/2048171659 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c4009400 con 0x7fa8d415d100 2023-11-09T03:52:48.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.470+0000 7fa8d9165640 1 --2- 172.21.15.76:0/2048171659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8d40d3b60 0x7fa8d4151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:52:48.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.471+0000 7fa8d8964640 1 --2- 172.21.15.76:0/2048171659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8d415d100 0x7fa8d40cb060 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4009890 tx=0x7fa8c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:48.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.471+0000 7fa8c9ffb640 1 -- 172.21.15.76:0/2048171659 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c403ba70 con 0x7fa8d415d100 2023-11-09T03:52:48.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.471+0000 7fa8c9ffb640 1 -- 172.21.15.76:0/2048171659 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c403bbd0 con 0x7fa8d415d100 2023-11-09T03:52:48.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.471+0000 7fa8db3c9640 1 -- 172.21.15.76:0/2048171659 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d40c9fc0 con 0x7fa8d415d100 2023-11-09T03:52:48.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.471+0000 7fa8c9ffb640 1 -- 172.21.15.76:0/2048171659 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c40453f0 con 0x7fa8d415d100 2023-11-09T03:52:48.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.471+0000 7fa8db3c9640 1 -- 172.21.15.76:0/2048171659 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d40ca580 con 0x7fa8d415d100 2023-11-09T03:52:48.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.472+0000 7fa8db3c9640 1 -- 172.21.15.76:0/2048171659 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8d40684b0 con 0x7fa8d415d100 2023-11-09T03:52:48.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.472+0000 7fa8c9ffb640 1 -- 172.21.15.76:0/2048171659 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa8c4045550 con 0x7fa8d415d100 2023-11-09T03:52:48.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.473+0000 7fa8c9ffb640 1 --2- 172.21.15.76:0/2048171659 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa8b806ae70 0x7fa8b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:48.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.473+0000 7fa8c9ffb640 1 -- 172.21.15.76:0/2048171659 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7952+0+0 (secure 0 0 0) 0x7fa8c40ac220 con 0x7fa8d415d100 2023-11-09T03:52:48.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.473+0000 7fa8d9165640 1 --2- 172.21.15.76:0/2048171659 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa8b806ae70 0x7fa8b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:48.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.473+0000 7fa8d9165640 1 --2- 172.21.15.76:0/2048171659 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa8b806ae70 0x7fa8b806d330 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa8d40b8a70 tx=0x7fa8d0007470 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:48.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.477+0000 7fa8c9ffb640 1 -- 172.21.15.76:0/2048171659 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa8c4076b30 con 0x7fa8d415d100 2023-11-09T03:52:48.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.643+0000 7fa8db3c9640 1 -- 172.21.15.76:0/2048171659 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa8d40d2cd0 con 0x7fa8d415d100 2023-11-09T03:52:48.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.644+0000 7fa8c9ffb640 1 -- 172.21.15.76:0/2048171659 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa8c4035020 con 0x7fa8d415d100 2023-11-09T03:52:48.644 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:52:48.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.646+0000 7fa8db3c9640 1 -- 172.21.15.76:0/2048171659 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa8b806ae70 msgr2=0x7fa8b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:48.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.646+0000 7fa8db3c9640 1 --2- 172.21.15.76:0/2048171659 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa8b806ae70 0x7fa8b806d330 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa8d40b8a70 tx=0x7fa8d0007470 comp rx=0 tx=0).stop 2023-11-09T03:52:48.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.646+0000 7fa8db3c9640 1 -- 172.21.15.76:0/2048171659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8d415d100 msgr2=0x7fa8d40cb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:48.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.646+0000 7fa8db3c9640 1 --2- 172.21.15.76:0/2048171659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8d415d100 0x7fa8d40cb060 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4009890 tx=0x7fa8c40046c0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.646+0000 7fa8db3c9640 1 -- 172.21.15.76:0/2048171659 shutdown_connections 2023-11-09T03:52:48.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.646+0000 7fa8db3c9640 1 --2- 172.21.15.76:0/2048171659 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa8b806ae70 0x7fa8b806d330 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.646+0000 7fa8db3c9640 1 --2- 172.21.15.76:0/2048171659 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8d415d4e0 0x7fa8d40b7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.646+0000 7fa8db3c9640 1 --2- 172.21.15.76:0/2048171659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8d415d100 0x7fa8d40cb060 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.646+0000 7fa8db3c9640 1 --2- 172.21.15.76:0/2048171659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8d40d3b60 0x7fa8d4151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.646+0000 7fa8db3c9640 1 -- 172.21.15.76:0/2048171659 >> 172.21.15.76:0/2048171659 conn(0x7fa8d40d2ef0 msgr2=0x7fa8d40d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:48.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.646+0000 7fa8db3c9640 1 -- 172.21.15.76:0/2048171659 shutdown_connections 2023-11-09T03:52:48.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.646+0000 7fa8db3c9640 1 -- 172.21.15.76:0/2048171659 wait complete. 2023-11-09T03:52:48.656 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-09T03:52:48.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.757+0000 7f4515790640 1 Processor -- start 2023-11-09T03:52:48.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.757+0000 7f4515790640 1 -- start start 2023-11-09T03:52:48.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.757+0000 7f4515790640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f451015d100 0x7f45100ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:48.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.757+0000 7f4515790640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45100d1be0 0x7f45100b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:48.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.757+0000 7f4515790640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45100b32b0 con 0x7f451015d4e0 2023-11-09T03:52:48.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.757+0000 7f4515790640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45100cee10 con 0x7f451015d100 2023-11-09T03:52:48.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.757+0000 7f4515790640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45100b2980 con 0x7f45100d1be0 2023-11-09T03:52:48.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.758+0000 7f450effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f451015d100 0x7f45100ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:48.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.758+0000 7f450f7fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45100d1be0 0x7f45100b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:48.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.758+0000 7f450effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f451015d100 0x7f45100ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:37428/0 (socket says 172.21.15.76:37428) 2023-11-09T03:52:48.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.758+0000 7f450effd640 1 -- 172.21.15.76:0/3139713125 learned_addr learned my addr 172.21.15.76:0/3139713125 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:48.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.758+0000 7f450effd640 1 -- 172.21.15.76:0/3139713125 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45100d1be0 msgr2=0x7f45100b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:48.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.758+0000 7f450effd640 1 --2- 172.21.15.76:0/3139713125 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45100d1be0 0x7f45100b3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.758+0000 7f450effd640 1 -- 172.21.15.76:0/3139713125 >> v1:172.21.15.113:6789/0 conn(0x7f451015d4e0 legacy=0x7f45100d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:48.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.758+0000 7f450effd640 1 -- 172.21.15.76:0/3139713125 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45100b2c60 con 0x7f451015d100 2023-11-09T03:52:48.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.758+0000 7f450f7fe640 1 --2- 172.21.15.76:0/3139713125 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45100d1be0 0x7f45100b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:52:48.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.758+0000 7f450dffb640 1 -- 172.21.15.76:0/3139713125 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1206676721 0 0) 0x7f45100b32b0 con 0x7f451015d4e0 2023-11-09T03:52:48.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.759+0000 7f450effd640 1 --2- 172.21.15.76:0/3139713125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f451015d100 0x7f45100ce510 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4500009ea0 tx=0x7f450002bfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb809c76e341e31a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:48.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.759+0000 7f450dffb640 1 -- 172.21.15.76:0/3139713125 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450002cb40 con 0x7f451015d100 2023-11-09T03:52:48.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.759+0000 7f450dffb640 1 -- 172.21.15.76:0/3139713125 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f450002cca0 con 0x7f451015d100 2023-11-09T03:52:48.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.759+0000 7f450dffb640 1 -- 172.21.15.76:0/3139713125 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4500034610 con 0x7f451015d100 2023-11-09T03:52:48.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.759+0000 7f4515790640 1 -- 172.21.15.76:0/3139713125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f451015d100 msgr2=0x7f45100ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:48.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.759+0000 7f4515790640 1 --2- 172.21.15.76:0/3139713125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f451015d100 0x7f45100ce510 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4500009ea0 tx=0x7f450002bfa0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.759+0000 7f4515790640 1 -- 172.21.15.76:0/3139713125 shutdown_connections 2023-11-09T03:52:48.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.759+0000 7f4515790640 1 --2- 172.21.15.76:0/3139713125 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45100d1be0 0x7f45100b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.759+0000 7f4515790640 1 --2- 172.21.15.76:0/3139713125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f451015d100 0x7f45100ce510 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.759+0000 7f4515790640 1 -- 172.21.15.76:0/3139713125 >> 172.21.15.76:0/3139713125 conn(0x7f45100d2ef0 msgr2=0x7f45100d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:48.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.760+0000 7f4515790640 1 -- 172.21.15.76:0/3139713125 shutdown_connections 2023-11-09T03:52:48.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.760+0000 7f4515790640 1 -- 172.21.15.76:0/3139713125 wait complete. 2023-11-09T03:52:48.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.761+0000 7f4515790640 1 Processor -- start 2023-11-09T03:52:48.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.761+0000 7f4515790640 1 -- start start 2023-11-09T03:52:48.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.762+0000 7f4515790640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45100d1be0 0x7f4510150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:48.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.762+0000 7f4515790640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f451015d100 0x7f45100cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:48.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.762+0000 7f4515790640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f451015d4e0 0x7f451015ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:48.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.762+0000 7f4515790640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4510151440 con 0x7f45100d1be0 2023-11-09T03:52:48.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.762+0000 7f4515790640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45100b5b10 con 0x7f451015d4e0 2023-11-09T03:52:48.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.762+0000 7f4515790640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45100b5d80 con 0x7f451015d100 2023-11-09T03:52:48.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.762+0000 7f450effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45100d1be0 0x7f4510150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:48.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.762+0000 7f450effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45100d1be0 0x7f4510150f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38332/0 (socket says 172.21.15.76:38332) 2023-11-09T03:52:48.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.762+0000 7f450effd640 1 -- 172.21.15.76:0/3318520154 learned_addr learned my addr 172.21.15.76:0/3318520154 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:48.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.762+0000 7f450f7fe640 1 --2- 172.21.15.76:0/3318520154 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f451015d4e0 0x7f451015ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:48.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.762+0000 7f450e7fc640 1 --2- 172.21.15.76:0/3318520154 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f451015d100 0x7f45100cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:48.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.762+0000 7f450effd640 1 -- 172.21.15.76:0/3318520154 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f451015d100 msgr2=0x7f45100cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:48.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.762+0000 7f450effd640 1 --2- 172.21.15.76:0/3318520154 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f451015d100 0x7f45100cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.762+0000 7f450effd640 1 -- 172.21.15.76:0/3318520154 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f451015d4e0 msgr2=0x7f451015ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:48.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.762+0000 7f450effd640 1 --2- 172.21.15.76:0/3318520154 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f451015d4e0 0x7f451015ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.762+0000 7f450effd640 1 -- 172.21.15.76:0/3318520154 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4508009400 con 0x7f45100d1be0 2023-11-09T03:52:48.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.762+0000 7f450effd640 1 --2- 172.21.15.76:0/3318520154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45100d1be0 0x7f4510150f00 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f4500036040 tx=0x7f4500004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:48.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.762+0000 7f44f77fe640 1 -- 172.21.15.76:0/3318520154 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4500009510 con 0x7f45100d1be0 2023-11-09T03:52:48.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.762+0000 7f44f77fe640 1 -- 172.21.15.76:0/3318520154 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f450003e450 con 0x7f45100d1be0 2023-11-09T03:52:48.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.762+0000 7f44f77fe640 1 -- 172.21.15.76:0/3318520154 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450003e800 con 0x7f45100d1be0 2023-11-09T03:52:48.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.762+0000 7f4515790640 1 -- 172.21.15.76:0/3318520154 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4500009b30 con 0x7f45100d1be0 2023-11-09T03:52:48.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.762+0000 7f4515790640 1 -- 172.21.15.76:0/3318520154 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45100b6340 con 0x7f45100d1be0 2023-11-09T03:52:48.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.763+0000 7f4515790640 1 -- 172.21.15.76:0/3318520154 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45100684b0 con 0x7f45100d1be0 2023-11-09T03:52:48.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.764+0000 7f44f77fe640 1 -- 172.21.15.76:0/3318520154 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f450003e960 con 0x7f45100d1be0 2023-11-09T03:52:48.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.764+0000 7f44f77fe640 1 --2- 172.21.15.76:0/3318520154 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f44fc06ae70 0x7f44fc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:48.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.764+0000 7f44f77fe640 1 -- 172.21.15.76:0/3318520154 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7952+0+0 (secure 0 0 0) 0x7f45000acb30 con 0x7f45100d1be0 2023-11-09T03:52:48.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.765+0000 7f450e7fc640 1 --2- 172.21.15.76:0/3318520154 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f44fc06ae70 0x7f44fc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:48.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.765+0000 7f450e7fc640 1 --2- 172.21.15.76:0/3318520154 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f44fc06ae70 0x7f44fc06d330 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f45100ca5e0 tx=0x7f4504009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:48.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.768+0000 7f44f77fe640 1 -- 172.21.15.76:0/3318520154 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4500077330 con 0x7f45100d1be0 2023-11-09T03:52:48.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.945+0000 7f4515790640 1 -- 172.21.15.76:0/3318520154 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f45100d2cd0 con 0x7f45100d1be0 2023-11-09T03:52:48.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.945+0000 7f44f77fe640 1 -- 172.21.15.76:0/3318520154 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 18 entries v109) v1 ==== 81+0+1093 (secure 0 0 0) 0x7f4500043460 con 0x7f45100d1be0 2023-11-09T03:52:48.945 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/3087804343 2023-11-09T04:52:42.513556+0000 2023-11-09T03:52:48.946 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6833/1536527256 2023-11-10T03:52:34.213804+0000 2023-11-09T03:52:48.946 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/445306598 2023-11-10T03:52:27.406838+0000 2023-11-09T03:52:48.946 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/3229239133 2023-11-10T03:51:32.624856+0000 2023-11-09T03:52:48.946 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/503424885 2023-11-09T04:51:47.727422+0000 2023-11-09T03:52:48.946 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/4011275120 2023-11-10T03:49:42.029057+0000 2023-11-09T03:52:48.947 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/4011275120 2023-11-10T03:49:42.029057+0000 2023-11-09T03:52:48.947 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/3280098776 2023-11-10T03:50:07.616780+0000 2023-11-09T03:52:48.947 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/3280098776 2023-11-10T03:50:07.616780+0000 2023-11-09T03:52:48.947 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/1663988060 2023-11-10T03:50:33.665092+0000 2023-11-09T03:52:48.947 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/1663988060 2023-11-10T03:50:33.665092+0000 2023-11-09T03:52:48.947 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/445306598 2023-11-10T03:52:27.406838+0000 2023-11-09T03:52:48.948 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6833/3697850101 2023-11-10T03:51:39.442929+0000 2023-11-09T03:52:48.948 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/3449910842 2023-11-10T03:50:59.611256+0000 2023-11-09T03:52:48.948 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/3229239133 2023-11-10T03:51:32.624856+0000 2023-11-09T03:52:48.948 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6832/1536527256 2023-11-10T03:52:34.213804+0000 2023-11-09T03:52:48.948 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/3449910842 2023-11-10T03:50:59.611256+0000 2023-11-09T03:52:48.948 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6832/3697850101 2023-11-10T03:51:39.442929+0000 2023-11-09T03:52:48.949 INFO:teuthology.orchestra.run.smithi076.stderr:listed 18 entries 2023-11-09T03:52:48.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.947+0000 7f4515790640 1 -- 172.21.15.76:0/3318520154 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f44fc06ae70 msgr2=0x7f44fc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:48.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.947+0000 7f4515790640 1 --2- 172.21.15.76:0/3318520154 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f44fc06ae70 0x7f44fc06d330 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f45100ca5e0 tx=0x7f4504009040 comp rx=0 tx=0).stop 2023-11-09T03:52:48.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.947+0000 7f4515790640 1 -- 172.21.15.76:0/3318520154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45100d1be0 msgr2=0x7f4510150f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:48.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.947+0000 7f4515790640 1 --2- 172.21.15.76:0/3318520154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45100d1be0 0x7f4510150f00 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f4500036040 tx=0x7f4500004a40 comp rx=0 tx=0).stop 2023-11-09T03:52:48.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.947+0000 7f4515790640 1 -- 172.21.15.76:0/3318520154 shutdown_connections 2023-11-09T03:52:48.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.947+0000 7f4515790640 1 --2- 172.21.15.76:0/3318520154 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f44fc06ae70 0x7f44fc06d330 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.947+0000 7f4515790640 1 --2- 172.21.15.76:0/3318520154 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f451015d4e0 0x7f451015ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.947+0000 7f4515790640 1 --2- 172.21.15.76:0/3318520154 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f451015d100 0x7f45100cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.947+0000 7f4515790640 1 --2- 172.21.15.76:0/3318520154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45100d1be0 0x7f4510150f00 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:48.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.947+0000 7f4515790640 1 -- 172.21.15.76:0/3318520154 >> 172.21.15.76:0/3318520154 conn(0x7f45100d2ef0 msgr2=0x7f45100d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:48.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.947+0000 7f4515790640 1 -- 172.21.15.76:0/3318520154 shutdown_connections 2023-11-09T03:52:48.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:48.947+0000 7f4515790640 1 -- 172.21.15.76:0/3318520154 wait complete. 2023-11-09T03:52:48.957 DEBUG:tasks.cephfs.mount:Force/lazy unmounting on client.0 2023-11-09T03:52:48.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:52:48.958 DEBUG:teuthology.orchestra.run.smithi076:> sudo umount --lazy --force /home/ubuntu/cephtest/mnt.0 2023-11-09T03:52:48.984 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[123533]: fuse finished with error 0 and tester_r 0 2023-11-09T03:52:48.985 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T03:52:48.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:52:48.986 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-09T03:52:49.040 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:52:49.040 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:52:49.095 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056381,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14761 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248868k,nr_inodes=812217,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-09T03:52:49.096 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-09T03:52:49.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.235+0000 7f6e888fc640 1 Processor -- start 2023-11-09T03:52:49.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.235+0000 7f6e888fc640 1 -- start start 2023-11-09T03:52:49.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.236+0000 7f6e888fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6e840b4160 0x7f6e840d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:49.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.236+0000 7f6e888fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e840b64b0 0x7f6e840b6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:49.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.236+0000 7f6e888fc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e840b5d40 con 0x7f6e840b4540 2023-11-09T03:52:49.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.236+0000 7f6e888fc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e840d07d0 con 0x7f6e840b4160 2023-11-09T03:52:49.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.236+0000 7f6e888fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e840ce8d0 con 0x7f6e840b64b0 2023-11-09T03:52:49.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.236+0000 7f6e8259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6e840b4160 0x7f6e840d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:49.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.236+0000 7f6e82d9d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e840b64b0 0x7f6e840b6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:49.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.236+0000 7f6e82d9d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e840b64b0 0x7f6e840b6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:35268/0 (socket says 172.21.15.76:35268) 2023-11-09T03:52:49.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.236+0000 7f6e81d9b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6e840b4540 0x7f6e840d1330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:48218/0 (socket says 172.21.15.76:48218) 2023-11-09T03:52:49.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.236+0000 7f6e8259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6e840b4160 0x7f6e840d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:37444/0 (socket says 172.21.15.76:37444) 2023-11-09T03:52:49.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.236+0000 7f6e82d9d640 1 -- 172.21.15.76:0/1301883364 learned_addr learned my addr 172.21.15.76:0/1301883364 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:49.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.237+0000 7f6e82d9d640 1 -- 172.21.15.76:0/1301883364 >> v2:172.21.15.113:3300/0 conn(0x7f6e840b4160 msgr2=0x7f6e840d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:49.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.237+0000 7f6e82d9d640 1 --2- 172.21.15.76:0/1301883364 >> v2:172.21.15.113:3300/0 conn(0x7f6e840b4160 0x7f6e840d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:49.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.237+0000 7f6e82d9d640 1 -- 172.21.15.76:0/1301883364 >> v1:172.21.15.113:6789/0 conn(0x7f6e840b4540 legacy=0x7f6e840d1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:49.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.237+0000 7f6e82d9d640 1 -- 172.21.15.76:0/1301883364 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e840cebb0 con 0x7f6e840b64b0 2023-11-09T03:52:49.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.237+0000 7f6e8259c640 1 --2- 172.21.15.76:0/1301883364 >> v2:172.21.15.113:3300/0 conn(0x7f6e840b4160 0x7f6e840d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:52:49.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.238+0000 7f6e82d9d640 1 --2- 172.21.15.76:0/1301883364 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e840b64b0 0x7f6e840b6930 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c009e50 tx=0x7f6e7c02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d086124be221945a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:49.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.238+0000 7f6e8159a640 1 -- 172.21.15.76:0/1301883364 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c02ca70 con 0x7f6e840b64b0 2023-11-09T03:52:49.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.238+0000 7f6e8159a640 1 -- 172.21.15.76:0/1301883364 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e7c02cbd0 con 0x7f6e840b64b0 2023-11-09T03:52:49.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.238+0000 7f6e8159a640 1 -- 172.21.15.76:0/1301883364 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c0343f0 con 0x7f6e840b64b0 2023-11-09T03:52:49.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.238+0000 7f6e888fc640 1 -- 172.21.15.76:0/1301883364 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e840b64b0 msgr2=0x7f6e840b6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:49.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.238+0000 7f6e888fc640 1 --2- 172.21.15.76:0/1301883364 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e840b64b0 0x7f6e840b6930 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c009e50 tx=0x7f6e7c02bed0 comp rx=0 tx=0).stop 2023-11-09T03:52:49.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.238+0000 7f6e888fc640 1 -- 172.21.15.76:0/1301883364 shutdown_connections 2023-11-09T03:52:49.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.238+0000 7f6e888fc640 1 --2- 172.21.15.76:0/1301883364 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e840b64b0 0x7f6e840b6930 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:49.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.238+0000 7f6e888fc640 1 --2- 172.21.15.76:0/1301883364 >> v2:172.21.15.113:3300/0 conn(0x7f6e840b4160 0x7f6e840d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:49.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.238+0000 7f6e888fc640 1 -- 172.21.15.76:0/1301883364 >> 172.21.15.76:0/1301883364 conn(0x7f6e840d2ef0 msgr2=0x7f6e840d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:49.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.238+0000 7f6e888fc640 1 -- 172.21.15.76:0/1301883364 shutdown_connections 2023-11-09T03:52:49.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.238+0000 7f6e888fc640 1 -- 172.21.15.76:0/1301883364 wait complete. 2023-11-09T03:52:49.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.240+0000 7f6e888fc640 1 Processor -- start 2023-11-09T03:52:49.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.240+0000 7f6e888fc640 1 -- start start 2023-11-09T03:52:49.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.241+0000 7f6e888fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e840b4160 0x7f6e841f6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:49.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.241+0000 7f6e888fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e840b4540 0x7f6e840d1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:49.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.241+0000 7f6e888fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e840b64b0 0x7f6e840d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:49.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.241+0000 7f6e888fc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e841f6cc0 con 0x7f6e840b4540 2023-11-09T03:52:49.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.241+0000 7f6e888fc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e84162c10 con 0x7f6e840b64b0 2023-11-09T03:52:49.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.241+0000 7f6e888fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e840c9d80 con 0x7f6e840b4160 2023-11-09T03:52:49.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.241+0000 7f6e81d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e840b4540 0x7f6e840d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:49.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.241+0000 7f6e8259c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e840b4160 0x7f6e841f6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:49.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.241+0000 7f6e81d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e840b4540 0x7f6e840d1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38342/0 (socket says 172.21.15.76:38342) 2023-11-09T03:52:49.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.241+0000 7f6e81d9b640 1 -- 172.21.15.76:0/1163444067 learned_addr learned my addr 172.21.15.76:0/1163444067 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:49.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.241+0000 7f6e82d9d640 1 --2- 172.21.15.76:0/1163444067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e840b64b0 0x7f6e840d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:49.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.241+0000 7f6e81d9b640 1 -- 172.21.15.76:0/1163444067 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e840b4160 msgr2=0x7f6e841f6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:49.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.241+0000 7f6e81d9b640 1 --2- 172.21.15.76:0/1163444067 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e840b4160 0x7f6e841f6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:49.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.241+0000 7f6e81d9b640 1 -- 172.21.15.76:0/1163444067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e840b64b0 msgr2=0x7f6e840d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:49.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.241+0000 7f6e81d9b640 1 --2- 172.21.15.76:0/1163444067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e840b64b0 0x7f6e840d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:49.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.241+0000 7f6e81d9b640 1 -- 172.21.15.76:0/1163444067 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e780093a0 con 0x7f6e840b4540 2023-11-09T03:52:49.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.241+0000 7f6e81d9b640 1 --2- 172.21.15.76:0/1163444067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e840b4540 0x7f6e840d1260 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400c450 tx=0x7f6e7400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:49.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.241+0000 7f6e72ffd640 1 -- 172.21.15.76:0/1163444067 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74007480 con 0x7f6e840b4540 2023-11-09T03:52:49.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.241+0000 7f6e72ffd640 1 -- 172.21.15.76:0/1163444067 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e740075e0 con 0x7f6e840b4540 2023-11-09T03:52:49.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.241+0000 7f6e72ffd640 1 -- 172.21.15.76:0/1163444067 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74004a00 con 0x7f6e840b4540 2023-11-09T03:52:49.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.241+0000 7f6e888fc640 1 -- 172.21.15.76:0/1163444067 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e7c009b30 con 0x7f6e840b4540 2023-11-09T03:52:49.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.241+0000 7f6e888fc640 1 -- 172.21.15.76:0/1163444067 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e84162130 con 0x7f6e840b4540 2023-11-09T03:52:49.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.242+0000 7f6e888fc640 1 -- 172.21.15.76:0/1163444067 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e840684b0 con 0x7f6e840b4540 2023-11-09T03:52:49.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.243+0000 7f6e72ffd640 1 -- 172.21.15.76:0/1163444067 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6e74004b60 con 0x7f6e840b4540 2023-11-09T03:52:49.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.243+0000 7f6e72ffd640 1 --2- 172.21.15.76:0/1163444067 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6e6406ae70 0x7f6e6406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:49.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.243+0000 7f6e72ffd640 1 -- 172.21.15.76:0/1163444067 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7952+0+0 (secure 0 0 0) 0x7f6e74088fb0 con 0x7f6e840b4540 2023-11-09T03:52:49.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.243+0000 7f6e8259c640 1 --2- 172.21.15.76:0/1163444067 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6e6406ae70 0x7f6e6406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:49.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.244+0000 7f6e8259c640 1 --2- 172.21.15.76:0/1163444067 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6e6406ae70 0x7f6e6406d330 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6e840c6c70 tx=0x7f6e78009110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:49.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.247+0000 7f6e72ffd640 1 -- 172.21.15.76:0/1163444067 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6e740bbca0 con 0x7f6e840b4540 2023-11-09T03:52:49.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.450+0000 7f6e888fc640 1 -- 172.21.15.76:0/1163444067 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6e840ce270 con 0x7f6e840b4540 2023-11-09T03:52:49.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.450+0000 7f6e72ffd640 1 -- 172.21.15.76:0/1163444067 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v105) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6e74018070 con 0x7f6e840b4540 2023-11-09T03:52:49.450 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-09T03:52:49.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.452+0000 7f6e888fc640 1 -- 172.21.15.76:0/1163444067 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6e6406ae70 msgr2=0x7f6e6406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:49.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.452+0000 7f6e888fc640 1 --2- 172.21.15.76:0/1163444067 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6e6406ae70 0x7f6e6406d330 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6e840c6c70 tx=0x7f6e78009110 comp rx=0 tx=0).stop 2023-11-09T03:52:49.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.452+0000 7f6e888fc640 1 -- 172.21.15.76:0/1163444067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e840b4540 msgr2=0x7f6e840d1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:49.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.452+0000 7f6e888fc640 1 --2- 172.21.15.76:0/1163444067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e840b4540 0x7f6e840d1260 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400c450 tx=0x7f6e7400c910 comp rx=0 tx=0).stop 2023-11-09T03:52:49.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.452+0000 7f6e888fc640 1 -- 172.21.15.76:0/1163444067 shutdown_connections 2023-11-09T03:52:49.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.452+0000 7f6e888fc640 1 --2- 172.21.15.76:0/1163444067 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6e6406ae70 0x7f6e6406d330 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:49.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.452+0000 7f6e888fc640 1 --2- 172.21.15.76:0/1163444067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e840b64b0 0x7f6e840d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:49.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.452+0000 7f6e888fc640 1 --2- 172.21.15.76:0/1163444067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e840b4540 0x7f6e840d1260 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:49.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.452+0000 7f6e888fc640 1 --2- 172.21.15.76:0/1163444067 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e840b4160 0x7f6e841f6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:49.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.452+0000 7f6e888fc640 1 -- 172.21.15.76:0/1163444067 >> 172.21.15.76:0/1163444067 conn(0x7f6e840d2ef0 msgr2=0x7f6e8406b610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:49.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.452+0000 7f6e888fc640 1 -- 172.21.15.76:0/1163444067 shutdown_connections 2023-11-09T03:52:49.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:49.452+0000 7f6e888fc640 1 -- 172.21.15.76:0/1163444067 wait complete. 2023-11-09T03:52:49.463 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-09T03:52:49.464 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-09T03:52:49.464 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2023-11-09T03:52:49.464 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-09T03:52:49.464 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-09T03:52:49.465 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-09T03:52:49.465 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-09T03:52:49.465 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-09T03:52:49.466 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:52:49.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:52:49.466 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2023-11-09T03:52:49.481 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-09T03:52:49.482 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-09T03:52:49.482 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2023-11-09T03:52:49.482 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:52:49.482 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2023-11-09T03:52:49.482 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:52:49.483 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:52:49.483 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:52:49.483 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2023-11-09T03:52:49.483 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:52:49.483 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:52:49.484 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2023-11-09T03:52:49.484 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:52:49.484 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:52:49.484 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-09T03:52:49.484 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:52:49.485 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-09T03:52:49.485 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42179sec preferred_lft 42179sec 2023-11-09T03:52:49.485 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2023-11-09T03:52:49.485 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:52:49.485 INFO:teuthology.orchestra.run.smithi076.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-09T03:52:49.486 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2e:15:bd:03:a8:2e brd ff:ff:ff:ff:ff:ff 2023-11-09T03:52:49.486 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-09T03:52:49.486 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:52:49.486 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::bcda:38ff:fed4:fdb0/64 scope link 2023-11-09T03:52:49.486 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:52:49.486 INFO:teuthology.orchestra.run.smithi076.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-09T03:52:49.487 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 6a:8d:94:d7:e5:00 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:52:49.487 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::688d:94ff:fed7:e500/64 scope link 2023-11-09T03:52:49.487 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:52:49.487 INFO:teuthology.orchestra.run.smithi076.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-09T03:52:49.487 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2e:15:bd:03:a8:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:52:49.487 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::2c15:bdff:fe03:a82e/64 scope link 2023-11-09T03:52:49.488 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:52:49.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:52:49.488 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-09T03:52:49.548 INFO:teuthology.orchestra.run.smithi076.stdout:1 2023-11-09T03:52:49.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:52:49.550 DEBUG:teuthology.orchestra.run.smithi076:> route 2023-11-09T03:52:49.605 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2023-11-09T03:52:49.606 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-09T03:52:49.606 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-09T03:52:49.606 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-09T03:52:49.606 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-09T03:52:49.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:52:49.607 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-09T03:52:49.607 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-09T03:52:49.608 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-09T03:52:49.608 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-09T03:52:49.609 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-09T03:52:49.609 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-09T03:52:49.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:52:49.702 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2023-11-09T03:52:49.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-09T03:52:49.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-09T03:52:49.717 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:52:49.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:52:49.717 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2023-11-09T03:52:49.783 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-09T03:52:49.783 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:52:49.784 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-09T03:52:49.837 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:52:49.838 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:52:49.838 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-09T03:52:49.892 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:52:49.947 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:52:49.948 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:52:49.948 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:52:49.948 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:52:49.948 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:52:49.948 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:52:49.949 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:52:49.949 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:52:49.949 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:52:49.949 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:52:49.949 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:52:49.950 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:52:49.950 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:52:49.950 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:52:49.950 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:52:49.950 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:52:49.951 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:52:49.951 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:52:49.951 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:52:49.951 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:52:49.951 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:52:49.951 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:52:49.952 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:52:49.952 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:52:49.952 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:52:49.952 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:52:49.952 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:52:49.953 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:52:49.953 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:52:49.953 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:52:49.953 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:52:49.953 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:52:49.953 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:52:49.954 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:52:49.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:52:49.954 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:52:50.001 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-09T03:52:50.002 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-09T03:52:50.043 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:52:50.044 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:52:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:52:50.058 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:52:50.059 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:52:50.059 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:52:50.059 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:52:50.059 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:52:50.059 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:52:50.060 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:52:50.060 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:52:50.060 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:52:50.060 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:52:50.060 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:52:50.060 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:52:50.061 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:52:50.061 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:52:50.061 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:52:50.061 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:52:50.061 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:52:50.061 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:52:50.062 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:52:50.062 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:52:50.062 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:52:50.062 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:52:50.062 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:52:50.063 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:52:50.063 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:52:50.063 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:52:50.063 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:52:50.063 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:52:50.064 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:52:50.064 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:52:50.064 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:52:50.064 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:52:50.064 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:52:50.064 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:52:50.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:52:50.066 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:52:50.134 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:52:50.135+0000 7f684e641480 -1 init, newargv = 0x558914bada30 newargc=15 2023-11-09T03:52:50.134 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[125642]: starting ceph client 2023-11-09T03:52:50.141 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[125642]: starting fuse 2023-11-09T03:52:50.457 INFO:tasks.ceph.mds.a.smithi076.stderr:2023-11-09T03:52:50.458+0000 7fca00d3a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:52:50.457 INFO:tasks.ceph.mds.a.smithi076.stderr:2023-11-09T03:52:50.458+0000 7fca00d3a640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-09T03:52:51.221 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:52:51.240 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:52:51.240 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:52:51.241 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:52:51.241 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:52:51.241 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:52:51.241 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:52:51.241 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:52:51.241 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:52:51.242 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:52:51.242 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:52:51.242 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:52:51.242 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:52:51.243 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:52:51.243 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:52:51.243 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:52:51.243 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:52:51.243 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:52:51.243 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:52:51.244 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:52:51.244 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:52:51.244 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:52:51.244 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:52:51.244 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:52:51.245 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:52:51.245 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:52:51.245 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:52:51.245 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:52:51.245 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:52:51.245 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:52:51.246 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:52:51.246 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:52:51.246 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:52:51.246 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:52:51.246 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:52:51.246 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-09T03:52:51.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:52:51.248 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:52:51.304 INFO:teuthology.orchestra.run.smithi076.stdout:43 2023-11-09T03:52:51.305 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-09T03:52:51.305 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer -- python3 -c ' 2023-11-09T03:52:51.306 DEBUG:teuthology.orchestra.run.smithi076:> import glob 2023-11-09T03:52:51.306 DEBUG:teuthology.orchestra.run.smithi076:> import re 2023-11-09T03:52:51.306 DEBUG:teuthology.orchestra.run.smithi076:> import os 2023-11-09T03:52:51.307 DEBUG:teuthology.orchestra.run.smithi076:> import subprocess 2023-11-09T03:52:51.307 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:52:51.307 DEBUG:teuthology.orchestra.run.smithi076:> def _find_admin_socket(client_name): 2023-11-09T03:52:51.308 DEBUG:teuthology.orchestra.run.smithi076:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-09T03:52:51.308 DEBUG:teuthology.orchestra.run.smithi076:> files = glob.glob(asok_path) 2023-11-09T03:52:51.308 DEBUG:teuthology.orchestra.run.smithi076:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-09T03:52:51.309 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:52:51.309 DEBUG:teuthology.orchestra.run.smithi076:> # Given a non-glob path, it better be there 2023-11-09T03:52:51.310 DEBUG:teuthology.orchestra.run.smithi076:> if "*" not in asok_path: 2023-11-09T03:52:51.310 DEBUG:teuthology.orchestra.run.smithi076:> assert(len(files) == 1) 2023-11-09T03:52:51.310 DEBUG:teuthology.orchestra.run.smithi076:> return files[0] 2023-11-09T03:52:51.311 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:52:51.311 DEBUG:teuthology.orchestra.run.smithi076:> for f in files: 2023-11-09T03:52:51.311 DEBUG:teuthology.orchestra.run.smithi076:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-09T03:52:51.312 DEBUG:teuthology.orchestra.run.smithi076:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-09T03:52:51.312 DEBUG:teuthology.orchestra.run.smithi076:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-09T03:52:51.313 DEBUG:teuthology.orchestra.run.smithi076:> contents = proc_f.read() 2023-11-09T03:52:51.313 DEBUG:teuthology.orchestra.run.smithi076:> if mountpoint in contents: 2023-11-09T03:52:51.313 DEBUG:teuthology.orchestra.run.smithi076:> return f 2023-11-09T03:52:51.314 DEBUG:teuthology.orchestra.run.smithi076:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-09T03:52:51.314 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:52:51.314 DEBUG:teuthology.orchestra.run.smithi076:> print(_find_admin_socket("client.0")) 2023-11-09T03:52:51.315 DEBUG:teuthology.orchestra.run.smithi076:> ' 2023-11-09T03:52:51.437 INFO:teuthology.orchestra.run.smithi076.stdout:/var/run/ceph/ceph-client.0.125642.asok 2023-11-09T03:52:51.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:52:51.449 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.125642.asok 2023-11-09T03:52:51.450 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:52:51.450 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.125642.asok status 2023-11-09T03:52:51.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:52:51.620 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:52:51.620 INFO:teuthology.orchestra.run.smithi076.stdout: "metadata": { 2023-11-09T03:52:51.620 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_sha1": "7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4", 2023-11-09T03:52:51.620 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev)", 2023-11-09T03:52:51.621 INFO:teuthology.orchestra.run.smithi076.stdout: "entity_id": "0", 2023-11-09T03:52:51.621 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": "smithi076", 2023-11-09T03:52:51.621 INFO:teuthology.orchestra.run.smithi076.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-09T03:52:51.621 INFO:teuthology.orchestra.run.smithi076.stdout: "pid": "125642", 2023-11-09T03:52:51.621 INFO:teuthology.orchestra.run.smithi076.stdout: "root": "/" 2023-11-09T03:52:51.621 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:52:51.622 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_count": 0, 2023-11-09T03:52:51.622 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_pinned_count": 0, 2023-11-09T03:52:51.622 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 8682, 2023-11-09T03:52:51.622 INFO:teuthology.orchestra.run.smithi076.stdout: "inst": { 2023-11-09T03:52:51.622 INFO:teuthology.orchestra.run.smithi076.stdout: "name": { 2023-11-09T03:52:51.622 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "client", 2023-11-09T03:52:51.623 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 8682 2023-11-09T03:52:51.623 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:52:51.623 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2023-11-09T03:52:51.623 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2023-11-09T03:52:51.623 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2023-11-09T03:52:51.624 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 334503200 2023-11-09T03:52:51.624 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:52:51.624 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:52:51.624 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2023-11-09T03:52:51.624 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2023-11-09T03:52:51.624 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2023-11-09T03:52:51.625 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 334503200 2023-11-09T03:52:51.625 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:52:51.625 INFO:teuthology.orchestra.run.smithi076.stdout: "inst_str": "client.8682 192.168.0.1:0/334503200", 2023-11-09T03:52:51.625 INFO:teuthology.orchestra.run.smithi076.stdout: "addr_str": "192.168.0.1:0/334503200", 2023-11-09T03:52:51.625 INFO:teuthology.orchestra.run.smithi076.stdout: "inode_count": 1, 2023-11-09T03:52:51.626 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_epoch": 105, 2023-11-09T03:52:51.626 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch": 109, 2023-11-09T03:52:51.626 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch_barrier": 0, 2023-11-09T03:52:51.626 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklisted": false, 2023-11-09T03:52:51.626 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs" 2023-11-09T03:52:51.627 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:52:51.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:52:51.634 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:52:51.635 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-11-09T03:52:51.635 DEBUG:teuthology.orchestra.run.smithi076:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-09T03:52:51.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:52:51.694 INFO:teuthology.orchestra.run.smithi076.stdout:fuseblk 2023-11-09T03:52:51.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:52:51.701 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-09T03:52:51.701 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:52:51.702 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-09T03:52:51.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:52:51.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:52:51.767 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 flush journal 2023-11-09T03:52:51.869 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.870+0000 7f7b3eb25640 1 Processor -- start 2023-11-09T03:52:51.870 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.870+0000 7f7b3eb25640 1 -- start start 2023-11-09T03:52:51.871 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.870+0000 7f7b3eb25640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b380b0c40 0x7f7b3806fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:51.871 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.870+0000 7f7b3eb25640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7b3806bb60 0x7f7b38070530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:51.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.870+0000 7f7b3eb25640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b38071500 con 0x7f7b38159e40 2023-11-09T03:52:51.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.870+0000 7f7b3eb25640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b38070a70 con 0x7f7b3806bb60 2023-11-09T03:52:51.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.871+0000 7f7b3eb25640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3815dae0 con 0x7f7b380b0c40 2023-11-09T03:52:51.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.871+0000 7f7b3d0c2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7b3806bb60 0x7f7b38070530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:51.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.871+0000 7f7b3c8c1640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f7b38159e40 0x7f7b380b0530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45218/0 (socket says 172.21.15.76:45218) 2023-11-09T03:52:51.872 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.871+0000 7f7b3c8c1640 1 -- 172.21.15.76:0/2793045524 learned_addr learned my addr 172.21.15.76:0/2793045524 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:52:51.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.871+0000 7f7b37fff640 1 --2- 172.21.15.76:0/2793045524 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b380b0c40 0x7f7b3806fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:51.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.871+0000 7f7b3d0c2640 1 -- 172.21.15.76:0/2793045524 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b380b0c40 msgr2=0x7f7b3806fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:51.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.871+0000 7f7b3d0c2640 1 --2- 172.21.15.76:0/2793045524 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b380b0c40 0x7f7b3806fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.871+0000 7f7b3d0c2640 1 -- 172.21.15.76:0/2793045524 >> v1:172.21.15.76:6789/0 conn(0x7f7b38159e40 legacy=0x7f7b380b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:51.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.871+0000 7f7b3d0c2640 1 -- 172.21.15.76:0/2793045524 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b38070c60 con 0x7f7b3806bb60 2023-11-09T03:52:51.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.872+0000 7f7b3d0c2640 1 --2- 172.21.15.76:0/2793045524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b3806bb60 0x7f7b38070530 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f7b30009d90 tx=0x7f7b3002e020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742b40453665e3ff server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:51.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.872+0000 7f7b377fe640 1 -- 172.21.15.76:0/2793045524 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3002edd0 con 0x7f7b3806bb60 2023-11-09T03:52:51.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.872+0000 7f7b377fe640 1 -- 172.21.15.76:0/2793045524 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b30036440 con 0x7f7b3806bb60 2023-11-09T03:52:51.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.872+0000 7f7b377fe640 1 -- 172.21.15.76:0/2793045524 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b300367d0 con 0x7f7b3806bb60 2023-11-09T03:52:51.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.872+0000 7f7b3eb25640 1 -- 172.21.15.76:0/2793045524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b3806bb60 msgr2=0x7f7b38070530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:51.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.872+0000 7f7b3eb25640 1 --2- 172.21.15.76:0/2793045524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b3806bb60 0x7f7b38070530 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f7b30009d90 tx=0x7f7b3002e020 comp rx=0 tx=0).stop 2023-11-09T03:52:51.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.872+0000 7f7b3eb25640 1 -- 172.21.15.76:0/2793045524 shutdown_connections 2023-11-09T03:52:51.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.872+0000 7f7b3eb25640 1 --2- 172.21.15.76:0/2793045524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b3806bb60 0x7f7b38070530 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.878 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.872+0000 7f7b3eb25640 1 --2- 172.21.15.76:0/2793045524 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b380b0c40 0x7f7b3806fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.872+0000 7f7b3eb25640 1 -- 172.21.15.76:0/2793045524 >> 172.21.15.76:0/2793045524 conn(0x7f7b380cb530 msgr2=0x7f7b380cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:51.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.872+0000 7f7b3eb25640 1 -- 172.21.15.76:0/2793045524 shutdown_connections 2023-11-09T03:52:51.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.872+0000 7f7b3eb25640 1 -- 172.21.15.76:0/2793045524 wait complete. 2023-11-09T03:52:51.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.874+0000 7f7b3eb25640 1 Processor -- start 2023-11-09T03:52:51.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.874+0000 7f7b3eb25640 1 -- start start 2023-11-09T03:52:51.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.874+0000 7f7b3eb25640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b3806bb60 0x7f7b38154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:51.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.874+0000 7f7b3eb25640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b380b0c40 0x7f7b3815a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:51.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.874+0000 7f7b3eb25640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b38159e40 0x7f7b38157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:51.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.874+0000 7f7b3eb25640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b381586b0 con 0x7f7b380b0c40 2023-11-09T03:52:51.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.874+0000 7f7b3eb25640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b38158990 con 0x7f7b3806bb60 2023-11-09T03:52:51.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.874+0000 7f7b3eb25640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b38155520 con 0x7f7b38159e40 2023-11-09T03:52:51.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.875+0000 7f7b37fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b380b0c40 0x7f7b3815a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:51.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.875+0000 7f7b37fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b380b0c40 0x7f7b3815a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38366/0 (socket says 172.21.15.76:38366) 2023-11-09T03:52:51.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.875+0000 7f7b37fff640 1 -- 172.21.15.76:0/1049992597 learned_addr learned my addr 172.21.15.76:0/1049992597 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:51.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.875+0000 7f7b3c8c1640 1 --2- 172.21.15.76:0/1049992597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b3806bb60 0x7f7b38154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:51.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.875+0000 7f7b3d0c2640 1 --2- 172.21.15.76:0/1049992597 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b38159e40 0x7f7b38157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:51.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.875+0000 7f7b37fff640 1 -- 172.21.15.76:0/1049992597 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b38159e40 msgr2=0x7f7b38157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:51.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.875+0000 7f7b37fff640 1 --2- 172.21.15.76:0/1049992597 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b38159e40 0x7f7b38157b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.875+0000 7f7b37fff640 1 -- 172.21.15.76:0/1049992597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b3806bb60 msgr2=0x7f7b38154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:51.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.875+0000 7f7b37fff640 1 --2- 172.21.15.76:0/1049992597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b3806bb60 0x7f7b38154e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.875+0000 7f7b37fff640 1 -- 172.21.15.76:0/1049992597 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b30009a70 con 0x7f7b380b0c40 2023-11-09T03:52:51.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.875+0000 7f7b37fff640 1 --2- 172.21.15.76:0/1049992597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b380b0c40 0x7f7b3815a220 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400d800 tx=0x7f7b2400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:51.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.875+0000 7f7b357fa640 1 -- 172.21.15.76:0/1049992597 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2400b6c0 con 0x7f7b380b0c40 2023-11-09T03:52:51.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.875+0000 7f7b357fa640 1 -- 172.21.15.76:0/1049992597 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2400b820 con 0x7f7b380b0c40 2023-11-09T03:52:51.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.875+0000 7f7b3eb25640 1 -- 172.21.15.76:0/1049992597 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b38155800 con 0x7f7b380b0c40 2023-11-09T03:52:51.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.875+0000 7f7b357fa640 1 -- 172.21.15.76:0/1049992597 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2400bbb0 con 0x7f7b380b0c40 2023-11-09T03:52:51.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.875+0000 7f7b3eb25640 1 -- 172.21.15.76:0/1049992597 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b38158c70 con 0x7f7b380b0c40 2023-11-09T03:52:51.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.877+0000 7f7b357fa640 1 -- 172.21.15.76:0/1049992597 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7b2400bd10 con 0x7f7b380b0c40 2023-11-09T03:52:51.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.877+0000 7f7b357fa640 1 --2- 172.21.15.76:0/1049992597 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7b1c06ae70 0x7f7b1c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:51.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.877+0000 7f7b357fa640 1 -- 172.21.15.76:0/1049992597 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7952+0+0 (secure 0 0 0) 0x7f7b2408a3e0 con 0x7f7b380b0c40 2023-11-09T03:52:51.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.877+0000 7f7b3c8c1640 1 --2- 172.21.15.76:0/1049992597 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7b1c06ae70 0x7f7b1c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:51.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.878+0000 7f7b3c8c1640 1 --2- 172.21.15.76:0/1049992597 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7b1c06ae70 0x7f7b1c06d330 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c005880 tx=0x7f7b2c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:51.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.878+0000 7f7b41794740 1 Processor -- start 2023-11-09T03:52:51.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.878+0000 7f7b41794740 1 -- start start 2023-11-09T03:52:51.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.879+0000 7f7b41794740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3388261f0 0x55f3388082d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:51.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.879+0000 7f7b41794740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f338823d10 0x55f338808810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:51.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.879+0000 7f7b41794740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f338805900 0x55f3387dc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:51.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.879+0000 7f7b41794740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33874bd70 con 0x55f338823d10 2023-11-09T03:52:51.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.879+0000 7f7b41794740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f338749460 con 0x55f3388261f0 2023-11-09T03:52:51.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.879+0000 7f7b41794740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f338744d50 con 0x55f338805900 2023-11-09T03:52:51.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.879+0000 7f7b37fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f338823d10 0x55f338808810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:51.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.879+0000 7f7b37fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f338823d10 0x55f338808810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38374/0 (socket says 172.21.15.76:38374) 2023-11-09T03:52:51.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.879+0000 7f7b37fff640 1 -- 172.21.15.76:0/1975454724 learned_addr learned my addr 172.21.15.76:0/1975454724 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:51.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.879+0000 7f7b3c8c1640 1 --2- 172.21.15.76:0/1975454724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3388261f0 0x55f3388082d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:51.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.879+0000 7f7b3d0c2640 1 --2- 172.21.15.76:0/1975454724 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f338805900 0x55f3387dc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:51.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.879+0000 7f7b37fff640 1 -- 172.21.15.76:0/1975454724 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f338805900 msgr2=0x55f3387dc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:51.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.879+0000 7f7b37fff640 1 --2- 172.21.15.76:0/1975454724 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f338805900 0x55f3387dc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.879+0000 7f7b37fff640 1 -- 172.21.15.76:0/1975454724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3388261f0 msgr2=0x55f3388082d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:51.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.879+0000 7f7b37fff640 1 --2- 172.21.15.76:0/1975454724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3388261f0 0x55f3388082d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.879+0000 7f7b37fff640 1 -- 172.21.15.76:0/1975454724 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f338707e30 con 0x55f338823d10 2023-11-09T03:52:51.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.879+0000 7f7b3c8c1640 1 --2- 172.21.15.76:0/1975454724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3388261f0 0x55f3388082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:52:51.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.879+0000 7f7b3d0c2640 1 --2- 172.21.15.76:0/1975454724 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f338805900 0x55f3387dc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:52:51.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.879+0000 7f7b37fff640 1 --2- 172.21.15.76:0/1975454724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f338823d10 0x55f338808810 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f7b240606f0 tx=0x7f7b24058590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33158a564a04f834 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:51.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.880+0000 7f7b227fc640 1 -- 172.21.15.76:0/1975454724 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b24060c20 con 0x55f338823d10 2023-11-09T03:52:51.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.880+0000 7f7b227fc640 1 -- 172.21.15.76:0/1975454724 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b24070180 con 0x55f338823d10 2023-11-09T03:52:51.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.880+0000 7f7b227fc640 1 -- 172.21.15.76:0/1975454724 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2408a0b0 con 0x55f338823d10 2023-11-09T03:52:51.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.880+0000 7f7b41794740 1 -- 172.21.15.76:0/1975454724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f338823d10 msgr2=0x55f338808810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:51.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.880+0000 7f7b41794740 1 --2- 172.21.15.76:0/1975454724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f338823d10 0x55f338808810 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f7b240606f0 tx=0x7f7b24058590 comp rx=0 tx=0).stop 2023-11-09T03:52:51.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.880+0000 7f7b41794740 1 -- 172.21.15.76:0/1975454724 shutdown_connections 2023-11-09T03:52:51.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.880+0000 7f7b41794740 1 --2- 172.21.15.76:0/1975454724 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f338805900 0x55f3387dc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.880+0000 7f7b41794740 1 --2- 172.21.15.76:0/1975454724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f338823d10 0x55f338808810 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.880+0000 7f7b41794740 1 --2- 172.21.15.76:0/1975454724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3388261f0 0x55f3388082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.880+0000 7f7b41794740 1 -- 172.21.15.76:0/1975454724 >> 172.21.15.76:0/1975454724 conn(0x55f3387da9e0 msgr2=0x55f3388232d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:51.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.880+0000 7f7b41794740 1 -- 172.21.15.76:0/1975454724 shutdown_connections 2023-11-09T03:52:51.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.880+0000 7f7b41794740 1 -- 172.21.15.76:0/1975454724 wait complete. 2023-11-09T03:52:51.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.881+0000 7f7b41794740 1 -- start start 2023-11-09T03:52:51.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.881+0000 7f7b41794740 1 Processor -- start 2023-11-09T03:52:51.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.882+0000 7f7b41794740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f338823d10 0x55f3387f4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:51.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.882+0000 7f7b41794740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3388261f0 0x55f3387f1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:51.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.882+0000 7f7b41794740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3387f4af0 0x55f33880b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:51.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.882+0000 7f7b41794740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3386feee0 con 0x55f3387f4af0 2023-11-09T03:52:51.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.882+0000 7f7b41794740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3386feb80 con 0x55f338823d10 2023-11-09T03:52:51.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.882+0000 7f7b41794740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f3386c43a0 con 0x55f3388261f0 2023-11-09T03:52:51.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.882+0000 7f7b41794740 10 client.0 _ll_register_callbacks cb 0x55f3386c88b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:52:51.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.882+0000 7f7b3d0c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3387f4af0 0x55f33880b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:51.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.882+0000 7f7b3d0c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3387f4af0 0x55f33880b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38388/0 (socket says 172.21.15.76:38388) 2023-11-09T03:52:51.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.882+0000 7f7b3d0c2640 1 -- 172.21.15.76:0/1500038373 learned_addr learned my addr 172.21.15.76:0/1500038373 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:51.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.882+0000 7f7b3c8c1640 1 --2- 172.21.15.76:0/1500038373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f338823d10 0x55f3387f4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:51.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.882+0000 7f7b37fff640 1 --2- 172.21.15.76:0/1500038373 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3388261f0 0x55f3387f1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:51.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.882+0000 7f7b3d0c2640 1 -- 172.21.15.76:0/1500038373 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3388261f0 msgr2=0x55f3387f1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:51.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.882+0000 7f7b3d0c2640 1 --2- 172.21.15.76:0/1500038373 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3388261f0 0x55f3387f1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.882+0000 7f7b3d0c2640 1 -- 172.21.15.76:0/1500038373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f338823d10 msgr2=0x55f3387f4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:51.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.882+0000 7f7b3d0c2640 1 --2- 172.21.15.76:0/1500038373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f338823d10 0x55f3387f4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.882+0000 7f7b3d0c2640 1 -- 172.21.15.76:0/1500038373 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b38158c70 con 0x55f3387f4af0 2023-11-09T03:52:51.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.882+0000 7f7b37fff640 1 --2- 172.21.15.76:0/1500038373 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3388261f0 0x55f3387f1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:52:51.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.882+0000 7f7b3d0c2640 1 --2- 172.21.15.76:0/1500038373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3387f4af0 0x55f33880b8c0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f7b30039040 tx=0x7f7b30030070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b886b80249dba6 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:51.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.882+0000 7f7b177fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:52:51.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.882+0000 7f7b177fe640 1 -- 172.21.15.76:0/1500038373 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b30036d70 con 0x55f3387f4af0 2023-11-09T03:52:51.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.882+0000 7f7b177fe640 1 -- 172.21.15.76:0/1500038373 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 7952+0+0 (secure 0 0 0) 0x7f7b30042450 con 0x55f3387f4af0 2023-11-09T03:52:51.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.882+0000 7f7b41794740 1 -- 172.21.15.76:0/1500038373 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f33873dd30 con 0x55f3387f4af0 2023-11-09T03:52:51.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.883+0000 7f7b177fe640 1 -- 172.21.15.76:0/1500038373 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b30003990 con 0x55f3387f4af0 2023-11-09T03:52:51.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.883+0000 7f7b41794740 10 client.8697 fetch_fsmap learned FSMap version 105 2023-11-09T03:52:51.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.883+0000 7f7b41794740 1 -- 172.21.15.76:0/1500038373 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x55f3387433d0 con 0x55f3387f4af0 2023-11-09T03:52:51.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.883+0000 7f7b177fe640 1 -- 172.21.15.76:0/1500038373 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f33873dd30 con 0x55f3387f4af0 2023-11-09T03:52:51.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.883+0000 7f7b177fe640 1 -- 172.21.15.76:0/1500038373 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 105) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f7b30003af0 con 0x55f3387f4af0 2023-11-09T03:52:51.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.883+0000 7f7b41794740 10 client.8697 fetch_fsmap finished waiting for FSMap version 105 2023-11-09T03:52:51.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.883+0000 7f7b41794740 10 client.8697 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2023-11-09T03:52:51.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.883+0000 7f7b41794740 20 client.8697 populate_metadata read hostname 'smithi076' 2023-11-09T03:52:51.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.883+0000 7f7b41794740 1 --2- 172.21.15.76:0/1500038373 >> [v2:172.21.15.76:6834/546338722,v1:172.21.15.76:6835/546338722] conn(0x55f3387d7bd0 0x55f33880a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:51.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.883+0000 7f7b41794740 4 client.8697 mds_command: new command op to 8310 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:52:51.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.883+0000 7f7b41794740 1 -- 172.21.15.76:0/1500038373 --> [v2:172.21.15.76:6834/546338722,v1:172.21.15.76:6835/546338722] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f33873da50 con 0x55f3387d7bd0 2023-11-09T03:52:51.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.883+0000 7f7b3c8c1640 1 --2- 172.21.15.76:0/1500038373 >> [v2:172.21.15.76:6834/546338722,v1:172.21.15.76:6835/546338722] conn(0x55f3387d7bd0 0x55f33880a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:51.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.883+0000 7f7b177fe640 10 client.8697 ms_handle_connect on v2:172.21.15.76:6834/546338722 2023-11-09T03:52:51.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.883+0000 7f7b3c8c1640 1 --2- 172.21.15.76:0/1500038373 >> [v2:172.21.15.76:6834/546338722,v1:172.21.15.76:6835/546338722] conn(0x55f3387d7bd0 0x55f33880a360 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c5da4ad5e5723df server_cookie=c72be2dc87c9c607 in_seq=0 out_seq=0 2023-11-09T03:52:51.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.884+0000 7f7b177fe640 1 -- 172.21.15.76:0/1500038373 <== mds.0 v2:172.21.15.76:6834/546338722 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f33873da50 con 0x55f3387d7bd0 2023-11-09T03:52:51.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.884+0000 7f7b177fe640 10 client.8697 handle_command_reply: tid=0 2023-11-09T03:52:51.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.884+0000 7f7b41794740 1 client.8697 shutdown 2023-11-09T03:52:51.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.884+0000 7f7b41794740 1 -- 172.21.15.76:0/1500038373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3387f4af0 msgr2=0x55f33880b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:51.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.884+0000 7f7b41794740 1 --2- 172.21.15.76:0/1500038373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3387f4af0 0x55f33880b8c0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f7b30039040 tx=0x7f7b30030070 comp rx=0 tx=0).stop 2023-11-09T03:52:51.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.884+0000 7f7b41794740 1 -- 172.21.15.76:0/1500038373 shutdown_connections 2023-11-09T03:52:51.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.884+0000 7f7b41794740 1 --2- 172.21.15.76:0/1500038373 >> [v2:172.21.15.76:6834/546338722,v1:172.21.15.76:6835/546338722] conn(0x55f3387d7bd0 0x55f33880a360 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.884+0000 7f7b41794740 1 --2- 172.21.15.76:0/1500038373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3387f4af0 0x55f33880b8c0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.884+0000 7f7b41794740 1 --2- 172.21.15.76:0/1500038373 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3388261f0 0x55f3387f1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.884+0000 7f7b177fe640 0 client.8697 ms_handle_reset on v2:172.21.15.76:6834/546338722 2023-11-09T03:52:51.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.884+0000 7f7b41794740 1 --2- 172.21.15.76:0/1500038373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f338823d10 0x55f3387f4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.884+0000 7f7b41794740 1 -- 172.21.15.76:0/1500038373 >> 172.21.15.76:0/1500038373 conn(0x55f338805900 msgr2=0x55f3387d9100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:51.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.884+0000 7f7b41794740 1 -- 172.21.15.76:0/1500038373 shutdown_connections 2023-11-09T03:52:51.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.884+0000 7f7b41794740 1 -- 172.21.15.76:0/1500038373 wait complete. 2023-11-09T03:52:51.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.884+0000 7f7b41794740 20 client.8697 trim_cache size 0 max 16384 2023-11-09T03:52:51.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.897+0000 7f7b41794740 1 Processor -- start 2023-11-09T03:52:51.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.897+0000 7f7b41794740 1 -- start start 2023-11-09T03:52:51.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.898+0000 7f7b41794740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3387f4af0 0x55f3388083b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:51.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.898+0000 7f7b41794740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f338823d10 0x55f338838b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:51.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.898+0000 7f7b41794740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3388261f0 0x55f338839050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:51.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.898+0000 7f7b41794740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f338801ec0 con 0x55f338823d10 2023-11-09T03:52:51.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.898+0000 7f7b41794740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f338804330 con 0x55f3387f4af0 2023-11-09T03:52:51.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.898+0000 7f7b41794740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f33870fdc0 con 0x55f3388261f0 2023-11-09T03:52:51.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.898+0000 7f7b37fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f338823d10 0x55f338838b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:51.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.898+0000 7f7b37fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f338823d10 0x55f338838b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38394/0 (socket says 172.21.15.76:38394) 2023-11-09T03:52:51.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.898+0000 7f7b37fff640 1 -- 172.21.15.76:0/3343855208 learned_addr learned my addr 172.21.15.76:0/3343855208 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:51.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.898+0000 7f7b37fff640 1 -- 172.21.15.76:0/3343855208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3388261f0 msgr2=0x55f338839050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:51.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.898+0000 7f7b37fff640 1 --2- 172.21.15.76:0/3343855208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3388261f0 0x55f338839050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.898+0000 7f7b37fff640 1 -- 172.21.15.76:0/3343855208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3387f4af0 msgr2=0x55f3388083b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:51.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.898+0000 7f7b37fff640 1 --2- 172.21.15.76:0/3343855208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3387f4af0 0x55f3388083b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.898+0000 7f7b37fff640 1 -- 172.21.15.76:0/3343855208 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3386e0ea0 con 0x55f338823d10 2023-11-09T03:52:51.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.898+0000 7f7b37fff640 1 --2- 172.21.15.76:0/3343855208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f338823d10 0x55f338838b10 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f7b24006fd0 tx=0x7f7b2408a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbdf09eb308fd299 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:51.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.898+0000 7f7b22ffd640 1 -- 172.21.15.76:0/3343855208 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b24060c20 con 0x55f338823d10 2023-11-09T03:52:51.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.898+0000 7f7b22ffd640 1 -- 172.21.15.76:0/3343855208 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b24056cc0 con 0x55f338823d10 2023-11-09T03:52:51.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.898+0000 7f7b22ffd640 1 -- 172.21.15.76:0/3343855208 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b24055030 con 0x55f338823d10 2023-11-09T03:52:51.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.899+0000 7f7b41794740 1 -- 172.21.15.76:0/3343855208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f338823d10 msgr2=0x55f338838b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:51.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.899+0000 7f7b41794740 1 --2- 172.21.15.76:0/3343855208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f338823d10 0x55f338838b10 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f7b24006fd0 tx=0x7f7b2408a040 comp rx=0 tx=0).stop 2023-11-09T03:52:51.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.899+0000 7f7b41794740 1 -- 172.21.15.76:0/3343855208 shutdown_connections 2023-11-09T03:52:51.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.899+0000 7f7b41794740 1 --2- 172.21.15.76:0/3343855208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f3388261f0 0x55f338839050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.899+0000 7f7b41794740 1 --2- 172.21.15.76:0/3343855208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f338823d10 0x55f338838b10 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.899+0000 7f7b41794740 1 --2- 172.21.15.76:0/3343855208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3387f4af0 0x55f3388083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.899+0000 7f7b41794740 1 -- 172.21.15.76:0/3343855208 >> 172.21.15.76:0/3343855208 conn(0x55f338805900 msgr2=0x55f3387d3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:51.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.899+0000 7f7b41794740 1 -- 172.21.15.76:0/3343855208 shutdown_connections 2023-11-09T03:52:51.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.899+0000 7f7b41794740 1 -- 172.21.15.76:0/3343855208 wait complete. 2023-11-09T03:52:51.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.899+0000 7f7b41794740 1 -- start start 2023-11-09T03:52:51.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.900+0000 7f7b41794740 1 Processor -- start 2023-11-09T03:52:51.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.900+0000 7f7b41794740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3387f4af0 0x55f338874f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:51.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.900+0000 7f7b41794740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f338823d10 0x55f338840940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:51.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.900+0000 7f7b41794740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3388261f0 0x55f3388230a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:51.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.900+0000 7f7b41794740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f338841420 con 0x55f3388261f0 2023-11-09T03:52:51.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.900+0000 7f7b41794740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3388754b0 con 0x55f3387f4af0 2023-11-09T03:52:51.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.900+0000 7f7b41794740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f3387ffa70 con 0x55f338823d10 2023-11-09T03:52:51.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.901+0000 7f7b41794740 10 client.0 _ll_register_callbacks cb 0x7f7b24054c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:52:51.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.901+0000 7f7b3d0c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3388261f0 0x55f3388230a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:51.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.901+0000 7f7b3d0c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3388261f0 0x55f3388230a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38396/0 (socket says 172.21.15.76:38396) 2023-11-09T03:52:51.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.901+0000 7f7b3d0c2640 1 -- 172.21.15.76:0/1175884487 learned_addr learned my addr 172.21.15.76:0/1175884487 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:51.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.901+0000 7f7b3c8c1640 1 --2- 172.21.15.76:0/1175884487 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3387f4af0 0x55f338874f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:51.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.901+0000 7f7b37fff640 1 --2- 172.21.15.76:0/1175884487 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f338823d10 0x55f338840940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:51.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.901+0000 7f7b3d0c2640 1 -- 172.21.15.76:0/1175884487 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f338823d10 msgr2=0x55f338840940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:51.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.901+0000 7f7b3d0c2640 1 --2- 172.21.15.76:0/1175884487 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f338823d10 0x55f338840940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.901+0000 7f7b3d0c2640 1 -- 172.21.15.76:0/1175884487 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3387f4af0 msgr2=0x55f338874f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:51.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.901+0000 7f7b3d0c2640 1 --2- 172.21.15.76:0/1175884487 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3387f4af0 0x55f338874f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.901+0000 7f7b3d0c2640 1 -- 172.21.15.76:0/1175884487 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f338707e30 con 0x55f3388261f0 2023-11-09T03:52:51.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.901+0000 7f7b3d0c2640 1 --2- 172.21.15.76:0/1175884487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3388261f0 0x55f3388230a0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f7b30002230 tx=0x7f7b30004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f67cfa63350eaa server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:51.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.901+0000 7f7b177fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:52:51.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.901+0000 7f7b177fe640 1 -- 172.21.15.76:0/1175884487 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b300367c0 con 0x55f3388261f0 2023-11-09T03:52:51.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.901+0000 7f7b41794740 1 -- 172.21.15.76:0/1175884487 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b24070180 con 0x55f3388261f0 2023-11-09T03:52:51.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.901+0000 7f7b177fe640 1 -- 172.21.15.76:0/1175884487 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 7952+0+0 (secure 0 0 0) 0x7f7b300043d0 con 0x55f3388261f0 2023-11-09T03:52:51.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.901+0000 7f7b177fe640 1 -- 172.21.15.76:0/1175884487 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b3000e120 con 0x55f3388261f0 2023-11-09T03:52:51.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.901+0000 7f7b41794740 10 client.8703 fetch_fsmap learned FSMap version 105 2023-11-09T03:52:51.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.901+0000 7f7b41794740 1 -- 172.21.15.76:0/1175884487 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x55f338706690 con 0x55f3388261f0 2023-11-09T03:52:51.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.901+0000 7f7b177fe640 1 -- 172.21.15.76:0/1175884487 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b3005f080 con 0x55f3388261f0 2023-11-09T03:52:51.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.902+0000 7f7b177fe640 1 -- 172.21.15.76:0/1175884487 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 105) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f7b30059030 con 0x55f3388261f0 2023-11-09T03:52:51.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.902+0000 7f7b41794740 10 client.8703 fetch_fsmap finished waiting for FSMap version 105 2023-11-09T03:52:51.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.902+0000 7f7b41794740 10 client.8703 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2023-11-09T03:52:51.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.902+0000 7f7b41794740 20 client.8703 populate_metadata read hostname 'smithi076' 2023-11-09T03:52:51.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.902+0000 7f7b41794740 1 --2- 172.21.15.76:0/1175884487 >> [v2:172.21.15.76:6834/546338722,v1:172.21.15.76:6835/546338722] conn(0x55f338840f90 0x55f3388235e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:51.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.902+0000 7f7b41794740 4 client.8703 mds_command: new command op to 8310 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-11-09T03:52:51.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.902+0000 7f7b41794740 1 -- 172.21.15.76:0/1175884487 --> [v2:172.21.15.76:6834/546338722,v1:172.21.15.76:6835/546338722] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f3384457e0 con 0x55f338840f90 2023-11-09T03:52:51.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.902+0000 7f7b3c8c1640 1 --2- 172.21.15.76:0/1175884487 >> [v2:172.21.15.76:6834/546338722,v1:172.21.15.76:6835/546338722] conn(0x55f338840f90 0x55f3388235e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:51.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.902+0000 7f7b177fe640 10 client.8703 ms_handle_connect on v2:172.21.15.76:6834/546338722 2023-11-09T03:52:51.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.902+0000 7f7b3c8c1640 1 --2- 172.21.15.76:0/1175884487 >> [v2:172.21.15.76:6834/546338722,v1:172.21.15.76:6835/546338722] conn(0x55f338840f90 0x55f3388235e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66b3f8e8c8669e4c server_cookie=26540b0195ed9d8c in_seq=0 out_seq=0 2023-11-09T03:52:51.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.913+0000 7f7b177fe640 1 -- 172.21.15.76:0/1175884487 <== mds.0 v2:172.21.15.76:6834/546338722 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f3384457e0 con 0x55f338840f90 2023-11-09T03:52:51.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.913+0000 7f7b177fe640 10 client.8703 handle_command_reply: tid=0 2023-11-09T03:52:51.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.913+0000 7f7b41794740 1 client.8703 shutdown 2023-11-09T03:52:51.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.913+0000 7f7b41794740 1 -- 172.21.15.76:0/1175884487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3388261f0 msgr2=0x55f3388230a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:51.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.913+0000 7f7b41794740 1 --2- 172.21.15.76:0/1175884487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3388261f0 0x55f3388230a0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f7b30002230 tx=0x7f7b30004ab0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.913+0000 7f7b41794740 1 -- 172.21.15.76:0/1175884487 shutdown_connections 2023-11-09T03:52:51.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.913+0000 7f7b41794740 1 --2- 172.21.15.76:0/1175884487 >> [v2:172.21.15.76:6834/546338722,v1:172.21.15.76:6835/546338722] conn(0x55f338840f90 0x55f3388235e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.913+0000 7f7b41794740 1 --2- 172.21.15.76:0/1175884487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f3388261f0 0x55f3388230a0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.913+0000 7f7b41794740 1 --2- 172.21.15.76:0/1175884487 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f338823d10 0x55f338840940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.913+0000 7f7b41794740 1 --2- 172.21.15.76:0/1175884487 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f3387f4af0 0x55f338874f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.913+0000 7f7b41794740 1 -- 172.21.15.76:0/1175884487 >> 172.21.15.76:0/1175884487 conn(0x55f338805900 msgr2=0x55f3387f4270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:51.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.913+0000 7f7b177fe640 0 client.8703 ms_handle_reset on v2:172.21.15.76:6834/546338722 2023-11-09T03:52:51.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.913+0000 7f7b41794740 1 -- 172.21.15.76:0/1175884487 shutdown_connections 2023-11-09T03:52:51.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.913+0000 7f7b41794740 1 -- 172.21.15.76:0/1175884487 wait complete. 2023-11-09T03:52:51.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.914+0000 7f7b41794740 20 client.8703 trim_cache size 0 max 16384 2023-11-09T03:52:51.926 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:52:51.926 INFO:teuthology.orchestra.run.smithi076.stdout: "message": "", 2023-11-09T03:52:51.927 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2023-11-09T03:52:51.927 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:52:51.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.914+0000 7f7b3eb25640 1 -- 172.21.15.76:0/1049992597 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7b1c06ae70 msgr2=0x7f7b1c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:51.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.914+0000 7f7b3eb25640 1 --2- 172.21.15.76:0/1049992597 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7b1c06ae70 0x7f7b1c06d330 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c005880 tx=0x7f7b2c009040 comp rx=0 tx=0).stop 2023-11-09T03:52:51.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.914+0000 7f7b3eb25640 1 -- 172.21.15.76:0/1049992597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b380b0c40 msgr2=0x7f7b3815a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:51.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.914+0000 7f7b3eb25640 1 --2- 172.21.15.76:0/1049992597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b380b0c40 0x7f7b3815a220 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400d800 tx=0x7f7b2400dcc0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.914+0000 7f7b3eb25640 1 -- 172.21.15.76:0/1049992597 shutdown_connections 2023-11-09T03:52:51.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.914+0000 7f7b3eb25640 1 --2- 172.21.15.76:0/1049992597 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7b1c06ae70 0x7f7b1c06d330 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.914+0000 7f7b3eb25640 1 --2- 172.21.15.76:0/1049992597 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7b38159e40 0x7f7b38157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.914+0000 7f7b3eb25640 1 --2- 172.21.15.76:0/1049992597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b380b0c40 0x7f7b3815a220 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.914+0000 7f7b3eb25640 1 --2- 172.21.15.76:0/1049992597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b3806bb60 0x7f7b38154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:51.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.914+0000 7f7b3eb25640 1 -- 172.21.15.76:0/1049992597 >> 172.21.15.76:0/1049992597 conn(0x7f7b380cb530 msgr2=0x7f7b380b03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:51.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.914+0000 7f7b3eb25640 1 -- 172.21.15.76:0/1049992597 shutdown_connections 2023-11-09T03:52:51.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:51.914+0000 7f7b3eb25640 1 -- 172.21.15.76:0/1049992597 wait complete. 2023-11-09T03:52:51.930 INFO:tasks.cephfs.test_snapshots:testing snapserver mds_kill_mdstable_at=6, also kill snapclient 2023-11-09T03:52:51.931 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:52:52.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.030+0000 7fac8fcd7640 1 Processor -- start 2023-11-09T03:52:52.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.030+0000 7fac8fcd7640 1 -- start start 2023-11-09T03:52:52.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.031+0000 7fac8fcd7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac880b0c40 0x7fac8806fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:52.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.031+0000 7fac8fcd7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac88071500 con 0x7fac88159e40 2023-11-09T03:52:52.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.031+0000 7fac8fcd7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac8815dae0 con 0x7fac8806bb60 2023-11-09T03:52:52.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.031+0000 7fac8fcd7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac88070c60 con 0x7fac880b0c40 2023-11-09T03:52:52.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.031+0000 7fac8da73640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fac88159e40 0x7fac880b0530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45224/0 (socket says 172.21.15.76:45224) 2023-11-09T03:52:52.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.031+0000 7fac8da73640 1 -- 172.21.15.76:0/1008554419 learned_addr learned my addr 172.21.15.76:0/1008554419 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:52:52.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.031+0000 7fac8ca71640 1 -- 172.21.15.76:0/1008554419 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2570864573 0 0) 0x7fac88071500 con 0x7fac88159e40 2023-11-09T03:52:52.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.031+0000 7fac8ca71640 1 -- 172.21.15.76:0/1008554419 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac74003520 con 0x7fac88159e40 2023-11-09T03:52:52.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.031+0000 7fac8d272640 1 --2- 172.21.15.76:0/1008554419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac880b0c40 0x7fac8806fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:52.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.032+0000 7fac8ca71640 1 -- 172.21.15.76:0/1008554419 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1572945774 0 0) 0x7fac74003520 con 0x7fac88159e40 2023-11-09T03:52:52.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.032+0000 7fac8ca71640 1 -- 172.21.15.76:0/1008554419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac880b0c40 msgr2=0x7fac8806fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:52.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.032+0000 7fac8ca71640 1 --2- 172.21.15.76:0/1008554419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac880b0c40 0x7fac8806fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:52.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.032+0000 7fac8ca71640 1 -- 172.21.15.76:0/1008554419 >> v1:172.21.15.113:6789/0 conn(0x7fac8806bb60 legacy=0x7fac880cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:52.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.032+0000 7fac8ca71640 1 -- 172.21.15.76:0/1008554419 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac88070f40 con 0x7fac88159e40 2023-11-09T03:52:52.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.032+0000 7fac8ca71640 1 -- 172.21.15.76:0/1008554419 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fac84001670 con 0x7fac88159e40 2023-11-09T03:52:52.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.032+0000 7fac8d272640 1 --2- 172.21.15.76:0/1008554419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac880b0c40 0x7fac8806fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:52:52.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.032+0000 7fac8ca71640 1 -- 172.21.15.76:0/1008554419 >> v1:172.21.15.76:6789/0 conn(0x7fac88159e40 legacy=0x7fac880b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:52.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.032+0000 7fac8ca71640 1 --2- 172.21.15.76:0/1008554419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac880b0c40 0x7fac740039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:52.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.032+0000 7fac8ca71640 1 --2- 172.21.15.76:0/1008554419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac74003f90 0x7fac740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:52.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.032+0000 7fac8ca71640 1 --2- 172.21.15.76:0/1008554419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac74004d00 0x7fac74005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:52.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.032+0000 7fac8ca71640 1 -- 172.21.15.76:0/1008554419 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac88070c60 con 0x7fac74003f90 2023-11-09T03:52:52.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.032+0000 7fac8ca71640 1 -- 172.21.15.76:0/1008554419 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7400bb70 con 0x7fac74004d00 2023-11-09T03:52:52.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.032+0000 7fac8ca71640 1 -- 172.21.15.76:0/1008554419 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac7400bd30 con 0x7fac880b0c40 2023-11-09T03:52:52.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.032+0000 7fac8d272640 1 --2- 172.21.15.76:0/1008554419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac74003f90 0x7fac740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:52.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.032+0000 7fac8da73640 1 --2- 172.21.15.76:0/1008554419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac880b0c40 0x7fac740039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:52.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.032+0000 7fac8e274640 1 --2- 172.21.15.76:0/1008554419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac74004d00 0x7fac74005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:52.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.032+0000 7fac8d272640 1 -- 172.21.15.76:0/1008554419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac880b0c40 msgr2=0x7fac740039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:52.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.032+0000 7fac8d272640 1 --2- 172.21.15.76:0/1008554419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac880b0c40 0x7fac740039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:52.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.032+0000 7fac8d272640 1 -- 172.21.15.76:0/1008554419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac74004d00 msgr2=0x7fac74005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:52.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.032+0000 7fac8d272640 1 --2- 172.21.15.76:0/1008554419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac74004d00 0x7fac74005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:52.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.032+0000 7fac8d272640 1 -- 172.21.15.76:0/1008554419 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac74003520 con 0x7fac74003f90 2023-11-09T03:52:52.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.032+0000 7fac8e274640 1 --2- 172.21.15.76:0/1008554419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac74004d00 0x7fac74005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:52:52.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.033+0000 7fac8d272640 1 --2- 172.21.15.76:0/1008554419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac74003f90 0x7fac740047c0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fac780098a0 tx=0x7fac740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5917235b97c96cee server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:52.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.033+0000 7fac8ca71640 1 -- 172.21.15.76:0/1008554419 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac78004230 con 0x7fac74003f90 2023-11-09T03:52:52.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.033+0000 7fac8ca71640 1 -- 172.21.15.76:0/1008554419 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac78004390 con 0x7fac74003f90 2023-11-09T03:52:52.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.033+0000 7fac8ca71640 1 -- 172.21.15.76:0/1008554419 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac78003830 con 0x7fac74003f90 2023-11-09T03:52:52.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.033+0000 7fac8fcd7640 1 -- 172.21.15.76:0/1008554419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac74003f90 msgr2=0x7fac740047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:52.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.033+0000 7fac8fcd7640 1 --2- 172.21.15.76:0/1008554419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac74003f90 0x7fac740047c0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fac780098a0 tx=0x7fac740057f0 comp rx=0 tx=0).stop 2023-11-09T03:52:52.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.033+0000 7fac8e274640 1 -- 172.21.15.76:0/1008554419 reap_dead start 2023-11-09T03:52:52.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.033+0000 7fac8fcd7640 1 -- 172.21.15.76:0/1008554419 shutdown_connections 2023-11-09T03:52:52.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.033+0000 7fac8fcd7640 1 -- 172.21.15.76:0/1008554419 >> 172.21.15.76:0/1008554419 conn(0x7fac880cb530 msgr2=0x7fac880cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:52.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.034+0000 7fac8fcd7640 1 -- 172.21.15.76:0/1008554419 shutdown_connections 2023-11-09T03:52:52.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.034+0000 7fac8fcd7640 1 -- 172.21.15.76:0/1008554419 wait complete. 2023-11-09T03:52:52.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.035+0000 7fac8fcd7640 1 Processor -- start 2023-11-09T03:52:52.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.035+0000 7fac8fcd7640 1 -- start start 2023-11-09T03:52:52.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.035+0000 7fac8fcd7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac74003f90 0x7fac880c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:52.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.035+0000 7fac8fcd7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac88155100 0x7fac881e3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:52.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.035+0000 7fac8fcd7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac881e3cc0 0x7fac880afe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:52.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.035+0000 7fac8fcd7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac881e4140 con 0x7fac74003f90 2023-11-09T03:52:52.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.035+0000 7fac8fcd7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac881f3c50 con 0x7fac88155100 2023-11-09T03:52:52.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.035+0000 7fac8fcd7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac881f4b20 con 0x7fac881e3cc0 2023-11-09T03:52:52.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.036+0000 7fac8da73640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac74003f90 0x7fac880c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:52.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.036+0000 7fac8da73640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac74003f90 0x7fac880c72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38408/0 (socket says 172.21.15.76:38408) 2023-11-09T03:52:52.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.036+0000 7fac8da73640 1 -- 172.21.15.76:0/2022270182 learned_addr learned my addr 172.21.15.76:0/2022270182 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:52.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.036+0000 7fac8d272640 1 --2- 172.21.15.76:0/2022270182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac88155100 0x7fac881e3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:52.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.036+0000 7fac8e274640 1 --2- 172.21.15.76:0/2022270182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac881e3cc0 0x7fac880afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:52.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.036+0000 7fac8da73640 1 -- 172.21.15.76:0/2022270182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac881e3cc0 msgr2=0x7fac880afe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:52.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.036+0000 7fac8da73640 1 --2- 172.21.15.76:0/2022270182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac881e3cc0 0x7fac880afe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:52.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.036+0000 7fac8da73640 1 -- 172.21.15.76:0/2022270182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac88155100 msgr2=0x7fac881e3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:52.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.036+0000 7fac8da73640 1 --2- 172.21.15.76:0/2022270182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac88155100 0x7fac881e3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:52.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.036+0000 7fac8da73640 1 -- 172.21.15.76:0/2022270182 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac881f3ea0 con 0x7fac74003f90 2023-11-09T03:52:52.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.036+0000 7fac8d272640 1 --2- 172.21.15.76:0/2022270182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac88155100 0x7fac881e3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:52:52.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.036+0000 7fac8da73640 1 --2- 172.21.15.76:0/2022270182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac74003f90 0x7fac880c72b0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fac840072c0 tx=0x7fac84007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:52.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.036+0000 7fac7e7fc640 1 -- 172.21.15.76:0/2022270182 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8400f960 con 0x7fac74003f90 2023-11-09T03:52:52.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.036+0000 7fac7e7fc640 1 -- 172.21.15.76:0/2022270182 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8400fae0 con 0x7fac74003f90 2023-11-09T03:52:52.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.036+0000 7fac8fcd7640 1 -- 172.21.15.76:0/2022270182 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac881f4180 con 0x7fac74003f90 2023-11-09T03:52:52.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.036+0000 7fac7e7fc640 1 -- 172.21.15.76:0/2022270182 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8400fe90 con 0x7fac74003f90 2023-11-09T03:52:52.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.036+0000 7fac8fcd7640 1 -- 172.21.15.76:0/2022270182 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac881f4740 con 0x7fac74003f90 2023-11-09T03:52:52.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.037+0000 7fac8fcd7640 1 -- 172.21.15.76:0/2022270182 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac880684b0 con 0x7fac74003f90 2023-11-09T03:52:52.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.037+0000 7fac7e7fc640 1 -- 172.21.15.76:0/2022270182 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fac8401e3b0 con 0x7fac74003f90 2023-11-09T03:52:52.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.038+0000 7fac7e7fc640 1 --2- 172.21.15.76:0/2022270182 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fac6c06ae70 0x7fac6c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:52.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.038+0000 7fac7e7fc640 1 -- 172.21.15.76:0/2022270182 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7952+0+0 (secure 0 0 0) 0x7fac84089ac0 con 0x7fac74003f90 2023-11-09T03:52:52.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.038+0000 7fac8d272640 1 --2- 172.21.15.76:0/2022270182 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fac6c06ae70 0x7fac6c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:52.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.039+0000 7fac8d272640 1 --2- 172.21.15.76:0/2022270182 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fac6c06ae70 0x7fac6c06d330 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fac78035000 tx=0x7fac7803a040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:52.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.042+0000 7fac7e7fc640 1 -- 172.21.15.76:0/2022270182 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fac84054340 con 0x7fac74003f90 2023-11-09T03:52:52.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.245+0000 7fac8fcd7640 1 -- 172.21.15.76:0/2022270182 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac881f0660 con 0x7fac74003f90 2023-11-09T03:52:52.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.245+0000 7fac7e7fc640 1 -- 172.21.15.76:0/2022270182 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4924 (secure 0 0 0) 0x7fac8401a020 con 0x7fac74003f90 2023-11-09T03:52:52.245 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:52:52.245 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":105,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8568,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:52:44.483938+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8310,"mds_1":7144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7144":{"gid":7144,"name":"b","rank":1,"incarnation":101,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/7360844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":7360844},{"type":"v1","addr":"172.21.15.113:6837","nonce":7360844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8310":{"gid":8310,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":19,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:52:52.246 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 105 2023-11-09T03:52:52.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.247+0000 7fac8fcd7640 1 -- 172.21.15.76:0/2022270182 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fac6c06ae70 msgr2=0x7fac6c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:52.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.247+0000 7fac8fcd7640 1 --2- 172.21.15.76:0/2022270182 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fac6c06ae70 0x7fac6c06d330 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fac78035000 tx=0x7fac7803a040 comp rx=0 tx=0).stop 2023-11-09T03:52:52.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.247+0000 7fac8fcd7640 1 -- 172.21.15.76:0/2022270182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac74003f90 msgr2=0x7fac880c72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:52.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.247+0000 7fac8fcd7640 1 --2- 172.21.15.76:0/2022270182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac74003f90 0x7fac880c72b0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fac840072c0 tx=0x7fac84007780 comp rx=0 tx=0).stop 2023-11-09T03:52:52.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.247+0000 7fac8fcd7640 1 -- 172.21.15.76:0/2022270182 shutdown_connections 2023-11-09T03:52:52.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.247+0000 7fac8fcd7640 1 --2- 172.21.15.76:0/2022270182 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fac6c06ae70 0x7fac6c06d330 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:52.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.247+0000 7fac8fcd7640 1 --2- 172.21.15.76:0/2022270182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac881e3cc0 0x7fac880afe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:52.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.247+0000 7fac8fcd7640 1 --2- 172.21.15.76:0/2022270182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac88155100 0x7fac881e3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:52.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.247+0000 7fac8fcd7640 1 --2- 172.21.15.76:0/2022270182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac74003f90 0x7fac880c72b0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:52.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.247+0000 7fac8fcd7640 1 -- 172.21.15.76:0/2022270182 >> 172.21.15.76:0/2022270182 conn(0x7fac880cb530 msgr2=0x7fac8815c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:52.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.247+0000 7fac8fcd7640 1 -- 172.21.15.76:0/2022270182 shutdown_connections 2023-11-09T03:52:52.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.247+0000 7fac8fcd7640 1 -- 172.21.15.76:0/2022270182 wait complete. 2023-11-09T03:52:52.258 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump snaps --server 2023-11-09T03:52:52.370 INFO:teuthology.orchestra.run.smithi076.stdout:{"last_snap":11,"last_created":11,"last_destroyed":9,"pending_noop":[],"snaps":[{"snapid":11,"ino":1099511627777,"stamp":"2023-11-09T03:52:04.689262+0000","name":"s25","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-09T03:52:52.370 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 11, "last_destroyed": 9, "last_snap": 11, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s25", "snapid": 11, "stamp": "2023-11-09T03:52:04.689262+0000" } ] } 2023-11-09T03:52:52.371 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:0 true 2023-11-09T03:52:52.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.473+0000 7f8f9a1cc640 1 Processor -- start 2023-11-09T03:52:52.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.473+0000 7f8f9a1cc640 1 -- start start 2023-11-09T03:52:52.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.474+0000 7f8f9a1cc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f940b0c40 0x7f8f9406fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.474+0000 7f8f9a1cc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8f9406bb60 0x7f8f94070530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.474+0000 7f8f9a1cc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f94071500 con 0x7f8f94159e40 2023-11-09T03:52:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.474+0000 7f8f9a1cc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f94070a70 con 0x7f8f9406bb60 2023-11-09T03:52:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.474+0000 7f8f9a1cc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9415dae0 con 0x7f8f940b0c40 2023-11-09T03:52:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.474+0000 7f8f937fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8f94159e40 0x7f8f940b0530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45236/0 (socket says 172.21.15.76:45236) 2023-11-09T03:52:52.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.474+0000 7f8f937fe640 1 -- 172.21.15.76:0/112370852 learned_addr learned my addr 172.21.15.76:0/112370852 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:52:52.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.474+0000 7f8f93fff640 1 --2- 172.21.15.76:0/112370852 >> v2:172.21.15.113:3300/0 conn(0x7f8f9406bb60 0x7f8f94070530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:52.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.474+0000 7f8f92ffd640 1 --2- 172.21.15.76:0/112370852 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f940b0c40 0x7f8f9406fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:52.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.474+0000 7f8f927fc640 1 -- 172.21.15.76:0/112370852 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1625403444 0 0) 0x7f8f94071500 con 0x7f8f94159e40 2023-11-09T03:52:52.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.474+0000 7f8f927fc640 1 -- 172.21.15.76:0/112370852 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f7c003520 con 0x7f8f94159e40 2023-11-09T03:52:52.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.474+0000 7f8f927fc640 1 -- 172.21.15.76:0/112370852 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 298567084 0 0) 0x7f8f7c003520 con 0x7f8f94159e40 2023-11-09T03:52:52.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.474+0000 7f8f927fc640 1 -- 172.21.15.76:0/112370852 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f940b0c40 msgr2=0x7f8f9406fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:52.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.474+0000 7f8f927fc640 1 --2- 172.21.15.76:0/112370852 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f940b0c40 0x7f8f9406fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:52.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.474+0000 7f8f927fc640 1 -- 172.21.15.76:0/112370852 >> v2:172.21.15.113:3300/0 conn(0x7f8f9406bb60 msgr2=0x7f8f94070530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:52.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.474+0000 7f8f927fc640 1 --2- 172.21.15.76:0/112370852 >> v2:172.21.15.113:3300/0 conn(0x7f8f9406bb60 0x7f8f94070530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.474+0000 7f8f927fc640 1 -- 172.21.15.76:0/112370852 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f94070c60 con 0x7f8f94159e40 2023-11-09T03:52:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.474+0000 7f8f927fc640 1 -- 172.21.15.76:0/112370852 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f8f84001670 con 0x7f8f94159e40 2023-11-09T03:52:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.474+0000 7f8f927fc640 1 -- 172.21.15.76:0/112370852 >> v1:172.21.15.76:6789/0 conn(0x7f8f94159e40 legacy=0x7f8f940b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.474+0000 7f8f927fc640 1 --2- 172.21.15.76:0/112370852 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f940b0c40 0x7f8f7c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.474+0000 7f8f927fc640 1 --2- 172.21.15.76:0/112370852 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f7c003ef0 0x7f8f7c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:52.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.474+0000 7f8f927fc640 1 --2- 172.21.15.76:0/112370852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f7c004d00 0x7f8f7c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:52.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.474+0000 7f8f927fc640 1 -- 172.21.15.76:0/112370852 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9415dae0 con 0x7f8f7c004d00 2023-11-09T03:52:52.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.474+0000 7f8f927fc640 1 -- 172.21.15.76:0/112370852 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7c00b980 con 0x7f8f7c003ef0 2023-11-09T03:52:52.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.474+0000 7f8f927fc640 1 -- 172.21.15.76:0/112370852 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7c00bb10 con 0x7f8f940b0c40 2023-11-09T03:52:52.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.475+0000 7f8f93fff640 1 --2- 172.21.15.76:0/112370852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f7c004d00 0x7f8f7c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:52.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.475+0000 7f8f937fe640 1 --2- 172.21.15.76:0/112370852 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f940b0c40 0x7f8f7c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:52.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.475+0000 7f8f92ffd640 1 --2- 172.21.15.76:0/112370852 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f7c003ef0 0x7f8f7c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:52.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.475+0000 7f8f93fff640 1 -- 172.21.15.76:0/112370852 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f940b0c40 msgr2=0x7f8f7c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:52.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.475+0000 7f8f93fff640 1 --2- 172.21.15.76:0/112370852 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f940b0c40 0x7f8f7c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:52.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.475+0000 7f8f93fff640 1 -- 172.21.15.76:0/112370852 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f7c003ef0 msgr2=0x7f8f7c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:52.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.475+0000 7f8f93fff640 1 --2- 172.21.15.76:0/112370852 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f7c003ef0 0x7f8f7c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:52.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.475+0000 7f8f93fff640 1 -- 172.21.15.76:0/112370852 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7c003520 con 0x7f8f7c004d00 2023-11-09T03:52:52.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.475+0000 7f8f92ffd640 1 --2- 172.21.15.76:0/112370852 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f7c003ef0 0x7f8f7c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:52:52.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.476+0000 7f8f93fff640 1 --2- 172.21.15.76:0/112370852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f7c004d00 0x7f8f7c005180 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c009bd0 tx=0x7f8f7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22403fecc749264b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:52.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.476+0000 7f8f927fc640 1 -- 172.21.15.76:0/112370852 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c02ee30 con 0x7f8f7c004d00 2023-11-09T03:52:52.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.476+0000 7f8f927fc640 1 -- 172.21.15.76:0/112370852 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8c00f040 con 0x7f8f7c004d00 2023-11-09T03:52:52.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.476+0000 7f8f927fc640 1 -- 172.21.15.76:0/112370852 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c036680 con 0x7f8f7c004d00 2023-11-09T03:52:52.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.476+0000 7f8f9a1cc640 1 -- 172.21.15.76:0/112370852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f7c004d00 msgr2=0x7f8f7c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:52.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.476+0000 7f8f9a1cc640 1 --2- 172.21.15.76:0/112370852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f7c004d00 0x7f8f7c005180 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c009bd0 tx=0x7f8f7c0057f0 comp rx=0 tx=0).stop 2023-11-09T03:52:52.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.476+0000 7f8f93fff640 1 -- 172.21.15.76:0/112370852 reap_dead start 2023-11-09T03:52:52.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.476+0000 7f8f9a1cc640 1 -- 172.21.15.76:0/112370852 shutdown_connections 2023-11-09T03:52:52.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.476+0000 7f8f9a1cc640 1 -- 172.21.15.76:0/112370852 >> 172.21.15.76:0/112370852 conn(0x7f8f940cb530 msgr2=0x7f8f940cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:52.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.476+0000 7f8f9a1cc640 1 -- 172.21.15.76:0/112370852 shutdown_connections 2023-11-09T03:52:52.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.476+0000 7f8f9a1cc640 1 -- 172.21.15.76:0/112370852 wait complete. 2023-11-09T03:52:52.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.477+0000 7f8f9a1cc640 1 Processor -- start 2023-11-09T03:52:52.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.478+0000 7f8f9a1cc640 1 -- start start 2023-11-09T03:52:52.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.478+0000 7f8f9a1cc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f7c004d00 0x7f8f940c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:52.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.478+0000 7f8f9a1cc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f940c60c0 0x7f8f940c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:52.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.478+0000 7f8f9a1cc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f940c8020 0x7f8f940c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:52.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.478+0000 7f8f9a1cc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f940c6540 con 0x7f8f940c60c0 2023-11-09T03:52:52.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.478+0000 7f8f9a1cc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9414ca20 con 0x7f8f940c8020 2023-11-09T03:52:52.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.478+0000 7f8f9a1cc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9415a220 con 0x7f8f7c004d00 2023-11-09T03:52:52.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.478+0000 7f8f92ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f940c60c0 0x7f8f940c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:52.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.478+0000 7f8f92ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f940c60c0 0x7f8f940c7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38416/0 (socket says 172.21.15.76:38416) 2023-11-09T03:52:52.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.478+0000 7f8f92ffd640 1 -- 172.21.15.76:0/2737477233 learned_addr learned my addr 172.21.15.76:0/2737477233 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:52.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.479+0000 7f8f92ffd640 1 -- 172.21.15.76:0/2737477233 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f7c004d00 msgr2=0x7f8f940c5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:52.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.479+0000 7f8f92ffd640 1 --2- 172.21.15.76:0/2737477233 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f7c004d00 0x7f8f940c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:52.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.479+0000 7f8f92ffd640 1 -- 172.21.15.76:0/2737477233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f940c8020 msgr2=0x7f8f940c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:52.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.479+0000 7f8f92ffd640 1 --2- 172.21.15.76:0/2737477233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f940c8020 0x7f8f940c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:52.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.479+0000 7f8f92ffd640 1 -- 172.21.15.76:0/2737477233 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f940c89e0 con 0x7f8f940c60c0 2023-11-09T03:52:52.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.479+0000 7f8f92ffd640 1 --2- 172.21.15.76:0/2737477233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f940c60c0 0x7f8f940c7ae0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f8f880115a0 tx=0x7f8f88011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:52.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.479+0000 7f8f7bfff640 1 -- 172.21.15.76:0/2737477233 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f88015870 con 0x7f8f940c60c0 2023-11-09T03:52:52.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.479+0000 7f8f7bfff640 1 -- 172.21.15.76:0/2737477233 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f880159d0 con 0x7f8f940c60c0 2023-11-09T03:52:52.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.479+0000 7f8f9a1cc640 1 -- 172.21.15.76:0/2737477233 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f940cdd40 con 0x7f8f940c60c0 2023-11-09T03:52:52.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.479+0000 7f8f7bfff640 1 -- 172.21.15.76:0/2737477233 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f88015d40 con 0x7f8f940c60c0 2023-11-09T03:52:52.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.479+0000 7f8f9a1cc640 1 -- 172.21.15.76:0/2737477233 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f9415a470 con 0x7f8f940c60c0 2023-11-09T03:52:52.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.480+0000 7f8f9a1cc640 1 -- 172.21.15.76:0/2737477233 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f940684b0 con 0x7f8f940c60c0 2023-11-09T03:52:52.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.480+0000 7f8f7bfff640 1 -- 172.21.15.76:0/2737477233 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8f8801d400 con 0x7f8f940c60c0 2023-11-09T03:52:52.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.481+0000 7f8f7bfff640 1 --2- 172.21.15.76:0/2737477233 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8f8006aea0 0x7f8f8006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:52.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.481+0000 7f8f7bfff640 1 -- 172.21.15.76:0/2737477233 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7952+0+0 (secure 0 0 0) 0x7f8f8808a680 con 0x7f8f940c60c0 2023-11-09T03:52:52.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.481+0000 7f8f937fe640 1 --2- 172.21.15.76:0/2737477233 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8f8006aea0 0x7f8f8006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:52.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.481+0000 7f8f937fe640 1 --2- 172.21.15.76:0/2737477233 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8f8006aea0 0x7f8f8006d360 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8f84000f60 tx=0x7f8f84000ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:52.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.485+0000 7f8f7bfff640 1 -- 172.21.15.76:0/2737477233 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8f88054e70 con 0x7f8f940c60c0 2023-11-09T03:52:52.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:52.718+0000 7f8f9a1cc640 1 -- 172.21.15.76:0/2737477233 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"} v 0) v1 -- 0x7f8f940c9750 con 0x7f8f940c60c0 2023-11-09T03:52:53.281 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:52:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:52:53.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.516+0000 7f8f7bfff640 1 -- 172.21.15.76:0/2737477233 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"}]=0 freezing mds.8310 v106) v1 ==== 112+0+0 (secure 0 0 0) 0x7f8f88058490 con 0x7f8f940c60c0 2023-11-09T03:52:53.516 INFO:teuthology.orchestra.run.smithi076.stderr:freezing mds.8310 2023-11-09T03:52:53.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.519+0000 7f8f9a1cc640 1 -- 172.21.15.76:0/2737477233 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8f8006aea0 msgr2=0x7f8f8006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:53.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.519+0000 7f8f9a1cc640 1 --2- 172.21.15.76:0/2737477233 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8f8006aea0 0x7f8f8006d360 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8f84000f60 tx=0x7f8f84000ed0 comp rx=0 tx=0).stop 2023-11-09T03:52:53.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.519+0000 7f8f9a1cc640 1 -- 172.21.15.76:0/2737477233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f940c60c0 msgr2=0x7f8f940c7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:53.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.519+0000 7f8f9a1cc640 1 --2- 172.21.15.76:0/2737477233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f940c60c0 0x7f8f940c7ae0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f8f880115a0 tx=0x7f8f88011a60 comp rx=0 tx=0).stop 2023-11-09T03:52:53.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.519+0000 7f8f9a1cc640 1 -- 172.21.15.76:0/2737477233 shutdown_connections 2023-11-09T03:52:53.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.519+0000 7f8f9a1cc640 1 --2- 172.21.15.76:0/2737477233 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8f8006aea0 0x7f8f8006d360 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:53.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.519+0000 7f8f9a1cc640 1 --2- 172.21.15.76:0/2737477233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f940c8020 0x7f8f940c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:53.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.519+0000 7f8f9a1cc640 1 --2- 172.21.15.76:0/2737477233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f940c60c0 0x7f8f940c7ae0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:53.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.519+0000 7f8f9a1cc640 1 --2- 172.21.15.76:0/2737477233 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f7c004d00 0x7f8f940c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:53.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.519+0000 7f8f9a1cc640 1 -- 172.21.15.76:0/2737477233 >> 172.21.15.76:0/2737477233 conn(0x7f8f940cb530 msgr2=0x7f8f94070350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:53.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.519+0000 7f8f9a1cc640 1 -- 172.21.15.76:0/2737477233 shutdown_connections 2023-11-09T03:52:53.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.519+0000 7f8f9a1cc640 1 -- 172.21.15.76:0/2737477233 wait complete. 2023-11-09T03:52:53.532 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:1 true 2023-11-09T03:52:53.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.641+0000 7f0222a4b640 1 Processor -- start 2023-11-09T03:52:53.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.641+0000 7f0222a4b640 1 -- start start 2023-11-09T03:52:53.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.641+0000 7f0222a4b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f021c15d110 0x7f021c06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:53.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.641+0000 7f0222a4b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f021c0b3ca0 0x7f021c0ace70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:53.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.642+0000 7f0222a4b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f021c15d4f0 con 0x7f021c0ad3b0 2023-11-09T03:52:53.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.642+0000 7f0222a4b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f021c0664f0 con 0x7f021c15d110 2023-11-09T03:52:53.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.642+0000 7f0222a4b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f021c06b9f0 con 0x7f021c0b3ca0 2023-11-09T03:52:53.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.642+0000 7f021b7fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f021c0b3ca0 0x7f021c0ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:53.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.642+0000 7f021bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f021c15d110 0x7f021c06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:53.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.642+0000 7f021b7fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f021c0b3ca0 0x7f021c0ace70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:50504/0 (socket says 172.21.15.76:50504) 2023-11-09T03:52:53.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.642+0000 7f021b7fe640 1 -- 172.21.15.76:0/1717151043 learned_addr learned my addr 172.21.15.76:0/1717151043 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:53.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.642+0000 7f0220fe8640 1 --1- 172.21.15.76:0/1717151043 >> v1:172.21.15.113:6789/0 conn(0x7f021c0ad3b0 0x7f021c0b2100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:60772/0 (socket says 172.21.15.76:60772) 2023-11-09T03:52:53.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.642+0000 7f021bfff640 1 -- 172.21.15.76:0/1717151043 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f021c0b3ca0 msgr2=0x7f021c0ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:53.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.642+0000 7f021bfff640 1 --2- 172.21.15.76:0/1717151043 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f021c0b3ca0 0x7f021c0ace70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:53.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.642+0000 7f021bfff640 1 -- 172.21.15.76:0/1717151043 >> v1:172.21.15.113:6789/0 conn(0x7f021c0ad3b0 legacy=0x7f021c0b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:53.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.642+0000 7f021bfff640 1 -- 172.21.15.76:0/1717151043 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021c06c2e0 con 0x7f021c15d110 2023-11-09T03:52:53.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.643+0000 7f021bfff640 1 --2- 172.21.15.76:0/1717151043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f021c15d110 0x7f021c06b2f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0210009790 tx=0x7f021002b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc60cb2e7bf75bf4 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:53.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.643+0000 7f021affd640 1 -- 172.21.15.76:0/1717151043 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0210031800 con 0x7f021c15d110 2023-11-09T03:52:53.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.643+0000 7f021affd640 1 -- 172.21.15.76:0/1717151043 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0210031960 con 0x7f021c15d110 2023-11-09T03:52:53.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.643+0000 7f021affd640 1 -- 172.21.15.76:0/1717151043 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0210031cf0 con 0x7f021c15d110 2023-11-09T03:52:53.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.644+0000 7f0222a4b640 1 -- 172.21.15.76:0/1717151043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f021c15d110 msgr2=0x7f021c06b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:53.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.644+0000 7f0222a4b640 1 --2- 172.21.15.76:0/1717151043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f021c15d110 0x7f021c06b2f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0210009790 tx=0x7f021002b8a0 comp rx=0 tx=0).stop 2023-11-09T03:52:53.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.644+0000 7f0222a4b640 1 -- 172.21.15.76:0/1717151043 shutdown_connections 2023-11-09T03:52:53.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.644+0000 7f0222a4b640 1 --2- 172.21.15.76:0/1717151043 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f021c0b3ca0 0x7f021c0ace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:53.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.644+0000 7f0222a4b640 1 --2- 172.21.15.76:0/1717151043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f021c15d110 0x7f021c06b2f0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:53.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.644+0000 7f0222a4b640 1 -- 172.21.15.76:0/1717151043 >> 172.21.15.76:0/1717151043 conn(0x7f021c0cbd30 msgr2=0x7f021c0cc140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:53.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.644+0000 7f0222a4b640 1 -- 172.21.15.76:0/1717151043 shutdown_connections 2023-11-09T03:52:53.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.644+0000 7f0222a4b640 1 -- 172.21.15.76:0/1717151043 wait complete. 2023-11-09T03:52:53.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.645+0000 7f0222a4b640 1 Processor -- start 2023-11-09T03:52:53.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.645+0000 7f0222a4b640 1 -- start start 2023-11-09T03:52:53.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.646+0000 7f0222a4b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f021c0ad3b0 0x7f021c071ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:53.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.646+0000 7f0222a4b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f021c0b3ca0 0x7f021c0ce860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:53.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.646+0000 7f0222a4b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f021c15d110 0x7f021c0700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:53.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.646+0000 7f0222a4b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021c0ccc30 con 0x7f021c15d110 2023-11-09T03:52:53.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.646+0000 7f0222a4b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021c072530 con 0x7f021c0b3ca0 2023-11-09T03:52:53.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.646+0000 7f0222a4b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f021c0ceda0 con 0x7f021c0ad3b0 2023-11-09T03:52:53.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.646+0000 7f0220fe8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f021c15d110 0x7f021c0700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:53.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.646+0000 7f0220fe8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f021c15d110 0x7f021c0700e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42028/0 (socket says 172.21.15.76:42028) 2023-11-09T03:52:53.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.646+0000 7f0220fe8640 1 -- 172.21.15.76:0/1748728371 learned_addr learned my addr 172.21.15.76:0/1748728371 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:53.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.646+0000 7f021bfff640 1 --2- 172.21.15.76:0/1748728371 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f021c0ad3b0 0x7f021c071ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:53.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.646+0000 7f021b7fe640 1 --2- 172.21.15.76:0/1748728371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f021c0b3ca0 0x7f021c0ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:53.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.646+0000 7f0220fe8640 1 -- 172.21.15.76:0/1748728371 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f021c0ad3b0 msgr2=0x7f021c071ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:53.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.646+0000 7f0220fe8640 1 --2- 172.21.15.76:0/1748728371 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f021c0ad3b0 0x7f021c071ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:53.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.646+0000 7f0220fe8640 1 -- 172.21.15.76:0/1748728371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f021c0b3ca0 msgr2=0x7f021c0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:53.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.646+0000 7f0220fe8640 1 --2- 172.21.15.76:0/1748728371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f021c0b3ca0 0x7f021c0ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:53.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.646+0000 7f0220fe8640 1 -- 172.21.15.76:0/1748728371 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02100093a0 con 0x7f021c15d110 2023-11-09T03:52:53.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.646+0000 7f0220fe8640 1 --2- 172.21.15.76:0/1748728371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f021c15d110 0x7f021c0700e0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f021400c6c0 tx=0x7f021400cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:53.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.647+0000 7f0218ff9640 1 -- 172.21.15.76:0/1748728371 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214004d60 con 0x7f021c15d110 2023-11-09T03:52:53.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.647+0000 7f0218ff9640 1 -- 172.21.15.76:0/1748728371 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021400a640 con 0x7f021c15d110 2023-11-09T03:52:53.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.647+0000 7f0218ff9640 1 -- 172.21.15.76:0/1748728371 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021400a9f0 con 0x7f021c15d110 2023-11-09T03:52:53.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.647+0000 7f0222a4b640 1 -- 172.21.15.76:0/1748728371 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f021c071bc0 con 0x7f021c15d110 2023-11-09T03:52:53.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.647+0000 7f0222a4b640 1 -- 172.21.15.76:0/1748728371 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f021c1f3370 con 0x7f021c15d110 2023-11-09T03:52:53.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.648+0000 7f0222a4b640 1 -- 172.21.15.76:0/1748728371 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f021c0ccda0 con 0x7f021c15d110 2023-11-09T03:52:53.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.648+0000 7f0218ff9640 1 -- 172.21.15.76:0/1748728371 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f021400ab50 con 0x7f021c15d110 2023-11-09T03:52:53.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.648+0000 7f0218ff9640 1 --2- 172.21.15.76:0/1748728371 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f020806ae70 0x7f020806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:53.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.648+0000 7f0218ff9640 1 -- 172.21.15.76:0/1748728371 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7952+0+0 (secure 0 0 0) 0x7f0214089200 con 0x7f021c15d110 2023-11-09T03:52:53.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.649+0000 7f021bfff640 1 --2- 172.21.15.76:0/1748728371 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f020806ae70 0x7f020806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:53.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.649+0000 7f021bfff640 1 --2- 172.21.15.76:0/1748728371 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f020806ae70 0x7f020806d330 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0210036000 tx=0x7f021003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:53.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.653+0000 7f0218ff9640 1 -- 172.21.15.76:0/1748728371 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f021c0ccda0 con 0x7f021c15d110 2023-11-09T03:52:53.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:53.888+0000 7f0222a4b640 1 -- 172.21.15.76:0/1748728371 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"} v 0) v1 -- 0x7f021c0ac8c0 con 0x7f021c15d110 2023-11-09T03:52:54.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.518+0000 7f0218ff9640 1 -- 172.21.15.76:0/1748728371 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"}]=0 freezing mds.7144 v107) v1 ==== 112+0+0 (secure 0 0 0) 0x7f0214057530 con 0x7f021c15d110 2023-11-09T03:52:54.518 INFO:teuthology.orchestra.run.smithi076.stderr:freezing mds.7144 2023-11-09T03:52:54.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.521+0000 7f0222a4b640 1 -- 172.21.15.76:0/1748728371 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f020806ae70 msgr2=0x7f020806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:54.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.521+0000 7f0222a4b640 1 --2- 172.21.15.76:0/1748728371 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f020806ae70 0x7f020806d330 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0210036000 tx=0x7f021003c040 comp rx=0 tx=0).stop 2023-11-09T03:52:54.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.521+0000 7f0222a4b640 1 -- 172.21.15.76:0/1748728371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f021c15d110 msgr2=0x7f021c0700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:54.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.521+0000 7f0222a4b640 1 --2- 172.21.15.76:0/1748728371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f021c15d110 0x7f021c0700e0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f021400c6c0 tx=0x7f021400cb80 comp rx=0 tx=0).stop 2023-11-09T03:52:54.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.521+0000 7f0222a4b640 1 -- 172.21.15.76:0/1748728371 shutdown_connections 2023-11-09T03:52:54.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.521+0000 7f0222a4b640 1 --2- 172.21.15.76:0/1748728371 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f020806ae70 0x7f020806d330 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:54.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.521+0000 7f0222a4b640 1 --2- 172.21.15.76:0/1748728371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f021c15d110 0x7f021c0700e0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:54.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.521+0000 7f0222a4b640 1 --2- 172.21.15.76:0/1748728371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f021c0b3ca0 0x7f021c0ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:54.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.521+0000 7f0222a4b640 1 --2- 172.21.15.76:0/1748728371 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f021c0ad3b0 0x7f021c071ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:54.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.521+0000 7f0222a4b640 1 -- 172.21.15.76:0/1748728371 >> 172.21.15.76:0/1748728371 conn(0x7f021c0cbd30 msgr2=0x7f021c06a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:54.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.521+0000 7f0222a4b640 1 -- 172.21.15.76:0/1748728371 shutdown_connections 2023-11-09T03:52:54.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.521+0000 7f0222a4b640 1 -- 172.21.15.76:0/1748728371 wait complete. 2023-11-09T03:52:54.533 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_kill_mdstable_at 6 2023-11-09T03:52:54.647 INFO:teuthology.orchestra.run.smithi076.stdout:{"success":"mds_kill_mdstable_at = '6' (not observed, change may require restart) "} 2023-11-09T03:52:54.647 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '6' (not observed, change may require restart) " } 2023-11-09T03:52:54.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:52:54.648 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s26) 2023-11-09T03:52:54.651 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2023-11-09T03:52:54.777 INFO:teuthology.orchestra.run.smithi076.stdout:{"mds_beacon_grace":"15.000000"} 2023-11-09T03:52:54.778 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2023-11-09T03:52:54.778 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:52:54.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.879+0000 7f0aad490640 1 Processor -- start 2023-11-09T03:52:54.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.879+0000 7f0aad490640 1 -- start start 2023-11-09T03:52:54.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.880+0000 7f0aad490640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0aa80b82c0 0x7f0aa806a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:54.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.880+0000 7f0aad490640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa80b6120 con 0x7f0aa80b75a0 2023-11-09T03:52:54.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.880+0000 7f0aad490640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa80d2750 con 0x7f0aa80d2ab0 2023-11-09T03:52:54.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.880+0000 7f0aad490640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aa80ce570 con 0x7f0aa80b82c0 2023-11-09T03:52:54.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.880+0000 7f0aa77fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0aa80b75a0 0x7f0aa80cde60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:42810/0 (socket says 172.21.15.76:42810) 2023-11-09T03:52:54.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.880+0000 7f0aa77fe640 1 -- 172.21.15.76:0/1745945750 learned_addr learned my addr 172.21.15.76:0/1745945750 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:52:54.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.880+0000 7f0aa6ffd640 1 --2- 172.21.15.76:0/1745945750 >> v2:172.21.15.113:3300/0 conn(0x7f0aa80b82c0 0x7f0aa806a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:54.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.880+0000 7f0aa5ffb640 1 -- 172.21.15.76:0/1745945750 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 26675439 0 0) 0x7f0aa80b6120 con 0x7f0aa80b75a0 2023-11-09T03:52:54.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.880+0000 7f0aa5ffb640 1 -- 172.21.15.76:0/1745945750 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a90003520 con 0x7f0aa80b75a0 2023-11-09T03:52:54.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.880+0000 7f0aa5ffb640 1 -- 172.21.15.76:0/1745945750 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2200262092 0 0) 0x7f0a90003520 con 0x7f0aa80b75a0 2023-11-09T03:52:54.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.880+0000 7f0aa5ffb640 1 -- 172.21.15.76:0/1745945750 >> v2:172.21.15.113:3300/0 conn(0x7f0aa80b82c0 msgr2=0x7f0aa806a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:54.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.880+0000 7f0aa5ffb640 1 --2- 172.21.15.76:0/1745945750 >> v2:172.21.15.113:3300/0 conn(0x7f0aa80b82c0 0x7f0aa806a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:54.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.880+0000 7f0aa5ffb640 1 -- 172.21.15.76:0/1745945750 >> v1:172.21.15.113:6789/0 conn(0x7f0aa80d2ab0 legacy=0x7f0aa80cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:54.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.881+0000 7f0aa5ffb640 1 -- 172.21.15.76:0/1745945750 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa80d1d70 con 0x7f0aa80b75a0 2023-11-09T03:52:54.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.881+0000 7f0aa5ffb640 1 -- 172.21.15.76:0/1745945750 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f0aa00019f0 con 0x7f0aa80b75a0 2023-11-09T03:52:54.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.881+0000 7f0aa5ffb640 1 -- 172.21.15.76:0/1745945750 >> v1:172.21.15.76:6789/0 conn(0x7f0aa80b75a0 legacy=0x7f0aa80cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:54.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.881+0000 7f0aa5ffb640 1 --2- 172.21.15.76:0/1745945750 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a90003970 0x7f0a90003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:54.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.881+0000 7f0aa5ffb640 1 --2- 172.21.15.76:0/1745945750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a900047c0 0x7f0a90004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:54.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.881+0000 7f0aa5ffb640 1 --2- 172.21.15.76:0/1745945750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a90005180 0x7f0a9000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:54.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.881+0000 7f0aa5ffb640 1 -- 172.21.15.76:0/1745945750 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa80ce570 con 0x7f0a90005180 2023-11-09T03:52:54.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.881+0000 7f0aa5ffb640 1 -- 172.21.15.76:0/1745945750 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9000c020 con 0x7f0a900047c0 2023-11-09T03:52:54.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.881+0000 7f0aa5ffb640 1 -- 172.21.15.76:0/1745945750 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9000c1e0 con 0x7f0a90003970 2023-11-09T03:52:54.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.881+0000 7f0aa77fe640 1 --2- 172.21.15.76:0/1745945750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a90005180 0x7f0a9000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:54.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.881+0000 7f0aa67fc640 1 --2- 172.21.15.76:0/1745945750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a900047c0 0x7f0a90004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:54.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.881+0000 7f0aa6ffd640 1 --2- 172.21.15.76:0/1745945750 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a90003970 0x7f0a90003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:54.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.881+0000 7f0aa77fe640 1 -- 172.21.15.76:0/1745945750 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a90003970 msgr2=0x7f0a90003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:54.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.881+0000 7f0aa77fe640 1 --2- 172.21.15.76:0/1745945750 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a90003970 0x7f0a90003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:54.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.881+0000 7f0aa77fe640 1 -- 172.21.15.76:0/1745945750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a900047c0 msgr2=0x7f0a90004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:52:54.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.881+0000 7f0aa77fe640 1 --2- 172.21.15.76:0/1745945750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a900047c0 0x7f0a90004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:54.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.881+0000 7f0aa77fe640 1 -- 172.21.15.76:0/1745945750 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a90003520 con 0x7f0a90005180 2023-11-09T03:52:54.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.882+0000 7f0aa77fe640 1 -- 172.21.15.76:0/1745945750 reap_dead start 2023-11-09T03:52:54.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.882+0000 7f0aa77fe640 1 --2- 172.21.15.76:0/1745945750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a90005180 0x7f0a9000b980 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f0aa000a820 tx=0x7f0a900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7dff7f5a2a40d72 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:54.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.882+0000 7f0aa5ffb640 1 -- 172.21.15.76:0/1745945750 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa00072d0 con 0x7f0a90005180 2023-11-09T03:52:54.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.882+0000 7f0aa5ffb640 1 -- 172.21.15.76:0/1745945750 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa002f070 con 0x7f0a90005180 2023-11-09T03:52:54.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.882+0000 7f0aa5ffb640 1 -- 172.21.15.76:0/1745945750 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa00076b0 con 0x7f0a90005180 2023-11-09T03:52:54.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.882+0000 7f0aad490640 1 -- 172.21.15.76:0/1745945750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a90005180 msgr2=0x7f0a9000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:54.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.882+0000 7f0aad490640 1 --2- 172.21.15.76:0/1745945750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a90005180 0x7f0a9000b980 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f0aa000a820 tx=0x7f0a900057f0 comp rx=0 tx=0).stop 2023-11-09T03:52:54.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.882+0000 7f0aad490640 1 -- 172.21.15.76:0/1745945750 shutdown_connections 2023-11-09T03:52:54.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.882+0000 7f0aad490640 1 --2- 172.21.15.76:0/1745945750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a90005180 0x7f0a9000b980 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:54.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.882+0000 7f0aad490640 1 -- 172.21.15.76:0/1745945750 >> 172.21.15.76:0/1745945750 conn(0x7f0aa80d2ef0 msgr2=0x7f0aa80d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:54.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.882+0000 7f0aad490640 1 -- 172.21.15.76:0/1745945750 shutdown_connections 2023-11-09T03:52:54.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.883+0000 7f0aad490640 1 -- 172.21.15.76:0/1745945750 wait complete. 2023-11-09T03:52:54.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.884+0000 7f0aad490640 1 Processor -- start 2023-11-09T03:52:54.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.884+0000 7f0aad490640 1 -- start start 2023-11-09T03:52:54.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.885+0000 7f0aad490640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a90005180 0x7f0aa81f3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:54.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.885+0000 7f0aad490640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0aa80cb010 0x7f0aa8152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:54.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.885+0000 7f0aad490640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0aa80cc650 0x7f0aa80abee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:54.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.885+0000 7f0aad490640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa815e880 con 0x7f0aa80cb010 2023-11-09T03:52:54.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.885+0000 7f0aad490640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa80cc1c0 con 0x7f0a90005180 2023-11-09T03:52:54.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.885+0000 7f0aad490640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa81f3590 con 0x7f0aa80cc650 2023-11-09T03:52:54.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.885+0000 7f0aa67fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0aa80cb010 0x7f0aa8152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:54.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.885+0000 7f0aa67fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0aa80cb010 0x7f0aa8152310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42034/0 (socket says 172.21.15.76:42034) 2023-11-09T03:52:54.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.885+0000 7f0aa67fc640 1 -- 172.21.15.76:0/2793266643 learned_addr learned my addr 172.21.15.76:0/2793266643 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:52:54.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.885+0000 7f0aa6ffd640 1 --2- 172.21.15.76:0/2793266643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a90005180 0x7f0aa81f3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:54.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.885+0000 7f0aa77fe640 1 --2- 172.21.15.76:0/2793266643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0aa80cc650 0x7f0aa80abee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:54.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.885+0000 7f0aa67fc640 1 -- 172.21.15.76:0/2793266643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0aa80cc650 msgr2=0x7f0aa80abee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:54.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.885+0000 7f0aa67fc640 1 --2- 172.21.15.76:0/2793266643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0aa80cc650 0x7f0aa80abee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:54.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.885+0000 7f0aa67fc640 1 -- 172.21.15.76:0/2793266643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a90005180 msgr2=0x7f0aa81f3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:54.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.885+0000 7f0aa67fc640 1 --2- 172.21.15.76:0/2793266643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a90005180 0x7f0aa81f3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:54.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.885+0000 7f0aa67fc640 1 -- 172.21.15.76:0/2793266643 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa80cb5b0 con 0x7f0aa80cb010 2023-11-09T03:52:54.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.885+0000 7f0aa6ffd640 1 --2- 172.21.15.76:0/2793266643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a90005180 0x7f0aa81f3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:52:54.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.885+0000 7f0aa67fc640 1 --2- 172.21.15.76:0/2793266643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0aa80cb010 0x7f0aa8152310 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c00f5c0 tx=0x7f0a9c00fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:54.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.885+0000 7f0a8f7fe640 1 -- 172.21.15.76:0/2793266643 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c0052e0 con 0x7f0aa80cb010 2023-11-09T03:52:54.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.885+0000 7f0a8f7fe640 1 -- 172.21.15.76:0/2793266643 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9c005440 con 0x7f0aa80cb010 2023-11-09T03:52:54.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.885+0000 7f0a8f7fe640 1 -- 172.21.15.76:0/2793266643 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c003830 con 0x7f0aa80cb010 2023-11-09T03:52:54.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.885+0000 7f0aad490640 1 -- 172.21.15.76:0/2793266643 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa80cb890 con 0x7f0aa80cb010 2023-11-09T03:52:54.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.885+0000 7f0aad490640 1 -- 172.21.15.76:0/2793266643 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa80cbe50 con 0x7f0aa80cb010 2023-11-09T03:52:54.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.886+0000 7f0aad490640 1 -- 172.21.15.76:0/2793266643 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa80684b0 con 0x7f0aa80cb010 2023-11-09T03:52:54.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.887+0000 7f0a8f7fe640 1 -- 172.21.15.76:0/2793266643 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0a9c003990 con 0x7f0aa80cb010 2023-11-09T03:52:54.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.887+0000 7f0a8f7fe640 1 --2- 172.21.15.76:0/2793266643 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a9406ae70 0x7f0a9406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:52:54.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.887+0000 7f0a8f7fe640 1 -- 172.21.15.76:0/2793266643 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7952+0+0 (secure 0 0 0) 0x7f0a9c089d60 con 0x7f0aa80cb010 2023-11-09T03:52:54.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.887+0000 7f0aa6ffd640 1 --2- 172.21.15.76:0/2793266643 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a9406ae70 0x7f0a9406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:52:54.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.888+0000 7f0aa6ffd640 1 --2- 172.21.15.76:0/2793266643 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a9406ae70 0x7f0a9406d330 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f0a98009b40 tx=0x7f0a98008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:52:54.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:54.891+0000 7f0a8f7fe640 1 -- 172.21.15.76:0/2793266643 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0a9c054550 con 0x7f0aa80cb010 2023-11-09T03:52:55.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:55.090+0000 7f0aad490640 1 -- 172.21.15.76:0/2793266643 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0aa80cbfe0 con 0x7f0aa80cb010 2023-11-09T03:52:55.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:55.090+0000 7f0a8f7fe640 1 -- 172.21.15.76:0/2793266643 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4924 (secure 0 0 0) 0x7f0a9c057b70 con 0x7f0aa80cb010 2023-11-09T03:52:55.090 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:52:55.091 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":107,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8568,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:52:53.889201+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8310,"mds_1":7144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7144":{"gid":7144,"name":"b","rank":1,"incarnation":101,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/7360844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":7360844},{"type":"v1","addr":"172.21.15.113:6837","nonce":7360844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8310":{"gid":8310,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":19,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:52:55.091 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 107 2023-11-09T03:52:55.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:55.092+0000 7f0aad490640 1 -- 172.21.15.76:0/2793266643 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a9406ae70 msgr2=0x7f0a9406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:55.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:55.092+0000 7f0aad490640 1 --2- 172.21.15.76:0/2793266643 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a9406ae70 0x7f0a9406d330 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f0a98009b40 tx=0x7f0a98008040 comp rx=0 tx=0).stop 2023-11-09T03:52:55.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:55.092+0000 7f0aad490640 1 -- 172.21.15.76:0/2793266643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0aa80cb010 msgr2=0x7f0aa8152310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:52:55.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:55.092+0000 7f0aad490640 1 --2- 172.21.15.76:0/2793266643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0aa80cb010 0x7f0aa8152310 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c00f5c0 tx=0x7f0a9c00fa80 comp rx=0 tx=0).stop 2023-11-09T03:52:55.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:55.092+0000 7f0aad490640 1 -- 172.21.15.76:0/2793266643 shutdown_connections 2023-11-09T03:52:55.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:55.092+0000 7f0aad490640 1 --2- 172.21.15.76:0/2793266643 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0a9406ae70 0x7f0a9406d330 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:55.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:55.092+0000 7f0aad490640 1 --2- 172.21.15.76:0/2793266643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0aa80cc650 0x7f0aa80abee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:55.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:55.092+0000 7f0aad490640 1 --2- 172.21.15.76:0/2793266643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0aa80cb010 0x7f0aa8152310 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:55.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:55.092+0000 7f0aad490640 1 --2- 172.21.15.76:0/2793266643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a90005180 0x7f0aa81f3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:52:55.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:55.092+0000 7f0aad490640 1 -- 172.21.15.76:0/2793266643 >> 172.21.15.76:0/2793266643 conn(0x7f0aa80d2ef0 msgr2=0x7f0aa80b39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:52:55.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:55.092+0000 7f0aad490640 1 -- 172.21.15.76:0/2793266643 shutdown_connections 2023-11-09T03:52:55.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:52:55.092+0000 7f0aad490640 1 -- 172.21.15.76:0/2793266643 wait complete. 2023-11-09T03:52:55.103 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:52:59.547 INFO:tasks.ceph.mds.a.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_commit_logged(ceph::cref_t&)' thread 7fc9fd533640 time 2023-11-09T03:52:59.547976+0000 2023-11-09T03:52:59.547 INFO:tasks.ceph.mds.a.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 171: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 6) 2023-11-09T03:52:59.548 INFO:tasks.ceph.mds.a.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:52:59.548 INFO:tasks.ceph.mds.a.smithi076.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fca0916efaf] 2023-11-09T03:52:59.548 INFO:tasks.ceph.mds.a.smithi076.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fca0916f16d] 2023-11-09T03:52:59.548 INFO:tasks.ceph.mds.a.smithi076.stderr: 3: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x41b) [0x55ad8fb1421b] 2023-11-09T03:52:59.548 INFO:tasks.ceph.mds.a.smithi076.stderr: 4: (MDSContext::complete(int)+0x5c) [0x55ad8fb43bcc] 2023-11-09T03:52:59.549 INFO:tasks.ceph.mds.a.smithi076.stderr: 5: (MDSIOContextBase::complete(int)+0x31c) [0x55ad8fb4cb0c] 2023-11-09T03:52:59.549 INFO:tasks.ceph.mds.a.smithi076.stderr: 6: (MDSLogContextBase::complete(int)+0x53) [0x55ad8fb4cc63] 2023-11-09T03:52:59.549 INFO:tasks.ceph.mds.a.smithi076.stderr: 7: (Finisher::finisher_thread_entry()+0x175) [0x7fca092164a5] 2023-11-09T03:52:59.549 INFO:tasks.ceph.mds.a.smithi076.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7fca0889f802] 2023-11-09T03:52:59.549 INFO:tasks.ceph.mds.a.smithi076.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7fca0883f450] 2023-11-09T03:52:59.550 INFO:tasks.ceph.mds.a.smithi076.stderr:2023-11-09T03:52:59.547+0000 7fc9fd533640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_commit_logged(ceph::cref_t&)' thread 7fc9fd533640 time 2023-11-09T03:52:59.547976+0000 2023-11-09T03:52:59.550 INFO:tasks.ceph.mds.a.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 171: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 6) 2023-11-09T03:52:59.550 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:52:59.550 INFO:tasks.ceph.mds.a.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:52:59.550 INFO:tasks.ceph.mds.a.smithi076.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fca0916efaf] 2023-11-09T03:52:59.550 INFO:tasks.ceph.mds.a.smithi076.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fca0916f16d] 2023-11-09T03:52:59.551 INFO:tasks.ceph.mds.a.smithi076.stderr: 3: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x41b) [0x55ad8fb1421b] 2023-11-09T03:52:59.551 INFO:tasks.ceph.mds.a.smithi076.stderr: 4: (MDSContext::complete(int)+0x5c) [0x55ad8fb43bcc] 2023-11-09T03:52:59.551 INFO:tasks.ceph.mds.a.smithi076.stderr: 5: (MDSIOContextBase::complete(int)+0x31c) [0x55ad8fb4cb0c] 2023-11-09T03:52:59.551 INFO:tasks.ceph.mds.a.smithi076.stderr: 6: (MDSLogContextBase::complete(int)+0x53) [0x55ad8fb4cc63] 2023-11-09T03:52:59.551 INFO:tasks.ceph.mds.a.smithi076.stderr: 7: (Finisher::finisher_thread_entry()+0x175) [0x7fca092164a5] 2023-11-09T03:52:59.551 INFO:tasks.ceph.mds.a.smithi076.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7fca0889f802] 2023-11-09T03:52:59.552 INFO:tasks.ceph.mds.a.smithi076.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7fca0883f450] 2023-11-09T03:52:59.552 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:52:59.552 INFO:tasks.ceph.mds.a.smithi076.stderr:*** Caught signal (Aborted) ** 2023-11-09T03:52:59.552 INFO:tasks.ceph.mds.a.smithi076.stderr: in thread 7fc9fd533640 thread_name:MR_Finisher 2023-11-09T03:52:59.552 INFO:tasks.ceph.mds.a.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:52:59.552 INFO:tasks.ceph.mds.a.smithi076.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fca08854db0] 2023-11-09T03:52:59.553 INFO:tasks.ceph.mds.a.smithi076.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fca088a154c] 2023-11-09T03:52:59.553 INFO:tasks.ceph.mds.a.smithi076.stderr: 3: raise() 2023-11-09T03:52:59.553 INFO:tasks.ceph.mds.a.smithi076.stderr: 4: abort() 2023-11-09T03:52:59.553 INFO:tasks.ceph.mds.a.smithi076.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fca0916f009] 2023-11-09T03:52:59.553 INFO:tasks.ceph.mds.a.smithi076.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fca0916f16d] 2023-11-09T03:52:59.554 INFO:tasks.ceph.mds.a.smithi076.stderr: 7: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x41b) [0x55ad8fb1421b] 2023-11-09T03:52:59.554 INFO:tasks.ceph.mds.a.smithi076.stderr: 8: (MDSContext::complete(int)+0x5c) [0x55ad8fb43bcc] 2023-11-09T03:52:59.554 INFO:tasks.ceph.mds.a.smithi076.stderr: 9: (MDSIOContextBase::complete(int)+0x31c) [0x55ad8fb4cb0c] 2023-11-09T03:52:59.554 INFO:tasks.ceph.mds.a.smithi076.stderr: 10: (MDSLogContextBase::complete(int)+0x53) [0x55ad8fb4cc63] 2023-11-09T03:52:59.554 INFO:tasks.ceph.mds.a.smithi076.stderr: 11: (Finisher::finisher_thread_entry()+0x175) [0x7fca092164a5] 2023-11-09T03:52:59.555 INFO:tasks.ceph.mds.a.smithi076.stderr: 12: /lib64/libc.so.6(+0x9f802) [0x7fca0889f802] 2023-11-09T03:52:59.555 INFO:tasks.ceph.mds.a.smithi076.stderr: 13: /lib64/libc.so.6(+0x3f450) [0x7fca0883f450] 2023-11-09T03:52:59.555 INFO:tasks.ceph.mds.a.smithi076.stderr:2023-11-09T03:52:59.548+0000 7fc9fd533640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:52:59.555 INFO:tasks.ceph.mds.a.smithi076.stderr: in thread 7fc9fd533640 thread_name:MR_Finisher 2023-11-09T03:52:59.555 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:52:59.555 INFO:tasks.ceph.mds.a.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:52:59.556 INFO:tasks.ceph.mds.a.smithi076.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fca08854db0] 2023-11-09T03:52:59.556 INFO:tasks.ceph.mds.a.smithi076.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fca088a154c] 2023-11-09T03:52:59.556 INFO:tasks.ceph.mds.a.smithi076.stderr: 3: raise() 2023-11-09T03:52:59.556 INFO:tasks.ceph.mds.a.smithi076.stderr: 4: abort() 2023-11-09T03:52:59.556 INFO:tasks.ceph.mds.a.smithi076.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fca0916f009] 2023-11-09T03:52:59.556 INFO:tasks.ceph.mds.a.smithi076.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fca0916f16d] 2023-11-09T03:52:59.557 INFO:tasks.ceph.mds.a.smithi076.stderr: 7: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x41b) [0x55ad8fb1421b] 2023-11-09T03:52:59.557 INFO:tasks.ceph.mds.a.smithi076.stderr: 8: (MDSContext::complete(int)+0x5c) [0x55ad8fb43bcc] 2023-11-09T03:52:59.557 INFO:tasks.ceph.mds.a.smithi076.stderr: 9: (MDSIOContextBase::complete(int)+0x31c) [0x55ad8fb4cb0c] 2023-11-09T03:52:59.557 INFO:tasks.ceph.mds.a.smithi076.stderr: 10: (MDSLogContextBase::complete(int)+0x53) [0x55ad8fb4cc63] 2023-11-09T03:52:59.557 INFO:tasks.ceph.mds.a.smithi076.stderr: 11: (Finisher::finisher_thread_entry()+0x175) [0x7fca092164a5] 2023-11-09T03:52:59.557 INFO:tasks.ceph.mds.a.smithi076.stderr: 12: /lib64/libc.so.6(+0x9f802) [0x7fca0889f802] 2023-11-09T03:52:59.558 INFO:tasks.ceph.mds.a.smithi076.stderr: 13: /lib64/libc.so.6(+0x3f450) [0x7fca0883f450] 2023-11-09T03:52:59.558 INFO:tasks.ceph.mds.a.smithi076.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:52:59.558 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:52:59.558 INFO:tasks.ceph.mds.a.smithi076.stderr: -789> 2023-11-09T03:52:50.458+0000 7fca00d3a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:52:59.559 INFO:tasks.ceph.mds.a.smithi076.stderr: -787> 2023-11-09T03:52:50.458+0000 7fca00d3a640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-09T03:52:59.559 INFO:tasks.ceph.mds.a.smithi076.stderr: -1> 2023-11-09T03:52:59.547+0000 7fc9fd533640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_commit_logged(ceph::cref_t&)' thread 7fc9fd533640 time 2023-11-09T03:52:59.547976+0000 2023-11-09T03:52:59.559 INFO:tasks.ceph.mds.a.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 171: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 6) 2023-11-09T03:52:59.559 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:52:59.560 INFO:tasks.ceph.mds.a.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:52:59.560 INFO:tasks.ceph.mds.a.smithi076.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fca0916efaf] 2023-11-09T03:52:59.560 INFO:tasks.ceph.mds.a.smithi076.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fca0916f16d] 2023-11-09T03:52:59.560 INFO:tasks.ceph.mds.a.smithi076.stderr: 3: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x41b) [0x55ad8fb1421b] 2023-11-09T03:52:59.560 INFO:tasks.ceph.mds.a.smithi076.stderr: 4: (MDSContext::complete(int)+0x5c) [0x55ad8fb43bcc] 2023-11-09T03:52:59.560 INFO:tasks.ceph.mds.a.smithi076.stderr: 5: (MDSIOContextBase::complete(int)+0x31c) [0x55ad8fb4cb0c] 2023-11-09T03:52:59.561 INFO:tasks.ceph.mds.a.smithi076.stderr: 6: (MDSLogContextBase::complete(int)+0x53) [0x55ad8fb4cc63] 2023-11-09T03:52:59.561 INFO:tasks.ceph.mds.a.smithi076.stderr: 7: (Finisher::finisher_thread_entry()+0x175) [0x7fca092164a5] 2023-11-09T03:52:59.561 INFO:tasks.ceph.mds.a.smithi076.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7fca0889f802] 2023-11-09T03:52:59.561 INFO:tasks.ceph.mds.a.smithi076.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7fca0883f450] 2023-11-09T03:52:59.561 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:52:59.561 INFO:tasks.ceph.mds.a.smithi076.stderr: 0> 2023-11-09T03:52:59.548+0000 7fc9fd533640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:52:59.562 INFO:tasks.ceph.mds.a.smithi076.stderr: in thread 7fc9fd533640 thread_name:MR_Finisher 2023-11-09T03:52:59.562 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:52:59.562 INFO:tasks.ceph.mds.a.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:52:59.562 INFO:tasks.ceph.mds.a.smithi076.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fca08854db0] 2023-11-09T03:52:59.562 INFO:tasks.ceph.mds.a.smithi076.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fca088a154c] 2023-11-09T03:52:59.562 INFO:tasks.ceph.mds.a.smithi076.stderr: 3: raise() 2023-11-09T03:52:59.563 INFO:tasks.ceph.mds.a.smithi076.stderr: 4: abort() 2023-11-09T03:52:59.563 INFO:tasks.ceph.mds.a.smithi076.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fca0916f009] 2023-11-09T03:52:59.563 INFO:tasks.ceph.mds.a.smithi076.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fca0916f16d] 2023-11-09T03:52:59.563 INFO:tasks.ceph.mds.a.smithi076.stderr: 7: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x41b) [0x55ad8fb1421b] 2023-11-09T03:52:59.563 INFO:tasks.ceph.mds.a.smithi076.stderr: 8: (MDSContext::complete(int)+0x5c) [0x55ad8fb43bcc] 2023-11-09T03:52:59.563 INFO:tasks.ceph.mds.a.smithi076.stderr: 9: (MDSIOContextBase::complete(int)+0x31c) [0x55ad8fb4cb0c] 2023-11-09T03:52:59.564 INFO:tasks.ceph.mds.a.smithi076.stderr: 10: (MDSLogContextBase::complete(int)+0x53) [0x55ad8fb4cc63] 2023-11-09T03:52:59.564 INFO:tasks.ceph.mds.a.smithi076.stderr: 11: (Finisher::finisher_thread_entry()+0x175) [0x7fca092164a5] 2023-11-09T03:52:59.564 INFO:tasks.ceph.mds.a.smithi076.stderr: 12: /lib64/libc.so.6(+0x9f802) [0x7fca0889f802] 2023-11-09T03:52:59.564 INFO:tasks.ceph.mds.a.smithi076.stderr: 13: /lib64/libc.so.6(+0x3f450) [0x7fca0883f450] 2023-11-09T03:52:59.564 INFO:tasks.ceph.mds.a.smithi076.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:52:59.564 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:52:59.565 INFO:tasks.ceph.mds.a.smithi076.stderr: -792> 2023-11-09T03:52:50.458+0000 7fca00d3a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-09T03:52:59.565 INFO:tasks.ceph.mds.a.smithi076.stderr: -790> 2023-11-09T03:52:50.458+0000 7fca00d3a640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-09T03:52:59.565 INFO:tasks.ceph.mds.a.smithi076.stderr: -4> 2023-11-09T03:52:59.547+0000 7fc9fd533640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_commit_logged(ceph::cref_t&)' thread 7fc9fd533640 time 2023-11-09T03:52:59.547976+0000 2023-11-09T03:52:59.565 INFO:tasks.ceph.mds.a.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 171: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 6) 2023-11-09T03:52:59.566 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:52:59.566 INFO:tasks.ceph.mds.a.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:52:59.566 INFO:tasks.ceph.mds.a.smithi076.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fca0916efaf] 2023-11-09T03:52:59.566 INFO:tasks.ceph.mds.a.smithi076.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fca0916f16d] 2023-11-09T03:52:59.566 INFO:tasks.ceph.mds.a.smithi076.stderr: 3: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x41b) [0x55ad8fb1421b] 2023-11-09T03:52:59.567 INFO:tasks.ceph.mds.a.smithi076.stderr: 4: (MDSContext::complete(int)+0x5c) [0x55ad8fb43bcc] 2023-11-09T03:52:59.567 INFO:tasks.ceph.mds.a.smithi076.stderr: 5: (MDSIOContextBase::complete(int)+0x31c) [0x55ad8fb4cb0c] 2023-11-09T03:52:59.567 INFO:tasks.ceph.mds.a.smithi076.stderr: 6: (MDSLogContextBase::complete(int)+0x53) [0x55ad8fb4cc63] 2023-11-09T03:52:59.567 INFO:tasks.ceph.mds.a.smithi076.stderr: 7: (Finisher::finisher_thread_entry()+0x175) [0x7fca092164a5] 2023-11-09T03:52:59.567 INFO:tasks.ceph.mds.a.smithi076.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7fca0889f802] 2023-11-09T03:52:59.567 INFO:tasks.ceph.mds.a.smithi076.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7fca0883f450] 2023-11-09T03:52:59.568 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:52:59.568 INFO:tasks.ceph.mds.a.smithi076.stderr: -3> 2023-11-09T03:52:59.548+0000 7fc9fd533640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:52:59.568 INFO:tasks.ceph.mds.a.smithi076.stderr: in thread 7fc9fd533640 thread_name:MR_Finisher 2023-11-09T03:52:59.568 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:52:59.568 INFO:tasks.ceph.mds.a.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:52:59.568 INFO:tasks.ceph.mds.a.smithi076.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fca08854db0] 2023-11-09T03:52:59.569 INFO:tasks.ceph.mds.a.smithi076.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fca088a154c] 2023-11-09T03:52:59.569 INFO:tasks.ceph.mds.a.smithi076.stderr: 3: raise() 2023-11-09T03:52:59.569 INFO:tasks.ceph.mds.a.smithi076.stderr: 4: abort() 2023-11-09T03:52:59.569 INFO:tasks.ceph.mds.a.smithi076.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fca0916f009] 2023-11-09T03:52:59.569 INFO:tasks.ceph.mds.a.smithi076.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fca0916f16d] 2023-11-09T03:52:59.569 INFO:tasks.ceph.mds.a.smithi076.stderr: 7: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x41b) [0x55ad8fb1421b] 2023-11-09T03:52:59.570 INFO:tasks.ceph.mds.a.smithi076.stderr: 8: (MDSContext::complete(int)+0x5c) [0x55ad8fb43bcc] 2023-11-09T03:52:59.570 INFO:tasks.ceph.mds.a.smithi076.stderr: 9: (MDSIOContextBase::complete(int)+0x31c) [0x55ad8fb4cb0c] 2023-11-09T03:52:59.570 INFO:tasks.ceph.mds.a.smithi076.stderr: 10: (MDSLogContextBase::complete(int)+0x53) [0x55ad8fb4cc63] 2023-11-09T03:52:59.570 INFO:tasks.ceph.mds.a.smithi076.stderr: 11: (Finisher::finisher_thread_entry()+0x175) [0x7fca092164a5] 2023-11-09T03:52:59.570 INFO:tasks.ceph.mds.a.smithi076.stderr: 12: /lib64/libc.so.6(+0x9f802) [0x7fca0889f802] 2023-11-09T03:52:59.570 INFO:tasks.ceph.mds.a.smithi076.stderr: 13: /lib64/libc.so.6(+0x3f450) [0x7fca0883f450] 2023-11-09T03:52:59.571 INFO:tasks.ceph.mds.a.smithi076.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:52:59.571 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:53:00.027 INFO:tasks.ceph.mds.a.smithi076.stderr:daemon-helper: command crashed with signal 6 2023-11-09T03:53:00.104 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:53:00.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.213+0000 7f1005abb640 1 Processor -- start 2023-11-09T03:53:00.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.213+0000 7f1005abb640 1 -- start start 2023-11-09T03:53:00.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.214+0000 7f1005abb640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f100015d4e0 0x7f10000ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:00.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.214+0000 7f1005abb640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10000b25a0 con 0x7f10000ced90 2023-11-09T03:53:00.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.214+0000 7f1005abb640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10000b2300 con 0x7f10000ca8b0 2023-11-09T03:53:00.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.214+0000 7f1005abb640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f100015d8c0 con 0x7f100015d4e0 2023-11-09T03:53:00.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.214+0000 7f0fff7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f100015d4e0 0x7f10000ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:00.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.214+0000 7f0ffeffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f10000ced90 0x7f10000ca1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:42824/0 (socket says 172.21.15.76:42824) 2023-11-09T03:53:00.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.214+0000 7f0fff7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f100015d4e0 0x7f10000ce850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42048/0 (socket says 172.21.15.76:42048) 2023-11-09T03:53:00.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.214+0000 7f0ffeffd640 1 -- 172.21.15.76:0/2970223642 learned_addr learned my addr 172.21.15.76:0/2970223642 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:53:00.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.215+0000 7f0fff7fe640 1 -- 172.21.15.76:0/2970223642 >> v1:172.21.15.113:6789/0 conn(0x7f10000ca8b0 legacy=0x7f100006a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:00.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.215+0000 7f0fff7fe640 1 -- 172.21.15.76:0/2970223642 >> v1:172.21.15.76:6789/0 conn(0x7f10000ced90 legacy=0x7f10000ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:00.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.215+0000 7f0fff7fe640 1 -- 172.21.15.76:0/2970223642 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100015e5f0 con 0x7f100015d4e0 2023-11-09T03:53:00.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.215+0000 7f0ffe7fc640 1 -- 172.21.15.76:0/2970223642 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3130338144 0 0) 0x7f10000b25a0 con 0x7f10000ced90 2023-11-09T03:53:00.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.215+0000 7f0fff7fe640 1 --2- 172.21.15.76:0/2970223642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f100015d4e0 0x7f10000ce850 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00097d0 tx=0x7f0ff002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a129c3d0f79ed server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:00.217 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.216+0000 7f0ffe7fc640 1 -- 172.21.15.76:0/2970223642 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0031830 con 0x7f100015d4e0 2023-11-09T03:53:00.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.216+0000 7f0ffe7fc640 1 -- 172.21.15.76:0/2970223642 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff0031990 con 0x7f100015d4e0 2023-11-09T03:53:00.218 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.216+0000 7f0ffe7fc640 1 -- 172.21.15.76:0/2970223642 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0031d20 con 0x7f100015d4e0 2023-11-09T03:53:00.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.216+0000 7f1005abb640 1 -- 172.21.15.76:0/2970223642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f100015d4e0 msgr2=0x7f10000ce850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:00.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.216+0000 7f1005abb640 1 --2- 172.21.15.76:0/2970223642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f100015d4e0 0x7f10000ce850 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00097d0 tx=0x7f0ff002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:53:00.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.216+0000 7f1005abb640 1 -- 172.21.15.76:0/2970223642 shutdown_connections 2023-11-09T03:53:00.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.216+0000 7f1005abb640 1 --2- 172.21.15.76:0/2970223642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f100015d4e0 0x7f10000ce850 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:00.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.216+0000 7f1005abb640 1 -- 172.21.15.76:0/2970223642 >> 172.21.15.76:0/2970223642 conn(0x7f10000b1640 msgr2=0x7f10000b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:00.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.216+0000 7f1005abb640 1 -- 172.21.15.76:0/2970223642 shutdown_connections 2023-11-09T03:53:00.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.216+0000 7f1005abb640 1 -- 172.21.15.76:0/2970223642 wait complete. 2023-11-09T03:53:00.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.217+0000 7f1005abb640 1 Processor -- start 2023-11-09T03:53:00.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.217+0000 7f1005abb640 1 -- start start 2023-11-09T03:53:00.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.218+0000 7f1005abb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10000ca8b0 0x7f10001549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:00.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.218+0000 7f1005abb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10000ced90 0x7f10001f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:00.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.218+0000 7f1005abb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f100015d4e0 0x7f10000c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:00.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.218+0000 7f1005abb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10000c7b50 con 0x7f10000ced90 2023-11-09T03:53:00.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.218+0000 7f1005abb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10001f78e0 con 0x7f100015d4e0 2023-11-09T03:53:00.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.218+0000 7f1005abb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1000155310 con 0x7f10000ca8b0 2023-11-09T03:53:00.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.218+0000 7f0ffeffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10000ced90 0x7f10001f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:00.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.218+0000 7f0ffeffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10000ced90 0x7f10001f73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42060/0 (socket says 172.21.15.76:42060) 2023-11-09T03:53:00.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.218+0000 7f0ffeffd640 1 -- 172.21.15.76:0/1536476822 learned_addr learned my addr 172.21.15.76:0/1536476822 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:00.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.219+0000 7f0ffeffd640 1 -- 172.21.15.76:0/1536476822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10000ca8b0 msgr2=0x7f10001549c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:00.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.219+0000 7f0ffeffd640 1 --2- 172.21.15.76:0/1536476822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10000ca8b0 0x7f10001549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:00.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.219+0000 7f0ffeffd640 1 -- 172.21.15.76:0/1536476822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f100015d4e0 msgr2=0x7f10000c8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:00.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.219+0000 7f0ffffff640 1 --2- 172.21.15.76:0/1536476822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f100015d4e0 0x7f10000c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:00.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.219+0000 7f0ffeffd640 1 --2- 172.21.15.76:0/1536476822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f100015d4e0 0x7f10000c8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:00.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.219+0000 7f0ffeffd640 1 -- 172.21.15.76:0/1536476822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff0009400 con 0x7f10000ced90 2023-11-09T03:53:00.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.219+0000 7f0ffeffd640 1 --2- 172.21.15.76:0/1536476822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10000ced90 0x7f10001f73a0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f0ff400c530 tx=0x7f0ff400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:00.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.219+0000 7f0fe7fff640 1 -- 172.21.15.76:0/1536476822 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff40074e0 con 0x7f10000ced90 2023-11-09T03:53:00.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.219+0000 7f0fe7fff640 1 -- 172.21.15.76:0/1536476822 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff4007660 con 0x7f10000ced90 2023-11-09T03:53:00.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.219+0000 7f1005abb640 1 -- 172.21.15.76:0/1536476822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10000c4850 con 0x7f10000ced90 2023-11-09T03:53:00.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.219+0000 7f0fe7fff640 1 -- 172.21.15.76:0/1536476822 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4004a20 con 0x7f10000ced90 2023-11-09T03:53:00.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.219+0000 7f1005abb640 1 -- 172.21.15.76:0/1536476822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10000b37d0 con 0x7f10000ced90 2023-11-09T03:53:00.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.220+0000 7f1005abb640 1 -- 172.21.15.76:0/1536476822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10000684b0 con 0x7f10000ced90 2023-11-09T03:53:00.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.220+0000 7f0fe7fff640 1 -- 172.21.15.76:0/1536476822 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0ff4004b80 con 0x7f10000ced90 2023-11-09T03:53:00.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.221+0000 7f0fe7fff640 1 --2- 172.21.15.76:0/1536476822 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0fec06ae70 0x7f0fec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:00.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.221+0000 7f0fe7fff640 1 -- 172.21.15.76:0/1536476822 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7952+0+0 (secure 0 0 0) 0x7f0ff4088fc0 con 0x7f10000ced90 2023-11-09T03:53:00.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.221+0000 7f0fff7fe640 1 --2- 172.21.15.76:0/1536476822 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0fec06ae70 0x7f0fec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:00.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.222+0000 7f0fff7fe640 1 --2- 172.21.15.76:0/1536476822 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0fec06ae70 0x7f0fec06d330 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0036000 tx=0x7f0ff0030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:00.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.225+0000 7f0fe7fff640 1 -- 172.21.15.76:0/1536476822 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0ff40bbca0 con 0x7f10000ced90 2023-11-09T03:53:00.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.424+0000 7f1005abb640 1 -- 172.21.15.76:0/1536476822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1000071aa0 con 0x7f10000ced90 2023-11-09T03:53:00.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.425+0000 7f0fe7fff640 1 -- 172.21.15.76:0/1536476822 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4924 (secure 0 0 0) 0x7f0ff4018070 con 0x7f10000ced90 2023-11-09T03:53:00.425 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:53:00.425 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":107,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8568,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:52:53.889201+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8310,"mds_1":7144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7144":{"gid":7144,"name":"b","rank":1,"incarnation":101,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/7360844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":7360844},{"type":"v1","addr":"172.21.15.113:6837","nonce":7360844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8310":{"gid":8310,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":19,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:53:00.425 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 107 2023-11-09T03:53:00.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.427+0000 7f1005abb640 1 -- 172.21.15.76:0/1536476822 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0fec06ae70 msgr2=0x7f0fec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:00.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.427+0000 7f1005abb640 1 --2- 172.21.15.76:0/1536476822 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0fec06ae70 0x7f0fec06d330 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0036000 tx=0x7f0ff0030000 comp rx=0 tx=0).stop 2023-11-09T03:53:00.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.427+0000 7f1005abb640 1 -- 172.21.15.76:0/1536476822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10000ced90 msgr2=0x7f10001f73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:00.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.427+0000 7f1005abb640 1 --2- 172.21.15.76:0/1536476822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10000ced90 0x7f10001f73a0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f0ff400c530 tx=0x7f0ff400c9f0 comp rx=0 tx=0).stop 2023-11-09T03:53:00.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.427+0000 7f1005abb640 1 -- 172.21.15.76:0/1536476822 shutdown_connections 2023-11-09T03:53:00.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.427+0000 7f1005abb640 1 --2- 172.21.15.76:0/1536476822 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0fec06ae70 0x7f0fec06d330 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:00.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.427+0000 7f1005abb640 1 --2- 172.21.15.76:0/1536476822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f100015d4e0 0x7f10000c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:00.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.427+0000 7f1005abb640 1 --2- 172.21.15.76:0/1536476822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10000ced90 0x7f10001f73a0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:00.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.427+0000 7f1005abb640 1 --2- 172.21.15.76:0/1536476822 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10000ca8b0 0x7f10001549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:00.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.427+0000 7f1005abb640 1 -- 172.21.15.76:0/1536476822 >> 172.21.15.76:0/1536476822 conn(0x7f10000b1640 msgr2=0x7f1000070900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:00.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.427+0000 7f1005abb640 1 -- 172.21.15.76:0/1536476822 shutdown_connections 2023-11-09T03:53:00.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:00.427+0000 7f1005abb640 1 -- 172.21.15.76:0/1536476822 wait complete. 2023-11-09T03:53:00.437 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:53:02.185 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:53:02.188 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:53:05.439 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:53:05.509 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-11-09T03:53:05.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.551+0000 7f5846155640 1 Processor -- start 2023-11-09T03:53:05.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.551+0000 7f5846155640 1 -- start start 2023-11-09T03:53:05.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.551+0000 7f5846155640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f584015d100 0x7f58400ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:05.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.551+0000 7f5846155640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f584015d4e0 0x7f58400d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:05.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.552+0000 7f5846155640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58400b32b0 con 0x7f58400d3b60 2023-11-09T03:53:05.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.552+0000 7f5846155640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58400cee10 con 0x7f584015d4e0 2023-11-09T03:53:05.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.552+0000 7f5846155640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58400b4750 con 0x7f584015d100 2023-11-09T03:53:05.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.552+0000 7f5836ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f584015d4e0 0x7f58400d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:05.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.552+0000 7f5836ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f584015d4e0 0x7f58400d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59360/0 (socket says 172.21.15.76:59360) 2023-11-09T03:53:05.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.552+0000 7f583ffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f58400d3b60 0x7f58400cf520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:55596/0 (socket says 172.21.15.76:55596) 2023-11-09T03:53:05.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.552+0000 7f5836ffd640 1 -- 172.21.15.76:0/701968357 learned_addr learned my addr 172.21.15.76:0/701968357 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:05.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.552+0000 7f583f7fe640 1 --2- 172.21.15.76:0/701968357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f584015d100 0x7f58400ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:05.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.552+0000 7f5836ffd640 1 -- 172.21.15.76:0/701968357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f584015d100 msgr2=0x7f58400ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:05.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.552+0000 7f5836ffd640 1 --2- 172.21.15.76:0/701968357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f584015d100 0x7f58400ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:05.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.552+0000 7f5836ffd640 1 -- 172.21.15.76:0/701968357 >> v1:172.21.15.76:6789/0 conn(0x7f58400d3b60 legacy=0x7f58400cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:05.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.552+0000 7f5836ffd640 1 -- 172.21.15.76:0/701968357 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58400b2980 con 0x7f584015d4e0 2023-11-09T03:53:05.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.553+0000 7f5836ffd640 1 --2- 172.21.15.76:0/701968357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f584015d4e0 0x7f58400d14d0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f5828009870 tx=0x7f582802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa426630b17f175a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:05.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.553+0000 7f583effd640 1 -- 172.21.15.76:0/701968357 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5828030840 con 0x7f584015d4e0 2023-11-09T03:53:05.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.553+0000 7f583effd640 1 -- 172.21.15.76:0/701968357 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58280309a0 con 0x7f584015d4e0 2023-11-09T03:53:05.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.553+0000 7f583effd640 1 -- 172.21.15.76:0/701968357 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5828030d30 con 0x7f584015d4e0 2023-11-09T03:53:05.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.553+0000 7f5846155640 1 -- 172.21.15.76:0/701968357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f584015d4e0 msgr2=0x7f58400d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:05.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.553+0000 7f5846155640 1 --2- 172.21.15.76:0/701968357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f584015d4e0 0x7f58400d14d0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f5828009870 tx=0x7f582802b980 comp rx=0 tx=0).stop 2023-11-09T03:53:05.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.553+0000 7f5846155640 1 -- 172.21.15.76:0/701968357 shutdown_connections 2023-11-09T03:53:05.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.553+0000 7f5846155640 1 --2- 172.21.15.76:0/701968357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f584015d4e0 0x7f58400d14d0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:05.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.553+0000 7f5846155640 1 --2- 172.21.15.76:0/701968357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f584015d100 0x7f58400ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:05.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.554+0000 7f5846155640 1 -- 172.21.15.76:0/701968357 >> 172.21.15.76:0/701968357 conn(0x7f58400d2ef0 msgr2=0x7f58400d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:05.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.554+0000 7f5846155640 1 -- 172.21.15.76:0/701968357 shutdown_connections 2023-11-09T03:53:05.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.554+0000 7f5846155640 1 -- 172.21.15.76:0/701968357 wait complete. 2023-11-09T03:53:05.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.555+0000 7f5846155640 1 Processor -- start 2023-11-09T03:53:05.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.555+0000 7f5846155640 1 -- start start 2023-11-09T03:53:05.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.556+0000 7f5846155640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58400d3b60 0x7f58401f3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:05.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.556+0000 7f5846155640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f584015d100 0x7f584006bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:05.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.556+0000 7f5846155640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f584015d4e0 0x7f584006c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:05.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.556+0000 7f5846155640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58401f3860 con 0x7f58400d3b60 2023-11-09T03:53:05.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.556+0000 7f5846155640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58400cc560 con 0x7f584015d4e0 2023-11-09T03:53:05.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.556+0000 7f5846155640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58400cc840 con 0x7f584015d100 2023-11-09T03:53:05.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.556+0000 7f583f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58400d3b60 0x7f58401f3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:05.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.556+0000 7f583f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58400d3b60 0x7f58401f3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59376/0 (socket says 172.21.15.76:59376) 2023-11-09T03:53:05.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.556+0000 7f583f7fe640 1 -- 172.21.15.76:0/3734117423 learned_addr learned my addr 172.21.15.76:0/3734117423 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:05.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.556+0000 7f5836ffd640 1 --2- 172.21.15.76:0/3734117423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f584015d100 0x7f584006bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:05.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.556+0000 7f583ffff640 1 --2- 172.21.15.76:0/3734117423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f584015d4e0 0x7f584006c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:05.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.556+0000 7f583f7fe640 1 -- 172.21.15.76:0/3734117423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f584015d100 msgr2=0x7f584006bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:05.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.556+0000 7f583f7fe640 1 --2- 172.21.15.76:0/3734117423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f584015d100 0x7f584006bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:05.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.556+0000 7f583f7fe640 1 -- 172.21.15.76:0/3734117423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f584015d4e0 msgr2=0x7f584006c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:05.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.556+0000 7f583f7fe640 1 --2- 172.21.15.76:0/3734117423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f584015d4e0 0x7f584006c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:05.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.556+0000 7f583f7fe640 1 -- 172.21.15.76:0/3734117423 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58280094d0 con 0x7f58400d3b60 2023-11-09T03:53:05.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.556+0000 7f5836ffd640 1 --2- 172.21.15.76:0/3734117423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f584015d100 0x7f584006bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:53:05.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.556+0000 7f583ffff640 1 --2- 172.21.15.76:0/3734117423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f584015d4e0 0x7f584006c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:53:05.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.556+0000 7f583f7fe640 1 --2- 172.21.15.76:0/3734117423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58400d3b60 0x7f58401f3320 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f583000d7d0 tx=0x7f583000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:05.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.556+0000 7f583cff9640 1 -- 172.21.15.76:0/3734117423 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58300096c0 con 0x7f58400d3b60 2023-11-09T03:53:05.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.556+0000 7f583cff9640 1 -- 172.21.15.76:0/3734117423 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5830009820 con 0x7f58400d3b60 2023-11-09T03:53:05.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.556+0000 7f583cff9640 1 -- 172.21.15.76:0/3734117423 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5830009bd0 con 0x7f58400d3b60 2023-11-09T03:53:05.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.556+0000 7f5846155640 1 -- 172.21.15.76:0/3734117423 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58400ccb20 con 0x7f58400d3b60 2023-11-09T03:53:05.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.556+0000 7f5846155640 1 -- 172.21.15.76:0/3734117423 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58401f39c0 con 0x7f58400d3b60 2023-11-09T03:53:05.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.557+0000 7f5846155640 1 -- 172.21.15.76:0/3734117423 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58400684b0 con 0x7f58400d3b60 2023-11-09T03:53:05.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.558+0000 7f583cff9640 1 -- 172.21.15.76:0/3734117423 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5830009d30 con 0x7f58400d3b60 2023-11-09T03:53:05.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.558+0000 7f583cff9640 1 --2- 172.21.15.76:0/3734117423 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f582006ae70 0x7f582006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:05.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.558+0000 7f583cff9640 1 -- 172.21.15.76:0/3734117423 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7952+0+0 (secure 0 0 0) 0x7f583008a6e0 con 0x7f58400d3b60 2023-11-09T03:53:05.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.559+0000 7f5836ffd640 1 --2- 172.21.15.76:0/3734117423 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f582006ae70 0x7f582006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:05.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.559+0000 7f5836ffd640 1 --2- 172.21.15.76:0/3734117423 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f582006ae70 0x7f582006d330 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5828036000 tx=0x7f582803b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:05.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.563+0000 7f583cff9640 1 -- 172.21.15.76:0/3734117423 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5830054ed0 con 0x7f58400d3b60 2023-11-09T03:53:05.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.763+0000 7f5846155640 1 -- 172.21.15.76:0/3734117423 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58400d2cd0 con 0x7f58400d3b60 2023-11-09T03:53:05.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.763+0000 7f583cff9640 1 -- 172.21.15.76:0/3734117423 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4924 (secure 0 0 0) 0x7f58300584f0 con 0x7f58400d3b60 2023-11-09T03:53:05.765 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:53:05.765 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":107,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8568,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:52:53.889201+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8310,"mds_1":7144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7144":{"gid":7144,"name":"b","rank":1,"incarnation":101,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/7360844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":7360844},{"type":"v1","addr":"172.21.15.113:6837","nonce":7360844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8310":{"gid":8310,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":19,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:53:05.765 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 107 2023-11-09T03:53:05.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.765+0000 7f5846155640 1 -- 172.21.15.76:0/3734117423 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f582006ae70 msgr2=0x7f582006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:05.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.765+0000 7f5846155640 1 --2- 172.21.15.76:0/3734117423 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f582006ae70 0x7f582006d330 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5828036000 tx=0x7f582803b040 comp rx=0 tx=0).stop 2023-11-09T03:53:05.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.765+0000 7f5846155640 1 -- 172.21.15.76:0/3734117423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58400d3b60 msgr2=0x7f58401f3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:05.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.765+0000 7f5846155640 1 --2- 172.21.15.76:0/3734117423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58400d3b60 0x7f58401f3320 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f583000d7d0 tx=0x7f583000dc90 comp rx=0 tx=0).stop 2023-11-09T03:53:05.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.765+0000 7f5846155640 1 -- 172.21.15.76:0/3734117423 shutdown_connections 2023-11-09T03:53:05.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.765+0000 7f5846155640 1 --2- 172.21.15.76:0/3734117423 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f582006ae70 0x7f582006d330 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:05.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.765+0000 7f5846155640 1 --2- 172.21.15.76:0/3734117423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f584015d4e0 0x7f584006c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:05.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.765+0000 7f5846155640 1 --2- 172.21.15.76:0/3734117423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f584015d100 0x7f584006bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:05.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.765+0000 7f5846155640 1 --2- 172.21.15.76:0/3734117423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58400d3b60 0x7f58401f3320 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:05.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.765+0000 7f5846155640 1 -- 172.21.15.76:0/3734117423 >> 172.21.15.76:0/3734117423 conn(0x7f58400d2ef0 msgr2=0x7f58400d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:05.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.765+0000 7f5846155640 1 -- 172.21.15.76:0/3734117423 shutdown_connections 2023-11-09T03:53:05.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:05.765+0000 7f5846155640 1 -- 172.21.15.76:0/3734117423 wait complete. 2023-11-09T03:53:05.777 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:53:10.779 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:53:10.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.956+0000 7f4dea59e640 1 Processor -- start 2023-11-09T03:53:10.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.956+0000 7f4dea59e640 1 -- start start 2023-11-09T03:53:10.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.957+0000 7f4dea59e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4de4159e50 0x7f4de4070840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:10.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.957+0000 7f4dea59e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4de406b7c0 0x7f4de40cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:10.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.957+0000 7f4dea59e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4de415d7f0 0x7f4de415dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:10.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.957+0000 7f4dea59e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4de415e6e0 con 0x7f4de415d7f0 2023-11-09T03:53:10.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.957+0000 7f4dea59e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4de415e1c0 con 0x7f4de4159e50 2023-11-09T03:53:10.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.957+0000 7f4dea59e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4de406c680 con 0x7f4de406b7c0 2023-11-09T03:53:10.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.957+0000 7f4de9d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4de415d7f0 0x7f4de415dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:10.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.957+0000 7f4de9d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4de415d7f0 0x7f4de415dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59384/0 (socket says 172.21.15.76:59384) 2023-11-09T03:53:10.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.957+0000 7f4de9d9d640 1 -- 172.21.15.76:0/3908073003 learned_addr learned my addr 172.21.15.76:0/3908073003 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:10.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.957+0000 7f4de959c640 1 --2- 172.21.15.76:0/3908073003 >> v2:172.21.15.113:3300/0 conn(0x7f4de4159e50 0x7f4de4070840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:10.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.957+0000 7f4de8d9b640 1 --2- 172.21.15.76:0/3908073003 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4de406b7c0 0x7f4de40cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:10.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.957+0000 7f4de9d9d640 1 -- 172.21.15.76:0/3908073003 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4de406b7c0 msgr2=0x7f4de40cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:10.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.957+0000 7f4de9d9d640 1 --2- 172.21.15.76:0/3908073003 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4de406b7c0 0x7f4de40cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:10.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.957+0000 7f4de9d9d640 1 -- 172.21.15.76:0/3908073003 >> v2:172.21.15.113:3300/0 conn(0x7f4de4159e50 msgr2=0x7f4de4070840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:10.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.957+0000 7f4de9d9d640 1 --2- 172.21.15.76:0/3908073003 >> v2:172.21.15.113:3300/0 conn(0x7f4de4159e50 0x7f4de4070840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:10.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.957+0000 7f4de9d9d640 1 -- 172.21.15.76:0/3908073003 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de4071100 con 0x7f4de415d7f0 2023-11-09T03:53:10.963 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.958+0000 7f4de9d9d640 1 --2- 172.21.15.76:0/3908073003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4de415d7f0 0x7f4de415dc80 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f4de40cf7e0 tx=0x7f4ddc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c10dd2e7e1b47b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:10.963 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.958+0000 7f4ddbfff640 1 -- 172.21.15.76:0/3908073003 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc02ca70 con 0x7f4de415d7f0 2023-11-09T03:53:10.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.958+0000 7f4ddbfff640 1 -- 172.21.15.76:0/3908073003 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ddc02cbd0 con 0x7f4de415d7f0 2023-11-09T03:53:10.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.958+0000 7f4ddbfff640 1 -- 172.21.15.76:0/3908073003 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc0343f0 con 0x7f4de415d7f0 2023-11-09T03:53:10.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.958+0000 7f4dea59e640 1 -- 172.21.15.76:0/3908073003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4de415d7f0 msgr2=0x7f4de415dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:10.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.958+0000 7f4dea59e640 1 --2- 172.21.15.76:0/3908073003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4de415d7f0 0x7f4de415dc80 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f4de40cf7e0 tx=0x7f4ddc02bed0 comp rx=0 tx=0).stop 2023-11-09T03:53:10.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.958+0000 7f4dea59e640 1 -- 172.21.15.76:0/3908073003 shutdown_connections 2023-11-09T03:53:10.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.958+0000 7f4dea59e640 1 --2- 172.21.15.76:0/3908073003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4de415d7f0 0x7f4de415dc80 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:10.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.958+0000 7f4dea59e640 1 --2- 172.21.15.76:0/3908073003 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4de406b7c0 0x7f4de40cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:10.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.958+0000 7f4dea59e640 1 --2- 172.21.15.76:0/3908073003 >> v2:172.21.15.113:3300/0 conn(0x7f4de4159e50 0x7f4de4070840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:10.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.958+0000 7f4dea59e640 1 -- 172.21.15.76:0/3908073003 >> 172.21.15.76:0/3908073003 conn(0x7f4de406a730 msgr2=0x7f4de40ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:10.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.959+0000 7f4dea59e640 1 -- 172.21.15.76:0/3908073003 shutdown_connections 2023-11-09T03:53:10.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.959+0000 7f4dea59e640 1 -- 172.21.15.76:0/3908073003 wait complete. 2023-11-09T03:53:10.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.960+0000 7f4dea59e640 1 Processor -- start 2023-11-09T03:53:10.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.960+0000 7f4dea59e640 1 -- start start 2023-11-09T03:53:10.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.961+0000 7f4dea59e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4de406b7c0 0x7f4de415a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:10.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.961+0000 7f4dea59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4de4159e50 0x7f4de414cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:10.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.961+0000 7f4dea59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4de415d7f0 0x7f4de40c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:10.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.961+0000 7f4dea59e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de40c45c0 con 0x7f4de4159e50 2023-11-09T03:53:10.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.961+0000 7f4dea59e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de415bd10 con 0x7f4de415d7f0 2023-11-09T03:53:10.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.961+0000 7f4dea59e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4de414d120 con 0x7f4de406b7c0 2023-11-09T03:53:10.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.961+0000 7f4de8d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4de4159e50 0x7f4de414cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:10.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.961+0000 7f4de8d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4de4159e50 0x7f4de414cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59400/0 (socket says 172.21.15.76:59400) 2023-11-09T03:53:10.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.961+0000 7f4de8d9b640 1 -- 172.21.15.76:0/605305882 learned_addr learned my addr 172.21.15.76:0/605305882 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:10.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.961+0000 7f4de9d9d640 1 --2- 172.21.15.76:0/605305882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4de415d7f0 0x7f4de40c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:10.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.961+0000 7f4de959c640 1 --2- 172.21.15.76:0/605305882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4de406b7c0 0x7f4de415a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:10.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.961+0000 7f4de8d9b640 1 -- 172.21.15.76:0/605305882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4de406b7c0 msgr2=0x7f4de415a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:10.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.961+0000 7f4de8d9b640 1 --2- 172.21.15.76:0/605305882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4de406b7c0 0x7f4de415a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:10.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.961+0000 7f4de8d9b640 1 -- 172.21.15.76:0/605305882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4de415d7f0 msgr2=0x7f4de40c49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:10.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.961+0000 7f4de8d9b640 1 --2- 172.21.15.76:0/605305882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4de415d7f0 0x7f4de40c49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:10.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.961+0000 7f4de8d9b640 1 -- 172.21.15.76:0/605305882 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ddc009b30 con 0x7f4de4159e50 2023-11-09T03:53:10.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.961+0000 7f4de8d9b640 1 --2- 172.21.15.76:0/605305882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4de4159e50 0x7f4de414cbe0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400d800 tx=0x7f4dd400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:10.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.961+0000 7f4dd9ffb640 1 -- 172.21.15.76:0/605305882 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd40096a0 con 0x7f4de4159e50 2023-11-09T03:53:10.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.961+0000 7f4dd9ffb640 1 -- 172.21.15.76:0/605305882 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd4009800 con 0x7f4de4159e50 2023-11-09T03:53:10.970 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.961+0000 7f4dea59e640 1 -- 172.21.15.76:0/605305882 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de40c3940 con 0x7f4de4159e50 2023-11-09T03:53:10.970 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.961+0000 7f4dd9ffb640 1 -- 172.21.15.76:0/605305882 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4009990 con 0x7f4de4159e50 2023-11-09T03:53:10.970 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.962+0000 7f4dea59e640 1 -- 172.21.15.76:0/605305882 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de40c3f00 con 0x7f4de4159e50 2023-11-09T03:53:10.970 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.963+0000 7f4dea59e640 1 -- 172.21.15.76:0/605305882 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4de4067c70 con 0x7f4de4159e50 2023-11-09T03:53:10.970 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.963+0000 7f4dd9ffb640 1 -- 172.21.15.76:0/605305882 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4dd4009af0 con 0x7f4de4159e50 2023-11-09T03:53:10.971 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.963+0000 7f4dd9ffb640 1 --2- 172.21.15.76:0/605305882 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4dc806ae70 0x7f4dc806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:10.971 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.963+0000 7f4dd9ffb640 1 -- 172.21.15.76:0/605305882 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7952+0+0 (secure 0 0 0) 0x7f4dd408a3b0 con 0x7f4de4159e50 2023-11-09T03:53:10.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.963+0000 7f4de959c640 1 --2- 172.21.15.76:0/605305882 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4dc806ae70 0x7f4dc806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:10.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.964+0000 7f4de959c640 1 --2- 172.21.15.76:0/605305882 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4dc806ae70 0x7f4dc806d330 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f4de0006fd0 tx=0x7f4de0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:10.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:10.967+0000 7f4dd9ffb640 1 -- 172.21.15.76:0/605305882 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4dd4054ba0 con 0x7f4de4159e50 2023-11-09T03:53:11.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:11.168+0000 7f4dea59e640 1 -- 172.21.15.76:0/605305882 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4de40742c0 con 0x7f4de4159e50 2023-11-09T03:53:11.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:11.168+0000 7f4dd9ffb640 1 -- 172.21.15.76:0/605305882 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4924 (secure 0 0 0) 0x7f4dd40581c0 con 0x7f4de4159e50 2023-11-09T03:53:11.169 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:53:11.169 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":107,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8568,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:52:53.889201+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8310,"mds_1":7144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7144":{"gid":7144,"name":"b","rank":1,"incarnation":101,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/7360844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":7360844},{"type":"v1","addr":"172.21.15.113:6837","nonce":7360844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8310":{"gid":8310,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":19,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:53:11.170 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 107 2023-11-09T03:53:11.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:11.170+0000 7f4dea59e640 1 -- 172.21.15.76:0/605305882 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4dc806ae70 msgr2=0x7f4dc806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:11.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:11.170+0000 7f4dea59e640 1 --2- 172.21.15.76:0/605305882 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4dc806ae70 0x7f4dc806d330 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f4de0006fd0 tx=0x7f4de0008040 comp rx=0 tx=0).stop 2023-11-09T03:53:11.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:11.170+0000 7f4dea59e640 1 -- 172.21.15.76:0/605305882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4de4159e50 msgr2=0x7f4de414cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:11.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:11.170+0000 7f4dea59e640 1 --2- 172.21.15.76:0/605305882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4de4159e50 0x7f4de414cbe0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400d800 tx=0x7f4dd400dcc0 comp rx=0 tx=0).stop 2023-11-09T03:53:11.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:11.170+0000 7f4dea59e640 1 -- 172.21.15.76:0/605305882 shutdown_connections 2023-11-09T03:53:11.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:11.170+0000 7f4dea59e640 1 --2- 172.21.15.76:0/605305882 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4dc806ae70 0x7f4dc806d330 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:11.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:11.170+0000 7f4dea59e640 1 --2- 172.21.15.76:0/605305882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4de415d7f0 0x7f4de40c49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:11.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:11.170+0000 7f4dea59e640 1 --2- 172.21.15.76:0/605305882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4de4159e50 0x7f4de414cbe0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:11.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:11.170+0000 7f4dea59e640 1 --2- 172.21.15.76:0/605305882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4de406b7c0 0x7f4de415a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:11.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:11.170+0000 7f4dea59e640 1 -- 172.21.15.76:0/605305882 >> 172.21.15.76:0/605305882 conn(0x7f4de406a730 msgr2=0x7f4de40cccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:11.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:11.170+0000 7f4dea59e640 1 -- 172.21.15.76:0/605305882 shutdown_connections 2023-11-09T03:53:11.173 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:11.170+0000 7f4dea59e640 1 -- 172.21.15.76:0/605305882 wait complete. 2023-11-09T03:53:11.182 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:53:12.153 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~7s 2023-11-09T03:53:16.183 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:53:16.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.293+0000 7fa0f341a640 1 Processor -- start 2023-11-09T03:53:16.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.293+0000 7fa0f341a640 1 -- start start 2023-11-09T03:53:16.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.294+0000 7fa0f341a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa0ec159e40 0x7fa0ec0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:16.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.294+0000 7fa0f341a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0ec06bb60 0x7fa0ec0b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:16.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.294+0000 7fa0f341a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0ec071500 con 0x7fa0ec06fff0 2023-11-09T03:53:16.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.294+0000 7fa0f341a640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0ec15dae0 con 0x7fa0ec159e40 2023-11-09T03:53:16.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.294+0000 7fa0f341a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec070c60 con 0x7fa0ec06bb60 2023-11-09T03:53:16.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.294+0000 7fa0f11b6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa0ec159e40 0x7fa0ec0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:16.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.294+0000 7fa0f11b6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa0ec159e40 0x7fa0ec0b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42366/0 (socket says 172.21.15.76:42366) 2023-11-09T03:53:16.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.294+0000 7fa0f11b6640 1 -- 172.21.15.76:0/1508825396 learned_addr learned my addr 172.21.15.76:0/1508825396 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:16.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.294+0000 7fa0f09b5640 1 --2- 172.21.15.76:0/1508825396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0ec06bb60 0x7fa0ec0b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:16.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.294+0000 7fa0f11b6640 1 -- 172.21.15.76:0/1508825396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0ec06bb60 msgr2=0x7fa0ec0b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:16.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.294+0000 7fa0f11b6640 1 --2- 172.21.15.76:0/1508825396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0ec06bb60 0x7fa0ec0b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:16.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.294+0000 7fa0f11b6640 1 -- 172.21.15.76:0/1508825396 >> v1:172.21.15.113:6789/0 conn(0x7fa0ec06fff0 legacy=0x7fa0ec070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:16.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.294+0000 7fa0f11b6640 1 -- 172.21.15.76:0/1508825396 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ec070f40 con 0x7fa0ec159e40 2023-11-09T03:53:16.298 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.294+0000 7fa0f09b5640 1 --2- 172.21.15.76:0/1508825396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0ec06bb60 0x7fa0ec0b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:53:16.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.295+0000 7fa0f11b6640 1 --2- 172.21.15.76:0/1508825396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0ec159e40 0x7fa0ec0b0530 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800bb10 tx=0x7fa0e802e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aeeae992c5a96d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:16.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.295+0000 7fa0e3fff640 1 -- 172.21.15.76:0/1508825396 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e802ee70 con 0x7fa0ec159e40 2023-11-09T03:53:16.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.295+0000 7fa0e3fff640 1 -- 172.21.15.76:0/1508825396 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e8036440 con 0x7fa0ec159e40 2023-11-09T03:53:16.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.295+0000 7fa0e3fff640 1 -- 172.21.15.76:0/1508825396 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e80367d0 con 0x7fa0ec159e40 2023-11-09T03:53:16.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.296+0000 7fa0f341a640 1 -- 172.21.15.76:0/1508825396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0ec159e40 msgr2=0x7fa0ec0b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:16.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.296+0000 7fa0f341a640 1 --2- 172.21.15.76:0/1508825396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0ec159e40 0x7fa0ec0b0530 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800bb10 tx=0x7fa0e802e0c0 comp rx=0 tx=0).stop 2023-11-09T03:53:16.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.296+0000 7fa0f341a640 1 -- 172.21.15.76:0/1508825396 shutdown_connections 2023-11-09T03:53:16.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.296+0000 7fa0f341a640 1 --2- 172.21.15.76:0/1508825396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0ec06bb60 0x7fa0ec0b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:16.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.296+0000 7fa0f341a640 1 --2- 172.21.15.76:0/1508825396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0ec159e40 0x7fa0ec0b0530 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:16.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.296+0000 7fa0f341a640 1 -- 172.21.15.76:0/1508825396 >> 172.21.15.76:0/1508825396 conn(0x7fa0ec0cb530 msgr2=0x7fa0ec0cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:16.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.296+0000 7fa0f341a640 1 -- 172.21.15.76:0/1508825396 shutdown_connections 2023-11-09T03:53:16.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.296+0000 7fa0f341a640 1 -- 172.21.15.76:0/1508825396 wait complete. 2023-11-09T03:53:16.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.297+0000 7fa0f341a640 1 Processor -- start 2023-11-09T03:53:16.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.297+0000 7fa0f341a640 1 -- start start 2023-11-09T03:53:16.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.298+0000 7fa0f341a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0ec06bb60 0x7fa0ec15dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:16.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.298+0000 7fa0f341a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0ec06fff0 0x7fa0ec0c6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:16.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.298+0000 7fa0f341a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0ec159e40 0x7fa0ec0c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:16.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.298+0000 7fa0f341a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec0af550 con 0x7fa0ec06bb60 2023-11-09T03:53:16.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.298+0000 7fa0f341a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec0c7100 con 0x7fa0ec159e40 2023-11-09T03:53:16.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.298+0000 7fa0f341a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec0c73e0 con 0x7fa0ec06fff0 2023-11-09T03:53:16.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.298+0000 7fa0f11b6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0ec06bb60 0x7fa0ec15dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:16.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.298+0000 7fa0f11b6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0ec06bb60 0x7fa0ec15dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42382/0 (socket says 172.21.15.76:42382) 2023-11-09T03:53:16.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.298+0000 7fa0f11b6640 1 -- 172.21.15.76:0/3704459284 learned_addr learned my addr 172.21.15.76:0/3704459284 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:16.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.298+0000 7fa0f09b5640 1 --2- 172.21.15.76:0/3704459284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0ec06fff0 0x7fa0ec0c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:16.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.298+0000 7fa0f19b7640 1 --2- 172.21.15.76:0/3704459284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0ec159e40 0x7fa0ec0c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:16.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.298+0000 7fa0f11b6640 1 -- 172.21.15.76:0/3704459284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0ec06fff0 msgr2=0x7fa0ec0c6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:16.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.298+0000 7fa0f11b6640 1 --2- 172.21.15.76:0/3704459284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0ec06fff0 0x7fa0ec0c6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:16.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.298+0000 7fa0f11b6640 1 -- 172.21.15.76:0/3704459284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0ec159e40 msgr2=0x7fa0ec0c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:16.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.298+0000 7fa0f11b6640 1 --2- 172.21.15.76:0/3704459284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0ec159e40 0x7fa0ec0c6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:16.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.298+0000 7fa0f11b6640 1 -- 172.21.15.76:0/3704459284 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e800b770 con 0x7fa0ec06bb60 2023-11-09T03:53:16.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.298+0000 7fa0f11b6640 1 --2- 172.21.15.76:0/3704459284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0ec06bb60 0x7fa0ec15dfb0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800b2c0 tx=0x7fa0e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:16.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.298+0000 7fa0e1ffb640 1 -- 172.21.15.76:0/3704459284 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e80348d0 con 0x7fa0ec06bb60 2023-11-09T03:53:16.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.298+0000 7fa0e1ffb640 1 -- 172.21.15.76:0/3704459284 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e8034a50 con 0x7fa0ec06bb60 2023-11-09T03:53:16.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.298+0000 7fa0f341a640 1 -- 172.21.15.76:0/3704459284 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0ec0c76c0 con 0x7fa0ec06bb60 2023-11-09T03:53:16.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.298+0000 7fa0e1ffb640 1 -- 172.21.15.76:0/3704459284 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8034e00 con 0x7fa0ec06bb60 2023-11-09T03:53:16.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.299+0000 7fa0f341a640 1 -- 172.21.15.76:0/3704459284 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0ec0c7c80 con 0x7fa0ec06bb60 2023-11-09T03:53:16.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.300+0000 7fa0f341a640 1 -- 172.21.15.76:0/3704459284 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0ec0684b0 con 0x7fa0ec06bb60 2023-11-09T03:53:16.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.300+0000 7fa0e1ffb640 1 -- 172.21.15.76:0/3704459284 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa0e8041480 con 0x7fa0ec06bb60 2023-11-09T03:53:16.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.300+0000 7fa0e1ffb640 1 --2- 172.21.15.76:0/3704459284 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa0d006ae70 0x7fa0d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:16.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.301+0000 7fa0e1ffb640 1 -- 172.21.15.76:0/3704459284 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7952+0+0 (secure 0 0 0) 0x7fa0e80ace60 con 0x7fa0ec06bb60 2023-11-09T03:53:16.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.301+0000 7fa0f09b5640 1 --2- 172.21.15.76:0/3704459284 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa0d006ae70 0x7fa0d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:16.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.301+0000 7fa0f09b5640 1 --2- 172.21.15.76:0/3704459284 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa0d006ae70 0x7fa0d006d330 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec06d0a0 tx=0x7fa0dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:16.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.305+0000 7fa0e1ffb640 1 -- 172.21.15.76:0/3704459284 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa0e8077790 con 0x7fa0ec06bb60 2023-11-09T03:53:16.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.506+0000 7fa0f341a640 1 -- 172.21.15.76:0/3704459284 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0ec0c9750 con 0x7fa0ec06bb60 2023-11-09T03:53:16.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.507+0000 7fa0e1ffb640 1 -- 172.21.15.76:0/3704459284 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+4972 (secure 0 0 0) 0x7fa0e803b090 con 0x7fa0ec06bb60 2023-11-09T03:53:16.508 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:53:16.508 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":108,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8568,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:53:16.471485+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8310,"mds_1":7144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7144":{"gid":7144,"name":"b","rank":1,"incarnation":101,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/7360844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":7360844},{"type":"v1","addr":"172.21.15.113:6837","nonce":7360844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8310":{"gid":8310,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":19,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"laggy_since":"2023-11-09T03:53:16.471485+0000","export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:53:16.508 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 108 2023-11-09T03:53:16.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.509+0000 7fa0f341a640 1 -- 172.21.15.76:0/3704459284 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa0d006ae70 msgr2=0x7fa0d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:16.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.509+0000 7fa0f341a640 1 --2- 172.21.15.76:0/3704459284 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa0d006ae70 0x7fa0d006d330 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec06d0a0 tx=0x7fa0dc008040 comp rx=0 tx=0).stop 2023-11-09T03:53:16.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.509+0000 7fa0f341a640 1 -- 172.21.15.76:0/3704459284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0ec06bb60 msgr2=0x7fa0ec15dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:16.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.509+0000 7fa0f341a640 1 --2- 172.21.15.76:0/3704459284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0ec06bb60 0x7fa0ec15dfb0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800b2c0 tx=0x7fa0e8004730 comp rx=0 tx=0).stop 2023-11-09T03:53:16.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.509+0000 7fa0f341a640 1 -- 172.21.15.76:0/3704459284 shutdown_connections 2023-11-09T03:53:16.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.509+0000 7fa0f341a640 1 --2- 172.21.15.76:0/3704459284 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa0d006ae70 0x7fa0d006d330 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:16.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.509+0000 7fa0f341a640 1 --2- 172.21.15.76:0/3704459284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0ec159e40 0x7fa0ec0c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:16.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.509+0000 7fa0f341a640 1 --2- 172.21.15.76:0/3704459284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa0ec06fff0 0x7fa0ec0c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:16.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.509+0000 7fa0f341a640 1 --2- 172.21.15.76:0/3704459284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0ec06bb60 0x7fa0ec15dfb0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:16.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.509+0000 7fa0f341a640 1 -- 172.21.15.76:0/3704459284 >> 172.21.15.76:0/3704459284 conn(0x7fa0ec0cb530 msgr2=0x7fa0ec070460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:16.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.509+0000 7fa0f341a640 1 -- 172.21.15.76:0/3704459284 shutdown_connections 2023-11-09T03:53:16.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.509+0000 7fa0f341a640 1 -- 172.21.15.76:0/3704459284 wait complete. 2023-11-09T03:53:16.520 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2023-11-09T03:53:16.521 DEBUG:teuthology.orchestra.run.smithi076:> sudo sysctl -n kernel.core_pattern 2023-11-09T03:53:16.546 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-09T03:53:16.547 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2023-11-09T03:53:16.548 DEBUG:teuthology.orchestra.run.smithi076:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2023-11-09T03:53:16.616 INFO:teuthology.orchestra.run.smithi076.stdout:1699501979.122454.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i a', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2023-11-09T03:53:16.618 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1699501979.122454.core'] 2023-11-09T03:53:16.619 INFO:tasks.cephfs.cephfs_test_case:Found core file 1699501979.122454.core, deleting it 2023-11-09T03:53:16.619 DEBUG:teuthology.orchestra.run.smithi076:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1699501979.122454.core 2023-11-09T03:53:16.688 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:53:16.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.827+0000 7f4617ce0640 1 Processor -- start 2023-11-09T03:53:16.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.827+0000 7f4617ce0640 1 -- start start 2023-11-09T03:53:16.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.828+0000 7f4617ce0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f46100addd0 0x7f46100cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:16.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.828+0000 7f4617ce0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f46100cc680 0x7f46100ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:16.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.828+0000 7f4617ce0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f461006bb10 0x7f461006bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:16.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.828+0000 7f4617ce0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f461006c4a0 con 0x7f46100addd0 2023-11-09T03:53:16.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.828+0000 7f4617ce0640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46100cd720 con 0x7f46100cc680 2023-11-09T03:53:16.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.828+0000 7f4617ce0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46100ae1b0 con 0x7f461006bb10 2023-11-09T03:53:16.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.828+0000 7f4615a7c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f46100addd0 0x7f46100cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:16.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.828+0000 7f4615a7c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f46100addd0 0x7f46100cdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42392/0 (socket says 172.21.15.76:42392) 2023-11-09T03:53:16.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.828+0000 7f4615a7c640 1 -- 172.21.15.76:0/757858174 learned_addr learned my addr 172.21.15.76:0/757858174 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:16.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.828+0000 7f4615a7c640 1 -- 172.21.15.76:0/757858174 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f461006bb10 msgr2=0x7f461006bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:16.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.828+0000 7f4615a7c640 1 --2- 172.21.15.76:0/757858174 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f461006bb10 0x7f461006bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:16.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.828+0000 7f4615a7c640 1 -- 172.21.15.76:0/757858174 >> v2:172.21.15.113:3300/0 conn(0x7f46100cc680 msgr2=0x7f46100ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:16.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.828+0000 7f4615a7c640 1 --2- 172.21.15.76:0/757858174 >> v2:172.21.15.113:3300/0 conn(0x7f46100cc680 0x7f46100ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:16.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.828+0000 7f4615a7c640 1 -- 172.21.15.76:0/757858174 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46100ccb10 con 0x7f46100addd0 2023-11-09T03:53:16.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.829+0000 7f4615a7c640 1 --2- 172.21.15.76:0/757858174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46100addd0 0x7f46100cdbb0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f46100b1650 tx=0x7f460c02dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d27b1d23d8177a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:16.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.829+0000 7f4614a7a640 1 -- 172.21.15.76:0/757858174 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c02ec50 con 0x7f46100addd0 2023-11-09T03:53:16.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.829+0000 7f4614a7a640 1 -- 172.21.15.76:0/757858174 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460c032070 con 0x7f46100addd0 2023-11-09T03:53:16.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.829+0000 7f4614a7a640 1 -- 172.21.15.76:0/757858174 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c038400 con 0x7f46100addd0 2023-11-09T03:53:16.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.829+0000 7f4617ce0640 1 -- 172.21.15.76:0/757858174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46100addd0 msgr2=0x7f46100cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:16.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.830+0000 7f4617ce0640 1 --2- 172.21.15.76:0/757858174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46100addd0 0x7f46100cdbb0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f46100b1650 tx=0x7f460c02dea0 comp rx=0 tx=0).stop 2023-11-09T03:53:16.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.830+0000 7f4617ce0640 1 -- 172.21.15.76:0/757858174 shutdown_connections 2023-11-09T03:53:16.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.830+0000 7f4617ce0640 1 --2- 172.21.15.76:0/757858174 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f461006bb10 0x7f461006bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:16.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.830+0000 7f4617ce0640 1 --2- 172.21.15.76:0/757858174 >> v2:172.21.15.113:3300/0 conn(0x7f46100cc680 0x7f46100ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:16.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.830+0000 7f4617ce0640 1 --2- 172.21.15.76:0/757858174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46100addd0 0x7f46100cdbb0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:16.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.830+0000 7f4617ce0640 1 -- 172.21.15.76:0/757858174 >> 172.21.15.76:0/757858174 conn(0x7f46100cb530 msgr2=0x7f46100cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:16.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.830+0000 7f4617ce0640 1 -- 172.21.15.76:0/757858174 shutdown_connections 2023-11-09T03:53:16.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.830+0000 7f4617ce0640 1 -- 172.21.15.76:0/757858174 wait complete. 2023-11-09T03:53:16.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.831+0000 7f4617ce0640 1 Processor -- start 2023-11-09T03:53:16.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.831+0000 7f4617ce0640 1 -- start start 2023-11-09T03:53:16.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.831+0000 7f4617ce0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f461006bb10 0x7f4610146f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:16.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.831+0000 7f4617ce0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46100addd0 0x7f4610070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:16.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.831+0000 7f4617ce0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46100c25c0 0x7f4610070a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:16.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.831+0000 7f4617ce0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46100b3340 con 0x7f461006bb10 2023-11-09T03:53:16.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.831+0000 7f4617ce0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46101474a0 con 0x7f46100c25c0 2023-11-09T03:53:16.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.831+0000 7f4617ce0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46100b29e0 con 0x7f46100addd0 2023-11-09T03:53:16.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.832+0000 7f4615a7c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f461006bb10 0x7f4610146f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:16.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.832+0000 7f4615a7c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f461006bb10 0x7f4610146f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42396/0 (socket says 172.21.15.76:42396) 2023-11-09T03:53:16.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.832+0000 7f4615a7c640 1 -- 172.21.15.76:0/498643373 learned_addr learned my addr 172.21.15.76:0/498643373 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:16.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.832+0000 7f461527b640 1 --2- 172.21.15.76:0/498643373 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46100addd0 0x7f4610070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:16.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.832+0000 7f461627d640 1 --2- 172.21.15.76:0/498643373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46100c25c0 0x7f4610070a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:16.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.832+0000 7f4615a7c640 1 -- 172.21.15.76:0/498643373 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46100addd0 msgr2=0x7f4610070540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:16.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.832+0000 7f4615a7c640 1 --2- 172.21.15.76:0/498643373 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46100addd0 0x7f4610070540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:16.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.832+0000 7f4615a7c640 1 -- 172.21.15.76:0/498643373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46100c25c0 msgr2=0x7f4610070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:16.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.832+0000 7f4615a7c640 1 --2- 172.21.15.76:0/498643373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46100c25c0 0x7f4610070a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:16.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.832+0000 7f4615a7c640 1 -- 172.21.15.76:0/498643373 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460c00b760 con 0x7f461006bb10 2023-11-09T03:53:16.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.832+0000 7f4615a7c640 1 --2- 172.21.15.76:0/498643373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f461006bb10 0x7f4610146f60 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f460c000c00 tx=0x7f460c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:16.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.832+0000 7f46067fc640 1 -- 172.21.15.76:0/498643373 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c043470 con 0x7f461006bb10 2023-11-09T03:53:16.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.832+0000 7f46067fc640 1 -- 172.21.15.76:0/498643373 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460c0435f0 con 0x7f461006bb10 2023-11-09T03:53:16.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.832+0000 7f46067fc640 1 -- 172.21.15.76:0/498643373 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c0439a0 con 0x7f461006bb10 2023-11-09T03:53:16.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.832+0000 7f4617ce0640 1 -- 172.21.15.76:0/498643373 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46100c7a30 con 0x7f461006bb10 2023-11-09T03:53:16.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.832+0000 7f4617ce0640 1 -- 172.21.15.76:0/498643373 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46100b2f60 con 0x7f461006bb10 2023-11-09T03:53:16.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.833+0000 7f4617ce0640 1 -- 172.21.15.76:0/498643373 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46100b3120 con 0x7f461006bb10 2023-11-09T03:53:16.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.834+0000 7f46067fc640 1 -- 172.21.15.76:0/498643373 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f460c043b00 con 0x7f461006bb10 2023-11-09T03:53:16.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.834+0000 7f46067fc640 1 --2- 172.21.15.76:0/498643373 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f45f406ae70 0x7f45f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:16.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.834+0000 7f46067fc640 1 -- 172.21.15.76:0/498643373 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7952+0+0 (secure 0 0 0) 0x7f460c0aecc0 con 0x7f461006bb10 2023-11-09T03:53:16.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.834+0000 7f461527b640 1 --2- 172.21.15.76:0/498643373 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f45f406ae70 0x7f45f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:16.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.835+0000 7f461527b640 1 --2- 172.21.15.76:0/498643373 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f45f406ae70 0x7f45f406d330 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f46100adab0 tx=0x7f4600008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:16.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:16.839+0000 7f46067fc640 1 -- 172.21.15.76:0/498643373 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f460c079560 con 0x7f461006bb10 2023-11-09T03:53:17.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.038+0000 7f4617ce0640 1 -- 172.21.15.76:0/498643373 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46100684b0 con 0x7f461006bb10 2023-11-09T03:53:17.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.039+0000 7f46067fc640 1 -- 172.21.15.76:0/498643373 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+4972 (secure 0 0 0) 0x7f460c047020 con 0x7f461006bb10 2023-11-09T03:53:17.040 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:53:17.040 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":108,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8568,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:53:16.471485+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8310,"mds_1":7144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7144":{"gid":7144,"name":"b","rank":1,"incarnation":101,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/7360844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":7360844},{"type":"v1","addr":"172.21.15.113:6837","nonce":7360844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8310":{"gid":8310,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":19,"addr":"172.21.15.76:6835/546338722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":546338722},{"type":"v1","addr":"172.21.15.76:6835","nonce":546338722}]},"join_fscid":-1,"laggy_since":"2023-11-09T03:53:16.471485+0000","export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:53:17.040 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 108 2023-11-09T03:53:17.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.041+0000 7f4617ce0640 1 -- 172.21.15.76:0/498643373 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f45f406ae70 msgr2=0x7f45f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:17.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.041+0000 7f4617ce0640 1 --2- 172.21.15.76:0/498643373 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f45f406ae70 0x7f45f406d330 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f46100adab0 tx=0x7f4600008040 comp rx=0 tx=0).stop 2023-11-09T03:53:17.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.041+0000 7f4617ce0640 1 -- 172.21.15.76:0/498643373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f461006bb10 msgr2=0x7f4610146f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:17.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.041+0000 7f4617ce0640 1 --2- 172.21.15.76:0/498643373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f461006bb10 0x7f4610146f60 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f460c000c00 tx=0x7f460c0046c0 comp rx=0 tx=0).stop 2023-11-09T03:53:17.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.041+0000 7f4617ce0640 1 -- 172.21.15.76:0/498643373 shutdown_connections 2023-11-09T03:53:17.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.041+0000 7f4617ce0640 1 --2- 172.21.15.76:0/498643373 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f45f406ae70 0x7f45f406d330 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:17.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.041+0000 7f4617ce0640 1 --2- 172.21.15.76:0/498643373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46100c25c0 0x7f4610070a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:17.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.041+0000 7f4617ce0640 1 --2- 172.21.15.76:0/498643373 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46100addd0 0x7f4610070540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:17.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.041+0000 7f4617ce0640 1 --2- 172.21.15.76:0/498643373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f461006bb10 0x7f4610146f60 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:17.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.041+0000 7f4617ce0640 1 -- 172.21.15.76:0/498643373 >> 172.21.15.76:0/498643373 conn(0x7f46100cb530 msgr2=0x7f46100afcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:17.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.041+0000 7f4617ce0640 1 -- 172.21.15.76:0/498643373 shutdown_connections 2023-11-09T03:53:17.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.041+0000 7f4617ce0640 1 -- 172.21.15.76:0/498643373 wait complete. 2023-11-09T03:53:17.052 INFO:tasks.ceph.mds.b:Sent signal 9 2023-11-09T03:53:17.053 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:53:17.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:53:17.053 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 down 2023-11-09T03:53:17.081 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:0 2023-11-09T03:53:17.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.223+0000 7f589beaf640 1 Processor -- start 2023-11-09T03:53:17.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.223+0000 7f589beaf640 1 -- start start 2023-11-09T03:53:17.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.224+0000 7f589beaf640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f58940b0c40 0x7f589406fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:17.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.224+0000 7f589beaf640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f589406bb60 0x7f5894070530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:17.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.224+0000 7f589beaf640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5894071500 con 0x7f5894159e40 2023-11-09T03:53:17.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.224+0000 7f589beaf640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5894070a70 con 0x7f58940b0c40 2023-11-09T03:53:17.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.224+0000 7f589beaf640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f589415dae0 con 0x7f589406bb60 2023-11-09T03:53:17.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.224+0000 7f589944a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f58940b0c40 0x7f589406fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:17.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.224+0000 7f589944a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f58940b0c40 0x7f589406fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42406/0 (socket says 172.21.15.76:42406) 2023-11-09T03:53:17.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.224+0000 7f589944a640 1 -- 172.21.15.76:0/1938077891 learned_addr learned my addr 172.21.15.76:0/1938077891 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:17.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.224+0000 7f589a44c640 1 --2- 172.21.15.76:0/1938077891 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f589406bb60 0x7f5894070530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:17.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.224+0000 7f589944a640 1 -- 172.21.15.76:0/1938077891 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f589406bb60 msgr2=0x7f5894070530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:17.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.224+0000 7f589944a640 1 --2- 172.21.15.76:0/1938077891 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f589406bb60 0x7f5894070530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:17.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.224+0000 7f589944a640 1 -- 172.21.15.76:0/1938077891 >> v1:172.21.15.113:6789/0 conn(0x7f5894159e40 legacy=0x7f58940b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:17.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.224+0000 7f589944a640 1 -- 172.21.15.76:0/1938077891 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5894070c60 con 0x7f58940b0c40 2023-11-09T03:53:17.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.224+0000 7f589a44c640 1 --2- 172.21.15.76:0/1938077891 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f589406bb60 0x7f5894070530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:53:17.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.225+0000 7f589944a640 1 --2- 172.21.15.76:0/1938077891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58940b0c40 0x7f589406fff0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f5884009870 tx=0x7f588402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a022433ad2510fdb server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:17.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.225+0000 7f5898c49640 1 -- 172.21.15.76:0/1938077891 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5884031640 con 0x7f58940b0c40 2023-11-09T03:53:17.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.225+0000 7f5898c49640 1 -- 172.21.15.76:0/1938077891 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58840317a0 con 0x7f58940b0c40 2023-11-09T03:53:17.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.225+0000 7f5898c49640 1 -- 172.21.15.76:0/1938077891 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5884031b30 con 0x7f58940b0c40 2023-11-09T03:53:17.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.225+0000 7f589beaf640 1 -- 172.21.15.76:0/1938077891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58940b0c40 msgr2=0x7f589406fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:17.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.225+0000 7f589beaf640 1 --2- 172.21.15.76:0/1938077891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58940b0c40 0x7f589406fff0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f5884009870 tx=0x7f588402b8f0 comp rx=0 tx=0).stop 2023-11-09T03:53:17.232 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.225+0000 7f589beaf640 1 -- 172.21.15.76:0/1938077891 shutdown_connections 2023-11-09T03:53:17.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.225+0000 7f589beaf640 1 --2- 172.21.15.76:0/1938077891 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f589406bb60 0x7f5894070530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:17.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.225+0000 7f589beaf640 1 --2- 172.21.15.76:0/1938077891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58940b0c40 0x7f589406fff0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:17.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.225+0000 7f589beaf640 1 -- 172.21.15.76:0/1938077891 >> 172.21.15.76:0/1938077891 conn(0x7f58940cb530 msgr2=0x7f58940cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:17.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.226+0000 7f589beaf640 1 -- 172.21.15.76:0/1938077891 shutdown_connections 2023-11-09T03:53:17.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.226+0000 7f589beaf640 1 -- 172.21.15.76:0/1938077891 wait complete. 2023-11-09T03:53:17.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.227+0000 7f589beaf640 1 Processor -- start 2023-11-09T03:53:17.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.227+0000 7f589beaf640 1 -- start start 2023-11-09T03:53:17.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.227+0000 7f589beaf640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f589406bb60 0x7f5894153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:17.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.227+0000 7f589beaf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58940b0c40 0x7f58940c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:17.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.227+0000 7f589beaf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5894159e40 0x7f58940c2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:17.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.227+0000 7f589beaf640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58941f0460 con 0x7f5894159e40 2023-11-09T03:53:17.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.227+0000 7f589beaf640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5894153660 con 0x7f58940b0c40 2023-11-09T03:53:17.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.227+0000 7f589beaf640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58941ef7e0 con 0x7f589406bb60 2023-11-09T03:53:17.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.228+0000 7f589a44c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5894159e40 0x7f58940c2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:17.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.228+0000 7f589a44c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5894159e40 0x7f58940c2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42420/0 (socket says 172.21.15.76:42420) 2023-11-09T03:53:17.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.228+0000 7f589a44c640 1 -- 172.21.15.76:0/2102146473 learned_addr learned my addr 172.21.15.76:0/2102146473 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:17.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.228+0000 7f589944a640 1 --2- 172.21.15.76:0/2102146473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58940b0c40 0x7f58940c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:17.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.228+0000 7f5899c4b640 1 --2- 172.21.15.76:0/2102146473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f589406bb60 0x7f5894153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:17.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.228+0000 7f589a44c640 1 -- 172.21.15.76:0/2102146473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f589406bb60 msgr2=0x7f5894153120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:17.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.228+0000 7f589a44c640 1 --2- 172.21.15.76:0/2102146473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f589406bb60 0x7f5894153120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:17.236 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.228+0000 7f589a44c640 1 -- 172.21.15.76:0/2102146473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58940b0c40 msgr2=0x7f58940c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:17.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.228+0000 7f589a44c640 1 --2- 172.21.15.76:0/2102146473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58940b0c40 0x7f58940c4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:17.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.228+0000 7f589a44c640 1 -- 172.21.15.76:0/2102146473 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58840094d0 con 0x7f5894159e40 2023-11-09T03:53:17.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.228+0000 7f589a44c640 1 --2- 172.21.15.76:0/2102146473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5894159e40 0x7f58940c2ed0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f588c00cef0 tx=0x7f588c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:17.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.228+0000 7f588a7fc640 1 -- 172.21.15.76:0/2102146473 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588c00d840 con 0x7f5894159e40 2023-11-09T03:53:17.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.228+0000 7f588a7fc640 1 -- 172.21.15.76:0/2102146473 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f588c00d9c0 con 0x7f5894159e40 2023-11-09T03:53:17.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.228+0000 7f589beaf640 1 -- 172.21.15.76:0/2102146473 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58941efac0 con 0x7f5894159e40 2023-11-09T03:53:17.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.228+0000 7f588a7fc640 1 -- 172.21.15.76:0/2102146473 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588c00dd70 con 0x7f5894159e40 2023-11-09T03:53:17.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.228+0000 7f589beaf640 1 -- 172.21.15.76:0/2102146473 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58941f0080 con 0x7f5894159e40 2023-11-09T03:53:17.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.229+0000 7f589beaf640 1 -- 172.21.15.76:0/2102146473 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58941f05a0 con 0x7f5894159e40 2023-11-09T03:53:17.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.230+0000 7f588a7fc640 1 -- 172.21.15.76:0/2102146473 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f588c01e3b0 con 0x7f5894159e40 2023-11-09T03:53:17.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.230+0000 7f588a7fc640 1 --2- 172.21.15.76:0/2102146473 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f587806ae70 0x7f587806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:17.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.230+0000 7f588a7fc640 1 -- 172.21.15.76:0/2102146473 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 7952+0+0 (secure 0 0 0) 0x7f588c08a8c0 con 0x7f5894159e40 2023-11-09T03:53:17.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.230+0000 7f5899c4b640 1 --2- 172.21.15.76:0/2102146473 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f587806ae70 0x7f587806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:17.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.231+0000 7f5899c4b640 1 --2- 172.21.15.76:0/2102146473 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f587806ae70 0x7f587806d330 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f58941596e0 tx=0x7f5890009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:17.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.234+0000 7f588a7fc640 1 -- 172.21.15.76:0/2102146473 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f58941f05a0 con 0x7f5894159e40 2023-11-09T03:53:17.253 INFO:tasks.ceph.mds.b.smithi113.stderr:daemon-helper: command crashed with signal 9 2023-11-09T03:53:17.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.468+0000 7f589beaf640 1 -- 172.21.15.76:0/2102146473 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:0"} v 0) v1 -- 0x7f58941f05a0 con 0x7f5894159e40 2023-11-09T03:53:17.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.470+0000 7f588a7fc640 1 -- 172.21.15.76:0/2102146473 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:0"}]=0 failed mds gid 8310 v109) v1 ==== 97+0+0 (secure 0 0 0) 0x7f58941f05a0 con 0x7f5894159e40 2023-11-09T03:53:17.471 INFO:teuthology.orchestra.run.smithi076.stderr:failed mds gid 8310 2023-11-09T03:53:17.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.472+0000 7f589beaf640 1 -- 172.21.15.76:0/2102146473 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f587806ae70 msgr2=0x7f587806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:17.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.472+0000 7f589beaf640 1 --2- 172.21.15.76:0/2102146473 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f587806ae70 0x7f587806d330 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f58941596e0 tx=0x7f5890009040 comp rx=0 tx=0).stop 2023-11-09T03:53:17.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.472+0000 7f589beaf640 1 -- 172.21.15.76:0/2102146473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5894159e40 msgr2=0x7f58940c2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:17.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.472+0000 7f589beaf640 1 --2- 172.21.15.76:0/2102146473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5894159e40 0x7f58940c2ed0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f588c00cef0 tx=0x7f588c00a710 comp rx=0 tx=0).stop 2023-11-09T03:53:17.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.472+0000 7f589beaf640 1 -- 172.21.15.76:0/2102146473 shutdown_connections 2023-11-09T03:53:17.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.472+0000 7f589beaf640 1 --2- 172.21.15.76:0/2102146473 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f587806ae70 0x7f587806d330 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:17.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.472+0000 7f589beaf640 1 --2- 172.21.15.76:0/2102146473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5894159e40 0x7f58940c2ed0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:17.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.472+0000 7f589beaf640 1 --2- 172.21.15.76:0/2102146473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58940b0c40 0x7f58940c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:17.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.472+0000 7f589beaf640 1 --2- 172.21.15.76:0/2102146473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f589406bb60 0x7f5894153120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:17.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.472+0000 7f589beaf640 1 -- 172.21.15.76:0/2102146473 >> 172.21.15.76:0/2102146473 conn(0x7f58940cb530 msgr2=0x7f58940cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:17.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.472+0000 7f589beaf640 1 -- 172.21.15.76:0/2102146473 shutdown_connections 2023-11-09T03:53:17.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.473+0000 7f589beaf640 1 -- 172.21.15.76:0/2102146473 wait complete. 2023-11-09T03:53:17.485 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-09T03:53:17.485 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-09T03:53:17.486 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-09T03:53:17.486 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:53:17.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:53:17.487 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi076 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2023-11-09T03:53:17.487 INFO:tasks.ceph.mds.a:Stopped 2023-11-09T03:53:17.488 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-09T03:53:17.490 INFO:tasks.ceph.mds.a:Started 2023-11-09T03:53:17.490 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:53:17.567 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2023-11-09T03:53:17.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.594+0000 7f3a6cee0640 1 Processor -- start 2023-11-09T03:53:17.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.594+0000 7f3a6cee0640 1 -- start start 2023-11-09T03:53:17.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.595+0000 7f3a6cee0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a6815d100 0x7f3a680ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:17.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.595+0000 7f3a6cee0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3a680d1be0 0x7f3a680b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:17.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.595+0000 7f3a6cee0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a680b32b0 con 0x7f3a6815d4e0 2023-11-09T03:53:17.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.595+0000 7f3a65d9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f3a6815d4e0 0x7f3a680d14d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59160/0 (socket says 172.21.15.76:59160) 2023-11-09T03:53:17.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.595+0000 7f3a65d9b640 1 -- 172.21.15.76:0/664014160 learned_addr learned my addr 172.21.15.76:0/664014160 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:53:17.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.595+0000 7f3a6659c640 1 --2- 172.21.15.76:0/664014160 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a6815d100 0x7f3a680ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:17.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.595+0000 7f3a6cee0640 1 -- 172.21.15.76:0/664014160 --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a680cee10 con 0x7f3a680d1be0 2023-11-09T03:53:17.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.595+0000 7f3a6cee0640 1 -- 172.21.15.76:0/664014160 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a680b2980 con 0x7f3a6815d100 2023-11-09T03:53:17.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.595+0000 7f3a66d9d640 1 --2- 172.21.15.76:0/664014160 >> v2:172.21.15.76:3300/0 conn(0x7f3a680d1be0 0x7f3a680b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:17.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.596+0000 7f3a6559a640 1 -- 172.21.15.76:0/664014160 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1577634432 0 0) 0x7f3a680b32b0 con 0x7f3a6815d4e0 2023-11-09T03:53:17.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.596+0000 7f3a6559a640 1 -- 172.21.15.76:0/664014160 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a50003520 con 0x7f3a6815d4e0 2023-11-09T03:53:17.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.596+0000 7f3a66d9d640 1 -- 172.21.15.76:0/664014160 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a6815d100 msgr2=0x7f3a680ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:17.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.596+0000 7f3a66d9d640 1 --2- 172.21.15.76:0/664014160 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a6815d100 0x7f3a680ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:17.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.596+0000 7f3a66d9d640 1 -- 172.21.15.76:0/664014160 >> v1:172.21.15.76:6789/0 conn(0x7f3a6815d4e0 legacy=0x7f3a680d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:17.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.596+0000 7f3a66d9d640 1 -- 172.21.15.76:0/664014160 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a680b2c60 con 0x7f3a680d1be0 2023-11-09T03:53:17.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.596+0000 7f3a66d9d640 1 --2- 172.21.15.76:0/664014160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a680d1be0 0x7f3a680b3740 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009dc0 tx=0x7f3a6002be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ffdbcd917166e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:17.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.597+0000 7f3a6559a640 1 -- 172.21.15.76:0/664014160 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6002cc40 con 0x7f3a680d1be0 2023-11-09T03:53:17.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.597+0000 7f3a6559a640 1 -- 172.21.15.76:0/664014160 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6002cda0 con 0x7f3a680d1be0 2023-11-09T03:53:17.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.597+0000 7f3a6559a640 1 -- 172.21.15.76:0/664014160 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a60034660 con 0x7f3a680d1be0 2023-11-09T03:53:17.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.597+0000 7f3a6cee0640 1 -- 172.21.15.76:0/664014160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a680d1be0 msgr2=0x7f3a680b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:17.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.597+0000 7f3a6cee0640 1 --2- 172.21.15.76:0/664014160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a680d1be0 0x7f3a680b3740 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009dc0 tx=0x7f3a6002be90 comp rx=0 tx=0).stop 2023-11-09T03:53:17.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.597+0000 7f3a6cee0640 1 -- 172.21.15.76:0/664014160 shutdown_connections 2023-11-09T03:53:17.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.597+0000 7f3a6cee0640 1 --2- 172.21.15.76:0/664014160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a680d1be0 0x7f3a680b3740 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:17.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.597+0000 7f3a6cee0640 1 --2- 172.21.15.76:0/664014160 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a6815d100 0x7f3a680ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:17.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.597+0000 7f3a6cee0640 1 -- 172.21.15.76:0/664014160 >> 172.21.15.76:0/664014160 conn(0x7f3a680d2ef0 msgr2=0x7f3a680d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:17.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.597+0000 7f3a6cee0640 1 -- 172.21.15.76:0/664014160 shutdown_connections 2023-11-09T03:53:17.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.597+0000 7f3a6cee0640 1 -- 172.21.15.76:0/664014160 wait complete. 2023-11-09T03:53:17.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.599+0000 7f3a6cee0640 1 Processor -- start 2023-11-09T03:53:17.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.599+0000 7f3a6cee0640 1 -- start start 2023-11-09T03:53:17.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.600+0000 7f3a6cee0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a680d1be0 0x7f3a681f2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:17.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.600+0000 7f3a6cee0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a6815d100 0x7f3a6815b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:17.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.600+0000 7f3a6cee0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a6815d4e0 0x7f3a680b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:17.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.600+0000 7f3a6cee0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a681f6fd0 con 0x7f3a680d1be0 2023-11-09T03:53:17.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.600+0000 7f3a6cee0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a681f6100 con 0x7f3a6815d4e0 2023-11-09T03:53:17.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.600+0000 7f3a6cee0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a681f6350 con 0x7f3a6815d100 2023-11-09T03:53:17.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.600+0000 7f3a6659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a680d1be0 0x7f3a681f2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:17.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.600+0000 7f3a6659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a680d1be0 0x7f3a681f2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42460/0 (socket says 172.21.15.76:42460) 2023-11-09T03:53:17.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.600+0000 7f3a6659c640 1 -- 172.21.15.76:0/3191150300 learned_addr learned my addr 172.21.15.76:0/3191150300 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:17.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.600+0000 7f3a66d9d640 1 --2- 172.21.15.76:0/3191150300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a6815d4e0 0x7f3a680b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:17.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.600+0000 7f3a65d9b640 1 --2- 172.21.15.76:0/3191150300 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a6815d100 0x7f3a6815b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:17.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.600+0000 7f3a6659c640 1 -- 172.21.15.76:0/3191150300 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a6815d100 msgr2=0x7f3a6815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:17.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.600+0000 7f3a6659c640 1 --2- 172.21.15.76:0/3191150300 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a6815d100 0x7f3a6815b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:17.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.600+0000 7f3a6659c640 1 -- 172.21.15.76:0/3191150300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a6815d4e0 msgr2=0x7f3a680b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:17.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.600+0000 7f3a6659c640 1 --2- 172.21.15.76:0/3191150300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a6815d4e0 0x7f3a680b1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:17.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.600+0000 7f3a6659c640 1 -- 172.21.15.76:0/3191150300 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a60009a70 con 0x7f3a680d1be0 2023-11-09T03:53:17.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.600+0000 7f3a66d9d640 1 --2- 172.21.15.76:0/3191150300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a6815d4e0 0x7f3a680b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:53:17.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.600+0000 7f3a6659c640 1 --2- 172.21.15.76:0/3191150300 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a680d1be0 0x7f3a681f2ad0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800e900 tx=0x7f3a5800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:17.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.600+0000 7f3a56ffd640 1 -- 172.21.15.76:0/3191150300 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58009660 con 0x7f3a680d1be0 2023-11-09T03:53:17.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.600+0000 7f3a56ffd640 1 -- 172.21.15.76:0/3191150300 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a5800ca00 con 0x7f3a680d1be0 2023-11-09T03:53:17.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.600+0000 7f3a56ffd640 1 -- 172.21.15.76:0/3191150300 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5800cd70 con 0x7f3a680d1be0 2023-11-09T03:53:17.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.600+0000 7f3a6cee0640 1 -- 172.21.15.76:0/3191150300 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a681f6630 con 0x7f3a680d1be0 2023-11-09T03:53:17.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.600+0000 7f3a6cee0640 1 -- 172.21.15.76:0/3191150300 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a681f6bf0 con 0x7f3a680d1be0 2023-11-09T03:53:17.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.602+0000 7f3a6cee0640 1 -- 172.21.15.76:0/3191150300 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a680b6fb0 con 0x7f3a680d1be0 2023-11-09T03:53:17.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.602+0000 7f3a56ffd640 1 -- 172.21.15.76:0/3191150300 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3a58013470 con 0x7f3a680d1be0 2023-11-09T03:53:17.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.602+0000 7f3a56ffd640 1 --2- 172.21.15.76:0/3191150300 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3a4806ae70 0x7f3a4806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:17.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.602+0000 7f3a56ffd640 1 -- 172.21.15.76:0/3191150300 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8038+0+0 (secure 0 0 0) 0x7f3a5808a520 con 0x7f3a680d1be0 2023-11-09T03:53:17.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.603+0000 7f3a65d9b640 1 --2- 172.21.15.76:0/3191150300 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3a4806ae70 0x7f3a4806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:17.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.603+0000 7f3a65d9b640 1 --2- 172.21.15.76:0/3191150300 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3a4806ae70 0x7f3a4806d330 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f3a680b0570 tx=0x7f3a5c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:17.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.607+0000 7f3a56ffd640 1 -- 172.21.15.76:0/3191150300 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3a58054cb0 con 0x7f3a680d1be0 2023-11-09T03:53:17.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.807+0000 7f3a6cee0640 1 -- 172.21.15.76:0/3191150300 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a680684b0 con 0x7f3a680d1be0 2023-11-09T03:53:17.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.807+0000 7f3a56ffd640 1 -- 172.21.15.76:0/3191150300 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4135 (secure 0 0 0) 0x7f3a58021090 con 0x7f3a680d1be0 2023-11-09T03:53:17.808 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:53:17.808 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":110,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:53:17.474703+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":7144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7144":{"gid":7144,"name":"b","rank":1,"incarnation":101,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/7360844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":7360844},{"type":"v1","addr":"172.21.15.113:6837","nonce":7360844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:replay","state_seq":1,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:53:17.808 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 110 2023-11-09T03:53:17.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.809+0000 7f3a6cee0640 1 -- 172.21.15.76:0/3191150300 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3a4806ae70 msgr2=0x7f3a4806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:17.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.809+0000 7f3a6cee0640 1 --2- 172.21.15.76:0/3191150300 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3a4806ae70 0x7f3a4806d330 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f3a680b0570 tx=0x7f3a5c009040 comp rx=0 tx=0).stop 2023-11-09T03:53:17.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.809+0000 7f3a6cee0640 1 -- 172.21.15.76:0/3191150300 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a680d1be0 msgr2=0x7f3a681f2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:17.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.809+0000 7f3a6cee0640 1 --2- 172.21.15.76:0/3191150300 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a680d1be0 0x7f3a681f2ad0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800e900 tx=0x7f3a5800edc0 comp rx=0 tx=0).stop 2023-11-09T03:53:17.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.809+0000 7f3a6cee0640 1 -- 172.21.15.76:0/3191150300 shutdown_connections 2023-11-09T03:53:17.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.809+0000 7f3a6cee0640 1 --2- 172.21.15.76:0/3191150300 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3a4806ae70 0x7f3a4806d330 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:17.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.809+0000 7f3a6cee0640 1 --2- 172.21.15.76:0/3191150300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a6815d4e0 0x7f3a680b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:17.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.809+0000 7f3a6cee0640 1 --2- 172.21.15.76:0/3191150300 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a6815d100 0x7f3a6815b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:17.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.809+0000 7f3a6cee0640 1 --2- 172.21.15.76:0/3191150300 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a680d1be0 0x7f3a681f2ad0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:17.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.809+0000 7f3a6cee0640 1 -- 172.21.15.76:0/3191150300 >> 172.21.15.76:0/3191150300 conn(0x7f3a680d2ef0 msgr2=0x7f3a680cd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:17.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.809+0000 7f3a6cee0640 1 -- 172.21.15.76:0/3191150300 shutdown_connections 2023-11-09T03:53:17.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:17.809+0000 7f3a6cee0640 1 -- 172.21.15.76:0/3191150300 wait complete. 2023-11-09T03:53:17.821 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-11-09T03:53:18.764 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-11-09T03:53:18.764 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-11-09T03:53:22.821 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:53:22.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.938+0000 7fc593da9640 1 Processor -- start 2023-11-09T03:53:22.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.938+0000 7fc593da9640 1 -- start start 2023-11-09T03:53:22.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.939+0000 7fc593da9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc58c0ade00 0x7fc58c0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:22.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.939+0000 7fc593da9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc58c06d9c0 0x7fc58c0710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:22.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.939+0000 7fc593da9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc58c06cc90 con 0x7fc58c06c5f0 2023-11-09T03:53:22.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.939+0000 7fc593da9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc58c0b0e60 con 0x7fc58c06d9c0 2023-11-09T03:53:22.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.939+0000 7fc593da9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc58c06cee0 con 0x7fc58c0ade00 2023-11-09T03:53:22.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.939+0000 7fc591344640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc58c06d9c0 0x7fc58c0710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:22.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.939+0000 7fc592346640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc58c06c5f0 0x7fc58c15e9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57012/0 (socket says 172.21.15.76:57012) 2023-11-09T03:53:22.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.939+0000 7fc592346640 1 -- 172.21.15.76:0/1432550330 learned_addr learned my addr 172.21.15.76:0/1432550330 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:53:22.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.939+0000 7fc591b45640 1 --2- 172.21.15.76:0/1432550330 >> v2:172.21.15.113:3300/0 conn(0x7fc58c0ade00 0x7fc58c0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:22.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.940+0000 7fc591344640 1 -- 172.21.15.76:0/1432550330 >> v2:172.21.15.113:3300/0 conn(0x7fc58c0ade00 msgr2=0x7fc58c0b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:22.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.940+0000 7fc591344640 1 --2- 172.21.15.76:0/1432550330 >> v2:172.21.15.113:3300/0 conn(0x7fc58c0ade00 0x7fc58c0b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:22.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.940+0000 7fc591344640 1 -- 172.21.15.76:0/1432550330 >> v1:172.21.15.76:6789/0 conn(0x7fc58c06c5f0 legacy=0x7fc58c15e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:22.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.940+0000 7fc591344640 1 -- 172.21.15.76:0/1432550330 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc58c06d1c0 con 0x7fc58c06d9c0 2023-11-09T03:53:22.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.940+0000 7fc590b43640 1 -- 172.21.15.76:0/1432550330 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3969247117 0 0) 0x7fc58c06cc90 con 0x7fc58c06c5f0 2023-11-09T03:53:22.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.940+0000 7fc591344640 1 --2- 172.21.15.76:0/1432550330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc58c06d9c0 0x7fc58c0710d0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fc58c0cb430 tx=0x7fc57c02b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d2f97886a77a8a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:22.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.941+0000 7fc590b43640 1 -- 172.21.15.76:0/1432550330 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c031710 con 0x7fc58c06d9c0 2023-11-09T03:53:22.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.941+0000 7fc590b43640 1 -- 172.21.15.76:0/1432550330 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc57c02f050 con 0x7fc58c06d9c0 2023-11-09T03:53:22.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.941+0000 7fc590b43640 1 -- 172.21.15.76:0/1432550330 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c031a90 con 0x7fc58c06d9c0 2023-11-09T03:53:22.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.941+0000 7fc593da9640 1 -- 172.21.15.76:0/1432550330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc58c06d9c0 msgr2=0x7fc58c0710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:22.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.941+0000 7fc593da9640 1 --2- 172.21.15.76:0/1432550330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc58c06d9c0 0x7fc58c0710d0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fc58c0cb430 tx=0x7fc57c02b7b0 comp rx=0 tx=0).stop 2023-11-09T03:53:22.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.941+0000 7fc593da9640 1 -- 172.21.15.76:0/1432550330 shutdown_connections 2023-11-09T03:53:22.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.941+0000 7fc593da9640 1 --2- 172.21.15.76:0/1432550330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc58c06d9c0 0x7fc58c0710d0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:22.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.941+0000 7fc593da9640 1 --2- 172.21.15.76:0/1432550330 >> v2:172.21.15.113:3300/0 conn(0x7fc58c0ade00 0x7fc58c0b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:22.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.941+0000 7fc593da9640 1 -- 172.21.15.76:0/1432550330 >> 172.21.15.76:0/1432550330 conn(0x7fc58c0cb530 msgr2=0x7fc58c0cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:22.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.941+0000 7fc593da9640 1 -- 172.21.15.76:0/1432550330 shutdown_connections 2023-11-09T03:53:22.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.941+0000 7fc593da9640 1 -- 172.21.15.76:0/1432550330 wait complete. 2023-11-09T03:53:22.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.942+0000 7fc593da9640 1 Processor -- start 2023-11-09T03:53:22.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.943+0000 7fc593da9640 1 -- start start 2023-11-09T03:53:22.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.943+0000 7fc593da9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc58c06c5f0 0x7fc58c155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:22.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.943+0000 7fc593da9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc58c06d9c0 0x7fc58c156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:22.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.943+0000 7fc593da9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc58c0ade00 0x7fc58c14a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:22.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.943+0000 7fc593da9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c0c1dd0 con 0x7fc58c06c5f0 2023-11-09T03:53:22.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.943+0000 7fc593da9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c154d50 con 0x7fc58c0ade00 2023-11-09T03:53:22.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.943+0000 7fc593da9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c155030 con 0x7fc58c06d9c0 2023-11-09T03:53:22.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.943+0000 7fc591b45640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc58c06c5f0 0x7fc58c155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:22.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.943+0000 7fc591b45640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc58c06c5f0 0x7fc58c155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39162/0 (socket says 172.21.15.76:39162) 2023-11-09T03:53:22.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.943+0000 7fc591b45640 1 -- 172.21.15.76:0/844895689 learned_addr learned my addr 172.21.15.76:0/844895689 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:22.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.944+0000 7fc591344640 1 --2- 172.21.15.76:0/844895689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc58c06d9c0 0x7fc58c156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:22.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.944+0000 7fc592346640 1 --2- 172.21.15.76:0/844895689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc58c0ade00 0x7fc58c14a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:22.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.944+0000 7fc591b45640 1 -- 172.21.15.76:0/844895689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc58c06d9c0 msgr2=0x7fc58c156330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:22.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.944+0000 7fc591b45640 1 --2- 172.21.15.76:0/844895689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc58c06d9c0 0x7fc58c156330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:22.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.944+0000 7fc591b45640 1 -- 172.21.15.76:0/844895689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc58c0ade00 msgr2=0x7fc58c14a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:22.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.944+0000 7fc591b45640 1 --2- 172.21.15.76:0/844895689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc58c0ade00 0x7fc58c14a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:22.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.944+0000 7fc591b45640 1 -- 172.21.15.76:0/844895689 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc57c009400 con 0x7fc58c06c5f0 2023-11-09T03:53:22.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.944+0000 7fc591344640 1 --2- 172.21.15.76:0/844895689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc58c06d9c0 0x7fc58c156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:53:22.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.944+0000 7fc592346640 1 --2- 172.21.15.76:0/844895689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc58c0ade00 0x7fc58c14a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:53:22.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.944+0000 7fc591b45640 1 --2- 172.21.15.76:0/844895689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc58c06c5f0 0x7fc58c155df0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fc58800d5f0 tx=0x7fc58800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:22.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.944+0000 7fc5827fc640 1 -- 172.21.15.76:0/844895689 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5880125f0 con 0x7fc58c06c5f0 2023-11-09T03:53:22.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.944+0000 7fc5827fc640 1 -- 172.21.15.76:0/844895689 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc588012750 con 0x7fc58c06c5f0 2023-11-09T03:53:22.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.944+0000 7fc593da9640 1 -- 172.21.15.76:0/844895689 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc58c155310 con 0x7fc58c06c5f0 2023-11-09T03:53:22.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.944+0000 7fc5827fc640 1 -- 172.21.15.76:0/844895689 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc588012b00 con 0x7fc58c06c5f0 2023-11-09T03:53:22.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.944+0000 7fc593da9640 1 -- 172.21.15.76:0/844895689 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc58c1558d0 con 0x7fc58c06c5f0 2023-11-09T03:53:22.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.945+0000 7fc593da9640 1 -- 172.21.15.76:0/844895689 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc58c155a60 con 0x7fc58c06c5f0 2023-11-09T03:53:22.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.946+0000 7fc5827fc640 1 -- 172.21.15.76:0/844895689 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc588012c60 con 0x7fc58c06c5f0 2023-11-09T03:53:22.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.946+0000 7fc5827fc640 1 --2- 172.21.15.76:0/844895689 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc57006ae70 0x7fc57006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:22.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.946+0000 7fc5827fc640 1 -- 172.21.15.76:0/844895689 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8038+0+0 (secure 0 0 0) 0x7fc58808a370 con 0x7fc58c06c5f0 2023-11-09T03:53:22.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.946+0000 7fc591344640 1 --2- 172.21.15.76:0/844895689 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc57006ae70 0x7fc57006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:22.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.947+0000 7fc591344640 1 --2- 172.21.15.76:0/844895689 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc57006ae70 0x7fc57006d330 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc57c035000 tx=0x7fc57c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:22.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:22.951+0000 7fc5827fc640 1 -- 172.21.15.76:0/844895689 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc588054b90 con 0x7fc58c06c5f0 2023-11-09T03:53:23.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.154+0000 7fc593da9640 1 -- 172.21.15.76:0/844895689 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc58c0684b0 con 0x7fc58c06c5f0 2023-11-09T03:53:23.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.155+0000 7fc5827fc640 1 -- 172.21.15.76:0/844895689 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4929 (secure 0 0 0) 0x7fc5880581b0 con 0x7fc58c06c5f0 2023-11-09T03:53:23.155 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:53:23.155 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":111,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2075292158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2075292158},{"type":"v1","addr":"172.21.15.76:6835","nonce":2075292158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:53:17.497627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":7144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7144":{"gid":7144,"name":"b","rank":1,"incarnation":101,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/7360844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":7360844},{"type":"v1","addr":"172.21.15.113:6837","nonce":7360844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:resolve","state_seq":12,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:53:23.155 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 111 2023-11-09T03:53:23.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.157+0000 7fc593da9640 1 -- 172.21.15.76:0/844895689 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc57006ae70 msgr2=0x7fc57006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:23.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.157+0000 7fc593da9640 1 --2- 172.21.15.76:0/844895689 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc57006ae70 0x7fc57006d330 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc57c035000 tx=0x7fc57c008030 comp rx=0 tx=0).stop 2023-11-09T03:53:23.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.157+0000 7fc593da9640 1 -- 172.21.15.76:0/844895689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc58c06c5f0 msgr2=0x7fc58c155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:23.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.157+0000 7fc593da9640 1 --2- 172.21.15.76:0/844895689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc58c06c5f0 0x7fc58c155df0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fc58800d5f0 tx=0x7fc58800dab0 comp rx=0 tx=0).stop 2023-11-09T03:53:23.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.157+0000 7fc593da9640 1 -- 172.21.15.76:0/844895689 shutdown_connections 2023-11-09T03:53:23.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.157+0000 7fc593da9640 1 --2- 172.21.15.76:0/844895689 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc57006ae70 0x7fc57006d330 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:23.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.157+0000 7fc593da9640 1 --2- 172.21.15.76:0/844895689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc58c0ade00 0x7fc58c14a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:23.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.157+0000 7fc593da9640 1 --2- 172.21.15.76:0/844895689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc58c06d9c0 0x7fc58c156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:23.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.157+0000 7fc593da9640 1 --2- 172.21.15.76:0/844895689 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc58c06c5f0 0x7fc58c155df0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:23.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.157+0000 7fc593da9640 1 -- 172.21.15.76:0/844895689 >> 172.21.15.76:0/844895689 conn(0x7fc58c0cb530 msgr2=0x7fc58c0b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:23.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.157+0000 7fc593da9640 1 -- 172.21.15.76:0/844895689 shutdown_connections 2023-11-09T03:53:23.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.157+0000 7fc593da9640 1 -- 172.21.15.76:0/844895689 wait complete. 2023-11-09T03:53:23.167 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-09T03:53:23.168 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:53:23.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.268+0000 7f4f87d61640 1 Processor -- start 2023-11-09T03:53:23.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.268+0000 7f4f87d61640 1 -- start start 2023-11-09T03:53:23.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.269+0000 7f4f87d61640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4f800b0c40 0x7f4f8006fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:23.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.269+0000 7f4f87d61640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f80071500 con 0x7f4f80159e40 2023-11-09T03:53:23.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.269+0000 7f4f87d61640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f8015dae0 con 0x7f4f8006bb60 2023-11-09T03:53:23.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.269+0000 7f4f87d61640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f80070c60 con 0x7f4f800b0c40 2023-11-09T03:53:23.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.269+0000 7f4f85afd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f4f80159e40 0x7f4f800b0530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57020/0 (socket says 172.21.15.76:57020) 2023-11-09T03:53:23.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.269+0000 7f4f85afd640 1 -- 172.21.15.76:0/316242800 learned_addr learned my addr 172.21.15.76:0/316242800 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:53:23.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.269+0000 7f4f84afb640 1 -- 172.21.15.76:0/316242800 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 665427688 0 0) 0x7f4f80071500 con 0x7f4f80159e40 2023-11-09T03:53:23.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.269+0000 7f4f84afb640 1 -- 172.21.15.76:0/316242800 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f6c003520 con 0x7f4f80159e40 2023-11-09T03:53:23.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.269+0000 7f4f852fc640 1 --2- 172.21.15.76:0/316242800 >> v2:172.21.15.113:3300/0 conn(0x7f4f800b0c40 0x7f4f8006fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:23.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.269+0000 7f4f84afb640 1 -- 172.21.15.76:0/316242800 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4026686927 0 0) 0x7f4f6c003520 con 0x7f4f80159e40 2023-11-09T03:53:23.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.269+0000 7f4f84afb640 1 -- 172.21.15.76:0/316242800 >> v2:172.21.15.113:3300/0 conn(0x7f4f800b0c40 msgr2=0x7f4f8006fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:23.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.269+0000 7f4f84afb640 1 --2- 172.21.15.76:0/316242800 >> v2:172.21.15.113:3300/0 conn(0x7f4f800b0c40 0x7f4f8006fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:23.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.269+0000 7f4f84afb640 1 -- 172.21.15.76:0/316242800 >> v1:172.21.15.113:6789/0 conn(0x7f4f8006bb60 legacy=0x7f4f800cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:23.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.269+0000 7f4f84afb640 1 -- 172.21.15.76:0/316242800 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f80070f40 con 0x7f4f80159e40 2023-11-09T03:53:23.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.269+0000 7f4f84afb640 1 -- 172.21.15.76:0/316242800 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f4f7c001670 con 0x7f4f80159e40 2023-11-09T03:53:23.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.269+0000 7f4f84afb640 1 -- 172.21.15.76:0/316242800 >> v1:172.21.15.76:6789/0 conn(0x7f4f80159e40 legacy=0x7f4f800b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:23.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.269+0000 7f4f84afb640 1 --2- 172.21.15.76:0/316242800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f6c003970 0x7f4f6c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:23.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.269+0000 7f4f84afb640 1 --2- 172.21.15.76:0/316242800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f6c0047c0 0x7f4f6c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:23.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.269+0000 7f4f84afb640 1 --2- 172.21.15.76:0/316242800 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f6c005180 0x7f4f6c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:23.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.269+0000 7f4f84afb640 1 -- 172.21.15.76:0/316242800 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f80070c60 con 0x7f4f6c0047c0 2023-11-09T03:53:23.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.269+0000 7f4f84afb640 1 -- 172.21.15.76:0/316242800 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c00c120 con 0x7f4f6c003970 2023-11-09T03:53:23.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.269+0000 7f4f84afb640 1 -- 172.21.15.76:0/316242800 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c00c2e0 con 0x7f4f6c005180 2023-11-09T03:53:23.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.269+0000 7f4f852fc640 1 --2- 172.21.15.76:0/316242800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f6c0047c0 0x7f4f6c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:23.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.270+0000 7f4f85afd640 1 --2- 172.21.15.76:0/316242800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f6c003970 0x7f4f6c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:23.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.270+0000 7f4f862fe640 1 --2- 172.21.15.76:0/316242800 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f6c005180 0x7f4f6c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:23.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.270+0000 7f4f852fc640 1 -- 172.21.15.76:0/316242800 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f6c005180 msgr2=0x7f4f6c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:23.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.270+0000 7f4f852fc640 1 --2- 172.21.15.76:0/316242800 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f6c005180 0x7f4f6c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:23.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.270+0000 7f4f852fc640 1 -- 172.21.15.76:0/316242800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f6c003970 msgr2=0x7f4f6c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:23.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.270+0000 7f4f852fc640 1 --2- 172.21.15.76:0/316242800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f6c003970 0x7f4f6c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:23.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.270+0000 7f4f852fc640 1 -- 172.21.15.76:0/316242800 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6c003520 con 0x7f4f6c0047c0 2023-11-09T03:53:23.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.270+0000 7f4f862fe640 1 -- 172.21.15.76:0/316242800 reap_dead start 2023-11-09T03:53:23.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.270+0000 7f4f85afd640 1 --2- 172.21.15.76:0/316242800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f6c003970 0x7f4f6c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:53:23.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.270+0000 7f4f852fc640 1 --2- 172.21.15.76:0/316242800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f6c0047c0 0x7f4f6c004c40 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f4f70009650 tx=0x7f4f6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1cfa3da26e1b530 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:23.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.270+0000 7f4f84afb640 1 -- 172.21.15.76:0/316242800 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7002e8c0 con 0x7f4f6c0047c0 2023-11-09T03:53:23.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.271+0000 7f4f84afb640 1 -- 172.21.15.76:0/316242800 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f7002ea20 con 0x7f4f6c0047c0 2023-11-09T03:53:23.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.271+0000 7f4f84afb640 1 -- 172.21.15.76:0/316242800 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7002edb0 con 0x7f4f6c0047c0 2023-11-09T03:53:23.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.271+0000 7f4f87d61640 1 -- 172.21.15.76:0/316242800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f6c0047c0 msgr2=0x7f4f6c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:23.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.271+0000 7f4f87d61640 1 --2- 172.21.15.76:0/316242800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f6c0047c0 0x7f4f6c004c40 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f4f70009650 tx=0x7f4f6c0057f0 comp rx=0 tx=0).stop 2023-11-09T03:53:23.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.271+0000 7f4f87d61640 1 -- 172.21.15.76:0/316242800 shutdown_connections 2023-11-09T03:53:23.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.271+0000 7f4f87d61640 1 --2- 172.21.15.76:0/316242800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f6c0047c0 0x7f4f6c004c40 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:23.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.271+0000 7f4f87d61640 1 -- 172.21.15.76:0/316242800 >> 172.21.15.76:0/316242800 conn(0x7f4f800cb530 msgr2=0x7f4f800cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:23.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.271+0000 7f4f87d61640 1 -- 172.21.15.76:0/316242800 shutdown_connections 2023-11-09T03:53:23.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.271+0000 7f4f87d61640 1 -- 172.21.15.76:0/316242800 wait complete. 2023-11-09T03:53:23.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.272+0000 7f4f87d61640 1 Processor -- start 2023-11-09T03:53:23.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.272+0000 7f4f87d61640 1 -- start start 2023-11-09T03:53:23.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.273+0000 7f4f87d61640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f6c0047c0 0x7f4f800c4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:23.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.273+0000 7f4f87d61640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f80156ad0 0x7f4f800c8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:23.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.273+0000 7f4f87d61640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f800c8c50 0x7f4f800cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:23.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.273+0000 7f4f87d61640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f800c3ad0 con 0x7f4f800c8c50 2023-11-09T03:53:23.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.273+0000 7f4f87d61640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f800c3db0 con 0x7f4f6c0047c0 2023-11-09T03:53:23.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.273+0000 7f4f87d61640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f800c4090 con 0x7f4f80156ad0 2023-11-09T03:53:23.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.273+0000 7f4f862fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f800c8c50 0x7f4f800cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:23.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.273+0000 7f4f862fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f800c8c50 0x7f4f800cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39186/0 (socket says 172.21.15.76:39186) 2023-11-09T03:53:23.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.273+0000 7f4f862fe640 1 -- 172.21.15.76:0/3262121055 learned_addr learned my addr 172.21.15.76:0/3262121055 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:23.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.273+0000 7f4f85afd640 1 --2- 172.21.15.76:0/3262121055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f6c0047c0 0x7f4f800c4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:23.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.273+0000 7f4f852fc640 1 --2- 172.21.15.76:0/3262121055 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f80156ad0 0x7f4f800c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:23.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.273+0000 7f4f862fe640 1 -- 172.21.15.76:0/3262121055 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f80156ad0 msgr2=0x7f4f800c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:23.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.273+0000 7f4f862fe640 1 --2- 172.21.15.76:0/3262121055 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f80156ad0 0x7f4f800c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:23.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.273+0000 7f4f862fe640 1 -- 172.21.15.76:0/3262121055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f6c0047c0 msgr2=0x7f4f800c4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:23.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.273+0000 7f4f862fe640 1 --2- 172.21.15.76:0/3262121055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f6c0047c0 0x7f4f800c4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:23.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.273+0000 7f4f862fe640 1 -- 172.21.15.76:0/3262121055 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f800c4370 con 0x7f4f800c8c50 2023-11-09T03:53:23.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.274+0000 7f4f862fe640 1 --2- 172.21.15.76:0/3262121055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f800c8c50 0x7f4f800cc4b0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f4f78007fd0 tx=0x7f4f7800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:23.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.274+0000 7f4f767fc640 1 -- 172.21.15.76:0/3262121055 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f78003970 con 0x7f4f800c8c50 2023-11-09T03:53:23.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.274+0000 7f4f767fc640 1 -- 172.21.15.76:0/3262121055 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f78003af0 con 0x7f4f800c8c50 2023-11-09T03:53:23.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.274+0000 7f4f87d61640 1 -- 172.21.15.76:0/3262121055 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f800c4650 con 0x7f4f800c8c50 2023-11-09T03:53:23.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.274+0000 7f4f767fc640 1 -- 172.21.15.76:0/3262121055 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f78003ea0 con 0x7f4f800c8c50 2023-11-09T03:53:23.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.274+0000 7f4f87d61640 1 -- 172.21.15.76:0/3262121055 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f800c5390 con 0x7f4f800c8c50 2023-11-09T03:53:23.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.275+0000 7f4f87d61640 1 -- 172.21.15.76:0/3262121055 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f800684b0 con 0x7f4f800c8c50 2023-11-09T03:53:23.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.275+0000 7f4f767fc640 1 -- 172.21.15.76:0/3262121055 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4f780050d0 con 0x7f4f800c8c50 2023-11-09T03:53:23.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.275+0000 7f4f767fc640 1 --2- 172.21.15.76:0/3262121055 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4f6406aea0 0x7f4f6406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:23.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.276+0000 7f4f767fc640 1 -- 172.21.15.76:0/3262121055 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8038+0+0 (secure 0 0 0) 0x7f4f78089cd0 con 0x7f4f800c8c50 2023-11-09T03:53:23.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.276+0000 7f4f85afd640 1 --2- 172.21.15.76:0/3262121055 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4f6406aea0 0x7f4f6406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:23.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.276+0000 7f4f85afd640 1 --2- 172.21.15.76:0/3262121055 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4f6406aea0 0x7f4f6406d360 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c001eb0 tx=0x7f4f7c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:23.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.280+0000 7f4f767fc640 1 -- 172.21.15.76:0/3262121055 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4f78054460 con 0x7f4f800c8c50 2023-11-09T03:53:23.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.480+0000 7f4f87d61640 1 -- 172.21.15.76:0/3262121055 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f8006c6d0 con 0x7f4f800c8c50 2023-11-09T03:53:23.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.481+0000 7f4f767fc640 1 -- 172.21.15.76:0/3262121055 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4929 (secure 0 0 0) 0x7f4f78057a80 con 0x7f4f800c8c50 2023-11-09T03:53:23.480 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:53:23.481 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":111,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2075292158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2075292158},{"type":"v1","addr":"172.21.15.76:6835","nonce":2075292158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:53:17.497627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":7144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7144":{"gid":7144,"name":"b","rank":1,"incarnation":101,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/7360844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":7360844},{"type":"v1","addr":"172.21.15.113:6837","nonce":7360844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:resolve","state_seq":12,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:53:23.481 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 111 2023-11-09T03:53:23.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.482+0000 7f4f87d61640 1 -- 172.21.15.76:0/3262121055 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4f6406aea0 msgr2=0x7f4f6406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:23.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.482+0000 7f4f87d61640 1 --2- 172.21.15.76:0/3262121055 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4f6406aea0 0x7f4f6406d360 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c001eb0 tx=0x7f4f7c009040 comp rx=0 tx=0).stop 2023-11-09T03:53:23.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.482+0000 7f4f87d61640 1 -- 172.21.15.76:0/3262121055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f800c8c50 msgr2=0x7f4f800cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:23.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.482+0000 7f4f87d61640 1 --2- 172.21.15.76:0/3262121055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f800c8c50 0x7f4f800cc4b0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f4f78007fd0 tx=0x7f4f7800ec10 comp rx=0 tx=0).stop 2023-11-09T03:53:23.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.483+0000 7f4f87d61640 1 -- 172.21.15.76:0/3262121055 shutdown_connections 2023-11-09T03:53:23.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.483+0000 7f4f87d61640 1 --2- 172.21.15.76:0/3262121055 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4f6406aea0 0x7f4f6406d360 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:23.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.483+0000 7f4f87d61640 1 --2- 172.21.15.76:0/3262121055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f800c8c50 0x7f4f800cc4b0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:23.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.483+0000 7f4f87d61640 1 --2- 172.21.15.76:0/3262121055 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f80156ad0 0x7f4f800c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:23.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.483+0000 7f4f87d61640 1 --2- 172.21.15.76:0/3262121055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f6c0047c0 0x7f4f800c4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:23.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.483+0000 7f4f87d61640 1 -- 172.21.15.76:0/3262121055 >> 172.21.15.76:0/3262121055 conn(0x7f4f800cb530 msgr2=0x7f4f800af0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:23.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.483+0000 7f4f87d61640 1 -- 172.21.15.76:0/3262121055 shutdown_connections 2023-11-09T03:53:23.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.483+0000 7f4f87d61640 1 -- 172.21.15.76:0/3262121055 wait complete. 2023-11-09T03:53:23.493 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:resolve 2023-11-09T03:53:23.493 DEBUG:tasks.cephfs.filesystem:reached state 'up:resolve' in 0.325685977935791s 2023-11-09T03:53:23.494 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:53:23.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.594+0000 7fd2c9916640 1 Processor -- start 2023-11-09T03:53:23.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.594+0000 7fd2c9916640 1 -- start start 2023-11-09T03:53:23.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.595+0000 7fd2c9916640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd2c415d100 0x7fd2c40ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:23.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.595+0000 7fd2c9916640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c40b32b0 con 0x7fd2c415d4e0 2023-11-09T03:53:23.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.595+0000 7fd2c9916640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c40cee10 con 0x7fd2c40d1be0 2023-11-09T03:53:23.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.595+0000 7fd2c9916640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c40b2980 con 0x7fd2c415d100 2023-11-09T03:53:23.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.595+0000 7fd2c2ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd2c415d100 0x7fd2c40ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:23.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.595+0000 7fd2c27fc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd2c415d4e0 0x7fd2c40d14d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57026/0 (socket says 172.21.15.76:57026) 2023-11-09T03:53:23.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.595+0000 7fd2c27fc640 1 -- 172.21.15.76:0/3711396949 learned_addr learned my addr 172.21.15.76:0/3711396949 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:53:23.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.595+0000 7fd2c2ffd640 1 -- 172.21.15.76:0/3711396949 >> v1:172.21.15.113:6789/0 conn(0x7fd2c40d1be0 legacy=0x7fd2c40b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:23.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.595+0000 7fd2c2ffd640 1 -- 172.21.15.76:0/3711396949 >> v1:172.21.15.76:6789/0 conn(0x7fd2c415d4e0 legacy=0x7fd2c40d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:23.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.595+0000 7fd2c2ffd640 1 -- 172.21.15.76:0/3711396949 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c40b2c60 con 0x7fd2c415d100 2023-11-09T03:53:23.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.596+0000 7fd2c2ffd640 1 --2- 172.21.15.76:0/3711396949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2c415d100 0x7fd2c40ce510 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4009790 tx=0x7fd2b402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc2d2848c0cb2c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:23.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.596+0000 7fd2c1ffb640 1 -- 172.21.15.76:0/3711396949 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b4031830 con 0x7fd2c415d100 2023-11-09T03:53:23.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.596+0000 7fd2c1ffb640 1 -- 172.21.15.76:0/3711396949 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b4031990 con 0x7fd2c415d100 2023-11-09T03:53:23.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.596+0000 7fd2c1ffb640 1 -- 172.21.15.76:0/3711396949 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b4031d20 con 0x7fd2c415d100 2023-11-09T03:53:23.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.597+0000 7fd2c9916640 1 -- 172.21.15.76:0/3711396949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2c415d100 msgr2=0x7fd2c40ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:23.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.597+0000 7fd2c9916640 1 --2- 172.21.15.76:0/3711396949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2c415d100 0x7fd2c40ce510 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4009790 tx=0x7fd2b402b8d0 comp rx=0 tx=0).stop 2023-11-09T03:53:23.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.597+0000 7fd2c9916640 1 -- 172.21.15.76:0/3711396949 shutdown_connections 2023-11-09T03:53:23.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.597+0000 7fd2c9916640 1 --2- 172.21.15.76:0/3711396949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2c415d100 0x7fd2c40ce510 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:23.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.597+0000 7fd2c9916640 1 -- 172.21.15.76:0/3711396949 >> 172.21.15.76:0/3711396949 conn(0x7fd2c40d2ef0 msgr2=0x7fd2c40d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:23.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.597+0000 7fd2c9916640 1 -- 172.21.15.76:0/3711396949 shutdown_connections 2023-11-09T03:53:23.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.597+0000 7fd2c9916640 1 -- 172.21.15.76:0/3711396949 wait complete. 2023-11-09T03:53:23.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.598+0000 7fd2c9916640 1 Processor -- start 2023-11-09T03:53:23.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.598+0000 7fd2c9916640 1 -- start start 2023-11-09T03:53:23.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.599+0000 7fd2c9916640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2c40d1be0 0x7fd2c4153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:23.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.599+0000 7fd2c9916640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2c415d100 0x7fd2c40ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:23.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.599+0000 7fd2c9916640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2c415d4e0 0x7fd2c409e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:23.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.599+0000 7fd2c9916640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c40cbe70 con 0x7fd2c40d1be0 2023-11-09T03:53:23.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.599+0000 7fd2c9916640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c415c780 con 0x7fd2c415d100 2023-11-09T03:53:23.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.599+0000 7fd2c9916640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c40cb1f0 con 0x7fd2c415d4e0 2023-11-09T03:53:23.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.599+0000 7fd2c2ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2c40d1be0 0x7fd2c4153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:23.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.599+0000 7fd2c2ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2c40d1be0 0x7fd2c4153260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39192/0 (socket says 172.21.15.76:39192) 2023-11-09T03:53:23.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.599+0000 7fd2c2ffd640 1 -- 172.21.15.76:0/281641967 learned_addr learned my addr 172.21.15.76:0/281641967 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:23.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.599+0000 7fd2c2ffd640 1 -- 172.21.15.76:0/281641967 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2c415d4e0 msgr2=0x7fd2c409e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:23.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.599+0000 7fd2c2ffd640 1 --2- 172.21.15.76:0/281641967 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2c415d4e0 0x7fd2c409e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:23.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.599+0000 7fd2c2ffd640 1 -- 172.21.15.76:0/281641967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2c415d100 msgr2=0x7fd2c40ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:23.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.599+0000 7fd2c2ffd640 1 --2- 172.21.15.76:0/281641967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2c415d100 0x7fd2c40ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:23.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.599+0000 7fd2c2ffd640 1 -- 172.21.15.76:0/281641967 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b40093a0 con 0x7fd2c40d1be0 2023-11-09T03:53:23.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.599+0000 7fd2c2ffd640 1 --2- 172.21.15.76:0/281641967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2c40d1be0 0x7fd2c4153260 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fd2b402bee0 tx=0x7fd2b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:23.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.599+0000 7fd2abfff640 1 -- 172.21.15.76:0/281641967 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b403ba70 con 0x7fd2c40d1be0 2023-11-09T03:53:23.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.599+0000 7fd2abfff640 1 -- 172.21.15.76:0/281641967 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b403bbd0 con 0x7fd2c40d1be0 2023-11-09T03:53:23.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.599+0000 7fd2abfff640 1 -- 172.21.15.76:0/281641967 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b40453f0 con 0x7fd2c40d1be0 2023-11-09T03:53:23.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.599+0000 7fd2c9916640 1 -- 172.21.15.76:0/281641967 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c40cb4d0 con 0x7fd2c40d1be0 2023-11-09T03:53:23.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.599+0000 7fd2c9916640 1 -- 172.21.15.76:0/281641967 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c40cba90 con 0x7fd2c40d1be0 2023-11-09T03:53:23.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.600+0000 7fd2c9916640 1 -- 172.21.15.76:0/281641967 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c40684b0 con 0x7fd2c40d1be0 2023-11-09T03:53:23.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.601+0000 7fd2abfff640 1 -- 172.21.15.76:0/281641967 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd2b4045550 con 0x7fd2c40d1be0 2023-11-09T03:53:23.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.601+0000 7fd2abfff640 1 --2- 172.21.15.76:0/281641967 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd2b006ae70 0x7fd2b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:23.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.601+0000 7fd2abfff640 1 -- 172.21.15.76:0/281641967 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8038+0+0 (secure 0 0 0) 0x7fd2b40ac330 con 0x7fd2c40d1be0 2023-11-09T03:53:23.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.601+0000 7fd2c27fc640 1 --2- 172.21.15.76:0/281641967 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd2b006ae70 0x7fd2b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:23.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.602+0000 7fd2c27fc640 1 --2- 172.21.15.76:0/281641967 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd2b006ae70 0x7fd2b006d330 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40d3c60 tx=0x7fd2b8007a40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:23.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.605+0000 7fd2abfff640 1 -- 172.21.15.76:0/281641967 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd2b4076be0 con 0x7fd2c40d1be0 2023-11-09T03:53:23.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.805+0000 7fd2c9916640 1 -- 172.21.15.76:0/281641967 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2c40b4d00 con 0x7fd2c40d1be0 2023-11-09T03:53:23.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.805+0000 7fd2abfff640 1 -- 172.21.15.76:0/281641967 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4929 (secure 0 0 0) 0x7fd2b407a200 con 0x7fd2c40d1be0 2023-11-09T03:53:23.805 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:53:23.805 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":111,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2075292158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2075292158},{"type":"v1","addr":"172.21.15.76:6835","nonce":2075292158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:53:17.497627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":7144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7144":{"gid":7144,"name":"b","rank":1,"incarnation":101,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/7360844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":7360844},{"type":"v1","addr":"172.21.15.113:6837","nonce":7360844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:resolve","state_seq":12,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:53:23.806 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 111 2023-11-09T03:53:23.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.807+0000 7fd2c9916640 1 -- 172.21.15.76:0/281641967 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd2b006ae70 msgr2=0x7fd2b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:23.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.807+0000 7fd2c9916640 1 --2- 172.21.15.76:0/281641967 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd2b006ae70 0x7fd2b006d330 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40d3c60 tx=0x7fd2b8007a40 comp rx=0 tx=0).stop 2023-11-09T03:53:23.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.807+0000 7fd2c9916640 1 -- 172.21.15.76:0/281641967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2c40d1be0 msgr2=0x7fd2c4153260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:23.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.807+0000 7fd2c9916640 1 --2- 172.21.15.76:0/281641967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2c40d1be0 0x7fd2c4153260 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fd2b402bee0 tx=0x7fd2b40046c0 comp rx=0 tx=0).stop 2023-11-09T03:53:23.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.807+0000 7fd2c9916640 1 -- 172.21.15.76:0/281641967 shutdown_connections 2023-11-09T03:53:23.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.807+0000 7fd2c9916640 1 --2- 172.21.15.76:0/281641967 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd2b006ae70 0x7fd2b006d330 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:23.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.807+0000 7fd2c9916640 1 --2- 172.21.15.76:0/281641967 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2c415d4e0 0x7fd2c409e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:23.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.807+0000 7fd2c9916640 1 --2- 172.21.15.76:0/281641967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2c415d100 0x7fd2c40ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:23.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.807+0000 7fd2c9916640 1 --2- 172.21.15.76:0/281641967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2c40d1be0 0x7fd2c4153260 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:23.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.807+0000 7fd2c9916640 1 -- 172.21.15.76:0/281641967 >> 172.21.15.76:0/281641967 conn(0x7fd2c40d2ef0 msgr2=0x7fd2c40b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:23.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.807+0000 7fd2c9916640 1 -- 172.21.15.76:0/281641967 shutdown_connections 2023-11-09T03:53:23.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:23.807+0000 7fd2c9916640 1 -- 172.21.15.76:0/281641967 wait complete. 2023-11-09T03:53:23.818 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2023-11-09T03:53:23.930 INFO:teuthology.orchestra.run.smithi076.stdout:{"last_snap":12,"last_created":12,"last_destroyed":9,"pending_noop":[],"snaps":[{"snapid":11,"ino":1099511627777,"stamp":"2023-11-09T03:52:04.689262+0000","name":"s25","metadata":{}},{"snapid":12,"ino":1099511627777,"stamp":"2023-11-09T03:52:54.669088+0000","name":"s26","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-09T03:53:23.931 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 12, "last_destroyed": 9, "last_snap": 12, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s25", "snapid": 11, "stamp": "2023-11-09T03:52:04.689262+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s26", "snapid": 12, "stamp": "2023-11-09T03:52:54.669088+0000" } ] } 2023-11-09T03:53:23.931 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:53:24.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.031+0000 7f92ab59e640 1 Processor -- start 2023-11-09T03:53:24.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.032+0000 7f92ab59e640 1 -- start start 2023-11-09T03:53:24.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.032+0000 7f92ab59e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f92ac15cf60 0x7f92ac1608c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:24.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.032+0000 7f92ab59e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92ac0b84e0 0x7f92ac0d14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:24.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.032+0000 7f92ab59e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92ac1611c0 con 0x7f92ac0cd820 2023-11-09T03:53:24.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.032+0000 7f92ab59e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92ac1614a0 con 0x7f92ac15cf60 2023-11-09T03:53:24.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.032+0000 7f92ab59e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac161780 con 0x7f92ac0b84e0 2023-11-09T03:53:24.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.033+0000 7f92aa59c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f92ac0cd820 0x7f92ac0cdc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57030/0 (socket says 172.21.15.76:57030) 2023-11-09T03:53:24.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.033+0000 7f92aa59c640 1 -- 172.21.15.76:0/722751169 learned_addr learned my addr 172.21.15.76:0/722751169 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:53:24.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.033+0000 7f92aad9d640 1 --2- 172.21.15.76:0/722751169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92ac0b84e0 0x7f92ac0d14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:24.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.033+0000 7f92a9d9b640 1 --2- 172.21.15.76:0/722751169 >> v2:172.21.15.113:3300/0 conn(0x7f92ac15cf60 0x7f92ac1608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:24.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.033+0000 7f92a959a640 1 -- 172.21.15.76:0/722751169 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 937581164 0 0) 0x7f92ac1611c0 con 0x7f92ac0cd820 2023-11-09T03:53:24.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.033+0000 7f92a959a640 1 -- 172.21.15.76:0/722751169 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9294003520 con 0x7f92ac0cd820 2023-11-09T03:53:24.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.033+0000 7f92a959a640 1 -- 172.21.15.76:0/722751169 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2133284674 0 0) 0x7f9294003520 con 0x7f92ac0cd820 2023-11-09T03:53:24.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.033+0000 7f92a959a640 1 -- 172.21.15.76:0/722751169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92ac0b84e0 msgr2=0x7f92ac0d14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:24.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.033+0000 7f92a959a640 1 --2- 172.21.15.76:0/722751169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92ac0b84e0 0x7f92ac0d14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:24.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.033+0000 7f92a959a640 1 -- 172.21.15.76:0/722751169 >> v2:172.21.15.113:3300/0 conn(0x7f92ac15cf60 msgr2=0x7f92ac1608c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:24.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.033+0000 7f92a959a640 1 --2- 172.21.15.76:0/722751169 >> v2:172.21.15.113:3300/0 conn(0x7f92ac15cf60 0x7f92ac1608c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:24.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.033+0000 7f92a959a640 1 -- 172.21.15.76:0/722751169 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac0d0ce0 con 0x7f92ac0cd820 2023-11-09T03:53:24.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.033+0000 7f92a959a640 1 -- 172.21.15.76:0/722751169 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f92a4001670 con 0x7f92ac0cd820 2023-11-09T03:53:24.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.033+0000 7f92a959a640 1 -- 172.21.15.76:0/722751169 >> v1:172.21.15.76:6789/0 conn(0x7f92ac0cd820 legacy=0x7f92ac0cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:24.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.033+0000 7f92a959a640 1 --2- 172.21.15.76:0/722751169 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9294003970 0x7f9294003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:24.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.033+0000 7f92a959a640 1 --2- 172.21.15.76:0/722751169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92940047c0 0x7f9294004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:24.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.033+0000 7f92a959a640 1 --2- 172.21.15.76:0/722751169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92ac0b84e0 0x7f9294005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:24.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.033+0000 7f92a959a640 1 -- 172.21.15.76:0/722751169 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac1614a0 con 0x7f9294003970 2023-11-09T03:53:24.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.033+0000 7f92a959a640 1 -- 172.21.15.76:0/722751169 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac161780 con 0x7f92940047c0 2023-11-09T03:53:24.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.033+0000 7f92a959a640 1 -- 172.21.15.76:0/722751169 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f929400b980 con 0x7f92ac0b84e0 2023-11-09T03:53:24.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.033+0000 7f92aa59c640 1 --2- 172.21.15.76:0/722751169 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9294003970 0x7f9294003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:24.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.033+0000 7f92a9d9b640 1 --2- 172.21.15.76:0/722751169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92940047c0 0x7f9294004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:24.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.033+0000 7f92aad9d640 1 --2- 172.21.15.76:0/722751169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92ac0b84e0 0x7f9294005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:24.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.034+0000 7f92aa59c640 1 -- 172.21.15.76:0/722751169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92ac0b84e0 msgr2=0x7f9294005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:24.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.034+0000 7f92aa59c640 1 --2- 172.21.15.76:0/722751169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92ac0b84e0 0x7f9294005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:24.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.034+0000 7f92aa59c640 1 -- 172.21.15.76:0/722751169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92940047c0 msgr2=0x7f9294004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:24.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.034+0000 7f92aa59c640 1 --2- 172.21.15.76:0/722751169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92940047c0 0x7f9294004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:24.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.034+0000 7f92aa59c640 1 -- 172.21.15.76:0/722751169 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9294003520 con 0x7f9294003970 2023-11-09T03:53:24.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.034+0000 7f92aad9d640 1 --2- 172.21.15.76:0/722751169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92ac0b84e0 0x7f9294005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:53:24.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.034+0000 7f92a9d9b640 1 --2- 172.21.15.76:0/722751169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92940047c0 0x7f9294004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:53:24.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.034+0000 7f92aa59c640 1 --2- 172.21.15.76:0/722751169 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9294003970 0x7f9294003d90 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f92a400c240 tx=0x7f92940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c641b5d6f1e427 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:24.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.034+0000 7f92a959a640 1 -- 172.21.15.76:0/722751169 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a402cb00 con 0x7f9294003970 2023-11-09T03:53:24.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.034+0000 7f92a959a640 1 -- 172.21.15.76:0/722751169 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a402cc60 con 0x7f9294003970 2023-11-09T03:53:24.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.034+0000 7f92a959a640 1 -- 172.21.15.76:0/722751169 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a4005b10 con 0x7f9294003970 2023-11-09T03:53:24.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.035+0000 7f92ab59e640 1 -- 172.21.15.76:0/722751169 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9294003970 msgr2=0x7f9294003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:24.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.035+0000 7f92ab59e640 1 --2- 172.21.15.76:0/722751169 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9294003970 0x7f9294003d90 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f92a400c240 tx=0x7f92940057f0 comp rx=0 tx=0).stop 2023-11-09T03:53:24.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.035+0000 7f92aad9d640 1 -- 172.21.15.76:0/722751169 reap_dead start 2023-11-09T03:53:24.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.035+0000 7f92ab59e640 1 -- 172.21.15.76:0/722751169 shutdown_connections 2023-11-09T03:53:24.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.035+0000 7f92ab59e640 1 -- 172.21.15.76:0/722751169 >> 172.21.15.76:0/722751169 conn(0x7f92ac06a5e0 msgr2=0x7f92ac06a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:24.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.035+0000 7f92ab59e640 1 -- 172.21.15.76:0/722751169 shutdown_connections 2023-11-09T03:53:24.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.035+0000 7f92ab59e640 1 -- 172.21.15.76:0/722751169 wait complete. 2023-11-09T03:53:24.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.036+0000 7f92ab59e640 1 Processor -- start 2023-11-09T03:53:24.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.036+0000 7f92ab59e640 1 -- start start 2023-11-09T03:53:24.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.037+0000 7f92ab59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9294003970 0x7f92ac0c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:24.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.037+0000 7f92ab59e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92ac0b1660 0x7f92ac156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:24.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.037+0000 7f92ab59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92ac0c64d0 0x7f92ac0c6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:24.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.037+0000 7f92ab59e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac0c6e90 con 0x7f9294003970 2023-11-09T03:53:24.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.037+0000 7f92ab59e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac1602f0 con 0x7f92ac0c64d0 2023-11-09T03:53:24.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.037+0000 7f92ab59e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac15dc20 con 0x7f92ac0b1660 2023-11-09T03:53:24.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.037+0000 7f92aa59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9294003970 0x7f92ac0c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:24.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.037+0000 7f92aa59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9294003970 0x7f92ac0c85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39216/0 (socket says 172.21.15.76:39216) 2023-11-09T03:53:24.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.037+0000 7f92aa59c640 1 -- 172.21.15.76:0/827747529 learned_addr learned my addr 172.21.15.76:0/827747529 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:24.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.037+0000 7f92aa59c640 1 -- 172.21.15.76:0/827747529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92ac0b1660 msgr2=0x7f92ac156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:24.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.037+0000 7f92aa59c640 1 --2- 172.21.15.76:0/827747529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92ac0b1660 0x7f92ac156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:24.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.037+0000 7f92aa59c640 1 -- 172.21.15.76:0/827747529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92ac0c64d0 msgr2=0x7f92ac0c6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:24.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.037+0000 7f92aa59c640 1 --2- 172.21.15.76:0/827747529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92ac0c64d0 0x7f92ac0c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:24.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.037+0000 7f92aa59c640 1 -- 172.21.15.76:0/827747529 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac0c8e30 con 0x7f9294003970 2023-11-09T03:53:24.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.038+0000 7f92aa59c640 1 --2- 172.21.15.76:0/827747529 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9294003970 0x7f92ac0c85e0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f92a402c4a0 tx=0x7f92a4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:24.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.038+0000 7f929affd640 1 -- 172.21.15.76:0/827747529 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a4004bf0 con 0x7f9294003970 2023-11-09T03:53:24.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.038+0000 7f929affd640 1 -- 172.21.15.76:0/827747529 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a4004d50 con 0x7f9294003970 2023-11-09T03:53:24.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.038+0000 7f929affd640 1 -- 172.21.15.76:0/827747529 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a4007290 con 0x7f9294003970 2023-11-09T03:53:24.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.038+0000 7f92ab59e640 1 -- 172.21.15.76:0/827747529 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92ac0c9110 con 0x7f9294003970 2023-11-09T03:53:24.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.038+0000 7f92ab59e640 1 -- 172.21.15.76:0/827747529 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92ac0c8b20 con 0x7f9294003970 2023-11-09T03:53:24.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.039+0000 7f92ab59e640 1 -- 172.21.15.76:0/827747529 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92ac0742c0 con 0x7f9294003970 2023-11-09T03:53:24.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.039+0000 7f929affd640 1 -- 172.21.15.76:0/827747529 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f92a4032070 con 0x7f9294003970 2023-11-09T03:53:24.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.039+0000 7f929affd640 1 --2- 172.21.15.76:0/827747529 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f928c06aea0 0x7f928c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:24.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.040+0000 7f929affd640 1 -- 172.21.15.76:0/827747529 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8038+0+0 (secure 0 0 0) 0x7f92a40aaee0 con 0x7f9294003970 2023-11-09T03:53:24.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.040+0000 7f92a9d9b640 1 --2- 172.21.15.76:0/827747529 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f928c06aea0 0x7f928c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:24.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.040+0000 7f92a9d9b640 1 --2- 172.21.15.76:0/827747529 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f928c06aea0 0x7f928c06d360 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f92ac160080 tx=0x7f92a0009750 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:24.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.044+0000 7f929affd640 1 -- 172.21.15.76:0/827747529 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f92a40756c0 con 0x7f9294003970 2023-11-09T03:53:24.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.244+0000 7f92ab59e640 1 -- 172.21.15.76:0/827747529 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92ac0b2290 con 0x7f9294003970 2023-11-09T03:53:24.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.244+0000 7f929affd640 1 -- 172.21.15.76:0/827747529 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4929 (secure 0 0 0) 0x7f92a4045030 con 0x7f9294003970 2023-11-09T03:53:24.244 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:53:24.244 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":111,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":8772,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2075292158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2075292158},{"type":"v1","addr":"172.21.15.76:6835","nonce":2075292158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:53:17.497627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":7144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7144":{"gid":7144,"name":"b","rank":1,"incarnation":101,"state":"up:active","state_seq":8,"addr":"172.21.15.113:6837/7360844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":7360844},{"type":"v1","addr":"172.21.15.113:6837","nonce":7360844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:resolve","state_seq":12,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:53:24.245 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 111 2023-11-09T03:53:24.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.246+0000 7f92ab59e640 1 -- 172.21.15.76:0/827747529 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f928c06aea0 msgr2=0x7f928c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:24.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.246+0000 7f92ab59e640 1 --2- 172.21.15.76:0/827747529 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f928c06aea0 0x7f928c06d360 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f92ac160080 tx=0x7f92a0009750 comp rx=0 tx=0).stop 2023-11-09T03:53:24.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.246+0000 7f92ab59e640 1 -- 172.21.15.76:0/827747529 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9294003970 msgr2=0x7f92ac0c85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:24.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.246+0000 7f92ab59e640 1 --2- 172.21.15.76:0/827747529 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9294003970 0x7f92ac0c85e0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f92a402c4a0 tx=0x7f92a4001a00 comp rx=0 tx=0).stop 2023-11-09T03:53:24.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.246+0000 7f92ab59e640 1 -- 172.21.15.76:0/827747529 shutdown_connections 2023-11-09T03:53:24.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.246+0000 7f92ab59e640 1 --2- 172.21.15.76:0/827747529 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f928c06aea0 0x7f928c06d360 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:24.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.246+0000 7f92ab59e640 1 --2- 172.21.15.76:0/827747529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92ac0c64d0 0x7f92ac0c6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:24.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.246+0000 7f92ab59e640 1 --2- 172.21.15.76:0/827747529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92ac0b1660 0x7f92ac156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:24.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.246+0000 7f92ab59e640 1 --2- 172.21.15.76:0/827747529 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9294003970 0x7f92ac0c85e0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:24.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.246+0000 7f92ab59e640 1 -- 172.21.15.76:0/827747529 >> 172.21.15.76:0/827747529 conn(0x7f92ac06a5e0 msgr2=0x7f92ac0b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:24.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.246+0000 7f92ab59e640 1 -- 172.21.15.76:0/827747529 shutdown_connections 2023-11-09T03:53:24.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.246+0000 7f92ab59e640 1 -- 172.21.15.76:0/827747529 wait complete. 2023-11-09T03:53:24.256 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2023-11-09T03:53:24.371 INFO:teuthology.orchestra.run.smithi076.stdout:{"last_snap":12,"last_created":12,"last_destroyed":9,"pending_noop":[],"snaps":[{"snapid":11,"ino":1099511627777,"stamp":"2023-11-09T03:52:04.689262+0000","name":"s25","metadata":{}},{"snapid":12,"ino":1099511627777,"stamp":"2023-11-09T03:52:54.669088+0000","name":"s26","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-09T03:53:24.371 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 12, "last_destroyed": 9, "last_snap": 12, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s25", "snapid": 11, "stamp": "2023-11-09T03:52:04.689262+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s26", "snapid": 12, "stamp": "2023-11-09T03:52:54.669088+0000" } ] } 2023-11-09T03:53:24.372 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:1 2023-11-09T03:53:24.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.472+0000 7f1450561640 1 Processor -- start 2023-11-09T03:53:24.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.472+0000 7f1450561640 1 -- start start 2023-11-09T03:53:24.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.473+0000 7f1450561640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f144815d1a0 0x7f14480d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:24.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.473+0000 7f1450561640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f14480ce8b0 0x7f144815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:24.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.473+0000 7f1450561640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f144815e5f0 con 0x7f14480d1ac0 2023-11-09T03:53:24.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.473+0000 7f1450561640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14480cf3c0 con 0x7f144815d1a0 2023-11-09T03:53:24.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.473+0000 7f1450561640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14480cf6a0 con 0x7f14480ce8b0 2023-11-09T03:53:24.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.473+0000 7f144e2fd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f144815d1a0 0x7f14480d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:24.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.473+0000 7f144dafc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f14480d1ac0 0x7f144815d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57038/0 (socket says 172.21.15.76:57038) 2023-11-09T03:53:24.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.473+0000 7f144dafc640 1 -- 172.21.15.76:0/3176904510 learned_addr learned my addr 172.21.15.76:0/3176904510 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:53:24.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.474+0000 7f144e2fd640 1 -- 172.21.15.76:0/3176904510 >> v2:172.21.15.113:3300/0 conn(0x7f14480ce8b0 msgr2=0x7f144815dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:24.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.474+0000 7f144e2fd640 1 --2- 172.21.15.76:0/3176904510 >> v2:172.21.15.113:3300/0 conn(0x7f14480ce8b0 0x7f144815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:24.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.474+0000 7f144e2fd640 1 -- 172.21.15.76:0/3176904510 >> v1:172.21.15.76:6789/0 conn(0x7f14480d1ac0 legacy=0x7f144815d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:24.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.474+0000 7f144e2fd640 1 -- 172.21.15.76:0/3176904510 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14480cece0 con 0x7f144815d1a0 2023-11-09T03:53:24.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.474+0000 7f144d2fb640 1 -- 172.21.15.76:0/3176904510 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 824270488 0 0) 0x7f144815e5f0 con 0x7f14480d1ac0 2023-11-09T03:53:24.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.474+0000 7f144e2fd640 1 --2- 172.21.15.76:0/3176904510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f144815d1a0 0x7f14480d1580 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f144808d220 tx=0x7f144402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b7b8cd9eaf5d8a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:24.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.475+0000 7f144d2fb640 1 -- 172.21.15.76:0/3176904510 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1444031830 con 0x7f144815d1a0 2023-11-09T03:53:24.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.475+0000 7f144d2fb640 1 -- 172.21.15.76:0/3176904510 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1444031990 con 0x7f144815d1a0 2023-11-09T03:53:24.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.475+0000 7f144d2fb640 1 -- 172.21.15.76:0/3176904510 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1444031d20 con 0x7f144815d1a0 2023-11-09T03:53:24.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.475+0000 7f1450561640 1 -- 172.21.15.76:0/3176904510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f144815d1a0 msgr2=0x7f14480d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:24.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.475+0000 7f1450561640 1 --2- 172.21.15.76:0/3176904510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f144815d1a0 0x7f14480d1580 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f144808d220 tx=0x7f144402b8d0 comp rx=0 tx=0).stop 2023-11-09T03:53:24.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.475+0000 7f1450561640 1 -- 172.21.15.76:0/3176904510 shutdown_connections 2023-11-09T03:53:24.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.475+0000 7f1450561640 1 --2- 172.21.15.76:0/3176904510 >> v2:172.21.15.113:3300/0 conn(0x7f14480ce8b0 0x7f144815dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:24.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.475+0000 7f1450561640 1 --2- 172.21.15.76:0/3176904510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f144815d1a0 0x7f14480d1580 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:24.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.475+0000 7f1450561640 1 -- 172.21.15.76:0/3176904510 >> 172.21.15.76:0/3176904510 conn(0x7f14480d2d90 msgr2=0x7f14480d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:24.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.475+0000 7f1450561640 1 -- 172.21.15.76:0/3176904510 shutdown_connections 2023-11-09T03:53:24.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.475+0000 7f1450561640 1 -- 172.21.15.76:0/3176904510 wait complete. 2023-11-09T03:53:24.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.476+0000 7f1450561640 1 Processor -- start 2023-11-09T03:53:24.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.476+0000 7f1450561640 1 -- start start 2023-11-09T03:53:24.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.477+0000 7f1450561640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14480ce8b0 0x7f1448159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:24.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.477+0000 7f1450561640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14480d1ac0 0x7f1448157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:24.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.477+0000 7f1450561640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f144815d1a0 0x7f14481f2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:24.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.477+0000 7f1450561640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1448157ad0 con 0x7f14480ce8b0 2023-11-09T03:53:24.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.477+0000 7f1450561640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14481f28a0 con 0x7f14480d1ac0 2023-11-09T03:53:24.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.477+0000 7f1450561640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14481571d0 con 0x7f144815d1a0 2023-11-09T03:53:24.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.477+0000 7f144e2fd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14480ce8b0 0x7f1448159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:24.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.477+0000 7f144e2fd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14480ce8b0 0x7f1448159b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39224/0 (socket says 172.21.15.76:39224) 2023-11-09T03:53:24.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.477+0000 7f144e2fd640 1 -- 172.21.15.76:0/1657253956 learned_addr learned my addr 172.21.15.76:0/1657253956 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:24.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.477+0000 7f144dafc640 1 --2- 172.21.15.76:0/1657253956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14480d1ac0 0x7f1448157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:24.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.478+0000 7f144eafe640 1 --2- 172.21.15.76:0/1657253956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f144815d1a0 0x7f14481f2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:24.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.478+0000 7f144e2fd640 1 -- 172.21.15.76:0/1657253956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f144815d1a0 msgr2=0x7f14481f2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:24.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.478+0000 7f144e2fd640 1 --2- 172.21.15.76:0/1657253956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f144815d1a0 0x7f14481f2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:24.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.478+0000 7f144e2fd640 1 -- 172.21.15.76:0/1657253956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14480d1ac0 msgr2=0x7f1448157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:24.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.478+0000 7f144e2fd640 1 --2- 172.21.15.76:0/1657253956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14480d1ac0 0x7f1448157590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:24.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.478+0000 7f144e2fd640 1 -- 172.21.15.76:0/1657253956 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14440093a0 con 0x7f14480ce8b0 2023-11-09T03:53:24.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.478+0000 7f144dafc640 1 --2- 172.21.15.76:0/1657253956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14480d1ac0 0x7f1448157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:53:24.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.478+0000 7f144eafe640 1 --2- 172.21.15.76:0/1657253956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f144815d1a0 0x7f14481f2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:53:24.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.478+0000 7f144e2fd640 1 --2- 172.21.15.76:0/1657253956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14480ce8b0 0x7f1448159b90 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f1444000c00 tx=0x7f14440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:24.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.478+0000 7f143effd640 1 -- 172.21.15.76:0/1657253956 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144403aa60 con 0x7f14480ce8b0 2023-11-09T03:53:24.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.478+0000 7f143effd640 1 -- 172.21.15.76:0/1657253956 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f144403abe0 con 0x7f14480ce8b0 2023-11-09T03:53:24.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.478+0000 7f1450561640 1 -- 172.21.15.76:0/1657253956 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1448156c10 con 0x7f14480ce8b0 2023-11-09T03:53:24.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.478+0000 7f143effd640 1 -- 172.21.15.76:0/1657253956 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14440443f0 con 0x7f14480ce8b0 2023-11-09T03:53:24.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.478+0000 7f1450561640 1 -- 172.21.15.76:0/1657253956 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14481f2530 con 0x7f14480ce8b0 2023-11-09T03:53:24.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.479+0000 7f1450561640 1 -- 172.21.15.76:0/1657253956 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1448074330 con 0x7f14480ce8b0 2023-11-09T03:53:24.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.479+0000 7f143effd640 1 -- 172.21.15.76:0/1657253956 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f144404d450 con 0x7f14480ce8b0 2023-11-09T03:53:24.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.480+0000 7f143effd640 1 --2- 172.21.15.76:0/1657253956 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f142c06aea0 0x7f142c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:24.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.480+0000 7f143effd640 1 -- 172.21.15.76:0/1657253956 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8038+0+0 (secure 0 0 0) 0x7f14440ac150 con 0x7f14480ce8b0 2023-11-09T03:53:24.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.480+0000 7f144dafc640 1 --2- 172.21.15.76:0/1657253956 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f142c06aea0 0x7f142c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:24.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.480+0000 7f144dafc640 1 --2- 172.21.15.76:0/1657253956 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f142c06aea0 0x7f142c06d360 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f14480b5d80 tx=0x7f1438009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:24.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.484+0000 7f143effd640 1 -- 172.21.15.76:0/1657253956 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f14440768e0 con 0x7f14480ce8b0 2023-11-09T03:53:24.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.717+0000 7f1450561640 1 -- 172.21.15.76:0/1657253956 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:1"} v 0) v1 -- 0x7f14480b5500 con 0x7f14480ce8b0 2023-11-09T03:53:24.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.718+0000 7f143effd640 1 -- 172.21.15.76:0/1657253956 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:1"}]=0 failed mds gid 7144 v112) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1444079f00 con 0x7f14480ce8b0 2023-11-09T03:53:24.718 INFO:teuthology.orchestra.run.smithi076.stderr:failed mds gid 7144 2023-11-09T03:53:24.721 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.721+0000 7f1450561640 1 -- 172.21.15.76:0/1657253956 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f142c06aea0 msgr2=0x7f142c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:24.721 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.721+0000 7f1450561640 1 --2- 172.21.15.76:0/1657253956 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f142c06aea0 0x7f142c06d360 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f14480b5d80 tx=0x7f1438009040 comp rx=0 tx=0).stop 2023-11-09T03:53:24.721 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.721+0000 7f1450561640 1 -- 172.21.15.76:0/1657253956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14480ce8b0 msgr2=0x7f1448159b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:24.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.721+0000 7f1450561640 1 --2- 172.21.15.76:0/1657253956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14480ce8b0 0x7f1448159b90 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f1444000c00 tx=0x7f14440046c0 comp rx=0 tx=0).stop 2023-11-09T03:53:24.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.721+0000 7f1450561640 1 -- 172.21.15.76:0/1657253956 shutdown_connections 2023-11-09T03:53:24.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.721+0000 7f1450561640 1 --2- 172.21.15.76:0/1657253956 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f142c06aea0 0x7f142c06d360 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:24.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.721+0000 7f1450561640 1 --2- 172.21.15.76:0/1657253956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f144815d1a0 0x7f14481f2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:24.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.721+0000 7f1450561640 1 --2- 172.21.15.76:0/1657253956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14480d1ac0 0x7f1448157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:24.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.721+0000 7f1450561640 1 --2- 172.21.15.76:0/1657253956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14480ce8b0 0x7f1448159b90 secure :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f1444000c00 tx=0x7f14440046c0 comp rx=0 tx=0).stop 2023-11-09T03:53:24.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.721+0000 7f1450561640 1 -- 172.21.15.76:0/1657253956 >> 172.21.15.76:0/1657253956 conn(0x7f14480d2d90 msgr2=0x7f144815d580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:24.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.722+0000 7f1450561640 1 -- 172.21.15.76:0/1657253956 shutdown_connections 2023-11-09T03:53:24.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.722+0000 7f1450561640 1 -- 172.21.15.76:0/1657253956 wait complete. 2023-11-09T03:53:24.734 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-09T03:53:24.734 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-09T03:53:24.735 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-09T03:53:24.735 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:53:24.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:53:24.736 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi113 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b' 2023-11-09T03:53:24.736 INFO:tasks.ceph.mds.b:Stopped 2023-11-09T03:53:24.737 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-09T03:53:24.739 INFO:tasks.ceph.mds.b:Started 2023-11-09T03:53:24.739 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:53:24.817 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2023-11-09T03:53:24.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.840+0000 7f60e5b05640 1 Processor -- start 2023-11-09T03:53:24.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.840+0000 7f60e5b05640 1 -- start start 2023-11-09T03:53:24.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.841+0000 7f60e5b05640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f60e015d100 0x7f60e00ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:24.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.841+0000 7f60e5b05640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f60e015d4e0 0x7f60e00d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:24.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.841+0000 7f60e5b05640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60e00b32b0 con 0x7f60e00d3b60 2023-11-09T03:53:24.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.841+0000 7f60e5b05640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60e00cee10 con 0x7f60e015d100 2023-11-09T03:53:24.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.841+0000 7f60e5b05640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60e00b4750 con 0x7f60e015d4e0 2023-11-09T03:53:24.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.841+0000 7f60df7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f60e015d100 0x7f60e00ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:24.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.841+0000 7f60dffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f60e00d3b60 0x7f60e00cf520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57044/0 (socket says 172.21.15.76:57044) 2023-11-09T03:53:24.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.841+0000 7f60dffff640 1 -- 172.21.15.76:0/2827490818 learned_addr learned my addr 172.21.15.76:0/2827490818 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:53:24.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.842+0000 7f60d6ffd640 1 --2- 172.21.15.76:0/2827490818 >> v2:172.21.15.113:3300/0 conn(0x7f60e015d4e0 0x7f60e00d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:24.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.842+0000 7f60df7fe640 1 -- 172.21.15.76:0/2827490818 >> v2:172.21.15.113:3300/0 conn(0x7f60e015d4e0 msgr2=0x7f60e00d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:24.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.842+0000 7f60df7fe640 1 --2- 172.21.15.76:0/2827490818 >> v2:172.21.15.113:3300/0 conn(0x7f60e015d4e0 0x7f60e00d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:24.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.842+0000 7f60df7fe640 1 -- 172.21.15.76:0/2827490818 >> v1:172.21.15.76:6789/0 conn(0x7f60e00d3b60 legacy=0x7f60e00cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:24.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.842+0000 7f60df7fe640 1 -- 172.21.15.76:0/2827490818 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e00b2980 con 0x7f60e015d100 2023-11-09T03:53:24.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.842+0000 7f60deffd640 1 -- 172.21.15.76:0/2827490818 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 447443074 0 0) 0x7f60e00b32b0 con 0x7f60e00d3b60 2023-11-09T03:53:24.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.842+0000 7f60df7fe640 1 --2- 172.21.15.76:0/2827490818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60e015d100 0x7f60e00ce510 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f60e0160da0 tx=0x7f60d002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857b8aaccdaeafd3 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:24.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.842+0000 7f60deffd640 1 -- 172.21.15.76:0/2827490818 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d0031830 con 0x7f60e015d100 2023-11-09T03:53:24.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.843+0000 7f60deffd640 1 -- 172.21.15.76:0/2827490818 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d0031990 con 0x7f60e015d100 2023-11-09T03:53:24.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.843+0000 7f60deffd640 1 -- 172.21.15.76:0/2827490818 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d0031d20 con 0x7f60e015d100 2023-11-09T03:53:24.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.843+0000 7f60e5b05640 1 -- 172.21.15.76:0/2827490818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60e015d100 msgr2=0x7f60e00ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:24.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.843+0000 7f60e5b05640 1 --2- 172.21.15.76:0/2827490818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60e015d100 0x7f60e00ce510 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f60e0160da0 tx=0x7f60d002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:53:24.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.843+0000 7f60e5b05640 1 -- 172.21.15.76:0/2827490818 shutdown_connections 2023-11-09T03:53:24.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.843+0000 7f60e5b05640 1 --2- 172.21.15.76:0/2827490818 >> v2:172.21.15.113:3300/0 conn(0x7f60e015d4e0 0x7f60e00d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:24.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.843+0000 7f60e5b05640 1 --2- 172.21.15.76:0/2827490818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60e015d100 0x7f60e00ce510 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:24.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.843+0000 7f60e5b05640 1 -- 172.21.15.76:0/2827490818 >> 172.21.15.76:0/2827490818 conn(0x7f60e00d2ef0 msgr2=0x7f60e00d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:24.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.843+0000 7f60e5b05640 1 -- 172.21.15.76:0/2827490818 shutdown_connections 2023-11-09T03:53:24.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.843+0000 7f60e5b05640 1 -- 172.21.15.76:0/2827490818 wait complete. 2023-11-09T03:53:24.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.844+0000 7f60e5b05640 1 Processor -- start 2023-11-09T03:53:24.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.844+0000 7f60e5b05640 1 -- start start 2023-11-09T03:53:24.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.845+0000 7f60e5b05640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60e00d3b60 0x7f60e00b7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:24.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.845+0000 7f60e5b05640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60e015d100 0x7f60e015e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:24.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.845+0000 7f60e5b05640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60e015d4e0 0x7f60e01f2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:24.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.845+0000 7f60e5b05640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e015de50 con 0x7f60e015d100 2023-11-09T03:53:24.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.845+0000 7f60e5b05640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e01f3a00 con 0x7f60e015d4e0 2023-11-09T03:53:24.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.845+0000 7f60e5b05640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60e00b7b80 con 0x7f60e00d3b60 2023-11-09T03:53:24.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.845+0000 7f60d6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60e015d100 0x7f60e015e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:24.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.845+0000 7f60d6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60e015d100 0x7f60e015e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39248/0 (socket says 172.21.15.76:39248) 2023-11-09T03:53:24.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.845+0000 7f60d6ffd640 1 -- 172.21.15.76:0/2109548004 learned_addr learned my addr 172.21.15.76:0/2109548004 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:24.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.845+0000 7f60df7fe640 1 --2- 172.21.15.76:0/2109548004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60e00d3b60 0x7f60e00b7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:24.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.845+0000 7f60dffff640 1 --2- 172.21.15.76:0/2109548004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60e015d4e0 0x7f60e01f2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:24.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.845+0000 7f60d6ffd640 1 -- 172.21.15.76:0/2109548004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60e00d3b60 msgr2=0x7f60e00b7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:24.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.845+0000 7f60d6ffd640 1 --2- 172.21.15.76:0/2109548004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60e00d3b60 0x7f60e00b7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:24.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.845+0000 7f60d6ffd640 1 -- 172.21.15.76:0/2109548004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60e015d4e0 msgr2=0x7f60e01f2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:24.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.846+0000 7f60d6ffd640 1 --2- 172.21.15.76:0/2109548004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60e015d4e0 0x7f60e01f2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:24.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.846+0000 7f60d6ffd640 1 -- 172.21.15.76:0/2109548004 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d00093a0 con 0x7f60e015d100 2023-11-09T03:53:24.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.846+0000 7f60df7fe640 1 --2- 172.21.15.76:0/2109548004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60e00d3b60 0x7f60e00b7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:53:24.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.846+0000 7f60d6ffd640 1 --2- 172.21.15.76:0/2109548004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60e015d100 0x7f60e015e270 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f60c800d800 tx=0x7f60c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:24.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.846+0000 7f60dcff9640 1 -- 172.21.15.76:0/2109548004 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c800b6c0 con 0x7f60e015d100 2023-11-09T03:53:24.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.846+0000 7f60dcff9640 1 -- 172.21.15.76:0/2109548004 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c800b820 con 0x7f60e015d100 2023-11-09T03:53:24.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.846+0000 7f60e5b05640 1 -- 172.21.15.76:0/2109548004 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e01f3440 con 0x7f60e015d100 2023-11-09T03:53:24.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.846+0000 7f60dcff9640 1 -- 172.21.15.76:0/2109548004 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c800bbb0 con 0x7f60e015d100 2023-11-09T03:53:24.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.846+0000 7f60e5b05640 1 -- 172.21.15.76:0/2109548004 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e015da70 con 0x7f60e015d100 2023-11-09T03:53:24.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.847+0000 7f60e5b05640 1 -- 172.21.15.76:0/2109548004 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60e015df90 con 0x7f60e015d100 2023-11-09T03:53:24.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.847+0000 7f60dcff9640 1 -- 172.21.15.76:0/2109548004 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f60c800bd10 con 0x7f60e015d100 2023-11-09T03:53:24.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.848+0000 7f60dcff9640 1 --2- 172.21.15.76:0/2109548004 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f60c006ae70 0x7f60c006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:24.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.848+0000 7f60dcff9640 1 -- 172.21.15.76:0/2109548004 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f60c808a480 con 0x7f60e015d100 2023-11-09T03:53:24.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.848+0000 7f60df7fe640 1 --2- 172.21.15.76:0/2109548004 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f60c006ae70 0x7f60c006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:24.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.848+0000 7f60df7fe640 1 --2- 172.21.15.76:0/2109548004 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f60c006ae70 0x7f60c006d330 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f60d0036000 tx=0x7f60d0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:24.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:24.852+0000 7f60dcff9640 1 -- 172.21.15.76:0/2109548004 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f60e015df90 con 0x7f60e015d100 2023-11-09T03:53:25.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:25.054+0000 7f60e5b05640 1 -- 172.21.15.76:0/2109548004 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60e015dc30 con 0x7f60e015d100 2023-11-09T03:53:25.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:25.054+0000 7f60dcff9640 1 -- 172.21.15.76:0/2109548004 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+4143 (secure 0 0 0) 0x7f60c80586e0 con 0x7f60e015d100 2023-11-09T03:53:25.054 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:53:25.054 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":113,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:53:24.722387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":8772},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:resolve","state_seq":12,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8772":{"gid":8772,"name":"a","rank":1,"incarnation":113,"state":"up:replay","state_seq":1,"addr":"172.21.15.76:6835/2075292158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2075292158},{"type":"v1","addr":"172.21.15.76:6835","nonce":2075292158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:53:25.055 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 113 2023-11-09T03:53:25.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:25.056+0000 7f60e5b05640 1 -- 172.21.15.76:0/2109548004 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f60c006ae70 msgr2=0x7f60c006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:25.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:25.056+0000 7f60e5b05640 1 --2- 172.21.15.76:0/2109548004 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f60c006ae70 0x7f60c006d330 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f60d0036000 tx=0x7f60d0008030 comp rx=0 tx=0).stop 2023-11-09T03:53:25.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:25.056+0000 7f60e5b05640 1 -- 172.21.15.76:0/2109548004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60e015d100 msgr2=0x7f60e015e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:25.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:25.056+0000 7f60e5b05640 1 --2- 172.21.15.76:0/2109548004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60e015d100 0x7f60e015e270 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f60c800d800 tx=0x7f60c800dcc0 comp rx=0 tx=0).stop 2023-11-09T03:53:25.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:25.056+0000 7f60e5b05640 1 -- 172.21.15.76:0/2109548004 shutdown_connections 2023-11-09T03:53:25.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:25.056+0000 7f60e5b05640 1 --2- 172.21.15.76:0/2109548004 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f60c006ae70 0x7f60c006d330 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:25.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:25.056+0000 7f60e5b05640 1 --2- 172.21.15.76:0/2109548004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60e015d4e0 0x7f60e01f2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:25.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:25.056+0000 7f60e5b05640 1 --2- 172.21.15.76:0/2109548004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60e015d100 0x7f60e015e270 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:25.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:25.056+0000 7f60e5b05640 1 --2- 172.21.15.76:0/2109548004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60e00d3b60 0x7f60e00b7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:25.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:25.056+0000 7f60e5b05640 1 -- 172.21.15.76:0/2109548004 >> 172.21.15.76:0/2109548004 conn(0x7f60e00d2ef0 msgr2=0x7f60e006bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:25.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:25.056+0000 7f60e5b05640 1 -- 172.21.15.76:0/2109548004 shutdown_connections 2023-11-09T03:53:25.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:25.056+0000 7f60e5b05640 1 -- 172.21.15.76:0/2109548004 wait complete. 2023-11-09T03:53:25.066 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-11-09T03:53:25.474 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-11-09T03:53:30.068 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:53:30.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.177+0000 7f059f3b6640 1 Processor -- start 2023-11-09T03:53:30.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.177+0000 7f059f3b6640 1 -- start start 2023-11-09T03:53:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.178+0000 7f059f3b6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f05980ccf50 0x7f059806a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.178+0000 7f059f3b6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f059806a850 0x7f05980b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.178+0000 7f059f3b6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059806aca0 con 0x7f0598158880 2023-11-09T03:53:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.178+0000 7f059f3b6640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05980ce2b0 con 0x7f059806a850 2023-11-09T03:53:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.178+0000 7f059f3b6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05980cd3e0 con 0x7f05980ccf50 2023-11-09T03:53:30.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.178+0000 7f059d953640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f059806a850 0x7f05980b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:30.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.178+0000 7f059d953640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f059806a850 0x7f05980b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39256/0 (socket says 172.21.15.76:39256) 2023-11-09T03:53:30.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.178+0000 7f059d953640 1 -- 172.21.15.76:0/1629722275 learned_addr learned my addr 172.21.15.76:0/1629722275 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:30.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.178+0000 7f059c951640 1 --2- 172.21.15.76:0/1629722275 >> v2:172.21.15.113:3300/0 conn(0x7f05980ccf50 0x7f059806a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:30.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.178+0000 7f059d953640 1 -- 172.21.15.76:0/1629722275 >> v2:172.21.15.113:3300/0 conn(0x7f05980ccf50 msgr2=0x7f059806a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:30.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.178+0000 7f059d953640 1 --2- 172.21.15.76:0/1629722275 >> v2:172.21.15.113:3300/0 conn(0x7f05980ccf50 0x7f059806a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:30.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.178+0000 7f059d953640 1 -- 172.21.15.76:0/1629722275 >> v1:172.21.15.113:6789/0 conn(0x7f0598158880 legacy=0x7f059815be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:30.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.178+0000 7f059d953640 1 -- 172.21.15.76:0/1629722275 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05980cd630 con 0x7f059806a850 2023-11-09T03:53:30.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.179+0000 7f059d953640 1 --2- 172.21.15.76:0/1629722275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f059806a850 0x7f05980b1b40 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f05980afda0 tx=0x7f059002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9dbfafb2b1f0f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:30.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.179+0000 7f058ffff640 1 -- 172.21.15.76:0/1629722275 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059002ca80 con 0x7f059806a850 2023-11-09T03:53:30.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.179+0000 7f058ffff640 1 -- 172.21.15.76:0/1629722275 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059002cbe0 con 0x7f059806a850 2023-11-09T03:53:30.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.179+0000 7f058ffff640 1 -- 172.21.15.76:0/1629722275 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05900343f0 con 0x7f059806a850 2023-11-09T03:53:30.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.180+0000 7f059f3b6640 1 -- 172.21.15.76:0/1629722275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f059806a850 msgr2=0x7f05980b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:30.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.180+0000 7f059f3b6640 1 --2- 172.21.15.76:0/1629722275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f059806a850 0x7f05980b1b40 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f05980afda0 tx=0x7f059002bee0 comp rx=0 tx=0).stop 2023-11-09T03:53:30.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.180+0000 7f059f3b6640 1 -- 172.21.15.76:0/1629722275 shutdown_connections 2023-11-09T03:53:30.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.180+0000 7f059f3b6640 1 --2- 172.21.15.76:0/1629722275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f059806a850 0x7f05980b1b40 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:30.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.180+0000 7f059f3b6640 1 --2- 172.21.15.76:0/1629722275 >> v2:172.21.15.113:3300/0 conn(0x7f05980ccf50 0x7f059806a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:30.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.180+0000 7f059f3b6640 1 -- 172.21.15.76:0/1629722275 >> 172.21.15.76:0/1629722275 conn(0x7f05980b0870 msgr2=0x7f05980b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:30.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.180+0000 7f059f3b6640 1 -- 172.21.15.76:0/1629722275 shutdown_connections 2023-11-09T03:53:30.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.180+0000 7f059f3b6640 1 -- 172.21.15.76:0/1629722275 wait complete. 2023-11-09T03:53:30.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.181+0000 7f059f3b6640 1 Processor -- start 2023-11-09T03:53:30.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.181+0000 7f059f3b6640 1 -- start start 2023-11-09T03:53:30.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.182+0000 7f059f3b6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f059806a850 0x7f05980aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:30.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.182+0000 7f059f3b6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05980ccf50 0x7f059814fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:30.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.182+0000 7f059f3b6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0598158880 0x7f059815d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:30.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.182+0000 7f059f3b6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0598154310 con 0x7f0598158880 2023-11-09T03:53:30.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.182+0000 7f059f3b6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05981539e0 con 0x7f05980ccf50 2023-11-09T03:53:30.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.182+0000 7f059f3b6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0598153cc0 con 0x7f059806a850 2023-11-09T03:53:30.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.182+0000 7f059d953640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0598158880 0x7f059815d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:30.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.182+0000 7f059d953640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0598158880 0x7f059815d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39262/0 (socket says 172.21.15.76:39262) 2023-11-09T03:53:30.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.182+0000 7f059d953640 1 -- 172.21.15.76:0/2386548050 learned_addr learned my addr 172.21.15.76:0/2386548050 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:30.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.182+0000 7f059d953640 1 -- 172.21.15.76:0/2386548050 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f059806a850 msgr2=0x7f05980aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:30.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.182+0000 7f059d953640 1 --2- 172.21.15.76:0/2386548050 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f059806a850 0x7f05980aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:30.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.182+0000 7f059d953640 1 -- 172.21.15.76:0/2386548050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05980ccf50 msgr2=0x7f059814fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:30.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.182+0000 7f059d953640 1 --2- 172.21.15.76:0/2386548050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05980ccf50 0x7f059814fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:30.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.182+0000 7f059d953640 1 -- 172.21.15.76:0/2386548050 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0590009b30 con 0x7f0598158880 2023-11-09T03:53:30.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.182+0000 7f059d953640 1 --2- 172.21.15.76:0/2386548050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0598158880 0x7f059815d760 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f0590036040 tx=0x7f0590004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:30.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.182+0000 7f058dffb640 1 -- 172.21.15.76:0/2386548050 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0590009510 con 0x7f0598158880 2023-11-09T03:53:30.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.182+0000 7f058dffb640 1 -- 172.21.15.76:0/2386548050 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059003e450 con 0x7f0598158880 2023-11-09T03:53:30.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.182+0000 7f058dffb640 1 -- 172.21.15.76:0/2386548050 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059003e800 con 0x7f0598158880 2023-11-09T03:53:30.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.182+0000 7f059f3b6640 1 -- 172.21.15.76:0/2386548050 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0598153fa0 con 0x7f0598158880 2023-11-09T03:53:30.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.183+0000 7f059f3b6640 1 -- 172.21.15.76:0/2386548050 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05981f17c0 con 0x7f0598158880 2023-11-09T03:53:30.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.184+0000 7f059f3b6640 1 -- 172.21.15.76:0/2386548050 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0598074330 con 0x7f0598158880 2023-11-09T03:53:30.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.184+0000 7f058dffb640 1 -- 172.21.15.76:0/2386548050 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f059002cd50 con 0x7f0598158880 2023-11-09T03:53:30.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.184+0000 7f058dffb640 1 --2- 172.21.15.76:0/2386548050 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f057c06ae70 0x7f057c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:30.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.185+0000 7f058dffb640 1 -- 172.21.15.76:0/2386548050 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f05900ac9f0 con 0x7f0598158880 2023-11-09T03:53:30.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.185+0000 7f059d152640 1 --2- 172.21.15.76:0/2386548050 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f057c06ae70 0x7f057c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:30.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.185+0000 7f059d152640 1 --2- 172.21.15.76:0/2386548050 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f057c06ae70 0x7f057c06d330 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f0588000f60 tx=0x7f05880079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:30.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.189+0000 7f058dffb640 1 -- 172.21.15.76:0/2386548050 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0590077130 con 0x7f0598158880 2023-11-09T03:53:30.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.389+0000 7f059f3b6640 1 -- 172.21.15.76:0/2386548050 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05980afdd0 con 0x7f0598158880 2023-11-09T03:53:30.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.389+0000 7f058dffb640 1 -- 172.21.15.76:0/2386548050 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+4943 (secure 0 0 0) 0x7f059007a750 con 0x7f0598158880 2023-11-09T03:53:30.389 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:53:30.389 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":115,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:53:25.733437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":8772},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8772":{"gid":8772,"name":"a","rank":1,"incarnation":113,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.76:6835/2075292158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2075292158},{"type":"v1","addr":"172.21.15.76:6835","nonce":2075292158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:53:30.390 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 115 2023-11-09T03:53:30.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.391+0000 7f059f3b6640 1 -- 172.21.15.76:0/2386548050 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f057c06ae70 msgr2=0x7f057c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:30.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.391+0000 7f059f3b6640 1 --2- 172.21.15.76:0/2386548050 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f057c06ae70 0x7f057c06d330 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f0588000f60 tx=0x7f05880079b0 comp rx=0 tx=0).stop 2023-11-09T03:53:30.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.391+0000 7f059f3b6640 1 -- 172.21.15.76:0/2386548050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0598158880 msgr2=0x7f059815d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:30.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.391+0000 7f059f3b6640 1 --2- 172.21.15.76:0/2386548050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0598158880 0x7f059815d760 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f0590036040 tx=0x7f0590004a40 comp rx=0 tx=0).stop 2023-11-09T03:53:30.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.391+0000 7f059f3b6640 1 -- 172.21.15.76:0/2386548050 shutdown_connections 2023-11-09T03:53:30.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.391+0000 7f059f3b6640 1 --2- 172.21.15.76:0/2386548050 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f057c06ae70 0x7f057c06d330 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:30.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.391+0000 7f059f3b6640 1 --2- 172.21.15.76:0/2386548050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0598158880 0x7f059815d760 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:30.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.391+0000 7f059f3b6640 1 --2- 172.21.15.76:0/2386548050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05980ccf50 0x7f059814fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:30.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.391+0000 7f059f3b6640 1 --2- 172.21.15.76:0/2386548050 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f059806a850 0x7f05980aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:30.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.391+0000 7f059f3b6640 1 -- 172.21.15.76:0/2386548050 >> 172.21.15.76:0/2386548050 conn(0x7f05980b0870 msgr2=0x7f05980b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:30.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.391+0000 7f059f3b6640 1 -- 172.21.15.76:0/2386548050 shutdown_connections 2023-11-09T03:53:30.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.391+0000 7f059f3b6640 1 -- 172.21.15.76:0/2386548050 wait complete. 2023-11-09T03:53:30.401 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-09T03:53:30.402 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:53:30.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.502+0000 7f58215b7640 1 Processor -- start 2023-11-09T03:53:30.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.502+0000 7f58215b7640 1 -- start start 2023-11-09T03:53:30.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.503+0000 7f58215b7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f581c15d100 0x7f581c0ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:30.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.503+0000 7f58215b7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f581c15d4e0 0x7f581c0d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:30.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.503+0000 7f58215b7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f581c0b32b0 con 0x7f581c0d3b60 2023-11-09T03:53:30.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.503+0000 7f58215b7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f581c0cee10 con 0x7f581c15d4e0 2023-11-09T03:53:30.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.503+0000 7f58215b7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0b4750 con 0x7f581c15d100 2023-11-09T03:53:30.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.504+0000 7f581affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f581c15d100 0x7f581c0ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:30.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.504+0000 7f581a7fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f581c15d4e0 0x7f581c0d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:30.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.504+0000 7f581affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f581c15d100 0x7f581c0ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:39784/0 (socket says 172.21.15.76:39784) 2023-11-09T03:53:30.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.504+0000 7f581affd640 1 -- 172.21.15.76:0/2565936649 learned_addr learned my addr 172.21.15.76:0/2565936649 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:30.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.504+0000 7f581affd640 1 -- 172.21.15.76:0/2565936649 >> v2:172.21.15.113:3300/0 conn(0x7f581c15d4e0 msgr2=0x7f581c0d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:30.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.504+0000 7f5819ffb640 1 -- 172.21.15.76:0/2565936649 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1823780264 0 0) 0x7f581c0b32b0 con 0x7f581c0d3b60 2023-11-09T03:53:30.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.504+0000 7f581affd640 1 --2- 172.21.15.76:0/2565936649 >> v2:172.21.15.113:3300/0 conn(0x7f581c15d4e0 0x7f581c0d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:30.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.504+0000 7f581affd640 1 -- 172.21.15.76:0/2565936649 >> v1:172.21.15.113:6789/0 conn(0x7f581c0d3b60 legacy=0x7f581c0cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:30.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.504+0000 7f581affd640 1 -- 172.21.15.76:0/2565936649 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c0b2980 con 0x7f581c15d100 2023-11-09T03:53:30.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.505+0000 7f581affd640 1 --2- 172.21.15.76:0/2565936649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f581c15d100 0x7f581c0ce510 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f580c009ea0 tx=0x7f580c02bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b065c3d710c7c6d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:30.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.505+0000 7f5819ffb640 1 -- 172.21.15.76:0/2565936649 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c02cb40 con 0x7f581c15d100 2023-11-09T03:53:30.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.505+0000 7f5819ffb640 1 -- 172.21.15.76:0/2565936649 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580c02cca0 con 0x7f581c15d100 2023-11-09T03:53:30.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.505+0000 7f5819ffb640 1 -- 172.21.15.76:0/2565936649 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c034610 con 0x7f581c15d100 2023-11-09T03:53:30.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.505+0000 7f58215b7640 1 -- 172.21.15.76:0/2565936649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f581c15d100 msgr2=0x7f581c0ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:30.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.505+0000 7f58215b7640 1 --2- 172.21.15.76:0/2565936649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f581c15d100 0x7f581c0ce510 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f580c009ea0 tx=0x7f580c02bfa0 comp rx=0 tx=0).stop 2023-11-09T03:53:30.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.506+0000 7f58215b7640 1 -- 172.21.15.76:0/2565936649 shutdown_connections 2023-11-09T03:53:30.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.506+0000 7f58215b7640 1 --2- 172.21.15.76:0/2565936649 >> v2:172.21.15.113:3300/0 conn(0x7f581c15d4e0 0x7f581c0d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:30.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.506+0000 7f58215b7640 1 --2- 172.21.15.76:0/2565936649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f581c15d100 0x7f581c0ce510 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:30.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.506+0000 7f58215b7640 1 -- 172.21.15.76:0/2565936649 >> 172.21.15.76:0/2565936649 conn(0x7f581c0d2ef0 msgr2=0x7f581c0d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:30.509 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.506+0000 7f58215b7640 1 -- 172.21.15.76:0/2565936649 shutdown_connections 2023-11-09T03:53:30.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.506+0000 7f58215b7640 1 -- 172.21.15.76:0/2565936649 wait complete. 2023-11-09T03:53:30.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.507+0000 7f58215b7640 1 Processor -- start 2023-11-09T03:53:30.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.507+0000 7f58215b7640 1 -- start start 2023-11-09T03:53:30.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.508+0000 7f58215b7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f581c0d3b60 0x7f581c1f67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:30.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.508+0000 7f58215b7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f581c15d100 0x7f581c156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:30.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.508+0000 7f58215b7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f581c15d4e0 0x7f581c159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:30.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.508+0000 7f58215b7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0b2060 con 0x7f581c0d3b60 2023-11-09T03:53:30.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.508+0000 7f58215b7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0b0510 con 0x7f581c15d4e0 2023-11-09T03:53:30.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.508+0000 7f58215b7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0b07f0 con 0x7f581c15d100 2023-11-09T03:53:30.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.508+0000 7f581affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f581c0d3b60 0x7f581c1f67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:30.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.508+0000 7f581affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f581c0d3b60 0x7f581c1f67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39266/0 (socket says 172.21.15.76:39266) 2023-11-09T03:53:30.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.508+0000 7f581affd640 1 -- 172.21.15.76:0/1907992491 learned_addr learned my addr 172.21.15.76:0/1907992491 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:30.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.508+0000 7f581a7fc640 1 --2- 172.21.15.76:0/1907992491 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f581c15d100 0x7f581c156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:30.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.508+0000 7f581b7fe640 1 --2- 172.21.15.76:0/1907992491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f581c15d4e0 0x7f581c159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:30.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.508+0000 7f581affd640 1 -- 172.21.15.76:0/1907992491 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f581c15d100 msgr2=0x7f581c156ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:30.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.508+0000 7f581affd640 1 --2- 172.21.15.76:0/1907992491 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f581c15d100 0x7f581c156ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:30.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.508+0000 7f581affd640 1 -- 172.21.15.76:0/1907992491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f581c15d4e0 msgr2=0x7f581c159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:30.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.508+0000 7f581affd640 1 --2- 172.21.15.76:0/1907992491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f581c15d4e0 0x7f581c159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:30.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.508+0000 7f581affd640 1 -- 172.21.15.76:0/1907992491 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c009b30 con 0x7f581c0d3b60 2023-11-09T03:53:30.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.508+0000 7f581a7fc640 1 --2- 172.21.15.76:0/1907992491 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f581c15d100 0x7f581c156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:53:30.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.508+0000 7f581b7fe640 1 --2- 172.21.15.76:0/1907992491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f581c15d4e0 0x7f581c159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:53:30.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.508+0000 7f581affd640 1 --2- 172.21.15.76:0/1907992491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f581c0d3b60 0x7f581c1f67f0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f580c009ea0 tx=0x7f580c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:30.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.508+0000 7f58037fe640 1 -- 172.21.15.76:0/1907992491 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c009510 con 0x7f581c0d3b60 2023-11-09T03:53:30.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.508+0000 7f58037fe640 1 -- 172.21.15.76:0/1907992491 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580c03e450 con 0x7f581c0d3b60 2023-11-09T03:53:30.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.508+0000 7f58215b7640 1 -- 172.21.15.76:0/1907992491 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f581c0b0ad0 con 0x7f581c0d3b60 2023-11-09T03:53:30.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.508+0000 7f58037fe640 1 -- 172.21.15.76:0/1907992491 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c03e600 con 0x7f581c0d3b60 2023-11-09T03:53:30.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.509+0000 7f58215b7640 1 -- 172.21.15.76:0/1907992491 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581c0b1090 con 0x7f581c0d3b60 2023-11-09T03:53:30.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.509+0000 7f58215b7640 1 -- 172.21.15.76:0/1907992491 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f581c0b1220 con 0x7f581c0d3b60 2023-11-09T03:53:30.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.510+0000 7f58037fe640 1 -- 172.21.15.76:0/1907992491 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f580c03e760 con 0x7f581c0d3b60 2023-11-09T03:53:30.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.510+0000 7f58037fe640 1 --2- 172.21.15.76:0/1907992491 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f580806ae70 0x7f580806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:30.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.510+0000 7f58037fe640 1 -- 172.21.15.76:0/1907992491 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f580c0ac8f0 con 0x7f581c0d3b60 2023-11-09T03:53:30.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.510+0000 7f581a7fc640 1 --2- 172.21.15.76:0/1907992491 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f580806ae70 0x7f580806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:30.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.511+0000 7f581a7fc640 1 --2- 172.21.15.76:0/1907992491 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f580806ae70 0x7f580806d330 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f581c0b1d40 tx=0x7f5810008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:30.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.514+0000 7f58037fe640 1 -- 172.21.15.76:0/1907992491 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f580c077040 con 0x7f581c0d3b60 2023-11-09T03:53:30.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.716+0000 7f58215b7640 1 -- 172.21.15.76:0/1907992491 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f581c0684b0 con 0x7f581c0d3b60 2023-11-09T03:53:30.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.716+0000 7f58037fe640 1 -- 172.21.15.76:0/1907992491 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+4943 (secure 0 0 0) 0x7f580c07a660 con 0x7f581c0d3b60 2023-11-09T03:53:30.716 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:53:30.716 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":115,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:53:25.733437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":8772},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8772":{"gid":8772,"name":"a","rank":1,"incarnation":113,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.76:6835/2075292158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2075292158},{"type":"v1","addr":"172.21.15.76:6835","nonce":2075292158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:53:30.717 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 115 2023-11-09T03:53:30.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.718+0000 7f58215b7640 1 -- 172.21.15.76:0/1907992491 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f580806ae70 msgr2=0x7f580806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:30.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.718+0000 7f58215b7640 1 --2- 172.21.15.76:0/1907992491 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f580806ae70 0x7f580806d330 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f581c0b1d40 tx=0x7f5810008040 comp rx=0 tx=0).stop 2023-11-09T03:53:30.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.718+0000 7f58215b7640 1 -- 172.21.15.76:0/1907992491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f581c0d3b60 msgr2=0x7f581c1f67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:30.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.718+0000 7f58215b7640 1 --2- 172.21.15.76:0/1907992491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f581c0d3b60 0x7f581c1f67f0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f580c009ea0 tx=0x7f580c004a40 comp rx=0 tx=0).stop 2023-11-09T03:53:30.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.718+0000 7f58215b7640 1 -- 172.21.15.76:0/1907992491 shutdown_connections 2023-11-09T03:53:30.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.718+0000 7f58215b7640 1 --2- 172.21.15.76:0/1907992491 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f580806ae70 0x7f580806d330 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:30.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.718+0000 7f58215b7640 1 --2- 172.21.15.76:0/1907992491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f581c15d4e0 0x7f581c159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:30.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.718+0000 7f58215b7640 1 --2- 172.21.15.76:0/1907992491 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f581c15d100 0x7f581c156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:30.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.718+0000 7f58215b7640 1 --2- 172.21.15.76:0/1907992491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f581c0d3b60 0x7f581c1f67f0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:30.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.718+0000 7f58215b7640 1 -- 172.21.15.76:0/1907992491 >> 172.21.15.76:0/1907992491 conn(0x7f581c0d2ef0 msgr2=0x7f581c15b200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:30.720 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.718+0000 7f58215b7640 1 -- 172.21.15.76:0/1907992491 shutdown_connections 2023-11-09T03:53:30.720 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:30.718+0000 7f58215b7640 1 -- 172.21.15.76:0/1907992491 wait complete. 2023-11-09T03:53:30.728 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:reconnect 2023-11-09T03:53:31.729 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:53:31.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.844+0000 7f6038ce5640 1 Processor -- start 2023-11-09T03:53:31.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.845+0000 7f6038ce5640 1 -- start start 2023-11-09T03:53:31.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.845+0000 7f6038ce5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f603415d100 0x7f60340ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:31.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.846+0000 7f6038ce5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f603415d4e0 0x7f60340d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:31.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.846+0000 7f6038ce5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60340b32b0 con 0x7f60340d3b60 2023-11-09T03:53:31.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.846+0000 7f6038ce5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60340cee10 con 0x7f603415d100 2023-11-09T03:53:31.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.846+0000 7f6038ce5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60340b4750 con 0x7f603415d4e0 2023-11-09T03:53:31.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.846+0000 7f6031d9b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f603415d4e0 0x7f60340d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:31.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.846+0000 7f603259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f603415d100 0x7f60340ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:31.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.846+0000 7f6031d9b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f603415d4e0 0x7f60340d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:39806/0 (socket says 172.21.15.76:39806) 2023-11-09T03:53:31.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.846+0000 7f6032d9d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f60340d3b60 0x7f60340cf520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:46970/0 (socket says 172.21.15.76:46970) 2023-11-09T03:53:31.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.846+0000 7f6031d9b640 1 -- 172.21.15.76:0/2780930314 learned_addr learned my addr 172.21.15.76:0/2780930314 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:31.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.846+0000 7f603259c640 1 -- 172.21.15.76:0/2780930314 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f603415d4e0 msgr2=0x7f60340d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:31.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.846+0000 7f603259c640 1 --2- 172.21.15.76:0/2780930314 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f603415d4e0 0x7f60340d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:31.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.846+0000 7f603259c640 1 -- 172.21.15.76:0/2780930314 >> v1:172.21.15.113:6789/0 conn(0x7f60340d3b60 legacy=0x7f60340cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:31.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.846+0000 7f603259c640 1 -- 172.21.15.76:0/2780930314 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60340b2980 con 0x7f603415d100 2023-11-09T03:53:31.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.846+0000 7f6031d9b640 1 --2- 172.21.15.76:0/2780930314 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f603415d4e0 0x7f60340d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:53:31.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.847+0000 7f603259c640 1 --2- 172.21.15.76:0/2780930314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f603415d100 0x7f60340ce510 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f602c009de0 tx=0x7f602c02bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f14bc421855dada server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:31.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.847+0000 7f603159a640 1 -- 172.21.15.76:0/2780930314 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c02cc90 con 0x7f603415d100 2023-11-09T03:53:31.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.848+0000 7f603159a640 1 -- 172.21.15.76:0/2780930314 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602c02cdf0 con 0x7f603415d100 2023-11-09T03:53:31.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.848+0000 7f603159a640 1 -- 172.21.15.76:0/2780930314 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c034660 con 0x7f603415d100 2023-11-09T03:53:31.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.848+0000 7f6038ce5640 1 -- 172.21.15.76:0/2780930314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f603415d100 msgr2=0x7f60340ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:31.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.848+0000 7f6038ce5640 1 --2- 172.21.15.76:0/2780930314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f603415d100 0x7f60340ce510 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f602c009de0 tx=0x7f602c02bee0 comp rx=0 tx=0).stop 2023-11-09T03:53:31.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.848+0000 7f6038ce5640 1 -- 172.21.15.76:0/2780930314 shutdown_connections 2023-11-09T03:53:31.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.848+0000 7f6038ce5640 1 --2- 172.21.15.76:0/2780930314 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f603415d4e0 0x7f60340d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:31.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.848+0000 7f6038ce5640 1 --2- 172.21.15.76:0/2780930314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f603415d100 0x7f60340ce510 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:31.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.848+0000 7f6038ce5640 1 -- 172.21.15.76:0/2780930314 >> 172.21.15.76:0/2780930314 conn(0x7f60340d2ef0 msgr2=0x7f60340d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:31.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.848+0000 7f6038ce5640 1 -- 172.21.15.76:0/2780930314 shutdown_connections 2023-11-09T03:53:31.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.848+0000 7f6038ce5640 1 -- 172.21.15.76:0/2780930314 wait complete. 2023-11-09T03:53:31.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.850+0000 7f6038ce5640 1 Processor -- start 2023-11-09T03:53:31.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.850+0000 7f6038ce5640 1 -- start start 2023-11-09T03:53:31.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.850+0000 7f6038ce5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60340d3b60 0x7f603415b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:31.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.850+0000 7f6038ce5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f603415d100 0x7f60340b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:31.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.850+0000 7f6038ce5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f603415d4e0 0x7f60340b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:31.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.850+0000 7f6038ce5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6034160660 con 0x7f603415d4e0 2023-11-09T03:53:31.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.850+0000 7f6038ce5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60340b5c10 con 0x7f60340d3b60 2023-11-09T03:53:31.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.850+0000 7f6038ce5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6034160380 con 0x7f603415d100 2023-11-09T03:53:31.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.851+0000 7f6032d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f603415d4e0 0x7f60340b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:31.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.851+0000 7f6032d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f603415d4e0 0x7f60340b0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39276/0 (socket says 172.21.15.76:39276) 2023-11-09T03:53:31.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.851+0000 7f6032d9d640 1 -- 172.21.15.76:0/4009485868 learned_addr learned my addr 172.21.15.76:0/4009485868 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:31.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.851+0000 7f6031d9b640 1 --2- 172.21.15.76:0/4009485868 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f603415d100 0x7f60340b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:31.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.851+0000 7f603259c640 1 --2- 172.21.15.76:0/4009485868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60340d3b60 0x7f603415b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:31.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.851+0000 7f6032d9d640 1 -- 172.21.15.76:0/4009485868 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f603415d100 msgr2=0x7f60340b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:31.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.851+0000 7f6032d9d640 1 --2- 172.21.15.76:0/4009485868 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f603415d100 0x7f60340b03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:31.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.851+0000 7f6032d9d640 1 -- 172.21.15.76:0/4009485868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60340d3b60 msgr2=0x7f603415b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:31.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.851+0000 7f6032d9d640 1 --2- 172.21.15.76:0/4009485868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60340d3b60 0x7f603415b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:31.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.851+0000 7f6032d9d640 1 -- 172.21.15.76:0/4009485868 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601c0094d0 con 0x7f603415d4e0 2023-11-09T03:53:31.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.851+0000 7f6032d9d640 1 --2- 172.21.15.76:0/4009485868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f603415d4e0 0x7f60340b0900 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f6024001eb0 tx=0x7f602400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:31.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.851+0000 7f602affd640 1 -- 172.21.15.76:0/4009485868 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60240074e0 con 0x7f603415d4e0 2023-11-09T03:53:31.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.851+0000 7f602affd640 1 -- 172.21.15.76:0/4009485868 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6024007640 con 0x7f603415d4e0 2023-11-09T03:53:31.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.851+0000 7f6038ce5640 1 -- 172.21.15.76:0/4009485868 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f602c009a70 con 0x7f603415d4e0 2023-11-09T03:53:31.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.851+0000 7f602affd640 1 -- 172.21.15.76:0/4009485868 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60240049e0 con 0x7f603415d4e0 2023-11-09T03:53:31.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.851+0000 7f6038ce5640 1 -- 172.21.15.76:0/4009485868 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60340b6190 con 0x7f603415d4e0 2023-11-09T03:53:31.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.852+0000 7f6038ce5640 1 -- 172.21.15.76:0/4009485868 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60340b5a70 con 0x7f603415d4e0 2023-11-09T03:53:31.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.853+0000 7f602affd640 1 -- 172.21.15.76:0/4009485868 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6024004b40 con 0x7f603415d4e0 2023-11-09T03:53:31.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.853+0000 7f602affd640 1 --2- 172.21.15.76:0/4009485868 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f601406ae70 0x7f601406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:31.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.853+0000 7f602affd640 1 -- 172.21.15.76:0/4009485868 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f60240898e0 con 0x7f603415d4e0 2023-11-09T03:53:31.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.853+0000 7f603259c640 1 --2- 172.21.15.76:0/4009485868 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f601406ae70 0x7f601406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:31.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.854+0000 7f603259c640 1 --2- 172.21.15.76:0/4009485868 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f601406ae70 0x7f601406d330 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f602c036000 tx=0x7f602c030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:31.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:31.857+0000 7f602affd640 1 -- 172.21.15.76:0/4009485868 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f60340b5a70 con 0x7f603415d4e0 2023-11-09T03:53:32.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:32.059+0000 7f6038ce5640 1 -- 172.21.15.76:0/4009485868 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60340b5a70 con 0x7f603415d4e0 2023-11-09T03:53:32.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:32.060+0000 7f602affd640 1 -- 172.21.15.76:0/4009485868 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+4943 (secure 0 0 0) 0x7f60340b5a70 con 0x7f603415d4e0 2023-11-09T03:53:32.060 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:53:32.060 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":115,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:53:25.733437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":8772},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8772":{"gid":8772,"name":"a","rank":1,"incarnation":113,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.76:6835/2075292158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2075292158},{"type":"v1","addr":"172.21.15.76:6835","nonce":2075292158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:53:32.060 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 115 2023-11-09T03:53:32.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:32.061+0000 7f6038ce5640 1 -- 172.21.15.76:0/4009485868 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f601406ae70 msgr2=0x7f601406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:32.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:32.061+0000 7f6038ce5640 1 --2- 172.21.15.76:0/4009485868 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f601406ae70 0x7f601406d330 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f602c036000 tx=0x7f602c030000 comp rx=0 tx=0).stop 2023-11-09T03:53:32.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:32.061+0000 7f6038ce5640 1 -- 172.21.15.76:0/4009485868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f603415d4e0 msgr2=0x7f60340b0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:32.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:32.061+0000 7f6038ce5640 1 --2- 172.21.15.76:0/4009485868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f603415d4e0 0x7f60340b0900 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f6024001eb0 tx=0x7f602400c820 comp rx=0 tx=0).stop 2023-11-09T03:53:32.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:32.061+0000 7f6038ce5640 1 -- 172.21.15.76:0/4009485868 shutdown_connections 2023-11-09T03:53:32.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:32.061+0000 7f6038ce5640 1 --2- 172.21.15.76:0/4009485868 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f601406ae70 0x7f601406d330 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:32.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:32.061+0000 7f6038ce5640 1 --2- 172.21.15.76:0/4009485868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f603415d4e0 0x7f60340b0900 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:32.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:32.061+0000 7f6038ce5640 1 --2- 172.21.15.76:0/4009485868 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f603415d100 0x7f60340b03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:32.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:32.061+0000 7f6038ce5640 1 --2- 172.21.15.76:0/4009485868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60340d3b60 0x7f603415b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:32.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:32.061+0000 7f6038ce5640 1 -- 172.21.15.76:0/4009485868 >> 172.21.15.76:0/4009485868 conn(0x7f60340d2ef0 msgr2=0x7f603406bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:32.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:32.062+0000 7f6038ce5640 1 -- 172.21.15.76:0/4009485868 shutdown_connections 2023-11-09T03:53:32.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:32.062+0000 7f6038ce5640 1 -- 172.21.15.76:0/4009485868 wait complete. 2023-11-09T03:53:32.072 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:reconnect 2023-11-09T03:53:32.270 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:53:32.273 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:53:33.073 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:53:33.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.181+0000 7f4d9cf73640 1 Processor -- start 2023-11-09T03:53:33.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.181+0000 7f4d9cf73640 1 -- start start 2023-11-09T03:53:33.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.182+0000 7f4d9cf73640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d9806c5f0 0x7f4d98165ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:33.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.182+0000 7f4d9cf73640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d9806cc90 con 0x7f4d9806d9c0 2023-11-09T03:53:33.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.182+0000 7f4d9cf73640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d980b0e60 con 0x7f4d980ade00 2023-11-09T03:53:33.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.182+0000 7f4d9cf73640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d980b0c40 con 0x7f4d9806c5f0 2023-11-09T03:53:33.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.182+0000 7f4d8ffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f4d9806d9c0 0x7f4d980710d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36076/0 (socket says 172.21.15.76:36076) 2023-11-09T03:53:33.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.182+0000 7f4d8ffff640 1 -- 172.21.15.76:0/1208324975 learned_addr learned my addr 172.21.15.76:0/1208324975 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:53:33.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.182+0000 7f4d96d9d640 1 --2- 172.21.15.76:0/1208324975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d9806c5f0 0x7f4d98165ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:33.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.182+0000 7f4d95d9b640 1 -- 172.21.15.76:0/1208324975 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1185330891 0 0) 0x7f4d9806cc90 con 0x7f4d9806d9c0 2023-11-09T03:53:33.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.182+0000 7f4d95d9b640 1 -- 172.21.15.76:0/1208324975 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d84003520 con 0x7f4d9806d9c0 2023-11-09T03:53:33.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.182+0000 7f4d95d9b640 1 -- 172.21.15.76:0/1208324975 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199428238 0 0) 0x7f4d980b0e60 con 0x7f4d980ade00 2023-11-09T03:53:33.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.182+0000 7f4d95d9b640 1 -- 172.21.15.76:0/1208324975 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d9806cc90 con 0x7f4d980ade00 2023-11-09T03:53:33.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.182+0000 7f4d95d9b640 1 -- 172.21.15.76:0/1208324975 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1418272877 0 0) 0x7f4d84003520 con 0x7f4d9806d9c0 2023-11-09T03:53:33.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.182+0000 7f4d95d9b640 1 -- 172.21.15.76:0/1208324975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d9806c5f0 msgr2=0x7f4d98165ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:33.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.182+0000 7f4d95d9b640 1 --2- 172.21.15.76:0/1208324975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d9806c5f0 0x7f4d98165ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:33.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.182+0000 7f4d95d9b640 1 -- 172.21.15.76:0/1208324975 >> v1:172.21.15.113:6789/0 conn(0x7f4d980ade00 legacy=0x7f4d980b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:33.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.182+0000 7f4d95d9b640 1 -- 172.21.15.76:0/1208324975 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d9806cee0 con 0x7f4d9806d9c0 2023-11-09T03:53:33.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.182+0000 7f4d95d9b640 1 -- 172.21.15.76:0/1208324975 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f4d80001670 con 0x7f4d9806d9c0 2023-11-09T03:53:33.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.182+0000 7f4d95d9b640 1 -- 172.21.15.76:0/1208324975 >> v1:172.21.15.76:6789/0 conn(0x7f4d9806d9c0 legacy=0x7f4d980710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:33.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.182+0000 7f4d95d9b640 1 --2- 172.21.15.76:0/1208324975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d84003b30 0x7f4d84003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:33.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.182+0000 7f4d95d9b640 1 --2- 172.21.15.76:0/1208324975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d9806c5f0 0x7f4d840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:33.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.182+0000 7f4d95d9b640 1 --2- 172.21.15.76:0/1208324975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d84004d00 0x7f4d84005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:33.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.182+0000 7f4d95d9b640 1 -- 172.21.15.76:0/1208324975 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d980b0c40 con 0x7f4d84003b30 2023-11-09T03:53:33.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.182+0000 7f4d95d9b640 1 -- 172.21.15.76:0/1208324975 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8400bb70 con 0x7f4d84004d00 2023-11-09T03:53:33.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.182+0000 7f4d95d9b640 1 -- 172.21.15.76:0/1208324975 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8400bd30 con 0x7f4d9806c5f0 2023-11-09T03:53:33.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.183+0000 7f4d9659c640 1 --2- 172.21.15.76:0/1208324975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d84003b30 0x7f4d84003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:33.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.183+0000 7f4d96d9d640 1 --2- 172.21.15.76:0/1208324975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d84004d00 0x7f4d84005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:33.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.183+0000 7f4d8ffff640 1 --2- 172.21.15.76:0/1208324975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d9806c5f0 0x7f4d840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:33.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.183+0000 7f4d9659c640 1 -- 172.21.15.76:0/1208324975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d9806c5f0 msgr2=0x7f4d840047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:33.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.183+0000 7f4d9659c640 1 --2- 172.21.15.76:0/1208324975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d9806c5f0 0x7f4d840047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:33.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.183+0000 7f4d9659c640 1 -- 172.21.15.76:0/1208324975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d84004d00 msgr2=0x7f4d84005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:33.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.183+0000 7f4d9659c640 1 --2- 172.21.15.76:0/1208324975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d84004d00 0x7f4d84005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:33.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.183+0000 7f4d9659c640 1 -- 172.21.15.76:0/1208324975 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d84003520 con 0x7f4d84003b30 2023-11-09T03:53:33.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.183+0000 7f4d8ffff640 1 --2- 172.21.15.76:0/1208324975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d9806c5f0 0x7f4d840047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:53:33.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.183+0000 7f4d9659c640 1 --2- 172.21.15.76:0/1208324975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d84003b30 0x7f4d84003f30 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f4d880018d0 tx=0x7f4d840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc45b790b921d9b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:33.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.184+0000 7f4d95d9b640 1 -- 172.21.15.76:0/1208324975 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d880052d0 con 0x7f4d84003b30 2023-11-09T03:53:33.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.184+0000 7f4d95d9b640 1 -- 172.21.15.76:0/1208324975 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d88005430 con 0x7f4d84003b30 2023-11-09T03:53:33.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.184+0000 7f4d95d9b640 1 -- 172.21.15.76:0/1208324975 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d88003810 con 0x7f4d84003b30 2023-11-09T03:53:33.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.184+0000 7f4d9cf73640 1 -- 172.21.15.76:0/1208324975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d84003b30 msgr2=0x7f4d84003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:33.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.184+0000 7f4d9cf73640 1 --2- 172.21.15.76:0/1208324975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d84003b30 0x7f4d84003f30 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f4d880018d0 tx=0x7f4d840057f0 comp rx=0 tx=0).stop 2023-11-09T03:53:33.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.184+0000 7f4d96d9d640 1 -- 172.21.15.76:0/1208324975 reap_dead start 2023-11-09T03:53:33.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.184+0000 7f4d9cf73640 1 -- 172.21.15.76:0/1208324975 shutdown_connections 2023-11-09T03:53:33.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.184+0000 7f4d9cf73640 1 -- 172.21.15.76:0/1208324975 >> 172.21.15.76:0/1208324975 conn(0x7f4d980cb530 msgr2=0x7f4d980cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:33.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.184+0000 7f4d9cf73640 1 -- 172.21.15.76:0/1208324975 shutdown_connections 2023-11-09T03:53:33.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.184+0000 7f4d9cf73640 1 -- 172.21.15.76:0/1208324975 wait complete. 2023-11-09T03:53:33.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.186+0000 7f4d9cf73640 1 Processor -- start 2023-11-09T03:53:33.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.186+0000 7f4d9cf73640 1 -- start start 2023-11-09T03:53:33.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.186+0000 7f4d9cf73640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d84003b30 0x7f4d980ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:33.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.186+0000 7f4d9cf73640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d981f1bb0 0x7f4d980af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:33.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.186+0000 7f4d9cf73640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d981ee570 0x7f4d980c6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:33.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.186+0000 7f4d9cf73640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d980c5b80 con 0x7f4d84003b30 2023-11-09T03:53:33.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.186+0000 7f4d9cf73640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d981f2120 con 0x7f4d981f1bb0 2023-11-09T03:53:33.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.186+0000 7f4d9cf73640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d981f2400 con 0x7f4d981ee570 2023-11-09T03:53:33.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.186+0000 7f4d9659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d84003b30 0x7f4d980ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:33.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.186+0000 7f4d9659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d84003b30 0x7f4d980ca2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37640/0 (socket says 172.21.15.76:37640) 2023-11-09T03:53:33.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.186+0000 7f4d9659c640 1 -- 172.21.15.76:0/3307060032 learned_addr learned my addr 172.21.15.76:0/3307060032 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:33.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.187+0000 7f4d8ffff640 1 --2- 172.21.15.76:0/3307060032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d981f1bb0 0x7f4d980af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:33.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.187+0000 7f4d96d9d640 1 --2- 172.21.15.76:0/3307060032 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d981ee570 0x7f4d980c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:33.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.187+0000 7f4d9659c640 1 -- 172.21.15.76:0/3307060032 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d981ee570 msgr2=0x7f4d980c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:33.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.187+0000 7f4d9659c640 1 --2- 172.21.15.76:0/3307060032 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d981ee570 0x7f4d980c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:33.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.187+0000 7f4d9659c640 1 -- 172.21.15.76:0/3307060032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d981f1bb0 msgr2=0x7f4d980af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:33.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.187+0000 7f4d9659c640 1 --2- 172.21.15.76:0/3307060032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d981f1bb0 0x7f4d980af4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:33.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.187+0000 7f4d9659c640 1 -- 172.21.15.76:0/3307060032 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d981f26e0 con 0x7f4d84003b30 2023-11-09T03:53:33.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.187+0000 7f4d9659c640 1 --2- 172.21.15.76:0/3307060032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d84003b30 0x7f4d980ca2a0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f4d88005040 tx=0x7f4d880012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:33.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.187+0000 7f4d8effd640 1 -- 172.21.15.76:0/3307060032 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d88007250 con 0x7f4d84003b30 2023-11-09T03:53:33.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.187+0000 7f4d8effd640 1 -- 172.21.15.76:0/3307060032 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d880073b0 con 0x7f4d84003b30 2023-11-09T03:53:33.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.187+0000 7f4d9cf73640 1 -- 172.21.15.76:0/3307060032 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d981f29c0 con 0x7f4d84003b30 2023-11-09T03:53:33.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.187+0000 7f4d8effd640 1 -- 172.21.15.76:0/3307060032 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d880035f0 con 0x7f4d84003b30 2023-11-09T03:53:33.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.187+0000 7f4d9cf73640 1 -- 172.21.15.76:0/3307060032 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d980af9f0 con 0x7f4d84003b30 2023-11-09T03:53:33.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.188+0000 7f4d9cf73640 1 -- 172.21.15.76:0/3307060032 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d980c5cf0 con 0x7f4d84003b30 2023-11-09T03:53:33.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.188+0000 7f4d8effd640 1 -- 172.21.15.76:0/3307060032 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4d88003750 con 0x7f4d84003b30 2023-11-09T03:53:33.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.189+0000 7f4d8effd640 1 --2- 172.21.15.76:0/3307060032 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d7806aea0 0x7f4d7806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:33.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.189+0000 7f4d8effd640 1 -- 172.21.15.76:0/3307060032 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f4d880abaa0 con 0x7f4d84003b30 2023-11-09T03:53:33.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.189+0000 7f4d8ffff640 1 --2- 172.21.15.76:0/3307060032 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d7806aea0 0x7f4d7806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:33.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.189+0000 7f4d8ffff640 1 --2- 172.21.15.76:0/3307060032 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d7806aea0 0x7f4d7806d360 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4d80001200 tx=0x7f4d80009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:33.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.193+0000 7f4d8effd640 1 -- 172.21.15.76:0/3307060032 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4d980c5cf0 con 0x7f4d84003b30 2023-11-09T03:53:33.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.394+0000 7f4d9cf73640 1 -- 172.21.15.76:0/3307060032 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d980c5cc0 con 0x7f4d84003b30 2023-11-09T03:53:33.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.395+0000 7f4d8effd640 1 -- 172.21.15.76:0/3307060032 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+4943 (secure 0 0 0) 0x7f4d88037090 con 0x7f4d84003b30 2023-11-09T03:53:33.395 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:53:33.395 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":115,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:53:25.733437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":8772},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8772":{"gid":8772,"name":"a","rank":1,"incarnation":113,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.76:6835/2075292158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2075292158},{"type":"v1","addr":"172.21.15.76:6835","nonce":2075292158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:53:33.395 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 115 2023-11-09T03:53:33.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.397+0000 7f4d9cf73640 1 -- 172.21.15.76:0/3307060032 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d7806aea0 msgr2=0x7f4d7806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:33.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.397+0000 7f4d9cf73640 1 --2- 172.21.15.76:0/3307060032 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d7806aea0 0x7f4d7806d360 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4d80001200 tx=0x7f4d80009040 comp rx=0 tx=0).stop 2023-11-09T03:53:33.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.397+0000 7f4d9cf73640 1 -- 172.21.15.76:0/3307060032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d84003b30 msgr2=0x7f4d980ca2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:33.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.397+0000 7f4d9cf73640 1 --2- 172.21.15.76:0/3307060032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d84003b30 0x7f4d980ca2a0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f4d88005040 tx=0x7f4d880012d0 comp rx=0 tx=0).stop 2023-11-09T03:53:33.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.397+0000 7f4d9cf73640 1 -- 172.21.15.76:0/3307060032 shutdown_connections 2023-11-09T03:53:33.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.397+0000 7f4d9cf73640 1 --2- 172.21.15.76:0/3307060032 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d7806aea0 0x7f4d7806d360 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:33.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.397+0000 7f4d9cf73640 1 --2- 172.21.15.76:0/3307060032 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d981ee570 0x7f4d980c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:33.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.397+0000 7f4d9cf73640 1 --2- 172.21.15.76:0/3307060032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d981f1bb0 0x7f4d980af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:33.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.397+0000 7f4d9cf73640 1 --2- 172.21.15.76:0/3307060032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d84003b30 0x7f4d980ca2a0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:33.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.397+0000 7f4d9cf73640 1 -- 172.21.15.76:0/3307060032 >> 172.21.15.76:0/3307060032 conn(0x7f4d980cb530 msgr2=0x7f4d980ae510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:33.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.397+0000 7f4d9cf73640 1 -- 172.21.15.76:0/3307060032 shutdown_connections 2023-11-09T03:53:33.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:33.397+0000 7f4d9cf73640 1 -- 172.21.15.76:0/3307060032 wait complete. 2023-11-09T03:53:33.407 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:reconnect 2023-11-09T03:53:34.409 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:53:34.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.513+0000 7f1937ac9640 1 Processor -- start 2023-11-09T03:53:34.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.513+0000 7f1937ac9640 1 -- start start 2023-11-09T03:53:34.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.514+0000 7f1937ac9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1930070700 0x7f1930164b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:34.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.514+0000 7f1937ac9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1930071500 con 0x7f19300b0c40 2023-11-09T03:53:34.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.514+0000 7f1937ac9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f193015dae0 con 0x7f1930159e40 2023-11-09T03:53:34.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.514+0000 7f1937ac9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1930070c60 con 0x7f1930070700 2023-11-09T03:53:34.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.514+0000 7f1935064640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f19300b0c40 0x7f193006fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36088/0 (socket says 172.21.15.76:36088) 2023-11-09T03:53:34.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.514+0000 7f1935064640 1 -- 172.21.15.76:0/1013421527 learned_addr learned my addr 172.21.15.76:0/1013421527 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:53:34.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.514+0000 7f1934863640 1 -- 172.21.15.76:0/1013421527 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 572194797 0 0) 0x7f1930071500 con 0x7f19300b0c40 2023-11-09T03:53:34.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.514+0000 7f1936066640 1 --2- 172.21.15.76:0/1013421527 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1930070700 0x7f1930164b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:34.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.514+0000 7f1934863640 1 -- 172.21.15.76:0/1013421527 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f191c005730 con 0x7f19300b0c40 2023-11-09T03:53:34.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.514+0000 7f1934863640 1 -- 172.21.15.76:0/1013421527 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3899811579 0 0) 0x7f191c005730 con 0x7f19300b0c40 2023-11-09T03:53:34.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.514+0000 7f1934863640 1 -- 172.21.15.76:0/1013421527 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1930070700 msgr2=0x7f1930164b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:34.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.514+0000 7f1934863640 1 --2- 172.21.15.76:0/1013421527 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1930070700 0x7f1930164b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:34.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.514+0000 7f1934863640 1 -- 172.21.15.76:0/1013421527 >> v1:172.21.15.113:6789/0 conn(0x7f1930159e40 legacy=0x7f19300b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:34.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.514+0000 7f1934863640 1 -- 172.21.15.76:0/1013421527 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1930070f40 con 0x7f19300b0c40 2023-11-09T03:53:34.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.514+0000 7f1934863640 1 -- 172.21.15.76:0/1013421527 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f1920001670 con 0x7f19300b0c40 2023-11-09T03:53:34.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.514+0000 7f1934863640 1 -- 172.21.15.76:0/1013421527 >> v1:172.21.15.76:6789/0 conn(0x7f19300b0c40 legacy=0x7f193006fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:34.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.514+0000 7f1934863640 1 --2- 172.21.15.76:0/1013421527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f191c005bb0 0x7f191c005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:34.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.514+0000 7f1934863640 1 --2- 172.21.15.76:0/1013421527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f191c0069d0 0x7f191c006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:34.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.514+0000 7f1934863640 1 --2- 172.21.15.76:0/1013421527 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1930070700 0x7f191c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:34.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.514+0000 7f1934863640 1 -- 172.21.15.76:0/1013421527 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1930070c60 con 0x7f191c0069d0 2023-11-09T03:53:34.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.514+0000 7f1934863640 1 -- 172.21.15.76:0/1013421527 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191c00dcc0 con 0x7f191c005bb0 2023-11-09T03:53:34.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.514+0000 7f1934863640 1 -- 172.21.15.76:0/1013421527 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f191c00de80 con 0x7f1930070700 2023-11-09T03:53:34.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.515+0000 7f1935064640 1 --2- 172.21.15.76:0/1013421527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f191c0069d0 0x7f191c006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:34.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.515+0000 7f1935865640 1 --2- 172.21.15.76:0/1013421527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f191c005bb0 0x7f191c005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:34.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.515+0000 7f1936066640 1 --2- 172.21.15.76:0/1013421527 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1930070700 0x7f191c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:34.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.515+0000 7f1935064640 1 -- 172.21.15.76:0/1013421527 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1930070700 msgr2=0x7f191c0073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:34.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.515+0000 7f1935064640 1 --2- 172.21.15.76:0/1013421527 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1930070700 0x7f191c0073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:34.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.515+0000 7f1935064640 1 -- 172.21.15.76:0/1013421527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f191c005bb0 msgr2=0x7f191c005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:34.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.515+0000 7f1935064640 1 --2- 172.21.15.76:0/1013421527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f191c005bb0 0x7f191c005fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:34.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.515+0000 7f1935064640 1 -- 172.21.15.76:0/1013421527 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191c005730 con 0x7f191c0069d0 2023-11-09T03:53:34.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.515+0000 7f1935064640 1 --2- 172.21.15.76:0/1013421527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f191c0069d0 0x7f191c006e50 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f192000c240 tx=0x7f191c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e021e58225fdd9 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:34.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.516+0000 7f1934863640 1 -- 172.21.15.76:0/1013421527 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920005ad0 con 0x7f191c0069d0 2023-11-09T03:53:34.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.516+0000 7f1934863640 1 -- 172.21.15.76:0/1013421527 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1920005c30 con 0x7f191c0069d0 2023-11-09T03:53:34.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.516+0000 7f1934863640 1 -- 172.21.15.76:0/1013421527 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192002c8b0 con 0x7f191c0069d0 2023-11-09T03:53:34.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.516+0000 7f1937ac9640 1 -- 172.21.15.76:0/1013421527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f191c0069d0 msgr2=0x7f191c006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:34.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.516+0000 7f1937ac9640 1 --2- 172.21.15.76:0/1013421527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f191c0069d0 0x7f191c006e50 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f192000c240 tx=0x7f191c007a00 comp rx=0 tx=0).stop 2023-11-09T03:53:34.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.516+0000 7f1936066640 1 -- 172.21.15.76:0/1013421527 reap_dead start 2023-11-09T03:53:34.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.516+0000 7f1937ac9640 1 -- 172.21.15.76:0/1013421527 shutdown_connections 2023-11-09T03:53:34.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.516+0000 7f1937ac9640 1 -- 172.21.15.76:0/1013421527 >> 172.21.15.76:0/1013421527 conn(0x7f19300cb530 msgr2=0x7f19300cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:34.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.516+0000 7f1937ac9640 1 -- 172.21.15.76:0/1013421527 shutdown_connections 2023-11-09T03:53:34.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.516+0000 7f1937ac9640 1 -- 172.21.15.76:0/1013421527 wait complete. 2023-11-09T03:53:34.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.517+0000 7f1937ac9640 1 Processor -- start 2023-11-09T03:53:34.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.517+0000 7f1937ac9640 1 -- start start 2023-11-09T03:53:34.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.518+0000 7f1937ac9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f191c0069d0 0x7f19301f3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:34.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.518+0000 7f1937ac9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19300acb40 0x7f19301f2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:34.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.518+0000 7f1937ac9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19300acfc0 0x7f19301f3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:34.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.518+0000 7f1937ac9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1930155760 con 0x7f19300acb40 2023-11-09T03:53:34.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.518+0000 7f1937ac9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19300c7780 con 0x7f19300acfc0 2023-11-09T03:53:34.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.518+0000 7f1937ac9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1930155150 con 0x7f191c0069d0 2023-11-09T03:53:34.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.518+0000 7f1935064640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19300acb40 0x7f19301f2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:34.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.518+0000 7f1935064640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19300acb40 0x7f19301f2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37648/0 (socket says 172.21.15.76:37648) 2023-11-09T03:53:34.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.518+0000 7f1935064640 1 -- 172.21.15.76:0/2349642902 learned_addr learned my addr 172.21.15.76:0/2349642902 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:34.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.518+0000 7f1936066640 1 --2- 172.21.15.76:0/2349642902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19300acfc0 0x7f19301f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:34.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.518+0000 7f1935865640 1 --2- 172.21.15.76:0/2349642902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f191c0069d0 0x7f19301f3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:34.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.518+0000 7f1935064640 1 -- 172.21.15.76:0/2349642902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f191c0069d0 msgr2=0x7f19301f3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:34.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.518+0000 7f1935064640 1 --2- 172.21.15.76:0/2349642902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f191c0069d0 0x7f19301f3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:34.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.519+0000 7f1935064640 1 -- 172.21.15.76:0/2349642902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19300acfc0 msgr2=0x7f19301f3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:34.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.519+0000 7f1935064640 1 --2- 172.21.15.76:0/2349642902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19300acfc0 0x7f19301f3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:34.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.519+0000 7f1935064640 1 -- 172.21.15.76:0/2349642902 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19301f3680 con 0x7f19300acb40 2023-11-09T03:53:34.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.519+0000 7f1935865640 1 --2- 172.21.15.76:0/2349642902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f191c0069d0 0x7f19301f3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:53:34.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.519+0000 7f1935064640 1 --2- 172.21.15.76:0/2349642902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19300acb40 0x7f19301f2b50 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f192000c240 tx=0x7f1920001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:34.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.519+0000 7f19267fc640 1 -- 172.21.15.76:0/2349642902 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920039070 con 0x7f19300acb40 2023-11-09T03:53:34.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.519+0000 7f19267fc640 1 -- 172.21.15.76:0/2349642902 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192002ccc0 con 0x7f19300acb40 2023-11-09T03:53:34.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.519+0000 7f19267fc640 1 -- 172.21.15.76:0/2349642902 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192003c040 con 0x7f19300acb40 2023-11-09T03:53:34.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.519+0000 7f1937ac9640 1 -- 172.21.15.76:0/2349642902 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19301f3960 con 0x7f19300acb40 2023-11-09T03:53:34.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.519+0000 7f1937ac9640 1 -- 172.21.15.76:0/2349642902 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19300c67a0 con 0x7f19300acb40 2023-11-09T03:53:34.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.520+0000 7f1937ac9640 1 -- 172.21.15.76:0/2349642902 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19301559d0 con 0x7f19300acb40 2023-11-09T03:53:34.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.520+0000 7f19267fc640 1 -- 172.21.15.76:0/2349642902 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1920032070 con 0x7f19300acb40 2023-11-09T03:53:34.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.521+0000 7f19267fc640 1 --2- 172.21.15.76:0/2349642902 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f191406ae70 0x7f191406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:34.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.521+0000 7f19267fc640 1 -- 172.21.15.76:0/2349642902 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8167+0+0 (secure 0 0 0) 0x7f19200ab140 con 0x7f19300acb40 2023-11-09T03:53:34.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.521+0000 7f1935865640 1 --2- 172.21.15.76:0/2349642902 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f191406ae70 0x7f191406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:34.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.521+0000 7f1935865640 1 --2- 172.21.15.76:0/2349642902 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f191406ae70 0x7f191406d330 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f19300ad4e0 tx=0x7f192c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:34.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.525+0000 7f19267fc640 1 -- 172.21.15.76:0/2349642902 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f19200758e0 con 0x7f19300acb40 2023-11-09T03:53:34.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.724+0000 7f1937ac9640 1 -- 172.21.15.76:0/2349642902 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19301f3b50 con 0x7f19300acb40 2023-11-09T03:53:34.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.724+0000 7f19267fc640 1 -- 172.21.15.76:0/2349642902 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+4937 (secure 0 0 0) 0x7f19301f3b50 con 0x7f19300acb40 2023-11-09T03:53:34.724 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:53:34.725 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":116,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:53:33.495574+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":8772},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:rejoin","state_seq":17,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8772":{"gid":8772,"name":"a","rank":1,"incarnation":113,"state":"up:rejoin","state_seq":6,"addr":"172.21.15.76:6835/2075292158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2075292158},{"type":"v1","addr":"172.21.15.76:6835","nonce":2075292158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:53:34.725 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 116 2023-11-09T03:53:34.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.726+0000 7f1937ac9640 1 -- 172.21.15.76:0/2349642902 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f191406ae70 msgr2=0x7f191406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:34.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.726+0000 7f1937ac9640 1 --2- 172.21.15.76:0/2349642902 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f191406ae70 0x7f191406d330 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f19300ad4e0 tx=0x7f192c009040 comp rx=0 tx=0).stop 2023-11-09T03:53:34.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.726+0000 7f1937ac9640 1 -- 172.21.15.76:0/2349642902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19300acb40 msgr2=0x7f19301f2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:34.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.726+0000 7f1937ac9640 1 --2- 172.21.15.76:0/2349642902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19300acb40 0x7f19301f2b50 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f192000c240 tx=0x7f1920001a00 comp rx=0 tx=0).stop 2023-11-09T03:53:34.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.726+0000 7f1937ac9640 1 -- 172.21.15.76:0/2349642902 shutdown_connections 2023-11-09T03:53:34.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.726+0000 7f1937ac9640 1 --2- 172.21.15.76:0/2349642902 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f191406ae70 0x7f191406d330 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:34.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.726+0000 7f1937ac9640 1 --2- 172.21.15.76:0/2349642902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19300acfc0 0x7f19301f3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:34.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.726+0000 7f1937ac9640 1 --2- 172.21.15.76:0/2349642902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19300acb40 0x7f19301f2b50 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:34.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.726+0000 7f1937ac9640 1 --2- 172.21.15.76:0/2349642902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f191c0069d0 0x7f19301f3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:34.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.726+0000 7f1937ac9640 1 -- 172.21.15.76:0/2349642902 >> 172.21.15.76:0/2349642902 conn(0x7f19300cb530 msgr2=0x7f193015dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:34.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.726+0000 7f1937ac9640 1 -- 172.21.15.76:0/2349642902 shutdown_connections 2023-11-09T03:53:34.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:34.726+0000 7f1937ac9640 1 -- 172.21.15.76:0/2349642902 wait complete. 2023-11-09T03:53:34.736 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:rejoin 2023-11-09T03:53:35.738 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:53:35.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.849+0000 7f6050c83640 1 Processor -- start 2023-11-09T03:53:35.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.849+0000 7f6050c83640 1 -- start start 2023-11-09T03:53:35.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.850+0000 7f6050c83640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f604c0cc680 0x7f604c0cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:35.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.850+0000 7f6050c83640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f604c06c220 0x7f604c0ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:35.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.850+0000 7f6050c83640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f604c0cd720 con 0x7f604c0addd0 2023-11-09T03:53:35.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.850+0000 7f6050c83640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f604c0ae1b0 con 0x7f604c0cc680 2023-11-09T03:53:35.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.850+0000 7f6050c83640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f604c0ccb10 con 0x7f604c06c220 2023-11-09T03:53:35.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.850+0000 7f6049d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f604c0cc680 0x7f604c0cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:35.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.850+0000 7f604a59c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f604c0addd0 0x7f604c06bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36094/0 (socket says 172.21.15.76:36094) 2023-11-09T03:53:35.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.850+0000 7f604a59c640 1 -- 172.21.15.76:0/3919690058 learned_addr learned my addr 172.21.15.76:0/3919690058 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:53:35.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.851+0000 7f6049d9b640 1 -- 172.21.15.76:0/3919690058 >> v2:172.21.15.113:3300/0 conn(0x7f604c06c220 msgr2=0x7f604c0ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:35.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.851+0000 7f6049d9b640 1 --2- 172.21.15.76:0/3919690058 >> v2:172.21.15.113:3300/0 conn(0x7f604c06c220 0x7f604c0ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:35.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.851+0000 7f6049d9b640 1 -- 172.21.15.76:0/3919690058 >> v1:172.21.15.76:6789/0 conn(0x7f604c0addd0 legacy=0x7f604c06bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:35.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.851+0000 7f604959a640 1 -- 172.21.15.76:0/3919690058 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1002277578 0 0) 0x7f604c0cd720 con 0x7f604c0addd0 2023-11-09T03:53:35.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.851+0000 7f6049d9b640 1 -- 172.21.15.76:0/3919690058 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f604c0ccdf0 con 0x7f604c0cc680 2023-11-09T03:53:35.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.851+0000 7f6049d9b640 1 --2- 172.21.15.76:0/3919690058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f604c0cc680 0x7f604c0cdbb0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f604c0ce6d0 tx=0x7f604002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3c18a3bdfcd523 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:35.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.851+0000 7f604959a640 1 -- 172.21.15.76:0/3919690058 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6040031830 con 0x7f604c0cc680 2023-11-09T03:53:35.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.852+0000 7f604959a640 1 -- 172.21.15.76:0/3919690058 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6040031990 con 0x7f604c0cc680 2023-11-09T03:53:35.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.852+0000 7f604959a640 1 -- 172.21.15.76:0/3919690058 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6040031d20 con 0x7f604c0cc680 2023-11-09T03:53:35.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.852+0000 7f6050c83640 1 -- 172.21.15.76:0/3919690058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f604c0cc680 msgr2=0x7f604c0cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:35.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.852+0000 7f6050c83640 1 --2- 172.21.15.76:0/3919690058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f604c0cc680 0x7f604c0cdbb0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f604c0ce6d0 tx=0x7f604002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:53:35.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.852+0000 7f6050c83640 1 -- 172.21.15.76:0/3919690058 shutdown_connections 2023-11-09T03:53:35.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.852+0000 7f6050c83640 1 --2- 172.21.15.76:0/3919690058 >> v2:172.21.15.113:3300/0 conn(0x7f604c06c220 0x7f604c0ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:35.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.852+0000 7f6050c83640 1 --2- 172.21.15.76:0/3919690058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f604c0cc680 0x7f604c0cdbb0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:35.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.852+0000 7f6050c83640 1 -- 172.21.15.76:0/3919690058 >> 172.21.15.76:0/3919690058 conn(0x7f604c0cb530 msgr2=0x7f604c0cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:35.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.852+0000 7f6050c83640 1 -- 172.21.15.76:0/3919690058 shutdown_connections 2023-11-09T03:53:35.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.852+0000 7f6050c83640 1 -- 172.21.15.76:0/3919690058 wait complete. 2023-11-09T03:53:35.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.854+0000 7f6050c83640 1 Processor -- start 2023-11-09T03:53:35.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.854+0000 7f6050c83640 1 -- start start 2023-11-09T03:53:35.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.854+0000 7f6050c83640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f604c06c220 0x7f604c0c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:35.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.854+0000 7f6050c83640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f604c0addd0 0x7f604c0b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:35.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.854+0000 7f6050c83640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f604c151070 0x7f604c0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:35.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.854+0000 7f6050c83640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c0c86a0 con 0x7f604c0addd0 2023-11-09T03:53:35.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.854+0000 7f6050c83640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c151520 con 0x7f604c151070 2023-11-09T03:53:35.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.854+0000 7f6050c83640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f604c0b3c40 con 0x7f604c06c220 2023-11-09T03:53:35.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.854+0000 7f6049d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f604c0addd0 0x7f604c0b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:35.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.854+0000 7f6049d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f604c0addd0 0x7f604c0b2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37652/0 (socket says 172.21.15.76:37652) 2023-11-09T03:53:35.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.854+0000 7f6049d9b640 1 -- 172.21.15.76:0/337879053 learned_addr learned my addr 172.21.15.76:0/337879053 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:35.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.855+0000 7f604ad9d640 1 --2- 172.21.15.76:0/337879053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f604c151070 0x7f604c0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:35.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.855+0000 7f604a59c640 1 --2- 172.21.15.76:0/337879053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f604c06c220 0x7f604c0c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:35.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.855+0000 7f6049d9b640 1 -- 172.21.15.76:0/337879053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f604c06c220 msgr2=0x7f604c0c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:35.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.855+0000 7f6049d9b640 1 --2- 172.21.15.76:0/337879053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f604c06c220 0x7f604c0c6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:35.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.855+0000 7f6049d9b640 1 -- 172.21.15.76:0/337879053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f604c151070 msgr2=0x7f604c0ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:35.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.855+0000 7f6049d9b640 1 --2- 172.21.15.76:0/337879053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f604c151070 0x7f604c0ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:35.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.855+0000 7f6049d9b640 1 -- 172.21.15.76:0/337879053 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6040009400 con 0x7f604c0addd0 2023-11-09T03:53:35.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.855+0000 7f6049d9b640 1 --2- 172.21.15.76:0/337879053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f604c0addd0 0x7f604c0b2ca0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f6040000c00 tx=0x7f60400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:35.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.855+0000 7f603affd640 1 -- 172.21.15.76:0/337879053 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604003aa60 con 0x7f604c0addd0 2023-11-09T03:53:35.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.855+0000 7f603affd640 1 -- 172.21.15.76:0/337879053 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f604003abe0 con 0x7f604c0addd0 2023-11-09T03:53:35.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.855+0000 7f6050c83640 1 -- 172.21.15.76:0/337879053 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f604c0b33a0 con 0x7f604c0addd0 2023-11-09T03:53:35.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.855+0000 7f603affd640 1 -- 172.21.15.76:0/337879053 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60400443f0 con 0x7f604c0addd0 2023-11-09T03:53:35.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.855+0000 7f6050c83640 1 -- 172.21.15.76:0/337879053 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f604c0b3960 con 0x7f604c0addd0 2023-11-09T03:53:35.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.856+0000 7f6050c83640 1 -- 172.21.15.76:0/337879053 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f604c0684b0 con 0x7f604c0addd0 2023-11-09T03:53:35.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.857+0000 7f603affd640 1 -- 172.21.15.76:0/337879053 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6040044550 con 0x7f604c0addd0 2023-11-09T03:53:35.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.857+0000 7f603affd640 1 --2- 172.21.15.76:0/337879053 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f602c06ae70 0x7f602c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:35.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.857+0000 7f603affd640 1 -- 172.21.15.76:0/337879053 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8167+0+0 (secure 0 0 0) 0x7f60400ac220 con 0x7f604c0addd0 2023-11-09T03:53:35.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.857+0000 7f604a59c640 1 --2- 172.21.15.76:0/337879053 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f602c06ae70 0x7f602c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:35.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.858+0000 7f604a59c640 1 --2- 172.21.15.76:0/337879053 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f602c06ae70 0x7f602c06d330 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f603c000f60 tx=0x7f603c007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:35.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:35.861+0000 7f603affd640 1 -- 172.21.15.76:0/337879053 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6040076a60 con 0x7f604c0addd0 2023-11-09T03:53:36.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.060+0000 7f6050c83640 1 -- 172.21.15.76:0/337879053 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f604c0b4a30 con 0x7f604c0addd0 2023-11-09T03:53:36.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.060+0000 7f603affd640 1 -- 172.21.15.76:0/337879053 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+4937 (secure 0 0 0) 0x7f604007a080 con 0x7f604c0addd0 2023-11-09T03:53:36.061 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:53:36.061 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":117,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:53:34.495220+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":8772},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8772":{"gid":8772,"name":"a","rank":1,"incarnation":113,"state":"up:active","state_seq":7,"addr":"172.21.15.76:6835/2075292158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2075292158},{"type":"v1","addr":"172.21.15.76:6835","nonce":2075292158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:53:36.062 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 117 2023-11-09T03:53:36.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.062+0000 7f6050c83640 1 -- 172.21.15.76:0/337879053 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f602c06ae70 msgr2=0x7f602c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:36.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.062+0000 7f6050c83640 1 --2- 172.21.15.76:0/337879053 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f602c06ae70 0x7f602c06d330 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f603c000f60 tx=0x7f603c007b50 comp rx=0 tx=0).stop 2023-11-09T03:53:36.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.062+0000 7f6050c83640 1 -- 172.21.15.76:0/337879053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f604c0addd0 msgr2=0x7f604c0b2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:36.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.062+0000 7f6050c83640 1 --2- 172.21.15.76:0/337879053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f604c0addd0 0x7f604c0b2ca0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f6040000c00 tx=0x7f60400046c0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.062+0000 7f6050c83640 1 -- 172.21.15.76:0/337879053 shutdown_connections 2023-11-09T03:53:36.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.062+0000 7f6050c83640 1 --2- 172.21.15.76:0/337879053 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f602c06ae70 0x7f602c06d330 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.062+0000 7f6050c83640 1 --2- 172.21.15.76:0/337879053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f604c151070 0x7f604c0ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.062+0000 7f6050c83640 1 --2- 172.21.15.76:0/337879053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f604c0addd0 0x7f604c0b2ca0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.062+0000 7f6050c83640 1 --2- 172.21.15.76:0/337879053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f604c06c220 0x7f604c0c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.062+0000 7f6050c83640 1 -- 172.21.15.76:0/337879053 >> 172.21.15.76:0/337879053 conn(0x7f604c0cb530 msgr2=0x7f604c153f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:36.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.062+0000 7f6050c83640 1 -- 172.21.15.76:0/337879053 shutdown_connections 2023-11-09T03:53:36.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.062+0000 7f6050c83640 1 -- 172.21.15.76:0/337879053 wait complete. 2023-11-09T03:53:36.073 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:active 2023-11-09T03:53:36.074 DEBUG:tasks.cephfs.filesystem:reached state 'up:active' in 5.671956539154053s 2023-11-09T03:53:36.074 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:53:36.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:53:36.075 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2023-11-09T03:53:36.102 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:53:36.103 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:53:36.157 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:53:36.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.296+0000 7f12d566f640 1 Processor -- start 2023-11-09T03:53:36.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.296+0000 7f12d566f640 1 -- start start 2023-11-09T03:53:36.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.297+0000 7f12d566f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f12d0159e40 0x7f12d00b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:36.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.297+0000 7f12d566f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f12d0070700 0x7f12d00b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:36.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.297+0000 7f12d566f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d0071500 con 0x7f12d006bb60 2023-11-09T03:53:36.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.297+0000 7f12d566f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d015dae0 con 0x7f12d0159e40 2023-11-09T03:53:36.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.297+0000 7f12d566f640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d0070c60 con 0x7f12d0070700 2023-11-09T03:53:36.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.297+0000 7f12ceffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f12d0159e40 0x7f12d00b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:36.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.297+0000 7f12ceffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f12d0159e40 0x7f12d00b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37654/0 (socket says 172.21.15.76:37654) 2023-11-09T03:53:36.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.297+0000 7f12ceffd640 1 -- 172.21.15.76:0/1334730209 learned_addr learned my addr 172.21.15.76:0/1334730209 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:36.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.297+0000 7f12ceffd640 1 -- 172.21.15.76:0/1334730209 >> v2:172.21.15.113:3300/0 conn(0x7f12d0070700 msgr2=0x7f12d00b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:36.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.297+0000 7f12ceffd640 1 --2- 172.21.15.76:0/1334730209 >> v2:172.21.15.113:3300/0 conn(0x7f12d0070700 0x7f12d00b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.297+0000 7f12ceffd640 1 -- 172.21.15.76:0/1334730209 >> v1:172.21.15.113:6789/0 conn(0x7f12d006bb60 legacy=0x7f12d006fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:36.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.297+0000 7f12ceffd640 1 -- 172.21.15.76:0/1334730209 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d0070f40 con 0x7f12d0159e40 2023-11-09T03:53:36.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.298+0000 7f12ceffd640 1 --2- 172.21.15.76:0/1334730209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12d0159e40 0x7f12d00b0530 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f12d00ce8f0 tx=0x7f12c002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2134a7338ac854bf server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:36.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.298+0000 7f12cdffb640 1 -- 172.21.15.76:0/1334730209 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0031750 con 0x7f12d0159e40 2023-11-09T03:53:36.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.298+0000 7f12cdffb640 1 -- 172.21.15.76:0/1334730209 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c00318b0 con 0x7f12d0159e40 2023-11-09T03:53:36.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.298+0000 7f12cdffb640 1 -- 172.21.15.76:0/1334730209 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0031c40 con 0x7f12d0159e40 2023-11-09T03:53:36.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.299+0000 7f12d566f640 1 -- 172.21.15.76:0/1334730209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12d0159e40 msgr2=0x7f12d00b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:36.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.299+0000 7f12d566f640 1 --2- 172.21.15.76:0/1334730209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12d0159e40 0x7f12d00b0530 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f12d00ce8f0 tx=0x7f12c002b7f0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.299+0000 7f12d566f640 1 -- 172.21.15.76:0/1334730209 shutdown_connections 2023-11-09T03:53:36.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.299+0000 7f12d566f640 1 --2- 172.21.15.76:0/1334730209 >> v2:172.21.15.113:3300/0 conn(0x7f12d0070700 0x7f12d00b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.299+0000 7f12d566f640 1 --2- 172.21.15.76:0/1334730209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12d0159e40 0x7f12d00b0530 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.299+0000 7f12d566f640 1 -- 172.21.15.76:0/1334730209 >> 172.21.15.76:0/1334730209 conn(0x7f12d00cb530 msgr2=0x7f12d00cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:36.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.299+0000 7f12d566f640 1 -- 172.21.15.76:0/1334730209 shutdown_connections 2023-11-09T03:53:36.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.299+0000 7f12d566f640 1 -- 172.21.15.76:0/1334730209 wait complete. 2023-11-09T03:53:36.306 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.300+0000 7f12d566f640 1 Processor -- start 2023-11-09T03:53:36.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.300+0000 7f12d566f640 1 -- start start 2023-11-09T03:53:36.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.301+0000 7f12d566f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12d006bb60 0x7f12d0156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:36.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.301+0000 7f12d566f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12d0070700 0x7f12d00ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:36.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.301+0000 7f12d566f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12d0159e40 0x7f12d00ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:36.307 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.301+0000 7f12d566f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d01f37b0 con 0x7f12d0159e40 2023-11-09T03:53:36.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.301+0000 7f12d566f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d00c8730 con 0x7f12d006bb60 2023-11-09T03:53:36.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.301+0000 7f12d566f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d01f34d0 con 0x7f12d0070700 2023-11-09T03:53:36.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.301+0000 7f12cf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12d0159e40 0x7f12d00ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:36.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.301+0000 7f12cf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12d0159e40 0x7f12d00ace10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37660/0 (socket says 172.21.15.76:37660) 2023-11-09T03:53:36.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.301+0000 7f12cf7fe640 1 -- 172.21.15.76:0/222647661 learned_addr learned my addr 172.21.15.76:0/222647661 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:36.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.301+0000 7f12ceffd640 1 --2- 172.21.15.76:0/222647661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12d006bb60 0x7f12d0156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:36.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.301+0000 7f12ce7fc640 1 --2- 172.21.15.76:0/222647661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12d0070700 0x7f12d00ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:36.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.301+0000 7f12cf7fe640 1 -- 172.21.15.76:0/222647661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12d0070700 msgr2=0x7f12d00ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:36.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.301+0000 7f12cf7fe640 1 --2- 172.21.15.76:0/222647661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12d0070700 0x7f12d00ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.301+0000 7f12cf7fe640 1 -- 172.21.15.76:0/222647661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12d006bb60 msgr2=0x7f12d0156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:36.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.301+0000 7f12cf7fe640 1 --2- 172.21.15.76:0/222647661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12d006bb60 0x7f12d0156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.301+0000 7f12cf7fe640 1 -- 172.21.15.76:0/222647661 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c00094d0 con 0x7f12d0159e40 2023-11-09T03:53:36.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.301+0000 7f12ce7fc640 1 --2- 172.21.15.76:0/222647661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12d0070700 0x7f12d00ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:53:36.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.301+0000 7f12ceffd640 1 --2- 172.21.15.76:0/222647661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12d006bb60 0x7f12d0156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:53:36.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.301+0000 7f12cf7fe640 1 --2- 172.21.15.76:0/222647661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12d0159e40 0x7f12d00ace10 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f12c800b9a0 tx=0x7f12c800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:36.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.302+0000 7f12b77fe640 1 -- 172.21.15.76:0/222647661 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c800cba0 con 0x7f12d0159e40 2023-11-09T03:53:36.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.302+0000 7f12b77fe640 1 -- 172.21.15.76:0/222647661 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c800cd20 con 0x7f12d0159e40 2023-11-09T03:53:36.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.302+0000 7f12d566f640 1 -- 172.21.15.76:0/222647661 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d01f5fa0 con 0x7f12d0159e40 2023-11-09T03:53:36.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.302+0000 7f12b77fe640 1 -- 172.21.15.76:0/222647661 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c8014680 con 0x7f12d0159e40 2023-11-09T03:53:36.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.302+0000 7f12d566f640 1 -- 172.21.15.76:0/222647661 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d01f6560 con 0x7f12d0159e40 2023-11-09T03:53:36.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.303+0000 7f12d566f640 1 -- 172.21.15.76:0/222647661 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d01f6780 con 0x7f12d0159e40 2023-11-09T03:53:36.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.303+0000 7f12b77fe640 1 -- 172.21.15.76:0/222647661 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f12c801d3b0 con 0x7f12d0159e40 2023-11-09T03:53:36.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.303+0000 7f12b77fe640 1 --2- 172.21.15.76:0/222647661 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f12bc06ae70 0x7f12bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:36.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.304+0000 7f12b77fe640 1 -- 172.21.15.76:0/222647661 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8167+0+0 (secure 0 0 0) 0x7f12c8089f30 con 0x7f12d0159e40 2023-11-09T03:53:36.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.304+0000 7f12ceffd640 1 --2- 172.21.15.76:0/222647661 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f12bc06ae70 0x7f12bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:36.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.304+0000 7f12ceffd640 1 --2- 172.21.15.76:0/222647661 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f12bc06ae70 0x7f12bc06d330 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f12c0035000 tx=0x7f12c0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:36.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.308+0000 7f12b77fe640 1 -- 172.21.15.76:0/222647661 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f12c8054650 con 0x7f12d0159e40 2023-11-09T03:53:36.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.475+0000 7f12d566f640 1 -- 172.21.15.76:0/222647661 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12d00684b0 con 0x7f12d0159e40 2023-11-09T03:53:36.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.478+0000 7f12b77fe640 1 -- 172.21.15.76:0/222647661 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+23650 (secure 0 0 0) 0x7f12c8019020 con 0x7f12d0159e40 2023-11-09T03:53:36.479 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:53:36.479 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":112,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:53:33.478377+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-09T03:47:59.280601+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-09T03:47:59.742039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:48:55.490832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-09T03:48:56.352624+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":9,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.17","mappings":[{"from":0,"to":2}]},{"pgid":"17.36","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/334503200":"2023-11-09T04:53:32.572840+0000","172.21.15.113:6837/7360844":"2023-11-10T03:53:24.718008+0000","172.21.15.113:6836/7360844":"2023-11-10T03:53:24.718008+0000","172.21.15.76:6835/546338722":"2023-11-10T03:53:17.470106+0000","172.21.15.76:6834/546338722":"2023-11-10T03:53:17.470106+0000","192.168.0.1:0/3087804343":"2023-11-09T04:52:42.513556+0000","172.21.15.76:6833/1536527256":"2023-11-10T03:52:34.213804+0000","172.21.15.113:6837/445306598":"2023-11-10T03:52:27.406838+0000","172.21.15.76:6834/3229239133":"2023-11-10T03:51:32.624856+0000","192.168.0.1:0/503424885":"2023-11-09T04:51:47.727422+0000","172.21.15.76:6834/4011275120":"2023-11-10T03:49:42.029057+0000","172.21.15.76:6835/4011275120":"2023-11-10T03:49:42.029057+0000","172.21.15.113:6836/3280098776":"2023-11-10T03:50:07.616780+0000","172.21.15.113:6837/3280098776":"2023-11-10T03:50:07.616780+0000","172.21.15.76:6834/1663988060":"2023-11-10T03:50:33.665092+0000","172.21.15.76:6835/1663988060":"2023-11-10T03:50:33.665092+0000","172.21.15.113:6836/445306598":"2023-11-10T03:52:27.406838+0000","172.21.15.76:6833/3697850101":"2023-11-10T03:51:39.442929+0000","172.21.15.113:6836/3449910842":"2023-11-10T03:50:59.611256+0000","172.21.15.76:6835/3229239133":"2023-11-10T03:51:32.624856+0000","172.21.15.76:6832/1536527256":"2023-11-10T03:52:34.213804+0000","172.21.15.113:6837/3449910842":"2023-11-10T03:50:59.611256+0000","172.21.15.76:6832/3697850101":"2023-11-10T03:51:39.442929+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:53:36.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.480+0000 7f12d566f640 1 -- 172.21.15.76:0/222647661 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f12bc06ae70 msgr2=0x7f12bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:36.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.480+0000 7f12d566f640 1 --2- 172.21.15.76:0/222647661 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f12bc06ae70 0x7f12bc06d330 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f12c0035000 tx=0x7f12c0008030 comp rx=0 tx=0).stop 2023-11-09T03:53:36.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.480+0000 7f12d566f640 1 -- 172.21.15.76:0/222647661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12d0159e40 msgr2=0x7f12d00ace10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:36.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.480+0000 7f12d566f640 1 --2- 172.21.15.76:0/222647661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12d0159e40 0x7f12d00ace10 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f12c800b9a0 tx=0x7f12c800be60 comp rx=0 tx=0).stop 2023-11-09T03:53:36.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.480+0000 7f12d566f640 1 -- 172.21.15.76:0/222647661 shutdown_connections 2023-11-09T03:53:36.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.480+0000 7f12d566f640 1 --2- 172.21.15.76:0/222647661 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f12bc06ae70 0x7f12bc06d330 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.480+0000 7f12d566f640 1 --2- 172.21.15.76:0/222647661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12d0159e40 0x7f12d00ace10 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.480+0000 7f12d566f640 1 --2- 172.21.15.76:0/222647661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12d0070700 0x7f12d00ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.480+0000 7f12d566f640 1 --2- 172.21.15.76:0/222647661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12d006bb60 0x7f12d0156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.480+0000 7f12d566f640 1 -- 172.21.15.76:0/222647661 >> 172.21.15.76:0/222647661 conn(0x7f12d00cb530 msgr2=0x7f12d00cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:36.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.480+0000 7f12d566f640 1 -- 172.21.15.76:0/222647661 shutdown_connections 2023-11-09T03:53:36.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.480+0000 7f12d566f640 1 -- 172.21.15.76:0/222647661 wait complete. 2023-11-09T03:53:36.491 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-09T03:47:00.036540+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_data_pool', 'create_time': '2023-11-09T03:47:59.280601+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 70, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-09T03:47:59.742039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:48:55.490832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:48:56.352624+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 9, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-09T03:53:36.492 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:53:36.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.594+0000 7fc2545ee640 1 Processor -- start 2023-11-09T03:53:36.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.594+0000 7fc2545ee640 1 -- start start 2023-11-09T03:53:36.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.595+0000 7fc2545ee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc24c15d110 0x7fc24c06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:36.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.595+0000 7fc2545ee640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc24c0b3ca0 0x7fc24c0ace70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:36.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.595+0000 7fc2545ee640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc24c15d4f0 con 0x7fc24c0ad3b0 2023-11-09T03:53:36.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.595+0000 7fc2545ee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc24c0664f0 con 0x7fc24c15d110 2023-11-09T03:53:36.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.595+0000 7fc2545ee640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c06b9f0 con 0x7fc24c0b3ca0 2023-11-09T03:53:36.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.595+0000 7fc25238a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc24c15d110 0x7fc24c06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:36.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.595+0000 7fc251b89640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc24c0b3ca0 0x7fc24c0ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:36.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.596+0000 7fc25238a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc24c15d110 0x7fc24c06b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:40928/0 (socket says 172.21.15.76:40928) 2023-11-09T03:53:36.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.596+0000 7fc25238a640 1 -- 172.21.15.76:0/250112745 learned_addr learned my addr 172.21.15.76:0/250112745 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:36.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.595+0000 7fc252b8b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc24c0ad3b0 0x7fc24c0b2100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:48104/0 (socket says 172.21.15.76:48104) 2023-11-09T03:53:36.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.596+0000 7fc251b89640 1 --2- 172.21.15.76:0/250112745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc24c0b3ca0 0x7fc24c0ace70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:46736/0 (socket says 172.21.15.76:46736) 2023-11-09T03:53:36.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.596+0000 7fc25238a640 1 -- 172.21.15.76:0/250112745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc24c0b3ca0 msgr2=0x7fc24c0ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:36.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.596+0000 7fc25238a640 1 --2- 172.21.15.76:0/250112745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc24c0b3ca0 0x7fc24c0ace70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.596+0000 7fc25238a640 1 -- 172.21.15.76:0/250112745 >> v1:172.21.15.113:6789/0 conn(0x7fc24c0ad3b0 legacy=0x7fc24c0b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:36.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.596+0000 7fc25238a640 1 -- 172.21.15.76:0/250112745 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24c06c2e0 con 0x7fc24c15d110 2023-11-09T03:53:36.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.596+0000 7fc251388640 1 -- 172.21.15.76:0/250112745 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4136988874 0 0) 0x7fc24c15d4f0 con 0x7fc24c0ad3b0 2023-11-09T03:53:36.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.596+0000 7fc251b89640 1 --2- 172.21.15.76:0/250112745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc24c0b3ca0 0x7fc24c0ace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:53:36.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.597+0000 7fc25238a640 1 --2- 172.21.15.76:0/250112745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc24c15d110 0x7fc24c06b2f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc248009ea0 tx=0x7fc24802bfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a1aad4cb266e8e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:36.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.597+0000 7fc251388640 1 -- 172.21.15.76:0/250112745 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24802cb40 con 0x7fc24c15d110 2023-11-09T03:53:36.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.597+0000 7fc251388640 1 -- 172.21.15.76:0/250112745 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24802cca0 con 0x7fc24c15d110 2023-11-09T03:53:36.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.597+0000 7fc251388640 1 -- 172.21.15.76:0/250112745 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc248034610 con 0x7fc24c15d110 2023-11-09T03:53:36.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.597+0000 7fc2545ee640 1 -- 172.21.15.76:0/250112745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc24c15d110 msgr2=0x7fc24c06b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:36.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.597+0000 7fc2545ee640 1 --2- 172.21.15.76:0/250112745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc24c15d110 0x7fc24c06b2f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc248009ea0 tx=0x7fc24802bfa0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.597+0000 7fc2545ee640 1 -- 172.21.15.76:0/250112745 shutdown_connections 2023-11-09T03:53:36.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.597+0000 7fc2545ee640 1 --2- 172.21.15.76:0/250112745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc24c0b3ca0 0x7fc24c0ace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.597+0000 7fc2545ee640 1 --2- 172.21.15.76:0/250112745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc24c15d110 0x7fc24c06b2f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.597+0000 7fc2545ee640 1 -- 172.21.15.76:0/250112745 >> 172.21.15.76:0/250112745 conn(0x7fc24c0cbd30 msgr2=0x7fc24c0cc140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:36.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.597+0000 7fc2545ee640 1 -- 172.21.15.76:0/250112745 shutdown_connections 2023-11-09T03:53:36.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.598+0000 7fc2545ee640 1 -- 172.21.15.76:0/250112745 wait complete. 2023-11-09T03:53:36.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.599+0000 7fc2545ee640 1 Processor -- start 2023-11-09T03:53:36.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.599+0000 7fc2545ee640 1 -- start start 2023-11-09T03:53:36.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.599+0000 7fc2545ee640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc24c0ad3b0 0x7fc24c1594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:36.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.599+0000 7fc2545ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc24c0b3ca0 0x7fc24c15aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:36.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.599+0000 7fc2545ee640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc24c15d110 0x7fc24c15afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:36.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.599+0000 7fc2545ee640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c09b300 con 0x7fc24c0ad3b0 2023-11-09T03:53:36.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.599+0000 7fc2545ee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c09a6f0 con 0x7fc24c0b3ca0 2023-11-09T03:53:36.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.599+0000 7fc2545ee640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c09a9d0 con 0x7fc24c15d110 2023-11-09T03:53:36.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.599+0000 7fc25238a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc24c0ad3b0 0x7fc24c1594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:36.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.599+0000 7fc25238a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc24c0ad3b0 0x7fc24c1594c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37674/0 (socket says 172.21.15.76:37674) 2023-11-09T03:53:36.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.599+0000 7fc25238a640 1 -- 172.21.15.76:0/1422615796 learned_addr learned my addr 172.21.15.76:0/1422615796 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:36.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.600+0000 7fc251b89640 1 --2- 172.21.15.76:0/1422615796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc24c0b3ca0 0x7fc24c15aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:36.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.600+0000 7fc252b8b640 1 --2- 172.21.15.76:0/1422615796 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc24c15d110 0x7fc24c15afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:36.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.600+0000 7fc25238a640 1 -- 172.21.15.76:0/1422615796 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc24c15d110 msgr2=0x7fc24c15afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:36.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.600+0000 7fc25238a640 1 --2- 172.21.15.76:0/1422615796 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc24c15d110 0x7fc24c15afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.600+0000 7fc25238a640 1 -- 172.21.15.76:0/1422615796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc24c0b3ca0 msgr2=0x7fc24c15aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:36.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.600+0000 7fc25238a640 1 --2- 172.21.15.76:0/1422615796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc24c0b3ca0 0x7fc24c15aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.600+0000 7fc25238a640 1 -- 172.21.15.76:0/1422615796 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc23c009400 con 0x7fc24c0ad3b0 2023-11-09T03:53:36.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.600+0000 7fc252b8b640 1 --2- 172.21.15.76:0/1422615796 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc24c15d110 0x7fc24c15afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:53:36.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.600+0000 7fc25238a640 1 --2- 172.21.15.76:0/1422615796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc24c0ad3b0 0x7fc24c1594c0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fc248036040 tx=0x7fc248004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:36.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.600+0000 7fc242ffd640 1 -- 172.21.15.76:0/1422615796 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc248009510 con 0x7fc24c0ad3b0 2023-11-09T03:53:36.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.600+0000 7fc242ffd640 1 -- 172.21.15.76:0/1422615796 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24803e450 con 0x7fc24c0ad3b0 2023-11-09T03:53:36.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.600+0000 7fc2545ee640 1 -- 172.21.15.76:0/1422615796 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc248009b30 con 0x7fc24c0ad3b0 2023-11-09T03:53:36.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.600+0000 7fc242ffd640 1 -- 172.21.15.76:0/1422615796 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24803e7e0 con 0x7fc24c0ad3b0 2023-11-09T03:53:36.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.600+0000 7fc2545ee640 1 -- 172.21.15.76:0/1422615796 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc24c09af90 con 0x7fc24c0ad3b0 2023-11-09T03:53:36.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.601+0000 7fc2545ee640 1 -- 172.21.15.76:0/1422615796 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc24c0684b0 con 0x7fc24c0ad3b0 2023-11-09T03:53:36.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.601+0000 7fc242ffd640 1 -- 172.21.15.76:0/1422615796 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc24803e940 con 0x7fc24c0ad3b0 2023-11-09T03:53:36.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.602+0000 7fc242ffd640 1 --2- 172.21.15.76:0/1422615796 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc23006ae70 0x7fc23006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:36.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.602+0000 7fc242ffd640 1 -- 172.21.15.76:0/1422615796 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8167+0+0 (secure 0 0 0) 0x7fc2480acd20 con 0x7fc24c0ad3b0 2023-11-09T03:53:36.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.602+0000 7fc251b89640 1 --2- 172.21.15.76:0/1422615796 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc23006ae70 0x7fc23006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:36.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.603+0000 7fc251b89640 1 --2- 172.21.15.76:0/1422615796 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc23006ae70 0x7fc23006d330 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0b1830 tx=0x7fc23c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:36.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.606+0000 7fc242ffd640 1 -- 172.21.15.76:0/1422615796 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc2480773b0 con 0x7fc24c0ad3b0 2023-11-09T03:53:36.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.772+0000 7fc2545ee640 1 -- 172.21.15.76:0/1422615796 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc24c0b1860 con 0x7fc24c0ad3b0 2023-11-09T03:53:36.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.773+0000 7fc242ffd640 1 -- 172.21.15.76:0/1422615796 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc248043020 con 0x7fc24c0ad3b0 2023-11-09T03:53:36.773 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:53:36.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.774+0000 7fc2545ee640 1 -- 172.21.15.76:0/1422615796 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc23006ae70 msgr2=0x7fc23006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:36.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.774+0000 7fc2545ee640 1 --2- 172.21.15.76:0/1422615796 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc23006ae70 0x7fc23006d330 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0b1830 tx=0x7fc23c009190 comp rx=0 tx=0).stop 2023-11-09T03:53:36.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.774+0000 7fc2545ee640 1 -- 172.21.15.76:0/1422615796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc24c0ad3b0 msgr2=0x7fc24c1594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:36.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.774+0000 7fc2545ee640 1 --2- 172.21.15.76:0/1422615796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc24c0ad3b0 0x7fc24c1594c0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fc248036040 tx=0x7fc248004a40 comp rx=0 tx=0).stop 2023-11-09T03:53:36.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.775+0000 7fc2545ee640 1 -- 172.21.15.76:0/1422615796 shutdown_connections 2023-11-09T03:53:36.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.775+0000 7fc2545ee640 1 --2- 172.21.15.76:0/1422615796 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc23006ae70 0x7fc23006d330 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.775+0000 7fc2545ee640 1 --2- 172.21.15.76:0/1422615796 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc24c15d110 0x7fc24c15afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.775+0000 7fc2545ee640 1 --2- 172.21.15.76:0/1422615796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc24c0b3ca0 0x7fc24c15aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.775+0000 7fc2545ee640 1 --2- 172.21.15.76:0/1422615796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc24c0ad3b0 0x7fc24c1594c0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.775+0000 7fc2545ee640 1 -- 172.21.15.76:0/1422615796 >> 172.21.15.76:0/1422615796 conn(0x7fc24c0cbd30 msgr2=0x7fc24c06a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:36.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.775+0000 7fc2545ee640 1 -- 172.21.15.76:0/1422615796 shutdown_connections 2023-11-09T03:53:36.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.775+0000 7fc2545ee640 1 -- 172.21.15.76:0/1422615796 wait complete. 2023-11-09T03:53:36.786 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-09T03:53:36.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.887+0000 7f53b3ae0640 1 Processor -- start 2023-11-09T03:53:36.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.888+0000 7f53b3ae0640 1 -- start start 2023-11-09T03:53:36.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.888+0000 7f53b3ae0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f53ac15d100 0x7f53ac0ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:36.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.888+0000 7f53b3ae0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f53ac15d4e0 0x7f53ac0d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:36.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.888+0000 7f53b3ae0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53ac0d3b60 0x7f53ac0d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:36.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.888+0000 7f53b3ae0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53ac0b32b0 con 0x7f53ac15d4e0 2023-11-09T03:53:36.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.888+0000 7f53b3ae0640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53ac0cee10 con 0x7f53ac15d100 2023-11-09T03:53:36.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.888+0000 7f53b3ae0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac0b2980 con 0x7f53ac0d3b60 2023-11-09T03:53:36.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.889+0000 7f53b107b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f53ac15d4e0 0x7f53ac0d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:36.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.889+0000 7f53b107b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f53ac15d4e0 0x7f53ac0d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37676/0 (socket says 172.21.15.76:37676) 2023-11-09T03:53:36.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.889+0000 7f53b107b640 1 -- 172.21.15.76:0/3818262048 learned_addr learned my addr 172.21.15.76:0/3818262048 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:36.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.889+0000 7f53b187c640 1 --2- 172.21.15.76:0/3818262048 >> v2:172.21.15.113:3300/0 conn(0x7f53ac15d100 0x7f53ac0ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:36.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.889+0000 7f53b207d640 1 --2- 172.21.15.76:0/3818262048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53ac0d3b60 0x7f53ac0d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:36.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.889+0000 7f53b107b640 1 -- 172.21.15.76:0/3818262048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53ac0d3b60 msgr2=0x7f53ac0d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:36.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.889+0000 7f53b107b640 1 --2- 172.21.15.76:0/3818262048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53ac0d3b60 0x7f53ac0d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.889+0000 7f53b107b640 1 -- 172.21.15.76:0/3818262048 >> v2:172.21.15.113:3300/0 conn(0x7f53ac15d100 msgr2=0x7f53ac0ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:36.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.889+0000 7f53b107b640 1 --2- 172.21.15.76:0/3818262048 >> v2:172.21.15.113:3300/0 conn(0x7f53ac15d100 0x7f53ac0ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.889+0000 7f53b107b640 1 -- 172.21.15.76:0/3818262048 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ac0b2c60 con 0x7f53ac15d4e0 2023-11-09T03:53:36.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.889+0000 7f53b207d640 1 --2- 172.21.15.76:0/3818262048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53ac0d3b60 0x7f53ac0d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:53:36.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.890+0000 7f53b107b640 1 --2- 172.21.15.76:0/3818262048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53ac15d4e0 0x7f53ac0d14d0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0d09f0 tx=0x7f539c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adbef1cc6ef04a06 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:36.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.890+0000 7f53b087a640 1 -- 172.21.15.76:0/3818262048 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539c031860 con 0x7f53ac15d4e0 2023-11-09T03:53:36.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.890+0000 7f53b087a640 1 -- 172.21.15.76:0/3818262048 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539c0319c0 con 0x7f53ac15d4e0 2023-11-09T03:53:36.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.890+0000 7f53b087a640 1 -- 172.21.15.76:0/3818262048 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539c031d50 con 0x7f53ac15d4e0 2023-11-09T03:53:36.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.890+0000 7f53b3ae0640 1 -- 172.21.15.76:0/3818262048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53ac15d4e0 msgr2=0x7f53ac0d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:36.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.890+0000 7f53b3ae0640 1 --2- 172.21.15.76:0/3818262048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53ac15d4e0 0x7f53ac0d14d0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0d09f0 tx=0x7f539c02b900 comp rx=0 tx=0).stop 2023-11-09T03:53:36.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.890+0000 7f53b3ae0640 1 -- 172.21.15.76:0/3818262048 shutdown_connections 2023-11-09T03:53:36.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.890+0000 7f53b3ae0640 1 --2- 172.21.15.76:0/3818262048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53ac0d3b60 0x7f53ac0d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.890+0000 7f53b3ae0640 1 --2- 172.21.15.76:0/3818262048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53ac15d4e0 0x7f53ac0d14d0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.890+0000 7f53b3ae0640 1 --2- 172.21.15.76:0/3818262048 >> v2:172.21.15.113:3300/0 conn(0x7f53ac15d100 0x7f53ac0ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.890+0000 7f53b3ae0640 1 -- 172.21.15.76:0/3818262048 >> 172.21.15.76:0/3818262048 conn(0x7f53ac0d2ef0 msgr2=0x7f53ac0d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:36.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.890+0000 7f53b3ae0640 1 -- 172.21.15.76:0/3818262048 shutdown_connections 2023-11-09T03:53:36.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.890+0000 7f53b3ae0640 1 -- 172.21.15.76:0/3818262048 wait complete. 2023-11-09T03:53:36.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.891+0000 7f53b3ae0640 1 Processor -- start 2023-11-09T03:53:36.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.892+0000 7f53b3ae0640 1 -- start start 2023-11-09T03:53:36.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.892+0000 7f53b3ae0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53ac0d3b60 0x7f53ac15e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:36.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.892+0000 7f53b3ae0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53ac15d100 0x7f53ac150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:36.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.892+0000 7f53b3ae0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53ac15d4e0 0x7f53ac06a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:36.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.892+0000 7f53b3ae0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac06b4b0 con 0x7f53ac0d3b60 2023-11-09T03:53:36.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.892+0000 7f53b3ae0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac151700 con 0x7f53ac15d100 2023-11-09T03:53:36.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.892+0000 7f53b3ae0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac15da20 con 0x7f53ac15d4e0 2023-11-09T03:53:36.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.892+0000 7f53b187c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53ac0d3b60 0x7f53ac15e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:36.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.892+0000 7f53b187c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53ac0d3b60 0x7f53ac15e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37680/0 (socket says 172.21.15.76:37680) 2023-11-09T03:53:36.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.892+0000 7f53b187c640 1 -- 172.21.15.76:0/4255638584 learned_addr learned my addr 172.21.15.76:0/4255638584 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:36.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.893+0000 7f53b187c640 1 -- 172.21.15.76:0/4255638584 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53ac15d4e0 msgr2=0x7f53ac06a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:36.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.893+0000 7f53b187c640 1 --2- 172.21.15.76:0/4255638584 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53ac15d4e0 0x7f53ac06a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.893+0000 7f53b187c640 1 -- 172.21.15.76:0/4255638584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53ac15d100 msgr2=0x7f53ac150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:36.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.893+0000 7f53b187c640 1 --2- 172.21.15.76:0/4255638584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53ac15d100 0x7f53ac150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:36.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.893+0000 7f53b187c640 1 -- 172.21.15.76:0/4255638584 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f539c0094d0 con 0x7f53ac0d3b60 2023-11-09T03:53:36.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.893+0000 7f53b187c640 1 --2- 172.21.15.76:0/4255638584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53ac0d3b60 0x7f53ac15e7e0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f53a800efc0 tx=0x7f53a800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:36.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.893+0000 7f53a27fc640 1 -- 172.21.15.76:0/4255638584 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a8014480 con 0x7f53ac0d3b60 2023-11-09T03:53:36.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.893+0000 7f53a27fc640 1 -- 172.21.15.76:0/4255638584 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a800f040 con 0x7f53ac0d3b60 2023-11-09T03:53:36.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.893+0000 7f53b3ae0640 1 -- 172.21.15.76:0/4255638584 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ac15dd00 con 0x7f53ac0d3b60 2023-11-09T03:53:36.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.893+0000 7f53a27fc640 1 -- 172.21.15.76:0/4255638584 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a8014800 con 0x7f53ac0d3b60 2023-11-09T03:53:36.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.893+0000 7f53b3ae0640 1 -- 172.21.15.76:0/4255638584 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ac15e2c0 con 0x7f53ac0d3b60 2023-11-09T03:53:36.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.894+0000 7f53b3ae0640 1 -- 172.21.15.76:0/4255638584 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53ac15e450 con 0x7f53ac0d3b60 2023-11-09T03:53:36.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.894+0000 7f53a27fc640 1 -- 172.21.15.76:0/4255638584 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f53a8014960 con 0x7f53ac0d3b60 2023-11-09T03:53:36.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.895+0000 7f53a27fc640 1 --2- 172.21.15.76:0/4255638584 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f539006ae70 0x7f539006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:36.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.895+0000 7f53a27fc640 1 -- 172.21.15.76:0/4255638584 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8167+0+0 (secure 0 0 0) 0x7f53a808abd0 con 0x7f53ac0d3b60 2023-11-09T03:53:36.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.895+0000 7f53b107b640 1 --2- 172.21.15.76:0/4255638584 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f539006ae70 0x7f539006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:36.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.895+0000 7f53b107b640 1 --2- 172.21.15.76:0/4255638584 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f539006ae70 0x7f539006d330 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f539c036000 tx=0x7f539c030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:36.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:36.899+0000 7f53a27fc640 1 -- 172.21.15.76:0/4255638584 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f53a8055370 con 0x7f53ac0d3b60 2023-11-09T03:53:37.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.066+0000 7f53b3ae0640 1 -- 172.21.15.76:0/4255638584 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f53ac0684b0 con 0x7f53ac0d3b60 2023-11-09T03:53:37.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.067+0000 7f53a27fc640 1 -- 172.21.15.76:0/4255638584 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v112) v1 ==== 103+0+11 (secure 0 0 0) 0x7f53a8058990 con 0x7f53ac0d3b60 2023-11-09T03:53:37.068 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:53:37.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.069+0000 7f53b3ae0640 1 -- 172.21.15.76:0/4255638584 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f539006ae70 msgr2=0x7f539006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:37.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.069+0000 7f53b3ae0640 1 --2- 172.21.15.76:0/4255638584 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f539006ae70 0x7f539006d330 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f539c036000 tx=0x7f539c030000 comp rx=0 tx=0).stop 2023-11-09T03:53:37.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.069+0000 7f53b3ae0640 1 -- 172.21.15.76:0/4255638584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53ac0d3b60 msgr2=0x7f53ac15e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:37.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.069+0000 7f53b3ae0640 1 --2- 172.21.15.76:0/4255638584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53ac0d3b60 0x7f53ac15e7e0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f53a800efc0 tx=0x7f53a800c490 comp rx=0 tx=0).stop 2023-11-09T03:53:37.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.069+0000 7f53b3ae0640 1 -- 172.21.15.76:0/4255638584 shutdown_connections 2023-11-09T03:53:37.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.069+0000 7f53b3ae0640 1 --2- 172.21.15.76:0/4255638584 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f539006ae70 0x7f539006d330 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.069+0000 7f53b3ae0640 1 --2- 172.21.15.76:0/4255638584 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f53ac15d4e0 0x7f53ac06a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.069+0000 7f53b3ae0640 1 --2- 172.21.15.76:0/4255638584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53ac15d100 0x7f53ac150bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.069+0000 7f53b3ae0640 1 --2- 172.21.15.76:0/4255638584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53ac0d3b60 0x7f53ac15e7e0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.069+0000 7f53b3ae0640 1 -- 172.21.15.76:0/4255638584 >> 172.21.15.76:0/4255638584 conn(0x7f53ac0d2ef0 msgr2=0x7f53ac0b4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:37.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.069+0000 7f53b3ae0640 1 -- 172.21.15.76:0/4255638584 shutdown_connections 2023-11-09T03:53:37.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.069+0000 7f53b3ae0640 1 -- 172.21.15.76:0/4255638584 wait complete. 2023-11-09T03:53:37.080 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-09T03:53:37.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.182+0000 7ff00adc9640 1 Processor -- start 2023-11-09T03:53:37.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.182+0000 7ff00adc9640 1 -- start start 2023-11-09T03:53:37.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.183+0000 7ff00adc9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff0040b0c40 0x7ff00406fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:37.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.183+0000 7ff00adc9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff004071500 con 0x7ff00406bb60 2023-11-09T03:53:37.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.183+0000 7ff00adc9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff00415dae0 con 0x7ff004159e40 2023-11-09T03:53:37.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.183+0000 7ff00adc9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff004070c60 con 0x7ff0040b0c40 2023-11-09T03:53:37.184 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.183+0000 7ff003fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff0040b0c40 0x7ff00406fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:37.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.183+0000 7ff009366640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff00406bb60 0x7ff0040cd790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36104/0 (socket says 172.21.15.76:36104) 2023-11-09T03:53:37.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.183+0000 7ff009366640 1 -- 172.21.15.76:0/4159343584 learned_addr learned my addr 172.21.15.76:0/4159343584 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:53:37.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.183+0000 7ff003fff640 1 -- 172.21.15.76:0/4159343584 >> v1:172.21.15.113:6789/0 conn(0x7ff004159e40 legacy=0x7ff0040b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:37.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.183+0000 7ff003fff640 1 -- 172.21.15.76:0/4159343584 >> v1:172.21.15.76:6789/0 conn(0x7ff00406bb60 legacy=0x7ff0040cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:37.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.183+0000 7ff003fff640 1 -- 172.21.15.76:0/4159343584 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff004070f40 con 0x7ff0040b0c40 2023-11-09T03:53:37.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.183+0000 7ff0037fe640 1 -- 172.21.15.76:0/4159343584 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4222166847 0 0) 0x7ff004071500 con 0x7ff00406bb60 2023-11-09T03:53:37.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.184+0000 7ff003fff640 1 --2- 172.21.15.76:0/4159343584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0040b0c40 0x7ff00406fff0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7feff00097a0 tx=0x7feff002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a87cc9708f3bbb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:37.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.184+0000 7ff0037fe640 1 -- 172.21.15.76:0/4159343584 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff002e070 con 0x7ff0040b0c40 2023-11-09T03:53:37.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.184+0000 7ff0037fe640 1 -- 172.21.15.76:0/4159343584 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff00315a0 con 0x7ff0040b0c40 2023-11-09T03:53:37.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.184+0000 7ff0037fe640 1 -- 172.21.15.76:0/4159343584 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff0031930 con 0x7ff0040b0c40 2023-11-09T03:53:37.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.185+0000 7ff00adc9640 1 -- 172.21.15.76:0/4159343584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0040b0c40 msgr2=0x7ff00406fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:37.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.185+0000 7ff00adc9640 1 --2- 172.21.15.76:0/4159343584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0040b0c40 0x7ff00406fff0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7feff00097a0 tx=0x7feff002b840 comp rx=0 tx=0).stop 2023-11-09T03:53:37.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.185+0000 7ff00adc9640 1 -- 172.21.15.76:0/4159343584 shutdown_connections 2023-11-09T03:53:37.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.185+0000 7ff00adc9640 1 --2- 172.21.15.76:0/4159343584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0040b0c40 0x7ff00406fff0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.185+0000 7ff00adc9640 1 -- 172.21.15.76:0/4159343584 >> 172.21.15.76:0/4159343584 conn(0x7ff0040cb530 msgr2=0x7ff0040cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:37.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.185+0000 7ff00adc9640 1 -- 172.21.15.76:0/4159343584 shutdown_connections 2023-11-09T03:53:37.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.185+0000 7ff00adc9640 1 -- 172.21.15.76:0/4159343584 wait complete. 2023-11-09T03:53:37.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.186+0000 7ff00adc9640 1 Processor -- start 2023-11-09T03:53:37.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.186+0000 7ff00adc9640 1 -- start start 2023-11-09T03:53:37.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.187+0000 7ff00adc9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff00406bb60 0x7ff0041e7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:37.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.187+0000 7ff00adc9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0040b0c40 0x7ff0040c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:37.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.187+0000 7ff00adc9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff004159e40 0x7ff004158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:37.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.187+0000 7ff00adc9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0040c8a30 con 0x7ff0040b0c40 2023-11-09T03:53:37.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.187+0000 7ff00adc9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0041f4050 con 0x7ff004159e40 2023-11-09T03:53:37.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.187+0000 7ff00adc9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0040c2090 con 0x7ff00406bb60 2023-11-09T03:53:37.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.187+0000 7ff003fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0040b0c40 0x7ff0040c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:37.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.187+0000 7ff003fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0040b0c40 0x7ff0040c7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37700/0 (socket says 172.21.15.76:37700) 2023-11-09T03:53:37.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.187+0000 7ff003fff640 1 -- 172.21.15.76:0/206103587 learned_addr learned my addr 172.21.15.76:0/206103587 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:37.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.187+0000 7ff003fff640 1 -- 172.21.15.76:0/206103587 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff00406bb60 msgr2=0x7ff0041e7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:37.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.187+0000 7ff003fff640 1 --2- 172.21.15.76:0/206103587 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff00406bb60 0x7ff0041e7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.187+0000 7ff003fff640 1 -- 172.21.15.76:0/206103587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff004159e40 msgr2=0x7ff004158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:37.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.187+0000 7ff003fff640 1 --2- 172.21.15.76:0/206103587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff004159e40 0x7ff004158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.187+0000 7ff003fff640 1 -- 172.21.15.76:0/206103587 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff0009400 con 0x7ff0040b0c40 2023-11-09T03:53:37.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.187+0000 7ff003fff640 1 --2- 172.21.15.76:0/206103587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0040b0c40 0x7ff0040c7d70 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7feff00097a0 tx=0x7feff00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:37.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.187+0000 7ff0017fa640 1 -- 172.21.15.76:0/206103587 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff0032840 con 0x7ff0040b0c40 2023-11-09T03:53:37.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.187+0000 7ff0017fa640 1 -- 172.21.15.76:0/206103587 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff00329a0 con 0x7ff0040b0c40 2023-11-09T03:53:37.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.187+0000 7ff00adc9640 1 -- 172.21.15.76:0/206103587 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0040c8470 con 0x7ff0040b0c40 2023-11-09T03:53:37.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.187+0000 7ff0017fa640 1 -- 172.21.15.76:0/206103587 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff0032d50 con 0x7ff0040b0c40 2023-11-09T03:53:37.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.187+0000 7ff00adc9640 1 -- 172.21.15.76:0/206103587 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0041f42f0 con 0x7ff0040b0c40 2023-11-09T03:53:37.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.188+0000 7ff00adc9640 1 -- 172.21.15.76:0/206103587 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0040684b0 con 0x7ff0040b0c40 2023-11-09T03:53:37.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.189+0000 7ff0017fa640 1 -- 172.21.15.76:0/206103587 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feff003b3f0 con 0x7ff0040b0c40 2023-11-09T03:53:37.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.189+0000 7ff0017fa640 1 --2- 172.21.15.76:0/206103587 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fefe806ae70 0x7fefe806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:37.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.189+0000 7ff0017fa640 1 -- 172.21.15.76:0/206103587 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8167+0+0 (secure 0 0 0) 0x7feff00ac090 con 0x7ff0040b0c40 2023-11-09T03:53:37.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.190+0000 7ff008b65640 1 --2- 172.21.15.76:0/206103587 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fefe806ae70 0x7fefe806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:37.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.190+0000 7ff008b65640 1 --2- 172.21.15.76:0/206103587 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fefe806ae70 0x7fefe806d330 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7feff8006fd0 tx=0x7feff8006bc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:37.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.193+0000 7ff0017fa640 1 -- 172.21.15.76:0/206103587 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feff0076830 con 0x7ff0040b0c40 2023-11-09T03:53:37.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.361+0000 7ff00adc9640 1 -- 172.21.15.76:0/206103587 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7ff00406c730 con 0x7ff0040b0c40 2023-11-09T03:53:37.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.362+0000 7ff0017fa640 1 -- 172.21.15.76:0/206103587 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v112) v1 ==== 103+0+11 (secure 0 0 0) 0x7feff0079e50 con 0x7ff0040b0c40 2023-11-09T03:53:37.363 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:53:37.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.364+0000 7ff00adc9640 1 -- 172.21.15.76:0/206103587 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fefe806ae70 msgr2=0x7fefe806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:37.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.364+0000 7ff00adc9640 1 --2- 172.21.15.76:0/206103587 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fefe806ae70 0x7fefe806d330 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7feff8006fd0 tx=0x7feff8006bc0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.364+0000 7ff00adc9640 1 -- 172.21.15.76:0/206103587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0040b0c40 msgr2=0x7ff0040c7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:37.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.364+0000 7ff00adc9640 1 --2- 172.21.15.76:0/206103587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0040b0c40 0x7ff0040c7d70 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7feff00097a0 tx=0x7feff00046c0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.364+0000 7ff00adc9640 1 -- 172.21.15.76:0/206103587 shutdown_connections 2023-11-09T03:53:37.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.364+0000 7ff00adc9640 1 --2- 172.21.15.76:0/206103587 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fefe806ae70 0x7fefe806d330 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.364+0000 7ff00adc9640 1 --2- 172.21.15.76:0/206103587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff004159e40 0x7ff004158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.364+0000 7ff00adc9640 1 --2- 172.21.15.76:0/206103587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0040b0c40 0x7ff0040c7d70 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.364+0000 7ff00adc9640 1 --2- 172.21.15.76:0/206103587 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff00406bb60 0x7ff0041e7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.364+0000 7ff00adc9640 1 -- 172.21.15.76:0/206103587 >> 172.21.15.76:0/206103587 conn(0x7ff0040cb530 msgr2=0x7ff00415c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:37.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.364+0000 7ff00adc9640 1 -- 172.21.15.76:0/206103587 shutdown_connections 2023-11-09T03:53:37.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.364+0000 7ff00adc9640 1 -- 172.21.15.76:0/206103587 wait complete. 2023-11-09T03:53:37.375 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-09T03:53:37.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.480+0000 7fe20a355640 1 Processor -- start 2023-11-09T03:53:37.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.480+0000 7fe20a355640 1 -- start start 2023-11-09T03:53:37.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.481+0000 7fe20a355640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe204159e40 0x7fe2040b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:37.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.481+0000 7fe20a355640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe204071500 con 0x7fe204070700 2023-11-09T03:53:37.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.481+0000 7fe20a355640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe20415dae0 con 0x7fe20406bb60 2023-11-09T03:53:37.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.481+0000 7fe20a355640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe204070c60 con 0x7fe204159e40 2023-11-09T03:53:37.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.481+0000 7fe203fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe204159e40 0x7fe2040b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:37.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.481+0000 7fe203fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe204159e40 0x7fe2040b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37714/0 (socket says 172.21.15.76:37714) 2023-11-09T03:53:37.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.481+0000 7fe2088f2640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe204070700 0x7fe2040cd790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36108/0 (socket says 172.21.15.76:36108) 2023-11-09T03:53:37.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.481+0000 7fe203fff640 1 -- 172.21.15.76:0/587433925 learned_addr learned my addr 172.21.15.76:0/587433925 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:37.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.481+0000 7fe203fff640 1 -- 172.21.15.76:0/587433925 >> v1:172.21.15.113:6789/0 conn(0x7fe20406bb60 legacy=0x7fe20406fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:37.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.482+0000 7fe203fff640 1 -- 172.21.15.76:0/587433925 >> v1:172.21.15.76:6789/0 conn(0x7fe204070700 legacy=0x7fe2040cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:37.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.482+0000 7fe203fff640 1 -- 172.21.15.76:0/587433925 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe204070f40 con 0x7fe204159e40 2023-11-09T03:53:37.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.482+0000 7fe202ffd640 1 -- 172.21.15.76:0/587433925 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2137642221 0 0) 0x7fe204071500 con 0x7fe204070700 2023-11-09T03:53:37.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.482+0000 7fe203fff640 1 --2- 172.21.15.76:0/587433925 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe204159e40 0x7fe2040b0530 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400bb10 tx=0x7fe1f402e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad64a0831d3c2dea server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:37.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.482+0000 7fe202ffd640 1 -- 172.21.15.76:0/587433925 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f402ee00 con 0x7fe204159e40 2023-11-09T03:53:37.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.482+0000 7fe202ffd640 1 -- 172.21.15.76:0/587433925 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f4036440 con 0x7fe204159e40 2023-11-09T03:53:37.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.482+0000 7fe202ffd640 1 -- 172.21.15.76:0/587433925 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f40367d0 con 0x7fe204159e40 2023-11-09T03:53:37.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.483+0000 7fe20a355640 1 -- 172.21.15.76:0/587433925 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe204159e40 msgr2=0x7fe2040b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:37.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.483+0000 7fe20a355640 1 --2- 172.21.15.76:0/587433925 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe204159e40 0x7fe2040b0530 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400bb10 tx=0x7fe1f402e050 comp rx=0 tx=0).stop 2023-11-09T03:53:37.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.483+0000 7fe20a355640 1 -- 172.21.15.76:0/587433925 shutdown_connections 2023-11-09T03:53:37.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.483+0000 7fe20a355640 1 --2- 172.21.15.76:0/587433925 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe204159e40 0x7fe2040b0530 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.483+0000 7fe20a355640 1 -- 172.21.15.76:0/587433925 >> 172.21.15.76:0/587433925 conn(0x7fe2040cb530 msgr2=0x7fe2040cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:37.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.483+0000 7fe20a355640 1 -- 172.21.15.76:0/587433925 shutdown_connections 2023-11-09T03:53:37.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.483+0000 7fe20a355640 1 -- 172.21.15.76:0/587433925 wait complete. 2023-11-09T03:53:37.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.484+0000 7fe20a355640 1 Processor -- start 2023-11-09T03:53:37.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.484+0000 7fe20a355640 1 -- start start 2023-11-09T03:53:37.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.485+0000 7fe20a355640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe20406bb60 0x7fe2040cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:37.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.485+0000 7fe20a355640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe204070700 0x7fe20415e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:37.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.485+0000 7fe20a355640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe204159e40 0x7fe2041ede90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:37.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.485+0000 7fe20a355640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2041446c0 con 0x7fe204159e40 2023-11-09T03:53:37.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.485+0000 7fe20a355640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20414aa10 con 0x7fe204070700 2023-11-09T03:53:37.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.485+0000 7fe20a355640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2040ccc40 con 0x7fe20406bb60 2023-11-09T03:53:37.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.485+0000 7fe2088f2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe204159e40 0x7fe2041ede90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:37.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.485+0000 7fe2088f2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe204159e40 0x7fe2041ede90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37722/0 (socket says 172.21.15.76:37722) 2023-11-09T03:53:37.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.485+0000 7fe2088f2640 1 -- 172.21.15.76:0/1935101861 learned_addr learned my addr 172.21.15.76:0/1935101861 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:37.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.485+0000 7fe2037fe640 1 --2- 172.21.15.76:0/1935101861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe204070700 0x7fe20415e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:37.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.485+0000 7fe203fff640 1 --2- 172.21.15.76:0/1935101861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe20406bb60 0x7fe2040cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:37.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.486+0000 7fe2088f2640 1 -- 172.21.15.76:0/1935101861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe20406bb60 msgr2=0x7fe2040cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:37.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.486+0000 7fe2088f2640 1 --2- 172.21.15.76:0/1935101861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe20406bb60 0x7fe2040cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.486+0000 7fe2088f2640 1 -- 172.21.15.76:0/1935101861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe204070700 msgr2=0x7fe20415e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:37.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.486+0000 7fe2088f2640 1 --2- 172.21.15.76:0/1935101861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe204070700 0x7fe20415e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.486+0000 7fe2088f2640 1 -- 172.21.15.76:0/1935101861 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f400b770 con 0x7fe204159e40 2023-11-09T03:53:37.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.486+0000 7fe2037fe640 1 --2- 172.21.15.76:0/1935101861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe204070700 0x7fe20415e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:53:37.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.486+0000 7fe2088f2640 1 --2- 172.21.15.76:0/1935101861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe204159e40 0x7fe2041ede90 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00c7e0 tx=0x7fe1fc00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:37.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.486+0000 7fe200ff9640 1 -- 172.21.15.76:0/1935101861 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc00a640 con 0x7fe204159e40 2023-11-09T03:53:37.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.486+0000 7fe200ff9640 1 -- 172.21.15.76:0/1935101861 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1fc00a7c0 con 0x7fe204159e40 2023-11-09T03:53:37.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.486+0000 7fe200ff9640 1 -- 172.21.15.76:0/1935101861 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc00ab70 con 0x7fe204159e40 2023-11-09T03:53:37.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.486+0000 7fe20a355640 1 -- 172.21.15.76:0/1935101861 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2040ccf20 con 0x7fe204159e40 2023-11-09T03:53:37.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.486+0000 7fe20a355640 1 -- 172.21.15.76:0/1935101861 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2040cd4e0 con 0x7fe204159e40 2023-11-09T03:53:37.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.487+0000 7fe20a355640 1 -- 172.21.15.76:0/1935101861 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2040684b0 con 0x7fe204159e40 2023-11-09T03:53:37.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.487+0000 7fe200ff9640 1 -- 172.21.15.76:0/1935101861 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe1fc00acd0 con 0x7fe204159e40 2023-11-09T03:53:37.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.488+0000 7fe200ff9640 1 --2- 172.21.15.76:0/1935101861 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe1f006ae70 0x7fe1f006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:37.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.488+0000 7fe200ff9640 1 -- 172.21.15.76:0/1935101861 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8167+0+0 (secure 0 0 0) 0x7fe1fc089540 con 0x7fe204159e40 2023-11-09T03:53:37.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.488+0000 7fe203fff640 1 --2- 172.21.15.76:0/1935101861 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe1f006ae70 0x7fe1f006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:37.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.488+0000 7fe203fff640 1 --2- 172.21.15.76:0/1935101861 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe1f006ae70 0x7fe1f006d330 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4038000 tx=0x7fe1f4032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:37.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.492+0000 7fe200ff9640 1 -- 172.21.15.76:0/1935101861 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe1fc053ce0 con 0x7fe204159e40 2023-11-09T03:53:37.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.660+0000 7fe20a355640 1 -- 172.21.15.76:0/1935101861 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7fe204144800 con 0x7fe204159e40 2023-11-09T03:53:37.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.660+0000 7fe200ff9640 1 -- 172.21.15.76:0/1935101861 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v112) v1 ==== 107+0+11 (secure 0 0 0) 0x7fe1fc018070 con 0x7fe204159e40 2023-11-09T03:53:37.661 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:53:37.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.662+0000 7fe20a355640 1 -- 172.21.15.76:0/1935101861 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe1f006ae70 msgr2=0x7fe1f006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:37.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.662+0000 7fe20a355640 1 --2- 172.21.15.76:0/1935101861 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe1f006ae70 0x7fe1f006d330 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4038000 tx=0x7fe1f4032000 comp rx=0 tx=0).stop 2023-11-09T03:53:37.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.662+0000 7fe20a355640 1 -- 172.21.15.76:0/1935101861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe204159e40 msgr2=0x7fe2041ede90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:37.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.662+0000 7fe20a355640 1 --2- 172.21.15.76:0/1935101861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe204159e40 0x7fe2041ede90 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00c7e0 tx=0x7fe1fc00cca0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.662+0000 7fe20a355640 1 -- 172.21.15.76:0/1935101861 shutdown_connections 2023-11-09T03:53:37.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.662+0000 7fe20a355640 1 --2- 172.21.15.76:0/1935101861 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe1f006ae70 0x7fe1f006d330 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.662+0000 7fe20a355640 1 --2- 172.21.15.76:0/1935101861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe204159e40 0x7fe2041ede90 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.662+0000 7fe20a355640 1 --2- 172.21.15.76:0/1935101861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe204070700 0x7fe20415e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.662+0000 7fe20a355640 1 --2- 172.21.15.76:0/1935101861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe20406bb60 0x7fe2040cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.662+0000 7fe20a355640 1 -- 172.21.15.76:0/1935101861 >> 172.21.15.76:0/1935101861 conn(0x7fe2040cb530 msgr2=0x7fe204159010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:37.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.662+0000 7fe20a355640 1 -- 172.21.15.76:0/1935101861 shutdown_connections 2023-11-09T03:53:37.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.662+0000 7fe20a355640 1 -- 172.21.15.76:0/1935101861 wait complete. 2023-11-09T03:53:37.673 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:53:37.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.777+0000 7f62e9f16640 1 Processor -- start 2023-11-09T03:53:37.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.777+0000 7f62e9f16640 1 -- start start 2023-11-09T03:53:37.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.777+0000 7f62e9f16640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f62e415d100 0x7f62e40ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:37.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.777+0000 7f62e9f16640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62e40d1be0 0x7f62e40b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:37.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.777+0000 7f62e9f16640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e40b32b0 con 0x7f62e415d4e0 2023-11-09T03:53:37.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.777+0000 7f62e9f16640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e40cee10 con 0x7f62e415d100 2023-11-09T03:53:37.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.777+0000 7f62e9f16640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e40b2980 con 0x7f62e40d1be0 2023-11-09T03:53:37.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.778+0000 7f62e37fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f62e415d100 0x7f62e40ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:37.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.778+0000 7f62e3fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62e40d1be0 0x7f62e40b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:37.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.778+0000 7f62daffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f62e415d4e0 0x7f62e40d14d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:48136/0 (socket says 172.21.15.76:48136) 2023-11-09T03:53:37.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.778+0000 7f62daffd640 1 -- 172.21.15.76:0/3192999400 learned_addr learned my addr 172.21.15.76:0/3192999400 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:53:37.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.778+0000 7f62e37fe640 1 -- 172.21.15.76:0/3192999400 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62e40d1be0 msgr2=0x7f62e40b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:37.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.778+0000 7f62e37fe640 1 --2- 172.21.15.76:0/3192999400 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62e40d1be0 0x7f62e40b3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.778+0000 7f62e37fe640 1 -- 172.21.15.76:0/3192999400 >> v1:172.21.15.113:6789/0 conn(0x7f62e415d4e0 legacy=0x7f62e40d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:37.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.778+0000 7f62e37fe640 1 -- 172.21.15.76:0/3192999400 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e40b2c60 con 0x7f62e415d100 2023-11-09T03:53:37.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.778+0000 7f62e2ffd640 1 -- 172.21.15.76:0/3192999400 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 940646558 0 0) 0x7f62e40b32b0 con 0x7f62e415d4e0 2023-11-09T03:53:37.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.779+0000 7f62e37fe640 1 --2- 172.21.15.76:0/3192999400 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62e415d100 0x7f62e40ce510 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f62d4009790 tx=0x7f62d402b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d99a13a0be987f3c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:37.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.779+0000 7f62e2ffd640 1 -- 172.21.15.76:0/3192999400 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d4031800 con 0x7f62e415d100 2023-11-09T03:53:37.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.779+0000 7f62e2ffd640 1 -- 172.21.15.76:0/3192999400 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d4031960 con 0x7f62e415d100 2023-11-09T03:53:37.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.779+0000 7f62e2ffd640 1 -- 172.21.15.76:0/3192999400 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d4031cf0 con 0x7f62e415d100 2023-11-09T03:53:37.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.779+0000 7f62e9f16640 1 -- 172.21.15.76:0/3192999400 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62e415d100 msgr2=0x7f62e40ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:37.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.779+0000 7f62e9f16640 1 --2- 172.21.15.76:0/3192999400 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62e415d100 0x7f62e40ce510 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f62d4009790 tx=0x7f62d402b8a0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.780+0000 7f62e9f16640 1 -- 172.21.15.76:0/3192999400 shutdown_connections 2023-11-09T03:53:37.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.780+0000 7f62e9f16640 1 --2- 172.21.15.76:0/3192999400 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62e40d1be0 0x7f62e40b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.780+0000 7f62e9f16640 1 --2- 172.21.15.76:0/3192999400 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62e415d100 0x7f62e40ce510 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.780+0000 7f62e9f16640 1 -- 172.21.15.76:0/3192999400 >> 172.21.15.76:0/3192999400 conn(0x7f62e40d2ef0 msgr2=0x7f62e40d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:37.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.780+0000 7f62e9f16640 1 -- 172.21.15.76:0/3192999400 shutdown_connections 2023-11-09T03:53:37.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.780+0000 7f62e9f16640 1 -- 172.21.15.76:0/3192999400 wait complete. 2023-11-09T03:53:37.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.781+0000 7f62e9f16640 1 Processor -- start 2023-11-09T03:53:37.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.781+0000 7f62e9f16640 1 -- start start 2023-11-09T03:53:37.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.782+0000 7f62e9f16640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62e40d1be0 0x7f62e4158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:37.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.782+0000 7f62e9f16640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62e415d100 0x7f62e40b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:37.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.782+0000 7f62e9f16640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62e415d4e0 0x7f62e415fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:37.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.782+0000 7f62e9f16640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e40b6a10 con 0x7f62e40d1be0 2023-11-09T03:53:37.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.782+0000 7f62e9f16640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e40b6450 con 0x7f62e415d4e0 2023-11-09T03:53:37.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.782+0000 7f62e9f16640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e40b6730 con 0x7f62e415d100 2023-11-09T03:53:37.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.782+0000 7f62e37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62e40d1be0 0x7f62e4158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:37.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.782+0000 7f62e37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62e40d1be0 0x7f62e4158930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37728/0 (socket says 172.21.15.76:37728) 2023-11-09T03:53:37.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.782+0000 7f62e37fe640 1 -- 172.21.15.76:0/1824275680 learned_addr learned my addr 172.21.15.76:0/1824275680 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:37.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.782+0000 7f62e3fff640 1 --2- 172.21.15.76:0/1824275680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62e415d4e0 0x7f62e415fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:37.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.782+0000 7f62daffd640 1 --2- 172.21.15.76:0/1824275680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62e415d100 0x7f62e40b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:37.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.782+0000 7f62e37fe640 1 -- 172.21.15.76:0/1824275680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62e415d100 msgr2=0x7f62e40b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:37.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.782+0000 7f62e37fe640 1 --2- 172.21.15.76:0/1824275680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62e415d100 0x7f62e40b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.782+0000 7f62e37fe640 1 -- 172.21.15.76:0/1824275680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62e415d4e0 msgr2=0x7f62e415fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:37.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.782+0000 7f62e37fe640 1 --2- 172.21.15.76:0/1824275680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62e415d4e0 0x7f62e415fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.782+0000 7f62e37fe640 1 -- 172.21.15.76:0/1824275680 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d40093a0 con 0x7f62e40d1be0 2023-11-09T03:53:37.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.782+0000 7f62e37fe640 1 --2- 172.21.15.76:0/1824275680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62e40d1be0 0x7f62e4158930 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f62d402b8e0 tx=0x7f62d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:37.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.782+0000 7f62e0ff9640 1 -- 172.21.15.76:0/1824275680 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d4005da0 con 0x7f62e40d1be0 2023-11-09T03:53:37.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.782+0000 7f62e0ff9640 1 -- 172.21.15.76:0/1824275680 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d4004030 con 0x7f62e40d1be0 2023-11-09T03:53:37.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.782+0000 7f62e9f16640 1 -- 172.21.15.76:0/1824275680 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e40b71f0 con 0x7f62e40d1be0 2023-11-09T03:53:37.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.782+0000 7f62e0ff9640 1 -- 172.21.15.76:0/1824275680 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d40043e0 con 0x7f62e40d1be0 2023-11-09T03:53:37.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.782+0000 7f62e9f16640 1 -- 172.21.15.76:0/1824275680 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e4158e70 con 0x7f62e40d1be0 2023-11-09T03:53:37.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.783+0000 7f62e9f16640 1 -- 172.21.15.76:0/1824275680 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62e40b7050 con 0x7f62e40d1be0 2023-11-09T03:53:37.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.784+0000 7f62e0ff9640 1 -- 172.21.15.76:0/1824275680 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f62d403c870 con 0x7f62e40d1be0 2023-11-09T03:53:37.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.784+0000 7f62e0ff9640 1 --2- 172.21.15.76:0/1824275680 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f62c406ae70 0x7f62c406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:37.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.784+0000 7f62e0ff9640 1 -- 172.21.15.76:0/1824275680 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8167+0+0 (secure 0 0 0) 0x7f62d402e070 con 0x7f62e40d1be0 2023-11-09T03:53:37.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.784+0000 7f62daffd640 1 --2- 172.21.15.76:0/1824275680 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f62c406ae70 0x7f62c406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:37.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.785+0000 7f62daffd640 1 --2- 172.21.15.76:0/1824275680 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f62c406ae70 0x7f62c406d330 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f62e4160720 tx=0x7f62cc007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:37.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.788+0000 7f62e0ff9640 1 -- 172.21.15.76:0/1824275680 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f62e40b7050 con 0x7f62e40d1be0 2023-11-09T03:53:37.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.952+0000 7f62e9f16640 1 -- 172.21.15.76:0/1824275680 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f62e40b7050 con 0x7f62e40d1be0 2023-11-09T03:53:37.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.953+0000 7f62e0ff9640 1 -- 172.21.15.76:0/1824275680 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) v1 ==== 104+0+11 (secure 0 0 0) 0x7f62e40b7050 con 0x7f62e40d1be0 2023-11-09T03:53:37.953 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:53:37.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.954+0000 7f62e9f16640 1 -- 172.21.15.76:0/1824275680 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f62c406ae70 msgr2=0x7f62c406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:37.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.954+0000 7f62e9f16640 1 --2- 172.21.15.76:0/1824275680 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f62c406ae70 0x7f62c406d330 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f62e4160720 tx=0x7f62cc007d50 comp rx=0 tx=0).stop 2023-11-09T03:53:37.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.954+0000 7f62e9f16640 1 -- 172.21.15.76:0/1824275680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62e40d1be0 msgr2=0x7f62e4158930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:37.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.954+0000 7f62e9f16640 1 --2- 172.21.15.76:0/1824275680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62e40d1be0 0x7f62e4158930 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f62d402b8e0 tx=0x7f62d40046c0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.954+0000 7f62e9f16640 1 -- 172.21.15.76:0/1824275680 shutdown_connections 2023-11-09T03:53:37.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.954+0000 7f62e9f16640 1 --2- 172.21.15.76:0/1824275680 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f62c406ae70 0x7f62c406d330 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.954+0000 7f62e9f16640 1 --2- 172.21.15.76:0/1824275680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62e415d4e0 0x7f62e415fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.954+0000 7f62e9f16640 1 --2- 172.21.15.76:0/1824275680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62e415d100 0x7f62e40b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.954+0000 7f62e9f16640 1 --2- 172.21.15.76:0/1824275680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62e40d1be0 0x7f62e4158930 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:37.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.954+0000 7f62e9f16640 1 -- 172.21.15.76:0/1824275680 >> 172.21.15.76:0/1824275680 conn(0x7f62e40d2ef0 msgr2=0x7f62e40d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:37.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.955+0000 7f62e9f16640 1 -- 172.21.15.76:0/1824275680 shutdown_connections 2023-11-09T03:53:37.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:37.955+0000 7f62e9f16640 1 -- 172.21.15.76:0/1824275680 wait complete. 2023-11-09T03:53:37.966 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:53:38.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.068+0000 7fdfcf3e2640 1 Processor -- start 2023-11-09T03:53:38.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.068+0000 7fdfcf3e2640 1 -- start start 2023-11-09T03:53:38.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.069+0000 7fdfcf3e2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdfc806b740 0x7fdfc806c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:38.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.069+0000 7fdfcf3e2640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc80ad6a0 con 0x7fdfc80b1440 2023-11-09T03:53:38.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.069+0000 7fdfcf3e2640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc80acaf0 con 0x7fdfc815ade0 2023-11-09T03:53:38.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.069+0000 7fdfcf3e2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfc80acdd0 con 0x7fdfc806b740 2023-11-09T03:53:38.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.069+0000 7fdfcc97d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fdfc80b1440 0x7fdfc806a0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36116/0 (socket says 172.21.15.76:36116) 2023-11-09T03:53:38.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.069+0000 7fdfcc97d640 1 -- 172.21.15.76:0/3914086818 learned_addr learned my addr 172.21.15.76:0/3914086818 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:53:38.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.070+0000 7fdfbffff640 1 -- 172.21.15.76:0/3914086818 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1780845064 0 0) 0x7fdfc80ad6a0 con 0x7fdfc80b1440 2023-11-09T03:53:38.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.070+0000 7fdfbffff640 1 -- 172.21.15.76:0/3914086818 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfb4003520 con 0x7fdfc80b1440 2023-11-09T03:53:38.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.070+0000 7fdfcd97f640 1 --2- 172.21.15.76:0/3914086818 >> v2:172.21.15.113:3300/0 conn(0x7fdfc806b740 0x7fdfc806c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:38.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.070+0000 7fdfbffff640 1 -- 172.21.15.76:0/3914086818 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3993430197 0 0) 0x7fdfb4003520 con 0x7fdfc80b1440 2023-11-09T03:53:38.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.070+0000 7fdfbffff640 1 -- 172.21.15.76:0/3914086818 >> v2:172.21.15.113:3300/0 conn(0x7fdfc806b740 msgr2=0x7fdfc806c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:38.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.070+0000 7fdfbffff640 1 --2- 172.21.15.76:0/3914086818 >> v2:172.21.15.113:3300/0 conn(0x7fdfc806b740 0x7fdfc806c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:38.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.070+0000 7fdfbffff640 1 -- 172.21.15.76:0/3914086818 >> v1:172.21.15.113:6789/0 conn(0x7fdfc815ade0 legacy=0x7fdfc80b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:38.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.070+0000 7fdfbffff640 1 -- 172.21.15.76:0/3914086818 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc80ad0e0 con 0x7fdfc80b1440 2023-11-09T03:53:38.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.070+0000 7fdfbffff640 1 -- 172.21.15.76:0/3914086818 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fdfb8001670 con 0x7fdfc80b1440 2023-11-09T03:53:38.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.070+0000 7fdfbffff640 1 -- 172.21.15.76:0/3914086818 >> v1:172.21.15.76:6789/0 conn(0x7fdfc80b1440 legacy=0x7fdfc806a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:38.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.070+0000 7fdfbffff640 1 --2- 172.21.15.76:0/3914086818 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfb4003970 0x7fdfb4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:38.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.070+0000 7fdfbffff640 1 --2- 172.21.15.76:0/3914086818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfb40047c0 0x7fdfb4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:38.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.070+0000 7fdfbffff640 1 --2- 172.21.15.76:0/3914086818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfb4005180 0x7fdfb400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:38.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.070+0000 7fdfbffff640 1 -- 172.21.15.76:0/3914086818 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc80acdd0 con 0x7fdfb4005180 2023-11-09T03:53:38.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.070+0000 7fdfbffff640 1 -- 172.21.15.76:0/3914086818 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb400c120 con 0x7fdfb40047c0 2023-11-09T03:53:38.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.070+0000 7fdfbffff640 1 -- 172.21.15.76:0/3914086818 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb400c2e0 con 0x7fdfb4003970 2023-11-09T03:53:38.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.070+0000 7fdfcd97f640 1 --2- 172.21.15.76:0/3914086818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfb4005180 0x7fdfb400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:38.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.070+0000 7fdfcc97d640 1 --2- 172.21.15.76:0/3914086818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfb40047c0 0x7fdfb4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:38.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.070+0000 7fdfcd17e640 1 --2- 172.21.15.76:0/3914086818 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfb4003970 0x7fdfb4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:38.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.070+0000 7fdfcd97f640 1 -- 172.21.15.76:0/3914086818 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfb4003970 msgr2=0x7fdfb4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:38.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.070+0000 7fdfcd97f640 1 --2- 172.21.15.76:0/3914086818 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfb4003970 0x7fdfb4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:38.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.070+0000 7fdfcd97f640 1 -- 172.21.15.76:0/3914086818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfb40047c0 msgr2=0x7fdfb4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:38.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.070+0000 7fdfcd97f640 1 --2- 172.21.15.76:0/3914086818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfb40047c0 0x7fdfb4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:38.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.070+0000 7fdfcd97f640 1 -- 172.21.15.76:0/3914086818 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb4003520 con 0x7fdfb4005180 2023-11-09T03:53:38.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.070+0000 7fdfcd17e640 1 --2- 172.21.15.76:0/3914086818 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfb4003970 0x7fdfb4003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:53:38.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.071+0000 7fdfcd97f640 1 -- 172.21.15.76:0/3914086818 reap_dead start 2023-11-09T03:53:38.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.071+0000 7fdfcd97f640 1 --2- 172.21.15.76:0/3914086818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfb4005180 0x7fdfb400b980 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0009ac0 tx=0x7fdfb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273b299a1d64800e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:38.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.071+0000 7fdfbffff640 1 -- 172.21.15.76:0/3914086818 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0004020 con 0x7fdfb4005180 2023-11-09T03:53:38.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.071+0000 7fdfbffff640 1 -- 172.21.15.76:0/3914086818 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc0004180 con 0x7fdfb4005180 2023-11-09T03:53:38.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.071+0000 7fdfbffff640 1 -- 172.21.15.76:0/3914086818 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc002c710 con 0x7fdfb4005180 2023-11-09T03:53:38.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.072+0000 7fdfcf3e2640 1 -- 172.21.15.76:0/3914086818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfb4005180 msgr2=0x7fdfb400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:38.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.072+0000 7fdfcf3e2640 1 --2- 172.21.15.76:0/3914086818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfb4005180 0x7fdfb400b980 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0009ac0 tx=0x7fdfb40057f0 comp rx=0 tx=0).stop 2023-11-09T03:53:38.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.072+0000 7fdfcf3e2640 1 -- 172.21.15.76:0/3914086818 shutdown_connections 2023-11-09T03:53:38.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.072+0000 7fdfcf3e2640 1 --2- 172.21.15.76:0/3914086818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfb4005180 0x7fdfb400b980 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:38.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.072+0000 7fdfcf3e2640 1 -- 172.21.15.76:0/3914086818 >> 172.21.15.76:0/3914086818 conn(0x7fdfc80cbd30 msgr2=0x7fdfc80cc140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:38.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.072+0000 7fdfcf3e2640 1 -- 172.21.15.76:0/3914086818 shutdown_connections 2023-11-09T03:53:38.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.072+0000 7fdfcf3e2640 1 -- 172.21.15.76:0/3914086818 wait complete. 2023-11-09T03:53:38.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.073+0000 7fdfcf3e2640 1 Processor -- start 2023-11-09T03:53:38.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.073+0000 7fdfcf3e2640 1 -- start start 2023-11-09T03:53:38.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.074+0000 7fdfcf3e2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfb4005180 0x7fdfc80c2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:38.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.074+0000 7fdfcf3e2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfc815baa0 0x7fdfc80c6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:38.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.074+0000 7fdfcf3e2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfc80c8d00 0x7fdfc815bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:38.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.074+0000 7fdfcf3e2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc80c8b90 con 0x7fdfc80c8d00 2023-11-09T03:53:38.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.074+0000 7fdfcf3e2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc80c2680 con 0x7fdfc815baa0 2023-11-09T03:53:38.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.074+0000 7fdfcf3e2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc8071590 con 0x7fdfb4005180 2023-11-09T03:53:38.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.074+0000 7fdfcd97f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfc80c8d00 0x7fdfc815bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:38.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.074+0000 7fdfcd97f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfc80c8d00 0x7fdfc815bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37734/0 (socket says 172.21.15.76:37734) 2023-11-09T03:53:38.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.074+0000 7fdfcd97f640 1 -- 172.21.15.76:0/411889712 learned_addr learned my addr 172.21.15.76:0/411889712 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:38.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.074+0000 7fdfcd17e640 1 --2- 172.21.15.76:0/411889712 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfb4005180 0x7fdfc80c2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:38.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.074+0000 7fdfcc97d640 1 --2- 172.21.15.76:0/411889712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfc815baa0 0x7fdfc80c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:38.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.074+0000 7fdfcd97f640 1 -- 172.21.15.76:0/411889712 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfb4005180 msgr2=0x7fdfc80c2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:38.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.074+0000 7fdfcd97f640 1 --2- 172.21.15.76:0/411889712 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfb4005180 0x7fdfc80c2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:38.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.074+0000 7fdfcd97f640 1 -- 172.21.15.76:0/411889712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfc815baa0 msgr2=0x7fdfc80c6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:38.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.074+0000 7fdfcd97f640 1 --2- 172.21.15.76:0/411889712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfc815baa0 0x7fdfc80c6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:38.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.074+0000 7fdfcd97f640 1 -- 172.21.15.76:0/411889712 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc8070c60 con 0x7fdfc80c8d00 2023-11-09T03:53:38.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.074+0000 7fdfcc97d640 1 --2- 172.21.15.76:0/411889712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfc815baa0 0x7fdfc80c6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:53:38.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.074+0000 7fdfcd97f640 1 --2- 172.21.15.76:0/411889712 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfc80c8d00 0x7fdfc815bef0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0009bb0 tx=0x7fdfc0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:38.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.074+0000 7fdfbdffb640 1 -- 172.21.15.76:0/411889712 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc003a070 con 0x7fdfc80c8d00 2023-11-09T03:53:38.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.074+0000 7fdfbdffb640 1 -- 172.21.15.76:0/411889712 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc000f0c0 con 0x7fdfc80c8d00 2023-11-09T03:53:38.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.074+0000 7fdfbdffb640 1 -- 172.21.15.76:0/411889712 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0003870 con 0x7fdfc80c8d00 2023-11-09T03:53:38.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.074+0000 7fdfcf3e2640 1 -- 172.21.15.76:0/411889712 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc8070f40 con 0x7fdfc80c8d00 2023-11-09T03:53:38.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.075+0000 7fdfcf3e2640 1 -- 172.21.15.76:0/411889712 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc80c87d0 con 0x7fdfc80c8d00 2023-11-09T03:53:38.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.076+0000 7fdfcf3e2640 1 -- 172.21.15.76:0/411889712 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc8071720 con 0x7fdfc80c8d00 2023-11-09T03:53:38.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.076+0000 7fdfbdffb640 1 -- 172.21.15.76:0/411889712 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdfc00039d0 con 0x7fdfc80c8d00 2023-11-09T03:53:38.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.076+0000 7fdfbdffb640 1 --2- 172.21.15.76:0/411889712 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdfac06aea0 0x7fdfac06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:38.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.076+0000 7fdfbdffb640 1 -- 172.21.15.76:0/411889712 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8167+0+0 (secure 0 0 0) 0x7fdfc00aa060 con 0x7fdfc80c8d00 2023-11-09T03:53:38.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.077+0000 7fdfcd17e640 1 --2- 172.21.15.76:0/411889712 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdfac06aea0 0x7fdfac06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:38.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.077+0000 7fdfcd17e640 1 --2- 172.21.15.76:0/411889712 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdfac06aea0 0x7fdfac06d360 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4000f80 tx=0x7fdfc4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:38.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.081+0000 7fdfbdffb640 1 -- 172.21.15.76:0/411889712 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdfc0074800 con 0x7fdfc80c8d00 2023-11-09T03:53:38.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.245+0000 7fdfcf3e2640 1 -- 172.21.15.76:0/411889712 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdfc80684b0 con 0x7fdfc80c8d00 2023-11-09T03:53:38.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.246+0000 7fdfbdffb640 1 -- 172.21.15.76:0/411889712 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdfc0077e20 con 0x7fdfc80c8d00 2023-11-09T03:53:38.247 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:53:38.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.248+0000 7fdfcf3e2640 1 -- 172.21.15.76:0/411889712 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdfac06aea0 msgr2=0x7fdfac06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:38.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.248+0000 7fdfcf3e2640 1 --2- 172.21.15.76:0/411889712 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdfac06aea0 0x7fdfac06d360 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4000f80 tx=0x7fdfc4009040 comp rx=0 tx=0).stop 2023-11-09T03:53:38.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.248+0000 7fdfcf3e2640 1 -- 172.21.15.76:0/411889712 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfc80c8d00 msgr2=0x7fdfc815bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:38.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.248+0000 7fdfcf3e2640 1 --2- 172.21.15.76:0/411889712 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfc80c8d00 0x7fdfc815bef0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0009bb0 tx=0x7fdfc0004730 comp rx=0 tx=0).stop 2023-11-09T03:53:38.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.248+0000 7fdfcf3e2640 1 -- 172.21.15.76:0/411889712 shutdown_connections 2023-11-09T03:53:38.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.248+0000 7fdfcf3e2640 1 --2- 172.21.15.76:0/411889712 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdfac06aea0 0x7fdfac06d360 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:38.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.248+0000 7fdfcf3e2640 1 --2- 172.21.15.76:0/411889712 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdfc80c8d00 0x7fdfc815bef0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:38.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.248+0000 7fdfcf3e2640 1 --2- 172.21.15.76:0/411889712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfc815baa0 0x7fdfc80c6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:38.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.248+0000 7fdfcf3e2640 1 --2- 172.21.15.76:0/411889712 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfb4005180 0x7fdfc80c2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:38.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.248+0000 7fdfcf3e2640 1 -- 172.21.15.76:0/411889712 >> 172.21.15.76:0/411889712 conn(0x7fdfc80cbd30 msgr2=0x7fdfc815a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:38.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.248+0000 7fdfcf3e2640 1 -- 172.21.15.76:0/411889712 shutdown_connections 2023-11-09T03:53:38.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.248+0000 7fdfcf3e2640 1 -- 172.21.15.76:0/411889712 wait complete. 2023-11-09T03:53:38.259 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-09T03:53:38.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.359+0000 7fe924be7640 1 Processor -- start 2023-11-09T03:53:38.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.359+0000 7fe924be7640 1 -- start start 2023-11-09T03:53:38.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.360+0000 7fe924be7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe920159e40 0x7fe9200b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:38.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.360+0000 7fe924be7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92006bb60 0x7fe9200b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:38.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.360+0000 7fe924be7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe920071500 con 0x7fe92006fff0 2023-11-09T03:53:38.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.360+0000 7fe924be7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe92015dae0 con 0x7fe920159e40 2023-11-09T03:53:38.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.360+0000 7fe924be7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe920070c60 con 0x7fe92006bb60 2023-11-09T03:53:38.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.360+0000 7fe91ed9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe92006fff0 0x7fe920070480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36130/0 (socket says 172.21.15.76:36130) 2023-11-09T03:53:38.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.360+0000 7fe91ed9d640 1 -- 172.21.15.76:0/214576911 learned_addr learned my addr 172.21.15.76:0/214576911 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:53:38.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.360+0000 7fe91e59c640 1 --2- 172.21.15.76:0/214576911 >> v2:172.21.15.113:3300/0 conn(0x7fe920159e40 0x7fe9200b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:38.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.360+0000 7fe91dd9b640 1 --2- 172.21.15.76:0/214576911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92006bb60 0x7fe9200b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:38.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.360+0000 7fe91d59a640 1 -- 172.21.15.76:0/214576911 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2753076030 0 0) 0x7fe920071500 con 0x7fe92006fff0 2023-11-09T03:53:38.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.361+0000 7fe91d59a640 1 -- 172.21.15.76:0/214576911 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe908003520 con 0x7fe92006fff0 2023-11-09T03:53:38.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.361+0000 7fe91d59a640 1 -- 172.21.15.76:0/214576911 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2171205115 0 0) 0x7fe908003520 con 0x7fe92006fff0 2023-11-09T03:53:38.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.361+0000 7fe91e59c640 1 -- 172.21.15.76:0/214576911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92006bb60 msgr2=0x7fe9200b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:38.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.361+0000 7fe91e59c640 1 --2- 172.21.15.76:0/214576911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92006bb60 0x7fe9200b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:38.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.361+0000 7fe91e59c640 1 -- 172.21.15.76:0/214576911 >> v1:172.21.15.76:6789/0 conn(0x7fe92006fff0 legacy=0x7fe920070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:38.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.361+0000 7fe91e59c640 1 -- 172.21.15.76:0/214576911 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe920070f40 con 0x7fe920159e40 2023-11-09T03:53:38.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.362+0000 7fe91e59c640 1 --2- 172.21.15.76:0/214576911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe920159e40 0x7fe9200b0530 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe910009d90 tx=0x7fe91002bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3db4343b0cd24e4b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:38.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.362+0000 7fe91d59a640 1 -- 172.21.15.76:0/214576911 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe918001670 con 0x7fe920159e40 2023-11-09T03:53:38.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.362+0000 7fe91d59a640 1 -- 172.21.15.76:0/214576911 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91002c810 con 0x7fe920159e40 2023-11-09T03:53:38.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.362+0000 7fe91d59a640 1 -- 172.21.15.76:0/214576911 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91002cb80 con 0x7fe920159e40 2023-11-09T03:53:38.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.362+0000 7fe924be7640 1 -- 172.21.15.76:0/214576911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe920159e40 msgr2=0x7fe9200b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:38.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.362+0000 7fe924be7640 1 --2- 172.21.15.76:0/214576911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe920159e40 0x7fe9200b0530 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe910009d90 tx=0x7fe91002bce0 comp rx=0 tx=0).stop 2023-11-09T03:53:38.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.362+0000 7fe924be7640 1 -- 172.21.15.76:0/214576911 shutdown_connections 2023-11-09T03:53:38.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.362+0000 7fe924be7640 1 --2- 172.21.15.76:0/214576911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92006bb60 0x7fe9200b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:38.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.362+0000 7fe924be7640 1 --2- 172.21.15.76:0/214576911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe920159e40 0x7fe9200b0530 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:38.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.362+0000 7fe924be7640 1 -- 172.21.15.76:0/214576911 >> 172.21.15.76:0/214576911 conn(0x7fe9200cb530 msgr2=0x7fe9200cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:38.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.362+0000 7fe924be7640 1 -- 172.21.15.76:0/214576911 shutdown_connections 2023-11-09T03:53:38.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.362+0000 7fe924be7640 1 -- 172.21.15.76:0/214576911 wait complete. 2023-11-09T03:53:38.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.364+0000 7fe924be7640 1 Processor -- start 2023-11-09T03:53:38.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.364+0000 7fe924be7640 1 -- start start 2023-11-09T03:53:38.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.364+0000 7fe924be7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92006bb60 0x7fe9200c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:38.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.364+0000 7fe924be7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe92006fff0 0x7fe92014e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:38.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.364+0000 7fe924be7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe920159e40 0x7fe9200cf010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:38.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.364+0000 7fe924be7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92014e580 con 0x7fe920159e40 2023-11-09T03:53:38.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.364+0000 7fe924be7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9200c6b50 con 0x7fe92006fff0 2023-11-09T03:53:38.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.364+0000 7fe924be7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9200c6e30 con 0x7fe92006bb60 2023-11-09T03:53:38.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.365+0000 7fe91ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe920159e40 0x7fe9200cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:38.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.365+0000 7fe91ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe920159e40 0x7fe9200cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37748/0 (socket says 172.21.15.76:37748) 2023-11-09T03:53:38.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.365+0000 7fe91ed9d640 1 -- 172.21.15.76:0/416060704 learned_addr learned my addr 172.21.15.76:0/416060704 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:38.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.365+0000 7fe91dd9b640 1 --2- 172.21.15.76:0/416060704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe92006fff0 0x7fe92014e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:38.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.365+0000 7fe91e59c640 1 --2- 172.21.15.76:0/416060704 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92006bb60 0x7fe9200c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:38.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.365+0000 7fe91ed9d640 1 -- 172.21.15.76:0/416060704 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92006bb60 msgr2=0x7fe9200c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:38.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.365+0000 7fe91ed9d640 1 --2- 172.21.15.76:0/416060704 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92006bb60 0x7fe9200c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:38.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.365+0000 7fe91ed9d640 1 -- 172.21.15.76:0/416060704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe92006fff0 msgr2=0x7fe92014e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:38.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.365+0000 7fe91ed9d640 1 --2- 172.21.15.76:0/416060704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe92006fff0 0x7fe92014e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:38.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.365+0000 7fe91ed9d640 1 -- 172.21.15.76:0/416060704 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe910009a70 con 0x7fe920159e40 2023-11-09T03:53:38.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.365+0000 7fe91ed9d640 1 --2- 172.21.15.76:0/416060704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe920159e40 0x7fe9200cf010 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fe9180011e0 tx=0x7fe91800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:38.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.365+0000 7fe90effd640 1 -- 172.21.15.76:0/416060704 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe918005ad0 con 0x7fe920159e40 2023-11-09T03:53:38.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.365+0000 7fe90effd640 1 -- 172.21.15.76:0/416060704 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe918005c30 con 0x7fe920159e40 2023-11-09T03:53:38.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.365+0000 7fe924be7640 1 -- 172.21.15.76:0/416060704 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9200c7110 con 0x7fe920159e40 2023-11-09T03:53:38.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.365+0000 7fe90effd640 1 -- 172.21.15.76:0/416060704 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe918007070 con 0x7fe920159e40 2023-11-09T03:53:38.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.365+0000 7fe924be7640 1 -- 172.21.15.76:0/416060704 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9200c76d0 con 0x7fe920159e40 2023-11-09T03:53:38.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.366+0000 7fe924be7640 1 -- 172.21.15.76:0/416060704 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9200c78f0 con 0x7fe920159e40 2023-11-09T03:53:38.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.367+0000 7fe90effd640 1 -- 172.21.15.76:0/416060704 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe9180071d0 con 0x7fe920159e40 2023-11-09T03:53:38.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.367+0000 7fe90effd640 1 --2- 172.21.15.76:0/416060704 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe90006ae70 0x7fe90006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:38.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.367+0000 7fe90effd640 1 -- 172.21.15.76:0/416060704 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8167+0+0 (secure 0 0 0) 0x7fe918089b90 con 0x7fe920159e40 2023-11-09T03:53:38.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.367+0000 7fe91e59c640 1 --2- 172.21.15.76:0/416060704 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe90006ae70 0x7fe90006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:38.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.368+0000 7fe91e59c640 1 --2- 172.21.15.76:0/416060704 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe90006ae70 0x7fe90006d330 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe910036000 tx=0x7fe910030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:38.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.371+0000 7fe90effd640 1 -- 172.21.15.76:0/416060704 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe918054350 con 0x7fe920159e40 2023-11-09T03:53:38.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.556+0000 7fe924be7640 1 -- 172.21.15.76:0/416060704 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe9200684b0 con 0x7fe920159e40 2023-11-09T03:53:38.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.557+0000 7fe90effd640 1 -- 172.21.15.76:0/416060704 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 23 entries v112) v1 ==== 81+0+1387 (secure 0 0 0) 0x7fe918057970 con 0x7fe920159e40 2023-11-09T03:53:38.557 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/334503200 2023-11-09T04:53:32.572840+0000 2023-11-09T03:53:38.558 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/7360844 2023-11-10T03:53:24.718008+0000 2023-11-09T03:53:38.558 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/7360844 2023-11-10T03:53:24.718008+0000 2023-11-09T03:53:38.558 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/546338722 2023-11-10T03:53:17.470106+0000 2023-11-09T03:53:38.558 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/546338722 2023-11-10T03:53:17.470106+0000 2023-11-09T03:53:38.558 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/3087804343 2023-11-09T04:52:42.513556+0000 2023-11-09T03:53:38.558 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6833/1536527256 2023-11-10T03:52:34.213804+0000 2023-11-09T03:53:38.559 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/445306598 2023-11-10T03:52:27.406838+0000 2023-11-09T03:53:38.559 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/3229239133 2023-11-10T03:51:32.624856+0000 2023-11-09T03:53:38.559 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/503424885 2023-11-09T04:51:47.727422+0000 2023-11-09T03:53:38.559 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/4011275120 2023-11-10T03:49:42.029057+0000 2023-11-09T03:53:38.559 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/4011275120 2023-11-10T03:49:42.029057+0000 2023-11-09T03:53:38.560 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/3280098776 2023-11-10T03:50:07.616780+0000 2023-11-09T03:53:38.560 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/3280098776 2023-11-10T03:50:07.616780+0000 2023-11-09T03:53:38.560 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/1663988060 2023-11-10T03:50:33.665092+0000 2023-11-09T03:53:38.560 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/1663988060 2023-11-10T03:50:33.665092+0000 2023-11-09T03:53:38.560 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/445306598 2023-11-10T03:52:27.406838+0000 2023-11-09T03:53:38.560 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6833/3697850101 2023-11-10T03:51:39.442929+0000 2023-11-09T03:53:38.561 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/3449910842 2023-11-10T03:50:59.611256+0000 2023-11-09T03:53:38.561 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/3229239133 2023-11-10T03:51:32.624856+0000 2023-11-09T03:53:38.561 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6832/1536527256 2023-11-10T03:52:34.213804+0000 2023-11-09T03:53:38.561 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/3449910842 2023-11-10T03:50:59.611256+0000 2023-11-09T03:53:38.561 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6832/3697850101 2023-11-10T03:51:39.442929+0000 2023-11-09T03:53:38.562 INFO:teuthology.orchestra.run.smithi076.stderr:listed 23 entries 2023-11-09T03:53:38.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.558+0000 7fe924be7640 1 -- 172.21.15.76:0/416060704 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe90006ae70 msgr2=0x7fe90006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:38.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.558+0000 7fe924be7640 1 --2- 172.21.15.76:0/416060704 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe90006ae70 0x7fe90006d330 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe910036000 tx=0x7fe910030000 comp rx=0 tx=0).stop 2023-11-09T03:53:38.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.558+0000 7fe924be7640 1 -- 172.21.15.76:0/416060704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe920159e40 msgr2=0x7fe9200cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:38.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.558+0000 7fe924be7640 1 --2- 172.21.15.76:0/416060704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe920159e40 0x7fe9200cf010 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fe9180011e0 tx=0x7fe91800cbe0 comp rx=0 tx=0).stop 2023-11-09T03:53:38.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.559+0000 7fe924be7640 1 -- 172.21.15.76:0/416060704 shutdown_connections 2023-11-09T03:53:38.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.559+0000 7fe924be7640 1 --2- 172.21.15.76:0/416060704 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe90006ae70 0x7fe90006d330 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:38.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.559+0000 7fe924be7640 1 --2- 172.21.15.76:0/416060704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe920159e40 0x7fe9200cf010 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:38.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.559+0000 7fe924be7640 1 --2- 172.21.15.76:0/416060704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe92006fff0 0x7fe92014e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:38.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.559+0000 7fe924be7640 1 --2- 172.21.15.76:0/416060704 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe92006bb60 0x7fe9200c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:38.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.559+0000 7fe924be7640 1 -- 172.21.15.76:0/416060704 >> 172.21.15.76:0/416060704 conn(0x7fe9200cb530 msgr2=0x7fe92015dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:38.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.559+0000 7fe924be7640 1 -- 172.21.15.76:0/416060704 shutdown_connections 2023-11-09T03:53:38.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.559+0000 7fe924be7640 1 -- 172.21.15.76:0/416060704 wait complete. 2023-11-09T03:53:38.569 DEBUG:tasks.cephfs.mount:Force/lazy unmounting on client.0 2023-11-09T03:53:38.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:53:38.570 DEBUG:teuthology.orchestra.run.smithi076:> sudo umount --lazy --force /home/ubuntu/cephtest/mnt.0 2023-11-09T03:53:38.595 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[125642]: fuse finished with error 0 and tester_r 0 2023-11-09T03:53:38.596 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T03:53:38.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:53:38.597 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-09T03:53:38.652 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:53:38.652 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:53:38.707 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056381,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14761 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248868k,nr_inodes=812217,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-09T03:53:38.707 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-09T03:53:38.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.846+0000 7f4c8524f640 1 Processor -- start 2023-11-09T03:53:38.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.846+0000 7f4c8524f640 1 -- start start 2023-11-09T03:53:38.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.846+0000 7f4c8524f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4c8015d4e0 0x7f4c800d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:38.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.846+0000 7f4c8524f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4c800d3b60 0x7f4c800d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:38.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.846+0000 7f4c8524f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c800b32b0 con 0x7f4c8015d100 2023-11-09T03:53:38.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.846+0000 7f4c8524f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c800cec20 con 0x7f4c8015d4e0 2023-11-09T03:53:38.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.846+0000 7f4c8524f640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c800cee10 con 0x7f4c800d3b60 2023-11-09T03:53:38.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.847+0000 7f4c77fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4c8015d4e0 0x7f4c800d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:38.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.847+0000 7f4c77fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4c8015d4e0 0x7f4c800d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37760/0 (socket says 172.21.15.76:37760) 2023-11-09T03:53:38.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.847+0000 7f4c77fff640 1 -- 172.21.15.76:0/865957271 learned_addr learned my addr 172.21.15.76:0/865957271 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:38.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.847+0000 7f4c7f59e640 1 --2- 172.21.15.76:0/865957271 >> v2:172.21.15.113:3300/0 conn(0x7f4c800d3b60 0x7f4c800d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:38.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.847+0000 7f4c77fff640 1 -- 172.21.15.76:0/865957271 >> v2:172.21.15.113:3300/0 conn(0x7f4c800d3b60 msgr2=0x7f4c800d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:38.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.847+0000 7f4c77fff640 1 --2- 172.21.15.76:0/865957271 >> v2:172.21.15.113:3300/0 conn(0x7f4c800d3b60 0x7f4c800d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:38.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.847+0000 7f4c77fff640 1 -- 172.21.15.76:0/865957271 >> v1:172.21.15.113:6789/0 conn(0x7f4c8015d100 legacy=0x7f4c800ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:38.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.847+0000 7f4c77fff640 1 -- 172.21.15.76:0/865957271 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c800b4750 con 0x7f4c8015d4e0 2023-11-09T03:53:38.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.848+0000 7f4c77fff640 1 --2- 172.21.15.76:0/865957271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c8015d4e0 0x7f4c800d14d0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f4c8008d6a0 tx=0x7f4c6802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a15b926a822ca6 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:38.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.848+0000 7f4c7e59c640 1 -- 172.21.15.76:0/865957271 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c68031870 con 0x7f4c8015d4e0 2023-11-09T03:53:38.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.848+0000 7f4c7e59c640 1 -- 172.21.15.76:0/865957271 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c680319d0 con 0x7f4c8015d4e0 2023-11-09T03:53:38.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.848+0000 7f4c7e59c640 1 -- 172.21.15.76:0/865957271 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c68031d60 con 0x7f4c8015d4e0 2023-11-09T03:53:38.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.848+0000 7f4c8524f640 1 -- 172.21.15.76:0/865957271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c8015d4e0 msgr2=0x7f4c800d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:38.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.848+0000 7f4c8524f640 1 --2- 172.21.15.76:0/865957271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c8015d4e0 0x7f4c800d14d0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f4c8008d6a0 tx=0x7f4c6802b910 comp rx=0 tx=0).stop 2023-11-09T03:53:38.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.848+0000 7f4c8524f640 1 -- 172.21.15.76:0/865957271 shutdown_connections 2023-11-09T03:53:38.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.848+0000 7f4c8524f640 1 --2- 172.21.15.76:0/865957271 >> v2:172.21.15.113:3300/0 conn(0x7f4c800d3b60 0x7f4c800d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.848+0000 7f4c8524f640 1 --2- 172.21.15.76:0/865957271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c8015d4e0 0x7f4c800d14d0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.848+0000 7f4c8524f640 1 -- 172.21.15.76:0/865957271 >> 172.21.15.76:0/865957271 conn(0x7f4c800d2ef0 msgr2=0x7f4c800d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.848+0000 7f4c8524f640 1 -- 172.21.15.76:0/865957271 shutdown_connections 2023-11-09T03:53:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.848+0000 7f4c8524f640 1 -- 172.21.15.76:0/865957271 wait complete. 2023-11-09T03:53:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.850+0000 7f4c8524f640 1 Processor -- start 2023-11-09T03:53:38.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.850+0000 7f4c8524f640 1 -- start start 2023-11-09T03:53:38.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.850+0000 7f4c8524f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c800d3b60 0x7f4c801f2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:38.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.850+0000 7f4c8524f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c8015d100 0x7f4c800b7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:38.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.850+0000 7f4c8524f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c8015d4e0 0x7f4c800b7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:38.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.850+0000 7f4c8524f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c801f27c0 con 0x7f4c8015d100 2023-11-09T03:53:38.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.850+0000 7f4c8524f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c801f5e90 con 0x7f4c800d3b60 2023-11-09T03:53:38.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.850+0000 7f4c8524f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c801f6730 con 0x7f4c8015d4e0 2023-11-09T03:53:38.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.850+0000 7f4c77fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c8015d100 0x7f4c800b7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:38.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.850+0000 7f4c77fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c8015d100 0x7f4c800b7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37772/0 (socket says 172.21.15.76:37772) 2023-11-09T03:53:38.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.850+0000 7f4c77fff640 1 -- 172.21.15.76:0/1061438334 learned_addr learned my addr 172.21.15.76:0/1061438334 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:38.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.851+0000 7f4c7ed9d640 1 --2- 172.21.15.76:0/1061438334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c800d3b60 0x7f4c801f2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:38.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.851+0000 7f4c7f59e640 1 --2- 172.21.15.76:0/1061438334 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c8015d4e0 0x7f4c800b7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:38.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.851+0000 7f4c77fff640 1 -- 172.21.15.76:0/1061438334 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c8015d4e0 msgr2=0x7f4c800b7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:38.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.851+0000 7f4c77fff640 1 --2- 172.21.15.76:0/1061438334 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c8015d4e0 0x7f4c800b7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:38.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.851+0000 7f4c77fff640 1 -- 172.21.15.76:0/1061438334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c800d3b60 msgr2=0x7f4c801f2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:38.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.851+0000 7f4c77fff640 1 --2- 172.21.15.76:0/1061438334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c800d3b60 0x7f4c801f2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:38.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.851+0000 7f4c77fff640 1 -- 172.21.15.76:0/1061438334 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c680094d0 con 0x7f4c8015d100 2023-11-09T03:53:38.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.851+0000 7f4c7f59e640 1 --2- 172.21.15.76:0/1061438334 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c8015d4e0 0x7f4c800b7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:53:38.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.851+0000 7f4c77fff640 1 --2- 172.21.15.76:0/1061438334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c8015d100 0x7f4c800b7a80 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f4c6802b950 tx=0x7f4c680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:38.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.851+0000 7f4c777fe640 1 -- 172.21.15.76:0/1061438334 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6803ac60 con 0x7f4c8015d100 2023-11-09T03:53:38.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.851+0000 7f4c777fe640 1 -- 172.21.15.76:0/1061438334 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6803adc0 con 0x7f4c8015d100 2023-11-09T03:53:38.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.851+0000 7f4c777fe640 1 -- 172.21.15.76:0/1061438334 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c68044630 con 0x7f4c8015d100 2023-11-09T03:53:38.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.851+0000 7f4c8524f640 1 -- 172.21.15.76:0/1061438334 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c80160020 con 0x7f4c8015d100 2023-11-09T03:53:38.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.851+0000 7f4c8524f640 1 -- 172.21.15.76:0/1061438334 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c801f63c0 con 0x7f4c8015d100 2023-11-09T03:53:38.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.852+0000 7f4c8524f640 1 -- 172.21.15.76:0/1061438334 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c800d0440 con 0x7f4c8015d100 2023-11-09T03:53:38.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.853+0000 7f4c777fe640 1 -- 172.21.15.76:0/1061438334 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4c6804d450 con 0x7f4c8015d100 2023-11-09T03:53:38.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.853+0000 7f4c777fe640 1 --2- 172.21.15.76:0/1061438334 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4c6006ae70 0x7f4c6006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:38.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.853+0000 7f4c777fe640 1 -- 172.21.15.76:0/1061438334 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8167+0+0 (secure 0 0 0) 0x7f4c6802d040 con 0x7f4c8015d100 2023-11-09T03:53:38.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.853+0000 7f4c7ed9d640 1 --2- 172.21.15.76:0/1061438334 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4c6006ae70 0x7f4c6006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:38.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.854+0000 7f4c7ed9d640 1 --2- 172.21.15.76:0/1061438334 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4c6006ae70 0x7f4c6006d330 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4c70007fd0 tx=0x7f4c70007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:38.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:38.857+0000 7f4c777fe640 1 -- 172.21.15.76:0/1061438334 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4c68076b80 con 0x7f4c8015d100 2023-11-09T03:53:39.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:39.056+0000 7f4c8524f640 1 -- 172.21.15.76:0/1061438334 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4c800684b0 con 0x7f4c8015d100 2023-11-09T03:53:39.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:39.057+0000 7f4c777fe640 1 -- 172.21.15.76:0/1061438334 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v117) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4c6807a1a0 con 0x7f4c8015d100 2023-11-09T03:53:39.057 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-09T03:53:39.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:39.059+0000 7f4c8524f640 1 -- 172.21.15.76:0/1061438334 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4c6006ae70 msgr2=0x7f4c6006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:39.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:39.059+0000 7f4c8524f640 1 --2- 172.21.15.76:0/1061438334 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4c6006ae70 0x7f4c6006d330 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4c70007fd0 tx=0x7f4c70007e80 comp rx=0 tx=0).stop 2023-11-09T03:53:39.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:39.059+0000 7f4c8524f640 1 -- 172.21.15.76:0/1061438334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c8015d100 msgr2=0x7f4c800b7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:39.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:39.059+0000 7f4c8524f640 1 --2- 172.21.15.76:0/1061438334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c8015d100 0x7f4c800b7a80 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f4c6802b950 tx=0x7f4c680046c0 comp rx=0 tx=0).stop 2023-11-09T03:53:39.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:39.059+0000 7f4c8524f640 1 -- 172.21.15.76:0/1061438334 shutdown_connections 2023-11-09T03:53:39.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:39.059+0000 7f4c8524f640 1 --2- 172.21.15.76:0/1061438334 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4c6006ae70 0x7f4c6006d330 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:39.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:39.059+0000 7f4c8524f640 1 --2- 172.21.15.76:0/1061438334 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c8015d4e0 0x7f4c800b7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:39.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:39.059+0000 7f4c8524f640 1 --2- 172.21.15.76:0/1061438334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c8015d100 0x7f4c800b7a80 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:39.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:39.059+0000 7f4c8524f640 1 --2- 172.21.15.76:0/1061438334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c800d3b60 0x7f4c801f2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:39.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:39.059+0000 7f4c8524f640 1 -- 172.21.15.76:0/1061438334 >> 172.21.15.76:0/1061438334 conn(0x7f4c800d2ef0 msgr2=0x7f4c8006bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:39.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:39.059+0000 7f4c8524f640 1 -- 172.21.15.76:0/1061438334 shutdown_connections 2023-11-09T03:53:39.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:39.059+0000 7f4c8524f640 1 -- 172.21.15.76:0/1061438334 wait complete. 2023-11-09T03:53:39.070 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-09T03:53:39.070 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-09T03:53:39.070 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2023-11-09T03:53:39.070 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-09T03:53:39.071 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-09T03:53:39.071 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-09T03:53:39.071 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-09T03:53:39.072 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-09T03:53:39.072 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:53:39.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:53:39.073 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2023-11-09T03:53:39.087 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-09T03:53:39.088 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-09T03:53:39.088 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2023-11-09T03:53:39.088 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:53:39.088 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2023-11-09T03:53:39.088 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:53:39.089 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:53:39.089 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:53:39.089 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2023-11-09T03:53:39.089 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:53:39.089 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:53:39.089 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2023-11-09T03:53:39.090 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:53:39.090 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:53:39.090 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-09T03:53:39.090 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:53:39.090 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-09T03:53:39.090 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42130sec preferred_lft 42130sec 2023-11-09T03:53:39.091 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2023-11-09T03:53:39.091 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:53:39.091 INFO:teuthology.orchestra.run.smithi076.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-09T03:53:39.091 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2e:15:bd:03:a8:2e brd ff:ff:ff:ff:ff:ff 2023-11-09T03:53:39.091 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-09T03:53:39.092 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:53:39.092 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::bcda:38ff:fed4:fdb0/64 scope link 2023-11-09T03:53:39.092 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:53:39.092 INFO:teuthology.orchestra.run.smithi076.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-09T03:53:39.092 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 6a:8d:94:d7:e5:00 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:53:39.092 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::688d:94ff:fed7:e500/64 scope link 2023-11-09T03:53:39.093 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:53:39.093 INFO:teuthology.orchestra.run.smithi076.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-09T03:53:39.093 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2e:15:bd:03:a8:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:53:39.093 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::2c15:bdff:fe03:a82e/64 scope link 2023-11-09T03:53:39.093 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:53:39.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:53:39.094 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-09T03:53:39.154 INFO:teuthology.orchestra.run.smithi076.stdout:1 2023-11-09T03:53:39.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:53:39.156 DEBUG:teuthology.orchestra.run.smithi076:> route 2023-11-09T03:53:39.210 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2023-11-09T03:53:39.210 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-09T03:53:39.211 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-09T03:53:39.211 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-09T03:53:39.211 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-09T03:53:39.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:53:39.212 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-09T03:53:39.212 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-09T03:53:39.212 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-09T03:53:39.213 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-09T03:53:39.213 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-09T03:53:39.213 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-09T03:53:39.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:53:39.306 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2023-11-09T03:53:39.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-09T03:53:39.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-09T03:53:39.321 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:53:39.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:53:39.322 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2023-11-09T03:53:39.388 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-09T03:53:39.388 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:53:39.388 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-09T03:53:39.443 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:53:39.443 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:53:39.443 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-09T03:53:39.497 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:53:39.552 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:53:39.552 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:53:39.552 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:53:39.553 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:53:39.553 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:53:39.553 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:53:39.553 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:53:39.553 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:53:39.553 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:53:39.554 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:53:39.554 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:53:39.554 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:53:39.554 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:53:39.554 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:53:39.554 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:53:39.555 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:53:39.555 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:53:39.555 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:53:39.555 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:53:39.555 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:53:39.556 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:53:39.556 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:53:39.556 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:53:39.556 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:53:39.556 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:53:39.556 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:53:39.557 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:53:39.557 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:53:39.557 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:53:39.557 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:53:39.557 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:53:39.557 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:53:39.558 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:53:39.558 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:53:39.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:53:39.559 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:53:39.614 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-09T03:53:39.614 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-09T03:53:39.657 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:53:39.657 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:53:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:53:39.671 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:53:39.671 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:53:39.672 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:53:39.672 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:53:39.672 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:53:39.672 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:53:39.672 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:53:39.673 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:53:39.673 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:53:39.673 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:53:39.673 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:53:39.673 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:53:39.673 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:53:39.674 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:53:39.674 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:53:39.674 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:53:39.674 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:53:39.674 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:53:39.675 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:53:39.675 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:53:39.675 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:53:39.675 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:53:39.675 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:53:39.675 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:53:39.676 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:53:39.676 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:53:39.676 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:53:39.676 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:53:39.676 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:53:39.676 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:53:39.677 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:53:39.677 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:53:39.677 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:53:39.677 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:53:39.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:53:39.678 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:53:39.747 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:53:39.746+0000 7f6fafb65480 -1 init, newargv = 0x562d35c7dc80 newargc=15 2023-11-09T03:53:39.747 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[127808]: starting ceph client 2023-11-09T03:53:39.753 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[127808]: starting fuse 2023-11-09T03:53:40.835 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:53:40.854 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:53:40.854 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:53:40.854 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:53:40.855 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:53:40.855 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:53:40.855 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:53:40.855 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:53:40.855 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:53:40.855 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:53:40.856 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:53:40.856 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:53:40.856 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:53:40.856 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:53:40.856 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:53:40.856 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:53:40.857 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:53:40.857 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:53:40.857 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:53:40.857 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:53:40.857 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:53:40.857 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:53:40.858 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:53:40.858 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:53:40.858 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:53:40.858 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:53:40.858 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:53:40.858 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:53:40.859 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:53:40.859 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:53:40.859 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:53:40.859 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:53:40.859 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:53:40.859 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:53:40.860 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:53:40.860 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-09T03:53:40.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:53:40.861 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:53:40.918 INFO:teuthology.orchestra.run.smithi076.stdout:43 2023-11-09T03:53:40.919 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-09T03:53:40.919 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer -- python3 -c ' 2023-11-09T03:53:40.919 DEBUG:teuthology.orchestra.run.smithi076:> import glob 2023-11-09T03:53:40.920 DEBUG:teuthology.orchestra.run.smithi076:> import re 2023-11-09T03:53:40.920 DEBUG:teuthology.orchestra.run.smithi076:> import os 2023-11-09T03:53:40.920 DEBUG:teuthology.orchestra.run.smithi076:> import subprocess 2023-11-09T03:53:40.921 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:53:40.921 DEBUG:teuthology.orchestra.run.smithi076:> def _find_admin_socket(client_name): 2023-11-09T03:53:40.921 DEBUG:teuthology.orchestra.run.smithi076:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-09T03:53:40.922 DEBUG:teuthology.orchestra.run.smithi076:> files = glob.glob(asok_path) 2023-11-09T03:53:40.922 DEBUG:teuthology.orchestra.run.smithi076:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-09T03:53:40.922 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:53:40.923 DEBUG:teuthology.orchestra.run.smithi076:> # Given a non-glob path, it better be there 2023-11-09T03:53:40.923 DEBUG:teuthology.orchestra.run.smithi076:> if "*" not in asok_path: 2023-11-09T03:53:40.923 DEBUG:teuthology.orchestra.run.smithi076:> assert(len(files) == 1) 2023-11-09T03:53:40.924 DEBUG:teuthology.orchestra.run.smithi076:> return files[0] 2023-11-09T03:53:40.924 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:53:40.924 DEBUG:teuthology.orchestra.run.smithi076:> for f in files: 2023-11-09T03:53:40.925 DEBUG:teuthology.orchestra.run.smithi076:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-09T03:53:40.925 DEBUG:teuthology.orchestra.run.smithi076:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-09T03:53:40.925 DEBUG:teuthology.orchestra.run.smithi076:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-09T03:53:40.926 DEBUG:teuthology.orchestra.run.smithi076:> contents = proc_f.read() 2023-11-09T03:53:40.926 DEBUG:teuthology.orchestra.run.smithi076:> if mountpoint in contents: 2023-11-09T03:53:40.926 DEBUG:teuthology.orchestra.run.smithi076:> return f 2023-11-09T03:53:40.927 DEBUG:teuthology.orchestra.run.smithi076:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-09T03:53:40.927 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:53:40.927 DEBUG:teuthology.orchestra.run.smithi076:> print(_find_admin_socket("client.0")) 2023-11-09T03:53:40.928 DEBUG:teuthology.orchestra.run.smithi076:> ' 2023-11-09T03:53:41.050 INFO:teuthology.orchestra.run.smithi076.stdout:/var/run/ceph/ceph-client.0.127808.asok 2023-11-09T03:53:41.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:53:41.061 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.127808.asok 2023-11-09T03:53:41.061 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:53:41.062 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.127808.asok status 2023-11-09T03:53:41.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:53:41.231 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:53:41.231 INFO:teuthology.orchestra.run.smithi076.stdout: "metadata": { 2023-11-09T03:53:41.231 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_sha1": "7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4", 2023-11-09T03:53:41.232 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev)", 2023-11-09T03:53:41.232 INFO:teuthology.orchestra.run.smithi076.stdout: "entity_id": "0", 2023-11-09T03:53:41.232 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": "smithi076", 2023-11-09T03:53:41.232 INFO:teuthology.orchestra.run.smithi076.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-09T03:53:41.232 INFO:teuthology.orchestra.run.smithi076.stdout: "pid": "127808", 2023-11-09T03:53:41.232 INFO:teuthology.orchestra.run.smithi076.stdout: "root": "/" 2023-11-09T03:53:41.233 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:53:41.233 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_count": 0, 2023-11-09T03:53:41.233 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_pinned_count": 0, 2023-11-09T03:53:41.233 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 8934, 2023-11-09T03:53:41.233 INFO:teuthology.orchestra.run.smithi076.stdout: "inst": { 2023-11-09T03:53:41.234 INFO:teuthology.orchestra.run.smithi076.stdout: "name": { 2023-11-09T03:53:41.234 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "client", 2023-11-09T03:53:41.234 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 8934 2023-11-09T03:53:41.234 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:53:41.234 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2023-11-09T03:53:41.234 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2023-11-09T03:53:41.235 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2023-11-09T03:53:41.235 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2519773110 2023-11-09T03:53:41.235 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:53:41.235 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:53:41.235 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2023-11-09T03:53:41.236 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2023-11-09T03:53:41.236 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2023-11-09T03:53:41.236 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2519773110 2023-11-09T03:53:41.236 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:53:41.236 INFO:teuthology.orchestra.run.smithi076.stdout: "inst_str": "client.8934 192.168.0.1:0/2519773110", 2023-11-09T03:53:41.236 INFO:teuthology.orchestra.run.smithi076.stdout: "addr_str": "192.168.0.1:0/2519773110", 2023-11-09T03:53:41.237 INFO:teuthology.orchestra.run.smithi076.stdout: "inode_count": 1, 2023-11-09T03:53:41.237 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_epoch": 117, 2023-11-09T03:53:41.237 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch": 112, 2023-11-09T03:53:41.237 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch_barrier": 0, 2023-11-09T03:53:41.237 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklisted": false, 2023-11-09T03:53:41.237 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs" 2023-11-09T03:53:41.238 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:53:41.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:53:41.245 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:53:41.245 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-11-09T03:53:41.246 DEBUG:teuthology.orchestra.run.smithi076:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-09T03:53:41.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:53:41.305 INFO:teuthology.orchestra.run.smithi076.stdout:fuseblk 2023-11-09T03:53:41.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:53:41.311 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-09T03:53:41.311 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:53:41.312 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-09T03:53:41.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:53:41.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:53:41.376 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 flush journal 2023-11-09T03:53:41.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.478+0000 7fdca026f640 1 Processor -- start 2023-11-09T03:53:41.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.478+0000 7fdca026f640 1 -- start start 2023-11-09T03:53:41.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.479+0000 7fdca026f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdc9815d100 0x7fdc980ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:41.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.479+0000 7fdca026f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc980b32b0 con 0x7fdc980d1be0 2023-11-09T03:53:41.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.479+0000 7fdca026f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc980cee10 con 0x7fdc9815d4e0 2023-11-09T03:53:41.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.479+0000 7fdca026f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc980b2980 con 0x7fdc9815d100 2023-11-09T03:53:41.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.479+0000 7fdc9e00b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdc9815d100 0x7fdc980ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:41.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.479+0000 7fdc9e00b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdc9815d100 0x7fdc980ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37788/0 (socket says 172.21.15.76:37788) 2023-11-09T03:53:41.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.479+0000 7fdc9e00b640 1 -- 172.21.15.76:0/1453130618 learned_addr learned my addr 172.21.15.76:0/1453130618 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:41.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.479+0000 7fdc9e80c640 1 --1- 172.21.15.76:0/1453130618 >> v1:172.21.15.76:6789/0 conn(0x7fdc980d1be0 0x7fdc980b3740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36140/0 (socket says 172.21.15.76:36140) 2023-11-09T03:53:41.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.479+0000 7fdc9e00b640 1 -- 172.21.15.76:0/1453130618 >> v1:172.21.15.113:6789/0 conn(0x7fdc9815d4e0 legacy=0x7fdc980d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:41.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.479+0000 7fdc9e00b640 1 -- 172.21.15.76:0/1453130618 >> v1:172.21.15.76:6789/0 conn(0x7fdc980d1be0 legacy=0x7fdc980b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:41.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.479+0000 7fdc9e00b640 1 -- 172.21.15.76:0/1453130618 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc980b2c60 con 0x7fdc9815d100 2023-11-09T03:53:41.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.479+0000 7fdc9d009640 1 -- 172.21.15.76:0/1453130618 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 867882099 0 0) 0x7fdc980b32b0 con 0x7fdc980d1be0 2023-11-09T03:53:41.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.480+0000 7fdc9e00b640 1 --2- 172.21.15.76:0/1453130618 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc9815d100 0x7fdc980ce510 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fdc94009850 tx=0x7fdc9402ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219b3f462841c201 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:41.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.480+0000 7fdc9d009640 1 -- 172.21.15.76:0/1453130618 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc94004230 con 0x7fdc9815d100 2023-11-09T03:53:41.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.480+0000 7fdc9d009640 1 -- 172.21.15.76:0/1453130618 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc94004390 con 0x7fdc9815d100 2023-11-09T03:53:41.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.480+0000 7fdc9d009640 1 -- 172.21.15.76:0/1453130618 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9402f830 con 0x7fdc9815d100 2023-11-09T03:53:41.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.480+0000 7fdca026f640 1 -- 172.21.15.76:0/1453130618 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc9815d100 msgr2=0x7fdc980ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:41.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.480+0000 7fdca026f640 1 --2- 172.21.15.76:0/1453130618 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc9815d100 0x7fdc980ce510 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fdc94009850 tx=0x7fdc9402ba10 comp rx=0 tx=0).stop 2023-11-09T03:53:41.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.480+0000 7fdca026f640 1 -- 172.21.15.76:0/1453130618 shutdown_connections 2023-11-09T03:53:41.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.480+0000 7fdca026f640 1 --2- 172.21.15.76:0/1453130618 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc9815d100 0x7fdc980ce510 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.480+0000 7fdca026f640 1 -- 172.21.15.76:0/1453130618 >> 172.21.15.76:0/1453130618 conn(0x7fdc980d2ef0 msgr2=0x7fdc980d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:41.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.480+0000 7fdca026f640 1 -- 172.21.15.76:0/1453130618 shutdown_connections 2023-11-09T03:53:41.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.481+0000 7fdca026f640 1 -- 172.21.15.76:0/1453130618 wait complete. 2023-11-09T03:53:41.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.482+0000 7fdca026f640 1 Processor -- start 2023-11-09T03:53:41.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.482+0000 7fdca026f640 1 -- start start 2023-11-09T03:53:41.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.482+0000 7fdca026f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc980d1be0 0x7fdc9814c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:41.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.482+0000 7fdca026f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc9815d100 0x7fdc980c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:41.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.482+0000 7fdca026f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc9815d4e0 0x7fdc981f2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:41.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.482+0000 7fdca026f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc980cb960 con 0x7fdc980d1be0 2023-11-09T03:53:41.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.482+0000 7fdca026f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc980ca8c0 con 0x7fdc9815d4e0 2023-11-09T03:53:41.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.482+0000 7fdca026f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc980caba0 con 0x7fdc9815d100 2023-11-09T03:53:41.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.482+0000 7fdc9e00b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc980d1be0 0x7fdc9814c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:41.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.482+0000 7fdc9e00b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc980d1be0 0x7fdc9814c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37794/0 (socket says 172.21.15.76:37794) 2023-11-09T03:53:41.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.482+0000 7fdc9e00b640 1 -- 172.21.15.76:0/487228059 learned_addr learned my addr 172.21.15.76:0/487228059 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:41.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.483+0000 7fdc9e80c640 1 --2- 172.21.15.76:0/487228059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc9815d4e0 0x7fdc981f2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:41.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.483+0000 7fdc9d80a640 1 --2- 172.21.15.76:0/487228059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc9815d100 0x7fdc980c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:41.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.483+0000 7fdc9e00b640 1 -- 172.21.15.76:0/487228059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc9815d100 msgr2=0x7fdc980c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:41.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.483+0000 7fdc9e00b640 1 --2- 172.21.15.76:0/487228059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc9815d100 0x7fdc980c9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.483+0000 7fdc9e00b640 1 -- 172.21.15.76:0/487228059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc9815d4e0 msgr2=0x7fdc981f2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:41.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.483+0000 7fdc9e00b640 1 --2- 172.21.15.76:0/487228059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc9815d4e0 0x7fdc981f2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.483+0000 7fdc9e00b640 1 -- 172.21.15.76:0/487228059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc94009460 con 0x7fdc980d1be0 2023-11-09T03:53:41.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.483+0000 7fdc9e00b640 1 --2- 172.21.15.76:0/487228059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc980d1be0 0x7fdc9814c4b0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fdc94000c00 tx=0x7fdc940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:41.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.483+0000 7fdc8effd640 1 -- 172.21.15.76:0/487228059 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9403d070 con 0x7fdc980d1be0 2023-11-09T03:53:41.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.483+0000 7fdc8effd640 1 -- 172.21.15.76:0/487228059 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc94004500 con 0x7fdc980d1be0 2023-11-09T03:53:41.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.483+0000 7fdc8effd640 1 -- 172.21.15.76:0/487228059 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9402fe30 con 0x7fdc980d1be0 2023-11-09T03:53:41.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.483+0000 7fdca026f640 1 -- 172.21.15.76:0/487228059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc980cae80 con 0x7fdc980d1be0 2023-11-09T03:53:41.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.483+0000 7fdca026f640 1 -- 172.21.15.76:0/487228059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc980cb440 con 0x7fdc980d1be0 2023-11-09T03:53:41.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.484+0000 7fdc8effd640 1 -- 172.21.15.76:0/487228059 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdc94003610 con 0x7fdc980d1be0 2023-11-09T03:53:41.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.485+0000 7fdc8effd640 1 --2- 172.21.15.76:0/487228059 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdc7c06aea0 0x7fdc7c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:41.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.485+0000 7fdc8effd640 1 -- 172.21.15.76:0/487228059 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8167+0+0 (secure 0 0 0) 0x7fdc940abe00 con 0x7fdc980d1be0 2023-11-09T03:53:41.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.485+0000 7fdc9d80a640 1 --2- 172.21.15.76:0/487228059 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdc7c06aea0 0x7fdc7c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:41.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.486+0000 7fdc9d80a640 1 --2- 172.21.15.76:0/487228059 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdc7c06aea0 0x7fdc7c06d360 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fdc981e9d90 tx=0x7fdc88009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:41.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.486+0000 7fdca2e01740 1 Processor -- start 2023-11-09T03:53:41.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.486+0000 7fdca2e01740 1 -- start start 2023-11-09T03:53:41.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.487+0000 7fdca2e01740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596fef4f4c0 0x5596fef4f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:41.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.487+0000 7fdca2e01740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596fef1f6d0 0x5596fef4c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:41.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.487+0000 7fdca2e01740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5596fef23040 0x5596fef4cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:41.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.487+0000 7fdca2e01740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596fee92d70 con 0x5596fef4f4c0 2023-11-09T03:53:41.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.487+0000 7fdca2e01740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596fee90460 con 0x5596fef1f6d0 2023-11-09T03:53:41.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.487+0000 7fdca2e01740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596fee8bd50 con 0x5596fef23040 2023-11-09T03:53:41.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.487+0000 7fdc9e00b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596fef4f4c0 0x5596fef4f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:41.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.487+0000 7fdc9e00b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596fef4f4c0 0x5596fef4f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37800/0 (socket says 172.21.15.76:37800) 2023-11-09T03:53:41.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.487+0000 7fdc9e00b640 1 -- 172.21.15.76:0/3832803303 learned_addr learned my addr 172.21.15.76:0/3832803303 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:41.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.487+0000 7fdc9d80a640 1 --2- 172.21.15.76:0/3832803303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596fef1f6d0 0x5596fef4c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:41.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.487+0000 7fdc9e80c640 1 --2- 172.21.15.76:0/3832803303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5596fef23040 0x5596fef4cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:41.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.487+0000 7fdc9e00b640 1 -- 172.21.15.76:0/3832803303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5596fef23040 msgr2=0x5596fef4cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:41.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.487+0000 7fdc9e00b640 1 --2- 172.21.15.76:0/3832803303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5596fef23040 0x5596fef4cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.487+0000 7fdc9e00b640 1 -- 172.21.15.76:0/3832803303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596fef1f6d0 msgr2=0x5596fef4c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:41.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.487+0000 7fdc9e00b640 1 --2- 172.21.15.76:0/3832803303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596fef1f6d0 0x5596fef4c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.488+0000 7fdc9e00b640 1 -- 172.21.15.76:0/3832803303 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596fee4ee30 con 0x5596fef4f4c0 2023-11-09T03:53:41.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.488+0000 7fdc9e80c640 1 --2- 172.21.15.76:0/3832803303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5596fef23040 0x5596fef4cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:53:41.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.488+0000 7fdc9e00b640 1 --2- 172.21.15.76:0/3832803303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596fef4f4c0 0x5596fef4f8a0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fdc94081fa0 tx=0x7fdc94004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d78c0392f46118a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:41.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.488+0000 7fdc7bfff640 1 -- 172.21.15.76:0/3832803303 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9407e320 con 0x5596fef4f4c0 2023-11-09T03:53:41.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.488+0000 7fdc7bfff640 1 -- 172.21.15.76:0/3832803303 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc940824d0 con 0x5596fef4f4c0 2023-11-09T03:53:41.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.488+0000 7fdc7bfff640 1 -- 172.21.15.76:0/3832803303 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9407e0d0 con 0x5596fef4f4c0 2023-11-09T03:53:41.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.488+0000 7fdca2e01740 1 -- 172.21.15.76:0/3832803303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596fef4f4c0 msgr2=0x5596fef4f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:41.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.488+0000 7fdca2e01740 1 --2- 172.21.15.76:0/3832803303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596fef4f4c0 0x5596fef4f8a0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fdc94081fa0 tx=0x7fdc94004320 comp rx=0 tx=0).stop 2023-11-09T03:53:41.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.489+0000 7fdca2e01740 1 -- 172.21.15.76:0/3832803303 shutdown_connections 2023-11-09T03:53:41.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.489+0000 7fdca2e01740 1 --2- 172.21.15.76:0/3832803303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5596fef23040 0x5596fef4cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.489+0000 7fdca2e01740 1 --2- 172.21.15.76:0/3832803303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596fef1f6d0 0x5596fef4c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.489+0000 7fdca2e01740 1 --2- 172.21.15.76:0/3832803303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596fef4f4c0 0x5596fef4f8a0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.489+0000 7fdca2e01740 1 -- 172.21.15.76:0/3832803303 >> 172.21.15.76:0/3832803303 conn(0x5596fef219e0 msgr2=0x5596fef6a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:41.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.489+0000 7fdca2e01740 1 -- 172.21.15.76:0/3832803303 shutdown_connections 2023-11-09T03:53:41.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.489+0000 7fdca2e01740 1 -- 172.21.15.76:0/3832803303 wait complete. 2023-11-09T03:53:41.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.489+0000 7fdca2e01740 1 -- start start 2023-11-09T03:53:41.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.489+0000 7fdca2e01740 1 Processor -- start 2023-11-09T03:53:41.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.490+0000 7fdca2e01740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596fef23040 0x5596fef354c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:41.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.490+0000 7fdca2e01740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5596fef4f4c0 0x5596fef33430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:41.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.490+0000 7fdca2e01740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596fef33970 0x5596fef47b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:41.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.490+0000 7fdca2e01740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596fee45ee0 con 0x5596fef23040 2023-11-09T03:53:41.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.490+0000 7fdca2e01740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596fee45b80 con 0x5596fef33970 2023-11-09T03:53:41.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.490+0000 7fdca2e01740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596fee0b3a0 con 0x5596fef4f4c0 2023-11-09T03:53:41.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.490+0000 7fdca2e01740 10 client.0 _ll_register_callbacks cb 0x5596fee0f8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:53:41.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.490+0000 7fdc9e00b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596fef23040 0x5596fef354c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:41.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.490+0000 7fdc9e00b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596fef23040 0x5596fef354c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37806/0 (socket says 172.21.15.76:37806) 2023-11-09T03:53:41.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.490+0000 7fdc9e00b640 1 -- 172.21.15.76:0/908616983 learned_addr learned my addr 172.21.15.76:0/908616983 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:41.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.490+0000 7fdc9d80a640 1 --2- 172.21.15.76:0/908616983 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5596fef4f4c0 0x5596fef33430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:41.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.490+0000 7fdc9e80c640 1 --2- 172.21.15.76:0/908616983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596fef33970 0x5596fef47b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:41.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.490+0000 7fdc9e00b640 1 -- 172.21.15.76:0/908616983 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5596fef4f4c0 msgr2=0x5596fef33430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:41.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.490+0000 7fdc9e00b640 1 --2- 172.21.15.76:0/908616983 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5596fef4f4c0 0x5596fef33430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.490+0000 7fdc9e00b640 1 -- 172.21.15.76:0/908616983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596fef33970 msgr2=0x5596fef47b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:41.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.490+0000 7fdc9e00b640 1 --2- 172.21.15.76:0/908616983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596fef33970 0x5596fef47b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.490+0000 7fdc9e00b640 1 -- 172.21.15.76:0/908616983 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc980cb440 con 0x5596fef23040 2023-11-09T03:53:41.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.491+0000 7fdc9e00b640 1 --2- 172.21.15.76:0/908616983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596fef23040 0x5596fef354c0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fdc940a99e0 tx=0x7fdc9402ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8845c091190c68 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:41.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.491+0000 7fdc797fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:53:41.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.491+0000 7fdc797fa640 1 -- 172.21.15.76:0/908616983 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9407a630 con 0x5596fef23040 2023-11-09T03:53:41.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.491+0000 7fdc797fa640 1 -- 172.21.15.76:0/908616983 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8167+0+0 (secure 0 0 0) 0x7fdc9407e0d0 con 0x5596fef23040 2023-11-09T03:53:41.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.491+0000 7fdca2e01740 1 -- 172.21.15.76:0/908616983 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596fee84d30 con 0x5596fef23040 2023-11-09T03:53:41.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.491+0000 7fdc797fa640 1 -- 172.21.15.76:0/908616983 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc940796c0 con 0x5596fef23040 2023-11-09T03:53:41.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.491+0000 7fdca2e01740 10 client.8949 fetch_fsmap learned FSMap version 117 2023-11-09T03:53:41.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.491+0000 7fdca2e01740 1 -- 172.21.15.76:0/908616983 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=117,monmap=2+}) v3 -- 0x5596fee8a3d0 con 0x5596fef23040 2023-11-09T03:53:41.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.491+0000 7fdc797fa640 1 -- 172.21.15.76:0/908616983 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5596fee84d30 con 0x5596fef23040 2023-11-09T03:53:41.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.491+0000 7fdc797fa640 1 -- 172.21.15.76:0/908616983 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 117) v1 ==== 3306+0+0 (secure 0 0 0) 0x7fdc94082040 con 0x5596fef23040 2023-11-09T03:53:41.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.491+0000 7fdca2e01740 10 client.8949 fetch_fsmap finished waiting for FSMap version 117 2023-11-09T03:53:41.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.491+0000 7fdca2e01740 10 client.8949 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2023-11-09T03:53:41.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.491+0000 7fdca2e01740 20 client.8949 populate_metadata read hostname 'smithi076' 2023-11-09T03:53:41.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.491+0000 7fdca2e01740 1 --2- 172.21.15.76:0/908616983 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x5596fef243c0 0x5596fef247c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:41.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.491+0000 7fdca2e01740 4 client.8949 mds_command: new command op to 8568 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:53:41.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.491+0000 7fdca2e01740 1 -- 172.21.15.76:0/908616983 --> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596fee84a50 con 0x5596fef243c0 2023-11-09T03:53:41.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.491+0000 7fdc9d80a640 1 --2- 172.21.15.76:0/908616983 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x5596fef243c0 0x5596fef247c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:41.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.492+0000 7fdc9d80a640 1 --2- 172.21.15.76:0/908616983 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x5596fef243c0 0x5596fef247c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af310cc4bdb8ff31 server_cookie=59f3fae459ead66b in_seq=0 out_seq=0 2023-11-09T03:53:41.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.492+0000 7fdc797fa640 10 client.8949 ms_handle_connect on v2:172.21.15.76:6832/153465374 2023-11-09T03:53:41.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.492+0000 7fdc797fa640 1 -- 172.21.15.76:0/908616983 <== mds.0 v2:172.21.15.76:6832/153465374 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5596fee84a50 con 0x5596fef243c0 2023-11-09T03:53:41.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.492+0000 7fdc797fa640 10 client.8949 handle_command_reply: tid=0 2023-11-09T03:53:41.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.492+0000 7fdca2e01740 1 client.8949 shutdown 2023-11-09T03:53:41.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.492+0000 7fdca2e01740 1 -- 172.21.15.76:0/908616983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596fef23040 msgr2=0x5596fef354c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:41.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.492+0000 7fdca2e01740 1 --2- 172.21.15.76:0/908616983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596fef23040 0x5596fef354c0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fdc940a99e0 tx=0x7fdc9402ff90 comp rx=0 tx=0).stop 2023-11-09T03:53:41.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.492+0000 7fdca2e01740 1 -- 172.21.15.76:0/908616983 shutdown_connections 2023-11-09T03:53:41.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.492+0000 7fdca2e01740 1 --2- 172.21.15.76:0/908616983 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x5596fef243c0 0x5596fef247c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.492+0000 7fdca2e01740 1 --2- 172.21.15.76:0/908616983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596fef33970 0x5596fef47b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.492+0000 7fdca2e01740 1 --2- 172.21.15.76:0/908616983 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5596fef4f4c0 0x5596fef33430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.492+0000 7fdca2e01740 1 --2- 172.21.15.76:0/908616983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596fef23040 0x5596fef354c0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.492+0000 7fdc797fa640 0 client.8949 ms_handle_reset on v2:172.21.15.76:6832/153465374 2023-11-09T03:53:41.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.492+0000 7fdca2e01740 1 -- 172.21.15.76:0/908616983 >> 172.21.15.76:0/908616983 conn(0x5596fef1f6d0 msgr2=0x5596fef53270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:41.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.492+0000 7fdca2e01740 1 -- 172.21.15.76:0/908616983 shutdown_connections 2023-11-09T03:53:41.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.492+0000 7fdca2e01740 1 -- 172.21.15.76:0/908616983 wait complete. 2023-11-09T03:53:41.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.493+0000 7fdca2e01740 20 client.8949 trim_cache size 0 max 16384 2023-11-09T03:53:41.585 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.506+0000 7fdca2e01740 1 Processor -- start 2023-11-09T03:53:41.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.506+0000 7fdca2e01740 1 -- start start 2023-11-09T03:53:41.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.506+0000 7fdca2e01740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596fef23040 0x5596fef32d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:41.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.506+0000 7fdca2e01740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5596fef4f4c0 0x5596fef6b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:41.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.506+0000 7fdca2e01740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596fef1fec0 0x5596fef6baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:41.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.506+0000 7fdca2e01740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596fef21280 con 0x5596fef1fec0 2023-11-09T03:53:41.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.506+0000 7fdca2e01740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596fef4b9f0 con 0x5596fef23040 2023-11-09T03:53:41.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.506+0000 7fdca2e01740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596fef1fb20 con 0x5596fef4f4c0 2023-11-09T03:53:41.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.506+0000 7fdc9e80c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596fef1fec0 0x5596fef6baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:41.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.506+0000 7fdc9e80c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596fef1fec0 0x5596fef6baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37822/0 (socket says 172.21.15.76:37822) 2023-11-09T03:53:41.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.506+0000 7fdc9e80c640 1 -- 172.21.15.76:0/1364813495 learned_addr learned my addr 172.21.15.76:0/1364813495 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:41.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.506+0000 7fdc9e00b640 1 --2- 172.21.15.76:0/1364813495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596fef23040 0x5596fef32d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:41.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.506+0000 7fdc9d80a640 1 --2- 172.21.15.76:0/1364813495 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5596fef4f4c0 0x5596fef6b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:41.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.507+0000 7fdc9e80c640 1 -- 172.21.15.76:0/1364813495 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5596fef4f4c0 msgr2=0x5596fef6b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:41.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.507+0000 7fdc9e80c640 1 --2- 172.21.15.76:0/1364813495 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5596fef4f4c0 0x5596fef6b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.507+0000 7fdc9e80c640 1 -- 172.21.15.76:0/1364813495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596fef23040 msgr2=0x5596fef32d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:41.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.507+0000 7fdc9e80c640 1 --2- 172.21.15.76:0/1364813495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596fef23040 0x5596fef32d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.507+0000 7fdc9e80c640 1 -- 172.21.15.76:0/1364813495 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596fee27ea0 con 0x5596fef1fec0 2023-11-09T03:53:41.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.507+0000 7fdc9d80a640 1 --2- 172.21.15.76:0/1364813495 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5596fef4f4c0 0x5596fef6b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:53:41.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.507+0000 7fdc9e80c640 1 --2- 172.21.15.76:0/1364813495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596fef1fec0 0x5596fef6baa0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fdc90001560 tx=0x7fdc900076a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db99252cd868e33 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:41.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.508+0000 7fdc8cdff640 1 -- 172.21.15.76:0/1364813495 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc900078f0 con 0x5596fef1fec0 2023-11-09T03:53:41.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.508+0000 7fdc8cdff640 1 -- 172.21.15.76:0/1364813495 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc90001c10 con 0x5596fef1fec0 2023-11-09T03:53:41.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.508+0000 7fdc8cdff640 1 -- 172.21.15.76:0/1364813495 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc90001f80 con 0x5596fef1fec0 2023-11-09T03:53:41.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.509+0000 7fdca2e01740 1 -- 172.21.15.76:0/1364813495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596fef1fec0 msgr2=0x5596fef6baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:41.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.509+0000 7fdca2e01740 1 --2- 172.21.15.76:0/1364813495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596fef1fec0 0x5596fef6baa0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fdc90001560 tx=0x7fdc900076a0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.509+0000 7fdca2e01740 1 -- 172.21.15.76:0/1364813495 shutdown_connections 2023-11-09T03:53:41.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.509+0000 7fdca2e01740 1 --2- 172.21.15.76:0/1364813495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596fef1fec0 0x5596fef6baa0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.509+0000 7fdca2e01740 1 --2- 172.21.15.76:0/1364813495 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5596fef4f4c0 0x5596fef6b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.509+0000 7fdca2e01740 1 --2- 172.21.15.76:0/1364813495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596fef23040 0x5596fef32d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.509+0000 7fdca2e01740 1 -- 172.21.15.76:0/1364813495 >> 172.21.15.76:0/1364813495 conn(0x5596fef1f6d0 msgr2=0x5596fef8c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:41.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.509+0000 7fdca2e01740 1 -- 172.21.15.76:0/1364813495 shutdown_connections 2023-11-09T03:53:41.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.509+0000 7fdca2e01740 1 -- 172.21.15.76:0/1364813495 wait complete. 2023-11-09T03:53:41.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.509+0000 7fdca2e01740 1 -- start start 2023-11-09T03:53:41.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.509+0000 7fdca2e01740 1 Processor -- start 2023-11-09T03:53:41.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.510+0000 7fdca2e01740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5596fef1fec0 0x5596fefd4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:41.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.510+0000 7fdca2e01740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596fef23040 0x5596fef4a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:41.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.510+0000 7fdca2e01740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596fef4f4c0 0x5596fef4aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:41.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.510+0000 7fdca2e01740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596fef89b30 con 0x5596fef4f4c0 2023-11-09T03:53:41.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.510+0000 7fdca2e01740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596fef47040 con 0x5596fef23040 2023-11-09T03:53:41.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.510+0000 7fdca2e01740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596fef7cdc0 con 0x5596fef1fec0 2023-11-09T03:53:41.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.510+0000 7fdca2e01740 10 client.0 _ll_register_callbacks cb 0x7fdc94076510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:53:41.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.510+0000 7fdc9e80c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596fef4f4c0 0x5596fef4aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:41.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.510+0000 7fdc9e80c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596fef4f4c0 0x5596fef4aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37834/0 (socket says 172.21.15.76:37834) 2023-11-09T03:53:41.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.510+0000 7fdc9e80c640 1 -- 172.21.15.76:0/2344992963 learned_addr learned my addr 172.21.15.76:0/2344992963 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:41.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.511+0000 7fdc9e00b640 1 --2- 172.21.15.76:0/2344992963 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5596fef1fec0 0x5596fefd4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:41.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.511+0000 7fdc9d80a640 1 --2- 172.21.15.76:0/2344992963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596fef23040 0x5596fef4a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:41.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.511+0000 7fdc9e80c640 1 -- 172.21.15.76:0/2344992963 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5596fef1fec0 msgr2=0x5596fefd4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:41.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.511+0000 7fdc9e80c640 1 --2- 172.21.15.76:0/2344992963 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5596fef1fec0 0x5596fefd4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.511+0000 7fdc9e80c640 1 -- 172.21.15.76:0/2344992963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596fef23040 msgr2=0x5596fef4a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:41.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.511+0000 7fdc9e80c640 1 --2- 172.21.15.76:0/2344992963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596fef23040 0x5596fef4a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.511+0000 7fdc9e80c640 1 -- 172.21.15.76:0/2344992963 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc9000f040 con 0x5596fef4f4c0 2023-11-09T03:53:41.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.511+0000 7fdc9e80c640 1 --2- 172.21.15.76:0/2344992963 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596fef4f4c0 0x5596fef4aad0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fdc90017000 tx=0x7fdc90001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9195871f998b202 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:41.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.511+0000 7fdc797fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:53:41.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.511+0000 7fdc797fa640 1 -- 172.21.15.76:0/2344992963 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc90016680 con 0x5596fef4f4c0 2023-11-09T03:53:41.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.511+0000 7fdc797fa640 1 -- 172.21.15.76:0/2344992963 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8167+0+0 (secure 0 0 0) 0x7fdc900167e0 con 0x5596fef4f4c0 2023-11-09T03:53:41.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.511+0000 7fdca2e01740 1 -- 172.21.15.76:0/2344992963 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc940824d0 con 0x5596fef4f4c0 2023-11-09T03:53:41.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.511+0000 7fdc797fa640 1 -- 172.21.15.76:0/2344992963 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc900020d0 con 0x5596fef4f4c0 2023-11-09T03:53:41.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.511+0000 7fdca2e01740 10 client.8955 fetch_fsmap learned FSMap version 117 2023-11-09T03:53:41.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.511+0000 7fdca2e01740 1 -- 172.21.15.76:0/2344992963 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=117,monmap=2+}) v3 -- 0x5596fee4d690 con 0x5596fef4f4c0 2023-11-09T03:53:41.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.511+0000 7fdc797fa640 1 -- 172.21.15.76:0/2344992963 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc940824d0 con 0x5596fef4f4c0 2023-11-09T03:53:41.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.511+0000 7fdc797fa640 1 -- 172.21.15.76:0/2344992963 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 117) v1 ==== 3306+0+0 (secure 0 0 0) 0x7fdc900169a0 con 0x5596fef4f4c0 2023-11-09T03:53:41.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.511+0000 7fdca2e01740 10 client.8955 fetch_fsmap finished waiting for FSMap version 117 2023-11-09T03:53:41.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.511+0000 7fdca2e01740 10 client.8955 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2023-11-09T03:53:41.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.511+0000 7fdca2e01740 20 client.8955 populate_metadata read hostname 'smithi076' 2023-11-09T03:53:41.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.511+0000 7fdca2e01740 1 --2- 172.21.15.76:0/2344992963 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x5596fef8c090 0x5596fef237e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:41.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.511+0000 7fdca2e01740 4 client.8955 mds_command: new command op to 8568 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-11-09T03:53:41.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.511+0000 7fdca2e01740 1 -- 172.21.15.76:0/2344992963 --> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5596feb8c7e0 con 0x5596fef8c090 2023-11-09T03:53:41.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.512+0000 7fdc9e00b640 1 --2- 172.21.15.76:0/2344992963 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x5596fef8c090 0x5596fef237e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:41.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.512+0000 7fdc797fa640 10 client.8955 ms_handle_connect on v2:172.21.15.76:6832/153465374 2023-11-09T03:53:41.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.512+0000 7fdc9e00b640 1 --2- 172.21.15.76:0/2344992963 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x5596fef8c090 0x5596fef237e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=273443ec13de11bb server_cookie=bb201bd3858812c5 in_seq=0 out_seq=0 2023-11-09T03:53:41.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.523+0000 7fdc797fa640 1 -- 172.21.15.76:0/2344992963 <== mds.0 v2:172.21.15.76:6832/153465374 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5596feb8c7e0 con 0x5596fef8c090 2023-11-09T03:53:41.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.523+0000 7fdc797fa640 10 client.8955 handle_command_reply: tid=0 2023-11-09T03:53:41.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.523+0000 7fdca2e01740 1 client.8955 shutdown 2023-11-09T03:53:41.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.523+0000 7fdca2e01740 1 -- 172.21.15.76:0/2344992963 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596fef4f4c0 msgr2=0x5596fef4aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:41.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.523+0000 7fdca2e01740 1 --2- 172.21.15.76:0/2344992963 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596fef4f4c0 0x5596fef4aad0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fdc90017000 tx=0x7fdc90001250 comp rx=0 tx=0).stop 2023-11-09T03:53:41.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.524+0000 7fdca2e01740 1 -- 172.21.15.76:0/2344992963 shutdown_connections 2023-11-09T03:53:41.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.524+0000 7fdca2e01740 1 --2- 172.21.15.76:0/2344992963 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x5596fef8c090 0x5596fef237e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.524+0000 7fdca2e01740 1 --2- 172.21.15.76:0/2344992963 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596fef4f4c0 0x5596fef4aad0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.524+0000 7fdca2e01740 1 --2- 172.21.15.76:0/2344992963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596fef23040 0x5596fef4a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.524+0000 7fdc797fa640 0 client.8955 ms_handle_reset on v2:172.21.15.76:6832/153465374 2023-11-09T03:53:41.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.524+0000 7fdca2e01740 1 --2- 172.21.15.76:0/2344992963 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5596fef1fec0 0x5596fefd4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.524+0000 7fdca2e01740 1 -- 172.21.15.76:0/2344992963 >> 172.21.15.76:0/2344992963 conn(0x5596fef1f6d0 msgr2=0x5596fef6b560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:41.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.524+0000 7fdca2e01740 1 -- 172.21.15.76:0/2344992963 shutdown_connections 2023-11-09T03:53:41.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.524+0000 7fdca2e01740 1 -- 172.21.15.76:0/2344992963 wait complete. 2023-11-09T03:53:41.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.524+0000 7fdca2e01740 20 client.8955 trim_cache size 0 max 16384 2023-11-09T03:53:41.601 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:53:41.601 INFO:teuthology.orchestra.run.smithi076.stdout: "message": "", 2023-11-09T03:53:41.601 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2023-11-09T03:53:41.602 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:53:41.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.524+0000 7fdca026f640 1 -- 172.21.15.76:0/487228059 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdc7c06aea0 msgr2=0x7fdc7c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:41.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.524+0000 7fdca026f640 1 --2- 172.21.15.76:0/487228059 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdc7c06aea0 0x7fdc7c06d360 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fdc981e9d90 tx=0x7fdc88009040 comp rx=0 tx=0).stop 2023-11-09T03:53:41.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.524+0000 7fdca026f640 1 -- 172.21.15.76:0/487228059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc980d1be0 msgr2=0x7fdc9814c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:41.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.524+0000 7fdca026f640 1 --2- 172.21.15.76:0/487228059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc980d1be0 0x7fdc9814c4b0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fdc94000c00 tx=0x7fdc940046c0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.524+0000 7fdca026f640 1 -- 172.21.15.76:0/487228059 shutdown_connections 2023-11-09T03:53:41.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.524+0000 7fdca026f640 1 --2- 172.21.15.76:0/487228059 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdc7c06aea0 0x7fdc7c06d360 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.524+0000 7fdca026f640 1 --2- 172.21.15.76:0/487228059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc9815d4e0 0x7fdc981f2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.524+0000 7fdca026f640 1 --2- 172.21.15.76:0/487228059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc9815d100 0x7fdc980c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.524+0000 7fdca026f640 1 --2- 172.21.15.76:0/487228059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc980d1be0 0x7fdc9814c4b0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.524+0000 7fdca026f640 1 -- 172.21.15.76:0/487228059 >> 172.21.15.76:0/487228059 conn(0x7fdc980d2ef0 msgr2=0x7fdc980b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:41.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.524+0000 7fdca026f640 1 -- 172.21.15.76:0/487228059 shutdown_connections 2023-11-09T03:53:41.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.524+0000 7fdca026f640 1 -- 172.21.15.76:0/487228059 wait complete. 2023-11-09T03:53:41.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:53:41.606 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir d1/dir/.snap/*) 2023-11-09T03:53:41.637 INFO:tasks.cephfs.test_snapshots:testing snapclient mds_kill_mdstable_at=3 2023-11-09T03:53:41.637 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:53:41.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.776+0000 7f8b03010640 1 Processor -- start 2023-11-09T03:53:41.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.776+0000 7f8b03010640 1 -- start start 2023-11-09T03:53:41.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.777+0000 7f8b03010640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8afc15b200 0x7f8afc0ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:41.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.777+0000 7f8b03010640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8afc0afda0 0x7f8afc0cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:41.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.777+0000 7f8b03010640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8afc0cdc20 con 0x7f8afc0cd7f0 2023-11-09T03:53:41.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.777+0000 7f8b03010640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8afc0cde80 con 0x7f8afc0afda0 2023-11-09T03:53:41.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.777+0000 7f8b03010640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8afc0cf670 con 0x7f8afc15b200 2023-11-09T03:53:41.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.777+0000 7f8afbfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8afc0afda0 0x7f8afc0cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:41.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.777+0000 7f8afbfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8afc0afda0 0x7f8afc0cd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37838/0 (socket says 172.21.15.76:37838) 2023-11-09T03:53:41.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.777+0000 7f8afbfff640 1 -- 172.21.15.76:0/586772675 learned_addr learned my addr 172.21.15.76:0/586772675 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:41.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.778+0000 7f8b00dac640 1 --2- 172.21.15.76:0/586772675 >> v2:172.21.15.113:3300/0 conn(0x7f8afc15b200 0x7f8afc0ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:41.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.778+0000 7f8afbfff640 1 -- 172.21.15.76:0/586772675 >> v2:172.21.15.113:3300/0 conn(0x7f8afc15b200 msgr2=0x7f8afc0ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:41.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.778+0000 7f8afbfff640 1 --2- 172.21.15.76:0/586772675 >> v2:172.21.15.113:3300/0 conn(0x7f8afc15b200 0x7f8afc0ceb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.778+0000 7f8afbfff640 1 -- 172.21.15.76:0/586772675 >> v1:172.21.15.113:6789/0 conn(0x7f8afc0cd7f0 legacy=0x7f8afc0cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:41.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.778+0000 7f8afbfff640 1 -- 172.21.15.76:0/586772675 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8afc0cf950 con 0x7f8afc0afda0 2023-11-09T03:53:41.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.778+0000 7f8b00dac640 1 --2- 172.21.15.76:0/586772675 >> v2:172.21.15.113:3300/0 conn(0x7f8afc15b200 0x7f8afc0ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:53:41.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.778+0000 7f8afbfff640 1 --2- 172.21.15.76:0/586772675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8afc0afda0 0x7f8afc0cd2b0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f8afc070130 tx=0x7f8ae802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950e22e6d1f4b036 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:41.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.779+0000 7f8afb7fe640 1 -- 172.21.15.76:0/586772675 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8031870 con 0x7f8afc0afda0 2023-11-09T03:53:41.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.779+0000 7f8afb7fe640 1 -- 172.21.15.76:0/586772675 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae80319d0 con 0x7f8afc0afda0 2023-11-09T03:53:41.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.779+0000 7f8afb7fe640 1 -- 172.21.15.76:0/586772675 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8031d60 con 0x7f8afc0afda0 2023-11-09T03:53:41.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.779+0000 7f8b03010640 1 -- 172.21.15.76:0/586772675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8afc0afda0 msgr2=0x7f8afc0cd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:41.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.779+0000 7f8b03010640 1 --2- 172.21.15.76:0/586772675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8afc0afda0 0x7f8afc0cd2b0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f8afc070130 tx=0x7f8ae802b910 comp rx=0 tx=0).stop 2023-11-09T03:53:41.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.779+0000 7f8b03010640 1 -- 172.21.15.76:0/586772675 shutdown_connections 2023-11-09T03:53:41.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.779+0000 7f8b03010640 1 --2- 172.21.15.76:0/586772675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8afc0afda0 0x7f8afc0cd2b0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.779+0000 7f8b03010640 1 --2- 172.21.15.76:0/586772675 >> v2:172.21.15.113:3300/0 conn(0x7f8afc15b200 0x7f8afc0ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.779+0000 7f8b03010640 1 -- 172.21.15.76:0/586772675 >> 172.21.15.76:0/586772675 conn(0x7f8afc0d0c30 msgr2=0x7f8afc0d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:41.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.779+0000 7f8b03010640 1 -- 172.21.15.76:0/586772675 shutdown_connections 2023-11-09T03:53:41.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.779+0000 7f8b03010640 1 -- 172.21.15.76:0/586772675 wait complete. 2023-11-09T03:53:41.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.780+0000 7f8b03010640 1 Processor -- start 2023-11-09T03:53:41.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.780+0000 7f8b03010640 1 -- start start 2023-11-09T03:53:41.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.781+0000 7f8b03010640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8afc0afda0 0x7f8afc0c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:41.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.781+0000 7f8b03010640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8afc0cd7f0 0x7f8afc1f0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:41.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.781+0000 7f8b03010640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8afc15b200 0x7f8afc1f07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:41.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.781+0000 7f8b03010640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc1f1a90 con 0x7f8afc0cd7f0 2023-11-09T03:53:41.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.781+0000 7f8b03010640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc0c4a00 con 0x7f8afc15b200 2023-11-09T03:53:41.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.781+0000 7f8b03010640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc1f0e30 con 0x7f8afc0afda0 2023-11-09T03:53:41.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.781+0000 7f8afbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8afc0cd7f0 0x7f8afc1f0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:41.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.781+0000 7f8afbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8afc0cd7f0 0x7f8afc1f0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37850/0 (socket says 172.21.15.76:37850) 2023-11-09T03:53:41.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.781+0000 7f8afbfff640 1 -- 172.21.15.76:0/3468152989 learned_addr learned my addr 172.21.15.76:0/3468152989 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:41.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.781+0000 7f8b00dac640 1 --2- 172.21.15.76:0/3468152989 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8afc0afda0 0x7f8afc0c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:41.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.781+0000 7f8b015ad640 1 --2- 172.21.15.76:0/3468152989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8afc15b200 0x7f8afc1f07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:41.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.782+0000 7f8afbfff640 1 -- 172.21.15.76:0/3468152989 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8afc0afda0 msgr2=0x7f8afc0c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:41.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.782+0000 7f8afbfff640 1 --2- 172.21.15.76:0/3468152989 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8afc0afda0 0x7f8afc0c7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.782+0000 7f8afbfff640 1 -- 172.21.15.76:0/3468152989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8afc15b200 msgr2=0x7f8afc1f07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:41.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.782+0000 7f8afbfff640 1 --2- 172.21.15.76:0/3468152989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8afc15b200 0x7f8afc1f07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.782+0000 7f8afbfff640 1 -- 172.21.15.76:0/3468152989 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae80094d0 con 0x7f8afc0cd7f0 2023-11-09T03:53:41.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.782+0000 7f8afbfff640 1 --2- 172.21.15.76:0/3468152989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8afc0cd7f0 0x7f8afc1f0270 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f8ae802b950 tx=0x7f8ae80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:41.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.782+0000 7f8af97fa640 1 -- 172.21.15.76:0/3468152989 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae803ad30 con 0x7f8afc0cd7f0 2023-11-09T03:53:41.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.782+0000 7f8af97fa640 1 -- 172.21.15.76:0/3468152989 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae80443f0 con 0x7f8afc0cd7f0 2023-11-09T03:53:41.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.782+0000 7f8b03010640 1 -- 172.21.15.76:0/3468152989 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8afc1f1110 con 0x7f8afc0cd7f0 2023-11-09T03:53:41.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.782+0000 7f8af97fa640 1 -- 172.21.15.76:0/3468152989 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae80447a0 con 0x7f8afc0cd7f0 2023-11-09T03:53:41.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.782+0000 7f8b03010640 1 -- 172.21.15.76:0/3468152989 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8afc1f16d0 con 0x7f8afc0cd7f0 2023-11-09T03:53:41.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.783+0000 7f8b03010640 1 -- 172.21.15.76:0/3468152989 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8afc1f1c00 con 0x7f8afc0cd7f0 2023-11-09T03:53:41.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.783+0000 7f8af97fa640 1 -- 172.21.15.76:0/3468152989 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8ae804d450 con 0x7f8afc0cd7f0 2023-11-09T03:53:41.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.784+0000 7f8af97fa640 1 --2- 172.21.15.76:0/3468152989 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8ae006ae70 0x7f8ae006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:41.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.784+0000 7f8af97fa640 1 -- 172.21.15.76:0/3468152989 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8167+0+0 (secure 0 0 0) 0x7f8ae802d040 con 0x7f8afc0cd7f0 2023-11-09T03:53:41.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.784+0000 7f8b00dac640 1 --2- 172.21.15.76:0/3468152989 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8ae006ae70 0x7f8ae006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:41.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.785+0000 7f8b00dac640 1 --2- 172.21.15.76:0/3468152989 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8ae006ae70 0x7f8ae006d330 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f8afc1e2ec0 tx=0x7f8af0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:41.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.788+0000 7f8af97fa640 1 -- 172.21.15.76:0/3468152989 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8ae8076c20 con 0x7f8afc0cd7f0 2023-11-09T03:53:41.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.987+0000 7f8b03010640 1 -- 172.21.15.76:0/3468152989 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8afc0684b0 con 0x7f8afc0cd7f0 2023-11-09T03:53:41.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.988+0000 7f8af97fa640 1 -- 172.21.15.76:0/3468152989 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+4937 (secure 0 0 0) 0x7f8ae8035070 con 0x7f8afc0cd7f0 2023-11-09T03:53:41.989 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:53:41.989 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":117,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:53:34.495220+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":8772},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8772":{"gid":8772,"name":"a","rank":1,"incarnation":113,"state":"up:active","state_seq":7,"addr":"172.21.15.76:6835/2075292158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2075292158},{"type":"v1","addr":"172.21.15.76:6835","nonce":2075292158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:53:41.989 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 117 2023-11-09T03:53:41.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.990+0000 7f8b03010640 1 -- 172.21.15.76:0/3468152989 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8ae006ae70 msgr2=0x7f8ae006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:41.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.990+0000 7f8b03010640 1 --2- 172.21.15.76:0/3468152989 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8ae006ae70 0x7f8ae006d330 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f8afc1e2ec0 tx=0x7f8af0008040 comp rx=0 tx=0).stop 2023-11-09T03:53:41.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.990+0000 7f8b03010640 1 -- 172.21.15.76:0/3468152989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8afc0cd7f0 msgr2=0x7f8afc1f0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:41.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.990+0000 7f8b03010640 1 --2- 172.21.15.76:0/3468152989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8afc0cd7f0 0x7f8afc1f0270 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f8ae802b950 tx=0x7f8ae80046c0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.990+0000 7f8b03010640 1 -- 172.21.15.76:0/3468152989 shutdown_connections 2023-11-09T03:53:41.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.990+0000 7f8b03010640 1 --2- 172.21.15.76:0/3468152989 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f8ae006ae70 0x7f8ae006d330 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.990+0000 7f8b03010640 1 --2- 172.21.15.76:0/3468152989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8afc15b200 0x7f8afc1f07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.990+0000 7f8b03010640 1 --2- 172.21.15.76:0/3468152989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8afc0cd7f0 0x7f8afc1f0270 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.990+0000 7f8b03010640 1 --2- 172.21.15.76:0/3468152989 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8afc0afda0 0x7f8afc0c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:41.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.990+0000 7f8b03010640 1 -- 172.21.15.76:0/3468152989 >> 172.21.15.76:0/3468152989 conn(0x7f8afc0d0c30 msgr2=0x7f8afc0b0180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:41.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.990+0000 7f8b03010640 1 -- 172.21.15.76:0/3468152989 shutdown_connections 2023-11-09T03:53:41.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:41.990+0000 7f8b03010640 1 -- 172.21.15.76:0/3468152989 wait complete. 2023-11-09T03:53:42.001 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2023-11-09T03:53:42.109 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:53:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:53:42.117 INFO:teuthology.orchestra.run.smithi076.stdout:{"last_snap":14,"last_created":12,"last_destroyed":13,"pending_noop":[],"snaps":[{"snapid":12,"ino":1099511627777,"stamp":"2023-11-09T03:52:54.669088+0000","name":"s26","metadata":{}}],"need_to_purge":{"17":[11,13]},"pending_update":[],"pending_destroy":[{"version":28,"removed_snap":12,"seq":14}]} 2023-11-09T03:53:42.117 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 12, "last_destroyed": 13, "last_snap": 14, "need_to_purge": { "17": [ 11, 13 ] }, "pending_destroy": [ { "removed_snap": 12, "seq": 14, "version": 28 } ], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s26", "snapid": 12, "stamp": "2023-11-09T03:52:54.669088+0000" } ] } 2023-11-09T03:53:42.118 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:53:42.257 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.257+0000 7f21ebd5e640 1 Processor -- start 2023-11-09T03:53:42.257 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.257+0000 7f21ebd5e640 1 -- start start 2023-11-09T03:53:42.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.258+0000 7f21ebd5e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21e40b2210 0x7f21e40cf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:42.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.258+0000 7f21ebd5e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f21e415d630 0x7f21e40cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:42.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.258+0000 7f21ebd5e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e40b0150 con 0x7f21e40ceda0 2023-11-09T03:53:42.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.258+0000 7f21ebd5e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e40b1df0 con 0x7f21e415d630 2023-11-09T03:53:42.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.258+0000 7f21ebd5e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e40b1170 con 0x7f21e40b2210 2023-11-09T03:53:42.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.258+0000 7f21ea2fb640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f21e415d630 0x7f21e40cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:42.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.258+0000 7f21ea2fb640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f21e415d630 0x7f21e40cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37854/0 (socket says 172.21.15.76:37854) 2023-11-09T03:53:42.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.258+0000 7f21ea2fb640 1 -- 172.21.15.76:0/2502902943 learned_addr learned my addr 172.21.15.76:0/2502902943 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:42.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.258+0000 7f21e92f9640 1 --2- 172.21.15.76:0/2502902943 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21e40b2210 0x7f21e40cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:42.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.258+0000 7f21ea2fb640 1 -- 172.21.15.76:0/2502902943 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21e40b2210 msgr2=0x7f21e40cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:42.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.258+0000 7f21ea2fb640 1 --2- 172.21.15.76:0/2502902943 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21e40b2210 0x7f21e40cf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:42.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.258+0000 7f21ea2fb640 1 -- 172.21.15.76:0/2502902943 >> v1:172.21.15.113:6789/0 conn(0x7f21e40ceda0 legacy=0x7f21e415cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:42.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.258+0000 7f21ea2fb640 1 -- 172.21.15.76:0/2502902943 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e40b1450 con 0x7f21e415d630 2023-11-09T03:53:42.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.259+0000 7f21ea2fb640 1 --2- 172.21.15.76:0/2502902943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21e415d630 0x7f21e40cf6c0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009870 tx=0x7f21dc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f25112f388410e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:42.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.259+0000 7f21e8af8640 1 -- 172.21.15.76:0/2502902943 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc031640 con 0x7f21e415d630 2023-11-09T03:53:42.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.259+0000 7f21e8af8640 1 -- 172.21.15.76:0/2502902943 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21dc0317a0 con 0x7f21e415d630 2023-11-09T03:53:42.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.259+0000 7f21e8af8640 1 -- 172.21.15.76:0/2502902943 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc031b30 con 0x7f21e415d630 2023-11-09T03:53:42.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.259+0000 7f21ebd5e640 1 -- 172.21.15.76:0/2502902943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21e415d630 msgr2=0x7f21e40cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:42.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.259+0000 7f21ebd5e640 1 --2- 172.21.15.76:0/2502902943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21e415d630 0x7f21e40cf6c0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009870 tx=0x7f21dc02b8f0 comp rx=0 tx=0).stop 2023-11-09T03:53:42.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.259+0000 7f21ebd5e640 1 -- 172.21.15.76:0/2502902943 shutdown_connections 2023-11-09T03:53:42.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.259+0000 7f21ebd5e640 1 --2- 172.21.15.76:0/2502902943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21e415d630 0x7f21e40cf6c0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:42.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.259+0000 7f21ebd5e640 1 --2- 172.21.15.76:0/2502902943 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21e40b2210 0x7f21e40cf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:42.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.259+0000 7f21ebd5e640 1 -- 172.21.15.76:0/2502902943 >> 172.21.15.76:0/2502902943 conn(0x7f21e40d0c30 msgr2=0x7f21e40d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:42.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.260+0000 7f21ebd5e640 1 -- 172.21.15.76:0/2502902943 shutdown_connections 2023-11-09T03:53:42.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.260+0000 7f21ebd5e640 1 -- 172.21.15.76:0/2502902943 wait complete. 2023-11-09T03:53:42.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.261+0000 7f21ebd5e640 1 Processor -- start 2023-11-09T03:53:42.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.261+0000 7f21ebd5e640 1 -- start start 2023-11-09T03:53:42.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.261+0000 7f21ebd5e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21e40b2210 0x7f21e41e5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:42.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.261+0000 7f21ebd5e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21e40ceda0 0x7f21e40cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:42.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.261+0000 7f21ebd5e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21e415d630 0x7f21e40cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:42.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.261+0000 7f21ebd5e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e409e630 con 0x7f21e415d630 2023-11-09T03:53:42.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.262+0000 7f21ebd5e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e41e4cd0 con 0x7f21e40ceda0 2023-11-09T03:53:42.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.262+0000 7f21ebd5e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e41e4fb0 con 0x7f21e40b2210 2023-11-09T03:53:42.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.262+0000 7f21ea2fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21e415d630 0x7f21e40cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:42.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.262+0000 7f21ea2fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21e415d630 0x7f21e40cc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37856/0 (socket says 172.21.15.76:37856) 2023-11-09T03:53:42.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.262+0000 7f21ea2fb640 1 -- 172.21.15.76:0/3009904320 learned_addr learned my addr 172.21.15.76:0/3009904320 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:42.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.262+0000 7f21e92f9640 1 --2- 172.21.15.76:0/3009904320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21e40ceda0 0x7f21e40cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:42.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.262+0000 7f21e9afa640 1 --2- 172.21.15.76:0/3009904320 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21e40b2210 0x7f21e41e5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:42.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.262+0000 7f21ea2fb640 1 -- 172.21.15.76:0/3009904320 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21e40b2210 msgr2=0x7f21e41e5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:42.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.262+0000 7f21ea2fb640 1 --2- 172.21.15.76:0/3009904320 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21e40b2210 0x7f21e41e5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:42.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.262+0000 7f21ea2fb640 1 -- 172.21.15.76:0/3009904320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21e40ceda0 msgr2=0x7f21e40cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:42.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.262+0000 7f21ea2fb640 1 --2- 172.21.15.76:0/3009904320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21e40ceda0 0x7f21e40cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:42.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.262+0000 7f21ea2fb640 1 -- 172.21.15.76:0/3009904320 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21dc0094d0 con 0x7f21e415d630 2023-11-09T03:53:42.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.262+0000 7f21ea2fb640 1 --2- 172.21.15.76:0/3009904320 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21e415d630 0x7f21e40cc1b0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009870 tx=0x7f21dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:42.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.262+0000 7f21da7fc640 1 -- 172.21.15.76:0/3009904320 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc03ac60 con 0x7f21e415d630 2023-11-09T03:53:42.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.262+0000 7f21da7fc640 1 -- 172.21.15.76:0/3009904320 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21dc03ade0 con 0x7f21e415d630 2023-11-09T03:53:42.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.262+0000 7f21ebd5e640 1 -- 172.21.15.76:0/3009904320 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e41e5290 con 0x7f21e415d630 2023-11-09T03:53:42.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.262+0000 7f21da7fc640 1 -- 172.21.15.76:0/3009904320 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc044630 con 0x7f21e415d630 2023-11-09T03:53:42.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.262+0000 7f21ebd5e640 1 -- 172.21.15.76:0/3009904320 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e41e5850 con 0x7f21e415d630 2023-11-09T03:53:42.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.263+0000 7f21ebd5e640 1 -- 172.21.15.76:0/3009904320 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e41e5a40 con 0x7f21e415d630 2023-11-09T03:53:42.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.264+0000 7f21da7fc640 1 -- 172.21.15.76:0/3009904320 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f21dc02f050 con 0x7f21e415d630 2023-11-09T03:53:42.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.264+0000 7f21da7fc640 1 --2- 172.21.15.76:0/3009904320 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f21c806ae70 0x7f21c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:42.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.264+0000 7f21da7fc640 1 -- 172.21.15.76:0/3009904320 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8167+0+0 (secure 0 0 0) 0x7f21dc0abbe0 con 0x7f21e415d630 2023-11-09T03:53:42.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.264+0000 7f21e9afa640 1 --2- 172.21.15.76:0/3009904320 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f21c806ae70 0x7f21c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:42.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.265+0000 7f21e9afa640 1 --2- 172.21.15.76:0/3009904320 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f21c806ae70 0x7f21c806d330 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f21e40ce260 tx=0x7f21e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:42.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.268+0000 7f21da7fc640 1 -- 172.21.15.76:0/3009904320 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f21dc076430 con 0x7f21e415d630 2023-11-09T03:53:42.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.469+0000 7f21ebd5e640 1 -- 172.21.15.76:0/3009904320 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21e40684b0 con 0x7f21e415d630 2023-11-09T03:53:42.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.470+0000 7f21da7fc640 1 -- 172.21.15.76:0/3009904320 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+4937 (secure 0 0 0) 0x7f21dc079a50 con 0x7f21e415d630 2023-11-09T03:53:42.471 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:53:42.471 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":117,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:53:34.495220+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":8772},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8772":{"gid":8772,"name":"a","rank":1,"incarnation":113,"state":"up:active","state_seq":7,"addr":"172.21.15.76:6835/2075292158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2075292158},{"type":"v1","addr":"172.21.15.76:6835","nonce":2075292158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:53:42.471 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 117 2023-11-09T03:53:42.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.472+0000 7f21ebd5e640 1 -- 172.21.15.76:0/3009904320 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f21c806ae70 msgr2=0x7f21c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:42.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.472+0000 7f21ebd5e640 1 --2- 172.21.15.76:0/3009904320 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f21c806ae70 0x7f21c806d330 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f21e40ce260 tx=0x7f21e0009040 comp rx=0 tx=0).stop 2023-11-09T03:53:42.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.472+0000 7f21ebd5e640 1 -- 172.21.15.76:0/3009904320 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21e415d630 msgr2=0x7f21e40cc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:42.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.472+0000 7f21ebd5e640 1 --2- 172.21.15.76:0/3009904320 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21e415d630 0x7f21e40cc1b0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009870 tx=0x7f21dc0046c0 comp rx=0 tx=0).stop 2023-11-09T03:53:42.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.472+0000 7f21ebd5e640 1 -- 172.21.15.76:0/3009904320 shutdown_connections 2023-11-09T03:53:42.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.472+0000 7f21ebd5e640 1 --2- 172.21.15.76:0/3009904320 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f21c806ae70 0x7f21c806d330 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:42.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.472+0000 7f21ebd5e640 1 --2- 172.21.15.76:0/3009904320 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21e415d630 0x7f21e40cc1b0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:42.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.472+0000 7f21ebd5e640 1 --2- 172.21.15.76:0/3009904320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21e40ceda0 0x7f21e40cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:42.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.472+0000 7f21ebd5e640 1 --2- 172.21.15.76:0/3009904320 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21e40b2210 0x7f21e41e5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:42.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.472+0000 7f21ebd5e640 1 -- 172.21.15.76:0/3009904320 >> 172.21.15.76:0/3009904320 conn(0x7f21e40d0c30 msgr2=0x7f21e40cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:42.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.472+0000 7f21ebd5e640 1 -- 172.21.15.76:0/3009904320 shutdown_connections 2023-11-09T03:53:42.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.472+0000 7f21ebd5e640 1 -- 172.21.15.76:0/3009904320 wait complete. 2023-11-09T03:53:42.483 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:1 true 2023-11-09T03:53:42.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.585+0000 7f34bf178640 1 Processor -- start 2023-11-09T03:53:42.586 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.585+0000 7f34bf178640 1 -- start start 2023-11-09T03:53:42.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.586+0000 7f34bf178640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f34b815d100 0x7f34b80ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:42.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.586+0000 7f34bf178640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34b815d4e0 0x7f34b80d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:42.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.586+0000 7f34bf178640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34b80b32b0 con 0x7f34b80d3b60 2023-11-09T03:53:42.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.586+0000 7f34bf178640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34b80cee10 con 0x7f34b815d100 2023-11-09T03:53:42.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.586+0000 7f34bf178640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b80b4750 con 0x7f34b815d4e0 2023-11-09T03:53:42.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.586+0000 7f34bd715640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f34b80d3b60 0x7f34b80cf520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48036/0 (socket says 172.21.15.76:48036) 2023-11-09T03:53:42.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.586+0000 7f34bd715640 1 -- 172.21.15.76:0/2406282290 learned_addr learned my addr 172.21.15.76:0/2406282290 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:53:42.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.586+0000 7f34bcf14640 1 --2- 172.21.15.76:0/2406282290 >> v2:172.21.15.113:3300/0 conn(0x7f34b815d100 0x7f34b80ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:42.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.586+0000 7f34b7fff640 1 --2- 172.21.15.76:0/2406282290 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34b815d4e0 0x7f34b80d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:42.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.587+0000 7f34b77fe640 1 -- 172.21.15.76:0/2406282290 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1446922176 0 0) 0x7f34b80b32b0 con 0x7f34b80d3b60 2023-11-09T03:53:42.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.587+0000 7f34b77fe640 1 -- 172.21.15.76:0/2406282290 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34a0003520 con 0x7f34b80d3b60 2023-11-09T03:53:42.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.587+0000 7f34b77fe640 1 -- 172.21.15.76:0/2406282290 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4239822772 0 0) 0x7f34a0003520 con 0x7f34b80d3b60 2023-11-09T03:53:42.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.587+0000 7f34bcf14640 1 -- 172.21.15.76:0/2406282290 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34b815d4e0 msgr2=0x7f34b80d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:42.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.587+0000 7f34bcf14640 1 --2- 172.21.15.76:0/2406282290 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34b815d4e0 0x7f34b80d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:42.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.587+0000 7f34bcf14640 1 -- 172.21.15.76:0/2406282290 >> v1:172.21.15.76:6789/0 conn(0x7f34b80d3b60 legacy=0x7f34b80cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:42.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.587+0000 7f34bcf14640 1 -- 172.21.15.76:0/2406282290 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b80b2980 con 0x7f34b815d100 2023-11-09T03:53:42.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.587+0000 7f34b7fff640 1 --2- 172.21.15.76:0/2406282290 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34b815d4e0 0x7f34b80d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:53:42.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.588+0000 7f34bcf14640 1 --2- 172.21.15.76:0/2406282290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34b815d100 0x7f34b80ce510 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f34a8009790 tx=0x7f34a802b770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5454f42cd982f91c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:42.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.588+0000 7f34b77fe640 1 -- 172.21.15.76:0/2406282290 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b00019f0 con 0x7f34b815d100 2023-11-09T03:53:42.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.588+0000 7f34b77fe640 1 -- 172.21.15.76:0/2406282290 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a802e050 con 0x7f34b815d100 2023-11-09T03:53:42.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.588+0000 7f34b77fe640 1 -- 172.21.15.76:0/2406282290 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a8030710 con 0x7f34b815d100 2023-11-09T03:53:42.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.588+0000 7f34bf178640 1 -- 172.21.15.76:0/2406282290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34b815d100 msgr2=0x7f34b80ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:42.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.588+0000 7f34bf178640 1 --2- 172.21.15.76:0/2406282290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34b815d100 0x7f34b80ce510 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f34a8009790 tx=0x7f34a802b770 comp rx=0 tx=0).stop 2023-11-09T03:53:42.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.588+0000 7f34bf178640 1 -- 172.21.15.76:0/2406282290 shutdown_connections 2023-11-09T03:53:42.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.588+0000 7f34bf178640 1 --2- 172.21.15.76:0/2406282290 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34b815d4e0 0x7f34b80d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:42.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.588+0000 7f34bf178640 1 --2- 172.21.15.76:0/2406282290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34b815d100 0x7f34b80ce510 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:42.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.588+0000 7f34bf178640 1 -- 172.21.15.76:0/2406282290 >> 172.21.15.76:0/2406282290 conn(0x7f34b80d2ef0 msgr2=0x7f34b80d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:42.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.588+0000 7f34bf178640 1 -- 172.21.15.76:0/2406282290 shutdown_connections 2023-11-09T03:53:42.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.588+0000 7f34bf178640 1 -- 172.21.15.76:0/2406282290 wait complete. 2023-11-09T03:53:42.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.589+0000 7f34bf178640 1 Processor -- start 2023-11-09T03:53:42.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.589+0000 7f34bf178640 1 -- start start 2023-11-09T03:53:42.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.590+0000 7f34bf178640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34b80d3b60 0x7f34b8155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:42.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.590+0000 7f34bf178640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34b815d100 0x7f34b80c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:42.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.590+0000 7f34bf178640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34b815d4e0 0x7f34b80cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:42.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.590+0000 7f34bf178640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b80ca9a0 con 0x7f34b80d3b60 2023-11-09T03:53:42.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.590+0000 7f34bf178640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b80cac80 con 0x7f34b815d100 2023-11-09T03:53:42.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.590+0000 7f34bf178640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b80caf60 con 0x7f34b815d4e0 2023-11-09T03:53:42.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.590+0000 7f34bcf14640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34b80d3b60 0x7f34b8155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:42.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.590+0000 7f34bcf14640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34b80d3b60 0x7f34b8155520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47878/0 (socket says 172.21.15.76:47878) 2023-11-09T03:53:42.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.590+0000 7f34bcf14640 1 -- 172.21.15.76:0/3685068718 learned_addr learned my addr 172.21.15.76:0/3685068718 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:42.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.591+0000 7f34bd715640 1 --2- 172.21.15.76:0/3685068718 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34b815d4e0 0x7f34b80cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:42.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.591+0000 7f34b7fff640 1 --2- 172.21.15.76:0/3685068718 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34b815d100 0x7f34b80c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:42.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.591+0000 7f34bcf14640 1 -- 172.21.15.76:0/3685068718 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34b815d4e0 msgr2=0x7f34b80cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:42.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.591+0000 7f34bcf14640 1 --2- 172.21.15.76:0/3685068718 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34b815d4e0 0x7f34b80cefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:42.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.591+0000 7f34bcf14640 1 -- 172.21.15.76:0/3685068718 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34b815d100 msgr2=0x7f34b80c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:42.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.591+0000 7f34bcf14640 1 --2- 172.21.15.76:0/3685068718 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34b815d100 0x7f34b80c9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:42.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.591+0000 7f34bcf14640 1 -- 172.21.15.76:0/3685068718 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ac009400 con 0x7f34b80d3b60 2023-11-09T03:53:42.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.591+0000 7f34bd715640 1 --2- 172.21.15.76:0/3685068718 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34b815d4e0 0x7f34b80cefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:53:42.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.591+0000 7f34b7fff640 1 --2- 172.21.15.76:0/3685068718 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34b815d100 0x7f34b80c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:53:42.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.591+0000 7f34bcf14640 1 --2- 172.21.15.76:0/3685068718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34b80d3b60 0x7f34b8155520 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f34a8000c00 tx=0x7f34a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:42.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.591+0000 7f34b57fa640 1 -- 172.21.15.76:0/3685068718 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a803c820 con 0x7f34b80d3b60 2023-11-09T03:53:42.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.591+0000 7f34b57fa640 1 -- 172.21.15.76:0/3685068718 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a803c980 con 0x7f34b80d3b60 2023-11-09T03:53:42.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.591+0000 7f34bf178640 1 -- 172.21.15.76:0/3685068718 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a80093a0 con 0x7f34b80d3b60 2023-11-09T03:53:42.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.591+0000 7f34b57fa640 1 -- 172.21.15.76:0/3685068718 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a803cd10 con 0x7f34b80d3b60 2023-11-09T03:53:42.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.591+0000 7f34bf178640 1 -- 172.21.15.76:0/3685068718 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34b80b6c80 con 0x7f34b80d3b60 2023-11-09T03:53:42.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.592+0000 7f34bf178640 1 -- 172.21.15.76:0/3685068718 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34b80684b0 con 0x7f34b80d3b60 2023-11-09T03:53:42.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.592+0000 7f34b57fa640 1 -- 172.21.15.76:0/3685068718 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f34a8039050 con 0x7f34b80d3b60 2023-11-09T03:53:42.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.593+0000 7f34b57fa640 1 --2- 172.21.15.76:0/3685068718 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f34a406ae70 0x7f34a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:42.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.593+0000 7f34b57fa640 1 -- 172.21.15.76:0/3685068718 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8167+0+0 (secure 0 0 0) 0x7f34a80ab600 con 0x7f34b80d3b60 2023-11-09T03:53:42.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.593+0000 7f34b7fff640 1 --2- 172.21.15.76:0/3685068718 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f34a406ae70 0x7f34a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:42.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.594+0000 7f34b7fff640 1 --2- 172.21.15.76:0/3685068718 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f34a406ae70 0x7f34a406d330 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f34b81e6070 tx=0x7f34ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:42.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.597+0000 7f34b57fa640 1 -- 172.21.15.76:0/3685068718 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f34a8075ec0 con 0x7f34b80d3b60 2023-11-09T03:53:42.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:42.830+0000 7f34bf178640 1 -- 172.21.15.76:0/3685068718 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"} v 0) v1 -- 0x7f34b80d2cd0 con 0x7f34b80d3b60 2023-11-09T03:53:43.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.513+0000 7f34b57fa640 1 -- 172.21.15.76:0/3685068718 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"}]=0 freezing mds.8772 v118) v1 ==== 112+0+0 (secure 0 0 0) 0x7f34a804c0a0 con 0x7f34b80d3b60 2023-11-09T03:53:43.514 INFO:teuthology.orchestra.run.smithi076.stderr:freezing mds.8772 2023-11-09T03:53:43.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.515+0000 7f34bf178640 1 -- 172.21.15.76:0/3685068718 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f34a406ae70 msgr2=0x7f34a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:43.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.515+0000 7f34bf178640 1 --2- 172.21.15.76:0/3685068718 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f34a406ae70 0x7f34a406d330 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f34b81e6070 tx=0x7f34ac008040 comp rx=0 tx=0).stop 2023-11-09T03:53:43.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.515+0000 7f34bf178640 1 -- 172.21.15.76:0/3685068718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34b80d3b60 msgr2=0x7f34b8155520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:43.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.515+0000 7f34bf178640 1 --2- 172.21.15.76:0/3685068718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34b80d3b60 0x7f34b8155520 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f34a8000c00 tx=0x7f34a80046c0 comp rx=0 tx=0).stop 2023-11-09T03:53:43.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.515+0000 7f34bf178640 1 -- 172.21.15.76:0/3685068718 shutdown_connections 2023-11-09T03:53:43.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.515+0000 7f34bf178640 1 --2- 172.21.15.76:0/3685068718 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f34a406ae70 0x7f34a406d330 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:43.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.515+0000 7f34bf178640 1 --2- 172.21.15.76:0/3685068718 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34b815d4e0 0x7f34b80cefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:43.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.515+0000 7f34bf178640 1 --2- 172.21.15.76:0/3685068718 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34b815d100 0x7f34b80c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:43.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.515+0000 7f34bf178640 1 --2- 172.21.15.76:0/3685068718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34b80d3b60 0x7f34b8155520 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:43.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.515+0000 7f34bf178640 1 -- 172.21.15.76:0/3685068718 >> 172.21.15.76:0/3685068718 conn(0x7f34b80d2ef0 msgr2=0x7f34b806beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:43.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.515+0000 7f34bf178640 1 -- 172.21.15.76:0/3685068718 shutdown_connections 2023-11-09T03:53:43.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.515+0000 7f34bf178640 1 -- 172.21.15.76:0/3685068718 wait complete. 2023-11-09T03:53:43.528 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_kill_mdstable_at 3 2023-11-09T03:53:43.641 INFO:teuthology.orchestra.run.smithi076.stdout:{"success":"mds_kill_mdstable_at = '3' (not observed, change may require restart) "} 2023-11-09T03:53:43.641 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '3' (not observed, change may require restart) " } 2023-11-09T03:53:43.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:53:43.642 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s33) 2023-11-09T03:53:43.645 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2023-11-09T03:53:43.660 INFO:tasks.ceph.mds.a.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::handle_request(ceph::cref_t&)' thread 7f14c71ff640 time 2023-11-09T03:53:43.661872+0000 2023-11-09T03:53:43.661 INFO:tasks.ceph.mds.a.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: 76: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 3) 2023-11-09T03:53:43.662 INFO:tasks.ceph.mds.a.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:53:43.662 INFO:tasks.ceph.mds.a.smithi076.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f14ccd6efaf] 2023-11-09T03:53:43.662 INFO:tasks.ceph.mds.a.smithi076.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f14ccd6f16d] 2023-11-09T03:53:43.663 INFO:tasks.ceph.mds.a.smithi076.stderr: 3: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x1187) [0x56056ba2d1b7] 2023-11-09T03:53:43.663 INFO:tasks.ceph.mds.a.smithi076.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xd55) [0x56056b7bad15] 2023-11-09T03:53:43.663 INFO:tasks.ceph.mds.a.smithi076.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x56056b7bcb2b] 2023-11-09T03:53:43.663 INFO:tasks.ceph.mds.a.smithi076.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56056b7bd01c] 2023-11-09T03:53:43.663 INFO:tasks.ceph.mds.a.smithi076.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x56056b7a6325] 2023-11-09T03:53:43.664 INFO:tasks.ceph.mds.a.smithi076.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7f14ccf5ff9a] 2023-11-09T03:53:43.664 INFO:tasks.ceph.mds.a.smithi076.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f14ccff1741] 2023-11-09T03:53:43.664 INFO:tasks.ceph.mds.a.smithi076.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7f14cc49f802] 2023-11-09T03:53:43.664 INFO:tasks.ceph.mds.a.smithi076.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7f14cc43f450] 2023-11-09T03:53:43.664 INFO:tasks.ceph.mds.a.smithi076.stderr:2023-11-09T03:53:43.660+0000 7f14c71ff640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::handle_request(ceph::cref_t&)' thread 7f14c71ff640 time 2023-11-09T03:53:43.661872+0000 2023-11-09T03:53:43.665 INFO:tasks.ceph.mds.a.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: 76: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 3) 2023-11-09T03:53:43.665 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:53:43.665 INFO:tasks.ceph.mds.a.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:53:43.665 INFO:tasks.ceph.mds.a.smithi076.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f14ccd6efaf] 2023-11-09T03:53:43.665 INFO:tasks.ceph.mds.a.smithi076.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f14ccd6f16d] 2023-11-09T03:53:43.666 INFO:tasks.ceph.mds.a.smithi076.stderr: 3: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x1187) [0x56056ba2d1b7] 2023-11-09T03:53:43.666 INFO:tasks.ceph.mds.a.smithi076.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xd55) [0x56056b7bad15] 2023-11-09T03:53:43.666 INFO:tasks.ceph.mds.a.smithi076.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x56056b7bcb2b] 2023-11-09T03:53:43.666 INFO:tasks.ceph.mds.a.smithi076.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56056b7bd01c] 2023-11-09T03:53:43.666 INFO:tasks.ceph.mds.a.smithi076.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x56056b7a6325] 2023-11-09T03:53:43.667 INFO:tasks.ceph.mds.a.smithi076.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7f14ccf5ff9a] 2023-11-09T03:53:43.667 INFO:tasks.ceph.mds.a.smithi076.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f14ccff1741] 2023-11-09T03:53:43.667 INFO:tasks.ceph.mds.a.smithi076.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7f14cc49f802] 2023-11-09T03:53:43.667 INFO:tasks.ceph.mds.a.smithi076.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7f14cc43f450] 2023-11-09T03:53:43.667 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:53:43.668 INFO:tasks.ceph.mds.a.smithi076.stderr:*** Caught signal (Aborted) ** 2023-11-09T03:53:43.668 INFO:tasks.ceph.mds.a.smithi076.stderr: in thread 7f14c71ff640 thread_name:ms_dispatch 2023-11-09T03:53:43.668 INFO:tasks.ceph.mds.a.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:53:43.668 INFO:tasks.ceph.mds.a.smithi076.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f14cc454db0] 2023-11-09T03:53:43.668 INFO:tasks.ceph.mds.a.smithi076.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f14cc4a154c] 2023-11-09T03:53:43.669 INFO:tasks.ceph.mds.a.smithi076.stderr: 3: raise() 2023-11-09T03:53:43.669 INFO:tasks.ceph.mds.a.smithi076.stderr: 4: abort() 2023-11-09T03:53:43.669 INFO:tasks.ceph.mds.a.smithi076.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f14ccd6f009] 2023-11-09T03:53:43.669 INFO:tasks.ceph.mds.a.smithi076.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f14ccd6f16d] 2023-11-09T03:53:43.669 INFO:tasks.ceph.mds.a.smithi076.stderr: 7: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x1187) [0x56056ba2d1b7] 2023-11-09T03:53:43.670 INFO:tasks.ceph.mds.a.smithi076.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xd55) [0x56056b7bad15] 2023-11-09T03:53:43.670 INFO:tasks.ceph.mds.a.smithi076.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x56056b7bcb2b] 2023-11-09T03:53:43.670 INFO:tasks.ceph.mds.a.smithi076.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56056b7bd01c] 2023-11-09T03:53:43.670 INFO:tasks.ceph.mds.a.smithi076.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x56056b7a6325] 2023-11-09T03:53:43.670 INFO:tasks.ceph.mds.a.smithi076.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7f14ccf5ff9a] 2023-11-09T03:53:43.671 INFO:tasks.ceph.mds.a.smithi076.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f14ccff1741] 2023-11-09T03:53:43.671 INFO:tasks.ceph.mds.a.smithi076.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7f14cc49f802] 2023-11-09T03:53:43.671 INFO:tasks.ceph.mds.a.smithi076.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7f14cc43f450] 2023-11-09T03:53:43.671 INFO:tasks.ceph.mds.a.smithi076.stderr:2023-11-09T03:53:43.661+0000 7f14c71ff640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:53:43.671 INFO:tasks.ceph.mds.a.smithi076.stderr: in thread 7f14c71ff640 thread_name:ms_dispatch 2023-11-09T03:53:43.671 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:53:43.672 INFO:tasks.ceph.mds.a.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:53:43.672 INFO:tasks.ceph.mds.a.smithi076.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f14cc454db0] 2023-11-09T03:53:43.672 INFO:tasks.ceph.mds.a.smithi076.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f14cc4a154c] 2023-11-09T03:53:43.672 INFO:tasks.ceph.mds.a.smithi076.stderr: 3: raise() 2023-11-09T03:53:43.673 INFO:tasks.ceph.mds.a.smithi076.stderr: 4: abort() 2023-11-09T03:53:43.673 INFO:tasks.ceph.mds.a.smithi076.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f14ccd6f009] 2023-11-09T03:53:43.673 INFO:tasks.ceph.mds.a.smithi076.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f14ccd6f16d] 2023-11-09T03:53:43.673 INFO:tasks.ceph.mds.a.smithi076.stderr: 7: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x1187) [0x56056ba2d1b7] 2023-11-09T03:53:43.673 INFO:tasks.ceph.mds.a.smithi076.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xd55) [0x56056b7bad15] 2023-11-09T03:53:43.674 INFO:tasks.ceph.mds.a.smithi076.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x56056b7bcb2b] 2023-11-09T03:53:43.674 INFO:tasks.ceph.mds.a.smithi076.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56056b7bd01c] 2023-11-09T03:53:43.674 INFO:tasks.ceph.mds.a.smithi076.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x56056b7a6325] 2023-11-09T03:53:43.674 INFO:tasks.ceph.mds.a.smithi076.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7f14ccf5ff9a] 2023-11-09T03:53:43.674 INFO:tasks.ceph.mds.a.smithi076.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f14ccff1741] 2023-11-09T03:53:43.675 INFO:tasks.ceph.mds.a.smithi076.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7f14cc49f802] 2023-11-09T03:53:43.675 INFO:tasks.ceph.mds.a.smithi076.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7f14cc43f450] 2023-11-09T03:53:43.675 INFO:tasks.ceph.mds.a.smithi076.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:53:43.675 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:53:43.676 INFO:tasks.ceph.mds.a.smithi076.stderr: -1> 2023-11-09T03:53:43.660+0000 7f14c71ff640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::handle_request(ceph::cref_t&)' thread 7f14c71ff640 time 2023-11-09T03:53:43.661872+0000 2023-11-09T03:53:43.676 INFO:tasks.ceph.mds.a.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: 76: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 3) 2023-11-09T03:53:43.676 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:53:43.676 INFO:tasks.ceph.mds.a.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:53:43.677 INFO:tasks.ceph.mds.a.smithi076.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f14ccd6efaf] 2023-11-09T03:53:43.677 INFO:tasks.ceph.mds.a.smithi076.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f14ccd6f16d] 2023-11-09T03:53:43.677 INFO:tasks.ceph.mds.a.smithi076.stderr: 3: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x1187) [0x56056ba2d1b7] 2023-11-09T03:53:43.677 INFO:tasks.ceph.mds.a.smithi076.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xd55) [0x56056b7bad15] 2023-11-09T03:53:43.677 INFO:tasks.ceph.mds.a.smithi076.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x56056b7bcb2b] 2023-11-09T03:53:43.678 INFO:tasks.ceph.mds.a.smithi076.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56056b7bd01c] 2023-11-09T03:53:43.678 INFO:tasks.ceph.mds.a.smithi076.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x56056b7a6325] 2023-11-09T03:53:43.678 INFO:tasks.ceph.mds.a.smithi076.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7f14ccf5ff9a] 2023-11-09T03:53:43.678 INFO:tasks.ceph.mds.a.smithi076.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f14ccff1741] 2023-11-09T03:53:43.678 INFO:tasks.ceph.mds.a.smithi076.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7f14cc49f802] 2023-11-09T03:53:43.679 INFO:tasks.ceph.mds.a.smithi076.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7f14cc43f450] 2023-11-09T03:53:43.679 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:53:43.679 INFO:tasks.ceph.mds.a.smithi076.stderr: 0> 2023-11-09T03:53:43.661+0000 7f14c71ff640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:53:43.679 INFO:tasks.ceph.mds.a.smithi076.stderr: in thread 7f14c71ff640 thread_name:ms_dispatch 2023-11-09T03:53:43.679 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:53:43.679 INFO:tasks.ceph.mds.a.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:53:43.680 INFO:tasks.ceph.mds.a.smithi076.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f14cc454db0] 2023-11-09T03:53:43.680 INFO:tasks.ceph.mds.a.smithi076.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f14cc4a154c] 2023-11-09T03:53:43.680 INFO:tasks.ceph.mds.a.smithi076.stderr: 3: raise() 2023-11-09T03:53:43.680 INFO:tasks.ceph.mds.a.smithi076.stderr: 4: abort() 2023-11-09T03:53:43.680 INFO:tasks.ceph.mds.a.smithi076.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f14ccd6f009] 2023-11-09T03:53:43.680 INFO:tasks.ceph.mds.a.smithi076.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f14ccd6f16d] 2023-11-09T03:53:43.681 INFO:tasks.ceph.mds.a.smithi076.stderr: 7: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x1187) [0x56056ba2d1b7] 2023-11-09T03:53:43.681 INFO:tasks.ceph.mds.a.smithi076.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xd55) [0x56056b7bad15] 2023-11-09T03:53:43.681 INFO:tasks.ceph.mds.a.smithi076.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x56056b7bcb2b] 2023-11-09T03:53:43.681 INFO:tasks.ceph.mds.a.smithi076.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56056b7bd01c] 2023-11-09T03:53:43.682 INFO:tasks.ceph.mds.a.smithi076.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x56056b7a6325] 2023-11-09T03:53:43.682 INFO:tasks.ceph.mds.a.smithi076.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7f14ccf5ff9a] 2023-11-09T03:53:43.682 INFO:tasks.ceph.mds.a.smithi076.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f14ccff1741] 2023-11-09T03:53:43.682 INFO:tasks.ceph.mds.a.smithi076.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7f14cc49f802] 2023-11-09T03:53:43.682 INFO:tasks.ceph.mds.a.smithi076.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7f14cc43f450] 2023-11-09T03:53:43.683 INFO:tasks.ceph.mds.a.smithi076.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:53:43.683 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:53:43.683 INFO:tasks.ceph.mds.a.smithi076.stderr: -1> 2023-11-09T03:53:43.660+0000 7f14c71ff640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::handle_request(ceph::cref_t&)' thread 7f14c71ff640 time 2023-11-09T03:53:43.661872+0000 2023-11-09T03:53:43.683 INFO:tasks.ceph.mds.a.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: 76: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 3) 2023-11-09T03:53:43.683 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:53:43.684 INFO:tasks.ceph.mds.a.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:53:43.684 INFO:tasks.ceph.mds.a.smithi076.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f14ccd6efaf] 2023-11-09T03:53:43.684 INFO:tasks.ceph.mds.a.smithi076.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f14ccd6f16d] 2023-11-09T03:53:43.684 INFO:tasks.ceph.mds.a.smithi076.stderr: 3: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x1187) [0x56056ba2d1b7] 2023-11-09T03:53:43.684 INFO:tasks.ceph.mds.a.smithi076.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xd55) [0x56056b7bad15] 2023-11-09T03:53:43.685 INFO:tasks.ceph.mds.a.smithi076.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x56056b7bcb2b] 2023-11-09T03:53:43.685 INFO:tasks.ceph.mds.a.smithi076.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56056b7bd01c] 2023-11-09T03:53:43.685 INFO:tasks.ceph.mds.a.smithi076.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x56056b7a6325] 2023-11-09T03:53:43.685 INFO:tasks.ceph.mds.a.smithi076.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7f14ccf5ff9a] 2023-11-09T03:53:43.685 INFO:tasks.ceph.mds.a.smithi076.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f14ccff1741] 2023-11-09T03:53:43.686 INFO:tasks.ceph.mds.a.smithi076.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7f14cc49f802] 2023-11-09T03:53:43.686 INFO:tasks.ceph.mds.a.smithi076.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7f14cc43f450] 2023-11-09T03:53:43.686 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:53:43.686 INFO:tasks.ceph.mds.a.smithi076.stderr: 0> 2023-11-09T03:53:43.661+0000 7f14c71ff640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:53:43.686 INFO:tasks.ceph.mds.a.smithi076.stderr: in thread 7f14c71ff640 thread_name:ms_dispatch 2023-11-09T03:53:43.687 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:53:43.687 INFO:tasks.ceph.mds.a.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:53:43.687 INFO:tasks.ceph.mds.a.smithi076.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f14cc454db0] 2023-11-09T03:53:43.687 INFO:tasks.ceph.mds.a.smithi076.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f14cc4a154c] 2023-11-09T03:53:43.687 INFO:tasks.ceph.mds.a.smithi076.stderr: 3: raise() 2023-11-09T03:53:43.687 INFO:tasks.ceph.mds.a.smithi076.stderr: 4: abort() 2023-11-09T03:53:43.688 INFO:tasks.ceph.mds.a.smithi076.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f14ccd6f009] 2023-11-09T03:53:43.688 INFO:tasks.ceph.mds.a.smithi076.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f14ccd6f16d] 2023-11-09T03:53:43.688 INFO:tasks.ceph.mds.a.smithi076.stderr: 7: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x1187) [0x56056ba2d1b7] 2023-11-09T03:53:43.688 INFO:tasks.ceph.mds.a.smithi076.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xd55) [0x56056b7bad15] 2023-11-09T03:53:43.688 INFO:tasks.ceph.mds.a.smithi076.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x56056b7bcb2b] 2023-11-09T03:53:43.690 INFO:tasks.ceph.mds.a.smithi076.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56056b7bd01c] 2023-11-09T03:53:43.691 INFO:tasks.ceph.mds.a.smithi076.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x56056b7a6325] 2023-11-09T03:53:43.691 INFO:tasks.ceph.mds.a.smithi076.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7f14ccf5ff9a] 2023-11-09T03:53:43.691 INFO:tasks.ceph.mds.a.smithi076.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f14ccff1741] 2023-11-09T03:53:43.691 INFO:tasks.ceph.mds.a.smithi076.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7f14cc49f802] 2023-11-09T03:53:43.692 INFO:tasks.ceph.mds.a.smithi076.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7f14cc43f450] 2023-11-09T03:53:43.692 INFO:tasks.ceph.mds.a.smithi076.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:53:43.692 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:53:43.768 INFO:teuthology.orchestra.run.smithi076.stdout:{"mds_beacon_grace":"15.000000"} 2023-11-09T03:53:43.768 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2023-11-09T03:53:43.769 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:53:43.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.908+0000 7fd573232640 1 Processor -- start 2023-11-09T03:53:43.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.908+0000 7fd573232640 1 -- start start 2023-11-09T03:53:43.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.909+0000 7fd573232640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd56c0ade00 0x7fd56c0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:43.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.909+0000 7fd573232640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd56c06d9c0 0x7fd56c0710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:43.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.909+0000 7fd573232640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd56c06cc90 con 0x7fd56c06c5f0 2023-11-09T03:53:43.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.909+0000 7fd573232640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd56c0b0e60 con 0x7fd56c06d9c0 2023-11-09T03:53:43.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.909+0000 7fd573232640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd56c06cee0 con 0x7fd56c0ade00 2023-11-09T03:53:43.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.909+0000 7fd56bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd56c06d9c0 0x7fd56c0710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:43.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.909+0000 7fd56bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd56c06d9c0 0x7fd56c0710d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47890/0 (socket says 172.21.15.76:47890) 2023-11-09T03:53:43.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.909+0000 7fd56bfff640 1 -- 172.21.15.76:0/105259731 learned_addr learned my addr 172.21.15.76:0/105259731 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:43.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.909+0000 7fd56bfff640 1 -- 172.21.15.76:0/105259731 >> v2:172.21.15.113:3300/0 conn(0x7fd56c0ade00 msgr2=0x7fd56c0b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:43.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.909+0000 7fd56bfff640 1 --2- 172.21.15.76:0/105259731 >> v2:172.21.15.113:3300/0 conn(0x7fd56c0ade00 0x7fd56c0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:43.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.909+0000 7fd56bfff640 1 -- 172.21.15.76:0/105259731 >> v1:172.21.15.113:6789/0 conn(0x7fd56c06c5f0 legacy=0x7fd56c15e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:43.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.909+0000 7fd56bfff640 1 -- 172.21.15.76:0/105259731 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd56c06d1c0 con 0x7fd56c06d9c0 2023-11-09T03:53:43.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.910+0000 7fd56bfff640 1 --2- 172.21.15.76:0/105259731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd56c06d9c0 0x7fd56c0710d0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0cb430 tx=0x7fd55802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7058b1d2dca41b7a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:43.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.910+0000 7fd56b7fe640 1 -- 172.21.15.76:0/105259731 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd558031750 con 0x7fd56c06d9c0 2023-11-09T03:53:43.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.910+0000 7fd56b7fe640 1 -- 172.21.15.76:0/105259731 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5580318b0 con 0x7fd56c06d9c0 2023-11-09T03:53:43.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.910+0000 7fd56b7fe640 1 -- 172.21.15.76:0/105259731 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd558031c40 con 0x7fd56c06d9c0 2023-11-09T03:53:43.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.911+0000 7fd573232640 1 -- 172.21.15.76:0/105259731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd56c06d9c0 msgr2=0x7fd56c0710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:43.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.911+0000 7fd573232640 1 --2- 172.21.15.76:0/105259731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd56c06d9c0 0x7fd56c0710d0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0cb430 tx=0x7fd55802b7f0 comp rx=0 tx=0).stop 2023-11-09T03:53:43.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.911+0000 7fd573232640 1 -- 172.21.15.76:0/105259731 shutdown_connections 2023-11-09T03:53:43.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.911+0000 7fd573232640 1 --2- 172.21.15.76:0/105259731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd56c06d9c0 0x7fd56c0710d0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:43.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.911+0000 7fd573232640 1 --2- 172.21.15.76:0/105259731 >> v2:172.21.15.113:3300/0 conn(0x7fd56c0ade00 0x7fd56c0b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:43.917 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.911+0000 7fd573232640 1 -- 172.21.15.76:0/105259731 >> 172.21.15.76:0/105259731 conn(0x7fd56c0cb530 msgr2=0x7fd56c0cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:43.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.911+0000 7fd573232640 1 -- 172.21.15.76:0/105259731 shutdown_connections 2023-11-09T03:53:43.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.911+0000 7fd573232640 1 -- 172.21.15.76:0/105259731 wait complete. 2023-11-09T03:53:43.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.912+0000 7fd573232640 1 Processor -- start 2023-11-09T03:53:43.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.912+0000 7fd573232640 1 -- start start 2023-11-09T03:53:43.918 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.913+0000 7fd573232640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd56c06c5f0 0x7fd56c1538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:43.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.913+0000 7fd573232640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd56c06d9c0 0x7fd56c143250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:43.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.913+0000 7fd573232640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd56c0ade00 0x7fd56c1f3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:43.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.913+0000 7fd573232640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c1574b0 con 0x7fd56c06c5f0 2023-11-09T03:53:43.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.913+0000 7fd573232640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c143790 con 0x7fd56c0ade00 2023-11-09T03:53:43.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.913+0000 7fd573232640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c157720 con 0x7fd56c06d9c0 2023-11-09T03:53:43.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.913+0000 7fd570fce640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd56c06c5f0 0x7fd56c1538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:43.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.913+0000 7fd570fce640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd56c06c5f0 0x7fd56c1538b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47904/0 (socket says 172.21.15.76:47904) 2023-11-09T03:53:43.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.913+0000 7fd570fce640 1 -- 172.21.15.76:0/2578724575 learned_addr learned my addr 172.21.15.76:0/2578724575 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:43.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.913+0000 7fd56bfff640 1 --2- 172.21.15.76:0/2578724575 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd56c06d9c0 0x7fd56c143250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:43.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.913+0000 7fd5717cf640 1 --2- 172.21.15.76:0/2578724575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd56c0ade00 0x7fd56c1f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:43.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.913+0000 7fd570fce640 1 -- 172.21.15.76:0/2578724575 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd56c06d9c0 msgr2=0x7fd56c143250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:43.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.913+0000 7fd570fce640 1 --2- 172.21.15.76:0/2578724575 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd56c06d9c0 0x7fd56c143250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:43.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.913+0000 7fd570fce640 1 -- 172.21.15.76:0/2578724575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd56c0ade00 msgr2=0x7fd56c1f3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:43.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.913+0000 7fd570fce640 1 --2- 172.21.15.76:0/2578724575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd56c0ade00 0x7fd56c1f3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:43.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.913+0000 7fd570fce640 1 -- 172.21.15.76:0/2578724575 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5580094d0 con 0x7fd56c06c5f0 2023-11-09T03:53:43.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.913+0000 7fd570fce640 1 --2- 172.21.15.76:0/2578724575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd56c06c5f0 0x7fd56c1538b0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fd56000b410 tx=0x7fd56000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:43.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.913+0000 7fd5697fa640 1 -- 172.21.15.76:0/2578724575 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd560011830 con 0x7fd56c06c5f0 2023-11-09T03:53:43.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.913+0000 7fd5697fa640 1 -- 172.21.15.76:0/2578724575 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5600119b0 con 0x7fd56c06c5f0 2023-11-09T03:53:43.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.913+0000 7fd573232640 1 -- 172.21.15.76:0/2578724575 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56c157a00 con 0x7fd56c06c5f0 2023-11-09T03:53:43.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.913+0000 7fd5697fa640 1 -- 172.21.15.76:0/2578724575 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd560011d60 con 0x7fd56c06c5f0 2023-11-09T03:53:43.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.913+0000 7fd573232640 1 -- 172.21.15.76:0/2578724575 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56c1597a0 con 0x7fd56c06c5f0 2023-11-09T03:53:43.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.914+0000 7fd573232640 1 -- 172.21.15.76:0/2578724575 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd56c0684b0 con 0x7fd56c06c5f0 2023-11-09T03:53:43.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.915+0000 7fd5697fa640 1 -- 172.21.15.76:0/2578724575 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd56001a400 con 0x7fd56c06c5f0 2023-11-09T03:53:43.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.915+0000 7fd5697fa640 1 --2- 172.21.15.76:0/2578724575 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd55006aea0 0x7fd55006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:43.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.915+0000 7fd5697fa640 1 -- 172.21.15.76:0/2578724575 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8167+0+0 (secure 0 0 0) 0x7fd56008a050 con 0x7fd56c06c5f0 2023-11-09T03:53:43.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.915+0000 7fd56bfff640 1 --2- 172.21.15.76:0/2578724575 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd55006aea0 0x7fd55006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:43.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.916+0000 7fd56bfff640 1 --2- 172.21.15.76:0/2578724575 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd55006aea0 0x7fd55006d360 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd558035000 tx=0x7fd558008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:43.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:43.920+0000 7fd5697fa640 1 -- 172.21.15.76:0/2578724575 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd5600547f0 con 0x7fd56c06c5f0 2023-11-09T03:53:44.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:44.124+0000 7fd573232640 1 -- 172.21.15.76:0/2578724575 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd56c0cddc0 con 0x7fd56c06c5f0 2023-11-09T03:53:44.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:44.125+0000 7fd5697fa640 1 -- 172.21.15.76:0/2578724575 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+4937 (secure 0 0 0) 0x7fd560015020 con 0x7fd56c06c5f0 2023-11-09T03:53:44.125 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:53:44.126 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":118,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:53:42.831863+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":8772},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8772":{"gid":8772,"name":"a","rank":1,"incarnation":113,"state":"up:active","state_seq":7,"addr":"172.21.15.76:6835/2075292158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2075292158},{"type":"v1","addr":"172.21.15.76:6835","nonce":2075292158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:53:44.126 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 118 2023-11-09T03:53:44.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:44.127+0000 7fd573232640 1 -- 172.21.15.76:0/2578724575 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd55006aea0 msgr2=0x7fd55006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:44.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:44.127+0000 7fd573232640 1 --2- 172.21.15.76:0/2578724575 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd55006aea0 0x7fd55006d360 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd558035000 tx=0x7fd558008030 comp rx=0 tx=0).stop 2023-11-09T03:53:44.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:44.127+0000 7fd573232640 1 -- 172.21.15.76:0/2578724575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd56c06c5f0 msgr2=0x7fd56c1538b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:44.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:44.127+0000 7fd573232640 1 --2- 172.21.15.76:0/2578724575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd56c06c5f0 0x7fd56c1538b0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fd56000b410 tx=0x7fd56000b8d0 comp rx=0 tx=0).stop 2023-11-09T03:53:44.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:44.127+0000 7fd573232640 1 -- 172.21.15.76:0/2578724575 shutdown_connections 2023-11-09T03:53:44.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:44.127+0000 7fd573232640 1 --2- 172.21.15.76:0/2578724575 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd55006aea0 0x7fd55006d360 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:44.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:44.127+0000 7fd573232640 1 --2- 172.21.15.76:0/2578724575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd56c0ade00 0x7fd56c1f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:44.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:44.127+0000 7fd573232640 1 --2- 172.21.15.76:0/2578724575 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd56c06d9c0 0x7fd56c143250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:44.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:44.127+0000 7fd573232640 1 --2- 172.21.15.76:0/2578724575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd56c06c5f0 0x7fd56c1538b0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:44.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:44.127+0000 7fd573232640 1 -- 172.21.15.76:0/2578724575 >> 172.21.15.76:0/2578724575 conn(0x7fd56c0cb530 msgr2=0x7fd56c0b1500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:44.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:44.127+0000 7fd573232640 1 -- 172.21.15.76:0/2578724575 shutdown_connections 2023-11-09T03:53:44.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:44.127+0000 7fd573232640 1 -- 172.21.15.76:0/2578724575 wait complete. 2023-11-09T03:53:44.138 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:53:44.181 INFO:tasks.ceph.mds.a.smithi076.stderr:daemon-helper: command crashed with signal 6 2023-11-09T03:53:45.515 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-11-09T03:53:49.139 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:53:49.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.253+0000 7f1570bee640 1 Processor -- start 2023-11-09T03:53:49.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.253+0000 7f1570bee640 1 -- start start 2023-11-09T03:53:49.257 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.254+0000 7f1570bee640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f156c070700 0x7f156c164b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:49.257 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.254+0000 7f1570bee640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156c071500 con 0x7f156c0b0c40 2023-11-09T03:53:49.257 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.254+0000 7f1570bee640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156c15dae0 con 0x7f156c159e40 2023-11-09T03:53:49.257 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.254+0000 7f1570bee640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f156c070c60 con 0x7f156c070700 2023-11-09T03:53:49.258 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.254+0000 7f1569d9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f156c0b0c40 0x7f156c06fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48044/0 (socket says 172.21.15.76:48044) 2023-11-09T03:53:49.258 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.254+0000 7f1569d9b640 1 -- 172.21.15.76:0/3562330035 learned_addr learned my addr 172.21.15.76:0/3562330035 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:53:49.258 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.255+0000 7f156ad9d640 1 --2- 172.21.15.76:0/3562330035 >> v2:172.21.15.113:3300/0 conn(0x7f156c070700 0x7f156c164b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:49.258 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.255+0000 7f156959a640 1 -- 172.21.15.76:0/3562330035 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 214827530 0 0) 0x7f156c071500 con 0x7f156c0b0c40 2023-11-09T03:53:49.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.255+0000 7f156959a640 1 -- 172.21.15.76:0/3562330035 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1554003520 con 0x7f156c0b0c40 2023-11-09T03:53:49.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.255+0000 7f156959a640 1 -- 172.21.15.76:0/3562330035 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2679420866 0 0) 0x7f1554003520 con 0x7f156c0b0c40 2023-11-09T03:53:49.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.255+0000 7f156959a640 1 -- 172.21.15.76:0/3562330035 >> v2:172.21.15.113:3300/0 conn(0x7f156c070700 msgr2=0x7f156c164b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:49.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.255+0000 7f156959a640 1 --2- 172.21.15.76:0/3562330035 >> v2:172.21.15.113:3300/0 conn(0x7f156c070700 0x7f156c164b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:49.259 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.255+0000 7f156959a640 1 -- 172.21.15.76:0/3562330035 >> v1:172.21.15.113:6789/0 conn(0x7f156c159e40 legacy=0x7f156c0b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:49.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.255+0000 7f156959a640 1 -- 172.21.15.76:0/3562330035 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c070f40 con 0x7f156c0b0c40 2023-11-09T03:53:49.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.255+0000 7f156959a640 1 -- 172.21.15.76:0/3562330035 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f1560001670 con 0x7f156c0b0c40 2023-11-09T03:53:49.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.255+0000 7f156959a640 1 -- 172.21.15.76:0/3562330035 >> v1:172.21.15.76:6789/0 conn(0x7f156c0b0c40 legacy=0x7f156c06fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:49.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.255+0000 7f156959a640 1 --2- 172.21.15.76:0/3562330035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1554003970 0x7f1554003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:49.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.255+0000 7f156959a640 1 --2- 172.21.15.76:0/3562330035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15540047c0 0x7f1554004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:49.260 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.255+0000 7f156959a640 1 --2- 172.21.15.76:0/3562330035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1554005180 0x7f155400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:49.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.255+0000 7f156959a640 1 -- 172.21.15.76:0/3562330035 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c070c60 con 0x7f1554003970 2023-11-09T03:53:49.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.255+0000 7f156959a640 1 -- 172.21.15.76:0/3562330035 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155400c120 con 0x7f15540047c0 2023-11-09T03:53:49.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.255+0000 7f156959a640 1 -- 172.21.15.76:0/3562330035 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f155400e4f0 con 0x7f1554005180 2023-11-09T03:53:49.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.255+0000 7f156a59c640 1 --2- 172.21.15.76:0/3562330035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1554003970 0x7f1554003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:49.261 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.255+0000 7f1569d9b640 1 --2- 172.21.15.76:0/3562330035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15540047c0 0x7f1554004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:49.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.255+0000 7f156ad9d640 1 --2- 172.21.15.76:0/3562330035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1554005180 0x7f155400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:49.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.255+0000 7f156a59c640 1 -- 172.21.15.76:0/3562330035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1554005180 msgr2=0x7f155400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:49.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.255+0000 7f156a59c640 1 --2- 172.21.15.76:0/3562330035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1554005180 0x7f155400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:49.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.255+0000 7f156a59c640 1 -- 172.21.15.76:0/3562330035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15540047c0 msgr2=0x7f1554004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:49.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.255+0000 7f156a59c640 1 --2- 172.21.15.76:0/3562330035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15540047c0 0x7f1554004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:49.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.255+0000 7f156a59c640 1 -- 172.21.15.76:0/3562330035 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1554003520 con 0x7f1554003970 2023-11-09T03:53:49.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.255+0000 7f156ad9d640 1 -- 172.21.15.76:0/3562330035 reap_dead start 2023-11-09T03:53:49.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.256+0000 7f156a59c640 1 --2- 172.21.15.76:0/3562330035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1554003970 0x7f1554003d90 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f155c001750 tx=0x7f15540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76b4e944e3d2bc server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:49.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.256+0000 7f156959a640 1 -- 172.21.15.76:0/3562330035 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c0052d0 con 0x7f1554003970 2023-11-09T03:53:49.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.256+0000 7f156959a640 1 -- 172.21.15.76:0/3562330035 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155c02e050 con 0x7f1554003970 2023-11-09T03:53:49.264 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.256+0000 7f156959a640 1 -- 172.21.15.76:0/3562330035 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c003830 con 0x7f1554003970 2023-11-09T03:53:49.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.257+0000 7f1570bee640 1 -- 172.21.15.76:0/3562330035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1554003970 msgr2=0x7f1554003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:49.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.257+0000 7f1570bee640 1 --2- 172.21.15.76:0/3562330035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1554003970 0x7f1554003d90 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f155c001750 tx=0x7f15540057f0 comp rx=0 tx=0).stop 2023-11-09T03:53:49.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.257+0000 7f1570bee640 1 -- 172.21.15.76:0/3562330035 shutdown_connections 2023-11-09T03:53:49.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.257+0000 7f1570bee640 1 --2- 172.21.15.76:0/3562330035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1554003970 0x7f1554003d90 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:49.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.257+0000 7f1570bee640 1 -- 172.21.15.76:0/3562330035 >> 172.21.15.76:0/3562330035 conn(0x7f156c0cb530 msgr2=0x7f156c0cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:49.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.257+0000 7f1570bee640 1 -- 172.21.15.76:0/3562330035 shutdown_connections 2023-11-09T03:53:49.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.257+0000 7f1570bee640 1 -- 172.21.15.76:0/3562330035 wait complete. 2023-11-09T03:53:49.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.258+0000 7f1570bee640 1 Processor -- start 2023-11-09T03:53:49.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.258+0000 7f1570bee640 1 -- start start 2023-11-09T03:53:49.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.259+0000 7f1570bee640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1554003970 0x7f156c1508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:49.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.259+0000 7f1570bee640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f156c06c6b0 0x7f156c0c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:49.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.259+0000 7f1570bee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f156c06cb30 0x7f156c0c44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:49.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.259+0000 7f1570bee640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c15b3a0 con 0x7f156c06c6b0 2023-11-09T03:53:49.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.259+0000 7f1570bee640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c1558d0 con 0x7f156c06cb30 2023-11-09T03:53:49.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.259+0000 7f1570bee640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156c1ee170 con 0x7f1554003970 2023-11-09T03:53:49.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.259+0000 7f1569d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f156c06c6b0 0x7f156c0c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:49.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.259+0000 7f1569d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f156c06c6b0 0x7f156c0c3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47924/0 (socket says 172.21.15.76:47924) 2023-11-09T03:53:49.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.259+0000 7f1569d9b640 1 -- 172.21.15.76:0/2850971504 learned_addr learned my addr 172.21.15.76:0/2850971504 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:49.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.259+0000 7f156a59c640 1 --2- 172.21.15.76:0/2850971504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1554003970 0x7f156c1508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:49.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.259+0000 7f156ad9d640 1 --2- 172.21.15.76:0/2850971504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f156c06cb30 0x7f156c0c44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:49.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.260+0000 7f1569d9b640 1 -- 172.21.15.76:0/2850971504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1554003970 msgr2=0x7f156c1508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:49.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.260+0000 7f1569d9b640 1 --2- 172.21.15.76:0/2850971504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1554003970 0x7f156c1508f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:49.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.260+0000 7f1569d9b640 1 -- 172.21.15.76:0/2850971504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f156c06cb30 msgr2=0x7f156c0c44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:49.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.260+0000 7f1569d9b640 1 --2- 172.21.15.76:0/2850971504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f156c06cb30 0x7f156c0c44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:49.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.260+0000 7f1569d9b640 1 -- 172.21.15.76:0/2850971504 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c0c2fd0 con 0x7f156c06c6b0 2023-11-09T03:53:49.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.260+0000 7f1569d9b640 1 --2- 172.21.15.76:0/2850971504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f156c06c6b0 0x7f156c0c3f70 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f15600072c0 tx=0x7f1560007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:49.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.260+0000 7f155affd640 1 -- 172.21.15.76:0/2850971504 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1560004ea0 con 0x7f156c06c6b0 2023-11-09T03:53:49.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.260+0000 7f155affd640 1 -- 172.21.15.76:0/2850971504 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156000f8e0 con 0x7f156c06c6b0 2023-11-09T03:53:49.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.260+0000 7f1570bee640 1 -- 172.21.15.76:0/2850971504 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156c0c32b0 con 0x7f156c06c6b0 2023-11-09T03:53:49.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.260+0000 7f155affd640 1 -- 172.21.15.76:0/2850971504 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156000fc90 con 0x7f156c06c6b0 2023-11-09T03:53:49.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.260+0000 7f1570bee640 1 -- 172.21.15.76:0/2850971504 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156c0c3870 con 0x7f156c06c6b0 2023-11-09T03:53:49.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.261+0000 7f1570bee640 1 -- 172.21.15.76:0/2850971504 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f156c15b4e0 con 0x7f156c06c6b0 2023-11-09T03:53:49.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.261+0000 7f155affd640 1 -- 172.21.15.76:0/2850971504 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f156001d460 con 0x7f156c06c6b0 2023-11-09T03:53:49.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.262+0000 7f155affd640 1 --2- 172.21.15.76:0/2850971504 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f154c06aea0 0x7f154c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:49.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.262+0000 7f155affd640 1 -- 172.21.15.76:0/2850971504 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8223+0+0 (secure 0 0 0) 0x7f1560089f30 con 0x7f156c06c6b0 2023-11-09T03:53:49.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.262+0000 7f156a59c640 1 --2- 172.21.15.76:0/2850971504 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f154c06aea0 0x7f154c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:49.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.263+0000 7f156a59c640 1 --2- 172.21.15.76:0/2850971504 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f154c06aea0 0x7f154c06d360 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f155c033000 tx=0x7f155c009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:49.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.266+0000 7f155affd640 1 -- 172.21.15.76:0/2850971504 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f156c15b4e0 con 0x7f156c06c6b0 2023-11-09T03:53:49.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.468+0000 7f1570bee640 1 -- 172.21.15.76:0/2850971504 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f156c15b4e0 con 0x7f156c06c6b0 2023-11-09T03:53:49.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.469+0000 7f155affd640 1 -- 172.21.15.76:0/2850971504 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+4937 (secure 0 0 0) 0x7f156c15b4e0 con 0x7f156c06c6b0 2023-11-09T03:53:49.469 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:53:49.470 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":118,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:53:42.831863+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":8772},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8772":{"gid":8772,"name":"a","rank":1,"incarnation":113,"state":"up:active","state_seq":7,"addr":"172.21.15.76:6835/2075292158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2075292158},{"type":"v1","addr":"172.21.15.76:6835","nonce":2075292158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:53:49.470 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 118 2023-11-09T03:53:49.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.470+0000 7f1570bee640 1 -- 172.21.15.76:0/2850971504 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f154c06aea0 msgr2=0x7f154c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:49.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.470+0000 7f1570bee640 1 --2- 172.21.15.76:0/2850971504 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f154c06aea0 0x7f154c06d360 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f155c033000 tx=0x7f155c009030 comp rx=0 tx=0).stop 2023-11-09T03:53:49.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.470+0000 7f1570bee640 1 -- 172.21.15.76:0/2850971504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f156c06c6b0 msgr2=0x7f156c0c3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:49.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.470+0000 7f1570bee640 1 --2- 172.21.15.76:0/2850971504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f156c06c6b0 0x7f156c0c3f70 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f15600072c0 tx=0x7f1560007780 comp rx=0 tx=0).stop 2023-11-09T03:53:49.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.470+0000 7f1570bee640 1 -- 172.21.15.76:0/2850971504 shutdown_connections 2023-11-09T03:53:49.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.470+0000 7f1570bee640 1 --2- 172.21.15.76:0/2850971504 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f154c06aea0 0x7f154c06d360 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:49.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.470+0000 7f1570bee640 1 --2- 172.21.15.76:0/2850971504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f156c06cb30 0x7f156c0c44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:49.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.471+0000 7f1570bee640 1 --2- 172.21.15.76:0/2850971504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f156c06c6b0 0x7f156c0c3f70 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:49.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.471+0000 7f1570bee640 1 --2- 172.21.15.76:0/2850971504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1554003970 0x7f156c1508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:49.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.471+0000 7f1570bee640 1 -- 172.21.15.76:0/2850971504 >> 172.21.15.76:0/2850971504 conn(0x7f156c0cb530 msgr2=0x7f156c0cea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:49.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.471+0000 7f1570bee640 1 -- 172.21.15.76:0/2850971504 shutdown_connections 2023-11-09T03:53:49.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:49.471+0000 7f1570bee640 1 -- 172.21.15.76:0/2850971504 wait complete. 2023-11-09T03:53:49.482 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:53:52.124 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~7s 2023-11-09T03:53:54.483 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:53:54.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.594+0000 7f915927c640 1 Processor -- start 2023-11-09T03:53:54.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.594+0000 7f915927c640 1 -- start start 2023-11-09T03:53:54.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.595+0000 7f915927c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f915415d100 0x7f91540ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:54.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.595+0000 7f915927c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91540b32b0 con 0x7f91540d1be0 2023-11-09T03:53:54.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.595+0000 7f915927c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91540cee10 con 0x7f915415d4e0 2023-11-09T03:53:54.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.595+0000 7f915927c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91540b2980 con 0x7f915415d100 2023-11-09T03:53:54.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.595+0000 7f91537fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f91540d1be0 0x7f91540b3740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40150/0 (socket says 172.21.15.76:40150) 2023-11-09T03:53:54.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.595+0000 7f91537fe640 1 -- 172.21.15.76:0/684321855 learned_addr learned my addr 172.21.15.76:0/684321855 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:53:54.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.596+0000 7f9152ffd640 1 --2- 172.21.15.76:0/684321855 >> v2:172.21.15.113:3300/0 conn(0x7f915415d100 0x7f91540ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:54.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.596+0000 7f9151ffb640 1 -- 172.21.15.76:0/684321855 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 122410033 0 0) 0x7f91540b32b0 con 0x7f91540d1be0 2023-11-09T03:53:54.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.596+0000 7f9151ffb640 1 -- 172.21.15.76:0/684321855 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f913c003520 con 0x7f91540d1be0 2023-11-09T03:53:54.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.596+0000 7f9151ffb640 1 -- 172.21.15.76:0/684321855 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2218481089 0 0) 0x7f913c003520 con 0x7f91540d1be0 2023-11-09T03:53:54.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.596+0000 7f9152ffd640 1 -- 172.21.15.76:0/684321855 >> v1:172.21.15.113:6789/0 conn(0x7f915415d4e0 legacy=0x7f91540d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:54.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.596+0000 7f9152ffd640 1 -- 172.21.15.76:0/684321855 >> v1:172.21.15.76:6789/0 conn(0x7f91540d1be0 legacy=0x7f91540b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:54.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.596+0000 7f9152ffd640 1 -- 172.21.15.76:0/684321855 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91540b2c60 con 0x7f915415d100 2023-11-09T03:53:54.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.597+0000 7f9152ffd640 1 --2- 172.21.15.76:0/684321855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f915415d100 0x7f91540ce510 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9144009de0 tx=0x7f914402be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8f518b8c2834fec server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:54.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.597+0000 7f9151ffb640 1 -- 172.21.15.76:0/684321855 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c001670 con 0x7f915415d100 2023-11-09T03:53:54.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.597+0000 7f9151ffb640 1 -- 172.21.15.76:0/684321855 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914402c980 con 0x7f915415d100 2023-11-09T03:53:54.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.597+0000 7f9151ffb640 1 -- 172.21.15.76:0/684321855 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914402cd10 con 0x7f915415d100 2023-11-09T03:53:54.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.597+0000 7f915927c640 1 -- 172.21.15.76:0/684321855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f915415d100 msgr2=0x7f91540ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:54.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.597+0000 7f915927c640 1 --2- 172.21.15.76:0/684321855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f915415d100 0x7f91540ce510 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9144009de0 tx=0x7f914402be50 comp rx=0 tx=0).stop 2023-11-09T03:53:54.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.597+0000 7f915927c640 1 -- 172.21.15.76:0/684321855 shutdown_connections 2023-11-09T03:53:54.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.597+0000 7f915927c640 1 --2- 172.21.15.76:0/684321855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f915415d100 0x7f91540ce510 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:54.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.597+0000 7f915927c640 1 -- 172.21.15.76:0/684321855 >> 172.21.15.76:0/684321855 conn(0x7f91540d2ef0 msgr2=0x7f91540d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:54.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.598+0000 7f915927c640 1 -- 172.21.15.76:0/684321855 shutdown_connections 2023-11-09T03:53:54.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.598+0000 7f915927c640 1 -- 172.21.15.76:0/684321855 wait complete. 2023-11-09T03:53:54.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.599+0000 7f915927c640 1 Processor -- start 2023-11-09T03:53:54.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.599+0000 7f915927c640 1 -- start start 2023-11-09T03:53:54.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.600+0000 7f915927c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91540d1be0 0x7f915415a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:54.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.600+0000 7f915927c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f915415d100 0x7f915406acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:54.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.600+0000 7f915927c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f915415d4e0 0x7f91540b8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:54.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.600+0000 7f915927c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91540cb140 con 0x7f91540d1be0 2023-11-09T03:53:54.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.600+0000 7f915927c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91540ca4c0 con 0x7f915415d4e0 2023-11-09T03:53:54.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.600+0000 7f915927c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91540ca7a0 con 0x7f915415d100 2023-11-09T03:53:54.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.600+0000 7f9152ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91540d1be0 0x7f915415a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:54.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.600+0000 7f9152ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91540d1be0 0x7f915415a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49194/0 (socket says 172.21.15.76:49194) 2023-11-09T03:53:54.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.600+0000 7f9152ffd640 1 -- 172.21.15.76:0/2001304248 learned_addr learned my addr 172.21.15.76:0/2001304248 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:54.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.600+0000 7f91537fe640 1 --2- 172.21.15.76:0/2001304248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f915415d4e0 0x7f91540b8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:54.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.600+0000 7f91527fc640 1 --2- 172.21.15.76:0/2001304248 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f915415d100 0x7f915406acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:54.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.600+0000 7f9152ffd640 1 -- 172.21.15.76:0/2001304248 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f915415d100 msgr2=0x7f915406acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:54.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.600+0000 7f9152ffd640 1 --2- 172.21.15.76:0/2001304248 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f915415d100 0x7f915406acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:54.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.600+0000 7f9152ffd640 1 -- 172.21.15.76:0/2001304248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f915415d4e0 msgr2=0x7f91540b8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:54.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.600+0000 7f9152ffd640 1 --2- 172.21.15.76:0/2001304248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f915415d4e0 0x7f91540b8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:54.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.600+0000 7f9152ffd640 1 -- 172.21.15.76:0/2001304248 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9144009a70 con 0x7f91540d1be0 2023-11-09T03:53:54.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.600+0000 7f9152ffd640 1 --2- 172.21.15.76:0/2001304248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91540d1be0 0x7f915415a590 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f9144009de0 tx=0x7f9144004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:54.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.600+0000 7f913b7fe640 1 -- 172.21.15.76:0/2001304248 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9144009450 con 0x7f91540d1be0 2023-11-09T03:53:54.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.600+0000 7f913b7fe640 1 -- 172.21.15.76:0/2001304248 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9144003990 con 0x7f91540d1be0 2023-11-09T03:53:54.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.600+0000 7f913b7fe640 1 -- 172.21.15.76:0/2001304248 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9144003d40 con 0x7f91540d1be0 2023-11-09T03:53:54.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.600+0000 7f915927c640 1 -- 172.21.15.76:0/2001304248 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91540caa80 con 0x7f91540d1be0 2023-11-09T03:53:54.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.601+0000 7f915927c640 1 -- 172.21.15.76:0/2001304248 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f915415aad0 con 0x7f91540d1be0 2023-11-09T03:53:54.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.601+0000 7f915927c640 1 -- 172.21.15.76:0/2001304248 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91540684b0 con 0x7f91540d1be0 2023-11-09T03:53:54.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.602+0000 7f913b7fe640 1 -- 172.21.15.76:0/2001304248 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f914403b050 con 0x7f91540d1be0 2023-11-09T03:53:54.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.602+0000 7f913b7fe640 1 --2- 172.21.15.76:0/2001304248 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f914006ae70 0x7f914006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:54.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.602+0000 7f913b7fe640 1 -- 172.21.15.76:0/2001304248 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8195+0+0 (secure 0 0 0) 0x7f91440ac580 con 0x7f91540d1be0 2023-11-09T03:53:54.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.603+0000 7f91527fc640 1 --2- 172.21.15.76:0/2001304248 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f914006ae70 0x7f914006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:54.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.603+0000 7f91527fc640 1 --2- 172.21.15.76:0/2001304248 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f914006ae70 0x7f914006d330 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f91540a0a90 tx=0x7f9148009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:54.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.607+0000 7f913b7fe640 1 -- 172.21.15.76:0/2001304248 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9144076cd0 con 0x7f91540d1be0 2023-11-09T03:53:54.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.807+0000 7f915927c640 1 -- 172.21.15.76:0/2001304248 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91540d2cd0 con 0x7f91540d1be0 2023-11-09T03:53:54.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.807+0000 7f913b7fe640 1 -- 172.21.15.76:0/2001304248 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+4937 (secure 0 0 0) 0x7f914404d020 con 0x7f91540d1be0 2023-11-09T03:53:54.808 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:53:54.808 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":118,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:53:42.831863+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":8772},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8772":{"gid":8772,"name":"a","rank":1,"incarnation":113,"state":"up:active","state_seq":7,"addr":"172.21.15.76:6835/2075292158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2075292158},{"type":"v1","addr":"172.21.15.76:6835","nonce":2075292158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:53:54.808 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 118 2023-11-09T03:53:54.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.809+0000 7f915927c640 1 -- 172.21.15.76:0/2001304248 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f914006ae70 msgr2=0x7f914006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:54.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.809+0000 7f915927c640 1 --2- 172.21.15.76:0/2001304248 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f914006ae70 0x7f914006d330 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f91540a0a90 tx=0x7f9148009040 comp rx=0 tx=0).stop 2023-11-09T03:53:54.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.809+0000 7f915927c640 1 -- 172.21.15.76:0/2001304248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91540d1be0 msgr2=0x7f915415a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:54.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.809+0000 7f915927c640 1 --2- 172.21.15.76:0/2001304248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91540d1be0 0x7f915415a590 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f9144009de0 tx=0x7f9144004a40 comp rx=0 tx=0).stop 2023-11-09T03:53:54.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.809+0000 7f915927c640 1 -- 172.21.15.76:0/2001304248 shutdown_connections 2023-11-09T03:53:54.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.809+0000 7f915927c640 1 --2- 172.21.15.76:0/2001304248 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f914006ae70 0x7f914006d330 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:54.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.809+0000 7f915927c640 1 --2- 172.21.15.76:0/2001304248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f915415d4e0 0x7f91540b8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:54.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.809+0000 7f915927c640 1 --2- 172.21.15.76:0/2001304248 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f915415d100 0x7f915406acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:54.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.809+0000 7f915927c640 1 --2- 172.21.15.76:0/2001304248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91540d1be0 0x7f915415a590 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:54.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.809+0000 7f915927c640 1 -- 172.21.15.76:0/2001304248 >> 172.21.15.76:0/2001304248 conn(0x7f91540d2ef0 msgr2=0x7f91540b4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:54.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.810+0000 7f915927c640 1 -- 172.21.15.76:0/2001304248 shutdown_connections 2023-11-09T03:53:54.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:54.810+0000 7f915927c640 1 -- 172.21.15.76:0/2001304248 wait complete. 2023-11-09T03:53:54.821 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:53:58.736 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~13s 2023-11-09T03:53:59.821 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:53:59.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.933+0000 7faf752da640 1 Processor -- start 2023-11-09T03:53:59.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.933+0000 7faf752da640 1 -- start start 2023-11-09T03:53:59.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.933+0000 7faf752da640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faf7015d100 0x7faf700ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:59.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.933+0000 7faf752da640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf7015d4e0 0x7faf700d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:59.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.933+0000 7faf752da640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf700b32b0 con 0x7faf700d3b60 2023-11-09T03:53:59.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.933+0000 7faf752da640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf700cee10 con 0x7faf7015d100 2023-11-09T03:53:59.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.933+0000 7faf752da640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf700b4750 con 0x7faf7015d4e0 2023-11-09T03:53:59.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.934+0000 7faf6f7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7faf700d3b60 0x7faf700cf520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40154/0 (socket says 172.21.15.76:40154) 2023-11-09T03:53:59.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.934+0000 7faf6f7fe640 1 -- 172.21.15.76:0/3549961102 learned_addr learned my addr 172.21.15.76:0/3549961102 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:53:59.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.934+0000 7faf6effd640 1 --2- 172.21.15.76:0/3549961102 >> v2:172.21.15.113:3300/0 conn(0x7faf7015d100 0x7faf700ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:59.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.934+0000 7faf6e7fc640 1 --2- 172.21.15.76:0/3549961102 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf7015d4e0 0x7faf700d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:59.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.934+0000 7faf6dffb640 1 -- 172.21.15.76:0/3549961102 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 849477261 0 0) 0x7faf700b32b0 con 0x7faf700d3b60 2023-11-09T03:53:59.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.934+0000 7faf6dffb640 1 -- 172.21.15.76:0/3549961102 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf58003520 con 0x7faf700d3b60 2023-11-09T03:53:59.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.934+0000 7faf6effd640 1 -- 172.21.15.76:0/3549961102 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf7015d4e0 msgr2=0x7faf700d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:53:59.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.934+0000 7faf6effd640 1 --2- 172.21.15.76:0/3549961102 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf7015d4e0 0x7faf700d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:59.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.934+0000 7faf6effd640 1 -- 172.21.15.76:0/3549961102 >> v1:172.21.15.76:6789/0 conn(0x7faf700d3b60 legacy=0x7faf700cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:59.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.934+0000 7faf6effd640 1 -- 172.21.15.76:0/3549961102 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf700b2980 con 0x7faf7015d100 2023-11-09T03:53:59.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.935+0000 7faf6effd640 1 --2- 172.21.15.76:0/3549961102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7015d100 0x7faf700ce510 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7faf60009790 tx=0x7faf6002b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1ed4c2f5c9717ee server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:59.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.935+0000 7faf6dffb640 1 -- 172.21.15.76:0/3549961102 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf60031800 con 0x7faf7015d100 2023-11-09T03:53:59.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.935+0000 7faf6dffb640 1 -- 172.21.15.76:0/3549961102 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf60031960 con 0x7faf7015d100 2023-11-09T03:53:59.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.935+0000 7faf6dffb640 1 -- 172.21.15.76:0/3549961102 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf60031cf0 con 0x7faf7015d100 2023-11-09T03:53:59.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.936+0000 7faf752da640 1 -- 172.21.15.76:0/3549961102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7015d100 msgr2=0x7faf700ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:59.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.936+0000 7faf752da640 1 --2- 172.21.15.76:0/3549961102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7015d100 0x7faf700ce510 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7faf60009790 tx=0x7faf6002b8a0 comp rx=0 tx=0).stop 2023-11-09T03:53:59.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.936+0000 7faf752da640 1 -- 172.21.15.76:0/3549961102 shutdown_connections 2023-11-09T03:53:59.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.936+0000 7faf752da640 1 --2- 172.21.15.76:0/3549961102 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf7015d4e0 0x7faf700d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:59.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.936+0000 7faf752da640 1 --2- 172.21.15.76:0/3549961102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7015d100 0x7faf700ce510 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:59.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.936+0000 7faf752da640 1 -- 172.21.15.76:0/3549961102 >> 172.21.15.76:0/3549961102 conn(0x7faf700d2ef0 msgr2=0x7faf700d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:53:59.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.936+0000 7faf752da640 1 -- 172.21.15.76:0/3549961102 shutdown_connections 2023-11-09T03:53:59.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.936+0000 7faf752da640 1 -- 172.21.15.76:0/3549961102 wait complete. 2023-11-09T03:53:59.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.937+0000 7faf752da640 1 Processor -- start 2023-11-09T03:53:59.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.937+0000 7faf752da640 1 -- start start 2023-11-09T03:53:59.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.938+0000 7faf752da640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf700d3b60 0x7faf700b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:59.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.938+0000 7faf752da640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf7015d100 0x7faf7015a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:59.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.938+0000 7faf752da640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7015d4e0 0x7faf7015d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:59.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.938+0000 7faf752da640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7015fda0 con 0x7faf7015d100 2023-11-09T03:53:59.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.938+0000 7faf752da640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf700b11c0 con 0x7faf7015d4e0 2023-11-09T03:53:59.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.938+0000 7faf752da640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf7015f120 con 0x7faf700d3b60 2023-11-09T03:53:59.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.938+0000 7faf6e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf7015d100 0x7faf7015a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:59.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.938+0000 7faf6e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf7015d100 0x7faf7015a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49204/0 (socket says 172.21.15.76:49204) 2023-11-09T03:53:59.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.938+0000 7faf6e7fc640 1 -- 172.21.15.76:0/3699928110 learned_addr learned my addr 172.21.15.76:0/3699928110 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:53:59.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.938+0000 7faf6f7fe640 1 --2- 172.21.15.76:0/3699928110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7015d4e0 0x7faf7015d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:59.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.938+0000 7faf6effd640 1 --2- 172.21.15.76:0/3699928110 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf700d3b60 0x7faf700b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:59.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.938+0000 7faf6e7fc640 1 -- 172.21.15.76:0/3699928110 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf700d3b60 msgr2=0x7faf700b0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:59.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.938+0000 7faf6e7fc640 1 --2- 172.21.15.76:0/3699928110 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf700d3b60 0x7faf700b0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:59.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.938+0000 7faf6e7fc640 1 -- 172.21.15.76:0/3699928110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7015d4e0 msgr2=0x7faf7015d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:53:59.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.938+0000 7faf6e7fc640 1 --2- 172.21.15.76:0/3699928110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7015d4e0 0x7faf7015d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:53:59.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.938+0000 7faf6e7fc640 1 -- 172.21.15.76:0/3699928110 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf600093a0 con 0x7faf7015d100 2023-11-09T03:53:59.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.938+0000 7faf6f7fe640 1 --2- 172.21.15.76:0/3699928110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7015d4e0 0x7faf7015d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:53:59.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.938+0000 7faf6e7fc640 1 --2- 172.21.15.76:0/3699928110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf7015d100 0x7faf7015a980 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7faf6400e9b0 tx=0x7faf6400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:59.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.938+0000 7faf577fe640 1 -- 172.21.15.76:0/3699928110 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf64009670 con 0x7faf7015d100 2023-11-09T03:53:59.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.939+0000 7faf577fe640 1 -- 172.21.15.76:0/3699928110 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6400c9a0 con 0x7faf7015d100 2023-11-09T03:53:59.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.939+0000 7faf752da640 1 -- 172.21.15.76:0/3699928110 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf7015f400 con 0x7faf7015d100 2023-11-09T03:53:59.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.939+0000 7faf577fe640 1 -- 172.21.15.76:0/3699928110 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6400cd50 con 0x7faf7015d100 2023-11-09T03:53:59.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.939+0000 7faf752da640 1 -- 172.21.15.76:0/3699928110 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf7015f9c0 con 0x7faf7015d100 2023-11-09T03:53:59.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.939+0000 7faf752da640 1 -- 172.21.15.76:0/3699928110 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf700684b0 con 0x7faf7015d100 2023-11-09T03:53:59.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.940+0000 7faf577fe640 1 -- 172.21.15.76:0/3699928110 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faf64013470 con 0x7faf7015d100 2023-11-09T03:53:59.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.940+0000 7faf577fe640 1 --2- 172.21.15.76:0/3699928110 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf5c06ae70 0x7faf5c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:53:59.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.941+0000 7faf577fe640 1 -- 172.21.15.76:0/3699928110 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8195+0+0 (secure 0 0 0) 0x7faf6408a240 con 0x7faf7015d100 2023-11-09T03:53:59.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.941+0000 7faf6effd640 1 --2- 172.21.15.76:0/3699928110 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf5c06ae70 0x7faf5c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:53:59.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.941+0000 7faf6effd640 1 --2- 172.21.15.76:0/3699928110 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf5c06ae70 0x7faf5c06d330 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7faf60036000 tx=0x7faf6003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:53:59.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:53:59.945+0000 7faf577fe640 1 -- 172.21.15.76:0/3699928110 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faf64054970 con 0x7faf7015d100 2023-11-09T03:54:00.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:00.148+0000 7faf752da640 1 -- 172.21.15.76:0/3699928110 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf700d2cd0 con 0x7faf7015d100 2023-11-09T03:54:00.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:00.148+0000 7faf577fe640 1 -- 172.21.15.76:0/3699928110 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+4937 (secure 0 0 0) 0x7faf64017090 con 0x7faf7015d100 2023-11-09T03:54:00.148 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:54:00.148 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":118,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:53:42.831863+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":8772},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8772":{"gid":8772,"name":"a","rank":1,"incarnation":113,"state":"up:active","state_seq":7,"addr":"172.21.15.76:6835/2075292158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2075292158},{"type":"v1","addr":"172.21.15.76:6835","nonce":2075292158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:54:00.149 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 118 2023-11-09T03:54:00.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:00.150+0000 7faf752da640 1 -- 172.21.15.76:0/3699928110 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf5c06ae70 msgr2=0x7faf5c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:00.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:00.150+0000 7faf752da640 1 --2- 172.21.15.76:0/3699928110 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf5c06ae70 0x7faf5c06d330 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7faf60036000 tx=0x7faf6003c040 comp rx=0 tx=0).stop 2023-11-09T03:54:00.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:00.150+0000 7faf752da640 1 -- 172.21.15.76:0/3699928110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf7015d100 msgr2=0x7faf7015a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:00.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:00.150+0000 7faf752da640 1 --2- 172.21.15.76:0/3699928110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf7015d100 0x7faf7015a980 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7faf6400e9b0 tx=0x7faf6400ee70 comp rx=0 tx=0).stop 2023-11-09T03:54:00.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:00.150+0000 7faf752da640 1 -- 172.21.15.76:0/3699928110 shutdown_connections 2023-11-09T03:54:00.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:00.150+0000 7faf752da640 1 --2- 172.21.15.76:0/3699928110 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf5c06ae70 0x7faf5c06d330 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:00.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:00.150+0000 7faf752da640 1 --2- 172.21.15.76:0/3699928110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7015d4e0 0x7faf7015d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:00.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:00.150+0000 7faf752da640 1 --2- 172.21.15.76:0/3699928110 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf7015d100 0x7faf7015a980 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:00.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:00.150+0000 7faf752da640 1 --2- 172.21.15.76:0/3699928110 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf700d3b60 0x7faf700b0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:00.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:00.150+0000 7faf752da640 1 -- 172.21.15.76:0/3699928110 >> 172.21.15.76:0/3699928110 conn(0x7faf700d2ef0 msgr2=0x7faf700c95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:00.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:00.150+0000 7faf752da640 1 -- 172.21.15.76:0/3699928110 shutdown_connections 2023-11-09T03:54:00.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:00.150+0000 7faf752da640 1 -- 172.21.15.76:0/3699928110 wait complete. 2023-11-09T03:54:00.161 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:54:02.424 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:54:02.428 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:54:05.162 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:54:05.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.271+0000 7f6fc73e7640 1 Processor -- start 2023-11-09T03:54:05.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.271+0000 7f6fc73e7640 1 -- start start 2023-11-09T03:54:05.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.272+0000 7f6fc73e7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6fc00b0c40 0x7f6fc006fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:05.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.272+0000 7f6fc73e7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6fc006bb60 0x7f6fc0070530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:05.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.272+0000 7f6fc73e7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc0071500 con 0x7f6fc0159e40 2023-11-09T03:54:05.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.272+0000 7f6fc73e7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc0070a70 con 0x7f6fc006bb60 2023-11-09T03:54:05.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.272+0000 7f6fc73e7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc015dae0 con 0x7f6fc00b0c40 2023-11-09T03:54:05.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.272+0000 7f6fc5984640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6fc006bb60 0x7f6fc0070530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:05.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.272+0000 7f6fc5984640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6fc006bb60 0x7f6fc0070530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40548/0 (socket says 172.21.15.76:40548) 2023-11-09T03:54:05.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.272+0000 7f6fc5984640 1 -- 172.21.15.76:0/2455697354 learned_addr learned my addr 172.21.15.76:0/2455697354 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:05.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.272+0000 7f6fc4982640 1 --2- 172.21.15.76:0/2455697354 >> v2:172.21.15.113:3300/0 conn(0x7f6fc00b0c40 0x7f6fc006fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:05.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.273+0000 7f6fc5984640 1 -- 172.21.15.76:0/2455697354 >> v2:172.21.15.113:3300/0 conn(0x7f6fc00b0c40 msgr2=0x7f6fc006fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:05.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.273+0000 7f6fc5984640 1 --2- 172.21.15.76:0/2455697354 >> v2:172.21.15.113:3300/0 conn(0x7f6fc00b0c40 0x7f6fc006fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:05.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.273+0000 7f6fc5984640 1 -- 172.21.15.76:0/2455697354 >> v1:172.21.15.113:6789/0 conn(0x7f6fc0159e40 legacy=0x7f6fc00b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:05.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.273+0000 7f6fc5984640 1 -- 172.21.15.76:0/2455697354 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc0070c60 con 0x7f6fc006bb60 2023-11-09T03:54:05.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.273+0000 7f6fc4982640 1 --2- 172.21.15.76:0/2455697354 >> v2:172.21.15.113:3300/0 conn(0x7f6fc00b0c40 0x7f6fc006fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:54:05.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.273+0000 7f6fc5984640 1 --2- 172.21.15.76:0/2455697354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fc006bb60 0x7f6fc0070530 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f6fc006fdc0 tx=0x7f6fb802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370e55a6bff9f30c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:05.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.273+0000 7f6fb7fff640 1 -- 172.21.15.76:0/2455697354 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb802c9f0 con 0x7f6fc006bb60 2023-11-09T03:54:05.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.273+0000 7f6fb7fff640 1 -- 172.21.15.76:0/2455697354 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb802cb50 con 0x7f6fc006bb60 2023-11-09T03:54:05.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.274+0000 7f6fb7fff640 1 -- 172.21.15.76:0/2455697354 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb80343f0 con 0x7f6fc006bb60 2023-11-09T03:54:05.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.274+0000 7f6fc73e7640 1 -- 172.21.15.76:0/2455697354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fc006bb60 msgr2=0x7f6fc0070530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:05.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.274+0000 7f6fc73e7640 1 --2- 172.21.15.76:0/2455697354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fc006bb60 0x7f6fc0070530 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f6fc006fdc0 tx=0x7f6fb802be50 comp rx=0 tx=0).stop 2023-11-09T03:54:05.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.274+0000 7f6fc73e7640 1 -- 172.21.15.76:0/2455697354 shutdown_connections 2023-11-09T03:54:05.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.274+0000 7f6fc73e7640 1 --2- 172.21.15.76:0/2455697354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fc006bb60 0x7f6fc0070530 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:05.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.274+0000 7f6fc73e7640 1 --2- 172.21.15.76:0/2455697354 >> v2:172.21.15.113:3300/0 conn(0x7f6fc00b0c40 0x7f6fc006fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:05.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.274+0000 7f6fc73e7640 1 -- 172.21.15.76:0/2455697354 >> 172.21.15.76:0/2455697354 conn(0x7f6fc00cb530 msgr2=0x7f6fc00cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:05.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.274+0000 7f6fc73e7640 1 -- 172.21.15.76:0/2455697354 shutdown_connections 2023-11-09T03:54:05.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.274+0000 7f6fc73e7640 1 -- 172.21.15.76:0/2455697354 wait complete. 2023-11-09T03:54:05.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.275+0000 7f6fc73e7640 1 Processor -- start 2023-11-09T03:54:05.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.275+0000 7f6fc73e7640 1 -- start start 2023-11-09T03:54:05.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.276+0000 7f6fc73e7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fc006bb60 0x7f6fc00c25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:05.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.276+0000 7f6fc73e7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fc00b0c40 0x7f6fc00ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:05.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.276+0000 7f6fc73e7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fc0159e40 0x7f6fc0151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:05.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.276+0000 7f6fc73e7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc00c4390 con 0x7f6fc006bb60 2023-11-09T03:54:05.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.276+0000 7f6fc73e7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc00c4670 con 0x7f6fc00b0c40 2023-11-09T03:54:05.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.276+0000 7f6fc73e7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc00c4950 con 0x7f6fc0159e40 2023-11-09T03:54:05.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.276+0000 7f6fc5183640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fc006bb60 0x7f6fc00c25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:05.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.276+0000 7f6fc5183640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fc006bb60 0x7f6fc00c25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40558/0 (socket says 172.21.15.76:40558) 2023-11-09T03:54:05.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.276+0000 7f6fc5183640 1 -- 172.21.15.76:0/248774389 learned_addr learned my addr 172.21.15.76:0/248774389 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:05.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.276+0000 7f6fc5183640 1 -- 172.21.15.76:0/248774389 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fc0159e40 msgr2=0x7f6fc0151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:05.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.276+0000 7f6fc5984640 1 --2- 172.21.15.76:0/248774389 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fc0159e40 0x7f6fc0151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:05.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.276+0000 7f6fc5183640 1 --2- 172.21.15.76:0/248774389 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fc0159e40 0x7f6fc0151fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:05.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.276+0000 7f6fc4982640 1 --2- 172.21.15.76:0/248774389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fc00b0c40 0x7f6fc00ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:05.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.276+0000 7f6fc5183640 1 -- 172.21.15.76:0/248774389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fc00b0c40 msgr2=0x7f6fc00ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:05.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.277+0000 7f6fc5183640 1 --2- 172.21.15.76:0/248774389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fc00b0c40 0x7f6fc00ca510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:05.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.277+0000 7f6fc5183640 1 -- 172.21.15.76:0/248774389 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb8009b30 con 0x7f6fc006bb60 2023-11-09T03:54:05.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.277+0000 7f6fc4982640 1 --2- 172.21.15.76:0/248774389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fc00b0c40 0x7f6fc00ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:54:05.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.277+0000 7f6fc5183640 1 --2- 172.21.15.76:0/248774389 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fc006bb60 0x7f6fc00c25c0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00d990 tx=0x7f6fbc00de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:05.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.277+0000 7f6fb5ffb640 1 -- 172.21.15.76:0/248774389 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc00ed60 con 0x7f6fc006bb60 2023-11-09T03:54:05.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.277+0000 7f6fb5ffb640 1 -- 172.21.15.76:0/248774389 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fbc016440 con 0x7f6fc006bb60 2023-11-09T03:54:05.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.277+0000 7f6fc73e7640 1 -- 172.21.15.76:0/248774389 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc00c4c30 con 0x7f6fc006bb60 2023-11-09T03:54:05.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.277+0000 7f6fb5ffb640 1 -- 172.21.15.76:0/248774389 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc016830 con 0x7f6fc006bb60 2023-11-09T03:54:05.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.277+0000 7f6fc73e7640 1 -- 172.21.15.76:0/248774389 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc00c2b00 con 0x7f6fc006bb60 2023-11-09T03:54:05.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.278+0000 7f6fc73e7640 1 -- 172.21.15.76:0/248774389 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc00684b0 con 0x7f6fc006bb60 2023-11-09T03:54:05.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.279+0000 7f6fb5ffb640 1 -- 172.21.15.76:0/248774389 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6fbc01f3b0 con 0x7f6fc006bb60 2023-11-09T03:54:05.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.279+0000 7f6fb5ffb640 1 --2- 172.21.15.76:0/248774389 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6fa406ae70 0x7f6fa406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:05.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.279+0000 7f6fb5ffb640 1 -- 172.21.15.76:0/248774389 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8195+0+0 (secure 0 0 0) 0x7f6fbc08bee0 con 0x7f6fc006bb60 2023-11-09T03:54:05.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.279+0000 7f6fc4982640 1 --2- 172.21.15.76:0/248774389 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6fa406ae70 0x7f6fa406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:05.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.280+0000 7f6fc4982640 1 --2- 172.21.15.76:0/248774389 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6fa406ae70 0x7f6fa406d330 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6fc00c19f0 tx=0x7f6fb0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:05.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.283+0000 7f6fb5ffb640 1 -- 172.21.15.76:0/248774389 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6fbc056610 con 0x7f6fc006bb60 2023-11-09T03:54:05.348 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~20s 2023-11-09T03:54:05.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.483+0000 7f6fc73e7640 1 -- 172.21.15.76:0/248774389 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6fc00c9e50 con 0x7f6fc006bb60 2023-11-09T03:54:05.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.483+0000 7f6fb5ffb640 1 -- 172.21.15.76:0/248774389 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+4985 (secure 0 0 0) 0x7f6fbc01b090 con 0x7f6fc006bb60 2023-11-09T03:54:05.483 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:54:05.484 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":119,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:01.482328+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":8772},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8772":{"gid":8772,"name":"a","rank":1,"incarnation":113,"state":"up:active","state_seq":7,"addr":"172.21.15.76:6835/2075292158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2075292158},{"type":"v1","addr":"172.21.15.76:6835","nonce":2075292158}]},"join_fscid":-1,"laggy_since":"2023-11-09T03:54:01.482328+0000","export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:54:05.485 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 119 2023-11-09T03:54:05.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.485+0000 7f6fc73e7640 1 -- 172.21.15.76:0/248774389 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6fa406ae70 msgr2=0x7f6fa406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:05.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.485+0000 7f6fc73e7640 1 --2- 172.21.15.76:0/248774389 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6fa406ae70 0x7f6fa406d330 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6fc00c19f0 tx=0x7f6fb0008040 comp rx=0 tx=0).stop 2023-11-09T03:54:05.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.485+0000 7f6fc73e7640 1 -- 172.21.15.76:0/248774389 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fc006bb60 msgr2=0x7f6fc00c25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:05.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.485+0000 7f6fc73e7640 1 --2- 172.21.15.76:0/248774389 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fc006bb60 0x7f6fc00c25c0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00d990 tx=0x7f6fbc00de50 comp rx=0 tx=0).stop 2023-11-09T03:54:05.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.485+0000 7f6fc73e7640 1 -- 172.21.15.76:0/248774389 shutdown_connections 2023-11-09T03:54:05.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.485+0000 7f6fc73e7640 1 --2- 172.21.15.76:0/248774389 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6fa406ae70 0x7f6fa406d330 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:05.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.485+0000 7f6fc73e7640 1 --2- 172.21.15.76:0/248774389 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fc0159e40 0x7f6fc0151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:05.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.485+0000 7f6fc73e7640 1 --2- 172.21.15.76:0/248774389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fc00b0c40 0x7f6fc00ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:05.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.485+0000 7f6fc73e7640 1 --2- 172.21.15.76:0/248774389 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fc006bb60 0x7f6fc00c25c0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:05.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.485+0000 7f6fc73e7640 1 -- 172.21.15.76:0/248774389 >> 172.21.15.76:0/248774389 conn(0x7f6fc00cb530 msgr2=0x7f6fc00cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:05.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.485+0000 7f6fc73e7640 1 -- 172.21.15.76:0/248774389 shutdown_connections 2023-11-09T03:54:05.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.485+0000 7f6fc73e7640 1 -- 172.21.15.76:0/248774389 wait complete. 2023-11-09T03:54:05.496 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2023-11-09T03:54:05.496 DEBUG:teuthology.orchestra.run.smithi076:> sudo sysctl -n kernel.core_pattern 2023-11-09T03:54:05.522 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-09T03:54:05.523 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2023-11-09T03:54:05.524 DEBUG:teuthology.orchestra.run.smithi076:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2023-11-09T03:54:05.593 INFO:teuthology.orchestra.run.smithi076.stdout:1699502023.126502.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i a', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2023-11-09T03:54:05.595 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1699502023.126502.core'] 2023-11-09T03:54:05.595 INFO:tasks.cephfs.cephfs_test_case:Found core file 1699502023.126502.core, deleting it 2023-11-09T03:54:05.596 DEBUG:teuthology.orchestra.run.smithi076:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1699502023.126502.core 2023-11-09T03:54:05.665 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:54:05.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:54:05.666 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 down 2023-11-09T03:54:05.733 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:54:05.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.874+0000 7f80f8f9e640 1 Processor -- start 2023-11-09T03:54:05.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.874+0000 7f80f8f9e640 1 -- start start 2023-11-09T03:54:05.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.875+0000 7f80f8f9e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80f4159e50 0x7f80f4070840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:05.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.875+0000 7f80f8f9e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f80f406b7c0 0x7f80f40cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:05.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.875+0000 7f80f8f9e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80f406c680 con 0x7f80f415d7f0 2023-11-09T03:54:05.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.875+0000 7f80f8f9e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80f415e6e0 con 0x7f80f406b7c0 2023-11-09T03:54:05.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.875+0000 7f80f8f9e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f4071100 con 0x7f80f4159e50 2023-11-09T03:54:05.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.875+0000 7f80f3fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f80f415d7f0 0x7f80f406cb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:55736/0 (socket says 172.21.15.76:55736) 2023-11-09T03:54:05.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.875+0000 7f80f3fff640 1 -- 172.21.15.76:0/4008122016 learned_addr learned my addr 172.21.15.76:0/4008122016 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:54:05.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.875+0000 7f80f37fe640 1 --2- 172.21.15.76:0/4008122016 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80f4159e50 0x7f80f4070840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:05.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.875+0000 7f80f2ffd640 1 --2- 172.21.15.76:0/4008122016 >> v2:172.21.15.113:3300/0 conn(0x7f80f406b7c0 0x7f80f40cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:05.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.875+0000 7f80f27fc640 1 -- 172.21.15.76:0/4008122016 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 190790770 0 0) 0x7f80f406c680 con 0x7f80f415d7f0 2023-11-09T03:54:05.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.875+0000 7f80f27fc640 1 -- 172.21.15.76:0/4008122016 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80dc003520 con 0x7f80f415d7f0 2023-11-09T03:54:05.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.876+0000 7f80f27fc640 1 -- 172.21.15.76:0/4008122016 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1921092527 0 0) 0x7f80dc003520 con 0x7f80f415d7f0 2023-11-09T03:54:05.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.876+0000 7f80f27fc640 1 -- 172.21.15.76:0/4008122016 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80f4159e50 msgr2=0x7f80f4070840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:05.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.876+0000 7f80f27fc640 1 --2- 172.21.15.76:0/4008122016 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80f4159e50 0x7f80f4070840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:05.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.876+0000 7f80f27fc640 1 -- 172.21.15.76:0/4008122016 >> v2:172.21.15.113:3300/0 conn(0x7f80f406b7c0 msgr2=0x7f80f40cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:05.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.876+0000 7f80f27fc640 1 --2- 172.21.15.76:0/4008122016 >> v2:172.21.15.113:3300/0 conn(0x7f80f406b7c0 0x7f80f40cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:05.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.876+0000 7f80f27fc640 1 -- 172.21.15.76:0/4008122016 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f4070e20 con 0x7f80f415d7f0 2023-11-09T03:54:05.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.876+0000 7f80f27fc640 1 -- 172.21.15.76:0/4008122016 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f80ec0019f0 con 0x7f80f415d7f0 2023-11-09T03:54:05.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.876+0000 7f80f2ffd640 1 --2- 172.21.15.76:0/4008122016 >> v2:172.21.15.113:3300/0 conn(0x7f80f406b7c0 0x7f80f40cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:54:05.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.876+0000 7f80f27fc640 1 -- 172.21.15.76:0/4008122016 >> v1:172.21.15.76:6789/0 conn(0x7f80f415d7f0 legacy=0x7f80f406cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:05.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.876+0000 7f80f27fc640 1 --2- 172.21.15.76:0/4008122016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80dc003970 0x7f80dc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:05.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.876+0000 7f80f27fc640 1 --2- 172.21.15.76:0/4008122016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80dc0047c0 0x7f80dc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:05.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.876+0000 7f80f27fc640 1 --2- 172.21.15.76:0/4008122016 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80f4159e50 0x7f80dc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:05.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.876+0000 7f80f27fc640 1 -- 172.21.15.76:0/4008122016 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f4071100 con 0x7f80dc003970 2023-11-09T03:54:05.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.876+0000 7f80f27fc640 1 -- 172.21.15.76:0/4008122016 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc00b980 con 0x7f80dc0047c0 2023-11-09T03:54:05.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.876+0000 7f80f27fc640 1 -- 172.21.15.76:0/4008122016 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc00bac0 con 0x7f80f4159e50 2023-11-09T03:54:05.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.876+0000 7f80f37fe640 1 --2- 172.21.15.76:0/4008122016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80dc003970 0x7f80dc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:05.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.876+0000 7f80f2ffd640 1 --2- 172.21.15.76:0/4008122016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80dc0047c0 0x7f80dc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:05.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.876+0000 7f80f3fff640 1 --2- 172.21.15.76:0/4008122016 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80f4159e50 0x7f80dc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:05.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.876+0000 7f80f37fe640 1 -- 172.21.15.76:0/4008122016 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80f4159e50 msgr2=0x7f80dc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:05.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.876+0000 7f80f37fe640 1 --2- 172.21.15.76:0/4008122016 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80f4159e50 0x7f80dc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:05.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.876+0000 7f80f37fe640 1 -- 172.21.15.76:0/4008122016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80dc0047c0 msgr2=0x7f80dc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:05.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.876+0000 7f80f37fe640 1 --2- 172.21.15.76:0/4008122016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80dc0047c0 0x7f80dc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:05.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.876+0000 7f80f37fe640 1 -- 172.21.15.76:0/4008122016 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80dc003520 con 0x7f80dc003970 2023-11-09T03:54:05.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.876+0000 7f80f3fff640 1 --2- 172.21.15.76:0/4008122016 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80f4159e50 0x7f80dc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:54:05.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.877+0000 7f80f37fe640 1 --2- 172.21.15.76:0/4008122016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80dc003970 0x7f80dc003d90 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009ac0 tx=0x7f80dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3879cef4a81def0d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:05.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.877+0000 7f80f27fc640 1 -- 172.21.15.76:0/4008122016 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e402c6b0 con 0x7f80dc003970 2023-11-09T03:54:05.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.877+0000 7f80f27fc640 1 -- 172.21.15.76:0/4008122016 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e402c810 con 0x7f80dc003970 2023-11-09T03:54:05.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.877+0000 7f80f27fc640 1 -- 172.21.15.76:0/4008122016 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e402cba0 con 0x7f80dc003970 2023-11-09T03:54:05.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.877+0000 7f80f8f9e640 1 -- 172.21.15.76:0/4008122016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80dc003970 msgr2=0x7f80dc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:05.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.877+0000 7f80f8f9e640 1 --2- 172.21.15.76:0/4008122016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80dc003970 0x7f80dc003d90 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009ac0 tx=0x7f80dc0057f0 comp rx=0 tx=0).stop 2023-11-09T03:54:05.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.877+0000 7f80f3fff640 1 -- 172.21.15.76:0/4008122016 reap_dead start 2023-11-09T03:54:05.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.878+0000 7f80f8f9e640 1 -- 172.21.15.76:0/4008122016 shutdown_connections 2023-11-09T03:54:05.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.878+0000 7f80f8f9e640 1 -- 172.21.15.76:0/4008122016 >> 172.21.15.76:0/4008122016 conn(0x7f80f406a730 msgr2=0x7f80f40ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:05.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.878+0000 7f80f8f9e640 1 -- 172.21.15.76:0/4008122016 shutdown_connections 2023-11-09T03:54:05.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.878+0000 7f80f8f9e640 1 -- 172.21.15.76:0/4008122016 wait complete. 2023-11-09T03:54:05.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.879+0000 7f80f8f9e640 1 Processor -- start 2023-11-09T03:54:05.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.879+0000 7f80f8f9e640 1 -- start start 2023-11-09T03:54:05.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.880+0000 7f80f8f9e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80dc003970 0x7f80f40c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:05.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.880+0000 7f80f8f9e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80f41f1250 0x7f80f41f3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:05.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.880+0000 7f80f8f9e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80f41f4300 0x7f80f40c4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:05.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.880+0000 7f80f8f9e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f40c7bc0 con 0x7f80f41f4300 2023-11-09T03:54:05.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.880+0000 7f80f8f9e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f414e9a0 con 0x7f80f41f1250 2023-11-09T03:54:05.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.880+0000 7f80f8f9e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f41f0dc0 con 0x7f80dc003970 2023-11-09T03:54:05.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.880+0000 7f80f3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80f41f4300 0x7f80f40c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:05.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.880+0000 7f80f3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80f41f4300 0x7f80f40c4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40570/0 (socket says 172.21.15.76:40570) 2023-11-09T03:54:05.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.880+0000 7f80f3fff640 1 -- 172.21.15.76:0/2067181274 learned_addr learned my addr 172.21.15.76:0/2067181274 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:05.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.880+0000 7f80f37fe640 1 --2- 172.21.15.76:0/2067181274 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80dc003970 0x7f80f40c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:05.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.880+0000 7f80f2ffd640 1 --2- 172.21.15.76:0/2067181274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80f41f1250 0x7f80f41f3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:05.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.880+0000 7f80f3fff640 1 -- 172.21.15.76:0/2067181274 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80dc003970 msgr2=0x7f80f40c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:05.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.880+0000 7f80f3fff640 1 --2- 172.21.15.76:0/2067181274 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80dc003970 0x7f80f40c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:05.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.880+0000 7f80f3fff640 1 -- 172.21.15.76:0/2067181274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80f41f1250 msgr2=0x7f80f41f3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:05.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.880+0000 7f80f3fff640 1 --2- 172.21.15.76:0/2067181274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80f41f1250 0x7f80f41f3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:05.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.880+0000 7f80f3fff640 1 -- 172.21.15.76:0/2067181274 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f41f07d0 con 0x7f80f41f4300 2023-11-09T03:54:05.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.880+0000 7f80f2ffd640 1 --2- 172.21.15.76:0/2067181274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80f41f1250 0x7f80f41f3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:54:05.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.880+0000 7f80f3fff640 1 --2- 172.21.15.76:0/2067181274 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80f41f4300 0x7f80f40c4540 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f80f415dbd0 tx=0x7f80ec00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:05.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.880+0000 7f80dbfff640 1 -- 172.21.15.76:0/2067181274 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ec00eb90 con 0x7f80f41f4300 2023-11-09T03:54:05.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.881+0000 7f80dbfff640 1 -- 172.21.15.76:0/2067181274 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80ec00ecf0 con 0x7f80f41f4300 2023-11-09T03:54:05.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.881+0000 7f80f8f9e640 1 -- 172.21.15.76:0/2067181274 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f41f4880 con 0x7f80f41f4300 2023-11-09T03:54:05.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.881+0000 7f80dbfff640 1 -- 172.21.15.76:0/2067181274 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ec013660 con 0x7f80f41f4300 2023-11-09T03:54:05.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.881+0000 7f80f8f9e640 1 -- 172.21.15.76:0/2067181274 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f41f0a50 con 0x7f80f41f4300 2023-11-09T03:54:05.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.881+0000 7f80f8f9e640 1 -- 172.21.15.76:0/2067181274 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80f4067c70 con 0x7f80f41f4300 2023-11-09T03:54:05.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.882+0000 7f80dbfff640 1 -- 172.21.15.76:0/2067181274 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f80ec0137c0 con 0x7f80f41f4300 2023-11-09T03:54:05.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.882+0000 7f80dbfff640 1 --2- 172.21.15.76:0/2067181274 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f80e006ae70 0x7f80e006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:05.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.882+0000 7f80dbfff640 1 -- 172.21.15.76:0/2067181274 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8195+0+0 (secure 0 0 0) 0x7f80ec08a520 con 0x7f80f41f4300 2023-11-09T03:54:05.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.883+0000 7f80f37fe640 1 --2- 172.21.15.76:0/2067181274 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f80e006ae70 0x7f80e006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:05.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.883+0000 7f80f37fe640 1 --2- 172.21.15.76:0/2067181274 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f80e006ae70 0x7f80e006d330 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f80e4036000 tx=0x7f80e4030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:05.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:05.887+0000 7f80dbfff640 1 -- 172.21.15.76:0/2067181274 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f80ec054b30 con 0x7f80f41f4300 2023-11-09T03:54:06.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.088+0000 7f80f8f9e640 1 -- 172.21.15.76:0/2067181274 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80f4067c70 con 0x7f80f41f4300 2023-11-09T03:54:06.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.088+0000 7f80dbfff640 1 -- 172.21.15.76:0/2067181274 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+4985 (secure 0 0 0) 0x7f80ec058150 con 0x7f80f41f4300 2023-11-09T03:54:06.088 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:54:06.088 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":119,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:01.482328+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":8772},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8772":{"gid":8772,"name":"a","rank":1,"incarnation":113,"state":"up:active","state_seq":7,"addr":"172.21.15.76:6835/2075292158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2075292158},{"type":"v1","addr":"172.21.15.76:6835","nonce":2075292158}]},"join_fscid":-1,"laggy_since":"2023-11-09T03:54:01.482328+0000","export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:54:06.090 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 119 2023-11-09T03:54:06.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.090+0000 7f80f8f9e640 1 -- 172.21.15.76:0/2067181274 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f80e006ae70 msgr2=0x7f80e006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:06.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.090+0000 7f80f8f9e640 1 --2- 172.21.15.76:0/2067181274 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f80e006ae70 0x7f80e006d330 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f80e4036000 tx=0x7f80e4030000 comp rx=0 tx=0).stop 2023-11-09T03:54:06.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.090+0000 7f80f8f9e640 1 -- 172.21.15.76:0/2067181274 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80f41f4300 msgr2=0x7f80f40c4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:06.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.090+0000 7f80f8f9e640 1 --2- 172.21.15.76:0/2067181274 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80f41f4300 0x7f80f40c4540 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f80f415dbd0 tx=0x7f80ec00e530 comp rx=0 tx=0).stop 2023-11-09T03:54:06.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.090+0000 7f80f8f9e640 1 -- 172.21.15.76:0/2067181274 shutdown_connections 2023-11-09T03:54:06.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.090+0000 7f80f8f9e640 1 --2- 172.21.15.76:0/2067181274 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f80e006ae70 0x7f80e006d330 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.090+0000 7f80f8f9e640 1 --2- 172.21.15.76:0/2067181274 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80f41f4300 0x7f80f40c4540 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.090+0000 7f80f8f9e640 1 --2- 172.21.15.76:0/2067181274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80f41f1250 0x7f80f41f3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.090+0000 7f80f8f9e640 1 --2- 172.21.15.76:0/2067181274 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80dc003970 0x7f80f40c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.090+0000 7f80f8f9e640 1 -- 172.21.15.76:0/2067181274 >> 172.21.15.76:0/2067181274 conn(0x7f80f406a730 msgr2=0x7f80f40c8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:06.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.090+0000 7f80f8f9e640 1 -- 172.21.15.76:0/2067181274 shutdown_connections 2023-11-09T03:54:06.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.090+0000 7f80f8f9e640 1 -- 172.21.15.76:0/2067181274 wait complete. 2023-11-09T03:54:06.101 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2023-11-09T03:54:06.215 INFO:teuthology.orchestra.run.smithi076.stdout:{"last_snap":15,"last_created":12,"last_destroyed":14,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[{"version":30,"snapinfo":{"snapid":15,"ino":1099511627777,"stamp":"2023-11-09T03:53:43.660126+0000","name":"s33","metadata":{}}}],"pending_destroy":[]} 2023-11-09T03:54:06.215 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 12, "last_destroyed": 14, "last_snap": 15, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [ { "snapInfinityo": { "ino": 1099511627777, "metadata": {}, "name": "s33", "snapid": 15, "stamp": "2023-11-09T03:53:43.660126+0000" }, "version": 30 } ], "snaps": [] } 2023-11-09T03:54:06.216 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:1 2023-11-09T03:54:06.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.317+0000 7f3f7015a640 1 Processor -- start 2023-11-09T03:54:06.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.317+0000 7f3f7015a640 1 -- start start 2023-11-09T03:54:06.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.318+0000 7f3f7015a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3f680ade00 0x7f3f680b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:06.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.318+0000 7f3f7015a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3f6806d9c0 0x7f3f680710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:06.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.318+0000 7f3f7015a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f6806c5f0 0x7f3f68071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:06.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.318+0000 7f3f7015a640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f6806cc90 con 0x7f3f680ade00 2023-11-09T03:54:06.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.318+0000 7f3f7015a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f68071b50 con 0x7f3f6806d9c0 2023-11-09T03:54:06.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.318+0000 7f3f7015a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f680b0e60 con 0x7f3f6806c5f0 2023-11-09T03:54:06.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.318+0000 7f3f6def6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3f680ade00 0x7f3f680b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:06.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.318+0000 7f3f6def6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3f680ade00 0x7f3f680b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40574/0 (socket says 172.21.15.76:40574) 2023-11-09T03:54:06.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.318+0000 7f3f6def6640 1 -- 172.21.15.76:0/512841484 learned_addr learned my addr 172.21.15.76:0/512841484 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:06.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.318+0000 7f3f6d6f5640 1 --2- 172.21.15.76:0/512841484 >> v2:172.21.15.113:3300/0 conn(0x7f3f6806d9c0 0x7f3f680710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:06.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.318+0000 7f3f6e6f7640 1 --2- 172.21.15.76:0/512841484 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f6806c5f0 0x7f3f68071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:06.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.318+0000 7f3f6def6640 1 -- 172.21.15.76:0/512841484 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f6806c5f0 msgr2=0x7f3f68071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:06.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.318+0000 7f3f6def6640 1 --2- 172.21.15.76:0/512841484 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f6806c5f0 0x7f3f68071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.318+0000 7f3f6def6640 1 -- 172.21.15.76:0/512841484 >> v2:172.21.15.113:3300/0 conn(0x7f3f6806d9c0 msgr2=0x7f3f680710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:06.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.318+0000 7f3f6def6640 1 --2- 172.21.15.76:0/512841484 >> v2:172.21.15.113:3300/0 conn(0x7f3f6806d9c0 0x7f3f680710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.318+0000 7f3f6def6640 1 -- 172.21.15.76:0/512841484 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6806cee0 con 0x7f3f680ade00 2023-11-09T03:54:06.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.319+0000 7f3f6def6640 1 --2- 172.21.15.76:0/512841484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f680ade00 0x7f3f680b0530 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f3f680c9080 tx=0x7f3f6402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7430e07a99015256 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:06.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.319+0000 7f3f6cef4640 1 -- 172.21.15.76:0/512841484 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f64031750 con 0x7f3f680ade00 2023-11-09T03:54:06.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.319+0000 7f3f6cef4640 1 -- 172.21.15.76:0/512841484 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f640318b0 con 0x7f3f680ade00 2023-11-09T03:54:06.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.319+0000 7f3f6cef4640 1 -- 172.21.15.76:0/512841484 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f64031c40 con 0x7f3f680ade00 2023-11-09T03:54:06.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.319+0000 7f3f7015a640 1 -- 172.21.15.76:0/512841484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f680ade00 msgr2=0x7f3f680b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:06.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.319+0000 7f3f7015a640 1 --2- 172.21.15.76:0/512841484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f680ade00 0x7f3f680b0530 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f3f680c9080 tx=0x7f3f6402b7f0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.320+0000 7f3f7015a640 1 -- 172.21.15.76:0/512841484 shutdown_connections 2023-11-09T03:54:06.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.320+0000 7f3f7015a640 1 --2- 172.21.15.76:0/512841484 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f6806c5f0 0x7f3f68071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.320+0000 7f3f7015a640 1 --2- 172.21.15.76:0/512841484 >> v2:172.21.15.113:3300/0 conn(0x7f3f6806d9c0 0x7f3f680710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.320+0000 7f3f7015a640 1 --2- 172.21.15.76:0/512841484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f680ade00 0x7f3f680b0530 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.320+0000 7f3f7015a640 1 -- 172.21.15.76:0/512841484 >> 172.21.15.76:0/512841484 conn(0x7f3f680cb530 msgr2=0x7f3f680cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:06.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.320+0000 7f3f7015a640 1 -- 172.21.15.76:0/512841484 shutdown_connections 2023-11-09T03:54:06.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.320+0000 7f3f7015a640 1 -- 172.21.15.76:0/512841484 wait complete. 2023-11-09T03:54:06.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.321+0000 7f3f7015a640 1 Processor -- start 2023-11-09T03:54:06.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.321+0000 7f3f7015a640 1 -- start start 2023-11-09T03:54:06.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.321+0000 7f3f7015a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f6806c5f0 0x7f3f680b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:06.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.321+0000 7f3f7015a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f6806d9c0 0x7f3f680cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:06.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.321+0000 7f3f7015a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f680ade00 0x7f3f680acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:06.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.321+0000 7f3f7015a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f680b25f0 con 0x7f3f6806c5f0 2023-11-09T03:54:06.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.321+0000 7f3f7015a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f680cb120 con 0x7f3f6806d9c0 2023-11-09T03:54:06.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.321+0000 7f3f7015a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f680b19e0 con 0x7f3f680ade00 2023-11-09T03:54:06.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.322+0000 7f3f6def6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f6806c5f0 0x7f3f680b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:06.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.322+0000 7f3f6def6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f6806c5f0 0x7f3f680b2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40582/0 (socket says 172.21.15.76:40582) 2023-11-09T03:54:06.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.322+0000 7f3f6def6640 1 -- 172.21.15.76:0/2282207051 learned_addr learned my addr 172.21.15.76:0/2282207051 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:06.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.322+0000 7f3f6e6f7640 1 --2- 172.21.15.76:0/2282207051 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f680ade00 0x7f3f680acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:06.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.322+0000 7f3f6d6f5640 1 --2- 172.21.15.76:0/2282207051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f6806d9c0 0x7f3f680cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:06.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.322+0000 7f3f6def6640 1 -- 172.21.15.76:0/2282207051 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f680ade00 msgr2=0x7f3f680acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:06.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.322+0000 7f3f6def6640 1 --2- 172.21.15.76:0/2282207051 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f680ade00 0x7f3f680acb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.322+0000 7f3f6def6640 1 -- 172.21.15.76:0/2282207051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f6806d9c0 msgr2=0x7f3f680cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:06.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.322+0000 7f3f6def6640 1 --2- 172.21.15.76:0/2282207051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f6806d9c0 0x7f3f680cabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.322+0000 7f3f6def6640 1 -- 172.21.15.76:0/2282207051 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f640094d0 con 0x7f3f6806c5f0 2023-11-09T03:54:06.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.322+0000 7f3f6def6640 1 --2- 172.21.15.76:0/2282207051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f6806c5f0 0x7f3f680b2a80 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f3f64000c00 tx=0x7f3f640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:06.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.322+0000 7f3f5e7fc640 1 -- 172.21.15.76:0/2282207051 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f64033c80 con 0x7f3f6806c5f0 2023-11-09T03:54:06.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.322+0000 7f3f5e7fc640 1 -- 172.21.15.76:0/2282207051 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f64033e00 con 0x7f3f6806c5f0 2023-11-09T03:54:06.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.322+0000 7f3f7015a640 1 -- 172.21.15.76:0/2282207051 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f680b1cc0 con 0x7f3f6806c5f0 2023-11-09T03:54:06.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.322+0000 7f3f5e7fc640 1 -- 172.21.15.76:0/2282207051 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6403c640 con 0x7f3f6806c5f0 2023-11-09T03:54:06.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.322+0000 7f3f7015a640 1 -- 172.21.15.76:0/2282207051 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f680b2280 con 0x7f3f6806c5f0 2023-11-09T03:54:06.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.323+0000 7f3f7015a640 1 -- 172.21.15.76:0/2282207051 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f680b2410 con 0x7f3f6806c5f0 2023-11-09T03:54:06.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.324+0000 7f3f5e7fc640 1 -- 172.21.15.76:0/2282207051 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3f6403a040 con 0x7f3f6806c5f0 2023-11-09T03:54:06.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.324+0000 7f3f5e7fc640 1 --2- 172.21.15.76:0/2282207051 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3f4c06ae70 0x7f3f4c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:06.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.324+0000 7f3f5e7fc640 1 -- 172.21.15.76:0/2282207051 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8195+0+0 (secure 0 0 0) 0x7f3f640aba50 con 0x7f3f6806c5f0 2023-11-09T03:54:06.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.324+0000 7f3f6d6f5640 1 --2- 172.21.15.76:0/2282207051 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3f4c06ae70 0x7f3f4c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:06.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.325+0000 7f3f6d6f5640 1 --2- 172.21.15.76:0/2282207051 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3f4c06ae70 0x7f3f4c06d330 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f3f6815d970 tx=0x7f3f58008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:06.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.328+0000 7f3f5e7fc640 1 -- 172.21.15.76:0/2282207051 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3f64076350 con 0x7f3f6806c5f0 2023-11-09T03:54:06.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.567+0000 7f3f7015a640 1 -- 172.21.15.76:0/2282207051 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:1"} v 0) v1 -- 0x7f3f680684b0 con 0x7f3f6806c5f0 2023-11-09T03:54:06.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.571+0000 7f3f5e7fc640 1 -- 172.21.15.76:0/2282207051 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:1"}]=0 failed mds gid 8772 v120) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3f64079970 con 0x7f3f6806c5f0 2023-11-09T03:54:06.570 INFO:teuthology.orchestra.run.smithi076.stderr:failed mds gid 8772 2023-11-09T03:54:06.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.572+0000 7f3f7015a640 1 -- 172.21.15.76:0/2282207051 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3f4c06ae70 msgr2=0x7f3f4c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:06.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.573+0000 7f3f7015a640 1 --2- 172.21.15.76:0/2282207051 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3f4c06ae70 0x7f3f4c06d330 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f3f6815d970 tx=0x7f3f58008040 comp rx=0 tx=0).stop 2023-11-09T03:54:06.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.573+0000 7f3f7015a640 1 -- 172.21.15.76:0/2282207051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f6806c5f0 msgr2=0x7f3f680b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:06.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.573+0000 7f3f7015a640 1 --2- 172.21.15.76:0/2282207051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f6806c5f0 0x7f3f680b2a80 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f3f64000c00 tx=0x7f3f640046c0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.573+0000 7f3f7015a640 1 -- 172.21.15.76:0/2282207051 shutdown_connections 2023-11-09T03:54:06.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.573+0000 7f3f7015a640 1 --2- 172.21.15.76:0/2282207051 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3f4c06ae70 0x7f3f4c06d330 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.573+0000 7f3f7015a640 1 --2- 172.21.15.76:0/2282207051 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f680ade00 0x7f3f680acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.573+0000 7f3f7015a640 1 --2- 172.21.15.76:0/2282207051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f6806d9c0 0x7f3f680cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.573+0000 7f3f7015a640 1 --2- 172.21.15.76:0/2282207051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f6806c5f0 0x7f3f680b2a80 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.573+0000 7f3f7015a640 1 -- 172.21.15.76:0/2282207051 >> 172.21.15.76:0/2282207051 conn(0x7f3f680cb530 msgr2=0x7f3f680ae720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:06.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.573+0000 7f3f7015a640 1 -- 172.21.15.76:0/2282207051 shutdown_connections 2023-11-09T03:54:06.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.573+0000 7f3f7015a640 1 -- 172.21.15.76:0/2282207051 wait complete. 2023-11-09T03:54:06.586 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-09T03:54:06.587 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-09T03:54:06.587 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-09T03:54:06.588 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:54:06.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:54:06.588 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi076 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2023-11-09T03:54:06.589 INFO:tasks.ceph.mds.a:Stopped 2023-11-09T03:54:06.589 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-09T03:54:06.592 INFO:tasks.ceph.mds.a:Started 2023-11-09T03:54:06.592 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:54:06.666 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2023-11-09T03:54:06.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.694+0000 7ff67b8e7640 1 Processor -- start 2023-11-09T03:54:06.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.694+0000 7ff67b8e7640 1 -- start start 2023-11-09T03:54:06.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.695+0000 7ff67b8e7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff67415d4e0 0x7ff6740d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:06.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.695+0000 7ff67b8e7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6740b32b0 con 0x7ff6740d3b60 2023-11-09T03:54:06.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.695+0000 7ff67b8e7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6740cee10 con 0x7ff67415d100 2023-11-09T03:54:06.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.695+0000 7ff67b8e7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6740cec20 con 0x7ff67415d4e0 2023-11-09T03:54:06.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.695+0000 7ff679e84640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff6740d3b60 0x7ff6740b3740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:55750/0 (socket says 172.21.15.76:55750) 2023-11-09T03:54:06.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.695+0000 7ff679e84640 1 -- 172.21.15.76:0/1981744158 learned_addr learned my addr 172.21.15.76:0/1981744158 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:54:06.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.695+0000 7ff66bfff640 1 -- 172.21.15.76:0/1981744158 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2251838120 0 0) 0x7ff6740b32b0 con 0x7ff6740d3b60 2023-11-09T03:54:06.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.695+0000 7ff66bfff640 1 -- 172.21.15.76:0/1981744158 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff660003520 con 0x7ff6740d3b60 2023-11-09T03:54:06.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.695+0000 7ff678e82640 1 --2- 172.21.15.76:0/1981744158 >> v2:172.21.15.113:3300/0 conn(0x7ff67415d4e0 0x7ff6740d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:06.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.695+0000 7ff66bfff640 1 -- 172.21.15.76:0/1981744158 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4185560970 0 0) 0x7ff660003520 con 0x7ff6740d3b60 2023-11-09T03:54:06.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.695+0000 7ff66bfff640 1 -- 172.21.15.76:0/1981744158 >> v2:172.21.15.113:3300/0 conn(0x7ff67415d4e0 msgr2=0x7ff6740d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:06.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.695+0000 7ff66bfff640 1 --2- 172.21.15.76:0/1981744158 >> v2:172.21.15.113:3300/0 conn(0x7ff67415d4e0 0x7ff6740d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.695+0000 7ff66bfff640 1 -- 172.21.15.76:0/1981744158 >> v1:172.21.15.113:6789/0 conn(0x7ff67415d100 legacy=0x7ff6740ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:06.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.695+0000 7ff66bfff640 1 -- 172.21.15.76:0/1981744158 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6740b2980 con 0x7ff6740d3b60 2023-11-09T03:54:06.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.695+0000 7ff66bfff640 1 -- 172.21.15.76:0/1981744158 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7ff66c0019f0 con 0x7ff6740d3b60 2023-11-09T03:54:06.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.695+0000 7ff66bfff640 1 -- 172.21.15.76:0/1981744158 >> v1:172.21.15.76:6789/0 conn(0x7ff6740d3b60 legacy=0x7ff6740b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:06.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.695+0000 7ff66bfff640 1 --2- 172.21.15.76:0/1981744158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff660003970 0x7ff660003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:06.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.695+0000 7ff66bfff640 1 --2- 172.21.15.76:0/1981744158 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6600047c0 0x7ff660004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:06.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.695+0000 7ff66bfff640 1 --2- 172.21.15.76:0/1981744158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff660005180 0x7ff66000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:06.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.695+0000 7ff66bfff640 1 -- 172.21.15.76:0/1981744158 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66000c0a0 con 0x7ff660003970 2023-11-09T03:54:06.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.695+0000 7ff66bfff640 1 -- 172.21.15.76:0/1981744158 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66000c210 con 0x7ff660005180 2023-11-09T03:54:06.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.695+0000 7ff66bfff640 1 -- 172.21.15.76:0/1981744158 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66000c3d0 con 0x7ff6600047c0 2023-11-09T03:54:06.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.695+0000 7ff678e82640 1 --2- 172.21.15.76:0/1981744158 >> v2:172.21.15.113:3300/0 conn(0x7ff67415d4e0 0x7ff6740d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:54:06.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.696+0000 7ff679683640 1 --2- 172.21.15.76:0/1981744158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff660003970 0x7ff660003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:06.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.696+0000 7ff679e84640 1 --2- 172.21.15.76:0/1981744158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff660005180 0x7ff66000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:06.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.696+0000 7ff678e82640 1 --2- 172.21.15.76:0/1981744158 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6600047c0 0x7ff660004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:06.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.696+0000 7ff679683640 1 -- 172.21.15.76:0/1981744158 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6600047c0 msgr2=0x7ff660004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:06.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.696+0000 7ff679683640 1 --2- 172.21.15.76:0/1981744158 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6600047c0 0x7ff660004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.696+0000 7ff679683640 1 -- 172.21.15.76:0/1981744158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff660005180 msgr2=0x7ff66000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:06.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.696+0000 7ff679683640 1 --2- 172.21.15.76:0/1981744158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff660005180 0x7ff66000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.696+0000 7ff679683640 1 -- 172.21.15.76:0/1981744158 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff660003520 con 0x7ff660003970 2023-11-09T03:54:06.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.696+0000 7ff679e84640 1 -- 172.21.15.76:0/1981744158 reap_dead start 2023-11-09T03:54:06.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.696+0000 7ff678e82640 1 --2- 172.21.15.76:0/1981744158 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6600047c0 0x7ff660004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:54:06.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.697+0000 7ff679683640 1 --2- 172.21.15.76:0/1981744158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff660003970 0x7ff660003d90 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7ff670001770 tx=0x7ff6600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f235232f3461032a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:06.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.697+0000 7ff66bfff640 1 -- 172.21.15.76:0/1981744158 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6700052d0 con 0x7ff660003970 2023-11-09T03:54:06.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.697+0000 7ff66bfff640 1 -- 172.21.15.76:0/1981744158 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67002e050 con 0x7ff660003970 2023-11-09T03:54:06.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.697+0000 7ff66bfff640 1 -- 172.21.15.76:0/1981744158 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff670003640 con 0x7ff660003970 2023-11-09T03:54:06.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.697+0000 7ff67b8e7640 1 -- 172.21.15.76:0/1981744158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff660003970 msgr2=0x7ff660003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:06.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.697+0000 7ff67b8e7640 1 --2- 172.21.15.76:0/1981744158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff660003970 0x7ff660003d90 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7ff670001770 tx=0x7ff6600057f0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.697+0000 7ff67b8e7640 1 -- 172.21.15.76:0/1981744158 shutdown_connections 2023-11-09T03:54:06.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.697+0000 7ff67b8e7640 1 --2- 172.21.15.76:0/1981744158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff660003970 0x7ff660003d90 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.697+0000 7ff67b8e7640 1 -- 172.21.15.76:0/1981744158 >> 172.21.15.76:0/1981744158 conn(0x7ff6740d2ef0 msgr2=0x7ff6740d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:06.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.697+0000 7ff67b8e7640 1 -- 172.21.15.76:0/1981744158 shutdown_connections 2023-11-09T03:54:06.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.697+0000 7ff67b8e7640 1 -- 172.21.15.76:0/1981744158 wait complete. 2023-11-09T03:54:06.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.698+0000 7ff67b8e7640 1 Processor -- start 2023-11-09T03:54:06.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.698+0000 7ff67b8e7640 1 -- start start 2023-11-09T03:54:06.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.699+0000 7ff67b8e7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff660003970 0x7ff6740b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:06.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.699+0000 7ff67b8e7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6741f3da0 0x7ff6740cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:06.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.699+0000 7ff67b8e7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6740b7eb0 0x7ff6741f2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:06.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.699+0000 7ff67b8e7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6740b87e0 con 0x7ff6740b7eb0 2023-11-09T03:54:06.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.699+0000 7ff67b8e7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67415ddc0 con 0x7ff6741f3da0 2023-11-09T03:54:06.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.699+0000 7ff67b8e7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67415e0a0 con 0x7ff660003970 2023-11-09T03:54:06.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.699+0000 7ff679e84640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6740b7eb0 0x7ff6741f2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:06.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.699+0000 7ff679e84640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6740b7eb0 0x7ff6741f2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40636/0 (socket says 172.21.15.76:40636) 2023-11-09T03:54:06.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.699+0000 7ff679e84640 1 -- 172.21.15.76:0/1258414100 learned_addr learned my addr 172.21.15.76:0/1258414100 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:06.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.699+0000 7ff678e82640 1 --2- 172.21.15.76:0/1258414100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6741f3da0 0x7ff6740cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:06.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.699+0000 7ff679683640 1 --2- 172.21.15.76:0/1258414100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff660003970 0x7ff6740b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:06.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.700+0000 7ff679e84640 1 -- 172.21.15.76:0/1258414100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff660003970 msgr2=0x7ff6740b5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:06.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.700+0000 7ff679e84640 1 --2- 172.21.15.76:0/1258414100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff660003970 0x7ff6740b5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.700+0000 7ff679e84640 1 -- 172.21.15.76:0/1258414100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6741f3da0 msgr2=0x7ff6740cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:06.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.700+0000 7ff679e84640 1 --2- 172.21.15.76:0/1258414100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6741f3da0 0x7ff6740cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.700+0000 7ff679e84640 1 -- 172.21.15.76:0/1258414100 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67415e380 con 0x7ff6740b7eb0 2023-11-09T03:54:06.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.700+0000 7ff679683640 1 --2- 172.21.15.76:0/1258414100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff660003970 0x7ff6740b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:54:06.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.700+0000 7ff679e84640 1 --2- 172.21.15.76:0/1258414100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6740b7eb0 0x7ff6741f2c20 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7ff6740cefa0 tx=0x7ff66c007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:06.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.700+0000 7ff669ffb640 1 -- 172.21.15.76:0/1258414100 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c00aaf0 con 0x7ff6740b7eb0 2023-11-09T03:54:06.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.700+0000 7ff669ffb640 1 -- 172.21.15.76:0/1258414100 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66c00ac50 con 0x7ff6740b7eb0 2023-11-09T03:54:06.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.700+0000 7ff669ffb640 1 -- 172.21.15.76:0/1258414100 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c0133f0 con 0x7ff6740b7eb0 2023-11-09T03:54:06.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.700+0000 7ff67b8e7640 1 -- 172.21.15.76:0/1258414100 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6740ccd80 con 0x7ff6740b7eb0 2023-11-09T03:54:06.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.700+0000 7ff67b8e7640 1 -- 172.21.15.76:0/1258414100 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6740b8540 con 0x7ff6740b7eb0 2023-11-09T03:54:06.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.701+0000 7ff67b8e7640 1 -- 172.21.15.76:0/1258414100 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff67415dc20 con 0x7ff6740b7eb0 2023-11-09T03:54:06.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.701+0000 7ff669ffb640 1 -- 172.21.15.76:0/1258414100 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff66c013550 con 0x7ff6740b7eb0 2023-11-09T03:54:06.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.702+0000 7ff669ffb640 1 --2- 172.21.15.76:0/1258414100 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff65806aea0 0x7ff65806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:06.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.702+0000 7ff669ffb640 1 -- 172.21.15.76:0/1258414100 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8253+0+0 (secure 0 0 0) 0x7ff66c08a890 con 0x7ff6740b7eb0 2023-11-09T03:54:06.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.702+0000 7ff679683640 1 --2- 172.21.15.76:0/1258414100 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff65806aea0 0x7ff65806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:06.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.703+0000 7ff679683640 1 --2- 172.21.15.76:0/1258414100 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff65806aea0 0x7ff65806d360 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff670033000 tx=0x7ff670009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:06.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.706+0000 7ff669ffb640 1 -- 172.21.15.76:0/1258414100 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff67415dc20 con 0x7ff6740b7eb0 2023-11-09T03:54:06.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.905+0000 7ff67b8e7640 1 -- 172.21.15.76:0/1258414100 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff67415dc20 con 0x7ff6740b7eb0 2023-11-09T03:54:06.905 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.905+0000 7ff669ffb640 1 -- 172.21.15.76:0/1258414100 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+4145 (secure 0 0 0) 0x7ff67415dc20 con 0x7ff6740b7eb0 2023-11-09T03:54:06.905 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:54:06.906 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":121,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:06.576254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":6470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6470":{"gid":6470,"name":"b","rank":1,"incarnation":121,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:54:06.906 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 121 2023-11-09T03:54:06.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.907+0000 7ff67b8e7640 1 -- 172.21.15.76:0/1258414100 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff65806aea0 msgr2=0x7ff65806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:06.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.907+0000 7ff67b8e7640 1 --2- 172.21.15.76:0/1258414100 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff65806aea0 0x7ff65806d360 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff670033000 tx=0x7ff670009030 comp rx=0 tx=0).stop 2023-11-09T03:54:06.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.907+0000 7ff67b8e7640 1 -- 172.21.15.76:0/1258414100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6740b7eb0 msgr2=0x7ff6741f2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:06.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.907+0000 7ff67b8e7640 1 --2- 172.21.15.76:0/1258414100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6740b7eb0 0x7ff6741f2c20 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7ff6740cefa0 tx=0x7ff66c007bd0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.907+0000 7ff67b8e7640 1 -- 172.21.15.76:0/1258414100 shutdown_connections 2023-11-09T03:54:06.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.907+0000 7ff67b8e7640 1 --2- 172.21.15.76:0/1258414100 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff65806aea0 0x7ff65806d360 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.907+0000 7ff67b8e7640 1 --2- 172.21.15.76:0/1258414100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6740b7eb0 0x7ff6741f2c20 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.907+0000 7ff67b8e7640 1 --2- 172.21.15.76:0/1258414100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6741f3da0 0x7ff6740cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.907+0000 7ff67b8e7640 1 --2- 172.21.15.76:0/1258414100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff660003970 0x7ff6740b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:06.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.907+0000 7ff67b8e7640 1 -- 172.21.15.76:0/1258414100 >> 172.21.15.76:0/1258414100 conn(0x7ff6740d2ef0 msgr2=0x7ff6740b5170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:06.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.907+0000 7ff67b8e7640 1 -- 172.21.15.76:0/1258414100 shutdown_connections 2023-11-09T03:54:06.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:06.907+0000 7ff67b8e7640 1 -- 172.21.15.76:0/1258414100 wait complete. 2023-11-09T03:54:06.918 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-11-09T03:54:11.919 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:54:12.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.028+0000 7f192e210640 1 Processor -- start 2023-11-09T03:54:12.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.028+0000 7f192e210640 1 -- start start 2023-11-09T03:54:12.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.029+0000 7f192e210640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1928159e40 0x7f19280b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:12.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.029+0000 7f192e210640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1928070700 0x7f19280b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:12.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.029+0000 7f192e210640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1928071500 con 0x7f192806bb60 2023-11-09T03:54:12.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.029+0000 7f192e210640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f192815dae0 con 0x7f1928070700 2023-11-09T03:54:12.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.029+0000 7f192e210640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1928070c60 con 0x7f1928159e40 2023-11-09T03:54:12.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.029+0000 7f1927fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1928070700 0x7f19280b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:12.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.029+0000 7f1927fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1928070700 0x7f19280b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40650/0 (socket says 172.21.15.76:40650) 2023-11-09T03:54:12.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.029+0000 7f1927fff640 1 -- 172.21.15.76:0/1879282176 learned_addr learned my addr 172.21.15.76:0/1879282176 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:12.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.029+0000 7f1927fff640 1 -- 172.21.15.76:0/1879282176 >> v2:172.21.15.113:3300/0 conn(0x7f1928159e40 msgr2=0x7f19280b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:12.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.029+0000 7f1927fff640 1 --2- 172.21.15.76:0/1879282176 >> v2:172.21.15.113:3300/0 conn(0x7f1928159e40 0x7f19280b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:12.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.029+0000 7f1927fff640 1 -- 172.21.15.76:0/1879282176 >> v1:172.21.15.113:6789/0 conn(0x7f192806bb60 legacy=0x7f192806fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:12.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.029+0000 7f1927fff640 1 -- 172.21.15.76:0/1879282176 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1928070f40 con 0x7f1928070700 2023-11-09T03:54:12.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.030+0000 7f1927fff640 1 --2- 172.21.15.76:0/1879282176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1928070700 0x7f19280b0a70 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f19280ce7f0 tx=0x7f192002bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed33470bce4c600 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:12.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.030+0000 7f19267fc640 1 -- 172.21.15.76:0/1879282176 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192002c960 con 0x7f1928070700 2023-11-09T03:54:12.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.030+0000 7f19267fc640 1 -- 172.21.15.76:0/1879282176 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192002cac0 con 0x7f1928070700 2023-11-09T03:54:12.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.030+0000 7f19267fc640 1 -- 172.21.15.76:0/1879282176 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192002ce50 con 0x7f1928070700 2023-11-09T03:54:12.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.031+0000 7f192e210640 1 -- 172.21.15.76:0/1879282176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1928070700 msgr2=0x7f19280b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:12.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.031+0000 7f192e210640 1 --2- 172.21.15.76:0/1879282176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1928070700 0x7f19280b0a70 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f19280ce7f0 tx=0x7f192002bdc0 comp rx=0 tx=0).stop 2023-11-09T03:54:12.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.031+0000 7f192e210640 1 -- 172.21.15.76:0/1879282176 shutdown_connections 2023-11-09T03:54:12.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.031+0000 7f192e210640 1 --2- 172.21.15.76:0/1879282176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1928070700 0x7f19280b0a70 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:12.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.031+0000 7f192e210640 1 --2- 172.21.15.76:0/1879282176 >> v2:172.21.15.113:3300/0 conn(0x7f1928159e40 0x7f19280b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:12.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.031+0000 7f192e210640 1 -- 172.21.15.76:0/1879282176 >> 172.21.15.76:0/1879282176 conn(0x7f19280cb530 msgr2=0x7f19280cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:12.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.031+0000 7f192e210640 1 -- 172.21.15.76:0/1879282176 shutdown_connections 2023-11-09T03:54:12.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.031+0000 7f192e210640 1 -- 172.21.15.76:0/1879282176 wait complete. 2023-11-09T03:54:12.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.032+0000 7f192e210640 1 Processor -- start 2023-11-09T03:54:12.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.032+0000 7f192e210640 1 -- start start 2023-11-09T03:54:12.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.033+0000 7f192e210640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f192806bb60 0x7f19280cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:12.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.033+0000 7f192e210640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1928070700 0x7f19281562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:12.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.033+0000 7f192e210640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1928159e40 0x7f19280aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:12.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.033+0000 7f192e210640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19280ccbc0 con 0x7f192806bb60 2023-11-09T03:54:12.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.033+0000 7f192e210640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19280c5230 con 0x7f1928070700 2023-11-09T03:54:12.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.033+0000 7f192e210640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1928157360 con 0x7f1928159e40 2023-11-09T03:54:12.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.033+0000 7f19277fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f192806bb60 0x7f19280cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:12.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.033+0000 7f19277fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f192806bb60 0x7f19280cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40652/0 (socket says 172.21.15.76:40652) 2023-11-09T03:54:12.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.033+0000 7f19277fe640 1 -- 172.21.15.76:0/3249977346 learned_addr learned my addr 172.21.15.76:0/3249977346 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:12.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.033+0000 7f1926ffd640 1 --2- 172.21.15.76:0/3249977346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1928070700 0x7f19281562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:12.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.033+0000 7f19277fe640 1 -- 172.21.15.76:0/3249977346 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1928159e40 msgr2=0x7f19280aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:12.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.033+0000 7f19277fe640 1 --2- 172.21.15.76:0/3249977346 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1928159e40 0x7f19280aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:12.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.033+0000 7f19277fe640 1 -- 172.21.15.76:0/3249977346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1928070700 msgr2=0x7f19281562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:12.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.033+0000 7f19277fe640 1 --2- 172.21.15.76:0/3249977346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1928070700 0x7f19281562e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:12.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.033+0000 7f19277fe640 1 -- 172.21.15.76:0/3249977346 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1920009b30 con 0x7f192806bb60 2023-11-09T03:54:12.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.034+0000 7f19277fe640 1 --2- 172.21.15.76:0/3249977346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f192806bb60 0x7f19280cc680 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f191800b440 tx=0x7f191800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:12.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.034+0000 7f190ffff640 1 -- 172.21.15.76:0/3249977346 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1918011890 con 0x7f192806bb60 2023-11-09T03:54:12.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.034+0000 7f190ffff640 1 -- 172.21.15.76:0/3249977346 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19180119f0 con 0x7f192806bb60 2023-11-09T03:54:12.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.034+0000 7f192e210640 1 -- 172.21.15.76:0/3249977346 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19280c4620 con 0x7f192806bb60 2023-11-09T03:54:12.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.034+0000 7f190ffff640 1 -- 172.21.15.76:0/3249977346 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1918011da0 con 0x7f192806bb60 2023-11-09T03:54:12.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.034+0000 7f192e210640 1 -- 172.21.15.76:0/3249977346 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19280c4be0 con 0x7f192806bb60 2023-11-09T03:54:12.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.035+0000 7f192e210640 1 -- 172.21.15.76:0/3249977346 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19280684b0 con 0x7f192806bb60 2023-11-09T03:54:12.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.035+0000 7f190ffff640 1 -- 172.21.15.76:0/3249977346 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f191801a400 con 0x7f192806bb60 2023-11-09T03:54:12.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.035+0000 7f190ffff640 1 --2- 172.21.15.76:0/3249977346 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f191406aea0 0x7f191406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:12.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.036+0000 7f190ffff640 1 -- 172.21.15.76:0/3249977346 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8253+0+0 (secure 0 0 0) 0x7f191808a380 con 0x7f192806bb60 2023-11-09T03:54:12.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.036+0000 7f1926ffd640 1 --2- 172.21.15.76:0/3249977346 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f191406aea0 0x7f191406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:12.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.036+0000 7f1926ffd640 1 --2- 172.21.15.76:0/3249977346 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f191406aea0 0x7f191406d360 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f19280adaa0 tx=0x7f191c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:12.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.040+0000 7f190ffff640 1 -- 172.21.15.76:0/3249977346 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f19180549f0 con 0x7f192806bb60 2023-11-09T03:54:12.061 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-11-09T03:54:12.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.243+0000 7f192e210640 1 -- 172.21.15.76:0/3249977346 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19280c9750 con 0x7f192806bb60 2023-11-09T03:54:12.262 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.244+0000 7f190ffff640 1 -- 172.21.15.76:0/3249977346 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+4938 (secure 0 0 0) 0x7f1918015020 con 0x7f192806bb60 2023-11-09T03:54:12.263 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:54:12.263 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":123,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":9024,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:07.587497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":6470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6470":{"gid":6470,"name":"b","rank":1,"incarnation":121,"state":"up:reconnect","state_seq":13,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:54:12.264 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 123 2023-11-09T03:54:12.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.246+0000 7f192e210640 1 -- 172.21.15.76:0/3249977346 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f191406aea0 msgr2=0x7f191406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:12.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.246+0000 7f192e210640 1 --2- 172.21.15.76:0/3249977346 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f191406aea0 0x7f191406d360 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f19280adaa0 tx=0x7f191c009040 comp rx=0 tx=0).stop 2023-11-09T03:54:12.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.246+0000 7f192e210640 1 -- 172.21.15.76:0/3249977346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f192806bb60 msgr2=0x7f19280cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:12.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.246+0000 7f192e210640 1 --2- 172.21.15.76:0/3249977346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f192806bb60 0x7f19280cc680 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f191800b440 tx=0x7f191800b900 comp rx=0 tx=0).stop 2023-11-09T03:54:12.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.246+0000 7f192e210640 1 -- 172.21.15.76:0/3249977346 shutdown_connections 2023-11-09T03:54:12.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.246+0000 7f192e210640 1 --2- 172.21.15.76:0/3249977346 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f191406aea0 0x7f191406d360 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:12.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.246+0000 7f192e210640 1 --2- 172.21.15.76:0/3249977346 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1928159e40 0x7f19280aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:12.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.246+0000 7f192e210640 1 --2- 172.21.15.76:0/3249977346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1928070700 0x7f19281562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:12.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.246+0000 7f192e210640 1 --2- 172.21.15.76:0/3249977346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f192806bb60 0x7f19280cc680 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:12.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.246+0000 7f192e210640 1 -- 172.21.15.76:0/3249977346 >> 172.21.15.76:0/3249977346 conn(0x7f19280cb530 msgr2=0x7f192815c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:12.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.246+0000 7f192e210640 1 -- 172.21.15.76:0/3249977346 shutdown_connections 2023-11-09T03:54:12.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.246+0000 7f192e210640 1 -- 172.21.15.76:0/3249977346 wait complete. 2023-11-09T03:54:12.267 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-09T03:54:12.268 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:54:12.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.367+0000 7fc6f1341640 1 Processor -- start 2023-11-09T03:54:12.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.367+0000 7fc6f1341640 1 -- start start 2023-11-09T03:54:12.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.368+0000 7fc6f1341640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6ec15d1b0 0x7fc6ec0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:12.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.368+0000 7fc6f1341640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc6ec0d14d0 0x7fc6ec0b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:12.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.368+0000 7fc6f1341640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6ec15d590 con 0x7fc6ec0b4e80 2023-11-09T03:54:12.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.368+0000 7fc6f1341640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6ec0ce6c0 con 0x7fc6ec0d14d0 2023-11-09T03:54:12.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.368+0000 7fc6f1341640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec0b5310 con 0x7fc6ec15d1b0 2023-11-09T03:54:12.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.368+0000 7fc6ea7fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc6ec0d14d0 0x7fc6ec0b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:12.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.368+0000 7fc6eb7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc6ec0b4e80 0x7fc6ec06bf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:55764/0 (socket says 172.21.15.76:55764) 2023-11-09T03:54:12.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.368+0000 7fc6ea7fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc6ec0d14d0 0x7fc6ec0b4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40666/0 (socket says 172.21.15.76:40666) 2023-11-09T03:54:12.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.368+0000 7fc6eb7fe640 1 -- 172.21.15.76:0/2815373659 learned_addr learned my addr 172.21.15.76:0/2815373659 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:54:12.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.368+0000 7fc6ea7fc640 1 -- 172.21.15.76:0/2815373659 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6ec15d1b0 msgr2=0x7fc6ec0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:12.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.368+0000 7fc6eaffd640 1 --2- 172.21.15.76:0/2815373659 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6ec15d1b0 0x7fc6ec0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:12.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.368+0000 7fc6e9ffb640 1 -- 172.21.15.76:0/2815373659 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3895282477 0 0) 0x7fc6e4001490 con 0x7fc6ec0b4e80 2023-11-09T03:54:12.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.368+0000 7fc6ea7fc640 1 --2- 172.21.15.76:0/2815373659 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6ec15d1b0 0x7fc6ec0d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:12.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.368+0000 7fc6ea7fc640 1 -- 172.21.15.76:0/2815373659 >> v1:172.21.15.76:6789/0 conn(0x7fc6ec0b4e80 legacy=0x7fc6ec06bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:12.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.368+0000 7fc6ea7fc640 1 -- 172.21.15.76:0/2815373659 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ec0b7000 con 0x7fc6ec0d14d0 2023-11-09T03:54:12.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.368+0000 7fc6eaffd640 1 --2- 172.21.15.76:0/2815373659 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6ec15d1b0 0x7fc6ec0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:54:12.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.369+0000 7fc6ea7fc640 1 --2- 172.21.15.76:0/2815373659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6ec0d14d0 0x7fc6ec0b4940 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fc6e00097a0 tx=0x7fc6e002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62d6d9848392d95 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:12.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.369+0000 7fc6e9ffb640 1 -- 172.21.15.76:0/2815373659 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0031830 con 0x7fc6ec0d14d0 2023-11-09T03:54:12.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.369+0000 7fc6e9ffb640 1 -- 172.21.15.76:0/2815373659 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e0031990 con 0x7fc6ec0d14d0 2023-11-09T03:54:12.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.369+0000 7fc6e9ffb640 1 -- 172.21.15.76:0/2815373659 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0031d20 con 0x7fc6ec0d14d0 2023-11-09T03:54:12.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.370+0000 7fc6f1341640 1 -- 172.21.15.76:0/2815373659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6ec0d14d0 msgr2=0x7fc6ec0b4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:12.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.370+0000 7fc6f1341640 1 --2- 172.21.15.76:0/2815373659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6ec0d14d0 0x7fc6ec0b4940 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fc6e00097a0 tx=0x7fc6e002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:54:12.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.370+0000 7fc6f1341640 1 -- 172.21.15.76:0/2815373659 shutdown_connections 2023-11-09T03:54:12.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.370+0000 7fc6f1341640 1 --2- 172.21.15.76:0/2815373659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6ec0d14d0 0x7fc6ec0b4940 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:12.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.370+0000 7fc6f1341640 1 --2- 172.21.15.76:0/2815373659 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6ec15d1b0 0x7fc6ec0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:12.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.370+0000 7fc6f1341640 1 -- 172.21.15.76:0/2815373659 >> 172.21.15.76:0/2815373659 conn(0x7fc6ec0cf940 msgr2=0x7fc6ec0cfd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:12.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.370+0000 7fc6f1341640 1 -- 172.21.15.76:0/2815373659 shutdown_connections 2023-11-09T03:54:12.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.370+0000 7fc6f1341640 1 -- 172.21.15.76:0/2815373659 wait complete. 2023-11-09T03:54:12.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.371+0000 7fc6f1341640 1 Processor -- start 2023-11-09T03:54:12.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.371+0000 7fc6f1341640 1 -- start start 2023-11-09T03:54:12.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.372+0000 7fc6f1341640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6ec0b4e80 0x7fc6ec0c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:12.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.372+0000 7fc6f1341640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6ec0d14d0 0x7fc6ec0c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:12.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.372+0000 7fc6f1341640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6ec15d1b0 0x7fc6ec0d32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:12.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.372+0000 7fc6f1341640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec0c6310 con 0x7fc6ec0d14d0 2023-11-09T03:54:12.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.372+0000 7fc6f1341640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec0ca7e0 con 0x7fc6ec15d1b0 2023-11-09T03:54:12.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.372+0000 7fc6f1341640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec0caac0 con 0x7fc6ec0b4e80 2023-11-09T03:54:12.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.372+0000 7fc6ea7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6ec0d14d0 0x7fc6ec0c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:12.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.372+0000 7fc6ea7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6ec0d14d0 0x7fc6ec0c9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40676/0 (socket says 172.21.15.76:40676) 2023-11-09T03:54:12.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.372+0000 7fc6ea7fc640 1 -- 172.21.15.76:0/2707436093 learned_addr learned my addr 172.21.15.76:0/2707436093 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:12.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.372+0000 7fc6ea7fc640 1 -- 172.21.15.76:0/2707436093 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6ec0b4e80 msgr2=0x7fc6ec0c5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:12.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.372+0000 7fc6ea7fc640 1 --2- 172.21.15.76:0/2707436093 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6ec0b4e80 0x7fc6ec0c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:12.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.372+0000 7fc6ea7fc640 1 -- 172.21.15.76:0/2707436093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6ec15d1b0 msgr2=0x7fc6ec0d32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:12.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.372+0000 7fc6ea7fc640 1 --2- 172.21.15.76:0/2707436093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6ec15d1b0 0x7fc6ec0d32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:12.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.372+0000 7fc6ea7fc640 1 -- 172.21.15.76:0/2707436093 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e0009400 con 0x7fc6ec0d14d0 2023-11-09T03:54:12.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.372+0000 7fc6ea7fc640 1 --2- 172.21.15.76:0/2707436093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6ec0d14d0 0x7fc6ec0c9f90 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fc6e00098c0 tx=0x7fc6e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:12.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.372+0000 7fc6d37fe640 1 -- 172.21.15.76:0/2707436093 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e003ba70 con 0x7fc6ec0d14d0 2023-11-09T03:54:12.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.373+0000 7fc6d37fe640 1 -- 172.21.15.76:0/2707436093 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e003bbd0 con 0x7fc6ec0d14d0 2023-11-09T03:54:12.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.373+0000 7fc6f1341640 1 -- 172.21.15.76:0/2707436093 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6ec0cada0 con 0x7fc6ec0d14d0 2023-11-09T03:54:12.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.373+0000 7fc6d37fe640 1 -- 172.21.15.76:0/2707436093 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e00453f0 con 0x7fc6ec0d14d0 2023-11-09T03:54:12.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.373+0000 7fc6f1341640 1 -- 172.21.15.76:0/2707436093 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6ec0cb360 con 0x7fc6ec0d14d0 2023-11-09T03:54:12.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.373+0000 7fc6f1341640 1 -- 172.21.15.76:0/2707436093 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6ec0684b0 con 0x7fc6ec0d14d0 2023-11-09T03:54:12.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.374+0000 7fc6d37fe640 1 -- 172.21.15.76:0/2707436093 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc6e004e450 con 0x7fc6ec0d14d0 2023-11-09T03:54:12.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.374+0000 7fc6d37fe640 1 --2- 172.21.15.76:0/2707436093 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc6d806ae70 0x7fc6d806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:12.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.374+0000 7fc6d37fe640 1 -- 172.21.15.76:0/2707436093 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8253+0+0 (secure 0 0 0) 0x7fc6e00ac310 con 0x7fc6ec0d14d0 2023-11-09T03:54:12.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.375+0000 7fc6eaffd640 1 --2- 172.21.15.76:0/2707436093 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc6d806ae70 0x7fc6d806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:12.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.375+0000 7fc6eaffd640 1 --2- 172.21.15.76:0/2707436093 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc6d806ae70 0x7fc6d806d330 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc006fd0 tx=0x7fc6dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:12.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.379+0000 7fc6d37fe640 1 -- 172.21.15.76:0/2707436093 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc6e0076980 con 0x7fc6ec0d14d0 2023-11-09T03:54:12.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.578+0000 7fc6f1341640 1 -- 172.21.15.76:0/2707436093 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6ec06c920 con 0x7fc6ec0d14d0 2023-11-09T03:54:12.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.579+0000 7fc6d37fe640 1 -- 172.21.15.76:0/2707436093 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+4938 (secure 0 0 0) 0x7fc6e0079fa0 con 0x7fc6ec0d14d0 2023-11-09T03:54:12.580 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:54:12.580 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":123,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":9024,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:07.587497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":6470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6470":{"gid":6470,"name":"b","rank":1,"incarnation":121,"state":"up:reconnect","state_seq":13,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:54:12.581 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 123 2023-11-09T03:54:12.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.581+0000 7fc6f1341640 1 -- 172.21.15.76:0/2707436093 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc6d806ae70 msgr2=0x7fc6d806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:12.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.581+0000 7fc6f1341640 1 --2- 172.21.15.76:0/2707436093 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc6d806ae70 0x7fc6d806d330 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc006fd0 tx=0x7fc6dc008040 comp rx=0 tx=0).stop 2023-11-09T03:54:12.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.581+0000 7fc6f1341640 1 -- 172.21.15.76:0/2707436093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6ec0d14d0 msgr2=0x7fc6ec0c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:12.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.581+0000 7fc6f1341640 1 --2- 172.21.15.76:0/2707436093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6ec0d14d0 0x7fc6ec0c9f90 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fc6e00098c0 tx=0x7fc6e00046c0 comp rx=0 tx=0).stop 2023-11-09T03:54:12.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.581+0000 7fc6f1341640 1 -- 172.21.15.76:0/2707436093 shutdown_connections 2023-11-09T03:54:12.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.581+0000 7fc6f1341640 1 --2- 172.21.15.76:0/2707436093 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc6d806ae70 0x7fc6d806d330 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:12.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.581+0000 7fc6f1341640 1 --2- 172.21.15.76:0/2707436093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6ec15d1b0 0x7fc6ec0d32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:12.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.581+0000 7fc6f1341640 1 --2- 172.21.15.76:0/2707436093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6ec0d14d0 0x7fc6ec0c9f90 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:12.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.581+0000 7fc6f1341640 1 --2- 172.21.15.76:0/2707436093 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6ec0b4e80 0x7fc6ec0c5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:12.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.581+0000 7fc6f1341640 1 -- 172.21.15.76:0/2707436093 >> 172.21.15.76:0/2707436093 conn(0x7fc6ec0cf940 msgr2=0x7fc6ec0b71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:12.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.581+0000 7fc6f1341640 1 -- 172.21.15.76:0/2707436093 shutdown_connections 2023-11-09T03:54:12.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:12.581+0000 7fc6f1341640 1 -- 172.21.15.76:0/2707436093 wait complete. 2023-11-09T03:54:12.994 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 123, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-09T03:48:57.383848+0000', 'modified': '2023-11-09T03:54:07.587497+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 115, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 8568, 'mds_1': 6470}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6470': {'gid': 6470, 'name': 'b', 'rank': 1, 'incarnation': 121, 'state': 'up:reconnect', 'state_seq': 13, 'addr': '172.21.15.113:6837/3462040782', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6836', 'nonce': 3462040782}, {'type': 'v1', 'addr': '172.21.15.113:6837', 'nonce': 3462040782}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_8568': {'gid': 8568, 'name': 'c', 'rank': 0, 'incarnation': 110, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.76:6833/153465374', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6832', 'nonce': 153465374}, {'type': 'v1', 'addr': '172.21.15.76:6833', 'nonce': 153465374}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-09T03:54:12.995 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6470:up:reconnect 2023-11-09T03:54:13.998 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:54:14.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.106+0000 7f08c279e640 1 Processor -- start 2023-11-09T03:54:14.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.106+0000 7f08c279e640 1 -- start start 2023-11-09T03:54:14.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.107+0000 7f08c279e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f08bc0cd820 0x7f08bc0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:14.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.107+0000 7f08c279e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f08bc15cf60 0x7f08bc1608c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:14.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.107+0000 7f08c279e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08bc0ce140 con 0x7f08bc0b84e0 2023-11-09T03:54:14.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.107+0000 7f08c279e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08bc1611c0 con 0x7f08bc15cf60 2023-11-09T03:54:14.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.107+0000 7f08c279e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08bc1614a0 con 0x7f08bc0cd820 2023-11-09T03:54:14.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.107+0000 7f08c0f9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f08bc15cf60 0x7f08bc1608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:14.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.107+0000 7f08c1f9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f08bc0b84e0 0x7f08bc0b45d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45738/0 (socket says 172.21.15.76:45738) 2023-11-09T03:54:14.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.107+0000 7f08c1f9d640 1 -- 172.21.15.76:0/2192375700 learned_addr learned my addr 172.21.15.76:0/2192375700 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:54:14.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.107+0000 7f08c179c640 1 --2- 172.21.15.76:0/2192375700 >> v2:172.21.15.113:3300/0 conn(0x7f08bc0cd820 0x7f08bc0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:14.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.107+0000 7f08c0f9b640 1 -- 172.21.15.76:0/2192375700 >> v2:172.21.15.113:3300/0 conn(0x7f08bc0cd820 msgr2=0x7f08bc0cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:14.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.107+0000 7f08c0f9b640 1 --2- 172.21.15.76:0/2192375700 >> v2:172.21.15.113:3300/0 conn(0x7f08bc0cd820 0x7f08bc0cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:14.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.107+0000 7f08c0f9b640 1 -- 172.21.15.76:0/2192375700 >> v1:172.21.15.76:6789/0 conn(0x7f08bc0b84e0 legacy=0x7f08bc0b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:14.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.107+0000 7f08b3fff640 1 -- 172.21.15.76:0/2192375700 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3416077995 0 0) 0x7f08bc0ce140 con 0x7f08bc0b84e0 2023-11-09T03:54:14.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.107+0000 7f08c0f9b640 1 -- 172.21.15.76:0/2192375700 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08bc161780 con 0x7f08bc15cf60 2023-11-09T03:54:14.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.108+0000 7f08c0f9b640 1 --2- 172.21.15.76:0/2192375700 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08bc15cf60 0x7f08bc1608c0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f08bc08b4e0 tx=0x7f08ac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21911299f0749299 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:14.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.108+0000 7f08b3fff640 1 -- 172.21.15.76:0/2192375700 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac031830 con 0x7f08bc15cf60 2023-11-09T03:54:14.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.108+0000 7f08b3fff640 1 -- 172.21.15.76:0/2192375700 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ac031990 con 0x7f08bc15cf60 2023-11-09T03:54:14.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.108+0000 7f08b3fff640 1 -- 172.21.15.76:0/2192375700 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac031d20 con 0x7f08bc15cf60 2023-11-09T03:54:14.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.108+0000 7f08c279e640 1 -- 172.21.15.76:0/2192375700 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08bc15cf60 msgr2=0x7f08bc1608c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:14.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.108+0000 7f08c279e640 1 --2- 172.21.15.76:0/2192375700 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08bc15cf60 0x7f08bc1608c0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f08bc08b4e0 tx=0x7f08ac02b8d0 comp rx=0 tx=0).stop 2023-11-09T03:54:14.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.109+0000 7f08c279e640 1 -- 172.21.15.76:0/2192375700 shutdown_connections 2023-11-09T03:54:14.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.109+0000 7f08c279e640 1 --2- 172.21.15.76:0/2192375700 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08bc15cf60 0x7f08bc1608c0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:14.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.109+0000 7f08c279e640 1 --2- 172.21.15.76:0/2192375700 >> v2:172.21.15.113:3300/0 conn(0x7f08bc0cd820 0x7f08bc0cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:14.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.109+0000 7f08c279e640 1 -- 172.21.15.76:0/2192375700 >> 172.21.15.76:0/2192375700 conn(0x7f08bc06a5e0 msgr2=0x7f08bc06a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:14.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.109+0000 7f08c279e640 1 -- 172.21.15.76:0/2192375700 shutdown_connections 2023-11-09T03:54:14.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.109+0000 7f08c279e640 1 -- 172.21.15.76:0/2192375700 wait complete. 2023-11-09T03:54:14.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.110+0000 7f08c279e640 1 Processor -- start 2023-11-09T03:54:14.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.110+0000 7f08c279e640 1 -- start start 2023-11-09T03:54:14.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.111+0000 7f08c279e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08bc0b84e0 0x7f08bc0ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:14.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.111+0000 7f08c279e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08bc0cd820 0x7f08bc0b4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:14.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.111+0000 7f08c279e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08bc15cf60 0x7f08bc1f3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:14.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.111+0000 7f08c279e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc0c7610 con 0x7f08bc0cd820 2023-11-09T03:54:14.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.111+0000 7f08c279e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc0cba90 con 0x7f08bc15cf60 2023-11-09T03:54:14.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.111+0000 7f08c279e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc0cab70 con 0x7f08bc0b84e0 2023-11-09T03:54:14.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.111+0000 7f08c0f9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08bc0cd820 0x7f08bc0b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:14.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.111+0000 7f08c0f9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08bc0cd820 0x7f08bc0b4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57068/0 (socket says 172.21.15.76:57068) 2023-11-09T03:54:14.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.111+0000 7f08c0f9b640 1 -- 172.21.15.76:0/3135903747 learned_addr learned my addr 172.21.15.76:0/3135903747 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:14.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.111+0000 7f08c1f9d640 1 --2- 172.21.15.76:0/3135903747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08bc15cf60 0x7f08bc1f3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:14.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.111+0000 7f08c0f9b640 1 -- 172.21.15.76:0/3135903747 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08bc0b84e0 msgr2=0x7f08bc0ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:14.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.111+0000 7f08c0f9b640 1 --2- 172.21.15.76:0/3135903747 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08bc0b84e0 0x7f08bc0ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:14.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.111+0000 7f08c0f9b640 1 -- 172.21.15.76:0/3135903747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08bc15cf60 msgr2=0x7f08bc1f3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:14.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.111+0000 7f08c0f9b640 1 --2- 172.21.15.76:0/3135903747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08bc15cf60 0x7f08bc1f3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:14.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.111+0000 7f08c0f9b640 1 -- 172.21.15.76:0/3135903747 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ac009400 con 0x7f08bc0cd820 2023-11-09T03:54:14.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.111+0000 7f08c1f9d640 1 --2- 172.21.15.76:0/3135903747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08bc15cf60 0x7f08bc1f3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:54:14.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.111+0000 7f08c0f9b640 1 --2- 172.21.15.76:0/3135903747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08bc0cd820 0x7f08bc0b4500 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f08ac000c00 tx=0x7f08ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:14.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.111+0000 7f08b1ffb640 1 -- 172.21.15.76:0/3135903747 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac03aa60 con 0x7f08bc0cd820 2023-11-09T03:54:14.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.111+0000 7f08b1ffb640 1 -- 172.21.15.76:0/3135903747 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ac03abe0 con 0x7f08bc0cd820 2023-11-09T03:54:14.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.111+0000 7f08c279e640 1 -- 172.21.15.76:0/3135903747 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08bc0c6990 con 0x7f08bc0cd820 2023-11-09T03:54:14.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.111+0000 7f08b1ffb640 1 -- 172.21.15.76:0/3135903747 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac0443f0 con 0x7f08bc0cd820 2023-11-09T03:54:14.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.112+0000 7f08c279e640 1 -- 172.21.15.76:0/3135903747 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08bc0c6f50 con 0x7f08bc0cd820 2023-11-09T03:54:14.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.113+0000 7f08c279e640 1 -- 172.21.15.76:0/3135903747 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08bc0742c0 con 0x7f08bc0cd820 2023-11-09T03:54:14.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.113+0000 7f08b1ffb640 1 -- 172.21.15.76:0/3135903747 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f08ac04d450 con 0x7f08bc0cd820 2023-11-09T03:54:14.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.113+0000 7f08b1ffb640 1 --2- 172.21.15.76:0/3135903747 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f08a006ae70 0x7f08a006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:14.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.113+0000 7f08b1ffb640 1 -- 172.21.15.76:0/3135903747 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8253+0+0 (secure 0 0 0) 0x7f08ac0ac1e0 con 0x7f08bc0cd820 2023-11-09T03:54:14.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.114+0000 7f08c179c640 1 --2- 172.21.15.76:0/3135903747 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f08a006ae70 0x7f08a006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:14.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.114+0000 7f08c179c640 1 --2- 172.21.15.76:0/3135903747 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f08a006ae70 0x7f08a006d330 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f08b8006fd0 tx=0x7f08b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:14.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.118+0000 7f08b1ffb640 1 -- 172.21.15.76:0/3135903747 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f08ac076850 con 0x7f08bc0cd820 2023-11-09T03:54:14.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.318+0000 7f08c279e640 1 -- 172.21.15.76:0/3135903747 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08bc0b2290 con 0x7f08bc0cd820 2023-11-09T03:54:14.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.319+0000 7f08b1ffb640 1 -- 172.21.15.76:0/3135903747 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+4938 (secure 0 0 0) 0x7f08ac035020 con 0x7f08bc0cd820 2023-11-09T03:54:14.319 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:54:14.320 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":123,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":9024,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:07.587497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":6470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6470":{"gid":6470,"name":"b","rank":1,"incarnation":121,"state":"up:reconnect","state_seq":13,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:54:14.320 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 123 2023-11-09T03:54:14.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.320+0000 7f08c279e640 1 -- 172.21.15.76:0/3135903747 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f08a006ae70 msgr2=0x7f08a006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:14.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.320+0000 7f08c279e640 1 --2- 172.21.15.76:0/3135903747 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f08a006ae70 0x7f08a006d330 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f08b8006fd0 tx=0x7f08b8008040 comp rx=0 tx=0).stop 2023-11-09T03:54:14.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.320+0000 7f08c279e640 1 -- 172.21.15.76:0/3135903747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08bc0cd820 msgr2=0x7f08bc0b4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:14.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.320+0000 7f08c279e640 1 --2- 172.21.15.76:0/3135903747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08bc0cd820 0x7f08bc0b4500 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f08ac000c00 tx=0x7f08ac0046c0 comp rx=0 tx=0).stop 2023-11-09T03:54:14.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.320+0000 7f08c279e640 1 -- 172.21.15.76:0/3135903747 shutdown_connections 2023-11-09T03:54:14.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.321+0000 7f08c279e640 1 --2- 172.21.15.76:0/3135903747 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f08a006ae70 0x7f08a006d330 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:14.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.321+0000 7f08c279e640 1 --2- 172.21.15.76:0/3135903747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08bc15cf60 0x7f08bc1f3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:14.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.321+0000 7f08c279e640 1 --2- 172.21.15.76:0/3135903747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08bc0cd820 0x7f08bc0b4500 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:14.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.321+0000 7f08c279e640 1 --2- 172.21.15.76:0/3135903747 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08bc0b84e0 0x7f08bc0ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:14.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.321+0000 7f08c279e640 1 -- 172.21.15.76:0/3135903747 >> 172.21.15.76:0/3135903747 conn(0x7f08bc06a5e0 msgr2=0x7f08bc0b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:14.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.321+0000 7f08c279e640 1 -- 172.21.15.76:0/3135903747 shutdown_connections 2023-11-09T03:54:14.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:14.321+0000 7f08c279e640 1 -- 172.21.15.76:0/3135903747 wait complete. 2023-11-09T03:54:14.734 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 123, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-09T03:48:57.383848+0000', 'modified': '2023-11-09T03:54:07.587497+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 115, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 8568, 'mds_1': 6470}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6470': {'gid': 6470, 'name': 'b', 'rank': 1, 'incarnation': 121, 'state': 'up:reconnect', 'state_seq': 13, 'addr': '172.21.15.113:6837/3462040782', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6836', 'nonce': 3462040782}, {'type': 'v1', 'addr': '172.21.15.113:6837', 'nonce': 3462040782}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_8568': {'gid': 8568, 'name': 'c', 'rank': 0, 'incarnation': 110, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.76:6833/153465374', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6832', 'nonce': 153465374}, {'type': 'v1', 'addr': '172.21.15.76:6833', 'nonce': 153465374}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-09T03:54:14.735 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6470:up:reconnect 2023-11-09T03:54:15.736 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:54:15.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.840+0000 7faf6d4bb640 1 Processor -- start 2023-11-09T03:54:15.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.840+0000 7faf6d4bb640 1 -- start start 2023-11-09T03:54:15.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.841+0000 7faf6d4bb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf6815d4e0 0x7faf680d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:15.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.841+0000 7faf6d4bb640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faf680d3b60 0x7faf680d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:15.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.841+0000 7faf6d4bb640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf680b32b0 con 0x7faf6815d100 2023-11-09T03:54:15.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.841+0000 7faf6d4bb640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf680cec20 con 0x7faf680d3b60 2023-11-09T03:54:15.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.841+0000 7faf6d4bb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf680cee10 con 0x7faf6815d4e0 2023-11-09T03:54:15.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.841+0000 7faf667fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf6815d4e0 0x7faf680d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:15.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.841+0000 7faf677fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faf680d3b60 0x7faf680d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:15.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.841+0000 7faf66ffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7faf6815d100 0x7faf680ce510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:40860/0 (socket says 172.21.15.76:40860) 2023-11-09T03:54:15.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.841+0000 7faf667fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf6815d4e0 0x7faf680d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:50022/0 (socket says 172.21.15.76:50022) 2023-11-09T03:54:15.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.841+0000 7faf66ffd640 1 -- 172.21.15.76:0/1387812208 learned_addr learned my addr 172.21.15.76:0/1387812208 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:54:15.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.842+0000 7faf667fc640 1 -- 172.21.15.76:0/1387812208 >> v2:172.21.15.113:3300/0 conn(0x7faf680d3b60 msgr2=0x7faf680d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:15.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.842+0000 7faf667fc640 1 --2- 172.21.15.76:0/1387812208 >> v2:172.21.15.113:3300/0 conn(0x7faf680d3b60 0x7faf680d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:15.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.842+0000 7faf667fc640 1 -- 172.21.15.76:0/1387812208 >> v1:172.21.15.113:6789/0 conn(0x7faf6815d100 legacy=0x7faf680ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:15.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.842+0000 7faf667fc640 1 -- 172.21.15.76:0/1387812208 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf680b4750 con 0x7faf6815d4e0 2023-11-09T03:54:15.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.842+0000 7faf65ffb640 1 -- 172.21.15.76:0/1387812208 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3847056131 0 0) 0x7faf680b32b0 con 0x7faf6815d100 2023-11-09T03:54:15.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.842+0000 7faf677fe640 1 --2- 172.21.15.76:0/1387812208 >> v2:172.21.15.113:3300/0 conn(0x7faf680d3b60 0x7faf680d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:54:15.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.843+0000 7faf667fc640 1 --2- 172.21.15.76:0/1387812208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf6815d4e0 0x7faf680d14d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0097a0 tx=0x7faf5c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d309064e9ed96726 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:15.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.843+0000 7faf65ffb640 1 -- 172.21.15.76:0/1387812208 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c031830 con 0x7faf6815d4e0 2023-11-09T03:54:15.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.843+0000 7faf65ffb640 1 -- 172.21.15.76:0/1387812208 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5c031990 con 0x7faf6815d4e0 2023-11-09T03:54:15.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.843+0000 7faf65ffb640 1 -- 172.21.15.76:0/1387812208 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c031d20 con 0x7faf6815d4e0 2023-11-09T03:54:15.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.843+0000 7faf6d4bb640 1 -- 172.21.15.76:0/1387812208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf6815d4e0 msgr2=0x7faf680d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:15.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.843+0000 7faf6d4bb640 1 --2- 172.21.15.76:0/1387812208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf6815d4e0 0x7faf680d14d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0097a0 tx=0x7faf5c02b8d0 comp rx=0 tx=0).stop 2023-11-09T03:54:15.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.843+0000 7faf6d4bb640 1 -- 172.21.15.76:0/1387812208 shutdown_connections 2023-11-09T03:54:15.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.843+0000 7faf6d4bb640 1 --2- 172.21.15.76:0/1387812208 >> v2:172.21.15.113:3300/0 conn(0x7faf680d3b60 0x7faf680d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:15.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.843+0000 7faf6d4bb640 1 --2- 172.21.15.76:0/1387812208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf6815d4e0 0x7faf680d14d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:15.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.843+0000 7faf6d4bb640 1 -- 172.21.15.76:0/1387812208 >> 172.21.15.76:0/1387812208 conn(0x7faf680d2ef0 msgr2=0x7faf680d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:15.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.843+0000 7faf6d4bb640 1 -- 172.21.15.76:0/1387812208 shutdown_connections 2023-11-09T03:54:15.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.843+0000 7faf6d4bb640 1 -- 172.21.15.76:0/1387812208 wait complete. 2023-11-09T03:54:15.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.845+0000 7faf6d4bb640 1 Processor -- start 2023-11-09T03:54:15.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.845+0000 7faf6d4bb640 1 -- start start 2023-11-09T03:54:15.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.845+0000 7faf6d4bb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf680d3b60 0x7faf68152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:15.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.845+0000 7faf6d4bb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf6815d100 0x7faf681600b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:15.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.845+0000 7faf6d4bb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf6815d4e0 0x7faf681f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:15.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.845+0000 7faf6d4bb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf681529a0 con 0x7faf6815d4e0 2023-11-09T03:54:15.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.845+0000 7faf6d4bb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf681550a0 con 0x7faf6815d100 2023-11-09T03:54:15.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.845+0000 7faf6d4bb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf68155380 con 0x7faf680d3b60 2023-11-09T03:54:15.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.845+0000 7faf677fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf6815d4e0 0x7faf681f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:15.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.846+0000 7faf677fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf6815d4e0 0x7faf681f2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57070/0 (socket says 172.21.15.76:57070) 2023-11-09T03:54:15.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.846+0000 7faf677fe640 1 -- 172.21.15.76:0/2181816010 learned_addr learned my addr 172.21.15.76:0/2181816010 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:15.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.846+0000 7faf66ffd640 1 --2- 172.21.15.76:0/2181816010 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf680d3b60 0x7faf68152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:15.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.846+0000 7faf667fc640 1 --2- 172.21.15.76:0/2181816010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf6815d100 0x7faf681600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:15.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.846+0000 7faf677fe640 1 -- 172.21.15.76:0/2181816010 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf680d3b60 msgr2=0x7faf68152460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:15.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.846+0000 7faf677fe640 1 --2- 172.21.15.76:0/2181816010 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf680d3b60 0x7faf68152460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:15.860 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.846+0000 7faf677fe640 1 -- 172.21.15.76:0/2181816010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf6815d100 msgr2=0x7faf681600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:15.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.846+0000 7faf677fe640 1 --2- 172.21.15.76:0/2181816010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf6815d100 0x7faf681600b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:15.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.846+0000 7faf677fe640 1 -- 172.21.15.76:0/2181816010 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf60009a70 con 0x7faf6815d4e0 2023-11-09T03:54:15.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.846+0000 7faf677fe640 1 --2- 172.21.15.76:0/2181816010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf6815d4e0 0x7faf681f2c90 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7faf6000efc0 tx=0x7faf60009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:15.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.846+0000 7faf4f7fe640 1 -- 172.21.15.76:0/2181816010 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf60009d70 con 0x7faf6815d4e0 2023-11-09T03:54:15.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.846+0000 7faf4f7fe640 1 -- 172.21.15.76:0/2181816010 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6000f040 con 0x7faf6815d4e0 2023-11-09T03:54:15.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.846+0000 7faf6d4bb640 1 -- 172.21.15.76:0/2181816010 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf5c009400 con 0x7faf6815d4e0 2023-11-09T03:54:15.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.846+0000 7faf4f7fe640 1 -- 172.21.15.76:0/2181816010 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf60003990 con 0x7faf6815d4e0 2023-11-09T03:54:15.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.846+0000 7faf6d4bb640 1 -- 172.21.15.76:0/2181816010 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf68155940 con 0x7faf6815d4e0 2023-11-09T03:54:15.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.847+0000 7faf6d4bb640 1 -- 172.21.15.76:0/2181816010 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf680684b0 con 0x7faf6815d4e0 2023-11-09T03:54:15.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.848+0000 7faf4f7fe640 1 -- 172.21.15.76:0/2181816010 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faf600144a0 con 0x7faf6815d4e0 2023-11-09T03:54:15.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.848+0000 7faf4f7fe640 1 --2- 172.21.15.76:0/2181816010 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf5406ae70 0x7faf5406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:15.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.848+0000 7faf4f7fe640 1 -- 172.21.15.76:0/2181816010 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7faf6008ab40 con 0x7faf6815d4e0 2023-11-09T03:54:15.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.848+0000 7faf66ffd640 1 --2- 172.21.15.76:0/2181816010 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf5406ae70 0x7faf5406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:15.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.849+0000 7faf66ffd640 1 --2- 172.21.15.76:0/2181816010 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf5406ae70 0x7faf5406d330 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7faf58000fb0 tx=0x7faf58007ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:15.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:15.852+0000 7faf4f7fe640 1 -- 172.21.15.76:0/2181816010 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faf600552c0 con 0x7faf6815d4e0 2023-11-09T03:54:16.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:16.054+0000 7faf6d4bb640 1 -- 172.21.15.76:0/2181816010 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf680d2cd0 con 0x7faf6815d4e0 2023-11-09T03:54:16.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:16.055+0000 7faf4f7fe640 1 -- 172.21.15.76:0/2181816010 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+4938 (secure 0 0 0) 0x7faf60018090 con 0x7faf6815d4e0 2023-11-09T03:54:16.056 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:54:16.056 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":123,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":9024,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:07.587497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":6470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6470":{"gid":6470,"name":"b","rank":1,"incarnation":121,"state":"up:reconnect","state_seq":13,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:54:16.058 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 123 2023-11-09T03:54:16.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:16.057+0000 7faf6d4bb640 1 -- 172.21.15.76:0/2181816010 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf5406ae70 msgr2=0x7faf5406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:16.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:16.057+0000 7faf6d4bb640 1 --2- 172.21.15.76:0/2181816010 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf5406ae70 0x7faf5406d330 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7faf58000fb0 tx=0x7faf58007ce0 comp rx=0 tx=0).stop 2023-11-09T03:54:16.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:16.057+0000 7faf6d4bb640 1 -- 172.21.15.76:0/2181816010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf6815d4e0 msgr2=0x7faf681f2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:16.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:16.057+0000 7faf6d4bb640 1 --2- 172.21.15.76:0/2181816010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf6815d4e0 0x7faf681f2c90 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7faf6000efc0 tx=0x7faf60009f90 comp rx=0 tx=0).stop 2023-11-09T03:54:16.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:16.057+0000 7faf6d4bb640 1 -- 172.21.15.76:0/2181816010 shutdown_connections 2023-11-09T03:54:16.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:16.057+0000 7faf6d4bb640 1 --2- 172.21.15.76:0/2181816010 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7faf5406ae70 0x7faf5406d330 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:16.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:16.057+0000 7faf6d4bb640 1 --2- 172.21.15.76:0/2181816010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf6815d4e0 0x7faf681f2c90 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:16.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:16.057+0000 7faf6d4bb640 1 --2- 172.21.15.76:0/2181816010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf6815d100 0x7faf681600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:16.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:16.057+0000 7faf6d4bb640 1 --2- 172.21.15.76:0/2181816010 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf680d3b60 0x7faf68152460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:16.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:16.057+0000 7faf6d4bb640 1 -- 172.21.15.76:0/2181816010 >> 172.21.15.76:0/2181816010 conn(0x7faf680d2ef0 msgr2=0x7faf680cf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:16.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:16.057+0000 7faf6d4bb640 1 -- 172.21.15.76:0/2181816010 shutdown_connections 2023-11-09T03:54:16.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:16.057+0000 7faf6d4bb640 1 -- 172.21.15.76:0/2181816010 wait complete. 2023-11-09T03:54:16.473 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 123, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-09T03:48:57.383848+0000', 'modified': '2023-11-09T03:54:07.587497+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 115, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 8568, 'mds_1': 6470}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6470': {'gid': 6470, 'name': 'b', 'rank': 1, 'incarnation': 121, 'state': 'up:reconnect', 'state_seq': 13, 'addr': '172.21.15.113:6837/3462040782', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6836', 'nonce': 3462040782}, {'type': 'v1', 'addr': '172.21.15.113:6837', 'nonce': 3462040782}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_8568': {'gid': 8568, 'name': 'c', 'rank': 0, 'incarnation': 110, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.76:6833/153465374', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6832', 'nonce': 153465374}, {'type': 'v1', 'addr': '172.21.15.76:6833', 'nonce': 153465374}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-09T03:54:16.474 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6470:up:reconnect 2023-11-09T03:54:17.475 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:54:17.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.591+0000 7fd4dbc39640 1 Processor -- start 2023-11-09T03:54:17.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.591+0000 7fd4dbc39640 1 -- start start 2023-11-09T03:54:17.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.592+0000 7fd4dbc39640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4d4159e40 0x7fd4d40b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:17.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.592+0000 7fd4dbc39640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd4d406bb60 0x7fd4d40b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:17.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.592+0000 7fd4dbc39640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd4d406fff0 0x7fd4d4070480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:17.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.592+0000 7fd4dbc39640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d4071500 con 0x7fd4d406fff0 2023-11-09T03:54:17.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.592+0000 7fd4dbc39640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d415dae0 con 0x7fd4d406bb60 2023-11-09T03:54:17.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.592+0000 7fd4dbc39640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d40709c0 con 0x7fd4d4159e40 2023-11-09T03:54:17.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.592+0000 7fd4da1d6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd4d406fff0 0x7fd4d4070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:17.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.592+0000 7fd4da1d6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd4d406fff0 0x7fd4d4070480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57076/0 (socket says 172.21.15.76:57076) 2023-11-09T03:54:17.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.592+0000 7fd4da1d6640 1 -- 172.21.15.76:0/237577336 learned_addr learned my addr 172.21.15.76:0/237577336 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:17.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.592+0000 7fd4d91d4640 1 --2- 172.21.15.76:0/237577336 >> v2:172.21.15.113:3300/0 conn(0x7fd4d406bb60 0x7fd4d40b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:17.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.592+0000 7fd4d99d5640 1 --2- 172.21.15.76:0/237577336 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4d4159e40 0x7fd4d40b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:17.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.593+0000 7fd4da1d6640 1 -- 172.21.15.76:0/237577336 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4d4159e40 msgr2=0x7fd4d40b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:17.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.593+0000 7fd4da1d6640 1 --2- 172.21.15.76:0/237577336 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4d4159e40 0x7fd4d40b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:17.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.593+0000 7fd4da1d6640 1 -- 172.21.15.76:0/237577336 >> v2:172.21.15.113:3300/0 conn(0x7fd4d406bb60 msgr2=0x7fd4d40b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:17.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.593+0000 7fd4da1d6640 1 --2- 172.21.15.76:0/237577336 >> v2:172.21.15.113:3300/0 conn(0x7fd4d406bb60 0x7fd4d40b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:17.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.593+0000 7fd4da1d6640 1 -- 172.21.15.76:0/237577336 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d4070c60 con 0x7fd4d406fff0 2023-11-09T03:54:17.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.593+0000 7fd4da1d6640 1 --2- 172.21.15.76:0/237577336 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4d406fff0 0x7fd4d4070480 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fd4d40ce820 tx=0x7fd4cc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462b39b6e70860d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:17.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.593+0000 7fd4d89d3640 1 -- 172.21.15.76:0/237577336 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc02c9e0 con 0x7fd4d406fff0 2023-11-09T03:54:17.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.594+0000 7fd4d89d3640 1 -- 172.21.15.76:0/237577336 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4cc02cb40 con 0x7fd4d406fff0 2023-11-09T03:54:17.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.594+0000 7fd4d89d3640 1 -- 172.21.15.76:0/237577336 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc0343f0 con 0x7fd4d406fff0 2023-11-09T03:54:17.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.594+0000 7fd4dbc39640 1 -- 172.21.15.76:0/237577336 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4d406fff0 msgr2=0x7fd4d4070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:17.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.594+0000 7fd4dbc39640 1 --2- 172.21.15.76:0/237577336 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4d406fff0 0x7fd4d4070480 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fd4d40ce820 tx=0x7fd4cc02be40 comp rx=0 tx=0).stop 2023-11-09T03:54:17.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.594+0000 7fd4dbc39640 1 -- 172.21.15.76:0/237577336 shutdown_connections 2023-11-09T03:54:17.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.594+0000 7fd4dbc39640 1 --2- 172.21.15.76:0/237577336 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4d406fff0 0x7fd4d4070480 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:17.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.594+0000 7fd4dbc39640 1 --2- 172.21.15.76:0/237577336 >> v2:172.21.15.113:3300/0 conn(0x7fd4d406bb60 0x7fd4d40b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:17.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.594+0000 7fd4dbc39640 1 --2- 172.21.15.76:0/237577336 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4d4159e40 0x7fd4d40b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:17.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.594+0000 7fd4dbc39640 1 -- 172.21.15.76:0/237577336 >> 172.21.15.76:0/237577336 conn(0x7fd4d40cb530 msgr2=0x7fd4d40cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:17.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.594+0000 7fd4dbc39640 1 -- 172.21.15.76:0/237577336 shutdown_connections 2023-11-09T03:54:17.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.594+0000 7fd4dbc39640 1 -- 172.21.15.76:0/237577336 wait complete. 2023-11-09T03:54:17.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.596+0000 7fd4dbc39640 1 Processor -- start 2023-11-09T03:54:17.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.596+0000 7fd4dbc39640 1 -- start start 2023-11-09T03:54:17.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.596+0000 7fd4dbc39640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4d406bb60 0x7fd4d4148fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:17.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.596+0000 7fd4dbc39640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4d406fff0 0x7fd4d40c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:17.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.596+0000 7fd4dbc39640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4d4159e40 0x7fd4d40c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:17.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.596+0000 7fd4dbc39640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d40a84c0 con 0x7fd4d406fff0 2023-11-09T03:54:17.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.596+0000 7fd4dbc39640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d40c4db0 con 0x7fd4d406bb60 2023-11-09T03:54:17.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.596+0000 7fd4dbc39640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d40c5090 con 0x7fd4d4159e40 2023-11-09T03:54:17.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.596+0000 7fd4d91d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4d406fff0 0x7fd4d40c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:17.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.596+0000 7fd4d91d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4d406fff0 0x7fd4d40c7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57086/0 (socket says 172.21.15.76:57086) 2023-11-09T03:54:17.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.597+0000 7fd4d91d4640 1 -- 172.21.15.76:0/3250691908 learned_addr learned my addr 172.21.15.76:0/3250691908 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:17.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.597+0000 7fd4d99d5640 1 --2- 172.21.15.76:0/3250691908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4d406bb60 0x7fd4d4148fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:17.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.597+0000 7fd4da1d6640 1 --2- 172.21.15.76:0/3250691908 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4d4159e40 0x7fd4d40c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:17.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.597+0000 7fd4d91d4640 1 -- 172.21.15.76:0/3250691908 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4d4159e40 msgr2=0x7fd4d40c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:17.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.597+0000 7fd4d91d4640 1 --2- 172.21.15.76:0/3250691908 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4d4159e40 0x7fd4d40c8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:17.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.597+0000 7fd4d91d4640 1 -- 172.21.15.76:0/3250691908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4d406bb60 msgr2=0x7fd4d4148fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:17.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.597+0000 7fd4d91d4640 1 --2- 172.21.15.76:0/3250691908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4d406bb60 0x7fd4d4148fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:17.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.597+0000 7fd4d91d4640 1 -- 172.21.15.76:0/3250691908 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc009b30 con 0x7fd4d406fff0 2023-11-09T03:54:17.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.597+0000 7fd4d91d4640 1 --2- 172.21.15.76:0/3250691908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4d406fff0 0x7fd4d40c7d50 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400d800 tx=0x7fd4c400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:17.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.597+0000 7fd4ca7fc640 1 -- 172.21.15.76:0/3250691908 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c40096a0 con 0x7fd4d406fff0 2023-11-09T03:54:17.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.597+0000 7fd4ca7fc640 1 -- 172.21.15.76:0/3250691908 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c4009800 con 0x7fd4d406fff0 2023-11-09T03:54:17.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.597+0000 7fd4dbc39640 1 -- 172.21.15.76:0/3250691908 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d40c5370 con 0x7fd4d406fff0 2023-11-09T03:54:17.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.597+0000 7fd4ca7fc640 1 -- 172.21.15.76:0/3250691908 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4009990 con 0x7fd4d406fff0 2023-11-09T03:54:17.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.597+0000 7fd4dbc39640 1 -- 172.21.15.76:0/3250691908 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d41494f0 con 0x7fd4d406fff0 2023-11-09T03:54:17.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.598+0000 7fd4dbc39640 1 -- 172.21.15.76:0/3250691908 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d41496b0 con 0x7fd4d406fff0 2023-11-09T03:54:17.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.599+0000 7fd4ca7fc640 1 -- 172.21.15.76:0/3250691908 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd4c4009af0 con 0x7fd4d406fff0 2023-11-09T03:54:17.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.599+0000 7fd4ca7fc640 1 --2- 172.21.15.76:0/3250691908 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd4b806ae70 0x7fd4b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:17.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.599+0000 7fd4ca7fc640 1 -- 172.21.15.76:0/3250691908 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7fd4c408a390 con 0x7fd4d406fff0 2023-11-09T03:54:17.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.599+0000 7fd4d99d5640 1 --2- 172.21.15.76:0/3250691908 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd4b806ae70 0x7fd4b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:17.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.600+0000 7fd4d99d5640 1 --2- 172.21.15.76:0/3250691908 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd4b806ae70 0x7fd4b806d330 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00079a0 tx=0x7fd4d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:17.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.604+0000 7fd4ca7fc640 1 -- 172.21.15.76:0/3250691908 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd4c4054a80 con 0x7fd4d406fff0 2023-11-09T03:54:17.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.807+0000 7fd4dbc39640 1 -- 172.21.15.76:0/3250691908 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4d40a8600 con 0x7fd4d406fff0 2023-11-09T03:54:17.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.808+0000 7fd4ca7fc640 1 -- 172.21.15.76:0/3250691908 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4935 (secure 0 0 0) 0x7fd4d40a8600 con 0x7fd4d406fff0 2023-11-09T03:54:17.809 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:54:17.809 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":125,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":9024,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:16.621433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":6470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6470":{"gid":6470,"name":"b","rank":1,"incarnation":121,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:54:17.810 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 125 2023-11-09T03:54:17.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.810+0000 7fd4dbc39640 1 -- 172.21.15.76:0/3250691908 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd4b806ae70 msgr2=0x7fd4b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:17.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.810+0000 7fd4dbc39640 1 --2- 172.21.15.76:0/3250691908 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd4b806ae70 0x7fd4b806d330 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00079a0 tx=0x7fd4d0008040 comp rx=0 tx=0).stop 2023-11-09T03:54:17.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.810+0000 7fd4dbc39640 1 -- 172.21.15.76:0/3250691908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4d406fff0 msgr2=0x7fd4d40c7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:17.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.810+0000 7fd4dbc39640 1 --2- 172.21.15.76:0/3250691908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4d406fff0 0x7fd4d40c7d50 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400d800 tx=0x7fd4c400dcc0 comp rx=0 tx=0).stop 2023-11-09T03:54:17.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.810+0000 7fd4dbc39640 1 -- 172.21.15.76:0/3250691908 shutdown_connections 2023-11-09T03:54:17.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.810+0000 7fd4dbc39640 1 --2- 172.21.15.76:0/3250691908 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd4b806ae70 0x7fd4b806d330 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:17.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.810+0000 7fd4dbc39640 1 --2- 172.21.15.76:0/3250691908 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4d4159e40 0x7fd4d40c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:17.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.810+0000 7fd4dbc39640 1 --2- 172.21.15.76:0/3250691908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4d406fff0 0x7fd4d40c7d50 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:17.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.810+0000 7fd4dbc39640 1 --2- 172.21.15.76:0/3250691908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4d406bb60 0x7fd4d4148fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:17.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.810+0000 7fd4dbc39640 1 -- 172.21.15.76:0/3250691908 >> 172.21.15.76:0/3250691908 conn(0x7fd4d40cb530 msgr2=0x7fd4d40b0700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:17.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.810+0000 7fd4dbc39640 1 -- 172.21.15.76:0/3250691908 shutdown_connections 2023-11-09T03:54:17.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:17.810+0000 7fd4dbc39640 1 -- 172.21.15.76:0/3250691908 wait complete. 2023-11-09T03:54:18.224 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-09T03:48:57.383848+0000', 'modified': '2023-11-09T03:54:16.621433+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 115, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 8568, 'mds_1': 6470}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6470': {'gid': 6470, 'name': 'b', 'rank': 1, 'incarnation': 121, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.113:6837/3462040782', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6836', 'nonce': 3462040782}, {'type': 'v1', 'addr': '172.21.15.113:6837', 'nonce': 3462040782}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_8568': {'gid': 8568, 'name': 'c', 'rank': 0, 'incarnation': 110, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.76:6833/153465374', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6832', 'nonce': 153465374}, {'type': 'v1', 'addr': '172.21.15.76:6833', 'nonce': 153465374}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-09T03:54:18.225 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2023-11-09T03:54:18.225 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-09T03:54:18.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.327+0000 7f80212d4640 1 Processor -- start 2023-11-09T03:54:18.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.327+0000 7f80212d4640 1 -- start start 2023-11-09T03:54:18.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.328+0000 7f80212d4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f801c0b1440 0x7f801c06c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.328+0000 7f80212d4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f801c0ad6a0 con 0x7f801c15ade0 2023-11-09T03:54:18.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.328+0000 7f80212d4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f801c0acaf0 con 0x7f801c06b740 2023-11-09T03:54:18.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.328+0000 7f80212d4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f801c0acdd0 con 0x7f801c0b1440 2023-11-09T03:54:18.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.328+0000 7f801a7fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f801c0b1440 0x7f801c06c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.328+0000 7f801affd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f801c15ade0 0x7f801c0b0d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45742/0 (socket says 172.21.15.76:45742) 2023-11-09T03:54:18.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.328+0000 7f801a7fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f801c0b1440 0x7f801c06c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57100/0 (socket says 172.21.15.76:57100) 2023-11-09T03:54:18.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.328+0000 7f801affd640 1 -- 172.21.15.76:0/3003249932 learned_addr learned my addr 172.21.15.76:0/3003249932 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:54:18.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.328+0000 7f801a7fc640 1 -- 172.21.15.76:0/3003249932 >> v1:172.21.15.113:6789/0 conn(0x7f801c06b740 legacy=0x7f801c06a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:18.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.328+0000 7f801a7fc640 1 -- 172.21.15.76:0/3003249932 >> v1:172.21.15.76:6789/0 conn(0x7f801c15ade0 legacy=0x7f801c0b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:18.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.328+0000 7f801a7fc640 1 -- 172.21.15.76:0/3003249932 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801c0ad0e0 con 0x7f801c0b1440 2023-11-09T03:54:18.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.328+0000 7f8019ffb640 1 -- 172.21.15.76:0/3003249932 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2482560284 0 0) 0x7f801c0ad6a0 con 0x7f801c15ade0 2023-11-09T03:54:18.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.329+0000 7f801a7fc640 1 --2- 172.21.15.76:0/3003249932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f801c0b1440 0x7f801c06c520 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f80100097a0 tx=0x7f801002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1017e152df697f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:18.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.329+0000 7f8019ffb640 1 -- 172.21.15.76:0/3003249932 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801002e070 con 0x7f801c0b1440 2023-11-09T03:54:18.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.329+0000 7f8019ffb640 1 -- 172.21.15.76:0/3003249932 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80100315a0 con 0x7f801c0b1440 2023-11-09T03:54:18.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.329+0000 7f8019ffb640 1 -- 172.21.15.76:0/3003249932 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8010031930 con 0x7f801c0b1440 2023-11-09T03:54:18.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.330+0000 7f80212d4640 1 -- 172.21.15.76:0/3003249932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f801c0b1440 msgr2=0x7f801c06c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:18.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.330+0000 7f80212d4640 1 --2- 172.21.15.76:0/3003249932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f801c0b1440 0x7f801c06c520 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f80100097a0 tx=0x7f801002b840 comp rx=0 tx=0).stop 2023-11-09T03:54:18.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.330+0000 7f80212d4640 1 -- 172.21.15.76:0/3003249932 shutdown_connections 2023-11-09T03:54:18.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.330+0000 7f80212d4640 1 --2- 172.21.15.76:0/3003249932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f801c0b1440 0x7f801c06c520 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.330+0000 7f80212d4640 1 -- 172.21.15.76:0/3003249932 >> 172.21.15.76:0/3003249932 conn(0x7f801c0cbd30 msgr2=0x7f801c0cc140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:18.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.330+0000 7f80212d4640 1 -- 172.21.15.76:0/3003249932 shutdown_connections 2023-11-09T03:54:18.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.330+0000 7f80212d4640 1 -- 172.21.15.76:0/3003249932 wait complete. 2023-11-09T03:54:18.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.331+0000 7f80212d4640 1 Processor -- start 2023-11-09T03:54:18.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.331+0000 7f80212d4640 1 -- start start 2023-11-09T03:54:18.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.332+0000 7f80212d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f801c06b740 0x7f801c0c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.332+0000 7f80212d4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f801c0b1440 0x7f801c0afc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.332+0000 7f80212d4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f801c15ade0 0x7f801c0b01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.332+0000 7f80212d4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c0c2db0 con 0x7f801c06b740 2023-11-09T03:54:18.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.332+0000 7f80212d4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c0c3000 con 0x7f801c15ade0 2023-11-09T03:54:18.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.332+0000 7f80212d4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f801c0c32e0 con 0x7f801c0b1440 2023-11-09T03:54:18.347 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.332+0000 7f801affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f801c06b740 0x7f801c0c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.332+0000 7f801affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f801c06b740 0x7f801c0c8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57104/0 (socket says 172.21.15.76:57104) 2023-11-09T03:54:18.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.332+0000 7f801affd640 1 -- 172.21.15.76:0/758131151 learned_addr learned my addr 172.21.15.76:0/758131151 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:18.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.332+0000 7f801b7fe640 1 --2- 172.21.15.76:0/758131151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f801c15ade0 0x7f801c0b01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.332+0000 7f801a7fc640 1 --2- 172.21.15.76:0/758131151 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f801c0b1440 0x7f801c0afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.332+0000 7f801affd640 1 -- 172.21.15.76:0/758131151 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f801c0b1440 msgr2=0x7f801c0afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:18.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.332+0000 7f801affd640 1 --2- 172.21.15.76:0/758131151 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f801c0b1440 0x7f801c0afc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.332+0000 7f801affd640 1 -- 172.21.15.76:0/758131151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f801c15ade0 msgr2=0x7f801c0b01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:18.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.332+0000 7f801affd640 1 --2- 172.21.15.76:0/758131151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f801c15ade0 0x7f801c0b01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.332+0000 7f801affd640 1 -- 172.21.15.76:0/758131151 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8010009400 con 0x7f801c06b740 2023-11-09T03:54:18.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.332+0000 7f801b7fe640 1 --2- 172.21.15.76:0/758131151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f801c15ade0 0x7f801c0b01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:54:18.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.332+0000 7f801a7fc640 1 --2- 172.21.15.76:0/758131151 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f801c0b1440 0x7f801c0afc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:54:18.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.332+0000 7f801affd640 1 --2- 172.21.15.76:0/758131151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f801c06b740 0x7f801c0c8c30 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f800c00c410 tx=0x7f800c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:18.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.333+0000 7f80037fe640 1 -- 172.21.15.76:0/758131151 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c0074e0 con 0x7f801c06b740 2023-11-09T03:54:18.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.333+0000 7f80037fe640 1 -- 172.21.15.76:0/758131151 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800c0057a0 con 0x7f801c06b740 2023-11-09T03:54:18.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.333+0000 7f80212d4640 1 -- 172.21.15.76:0/758131151 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f801c0c35c0 con 0x7f801c06b740 2023-11-09T03:54:18.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.333+0000 7f80037fe640 1 -- 172.21.15.76:0/758131151 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c007660 con 0x7f801c06b740 2023-11-09T03:54:18.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.333+0000 7f80212d4640 1 -- 172.21.15.76:0/758131151 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f801c0c3b80 con 0x7f801c06b740 2023-11-09T03:54:18.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.334+0000 7f80037fe640 1 -- 172.21.15.76:0/758131151 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f800c005bd0 con 0x7f801c06b740 2023-11-09T03:54:18.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.334+0000 7f80037fe640 1 --2- 172.21.15.76:0/758131151 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f800806ae70 0x7f800806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.335+0000 7f80037fe640 1 -- 172.21.15.76:0/758131151 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7f800c0896c0 con 0x7f801c06b740 2023-11-09T03:54:18.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.335+0000 7f801a7fc640 1 --2- 172.21.15.76:0/758131151 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f800806ae70 0x7f800806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.335+0000 7f801a7fc640 1 --2- 172.21.15.76:0/758131151 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f800806ae70 0x7f800806d330 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f8010036000 tx=0x7f801003d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:18.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.335+0000 7f8023e60740 1 Processor -- start 2023-11-09T03:54:18.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.336+0000 7f8023e60740 1 -- start start 2023-11-09T03:54:18.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.336+0000 7f8023e60740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c46e3e6850 0x55c46e3b9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.336+0000 7f8023e60740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c46e3e6c30 0x55c46e3e95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.336+0000 7f8023e60740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c46e3b9c90 0x55c46e3e9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.336+0000 7f8023e60740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46e2ca740 con 0x55c46e3b9c90 2023-11-09T03:54:18.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.336+0000 7f8023e60740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46e32c8d0 con 0x55c46e3e6c30 2023-11-09T03:54:18.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.336+0000 7f8023e60740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c46e329fc0 con 0x55c46e3e6850 2023-11-09T03:54:18.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.336+0000 7f801b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c46e3b9c90 0x55c46e3e9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.337+0000 7f801b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c46e3b9c90 0x55c46e3e9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57118/0 (socket says 172.21.15.76:57118) 2023-11-09T03:54:18.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.337+0000 7f801b7fe640 1 -- 172.21.15.76:0/2576435580 learned_addr learned my addr 172.21.15.76:0/2576435580 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:18.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.337+0000 7f801affd640 1 --2- 172.21.15.76:0/2576435580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c46e3e6850 0x55c46e3b9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.337+0000 7f801a7fc640 1 --2- 172.21.15.76:0/2576435580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c46e3e6c30 0x55c46e3e95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.337+0000 7f801b7fe640 1 -- 172.21.15.76:0/2576435580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c46e3e6850 msgr2=0x55c46e3b9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:18.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.337+0000 7f801b7fe640 1 --2- 172.21.15.76:0/2576435580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c46e3e6850 0x55c46e3b9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.337+0000 7f801b7fe640 1 -- 172.21.15.76:0/2576435580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c46e3e6c30 msgr2=0x55c46e3e95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:18.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.337+0000 7f801b7fe640 1 --2- 172.21.15.76:0/2576435580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c46e3e6c30 0x55c46e3e95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.337+0000 7f801b7fe640 1 -- 172.21.15.76:0/2576435580 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c46e2e8ca0 con 0x55c46e3b9c90 2023-11-09T03:54:18.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.337+0000 7f801b7fe640 1 --2- 172.21.15.76:0/2576435580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c46e3b9c90 0x55c46e3e9b20 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f80140045d0 tx=0x7f801400d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7682dfcd3a5c9bb6 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:18.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.337+0000 7f8000ff9640 1 -- 172.21.15.76:0/2576435580 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8014006d50 con 0x55c46e3b9c90 2023-11-09T03:54:18.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.337+0000 7f8000ff9640 1 -- 172.21.15.76:0/2576435580 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801400de40 con 0x55c46e3b9c90 2023-11-09T03:54:18.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.337+0000 7f8000ff9640 1 -- 172.21.15.76:0/2576435580 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8014004a20 con 0x55c46e3b9c90 2023-11-09T03:54:18.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.338+0000 7f8023e60740 1 -- 172.21.15.76:0/2576435580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c46e3b9c90 msgr2=0x55c46e3e9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:18.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.338+0000 7f8023e60740 1 --2- 172.21.15.76:0/2576435580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c46e3b9c90 0x55c46e3e9b20 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f80140045d0 tx=0x7f801400d520 comp rx=0 tx=0).stop 2023-11-09T03:54:18.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.338+0000 7f8023e60740 1 -- 172.21.15.76:0/2576435580 shutdown_connections 2023-11-09T03:54:18.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.338+0000 7f8023e60740 1 --2- 172.21.15.76:0/2576435580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c46e3b9c90 0x55c46e3e9b20 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.338+0000 7f8023e60740 1 --2- 172.21.15.76:0/2576435580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c46e3e6c30 0x55c46e3e95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.338+0000 7f8023e60740 1 --2- 172.21.15.76:0/2576435580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c46e3e6850 0x55c46e3b9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.338+0000 7f8023e60740 1 -- 172.21.15.76:0/2576435580 >> 172.21.15.76:0/2576435580 conn(0x55c46e3bba60 msgr2=0x55c46e4045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:18.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.338+0000 7f8023e60740 1 -- 172.21.15.76:0/2576435580 shutdown_connections 2023-11-09T03:54:18.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.338+0000 7f8023e60740 1 -- 172.21.15.76:0/2576435580 wait complete. 2023-11-09T03:54:18.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.338+0000 7f8023e60740 1 -- start start 2023-11-09T03:54:18.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.338+0000 7f8023e60740 1 Processor -- start 2023-11-09T03:54:18.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.339+0000 7f8023e60740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c46e3e6850 0x55c46e3fba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.339+0000 7f8023e60740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c46e3e6c30 0x55c46e404b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.339+0000 7f8023e60740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c46e3eb280 0x55c46e3eca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.339+0000 7f8023e60740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46e3258b0 con 0x55c46e3e6850 2023-11-09T03:54:18.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.339+0000 7f8023e60740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46e2dfc80 con 0x55c46e3e6c30 2023-11-09T03:54:18.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.339+0000 7f8023e60740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c46e2df920 con 0x55c46e3eb280 2023-11-09T03:54:18.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.339+0000 7f8023e60740 10 client.0 _ll_register_callbacks cb 0x55c46e037ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:54:18.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.339+0000 7f801affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c46e3e6850 0x55c46e3fba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.339+0000 7f801affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c46e3e6850 0x55c46e3fba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57126/0 (socket says 172.21.15.76:57126) 2023-11-09T03:54:18.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.339+0000 7f801affd640 1 -- 172.21.15.76:0/3164066844 learned_addr learned my addr 172.21.15.76:0/3164066844 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:18.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.339+0000 7f801a7fc640 1 --2- 172.21.15.76:0/3164066844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c46e3e6c30 0x55c46e404b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.340+0000 7f801b7fe640 1 --2- 172.21.15.76:0/3164066844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c46e3eb280 0x55c46e3eca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.340+0000 7f801affd640 1 -- 172.21.15.76:0/3164066844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c46e3eb280 msgr2=0x55c46e3eca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:18.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.340+0000 7f801affd640 1 --2- 172.21.15.76:0/3164066844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c46e3eb280 0x55c46e3eca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.340+0000 7f801affd640 1 -- 172.21.15.76:0/3164066844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c46e3e6c30 msgr2=0x55c46e404b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:18.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.340+0000 7f801affd640 1 --2- 172.21.15.76:0/3164066844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c46e3e6c30 0x55c46e404b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.340+0000 7f801affd640 1 -- 172.21.15.76:0/3164066844 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80140041d0 con 0x55c46e3e6850 2023-11-09T03:54:18.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.340+0000 7f801a7fc640 1 --2- 172.21.15.76:0/3164066844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c46e3e6c30 0x55c46e404b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:54:18.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.340+0000 7f801affd640 1 --2- 172.21.15.76:0/3164066844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c46e3e6850 0x55c46e3fba00 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f800c00cdd0 tx=0x7f800c05bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0170afeb15f669 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:18.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.340+0000 7f7ff9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:54:18.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.340+0000 7f7ff9ffb640 1 -- 172.21.15.76:0/3164066844 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c06f170 con 0x55c46e3e6850 2023-11-09T03:54:18.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.340+0000 7f7ff9ffb640 1 -- 172.21.15.76:0/3164066844 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7f800c0537b0 con 0x55c46e3e6850 2023-11-09T03:54:18.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.340+0000 7f8023e60740 1 -- 172.21.15.76:0/3164066844 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c46e2d1bf0 con 0x55c46e3e6850 2023-11-09T03:54:18.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.340+0000 7f7ff9ffb640 1 -- 172.21.15.76:0/3164066844 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f800c06f2d0 con 0x55c46e3e6850 2023-11-09T03:54:18.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.340+0000 7f8023e60740 10 client.9078 fetch_fsmap learned FSMap version 125 2023-11-09T03:54:18.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.340+0000 7f8023e60740 1 -- 172.21.15.76:0/3164066844 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x55c46e323f30 con 0x55c46e3e6850 2023-11-09T03:54:18.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.340+0000 7f7ff9ffb640 1 -- 172.21.15.76:0/3164066844 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800c073020 con 0x55c46e3e6850 2023-11-09T03:54:18.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.340+0000 7f7ff9ffb640 1 -- 172.21.15.76:0/3164066844 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 125) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f800c053970 con 0x55c46e3e6850 2023-11-09T03:54:18.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.340+0000 7f8023e60740 10 client.9078 fetch_fsmap finished waiting for FSMap version 125 2023-11-09T03:54:18.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.340+0000 7f8023e60740 10 client.9078 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-09T03:54:18.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.340+0000 7f8023e60740 20 client.9078 populate_metadata read hostname 'smithi076' 2023-11-09T03:54:18.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.340+0000 7f8023e60740 1 --2- 172.21.15.76:0/3164066844 >> [v2:172.21.15.113:6836/3462040782,v1:172.21.15.113:6837/3462040782] conn(0x55c46e3e8740 0x55c46e421ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.340+0000 7f8023e60740 4 client.9078 mds_command: new command op to 6470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:54:18.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.340+0000 7f8023e60740 1 -- 172.21.15.76:0/3164066844 --> [v2:172.21.15.113:6836/3462040782,v1:172.21.15.113:6837/3462040782] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c46e31e890 con 0x55c46e3e8740 2023-11-09T03:54:18.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.341+0000 7f801a7fc640 1 --2- 172.21.15.76:0/3164066844 >> [v2:172.21.15.113:6836/3462040782,v1:172.21.15.113:6837/3462040782] conn(0x55c46e3e8740 0x55c46e421ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.341+0000 7f801a7fc640 1 --2- 172.21.15.76:0/3164066844 >> [v2:172.21.15.113:6836/3462040782,v1:172.21.15.113:6837/3462040782] conn(0x55c46e3e8740 0x55c46e421ae0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=40f0582dc29a509 server_cookie=490a90d6852cf306 in_seq=0 out_seq=0 2023-11-09T03:54:18.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.341+0000 7f7ff9ffb640 10 client.9078 ms_handle_connect on v2:172.21.15.113:6836/3462040782 2023-11-09T03:54:18.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.342+0000 7f7ff9ffb640 1 -- 172.21.15.76:0/3164066844 <== mds.1 v2:172.21.15.113:6836/3462040782 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c46e31e890 con 0x55c46e3e8740 2023-11-09T03:54:18.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.342+0000 7f7ff9ffb640 10 client.9078 handle_command_reply: tid=0 2023-11-09T03:54:18.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.342+0000 7f8023e60740 1 client.9078 shutdown 2023-11-09T03:54:18.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.342+0000 7f8023e60740 1 -- 172.21.15.76:0/3164066844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c46e3e6850 msgr2=0x55c46e3fba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:18.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.342+0000 7f8023e60740 1 --2- 172.21.15.76:0/3164066844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c46e3e6850 0x55c46e3fba00 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f800c00cdd0 tx=0x7f800c05bbd0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.342+0000 7f8023e60740 1 -- 172.21.15.76:0/3164066844 shutdown_connections 2023-11-09T03:54:18.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.342+0000 7f8023e60740 1 --2- 172.21.15.76:0/3164066844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c46e3eb280 0x55c46e3eca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.342+0000 7f8023e60740 1 --2- 172.21.15.76:0/3164066844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c46e3e6c30 0x55c46e404b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.342+0000 7f8023e60740 1 --2- 172.21.15.76:0/3164066844 >> [v2:172.21.15.113:6836/3462040782,v1:172.21.15.113:6837/3462040782] conn(0x55c46e3e8740 0x55c46e421ae0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.342+0000 7f8023e60740 1 --2- 172.21.15.76:0/3164066844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c46e3e6850 0x55c46e3fba00 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.342+0000 7f7ff9ffb640 0 client.9078 ms_handle_reset on v2:172.21.15.113:6836/3462040782 2023-11-09T03:54:18.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.342+0000 7f8023e60740 1 -- 172.21.15.76:0/3164066844 >> 172.21.15.76:0/3164066844 conn(0x55c46e3b9c90 msgr2=0x55c46e408380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:18.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.342+0000 7f8023e60740 1 -- 172.21.15.76:0/3164066844 shutdown_connections 2023-11-09T03:54:18.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.342+0000 7f8023e60740 1 -- 172.21.15.76:0/3164066844 wait complete. 2023-11-09T03:54:18.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.342+0000 7f8023e60740 20 client.9078 trim_cache size 0 max 16384 2023-11-09T03:54:18.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.357+0000 7f8023e60740 1 Processor -- start 2023-11-09T03:54:18.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.357+0000 7f8023e60740 1 -- start start 2023-11-09T03:54:18.377 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.358+0000 7f8023e60740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c46e3e6850 0x55c46e3fc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.358+0000 7f8023e60740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c46e3e6c30 0x55c46e3bd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.358+0000 7f8023e60740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c46e3eb280 0x55c46e407af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.358+0000 7f8023e60740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46e2a5280 con 0x55c46e3e6c30 2023-11-09T03:54:18.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.358+0000 7f8023e60740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46e4035d0 con 0x55c46e3eb280 2023-11-09T03:54:18.378 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.358+0000 7f8023e60740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c46e3d3bd0 con 0x55c46e3e6850 2023-11-09T03:54:18.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.358+0000 7f801a7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c46e3e6c30 0x55c46e3bd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.358+0000 7f801a7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c46e3e6c30 0x55c46e3bd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57128/0 (socket says 172.21.15.76:57128) 2023-11-09T03:54:18.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.358+0000 7f801a7fc640 1 -- 172.21.15.76:0/555921479 learned_addr learned my addr 172.21.15.76:0/555921479 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:18.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.358+0000 7f801a7fc640 1 -- 172.21.15.76:0/555921479 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c46e3e6850 msgr2=0x55c46e3fc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:18.379 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.358+0000 7f801a7fc640 1 --2- 172.21.15.76:0/555921479 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c46e3e6850 0x55c46e3fc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.358+0000 7f801a7fc640 1 -- 172.21.15.76:0/555921479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c46e3eb280 msgr2=0x55c46e407af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:18.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.358+0000 7f801a7fc640 1 --2- 172.21.15.76:0/555921479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c46e3eb280 0x55c46e407af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.358+0000 7f801a7fc640 1 -- 172.21.15.76:0/555921479 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c46e3b9b00 con 0x55c46e3e6c30 2023-11-09T03:54:18.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.359+0000 7f801a7fc640 1 --2- 172.21.15.76:0/555921479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c46e3e6c30 0x55c46e3bd540 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f8010000c00 tx=0x7f8010030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec635d29ad68ef1b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:18.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.359+0000 7f80017fa640 1 -- 172.21.15.76:0/555921479 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8010032cd0 con 0x55c46e3e6c30 2023-11-09T03:54:18.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.359+0000 7f80017fa640 1 -- 172.21.15.76:0/555921479 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801004c040 con 0x55c46e3e6c30 2023-11-09T03:54:18.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.359+0000 7f80017fa640 1 -- 172.21.15.76:0/555921479 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8010032e30 con 0x55c46e3e6c30 2023-11-09T03:54:18.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.359+0000 7f8023e60740 1 -- 172.21.15.76:0/555921479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c46e3e6c30 msgr2=0x55c46e3bd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:18.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.359+0000 7f8023e60740 1 --2- 172.21.15.76:0/555921479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c46e3e6c30 0x55c46e3bd540 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f8010000c00 tx=0x7f8010030000 comp rx=0 tx=0).stop 2023-11-09T03:54:18.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.359+0000 7f8023e60740 1 -- 172.21.15.76:0/555921479 shutdown_connections 2023-11-09T03:54:18.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.359+0000 7f8023e60740 1 --2- 172.21.15.76:0/555921479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c46e3eb280 0x55c46e407af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.359+0000 7f8023e60740 1 --2- 172.21.15.76:0/555921479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c46e3e6c30 0x55c46e3bd540 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.359+0000 7f8023e60740 1 --2- 172.21.15.76:0/555921479 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c46e3e6850 0x55c46e3fc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.359+0000 7f8023e60740 1 -- 172.21.15.76:0/555921479 >> 172.21.15.76:0/555921479 conn(0x55c46e3b9c90 msgr2=0x55c46e3b4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:18.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.359+0000 7f8023e60740 1 -- 172.21.15.76:0/555921479 shutdown_connections 2023-11-09T03:54:18.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.359+0000 7f8023e60740 1 -- 172.21.15.76:0/555921479 wait complete. 2023-11-09T03:54:18.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.360+0000 7f8023e60740 1 -- start start 2023-11-09T03:54:18.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.360+0000 7f8023e60740 1 Processor -- start 2023-11-09T03:54:18.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.361+0000 7f8023e60740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c46e3e6850 0x55c46e3fba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.361+0000 7f8023e60740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c46e3e6c30 0x55c46e3e9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.361+0000 7f8023e60740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c46e3eb280 0x55c46e46c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.361+0000 7f8023e60740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46e413b00 con 0x55c46e3eb280 2023-11-09T03:54:18.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.361+0000 7f8023e60740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46e416320 con 0x55c46e3e6c30 2023-11-09T03:54:18.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.361+0000 7f8023e60740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c46e3fbfa0 con 0x55c46e3e6850 2023-11-09T03:54:18.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.361+0000 7f8023e60740 10 client.0 _ll_register_callbacks cb 0x7f8014003d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:54:18.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.361+0000 7f801b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c46e3eb280 0x55c46e46c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.361+0000 7f801b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c46e3eb280 0x55c46e46c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57140/0 (socket says 172.21.15.76:57140) 2023-11-09T03:54:18.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.361+0000 7f801b7fe640 1 -- 172.21.15.76:0/3219784688 learned_addr learned my addr 172.21.15.76:0/3219784688 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:18.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.361+0000 7f801affd640 1 --2- 172.21.15.76:0/3219784688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c46e3e6850 0x55c46e3fba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.361+0000 7f801a7fc640 1 --2- 172.21.15.76:0/3219784688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c46e3e6c30 0x55c46e3e9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.386 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.361+0000 7f801b7fe640 1 -- 172.21.15.76:0/3219784688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c46e3e6850 msgr2=0x55c46e3fba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:18.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.361+0000 7f801b7fe640 1 --2- 172.21.15.76:0/3219784688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c46e3e6850 0x55c46e3fba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.361+0000 7f801b7fe640 1 -- 172.21.15.76:0/3219784688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c46e3e6c30 msgr2=0x55c46e3e9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:18.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.361+0000 7f801b7fe640 1 --2- 172.21.15.76:0/3219784688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c46e3e6c30 0x55c46e3e9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.387 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.361+0000 7f801b7fe640 1 -- 172.21.15.76:0/3219784688 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f801c0ad0e0 con 0x55c46e3eb280 2023-11-09T03:54:18.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.361+0000 7f801b7fe640 1 --2- 172.21.15.76:0/3219784688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c46e3eb280 0x55c46e46c9f0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f8014004570 tx=0x7f8014003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a8f4252fc72e1d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:18.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.361+0000 7f7ff9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:54:18.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.361+0000 7f7ff9ffb640 1 -- 172.21.15.76:0/3219784688 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8014003e60 con 0x55c46e3eb280 2023-11-09T03:54:18.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.361+0000 7f7ff9ffb640 1 -- 172.21.15.76:0/3219784688 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7f8014005040 con 0x55c46e3eb280 2023-11-09T03:54:18.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.361+0000 7f8023e60740 1 -- 172.21.15.76:0/3219784688 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f801400de40 con 0x55c46e3eb280 2023-11-09T03:54:18.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.361+0000 7f7ff9ffb640 1 -- 172.21.15.76:0/3219784688 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8014003be0 con 0x55c46e3eb280 2023-11-09T03:54:18.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.361+0000 7f8023e60740 10 client.9084 fetch_fsmap learned FSMap version 125 2023-11-09T03:54:18.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.361+0000 7f8023e60740 1 -- 172.21.15.76:0/3219784688 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x55c46e2c1d10 con 0x55c46e3eb280 2023-11-09T03:54:18.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.361+0000 7f7ff9ffb640 1 -- 172.21.15.76:0/3219784688 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801400de40 con 0x55c46e3eb280 2023-11-09T03:54:18.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.362+0000 7f7ff9ffb640 1 -- 172.21.15.76:0/3219784688 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 125) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f8014019070 con 0x55c46e3eb280 2023-11-09T03:54:18.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.362+0000 7f8023e60740 10 client.9084 fetch_fsmap finished waiting for FSMap version 125 2023-11-09T03:54:18.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.362+0000 7f8023e60740 10 client.9084 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-09T03:54:18.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.362+0000 7f8023e60740 20 client.9084 populate_metadata read hostname 'smithi076' 2023-11-09T03:54:18.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.362+0000 7f8023e60740 1 --2- 172.21.15.76:0/3219784688 >> [v2:172.21.15.113:6836/3462040782,v1:172.21.15.113:6837/3462040782] conn(0x55c46e41de50 0x55c46e41e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.362+0000 7f8023e60740 4 client.9084 mds_command: new command op to 6470 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-09T03:54:18.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.362+0000 7f8023e60740 1 -- 172.21.15.76:0/3219784688 --> [v2:172.21.15.113:6836/3462040782,v1:172.21.15.113:6837/3462040782] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c46e31e5b0 con 0x55c46e41de50 2023-11-09T03:54:18.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.362+0000 7f801affd640 1 --2- 172.21.15.76:0/3219784688 >> [v2:172.21.15.113:6836/3462040782,v1:172.21.15.113:6837/3462040782] conn(0x55c46e41de50 0x55c46e41e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.362+0000 7f7ff9ffb640 10 client.9084 ms_handle_connect on v2:172.21.15.113:6836/3462040782 2023-11-09T03:54:18.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.362+0000 7f801affd640 1 --2- 172.21.15.76:0/3219784688 >> [v2:172.21.15.113:6836/3462040782,v1:172.21.15.113:6837/3462040782] conn(0x55c46e41de50 0x55c46e41e230 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=53dad4bd381fd3e4 server_cookie=753a79e0cf471e88 in_seq=0 out_seq=0 2023-11-09T03:54:18.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.362+0000 7f7ff9ffb640 1 -- 172.21.15.76:0/3219784688 <== mds.1 v2:172.21.15.113:6836/3462040782 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x55c46e31e5b0 con 0x55c46e41de50 2023-11-09T03:54:18.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.362+0000 7f7ff9ffb640 10 client.9084 handle_command_reply: tid=0 2023-11-09T03:54:18.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.362+0000 7f8023e60740 1 client.9084 shutdown 2023-11-09T03:54:18.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.363+0000 7f8023e60740 1 -- 172.21.15.76:0/3219784688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c46e3eb280 msgr2=0x55c46e46c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:18.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.363+0000 7f8023e60740 1 --2- 172.21.15.76:0/3219784688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c46e3eb280 0x55c46e46c9f0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f8014004570 tx=0x7f8014003ae0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.363+0000 7f8023e60740 1 -- 172.21.15.76:0/3219784688 shutdown_connections 2023-11-09T03:54:18.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.363+0000 7f8023e60740 1 --2- 172.21.15.76:0/3219784688 >> [v2:172.21.15.113:6836/3462040782,v1:172.21.15.113:6837/3462040782] conn(0x55c46e41de50 0x55c46e41e230 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.363+0000 7f8023e60740 1 --2- 172.21.15.76:0/3219784688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c46e3eb280 0x55c46e46c9f0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.363+0000 7f8023e60740 1 --2- 172.21.15.76:0/3219784688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c46e3e6c30 0x55c46e3e9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.363+0000 7f7ff9ffb640 0 client.9084 ms_handle_reset on v2:172.21.15.113:6836/3462040782 2023-11-09T03:54:18.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.363+0000 7f8023e60740 1 --2- 172.21.15.76:0/3219784688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c46e3e6850 0x55c46e3fba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.363+0000 7f8023e60740 1 -- 172.21.15.76:0/3219784688 >> 172.21.15.76:0/3219784688 conn(0x55c46e3b9c90 msgr2=0x55c46e3b4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:18.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.363+0000 7f8023e60740 1 -- 172.21.15.76:0/3219784688 shutdown_connections 2023-11-09T03:54:18.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.363+0000 7f8023e60740 1 -- 172.21.15.76:0/3219784688 wait complete. 2023-11-09T03:54:18.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.363+0000 7f8023e60740 20 client.9084 trim_cache size 0 max 16384 2023-11-09T03:54:18.395 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:54:18.395 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "c213b26f-d0da-45d5-8e34-bb75c096d4f2", 2023-11-09T03:54:18.395 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 1, 2023-11-09T03:54:18.395 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 6470, 2023-11-09T03:54:18.396 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2023-11-09T03:54:18.396 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2023-11-09T03:54:18.396 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2023-11-09T03:54:18.396 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 11.783743732, 2023-11-09T03:54:18.396 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 125, 2023-11-09T03:54:18.397 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 116, 2023-11-09T03:54:18.397 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 116, 2023-11-09T03:54:18.397 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 53.544103954999997 2023-11-09T03:54:18.397 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:54:18.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.363+0000 7f80212d4640 1 -- 172.21.15.76:0/758131151 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f800806ae70 msgr2=0x7f800806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:18.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.363+0000 7f80212d4640 1 --2- 172.21.15.76:0/758131151 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f800806ae70 0x7f800806d330 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f8010036000 tx=0x7f801003d040 comp rx=0 tx=0).stop 2023-11-09T03:54:18.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.363+0000 7f80212d4640 1 -- 172.21.15.76:0/758131151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f801c06b740 msgr2=0x7f801c0c8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:18.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.363+0000 7f80212d4640 1 --2- 172.21.15.76:0/758131151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f801c06b740 0x7f801c0c8c30 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f800c00c410 tx=0x7f800c00c8d0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.363+0000 7f80212d4640 1 -- 172.21.15.76:0/758131151 shutdown_connections 2023-11-09T03:54:18.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.363+0000 7f80212d4640 1 --2- 172.21.15.76:0/758131151 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f800806ae70 0x7f800806d330 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.363+0000 7f80212d4640 1 --2- 172.21.15.76:0/758131151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f801c15ade0 0x7f801c0b01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.363+0000 7f80212d4640 1 --2- 172.21.15.76:0/758131151 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f801c0b1440 0x7f801c0afc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.363+0000 7f80212d4640 1 --2- 172.21.15.76:0/758131151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f801c06b740 0x7f801c0c8c30 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.363+0000 7f80212d4640 1 -- 172.21.15.76:0/758131151 >> 172.21.15.76:0/758131151 conn(0x7f801c0cbd30 msgr2=0x7f801c1583d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:18.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.363+0000 7f80212d4640 1 -- 172.21.15.76:0/758131151 shutdown_connections 2023-11-09T03:54:18.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.363+0000 7f80212d4640 1 -- 172.21.15.76:0/758131151 wait complete. 2023-11-09T03:54:18.401 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-09T03:54:18.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.502+0000 7fc8e894c640 1 Processor -- start 2023-11-09T03:54:18.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.502+0000 7fc8e894c640 1 -- start start 2023-11-09T03:54:18.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.502+0000 7fc8e894c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8e406d9c0 0x7fc8e40710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.503+0000 7fc8e894c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e406cc90 con 0x7fc8e406c5f0 2023-11-09T03:54:18.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.503+0000 7fc8e894c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e40b0e60 con 0x7fc8e40ade00 2023-11-09T03:54:18.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.503+0000 7fc8e894c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e40b0c40 con 0x7fc8e406d9c0 2023-11-09T03:54:18.505 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.503+0000 7fc8e2d9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc8e406c5f0 0x7fc8e40cc680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45756/0 (socket says 172.21.15.76:45756) 2023-11-09T03:54:18.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.503+0000 7fc8e2d9d640 1 -- 172.21.15.76:0/2292180383 learned_addr learned my addr 172.21.15.76:0/2292180383 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:54:18.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.503+0000 7fc8e1d9b640 1 --2- 172.21.15.76:0/2292180383 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8e406d9c0 0x7fc8e40710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.503+0000 7fc8e159a640 1 -- 172.21.15.76:0/2292180383 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 499547682 0 0) 0x7fc8e406cc90 con 0x7fc8e406c5f0 2023-11-09T03:54:18.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.503+0000 7fc8e159a640 1 -- 172.21.15.76:0/2292180383 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8cc003520 con 0x7fc8e406c5f0 2023-11-09T03:54:18.506 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.503+0000 7fc8e159a640 1 -- 172.21.15.76:0/2292180383 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1819618929 0 0) 0x7fc8e40b0e60 con 0x7fc8e40ade00 2023-11-09T03:54:18.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.503+0000 7fc8e159a640 1 -- 172.21.15.76:0/2292180383 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8e406cc90 con 0x7fc8e40ade00 2023-11-09T03:54:18.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.503+0000 7fc8e1d9b640 1 -- 172.21.15.76:0/2292180383 >> v1:172.21.15.113:6789/0 conn(0x7fc8e40ade00 legacy=0x7fc8e40b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:18.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.503+0000 7fc8e1d9b640 1 -- 172.21.15.76:0/2292180383 >> v1:172.21.15.76:6789/0 conn(0x7fc8e406c5f0 legacy=0x7fc8e40cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:18.507 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.503+0000 7fc8e1d9b640 1 -- 172.21.15.76:0/2292180383 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e406cee0 con 0x7fc8e406d9c0 2023-11-09T03:54:18.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.504+0000 7fc8e1d9b640 1 --2- 172.21.15.76:0/2292180383 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8e406d9c0 0x7fc8e40710d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40097d0 tx=0x7fc8d402b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a31fee50fe310aee server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:18.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.504+0000 7fc8e159a640 1 -- 172.21.15.76:0/2292180383 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4031770 con 0x7fc8e406d9c0 2023-11-09T03:54:18.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.504+0000 7fc8e159a640 1 -- 172.21.15.76:0/2292180383 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d40318d0 con 0x7fc8e406d9c0 2023-11-09T03:54:18.508 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.504+0000 7fc8e159a640 1 -- 172.21.15.76:0/2292180383 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4031c60 con 0x7fc8e406d9c0 2023-11-09T03:54:18.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.505+0000 7fc8e894c640 1 -- 172.21.15.76:0/2292180383 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8e406d9c0 msgr2=0x7fc8e40710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:18.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.505+0000 7fc8e894c640 1 --2- 172.21.15.76:0/2292180383 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8e406d9c0 0x7fc8e40710d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40097d0 tx=0x7fc8d402b810 comp rx=0 tx=0).stop 2023-11-09T03:54:18.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.505+0000 7fc8e894c640 1 -- 172.21.15.76:0/2292180383 shutdown_connections 2023-11-09T03:54:18.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.505+0000 7fc8e894c640 1 --2- 172.21.15.76:0/2292180383 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8e406d9c0 0x7fc8e40710d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.505+0000 7fc8e894c640 1 -- 172.21.15.76:0/2292180383 >> 172.21.15.76:0/2292180383 conn(0x7fc8e40cb530 msgr2=0x7fc8e40cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:18.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.505+0000 7fc8e894c640 1 -- 172.21.15.76:0/2292180383 shutdown_connections 2023-11-09T03:54:18.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.505+0000 7fc8e894c640 1 -- 172.21.15.76:0/2292180383 wait complete. 2023-11-09T03:54:18.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.506+0000 7fc8e894c640 1 Processor -- start 2023-11-09T03:54:18.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.506+0000 7fc8e894c640 1 -- start start 2023-11-09T03:54:18.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.507+0000 7fc8e894c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8e406c5f0 0x7fc8e414f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.507+0000 7fc8e894c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8e406d9c0 0x7fc8e4152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.507+0000 7fc8e894c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e40ade00 0x7fc8e40c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.507+0000 7fc8e894c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e4150b60 con 0x7fc8e406c5f0 2023-11-09T03:54:18.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.507+0000 7fc8e894c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e4152b70 con 0x7fc8e40ade00 2023-11-09T03:54:18.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.507+0000 7fc8e894c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e414ff50 con 0x7fc8e406d9c0 2023-11-09T03:54:18.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.507+0000 7fc8e259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8e406c5f0 0x7fc8e414f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.507+0000 7fc8e259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8e406c5f0 0x7fc8e414f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57146/0 (socket says 172.21.15.76:57146) 2023-11-09T03:54:18.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.507+0000 7fc8e259c640 1 -- 172.21.15.76:0/579383460 learned_addr learned my addr 172.21.15.76:0/579383460 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:18.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.507+0000 7fc8e2d9d640 1 --2- 172.21.15.76:0/579383460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e40ade00 0x7fc8e40c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.507+0000 7fc8e1d9b640 1 --2- 172.21.15.76:0/579383460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8e406d9c0 0x7fc8e4152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.507+0000 7fc8e259c640 1 -- 172.21.15.76:0/579383460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8e406d9c0 msgr2=0x7fc8e4152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:18.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.507+0000 7fc8e259c640 1 --2- 172.21.15.76:0/579383460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8e406d9c0 0x7fc8e4152630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.507+0000 7fc8e259c640 1 -- 172.21.15.76:0/579383460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e40ade00 msgr2=0x7fc8e40c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:18.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.507+0000 7fc8e259c640 1 --2- 172.21.15.76:0/579383460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e40ade00 0x7fc8e40c61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.507+0000 7fc8e259c640 1 -- 172.21.15.76:0/579383460 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d4009400 con 0x7fc8e406c5f0 2023-11-09T03:54:18.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.507+0000 7fc8e1d9b640 1 --2- 172.21.15.76:0/579383460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8e406d9c0 0x7fc8e4152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:54:18.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.507+0000 7fc8e2d9d640 1 --2- 172.21.15.76:0/579383460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e40ade00 0x7fc8e40c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:54:18.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.507+0000 7fc8e259c640 1 --2- 172.21.15.76:0/579383460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8e406c5f0 0x7fc8e414f5a0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800c430 tx=0x7fc8d800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:18.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.507+0000 7fc8d2ffd640 1 -- 172.21.15.76:0/579383460 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d80052d0 con 0x7fc8e406c5f0 2023-11-09T03:54:18.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.507+0000 7fc8d2ffd640 1 -- 172.21.15.76:0/579383460 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d8005450 con 0x7fc8e406c5f0 2023-11-09T03:54:18.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.508+0000 7fc8e894c640 1 -- 172.21.15.76:0/579383460 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e4150230 con 0x7fc8e406c5f0 2023-11-09T03:54:18.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.508+0000 7fc8d2ffd640 1 -- 172.21.15.76:0/579383460 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8003830 con 0x7fc8e406c5f0 2023-11-09T03:54:18.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.508+0000 7fc8e894c640 1 -- 172.21.15.76:0/579383460 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e41507f0 con 0x7fc8e406c5f0 2023-11-09T03:54:18.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.509+0000 7fc8d2ffd640 1 -- 172.21.15.76:0/579383460 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc8d8003990 con 0x7fc8e406c5f0 2023-11-09T03:54:18.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.509+0000 7fc8d2ffd640 1 --2- 172.21.15.76:0/579383460 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc8c406ae70 0x7fc8c406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.509+0000 7fc8d2ffd640 1 -- 172.21.15.76:0/579383460 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7fc8d8089740 con 0x7fc8e406c5f0 2023-11-09T03:54:18.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.510+0000 7fc8e1d9b640 1 --2- 172.21.15.76:0/579383460 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc8c406ae70 0x7fc8c406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.510+0000 7fc8e1d9b640 1 --2- 172.21.15.76:0/579383460 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc8c406ae70 0x7fc8c406d330 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4036000 tx=0x7fc8d403d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:18.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.510+0000 7fc8eb5b9740 1 Processor -- start 2023-11-09T03:54:18.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.511+0000 7fc8eb5b9740 1 -- start start 2023-11-09T03:54:18.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.511+0000 7fc8eb5b9740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f541ae6850 0x55f541ab9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.511+0000 7fc8eb5b9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f541ae6c30 0x55f541ae95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.511+0000 7fc8eb5b9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f541ab9c90 0x55f541ae9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.527 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.511+0000 7fc8eb5b9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5419ca740 con 0x55f541ab9c90 2023-11-09T03:54:18.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.511+0000 7fc8eb5b9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f541a2c8d0 con 0x55f541ae6c30 2023-11-09T03:54:18.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.511+0000 7fc8eb5b9740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f541a29fc0 con 0x55f541ae6850 2023-11-09T03:54:18.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.511+0000 7fc8e2d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f541ab9c90 0x55f541ae9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.511+0000 7fc8e2d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f541ab9c90 0x55f541ae9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57154/0 (socket says 172.21.15.76:57154) 2023-11-09T03:54:18.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.511+0000 7fc8e2d9d640 1 -- 172.21.15.76:0/3676840991 learned_addr learned my addr 172.21.15.76:0/3676840991 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:18.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.511+0000 7fc8e259c640 1 --2- 172.21.15.76:0/3676840991 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f541ae6850 0x55f541ab9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.511+0000 7fc8e1d9b640 1 --2- 172.21.15.76:0/3676840991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f541ae6c30 0x55f541ae95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.512+0000 7fc8e2d9d640 1 -- 172.21.15.76:0/3676840991 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f541ae6850 msgr2=0x55f541ab9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:18.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.512+0000 7fc8e2d9d640 1 --2- 172.21.15.76:0/3676840991 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f541ae6850 0x55f541ab9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.512+0000 7fc8e2d9d640 1 -- 172.21.15.76:0/3676840991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f541ae6c30 msgr2=0x55f541ae95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:18.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.512+0000 7fc8e2d9d640 1 --2- 172.21.15.76:0/3676840991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f541ae6c30 0x55f541ae95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.512+0000 7fc8e2d9d640 1 -- 172.21.15.76:0/3676840991 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5419e8ca0 con 0x55f541ab9c90 2023-11-09T03:54:18.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.512+0000 7fc8e2d9d640 1 --2- 172.21.15.76:0/3676840991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f541ab9c90 0x55f541ae9b20 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc002090 tx=0x7fc8dc00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23a6469e64608a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:18.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.512+0000 7fc8c3fff640 1 -- 172.21.15.76:0/3676840991 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc014480 con 0x55f541ab9c90 2023-11-09T03:54:18.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.512+0000 7fc8c3fff640 1 -- 172.21.15.76:0/3676840991 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8dc0145e0 con 0x55f541ab9c90 2023-11-09T03:54:18.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.512+0000 7fc8c3fff640 1 -- 172.21.15.76:0/3676840991 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc014950 con 0x55f541ab9c90 2023-11-09T03:54:18.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.512+0000 7fc8eb5b9740 1 -- 172.21.15.76:0/3676840991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f541ab9c90 msgr2=0x55f541ae9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:18.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.512+0000 7fc8eb5b9740 1 --2- 172.21.15.76:0/3676840991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f541ab9c90 0x55f541ae9b20 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc002090 tx=0x7fc8dc00e530 comp rx=0 tx=0).stop 2023-11-09T03:54:18.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.513+0000 7fc8eb5b9740 1 -- 172.21.15.76:0/3676840991 shutdown_connections 2023-11-09T03:54:18.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.513+0000 7fc8eb5b9740 1 --2- 172.21.15.76:0/3676840991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f541ab9c90 0x55f541ae9b20 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.513+0000 7fc8eb5b9740 1 --2- 172.21.15.76:0/3676840991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f541ae6c30 0x55f541ae95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.513+0000 7fc8eb5b9740 1 --2- 172.21.15.76:0/3676840991 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f541ae6850 0x55f541ab9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.513+0000 7fc8eb5b9740 1 -- 172.21.15.76:0/3676840991 >> 172.21.15.76:0/3676840991 conn(0x55f541abba60 msgr2=0x55f541b045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:18.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.513+0000 7fc8eb5b9740 1 -- 172.21.15.76:0/3676840991 shutdown_connections 2023-11-09T03:54:18.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.513+0000 7fc8eb5b9740 1 -- 172.21.15.76:0/3676840991 wait complete. 2023-11-09T03:54:18.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.513+0000 7fc8eb5b9740 1 -- start start 2023-11-09T03:54:18.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.513+0000 7fc8eb5b9740 1 Processor -- start 2023-11-09T03:54:18.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.514+0000 7fc8eb5b9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f541ae6850 0x55f541aec030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.514+0000 7fc8eb5b9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f541ae6c30 0x55f541aff140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.514+0000 7fc8eb5b9740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f541adab40 0x55f541afe5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.514+0000 7fc8eb5b9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f541a258b0 con 0x55f541ae6c30 2023-11-09T03:54:18.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.514+0000 7fc8eb5b9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5419dfc80 con 0x55f541ae6850 2023-11-09T03:54:18.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.514+0000 7fc8eb5b9740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f5419df920 con 0x55f541adab40 2023-11-09T03:54:18.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.514+0000 7fc8eb5b9740 10 client.0 _ll_register_callbacks cb 0x55f541737ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:54:18.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.514+0000 7fc8e1d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f541ae6c30 0x55f541aff140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.514+0000 7fc8e1d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f541ae6c30 0x55f541aff140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57162/0 (socket says 172.21.15.76:57162) 2023-11-09T03:54:18.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.514+0000 7fc8e1d9b640 1 -- 172.21.15.76:0/1968715628 learned_addr learned my addr 172.21.15.76:0/1968715628 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:18.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.514+0000 7fc8e2d9d640 1 --2- 172.21.15.76:0/1968715628 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f541adab40 0x55f541afe5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.514+0000 7fc8e259c640 1 --2- 172.21.15.76:0/1968715628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f541ae6850 0x55f541aec030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.515+0000 7fc8e1d9b640 1 -- 172.21.15.76:0/1968715628 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f541adab40 msgr2=0x55f541afe5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:18.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.515+0000 7fc8e1d9b640 1 --2- 172.21.15.76:0/1968715628 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f541adab40 0x55f541afe5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.515+0000 7fc8e1d9b640 1 -- 172.21.15.76:0/1968715628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f541ae6850 msgr2=0x55f541aec030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:18.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.515+0000 7fc8e1d9b640 1 --2- 172.21.15.76:0/1968715628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f541ae6850 0x55f541aec030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.515+0000 7fc8e1d9b640 1 -- 172.21.15.76:0/1968715628 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8dc00f040 con 0x55f541ae6c30 2023-11-09T03:54:18.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.515+0000 7fc8e1d9b640 1 --2- 172.21.15.76:0/1968715628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f541ae6c30 0x55f541aff140 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4034040 tx=0x7fc8d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79c4cd3ee8270fe server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:18.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.515+0000 7fc8c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:54:18.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.515+0000 7fc8c17fa640 1 -- 172.21.15.76:0/1968715628 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4032810 con 0x55f541ae6c30 2023-11-09T03:54:18.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.515+0000 7fc8c17fa640 1 -- 172.21.15.76:0/1968715628 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7fc8d4040070 con 0x55f541ae6c30 2023-11-09T03:54:18.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.515+0000 7fc8eb5b9740 1 -- 172.21.15.76:0/1968715628 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5419d1bf0 con 0x55f541ae6c30 2023-11-09T03:54:18.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.515+0000 7fc8c17fa640 1 -- 172.21.15.76:0/1968715628 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8d4045e70 con 0x55f541ae6c30 2023-11-09T03:54:18.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.515+0000 7fc8eb5b9740 10 client.9096 fetch_fsmap learned FSMap version 125 2023-11-09T03:54:18.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.515+0000 7fc8eb5b9740 1 -- 172.21.15.76:0/1968715628 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x55f541a23f30 con 0x55f541ae6c30 2023-11-09T03:54:18.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.515+0000 7fc8c17fa640 1 -- 172.21.15.76:0/1968715628 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d405f020 con 0x55f541ae6c30 2023-11-09T03:54:18.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.515+0000 7fc8c17fa640 1 -- 172.21.15.76:0/1968715628 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 125) v1 ==== 3306+0+0 (secure 0 0 0) 0x7fc8d404f070 con 0x55f541ae6c30 2023-11-09T03:54:18.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.515+0000 7fc8eb5b9740 10 client.9096 fetch_fsmap finished waiting for FSMap version 125 2023-11-09T03:54:18.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.515+0000 7fc8eb5b9740 10 client.9096 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-09T03:54:18.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.515+0000 7fc8eb5b9740 20 client.9096 populate_metadata read hostname 'smithi076' 2023-11-09T03:54:18.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.515+0000 7fc8eb5b9740 1 --2- 172.21.15.76:0/1968715628 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x55f541b17c80 0x55f541afc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.515+0000 7fc8eb5b9740 4 client.9096 mds_command: new command op to 8568 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:54:18.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.515+0000 7fc8eb5b9740 1 -- 172.21.15.76:0/1968715628 --> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f541a1e890 con 0x55f541b17c80 2023-11-09T03:54:18.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.516+0000 7fc8e259c640 1 --2- 172.21.15.76:0/1968715628 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x55f541b17c80 0x55f541afc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.516+0000 7fc8c17fa640 10 client.9096 ms_handle_connect on v2:172.21.15.76:6832/153465374 2023-11-09T03:54:18.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.516+0000 7fc8e259c640 1 --2- 172.21.15.76:0/1968715628 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x55f541b17c80 0x55f541afc720 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82303477cccaa220 server_cookie=daccd0510dd11b56 in_seq=0 out_seq=0 2023-11-09T03:54:18.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.516+0000 7fc8c17fa640 1 -- 172.21.15.76:0/1968715628 <== mds.0 v2:172.21.15.76:6832/153465374 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f541a1e890 con 0x55f541b17c80 2023-11-09T03:54:18.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.516+0000 7fc8c17fa640 10 client.9096 handle_command_reply: tid=0 2023-11-09T03:54:18.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.516+0000 7fc8eb5b9740 1 client.9096 shutdown 2023-11-09T03:54:18.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.516+0000 7fc8eb5b9740 1 -- 172.21.15.76:0/1968715628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f541ae6c30 msgr2=0x55f541aff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:18.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.516+0000 7fc8eb5b9740 1 --2- 172.21.15.76:0/1968715628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f541ae6c30 0x55f541aff140 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4034040 tx=0x7fc8d4004730 comp rx=0 tx=0).stop 2023-11-09T03:54:18.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.516+0000 7fc8eb5b9740 1 -- 172.21.15.76:0/1968715628 shutdown_connections 2023-11-09T03:54:18.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.516+0000 7fc8eb5b9740 1 --2- 172.21.15.76:0/1968715628 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x55f541b17c80 0x55f541afc720 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.516+0000 7fc8eb5b9740 1 --2- 172.21.15.76:0/1968715628 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f541adab40 0x55f541afe5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.516+0000 7fc8eb5b9740 1 --2- 172.21.15.76:0/1968715628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f541ae6c30 0x55f541aff140 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.516+0000 7fc8eb5b9740 1 --2- 172.21.15.76:0/1968715628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f541ae6850 0x55f541aec030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.516+0000 7fc8c17fa640 0 client.9096 ms_handle_reset on v2:172.21.15.76:6832/153465374 2023-11-09T03:54:18.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.517+0000 7fc8eb5b9740 1 -- 172.21.15.76:0/1968715628 >> 172.21.15.76:0/1968715628 conn(0x55f541ab9c90 msgr2=0x55f541abd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:18.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.517+0000 7fc8eb5b9740 1 -- 172.21.15.76:0/1968715628 shutdown_connections 2023-11-09T03:54:18.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.517+0000 7fc8eb5b9740 1 -- 172.21.15.76:0/1968715628 wait complete. 2023-11-09T03:54:18.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.517+0000 7fc8eb5b9740 20 client.9096 trim_cache size 0 max 16384 2023-11-09T03:54:18.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.532+0000 7fc8eb5b9740 1 Processor -- start 2023-11-09T03:54:18.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.532+0000 7fc8eb5b9740 1 -- start start 2023-11-09T03:54:18.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.533+0000 7fc8eb5b9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f541adab40 0x55f541b04ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.533+0000 7fc8eb5b9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f541ae6850 0x55f541afdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.533+0000 7fc8eb5b9740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f541ae6c30 0x55f541aebd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.533+0000 7fc8eb5b9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5419a5280 con 0x55f541ae6850 2023-11-09T03:54:18.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.533+0000 7fc8eb5b9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f541b00050 con 0x55f541adab40 2023-11-09T03:54:18.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.533+0000 7fc8eb5b9740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f541abaac0 con 0x55f541ae6c30 2023-11-09T03:54:18.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.533+0000 7fc8e1d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f541ae6850 0x55f541afdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.533+0000 7fc8e1d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f541ae6850 0x55f541afdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57166/0 (socket says 172.21.15.76:57166) 2023-11-09T03:54:18.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.533+0000 7fc8e1d9b640 1 -- 172.21.15.76:0/1920122111 learned_addr learned my addr 172.21.15.76:0/1920122111 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:18.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.533+0000 7fc8e1d9b640 1 -- 172.21.15.76:0/1920122111 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f541ae6c30 msgr2=0x55f541aebd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:18.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.533+0000 7fc8e1d9b640 1 --2- 172.21.15.76:0/1920122111 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f541ae6c30 0x55f541aebd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.533+0000 7fc8e1d9b640 1 -- 172.21.15.76:0/1920122111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f541adab40 msgr2=0x55f541b04ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:18.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.533+0000 7fc8e1d9b640 1 --2- 172.21.15.76:0/1920122111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f541adab40 0x55f541b04ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.533+0000 7fc8e1d9b640 1 -- 172.21.15.76:0/1920122111 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5419c1d10 con 0x55f541ae6850 2023-11-09T03:54:18.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.533+0000 7fc8e1d9b640 1 --2- 172.21.15.76:0/1920122111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f541ae6850 0x55f541afdbf0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fc8d402bda0 tx=0x7fc8d4048070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555cd8ecafd959ab server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:18.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.533+0000 7fc8d0ff9640 1 -- 172.21.15.76:0/1920122111 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4032ab0 con 0x55f541ae6850 2023-11-09T03:54:18.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.533+0000 7fc8d0ff9640 1 -- 172.21.15.76:0/1920122111 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d405f040 con 0x55f541ae6850 2023-11-09T03:54:18.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.533+0000 7fc8d0ff9640 1 -- 172.21.15.76:0/1920122111 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4032e10 con 0x55f541ae6850 2023-11-09T03:54:18.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.534+0000 7fc8eb5b9740 1 -- 172.21.15.76:0/1920122111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f541ae6850 msgr2=0x55f541afdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:18.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.534+0000 7fc8eb5b9740 1 --2- 172.21.15.76:0/1920122111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f541ae6850 0x55f541afdbf0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fc8d402bda0 tx=0x7fc8d4048070 comp rx=0 tx=0).stop 2023-11-09T03:54:18.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.534+0000 7fc8eb5b9740 1 -- 172.21.15.76:0/1920122111 shutdown_connections 2023-11-09T03:54:18.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.534+0000 7fc8eb5b9740 1 --2- 172.21.15.76:0/1920122111 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f541ae6c30 0x55f541aebd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.534+0000 7fc8eb5b9740 1 --2- 172.21.15.76:0/1920122111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f541ae6850 0x55f541afdbf0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.534+0000 7fc8eb5b9740 1 --2- 172.21.15.76:0/1920122111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f541adab40 0x55f541b04ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.534+0000 7fc8eb5b9740 1 -- 172.21.15.76:0/1920122111 >> 172.21.15.76:0/1920122111 conn(0x55f541ab9c90 msgr2=0x55f541ad9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:18.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.534+0000 7fc8eb5b9740 1 -- 172.21.15.76:0/1920122111 shutdown_connections 2023-11-09T03:54:18.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.534+0000 7fc8eb5b9740 1 -- 172.21.15.76:0/1920122111 wait complete. 2023-11-09T03:54:18.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.534+0000 7fc8eb5b9740 1 -- start start 2023-11-09T03:54:18.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.535+0000 7fc8eb5b9740 1 Processor -- start 2023-11-09T03:54:18.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.535+0000 7fc8eb5b9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f541adab40 0x55f541ae9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.535+0000 7fc8eb5b9740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f541ae6850 0x55f541ae9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.535+0000 7fc8eb5b9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f541ae6c30 0x55f541afc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.535+0000 7fc8eb5b9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f541afd4e0 con 0x55f541ae6c30 2023-11-09T03:54:18.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.535+0000 7fc8eb5b9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f541affbf0 con 0x55f541adab40 2023-11-09T03:54:18.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.535+0000 7fc8eb5b9740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f541b16c70 con 0x55f541ae6850 2023-11-09T03:54:18.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.535+0000 7fc8eb5b9740 10 client.0 _ll_register_callbacks cb 0x7fc8dc001ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:54:18.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.535+0000 7fc8e2d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f541ae6c30 0x55f541afc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.535+0000 7fc8e2d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f541ae6c30 0x55f541afc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57176/0 (socket says 172.21.15.76:57176) 2023-11-09T03:54:18.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.535+0000 7fc8e2d9d640 1 -- 172.21.15.76:0/1246307245 learned_addr learned my addr 172.21.15.76:0/1246307245 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:18.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.536+0000 7fc8e1d9b640 1 --2- 172.21.15.76:0/1246307245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f541ae6850 0x55f541ae9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.536+0000 7fc8e259c640 1 --2- 172.21.15.76:0/1246307245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f541adab40 0x55f541ae9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.536+0000 7fc8e2d9d640 1 -- 172.21.15.76:0/1246307245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f541ae6850 msgr2=0x55f541ae9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:18.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.536+0000 7fc8e2d9d640 1 --2- 172.21.15.76:0/1246307245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f541ae6850 0x55f541ae9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.536+0000 7fc8e2d9d640 1 -- 172.21.15.76:0/1246307245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f541adab40 msgr2=0x55f541ae9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:18.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.536+0000 7fc8e2d9d640 1 --2- 172.21.15.76:0/1246307245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f541adab40 0x55f541ae9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.536+0000 7fc8e2d9d640 1 -- 172.21.15.76:0/1246307245 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f541a23f30 con 0x55f541ae6c30 2023-11-09T03:54:18.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.536+0000 7fc8e2d9d640 1 --2- 172.21.15.76:0/1246307245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f541ae6c30 0x55f541afc3d0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc017040 tx=0x7fc8dc0014d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3452e74b8fe2dc server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:18.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.536+0000 7fc8c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:54:18.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.536+0000 7fc8c17fa640 1 -- 172.21.15.76:0/1246307245 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc014480 con 0x55f541ae6c30 2023-11-09T03:54:18.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.536+0000 7fc8c17fa640 1 -- 172.21.15.76:0/1246307245 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7fc8dc001910 con 0x55f541ae6c30 2023-11-09T03:54:18.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.536+0000 7fc8eb5b9740 1 -- 172.21.15.76:0/1246307245 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc8dc0145e0 con 0x55f541ae6c30 2023-11-09T03:54:18.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.536+0000 7fc8c17fa640 1 -- 172.21.15.76:0/1246307245 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8dc02e020 con 0x55f541ae6c30 2023-11-09T03:54:18.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.536+0000 7fc8eb5b9740 10 client.9102 fetch_fsmap learned FSMap version 125 2023-11-09T03:54:18.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.536+0000 7fc8eb5b9740 1 -- 172.21.15.76:0/1246307245 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x55f5419e7500 con 0x55f541ae6c30 2023-11-09T03:54:18.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.536+0000 7fc8c17fa640 1 -- 172.21.15.76:0/1246307245 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8dc0145e0 con 0x55f541ae6c30 2023-11-09T03:54:18.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.537+0000 7fc8c17fa640 1 -- 172.21.15.76:0/1246307245 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 125) v1 ==== 3306+0+0 (secure 0 0 0) 0x7fc8dc014c90 con 0x55f541ae6c30 2023-11-09T03:54:18.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.537+0000 7fc8eb5b9740 10 client.9102 fetch_fsmap finished waiting for FSMap version 125 2023-11-09T03:54:18.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.537+0000 7fc8eb5b9740 10 client.9102 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-09T03:54:18.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.537+0000 7fc8eb5b9740 20 client.9102 populate_metadata read hostname 'smithi076' 2023-11-09T03:54:18.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.537+0000 7fc8eb5b9740 1 --2- 172.21.15.76:0/1246307245 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x55f541afc910 0x55f541b12cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.537+0000 7fc8eb5b9740 4 client.9102 mds_command: new command op to 8568 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-09T03:54:18.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.537+0000 7fc8eb5b9740 1 -- 172.21.15.76:0/1246307245 --> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f541a1e5b0 con 0x55f541afc910 2023-11-09T03:54:18.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.537+0000 7fc8e259c640 1 --2- 172.21.15.76:0/1246307245 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x55f541afc910 0x55f541b12cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.537+0000 7fc8c17fa640 10 client.9102 ms_handle_connect on v2:172.21.15.76:6832/153465374 2023-11-09T03:54:18.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.537+0000 7fc8e259c640 1 --2- 172.21.15.76:0/1246307245 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x55f541afc910 0x55f541b12cb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dbd425851a9cd6c server_cookie=2663f56ebf7d281e in_seq=0 out_seq=0 2023-11-09T03:54:18.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.537+0000 7fc8c17fa640 1 -- 172.21.15.76:0/1246307245 <== mds.0 v2:172.21.15.76:6832/153465374 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+329 (crc 0 0 0) 0x55f541a1e5b0 con 0x55f541afc910 2023-11-09T03:54:18.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.537+0000 7fc8c17fa640 10 client.9102 handle_command_reply: tid=0 2023-11-09T03:54:18.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.537+0000 7fc8eb5b9740 1 client.9102 shutdown 2023-11-09T03:54:18.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.537+0000 7fc8eb5b9740 1 -- 172.21.15.76:0/1246307245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f541ae6c30 msgr2=0x55f541afc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:18.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.537+0000 7fc8eb5b9740 1 --2- 172.21.15.76:0/1246307245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f541ae6c30 0x55f541afc3d0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc017040 tx=0x7fc8dc0014d0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.537+0000 7fc8eb5b9740 1 -- 172.21.15.76:0/1246307245 shutdown_connections 2023-11-09T03:54:18.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.537+0000 7fc8eb5b9740 1 --2- 172.21.15.76:0/1246307245 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x55f541afc910 0x55f541b12cb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.537+0000 7fc8eb5b9740 1 --2- 172.21.15.76:0/1246307245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f541ae6c30 0x55f541afc3d0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.537+0000 7fc8c17fa640 0 client.9102 ms_handle_reset on v2:172.21.15.76:6832/153465374 2023-11-09T03:54:18.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.537+0000 7fc8eb5b9740 1 --2- 172.21.15.76:0/1246307245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f541ae6850 0x55f541ae9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.537+0000 7fc8eb5b9740 1 --2- 172.21.15.76:0/1246307245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f541adab40 0x55f541ae9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.537+0000 7fc8eb5b9740 1 -- 172.21.15.76:0/1246307245 >> 172.21.15.76:0/1246307245 conn(0x55f541ab9c90 msgr2=0x55f541b04f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:18.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.537+0000 7fc8eb5b9740 1 -- 172.21.15.76:0/1246307245 shutdown_connections 2023-11-09T03:54:18.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.537+0000 7fc8eb5b9740 1 -- 172.21.15.76:0/1246307245 wait complete. 2023-11-09T03:54:18.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.538+0000 7fc8eb5b9740 20 client.9102 trim_cache size 0 max 16384 2023-11-09T03:54:18.565 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:54:18.565 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "c213b26f-d0da-45d5-8e34-bb75c096d4f2", 2023-11-09T03:54:18.566 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2023-11-09T03:54:18.566 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 8568, 2023-11-09T03:54:18.566 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2023-11-09T03:54:18.566 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2023-11-09T03:54:18.566 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2023-11-09T03:54:18.566 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 61.056439795000003, 2023-11-09T03:54:18.567 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 125, 2023-11-09T03:54:18.567 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 116, 2023-11-09T03:54:18.567 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 116, 2023-11-09T03:54:18.567 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 104.226468873 2023-11-09T03:54:18.567 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:54:18.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.538+0000 7fc8e894c640 1 -- 172.21.15.76:0/579383460 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc8c406ae70 msgr2=0x7fc8c406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:18.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.538+0000 7fc8e894c640 1 --2- 172.21.15.76:0/579383460 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc8c406ae70 0x7fc8c406d330 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4036000 tx=0x7fc8d403d040 comp rx=0 tx=0).stop 2023-11-09T03:54:18.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.538+0000 7fc8e894c640 1 -- 172.21.15.76:0/579383460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8e406c5f0 msgr2=0x7fc8e414f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:18.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.538+0000 7fc8e894c640 1 --2- 172.21.15.76:0/579383460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8e406c5f0 0x7fc8e414f5a0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800c430 tx=0x7fc8d800c8f0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.538+0000 7fc8e894c640 1 -- 172.21.15.76:0/579383460 shutdown_connections 2023-11-09T03:54:18.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.538+0000 7fc8e894c640 1 --2- 172.21.15.76:0/579383460 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc8c406ae70 0x7fc8c406d330 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.538+0000 7fc8e894c640 1 --2- 172.21.15.76:0/579383460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e40ade00 0x7fc8e40c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.538+0000 7fc8e894c640 1 --2- 172.21.15.76:0/579383460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8e406d9c0 0x7fc8e4152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.538+0000 7fc8e894c640 1 --2- 172.21.15.76:0/579383460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8e406c5f0 0x7fc8e414f5a0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.538+0000 7fc8e894c640 1 -- 172.21.15.76:0/579383460 >> 172.21.15.76:0/579383460 conn(0x7fc8e40cb530 msgr2=0x7fc8e40ae4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:18.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.538+0000 7fc8e894c640 1 -- 172.21.15.76:0/579383460 shutdown_connections 2023-11-09T03:54:18.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.538+0000 7fc8e894c640 1 -- 172.21.15.76:0/579383460 wait complete. 2023-11-09T03:54:18.571 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:54:18.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.672+0000 7f29060a9640 1 Processor -- start 2023-11-09T03:54:18.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.672+0000 7f29060a9640 1 -- start start 2023-11-09T03:54:18.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.673+0000 7f29060a9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f290015d4e0 0x7f29000d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.673+0000 7f29060a9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29000b32b0 con 0x7f29000d3b60 2023-11-09T03:54:18.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.673+0000 7f29060a9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29000cee10 con 0x7f290015d100 2023-11-09T03:54:18.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.673+0000 7f29060a9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29000cec20 con 0x7f290015d4e0 2023-11-09T03:54:18.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.673+0000 7f28fffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f29000d3b60 0x7f29000b3740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45772/0 (socket says 172.21.15.76:45772) 2023-11-09T03:54:18.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.673+0000 7f28fffff640 1 -- 172.21.15.76:0/2312244595 learned_addr learned my addr 172.21.15.76:0/2312244595 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:54:18.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.674+0000 7f28f6ffd640 1 --2- 172.21.15.76:0/2312244595 >> v2:172.21.15.113:3300/0 conn(0x7f290015d4e0 0x7f29000d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.678 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.674+0000 7f28feffd640 1 -- 172.21.15.76:0/2312244595 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2987066889 0 0) 0x7f29000b32b0 con 0x7f29000d3b60 2023-11-09T03:54:18.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.674+0000 7f28feffd640 1 -- 172.21.15.76:0/2312244595 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28ec003520 con 0x7f29000d3b60 2023-11-09T03:54:18.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.674+0000 7f28feffd640 1 -- 172.21.15.76:0/2312244595 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2456118755 0 0) 0x7f28ec003520 con 0x7f29000d3b60 2023-11-09T03:54:18.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.674+0000 7f28feffd640 1 -- 172.21.15.76:0/2312244595 >> v2:172.21.15.113:3300/0 conn(0x7f290015d4e0 msgr2=0x7f29000d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:18.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.674+0000 7f28feffd640 1 --2- 172.21.15.76:0/2312244595 >> v2:172.21.15.113:3300/0 conn(0x7f290015d4e0 0x7f29000d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.674+0000 7f28feffd640 1 -- 172.21.15.76:0/2312244595 >> v1:172.21.15.113:6789/0 conn(0x7f290015d100 legacy=0x7f29000ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:18.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.674+0000 7f28feffd640 1 -- 172.21.15.76:0/2312244595 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29000b2980 con 0x7f29000d3b60 2023-11-09T03:54:18.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.674+0000 7f28feffd640 1 -- 172.21.15.76:0/2312244595 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f28f80019f0 con 0x7f29000d3b60 2023-11-09T03:54:18.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.674+0000 7f28feffd640 1 -- 172.21.15.76:0/2312244595 >> v1:172.21.15.76:6789/0 conn(0x7f29000d3b60 legacy=0x7f29000b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:18.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.674+0000 7f28feffd640 1 --2- 172.21.15.76:0/2312244595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28ec003970 0x7f28ec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.674+0000 7f28feffd640 1 --2- 172.21.15.76:0/2312244595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28ec0047c0 0x7f28ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.674+0000 7f28feffd640 1 --2- 172.21.15.76:0/2312244595 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f28ec005180 0x7f28ec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.674+0000 7f28feffd640 1 -- 172.21.15.76:0/2312244595 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec00bff0 con 0x7f28ec0047c0 2023-11-09T03:54:18.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.674+0000 7f28feffd640 1 -- 172.21.15.76:0/2312244595 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec00c160 con 0x7f28ec003970 2023-11-09T03:54:18.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.674+0000 7f28feffd640 1 -- 172.21.15.76:0/2312244595 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec00c320 con 0x7f28ec005180 2023-11-09T03:54:18.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.674+0000 7f28f6ffd640 1 --2- 172.21.15.76:0/2312244595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28ec0047c0 0x7f28ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.674+0000 7f28fffff640 1 --2- 172.21.15.76:0/2312244595 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f28ec005180 0x7f28ec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.674+0000 7f28ff7fe640 1 --2- 172.21.15.76:0/2312244595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28ec003970 0x7f28ec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.674+0000 7f28f6ffd640 1 -- 172.21.15.76:0/2312244595 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f28ec005180 msgr2=0x7f28ec00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:18.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.674+0000 7f28f6ffd640 1 --2- 172.21.15.76:0/2312244595 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f28ec005180 0x7f28ec00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.674+0000 7f28f6ffd640 1 -- 172.21.15.76:0/2312244595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28ec003970 msgr2=0x7f28ec003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:18.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.674+0000 7f28f6ffd640 1 --2- 172.21.15.76:0/2312244595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28ec003970 0x7f28ec003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.674+0000 7f28fffff640 1 -- 172.21.15.76:0/2312244595 reap_dead start 2023-11-09T03:54:18.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.674+0000 7f28f6ffd640 1 -- 172.21.15.76:0/2312244595 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ec003520 con 0x7f28ec0047c0 2023-11-09T03:54:18.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.674+0000 7f28ff7fe640 1 --2- 172.21.15.76:0/2312244595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28ec003970 0x7f28ec003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:54:18.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.675+0000 7f28f6ffd640 1 --2- 172.21.15.76:0/2312244595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28ec0047c0 0x7f28ec004c40 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009a90 tx=0x7f28ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d2fe7f4f3bed38 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:18.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.675+0000 7f28feffd640 1 -- 172.21.15.76:0/2312244595 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e802c6b0 con 0x7f28ec0047c0 2023-11-09T03:54:18.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.675+0000 7f28feffd640 1 -- 172.21.15.76:0/2312244595 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e802c810 con 0x7f28ec0047c0 2023-11-09T03:54:18.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.675+0000 7f28feffd640 1 -- 172.21.15.76:0/2312244595 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e802cba0 con 0x7f28ec0047c0 2023-11-09T03:54:18.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.676+0000 7f29060a9640 1 -- 172.21.15.76:0/2312244595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28ec0047c0 msgr2=0x7f28ec004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:18.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.676+0000 7f29060a9640 1 --2- 172.21.15.76:0/2312244595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28ec0047c0 0x7f28ec004c40 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009a90 tx=0x7f28ec0057f0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.676+0000 7f29060a9640 1 -- 172.21.15.76:0/2312244595 shutdown_connections 2023-11-09T03:54:18.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.676+0000 7f29060a9640 1 --2- 172.21.15.76:0/2312244595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28ec0047c0 0x7f28ec004c40 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.676+0000 7f29060a9640 1 -- 172.21.15.76:0/2312244595 >> 172.21.15.76:0/2312244595 conn(0x7f29000d2ef0 msgr2=0x7f29000d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:18.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.676+0000 7f29060a9640 1 -- 172.21.15.76:0/2312244595 shutdown_connections 2023-11-09T03:54:18.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.676+0000 7f29060a9640 1 -- 172.21.15.76:0/2312244595 wait complete. 2023-11-09T03:54:18.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.677+0000 7f29060a9640 1 Processor -- start 2023-11-09T03:54:18.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.677+0000 7f29060a9640 1 -- start start 2023-11-09T03:54:18.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.678+0000 7f29060a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28ec0047c0 0x7f2900152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.678+0000 7f29060a9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29000cc260 0x7f290015fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.678+0000 7f29060a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29000cc6e0 0x7f29001602e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.678+0000 7f29060a9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29000b77d0 con 0x7f29000cc260 2023-11-09T03:54:18.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.678+0000 7f29060a9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29000b6bc0 con 0x7f28ec0047c0 2023-11-09T03:54:18.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.678+0000 7f29060a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29000b6ea0 con 0x7f29000cc6e0 2023-11-09T03:54:18.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.678+0000 7f28f6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29000cc260 0x7f290015fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.678+0000 7f28f6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29000cc260 0x7f290015fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57194/0 (socket says 172.21.15.76:57194) 2023-11-09T03:54:18.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.678+0000 7f28f6ffd640 1 -- 172.21.15.76:0/1331922197 learned_addr learned my addr 172.21.15.76:0/1331922197 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:18.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.678+0000 7f28ff7fe640 1 --2- 172.21.15.76:0/1331922197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28ec0047c0 0x7f2900152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.678+0000 7f28fffff640 1 --2- 172.21.15.76:0/1331922197 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29000cc6e0 0x7f29001602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.678+0000 7f28f6ffd640 1 -- 172.21.15.76:0/1331922197 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29000cc6e0 msgr2=0x7f29001602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:18.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.678+0000 7f28f6ffd640 1 --2- 172.21.15.76:0/1331922197 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29000cc6e0 0x7f29001602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.678+0000 7f28f6ffd640 1 -- 172.21.15.76:0/1331922197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28ec0047c0 msgr2=0x7f2900152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:18.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.678+0000 7f28f6ffd640 1 --2- 172.21.15.76:0/1331922197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28ec0047c0 0x7f2900152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.678+0000 7f28f6ffd640 1 -- 172.21.15.76:0/1331922197 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29000b7180 con 0x7f29000cc260 2023-11-09T03:54:18.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.678+0000 7f28ff7fe640 1 --2- 172.21.15.76:0/1331922197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28ec0047c0 0x7f2900152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:54:18.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.678+0000 7f28fffff640 1 --2- 172.21.15.76:0/1331922197 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29000cc6e0 0x7f29001602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:54:18.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.678+0000 7f28f6ffd640 1 --2- 172.21.15.76:0/1331922197 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29000cc260 0x7f290015fda0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f28e802c250 tx=0x7f28e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:18.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.678+0000 7f28fcff9640 1 -- 172.21.15.76:0/1331922197 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8003810 con 0x7f29000cc260 2023-11-09T03:54:18.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.678+0000 7f28fcff9640 1 -- 172.21.15.76:0/1331922197 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e8003970 con 0x7f29000cc260 2023-11-09T03:54:18.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.679+0000 7f29060a9640 1 -- 172.21.15.76:0/1331922197 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29000b7460 con 0x7f29000cc260 2023-11-09T03:54:18.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.679+0000 7f28fcff9640 1 -- 172.21.15.76:0/1331922197 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8003ce0 con 0x7f29000cc260 2023-11-09T03:54:18.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.679+0000 7f29060a9640 1 -- 172.21.15.76:0/1331922197 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29000b8190 con 0x7f29000cc260 2023-11-09T03:54:18.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.680+0000 7f29060a9640 1 -- 172.21.15.76:0/1331922197 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29000b75f0 con 0x7f29000cc260 2023-11-09T03:54:18.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.680+0000 7f28fcff9640 1 -- 172.21.15.76:0/1331922197 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f28e8040450 con 0x7f29000cc260 2023-11-09T03:54:18.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.680+0000 7f28fcff9640 1 --2- 172.21.15.76:0/1331922197 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f28e006ae70 0x7f28e006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:18.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.680+0000 7f28fcff9640 1 -- 172.21.15.76:0/1331922197 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7f28e80abf80 con 0x7f29000cc260 2023-11-09T03:54:18.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.681+0000 7f28ff7fe640 1 --2- 172.21.15.76:0/1331922197 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f28e006ae70 0x7f28e006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:18.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.681+0000 7f28ff7fe640 1 --2- 172.21.15.76:0/1331922197 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f28e006ae70 0x7f28e006d330 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f28f0000c00 tx=0x7f28f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:18.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.685+0000 7f28fcff9640 1 -- 172.21.15.76:0/1331922197 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f28e80deca0 con 0x7f29000cc260 2023-11-09T03:54:18.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.886+0000 7f29060a9640 1 -- 172.21.15.76:0/1331922197 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29000684b0 con 0x7f29000cc260 2023-11-09T03:54:18.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.887+0000 7f28fcff9640 1 -- 172.21.15.76:0/1331922197 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4935 (secure 0 0 0) 0x7f28e80765c0 con 0x7f29000cc260 2023-11-09T03:54:18.888 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:54:18.888 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":125,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":9024,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:16.621433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":6470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6470":{"gid":6470,"name":"b","rank":1,"incarnation":121,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:54:18.889 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 125 2023-11-09T03:54:18.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.889+0000 7f29060a9640 1 -- 172.21.15.76:0/1331922197 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f28e006ae70 msgr2=0x7f28e006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:18.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.889+0000 7f29060a9640 1 --2- 172.21.15.76:0/1331922197 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f28e006ae70 0x7f28e006d330 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f28f0000c00 tx=0x7f28f0009040 comp rx=0 tx=0).stop 2023-11-09T03:54:18.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.889+0000 7f29060a9640 1 -- 172.21.15.76:0/1331922197 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29000cc260 msgr2=0x7f290015fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:18.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.889+0000 7f29060a9640 1 --2- 172.21.15.76:0/1331922197 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29000cc260 0x7f290015fda0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f28e802c250 tx=0x7f28e8004730 comp rx=0 tx=0).stop 2023-11-09T03:54:18.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.889+0000 7f29060a9640 1 -- 172.21.15.76:0/1331922197 shutdown_connections 2023-11-09T03:54:18.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.889+0000 7f29060a9640 1 --2- 172.21.15.76:0/1331922197 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f28e006ae70 0x7f28e006d330 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.889+0000 7f29060a9640 1 --2- 172.21.15.76:0/1331922197 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29000cc6e0 0x7f29001602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.889+0000 7f29060a9640 1 --2- 172.21.15.76:0/1331922197 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29000cc260 0x7f290015fda0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.889+0000 7f29060a9640 1 --2- 172.21.15.76:0/1331922197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28ec0047c0 0x7f2900152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:18.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.889+0000 7f29060a9640 1 -- 172.21.15.76:0/1331922197 >> 172.21.15.76:0/1331922197 conn(0x7f29000d2ef0 msgr2=0x7f290015c280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:18.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.889+0000 7f29060a9640 1 -- 172.21.15.76:0/1331922197 shutdown_connections 2023-11-09T03:54:18.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:18.889+0000 7f29060a9640 1 -- 172.21.15.76:0/1331922197 wait complete. 2023-11-09T03:54:18.901 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2023-11-09T03:54:19.013 INFO:teuthology.orchestra.run.smithi076.stdout:{"last_snap":15,"last_created":12,"last_destroyed":14,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-09T03:54:19.014 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 12, "last_destroyed": 14, "last_snap": 15, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [] } 2023-11-09T03:54:19.014 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-09T03:54:19.015 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:54:19.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.114+0000 7fbc31f25640 1 Processor -- start 2023-11-09T03:54:19.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.114+0000 7fbc31f25640 1 -- start start 2023-11-09T03:54:19.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.115+0000 7fbc31f25640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc2c15d1b0 0x7fbc2c0b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:19.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.115+0000 7fbc31f25640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbc2c06c240 0x7fbc2c0b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:19.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.115+0000 7fbc31f25640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc2c0b2310 con 0x7fbc2c06c6a0 2023-11-09T03:54:19.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.115+0000 7fbc31f25640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc2c06cbe0 con 0x7fbc2c06c240 2023-11-09T03:54:19.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.115+0000 7fbc31f25640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c0b2100 con 0x7fbc2c15d1b0 2023-11-09T03:54:19.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.115+0000 7fbc2bfff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fbc2c06c6a0 0x7fbc2c06a0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45786/0 (socket says 172.21.15.76:45786) 2023-11-09T03:54:19.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.115+0000 7fbc2bfff640 1 -- 172.21.15.76:0/994550840 learned_addr learned my addr 172.21.15.76:0/994550840 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:54:19.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.116+0000 7fbc2b7fe640 1 --2- 172.21.15.76:0/994550840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc2c15d1b0 0x7fbc2c0b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:19.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.116+0000 7fbc2a7fc640 1 -- 172.21.15.76:0/994550840 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1904444889 0 0) 0x7fbc2c0b2310 con 0x7fbc2c06c6a0 2023-11-09T03:54:19.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.116+0000 7fbc2affd640 1 --2- 172.21.15.76:0/994550840 >> v2:172.21.15.113:3300/0 conn(0x7fbc2c06c240 0x7fbc2c0b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:19.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.116+0000 7fbc2a7fc640 1 -- 172.21.15.76:0/994550840 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc14003520 con 0x7fbc2c06c6a0 2023-11-09T03:54:19.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.116+0000 7fbc2a7fc640 1 -- 172.21.15.76:0/994550840 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4268719566 0 0) 0x7fbc14003520 con 0x7fbc2c06c6a0 2023-11-09T03:54:19.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.116+0000 7fbc2a7fc640 1 -- 172.21.15.76:0/994550840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc2c15d1b0 msgr2=0x7fbc2c0b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:19.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.116+0000 7fbc2a7fc640 1 --2- 172.21.15.76:0/994550840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc2c15d1b0 0x7fbc2c0b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:19.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.116+0000 7fbc2a7fc640 1 -- 172.21.15.76:0/994550840 >> v2:172.21.15.113:3300/0 conn(0x7fbc2c06c240 msgr2=0x7fbc2c0b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:19.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.116+0000 7fbc2a7fc640 1 --2- 172.21.15.76:0/994550840 >> v2:172.21.15.113:3300/0 conn(0x7fbc2c06c240 0x7fbc2c0b0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:19.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.116+0000 7fbc2a7fc640 1 -- 172.21.15.76:0/994550840 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2c0b25f0 con 0x7fbc2c06c6a0 2023-11-09T03:54:19.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.116+0000 7fbc2a7fc640 1 -- 172.21.15.76:0/994550840 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fbc240019f0 con 0x7fbc2c06c6a0 2023-11-09T03:54:19.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.116+0000 7fbc2a7fc640 1 -- 172.21.15.76:0/994550840 >> v1:172.21.15.76:6789/0 conn(0x7fbc2c06c6a0 legacy=0x7fbc2c06a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:19.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.116+0000 7fbc2a7fc640 1 --2- 172.21.15.76:0/994550840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc2c15d1b0 0x7fbc14003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:19.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.116+0000 7fbc2a7fc640 1 --2- 172.21.15.76:0/994550840 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc14003ef0 0x7fbc140047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:19.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.116+0000 7fbc2a7fc640 1 --2- 172.21.15.76:0/994550840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc14004d00 0x7fbc14005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:19.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.116+0000 7fbc2a7fc640 1 -- 172.21.15.76:0/994550840 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c06cbe0 con 0x7fbc14003ef0 2023-11-09T03:54:19.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.116+0000 7fbc2a7fc640 1 -- 172.21.15.76:0/994550840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c0b2100 con 0x7fbc14004d00 2023-11-09T03:54:19.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.116+0000 7fbc2a7fc640 1 -- 172.21.15.76:0/994550840 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1400b9b0 con 0x7fbc2c15d1b0 2023-11-09T03:54:19.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.116+0000 7fbc2affd640 1 --2- 172.21.15.76:0/994550840 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc14003ef0 0x7fbc140047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:19.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.116+0000 7fbc2b7fe640 1 --2- 172.21.15.76:0/994550840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc2c15d1b0 0x7fbc14003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:19.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.116+0000 7fbc2bfff640 1 --2- 172.21.15.76:0/994550840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc14004d00 0x7fbc14005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:19.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.116+0000 7fbc2affd640 1 -- 172.21.15.76:0/994550840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc2c15d1b0 msgr2=0x7fbc14003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:19.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.116+0000 7fbc2affd640 1 --2- 172.21.15.76:0/994550840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc2c15d1b0 0x7fbc14003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:19.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.116+0000 7fbc2affd640 1 -- 172.21.15.76:0/994550840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc14004d00 msgr2=0x7fbc14005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:19.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.116+0000 7fbc2affd640 1 --2- 172.21.15.76:0/994550840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc14004d00 0x7fbc14005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:19.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.116+0000 7fbc2affd640 1 -- 172.21.15.76:0/994550840 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc14003520 con 0x7fbc14003ef0 2023-11-09T03:54:19.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.117+0000 7fbc2affd640 1 --2- 172.21.15.76:0/994550840 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc14003ef0 0x7fbc140047c0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fbc20009ac0 tx=0x7fbc140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01c9bdcec09531 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:19.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.117+0000 7fbc2a7fc640 1 -- 172.21.15.76:0/994550840 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2002c710 con 0x7fbc14003ef0 2023-11-09T03:54:19.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.117+0000 7fbc2a7fc640 1 -- 172.21.15.76:0/994550840 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2002c870 con 0x7fbc14003ef0 2023-11-09T03:54:19.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.117+0000 7fbc2a7fc640 1 -- 172.21.15.76:0/994550840 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2002cc00 con 0x7fbc14003ef0 2023-11-09T03:54:19.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.118+0000 7fbc31f25640 1 -- 172.21.15.76:0/994550840 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc14003ef0 msgr2=0x7fbc140047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:19.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.118+0000 7fbc31f25640 1 --2- 172.21.15.76:0/994550840 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc14003ef0 0x7fbc140047c0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fbc20009ac0 tx=0x7fbc140057f0 comp rx=0 tx=0).stop 2023-11-09T03:54:19.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.118+0000 7fbc2bfff640 1 -- 172.21.15.76:0/994550840 reap_dead start 2023-11-09T03:54:19.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.118+0000 7fbc31f25640 1 -- 172.21.15.76:0/994550840 shutdown_connections 2023-11-09T03:54:19.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.118+0000 7fbc31f25640 1 -- 172.21.15.76:0/994550840 >> 172.21.15.76:0/994550840 conn(0x7fbc2c0cbd30 msgr2=0x7fbc2c0cc140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:19.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.118+0000 7fbc31f25640 1 -- 172.21.15.76:0/994550840 shutdown_connections 2023-11-09T03:54:19.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.118+0000 7fbc31f25640 1 -- 172.21.15.76:0/994550840 wait complete. 2023-11-09T03:54:19.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.119+0000 7fbc31f25640 1 Processor -- start 2023-11-09T03:54:19.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.119+0000 7fbc31f25640 1 -- start start 2023-11-09T03:54:19.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.120+0000 7fbc31f25640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc14003ef0 0x7fbc2c1529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:19.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.120+0000 7fbc31f25640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc2c0c8220 0x7fbc2c09a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:19.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.120+0000 7fbc31f25640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc2c0acb90 0x7fbc2c0ad010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:19.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.120+0000 7fbc31f25640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c0ad550 con 0x7fbc2c0acb90 2023-11-09T03:54:19.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.120+0000 7fbc31f25640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c09b700 con 0x7fbc14003ef0 2023-11-09T03:54:19.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.120+0000 7fbc31f25640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c09b9e0 con 0x7fbc2c0c8220 2023-11-09T03:54:19.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.120+0000 7fbc2bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc2c0acb90 0x7fbc2c0ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:19.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.120+0000 7fbc2bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc2c0acb90 0x7fbc2c0ad010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57210/0 (socket says 172.21.15.76:57210) 2023-11-09T03:54:19.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.120+0000 7fbc2bfff640 1 -- 172.21.15.76:0/3223896442 learned_addr learned my addr 172.21.15.76:0/3223896442 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:19.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.120+0000 7fbc2b7fe640 1 --2- 172.21.15.76:0/3223896442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc14003ef0 0x7fbc2c1529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:19.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.120+0000 7fbc2bfff640 1 -- 172.21.15.76:0/3223896442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc2c0c8220 msgr2=0x7fbc2c09a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:19.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.120+0000 7fbc2affd640 1 --2- 172.21.15.76:0/3223896442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc2c0c8220 0x7fbc2c09a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:19.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.120+0000 7fbc2bfff640 1 --2- 172.21.15.76:0/3223896442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc2c0c8220 0x7fbc2c09a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:19.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.120+0000 7fbc2bfff640 1 -- 172.21.15.76:0/3223896442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc14003ef0 msgr2=0x7fbc2c1529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:19.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.120+0000 7fbc2bfff640 1 --2- 172.21.15.76:0/3223896442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc14003ef0 0x7fbc2c1529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:19.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.120+0000 7fbc2bfff640 1 -- 172.21.15.76:0/3223896442 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2c09bcc0 con 0x7fbc2c0acb90 2023-11-09T03:54:19.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.120+0000 7fbc2b7fe640 1 --2- 172.21.15.76:0/3223896442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc14003ef0 0x7fbc2c1529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:54:19.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.120+0000 7fbc2affd640 1 --2- 172.21.15.76:0/3223896442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc2c0c8220 0x7fbc2c09a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:54:19.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.120+0000 7fbc2bfff640 1 --2- 172.21.15.76:0/3223896442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc2c0acb90 0x7fbc2c0ad010 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fbc24000f80 tx=0x7fbc24007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:19.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.120+0000 7fbc13fff640 1 -- 172.21.15.76:0/3223896442 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2400ac20 con 0x7fbc2c0acb90 2023-11-09T03:54:19.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.121+0000 7fbc13fff640 1 -- 172.21.15.76:0/3223896442 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2400ada0 con 0x7fbc2c0acb90 2023-11-09T03:54:19.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.121+0000 7fbc13fff640 1 -- 172.21.15.76:0/3223896442 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc24013630 con 0x7fbc2c0acb90 2023-11-09T03:54:19.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.121+0000 7fbc31f25640 1 -- 172.21.15.76:0/3223896442 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc2c09bfa0 con 0x7fbc2c0acb90 2023-11-09T03:54:19.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.121+0000 7fbc31f25640 1 -- 172.21.15.76:0/3223896442 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc2c09c7a0 con 0x7fbc2c0acb90 2023-11-09T03:54:19.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.121+0000 7fbc31f25640 1 -- 172.21.15.76:0/3223896442 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc2c066600 con 0x7fbc2c0acb90 2023-11-09T03:54:19.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.122+0000 7fbc13fff640 1 -- 172.21.15.76:0/3223896442 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbc24013790 con 0x7fbc2c0acb90 2023-11-09T03:54:19.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.122+0000 7fbc13fff640 1 --2- 172.21.15.76:0/3223896442 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbc1806ae70 0x7fbc1806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:19.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.122+0000 7fbc13fff640 1 -- 172.21.15.76:0/3223896442 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7fbc2408a730 con 0x7fbc2c0acb90 2023-11-09T03:54:19.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.122+0000 7fbc2b7fe640 1 --2- 172.21.15.76:0/3223896442 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbc1806ae70 0x7fbc1806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:19.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.123+0000 7fbc2b7fe640 1 --2- 172.21.15.76:0/3223896442 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbc1806ae70 0x7fbc1806d330 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c0099d0 tx=0x7fbc1c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:19.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.126+0000 7fbc13fff640 1 -- 172.21.15.76:0/3223896442 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbc24054e30 con 0x7fbc2c0acb90 2023-11-09T03:54:19.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.330+0000 7fbc31f25640 1 -- 172.21.15.76:0/3223896442 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc2c0684b0 con 0x7fbc2c0acb90 2023-11-09T03:54:19.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.330+0000 7fbc13fff640 1 -- 172.21.15.76:0/3223896442 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4935 (secure 0 0 0) 0x7fbc24058450 con 0x7fbc2c0acb90 2023-11-09T03:54:19.331 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:54:19.331 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":125,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":9024,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:16.621433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":6470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6470":{"gid":6470,"name":"b","rank":1,"incarnation":121,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:54:19.332 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 125 2023-11-09T03:54:19.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.332+0000 7fbc31f25640 1 -- 172.21.15.76:0/3223896442 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbc1806ae70 msgr2=0x7fbc1806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:19.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.332+0000 7fbc31f25640 1 --2- 172.21.15.76:0/3223896442 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbc1806ae70 0x7fbc1806d330 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c0099d0 tx=0x7fbc1c008040 comp rx=0 tx=0).stop 2023-11-09T03:54:19.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.332+0000 7fbc31f25640 1 -- 172.21.15.76:0/3223896442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc2c0acb90 msgr2=0x7fbc2c0ad010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:19.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.332+0000 7fbc31f25640 1 --2- 172.21.15.76:0/3223896442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc2c0acb90 0x7fbc2c0ad010 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fbc24000f80 tx=0x7fbc24007bd0 comp rx=0 tx=0).stop 2023-11-09T03:54:19.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.333+0000 7fbc31f25640 1 -- 172.21.15.76:0/3223896442 shutdown_connections 2023-11-09T03:54:19.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.333+0000 7fbc31f25640 1 --2- 172.21.15.76:0/3223896442 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbc1806ae70 0x7fbc1806d330 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:19.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.333+0000 7fbc31f25640 1 --2- 172.21.15.76:0/3223896442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc2c0acb90 0x7fbc2c0ad010 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:19.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.333+0000 7fbc31f25640 1 --2- 172.21.15.76:0/3223896442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc2c0c8220 0x7fbc2c09a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:19.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.333+0000 7fbc31f25640 1 --2- 172.21.15.76:0/3223896442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc14003ef0 0x7fbc2c1529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:19.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.333+0000 7fbc31f25640 1 -- 172.21.15.76:0/3223896442 >> 172.21.15.76:0/3223896442 conn(0x7fbc2c0cbd30 msgr2=0x7fbc2c070e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:19.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.333+0000 7fbc31f25640 1 -- 172.21.15.76:0/3223896442 shutdown_connections 2023-11-09T03:54:19.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:19.333+0000 7fbc31f25640 1 -- 172.21.15.76:0/3223896442 wait complete. 2023-11-09T03:54:19.344 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2023-11-09T03:54:19.458 INFO:teuthology.orchestra.run.smithi076.stdout:{"last_snap":15,"last_created":12,"last_destroyed":14,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-09T03:54:19.459 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 12, "last_destroyed": 14, "last_snap": 15, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [] } 2023-11-09T03:54:19.459 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:54:19.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:54:19.460 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2023-11-09T03:54:20.116 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:54:20.116+0000 7f6f94ff9640 -1 client.8934 I was blocklisted at osd epoch 116 2023-11-09T03:54:20.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:54:20.118 INFO:teuthology.orchestra.run.smithi076.stderr:mkdir: cannot create directory ‘d1/dir/.snap/s33’: Cannot send after transport endpoint shutdown 2023-11-09T03:54:20.118 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:54:20.119 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:54:20.135 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:54:20.235 INFO:tasks.ceph.mds.c.smithi076.stderr:2023-11-09T03:54:20.235+0000 7f1a9c20f640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-09T03:54:20.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.279+0000 7fd566105640 1 Processor -- start 2023-11-09T03:54:20.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.279+0000 7fd566105640 1 -- start start 2023-11-09T03:54:20.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.280+0000 7fd566105640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd56015d100 0x7fd5600ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:20.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.280+0000 7fd566105640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd56015d4e0 0x7fd5600d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:20.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.280+0000 7fd566105640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5600b32b0 con 0x7fd5600d3b60 2023-11-09T03:54:20.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.280+0000 7fd566105640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5600cee10 con 0x7fd56015d4e0 2023-11-09T03:54:20.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.280+0000 7fd566105640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5600b4750 con 0x7fd56015d100 2023-11-09T03:54:20.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.280+0000 7fd55effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd56015d4e0 0x7fd5600d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:20.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.280+0000 7fd55effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd56015d4e0 0x7fd5600d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57214/0 (socket says 172.21.15.76:57214) 2023-11-09T03:54:20.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.280+0000 7fd55effd640 1 -- 172.21.15.76:0/2258056278 learned_addr learned my addr 172.21.15.76:0/2258056278 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:20.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.280+0000 7fd55f7fe640 1 --2- 172.21.15.76:0/2258056278 >> v2:172.21.15.113:3300/0 conn(0x7fd56015d100 0x7fd5600ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:20.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.280+0000 7fd55effd640 1 -- 172.21.15.76:0/2258056278 >> v2:172.21.15.113:3300/0 conn(0x7fd56015d100 msgr2=0x7fd5600ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:20.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.280+0000 7fd55effd640 1 --2- 172.21.15.76:0/2258056278 >> v2:172.21.15.113:3300/0 conn(0x7fd56015d100 0x7fd5600ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:20.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.280+0000 7fd55effd640 1 -- 172.21.15.76:0/2258056278 >> v1:172.21.15.113:6789/0 conn(0x7fd5600d3b60 legacy=0x7fd5600cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:20.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.280+0000 7fd55effd640 1 -- 172.21.15.76:0/2258056278 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5600b2980 con 0x7fd56015d4e0 2023-11-09T03:54:20.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.280+0000 7fd55f7fe640 1 --2- 172.21.15.76:0/2258056278 >> v2:172.21.15.113:3300/0 conn(0x7fd56015d100 0x7fd5600ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:54:20.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.281+0000 7fd55effd640 1 --2- 172.21.15.76:0/2258056278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd56015d4e0 0x7fd5600d14d0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fd5600d09f0 tx=0x7fd55402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20affa017fcdadc3 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:20.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.281+0000 7fd55e7fc640 1 -- 172.21.15.76:0/2258056278 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd554031870 con 0x7fd56015d4e0 2023-11-09T03:54:20.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.281+0000 7fd55e7fc640 1 -- 172.21.15.76:0/2258056278 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5540319d0 con 0x7fd56015d4e0 2023-11-09T03:54:20.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.281+0000 7fd55e7fc640 1 -- 172.21.15.76:0/2258056278 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd554031d60 con 0x7fd56015d4e0 2023-11-09T03:54:20.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.281+0000 7fd566105640 1 -- 172.21.15.76:0/2258056278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd56015d4e0 msgr2=0x7fd5600d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:20.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.281+0000 7fd566105640 1 --2- 172.21.15.76:0/2258056278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd56015d4e0 0x7fd5600d14d0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fd5600d09f0 tx=0x7fd55402b910 comp rx=0 tx=0).stop 2023-11-09T03:54:20.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.281+0000 7fd566105640 1 -- 172.21.15.76:0/2258056278 shutdown_connections 2023-11-09T03:54:20.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.282+0000 7fd566105640 1 --2- 172.21.15.76:0/2258056278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd56015d4e0 0x7fd5600d14d0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:20.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.282+0000 7fd566105640 1 --2- 172.21.15.76:0/2258056278 >> v2:172.21.15.113:3300/0 conn(0x7fd56015d100 0x7fd5600ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:20.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.282+0000 7fd566105640 1 -- 172.21.15.76:0/2258056278 >> 172.21.15.76:0/2258056278 conn(0x7fd5600d2ef0 msgr2=0x7fd5600d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:20.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.282+0000 7fd566105640 1 -- 172.21.15.76:0/2258056278 shutdown_connections 2023-11-09T03:54:20.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.282+0000 7fd566105640 1 -- 172.21.15.76:0/2258056278 wait complete. 2023-11-09T03:54:20.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.283+0000 7fd566105640 1 Processor -- start 2023-11-09T03:54:20.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.283+0000 7fd566105640 1 -- start start 2023-11-09T03:54:20.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.284+0000 7fd566105640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5600d3b60 0x7fd56015dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:20.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.284+0000 7fd566105640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd56015d100 0x7fd5600b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:20.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.284+0000 7fd566105640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd56015d4e0 0x7fd5600b6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:20.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.284+0000 7fd566105640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5600cd070 con 0x7fd5600d3b60 2023-11-09T03:54:20.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.284+0000 7fd566105640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5600cc7d0 con 0x7fd56015d100 2023-11-09T03:54:20.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.284+0000 7fd566105640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5600ccab0 con 0x7fd56015d4e0 2023-11-09T03:54:20.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.284+0000 7fd55f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5600d3b60 0x7fd56015dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:20.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.284+0000 7fd55f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5600d3b60 0x7fd56015dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57218/0 (socket says 172.21.15.76:57218) 2023-11-09T03:54:20.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.284+0000 7fd55f7fe640 1 -- 172.21.15.76:0/1922280828 learned_addr learned my addr 172.21.15.76:0/1922280828 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:20.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.284+0000 7fd55effd640 1 --2- 172.21.15.76:0/1922280828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd56015d100 0x7fd5600b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:20.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.284+0000 7fd55ffff640 1 --2- 172.21.15.76:0/1922280828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd56015d4e0 0x7fd5600b6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:20.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.284+0000 7fd55f7fe640 1 -- 172.21.15.76:0/1922280828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd56015d4e0 msgr2=0x7fd5600b6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:20.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.284+0000 7fd55f7fe640 1 --2- 172.21.15.76:0/1922280828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd56015d4e0 0x7fd5600b6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:20.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.284+0000 7fd55f7fe640 1 -- 172.21.15.76:0/1922280828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd56015d100 msgr2=0x7fd5600b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:20.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.284+0000 7fd55f7fe640 1 --2- 172.21.15.76:0/1922280828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd56015d100 0x7fd5600b5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:20.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.284+0000 7fd55f7fe640 1 -- 172.21.15.76:0/1922280828 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5540094d0 con 0x7fd5600d3b60 2023-11-09T03:54:20.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.284+0000 7fd55ffff640 1 --2- 172.21.15.76:0/1922280828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd56015d4e0 0x7fd5600b6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:54:20.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.284+0000 7fd55f7fe640 1 --2- 172.21.15.76:0/1922280828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5600d3b60 0x7fd56015dd90 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fd55000c7f0 tx=0x7fd55000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:20.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.284+0000 7fd547fff640 1 -- 172.21.15.76:0/1922280828 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd550004220 con 0x7fd5600d3b60 2023-11-09T03:54:20.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.284+0000 7fd547fff640 1 -- 172.21.15.76:0/1922280828 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd550004380 con 0x7fd5600d3b60 2023-11-09T03:54:20.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.284+0000 7fd566105640 1 -- 172.21.15.76:0/1922280828 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5600ccd90 con 0x7fd5600d3b60 2023-11-09T03:54:20.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.284+0000 7fd547fff640 1 -- 172.21.15.76:0/1922280828 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd550005bb0 con 0x7fd5600d3b60 2023-11-09T03:54:20.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.284+0000 7fd566105640 1 -- 172.21.15.76:0/1922280828 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5600ca180 con 0x7fd5600d3b60 2023-11-09T03:54:20.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.285+0000 7fd566105640 1 -- 172.21.15.76:0/1922280828 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5600ca310 con 0x7fd5600d3b60 2023-11-09T03:54:20.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.286+0000 7fd547fff640 1 -- 172.21.15.76:0/1922280828 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd550006030 con 0x7fd5600d3b60 2023-11-09T03:54:20.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.286+0000 7fd547fff640 1 --2- 172.21.15.76:0/1922280828 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd54c06aea0 0x7fd54c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:20.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.286+0000 7fd547fff640 1 -- 172.21.15.76:0/1922280828 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7fd55008a3d0 con 0x7fd5600d3b60 2023-11-09T03:54:20.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.286+0000 7fd55effd640 1 --2- 172.21.15.76:0/1922280828 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd54c06aea0 0x7fd54c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:20.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.287+0000 7fd55effd640 1 --2- 172.21.15.76:0/1922280828 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd54c06aea0 0x7fd54c06d360 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fd554036000 tx=0x7fd55403d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:20.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.290+0000 7fd547fff640 1 -- 172.21.15.76:0/1922280828 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd550054aa0 con 0x7fd5600d3b60 2023-11-09T03:54:20.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.458+0000 7fd566105640 1 -- 172.21.15.76:0/1922280828 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5600684b0 con 0x7fd5600d3b60 2023-11-09T03:54:20.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.459+0000 7fd547fff640 1 -- 172.21.15.76:0/1922280828 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+23842 (secure 0 0 0) 0x7fd5500580c0 con 0x7fd5600d3b60 2023-11-09T03:54:20.460 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:54:20.460 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":116,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:54:15.604172+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-09T03:47:59.280601+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-09T03:47:59.742039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:48:55.490832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-09T03:48:56.352624+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":14,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.17","mappings":[{"from":0,"to":2}]},{"pgid":"17.36","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:6834/2075292158":"2023-11-10T03:54:06.568191+0000","192.168.0.1:0/334503200":"2023-11-09T04:53:32.572840+0000","172.21.15.76:6835/2075292158":"2023-11-10T03:54:06.568191+0000","172.21.15.113:6837/7360844":"2023-11-10T03:53:24.718008+0000","172.21.15.113:6836/7360844":"2023-11-10T03:53:24.718008+0000","172.21.15.76:6835/546338722":"2023-11-10T03:53:17.470106+0000","172.21.15.76:6834/546338722":"2023-11-10T03:53:17.470106+0000","192.168.0.1:0/3087804343":"2023-11-09T04:52:42.513556+0000","172.21.15.76:6833/1536527256":"2023-11-10T03:52:34.213804+0000","172.21.15.113:6837/445306598":"2023-11-10T03:52:27.406838+0000","172.21.15.76:6834/3229239133":"2023-11-10T03:51:32.624856+0000","192.168.0.1:0/503424885":"2023-11-09T04:51:47.727422+0000","172.21.15.76:6834/4011275120":"2023-11-10T03:49:42.029057+0000","172.21.15.76:6835/4011275120":"2023-11-10T03:49:42.029057+0000","192.168.0.1:0/2519773110":"2023-11-09T04:54:14.823765+0000","172.21.15.113:6836/3280098776":"2023-11-10T03:50:07.616780+0000","172.21.15.113:6837/3280098776":"2023-11-10T03:50:07.616780+0000","172.21.15.76:6834/1663988060":"2023-11-10T03:50:33.665092+0000","172.21.15.76:6835/1663988060":"2023-11-10T03:50:33.665092+0000","172.21.15.113:6836/445306598":"2023-11-10T03:52:27.406838+0000","172.21.15.76:6833/3697850101":"2023-11-10T03:51:39.442929+0000","172.21.15.113:6836/3449910842":"2023-11-10T03:50:59.611256+0000","172.21.15.76:6835/3229239133":"2023-11-10T03:51:32.624856+0000","172.21.15.76:6832/1536527256":"2023-11-10T03:52:34.213804+0000","172.21.15.113:6837/3449910842":"2023-11-10T03:50:59.611256+0000","172.21.15.76:6832/3697850101":"2023-11-10T03:51:39.442929+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:54:20.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.461+0000 7fd566105640 1 -- 172.21.15.76:0/1922280828 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd54c06aea0 msgr2=0x7fd54c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:20.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.461+0000 7fd566105640 1 --2- 172.21.15.76:0/1922280828 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd54c06aea0 0x7fd54c06d360 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fd554036000 tx=0x7fd55403d040 comp rx=0 tx=0).stop 2023-11-09T03:54:20.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.461+0000 7fd566105640 1 -- 172.21.15.76:0/1922280828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5600d3b60 msgr2=0x7fd56015dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:20.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.461+0000 7fd566105640 1 --2- 172.21.15.76:0/1922280828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5600d3b60 0x7fd56015dd90 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fd55000c7f0 tx=0x7fd55000ccb0 comp rx=0 tx=0).stop 2023-11-09T03:54:20.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.461+0000 7fd566105640 1 -- 172.21.15.76:0/1922280828 shutdown_connections 2023-11-09T03:54:20.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.461+0000 7fd566105640 1 --2- 172.21.15.76:0/1922280828 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd54c06aea0 0x7fd54c06d360 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:20.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.461+0000 7fd566105640 1 --2- 172.21.15.76:0/1922280828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd56015d4e0 0x7fd5600b6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:20.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.461+0000 7fd566105640 1 --2- 172.21.15.76:0/1922280828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd56015d100 0x7fd5600b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:20.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.461+0000 7fd566105640 1 --2- 172.21.15.76:0/1922280828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5600d3b60 0x7fd56015dd90 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:20.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.461+0000 7fd566105640 1 -- 172.21.15.76:0/1922280828 >> 172.21.15.76:0/1922280828 conn(0x7fd5600d2ef0 msgr2=0x7fd56006ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:20.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.462+0000 7fd566105640 1 -- 172.21.15.76:0/1922280828 shutdown_connections 2023-11-09T03:54:20.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.462+0000 7fd566105640 1 -- 172.21.15.76:0/1922280828 wait complete. 2023-11-09T03:54:20.473 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-09T03:47:00.036540+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_data_pool', 'create_time': '2023-11-09T03:47:59.280601+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 70, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-09T03:47:59.742039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:48:55.490832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:48:56.352624+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 14, 'snap_epoch': 113, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-09T03:54:20.475 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:54:20.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.577+0000 7f5415cbb640 1 Processor -- start 2023-11-09T03:54:20.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.577+0000 7f5415cbb640 1 -- start start 2023-11-09T03:54:20.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.578+0000 7f5415cbb640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f541015d100 0x7f54100ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:20.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.578+0000 7f5415cbb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54100d1be0 0x7f54100b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:20.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.578+0000 7f5415cbb640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54100b32b0 con 0x7f541015d4e0 2023-11-09T03:54:20.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.578+0000 7f5415cbb640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54100cee10 con 0x7f541015d100 2023-11-09T03:54:20.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.578+0000 7f5415cbb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54100b2980 con 0x7f54100d1be0 2023-11-09T03:54:20.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.578+0000 7f540f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f541015d100 0x7f54100ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:20.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.578+0000 7f540f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f541015d100 0x7f54100ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57222/0 (socket says 172.21.15.76:57222) 2023-11-09T03:54:20.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.578+0000 7f540f7fe640 1 -- 172.21.15.76:0/1265922273 learned_addr learned my addr 172.21.15.76:0/1265922273 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:20.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.579+0000 7f540ffff640 1 --2- 172.21.15.76:0/1265922273 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54100d1be0 0x7f54100b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:20.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.579+0000 7f540f7fe640 1 -- 172.21.15.76:0/1265922273 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54100d1be0 msgr2=0x7f54100b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:20.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.579+0000 7f540f7fe640 1 --2- 172.21.15.76:0/1265922273 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54100d1be0 0x7f54100b3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:20.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.579+0000 7f540f7fe640 1 -- 172.21.15.76:0/1265922273 >> v1:172.21.15.113:6789/0 conn(0x7f541015d4e0 legacy=0x7f54100d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:20.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.579+0000 7f540f7fe640 1 -- 172.21.15.76:0/1265922273 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54100b2c60 con 0x7f541015d100 2023-11-09T03:54:20.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.579+0000 7f540ffff640 1 --2- 172.21.15.76:0/1265922273 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54100d1be0 0x7f54100b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:54:20.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.579+0000 7f540f7fe640 1 --2- 172.21.15.76:0/1265922273 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f541015d100 0x7f54100ce510 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f5400009850 tx=0x7f540002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec0a9999a7e59e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:20.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.580+0000 7f540effd640 1 -- 172.21.15.76:0/1265922273 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5400004230 con 0x7f541015d100 2023-11-09T03:54:20.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.580+0000 7f540effd640 1 -- 172.21.15.76:0/1265922273 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5400004390 con 0x7f541015d100 2023-11-09T03:54:20.584 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.580+0000 7f540effd640 1 -- 172.21.15.76:0/1265922273 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5400031830 con 0x7f541015d100 2023-11-09T03:54:20.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.580+0000 7f5415cbb640 1 -- 172.21.15.76:0/1265922273 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f541015d100 msgr2=0x7f54100ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:20.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.580+0000 7f5415cbb640 1 --2- 172.21.15.76:0/1265922273 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f541015d100 0x7f54100ce510 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f5400009850 tx=0x7f540002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:54:20.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.580+0000 7f5415cbb640 1 -- 172.21.15.76:0/1265922273 shutdown_connections 2023-11-09T03:54:20.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.580+0000 7f5415cbb640 1 --2- 172.21.15.76:0/1265922273 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54100d1be0 0x7f54100b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:20.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.580+0000 7f5415cbb640 1 --2- 172.21.15.76:0/1265922273 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f541015d100 0x7f54100ce510 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:20.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.580+0000 7f5415cbb640 1 -- 172.21.15.76:0/1265922273 >> 172.21.15.76:0/1265922273 conn(0x7f54100d2ef0 msgr2=0x7f54100d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:20.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.580+0000 7f5415cbb640 1 -- 172.21.15.76:0/1265922273 shutdown_connections 2023-11-09T03:54:20.588 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.580+0000 7f5415cbb640 1 -- 172.21.15.76:0/1265922273 wait complete. 2023-11-09T03:54:20.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.581+0000 7f5415cbb640 1 Processor -- start 2023-11-09T03:54:20.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.581+0000 7f5415cbb640 1 -- start start 2023-11-09T03:54:20.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.582+0000 7f5415cbb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54100d1be0 0x7f5410158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:20.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.582+0000 7f5415cbb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f541015d100 0x7f54100b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:20.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.582+0000 7f5415cbb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f541015d4e0 0x7f54101549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:20.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.582+0000 7f5415cbb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5410154f10 con 0x7f541015d100 2023-11-09T03:54:20.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.582+0000 7f5415cbb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5410159880 con 0x7f541015d4e0 2023-11-09T03:54:20.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.582+0000 7f5415cbb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5410158c70 con 0x7f54100d1be0 2023-11-09T03:54:20.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.582+0000 7f5406ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f541015d100 0x7f54100b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:20.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.582+0000 7f5406ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f541015d100 0x7f54100b5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57228/0 (socket says 172.21.15.76:57228) 2023-11-09T03:54:20.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.582+0000 7f5406ffd640 1 -- 172.21.15.76:0/303148727 learned_addr learned my addr 172.21.15.76:0/303148727 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:20.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.582+0000 7f540f7fe640 1 --2- 172.21.15.76:0/303148727 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54100d1be0 0x7f5410158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:20.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.582+0000 7f540ffff640 1 --2- 172.21.15.76:0/303148727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f541015d4e0 0x7f54101549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:20.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.583+0000 7f5406ffd640 1 -- 172.21.15.76:0/303148727 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54100d1be0 msgr2=0x7f5410158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:20.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.583+0000 7f5406ffd640 1 --2- 172.21.15.76:0/303148727 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54100d1be0 0x7f5410158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:20.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.583+0000 7f5406ffd640 1 -- 172.21.15.76:0/303148727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f541015d4e0 msgr2=0x7f54101549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:20.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.583+0000 7f5406ffd640 1 --2- 172.21.15.76:0/303148727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f541015d4e0 0x7f54101549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:20.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.583+0000 7f5406ffd640 1 -- 172.21.15.76:0/303148727 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5400009460 con 0x7f541015d100 2023-11-09T03:54:20.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.583+0000 7f5406ffd640 1 --2- 172.21.15.76:0/303148727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f541015d100 0x7f54100b5a70 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f53f800b3c0 tx=0x7f53f800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:20.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.583+0000 7f540cff9640 1 -- 172.21.15.76:0/303148727 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f8011830 con 0x7f541015d100 2023-11-09T03:54:20.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.583+0000 7f540cff9640 1 -- 172.21.15.76:0/303148727 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f8011990 con 0x7f541015d100 2023-11-09T03:54:20.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.583+0000 7f540cff9640 1 -- 172.21.15.76:0/303148727 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f8011d40 con 0x7f541015d100 2023-11-09T03:54:20.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.583+0000 7f5415cbb640 1 -- 172.21.15.76:0/303148727 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5410158f50 con 0x7f541015d100 2023-11-09T03:54:20.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.583+0000 7f5415cbb640 1 -- 172.21.15.76:0/303148727 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5410159510 con 0x7f541015d100 2023-11-09T03:54:20.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.584+0000 7f5415cbb640 1 -- 172.21.15.76:0/303148727 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54101599f0 con 0x7f541015d100 2023-11-09T03:54:20.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.584+0000 7f540cff9640 1 -- 172.21.15.76:0/303148727 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f53f801b460 con 0x7f541015d100 2023-11-09T03:54:20.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.585+0000 7f540cff9640 1 --2- 172.21.15.76:0/303148727 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f53f006ae70 0x7f53f006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:20.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.585+0000 7f540cff9640 1 -- 172.21.15.76:0/303148727 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7f53f80899b0 con 0x7f541015d100 2023-11-09T03:54:20.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.585+0000 7f540f7fe640 1 --2- 172.21.15.76:0/303148727 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f53f006ae70 0x7f53f006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:20.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.586+0000 7f540f7fe640 1 --2- 172.21.15.76:0/303148727 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f53f006ae70 0x7f53f006d330 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5400035000 tx=0x7f5400008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:20.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.589+0000 7f540cff9640 1 -- 172.21.15.76:0/303148727 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f53f8054130 con 0x7f541015d100 2023-11-09T03:54:20.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.757+0000 7f5415cbb640 1 -- 172.21.15.76:0/303148727 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f54101596a0 con 0x7f541015d100 2023-11-09T03:54:20.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.757+0000 7f540cff9640 1 -- 172.21.15.76:0/303148727 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) v1 ==== 93+0+10 (secure 0 0 0) 0x7f53f8057750 con 0x7f541015d100 2023-11-09T03:54:20.758 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:54:20.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.759+0000 7f5415cbb640 1 -- 172.21.15.76:0/303148727 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f53f006ae70 msgr2=0x7f53f006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:20.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.759+0000 7f5415cbb640 1 --2- 172.21.15.76:0/303148727 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f53f006ae70 0x7f53f006d330 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5400035000 tx=0x7f5400008030 comp rx=0 tx=0).stop 2023-11-09T03:54:20.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.759+0000 7f5415cbb640 1 -- 172.21.15.76:0/303148727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f541015d100 msgr2=0x7f54100b5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:20.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.759+0000 7f5415cbb640 1 --2- 172.21.15.76:0/303148727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f541015d100 0x7f54100b5a70 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f53f800b3c0 tx=0x7f53f800b880 comp rx=0 tx=0).stop 2023-11-09T03:54:20.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.759+0000 7f5415cbb640 1 -- 172.21.15.76:0/303148727 shutdown_connections 2023-11-09T03:54:20.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.759+0000 7f5415cbb640 1 --2- 172.21.15.76:0/303148727 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f53f006ae70 0x7f53f006d330 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:20.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.759+0000 7f5415cbb640 1 --2- 172.21.15.76:0/303148727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f541015d4e0 0x7f54101549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:20.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.759+0000 7f5415cbb640 1 --2- 172.21.15.76:0/303148727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f541015d100 0x7f54100b5a70 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:20.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.759+0000 7f5415cbb640 1 --2- 172.21.15.76:0/303148727 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54100d1be0 0x7f5410158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:20.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.759+0000 7f5415cbb640 1 -- 172.21.15.76:0/303148727 >> 172.21.15.76:0/303148727 conn(0x7f54100d2ef0 msgr2=0x7f54100b4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:20.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.759+0000 7f5415cbb640 1 -- 172.21.15.76:0/303148727 shutdown_connections 2023-11-09T03:54:20.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.759+0000 7f5415cbb640 1 -- 172.21.15.76:0/303148727 wait complete. 2023-11-09T03:54:20.770 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-09T03:54:20.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.872+0000 7f9ca658a640 1 Processor -- start 2023-11-09T03:54:20.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.872+0000 7f9ca658a640 1 -- start start 2023-11-09T03:54:20.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.873+0000 7f9ca658a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca0159e40 0x7f9ca00b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:20.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.873+0000 7f9ca658a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9ca006bb60 0x7f9ca00b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:20.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.873+0000 7f9ca658a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca0071500 con 0x7f9ca006fff0 2023-11-09T03:54:20.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.873+0000 7f9ca658a640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca015dae0 con 0x7f9ca006bb60 2023-11-09T03:54:20.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.873+0000 7f9ca658a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca0070c60 con 0x7f9ca0159e40 2023-11-09T03:54:20.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.873+0000 7f9c9f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9ca006bb60 0x7f9ca00b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:20.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.873+0000 7f9c9f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9ca006bb60 0x7f9ca00b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57244/0 (socket says 172.21.15.76:57244) 2023-11-09T03:54:20.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.873+0000 7f9c9f7fe640 1 -- 172.21.15.76:0/4019791202 learned_addr learned my addr 172.21.15.76:0/4019791202 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:20.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.873+0000 7f9c9ffff640 1 --2- 172.21.15.76:0/4019791202 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca0159e40 0x7f9ca00b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:20.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.873+0000 7f9c9f7fe640 1 -- 172.21.15.76:0/4019791202 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca0159e40 msgr2=0x7f9ca00b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:20.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.874+0000 7f9c9f7fe640 1 --2- 172.21.15.76:0/4019791202 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca0159e40 0x7f9ca00b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:20.876 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.874+0000 7f9c9f7fe640 1 -- 172.21.15.76:0/4019791202 >> v1:172.21.15.113:6789/0 conn(0x7f9ca006fff0 legacy=0x7f9ca0070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:20.877 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.874+0000 7f9c9f7fe640 1 -- 172.21.15.76:0/4019791202 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca0070f40 con 0x7f9ca006bb60 2023-11-09T03:54:20.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.874+0000 7f9c9f7fe640 1 --2- 172.21.15.76:0/4019791202 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ca006bb60 0x7f9ca00b0a70 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f9c90009870 tx=0x7f9c9002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7277375e17ccd8d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:20.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.874+0000 7f9c9effd640 1 -- 172.21.15.76:0/4019791202 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c90031640 con 0x7f9ca006bb60 2023-11-09T03:54:20.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.874+0000 7f9c9effd640 1 -- 172.21.15.76:0/4019791202 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c900317a0 con 0x7f9ca006bb60 2023-11-09T03:54:20.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.874+0000 7f9c9effd640 1 -- 172.21.15.76:0/4019791202 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c90031b30 con 0x7f9ca006bb60 2023-11-09T03:54:20.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.875+0000 7f9ca658a640 1 -- 172.21.15.76:0/4019791202 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ca006bb60 msgr2=0x7f9ca00b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:20.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.875+0000 7f9ca658a640 1 --2- 172.21.15.76:0/4019791202 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ca006bb60 0x7f9ca00b0a70 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f9c90009870 tx=0x7f9c9002b8f0 comp rx=0 tx=0).stop 2023-11-09T03:54:20.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.875+0000 7f9ca658a640 1 -- 172.21.15.76:0/4019791202 shutdown_connections 2023-11-09T03:54:20.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.875+0000 7f9ca658a640 1 --2- 172.21.15.76:0/4019791202 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ca006bb60 0x7f9ca00b0a70 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:20.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.875+0000 7f9ca658a640 1 --2- 172.21.15.76:0/4019791202 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca0159e40 0x7f9ca00b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:20.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.875+0000 7f9ca658a640 1 -- 172.21.15.76:0/4019791202 >> 172.21.15.76:0/4019791202 conn(0x7f9ca00cb530 msgr2=0x7f9ca00cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:20.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.875+0000 7f9ca658a640 1 -- 172.21.15.76:0/4019791202 shutdown_connections 2023-11-09T03:54:20.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.875+0000 7f9ca658a640 1 -- 172.21.15.76:0/4019791202 wait complete. 2023-11-09T03:54:20.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.876+0000 7f9ca658a640 1 Processor -- start 2023-11-09T03:54:20.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.876+0000 7f9ca658a640 1 -- start start 2023-11-09T03:54:20.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.877+0000 7f9ca658a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ca006bb60 0x7f9ca01590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:20.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.877+0000 7f9ca658a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca006fff0 0x7f9ca01f1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:20.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.877+0000 7f9ca658a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ca0159e40 0x7f9ca00acff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:20.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.877+0000 7f9ca658a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca0159630 con 0x7f9ca0159e40 2023-11-09T03:54:20.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.877+0000 7f9ca658a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca00acb60 con 0x7f9ca006bb60 2023-11-09T03:54:20.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.877+0000 7f9ca658a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca00c4b10 con 0x7f9ca006fff0 2023-11-09T03:54:20.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.877+0000 7f9ca4b27640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ca0159e40 0x7f9ca00acff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:20.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.877+0000 7f9ca4b27640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ca0159e40 0x7f9ca00acff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57254/0 (socket says 172.21.15.76:57254) 2023-11-09T03:54:20.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.877+0000 7f9ca4b27640 1 -- 172.21.15.76:0/642371027 learned_addr learned my addr 172.21.15.76:0/642371027 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:20.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.877+0000 7f9c9f7fe640 1 --2- 172.21.15.76:0/642371027 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca006fff0 0x7f9ca01f1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:20.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.877+0000 7f9ca4b27640 1 -- 172.21.15.76:0/642371027 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca006fff0 msgr2=0x7f9ca01f1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:20.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.877+0000 7f9ca4b27640 1 --2- 172.21.15.76:0/642371027 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca006fff0 0x7f9ca01f1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:20.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.877+0000 7f9ca4b27640 1 -- 172.21.15.76:0/642371027 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ca006bb60 msgr2=0x7f9ca01590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:20.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.877+0000 7f9ca4b27640 1 --2- 172.21.15.76:0/642371027 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ca006bb60 0x7f9ca01590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:20.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.877+0000 7f9ca4b27640 1 -- 172.21.15.76:0/642371027 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c900094d0 con 0x7f9ca0159e40 2023-11-09T03:54:20.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.878+0000 7f9ca4b27640 1 --2- 172.21.15.76:0/642371027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ca0159e40 0x7f9ca00acff0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800c760 tx=0x7f9c9800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:20.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.878+0000 7f9c9cff9640 1 -- 172.21.15.76:0/642371027 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c98003b70 con 0x7f9ca0159e40 2023-11-09T03:54:20.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.878+0000 7f9c9cff9640 1 -- 172.21.15.76:0/642371027 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c98003cd0 con 0x7f9ca0159e40 2023-11-09T03:54:20.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.878+0000 7f9ca658a640 1 -- 172.21.15.76:0/642371027 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca01f18a0 con 0x7f9ca0159e40 2023-11-09T03:54:20.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.878+0000 7f9c9cff9640 1 -- 172.21.15.76:0/642371027 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9800aa90 con 0x7f9ca0159e40 2023-11-09T03:54:20.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.878+0000 7f9ca658a640 1 -- 172.21.15.76:0/642371027 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca00ac230 con 0x7f9ca0159e40 2023-11-09T03:54:20.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.879+0000 7f9ca658a640 1 -- 172.21.15.76:0/642371027 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca00cd750 con 0x7f9ca0159e40 2023-11-09T03:54:20.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.879+0000 7f9c9cff9640 1 -- 172.21.15.76:0/642371027 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9c9800abf0 con 0x7f9ca0159e40 2023-11-09T03:54:20.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.879+0000 7f9c9cff9640 1 --2- 172.21.15.76:0/642371027 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c8c06ae70 0x7f9c8c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:20.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.880+0000 7f9c9cff9640 1 -- 172.21.15.76:0/642371027 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7f9c9808b190 con 0x7f9ca0159e40 2023-11-09T03:54:20.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.880+0000 7f9c9ffff640 1 --2- 172.21.15.76:0/642371027 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c8c06ae70 0x7f9c8c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:20.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.880+0000 7f9c9ffff640 1 --2- 172.21.15.76:0/642371027 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c8c06ae70 0x7f9c8c06d330 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f9c94006fd0 tx=0x7f9c94008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:20.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:20.884+0000 7f9c9cff9640 1 -- 172.21.15.76:0/642371027 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9c980544f0 con 0x7f9ca0159e40 2023-11-09T03:54:21.049 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.049+0000 7f9ca658a640 1 -- 172.21.15.76:0/642371027 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f9ca00684b0 con 0x7f9ca0159e40 2023-11-09T03:54:21.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.049+0000 7f9c9cff9640 1 -- 172.21.15.76:0/642371027 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v116) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9c98057b10 con 0x7f9ca0159e40 2023-11-09T03:54:21.050 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:54:21.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.051+0000 7f9ca658a640 1 -- 172.21.15.76:0/642371027 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c8c06ae70 msgr2=0x7f9c8c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:21.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.051+0000 7f9ca658a640 1 --2- 172.21.15.76:0/642371027 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c8c06ae70 0x7f9c8c06d330 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f9c94006fd0 tx=0x7f9c94008040 comp rx=0 tx=0).stop 2023-11-09T03:54:21.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.051+0000 7f9ca658a640 1 -- 172.21.15.76:0/642371027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ca0159e40 msgr2=0x7f9ca00acff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:21.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.051+0000 7f9ca658a640 1 --2- 172.21.15.76:0/642371027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ca0159e40 0x7f9ca00acff0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800c760 tx=0x7f9c9800cc20 comp rx=0 tx=0).stop 2023-11-09T03:54:21.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.051+0000 7f9ca658a640 1 -- 172.21.15.76:0/642371027 shutdown_connections 2023-11-09T03:54:21.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.051+0000 7f9ca658a640 1 --2- 172.21.15.76:0/642371027 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c8c06ae70 0x7f9c8c06d330 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.051+0000 7f9ca658a640 1 --2- 172.21.15.76:0/642371027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ca0159e40 0x7f9ca00acff0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.051+0000 7f9ca658a640 1 --2- 172.21.15.76:0/642371027 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca006fff0 0x7f9ca01f1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.051+0000 7f9ca658a640 1 --2- 172.21.15.76:0/642371027 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ca006bb60 0x7f9ca01590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.051+0000 7f9ca658a640 1 -- 172.21.15.76:0/642371027 >> 172.21.15.76:0/642371027 conn(0x7f9ca00cb530 msgr2=0x7f9ca015c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:21.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.051+0000 7f9ca658a640 1 -- 172.21.15.76:0/642371027 shutdown_connections 2023-11-09T03:54:21.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.051+0000 7f9ca658a640 1 -- 172.21.15.76:0/642371027 wait complete. 2023-11-09T03:54:21.062 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-09T03:54:21.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.164+0000 7f99b244b640 1 Processor -- start 2023-11-09T03:54:21.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.164+0000 7f99b244b640 1 -- start start 2023-11-09T03:54:21.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.165+0000 7f99b244b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f99ac0d1be0 0x7f99ac167e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:21.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.165+0000 7f99b244b640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99ac0b32b0 con 0x7f99ac15d100 2023-11-09T03:54:21.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.165+0000 7f99b244b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99ac0cee10 con 0x7f99ac15d4e0 2023-11-09T03:54:21.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.165+0000 7f99b244b640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99ac0cec20 con 0x7f99ac0d1be0 2023-11-09T03:54:21.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.166+0000 7f99b09e8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f99ac0d1be0 0x7f99ac167e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:21.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.166+0000 7f99b09e8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f99ac0d1be0 0x7f99ac167e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57270/0 (socket says 172.21.15.76:57270) 2023-11-09T03:54:21.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.166+0000 7f99abfff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f99ac15d100 0x7f99ac0ce510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45794/0 (socket says 172.21.15.76:45794) 2023-11-09T03:54:21.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.166+0000 7f99b09e8640 1 -- 172.21.15.76:0/3889064785 learned_addr learned my addr 172.21.15.76:0/3889064785 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:21.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.166+0000 7f99b09e8640 1 -- 172.21.15.76:0/3889064785 >> v1:172.21.15.113:6789/0 conn(0x7f99ac15d4e0 legacy=0x7f99ac0d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:21.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.166+0000 7f99b09e8640 1 -- 172.21.15.76:0/3889064785 >> v1:172.21.15.76:6789/0 conn(0x7f99ac15d100 legacy=0x7f99ac0ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:21.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.166+0000 7f99b09e8640 1 -- 172.21.15.76:0/3889064785 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ac0b4750 con 0x7f99ac0d1be0 2023-11-09T03:54:21.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.166+0000 7f99aaffd640 1 -- 172.21.15.76:0/3889064785 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1401516020 0 0) 0x7f99ac0b32b0 con 0x7f99ac15d100 2023-11-09T03:54:21.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.167+0000 7f99b09e8640 1 --2- 172.21.15.76:0/3889064785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99ac0d1be0 0x7f99ac167e80 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009e50 tx=0x7f99a402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c534720fd78a41c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:21.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.167+0000 7f99aaffd640 1 -- 172.21.15.76:0/3889064785 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a402cb20 con 0x7f99ac0d1be0 2023-11-09T03:54:21.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.167+0000 7f99aaffd640 1 -- 172.21.15.76:0/3889064785 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a402cc80 con 0x7f99ac0d1be0 2023-11-09T03:54:21.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.167+0000 7f99aaffd640 1 -- 172.21.15.76:0/3889064785 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a40343f0 con 0x7f99ac0d1be0 2023-11-09T03:54:21.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.167+0000 7f99b244b640 1 -- 172.21.15.76:0/3889064785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99ac0d1be0 msgr2=0x7f99ac167e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:21.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.167+0000 7f99b244b640 1 --2- 172.21.15.76:0/3889064785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99ac0d1be0 0x7f99ac167e80 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009e50 tx=0x7f99a402bf80 comp rx=0 tx=0).stop 2023-11-09T03:54:21.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.167+0000 7f99b244b640 1 -- 172.21.15.76:0/3889064785 shutdown_connections 2023-11-09T03:54:21.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.167+0000 7f99b244b640 1 --2- 172.21.15.76:0/3889064785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99ac0d1be0 0x7f99ac167e80 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.167+0000 7f99b244b640 1 -- 172.21.15.76:0/3889064785 >> 172.21.15.76:0/3889064785 conn(0x7f99ac0d2ef0 msgr2=0x7f99ac0d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:21.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.167+0000 7f99b244b640 1 -- 172.21.15.76:0/3889064785 shutdown_connections 2023-11-09T03:54:21.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.167+0000 7f99b244b640 1 -- 172.21.15.76:0/3889064785 wait complete. 2023-11-09T03:54:21.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.168+0000 7f99b244b640 1 Processor -- start 2023-11-09T03:54:21.172 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.169+0000 7f99b244b640 1 -- start start 2023-11-09T03:54:21.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.169+0000 7f99b244b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99ac0d1be0 0x7f99ac15b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:21.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.169+0000 7f99b244b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99ac15d100 0x7f99ac1f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:21.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.169+0000 7f99b244b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99ac15d4e0 0x7f99ac0ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:21.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.169+0000 7f99b244b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac0d38f0 con 0x7f99ac15d100 2023-11-09T03:54:21.174 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.169+0000 7f99b244b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac0b6c40 con 0x7f99ac15d4e0 2023-11-09T03:54:21.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.169+0000 7f99b244b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac1f67a0 con 0x7f99ac0d1be0 2023-11-09T03:54:21.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.169+0000 7f99ab7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99ac15d100 0x7f99ac1f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:21.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.170+0000 7f99ab7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99ac15d100 0x7f99ac1f60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57278/0 (socket says 172.21.15.76:57278) 2023-11-09T03:54:21.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.170+0000 7f99ab7fe640 1 -- 172.21.15.76:0/1803926563 learned_addr learned my addr 172.21.15.76:0/1803926563 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:21.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.170+0000 7f99abfff640 1 --2- 172.21.15.76:0/1803926563 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99ac0d1be0 0x7f99ac15b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:21.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.170+0000 7f99b09e8640 1 --2- 172.21.15.76:0/1803926563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99ac15d4e0 0x7f99ac0ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:21.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.170+0000 7f99ab7fe640 1 -- 172.21.15.76:0/1803926563 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99ac0d1be0 msgr2=0x7f99ac15b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:21.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.170+0000 7f99ab7fe640 1 --2- 172.21.15.76:0/1803926563 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99ac0d1be0 0x7f99ac15b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.170+0000 7f99ab7fe640 1 -- 172.21.15.76:0/1803926563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99ac15d4e0 msgr2=0x7f99ac0ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:21.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.170+0000 7f99ab7fe640 1 --2- 172.21.15.76:0/1803926563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99ac15d4e0 0x7f99ac0ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.176 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.170+0000 7f99ab7fe640 1 -- 172.21.15.76:0/1803926563 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a4009b30 con 0x7f99ac15d100 2023-11-09T03:54:21.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.170+0000 7f99ab7fe640 1 --2- 172.21.15.76:0/1803926563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99ac15d100 0x7f99ac1f60c0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f99a000b3c0 tx=0x7f99a000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:21.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.170+0000 7f99a8ff9640 1 -- 172.21.15.76:0/1803926563 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a0011830 con 0x7f99ac15d100 2023-11-09T03:54:21.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.170+0000 7f99a8ff9640 1 -- 172.21.15.76:0/1803926563 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a0011990 con 0x7f99ac15d100 2023-11-09T03:54:21.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.170+0000 7f99b244b640 1 -- 172.21.15.76:0/1803926563 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99ac1f6a80 con 0x7f99ac15d100 2023-11-09T03:54:21.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.170+0000 7f99a8ff9640 1 -- 172.21.15.76:0/1803926563 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a0011d80 con 0x7f99ac15d100 2023-11-09T03:54:21.177 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.170+0000 7f99b244b640 1 -- 172.21.15.76:0/1803926563 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99ac0ca9d0 con 0x7f99ac15d100 2023-11-09T03:54:21.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.171+0000 7f99b244b640 1 -- 172.21.15.76:0/1803926563 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99ac0b6e60 con 0x7f99ac15d100 2023-11-09T03:54:21.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.172+0000 7f99a8ff9640 1 -- 172.21.15.76:0/1803926563 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f99a001b460 con 0x7f99ac15d100 2023-11-09T03:54:21.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.172+0000 7f99a8ff9640 1 --2- 172.21.15.76:0/1803926563 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f999806ae70 0x7f999806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:21.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.172+0000 7f99a8ff9640 1 -- 172.21.15.76:0/1803926563 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7f99a0089be0 con 0x7f99ac15d100 2023-11-09T03:54:21.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.172+0000 7f99abfff640 1 --2- 172.21.15.76:0/1803926563 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f999806ae70 0x7f999806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:21.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.173+0000 7f99abfff640 1 --2- 172.21.15.76:0/1803926563 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f999806ae70 0x7f999806d330 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f999c001700 tx=0x7f999c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:21.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.176+0000 7f99a8ff9640 1 -- 172.21.15.76:0/1803926563 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f99ac0b6e60 con 0x7f99ac15d100 2023-11-09T03:54:21.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.339+0000 7f99b244b640 1 -- 172.21.15.76:0/1803926563 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7f99ac0cac20 con 0x7f99ac15d100 2023-11-09T03:54:21.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.340+0000 7f99a8ff9640 1 -- 172.21.15.76:0/1803926563 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v116) v1 ==== 103+0+11 (secure 0 0 0) 0x7f99a0054220 con 0x7f99ac15d100 2023-11-09T03:54:21.340 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:54:21.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.341+0000 7f99b244b640 1 -- 172.21.15.76:0/1803926563 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f999806ae70 msgr2=0x7f999806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:21.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.341+0000 7f99b244b640 1 --2- 172.21.15.76:0/1803926563 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f999806ae70 0x7f999806d330 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f999c001700 tx=0x7f999c009040 comp rx=0 tx=0).stop 2023-11-09T03:54:21.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.341+0000 7f99b244b640 1 -- 172.21.15.76:0/1803926563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99ac15d100 msgr2=0x7f99ac1f60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:21.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.341+0000 7f99b244b640 1 --2- 172.21.15.76:0/1803926563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99ac15d100 0x7f99ac1f60c0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f99a000b3c0 tx=0x7f99a000b880 comp rx=0 tx=0).stop 2023-11-09T03:54:21.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.341+0000 7f99b244b640 1 -- 172.21.15.76:0/1803926563 shutdown_connections 2023-11-09T03:54:21.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.341+0000 7f99b244b640 1 --2- 172.21.15.76:0/1803926563 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f999806ae70 0x7f999806d330 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.341+0000 7f99b244b640 1 --2- 172.21.15.76:0/1803926563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99ac15d4e0 0x7f99ac0ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.341+0000 7f99b244b640 1 --2- 172.21.15.76:0/1803926563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99ac15d100 0x7f99ac1f60c0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.341+0000 7f99b244b640 1 --2- 172.21.15.76:0/1803926563 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99ac0d1be0 0x7f99ac15b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.341+0000 7f99b244b640 1 -- 172.21.15.76:0/1803926563 >> 172.21.15.76:0/1803926563 conn(0x7f99ac0d2ef0 msgr2=0x7f99ac15c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:21.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.342+0000 7f99b244b640 1 -- 172.21.15.76:0/1803926563 shutdown_connections 2023-11-09T03:54:21.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.342+0000 7f99b244b640 1 -- 172.21.15.76:0/1803926563 wait complete. 2023-11-09T03:54:21.353 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-09T03:54:21.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.454+0000 7f02ded55640 1 Processor -- start 2023-11-09T03:54:21.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.454+0000 7f02ded55640 1 -- start start 2023-11-09T03:54:21.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.455+0000 7f02ded55640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d8070700 0x7f02d8164b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:21.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.455+0000 7f02ded55640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02d8071500 con 0x7f02d80b0c40 2023-11-09T03:54:21.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.455+0000 7f02ded55640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02d815dae0 con 0x7f02d8159e40 2023-11-09T03:54:21.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.455+0000 7f02ded55640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d8070c60 con 0x7f02d8070700 2023-11-09T03:54:21.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.455+0000 7f02d7fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f02d80b0c40 0x7f02d806fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45804/0 (socket says 172.21.15.76:45804) 2023-11-09T03:54:21.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.455+0000 7f02d7fff640 1 -- 172.21.15.76:0/4175421724 learned_addr learned my addr 172.21.15.76:0/4175421724 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:54:21.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.456+0000 7f02dd2f2640 1 --2- 172.21.15.76:0/4175421724 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d8070700 0x7f02d8164b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:21.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.456+0000 7f02d77fe640 1 -- 172.21.15.76:0/4175421724 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2957392687 0 0) 0x7f02d8071500 con 0x7f02d80b0c40 2023-11-09T03:54:21.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.456+0000 7f02d77fe640 1 -- 172.21.15.76:0/4175421724 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02c0003520 con 0x7f02d80b0c40 2023-11-09T03:54:21.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.456+0000 7f02d77fe640 1 -- 172.21.15.76:0/4175421724 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4056920573 0 0) 0x7f02c0003520 con 0x7f02d80b0c40 2023-11-09T03:54:21.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.456+0000 7f02d77fe640 1 -- 172.21.15.76:0/4175421724 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d8070700 msgr2=0x7f02d8164b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:21.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.456+0000 7f02d77fe640 1 --2- 172.21.15.76:0/4175421724 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d8070700 0x7f02d8164b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.456+0000 7f02d77fe640 1 -- 172.21.15.76:0/4175421724 >> v1:172.21.15.113:6789/0 conn(0x7f02d8159e40 legacy=0x7f02d80b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:21.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.456+0000 7f02d77fe640 1 -- 172.21.15.76:0/4175421724 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d8070f40 con 0x7f02d80b0c40 2023-11-09T03:54:21.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.456+0000 7f02d77fe640 1 -- 172.21.15.76:0/4175421724 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f02cc001670 con 0x7f02d80b0c40 2023-11-09T03:54:21.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.456+0000 7f02d77fe640 1 -- 172.21.15.76:0/4175421724 >> v1:172.21.15.76:6789/0 conn(0x7f02d80b0c40 legacy=0x7f02d806fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:21.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.456+0000 7f02d77fe640 1 --2- 172.21.15.76:0/4175421724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02c00039a0 0x7f02c0003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:21.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.456+0000 7f02d77fe640 1 --2- 172.21.15.76:0/4175421724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02c0004350 0x7f02c0004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:21.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.456+0000 7f02d77fe640 1 --2- 172.21.15.76:0/4175421724 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d8070700 0x7f02c0005120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:21.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.456+0000 7f02d77fe640 1 -- 172.21.15.76:0/4175421724 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d8070c60 con 0x7f02c0004350 2023-11-09T03:54:21.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.456+0000 7f02d77fe640 1 -- 172.21.15.76:0/4175421724 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c000ba30 con 0x7f02c00039a0 2023-11-09T03:54:21.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.456+0000 7f02d77fe640 1 -- 172.21.15.76:0/4175421724 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c000bbf0 con 0x7f02d8070700 2023-11-09T03:54:21.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.456+0000 7f02d7fff640 1 --2- 172.21.15.76:0/4175421724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02c0004350 0x7f02c0004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:21.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.456+0000 7f02dcaf1640 1 --2- 172.21.15.76:0/4175421724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02c00039a0 0x7f02c0003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:21.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.456+0000 7f02dd2f2640 1 --2- 172.21.15.76:0/4175421724 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d8070700 0x7f02c0005120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:21.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.456+0000 7f02d7fff640 1 -- 172.21.15.76:0/4175421724 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d8070700 msgr2=0x7f02c0005120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:21.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.456+0000 7f02d7fff640 1 --2- 172.21.15.76:0/4175421724 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d8070700 0x7f02c0005120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.456+0000 7f02d7fff640 1 -- 172.21.15.76:0/4175421724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02c00039a0 msgr2=0x7f02c0003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:21.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.456+0000 7f02d7fff640 1 --2- 172.21.15.76:0/4175421724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02c00039a0 0x7f02c0003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.456+0000 7f02d7fff640 1 -- 172.21.15.76:0/4175421724 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c0003520 con 0x7f02c0004350 2023-11-09T03:54:21.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.456+0000 7f02dcaf1640 1 --2- 172.21.15.76:0/4175421724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02c00039a0 0x7f02c0003d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:54:21.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.457+0000 7f02d7fff640 1 --2- 172.21.15.76:0/4175421724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02c0004350 0x7f02c0004820 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00c210 tx=0x7f02c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b2fa9093f6afda server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:21.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.457+0000 7f02d77fe640 1 -- 172.21.15.76:0/4175421724 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc005ad0 con 0x7f02c0004350 2023-11-09T03:54:21.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.457+0000 7f02d77fe640 1 -- 172.21.15.76:0/4175421724 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02cc005c50 con 0x7f02c0004350 2023-11-09T03:54:21.464 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.457+0000 7f02d77fe640 1 -- 172.21.15.76:0/4175421724 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc02c8d0 con 0x7f02c0004350 2023-11-09T03:54:21.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.458+0000 7f02ded55640 1 -- 172.21.15.76:0/4175421724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02c0004350 msgr2=0x7f02c0004820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:21.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.458+0000 7f02ded55640 1 --2- 172.21.15.76:0/4175421724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02c0004350 0x7f02c0004820 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00c210 tx=0x7f02c00057f0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.458+0000 7f02dd2f2640 1 -- 172.21.15.76:0/4175421724 reap_dead start 2023-11-09T03:54:21.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.458+0000 7f02ded55640 1 -- 172.21.15.76:0/4175421724 shutdown_connections 2023-11-09T03:54:21.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.458+0000 7f02ded55640 1 -- 172.21.15.76:0/4175421724 >> 172.21.15.76:0/4175421724 conn(0x7f02d80cb530 msgr2=0x7f02d80cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:21.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.458+0000 7f02ded55640 1 -- 172.21.15.76:0/4175421724 shutdown_connections 2023-11-09T03:54:21.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.458+0000 7f02ded55640 1 -- 172.21.15.76:0/4175421724 wait complete. 2023-11-09T03:54:21.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.459+0000 7f02ded55640 1 Processor -- start 2023-11-09T03:54:21.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.459+0000 7f02ded55640 1 -- start start 2023-11-09T03:54:21.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.460+0000 7f02ded55640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02c0004350 0x7f02d8153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:21.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.460+0000 7f02ded55640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d81529e0 0x7f02d8154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:21.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.460+0000 7f02ded55640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02d814eb90 0x7f02d814eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:21.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.460+0000 7f02ded55640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d8152e40 con 0x7f02d814eb90 2023-11-09T03:54:21.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.460+0000 7f02ded55640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d8154100 con 0x7f02d81529e0 2023-11-09T03:54:21.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.460+0000 7f02ded55640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d81543e0 con 0x7f02c0004350 2023-11-09T03:54:21.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.460+0000 7f02dd2f2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02d814eb90 0x7f02d814eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:21.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.460+0000 7f02dd2f2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02d814eb90 0x7f02d814eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57288/0 (socket says 172.21.15.76:57288) 2023-11-09T03:54:21.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.460+0000 7f02dd2f2640 1 -- 172.21.15.76:0/1710138170 learned_addr learned my addr 172.21.15.76:0/1710138170 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:21.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.460+0000 7f02d7fff640 1 --2- 172.21.15.76:0/1710138170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d81529e0 0x7f02d8154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:21.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.460+0000 7f02dcaf1640 1 --2- 172.21.15.76:0/1710138170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02c0004350 0x7f02d8153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:21.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.460+0000 7f02dd2f2640 1 -- 172.21.15.76:0/1710138170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02c0004350 msgr2=0x7f02d8153a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:21.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.460+0000 7f02dd2f2640 1 --2- 172.21.15.76:0/1710138170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02c0004350 0x7f02d8153a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.460+0000 7f02dd2f2640 1 -- 172.21.15.76:0/1710138170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d81529e0 msgr2=0x7f02d8154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:21.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.460+0000 7f02dd2f2640 1 --2- 172.21.15.76:0/1710138170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d81529e0 0x7f02d8154c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.460+0000 7f02dd2f2640 1 -- 172.21.15.76:0/1710138170 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d81546c0 con 0x7f02d814eb90 2023-11-09T03:54:21.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.460+0000 7f02dcaf1640 1 --2- 172.21.15.76:0/1710138170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02c0004350 0x7f02d8153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:54:21.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.460+0000 7f02d7fff640 1 --2- 172.21.15.76:0/1710138170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d81529e0 0x7f02d8154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:54:21.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.460+0000 7f02dd2f2640 1 --2- 172.21.15.76:0/1710138170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02d814eb90 0x7f02d814eff0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f02d80c9720 tx=0x7f02d00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:21.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.460+0000 7f02d57fa640 1 -- 172.21.15.76:0/1710138170 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d000e070 con 0x7f02d814eb90 2023-11-09T03:54:21.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.460+0000 7f02d57fa640 1 -- 172.21.15.76:0/1710138170 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d0011ed0 con 0x7f02d814eb90 2023-11-09T03:54:21.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.460+0000 7f02ded55640 1 -- 172.21.15.76:0/1710138170 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d814fba0 con 0x7f02d814eb90 2023-11-09T03:54:21.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.461+0000 7f02d57fa640 1 -- 172.21.15.76:0/1710138170 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d0003990 con 0x7f02d814eb90 2023-11-09T03:54:21.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.461+0000 7f02ded55640 1 -- 172.21.15.76:0/1710138170 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d814f530 con 0x7f02d814eb90 2023-11-09T03:54:21.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.461+0000 7f02ded55640 1 -- 172.21.15.76:0/1710138170 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02d81500c0 con 0x7f02d814eb90 2023-11-09T03:54:21.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.462+0000 7f02d57fa640 1 -- 172.21.15.76:0/1710138170 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f02d0003b10 con 0x7f02d814eb90 2023-11-09T03:54:21.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.462+0000 7f02d57fa640 1 --2- 172.21.15.76:0/1710138170 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f02c406ae70 0x7f02c406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:21.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.462+0000 7f02d57fa640 1 -- 172.21.15.76:0/1710138170 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7f02d008a730 con 0x7f02d814eb90 2023-11-09T03:54:21.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.462+0000 7f02dcaf1640 1 --2- 172.21.15.76:0/1710138170 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f02c406ae70 0x7f02c406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:21.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.463+0000 7f02dcaf1640 1 --2- 172.21.15.76:0/1710138170 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f02c406ae70 0x7f02c406d330 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f02c8005580 tx=0x7f02c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:21.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.466+0000 7f02d57fa640 1 -- 172.21.15.76:0/1710138170 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f02d81500c0 con 0x7f02d814eb90 2023-11-09T03:54:21.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.635+0000 7f02ded55640 1 -- 172.21.15.76:0/1710138170 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7f02d81500c0 con 0x7f02d814eb90 2023-11-09T03:54:21.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.637+0000 7f02d57fa640 1 -- 172.21.15.76:0/1710138170 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v116) v1 ==== 107+0+11 (secure 0 0 0) 0x7f02d81500c0 con 0x7f02d814eb90 2023-11-09T03:54:21.638 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:54:21.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.639+0000 7f02ded55640 1 -- 172.21.15.76:0/1710138170 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f02c406ae70 msgr2=0x7f02c406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:21.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.639+0000 7f02ded55640 1 --2- 172.21.15.76:0/1710138170 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f02c406ae70 0x7f02c406d330 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f02c8005580 tx=0x7f02c8009040 comp rx=0 tx=0).stop 2023-11-09T03:54:21.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.639+0000 7f02ded55640 1 -- 172.21.15.76:0/1710138170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02d814eb90 msgr2=0x7f02d814eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:21.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.639+0000 7f02ded55640 1 --2- 172.21.15.76:0/1710138170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02d814eb90 0x7f02d814eff0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f02d80c9720 tx=0x7f02d00115a0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.639+0000 7f02ded55640 1 -- 172.21.15.76:0/1710138170 shutdown_connections 2023-11-09T03:54:21.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.639+0000 7f02ded55640 1 --2- 172.21.15.76:0/1710138170 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f02c406ae70 0x7f02c406d330 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.639+0000 7f02ded55640 1 --2- 172.21.15.76:0/1710138170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02d814eb90 0x7f02d814eff0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.639+0000 7f02ded55640 1 --2- 172.21.15.76:0/1710138170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d81529e0 0x7f02d8154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.639+0000 7f02ded55640 1 --2- 172.21.15.76:0/1710138170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02c0004350 0x7f02d8153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.639+0000 7f02ded55640 1 -- 172.21.15.76:0/1710138170 >> 172.21.15.76:0/1710138170 conn(0x7f02d80cb530 msgr2=0x7f02d806bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:21.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.639+0000 7f02ded55640 1 -- 172.21.15.76:0/1710138170 shutdown_connections 2023-11-09T03:54:21.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.639+0000 7f02ded55640 1 -- 172.21.15.76:0/1710138170 wait complete. 2023-11-09T03:54:21.651 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:54:21.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.754+0000 7f5fcc5c0640 1 Processor -- start 2023-11-09T03:54:21.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.754+0000 7f5fcc5c0640 1 -- start start 2023-11-09T03:54:21.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.755+0000 7f5fcc5c0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5fc415d100 0x7f5fc40ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:21.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.755+0000 7f5fcc5c0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5fc415d4e0 0x7f5fc40d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:21.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.755+0000 7f5fcc5c0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fc40b32b0 con 0x7f5fc40d3b60 2023-11-09T03:54:21.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.755+0000 7f5fcc5c0640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fc40cee10 con 0x7f5fc415d100 2023-11-09T03:54:21.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.755+0000 7f5fcc5c0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc40b4750 con 0x7f5fc415d4e0 2023-11-09T03:54:21.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.756+0000 7f5fcab5d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f5fc40d3b60 0x7f5fc40cf520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45818/0 (socket says 172.21.15.76:45818) 2023-11-09T03:54:21.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.756+0000 7f5fcab5d640 1 -- 172.21.15.76:0/3147390419 learned_addr learned my addr 172.21.15.76:0/3147390419 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:54:21.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.756+0000 7f5fca35c640 1 --2- 172.21.15.76:0/3147390419 >> v2:172.21.15.113:3300/0 conn(0x7f5fc415d100 0x7f5fc40ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:21.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.756+0000 7f5fc9b5b640 1 --2- 172.21.15.76:0/3147390419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5fc415d4e0 0x7f5fc40d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:21.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.756+0000 7f5fc935a640 1 -- 172.21.15.76:0/3147390419 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3528790814 0 0) 0x7f5fc40b32b0 con 0x7f5fc40d3b60 2023-11-09T03:54:21.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.756+0000 7f5fc935a640 1 -- 172.21.15.76:0/3147390419 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fb0003520 con 0x7f5fc40d3b60 2023-11-09T03:54:21.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.756+0000 7f5fc935a640 1 -- 172.21.15.76:0/3147390419 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 267171900 0 0) 0x7f5fb0003520 con 0x7f5fc40d3b60 2023-11-09T03:54:21.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.756+0000 7f5fc935a640 1 -- 172.21.15.76:0/3147390419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5fc415d4e0 msgr2=0x7f5fc40d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:21.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.756+0000 7f5fc935a640 1 --2- 172.21.15.76:0/3147390419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5fc415d4e0 0x7f5fc40d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.756+0000 7f5fc935a640 1 -- 172.21.15.76:0/3147390419 >> v2:172.21.15.113:3300/0 conn(0x7f5fc415d100 msgr2=0x7f5fc40ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:21.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.756+0000 7f5fc935a640 1 --2- 172.21.15.76:0/3147390419 >> v2:172.21.15.113:3300/0 conn(0x7f5fc415d100 0x7f5fc40ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.756+0000 7f5fc935a640 1 -- 172.21.15.76:0/3147390419 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc40b2980 con 0x7f5fc40d3b60 2023-11-09T03:54:21.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.756+0000 7f5fc935a640 1 -- 172.21.15.76:0/3147390419 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f5fbc0019f0 con 0x7f5fc40d3b60 2023-11-09T03:54:21.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.756+0000 7f5fca35c640 1 --2- 172.21.15.76:0/3147390419 >> v2:172.21.15.113:3300/0 conn(0x7f5fc415d100 0x7f5fc40ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:54:21.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.756+0000 7f5fc935a640 1 -- 172.21.15.76:0/3147390419 >> v1:172.21.15.76:6789/0 conn(0x7f5fc40d3b60 legacy=0x7f5fc40cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:21.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.756+0000 7f5fc935a640 1 --2- 172.21.15.76:0/3147390419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5fc415d4e0 0x7f5fb0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:21.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.756+0000 7f5fc935a640 1 --2- 172.21.15.76:0/3147390419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fb0003ef0 0x7f5fb00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:21.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.756+0000 7f5fc935a640 1 --2- 172.21.15.76:0/3147390419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fb0004d00 0x7f5fb0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:21.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.756+0000 7f5fc935a640 1 -- 172.21.15.76:0/3147390419 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc40cee10 con 0x7f5fb0003ef0 2023-11-09T03:54:21.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.756+0000 7f5fc935a640 1 -- 172.21.15.76:0/3147390419 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc40b4750 con 0x7f5fb0004d00 2023-11-09T03:54:21.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.756+0000 7f5fc935a640 1 -- 172.21.15.76:0/3147390419 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb000b980 con 0x7f5fc415d4e0 2023-11-09T03:54:21.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.756+0000 7f5fc9b5b640 1 --2- 172.21.15.76:0/3147390419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fb0003ef0 0x7f5fb00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:21.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.756+0000 7f5fca35c640 1 --2- 172.21.15.76:0/3147390419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5fc415d4e0 0x7f5fb0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:21.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.756+0000 7f5fcab5d640 1 --2- 172.21.15.76:0/3147390419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fb0004d00 0x7f5fb0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:21.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.757+0000 7f5fc9b5b640 1 -- 172.21.15.76:0/3147390419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5fc415d4e0 msgr2=0x7f5fb0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:21.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.757+0000 7f5fc9b5b640 1 --2- 172.21.15.76:0/3147390419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5fc415d4e0 0x7f5fb0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.757+0000 7f5fc9b5b640 1 -- 172.21.15.76:0/3147390419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fb0004d00 msgr2=0x7f5fb0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:21.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.757+0000 7f5fc9b5b640 1 --2- 172.21.15.76:0/3147390419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fb0004d00 0x7f5fb0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.757+0000 7f5fc9b5b640 1 -- 172.21.15.76:0/3147390419 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb0003520 con 0x7f5fb0003ef0 2023-11-09T03:54:21.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.757+0000 7f5fcab5d640 1 --2- 172.21.15.76:0/3147390419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fb0004d00 0x7f5fb0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:54:21.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.757+0000 7f5fc9b5b640 1 --2- 172.21.15.76:0/3147390419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fb0003ef0 0x7f5fb00047c0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009ac0 tx=0x7f5fb00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897d0c02bb8bac server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:21.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.757+0000 7f5fc935a640 1 -- 172.21.15.76:0/3147390419 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb402c6b0 con 0x7f5fb0003ef0 2023-11-09T03:54:21.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.757+0000 7f5fc935a640 1 -- 172.21.15.76:0/3147390419 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb402c810 con 0x7f5fb0003ef0 2023-11-09T03:54:21.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.757+0000 7f5fc935a640 1 -- 172.21.15.76:0/3147390419 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb402cba0 con 0x7f5fb0003ef0 2023-11-09T03:54:21.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.758+0000 7f5fcc5c0640 1 -- 172.21.15.76:0/3147390419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fb0003ef0 msgr2=0x7f5fb00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:21.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.758+0000 7f5fcc5c0640 1 --2- 172.21.15.76:0/3147390419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fb0003ef0 0x7f5fb00047c0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009ac0 tx=0x7f5fb00057f0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.758+0000 7f5fcab5d640 1 -- 172.21.15.76:0/3147390419 reap_dead start 2023-11-09T03:54:21.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.758+0000 7f5fcc5c0640 1 -- 172.21.15.76:0/3147390419 shutdown_connections 2023-11-09T03:54:21.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.758+0000 7f5fcc5c0640 1 -- 172.21.15.76:0/3147390419 >> 172.21.15.76:0/3147390419 conn(0x7f5fc40d2ef0 msgr2=0x7f5fc40d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:21.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.758+0000 7f5fcc5c0640 1 -- 172.21.15.76:0/3147390419 shutdown_connections 2023-11-09T03:54:21.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.758+0000 7f5fcc5c0640 1 -- 172.21.15.76:0/3147390419 wait complete. 2023-11-09T03:54:21.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.759+0000 7f5fcc5c0640 1 Processor -- start 2023-11-09T03:54:21.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.759+0000 7f5fcc5c0640 1 -- start start 2023-11-09T03:54:21.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.760+0000 7f5fcc5c0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fb0003ef0 0x7f5fc4151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:21.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.760+0000 7f5fcc5c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fc41f6970 0x7f5fc41f6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:21.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.760+0000 7f5fcc5c0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5fc41f7330 0x7f5fc40b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:21.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.760+0000 7f5fcc5c0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc415d920 con 0x7f5fc41f6970 2023-11-09T03:54:21.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.760+0000 7f5fcc5c0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc40cbca0 con 0x7f5fb0003ef0 2023-11-09T03:54:21.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.760+0000 7f5fcc5c0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc40cbf60 con 0x7f5fc41f7330 2023-11-09T03:54:21.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.760+0000 7f5fc9b5b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fc41f6970 0x7f5fc41f6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:21.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.760+0000 7f5fc9b5b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fc41f6970 0x7f5fc41f6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57314/0 (socket says 172.21.15.76:57314) 2023-11-09T03:54:21.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.760+0000 7f5fc9b5b640 1 -- 172.21.15.76:0/2125703974 learned_addr learned my addr 172.21.15.76:0/2125703974 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:21.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.760+0000 7f5fcab5d640 1 --2- 172.21.15.76:0/2125703974 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5fc41f7330 0x7f5fc40b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:21.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.760+0000 7f5fca35c640 1 --2- 172.21.15.76:0/2125703974 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fb0003ef0 0x7f5fc4151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:21.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.760+0000 7f5fc9b5b640 1 -- 172.21.15.76:0/2125703974 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5fc41f7330 msgr2=0x7f5fc40b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:21.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.760+0000 7f5fc9b5b640 1 --2- 172.21.15.76:0/2125703974 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5fc41f7330 0x7f5fc40b77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.760+0000 7f5fc9b5b640 1 -- 172.21.15.76:0/2125703974 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fb0003ef0 msgr2=0x7f5fc4151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:21.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.760+0000 7f5fc9b5b640 1 --2- 172.21.15.76:0/2125703974 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fb0003ef0 0x7f5fc4151f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.760+0000 7f5fc9b5b640 1 -- 172.21.15.76:0/2125703974 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc40cc240 con 0x7f5fc41f6970 2023-11-09T03:54:21.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.760+0000 7f5fc9b5b640 1 --2- 172.21.15.76:0/2125703974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fc41f6970 0x7f5fc41f6df0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009ac0 tx=0x7f5fb4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:21.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.761+0000 7f5fbaffd640 1 -- 172.21.15.76:0/2125703974 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4003810 con 0x7f5fc41f6970 2023-11-09T03:54:21.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.761+0000 7f5fbaffd640 1 -- 172.21.15.76:0/2125703974 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb4003970 con 0x7f5fc41f6970 2023-11-09T03:54:21.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.761+0000 7f5fcc5c0640 1 -- 172.21.15.76:0/2125703974 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fc40cc520 con 0x7f5fc41f6970 2023-11-09T03:54:21.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.761+0000 7f5fbaffd640 1 -- 172.21.15.76:0/2125703974 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4003d00 con 0x7f5fc41f6970 2023-11-09T03:54:21.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.761+0000 7f5fcc5c0640 1 -- 172.21.15.76:0/2125703974 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc40cc800 con 0x7f5fc41f6970 2023-11-09T03:54:21.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.761+0000 7f5fcc5c0640 1 -- 172.21.15.76:0/2125703974 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fc40684b0 con 0x7f5fc41f6970 2023-11-09T03:54:21.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.762+0000 7f5fbaffd640 1 -- 172.21.15.76:0/2125703974 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5fb4040450 con 0x7f5fc41f6970 2023-11-09T03:54:21.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.762+0000 7f5fbaffd640 1 --2- 172.21.15.76:0/2125703974 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5fa806ae70 0x7f5fa806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:21.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.762+0000 7f5fbaffd640 1 -- 172.21.15.76:0/2125703974 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7f5fb40abfa0 con 0x7f5fc41f6970 2023-11-09T03:54:21.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.762+0000 7f5fca35c640 1 --2- 172.21.15.76:0/2125703974 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5fa806ae70 0x7f5fa806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:21.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.763+0000 7f5fca35c640 1 --2- 172.21.15.76:0/2125703974 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5fa806ae70 0x7f5fa806d330 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009d00 tx=0x7f5fc0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:21.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.766+0000 7f5fbaffd640 1 -- 172.21.15.76:0/2125703974 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5fb4076580 con 0x7f5fc41f6970 2023-11-09T03:54:21.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.931+0000 7f5fcc5c0640 1 -- 172.21.15.76:0/2125703974 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5fc406a8d0 con 0x7f5fc41f6970 2023-11-09T03:54:21.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.932+0000 7f5fbaffd640 1 -- 172.21.15.76:0/2125703974 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v116) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5fb4044020 con 0x7f5fc41f6970 2023-11-09T03:54:21.933 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:54:21.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.934+0000 7f5fcc5c0640 1 -- 172.21.15.76:0/2125703974 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5fa806ae70 msgr2=0x7f5fa806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:21.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.934+0000 7f5fcc5c0640 1 --2- 172.21.15.76:0/2125703974 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5fa806ae70 0x7f5fa806d330 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009d00 tx=0x7f5fc0008040 comp rx=0 tx=0).stop 2023-11-09T03:54:21.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.934+0000 7f5fcc5c0640 1 -- 172.21.15.76:0/2125703974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fc41f6970 msgr2=0x7f5fc41f6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:21.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.934+0000 7f5fcc5c0640 1 --2- 172.21.15.76:0/2125703974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fc41f6970 0x7f5fc41f6df0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009ac0 tx=0x7f5fb4004730 comp rx=0 tx=0).stop 2023-11-09T03:54:21.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.934+0000 7f5fcc5c0640 1 -- 172.21.15.76:0/2125703974 shutdown_connections 2023-11-09T03:54:21.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.934+0000 7f5fcc5c0640 1 --2- 172.21.15.76:0/2125703974 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5fa806ae70 0x7f5fa806d330 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.934+0000 7f5fcc5c0640 1 --2- 172.21.15.76:0/2125703974 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5fc41f7330 0x7f5fc40b77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.934+0000 7f5fcc5c0640 1 --2- 172.21.15.76:0/2125703974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fc41f6970 0x7f5fc41f6df0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.934+0000 7f5fcc5c0640 1 --2- 172.21.15.76:0/2125703974 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fb0003ef0 0x7f5fc4151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:21.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.934+0000 7f5fcc5c0640 1 -- 172.21.15.76:0/2125703974 >> 172.21.15.76:0/2125703974 conn(0x7f5fc40d2ef0 msgr2=0x7f5fc40d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:21.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.934+0000 7f5fcc5c0640 1 -- 172.21.15.76:0/2125703974 shutdown_connections 2023-11-09T03:54:21.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:21.934+0000 7f5fcc5c0640 1 -- 172.21.15.76:0/2125703974 wait complete. 2023-11-09T03:54:21.945 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:54:22.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.047+0000 7fadea11d640 1 Processor -- start 2023-11-09T03:54:22.048 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.047+0000 7fadea11d640 1 -- start start 2023-11-09T03:54:22.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.048+0000 7fadea11d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fade4159e40 0x7fade40b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:22.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.048+0000 7fadea11d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fade406bb60 0x7fade40b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:22.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.048+0000 7fadea11d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade4071500 con 0x7fade406fff0 2023-11-09T03:54:22.050 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.048+0000 7fadea11d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade415dae0 con 0x7fade406bb60 2023-11-09T03:54:22.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.048+0000 7fadea11d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade4070c60 con 0x7fade4159e40 2023-11-09T03:54:22.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.048+0000 7faddbfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fade406bb60 0x7fade40b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:22.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.048+0000 7faddbfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fade406bb60 0x7fade40b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57328/0 (socket says 172.21.15.76:57328) 2023-11-09T03:54:22.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.048+0000 7faddbfff640 1 -- 172.21.15.76:0/1136480354 learned_addr learned my addr 172.21.15.76:0/1136480354 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:22.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.048+0000 7faddbfff640 1 -- 172.21.15.76:0/1136480354 >> v2:172.21.15.113:3300/0 conn(0x7fade4159e40 msgr2=0x7fade40b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:22.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.048+0000 7faddbfff640 1 --2- 172.21.15.76:0/1136480354 >> v2:172.21.15.113:3300/0 conn(0x7fade4159e40 0x7fade40b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:22.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.048+0000 7faddbfff640 1 -- 172.21.15.76:0/1136480354 >> v1:172.21.15.113:6789/0 conn(0x7fade406fff0 legacy=0x7fade4070420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:22.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.048+0000 7faddbfff640 1 -- 172.21.15.76:0/1136480354 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade4070f40 con 0x7fade406bb60 2023-11-09T03:54:22.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.049+0000 7faddbfff640 1 --2- 172.21.15.76:0/1136480354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade406bb60 0x7fade40b0a70 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fade40ce7f0 tx=0x7fadcc02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e121091afb4432c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:22.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.049+0000 7fade2ffd640 1 -- 172.21.15.76:0/1136480354 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc031750 con 0x7fade406bb60 2023-11-09T03:54:22.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.049+0000 7fade2ffd640 1 -- 172.21.15.76:0/1136480354 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadcc0318b0 con 0x7fade406bb60 2023-11-09T03:54:22.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.049+0000 7fade2ffd640 1 -- 172.21.15.76:0/1136480354 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc031c40 con 0x7fade406bb60 2023-11-09T03:54:22.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.050+0000 7fadea11d640 1 -- 172.21.15.76:0/1136480354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade406bb60 msgr2=0x7fade40b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:22.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.050+0000 7fadea11d640 1 --2- 172.21.15.76:0/1136480354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade406bb60 0x7fade40b0a70 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fade40ce7f0 tx=0x7fadcc02b7f0 comp rx=0 tx=0).stop 2023-11-09T03:54:22.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.050+0000 7fadea11d640 1 -- 172.21.15.76:0/1136480354 shutdown_connections 2023-11-09T03:54:22.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.050+0000 7fadea11d640 1 --2- 172.21.15.76:0/1136480354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade406bb60 0x7fade40b0a70 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:22.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.050+0000 7fadea11d640 1 --2- 172.21.15.76:0/1136480354 >> v2:172.21.15.113:3300/0 conn(0x7fade4159e40 0x7fade40b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:22.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.050+0000 7fadea11d640 1 -- 172.21.15.76:0/1136480354 >> 172.21.15.76:0/1136480354 conn(0x7fade40cb530 msgr2=0x7fade40cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:22.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.050+0000 7fadea11d640 1 -- 172.21.15.76:0/1136480354 shutdown_connections 2023-11-09T03:54:22.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.050+0000 7fadea11d640 1 -- 172.21.15.76:0/1136480354 wait complete. 2023-11-09T03:54:22.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.051+0000 7fadea11d640 1 Processor -- start 2023-11-09T03:54:22.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.051+0000 7fadea11d640 1 -- start start 2023-11-09T03:54:22.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.052+0000 7fadea11d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fade406bb60 0x7fade4152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:22.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.052+0000 7fadea11d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fade406fff0 0x7fade41504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:22.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.052+0000 7fadea11d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade4159e40 0x7fade4150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:22.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.052+0000 7fadea11d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade40c6140 con 0x7fade4159e40 2023-11-09T03:54:22.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.052+0000 7fadea11d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade40c2660 con 0x7fade406fff0 2023-11-09T03:54:22.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.052+0000 7fadea11d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade4150f50 con 0x7fade406bb60 2023-11-09T03:54:22.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.052+0000 7fade3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade4159e40 0x7fade4150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:22.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.052+0000 7fade3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade4159e40 0x7fade4150a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57330/0 (socket says 172.21.15.76:57330) 2023-11-09T03:54:22.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.052+0000 7fade3fff640 1 -- 172.21.15.76:0/589417279 learned_addr learned my addr 172.21.15.76:0/589417279 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:22.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.052+0000 7fade3fff640 1 -- 172.21.15.76:0/589417279 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fade406bb60 msgr2=0x7fade4152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:22.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.052+0000 7fade3fff640 1 --2- 172.21.15.76:0/589417279 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fade406bb60 0x7fade4152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:22.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.052+0000 7fade3fff640 1 -- 172.21.15.76:0/589417279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fade406fff0 msgr2=0x7fade41504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:22.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.052+0000 7faddbfff640 1 --2- 172.21.15.76:0/589417279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fade406fff0 0x7fade41504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:22.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.052+0000 7fade3fff640 1 --2- 172.21.15.76:0/589417279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fade406fff0 0x7fade41504d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:22.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.052+0000 7fade3fff640 1 -- 172.21.15.76:0/589417279 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadcc0094d0 con 0x7fade4159e40 2023-11-09T03:54:22.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.052+0000 7fade3fff640 1 --2- 172.21.15.76:0/589417279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade4159e40 0x7fade4150a10 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7faddc00c780 tx=0x7faddc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:22.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.053+0000 7fade0ff9640 1 -- 172.21.15.76:0/589417279 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc003d80 con 0x7fade4159e40 2023-11-09T03:54:22.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.053+0000 7fade0ff9640 1 -- 172.21.15.76:0/589417279 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faddc00ce90 con 0x7fade4159e40 2023-11-09T03:54:22.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.053+0000 7fade0ff9640 1 -- 172.21.15.76:0/589417279 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc0052b0 con 0x7fade4159e40 2023-11-09T03:54:22.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.053+0000 7fadea11d640 1 -- 172.21.15.76:0/589417279 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade40c2900 con 0x7fade4159e40 2023-11-09T03:54:22.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.053+0000 7fadea11d640 1 -- 172.21.15.76:0/589417279 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade40c2ec0 con 0x7fade4159e40 2023-11-09T03:54:22.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.054+0000 7fadea11d640 1 -- 172.21.15.76:0/589417279 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade40684b0 con 0x7fade4159e40 2023-11-09T03:54:22.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.054+0000 7fade0ff9640 1 -- 172.21.15.76:0/589417279 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faddc005410 con 0x7fade4159e40 2023-11-09T03:54:22.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.054+0000 7fade0ff9640 1 --2- 172.21.15.76:0/589417279 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fadc406aea0 0x7fadc406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:22.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.054+0000 7fade0ff9640 1 -- 172.21.15.76:0/589417279 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7faddc089430 con 0x7fade4159e40 2023-11-09T03:54:22.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.055+0000 7fade37fe640 1 --2- 172.21.15.76:0/589417279 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fadc406aea0 0x7fadc406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:22.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.055+0000 7fade37fe640 1 --2- 172.21.15.76:0/589417279 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fadc406aea0 0x7fadc406d360 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009670 tx=0x7fadd4006bf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:22.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.059+0000 7fade0ff9640 1 -- 172.21.15.76:0/589417279 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faddc053a70 con 0x7fade4159e40 2023-11-09T03:54:22.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.224+0000 7fadea11d640 1 -- 172.21.15.76:0/589417279 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fade4151280 con 0x7fade4159e40 2023-11-09T03:54:22.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.224+0000 7fade0ff9640 1 -- 172.21.15.76:0/589417279 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v116) v1 ==== 100+0+11 (secure 0 0 0) 0x7faddc0170a0 con 0x7fade4159e40 2023-11-09T03:54:22.225 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:54:22.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.226+0000 7fadea11d640 1 -- 172.21.15.76:0/589417279 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fadc406aea0 msgr2=0x7fadc406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:22.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.226+0000 7fadea11d640 1 --2- 172.21.15.76:0/589417279 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fadc406aea0 0x7fadc406d360 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009670 tx=0x7fadd4006bf0 comp rx=0 tx=0).stop 2023-11-09T03:54:22.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.226+0000 7fadea11d640 1 -- 172.21.15.76:0/589417279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade4159e40 msgr2=0x7fade4150a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:22.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.226+0000 7fadea11d640 1 --2- 172.21.15.76:0/589417279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade4159e40 0x7fade4150a10 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7faddc00c780 tx=0x7faddc00cc40 comp rx=0 tx=0).stop 2023-11-09T03:54:22.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.226+0000 7fadea11d640 1 -- 172.21.15.76:0/589417279 shutdown_connections 2023-11-09T03:54:22.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.226+0000 7fadea11d640 1 --2- 172.21.15.76:0/589417279 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fadc406aea0 0x7fadc406d360 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:22.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.226+0000 7fadea11d640 1 --2- 172.21.15.76:0/589417279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade4159e40 0x7fade4150a10 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:22.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.226+0000 7fadea11d640 1 --2- 172.21.15.76:0/589417279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fade406fff0 0x7fade41504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:22.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.226+0000 7fadea11d640 1 --2- 172.21.15.76:0/589417279 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fade406bb60 0x7fade4152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:22.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.226+0000 7fadea11d640 1 -- 172.21.15.76:0/589417279 >> 172.21.15.76:0/589417279 conn(0x7fade40cb530 msgr2=0x7fade40cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:22.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.226+0000 7fadea11d640 1 -- 172.21.15.76:0/589417279 shutdown_connections 2023-11-09T03:54:22.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.226+0000 7fadea11d640 1 -- 172.21.15.76:0/589417279 wait complete. 2023-11-09T03:54:22.238 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-09T03:54:22.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.339+0000 7fa96edb6640 1 Processor -- start 2023-11-09T03:54:22.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.339+0000 7fa96edb6640 1 -- start start 2023-11-09T03:54:22.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.340+0000 7fa96edb6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa96815d100 0x7fa9680ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:22.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.340+0000 7fa96edb6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa96815d4e0 0x7fa9680d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:22.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.340+0000 7fa96edb6640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9680b32b0 con 0x7fa9680d3b60 2023-11-09T03:54:22.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.340+0000 7fa96edb6640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9680cee10 con 0x7fa96815d4e0 2023-11-09T03:54:22.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.340+0000 7fa96edb6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9680b4750 con 0x7fa96815d100 2023-11-09T03:54:22.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.340+0000 7fa967fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa96815d4e0 0x7fa9680d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:22.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.340+0000 7fa96d353640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa9680d3b60 0x7fa9680cf520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45828/0 (socket says 172.21.15.76:45828) 2023-11-09T03:54:22.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.340+0000 7fa96d353640 1 -- 172.21.15.76:0/617333869 learned_addr learned my addr 172.21.15.76:0/617333869 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:54:22.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.341+0000 7fa96cb52640 1 --2- 172.21.15.76:0/617333869 >> v2:172.21.15.113:3300/0 conn(0x7fa96815d100 0x7fa9680ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:22.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.341+0000 7fa967fff640 1 -- 172.21.15.76:0/617333869 >> v2:172.21.15.113:3300/0 conn(0x7fa96815d100 msgr2=0x7fa9680ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:22.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.341+0000 7fa967fff640 1 --2- 172.21.15.76:0/617333869 >> v2:172.21.15.113:3300/0 conn(0x7fa96815d100 0x7fa9680ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:22.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.341+0000 7fa967fff640 1 -- 172.21.15.76:0/617333869 >> v1:172.21.15.76:6789/0 conn(0x7fa9680d3b60 legacy=0x7fa9680cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:22.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.341+0000 7fa967fff640 1 -- 172.21.15.76:0/617333869 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9680b2980 con 0x7fa96815d4e0 2023-11-09T03:54:22.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.341+0000 7fa9677fe640 1 -- 172.21.15.76:0/617333869 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1796435603 0 0) 0x7fa9680b32b0 con 0x7fa9680d3b60 2023-11-09T03:54:22.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.341+0000 7fa967fff640 1 --2- 172.21.15.76:0/617333869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa96815d4e0 0x7fa9680d14d0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fa9680d09f0 tx=0x7fa95c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95426e242392a70d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:22.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.342+0000 7fa9677fe640 1 -- 172.21.15.76:0/617333869 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c031830 con 0x7fa96815d4e0 2023-11-09T03:54:22.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.342+0000 7fa9677fe640 1 -- 172.21.15.76:0/617333869 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95c031990 con 0x7fa96815d4e0 2023-11-09T03:54:22.345 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.342+0000 7fa9677fe640 1 -- 172.21.15.76:0/617333869 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c031d20 con 0x7fa96815d4e0 2023-11-09T03:54:22.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.342+0000 7fa96edb6640 1 -- 172.21.15.76:0/617333869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa96815d4e0 msgr2=0x7fa9680d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:22.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.342+0000 7fa96edb6640 1 --2- 172.21.15.76:0/617333869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa96815d4e0 0x7fa9680d14d0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fa9680d09f0 tx=0x7fa95c02b8d0 comp rx=0 tx=0).stop 2023-11-09T03:54:22.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.342+0000 7fa96edb6640 1 -- 172.21.15.76:0/617333869 shutdown_connections 2023-11-09T03:54:22.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.342+0000 7fa96edb6640 1 --2- 172.21.15.76:0/617333869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa96815d4e0 0x7fa9680d14d0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:22.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.342+0000 7fa96edb6640 1 --2- 172.21.15.76:0/617333869 >> v2:172.21.15.113:3300/0 conn(0x7fa96815d100 0x7fa9680ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:22.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.342+0000 7fa96edb6640 1 -- 172.21.15.76:0/617333869 >> 172.21.15.76:0/617333869 conn(0x7fa9680d2ef0 msgr2=0x7fa9680d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:22.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.342+0000 7fa96edb6640 1 -- 172.21.15.76:0/617333869 shutdown_connections 2023-11-09T03:54:22.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.342+0000 7fa96edb6640 1 -- 172.21.15.76:0/617333869 wait complete. 2023-11-09T03:54:22.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.343+0000 7fa96edb6640 1 Processor -- start 2023-11-09T03:54:22.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.343+0000 7fa96edb6640 1 -- start start 2023-11-09T03:54:22.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.344+0000 7fa96edb6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9680d3b60 0x7fa968151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:22.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.344+0000 7fa96edb6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa96815d100 0x7fa9680cb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:22.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.344+0000 7fa96edb6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa96815d4e0 0x7fa9680b7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:22.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.344+0000 7fa96edb6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9681519b0 con 0x7fa96815d100 2023-11-09T03:54:22.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.344+0000 7fa96edb6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9680cce10 con 0x7fa9680d3b60 2023-11-09T03:54:22.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.344+0000 7fa96edb6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9680cb5a0 con 0x7fa96815d4e0 2023-11-09T03:54:22.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.344+0000 7fa967fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa96815d100 0x7fa9680cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:22.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.344+0000 7fa967fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa96815d100 0x7fa9680cb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57350/0 (socket says 172.21.15.76:57350) 2023-11-09T03:54:22.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.344+0000 7fa967fff640 1 -- 172.21.15.76:0/3039395329 learned_addr learned my addr 172.21.15.76:0/3039395329 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:22.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.344+0000 7fa96d353640 1 --2- 172.21.15.76:0/3039395329 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa96815d4e0 0x7fa9680b7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:22.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.344+0000 7fa96cb52640 1 --2- 172.21.15.76:0/3039395329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9680d3b60 0x7fa968151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:22.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.345+0000 7fa967fff640 1 -- 172.21.15.76:0/3039395329 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa96815d4e0 msgr2=0x7fa9680b7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:22.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.345+0000 7fa967fff640 1 --2- 172.21.15.76:0/3039395329 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa96815d4e0 0x7fa9680b7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:22.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.345+0000 7fa967fff640 1 -- 172.21.15.76:0/3039395329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9680d3b60 msgr2=0x7fa968151470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:22.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.345+0000 7fa967fff640 1 --2- 172.21.15.76:0/3039395329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9680d3b60 0x7fa968151470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:22.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.345+0000 7fa967fff640 1 -- 172.21.15.76:0/3039395329 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95c009400 con 0x7fa96815d100 2023-11-09T03:54:22.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.345+0000 7fa96d353640 1 --2- 172.21.15.76:0/3039395329 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa96815d4e0 0x7fa9680b7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:54:22.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.345+0000 7fa96cb52640 1 --2- 172.21.15.76:0/3039395329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9680d3b60 0x7fa968151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:54:22.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.345+0000 7fa967fff640 1 --2- 172.21.15.76:0/3039395329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa96815d100 0x7fa9680cb060 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fa95c009890 tx=0x7fa95c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:22.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.345+0000 7fa9657fa640 1 -- 172.21.15.76:0/3039395329 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c0317d0 con 0x7fa96815d100 2023-11-09T03:54:22.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.345+0000 7fa9657fa640 1 -- 172.21.15.76:0/3039395329 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95c03ba70 con 0x7fa96815d100 2023-11-09T03:54:22.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.345+0000 7fa96edb6640 1 -- 172.21.15.76:0/3039395329 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9680c9fc0 con 0x7fa96815d100 2023-11-09T03:54:22.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.345+0000 7fa9657fa640 1 -- 172.21.15.76:0/3039395329 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c03be20 con 0x7fa96815d100 2023-11-09T03:54:22.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.345+0000 7fa96edb6640 1 -- 172.21.15.76:0/3039395329 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9680ca580 con 0x7fa96815d100 2023-11-09T03:54:22.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.346+0000 7fa96edb6640 1 -- 172.21.15.76:0/3039395329 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9680684b0 con 0x7fa96815d100 2023-11-09T03:54:22.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.346+0000 7fa9657fa640 1 -- 172.21.15.76:0/3039395329 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa95c0453f0 con 0x7fa96815d100 2023-11-09T03:54:22.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.347+0000 7fa9657fa640 1 --2- 172.21.15.76:0/3039395329 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa95406ae70 0x7fa95406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:22.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.347+0000 7fa9657fa640 1 -- 172.21.15.76:0/3039395329 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7fa95c0ac430 con 0x7fa96815d100 2023-11-09T03:54:22.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.347+0000 7fa96cb52640 1 --2- 172.21.15.76:0/3039395329 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa95406ae70 0x7fa95406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:22.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.348+0000 7fa96cb52640 1 --2- 172.21.15.76:0/3039395329 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa95406ae70 0x7fa95406d330 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa9680b8a70 tx=0x7fa958008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:22.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.351+0000 7fa9657fa640 1 -- 172.21.15.76:0/3039395329 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa95c076b90 con 0x7fa96815d100 2023-11-09T03:54:22.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.530+0000 7fa96edb6640 1 -- 172.21.15.76:0/3039395329 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa9680d2cd0 con 0x7fa96815d100 2023-11-09T03:54:22.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.531+0000 7fa9657fa640 1 -- 172.21.15.76:0/3039395329 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 26 entries v116) v1 ==== 81+0+1566 (secure 0 0 0) 0x7fa95c035080 con 0x7fa96815d100 2023-11-09T03:54:22.532 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/2075292158 2023-11-10T03:54:06.568191+0000 2023-11-09T03:54:22.532 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/334503200 2023-11-09T04:53:32.572840+0000 2023-11-09T03:54:22.532 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/2075292158 2023-11-10T03:54:06.568191+0000 2023-11-09T03:54:22.532 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/7360844 2023-11-10T03:53:24.718008+0000 2023-11-09T03:54:22.533 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/7360844 2023-11-10T03:53:24.718008+0000 2023-11-09T03:54:22.533 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/546338722 2023-11-10T03:53:17.470106+0000 2023-11-09T03:54:22.533 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/546338722 2023-11-10T03:53:17.470106+0000 2023-11-09T03:54:22.533 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/3087804343 2023-11-09T04:52:42.513556+0000 2023-11-09T03:54:22.533 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6833/1536527256 2023-11-10T03:52:34.213804+0000 2023-11-09T03:54:22.534 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/445306598 2023-11-10T03:52:27.406838+0000 2023-11-09T03:54:22.534 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/3229239133 2023-11-10T03:51:32.624856+0000 2023-11-09T03:54:22.534 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/503424885 2023-11-09T04:51:47.727422+0000 2023-11-09T03:54:22.534 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/4011275120 2023-11-10T03:49:42.029057+0000 2023-11-09T03:54:22.535 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/4011275120 2023-11-10T03:49:42.029057+0000 2023-11-09T03:54:22.535 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/2519773110 2023-11-09T04:54:14.823765+0000 2023-11-09T03:54:22.535 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/3280098776 2023-11-10T03:50:07.616780+0000 2023-11-09T03:54:22.535 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/3280098776 2023-11-10T03:50:07.616780+0000 2023-11-09T03:54:22.535 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/1663988060 2023-11-10T03:50:33.665092+0000 2023-11-09T03:54:22.536 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/1663988060 2023-11-10T03:50:33.665092+0000 2023-11-09T03:54:22.536 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/445306598 2023-11-10T03:52:27.406838+0000 2023-11-09T03:54:22.536 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6833/3697850101 2023-11-10T03:51:39.442929+0000 2023-11-09T03:54:22.536 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/3449910842 2023-11-10T03:50:59.611256+0000 2023-11-09T03:54:22.536 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/3229239133 2023-11-10T03:51:32.624856+0000 2023-11-09T03:54:22.537 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6832/1536527256 2023-11-10T03:52:34.213804+0000 2023-11-09T03:54:22.537 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/3449910842 2023-11-10T03:50:59.611256+0000 2023-11-09T03:54:22.537 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6832/3697850101 2023-11-10T03:51:39.442929+0000 2023-11-09T03:54:22.538 INFO:teuthology.orchestra.run.smithi076.stderr:listed 26 entries 2023-11-09T03:54:22.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.532+0000 7fa96edb6640 1 -- 172.21.15.76:0/3039395329 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa95406ae70 msgr2=0x7fa95406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:22.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.532+0000 7fa96edb6640 1 --2- 172.21.15.76:0/3039395329 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa95406ae70 0x7fa95406d330 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa9680b8a70 tx=0x7fa958008040 comp rx=0 tx=0).stop 2023-11-09T03:54:22.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.532+0000 7fa96edb6640 1 -- 172.21.15.76:0/3039395329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa96815d100 msgr2=0x7fa9680cb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:22.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.532+0000 7fa96edb6640 1 --2- 172.21.15.76:0/3039395329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa96815d100 0x7fa9680cb060 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fa95c009890 tx=0x7fa95c0046c0 comp rx=0 tx=0).stop 2023-11-09T03:54:22.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.532+0000 7fa96edb6640 1 -- 172.21.15.76:0/3039395329 shutdown_connections 2023-11-09T03:54:22.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.532+0000 7fa96edb6640 1 --2- 172.21.15.76:0/3039395329 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa95406ae70 0x7fa95406d330 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:22.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.532+0000 7fa96edb6640 1 --2- 172.21.15.76:0/3039395329 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa96815d4e0 0x7fa9680b7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:22.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.532+0000 7fa96edb6640 1 --2- 172.21.15.76:0/3039395329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa96815d100 0x7fa9680cb060 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:22.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.532+0000 7fa96edb6640 1 --2- 172.21.15.76:0/3039395329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9680d3b60 0x7fa968151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:22.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.532+0000 7fa96edb6640 1 -- 172.21.15.76:0/3039395329 >> 172.21.15.76:0/3039395329 conn(0x7fa9680d2ef0 msgr2=0x7fa9680d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:22.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.533+0000 7fa96edb6640 1 -- 172.21.15.76:0/3039395329 shutdown_connections 2023-11-09T03:54:22.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.533+0000 7fa96edb6640 1 -- 172.21.15.76:0/3039395329 wait complete. 2023-11-09T03:54:22.544 DEBUG:tasks.cephfs.mount:Force/lazy unmounting on client.0 2023-11-09T03:54:22.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:54:22.544 DEBUG:teuthology.orchestra.run.smithi076:> sudo umount --lazy --force /home/ubuntu/cephtest/mnt.0 2023-11-09T03:54:22.571 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[127808]: fuse finished with error 0 and tester_r 0 2023-11-09T03:54:22.572 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T03:54:22.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:54:22.573 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-09T03:54:22.628 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:54:22.629 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:54:22.684 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056381,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14761 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248868k,nr_inodes=812217,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-09T03:54:22.684 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-09T03:54:22.759 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:54:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:54:22.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.828+0000 7ff0805cc640 1 Processor -- start 2023-11-09T03:54:22.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.828+0000 7ff0805cc640 1 -- start start 2023-11-09T03:54:22.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.829+0000 7ff0805cc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff078159e40 0x7ff0780b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:22.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.829+0000 7ff0805cc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff078071500 con 0x7ff078070700 2023-11-09T03:54:22.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.829+0000 7ff0805cc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff07815dae0 con 0x7ff07806bb60 2023-11-09T03:54:22.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.829+0000 7ff0805cc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff078070c60 con 0x7ff078159e40 2023-11-09T03:54:22.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.829+0000 7ff07eb69640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff078070700 0x7ff0780cd790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35350/0 (socket says 172.21.15.76:35350) 2023-11-09T03:54:22.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.829+0000 7ff07eb69640 1 -- 172.21.15.76:0/3596599781 learned_addr learned my addr 172.21.15.76:0/3596599781 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:54:22.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.829+0000 7ff07e368640 1 --2- 172.21.15.76:0/3596599781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff078159e40 0x7ff0780b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:22.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.829+0000 7ff07d366640 1 -- 172.21.15.76:0/3596599781 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1450723235 0 0) 0x7ff078071500 con 0x7ff078070700 2023-11-09T03:54:22.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.829+0000 7ff07d366640 1 -- 172.21.15.76:0/3596599781 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff064003520 con 0x7ff078070700 2023-11-09T03:54:22.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.829+0000 7ff07d366640 1 -- 172.21.15.76:0/3596599781 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4002900715 0 0) 0x7ff064003520 con 0x7ff078070700 2023-11-09T03:54:22.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.829+0000 7ff07d366640 1 -- 172.21.15.76:0/3596599781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff078159e40 msgr2=0x7ff0780b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:22.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.829+0000 7ff07d366640 1 --2- 172.21.15.76:0/3596599781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff078159e40 0x7ff0780b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:22.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.829+0000 7ff07d366640 1 -- 172.21.15.76:0/3596599781 >> v1:172.21.15.113:6789/0 conn(0x7ff07806bb60 legacy=0x7ff07806fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:22.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.829+0000 7ff07d366640 1 -- 172.21.15.76:0/3596599781 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff078070f40 con 0x7ff078070700 2023-11-09T03:54:22.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.829+0000 7ff07d366640 1 -- 172.21.15.76:0/3596599781 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7ff070001670 con 0x7ff078070700 2023-11-09T03:54:22.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.830+0000 7ff07d366640 1 -- 172.21.15.76:0/3596599781 >> v1:172.21.15.76:6789/0 conn(0x7ff078070700 legacy=0x7ff0780cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:22.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.830+0000 7ff07d366640 1 --2- 172.21.15.76:0/3596599781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0640039a0 0x7ff064003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:22.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.830+0000 7ff07d366640 1 --2- 172.21.15.76:0/3596599781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff078159e40 0x7ff0640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:22.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.830+0000 7ff07d366640 1 --2- 172.21.15.76:0/3596599781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff064004d00 0x7ff064005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:22.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.830+0000 7ff07d366640 1 -- 172.21.15.76:0/3596599781 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff078070c60 con 0x7ff064004d00 2023-11-09T03:54:22.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.830+0000 7ff07d366640 1 -- 172.21.15.76:0/3596599781 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06400bab0 con 0x7ff0640039a0 2023-11-09T03:54:22.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.830+0000 7ff07d366640 1 -- 172.21.15.76:0/3596599781 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06400de80 con 0x7ff078159e40 2023-11-09T03:54:22.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.830+0000 7ff07eb69640 1 --2- 172.21.15.76:0/3596599781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff064004d00 0x7ff064005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:22.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.830+0000 7ff07e368640 1 --2- 172.21.15.76:0/3596599781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0640039a0 0x7ff064003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:22.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.830+0000 7ff07db67640 1 --2- 172.21.15.76:0/3596599781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff078159e40 0x7ff0640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:22.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.830+0000 7ff07eb69640 1 -- 172.21.15.76:0/3596599781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff078159e40 msgr2=0x7ff0640047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:22.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.830+0000 7ff07eb69640 1 --2- 172.21.15.76:0/3596599781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff078159e40 0x7ff0640047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:22.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.830+0000 7ff07eb69640 1 -- 172.21.15.76:0/3596599781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0640039a0 msgr2=0x7ff064003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:22.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.830+0000 7ff07eb69640 1 --2- 172.21.15.76:0/3596599781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0640039a0 0x7ff064003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:22.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.830+0000 7ff07eb69640 1 -- 172.21.15.76:0/3596599781 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff064003520 con 0x7ff064004d00 2023-11-09T03:54:22.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.830+0000 7ff07db67640 1 --2- 172.21.15.76:0/3596599781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff078159e40 0x7ff0640047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:54:22.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.831+0000 7ff07eb69640 1 --2- 172.21.15.76:0/3596599781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff064004d00 0x7ff064005180 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7ff07000c210 tx=0x7ff0640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169e10606c12763f server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:22.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.831+0000 7ff07d366640 1 -- 172.21.15.76:0/3596599781 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff070005ad0 con 0x7ff064004d00 2023-11-09T03:54:22.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.831+0000 7ff07d366640 1 -- 172.21.15.76:0/3596599781 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff070005c50 con 0x7ff064004d00 2023-11-09T03:54:22.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.831+0000 7ff07d366640 1 -- 172.21.15.76:0/3596599781 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07002c890 con 0x7ff064004d00 2023-11-09T03:54:22.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.831+0000 7ff0805cc640 1 -- 172.21.15.76:0/3596599781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff064004d00 msgr2=0x7ff064005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:22.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.831+0000 7ff0805cc640 1 --2- 172.21.15.76:0/3596599781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff064004d00 0x7ff064005180 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7ff07000c210 tx=0x7ff0640057f0 comp rx=0 tx=0).stop 2023-11-09T03:54:22.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.831+0000 7ff07eb69640 1 -- 172.21.15.76:0/3596599781 reap_dead start 2023-11-09T03:54:22.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.831+0000 7ff0805cc640 1 -- 172.21.15.76:0/3596599781 shutdown_connections 2023-11-09T03:54:22.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.831+0000 7ff0805cc640 1 -- 172.21.15.76:0/3596599781 >> 172.21.15.76:0/3596599781 conn(0x7ff0780cb530 msgr2=0x7ff0780cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:22.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.831+0000 7ff0805cc640 1 -- 172.21.15.76:0/3596599781 shutdown_connections 2023-11-09T03:54:22.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.831+0000 7ff0805cc640 1 -- 172.21.15.76:0/3596599781 wait complete. 2023-11-09T03:54:22.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.833+0000 7ff0805cc640 1 Processor -- start 2023-11-09T03:54:22.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.833+0000 7ff0805cc640 1 -- start start 2023-11-09T03:54:22.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.833+0000 7ff0805cc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff064004d00 0x7ff07815a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:22.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.833+0000 7ff0805cc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0780afd10 0x7ff07813db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:22.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.833+0000 7ff0805cc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff07813e070 0x7ff07815c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:22.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.833+0000 7ff0805cc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0780b0700 con 0x7ff07813e070 2023-11-09T03:54:22.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.833+0000 7ff0805cc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0780b09e0 con 0x7ff064004d00 2023-11-09T03:54:22.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.833+0000 7ff0805cc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff07815be70 con 0x7ff0780afd10 2023-11-09T03:54:22.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.833+0000 7ff07eb69640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff07813e070 0x7ff07815c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:22.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.833+0000 7ff07eb69640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff07813e070 0x7ff07815c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43632/0 (socket says 172.21.15.76:43632) 2023-11-09T03:54:22.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.833+0000 7ff07eb69640 1 -- 172.21.15.76:0/3903868223 learned_addr learned my addr 172.21.15.76:0/3903868223 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:22.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.834+0000 7ff07e368640 1 --2- 172.21.15.76:0/3903868223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff064004d00 0x7ff07815a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:22.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.834+0000 7ff07db67640 1 --2- 172.21.15.76:0/3903868223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0780afd10 0x7ff07813db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:22.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.834+0000 7ff07eb69640 1 -- 172.21.15.76:0/3903868223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0780afd10 msgr2=0x7ff07813db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:22.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.834+0000 7ff07eb69640 1 --2- 172.21.15.76:0/3903868223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0780afd10 0x7ff07813db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:22.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.834+0000 7ff07eb69640 1 -- 172.21.15.76:0/3903868223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff064004d00 msgr2=0x7ff07815a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:22.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.834+0000 7ff07eb69640 1 --2- 172.21.15.76:0/3903868223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff064004d00 0x7ff07815a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:22.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.834+0000 7ff07eb69640 1 -- 172.21.15.76:0/3903868223 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07815c150 con 0x7ff07813e070 2023-11-09T03:54:22.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.834+0000 7ff07eb69640 1 --2- 172.21.15.76:0/3903868223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff07813e070 0x7ff07815c950 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7ff07002c430 tx=0x7ff07002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:22.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.834+0000 7ff06effd640 1 -- 172.21.15.76:0/3903868223 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07003c040 con 0x7ff07813e070 2023-11-09T03:54:22.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.834+0000 7ff06effd640 1 -- 172.21.15.76:0/3903868223 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07002cdd0 con 0x7ff07813e070 2023-11-09T03:54:22.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.834+0000 7ff06effd640 1 -- 172.21.15.76:0/3903868223 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff070007290 con 0x7ff07813e070 2023-11-09T03:54:22.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.834+0000 7ff0805cc640 1 -- 172.21.15.76:0/3903868223 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff07815c430 con 0x7ff07813e070 2023-11-09T03:54:22.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.834+0000 7ff0805cc640 1 -- 172.21.15.76:0/3903868223 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff07814f2b0 con 0x7ff07813e070 2023-11-09T03:54:22.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.835+0000 7ff0805cc640 1 -- 172.21.15.76:0/3903868223 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff07815bcd0 con 0x7ff07813e070 2023-11-09T03:54:22.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.836+0000 7ff06effd640 1 -- 172.21.15.76:0/3903868223 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff070032070 con 0x7ff07813e070 2023-11-09T03:54:22.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.836+0000 7ff06effd640 1 --2- 172.21.15.76:0/3903868223 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff05c06ae70 0x7ff05c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:22.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.836+0000 7ff06effd640 1 -- 172.21.15.76:0/3903868223 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7ff0700aaf40 con 0x7ff07813e070 2023-11-09T03:54:22.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.836+0000 7ff07e368640 1 --2- 172.21.15.76:0/3903868223 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff05c06ae70 0x7ff05c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:22.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.837+0000 7ff07e368640 1 --2- 172.21.15.76:0/3903868223 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff05c06ae70 0x7ff05c06d330 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff074014510 tx=0x7ff07400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:22.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:22.840+0000 7ff06effd640 1 -- 172.21.15.76:0/3903868223 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff07815bcd0 con 0x7ff07813e070 2023-11-09T03:54:23.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:23.041+0000 7ff0805cc640 1 -- 172.21.15.76:0/3903868223 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff07815bcd0 con 0x7ff07813e070 2023-11-09T03:54:23.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:23.041+0000 7ff06effd640 1 -- 172.21.15.76:0/3903868223 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v125) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff07815bcd0 con 0x7ff07813e070 2023-11-09T03:54:23.042 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-09T03:54:23.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:23.043+0000 7ff0805cc640 1 -- 172.21.15.76:0/3903868223 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff05c06ae70 msgr2=0x7ff05c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:23.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:23.043+0000 7ff0805cc640 1 --2- 172.21.15.76:0/3903868223 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff05c06ae70 0x7ff05c06d330 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff074014510 tx=0x7ff07400b040 comp rx=0 tx=0).stop 2023-11-09T03:54:23.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:23.043+0000 7ff0805cc640 1 -- 172.21.15.76:0/3903868223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff07813e070 msgr2=0x7ff07815c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:23.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:23.043+0000 7ff0805cc640 1 --2- 172.21.15.76:0/3903868223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff07813e070 0x7ff07815c950 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7ff07002c430 tx=0x7ff07002e070 comp rx=0 tx=0).stop 2023-11-09T03:54:23.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:23.043+0000 7ff0805cc640 1 -- 172.21.15.76:0/3903868223 shutdown_connections 2023-11-09T03:54:23.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:23.043+0000 7ff0805cc640 1 --2- 172.21.15.76:0/3903868223 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff05c06ae70 0x7ff05c06d330 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:23.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:23.043+0000 7ff0805cc640 1 --2- 172.21.15.76:0/3903868223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff07813e070 0x7ff07815c950 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:23.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:23.043+0000 7ff0805cc640 1 --2- 172.21.15.76:0/3903868223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0780afd10 0x7ff07813db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:23.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:23.043+0000 7ff0805cc640 1 --2- 172.21.15.76:0/3903868223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff064004d00 0x7ff07815a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:23.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:23.043+0000 7ff0805cc640 1 -- 172.21.15.76:0/3903868223 >> 172.21.15.76:0/3903868223 conn(0x7ff0780cb530 msgr2=0x7ff07806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:23.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:23.043+0000 7ff0805cc640 1 -- 172.21.15.76:0/3903868223 shutdown_connections 2023-11-09T03:54:23.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:23.043+0000 7ff0805cc640 1 -- 172.21.15.76:0/3903868223 wait complete. 2023-11-09T03:54:23.054 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-09T03:54:23.055 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-09T03:54:23.055 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2023-11-09T03:54:23.056 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-09T03:54:23.056 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-09T03:54:23.056 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-09T03:54:23.057 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-09T03:54:23.057 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-09T03:54:23.057 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:54:23.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:54:23.058 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2023-11-09T03:54:23.110 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-09T03:54:23.110 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-09T03:54:23.110 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2023-11-09T03:54:23.111 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:54:23.111 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2023-11-09T03:54:23.111 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:54:23.111 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:54:23.112 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:54:23.112 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2023-11-09T03:54:23.112 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:54:23.112 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:54:23.113 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2023-11-09T03:54:23.113 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:54:23.113 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:54:23.113 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-09T03:54:23.114 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:54:23.114 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-09T03:54:23.114 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42086sec preferred_lft 42086sec 2023-11-09T03:54:23.114 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2023-11-09T03:54:23.115 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:54:23.115 INFO:teuthology.orchestra.run.smithi076.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-09T03:54:23.115 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2e:15:bd:03:a8:2e brd ff:ff:ff:ff:ff:ff 2023-11-09T03:54:23.115 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-09T03:54:23.116 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:54:23.116 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::bcda:38ff:fed4:fdb0/64 scope link 2023-11-09T03:54:23.116 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:54:23.116 INFO:teuthology.orchestra.run.smithi076.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-09T03:54:23.116 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 6a:8d:94:d7:e5:00 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:54:23.117 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::688d:94ff:fed7:e500/64 scope link 2023-11-09T03:54:23.117 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:54:23.117 INFO:teuthology.orchestra.run.smithi076.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-09T03:54:23.117 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2e:15:bd:03:a8:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:54:23.118 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::2c15:bdff:fe03:a82e/64 scope link 2023-11-09T03:54:23.118 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:54:23.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:54:23.119 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-09T03:54:23.176 INFO:teuthology.orchestra.run.smithi076.stdout:1 2023-11-09T03:54:23.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:54:23.178 DEBUG:teuthology.orchestra.run.smithi076:> route 2023-11-09T03:54:23.234 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2023-11-09T03:54:23.234 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-09T03:54:23.234 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-09T03:54:23.235 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-09T03:54:23.235 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-09T03:54:23.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:54:23.236 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-09T03:54:23.236 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-09T03:54:23.237 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-09T03:54:23.237 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-09T03:54:23.237 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-09T03:54:23.238 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-09T03:54:23.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:54:23.330 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2023-11-09T03:54:23.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-09T03:54:23.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-09T03:54:23.345 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:54:23.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:54:23.346 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2023-11-09T03:54:23.412 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-09T03:54:23.413 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:54:23.413 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-09T03:54:23.467 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:54:23.468 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:54:23.468 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-09T03:54:23.522 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:54:23.577 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:54:23.577 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:54:23.578 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:54:23.578 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:54:23.578 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:54:23.578 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:54:23.579 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:54:23.579 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:54:23.579 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:54:23.579 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:54:23.580 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:54:23.580 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:54:23.580 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:54:23.580 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:54:23.580 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:54:23.580 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:54:23.581 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:54:23.581 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:54:23.581 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:54:23.581 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:54:23.581 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:54:23.582 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:54:23.582 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:54:23.582 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:54:23.582 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:54:23.582 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:54:23.583 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:54:23.583 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:54:23.583 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:54:23.583 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:54:23.583 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:54:23.584 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:54:23.584 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:54:23.584 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:54:23.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:54:23.585 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:54:23.631 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-09T03:54:23.632 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-09T03:54:23.674 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:54:23.688 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:54:23.689 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:54:23.689 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:54:23.689 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:54:23.689 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:54:23.689 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:54:23.690 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:54:23.690 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:54:23.690 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:54:23.690 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:54:23.691 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:54:23.691 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:54:23.691 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:54:23.691 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:54:23.691 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:54:23.692 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:54:23.692 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:54:23.692 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:54:23.692 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:54:23.692 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:54:23.693 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:54:23.693 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:54:23.693 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:54:23.693 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:54:23.693 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:54:23.694 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:54:23.694 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:54:23.694 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:54:23.694 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:54:23.694 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:54:23.695 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:54:23.695 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:54:23.695 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:54:23.695 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:54:23.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:54:23.696 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:54:23.763 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:54:23.763+0000 7ff3b0c34480 -1 init, newargv = 0x559ad868cef0 newargc=15 2023-11-09T03:54:23.764 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[129903]: starting ceph client 2023-11-09T03:54:23.770 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[129903]: starting fuse 2023-11-09T03:54:24.844 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:54:24.863 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:54:24.863 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:54:24.863 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:54:24.864 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:54:24.864 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:54:24.864 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:54:24.864 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:54:24.864 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:54:24.865 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:54:24.865 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:54:24.865 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:54:24.865 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:54:24.865 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:54:24.865 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:54:24.866 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:54:24.866 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:54:24.866 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:54:24.866 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:54:24.866 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:54:24.867 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:54:24.867 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:54:24.867 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:54:24.867 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:54:24.867 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:54:24.868 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:54:24.868 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:54:24.868 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:54:24.868 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:54:24.868 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:54:24.868 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:54:24.869 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:54:24.869 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:54:24.869 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:54:24.869 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:54:24.869 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-09T03:54:24.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:54:24.870 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:54:24.927 INFO:teuthology.orchestra.run.smithi076.stdout:43 2023-11-09T03:54:24.928 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-09T03:54:24.928 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer -- python3 -c ' 2023-11-09T03:54:24.929 DEBUG:teuthology.orchestra.run.smithi076:> import glob 2023-11-09T03:54:24.929 DEBUG:teuthology.orchestra.run.smithi076:> import re 2023-11-09T03:54:24.929 DEBUG:teuthology.orchestra.run.smithi076:> import os 2023-11-09T03:54:24.930 DEBUG:teuthology.orchestra.run.smithi076:> import subprocess 2023-11-09T03:54:24.930 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:54:24.930 DEBUG:teuthology.orchestra.run.smithi076:> def _find_admin_socket(client_name): 2023-11-09T03:54:24.931 DEBUG:teuthology.orchestra.run.smithi076:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-09T03:54:24.931 DEBUG:teuthology.orchestra.run.smithi076:> files = glob.glob(asok_path) 2023-11-09T03:54:24.931 DEBUG:teuthology.orchestra.run.smithi076:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-09T03:54:24.932 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:54:24.932 DEBUG:teuthology.orchestra.run.smithi076:> # Given a non-glob path, it better be there 2023-11-09T03:54:24.932 DEBUG:teuthology.orchestra.run.smithi076:> if "*" not in asok_path: 2023-11-09T03:54:24.933 DEBUG:teuthology.orchestra.run.smithi076:> assert(len(files) == 1) 2023-11-09T03:54:24.933 DEBUG:teuthology.orchestra.run.smithi076:> return files[0] 2023-11-09T03:54:24.933 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:54:24.934 DEBUG:teuthology.orchestra.run.smithi076:> for f in files: 2023-11-09T03:54:24.934 DEBUG:teuthology.orchestra.run.smithi076:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-09T03:54:24.934 DEBUG:teuthology.orchestra.run.smithi076:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-09T03:54:24.935 DEBUG:teuthology.orchestra.run.smithi076:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-09T03:54:24.935 DEBUG:teuthology.orchestra.run.smithi076:> contents = proc_f.read() 2023-11-09T03:54:24.935 DEBUG:teuthology.orchestra.run.smithi076:> if mountpoint in contents: 2023-11-09T03:54:24.936 DEBUG:teuthology.orchestra.run.smithi076:> return f 2023-11-09T03:54:24.936 DEBUG:teuthology.orchestra.run.smithi076:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-09T03:54:24.936 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:54:24.937 DEBUG:teuthology.orchestra.run.smithi076:> print(_find_admin_socket("client.0")) 2023-11-09T03:54:24.937 DEBUG:teuthology.orchestra.run.smithi076:> ' 2023-11-09T03:54:25.060 INFO:teuthology.orchestra.run.smithi076.stdout:/var/run/ceph/ceph-client.0.129903.asok 2023-11-09T03:54:25.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:54:25.071 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.129903.asok 2023-11-09T03:54:25.072 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:54:25.072 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.129903.asok status 2023-11-09T03:54:25.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:54:25.241 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:54:25.241 INFO:teuthology.orchestra.run.smithi076.stdout: "metadata": { 2023-11-09T03:54:25.241 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_sha1": "7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4", 2023-11-09T03:54:25.241 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev)", 2023-11-09T03:54:25.241 INFO:teuthology.orchestra.run.smithi076.stdout: "entity_id": "0", 2023-11-09T03:54:25.241 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": "smithi076", 2023-11-09T03:54:25.242 INFO:teuthology.orchestra.run.smithi076.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-09T03:54:25.242 INFO:teuthology.orchestra.run.smithi076.stdout: "pid": "129903", 2023-11-09T03:54:25.242 INFO:teuthology.orchestra.run.smithi076.stdout: "root": "/" 2023-11-09T03:54:25.242 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:54:25.242 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_count": 0, 2023-11-09T03:54:25.243 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_pinned_count": 0, 2023-11-09T03:54:25.243 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 9180, 2023-11-09T03:54:25.243 INFO:teuthology.orchestra.run.smithi076.stdout: "inst": { 2023-11-09T03:54:25.243 INFO:teuthology.orchestra.run.smithi076.stdout: "name": { 2023-11-09T03:54:25.243 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "client", 2023-11-09T03:54:25.243 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 9180 2023-11-09T03:54:25.244 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:54:25.244 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2023-11-09T03:54:25.244 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2023-11-09T03:54:25.244 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2023-11-09T03:54:25.244 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3870807407 2023-11-09T03:54:25.244 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:54:25.245 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:54:25.245 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2023-11-09T03:54:25.245 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2023-11-09T03:54:25.245 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2023-11-09T03:54:25.245 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3870807407 2023-11-09T03:54:25.246 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:54:25.246 INFO:teuthology.orchestra.run.smithi076.stdout: "inst_str": "client.9180 192.168.0.1:0/3870807407", 2023-11-09T03:54:25.246 INFO:teuthology.orchestra.run.smithi076.stdout: "addr_str": "192.168.0.1:0/3870807407", 2023-11-09T03:54:25.246 INFO:teuthology.orchestra.run.smithi076.stdout: "inode_count": 1, 2023-11-09T03:54:25.246 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_epoch": 125, 2023-11-09T03:54:25.246 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch": 116, 2023-11-09T03:54:25.247 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch_barrier": 0, 2023-11-09T03:54:25.247 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklisted": false, 2023-11-09T03:54:25.247 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs" 2023-11-09T03:54:25.247 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:54:25.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:54:25.255 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:54:25.256 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-11-09T03:54:25.256 DEBUG:teuthology.orchestra.run.smithi076:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-09T03:54:25.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:54:25.315 INFO:teuthology.orchestra.run.smithi076.stdout:fuseblk 2023-11-09T03:54:25.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:54:25.321 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-09T03:54:25.322 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:54:25.322 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-09T03:54:25.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:54:25.381 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:54:25.387 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 flush journal 2023-11-09T03:54:25.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.492+0000 7f79879d7640 1 Processor -- start 2023-11-09T03:54:25.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.492+0000 7f79879d7640 1 -- start start 2023-11-09T03:54:25.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.493+0000 7f79879d7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f798006d9c0 0x7f79800710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:25.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.493+0000 7f79879d7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f798006cc90 con 0x7f798006c5f0 2023-11-09T03:54:25.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.493+0000 7f79879d7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79800b0e60 con 0x7f79800ade00 2023-11-09T03:54:25.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.493+0000 7f79879d7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79800b0c40 con 0x7f798006d9c0 2023-11-09T03:54:25.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.493+0000 7f7984f72640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f798006d9c0 0x7f79800710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:25.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.493+0000 7f7985f74640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f798006c5f0 0x7f79800cc680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35364/0 (socket says 172.21.15.76:35364) 2023-11-09T03:54:25.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.493+0000 7f7985f74640 1 -- 172.21.15.76:0/132062868 learned_addr learned my addr 172.21.15.76:0/132062868 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:54:25.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.493+0000 7f7984f72640 1 -- 172.21.15.76:0/132062868 >> v1:172.21.15.113:6789/0 conn(0x7f79800ade00 legacy=0x7f79800b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:25.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.493+0000 7f7984f72640 1 -- 172.21.15.76:0/132062868 >> v1:172.21.15.76:6789/0 conn(0x7f798006c5f0 legacy=0x7f79800cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:25.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.493+0000 7f7984f72640 1 -- 172.21.15.76:0/132062868 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798006cee0 con 0x7f798006d9c0 2023-11-09T03:54:25.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.494+0000 7f7984f72640 1 --2- 172.21.15.76:0/132062868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f798006d9c0 0x7f79800710d0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f79700097a0 tx=0x7f797002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0b30dc4b3ebbb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:25.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.494+0000 7f7977fff640 1 -- 172.21.15.76:0/132062868 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797002e070 con 0x7f798006d9c0 2023-11-09T03:54:25.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.494+0000 7f7977fff640 1 -- 172.21.15.76:0/132062868 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79700315a0 con 0x7f798006d9c0 2023-11-09T03:54:25.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.494+0000 7f7977fff640 1 -- 172.21.15.76:0/132062868 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7970031930 con 0x7f798006d9c0 2023-11-09T03:54:25.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.494+0000 7f79879d7640 1 -- 172.21.15.76:0/132062868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f798006d9c0 msgr2=0x7f79800710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:25.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.494+0000 7f79879d7640 1 --2- 172.21.15.76:0/132062868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f798006d9c0 0x7f79800710d0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f79700097a0 tx=0x7f797002b840 comp rx=0 tx=0).stop 2023-11-09T03:54:25.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.494+0000 7f79879d7640 1 -- 172.21.15.76:0/132062868 shutdown_connections 2023-11-09T03:54:25.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.494+0000 7f79879d7640 1 --2- 172.21.15.76:0/132062868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f798006d9c0 0x7f79800710d0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.494+0000 7f79879d7640 1 -- 172.21.15.76:0/132062868 >> 172.21.15.76:0/132062868 conn(0x7f79800cb530 msgr2=0x7f79800cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:25.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.495+0000 7f79879d7640 1 -- 172.21.15.76:0/132062868 shutdown_connections 2023-11-09T03:54:25.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.495+0000 7f79879d7640 1 -- 172.21.15.76:0/132062868 wait complete. 2023-11-09T03:54:25.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.496+0000 7f79879d7640 1 Processor -- start 2023-11-09T03:54:25.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.496+0000 7f79879d7640 1 -- start start 2023-11-09T03:54:25.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.496+0000 7f79879d7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f798006c5f0 0x7f798015cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:25.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.496+0000 7f79879d7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f798006d9c0 0x7f79800a9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:25.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.496+0000 7f79879d7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79800ade00 0x7f79800b5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:25.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.496+0000 7f79879d7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79800aa970 con 0x7f79800ade00 2023-11-09T03:54:25.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.496+0000 7f79879d7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79800aac50 con 0x7f798006d9c0 2023-11-09T03:54:25.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.496+0000 7f79879d7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79800aaf30 con 0x7f798006c5f0 2023-11-09T03:54:25.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.496+0000 7f7985f74640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79800ade00 0x7f79800b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:25.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.496+0000 7f7985f74640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79800ade00 0x7f79800b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43646/0 (socket says 172.21.15.76:43646) 2023-11-09T03:54:25.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.496+0000 7f7985f74640 1 -- 172.21.15.76:0/2940091783 learned_addr learned my addr 172.21.15.76:0/2940091783 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:25.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.497+0000 7f7984f72640 1 --2- 172.21.15.76:0/2940091783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f798006d9c0 0x7f79800a9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:25.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.497+0000 7f7985773640 1 --2- 172.21.15.76:0/2940091783 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f798006c5f0 0x7f798015cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:25.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.497+0000 7f7985f74640 1 -- 172.21.15.76:0/2940091783 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f798006c5f0 msgr2=0x7f798015cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:25.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.497+0000 7f7985f74640 1 --2- 172.21.15.76:0/2940091783 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f798006c5f0 0x7f798015cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.497+0000 7f7985f74640 1 -- 172.21.15.76:0/2940091783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f798006d9c0 msgr2=0x7f79800a9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:25.502 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.497+0000 7f7985f74640 1 --2- 172.21.15.76:0/2940091783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f798006d9c0 0x7f79800a9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.497+0000 7f7985f74640 1 -- 172.21.15.76:0/2940091783 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7970009400 con 0x7f79800ade00 2023-11-09T03:54:25.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.497+0000 7f7985773640 1 --2- 172.21.15.76:0/2940091783 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f798006c5f0 0x7f798015cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:54:25.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.497+0000 7f7984f72640 1 --2- 172.21.15.76:0/2940091783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f798006d9c0 0x7f79800a9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:54:25.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.497+0000 7f7985f74640 1 --2- 172.21.15.76:0/2940091783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79800ade00 0x7f79800b5430 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f797800ca80 tx=0x7f797800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:25.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.497+0000 7f7975ffb640 1 -- 172.21.15.76:0/2940091783 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797800ac30 con 0x7f79800ade00 2023-11-09T03:54:25.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.497+0000 7f7975ffb640 1 -- 172.21.15.76:0/2940091783 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797800ad90 con 0x7f79800ade00 2023-11-09T03:54:25.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.497+0000 7f79879d7640 1 -- 172.21.15.76:0/2940091783 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79801f13e0 con 0x7f79800ade00 2023-11-09T03:54:25.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.497+0000 7f7975ffb640 1 -- 172.21.15.76:0/2940091783 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7978012600 con 0x7f79800ade00 2023-11-09T03:54:25.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.497+0000 7f79879d7640 1 -- 172.21.15.76:0/2940091783 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79800abce0 con 0x7f79800ade00 2023-11-09T03:54:25.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.499+0000 7f7975ffb640 1 -- 172.21.15.76:0/2940091783 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7978007070 con 0x7f79800ade00 2023-11-09T03:54:25.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.499+0000 7f7975ffb640 1 --2- 172.21.15.76:0/2940091783 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f796406aea0 0x7f796406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:25.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.499+0000 7f7975ffb640 1 -- 172.21.15.76:0/2940091783 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7f7978089df0 con 0x7f79800ade00 2023-11-09T03:54:25.510 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.499+0000 7f7985773640 1 --2- 172.21.15.76:0/2940091783 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f796406aea0 0x7f796406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:25.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.500+0000 7f7985773640 1 --2- 172.21.15.76:0/2940091783 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f796406aea0 0x7f796406d360 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f7980071a50 tx=0x7f797c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:25.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.500+0000 7f798a56a740 1 Processor -- start 2023-11-09T03:54:25.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.500+0000 7f798a56a740 1 -- start start 2023-11-09T03:54:25.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.501+0000 7f798a56a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555a5532b4c0 0x555a5532b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:25.511 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.501+0000 7f798a56a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a552fb6d0 0x555a55328730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:25.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.501+0000 7f798a56a740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555a552ff040 0x555a55328c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:25.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.501+0000 7f798a56a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a5526ed70 con 0x555a552fb6d0 2023-11-09T03:54:25.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.501+0000 7f798a56a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a5526c460 con 0x555a5532b4c0 2023-11-09T03:54:25.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.501+0000 7f798a56a740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a55267d50 con 0x555a552ff040 2023-11-09T03:54:25.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.501+0000 7f7984f72640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a552fb6d0 0x555a55328730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:25.512 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.501+0000 7f7984f72640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a552fb6d0 0x555a55328730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43660/0 (socket says 172.21.15.76:43660) 2023-11-09T03:54:25.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.501+0000 7f7984f72640 1 -- 172.21.15.76:0/2993886584 learned_addr learned my addr 172.21.15.76:0/2993886584 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:25.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.501+0000 7f7985773640 1 --2- 172.21.15.76:0/2993886584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555a5532b4c0 0x555a5532b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:25.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.501+0000 7f7985f74640 1 --2- 172.21.15.76:0/2993886584 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555a552ff040 0x555a55328c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:25.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.501+0000 7f7984f72640 1 -- 172.21.15.76:0/2993886584 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555a552ff040 msgr2=0x555a55328c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:25.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.501+0000 7f7984f72640 1 --2- 172.21.15.76:0/2993886584 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555a552ff040 0x555a55328c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.513 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.501+0000 7f7984f72640 1 -- 172.21.15.76:0/2993886584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555a5532b4c0 msgr2=0x555a5532b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:25.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.501+0000 7f7984f72640 1 --2- 172.21.15.76:0/2993886584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555a5532b4c0 0x555a5532b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.501+0000 7f7984f72640 1 -- 172.21.15.76:0/2993886584 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a5522ae30 con 0x555a552fb6d0 2023-11-09T03:54:25.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.501+0000 7f7985f74640 1 --2- 172.21.15.76:0/2993886584 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555a552ff040 0x555a55328c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:54:25.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.501+0000 7f7984f72640 1 --2- 172.21.15.76:0/2993886584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a552fb6d0 0x555a55328730 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f79700098c0 tx=0x7f7970008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ff40f72209d78d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:25.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.501+0000 7f796affd640 1 -- 172.21.15.76:0/2993886584 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7970049070 con 0x555a552fb6d0 2023-11-09T03:54:25.514 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.502+0000 7f796affd640 1 -- 172.21.15.76:0/2993886584 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7970031a50 con 0x555a552fb6d0 2023-11-09T03:54:25.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.502+0000 7f796affd640 1 -- 172.21.15.76:0/2993886584 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7970031dc0 con 0x555a552fb6d0 2023-11-09T03:54:25.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.502+0000 7f798a56a740 1 -- 172.21.15.76:0/2993886584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a552fb6d0 msgr2=0x555a55328730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:25.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.502+0000 7f798a56a740 1 --2- 172.21.15.76:0/2993886584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a552fb6d0 0x555a55328730 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f79700098c0 tx=0x7f7970008030 comp rx=0 tx=0).stop 2023-11-09T03:54:25.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.502+0000 7f798a56a740 1 -- 172.21.15.76:0/2993886584 shutdown_connections 2023-11-09T03:54:25.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.502+0000 7f798a56a740 1 --2- 172.21.15.76:0/2993886584 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555a552ff040 0x555a55328c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.502+0000 7f798a56a740 1 --2- 172.21.15.76:0/2993886584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a552fb6d0 0x555a55328730 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.502+0000 7f798a56a740 1 --2- 172.21.15.76:0/2993886584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555a5532b4c0 0x555a5532b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.502+0000 7f798a56a740 1 -- 172.21.15.76:0/2993886584 >> 172.21.15.76:0/2993886584 conn(0x555a552fd9e0 msgr2=0x555a553462d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:25.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.502+0000 7f798a56a740 1 -- 172.21.15.76:0/2993886584 shutdown_connections 2023-11-09T03:54:25.516 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.502+0000 7f798a56a740 1 -- 172.21.15.76:0/2993886584 wait complete. 2023-11-09T03:54:25.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.503+0000 7f798a56a740 1 -- start start 2023-11-09T03:54:25.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.503+0000 7f798a56a740 1 Processor -- start 2023-11-09T03:54:25.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.504+0000 7f798a56a740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555a552ff040 0x555a55325c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:25.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.504+0000 7f798a56a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555a5532b4c0 0x555a55326190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:25.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.504+0000 7f798a56a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a553242a0 0x555a5532dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:25.517 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.504+0000 7f798a56a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a55221ee0 con 0x555a553242a0 2023-11-09T03:54:25.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.504+0000 7f798a56a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a55221b80 con 0x555a5532b4c0 2023-11-09T03:54:25.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.504+0000 7f798a56a740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a551e73a0 con 0x555a552ff040 2023-11-09T03:54:25.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.504+0000 7f798a56a740 10 client.0 _ll_register_callbacks cb 0x555a551eb8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:54:25.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.504+0000 7f7985f74640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a553242a0 0x555a5532dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:25.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.504+0000 7f7985f74640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a553242a0 0x555a5532dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43670/0 (socket says 172.21.15.76:43670) 2023-11-09T03:54:25.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.504+0000 7f7985f74640 1 -- 172.21.15.76:0/3874503751 learned_addr learned my addr 172.21.15.76:0/3874503751 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:25.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.504+0000 7f7984f72640 1 --2- 172.21.15.76:0/3874503751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555a5532b4c0 0x555a55326190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:25.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.504+0000 7f7985773640 1 --2- 172.21.15.76:0/3874503751 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555a552ff040 0x555a55325c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:25.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.504+0000 7f7985f74640 1 -- 172.21.15.76:0/3874503751 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555a552ff040 msgr2=0x555a55325c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:25.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.504+0000 7f7985f74640 1 --2- 172.21.15.76:0/3874503751 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555a552ff040 0x555a55325c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.504+0000 7f7985f74640 1 -- 172.21.15.76:0/3874503751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555a5532b4c0 msgr2=0x555a55326190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:25.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.504+0000 7f7985f74640 1 --2- 172.21.15.76:0/3874503751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555a5532b4c0 0x555a55326190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.504+0000 7f7985f74640 1 -- 172.21.15.76:0/3874503751 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f798006cee0 con 0x555a553242a0 2023-11-09T03:54:25.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.504+0000 7f7985773640 1 --2- 172.21.15.76:0/3874503751 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555a552ff040 0x555a55325c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:54:25.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.504+0000 7f7985f74640 1 --2- 172.21.15.76:0/3874503751 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a553242a0 0x555a5532dfb0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f7978058130 tx=0x7f79780582e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912ae3f8e910b16d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:25.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.504+0000 7f795ffff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:54:25.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.504+0000 7f795ffff640 1 -- 172.21.15.76:0/3874503751 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7978056600 con 0x555a553242a0 2023-11-09T03:54:25.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.504+0000 7f795ffff640 1 -- 172.21.15.76:0/3874503751 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7f797806be40 con 0x555a553242a0 2023-11-09T03:54:25.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.504+0000 7f798a56a740 1 -- 172.21.15.76:0/3874503751 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a55260d30 con 0x555a553242a0 2023-11-09T03:54:25.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.504+0000 7f795ffff640 1 -- 172.21.15.76:0/3874503751 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f797805e0c0 con 0x555a553242a0 2023-11-09T03:54:25.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.504+0000 7f798a56a740 10 client.9195 fetch_fsmap learned FSMap version 125 2023-11-09T03:54:25.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.504+0000 7f798a56a740 1 -- 172.21.15.76:0/3874503751 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x555a552663d0 con 0x555a553242a0 2023-11-09T03:54:25.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.505+0000 7f795ffff640 1 -- 172.21.15.76:0/3874503751 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7978087030 con 0x555a553242a0 2023-11-09T03:54:25.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.505+0000 7f795ffff640 1 -- 172.21.15.76:0/3874503751 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 125) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f797805ff60 con 0x555a553242a0 2023-11-09T03:54:25.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.505+0000 7f798a56a740 10 client.9195 fetch_fsmap finished waiting for FSMap version 125 2023-11-09T03:54:25.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.505+0000 7f798a56a740 10 client.9195 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2023-11-09T03:54:25.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.505+0000 7f798a56a740 20 client.9195 populate_metadata read hostname 'smithi076' 2023-11-09T03:54:25.522 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.505+0000 7f798a56a740 1 --2- 172.21.15.76:0/3874503751 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x555a5532e9c0 0x555a55359930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:25.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.505+0000 7f798a56a740 4 client.9195 mds_command: new command op to 8568 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:54:25.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.505+0000 7f798a56a740 1 -- 172.21.15.76:0/3874503751 --> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a55260a50 con 0x555a5532e9c0 2023-11-09T03:54:25.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.505+0000 7f7985773640 1 --2- 172.21.15.76:0/3874503751 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x555a5532e9c0 0x555a55359930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:25.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.505+0000 7f795ffff640 10 client.9195 ms_handle_connect on v2:172.21.15.76:6832/153465374 2023-11-09T03:54:25.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.505+0000 7f7985773640 1 --2- 172.21.15.76:0/3874503751 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x555a5532e9c0 0x555a55359930 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66f49e43efb43d82 server_cookie=7d726eecd1d9206e in_seq=0 out_seq=0 2023-11-09T03:54:25.523 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.506+0000 7f795ffff640 1 -- 172.21.15.76:0/3874503751 <== mds.0 v2:172.21.15.76:6832/153465374 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555a55260a50 con 0x555a5532e9c0 2023-11-09T03:54:25.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.506+0000 7f795ffff640 10 client.9195 handle_command_reply: tid=0 2023-11-09T03:54:25.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.506+0000 7f798a56a740 1 client.9195 shutdown 2023-11-09T03:54:25.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.506+0000 7f798a56a740 1 -- 172.21.15.76:0/3874503751 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a553242a0 msgr2=0x555a5532dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:25.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.506+0000 7f798a56a740 1 --2- 172.21.15.76:0/3874503751 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a553242a0 0x555a5532dfb0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f7978058130 tx=0x7f79780582e0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.506+0000 7f798a56a740 1 -- 172.21.15.76:0/3874503751 shutdown_connections 2023-11-09T03:54:25.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.506+0000 7f798a56a740 1 --2- 172.21.15.76:0/3874503751 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x555a5532e9c0 0x555a55359930 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.506+0000 7f798a56a740 1 --2- 172.21.15.76:0/3874503751 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a553242a0 0x555a5532dfb0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.506+0000 7f798a56a740 1 --2- 172.21.15.76:0/3874503751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555a5532b4c0 0x555a55326190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.506+0000 7f795ffff640 0 client.9195 ms_handle_reset on v2:172.21.15.76:6832/153465374 2023-11-09T03:54:25.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.506+0000 7f798a56a740 1 --2- 172.21.15.76:0/3874503751 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555a552ff040 0x555a55325c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.506+0000 7f798a56a740 1 -- 172.21.15.76:0/3874503751 >> 172.21.15.76:0/3874503751 conn(0x555a552fb6d0 msgr2=0x555a552ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:25.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.506+0000 7f798a56a740 1 -- 172.21.15.76:0/3874503751 shutdown_connections 2023-11-09T03:54:25.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.506+0000 7f798a56a740 1 -- 172.21.15.76:0/3874503751 wait complete. 2023-11-09T03:54:25.526 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.506+0000 7f798a56a740 20 client.9195 trim_cache size 0 max 16384 2023-11-09T03:54:25.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.519+0000 7f798a56a740 1 Processor -- start 2023-11-09T03:54:25.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.519+0000 7f798a56a740 1 -- start start 2023-11-09T03:54:25.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.520+0000 7f798a56a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555a552ff040 0x555a55328620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:25.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.520+0000 7f798a56a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a553242a0 0x555a552fbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:25.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.520+0000 7f798a56a740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555a5532b4c0 0x555a55325e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:25.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.520+0000 7f798a56a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a553282e0 con 0x555a553242a0 2023-11-09T03:54:25.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.520+0000 7f798a56a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a55369340 con 0x555a552ff040 2023-11-09T03:54:25.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.520+0000 7f798a56a740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a552fd1b0 con 0x555a5532b4c0 2023-11-09T03:54:25.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.520+0000 7f7984f72640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a553242a0 0x555a552fbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:25.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.520+0000 7f7984f72640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a553242a0 0x555a552fbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43678/0 (socket says 172.21.15.76:43678) 2023-11-09T03:54:25.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.520+0000 7f7984f72640 1 -- 172.21.15.76:0/113721990 learned_addr learned my addr 172.21.15.76:0/113721990 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:25.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.520+0000 7f7984f72640 1 -- 172.21.15.76:0/113721990 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555a5532b4c0 msgr2=0x555a55325e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:25.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.520+0000 7f7984f72640 1 --2- 172.21.15.76:0/113721990 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555a5532b4c0 0x555a55325e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.520+0000 7f7984f72640 1 -- 172.21.15.76:0/113721990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555a552ff040 msgr2=0x555a55328620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:25.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.520+0000 7f7984f72640 1 --2- 172.21.15.76:0/113721990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555a552ff040 0x555a55328620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.520+0000 7f7984f72640 1 -- 172.21.15.76:0/113721990 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a55203ea0 con 0x555a553242a0 2023-11-09T03:54:25.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.520+0000 7f7984f72640 1 --2- 172.21.15.76:0/113721990 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a553242a0 0x555a552fbc00 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f7970047040 tx=0x7f7970030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939929c78aa98d1c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:25.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.521+0000 7f796b7fe640 1 -- 172.21.15.76:0/113721990 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7970049070 con 0x555a553242a0 2023-11-09T03:54:25.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.521+0000 7f796b7fe640 1 -- 172.21.15.76:0/113721990 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7970031bc0 con 0x555a553242a0 2023-11-09T03:54:25.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.521+0000 7f796b7fe640 1 -- 172.21.15.76:0/113721990 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7970004040 con 0x555a553242a0 2023-11-09T03:54:25.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.521+0000 7f798a56a740 1 -- 172.21.15.76:0/113721990 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a553242a0 msgr2=0x555a552fbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:25.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.521+0000 7f798a56a740 1 --2- 172.21.15.76:0/113721990 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a553242a0 0x555a552fbc00 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f7970047040 tx=0x7f7970030000 comp rx=0 tx=0).stop 2023-11-09T03:54:25.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.521+0000 7f798a56a740 1 -- 172.21.15.76:0/113721990 shutdown_connections 2023-11-09T03:54:25.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.521+0000 7f798a56a740 1 --2- 172.21.15.76:0/113721990 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555a5532b4c0 0x555a55325e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.521+0000 7f798a56a740 1 --2- 172.21.15.76:0/113721990 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a553242a0 0x555a552fbc00 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.521+0000 7f798a56a740 1 --2- 172.21.15.76:0/113721990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555a552ff040 0x555a55328620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.521+0000 7f798a56a740 1 -- 172.21.15.76:0/113721990 >> 172.21.15.76:0/113721990 conn(0x555a552fb6d0 msgr2=0x555a553202a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:25.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.521+0000 7f798a56a740 1 -- 172.21.15.76:0/113721990 shutdown_connections 2023-11-09T03:54:25.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.521+0000 7f798a56a740 1 -- 172.21.15.76:0/113721990 wait complete. 2023-11-09T03:54:25.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.522+0000 7f798a56a740 1 -- start start 2023-11-09T03:54:25.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.522+0000 7f798a56a740 1 Processor -- start 2023-11-09T03:54:25.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.523+0000 7f798a56a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a552ff040 0x555a553589c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:25.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.523+0000 7f798a56a740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555a553242a0 0x555a55358f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:25.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.523+0000 7f798a56a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555a5532b4c0 0x555a553b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:25.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.523+0000 7f798a56a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a553232e0 con 0x555a552ff040 2023-11-09T03:54:25.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.523+0000 7f798a56a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a553b5880 con 0x555a5532b4c0 2023-11-09T03:54:25.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.523+0000 7f798a56a740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a553234e0 con 0x555a553242a0 2023-11-09T03:54:25.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.523+0000 7f798a56a740 10 client.0 _ll_register_callbacks cb 0x7f7970009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:54:25.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.523+0000 7f7985773640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a552ff040 0x555a553589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:25.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.523+0000 7f7985773640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a552ff040 0x555a553589c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43682/0 (socket says 172.21.15.76:43682) 2023-11-09T03:54:25.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.523+0000 7f7985773640 1 -- 172.21.15.76:0/3825695204 learned_addr learned my addr 172.21.15.76:0/3825695204 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:25.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.523+0000 7f7985773640 1 -- 172.21.15.76:0/3825695204 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555a553242a0 msgr2=0x555a55358f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:25.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.523+0000 7f7985773640 1 --2- 172.21.15.76:0/3825695204 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555a553242a0 0x555a55358f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.523+0000 7f7985773640 1 -- 172.21.15.76:0/3825695204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555a5532b4c0 msgr2=0x555a553b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:25.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.523+0000 7f7985773640 1 --2- 172.21.15.76:0/3825695204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555a5532b4c0 0x555a553b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.523+0000 7f7985773640 1 -- 172.21.15.76:0/3825695204 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a5522ae30 con 0x555a552ff040 2023-11-09T03:54:25.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.523+0000 7f7985773640 1 --2- 172.21.15.76:0/3825695204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a552ff040 0x555a553589c0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f797c007e80 tx=0x7f797c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8015b620227b542d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:25.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.523+0000 7f795ffff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:54:25.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.523+0000 7f795ffff640 1 -- 172.21.15.76:0/3825695204 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c003a40 con 0x555a552ff040 2023-11-09T03:54:25.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.523+0000 7f795ffff640 1 -- 172.21.15.76:0/3825695204 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7f797c003ba0 con 0x555a552ff040 2023-11-09T03:54:25.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.523+0000 7f798a56a740 1 -- 172.21.15.76:0/3825695204 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7970031a50 con 0x555a552ff040 2023-11-09T03:54:25.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.524+0000 7f795ffff640 1 -- 172.21.15.76:0/3825695204 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f797c003d60 con 0x555a552ff040 2023-11-09T03:54:25.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.524+0000 7f798a56a740 10 client.9201 fetch_fsmap learned FSMap version 125 2023-11-09T03:54:25.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.524+0000 7f798a56a740 1 -- 172.21.15.76:0/3825695204 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x555a55229690 con 0x555a552ff040 2023-11-09T03:54:25.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.524+0000 7f795ffff640 1 -- 172.21.15.76:0/3825695204 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7970031a50 con 0x555a552ff040 2023-11-09T03:54:25.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.524+0000 7f795ffff640 1 -- 172.21.15.76:0/3825695204 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 125) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f797c026040 con 0x555a552ff040 2023-11-09T03:54:25.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.524+0000 7f798a56a740 10 client.9201 fetch_fsmap finished waiting for FSMap version 125 2023-11-09T03:54:25.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.524+0000 7f798a56a740 10 client.9201 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2023-11-09T03:54:25.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.524+0000 7f798a56a740 20 client.9201 populate_metadata read hostname 'smithi076' 2023-11-09T03:54:25.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.524+0000 7f798a56a740 1 --2- 172.21.15.76:0/3825695204 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x555a5532f3a0 0x555a553b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:25.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.524+0000 7f798a56a740 4 client.9201 mds_command: new command op to 8568 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-11-09T03:54:25.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.524+0000 7f798a56a740 1 -- 172.21.15.76:0/3825695204 --> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555a54f687e0 con 0x555a5532f3a0 2023-11-09T03:54:25.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.524+0000 7f7984f72640 1 --2- 172.21.15.76:0/3825695204 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x555a5532f3a0 0x555a553b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:25.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.524+0000 7f795ffff640 10 client.9201 ms_handle_connect on v2:172.21.15.76:6832/153465374 2023-11-09T03:54:25.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.524+0000 7f7984f72640 1 --2- 172.21.15.76:0/3825695204 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x555a5532f3a0 0x555a553b26a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b48c7cb0946d22e8 server_cookie=34874937699349b9 in_seq=0 out_seq=0 2023-11-09T03:54:25.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.534+0000 7f795ffff640 1 -- 172.21.15.76:0/3825695204 <== mds.0 v2:172.21.15.76:6832/153465374 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555a54f687e0 con 0x555a5532f3a0 2023-11-09T03:54:25.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.534+0000 7f795ffff640 10 client.9201 handle_command_reply: tid=0 2023-11-09T03:54:25.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.534+0000 7f798a56a740 1 client.9201 shutdown 2023-11-09T03:54:25.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.534+0000 7f798a56a740 1 -- 172.21.15.76:0/3825695204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a552ff040 msgr2=0x555a553589c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:25.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.534+0000 7f798a56a740 1 --2- 172.21.15.76:0/3825695204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a552ff040 0x555a553589c0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f797c007e80 tx=0x7f797c000ed0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.534+0000 7f798a56a740 1 -- 172.21.15.76:0/3825695204 shutdown_connections 2023-11-09T03:54:25.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.534+0000 7f798a56a740 1 --2- 172.21.15.76:0/3825695204 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x555a5532f3a0 0x555a553b26a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.534+0000 7f798a56a740 1 --2- 172.21.15.76:0/3825695204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555a5532b4c0 0x555a553b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.534+0000 7f798a56a740 1 --2- 172.21.15.76:0/3825695204 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555a553242a0 0x555a55358f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.534+0000 7f795ffff640 0 client.9201 ms_handle_reset on v2:172.21.15.76:6832/153465374 2023-11-09T03:54:25.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.534+0000 7f798a56a740 1 --2- 172.21.15.76:0/3825695204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555a552ff040 0x555a553589c0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.534+0000 7f798a56a740 1 -- 172.21.15.76:0/3825695204 >> 172.21.15.76:0/3825695204 conn(0x555a552fb6d0 msgr2=0x555a55325e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:25.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.535+0000 7f798a56a740 1 -- 172.21.15.76:0/3825695204 shutdown_connections 2023-11-09T03:54:25.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.535+0000 7f798a56a740 1 -- 172.21.15.76:0/3825695204 wait complete. 2023-11-09T03:54:25.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.535+0000 7f798a56a740 20 client.9201 trim_cache size 0 max 16384 2023-11-09T03:54:25.546 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:54:25.546 INFO:teuthology.orchestra.run.smithi076.stdout: "message": "", 2023-11-09T03:54:25.546 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2023-11-09T03:54:25.546 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:54:25.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.535+0000 7f79879d7640 1 -- 172.21.15.76:0/2940091783 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f796406aea0 msgr2=0x7f796406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:25.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.535+0000 7f79879d7640 1 --2- 172.21.15.76:0/2940091783 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f796406aea0 0x7f796406d360 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f7980071a50 tx=0x7f797c009040 comp rx=0 tx=0).stop 2023-11-09T03:54:25.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.535+0000 7f79879d7640 1 -- 172.21.15.76:0/2940091783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79800ade00 msgr2=0x7f79800b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:25.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.535+0000 7f79879d7640 1 --2- 172.21.15.76:0/2940091783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79800ade00 0x7f79800b5430 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f797800ca80 tx=0x7f797800cf40 comp rx=0 tx=0).stop 2023-11-09T03:54:25.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.535+0000 7f79879d7640 1 -- 172.21.15.76:0/2940091783 shutdown_connections 2023-11-09T03:54:25.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.535+0000 7f79879d7640 1 --2- 172.21.15.76:0/2940091783 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f796406aea0 0x7f796406d360 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.535+0000 7f79879d7640 1 --2- 172.21.15.76:0/2940091783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79800ade00 0x7f79800b5430 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.535+0000 7f79879d7640 1 --2- 172.21.15.76:0/2940091783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f798006d9c0 0x7f79800a9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.535+0000 7f79879d7640 1 --2- 172.21.15.76:0/2940091783 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f798006c5f0 0x7f798015cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.535+0000 7f79879d7640 1 -- 172.21.15.76:0/2940091783 >> 172.21.15.76:0/2940091783 conn(0x7f79800cb530 msgr2=0x7f79800ccec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:25.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.535+0000 7f79879d7640 1 -- 172.21.15.76:0/2940091783 shutdown_connections 2023-11-09T03:54:25.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.535+0000 7f79879d7640 1 -- 172.21.15.76:0/2940091783 wait complete. 2023-11-09T03:54:25.550 INFO:tasks.cephfs.test_snapshots:testing snapclient mds_kill_mdstable_at=4 2023-11-09T03:54:25.550 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:54:25.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.651+0000 7fd07be26640 1 Processor -- start 2023-11-09T03:54:25.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.651+0000 7fd07be26640 1 -- start start 2023-11-09T03:54:25.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.652+0000 7fd07be26640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd074070700 0x7fd074164b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:25.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.652+0000 7fd07be26640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd074071500 con 0x7fd0740b0c40 2023-11-09T03:54:25.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.652+0000 7fd07be26640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd07415dae0 con 0x7fd074159e40 2023-11-09T03:54:25.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.652+0000 7fd07be26640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd074070c60 con 0x7fd074070700 2023-11-09T03:54:25.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.652+0000 7fd0793c1640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd0740b0c40 0x7fd07406fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35368/0 (socket says 172.21.15.76:35368) 2023-11-09T03:54:25.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.652+0000 7fd0793c1640 1 -- 172.21.15.76:0/24030034 learned_addr learned my addr 172.21.15.76:0/24030034 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:54:25.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.652+0000 7fd07a3c3640 1 --2- 172.21.15.76:0/24030034 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd074070700 0x7fd074164b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:25.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.652+0000 7fd078bc0640 1 -- 172.21.15.76:0/24030034 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4287775309 0 0) 0x7fd074071500 con 0x7fd0740b0c40 2023-11-09T03:54:25.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.652+0000 7fd078bc0640 1 -- 172.21.15.76:0/24030034 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd060003520 con 0x7fd0740b0c40 2023-11-09T03:54:25.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.652+0000 7fd07a3c3640 1 -- 172.21.15.76:0/24030034 >> v1:172.21.15.113:6789/0 conn(0x7fd074159e40 legacy=0x7fd0740b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:25.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.652+0000 7fd07a3c3640 1 -- 172.21.15.76:0/24030034 >> v1:172.21.15.76:6789/0 conn(0x7fd0740b0c40 legacy=0x7fd07406fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:25.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.652+0000 7fd07a3c3640 1 -- 172.21.15.76:0/24030034 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd074070f40 con 0x7fd074070700 2023-11-09T03:54:25.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.653+0000 7fd07a3c3640 1 --2- 172.21.15.76:0/24030034 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd074070700 0x7fd074164b60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd0700097d0 tx=0x7fd07002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bb27d4617b51102 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:25.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.655+0000 7fd078bc0640 1 -- 172.21.15.76:0/24030034 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd070031800 con 0x7fd074070700 2023-11-09T03:54:25.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.655+0000 7fd078bc0640 1 -- 172.21.15.76:0/24030034 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd070031960 con 0x7fd074070700 2023-11-09T03:54:25.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.655+0000 7fd078bc0640 1 -- 172.21.15.76:0/24030034 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd070031cf0 con 0x7fd074070700 2023-11-09T03:54:25.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.656+0000 7fd07be26640 1 -- 172.21.15.76:0/24030034 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd074070700 msgr2=0x7fd074164b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:25.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.656+0000 7fd07be26640 1 --2- 172.21.15.76:0/24030034 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd074070700 0x7fd074164b60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd0700097d0 tx=0x7fd07002b8a0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.656+0000 7fd07be26640 1 -- 172.21.15.76:0/24030034 shutdown_connections 2023-11-09T03:54:25.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.656+0000 7fd07be26640 1 --2- 172.21.15.76:0/24030034 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd074070700 0x7fd074164b60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.656+0000 7fd07be26640 1 -- 172.21.15.76:0/24030034 >> 172.21.15.76:0/24030034 conn(0x7fd0740cb530 msgr2=0x7fd0740cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:25.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.656+0000 7fd07be26640 1 -- 172.21.15.76:0/24030034 shutdown_connections 2023-11-09T03:54:25.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.656+0000 7fd07be26640 1 -- 172.21.15.76:0/24030034 wait complete. 2023-11-09T03:54:25.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.657+0000 7fd07be26640 1 Processor -- start 2023-11-09T03:54:25.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.657+0000 7fd07be26640 1 -- start start 2023-11-09T03:54:25.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.658+0000 7fd07be26640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd074070700 0x7fd0741ef750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:25.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.658+0000 7fd07be26640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0740b0c40 0x7fd0740accc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:25.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.658+0000 7fd07be26640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd074159e40 0x7fd0740aea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:25.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.658+0000 7fd07be26640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0741f1850 con 0x7fd0740b0c40 2023-11-09T03:54:25.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.658+0000 7fd07be26640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0741f1aa0 con 0x7fd074070700 2023-11-09T03:54:25.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.658+0000 7fd07be26640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0741f1d80 con 0x7fd074159e40 2023-11-09T03:54:25.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.658+0000 7fd0793c1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0740b0c40 0x7fd0740accc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:25.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.658+0000 7fd0793c1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0740b0c40 0x7fd0740accc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43686/0 (socket says 172.21.15.76:43686) 2023-11-09T03:54:25.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.658+0000 7fd0793c1640 1 -- 172.21.15.76:0/3875995380 learned_addr learned my addr 172.21.15.76:0/3875995380 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:25.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.658+0000 7fd079bc2640 1 --2- 172.21.15.76:0/3875995380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd074070700 0x7fd0741ef750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:25.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.658+0000 7fd07a3c3640 1 --2- 172.21.15.76:0/3875995380 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd074159e40 0x7fd0740aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:25.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.658+0000 7fd0793c1640 1 -- 172.21.15.76:0/3875995380 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd074159e40 msgr2=0x7fd0740aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:25.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.658+0000 7fd0793c1640 1 --2- 172.21.15.76:0/3875995380 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd074159e40 0x7fd0740aea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.658+0000 7fd0793c1640 1 -- 172.21.15.76:0/3875995380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd074070700 msgr2=0x7fd0741ef750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:25.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.658+0000 7fd0793c1640 1 --2- 172.21.15.76:0/3875995380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd074070700 0x7fd0741ef750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.658+0000 7fd0793c1640 1 -- 172.21.15.76:0/3875995380 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd070009400 con 0x7fd0740b0c40 2023-11-09T03:54:25.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.658+0000 7fd07a3c3640 1 --2- 172.21.15.76:0/3875995380 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd074159e40 0x7fd0740aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:54:25.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.658+0000 7fd0793c1640 1 --2- 172.21.15.76:0/3875995380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0740b0c40 0x7fd0740accc0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fd06400c500 tx=0x7fd06400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:25.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.658+0000 7fd06a7fc640 1 -- 172.21.15.76:0/3875995380 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd064005ce0 con 0x7fd0740b0c40 2023-11-09T03:54:25.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.658+0000 7fd06a7fc640 1 -- 172.21.15.76:0/3875995380 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd064005e40 con 0x7fd0740b0c40 2023-11-09T03:54:25.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.658+0000 7fd07be26640 1 -- 172.21.15.76:0/3875995380 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0741f2060 con 0x7fd0740b0c40 2023-11-09T03:54:25.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.658+0000 7fd06a7fc640 1 -- 172.21.15.76:0/3875995380 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd064007330 con 0x7fd0740b0c40 2023-11-09T03:54:25.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.658+0000 7fd07be26640 1 -- 172.21.15.76:0/3875995380 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0741f2620 con 0x7fd0740b0c40 2023-11-09T03:54:25.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.659+0000 7fd07be26640 1 -- 172.21.15.76:0/3875995380 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0740684b0 con 0x7fd0740b0c40 2023-11-09T03:54:25.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.660+0000 7fd06a7fc640 1 -- 172.21.15.76:0/3875995380 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd064007550 con 0x7fd0740b0c40 2023-11-09T03:54:25.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.660+0000 7fd06a7fc640 1 --2- 172.21.15.76:0/3875995380 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd05806ae70 0x7fd05806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:25.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.660+0000 7fd06a7fc640 1 -- 172.21.15.76:0/3875995380 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7fd064089660 con 0x7fd0740b0c40 2023-11-09T03:54:25.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.660+0000 7fd079bc2640 1 --2- 172.21.15.76:0/3875995380 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd05806ae70 0x7fd05806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:25.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.661+0000 7fd079bc2640 1 --2- 172.21.15.76:0/3875995380 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd05806ae70 0x7fd05806d330 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0012e0 tx=0x7fd06c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:25.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.664+0000 7fd06a7fc640 1 -- 172.21.15.76:0/3875995380 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd064053e20 con 0x7fd0740b0c40 2023-11-09T03:54:25.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.863+0000 7fd07be26640 1 -- 172.21.15.76:0/3875995380 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0740c9750 con 0x7fd0740b0c40 2023-11-09T03:54:25.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.864+0000 7fd06a7fc640 1 -- 172.21.15.76:0/3875995380 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4935 (secure 0 0 0) 0x7fd064057440 con 0x7fd0740b0c40 2023-11-09T03:54:25.864 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:54:25.864 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":125,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":9024,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:16.621433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":6470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6470":{"gid":6470,"name":"b","rank":1,"incarnation":121,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:54:25.864 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 125 2023-11-09T03:54:25.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.866+0000 7fd07be26640 1 -- 172.21.15.76:0/3875995380 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd05806ae70 msgr2=0x7fd05806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:25.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.866+0000 7fd07be26640 1 --2- 172.21.15.76:0/3875995380 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd05806ae70 0x7fd05806d330 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0012e0 tx=0x7fd06c009040 comp rx=0 tx=0).stop 2023-11-09T03:54:25.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.866+0000 7fd07be26640 1 -- 172.21.15.76:0/3875995380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0740b0c40 msgr2=0x7fd0740accc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:25.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.866+0000 7fd07be26640 1 --2- 172.21.15.76:0/3875995380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0740b0c40 0x7fd0740accc0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fd06400c500 tx=0x7fd06400c9c0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.866+0000 7fd07be26640 1 -- 172.21.15.76:0/3875995380 shutdown_connections 2023-11-09T03:54:25.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.866+0000 7fd07be26640 1 --2- 172.21.15.76:0/3875995380 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd05806ae70 0x7fd05806d330 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.866+0000 7fd07be26640 1 --2- 172.21.15.76:0/3875995380 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd074159e40 0x7fd0740aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.866+0000 7fd07be26640 1 --2- 172.21.15.76:0/3875995380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0740b0c40 0x7fd0740accc0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.866+0000 7fd07be26640 1 --2- 172.21.15.76:0/3875995380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd074070700 0x7fd0741ef750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:25.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.866+0000 7fd07be26640 1 -- 172.21.15.76:0/3875995380 >> 172.21.15.76:0/3875995380 conn(0x7fd0740cb530 msgr2=0x7fd0740cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:25.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.866+0000 7fd07be26640 1 -- 172.21.15.76:0/3875995380 shutdown_connections 2023-11-09T03:54:25.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:25.866+0000 7fd07be26640 1 -- 172.21.15.76:0/3875995380 wait complete. 2023-11-09T03:54:25.876 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2023-11-09T03:54:25.990 INFO:teuthology.orchestra.run.smithi076.stdout:{"last_snap":15,"last_created":12,"last_destroyed":14,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-09T03:54:25.991 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 12, "last_destroyed": 14, "last_snap": 15, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [] } 2023-11-09T03:54:25.991 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:54:26.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.093+0000 7f7bd6f1d640 1 Processor -- start 2023-11-09T03:54:26.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.093+0000 7f7bd6f1d640 1 -- start start 2023-11-09T03:54:26.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.094+0000 7f7bd6f1d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bd00b0c40 0x7f7bd006fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:26.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.094+0000 7f7bd6f1d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bd006bb60 0x7f7bd0070530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:26.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.094+0000 7f7bd6f1d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd0071500 con 0x7f7bd0159e40 2023-11-09T03:54:26.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.094+0000 7f7bd6f1d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0070a70 con 0x7f7bd006bb60 2023-11-09T03:54:26.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.094+0000 7f7bd6f1d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd015dae0 con 0x7f7bd00b0c40 2023-11-09T03:54:26.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.094+0000 7f7bd4cb9640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f7bd0159e40 0x7f7bd00b0530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35370/0 (socket says 172.21.15.76:35370) 2023-11-09T03:54:26.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.094+0000 7f7bd4cb9640 1 -- 172.21.15.76:0/3500675411 learned_addr learned my addr 172.21.15.76:0/3500675411 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:54:26.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.094+0000 7f7bd54ba640 1 --2- 172.21.15.76:0/3500675411 >> v2:172.21.15.113:3300/0 conn(0x7f7bd006bb60 0x7f7bd0070530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:26.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.094+0000 7f7bcffff640 1 --2- 172.21.15.76:0/3500675411 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bd00b0c40 0x7f7bd006fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:26.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.094+0000 7f7bcf7fe640 1 -- 172.21.15.76:0/3500675411 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3113534194 0 0) 0x7f7bd0071500 con 0x7f7bd0159e40 2023-11-09T03:54:26.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.094+0000 7f7bcf7fe640 1 -- 172.21.15.76:0/3500675411 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bb8003520 con 0x7f7bd0159e40 2023-11-09T03:54:26.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.095+0000 7f7bcf7fe640 1 -- 172.21.15.76:0/3500675411 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3556237399 0 0) 0x7f7bb8003520 con 0x7f7bd0159e40 2023-11-09T03:54:26.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.095+0000 7f7bcf7fe640 1 -- 172.21.15.76:0/3500675411 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bd00b0c40 msgr2=0x7f7bd006fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:26.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.095+0000 7f7bcf7fe640 1 --2- 172.21.15.76:0/3500675411 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bd00b0c40 0x7f7bd006fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:26.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.095+0000 7f7bcf7fe640 1 -- 172.21.15.76:0/3500675411 >> v2:172.21.15.113:3300/0 conn(0x7f7bd006bb60 msgr2=0x7f7bd0070530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:26.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.095+0000 7f7bcf7fe640 1 --2- 172.21.15.76:0/3500675411 >> v2:172.21.15.113:3300/0 conn(0x7f7bd006bb60 0x7f7bd0070530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:26.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.095+0000 7f7bcf7fe640 1 -- 172.21.15.76:0/3500675411 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd0070c60 con 0x7f7bd0159e40 2023-11-09T03:54:26.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.095+0000 7f7bcf7fe640 1 -- 172.21.15.76:0/3500675411 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f7bc0001670 con 0x7f7bd0159e40 2023-11-09T03:54:26.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.095+0000 7f7bcf7fe640 1 -- 172.21.15.76:0/3500675411 >> v1:172.21.15.76:6789/0 conn(0x7f7bd0159e40 legacy=0x7f7bd00b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:26.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.095+0000 7f7bcf7fe640 1 --2- 172.21.15.76:0/3500675411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7bb8003970 0x7f7bb8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:26.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.095+0000 7f7bcf7fe640 1 --2- 172.21.15.76:0/3500675411 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bd00b0c40 0x7f7bb80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:26.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.095+0000 7f7bcf7fe640 1 --2- 172.21.15.76:0/3500675411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb8004d00 0x7f7bb8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:26.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.095+0000 7f7bcf7fe640 1 -- 172.21.15.76:0/3500675411 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd015dae0 con 0x7f7bb8003970 2023-11-09T03:54:26.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.095+0000 7f7bcf7fe640 1 -- 172.21.15.76:0/3500675411 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb800b980 con 0x7f7bb8004d00 2023-11-09T03:54:26.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.095+0000 7f7bcf7fe640 1 -- 172.21.15.76:0/3500675411 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb800bac0 con 0x7f7bd00b0c40 2023-11-09T03:54:26.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.095+0000 7f7bd54ba640 1 --2- 172.21.15.76:0/3500675411 >> v2:172.21.15.113:3300/0 conn(0x7f7bd006bb60 0x7f7bd0070530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:54:26.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.095+0000 7f7bd4cb9640 1 --2- 172.21.15.76:0/3500675411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7bb8003970 0x7f7bb8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:26.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.095+0000 7f7bcffff640 1 --2- 172.21.15.76:0/3500675411 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bd00b0c40 0x7f7bb80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:26.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.095+0000 7f7bd54ba640 1 --2- 172.21.15.76:0/3500675411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb8004d00 0x7f7bb8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:26.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.095+0000 7f7bd4cb9640 1 -- 172.21.15.76:0/3500675411 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bd00b0c40 msgr2=0x7f7bb80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:26.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.095+0000 7f7bd4cb9640 1 --2- 172.21.15.76:0/3500675411 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bd00b0c40 0x7f7bb80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:26.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.095+0000 7f7bd4cb9640 1 -- 172.21.15.76:0/3500675411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb8004d00 msgr2=0x7f7bb8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:26.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.095+0000 7f7bd4cb9640 1 --2- 172.21.15.76:0/3500675411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb8004d00 0x7f7bb8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:26.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.095+0000 7f7bd4cb9640 1 -- 172.21.15.76:0/3500675411 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb8003520 con 0x7f7bb8003970 2023-11-09T03:54:26.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.095+0000 7f7bd54ba640 1 --2- 172.21.15.76:0/3500675411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb8004d00 0x7f7bb8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:54:26.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.096+0000 7f7bd4cb9640 1 --2- 172.21.15.76:0/3500675411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7bb8003970 0x7f7bb8003d90 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f7bc000c210 tx=0x7f7bb80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56498f239583e9de server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:26.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.096+0000 7f7bcf7fe640 1 -- 172.21.15.76:0/3500675411 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc002cb00 con 0x7f7bb8003970 2023-11-09T03:54:26.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.096+0000 7f7bcf7fe640 1 -- 172.21.15.76:0/3500675411 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc002cc80 con 0x7f7bb8003970 2023-11-09T03:54:26.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.096+0000 7f7bcf7fe640 1 -- 172.21.15.76:0/3500675411 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc00058d0 con 0x7f7bb8003970 2023-11-09T03:54:26.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.096+0000 7f7bd6f1d640 1 -- 172.21.15.76:0/3500675411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7bb8003970 msgr2=0x7f7bb8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:26.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.096+0000 7f7bd6f1d640 1 --2- 172.21.15.76:0/3500675411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7bb8003970 0x7f7bb8003d90 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f7bc000c210 tx=0x7f7bb80057f0 comp rx=0 tx=0).stop 2023-11-09T03:54:26.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.096+0000 7f7bd54ba640 1 -- 172.21.15.76:0/3500675411 reap_dead start 2023-11-09T03:54:26.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.096+0000 7f7bd6f1d640 1 -- 172.21.15.76:0/3500675411 shutdown_connections 2023-11-09T03:54:26.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.096+0000 7f7bd6f1d640 1 -- 172.21.15.76:0/3500675411 >> 172.21.15.76:0/3500675411 conn(0x7f7bd00cb530 msgr2=0x7f7bd00cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:26.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.097+0000 7f7bd6f1d640 1 -- 172.21.15.76:0/3500675411 shutdown_connections 2023-11-09T03:54:26.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.097+0000 7f7bd6f1d640 1 -- 172.21.15.76:0/3500675411 wait complete. 2023-11-09T03:54:26.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.098+0000 7f7bd6f1d640 1 Processor -- start 2023-11-09T03:54:26.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.098+0000 7f7bd6f1d640 1 -- start start 2023-11-09T03:54:26.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.099+0000 7f7bd6f1d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb8003970 0x7f7bd0155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:26.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.099+0000 7f7bd6f1d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7bd0152250 0x7f7bd01f0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:26.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.099+0000 7f7bd6f1d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bd0151280 0x7f7bd015c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:26.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.099+0000 7f7bd6f1d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd01557d0 con 0x7f7bd0152250 2023-11-09T03:54:26.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.099+0000 7f7bd6f1d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd01f12c0 con 0x7f7bb8003970 2023-11-09T03:54:26.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.099+0000 7f7bd6f1d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd01f15a0 con 0x7f7bd0151280 2023-11-09T03:54:26.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.099+0000 7f7bcffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7bd0152250 0x7f7bd01f0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:26.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.099+0000 7f7bcffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7bd0152250 0x7f7bd01f0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43696/0 (socket says 172.21.15.76:43696) 2023-11-09T03:54:26.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.099+0000 7f7bcffff640 1 -- 172.21.15.76:0/3879352069 learned_addr learned my addr 172.21.15.76:0/3879352069 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:26.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.099+0000 7f7bd4cb9640 1 --2- 172.21.15.76:0/3879352069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb8003970 0x7f7bd0155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:26.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.099+0000 7f7bd54ba640 1 --2- 172.21.15.76:0/3879352069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bd0151280 0x7f7bd015c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:26.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.099+0000 7f7bcffff640 1 -- 172.21.15.76:0/3879352069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bd0151280 msgr2=0x7f7bd015c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:26.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.099+0000 7f7bcffff640 1 --2- 172.21.15.76:0/3879352069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bd0151280 0x7f7bd015c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:26.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.099+0000 7f7bcffff640 1 -- 172.21.15.76:0/3879352069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb8003970 msgr2=0x7f7bd0155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:26.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.099+0000 7f7bcffff640 1 --2- 172.21.15.76:0/3879352069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb8003970 0x7f7bd0155290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:26.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.099+0000 7f7bcffff640 1 -- 172.21.15.76:0/3879352069 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd0150a80 con 0x7f7bd0152250 2023-11-09T03:54:26.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.099+0000 7f7bd4cb9640 1 --2- 172.21.15.76:0/3879352069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb8003970 0x7f7bd0155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:54:26.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.099+0000 7f7bcffff640 1 --2- 172.21.15.76:0/3879352069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7bd0152250 0x7f7bd01f0a00 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f7bc40119a0 tx=0x7f7bc4011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:26.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.099+0000 7f7bcd7fa640 1 -- 172.21.15.76:0/3879352069 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc400fa70 con 0x7f7bd0152250 2023-11-09T03:54:26.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.099+0000 7f7bcd7fa640 1 -- 172.21.15.76:0/3879352069 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc400fbd0 con 0x7f7bd0152250 2023-11-09T03:54:26.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.099+0000 7f7bd6f1d640 1 -- 172.21.15.76:0/3879352069 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd0150d60 con 0x7f7bd0152250 2023-11-09T03:54:26.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.099+0000 7f7bcd7fa640 1 -- 172.21.15.76:0/3879352069 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4014440 con 0x7f7bd0152250 2023-11-09T03:54:26.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.099+0000 7f7bd6f1d640 1 -- 172.21.15.76:0/3879352069 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd01f0f40 con 0x7f7bd0152250 2023-11-09T03:54:26.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.100+0000 7f7bd6f1d640 1 -- 172.21.15.76:0/3879352069 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bd00684b0 con 0x7f7bd0152250 2023-11-09T03:54:26.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.101+0000 7f7bcd7fa640 1 -- 172.21.15.76:0/3879352069 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7bc40145a0 con 0x7f7bd0152250 2023-11-09T03:54:26.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.101+0000 7f7bcd7fa640 1 --2- 172.21.15.76:0/3879352069 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7bbc06aea0 0x7f7bbc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:26.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.101+0000 7f7bcd7fa640 1 -- 172.21.15.76:0/3879352069 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7f7bc408a8f0 con 0x7f7bd0152250 2023-11-09T03:54:26.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.101+0000 7f7bd4cb9640 1 --2- 172.21.15.76:0/3879352069 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7bbc06aea0 0x7f7bbc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:26.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.102+0000 7f7bd4cb9640 1 --2- 172.21.15.76:0/3879352069 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7bbc06aea0 0x7f7bbc06d360 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0034000 tx=0x7f7bc0009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:26.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.106+0000 7f7bcd7fa640 1 -- 172.21.15.76:0/3879352069 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7bc4054f30 con 0x7f7bd0152250 2023-11-09T03:54:26.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.305+0000 7f7bd6f1d640 1 -- 172.21.15.76:0/3879352069 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7bd00c9750 con 0x7f7bd0152250 2023-11-09T03:54:26.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.306+0000 7f7bcd7fa640 1 -- 172.21.15.76:0/3879352069 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4935 (secure 0 0 0) 0x7f7bc4058550 con 0x7f7bd0152250 2023-11-09T03:54:26.306 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:54:26.306 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":125,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":9024,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:16.621433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":6470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6470":{"gid":6470,"name":"b","rank":1,"incarnation":121,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:54:26.306 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 125 2023-11-09T03:54:26.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.308+0000 7f7bd6f1d640 1 -- 172.21.15.76:0/3879352069 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7bbc06aea0 msgr2=0x7f7bbc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:26.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.308+0000 7f7bd6f1d640 1 --2- 172.21.15.76:0/3879352069 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7bbc06aea0 0x7f7bbc06d360 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0034000 tx=0x7f7bc0009030 comp rx=0 tx=0).stop 2023-11-09T03:54:26.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.308+0000 7f7bd6f1d640 1 -- 172.21.15.76:0/3879352069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7bd0152250 msgr2=0x7f7bd01f0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:26.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.308+0000 7f7bd6f1d640 1 --2- 172.21.15.76:0/3879352069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7bd0152250 0x7f7bd01f0a00 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f7bc40119a0 tx=0x7f7bc4011e60 comp rx=0 tx=0).stop 2023-11-09T03:54:26.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.308+0000 7f7bd6f1d640 1 -- 172.21.15.76:0/3879352069 shutdown_connections 2023-11-09T03:54:26.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.308+0000 7f7bd6f1d640 1 --2- 172.21.15.76:0/3879352069 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f7bbc06aea0 0x7f7bbc06d360 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:26.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.308+0000 7f7bd6f1d640 1 --2- 172.21.15.76:0/3879352069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bd0151280 0x7f7bd015c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:26.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.308+0000 7f7bd6f1d640 1 --2- 172.21.15.76:0/3879352069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7bd0152250 0x7f7bd01f0a00 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:26.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.308+0000 7f7bd6f1d640 1 --2- 172.21.15.76:0/3879352069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb8003970 0x7f7bd0155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:26.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.308+0000 7f7bd6f1d640 1 -- 172.21.15.76:0/3879352069 >> 172.21.15.76:0/3879352069 conn(0x7f7bd00cb530 msgr2=0x7f7bd0158f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:26.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.308+0000 7f7bd6f1d640 1 -- 172.21.15.76:0/3879352069 shutdown_connections 2023-11-09T03:54:26.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.308+0000 7f7bd6f1d640 1 -- 172.21.15.76:0/3879352069 wait complete. 2023-11-09T03:54:26.318 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:1 true 2023-11-09T03:54:26.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.419+0000 7f8609a0e640 1 Processor -- start 2023-11-09T03:54:26.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.419+0000 7f8609a0e640 1 -- start start 2023-11-09T03:54:26.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.420+0000 7f8609a0e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8604159e40 0x7f86040b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:26.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.420+0000 7f8609a0e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f860406bb60 0x7f86040b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:26.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.420+0000 7f8609a0e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f860406fff0 0x7f8604070460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:26.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.420+0000 7f8609a0e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8604071500 con 0x7f8604159e40 2023-11-09T03:54:26.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.420+0000 7f8609a0e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f860415dae0 con 0x7f860406bb60 2023-11-09T03:54:26.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.420+0000 7f8609a0e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86040709a0 con 0x7f860406fff0 2023-11-09T03:54:26.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.420+0000 7f8602ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8604159e40 0x7f86040b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:26.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.420+0000 7f8602ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8604159e40 0x7f86040b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43708/0 (socket says 172.21.15.76:43708) 2023-11-09T03:54:26.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.420+0000 7f8602ffd640 1 -- 172.21.15.76:0/1477962182 learned_addr learned my addr 172.21.15.76:0/1477962182 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:26.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.420+0000 7f86037fe640 1 --2- 172.21.15.76:0/1477962182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f860406fff0 0x7f8604070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:26.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.420+0000 7f86027fc640 1 --2- 172.21.15.76:0/1477962182 >> v2:172.21.15.113:3300/0 conn(0x7f860406bb60 0x7f86040b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:26.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.420+0000 7f8602ffd640 1 -- 172.21.15.76:0/1477962182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f860406fff0 msgr2=0x7f8604070460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:26.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.420+0000 7f8602ffd640 1 --2- 172.21.15.76:0/1477962182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f860406fff0 0x7f8604070460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:26.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.420+0000 7f8602ffd640 1 -- 172.21.15.76:0/1477962182 >> v2:172.21.15.113:3300/0 conn(0x7f860406bb60 msgr2=0x7f86040b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:26.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.420+0000 7f8602ffd640 1 --2- 172.21.15.76:0/1477962182 >> v2:172.21.15.113:3300/0 conn(0x7f860406bb60 0x7f86040b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:26.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.420+0000 7f8602ffd640 1 -- 172.21.15.76:0/1477962182 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8604070c60 con 0x7f8604159e40 2023-11-09T03:54:26.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.421+0000 7f8602ffd640 1 --2- 172.21.15.76:0/1477962182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8604159e40 0x7f86040b0530 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f860406fdc0 tx=0x7f85f402e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c76ec25f53a109c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:26.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.421+0000 7f8601ffb640 1 -- 172.21.15.76:0/1477962182 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f402ea00 con 0x7f8604159e40 2023-11-09T03:54:26.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.421+0000 7f8601ffb640 1 -- 172.21.15.76:0/1477962182 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f402eb60 con 0x7f8604159e40 2023-11-09T03:54:26.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.421+0000 7f8601ffb640 1 -- 172.21.15.76:0/1477962182 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f4009600 con 0x7f8604159e40 2023-11-09T03:54:26.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.422+0000 7f8609a0e640 1 -- 172.21.15.76:0/1477962182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8604159e40 msgr2=0x7f86040b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:26.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.422+0000 7f8609a0e640 1 --2- 172.21.15.76:0/1477962182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8604159e40 0x7f86040b0530 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f860406fdc0 tx=0x7f85f402e060 comp rx=0 tx=0).stop 2023-11-09T03:54:26.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.422+0000 7f8609a0e640 1 -- 172.21.15.76:0/1477962182 shutdown_connections 2023-11-09T03:54:26.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.422+0000 7f8609a0e640 1 --2- 172.21.15.76:0/1477962182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f860406fff0 0x7f8604070460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:26.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.422+0000 7f8609a0e640 1 --2- 172.21.15.76:0/1477962182 >> v2:172.21.15.113:3300/0 conn(0x7f860406bb60 0x7f86040b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:26.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.422+0000 7f8609a0e640 1 --2- 172.21.15.76:0/1477962182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8604159e40 0x7f86040b0530 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:26.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.422+0000 7f8609a0e640 1 -- 172.21.15.76:0/1477962182 >> 172.21.15.76:0/1477962182 conn(0x7f86040cb530 msgr2=0x7f86040cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:26.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.422+0000 7f8609a0e640 1 -- 172.21.15.76:0/1477962182 shutdown_connections 2023-11-09T03:54:26.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.422+0000 7f8609a0e640 1 -- 172.21.15.76:0/1477962182 wait complete. 2023-11-09T03:54:26.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.423+0000 7f8609a0e640 1 Processor -- start 2023-11-09T03:54:26.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.423+0000 7f8609a0e640 1 -- start start 2023-11-09T03:54:26.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.424+0000 7f8609a0e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f860406bb60 0x7f86040c4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:26.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.424+0000 7f8609a0e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f860406fff0 0x7f86041f3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:26.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.424+0000 7f8609a0e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8604159e40 0x7f86041f5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:26.430 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.424+0000 7f8609a0e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86041f39c0 con 0x7f860406bb60 2023-11-09T03:54:26.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.424+0000 7f8609a0e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86041f2ff0 con 0x7f860406fff0 2023-11-09T03:54:26.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.424+0000 7f8609a0e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86041f6a20 con 0x7f8604159e40 2023-11-09T03:54:26.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.424+0000 7f8602ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f860406bb60 0x7f86040c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:26.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.424+0000 7f8602ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f860406bb60 0x7f86040c4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43712/0 (socket says 172.21.15.76:43712) 2023-11-09T03:54:26.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.424+0000 7f8602ffd640 1 -- 172.21.15.76:0/3236683566 learned_addr learned my addr 172.21.15.76:0/3236683566 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:26.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.424+0000 7f8602ffd640 1 -- 172.21.15.76:0/3236683566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8604159e40 msgr2=0x7f86041f5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:26.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.424+0000 7f8602ffd640 1 --2- 172.21.15.76:0/3236683566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8604159e40 0x7f86041f5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:26.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.424+0000 7f8602ffd640 1 -- 172.21.15.76:0/3236683566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f860406fff0 msgr2=0x7f86041f3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:26.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.424+0000 7f8602ffd640 1 --2- 172.21.15.76:0/3236683566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f860406fff0 0x7f86041f3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:26.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.424+0000 7f8602ffd640 1 -- 172.21.15.76:0/3236683566 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f400bd40 con 0x7f860406bb60 2023-11-09T03:54:26.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.424+0000 7f8602ffd640 1 --2- 172.21.15.76:0/3236683566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f860406bb60 0x7f86040c4310 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f85f4038040 tx=0x7f85f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:26.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.424+0000 7f85ebfff640 1 -- 172.21.15.76:0/3236683566 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f402ea00 con 0x7f860406bb60 2023-11-09T03:54:26.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.424+0000 7f85ebfff640 1 -- 172.21.15.76:0/3236683566 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f4036820 con 0x7f860406bb60 2023-11-09T03:54:26.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.424+0000 7f8609a0e640 1 -- 172.21.15.76:0/3236683566 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86041f6d00 con 0x7f860406bb60 2023-11-09T03:54:26.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.424+0000 7f85ebfff640 1 -- 172.21.15.76:0/3236683566 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f4036bd0 con 0x7f860406bb60 2023-11-09T03:54:26.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.425+0000 7f8609a0e640 1 -- 172.21.15.76:0/3236683566 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86041f2970 con 0x7f860406bb60 2023-11-09T03:54:26.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.425+0000 7f8609a0e640 1 -- 172.21.15.76:0/3236683566 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86040684b0 con 0x7f860406bb60 2023-11-09T03:54:26.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.426+0000 7f85ebfff640 1 -- 172.21.15.76:0/3236683566 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f85f4036d30 con 0x7f860406bb60 2023-11-09T03:54:26.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.426+0000 7f85ebfff640 1 --2- 172.21.15.76:0/3236683566 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f85f006ae70 0x7f85f006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:26.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.426+0000 7f85ebfff640 1 -- 172.21.15.76:0/3236683566 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7f85f40ad3c0 con 0x7f860406bb60 2023-11-09T03:54:26.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.426+0000 7f86027fc640 1 --2- 172.21.15.76:0/3236683566 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f85f006ae70 0x7f85f006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:26.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.427+0000 7f86027fc640 1 --2- 172.21.15.76:0/3236683566 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f85f006ae70 0x7f85f006d330 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f86040c23c0 tx=0x7f85f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:26.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.430+0000 7f85ebfff640 1 -- 172.21.15.76:0/3236683566 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f85f4077b30 con 0x7f860406bb60 2023-11-09T03:54:26.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:26.667+0000 7f8609a0e640 1 -- 172.21.15.76:0/3236683566 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"} v 0) v1 -- 0x7f86040c9750 con 0x7f860406bb60 2023-11-09T03:54:27.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:27.664+0000 7f85ebfff640 1 -- 172.21.15.76:0/3236683566 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"}]=0 freezing mds.6470 v126) v1 ==== 112+0+0 (secure 0 0 0) 0x7f85f400b720 con 0x7f860406bb60 2023-11-09T03:54:27.664 INFO:teuthology.orchestra.run.smithi076.stderr:freezing mds.6470 2023-11-09T03:54:27.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:27.667+0000 7f8609a0e640 1 -- 172.21.15.76:0/3236683566 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f85f006ae70 msgr2=0x7f85f006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:27.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:27.667+0000 7f8609a0e640 1 --2- 172.21.15.76:0/3236683566 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f85f006ae70 0x7f85f006d330 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f86040c23c0 tx=0x7f85f8008040 comp rx=0 tx=0).stop 2023-11-09T03:54:27.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:27.667+0000 7f8609a0e640 1 -- 172.21.15.76:0/3236683566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f860406bb60 msgr2=0x7f86040c4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:27.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:27.667+0000 7f8609a0e640 1 --2- 172.21.15.76:0/3236683566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f860406bb60 0x7f86040c4310 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f85f4038040 tx=0x7f85f4004a40 comp rx=0 tx=0).stop 2023-11-09T03:54:27.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:27.667+0000 7f8609a0e640 1 -- 172.21.15.76:0/3236683566 shutdown_connections 2023-11-09T03:54:27.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:27.667+0000 7f8609a0e640 1 --2- 172.21.15.76:0/3236683566 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f85f006ae70 0x7f85f006d330 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:27.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:27.667+0000 7f8609a0e640 1 --2- 172.21.15.76:0/3236683566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8604159e40 0x7f86041f5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:27.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:27.667+0000 7f8609a0e640 1 --2- 172.21.15.76:0/3236683566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f860406fff0 0x7f86041f3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:27.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:27.667+0000 7f8609a0e640 1 --2- 172.21.15.76:0/3236683566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f860406bb60 0x7f86040c4310 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:27.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:27.667+0000 7f8609a0e640 1 -- 172.21.15.76:0/3236683566 >> 172.21.15.76:0/3236683566 conn(0x7f86040cb530 msgr2=0x7f86040cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:27.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:27.667+0000 7f8609a0e640 1 -- 172.21.15.76:0/3236683566 shutdown_connections 2023-11-09T03:54:27.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:27.667+0000 7f8609a0e640 1 -- 172.21.15.76:0/3236683566 wait complete. 2023-11-09T03:54:27.680 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_kill_mdstable_at 4 2023-11-09T03:54:27.802 INFO:teuthology.orchestra.run.smithi113.stdout:{"success":"mds_kill_mdstable_at = '4' (not observed, change may require restart) "} 2023-11-09T03:54:27.802 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '4' (not observed, change may require restart) " } 2023-11-09T03:54:27.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:54:27.803 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s34) 2023-11-09T03:54:27.806 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2023-11-09T03:54:27.831 INFO:tasks.ceph.mds.b.smithi113.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::commit(version_t, LogSegment*)' thread 7f5c8b165640 time 2023-11-09T03:54:27.833022+0000 2023-11-09T03:54:27.832 INFO:tasks.ceph.mds.b.smithi113.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: 195: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 4) 2023-11-09T03:54:27.833 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:54:27.833 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f5c96d6efaf] 2023-11-09T03:54:27.833 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f5c96d6f16d] 2023-11-09T03:54:27.833 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: (MDSTableClient::commit(unsigned long, LogSegment*)+0x62c) [0x55a0997097fc] 2023-11-09T03:54:27.833 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: (Server::_mksnap_finish(boost::intrusive_ptr&, CInode*, SnapInfo&)+0xa1) [0x55a09956dcc1] 2023-11-09T03:54:27.834 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (MDSContext::complete(int)+0x5c) [0x55a09973abcc] 2023-11-09T03:54:27.834 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: (MDSIOContextBase::complete(int)+0x31c) [0x55a099743b0c] 2023-11-09T03:54:27.834 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSLogContextBase::complete(int)+0x53) [0x55a099743c63] 2023-11-09T03:54:27.834 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (Finisher::finisher_thread_entry()+0x175) [0x7f5c96e164a5] 2023-11-09T03:54:27.834 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: /lib64/libc.so.6(+0x9f802) [0x7f5c9649f802] 2023-11-09T03:54:27.835 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: /lib64/libc.so.6(+0x3f450) [0x7f5c9643f450] 2023-11-09T03:54:27.835 INFO:tasks.ceph.mds.b.smithi113.stderr:*** Caught signal (Aborted) ** 2023-11-09T03:54:27.835 INFO:tasks.ceph.mds.b.smithi113.stderr: in thread 7f5c8b165640 thread_name:MR_Finisher 2023-11-09T03:54:27.835 INFO:tasks.ceph.mds.b.smithi113.stderr:2023-11-09T03:54:27.832+0000 7f5c8b165640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::commit(version_t, LogSegment*)' thread 7f5c8b165640 time 2023-11-09T03:54:27.833022+0000 2023-11-09T03:54:27.835 INFO:tasks.ceph.mds.b.smithi113.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: 195: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 4) 2023-11-09T03:54:27.836 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:54:27.836 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:54:27.836 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f5c96d6efaf] 2023-11-09T03:54:27.836 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f5c96d6f16d] 2023-11-09T03:54:27.836 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: (MDSTableClient::commit(unsigned long, LogSegment*)+0x62c) [0x55a0997097fc] 2023-11-09T03:54:27.837 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: (Server::_mksnap_finish(boost::intrusive_ptr&, CInode*, SnapInfo&)+0xa1) [0x55a09956dcc1] 2023-11-09T03:54:27.837 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (MDSContext::complete(int)+0x5c) [0x55a09973abcc] 2023-11-09T03:54:27.837 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: (MDSIOContextBase::complete(int)+0x31c) [0x55a099743b0c] 2023-11-09T03:54:27.837 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSLogContextBase::complete(int)+0x53) [0x55a099743c63] 2023-11-09T03:54:27.837 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (Finisher::finisher_thread_entry()+0x175) [0x7f5c96e164a5] 2023-11-09T03:54:27.838 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: /lib64/libc.so.6(+0x9f802) [0x7f5c9649f802] 2023-11-09T03:54:27.838 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: /lib64/libc.so.6(+0x3f450) [0x7f5c9643f450] 2023-11-09T03:54:27.838 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:54:27.838 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:54:27.838 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f5c96454db0] 2023-11-09T03:54:27.838 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f5c964a154c] 2023-11-09T03:54:27.839 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: raise() 2023-11-09T03:54:27.839 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: abort() 2023-11-09T03:54:27.839 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f5c96d6f009] 2023-11-09T03:54:27.839 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f5c96d6f16d] 2023-11-09T03:54:27.839 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSTableClient::commit(unsigned long, LogSegment*)+0x62c) [0x55a0997097fc] 2023-11-09T03:54:27.840 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (Server::_mksnap_finish(boost::intrusive_ptr&, CInode*, SnapInfo&)+0xa1) [0x55a09956dcc1] 2023-11-09T03:54:27.840 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: (MDSContext::complete(int)+0x5c) [0x55a09973abcc] 2023-11-09T03:54:27.840 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: (MDSIOContextBase::complete(int)+0x31c) [0x55a099743b0c] 2023-11-09T03:54:27.840 INFO:tasks.ceph.mds.b.smithi113.stderr: 11: (MDSLogContextBase::complete(int)+0x53) [0x55a099743c63] 2023-11-09T03:54:27.840 INFO:tasks.ceph.mds.b.smithi113.stderr: 12: (Finisher::finisher_thread_entry()+0x175) [0x7f5c96e164a5] 2023-11-09T03:54:27.840 INFO:tasks.ceph.mds.b.smithi113.stderr: 13: /lib64/libc.so.6(+0x9f802) [0x7f5c9649f802] 2023-11-09T03:54:27.841 INFO:tasks.ceph.mds.b.smithi113.stderr: 14: /lib64/libc.so.6(+0x3f450) [0x7f5c9643f450] 2023-11-09T03:54:27.841 INFO:tasks.ceph.mds.b.smithi113.stderr:2023-11-09T03:54:27.832+0000 7f5c8b165640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:54:27.841 INFO:tasks.ceph.mds.b.smithi113.stderr: in thread 7f5c8b165640 thread_name:MR_Finisher 2023-11-09T03:54:27.841 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:54:27.841 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:54:27.842 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f5c96454db0] 2023-11-09T03:54:27.842 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f5c964a154c] 2023-11-09T03:54:27.842 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: raise() 2023-11-09T03:54:27.842 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: abort() 2023-11-09T03:54:27.842 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f5c96d6f009] 2023-11-09T03:54:27.842 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f5c96d6f16d] 2023-11-09T03:54:27.843 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSTableClient::commit(unsigned long, LogSegment*)+0x62c) [0x55a0997097fc] 2023-11-09T03:54:27.843 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (Server::_mksnap_finish(boost::intrusive_ptr&, CInode*, SnapInfo&)+0xa1) [0x55a09956dcc1] 2023-11-09T03:54:27.843 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: (MDSContext::complete(int)+0x5c) [0x55a09973abcc] 2023-11-09T03:54:27.843 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: (MDSIOContextBase::complete(int)+0x31c) [0x55a099743b0c] 2023-11-09T03:54:27.843 INFO:tasks.ceph.mds.b.smithi113.stderr: 11: (MDSLogContextBase::complete(int)+0x53) [0x55a099743c63] 2023-11-09T03:54:27.844 INFO:tasks.ceph.mds.b.smithi113.stderr: 12: (Finisher::finisher_thread_entry()+0x175) [0x7f5c96e164a5] 2023-11-09T03:54:27.844 INFO:tasks.ceph.mds.b.smithi113.stderr: 13: /lib64/libc.so.6(+0x9f802) [0x7f5c9649f802] 2023-11-09T03:54:27.844 INFO:tasks.ceph.mds.b.smithi113.stderr: 14: /lib64/libc.so.6(+0x3f450) [0x7f5c9643f450] 2023-11-09T03:54:27.844 INFO:tasks.ceph.mds.b.smithi113.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:54:27.844 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:54:27.845 INFO:tasks.ceph.mds.b.smithi113.stderr: -1> 2023-11-09T03:54:27.832+0000 7f5c8b165640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::commit(version_t, LogSegment*)' thread 7f5c8b165640 time 2023-11-09T03:54:27.833022+0000 2023-11-09T03:54:27.845 INFO:tasks.ceph.mds.b.smithi113.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: 195: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 4) 2023-11-09T03:54:27.845 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:54:27.845 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:54:27.846 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f5c96d6efaf] 2023-11-09T03:54:27.846 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f5c96d6f16d] 2023-11-09T03:54:27.846 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: (MDSTableClient::commit(unsigned long, LogSegment*)+0x62c) [0x55a0997097fc] 2023-11-09T03:54:27.846 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: (Server::_mksnap_finish(boost::intrusive_ptr&, CInode*, SnapInfo&)+0xa1) [0x55a09956dcc1] 2023-11-09T03:54:27.846 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (MDSContext::complete(int)+0x5c) [0x55a09973abcc] 2023-11-09T03:54:27.847 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: (MDSIOContextBase::complete(int)+0x31c) [0x55a099743b0c] 2023-11-09T03:54:27.847 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSLogContextBase::complete(int)+0x53) [0x55a099743c63] 2023-11-09T03:54:27.847 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (Finisher::finisher_thread_entry()+0x175) [0x7f5c96e164a5] 2023-11-09T03:54:27.847 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: /lib64/libc.so.6(+0x9f802) [0x7f5c9649f802] 2023-11-09T03:54:27.847 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: /lib64/libc.so.6(+0x3f450) [0x7f5c9643f450] 2023-11-09T03:54:27.847 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:54:27.848 INFO:tasks.ceph.mds.b.smithi113.stderr: 0> 2023-11-09T03:54:27.832+0000 7f5c8b165640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:54:27.848 INFO:tasks.ceph.mds.b.smithi113.stderr: in thread 7f5c8b165640 thread_name:MR_Finisher 2023-11-09T03:54:27.848 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:54:27.848 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:54:27.848 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f5c96454db0] 2023-11-09T03:54:27.849 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f5c964a154c] 2023-11-09T03:54:27.849 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: raise() 2023-11-09T03:54:27.849 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: abort() 2023-11-09T03:54:27.849 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f5c96d6f009] 2023-11-09T03:54:27.849 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f5c96d6f16d] 2023-11-09T03:54:27.850 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSTableClient::commit(unsigned long, LogSegment*)+0x62c) [0x55a0997097fc] 2023-11-09T03:54:27.850 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (Server::_mksnap_finish(boost::intrusive_ptr&, CInode*, SnapInfo&)+0xa1) [0x55a09956dcc1] 2023-11-09T03:54:27.850 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: (MDSContext::complete(int)+0x5c) [0x55a09973abcc] 2023-11-09T03:54:27.850 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: (MDSIOContextBase::complete(int)+0x31c) [0x55a099743b0c] 2023-11-09T03:54:27.850 INFO:tasks.ceph.mds.b.smithi113.stderr: 11: (MDSLogContextBase::complete(int)+0x53) [0x55a099743c63] 2023-11-09T03:54:27.851 INFO:tasks.ceph.mds.b.smithi113.stderr: 12: (Finisher::finisher_thread_entry()+0x175) [0x7f5c96e164a5] 2023-11-09T03:54:27.851 INFO:tasks.ceph.mds.b.smithi113.stderr: 13: /lib64/libc.so.6(+0x9f802) [0x7f5c9649f802] 2023-11-09T03:54:27.851 INFO:tasks.ceph.mds.b.smithi113.stderr: 14: /lib64/libc.so.6(+0x3f450) [0x7f5c9643f450] 2023-11-09T03:54:27.851 INFO:tasks.ceph.mds.b.smithi113.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:54:27.851 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:54:27.851 INFO:tasks.ceph.mds.b.smithi113.stderr: -1> 2023-11-09T03:54:27.832+0000 7f5c8b165640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::commit(version_t, LogSegment*)' thread 7f5c8b165640 time 2023-11-09T03:54:27.833022+0000 2023-11-09T03:54:27.852 INFO:tasks.ceph.mds.b.smithi113.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: 195: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 4) 2023-11-09T03:54:27.852 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:54:27.852 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:54:27.852 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f5c96d6efaf] 2023-11-09T03:54:27.852 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f5c96d6f16d] 2023-11-09T03:54:27.852 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: (MDSTableClient::commit(unsigned long, LogSegment*)+0x62c) [0x55a0997097fc] 2023-11-09T03:54:27.853 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: (Server::_mksnap_finish(boost::intrusive_ptr&, CInode*, SnapInfo&)+0xa1) [0x55a09956dcc1] 2023-11-09T03:54:27.853 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (MDSContext::complete(int)+0x5c) [0x55a09973abcc] 2023-11-09T03:54:27.853 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: (MDSIOContextBase::complete(int)+0x31c) [0x55a099743b0c] 2023-11-09T03:54:27.853 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSLogContextBase::complete(int)+0x53) [0x55a099743c63] 2023-11-09T03:54:27.853 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (Finisher::finisher_thread_entry()+0x175) [0x7f5c96e164a5] 2023-11-09T03:54:27.853 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: /lib64/libc.so.6(+0x9f802) [0x7f5c9649f802] 2023-11-09T03:54:27.854 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: /lib64/libc.so.6(+0x3f450) [0x7f5c9643f450] 2023-11-09T03:54:27.854 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:54:27.854 INFO:tasks.ceph.mds.b.smithi113.stderr: 0> 2023-11-09T03:54:27.832+0000 7f5c8b165640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:54:27.854 INFO:tasks.ceph.mds.b.smithi113.stderr: in thread 7f5c8b165640 thread_name:MR_Finisher 2023-11-09T03:54:27.854 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:54:27.855 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:54:27.855 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f5c96454db0] 2023-11-09T03:54:27.855 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f5c964a154c] 2023-11-09T03:54:27.855 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: raise() 2023-11-09T03:54:27.855 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: abort() 2023-11-09T03:54:27.855 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f5c96d6f009] 2023-11-09T03:54:27.856 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f5c96d6f16d] 2023-11-09T03:54:27.856 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSTableClient::commit(unsigned long, LogSegment*)+0x62c) [0x55a0997097fc] 2023-11-09T03:54:27.856 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (Server::_mksnap_finish(boost::intrusive_ptr&, CInode*, SnapInfo&)+0xa1) [0x55a09956dcc1] 2023-11-09T03:54:27.856 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: (MDSContext::complete(int)+0x5c) [0x55a09973abcc] 2023-11-09T03:54:27.856 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: (MDSIOContextBase::complete(int)+0x31c) [0x55a099743b0c] 2023-11-09T03:54:27.856 INFO:tasks.ceph.mds.b.smithi113.stderr: 11: (MDSLogContextBase::complete(int)+0x53) [0x55a099743c63] 2023-11-09T03:54:27.857 INFO:tasks.ceph.mds.b.smithi113.stderr: 12: (Finisher::finisher_thread_entry()+0x175) [0x7f5c96e164a5] 2023-11-09T03:54:27.857 INFO:tasks.ceph.mds.b.smithi113.stderr: 13: /lib64/libc.so.6(+0x9f802) [0x7f5c9649f802] 2023-11-09T03:54:27.857 INFO:tasks.ceph.mds.b.smithi113.stderr: 14: /lib64/libc.so.6(+0x3f450) [0x7f5c9643f450] 2023-11-09T03:54:27.857 INFO:tasks.ceph.mds.b.smithi113.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:54:27.857 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:54:27.960 INFO:teuthology.orchestra.run.smithi076.stdout:{"mds_beacon_grace":"15.000000"} 2023-11-09T03:54:27.961 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2023-11-09T03:54:27.961 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:54:28.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.063+0000 7fc2e94ec640 1 Processor -- start 2023-11-09T03:54:28.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.063+0000 7fc2e94ec640 1 -- start start 2023-11-09T03:54:28.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.063+0000 7fc2e94ec640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2e415d4e0 0x7fc2e40d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:28.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.063+0000 7fc2e94ec640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc2e40d3b60 0x7fc2e40d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:28.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.063+0000 7fc2e94ec640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2e40b32b0 con 0x7fc2e415d100 2023-11-09T03:54:28.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.063+0000 7fc2e94ec640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e40cec20 con 0x7fc2e40d3b60 2023-11-09T03:54:28.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.063+0000 7fc2e94ec640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e40cee10 con 0x7fc2e415d4e0 2023-11-09T03:54:28.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.063+0000 7fc2e2ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc2e415d100 0x7fc2e40ce510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35382/0 (socket says 172.21.15.76:35382) 2023-11-09T03:54:28.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.064+0000 7fc2e2ffd640 1 -- 172.21.15.76:0/3509076044 learned_addr learned my addr 172.21.15.76:0/3509076044 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:54:28.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.064+0000 7fc2e37fe640 1 --2- 172.21.15.76:0/3509076044 >> v2:172.21.15.113:3300/0 conn(0x7fc2e40d3b60 0x7fc2e40d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:28.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.064+0000 7fc2e27fc640 1 --2- 172.21.15.76:0/3509076044 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2e415d4e0 0x7fc2e40d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:28.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.064+0000 7fc2e1ffb640 1 -- 172.21.15.76:0/3509076044 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2761083863 0 0) 0x7fc2e40b32b0 con 0x7fc2e415d100 2023-11-09T03:54:28.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.064+0000 7fc2e1ffb640 1 -- 172.21.15.76:0/3509076044 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2cc003520 con 0x7fc2e415d100 2023-11-09T03:54:28.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.064+0000 7fc2e1ffb640 1 -- 172.21.15.76:0/3509076044 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1384217619 0 0) 0x7fc2cc003520 con 0x7fc2e415d100 2023-11-09T03:54:28.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.064+0000 7fc2e37fe640 1 -- 172.21.15.76:0/3509076044 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2e415d4e0 msgr2=0x7fc2e40d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:28.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.064+0000 7fc2e37fe640 1 --2- 172.21.15.76:0/3509076044 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2e415d4e0 0x7fc2e40d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:28.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.064+0000 7fc2e37fe640 1 -- 172.21.15.76:0/3509076044 >> v1:172.21.15.76:6789/0 conn(0x7fc2e415d100 legacy=0x7fc2e40ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:28.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.064+0000 7fc2e37fe640 1 -- 172.21.15.76:0/3509076044 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e40b4750 con 0x7fc2e40d3b60 2023-11-09T03:54:28.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.064+0000 7fc2e27fc640 1 --2- 172.21.15.76:0/3509076044 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2e415d4e0 0x7fc2e40d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:54:28.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.065+0000 7fc2e37fe640 1 --2- 172.21.15.76:0/3509076044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2e40d3b60 0x7fc2e40d1a10 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc0097a0 tx=0x7fc2dc02b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b922bdb04b3854d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:28.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.065+0000 7fc2e1ffb640 1 -- 172.21.15.76:0/3509076044 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4001b70 con 0x7fc2e40d3b60 2023-11-09T03:54:28.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.065+0000 7fc2e1ffb640 1 -- 172.21.15.76:0/3509076044 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2dc02e050 con 0x7fc2e40d3b60 2023-11-09T03:54:28.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.065+0000 7fc2e1ffb640 1 -- 172.21.15.76:0/3509076044 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc030730 con 0x7fc2e40d3b60 2023-11-09T03:54:28.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.065+0000 7fc2e94ec640 1 -- 172.21.15.76:0/3509076044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2e40d3b60 msgr2=0x7fc2e40d1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:28.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.065+0000 7fc2e94ec640 1 --2- 172.21.15.76:0/3509076044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2e40d3b60 0x7fc2e40d1a10 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc0097a0 tx=0x7fc2dc02b790 comp rx=0 tx=0).stop 2023-11-09T03:54:28.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.065+0000 7fc2e94ec640 1 -- 172.21.15.76:0/3509076044 shutdown_connections 2023-11-09T03:54:28.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.065+0000 7fc2e94ec640 1 --2- 172.21.15.76:0/3509076044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2e40d3b60 0x7fc2e40d1a10 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:28.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.065+0000 7fc2e94ec640 1 --2- 172.21.15.76:0/3509076044 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2e415d4e0 0x7fc2e40d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:28.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.065+0000 7fc2e94ec640 1 -- 172.21.15.76:0/3509076044 >> 172.21.15.76:0/3509076044 conn(0x7fc2e40d2ef0 msgr2=0x7fc2e40d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:28.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.066+0000 7fc2e94ec640 1 -- 172.21.15.76:0/3509076044 shutdown_connections 2023-11-09T03:54:28.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.066+0000 7fc2e94ec640 1 -- 172.21.15.76:0/3509076044 wait complete. 2023-11-09T03:54:28.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.067+0000 7fc2e94ec640 1 Processor -- start 2023-11-09T03:54:28.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.067+0000 7fc2e94ec640 1 -- start start 2023-11-09T03:54:28.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.068+0000 7fc2e94ec640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2e40d3b60 0x7fc2e4152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:28.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.068+0000 7fc2e94ec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2e415d100 0x7fc2e4154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:28.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.068+0000 7fc2e94ec640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2e415d4e0 0x7fc2e415eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:28.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.068+0000 7fc2e94ec640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e4152a50 con 0x7fc2e415d100 2023-11-09T03:54:28.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.068+0000 7fc2e94ec640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e4153590 con 0x7fc2e415d4e0 2023-11-09T03:54:28.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.068+0000 7fc2e94ec640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e41559e0 con 0x7fc2e40d3b60 2023-11-09T03:54:28.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.068+0000 7fc2e27fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2e415d100 0x7fc2e4154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:28.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.068+0000 7fc2e27fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2e415d100 0x7fc2e4154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43726/0 (socket says 172.21.15.76:43726) 2023-11-09T03:54:28.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.068+0000 7fc2e27fc640 1 -- 172.21.15.76:0/595796519 learned_addr learned my addr 172.21.15.76:0/595796519 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:28.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.068+0000 7fc2e37fe640 1 --2- 172.21.15.76:0/595796519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2e415d4e0 0x7fc2e415eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:28.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.068+0000 7fc2e2ffd640 1 --2- 172.21.15.76:0/595796519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2e40d3b60 0x7fc2e4152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:28.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.068+0000 7fc2e27fc640 1 -- 172.21.15.76:0/595796519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2e40d3b60 msgr2=0x7fc2e4152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:28.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.068+0000 7fc2e27fc640 1 --2- 172.21.15.76:0/595796519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2e40d3b60 0x7fc2e4152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:28.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.068+0000 7fc2e27fc640 1 -- 172.21.15.76:0/595796519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2e415d4e0 msgr2=0x7fc2e415eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:28.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.068+0000 7fc2e27fc640 1 --2- 172.21.15.76:0/595796519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2e415d4e0 0x7fc2e415eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:28.078 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.068+0000 7fc2e27fc640 1 -- 172.21.15.76:0/595796519 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d8009400 con 0x7fc2e415d100 2023-11-09T03:54:28.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.068+0000 7fc2e27fc640 1 --2- 172.21.15.76:0/595796519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2e415d100 0x7fc2e4154f60 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800e990 tx=0x7fc2d800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:28.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.068+0000 7fc2cb7fe640 1 -- 172.21.15.76:0/595796519 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d800c9e0 con 0x7fc2e415d100 2023-11-09T03:54:28.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.068+0000 7fc2e94ec640 1 -- 172.21.15.76:0/595796519 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2dc009400 con 0x7fc2e415d100 2023-11-09T03:54:28.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.068+0000 7fc2cb7fe640 1 -- 172.21.15.76:0/595796519 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d800cb40 con 0x7fc2e415d100 2023-11-09T03:54:28.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.068+0000 7fc2cb7fe640 1 -- 172.21.15.76:0/595796519 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d800ced0 con 0x7fc2e415d100 2023-11-09T03:54:28.079 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.068+0000 7fc2e94ec640 1 -- 172.21.15.76:0/595796519 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e4153220 con 0x7fc2e415d100 2023-11-09T03:54:28.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.069+0000 7fc2e94ec640 1 -- 172.21.15.76:0/595796519 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2e4152da0 con 0x7fc2e415d100 2023-11-09T03:54:28.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.070+0000 7fc2cb7fe640 1 -- 172.21.15.76:0/595796519 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc2d8013470 con 0x7fc2e415d100 2023-11-09T03:54:28.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.070+0000 7fc2cb7fe640 1 --2- 172.21.15.76:0/595796519 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc2d006ae70 0x7fc2d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:28.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.070+0000 7fc2cb7fe640 1 -- 172.21.15.76:0/595796519 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7fc2d808a2d0 con 0x7fc2e415d100 2023-11-09T03:54:28.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.070+0000 7fc2e2ffd640 1 --2- 172.21.15.76:0/595796519 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc2d006ae70 0x7fc2d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:28.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.071+0000 7fc2e2ffd640 1 --2- 172.21.15.76:0/595796519 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc2d006ae70 0x7fc2d006d330 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4007fd0 tx=0x7fc2d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:28.081 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.074+0000 7fc2cb7fe640 1 -- 172.21.15.76:0/595796519 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc2e4152da0 con 0x7fc2e415d100 2023-11-09T03:54:28.084 INFO:tasks.ceph.mds.b.smithi113.stderr:daemon-helper: command crashed with signal 6 2023-11-09T03:54:28.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.274+0000 7fc2e94ec640 1 -- 172.21.15.76:0/595796519 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2e4152da0 con 0x7fc2e415d100 2023-11-09T03:54:28.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.275+0000 7fc2cb7fe640 1 -- 172.21.15.76:0/595796519 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+4935 (secure 0 0 0) 0x7fc2e4152da0 con 0x7fc2e415d100 2023-11-09T03:54:28.274 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:54:28.275 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":126,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":9024,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:26.668321+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":6470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6470":{"gid":6470,"name":"b","rank":1,"incarnation":121,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:54:28.275 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 126 2023-11-09T03:54:28.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.276+0000 7fc2e94ec640 1 -- 172.21.15.76:0/595796519 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc2d006ae70 msgr2=0x7fc2d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:28.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.276+0000 7fc2e94ec640 1 --2- 172.21.15.76:0/595796519 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc2d006ae70 0x7fc2d006d330 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4007fd0 tx=0x7fc2d4009040 comp rx=0 tx=0).stop 2023-11-09T03:54:28.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.276+0000 7fc2e94ec640 1 -- 172.21.15.76:0/595796519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2e415d100 msgr2=0x7fc2e4154f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:28.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.276+0000 7fc2e94ec640 1 --2- 172.21.15.76:0/595796519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2e415d100 0x7fc2e4154f60 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800e990 tx=0x7fc2d800ee50 comp rx=0 tx=0).stop 2023-11-09T03:54:28.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.276+0000 7fc2e94ec640 1 -- 172.21.15.76:0/595796519 shutdown_connections 2023-11-09T03:54:28.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.276+0000 7fc2e94ec640 1 --2- 172.21.15.76:0/595796519 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fc2d006ae70 0x7fc2d006d330 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:28.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.276+0000 7fc2e94ec640 1 --2- 172.21.15.76:0/595796519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2e415d4e0 0x7fc2e415eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:28.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.276+0000 7fc2e94ec640 1 --2- 172.21.15.76:0/595796519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2e415d100 0x7fc2e4154f60 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:28.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.276+0000 7fc2e94ec640 1 --2- 172.21.15.76:0/595796519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2e40d3b60 0x7fc2e4152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:28.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.276+0000 7fc2e94ec640 1 -- 172.21.15.76:0/595796519 >> 172.21.15.76:0/595796519 conn(0x7fc2e40d2ef0 msgr2=0x7fc2e40b1220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:28.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.276+0000 7fc2e94ec640 1 -- 172.21.15.76:0/595796519 shutdown_connections 2023-11-09T03:54:28.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:28.277+0000 7fc2e94ec640 1 -- 172.21.15.76:0/595796519 wait complete. 2023-11-09T03:54:28.287 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:54:32.093 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-11-09T03:54:32.585 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:54:32.588 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:54:33.288 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:54:33.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.399+0000 7f4af44c2640 1 Processor -- start 2023-11-09T03:54:33.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.399+0000 7f4af44c2640 1 -- start start 2023-11-09T03:54:33.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.400+0000 7f4af44c2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4aec06c220 0x7f4aec0cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:33.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.400+0000 7f4af44c2640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aec0cd720 con 0x7f4aec0addd0 2023-11-09T03:54:33.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.400+0000 7f4af44c2640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aec0ae1b0 con 0x7f4aec0cc680 2023-11-09T03:54:33.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.400+0000 7f4af44c2640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aec0ccb10 con 0x7f4aec06c220 2023-11-09T03:54:33.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.400+0000 7f4af2a5f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4aec06c220 0x7f4aec0cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:33.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.400+0000 7f4af225e640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f4aec0addd0 0x7f4aec06bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:32888/0 (socket says 172.21.15.76:32888) 2023-11-09T03:54:33.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.400+0000 7f4af225e640 1 -- 172.21.15.76:0/2263236615 learned_addr learned my addr 172.21.15.76:0/2263236615 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:54:33.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.400+0000 7f4af2a5f640 1 --2- 172.21.15.76:0/2263236615 >> v2:172.21.15.76:3300/0 conn(0x7f4aec06c220 0x7f4aec0cdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40444/0 (socket says 172.21.15.76:40444) 2023-11-09T03:54:33.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.400+0000 7f4af2a5f640 1 -- 172.21.15.76:0/2263236615 >> v1:172.21.15.113:6789/0 conn(0x7f4aec0cc680 legacy=0x7f4aec06f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:33.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.400+0000 7f4af2a5f640 1 -- 172.21.15.76:0/2263236615 >> v1:172.21.15.76:6789/0 conn(0x7f4aec0addd0 legacy=0x7f4aec06bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:33.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.400+0000 7f4af125c640 1 -- 172.21.15.76:0/2263236615 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2318954957 0 0) 0x7f4aec0cd720 con 0x7f4aec0addd0 2023-11-09T03:54:33.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.400+0000 7f4af2a5f640 1 -- 172.21.15.76:0/2263236615 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aec0ccdf0 con 0x7f4aec06c220 2023-11-09T03:54:33.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.401+0000 7f4af2a5f640 1 --2- 172.21.15.76:0/2263236615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aec06c220 0x7f4aec0cdbb0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4009d90 tx=0x7f4ae402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba30fe3fe4470aad server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:33.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.401+0000 7f4af125c640 1 -- 172.21.15.76:0/2263236615 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae402cd00 con 0x7f4aec06c220 2023-11-09T03:54:33.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.401+0000 7f4af125c640 1 -- 172.21.15.76:0/2263236615 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae402ce60 con 0x7f4aec06c220 2023-11-09T03:54:33.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.401+0000 7f4af125c640 1 -- 172.21.15.76:0/2263236615 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4034660 con 0x7f4aec06c220 2023-11-09T03:54:33.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.402+0000 7f4af44c2640 1 -- 172.21.15.76:0/2263236615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aec06c220 msgr2=0x7f4aec0cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:33.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.402+0000 7f4af44c2640 1 --2- 172.21.15.76:0/2263236615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aec06c220 0x7f4aec0cdbb0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4009d90 tx=0x7f4ae402bf50 comp rx=0 tx=0).stop 2023-11-09T03:54:33.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.402+0000 7f4af44c2640 1 -- 172.21.15.76:0/2263236615 shutdown_connections 2023-11-09T03:54:33.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.402+0000 7f4af44c2640 1 --2- 172.21.15.76:0/2263236615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aec06c220 0x7f4aec0cdbb0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:33.406 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.402+0000 7f4af44c2640 1 -- 172.21.15.76:0/2263236615 >> 172.21.15.76:0/2263236615 conn(0x7f4aec0cb530 msgr2=0x7f4aec0cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:33.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.402+0000 7f4af44c2640 1 -- 172.21.15.76:0/2263236615 shutdown_connections 2023-11-09T03:54:33.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.402+0000 7f4af44c2640 1 -- 172.21.15.76:0/2263236615 wait complete. 2023-11-09T03:54:33.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.403+0000 7f4af44c2640 1 Processor -- start 2023-11-09T03:54:33.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.403+0000 7f4af44c2640 1 -- start start 2023-11-09T03:54:33.407 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.404+0000 7f4af44c2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4aec06c220 0x7f4aec0c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:33.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.404+0000 7f4af44c2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4aec0addd0 0x7f4aec0c4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:33.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.404+0000 7f4af44c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aec0c4e80 0x7f4aec0c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:33.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.404+0000 7f4af44c2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec1557f0 con 0x7f4aec0c4e80 2023-11-09T03:54:33.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.404+0000 7f4af44c2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec15b770 con 0x7f4aec0addd0 2023-11-09T03:54:33.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.404+0000 7f4af44c2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec15b490 con 0x7f4aec06c220 2023-11-09T03:54:33.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.404+0000 7f4af2a5f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aec0c4e80 0x7f4aec0c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:33.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.404+0000 7f4af2a5f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aec0c4e80 0x7f4aec0c31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40458/0 (socket says 172.21.15.76:40458) 2023-11-09T03:54:33.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.404+0000 7f4af2a5f640 1 -- 172.21.15.76:0/2463248449 learned_addr learned my addr 172.21.15.76:0/2463248449 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:33.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.404+0000 7f4af1a5d640 1 --2- 172.21.15.76:0/2463248449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4aec0addd0 0x7f4aec0c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:33.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.404+0000 7f4af225e640 1 --2- 172.21.15.76:0/2463248449 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4aec06c220 0x7f4aec0c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:33.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.404+0000 7f4af2a5f640 1 -- 172.21.15.76:0/2463248449 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4aec06c220 msgr2=0x7f4aec0c6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:33.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.404+0000 7f4af2a5f640 1 --2- 172.21.15.76:0/2463248449 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4aec06c220 0x7f4aec0c6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:33.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.404+0000 7f4af2a5f640 1 -- 172.21.15.76:0/2463248449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4aec0addd0 msgr2=0x7f4aec0c4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:33.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.404+0000 7f4af2a5f640 1 --2- 172.21.15.76:0/2463248449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4aec0addd0 0x7f4aec0c4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:33.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.404+0000 7f4af2a5f640 1 -- 172.21.15.76:0/2463248449 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae4009a70 con 0x7f4aec0c4e80 2023-11-09T03:54:33.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.404+0000 7f4af1a5d640 1 --2- 172.21.15.76:0/2463248449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4aec0addd0 0x7f4aec0c4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:54:33.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.404+0000 7f4af2a5f640 1 --2- 172.21.15.76:0/2463248449 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aec0c4e80 0x7f4aec0c31b0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4009d90 tx=0x7f4ae4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:33.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.405+0000 7f4ae2ffd640 1 -- 172.21.15.76:0/2463248449 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae403d600 con 0x7f4aec0c4e80 2023-11-09T03:54:33.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.405+0000 7f4ae2ffd640 1 -- 172.21.15.76:0/2463248449 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae403d780 con 0x7f4aec0c4e80 2023-11-09T03:54:33.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.405+0000 7f4af44c2640 1 -- 172.21.15.76:0/2463248449 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aec0c26d0 con 0x7f4aec0c4e80 2023-11-09T03:54:33.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.405+0000 7f4ae2ffd640 1 -- 172.21.15.76:0/2463248449 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae403db30 con 0x7f4aec0c4e80 2023-11-09T03:54:33.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.405+0000 7f4af44c2640 1 -- 172.21.15.76:0/2463248449 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aec0c2c90 con 0x7f4aec0c4e80 2023-11-09T03:54:33.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.406+0000 7f4af44c2640 1 -- 172.21.15.76:0/2463248449 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aec0684b0 con 0x7f4aec0c4e80 2023-11-09T03:54:33.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.406+0000 7f4ae2ffd640 1 -- 172.21.15.76:0/2463248449 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4ae403b040 con 0x7f4aec0c4e80 2023-11-09T03:54:33.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.407+0000 7f4ae2ffd640 1 --2- 172.21.15.76:0/2463248449 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4ad006ae70 0x7f4ad006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:33.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.407+0000 7f4ae2ffd640 1 -- 172.21.15.76:0/2463248449 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7f4ae40ac0d0 con 0x7f4aec0c4e80 2023-11-09T03:54:33.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.407+0000 7f4af225e640 1 --2- 172.21.15.76:0/2463248449 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4ad006ae70 0x7f4ad006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:33.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.407+0000 7f4af225e640 1 --2- 172.21.15.76:0/2463248449 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4ad006ae70 0x7f4ad006d330 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8000f60 tx=0x7f4ae8007b90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:33.413 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.411+0000 7f4ae2ffd640 1 -- 172.21.15.76:0/2463248449 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4ae4076830 con 0x7f4aec0c4e80 2023-11-09T03:54:33.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.618+0000 7f4af44c2640 1 -- 172.21.15.76:0/2463248449 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4aec0b4a30 con 0x7f4aec0c4e80 2023-11-09T03:54:33.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.618+0000 7f4ae2ffd640 1 -- 172.21.15.76:0/2463248449 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+4935 (secure 0 0 0) 0x7f4ae4039090 con 0x7f4aec0c4e80 2023-11-09T03:54:33.618 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:54:33.619 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":126,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":9024,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:26.668321+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":6470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6470":{"gid":6470,"name":"b","rank":1,"incarnation":121,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:54:33.619 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 126 2023-11-09T03:54:33.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.620+0000 7f4af44c2640 1 -- 172.21.15.76:0/2463248449 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4ad006ae70 msgr2=0x7f4ad006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:33.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.620+0000 7f4af44c2640 1 --2- 172.21.15.76:0/2463248449 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4ad006ae70 0x7f4ad006d330 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8000f60 tx=0x7f4ae8007b90 comp rx=0 tx=0).stop 2023-11-09T03:54:33.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.620+0000 7f4af44c2640 1 -- 172.21.15.76:0/2463248449 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aec0c4e80 msgr2=0x7f4aec0c31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:33.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.620+0000 7f4af44c2640 1 --2- 172.21.15.76:0/2463248449 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aec0c4e80 0x7f4aec0c31b0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4009d90 tx=0x7f4ae4004a40 comp rx=0 tx=0).stop 2023-11-09T03:54:33.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.620+0000 7f4af44c2640 1 -- 172.21.15.76:0/2463248449 shutdown_connections 2023-11-09T03:54:33.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.620+0000 7f4af44c2640 1 --2- 172.21.15.76:0/2463248449 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4ad006ae70 0x7f4ad006d330 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:33.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.620+0000 7f4af44c2640 1 --2- 172.21.15.76:0/2463248449 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aec0c4e80 0x7f4aec0c31b0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:33.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.620+0000 7f4af44c2640 1 --2- 172.21.15.76:0/2463248449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4aec0addd0 0x7f4aec0c4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:33.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.620+0000 7f4af44c2640 1 --2- 172.21.15.76:0/2463248449 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4aec06c220 0x7f4aec0c6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:33.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.620+0000 7f4af44c2640 1 -- 172.21.15.76:0/2463248449 >> 172.21.15.76:0/2463248449 conn(0x7f4aec0cb530 msgr2=0x7f4aec158c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:33.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.620+0000 7f4af44c2640 1 -- 172.21.15.76:0/2463248449 shutdown_connections 2023-11-09T03:54:33.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:33.620+0000 7f4af44c2640 1 -- 172.21.15.76:0/2463248449 wait complete. 2023-11-09T03:54:33.631 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:54:38.631 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:54:38.702 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~7s 2023-11-09T03:54:38.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.742+0000 7f98ae74c640 1 Processor -- start 2023-11-09T03:54:38.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.742+0000 7f98ae74c640 1 -- start start 2023-11-09T03:54:38.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.743+0000 7f98ae74c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98a8159e40 0x7f98a80b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:38.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.743+0000 7f98ae74c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f98a8070700 0x7f98a80b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:38.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.743+0000 7f98ae74c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a8071500 con 0x7f98a806bb60 2023-11-09T03:54:38.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.743+0000 7f98ae74c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a815dae0 con 0x7f98a8070700 2023-11-09T03:54:38.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.743+0000 7f98ae74c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a8070c60 con 0x7f98a8159e40 2023-11-09T03:54:38.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.743+0000 7f98a77fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f98a806bb60 0x7f98a806fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:32904/0 (socket says 172.21.15.76:32904) 2023-11-09T03:54:38.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.743+0000 7f98a77fe640 1 -- 172.21.15.76:0/3570126588 learned_addr learned my addr 172.21.15.76:0/3570126588 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:54:38.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.744+0000 7f98a7fff640 1 --2- 172.21.15.76:0/3570126588 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98a8159e40 0x7f98a80b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:38.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.744+0000 7f98acce9640 1 --2- 172.21.15.76:0/3570126588 >> v2:172.21.15.113:3300/0 conn(0x7f98a8070700 0x7f98a80b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:38.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.744+0000 7f98a6ffd640 1 -- 172.21.15.76:0/3570126588 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3899777156 0 0) 0x7f98a8071500 con 0x7f98a806bb60 2023-11-09T03:54:38.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.744+0000 7f98a6ffd640 1 -- 172.21.15.76:0/3570126588 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9890003520 con 0x7f98a806bb60 2023-11-09T03:54:38.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.744+0000 7f98a6ffd640 1 -- 172.21.15.76:0/3570126588 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1360672329 0 0) 0x7f9890003520 con 0x7f98a806bb60 2023-11-09T03:54:38.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.744+0000 7f98a6ffd640 1 -- 172.21.15.76:0/3570126588 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98a8159e40 msgr2=0x7f98a80b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:38.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.744+0000 7f98a6ffd640 1 --2- 172.21.15.76:0/3570126588 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98a8159e40 0x7f98a80b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:38.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.744+0000 7f98a6ffd640 1 -- 172.21.15.76:0/3570126588 >> v2:172.21.15.113:3300/0 conn(0x7f98a8070700 msgr2=0x7f98a80b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:38.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.744+0000 7f98a6ffd640 1 --2- 172.21.15.76:0/3570126588 >> v2:172.21.15.113:3300/0 conn(0x7f98a8070700 0x7f98a80b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:38.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.744+0000 7f98a6ffd640 1 -- 172.21.15.76:0/3570126588 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a8070f40 con 0x7f98a806bb60 2023-11-09T03:54:38.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.744+0000 7f98a6ffd640 1 -- 172.21.15.76:0/3570126588 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f989c001670 con 0x7f98a806bb60 2023-11-09T03:54:38.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.744+0000 7f98a6ffd640 1 -- 172.21.15.76:0/3570126588 >> v1:172.21.15.76:6789/0 conn(0x7f98a806bb60 legacy=0x7f98a806fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:38.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.744+0000 7f98a6ffd640 1 --2- 172.21.15.76:0/3570126588 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98a8159e40 0x7f9890003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:38.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.744+0000 7f98a6ffd640 1 --2- 172.21.15.76:0/3570126588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9890003ef0 0x7f98900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:38.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.744+0000 7f98a6ffd640 1 --2- 172.21.15.76:0/3570126588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9890004d00 0x7f9890005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:38.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.744+0000 7f98a6ffd640 1 -- 172.21.15.76:0/3570126588 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a815dae0 con 0x7f9890004d00 2023-11-09T03:54:38.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.744+0000 7f98a6ffd640 1 -- 172.21.15.76:0/3570126588 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a8070c60 con 0x7f9890003ef0 2023-11-09T03:54:38.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.744+0000 7f98a6ffd640 1 -- 172.21.15.76:0/3570126588 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f989000dbe0 con 0x7f98a8159e40 2023-11-09T03:54:38.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.744+0000 7f98acce9640 1 --2- 172.21.15.76:0/3570126588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9890004d00 0x7f9890005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:38.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.744+0000 7f98a7fff640 1 --2- 172.21.15.76:0/3570126588 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98a8159e40 0x7f9890003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:38.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.744+0000 7f98a77fe640 1 --2- 172.21.15.76:0/3570126588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9890003ef0 0x7f98900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:38.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.744+0000 7f98acce9640 1 -- 172.21.15.76:0/3570126588 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98a8159e40 msgr2=0x7f9890003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:38.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.744+0000 7f98acce9640 1 --2- 172.21.15.76:0/3570126588 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98a8159e40 0x7f9890003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:38.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.744+0000 7f98acce9640 1 -- 172.21.15.76:0/3570126588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9890003ef0 msgr2=0x7f98900047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:38.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.744+0000 7f98acce9640 1 --2- 172.21.15.76:0/3570126588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9890003ef0 0x7f98900047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:38.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.744+0000 7f98acce9640 1 -- 172.21.15.76:0/3570126588 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9890003520 con 0x7f9890004d00 2023-11-09T03:54:38.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.745+0000 7f98acce9640 1 --2- 172.21.15.76:0/3570126588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9890004d00 0x7f9890005180 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f98a0009fc0 tx=0x7f98900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18fcb2080825e998 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:38.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.745+0000 7f98a6ffd640 1 -- 172.21.15.76:0/3570126588 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a002cbf0 con 0x7f9890004d00 2023-11-09T03:54:38.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.745+0000 7f98a6ffd640 1 -- 172.21.15.76:0/3570126588 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a000f040 con 0x7f9890004d00 2023-11-09T03:54:38.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.745+0000 7f98a6ffd640 1 -- 172.21.15.76:0/3570126588 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a0034440 con 0x7f9890004d00 2023-11-09T03:54:38.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.746+0000 7f98ae74c640 1 -- 172.21.15.76:0/3570126588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9890004d00 msgr2=0x7f9890005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:38.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.746+0000 7f98ae74c640 1 --2- 172.21.15.76:0/3570126588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9890004d00 0x7f9890005180 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f98a0009fc0 tx=0x7f98900057f0 comp rx=0 tx=0).stop 2023-11-09T03:54:38.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.746+0000 7f98acce9640 1 -- 172.21.15.76:0/3570126588 reap_dead start 2023-11-09T03:54:38.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.746+0000 7f98ae74c640 1 -- 172.21.15.76:0/3570126588 shutdown_connections 2023-11-09T03:54:38.755 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.746+0000 7f98ae74c640 1 -- 172.21.15.76:0/3570126588 >> 172.21.15.76:0/3570126588 conn(0x7f98a80cb530 msgr2=0x7f98a80cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:38.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.746+0000 7f98ae74c640 1 -- 172.21.15.76:0/3570126588 shutdown_connections 2023-11-09T03:54:38.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.746+0000 7f98ae74c640 1 -- 172.21.15.76:0/3570126588 wait complete. 2023-11-09T03:54:38.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.747+0000 7f98ae74c640 1 Processor -- start 2023-11-09T03:54:38.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.747+0000 7f98ae74c640 1 -- start start 2023-11-09T03:54:38.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.748+0000 7f98ae74c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98a8144cf0 0x7f98a815c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:38.756 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.748+0000 7f98ae74c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98a81450d0 0x7f98a80ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:38.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.748+0000 7f98ae74c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98a815c670 0x7f98a80cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:38.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.748+0000 7f98ae74c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a81455d0 con 0x7f98a81450d0 2023-11-09T03:54:38.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.748+0000 7f98ae74c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a81458b0 con 0x7f98a8144cf0 2023-11-09T03:54:38.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.748+0000 7f98ae74c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a815cb40 con 0x7f98a815c670 2023-11-09T03:54:38.757 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.748+0000 7f98a77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98a81450d0 0x7f98a80ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:38.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.748+0000 7f98a77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98a81450d0 0x7f98a80ccc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40466/0 (socket says 172.21.15.76:40466) 2023-11-09T03:54:38.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.748+0000 7f98a77fe640 1 -- 172.21.15.76:0/1937442434 learned_addr learned my addr 172.21.15.76:0/1937442434 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:38.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.748+0000 7f98acce9640 1 --2- 172.21.15.76:0/1937442434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98a815c670 0x7f98a80cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:38.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.748+0000 7f98a7fff640 1 --2- 172.21.15.76:0/1937442434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98a8144cf0 0x7f98a815c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:38.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.748+0000 7f98a77fe640 1 -- 172.21.15.76:0/1937442434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98a815c670 msgr2=0x7f98a80cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:38.758 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.748+0000 7f98a77fe640 1 --2- 172.21.15.76:0/1937442434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98a815c670 0x7f98a80cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:38.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.748+0000 7f98a77fe640 1 -- 172.21.15.76:0/1937442434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98a8144cf0 msgr2=0x7f98a815c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:38.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.748+0000 7f98a77fe640 1 --2- 172.21.15.76:0/1937442434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98a8144cf0 0x7f98a815c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:38.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.748+0000 7f98a77fe640 1 -- 172.21.15.76:0/1937442434 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a815ce20 con 0x7f98a81450d0 2023-11-09T03:54:38.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.749+0000 7f98a77fe640 1 --2- 172.21.15.76:0/1937442434 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98a81450d0 0x7f98a80ccc70 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f989c007290 tx=0x7f989c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:38.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.749+0000 7f98a4ff9640 1 -- 172.21.15.76:0/1937442434 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c00f960 con 0x7f98a81450d0 2023-11-09T03:54:38.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.749+0000 7f98a4ff9640 1 -- 172.21.15.76:0/1937442434 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c00fac0 con 0x7f98a81450d0 2023-11-09T03:54:38.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.749+0000 7f98ae74c640 1 -- 172.21.15.76:0/1937442434 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a815d100 con 0x7f98a81450d0 2023-11-09T03:54:38.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.749+0000 7f98a4ff9640 1 -- 172.21.15.76:0/1937442434 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c00fe50 con 0x7f98a81450d0 2023-11-09T03:54:38.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.749+0000 7f98ae74c640 1 -- 172.21.15.76:0/1937442434 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a80c8a50 con 0x7f98a81450d0 2023-11-09T03:54:38.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.750+0000 7f98ae74c640 1 -- 172.21.15.76:0/1937442434 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a806dc50 con 0x7f98a81450d0 2023-11-09T03:54:38.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.750+0000 7f98a4ff9640 1 -- 172.21.15.76:0/1937442434 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f989c01d460 con 0x7f98a81450d0 2023-11-09T03:54:38.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.751+0000 7f98a4ff9640 1 --2- 172.21.15.76:0/1937442434 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f989406ae70 0x7f989406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:38.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.751+0000 7f98a4ff9640 1 -- 172.21.15.76:0/1937442434 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7f989c08a250 con 0x7f98a81450d0 2023-11-09T03:54:38.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.751+0000 7f98a7fff640 1 --2- 172.21.15.76:0/1937442434 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f989406ae70 0x7f989406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:38.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.751+0000 7f98a7fff640 1 --2- 172.21.15.76:0/1937442434 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f989406ae70 0x7f989406d330 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9898009b00 tx=0x7f9898008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:38.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.755+0000 7f98a4ff9640 1 -- 172.21.15.76:0/1937442434 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f989c054890 con 0x7f98a81450d0 2023-11-09T03:54:38.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.960+0000 7f98ae74c640 1 -- 172.21.15.76:0/1937442434 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98a80684b0 con 0x7f98a81450d0 2023-11-09T03:54:38.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.960+0000 7f98a4ff9640 1 -- 172.21.15.76:0/1937442434 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+4935 (secure 0 0 0) 0x7f989c057eb0 con 0x7f98a81450d0 2023-11-09T03:54:38.960 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:54:38.961 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":126,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":9024,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:26.668321+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":6470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6470":{"gid":6470,"name":"b","rank":1,"incarnation":121,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:54:38.961 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 126 2023-11-09T03:54:38.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.963+0000 7f98ae74c640 1 -- 172.21.15.76:0/1937442434 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f989406ae70 msgr2=0x7f989406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:38.963 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.963+0000 7f98ae74c640 1 --2- 172.21.15.76:0/1937442434 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f989406ae70 0x7f989406d330 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9898009b00 tx=0x7f9898008040 comp rx=0 tx=0).stop 2023-11-09T03:54:38.963 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.963+0000 7f98ae74c640 1 -- 172.21.15.76:0/1937442434 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98a81450d0 msgr2=0x7f98a80ccc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:38.963 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.963+0000 7f98ae74c640 1 --2- 172.21.15.76:0/1937442434 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98a81450d0 0x7f98a80ccc70 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f989c007290 tx=0x7f989c007750 comp rx=0 tx=0).stop 2023-11-09T03:54:38.963 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.963+0000 7f98ae74c640 1 -- 172.21.15.76:0/1937442434 shutdown_connections 2023-11-09T03:54:38.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.963+0000 7f98ae74c640 1 --2- 172.21.15.76:0/1937442434 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f989406ae70 0x7f989406d330 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:38.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.963+0000 7f98ae74c640 1 --2- 172.21.15.76:0/1937442434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98a815c670 0x7f98a80cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:38.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.963+0000 7f98ae74c640 1 --2- 172.21.15.76:0/1937442434 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98a81450d0 0x7f98a80ccc70 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:38.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.963+0000 7f98ae74c640 1 --2- 172.21.15.76:0/1937442434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98a8144cf0 0x7f98a815c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:38.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.963+0000 7f98ae74c640 1 -- 172.21.15.76:0/1937442434 >> 172.21.15.76:0/1937442434 conn(0x7f98a80cb530 msgr2=0x7f98a815dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:38.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.963+0000 7f98ae74c640 1 -- 172.21.15.76:0/1937442434 shutdown_connections 2023-11-09T03:54:38.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:38.963+0000 7f98ae74c640 1 -- 172.21.15.76:0/1937442434 wait complete. 2023-11-09T03:54:38.973 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:54:43.974 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:54:44.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.084+0000 7f5fbcea2640 1 Processor -- start 2023-11-09T03:54:44.085 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.084+0000 7f5fbcea2640 1 -- start start 2023-11-09T03:54:44.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.085+0000 7f5fbcea2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5fb806d9c0 0x7f5fb80710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:44.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.085+0000 7f5fbcea2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5fb806c5f0 0x7f5fb8071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:44.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.085+0000 7f5fbcea2640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fb806cc90 con 0x7f5fb80ade00 2023-11-09T03:54:44.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.085+0000 7f5fbcea2640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fb80b0c40 con 0x7f5fb806c5f0 2023-11-09T03:54:44.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.085+0000 7f5fbcea2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fb8071b50 con 0x7f5fb806d9c0 2023-11-09T03:54:44.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.085+0000 7f5fb6d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5fb806c5f0 0x7f5fb8071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:44.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.085+0000 7f5fb6d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5fb806c5f0 0x7f5fb8071610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58024/0 (socket says 172.21.15.76:58024) 2023-11-09T03:54:44.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.085+0000 7f5fb6d9d640 1 -- 172.21.15.76:0/172641125 learned_addr learned my addr 172.21.15.76:0/172641125 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:44.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.085+0000 7f5fb659c640 1 --1- 172.21.15.76:0/172641125 >> v1:172.21.15.76:6789/0 conn(0x7f5fb80ade00 0x7f5fb80b0530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:49022/0 (socket says 172.21.15.76:49022) 2023-11-09T03:54:44.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.085+0000 7f5faffff640 1 --2- 172.21.15.76:0/172641125 >> v2:172.21.15.113:3300/0 conn(0x7f5fb806d9c0 0x7f5fb80710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:44.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.085+0000 7f5fb6d9d640 1 -- 172.21.15.76:0/172641125 >> v2:172.21.15.113:3300/0 conn(0x7f5fb806d9c0 msgr2=0x7f5fb80710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:44.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.085+0000 7f5fb6d9d640 1 --2- 172.21.15.76:0/172641125 >> v2:172.21.15.113:3300/0 conn(0x7f5fb806d9c0 0x7f5fb80710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:44.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.085+0000 7f5fb6d9d640 1 -- 172.21.15.76:0/172641125 >> v1:172.21.15.76:6789/0 conn(0x7f5fb80ade00 legacy=0x7f5fb80b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:44.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.085+0000 7f5fb6d9d640 1 -- 172.21.15.76:0/172641125 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb80b0e60 con 0x7f5fb806c5f0 2023-11-09T03:54:44.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.085+0000 7f5fb5d9b640 1 -- 172.21.15.76:0/172641125 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 137393577 0 0) 0x7f5fb806cc90 con 0x7f5fb80ade00 2023-11-09T03:54:44.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.086+0000 7f5fb6d9d640 1 --2- 172.21.15.76:0/172641125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fb806c5f0 0x7f5fb8071610 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f5fb80c9080 tx=0x7f5fb002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe9fa7a74d07fe4 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:44.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.086+0000 7f5fb5d9b640 1 -- 172.21.15.76:0/172641125 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb002c9f0 con 0x7f5fb806c5f0 2023-11-09T03:54:44.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.086+0000 7f5fb5d9b640 1 -- 172.21.15.76:0/172641125 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb002cb50 con 0x7f5fb806c5f0 2023-11-09T03:54:44.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.086+0000 7f5fb5d9b640 1 -- 172.21.15.76:0/172641125 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb00343f0 con 0x7f5fb806c5f0 2023-11-09T03:54:44.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.086+0000 7f5fbcea2640 1 -- 172.21.15.76:0/172641125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fb806c5f0 msgr2=0x7f5fb8071610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:44.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.086+0000 7f5fbcea2640 1 --2- 172.21.15.76:0/172641125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fb806c5f0 0x7f5fb8071610 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f5fb80c9080 tx=0x7f5fb002be50 comp rx=0 tx=0).stop 2023-11-09T03:54:44.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.087+0000 7f5fbcea2640 1 -- 172.21.15.76:0/172641125 shutdown_connections 2023-11-09T03:54:44.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.087+0000 7f5fbcea2640 1 --2- 172.21.15.76:0/172641125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fb806c5f0 0x7f5fb8071610 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:44.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.087+0000 7f5fbcea2640 1 --2- 172.21.15.76:0/172641125 >> v2:172.21.15.113:3300/0 conn(0x7f5fb806d9c0 0x7f5fb80710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:44.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.087+0000 7f5fbcea2640 1 -- 172.21.15.76:0/172641125 >> 172.21.15.76:0/172641125 conn(0x7f5fb80cb530 msgr2=0x7f5fb80cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:44.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.087+0000 7f5fbcea2640 1 -- 172.21.15.76:0/172641125 shutdown_connections 2023-11-09T03:54:44.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.087+0000 7f5fbcea2640 1 -- 172.21.15.76:0/172641125 wait complete. 2023-11-09T03:54:44.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.088+0000 7f5fbcea2640 1 Processor -- start 2023-11-09T03:54:44.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.088+0000 7f5fbcea2640 1 -- start start 2023-11-09T03:54:44.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.089+0000 7f5fbcea2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fb806c5f0 0x7f5fb81eff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:44.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.089+0000 7f5fbcea2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5fb806d9c0 0x7f5fb80c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:44.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.089+0000 7f5fbcea2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fb80ade00 0x7f5fb80c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:44.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.089+0000 7f5fbcea2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb81f3fc0 con 0x7f5fb806c5f0 2023-11-09T03:54:44.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.089+0000 7f5fbcea2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb81497a0 con 0x7f5fb80ade00 2023-11-09T03:54:44.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.089+0000 7f5fbcea2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb80c2580 con 0x7f5fb806d9c0 2023-11-09T03:54:44.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.089+0000 7f5fb659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fb806c5f0 0x7f5fb81eff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:44.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.089+0000 7f5fb659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fb806c5f0 0x7f5fb81eff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58034/0 (socket says 172.21.15.76:58034) 2023-11-09T03:54:44.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.089+0000 7f5fb659c640 1 -- 172.21.15.76:0/218423568 learned_addr learned my addr 172.21.15.76:0/218423568 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:44.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.089+0000 7f5fb6d9d640 1 --2- 172.21.15.76:0/218423568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fb80ade00 0x7f5fb80c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:44.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.089+0000 7f5faffff640 1 --2- 172.21.15.76:0/218423568 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5fb806d9c0 0x7f5fb80c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:44.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.089+0000 7f5fb659c640 1 -- 172.21.15.76:0/218423568 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5fb806d9c0 msgr2=0x7f5fb80c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:44.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.089+0000 7f5fb659c640 1 --2- 172.21.15.76:0/218423568 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5fb806d9c0 0x7f5fb80c4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:44.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.089+0000 7f5fb659c640 1 -- 172.21.15.76:0/218423568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fb80ade00 msgr2=0x7f5fb80c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:44.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.089+0000 7f5fb659c640 1 --2- 172.21.15.76:0/218423568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fb80ade00 0x7f5fb80c4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:44.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.089+0000 7f5fb659c640 1 -- 172.21.15.76:0/218423568 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb0009b30 con 0x7f5fb806c5f0 2023-11-09T03:54:44.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.089+0000 7f5fb6d9d640 1 --2- 172.21.15.76:0/218423568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fb80ade00 0x7f5fb80c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:54:44.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.089+0000 7f5fb659c640 1 --2- 172.21.15.76:0/218423568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fb806c5f0 0x7f5fb81eff50 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800c430 tx=0x7f5fa800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:44.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.089+0000 7f5faeffd640 1 -- 172.21.15.76:0/218423568 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa80074e0 con 0x7f5fb806c5f0 2023-11-09T03:54:44.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.089+0000 7f5faeffd640 1 -- 172.21.15.76:0/218423568 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa8007660 con 0x7f5fb806c5f0 2023-11-09T03:54:44.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.089+0000 7f5fbcea2640 1 -- 172.21.15.76:0/218423568 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb81f0490 con 0x7f5fb806c5f0 2023-11-09T03:54:44.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.089+0000 7f5faeffd640 1 -- 172.21.15.76:0/218423568 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8004a20 con 0x7f5fb806c5f0 2023-11-09T03:54:44.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.089+0000 7f5fbcea2640 1 -- 172.21.15.76:0/218423568 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb80c6ae0 con 0x7f5fb806c5f0 2023-11-09T03:54:44.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.090+0000 7f5fbcea2640 1 -- 172.21.15.76:0/218423568 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb806d480 con 0x7f5fb806c5f0 2023-11-09T03:54:44.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.091+0000 7f5faeffd640 1 -- 172.21.15.76:0/218423568 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5fa8004b80 con 0x7f5fb806c5f0 2023-11-09T03:54:44.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.091+0000 7f5faeffd640 1 --2- 172.21.15.76:0/218423568 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5f9806ae70 0x7f5f9806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:44.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.091+0000 7f5faeffd640 1 -- 172.21.15.76:0/218423568 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7f5fa8089de0 con 0x7f5fb806c5f0 2023-11-09T03:54:44.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.091+0000 7f5faffff640 1 --2- 172.21.15.76:0/218423568 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5f9806ae70 0x7f5f9806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:44.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.092+0000 7f5faffff640 1 --2- 172.21.15.76:0/218423568 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5f9806ae70 0x7f5f9806d330 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0006fd0 tx=0x7f5fa0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:44.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.096+0000 7f5faeffd640 1 -- 172.21.15.76:0/218423568 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5fb806d480 con 0x7f5fb806c5f0 2023-11-09T03:54:44.299 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.299+0000 7f5fbcea2640 1 -- 172.21.15.76:0/218423568 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5fb806d480 con 0x7f5fb806c5f0 2023-11-09T03:54:44.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.299+0000 7f5faeffd640 1 -- 172.21.15.76:0/218423568 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 127 v127) v1 ==== 95+0+4983 (secure 0 0 0) 0x7f5fb806d480 con 0x7f5fb806c5f0 2023-11-09T03:54:44.300 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:54:44.300 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":127,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":9024,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":127,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:41.489585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":6470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6470":{"gid":6470,"name":"b","rank":1,"incarnation":121,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"laggy_since":"2023-11-09T03:54:41.489584+0000","export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:54:44.301 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 127 2023-11-09T03:54:44.302 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.301+0000 7f5fbcea2640 1 -- 172.21.15.76:0/218423568 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5f9806ae70 msgr2=0x7f5f9806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:44.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.301+0000 7f5fbcea2640 1 --2- 172.21.15.76:0/218423568 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5f9806ae70 0x7f5f9806d330 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0006fd0 tx=0x7f5fa0008040 comp rx=0 tx=0).stop 2023-11-09T03:54:44.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.301+0000 7f5fbcea2640 1 -- 172.21.15.76:0/218423568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fb806c5f0 msgr2=0x7f5fb81eff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:44.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.301+0000 7f5fbcea2640 1 --2- 172.21.15.76:0/218423568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fb806c5f0 0x7f5fb81eff50 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800c430 tx=0x7f5fa800c8f0 comp rx=0 tx=0).stop 2023-11-09T03:54:44.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.301+0000 7f5fbcea2640 1 -- 172.21.15.76:0/218423568 shutdown_connections 2023-11-09T03:54:44.303 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.301+0000 7f5fbcea2640 1 --2- 172.21.15.76:0/218423568 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5f9806ae70 0x7f5f9806d330 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:44.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.301+0000 7f5fbcea2640 1 --2- 172.21.15.76:0/218423568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fb80ade00 0x7f5fb80c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:44.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.301+0000 7f5fbcea2640 1 --2- 172.21.15.76:0/218423568 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5fb806d9c0 0x7f5fb80c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:44.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.301+0000 7f5fbcea2640 1 --2- 172.21.15.76:0/218423568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fb806c5f0 0x7f5fb81eff50 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:44.304 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.301+0000 7f5fbcea2640 1 -- 172.21.15.76:0/218423568 >> 172.21.15.76:0/218423568 conn(0x7f5fb80cb530 msgr2=0x7f5fb80b0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:44.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.302+0000 7f5fbcea2640 1 -- 172.21.15.76:0/218423568 shutdown_connections 2023-11-09T03:54:44.305 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.302+0000 7f5fbcea2640 1 -- 172.21.15.76:0/218423568 wait complete. 2023-11-09T03:54:44.313 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2023-11-09T03:54:44.313 DEBUG:teuthology.orchestra.run.smithi113:> sudo sysctl -n kernel.core_pattern 2023-11-09T03:54:44.343 INFO:teuthology.orchestra.run.smithi113.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-09T03:54:44.345 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2023-11-09T03:54:44.345 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2023-11-09T03:54:44.417 INFO:teuthology.orchestra.run.smithi113.stdout:1699502067.96335.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i b', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2023-11-09T03:54:44.419 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1699502067.96335.core'] 2023-11-09T03:54:44.419 INFO:tasks.cephfs.cephfs_test_case:Found core file 1699502067.96335.core, deleting it 2023-11-09T03:54:44.420 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1699502067.96335.core 2023-11-09T03:54:44.490 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:54:44.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:54:44.491 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 down 2023-11-09T03:54:44.519 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:54:44.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.658+0000 7f24f6797640 1 Processor -- start 2023-11-09T03:54:44.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.658+0000 7f24f6797640 1 -- start start 2023-11-09T03:54:44.660 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.659+0000 7f24f6797640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f24f015d110 0x7f24f006b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:44.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.659+0000 7f24f6797640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24f00b2100 0x7f24f00b2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:44.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.659+0000 7f24f6797640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f00b2a90 con 0x7f24f00b3ca0 2023-11-09T03:54:44.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.659+0000 7f24f6797640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f00664f0 con 0x7f24f015d110 2023-11-09T03:54:44.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.659+0000 7f24f6797640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f015d4f0 con 0x7f24f00b2100 2023-11-09T03:54:44.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.659+0000 7f24effff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f24f015d110 0x7f24f006b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:44.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.659+0000 7f24ef7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f24f00b3ca0 0x7f24f00ace70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:49036/0 (socket says 172.21.15.76:49036) 2023-11-09T03:54:44.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.659+0000 7f24effff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f24f015d110 0x7f24f006b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58044/0 (socket says 172.21.15.76:58044) 2023-11-09T03:54:44.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.659+0000 7f24ef7fe640 1 -- 172.21.15.76:0/2205312370 learned_addr learned my addr 172.21.15.76:0/2205312370 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:54:44.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.660+0000 7f24f4d34640 1 --2- 172.21.15.76:0/2205312370 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24f00b2100 0x7f24f00b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:44.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.660+0000 7f24effff640 1 -- 172.21.15.76:0/2205312370 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24f00b2100 msgr2=0x7f24f00b2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:44.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.660+0000 7f24effff640 1 --2- 172.21.15.76:0/2205312370 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24f00b2100 0x7f24f00b2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:44.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.660+0000 7f24effff640 1 -- 172.21.15.76:0/2205312370 >> v1:172.21.15.76:6789/0 conn(0x7f24f00b3ca0 legacy=0x7f24f00ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:44.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.660+0000 7f24effff640 1 -- 172.21.15.76:0/2205312370 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f006b9f0 con 0x7f24f015d110 2023-11-09T03:54:44.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.660+0000 7f24eeffd640 1 -- 172.21.15.76:0/2205312370 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1237292376 0 0) 0x7f24e4001110 con 0x7f24f00b3ca0 2023-11-09T03:54:44.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.660+0000 7f24f4d34640 1 --2- 172.21.15.76:0/2205312370 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24f00b2100 0x7f24f00b2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:54:44.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.660+0000 7f24effff640 1 --2- 172.21.15.76:0/2205312370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24f015d110 0x7f24f006b2f0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009790 tx=0x7f24e002b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df2c68283c3e23f server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:44.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.660+0000 7f24eeffd640 1 -- 172.21.15.76:0/2205312370 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e0031800 con 0x7f24f015d110 2023-11-09T03:54:44.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.660+0000 7f24eeffd640 1 -- 172.21.15.76:0/2205312370 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e0031960 con 0x7f24f015d110 2023-11-09T03:54:44.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.660+0000 7f24eeffd640 1 -- 172.21.15.76:0/2205312370 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e0031cf0 con 0x7f24f015d110 2023-11-09T03:54:44.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.661+0000 7f24f6797640 1 -- 172.21.15.76:0/2205312370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24f015d110 msgr2=0x7f24f006b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:44.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.661+0000 7f24f6797640 1 --2- 172.21.15.76:0/2205312370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24f015d110 0x7f24f006b2f0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009790 tx=0x7f24e002b8a0 comp rx=0 tx=0).stop 2023-11-09T03:54:44.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.661+0000 7f24f6797640 1 -- 172.21.15.76:0/2205312370 shutdown_connections 2023-11-09T03:54:44.668 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.661+0000 7f24f6797640 1 --2- 172.21.15.76:0/2205312370 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24f00b2100 0x7f24f00b2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:44.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.661+0000 7f24f6797640 1 --2- 172.21.15.76:0/2205312370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24f015d110 0x7f24f006b2f0 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:44.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.661+0000 7f24f6797640 1 -- 172.21.15.76:0/2205312370 >> 172.21.15.76:0/2205312370 conn(0x7f24f00cbd30 msgr2=0x7f24f00cc140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:44.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.661+0000 7f24f6797640 1 -- 172.21.15.76:0/2205312370 shutdown_connections 2023-11-09T03:54:44.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.661+0000 7f24f6797640 1 -- 172.21.15.76:0/2205312370 wait complete. 2023-11-09T03:54:44.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.662+0000 7f24f6797640 1 Processor -- start 2023-11-09T03:54:44.669 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.662+0000 7f24f6797640 1 -- start start 2023-11-09T03:54:44.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.663+0000 7f24f6797640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24f00b2100 0x7f24f00ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:44.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.663+0000 7f24f6797640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f00b3ca0 0x7f24f015d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:44.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.663+0000 7f24f6797640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24f015d110 0x7f24f0159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:44.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.663+0000 7f24f6797640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f0071380 con 0x7f24f015d110 2023-11-09T03:54:44.670 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.663+0000 7f24f6797640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f015ddc0 con 0x7f24f00b3ca0 2023-11-09T03:54:44.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.663+0000 7f24f6797640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f01596b0 con 0x7f24f00b2100 2023-11-09T03:54:44.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.663+0000 7f24f4d34640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24f015d110 0x7f24f0159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:44.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.663+0000 7f24f4d34640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24f015d110 0x7f24f0159170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58056/0 (socket says 172.21.15.76:58056) 2023-11-09T03:54:44.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.663+0000 7f24f4d34640 1 -- 172.21.15.76:0/2831433797 learned_addr learned my addr 172.21.15.76:0/2831433797 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:44.671 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.663+0000 7f24effff640 1 --2- 172.21.15.76:0/2831433797 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24f00b2100 0x7f24f00ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:44.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.664+0000 7f24f4d34640 1 -- 172.21.15.76:0/2831433797 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24f00b2100 msgr2=0x7f24f00ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:44.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.664+0000 7f24ef7fe640 1 --2- 172.21.15.76:0/2831433797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f00b3ca0 0x7f24f015d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:44.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.664+0000 7f24f4d34640 1 --2- 172.21.15.76:0/2831433797 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24f00b2100 0x7f24f00ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:44.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.664+0000 7f24f4d34640 1 -- 172.21.15.76:0/2831433797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f00b3ca0 msgr2=0x7f24f015d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:44.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.664+0000 7f24f4d34640 1 --2- 172.21.15.76:0/2831433797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f00b3ca0 0x7f24f015d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:44.672 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.664+0000 7f24f4d34640 1 -- 172.21.15.76:0/2831433797 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e00093a0 con 0x7f24f015d110 2023-11-09T03:54:44.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.664+0000 7f24effff640 1 --2- 172.21.15.76:0/2831433797 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24f00b2100 0x7f24f00ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:54:44.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.664+0000 7f24f4d34640 1 --2- 172.21.15.76:0/2831433797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24f015d110 0x7f24f0159170 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f24e800cd80 tx=0x7f24e800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:44.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.664+0000 7f24ecff9640 1 -- 172.21.15.76:0/2831433797 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e800f680 con 0x7f24f015d110 2023-11-09T03:54:44.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.664+0000 7f24ecff9640 1 -- 172.21.15.76:0/2831433797 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e800f800 con 0x7f24f015d110 2023-11-09T03:54:44.673 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.664+0000 7f24f6797640 1 -- 172.21.15.76:0/2831433797 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f00709e0 con 0x7f24f015d110 2023-11-09T03:54:44.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.664+0000 7f24ecff9640 1 -- 172.21.15.76:0/2831433797 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e800fbb0 con 0x7f24f015d110 2023-11-09T03:54:44.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.664+0000 7f24f6797640 1 -- 172.21.15.76:0/2831433797 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f0070fa0 con 0x7f24f015d110 2023-11-09T03:54:44.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.665+0000 7f24f6797640 1 -- 172.21.15.76:0/2831433797 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f00684b0 con 0x7f24f015d110 2023-11-09T03:54:44.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.665+0000 7f24ecff9640 1 -- 172.21.15.76:0/2831433797 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f24e800b040 con 0x7f24f015d110 2023-11-09T03:54:44.674 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.666+0000 7f24ecff9640 1 --2- 172.21.15.76:0/2831433797 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f24dc06ae70 0x7f24dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:44.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.666+0000 7f24ecff9640 1 -- 172.21.15.76:0/2831433797 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7f24e808a980 con 0x7f24f015d110 2023-11-09T03:54:44.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.666+0000 7f24effff640 1 --2- 172.21.15.76:0/2831433797 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f24dc06ae70 0x7f24dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:44.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.666+0000 7f24effff640 1 --2- 172.21.15.76:0/2831433797 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f24dc06ae70 0x7f24dc06d330 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f24e0036000 tx=0x7f24e0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:44.675 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.670+0000 7f24ecff9640 1 -- 172.21.15.76:0/2831433797 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f24e8055050 con 0x7f24f015d110 2023-11-09T03:54:44.870 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.870+0000 7f24f6797640 1 -- 172.21.15.76:0/2831433797 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24f00c9320 con 0x7f24f015d110 2023-11-09T03:54:44.871 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.870+0000 7f24ecff9640 1 -- 172.21.15.76:0/2831433797 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 127 v127) v1 ==== 95+0+4983 (secure 0 0 0) 0x7f24e8058670 con 0x7f24f015d110 2023-11-09T03:54:44.871 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:54:44.871 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":127,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":9024,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":127,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:41.489585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":6470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6470":{"gid":6470,"name":"b","rank":1,"incarnation":121,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6837/3462040782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3462040782},{"type":"v1","addr":"172.21.15.113:6837","nonce":3462040782}]},"join_fscid":-1,"laggy_since":"2023-11-09T03:54:41.489584+0000","export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:54:44.872 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 127 2023-11-09T03:54:44.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.872+0000 7f24f6797640 1 -- 172.21.15.76:0/2831433797 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f24dc06ae70 msgr2=0x7f24dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:44.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.872+0000 7f24f6797640 1 --2- 172.21.15.76:0/2831433797 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f24dc06ae70 0x7f24dc06d330 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f24e0036000 tx=0x7f24e0008030 comp rx=0 tx=0).stop 2023-11-09T03:54:44.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.872+0000 7f24f6797640 1 -- 172.21.15.76:0/2831433797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24f015d110 msgr2=0x7f24f0159170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:44.873 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.872+0000 7f24f6797640 1 --2- 172.21.15.76:0/2831433797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24f015d110 0x7f24f0159170 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f24e800cd80 tx=0x7f24e800a650 comp rx=0 tx=0).stop 2023-11-09T03:54:44.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.872+0000 7f24f6797640 1 -- 172.21.15.76:0/2831433797 shutdown_connections 2023-11-09T03:54:44.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.872+0000 7f24f6797640 1 --2- 172.21.15.76:0/2831433797 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f24dc06ae70 0x7f24dc06d330 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:44.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.872+0000 7f24f6797640 1 --2- 172.21.15.76:0/2831433797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24f015d110 0x7f24f0159170 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:44.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.872+0000 7f24f6797640 1 --2- 172.21.15.76:0/2831433797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f00b3ca0 0x7f24f015d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:44.874 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.872+0000 7f24f6797640 1 --2- 172.21.15.76:0/2831433797 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24f00b2100 0x7f24f00ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:44.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.872+0000 7f24f6797640 1 -- 172.21.15.76:0/2831433797 >> 172.21.15.76:0/2831433797 conn(0x7f24f00cbd30 msgr2=0x7f24f00acd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:44.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.872+0000 7f24f6797640 1 -- 172.21.15.76:0/2831433797 shutdown_connections 2023-11-09T03:54:44.875 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:44.872+0000 7f24f6797640 1 -- 172.21.15.76:0/2831433797 wait complete. 2023-11-09T03:54:44.884 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2023-11-09T03:54:44.997 INFO:teuthology.orchestra.run.smithi076.stdout:{"last_snap":16,"last_created":12,"last_destroyed":14,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[{"version":34,"snapinfo":{"snapid":16,"ino":1099511627777,"stamp":"2023-11-09T03:54:27.826603+0000","name":"s34","metadata":{}}}],"pending_destroy":[]} 2023-11-09T03:54:44.997 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 12, "last_destroyed": 14, "last_snap": 16, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [ { "snapInfinityo": { "ino": 1099511627777, "metadata": {}, "name": "s34", "snapid": 16, "stamp": "2023-11-09T03:54:27.826603+0000" }, "version": 34 } ], "snaps": [] } 2023-11-09T03:54:44.998 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:1 2023-11-09T03:54:45.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.098+0000 7f6aae087640 1 Processor -- start 2023-11-09T03:54:45.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.098+0000 7f6aae087640 1 -- start start 2023-11-09T03:54:45.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.099+0000 7f6aae087640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6aa8159e40 0x7f6aa80b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:45.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.099+0000 7f6aae087640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6aa806bb60 0x7f6aa80b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:45.100 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.099+0000 7f6aae087640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa8071500 con 0x7f6aa806fff0 2023-11-09T03:54:45.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.099+0000 7f6aae087640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aa815dae0 con 0x7f6aa806bb60 2023-11-09T03:54:45.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.099+0000 7f6aae087640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aa8070c60 con 0x7f6aa8159e40 2023-11-09T03:54:45.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.099+0000 7f6aa6ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6aa806bb60 0x7f6aa80b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:45.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.099+0000 7f6aa6ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6aa806bb60 0x7f6aa80b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58072/0 (socket says 172.21.15.76:58072) 2023-11-09T03:54:45.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.099+0000 7f6aa6ffd640 1 -- 172.21.15.76:0/3571597082 learned_addr learned my addr 172.21.15.76:0/3571597082 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:45.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.099+0000 7f6aa77fe640 1 --2- 172.21.15.76:0/3571597082 >> v2:172.21.15.113:3300/0 conn(0x7f6aa8159e40 0x7f6aa80b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:45.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.099+0000 7f6aa6ffd640 1 -- 172.21.15.76:0/3571597082 >> v2:172.21.15.113:3300/0 conn(0x7f6aa8159e40 msgr2=0x7f6aa80b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:45.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.099+0000 7f6aa6ffd640 1 --2- 172.21.15.76:0/3571597082 >> v2:172.21.15.113:3300/0 conn(0x7f6aa8159e40 0x7f6aa80b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:45.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.099+0000 7f6aa6ffd640 1 -- 172.21.15.76:0/3571597082 >> v1:172.21.15.113:6789/0 conn(0x7f6aa806fff0 legacy=0x7f6aa8070420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:45.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.099+0000 7f6aa6ffd640 1 -- 172.21.15.76:0/3571597082 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa8070f40 con 0x7f6aa806bb60 2023-11-09T03:54:45.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.100+0000 7f6aa6ffd640 1 --2- 172.21.15.76:0/3571597082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aa806bb60 0x7f6aa80b0a70 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f6aa80ce7f0 tx=0x7f6a9c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f15fe287d7578d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:45.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.100+0000 7f6aa67fc640 1 -- 172.21.15.76:0/3571597082 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c0317e0 con 0x7f6aa806bb60 2023-11-09T03:54:45.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.100+0000 7f6aa67fc640 1 -- 172.21.15.76:0/3571597082 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9c031940 con 0x7f6aa806bb60 2023-11-09T03:54:45.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.100+0000 7f6aa67fc640 1 -- 172.21.15.76:0/3571597082 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c031cd0 con 0x7f6aa806bb60 2023-11-09T03:54:45.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.101+0000 7f6aae087640 1 -- 172.21.15.76:0/3571597082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aa806bb60 msgr2=0x7f6aa80b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:45.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.101+0000 7f6aae087640 1 --2- 172.21.15.76:0/3571597082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aa806bb60 0x7f6aa80b0a70 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f6aa80ce7f0 tx=0x7f6a9c02b880 comp rx=0 tx=0).stop 2023-11-09T03:54:45.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.101+0000 7f6aae087640 1 -- 172.21.15.76:0/3571597082 shutdown_connections 2023-11-09T03:54:45.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.101+0000 7f6aae087640 1 --2- 172.21.15.76:0/3571597082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aa806bb60 0x7f6aa80b0a70 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:45.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.101+0000 7f6aae087640 1 --2- 172.21.15.76:0/3571597082 >> v2:172.21.15.113:3300/0 conn(0x7f6aa8159e40 0x7f6aa80b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:45.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.101+0000 7f6aae087640 1 -- 172.21.15.76:0/3571597082 >> 172.21.15.76:0/3571597082 conn(0x7f6aa80cb530 msgr2=0x7f6aa80cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:45.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.101+0000 7f6aae087640 1 -- 172.21.15.76:0/3571597082 shutdown_connections 2023-11-09T03:54:45.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.101+0000 7f6aae087640 1 -- 172.21.15.76:0/3571597082 wait complete. 2023-11-09T03:54:45.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.102+0000 7f6aae087640 1 Processor -- start 2023-11-09T03:54:45.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.102+0000 7f6aae087640 1 -- start start 2023-11-09T03:54:45.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.103+0000 7f6aae087640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6aa806bb60 0x7f6aa8153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:45.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.103+0000 7f6aae087640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6aa806fff0 0x7f6aa80c3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:45.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.103+0000 7f6aae087640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aa8159e40 0x7f6aa80c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:45.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.103+0000 7f6aae087640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa8153640 con 0x7f6aa8159e40 2023-11-09T03:54:45.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.103+0000 7f6aae087640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa80c3b20 con 0x7f6aa806fff0 2023-11-09T03:54:45.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.103+0000 7f6aae087640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa80c3e00 con 0x7f6aa806bb60 2023-11-09T03:54:45.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.103+0000 7f6aa7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aa8159e40 0x7f6aa80c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:45.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.103+0000 7f6aa7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aa8159e40 0x7f6aa80c7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58078/0 (socket says 172.21.15.76:58078) 2023-11-09T03:54:45.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.103+0000 7f6aa7fff640 1 -- 172.21.15.76:0/825148835 learned_addr learned my addr 172.21.15.76:0/825148835 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:45.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.103+0000 7f6aa6ffd640 1 --2- 172.21.15.76:0/825148835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6aa806fff0 0x7f6aa80c3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:45.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.103+0000 7f6aa77fe640 1 --2- 172.21.15.76:0/825148835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6aa806bb60 0x7f6aa8153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:45.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.103+0000 7f6aa7fff640 1 -- 172.21.15.76:0/825148835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6aa806bb60 msgr2=0x7f6aa8153100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:45.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.103+0000 7f6aa7fff640 1 --2- 172.21.15.76:0/825148835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6aa806bb60 0x7f6aa8153100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:45.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.103+0000 7f6aa7fff640 1 -- 172.21.15.76:0/825148835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6aa806fff0 msgr2=0x7f6aa80c3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:45.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.103+0000 7f6aa7fff640 1 --2- 172.21.15.76:0/825148835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6aa806fff0 0x7f6aa80c3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:45.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.103+0000 7f6aa7fff640 1 -- 172.21.15.76:0/825148835 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9c0094d0 con 0x7f6aa8159e40 2023-11-09T03:54:45.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.103+0000 7f6aa77fe640 1 --2- 172.21.15.76:0/825148835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6aa806bb60 0x7f6aa8153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:54:45.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.103+0000 7f6aa7fff640 1 --2- 172.21.15.76:0/825148835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aa8159e40 0x7f6aa80c7d90 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000c7a0 tx=0x7f6aa000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:45.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.103+0000 7f6a8ffff640 1 -- 172.21.15.76:0/825148835 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0003b70 con 0x7f6aa8159e40 2023-11-09T03:54:45.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.103+0000 7f6a8ffff640 1 -- 172.21.15.76:0/825148835 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa0003cf0 con 0x7f6aa8159e40 2023-11-09T03:54:45.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.104+0000 7f6aae087640 1 -- 172.21.15.76:0/825148835 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa80c40e0 con 0x7f6aa8159e40 2023-11-09T03:54:45.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.104+0000 7f6a8ffff640 1 -- 172.21.15.76:0/825148835 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0003ea0 con 0x7f6aa8159e40 2023-11-09T03:54:45.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.104+0000 7f6aae087640 1 -- 172.21.15.76:0/825148835 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa8153ce0 con 0x7f6aa8159e40 2023-11-09T03:54:45.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.105+0000 7f6aae087640 1 -- 172.21.15.76:0/825148835 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa80684b0 con 0x7f6aa8159e40 2023-11-09T03:54:45.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.105+0000 7f6a8ffff640 1 -- 172.21.15.76:0/825148835 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6aa000aa90 con 0x7f6aa8159e40 2023-11-09T03:54:45.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.105+0000 7f6a8ffff640 1 --2- 172.21.15.76:0/825148835 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6a9406ae70 0x7f6a9406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:45.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.105+0000 7f6a8ffff640 1 -- 172.21.15.76:0/825148835 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8296+0+0 (secure 0 0 0) 0x7f6aa0089ee0 con 0x7f6aa8159e40 2023-11-09T03:54:45.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.105+0000 7f6aa77fe640 1 --2- 172.21.15.76:0/825148835 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6a9406ae70 0x7f6a9406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:45.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.106+0000 7f6aa77fe640 1 --2- 172.21.15.76:0/825148835 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6a9406ae70 0x7f6a9406d330 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f6a98006fd0 tx=0x7f6a98008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:45.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.110+0000 7f6a8ffff640 1 -- 172.21.15.76:0/825148835 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6aa0054520 con 0x7f6aa8159e40 2023-11-09T03:54:45.317 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~13s 2023-11-09T03:54:45.344 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.343+0000 7f6aae087640 1 -- 172.21.15.76:0/825148835 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:1"} v 0) v1 -- 0x7f6aa80c9750 con 0x7f6aa8159e40 2023-11-09T03:54:45.346 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.345+0000 7f6a8ffff640 1 -- 172.21.15.76:0/825148835 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:1"}]=0 failed mds gid 6470 v128) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6aa0087e60 con 0x7f6aa8159e40 2023-11-09T03:54:45.346 INFO:teuthology.orchestra.run.smithi076.stderr:failed mds gid 6470 2023-11-09T03:54:45.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.348+0000 7f6aae087640 1 -- 172.21.15.76:0/825148835 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6a9406ae70 msgr2=0x7f6a9406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:45.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.348+0000 7f6aae087640 1 --2- 172.21.15.76:0/825148835 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6a9406ae70 0x7f6a9406d330 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f6a98006fd0 tx=0x7f6a98008040 comp rx=0 tx=0).stop 2023-11-09T03:54:45.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.348+0000 7f6aae087640 1 -- 172.21.15.76:0/825148835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aa8159e40 msgr2=0x7f6aa80c7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:45.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.348+0000 7f6aae087640 1 --2- 172.21.15.76:0/825148835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aa8159e40 0x7f6aa80c7d90 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000c7a0 tx=0x7f6aa000cc60 comp rx=0 tx=0).stop 2023-11-09T03:54:45.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.348+0000 7f6aae087640 1 -- 172.21.15.76:0/825148835 shutdown_connections 2023-11-09T03:54:45.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.348+0000 7f6aae087640 1 --2- 172.21.15.76:0/825148835 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f6a9406ae70 0x7f6a9406d330 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:45.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.348+0000 7f6aae087640 1 --2- 172.21.15.76:0/825148835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6aa8159e40 0x7f6aa80c7d90 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:45.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.348+0000 7f6aae087640 1 --2- 172.21.15.76:0/825148835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6aa806fff0 0x7f6aa80c3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:45.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.348+0000 7f6aae087640 1 --2- 172.21.15.76:0/825148835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6aa806bb60 0x7f6aa8153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:45.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.348+0000 7f6aae087640 1 -- 172.21.15.76:0/825148835 >> 172.21.15.76:0/825148835 conn(0x7f6aa80cb530 msgr2=0x7f6aa8070420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:45.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.348+0000 7f6aae087640 1 -- 172.21.15.76:0/825148835 shutdown_connections 2023-11-09T03:54:45.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.348+0000 7f6aae087640 1 -- 172.21.15.76:0/825148835 wait complete. 2023-11-09T03:54:45.362 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-09T03:54:45.362 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-09T03:54:45.363 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-09T03:54:45.363 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:54:45.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:54:45.364 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi113 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b' 2023-11-09T03:54:45.364 INFO:tasks.ceph.mds.b:Stopped 2023-11-09T03:54:45.365 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-09T03:54:45.368 INFO:tasks.ceph.mds.b:Started 2023-11-09T03:54:45.368 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:54:45.445 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2023-11-09T03:54:45.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.468+0000 7f187de32640 1 Processor -- start 2023-11-09T03:54:45.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.468+0000 7f187de32640 1 -- start start 2023-11-09T03:54:45.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.469+0000 7f187de32640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18780addd0 0x7f18780cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:45.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.469+0000 7f187de32640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f18780cc680 0x7f18780ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:45.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.469+0000 7f187de32640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18780cd720 con 0x7f187806bb10 2023-11-09T03:54:45.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.469+0000 7f187de32640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18780ae1b0 con 0x7f18780cc680 2023-11-09T03:54:45.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.469+0000 7f187de32640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18780ccb10 con 0x7f18780addd0 2023-11-09T03:54:45.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.469+0000 7f18777fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18780addd0 0x7f18780cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:45.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.469+0000 7f1876ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f18780cc680 0x7f18780ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:45.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.469+0000 7f18777fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18780addd0 0x7f18780cdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:55172/0 (socket says 172.21.15.76:55172) 2023-11-09T03:54:45.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.469+0000 7f18777fe640 1 -- 172.21.15.76:0/3161519793 learned_addr learned my addr 172.21.15.76:0/3161519793 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:45.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.469+0000 7f1877fff640 1 --1- 172.21.15.76:0/3161519793 >> v1:172.21.15.113:6789/0 conn(0x7f187806bb10 0x7f187806f8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:46342/0 (socket says 172.21.15.76:46342) 2023-11-09T03:54:45.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.469+0000 7f18767fc640 1 -- 172.21.15.76:0/3161519793 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1706268994 0 0) 0x7f18780cd720 con 0x7f187806bb10 2023-11-09T03:54:45.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.470+0000 7f18767fc640 1 -- 172.21.15.76:0/3161519793 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1860003520 con 0x7f187806bb10 2023-11-09T03:54:45.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.470+0000 7f18777fe640 1 -- 172.21.15.76:0/3161519793 >> v2:172.21.15.113:3300/0 conn(0x7f18780cc680 msgr2=0x7f18780ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:45.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.470+0000 7f18777fe640 1 --2- 172.21.15.76:0/3161519793 >> v2:172.21.15.113:3300/0 conn(0x7f18780cc680 0x7f18780ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:45.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.470+0000 7f18777fe640 1 -- 172.21.15.76:0/3161519793 >> v1:172.21.15.113:6789/0 conn(0x7f187806bb10 legacy=0x7f187806f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:45.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.470+0000 7f18777fe640 1 -- 172.21.15.76:0/3161519793 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18780ccdf0 con 0x7f18780addd0 2023-11-09T03:54:45.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.470+0000 7f1876ffd640 1 --2- 172.21.15.76:0/3161519793 >> v2:172.21.15.113:3300/0 conn(0x7f18780cc680 0x7f18780ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:54:45.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.470+0000 7f18777fe640 1 --2- 172.21.15.76:0/3161519793 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18780addd0 0x7f18780cdbb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f186800aa10 tx=0x7f186802ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bf346eb719f9f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:45.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.470+0000 7f18767fc640 1 -- 172.21.15.76:0/3161519793 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186802dc10 con 0x7f18780addd0 2023-11-09T03:54:45.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.470+0000 7f18767fc640 1 -- 172.21.15.76:0/3161519793 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f186802dd70 con 0x7f18780addd0 2023-11-09T03:54:45.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.470+0000 7f18767fc640 1 -- 172.21.15.76:0/3161519793 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1868035660 con 0x7f18780addd0 2023-11-09T03:54:45.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.471+0000 7f187de32640 1 -- 172.21.15.76:0/3161519793 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18780addd0 msgr2=0x7f18780cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:45.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.471+0000 7f187de32640 1 --2- 172.21.15.76:0/3161519793 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18780addd0 0x7f18780cdbb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f186800aa10 tx=0x7f186802ce60 comp rx=0 tx=0).stop 2023-11-09T03:54:45.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.471+0000 7f187de32640 1 -- 172.21.15.76:0/3161519793 shutdown_connections 2023-11-09T03:54:45.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.471+0000 7f187de32640 1 --2- 172.21.15.76:0/3161519793 >> v2:172.21.15.113:3300/0 conn(0x7f18780cc680 0x7f18780ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:45.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.471+0000 7f187de32640 1 --2- 172.21.15.76:0/3161519793 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18780addd0 0x7f18780cdbb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:45.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.471+0000 7f187de32640 1 -- 172.21.15.76:0/3161519793 >> 172.21.15.76:0/3161519793 conn(0x7f18780cb530 msgr2=0x7f18780cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:45.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.471+0000 7f187de32640 1 -- 172.21.15.76:0/3161519793 shutdown_connections 2023-11-09T03:54:45.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.471+0000 7f187de32640 1 -- 172.21.15.76:0/3161519793 wait complete. 2023-11-09T03:54:45.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.472+0000 7f187de32640 1 Processor -- start 2023-11-09T03:54:45.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.472+0000 7f187de32640 1 -- start start 2023-11-09T03:54:45.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.473+0000 7f187de32640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f187806bb10 0x7f18780c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:45.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.473+0000 7f187de32640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18780addd0 0x7f18780c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:45.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.473+0000 7f187de32640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18781520c0 0x7f18781f6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:45.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.473+0000 7f187de32640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18780c5910 con 0x7f187806bb10 2023-11-09T03:54:45.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.473+0000 7f187de32640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18780c8bd0 con 0x7f18781520c0 2023-11-09T03:54:45.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.473+0000 7f187de32640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18780c65a0 con 0x7f18780addd0 2023-11-09T03:54:45.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.473+0000 7f18777fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f187806bb10 0x7f18780c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:45.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.473+0000 7f18777fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f187806bb10 0x7f18780c53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58086/0 (socket says 172.21.15.76:58086) 2023-11-09T03:54:45.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.473+0000 7f18777fe640 1 -- 172.21.15.76:0/1130655509 learned_addr learned my addr 172.21.15.76:0/1130655509 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:45.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.473+0000 7f1877fff640 1 --2- 172.21.15.76:0/1130655509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18781520c0 0x7f18781f6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:45.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.473+0000 7f1876ffd640 1 --2- 172.21.15.76:0/1130655509 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18780addd0 0x7f18780c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:45.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.473+0000 7f18777fe640 1 -- 172.21.15.76:0/1130655509 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18780addd0 msgr2=0x7f18780c8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:45.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.473+0000 7f18777fe640 1 --2- 172.21.15.76:0/1130655509 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18780addd0 0x7f18780c8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:45.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.473+0000 7f18777fe640 1 -- 172.21.15.76:0/1130655509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18781520c0 msgr2=0x7f18781f6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:45.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.473+0000 7f18777fe640 1 --2- 172.21.15.76:0/1130655509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18781520c0 0x7f18781f6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:45.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.473+0000 7f18777fe640 1 -- 172.21.15.76:0/1130655509 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186c009400 con 0x7f187806bb10 2023-11-09T03:54:45.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.474+0000 7f18777fe640 1 --2- 172.21.15.76:0/1130655509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f187806bb10 0x7f18780c53d0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f186802cea0 tx=0x7f18680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:45.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.474+0000 7f185ffff640 1 -- 172.21.15.76:0/1130655509 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186803f600 con 0x7f187806bb10 2023-11-09T03:54:45.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.474+0000 7f185ffff640 1 -- 172.21.15.76:0/1130655509 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f186803f780 con 0x7f187806bb10 2023-11-09T03:54:45.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.474+0000 7f187de32640 1 -- 172.21.15.76:0/1130655509 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f186800a670 con 0x7f187806bb10 2023-11-09T03:54:45.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.474+0000 7f185ffff640 1 -- 172.21.15.76:0/1130655509 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186803fb30 con 0x7f187806bb10 2023-11-09T03:54:45.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.474+0000 7f187de32640 1 -- 172.21.15.76:0/1130655509 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18780c6b60 con 0x7f187806bb10 2023-11-09T03:54:45.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.475+0000 7f187de32640 1 -- 172.21.15.76:0/1130655509 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18780684b0 con 0x7f187806bb10 2023-11-09T03:54:45.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.475+0000 7f185ffff640 1 -- 172.21.15.76:0/1130655509 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f186803fc90 con 0x7f187806bb10 2023-11-09T03:54:45.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.476+0000 7f185ffff640 1 --2- 172.21.15.76:0/1130655509 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f186406aea0 0x7f186406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:45.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.476+0000 7f185ffff640 1 -- 172.21.15.76:0/1130655509 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f18680ac3d0 con 0x7f187806bb10 2023-11-09T03:54:45.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.476+0000 7f1876ffd640 1 --2- 172.21.15.76:0/1130655509 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f186406aea0 0x7f186406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:45.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.476+0000 7f1876ffd640 1 --2- 172.21.15.76:0/1130655509 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f186406aea0 0x7f186406d360 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f187806c5b0 tx=0x7f186c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:45.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.480+0000 7f185ffff640 1 -- 172.21.15.76:0/1130655509 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1868076a70 con 0x7f187806bb10 2023-11-09T03:54:45.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.683+0000 7f187de32640 1 -- 172.21.15.76:0/1130655509 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18780ca7a0 con 0x7f187806bb10 2023-11-09T03:54:45.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.684+0000 7f185ffff640 1 -- 172.21.15.76:0/1130655509 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 129 v129) v1 ==== 95+0+4139 (secure 0 0 0) 0x7f186803a090 con 0x7f187806bb10 2023-11-09T03:54:45.685 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:54:45.686 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":129,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:45.350167+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":9024},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9024":{"gid":9024,"name":"a","rank":1,"incarnation":129,"state":"up:replay","state_seq":1,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:54:45.687 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 129 2023-11-09T03:54:45.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.686+0000 7f187de32640 1 -- 172.21.15.76:0/1130655509 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f186406aea0 msgr2=0x7f186406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:45.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.686+0000 7f187de32640 1 --2- 172.21.15.76:0/1130655509 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f186406aea0 0x7f186406d360 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f187806c5b0 tx=0x7f186c009190 comp rx=0 tx=0).stop 2023-11-09T03:54:45.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.686+0000 7f187de32640 1 -- 172.21.15.76:0/1130655509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f187806bb10 msgr2=0x7f18780c53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:45.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.686+0000 7f187de32640 1 --2- 172.21.15.76:0/1130655509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f187806bb10 0x7f18780c53d0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f186802cea0 tx=0x7f18680046c0 comp rx=0 tx=0).stop 2023-11-09T03:54:45.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.686+0000 7f187de32640 1 -- 172.21.15.76:0/1130655509 shutdown_connections 2023-11-09T03:54:45.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.686+0000 7f187de32640 1 --2- 172.21.15.76:0/1130655509 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f186406aea0 0x7f186406d360 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:45.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.686+0000 7f187de32640 1 --2- 172.21.15.76:0/1130655509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18781520c0 0x7f18781f6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:45.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.686+0000 7f187de32640 1 --2- 172.21.15.76:0/1130655509 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18780addd0 0x7f18780c8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:45.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.686+0000 7f187de32640 1 --2- 172.21.15.76:0/1130655509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f187806bb10 0x7f18780c53d0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:45.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.686+0000 7f187de32640 1 -- 172.21.15.76:0/1130655509 >> 172.21.15.76:0/1130655509 conn(0x7f18780cb530 msgr2=0x7f18780afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:45.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.686+0000 7f187de32640 1 -- 172.21.15.76:0/1130655509 shutdown_connections 2023-11-09T03:54:45.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:45.686+0000 7f187de32640 1 -- 172.21.15.76:0/1130655509 wait complete. 2023-11-09T03:54:45.697 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-11-09T03:54:50.699 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:54:50.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.808+0000 7f24eb426640 1 Processor -- start 2023-11-09T03:54:50.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.808+0000 7f24eb426640 1 -- start start 2023-11-09T03:54:50.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.809+0000 7f24eb426640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f24e40ade00 0x7f24e40b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:50.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.809+0000 7f24eb426640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24e406c5f0 0x7f24e40cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:50.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.809+0000 7f24eb426640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24e406cc90 con 0x7f24e406d9c0 2023-11-09T03:54:50.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.809+0000 7f24eb426640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24e40b0e60 con 0x7f24e40ade00 2023-11-09T03:54:50.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.809+0000 7f24eb426640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e406cee0 con 0x7f24e406c5f0 2023-11-09T03:54:50.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.809+0000 7f24e91c2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f24e40ade00 0x7f24e40b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:50.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.809+0000 7f24e89c1640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f24e406d9c0 0x7f24e40710d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:49044/0 (socket says 172.21.15.76:49044) 2023-11-09T03:54:50.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.809+0000 7f24e91c2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f24e40ade00 0x7f24e40b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58088/0 (socket says 172.21.15.76:58088) 2023-11-09T03:54:50.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.809+0000 7f24e89c1640 1 -- 172.21.15.76:0/2411827545 learned_addr learned my addr 172.21.15.76:0/2411827545 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:54:50.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.809+0000 7f24e99c3640 1 --2- 172.21.15.76:0/2411827545 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24e406c5f0 0x7f24e40cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:50.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.809+0000 7f24e91c2640 1 -- 172.21.15.76:0/2411827545 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24e406c5f0 msgr2=0x7f24e40cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:50.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.809+0000 7f24e91c2640 1 --2- 172.21.15.76:0/2411827545 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24e406c5f0 0x7f24e40cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:50.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.809+0000 7f24e91c2640 1 -- 172.21.15.76:0/2411827545 >> v1:172.21.15.76:6789/0 conn(0x7f24e406d9c0 legacy=0x7f24e40710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:50.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.809+0000 7f24e91c2640 1 -- 172.21.15.76:0/2411827545 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e406d1c0 con 0x7f24e40ade00 2023-11-09T03:54:50.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.809+0000 7f24d7fff640 1 -- 172.21.15.76:0/2411827545 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1841023717 0 0) 0x7f24e406cc90 con 0x7f24e406d9c0 2023-11-09T03:54:50.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.810+0000 7f24e91c2640 1 --2- 172.21.15.76:0/2411827545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24e40ade00 0x7f24e40b0530 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f24d80097a0 tx=0x7f24d802b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f02567d48f0f2e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:50.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.810+0000 7f24d7fff640 1 -- 172.21.15.76:0/2411827545 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d8031710 con 0x7f24e40ade00 2023-11-09T03:54:50.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.810+0000 7f24d7fff640 1 -- 172.21.15.76:0/2411827545 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d802f050 con 0x7f24e40ade00 2023-11-09T03:54:50.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.810+0000 7f24d7fff640 1 -- 172.21.15.76:0/2411827545 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d8031a90 con 0x7f24e40ade00 2023-11-09T03:54:50.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.810+0000 7f24eb426640 1 -- 172.21.15.76:0/2411827545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24e40ade00 msgr2=0x7f24e40b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:50.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.810+0000 7f24eb426640 1 --2- 172.21.15.76:0/2411827545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24e40ade00 0x7f24e40b0530 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f24d80097a0 tx=0x7f24d802b7b0 comp rx=0 tx=0).stop 2023-11-09T03:54:50.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.810+0000 7f24eb426640 1 -- 172.21.15.76:0/2411827545 shutdown_connections 2023-11-09T03:54:50.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.810+0000 7f24eb426640 1 --2- 172.21.15.76:0/2411827545 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24e406c5f0 0x7f24e40cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:50.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.810+0000 7f24eb426640 1 --2- 172.21.15.76:0/2411827545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24e40ade00 0x7f24e40b0530 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:50.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.810+0000 7f24eb426640 1 -- 172.21.15.76:0/2411827545 >> 172.21.15.76:0/2411827545 conn(0x7f24e40cb530 msgr2=0x7f24e40cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:50.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.811+0000 7f24eb426640 1 -- 172.21.15.76:0/2411827545 shutdown_connections 2023-11-09T03:54:50.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.811+0000 7f24eb426640 1 -- 172.21.15.76:0/2411827545 wait complete. 2023-11-09T03:54:50.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.812+0000 7f24eb426640 1 Processor -- start 2023-11-09T03:54:50.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.812+0000 7f24eb426640 1 -- start start 2023-11-09T03:54:50.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.812+0000 7f24eb426640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24e406c5f0 0x7f24e4152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:50.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.812+0000 7f24eb426640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24e406d9c0 0x7f24e40a8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:50.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.812+0000 7f24eb426640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24e40ade00 0x7f24e40a91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:50.824 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.812+0000 7f24eb426640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e40ac6a0 con 0x7f24e406d9c0 2023-11-09T03:54:50.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.812+0000 7f24eb426640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e40ac980 con 0x7f24e40ade00 2023-11-09T03:54:50.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.812+0000 7f24eb426640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e40acc60 con 0x7f24e406c5f0 2023-11-09T03:54:50.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.813+0000 7f24e89c1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24e406d9c0 0x7f24e40a8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:50.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.813+0000 7f24e89c1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24e406d9c0 0x7f24e40a8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58094/0 (socket says 172.21.15.76:58094) 2023-11-09T03:54:50.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.813+0000 7f24e89c1640 1 -- 172.21.15.76:0/1374618220 learned_addr learned my addr 172.21.15.76:0/1374618220 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:50.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.813+0000 7f24e99c3640 1 --2- 172.21.15.76:0/1374618220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24e40ade00 0x7f24e40a91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:50.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.813+0000 7f24e91c2640 1 --2- 172.21.15.76:0/1374618220 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24e406c5f0 0x7f24e4152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:50.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.813+0000 7f24e89c1640 1 -- 172.21.15.76:0/1374618220 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24e406c5f0 msgr2=0x7f24e4152010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:50.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.813+0000 7f24e89c1640 1 --2- 172.21.15.76:0/1374618220 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24e406c5f0 0x7f24e4152010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:50.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.813+0000 7f24e89c1640 1 -- 172.21.15.76:0/1374618220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24e40ade00 msgr2=0x7f24e40a91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:50.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.813+0000 7f24e89c1640 1 --2- 172.21.15.76:0/1374618220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24e40ade00 0x7f24e40a91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:50.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.813+0000 7f24e89c1640 1 -- 172.21.15.76:0/1374618220 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d8009400 con 0x7f24e406d9c0 2023-11-09T03:54:50.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.813+0000 7f24e89c1640 1 --2- 172.21.15.76:0/1374618220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24e406d9c0 0x7f24e40a8cb0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f24e000c410 tx=0x7f24e0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:50.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.813+0000 7f24d5ffb640 1 -- 172.21.15.76:0/1374618220 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e00074e0 con 0x7f24e406d9c0 2023-11-09T03:54:50.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.813+0000 7f24d5ffb640 1 -- 172.21.15.76:0/1374618220 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e0007660 con 0x7f24e406d9c0 2023-11-09T03:54:50.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.813+0000 7f24eb426640 1 -- 172.21.15.76:0/1374618220 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e40acf40 con 0x7f24e406d9c0 2023-11-09T03:54:50.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.813+0000 7f24d5ffb640 1 -- 172.21.15.76:0/1374618220 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e0004a20 con 0x7f24e406d9c0 2023-11-09T03:54:50.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.813+0000 7f24eb426640 1 -- 172.21.15.76:0/1374618220 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e40ad740 con 0x7f24e406d9c0 2023-11-09T03:54:50.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.814+0000 7f24eb426640 1 -- 172.21.15.76:0/1374618220 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24e40684b0 con 0x7f24e406d9c0 2023-11-09T03:54:50.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.815+0000 7f24d5ffb640 1 -- 172.21.15.76:0/1374618220 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f24e0004b80 con 0x7f24e406d9c0 2023-11-09T03:54:50.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.815+0000 7f24d5ffb640 1 --2- 172.21.15.76:0/1374618220 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f24c806ae70 0x7f24c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:50.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.815+0000 7f24d5ffb640 1 -- 172.21.15.76:0/1374618220 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f24e0089ee0 con 0x7f24e406d9c0 2023-11-09T03:54:50.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.816+0000 7f24e91c2640 1 --2- 172.21.15.76:0/1374618220 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f24c806ae70 0x7f24c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:50.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.816+0000 7f24e91c2640 1 --2- 172.21.15.76:0/1374618220 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f24c806ae70 0x7f24c806d330 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f24d8035000 tx=0x7f24d8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:50.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:50.819+0000 7f24d5ffb640 1 -- 172.21.15.76:0/1374618220 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f24e00544d0 con 0x7f24e406d9c0 2023-11-09T03:54:51.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.022+0000 7f24eb426640 1 -- 172.21.15.76:0/1374618220 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24e40cddc0 con 0x7f24e406d9c0 2023-11-09T03:54:51.023 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.022+0000 7f24d5ffb640 1 -- 172.21.15.76:0/1374618220 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+4935 (secure 0 0 0) 0x7f24e0057af0 con 0x7f24e406d9c0 2023-11-09T03:54:51.024 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:54:51.024 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":131,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6698,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:46.356076+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":9024},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9024":{"gid":9024,"name":"a","rank":1,"incarnation":129,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:54:51.026 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 131 2023-11-09T03:54:51.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.024+0000 7f24eb426640 1 -- 172.21.15.76:0/1374618220 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f24c806ae70 msgr2=0x7f24c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:51.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.024+0000 7f24eb426640 1 --2- 172.21.15.76:0/1374618220 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f24c806ae70 0x7f24c806d330 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f24d8035000 tx=0x7f24d8008030 comp rx=0 tx=0).stop 2023-11-09T03:54:51.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.024+0000 7f24eb426640 1 -- 172.21.15.76:0/1374618220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24e406d9c0 msgr2=0x7f24e40a8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:51.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.024+0000 7f24eb426640 1 --2- 172.21.15.76:0/1374618220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24e406d9c0 0x7f24e40a8cb0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f24e000c410 tx=0x7f24e0001e70 comp rx=0 tx=0).stop 2023-11-09T03:54:51.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.024+0000 7f24eb426640 1 -- 172.21.15.76:0/1374618220 shutdown_connections 2023-11-09T03:54:51.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.024+0000 7f24eb426640 1 --2- 172.21.15.76:0/1374618220 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f24c806ae70 0x7f24c806d330 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:51.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.024+0000 7f24eb426640 1 --2- 172.21.15.76:0/1374618220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24e40ade00 0x7f24e40a91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:51.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.024+0000 7f24eb426640 1 --2- 172.21.15.76:0/1374618220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24e406d9c0 0x7f24e40a8cb0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:51.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.024+0000 7f24eb426640 1 --2- 172.21.15.76:0/1374618220 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24e406c5f0 0x7f24e4152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:51.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.024+0000 7f24eb426640 1 -- 172.21.15.76:0/1374618220 >> 172.21.15.76:0/1374618220 conn(0x7f24e40cb530 msgr2=0x7f24e40b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:51.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.024+0000 7f24eb426640 1 -- 172.21.15.76:0/1374618220 shutdown_connections 2023-11-09T03:54:51.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.025+0000 7f24eb426640 1 -- 172.21.15.76:0/1374618220 wait complete. 2023-11-09T03:54:51.036 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-09T03:54:51.037 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:54:51.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.136+0000 7f645c6da640 1 Processor -- start 2023-11-09T03:54:51.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.136+0000 7f645c6da640 1 -- start start 2023-11-09T03:54:51.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.137+0000 7f645c6da640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64540b0c40 0x7f645406fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:51.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.137+0000 7f645c6da640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f645406bb60 0x7f6454070530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:51.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.137+0000 7f645c6da640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6454071500 con 0x7f6454159e40 2023-11-09T03:54:51.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.137+0000 7f645c6da640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6454070a70 con 0x7f645406bb60 2023-11-09T03:54:51.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.137+0000 7f645c6da640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645415dae0 con 0x7f64540b0c40 2023-11-09T03:54:51.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.137+0000 7f645a476640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f6454159e40 0x7f64540b0530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:49048/0 (socket says 172.21.15.76:49048) 2023-11-09T03:54:51.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.137+0000 7f645a476640 1 -- 172.21.15.76:0/3827439141 learned_addr learned my addr 172.21.15.76:0/3827439141 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:54:51.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.137+0000 7f6459474640 1 -- 172.21.15.76:0/3827439141 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1689679813 0 0) 0x7f6454071500 con 0x7f6454159e40 2023-11-09T03:54:51.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.137+0000 7f6459474640 1 -- 172.21.15.76:0/3827439141 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6440003520 con 0x7f6454159e40 2023-11-09T03:54:51.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.137+0000 7f645ac77640 1 --2- 172.21.15.76:0/3827439141 >> v2:172.21.15.113:3300/0 conn(0x7f645406bb60 0x7f6454070530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:51.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.137+0000 7f6459c75640 1 --2- 172.21.15.76:0/3827439141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64540b0c40 0x7f645406fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:51.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.137+0000 7f6459474640 1 -- 172.21.15.76:0/3827439141 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4263371660 0 0) 0x7f6440003520 con 0x7f6454159e40 2023-11-09T03:54:51.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.137+0000 7f6459474640 1 -- 172.21.15.76:0/3827439141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64540b0c40 msgr2=0x7f645406fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:51.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.137+0000 7f6459474640 1 --2- 172.21.15.76:0/3827439141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64540b0c40 0x7f645406fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:51.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.137+0000 7f6459474640 1 -- 172.21.15.76:0/3827439141 >> v2:172.21.15.113:3300/0 conn(0x7f645406bb60 msgr2=0x7f6454070530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:51.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.137+0000 7f6459474640 1 --2- 172.21.15.76:0/3827439141 >> v2:172.21.15.113:3300/0 conn(0x7f645406bb60 0x7f6454070530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:51.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.137+0000 7f6459474640 1 -- 172.21.15.76:0/3827439141 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6454070c60 con 0x7f6454159e40 2023-11-09T03:54:51.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.137+0000 7f6459474640 1 -- 172.21.15.76:0/3827439141 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f6450001670 con 0x7f6454159e40 2023-11-09T03:54:51.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.137+0000 7f6459474640 1 -- 172.21.15.76:0/3827439141 >> v1:172.21.15.76:6789/0 conn(0x7f6454159e40 legacy=0x7f64540b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:51.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.137+0000 7f6459474640 1 --2- 172.21.15.76:0/3827439141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64540b0c40 0x7f6440003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:51.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.137+0000 7f6459474640 1 --2- 172.21.15.76:0/3827439141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6440003ef0 0x7f64400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:51.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.137+0000 7f6459474640 1 --2- 172.21.15.76:0/3827439141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6440004d00 0x7f6440005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:51.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.137+0000 7f6459474640 1 -- 172.21.15.76:0/3827439141 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645415dae0 con 0x7f6440003ef0 2023-11-09T03:54:51.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.137+0000 7f6459474640 1 -- 172.21.15.76:0/3827439141 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644000b980 con 0x7f6440004d00 2023-11-09T03:54:51.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.137+0000 7f6459474640 1 -- 172.21.15.76:0/3827439141 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f644000bb10 con 0x7f64540b0c40 2023-11-09T03:54:51.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.138+0000 7f6459c75640 1 --2- 172.21.15.76:0/3827439141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6440003ef0 0x7f64400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:51.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.138+0000 7f645a476640 1 --2- 172.21.15.76:0/3827439141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64540b0c40 0x7f6440003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:51.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.138+0000 7f645ac77640 1 --2- 172.21.15.76:0/3827439141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6440004d00 0x7f6440005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:51.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.138+0000 7f6459c75640 1 -- 172.21.15.76:0/3827439141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64540b0c40 msgr2=0x7f6440003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:51.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.138+0000 7f6459c75640 1 --2- 172.21.15.76:0/3827439141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64540b0c40 0x7f6440003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:51.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.138+0000 7f6459c75640 1 -- 172.21.15.76:0/3827439141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6440004d00 msgr2=0x7f6440005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:51.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.138+0000 7f6459c75640 1 --2- 172.21.15.76:0/3827439141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6440004d00 0x7f6440005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:51.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.138+0000 7f6459c75640 1 -- 172.21.15.76:0/3827439141 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6440003520 con 0x7f6440003ef0 2023-11-09T03:54:51.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.138+0000 7f645ac77640 1 --2- 172.21.15.76:0/3827439141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6440004d00 0x7f6440005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:54:51.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.138+0000 7f645a476640 1 --2- 172.21.15.76:0/3827439141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f64540b0c40 0x7f6440003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:54:51.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.138+0000 7f6459c75640 1 --2- 172.21.15.76:0/3827439141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6440003ef0 0x7f64400047c0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f6444009af0 tx=0x7f64400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdfbf9747b801cf4 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:51.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.139+0000 7f6459474640 1 -- 172.21.15.76:0/3827439141 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644402c6b0 con 0x7f6440003ef0 2023-11-09T03:54:51.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.139+0000 7f6459474640 1 -- 172.21.15.76:0/3827439141 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644402c810 con 0x7f6440003ef0 2023-11-09T03:54:51.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.139+0000 7f6459474640 1 -- 172.21.15.76:0/3827439141 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644402cba0 con 0x7f6440003ef0 2023-11-09T03:54:51.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.139+0000 7f645c6da640 1 -- 172.21.15.76:0/3827439141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6440003ef0 msgr2=0x7f64400047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:51.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.139+0000 7f645c6da640 1 --2- 172.21.15.76:0/3827439141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6440003ef0 0x7f64400047c0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f6444009af0 tx=0x7f64400057f0 comp rx=0 tx=0).stop 2023-11-09T03:54:51.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.139+0000 7f645ac77640 1 -- 172.21.15.76:0/3827439141 reap_dead start 2023-11-09T03:54:51.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.139+0000 7f645c6da640 1 -- 172.21.15.76:0/3827439141 shutdown_connections 2023-11-09T03:54:51.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.139+0000 7f645c6da640 1 -- 172.21.15.76:0/3827439141 >> 172.21.15.76:0/3827439141 conn(0x7f64540cb530 msgr2=0x7f64540cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:51.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.139+0000 7f645c6da640 1 -- 172.21.15.76:0/3827439141 shutdown_connections 2023-11-09T03:54:51.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.139+0000 7f645c6da640 1 -- 172.21.15.76:0/3827439141 wait complete. 2023-11-09T03:54:51.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.140+0000 7f645c6da640 1 Processor -- start 2023-11-09T03:54:51.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.140+0000 7f645c6da640 1 -- start start 2023-11-09T03:54:51.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.141+0000 7f645c6da640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6440003ef0 0x7f645414e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:51.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.141+0000 7f645c6da640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6454152310 0x7f64541f0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:51.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.141+0000 7f645c6da640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6454154430 0x7f64540af120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:51.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.141+0000 7f645c6da640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64541f1520 con 0x7f6454152310 2023-11-09T03:54:51.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.141+0000 7f645c6da640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64541f0530 con 0x7f6440003ef0 2023-11-09T03:54:51.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.141+0000 7f645c6da640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6454154d80 con 0x7f6454154430 2023-11-09T03:54:51.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.141+0000 7f6459c75640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6454152310 0x7f64541f0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:51.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.141+0000 7f6459c75640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6454152310 0x7f64541f0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58120/0 (socket says 172.21.15.76:58120) 2023-11-09T03:54:51.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.141+0000 7f6459c75640 1 -- 172.21.15.76:0/11870585 learned_addr learned my addr 172.21.15.76:0/11870585 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:51.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.141+0000 7f645a476640 1 --2- 172.21.15.76:0/11870585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6440003ef0 0x7f645414e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:51.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.141+0000 7f645ac77640 1 --2- 172.21.15.76:0/11870585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6454154430 0x7f64540af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:51.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.141+0000 7f6459c75640 1 -- 172.21.15.76:0/11870585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6454154430 msgr2=0x7f64540af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:51.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.141+0000 7f6459c75640 1 --2- 172.21.15.76:0/11870585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6454154430 0x7f64540af120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:51.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.141+0000 7f6459c75640 1 -- 172.21.15.76:0/11870585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6440003ef0 msgr2=0x7f645414e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:51.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.141+0000 7f6459c75640 1 --2- 172.21.15.76:0/11870585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6440003ef0 0x7f645414e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:51.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.141+0000 7f6459c75640 1 -- 172.21.15.76:0/11870585 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6454155060 con 0x7f6454152310 2023-11-09T03:54:51.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.141+0000 7f645a476640 1 --2- 172.21.15.76:0/11870585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6440003ef0 0x7f645414e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:54:51.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.142+0000 7f6459c75640 1 --2- 172.21.15.76:0/11870585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6454152310 0x7f64541f0fe0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f6444037040 tx=0x7f6444004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:51.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.142+0000 7f644affd640 1 -- 172.21.15.76:0/11870585 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6444003810 con 0x7f6454152310 2023-11-09T03:54:51.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.142+0000 7f644affd640 1 -- 172.21.15.76:0/11870585 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6444003970 con 0x7f6454152310 2023-11-09T03:54:51.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.142+0000 7f645c6da640 1 -- 172.21.15.76:0/11870585 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64541f07e0 con 0x7f6454152310 2023-11-09T03:54:51.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.142+0000 7f644affd640 1 -- 172.21.15.76:0/11870585 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6444003ce0 con 0x7f6454152310 2023-11-09T03:54:51.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.142+0000 7f645c6da640 1 -- 172.21.15.76:0/11870585 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64541548c0 con 0x7f6454152310 2023-11-09T03:54:51.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.143+0000 7f645c6da640 1 -- 172.21.15.76:0/11870585 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6454154b10 con 0x7f6454152310 2023-11-09T03:54:51.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.143+0000 7f644affd640 1 -- 172.21.15.76:0/11870585 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6444047450 con 0x7f6454152310 2023-11-09T03:54:51.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.144+0000 7f644affd640 1 --2- 172.21.15.76:0/11870585 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f643806aea0 0x7f643806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:51.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.144+0000 7f644affd640 1 -- 172.21.15.76:0/11870585 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f64440ac2b0 con 0x7f6454152310 2023-11-09T03:54:51.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.144+0000 7f645a476640 1 --2- 172.21.15.76:0/11870585 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f643806aea0 0x7f643806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:51.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.144+0000 7f645a476640 1 --2- 172.21.15.76:0/11870585 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f643806aea0 0x7f643806d360 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6450001cf0 tx=0x7f6450001c60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:51.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.148+0000 7f644affd640 1 -- 172.21.15.76:0/11870585 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6444076950 con 0x7f6454152310 2023-11-09T03:54:51.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.347+0000 7f645c6da640 1 -- 172.21.15.76:0/11870585 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64540684b0 con 0x7f6454152310 2023-11-09T03:54:51.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.347+0000 7f644affd640 1 -- 172.21.15.76:0/11870585 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+4935 (secure 0 0 0) 0x7f64440044e0 con 0x7f6454152310 2023-11-09T03:54:51.349 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:54:51.349 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":131,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6698,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:46.356076+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":9024},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9024":{"gid":9024,"name":"a","rank":1,"incarnation":129,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:54:51.350 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 131 2023-11-09T03:54:51.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.349+0000 7f645c6da640 1 -- 172.21.15.76:0/11870585 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f643806aea0 msgr2=0x7f643806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:51.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.349+0000 7f645c6da640 1 --2- 172.21.15.76:0/11870585 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f643806aea0 0x7f643806d360 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6450001cf0 tx=0x7f6450001c60 comp rx=0 tx=0).stop 2023-11-09T03:54:51.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.349+0000 7f645c6da640 1 -- 172.21.15.76:0/11870585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6454152310 msgr2=0x7f64541f0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:51.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.349+0000 7f645c6da640 1 --2- 172.21.15.76:0/11870585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6454152310 0x7f64541f0fe0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f6444037040 tx=0x7f6444004730 comp rx=0 tx=0).stop 2023-11-09T03:54:51.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.349+0000 7f645c6da640 1 -- 172.21.15.76:0/11870585 shutdown_connections 2023-11-09T03:54:51.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.349+0000 7f645c6da640 1 --2- 172.21.15.76:0/11870585 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f643806aea0 0x7f643806d360 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:51.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.349+0000 7f645c6da640 1 --2- 172.21.15.76:0/11870585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6454154430 0x7f64540af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:51.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.349+0000 7f645c6da640 1 --2- 172.21.15.76:0/11870585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6454152310 0x7f64541f0fe0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:51.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.349+0000 7f645c6da640 1 --2- 172.21.15.76:0/11870585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6440003ef0 0x7f645414e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:51.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.349+0000 7f645c6da640 1 -- 172.21.15.76:0/11870585 >> 172.21.15.76:0/11870585 conn(0x7f64540cb530 msgr2=0x7f64540ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:51.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.349+0000 7f645c6da640 1 -- 172.21.15.76:0/11870585 shutdown_connections 2023-11-09T03:54:51.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:51.349+0000 7f645c6da640 1 -- 172.21.15.76:0/11870585 wait complete. 2023-11-09T03:54:51.764 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 131, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-09T03:48:57.383848+0000', 'modified': '2023-11-09T03:54:46.356076+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 117, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 8568, 'mds_1': 9024}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8568': {'gid': 8568, 'name': 'c', 'rank': 0, 'incarnation': 110, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.76:6833/153465374', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6832', 'nonce': 153465374}, {'type': 'v1', 'addr': '172.21.15.76:6833', 'nonce': 153465374}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_9024': {'gid': 9024, 'name': 'a', 'rank': 1, 'incarnation': 129, 'state': 'up:reconnect', 'state_seq': 12, 'addr': '172.21.15.76:6835/757529522', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 757529522}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 757529522}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-09T03:54:51.765 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9024:up:reconnect 2023-11-09T03:54:52.039 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-11-09T03:54:52.766 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:54:52.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.879+0000 7f9ffa61a640 1 Processor -- start 2023-11-09T03:54:52.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.879+0000 7f9ffa61a640 1 -- start start 2023-11-09T03:54:52.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.880+0000 7f9ffa61a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ff40ade00 0x7f9ff40b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:52.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.880+0000 7f9ffa61a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9ff406d9c0 0x7f9ff40710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:52.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.880+0000 7f9ffa61a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9ff406c5f0 0x7f9ff4071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:52.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.880+0000 7f9ffa61a640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ff406cc90 con 0x7f9ff406c5f0 2023-11-09T03:54:52.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.880+0000 7f9ffa61a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ff4071b50 con 0x7f9ff406d9c0 2023-11-09T03:54:52.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.880+0000 7f9ffa61a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff40b0e60 con 0x7f9ff40ade00 2023-11-09T03:54:52.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.880+0000 7f9ff8bb7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9ff406c5f0 0x7f9ff4071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:52.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.880+0000 7f9ff8bb7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9ff406c5f0 0x7f9ff4071610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40890/0 (socket says 172.21.15.76:40890) 2023-11-09T03:54:52.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.880+0000 7f9ff8bb7640 1 -- 172.21.15.76:0/265400355 learned_addr learned my addr 172.21.15.76:0/265400355 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:52.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.880+0000 7f9ff3fff640 1 --2- 172.21.15.76:0/265400355 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ff40ade00 0x7f9ff40b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:52.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.880+0000 7f9ff37fe640 1 --2- 172.21.15.76:0/265400355 >> v2:172.21.15.113:3300/0 conn(0x7f9ff406d9c0 0x7f9ff40710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:52.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.880+0000 7f9ff8bb7640 1 -- 172.21.15.76:0/265400355 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ff40ade00 msgr2=0x7f9ff40b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:52.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.880+0000 7f9ff8bb7640 1 --2- 172.21.15.76:0/265400355 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ff40ade00 0x7f9ff40b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:52.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.880+0000 7f9ff8bb7640 1 -- 172.21.15.76:0/265400355 >> v2:172.21.15.113:3300/0 conn(0x7f9ff406d9c0 msgr2=0x7f9ff40710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:52.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.880+0000 7f9ff8bb7640 1 --2- 172.21.15.76:0/265400355 >> v2:172.21.15.113:3300/0 conn(0x7f9ff406d9c0 0x7f9ff40710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:52.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.880+0000 7f9ff8bb7640 1 -- 172.21.15.76:0/265400355 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff406cee0 con 0x7f9ff406c5f0 2023-11-09T03:54:52.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.881+0000 7f9ff8bb7640 1 --2- 172.21.15.76:0/265400355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ff406c5f0 0x7f9ff4071610 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f9ff40cb390 tx=0x7f9fec02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8645833e65d63b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:52.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.881+0000 7f9ff2ffd640 1 -- 172.21.15.76:0/265400355 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec02c950 con 0x7f9ff406c5f0 2023-11-09T03:54:52.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.881+0000 7f9ff2ffd640 1 -- 172.21.15.76:0/265400355 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fec02cab0 con 0x7f9ff406c5f0 2023-11-09T03:54:52.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.881+0000 7f9ff2ffd640 1 -- 172.21.15.76:0/265400355 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec02ce40 con 0x7f9ff406c5f0 2023-11-09T03:54:52.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.882+0000 7f9ffa61a640 1 -- 172.21.15.76:0/265400355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ff406c5f0 msgr2=0x7f9ff4071610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:52.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.882+0000 7f9ffa61a640 1 --2- 172.21.15.76:0/265400355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ff406c5f0 0x7f9ff4071610 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f9ff40cb390 tx=0x7f9fec02bdb0 comp rx=0 tx=0).stop 2023-11-09T03:54:52.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.882+0000 7f9ffa61a640 1 -- 172.21.15.76:0/265400355 shutdown_connections 2023-11-09T03:54:52.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.882+0000 7f9ffa61a640 1 --2- 172.21.15.76:0/265400355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ff406c5f0 0x7f9ff4071610 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:52.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.882+0000 7f9ffa61a640 1 --2- 172.21.15.76:0/265400355 >> v2:172.21.15.113:3300/0 conn(0x7f9ff406d9c0 0x7f9ff40710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:52.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.882+0000 7f9ffa61a640 1 --2- 172.21.15.76:0/265400355 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ff40ade00 0x7f9ff40b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:52.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.882+0000 7f9ffa61a640 1 -- 172.21.15.76:0/265400355 >> 172.21.15.76:0/265400355 conn(0x7f9ff40cb530 msgr2=0x7f9ff40cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:52.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.882+0000 7f9ffa61a640 1 -- 172.21.15.76:0/265400355 shutdown_connections 2023-11-09T03:54:52.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.882+0000 7f9ffa61a640 1 -- 172.21.15.76:0/265400355 wait complete. 2023-11-09T03:54:52.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.883+0000 7f9ffa61a640 1 Processor -- start 2023-11-09T03:54:52.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.883+0000 7f9ffa61a640 1 -- start start 2023-11-09T03:54:52.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.884+0000 7f9ffa61a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ff406c5f0 0x7f9ff40c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:52.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.884+0000 7f9ffa61a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ff406d9c0 0x7f9ff40c2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:52.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.884+0000 7f9ffa61a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ff40ade00 0x7f9ff40c3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:52.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.884+0000 7f9ffa61a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff40c8980 con 0x7f9ff406d9c0 2023-11-09T03:54:52.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.884+0000 7f9ffa61a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff4153920 con 0x7f9ff40ade00 2023-11-09T03:54:52.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.884+0000 7f9ffa61a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff40c2c20 con 0x7f9ff406c5f0 2023-11-09T03:54:52.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.884+0000 7f9ff37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ff406d9c0 0x7f9ff40c2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:52.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.884+0000 7f9ff37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ff406d9c0 0x7f9ff40c2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40904/0 (socket says 172.21.15.76:40904) 2023-11-09T03:54:52.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.884+0000 7f9ff37fe640 1 -- 172.21.15.76:0/2516800784 learned_addr learned my addr 172.21.15.76:0/2516800784 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:52.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.884+0000 7f9ff3fff640 1 --2- 172.21.15.76:0/2516800784 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ff406c5f0 0x7f9ff40c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:52.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.884+0000 7f9ff8bb7640 1 --2- 172.21.15.76:0/2516800784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ff40ade00 0x7f9ff40c3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:52.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.884+0000 7f9ff37fe640 1 -- 172.21.15.76:0/2516800784 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ff406c5f0 msgr2=0x7f9ff40c5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:52.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.884+0000 7f9ff37fe640 1 --2- 172.21.15.76:0/2516800784 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ff406c5f0 0x7f9ff40c5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:52.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.884+0000 7f9ff37fe640 1 -- 172.21.15.76:0/2516800784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ff40ade00 msgr2=0x7f9ff40c3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:52.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.884+0000 7f9ff37fe640 1 --2- 172.21.15.76:0/2516800784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ff40ade00 0x7f9ff40c3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:52.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.884+0000 7f9ff37fe640 1 -- 172.21.15.76:0/2516800784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fec009b30 con 0x7f9ff406d9c0 2023-11-09T03:54:52.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.884+0000 7f9ff8bb7640 1 --2- 172.21.15.76:0/2516800784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ff40ade00 0x7f9ff40c3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:54:52.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.885+0000 7f9ff37fe640 1 --2- 172.21.15.76:0/2516800784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ff406d9c0 0x7f9ff40c2f00 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800d5d0 tx=0x7f9fe800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:52.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.885+0000 7f9ff0ff9640 1 -- 172.21.15.76:0/2516800784 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8004240 con 0x7f9ff406d9c0 2023-11-09T03:54:52.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.885+0000 7f9ff0ff9640 1 -- 172.21.15.76:0/2516800784 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe80043a0 con 0x7f9ff406d9c0 2023-11-09T03:54:52.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.885+0000 7f9ff0ff9640 1 -- 172.21.15.76:0/2516800784 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8003830 con 0x7f9ff406d9c0 2023-11-09T03:54:52.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.885+0000 7f9ffa61a640 1 -- 172.21.15.76:0/2516800784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff40c8330 con 0x7f9ff406d9c0 2023-11-09T03:54:52.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.885+0000 7f9ffa61a640 1 -- 172.21.15.76:0/2516800784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff40c27d0 con 0x7f9ff406d9c0 2023-11-09T03:54:52.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.886+0000 7f9ffa61a640 1 -- 172.21.15.76:0/2516800784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff40684b0 con 0x7f9ff406d9c0 2023-11-09T03:54:52.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.886+0000 7f9ff0ff9640 1 -- 172.21.15.76:0/2516800784 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9fe8003990 con 0x7f9ff406d9c0 2023-11-09T03:54:52.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.887+0000 7f9ff0ff9640 1 --2- 172.21.15.76:0/2516800784 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9fe006aea0 0x7f9fe006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:52.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.887+0000 7f9ff0ff9640 1 -- 172.21.15.76:0/2516800784 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f9fe808a2a0 con 0x7f9ff406d9c0 2023-11-09T03:54:52.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.887+0000 7f9ff3fff640 1 --2- 172.21.15.76:0/2516800784 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9fe006aea0 0x7f9fe006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:52.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.887+0000 7f9ff3fff640 1 --2- 172.21.15.76:0/2516800784 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9fe006aea0 0x7f9fe006d360 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4006fd0 tx=0x7f9fe4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:52.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:52.891+0000 7f9ff0ff9640 1 -- 172.21.15.76:0/2516800784 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9fe8054890 con 0x7f9ff406d9c0 2023-11-09T03:54:53.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:53.092+0000 7f9ffa61a640 1 -- 172.21.15.76:0/2516800784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ff40cddc0 con 0x7f9ff406d9c0 2023-11-09T03:54:53.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:53.092+0000 7f9ff0ff9640 1 -- 172.21.15.76:0/2516800784 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+4935 (secure 0 0 0) 0x7f9fe8057eb0 con 0x7f9ff406d9c0 2023-11-09T03:54:53.094 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:54:53.094 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":131,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6698,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:46.356076+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":9024},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9024":{"gid":9024,"name":"a","rank":1,"incarnation":129,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:54:53.096 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 131 2023-11-09T03:54:53.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:53.094+0000 7f9ffa61a640 1 -- 172.21.15.76:0/2516800784 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9fe006aea0 msgr2=0x7f9fe006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:53.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:53.094+0000 7f9ffa61a640 1 --2- 172.21.15.76:0/2516800784 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9fe006aea0 0x7f9fe006d360 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4006fd0 tx=0x7f9fe4008040 comp rx=0 tx=0).stop 2023-11-09T03:54:53.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:53.094+0000 7f9ffa61a640 1 -- 172.21.15.76:0/2516800784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ff406d9c0 msgr2=0x7f9ff40c2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:53.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:53.094+0000 7f9ffa61a640 1 --2- 172.21.15.76:0/2516800784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ff406d9c0 0x7f9ff40c2f00 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800d5d0 tx=0x7f9fe800da90 comp rx=0 tx=0).stop 2023-11-09T03:54:53.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:53.094+0000 7f9ffa61a640 1 -- 172.21.15.76:0/2516800784 shutdown_connections 2023-11-09T03:54:53.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:53.094+0000 7f9ffa61a640 1 --2- 172.21.15.76:0/2516800784 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9fe006aea0 0x7f9fe006d360 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:53.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:53.094+0000 7f9ffa61a640 1 --2- 172.21.15.76:0/2516800784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ff40ade00 0x7f9ff40c3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:53.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:53.094+0000 7f9ffa61a640 1 --2- 172.21.15.76:0/2516800784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ff406d9c0 0x7f9ff40c2f00 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:53.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:53.094+0000 7f9ffa61a640 1 --2- 172.21.15.76:0/2516800784 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ff406c5f0 0x7f9ff40c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:53.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:53.094+0000 7f9ffa61a640 1 -- 172.21.15.76:0/2516800784 >> 172.21.15.76:0/2516800784 conn(0x7f9ff40cb530 msgr2=0x7f9ff40cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:53.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:53.094+0000 7f9ffa61a640 1 -- 172.21.15.76:0/2516800784 shutdown_connections 2023-11-09T03:54:53.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:53.094+0000 7f9ffa61a640 1 -- 172.21.15.76:0/2516800784 wait complete. 2023-11-09T03:54:53.508 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 131, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-09T03:48:57.383848+0000', 'modified': '2023-11-09T03:54:46.356076+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 117, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 8568, 'mds_1': 9024}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8568': {'gid': 8568, 'name': 'c', 'rank': 0, 'incarnation': 110, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.76:6833/153465374', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6832', 'nonce': 153465374}, {'type': 'v1', 'addr': '172.21.15.76:6833', 'nonce': 153465374}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_9024': {'gid': 9024, 'name': 'a', 'rank': 1, 'incarnation': 129, 'state': 'up:reconnect', 'state_seq': 12, 'addr': '172.21.15.76:6835/757529522', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 757529522}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 757529522}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-09T03:54:53.509 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9024:up:reconnect 2023-11-09T03:54:54.510 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:54:54.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.616+0000 7f21ff2fc640 1 Processor -- start 2023-11-09T03:54:54.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.617+0000 7f21ff2fc640 1 -- start start 2023-11-09T03:54:54.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.617+0000 7f21ff2fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f21f815d4e0 0x7f21f80d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:54.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.617+0000 7f21ff2fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f21f80d3b60 0x7f21f80d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:54.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.617+0000 7f21ff2fc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f80b32b0 con 0x7f21f815d100 2023-11-09T03:54:54.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.617+0000 7f21ff2fc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21f80cec20 con 0x7f21f80d3b60 2023-11-09T03:54:54.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.617+0000 7f21ff2fc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21f80cee10 con 0x7f21f815d4e0 2023-11-09T03:54:54.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.617+0000 7f21fd899640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f21f80d3b60 0x7f21f80d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:54.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.618+0000 7f21fd899640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f21f80d3b60 0x7f21f80d1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40920/0 (socket says 172.21.15.76:40920) 2023-11-09T03:54:54.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.618+0000 7f21fd899640 1 -- 172.21.15.76:0/821755359 learned_addr learned my addr 172.21.15.76:0/821755359 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:54.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.618+0000 7f21fc897640 1 --2- 172.21.15.76:0/821755359 >> v2:172.21.15.113:3300/0 conn(0x7f21f815d4e0 0x7f21f80d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:54.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.618+0000 7f21fd899640 1 -- 172.21.15.76:0/821755359 >> v2:172.21.15.113:3300/0 conn(0x7f21f815d4e0 msgr2=0x7f21f80d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:54.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.618+0000 7f21fd899640 1 --2- 172.21.15.76:0/821755359 >> v2:172.21.15.113:3300/0 conn(0x7f21f815d4e0 0x7f21f80d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:54.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.618+0000 7f21fd899640 1 -- 172.21.15.76:0/821755359 >> v1:172.21.15.113:6789/0 conn(0x7f21f815d100 legacy=0x7f21f80ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:54.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.618+0000 7f21fd899640 1 -- 172.21.15.76:0/821755359 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f80b4750 con 0x7f21f80d3b60 2023-11-09T03:54:54.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.618+0000 7f21fc897640 1 --2- 172.21.15.76:0/821755359 >> v2:172.21.15.113:3300/0 conn(0x7f21f815d4e0 0x7f21f80d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:54:54.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.619+0000 7f21fd899640 1 --2- 172.21.15.76:0/821755359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21f80d3b60 0x7f21f80d1a10 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f21f8160da0 tx=0x7f21f002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b1292ecb4c6359 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:54.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.619+0000 7f21effff640 1 -- 172.21.15.76:0/821755359 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f002ca80 con 0x7f21f80d3b60 2023-11-09T03:54:54.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.619+0000 7f21effff640 1 -- 172.21.15.76:0/821755359 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f002cbe0 con 0x7f21f80d3b60 2023-11-09T03:54:54.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.619+0000 7f21effff640 1 -- 172.21.15.76:0/821755359 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f00343f0 con 0x7f21f80d3b60 2023-11-09T03:54:54.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.619+0000 7f21ff2fc640 1 -- 172.21.15.76:0/821755359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21f80d3b60 msgr2=0x7f21f80d1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:54.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.619+0000 7f21ff2fc640 1 --2- 172.21.15.76:0/821755359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21f80d3b60 0x7f21f80d1a10 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f21f8160da0 tx=0x7f21f002bee0 comp rx=0 tx=0).stop 2023-11-09T03:54:54.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.619+0000 7f21ff2fc640 1 -- 172.21.15.76:0/821755359 shutdown_connections 2023-11-09T03:54:54.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.619+0000 7f21ff2fc640 1 --2- 172.21.15.76:0/821755359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21f80d3b60 0x7f21f80d1a10 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:54.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.619+0000 7f21ff2fc640 1 --2- 172.21.15.76:0/821755359 >> v2:172.21.15.113:3300/0 conn(0x7f21f815d4e0 0x7f21f80d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:54.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.619+0000 7f21ff2fc640 1 -- 172.21.15.76:0/821755359 >> 172.21.15.76:0/821755359 conn(0x7f21f80d2ef0 msgr2=0x7f21f80d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:54.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.619+0000 7f21ff2fc640 1 -- 172.21.15.76:0/821755359 shutdown_connections 2023-11-09T03:54:54.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.619+0000 7f21ff2fc640 1 -- 172.21.15.76:0/821755359 wait complete. 2023-11-09T03:54:54.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.620+0000 7f21ff2fc640 1 Processor -- start 2023-11-09T03:54:54.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.621+0000 7f21ff2fc640 1 -- start start 2023-11-09T03:54:54.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.621+0000 7f21ff2fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21f80d3b60 0x7f21f815a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:54.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.621+0000 7f21ff2fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21f815d100 0x7f21f80c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:54.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.621+0000 7f21ff2fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21f815d4e0 0x7f21f806af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:54.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.621+0000 7f21ff2fc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f80cb590 con 0x7f21f80d3b60 2023-11-09T03:54:54.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.621+0000 7f21ff2fc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f815abf0 con 0x7f21f815d4e0 2023-11-09T03:54:54.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.621+0000 7f21ff2fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f80caa10 con 0x7f21f815d100 2023-11-09T03:54:54.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.621+0000 7f21fd098640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21f80d3b60 0x7f21f815a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:54.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.621+0000 7f21fd098640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21f80d3b60 0x7f21f815a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40926/0 (socket says 172.21.15.76:40926) 2023-11-09T03:54:54.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.621+0000 7f21fd098640 1 -- 172.21.15.76:0/481499535 learned_addr learned my addr 172.21.15.76:0/481499535 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:54.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.622+0000 7f21fd899640 1 --2- 172.21.15.76:0/481499535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21f815d4e0 0x7f21f806af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:54.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.622+0000 7f21fc897640 1 --2- 172.21.15.76:0/481499535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21f815d100 0x7f21f80c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:54.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.622+0000 7f21fd098640 1 -- 172.21.15.76:0/481499535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21f815d100 msgr2=0x7f21f80c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:54.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.622+0000 7f21fd098640 1 --2- 172.21.15.76:0/481499535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21f815d100 0x7f21f80c9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:54.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.622+0000 7f21fd098640 1 -- 172.21.15.76:0/481499535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21f815d4e0 msgr2=0x7f21f806af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:54.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.622+0000 7f21fd098640 1 --2- 172.21.15.76:0/481499535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21f815d4e0 0x7f21f806af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:54.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.622+0000 7f21fd098640 1 -- 172.21.15.76:0/481499535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f0009b30 con 0x7f21f80d3b60 2023-11-09T03:54:54.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.622+0000 7f21fd098640 1 --2- 172.21.15.76:0/481499535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21f80d3b60 0x7f21f815a6b0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f21f4004640 tx=0x7f21f400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:54.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.622+0000 7f21edffb640 1 -- 172.21.15.76:0/481499535 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f4011790 con 0x7f21f80d3b60 2023-11-09T03:54:54.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.622+0000 7f21edffb640 1 -- 172.21.15.76:0/481499535 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f40118f0 con 0x7f21f80d3b60 2023-11-09T03:54:54.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.622+0000 7f21ff2fc640 1 -- 172.21.15.76:0/481499535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21f80cacf0 con 0x7f21f80d3b60 2023-11-09T03:54:54.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.622+0000 7f21edffb640 1 -- 172.21.15.76:0/481499535 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f4011ca0 con 0x7f21f80d3b60 2023-11-09T03:54:54.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.622+0000 7f21ff2fc640 1 -- 172.21.15.76:0/481499535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f80cb2b0 con 0x7f21f80d3b60 2023-11-09T03:54:54.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.623+0000 7f21ff2fc640 1 -- 172.21.15.76:0/481499535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21f80684b0 con 0x7f21f80d3b60 2023-11-09T03:54:54.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.624+0000 7f21edffb640 1 -- 172.21.15.76:0/481499535 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f21f4012420 con 0x7f21f80d3b60 2023-11-09T03:54:54.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.624+0000 7f21edffb640 1 --2- 172.21.15.76:0/481499535 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f21dc06ae70 0x7f21dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:54.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.624+0000 7f21edffb640 1 -- 172.21.15.76:0/481499535 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f21f4089cd0 con 0x7f21f80d3b60 2023-11-09T03:54:54.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.624+0000 7f21fc897640 1 --2- 172.21.15.76:0/481499535 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f21dc06ae70 0x7f21dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:54.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.625+0000 7f21fc897640 1 --2- 172.21.15.76:0/481499535 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f21dc06ae70 0x7f21dc06d330 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f21f80b1a50 tx=0x7f21e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:54.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.628+0000 7f21edffb640 1 -- 172.21.15.76:0/481499535 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f21f40542c0 con 0x7f21f80d3b60 2023-11-09T03:54:54.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.829+0000 7f21ff2fc640 1 -- 172.21.15.76:0/481499535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21f80d0430 con 0x7f21f80d3b60 2023-11-09T03:54:54.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.829+0000 7f21edffb640 1 -- 172.21.15.76:0/481499535 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+4935 (secure 0 0 0) 0x7f21f40578e0 con 0x7f21f80d3b60 2023-11-09T03:54:54.830 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:54:54.830 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":131,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6698,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:46.356076+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":9024},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9024":{"gid":9024,"name":"a","rank":1,"incarnation":129,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:54:54.831 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 131 2023-11-09T03:54:54.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.831+0000 7f21ff2fc640 1 -- 172.21.15.76:0/481499535 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f21dc06ae70 msgr2=0x7f21dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:54.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.831+0000 7f21ff2fc640 1 --2- 172.21.15.76:0/481499535 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f21dc06ae70 0x7f21dc06d330 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f21f80b1a50 tx=0x7f21e8008040 comp rx=0 tx=0).stop 2023-11-09T03:54:54.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.831+0000 7f21ff2fc640 1 -- 172.21.15.76:0/481499535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21f80d3b60 msgr2=0x7f21f815a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:54.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.831+0000 7f21ff2fc640 1 --2- 172.21.15.76:0/481499535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21f80d3b60 0x7f21f815a6b0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f21f4004640 tx=0x7f21f400c8e0 comp rx=0 tx=0).stop 2023-11-09T03:54:54.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.831+0000 7f21ff2fc640 1 -- 172.21.15.76:0/481499535 shutdown_connections 2023-11-09T03:54:54.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.831+0000 7f21ff2fc640 1 --2- 172.21.15.76:0/481499535 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f21dc06ae70 0x7f21dc06d330 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:54.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.831+0000 7f21ff2fc640 1 --2- 172.21.15.76:0/481499535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21f815d4e0 0x7f21f806af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:54.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.831+0000 7f21ff2fc640 1 --2- 172.21.15.76:0/481499535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21f815d100 0x7f21f80c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:54.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.831+0000 7f21ff2fc640 1 --2- 172.21.15.76:0/481499535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21f80d3b60 0x7f21f815a6b0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:54.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.831+0000 7f21ff2fc640 1 -- 172.21.15.76:0/481499535 >> 172.21.15.76:0/481499535 conn(0x7f21f80d2ef0 msgr2=0x7f21f80d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:54.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.831+0000 7f21ff2fc640 1 -- 172.21.15.76:0/481499535 shutdown_connections 2023-11-09T03:54:54.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:54.831+0000 7f21ff2fc640 1 -- 172.21.15.76:0/481499535 wait complete. 2023-11-09T03:54:55.244 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 131, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-09T03:48:57.383848+0000', 'modified': '2023-11-09T03:54:46.356076+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 117, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 8568, 'mds_1': 9024}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8568': {'gid': 8568, 'name': 'c', 'rank': 0, 'incarnation': 110, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.76:6833/153465374', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6832', 'nonce': 153465374}, {'type': 'v1', 'addr': '172.21.15.76:6833', 'nonce': 153465374}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_9024': {'gid': 9024, 'name': 'a', 'rank': 1, 'incarnation': 129, 'state': 'up:reconnect', 'state_seq': 12, 'addr': '172.21.15.76:6835/757529522', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 757529522}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 757529522}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-09T03:54:55.245 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9024:up:reconnect 2023-11-09T03:54:56.246 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:54:56.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.359+0000 7f4d368b4640 1 Processor -- start 2023-11-09T03:54:56.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.360+0000 7f4d368b4640 1 -- start start 2023-11-09T03:54:56.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.360+0000 7f4d368b4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4d300d1be0 0x7f4d30167e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:56.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.360+0000 7f4d368b4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d300b32b0 con 0x7f4d3015d4e0 2023-11-09T03:54:56.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.360+0000 7f4d368b4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d300cee10 con 0x7f4d3015d100 2023-11-09T03:54:56.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.360+0000 7f4d368b4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d300cec20 con 0x7f4d300d1be0 2023-11-09T03:54:56.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.360+0000 7f4d34e51640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4d300d1be0 0x7f4d30167e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:56.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.361+0000 7f4d2f7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f4d3015d4e0 0x7f4d300d14d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45422/0 (socket says 172.21.15.76:45422) 2023-11-09T03:54:56.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.361+0000 7f4d2f7fe640 1 -- 172.21.15.76:0/2808473165 learned_addr learned my addr 172.21.15.76:0/2808473165 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:54:56.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.361+0000 7f4d34e51640 1 -- 172.21.15.76:0/2808473165 >> v1:172.21.15.113:6789/0 conn(0x7f4d3015d100 legacy=0x7f4d300ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:56.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.361+0000 7f4d34e51640 1 -- 172.21.15.76:0/2808473165 >> v1:172.21.15.76:6789/0 conn(0x7f4d3015d4e0 legacy=0x7f4d300d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:56.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.361+0000 7f4d34e51640 1 -- 172.21.15.76:0/2808473165 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d300b4750 con 0x7f4d300d1be0 2023-11-09T03:54:56.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.361+0000 7f4d2effd640 1 -- 172.21.15.76:0/2808473165 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 179139484 0 0) 0x7f4d300b32b0 con 0x7f4d3015d4e0 2023-11-09T03:54:56.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.362+0000 7f4d34e51640 1 --2- 172.21.15.76:0/2808473165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d300d1be0 0x7f4d30167e80 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009d90 tx=0x7f4d2802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bcc23143ef66210 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:56.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.362+0000 7f4d2effd640 1 -- 172.21.15.76:0/2808473165 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2802cc60 con 0x7f4d300d1be0 2023-11-09T03:54:56.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.362+0000 7f4d2effd640 1 -- 172.21.15.76:0/2808473165 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2802cdc0 con 0x7f4d300d1be0 2023-11-09T03:54:56.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.362+0000 7f4d2effd640 1 -- 172.21.15.76:0/2808473165 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d28034660 con 0x7f4d300d1be0 2023-11-09T03:54:56.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.362+0000 7f4d368b4640 1 -- 172.21.15.76:0/2808473165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d300d1be0 msgr2=0x7f4d30167e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:56.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.362+0000 7f4d368b4640 1 --2- 172.21.15.76:0/2808473165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d300d1be0 0x7f4d30167e80 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009d90 tx=0x7f4d2802beb0 comp rx=0 tx=0).stop 2023-11-09T03:54:56.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.362+0000 7f4d368b4640 1 -- 172.21.15.76:0/2808473165 shutdown_connections 2023-11-09T03:54:56.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.362+0000 7f4d368b4640 1 --2- 172.21.15.76:0/2808473165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d300d1be0 0x7f4d30167e80 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:56.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.362+0000 7f4d368b4640 1 -- 172.21.15.76:0/2808473165 >> 172.21.15.76:0/2808473165 conn(0x7f4d300d2ef0 msgr2=0x7f4d300d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:56.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.362+0000 7f4d368b4640 1 -- 172.21.15.76:0/2808473165 shutdown_connections 2023-11-09T03:54:56.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.362+0000 7f4d368b4640 1 -- 172.21.15.76:0/2808473165 wait complete. 2023-11-09T03:54:56.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.364+0000 7f4d368b4640 1 Processor -- start 2023-11-09T03:54:56.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.364+0000 7f4d368b4640 1 -- start start 2023-11-09T03:54:56.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.365+0000 7f4d368b4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d300d1be0 0x7f4d3006c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:56.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.365+0000 7f4d368b4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d3015d100 0x7f4d301f2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:56.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.365+0000 7f4d368b4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d3015d4e0 0x7f4d3015fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:56.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.365+0000 7f4d368b4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d300c95e0 con 0x7f4d3015d100 2023-11-09T03:54:56.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.365+0000 7f4d368b4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d30150300 con 0x7f4d300d1be0 2023-11-09T03:54:56.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.365+0000 7f4d368b4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d300b11d0 con 0x7f4d3015d4e0 2023-11-09T03:54:56.369 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.365+0000 7f4d2f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d3015d100 0x7f4d301f2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:56.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.365+0000 7f4d2f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d3015d100 0x7f4d301f2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40944/0 (socket says 172.21.15.76:40944) 2023-11-09T03:54:56.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.365+0000 7f4d2f7fe640 1 -- 172.21.15.76:0/3111948092 learned_addr learned my addr 172.21.15.76:0/3111948092 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:56.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.365+0000 7f4d34e51640 1 --2- 172.21.15.76:0/3111948092 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d3015d4e0 0x7f4d3015fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:56.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.365+0000 7f4d2ffff640 1 --2- 172.21.15.76:0/3111948092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d300d1be0 0x7f4d3006c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:56.370 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.365+0000 7f4d2f7fe640 1 -- 172.21.15.76:0/3111948092 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d3015d4e0 msgr2=0x7f4d3015fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:56.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.365+0000 7f4d2f7fe640 1 --2- 172.21.15.76:0/3111948092 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d3015d4e0 0x7f4d3015fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:56.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.365+0000 7f4d2f7fe640 1 -- 172.21.15.76:0/3111948092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d300d1be0 msgr2=0x7f4d3006c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:56.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.365+0000 7f4d2f7fe640 1 --2- 172.21.15.76:0/3111948092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d300d1be0 0x7f4d3006c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:56.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.365+0000 7f4d2f7fe640 1 -- 172.21.15.76:0/3111948092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d28009a70 con 0x7f4d3015d100 2023-11-09T03:54:56.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.365+0000 7f4d34e51640 1 --2- 172.21.15.76:0/3111948092 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d3015d4e0 0x7f4d3015fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:54:56.371 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.365+0000 7f4d2f7fe640 1 --2- 172.21.15.76:0/3111948092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d3015d100 0x7f4d301f2360 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400c410 tx=0x7f4d2400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:56.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.365+0000 7f4d2cff9640 1 -- 172.21.15.76:0/3111948092 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d240074e0 con 0x7f4d3015d100 2023-11-09T03:54:56.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.365+0000 7f4d2cff9640 1 -- 172.21.15.76:0/3111948092 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d24007660 con 0x7f4d3015d100 2023-11-09T03:54:56.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.365+0000 7f4d368b4640 1 -- 172.21.15.76:0/3111948092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d300b14b0 con 0x7f4d3015d100 2023-11-09T03:54:56.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.365+0000 7f4d2cff9640 1 -- 172.21.15.76:0/3111948092 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d24004a20 con 0x7f4d3015d100 2023-11-09T03:54:56.372 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.365+0000 7f4d368b4640 1 -- 172.21.15.76:0/3111948092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d300b1a70 con 0x7f4d3015d100 2023-11-09T03:54:56.373 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.366+0000 7f4d368b4640 1 -- 172.21.15.76:0/3111948092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d300684b0 con 0x7f4d3015d100 2023-11-09T03:54:56.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.367+0000 7f4d2cff9640 1 -- 172.21.15.76:0/3111948092 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4d24004b80 con 0x7f4d3015d100 2023-11-09T03:54:56.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.367+0000 7f4d2cff9640 1 --2- 172.21.15.76:0/3111948092 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d1c06ae70 0x7f4d1c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:56.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.367+0000 7f4d2cff9640 1 -- 172.21.15.76:0/3111948092 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f4d24089e40 con 0x7f4d3015d100 2023-11-09T03:54:56.374 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.367+0000 7f4d2ffff640 1 --2- 172.21.15.76:0/3111948092 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d1c06ae70 0x7f4d1c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:56.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.368+0000 7f4d2ffff640 1 --2- 172.21.15.76:0/3111948092 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d1c06ae70 0x7f4d1c06d330 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4d30159d70 tx=0x7f4d20009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:56.375 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.371+0000 7f4d2cff9640 1 -- 172.21.15.76:0/3111948092 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4d24057f50 con 0x7f4d3015d100 2023-11-09T03:54:56.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.575+0000 7f4d368b4640 1 -- 172.21.15.76:0/3111948092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d300d2cd0 con 0x7f4d3015d100 2023-11-09T03:54:56.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.576+0000 7f4d2cff9640 1 -- 172.21.15.76:0/3111948092 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+4935 (secure 0 0 0) 0x7f4d24057ae0 con 0x7f4d3015d100 2023-11-09T03:54:56.576 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:54:56.576 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":131,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6698,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:46.356076+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":9024},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9024":{"gid":9024,"name":"a","rank":1,"incarnation":129,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:54:56.576 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 131 2023-11-09T03:54:56.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.577+0000 7f4d368b4640 1 -- 172.21.15.76:0/3111948092 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d1c06ae70 msgr2=0x7f4d1c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:56.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.577+0000 7f4d368b4640 1 --2- 172.21.15.76:0/3111948092 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d1c06ae70 0x7f4d1c06d330 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4d30159d70 tx=0x7f4d20009040 comp rx=0 tx=0).stop 2023-11-09T03:54:56.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.577+0000 7f4d368b4640 1 -- 172.21.15.76:0/3111948092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d3015d100 msgr2=0x7f4d301f2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:56.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.577+0000 7f4d368b4640 1 --2- 172.21.15.76:0/3111948092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d3015d100 0x7f4d301f2360 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400c410 tx=0x7f4d2400c8d0 comp rx=0 tx=0).stop 2023-11-09T03:54:56.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.578+0000 7f4d368b4640 1 -- 172.21.15.76:0/3111948092 shutdown_connections 2023-11-09T03:54:56.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.578+0000 7f4d368b4640 1 --2- 172.21.15.76:0/3111948092 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d1c06ae70 0x7f4d1c06d330 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:56.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.578+0000 7f4d368b4640 1 --2- 172.21.15.76:0/3111948092 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d3015d4e0 0x7f4d3015fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:56.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.578+0000 7f4d368b4640 1 --2- 172.21.15.76:0/3111948092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d3015d100 0x7f4d301f2360 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:56.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.578+0000 7f4d368b4640 1 --2- 172.21.15.76:0/3111948092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d300d1be0 0x7f4d3006c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:56.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.578+0000 7f4d368b4640 1 -- 172.21.15.76:0/3111948092 >> 172.21.15.76:0/3111948092 conn(0x7f4d300d2ef0 msgr2=0x7f4d300d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:56.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.578+0000 7f4d368b4640 1 -- 172.21.15.76:0/3111948092 shutdown_connections 2023-11-09T03:54:56.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:56.578+0000 7f4d368b4640 1 -- 172.21.15.76:0/3111948092 wait complete. 2023-11-09T03:54:56.989 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 131, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-09T03:48:57.383848+0000', 'modified': '2023-11-09T03:54:46.356076+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 117, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 8568, 'mds_1': 9024}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8568': {'gid': 8568, 'name': 'c', 'rank': 0, 'incarnation': 110, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.76:6833/153465374', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6832', 'nonce': 153465374}, {'type': 'v1', 'addr': '172.21.15.76:6833', 'nonce': 153465374}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_9024': {'gid': 9024, 'name': 'a', 'rank': 1, 'incarnation': 129, 'state': 'up:reconnect', 'state_seq': 12, 'addr': '172.21.15.76:6835/757529522', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 757529522}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 757529522}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-09T03:54:56.990 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9024:up:reconnect 2023-11-09T03:54:57.991 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:54:58.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.101+0000 7fdc957a9640 1 Processor -- start 2023-11-09T03:54:58.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.101+0000 7fdc957a9640 1 -- start start 2023-11-09T03:54:58.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.102+0000 7fdc957a9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdc9015d100 0x7fdc900ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:58.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.102+0000 7fdc957a9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdc9015d4e0 0x7fdc900d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:58.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.102+0000 7fdc957a9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc900b32b0 con 0x7fdc900d3b60 2023-11-09T03:54:58.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.102+0000 7fdc957a9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc900cee10 con 0x7fdc9015d4e0 2023-11-09T03:54:58.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.102+0000 7fdc957a9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc900b4750 con 0x7fdc9015d100 2023-11-09T03:54:58.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.102+0000 7fdc8e7fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdc9015d4e0 0x7fdc900d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:58.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.102+0000 7fdc8e7fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdc9015d4e0 0x7fdc900d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40958/0 (socket says 172.21.15.76:40958) 2023-11-09T03:54:58.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.102+0000 7fdc8e7fc640 1 -- 172.21.15.76:0/1602630442 learned_addr learned my addr 172.21.15.76:0/1602630442 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:58.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.103+0000 7fdc8e7fc640 1 -- 172.21.15.76:0/1602630442 >> v2:172.21.15.113:3300/0 conn(0x7fdc9015d100 msgr2=0x7fdc900ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:58.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.103+0000 7fdc8effd640 1 --2- 172.21.15.76:0/1602630442 >> v2:172.21.15.113:3300/0 conn(0x7fdc9015d100 0x7fdc900ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:58.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.103+0000 7fdc8e7fc640 1 --2- 172.21.15.76:0/1602630442 >> v2:172.21.15.113:3300/0 conn(0x7fdc9015d100 0x7fdc900ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:58.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.103+0000 7fdc8e7fc640 1 -- 172.21.15.76:0/1602630442 >> v1:172.21.15.113:6789/0 conn(0x7fdc900d3b60 legacy=0x7fdc900cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:58.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.103+0000 7fdc8e7fc640 1 -- 172.21.15.76:0/1602630442 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc900b2980 con 0x7fdc9015d4e0 2023-11-09T03:54:58.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.103+0000 7fdc8e7fc640 1 --2- 172.21.15.76:0/1602630442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc9015d4e0 0x7fdc900d14d0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fdc900d09f0 tx=0x7fdc8402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d95fce22de6f5c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:58.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.104+0000 7fdc8dffb640 1 -- 172.21.15.76:0/1602630442 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc84031870 con 0x7fdc9015d4e0 2023-11-09T03:54:58.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.104+0000 7fdc8dffb640 1 -- 172.21.15.76:0/1602630442 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc840319d0 con 0x7fdc9015d4e0 2023-11-09T03:54:58.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.104+0000 7fdc8dffb640 1 -- 172.21.15.76:0/1602630442 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc84031d60 con 0x7fdc9015d4e0 2023-11-09T03:54:58.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.104+0000 7fdc957a9640 1 -- 172.21.15.76:0/1602630442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc9015d4e0 msgr2=0x7fdc900d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:58.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.104+0000 7fdc957a9640 1 --2- 172.21.15.76:0/1602630442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc9015d4e0 0x7fdc900d14d0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fdc900d09f0 tx=0x7fdc8402b910 comp rx=0 tx=0).stop 2023-11-09T03:54:58.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.104+0000 7fdc957a9640 1 -- 172.21.15.76:0/1602630442 shutdown_connections 2023-11-09T03:54:58.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.104+0000 7fdc957a9640 1 --2- 172.21.15.76:0/1602630442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc9015d4e0 0x7fdc900d14d0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:58.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.104+0000 7fdc957a9640 1 --2- 172.21.15.76:0/1602630442 >> v2:172.21.15.113:3300/0 conn(0x7fdc9015d100 0x7fdc900ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:58.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.104+0000 7fdc957a9640 1 -- 172.21.15.76:0/1602630442 >> 172.21.15.76:0/1602630442 conn(0x7fdc900d2ef0 msgr2=0x7fdc900d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:58.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.104+0000 7fdc957a9640 1 -- 172.21.15.76:0/1602630442 shutdown_connections 2023-11-09T03:54:58.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.104+0000 7fdc957a9640 1 -- 172.21.15.76:0/1602630442 wait complete. 2023-11-09T03:54:58.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.106+0000 7fdc957a9640 1 Processor -- start 2023-11-09T03:54:58.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.106+0000 7fdc957a9640 1 -- start start 2023-11-09T03:54:58.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.106+0000 7fdc957a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc900d3b60 0x7fdc90160370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:58.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.106+0000 7fdc957a9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc9015d100 0x7fdc900cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:58.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.106+0000 7fdc957a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc9015d4e0 0x7fdc900ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:58.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.106+0000 7fdc957a9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc900ca830 con 0x7fdc9015d100 2023-11-09T03:54:58.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.106+0000 7fdc957a9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc900b6ac0 con 0x7fdc900d3b60 2023-11-09T03:54:58.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.106+0000 7fdc957a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc900cceb0 con 0x7fdc9015d4e0 2023-11-09T03:54:58.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.106+0000 7fdc8e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc9015d100 0x7fdc900cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:58.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.106+0000 7fdc8e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc9015d100 0x7fdc900cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40970/0 (socket says 172.21.15.76:40970) 2023-11-09T03:54:58.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.107+0000 7fdc8e7fc640 1 -- 172.21.15.76:0/2095445544 learned_addr learned my addr 172.21.15.76:0/2095445544 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:58.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.107+0000 7fdc8effd640 1 --2- 172.21.15.76:0/2095445544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc900d3b60 0x7fdc90160370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:58.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.107+0000 7fdc8f7fe640 1 --2- 172.21.15.76:0/2095445544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc9015d4e0 0x7fdc900ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:58.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.107+0000 7fdc8e7fc640 1 -- 172.21.15.76:0/2095445544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc9015d4e0 msgr2=0x7fdc900ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:58.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.107+0000 7fdc8e7fc640 1 --2- 172.21.15.76:0/2095445544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc9015d4e0 0x7fdc900ca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:58.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.107+0000 7fdc8e7fc640 1 -- 172.21.15.76:0/2095445544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc900d3b60 msgr2=0x7fdc90160370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:58.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.107+0000 7fdc8e7fc640 1 --2- 172.21.15.76:0/2095445544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc900d3b60 0x7fdc90160370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:58.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.107+0000 7fdc8e7fc640 1 -- 172.21.15.76:0/2095445544 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc840094d0 con 0x7fdc9015d100 2023-11-09T03:54:58.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.107+0000 7fdc8e7fc640 1 --2- 172.21.15.76:0/2095445544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc9015d100 0x7fdc900cc7b0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fdc8402b950 tx=0x7fdc840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:58.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.107+0000 7fdc777fe640 1 -- 172.21.15.76:0/2095445544 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc84031820 con 0x7fdc9015d100 2023-11-09T03:54:58.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.107+0000 7fdc777fe640 1 -- 172.21.15.76:0/2095445544 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc840453f0 con 0x7fdc9015d100 2023-11-09T03:54:58.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.107+0000 7fdc957a9640 1 -- 172.21.15.76:0/2095445544 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc900b6d60 con 0x7fdc9015d100 2023-11-09T03:54:58.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.107+0000 7fdc777fe640 1 -- 172.21.15.76:0/2095445544 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc840457a0 con 0x7fdc9015d100 2023-11-09T03:54:58.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.107+0000 7fdc957a9640 1 -- 172.21.15.76:0/2095445544 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc900b7320 con 0x7fdc9015d100 2023-11-09T03:54:58.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.108+0000 7fdc957a9640 1 -- 172.21.15.76:0/2095445544 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc9015cad0 con 0x7fdc9015d100 2023-11-09T03:54:58.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.109+0000 7fdc777fe640 1 -- 172.21.15.76:0/2095445544 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdc8404e450 con 0x7fdc9015d100 2023-11-09T03:54:58.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.109+0000 7fdc777fe640 1 --2- 172.21.15.76:0/2095445544 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdc7c06ae70 0x7fdc7c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:58.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.109+0000 7fdc777fe640 1 -- 172.21.15.76:0/2095445544 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7fdc8402d040 con 0x7fdc9015d100 2023-11-09T03:54:58.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.110+0000 7fdc8effd640 1 --2- 172.21.15.76:0/2095445544 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdc7c06ae70 0x7fdc7c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:58.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.110+0000 7fdc8effd640 1 --2- 172.21.15.76:0/2095445544 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdc7c06ae70 0x7fdc7c06d330 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000f400 tx=0x7fdc80007740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:58.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.114+0000 7fdc777fe640 1 -- 172.21.15.76:0/2095445544 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdc84076e20 con 0x7fdc9015d100 2023-11-09T03:54:58.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.313+0000 7fdc957a9640 1 -- 172.21.15.76:0/2095445544 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc900684b0 con 0x7fdc9015d100 2023-11-09T03:54:58.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.314+0000 7fdc777fe640 1 -- 172.21.15.76:0/2095445544 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+4935 (secure 0 0 0) 0x7fdc8407a440 con 0x7fdc9015d100 2023-11-09T03:54:58.314 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:54:58.314 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":131,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6698,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:46.356076+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":9024},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9024":{"gid":9024,"name":"a","rank":1,"incarnation":129,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:54:58.314 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 131 2023-11-09T03:54:58.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.316+0000 7fdc957a9640 1 -- 172.21.15.76:0/2095445544 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdc7c06ae70 msgr2=0x7fdc7c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:58.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.316+0000 7fdc957a9640 1 --2- 172.21.15.76:0/2095445544 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdc7c06ae70 0x7fdc7c06d330 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000f400 tx=0x7fdc80007740 comp rx=0 tx=0).stop 2023-11-09T03:54:58.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.316+0000 7fdc957a9640 1 -- 172.21.15.76:0/2095445544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc9015d100 msgr2=0x7fdc900cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:58.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.316+0000 7fdc957a9640 1 --2- 172.21.15.76:0/2095445544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc9015d100 0x7fdc900cc7b0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fdc8402b950 tx=0x7fdc840046c0 comp rx=0 tx=0).stop 2023-11-09T03:54:58.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.316+0000 7fdc957a9640 1 -- 172.21.15.76:0/2095445544 shutdown_connections 2023-11-09T03:54:58.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.316+0000 7fdc957a9640 1 --2- 172.21.15.76:0/2095445544 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdc7c06ae70 0x7fdc7c06d330 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:58.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.316+0000 7fdc957a9640 1 --2- 172.21.15.76:0/2095445544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdc9015d4e0 0x7fdc900ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:58.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.316+0000 7fdc957a9640 1 --2- 172.21.15.76:0/2095445544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc9015d100 0x7fdc900cc7b0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:58.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.316+0000 7fdc957a9640 1 --2- 172.21.15.76:0/2095445544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc900d3b60 0x7fdc90160370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:58.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.316+0000 7fdc957a9640 1 -- 172.21.15.76:0/2095445544 >> 172.21.15.76:0/2095445544 conn(0x7fdc900d2ef0 msgr2=0x7fdc900d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:58.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.316+0000 7fdc957a9640 1 -- 172.21.15.76:0/2095445544 shutdown_connections 2023-11-09T03:54:58.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:58.316+0000 7fdc957a9640 1 -- 172.21.15.76:0/2095445544 wait complete. 2023-11-09T03:54:58.728 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 131, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-09T03:48:57.383848+0000', 'modified': '2023-11-09T03:54:46.356076+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 117, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 8568, 'mds_1': 9024}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8568': {'gid': 8568, 'name': 'c', 'rank': 0, 'incarnation': 110, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.76:6833/153465374', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6832', 'nonce': 153465374}, {'type': 'v1', 'addr': '172.21.15.76:6833', 'nonce': 153465374}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_9024': {'gid': 9024, 'name': 'a', 'rank': 1, 'incarnation': 129, 'state': 'up:reconnect', 'state_seq': 12, 'addr': '172.21.15.76:6835/757529522', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 757529522}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 757529522}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-09T03:54:58.729 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9024:up:reconnect 2023-11-09T03:54:59.729 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:54:59.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.840+0000 7f64a2032640 1 Processor -- start 2023-11-09T03:54:59.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.841+0000 7f64a2032640 1 -- start start 2023-11-09T03:54:59.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.841+0000 7f64a2032640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f649c0d1be0 0x7f649c167e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:59.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.841+0000 7f64a2032640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f649c0b32b0 con 0x7f649c15d100 2023-11-09T03:54:59.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.841+0000 7f64a2032640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f649c0cee10 con 0x7f649c15d4e0 2023-11-09T03:54:59.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.841+0000 7f64a2032640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f649c0cec20 con 0x7f649c0d1be0 2023-11-09T03:54:59.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.841+0000 7f649bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f649c0d1be0 0x7f649c167e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:59.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.841+0000 7f649b7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f649c15d100 0x7f649c0ce510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45438/0 (socket says 172.21.15.76:45438) 2023-11-09T03:54:59.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.841+0000 7f649b7fe640 1 -- 172.21.15.76:0/1262100500 learned_addr learned my addr 172.21.15.76:0/1262100500 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:54:59.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.841+0000 7f649bfff640 1 --2- 172.21.15.76:0/1262100500 >> v2:172.21.15.76:3300/0 conn(0x7f649c0d1be0 0x7f649c167e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40978/0 (socket says 172.21.15.76:40978) 2023-11-09T03:54:59.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.842+0000 7f649bfff640 1 -- 172.21.15.76:0/1262100500 >> v1:172.21.15.113:6789/0 conn(0x7f649c15d4e0 legacy=0x7f649c0d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:54:59.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.842+0000 7f649bfff640 1 -- 172.21.15.76:0/1262100500 >> v1:172.21.15.76:6789/0 conn(0x7f649c15d100 legacy=0x7f649c0ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:59.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.842+0000 7f649bfff640 1 -- 172.21.15.76:0/1262100500 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f649c0b4750 con 0x7f649c0d1be0 2023-11-09T03:54:59.847 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.843+0000 7f649bfff640 1 --2- 172.21.15.76:0/1262100500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f649c0d1be0 0x7f649c167e80 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f6494009d90 tx=0x7f649402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2de2bb7249ab0af server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:59.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.843+0000 7f649a7fc640 1 -- 172.21.15.76:0/1262100500 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649402cc60 con 0x7f649c0d1be0 2023-11-09T03:54:59.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.843+0000 7f649a7fc640 1 -- 172.21.15.76:0/1262100500 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649402cdc0 con 0x7f649c0d1be0 2023-11-09T03:54:59.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.843+0000 7f649a7fc640 1 -- 172.21.15.76:0/1262100500 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6494034660 con 0x7f649c0d1be0 2023-11-09T03:54:59.848 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.843+0000 7f64a2032640 1 -- 172.21.15.76:0/1262100500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f649c0d1be0 msgr2=0x7f649c167e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:59.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.843+0000 7f64a2032640 1 --2- 172.21.15.76:0/1262100500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f649c0d1be0 0x7f649c167e80 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f6494009d90 tx=0x7f649402beb0 comp rx=0 tx=0).stop 2023-11-09T03:54:59.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.843+0000 7f64a2032640 1 -- 172.21.15.76:0/1262100500 shutdown_connections 2023-11-09T03:54:59.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.843+0000 7f64a2032640 1 --2- 172.21.15.76:0/1262100500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f649c0d1be0 0x7f649c167e80 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:59.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.843+0000 7f64a2032640 1 -- 172.21.15.76:0/1262100500 >> 172.21.15.76:0/1262100500 conn(0x7f649c0d2ef0 msgr2=0x7f649c0d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:54:59.849 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.843+0000 7f64a2032640 1 -- 172.21.15.76:0/1262100500 shutdown_connections 2023-11-09T03:54:59.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.843+0000 7f64a2032640 1 -- 172.21.15.76:0/1262100500 wait complete. 2023-11-09T03:54:59.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.844+0000 7f64a2032640 1 Processor -- start 2023-11-09T03:54:59.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.845+0000 7f64a2032640 1 -- start start 2023-11-09T03:54:59.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.845+0000 7f64a2032640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f649c0d1be0 0x7f649c0cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:59.850 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.845+0000 7f64a2032640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f649c15d100 0x7f649c0b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:59.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.845+0000 7f64a2032640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f649c15d4e0 0x7f649c0ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:59.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.845+0000 7f64a2032640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649c0cc6d0 con 0x7f649c15d100 2023-11-09T03:54:59.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.845+0000 7f64a2032640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649c0b6620 con 0x7f649c0d1be0 2023-11-09T03:54:59.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.845+0000 7f64a2032640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f649c0b6900 con 0x7f649c15d4e0 2023-11-09T03:54:59.851 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.845+0000 7f649affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f649c15d100 0x7f649c0b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:59.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.846+0000 7f649affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f649c15d100 0x7f649c0b76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40990/0 (socket says 172.21.15.76:40990) 2023-11-09T03:54:59.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.846+0000 7f649affd640 1 -- 172.21.15.76:0/3556196437 learned_addr learned my addr 172.21.15.76:0/3556196437 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:54:59.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.846+0000 7f649b7fe640 1 --2- 172.21.15.76:0/3556196437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f649c0d1be0 0x7f649c0cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:59.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.846+0000 7f649bfff640 1 --2- 172.21.15.76:0/3556196437 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f649c15d4e0 0x7f649c0ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:59.852 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.846+0000 7f649affd640 1 -- 172.21.15.76:0/3556196437 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f649c15d4e0 msgr2=0x7f649c0ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:59.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.846+0000 7f649affd640 1 --2- 172.21.15.76:0/3556196437 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f649c15d4e0 0x7f649c0ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:59.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.846+0000 7f649affd640 1 -- 172.21.15.76:0/3556196437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f649c0d1be0 msgr2=0x7f649c0cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:54:59.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.846+0000 7f649affd640 1 --2- 172.21.15.76:0/3556196437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f649c0d1be0 0x7f649c0cc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:54:59.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.846+0000 7f649affd640 1 -- 172.21.15.76:0/3556196437 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6494009a70 con 0x7f649c15d100 2023-11-09T03:54:59.853 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.846+0000 7f649b7fe640 1 --2- 172.21.15.76:0/3556196437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f649c0d1be0 0x7f649c0cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:54:59.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.846+0000 7f649affd640 1 --2- 172.21.15.76:0/3556196437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f649c15d100 0x7f649c0b76c0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f649000b3c0 tx=0x7f649000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:59.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.846+0000 7f6483fff640 1 -- 172.21.15.76:0/3556196437 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6490011830 con 0x7f649c15d100 2023-11-09T03:54:59.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.846+0000 7f6483fff640 1 -- 172.21.15.76:0/3556196437 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6490011990 con 0x7f649c15d100 2023-11-09T03:54:59.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.846+0000 7f6483fff640 1 -- 172.21.15.76:0/3556196437 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6490011d80 con 0x7f649c15d100 2023-11-09T03:54:59.854 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.846+0000 7f64a2032640 1 -- 172.21.15.76:0/3556196437 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f649c0b6be0 con 0x7f649c15d100 2023-11-09T03:54:59.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.846+0000 7f64a2032640 1 -- 172.21.15.76:0/3556196437 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f649c0b71a0 con 0x7f649c15d100 2023-11-09T03:54:59.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.847+0000 7f64a2032640 1 -- 172.21.15.76:0/3556196437 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f649c0b73c0 con 0x7f649c15d100 2023-11-09T03:54:59.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.848+0000 7f6483fff640 1 -- 172.21.15.76:0/3556196437 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f649001b460 con 0x7f649c15d100 2023-11-09T03:54:59.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.848+0000 7f6483fff640 1 --2- 172.21.15.76:0/3556196437 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f648806ae70 0x7f648806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:54:59.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.848+0000 7f6483fff640 1 -- 172.21.15.76:0/3556196437 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7f6490089d00 con 0x7f649c15d100 2023-11-09T03:54:59.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.848+0000 7f649b7fe640 1 --2- 172.21.15.76:0/3556196437 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f648806ae70 0x7f648806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:54:59.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.849+0000 7f649b7fe640 1 --2- 172.21.15.76:0/3556196437 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f648806ae70 0x7f648806d330 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f648c0010e0 tx=0x7f648c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:54:59.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:54:59.852+0000 7f6483fff640 1 -- 172.21.15.76:0/3556196437 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f64900542c0 con 0x7f649c15d100 2023-11-09T03:55:00.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.052+0000 7f64a2032640 1 -- 172.21.15.76:0/3556196437 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f649c0684b0 con 0x7f649c15d100 2023-11-09T03:55:00.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.053+0000 7f6483fff640 1 -- 172.21.15.76:0/3556196437 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+4932 (secure 0 0 0) 0x7f64900578e0 con 0x7f649c15d100 2023-11-09T03:55:00.053 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:55:00.053 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":133,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6698,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":133,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:58.523123+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":9024},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9024":{"gid":9024,"name":"a","rank":1,"incarnation":129,"state":"up:active","state_seq":16,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:55:00.053 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 133 2023-11-09T03:55:00.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.054+0000 7f64a2032640 1 -- 172.21.15.76:0/3556196437 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f648806ae70 msgr2=0x7f648806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:00.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.054+0000 7f64a2032640 1 --2- 172.21.15.76:0/3556196437 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f648806ae70 0x7f648806d330 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f648c0010e0 tx=0x7f648c009040 comp rx=0 tx=0).stop 2023-11-09T03:55:00.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.054+0000 7f64a2032640 1 -- 172.21.15.76:0/3556196437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f649c15d100 msgr2=0x7f649c0b76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:00.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.054+0000 7f64a2032640 1 --2- 172.21.15.76:0/3556196437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f649c15d100 0x7f649c0b76c0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f649000b3c0 tx=0x7f649000b880 comp rx=0 tx=0).stop 2023-11-09T03:55:00.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.054+0000 7f64a2032640 1 -- 172.21.15.76:0/3556196437 shutdown_connections 2023-11-09T03:55:00.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.054+0000 7f64a2032640 1 --2- 172.21.15.76:0/3556196437 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f648806ae70 0x7f648806d330 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.054+0000 7f64a2032640 1 --2- 172.21.15.76:0/3556196437 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f649c15d4e0 0x7f649c0ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.054+0000 7f64a2032640 1 --2- 172.21.15.76:0/3556196437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f649c15d100 0x7f649c0b76c0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.054+0000 7f64a2032640 1 --2- 172.21.15.76:0/3556196437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f649c0d1be0 0x7f649c0cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.055+0000 7f64a2032640 1 -- 172.21.15.76:0/3556196437 >> 172.21.15.76:0/3556196437 conn(0x7f649c0d2ef0 msgr2=0x7f649c0cfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:00.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.055+0000 7f64a2032640 1 -- 172.21.15.76:0/3556196437 shutdown_connections 2023-11-09T03:55:00.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.055+0000 7f64a2032640 1 -- 172.21.15.76:0/3556196437 wait complete. 2023-11-09T03:55:00.238 INFO:tasks.ceph.mds.c.smithi076.stderr:2023-11-09T03:55:00.239+0000 7f1a9c20f640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-09T03:55:00.467 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 133, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-09T03:48:57.383848+0000', 'modified': '2023-11-09T03:54:58.523123+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 117, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 8568, 'mds_1': 9024}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8568': {'gid': 8568, 'name': 'c', 'rank': 0, 'incarnation': 110, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.76:6833/153465374', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6832', 'nonce': 153465374}, {'type': 'v1', 'addr': '172.21.15.76:6833', 'nonce': 153465374}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_9024': {'gid': 9024, 'name': 'a', 'rank': 1, 'incarnation': 129, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.76:6835/757529522', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 757529522}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 757529522}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-09T03:55:00.467 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2023-11-09T03:55:00.467 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-09T03:55:00.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.572+0000 7fe42d489640 1 Processor -- start 2023-11-09T03:55:00.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.572+0000 7fe42d489640 1 -- start start 2023-11-09T03:55:00.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.573+0000 7fe42d489640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe42815d100 0x7fe4280ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.573+0000 7fe42d489640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4280b32b0 con 0x7fe42815d4e0 2023-11-09T03:55:00.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.573+0000 7fe42d489640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4280cee10 con 0x7fe4280d1be0 2023-11-09T03:55:00.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.573+0000 7fe42d489640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4280b2980 con 0x7fe42815d100 2023-11-09T03:55:00.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.573+0000 7fe41e7fc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe42815d4e0 0x7fe4280d14d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45454/0 (socket says 172.21.15.76:45454) 2023-11-09T03:55:00.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.573+0000 7fe41e7fc640 1 -- 172.21.15.76:0/926962007 learned_addr learned my addr 172.21.15.76:0/926962007 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:55:00.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.573+0000 7fe426ffd640 1 --2- 172.21.15.76:0/926962007 >> v2:172.21.15.113:3300/0 conn(0x7fe42815d100 0x7fe4280ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.573+0000 7fe4267fc640 1 -- 172.21.15.76:0/926962007 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1797597288 0 0) 0x7fe4280b32b0 con 0x7fe42815d4e0 2023-11-09T03:55:00.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.573+0000 7fe4267fc640 1 -- 172.21.15.76:0/926962007 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe414003520 con 0x7fe42815d4e0 2023-11-09T03:55:00.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.573+0000 7fe4267fc640 1 -- 172.21.15.76:0/926962007 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 875168949 0 0) 0x7fe414003520 con 0x7fe42815d4e0 2023-11-09T03:55:00.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.573+0000 7fe4267fc640 1 -- 172.21.15.76:0/926962007 >> v2:172.21.15.113:3300/0 conn(0x7fe42815d100 msgr2=0x7fe4280ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:00.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.573+0000 7fe4267fc640 1 --2- 172.21.15.76:0/926962007 >> v2:172.21.15.113:3300/0 conn(0x7fe42815d100 0x7fe4280ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.573+0000 7fe4267fc640 1 -- 172.21.15.76:0/926962007 >> v1:172.21.15.113:6789/0 conn(0x7fe4280d1be0 legacy=0x7fe4280b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:00.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.573+0000 7fe4267fc640 1 -- 172.21.15.76:0/926962007 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4280b2c60 con 0x7fe42815d4e0 2023-11-09T03:55:00.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.573+0000 7fe4267fc640 1 -- 172.21.15.76:0/926962007 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fe410001670 con 0x7fe42815d4e0 2023-11-09T03:55:00.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.573+0000 7fe4267fc640 1 -- 172.21.15.76:0/926962007 >> v1:172.21.15.76:6789/0 conn(0x7fe42815d4e0 legacy=0x7fe4280d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:00.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.573+0000 7fe4267fc640 1 --2- 172.21.15.76:0/926962007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe414003970 0x7fe414003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.573+0000 7fe4267fc640 1 --2- 172.21.15.76:0/926962007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4140047c0 0x7fe414004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.573+0000 7fe4267fc640 1 --2- 172.21.15.76:0/926962007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe414005180 0x7fe41400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.574+0000 7fe4267fc640 1 -- 172.21.15.76:0/926962007 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4280b2980 con 0x7fe414005180 2023-11-09T03:55:00.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.574+0000 7fe4267fc640 1 -- 172.21.15.76:0/926962007 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe41400c020 con 0x7fe4140047c0 2023-11-09T03:55:00.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.574+0000 7fe4267fc640 1 -- 172.21.15.76:0/926962007 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe41400c1e0 con 0x7fe414003970 2023-11-09T03:55:00.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.574+0000 7fe4277fe640 1 --2- 172.21.15.76:0/926962007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe414005180 0x7fe41400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.579 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.574+0000 7fe41e7fc640 1 --2- 172.21.15.76:0/926962007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4140047c0 0x7fe414004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.574+0000 7fe426ffd640 1 --2- 172.21.15.76:0/926962007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe414003970 0x7fe414003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.574+0000 7fe4277fe640 1 -- 172.21.15.76:0/926962007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe414003970 msgr2=0x7fe414003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:00.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.574+0000 7fe4277fe640 1 --2- 172.21.15.76:0/926962007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe414003970 0x7fe414003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.574+0000 7fe4277fe640 1 -- 172.21.15.76:0/926962007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4140047c0 msgr2=0x7fe414004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:00.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.574+0000 7fe4277fe640 1 --2- 172.21.15.76:0/926962007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4140047c0 0x7fe414004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.580 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.574+0000 7fe4277fe640 1 -- 172.21.15.76:0/926962007 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe414003520 con 0x7fe414005180 2023-11-09T03:55:00.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.574+0000 7fe41e7fc640 1 --2- 172.21.15.76:0/926962007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4140047c0 0x7fe414004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:55:00.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.574+0000 7fe426ffd640 1 --2- 172.21.15.76:0/926962007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe414003970 0x7fe414003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:55:00.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.574+0000 7fe4277fe640 1 -- 172.21.15.76:0/926962007 reap_dead start 2023-11-09T03:55:00.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.575+0000 7fe4277fe640 1 --2- 172.21.15.76:0/926962007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe414005180 0x7fe41400b980 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fe420001e50 tx=0x7fe4140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a806fcd225d60d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:00.581 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.575+0000 7fe4267fc640 1 -- 172.21.15.76:0/926962007 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe420003b70 con 0x7fe414005180 2023-11-09T03:55:00.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.575+0000 7fe4267fc640 1 -- 172.21.15.76:0/926962007 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe420003d10 con 0x7fe414005180 2023-11-09T03:55:00.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.575+0000 7fe4267fc640 1 -- 172.21.15.76:0/926962007 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe420005310 con 0x7fe414005180 2023-11-09T03:55:00.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.575+0000 7fe42d489640 1 -- 172.21.15.76:0/926962007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe414005180 msgr2=0x7fe41400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:00.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.575+0000 7fe42d489640 1 --2- 172.21.15.76:0/926962007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe414005180 0x7fe41400b980 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fe420001e50 tx=0x7fe4140057f0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.575+0000 7fe42d489640 1 -- 172.21.15.76:0/926962007 shutdown_connections 2023-11-09T03:55:00.582 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.575+0000 7fe42d489640 1 --2- 172.21.15.76:0/926962007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe414005180 0x7fe41400b980 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.575+0000 7fe42d489640 1 -- 172.21.15.76:0/926962007 >> 172.21.15.76:0/926962007 conn(0x7fe4280d2ef0 msgr2=0x7fe4280d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:00.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.575+0000 7fe42d489640 1 -- 172.21.15.76:0/926962007 shutdown_connections 2023-11-09T03:55:00.583 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.575+0000 7fe42d489640 1 -- 172.21.15.76:0/926962007 wait complete. 2023-11-09T03:55:00.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.576+0000 7fe42d489640 1 Processor -- start 2023-11-09T03:55:00.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.577+0000 7fe42d489640 1 -- start start 2023-11-09T03:55:00.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.577+0000 7fe42d489640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe414005180 0x7fe4280cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.577+0000 7fe42d489640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4281522f0 0x7fe4280b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.577+0000 7fe42d489640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4280b1bf0 0x7fe4280ca030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.577+0000 7fe42d489640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4280b2070 con 0x7fe4280b1bf0 2023-11-09T03:55:00.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.577+0000 7fe42d489640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4280cb970 con 0x7fe4281522f0 2023-11-09T03:55:00.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.577+0000 7fe42d489640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4280cad60 con 0x7fe414005180 2023-11-09T03:55:00.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.577+0000 7fe4277fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4280b1bf0 0x7fe4280ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.577+0000 7fe4277fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4280b1bf0 0x7fe4280ca030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41012/0 (socket says 172.21.15.76:41012) 2023-11-09T03:55:00.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.577+0000 7fe4277fe640 1 -- 172.21.15.76:0/346478762 learned_addr learned my addr 172.21.15.76:0/346478762 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:00.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.578+0000 7fe426ffd640 1 --2- 172.21.15.76:0/346478762 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe414005180 0x7fe4280cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.578+0000 7fe41e7fc640 1 --2- 172.21.15.76:0/346478762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4281522f0 0x7fe4280b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.578+0000 7fe4277fe640 1 -- 172.21.15.76:0/346478762 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe414005180 msgr2=0x7fe4280cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:00.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.578+0000 7fe4277fe640 1 --2- 172.21.15.76:0/346478762 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe414005180 0x7fe4280cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.578+0000 7fe4277fe640 1 -- 172.21.15.76:0/346478762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4281522f0 msgr2=0x7fe4280b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:00.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.578+0000 7fe4277fe640 1 --2- 172.21.15.76:0/346478762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4281522f0 0x7fe4280b16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.578+0000 7fe4277fe640 1 -- 172.21.15.76:0/346478762 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4280cb040 con 0x7fe4280b1bf0 2023-11-09T03:55:00.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.578+0000 7fe4277fe640 1 --2- 172.21.15.76:0/346478762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4280b1bf0 0x7fe4280ca030 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fe42002c060 tx=0x7fe4200014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:00.594 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.578+0000 7fe41ffff640 1 -- 172.21.15.76:0/346478762 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe420006d50 con 0x7fe4280b1bf0 2023-11-09T03:55:00.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.578+0000 7fe41ffff640 1 -- 172.21.15.76:0/346478762 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe420007030 con 0x7fe4280b1bf0 2023-11-09T03:55:00.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.578+0000 7fe42d489640 1 -- 172.21.15.76:0/346478762 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4280cb320 con 0x7fe4280b1bf0 2023-11-09T03:55:00.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.578+0000 7fe41ffff640 1 -- 172.21.15.76:0/346478762 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4200073e0 con 0x7fe4280b1bf0 2023-11-09T03:55:00.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.578+0000 7fe42d489640 1 -- 172.21.15.76:0/346478762 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe428152750 con 0x7fe4280b1bf0 2023-11-09T03:55:00.595 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.579+0000 7fe41ffff640 1 -- 172.21.15.76:0/346478762 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe420007540 con 0x7fe4280b1bf0 2023-11-09T03:55:00.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.580+0000 7fe41ffff640 1 --2- 172.21.15.76:0/346478762 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe40806ae70 0x7fe40806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.580+0000 7fe41ffff640 1 -- 172.21.15.76:0/346478762 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7fe4200ac200 con 0x7fe4280b1bf0 2023-11-09T03:55:00.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.580+0000 7fe426ffd640 1 --2- 172.21.15.76:0/346478762 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe40806ae70 0x7fe40806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.581+0000 7fe43001d740 1 Processor -- start 2023-11-09T03:55:00.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.581+0000 7fe426ffd640 1 --2- 172.21.15.76:0/346478762 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe40806ae70 0x7fe40806d330 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fe418009980 tx=0x7fe418008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:00.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.581+0000 7fe43001d740 1 -- start start 2023-11-09T03:55:00.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.582+0000 7fe43001d740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561e19209850 0x561e191dc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.582+0000 7fe43001d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e19209c30 0x561e1920c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.582+0000 7fe43001d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561e191dcc90 0x561e1920cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.582+0000 7fe43001d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e190ed740 con 0x561e19209c30 2023-11-09T03:55:00.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.582+0000 7fe43001d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e1914f8d0 con 0x561e191dcc90 2023-11-09T03:55:00.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.582+0000 7fe43001d740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e1914cfc0 con 0x561e19209850 2023-11-09T03:55:00.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.582+0000 7fe41e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e19209c30 0x561e1920c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.582+0000 7fe41e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e19209c30 0x561e1920c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41024/0 (socket says 172.21.15.76:41024) 2023-11-09T03:55:00.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.582+0000 7fe41e7fc640 1 -- 172.21.15.76:0/2722788800 learned_addr learned my addr 172.21.15.76:0/2722788800 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:00.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.582+0000 7fe4277fe640 1 --2- 172.21.15.76:0/2722788800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561e191dcc90 0x561e1920cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.582+0000 7fe426ffd640 1 --2- 172.21.15.76:0/2722788800 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561e19209850 0x561e191dc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.582+0000 7fe41e7fc640 1 -- 172.21.15.76:0/2722788800 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561e19209850 msgr2=0x561e191dc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:00.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.582+0000 7fe41e7fc640 1 --2- 172.21.15.76:0/2722788800 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561e19209850 0x561e191dc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.582+0000 7fe41e7fc640 1 -- 172.21.15.76:0/2722788800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561e191dcc90 msgr2=0x561e1920cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:00.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.582+0000 7fe41e7fc640 1 --2- 172.21.15.76:0/2722788800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561e191dcc90 0x561e1920cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.582+0000 7fe41e7fc640 1 -- 172.21.15.76:0/2722788800 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e1910bca0 con 0x561e19209c30 2023-11-09T03:55:00.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.582+0000 7fe41e7fc640 1 --2- 172.21.15.76:0/2722788800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e19209c30 0x561e1920c5e0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fe410005950 tx=0x7fe410005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce85f81d40b3f3b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:00.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.582+0000 7fe41cff9640 1 -- 172.21.15.76:0/2722788800 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe410010a80 con 0x561e19209c30 2023-11-09T03:55:00.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.582+0000 7fe41cff9640 1 -- 172.21.15.76:0/2722788800 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe410010be0 con 0x561e19209c30 2023-11-09T03:55:00.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.582+0000 7fe41cff9640 1 -- 172.21.15.76:0/2722788800 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe410014400 con 0x561e19209c30 2023-11-09T03:55:00.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.583+0000 7fe43001d740 1 -- 172.21.15.76:0/2722788800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e19209c30 msgr2=0x561e1920c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:00.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.583+0000 7fe43001d740 1 --2- 172.21.15.76:0/2722788800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e19209c30 0x561e1920c5e0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fe410005950 tx=0x7fe410005e10 comp rx=0 tx=0).stop 2023-11-09T03:55:00.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.583+0000 7fe43001d740 1 -- 172.21.15.76:0/2722788800 shutdown_connections 2023-11-09T03:55:00.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.583+0000 7fe43001d740 1 --2- 172.21.15.76:0/2722788800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561e191dcc90 0x561e1920cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.583+0000 7fe43001d740 1 --2- 172.21.15.76:0/2722788800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e19209c30 0x561e1920c5e0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.583+0000 7fe43001d740 1 --2- 172.21.15.76:0/2722788800 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561e19209850 0x561e191dc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.583+0000 7fe43001d740 1 -- 172.21.15.76:0/2722788800 >> 172.21.15.76:0/2722788800 conn(0x561e191dea60 msgr2=0x561e192275e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:00.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.583+0000 7fe43001d740 1 -- 172.21.15.76:0/2722788800 shutdown_connections 2023-11-09T03:55:00.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.583+0000 7fe43001d740 1 -- 172.21.15.76:0/2722788800 wait complete. 2023-11-09T03:55:00.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.584+0000 7fe43001d740 1 -- start start 2023-11-09T03:55:00.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.584+0000 7fe43001d740 1 Processor -- start 2023-11-09T03:55:00.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.584+0000 7fe43001d740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561e19209850 0x561e19210090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.584+0000 7fe43001d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e19209c30 0x561e192206f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.584+0000 7fe43001d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561e19207300 0x561e19207780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.584+0000 7fe43001d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e191488b0 con 0x561e19209c30 2023-11-09T03:55:00.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.584+0000 7fe43001d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e19102c80 con 0x561e19207300 2023-11-09T03:55:00.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.584+0000 7fe43001d740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e19102920 con 0x561e19209850 2023-11-09T03:55:00.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.584+0000 7fe43001d740 10 client.0 _ll_register_callbacks cb 0x561e18e5aac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:55:00.604 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.584+0000 7fe41e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e19209c30 0x561e192206f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.585+0000 7fe41e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e19209c30 0x561e192206f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41030/0 (socket says 172.21.15.76:41030) 2023-11-09T03:55:00.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.585+0000 7fe41e7fc640 1 -- 172.21.15.76:0/2633270050 learned_addr learned my addr 172.21.15.76:0/2633270050 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:00.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.585+0000 7fe4277fe640 1 --2- 172.21.15.76:0/2633270050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561e19207300 0x561e19207780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.585+0000 7fe426ffd640 1 --2- 172.21.15.76:0/2633270050 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561e19209850 0x561e19210090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.585+0000 7fe41e7fc640 1 -- 172.21.15.76:0/2633270050 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561e19209850 msgr2=0x561e19210090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:00.605 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.585+0000 7fe41e7fc640 1 --2- 172.21.15.76:0/2633270050 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561e19209850 0x561e19210090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.585+0000 7fe41e7fc640 1 -- 172.21.15.76:0/2633270050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561e19207300 msgr2=0x561e19207780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:00.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.585+0000 7fe41e7fc640 1 --2- 172.21.15.76:0/2633270050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561e19207300 0x561e19207780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.585+0000 7fe41e7fc640 1 -- 172.21.15.76:0/2633270050 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe41000f040 con 0x561e19209c30 2023-11-09T03:55:00.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.585+0000 7fe41e7fc640 1 --2- 172.21.15.76:0/2633270050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e19209c30 0x561e192206f0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fe410006fd0 tx=0x7fe410001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63fb2c1b6e2e3f9a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:00.606 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.585+0000 7fe405ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:55:00.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.585+0000 7fe405ffb640 1 -- 172.21.15.76:0/2633270050 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe410028630 con 0x561e19209c30 2023-11-09T03:55:00.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.585+0000 7fe43001d740 1 -- 172.21.15.76:0/2633270050 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e190f4bf0 con 0x561e19209c30 2023-11-09T03:55:00.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.585+0000 7fe405ffb640 1 -- 172.21.15.76:0/2633270050 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7fe41001f410 con 0x561e19209c30 2023-11-09T03:55:00.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.585+0000 7fe405ffb640 1 -- 172.21.15.76:0/2633270050 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=133) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe410028790 con 0x561e19209c30 2023-11-09T03:55:00.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.585+0000 7fe43001d740 10 client.9339 fetch_fsmap learned FSMap version 133 2023-11-09T03:55:00.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.585+0000 7fe43001d740 1 -- 172.21.15.76:0/2633270050 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=133,monmap=2+}) v3 -- 0x561e19146f30 con 0x561e19209c30 2023-11-09T03:55:00.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.585+0000 7fe405ffb640 1 -- 172.21.15.76:0/2633270050 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561e190f4bf0 con 0x561e19209c30 2023-11-09T03:55:00.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.585+0000 7fe405ffb640 1 -- 172.21.15.76:0/2633270050 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 133) v1 ==== 3306+0+0 (secure 0 0 0) 0x7fe4100288f0 con 0x561e19209c30 2023-11-09T03:55:00.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.585+0000 7fe43001d740 10 client.9339 fetch_fsmap finished waiting for FSMap version 133 2023-11-09T03:55:00.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.586+0000 7fe43001d740 10 client.9339 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-09T03:55:00.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.586+0000 7fe43001d740 20 client.9339 populate_metadata read hostname 'smithi076' 2023-11-09T03:55:00.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.586+0000 7fe43001d740 1 --2- 172.21.15.76:0/2633270050 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x561e19242be0 0x561e19244fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.586+0000 7fe43001d740 4 client.9339 mds_command: new command op to 8568 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:55:00.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.586+0000 7fe43001d740 1 -- 172.21.15.76:0/2633270050 --> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e19141890 con 0x561e19242be0 2023-11-09T03:55:00.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.586+0000 7fe426ffd640 1 --2- 172.21.15.76:0/2633270050 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x561e19242be0 0x561e19244fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.586+0000 7fe405ffb640 10 client.9339 ms_handle_connect on v2:172.21.15.76:6832/153465374 2023-11-09T03:55:00.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.586+0000 7fe426ffd640 1 --2- 172.21.15.76:0/2633270050 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x561e19242be0 0x561e19244fd0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6b35ceb3868793c server_cookie=ead14ec47bd3154d in_seq=0 out_seq=0 2023-11-09T03:55:00.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.586+0000 7fe405ffb640 1 -- 172.21.15.76:0/2633270050 <== mds.0 v2:172.21.15.76:6832/153465374 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561e19141890 con 0x561e19242be0 2023-11-09T03:55:00.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.586+0000 7fe405ffb640 10 client.9339 handle_command_reply: tid=0 2023-11-09T03:55:00.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.586+0000 7fe43001d740 1 client.9339 shutdown 2023-11-09T03:55:00.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.587+0000 7fe43001d740 1 -- 172.21.15.76:0/2633270050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e19209c30 msgr2=0x561e192206f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:00.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.587+0000 7fe43001d740 1 --2- 172.21.15.76:0/2633270050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e19209c30 0x561e192206f0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fe410006fd0 tx=0x7fe410001a00 comp rx=0 tx=0).stop 2023-11-09T03:55:00.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.587+0000 7fe43001d740 1 -- 172.21.15.76:0/2633270050 shutdown_connections 2023-11-09T03:55:00.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.587+0000 7fe43001d740 1 --2- 172.21.15.76:0/2633270050 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x561e19242be0 0x561e19244fd0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.587+0000 7fe43001d740 1 --2- 172.21.15.76:0/2633270050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561e19207300 0x561e19207780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.587+0000 7fe405ffb640 0 client.9339 ms_handle_reset on v2:172.21.15.76:6832/153465374 2023-11-09T03:55:00.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.587+0000 7fe43001d740 1 --2- 172.21.15.76:0/2633270050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e19209c30 0x561e192206f0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.587+0000 7fe43001d740 1 --2- 172.21.15.76:0/2633270050 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561e19209850 0x561e19210090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.587+0000 7fe43001d740 1 -- 172.21.15.76:0/2633270050 >> 172.21.15.76:0/2633270050 conn(0x561e191dcc90 msgr2=0x561e191e06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:00.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.587+0000 7fe43001d740 1 -- 172.21.15.76:0/2633270050 shutdown_connections 2023-11-09T03:55:00.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.587+0000 7fe43001d740 1 -- 172.21.15.76:0/2633270050 wait complete. 2023-11-09T03:55:00.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.587+0000 7fe43001d740 20 client.9339 trim_cache size 0 max 16384 2023-11-09T03:55:00.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.602+0000 7fe43001d740 1 Processor -- start 2023-11-09T03:55:00.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.602+0000 7fe43001d740 1 -- start start 2023-11-09T03:55:00.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.603+0000 7fe43001d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561e191dcc90 0x561e19208b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.603+0000 7fe43001d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e19207300 0x561e191db4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.603+0000 7fe43001d740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561e19209850 0x561e191dba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.603+0000 7fe43001d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e190c8280 con 0x561e19207300 2023-11-09T03:55:00.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.603+0000 7fe43001d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e191d7c10 con 0x561e191dcc90 2023-11-09T03:55:00.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.603+0000 7fe43001d740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e191479b0 con 0x561e19209850 2023-11-09T03:55:00.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.603+0000 7fe41e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e19207300 0x561e191db4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.603+0000 7fe41e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e19207300 0x561e191db4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41040/0 (socket says 172.21.15.76:41040) 2023-11-09T03:55:00.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.603+0000 7fe41e7fc640 1 -- 172.21.15.76:0/1147529294 learned_addr learned my addr 172.21.15.76:0/1147529294 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:00.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.603+0000 7fe41e7fc640 1 -- 172.21.15.76:0/1147529294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561e19209850 msgr2=0x561e191dba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:00.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.603+0000 7fe426ffd640 1 --2- 172.21.15.76:0/1147529294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561e191dcc90 0x561e19208b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.603+0000 7fe4277fe640 1 --2- 172.21.15.76:0/1147529294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561e19209850 0x561e191dba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.603+0000 7fe41e7fc640 1 --2- 172.21.15.76:0/1147529294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561e19209850 0x561e191dba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.603+0000 7fe41e7fc640 1 -- 172.21.15.76:0/1147529294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561e191dcc90 msgr2=0x561e19208b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:00.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.603+0000 7fe41e7fc640 1 --2- 172.21.15.76:0/1147529294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561e191dcc90 0x561e19208b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.603+0000 7fe41e7fc640 1 -- 172.21.15.76:0/1147529294 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e190e4d10 con 0x561e19207300 2023-11-09T03:55:00.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.603+0000 7fe426ffd640 1 --2- 172.21.15.76:0/1147529294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561e191dcc90 0x561e19208b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:55:00.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.603+0000 7fe4277fe640 1 --2- 172.21.15.76:0/1147529294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561e19209850 0x561e191dba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:55:00.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.603+0000 7fe41e7fc640 1 --2- 172.21.15.76:0/1147529294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e19207300 0x561e191db4f0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fe410001820 tx=0x7fe410021070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5860e2599fd6dd45 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:00.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.603+0000 7fe41d7fa640 1 -- 172.21.15.76:0/1147529294 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe410028630 con 0x561e19207300 2023-11-09T03:55:00.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.604+0000 7fe41d7fa640 1 -- 172.21.15.76:0/1147529294 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe410023070 con 0x561e19207300 2023-11-09T03:55:00.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.604+0000 7fe41d7fa640 1 -- 172.21.15.76:0/1147529294 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4100289b0 con 0x561e19207300 2023-11-09T03:55:00.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.604+0000 7fe43001d740 1 -- 172.21.15.76:0/1147529294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e19207300 msgr2=0x561e191db4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:00.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.604+0000 7fe43001d740 1 --2- 172.21.15.76:0/1147529294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e19207300 0x561e191db4f0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fe410001820 tx=0x7fe410021070 comp rx=0 tx=0).stop 2023-11-09T03:55:00.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.604+0000 7fe43001d740 1 -- 172.21.15.76:0/1147529294 shutdown_connections 2023-11-09T03:55:00.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.604+0000 7fe43001d740 1 --2- 172.21.15.76:0/1147529294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561e19209850 0x561e191dba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.604+0000 7fe43001d740 1 --2- 172.21.15.76:0/1147529294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e19207300 0x561e191db4f0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.604+0000 7fe43001d740 1 --2- 172.21.15.76:0/1147529294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561e191dcc90 0x561e19208b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.604+0000 7fe43001d740 1 -- 172.21.15.76:0/1147529294 >> 172.21.15.76:0/1147529294 conn(0x561e191e2cc0 msgr2=0x561e19221240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:00.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.604+0000 7fe43001d740 1 -- 172.21.15.76:0/1147529294 shutdown_connections 2023-11-09T03:55:00.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.604+0000 7fe43001d740 1 -- 172.21.15.76:0/1147529294 wait complete. 2023-11-09T03:55:00.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.605+0000 7fe43001d740 1 -- start start 2023-11-09T03:55:00.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.605+0000 7fe43001d740 1 Processor -- start 2023-11-09T03:55:00.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.605+0000 7fe43001d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561e191dcc90 0x561e19206250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.605+0000 7fe43001d740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561e19207300 0x561e191db4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.605+0000 7fe43001d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e19209850 0x561e191dba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.605+0000 7fe43001d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e1922a290 con 0x561e19209850 2023-11-09T03:55:00.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.605+0000 7fe43001d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e19205840 con 0x561e191dcc90 2023-11-09T03:55:00.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.605+0000 7fe43001d740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e19206790 con 0x561e19207300 2023-11-09T03:55:00.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.605+0000 7fe43001d740 10 client.0 _ll_register_callbacks cb 0x7fe420006eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:55:00.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.605+0000 7fe4277fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e19209850 0x561e191dba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.605+0000 7fe4277fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e19209850 0x561e191dba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41046/0 (socket says 172.21.15.76:41046) 2023-11-09T03:55:00.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.605+0000 7fe4277fe640 1 -- 172.21.15.76:0/1405083616 learned_addr learned my addr 172.21.15.76:0/1405083616 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:00.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.606+0000 7fe426ffd640 1 --2- 172.21.15.76:0/1405083616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561e191dcc90 0x561e19206250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.606+0000 7fe41e7fc640 1 --2- 172.21.15.76:0/1405083616 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561e19207300 0x561e191db4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.606+0000 7fe4277fe640 1 -- 172.21.15.76:0/1405083616 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561e19207300 msgr2=0x561e191db4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:00.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.606+0000 7fe4277fe640 1 --2- 172.21.15.76:0/1405083616 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561e19207300 0x561e191db4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.606+0000 7fe4277fe640 1 -- 172.21.15.76:0/1405083616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561e191dcc90 msgr2=0x561e19206250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:00.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.606+0000 7fe4277fe640 1 --2- 172.21.15.76:0/1405083616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561e191dcc90 0x561e19206250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.606+0000 7fe4277fe640 1 -- 172.21.15.76:0/1405083616 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e19146f30 con 0x561e19209850 2023-11-09T03:55:00.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.606+0000 7fe4277fe640 1 --2- 172.21.15.76:0/1405083616 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e19209850 0x561e191dba30 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fe42007a320 tx=0x7fe42007a220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac391fdd0409e4f server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:00.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.606+0000 7fe405ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:55:00.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.606+0000 7fe405ffb640 1 -- 172.21.15.76:0/1405083616 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42007e4e0 con 0x561e19209850 2023-11-09T03:55:00.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.606+0000 7fe405ffb640 1 -- 172.21.15.76:0/1405083616 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7fe420082720 con 0x561e19209850 2023-11-09T03:55:00.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.606+0000 7fe43001d740 1 -- 172.21.15.76:0/1405083616 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe410010be0 con 0x561e19209850 2023-11-09T03:55:00.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.606+0000 7fe405ffb640 1 -- 172.21.15.76:0/1405083616 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=133) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4200a9cf0 con 0x561e19209850 2023-11-09T03:55:00.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.606+0000 7fe43001d740 10 client.9345 fetch_fsmap learned FSMap version 133 2023-11-09T03:55:00.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.606+0000 7fe43001d740 1 -- 172.21.15.76:0/1405083616 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=133,monmap=2+}) v3 -- 0x561e1910a500 con 0x561e19209850 2023-11-09T03:55:00.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.606+0000 7fe405ffb640 1 -- 172.21.15.76:0/1405083616 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe410010be0 con 0x561e19209850 2023-11-09T03:55:00.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.606+0000 7fe405ffb640 1 -- 172.21.15.76:0/1405083616 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 133) v1 ==== 3306+0+0 (secure 0 0 0) 0x7fe420035070 con 0x561e19209850 2023-11-09T03:55:00.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.606+0000 7fe43001d740 10 client.9345 fetch_fsmap finished waiting for FSMap version 133 2023-11-09T03:55:00.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.606+0000 7fe43001d740 10 client.9345 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-09T03:55:00.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.606+0000 7fe43001d740 20 client.9345 populate_metadata read hostname 'smithi076' 2023-11-09T03:55:00.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.606+0000 7fe43001d740 1 --2- 172.21.15.76:0/1405083616 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x561e19209c30 0x561e191e1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.606+0000 7fe43001d740 4 client.9345 mds_command: new command op to 8568 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-09T03:55:00.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.606+0000 7fe43001d740 1 -- 172.21.15.76:0/1405083616 --> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561e191415b0 con 0x561e19209c30 2023-11-09T03:55:00.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.606+0000 7fe426ffd640 1 --2- 172.21.15.76:0/1405083616 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x561e19209c30 0x561e191e1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.607+0000 7fe405ffb640 10 client.9345 ms_handle_connect on v2:172.21.15.76:6832/153465374 2023-11-09T03:55:00.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.607+0000 7fe426ffd640 1 --2- 172.21.15.76:0/1405083616 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x561e19209c30 0x561e191e1680 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d25138c15b88d4a8 server_cookie=702f70a5b45cfba8 in_seq=0 out_seq=0 2023-11-09T03:55:00.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.607+0000 7fe405ffb640 1 -- 172.21.15.76:0/1405083616 <== mds.0 v2:172.21.15.76:6832/153465374 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x561e191415b0 con 0x561e19209c30 2023-11-09T03:55:00.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.607+0000 7fe405ffb640 10 client.9345 handle_command_reply: tid=0 2023-11-09T03:55:00.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.607+0000 7fe43001d740 1 client.9345 shutdown 2023-11-09T03:55:00.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.607+0000 7fe43001d740 1 -- 172.21.15.76:0/1405083616 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e19209850 msgr2=0x561e191dba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:00.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.607+0000 7fe43001d740 1 --2- 172.21.15.76:0/1405083616 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e19209850 0x561e191dba30 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fe42007a320 tx=0x7fe42007a220 comp rx=0 tx=0).stop 2023-11-09T03:55:00.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.607+0000 7fe43001d740 1 -- 172.21.15.76:0/1405083616 shutdown_connections 2023-11-09T03:55:00.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.607+0000 7fe43001d740 1 --2- 172.21.15.76:0/1405083616 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x561e19209c30 0x561e191e1680 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.607+0000 7fe43001d740 1 --2- 172.21.15.76:0/1405083616 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e19209850 0x561e191dba30 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.607+0000 7fe405ffb640 0 client.9345 ms_handle_reset on v2:172.21.15.76:6832/153465374 2023-11-09T03:55:00.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.607+0000 7fe43001d740 1 --2- 172.21.15.76:0/1405083616 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561e19207300 0x561e191db4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.607+0000 7fe43001d740 1 --2- 172.21.15.76:0/1405083616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561e191dcc90 0x561e19206250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.607+0000 7fe43001d740 1 -- 172.21.15.76:0/1405083616 >> 172.21.15.76:0/1405083616 conn(0x561e191e2cc0 msgr2=0x561e1920c580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:00.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.607+0000 7fe43001d740 1 -- 172.21.15.76:0/1405083616 shutdown_connections 2023-11-09T03:55:00.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.607+0000 7fe43001d740 1 -- 172.21.15.76:0/1405083616 wait complete. 2023-11-09T03:55:00.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.607+0000 7fe43001d740 20 client.9345 trim_cache size 0 max 16384 2023-11-09T03:55:00.634 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:55:00.634 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "c213b26f-d0da-45d5-8e34-bb75c096d4f2", 2023-11-09T03:55:00.634 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2023-11-09T03:55:00.634 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 8568, 2023-11-09T03:55:00.635 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2023-11-09T03:55:00.635 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2023-11-09T03:55:00.635 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2023-11-09T03:55:00.635 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 103.12518466100001, 2023-11-09T03:55:00.635 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 133, 2023-11-09T03:55:00.635 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 118, 2023-11-09T03:55:00.636 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 118, 2023-11-09T03:55:00.636 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 146.29521388800001 2023-11-09T03:55:00.636 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:55:00.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.608+0000 7fe42d489640 1 -- 172.21.15.76:0/346478762 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe40806ae70 msgr2=0x7fe40806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:00.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.608+0000 7fe42d489640 1 --2- 172.21.15.76:0/346478762 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe40806ae70 0x7fe40806d330 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fe418009980 tx=0x7fe418008040 comp rx=0 tx=0).stop 2023-11-09T03:55:00.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.608+0000 7fe42d489640 1 -- 172.21.15.76:0/346478762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4280b1bf0 msgr2=0x7fe4280ca030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:00.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.608+0000 7fe42d489640 1 --2- 172.21.15.76:0/346478762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4280b1bf0 0x7fe4280ca030 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fe42002c060 tx=0x7fe4200014a0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.608+0000 7fe42d489640 1 -- 172.21.15.76:0/346478762 shutdown_connections 2023-11-09T03:55:00.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.608+0000 7fe42d489640 1 --2- 172.21.15.76:0/346478762 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe40806ae70 0x7fe40806d330 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.608+0000 7fe42d489640 1 --2- 172.21.15.76:0/346478762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4280b1bf0 0x7fe4280ca030 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.608+0000 7fe42d489640 1 --2- 172.21.15.76:0/346478762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4281522f0 0x7fe4280b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.608+0000 7fe42d489640 1 --2- 172.21.15.76:0/346478762 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe414005180 0x7fe4280cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.608+0000 7fe42d489640 1 -- 172.21.15.76:0/346478762 >> 172.21.15.76:0/346478762 conn(0x7fe4280d2ef0 msgr2=0x7fe4280d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:00.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.608+0000 7fe42d489640 1 -- 172.21.15.76:0/346478762 shutdown_connections 2023-11-09T03:55:00.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.608+0000 7fe42d489640 1 -- 172.21.15.76:0/346478762 wait complete. 2023-11-09T03:55:00.640 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-09T03:55:00.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.741+0000 7fae9b905640 1 Processor -- start 2023-11-09T03:55:00.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.741+0000 7fae9b905640 1 -- start start 2023-11-09T03:55:00.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.742+0000 7fae9b905640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fae940addd0 0x7fae940cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.742+0000 7fae9b905640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae940cc680 0x7fae940ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.742+0000 7fae9b905640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae940cd720 con 0x7fae9406bb10 2023-11-09T03:55:00.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.742+0000 7fae9b905640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae940ae1b0 con 0x7fae940addd0 2023-11-09T03:55:00.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.742+0000 7fae9b905640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae940ccb10 con 0x7fae940cc680 2023-11-09T03:55:00.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.742+0000 7fae99ea2640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fae9406bb10 0x7fae9406f8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45468/0 (socket says 172.21.15.76:45468) 2023-11-09T03:55:00.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.742+0000 7fae99ea2640 1 -- 172.21.15.76:0/1052153834 learned_addr learned my addr 172.21.15.76:0/1052153834 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:55:00.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.742+0000 7fae996a1640 1 --2- 172.21.15.76:0/1052153834 >> v2:172.21.15.113:3300/0 conn(0x7fae940addd0 0x7fae940cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.742+0000 7fae98ea0640 1 --2- 172.21.15.76:0/1052153834 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae940cc680 0x7fae940ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.742+0000 7fae8bfff640 1 -- 172.21.15.76:0/1052153834 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 146736800 0 0) 0x7fae940cd720 con 0x7fae9406bb10 2023-11-09T03:55:00.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.742+0000 7fae8bfff640 1 -- 172.21.15.76:0/1052153834 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae80003520 con 0x7fae9406bb10 2023-11-09T03:55:00.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.742+0000 7fae8bfff640 1 -- 172.21.15.76:0/1052153834 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4189402274 0 0) 0x7fae80003520 con 0x7fae9406bb10 2023-11-09T03:55:00.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.742+0000 7fae8bfff640 1 -- 172.21.15.76:0/1052153834 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae940cc680 msgr2=0x7fae940ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:00.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.742+0000 7fae8bfff640 1 --2- 172.21.15.76:0/1052153834 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae940cc680 0x7fae940ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.742+0000 7fae8bfff640 1 -- 172.21.15.76:0/1052153834 >> v2:172.21.15.113:3300/0 conn(0x7fae940addd0 msgr2=0x7fae940cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:00.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.742+0000 7fae8bfff640 1 --2- 172.21.15.76:0/1052153834 >> v2:172.21.15.113:3300/0 conn(0x7fae940addd0 0x7fae940cdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.742+0000 7fae8bfff640 1 -- 172.21.15.76:0/1052153834 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae940ccdf0 con 0x7fae9406bb10 2023-11-09T03:55:00.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.742+0000 7fae8bfff640 1 -- 172.21.15.76:0/1052153834 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fae8c0019f0 con 0x7fae9406bb10 2023-11-09T03:55:00.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.742+0000 7fae996a1640 1 --2- 172.21.15.76:0/1052153834 >> v2:172.21.15.113:3300/0 conn(0x7fae940addd0 0x7fae940cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:55:00.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.742+0000 7fae8bfff640 1 -- 172.21.15.76:0/1052153834 >> v1:172.21.15.76:6789/0 conn(0x7fae9406bb10 legacy=0x7fae9406f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:00.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.742+0000 7fae8bfff640 1 --2- 172.21.15.76:0/1052153834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae80003970 0x7fae80003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.742+0000 7fae8bfff640 1 --2- 172.21.15.76:0/1052153834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae800047c0 0x7fae80004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.742+0000 7fae8bfff640 1 --2- 172.21.15.76:0/1052153834 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae80005180 0x7fae8000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.742+0000 7fae8bfff640 1 -- 172.21.15.76:0/1052153834 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae940ae1b0 con 0x7fae80003970 2023-11-09T03:55:00.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.742+0000 7fae8bfff640 1 -- 172.21.15.76:0/1052153834 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae940ccb10 con 0x7fae800047c0 2023-11-09T03:55:00.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.742+0000 7fae8bfff640 1 -- 172.21.15.76:0/1052153834 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae80004560 con 0x7fae80005180 2023-11-09T03:55:00.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.742+0000 7fae996a1640 1 --2- 172.21.15.76:0/1052153834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae80003970 0x7fae80003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.749 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.743+0000 7fae98ea0640 1 --2- 172.21.15.76:0/1052153834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae800047c0 0x7fae80004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.743+0000 7fae99ea2640 1 --2- 172.21.15.76:0/1052153834 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae80005180 0x7fae8000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.743+0000 7fae996a1640 1 -- 172.21.15.76:0/1052153834 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae80005180 msgr2=0x7fae8000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:00.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.743+0000 7fae996a1640 1 --2- 172.21.15.76:0/1052153834 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae80005180 0x7fae8000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.743+0000 7fae996a1640 1 -- 172.21.15.76:0/1052153834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae800047c0 msgr2=0x7fae80004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:00.750 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.743+0000 7fae996a1640 1 --2- 172.21.15.76:0/1052153834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae800047c0 0x7fae80004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.743+0000 7fae996a1640 1 -- 172.21.15.76:0/1052153834 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae80003520 con 0x7fae80003970 2023-11-09T03:55:00.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.743+0000 7fae99ea2640 1 --2- 172.21.15.76:0/1052153834 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae80005180 0x7fae8000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:55:00.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.743+0000 7fae996a1640 1 --2- 172.21.15.76:0/1052153834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae80003970 0x7fae80003d90 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fae90009c90 tx=0x7fae800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da863ff2dd8b45e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:00.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.744+0000 7fae8bfff640 1 -- 172.21.15.76:0/1052153834 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9002c980 con 0x7fae80003970 2023-11-09T03:55:00.751 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.744+0000 7fae8bfff640 1 -- 172.21.15.76:0/1052153834 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9002cae0 con 0x7fae80003970 2023-11-09T03:55:00.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.744+0000 7fae8bfff640 1 -- 172.21.15.76:0/1052153834 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9002ce90 con 0x7fae80003970 2023-11-09T03:55:00.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.744+0000 7fae9b905640 1 -- 172.21.15.76:0/1052153834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae80003970 msgr2=0x7fae80003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:00.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.744+0000 7fae9b905640 1 --2- 172.21.15.76:0/1052153834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae80003970 0x7fae80003d90 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fae90009c90 tx=0x7fae800057f0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.744+0000 7fae99ea2640 1 -- 172.21.15.76:0/1052153834 reap_dead start 2023-11-09T03:55:00.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.744+0000 7fae9b905640 1 -- 172.21.15.76:0/1052153834 shutdown_connections 2023-11-09T03:55:00.752 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.744+0000 7fae9b905640 1 -- 172.21.15.76:0/1052153834 >> 172.21.15.76:0/1052153834 conn(0x7fae940cb530 msgr2=0x7fae940cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:00.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.744+0000 7fae9b905640 1 -- 172.21.15.76:0/1052153834 shutdown_connections 2023-11-09T03:55:00.753 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.744+0000 7fae9b905640 1 -- 172.21.15.76:0/1052153834 wait complete. 2023-11-09T03:55:00.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.745+0000 7fae9b905640 1 Processor -- start 2023-11-09T03:55:00.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.745+0000 7fae9b905640 1 -- start start 2023-11-09T03:55:00.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.746+0000 7fae9b905640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae80003970 0x7fae94070200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.746+0000 7fae9b905640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae9414e380 0x7fae94159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.761 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.746+0000 7fae9b905640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae9414c8b0 0x7fae9415a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.746+0000 7fae9b905640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae94154e40 con 0x7fae9414c8b0 2023-11-09T03:55:00.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.746+0000 7fae9b905640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9414cee0 con 0x7fae9414e380 2023-11-09T03:55:00.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.746+0000 7fae9b905640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9414d1c0 con 0x7fae80003970 2023-11-09T03:55:00.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.746+0000 7fae99ea2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae9414c8b0 0x7fae9415a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.746+0000 7fae99ea2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae9414c8b0 0x7fae9415a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41072/0 (socket says 172.21.15.76:41072) 2023-11-09T03:55:00.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.746+0000 7fae99ea2640 1 -- 172.21.15.76:0/3063709612 learned_addr learned my addr 172.21.15.76:0/3063709612 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:00.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.746+0000 7fae996a1640 1 --2- 172.21.15.76:0/3063709612 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae80003970 0x7fae94070200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.746+0000 7fae98ea0640 1 --2- 172.21.15.76:0/3063709612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae9414e380 0x7fae94159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.746+0000 7fae99ea2640 1 -- 172.21.15.76:0/3063709612 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae80003970 msgr2=0x7fae94070200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:00.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.746+0000 7fae99ea2640 1 --2- 172.21.15.76:0/3063709612 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae80003970 0x7fae94070200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.746+0000 7fae99ea2640 1 -- 172.21.15.76:0/3063709612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae9414e380 msgr2=0x7fae94159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:00.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.746+0000 7fae99ea2640 1 --2- 172.21.15.76:0/3063709612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae9414e380 0x7fae94159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.746+0000 7fae99ea2640 1 -- 172.21.15.76:0/3063709612 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9414d4a0 con 0x7fae9414c8b0 2023-11-09T03:55:00.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.746+0000 7fae98ea0640 1 --2- 172.21.15.76:0/3063709612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae9414e380 0x7fae94159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:55:00.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.746+0000 7fae996a1640 1 --2- 172.21.15.76:0/3063709612 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae80003970 0x7fae94070200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:55:00.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.747+0000 7fae99ea2640 1 --2- 172.21.15.76:0/3063709612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae9414c8b0 0x7fae9415a060 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fae940ce9e0 tx=0x7fae8c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:00.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.747+0000 7fae89ffb640 1 -- 172.21.15.76:0/3063709612 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c00eb90 con 0x7fae9414c8b0 2023-11-09T03:55:00.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.747+0000 7fae89ffb640 1 -- 172.21.15.76:0/3063709612 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8c00ed10 con 0x7fae9414c8b0 2023-11-09T03:55:00.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.747+0000 7fae89ffb640 1 -- 172.21.15.76:0/3063709612 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c013660 con 0x7fae9414c8b0 2023-11-09T03:55:00.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.747+0000 7fae9b905640 1 -- 172.21.15.76:0/3063709612 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae9414d780 con 0x7fae9414c8b0 2023-11-09T03:55:00.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.747+0000 7fae9b905640 1 -- 172.21.15.76:0/3063709612 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae9414df80 con 0x7fae9414c8b0 2023-11-09T03:55:00.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.748+0000 7fae89ffb640 1 -- 172.21.15.76:0/3063709612 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fae8c0137c0 con 0x7fae9414c8b0 2023-11-09T03:55:00.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.749+0000 7fae89ffb640 1 --2- 172.21.15.76:0/3063709612 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae7806ae70 0x7fae7806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.749+0000 7fae89ffb640 1 -- 172.21.15.76:0/3063709612 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7fae8c08a4b0 con 0x7fae9414c8b0 2023-11-09T03:55:00.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.749+0000 7fae996a1640 1 --2- 172.21.15.76:0/3063709612 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae7806ae70 0x7fae7806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.750+0000 7fae996a1640 1 --2- 172.21.15.76:0/3063709612 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae7806ae70 0x7fae7806d330 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fae90009c90 tx=0x7fae90030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:00.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.750+0000 7fae9e56a740 1 Processor -- start 2023-11-09T03:55:00.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.750+0000 7fae9e56a740 1 -- start start 2023-11-09T03:55:00.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.751+0000 7fae9e56a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba99639850 0x55ba9960c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.751+0000 7fae9e56a740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba99639c30 0x55ba9963c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.751+0000 7fae9e56a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba9960cc90 0x55ba9963cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.751+0000 7fae9e56a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba9951d740 con 0x55ba9960cc90 2023-11-09T03:55:00.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.751+0000 7fae9e56a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba9957f8d0 con 0x55ba99639850 2023-11-09T03:55:00.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.751+0000 7fae9e56a740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba9957cfc0 con 0x55ba99639c30 2023-11-09T03:55:00.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.751+0000 7fae99ea2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba9960cc90 0x55ba9963cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.751+0000 7fae99ea2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba9960cc90 0x55ba9963cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41074/0 (socket says 172.21.15.76:41074) 2023-11-09T03:55:00.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.751+0000 7fae99ea2640 1 -- 172.21.15.76:0/3817191395 learned_addr learned my addr 172.21.15.76:0/3817191395 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:00.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.751+0000 7fae996a1640 1 --2- 172.21.15.76:0/3817191395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba99639850 0x55ba9960c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.751+0000 7fae98ea0640 1 --2- 172.21.15.76:0/3817191395 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba99639c30 0x55ba9963c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.751+0000 7fae99ea2640 1 -- 172.21.15.76:0/3817191395 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba99639c30 msgr2=0x55ba9963c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:00.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.751+0000 7fae99ea2640 1 --2- 172.21.15.76:0/3817191395 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba99639c30 0x55ba9963c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.751+0000 7fae99ea2640 1 -- 172.21.15.76:0/3817191395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba99639850 msgr2=0x55ba9960c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:00.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.751+0000 7fae99ea2640 1 --2- 172.21.15.76:0/3817191395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba99639850 0x55ba9960c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.751+0000 7fae99ea2640 1 -- 172.21.15.76:0/3817191395 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba9953bca0 con 0x55ba9960cc90 2023-11-09T03:55:00.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.751+0000 7fae99ea2640 1 --2- 172.21.15.76:0/3817191395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba9960cc90 0x55ba9963cb20 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fae8c060500 tx=0x7fae8c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2bdb3463d95aa6a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:00.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.751+0000 7fae76ffd640 1 -- 172.21.15.76:0/3817191395 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c05c880 con 0x55ba9960cc90 2023-11-09T03:55:00.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.751+0000 7fae76ffd640 1 -- 172.21.15.76:0/3817191395 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8c060a30 con 0x55ba9960cc90 2023-11-09T03:55:00.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.751+0000 7fae76ffd640 1 -- 172.21.15.76:0/3817191395 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c054e40 con 0x55ba9960cc90 2023-11-09T03:55:00.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.752+0000 7fae9e56a740 1 -- 172.21.15.76:0/3817191395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba9960cc90 msgr2=0x55ba9963cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:00.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.752+0000 7fae9e56a740 1 --2- 172.21.15.76:0/3817191395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba9960cc90 0x55ba9963cb20 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fae8c060500 tx=0x7fae8c00ee80 comp rx=0 tx=0).stop 2023-11-09T03:55:00.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.752+0000 7fae9e56a740 1 -- 172.21.15.76:0/3817191395 shutdown_connections 2023-11-09T03:55:00.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.752+0000 7fae9e56a740 1 --2- 172.21.15.76:0/3817191395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba9960cc90 0x55ba9963cb20 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.752+0000 7fae9e56a740 1 --2- 172.21.15.76:0/3817191395 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba99639c30 0x55ba9963c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.752+0000 7fae9e56a740 1 --2- 172.21.15.76:0/3817191395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba99639850 0x55ba9960c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.752+0000 7fae9e56a740 1 -- 172.21.15.76:0/3817191395 >> 172.21.15.76:0/3817191395 conn(0x55ba9960ea60 msgr2=0x55ba996575e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:00.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.752+0000 7fae9e56a740 1 -- 172.21.15.76:0/3817191395 shutdown_connections 2023-11-09T03:55:00.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.752+0000 7fae9e56a740 1 -- 172.21.15.76:0/3817191395 wait complete. 2023-11-09T03:55:00.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.753+0000 7fae9e56a740 1 -- start start 2023-11-09T03:55:00.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.753+0000 7fae9e56a740 1 Processor -- start 2023-11-09T03:55:00.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.754+0000 7fae9e56a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba99639850 0x55ba9966a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.754+0000 7fae9e56a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba99639c30 0x55ba9963c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.754+0000 7fae9e56a740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba996324a0 0x55ba9963e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.754+0000 7fae9e56a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba995788b0 con 0x55ba99639850 2023-11-09T03:55:00.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.754+0000 7fae9e56a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba99532c80 con 0x55ba99639c30 2023-11-09T03:55:00.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.754+0000 7fae9e56a740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba99532920 con 0x55ba996324a0 2023-11-09T03:55:00.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.754+0000 7fae9e56a740 10 client.0 _ll_register_callbacks cb 0x55ba9928aac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:55:00.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.754+0000 7fae996a1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba99639850 0x55ba9966a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.754+0000 7fae996a1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba99639850 0x55ba9966a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41086/0 (socket says 172.21.15.76:41086) 2023-11-09T03:55:00.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.754+0000 7fae996a1640 1 -- 172.21.15.76:0/3981951733 learned_addr learned my addr 172.21.15.76:0/3981951733 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:00.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.754+0000 7fae99ea2640 1 --2- 172.21.15.76:0/3981951733 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba996324a0 0x55ba9963e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.754+0000 7fae98ea0640 1 --2- 172.21.15.76:0/3981951733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba99639c30 0x55ba9963c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.754+0000 7fae996a1640 1 -- 172.21.15.76:0/3981951733 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba996324a0 msgr2=0x55ba9963e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:00.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.754+0000 7fae996a1640 1 --2- 172.21.15.76:0/3981951733 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba996324a0 0x55ba9963e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.754+0000 7fae996a1640 1 -- 172.21.15.76:0/3981951733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba99639c30 msgr2=0x55ba9963c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:00.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.754+0000 7fae996a1640 1 --2- 172.21.15.76:0/3981951733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba99639c30 0x55ba9963c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.754+0000 7fae996a1640 1 -- 172.21.15.76:0/3981951733 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae9414df80 con 0x55ba99639850 2023-11-09T03:55:00.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.754+0000 7fae98ea0640 1 --2- 172.21.15.76:0/3981951733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba99639c30 0x55ba9963c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:55:00.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.754+0000 7fae99ea2640 1 --2- 172.21.15.76:0/3981951733 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba996324a0 0x55ba9963e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:55:00.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.754+0000 7fae996a1640 1 --2- 172.21.15.76:0/3981951733 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba99639850 0x55ba9966a660 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fae90009d80 tx=0x7fae900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdcede005969d13e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:00.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.754+0000 7fae6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:55:00.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.754+0000 7fae6ffff640 1 -- 172.21.15.76:0/3981951733 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9003f9d0 con 0x55ba99639850 2023-11-09T03:55:00.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.754+0000 7fae6ffff640 1 -- 172.21.15.76:0/3981951733 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7fae9003fb30 con 0x55ba99639850 2023-11-09T03:55:00.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.754+0000 7fae9e56a740 1 -- 172.21.15.76:0/3981951733 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba99524bf0 con 0x55ba99639850 2023-11-09T03:55:00.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.754+0000 7fae6ffff640 1 -- 172.21.15.76:0/3981951733 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=133) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae9003fcf0 con 0x55ba99639850 2023-11-09T03:55:00.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.754+0000 7fae9e56a740 10 client.9357 fetch_fsmap learned FSMap version 133 2023-11-09T03:55:00.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.754+0000 7fae9e56a740 1 -- 172.21.15.76:0/3981951733 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=133,monmap=2+}) v3 -- 0x55ba99576f30 con 0x55ba99639850 2023-11-09T03:55:00.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.755+0000 7fae6ffff640 1 -- 172.21.15.76:0/3981951733 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ba99524bf0 con 0x55ba99639850 2023-11-09T03:55:00.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.755+0000 7fae6ffff640 1 -- 172.21.15.76:0/3981951733 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 133) v1 ==== 3306+0+0 (secure 0 0 0) 0x7fae9004c440 con 0x55ba99639850 2023-11-09T03:55:00.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.755+0000 7fae9e56a740 10 client.9357 fetch_fsmap finished waiting for FSMap version 133 2023-11-09T03:55:00.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.755+0000 7fae9e56a740 10 client.9357 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-09T03:55:00.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.755+0000 7fae9e56a740 20 client.9357 populate_metadata read hostname 'smithi076' 2023-11-09T03:55:00.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.755+0000 7fae9e56a740 1 --2- 172.21.15.76:0/3981951733 >> [v2:172.21.15.76:6834/757529522,v1:172.21.15.76:6835/757529522] conn(0x55ba9966ac70 0x55ba9966b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.755+0000 7fae9e56a740 4 client.9357 mds_command: new command op to 9024 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:55:00.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.755+0000 7fae9e56a740 1 -- 172.21.15.76:0/3981951733 --> [v2:172.21.15.76:6834/757529522,v1:172.21.15.76:6835/757529522] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba99571890 con 0x55ba9966ac70 2023-11-09T03:55:00.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.755+0000 7fae98ea0640 1 --2- 172.21.15.76:0/3981951733 >> [v2:172.21.15.76:6834/757529522,v1:172.21.15.76:6835/757529522] conn(0x55ba9966ac70 0x55ba9966b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.755+0000 7fae98ea0640 1 --2- 172.21.15.76:0/3981951733 >> [v2:172.21.15.76:6834/757529522,v1:172.21.15.76:6835/757529522] conn(0x55ba9966ac70 0x55ba9966b050 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=454ac486ca2ef00d server_cookie=71d9861db8839a0d in_seq=0 out_seq=0 2023-11-09T03:55:00.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.755+0000 7fae6ffff640 10 client.9357 ms_handle_connect on v2:172.21.15.76:6834/757529522 2023-11-09T03:55:00.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.756+0000 7fae6ffff640 1 -- 172.21.15.76:0/3981951733 <== mds.1 v2:172.21.15.76:6834/757529522 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ba99571890 con 0x55ba9966ac70 2023-11-09T03:55:00.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.756+0000 7fae6ffff640 10 client.9357 handle_command_reply: tid=0 2023-11-09T03:55:00.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.756+0000 7fae9e56a740 1 client.9357 shutdown 2023-11-09T03:55:00.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.756+0000 7fae9e56a740 1 -- 172.21.15.76:0/3981951733 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba99639850 msgr2=0x55ba9966a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:00.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.756+0000 7fae9e56a740 1 --2- 172.21.15.76:0/3981951733 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba99639850 0x55ba9966a660 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fae90009d80 tx=0x7fae900046c0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.756+0000 7fae9e56a740 1 -- 172.21.15.76:0/3981951733 shutdown_connections 2023-11-09T03:55:00.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.756+0000 7fae9e56a740 1 --2- 172.21.15.76:0/3981951733 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba996324a0 0x55ba9963e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.756+0000 7fae9e56a740 1 --2- 172.21.15.76:0/3981951733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba99639c30 0x55ba9963c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.756+0000 7fae9e56a740 1 --2- 172.21.15.76:0/3981951733 >> [v2:172.21.15.76:6834/757529522,v1:172.21.15.76:6835/757529522] conn(0x55ba9966ac70 0x55ba9966b050 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.756+0000 7fae9e56a740 1 --2- 172.21.15.76:0/3981951733 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba99639850 0x55ba9966a660 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.756+0000 7fae6ffff640 0 client.9357 ms_handle_reset on v2:172.21.15.76:6834/757529522 2023-11-09T03:55:00.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.756+0000 7fae9e56a740 1 -- 172.21.15.76:0/3981951733 >> 172.21.15.76:0/3981951733 conn(0x55ba9960cc90 msgr2=0x55ba996106d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:00.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.756+0000 7fae9e56a740 1 -- 172.21.15.76:0/3981951733 shutdown_connections 2023-11-09T03:55:00.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.756+0000 7fae9e56a740 1 -- 172.21.15.76:0/3981951733 wait complete. 2023-11-09T03:55:00.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.756+0000 7fae9e56a740 20 client.9357 trim_cache size 0 max 16384 2023-11-09T03:55:00.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.771+0000 7fae9e56a740 1 Processor -- start 2023-11-09T03:55:00.788 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.771+0000 7fae9e56a740 1 -- start start 2023-11-09T03:55:00.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.772+0000 7fae9e56a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba996324a0 0x55ba99613910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.772+0000 7fae9e56a740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba99639850 0x55ba9966a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.772+0000 7fae9e56a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba99639c30 0x55ba99638540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.772+0000 7fae9e56a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba994f8280 con 0x55ba99639c30 2023-11-09T03:55:00.789 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.772+0000 7fae9e56a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba9960d730 con 0x55ba996324a0 2023-11-09T03:55:00.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.772+0000 7fae9e56a740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba9960be20 con 0x55ba99639850 2023-11-09T03:55:00.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.772+0000 7fae99ea2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba99639c30 0x55ba99638540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.772+0000 7fae99ea2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba99639c30 0x55ba99638540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41094/0 (socket says 172.21.15.76:41094) 2023-11-09T03:55:00.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.772+0000 7fae99ea2640 1 -- 172.21.15.76:0/2508133589 learned_addr learned my addr 172.21.15.76:0/2508133589 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:00.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.772+0000 7fae98ea0640 1 --2- 172.21.15.76:0/2508133589 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba99639850 0x55ba9966a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.772+0000 7fae996a1640 1 --2- 172.21.15.76:0/2508133589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba996324a0 0x55ba99613910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.773+0000 7fae99ea2640 1 -- 172.21.15.76:0/2508133589 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba99639850 msgr2=0x55ba9966a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:00.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.773+0000 7fae99ea2640 1 --2- 172.21.15.76:0/2508133589 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba99639850 0x55ba9966a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.773+0000 7fae99ea2640 1 -- 172.21.15.76:0/2508133589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba996324a0 msgr2=0x55ba99613910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:00.791 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.773+0000 7fae99ea2640 1 --2- 172.21.15.76:0/2508133589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba996324a0 0x55ba99613910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.773+0000 7fae99ea2640 1 -- 172.21.15.76:0/2508133589 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba99514d10 con 0x55ba99639c30 2023-11-09T03:55:00.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.773+0000 7fae99ea2640 1 --2- 172.21.15.76:0/2508133589 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba99639c30 0x55ba99638540 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00eec0 tx=0x7fae8c05c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc394cde3b12f07 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:00.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.773+0000 7fae777fe640 1 -- 172.21.15.76:0/2508133589 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c05c880 con 0x55ba99639c30 2023-11-09T03:55:00.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.773+0000 7fae777fe640 1 -- 172.21.15.76:0/2508133589 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8c058730 con 0x55ba99639c30 2023-11-09T03:55:00.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.773+0000 7fae777fe640 1 -- 172.21.15.76:0/2508133589 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c056ad0 con 0x55ba99639c30 2023-11-09T03:55:00.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.773+0000 7fae9e56a740 1 -- 172.21.15.76:0/2508133589 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba99639c30 msgr2=0x55ba99638540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:00.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.773+0000 7fae9e56a740 1 --2- 172.21.15.76:0/2508133589 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba99639c30 0x55ba99638540 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00eec0 tx=0x7fae8c05c000 comp rx=0 tx=0).stop 2023-11-09T03:55:00.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.773+0000 7fae9e56a740 1 -- 172.21.15.76:0/2508133589 shutdown_connections 2023-11-09T03:55:00.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.773+0000 7fae9e56a740 1 --2- 172.21.15.76:0/2508133589 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba99639c30 0x55ba99638540 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.773+0000 7fae9e56a740 1 --2- 172.21.15.76:0/2508133589 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba99639850 0x55ba9966a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.773+0000 7fae9e56a740 1 --2- 172.21.15.76:0/2508133589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba996324a0 0x55ba99613910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.773+0000 7fae9e56a740 1 -- 172.21.15.76:0/2508133589 >> 172.21.15.76:0/2508133589 conn(0x55ba9960cc90 msgr2=0x55ba9963de20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:00.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.773+0000 7fae9e56a740 1 -- 172.21.15.76:0/2508133589 shutdown_connections 2023-11-09T03:55:00.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.773+0000 7fae9e56a740 1 -- 172.21.15.76:0/2508133589 wait complete. 2023-11-09T03:55:00.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.774+0000 7fae9e56a740 1 -- start start 2023-11-09T03:55:00.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.774+0000 7fae9e56a740 1 Processor -- start 2023-11-09T03:55:00.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.775+0000 7fae9e56a740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba996324a0 0x55ba99611580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.775+0000 7fae9e56a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba99639850 0x55ba99611ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.775+0000 7fae9e56a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba99639c30 0x55ba9965a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.775+0000 7fae9e56a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba996c3120 con 0x55ba99639c30 2023-11-09T03:55:00.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.775+0000 7fae9e56a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba9963ae50 con 0x55ba99639850 2023-11-09T03:55:00.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.775+0000 7fae9e56a740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba996c2a00 con 0x55ba996324a0 2023-11-09T03:55:00.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.775+0000 7fae9e56a740 10 client.0 _ll_register_callbacks cb 0x7fae8c054a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:55:00.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.775+0000 7fae99ea2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba99639c30 0x55ba9965a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.775+0000 7fae99ea2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba99639c30 0x55ba9965a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41106/0 (socket says 172.21.15.76:41106) 2023-11-09T03:55:00.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.775+0000 7fae99ea2640 1 -- 172.21.15.76:0/3923235706 learned_addr learned my addr 172.21.15.76:0/3923235706 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:00.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.775+0000 7fae98ea0640 1 --2- 172.21.15.76:0/3923235706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba99639850 0x55ba99611ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.775+0000 7fae996a1640 1 --2- 172.21.15.76:0/3923235706 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba996324a0 0x55ba99611580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.775+0000 7fae99ea2640 1 -- 172.21.15.76:0/3923235706 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba996324a0 msgr2=0x55ba99611580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:00.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.775+0000 7fae99ea2640 1 --2- 172.21.15.76:0/3923235706 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba996324a0 0x55ba99611580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.775+0000 7fae99ea2640 1 -- 172.21.15.76:0/3923235706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba99639850 msgr2=0x55ba99611ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:00.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.775+0000 7fae99ea2640 1 --2- 172.21.15.76:0/3923235706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba99639850 0x55ba99611ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.775+0000 7fae99ea2640 1 -- 172.21.15.76:0/3923235706 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba9953bca0 con 0x55ba99639c30 2023-11-09T03:55:00.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.775+0000 7fae99ea2640 1 --2- 172.21.15.76:0/3923235706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba99639c30 0x55ba9965a2e0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00eec0 tx=0x7fae8c01f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ef4a2d68248b22 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:00.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.775+0000 7fae6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:55:00.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.776+0000 7fae6ffff640 1 -- 172.21.15.76:0/3923235706 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c054e40 con 0x55ba99639c30 2023-11-09T03:55:00.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.776+0000 7fae6ffff640 1 -- 172.21.15.76:0/3923235706 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7fae8c054fa0 con 0x55ba99639c30 2023-11-09T03:55:00.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.776+0000 7fae9e56a740 1 -- 172.21.15.76:0/3923235706 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae8c060a30 con 0x55ba99639c30 2023-11-09T03:55:00.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.776+0000 7fae6ffff640 1 -- 172.21.15.76:0/3923235706 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=133) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae8c0605a0 con 0x55ba99639c30 2023-11-09T03:55:00.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.776+0000 7fae9e56a740 10 client.9363 fetch_fsmap learned FSMap version 133 2023-11-09T03:55:00.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.776+0000 7fae9e56a740 1 -- 172.21.15.76:0/3923235706 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=133,monmap=2+}) v3 -- 0x55ba9953a500 con 0x55ba99639c30 2023-11-09T03:55:00.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.776+0000 7fae6ffff640 1 -- 172.21.15.76:0/3923235706 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8c060a30 con 0x55ba99639c30 2023-11-09T03:55:00.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.776+0000 7fae6ffff640 1 -- 172.21.15.76:0/3923235706 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 133) v1 ==== 3306+0+0 (secure 0 0 0) 0x7fae8c067020 con 0x55ba99639c30 2023-11-09T03:55:00.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.776+0000 7fae9e56a740 10 client.9363 fetch_fsmap finished waiting for FSMap version 133 2023-11-09T03:55:00.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.776+0000 7fae9e56a740 10 client.9363 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-09T03:55:00.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.776+0000 7fae9e56a740 20 client.9363 populate_metadata read hostname 'smithi076' 2023-11-09T03:55:00.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.776+0000 7fae9e56a740 1 --2- 172.21.15.76:0/3923235706 >> [v2:172.21.15.76:6834/757529522,v1:172.21.15.76:6835/757529522] conn(0x55ba996c1ae0 0x55ba9965a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.776+0000 7fae9e56a740 4 client.9363 mds_command: new command op to 9024 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-09T03:55:00.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.776+0000 7fae9e56a740 1 -- 172.21.15.76:0/3923235706 --> [v2:172.21.15.76:6834/757529522,v1:172.21.15.76:6835/757529522] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ba995715b0 con 0x55ba996c1ae0 2023-11-09T03:55:00.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.776+0000 7fae996a1640 1 --2- 172.21.15.76:0/3923235706 >> [v2:172.21.15.76:6834/757529522,v1:172.21.15.76:6835/757529522] conn(0x55ba996c1ae0 0x55ba9965a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.776+0000 7fae996a1640 1 --2- 172.21.15.76:0/3923235706 >> [v2:172.21.15.76:6834/757529522,v1:172.21.15.76:6835/757529522] conn(0x55ba996c1ae0 0x55ba9965a820 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2ddf6954957b6247 server_cookie=9e66b46ed807631c in_seq=0 out_seq=0 2023-11-09T03:55:00.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.776+0000 7fae6ffff640 10 client.9363 ms_handle_connect on v2:172.21.15.76:6834/757529522 2023-11-09T03:55:00.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.777+0000 7fae6ffff640 1 -- 172.21.15.76:0/3923235706 <== mds.1 v2:172.21.15.76:6834/757529522 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x55ba995715b0 con 0x55ba996c1ae0 2023-11-09T03:55:00.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.777+0000 7fae6ffff640 10 client.9363 handle_command_reply: tid=0 2023-11-09T03:55:00.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.777+0000 7fae9e56a740 1 client.9363 shutdown 2023-11-09T03:55:00.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.777+0000 7fae9e56a740 1 -- 172.21.15.76:0/3923235706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba99639c30 msgr2=0x55ba9965a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:00.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.777+0000 7fae9e56a740 1 --2- 172.21.15.76:0/3923235706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba99639c30 0x55ba9965a2e0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00eec0 tx=0x7fae8c01f040 comp rx=0 tx=0).stop 2023-11-09T03:55:00.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.777+0000 7fae9e56a740 1 -- 172.21.15.76:0/3923235706 shutdown_connections 2023-11-09T03:55:00.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.777+0000 7fae9e56a740 1 --2- 172.21.15.76:0/3923235706 >> [v2:172.21.15.76:6834/757529522,v1:172.21.15.76:6835/757529522] conn(0x55ba996c1ae0 0x55ba9965a820 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.777+0000 7fae9e56a740 1 --2- 172.21.15.76:0/3923235706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ba99639c30 0x55ba9965a2e0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.777+0000 7fae6ffff640 0 client.9363 ms_handle_reset on v2:172.21.15.76:6834/757529522 2023-11-09T03:55:00.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.777+0000 7fae9e56a740 1 --2- 172.21.15.76:0/3923235706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ba99639850 0x55ba99611ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.777+0000 7fae9e56a740 1 --2- 172.21.15.76:0/3923235706 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ba996324a0 0x55ba99611580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.777+0000 7fae9e56a740 1 -- 172.21.15.76:0/3923235706 >> 172.21.15.76:0/3923235706 conn(0x55ba9960cc90 msgr2=0x55ba9966a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:00.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.777+0000 7fae9e56a740 1 -- 172.21.15.76:0/3923235706 shutdown_connections 2023-11-09T03:55:00.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.777+0000 7fae9e56a740 1 -- 172.21.15.76:0/3923235706 wait complete. 2023-11-09T03:55:00.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.777+0000 7fae9e56a740 20 client.9363 trim_cache size 0 max 16384 2023-11-09T03:55:00.804 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:55:00.805 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "c213b26f-d0da-45d5-8e34-bb75c096d4f2", 2023-11-09T03:55:00.805 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 1, 2023-11-09T03:55:00.805 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 9024, 2023-11-09T03:55:00.805 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2023-11-09T03:55:00.805 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2023-11-09T03:55:00.806 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2023-11-09T03:55:00.806 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 15.424543021, 2023-11-09T03:55:00.806 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 133, 2023-11-09T03:55:00.806 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 118, 2023-11-09T03:55:00.806 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 118, 2023-11-09T03:55:00.807 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 54.109059877999997 2023-11-09T03:55:00.807 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:55:00.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.777+0000 7fae9b905640 1 -- 172.21.15.76:0/3063709612 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae7806ae70 msgr2=0x7fae7806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:00.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.777+0000 7fae9b905640 1 --2- 172.21.15.76:0/3063709612 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae7806ae70 0x7fae7806d330 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fae90009c90 tx=0x7fae90030000 comp rx=0 tx=0).stop 2023-11-09T03:55:00.808 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.777+0000 7fae9b905640 1 -- 172.21.15.76:0/3063709612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae9414c8b0 msgr2=0x7fae9415a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:00.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.777+0000 7fae9b905640 1 --2- 172.21.15.76:0/3063709612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae9414c8b0 0x7fae9415a060 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fae940ce9e0 tx=0x7fae8c00e530 comp rx=0 tx=0).stop 2023-11-09T03:55:00.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.777+0000 7fae9b905640 1 -- 172.21.15.76:0/3063709612 shutdown_connections 2023-11-09T03:55:00.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.777+0000 7fae9b905640 1 --2- 172.21.15.76:0/3063709612 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae7806ae70 0x7fae7806d330 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.777+0000 7fae9b905640 1 --2- 172.21.15.76:0/3063709612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae9414c8b0 0x7fae9415a060 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.777+0000 7fae9b905640 1 --2- 172.21.15.76:0/3063709612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae9414e380 0x7fae94159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.777+0000 7fae9b905640 1 --2- 172.21.15.76:0/3063709612 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae80003970 0x7fae94070200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.778+0000 7fae9b905640 1 -- 172.21.15.76:0/3063709612 >> 172.21.15.76:0/3063709612 conn(0x7fae940cb530 msgr2=0x7fae940715b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:00.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.778+0000 7fae9b905640 1 -- 172.21.15.76:0/3063709612 shutdown_connections 2023-11-09T03:55:00.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.778+0000 7fae9b905640 1 -- 172.21.15.76:0/3063709612 wait complete. 2023-11-09T03:55:00.810 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:55:00.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.912+0000 7ff66e207640 1 Processor -- start 2023-11-09T03:55:00.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.912+0000 7ff66e207640 1 -- start start 2023-11-09T03:55:00.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.913+0000 7ff66e207640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff6680ade00 0x7ff6680b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.913+0000 7ff66e207640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff66806c5f0 0x7ff6680cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.913 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.913+0000 7ff66e207640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff66806cc90 con 0x7ff66806d9c0 2023-11-09T03:55:00.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.913+0000 7ff66e207640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6680b0e60 con 0x7ff6680ade00 2023-11-09T03:55:00.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.913+0000 7ff66e207640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff66806cee0 con 0x7ff66806c5f0 2023-11-09T03:55:00.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.913+0000 7ff6677fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff6680ade00 0x7ff6680b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.913+0000 7ff6677fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff6680ade00 0x7ff6680b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41108/0 (socket says 172.21.15.76:41108) 2023-11-09T03:55:00.914 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.913+0000 7ff65ffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff66806d9c0 0x7ff6680710d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45484/0 (socket says 172.21.15.76:45484) 2023-11-09T03:55:00.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.913+0000 7ff6677fe640 1 -- 172.21.15.76:0/3457896018 learned_addr learned my addr 172.21.15.76:0/3457896018 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:00.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.913+0000 7ff667fff640 1 --2- 172.21.15.76:0/3457896018 >> v2:172.21.15.113:3300/0 conn(0x7ff66806c5f0 0x7ff6680cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.913+0000 7ff6677fe640 1 -- 172.21.15.76:0/3457896018 >> v2:172.21.15.113:3300/0 conn(0x7ff66806c5f0 msgr2=0x7ff6680cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:00.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.913+0000 7ff6677fe640 1 --2- 172.21.15.76:0/3457896018 >> v2:172.21.15.113:3300/0 conn(0x7ff66806c5f0 0x7ff6680cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.915 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.913+0000 7ff6677fe640 1 -- 172.21.15.76:0/3457896018 >> v1:172.21.15.76:6789/0 conn(0x7ff66806d9c0 legacy=0x7ff6680710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:00.916 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.913+0000 7ff6677fe640 1 -- 172.21.15.76:0/3457896018 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff66806d1c0 con 0x7ff6680ade00 2023-11-09T03:55:00.919 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.914+0000 7ff6677fe640 1 --2- 172.21.15.76:0/3457896018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6680ade00 0x7ff6680b0530 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7ff6680b44f0 tx=0x7ff65802b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b62cec57a655af server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:00.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.914+0000 7ff666ffd640 1 -- 172.21.15.76:0/3457896018 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff658030710 con 0x7ff6680ade00 2023-11-09T03:55:00.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.914+0000 7ff666ffd640 1 -- 172.21.15.76:0/3457896018 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65802e050 con 0x7ff6680ade00 2023-11-09T03:55:00.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.914+0000 7ff666ffd640 1 -- 172.21.15.76:0/3457896018 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff658030a90 con 0x7ff6680ade00 2023-11-09T03:55:00.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.914+0000 7ff66e207640 1 -- 172.21.15.76:0/3457896018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6680ade00 msgr2=0x7ff6680b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:00.920 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.914+0000 7ff66e207640 1 --2- 172.21.15.76:0/3457896018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6680ade00 0x7ff6680b0530 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7ff6680b44f0 tx=0x7ff65802b6d0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.915+0000 7ff66e207640 1 -- 172.21.15.76:0/3457896018 shutdown_connections 2023-11-09T03:55:00.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.915+0000 7ff66e207640 1 --2- 172.21.15.76:0/3457896018 >> v2:172.21.15.113:3300/0 conn(0x7ff66806c5f0 0x7ff6680cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.915+0000 7ff66e207640 1 --2- 172.21.15.76:0/3457896018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6680ade00 0x7ff6680b0530 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.915+0000 7ff66e207640 1 -- 172.21.15.76:0/3457896018 >> 172.21.15.76:0/3457896018 conn(0x7ff6680cb530 msgr2=0x7ff6680cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:00.921 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.915+0000 7ff66e207640 1 -- 172.21.15.76:0/3457896018 shutdown_connections 2023-11-09T03:55:00.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.915+0000 7ff66e207640 1 -- 172.21.15.76:0/3457896018 wait complete. 2023-11-09T03:55:00.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.916+0000 7ff66e207640 1 Processor -- start 2023-11-09T03:55:00.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.916+0000 7ff66e207640 1 -- start start 2023-11-09T03:55:00.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.916+0000 7ff66e207640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff66806c5f0 0x7ff668154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.922 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.917+0000 7ff66e207640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff66806d9c0 0x7ff6680c45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.917+0000 7ff66e207640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6680ade00 0x7ff668157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.917+0000 7ff66e207640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6681551b0 con 0x7ff66806c5f0 2023-11-09T03:55:00.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.917+0000 7ff66e207640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff668155610 con 0x7ff6680ade00 2023-11-09T03:55:00.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.917+0000 7ff66e207640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6681558f0 con 0x7ff66806d9c0 2023-11-09T03:55:00.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.917+0000 7ff6677fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff66806c5f0 0x7ff668154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.917+0000 7ff6677fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff66806c5f0 0x7ff668154c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41116/0 (socket says 172.21.15.76:41116) 2023-11-09T03:55:00.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.917+0000 7ff6677fe640 1 -- 172.21.15.76:0/2953206036 learned_addr learned my addr 172.21.15.76:0/2953206036 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:00.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.917+0000 7ff65ffff640 1 --2- 172.21.15.76:0/2953206036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff66806d9c0 0x7ff6680c45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.917+0000 7ff667fff640 1 --2- 172.21.15.76:0/2953206036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6680ade00 0x7ff668157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.917+0000 7ff6677fe640 1 -- 172.21.15.76:0/2953206036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff66806d9c0 msgr2=0x7ff6680c45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:00.924 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.917+0000 7ff6677fe640 1 --2- 172.21.15.76:0/2953206036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff66806d9c0 0x7ff6680c45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.917+0000 7ff6677fe640 1 -- 172.21.15.76:0/2953206036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6680ade00 msgr2=0x7ff668157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:00.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.917+0000 7ff6677fe640 1 --2- 172.21.15.76:0/2953206036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6680ade00 0x7ff668157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:00.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.917+0000 7ff6677fe640 1 -- 172.21.15.76:0/2953206036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6580094d0 con 0x7ff66806c5f0 2023-11-09T03:55:00.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.917+0000 7ff667fff640 1 --2- 172.21.15.76:0/2953206036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6680ade00 0x7ff668157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:55:00.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.917+0000 7ff6677fe640 1 --2- 172.21.15.76:0/2953206036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff66806c5f0 0x7ff668154c70 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7ff658000c00 tx=0x7ff6580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:00.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.917+0000 7ff664ff9640 1 -- 172.21.15.76:0/2953206036 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff658032890 con 0x7ff66806c5f0 2023-11-09T03:55:00.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.917+0000 7ff664ff9640 1 -- 172.21.15.76:0/2953206036 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff658032a10 con 0x7ff66806c5f0 2023-11-09T03:55:00.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.917+0000 7ff66e207640 1 -- 172.21.15.76:0/2953206036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff668155bd0 con 0x7ff66806c5f0 2023-11-09T03:55:00.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.917+0000 7ff664ff9640 1 -- 172.21.15.76:0/2953206036 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff658032dc0 con 0x7ff66806c5f0 2023-11-09T03:55:00.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.917+0000 7ff66e207640 1 -- 172.21.15.76:0/2953206036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff668146ef0 con 0x7ff66806c5f0 2023-11-09T03:55:00.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.918+0000 7ff66e207640 1 -- 172.21.15.76:0/2953206036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff668157180 con 0x7ff66806c5f0 2023-11-09T03:55:00.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.919+0000 7ff664ff9640 1 -- 172.21.15.76:0/2953206036 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff658039040 con 0x7ff66806c5f0 2023-11-09T03:55:00.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.919+0000 7ff664ff9640 1 --2- 172.21.15.76:0/2953206036 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff64806ae70 0x7ff64806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:00.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.919+0000 7ff664ff9640 1 -- 172.21.15.76:0/2953206036 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7ff6580abdb0 con 0x7ff66806c5f0 2023-11-09T03:55:00.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.919+0000 7ff65ffff640 1 --2- 172.21.15.76:0/2953206036 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff64806ae70 0x7ff64806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:00.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.920+0000 7ff65ffff640 1 --2- 172.21.15.76:0/2953206036 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff64806ae70 0x7ff64806d330 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff6680b0240 tx=0x7ff650007e20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:00.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:00.923+0000 7ff664ff9640 1 -- 172.21.15.76:0/2953206036 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff668157180 con 0x7ff66806c5f0 2023-11-09T03:55:01.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.123+0000 7ff66e207640 1 -- 172.21.15.76:0/2953206036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff668157560 con 0x7ff66806c5f0 2023-11-09T03:55:01.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.124+0000 7ff664ff9640 1 -- 172.21.15.76:0/2953206036 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+4932 (secure 0 0 0) 0x7ff668157560 con 0x7ff66806c5f0 2023-11-09T03:55:01.124 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:55:01.124 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":133,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6698,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":133,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:58.523123+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":9024},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9024":{"gid":9024,"name":"a","rank":1,"incarnation":129,"state":"up:active","state_seq":16,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:55:01.124 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 133 2023-11-09T03:55:01.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.125+0000 7ff66e207640 1 -- 172.21.15.76:0/2953206036 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff64806ae70 msgr2=0x7ff64806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:01.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.125+0000 7ff66e207640 1 --2- 172.21.15.76:0/2953206036 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff64806ae70 0x7ff64806d330 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff6680b0240 tx=0x7ff650007e20 comp rx=0 tx=0).stop 2023-11-09T03:55:01.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.125+0000 7ff66e207640 1 -- 172.21.15.76:0/2953206036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff66806c5f0 msgr2=0x7ff668154c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:01.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.125+0000 7ff66e207640 1 --2- 172.21.15.76:0/2953206036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff66806c5f0 0x7ff668154c70 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7ff658000c00 tx=0x7ff6580046c0 comp rx=0 tx=0).stop 2023-11-09T03:55:01.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.126+0000 7ff66e207640 1 -- 172.21.15.76:0/2953206036 shutdown_connections 2023-11-09T03:55:01.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.126+0000 7ff66e207640 1 --2- 172.21.15.76:0/2953206036 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff64806ae70 0x7ff64806d330 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:01.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.126+0000 7ff66e207640 1 --2- 172.21.15.76:0/2953206036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6680ade00 0x7ff668157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:01.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.126+0000 7ff66e207640 1 --2- 172.21.15.76:0/2953206036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff66806d9c0 0x7ff6680c45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:01.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.126+0000 7ff66e207640 1 --2- 172.21.15.76:0/2953206036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff66806c5f0 0x7ff668154c70 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:01.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.126+0000 7ff66e207640 1 -- 172.21.15.76:0/2953206036 >> 172.21.15.76:0/2953206036 conn(0x7ff6680cb530 msgr2=0x7ff668071090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:01.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.126+0000 7ff66e207640 1 -- 172.21.15.76:0/2953206036 shutdown_connections 2023-11-09T03:55:01.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.126+0000 7ff66e207640 1 -- 172.21.15.76:0/2953206036 wait complete. 2023-11-09T03:55:01.136 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2023-11-09T03:55:01.248 INFO:teuthology.orchestra.run.smithi076.stdout:{"last_snap":16,"last_created":16,"last_destroyed":14,"pending_noop":[],"snaps":[{"snapid":16,"ino":1099511627777,"stamp":"2023-11-09T03:54:27.826603+0000","name":"s34","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-09T03:55:01.249 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 16, "last_destroyed": 14, "last_snap": 16, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s34", "snapid": 16, "stamp": "2023-11-09T03:54:27.826603+0000" } ] } 2023-11-09T03:55:01.249 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-09T03:55:01.249 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:55:01.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.350+0000 7f62a7432640 1 Processor -- start 2023-11-09T03:55:01.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.350+0000 7f62a7432640 1 -- start start 2023-11-09T03:55:01.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.351+0000 7f62a7432640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62a0159e40 0x7f62a00b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:01.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.351+0000 7f62a7432640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f62a0070700 0x7f62a00b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:01.351 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.351+0000 7f62a7432640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62a0071500 con 0x7f62a006bb60 2023-11-09T03:55:01.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.351+0000 7f62a7432640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a015dae0 con 0x7f62a0070700 2023-11-09T03:55:01.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.351+0000 7f62a7432640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a0070c60 con 0x7f62a0159e40 2023-11-09T03:55:01.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.351+0000 7f62a59cf640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f62a0070700 0x7f62a00b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:01.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.351+0000 7f62a59cf640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f62a0070700 0x7f62a00b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41126/0 (socket says 172.21.15.76:41126) 2023-11-09T03:55:01.352 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.351+0000 7f62a59cf640 1 -- 172.21.15.76:0/4197068198 learned_addr learned my addr 172.21.15.76:0/4197068198 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:01.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.351+0000 7f62a51ce640 1 --2- 172.21.15.76:0/4197068198 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62a0159e40 0x7f62a00b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:01.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.351+0000 7f62a59cf640 1 -- 172.21.15.76:0/4197068198 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62a0159e40 msgr2=0x7f62a00b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:01.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.351+0000 7f62a59cf640 1 --2- 172.21.15.76:0/4197068198 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62a0159e40 0x7f62a00b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:01.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.351+0000 7f62a59cf640 1 -- 172.21.15.76:0/4197068198 >> v1:172.21.15.113:6789/0 conn(0x7f62a006bb60 legacy=0x7f62a006fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:01.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.351+0000 7f62a59cf640 1 -- 172.21.15.76:0/4197068198 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a0070f40 con 0x7f62a0070700 2023-11-09T03:55:01.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.352+0000 7f62a59cf640 1 --2- 172.21.15.76:0/4197068198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a0070700 0x7f62a00b0a70 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f6298009e50 tx=0x7f629802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5098dd418710c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:01.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.352+0000 7f6297fff640 1 -- 172.21.15.76:0/4197068198 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629802ca70 con 0x7f62a0070700 2023-11-09T03:55:01.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.352+0000 7f6297fff640 1 -- 172.21.15.76:0/4197068198 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629802cbd0 con 0x7f62a0070700 2023-11-09T03:55:01.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.352+0000 7f6297fff640 1 -- 172.21.15.76:0/4197068198 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62980343f0 con 0x7f62a0070700 2023-11-09T03:55:01.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.353+0000 7f62a7432640 1 -- 172.21.15.76:0/4197068198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a0070700 msgr2=0x7f62a00b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:01.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.353+0000 7f62a7432640 1 --2- 172.21.15.76:0/4197068198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a0070700 0x7f62a00b0a70 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f6298009e50 tx=0x7f629802bed0 comp rx=0 tx=0).stop 2023-11-09T03:55:01.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.353+0000 7f62a7432640 1 -- 172.21.15.76:0/4197068198 shutdown_connections 2023-11-09T03:55:01.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.353+0000 7f62a7432640 1 --2- 172.21.15.76:0/4197068198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a0070700 0x7f62a00b0a70 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:01.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.353+0000 7f62a7432640 1 --2- 172.21.15.76:0/4197068198 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62a0159e40 0x7f62a00b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:01.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.353+0000 7f62a7432640 1 -- 172.21.15.76:0/4197068198 >> 172.21.15.76:0/4197068198 conn(0x7f62a00cb530 msgr2=0x7f62a00cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:01.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.353+0000 7f62a7432640 1 -- 172.21.15.76:0/4197068198 shutdown_connections 2023-11-09T03:55:01.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.353+0000 7f62a7432640 1 -- 172.21.15.76:0/4197068198 wait complete. 2023-11-09T03:55:01.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.354+0000 7f62a7432640 1 Processor -- start 2023-11-09T03:55:01.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.354+0000 7f62a7432640 1 -- start start 2023-11-09T03:55:01.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.355+0000 7f62a7432640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62a006bb60 0x7f62a0148d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:01.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.355+0000 7f62a7432640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a0070700 0x7f62a00c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:01.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.355+0000 7f62a7432640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62a0159e40 0x7f62a00af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:01.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.355+0000 7f62a7432640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a0154610 con 0x7f62a0070700 2023-11-09T03:55:01.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.355+0000 7f62a7432640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a01548f0 con 0x7f62a006bb60 2023-11-09T03:55:01.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.355+0000 7f62a7432640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a0154bd0 con 0x7f62a0159e40 2023-11-09T03:55:01.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.355+0000 7f62a49cd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a0070700 0x7f62a00c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:01.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.355+0000 7f62a49cd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a0070700 0x7f62a00c7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41134/0 (socket says 172.21.15.76:41134) 2023-11-09T03:55:01.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.355+0000 7f62a49cd640 1 -- 172.21.15.76:0/2007107708 learned_addr learned my addr 172.21.15.76:0/2007107708 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:01.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.355+0000 7f62a59cf640 1 --2- 172.21.15.76:0/2007107708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62a0159e40 0x7f62a00af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:01.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.355+0000 7f62a51ce640 1 --2- 172.21.15.76:0/2007107708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62a006bb60 0x7f62a0148d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:01.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.355+0000 7f62a49cd640 1 -- 172.21.15.76:0/2007107708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62a0159e40 msgr2=0x7f62a00af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:01.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.355+0000 7f62a49cd640 1 --2- 172.21.15.76:0/2007107708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62a0159e40 0x7f62a00af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:01.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.355+0000 7f62a49cd640 1 -- 172.21.15.76:0/2007107708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62a006bb60 msgr2=0x7f62a0148d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:01.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.355+0000 7f62a49cd640 1 --2- 172.21.15.76:0/2007107708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62a006bb60 0x7f62a0148d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:01.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.355+0000 7f62a49cd640 1 -- 172.21.15.76:0/2007107708 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6298009b30 con 0x7f62a0070700 2023-11-09T03:55:01.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.355+0000 7f62a49cd640 1 --2- 172.21.15.76:0/2007107708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a0070700 0x7f62a00c7a70 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f629000c450 tx=0x7f629000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:01.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.355+0000 7f6295ffb640 1 -- 172.21.15.76:0/2007107708 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62900052d0 con 0x7f62a0070700 2023-11-09T03:55:01.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.355+0000 7f6295ffb640 1 -- 172.21.15.76:0/2007107708 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6290005430 con 0x7f62a0070700 2023-11-09T03:55:01.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.355+0000 7f62a7432640 1 -- 172.21.15.76:0/2007107708 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a0154eb0 con 0x7f62a0070700 2023-11-09T03:55:01.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.355+0000 7f6295ffb640 1 -- 172.21.15.76:0/2007107708 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6290012870 con 0x7f62a0070700 2023-11-09T03:55:01.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.356+0000 7f62a7432640 1 -- 172.21.15.76:0/2007107708 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a0149290 con 0x7f62a0070700 2023-11-09T03:55:01.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.356+0000 7f62a7432640 1 -- 172.21.15.76:0/2007107708 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a01efb50 con 0x7f62a0070700 2023-11-09T03:55:01.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.357+0000 7f6295ffb640 1 -- 172.21.15.76:0/2007107708 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f62900129d0 con 0x7f62a0070700 2023-11-09T03:55:01.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.357+0000 7f6295ffb640 1 --2- 172.21.15.76:0/2007107708 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f628406ae70 0x7f628406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:01.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.357+0000 7f6295ffb640 1 -- 172.21.15.76:0/2007107708 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7f6290089cc0 con 0x7f62a0070700 2023-11-09T03:55:01.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.357+0000 7f62a51ce640 1 --2- 172.21.15.76:0/2007107708 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f628406ae70 0x7f628406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:01.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.358+0000 7f62a51ce640 1 --2- 172.21.15.76:0/2007107708 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f628406ae70 0x7f628406d330 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f629c006fd0 tx=0x7f629c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:01.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.362+0000 7f6295ffb640 1 -- 172.21.15.76:0/2007107708 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f62900541f0 con 0x7f62a0070700 2023-11-09T03:55:01.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.563+0000 7f62a7432640 1 -- 172.21.15.76:0/2007107708 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62a00711f0 con 0x7f62a0070700 2023-11-09T03:55:01.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.564+0000 7f6295ffb640 1 -- 172.21.15.76:0/2007107708 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+4932 (secure 0 0 0) 0x7f6290057810 con 0x7f62a0070700 2023-11-09T03:55:01.564 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:55:01.564 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":133,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6698,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":133,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:58.523123+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":9024},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9024":{"gid":9024,"name":"a","rank":1,"incarnation":129,"state":"up:active","state_seq":16,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:55:01.565 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 133 2023-11-09T03:55:01.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.566+0000 7f62a7432640 1 -- 172.21.15.76:0/2007107708 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f628406ae70 msgr2=0x7f628406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:01.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.566+0000 7f62a7432640 1 --2- 172.21.15.76:0/2007107708 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f628406ae70 0x7f628406d330 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f629c006fd0 tx=0x7f629c008040 comp rx=0 tx=0).stop 2023-11-09T03:55:01.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.566+0000 7f62a7432640 1 -- 172.21.15.76:0/2007107708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a0070700 msgr2=0x7f62a00c7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:01.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.566+0000 7f62a7432640 1 --2- 172.21.15.76:0/2007107708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a0070700 0x7f62a00c7a70 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f629000c450 tx=0x7f629000c910 comp rx=0 tx=0).stop 2023-11-09T03:55:01.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.566+0000 7f62a7432640 1 -- 172.21.15.76:0/2007107708 shutdown_connections 2023-11-09T03:55:01.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.566+0000 7f62a7432640 1 --2- 172.21.15.76:0/2007107708 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f628406ae70 0x7f628406d330 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:01.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.566+0000 7f62a7432640 1 --2- 172.21.15.76:0/2007107708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62a0159e40 0x7f62a00af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:01.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.566+0000 7f62a7432640 1 --2- 172.21.15.76:0/2007107708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a0070700 0x7f62a00c7a70 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:01.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.566+0000 7f62a7432640 1 --2- 172.21.15.76:0/2007107708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62a006bb60 0x7f62a0148d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:01.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.566+0000 7f62a7432640 1 -- 172.21.15.76:0/2007107708 >> 172.21.15.76:0/2007107708 conn(0x7f62a00cb530 msgr2=0x7f62a00c1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:01.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.566+0000 7f62a7432640 1 -- 172.21.15.76:0/2007107708 shutdown_connections 2023-11-09T03:55:01.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:01.566+0000 7f62a7432640 1 -- 172.21.15.76:0/2007107708 wait complete. 2023-11-09T03:55:01.576 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2023-11-09T03:55:01.692 INFO:teuthology.orchestra.run.smithi076.stdout:{"last_snap":16,"last_created":16,"last_destroyed":14,"pending_noop":[],"snaps":[{"snapid":16,"ino":1099511627777,"stamp":"2023-11-09T03:54:27.826603+0000","name":"s34","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-09T03:55:01.692 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 16, "last_destroyed": 14, "last_snap": 16, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s34", "snapid": 16, "stamp": "2023-11-09T03:54:27.826603+0000" } ] } 2023-11-09T03:55:01.693 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:55:01.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:55:01.693 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2023-11-09T03:55:02.710 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:55:02.713 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:55:06.770 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:55:06.770+0000 7ff3957fa640 -1 client.9180 I was blocklisted at osd epoch 118 2023-11-09T03:55:06.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:55:06.772 INFO:teuthology.orchestra.run.smithi076.stderr:mkdir: cannot create directory ‘d1/dir/.snap/s34’: Cannot send after transport endpoint shutdown 2023-11-09T03:55:06.772 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:55:06.773 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:55:06.791 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:55:06.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.939+0000 7f2777f39640 1 Processor -- start 2023-11-09T03:55:06.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.940+0000 7f2777f39640 1 -- start start 2023-11-09T03:55:06.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.940+0000 7f2777f39640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f277015d4e0 0x7f27700d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:06.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.940+0000 7f2777f39640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f27700d3b60 0x7f27700d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:06.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.940+0000 7f2777f39640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27700b32b0 con 0x7f277015d100 2023-11-09T03:55:06.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.940+0000 7f2777f39640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27700cec20 con 0x7f27700d3b60 2023-11-09T03:55:06.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.940+0000 7f2777f39640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27700cee10 con 0x7f277015d4e0 2023-11-09T03:55:06.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.941+0000 7f27754d4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f277015d4e0 0x7f27700d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:06.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.941+0000 7f27764d6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f27700d3b60 0x7f27700d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:06.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.941+0000 7f2775cd5640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f277015d100 0x7f27700ce510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:56006/0 (socket says 172.21.15.76:56006) 2023-11-09T03:55:06.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.941+0000 7f2775cd5640 1 -- 172.21.15.76:0/2105012537 learned_addr learned my addr 172.21.15.76:0/2105012537 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:55:06.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.941+0000 7f2774cd3640 1 -- 172.21.15.76:0/2105012537 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1186960878 0 0) 0x7f27700b32b0 con 0x7f277015d100 2023-11-09T03:55:06.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.941+0000 7f2774cd3640 1 -- 172.21.15.76:0/2105012537 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f275c003520 con 0x7f277015d100 2023-11-09T03:55:06.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.941+0000 7f27754d4640 1 -- 172.21.15.76:0/2105012537 >> v2:172.21.15.113:3300/0 conn(0x7f27700d3b60 msgr2=0x7f27700d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:06.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.941+0000 7f27754d4640 1 --2- 172.21.15.76:0/2105012537 >> v2:172.21.15.113:3300/0 conn(0x7f27700d3b60 0x7f27700d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:06.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.941+0000 7f27754d4640 1 -- 172.21.15.76:0/2105012537 >> v1:172.21.15.113:6789/0 conn(0x7f277015d100 legacy=0x7f27700ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:06.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.941+0000 7f27754d4640 1 -- 172.21.15.76:0/2105012537 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27700b4750 con 0x7f277015d4e0 2023-11-09T03:55:06.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.941+0000 7f27764d6640 1 --2- 172.21.15.76:0/2105012537 >> v2:172.21.15.113:3300/0 conn(0x7f27700d3b60 0x7f27700d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:55:06.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.942+0000 7f27754d4640 1 --2- 172.21.15.76:0/2105012537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f277015d4e0 0x7f27700d14d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f27600097a0 tx=0x7f276002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de73b1a6015cc0a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:06.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.942+0000 7f2774cd3640 1 -- 172.21.15.76:0/2105012537 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2760031830 con 0x7f277015d4e0 2023-11-09T03:55:06.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.942+0000 7f2774cd3640 1 -- 172.21.15.76:0/2105012537 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2760031990 con 0x7f277015d4e0 2023-11-09T03:55:06.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.942+0000 7f2774cd3640 1 -- 172.21.15.76:0/2105012537 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2760031d20 con 0x7f277015d4e0 2023-11-09T03:55:06.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.943+0000 7f2777f39640 1 -- 172.21.15.76:0/2105012537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f277015d4e0 msgr2=0x7f27700d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:06.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.943+0000 7f2777f39640 1 --2- 172.21.15.76:0/2105012537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f277015d4e0 0x7f27700d14d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f27600097a0 tx=0x7f276002b8d0 comp rx=0 tx=0).stop 2023-11-09T03:55:06.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.943+0000 7f2777f39640 1 -- 172.21.15.76:0/2105012537 shutdown_connections 2023-11-09T03:55:06.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.943+0000 7f2777f39640 1 --2- 172.21.15.76:0/2105012537 >> v2:172.21.15.113:3300/0 conn(0x7f27700d3b60 0x7f27700d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:06.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.943+0000 7f2777f39640 1 --2- 172.21.15.76:0/2105012537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f277015d4e0 0x7f27700d14d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:06.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.943+0000 7f2777f39640 1 -- 172.21.15.76:0/2105012537 >> 172.21.15.76:0/2105012537 conn(0x7f27700d2ef0 msgr2=0x7f27700d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:06.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.943+0000 7f2777f39640 1 -- 172.21.15.76:0/2105012537 shutdown_connections 2023-11-09T03:55:06.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.943+0000 7f2777f39640 1 -- 172.21.15.76:0/2105012537 wait complete. 2023-11-09T03:55:06.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.944+0000 7f2777f39640 1 Processor -- start 2023-11-09T03:55:06.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.944+0000 7f2777f39640 1 -- start start 2023-11-09T03:55:06.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.945+0000 7f2777f39640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27700d3b60 0x7f277014fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:06.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.945+0000 7f2777f39640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f277015d100 0x7f277015fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:06.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.945+0000 7f2777f39640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f277015d4e0 0x7f27701601b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:06.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.945+0000 7f2777f39640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27701573c0 con 0x7f277015d100 2023-11-09T03:55:06.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.945+0000 7f2777f39640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277015e730 con 0x7f277015d4e0 2023-11-09T03:55:06.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.945+0000 7f2777f39640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277015ea10 con 0x7f27700d3b60 2023-11-09T03:55:06.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.945+0000 7f27754d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f277015d100 0x7f277015fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:06.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.945+0000 7f27754d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f277015d100 0x7f277015fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47424/0 (socket says 172.21.15.76:47424) 2023-11-09T03:55:06.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.945+0000 7f27754d4640 1 -- 172.21.15.76:0/3577696579 learned_addr learned my addr 172.21.15.76:0/3577696579 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:06.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.945+0000 7f2775cd5640 1 --2- 172.21.15.76:0/3577696579 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27700d3b60 0x7f277014fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:06.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.945+0000 7f27764d6640 1 --2- 172.21.15.76:0/3577696579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f277015d4e0 0x7f27701601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:06.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.945+0000 7f27754d4640 1 -- 172.21.15.76:0/3577696579 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27700d3b60 msgr2=0x7f277014fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:06.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.945+0000 7f27754d4640 1 --2- 172.21.15.76:0/3577696579 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27700d3b60 0x7f277014fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:06.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.945+0000 7f27754d4640 1 -- 172.21.15.76:0/3577696579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f277015d4e0 msgr2=0x7f27701601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:06.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.945+0000 7f27754d4640 1 --2- 172.21.15.76:0/3577696579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f277015d4e0 0x7f27701601b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:06.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.945+0000 7f27754d4640 1 -- 172.21.15.76:0/3577696579 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2768009400 con 0x7f277015d100 2023-11-09T03:55:06.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.945+0000 7f27754d4640 1 --2- 172.21.15.76:0/3577696579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f277015d100 0x7f277015fc70 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f2760034040 tx=0x7f27600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:06.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.945+0000 7f27667fc640 1 -- 172.21.15.76:0/3577696579 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276003a830 con 0x7f277015d100 2023-11-09T03:55:06.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.946+0000 7f27667fc640 1 -- 172.21.15.76:0/3577696579 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276003a9b0 con 0x7f277015d100 2023-11-09T03:55:06.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.946+0000 7f2777f39640 1 -- 172.21.15.76:0/3577696579 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2760009400 con 0x7f277015d100 2023-11-09T03:55:06.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.946+0000 7f27667fc640 1 -- 172.21.15.76:0/3577696579 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276003ad60 con 0x7f277015d100 2023-11-09T03:55:06.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.946+0000 7f2777f39640 1 -- 172.21.15.76:0/3577696579 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2770150170 con 0x7f277015d100 2023-11-09T03:55:06.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.946+0000 7f2777f39640 1 -- 172.21.15.76:0/3577696579 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2770157590 con 0x7f277015d100 2023-11-09T03:55:06.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.947+0000 7f27667fc640 1 -- 172.21.15.76:0/3577696579 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f276004d440 con 0x7f277015d100 2023-11-09T03:55:06.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.947+0000 7f27667fc640 1 --2- 172.21.15.76:0/3577696579 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f275406ae70 0x7f275406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:06.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.947+0000 7f27667fc640 1 -- 172.21.15.76:0/3577696579 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7f27600ac1e0 con 0x7f277015d100 2023-11-09T03:55:06.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.948+0000 7f2775cd5640 1 --2- 172.21.15.76:0/3577696579 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f275406ae70 0x7f275406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:06.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.948+0000 7f2775cd5640 1 --2- 172.21.15.76:0/3577696579 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f275406ae70 0x7f275406d330 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f276c0015b0 tx=0x7f276c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:06.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:06.952+0000 7f27667fc640 1 -- 172.21.15.76:0/3577696579 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2770157590 con 0x7f277015d100 2023-11-09T03:55:07.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.119+0000 7f2777f39640 1 -- 172.21.15.76:0/3577696579 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27700684b0 con 0x7f277015d100 2023-11-09T03:55:07.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.121+0000 7f27667fc640 1 -- 172.21.15.76:0/3577696579 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+24035 (secure 0 0 0) 0x7f2760035090 con 0x7f277015d100 2023-11-09T03:55:07.121 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:55:07.121 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":118,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:54:57.509744+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-09T03:47:59.280601+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-09T03:47:59.742039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:48:55.490832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-09T03:48:56.352624+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":14,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.17","mappings":[{"from":0,"to":2}]},{"pgid":"17.36","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/3870807407":"2023-11-09T04:54:56.671385+0000","172.21.15.113:6836/3462040782":"2023-11-10T03:54:45.345437+0000","172.21.15.76:6834/2075292158":"2023-11-10T03:54:06.568191+0000","192.168.0.1:0/334503200":"2023-11-09T04:53:32.572840+0000","172.21.15.76:6835/2075292158":"2023-11-10T03:54:06.568191+0000","172.21.15.113:6837/7360844":"2023-11-10T03:53:24.718008+0000","172.21.15.113:6836/7360844":"2023-11-10T03:53:24.718008+0000","172.21.15.76:6835/546338722":"2023-11-10T03:53:17.470106+0000","172.21.15.76:6834/546338722":"2023-11-10T03:53:17.470106+0000","192.168.0.1:0/3087804343":"2023-11-09T04:52:42.513556+0000","172.21.15.76:6833/1536527256":"2023-11-10T03:52:34.213804+0000","172.21.15.113:6837/445306598":"2023-11-10T03:52:27.406838+0000","172.21.15.76:6834/3229239133":"2023-11-10T03:51:32.624856+0000","192.168.0.1:0/503424885":"2023-11-09T04:51:47.727422+0000","172.21.15.113:6837/3462040782":"2023-11-10T03:54:45.345437+0000","172.21.15.76:6834/4011275120":"2023-11-10T03:49:42.029057+0000","172.21.15.76:6835/4011275120":"2023-11-10T03:49:42.029057+0000","192.168.0.1:0/2519773110":"2023-11-09T04:54:14.823765+0000","172.21.15.113:6836/3280098776":"2023-11-10T03:50:07.616780+0000","172.21.15.113:6837/3280098776":"2023-11-10T03:50:07.616780+0000","172.21.15.76:6834/1663988060":"2023-11-10T03:50:33.665092+0000","172.21.15.76:6835/1663988060":"2023-11-10T03:50:33.665092+0000","172.21.15.113:6836/445306598":"2023-11-10T03:52:27.406838+0000","172.21.15.76:6833/3697850101":"2023-11-10T03:51:39.442929+0000","172.21.15.113:6836/3449910842":"2023-11-10T03:50:59.611256+0000","172.21.15.76:6835/3229239133":"2023-11-10T03:51:32.624856+0000","172.21.15.76:6832/1536527256":"2023-11-10T03:52:34.213804+0000","172.21.15.113:6837/3449910842":"2023-11-10T03:50:59.611256+0000","172.21.15.76:6832/3697850101":"2023-11-10T03:51:39.442929+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:55:07.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.123+0000 7f2777f39640 1 -- 172.21.15.76:0/3577696579 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f275406ae70 msgr2=0x7f275406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:07.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.123+0000 7f2777f39640 1 --2- 172.21.15.76:0/3577696579 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f275406ae70 0x7f275406d330 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f276c0015b0 tx=0x7f276c009040 comp rx=0 tx=0).stop 2023-11-09T03:55:07.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.123+0000 7f2777f39640 1 -- 172.21.15.76:0/3577696579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f277015d100 msgr2=0x7f277015fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:07.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.123+0000 7f2777f39640 1 --2- 172.21.15.76:0/3577696579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f277015d100 0x7f277015fc70 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f2760034040 tx=0x7f27600046c0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.123+0000 7f2777f39640 1 -- 172.21.15.76:0/3577696579 shutdown_connections 2023-11-09T03:55:07.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.123+0000 7f2777f39640 1 --2- 172.21.15.76:0/3577696579 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f275406ae70 0x7f275406d330 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.123+0000 7f2777f39640 1 --2- 172.21.15.76:0/3577696579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f277015d4e0 0x7f27701601b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.123+0000 7f2777f39640 1 --2- 172.21.15.76:0/3577696579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f277015d100 0x7f277015fc70 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.123+0000 7f2777f39640 1 --2- 172.21.15.76:0/3577696579 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27700d3b60 0x7f277014fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.123+0000 7f2777f39640 1 -- 172.21.15.76:0/3577696579 >> 172.21.15.76:0/3577696579 conn(0x7f27700d2ef0 msgr2=0x7f27700b01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:07.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.123+0000 7f2777f39640 1 -- 172.21.15.76:0/3577696579 shutdown_connections 2023-11-09T03:55:07.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.123+0000 7f2777f39640 1 -- 172.21.15.76:0/3577696579 wait complete. 2023-11-09T03:55:07.133 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-09T03:47:00.036540+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_data_pool', 'create_time': '2023-11-09T03:47:59.280601+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 70, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-09T03:47:59.742039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:48:55.490832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:48:56.352624+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 14, 'snap_epoch': 113, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-09T03:55:07.134 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:55:07.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.237+0000 7fbf6e983640 1 Processor -- start 2023-11-09T03:55:07.237 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.237+0000 7fbf6e983640 1 -- start start 2023-11-09T03:55:07.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.238+0000 7fbf6e983640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbf680b3ca0 0x7fbf6806b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:07.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.238+0000 7fbf6e983640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbf680b2100 0x7fbf680b2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:07.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.238+0000 7fbf6e983640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf680b2a50 con 0x7fbf6815d110 2023-11-09T03:55:07.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.238+0000 7fbf6e983640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf680664f0 con 0x7fbf680b2100 2023-11-09T03:55:07.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.238+0000 7fbf6e983640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf6815d4f0 con 0x7fbf680b3ca0 2023-11-09T03:55:07.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.238+0000 7fbf6cf20640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbf680b2100 0x7fbf680b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:07.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.238+0000 7fbf6cf20640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbf680b2100 0x7fbf680b2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47440/0 (socket says 172.21.15.76:47440) 2023-11-09T03:55:07.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.238+0000 7fbf6cf20640 1 -- 172.21.15.76:0/1597614359 learned_addr learned my addr 172.21.15.76:0/1597614359 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:07.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.238+0000 7fbf677fe640 1 --2- 172.21.15.76:0/1597614359 >> v2:172.21.15.113:3300/0 conn(0x7fbf680b3ca0 0x7fbf6806b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:07.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.238+0000 7fbf6cf20640 1 -- 172.21.15.76:0/1597614359 >> v2:172.21.15.113:3300/0 conn(0x7fbf680b3ca0 msgr2=0x7fbf6806b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:07.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.238+0000 7fbf6cf20640 1 --2- 172.21.15.76:0/1597614359 >> v2:172.21.15.113:3300/0 conn(0x7fbf680b3ca0 0x7fbf6806b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.238+0000 7fbf6cf20640 1 -- 172.21.15.76:0/1597614359 >> v1:172.21.15.113:6789/0 conn(0x7fbf6815d110 legacy=0x7fbf680ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:07.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.238+0000 7fbf6cf20640 1 -- 172.21.15.76:0/1597614359 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf6806b9f0 con 0x7fbf680b2100 2023-11-09T03:55:07.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.238+0000 7fbf677fe640 1 --2- 172.21.15.76:0/1597614359 >> v2:172.21.15.113:3300/0 conn(0x7fbf680b3ca0 0x7fbf6806b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:55:07.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.239+0000 7fbf6cf20640 1 --2- 172.21.15.76:0/1597614359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf680b2100 0x7fbf680b2510 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fbf6806fd80 tx=0x7fbf6002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b99fa7205ab7b63 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:07.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.239+0000 7fbf66ffd640 1 -- 172.21.15.76:0/1597614359 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6002ca80 con 0x7fbf680b2100 2023-11-09T03:55:07.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.239+0000 7fbf66ffd640 1 -- 172.21.15.76:0/1597614359 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf6002cbe0 con 0x7fbf680b2100 2023-11-09T03:55:07.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.239+0000 7fbf66ffd640 1 -- 172.21.15.76:0/1597614359 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf600343f0 con 0x7fbf680b2100 2023-11-09T03:55:07.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.240+0000 7fbf6e983640 1 -- 172.21.15.76:0/1597614359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf680b2100 msgr2=0x7fbf680b2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:07.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.240+0000 7fbf6e983640 1 --2- 172.21.15.76:0/1597614359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf680b2100 0x7fbf680b2510 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fbf6806fd80 tx=0x7fbf6002bee0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.240+0000 7fbf6e983640 1 -- 172.21.15.76:0/1597614359 shutdown_connections 2023-11-09T03:55:07.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.240+0000 7fbf6e983640 1 --2- 172.21.15.76:0/1597614359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf680b2100 0x7fbf680b2510 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.240+0000 7fbf6e983640 1 --2- 172.21.15.76:0/1597614359 >> v2:172.21.15.113:3300/0 conn(0x7fbf680b3ca0 0x7fbf6806b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.240+0000 7fbf6e983640 1 -- 172.21.15.76:0/1597614359 >> 172.21.15.76:0/1597614359 conn(0x7fbf680cbd30 msgr2=0x7fbf680cc140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:07.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.240+0000 7fbf6e983640 1 -- 172.21.15.76:0/1597614359 shutdown_connections 2023-11-09T03:55:07.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.240+0000 7fbf6e983640 1 -- 172.21.15.76:0/1597614359 wait complete. 2023-11-09T03:55:07.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.241+0000 7fbf6e983640 1 Processor -- start 2023-11-09T03:55:07.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.241+0000 7fbf6e983640 1 -- start start 2023-11-09T03:55:07.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.242+0000 7fbf6e983640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf680b2100 0x7fbf681f33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:07.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.242+0000 7fbf6e983640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf680b3ca0 0x7fbf681f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:07.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.242+0000 7fbf6e983640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf6815d110 0x7fbf681f6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:07.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.242+0000 7fbf6e983640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf68148240 con 0x7fbf680b3ca0 2023-11-09T03:55:07.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.242+0000 7fbf6e983640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf681f38f0 con 0x7fbf680b2100 2023-11-09T03:55:07.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.242+0000 7fbf6e983640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf680c93c0 con 0x7fbf6815d110 2023-11-09T03:55:07.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.242+0000 7fbf677fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf680b3ca0 0x7fbf681f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:07.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.242+0000 7fbf677fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf680b3ca0 0x7fbf681f5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47444/0 (socket says 172.21.15.76:47444) 2023-11-09T03:55:07.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.242+0000 7fbf677fe640 1 -- 172.21.15.76:0/2595857879 learned_addr learned my addr 172.21.15.76:0/2595857879 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:07.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.242+0000 7fbf67fff640 1 --2- 172.21.15.76:0/2595857879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf680b2100 0x7fbf681f33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:07.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.242+0000 7fbf6cf20640 1 --2- 172.21.15.76:0/2595857879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf6815d110 0x7fbf681f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:07.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.242+0000 7fbf677fe640 1 -- 172.21.15.76:0/2595857879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf6815d110 msgr2=0x7fbf681f6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:07.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.242+0000 7fbf677fe640 1 --2- 172.21.15.76:0/2595857879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf6815d110 0x7fbf681f6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.242+0000 7fbf677fe640 1 -- 172.21.15.76:0/2595857879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf680b2100 msgr2=0x7fbf681f33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:07.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.242+0000 7fbf677fe640 1 --2- 172.21.15.76:0/2595857879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf680b2100 0x7fbf681f33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.242+0000 7fbf677fe640 1 -- 172.21.15.76:0/2595857879 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf60009b30 con 0x7fbf680b3ca0 2023-11-09T03:55:07.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.242+0000 7fbf6cf20640 1 --2- 172.21.15.76:0/2595857879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf6815d110 0x7fbf681f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:55:07.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.243+0000 7fbf677fe640 1 --2- 172.21.15.76:0/2595857879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf680b3ca0 0x7fbf681f5f20 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00c7f0 tx=0x7fbf5c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:07.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.243+0000 7fbf64ff9640 1 -- 172.21.15.76:0/2595857879 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c007820 con 0x7fbf680b3ca0 2023-11-09T03:55:07.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.243+0000 7fbf64ff9640 1 -- 172.21.15.76:0/2595857879 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5c007980 con 0x7fbf680b3ca0 2023-11-09T03:55:07.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.243+0000 7fbf6e983640 1 -- 172.21.15.76:0/2595857879 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf680c8320 con 0x7fbf680b3ca0 2023-11-09T03:55:07.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.243+0000 7fbf64ff9640 1 -- 172.21.15.76:0/2595857879 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c007cf0 con 0x7fbf680b3ca0 2023-11-09T03:55:07.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.243+0000 7fbf6e983640 1 -- 172.21.15.76:0/2595857879 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf680c88e0 con 0x7fbf680b3ca0 2023-11-09T03:55:07.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.244+0000 7fbf6e983640 1 -- 172.21.15.76:0/2595857879 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf680684b0 con 0x7fbf680b3ca0 2023-11-09T03:55:07.253 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.244+0000 7fbf64ff9640 1 -- 172.21.15.76:0/2595857879 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbf5c0113f0 con 0x7fbf680b3ca0 2023-11-09T03:55:07.253 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.244+0000 7fbf64ff9640 1 --2- 172.21.15.76:0/2595857879 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbf5406ae70 0x7fbf5406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:07.253 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.245+0000 7fbf64ff9640 1 -- 172.21.15.76:0/2595857879 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7fbf5c08a0b0 con 0x7fbf680b3ca0 2023-11-09T03:55:07.253 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.245+0000 7fbf67fff640 1 --2- 172.21.15.76:0/2595857879 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbf5406ae70 0x7fbf5406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:07.253 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.245+0000 7fbf67fff640 1 --2- 172.21.15.76:0/2595857879 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbf5406ae70 0x7fbf5406d330 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fbf580098c0 tx=0x7fbf58009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:07.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.249+0000 7fbf64ff9640 1 -- 172.21.15.76:0/2595857879 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbf5c0547b0 con 0x7fbf680b3ca0 2023-11-09T03:55:07.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.414+0000 7fbf6e983640 1 -- 172.21.15.76:0/2595857879 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbf680cd360 con 0x7fbf680b3ca0 2023-11-09T03:55:07.414 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.414+0000 7fbf64ff9640 1 -- 172.21.15.76:0/2595857879 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbf5c01f0a0 con 0x7fbf680b3ca0 2023-11-09T03:55:07.415 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:55:07.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.416+0000 7fbf6e983640 1 -- 172.21.15.76:0/2595857879 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbf5406ae70 msgr2=0x7fbf5406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:07.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.416+0000 7fbf6e983640 1 --2- 172.21.15.76:0/2595857879 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbf5406ae70 0x7fbf5406d330 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fbf580098c0 tx=0x7fbf58009060 comp rx=0 tx=0).stop 2023-11-09T03:55:07.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.416+0000 7fbf6e983640 1 -- 172.21.15.76:0/2595857879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf680b3ca0 msgr2=0x7fbf681f5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:07.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.416+0000 7fbf6e983640 1 --2- 172.21.15.76:0/2595857879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf680b3ca0 0x7fbf681f5f20 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00c7f0 tx=0x7fbf5c00ccb0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.416+0000 7fbf6e983640 1 -- 172.21.15.76:0/2595857879 shutdown_connections 2023-11-09T03:55:07.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.416+0000 7fbf6e983640 1 --2- 172.21.15.76:0/2595857879 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbf5406ae70 0x7fbf5406d330 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.416+0000 7fbf6e983640 1 --2- 172.21.15.76:0/2595857879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf6815d110 0x7fbf681f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.416+0000 7fbf6e983640 1 --2- 172.21.15.76:0/2595857879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf680b3ca0 0x7fbf681f5f20 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.416+0000 7fbf6e983640 1 --2- 172.21.15.76:0/2595857879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf680b2100 0x7fbf681f33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.416+0000 7fbf6e983640 1 -- 172.21.15.76:0/2595857879 >> 172.21.15.76:0/2595857879 conn(0x7fbf680cbd30 msgr2=0x7fbf680acd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:07.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.416+0000 7fbf6e983640 1 -- 172.21.15.76:0/2595857879 shutdown_connections 2023-11-09T03:55:07.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.416+0000 7fbf6e983640 1 -- 172.21.15.76:0/2595857879 wait complete. 2023-11-09T03:55:07.427 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-09T03:55:07.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.531+0000 7ff206f25640 1 Processor -- start 2023-11-09T03:55:07.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.531+0000 7ff206f25640 1 -- start start 2023-11-09T03:55:07.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.532+0000 7ff206f25640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff200159e40 0x7ff2000b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:07.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.532+0000 7ff206f25640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff200071500 con 0x7ff200070700 2023-11-09T03:55:07.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.532+0000 7ff206f25640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff20015dae0 con 0x7ff20006bb60 2023-11-09T03:55:07.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.532+0000 7ff206f25640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff200070c60 con 0x7ff200159e40 2023-11-09T03:55:07.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.532+0000 7ff2054c2640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff200070700 0x7ff2000cd790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45748/0 (socket says 172.21.15.76:45748) 2023-11-09T03:55:07.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.532+0000 7ff2054c2640 1 -- 172.21.15.76:0/2343606899 learned_addr learned my addr 172.21.15.76:0/2343606899 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:55:07.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.532+0000 7ff1ff7fe640 1 -- 172.21.15.76:0/2343606899 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1579118108 0 0) 0x7ff200071500 con 0x7ff200070700 2023-11-09T03:55:07.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.532+0000 7ff204cc1640 1 --2- 172.21.15.76:0/2343606899 >> v2:172.21.15.113:3300/0 conn(0x7ff200159e40 0x7ff2000b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:07.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.532+0000 7ff1ff7fe640 1 -- 172.21.15.76:0/2343606899 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1f0005730 con 0x7ff200070700 2023-11-09T03:55:07.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.532+0000 7ff1ff7fe640 1 -- 172.21.15.76:0/2343606899 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 403597454 0 0) 0x7ff1f0005730 con 0x7ff200070700 2023-11-09T03:55:07.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.532+0000 7ff1ff7fe640 1 -- 172.21.15.76:0/2343606899 >> v2:172.21.15.113:3300/0 conn(0x7ff200159e40 msgr2=0x7ff2000b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:07.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.532+0000 7ff1ff7fe640 1 --2- 172.21.15.76:0/2343606899 >> v2:172.21.15.113:3300/0 conn(0x7ff200159e40 0x7ff2000b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.532+0000 7ff1ff7fe640 1 -- 172.21.15.76:0/2343606899 >> v1:172.21.15.113:6789/0 conn(0x7ff20006bb60 legacy=0x7ff20006fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:07.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.532+0000 7ff1ff7fe640 1 -- 172.21.15.76:0/2343606899 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff200070f40 con 0x7ff200070700 2023-11-09T03:55:07.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.532+0000 7ff1ff7fe640 1 -- 172.21.15.76:0/2343606899 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7ff1f80019f0 con 0x7ff200070700 2023-11-09T03:55:07.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.532+0000 7ff1ff7fe640 1 -- 172.21.15.76:0/2343606899 >> v1:172.21.15.76:6789/0 conn(0x7ff200070700 legacy=0x7ff2000cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:07.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.532+0000 7ff1ff7fe640 1 --2- 172.21.15.76:0/2343606899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1f0005b80 0x7ff1f0005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:07.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.532+0000 7ff1ff7fe640 1 --2- 172.21.15.76:0/2343606899 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1f00069d0 0x7ff1f0006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:07.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.532+0000 7ff1ff7fe640 1 --2- 172.21.15.76:0/2343606899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1f00074b0 0x7ff1f000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:07.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.532+0000 7ff1ff7fe640 1 -- 172.21.15.76:0/2343606899 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff200070c60 con 0x7ff1f00074b0 2023-11-09T03:55:07.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.532+0000 7ff1ff7fe640 1 -- 172.21.15.76:0/2343606899 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f000e380 con 0x7ff1f0005b80 2023-11-09T03:55:07.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.532+0000 7ff1ff7fe640 1 -- 172.21.15.76:0/2343606899 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f000e540 con 0x7ff1f00069d0 2023-11-09T03:55:07.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.533+0000 7ff2054c2640 1 --2- 172.21.15.76:0/2343606899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1f00074b0 0x7ff1f000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:07.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.533+0000 7ff204cc1640 1 --2- 172.21.15.76:0/2343606899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1f0005b80 0x7ff1f0005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:07.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.533+0000 7ff1fffff640 1 --2- 172.21.15.76:0/2343606899 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1f00069d0 0x7ff1f0006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:07.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.533+0000 7ff2054c2640 1 -- 172.21.15.76:0/2343606899 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1f00069d0 msgr2=0x7ff1f0006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:07.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.533+0000 7ff2054c2640 1 --2- 172.21.15.76:0/2343606899 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1f00069d0 0x7ff1f0006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.533+0000 7ff2054c2640 1 -- 172.21.15.76:0/2343606899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1f0005b80 msgr2=0x7ff1f0005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:07.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.533+0000 7ff2054c2640 1 --2- 172.21.15.76:0/2343606899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1f0005b80 0x7ff1f0005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.533+0000 7ff2054c2640 1 -- 172.21.15.76:0/2343606899 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f0005730 con 0x7ff1f00074b0 2023-11-09T03:55:07.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.533+0000 7ff2054c2640 1 -- 172.21.15.76:0/2343606899 reap_dead start 2023-11-09T03:55:07.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.534+0000 7ff2054c2640 1 --2- 172.21.15.76:0/2343606899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1f00074b0 0x7ff1f000db90 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7ff1f800a800 tx=0x7ff1f0007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b2aa6dd0dd51d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:07.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.534+0000 7ff1ff7fe640 1 -- 172.21.15.76:0/2343606899 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f80072d0 con 0x7ff1f00074b0 2023-11-09T03:55:07.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.534+0000 7ff1ff7fe640 1 -- 172.21.15.76:0/2343606899 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f802f070 con 0x7ff1f00074b0 2023-11-09T03:55:07.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.534+0000 7ff1ff7fe640 1 -- 172.21.15.76:0/2343606899 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8007670 con 0x7ff1f00074b0 2023-11-09T03:55:07.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.534+0000 7ff206f25640 1 -- 172.21.15.76:0/2343606899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1f00074b0 msgr2=0x7ff1f000db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:07.544 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.534+0000 7ff206f25640 1 --2- 172.21.15.76:0/2343606899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1f00074b0 0x7ff1f000db90 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7ff1f800a800 tx=0x7ff1f0007a00 comp rx=0 tx=0).stop 2023-11-09T03:55:07.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.534+0000 7ff206f25640 1 -- 172.21.15.76:0/2343606899 shutdown_connections 2023-11-09T03:55:07.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.534+0000 7ff206f25640 1 --2- 172.21.15.76:0/2343606899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1f00074b0 0x7ff1f000db90 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.534+0000 7ff206f25640 1 -- 172.21.15.76:0/2343606899 >> 172.21.15.76:0/2343606899 conn(0x7ff2000cb530 msgr2=0x7ff2000cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:07.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.534+0000 7ff206f25640 1 -- 172.21.15.76:0/2343606899 shutdown_connections 2023-11-09T03:55:07.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.534+0000 7ff206f25640 1 -- 172.21.15.76:0/2343606899 wait complete. 2023-11-09T03:55:07.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.535+0000 7ff206f25640 1 Processor -- start 2023-11-09T03:55:07.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.535+0000 7ff206f25640 1 -- start start 2023-11-09T03:55:07.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.536+0000 7ff206f25640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1f00074b0 0x7ff2001575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:07.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.536+0000 7ff206f25640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2000c7f10 0x7ff2000c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:07.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.536+0000 7ff206f25640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff200151ac0 0x7ff200151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:07.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.536+0000 7ff206f25640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20015b800 con 0x7ff1f00074b0 2023-11-09T03:55:07.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.536+0000 7ff206f25640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20015ba50 con 0x7ff200151ac0 2023-11-09T03:55:07.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.536+0000 7ff206f25640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff20015bd30 con 0x7ff2000c7f10 2023-11-09T03:55:07.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.536+0000 7ff204cc1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1f00074b0 0x7ff2001575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:07.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.536+0000 7ff204cc1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1f00074b0 0x7ff2001575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47472/0 (socket says 172.21.15.76:47472) 2023-11-09T03:55:07.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.536+0000 7ff204cc1640 1 -- 172.21.15.76:0/1647385914 learned_addr learned my addr 172.21.15.76:0/1647385914 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:07.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.537+0000 7ff2054c2640 1 --2- 172.21.15.76:0/1647385914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff200151ac0 0x7ff200151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:07.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.537+0000 7ff1fffff640 1 --2- 172.21.15.76:0/1647385914 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2000c7f10 0x7ff2000c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:07.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.537+0000 7ff204cc1640 1 -- 172.21.15.76:0/1647385914 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2000c7f10 msgr2=0x7ff2000c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:07.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.537+0000 7ff204cc1640 1 --2- 172.21.15.76:0/1647385914 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2000c7f10 0x7ff2000c8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.537+0000 7ff204cc1640 1 -- 172.21.15.76:0/1647385914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff200151ac0 msgr2=0x7ff200151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:07.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.537+0000 7ff204cc1640 1 --2- 172.21.15.76:0/1647385914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff200151ac0 0x7ff200151f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.537+0000 7ff204cc1640 1 -- 172.21.15.76:0/1647385914 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff20015c010 con 0x7ff1f00074b0 2023-11-09T03:55:07.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.537+0000 7ff204cc1640 1 --2- 172.21.15.76:0/1647385914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1f00074b0 0x7ff2001575a0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7ff1f40118e0 tx=0x7ff1f4011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:07.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.537+0000 7ff1fd7fa640 1 -- 172.21.15.76:0/1647385914 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f400faf0 con 0x7ff1f00074b0 2023-11-09T03:55:07.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.537+0000 7ff1fd7fa640 1 -- 172.21.15.76:0/1647385914 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f400fc50 con 0x7ff1f00074b0 2023-11-09T03:55:07.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.537+0000 7ff206f25640 1 -- 172.21.15.76:0/1647385914 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff20015c2f0 con 0x7ff1f00074b0 2023-11-09T03:55:07.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.537+0000 7ff1fd7fa640 1 -- 172.21.15.76:0/1647385914 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4014440 con 0x7ff1f00074b0 2023-11-09T03:55:07.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.537+0000 7ff206f25640 1 -- 172.21.15.76:0/1647385914 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff200152460 con 0x7ff1f00074b0 2023-11-09T03:55:07.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.538+0000 7ff206f25640 1 -- 172.21.15.76:0/1647385914 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2000cfa20 con 0x7ff1f00074b0 2023-11-09T03:55:07.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.538+0000 7ff1fd7fa640 1 -- 172.21.15.76:0/1647385914 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff1f40145a0 con 0x7ff1f00074b0 2023-11-09T03:55:07.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.539+0000 7ff1fd7fa640 1 --2- 172.21.15.76:0/1647385914 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff1e406aea0 0x7ff1e406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:07.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.539+0000 7ff1fd7fa640 1 -- 172.21.15.76:0/1647385914 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7ff1f408a960 con 0x7ff1f00074b0 2023-11-09T03:55:07.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.539+0000 7ff1fffff640 1 --2- 172.21.15.76:0/1647385914 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff1e406aea0 0x7ff1e406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:07.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.540+0000 7ff1fffff640 1 --2- 172.21.15.76:0/1647385914 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff1e406aea0 0x7ff1e406d360 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec000c00 tx=0x7ff1ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:07.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.543+0000 7ff1fd7fa640 1 -- 172.21.15.76:0/1647385914 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff1f4054fb0 con 0x7ff1f00074b0 2023-11-09T03:55:07.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.708+0000 7ff206f25640 1 -- 172.21.15.76:0/1647385914 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7ff2000684b0 con 0x7ff1f00074b0 2023-11-09T03:55:07.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.708+0000 7ff1fd7fa640 1 -- 172.21.15.76:0/1647385914 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v118) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff1f40585d0 con 0x7ff1f00074b0 2023-11-09T03:55:07.709 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:55:07.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.710+0000 7ff206f25640 1 -- 172.21.15.76:0/1647385914 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff1e406aea0 msgr2=0x7ff1e406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:07.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.710+0000 7ff206f25640 1 --2- 172.21.15.76:0/1647385914 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff1e406aea0 0x7ff1e406d360 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec000c00 tx=0x7ff1ec009040 comp rx=0 tx=0).stop 2023-11-09T03:55:07.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.710+0000 7ff206f25640 1 -- 172.21.15.76:0/1647385914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1f00074b0 msgr2=0x7ff2001575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:07.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.710+0000 7ff206f25640 1 --2- 172.21.15.76:0/1647385914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1f00074b0 0x7ff2001575a0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7ff1f40118e0 tx=0x7ff1f4011da0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.710+0000 7ff206f25640 1 -- 172.21.15.76:0/1647385914 shutdown_connections 2023-11-09T03:55:07.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.710+0000 7ff206f25640 1 --2- 172.21.15.76:0/1647385914 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff1e406aea0 0x7ff1e406d360 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.711 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.710+0000 7ff206f25640 1 --2- 172.21.15.76:0/1647385914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff200151ac0 0x7ff200151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.710+0000 7ff206f25640 1 --2- 172.21.15.76:0/1647385914 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2000c7f10 0x7ff2000c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.710+0000 7ff206f25640 1 --2- 172.21.15.76:0/1647385914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1f00074b0 0x7ff2001575a0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.710+0000 7ff206f25640 1 -- 172.21.15.76:0/1647385914 >> 172.21.15.76:0/1647385914 conn(0x7ff2000cb530 msgr2=0x7ff2000cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:07.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.710+0000 7ff206f25640 1 -- 172.21.15.76:0/1647385914 shutdown_connections 2023-11-09T03:55:07.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.710+0000 7ff206f25640 1 -- 172.21.15.76:0/1647385914 wait complete. 2023-11-09T03:55:07.721 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-09T03:55:07.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.823+0000 7f51c8355640 1 Processor -- start 2023-11-09T03:55:07.823 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.823+0000 7f51c8355640 1 -- start start 2023-11-09T03:55:07.825 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.824+0000 7f51c8355640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f51c015d100 0x7f51c00ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:07.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.824+0000 7f51c8355640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51c015d4e0 0x7f51c00d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:07.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.824+0000 7f51c8355640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c00b32b0 con 0x7f51c00d3b60 2023-11-09T03:55:07.826 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.824+0000 7f51c8355640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51c00cee10 con 0x7f51c015d100 2023-11-09T03:55:07.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.824+0000 7f51c8355640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c00b4750 con 0x7f51c015d4e0 2023-11-09T03:55:07.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.824+0000 7f51c68f2640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f51c00d3b60 0x7f51c00cf520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45750/0 (socket says 172.21.15.76:45750) 2023-11-09T03:55:07.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.824+0000 7f51c68f2640 1 -- 172.21.15.76:0/1111742003 learned_addr learned my addr 172.21.15.76:0/1111742003 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:55:07.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.825+0000 7f51c60f1640 1 --2- 172.21.15.76:0/1111742003 >> v2:172.21.15.113:3300/0 conn(0x7f51c015d100 0x7f51c00ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:07.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.825+0000 7f51c58f0640 1 --2- 172.21.15.76:0/1111742003 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51c015d4e0 0x7f51c00d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:07.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.825+0000 7f51c50ef640 1 -- 172.21.15.76:0/1111742003 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3136435261 0 0) 0x7f51c00b32b0 con 0x7f51c00d3b60 2023-11-09T03:55:07.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.825+0000 7f51c50ef640 1 -- 172.21.15.76:0/1111742003 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51ac003520 con 0x7f51c00d3b60 2023-11-09T03:55:07.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.825+0000 7f51c60f1640 1 -- 172.21.15.76:0/1111742003 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51c015d4e0 msgr2=0x7f51c00d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:07.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.825+0000 7f51c60f1640 1 --2- 172.21.15.76:0/1111742003 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51c015d4e0 0x7f51c00d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.825+0000 7f51c60f1640 1 -- 172.21.15.76:0/1111742003 >> v1:172.21.15.76:6789/0 conn(0x7f51c00d3b60 legacy=0x7f51c00cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:07.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.825+0000 7f51c60f1640 1 -- 172.21.15.76:0/1111742003 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c00b2980 con 0x7f51c015d100 2023-11-09T03:55:07.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.826+0000 7f51c60f1640 1 --2- 172.21.15.76:0/1111742003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51c015d100 0x7f51c00ce510 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f51bc009de0 tx=0x7f51bc02bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5017351586e0b177 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:07.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.826+0000 7f51c50ef640 1 -- 172.21.15.76:0/1111742003 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc02cc90 con 0x7f51c015d100 2023-11-09T03:55:07.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.826+0000 7f51c50ef640 1 -- 172.21.15.76:0/1111742003 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51bc02cdf0 con 0x7f51c015d100 2023-11-09T03:55:07.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.826+0000 7f51c50ef640 1 -- 172.21.15.76:0/1111742003 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc034640 con 0x7f51c015d100 2023-11-09T03:55:07.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.826+0000 7f51c8355640 1 -- 172.21.15.76:0/1111742003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51c015d100 msgr2=0x7f51c00ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:07.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.826+0000 7f51c8355640 1 --2- 172.21.15.76:0/1111742003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51c015d100 0x7f51c00ce510 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f51bc009de0 tx=0x7f51bc02bee0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.826+0000 7f51c8355640 1 -- 172.21.15.76:0/1111742003 shutdown_connections 2023-11-09T03:55:07.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.826+0000 7f51c8355640 1 --2- 172.21.15.76:0/1111742003 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51c015d4e0 0x7f51c00d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.826+0000 7f51c8355640 1 --2- 172.21.15.76:0/1111742003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51c015d100 0x7f51c00ce510 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.826+0000 7f51c8355640 1 -- 172.21.15.76:0/1111742003 >> 172.21.15.76:0/1111742003 conn(0x7f51c00d2ef0 msgr2=0x7f51c00d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:07.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.826+0000 7f51c8355640 1 -- 172.21.15.76:0/1111742003 shutdown_connections 2023-11-09T03:55:07.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.826+0000 7f51c8355640 1 -- 172.21.15.76:0/1111742003 wait complete. 2023-11-09T03:55:07.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.828+0000 7f51c8355640 1 Processor -- start 2023-11-09T03:55:07.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.828+0000 7f51c8355640 1 -- start start 2023-11-09T03:55:07.835 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.828+0000 7f51c8355640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51c00d3b60 0x7f51c0152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:07.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.828+0000 7f51c8355640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51c015d100 0x7f51c0155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:07.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.828+0000 7f51c8355640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51c015d4e0 0x7f51c00b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:07.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.828+0000 7f51c8355640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c01f7160 con 0x7f51c00d3b60 2023-11-09T03:55:07.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.828+0000 7f51c8355640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c0154da0 con 0x7f51c015d4e0 2023-11-09T03:55:07.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.828+0000 7f51c8355640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c0154210 con 0x7f51c015d100 2023-11-09T03:55:07.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.828+0000 7f51c60f1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51c00d3b60 0x7f51c0152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:07.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.828+0000 7f51c60f1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51c00d3b60 0x7f51c0152250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47482/0 (socket says 172.21.15.76:47482) 2023-11-09T03:55:07.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.828+0000 7f51c60f1640 1 -- 172.21.15.76:0/4001526440 learned_addr learned my addr 172.21.15.76:0/4001526440 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:07.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.828+0000 7f51c68f2640 1 --2- 172.21.15.76:0/4001526440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51c015d4e0 0x7f51c00b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:07.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.828+0000 7f51c58f0640 1 --2- 172.21.15.76:0/4001526440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51c015d100 0x7f51c0155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:07.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.829+0000 7f51c60f1640 1 -- 172.21.15.76:0/4001526440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51c015d100 msgr2=0x7f51c0155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:07.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.829+0000 7f51c60f1640 1 --2- 172.21.15.76:0/4001526440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51c015d100 0x7f51c0155230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.829+0000 7f51c60f1640 1 -- 172.21.15.76:0/4001526440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51c015d4e0 msgr2=0x7f51c00b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:07.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.829+0000 7f51c60f1640 1 --2- 172.21.15.76:0/4001526440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51c015d4e0 0x7f51c00b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:07.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.829+0000 7f51c60f1640 1 -- 172.21.15.76:0/4001526440 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51bc009a70 con 0x7f51c00d3b60 2023-11-09T03:55:07.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.829+0000 7f51c60f1640 1 --2- 172.21.15.76:0/4001526440 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51c00d3b60 0x7f51c0152250 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f51bc009f00 tx=0x7f51bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:07.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.829+0000 7f51b6ffd640 1 -- 172.21.15.76:0/4001526440 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc03e610 con 0x7f51c00d3b60 2023-11-09T03:55:07.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.829+0000 7f51b6ffd640 1 -- 172.21.15.76:0/4001526440 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51bc03e770 con 0x7f51c00d3b60 2023-11-09T03:55:07.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.829+0000 7f51c8355640 1 -- 172.21.15.76:0/4001526440 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c01544b0 con 0x7f51c00d3b60 2023-11-09T03:55:07.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.829+0000 7f51b6ffd640 1 -- 172.21.15.76:0/4001526440 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc03eb20 con 0x7f51c00d3b60 2023-11-09T03:55:07.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.829+0000 7f51c8355640 1 -- 172.21.15.76:0/4001526440 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c0154a30 con 0x7f51c00d3b60 2023-11-09T03:55:07.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.830+0000 7f51c8355640 1 -- 172.21.15.76:0/4001526440 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c00684b0 con 0x7f51c00d3b60 2023-11-09T03:55:07.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.831+0000 7f51b6ffd640 1 -- 172.21.15.76:0/4001526440 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f51bc03b050 con 0x7f51c00d3b60 2023-11-09T03:55:07.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.831+0000 7f51b6ffd640 1 --2- 172.21.15.76:0/4001526440 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f51a406aea0 0x7f51a406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:07.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.831+0000 7f51b6ffd640 1 -- 172.21.15.76:0/4001526440 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7f51bc0ac510 con 0x7f51c00d3b60 2023-11-09T03:55:07.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.831+0000 7f51c58f0640 1 --2- 172.21.15.76:0/4001526440 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f51a406aea0 0x7f51a406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:07.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.832+0000 7f51c58f0640 1 --2- 172.21.15.76:0/4001526440 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f51a406aea0 0x7f51a406d360 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f51c00cb660 tx=0x7f51b0009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:07.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:07.835+0000 7f51b6ffd640 1 -- 172.21.15.76:0/4001526440 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f51bc076b80 con 0x7f51c00d3b60 2023-11-09T03:55:08.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.001+0000 7f51c8355640 1 -- 172.21.15.76:0/4001526440 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7f51c00ce6b0 con 0x7f51c00d3b60 2023-11-09T03:55:08.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.001+0000 7f51b6ffd640 1 -- 172.21.15.76:0/4001526440 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v118) v1 ==== 103+0+11 (secure 0 0 0) 0x7f51bc0390a0 con 0x7f51c00d3b60 2023-11-09T03:55:08.002 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:55:08.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.003+0000 7f51c8355640 1 -- 172.21.15.76:0/4001526440 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f51a406aea0 msgr2=0x7f51a406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:08.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.003+0000 7f51c8355640 1 --2- 172.21.15.76:0/4001526440 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f51a406aea0 0x7f51a406d360 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f51c00cb660 tx=0x7f51b0009190 comp rx=0 tx=0).stop 2023-11-09T03:55:08.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.003+0000 7f51c8355640 1 -- 172.21.15.76:0/4001526440 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51c00d3b60 msgr2=0x7f51c0152250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:08.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.003+0000 7f51c8355640 1 --2- 172.21.15.76:0/4001526440 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51c00d3b60 0x7f51c0152250 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f51bc009f00 tx=0x7f51bc004a40 comp rx=0 tx=0).stop 2023-11-09T03:55:08.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.003+0000 7f51c8355640 1 -- 172.21.15.76:0/4001526440 shutdown_connections 2023-11-09T03:55:08.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.003+0000 7f51c8355640 1 --2- 172.21.15.76:0/4001526440 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f51a406aea0 0x7f51a406d360 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.003+0000 7f51c8355640 1 --2- 172.21.15.76:0/4001526440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51c015d4e0 0x7f51c00b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.003+0000 7f51c8355640 1 --2- 172.21.15.76:0/4001526440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51c015d100 0x7f51c0155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.003+0000 7f51c8355640 1 --2- 172.21.15.76:0/4001526440 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51c00d3b60 0x7f51c0152250 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.003+0000 7f51c8355640 1 -- 172.21.15.76:0/4001526440 >> 172.21.15.76:0/4001526440 conn(0x7f51c00d2ef0 msgr2=0x7f51c00c9880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:08.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.003+0000 7f51c8355640 1 -- 172.21.15.76:0/4001526440 shutdown_connections 2023-11-09T03:55:08.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.003+0000 7f51c8355640 1 -- 172.21.15.76:0/4001526440 wait complete. 2023-11-09T03:55:08.014 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-09T03:55:08.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.118+0000 7facd8133640 1 Processor -- start 2023-11-09T03:55:08.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.118+0000 7facd8133640 1 -- start start 2023-11-09T03:55:08.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.119+0000 7facd8133640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7facd0159e40 0x7facd00b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:08.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.119+0000 7facd8133640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7facd006bb60 0x7facd00b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:08.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.119+0000 7facd8133640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facd006fff0 0x7facd0070460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:08.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.119+0000 7facd8133640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facd0071500 con 0x7facd006bb60 2023-11-09T03:55:08.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.119+0000 7facd8133640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facd015dae0 con 0x7facd0159e40 2023-11-09T03:55:08.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.119+0000 7facd8133640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facd00709a0 con 0x7facd006fff0 2023-11-09T03:55:08.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.119+0000 7facd56ce640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7facd006bb60 0x7facd00b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:08.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.119+0000 7facd56ce640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7facd006bb60 0x7facd00b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47496/0 (socket says 172.21.15.76:47496) 2023-11-09T03:55:08.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.119+0000 7facd56ce640 1 -- 172.21.15.76:0/2958187679 learned_addr learned my addr 172.21.15.76:0/2958187679 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:08.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.119+0000 7facd56ce640 1 -- 172.21.15.76:0/2958187679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facd006fff0 msgr2=0x7facd0070460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:08.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.119+0000 7facd56ce640 1 --2- 172.21.15.76:0/2958187679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facd006fff0 0x7facd0070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.119+0000 7facd56ce640 1 -- 172.21.15.76:0/2958187679 >> v2:172.21.15.113:3300/0 conn(0x7facd0159e40 msgr2=0x7facd00b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:08.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.119+0000 7facd5ecf640 1 --2- 172.21.15.76:0/2958187679 >> v2:172.21.15.113:3300/0 conn(0x7facd0159e40 0x7facd00b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:08.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.119+0000 7facd56ce640 1 --2- 172.21.15.76:0/2958187679 >> v2:172.21.15.113:3300/0 conn(0x7facd0159e40 0x7facd00b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.119+0000 7facd56ce640 1 -- 172.21.15.76:0/2958187679 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd0070c60 con 0x7facd006bb60 2023-11-09T03:55:08.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.119+0000 7facd5ecf640 1 --2- 172.21.15.76:0/2958187679 >> v2:172.21.15.113:3300/0 conn(0x7facd0159e40 0x7facd00b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:55:08.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.120+0000 7facd56ce640 1 --2- 172.21.15.76:0/2958187679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facd006bb60 0x7facd00b0a70 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7facd00ce7f0 tx=0x7facc002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61bde53ced33e57d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:08.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.120+0000 7facd4ecd640 1 -- 172.21.15.76:0/2958187679 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc00317e0 con 0x7facd006bb60 2023-11-09T03:55:08.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.120+0000 7facd4ecd640 1 -- 172.21.15.76:0/2958187679 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc0031940 con 0x7facd006bb60 2023-11-09T03:55:08.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.120+0000 7facd4ecd640 1 -- 172.21.15.76:0/2958187679 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc0031cd0 con 0x7facd006bb60 2023-11-09T03:55:08.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.121+0000 7facd8133640 1 -- 172.21.15.76:0/2958187679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facd006bb60 msgr2=0x7facd00b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:08.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.121+0000 7facd8133640 1 --2- 172.21.15.76:0/2958187679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facd006bb60 0x7facd00b0a70 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7facd00ce7f0 tx=0x7facc002b880 comp rx=0 tx=0).stop 2023-11-09T03:55:08.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.121+0000 7facd8133640 1 -- 172.21.15.76:0/2958187679 shutdown_connections 2023-11-09T03:55:08.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.121+0000 7facd8133640 1 --2- 172.21.15.76:0/2958187679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facd006fff0 0x7facd0070460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.121+0000 7facd8133640 1 --2- 172.21.15.76:0/2958187679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facd006bb60 0x7facd00b0a70 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.121+0000 7facd8133640 1 --2- 172.21.15.76:0/2958187679 >> v2:172.21.15.113:3300/0 conn(0x7facd0159e40 0x7facd00b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.121+0000 7facd8133640 1 -- 172.21.15.76:0/2958187679 >> 172.21.15.76:0/2958187679 conn(0x7facd00cb530 msgr2=0x7facd00cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:08.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.121+0000 7facd8133640 1 -- 172.21.15.76:0/2958187679 shutdown_connections 2023-11-09T03:55:08.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.121+0000 7facd8133640 1 -- 172.21.15.76:0/2958187679 wait complete. 2023-11-09T03:55:08.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.122+0000 7facd8133640 1 Processor -- start 2023-11-09T03:55:08.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.122+0000 7facd8133640 1 -- start start 2023-11-09T03:55:08.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.122+0000 7facd8133640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facd006bb60 0x7facd0149f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:08.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.122+0000 7facd8133640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facd006fff0 0x7facd00c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:08.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.122+0000 7facd8133640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facd0159e40 0x7facd015ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:08.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.122+0000 7facd8133640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd00c7050 con 0x7facd0159e40 2023-11-09T03:55:08.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.122+0000 7facd8133640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd014a4b0 con 0x7facd006bb60 2023-11-09T03:55:08.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.122+0000 7facd8133640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facd00c63d0 con 0x7facd006fff0 2023-11-09T03:55:08.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.123+0000 7facd66d0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facd0159e40 0x7facd015ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:08.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.123+0000 7facd66d0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facd0159e40 0x7facd015ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47504/0 (socket says 172.21.15.76:47504) 2023-11-09T03:55:08.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.123+0000 7facd66d0640 1 -- 172.21.15.76:0/2656241129 learned_addr learned my addr 172.21.15.76:0/2656241129 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:08.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.123+0000 7facd5ecf640 1 --2- 172.21.15.76:0/2656241129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facd006bb60 0x7facd0149f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:08.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.123+0000 7facd56ce640 1 --2- 172.21.15.76:0/2656241129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facd006fff0 0x7facd00c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:08.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.123+0000 7facd66d0640 1 -- 172.21.15.76:0/2656241129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facd006fff0 msgr2=0x7facd00c5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:08.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.123+0000 7facd66d0640 1 --2- 172.21.15.76:0/2656241129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facd006fff0 0x7facd00c5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.123+0000 7facd66d0640 1 -- 172.21.15.76:0/2656241129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facd006bb60 msgr2=0x7facd0149f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:08.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.123+0000 7facd66d0640 1 --2- 172.21.15.76:0/2656241129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facd006bb60 0x7facd0149f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.123+0000 7facd66d0640 1 -- 172.21.15.76:0/2656241129 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc00094d0 con 0x7facd0159e40 2023-11-09T03:55:08.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.123+0000 7facd66d0640 1 --2- 172.21.15.76:0/2656241129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facd0159e40 0x7facd015ba40 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7facc800b970 tx=0x7facc800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:08.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.123+0000 7facc67fc640 1 -- 172.21.15.76:0/2656241129 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc800cb40 con 0x7facd0159e40 2023-11-09T03:55:08.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.123+0000 7facc67fc640 1 -- 172.21.15.76:0/2656241129 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc800ccc0 con 0x7facd0159e40 2023-11-09T03:55:08.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.123+0000 7facc67fc640 1 -- 172.21.15.76:0/2656241129 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc8014680 con 0x7facd0159e40 2023-11-09T03:55:08.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.123+0000 7facd8133640 1 -- 172.21.15.76:0/2656241129 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd00c66b0 con 0x7facd0159e40 2023-11-09T03:55:08.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.123+0000 7facd8133640 1 -- 172.21.15.76:0/2656241129 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd00c6c70 con 0x7facd0159e40 2023-11-09T03:55:08.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.124+0000 7facd8133640 1 -- 172.21.15.76:0/2656241129 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facd00c7190 con 0x7facd0159e40 2023-11-09T03:55:08.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.125+0000 7facc67fc640 1 -- 172.21.15.76:0/2656241129 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7facc801d3b0 con 0x7facd0159e40 2023-11-09T03:55:08.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.125+0000 7facc67fc640 1 --2- 172.21.15.76:0/2656241129 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7facb406ae70 0x7facb406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:08.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.125+0000 7facc67fc640 1 -- 172.21.15.76:0/2656241129 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7facc8089f60 con 0x7facd0159e40 2023-11-09T03:55:08.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.126+0000 7facd5ecf640 1 --2- 172.21.15.76:0/2656241129 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7facb406ae70 0x7facb406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:08.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.126+0000 7facd5ecf640 1 --2- 172.21.15.76:0/2656241129 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7facb406ae70 0x7facb406d330 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7facd015e4b0 tx=0x7faccc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:08.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.129+0000 7facc67fc640 1 -- 172.21.15.76:0/2656241129 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7facd00c7190 con 0x7facd0159e40 2023-11-09T03:55:08.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.293+0000 7facd8133640 1 -- 172.21.15.76:0/2656241129 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7facd00c7190 con 0x7facd0159e40 2023-11-09T03:55:08.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.293+0000 7facc67fc640 1 -- 172.21.15.76:0/2656241129 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v118) v1 ==== 107+0+11 (secure 0 0 0) 0x7facc8087e60 con 0x7facd0159e40 2023-11-09T03:55:08.293 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:55:08.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.295+0000 7facd8133640 1 -- 172.21.15.76:0/2656241129 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7facb406ae70 msgr2=0x7facb406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:08.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.295+0000 7facd8133640 1 --2- 172.21.15.76:0/2656241129 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7facb406ae70 0x7facb406d330 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7facd015e4b0 tx=0x7faccc008040 comp rx=0 tx=0).stop 2023-11-09T03:55:08.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.295+0000 7facd8133640 1 -- 172.21.15.76:0/2656241129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facd0159e40 msgr2=0x7facd015ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:08.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.295+0000 7facd8133640 1 --2- 172.21.15.76:0/2656241129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facd0159e40 0x7facd015ba40 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7facc800b970 tx=0x7facc800be30 comp rx=0 tx=0).stop 2023-11-09T03:55:08.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.295+0000 7facd8133640 1 -- 172.21.15.76:0/2656241129 shutdown_connections 2023-11-09T03:55:08.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.295+0000 7facd8133640 1 --2- 172.21.15.76:0/2656241129 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7facb406ae70 0x7facb406d330 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.295+0000 7facd8133640 1 --2- 172.21.15.76:0/2656241129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facd0159e40 0x7facd015ba40 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.295+0000 7facd8133640 1 --2- 172.21.15.76:0/2656241129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facd006fff0 0x7facd00c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.295+0000 7facd8133640 1 --2- 172.21.15.76:0/2656241129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facd006bb60 0x7facd0149f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.295+0000 7facd8133640 1 -- 172.21.15.76:0/2656241129 >> 172.21.15.76:0/2656241129 conn(0x7facd00cb530 msgr2=0x7facd00cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:08.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.295+0000 7facd8133640 1 -- 172.21.15.76:0/2656241129 shutdown_connections 2023-11-09T03:55:08.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.295+0000 7facd8133640 1 -- 172.21.15.76:0/2656241129 wait complete. 2023-11-09T03:55:08.306 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:55:08.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.408+0000 7f9f3b71d640 1 Processor -- start 2023-11-09T03:55:08.408 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.408+0000 7f9f3b71d640 1 -- start start 2023-11-09T03:55:08.409 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.409+0000 7f9f3b71d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9f3415d100 0x7f9f340ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:08.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.409+0000 7f9f3b71d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9f3415d4e0 0x7f9f340d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:08.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.409+0000 7f9f3b71d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f340b32b0 con 0x7f9f340d3b60 2023-11-09T03:55:08.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.409+0000 7f9f3b71d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f340cee10 con 0x7f9f3415d4e0 2023-11-09T03:55:08.410 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.409+0000 7f9f3b71d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f340b4750 con 0x7f9f3415d100 2023-11-09T03:55:08.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.409+0000 7f9f38cb8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9f3415d4e0 0x7f9f340d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:08.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.409+0000 7f9f38cb8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9f3415d4e0 0x7f9f340d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47512/0 (socket says 172.21.15.76:47512) 2023-11-09T03:55:08.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.409+0000 7f9f38cb8640 1 -- 172.21.15.76:0/2032989173 learned_addr learned my addr 172.21.15.76:0/2032989173 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:08.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.409+0000 7f9f394b9640 1 --2- 172.21.15.76:0/2032989173 >> v2:172.21.15.113:3300/0 conn(0x7f9f3415d100 0x7f9f340ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:08.411 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.410+0000 7f9f38cb8640 1 -- 172.21.15.76:0/2032989173 >> v2:172.21.15.113:3300/0 conn(0x7f9f3415d100 msgr2=0x7f9f340ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:08.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.410+0000 7f9f38cb8640 1 --2- 172.21.15.76:0/2032989173 >> v2:172.21.15.113:3300/0 conn(0x7f9f3415d100 0x7f9f340ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.410+0000 7f9f38cb8640 1 -- 172.21.15.76:0/2032989173 >> v1:172.21.15.113:6789/0 conn(0x7f9f340d3b60 legacy=0x7f9f340cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:08.412 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.410+0000 7f9f38cb8640 1 -- 172.21.15.76:0/2032989173 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f340b2980 con 0x7f9f3415d4e0 2023-11-09T03:55:08.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.410+0000 7f9f38cb8640 1 --2- 172.21.15.76:0/2032989173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f3415d4e0 0x7f9f340d14d0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f9f340d09f0 tx=0x7f9f2402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7bc318469e599d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:08.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.411+0000 7f9f2bfff640 1 -- 172.21.15.76:0/2032989173 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f24030860 con 0x7f9f3415d4e0 2023-11-09T03:55:08.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.411+0000 7f9f2bfff640 1 -- 172.21.15.76:0/2032989173 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f240309c0 con 0x7f9f3415d4e0 2023-11-09T03:55:08.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.411+0000 7f9f2bfff640 1 -- 172.21.15.76:0/2032989173 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f24030d50 con 0x7f9f3415d4e0 2023-11-09T03:55:08.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.411+0000 7f9f3b71d640 1 -- 172.21.15.76:0/2032989173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f3415d4e0 msgr2=0x7f9f340d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:08.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.411+0000 7f9f3b71d640 1 --2- 172.21.15.76:0/2032989173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f3415d4e0 0x7f9f340d14d0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f9f340d09f0 tx=0x7f9f2402b9a0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.417 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.411+0000 7f9f3b71d640 1 -- 172.21.15.76:0/2032989173 shutdown_connections 2023-11-09T03:55:08.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.411+0000 7f9f3b71d640 1 --2- 172.21.15.76:0/2032989173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f3415d4e0 0x7f9f340d14d0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.411+0000 7f9f3b71d640 1 --2- 172.21.15.76:0/2032989173 >> v2:172.21.15.113:3300/0 conn(0x7f9f3415d100 0x7f9f340ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.411+0000 7f9f3b71d640 1 -- 172.21.15.76:0/2032989173 >> 172.21.15.76:0/2032989173 conn(0x7f9f340d2ef0 msgr2=0x7f9f340d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:08.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.411+0000 7f9f3b71d640 1 -- 172.21.15.76:0/2032989173 shutdown_connections 2023-11-09T03:55:08.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.411+0000 7f9f3b71d640 1 -- 172.21.15.76:0/2032989173 wait complete. 2023-11-09T03:55:08.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.412+0000 7f9f3b71d640 1 Processor -- start 2023-11-09T03:55:08.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.412+0000 7f9f3b71d640 1 -- start start 2023-11-09T03:55:08.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.413+0000 7f9f3b71d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f340d3b60 0x7f9f34160260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:08.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.413+0000 7f9f3b71d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f3415d100 0x7f9f34158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:08.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.413+0000 7f9f3b71d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f3415d4e0 0x7f9f340b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:08.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.413+0000 7f9f3b71d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f341607a0 con 0x7f9f3415d100 2023-11-09T03:55:08.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.413+0000 7f9f3b71d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3415ff40 con 0x7f9f3415d4e0 2023-11-09T03:55:08.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.413+0000 7f9f3b71d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f340b7ff0 con 0x7f9f340d3b60 2023-11-09T03:55:08.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.413+0000 7f9f38cb8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f3415d100 0x7f9f34158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:08.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.413+0000 7f9f38cb8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f3415d100 0x7f9f34158d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47526/0 (socket says 172.21.15.76:47526) 2023-11-09T03:55:08.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.413+0000 7f9f38cb8640 1 -- 172.21.15.76:0/1133068053 learned_addr learned my addr 172.21.15.76:0/1133068053 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:08.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.413+0000 7f9f39cba640 1 --2- 172.21.15.76:0/1133068053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f3415d4e0 0x7f9f340b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:08.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.413+0000 7f9f394b9640 1 --2- 172.21.15.76:0/1133068053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f340d3b60 0x7f9f34160260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:08.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.413+0000 7f9f38cb8640 1 -- 172.21.15.76:0/1133068053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f340d3b60 msgr2=0x7f9f34160260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:08.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.413+0000 7f9f38cb8640 1 --2- 172.21.15.76:0/1133068053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f340d3b60 0x7f9f34160260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.413+0000 7f9f38cb8640 1 -- 172.21.15.76:0/1133068053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f3415d4e0 msgr2=0x7f9f340b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:08.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.413+0000 7f9f38cb8640 1 --2- 172.21.15.76:0/1133068053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f3415d4e0 0x7f9f340b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.413+0000 7f9f38cb8640 1 -- 172.21.15.76:0/1133068053 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f240094d0 con 0x7f9f3415d100 2023-11-09T03:55:08.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.413+0000 7f9f39cba640 1 --2- 172.21.15.76:0/1133068053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f3415d4e0 0x7f9f340b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:55:08.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.414+0000 7f9f38cb8640 1 --2- 172.21.15.76:0/1133068053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f3415d100 0x7f9f34158d30 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f9f2402b9e0 tx=0x7f9f24004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:08.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.414+0000 7f9f29ffb640 1 -- 172.21.15.76:0/1133068053 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2403e040 con 0x7f9f3415d100 2023-11-09T03:55:08.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.414+0000 7f9f29ffb640 1 -- 172.21.15.76:0/1133068053 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f24039c30 con 0x7f9f3415d100 2023-11-09T03:55:08.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.414+0000 7f9f3b71d640 1 -- 172.21.15.76:0/1133068053 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f340b82d0 con 0x7f9f3415d100 2023-11-09T03:55:08.423 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.414+0000 7f9f29ffb640 1 -- 172.21.15.76:0/1133068053 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f240443f0 con 0x7f9f3415d100 2023-11-09T03:55:08.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.414+0000 7f9f3b71d640 1 -- 172.21.15.76:0/1133068053 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f340b8890 con 0x7f9f3415d100 2023-11-09T03:55:08.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.415+0000 7f9f3b71d640 1 -- 172.21.15.76:0/1133068053 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f340684b0 con 0x7f9f3415d100 2023-11-09T03:55:08.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.415+0000 7f9f29ffb640 1 -- 172.21.15.76:0/1133068053 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9f2404d3b0 con 0x7f9f3415d100 2023-11-09T03:55:08.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.416+0000 7f9f29ffb640 1 --2- 172.21.15.76:0/1133068053 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9f1806aea0 0x7f9f1806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:08.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.416+0000 7f9f29ffb640 1 -- 172.21.15.76:0/1133068053 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7f9f240ac8e0 con 0x7f9f3415d100 2023-11-09T03:55:08.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.416+0000 7f9f394b9640 1 --2- 172.21.15.76:0/1133068053 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9f1806aea0 0x7f9f1806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:08.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.416+0000 7f9f394b9640 1 --2- 172.21.15.76:0/1133068053 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9f1806aea0 0x7f9f1806d360 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9f30007440 tx=0x7f9f3000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:08.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.420+0000 7f9f29ffb640 1 -- 172.21.15.76:0/1133068053 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9f24076ea0 con 0x7f9f3415d100 2023-11-09T03:55:08.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.587+0000 7f9f3b71d640 1 -- 172.21.15.76:0/1133068053 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9f340d2cd0 con 0x7f9f3415d100 2023-11-09T03:55:08.587 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.587+0000 7f9f29ffb640 1 -- 172.21.15.76:0/1133068053 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9f2407a4c0 con 0x7f9f3415d100 2023-11-09T03:55:08.587 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:55:08.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.589+0000 7f9f3b71d640 1 -- 172.21.15.76:0/1133068053 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9f1806aea0 msgr2=0x7f9f1806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:08.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.589+0000 7f9f3b71d640 1 --2- 172.21.15.76:0/1133068053 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9f1806aea0 0x7f9f1806d360 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9f30007440 tx=0x7f9f3000b040 comp rx=0 tx=0).stop 2023-11-09T03:55:08.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.589+0000 7f9f3b71d640 1 -- 172.21.15.76:0/1133068053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f3415d100 msgr2=0x7f9f34158d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:08.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.589+0000 7f9f3b71d640 1 --2- 172.21.15.76:0/1133068053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f3415d100 0x7f9f34158d30 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f9f2402b9e0 tx=0x7f9f24004730 comp rx=0 tx=0).stop 2023-11-09T03:55:08.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.589+0000 7f9f3b71d640 1 -- 172.21.15.76:0/1133068053 shutdown_connections 2023-11-09T03:55:08.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.589+0000 7f9f3b71d640 1 --2- 172.21.15.76:0/1133068053 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9f1806aea0 0x7f9f1806d360 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.589+0000 7f9f3b71d640 1 --2- 172.21.15.76:0/1133068053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f3415d4e0 0x7f9f340b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.589+0000 7f9f3b71d640 1 --2- 172.21.15.76:0/1133068053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f3415d100 0x7f9f34158d30 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.589+0000 7f9f3b71d640 1 --2- 172.21.15.76:0/1133068053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9f340d3b60 0x7f9f34160260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.589+0000 7f9f3b71d640 1 -- 172.21.15.76:0/1133068053 >> 172.21.15.76:0/1133068053 conn(0x7f9f340d2ef0 msgr2=0x7f9f3406ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:08.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.589+0000 7f9f3b71d640 1 -- 172.21.15.76:0/1133068053 shutdown_connections 2023-11-09T03:55:08.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.589+0000 7f9f3b71d640 1 -- 172.21.15.76:0/1133068053 wait complete. 2023-11-09T03:55:08.599 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:55:08.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.702+0000 7fae7f3f5640 1 Processor -- start 2023-11-09T03:55:08.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.703+0000 7fae7f3f5640 1 -- start start 2023-11-09T03:55:08.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.703+0000 7fae7f3f5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fae7815d100 0x7fae780ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:08.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.703+0000 7fae7f3f5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae7815d4e0 0x7fae780d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:08.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.703+0000 7fae7f3f5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae780b32b0 con 0x7fae780d3b60 2023-11-09T03:55:08.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.703+0000 7fae7f3f5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae780cee10 con 0x7fae7815d100 2023-11-09T03:55:08.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.703+0000 7fae7f3f5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae780b4750 con 0x7fae7815d4e0 2023-11-09T03:55:08.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.703+0000 7fae7d191640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fae7815d100 0x7fae780ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:08.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.704+0000 7fae7d191640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fae7815d100 0x7fae780ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47540/0 (socket says 172.21.15.76:47540) 2023-11-09T03:55:08.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.704+0000 7fae7d191640 1 -- 172.21.15.76:0/3449244517 learned_addr learned my addr 172.21.15.76:0/3449244517 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:08.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.704+0000 7fae7c990640 1 --2- 172.21.15.76:0/3449244517 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae7815d4e0 0x7fae780d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:08.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.704+0000 7fae7d191640 1 -- 172.21.15.76:0/3449244517 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae7815d4e0 msgr2=0x7fae780d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:08.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.704+0000 7fae7d191640 1 --2- 172.21.15.76:0/3449244517 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae7815d4e0 0x7fae780d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.704+0000 7fae7d191640 1 -- 172.21.15.76:0/3449244517 >> v1:172.21.15.113:6789/0 conn(0x7fae780d3b60 legacy=0x7fae780cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:08.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.704+0000 7fae7d191640 1 -- 172.21.15.76:0/3449244517 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae780b2980 con 0x7fae7815d100 2023-11-09T03:55:08.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.705+0000 7fae7d191640 1 --2- 172.21.15.76:0/3449244517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae7815d100 0x7fae780ce510 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fae74009850 tx=0x7fae7402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d046e3e7fb20ff31 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:08.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.705+0000 7fae6ffff640 1 -- 172.21.15.76:0/3449244517 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae74004230 con 0x7fae7815d100 2023-11-09T03:55:08.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.705+0000 7fae6ffff640 1 -- 172.21.15.76:0/3449244517 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae74004390 con 0x7fae7815d100 2023-11-09T03:55:08.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.705+0000 7fae6ffff640 1 -- 172.21.15.76:0/3449244517 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae74003810 con 0x7fae7815d100 2023-11-09T03:55:08.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.705+0000 7fae7f3f5640 1 -- 172.21.15.76:0/3449244517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae7815d100 msgr2=0x7fae780ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:08.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.705+0000 7fae7f3f5640 1 --2- 172.21.15.76:0/3449244517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae7815d100 0x7fae780ce510 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fae74009850 tx=0x7fae7402b960 comp rx=0 tx=0).stop 2023-11-09T03:55:08.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.705+0000 7fae7f3f5640 1 -- 172.21.15.76:0/3449244517 shutdown_connections 2023-11-09T03:55:08.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.705+0000 7fae7f3f5640 1 --2- 172.21.15.76:0/3449244517 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae7815d4e0 0x7fae780d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.705+0000 7fae7f3f5640 1 --2- 172.21.15.76:0/3449244517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae7815d100 0x7fae780ce510 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.705+0000 7fae7f3f5640 1 -- 172.21.15.76:0/3449244517 >> 172.21.15.76:0/3449244517 conn(0x7fae780d2ef0 msgr2=0x7fae780d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:08.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.705+0000 7fae7f3f5640 1 -- 172.21.15.76:0/3449244517 shutdown_connections 2023-11-09T03:55:08.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.705+0000 7fae7f3f5640 1 -- 172.21.15.76:0/3449244517 wait complete. 2023-11-09T03:55:08.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.706+0000 7fae7f3f5640 1 Processor -- start 2023-11-09T03:55:08.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.707+0000 7fae7f3f5640 1 -- start start 2023-11-09T03:55:08.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.707+0000 7fae7f3f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae780d3b60 0x7fae781521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:08.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.707+0000 7fae7f3f5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae7815d100 0x7fae780d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:08.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.707+0000 7fae7f3f5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae7815d4e0 0x7fae780d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:08.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.707+0000 7fae7f3f5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae780c9fc0 con 0x7fae7815d100 2023-11-09T03:55:08.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.707+0000 7fae7f3f5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae780ca210 con 0x7fae780d3b60 2023-11-09T03:55:08.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.707+0000 7fae7f3f5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae780ca4f0 con 0x7fae7815d4e0 2023-11-09T03:55:08.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.707+0000 7fae7c990640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae7815d100 0x7fae780d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:08.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.707+0000 7fae7c990640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae7815d100 0x7fae780d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47548/0 (socket says 172.21.15.76:47548) 2023-11-09T03:55:08.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.707+0000 7fae7c990640 1 -- 172.21.15.76:0/2808907557 learned_addr learned my addr 172.21.15.76:0/2808907557 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:08.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.707+0000 7fae7d191640 1 --2- 172.21.15.76:0/2808907557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae780d3b60 0x7fae781521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:08.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.707+0000 7fae7d992640 1 --2- 172.21.15.76:0/2808907557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae7815d4e0 0x7fae780d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:08.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.708+0000 7fae7c990640 1 -- 172.21.15.76:0/2808907557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae7815d4e0 msgr2=0x7fae780d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:08.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.708+0000 7fae7c990640 1 --2- 172.21.15.76:0/2808907557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae7815d4e0 0x7fae780d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.708+0000 7fae7c990640 1 -- 172.21.15.76:0/2808907557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae780d3b60 msgr2=0x7fae781521c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:08.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.708+0000 7fae7c990640 1 --2- 172.21.15.76:0/2808907557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae780d3b60 0x7fae781521c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.708+0000 7fae7c990640 1 -- 172.21.15.76:0/2808907557 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae74009460 con 0x7fae7815d100 2023-11-09T03:55:08.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.708+0000 7fae7c990640 1 --2- 172.21.15.76:0/2808907557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae7815d100 0x7fae780d14d0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fae6800d800 tx=0x7fae6800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:08.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.708+0000 7fae6dffb640 1 -- 172.21.15.76:0/2808907557 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6800b6c0 con 0x7fae7815d100 2023-11-09T03:55:08.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.708+0000 7fae6dffb640 1 -- 172.21.15.76:0/2808907557 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae6800b820 con 0x7fae7815d100 2023-11-09T03:55:08.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.708+0000 7fae7f3f5640 1 -- 172.21.15.76:0/2808907557 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae780ca7d0 con 0x7fae7815d100 2023-11-09T03:55:08.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.708+0000 7fae6dffb640 1 -- 172.21.15.76:0/2808907557 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6800bbb0 con 0x7fae7815d100 2023-11-09T03:55:08.720 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.708+0000 7fae7f3f5640 1 -- 172.21.15.76:0/2808907557 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae78152700 con 0x7fae7815d100 2023-11-09T03:55:08.720 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.709+0000 7fae7f3f5640 1 -- 172.21.15.76:0/2808907557 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae780684b0 con 0x7fae7815d100 2023-11-09T03:55:08.720 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.709+0000 7fae6dffb640 1 -- 172.21.15.76:0/2808907557 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fae6800bd10 con 0x7fae7815d100 2023-11-09T03:55:08.720 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.710+0000 7fae6dffb640 1 --2- 172.21.15.76:0/2808907557 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae5c06ae70 0x7fae5c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:08.721 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.710+0000 7fae6dffb640 1 -- 172.21.15.76:0/2808907557 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7fae6808a530 con 0x7fae7815d100 2023-11-09T03:55:08.721 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.710+0000 7fae7d191640 1 --2- 172.21.15.76:0/2808907557 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae5c06ae70 0x7fae5c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:08.721 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.710+0000 7fae7d191640 1 --2- 172.21.15.76:0/2808907557 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae5c06ae70 0x7fae5c06d330 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fae74035000 tx=0x7fae7403a040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:08.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.714+0000 7fae6dffb640 1 -- 172.21.15.76:0/2808907557 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fae68054b80 con 0x7fae7815d100 2023-11-09T03:55:08.879 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.880+0000 7fae7f3f5640 1 -- 172.21.15.76:0/2808907557 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fae780d2cd0 con 0x7fae7815d100 2023-11-09T03:55:08.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.880+0000 7fae6dffb640 1 -- 172.21.15.76:0/2808907557 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) v1 ==== 100+0+11 (secure 0 0 0) 0x7fae680581a0 con 0x7fae7815d100 2023-11-09T03:55:08.880 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:55:08.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.882+0000 7fae7f3f5640 1 -- 172.21.15.76:0/2808907557 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae5c06ae70 msgr2=0x7fae5c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:08.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.882+0000 7fae7f3f5640 1 --2- 172.21.15.76:0/2808907557 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae5c06ae70 0x7fae5c06d330 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fae74035000 tx=0x7fae7403a040 comp rx=0 tx=0).stop 2023-11-09T03:55:08.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.882+0000 7fae7f3f5640 1 -- 172.21.15.76:0/2808907557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae7815d100 msgr2=0x7fae780d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:08.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.882+0000 7fae7f3f5640 1 --2- 172.21.15.76:0/2808907557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae7815d100 0x7fae780d14d0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fae6800d800 tx=0x7fae6800dcc0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.882+0000 7fae7f3f5640 1 -- 172.21.15.76:0/2808907557 shutdown_connections 2023-11-09T03:55:08.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.882+0000 7fae7f3f5640 1 --2- 172.21.15.76:0/2808907557 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae5c06ae70 0x7fae5c06d330 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.882+0000 7fae7f3f5640 1 --2- 172.21.15.76:0/2808907557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae7815d4e0 0x7fae780d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.882+0000 7fae7f3f5640 1 --2- 172.21.15.76:0/2808907557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae7815d100 0x7fae780d14d0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.882+0000 7fae7f3f5640 1 --2- 172.21.15.76:0/2808907557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae780d3b60 0x7fae781521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:08.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.882+0000 7fae7f3f5640 1 -- 172.21.15.76:0/2808907557 >> 172.21.15.76:0/2808907557 conn(0x7fae780d2ef0 msgr2=0x7fae7806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:08.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.882+0000 7fae7f3f5640 1 -- 172.21.15.76:0/2808907557 shutdown_connections 2023-11-09T03:55:08.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.882+0000 7fae7f3f5640 1 -- 172.21.15.76:0/2808907557 wait complete. 2023-11-09T03:55:08.892 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-09T03:55:08.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.995+0000 7fa42d24b640 1 Processor -- start 2023-11-09T03:55:08.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.995+0000 7fa42d24b640 1 -- start start 2023-11-09T03:55:08.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.996+0000 7fa42d24b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa428159e40 0x7fa4280b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:08.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.996+0000 7fa42d24b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa42806bb60 0x7fa4280b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:08.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.996+0000 7fa42d24b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa428071500 con 0x7fa42806fff0 2023-11-09T03:55:08.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.996+0000 7fa42d24b640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa42815dae0 con 0x7fa42806bb60 2023-11-09T03:55:08.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.996+0000 7fa42d24b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa428070c60 con 0x7fa428159e40 2023-11-09T03:55:08.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.996+0000 7fa42659c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa42806bb60 0x7fa4280b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:08.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.996+0000 7fa42659c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa42806bb60 0x7fa4280b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47554/0 (socket says 172.21.15.76:47554) 2023-11-09T03:55:08.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.996+0000 7fa42659c640 1 -- 172.21.15.76:0/3774309607 learned_addr learned my addr 172.21.15.76:0/3774309607 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:09.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.996+0000 7fa426d9d640 1 --2- 172.21.15.76:0/3774309607 >> v2:172.21.15.113:3300/0 conn(0x7fa428159e40 0x7fa4280b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:09.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.996+0000 7fa42659c640 1 -- 172.21.15.76:0/3774309607 >> v2:172.21.15.113:3300/0 conn(0x7fa428159e40 msgr2=0x7fa4280b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:09.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.996+0000 7fa42659c640 1 --2- 172.21.15.76:0/3774309607 >> v2:172.21.15.113:3300/0 conn(0x7fa428159e40 0x7fa4280b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:09.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.996+0000 7fa42659c640 1 -- 172.21.15.76:0/3774309607 >> v1:172.21.15.113:6789/0 conn(0x7fa42806fff0 legacy=0x7fa428070420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:09.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.996+0000 7fa42659c640 1 -- 172.21.15.76:0/3774309607 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa428070f40 con 0x7fa42806bb60 2023-11-09T03:55:09.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.997+0000 7fa42659c640 1 --2- 172.21.15.76:0/3774309607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa42806bb60 0x7fa4280b0a70 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fa4280ce7f0 tx=0x7fa41c02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3ea3786dc6a40 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:09.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.997+0000 7fa425d9b640 1 -- 172.21.15.76:0/3774309607 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c031750 con 0x7fa42806bb60 2023-11-09T03:55:09.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.997+0000 7fa425d9b640 1 -- 172.21.15.76:0/3774309607 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41c0318b0 con 0x7fa42806bb60 2023-11-09T03:55:09.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.997+0000 7fa425d9b640 1 -- 172.21.15.76:0/3774309607 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c031c40 con 0x7fa42806bb60 2023-11-09T03:55:09.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.998+0000 7fa42d24b640 1 -- 172.21.15.76:0/3774309607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa42806bb60 msgr2=0x7fa4280b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:09.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.998+0000 7fa42d24b640 1 --2- 172.21.15.76:0/3774309607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa42806bb60 0x7fa4280b0a70 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fa4280ce7f0 tx=0x7fa41c02b7f0 comp rx=0 tx=0).stop 2023-11-09T03:55:09.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.998+0000 7fa42d24b640 1 -- 172.21.15.76:0/3774309607 shutdown_connections 2023-11-09T03:55:09.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.998+0000 7fa42d24b640 1 --2- 172.21.15.76:0/3774309607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa42806bb60 0x7fa4280b0a70 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:09.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.998+0000 7fa42d24b640 1 --2- 172.21.15.76:0/3774309607 >> v2:172.21.15.113:3300/0 conn(0x7fa428159e40 0x7fa4280b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:09.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.998+0000 7fa42d24b640 1 -- 172.21.15.76:0/3774309607 >> 172.21.15.76:0/3774309607 conn(0x7fa4280cb530 msgr2=0x7fa4280cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:09.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.998+0000 7fa42d24b640 1 -- 172.21.15.76:0/3774309607 shutdown_connections 2023-11-09T03:55:09.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.998+0000 7fa42d24b640 1 -- 172.21.15.76:0/3774309607 wait complete. 2023-11-09T03:55:09.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.999+0000 7fa42d24b640 1 Processor -- start 2023-11-09T03:55:09.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:08.999+0000 7fa42d24b640 1 -- start start 2023-11-09T03:55:09.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.000+0000 7fa42d24b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa42806bb60 0x7fa428158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:09.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.000+0000 7fa42d24b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa42806fff0 0x7fa4280b0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:09.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.000+0000 7fa42d24b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa428159e40 0x7fa4280b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:09.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.000+0000 7fa42d24b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4280c8bf0 con 0x7fa428159e40 2023-11-09T03:55:09.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.000+0000 7fa42d24b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42815d200 con 0x7fa42806bb60 2023-11-09T03:55:09.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.000+0000 7fa42d24b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4280c6730 con 0x7fa42806fff0 2023-11-09T03:55:09.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.000+0000 7fa42759e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa428159e40 0x7fa4280b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:09.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.000+0000 7fa42759e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa428159e40 0x7fa4280b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47558/0 (socket says 172.21.15.76:47558) 2023-11-09T03:55:09.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.000+0000 7fa42759e640 1 -- 172.21.15.76:0/702150721 learned_addr learned my addr 172.21.15.76:0/702150721 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:09.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.000+0000 7fa426d9d640 1 --2- 172.21.15.76:0/702150721 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa42806bb60 0x7fa428158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:09.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.000+0000 7fa42659c640 1 --2- 172.21.15.76:0/702150721 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa42806fff0 0x7fa4280b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:09.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.000+0000 7fa42759e640 1 -- 172.21.15.76:0/702150721 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa42806fff0 msgr2=0x7fa4280b0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:09.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.000+0000 7fa42759e640 1 --2- 172.21.15.76:0/702150721 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa42806fff0 0x7fa4280b0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:09.010 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.000+0000 7fa42759e640 1 -- 172.21.15.76:0/702150721 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa42806bb60 msgr2=0x7fa428158d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:09.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.000+0000 7fa42759e640 1 --2- 172.21.15.76:0/702150721 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa42806bb60 0x7fa428158d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:09.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.000+0000 7fa42759e640 1 -- 172.21.15.76:0/702150721 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41c0094d0 con 0x7fa428159e40 2023-11-09T03:55:09.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.000+0000 7fa426d9d640 1 --2- 172.21.15.76:0/702150721 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa42806bb60 0x7fa428158d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:55:09.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.000+0000 7fa42659c640 1 --2- 172.21.15.76:0/702150721 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa42806fff0 0x7fa4280b0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:55:09.011 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.001+0000 7fa42759e640 1 --2- 172.21.15.76:0/702150721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa428159e40 0x7fa4280b0bc0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fa42000c7b0 tx=0x7fa42000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:09.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.001+0000 7fa40f7fe640 1 -- 172.21.15.76:0/702150721 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa420004b60 con 0x7fa428159e40 2023-11-09T03:55:09.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.001+0000 7fa40f7fe640 1 -- 172.21.15.76:0/702150721 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa420004cc0 con 0x7fa428159e40 2023-11-09T03:55:09.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.001+0000 7fa42d24b640 1 -- 172.21.15.76:0/702150721 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4280c6a10 con 0x7fa428159e40 2023-11-09T03:55:09.012 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.001+0000 7fa40f7fe640 1 -- 172.21.15.76:0/702150721 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42000aa90 con 0x7fa428159e40 2023-11-09T03:55:09.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.001+0000 7fa42d24b640 1 -- 172.21.15.76:0/702150721 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4280c6fd0 con 0x7fa428159e40 2023-11-09T03:55:09.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.002+0000 7fa42d24b640 1 -- 172.21.15.76:0/702150721 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4281ee9b0 con 0x7fa428159e40 2023-11-09T03:55:09.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.002+0000 7fa40f7fe640 1 -- 172.21.15.76:0/702150721 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa42000abf0 con 0x7fa428159e40 2023-11-09T03:55:09.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.003+0000 7fa40f7fe640 1 --2- 172.21.15.76:0/702150721 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa41406ae70 0x7fa41406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:09.013 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.003+0000 7fa40f7fe640 1 -- 172.21.15.76:0/702150721 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7fa42008a240 con 0x7fa428159e40 2023-11-09T03:55:09.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.003+0000 7fa426d9d640 1 --2- 172.21.15.76:0/702150721 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa41406ae70 0x7fa41406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:09.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.003+0000 7fa426d9d640 1 --2- 172.21.15.76:0/702150721 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa41406ae70 0x7fa41406d330 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fa418006fd0 tx=0x7fa418008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:09.014 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.007+0000 7fa40f7fe640 1 -- 172.21.15.76:0/702150721 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa420054800 con 0x7fa428159e40 2023-11-09T03:55:09.185 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.186+0000 7fa42d24b640 1 -- 172.21.15.76:0/702150721 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa4280684b0 con 0x7fa428159e40 2023-11-09T03:55:09.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.186+0000 7fa40f7fe640 1 -- 172.21.15.76:0/702150721 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 29 entries v118) v1 ==== 81+0+1747 (secure 0 0 0) 0x7fa420057e20 con 0x7fa428159e40 2023-11-09T03:55:09.187 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/3870807407 2023-11-09T04:54:56.671385+0000 2023-11-09T03:55:09.187 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/3462040782 2023-11-10T03:54:45.345437+0000 2023-11-09T03:55:09.187 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/2075292158 2023-11-10T03:54:06.568191+0000 2023-11-09T03:55:09.188 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/334503200 2023-11-09T04:53:32.572840+0000 2023-11-09T03:55:09.188 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/2075292158 2023-11-10T03:54:06.568191+0000 2023-11-09T03:55:09.188 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/7360844 2023-11-10T03:53:24.718008+0000 2023-11-09T03:55:09.188 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/7360844 2023-11-10T03:53:24.718008+0000 2023-11-09T03:55:09.189 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/546338722 2023-11-10T03:53:17.470106+0000 2023-11-09T03:55:09.189 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/546338722 2023-11-10T03:53:17.470106+0000 2023-11-09T03:55:09.189 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/3087804343 2023-11-09T04:52:42.513556+0000 2023-11-09T03:55:09.190 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6833/1536527256 2023-11-10T03:52:34.213804+0000 2023-11-09T03:55:09.190 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/445306598 2023-11-10T03:52:27.406838+0000 2023-11-09T03:55:09.190 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/3229239133 2023-11-10T03:51:32.624856+0000 2023-11-09T03:55:09.190 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/503424885 2023-11-09T04:51:47.727422+0000 2023-11-09T03:55:09.191 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/3462040782 2023-11-10T03:54:45.345437+0000 2023-11-09T03:55:09.191 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/4011275120 2023-11-10T03:49:42.029057+0000 2023-11-09T03:55:09.191 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/4011275120 2023-11-10T03:49:42.029057+0000 2023-11-09T03:55:09.192 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/2519773110 2023-11-09T04:54:14.823765+0000 2023-11-09T03:55:09.192 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/3280098776 2023-11-10T03:50:07.616780+0000 2023-11-09T03:55:09.192 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/3280098776 2023-11-10T03:50:07.616780+0000 2023-11-09T03:55:09.192 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/1663988060 2023-11-10T03:50:33.665092+0000 2023-11-09T03:55:09.193 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/1663988060 2023-11-10T03:50:33.665092+0000 2023-11-09T03:55:09.193 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/445306598 2023-11-10T03:52:27.406838+0000 2023-11-09T03:55:09.193 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6833/3697850101 2023-11-10T03:51:39.442929+0000 2023-11-09T03:55:09.193 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/3449910842 2023-11-10T03:50:59.611256+0000 2023-11-09T03:55:09.194 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/3229239133 2023-11-10T03:51:32.624856+0000 2023-11-09T03:55:09.194 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6832/1536527256 2023-11-10T03:52:34.213804+0000 2023-11-09T03:55:09.194 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/3449910842 2023-11-10T03:50:59.611256+0000 2023-11-09T03:55:09.195 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6832/3697850101 2023-11-10T03:51:39.442929+0000 2023-11-09T03:55:09.196 INFO:teuthology.orchestra.run.smithi076.stderr:listed 29 entries 2023-11-09T03:55:09.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.188+0000 7fa42d24b640 1 -- 172.21.15.76:0/702150721 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa41406ae70 msgr2=0x7fa41406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:09.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.188+0000 7fa42d24b640 1 --2- 172.21.15.76:0/702150721 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa41406ae70 0x7fa41406d330 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fa418006fd0 tx=0x7fa418008040 comp rx=0 tx=0).stop 2023-11-09T03:55:09.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.188+0000 7fa42d24b640 1 -- 172.21.15.76:0/702150721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa428159e40 msgr2=0x7fa4280b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:09.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.188+0000 7fa42d24b640 1 --2- 172.21.15.76:0/702150721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa428159e40 0x7fa4280b0bc0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fa42000c7b0 tx=0x7fa42000cc70 comp rx=0 tx=0).stop 2023-11-09T03:55:09.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.188+0000 7fa42d24b640 1 -- 172.21.15.76:0/702150721 shutdown_connections 2023-11-09T03:55:09.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.188+0000 7fa42d24b640 1 --2- 172.21.15.76:0/702150721 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa41406ae70 0x7fa41406d330 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:09.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.188+0000 7fa42d24b640 1 --2- 172.21.15.76:0/702150721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa428159e40 0x7fa4280b0bc0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:09.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.188+0000 7fa42d24b640 1 --2- 172.21.15.76:0/702150721 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa42806fff0 0x7fa4280b0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:09.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.188+0000 7fa42d24b640 1 --2- 172.21.15.76:0/702150721 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa42806bb60 0x7fa428158d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:09.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.188+0000 7fa42d24b640 1 -- 172.21.15.76:0/702150721 >> 172.21.15.76:0/702150721 conn(0x7fa4280cb530 msgr2=0x7fa4280cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:09.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.188+0000 7fa42d24b640 1 -- 172.21.15.76:0/702150721 shutdown_connections 2023-11-09T03:55:09.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.188+0000 7fa42d24b640 1 -- 172.21.15.76:0/702150721 wait complete. 2023-11-09T03:55:09.200 DEBUG:tasks.cephfs.mount:Force/lazy unmounting on client.0 2023-11-09T03:55:09.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:55:09.201 DEBUG:teuthology.orchestra.run.smithi076:> sudo umount --lazy --force /home/ubuntu/cephtest/mnt.0 2023-11-09T03:55:09.228 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[129903]: fuse finished with error 0 and tester_r 0 2023-11-09T03:55:09.229 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T03:55:09.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:55:09.230 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-09T03:55:09.285 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:55:09.285 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:55:09.341 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056381,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14761 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248868k,nr_inodes=812217,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-09T03:55:09.342 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-09T03:55:09.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.481+0000 7f28002d7640 1 Processor -- start 2023-11-09T03:55:09.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.481+0000 7f28002d7640 1 -- start start 2023-11-09T03:55:09.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.482+0000 7f28002d7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f27f80cc680 0x7f27f80cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:09.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.482+0000 7f28002d7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27f806c220 0x7f27f80ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:09.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.482+0000 7f28002d7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27f80cd720 con 0x7f27f80addd0 2023-11-09T03:55:09.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.482+0000 7f28002d7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27f80ae1b0 con 0x7f27f80cc680 2023-11-09T03:55:09.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.482+0000 7f28002d7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f80ccb10 con 0x7f27f806c220 2023-11-09T03:55:09.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.482+0000 7f27fd872640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f27f80cc680 0x7f27f80cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:09.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.483+0000 7f27fe073640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f27f80addd0 0x7f27f806bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45760/0 (socket says 172.21.15.76:45760) 2023-11-09T03:55:09.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.483+0000 7f27fe073640 1 -- 172.21.15.76:0/2512447827 learned_addr learned my addr 172.21.15.76:0/2512447827 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:55:09.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.483+0000 7f27fe874640 1 --2- 172.21.15.76:0/2512447827 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27f806c220 0x7f27f80ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:09.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.483+0000 7f27fd872640 1 -- 172.21.15.76:0/2512447827 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27f806c220 msgr2=0x7f27f80ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:09.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.483+0000 7f27fd872640 1 --2- 172.21.15.76:0/2512447827 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27f806c220 0x7f27f80ca2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:09.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.483+0000 7f27fd872640 1 -- 172.21.15.76:0/2512447827 >> v1:172.21.15.76:6789/0 conn(0x7f27f80addd0 legacy=0x7f27f806bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:09.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.483+0000 7f27fd071640 1 -- 172.21.15.76:0/2512447827 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 544602194 0 0) 0x7f27f80cd720 con 0x7f27f80addd0 2023-11-09T03:55:09.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.483+0000 7f27fd872640 1 -- 172.21.15.76:0/2512447827 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f80ccdf0 con 0x7f27f80cc680 2023-11-09T03:55:09.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.484+0000 7f27fd872640 1 --2- 172.21.15.76:0/2512447827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27f80cc680 0x7f27f80cdbb0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f27e800ba40 tx=0x7f27e802de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0cb7f897d19a0e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:09.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.484+0000 7f27fd071640 1 -- 172.21.15.76:0/2512447827 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e802ec10 con 0x7f27f80cc680 2023-11-09T03:55:09.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.484+0000 7f27fd071640 1 -- 172.21.15.76:0/2512447827 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e8032070 con 0x7f27f80cc680 2023-11-09T03:55:09.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.484+0000 7f27fd071640 1 -- 172.21.15.76:0/2512447827 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e8038400 con 0x7f27f80cc680 2023-11-09T03:55:09.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.484+0000 7f28002d7640 1 -- 172.21.15.76:0/2512447827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27f80cc680 msgr2=0x7f27f80cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:09.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.484+0000 7f28002d7640 1 --2- 172.21.15.76:0/2512447827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27f80cc680 0x7f27f80cdbb0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f27e800ba40 tx=0x7f27e802de60 comp rx=0 tx=0).stop 2023-11-09T03:55:09.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.484+0000 7f28002d7640 1 -- 172.21.15.76:0/2512447827 shutdown_connections 2023-11-09T03:55:09.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.484+0000 7f28002d7640 1 --2- 172.21.15.76:0/2512447827 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27f806c220 0x7f27f80ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:09.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.484+0000 7f28002d7640 1 --2- 172.21.15.76:0/2512447827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27f80cc680 0x7f27f80cdbb0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:09.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.484+0000 7f28002d7640 1 -- 172.21.15.76:0/2512447827 >> 172.21.15.76:0/2512447827 conn(0x7f27f80cb530 msgr2=0x7f27f80cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:09.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.484+0000 7f28002d7640 1 -- 172.21.15.76:0/2512447827 shutdown_connections 2023-11-09T03:55:09.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.484+0000 7f28002d7640 1 -- 172.21.15.76:0/2512447827 wait complete. 2023-11-09T03:55:09.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.485+0000 7f28002d7640 1 Processor -- start 2023-11-09T03:55:09.495 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.485+0000 7f28002d7640 1 -- start start 2023-11-09T03:55:09.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.486+0000 7f28002d7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27f806c220 0x7f27f8153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:09.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.486+0000 7f28002d7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27f80addd0 0x7f27f80aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:09.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.486+0000 7f28002d7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27f80af270 0x7f27f80c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:09.496 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.486+0000 7f28002d7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f80af720 con 0x7f27f80af270 2023-11-09T03:55:09.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.486+0000 7f28002d7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f80c76f0 con 0x7f27f80addd0 2023-11-09T03:55:09.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.486+0000 7f28002d7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f80c65e0 con 0x7f27f806c220 2023-11-09T03:55:09.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.486+0000 7f27fe874640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27f80af270 0x7f27f80c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:09.497 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.486+0000 7f27fe874640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27f80af270 0x7f27f80c6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47588/0 (socket says 172.21.15.76:47588) 2023-11-09T03:55:09.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.486+0000 7f27fe874640 1 -- 172.21.15.76:0/1040383396 learned_addr learned my addr 172.21.15.76:0/1040383396 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:09.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.487+0000 7f27fe073640 1 --2- 172.21.15.76:0/1040383396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27f806c220 0x7f27f8153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:09.498 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.487+0000 7f27fe874640 1 -- 172.21.15.76:0/1040383396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27f806c220 msgr2=0x7f27f8153920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:09.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.487+0000 7f27fe874640 1 --2- 172.21.15.76:0/1040383396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27f806c220 0x7f27f8153920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:09.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.487+0000 7f27fe874640 1 -- 172.21.15.76:0/1040383396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27f80addd0 msgr2=0x7f27f80aed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:09.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.487+0000 7f27fe874640 1 --2- 172.21.15.76:0/1040383396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27f80addd0 0x7f27f80aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:09.499 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.487+0000 7f27fe874640 1 -- 172.21.15.76:0/1040383396 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e800b6a0 con 0x7f27f80af270 2023-11-09T03:55:09.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.487+0000 7f27fe874640 1 --2- 172.21.15.76:0/1040383396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27f80af270 0x7f27f80c6a70 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f27f400cd40 tx=0x7f27f400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:09.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.487+0000 7f27eeffd640 1 -- 172.21.15.76:0/1040383396 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f4003b90 con 0x7f27f80af270 2023-11-09T03:55:09.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.487+0000 7f27eeffd640 1 -- 172.21.15.76:0/1040383396 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f4003cf0 con 0x7f27f80af270 2023-11-09T03:55:09.500 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.487+0000 7f28002d7640 1 -- 172.21.15.76:0/1040383396 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f80c5cb0 con 0x7f27f80af270 2023-11-09T03:55:09.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.487+0000 7f27eeffd640 1 -- 172.21.15.76:0/1040383396 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f4004240 con 0x7f27f80af270 2023-11-09T03:55:09.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.487+0000 7f28002d7640 1 -- 172.21.15.76:0/1040383396 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f80c6270 con 0x7f27f80af270 2023-11-09T03:55:09.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.488+0000 7f28002d7640 1 -- 172.21.15.76:0/1040383396 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27f80c6750 con 0x7f27f80af270 2023-11-09T03:55:09.501 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.488+0000 7f27eeffd640 1 -- 172.21.15.76:0/1040383396 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f27f40043a0 con 0x7f27f80af270 2023-11-09T03:55:09.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.489+0000 7f27eeffd640 1 --2- 172.21.15.76:0/1040383396 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f27dc06ae70 0x7f27dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:09.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.489+0000 7f27eeffd640 1 -- 172.21.15.76:0/1040383396 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7f27f4012070 con 0x7f27f80af270 2023-11-09T03:55:09.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.489+0000 7f27fe073640 1 --2- 172.21.15.76:0/1040383396 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f27dc06ae70 0x7f27dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:09.503 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.490+0000 7f27fe073640 1 --2- 172.21.15.76:0/1040383396 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f27dc06ae70 0x7f27dc06d330 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f27f0000f60 tx=0x7f27f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:09.504 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.493+0000 7f27eeffd640 1 -- 172.21.15.76:0/1040383396 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f27f4055010 con 0x7f27f80af270 2023-11-09T03:55:09.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.700+0000 7f28002d7640 1 -- 172.21.15.76:0/1040383396 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f27f80684b0 con 0x7f27f80af270 2023-11-09T03:55:09.700 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.700+0000 7f27eeffd640 1 -- 172.21.15.76:0/1040383396 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v133) v1 ==== 53+0+73 (secure 0 0 0) 0x7f27f4058630 con 0x7f27f80af270 2023-11-09T03:55:09.700 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-09T03:55:09.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.702+0000 7f28002d7640 1 -- 172.21.15.76:0/1040383396 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f27dc06ae70 msgr2=0x7f27dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:09.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.702+0000 7f28002d7640 1 --2- 172.21.15.76:0/1040383396 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f27dc06ae70 0x7f27dc06d330 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f27f0000f60 tx=0x7f27f0009040 comp rx=0 tx=0).stop 2023-11-09T03:55:09.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.702+0000 7f28002d7640 1 -- 172.21.15.76:0/1040383396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27f80af270 msgr2=0x7f27f80c6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:09.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.702+0000 7f28002d7640 1 --2- 172.21.15.76:0/1040383396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27f80af270 0x7f27f80c6a70 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f27f400cd40 tx=0x7f27f400a660 comp rx=0 tx=0).stop 2023-11-09T03:55:09.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.702+0000 7f28002d7640 1 -- 172.21.15.76:0/1040383396 shutdown_connections 2023-11-09T03:55:09.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.702+0000 7f28002d7640 1 --2- 172.21.15.76:0/1040383396 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f27dc06ae70 0x7f27dc06d330 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:09.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.702+0000 7f28002d7640 1 --2- 172.21.15.76:0/1040383396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f27f80af270 0x7f27f80c6a70 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:09.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.702+0000 7f28002d7640 1 --2- 172.21.15.76:0/1040383396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27f80addd0 0x7f27f80aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:09.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.702+0000 7f28002d7640 1 --2- 172.21.15.76:0/1040383396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f27f806c220 0x7f27f8153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:09.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.702+0000 7f28002d7640 1 -- 172.21.15.76:0/1040383396 >> 172.21.15.76:0/1040383396 conn(0x7f27f80cb530 msgr2=0x7f27f80ae380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:09.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.702+0000 7f28002d7640 1 -- 172.21.15.76:0/1040383396 shutdown_connections 2023-11-09T03:55:09.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:09.702+0000 7f28002d7640 1 -- 172.21.15.76:0/1040383396 wait complete. 2023-11-09T03:55:09.712 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-09T03:55:09.713 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-09T03:55:09.713 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2023-11-09T03:55:09.714 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-09T03:55:09.714 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-09T03:55:09.715 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-09T03:55:09.715 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-09T03:55:09.715 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-09T03:55:09.716 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:55:09.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:55:09.717 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2023-11-09T03:55:09.733 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-09T03:55:09.733 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-09T03:55:09.734 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2023-11-09T03:55:09.734 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:55:09.734 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2023-11-09T03:55:09.734 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:55:09.735 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:55:09.735 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:55:09.735 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2023-11-09T03:55:09.736 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:55:09.736 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:55:09.736 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2023-11-09T03:55:09.736 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:55:09.737 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:55:09.737 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-09T03:55:09.737 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:55:09.737 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-09T03:55:09.738 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42039sec preferred_lft 42039sec 2023-11-09T03:55:09.738 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2023-11-09T03:55:09.738 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:55:09.739 INFO:teuthology.orchestra.run.smithi076.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-09T03:55:09.739 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2e:15:bd:03:a8:2e brd ff:ff:ff:ff:ff:ff 2023-11-09T03:55:09.739 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-09T03:55:09.739 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:55:09.740 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::bcda:38ff:fed4:fdb0/64 scope link 2023-11-09T03:55:09.740 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:55:09.740 INFO:teuthology.orchestra.run.smithi076.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-09T03:55:09.740 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 6a:8d:94:d7:e5:00 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:55:09.741 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::688d:94ff:fed7:e500/64 scope link 2023-11-09T03:55:09.741 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:55:09.741 INFO:teuthology.orchestra.run.smithi076.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-09T03:55:09.741 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2e:15:bd:03:a8:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:55:09.742 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::2c15:bdff:fe03:a82e/64 scope link 2023-11-09T03:55:09.742 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:55:09.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:55:09.743 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-09T03:55:09.777 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:55:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:55:09.800 INFO:teuthology.orchestra.run.smithi076.stdout:1 2023-11-09T03:55:09.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:55:09.802 DEBUG:teuthology.orchestra.run.smithi076:> route 2023-11-09T03:55:09.857 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2023-11-09T03:55:09.857 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-09T03:55:09.857 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-09T03:55:09.858 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-09T03:55:09.858 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-09T03:55:09.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:55:09.859 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-09T03:55:09.859 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-09T03:55:09.860 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-09T03:55:09.860 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-09T03:55:09.860 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-09T03:55:09.861 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-09T03:55:09.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:55:09.953 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2023-11-09T03:55:09.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-09T03:55:09.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-09T03:55:09.969 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:55:09.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:55:09.970 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2023-11-09T03:55:10.037 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-09T03:55:10.038 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:55:10.038 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-09T03:55:10.092 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:55:10.093 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:55:10.093 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-09T03:55:10.147 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:55:10.202 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:55:10.203 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:55:10.203 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:55:10.203 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:55:10.203 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:55:10.204 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:55:10.204 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:55:10.204 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:55:10.204 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:55:10.205 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:55:10.205 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:55:10.205 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:55:10.205 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:55:10.205 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:55:10.206 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:55:10.206 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:55:10.206 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:55:10.206 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:55:10.206 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:55:10.207 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:55:10.207 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:55:10.207 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:55:10.207 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:55:10.207 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:55:10.208 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:55:10.208 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:55:10.208 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:55:10.208 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:55:10.208 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:55:10.209 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:55:10.209 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:55:10.209 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:55:10.209 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:55:10.209 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:55:10.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:55:10.210 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:55:10.257 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-09T03:55:10.257 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-09T03:55:10.299 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:55:10.315 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:55:10.315 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:55:10.315 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:55:10.315 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:55:10.316 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:55:10.316 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:55:10.316 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:55:10.316 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:55:10.316 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:55:10.317 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:55:10.317 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:55:10.317 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:55:10.317 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:55:10.317 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:55:10.318 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:55:10.318 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:55:10.318 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:55:10.318 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:55:10.318 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:55:10.319 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:55:10.319 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:55:10.319 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:55:10.319 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:55:10.319 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:55:10.320 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:55:10.320 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:55:10.320 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:55:10.320 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:55:10.320 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:55:10.321 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:55:10.321 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:55:10.321 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:55:10.321 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:55:10.321 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:55:10.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:55:10.322 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:55:10.391 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:55:10.391+0000 7f6af946d480 -1 init, newargv = 0x558d19a7d8c0 newargc=15 2023-11-09T03:55:10.391 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[131924]: starting ceph client 2023-11-09T03:55:10.397 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[131924]: starting fuse 2023-11-09T03:55:11.471 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:55:11.492 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:55:11.492 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:55:11.492 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:55:11.492 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:55:11.493 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:55:11.493 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:55:11.493 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:55:11.493 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:55:11.494 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:55:11.494 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:55:11.494 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:55:11.495 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:55:11.495 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:55:11.495 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:55:11.495 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:55:11.496 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:55:11.496 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:55:11.496 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:55:11.496 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:55:11.497 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:55:11.497 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:55:11.497 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:55:11.497 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:55:11.498 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:55:11.498 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:55:11.498 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:55:11.498 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:55:11.499 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:55:11.499 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:55:11.499 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:55:11.499 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:55:11.500 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:55:11.500 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:55:11.500 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:55:11.501 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-09T03:55:11.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:55:11.501 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:55:11.550 INFO:teuthology.orchestra.run.smithi076.stdout:43 2023-11-09T03:55:11.550 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-09T03:55:11.551 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer -- python3 -c ' 2023-11-09T03:55:11.551 DEBUG:teuthology.orchestra.run.smithi076:> import glob 2023-11-09T03:55:11.552 DEBUG:teuthology.orchestra.run.smithi076:> import re 2023-11-09T03:55:11.552 DEBUG:teuthology.orchestra.run.smithi076:> import os 2023-11-09T03:55:11.553 DEBUG:teuthology.orchestra.run.smithi076:> import subprocess 2023-11-09T03:55:11.553 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:55:11.554 DEBUG:teuthology.orchestra.run.smithi076:> def _find_admin_socket(client_name): 2023-11-09T03:55:11.555 DEBUG:teuthology.orchestra.run.smithi076:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-09T03:55:11.555 DEBUG:teuthology.orchestra.run.smithi076:> files = glob.glob(asok_path) 2023-11-09T03:55:11.556 DEBUG:teuthology.orchestra.run.smithi076:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-09T03:55:11.556 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:55:11.557 DEBUG:teuthology.orchestra.run.smithi076:> # Given a non-glob path, it better be there 2023-11-09T03:55:11.557 DEBUG:teuthology.orchestra.run.smithi076:> if "*" not in asok_path: 2023-11-09T03:55:11.558 DEBUG:teuthology.orchestra.run.smithi076:> assert(len(files) == 1) 2023-11-09T03:55:11.558 DEBUG:teuthology.orchestra.run.smithi076:> return files[0] 2023-11-09T03:55:11.559 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:55:11.559 DEBUG:teuthology.orchestra.run.smithi076:> for f in files: 2023-11-09T03:55:11.559 DEBUG:teuthology.orchestra.run.smithi076:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-09T03:55:11.560 DEBUG:teuthology.orchestra.run.smithi076:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-09T03:55:11.560 DEBUG:teuthology.orchestra.run.smithi076:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-09T03:55:11.561 DEBUG:teuthology.orchestra.run.smithi076:> contents = proc_f.read() 2023-11-09T03:55:11.561 DEBUG:teuthology.orchestra.run.smithi076:> if mountpoint in contents: 2023-11-09T03:55:11.561 DEBUG:teuthology.orchestra.run.smithi076:> return f 2023-11-09T03:55:11.562 DEBUG:teuthology.orchestra.run.smithi076:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-09T03:55:11.562 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:55:11.563 DEBUG:teuthology.orchestra.run.smithi076:> print(_find_admin_socket("client.0")) 2023-11-09T03:55:11.563 DEBUG:teuthology.orchestra.run.smithi076:> ' 2023-11-09T03:55:11.675 INFO:teuthology.orchestra.run.smithi076.stdout:/var/run/ceph/ceph-client.0.131924.asok 2023-11-09T03:55:11.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:55:11.687 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.131924.asok 2023-11-09T03:55:11.687 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:55:11.688 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.131924.asok status 2023-11-09T03:55:11.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:55:11.854 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:55:11.855 INFO:teuthology.orchestra.run.smithi076.stdout: "metadata": { 2023-11-09T03:55:11.855 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_sha1": "7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4", 2023-11-09T03:55:11.855 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev)", 2023-11-09T03:55:11.855 INFO:teuthology.orchestra.run.smithi076.stdout: "entity_id": "0", 2023-11-09T03:55:11.855 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": "smithi076", 2023-11-09T03:55:11.855 INFO:teuthology.orchestra.run.smithi076.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-09T03:55:11.856 INFO:teuthology.orchestra.run.smithi076.stdout: "pid": "131924", 2023-11-09T03:55:11.856 INFO:teuthology.orchestra.run.smithi076.stdout: "root": "/" 2023-11-09T03:55:11.856 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:55:11.856 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_count": 0, 2023-11-09T03:55:11.856 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_pinned_count": 0, 2023-11-09T03:55:11.857 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 9441, 2023-11-09T03:55:11.857 INFO:teuthology.orchestra.run.smithi076.stdout: "inst": { 2023-11-09T03:55:11.857 INFO:teuthology.orchestra.run.smithi076.stdout: "name": { 2023-11-09T03:55:11.857 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "client", 2023-11-09T03:55:11.857 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 9441 2023-11-09T03:55:11.857 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:55:11.858 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2023-11-09T03:55:11.858 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2023-11-09T03:55:11.858 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2023-11-09T03:55:11.858 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 497547142 2023-11-09T03:55:11.858 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:55:11.859 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:55:11.859 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2023-11-09T03:55:11.859 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2023-11-09T03:55:11.859 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2023-11-09T03:55:11.859 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 497547142 2023-11-09T03:55:11.860 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:55:11.860 INFO:teuthology.orchestra.run.smithi076.stdout: "inst_str": "client.9441 192.168.0.1:0/497547142", 2023-11-09T03:55:11.860 INFO:teuthology.orchestra.run.smithi076.stdout: "addr_str": "192.168.0.1:0/497547142", 2023-11-09T03:55:11.860 INFO:teuthology.orchestra.run.smithi076.stdout: "inode_count": 1, 2023-11-09T03:55:11.860 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_epoch": 133, 2023-11-09T03:55:11.861 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch": 118, 2023-11-09T03:55:11.861 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch_barrier": 0, 2023-11-09T03:55:11.861 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklisted": false, 2023-11-09T03:55:11.861 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs" 2023-11-09T03:55:11.861 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:55:11.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:55:11.868 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:55:11.869 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-11-09T03:55:11.869 DEBUG:teuthology.orchestra.run.smithi076:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-09T03:55:11.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:55:11.928 INFO:teuthology.orchestra.run.smithi076.stdout:fuseblk 2023-11-09T03:55:11.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:55:11.935 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-09T03:55:11.935 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:55:11.935 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-09T03:55:11.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:55:11.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:55:12.001 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 flush journal 2023-11-09T03:55:12.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.103+0000 7f4d2a5b6640 1 Processor -- start 2023-11-09T03:55:12.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.103+0000 7f4d2a5b6640 1 -- start start 2023-11-09T03:55:12.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.104+0000 7f4d2a5b6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4d2415d100 0x7f4d240ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.104+0000 7f4d2a5b6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d2415d4e0 0x7f4d240d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.104+0000 7f4d2a5b6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4d240d3b60 0x7f4d240d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.104+0000 7f4d2a5b6640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d240b32b0 con 0x7f4d2415d100 2023-11-09T03:55:12.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.104+0000 7f4d2a5b6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d240cee10 con 0x7f4d240d3b60 2023-11-09T03:55:12.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.104+0000 7f4d2a5b6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d240b2980 con 0x7f4d2415d4e0 2023-11-09T03:55:12.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.104+0000 7f4d23fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4d2415d100 0x7f4d240ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.104+0000 7f4d23fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4d2415d100 0x7f4d240ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47604/0 (socket says 172.21.15.76:47604) 2023-11-09T03:55:12.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.104+0000 7f4d23fff640 1 -- 172.21.15.76:0/915828954 learned_addr learned my addr 172.21.15.76:0/915828954 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:12.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.104+0000 7f4d28b53640 1 --2- 172.21.15.76:0/915828954 >> v2:172.21.15.113:3300/0 conn(0x7f4d240d3b60 0x7f4d240d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.104+0000 7f4d237fe640 1 --2- 172.21.15.76:0/915828954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d2415d4e0 0x7f4d240d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.105+0000 7f4d23fff640 1 -- 172.21.15.76:0/915828954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d2415d4e0 msgr2=0x7f4d240d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:12.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.105+0000 7f4d23fff640 1 --2- 172.21.15.76:0/915828954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d2415d4e0 0x7f4d240d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.105+0000 7f4d23fff640 1 -- 172.21.15.76:0/915828954 >> v2:172.21.15.113:3300/0 conn(0x7f4d240d3b60 msgr2=0x7f4d240d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:12.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.105+0000 7f4d23fff640 1 --2- 172.21.15.76:0/915828954 >> v2:172.21.15.113:3300/0 conn(0x7f4d240d3b60 0x7f4d240d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.105+0000 7f4d23fff640 1 -- 172.21.15.76:0/915828954 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d240b2c60 con 0x7f4d2415d100 2023-11-09T03:55:12.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.105+0000 7f4d23fff640 1 --2- 172.21.15.76:0/915828954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d2415d100 0x7f4d240ce510 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f4d240b27a0 tx=0x7f4d1402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39e3f6f66372a2a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:12.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.105+0000 7f4d22ffd640 1 -- 172.21.15.76:0/915828954 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14004230 con 0x7f4d2415d100 2023-11-09T03:55:12.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.105+0000 7f4d22ffd640 1 -- 172.21.15.76:0/915828954 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d14004390 con 0x7f4d2415d100 2023-11-09T03:55:12.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.105+0000 7f4d22ffd640 1 -- 172.21.15.76:0/915828954 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14003810 con 0x7f4d2415d100 2023-11-09T03:55:12.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.106+0000 7f4d2a5b6640 1 -- 172.21.15.76:0/915828954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d2415d100 msgr2=0x7f4d240ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:12.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.106+0000 7f4d2a5b6640 1 --2- 172.21.15.76:0/915828954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d2415d100 0x7f4d240ce510 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f4d240b27a0 tx=0x7f4d1402b8f0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.106+0000 7f4d2a5b6640 1 -- 172.21.15.76:0/915828954 shutdown_connections 2023-11-09T03:55:12.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.106+0000 7f4d2a5b6640 1 --2- 172.21.15.76:0/915828954 >> v2:172.21.15.113:3300/0 conn(0x7f4d240d3b60 0x7f4d240d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.106+0000 7f4d2a5b6640 1 --2- 172.21.15.76:0/915828954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d2415d4e0 0x7f4d240d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.106+0000 7f4d2a5b6640 1 --2- 172.21.15.76:0/915828954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d2415d100 0x7f4d240ce510 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.106+0000 7f4d2a5b6640 1 -- 172.21.15.76:0/915828954 >> 172.21.15.76:0/915828954 conn(0x7f4d240d2ef0 msgr2=0x7f4d240d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:12.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.106+0000 7f4d2a5b6640 1 -- 172.21.15.76:0/915828954 shutdown_connections 2023-11-09T03:55:12.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.106+0000 7f4d2a5b6640 1 -- 172.21.15.76:0/915828954 wait complete. 2023-11-09T03:55:12.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.107+0000 7f4d2a5b6640 1 Processor -- start 2023-11-09T03:55:12.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.107+0000 7f4d2a5b6640 1 -- start start 2023-11-09T03:55:12.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.108+0000 7f4d2a5b6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d240d3b60 0x7f4d2414bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.108+0000 7f4d2a5b6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d2415d100 0x7f4d240b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.108+0000 7f4d2a5b6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d2415d4e0 0x7f4d240cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.108+0000 7f4d2a5b6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2409ce30 con 0x7f4d2415d4e0 2023-11-09T03:55:12.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.108+0000 7f4d2a5b6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2409d110 con 0x7f4d240d3b60 2023-11-09T03:55:12.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.108+0000 7f4d2a5b6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2409d3f0 con 0x7f4d2415d100 2023-11-09T03:55:12.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.108+0000 7f4d28b53640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d2415d4e0 0x7f4d240cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.108+0000 7f4d28b53640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d2415d4e0 0x7f4d240cbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47612/0 (socket says 172.21.15.76:47612) 2023-11-09T03:55:12.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.108+0000 7f4d28b53640 1 -- 172.21.15.76:0/133707415 learned_addr learned my addr 172.21.15.76:0/133707415 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:12.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.108+0000 7f4d23fff640 1 --2- 172.21.15.76:0/133707415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d240d3b60 0x7f4d2414bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.108+0000 7f4d237fe640 1 --2- 172.21.15.76:0/133707415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d2415d100 0x7f4d240b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.108+0000 7f4d28b53640 1 -- 172.21.15.76:0/133707415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d2415d100 msgr2=0x7f4d240b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:12.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.108+0000 7f4d28b53640 1 --2- 172.21.15.76:0/133707415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d2415d100 0x7f4d240b3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.108+0000 7f4d28b53640 1 -- 172.21.15.76:0/133707415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d240d3b60 msgr2=0x7f4d2414bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:12.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.108+0000 7f4d28b53640 1 --2- 172.21.15.76:0/133707415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d240d3b60 0x7f4d2414bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.108+0000 7f4d28b53640 1 -- 172.21.15.76:0/133707415 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d14009460 con 0x7f4d2415d4e0 2023-11-09T03:55:12.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.109+0000 7f4d28b53640 1 --2- 172.21.15.76:0/133707415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d2415d4e0 0x7f4d240cbbc0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c007e60 tx=0x7f4d1c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:12.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.109+0000 7f4d20ff9640 1 -- 172.21.15.76:0/133707415 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c014620 con 0x7f4d2415d4e0 2023-11-09T03:55:12.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.109+0000 7f4d20ff9640 1 -- 172.21.15.76:0/133707415 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1c00ee90 con 0x7f4d2415d4e0 2023-11-09T03:55:12.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.109+0000 7f4d2a5b6640 1 -- 172.21.15.76:0/133707415 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d2409d6d0 con 0x7f4d2415d4e0 2023-11-09T03:55:12.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.109+0000 7f4d20ff9640 1 -- 172.21.15.76:0/133707415 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c0149e0 con 0x7f4d2415d4e0 2023-11-09T03:55:12.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.109+0000 7f4d2a5b6640 1 -- 172.21.15.76:0/133707415 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d2414c410 con 0x7f4d2415d4e0 2023-11-09T03:55:12.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.110+0000 7f4d20ff9640 1 -- 172.21.15.76:0/133707415 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4d1c014b40 con 0x7f4d2415d4e0 2023-11-09T03:55:12.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.110+0000 7f4d20ff9640 1 --2- 172.21.15.76:0/133707415 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d1006ae70 0x7f4d1006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.111+0000 7f4d20ff9640 1 -- 172.21.15.76:0/133707415 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7f4d1c08aa60 con 0x7f4d2415d4e0 2023-11-09T03:55:12.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.111+0000 7f4d23fff640 1 --2- 172.21.15.76:0/133707415 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d1006ae70 0x7f4d1006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.111+0000 7f4d23fff640 1 --2- 172.21.15.76:0/133707415 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d1006ae70 0x7f4d1006d330 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4d14035000 tx=0x7f4d14008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:12.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.112+0000 7f4d2d223740 1 Processor -- start 2023-11-09T03:55:12.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.112+0000 7f4d2d223740 1 -- start start 2023-11-09T03:55:12.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.112+0000 7f4d2d223740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563ebed944c0 0x563ebed948a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.112+0000 7f4d2d223740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563ebed646d0 0x563ebed91730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.112+0000 7f4d2d223740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563ebed68040 0x563ebed91c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.112+0000 7f4d2d223740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebecd7d70 con 0x563ebed944c0 2023-11-09T03:55:12.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.112+0000 7f4d2d223740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebecd5460 con 0x563ebed646d0 2023-11-09T03:55:12.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.112+0000 7f4d2d223740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ebecd0d50 con 0x563ebed68040 2023-11-09T03:55:12.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.112+0000 7f4d23fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563ebed944c0 0x563ebed948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.112+0000 7f4d23fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563ebed944c0 0x563ebed948a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47620/0 (socket says 172.21.15.76:47620) 2023-11-09T03:55:12.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.112+0000 7f4d23fff640 1 -- 172.21.15.76:0/359276358 learned_addr learned my addr 172.21.15.76:0/359276358 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:12.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.113+0000 7f4d237fe640 1 --2- 172.21.15.76:0/359276358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563ebed646d0 0x563ebed91730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.113+0000 7f4d28b53640 1 --2- 172.21.15.76:0/359276358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563ebed68040 0x563ebed91c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.113+0000 7f4d23fff640 1 -- 172.21.15.76:0/359276358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563ebed68040 msgr2=0x563ebed91c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:12.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.113+0000 7f4d23fff640 1 --2- 172.21.15.76:0/359276358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563ebed68040 0x563ebed91c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.113+0000 7f4d23fff640 1 -- 172.21.15.76:0/359276358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563ebed646d0 msgr2=0x563ebed91730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:12.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.113+0000 7f4d23fff640 1 --2- 172.21.15.76:0/359276358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563ebed646d0 0x563ebed91730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.113+0000 7f4d23fff640 1 -- 172.21.15.76:0/359276358 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ebec93e30 con 0x563ebed944c0 2023-11-09T03:55:12.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.113+0000 7f4d237fe640 1 --2- 172.21.15.76:0/359276358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563ebed646d0 0x563ebed91730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:55:12.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.113+0000 7f4d23fff640 1 --2- 172.21.15.76:0/359276358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563ebed944c0 0x563ebed948a0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f4d1402bf00 tx=0x7f4d14030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb32989b00bfdf5 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:12.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.113+0000 7f4d09ffb640 1 -- 172.21.15.76:0/359276358 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14003810 con 0x563ebed944c0 2023-11-09T03:55:12.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.113+0000 7f4d09ffb640 1 -- 172.21.15.76:0/359276358 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1404b070 con 0x563ebed944c0 2023-11-09T03:55:12.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.113+0000 7f4d09ffb640 1 -- 172.21.15.76:0/359276358 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14003b90 con 0x563ebed944c0 2023-11-09T03:55:12.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.114+0000 7f4d2d223740 1 -- 172.21.15.76:0/359276358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563ebed944c0 msgr2=0x563ebed948a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:12.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.114+0000 7f4d2d223740 1 --2- 172.21.15.76:0/359276358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563ebed944c0 0x563ebed948a0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f4d1402bf00 tx=0x7f4d14030000 comp rx=0 tx=0).stop 2023-11-09T03:55:12.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.114+0000 7f4d2d223740 1 -- 172.21.15.76:0/359276358 shutdown_connections 2023-11-09T03:55:12.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.114+0000 7f4d2d223740 1 --2- 172.21.15.76:0/359276358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563ebed68040 0x563ebed91c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.114+0000 7f4d2d223740 1 --2- 172.21.15.76:0/359276358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563ebed646d0 0x563ebed91730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.114+0000 7f4d2d223740 1 --2- 172.21.15.76:0/359276358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563ebed944c0 0x563ebed948a0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.114+0000 7f4d2d223740 1 -- 172.21.15.76:0/359276358 >> 172.21.15.76:0/359276358 conn(0x563ebed669e0 msgr2=0x563ebedaf2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:12.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.114+0000 7f4d2d223740 1 -- 172.21.15.76:0/359276358 shutdown_connections 2023-11-09T03:55:12.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.114+0000 7f4d2d223740 1 -- 172.21.15.76:0/359276358 wait complete. 2023-11-09T03:55:12.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.114+0000 7f4d2d223740 1 -- start start 2023-11-09T03:55:12.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.114+0000 7f4d2d223740 1 Processor -- start 2023-11-09T03:55:12.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.115+0000 7f4d2d223740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563ebed68040 0x563ebedc0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.115+0000 7f4d2d223740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563ebed944c0 0x563ebed96390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.115+0000 7f4d2d223740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563ebed7fa60 0x563ebed968d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.115+0000 7f4d2d223740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebec8aee0 con 0x563ebed68040 2023-11-09T03:55:12.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.115+0000 7f4d2d223740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebec8ab80 con 0x563ebed944c0 2023-11-09T03:55:12.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.115+0000 7f4d2d223740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ebec503a0 con 0x563ebed7fa60 2023-11-09T03:55:12.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.115+0000 7f4d2d223740 10 client.0 _ll_register_callbacks cb 0x563ebec548b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:55:12.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.115+0000 7f4d23fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563ebed68040 0x563ebedc0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.115+0000 7f4d23fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563ebed68040 0x563ebedc0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47622/0 (socket says 172.21.15.76:47622) 2023-11-09T03:55:12.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.115+0000 7f4d23fff640 1 -- 172.21.15.76:0/4132317675 learned_addr learned my addr 172.21.15.76:0/4132317675 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:12.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.115+0000 7f4d237fe640 1 --2- 172.21.15.76:0/4132317675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563ebed944c0 0x563ebed96390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.115+0000 7f4d28b53640 1 --2- 172.21.15.76:0/4132317675 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563ebed7fa60 0x563ebed968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.116+0000 7f4d23fff640 1 -- 172.21.15.76:0/4132317675 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563ebed7fa60 msgr2=0x563ebed968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:12.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.116+0000 7f4d23fff640 1 --2- 172.21.15.76:0/4132317675 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563ebed7fa60 0x563ebed968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.116+0000 7f4d23fff640 1 -- 172.21.15.76:0/4132317675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563ebed944c0 msgr2=0x563ebed96390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:12.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.116+0000 7f4d23fff640 1 --2- 172.21.15.76:0/4132317675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563ebed944c0 0x563ebed96390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.116+0000 7f4d23fff640 1 -- 172.21.15.76:0/4132317675 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d14041070 con 0x563ebed68040 2023-11-09T03:55:12.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.116+0000 7f4d23fff640 1 --2- 172.21.15.76:0/4132317675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563ebed68040 0x563ebedc0080 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f4d140090b0 tx=0x7f4d14004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa81201b5cfab5b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:12.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.116+0000 7f4d02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:55:12.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.116+0000 7f4d02ffd640 1 -- 172.21.15.76:0/4132317675 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14003c20 con 0x563ebed68040 2023-11-09T03:55:12.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.116+0000 7f4d02ffd640 1 -- 172.21.15.76:0/4132317675 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7f4d14003d80 con 0x563ebed68040 2023-11-09T03:55:12.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.116+0000 7f4d2d223740 1 -- 172.21.15.76:0/4132317675 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563ebecc9d30 con 0x563ebed68040 2023-11-09T03:55:12.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.116+0000 7f4d02ffd640 1 -- 172.21.15.76:0/4132317675 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=133) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d1405f550 con 0x563ebed68040 2023-11-09T03:55:12.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.116+0000 7f4d2d223740 10 client.9453 fetch_fsmap learned FSMap version 133 2023-11-09T03:55:12.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.116+0000 7f4d2d223740 1 -- 172.21.15.76:0/4132317675 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=133,monmap=2+}) v3 -- 0x563ebeccf3d0 con 0x563ebed68040 2023-11-09T03:55:12.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.116+0000 7f4d02ffd640 1 -- 172.21.15.76:0/4132317675 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563ebecc9d30 con 0x563ebed68040 2023-11-09T03:55:12.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.116+0000 7f4d02ffd640 1 -- 172.21.15.76:0/4132317675 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 133) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f4d14038070 con 0x563ebed68040 2023-11-09T03:55:12.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.116+0000 7f4d2d223740 10 client.9453 fetch_fsmap finished waiting for FSMap version 133 2023-11-09T03:55:12.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.116+0000 7f4d2d223740 10 client.9453 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2023-11-09T03:55:12.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.116+0000 7f4d2d223740 20 client.9453 populate_metadata read hostname 'smithi076' 2023-11-09T03:55:12.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.116+0000 7f4d2d223740 1 --2- 172.21.15.76:0/4132317675 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x563ebed990d0 0x563ebed994e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.116+0000 7f4d2d223740 4 client.9453 mds_command: new command op to 8568 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:55:12.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.116+0000 7f4d2d223740 1 -- 172.21.15.76:0/4132317675 --> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ebecc9a50 con 0x563ebed990d0 2023-11-09T03:55:12.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.116+0000 7f4d237fe640 1 --2- 172.21.15.76:0/4132317675 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x563ebed990d0 0x563ebed994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.117+0000 7f4d02ffd640 10 client.9453 ms_handle_connect on v2:172.21.15.76:6832/153465374 2023-11-09T03:55:12.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.117+0000 7f4d237fe640 1 --2- 172.21.15.76:0/4132317675 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x563ebed990d0 0x563ebed994e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3bf6a04c897a7f4 server_cookie=981d9e1520771a4b in_seq=0 out_seq=0 2023-11-09T03:55:12.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.117+0000 7f4d02ffd640 1 -- 172.21.15.76:0/4132317675 <== mds.0 v2:172.21.15.76:6832/153465374 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563ebecc9a50 con 0x563ebed990d0 2023-11-09T03:55:12.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.117+0000 7f4d02ffd640 10 client.9453 handle_command_reply: tid=0 2023-11-09T03:55:12.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.117+0000 7f4d2d223740 1 client.9453 shutdown 2023-11-09T03:55:12.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.117+0000 7f4d2d223740 1 -- 172.21.15.76:0/4132317675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563ebed68040 msgr2=0x563ebedc0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:12.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.117+0000 7f4d2d223740 1 --2- 172.21.15.76:0/4132317675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563ebed68040 0x563ebedc0080 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f4d140090b0 tx=0x7f4d14004730 comp rx=0 tx=0).stop 2023-11-09T03:55:12.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.117+0000 7f4d2d223740 1 -- 172.21.15.76:0/4132317675 shutdown_connections 2023-11-09T03:55:12.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.117+0000 7f4d2d223740 1 --2- 172.21.15.76:0/4132317675 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x563ebed990d0 0x563ebed994e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.117+0000 7f4d2d223740 1 --2- 172.21.15.76:0/4132317675 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563ebed7fa60 0x563ebed968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.117+0000 7f4d2d223740 1 --2- 172.21.15.76:0/4132317675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563ebed944c0 0x563ebed96390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.117+0000 7f4d02ffd640 0 client.9453 ms_handle_reset on v2:172.21.15.76:6832/153465374 2023-11-09T03:55:12.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.117+0000 7f4d2d223740 1 --2- 172.21.15.76:0/4132317675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563ebed68040 0x563ebedc0080 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.117+0000 7f4d2d223740 1 -- 172.21.15.76:0/4132317675 >> 172.21.15.76:0/4132317675 conn(0x563ebed646d0 msgr2=0x563ebed98160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:12.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.118+0000 7f4d2d223740 1 -- 172.21.15.76:0/4132317675 shutdown_connections 2023-11-09T03:55:12.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.118+0000 7f4d2d223740 1 -- 172.21.15.76:0/4132317675 wait complete. 2023-11-09T03:55:12.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.118+0000 7f4d2d223740 20 client.9453 trim_cache size 0 max 16384 2023-11-09T03:55:12.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.131+0000 7f4d2d223740 1 Processor -- start 2023-11-09T03:55:12.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.131+0000 7f4d2d223740 1 -- start start 2023-11-09T03:55:12.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.131+0000 7f4d2d223740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563ebed68040 0x563ebedb27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.131+0000 7f4d2d223740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563ebed7fa60 0x563ebedb2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.131+0000 7f4d2d223740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563ebed944c0 0x563ebedc8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.131+0000 7f4d2d223740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebedb19f0 con 0x563ebed7fa60 2023-11-09T03:55:12.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.131+0000 7f4d2d223740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebed7f870 con 0x563ebed68040 2023-11-09T03:55:12.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.131+0000 7f4d2d223740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ebed9db30 con 0x563ebed944c0 2023-11-09T03:55:12.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.131+0000 7f4d237fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563ebed7fa60 0x563ebedb2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.131+0000 7f4d237fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563ebed7fa60 0x563ebedb2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47638/0 (socket says 172.21.15.76:47638) 2023-11-09T03:55:12.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.131+0000 7f4d237fe640 1 -- 172.21.15.76:0/4169056789 learned_addr learned my addr 172.21.15.76:0/4169056789 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:12.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.132+0000 7f4d28b53640 1 --2- 172.21.15.76:0/4169056789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563ebed944c0 0x563ebedc8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.132+0000 7f4d237fe640 1 -- 172.21.15.76:0/4169056789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563ebed944c0 msgr2=0x563ebedc8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:12.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.132+0000 7f4d237fe640 1 --2- 172.21.15.76:0/4169056789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563ebed944c0 0x563ebedc8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.132+0000 7f4d237fe640 1 -- 172.21.15.76:0/4169056789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563ebed68040 msgr2=0x563ebedb27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:12.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.132+0000 7f4d23fff640 1 --2- 172.21.15.76:0/4169056789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563ebed68040 0x563ebedb27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.132+0000 7f4d237fe640 1 --2- 172.21.15.76:0/4169056789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563ebed68040 0x563ebedb27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.132+0000 7f4d237fe640 1 -- 172.21.15.76:0/4169056789 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ebec6cea0 con 0x563ebed7fa60 2023-11-09T03:55:12.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.132+0000 7f4d28b53640 1 --2- 172.21.15.76:0/4169056789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563ebed944c0 0x563ebedc8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:55:12.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.132+0000 7f4d23fff640 1 --2- 172.21.15.76:0/4169056789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563ebed68040 0x563ebedb27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:55:12.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.132+0000 7f4d237fe640 1 --2- 172.21.15.76:0/4169056789 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563ebed7fa60 0x563ebedb2d20 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800b1f0 tx=0x7f4d18008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe3b7ab56d2a7fe server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:12.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.132+0000 7f4d0a7fc640 1 -- 172.21.15.76:0/4169056789 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1801c070 con 0x563ebed7fa60 2023-11-09T03:55:12.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.132+0000 7f4d0a7fc640 1 -- 172.21.15.76:0/4169056789 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d18004020 con 0x563ebed7fa60 2023-11-09T03:55:12.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.132+0000 7f4d0a7fc640 1 -- 172.21.15.76:0/4169056789 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d18004390 con 0x563ebed7fa60 2023-11-09T03:55:12.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.132+0000 7f4d2d223740 1 -- 172.21.15.76:0/4169056789 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563ebed7fa60 msgr2=0x563ebedb2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:12.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.132+0000 7f4d2d223740 1 --2- 172.21.15.76:0/4169056789 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563ebed7fa60 0x563ebedb2d20 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800b1f0 tx=0x7f4d18008040 comp rx=0 tx=0).stop 2023-11-09T03:55:12.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.133+0000 7f4d2d223740 1 -- 172.21.15.76:0/4169056789 shutdown_connections 2023-11-09T03:55:12.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.133+0000 7f4d2d223740 1 --2- 172.21.15.76:0/4169056789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563ebed944c0 0x563ebedc8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.133+0000 7f4d2d223740 1 --2- 172.21.15.76:0/4169056789 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563ebed7fa60 0x563ebedb2d20 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.133+0000 7f4d2d223740 1 --2- 172.21.15.76:0/4169056789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563ebed68040 0x563ebedb27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.133+0000 7f4d2d223740 1 -- 172.21.15.76:0/4169056789 >> 172.21.15.76:0/4169056789 conn(0x563ebed646d0 msgr2=0x563ebed64ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:12.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.133+0000 7f4d2d223740 1 -- 172.21.15.76:0/4169056789 shutdown_connections 2023-11-09T03:55:12.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.133+0000 7f4d2d223740 1 -- 172.21.15.76:0/4169056789 wait complete. 2023-11-09T03:55:12.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.133+0000 7f4d2d223740 1 -- start start 2023-11-09T03:55:12.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.133+0000 7f4d2d223740 1 Processor -- start 2023-11-09T03:55:12.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.134+0000 7f4d2d223740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563ebed68040 0x563ebee1e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.134+0000 7f4d2d223740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563ebed7fa60 0x563ebed9c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.134+0000 7f4d2d223740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563ebed944c0 0x563ebed9c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.134+0000 7f4d2d223740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebee1eb00 con 0x563ebed68040 2023-11-09T03:55:12.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.134+0000 7f4d2d223740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebedbf000 con 0x563ebed944c0 2023-11-09T03:55:12.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.134+0000 7f4d2d223740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ebedb27e0 con 0x563ebed7fa60 2023-11-09T03:55:12.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.134+0000 7f4d2d223740 10 client.0 _ll_register_callbacks cb 0x7f4d14009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:55:12.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.134+0000 7f4d23fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563ebed68040 0x563ebee1e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.134+0000 7f4d23fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563ebed68040 0x563ebee1e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47644/0 (socket says 172.21.15.76:47644) 2023-11-09T03:55:12.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.134+0000 7f4d23fff640 1 -- 172.21.15.76:0/2651007987 learned_addr learned my addr 172.21.15.76:0/2651007987 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:12.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.134+0000 7f4d237fe640 1 --2- 172.21.15.76:0/2651007987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563ebed7fa60 0x563ebed9c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.134+0000 7f4d28b53640 1 --2- 172.21.15.76:0/2651007987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563ebed944c0 0x563ebed9c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.134+0000 7f4d23fff640 1 -- 172.21.15.76:0/2651007987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563ebed7fa60 msgr2=0x563ebed9c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:12.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.134+0000 7f4d23fff640 1 --2- 172.21.15.76:0/2651007987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563ebed7fa60 0x563ebed9c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.134+0000 7f4d23fff640 1 -- 172.21.15.76:0/2651007987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563ebed944c0 msgr2=0x563ebed9c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:12.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.134+0000 7f4d23fff640 1 --2- 172.21.15.76:0/2651007987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563ebed944c0 0x563ebed9c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.134+0000 7f4d23fff640 1 -- 172.21.15.76:0/2651007987 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1800e040 con 0x563ebed68040 2023-11-09T03:55:12.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.134+0000 7f4d237fe640 1 --2- 172.21.15.76:0/2651007987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563ebed7fa60 0x563ebed9c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:55:12.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.135+0000 7f4d23fff640 1 --2- 172.21.15.76:0/2651007987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563ebed68040 0x563ebee1e5c0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f4d1402bf00 tx=0x7f4d14031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b284ffc1acda9d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:12.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.135+0000 7f4d02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:55:12.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.135+0000 7f4d02ffd640 1 -- 172.21.15.76:0/2651007987 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14003e40 con 0x563ebed68040 2023-11-09T03:55:12.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.135+0000 7f4d02ffd640 1 -- 172.21.15.76:0/2651007987 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7f4d1405f9a0 con 0x563ebed68040 2023-11-09T03:55:12.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.135+0000 7f4d2d223740 1 -- 172.21.15.76:0/2651007987 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d1404b070 con 0x563ebed68040 2023-11-09T03:55:12.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.135+0000 7f4d02ffd640 1 -- 172.21.15.76:0/2651007987 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=133) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d1405fb60 con 0x563ebed68040 2023-11-09T03:55:12.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.135+0000 7f4d2d223740 10 client.9459 fetch_fsmap learned FSMap version 133 2023-11-09T03:55:12.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.135+0000 7f4d2d223740 1 -- 172.21.15.76:0/2651007987 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=133,monmap=2+}) v3 -- 0x563ebec92690 con 0x563ebed68040 2023-11-09T03:55:12.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.135+0000 7f4d02ffd640 1 -- 172.21.15.76:0/2651007987 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d14076020 con 0x563ebed68040 2023-11-09T03:55:12.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.135+0000 7f4d02ffd640 1 -- 172.21.15.76:0/2651007987 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 133) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f4d14070020 con 0x563ebed68040 2023-11-09T03:55:12.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.135+0000 7f4d2d223740 10 client.9459 fetch_fsmap finished waiting for FSMap version 133 2023-11-09T03:55:12.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.135+0000 7f4d2d223740 10 client.9459 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2023-11-09T03:55:12.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.135+0000 7f4d2d223740 20 client.9459 populate_metadata read hostname 'smithi076' 2023-11-09T03:55:12.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.135+0000 7f4d2d223740 1 --2- 172.21.15.76:0/2651007987 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x563ebee1b240 0x563ebedbe070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.135+0000 7f4d2d223740 4 client.9459 mds_command: new command op to 8568 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-11-09T03:55:12.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.135+0000 7f4d2d223740 1 -- 172.21.15.76:0/2651007987 --> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563ebe9d17e0 con 0x563ebee1b240 2023-11-09T03:55:12.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.135+0000 7f4d237fe640 1 --2- 172.21.15.76:0/2651007987 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x563ebee1b240 0x563ebedbe070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.136+0000 7f4d237fe640 1 --2- 172.21.15.76:0/2651007987 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x563ebee1b240 0x563ebedbe070 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fdd84ce30ff8889 server_cookie=62e1f91837caf69c in_seq=0 out_seq=0 2023-11-09T03:55:12.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.136+0000 7f4d02ffd640 10 client.9459 ms_handle_connect on v2:172.21.15.76:6832/153465374 2023-11-09T03:55:12.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.146+0000 7f4d02ffd640 1 -- 172.21.15.76:0/2651007987 <== mds.0 v2:172.21.15.76:6832/153465374 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563ebe9d17e0 con 0x563ebee1b240 2023-11-09T03:55:12.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.146+0000 7f4d02ffd640 10 client.9459 handle_command_reply: tid=0 2023-11-09T03:55:12.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.147+0000 7f4d2d223740 1 client.9459 shutdown 2023-11-09T03:55:12.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.147+0000 7f4d2d223740 1 -- 172.21.15.76:0/2651007987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563ebed68040 msgr2=0x563ebee1e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:12.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.147+0000 7f4d2d223740 1 --2- 172.21.15.76:0/2651007987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563ebed68040 0x563ebee1e5c0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f4d1402bf00 tx=0x7f4d14031000 comp rx=0 tx=0).stop 2023-11-09T03:55:12.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.147+0000 7f4d2d223740 1 -- 172.21.15.76:0/2651007987 shutdown_connections 2023-11-09T03:55:12.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.147+0000 7f4d2d223740 1 --2- 172.21.15.76:0/2651007987 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x563ebee1b240 0x563ebedbe070 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.147+0000 7f4d2d223740 1 --2- 172.21.15.76:0/2651007987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563ebed944c0 0x563ebed9c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.147+0000 7f4d02ffd640 0 client.9459 ms_handle_reset on v2:172.21.15.76:6832/153465374 2023-11-09T03:55:12.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.147+0000 7f4d2d223740 1 --2- 172.21.15.76:0/2651007987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563ebed7fa60 0x563ebed9c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.147+0000 7f4d2d223740 1 --2- 172.21.15.76:0/2651007987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563ebed68040 0x563ebee1e5c0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.147+0000 7f4d2d223740 1 -- 172.21.15.76:0/2651007987 >> 172.21.15.76:0/2651007987 conn(0x563ebed646d0 msgr2=0x563ebed901d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:12.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.147+0000 7f4d2d223740 1 -- 172.21.15.76:0/2651007987 shutdown_connections 2023-11-09T03:55:12.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.147+0000 7f4d2d223740 1 -- 172.21.15.76:0/2651007987 wait complete. 2023-11-09T03:55:12.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.147+0000 7f4d2d223740 20 client.9459 trim_cache size 0 max 16384 2023-11-09T03:55:12.167 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:55:12.168 INFO:teuthology.orchestra.run.smithi076.stdout: "message": "", 2023-11-09T03:55:12.168 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2023-11-09T03:55:12.168 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:55:12.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.147+0000 7f4d2a5b6640 1 -- 172.21.15.76:0/133707415 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d1006ae70 msgr2=0x7f4d1006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:12.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.147+0000 7f4d2a5b6640 1 --2- 172.21.15.76:0/133707415 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d1006ae70 0x7f4d1006d330 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4d14035000 tx=0x7f4d14008030 comp rx=0 tx=0).stop 2023-11-09T03:55:12.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.147+0000 7f4d2a5b6640 1 -- 172.21.15.76:0/133707415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d2415d4e0 msgr2=0x7f4d240cbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:12.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.147+0000 7f4d2a5b6640 1 --2- 172.21.15.76:0/133707415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d2415d4e0 0x7f4d240cbbc0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c007e60 tx=0x7f4d1c00e4b0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.148+0000 7f4d2a5b6640 1 -- 172.21.15.76:0/133707415 shutdown_connections 2023-11-09T03:55:12.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.148+0000 7f4d2a5b6640 1 --2- 172.21.15.76:0/133707415 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4d1006ae70 0x7f4d1006d330 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.170 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.148+0000 7f4d2a5b6640 1 --2- 172.21.15.76:0/133707415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d2415d4e0 0x7f4d240cbbc0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.148+0000 7f4d2a5b6640 1 --2- 172.21.15.76:0/133707415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d2415d100 0x7f4d240b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.148+0000 7f4d2a5b6640 1 --2- 172.21.15.76:0/133707415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d240d3b60 0x7f4d2414bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.148+0000 7f4d2a5b6640 1 -- 172.21.15.76:0/133707415 >> 172.21.15.76:0/133707415 conn(0x7f4d240d2ef0 msgr2=0x7f4d240b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:12.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.148+0000 7f4d2a5b6640 1 -- 172.21.15.76:0/133707415 shutdown_connections 2023-11-09T03:55:12.171 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.148+0000 7f4d2a5b6640 1 -- 172.21.15.76:0/133707415 wait complete. 2023-11-09T03:55:12.172 INFO:tasks.cephfs.test_snapshots:testing snapclient mds_kill_mdstable_at=7 2023-11-09T03:55:12.172 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:55:12.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.273+0000 7f37b6c90640 1 Processor -- start 2023-11-09T03:55:12.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.273+0000 7f37b6c90640 1 -- start start 2023-11-09T03:55:12.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.274+0000 7f37b6c90640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f37b00b0c40 0x7f37b00ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.274+0000 7f37b6c90640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37b00c9100 con 0x7f37b01605b0 2023-11-09T03:55:12.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.274+0000 7f37b6c90640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37b00ae230 con 0x7f37b006b950 2023-11-09T03:55:12.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.274+0000 7f37b6c90640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37b00aceb0 con 0x7f37b00b0c40 2023-11-09T03:55:12.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.274+0000 7f37b4a2c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f37b01605b0 0x7f37b00b0530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45762/0 (socket says 172.21.15.76:45762) 2023-11-09T03:55:12.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.274+0000 7f37b4a2c640 1 -- 172.21.15.76:0/1256537663 learned_addr learned my addr 172.21.15.76:0/1256537663 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:55:12.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.274+0000 7f37af7fe640 1 -- 172.21.15.76:0/1256537663 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 508741027 0 0) 0x7f37b00c9100 con 0x7f37b01605b0 2023-11-09T03:55:12.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.274+0000 7f37af7fe640 1 -- 172.21.15.76:0/1256537663 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37a0003520 con 0x7f37b01605b0 2023-11-09T03:55:12.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.274+0000 7f37affff640 1 --2- 172.21.15.76:0/1256537663 >> v2:172.21.15.113:3300/0 conn(0x7f37b00b0c40 0x7f37b00ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.274+0000 7f37af7fe640 1 -- 172.21.15.76:0/1256537663 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 95688962 0 0) 0x7f37a0003520 con 0x7f37b01605b0 2023-11-09T03:55:12.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.274+0000 7f37af7fe640 1 -- 172.21.15.76:0/1256537663 >> v2:172.21.15.113:3300/0 conn(0x7f37b00b0c40 msgr2=0x7f37b00ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:12.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.274+0000 7f37af7fe640 1 --2- 172.21.15.76:0/1256537663 >> v2:172.21.15.113:3300/0 conn(0x7f37b00b0c40 0x7f37b00ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.274+0000 7f37af7fe640 1 -- 172.21.15.76:0/1256537663 >> v1:172.21.15.113:6789/0 conn(0x7f37b006b950 legacy=0x7f37b00c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:12.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.274+0000 7f37af7fe640 1 -- 172.21.15.76:0/1256537663 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b00ad190 con 0x7f37b01605b0 2023-11-09T03:55:12.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.274+0000 7f37af7fe640 1 -- 172.21.15.76:0/1256537663 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f37a4001670 con 0x7f37b01605b0 2023-11-09T03:55:12.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.274+0000 7f37af7fe640 1 -- 172.21.15.76:0/1256537663 >> v1:172.21.15.76:6789/0 conn(0x7f37b01605b0 legacy=0x7f37b00b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:12.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.274+0000 7f37af7fe640 1 --2- 172.21.15.76:0/1256537663 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37a0003970 0x7f37a0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.274+0000 7f37af7fe640 1 --2- 172.21.15.76:0/1256537663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37a00047c0 0x7f37a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.274+0000 7f37af7fe640 1 --2- 172.21.15.76:0/1256537663 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37a0005180 0x7f37a000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.274+0000 7f37af7fe640 1 -- 172.21.15.76:0/1256537663 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b00aceb0 con 0x7f37a0005180 2023-11-09T03:55:12.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.274+0000 7f37af7fe640 1 -- 172.21.15.76:0/1256537663 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a000c120 con 0x7f37a00047c0 2023-11-09T03:55:12.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.274+0000 7f37af7fe640 1 -- 172.21.15.76:0/1256537663 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a000c2e0 con 0x7f37a0003970 2023-11-09T03:55:12.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.274+0000 7f37b522d640 1 --2- 172.21.15.76:0/1256537663 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37a0005180 0x7f37a000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.275+0000 7f37b4a2c640 1 --2- 172.21.15.76:0/1256537663 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37a0003970 0x7f37a0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.275+0000 7f37affff640 1 --2- 172.21.15.76:0/1256537663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37a00047c0 0x7f37a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.275+0000 7f37b522d640 1 -- 172.21.15.76:0/1256537663 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37a0003970 msgr2=0x7f37a0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:12.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.275+0000 7f37b522d640 1 --2- 172.21.15.76:0/1256537663 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37a0003970 0x7f37a0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.275+0000 7f37b522d640 1 -- 172.21.15.76:0/1256537663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37a00047c0 msgr2=0x7f37a0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:12.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.275+0000 7f37b522d640 1 --2- 172.21.15.76:0/1256537663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37a00047c0 0x7f37a0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.275+0000 7f37b522d640 1 -- 172.21.15.76:0/1256537663 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a0003520 con 0x7f37a0005180 2023-11-09T03:55:12.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.275+0000 7f37b522d640 1 -- 172.21.15.76:0/1256537663 reap_dead start 2023-11-09T03:55:12.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.275+0000 7f37b522d640 1 --2- 172.21.15.76:0/1256537663 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37a0005180 0x7f37a000b980 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f37a8001e20 tx=0x7f37a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb58d3c51c39ad3 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:12.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.276+0000 7f37af7fe640 1 -- 172.21.15.76:0/1256537663 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a8003b70 con 0x7f37a0005180 2023-11-09T03:55:12.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.276+0000 7f37af7fe640 1 -- 172.21.15.76:0/1256537663 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a8003cd0 con 0x7f37a0005180 2023-11-09T03:55:12.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.276+0000 7f37af7fe640 1 -- 172.21.15.76:0/1256537663 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a8005310 con 0x7f37a0005180 2023-11-09T03:55:12.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.276+0000 7f37b6c90640 1 -- 172.21.15.76:0/1256537663 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37a0005180 msgr2=0x7f37a000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:12.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.276+0000 7f37b6c90640 1 --2- 172.21.15.76:0/1256537663 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37a0005180 0x7f37a000b980 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f37a8001e20 tx=0x7f37a00057f0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.276+0000 7f37b6c90640 1 -- 172.21.15.76:0/1256537663 shutdown_connections 2023-11-09T03:55:12.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.276+0000 7f37b6c90640 1 --2- 172.21.15.76:0/1256537663 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37a0005180 0x7f37a000b980 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.276+0000 7f37b6c90640 1 -- 172.21.15.76:0/1256537663 >> 172.21.15.76:0/1256537663 conn(0x7f37b00cb530 msgr2=0x7f37b00cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:12.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.276+0000 7f37b6c90640 1 -- 172.21.15.76:0/1256537663 shutdown_connections 2023-11-09T03:55:12.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.276+0000 7f37b6c90640 1 -- 172.21.15.76:0/1256537663 wait complete. 2023-11-09T03:55:12.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.277+0000 7f37b6c90640 1 Processor -- start 2023-11-09T03:55:12.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.277+0000 7f37b6c90640 1 -- start start 2023-11-09T03:55:12.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.278+0000 7f37b6c90640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37a0005180 0x7f37b00c4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.278+0000 7f37b6c90640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37b00c2580 0x7f37b015d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.278+0000 7f37b6c90640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37b015d580 0x7f37b0151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.278+0000 7f37b6c90640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b00b17e0 con 0x7f37b015d580 2023-11-09T03:55:12.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.278+0000 7f37b6c90640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b00b1ac0 con 0x7f37b00c2580 2023-11-09T03:55:12.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.278+0000 7f37b6c90640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b00b1da0 con 0x7f37a0005180 2023-11-09T03:55:12.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.278+0000 7f37b522d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37b015d580 0x7f37b0151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.278+0000 7f37b522d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37b015d580 0x7f37b0151aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47656/0 (socket says 172.21.15.76:47656) 2023-11-09T03:55:12.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.278+0000 7f37b522d640 1 -- 172.21.15.76:0/263785238 learned_addr learned my addr 172.21.15.76:0/263785238 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:12.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.279+0000 7f37b4a2c640 1 --2- 172.21.15.76:0/263785238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37a0005180 0x7f37b00c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.279+0000 7f37affff640 1 --2- 172.21.15.76:0/263785238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37b00c2580 0x7f37b015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.279+0000 7f37b522d640 1 -- 172.21.15.76:0/263785238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37a0005180 msgr2=0x7f37b00c4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:12.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.279+0000 7f37b522d640 1 --2- 172.21.15.76:0/263785238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37a0005180 0x7f37b00c4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.279+0000 7f37b522d640 1 -- 172.21.15.76:0/263785238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37b00c2580 msgr2=0x7f37b015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:12.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.279+0000 7f37b522d640 1 --2- 172.21.15.76:0/263785238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37b00c2580 0x7f37b015d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.279+0000 7f37b522d640 1 -- 172.21.15.76:0/263785238 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b00b2080 con 0x7f37b015d580 2023-11-09T03:55:12.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.279+0000 7f37affff640 1 --2- 172.21.15.76:0/263785238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37b00c2580 0x7f37b015d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:55:12.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.279+0000 7f37b522d640 1 --2- 172.21.15.76:0/263785238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37b015d580 0x7f37b0151aa0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f37a8001f40 tx=0x7f37a80014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:12.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.279+0000 7f37ad7fa640 1 -- 172.21.15.76:0/263785238 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a8006d50 con 0x7f37b015d580 2023-11-09T03:55:12.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.279+0000 7f37ad7fa640 1 -- 172.21.15.76:0/263785238 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a8007030 con 0x7f37b015d580 2023-11-09T03:55:12.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.279+0000 7f37b6c90640 1 -- 172.21.15.76:0/263785238 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b00b2360 con 0x7f37b015d580 2023-11-09T03:55:12.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.279+0000 7f37ad7fa640 1 -- 172.21.15.76:0/263785238 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a80073c0 con 0x7f37b015d580 2023-11-09T03:55:12.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.279+0000 7f37b6c90640 1 -- 172.21.15.76:0/263785238 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b00b4500 con 0x7f37b015d580 2023-11-09T03:55:12.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.280+0000 7f37b6c90640 1 -- 172.21.15.76:0/263785238 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37b00b2550 con 0x7f37b015d580 2023-11-09T03:55:12.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.280+0000 7f37ad7fa640 1 -- 172.21.15.76:0/263785238 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f37a8007520 con 0x7f37b015d580 2023-11-09T03:55:12.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.281+0000 7f37ad7fa640 1 --2- 172.21.15.76:0/263785238 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f379406ae70 0x7f379406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.281+0000 7f37ad7fa640 1 -- 172.21.15.76:0/263785238 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7f37a80abb30 con 0x7f37b015d580 2023-11-09T03:55:12.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.281+0000 7f37b4a2c640 1 --2- 172.21.15.76:0/263785238 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f379406ae70 0x7f379406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.282+0000 7f37b4a2c640 1 --2- 172.21.15.76:0/263785238 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f379406ae70 0x7f379406d330 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f37a4000f60 tx=0x7f37a4000ef0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:12.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.285+0000 7f37ad7fa640 1 -- 172.21.15.76:0/263785238 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f37a8076200 con 0x7f37b015d580 2023-11-09T03:55:12.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.487+0000 7f37b6c90640 1 -- 172.21.15.76:0/263785238 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37b00ad440 con 0x7f37b015d580 2023-11-09T03:55:12.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.487+0000 7f37ad7fa640 1 -- 172.21.15.76:0/263785238 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+4932 (secure 0 0 0) 0x7f37a8047090 con 0x7f37b015d580 2023-11-09T03:55:12.487 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:55:12.487 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":133,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6698,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":133,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:58.523123+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":9024},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9024":{"gid":9024,"name":"a","rank":1,"incarnation":129,"state":"up:active","state_seq":16,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:55:12.488 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 133 2023-11-09T03:55:12.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.489+0000 7f37b6c90640 1 -- 172.21.15.76:0/263785238 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f379406ae70 msgr2=0x7f379406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:12.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.489+0000 7f37b6c90640 1 --2- 172.21.15.76:0/263785238 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f379406ae70 0x7f379406d330 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f37a4000f60 tx=0x7f37a4000ef0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.489+0000 7f37b6c90640 1 -- 172.21.15.76:0/263785238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37b015d580 msgr2=0x7f37b0151aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:12.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.489+0000 7f37b6c90640 1 --2- 172.21.15.76:0/263785238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37b015d580 0x7f37b0151aa0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f37a8001f40 tx=0x7f37a80014a0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.489+0000 7f37b6c90640 1 -- 172.21.15.76:0/263785238 shutdown_connections 2023-11-09T03:55:12.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.489+0000 7f37b6c90640 1 --2- 172.21.15.76:0/263785238 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f379406ae70 0x7f379406d330 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.489+0000 7f37b6c90640 1 --2- 172.21.15.76:0/263785238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37b015d580 0x7f37b0151aa0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.490 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.489+0000 7f37b6c90640 1 --2- 172.21.15.76:0/263785238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37b00c2580 0x7f37b015d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.489+0000 7f37b6c90640 1 --2- 172.21.15.76:0/263785238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37a0005180 0x7f37b00c4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.489+0000 7f37b6c90640 1 -- 172.21.15.76:0/263785238 >> 172.21.15.76:0/263785238 conn(0x7f37b00cb530 msgr2=0x7f37b00cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:12.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.489+0000 7f37b6c90640 1 -- 172.21.15.76:0/263785238 shutdown_connections 2023-11-09T03:55:12.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.489+0000 7f37b6c90640 1 -- 172.21.15.76:0/263785238 wait complete. 2023-11-09T03:55:12.500 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2023-11-09T03:55:12.615 INFO:teuthology.orchestra.run.smithi076.stdout:{"last_snap":16,"last_created":16,"last_destroyed":14,"pending_noop":[],"snaps":[{"snapid":16,"ino":1099511627777,"stamp":"2023-11-09T03:54:27.826603+0000","name":"s34","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-09T03:55:12.615 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 16, "last_destroyed": 14, "last_snap": 16, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s34", "snapid": 16, "stamp": "2023-11-09T03:54:27.826603+0000" } ] } 2023-11-09T03:55:12.616 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:55:12.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.716+0000 7f2bfb7cc640 1 Processor -- start 2023-11-09T03:55:12.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.716+0000 7f2bfb7cc640 1 -- start start 2023-11-09T03:55:12.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.717+0000 7f2bfb7cc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bf4159e40 0x7f2bf40b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.717+0000 7f2bfb7cc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bf4071500 con 0x7f2bf4070700 2023-11-09T03:55:12.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.717+0000 7f2bfb7cc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bf415dae0 con 0x7f2bf406bb60 2023-11-09T03:55:12.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.717+0000 7f2bfb7cc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf4070c60 con 0x7f2bf4159e40 2023-11-09T03:55:12.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.717+0000 7f2bf9d69640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2bf4070700 0x7f2bf40cd790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47904/0 (socket says 172.21.15.76:47904) 2023-11-09T03:55:12.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.717+0000 7f2bf9d69640 1 -- 172.21.15.76:0/1262386844 learned_addr learned my addr 172.21.15.76:0/1262386844 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:55:12.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.717+0000 7f2bf9568640 1 --2- 172.21.15.76:0/1262386844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bf4159e40 0x7f2bf40b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.717+0000 7f2bebfff640 1 -- 172.21.15.76:0/1262386844 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3071636181 0 0) 0x7f2bf4071500 con 0x7f2bf4070700 2023-11-09T03:55:12.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.717+0000 7f2bebfff640 1 -- 172.21.15.76:0/1262386844 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2be0003520 con 0x7f2bf4070700 2023-11-09T03:55:12.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.717+0000 7f2bebfff640 1 -- 172.21.15.76:0/1262386844 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1745294662 0 0) 0x7f2bf415dae0 con 0x7f2bf406bb60 2023-11-09T03:55:12.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.717+0000 7f2bebfff640 1 -- 172.21.15.76:0/1262386844 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bf4071500 con 0x7f2bf406bb60 2023-11-09T03:55:12.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.717+0000 7f2bf9568640 1 -- 172.21.15.76:0/1262386844 >> v1:172.21.15.113:6789/0 conn(0x7f2bf406bb60 legacy=0x7f2bf406fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:12.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.717+0000 7f2bf9568640 1 -- 172.21.15.76:0/1262386844 >> v1:172.21.15.76:6789/0 conn(0x7f2bf4070700 legacy=0x7f2bf40cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:12.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.717+0000 7f2bf9568640 1 -- 172.21.15.76:0/1262386844 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf4070f40 con 0x7f2bf4159e40 2023-11-09T03:55:12.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.718+0000 7f2bf9568640 1 --2- 172.21.15.76:0/1262386844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bf4159e40 0x7f2bf40b0530 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00097d0 tx=0x7f2bf002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8f912850c7cc815 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:12.722 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.718+0000 7f2bebfff640 1 -- 172.21.15.76:0/1262386844 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0031800 con 0x7f2bf4159e40 2023-11-09T03:55:12.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.718+0000 7f2bebfff640 1 -- 172.21.15.76:0/1262386844 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf0031960 con 0x7f2bf4159e40 2023-11-09T03:55:12.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.718+0000 7f2bebfff640 1 -- 172.21.15.76:0/1262386844 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0031cf0 con 0x7f2bf4159e40 2023-11-09T03:55:12.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.719+0000 7f2bfb7cc640 1 -- 172.21.15.76:0/1262386844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bf4159e40 msgr2=0x7f2bf40b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:12.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.719+0000 7f2bfb7cc640 1 --2- 172.21.15.76:0/1262386844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bf4159e40 0x7f2bf40b0530 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00097d0 tx=0x7f2bf002b8a0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.723 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.719+0000 7f2bfb7cc640 1 -- 172.21.15.76:0/1262386844 shutdown_connections 2023-11-09T03:55:12.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.719+0000 7f2bfb7cc640 1 --2- 172.21.15.76:0/1262386844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bf4159e40 0x7f2bf40b0530 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.719+0000 7f2bfb7cc640 1 -- 172.21.15.76:0/1262386844 >> 172.21.15.76:0/1262386844 conn(0x7f2bf40cb530 msgr2=0x7f2bf40cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:12.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.719+0000 7f2bfb7cc640 1 -- 172.21.15.76:0/1262386844 shutdown_connections 2023-11-09T03:55:12.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.719+0000 7f2bfb7cc640 1 -- 172.21.15.76:0/1262386844 wait complete. 2023-11-09T03:55:12.724 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.720+0000 7f2bfb7cc640 1 Processor -- start 2023-11-09T03:55:12.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.720+0000 7f2bfb7cc640 1 -- start start 2023-11-09T03:55:12.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.721+0000 7f2bfb7cc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bf406bb60 0x7f2bf414e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.721+0000 7f2bfb7cc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bf4070700 0x7f2bf4150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.721+0000 7f2bfb7cc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bf4159e40 0x7f2bf414f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.725 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.721+0000 7f2bfb7cc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf40c19d0 con 0x7f2bf4159e40 2023-11-09T03:55:12.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.721+0000 7f2bfb7cc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf4151210 con 0x7f2bf406bb60 2023-11-09T03:55:12.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.721+0000 7f2bfb7cc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf414f850 con 0x7f2bf4070700 2023-11-09T03:55:12.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.721+0000 7f2bf9d69640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bf4159e40 0x7f2bf414f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.721+0000 7f2bf9d69640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bf4159e40 0x7f2bf414f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57282/0 (socket says 172.21.15.76:57282) 2023-11-09T03:55:12.726 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.721+0000 7f2bf9d69640 1 -- 172.21.15.76:0/3615788247 learned_addr learned my addr 172.21.15.76:0/3615788247 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:12.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.721+0000 7f2bf8d67640 1 --2- 172.21.15.76:0/3615788247 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bf4070700 0x7f2bf4150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.721+0000 7f2bf9568640 1 --2- 172.21.15.76:0/3615788247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bf406bb60 0x7f2bf414e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.721+0000 7f2bf9d69640 1 -- 172.21.15.76:0/3615788247 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bf4070700 msgr2=0x7f2bf4150cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:12.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.721+0000 7f2bf9d69640 1 --2- 172.21.15.76:0/3615788247 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bf4070700 0x7f2bf4150cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.721+0000 7f2bf9d69640 1 -- 172.21.15.76:0/3615788247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bf406bb60 msgr2=0x7f2bf414e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:12.727 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.721+0000 7f2bf9d69640 1 --2- 172.21.15.76:0/3615788247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bf406bb60 0x7f2bf414e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.721+0000 7f2bf9d69640 1 -- 172.21.15.76:0/3615788247 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf0009400 con 0x7f2bf4159e40 2023-11-09T03:55:12.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.721+0000 7f2bf9d69640 1 --2- 172.21.15.76:0/3615788247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bf4159e40 0x7f2bf414f310 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00cb70 tx=0x7f2bec00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:12.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.721+0000 7f2be9ffb640 1 -- 172.21.15.76:0/3615788247 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec00ace0 con 0x7f2bf4159e40 2023-11-09T03:55:12.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.721+0000 7f2be9ffb640 1 -- 172.21.15.76:0/3615788247 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bec00ae40 con 0x7f2bf4159e40 2023-11-09T03:55:12.728 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.721+0000 7f2bfb7cc640 1 -- 172.21.15.76:0/3615788247 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bf41514b0 con 0x7f2bf4159e40 2023-11-09T03:55:12.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.721+0000 7f2be9ffb640 1 -- 172.21.15.76:0/3615788247 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec012620 con 0x7f2bf4159e40 2023-11-09T03:55:12.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.722+0000 7f2bfb7cc640 1 -- 172.21.15.76:0/3615788247 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf4151a70 con 0x7f2bf4159e40 2023-11-09T03:55:12.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.723+0000 7f2bfb7cc640 1 -- 172.21.15.76:0/3615788247 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bf414dfd0 con 0x7f2bf4159e40 2023-11-09T03:55:12.729 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.723+0000 7f2be9ffb640 1 -- 172.21.15.76:0/3615788247 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2bec007070 con 0x7f2bf4159e40 2023-11-09T03:55:12.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.723+0000 7f2be9ffb640 1 --2- 172.21.15.76:0/3615788247 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2bd806aea0 0x7f2bd806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:12.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.723+0000 7f2be9ffb640 1 -- 172.21.15.76:0/3615788247 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7f2bec08a0b0 con 0x7f2bf4159e40 2023-11-09T03:55:12.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.724+0000 7f2bf9568640 1 --2- 172.21.15.76:0/3615788247 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2bd806aea0 0x7f2bd806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:12.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.724+0000 7f2bf9568640 1 --2- 172.21.15.76:0/3615788247 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2bd806aea0 0x7f2bd806d360 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0036000 tx=0x7f2bf003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:12.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.727+0000 7f2be9ffb640 1 -- 172.21.15.76:0/3615788247 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2bec058100 con 0x7f2bf4159e40 2023-11-09T03:55:12.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.928+0000 7f2bfb7cc640 1 -- 172.21.15.76:0/3615788247 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2bf40684b0 con 0x7f2bf4159e40 2023-11-09T03:55:12.928 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.929+0000 7f2be9ffb640 1 -- 172.21.15.76:0/3615788247 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+4932 (secure 0 0 0) 0x7f2bec017020 con 0x7f2bf4159e40 2023-11-09T03:55:12.929 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:55:12.929 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":133,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6698,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":133,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:54:58.523123+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":9024},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9024":{"gid":9024,"name":"a","rank":1,"incarnation":129,"state":"up:active","state_seq":16,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:55:12.929 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 133 2023-11-09T03:55:12.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.931+0000 7f2bfb7cc640 1 -- 172.21.15.76:0/3615788247 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2bd806aea0 msgr2=0x7f2bd806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:12.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.931+0000 7f2bfb7cc640 1 --2- 172.21.15.76:0/3615788247 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2bd806aea0 0x7f2bd806d360 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0036000 tx=0x7f2bf003c040 comp rx=0 tx=0).stop 2023-11-09T03:55:12.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.931+0000 7f2bfb7cc640 1 -- 172.21.15.76:0/3615788247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bf4159e40 msgr2=0x7f2bf414f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:12.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.931+0000 7f2bfb7cc640 1 --2- 172.21.15.76:0/3615788247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bf4159e40 0x7f2bf414f310 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00cb70 tx=0x7f2bec00a430 comp rx=0 tx=0).stop 2023-11-09T03:55:12.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.931+0000 7f2bfb7cc640 1 -- 172.21.15.76:0/3615788247 shutdown_connections 2023-11-09T03:55:12.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.931+0000 7f2bfb7cc640 1 --2- 172.21.15.76:0/3615788247 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2bd806aea0 0x7f2bd806d360 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.931+0000 7f2bfb7cc640 1 --2- 172.21.15.76:0/3615788247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bf4159e40 0x7f2bf414f310 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.931+0000 7f2bfb7cc640 1 --2- 172.21.15.76:0/3615788247 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bf4070700 0x7f2bf4150cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.931+0000 7f2bfb7cc640 1 --2- 172.21.15.76:0/3615788247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bf406bb60 0x7f2bf414e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:12.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.931+0000 7f2bfb7cc640 1 -- 172.21.15.76:0/3615788247 >> 172.21.15.76:0/3615788247 conn(0x7f2bf40cb530 msgr2=0x7f2bf40cf300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:12.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.931+0000 7f2bfb7cc640 1 -- 172.21.15.76:0/3615788247 shutdown_connections 2023-11-09T03:55:12.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:12.931+0000 7f2bfb7cc640 1 -- 172.21.15.76:0/3615788247 wait complete. 2023-11-09T03:55:12.941 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:1 true 2023-11-09T03:55:13.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.044+0000 7fbede72c640 1 Processor -- start 2023-11-09T03:55:13.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.044+0000 7fbede72c640 1 -- start start 2023-11-09T03:55:13.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.045+0000 7fbede72c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbed80b0c40 0x7fbed806fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:13.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.045+0000 7fbede72c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbed806bb60 0x7fbed8070530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:13.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.045+0000 7fbede72c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed8071500 con 0x7fbed8159e40 2023-11-09T03:55:13.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.045+0000 7fbede72c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed8070a70 con 0x7fbed806bb60 2023-11-09T03:55:13.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.045+0000 7fbede72c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed815dae0 con 0x7fbed80b0c40 2023-11-09T03:55:13.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.045+0000 7fbedccc9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbed806bb60 0x7fbed8070530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:13.046 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.045+0000 7fbedccc9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbed806bb60 0x7fbed8070530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57294/0 (socket says 172.21.15.76:57294) 2023-11-09T03:55:13.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.045+0000 7fbedccc9640 1 -- 172.21.15.76:0/2612530753 learned_addr learned my addr 172.21.15.76:0/2612530753 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:13.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.045+0000 7fbedccc9640 1 -- 172.21.15.76:0/2612530753 >> v2:172.21.15.113:3300/0 conn(0x7fbed80b0c40 msgr2=0x7fbed806fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:13.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.045+0000 7fbedccc9640 1 --2- 172.21.15.76:0/2612530753 >> v2:172.21.15.113:3300/0 conn(0x7fbed80b0c40 0x7fbed806fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:13.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.045+0000 7fbedccc9640 1 -- 172.21.15.76:0/2612530753 >> v1:172.21.15.113:6789/0 conn(0x7fbed8159e40 legacy=0x7fbed80b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:13.047 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.045+0000 7fbedccc9640 1 -- 172.21.15.76:0/2612530753 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed8070c60 con 0x7fbed806bb60 2023-11-09T03:55:13.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.046+0000 7fbedccc9640 1 --2- 172.21.15.76:0/2612530753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbed806bb60 0x7fbed8070530 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fbed806fdc0 tx=0x7fbed002e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5357f5f61a7c93c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:13.051 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.046+0000 7fbed6ffd640 1 -- 172.21.15.76:0/2612530753 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed002ea00 con 0x7fbed806bb60 2023-11-09T03:55:13.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.046+0000 7fbed6ffd640 1 -- 172.21.15.76:0/2612530753 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed002eb60 con 0x7fbed806bb60 2023-11-09T03:55:13.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.046+0000 7fbed6ffd640 1 -- 172.21.15.76:0/2612530753 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed0009600 con 0x7fbed806bb60 2023-11-09T03:55:13.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.047+0000 7fbede72c640 1 -- 172.21.15.76:0/2612530753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbed806bb60 msgr2=0x7fbed8070530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:13.052 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.047+0000 7fbede72c640 1 --2- 172.21.15.76:0/2612530753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbed806bb60 0x7fbed8070530 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fbed806fdc0 tx=0x7fbed002e060 comp rx=0 tx=0).stop 2023-11-09T03:55:13.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.047+0000 7fbede72c640 1 -- 172.21.15.76:0/2612530753 shutdown_connections 2023-11-09T03:55:13.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.047+0000 7fbede72c640 1 --2- 172.21.15.76:0/2612530753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbed806bb60 0x7fbed8070530 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:13.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.047+0000 7fbede72c640 1 --2- 172.21.15.76:0/2612530753 >> v2:172.21.15.113:3300/0 conn(0x7fbed80b0c40 0x7fbed806fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:13.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.047+0000 7fbede72c640 1 -- 172.21.15.76:0/2612530753 >> 172.21.15.76:0/2612530753 conn(0x7fbed80cb530 msgr2=0x7fbed80cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:13.053 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.047+0000 7fbede72c640 1 -- 172.21.15.76:0/2612530753 shutdown_connections 2023-11-09T03:55:13.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.047+0000 7fbede72c640 1 -- 172.21.15.76:0/2612530753 wait complete. 2023-11-09T03:55:13.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.048+0000 7fbede72c640 1 Processor -- start 2023-11-09T03:55:13.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.048+0000 7fbede72c640 1 -- start start 2023-11-09T03:55:13.054 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.049+0000 7fbede72c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbed806bb60 0x7fbed814e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:13.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.049+0000 7fbede72c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbed80b0c40 0x7fbed815b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:13.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.049+0000 7fbede72c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbed8159e40 0x7fbed815bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:13.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.049+0000 7fbede72c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed81ee430 con 0x7fbed8159e40 2023-11-09T03:55:13.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.049+0000 7fbede72c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed80c78f0 con 0x7fbed80b0c40 2023-11-09T03:55:13.055 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.049+0000 7fbede72c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed815a3c0 con 0x7fbed806bb60 2023-11-09T03:55:13.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.049+0000 7fbedccc9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbed8159e40 0x7fbed815bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:13.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.049+0000 7fbedccc9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbed8159e40 0x7fbed815bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57302/0 (socket says 172.21.15.76:57302) 2023-11-09T03:55:13.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.049+0000 7fbedccc9640 1 -- 172.21.15.76:0/3836529496 learned_addr learned my addr 172.21.15.76:0/3836529496 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:13.056 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.049+0000 7fbed77fe640 1 --2- 172.21.15.76:0/3836529496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbed80b0c40 0x7fbed815b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:13.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.049+0000 7fbedccc9640 1 -- 172.21.15.76:0/3836529496 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbed806bb60 msgr2=0x7fbed814e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:13.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.049+0000 7fbed7fff640 1 --2- 172.21.15.76:0/3836529496 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbed806bb60 0x7fbed814e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:13.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.049+0000 7fbedccc9640 1 --2- 172.21.15.76:0/3836529496 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbed806bb60 0x7fbed814e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:13.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.049+0000 7fbedccc9640 1 -- 172.21.15.76:0/3836529496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbed80b0c40 msgr2=0x7fbed815b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:13.057 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.049+0000 7fbedccc9640 1 --2- 172.21.15.76:0/3836529496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbed80b0c40 0x7fbed815b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:13.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.049+0000 7fbedccc9640 1 -- 172.21.15.76:0/3836529496 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed000bd40 con 0x7fbed8159e40 2023-11-09T03:55:13.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.049+0000 7fbed77fe640 1 --2- 172.21.15.76:0/3836529496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbed80b0c40 0x7fbed815b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:55:13.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.049+0000 7fbedccc9640 1 --2- 172.21.15.76:0/3836529496 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbed8159e40 0x7fbed815bad0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fbed0038040 tx=0x7fbed0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:13.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.049+0000 7fbed4ff9640 1 -- 172.21.15.76:0/3836529496 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed000b720 con 0x7fbed8159e40 2023-11-09T03:55:13.058 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.049+0000 7fbed4ff9640 1 -- 172.21.15.76:0/3836529496 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed0040660 con 0x7fbed8159e40 2023-11-09T03:55:13.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.049+0000 7fbede72c640 1 -- 172.21.15.76:0/3836529496 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed815a6a0 con 0x7fbed8159e40 2023-11-09T03:55:13.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.049+0000 7fbed4ff9640 1 -- 172.21.15.76:0/3836529496 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed00409f0 con 0x7fbed8159e40 2023-11-09T03:55:13.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.049+0000 7fbede72c640 1 -- 172.21.15.76:0/3836529496 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed815ac60 con 0x7fbed8159e40 2023-11-09T03:55:13.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.051+0000 7fbede72c640 1 -- 172.21.15.76:0/3836529496 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed815ae80 con 0x7fbed8159e40 2023-11-09T03:55:13.059 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.051+0000 7fbed4ff9640 1 -- 172.21.15.76:0/3836529496 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbed0040b50 con 0x7fbed8159e40 2023-11-09T03:55:13.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.051+0000 7fbed4ff9640 1 --2- 172.21.15.76:0/3836529496 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbec406ae70 0x7fbec406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:13.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.052+0000 7fbed4ff9640 1 -- 172.21.15.76:0/3836529496 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7fbed00af5b0 con 0x7fbed8159e40 2023-11-09T03:55:13.060 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.052+0000 7fbed7fff640 1 --2- 172.21.15.76:0/3836529496 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbec406ae70 0x7fbec406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:13.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.052+0000 7fbed7fff640 1 --2- 172.21.15.76:0/3836529496 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbec406ae70 0x7fbec406d330 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fbec800a730 tx=0x7fbec8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:13.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.056+0000 7fbed4ff9640 1 -- 172.21.15.76:0/3836529496 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbed0079c10 con 0x7fbed8159e40 2023-11-09T03:55:13.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.287+0000 7fbede72c640 1 -- 172.21.15.76:0/3836529496 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"} v 0) v1 -- 0x7fbed80684b0 con 0x7fbed8159e40 2023-11-09T03:55:13.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.567+0000 7fbed4ff9640 1 -- 172.21.15.76:0/3836529496 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"}]=0 freezing mds.9024 v134) v1 ==== 112+0+0 (secure 0 0 0) 0x7fbed007d230 con 0x7fbed8159e40 2023-11-09T03:55:13.568 INFO:teuthology.orchestra.run.smithi076.stderr:freezing mds.9024 2023-11-09T03:55:13.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.569+0000 7fbede72c640 1 -- 172.21.15.76:0/3836529496 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbec406ae70 msgr2=0x7fbec406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:13.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.569+0000 7fbede72c640 1 --2- 172.21.15.76:0/3836529496 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbec406ae70 0x7fbec406d330 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fbec800a730 tx=0x7fbec8008040 comp rx=0 tx=0).stop 2023-11-09T03:55:13.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.569+0000 7fbede72c640 1 -- 172.21.15.76:0/3836529496 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbed8159e40 msgr2=0x7fbed815bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:13.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.569+0000 7fbede72c640 1 --2- 172.21.15.76:0/3836529496 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbed8159e40 0x7fbed815bad0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fbed0038040 tx=0x7fbed0004a40 comp rx=0 tx=0).stop 2023-11-09T03:55:13.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.570+0000 7fbede72c640 1 -- 172.21.15.76:0/3836529496 shutdown_connections 2023-11-09T03:55:13.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.570+0000 7fbede72c640 1 --2- 172.21.15.76:0/3836529496 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fbec406ae70 0x7fbec406d330 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:13.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.570+0000 7fbede72c640 1 --2- 172.21.15.76:0/3836529496 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbed8159e40 0x7fbed815bad0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:13.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.570+0000 7fbede72c640 1 --2- 172.21.15.76:0/3836529496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbed80b0c40 0x7fbed815b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:13.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.570+0000 7fbede72c640 1 --2- 172.21.15.76:0/3836529496 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbed806bb60 0x7fbed814e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:13.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.570+0000 7fbede72c640 1 -- 172.21.15.76:0/3836529496 >> 172.21.15.76:0/3836529496 conn(0x7fbed80cb530 msgr2=0x7fbed80cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:13.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.570+0000 7fbede72c640 1 -- 172.21.15.76:0/3836529496 shutdown_connections 2023-11-09T03:55:13.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.570+0000 7fbede72c640 1 -- 172.21.15.76:0/3836529496 wait complete. 2023-11-09T03:55:13.581 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_kill_mdstable_at 7 2023-11-09T03:55:13.695 INFO:teuthology.orchestra.run.smithi076.stdout:{"success":"mds_kill_mdstable_at = '7' (not observed, change may require restart) "} 2023-11-09T03:55:13.695 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '7' (not observed, change may require restart) " } 2023-11-09T03:55:13.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:55:13.696 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s37) 2023-11-09T03:55:13.698 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2023-11-09T03:55:13.822 INFO:teuthology.orchestra.run.smithi076.stdout:{"mds_beacon_grace":"15.000000"} 2023-11-09T03:55:13.822 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2023-11-09T03:55:13.823 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:55:13.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.922+0000 7ff017db0640 1 Processor -- start 2023-11-09T03:55:13.923 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.923+0000 7ff017db0640 1 -- start start 2023-11-09T03:55:13.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.923+0000 7ff017db0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff01015d100 0x7ff0100ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:13.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.923+0000 7ff017db0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff0100d1be0 0x7ff0100b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:13.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.923+0000 7ff017db0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0100b32b0 con 0x7ff01015d4e0 2023-11-09T03:55:13.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.923+0000 7ff017db0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0100cee10 con 0x7ff0100d1be0 2023-11-09T03:55:13.925 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.923+0000 7ff017db0640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0100b2980 con 0x7ff01015d100 2023-11-09T03:55:13.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.924+0000 7ff01634d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff0100d1be0 0x7ff0100b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:13.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.924+0000 7ff01634d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff0100d1be0 0x7ff0100b3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57306/0 (socket says 172.21.15.76:57306) 2023-11-09T03:55:13.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.924+0000 7ff01634d640 1 -- 172.21.15.76:0/376239846 learned_addr learned my addr 172.21.15.76:0/376239846 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:13.926 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.924+0000 7ff015b4c640 1 --2- 172.21.15.76:0/376239846 >> v2:172.21.15.113:3300/0 conn(0x7ff01015d100 0x7ff0100ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:13.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.924+0000 7ff01634d640 1 -- 172.21.15.76:0/376239846 >> v2:172.21.15.113:3300/0 conn(0x7ff01015d100 msgr2=0x7ff0100ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:13.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.924+0000 7ff01634d640 1 --2- 172.21.15.76:0/376239846 >> v2:172.21.15.113:3300/0 conn(0x7ff01015d100 0x7ff0100ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:13.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.924+0000 7ff01634d640 1 -- 172.21.15.76:0/376239846 >> v1:172.21.15.113:6789/0 conn(0x7ff01015d4e0 legacy=0x7ff0100d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:13.927 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.924+0000 7ff01634d640 1 -- 172.21.15.76:0/376239846 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0100b2c60 con 0x7ff0100d1be0 2023-11-09T03:55:13.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.925+0000 7ff01634d640 1 --2- 172.21.15.76:0/376239846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0100d1be0 0x7ff0100b3740 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7ff0100d09f0 tx=0x7ff00802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd790e3d574af988 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:13.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.925+0000 7ff014b4a640 1 -- 172.21.15.76:0/376239846 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00802ca80 con 0x7ff0100d1be0 2023-11-09T03:55:13.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.925+0000 7ff014b4a640 1 -- 172.21.15.76:0/376239846 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00802cbe0 con 0x7ff0100d1be0 2023-11-09T03:55:13.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.925+0000 7ff014b4a640 1 -- 172.21.15.76:0/376239846 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0080343f0 con 0x7ff0100d1be0 2023-11-09T03:55:13.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.925+0000 7ff017db0640 1 -- 172.21.15.76:0/376239846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0100d1be0 msgr2=0x7ff0100b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:13.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.925+0000 7ff017db0640 1 --2- 172.21.15.76:0/376239846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0100d1be0 0x7ff0100b3740 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7ff0100d09f0 tx=0x7ff00802bee0 comp rx=0 tx=0).stop 2023-11-09T03:55:13.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.925+0000 7ff017db0640 1 -- 172.21.15.76:0/376239846 shutdown_connections 2023-11-09T03:55:13.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.925+0000 7ff017db0640 1 --2- 172.21.15.76:0/376239846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0100d1be0 0x7ff0100b3740 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:13.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.925+0000 7ff017db0640 1 --2- 172.21.15.76:0/376239846 >> v2:172.21.15.113:3300/0 conn(0x7ff01015d100 0x7ff0100ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:13.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.925+0000 7ff017db0640 1 -- 172.21.15.76:0/376239846 >> 172.21.15.76:0/376239846 conn(0x7ff0100d2ef0 msgr2=0x7ff0100d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:13.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.926+0000 7ff017db0640 1 -- 172.21.15.76:0/376239846 shutdown_connections 2023-11-09T03:55:13.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.926+0000 7ff017db0640 1 -- 172.21.15.76:0/376239846 wait complete. 2023-11-09T03:55:13.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.927+0000 7ff017db0640 1 Processor -- start 2023-11-09T03:55:13.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.927+0000 7ff017db0640 1 -- start start 2023-11-09T03:55:13.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.927+0000 7ff017db0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0100d1be0 0x7ff01015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:13.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.927+0000 7ff017db0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff01015d100 0x7ff0100ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:13.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.927+0000 7ff017db0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff01015d4e0 0x7ff0100b7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:13.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.927+0000 7ff017db0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0100b5a70 con 0x7ff01015d100 2023-11-09T03:55:13.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.927+0000 7ff017db0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0100cb220 con 0x7ff0100d1be0 2023-11-09T03:55:13.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.927+0000 7ff017db0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0100cb500 con 0x7ff01015d4e0 2023-11-09T03:55:13.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.927+0000 7ff01534b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff01015d100 0x7ff0100ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:13.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.927+0000 7ff01534b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff01015d100 0x7ff0100ca9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57316/0 (socket says 172.21.15.76:57316) 2023-11-09T03:55:13.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.927+0000 7ff01534b640 1 -- 172.21.15.76:0/1032864857 learned_addr learned my addr 172.21.15.76:0/1032864857 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:13.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.928+0000 7ff015b4c640 1 --2- 172.21.15.76:0/1032864857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0100d1be0 0x7ff01015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:13.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.928+0000 7ff01634d640 1 --2- 172.21.15.76:0/1032864857 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff01015d4e0 0x7ff0100b7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:13.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.928+0000 7ff01534b640 1 -- 172.21.15.76:0/1032864857 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff01015d4e0 msgr2=0x7ff0100b7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:13.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.928+0000 7ff01534b640 1 --2- 172.21.15.76:0/1032864857 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff01015d4e0 0x7ff0100b7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:13.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.928+0000 7ff01534b640 1 -- 172.21.15.76:0/1032864857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0100d1be0 msgr2=0x7ff01015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:13.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.928+0000 7ff01534b640 1 --2- 172.21.15.76:0/1032864857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0100d1be0 0x7ff01015b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:13.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.928+0000 7ff01534b640 1 -- 172.21.15.76:0/1032864857 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff008009b30 con 0x7ff01015d100 2023-11-09T03:55:13.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.928+0000 7ff01534b640 1 --2- 172.21.15.76:0/1032864857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff01015d100 0x7ff0100ca9d0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00c400 tx=0x7ff00c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:13.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.928+0000 7ff0067fc640 1 -- 172.21.15.76:0/1032864857 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00c011790 con 0x7ff01015d100 2023-11-09T03:55:13.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.928+0000 7ff0067fc640 1 -- 172.21.15.76:0/1032864857 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00c011930 con 0x7ff01015d100 2023-11-09T03:55:13.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.928+0000 7ff017db0640 1 -- 172.21.15.76:0/1032864857 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0100cb7e0 con 0x7ff01015d100 2023-11-09T03:55:13.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.928+0000 7ff0067fc640 1 -- 172.21.15.76:0/1032864857 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00c011ce0 con 0x7ff01015d100 2023-11-09T03:55:13.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.928+0000 7ff017db0640 1 -- 172.21.15.76:0/1032864857 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0100caf10 con 0x7ff01015d100 2023-11-09T03:55:13.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.929+0000 7ff017db0640 1 -- 172.21.15.76:0/1032864857 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0100684b0 con 0x7ff01015d100 2023-11-09T03:55:13.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.929+0000 7ff0067fc640 1 -- 172.21.15.76:0/1032864857 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff00c01c460 con 0x7ff01015d100 2023-11-09T03:55:13.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.930+0000 7ff0067fc640 1 --2- 172.21.15.76:0/1032864857 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7feff406ae70 0x7feff406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:13.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.930+0000 7ff0067fc640 1 -- 172.21.15.76:0/1032864857 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7ff00c08a070 con 0x7ff01015d100 2023-11-09T03:55:13.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.930+0000 7ff015b4c640 1 --2- 172.21.15.76:0/1032864857 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7feff406ae70 0x7feff406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:13.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.931+0000 7ff015b4c640 1 --2- 172.21.15.76:0/1032864857 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7feff406ae70 0x7feff406d330 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff00000b9b0 tx=0x7ff000008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:13.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:13.934+0000 7ff0067fc640 1 -- 172.21.15.76:0/1032864857 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff00c054630 con 0x7ff01015d100 2023-11-09T03:55:14.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:14.134+0000 7ff017db0640 1 -- 172.21.15.76:0/1032864857 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0100d2cd0 con 0x7ff01015d100 2023-11-09T03:55:14.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:14.135+0000 7ff0067fc640 1 -- 172.21.15.76:0/1032864857 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 134 v134) v1 ==== 95+0+4932 (secure 0 0 0) 0x7ff00c057c50 con 0x7ff01015d100 2023-11-09T03:55:14.135 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:55:14.136 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":134,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6698,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":134,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:55:13.289237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":9024},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9024":{"gid":9024,"name":"a","rank":1,"incarnation":129,"state":"up:active","state_seq":16,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:55:14.136 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 134 2023-11-09T03:55:14.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:14.136+0000 7ff017db0640 1 -- 172.21.15.76:0/1032864857 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7feff406ae70 msgr2=0x7feff406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:14.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:14.136+0000 7ff017db0640 1 --2- 172.21.15.76:0/1032864857 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7feff406ae70 0x7feff406d330 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff00000b9b0 tx=0x7ff000008040 comp rx=0 tx=0).stop 2023-11-09T03:55:14.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:14.136+0000 7ff017db0640 1 -- 172.21.15.76:0/1032864857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff01015d100 msgr2=0x7ff0100ca9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:14.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:14.136+0000 7ff017db0640 1 --2- 172.21.15.76:0/1032864857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff01015d100 0x7ff0100ca9d0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00c400 tx=0x7ff00c00c8c0 comp rx=0 tx=0).stop 2023-11-09T03:55:14.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:14.136+0000 7ff017db0640 1 -- 172.21.15.76:0/1032864857 shutdown_connections 2023-11-09T03:55:14.138 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:14.136+0000 7ff017db0640 1 --2- 172.21.15.76:0/1032864857 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7feff406ae70 0x7feff406d330 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:14.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:14.136+0000 7ff017db0640 1 --2- 172.21.15.76:0/1032864857 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff01015d4e0 0x7ff0100b7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:14.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:14.136+0000 7ff017db0640 1 --2- 172.21.15.76:0/1032864857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff01015d100 0x7ff0100ca9d0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:14.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:14.136+0000 7ff017db0640 1 --2- 172.21.15.76:0/1032864857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0100d1be0 0x7ff01015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:14.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:14.136+0000 7ff017db0640 1 -- 172.21.15.76:0/1032864857 >> 172.21.15.76:0/1032864857 conn(0x7ff0100d2ef0 msgr2=0x7ff0100b1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:14.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:14.137+0000 7ff017db0640 1 -- 172.21.15.76:0/1032864857 shutdown_connections 2023-11-09T03:55:14.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:14.137+0000 7ff017db0640 1 -- 172.21.15.76:0/1032864857 wait complete. 2023-11-09T03:55:14.148 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:55:14.318 INFO:tasks.ceph.mds.a.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::handle_request(ceph::cref_t&)' thread 7f68af120640 time 2023-11-09T03:55:14.319061+0000 2023-11-09T03:55:14.318 INFO:tasks.ceph.mds.a.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: 113: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 7) 2023-11-09T03:55:14.319 INFO:tasks.ceph.mds.a.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:55:14.319 INFO:tasks.ceph.mds.a.smithi076.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f68b4d6efaf] 2023-11-09T03:55:14.319 INFO:tasks.ceph.mds.a.smithi076.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f68b4d6f16d] 2023-11-09T03:55:14.320 INFO:tasks.ceph.mds.a.smithi076.stderr: 3: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x11ab) [0x5607507b71db] 2023-11-09T03:55:14.320 INFO:tasks.ceph.mds.a.smithi076.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xd55) [0x560750544d15] 2023-11-09T03:55:14.320 INFO:tasks.ceph.mds.a.smithi076.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x560750546b2b] 2023-11-09T03:55:14.320 INFO:tasks.ceph.mds.a.smithi076.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56075054701c] 2023-11-09T03:55:14.320 INFO:tasks.ceph.mds.a.smithi076.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x560750530325] 2023-11-09T03:55:14.320 INFO:tasks.ceph.mds.a.smithi076.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7f68b4f5ff9a] 2023-11-09T03:55:14.321 INFO:tasks.ceph.mds.a.smithi076.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f68b4ff1741] 2023-11-09T03:55:14.321 INFO:tasks.ceph.mds.a.smithi076.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7f68b449f802] 2023-11-09T03:55:14.321 INFO:tasks.ceph.mds.a.smithi076.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7f68b443f450] 2023-11-09T03:55:14.321 INFO:tasks.ceph.mds.a.smithi076.stderr:2023-11-09T03:55:14.318+0000 7f68af120640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::handle_request(ceph::cref_t&)' thread 7f68af120640 time 2023-11-09T03:55:14.319061+0000 2023-11-09T03:55:14.321 INFO:tasks.ceph.mds.a.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: 113: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 7) 2023-11-09T03:55:14.322 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:55:14.322 INFO:tasks.ceph.mds.a.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:55:14.322 INFO:tasks.ceph.mds.a.smithi076.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f68b4d6efaf] 2023-11-09T03:55:14.322 INFO:tasks.ceph.mds.a.smithi076.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f68b4d6f16d] 2023-11-09T03:55:14.322 INFO:tasks.ceph.mds.a.smithi076.stderr: 3: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x11ab) [0x5607507b71db] 2023-11-09T03:55:14.323 INFO:tasks.ceph.mds.a.smithi076.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xd55) [0x560750544d15] 2023-11-09T03:55:14.323 INFO:tasks.ceph.mds.a.smithi076.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x560750546b2b] 2023-11-09T03:55:14.323 INFO:tasks.ceph.mds.a.smithi076.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56075054701c] 2023-11-09T03:55:14.323 INFO:tasks.ceph.mds.a.smithi076.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x560750530325] 2023-11-09T03:55:14.323 INFO:tasks.ceph.mds.a.smithi076.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7f68b4f5ff9a] 2023-11-09T03:55:14.324 INFO:tasks.ceph.mds.a.smithi076.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f68b4ff1741] 2023-11-09T03:55:14.324 INFO:tasks.ceph.mds.a.smithi076.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7f68b449f802] 2023-11-09T03:55:14.324 INFO:tasks.ceph.mds.a.smithi076.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7f68b443f450] 2023-11-09T03:55:14.324 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:55:14.324 INFO:tasks.ceph.mds.a.smithi076.stderr:*** Caught signal (Aborted) ** 2023-11-09T03:55:14.325 INFO:tasks.ceph.mds.a.smithi076.stderr: in thread 7f68af120640 thread_name:ms_dispatch 2023-11-09T03:55:14.325 INFO:tasks.ceph.mds.a.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:55:14.325 INFO:tasks.ceph.mds.a.smithi076.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f68b4454db0] 2023-11-09T03:55:14.325 INFO:tasks.ceph.mds.a.smithi076.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f68b44a154c] 2023-11-09T03:55:14.325 INFO:tasks.ceph.mds.a.smithi076.stderr: 3: raise() 2023-11-09T03:55:14.326 INFO:tasks.ceph.mds.a.smithi076.stderr: 4: abort() 2023-11-09T03:55:14.326 INFO:tasks.ceph.mds.a.smithi076.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f68b4d6f009] 2023-11-09T03:55:14.326 INFO:tasks.ceph.mds.a.smithi076.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f68b4d6f16d] 2023-11-09T03:55:14.326 INFO:tasks.ceph.mds.a.smithi076.stderr: 7: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x11ab) [0x5607507b71db] 2023-11-09T03:55:14.326 INFO:tasks.ceph.mds.a.smithi076.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xd55) [0x560750544d15] 2023-11-09T03:55:14.327 INFO:tasks.ceph.mds.a.smithi076.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x560750546b2b] 2023-11-09T03:55:14.327 INFO:tasks.ceph.mds.a.smithi076.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56075054701c] 2023-11-09T03:55:14.327 INFO:tasks.ceph.mds.a.smithi076.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x560750530325] 2023-11-09T03:55:14.327 INFO:tasks.ceph.mds.a.smithi076.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7f68b4f5ff9a] 2023-11-09T03:55:14.327 INFO:tasks.ceph.mds.a.smithi076.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f68b4ff1741] 2023-11-09T03:55:14.328 INFO:tasks.ceph.mds.a.smithi076.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7f68b449f802] 2023-11-09T03:55:14.328 INFO:tasks.ceph.mds.a.smithi076.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7f68b443f450] 2023-11-09T03:55:14.328 INFO:tasks.ceph.mds.a.smithi076.stderr:2023-11-09T03:55:14.318+0000 7f68af120640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:55:14.328 INFO:tasks.ceph.mds.a.smithi076.stderr: in thread 7f68af120640 thread_name:ms_dispatch 2023-11-09T03:55:14.328 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:55:14.329 INFO:tasks.ceph.mds.a.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:55:14.329 INFO:tasks.ceph.mds.a.smithi076.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f68b4454db0] 2023-11-09T03:55:14.329 INFO:tasks.ceph.mds.a.smithi076.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f68b44a154c] 2023-11-09T03:55:14.329 INFO:tasks.ceph.mds.a.smithi076.stderr: 3: raise() 2023-11-09T03:55:14.329 INFO:tasks.ceph.mds.a.smithi076.stderr: 4: abort() 2023-11-09T03:55:14.330 INFO:tasks.ceph.mds.a.smithi076.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f68b4d6f009] 2023-11-09T03:55:14.330 INFO:tasks.ceph.mds.a.smithi076.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f68b4d6f16d] 2023-11-09T03:55:14.330 INFO:tasks.ceph.mds.a.smithi076.stderr: 7: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x11ab) [0x5607507b71db] 2023-11-09T03:55:14.330 INFO:tasks.ceph.mds.a.smithi076.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xd55) [0x560750544d15] 2023-11-09T03:55:14.330 INFO:tasks.ceph.mds.a.smithi076.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x560750546b2b] 2023-11-09T03:55:14.331 INFO:tasks.ceph.mds.a.smithi076.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56075054701c] 2023-11-09T03:55:14.331 INFO:tasks.ceph.mds.a.smithi076.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x560750530325] 2023-11-09T03:55:14.331 INFO:tasks.ceph.mds.a.smithi076.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7f68b4f5ff9a] 2023-11-09T03:55:14.331 INFO:tasks.ceph.mds.a.smithi076.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f68b4ff1741] 2023-11-09T03:55:14.331 INFO:tasks.ceph.mds.a.smithi076.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7f68b449f802] 2023-11-09T03:55:14.332 INFO:tasks.ceph.mds.a.smithi076.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7f68b443f450] 2023-11-09T03:55:14.332 INFO:tasks.ceph.mds.a.smithi076.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:55:14.332 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:55:14.333 INFO:tasks.ceph.mds.a.smithi076.stderr: -1> 2023-11-09T03:55:14.318+0000 7f68af120640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::handle_request(ceph::cref_t&)' thread 7f68af120640 time 2023-11-09T03:55:14.319061+0000 2023-11-09T03:55:14.333 INFO:tasks.ceph.mds.a.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: 113: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 7) 2023-11-09T03:55:14.333 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:55:14.333 INFO:tasks.ceph.mds.a.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:55:14.333 INFO:tasks.ceph.mds.a.smithi076.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f68b4d6efaf] 2023-11-09T03:55:14.334 INFO:tasks.ceph.mds.a.smithi076.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f68b4d6f16d] 2023-11-09T03:55:14.334 INFO:tasks.ceph.mds.a.smithi076.stderr: 3: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x11ab) [0x5607507b71db] 2023-11-09T03:55:14.334 INFO:tasks.ceph.mds.a.smithi076.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xd55) [0x560750544d15] 2023-11-09T03:55:14.334 INFO:tasks.ceph.mds.a.smithi076.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x560750546b2b] 2023-11-09T03:55:14.334 INFO:tasks.ceph.mds.a.smithi076.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56075054701c] 2023-11-09T03:55:14.335 INFO:tasks.ceph.mds.a.smithi076.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x560750530325] 2023-11-09T03:55:14.335 INFO:tasks.ceph.mds.a.smithi076.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7f68b4f5ff9a] 2023-11-09T03:55:14.335 INFO:tasks.ceph.mds.a.smithi076.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f68b4ff1741] 2023-11-09T03:55:14.335 INFO:tasks.ceph.mds.a.smithi076.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7f68b449f802] 2023-11-09T03:55:14.335 INFO:tasks.ceph.mds.a.smithi076.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7f68b443f450] 2023-11-09T03:55:14.336 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:55:14.336 INFO:tasks.ceph.mds.a.smithi076.stderr: 0> 2023-11-09T03:55:14.318+0000 7f68af120640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:55:14.336 INFO:tasks.ceph.mds.a.smithi076.stderr: in thread 7f68af120640 thread_name:ms_dispatch 2023-11-09T03:55:14.336 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:55:14.336 INFO:tasks.ceph.mds.a.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:55:14.337 INFO:tasks.ceph.mds.a.smithi076.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f68b4454db0] 2023-11-09T03:55:14.337 INFO:tasks.ceph.mds.a.smithi076.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f68b44a154c] 2023-11-09T03:55:14.337 INFO:tasks.ceph.mds.a.smithi076.stderr: 3: raise() 2023-11-09T03:55:14.337 INFO:tasks.ceph.mds.a.smithi076.stderr: 4: abort() 2023-11-09T03:55:14.337 INFO:tasks.ceph.mds.a.smithi076.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f68b4d6f009] 2023-11-09T03:55:14.337 INFO:tasks.ceph.mds.a.smithi076.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f68b4d6f16d] 2023-11-09T03:55:14.338 INFO:tasks.ceph.mds.a.smithi076.stderr: 7: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x11ab) [0x5607507b71db] 2023-11-09T03:55:14.338 INFO:tasks.ceph.mds.a.smithi076.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xd55) [0x560750544d15] 2023-11-09T03:55:14.338 INFO:tasks.ceph.mds.a.smithi076.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x560750546b2b] 2023-11-09T03:55:14.338 INFO:tasks.ceph.mds.a.smithi076.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56075054701c] 2023-11-09T03:55:14.338 INFO:tasks.ceph.mds.a.smithi076.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x560750530325] 2023-11-09T03:55:14.339 INFO:tasks.ceph.mds.a.smithi076.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7f68b4f5ff9a] 2023-11-09T03:55:14.339 INFO:tasks.ceph.mds.a.smithi076.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f68b4ff1741] 2023-11-09T03:55:14.339 INFO:tasks.ceph.mds.a.smithi076.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7f68b449f802] 2023-11-09T03:55:14.339 INFO:tasks.ceph.mds.a.smithi076.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7f68b443f450] 2023-11-09T03:55:14.339 INFO:tasks.ceph.mds.a.smithi076.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:55:14.339 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:55:14.340 INFO:tasks.ceph.mds.a.smithi076.stderr: -1> 2023-11-09T03:55:14.318+0000 7f68af120640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::handle_request(ceph::cref_t&)' thread 7f68af120640 time 2023-11-09T03:55:14.319061+0000 2023-11-09T03:55:14.340 INFO:tasks.ceph.mds.a.smithi076.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: 113: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 7) 2023-11-09T03:55:14.340 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:55:14.340 INFO:tasks.ceph.mds.a.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:55:14.340 INFO:tasks.ceph.mds.a.smithi076.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f68b4d6efaf] 2023-11-09T03:55:14.341 INFO:tasks.ceph.mds.a.smithi076.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f68b4d6f16d] 2023-11-09T03:55:14.341 INFO:tasks.ceph.mds.a.smithi076.stderr: 3: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x11ab) [0x5607507b71db] 2023-11-09T03:55:14.341 INFO:tasks.ceph.mds.a.smithi076.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xd55) [0x560750544d15] 2023-11-09T03:55:14.341 INFO:tasks.ceph.mds.a.smithi076.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x560750546b2b] 2023-11-09T03:55:14.341 INFO:tasks.ceph.mds.a.smithi076.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56075054701c] 2023-11-09T03:55:14.342 INFO:tasks.ceph.mds.a.smithi076.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x560750530325] 2023-11-09T03:55:14.342 INFO:tasks.ceph.mds.a.smithi076.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7f68b4f5ff9a] 2023-11-09T03:55:14.342 INFO:tasks.ceph.mds.a.smithi076.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f68b4ff1741] 2023-11-09T03:55:14.342 INFO:tasks.ceph.mds.a.smithi076.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7f68b449f802] 2023-11-09T03:55:14.342 INFO:tasks.ceph.mds.a.smithi076.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7f68b443f450] 2023-11-09T03:55:14.342 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:55:14.343 INFO:tasks.ceph.mds.a.smithi076.stderr: 0> 2023-11-09T03:55:14.318+0000 7f68af120640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:55:14.343 INFO:tasks.ceph.mds.a.smithi076.stderr: in thread 7f68af120640 thread_name:ms_dispatch 2023-11-09T03:55:14.343 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:55:14.343 INFO:tasks.ceph.mds.a.smithi076.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:55:14.343 INFO:tasks.ceph.mds.a.smithi076.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f68b4454db0] 2023-11-09T03:55:14.344 INFO:tasks.ceph.mds.a.smithi076.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f68b44a154c] 2023-11-09T03:55:14.344 INFO:tasks.ceph.mds.a.smithi076.stderr: 3: raise() 2023-11-09T03:55:14.344 INFO:tasks.ceph.mds.a.smithi076.stderr: 4: abort() 2023-11-09T03:55:14.344 INFO:tasks.ceph.mds.a.smithi076.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f68b4d6f009] 2023-11-09T03:55:14.344 INFO:tasks.ceph.mds.a.smithi076.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f68b4d6f16d] 2023-11-09T03:55:14.345 INFO:tasks.ceph.mds.a.smithi076.stderr: 7: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x11ab) [0x5607507b71db] 2023-11-09T03:55:14.345 INFO:tasks.ceph.mds.a.smithi076.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xd55) [0x560750544d15] 2023-11-09T03:55:14.345 INFO:tasks.ceph.mds.a.smithi076.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x560750546b2b] 2023-11-09T03:55:14.345 INFO:tasks.ceph.mds.a.smithi076.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56075054701c] 2023-11-09T03:55:14.345 INFO:tasks.ceph.mds.a.smithi076.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x560750530325] 2023-11-09T03:55:14.346 INFO:tasks.ceph.mds.a.smithi076.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7f68b4f5ff9a] 2023-11-09T03:55:14.346 INFO:tasks.ceph.mds.a.smithi076.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f68b4ff1741] 2023-11-09T03:55:14.346 INFO:tasks.ceph.mds.a.smithi076.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7f68b449f802] 2023-11-09T03:55:14.346 INFO:tasks.ceph.mds.a.smithi076.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7f68b443f450] 2023-11-09T03:55:14.346 INFO:tasks.ceph.mds.a.smithi076.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:55:14.347 INFO:tasks.ceph.mds.a.smithi076.stderr: 2023-11-09T03:55:14.535 INFO:tasks.ceph.mds.a.smithi076.stderr:daemon-helper: command crashed with signal 6 2023-11-09T03:55:18.783 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-11-09T03:55:19.149 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:55:19.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.262+0000 7fb74f8e0640 1 Processor -- start 2023-11-09T03:55:19.263 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.262+0000 7fb74f8e0640 1 -- start start 2023-11-09T03:55:19.264 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.263+0000 7fb74f8e0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb748159e40 0x7fb7480b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:19.264 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.263+0000 7fb74f8e0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb748070700 0x7fb7480b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:19.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.263+0000 7fb74f8e0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb748071500 con 0x7fb74806bb60 2023-11-09T03:55:19.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.263+0000 7fb74f8e0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb74815dae0 con 0x7fb748070700 2023-11-09T03:55:19.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.263+0000 7fb74f8e0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb748070c60 con 0x7fb748159e40 2023-11-09T03:55:19.265 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.263+0000 7fb74de7d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb748070700 0x7fb7480b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:19.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.263+0000 7fb74de7d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb748070700 0x7fb7480b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57322/0 (socket says 172.21.15.76:57322) 2023-11-09T03:55:19.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.263+0000 7fb74de7d640 1 -- 172.21.15.76:0/3809604987 learned_addr learned my addr 172.21.15.76:0/3809604987 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:19.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.264+0000 7fb74d67c640 1 --2- 172.21.15.76:0/3809604987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb748159e40 0x7fb7480b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:19.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.264+0000 7fb74de7d640 1 -- 172.21.15.76:0/3809604987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb748159e40 msgr2=0x7fb7480b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:19.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.264+0000 7fb74de7d640 1 --2- 172.21.15.76:0/3809604987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb748159e40 0x7fb7480b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:19.266 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.264+0000 7fb74de7d640 1 -- 172.21.15.76:0/3809604987 >> v1:172.21.15.113:6789/0 conn(0x7fb74806bb60 legacy=0x7fb74806fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:19.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.264+0000 7fb74de7d640 1 -- 172.21.15.76:0/3809604987 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb748070f40 con 0x7fb748070700 2023-11-09T03:55:19.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.265+0000 7fb74de7d640 1 --2- 172.21.15.76:0/3809604987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb748070700 0x7fb7480b0a70 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fb740009870 tx=0x7fb74002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9939752931928336 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:19.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.265+0000 7fb73ffff640 1 -- 172.21.15.76:0/3809604987 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb740031640 con 0x7fb748070700 2023-11-09T03:55:19.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.265+0000 7fb73ffff640 1 -- 172.21.15.76:0/3809604987 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7400317a0 con 0x7fb748070700 2023-11-09T03:55:19.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.265+0000 7fb73ffff640 1 -- 172.21.15.76:0/3809604987 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb740031b30 con 0x7fb748070700 2023-11-09T03:55:19.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.265+0000 7fb74f8e0640 1 -- 172.21.15.76:0/3809604987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb748070700 msgr2=0x7fb7480b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:19.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.265+0000 7fb74f8e0640 1 --2- 172.21.15.76:0/3809604987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb748070700 0x7fb7480b0a70 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fb740009870 tx=0x7fb74002b8f0 comp rx=0 tx=0).stop 2023-11-09T03:55:19.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.265+0000 7fb74f8e0640 1 -- 172.21.15.76:0/3809604987 shutdown_connections 2023-11-09T03:55:19.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.265+0000 7fb74f8e0640 1 --2- 172.21.15.76:0/3809604987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb748070700 0x7fb7480b0a70 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:19.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.265+0000 7fb74f8e0640 1 --2- 172.21.15.76:0/3809604987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb748159e40 0x7fb7480b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:19.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.265+0000 7fb74f8e0640 1 -- 172.21.15.76:0/3809604987 >> 172.21.15.76:0/3809604987 conn(0x7fb7480cb530 msgr2=0x7fb7480cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:19.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.265+0000 7fb74f8e0640 1 -- 172.21.15.76:0/3809604987 shutdown_connections 2023-11-09T03:55:19.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.265+0000 7fb74f8e0640 1 -- 172.21.15.76:0/3809604987 wait complete. 2023-11-09T03:55:19.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.267+0000 7fb74f8e0640 1 Processor -- start 2023-11-09T03:55:19.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.267+0000 7fb74f8e0640 1 -- start start 2023-11-09T03:55:19.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.267+0000 7fb74f8e0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb74806bb60 0x7fb7480c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:19.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.267+0000 7fb74f8e0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb748070700 0x7fb7481548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:19.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.267+0000 7fb74f8e0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb748159e40 0x7fb748157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:19.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.267+0000 7fb74f8e0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7480c49b0 con 0x7fb748159e40 2023-11-09T03:55:19.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.267+0000 7fb74f8e0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb748157c60 con 0x7fb74806bb60 2023-11-09T03:55:19.273 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.267+0000 7fb74f8e0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7480c40c0 con 0x7fb748070700 2023-11-09T03:55:19.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.268+0000 7fb74de7d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb748159e40 0x7fb748157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:19.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.268+0000 7fb74de7d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb748159e40 0x7fb748157720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57326/0 (socket says 172.21.15.76:57326) 2023-11-09T03:55:19.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.268+0000 7fb74de7d640 1 -- 172.21.15.76:0/3421151467 learned_addr learned my addr 172.21.15.76:0/3421151467 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:19.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.268+0000 7fb74d67c640 1 --2- 172.21.15.76:0/3421151467 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb74806bb60 0x7fb7480c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:19.274 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.268+0000 7fb74ce7b640 1 --2- 172.21.15.76:0/3421151467 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb748070700 0x7fb7481548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:19.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.268+0000 7fb74de7d640 1 -- 172.21.15.76:0/3421151467 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb748070700 msgr2=0x7fb7481548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:19.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.268+0000 7fb74de7d640 1 --2- 172.21.15.76:0/3421151467 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb748070700 0x7fb7481548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:19.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.268+0000 7fb74de7d640 1 -- 172.21.15.76:0/3421151467 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb74806bb60 msgr2=0x7fb7480c87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:19.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.268+0000 7fb74de7d640 1 --2- 172.21.15.76:0/3421151467 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb74806bb60 0x7fb7480c87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:19.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.268+0000 7fb74de7d640 1 -- 172.21.15.76:0/3421151467 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7400094d0 con 0x7fb748159e40 2023-11-09T03:55:19.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.268+0000 7fb74ce7b640 1 --2- 172.21.15.76:0/3421151467 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb748070700 0x7fb7481548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:55:19.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.268+0000 7fb74de7d640 1 --2- 172.21.15.76:0/3421151467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb748159e40 0x7fb748157720 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fb74002b930 tx=0x7fb7400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:19.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.268+0000 7fb73dffb640 1 -- 172.21.15.76:0/3421151467 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74003ac60 con 0x7fb748159e40 2023-11-09T03:55:19.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.268+0000 7fb73dffb640 1 -- 172.21.15.76:0/3421151467 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb74003adc0 con 0x7fb748159e40 2023-11-09T03:55:19.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.268+0000 7fb74f8e0640 1 -- 172.21.15.76:0/3421151467 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7480c4360 con 0x7fb748159e40 2023-11-09T03:55:19.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.268+0000 7fb73dffb640 1 -- 172.21.15.76:0/3421151467 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb740044630 con 0x7fb748159e40 2023-11-09T03:55:19.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.269+0000 7fb74f8e0640 1 -- 172.21.15.76:0/3421151467 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb748154fe0 con 0x7fb748159e40 2023-11-09T03:55:19.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.270+0000 7fb74f8e0640 1 -- 172.21.15.76:0/3421151467 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7480c4b20 con 0x7fb748159e40 2023-11-09T03:55:19.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.270+0000 7fb73dffb640 1 -- 172.21.15.76:0/3421151467 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb74002f050 con 0x7fb748159e40 2023-11-09T03:55:19.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.270+0000 7fb73dffb640 1 --2- 172.21.15.76:0/3421151467 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb72c06ae70 0x7fb72c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:19.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.270+0000 7fb73dffb640 1 -- 172.21.15.76:0/3421151467 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7fb7400abee0 con 0x7fb748159e40 2023-11-09T03:55:19.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.271+0000 7fb74d67c640 1 --2- 172.21.15.76:0/3421151467 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb72c06ae70 0x7fb72c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:19.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.271+0000 7fb74d67c640 1 --2- 172.21.15.76:0/3421151467 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb72c06ae70 0x7fb72c06d330 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fb744003aa0 tx=0x7fb744009830 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:19.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.275+0000 7fb73dffb640 1 -- 172.21.15.76:0/3421151467 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb7400764a0 con 0x7fb748159e40 2023-11-09T03:55:19.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.475+0000 7fb74f8e0640 1 -- 172.21.15.76:0/3421151467 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7480684b0 con 0x7fb748159e40 2023-11-09T03:55:19.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.475+0000 7fb73dffb640 1 -- 172.21.15.76:0/3421151467 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 134 v134) v1 ==== 95+0+4932 (secure 0 0 0) 0x7fb740031e60 con 0x7fb748159e40 2023-11-09T03:55:19.476 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:55:19.476 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":134,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6698,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":134,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:55:13.289237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":9024},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9024":{"gid":9024,"name":"a","rank":1,"incarnation":129,"state":"up:active","state_seq":16,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:55:19.477 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 134 2023-11-09T03:55:19.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.477+0000 7fb74f8e0640 1 -- 172.21.15.76:0/3421151467 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb72c06ae70 msgr2=0x7fb72c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:19.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.477+0000 7fb74f8e0640 1 --2- 172.21.15.76:0/3421151467 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb72c06ae70 0x7fb72c06d330 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fb744003aa0 tx=0x7fb744009830 comp rx=0 tx=0).stop 2023-11-09T03:55:19.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.477+0000 7fb74f8e0640 1 -- 172.21.15.76:0/3421151467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb748159e40 msgr2=0x7fb748157720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:19.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.477+0000 7fb74f8e0640 1 --2- 172.21.15.76:0/3421151467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb748159e40 0x7fb748157720 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fb74002b930 tx=0x7fb7400046c0 comp rx=0 tx=0).stop 2023-11-09T03:55:19.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.477+0000 7fb74f8e0640 1 -- 172.21.15.76:0/3421151467 shutdown_connections 2023-11-09T03:55:19.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.477+0000 7fb74f8e0640 1 --2- 172.21.15.76:0/3421151467 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fb72c06ae70 0x7fb72c06d330 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:19.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.477+0000 7fb74f8e0640 1 --2- 172.21.15.76:0/3421151467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb748159e40 0x7fb748157720 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:19.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.477+0000 7fb74f8e0640 1 --2- 172.21.15.76:0/3421151467 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb748070700 0x7fb7481548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:19.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.477+0000 7fb74f8e0640 1 --2- 172.21.15.76:0/3421151467 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb74806bb60 0x7fb7480c87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:19.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.477+0000 7fb74f8e0640 1 -- 172.21.15.76:0/3421151467 >> 172.21.15.76:0/3421151467 conn(0x7fb7480cb530 msgr2=0x7fb7480cf2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:19.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.477+0000 7fb74f8e0640 1 -- 172.21.15.76:0/3421151467 shutdown_connections 2023-11-09T03:55:19.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:19.477+0000 7fb74f8e0640 1 -- 172.21.15.76:0/3421151467 wait complete. 2023-11-09T03:55:19.489 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:55:24.490 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:55:24.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.599+0000 7fe9e36af640 1 Processor -- start 2023-11-09T03:55:24.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.599+0000 7fe9e36af640 1 -- start start 2023-11-09T03:55:24.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.600+0000 7fe9e36af640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe9dc0b0c40 0x7fe9dc06fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:24.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.600+0000 7fe9e36af640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9dc06bb60 0x7fe9dc070530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:24.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.600+0000 7fe9e36af640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9dc071500 con 0x7fe9dc159e40 2023-11-09T03:55:24.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.600+0000 7fe9e36af640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9dc070a70 con 0x7fe9dc0b0c40 2023-11-09T03:55:24.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.600+0000 7fe9e36af640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc15dae0 con 0x7fe9dc06bb60 2023-11-09T03:55:24.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.600+0000 7fe9e1c4c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9dc06bb60 0x7fe9dc070530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:24.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.600+0000 7fe9e0c4a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe9dc0b0c40 0x7fe9dc06fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:24.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.600+0000 7fe9e144b640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe9dc159e40 0x7fe9dc0b0530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:35806/0 (socket says 172.21.15.76:35806) 2023-11-09T03:55:24.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.600+0000 7fe9e144b640 1 -- 172.21.15.76:0/1113504809 learned_addr learned my addr 172.21.15.76:0/1113504809 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:55:24.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.600+0000 7fe9e1c4c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9dc06bb60 0x7fe9dc070530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:49788/0 (socket says 172.21.15.76:49788) 2023-11-09T03:55:24.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.600+0000 7fe9e0c4a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe9dc0b0c40 0x7fe9dc06fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:54170/0 (socket says 172.21.15.76:54170) 2023-11-09T03:55:24.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.601+0000 7fe9e1c4c640 1 -- 172.21.15.76:0/1113504809 >> v2:172.21.15.113:3300/0 conn(0x7fe9dc0b0c40 msgr2=0x7fe9dc06fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:24.607 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.601+0000 7fe9e1c4c640 1 --2- 172.21.15.76:0/1113504809 >> v2:172.21.15.113:3300/0 conn(0x7fe9dc0b0c40 0x7fe9dc06fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:24.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.601+0000 7fe9e1c4c640 1 -- 172.21.15.76:0/1113504809 >> v1:172.21.15.113:6789/0 conn(0x7fe9dc159e40 legacy=0x7fe9dc0b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:24.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.601+0000 7fe9e1c4c640 1 -- 172.21.15.76:0/1113504809 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9dc070c60 con 0x7fe9dc06bb60 2023-11-09T03:55:24.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.601+0000 7fe9d3fff640 1 -- 172.21.15.76:0/1113504809 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4174346133 0 0) 0x7fe9dc071500 con 0x7fe9dc159e40 2023-11-09T03:55:24.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.601+0000 7fe9e0c4a640 1 --2- 172.21.15.76:0/1113504809 >> v2:172.21.15.113:3300/0 conn(0x7fe9dc0b0c40 0x7fe9dc06fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:55:24.608 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.601+0000 7fe9e1c4c640 1 --2- 172.21.15.76:0/1113504809 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9dc06bb60 0x7fe9dc070530 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4009d90 tx=0x7fe9d402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faa29af4284fbb67 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:24.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.602+0000 7fe9d3fff640 1 -- 172.21.15.76:0/1113504809 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d402cbc0 con 0x7fe9dc06bb60 2023-11-09T03:55:24.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.602+0000 7fe9d3fff640 1 -- 172.21.15.76:0/1113504809 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d402cd20 con 0x7fe9dc06bb60 2023-11-09T03:55:24.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.602+0000 7fe9d3fff640 1 -- 172.21.15.76:0/1113504809 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4034660 con 0x7fe9dc06bb60 2023-11-09T03:55:24.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.602+0000 7fe9e36af640 1 -- 172.21.15.76:0/1113504809 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9dc06bb60 msgr2=0x7fe9dc070530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:24.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.602+0000 7fe9e36af640 1 --2- 172.21.15.76:0/1113504809 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9dc06bb60 0x7fe9dc070530 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4009d90 tx=0x7fe9d402be10 comp rx=0 tx=0).stop 2023-11-09T03:55:24.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.602+0000 7fe9e36af640 1 -- 172.21.15.76:0/1113504809 shutdown_connections 2023-11-09T03:55:24.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.602+0000 7fe9e36af640 1 --2- 172.21.15.76:0/1113504809 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9dc06bb60 0x7fe9dc070530 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:24.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.602+0000 7fe9e36af640 1 --2- 172.21.15.76:0/1113504809 >> v2:172.21.15.113:3300/0 conn(0x7fe9dc0b0c40 0x7fe9dc06fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:24.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.602+0000 7fe9e36af640 1 -- 172.21.15.76:0/1113504809 >> 172.21.15.76:0/1113504809 conn(0x7fe9dc0cb530 msgr2=0x7fe9dc0cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:24.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.602+0000 7fe9e36af640 1 -- 172.21.15.76:0/1113504809 shutdown_connections 2023-11-09T03:55:24.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.602+0000 7fe9e36af640 1 -- 172.21.15.76:0/1113504809 wait complete. 2023-11-09T03:55:24.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.603+0000 7fe9e36af640 1 Processor -- start 2023-11-09T03:55:24.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.603+0000 7fe9e36af640 1 -- start start 2023-11-09T03:55:24.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.604+0000 7fe9e36af640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9dc06bb60 0x7fe9dc155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:24.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.604+0000 7fe9e36af640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9dc0b0c40 0x7fe9dc15b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:24.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.604+0000 7fe9e36af640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9dc159e40 0x7fe9dc14ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:24.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.604+0000 7fe9e36af640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc15b8e0 con 0x7fe9dc159e40 2023-11-09T03:55:24.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.604+0000 7fe9e36af640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc15bb60 con 0x7fe9dc06bb60 2023-11-09T03:55:24.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.604+0000 7fe9e36af640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc15be40 con 0x7fe9dc0b0c40 2023-11-09T03:55:24.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.604+0000 7fe9e1c4c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9dc159e40 0x7fe9dc14ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:24.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.604+0000 7fe9e1c4c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9dc159e40 0x7fe9dc14ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50112/0 (socket says 172.21.15.76:50112) 2023-11-09T03:55:24.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.604+0000 7fe9e1c4c640 1 -- 172.21.15.76:0/2988630293 learned_addr learned my addr 172.21.15.76:0/2988630293 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:24.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.604+0000 7fe9e0c4a640 1 --2- 172.21.15.76:0/2988630293 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9dc0b0c40 0x7fe9dc15b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:24.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.604+0000 7fe9e144b640 1 --2- 172.21.15.76:0/2988630293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9dc06bb60 0x7fe9dc155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:24.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.605+0000 7fe9e1c4c640 1 -- 172.21.15.76:0/2988630293 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9dc0b0c40 msgr2=0x7fe9dc15b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:24.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.605+0000 7fe9e1c4c640 1 --2- 172.21.15.76:0/2988630293 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9dc0b0c40 0x7fe9dc15b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:24.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.605+0000 7fe9e1c4c640 1 -- 172.21.15.76:0/2988630293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9dc06bb60 msgr2=0x7fe9dc155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:24.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.605+0000 7fe9e1c4c640 1 --2- 172.21.15.76:0/2988630293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9dc06bb60 0x7fe9dc155400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:24.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.605+0000 7fe9e1c4c640 1 -- 172.21.15.76:0/2988630293 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9cc009400 con 0x7fe9dc159e40 2023-11-09T03:55:24.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.605+0000 7fe9e1c4c640 1 --2- 172.21.15.76:0/2988630293 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9dc159e40 0x7fe9dc14ded0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4009d90 tx=0x7fe9d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:24.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.605+0000 7fe9d1ffb640 1 -- 172.21.15.76:0/2988630293 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4003b90 con 0x7fe9dc159e40 2023-11-09T03:55:24.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.605+0000 7fe9d1ffb640 1 -- 172.21.15.76:0/2988630293 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d402ce90 con 0x7fe9dc159e40 2023-11-09T03:55:24.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.605+0000 7fe9e36af640 1 -- 172.21.15.76:0/2988630293 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d4009a70 con 0x7fe9dc159e40 2023-11-09T03:55:24.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.605+0000 7fe9d1ffb640 1 -- 172.21.15.76:0/2988630293 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4009450 con 0x7fe9dc159e40 2023-11-09T03:55:24.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.605+0000 7fe9e36af640 1 -- 172.21.15.76:0/2988630293 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9dc15c400 con 0x7fe9dc159e40 2023-11-09T03:55:24.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.606+0000 7fe9e36af640 1 -- 172.21.15.76:0/2988630293 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9dc0684b0 con 0x7fe9dc159e40 2023-11-09T03:55:24.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.607+0000 7fe9d1ffb640 1 -- 172.21.15.76:0/2988630293 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe9d403b050 con 0x7fe9dc159e40 2023-11-09T03:55:24.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.607+0000 7fe9d1ffb640 1 --2- 172.21.15.76:0/2988630293 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe9c006ae70 0x7fe9c006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:24.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.607+0000 7fe9d1ffb640 1 -- 172.21.15.76:0/2988630293 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7fe9d40ac490 con 0x7fe9dc159e40 2023-11-09T03:55:24.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.607+0000 7fe9e144b640 1 --2- 172.21.15.76:0/2988630293 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe9c006ae70 0x7fe9c006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:24.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.608+0000 7fe9e144b640 1 --2- 172.21.15.76:0/2988630293 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe9c006ae70 0x7fe9c006d330 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8000f60 tx=0x7fe9d8007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:24.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.611+0000 7fe9d1ffb640 1 -- 172.21.15.76:0/2988630293 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe9d4076a60 con 0x7fe9dc159e40 2023-11-09T03:55:24.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.813+0000 7fe9e36af640 1 -- 172.21.15.76:0/2988630293 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9dc0c9750 con 0x7fe9dc159e40 2023-11-09T03:55:24.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.814+0000 7fe9d1ffb640 1 -- 172.21.15.76:0/2988630293 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 134 v134) v1 ==== 95+0+4932 (secure 0 0 0) 0x7fe9d407a080 con 0x7fe9dc159e40 2023-11-09T03:55:24.815 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:55:24.815 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":134,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6698,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":134,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:55:13.289237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":9024},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9024":{"gid":9024,"name":"a","rank":1,"incarnation":129,"state":"up:active","state_seq":16,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:55:24.816 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 134 2023-11-09T03:55:24.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.816+0000 7fe9e36af640 1 -- 172.21.15.76:0/2988630293 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe9c006ae70 msgr2=0x7fe9c006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:24.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.816+0000 7fe9e36af640 1 --2- 172.21.15.76:0/2988630293 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe9c006ae70 0x7fe9c006d330 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8000f60 tx=0x7fe9d8007c00 comp rx=0 tx=0).stop 2023-11-09T03:55:24.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.816+0000 7fe9e36af640 1 -- 172.21.15.76:0/2988630293 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9dc159e40 msgr2=0x7fe9dc14ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:24.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.816+0000 7fe9e36af640 1 --2- 172.21.15.76:0/2988630293 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9dc159e40 0x7fe9dc14ded0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4009d90 tx=0x7fe9d4004a40 comp rx=0 tx=0).stop 2023-11-09T03:55:24.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.816+0000 7fe9e36af640 1 -- 172.21.15.76:0/2988630293 shutdown_connections 2023-11-09T03:55:24.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.816+0000 7fe9e36af640 1 --2- 172.21.15.76:0/2988630293 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe9c006ae70 0x7fe9c006d330 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:24.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.816+0000 7fe9e36af640 1 --2- 172.21.15.76:0/2988630293 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9dc159e40 0x7fe9dc14ded0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:24.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.816+0000 7fe9e36af640 1 --2- 172.21.15.76:0/2988630293 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9dc0b0c40 0x7fe9dc15b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:24.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.816+0000 7fe9e36af640 1 --2- 172.21.15.76:0/2988630293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9dc06bb60 0x7fe9dc155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:24.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.816+0000 7fe9e36af640 1 -- 172.21.15.76:0/2988630293 >> 172.21.15.76:0/2988630293 conn(0x7fe9dc0cb530 msgr2=0x7fe9dc0cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:24.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.816+0000 7fe9e36af640 1 -- 172.21.15.76:0/2988630293 shutdown_connections 2023-11-09T03:55:24.819 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:24.816+0000 7fe9e36af640 1 -- 172.21.15.76:0/2988630293 wait complete. 2023-11-09T03:55:24.827 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:55:25.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~7s 2023-11-09T03:55:29.828 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:55:29.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.940+0000 7f083a505640 1 Processor -- start 2023-11-09T03:55:29.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.940+0000 7f083a505640 1 -- start start 2023-11-09T03:55:29.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.941+0000 7f083a505640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f08340ade00 0x7f08340b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:29.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.941+0000 7f083a505640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f083406d9c0 0x7f08340710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:29.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.941+0000 7f083a505640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f083406cc90 con 0x7f083406c5f0 2023-11-09T03:55:29.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.941+0000 7f083a505640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08340b0e60 con 0x7f083406d9c0 2023-11-09T03:55:29.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.941+0000 7f083a505640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f083406cee0 con 0x7f08340ade00 2023-11-09T03:55:29.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.941+0000 7f08337fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f083406d9c0 0x7f08340710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:29.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.941+0000 7f0838aa2640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f083406c5f0 0x7f083415e9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:58536/0 (socket says 172.21.15.76:58536) 2023-11-09T03:55:29.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.941+0000 7f0838aa2640 1 -- 172.21.15.76:0/627528770 learned_addr learned my addr 172.21.15.76:0/627528770 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:55:29.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.941+0000 7f0833fff640 1 --2- 172.21.15.76:0/627528770 >> v2:172.21.15.113:3300/0 conn(0x7f08340ade00 0x7f08340b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:29.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.941+0000 7f08337fe640 1 -- 172.21.15.76:0/627528770 >> v2:172.21.15.113:3300/0 conn(0x7f08340ade00 msgr2=0x7f08340b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:29.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.941+0000 7f08337fe640 1 --2- 172.21.15.76:0/627528770 >> v2:172.21.15.113:3300/0 conn(0x7f08340ade00 0x7f08340b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:29.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.941+0000 7f08337fe640 1 -- 172.21.15.76:0/627528770 >> v1:172.21.15.76:6789/0 conn(0x7f083406c5f0 legacy=0x7f083415e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:29.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.941+0000 7f08337fe640 1 -- 172.21.15.76:0/627528770 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f083406d1c0 con 0x7f083406d9c0 2023-11-09T03:55:29.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.942+0000 7f08337fe640 1 --2- 172.21.15.76:0/627528770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f083406d9c0 0x7f08340710d0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f08340cb430 tx=0x7f082802b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450141bf17030b97 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:29.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.942+0000 7f0832ffd640 1 -- 172.21.15.76:0/627528770 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0828031710 con 0x7f083406d9c0 2023-11-09T03:55:29.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.942+0000 7f0832ffd640 1 -- 172.21.15.76:0/627528770 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f082802f050 con 0x7f083406d9c0 2023-11-09T03:55:29.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.942+0000 7f0832ffd640 1 -- 172.21.15.76:0/627528770 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0828031a90 con 0x7f083406d9c0 2023-11-09T03:55:29.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.943+0000 7f083a505640 1 -- 172.21.15.76:0/627528770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f083406d9c0 msgr2=0x7f08340710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:29.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.943+0000 7f083a505640 1 --2- 172.21.15.76:0/627528770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f083406d9c0 0x7f08340710d0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f08340cb430 tx=0x7f082802b7b0 comp rx=0 tx=0).stop 2023-11-09T03:55:29.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.943+0000 7f083a505640 1 -- 172.21.15.76:0/627528770 shutdown_connections 2023-11-09T03:55:29.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.943+0000 7f083a505640 1 --2- 172.21.15.76:0/627528770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f083406d9c0 0x7f08340710d0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:29.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.943+0000 7f083a505640 1 --2- 172.21.15.76:0/627528770 >> v2:172.21.15.113:3300/0 conn(0x7f08340ade00 0x7f08340b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:29.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.943+0000 7f083a505640 1 -- 172.21.15.76:0/627528770 >> 172.21.15.76:0/627528770 conn(0x7f08340cb530 msgr2=0x7f08340cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:29.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.943+0000 7f083a505640 1 -- 172.21.15.76:0/627528770 shutdown_connections 2023-11-09T03:55:29.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.943+0000 7f083a505640 1 -- 172.21.15.76:0/627528770 wait complete. 2023-11-09T03:55:29.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.944+0000 7f083a505640 1 Processor -- start 2023-11-09T03:55:29.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.944+0000 7f083a505640 1 -- start start 2023-11-09T03:55:29.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.945+0000 7f083a505640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f083406c5f0 0x7f083414eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:29.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.945+0000 7f083a505640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f083406d9c0 0x7f08340ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:29.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.945+0000 7f083a505640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08340ade00 0x7f08340ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:29.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.945+0000 7f083a505640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0834150f70 con 0x7f083406d9c0 2023-11-09T03:55:29.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.945+0000 7f083a505640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0834151250 con 0x7f083406c5f0 2023-11-09T03:55:29.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.945+0000 7f083a505640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0834151530 con 0x7f08340ade00 2023-11-09T03:55:29.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.945+0000 7f08337fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f083406d9c0 0x7f08340ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:29.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.945+0000 7f08337fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f083406d9c0 0x7f08340ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50124/0 (socket says 172.21.15.76:50124) 2023-11-09T03:55:29.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.945+0000 7f08337fe640 1 -- 172.21.15.76:0/1951785544 learned_addr learned my addr 172.21.15.76:0/1951785544 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:29.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.945+0000 7f0833fff640 1 --2- 172.21.15.76:0/1951785544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f083406c5f0 0x7f083414eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:29.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.945+0000 7f0838aa2640 1 --2- 172.21.15.76:0/1951785544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08340ade00 0x7f08340ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:29.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.945+0000 7f08337fe640 1 -- 172.21.15.76:0/1951785544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08340ade00 msgr2=0x7f08340ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:29.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.945+0000 7f08337fe640 1 --2- 172.21.15.76:0/1951785544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08340ade00 0x7f08340ace10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:29.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.945+0000 7f08337fe640 1 -- 172.21.15.76:0/1951785544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f083406c5f0 msgr2=0x7f083414eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:29.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.945+0000 7f08337fe640 1 --2- 172.21.15.76:0/1951785544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f083406c5f0 0x7f083414eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:29.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.945+0000 7f08337fe640 1 -- 172.21.15.76:0/1951785544 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0828009400 con 0x7f083406d9c0 2023-11-09T03:55:29.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.945+0000 7f0833fff640 1 --2- 172.21.15.76:0/1951785544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f083406c5f0 0x7f083414eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:55:29.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.945+0000 7f0838aa2640 1 --2- 172.21.15.76:0/1951785544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08340ade00 0x7f08340ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:55:29.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.946+0000 7f08337fe640 1 --2- 172.21.15.76:0/1951785544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f083406d9c0 0x7f08340ac8d0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f0828000c00 tx=0x7f08280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:29.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.946+0000 7f0830ff9640 1 -- 172.21.15.76:0/1951785544 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082803c8d0 con 0x7f083406d9c0 2023-11-09T03:55:29.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.946+0000 7f0830ff9640 1 -- 172.21.15.76:0/1951785544 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f082803ca30 con 0x7f083406d9c0 2023-11-09T03:55:29.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.946+0000 7f083a505640 1 -- 172.21.15.76:0/1951785544 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0834151810 con 0x7f083406d9c0 2023-11-09T03:55:29.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.946+0000 7f0830ff9640 1 -- 172.21.15.76:0/1951785544 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082803cde0 con 0x7f083406d9c0 2023-11-09T03:55:29.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.946+0000 7f083a505640 1 -- 172.21.15.76:0/1951785544 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0834150c20 con 0x7f083406d9c0 2023-11-09T03:55:29.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.947+0000 7f083a505640 1 -- 172.21.15.76:0/1951785544 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08340684b0 con 0x7f083406d9c0 2023-11-09T03:55:29.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.947+0000 7f0830ff9640 1 -- 172.21.15.76:0/1951785544 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f082803a040 con 0x7f083406d9c0 2023-11-09T03:55:29.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.947+0000 7f0830ff9640 1 --2- 172.21.15.76:0/1951785544 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f082006ae70 0x7f082006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:29.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.948+0000 7f0830ff9640 1 -- 172.21.15.76:0/1951785544 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7f08280ab610 con 0x7f083406d9c0 2023-11-09T03:55:29.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.948+0000 7f0833fff640 1 --2- 172.21.15.76:0/1951785544 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f082006ae70 0x7f082006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:29.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.948+0000 7f0833fff640 1 --2- 172.21.15.76:0/1951785544 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f082006ae70 0x7f082006d330 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0824006fd0 tx=0x7f0824008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:29.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:29.952+0000 7f0830ff9640 1 -- 172.21.15.76:0/1951785544 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0828075c60 con 0x7f083406d9c0 2023-11-09T03:55:30.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.153+0000 7f083a505640 1 -- 172.21.15.76:0/1951785544 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08341510b0 con 0x7f083406d9c0 2023-11-09T03:55:30.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.153+0000 7f0830ff9640 1 -- 172.21.15.76:0/1951785544 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 135 v135) v1 ==== 95+0+4980 (secure 0 0 0) 0x7f08341510b0 con 0x7f083406d9c0 2023-11-09T03:55:30.153 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:55:30.154 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":135,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6698,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:55:26.497949+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":9024},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9024":{"gid":9024,"name":"a","rank":1,"incarnation":129,"state":"up:active","state_seq":16,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"laggy_since":"2023-11-09T03:55:26.497949+0000","export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:55:30.154 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 135 2023-11-09T03:55:30.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.155+0000 7f083a505640 1 -- 172.21.15.76:0/1951785544 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f082006ae70 msgr2=0x7f082006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:30.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.155+0000 7f083a505640 1 --2- 172.21.15.76:0/1951785544 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f082006ae70 0x7f082006d330 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0824006fd0 tx=0x7f0824008040 comp rx=0 tx=0).stop 2023-11-09T03:55:30.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.155+0000 7f083a505640 1 -- 172.21.15.76:0/1951785544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f083406d9c0 msgr2=0x7f08340ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:30.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.155+0000 7f083a505640 1 --2- 172.21.15.76:0/1951785544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f083406d9c0 0x7f08340ac8d0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f0828000c00 tx=0x7f08280046c0 comp rx=0 tx=0).stop 2023-11-09T03:55:30.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.155+0000 7f083a505640 1 -- 172.21.15.76:0/1951785544 shutdown_connections 2023-11-09T03:55:30.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.155+0000 7f083a505640 1 --2- 172.21.15.76:0/1951785544 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f082006ae70 0x7f082006d330 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:30.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.155+0000 7f083a505640 1 --2- 172.21.15.76:0/1951785544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08340ade00 0x7f08340ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:30.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.155+0000 7f083a505640 1 --2- 172.21.15.76:0/1951785544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f083406d9c0 0x7f08340ac8d0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:30.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.155+0000 7f083a505640 1 --2- 172.21.15.76:0/1951785544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f083406c5f0 0x7f083414eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:30.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.155+0000 7f083a505640 1 -- 172.21.15.76:0/1951785544 >> 172.21.15.76:0/1951785544 conn(0x7f08340cb530 msgr2=0x7f08340b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:30.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.155+0000 7f083a505640 1 -- 172.21.15.76:0/1951785544 shutdown_connections 2023-11-09T03:55:30.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.155+0000 7f083a505640 1 -- 172.21.15.76:0/1951785544 wait complete. 2023-11-09T03:55:30.166 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2023-11-09T03:55:30.166 DEBUG:teuthology.orchestra.run.smithi076:> sudo sysctl -n kernel.core_pattern 2023-11-09T03:55:30.192 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-09T03:55:30.193 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2023-11-09T03:55:30.194 DEBUG:teuthology.orchestra.run.smithi076:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2023-11-09T03:55:30.263 INFO:teuthology.orchestra.run.smithi076.stdout:1699502114.128695.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i a', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2023-11-09T03:55:30.265 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1699502114.128695.core'] 2023-11-09T03:55:30.265 INFO:tasks.cephfs.cephfs_test_case:Found core file 1699502114.128695.core, deleting it 2023-11-09T03:55:30.265 DEBUG:teuthology.orchestra.run.smithi076:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1699502114.128695.core 2023-11-09T03:55:30.335 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:55:30.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:55:30.336 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 down 2023-11-09T03:55:30.403 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:55:30.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.543+0000 7f0e2a7a9640 1 Processor -- start 2023-11-09T03:55:30.543 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.543+0000 7f0e2a7a9640 1 -- start start 2023-11-09T03:55:30.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.544+0000 7f0e2a7a9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0e2415d4e0 0x7f0e240d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:30.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.544+0000 7f0e2a7a9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e240b32b0 con 0x7f0e2415d100 2023-11-09T03:55:30.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.544+0000 7f0e2a7a9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e240cee10 con 0x7f0e240d3b60 2023-11-09T03:55:30.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.544+0000 7f0e2a7a9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e240cec20 con 0x7f0e2415d4e0 2023-11-09T03:55:30.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.544+0000 7f0e23fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0e2415d100 0x7f0e240ce510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:58544/0 (socket says 172.21.15.76:58544) 2023-11-09T03:55:30.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.544+0000 7f0e23fff640 1 -- 172.21.15.76:0/2441873301 learned_addr learned my addr 172.21.15.76:0/2441873301 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:55:30.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.545+0000 7f0e22ffd640 1 -- 172.21.15.76:0/2441873301 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2682695547 0 0) 0x7f0e240b32b0 con 0x7f0e2415d100 2023-11-09T03:55:30.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.545+0000 7f0e22ffd640 1 -- 172.21.15.76:0/2441873301 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e0c003520 con 0x7f0e2415d100 2023-11-09T03:55:30.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.545+0000 7f0e237fe640 1 --2- 172.21.15.76:0/2441873301 >> v2:172.21.15.113:3300/0 conn(0x7f0e2415d4e0 0x7f0e240d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:30.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.545+0000 7f0e22ffd640 1 -- 172.21.15.76:0/2441873301 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4038830970 0 0) 0x7f0e0c003520 con 0x7f0e2415d100 2023-11-09T03:55:30.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.545+0000 7f0e22ffd640 1 -- 172.21.15.76:0/2441873301 >> v2:172.21.15.113:3300/0 conn(0x7f0e2415d4e0 msgr2=0x7f0e240d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:30.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.545+0000 7f0e22ffd640 1 --2- 172.21.15.76:0/2441873301 >> v2:172.21.15.113:3300/0 conn(0x7f0e2415d4e0 0x7f0e240d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:30.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.545+0000 7f0e22ffd640 1 -- 172.21.15.76:0/2441873301 >> v1:172.21.15.113:6789/0 conn(0x7f0e240d3b60 legacy=0x7f0e240b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:30.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.545+0000 7f0e22ffd640 1 -- 172.21.15.76:0/2441873301 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e240b2980 con 0x7f0e2415d100 2023-11-09T03:55:30.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.545+0000 7f0e22ffd640 1 -- 172.21.15.76:0/2441873301 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f0e140017f0 con 0x7f0e2415d100 2023-11-09T03:55:30.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.545+0000 7f0e22ffd640 1 -- 172.21.15.76:0/2441873301 >> v1:172.21.15.76:6789/0 conn(0x7f0e2415d100 legacy=0x7f0e240ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:30.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.545+0000 7f0e22ffd640 1 --2- 172.21.15.76:0/2441873301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e0c003970 0x7f0e0c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:30.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.545+0000 7f0e22ffd640 1 --2- 172.21.15.76:0/2441873301 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e0c0047c0 0x7f0e0c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:30.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.545+0000 7f0e22ffd640 1 --2- 172.21.15.76:0/2441873301 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e0c005180 0x7f0e0c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:30.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.545+0000 7f0e22ffd640 1 -- 172.21.15.76:0/2441873301 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c00bff0 con 0x7f0e0c003970 2023-11-09T03:55:30.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.545+0000 7f0e22ffd640 1 -- 172.21.15.76:0/2441873301 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c00c160 con 0x7f0e0c0047c0 2023-11-09T03:55:30.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.545+0000 7f0e22ffd640 1 -- 172.21.15.76:0/2441873301 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c00c320 con 0x7f0e0c005180 2023-11-09T03:55:30.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.545+0000 7f0e23fff640 1 --2- 172.21.15.76:0/2441873301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e0c003970 0x7f0e0c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:30.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.545+0000 7f0e237fe640 1 --2- 172.21.15.76:0/2441873301 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e0c0047c0 0x7f0e0c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:30.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.545+0000 7f0e28d46640 1 --2- 172.21.15.76:0/2441873301 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e0c005180 0x7f0e0c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:30.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.545+0000 7f0e23fff640 1 -- 172.21.15.76:0/2441873301 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e0c005180 msgr2=0x7f0e0c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:30.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.545+0000 7f0e23fff640 1 --2- 172.21.15.76:0/2441873301 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e0c005180 0x7f0e0c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:30.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.545+0000 7f0e23fff640 1 -- 172.21.15.76:0/2441873301 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e0c0047c0 msgr2=0x7f0e0c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:30.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.545+0000 7f0e23fff640 1 --2- 172.21.15.76:0/2441873301 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e0c0047c0 0x7f0e0c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:30.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.545+0000 7f0e23fff640 1 -- 172.21.15.76:0/2441873301 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0c003520 con 0x7f0e0c003970 2023-11-09T03:55:30.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.545+0000 7f0e28d46640 1 -- 172.21.15.76:0/2441873301 reap_dead start 2023-11-09T03:55:30.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.546+0000 7f0e23fff640 1 --2- 172.21.15.76:0/2441873301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e0c003970 0x7f0e0c003d90 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400c190 tx=0x7f0e0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43af80f8600d8c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:30.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.546+0000 7f0e22ffd640 1 -- 172.21.15.76:0/2441873301 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14007270 con 0x7f0e0c003970 2023-11-09T03:55:30.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.546+0000 7f0e22ffd640 1 -- 172.21.15.76:0/2441873301 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e140073f0 con 0x7f0e0c003970 2023-11-09T03:55:30.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.546+0000 7f0e22ffd640 1 -- 172.21.15.76:0/2441873301 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1402c8a0 con 0x7f0e0c003970 2023-11-09T03:55:30.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.547+0000 7f0e2a7a9640 1 -- 172.21.15.76:0/2441873301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e0c003970 msgr2=0x7f0e0c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:30.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.547+0000 7f0e2a7a9640 1 --2- 172.21.15.76:0/2441873301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e0c003970 0x7f0e0c003d90 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400c190 tx=0x7f0e0c0057f0 comp rx=0 tx=0).stop 2023-11-09T03:55:30.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.547+0000 7f0e2a7a9640 1 -- 172.21.15.76:0/2441873301 shutdown_connections 2023-11-09T03:55:30.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.547+0000 7f0e2a7a9640 1 --2- 172.21.15.76:0/2441873301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e0c003970 0x7f0e0c003d90 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:30.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.547+0000 7f0e2a7a9640 1 -- 172.21.15.76:0/2441873301 >> 172.21.15.76:0/2441873301 conn(0x7f0e240d2ef0 msgr2=0x7f0e240d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:30.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.547+0000 7f0e2a7a9640 1 -- 172.21.15.76:0/2441873301 shutdown_connections 2023-11-09T03:55:30.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.547+0000 7f0e2a7a9640 1 -- 172.21.15.76:0/2441873301 wait complete. 2023-11-09T03:55:30.557 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.548+0000 7f0e2a7a9640 1 Processor -- start 2023-11-09T03:55:30.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.548+0000 7f0e2a7a9640 1 -- start start 2023-11-09T03:55:30.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.549+0000 7f0e2a7a9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e0c003970 0x7f0e240ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:30.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.549+0000 7f0e2a7a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e2414e2d0 0x7f0e240b4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:30.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.549+0000 7f0e2a7a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e240b4cb0 0x7f0e240cc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:30.558 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.549+0000 7f0e2a7a9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e240cd3f0 con 0x7f0e0c003970 2023-11-09T03:55:30.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.549+0000 7f0e2a7a9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e240b5140 con 0x7f0e2414e2d0 2023-11-09T03:55:30.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.549+0000 7f0e2a7a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2414b8b0 con 0x7f0e240b4cb0 2023-11-09T03:55:30.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.549+0000 7f0e23fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e0c003970 0x7f0e240ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:30.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.549+0000 7f0e23fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e0c003970 0x7f0e240ca8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50136/0 (socket says 172.21.15.76:50136) 2023-11-09T03:55:30.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.549+0000 7f0e23fff640 1 -- 172.21.15.76:0/3194813428 learned_addr learned my addr 172.21.15.76:0/3194813428 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:30.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.549+0000 7f0e23fff640 1 -- 172.21.15.76:0/3194813428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e240b4cb0 msgr2=0x7f0e240cc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:30.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.549+0000 7f0e23fff640 1 --2- 172.21.15.76:0/3194813428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e240b4cb0 0x7f0e240cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:30.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.549+0000 7f0e23fff640 1 -- 172.21.15.76:0/3194813428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e2414e2d0 msgr2=0x7f0e240b4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:30.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.549+0000 7f0e23fff640 1 --2- 172.21.15.76:0/3194813428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e2414e2d0 0x7f0e240b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:30.560 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.549+0000 7f0e23fff640 1 -- 172.21.15.76:0/3194813428 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e240cadf0 con 0x7f0e0c003970 2023-11-09T03:55:30.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.549+0000 7f0e23fff640 1 --2- 172.21.15.76:0/3194813428 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e0c003970 0x7f0e240ca8b0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f0e1402c440 tx=0x7f0e1402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:30.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.549+0000 7f0e20ff9640 1 -- 172.21.15.76:0/3194813428 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1403c040 con 0x7f0e0c003970 2023-11-09T03:55:30.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.549+0000 7f0e20ff9640 1 -- 172.21.15.76:0/3194813428 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1402ccb0 con 0x7f0e0c003970 2023-11-09T03:55:30.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.549+0000 7f0e20ff9640 1 -- 172.21.15.76:0/3194813428 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e140059e0 con 0x7f0e0c003970 2023-11-09T03:55:30.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.549+0000 7f0e2a7a9640 1 -- 172.21.15.76:0/3194813428 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e2414af80 con 0x7f0e0c003970 2023-11-09T03:55:30.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.550+0000 7f0e2a7a9640 1 -- 172.21.15.76:0/3194813428 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e2414b540 con 0x7f0e0c003970 2023-11-09T03:55:30.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.551+0000 7f0e2a7a9640 1 -- 172.21.15.76:0/3194813428 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e240684b0 con 0x7f0e0c003970 2023-11-09T03:55:30.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.551+0000 7f0e20ff9640 1 -- 172.21.15.76:0/3194813428 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0e14032070 con 0x7f0e0c003970 2023-11-09T03:55:30.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.551+0000 7f0e20ff9640 1 --2- 172.21.15.76:0/3194813428 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0e1006ae70 0x7f0e1006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:30.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.551+0000 7f0e20ff9640 1 -- 172.21.15.76:0/3194813428 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7f0e140ab450 con 0x7f0e0c003970 2023-11-09T03:55:30.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.552+0000 7f0e237fe640 1 --2- 172.21.15.76:0/3194813428 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0e1006ae70 0x7f0e1006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:30.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.552+0000 7f0e237fe640 1 --2- 172.21.15.76:0/3194813428 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0e1006ae70 0x7f0e1006d330 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f0e18006e80 tx=0x7f0e18006df0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:30.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.556+0000 7f0e20ff9640 1 -- 172.21.15.76:0/3194813428 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0e14075a10 con 0x7f0e0c003970 2023-11-09T03:55:30.759 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.759+0000 7f0e2a7a9640 1 -- 172.21.15.76:0/3194813428 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e240d2cd0 con 0x7f0e0c003970 2023-11-09T03:55:30.760 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.760+0000 7f0e20ff9640 1 -- 172.21.15.76:0/3194813428 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 135 v135) v1 ==== 95+0+4980 (secure 0 0 0) 0x7f0e14079030 con 0x7f0e0c003970 2023-11-09T03:55:30.760 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:55:30.760 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":135,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6698,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:55:26.497949+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":9024},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9024":{"gid":9024,"name":"a","rank":1,"incarnation":129,"state":"up:active","state_seq":16,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"laggy_since":"2023-11-09T03:55:26.497949+0000","export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:55:30.762 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 135 2023-11-09T03:55:30.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.762+0000 7f0e2a7a9640 1 -- 172.21.15.76:0/3194813428 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0e1006ae70 msgr2=0x7f0e1006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:30.762 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.762+0000 7f0e2a7a9640 1 --2- 172.21.15.76:0/3194813428 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0e1006ae70 0x7f0e1006d330 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f0e18006e80 tx=0x7f0e18006df0 comp rx=0 tx=0).stop 2023-11-09T03:55:30.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.762+0000 7f0e2a7a9640 1 -- 172.21.15.76:0/3194813428 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e0c003970 msgr2=0x7f0e240ca8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:30.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.762+0000 7f0e2a7a9640 1 --2- 172.21.15.76:0/3194813428 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e0c003970 0x7f0e240ca8b0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f0e1402c440 tx=0x7f0e1402e070 comp rx=0 tx=0).stop 2023-11-09T03:55:30.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.762+0000 7f0e2a7a9640 1 -- 172.21.15.76:0/3194813428 shutdown_connections 2023-11-09T03:55:30.763 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.762+0000 7f0e2a7a9640 1 --2- 172.21.15.76:0/3194813428 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f0e1006ae70 0x7f0e1006d330 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:30.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.762+0000 7f0e2a7a9640 1 --2- 172.21.15.76:0/3194813428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e240b4cb0 0x7f0e240cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:30.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.762+0000 7f0e2a7a9640 1 --2- 172.21.15.76:0/3194813428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e2414e2d0 0x7f0e240b4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:30.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.762+0000 7f0e2a7a9640 1 --2- 172.21.15.76:0/3194813428 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e0c003970 0x7f0e240ca8b0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:30.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.762+0000 7f0e2a7a9640 1 -- 172.21.15.76:0/3194813428 >> 172.21.15.76:0/3194813428 conn(0x7f0e240d2ef0 msgr2=0x7f0e240b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:30.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.762+0000 7f0e2a7a9640 1 -- 172.21.15.76:0/3194813428 shutdown_connections 2023-11-09T03:55:30.765 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.762+0000 7f0e2a7a9640 1 -- 172.21.15.76:0/3194813428 wait complete. 2023-11-09T03:55:30.773 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2023-11-09T03:55:30.887 INFO:teuthology.orchestra.run.smithi076.stdout:{"last_snap":17,"last_created":17,"last_destroyed":14,"pending_noop":[],"snaps":[{"snapid":16,"ino":1099511627777,"stamp":"2023-11-09T03:54:27.826603+0000","name":"s34","metadata":{}},{"snapid":17,"ino":1099511627777,"stamp":"2023-11-09T03:55:13.714891+0000","name":"s37","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-09T03:55:30.887 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 17, "last_destroyed": 14, "last_snap": 17, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s34", "snapid": 16, "stamp": "2023-11-09T03:54:27.826603+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s37", "snapid": 17, "stamp": "2023-11-09T03:55:13.714891+0000" } ] } 2023-11-09T03:55:30.888 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:55:30.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.989+0000 7f334dbfb640 1 Processor -- start 2023-11-09T03:55:30.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.989+0000 7f334dbfb640 1 -- start start 2023-11-09T03:55:30.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.989+0000 7f334dbfb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f334815d100 0x7f33480ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:30.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.990+0000 7f334dbfb640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f33480d1be0 0x7f33480b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:30.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.990+0000 7f334dbfb640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33480b32b0 con 0x7f334815d4e0 2023-11-09T03:55:30.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.990+0000 7f334dbfb640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33480cee10 con 0x7f33480d1be0 2023-11-09T03:55:30.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.990+0000 7f334dbfb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33480b2980 con 0x7f334815d100 2023-11-09T03:55:30.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.990+0000 7f3347fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f33480d1be0 0x7f33480b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:30.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.990+0000 7f3347fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f33480d1be0 0x7f33480b3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50140/0 (socket says 172.21.15.76:50140) 2023-11-09T03:55:30.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.990+0000 7f3347fff640 1 -- 172.21.15.76:0/2496891413 learned_addr learned my addr 172.21.15.76:0/2496891413 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:30.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.990+0000 7f33477fe640 1 --2- 172.21.15.76:0/2496891413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f334815d100 0x7f33480ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:30.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.990+0000 7f3347fff640 1 -- 172.21.15.76:0/2496891413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f334815d100 msgr2=0x7f33480ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:30.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.990+0000 7f3347fff640 1 --2- 172.21.15.76:0/2496891413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f334815d100 0x7f33480ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:30.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.990+0000 7f3347fff640 1 -- 172.21.15.76:0/2496891413 >> v1:172.21.15.113:6789/0 conn(0x7f334815d4e0 legacy=0x7f33480d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:30.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.990+0000 7f3347fff640 1 -- 172.21.15.76:0/2496891413 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33480b2c60 con 0x7f33480d1be0 2023-11-09T03:55:30.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.990+0000 7f33477fe640 1 --2- 172.21.15.76:0/2496891413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f334815d100 0x7f33480ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:55:30.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.991+0000 7f3347fff640 1 --2- 172.21.15.76:0/2496891413 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33480d1be0 0x7f33480b3740 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f3340009e80 tx=0x7f334002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43fa1c2d22b87855 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:30.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.991+0000 7f3346ffd640 1 -- 172.21.15.76:0/2496891413 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334002ca50 con 0x7f33480d1be0 2023-11-09T03:55:30.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.991+0000 7f3346ffd640 1 -- 172.21.15.76:0/2496891413 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f334002cbb0 con 0x7f33480d1be0 2023-11-09T03:55:30.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.991+0000 7f3346ffd640 1 -- 172.21.15.76:0/2496891413 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33400343f0 con 0x7f33480d1be0 2023-11-09T03:55:30.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.991+0000 7f334dbfb640 1 -- 172.21.15.76:0/2496891413 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33480d1be0 msgr2=0x7f33480b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:30.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.991+0000 7f334dbfb640 1 --2- 172.21.15.76:0/2496891413 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33480d1be0 0x7f33480b3740 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f3340009e80 tx=0x7f334002beb0 comp rx=0 tx=0).stop 2023-11-09T03:55:30.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.992+0000 7f334dbfb640 1 -- 172.21.15.76:0/2496891413 shutdown_connections 2023-11-09T03:55:30.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.992+0000 7f334dbfb640 1 --2- 172.21.15.76:0/2496891413 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33480d1be0 0x7f33480b3740 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:30.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.992+0000 7f334dbfb640 1 --2- 172.21.15.76:0/2496891413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f334815d100 0x7f33480ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:30.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.992+0000 7f334dbfb640 1 -- 172.21.15.76:0/2496891413 >> 172.21.15.76:0/2496891413 conn(0x7f33480d2ef0 msgr2=0x7f33480d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:30.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.992+0000 7f334dbfb640 1 -- 172.21.15.76:0/2496891413 shutdown_connections 2023-11-09T03:55:30.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.992+0000 7f334dbfb640 1 -- 172.21.15.76:0/2496891413 wait complete. 2023-11-09T03:55:30.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.993+0000 7f334dbfb640 1 Processor -- start 2023-11-09T03:55:30.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.993+0000 7f334dbfb640 1 -- start start 2023-11-09T03:55:31.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.993+0000 7f334dbfb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33480d1be0 0x7f33480b7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:31.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.993+0000 7f334dbfb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f334815d100 0x7f33481f3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:31.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.993+0000 7f334dbfb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f334815d4e0 0x7f33481f3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:31.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.993+0000 7f334dbfb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33481f0620 con 0x7f33480d1be0 2023-11-09T03:55:31.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.993+0000 7f334dbfb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33481f3b50 con 0x7f334815d100 2023-11-09T03:55:31.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.993+0000 7f334dbfb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33480b7f60 con 0x7f334815d4e0 2023-11-09T03:55:31.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.994+0000 7f33477fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33480d1be0 0x7f33480b7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:31.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.994+0000 7f33477fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33480d1be0 0x7f33480b7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50146/0 (socket says 172.21.15.76:50146) 2023-11-09T03:55:31.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.994+0000 7f33477fe640 1 -- 172.21.15.76:0/2779312504 learned_addr learned my addr 172.21.15.76:0/2779312504 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:31.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.994+0000 7f3347fff640 1 --2- 172.21.15.76:0/2779312504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f334815d4e0 0x7f33481f3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:31.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.994+0000 7f333ffff640 1 --2- 172.21.15.76:0/2779312504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f334815d100 0x7f33481f3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:31.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.994+0000 7f33477fe640 1 -- 172.21.15.76:0/2779312504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f334815d4e0 msgr2=0x7f33481f3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:31.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.994+0000 7f33477fe640 1 --2- 172.21.15.76:0/2779312504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f334815d4e0 0x7f33481f3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:31.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.994+0000 7f33477fe640 1 -- 172.21.15.76:0/2779312504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f334815d100 msgr2=0x7f33481f3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:31.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.994+0000 7f33477fe640 1 --2- 172.21.15.76:0/2779312504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f334815d100 0x7f33481f3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:31.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.994+0000 7f33477fe640 1 -- 172.21.15.76:0/2779312504 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3340009b30 con 0x7f33480d1be0 2023-11-09T03:55:31.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.994+0000 7f3347fff640 1 --2- 172.21.15.76:0/2779312504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f334815d4e0 0x7f33481f3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:55:31.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.994+0000 7f333ffff640 1 --2- 172.21.15.76:0/2779312504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f334815d100 0x7f33481f3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:55:31.006 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.994+0000 7f33477fe640 1 --2- 172.21.15.76:0/2779312504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33480d1be0 0x7f33480b7980 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f333800c7f0 tx=0x7f333800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:31.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.994+0000 7f3344ff9640 1 -- 172.21.15.76:0/2779312504 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3338005b90 con 0x7f33480d1be0 2023-11-09T03:55:31.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.994+0000 7f3344ff9640 1 -- 172.21.15.76:0/2779312504 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3338005cf0 con 0x7f33480d1be0 2023-11-09T03:55:31.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.994+0000 7f3344ff9640 1 -- 172.21.15.76:0/2779312504 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3338007a30 con 0x7f33480d1be0 2023-11-09T03:55:31.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.994+0000 7f334dbfb640 1 -- 172.21.15.76:0/2779312504 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33480b8240 con 0x7f33480d1be0 2023-11-09T03:55:31.007 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.994+0000 7f334dbfb640 1 -- 172.21.15.76:0/2779312504 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33480b8800 con 0x7f33480d1be0 2023-11-09T03:55:31.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.995+0000 7f334dbfb640 1 -- 172.21.15.76:0/2779312504 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33480684b0 con 0x7f33480d1be0 2023-11-09T03:55:31.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.996+0000 7f3344ff9640 1 -- 172.21.15.76:0/2779312504 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3338007b90 con 0x7f33480d1be0 2023-11-09T03:55:31.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.996+0000 7f3344ff9640 1 --2- 172.21.15.76:0/2779312504 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f332806ae70 0x7f332806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:31.008 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.996+0000 7f3344ff9640 1 -- 172.21.15.76:0/2779312504 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7f333808a080 con 0x7f33480d1be0 2023-11-09T03:55:31.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.996+0000 7f333ffff640 1 --2- 172.21.15.76:0/2779312504 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f332806ae70 0x7f332806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:31.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:30.997+0000 7f333ffff640 1 --2- 172.21.15.76:0/2779312504 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f332806ae70 0x7f332806d330 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f33480cb4b0 tx=0x7f3330009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:31.009 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.000+0000 7f3344ff9640 1 -- 172.21.15.76:0/2779312504 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3338054640 con 0x7f33480d1be0 2023-11-09T03:55:31.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.203+0000 7f334dbfb640 1 -- 172.21.15.76:0/2779312504 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33480d0430 con 0x7f33480d1be0 2023-11-09T03:55:31.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.204+0000 7f3344ff9640 1 -- 172.21.15.76:0/2779312504 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 135 v135) v1 ==== 95+0+4980 (secure 0 0 0) 0x7f333801e020 con 0x7f33480d1be0 2023-11-09T03:55:31.204 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:55:31.204 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":135,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6698,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:55:26.497949+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":9024},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9024":{"gid":9024,"name":"a","rank":1,"incarnation":129,"state":"up:active","state_seq":16,"addr":"172.21.15.76:6835/757529522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":757529522},{"type":"v1","addr":"172.21.15.76:6835","nonce":757529522}]},"join_fscid":-1,"laggy_since":"2023-11-09T03:55:26.497949+0000","export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:55:31.204 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 135 2023-11-09T03:55:31.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.205+0000 7f334dbfb640 1 -- 172.21.15.76:0/2779312504 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f332806ae70 msgr2=0x7f332806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:31.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.205+0000 7f334dbfb640 1 --2- 172.21.15.76:0/2779312504 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f332806ae70 0x7f332806d330 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f33480cb4b0 tx=0x7f3330009040 comp rx=0 tx=0).stop 2023-11-09T03:55:31.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.205+0000 7f334dbfb640 1 -- 172.21.15.76:0/2779312504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33480d1be0 msgr2=0x7f33480b7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:31.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.205+0000 7f334dbfb640 1 --2- 172.21.15.76:0/2779312504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33480d1be0 0x7f33480b7980 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f333800c7f0 tx=0x7f333800cbf0 comp rx=0 tx=0).stop 2023-11-09T03:55:31.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.206+0000 7f334dbfb640 1 -- 172.21.15.76:0/2779312504 shutdown_connections 2023-11-09T03:55:31.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.206+0000 7f334dbfb640 1 --2- 172.21.15.76:0/2779312504 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f332806ae70 0x7f332806d330 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:31.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.206+0000 7f334dbfb640 1 --2- 172.21.15.76:0/2779312504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f334815d4e0 0x7f33481f3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:31.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.206+0000 7f334dbfb640 1 --2- 172.21.15.76:0/2779312504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f334815d100 0x7f33481f3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:31.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.206+0000 7f334dbfb640 1 --2- 172.21.15.76:0/2779312504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33480d1be0 0x7f33480b7980 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:31.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.206+0000 7f334dbfb640 1 -- 172.21.15.76:0/2779312504 >> 172.21.15.76:0/2779312504 conn(0x7f33480d2ef0 msgr2=0x7f33480b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:31.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.206+0000 7f334dbfb640 1 -- 172.21.15.76:0/2779312504 shutdown_connections 2023-11-09T03:55:31.207 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.206+0000 7f334dbfb640 1 -- 172.21.15.76:0/2779312504 wait complete. 2023-11-09T03:55:31.216 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2023-11-09T03:55:31.329 INFO:teuthology.orchestra.run.smithi076.stdout:{"last_snap":17,"last_created":17,"last_destroyed":14,"pending_noop":[],"snaps":[{"snapid":16,"ino":1099511627777,"stamp":"2023-11-09T03:54:27.826603+0000","name":"s34","metadata":{}},{"snapid":17,"ino":1099511627777,"stamp":"2023-11-09T03:55:13.714891+0000","name":"s37","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-09T03:55:31.330 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 17, "last_destroyed": 14, "last_snap": 17, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s34", "snapid": 16, "stamp": "2023-11-09T03:54:27.826603+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s37", "snapid": 17, "stamp": "2023-11-09T03:55:13.714891+0000" } ] } 2023-11-09T03:55:31.330 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:1 2023-11-09T03:55:31.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.431+0000 7f32c2afc640 1 Processor -- start 2023-11-09T03:55:31.431 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.432+0000 7f32c2afc640 1 -- start start 2023-11-09T03:55:31.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.432+0000 7f32c2afc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f32bc0addd0 0x7f32bc0cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:31.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.432+0000 7f32c2afc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32bc06c220 0x7f32bc0ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:31.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.432+0000 7f32c2afc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32bc0cd720 con 0x7f32bc0cc680 2023-11-09T03:55:31.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.432+0000 7f32c2afc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32bc0ae1b0 con 0x7f32bc0addd0 2023-11-09T03:55:31.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.432+0000 7f32c2afc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc0ccb10 con 0x7f32bc06c220 2023-11-09T03:55:31.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.432+0000 7f32c0898640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f32bc0addd0 0x7f32bc0cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:31.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.432+0000 7f32c0898640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f32bc0addd0 0x7f32bc0cdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50152/0 (socket says 172.21.15.76:50152) 2023-11-09T03:55:31.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.433+0000 7f32c0898640 1 -- 172.21.15.76:0/4210091714 learned_addr learned my addr 172.21.15.76:0/4210091714 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:31.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.433+0000 7f32c1099640 1 --2- 172.21.15.76:0/4210091714 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32bc06c220 0x7f32bc0ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:31.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.433+0000 7f32c0898640 1 -- 172.21.15.76:0/4210091714 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32bc06c220 msgr2=0x7f32bc0ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:31.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.433+0000 7f32c0898640 1 --2- 172.21.15.76:0/4210091714 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32bc06c220 0x7f32bc0ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:31.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.433+0000 7f32c0898640 1 -- 172.21.15.76:0/4210091714 >> v1:172.21.15.113:6789/0 conn(0x7f32bc0cc680 legacy=0x7f32bc06bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:31.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.433+0000 7f32c0898640 1 -- 172.21.15.76:0/4210091714 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32bc0ccdf0 con 0x7f32bc0addd0 2023-11-09T03:55:31.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.433+0000 7f32c0898640 1 --2- 172.21.15.76:0/4210091714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32bc0addd0 0x7f32bc0cdbb0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009870 tx=0x7f32b802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1565182eec8d2f server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:31.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.434+0000 7f32b37fe640 1 -- 172.21.15.76:0/4210091714 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b8030840 con 0x7f32bc0addd0 2023-11-09T03:55:31.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.434+0000 7f32b37fe640 1 -- 172.21.15.76:0/4210091714 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b80309a0 con 0x7f32bc0addd0 2023-11-09T03:55:31.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.434+0000 7f32b37fe640 1 -- 172.21.15.76:0/4210091714 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b8030d30 con 0x7f32bc0addd0 2023-11-09T03:55:31.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.434+0000 7f32c2afc640 1 -- 172.21.15.76:0/4210091714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32bc0addd0 msgr2=0x7f32bc0cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:31.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.434+0000 7f32c2afc640 1 --2- 172.21.15.76:0/4210091714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32bc0addd0 0x7f32bc0cdbb0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009870 tx=0x7f32b802b980 comp rx=0 tx=0).stop 2023-11-09T03:55:31.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.434+0000 7f32c2afc640 1 -- 172.21.15.76:0/4210091714 shutdown_connections 2023-11-09T03:55:31.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.434+0000 7f32c2afc640 1 --2- 172.21.15.76:0/4210091714 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32bc06c220 0x7f32bc0ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:31.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.434+0000 7f32c2afc640 1 --2- 172.21.15.76:0/4210091714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32bc0addd0 0x7f32bc0cdbb0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:31.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.434+0000 7f32c2afc640 1 -- 172.21.15.76:0/4210091714 >> 172.21.15.76:0/4210091714 conn(0x7f32bc0cb530 msgr2=0x7f32bc0cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:31.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.434+0000 7f32c2afc640 1 -- 172.21.15.76:0/4210091714 shutdown_connections 2023-11-09T03:55:31.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.434+0000 7f32c2afc640 1 -- 172.21.15.76:0/4210091714 wait complete. 2023-11-09T03:55:31.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.435+0000 7f32c2afc640 1 Processor -- start 2023-11-09T03:55:31.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.435+0000 7f32c2afc640 1 -- start start 2023-11-09T03:55:31.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.436+0000 7f32c2afc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32bc06c220 0x7f32bc1e4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:31.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.436+0000 7f32c2afc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32bc0addd0 0x7f32bc0c44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:31.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.436+0000 7f32c2afc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32bc0c50c0 0x7f32bc0c25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:31.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.436+0000 7f32c2afc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc15b5d0 con 0x7f32bc0c50c0 2023-11-09T03:55:31.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.436+0000 7f32c2afc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc1e2eb0 con 0x7f32bc0addd0 2023-11-09T03:55:31.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.436+0000 7f32c2afc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc0c2b00 con 0x7f32bc06c220 2023-11-09T03:55:31.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.436+0000 7f32c1099640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32bc0c50c0 0x7f32bc0c25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:31.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.437+0000 7f32c1099640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32bc0c50c0 0x7f32bc0c25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50156/0 (socket says 172.21.15.76:50156) 2023-11-09T03:55:31.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.437+0000 7f32c1099640 1 -- 172.21.15.76:0/1845128326 learned_addr learned my addr 172.21.15.76:0/1845128326 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:31.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.437+0000 7f32c0898640 1 --2- 172.21.15.76:0/1845128326 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32bc06c220 0x7f32bc1e4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:31.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.437+0000 7f32b3fff640 1 --2- 172.21.15.76:0/1845128326 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32bc0addd0 0x7f32bc0c44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:31.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.437+0000 7f32c1099640 1 -- 172.21.15.76:0/1845128326 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32bc06c220 msgr2=0x7f32bc1e4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:31.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.437+0000 7f32c1099640 1 --2- 172.21.15.76:0/1845128326 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32bc06c220 0x7f32bc1e4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:31.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.437+0000 7f32c1099640 1 -- 172.21.15.76:0/1845128326 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32bc0addd0 msgr2=0x7f32bc0c44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:31.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.437+0000 7f32c1099640 1 --2- 172.21.15.76:0/1845128326 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32bc0addd0 0x7f32bc0c44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:31.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.437+0000 7f32c1099640 1 -- 172.21.15.76:0/1845128326 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b80094d0 con 0x7f32bc0c50c0 2023-11-09T03:55:31.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.437+0000 7f32c1099640 1 --2- 172.21.15.76:0/1845128326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32bc0c50c0 0x7f32bc0c25c0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f32b400efc0 tx=0x7f32b400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:31.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.437+0000 7f32b17fa640 1 -- 172.21.15.76:0/1845128326 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b4009b20 con 0x7f32bc0c50c0 2023-11-09T03:55:31.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.437+0000 7f32b17fa640 1 -- 172.21.15.76:0/1845128326 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b400f040 con 0x7f32bc0c50c0 2023-11-09T03:55:31.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.437+0000 7f32c2afc640 1 -- 172.21.15.76:0/1845128326 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32bc1e48e0 con 0x7f32bc0c50c0 2023-11-09T03:55:31.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.437+0000 7f32b17fa640 1 -- 172.21.15.76:0/1845128326 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b40146c0 con 0x7f32bc0c50c0 2023-11-09T03:55:31.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.437+0000 7f32c2afc640 1 -- 172.21.15.76:0/1845128326 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32bc15a340 con 0x7f32bc0c50c0 2023-11-09T03:55:31.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.438+0000 7f32c2afc640 1 -- 172.21.15.76:0/1845128326 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32bc0684b0 con 0x7f32bc0c50c0 2023-11-09T03:55:31.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.439+0000 7f32b17fa640 1 -- 172.21.15.76:0/1845128326 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f32b4014820 con 0x7f32bc0c50c0 2023-11-09T03:55:31.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.439+0000 7f32b17fa640 1 --2- 172.21.15.76:0/1845128326 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f32a806ae70 0x7f32a806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:31.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.439+0000 7f32b17fa640 1 -- 172.21.15.76:0/1845128326 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8425+0+0 (secure 0 0 0) 0x7f32b408ade0 con 0x7f32bc0c50c0 2023-11-09T03:55:31.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.439+0000 7f32c0898640 1 --2- 172.21.15.76:0/1845128326 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f32a806ae70 0x7f32a806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:31.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.440+0000 7f32c0898640 1 --2- 172.21.15.76:0/1845128326 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f32a806ae70 0x7f32a806d330 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f32b8036000 tx=0x7f32b8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:31.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.443+0000 7f32b17fa640 1 -- 172.21.15.76:0/1845128326 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f32b4055450 con 0x7f32bc0c50c0 2023-11-09T03:55:31.677 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.677+0000 7f32c2afc640 1 -- 172.21.15.76:0/1845128326 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:1"} v 0) v1 -- 0x7f32bc1e2ff0 con 0x7f32bc0c50c0 2023-11-09T03:55:31.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.681+0000 7f32b17fa640 1 -- 172.21.15.76:0/1845128326 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:1"}]=0 failed mds gid 9024 v136) v1 ==== 97+0+0 (secure 0 0 0) 0x7f32b4018020 con 0x7f32bc0c50c0 2023-11-09T03:55:31.680 INFO:teuthology.orchestra.run.smithi076.stderr:failed mds gid 9024 2023-11-09T03:55:31.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.683+0000 7f32c2afc640 1 -- 172.21.15.76:0/1845128326 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f32a806ae70 msgr2=0x7f32a806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:31.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.683+0000 7f32c2afc640 1 --2- 172.21.15.76:0/1845128326 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f32a806ae70 0x7f32a806d330 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f32b8036000 tx=0x7f32b8008030 comp rx=0 tx=0).stop 2023-11-09T03:55:31.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.683+0000 7f32c2afc640 1 -- 172.21.15.76:0/1845128326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32bc0c50c0 msgr2=0x7f32bc0c25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:31.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.683+0000 7f32c2afc640 1 --2- 172.21.15.76:0/1845128326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32bc0c50c0 0x7f32bc0c25c0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f32b400efc0 tx=0x7f32b400c490 comp rx=0 tx=0).stop 2023-11-09T03:55:31.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.683+0000 7f32c2afc640 1 -- 172.21.15.76:0/1845128326 shutdown_connections 2023-11-09T03:55:31.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.683+0000 7f32c2afc640 1 --2- 172.21.15.76:0/1845128326 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f32a806ae70 0x7f32a806d330 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:31.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.683+0000 7f32c2afc640 1 --2- 172.21.15.76:0/1845128326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32bc0c50c0 0x7f32bc0c25c0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:31.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.683+0000 7f32c2afc640 1 --2- 172.21.15.76:0/1845128326 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32bc0addd0 0x7f32bc0c44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:31.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.683+0000 7f32c2afc640 1 --2- 172.21.15.76:0/1845128326 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32bc06c220 0x7f32bc1e4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:31.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.683+0000 7f32c2afc640 1 -- 172.21.15.76:0/1845128326 >> 172.21.15.76:0/1845128326 conn(0x7f32bc0cb530 msgr2=0x7f32bc158ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:31.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.683+0000 7f32c2afc640 1 -- 172.21.15.76:0/1845128326 shutdown_connections 2023-11-09T03:55:31.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.683+0000 7f32c2afc640 1 -- 172.21.15.76:0/1845128326 wait complete. 2023-11-09T03:55:31.695 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-09T03:55:31.695 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-09T03:55:31.696 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-09T03:55:31.696 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:55:31.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:55:31.697 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi076 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2023-11-09T03:55:31.697 INFO:tasks.ceph.mds.a:Stopped 2023-11-09T03:55:31.698 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-09T03:55:31.700 INFO:tasks.ceph.mds.a:Started 2023-11-09T03:55:31.700 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:55:31.775 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2023-11-09T03:55:31.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.802+0000 7f5dbd3f5640 1 Processor -- start 2023-11-09T03:55:31.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.803+0000 7f5dbd3f5640 1 -- start start 2023-11-09T03:55:31.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.803+0000 7f5dbd3f5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5db8159e40 0x7f5db80b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:31.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.803+0000 7f5dbd3f5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5db8070700 0x7f5db80b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:31.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.803+0000 7f5dbd3f5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db8071500 con 0x7f5db806bb60 2023-11-09T03:55:31.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.803+0000 7f5dbd3f5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5db815dae0 con 0x7f5db8070700 2023-11-09T03:55:31.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.803+0000 7f5dbd3f5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db8070c60 con 0x7f5db8159e40 2023-11-09T03:55:31.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.804+0000 7f5db6ffd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5db8159e40 0x7f5db80b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:31.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.804+0000 7f5db6ffd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5db8159e40 0x7f5db80b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.76:49888/0 (socket says 172.21.15.76:49888) 2023-11-09T03:55:31.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.804+0000 7f5db6ffd640 1 -- 172.21.15.76:0/4161871216 learned_addr learned my addr 172.21.15.76:0/4161871216 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:31.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.804+0000 7f5db77fe640 1 --2- 172.21.15.76:0/4161871216 >> v2:172.21.15.113:3300/0 conn(0x7f5db8070700 0x7f5db80b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:31.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.804+0000 7f5db6ffd640 1 -- 172.21.15.76:0/4161871216 >> v2:172.21.15.113:3300/0 conn(0x7f5db8070700 msgr2=0x7f5db80b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:31.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.804+0000 7f5db6ffd640 1 --2- 172.21.15.76:0/4161871216 >> v2:172.21.15.113:3300/0 conn(0x7f5db8070700 0x7f5db80b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:31.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.804+0000 7f5db6ffd640 1 -- 172.21.15.76:0/4161871216 >> v1:172.21.15.113:6789/0 conn(0x7f5db806bb60 legacy=0x7f5db806fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:31.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.804+0000 7f5db6ffd640 1 -- 172.21.15.76:0/4161871216 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db8070f40 con 0x7f5db8159e40 2023-11-09T03:55:31.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.805+0000 7f5db6ffd640 1 --2- 172.21.15.76:0/4161871216 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5db8159e40 0x7f5db80b0530 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009870 tx=0x7f5dac02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9b2f2bf9ff5ebb5 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:31.809 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.805+0000 7f5db5ffb640 1 -- 172.21.15.76:0/4161871216 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac031640 con 0x7f5db8159e40 2023-11-09T03:55:31.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.805+0000 7f5db5ffb640 1 -- 172.21.15.76:0/4161871216 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dac0317a0 con 0x7f5db8159e40 2023-11-09T03:55:31.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.805+0000 7f5db5ffb640 1 -- 172.21.15.76:0/4161871216 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac031b30 con 0x7f5db8159e40 2023-11-09T03:55:31.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.805+0000 7f5dbd3f5640 1 -- 172.21.15.76:0/4161871216 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5db8159e40 msgr2=0x7f5db80b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:31.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.805+0000 7f5dbd3f5640 1 --2- 172.21.15.76:0/4161871216 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5db8159e40 0x7f5db80b0530 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009870 tx=0x7f5dac02b8f0 comp rx=0 tx=0).stop 2023-11-09T03:55:31.810 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.805+0000 7f5dbd3f5640 1 -- 172.21.15.76:0/4161871216 shutdown_connections 2023-11-09T03:55:31.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.805+0000 7f5dbd3f5640 1 --2- 172.21.15.76:0/4161871216 >> v2:172.21.15.113:3300/0 conn(0x7f5db8070700 0x7f5db80b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:31.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.805+0000 7f5dbd3f5640 1 --2- 172.21.15.76:0/4161871216 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5db8159e40 0x7f5db80b0530 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:31.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.805+0000 7f5dbd3f5640 1 -- 172.21.15.76:0/4161871216 >> 172.21.15.76:0/4161871216 conn(0x7f5db80cb530 msgr2=0x7f5db80cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:31.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.806+0000 7f5dbd3f5640 1 -- 172.21.15.76:0/4161871216 shutdown_connections 2023-11-09T03:55:31.811 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.806+0000 7f5dbd3f5640 1 -- 172.21.15.76:0/4161871216 wait complete. 2023-11-09T03:55:31.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.807+0000 7f5dbd3f5640 1 Processor -- start 2023-11-09T03:55:31.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.807+0000 7f5dbd3f5640 1 -- start start 2023-11-09T03:55:31.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.807+0000 7f5dbd3f5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5db806bb60 0x7f5db80c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:31.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.808+0000 7f5dbd3f5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5db8070700 0x7f5db80c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:31.812 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.808+0000 7f5dbd3f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5db8159e40 0x7f5db81f5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:31.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.808+0000 7f5dbd3f5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db81f7270 con 0x7f5db806bb60 2023-11-09T03:55:31.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.808+0000 7f5dbd3f5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db80c8fb0 con 0x7f5db8159e40 2023-11-09T03:55:31.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.808+0000 7f5dbd3f5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db81f6270 con 0x7f5db8070700 2023-11-09T03:55:31.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.808+0000 7f5db6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5db806bb60 0x7f5db80c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:31.813 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.808+0000 7f5db6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5db806bb60 0x7f5db80c6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50186/0 (socket says 172.21.15.76:50186) 2023-11-09T03:55:31.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.808+0000 7f5db6ffd640 1 -- 172.21.15.76:0/2624304149 learned_addr learned my addr 172.21.15.76:0/2624304149 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:31.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.808+0000 7f5db77fe640 1 --2- 172.21.15.76:0/2624304149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5db8159e40 0x7f5db81f5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:31.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.808+0000 7f5db67fc640 1 --2- 172.21.15.76:0/2624304149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5db8070700 0x7f5db80c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:31.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.808+0000 7f5db6ffd640 1 -- 172.21.15.76:0/2624304149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5db8070700 msgr2=0x7f5db80c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:31.814 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.808+0000 7f5db6ffd640 1 --2- 172.21.15.76:0/2624304149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5db8070700 0x7f5db80c8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:31.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.808+0000 7f5db6ffd640 1 -- 172.21.15.76:0/2624304149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5db8159e40 msgr2=0x7f5db81f5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:31.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.808+0000 7f5db6ffd640 1 --2- 172.21.15.76:0/2624304149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5db8159e40 0x7f5db81f5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:31.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.808+0000 7f5db6ffd640 1 -- 172.21.15.76:0/2624304149 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dac0094d0 con 0x7f5db806bb60 2023-11-09T03:55:31.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.808+0000 7f5db6ffd640 1 --2- 172.21.15.76:0/2624304149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5db806bb60 0x7f5db80c6720 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009870 tx=0x7f5dac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:31.815 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.808+0000 7f5d9f7fe640 1 -- 172.21.15.76:0/2624304149 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac03a930 con 0x7f5db806bb60 2023-11-09T03:55:31.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.808+0000 7f5d9f7fe640 1 -- 172.21.15.76:0/2624304149 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dac03aa90 con 0x7f5db806bb60 2023-11-09T03:55:31.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.808+0000 7f5dbd3f5640 1 -- 172.21.15.76:0/2624304149 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db81f6660 con 0x7f5db806bb60 2023-11-09T03:55:31.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.808+0000 7f5d9f7fe640 1 -- 172.21.15.76:0/2624304149 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac03ae40 con 0x7f5db806bb60 2023-11-09T03:55:31.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.808+0000 7f5dbd3f5640 1 -- 172.21.15.76:0/2624304149 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db81f6c20 con 0x7f5db806bb60 2023-11-09T03:55:31.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.809+0000 7f5dbd3f5640 1 -- 172.21.15.76:0/2624304149 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db80684b0 con 0x7f5db806bb60 2023-11-09T03:55:31.816 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.810+0000 7f5d9f7fe640 1 -- 172.21.15.76:0/2624304149 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5dac02f050 con 0x7f5db806bb60 2023-11-09T03:55:31.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.810+0000 7f5d9f7fe640 1 --2- 172.21.15.76:0/2624304149 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5da406ae70 0x7f5da406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:31.817 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.810+0000 7f5d9f7fe640 1 -- 172.21.15.76:0/2624304149 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8511+0+0 (secure 0 0 0) 0x7f5dac0abfc0 con 0x7f5db806bb60 2023-11-09T03:55:31.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.810+0000 7f5db67fc640 1 --2- 172.21.15.76:0/2624304149 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5da406ae70 0x7f5da406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:31.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.811+0000 7f5db67fc640 1 --2- 172.21.15.76:0/2624304149 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5da406ae70 0x7f5da406d330 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5db81f0ac0 tx=0x7f5da8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:31.818 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:31.815+0000 7f5d9f7fe640 1 -- 172.21.15.76:0/2624304149 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5dac0deca0 con 0x7f5db806bb60 2023-11-09T03:55:32.017 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:32.018+0000 7f5dbd3f5640 1 -- 172.21.15.76:0/2624304149 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5db80711f0 con 0x7f5db806bb60 2023-11-09T03:55:32.018 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:32.018+0000 7f5d9f7fe640 1 -- 172.21.15.76:0/2624304149 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) v1 ==== 95+0+4142 (secure 0 0 0) 0x7f5dac076560 con 0x7f5db806bb60 2023-11-09T03:55:32.018 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:55:32.019 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":137,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:55:31.686614+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":6698},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6698":{"gid":6698,"name":"b","rank":1,"incarnation":137,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:55:32.019 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 137 2023-11-09T03:55:32.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:32.020+0000 7f5dbd3f5640 1 -- 172.21.15.76:0/2624304149 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5da406ae70 msgr2=0x7f5da406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:32.020 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:32.020+0000 7f5dbd3f5640 1 --2- 172.21.15.76:0/2624304149 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5da406ae70 0x7f5da406d330 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5db81f0ac0 tx=0x7f5da8009040 comp rx=0 tx=0).stop 2023-11-09T03:55:32.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:32.020+0000 7f5dbd3f5640 1 -- 172.21.15.76:0/2624304149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5db806bb60 msgr2=0x7f5db80c6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:32.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:32.020+0000 7f5dbd3f5640 1 --2- 172.21.15.76:0/2624304149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5db806bb60 0x7f5db80c6720 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009870 tx=0x7f5dac0046c0 comp rx=0 tx=0).stop 2023-11-09T03:55:32.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:32.020+0000 7f5dbd3f5640 1 -- 172.21.15.76:0/2624304149 shutdown_connections 2023-11-09T03:55:32.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:32.020+0000 7f5dbd3f5640 1 --2- 172.21.15.76:0/2624304149 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5da406ae70 0x7f5da406d330 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:32.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:32.020+0000 7f5dbd3f5640 1 --2- 172.21.15.76:0/2624304149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5db8159e40 0x7f5db81f5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:32.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:32.020+0000 7f5dbd3f5640 1 --2- 172.21.15.76:0/2624304149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5db8070700 0x7f5db80c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:32.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:32.020+0000 7f5dbd3f5640 1 --2- 172.21.15.76:0/2624304149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5db806bb60 0x7f5db80c6720 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:32.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:32.020+0000 7f5dbd3f5640 1 -- 172.21.15.76:0/2624304149 >> 172.21.15.76:0/2624304149 conn(0x7f5db80cb530 msgr2=0x7f5db815c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:32.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:32.020+0000 7f5dbd3f5640 1 -- 172.21.15.76:0/2624304149 shutdown_connections 2023-11-09T03:55:32.022 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:32.020+0000 7f5dbd3f5640 1 -- 172.21.15.76:0/2624304149 wait complete. 2023-11-09T03:55:32.031 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-11-09T03:55:32.038 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~13s 2023-11-09T03:55:32.851 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:55:32.854 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:55:37.032 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:55:37.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.142+0000 7f268d0c2640 1 Processor -- start 2023-11-09T03:55:37.142 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.142+0000 7f268d0c2640 1 -- start start 2023-11-09T03:55:37.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.143+0000 7f268d0c2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2688159e40 0x7f26880b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:37.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.143+0000 7f268d0c2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2688070700 0x7f26880b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:37.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.143+0000 7f268d0c2640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2688071500 con 0x7f268806bb60 2023-11-09T03:55:37.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.143+0000 7f268d0c2640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f268815dae0 con 0x7f2688159e40 2023-11-09T03:55:37.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.143+0000 7f268d0c2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2688070c60 con 0x7f2688070700 2023-11-09T03:55:37.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.143+0000 7f2686d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2688159e40 0x7f26880b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:37.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.143+0000 7f2686d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2688159e40 0x7f26880b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58112/0 (socket says 172.21.15.76:58112) 2023-11-09T03:55:37.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.143+0000 7f2686d9d640 1 -- 172.21.15.76:0/3554822666 learned_addr learned my addr 172.21.15.76:0/3554822666 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:37.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.143+0000 7f268759e640 1 --2- 172.21.15.76:0/3554822666 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2688070700 0x7f26880b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:37.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.143+0000 7f2686d9d640 1 -- 172.21.15.76:0/3554822666 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2688070700 msgr2=0x7f26880b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:37.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.143+0000 7f2686d9d640 1 --2- 172.21.15.76:0/3554822666 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2688070700 0x7f26880b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:37.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.143+0000 7f2686d9d640 1 -- 172.21.15.76:0/3554822666 >> v1:172.21.15.113:6789/0 conn(0x7f268806bb60 legacy=0x7f268806fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:37.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.143+0000 7f2686d9d640 1 -- 172.21.15.76:0/3554822666 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2688070f40 con 0x7f2688159e40 2023-11-09T03:55:37.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.143+0000 7f268759e640 1 --2- 172.21.15.76:0/3554822666 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2688070700 0x7f26880b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:55:37.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.144+0000 7f2686d9d640 1 --2- 172.21.15.76:0/3554822666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2688159e40 0x7f26880b0530 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f2678009e50 tx=0x7f267802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34144b05f0427a74 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:37.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.144+0000 7f2685d9b640 1 -- 172.21.15.76:0/3554822666 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267802c9f0 con 0x7f2688159e40 2023-11-09T03:55:37.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.144+0000 7f2685d9b640 1 -- 172.21.15.76:0/3554822666 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267802cb50 con 0x7f2688159e40 2023-11-09T03:55:37.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.144+0000 7f2685d9b640 1 -- 172.21.15.76:0/3554822666 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26780343f0 con 0x7f2688159e40 2023-11-09T03:55:37.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.145+0000 7f268d0c2640 1 -- 172.21.15.76:0/3554822666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2688159e40 msgr2=0x7f26880b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:37.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.145+0000 7f268d0c2640 1 --2- 172.21.15.76:0/3554822666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2688159e40 0x7f26880b0530 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f2678009e50 tx=0x7f267802be50 comp rx=0 tx=0).stop 2023-11-09T03:55:37.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.145+0000 7f268d0c2640 1 -- 172.21.15.76:0/3554822666 shutdown_connections 2023-11-09T03:55:37.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.145+0000 7f268d0c2640 1 --2- 172.21.15.76:0/3554822666 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2688070700 0x7f26880b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:37.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.145+0000 7f268d0c2640 1 --2- 172.21.15.76:0/3554822666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2688159e40 0x7f26880b0530 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:37.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.145+0000 7f268d0c2640 1 -- 172.21.15.76:0/3554822666 >> 172.21.15.76:0/3554822666 conn(0x7f26880cb530 msgr2=0x7f26880cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:37.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.145+0000 7f268d0c2640 1 -- 172.21.15.76:0/3554822666 shutdown_connections 2023-11-09T03:55:37.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.145+0000 7f268d0c2640 1 -- 172.21.15.76:0/3554822666 wait complete. 2023-11-09T03:55:37.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.146+0000 7f268d0c2640 1 Processor -- start 2023-11-09T03:55:37.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.146+0000 7f268d0c2640 1 -- start start 2023-11-09T03:55:37.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.147+0000 7f268d0c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f268806bb60 0x7f268814be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:37.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.147+0000 7f268d0c2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2688070700 0x7f26880a8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:37.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.147+0000 7f268d0c2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2688159e40 0x7f26880c4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:37.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.147+0000 7f268d0c2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26880c7c00 con 0x7f268806bb60 2023-11-09T03:55:37.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.147+0000 7f268d0c2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26880c7ee0 con 0x7f2688070700 2023-11-09T03:55:37.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.147+0000 7f268d0c2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26880c81c0 con 0x7f2688159e40 2023-11-09T03:55:37.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.147+0000 7f2686d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f268806bb60 0x7f268814be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:37.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.147+0000 7f2686d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f268806bb60 0x7f268814be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58116/0 (socket says 172.21.15.76:58116) 2023-11-09T03:55:37.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.147+0000 7f2686d9d640 1 -- 172.21.15.76:0/1240125518 learned_addr learned my addr 172.21.15.76:0/1240125518 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:37.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.147+0000 7f268759e640 1 --2- 172.21.15.76:0/1240125518 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2688159e40 0x7f26880c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:37.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.147+0000 7f268659c640 1 --2- 172.21.15.76:0/1240125518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2688070700 0x7f26880a8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:37.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.148+0000 7f2686d9d640 1 -- 172.21.15.76:0/1240125518 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2688159e40 msgr2=0x7f26880c4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:37.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.148+0000 7f2686d9d640 1 --2- 172.21.15.76:0/1240125518 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2688159e40 0x7f26880c4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:37.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.148+0000 7f2686d9d640 1 -- 172.21.15.76:0/1240125518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2688070700 msgr2=0x7f26880a8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:37.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.148+0000 7f2686d9d640 1 --2- 172.21.15.76:0/1240125518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2688070700 0x7f26880a8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:37.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.148+0000 7f2686d9d640 1 -- 172.21.15.76:0/1240125518 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2678009b30 con 0x7f268806bb60 2023-11-09T03:55:37.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.148+0000 7f268759e640 1 --2- 172.21.15.76:0/1240125518 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2688159e40 0x7f26880c4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:55:37.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.148+0000 7f268659c640 1 --2- 172.21.15.76:0/1240125518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2688070700 0x7f26880a8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:55:37.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.148+0000 7f2686d9d640 1 --2- 172.21.15.76:0/1240125518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f268806bb60 0x7f268814be90 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f2678009e50 tx=0x7f2678004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:37.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.148+0000 7f266f7fe640 1 -- 172.21.15.76:0/1240125518 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267803e450 con 0x7f268806bb60 2023-11-09T03:55:37.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.148+0000 7f266f7fe640 1 -- 172.21.15.76:0/1240125518 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267803e5f0 con 0x7f268806bb60 2023-11-09T03:55:37.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.148+0000 7f268d0c2640 1 -- 172.21.15.76:0/1240125518 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26880c84a0 con 0x7f268806bb60 2023-11-09T03:55:37.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.148+0000 7f266f7fe640 1 -- 172.21.15.76:0/1240125518 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267803e9a0 con 0x7f268806bb60 2023-11-09T03:55:37.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.148+0000 7f268d0c2640 1 -- 172.21.15.76:0/1240125518 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f268814c3d0 con 0x7f268806bb60 2023-11-09T03:55:37.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.149+0000 7f268d0c2640 1 -- 172.21.15.76:0/1240125518 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26880c8960 con 0x7f268806bb60 2023-11-09T03:55:37.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.149+0000 7f266f7fe640 1 -- 172.21.15.76:0/1240125518 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f267803eb00 con 0x7f268806bb60 2023-11-09T03:55:37.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.150+0000 7f266f7fe640 1 --2- 172.21.15.76:0/1240125518 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f267406ae70 0x7f267406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:37.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.150+0000 7f266f7fe640 1 -- 172.21.15.76:0/1240125518 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8511+0+0 (secure 0 0 0) 0x7f26780ad0c0 con 0x7f268806bb60 2023-11-09T03:55:37.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.150+0000 7f268659c640 1 --2- 172.21.15.76:0/1240125518 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f267406ae70 0x7f267406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:37.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.151+0000 7f268659c640 1 --2- 172.21.15.76:0/1240125518 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f267406ae70 0x7f267406d330 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f2688159510 tx=0x7f267c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:37.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.154+0000 7f266f7fe640 1 -- 172.21.15.76:0/1240125518 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2678077700 con 0x7f268806bb60 2023-11-09T03:55:37.353 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.354+0000 7f268d0c2640 1 -- 172.21.15.76:0/1240125518 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26880684b0 con 0x7f268806bb60 2023-11-09T03:55:37.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.354+0000 7f266f7fe640 1 -- 172.21.15.76:0/1240125518 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) v1 ==== 95+0+4938 (secure 0 0 0) 0x7f267803edb0 con 0x7f268806bb60 2023-11-09T03:55:37.354 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:55:37.354 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":139,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":9525,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1713748386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1713748386},{"type":"v1","addr":"172.21.15.76:6835","nonce":1713748386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":138}],"filesystems":[{"mdsmap":{"epoch":139,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:55:32.694340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":6698},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6698":{"gid":6698,"name":"b","rank":1,"incarnation":137,"state":"up:reconnect","state_seq":14,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:55:37.355 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 139 2023-11-09T03:55:37.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.356+0000 7f268d0c2640 1 -- 172.21.15.76:0/1240125518 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f267406ae70 msgr2=0x7f267406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:37.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.356+0000 7f268d0c2640 1 --2- 172.21.15.76:0/1240125518 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f267406ae70 0x7f267406d330 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f2688159510 tx=0x7f267c009040 comp rx=0 tx=0).stop 2023-11-09T03:55:37.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.356+0000 7f268d0c2640 1 -- 172.21.15.76:0/1240125518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f268806bb60 msgr2=0x7f268814be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:37.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.356+0000 7f268d0c2640 1 --2- 172.21.15.76:0/1240125518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f268806bb60 0x7f268814be90 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f2678009e50 tx=0x7f2678004a40 comp rx=0 tx=0).stop 2023-11-09T03:55:37.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.356+0000 7f268d0c2640 1 -- 172.21.15.76:0/1240125518 shutdown_connections 2023-11-09T03:55:37.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.356+0000 7f268d0c2640 1 --2- 172.21.15.76:0/1240125518 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f267406ae70 0x7f267406d330 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:37.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.357+0000 7f268d0c2640 1 --2- 172.21.15.76:0/1240125518 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2688159e40 0x7f26880c4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:37.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.357+0000 7f268d0c2640 1 --2- 172.21.15.76:0/1240125518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2688070700 0x7f26880a8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:37.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.357+0000 7f268d0c2640 1 --2- 172.21.15.76:0/1240125518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f268806bb60 0x7f268814be90 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:37.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.357+0000 7f268d0c2640 1 -- 172.21.15.76:0/1240125518 >> 172.21.15.76:0/1240125518 conn(0x7f26880cb530 msgr2=0x7f268815c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:37.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.357+0000 7f268d0c2640 1 -- 172.21.15.76:0/1240125518 shutdown_connections 2023-11-09T03:55:37.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.357+0000 7f268d0c2640 1 -- 172.21.15.76:0/1240125518 wait complete. 2023-11-09T03:55:37.367 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-09T03:55:37.367 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:55:37.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.470+0000 7f1d991a9640 1 Processor -- start 2023-11-09T03:55:37.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.470+0000 7f1d991a9640 1 -- start start 2023-11-09T03:55:37.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.471+0000 7f1d991a9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1d94159e40 0x7f1d940b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:37.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.471+0000 7f1d991a9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1d94070700 0x7f1d940b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:37.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.471+0000 7f1d991a9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d94071500 con 0x7f1d9406bb60 2023-11-09T03:55:37.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.471+0000 7f1d991a9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d9415dae0 con 0x7f1d94070700 2023-11-09T03:55:37.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.471+0000 7f1d991a9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d94070c60 con 0x7f1d94159e40 2023-11-09T03:55:37.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.471+0000 7f1d9359e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1d94070700 0x7f1d940b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:37.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.471+0000 7f1d9359e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1d94070700 0x7f1d940b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58126/0 (socket says 172.21.15.76:58126) 2023-11-09T03:55:37.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.471+0000 7f1d9359e640 1 -- 172.21.15.76:0/608361547 learned_addr learned my addr 172.21.15.76:0/608361547 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:37.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.471+0000 7f1d8bfff640 1 --1- 172.21.15.76:0/608361547 >> v1:172.21.15.76:6789/0 conn(0x7f1d9406bb60 0x7f1d9406fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47548/0 (socket says 172.21.15.76:47548) 2023-11-09T03:55:37.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.471+0000 7f1d92d9d640 1 --2- 172.21.15.76:0/608361547 >> v2:172.21.15.113:3300/0 conn(0x7f1d94159e40 0x7f1d940b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:37.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.471+0000 7f1d9359e640 1 -- 172.21.15.76:0/608361547 >> v2:172.21.15.113:3300/0 conn(0x7f1d94159e40 msgr2=0x7f1d940b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:37.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.471+0000 7f1d9359e640 1 --2- 172.21.15.76:0/608361547 >> v2:172.21.15.113:3300/0 conn(0x7f1d94159e40 0x7f1d940b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:37.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.471+0000 7f1d9359e640 1 -- 172.21.15.76:0/608361547 >> v1:172.21.15.76:6789/0 conn(0x7f1d9406bb60 legacy=0x7f1d9406fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:37.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.471+0000 7f1d9359e640 1 -- 172.21.15.76:0/608361547 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d94070f40 con 0x7f1d94070700 2023-11-09T03:55:37.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.471+0000 7f1d9259c640 1 -- 172.21.15.76:0/608361547 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1500144448 0 0) 0x7f1d94071500 con 0x7f1d9406bb60 2023-11-09T03:55:37.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.472+0000 7f1d9359e640 1 --2- 172.21.15.76:0/608361547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d94070700 0x7f1d940b0a70 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f1d940ce7f0 tx=0x7f1d8c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87902a03752e461c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:37.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.472+0000 7f1d9259c640 1 -- 172.21.15.76:0/608361547 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c02c9f0 con 0x7f1d94070700 2023-11-09T03:55:37.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.472+0000 7f1d9259c640 1 -- 172.21.15.76:0/608361547 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8c02cb50 con 0x7f1d94070700 2023-11-09T03:55:37.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.472+0000 7f1d9259c640 1 -- 172.21.15.76:0/608361547 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c0343f0 con 0x7f1d94070700 2023-11-09T03:55:37.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.472+0000 7f1d991a9640 1 -- 172.21.15.76:0/608361547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d94070700 msgr2=0x7f1d940b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:37.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.472+0000 7f1d991a9640 1 --2- 172.21.15.76:0/608361547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d94070700 0x7f1d940b0a70 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f1d940ce7f0 tx=0x7f1d8c02be50 comp rx=0 tx=0).stop 2023-11-09T03:55:37.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.472+0000 7f1d991a9640 1 -- 172.21.15.76:0/608361547 shutdown_connections 2023-11-09T03:55:37.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.472+0000 7f1d991a9640 1 --2- 172.21.15.76:0/608361547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d94070700 0x7f1d940b0a70 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:37.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.472+0000 7f1d991a9640 1 --2- 172.21.15.76:0/608361547 >> v2:172.21.15.113:3300/0 conn(0x7f1d94159e40 0x7f1d940b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:37.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.472+0000 7f1d991a9640 1 -- 172.21.15.76:0/608361547 >> 172.21.15.76:0/608361547 conn(0x7f1d940cb530 msgr2=0x7f1d940cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:37.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.472+0000 7f1d991a9640 1 -- 172.21.15.76:0/608361547 shutdown_connections 2023-11-09T03:55:37.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.472+0000 7f1d991a9640 1 -- 172.21.15.76:0/608361547 wait complete. 2023-11-09T03:55:37.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.474+0000 7f1d991a9640 1 Processor -- start 2023-11-09T03:55:37.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.474+0000 7f1d991a9640 1 -- start start 2023-11-09T03:55:37.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.474+0000 7f1d991a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d9406bb60 0x7f1d9415cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:37.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.474+0000 7f1d991a9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d94070700 0x7f1d940c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:37.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.474+0000 7f1d991a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d94159e40 0x7f1d940c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:37.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.474+0000 7f1d991a9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9415d2c0 con 0x7f1d94070700 2023-11-09T03:55:37.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.474+0000 7f1d991a9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d940c66d0 con 0x7f1d9406bb60 2023-11-09T03:55:37.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.474+0000 7f1d991a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d940c69b0 con 0x7f1d94159e40 2023-11-09T03:55:37.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.475+0000 7f1d8bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d94070700 0x7f1d940c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:37.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.475+0000 7f1d8bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d94070700 0x7f1d940c5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58130/0 (socket says 172.21.15.76:58130) 2023-11-09T03:55:37.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.475+0000 7f1d8bfff640 1 -- 172.21.15.76:0/3050168331 learned_addr learned my addr 172.21.15.76:0/3050168331 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:37.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.475+0000 7f1d92d9d640 1 --2- 172.21.15.76:0/3050168331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d9406bb60 0x7f1d9415cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:37.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.475+0000 7f1d9359e640 1 --2- 172.21.15.76:0/3050168331 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d94159e40 0x7f1d940c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:37.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.475+0000 7f1d8bfff640 1 -- 172.21.15.76:0/3050168331 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d94159e40 msgr2=0x7f1d940c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:37.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.475+0000 7f1d8bfff640 1 --2- 172.21.15.76:0/3050168331 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d94159e40 0x7f1d940c60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:37.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.475+0000 7f1d8bfff640 1 -- 172.21.15.76:0/3050168331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d9406bb60 msgr2=0x7f1d9415cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:37.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.475+0000 7f1d8bfff640 1 --2- 172.21.15.76:0/3050168331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d9406bb60 0x7f1d9415cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:37.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.475+0000 7f1d8bfff640 1 -- 172.21.15.76:0/3050168331 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8c009b30 con 0x7f1d94070700 2023-11-09T03:55:37.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.475+0000 7f1d92d9d640 1 --2- 172.21.15.76:0/3050168331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d9406bb60 0x7f1d9415cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:55:37.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.475+0000 7f1d9359e640 1 --2- 172.21.15.76:0/3050168331 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d94159e40 0x7f1d940c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:55:37.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.475+0000 7f1d8bfff640 1 --2- 172.21.15.76:0/3050168331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d94070700 0x7f1d940c5b80 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c00e6e0 tx=0x7f1d7c00eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:37.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.475+0000 7f1d8b7fe640 1 -- 172.21.15.76:0/3050168331 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c006bf0 con 0x7f1d94070700 2023-11-09T03:55:37.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.475+0000 7f1d8b7fe640 1 -- 172.21.15.76:0/3050168331 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7c006d50 con 0x7f1d94070700 2023-11-09T03:55:37.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.475+0000 7f1d8b7fe640 1 -- 172.21.15.76:0/3050168331 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c009310 con 0x7f1d94070700 2023-11-09T03:55:37.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.475+0000 7f1d991a9640 1 -- 172.21.15.76:0/3050168331 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d940c6c90 con 0x7f1d94070700 2023-11-09T03:55:37.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.475+0000 7f1d991a9640 1 -- 172.21.15.76:0/3050168331 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d940c7250 con 0x7f1d94070700 2023-11-09T03:55:37.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.476+0000 7f1d991a9640 1 -- 172.21.15.76:0/3050168331 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d940684b0 con 0x7f1d94070700 2023-11-09T03:55:37.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.477+0000 7f1d8b7fe640 1 -- 172.21.15.76:0/3050168331 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1d7c009470 con 0x7f1d94070700 2023-11-09T03:55:37.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.477+0000 7f1d8b7fe640 1 --2- 172.21.15.76:0/3050168331 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1d7406ae70 0x7f1d7406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:37.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.477+0000 7f1d8b7fe640 1 -- 172.21.15.76:0/3050168331 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8511+0+0 (secure 0 0 0) 0x7f1d7c08b6a0 con 0x7f1d94070700 2023-11-09T03:55:37.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.477+0000 7f1d92d9d640 1 --2- 172.21.15.76:0/3050168331 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1d7406ae70 0x7f1d7406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:37.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.478+0000 7f1d92d9d640 1 --2- 172.21.15.76:0/3050168331 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1d7406ae70 0x7f1d7406d330 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1d940c54e0 tx=0x7f1d84008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:37.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.481+0000 7f1d8b7fe640 1 -- 172.21.15.76:0/3050168331 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1d7c055b80 con 0x7f1d94070700 2023-11-09T03:55:37.680 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.681+0000 7f1d991a9640 1 -- 172.21.15.76:0/3050168331 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d940711f0 con 0x7f1d94070700 2023-11-09T03:55:37.681 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.681+0000 7f1d8b7fe640 1 -- 172.21.15.76:0/3050168331 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) v1 ==== 95+0+4938 (secure 0 0 0) 0x7f1d7c01a030 con 0x7f1d94070700 2023-11-09T03:55:37.681 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:55:37.682 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":139,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":9525,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1713748386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1713748386},{"type":"v1","addr":"172.21.15.76:6835","nonce":1713748386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":138}],"filesystems":[{"mdsmap":{"epoch":139,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:55:32.694340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":6698},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6698":{"gid":6698,"name":"b","rank":1,"incarnation":137,"state":"up:reconnect","state_seq":14,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:55:37.682 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 139 2023-11-09T03:55:37.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.683+0000 7f1d991a9640 1 -- 172.21.15.76:0/3050168331 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1d7406ae70 msgr2=0x7f1d7406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:37.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.683+0000 7f1d991a9640 1 --2- 172.21.15.76:0/3050168331 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1d7406ae70 0x7f1d7406d330 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1d940c54e0 tx=0x7f1d84008040 comp rx=0 tx=0).stop 2023-11-09T03:55:37.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.683+0000 7f1d991a9640 1 -- 172.21.15.76:0/3050168331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d94070700 msgr2=0x7f1d940c5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:37.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.683+0000 7f1d991a9640 1 --2- 172.21.15.76:0/3050168331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d94070700 0x7f1d940c5b80 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c00e6e0 tx=0x7f1d7c00eba0 comp rx=0 tx=0).stop 2023-11-09T03:55:37.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.683+0000 7f1d991a9640 1 -- 172.21.15.76:0/3050168331 shutdown_connections 2023-11-09T03:55:37.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.683+0000 7f1d991a9640 1 --2- 172.21.15.76:0/3050168331 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1d7406ae70 0x7f1d7406d330 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:37.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.683+0000 7f1d991a9640 1 --2- 172.21.15.76:0/3050168331 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d94159e40 0x7f1d940c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:37.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.683+0000 7f1d991a9640 1 --2- 172.21.15.76:0/3050168331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d94070700 0x7f1d940c5b80 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:37.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.683+0000 7f1d991a9640 1 --2- 172.21.15.76:0/3050168331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d9406bb60 0x7f1d9415cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:37.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.683+0000 7f1d991a9640 1 -- 172.21.15.76:0/3050168331 >> 172.21.15.76:0/3050168331 conn(0x7f1d940cb530 msgr2=0x7f1d940cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:37.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.683+0000 7f1d991a9640 1 -- 172.21.15.76:0/3050168331 shutdown_connections 2023-11-09T03:55:37.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:37.683+0000 7f1d991a9640 1 -- 172.21.15.76:0/3050168331 wait complete. 2023-11-09T03:55:38.096 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 139, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-09T03:48:57.383848+0000', 'modified': '2023-11-09T03:55:32.694340+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 119, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 8568, 'mds_1': 6698}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6698': {'gid': 6698, 'name': 'b', 'rank': 1, 'incarnation': 137, 'state': 'up:reconnect', 'state_seq': 14, 'addr': '172.21.15.113:6837/366683575', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6836', 'nonce': 366683575}, {'type': 'v1', 'addr': '172.21.15.113:6837', 'nonce': 366683575}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_8568': {'gid': 8568, 'name': 'c', 'rank': 0, 'incarnation': 110, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.76:6833/153465374', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6832', 'nonce': 153465374}, {'type': 'v1', 'addr': '172.21.15.76:6833', 'nonce': 153465374}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-09T03:55:38.096 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6698:up:reconnect 2023-11-09T03:55:38.750 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-11-09T03:55:39.097 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:55:39.210 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.211+0000 7f663db61640 1 Processor -- start 2023-11-09T03:55:39.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.211+0000 7f663db61640 1 -- start start 2023-11-09T03:55:39.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.211+0000 7f663db61640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f663815d100 0x7f66380ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:39.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.211+0000 7f663db61640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f663815d4e0 0x7f66380d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:39.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.211+0000 7f663db61640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f66380d3b60 0x7f66380d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:39.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.211+0000 7f663db61640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66380b32b0 con 0x7f66380d3b60 2023-11-09T03:55:39.213 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.211+0000 7f663db61640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66380cee10 con 0x7f663815d4e0 2023-11-09T03:55:39.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.211+0000 7f663db61640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66380b2980 con 0x7f663815d100 2023-11-09T03:55:39.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.212+0000 7f6637fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f66380d3b60 0x7f66380d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:39.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.212+0000 7f6637fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f66380d3b60 0x7f66380d1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58132/0 (socket says 172.21.15.76:58132) 2023-11-09T03:55:39.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.212+0000 7f6637fff640 1 -- 172.21.15.76:0/4186751952 learned_addr learned my addr 172.21.15.76:0/4186751952 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:39.214 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.212+0000 7f6636ffd640 1 --2- 172.21.15.76:0/4186751952 >> v2:172.21.15.113:3300/0 conn(0x7f663815d4e0 0x7f66380d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:39.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.212+0000 7f66377fe640 1 --2- 172.21.15.76:0/4186751952 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f663815d100 0x7f66380ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:39.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.212+0000 7f6637fff640 1 -- 172.21.15.76:0/4186751952 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f663815d100 msgr2=0x7f66380ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:39.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.212+0000 7f6637fff640 1 --2- 172.21.15.76:0/4186751952 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f663815d100 0x7f66380ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:39.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.212+0000 7f6637fff640 1 -- 172.21.15.76:0/4186751952 >> v2:172.21.15.113:3300/0 conn(0x7f663815d4e0 msgr2=0x7f66380d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:39.215 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.212+0000 7f6637fff640 1 --2- 172.21.15.76:0/4186751952 >> v2:172.21.15.113:3300/0 conn(0x7f663815d4e0 0x7f66380d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:39.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.212+0000 7f6637fff640 1 -- 172.21.15.76:0/4186751952 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66380b2c60 con 0x7f66380d3b60 2023-11-09T03:55:39.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.212+0000 7f66377fe640 1 --2- 172.21.15.76:0/4186751952 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f663815d100 0x7f66380ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:55:39.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.213+0000 7f6637fff640 1 --2- 172.21.15.76:0/4186751952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66380d3b60 0x7f66380d1a10 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f663808bf20 tx=0x7f663002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3767f6d8375ee037 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:39.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.213+0000 7f66367fc640 1 -- 172.21.15.76:0/4186751952 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663002ca70 con 0x7f66380d3b60 2023-11-09T03:55:39.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.213+0000 7f66367fc640 1 -- 172.21.15.76:0/4186751952 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663002cbd0 con 0x7f66380d3b60 2023-11-09T03:55:39.216 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.213+0000 7f66367fc640 1 -- 172.21.15.76:0/4186751952 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66300343f0 con 0x7f66380d3b60 2023-11-09T03:55:39.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.213+0000 7f663db61640 1 -- 172.21.15.76:0/4186751952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66380d3b60 msgr2=0x7f66380d1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:39.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.213+0000 7f663db61640 1 --2- 172.21.15.76:0/4186751952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66380d3b60 0x7f66380d1a10 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f663808bf20 tx=0x7f663002bed0 comp rx=0 tx=0).stop 2023-11-09T03:55:39.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.214+0000 7f663db61640 1 -- 172.21.15.76:0/4186751952 shutdown_connections 2023-11-09T03:55:39.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.214+0000 7f663db61640 1 --2- 172.21.15.76:0/4186751952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66380d3b60 0x7f66380d1a10 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:39.220 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.214+0000 7f663db61640 1 --2- 172.21.15.76:0/4186751952 >> v2:172.21.15.113:3300/0 conn(0x7f663815d4e0 0x7f66380d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:39.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.214+0000 7f663db61640 1 --2- 172.21.15.76:0/4186751952 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f663815d100 0x7f66380ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:39.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.214+0000 7f663db61640 1 -- 172.21.15.76:0/4186751952 >> 172.21.15.76:0/4186751952 conn(0x7f66380d2ef0 msgr2=0x7f66380d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:39.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.214+0000 7f663db61640 1 -- 172.21.15.76:0/4186751952 shutdown_connections 2023-11-09T03:55:39.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.214+0000 7f663db61640 1 -- 172.21.15.76:0/4186751952 wait complete. 2023-11-09T03:55:39.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.215+0000 7f663db61640 1 Processor -- start 2023-11-09T03:55:39.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.215+0000 7f663db61640 1 -- start start 2023-11-09T03:55:39.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.216+0000 7f663db61640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66380d3b60 0x7f66380cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:39.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.216+0000 7f663db61640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f663815d100 0x7f66380cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:39.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.216+0000 7f663db61640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f663815d4e0 0x7f66380cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:39.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.216+0000 7f663db61640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6638155230 con 0x7f66380d3b60 2023-11-09T03:55:39.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.216+0000 7f663db61640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66381545d0 con 0x7f663815d4e0 2023-11-09T03:55:39.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.216+0000 7f663db61640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6638154890 con 0x7f663815d100 2023-11-09T03:55:39.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.216+0000 7f66377fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66380d3b60 0x7f66380cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:39.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.216+0000 7f66377fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66380d3b60 0x7f66380cb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58136/0 (socket says 172.21.15.76:58136) 2023-11-09T03:55:39.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.216+0000 7f66377fe640 1 -- 172.21.15.76:0/2904271994 learned_addr learned my addr 172.21.15.76:0/2904271994 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:39.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.216+0000 7f6636ffd640 1 --2- 172.21.15.76:0/2904271994 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f663815d100 0x7f66380cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:39.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.216+0000 7f6637fff640 1 --2- 172.21.15.76:0/2904271994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f663815d4e0 0x7f66380cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:39.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.216+0000 7f66377fe640 1 -- 172.21.15.76:0/2904271994 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f663815d100 msgr2=0x7f66380cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:39.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.216+0000 7f66377fe640 1 --2- 172.21.15.76:0/2904271994 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f663815d100 0x7f66380cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:39.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.216+0000 7f66377fe640 1 -- 172.21.15.76:0/2904271994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f663815d4e0 msgr2=0x7f66380cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:39.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.216+0000 7f66377fe640 1 --2- 172.21.15.76:0/2904271994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f663815d4e0 0x7f66380cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:39.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.216+0000 7f66377fe640 1 -- 172.21.15.76:0/2904271994 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6630009b30 con 0x7f66380d3b60 2023-11-09T03:55:39.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.216+0000 7f6637fff640 1 --2- 172.21.15.76:0/2904271994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f663815d4e0 0x7f66380cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:55:39.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.216+0000 7f66377fe640 1 --2- 172.21.15.76:0/2904271994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66380d3b60 0x7f66380cb450 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f662800c7c0 tx=0x7f662800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:39.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.216+0000 7f661ffff640 1 -- 172.21.15.76:0/2904271994 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6628005b90 con 0x7f66380d3b60 2023-11-09T03:55:39.225 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.217+0000 7f661ffff640 1 -- 172.21.15.76:0/2904271994 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6628005cf0 con 0x7f66380d3b60 2023-11-09T03:55:39.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.217+0000 7f663db61640 1 -- 172.21.15.76:0/2904271994 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6638154b70 con 0x7f66380d3b60 2023-11-09T03:55:39.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.217+0000 7f661ffff640 1 -- 172.21.15.76:0/2904271994 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6628006250 con 0x7f66380d3b60 2023-11-09T03:55:39.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.217+0000 7f663db61640 1 -- 172.21.15.76:0/2904271994 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66380cb990 con 0x7f66380d3b60 2023-11-09T03:55:39.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.218+0000 7f663db61640 1 -- 172.21.15.76:0/2904271994 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66380684b0 con 0x7f66380d3b60 2023-11-09T03:55:39.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.218+0000 7f661ffff640 1 -- 172.21.15.76:0/2904271994 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f66280063b0 con 0x7f66380d3b60 2023-11-09T03:55:39.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.218+0000 7f661ffff640 1 --2- 172.21.15.76:0/2904271994 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f662406ae70 0x7f662406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:39.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.219+0000 7f661ffff640 1 -- 172.21.15.76:0/2904271994 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8511+0+0 (secure 0 0 0) 0x7f6628089f40 con 0x7f66380d3b60 2023-11-09T03:55:39.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.219+0000 7f6636ffd640 1 --2- 172.21.15.76:0/2904271994 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f662406ae70 0x7f662406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:39.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.219+0000 7f6636ffd640 1 --2- 172.21.15.76:0/2904271994 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f662406ae70 0x7f662406d330 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f6638159bf0 tx=0x7f662c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:39.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.223+0000 7f661ffff640 1 -- 172.21.15.76:0/2904271994 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f66280545f0 con 0x7f66380d3b60 2023-11-09T03:55:39.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.422+0000 7f663db61640 1 -- 172.21.15.76:0/2904271994 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66380d2cd0 con 0x7f66380d3b60 2023-11-09T03:55:39.422 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.423+0000 7f661ffff640 1 -- 172.21.15.76:0/2904271994 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) v1 ==== 95+0+4938 (secure 0 0 0) 0x7f662801e070 con 0x7f66380d3b60 2023-11-09T03:55:39.423 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:55:39.423 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":139,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":9525,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1713748386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1713748386},{"type":"v1","addr":"172.21.15.76:6835","nonce":1713748386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":138}],"filesystems":[{"mdsmap":{"epoch":139,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:55:32.694340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":6698},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6698":{"gid":6698,"name":"b","rank":1,"incarnation":137,"state":"up:reconnect","state_seq":14,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:55:39.423 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 139 2023-11-09T03:55:39.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.424+0000 7f663db61640 1 -- 172.21.15.76:0/2904271994 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f662406ae70 msgr2=0x7f662406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:39.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.424+0000 7f663db61640 1 --2- 172.21.15.76:0/2904271994 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f662406ae70 0x7f662406d330 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f6638159bf0 tx=0x7f662c008040 comp rx=0 tx=0).stop 2023-11-09T03:55:39.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.424+0000 7f663db61640 1 -- 172.21.15.76:0/2904271994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66380d3b60 msgr2=0x7f66380cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:39.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.424+0000 7f663db61640 1 --2- 172.21.15.76:0/2904271994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66380d3b60 0x7f66380cb450 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f662800c7c0 tx=0x7f662800cc80 comp rx=0 tx=0).stop 2023-11-09T03:55:39.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.425+0000 7f663db61640 1 -- 172.21.15.76:0/2904271994 shutdown_connections 2023-11-09T03:55:39.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.425+0000 7f663db61640 1 --2- 172.21.15.76:0/2904271994 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f662406ae70 0x7f662406d330 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:39.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.425+0000 7f663db61640 1 --2- 172.21.15.76:0/2904271994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f663815d4e0 0x7f66380cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:39.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.425+0000 7f663db61640 1 --2- 172.21.15.76:0/2904271994 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f663815d100 0x7f66380cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:39.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.425+0000 7f663db61640 1 --2- 172.21.15.76:0/2904271994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66380d3b60 0x7f66380cb450 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:39.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.425+0000 7f663db61640 1 -- 172.21.15.76:0/2904271994 >> 172.21.15.76:0/2904271994 conn(0x7f66380d2ef0 msgr2=0x7f66380d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:39.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.425+0000 7f663db61640 1 -- 172.21.15.76:0/2904271994 shutdown_connections 2023-11-09T03:55:39.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:39.425+0000 7f663db61640 1 -- 172.21.15.76:0/2904271994 wait complete. 2023-11-09T03:55:39.837 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 139, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-09T03:48:57.383848+0000', 'modified': '2023-11-09T03:55:32.694340+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 119, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 8568, 'mds_1': 6698}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6698': {'gid': 6698, 'name': 'b', 'rank': 1, 'incarnation': 137, 'state': 'up:reconnect', 'state_seq': 14, 'addr': '172.21.15.113:6837/366683575', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6836', 'nonce': 366683575}, {'type': 'v1', 'addr': '172.21.15.113:6837', 'nonce': 366683575}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_8568': {'gid': 8568, 'name': 'c', 'rank': 0, 'incarnation': 110, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.76:6833/153465374', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6832', 'nonce': 153465374}, {'type': 'v1', 'addr': '172.21.15.76:6833', 'nonce': 153465374}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-09T03:55:39.837 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6698:up:reconnect 2023-11-09T03:55:40.838 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:55:40.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.949+0000 7f81a9a6b640 1 Processor -- start 2023-11-09T03:55:40.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.949+0000 7f81a9a6b640 1 -- start start 2023-11-09T03:55:40.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.950+0000 7f81a9a6b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f81a40addd0 0x7f81a40cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:40.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.950+0000 7f81a9a6b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f81a40cc680 0x7f81a40ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:40.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.950+0000 7f81a9a6b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81a406bb10 0x7f81a406bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:40.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.950+0000 7f81a9a6b640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a406c4a0 con 0x7f81a40cc680 2023-11-09T03:55:40.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.950+0000 7f81a9a6b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a40cd720 con 0x7f81a40addd0 2023-11-09T03:55:40.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.950+0000 7f81a9a6b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a40ae1b0 con 0x7f81a406bb10 2023-11-09T03:55:40.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.950+0000 7f81a2ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f81a40cc680 0x7f81a40ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:40.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.950+0000 7f81a2ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f81a40cc680 0x7f81a40ca2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58148/0 (socket says 172.21.15.76:58148) 2023-11-09T03:55:40.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.950+0000 7f81a2ffd640 1 -- 172.21.15.76:0/3754098109 learned_addr learned my addr 172.21.15.76:0/3754098109 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:40.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.950+0000 7f81a2ffd640 1 -- 172.21.15.76:0/3754098109 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81a406bb10 msgr2=0x7f81a406bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:40.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.950+0000 7f81a2ffd640 1 --2- 172.21.15.76:0/3754098109 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81a406bb10 0x7f81a406bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:40.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.950+0000 7f81a2ffd640 1 -- 172.21.15.76:0/3754098109 >> v2:172.21.15.113:3300/0 conn(0x7f81a40addd0 msgr2=0x7f81a40cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:40.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.950+0000 7f81a2ffd640 1 --2- 172.21.15.76:0/3754098109 >> v2:172.21.15.113:3300/0 conn(0x7f81a40addd0 0x7f81a40cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:40.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.950+0000 7f81a2ffd640 1 -- 172.21.15.76:0/3754098109 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a40ccb10 con 0x7f81a40cc680 2023-11-09T03:55:40.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.951+0000 7f81a2ffd640 1 --2- 172.21.15.76:0/3754098109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81a40cc680 0x7f81a40ca2d0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f81a40cc630 tx=0x7f819402dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee2f00ece74e8ec server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:40.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.951+0000 7f81a27fc640 1 -- 172.21.15.76:0/3754098109 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819402ec50 con 0x7f81a40cc680 2023-11-09T03:55:40.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.951+0000 7f81a27fc640 1 -- 172.21.15.76:0/3754098109 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8194032070 con 0x7f81a40cc680 2023-11-09T03:55:40.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.951+0000 7f81a27fc640 1 -- 172.21.15.76:0/3754098109 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194038400 con 0x7f81a40cc680 2023-11-09T03:55:40.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.951+0000 7f81a9a6b640 1 -- 172.21.15.76:0/3754098109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81a40cc680 msgr2=0x7f81a40ca2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:40.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.951+0000 7f81a9a6b640 1 --2- 172.21.15.76:0/3754098109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81a40cc680 0x7f81a40ca2d0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f81a40cc630 tx=0x7f819402dea0 comp rx=0 tx=0).stop 2023-11-09T03:55:40.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.951+0000 7f81a9a6b640 1 -- 172.21.15.76:0/3754098109 shutdown_connections 2023-11-09T03:55:40.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.951+0000 7f81a9a6b640 1 --2- 172.21.15.76:0/3754098109 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81a406bb10 0x7f81a406bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:40.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.951+0000 7f81a9a6b640 1 --2- 172.21.15.76:0/3754098109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81a40cc680 0x7f81a40ca2d0 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:40.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.951+0000 7f81a9a6b640 1 --2- 172.21.15.76:0/3754098109 >> v2:172.21.15.113:3300/0 conn(0x7f81a40addd0 0x7f81a40cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:40.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.951+0000 7f81a9a6b640 1 -- 172.21.15.76:0/3754098109 >> 172.21.15.76:0/3754098109 conn(0x7f81a40cb530 msgr2=0x7f81a40cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:40.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.952+0000 7f81a9a6b640 1 -- 172.21.15.76:0/3754098109 shutdown_connections 2023-11-09T03:55:40.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.952+0000 7f81a9a6b640 1 -- 172.21.15.76:0/3754098109 wait complete. 2023-11-09T03:55:40.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.953+0000 7f81a9a6b640 1 Processor -- start 2023-11-09T03:55:40.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.953+0000 7f81a9a6b640 1 -- start start 2023-11-09T03:55:40.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.954+0000 7f81a9a6b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81a406bb10 0x7f81a40c1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:40.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.954+0000 7f81a9a6b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81a40addd0 0x7f81a41598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:40.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.954+0000 7f81a9a6b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81a4159de0 0x7f81a41589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:40.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.954+0000 7f81a9a6b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a41585a0 con 0x7f81a4159de0 2023-11-09T03:55:40.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.954+0000 7f81a9a6b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a40c1cd0 con 0x7f81a406bb10 2023-11-09T03:55:40.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.954+0000 7f81a9a6b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a41ef950 con 0x7f81a40addd0 2023-11-09T03:55:40.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.954+0000 7f81a3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81a4159de0 0x7f81a41589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:40.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.954+0000 7f81a3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81a4159de0 0x7f81a41589c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58156/0 (socket says 172.21.15.76:58156) 2023-11-09T03:55:40.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.954+0000 7f81a3fff640 1 -- 172.21.15.76:0/4109151365 learned_addr learned my addr 172.21.15.76:0/4109151365 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:40.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.954+0000 7f81a2ffd640 1 --2- 172.21.15.76:0/4109151365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81a40addd0 0x7f81a41598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:40.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.954+0000 7f81a37fe640 1 --2- 172.21.15.76:0/4109151365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81a406bb10 0x7f81a40c1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:40.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.954+0000 7f81a3fff640 1 -- 172.21.15.76:0/4109151365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81a40addd0 msgr2=0x7f81a41598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:40.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.954+0000 7f81a3fff640 1 --2- 172.21.15.76:0/4109151365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81a40addd0 0x7f81a41598a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:40.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.954+0000 7f81a3fff640 1 -- 172.21.15.76:0/4109151365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81a406bb10 msgr2=0x7f81a40c1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:40.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.954+0000 7f81a3fff640 1 --2- 172.21.15.76:0/4109151365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81a406bb10 0x7f81a40c1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:40.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.954+0000 7f81a3fff640 1 -- 172.21.15.76:0/4109151365 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819400b760 con 0x7f81a4159de0 2023-11-09T03:55:40.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.954+0000 7f81a37fe640 1 --2- 172.21.15.76:0/4109151365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81a406bb10 0x7f81a40c1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:55:40.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.954+0000 7f81a3fff640 1 --2- 172.21.15.76:0/4109151365 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81a4159de0 0x7f81a41589c0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f819c00b970 tx=0x7f819c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:40.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.954+0000 7f818bfff640 1 -- 172.21.15.76:0/4109151365 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c00cae0 con 0x7f81a4159de0 2023-11-09T03:55:40.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.954+0000 7f818bfff640 1 -- 172.21.15.76:0/4109151365 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819c00cc60 con 0x7f81a4159de0 2023-11-09T03:55:40.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.954+0000 7f818bfff640 1 -- 172.21.15.76:0/4109151365 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c014440 con 0x7f81a4159de0 2023-11-09T03:55:40.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.954+0000 7f81a9a6b640 1 -- 172.21.15.76:0/4109151365 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a4157c00 con 0x7f81a4159de0 2023-11-09T03:55:40.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.955+0000 7f81a9a6b640 1 -- 172.21.15.76:0/4109151365 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a41581c0 con 0x7f81a4159de0 2023-11-09T03:55:40.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.956+0000 7f81a9a6b640 1 -- 172.21.15.76:0/4109151365 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a41586e0 con 0x7f81a4159de0 2023-11-09T03:55:40.963 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.956+0000 7f818bfff640 1 -- 172.21.15.76:0/4109151365 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f819c01d3b0 con 0x7f81a4159de0 2023-11-09T03:55:40.963 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.956+0000 7f818bfff640 1 --2- 172.21.15.76:0/4109151365 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f819006aea0 0x7f819006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:40.963 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.956+0000 7f818bfff640 1 -- 172.21.15.76:0/4109151365 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8554+0+0 (secure 0 0 0) 0x7f819c089f80 con 0x7f81a4159de0 2023-11-09T03:55:40.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.957+0000 7f81a37fe640 1 --2- 172.21.15.76:0/4109151365 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f819006aea0 0x7f819006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:40.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.957+0000 7f81a37fe640 1 --2- 172.21.15.76:0/4109151365 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f819006aea0 0x7f819006d360 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8198009930 tx=0x7f8198008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:40.964 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:40.961+0000 7f818bfff640 1 -- 172.21.15.76:0/4109151365 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f819c0bcca0 con 0x7f81a4159de0 2023-11-09T03:55:41.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:41.164+0000 7f81a9a6b640 1 -- 172.21.15.76:0/4109151365 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81a40684b0 con 0x7f81a4159de0 2023-11-09T03:55:41.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:41.165+0000 7f818bfff640 1 -- 172.21.15.76:0/4109151365 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) v1 ==== 95+0+4938 (secure 0 0 0) 0x7f819c019070 con 0x7f81a4159de0 2023-11-09T03:55:41.165 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:55:41.165 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":139,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":9525,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1713748386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1713748386},{"type":"v1","addr":"172.21.15.76:6835","nonce":1713748386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":138}],"filesystems":[{"mdsmap":{"epoch":139,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:55:32.694340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":6698},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6698":{"gid":6698,"name":"b","rank":1,"incarnation":137,"state":"up:reconnect","state_seq":14,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:55:41.165 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 139 2023-11-09T03:55:41.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:41.167+0000 7f81a9a6b640 1 -- 172.21.15.76:0/4109151365 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f819006aea0 msgr2=0x7f819006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:41.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:41.167+0000 7f81a9a6b640 1 --2- 172.21.15.76:0/4109151365 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f819006aea0 0x7f819006d360 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8198009930 tx=0x7f8198008040 comp rx=0 tx=0).stop 2023-11-09T03:55:41.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:41.167+0000 7f81a9a6b640 1 -- 172.21.15.76:0/4109151365 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81a4159de0 msgr2=0x7f81a41589c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:41.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:41.167+0000 7f81a9a6b640 1 --2- 172.21.15.76:0/4109151365 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81a4159de0 0x7f81a41589c0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f819c00b970 tx=0x7f819c00be30 comp rx=0 tx=0).stop 2023-11-09T03:55:41.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:41.167+0000 7f81a9a6b640 1 -- 172.21.15.76:0/4109151365 shutdown_connections 2023-11-09T03:55:41.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:41.167+0000 7f81a9a6b640 1 --2- 172.21.15.76:0/4109151365 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f819006aea0 0x7f819006d360 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:41.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:41.167+0000 7f81a9a6b640 1 --2- 172.21.15.76:0/4109151365 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81a4159de0 0x7f81a41589c0 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:41.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:41.167+0000 7f81a9a6b640 1 --2- 172.21.15.76:0/4109151365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81a40addd0 0x7f81a41598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:41.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:41.167+0000 7f81a9a6b640 1 --2- 172.21.15.76:0/4109151365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81a406bb10 0x7f81a40c1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:41.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:41.167+0000 7f81a9a6b640 1 -- 172.21.15.76:0/4109151365 >> 172.21.15.76:0/4109151365 conn(0x7f81a40cb530 msgr2=0x7f81a40cea10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:41.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:41.167+0000 7f81a9a6b640 1 -- 172.21.15.76:0/4109151365 shutdown_connections 2023-11-09T03:55:41.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:41.167+0000 7f81a9a6b640 1 -- 172.21.15.76:0/4109151365 wait complete. 2023-11-09T03:55:41.579 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 139, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-09T03:48:57.383848+0000', 'modified': '2023-11-09T03:55:32.694340+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 119, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 8568, 'mds_1': 6698}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6698': {'gid': 6698, 'name': 'b', 'rank': 1, 'incarnation': 137, 'state': 'up:reconnect', 'state_seq': 14, 'addr': '172.21.15.113:6837/366683575', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6836', 'nonce': 366683575}, {'type': 'v1', 'addr': '172.21.15.113:6837', 'nonce': 366683575}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_8568': {'gid': 8568, 'name': 'c', 'rank': 0, 'incarnation': 110, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.76:6833/153465374', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6832', 'nonce': 153465374}, {'type': 'v1', 'addr': '172.21.15.76:6833', 'nonce': 153465374}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-09T03:55:41.580 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6698:up:reconnect 2023-11-09T03:55:42.580 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:55:42.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.691+0000 7f599e30f640 1 Processor -- start 2023-11-09T03:55:42.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.691+0000 7f599e30f640 1 -- start start 2023-11-09T03:55:42.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.692+0000 7f599e30f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f59980b0c40 0x7f599806fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:42.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.692+0000 7f599e30f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f599806bb60 0x7f5998070530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:42.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.692+0000 7f599e30f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5998071500 con 0x7f5998159e40 2023-11-09T03:55:42.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.692+0000 7f599e30f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5998070a70 con 0x7f59980b0c40 2023-11-09T03:55:42.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.692+0000 7f599e30f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f599815dae0 con 0x7f599806bb60 2023-11-09T03:55:42.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.692+0000 7f59977fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f59980b0c40 0x7f599806fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:42.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.692+0000 7f59977fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f59980b0c40 0x7f599806fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49484/0 (socket says 172.21.15.76:49484) 2023-11-09T03:55:42.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.692+0000 7f5997fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f5998159e40 0x7f59980b0530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37628/0 (socket says 172.21.15.76:37628) 2023-11-09T03:55:42.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.692+0000 7f59977fe640 1 -- 172.21.15.76:0/1136573550 learned_addr learned my addr 172.21.15.76:0/1136573550 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:42.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.692+0000 7f599c8ac640 1 --2- 172.21.15.76:0/1136573550 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f599806bb60 0x7f5998070530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:42.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.692+0000 7f59977fe640 1 -- 172.21.15.76:0/1136573550 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f599806bb60 msgr2=0x7f5998070530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:42.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.692+0000 7f59977fe640 1 --2- 172.21.15.76:0/1136573550 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f599806bb60 0x7f5998070530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:42.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.692+0000 7f59977fe640 1 -- 172.21.15.76:0/1136573550 >> v1:172.21.15.76:6789/0 conn(0x7f5998159e40 legacy=0x7f59980b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:42.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.692+0000 7f59977fe640 1 -- 172.21.15.76:0/1136573550 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5998070c60 con 0x7f59980b0c40 2023-11-09T03:55:42.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.692+0000 7f5996ffd640 1 -- 172.21.15.76:0/1136573550 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 937529804 0 0) 0x7f5998071500 con 0x7f5998159e40 2023-11-09T03:55:42.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.693+0000 7f59977fe640 1 --2- 172.21.15.76:0/1136573550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59980b0c40 0x7f599806fff0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f598c009870 tx=0x7f598c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7663df05d4004b6b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:42.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.693+0000 7f5996ffd640 1 -- 172.21.15.76:0/1136573550 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c031640 con 0x7f59980b0c40 2023-11-09T03:55:42.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.693+0000 7f5996ffd640 1 -- 172.21.15.76:0/1136573550 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598c0317a0 con 0x7f59980b0c40 2023-11-09T03:55:42.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.693+0000 7f5996ffd640 1 -- 172.21.15.76:0/1136573550 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c031b30 con 0x7f59980b0c40 2023-11-09T03:55:42.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.694+0000 7f599e30f640 1 -- 172.21.15.76:0/1136573550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59980b0c40 msgr2=0x7f599806fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:42.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.694+0000 7f599e30f640 1 --2- 172.21.15.76:0/1136573550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59980b0c40 0x7f599806fff0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f598c009870 tx=0x7f598c02b8f0 comp rx=0 tx=0).stop 2023-11-09T03:55:42.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.694+0000 7f599e30f640 1 -- 172.21.15.76:0/1136573550 shutdown_connections 2023-11-09T03:55:42.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.694+0000 7f599e30f640 1 --2- 172.21.15.76:0/1136573550 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f599806bb60 0x7f5998070530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:42.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.694+0000 7f599e30f640 1 --2- 172.21.15.76:0/1136573550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59980b0c40 0x7f599806fff0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:42.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.694+0000 7f599e30f640 1 -- 172.21.15.76:0/1136573550 >> 172.21.15.76:0/1136573550 conn(0x7f59980cb530 msgr2=0x7f59980cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:42.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.694+0000 7f599e30f640 1 -- 172.21.15.76:0/1136573550 shutdown_connections 2023-11-09T03:55:42.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.694+0000 7f599e30f640 1 -- 172.21.15.76:0/1136573550 wait complete. 2023-11-09T03:55:42.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.695+0000 7f599e30f640 1 Processor -- start 2023-11-09T03:55:42.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.695+0000 7f599e30f640 1 -- start start 2023-11-09T03:55:42.701 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.696+0000 7f599e30f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f599806bb60 0x7f5998154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:42.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.696+0000 7f599e30f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59980b0c40 0x7f59980c2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:42.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.696+0000 7f599e30f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5998159e40 0x7f59980c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:42.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.696+0000 7f599e30f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599815a220 con 0x7f59980b0c40 2023-11-09T03:55:42.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.696+0000 7f599e30f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59980c3450 con 0x7f599806bb60 2023-11-09T03:55:42.702 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.696+0000 7f599e30f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59980c3730 con 0x7f5998159e40 2023-11-09T03:55:42.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.696+0000 7f59977fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59980b0c40 0x7f59980c2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:42.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.696+0000 7f59977fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59980b0c40 0x7f59980c2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49494/0 (socket says 172.21.15.76:49494) 2023-11-09T03:55:42.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.696+0000 7f59977fe640 1 -- 172.21.15.76:0/2532499007 learned_addr learned my addr 172.21.15.76:0/2532499007 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:42.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.696+0000 7f599c8ac640 1 --2- 172.21.15.76:0/2532499007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5998159e40 0x7f59980c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:42.703 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.696+0000 7f5997fff640 1 --2- 172.21.15.76:0/2532499007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f599806bb60 0x7f5998154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:42.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.696+0000 7f59977fe640 1 -- 172.21.15.76:0/2532499007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5998159e40 msgr2=0x7f59980c6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:42.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.696+0000 7f59977fe640 1 --2- 172.21.15.76:0/2532499007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5998159e40 0x7f59980c6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:42.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.696+0000 7f59977fe640 1 -- 172.21.15.76:0/2532499007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f599806bb60 msgr2=0x7f5998154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:42.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.696+0000 7f59977fe640 1 --2- 172.21.15.76:0/2532499007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f599806bb60 0x7f5998154230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:42.704 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.696+0000 7f59977fe640 1 -- 172.21.15.76:0/2532499007 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598c0094d0 con 0x7f59980b0c40 2023-11-09T03:55:42.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.696+0000 7f59977fe640 1 --2- 172.21.15.76:0/2532499007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59980b0c40 0x7f59980c2a70 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f598c000c00 tx=0x7f598c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:42.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.697+0000 7f5994ff9640 1 -- 172.21.15.76:0/2532499007 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c03ac60 con 0x7f59980b0c40 2023-11-09T03:55:42.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.697+0000 7f5994ff9640 1 -- 172.21.15.76:0/2532499007 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598c03ade0 con 0x7f59980b0c40 2023-11-09T03:55:42.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.697+0000 7f599e30f640 1 -- 172.21.15.76:0/2532499007 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59980c3a10 con 0x7f59980b0c40 2023-11-09T03:55:42.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.697+0000 7f5994ff9640 1 -- 172.21.15.76:0/2532499007 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c044630 con 0x7f59980b0c40 2023-11-09T03:55:42.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.697+0000 7f599e30f640 1 -- 172.21.15.76:0/2532499007 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59981f2880 con 0x7f59980b0c40 2023-11-09T03:55:42.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.698+0000 7f599e30f640 1 -- 172.21.15.76:0/2532499007 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f599815a720 con 0x7f59980b0c40 2023-11-09T03:55:42.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.698+0000 7f5994ff9640 1 -- 172.21.15.76:0/2532499007 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f598c02f050 con 0x7f59980b0c40 2023-11-09T03:55:42.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.698+0000 7f5994ff9640 1 --2- 172.21.15.76:0/2532499007 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f598406ae70 0x7f598406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:42.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.698+0000 7f5994ff9640 1 -- 172.21.15.76:0/2532499007 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8554+0+0 (secure 0 0 0) 0x7f598c0ac0f0 con 0x7f59980b0c40 2023-11-09T03:55:42.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.699+0000 7f5997fff640 1 --2- 172.21.15.76:0/2532499007 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f598406ae70 0x7f598406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:42.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.699+0000 7f5997fff640 1 --2- 172.21.15.76:0/2532499007 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f598406ae70 0x7f598406d330 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f5988007fd0 tx=0x7f5988007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:42.707 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.703+0000 7f5994ff9640 1 -- 172.21.15.76:0/2532499007 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f599815a720 con 0x7f59980b0c40 2023-11-09T03:55:42.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.906+0000 7f599e30f640 1 -- 172.21.15.76:0/2532499007 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f599815a720 con 0x7f59980b0c40 2023-11-09T03:55:42.906 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.907+0000 7f5994ff9640 1 -- 172.21.15.76:0/2532499007 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+4935 (secure 0 0 0) 0x7f599815a720 con 0x7f59980b0c40 2023-11-09T03:55:42.907 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:55:42.907 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":141,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":9525,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1713748386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1713748386},{"type":"v1","addr":"172.21.15.76:6835","nonce":1713748386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":138}],"filesystems":[{"mdsmap":{"epoch":141,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:55:41.732323+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":6698},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6698":{"gid":6698,"name":"b","rank":1,"incarnation":137,"state":"up:active","state_seq":18,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:55:42.907 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 141 2023-11-09T03:55:42.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.908+0000 7f599e30f640 1 -- 172.21.15.76:0/2532499007 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f598406ae70 msgr2=0x7f598406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:42.908 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.908+0000 7f599e30f640 1 --2- 172.21.15.76:0/2532499007 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f598406ae70 0x7f598406d330 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f5988007fd0 tx=0x7f5988007de0 comp rx=0 tx=0).stop 2023-11-09T03:55:42.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.908+0000 7f599e30f640 1 -- 172.21.15.76:0/2532499007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59980b0c40 msgr2=0x7f59980c2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:42.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.908+0000 7f599e30f640 1 --2- 172.21.15.76:0/2532499007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59980b0c40 0x7f59980c2a70 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f598c000c00 tx=0x7f598c0046c0 comp rx=0 tx=0).stop 2023-11-09T03:55:42.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.909+0000 7f599e30f640 1 -- 172.21.15.76:0/2532499007 shutdown_connections 2023-11-09T03:55:42.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.909+0000 7f599e30f640 1 --2- 172.21.15.76:0/2532499007 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f598406ae70 0x7f598406d330 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:42.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.909+0000 7f599e30f640 1 --2- 172.21.15.76:0/2532499007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5998159e40 0x7f59980c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:42.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.909+0000 7f599e30f640 1 --2- 172.21.15.76:0/2532499007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59980b0c40 0x7f59980c2a70 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:42.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.909+0000 7f599e30f640 1 --2- 172.21.15.76:0/2532499007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f599806bb60 0x7f5998154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:42.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.909+0000 7f599e30f640 1 -- 172.21.15.76:0/2532499007 >> 172.21.15.76:0/2532499007 conn(0x7f59980cb530 msgr2=0x7f59980cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:42.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.909+0000 7f599e30f640 1 -- 172.21.15.76:0/2532499007 shutdown_connections 2023-11-09T03:55:42.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:42.909+0000 7f599e30f640 1 -- 172.21.15.76:0/2532499007 wait complete. 2023-11-09T03:55:43.321 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 141, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-09T03:48:57.383848+0000', 'modified': '2023-11-09T03:55:41.732323+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 119, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 8568, 'mds_1': 6698}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6698': {'gid': 6698, 'name': 'b', 'rank': 1, 'incarnation': 137, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.113:6837/366683575', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6836', 'nonce': 366683575}, {'type': 'v1', 'addr': '172.21.15.113:6837', 'nonce': 366683575}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_8568': {'gid': 8568, 'name': 'c', 'rank': 0, 'incarnation': 110, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.76:6833/153465374', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6832', 'nonce': 153465374}, {'type': 'v1', 'addr': '172.21.15.76:6833', 'nonce': 153465374}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-09T03:55:43.321 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2023-11-09T03:55:43.322 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-09T03:55:43.424 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.425+0000 7fa8d12c9640 1 Processor -- start 2023-11-09T03:55:43.425 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.425+0000 7fa8d12c9640 1 -- start start 2023-11-09T03:55:43.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.426+0000 7fa8d12c9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8cc15d100 0x7fa8cc0ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.426+0000 7fa8d12c9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa8cc15d4e0 0x7fa8cc0d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.426 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.426+0000 7fa8d12c9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa8cc0d3b60 0x7fa8cc0d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.426+0000 7fa8d12c9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8cc0b32b0 con 0x7fa8cc15d4e0 2023-11-09T03:55:43.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.426+0000 7fa8d12c9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8cc0cee10 con 0x7fa8cc0d3b60 2023-11-09T03:55:43.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.426+0000 7fa8d12c9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc0b2980 con 0x7fa8cc15d100 2023-11-09T03:55:43.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.426+0000 7fa8c3fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa8cc15d4e0 0x7fa8cc0d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.427 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.426+0000 7fa8c3fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa8cc15d4e0 0x7fa8cc0d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49500/0 (socket says 172.21.15.76:49500) 2023-11-09T03:55:43.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.426+0000 7fa8c3fff640 1 -- 172.21.15.76:0/3510341308 learned_addr learned my addr 172.21.15.76:0/3510341308 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:43.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.426+0000 7fa8c3fff640 1 -- 172.21.15.76:0/3510341308 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8cc15d100 msgr2=0x7fa8cc0ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:43.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.426+0000 7fa8caffd640 1 --2- 172.21.15.76:0/3510341308 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8cc15d100 0x7fa8cc0ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.426+0000 7fa8cb7fe640 1 --2- 172.21.15.76:0/3510341308 >> v2:172.21.15.113:3300/0 conn(0x7fa8cc0d3b60 0x7fa8cc0d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.428 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.426+0000 7fa8c3fff640 1 --2- 172.21.15.76:0/3510341308 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8cc15d100 0x7fa8cc0ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.426+0000 7fa8c3fff640 1 -- 172.21.15.76:0/3510341308 >> v2:172.21.15.113:3300/0 conn(0x7fa8cc0d3b60 msgr2=0x7fa8cc0d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:43.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.426+0000 7fa8c3fff640 1 --2- 172.21.15.76:0/3510341308 >> v2:172.21.15.113:3300/0 conn(0x7fa8cc0d3b60 0x7fa8cc0d1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.426+0000 7fa8c3fff640 1 -- 172.21.15.76:0/3510341308 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8cc0b2c60 con 0x7fa8cc15d4e0 2023-11-09T03:55:43.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.427+0000 7fa8c3fff640 1 --2- 172.21.15.76:0/3510341308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8cc15d4e0 0x7fa8cc0d14d0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc0b27a0 tx=0x7fa8b402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62365e0b35d87d2a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:43.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.427+0000 7fa8ca7fc640 1 -- 172.21.15.76:0/3510341308 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4031870 con 0x7fa8cc15d4e0 2023-11-09T03:55:43.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.427+0000 7fa8ca7fc640 1 -- 172.21.15.76:0/3510341308 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b40319d0 con 0x7fa8cc15d4e0 2023-11-09T03:55:43.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.427+0000 7fa8ca7fc640 1 -- 172.21.15.76:0/3510341308 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4031d60 con 0x7fa8cc15d4e0 2023-11-09T03:55:43.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.427+0000 7fa8d12c9640 1 -- 172.21.15.76:0/3510341308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8cc15d4e0 msgr2=0x7fa8cc0d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:43.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.427+0000 7fa8d12c9640 1 --2- 172.21.15.76:0/3510341308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8cc15d4e0 0x7fa8cc0d14d0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc0b27a0 tx=0x7fa8b402b910 comp rx=0 tx=0).stop 2023-11-09T03:55:43.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.428+0000 7fa8d12c9640 1 -- 172.21.15.76:0/3510341308 shutdown_connections 2023-11-09T03:55:43.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.428+0000 7fa8d12c9640 1 --2- 172.21.15.76:0/3510341308 >> v2:172.21.15.113:3300/0 conn(0x7fa8cc0d3b60 0x7fa8cc0d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.428+0000 7fa8d12c9640 1 --2- 172.21.15.76:0/3510341308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8cc15d4e0 0x7fa8cc0d14d0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.428+0000 7fa8d12c9640 1 --2- 172.21.15.76:0/3510341308 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8cc15d100 0x7fa8cc0ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.428+0000 7fa8d12c9640 1 -- 172.21.15.76:0/3510341308 >> 172.21.15.76:0/3510341308 conn(0x7fa8cc0d2ef0 msgr2=0x7fa8cc0d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:43.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.428+0000 7fa8d12c9640 1 -- 172.21.15.76:0/3510341308 shutdown_connections 2023-11-09T03:55:43.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.428+0000 7fa8d12c9640 1 -- 172.21.15.76:0/3510341308 wait complete. 2023-11-09T03:55:43.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.429+0000 7fa8d12c9640 1 Processor -- start 2023-11-09T03:55:43.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.429+0000 7fa8d12c9640 1 -- start start 2023-11-09T03:55:43.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.429+0000 7fa8d12c9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8cc0d3b60 0x7fa8cc0b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.429+0000 7fa8d12c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8cc15d100 0x7fa8cc09d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.429+0000 7fa8d12c9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8cc15d4e0 0x7fa8cc0a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.429+0000 7fa8d12c9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc09e3f0 con 0x7fa8cc15d100 2023-11-09T03:55:43.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.429+0000 7fa8d12c9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc09e6d0 con 0x7fa8cc15d4e0 2023-11-09T03:55:43.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.429+0000 7fa8d12c9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc09e9b0 con 0x7fa8cc0d3b60 2023-11-09T03:55:43.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.430+0000 7fa8c3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8cc15d100 0x7fa8cc09d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.430+0000 7fa8c3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8cc15d100 0x7fa8cc09d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49504/0 (socket says 172.21.15.76:49504) 2023-11-09T03:55:43.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.430+0000 7fa8c3fff640 1 -- 172.21.15.76:0/3820498830 learned_addr learned my addr 172.21.15.76:0/3820498830 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:43.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.430+0000 7fa8caffd640 1 --2- 172.21.15.76:0/3820498830 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8cc0d3b60 0x7fa8cc0b81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.430+0000 7fa8c3fff640 1 -- 172.21.15.76:0/3820498830 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8cc0d3b60 msgr2=0x7fa8cc0b81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:43.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.430+0000 7fa8c3fff640 1 --2- 172.21.15.76:0/3820498830 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8cc0d3b60 0x7fa8cc0b81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.430+0000 7fa8cb7fe640 1 --2- 172.21.15.76:0/3820498830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8cc15d4e0 0x7fa8cc0a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.430+0000 7fa8c3fff640 1 -- 172.21.15.76:0/3820498830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8cc15d4e0 msgr2=0x7fa8cc0a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:43.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.430+0000 7fa8c3fff640 1 --2- 172.21.15.76:0/3820498830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8cc15d4e0 0x7fa8cc0a0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.430+0000 7fa8c3fff640 1 -- 172.21.15.76:0/3820498830 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b40094d0 con 0x7fa8cc15d100 2023-11-09T03:55:43.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.430+0000 7fa8c3fff640 1 --2- 172.21.15.76:0/3820498830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8cc15d100 0x7fa8cc09d970 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fa8b402b950 tx=0x7fa8b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:43.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.430+0000 7fa8c37fe640 1 -- 172.21.15.76:0/3820498830 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4031860 con 0x7fa8cc15d100 2023-11-09T03:55:43.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.430+0000 7fa8c37fe640 1 -- 172.21.15.76:0/3820498830 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b40443f0 con 0x7fa8cc15d100 2023-11-09T03:55:43.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.430+0000 7fa8d12c9640 1 -- 172.21.15.76:0/3820498830 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8cc09ec90 con 0x7fa8cc15d100 2023-11-09T03:55:43.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.430+0000 7fa8c37fe640 1 -- 172.21.15.76:0/3820498830 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b40447a0 con 0x7fa8cc15d100 2023-11-09T03:55:43.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.430+0000 7fa8d12c9640 1 -- 172.21.15.76:0/3820498830 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8cc152620 con 0x7fa8cc15d100 2023-11-09T03:55:43.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.432+0000 7fa8c37fe640 1 -- 172.21.15.76:0/3820498830 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa8b404d450 con 0x7fa8cc15d100 2023-11-09T03:55:43.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.432+0000 7fa8c37fe640 1 --2- 172.21.15.76:0/3820498830 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa8ac06ae70 0x7fa8ac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.432+0000 7fa8c37fe640 1 -- 172.21.15.76:0/3820498830 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8554+0+0 (secure 0 0 0) 0x7fa8b402d040 con 0x7fa8cc15d100 2023-11-09T03:55:43.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.432+0000 7fa8caffd640 1 --2- 172.21.15.76:0/3820498830 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa8ac06ae70 0x7fa8ac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.433+0000 7fa8caffd640 1 --2- 172.21.15.76:0/3820498830 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa8ac06ae70 0x7fa8ac06d330 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00b840 tx=0x7fa8bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:43.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.433+0000 7fa8d3e5c740 1 Processor -- start 2023-11-09T03:55:43.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.433+0000 7fa8d3e5c740 1 -- start start 2023-11-09T03:55:43.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.434+0000 7fa8d3e5c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1c8558850 0x55c1c852b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.434+0000 7fa8d3e5c740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1c8558c30 0x55c1c855b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.434+0000 7fa8d3e5c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c1c852bc90 0x55c1c855bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.434+0000 7fa8d3e5c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c843c740 con 0x55c1c852bc90 2023-11-09T03:55:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.434+0000 7fa8d3e5c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c849e8d0 con 0x55c1c8558850 2023-11-09T03:55:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.434+0000 7fa8d3e5c740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1c849bfc0 con 0x55c1c8558c30 2023-11-09T03:55:43.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.434+0000 7fa8cb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c1c852bc90 0x55c1c855bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.434+0000 7fa8cb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c1c852bc90 0x55c1c855bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49520/0 (socket says 172.21.15.76:49520) 2023-11-09T03:55:43.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.434+0000 7fa8cb7fe640 1 -- 172.21.15.76:0/1368880231 learned_addr learned my addr 172.21.15.76:0/1368880231 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:43.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.434+0000 7fa8caffd640 1 --2- 172.21.15.76:0/1368880231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1c8558850 0x55c1c852b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.434+0000 7fa8c3fff640 1 --2- 172.21.15.76:0/1368880231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1c8558c30 0x55c1c855b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.434+0000 7fa8cb7fe640 1 -- 172.21.15.76:0/1368880231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1c8558c30 msgr2=0x55c1c855b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:43.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.434+0000 7fa8cb7fe640 1 --2- 172.21.15.76:0/1368880231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1c8558c30 0x55c1c855b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.434+0000 7fa8cb7fe640 1 -- 172.21.15.76:0/1368880231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1c8558850 msgr2=0x55c1c852b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:43.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.434+0000 7fa8cb7fe640 1 --2- 172.21.15.76:0/1368880231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1c8558850 0x55c1c852b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.434+0000 7fa8cb7fe640 1 -- 172.21.15.76:0/1368880231 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1c845aca0 con 0x55c1c852bc90 2023-11-09T03:55:43.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.434+0000 7fa8c3fff640 1 --2- 172.21.15.76:0/1368880231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1c8558c30 0x55c1c855b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:55:43.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.434+0000 7fa8cb7fe640 1 --2- 172.21.15.76:0/1368880231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c1c852bc90 0x55c1c855bb20 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4008f00 tx=0x7fa8c400d750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bab58a3cde3f9da server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:43.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.435+0000 7fa8c0ff9640 1 -- 172.21.15.76:0/1368880231 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4003da0 con 0x55c1c852bc90 2023-11-09T03:55:43.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.435+0000 7fa8c0ff9640 1 -- 172.21.15.76:0/1368880231 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c4004030 con 0x55c1c852bc90 2023-11-09T03:55:43.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.435+0000 7fa8c0ff9640 1 -- 172.21.15.76:0/1368880231 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c40043c0 con 0x55c1c852bc90 2023-11-09T03:55:43.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.435+0000 7fa8d3e5c740 1 -- 172.21.15.76:0/1368880231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c1c852bc90 msgr2=0x55c1c855bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:43.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.435+0000 7fa8d3e5c740 1 --2- 172.21.15.76:0/1368880231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c1c852bc90 0x55c1c855bb20 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4008f00 tx=0x7fa8c400d750 comp rx=0 tx=0).stop 2023-11-09T03:55:43.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.435+0000 7fa8d3e5c740 1 -- 172.21.15.76:0/1368880231 shutdown_connections 2023-11-09T03:55:43.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.435+0000 7fa8d3e5c740 1 --2- 172.21.15.76:0/1368880231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c1c852bc90 0x55c1c855bb20 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.435+0000 7fa8d3e5c740 1 --2- 172.21.15.76:0/1368880231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1c8558c30 0x55c1c855b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.435+0000 7fa8d3e5c740 1 --2- 172.21.15.76:0/1368880231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1c8558850 0x55c1c852b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.435+0000 7fa8d3e5c740 1 -- 172.21.15.76:0/1368880231 >> 172.21.15.76:0/1368880231 conn(0x55c1c852da60 msgr2=0x55c1c85765e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:43.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.435+0000 7fa8d3e5c740 1 -- 172.21.15.76:0/1368880231 shutdown_connections 2023-11-09T03:55:43.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.435+0000 7fa8d3e5c740 1 -- 172.21.15.76:0/1368880231 wait complete. 2023-11-09T03:55:43.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.436+0000 7fa8d3e5c740 1 -- start start 2023-11-09T03:55:43.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.436+0000 7fa8d3e5c740 1 Processor -- start 2023-11-09T03:55:43.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.437+0000 7fa8d3e5c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c1c8558850 0x55c1c8562c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.437+0000 7fa8d3e5c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1c8558c30 0x55c1c855c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.437+0000 7fa8d3e5c740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1c8576ac0 0x55c1c8576f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.437+0000 7fa8d3e5c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c84978b0 con 0x55c1c8558850 2023-11-09T03:55:43.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.437+0000 7fa8d3e5c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c8451c80 con 0x55c1c8558c30 2023-11-09T03:55:43.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.437+0000 7fa8d3e5c740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1c8451920 con 0x55c1c8576ac0 2023-11-09T03:55:43.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.437+0000 7fa8d3e5c740 10 client.0 _ll_register_callbacks cb 0x55c1c81a9ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:55:43.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.437+0000 7fa8caffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c1c8558850 0x55c1c8562c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.437+0000 7fa8caffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c1c8558850 0x55c1c8562c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49526/0 (socket says 172.21.15.76:49526) 2023-11-09T03:55:43.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.437+0000 7fa8caffd640 1 -- 172.21.15.76:0/3081882326 learned_addr learned my addr 172.21.15.76:0/3081882326 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:43.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.437+0000 7fa8caffd640 1 -- 172.21.15.76:0/3081882326 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1c8576ac0 msgr2=0x55c1c8576f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:43.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.437+0000 7fa8caffd640 1 --2- 172.21.15.76:0/3081882326 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1c8576ac0 0x55c1c8576f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.437+0000 7fa8caffd640 1 -- 172.21.15.76:0/3081882326 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1c8558c30 msgr2=0x55c1c855c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:43.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.437+0000 7fa8caffd640 1 --2- 172.21.15.76:0/3081882326 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1c8558c30 0x55c1c855c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.437+0000 7fa8caffd640 1 -- 172.21.15.76:0/3081882326 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c4008b00 con 0x55c1c8558850 2023-11-09T03:55:43.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.437+0000 7fa8caffd640 1 --2- 172.21.15.76:0/3081882326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c1c8558850 0x55c1c8562c10 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0127c0 tx=0x7fa8bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbdb73c599b19db7 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:43.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.437+0000 7fa8a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:55:43.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.437+0000 7fa8a9ffb640 1 -- 172.21.15.76:0/3081882326 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc00b050 con 0x55c1c8558850 2023-11-09T03:55:43.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.437+0000 7fa8d3e5c740 1 -- 172.21.15.76:0/3081882326 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c1c8443bf0 con 0x55c1c8558850 2023-11-09T03:55:43.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.437+0000 7fa8a9ffb640 1 -- 172.21.15.76:0/3081882326 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8554+0+0 (secure 0 0 0) 0x7fa8bc01f800 con 0x55c1c8558850 2023-11-09T03:55:43.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.438+0000 7fa8a9ffb640 1 -- 172.21.15.76:0/3081882326 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=141) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8bc01f9c0 con 0x55c1c8558850 2023-11-09T03:55:43.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.438+0000 7fa8d3e5c740 10 client.9576 fetch_fsmap learned FSMap version 141 2023-11-09T03:55:43.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.438+0000 7fa8d3e5c740 1 -- 172.21.15.76:0/3081882326 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=141,monmap=2+}) v3 -- 0x55c1c8495f30 con 0x55c1c8558850 2023-11-09T03:55:43.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.438+0000 7fa8a9ffb640 1 -- 172.21.15.76:0/3081882326 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c1c8443bf0 con 0x55c1c8558850 2023-11-09T03:55:43.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.438+0000 7fa8a9ffb640 1 -- 172.21.15.76:0/3081882326 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 141) v1 ==== 3306+0+0 (secure 0 0 0) 0x7fa8bc01fb20 con 0x55c1c8558850 2023-11-09T03:55:43.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.438+0000 7fa8d3e5c740 10 client.9576 fetch_fsmap finished waiting for FSMap version 141 2023-11-09T03:55:43.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.438+0000 7fa8d3e5c740 10 client.9576 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-09T03:55:43.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.438+0000 7fa8d3e5c740 20 client.9576 populate_metadata read hostname 'smithi076' 2023-11-09T03:55:43.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.438+0000 7fa8d3e5c740 1 --2- 172.21.15.76:0/3081882326 >> [v2:172.21.15.113:6836/366683575,v1:172.21.15.113:6837/366683575] conn(0x55c1c8545e30 0x55c1c8546210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.438+0000 7fa8d3e5c740 4 client.9576 mds_command: new command op to 6698 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:55:43.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.438+0000 7fa8d3e5c740 1 -- 172.21.15.76:0/3081882326 --> [v2:172.21.15.113:6836/366683575,v1:172.21.15.113:6837/366683575] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1c8490890 con 0x55c1c8545e30 2023-11-09T03:55:43.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.438+0000 7fa8c3fff640 1 --2- 172.21.15.76:0/3081882326 >> [v2:172.21.15.113:6836/366683575,v1:172.21.15.113:6837/366683575] conn(0x55c1c8545e30 0x55c1c8546210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.439+0000 7fa8a9ffb640 10 client.9576 ms_handle_connect on v2:172.21.15.113:6836/366683575 2023-11-09T03:55:43.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.439+0000 7fa8c3fff640 1 --2- 172.21.15.76:0/3081882326 >> [v2:172.21.15.113:6836/366683575,v1:172.21.15.113:6837/366683575] conn(0x55c1c8545e30 0x55c1c8546210 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a6320893a4dcee8 server_cookie=ad6861a68a272cb1 in_seq=0 out_seq=0 2023-11-09T03:55:43.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.439+0000 7fa8a9ffb640 1 -- 172.21.15.76:0/3081882326 <== mds.1 v2:172.21.15.113:6836/366683575 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c1c8490890 con 0x55c1c8545e30 2023-11-09T03:55:43.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.439+0000 7fa8a9ffb640 10 client.9576 handle_command_reply: tid=0 2023-11-09T03:55:43.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.439+0000 7fa8d3e5c740 1 client.9576 shutdown 2023-11-09T03:55:43.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.439+0000 7fa8d3e5c740 1 -- 172.21.15.76:0/3081882326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c1c8558850 msgr2=0x55c1c8562c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:43.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.439+0000 7fa8d3e5c740 1 --2- 172.21.15.76:0/3081882326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c1c8558850 0x55c1c8562c10 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0127c0 tx=0x7fa8bc0046c0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.439+0000 7fa8d3e5c740 1 -- 172.21.15.76:0/3081882326 shutdown_connections 2023-11-09T03:55:43.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.439+0000 7fa8d3e5c740 1 --2- 172.21.15.76:0/3081882326 >> [v2:172.21.15.113:6836/366683575,v1:172.21.15.113:6837/366683575] conn(0x55c1c8545e30 0x55c1c8546210 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.439+0000 7fa8d3e5c740 1 --2- 172.21.15.76:0/3081882326 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1c8576ac0 0x55c1c8576f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.440+0000 7fa8a9ffb640 0 client.9576 ms_handle_reset on v2:172.21.15.113:6836/366683575 2023-11-09T03:55:43.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.440+0000 7fa8d3e5c740 1 --2- 172.21.15.76:0/3081882326 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1c8558c30 0x55c1c855c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.440+0000 7fa8d3e5c740 1 --2- 172.21.15.76:0/3081882326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c1c8558850 0x55c1c8562c10 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.440+0000 7fa8d3e5c740 1 -- 172.21.15.76:0/3081882326 >> 172.21.15.76:0/3081882326 conn(0x55c1c852bc90 msgr2=0x55c1c855b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:43.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.440+0000 7fa8d3e5c740 1 -- 172.21.15.76:0/3081882326 shutdown_connections 2023-11-09T03:55:43.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.440+0000 7fa8d3e5c740 1 -- 172.21.15.76:0/3081882326 wait complete. 2023-11-09T03:55:43.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.440+0000 7fa8d3e5c740 20 client.9576 trim_cache size 0 max 16384 2023-11-09T03:55:43.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.455+0000 7fa8d3e5c740 1 Processor -- start 2023-11-09T03:55:43.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.455+0000 7fa8d3e5c740 1 -- start start 2023-11-09T03:55:43.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.456+0000 7fa8d3e5c740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1c8558c30 0x55c1c8526f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.456+0000 7fa8d3e5c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c1c8576ac0 0x55c1c857a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.456+0000 7fa8d3e5c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1c855e7b0 0x55c1c85480f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.456+0000 7fa8d3e5c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c8417280 con 0x55c1c8576ac0 2023-11-09T03:55:43.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.456+0000 7fa8d3e5c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c857a400 con 0x55c1c855e7b0 2023-11-09T03:55:43.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.456+0000 7fa8d3e5c740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1c852b660 con 0x55c1c8558c30 2023-11-09T03:55:43.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.456+0000 7fa8c3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c1c8576ac0 0x55c1c857a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.456+0000 7fa8c3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c1c8576ac0 0x55c1c857a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49534/0 (socket says 172.21.15.76:49534) 2023-11-09T03:55:43.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.456+0000 7fa8c3fff640 1 -- 172.21.15.76:0/2943720108 learned_addr learned my addr 172.21.15.76:0/2943720108 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:43.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.456+0000 7fa8c3fff640 1 -- 172.21.15.76:0/2943720108 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1c8558c30 msgr2=0x55c1c8526f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:43.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.456+0000 7fa8c3fff640 1 --2- 172.21.15.76:0/2943720108 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1c8558c30 0x55c1c8526f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.456+0000 7fa8c3fff640 1 -- 172.21.15.76:0/2943720108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1c855e7b0 msgr2=0x55c1c85480f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:43.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.456+0000 7fa8c3fff640 1 --2- 172.21.15.76:0/2943720108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1c855e7b0 0x55c1c85480f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.456+0000 7fa8c3fff640 1 -- 172.21.15.76:0/2943720108 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1c8433d10 con 0x55c1c8576ac0 2023-11-09T03:55:43.469 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.456+0000 7fa8c3fff640 1 --2- 172.21.15.76:0/2943720108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c1c8576ac0 0x55c1c857a6b0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4000c00 tx=0x7fa8b4046040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a5bfe9dd21a44e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:43.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.457+0000 7fa8c17fa640 1 -- 172.21.15.76:0/2943720108 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4082dc0 con 0x55c1c8576ac0 2023-11-09T03:55:43.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.457+0000 7fa8c17fa640 1 -- 172.21.15.76:0/2943720108 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b40aa2d0 con 0x55c1c8576ac0 2023-11-09T03:55:43.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.457+0000 7fa8c17fa640 1 -- 172.21.15.76:0/2943720108 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b40771d0 con 0x55c1c8576ac0 2023-11-09T03:55:43.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.457+0000 7fa8d3e5c740 1 -- 172.21.15.76:0/2943720108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c1c8576ac0 msgr2=0x55c1c857a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:43.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.457+0000 7fa8d3e5c740 1 --2- 172.21.15.76:0/2943720108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c1c8576ac0 0x55c1c857a6b0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4000c00 tx=0x7fa8b4046040 comp rx=0 tx=0).stop 2023-11-09T03:55:43.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.457+0000 7fa8d3e5c740 1 -- 172.21.15.76:0/2943720108 shutdown_connections 2023-11-09T03:55:43.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.457+0000 7fa8d3e5c740 1 --2- 172.21.15.76:0/2943720108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1c855e7b0 0x55c1c85480f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.457+0000 7fa8d3e5c740 1 --2- 172.21.15.76:0/2943720108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c1c8576ac0 0x55c1c857a6b0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.457+0000 7fa8d3e5c740 1 --2- 172.21.15.76:0/2943720108 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1c8558c30 0x55c1c8526f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.457+0000 7fa8d3e5c740 1 -- 172.21.15.76:0/2943720108 >> 172.21.15.76:0/2943720108 conn(0x55c1c852bc90 msgr2=0x55c1c8588cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:43.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.457+0000 7fa8d3e5c740 1 -- 172.21.15.76:0/2943720108 shutdown_connections 2023-11-09T03:55:43.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.457+0000 7fa8d3e5c740 1 -- 172.21.15.76:0/2943720108 wait complete. 2023-11-09T03:55:43.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.458+0000 7fa8d3e5c740 1 -- start start 2023-11-09T03:55:43.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.458+0000 7fa8d3e5c740 1 Processor -- start 2023-11-09T03:55:43.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.458+0000 7fa8d3e5c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c1c8558c30 0x55c1c85e9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.458+0000 7fa8d3e5c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1c855e7b0 0x55c1c85448a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.458+0000 7fa8d3e5c740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1c8576ac0 0x55c1c8544de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.458+0000 7fa8d3e5c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c857a520 con 0x55c1c8558c30 2023-11-09T03:55:43.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.458+0000 7fa8d3e5c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c85ea6e0 con 0x55c1c855e7b0 2023-11-09T03:55:43.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.458+0000 7fa8d3e5c740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1c857a2d0 con 0x55c1c8576ac0 2023-11-09T03:55:43.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.458+0000 7fa8d3e5c740 10 client.0 _ll_register_callbacks cb 0x7fa8c4008660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:55:43.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.458+0000 7fa8caffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c1c8558c30 0x55c1c85e9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.459+0000 7fa8caffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c1c8558c30 0x55c1c85e9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49548/0 (socket says 172.21.15.76:49548) 2023-11-09T03:55:43.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.459+0000 7fa8caffd640 1 -- 172.21.15.76:0/313806135 learned_addr learned my addr 172.21.15.76:0/313806135 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:43.474 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.459+0000 7fa8c3fff640 1 --2- 172.21.15.76:0/313806135 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1c855e7b0 0x55c1c85448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.459+0000 7fa8cb7fe640 1 --2- 172.21.15.76:0/313806135 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1c8576ac0 0x55c1c8544de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.459+0000 7fa8caffd640 1 -- 172.21.15.76:0/313806135 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1c8576ac0 msgr2=0x55c1c8544de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:43.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.459+0000 7fa8caffd640 1 --2- 172.21.15.76:0/313806135 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1c8576ac0 0x55c1c8544de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.459+0000 7fa8caffd640 1 -- 172.21.15.76:0/313806135 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1c855e7b0 msgr2=0x55c1c85448a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:43.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.459+0000 7fa8caffd640 1 --2- 172.21.15.76:0/313806135 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1c855e7b0 0x55c1c85448a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.459+0000 7fa8caffd640 1 -- 172.21.15.76:0/313806135 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8cc152620 con 0x55c1c8558c30 2023-11-09T03:55:43.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.459+0000 7fa8cb7fe640 1 --2- 172.21.15.76:0/313806135 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1c8576ac0 0x55c1c8544de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:55:43.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.459+0000 7fa8caffd640 1 --2- 172.21.15.76:0/313806135 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c1c8558c30 0x55c1c85e9ee0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc000c00 tx=0x7fa8bc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d2a779ee999a6b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:43.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.459+0000 7fa8a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:55:43.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.459+0000 7fa8a9ffb640 1 -- 172.21.15.76:0/313806135 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc012d40 con 0x55c1c8558c30 2023-11-09T03:55:43.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.459+0000 7fa8a9ffb640 1 -- 172.21.15.76:0/313806135 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8554+0+0 (secure 0 0 0) 0x7fa8bc01fbc0 con 0x55c1c8558c30 2023-11-09T03:55:43.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.459+0000 7fa8d3e5c740 1 -- 172.21.15.76:0/313806135 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa8c4004030 con 0x55c1c8558c30 2023-11-09T03:55:43.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.459+0000 7fa8a9ffb640 1 -- 172.21.15.76:0/313806135 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=141) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8bc004030 con 0x55c1c8558c30 2023-11-09T03:55:43.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.459+0000 7fa8d3e5c740 10 client.9582 fetch_fsmap learned FSMap version 141 2023-11-09T03:55:43.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.459+0000 7fa8d3e5c740 1 -- 172.21.15.76:0/313806135 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=141,monmap=2+}) v3 -- 0x55c1c8459500 con 0x55c1c8558c30 2023-11-09T03:55:43.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.459+0000 7fa8a9ffb640 1 -- 172.21.15.76:0/313806135 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c4004030 con 0x55c1c8558c30 2023-11-09T03:55:43.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.459+0000 7fa8a9ffb640 1 -- 172.21.15.76:0/313806135 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 141) v1 ==== 3306+0+0 (secure 0 0 0) 0x7fa8bc004190 con 0x55c1c8558c30 2023-11-09T03:55:43.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.459+0000 7fa8d3e5c740 10 client.9582 fetch_fsmap finished waiting for FSMap version 141 2023-11-09T03:55:43.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.459+0000 7fa8d3e5c740 10 client.9582 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-09T03:55:43.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.459+0000 7fa8d3e5c740 20 client.9582 populate_metadata read hostname 'smithi076' 2023-11-09T03:55:43.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.459+0000 7fa8d3e5c740 1 --2- 172.21.15.76:0/313806135 >> [v2:172.21.15.113:6836/366683575,v1:172.21.15.113:6837/366683575] conn(0x55c1c852f410 0x55c1c852f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.459+0000 7fa8d3e5c740 4 client.9582 mds_command: new command op to 6698 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-09T03:55:43.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.459+0000 7fa8d3e5c740 1 -- 172.21.15.76:0/313806135 --> [v2:172.21.15.113:6836/366683575,v1:172.21.15.113:6837/366683575] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c1c84905b0 con 0x55c1c852f410 2023-11-09T03:55:43.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.460+0000 7fa8c3fff640 1 --2- 172.21.15.76:0/313806135 >> [v2:172.21.15.113:6836/366683575,v1:172.21.15.113:6837/366683575] conn(0x55c1c852f410 0x55c1c852f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.460+0000 7fa8c3fff640 1 --2- 172.21.15.76:0/313806135 >> [v2:172.21.15.113:6836/366683575,v1:172.21.15.113:6837/366683575] conn(0x55c1c852f410 0x55c1c852f7f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e98939543a044ff7 server_cookie=2301f62c1f8a53ed in_seq=0 out_seq=0 2023-11-09T03:55:43.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.460+0000 7fa8a9ffb640 10 client.9582 ms_handle_connect on v2:172.21.15.113:6836/366683575 2023-11-09T03:55:43.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.460+0000 7fa8a9ffb640 1 -- 172.21.15.76:0/313806135 <== mds.1 v2:172.21.15.113:6836/366683575 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55c1c84905b0 con 0x55c1c852f410 2023-11-09T03:55:43.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.460+0000 7fa8a9ffb640 10 client.9582 handle_command_reply: tid=0 2023-11-09T03:55:43.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.460+0000 7fa8d3e5c740 1 client.9582 shutdown 2023-11-09T03:55:43.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.460+0000 7fa8d3e5c740 1 -- 172.21.15.76:0/313806135 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c1c8558c30 msgr2=0x55c1c85e9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:43.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.460+0000 7fa8d3e5c740 1 --2- 172.21.15.76:0/313806135 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c1c8558c30 0x55c1c85e9ee0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc000c00 tx=0x7fa8bc012000 comp rx=0 tx=0).stop 2023-11-09T03:55:43.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.461+0000 7fa8d3e5c740 1 -- 172.21.15.76:0/313806135 shutdown_connections 2023-11-09T03:55:43.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.461+0000 7fa8d3e5c740 1 --2- 172.21.15.76:0/313806135 >> [v2:172.21.15.113:6836/366683575,v1:172.21.15.113:6837/366683575] conn(0x55c1c852f410 0x55c1c852f7f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.461+0000 7fa8d3e5c740 1 --2- 172.21.15.76:0/313806135 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1c8576ac0 0x55c1c8544de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.461+0000 7fa8d3e5c740 1 --2- 172.21.15.76:0/313806135 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1c855e7b0 0x55c1c85448a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.461+0000 7fa8a9ffb640 0 client.9582 ms_handle_reset on v2:172.21.15.113:6836/366683575 2023-11-09T03:55:43.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.461+0000 7fa8d3e5c740 1 --2- 172.21.15.76:0/313806135 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c1c8558c30 0x55c1c85e9ee0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.461+0000 7fa8d3e5c740 1 -- 172.21.15.76:0/313806135 >> 172.21.15.76:0/313806135 conn(0x55c1c852bc90 msgr2=0x55c1c85462a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:43.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.461+0000 7fa8d3e5c740 1 -- 172.21.15.76:0/313806135 shutdown_connections 2023-11-09T03:55:43.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.461+0000 7fa8d3e5c740 1 -- 172.21.15.76:0/313806135 wait complete. 2023-11-09T03:55:43.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.461+0000 7fa8d3e5c740 20 client.9582 trim_cache size 0 max 16384 2023-11-09T03:55:43.482 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:55:43.482 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "c213b26f-d0da-45d5-8e34-bb75c096d4f2", 2023-11-09T03:55:43.482 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 1, 2023-11-09T03:55:43.483 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 6698, 2023-11-09T03:55:43.483 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2023-11-09T03:55:43.483 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2023-11-09T03:55:43.483 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2023-11-09T03:55:43.483 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 11.770416433999999, 2023-11-09T03:55:43.483 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 141, 2023-11-09T03:55:43.484 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 120, 2023-11-09T03:55:43.484 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 120, 2023-11-09T03:55:43.484 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 58.01369467 2023-11-09T03:55:43.484 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:55:43.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.461+0000 7fa8d12c9640 1 -- 172.21.15.76:0/3820498830 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa8ac06ae70 msgr2=0x7fa8ac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:43.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.461+0000 7fa8d12c9640 1 --2- 172.21.15.76:0/3820498830 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa8ac06ae70 0x7fa8ac06d330 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00b840 tx=0x7fa8bc008040 comp rx=0 tx=0).stop 2023-11-09T03:55:43.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.461+0000 7fa8d12c9640 1 -- 172.21.15.76:0/3820498830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8cc15d100 msgr2=0x7fa8cc09d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:43.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.461+0000 7fa8d12c9640 1 --2- 172.21.15.76:0/3820498830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8cc15d100 0x7fa8cc09d970 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fa8b402b950 tx=0x7fa8b40046c0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.461+0000 7fa8d12c9640 1 -- 172.21.15.76:0/3820498830 shutdown_connections 2023-11-09T03:55:43.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.461+0000 7fa8d12c9640 1 --2- 172.21.15.76:0/3820498830 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fa8ac06ae70 0x7fa8ac06d330 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.461+0000 7fa8d12c9640 1 --2- 172.21.15.76:0/3820498830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8cc15d4e0 0x7fa8cc0a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.461+0000 7fa8d12c9640 1 --2- 172.21.15.76:0/3820498830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8cc15d100 0x7fa8cc09d970 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.461+0000 7fa8d12c9640 1 --2- 172.21.15.76:0/3820498830 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8cc0d3b60 0x7fa8cc0b81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.461+0000 7fa8d12c9640 1 -- 172.21.15.76:0/3820498830 >> 172.21.15.76:0/3820498830 conn(0x7fa8cc0d2ef0 msgr2=0x7fa8cc06bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:43.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.461+0000 7fa8d12c9640 1 -- 172.21.15.76:0/3820498830 shutdown_connections 2023-11-09T03:55:43.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.461+0000 7fa8d12c9640 1 -- 172.21.15.76:0/3820498830 wait complete. 2023-11-09T03:55:43.488 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-09T03:55:43.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.589+0000 7f1f32e64640 1 Processor -- start 2023-11-09T03:55:43.589 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.589+0000 7f1f32e64640 1 -- start start 2023-11-09T03:55:43.590 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.590+0000 7f1f32e64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1f2c06d9c0 0x7f1f2c0710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.590+0000 7f1f32e64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f2c06c5f0 0x7f1f2c071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.590+0000 7f1f32e64640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f2c06cc90 con 0x7f1f2c0ade00 2023-11-09T03:55:43.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.590+0000 7f1f32e64640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f2c0b0c40 con 0x7f1f2c06d9c0 2023-11-09T03:55:43.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.590+0000 7f1f32e64640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c071b50 con 0x7f1f2c06c5f0 2023-11-09T03:55:43.591 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.590+0000 7f1f2bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1f2c06d9c0 0x7f1f2c0710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.590+0000 7f1f2bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1f2c06d9c0 0x7f1f2c0710d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.76:52862/0 (socket says 172.21.15.76:52862) 2023-11-09T03:55:43.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.591+0000 7f1f2bfff640 1 -- 172.21.15.76:0/1484996491 learned_addr learned my addr 172.21.15.76:0/1484996491 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:43.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.591+0000 7f1f31401640 1 --2- 172.21.15.76:0/1484996491 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f2c06c5f0 0x7f1f2c071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.590+0000 7f1f30c00640 1 --1- 172.21.15.76:0/1484996491 >> v1:172.21.15.113:6789/0 conn(0x7f1f2c0ade00 0x7f1f2c0b0530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.76:57772/0 (socket says 172.21.15.76:57772) 2023-11-09T03:55:43.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.591+0000 7f1f2bfff640 1 -- 172.21.15.76:0/1484996491 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f2c06c5f0 msgr2=0x7f1f2c071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:43.592 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.591+0000 7f1f2bfff640 1 --2- 172.21.15.76:0/1484996491 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f2c06c5f0 0x7f1f2c071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.591+0000 7f1f2bfff640 1 -- 172.21.15.76:0/1484996491 >> v1:172.21.15.113:6789/0 conn(0x7f1f2c0ade00 legacy=0x7f1f2c0b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:43.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.591+0000 7f1f2bfff640 1 -- 172.21.15.76:0/1484996491 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2c0b0e60 con 0x7f1f2c06d9c0 2023-11-09T03:55:43.593 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.591+0000 7f1f2b7fe640 1 -- 172.21.15.76:0/1484996491 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2474801766 0 0) 0x7f1f2c06cc90 con 0x7f1f2c0ade00 2023-11-09T03:55:43.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.592+0000 7f1f2bfff640 1 --2- 172.21.15.76:0/1484996491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f2c06d9c0 0x7f1f2c0710d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009870 tx=0x7f1f1402b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b434bde0146b48c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:43.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.592+0000 7f1f2b7fe640 1 -- 172.21.15.76:0/1484996491 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f140317c0 con 0x7f1f2c06d9c0 2023-11-09T03:55:43.596 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.592+0000 7f1f2b7fe640 1 -- 172.21.15.76:0/1484996491 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f14031920 con 0x7f1f2c06d9c0 2023-11-09T03:55:43.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.592+0000 7f1f2b7fe640 1 -- 172.21.15.76:0/1484996491 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f14031cb0 con 0x7f1f2c06d9c0 2023-11-09T03:55:43.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.592+0000 7f1f32e64640 1 -- 172.21.15.76:0/1484996491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f2c06d9c0 msgr2=0x7f1f2c0710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:43.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.592+0000 7f1f32e64640 1 --2- 172.21.15.76:0/1484996491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f2c06d9c0 0x7f1f2c0710d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009870 tx=0x7f1f1402b860 comp rx=0 tx=0).stop 2023-11-09T03:55:43.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.592+0000 7f1f32e64640 1 -- 172.21.15.76:0/1484996491 shutdown_connections 2023-11-09T03:55:43.597 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.592+0000 7f1f32e64640 1 --2- 172.21.15.76:0/1484996491 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f2c06c5f0 0x7f1f2c071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.592+0000 7f1f32e64640 1 --2- 172.21.15.76:0/1484996491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f2c06d9c0 0x7f1f2c0710d0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.592+0000 7f1f32e64640 1 -- 172.21.15.76:0/1484996491 >> 172.21.15.76:0/1484996491 conn(0x7f1f2c0cb530 msgr2=0x7f1f2c0cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:43.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.592+0000 7f1f32e64640 1 -- 172.21.15.76:0/1484996491 shutdown_connections 2023-11-09T03:55:43.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.592+0000 7f1f32e64640 1 -- 172.21.15.76:0/1484996491 wait complete. 2023-11-09T03:55:43.598 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.594+0000 7f1f32e64640 1 Processor -- start 2023-11-09T03:55:43.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.594+0000 7f1f32e64640 1 -- start start 2023-11-09T03:55:43.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.594+0000 7f1f32e64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f2c06c5f0 0x7f1f2c0cad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.594+0000 7f1f32e64640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f2c06d9c0 0x7f1f2c0c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.594+0000 7f1f32e64640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f2c0ade00 0x7f1f2c0c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.599 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.594+0000 7f1f32e64640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c0c6520 con 0x7f1f2c06d9c0 2023-11-09T03:55:43.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.594+0000 7f1f32e64640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c0c6940 con 0x7f1f2c0ade00 2023-11-09T03:55:43.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.594+0000 7f1f32e64640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c0c8930 con 0x7f1f2c06c5f0 2023-11-09T03:55:43.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.595+0000 7f1f2bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f2c06d9c0 0x7f1f2c0c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.595+0000 7f1f2bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f2c06d9c0 0x7f1f2c0c7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49564/0 (socket says 172.21.15.76:49564) 2023-11-09T03:55:43.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.595+0000 7f1f2bfff640 1 -- 172.21.15.76:0/3760912821 learned_addr learned my addr 172.21.15.76:0/3760912821 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:43.600 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.595+0000 7f1f30c00640 1 --2- 172.21.15.76:0/3760912821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f2c06c5f0 0x7f1f2c0cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.595+0000 7f1f31401640 1 --2- 172.21.15.76:0/3760912821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f2c0ade00 0x7f1f2c0c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.595+0000 7f1f2bfff640 1 -- 172.21.15.76:0/3760912821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f2c06c5f0 msgr2=0x7f1f2c0cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:43.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.595+0000 7f1f2bfff640 1 --2- 172.21.15.76:0/3760912821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f2c06c5f0 0x7f1f2c0cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.595+0000 7f1f2bfff640 1 -- 172.21.15.76:0/3760912821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f2c0ade00 msgr2=0x7f1f2c0c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:43.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.595+0000 7f1f2bfff640 1 --2- 172.21.15.76:0/3760912821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f2c0ade00 0x7f1f2c0c82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.601 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.595+0000 7f1f2bfff640 1 -- 172.21.15.76:0/3760912821 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f140094d0 con 0x7f1f2c06d9c0 2023-11-09T03:55:43.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.595+0000 7f1f2bfff640 1 --2- 172.21.15.76:0/3760912821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f2c06d9c0 0x7f1f2c0c7d70 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f1f14034040 tx=0x7f1f140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:43.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.595+0000 7f1f297fa640 1 -- 172.21.15.76:0/3760912821 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f14004230 con 0x7f1f2c06d9c0 2023-11-09T03:55:43.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.595+0000 7f1f297fa640 1 -- 172.21.15.76:0/3760912821 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f14004390 con 0x7f1f2c06d9c0 2023-11-09T03:55:43.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.595+0000 7f1f32e64640 1 -- 172.21.15.76:0/3760912821 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f2c0c5b80 con 0x7f1f2c06d9c0 2023-11-09T03:55:43.602 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.595+0000 7f1f297fa640 1 -- 172.21.15.76:0/3760912821 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f14003830 con 0x7f1f2c06d9c0 2023-11-09T03:55:43.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.595+0000 7f1f32e64640 1 -- 172.21.15.76:0/3760912821 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2c0c6140 con 0x7f1f2c06d9c0 2023-11-09T03:55:43.603 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.597+0000 7f1f297fa640 1 -- 172.21.15.76:0/3760912821 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1f14043070 con 0x7f1f2c06d9c0 2023-11-09T03:55:43.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.597+0000 7f1f297fa640 1 --2- 172.21.15.76:0/3760912821 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1f0c06aea0 0x7f1f0c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.597+0000 7f1f297fa640 1 -- 172.21.15.76:0/3760912821 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8554+0+0 (secure 0 0 0) 0x7f1f140abef0 con 0x7f1f2c06d9c0 2023-11-09T03:55:43.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.597+0000 7f1f30c00640 1 --2- 172.21.15.76:0/3760912821 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1f0c06aea0 0x7f1f0c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.598+0000 7f1f30c00640 1 --2- 172.21.15.76:0/3760912821 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1f0c06aea0 0x7f1f0c06d360 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c0012e0 tx=0x7f1f1c001630 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:43.609 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.598+0000 7f1f359f6740 1 Processor -- start 2023-11-09T03:55:43.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.598+0000 7f1f359f6740 1 -- start start 2023-11-09T03:55:43.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.599+0000 7f1f359f6740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563b5002d850 0x563b50000750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.599+0000 7f1f359f6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b5002dc30 0x563b500305e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.599+0000 7f1f359f6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b50000c90 0x563b50030b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.610 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.599+0000 7f1f359f6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b4ff11740 con 0x563b5002dc30 2023-11-09T03:55:43.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.599+0000 7f1f359f6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b4ff738d0 con 0x563b50000c90 2023-11-09T03:55:43.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.599+0000 7f1f359f6740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b4ff70fc0 con 0x563b5002d850 2023-11-09T03:55:43.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.599+0000 7f1f2bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b5002dc30 0x563b500305e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.599+0000 7f1f2bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b5002dc30 0x563b500305e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49578/0 (socket says 172.21.15.76:49578) 2023-11-09T03:55:43.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.599+0000 7f1f2bfff640 1 -- 172.21.15.76:0/3392597684 learned_addr learned my addr 172.21.15.76:0/3392597684 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:43.611 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.599+0000 7f1f30c00640 1 --2- 172.21.15.76:0/3392597684 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563b5002d850 0x563b50000750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.599+0000 7f1f31401640 1 --2- 172.21.15.76:0/3392597684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b50000c90 0x563b50030b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.599+0000 7f1f2bfff640 1 -- 172.21.15.76:0/3392597684 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563b5002d850 msgr2=0x563b50000750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:43.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.599+0000 7f1f2bfff640 1 --2- 172.21.15.76:0/3392597684 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563b5002d850 0x563b50000750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.599+0000 7f1f2bfff640 1 -- 172.21.15.76:0/3392597684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b50000c90 msgr2=0x563b50030b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:43.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.599+0000 7f1f2bfff640 1 --2- 172.21.15.76:0/3392597684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b50000c90 0x563b50030b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.599+0000 7f1f2bfff640 1 -- 172.21.15.76:0/3392597684 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b4ff2fca0 con 0x563b5002dc30 2023-11-09T03:55:43.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.599+0000 7f1f2bfff640 1 --2- 172.21.15.76:0/3392597684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b5002dc30 0x563b500305e0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f1f1407a250 tx=0x7f1f14079ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac41ff4790cad77 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:43.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.600+0000 7f1f227fc640 1 -- 172.21.15.76:0/3392597684 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f14082510 con 0x563b5002dc30 2023-11-09T03:55:43.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.600+0000 7f1f227fc640 1 -- 172.21.15.76:0/3392597684 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f14078660 con 0x563b5002dc30 2023-11-09T03:55:43.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.600+0000 7f1f227fc640 1 -- 172.21.15.76:0/3392597684 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f14082080 con 0x563b5002dc30 2023-11-09T03:55:43.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.600+0000 7f1f359f6740 1 -- 172.21.15.76:0/3392597684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b5002dc30 msgr2=0x563b500305e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:43.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.600+0000 7f1f359f6740 1 --2- 172.21.15.76:0/3392597684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b5002dc30 0x563b500305e0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f1f1407a250 tx=0x7f1f14079ce0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.600+0000 7f1f359f6740 1 -- 172.21.15.76:0/3392597684 shutdown_connections 2023-11-09T03:55:43.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.600+0000 7f1f359f6740 1 --2- 172.21.15.76:0/3392597684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b50000c90 0x563b50030b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.614 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.600+0000 7f1f359f6740 1 --2- 172.21.15.76:0/3392597684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b5002dc30 0x563b500305e0 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.600+0000 7f1f359f6740 1 --2- 172.21.15.76:0/3392597684 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563b5002d850 0x563b50000750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.600+0000 7f1f359f6740 1 -- 172.21.15.76:0/3392597684 >> 172.21.15.76:0/3392597684 conn(0x563b50002a60 msgr2=0x563b5004b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:43.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.600+0000 7f1f359f6740 1 -- 172.21.15.76:0/3392597684 shutdown_connections 2023-11-09T03:55:43.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.600+0000 7f1f359f6740 1 -- 172.21.15.76:0/3392597684 wait complete. 2023-11-09T03:55:43.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.601+0000 7f1f359f6740 1 -- start start 2023-11-09T03:55:43.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.601+0000 7f1f359f6740 1 Processor -- start 2023-11-09T03:55:43.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.602+0000 7f1f359f6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b5002d850 0x563b5005b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.602+0000 7f1f359f6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b5002dc30 0x563b5002a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.602+0000 7f1f359f6740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563b50033e70 0x563b5002be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.602+0000 7f1f359f6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b4ff6c8b0 con 0x563b5002d850 2023-11-09T03:55:43.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.602+0000 7f1f359f6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b4ff26c80 con 0x563b5002dc30 2023-11-09T03:55:43.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.602+0000 7f1f359f6740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b4ff26920 con 0x563b50033e70 2023-11-09T03:55:43.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.602+0000 7f1f359f6740 10 client.0 _ll_register_callbacks cb 0x563b4fc7eac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:55:43.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.602+0000 7f1f30c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b5002d850 0x563b5005b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.602+0000 7f1f30c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b5002d850 0x563b5005b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49588/0 (socket says 172.21.15.76:49588) 2023-11-09T03:55:43.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.602+0000 7f1f30c00640 1 -- 172.21.15.76:0/2485286561 learned_addr learned my addr 172.21.15.76:0/2485286561 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:43.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.602+0000 7f1f2bfff640 1 --2- 172.21.15.76:0/2485286561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b5002dc30 0x563b5002a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.602+0000 7f1f31401640 1 --2- 172.21.15.76:0/2485286561 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563b50033e70 0x563b5002be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.602+0000 7f1f30c00640 1 -- 172.21.15.76:0/2485286561 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563b50033e70 msgr2=0x563b5002be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:43.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.602+0000 7f1f30c00640 1 --2- 172.21.15.76:0/2485286561 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563b50033e70 0x563b5002be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.602+0000 7f1f30c00640 1 -- 172.21.15.76:0/2485286561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b5002dc30 msgr2=0x563b5002a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:43.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.602+0000 7f1f30c00640 1 --2- 172.21.15.76:0/2485286561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b5002dc30 0x563b5002a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.602+0000 7f1f30c00640 1 -- 172.21.15.76:0/2485286561 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2c0c6140 con 0x563b5002d850 2023-11-09T03:55:43.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.602+0000 7f1f31401640 1 --2- 172.21.15.76:0/2485286561 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563b50033e70 0x563b5002be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:55:43.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.602+0000 7f1f30c00640 1 --2- 172.21.15.76:0/2485286561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b5002d850 0x563b5005b6e0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c004ef0 tx=0x7f1f1c001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e2304448176590 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:43.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.602+0000 7f1f0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:55:43.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.602+0000 7f1f0b7fe640 1 -- 172.21.15.76:0/2485286561 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c013e00 con 0x563b5002d850 2023-11-09T03:55:43.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.602+0000 7f1f0b7fe640 1 -- 172.21.15.76:0/2485286561 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8554+0+0 (secure 0 0 0) 0x7f1f1c01b070 con 0x563b5002d850 2023-11-09T03:55:43.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.603+0000 7f1f359f6740 1 -- 172.21.15.76:0/2485286561 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563b4ff18bf0 con 0x563b5002d850 2023-11-09T03:55:43.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.603+0000 7f1f0b7fe640 1 -- 172.21.15.76:0/2485286561 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=141) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f1c012ae0 con 0x563b5002d850 2023-11-09T03:55:43.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.603+0000 7f1f359f6740 10 client.9591 fetch_fsmap learned FSMap version 141 2023-11-09T03:55:43.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.603+0000 7f1f359f6740 1 -- 172.21.15.76:0/2485286561 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=141,monmap=2+}) v3 -- 0x563b4ff6af30 con 0x563b5002d850 2023-11-09T03:55:43.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.603+0000 7f1f0b7fe640 1 -- 172.21.15.76:0/2485286561 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563b4ff18bf0 con 0x563b5002d850 2023-11-09T03:55:43.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.603+0000 7f1f0b7fe640 1 -- 172.21.15.76:0/2485286561 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 141) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f1f1c012070 con 0x563b5002d850 2023-11-09T03:55:43.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.603+0000 7f1f359f6740 10 client.9591 fetch_fsmap finished waiting for FSMap version 141 2023-11-09T03:55:43.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.603+0000 7f1f359f6740 10 client.9591 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-09T03:55:43.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.603+0000 7f1f359f6740 20 client.9591 populate_metadata read hostname 'smithi076' 2023-11-09T03:55:43.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.603+0000 7f1f359f6740 1 --2- 172.21.15.76:0/2485286561 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x563b50027670 0x563b50017490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.621 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.603+0000 7f1f359f6740 4 client.9591 mds_command: new command op to 8568 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:55:43.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.603+0000 7f1f359f6740 1 -- 172.21.15.76:0/2485286561 --> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b4ff65890 con 0x563b50027670 2023-11-09T03:55:43.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.603+0000 7f1f2bfff640 1 --2- 172.21.15.76:0/2485286561 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x563b50027670 0x563b50017490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.603+0000 7f1f0b7fe640 10 client.9591 ms_handle_connect on v2:172.21.15.76:6832/153465374 2023-11-09T03:55:43.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.603+0000 7f1f2bfff640 1 --2- 172.21.15.76:0/2485286561 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x563b50027670 0x563b50017490 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cf96f43240a23cc server_cookie=e2b8e005a4174cc9 in_seq=0 out_seq=0 2023-11-09T03:55:43.622 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.604+0000 7f1f0b7fe640 1 -- 172.21.15.76:0/2485286561 <== mds.0 v2:172.21.15.76:6832/153465374 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563b4ff65890 con 0x563b50027670 2023-11-09T03:55:43.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.604+0000 7f1f0b7fe640 10 client.9591 handle_command_reply: tid=0 2023-11-09T03:55:43.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.604+0000 7f1f359f6740 1 client.9591 shutdown 2023-11-09T03:55:43.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.604+0000 7f1f359f6740 1 -- 172.21.15.76:0/2485286561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b5002d850 msgr2=0x563b5005b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:43.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.604+0000 7f1f359f6740 1 --2- 172.21.15.76:0/2485286561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b5002d850 0x563b5005b6e0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c004ef0 tx=0x7f1f1c001250 comp rx=0 tx=0).stop 2023-11-09T03:55:43.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.604+0000 7f1f359f6740 1 -- 172.21.15.76:0/2485286561 shutdown_connections 2023-11-09T03:55:43.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.604+0000 7f1f359f6740 1 --2- 172.21.15.76:0/2485286561 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x563b50027670 0x563b50017490 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.604+0000 7f1f359f6740 1 --2- 172.21.15.76:0/2485286561 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563b50033e70 0x563b5002be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.604+0000 7f1f359f6740 1 --2- 172.21.15.76:0/2485286561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b5002dc30 0x563b5002a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.604+0000 7f1f0b7fe640 0 client.9591 ms_handle_reset on v2:172.21.15.76:6832/153465374 2023-11-09T03:55:43.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.604+0000 7f1f359f6740 1 --2- 172.21.15.76:0/2485286561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b5002d850 0x563b5005b6e0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.604+0000 7f1f359f6740 1 -- 172.21.15.76:0/2485286561 >> 172.21.15.76:0/2485286561 conn(0x563b50000c90 msgr2=0x563b500300a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:43.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.604+0000 7f1f359f6740 1 -- 172.21.15.76:0/2485286561 shutdown_connections 2023-11-09T03:55:43.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.604+0000 7f1f359f6740 1 -- 172.21.15.76:0/2485286561 wait complete. 2023-11-09T03:55:43.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.604+0000 7f1f359f6740 20 client.9591 trim_cache size 0 max 16384 2023-11-09T03:55:43.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.620+0000 7f1f359f6740 1 Processor -- start 2023-11-09T03:55:43.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.620+0000 7f1f359f6740 1 -- start start 2023-11-09T03:55:43.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.620+0000 7f1f359f6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b5002d850 0x563b50004440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.620+0000 7f1f359f6740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563b5002dc30 0x563b50059d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.620+0000 7f1f359f6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b50033e70 0x563b5005a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.620+0000 7f1f359f6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b50005350 con 0x563b50033e70 2023-11-09T03:55:43.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.620+0000 7f1f359f6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b4feec280 con 0x563b5002d850 2023-11-09T03:55:43.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.620+0000 7f1f359f6740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b5002ae90 con 0x563b5002dc30 2023-11-09T03:55:43.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.620+0000 7f1f31401640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b50033e70 0x563b5005a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.621+0000 7f1f31401640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b50033e70 0x563b5005a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49590/0 (socket says 172.21.15.76:49590) 2023-11-09T03:55:43.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.621+0000 7f1f31401640 1 -- 172.21.15.76:0/343984916 learned_addr learned my addr 172.21.15.76:0/343984916 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:43.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.621+0000 7f1f2bfff640 1 --2- 172.21.15.76:0/343984916 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563b5002dc30 0x563b50059d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.621+0000 7f1f30c00640 1 --2- 172.21.15.76:0/343984916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b5002d850 0x563b50004440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.621+0000 7f1f31401640 1 -- 172.21.15.76:0/343984916 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563b5002dc30 msgr2=0x563b50059d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:43.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.621+0000 7f1f31401640 1 --2- 172.21.15.76:0/343984916 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563b5002dc30 0x563b50059d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.621+0000 7f1f31401640 1 -- 172.21.15.76:0/343984916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b5002d850 msgr2=0x563b50004440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:43.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.621+0000 7f1f31401640 1 --2- 172.21.15.76:0/343984916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b5002d850 0x563b50004440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.621+0000 7f1f31401640 1 -- 172.21.15.76:0/343984916 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b4ff08d10 con 0x563b50033e70 2023-11-09T03:55:43.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.621+0000 7f1f30c00640 1 --2- 172.21.15.76:0/343984916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b5002d850 0x563b50004440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:55:43.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.621+0000 7f1f2bfff640 1 --2- 172.21.15.76:0/343984916 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563b5002dc30 0x563b50059d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:55:43.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.621+0000 7f1f31401640 1 --2- 172.21.15.76:0/343984916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b50033e70 0x563b5005a240 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009c90 tx=0x7f1f24009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17ad0d42660a31d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:43.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.621+0000 7f1f22ffd640 1 -- 172.21.15.76:0/343984916 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f24009e40 con 0x563b50033e70 2023-11-09T03:55:43.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.621+0000 7f1f22ffd640 1 -- 172.21.15.76:0/343984916 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f24014610 con 0x563b50033e70 2023-11-09T03:55:43.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.621+0000 7f1f22ffd640 1 -- 172.21.15.76:0/343984916 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f240149c0 con 0x563b50033e70 2023-11-09T03:55:43.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.622+0000 7f1f359f6740 1 -- 172.21.15.76:0/343984916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b50033e70 msgr2=0x563b5005a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:43.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.622+0000 7f1f359f6740 1 --2- 172.21.15.76:0/343984916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b50033e70 0x563b5005a240 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009c90 tx=0x7f1f24009190 comp rx=0 tx=0).stop 2023-11-09T03:55:43.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.622+0000 7f1f359f6740 1 -- 172.21.15.76:0/343984916 shutdown_connections 2023-11-09T03:55:43.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.622+0000 7f1f359f6740 1 --2- 172.21.15.76:0/343984916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b50033e70 0x563b5005a240 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.635 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.622+0000 7f1f359f6740 1 --2- 172.21.15.76:0/343984916 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563b5002dc30 0x563b50059d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.622+0000 7f1f359f6740 1 --2- 172.21.15.76:0/343984916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b5002d850 0x563b50004440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.622+0000 7f1f359f6740 1 -- 172.21.15.76:0/343984916 >> 172.21.15.76:0/343984916 conn(0x563b50000c90 msgr2=0x563b50065a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:43.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.622+0000 7f1f359f6740 1 -- 172.21.15.76:0/343984916 shutdown_connections 2023-11-09T03:55:43.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.622+0000 7f1f359f6740 1 -- 172.21.15.76:0/343984916 wait complete. 2023-11-09T03:55:43.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.622+0000 7f1f359f6740 1 -- start start 2023-11-09T03:55:43.636 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.622+0000 7f1f359f6740 1 Processor -- start 2023-11-09T03:55:43.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.623+0000 7f1f359f6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b5002d850 0x563b500acd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.623+0000 7f1f359f6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b5002dc30 0x563b4fffff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.623+0000 7f1f359f6740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563b50033e70 0x563b500b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.623+0000 7f1f359f6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b500b6ad0 con 0x563b5002d850 2023-11-09T03:55:43.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.623+0000 7f1f359f6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b50031550 con 0x563b5002dc30 2023-11-09T03:55:43.637 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.623+0000 7f1f359f6740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b500317a0 con 0x563b50033e70 2023-11-09T03:55:43.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.623+0000 7f1f359f6740 10 client.0 _ll_register_callbacks cb 0x7f1f14004580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:55:43.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.623+0000 7f1f30c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b5002d850 0x563b500acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.623+0000 7f1f30c00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b5002d850 0x563b500acd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49606/0 (socket says 172.21.15.76:49606) 2023-11-09T03:55:43.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.623+0000 7f1f30c00640 1 -- 172.21.15.76:0/1612286716 learned_addr learned my addr 172.21.15.76:0/1612286716 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:43.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.623+0000 7f1f2bfff640 1 --2- 172.21.15.76:0/1612286716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b5002dc30 0x563b4fffff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.638 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.623+0000 7f1f31401640 1 --2- 172.21.15.76:0/1612286716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563b50033e70 0x563b500b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.623+0000 7f1f30c00640 1 -- 172.21.15.76:0/1612286716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563b50033e70 msgr2=0x563b500b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:43.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.623+0000 7f1f30c00640 1 --2- 172.21.15.76:0/1612286716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563b50033e70 0x563b500b62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.623+0000 7f1f30c00640 1 -- 172.21.15.76:0/1612286716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b5002dc30 msgr2=0x563b4fffff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:43.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.623+0000 7f1f30c00640 1 --2- 172.21.15.76:0/1612286716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b5002dc30 0x563b4fffff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.639 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.624+0000 7f1f30c00640 1 -- 172.21.15.76:0/1612286716 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f24009970 con 0x563b5002d850 2023-11-09T03:55:43.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.624+0000 7f1f31401640 1 --2- 172.21.15.76:0/1612286716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563b50033e70 0x563b500b62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:55:43.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.624+0000 7f1f30c00640 1 --2- 172.21.15.76:0/1612286716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b5002d850 0x563b500acd70 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c013ee0 tx=0x7f1f1c00a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c212b5542c35bdd7 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:43.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.624+0000 7f1f0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:55:43.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.624+0000 7f1f0b7fe640 1 -- 172.21.15.76:0/1612286716 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c0074b0 con 0x563b5002d850 2023-11-09T03:55:43.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.624+0000 7f1f0b7fe640 1 -- 172.21.15.76:0/1612286716 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8554+0+0 (secure 0 0 0) 0x7f1f1c012cf0 con 0x563b5002d850 2023-11-09T03:55:43.640 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.624+0000 7f1f359f6740 1 -- 172.21.15.76:0/1612286716 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f14078660 con 0x563b5002d850 2023-11-09T03:55:43.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.624+0000 7f1f0b7fe640 1 -- 172.21.15.76:0/1612286716 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=141) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f1c034410 con 0x563b5002d850 2023-11-09T03:55:43.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.624+0000 7f1f359f6740 10 client.9597 fetch_fsmap learned FSMap version 141 2023-11-09T03:55:43.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.624+0000 7f1f359f6740 1 -- 172.21.15.76:0/1612286716 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=141,monmap=2+}) v3 -- 0x563b4ff2e500 con 0x563b5002d850 2023-11-09T03:55:43.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.624+0000 7f1f0b7fe640 1 -- 172.21.15.76:0/1612286716 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f14078660 con 0x563b5002d850 2023-11-09T03:55:43.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.624+0000 7f1f0b7fe640 1 -- 172.21.15.76:0/1612286716 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 141) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f1f1c012050 con 0x563b5002d850 2023-11-09T03:55:43.641 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.624+0000 7f1f359f6740 10 client.9597 fetch_fsmap finished waiting for FSMap version 141 2023-11-09T03:55:43.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.624+0000 7f1f359f6740 10 client.9597 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-09T03:55:43.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.624+0000 7f1f359f6740 20 client.9597 populate_metadata read hostname 'smithi076' 2023-11-09T03:55:43.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.624+0000 7f1f359f6740 1 --2- 172.21.15.76:0/1612286716 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x563b50030780 0x563b50065730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.624+0000 7f1f359f6740 4 client.9597 mds_command: new command op to 8568 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-09T03:55:43.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.624+0000 7f1f359f6740 1 -- 172.21.15.76:0/1612286716 --> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563b4ff655b0 con 0x563b50030780 2023-11-09T03:55:43.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.624+0000 7f1f2bfff640 1 --2- 172.21.15.76:0/1612286716 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x563b50030780 0x563b50065730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.625+0000 7f1f0b7fe640 10 client.9597 ms_handle_connect on v2:172.21.15.76:6832/153465374 2023-11-09T03:55:43.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.625+0000 7f1f2bfff640 1 --2- 172.21.15.76:0/1612286716 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x563b50030780 0x563b50065730 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=610fce0424ff44f6 server_cookie=d86a4bcef3884550 in_seq=0 out_seq=0 2023-11-09T03:55:43.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.625+0000 7f1f0b7fe640 1 -- 172.21.15.76:0/1612286716 <== mds.0 v2:172.21.15.76:6832/153465374 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x563b4ff655b0 con 0x563b50030780 2023-11-09T03:55:43.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.625+0000 7f1f0b7fe640 10 client.9597 handle_command_reply: tid=0 2023-11-09T03:55:43.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.625+0000 7f1f359f6740 1 client.9597 shutdown 2023-11-09T03:55:43.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.625+0000 7f1f359f6740 1 -- 172.21.15.76:0/1612286716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b5002d850 msgr2=0x563b500acd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:43.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.625+0000 7f1f359f6740 1 --2- 172.21.15.76:0/1612286716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b5002d850 0x563b500acd70 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c013ee0 tx=0x7f1f1c00a000 comp rx=0 tx=0).stop 2023-11-09T03:55:43.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.625+0000 7f1f359f6740 1 -- 172.21.15.76:0/1612286716 shutdown_connections 2023-11-09T03:55:43.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.625+0000 7f1f359f6740 1 --2- 172.21.15.76:0/1612286716 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x563b50030780 0x563b50065730 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.625+0000 7f1f359f6740 1 --2- 172.21.15.76:0/1612286716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563b50033e70 0x563b500b62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.625+0000 7f1f359f6740 1 --2- 172.21.15.76:0/1612286716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b5002dc30 0x563b4fffff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.625+0000 7f1f0b7fe640 0 client.9597 ms_handle_reset on v2:172.21.15.76:6832/153465374 2023-11-09T03:55:43.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.625+0000 7f1f359f6740 1 --2- 172.21.15.76:0/1612286716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563b5002d850 0x563b500acd70 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.625+0000 7f1f359f6740 1 -- 172.21.15.76:0/1612286716 >> 172.21.15.76:0/1612286716 conn(0x563b50000c90 msgr2=0x563b50026240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:43.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.625+0000 7f1f359f6740 1 -- 172.21.15.76:0/1612286716 shutdown_connections 2023-11-09T03:55:43.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.625+0000 7f1f359f6740 1 -- 172.21.15.76:0/1612286716 wait complete. 2023-11-09T03:55:43.645 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.625+0000 7f1f359f6740 20 client.9597 trim_cache size 0 max 16384 2023-11-09T03:55:43.646 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:55:43.646 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "c213b26f-d0da-45d5-8e34-bb75c096d4f2", 2023-11-09T03:55:43.646 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2023-11-09T03:55:43.647 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 8568, 2023-11-09T03:55:43.647 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2023-11-09T03:55:43.647 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2023-11-09T03:55:43.647 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2023-11-09T03:55:43.647 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 146.14315107100001, 2023-11-09T03:55:43.647 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 141, 2023-11-09T03:55:43.648 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 120, 2023-11-09T03:55:43.648 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 120, 2023-11-09T03:55:43.648 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 189.31318270599999 2023-11-09T03:55:43.648 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:55:43.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.626+0000 7f1f32e64640 1 -- 172.21.15.76:0/3760912821 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1f0c06aea0 msgr2=0x7f1f0c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:43.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.626+0000 7f1f32e64640 1 --2- 172.21.15.76:0/3760912821 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1f0c06aea0 0x7f1f0c06d360 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c0012e0 tx=0x7f1f1c001630 comp rx=0 tx=0).stop 2023-11-09T03:55:43.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.626+0000 7f1f32e64640 1 -- 172.21.15.76:0/3760912821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f2c06d9c0 msgr2=0x7f1f2c0c7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:43.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.626+0000 7f1f32e64640 1 --2- 172.21.15.76:0/3760912821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f2c06d9c0 0x7f1f2c0c7d70 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f1f14034040 tx=0x7f1f140046c0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.626+0000 7f1f32e64640 1 -- 172.21.15.76:0/3760912821 shutdown_connections 2023-11-09T03:55:43.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.626+0000 7f1f32e64640 1 --2- 172.21.15.76:0/3760912821 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f1f0c06aea0 0x7f1f0c06d360 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.626+0000 7f1f32e64640 1 --2- 172.21.15.76:0/3760912821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f2c0ade00 0x7f1f2c0c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.626+0000 7f1f32e64640 1 --2- 172.21.15.76:0/3760912821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f2c06d9c0 0x7f1f2c0c7d70 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.626+0000 7f1f32e64640 1 --2- 172.21.15.76:0/3760912821 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f2c06c5f0 0x7f1f2c0cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.626+0000 7f1f32e64640 1 -- 172.21.15.76:0/3760912821 >> 172.21.15.76:0/3760912821 conn(0x7f1f2c0cb530 msgr2=0x7f1f2c0b0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:43.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.626+0000 7f1f32e64640 1 -- 172.21.15.76:0/3760912821 shutdown_connections 2023-11-09T03:55:43.651 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.626+0000 7f1f32e64640 1 -- 172.21.15.76:0/3760912821 wait complete. 2023-11-09T03:55:43.652 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:55:43.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:55:43.652 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2023-11-09T03:55:43.679 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:55:43.680 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:55:43.740 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:55:43.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.880+0000 7ff3d0378640 1 Processor -- start 2023-11-09T03:55:43.880 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.880+0000 7ff3d0378640 1 -- start start 2023-11-09T03:55:43.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.881+0000 7ff3d0378640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff3c815d100 0x7ff3c80ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.881 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.881+0000 7ff3d0378640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3c80d1be0 0x7ff3c80b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.881+0000 7ff3d0378640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c80b32b0 con 0x7ff3c815d4e0 2023-11-09T03:55:43.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.881+0000 7ff3d0378640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3c80cee10 con 0x7ff3c815d100 2023-11-09T03:55:43.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.881+0000 7ff3d0378640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3c80b2980 con 0x7ff3c80d1be0 2023-11-09T03:55:43.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.881+0000 7ff3ce114640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff3c815d100 0x7ff3c80ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.881+0000 7ff3cd913640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff3c815d4e0 0x7ff3c80d14d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37640/0 (socket says 172.21.15.76:37640) 2023-11-09T03:55:43.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.881+0000 7ff3ce114640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff3c815d100 0x7ff3c80ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49622/0 (socket says 172.21.15.76:49622) 2023-11-09T03:55:43.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.881+0000 7ff3cd913640 1 -- 172.21.15.76:0/2828568495 learned_addr learned my addr 172.21.15.76:0/2828568495 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:55:43.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.882+0000 7ff3ce114640 1 -- 172.21.15.76:0/2828568495 >> v2:172.21.15.113:3300/0 conn(0x7ff3c80d1be0 msgr2=0x7ff3c80b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:43.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.882+0000 7ff3ce915640 1 --2- 172.21.15.76:0/2828568495 >> v2:172.21.15.113:3300/0 conn(0x7ff3c80d1be0 0x7ff3c80b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.882+0000 7ff3ce114640 1 --2- 172.21.15.76:0/2828568495 >> v2:172.21.15.113:3300/0 conn(0x7ff3c80d1be0 0x7ff3c80b3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.882+0000 7ff3ce114640 1 -- 172.21.15.76:0/2828568495 >> v1:172.21.15.76:6789/0 conn(0x7ff3c815d4e0 legacy=0x7ff3c80d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:43.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.882+0000 7ff3ce114640 1 -- 172.21.15.76:0/2828568495 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c80b2c60 con 0x7ff3c815d100 2023-11-09T03:55:43.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.882+0000 7ff3cd112640 1 -- 172.21.15.76:0/2828568495 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3266011475 0 0) 0x7ff3c80b32b0 con 0x7ff3c815d4e0 2023-11-09T03:55:43.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.882+0000 7ff3ce114640 1 --2- 172.21.15.76:0/2828568495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3c815d100 0x7ff3c80ce510 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7ff3c808d6a0 tx=0x7ff3c402bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991550aea8754828 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:43.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.883+0000 7ff3cd112640 1 -- 172.21.15.76:0/2828568495 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c402ccb0 con 0x7ff3c815d100 2023-11-09T03:55:43.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.883+0000 7ff3cd112640 1 -- 172.21.15.76:0/2828568495 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c402ce10 con 0x7ff3c815d100 2023-11-09T03:55:43.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.883+0000 7ff3cd112640 1 -- 172.21.15.76:0/2828568495 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4034660 con 0x7ff3c815d100 2023-11-09T03:55:43.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.883+0000 7ff3d0378640 1 -- 172.21.15.76:0/2828568495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3c815d100 msgr2=0x7ff3c80ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:43.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.883+0000 7ff3d0378640 1 --2- 172.21.15.76:0/2828568495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3c815d100 0x7ff3c80ce510 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7ff3c808d6a0 tx=0x7ff3c402bf00 comp rx=0 tx=0).stop 2023-11-09T03:55:43.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.883+0000 7ff3d0378640 1 -- 172.21.15.76:0/2828568495 shutdown_connections 2023-11-09T03:55:43.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.883+0000 7ff3d0378640 1 --2- 172.21.15.76:0/2828568495 >> v2:172.21.15.113:3300/0 conn(0x7ff3c80d1be0 0x7ff3c80b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.883+0000 7ff3d0378640 1 --2- 172.21.15.76:0/2828568495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3c815d100 0x7ff3c80ce510 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.883+0000 7ff3d0378640 1 -- 172.21.15.76:0/2828568495 >> 172.21.15.76:0/2828568495 conn(0x7ff3c80d2ef0 msgr2=0x7ff3c80d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:43.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.883+0000 7ff3d0378640 1 -- 172.21.15.76:0/2828568495 shutdown_connections 2023-11-09T03:55:43.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.883+0000 7ff3d0378640 1 -- 172.21.15.76:0/2828568495 wait complete. 2023-11-09T03:55:43.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.884+0000 7ff3d0378640 1 Processor -- start 2023-11-09T03:55:43.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.884+0000 7ff3d0378640 1 -- start start 2023-11-09T03:55:43.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.885+0000 7ff3d0378640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3c80d1be0 0x7ff3c8149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.885+0000 7ff3d0378640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3c815d100 0x7ff3c81f2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.885+0000 7ff3d0378640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3c815d4e0 0x7ff3c814be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.885+0000 7ff3d0378640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c80b7c40 con 0x7ff3c815d100 2023-11-09T03:55:43.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.885+0000 7ff3d0378640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c81601b0 con 0x7ff3c815d4e0 2023-11-09T03:55:43.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.885+0000 7ff3d0378640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c80b7580 con 0x7ff3c80d1be0 2023-11-09T03:55:43.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.885+0000 7ff3cd913640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3c815d100 0x7ff3c81f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.885+0000 7ff3cd913640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3c815d100 0x7ff3c81f2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49638/0 (socket says 172.21.15.76:49638) 2023-11-09T03:55:43.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.885+0000 7ff3cd913640 1 -- 172.21.15.76:0/1987517770 learned_addr learned my addr 172.21.15.76:0/1987517770 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:43.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.886+0000 7ff3ce114640 1 --2- 172.21.15.76:0/1987517770 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3c80d1be0 0x7ff3c8149dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.885+0000 7ff3ce915640 1 --2- 172.21.15.76:0/1987517770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3c815d4e0 0x7ff3c814be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.886+0000 7ff3cd913640 1 -- 172.21.15.76:0/1987517770 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3c80d1be0 msgr2=0x7ff3c8149dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:43.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.886+0000 7ff3cd913640 1 --2- 172.21.15.76:0/1987517770 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3c80d1be0 0x7ff3c8149dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.886+0000 7ff3cd913640 1 -- 172.21.15.76:0/1987517770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3c815d4e0 msgr2=0x7ff3c814be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:43.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.886+0000 7ff3cd913640 1 --2- 172.21.15.76:0/1987517770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3c815d4e0 0x7ff3c814be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:43.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.886+0000 7ff3cd913640 1 -- 172.21.15.76:0/1987517770 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c4009a70 con 0x7ff3c815d100 2023-11-09T03:55:43.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.886+0000 7ff3ce114640 1 --2- 172.21.15.76:0/1987517770 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3c80d1be0 0x7ff3c8149dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:55:43.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.886+0000 7ff3ce915640 1 --2- 172.21.15.76:0/1987517770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3c815d4e0 0x7ff3c814be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:55:43.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.886+0000 7ff3cd913640 1 --2- 172.21.15.76:0/1987517770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3c815d100 0x7ff3c81f2590 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7ff3b800c4b0 tx=0x7ff3b800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:43.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.886+0000 7ff3beffd640 1 -- 172.21.15.76:0/1987517770 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b80074e0 con 0x7ff3c815d100 2023-11-09T03:55:43.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.886+0000 7ff3beffd640 1 -- 172.21.15.76:0/1987517770 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b8007660 con 0x7ff3c815d100 2023-11-09T03:55:43.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.886+0000 7ff3d0378640 1 -- 172.21.15.76:0/1987517770 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c80b7860 con 0x7ff3c815d100 2023-11-09T03:55:43.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.886+0000 7ff3beffd640 1 -- 172.21.15.76:0/1987517770 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8004a20 con 0x7ff3c815d100 2023-11-09T03:55:43.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.886+0000 7ff3d0378640 1 -- 172.21.15.76:0/1987517770 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c814c340 con 0x7ff3c815d100 2023-11-09T03:55:43.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.887+0000 7ff3d0378640 1 -- 172.21.15.76:0/1987517770 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c80b73c0 con 0x7ff3c815d100 2023-11-09T03:55:43.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.887+0000 7ff3beffd640 1 -- 172.21.15.76:0/1987517770 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff3b8004b80 con 0x7ff3c815d100 2023-11-09T03:55:43.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.888+0000 7ff3beffd640 1 --2- 172.21.15.76:0/1987517770 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff3ac06aea0 0x7ff3ac06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:43.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.888+0000 7ff3beffd640 1 -- 172.21.15.76:0/1987517770 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8554+0+0 (secure 0 0 0) 0x7ff3b80895b0 con 0x7ff3c815d100 2023-11-09T03:55:43.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.888+0000 7ff3ce114640 1 --2- 172.21.15.76:0/1987517770 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff3ac06aea0 0x7ff3ac06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:43.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.889+0000 7ff3ce114640 1 --2- 172.21.15.76:0/1987517770 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff3ac06aea0 0x7ff3ac06d360 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4036000 tx=0x7ff3c4030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:43.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:43.892+0000 7ff3beffd640 1 -- 172.21.15.76:0/1987517770 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff3b8053c40 con 0x7ff3c815d100 2023-11-09T03:55:44.061 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.062+0000 7ff3d0378640 1 -- 172.21.15.76:0/1987517770 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3c80b7d80 con 0x7ff3c815d100 2023-11-09T03:55:44.063 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.063+0000 7ff3beffd640 1 -- 172.21.15.76:0/1987517770 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+24223 (secure 0 0 0) 0x7ff3b8018020 con 0x7ff3c815d100 2023-11-09T03:55:44.063 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:55:44.063 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":120,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:55:40.713638+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-09T03:47:59.280601+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-09T03:47:59.742039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:48:55.490832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-09T03:48:56.352624+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":14,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.17","mappings":[{"from":0,"to":2}]},{"pgid":"17.36","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/497547142":"2023-11-09T04:55:40.451393+0000","172.21.15.76:6834/757529522":"2023-11-10T03:55:31.678438+0000","172.21.15.76:6832/3697850101":"2023-11-10T03:51:39.442929+0000","172.21.15.113:6837/3449910842":"2023-11-10T03:50:59.611256+0000","172.21.15.76:6832/1536527256":"2023-11-10T03:52:34.213804+0000","172.21.15.76:6835/3229239133":"2023-11-10T03:51:32.624856+0000","172.21.15.113:6836/445306598":"2023-11-10T03:52:27.406838+0000","172.21.15.76:6835/1663988060":"2023-11-10T03:50:33.665092+0000","172.21.15.76:6834/1663988060":"2023-11-10T03:50:33.665092+0000","172.21.15.113:6837/3280098776":"2023-11-10T03:50:07.616780+0000","192.168.0.1:0/2519773110":"2023-11-09T04:54:14.823765+0000","172.21.15.76:6835/4011275120":"2023-11-10T03:49:42.029057+0000","172.21.15.76:6835/757529522":"2023-11-10T03:55:31.678438+0000","172.21.15.76:6834/4011275120":"2023-11-10T03:49:42.029057+0000","172.21.15.113:6837/3462040782":"2023-11-10T03:54:45.345437+0000","192.168.0.1:0/503424885":"2023-11-09T04:51:47.727422+0000","172.21.15.76:6834/3229239133":"2023-11-10T03:51:32.624856+0000","172.21.15.113:6837/445306598":"2023-11-10T03:52:27.406838+0000","172.21.15.113:6836/3280098776":"2023-11-10T03:50:07.616780+0000","192.168.0.1:0/3087804343":"2023-11-09T04:52:42.513556+0000","172.21.15.76:6834/546338722":"2023-11-10T03:53:17.470106+0000","172.21.15.76:6833/1536527256":"2023-11-10T03:52:34.213804+0000","172.21.15.76:6835/546338722":"2023-11-10T03:53:17.470106+0000","172.21.15.113:6836/7360844":"2023-11-10T03:53:24.718008+0000","172.21.15.113:6837/7360844":"2023-11-10T03:53:24.718008+0000","172.21.15.76:6835/2075292158":"2023-11-10T03:54:06.568191+0000","192.168.0.1:0/334503200":"2023-11-09T04:53:32.572840+0000","172.21.15.76:6834/2075292158":"2023-11-10T03:54:06.568191+0000","172.21.15.76:6833/3697850101":"2023-11-10T03:51:39.442929+0000","172.21.15.113:6836/3462040782":"2023-11-10T03:54:45.345437+0000","172.21.15.113:6836/3449910842":"2023-11-10T03:50:59.611256+0000","192.168.0.1:0/3870807407":"2023-11-09T04:54:56.671385+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:55:44.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.065+0000 7ff3d0378640 1 -- 172.21.15.76:0/1987517770 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff3ac06aea0 msgr2=0x7ff3ac06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:44.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.065+0000 7ff3d0378640 1 --2- 172.21.15.76:0/1987517770 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff3ac06aea0 0x7ff3ac06d360 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4036000 tx=0x7ff3c4030000 comp rx=0 tx=0).stop 2023-11-09T03:55:44.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.065+0000 7ff3d0378640 1 -- 172.21.15.76:0/1987517770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3c815d100 msgr2=0x7ff3c81f2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:44.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.065+0000 7ff3d0378640 1 --2- 172.21.15.76:0/1987517770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3c815d100 0x7ff3c81f2590 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7ff3b800c4b0 tx=0x7ff3b800c970 comp rx=0 tx=0).stop 2023-11-09T03:55:44.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.065+0000 7ff3d0378640 1 -- 172.21.15.76:0/1987517770 shutdown_connections 2023-11-09T03:55:44.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.065+0000 7ff3d0378640 1 --2- 172.21.15.76:0/1987517770 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff3ac06aea0 0x7ff3ac06d360 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.065+0000 7ff3d0378640 1 --2- 172.21.15.76:0/1987517770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3c815d4e0 0x7ff3c814be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.065+0000 7ff3d0378640 1 --2- 172.21.15.76:0/1987517770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3c815d100 0x7ff3c81f2590 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.065+0000 7ff3d0378640 1 --2- 172.21.15.76:0/1987517770 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3c80d1be0 0x7ff3c8149dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.065+0000 7ff3d0378640 1 -- 172.21.15.76:0/1987517770 >> 172.21.15.76:0/1987517770 conn(0x7ff3c80d2ef0 msgr2=0x7ff3c80b4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:44.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.065+0000 7ff3d0378640 1 -- 172.21.15.76:0/1987517770 shutdown_connections 2023-11-09T03:55:44.067 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.065+0000 7ff3d0378640 1 -- 172.21.15.76:0/1987517770 wait complete. 2023-11-09T03:55:44.075 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-09T03:47:00.036540+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_data_pool', 'create_time': '2023-11-09T03:47:59.280601+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 70, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-09T03:47:59.742039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:48:55.490832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:48:56.352624+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 14, 'snap_epoch': 113, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-09T03:55:44.076 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:55:44.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.179+0000 7f525bb25640 1 Processor -- start 2023-11-09T03:55:44.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.179+0000 7f525bb25640 1 -- start start 2023-11-09T03:55:44.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.180+0000 7f525bb25640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f525415d4e0 0x7f52540ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:44.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.180+0000 7f525bb25640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f52540ced90 0x7f52540ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:44.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.180+0000 7f525bb25640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52540b25a0 con 0x7f52540ca6e0 2023-11-09T03:55:44.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.180+0000 7f525bb25640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52540cab70 con 0x7f52540ced90 2023-11-09T03:55:44.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.180+0000 7f525bb25640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52540b2300 con 0x7f525415d4e0 2023-11-09T03:55:44.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.180+0000 7f52590c0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f52540ced90 0x7f52540ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:44.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.180+0000 7f525a0c2640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f52540ca6e0 0x7f5254160fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37652/0 (socket says 172.21.15.76:37652) 2023-11-09T03:55:44.181 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.180+0000 7f525a0c2640 1 -- 172.21.15.76:0/1491801256 learned_addr learned my addr 172.21.15.76:0/1491801256 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:55:44.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.180+0000 7f52590c0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f52540ced90 0x7f52540ca1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49640/0 (socket says 172.21.15.76:49640) 2023-11-09T03:55:44.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.180+0000 7f52598c1640 1 --2- 172.21.15.76:0/1491801256 >> v2:172.21.15.113:3300/0 conn(0x7f525415d4e0 0x7f52540ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:44.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.180+0000 7f52590c0640 1 -- 172.21.15.76:0/1491801256 >> v2:172.21.15.113:3300/0 conn(0x7f525415d4e0 msgr2=0x7f52540ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:44.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.180+0000 7f52590c0640 1 --2- 172.21.15.76:0/1491801256 >> v2:172.21.15.113:3300/0 conn(0x7f525415d4e0 0x7f52540ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.180+0000 7f52590c0640 1 -- 172.21.15.76:0/1491801256 >> v1:172.21.15.76:6789/0 conn(0x7f52540ca6e0 legacy=0x7f5254160fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:44.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.180+0000 7f52590c0640 1 -- 172.21.15.76:0/1491801256 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525415d8c0 con 0x7f52540ced90 2023-11-09T03:55:44.183 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.180+0000 7f52588bf640 1 -- 172.21.15.76:0/1491801256 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897569800 0 0) 0x7f52540b25a0 con 0x7f52540ca6e0 2023-11-09T03:55:44.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.181+0000 7f52590c0640 1 --2- 172.21.15.76:0/1491801256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52540ced90 0x7f52540ca1a0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f52540699b0 tx=0x7f524402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4abdc6d6bb7610 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:44.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.181+0000 7f52588bf640 1 -- 172.21.15.76:0/1491801256 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5244031830 con 0x7f52540ced90 2023-11-09T03:55:44.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.181+0000 7f52588bf640 1 -- 172.21.15.76:0/1491801256 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5244031990 con 0x7f52540ced90 2023-11-09T03:55:44.186 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.181+0000 7f52588bf640 1 -- 172.21.15.76:0/1491801256 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5244031d20 con 0x7f52540ced90 2023-11-09T03:55:44.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.182+0000 7f525bb25640 1 -- 172.21.15.76:0/1491801256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52540ced90 msgr2=0x7f52540ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:44.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.182+0000 7f525bb25640 1 --2- 172.21.15.76:0/1491801256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52540ced90 0x7f52540ca1a0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f52540699b0 tx=0x7f524402b8d0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.182+0000 7f525bb25640 1 -- 172.21.15.76:0/1491801256 shutdown_connections 2023-11-09T03:55:44.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.182+0000 7f525bb25640 1 --2- 172.21.15.76:0/1491801256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52540ced90 0x7f52540ca1a0 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.187 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.182+0000 7f525bb25640 1 --2- 172.21.15.76:0/1491801256 >> v2:172.21.15.113:3300/0 conn(0x7f525415d4e0 0x7f52540ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.182+0000 7f525bb25640 1 -- 172.21.15.76:0/1491801256 >> 172.21.15.76:0/1491801256 conn(0x7f52540b1640 msgr2=0x7f52540b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:44.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.182+0000 7f525bb25640 1 -- 172.21.15.76:0/1491801256 shutdown_connections 2023-11-09T03:55:44.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.182+0000 7f525bb25640 1 -- 172.21.15.76:0/1491801256 wait complete. 2023-11-09T03:55:44.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.183+0000 7f525bb25640 1 Processor -- start 2023-11-09T03:55:44.188 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.183+0000 7f525bb25640 1 -- start start 2023-11-09T03:55:44.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.184+0000 7f525bb25640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52540ca6e0 0x7f52540c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:44.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.184+0000 7f525bb25640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52540ced90 0x7f525414efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:44.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.184+0000 7f525bb25640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f525415d4e0 0x7f525414f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:44.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.184+0000 7f525bb25640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5254159660 con 0x7f525415d4e0 2023-11-09T03:55:44.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.184+0000 7f525bb25640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525414f500 con 0x7f52540ca6e0 2023-11-09T03:55:44.189 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.184+0000 7f525bb25640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52540c6500 con 0x7f52540ced90 2023-11-09T03:55:44.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.184+0000 7f525a0c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f525415d4e0 0x7f525414f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:44.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.184+0000 7f525a0c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f525415d4e0 0x7f525414f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49654/0 (socket says 172.21.15.76:49654) 2023-11-09T03:55:44.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.184+0000 7f525a0c2640 1 -- 172.21.15.76:0/814275766 learned_addr learned my addr 172.21.15.76:0/814275766 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:44.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.184+0000 7f52598c1640 1 --2- 172.21.15.76:0/814275766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52540ca6e0 0x7f52540c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:44.190 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.184+0000 7f52590c0640 1 --2- 172.21.15.76:0/814275766 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52540ced90 0x7f525414efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:44.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.184+0000 7f525a0c2640 1 -- 172.21.15.76:0/814275766 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52540ced90 msgr2=0x7f525414efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:44.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.184+0000 7f525a0c2640 1 --2- 172.21.15.76:0/814275766 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52540ced90 0x7f525414efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.184+0000 7f525a0c2640 1 -- 172.21.15.76:0/814275766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52540ca6e0 msgr2=0x7f52540c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:44.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.184+0000 7f525a0c2640 1 --2- 172.21.15.76:0/814275766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52540ca6e0 0x7f52540c5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.184+0000 7f525a0c2640 1 -- 172.21.15.76:0/814275766 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5244009400 con 0x7f525415d4e0 2023-11-09T03:55:44.191 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.184+0000 7f525a0c2640 1 --2- 172.21.15.76:0/814275766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f525415d4e0 0x7f525414f800 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f524c00c7e0 tx=0x7f524c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:44.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.184+0000 7f524a7fc640 1 -- 172.21.15.76:0/814275766 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c00a640 con 0x7f525415d4e0 2023-11-09T03:55:44.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.184+0000 7f524a7fc640 1 -- 172.21.15.76:0/814275766 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524c00a7c0 con 0x7f525415d4e0 2023-11-09T03:55:44.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.184+0000 7f525bb25640 1 -- 172.21.15.76:0/814275766 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52540b3ee0 con 0x7f525415d4e0 2023-11-09T03:55:44.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.184+0000 7f524a7fc640 1 -- 172.21.15.76:0/814275766 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c00ab70 con 0x7f525415d4e0 2023-11-09T03:55:44.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.184+0000 7f525bb25640 1 -- 172.21.15.76:0/814275766 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52540b44a0 con 0x7f525415d4e0 2023-11-09T03:55:44.192 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.185+0000 7f525bb25640 1 -- 172.21.15.76:0/814275766 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52540684b0 con 0x7f525415d4e0 2023-11-09T03:55:44.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.186+0000 7f524a7fc640 1 -- 172.21.15.76:0/814275766 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f524c00acd0 con 0x7f525415d4e0 2023-11-09T03:55:44.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.186+0000 7f524a7fc640 1 --2- 172.21.15.76:0/814275766 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f523806ae70 0x7f523806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:44.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.186+0000 7f524a7fc640 1 -- 172.21.15.76:0/814275766 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8554+0+0 (secure 0 0 0) 0x7f524c0899b0 con 0x7f525415d4e0 2023-11-09T03:55:44.193 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.186+0000 7f52598c1640 1 --2- 172.21.15.76:0/814275766 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f523806ae70 0x7f523806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:44.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.187+0000 7f52598c1640 1 --2- 172.21.15.76:0/814275766 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f523806ae70 0x7f523806d330 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5250006fd0 tx=0x7f5250008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:44.194 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.190+0000 7f524a7fc640 1 -- 172.21.15.76:0/814275766 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f524c053f80 con 0x7f525415d4e0 2023-11-09T03:55:44.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.356+0000 7f525bb25640 1 -- 172.21.15.76:0/814275766 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f52540cacd0 con 0x7f525415d4e0 2023-11-09T03:55:44.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.356+0000 7f524a7fc640 1 -- 172.21.15.76:0/814275766 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v120) v1 ==== 93+0+10 (secure 0 0 0) 0x7f524c018070 con 0x7f525415d4e0 2023-11-09T03:55:44.356 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:55:44.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.358+0000 7f525bb25640 1 -- 172.21.15.76:0/814275766 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f523806ae70 msgr2=0x7f523806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:44.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.358+0000 7f525bb25640 1 --2- 172.21.15.76:0/814275766 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f523806ae70 0x7f523806d330 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5250006fd0 tx=0x7f5250008040 comp rx=0 tx=0).stop 2023-11-09T03:55:44.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.358+0000 7f525bb25640 1 -- 172.21.15.76:0/814275766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f525415d4e0 msgr2=0x7f525414f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:44.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.358+0000 7f525bb25640 1 --2- 172.21.15.76:0/814275766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f525415d4e0 0x7f525414f800 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f524c00c7e0 tx=0x7f524c00cca0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.358+0000 7f525bb25640 1 -- 172.21.15.76:0/814275766 shutdown_connections 2023-11-09T03:55:44.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.358+0000 7f525bb25640 1 --2- 172.21.15.76:0/814275766 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f523806ae70 0x7f523806d330 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.359 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.358+0000 7f525bb25640 1 --2- 172.21.15.76:0/814275766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f525415d4e0 0x7f525414f800 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.358+0000 7f525bb25640 1 --2- 172.21.15.76:0/814275766 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52540ced90 0x7f525414efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.358+0000 7f525bb25640 1 --2- 172.21.15.76:0/814275766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52540ca6e0 0x7f52540c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.358+0000 7f525bb25640 1 -- 172.21.15.76:0/814275766 >> 172.21.15.76:0/814275766 conn(0x7f52540b1640 msgr2=0x7f525415ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:44.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.358+0000 7f525bb25640 1 -- 172.21.15.76:0/814275766 shutdown_connections 2023-11-09T03:55:44.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.358+0000 7f525bb25640 1 -- 172.21.15.76:0/814275766 wait complete. 2023-11-09T03:55:44.369 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-09T03:55:44.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.473+0000 7f3bb667c640 1 Processor -- start 2023-11-09T03:55:44.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.473+0000 7f3bb667c640 1 -- start start 2023-11-09T03:55:44.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.473+0000 7f3bb667c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3bb015d100 0x7f3bb00ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:44.475 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.473+0000 7f3bb667c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3bb015d4e0 0x7f3bb00d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:44.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.473+0000 7f3bb667c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bb00b32b0 con 0x7f3bb00d3b60 2023-11-09T03:55:44.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.473+0000 7f3bb667c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bb00cee10 con 0x7f3bb015d4e0 2023-11-09T03:55:44.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.473+0000 7f3bb667c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb00b4750 con 0x7f3bb015d100 2023-11-09T03:55:44.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.474+0000 7f3baf7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3bb015d4e0 0x7f3bb00d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:44.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.474+0000 7f3bb4c19640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f3bb00d3b60 0x7f3bb00cf520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37666/0 (socket says 172.21.15.76:37666) 2023-11-09T03:55:44.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.474+0000 7f3baf7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3bb015d4e0 0x7f3bb00d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49660/0 (socket says 172.21.15.76:49660) 2023-11-09T03:55:44.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.474+0000 7f3bb4c19640 1 -- 172.21.15.76:0/1301059261 learned_addr learned my addr 172.21.15.76:0/1301059261 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:55:44.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.474+0000 7f3baffff640 1 --2- 172.21.15.76:0/1301059261 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3bb015d100 0x7f3bb00ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:44.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.474+0000 7f3baf7fe640 1 -- 172.21.15.76:0/1301059261 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3bb015d100 msgr2=0x7f3bb00ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:44.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.474+0000 7f3baf7fe640 1 --2- 172.21.15.76:0/1301059261 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3bb015d100 0x7f3bb00ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.474+0000 7f3baf7fe640 1 -- 172.21.15.76:0/1301059261 >> v1:172.21.15.76:6789/0 conn(0x7f3bb00d3b60 legacy=0x7f3bb00cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:44.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.474+0000 7f3baf7fe640 1 -- 172.21.15.76:0/1301059261 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb00b2980 con 0x7f3bb015d4e0 2023-11-09T03:55:44.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.474+0000 7f3baeffd640 1 -- 172.21.15.76:0/1301059261 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 789201055 0 0) 0x7f3bb00b32b0 con 0x7f3bb00d3b60 2023-11-09T03:55:44.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.475+0000 7f3baf7fe640 1 --2- 172.21.15.76:0/1301059261 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bb015d4e0 0x7f3bb00d14d0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40097a0 tx=0x7f3ba402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0d56a925352d7e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:44.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.475+0000 7f3baeffd640 1 -- 172.21.15.76:0/1301059261 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4031830 con 0x7f3bb015d4e0 2023-11-09T03:55:44.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.475+0000 7f3baeffd640 1 -- 172.21.15.76:0/1301059261 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba4031990 con 0x7f3bb015d4e0 2023-11-09T03:55:44.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.475+0000 7f3baeffd640 1 -- 172.21.15.76:0/1301059261 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4031d20 con 0x7f3bb015d4e0 2023-11-09T03:55:44.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.475+0000 7f3bb667c640 1 -- 172.21.15.76:0/1301059261 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bb015d4e0 msgr2=0x7f3bb00d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:44.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.475+0000 7f3bb667c640 1 --2- 172.21.15.76:0/1301059261 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bb015d4e0 0x7f3bb00d14d0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40097a0 tx=0x7f3ba402b8d0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.475+0000 7f3bb667c640 1 -- 172.21.15.76:0/1301059261 shutdown_connections 2023-11-09T03:55:44.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.475+0000 7f3bb667c640 1 --2- 172.21.15.76:0/1301059261 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bb015d4e0 0x7f3bb00d14d0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.475+0000 7f3bb667c640 1 --2- 172.21.15.76:0/1301059261 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3bb015d100 0x7f3bb00ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.475+0000 7f3bb667c640 1 -- 172.21.15.76:0/1301059261 >> 172.21.15.76:0/1301059261 conn(0x7f3bb00d2ef0 msgr2=0x7f3bb00d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:44.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.475+0000 7f3bb667c640 1 -- 172.21.15.76:0/1301059261 shutdown_connections 2023-11-09T03:55:44.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.475+0000 7f3bb667c640 1 -- 172.21.15.76:0/1301059261 wait complete. 2023-11-09T03:55:44.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.477+0000 7f3bb667c640 1 Processor -- start 2023-11-09T03:55:44.482 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.477+0000 7f3bb667c640 1 -- start start 2023-11-09T03:55:44.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.477+0000 7f3bb667c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3bb00d3b60 0x7f3bb01574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:44.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.477+0000 7f3bb667c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3bb015d100 0x7f3bb00ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:44.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.477+0000 7f3bb667c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bb015d4e0 0x7f3bb015a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:44.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.477+0000 7f3bb667c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb0157a10 con 0x7f3bb015d4e0 2023-11-09T03:55:44.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.477+0000 7f3bb667c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb01f6510 con 0x7f3bb00d3b60 2023-11-09T03:55:44.483 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.477+0000 7f3bb667c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb01f67f0 con 0x7f3bb015d100 2023-11-09T03:55:44.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.478+0000 7f3bb4c19640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bb015d4e0 0x7f3bb015a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:44.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.478+0000 7f3bb4c19640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bb015d4e0 0x7f3bb015a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49672/0 (socket says 172.21.15.76:49672) 2023-11-09T03:55:44.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.478+0000 7f3bb4c19640 1 -- 172.21.15.76:0/1270292813 learned_addr learned my addr 172.21.15.76:0/1270292813 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:44.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.478+0000 7f3baffff640 1 --2- 172.21.15.76:0/1270292813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3bb00d3b60 0x7f3bb01574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:44.484 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.478+0000 7f3baf7fe640 1 --2- 172.21.15.76:0/1270292813 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3bb015d100 0x7f3bb00ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:44.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.478+0000 7f3bb4c19640 1 -- 172.21.15.76:0/1270292813 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3bb015d100 msgr2=0x7f3bb00ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:44.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.478+0000 7f3bb4c19640 1 --2- 172.21.15.76:0/1270292813 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3bb015d100 0x7f3bb00ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.478+0000 7f3bb4c19640 1 -- 172.21.15.76:0/1270292813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3bb00d3b60 msgr2=0x7f3bb01574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:44.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.478+0000 7f3bb4c19640 1 --2- 172.21.15.76:0/1270292813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3bb00d3b60 0x7f3bb01574d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.478+0000 7f3bb4c19640 1 -- 172.21.15.76:0/1270292813 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba4009400 con 0x7f3bb015d4e0 2023-11-09T03:55:44.485 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.478+0000 7f3bb4c19640 1 --2- 172.21.15.76:0/1270292813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bb015d4e0 0x7f3bb015a5b0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800c700 tx=0x7f3ba800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:44.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.478+0000 7f3bacff9640 1 -- 172.21.15.76:0/1270292813 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba800ce10 con 0x7f3bb015d4e0 2023-11-09T03:55:44.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.478+0000 7f3bacff9640 1 -- 172.21.15.76:0/1270292813 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba800a640 con 0x7f3bb015d4e0 2023-11-09T03:55:44.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.478+0000 7f3bacff9640 1 -- 172.21.15.76:0/1270292813 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba800aa70 con 0x7f3bb015d4e0 2023-11-09T03:55:44.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.478+0000 7f3bb667c640 1 -- 172.21.15.76:0/1270292813 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb01f6ad0 con 0x7f3bb015d4e0 2023-11-09T03:55:44.486 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.478+0000 7f3bb667c640 1 -- 172.21.15.76:0/1270292813 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb01f7090 con 0x7f3bb015d4e0 2023-11-09T03:55:44.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.479+0000 7f3bb667c640 1 -- 172.21.15.76:0/1270292813 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bb01f7280 con 0x7f3bb015d4e0 2023-11-09T03:55:44.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.480+0000 7f3bacff9640 1 -- 172.21.15.76:0/1270292813 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3ba800abd0 con 0x7f3bb015d4e0 2023-11-09T03:55:44.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.480+0000 7f3bacff9640 1 --2- 172.21.15.76:0/1270292813 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3b9c06ae70 0x7f3b9c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:44.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.480+0000 7f3bacff9640 1 -- 172.21.15.76:0/1270292813 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8554+0+0 (secure 0 0 0) 0x7f3ba8089560 con 0x7f3bb015d4e0 2023-11-09T03:55:44.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.480+0000 7f3baffff640 1 --2- 172.21.15.76:0/1270292813 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3b9c06ae70 0x7f3b9c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:44.487 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.481+0000 7f3baffff640 1 --2- 172.21.15.76:0/1270292813 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3b9c06ae70 0x7f3b9c06d330 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0006fd0 tx=0x7f3ba0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:44.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.484+0000 7f3bacff9640 1 -- 172.21.15.76:0/1270292813 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3ba8053be0 con 0x7f3bb015d4e0 2023-11-09T03:55:44.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.650+0000 7f3bb667c640 1 -- 172.21.15.76:0/1270292813 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f3bb00684b0 con 0x7f3bb015d4e0 2023-11-09T03:55:44.650 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.651+0000 7f3bacff9640 1 -- 172.21.15.76:0/1270292813 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v120) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3ba8018070 con 0x7f3bb015d4e0 2023-11-09T03:55:44.651 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:55:44.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.652+0000 7f3bb667c640 1 -- 172.21.15.76:0/1270292813 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3b9c06ae70 msgr2=0x7f3b9c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:44.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.652+0000 7f3bb667c640 1 --2- 172.21.15.76:0/1270292813 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3b9c06ae70 0x7f3b9c06d330 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0006fd0 tx=0x7f3ba0008040 comp rx=0 tx=0).stop 2023-11-09T03:55:44.652 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.652+0000 7f3bb667c640 1 -- 172.21.15.76:0/1270292813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bb015d4e0 msgr2=0x7f3bb015a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:44.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.652+0000 7f3bb667c640 1 --2- 172.21.15.76:0/1270292813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bb015d4e0 0x7f3bb015a5b0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800c700 tx=0x7f3ba800cbc0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.652+0000 7f3bb667c640 1 -- 172.21.15.76:0/1270292813 shutdown_connections 2023-11-09T03:55:44.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.652+0000 7f3bb667c640 1 --2- 172.21.15.76:0/1270292813 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3b9c06ae70 0x7f3b9c06d330 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.652+0000 7f3bb667c640 1 --2- 172.21.15.76:0/1270292813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bb015d4e0 0x7f3bb015a5b0 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.652+0000 7f3bb667c640 1 --2- 172.21.15.76:0/1270292813 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3bb015d100 0x7f3bb00ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.652+0000 7f3bb667c640 1 --2- 172.21.15.76:0/1270292813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3bb00d3b60 0x7f3bb01574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.652+0000 7f3bb667c640 1 -- 172.21.15.76:0/1270292813 >> 172.21.15.76:0/1270292813 conn(0x7f3bb00d2ef0 msgr2=0x7f3bb006ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:44.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.652+0000 7f3bb667c640 1 -- 172.21.15.76:0/1270292813 shutdown_connections 2023-11-09T03:55:44.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.652+0000 7f3bb667c640 1 -- 172.21.15.76:0/1270292813 wait complete. 2023-11-09T03:55:44.663 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-09T03:55:44.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.766+0000 7f6988ae0640 1 Processor -- start 2023-11-09T03:55:44.766 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.766+0000 7f6988ae0640 1 -- start start 2023-11-09T03:55:44.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.767+0000 7f6988ae0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f698415d100 0x7f69840ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:44.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.767+0000 7f6988ae0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f69840d1be0 0x7f69840b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:44.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.767+0000 7f6988ae0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69840b32b0 con 0x7f698415d4e0 2023-11-09T03:55:44.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.767+0000 7f6988ae0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69840cee10 con 0x7f69840d1be0 2023-11-09T03:55:44.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.767+0000 7f6988ae0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69840b2980 con 0x7f698415d100 2023-11-09T03:55:44.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.767+0000 7f6982d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f69840d1be0 0x7f69840b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:44.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.767+0000 7f6981d9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f698415d4e0 0x7f69840d14d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37682/0 (socket says 172.21.15.76:37682) 2023-11-09T03:55:44.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.768+0000 7f6981d9b640 1 -- 172.21.15.76:0/1631120420 learned_addr learned my addr 172.21.15.76:0/1631120420 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:55:44.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.768+0000 7f6982d9d640 1 -- 172.21.15.76:0/1631120420 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f698415d100 msgr2=0x7f69840ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:44.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.768+0000 7f6982d9d640 1 --2- 172.21.15.76:0/1631120420 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f698415d100 0x7f69840ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.768+0000 7f6982d9d640 1 -- 172.21.15.76:0/1631120420 >> v1:172.21.15.76:6789/0 conn(0x7f698415d4e0 legacy=0x7f69840d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:44.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.768+0000 7f6982d9d640 1 -- 172.21.15.76:0/1631120420 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69840b2c60 con 0x7f69840d1be0 2023-11-09T03:55:44.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.768+0000 7f698159a640 1 -- 172.21.15.76:0/1631120420 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2825496673 0 0) 0x7f69840b32b0 con 0x7f698415d4e0 2023-11-09T03:55:44.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.768+0000 7f6982d9d640 1 --2- 172.21.15.76:0/1631120420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69840d1be0 0x7f69840b3740 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f697c0097d0 tx=0x7f697c02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139f5ec2398f4181 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:44.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.769+0000 7f698159a640 1 -- 172.21.15.76:0/1631120420 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c031800 con 0x7f69840d1be0 2023-11-09T03:55:44.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.769+0000 7f698159a640 1 -- 172.21.15.76:0/1631120420 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697c031960 con 0x7f69840d1be0 2023-11-09T03:55:44.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.769+0000 7f698159a640 1 -- 172.21.15.76:0/1631120420 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c031cf0 con 0x7f69840d1be0 2023-11-09T03:55:44.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.769+0000 7f6988ae0640 1 -- 172.21.15.76:0/1631120420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69840d1be0 msgr2=0x7f69840b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:44.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.769+0000 7f6988ae0640 1 --2- 172.21.15.76:0/1631120420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69840d1be0 0x7f69840b3740 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f697c0097d0 tx=0x7f697c02b8a0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.769+0000 7f6988ae0640 1 -- 172.21.15.76:0/1631120420 shutdown_connections 2023-11-09T03:55:44.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.769+0000 7f6988ae0640 1 --2- 172.21.15.76:0/1631120420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69840d1be0 0x7f69840b3740 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.769+0000 7f6988ae0640 1 --2- 172.21.15.76:0/1631120420 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f698415d100 0x7f69840ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.769+0000 7f6988ae0640 1 -- 172.21.15.76:0/1631120420 >> 172.21.15.76:0/1631120420 conn(0x7f69840d2ef0 msgr2=0x7f69840d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:44.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.769+0000 7f6988ae0640 1 -- 172.21.15.76:0/1631120420 shutdown_connections 2023-11-09T03:55:44.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.769+0000 7f6988ae0640 1 -- 172.21.15.76:0/1631120420 wait complete. 2023-11-09T03:55:44.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.771+0000 7f6988ae0640 1 Processor -- start 2023-11-09T03:55:44.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.771+0000 7f6988ae0640 1 -- start start 2023-11-09T03:55:44.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.771+0000 7f6988ae0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69840d1be0 0x7f6984150780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:44.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.771+0000 7f6988ae0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f698415d100 0x7f698415eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:44.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.771+0000 7f6988ae0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f698415d4e0 0x7f698415f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:44.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.771+0000 7f6988ae0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69840cba60 con 0x7f698415d4e0 2023-11-09T03:55:44.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.771+0000 7f6988ae0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69840b8dc0 con 0x7f698415d100 2023-11-09T03:55:44.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.771+0000 7f6988ae0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69840b8ae0 con 0x7f69840d1be0 2023-11-09T03:55:44.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.771+0000 7f6982d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f698415d4e0 0x7f698415f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:44.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.771+0000 7f6982d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f698415d4e0 0x7f698415f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49688/0 (socket says 172.21.15.76:49688) 2023-11-09T03:55:44.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.771+0000 7f6982d9d640 1 -- 172.21.15.76:0/3117449322 learned_addr learned my addr 172.21.15.76:0/3117449322 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:44.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.772+0000 7f6982d9d640 1 -- 172.21.15.76:0/3117449322 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69840d1be0 msgr2=0x7f6984150780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:44.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.772+0000 7f6982d9d640 1 --2- 172.21.15.76:0/3117449322 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69840d1be0 0x7f6984150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.772+0000 7f6982d9d640 1 -- 172.21.15.76:0/3117449322 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f698415d100 msgr2=0x7f698415eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:44.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.772+0000 7f6982d9d640 1 --2- 172.21.15.76:0/3117449322 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f698415d100 0x7f698415eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.772+0000 7f6982d9d640 1 -- 172.21.15.76:0/3117449322 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c009400 con 0x7f698415d4e0 2023-11-09T03:55:44.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.772+0000 7f6982d9d640 1 --2- 172.21.15.76:0/3117449322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f698415d4e0 0x7f698415f430 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f697c02beb0 tx=0x7f697c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:44.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.772+0000 7f6972ffd640 1 -- 172.21.15.76:0/3117449322 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c03ac30 con 0x7f698415d4e0 2023-11-09T03:55:44.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.772+0000 7f6972ffd640 1 -- 172.21.15.76:0/3117449322 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697c03ad90 con 0x7f698415d4e0 2023-11-09T03:55:44.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.772+0000 7f6988ae0640 1 -- 172.21.15.76:0/3117449322 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69840cb410 con 0x7f698415d4e0 2023-11-09T03:55:44.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.772+0000 7f6972ffd640 1 -- 172.21.15.76:0/3117449322 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c044630 con 0x7f698415d4e0 2023-11-09T03:55:44.778 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.772+0000 7f6988ae0640 1 -- 172.21.15.76:0/3117449322 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6984150cc0 con 0x7f698415d4e0 2023-11-09T03:55:44.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.773+0000 7f6988ae0640 1 -- 172.21.15.76:0/3117449322 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69840684b0 con 0x7f698415d4e0 2023-11-09T03:55:44.779 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.773+0000 7f6972ffd640 1 -- 172.21.15.76:0/3117449322 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f697c044790 con 0x7f698415d4e0 2023-11-09T03:55:44.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.774+0000 7f6972ffd640 1 --2- 172.21.15.76:0/3117449322 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f696406ae70 0x7f696406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:44.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.774+0000 7f6972ffd640 1 -- 172.21.15.76:0/3117449322 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8554+0+0 (secure 0 0 0) 0x7f697c02e070 con 0x7f698415d4e0 2023-11-09T03:55:44.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.774+0000 7f698259c640 1 --2- 172.21.15.76:0/3117449322 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f696406ae70 0x7f696406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:44.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.775+0000 7f698259c640 1 --2- 172.21.15.76:0/3117449322 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f696406ae70 0x7f696406d330 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f697400ab90 tx=0x7f6974009250 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:44.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.778+0000 7f6972ffd640 1 -- 172.21.15.76:0/3117449322 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f697c076b30 con 0x7f698415d4e0 2023-11-09T03:55:44.787 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:55:44.788+0000 7f6ade7fc640 -1 client.9441 I was blocklisted at osd epoch 120 2023-11-09T03:55:44.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.947+0000 7f6988ae0640 1 -- 172.21.15.76:0/3117449322 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7f69840d2cd0 con 0x7f698415d4e0 2023-11-09T03:55:44.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.947+0000 7f6972ffd640 1 -- 172.21.15.76:0/3117449322 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v120) v1 ==== 103+0+11 (secure 0 0 0) 0x7f697c07a150 con 0x7f698415d4e0 2023-11-09T03:55:44.947 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:55:44.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.949+0000 7f6988ae0640 1 -- 172.21.15.76:0/3117449322 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f696406ae70 msgr2=0x7f696406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:44.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.949+0000 7f6988ae0640 1 --2- 172.21.15.76:0/3117449322 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f696406ae70 0x7f696406d330 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f697400ab90 tx=0x7f6974009250 comp rx=0 tx=0).stop 2023-11-09T03:55:44.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.949+0000 7f6988ae0640 1 -- 172.21.15.76:0/3117449322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f698415d4e0 msgr2=0x7f698415f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:44.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.949+0000 7f6988ae0640 1 --2- 172.21.15.76:0/3117449322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f698415d4e0 0x7f698415f430 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f697c02beb0 tx=0x7f697c0046c0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.949+0000 7f6988ae0640 1 -- 172.21.15.76:0/3117449322 shutdown_connections 2023-11-09T03:55:44.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.949+0000 7f6988ae0640 1 --2- 172.21.15.76:0/3117449322 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f696406ae70 0x7f696406d330 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.949+0000 7f6988ae0640 1 --2- 172.21.15.76:0/3117449322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f698415d4e0 0x7f698415f430 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.949+0000 7f6988ae0640 1 --2- 172.21.15.76:0/3117449322 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f698415d100 0x7f698415eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.949+0000 7f6988ae0640 1 --2- 172.21.15.76:0/3117449322 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69840d1be0 0x7f6984150780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:44.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.949+0000 7f6988ae0640 1 -- 172.21.15.76:0/3117449322 >> 172.21.15.76:0/3117449322 conn(0x7f69840d2ef0 msgr2=0x7f69840b4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:44.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.949+0000 7f6988ae0640 1 -- 172.21.15.76:0/3117449322 shutdown_connections 2023-11-09T03:55:44.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:44.949+0000 7f6988ae0640 1 -- 172.21.15.76:0/3117449322 wait complete. 2023-11-09T03:55:44.959 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-09T03:55:45.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.063+0000 7fe7a7d2c640 1 Processor -- start 2023-11-09T03:55:45.062 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.063+0000 7fe7a7d2c640 1 -- start start 2023-11-09T03:55:45.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.064+0000 7fe7a7d2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe7a015d100 0x7fe7a00ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:45.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.064+0000 7fe7a7d2c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7a015d4e0 0x7fe7a00d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:45.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.064+0000 7fe7a7d2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe7a00d3b60 0x7fe7a00d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:45.064 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.064+0000 7fe7a7d2c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a00b32b0 con 0x7fe7a015d100 2023-11-09T03:55:45.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.064+0000 7fe7a7d2c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a00cee10 con 0x7fe7a00d3b60 2023-11-09T03:55:45.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.064+0000 7fe7a7d2c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a00b2980 con 0x7fe7a015d4e0 2023-11-09T03:55:45.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.064+0000 7fe7a5ac8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe7a015d100 0x7fe7a00ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:45.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.064+0000 7fe7a5ac8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe7a015d100 0x7fe7a00ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49702/0 (socket says 172.21.15.76:49702) 2023-11-09T03:55:45.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.064+0000 7fe7a5ac8640 1 -- 172.21.15.76:0/3291674431 learned_addr learned my addr 172.21.15.76:0/3291674431 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:45.065 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.064+0000 7fe7a5ac8640 1 -- 172.21.15.76:0/3291674431 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7a015d4e0 msgr2=0x7fe7a00d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:45.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.064+0000 7fe7a5ac8640 1 --2- 172.21.15.76:0/3291674431 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7a015d4e0 0x7fe7a00d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.064+0000 7fe7a5ac8640 1 -- 172.21.15.76:0/3291674431 >> v2:172.21.15.113:3300/0 conn(0x7fe7a00d3b60 msgr2=0x7fe7a00d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:45.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.064+0000 7fe7a62c9640 1 --2- 172.21.15.76:0/3291674431 >> v2:172.21.15.113:3300/0 conn(0x7fe7a00d3b60 0x7fe7a00d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:45.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.064+0000 7fe7a5ac8640 1 --2- 172.21.15.76:0/3291674431 >> v2:172.21.15.113:3300/0 conn(0x7fe7a00d3b60 0x7fe7a00d1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.066 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.064+0000 7fe7a5ac8640 1 -- 172.21.15.76:0/3291674431 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a00b2c60 con 0x7fe7a015d100 2023-11-09T03:55:45.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.065+0000 7fe7a5ac8640 1 --2- 172.21.15.76:0/3291674431 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7a015d100 0x7fe7a00ce510 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00b27a0 tx=0x7fe79c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d491c7448b1f11d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:45.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.065+0000 7fe7a4ac6640 1 -- 172.21.15.76:0/3291674431 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c004230 con 0x7fe7a015d100 2023-11-09T03:55:45.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.065+0000 7fe7a4ac6640 1 -- 172.21.15.76:0/3291674431 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79c004390 con 0x7fe7a015d100 2023-11-09T03:55:45.069 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.065+0000 7fe7a4ac6640 1 -- 172.21.15.76:0/3291674431 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c003810 con 0x7fe7a015d100 2023-11-09T03:55:45.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.066+0000 7fe7a7d2c640 1 -- 172.21.15.76:0/3291674431 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7a015d100 msgr2=0x7fe7a00ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:45.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.066+0000 7fe7a7d2c640 1 --2- 172.21.15.76:0/3291674431 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7a015d100 0x7fe7a00ce510 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00b27a0 tx=0x7fe79c02b8f0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.066+0000 7fe7a7d2c640 1 -- 172.21.15.76:0/3291674431 shutdown_connections 2023-11-09T03:55:45.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.066+0000 7fe7a7d2c640 1 --2- 172.21.15.76:0/3291674431 >> v2:172.21.15.113:3300/0 conn(0x7fe7a00d3b60 0x7fe7a00d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.070 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.066+0000 7fe7a7d2c640 1 --2- 172.21.15.76:0/3291674431 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7a015d4e0 0x7fe7a00d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.066+0000 7fe7a7d2c640 1 --2- 172.21.15.76:0/3291674431 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7a015d100 0x7fe7a00ce510 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.066+0000 7fe7a7d2c640 1 -- 172.21.15.76:0/3291674431 >> 172.21.15.76:0/3291674431 conn(0x7fe7a00d2ef0 msgr2=0x7fe7a00d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:45.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.066+0000 7fe7a7d2c640 1 -- 172.21.15.76:0/3291674431 shutdown_connections 2023-11-09T03:55:45.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.066+0000 7fe7a7d2c640 1 -- 172.21.15.76:0/3291674431 wait complete. 2023-11-09T03:55:45.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.067+0000 7fe7a7d2c640 1 Processor -- start 2023-11-09T03:55:45.071 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.067+0000 7fe7a7d2c640 1 -- start start 2023-11-09T03:55:45.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.068+0000 7fe7a7d2c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7a00d3b60 0x7fe7a01f6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:45.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.068+0000 7fe7a7d2c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7a015d100 0x7fe7a014bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:45.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.068+0000 7fe7a7d2c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7a015d4e0 0x7fe7a009db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:45.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.068+0000 7fe7a7d2c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a014c1e0 con 0x7fe7a015d100 2023-11-09T03:55:45.072 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.068+0000 7fe7a7d2c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a00cbeb0 con 0x7fe7a015d4e0 2023-11-09T03:55:45.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.068+0000 7fe7a7d2c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a009e060 con 0x7fe7a00d3b60 2023-11-09T03:55:45.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.068+0000 7fe7a52c7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7a015d100 0x7fe7a014bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:45.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.068+0000 7fe7a52c7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7a015d100 0x7fe7a014bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49710/0 (socket says 172.21.15.76:49710) 2023-11-09T03:55:45.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.068+0000 7fe7a52c7640 1 -- 172.21.15.76:0/1519710651 learned_addr learned my addr 172.21.15.76:0/1519710651 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:45.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.068+0000 7fe7a62c9640 1 --2- 172.21.15.76:0/1519710651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7a015d4e0 0x7fe7a009db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:45.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.068+0000 7fe7a5ac8640 1 --2- 172.21.15.76:0/1519710651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7a00d3b60 0x7fe7a01f6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:45.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.068+0000 7fe7a52c7640 1 -- 172.21.15.76:0/1519710651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7a00d3b60 msgr2=0x7fe7a01f6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:45.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.068+0000 7fe7a52c7640 1 --2- 172.21.15.76:0/1519710651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7a00d3b60 0x7fe7a01f6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.068+0000 7fe7a52c7640 1 -- 172.21.15.76:0/1519710651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7a015d4e0 msgr2=0x7fe7a009db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:45.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.068+0000 7fe7a52c7640 1 --2- 172.21.15.76:0/1519710651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7a015d4e0 0x7fe7a009db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.074 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.068+0000 7fe7a52c7640 1 -- 172.21.15.76:0/1519710651 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe79c009460 con 0x7fe7a015d100 2023-11-09T03:55:45.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.068+0000 7fe7a52c7640 1 --2- 172.21.15.76:0/1519710651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7a015d100 0x7fe7a014bca0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fe79000b410 tx=0x7fe79000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:45.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.068+0000 7fe7967fc640 1 -- 172.21.15.76:0/1519710651 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790011830 con 0x7fe7a015d100 2023-11-09T03:55:45.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.068+0000 7fe7967fc640 1 -- 172.21.15.76:0/1519710651 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7900119b0 con 0x7fe7a015d100 2023-11-09T03:55:45.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.068+0000 7fe7967fc640 1 -- 172.21.15.76:0/1519710651 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790011d60 con 0x7fe7a015d100 2023-11-09T03:55:45.075 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.068+0000 7fe7a7d2c640 1 -- 172.21.15.76:0/1519710651 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a00a05e0 con 0x7fe7a015d100 2023-11-09T03:55:45.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.069+0000 7fe7a7d2c640 1 -- 172.21.15.76:0/1519710651 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a00a0ba0 con 0x7fe7a015d100 2023-11-09T03:55:45.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.069+0000 7fe7a7d2c640 1 -- 172.21.15.76:0/1519710651 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a00a0d60 con 0x7fe7a015d100 2023-11-09T03:55:45.076 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.070+0000 7fe7967fc640 1 -- 172.21.15.76:0/1519710651 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe79001a400 con 0x7fe7a015d100 2023-11-09T03:55:45.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.070+0000 7fe7967fc640 1 --2- 172.21.15.76:0/1519710651 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe78406ae70 0x7fe78406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:45.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.070+0000 7fe7967fc640 1 -- 172.21.15.76:0/1519710651 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8554+0+0 (secure 0 0 0) 0x7fe79008a320 con 0x7fe7a015d100 2023-11-09T03:55:45.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.071+0000 7fe7a5ac8640 1 --2- 172.21.15.76:0/1519710651 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe78406ae70 0x7fe78406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:45.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.071+0000 7fe7a5ac8640 1 --2- 172.21.15.76:0/1519710651 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe78406ae70 0x7fe78406d330 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fe79c035000 tx=0x7fe79c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:45.077 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.075+0000 7fe7967fc640 1 -- 172.21.15.76:0/1519710651 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe7a00a0d60 con 0x7fe7a015d100 2023-11-09T03:55:45.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.240+0000 7fe7a7d2c640 1 -- 172.21.15.76:0/1519710651 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7fe7a00a0d60 con 0x7fe7a015d100 2023-11-09T03:55:45.240 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.240+0000 7fe7967fc640 1 -- 172.21.15.76:0/1519710651 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v120) v1 ==== 107+0+11 (secure 0 0 0) 0x7fe7a00a0d60 con 0x7fe7a015d100 2023-11-09T03:55:45.240 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:55:45.241 INFO:tasks.ceph.mds.c.smithi076.stderr:2023-11-09T03:55:45.242+0000 7f1a9c20f640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-09T03:55:45.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.242+0000 7fe7a7d2c640 1 -- 172.21.15.76:0/1519710651 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe78406ae70 msgr2=0x7fe78406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:45.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.242+0000 7fe7a7d2c640 1 --2- 172.21.15.76:0/1519710651 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe78406ae70 0x7fe78406d330 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fe79c035000 tx=0x7fe79c008030 comp rx=0 tx=0).stop 2023-11-09T03:55:45.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.242+0000 7fe7a7d2c640 1 -- 172.21.15.76:0/1519710651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7a015d100 msgr2=0x7fe7a014bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:45.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.242+0000 7fe7a7d2c640 1 --2- 172.21.15.76:0/1519710651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7a015d100 0x7fe7a014bca0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fe79000b410 tx=0x7fe79000b8d0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.242+0000 7fe7a7d2c640 1 -- 172.21.15.76:0/1519710651 shutdown_connections 2023-11-09T03:55:45.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.242+0000 7fe7a7d2c640 1 --2- 172.21.15.76:0/1519710651 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe78406ae70 0x7fe78406d330 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.242+0000 7fe7a7d2c640 1 --2- 172.21.15.76:0/1519710651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7a015d4e0 0x7fe7a009db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.242+0000 7fe7a7d2c640 1 --2- 172.21.15.76:0/1519710651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7a015d100 0x7fe7a014bca0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.242+0000 7fe7a7d2c640 1 --2- 172.21.15.76:0/1519710651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7a00d3b60 0x7fe7a01f6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.242+0000 7fe7a7d2c640 1 -- 172.21.15.76:0/1519710651 >> 172.21.15.76:0/1519710651 conn(0x7fe7a00d2ef0 msgr2=0x7fe7a00b15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:45.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.242+0000 7fe7a7d2c640 1 -- 172.21.15.76:0/1519710651 shutdown_connections 2023-11-09T03:55:45.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.242+0000 7fe7a7d2c640 1 -- 172.21.15.76:0/1519710651 wait complete. 2023-11-09T03:55:45.252 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:55:45.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.353+0000 7fab8791d640 1 Processor -- start 2023-11-09T03:55:45.354 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.354+0000 7fab8791d640 1 -- start start 2023-11-09T03:55:45.355 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.354+0000 7fab8791d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab8015d100 0x7fab800ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:45.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.354+0000 7fab8791d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fab800d1be0 0x7fab800b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:45.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.354+0000 7fab8791d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab800b32b0 con 0x7fab8015d4e0 2023-11-09T03:55:45.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.354+0000 7fab8791d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab800cee10 con 0x7fab800d1be0 2023-11-09T03:55:45.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.354+0000 7fab8791d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab800b2980 con 0x7fab8015d100 2023-11-09T03:55:45.356 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.355+0000 7fab85eba640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fab800d1be0 0x7fab800b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:45.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.355+0000 7fab85eba640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fab800d1be0 0x7fab800b3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49716/0 (socket says 172.21.15.76:49716) 2023-11-09T03:55:45.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.355+0000 7fab85eba640 1 -- 172.21.15.76:0/2063495534 learned_addr learned my addr 172.21.15.76:0/2063495534 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:45.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.355+0000 7fab856b9640 1 --2- 172.21.15.76:0/2063495534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab8015d100 0x7fab800ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:45.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.355+0000 7fab85eba640 1 -- 172.21.15.76:0/2063495534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab8015d100 msgr2=0x7fab800ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:45.357 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.355+0000 7fab85eba640 1 --2- 172.21.15.76:0/2063495534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab8015d100 0x7fab800ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.355+0000 7fab85eba640 1 -- 172.21.15.76:0/2063495534 >> v1:172.21.15.113:6789/0 conn(0x7fab8015d4e0 legacy=0x7fab800d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:45.358 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.355+0000 7fab85eba640 1 -- 172.21.15.76:0/2063495534 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab800b2c60 con 0x7fab800d1be0 2023-11-09T03:55:45.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.356+0000 7fab85eba640 1 --2- 172.21.15.76:0/2063495534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab800d1be0 0x7fab800b3740 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fab78009e80 tx=0x7fab7802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf5856d2c42bc42 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:45.360 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.356+0000 7fab77fff640 1 -- 172.21.15.76:0/2063495534 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7802caf0 con 0x7fab800d1be0 2023-11-09T03:55:45.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.356+0000 7fab77fff640 1 -- 172.21.15.76:0/2063495534 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7802cc50 con 0x7fab800d1be0 2023-11-09T03:55:45.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.356+0000 7fab77fff640 1 -- 172.21.15.76:0/2063495534 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab780343f0 con 0x7fab800d1be0 2023-11-09T03:55:45.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.356+0000 7fab8791d640 1 -- 172.21.15.76:0/2063495534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab800d1be0 msgr2=0x7fab800b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:45.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.356+0000 7fab8791d640 1 --2- 172.21.15.76:0/2063495534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab800d1be0 0x7fab800b3740 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fab78009e80 tx=0x7fab7802bf50 comp rx=0 tx=0).stop 2023-11-09T03:55:45.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.356+0000 7fab8791d640 1 -- 172.21.15.76:0/2063495534 shutdown_connections 2023-11-09T03:55:45.361 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.356+0000 7fab8791d640 1 --2- 172.21.15.76:0/2063495534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab800d1be0 0x7fab800b3740 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.356+0000 7fab8791d640 1 --2- 172.21.15.76:0/2063495534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab8015d100 0x7fab800ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.356+0000 7fab8791d640 1 -- 172.21.15.76:0/2063495534 >> 172.21.15.76:0/2063495534 conn(0x7fab800d2ef0 msgr2=0x7fab800d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:45.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.356+0000 7fab8791d640 1 -- 172.21.15.76:0/2063495534 shutdown_connections 2023-11-09T03:55:45.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.356+0000 7fab8791d640 1 -- 172.21.15.76:0/2063495534 wait complete. 2023-11-09T03:55:45.362 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.358+0000 7fab8791d640 1 Processor -- start 2023-11-09T03:55:45.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.358+0000 7fab8791d640 1 -- start start 2023-11-09T03:55:45.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.358+0000 7fab8791d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab800d1be0 0x7fab800cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:45.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.358+0000 7fab8791d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab8015d100 0x7fab8006b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:45.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.358+0000 7fab8791d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab8015d4e0 0x7fab8006be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:45.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.358+0000 7fab8791d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab800b7450 con 0x7fab800d1be0 2023-11-09T03:55:45.363 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.358+0000 7fab8791d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab800b7870 con 0x7fab8015d100 2023-11-09T03:55:45.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.358+0000 7fab8791d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab800cc2a0 con 0x7fab8015d4e0 2023-11-09T03:55:45.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.358+0000 7fab856b9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab800d1be0 0x7fab800cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:45.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.358+0000 7fab856b9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab800d1be0 0x7fab800cbd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49720/0 (socket says 172.21.15.76:49720) 2023-11-09T03:55:45.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.358+0000 7fab856b9640 1 -- 172.21.15.76:0/3182575725 learned_addr learned my addr 172.21.15.76:0/3182575725 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:45.364 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.359+0000 7fab856b9640 1 -- 172.21.15.76:0/3182575725 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab8015d4e0 msgr2=0x7fab8006be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:45.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.359+0000 7fab856b9640 1 --2- 172.21.15.76:0/3182575725 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab8015d4e0 0x7fab8006be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.359+0000 7fab856b9640 1 -- 172.21.15.76:0/3182575725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab8015d100 msgr2=0x7fab8006b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:45.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.359+0000 7fab856b9640 1 --2- 172.21.15.76:0/3182575725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab8015d100 0x7fab8006b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.359+0000 7fab856b9640 1 -- 172.21.15.76:0/3182575725 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab78009b30 con 0x7fab800d1be0 2023-11-09T03:55:45.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.359+0000 7fab856b9640 1 --2- 172.21.15.76:0/3182575725 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab800d1be0 0x7fab800cbd60 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fab7000d800 tx=0x7fab7000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:45.365 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.359+0000 7fab75ffb640 1 -- 172.21.15.76:0/3182575725 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7000b6c0 con 0x7fab800d1be0 2023-11-09T03:55:45.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.359+0000 7fab75ffb640 1 -- 172.21.15.76:0/3182575725 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7000b820 con 0x7fab800d1be0 2023-11-09T03:55:45.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.359+0000 7fab8791d640 1 -- 172.21.15.76:0/3182575725 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab800b67d0 con 0x7fab800d1be0 2023-11-09T03:55:45.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.359+0000 7fab75ffb640 1 -- 172.21.15.76:0/3182575725 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7000bbd0 con 0x7fab800d1be0 2023-11-09T03:55:45.366 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.359+0000 7fab8791d640 1 -- 172.21.15.76:0/3182575725 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab800b6d90 con 0x7fab800d1be0 2023-11-09T03:55:45.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.360+0000 7fab8791d640 1 -- 172.21.15.76:0/3182575725 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab800684b0 con 0x7fab800d1be0 2023-11-09T03:55:45.367 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.360+0000 7fab75ffb640 1 -- 172.21.15.76:0/3182575725 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fab7000bd30 con 0x7fab800d1be0 2023-11-09T03:55:45.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.361+0000 7fab75ffb640 1 --2- 172.21.15.76:0/3182575725 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fab6406ae70 0x7fab6406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:45.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.361+0000 7fab75ffb640 1 -- 172.21.15.76:0/3182575725 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8554+0+0 (secure 0 0 0) 0x7fab7008a650 con 0x7fab800d1be0 2023-11-09T03:55:45.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.361+0000 7fab84eb8640 1 --2- 172.21.15.76:0/3182575725 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fab6406ae70 0x7fab6406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:45.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.361+0000 7fab84eb8640 1 --2- 172.21.15.76:0/3182575725 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fab6406ae70 0x7fab6406d330 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fab7c000f60 tx=0x7fab7c0079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:45.368 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.365+0000 7fab75ffb640 1 -- 172.21.15.76:0/3182575725 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fab70054c20 con 0x7fab800d1be0 2023-11-09T03:55:45.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.537+0000 7fab8791d640 1 -- 172.21.15.76:0/3182575725 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fab800b4f80 con 0x7fab800d1be0 2023-11-09T03:55:45.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.537+0000 7fab75ffb640 1 -- 172.21.15.76:0/3182575725 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v120) v1 ==== 104+0+11 (secure 0 0 0) 0x7fab70058240 con 0x7fab800d1be0 2023-11-09T03:55:45.538 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:55:45.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.539+0000 7fab8791d640 1 -- 172.21.15.76:0/3182575725 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fab6406ae70 msgr2=0x7fab6406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:45.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.539+0000 7fab8791d640 1 --2- 172.21.15.76:0/3182575725 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fab6406ae70 0x7fab6406d330 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fab7c000f60 tx=0x7fab7c0079b0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.539+0000 7fab8791d640 1 -- 172.21.15.76:0/3182575725 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab800d1be0 msgr2=0x7fab800cbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:45.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.539+0000 7fab8791d640 1 --2- 172.21.15.76:0/3182575725 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab800d1be0 0x7fab800cbd60 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fab7000d800 tx=0x7fab7000dcc0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.539+0000 7fab8791d640 1 -- 172.21.15.76:0/3182575725 shutdown_connections 2023-11-09T03:55:45.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.539+0000 7fab8791d640 1 --2- 172.21.15.76:0/3182575725 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fab6406ae70 0x7fab6406d330 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.541 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.539+0000 7fab8791d640 1 --2- 172.21.15.76:0/3182575725 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab8015d4e0 0x7fab8006be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.539+0000 7fab8791d640 1 --2- 172.21.15.76:0/3182575725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab8015d100 0x7fab8006b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.539+0000 7fab8791d640 1 --2- 172.21.15.76:0/3182575725 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab800d1be0 0x7fab800cbd60 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.539+0000 7fab8791d640 1 -- 172.21.15.76:0/3182575725 >> 172.21.15.76:0/3182575725 conn(0x7fab800d2ef0 msgr2=0x7fab800d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:45.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.539+0000 7fab8791d640 1 -- 172.21.15.76:0/3182575725 shutdown_connections 2023-11-09T03:55:45.542 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.539+0000 7fab8791d640 1 -- 172.21.15.76:0/3182575725 wait complete. 2023-11-09T03:55:45.550 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:55:45.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.652+0000 7f5ea88af640 1 Processor -- start 2023-11-09T03:55:45.653 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.652+0000 7f5ea88af640 1 -- start start 2023-11-09T03:55:45.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.653+0000 7f5ea88af640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5ea40ade00 0x7f5ea40b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:45.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.653+0000 7f5ea88af640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea406cc90 con 0x7f5ea406d9c0 2023-11-09T03:55:45.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.653+0000 7f5ea88af640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea40b0e60 con 0x7f5ea406c5f0 2023-11-09T03:55:45.654 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.653+0000 7f5ea88af640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea406cee0 con 0x7f5ea40ade00 2023-11-09T03:55:45.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.653+0000 7f5ea259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5ea40ade00 0x7f5ea40b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:45.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.653+0000 7f5ea259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5ea40ade00 0x7f5ea40b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49726/0 (socket says 172.21.15.76:49726) 2023-11-09T03:55:45.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.653+0000 7f5ea1d9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f5ea406d9c0 0x7f5ea40710d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37696/0 (socket says 172.21.15.76:37696) 2023-11-09T03:55:45.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.653+0000 7f5ea259c640 1 -- 172.21.15.76:0/610690695 learned_addr learned my addr 172.21.15.76:0/610690695 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:45.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.653+0000 7f5ea259c640 1 -- 172.21.15.76:0/610690695 >> v1:172.21.15.113:6789/0 conn(0x7f5ea406c5f0 legacy=0x7f5ea40cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:45.655 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.653+0000 7f5ea259c640 1 -- 172.21.15.76:0/610690695 >> v1:172.21.15.76:6789/0 conn(0x7f5ea406d9c0 legacy=0x7f5ea40710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:45.656 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.653+0000 7f5ea259c640 1 -- 172.21.15.76:0/610690695 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea406d1c0 con 0x7f5ea40ade00 2023-11-09T03:55:45.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.654+0000 7f5ea259c640 1 --2- 172.21.15.76:0/610690695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea40ade00 0x7f5ea40b0530 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f5e94009870 tx=0x7f5e9402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159a272da09bf376 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:45.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.654+0000 7f5ea159a640 1 -- 172.21.15.76:0/610690695 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e940317e0 con 0x7f5ea40ade00 2023-11-09T03:55:45.657 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.654+0000 7f5ea159a640 1 -- 172.21.15.76:0/610690695 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e94031940 con 0x7f5ea40ade00 2023-11-09T03:55:45.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.654+0000 7f5ea159a640 1 -- 172.21.15.76:0/610690695 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e94031cd0 con 0x7f5ea40ade00 2023-11-09T03:55:45.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.655+0000 7f5ea88af640 1 -- 172.21.15.76:0/610690695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea40ade00 msgr2=0x7f5ea40b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:45.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.655+0000 7f5ea88af640 1 --2- 172.21.15.76:0/610690695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea40ade00 0x7f5ea40b0530 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f5e94009870 tx=0x7f5e9402b880 comp rx=0 tx=0).stop 2023-11-09T03:55:45.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.655+0000 7f5ea88af640 1 -- 172.21.15.76:0/610690695 shutdown_connections 2023-11-09T03:55:45.658 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.655+0000 7f5ea88af640 1 --2- 172.21.15.76:0/610690695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea40ade00 0x7f5ea40b0530 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.655+0000 7f5ea88af640 1 -- 172.21.15.76:0/610690695 >> 172.21.15.76:0/610690695 conn(0x7f5ea40cb530 msgr2=0x7f5ea40cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:45.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.655+0000 7f5ea88af640 1 -- 172.21.15.76:0/610690695 shutdown_connections 2023-11-09T03:55:45.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.655+0000 7f5ea88af640 1 -- 172.21.15.76:0/610690695 wait complete. 2023-11-09T03:55:45.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.656+0000 7f5ea88af640 1 Processor -- start 2023-11-09T03:55:45.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.656+0000 7f5ea88af640 1 -- start start 2023-11-09T03:55:45.661 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.657+0000 7f5ea88af640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea406c5f0 0x7f5ea4158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:45.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.657+0000 7f5ea88af640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ea406d9c0 0x7f5ea4145700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:45.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.657+0000 7f5ea88af640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ea40ade00 0x7f5ea41f2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:45.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.657+0000 7f5ea88af640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea41f3c90 con 0x7f5ea406c5f0 2023-11-09T03:55:45.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.657+0000 7f5ea88af640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea4145270 con 0x7f5ea40ade00 2023-11-09T03:55:45.662 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.657+0000 7f5ea88af640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea41f36d0 con 0x7f5ea406d9c0 2023-11-09T03:55:45.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.657+0000 7f5ea259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea406c5f0 0x7f5ea4158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:45.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.657+0000 7f5ea259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea406c5f0 0x7f5ea4158a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49742/0 (socket says 172.21.15.76:49742) 2023-11-09T03:55:45.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.657+0000 7f5ea259c640 1 -- 172.21.15.76:0/49185838 learned_addr learned my addr 172.21.15.76:0/49185838 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:45.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.657+0000 7f5ea259c640 1 -- 172.21.15.76:0/49185838 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ea406d9c0 msgr2=0x7f5ea4145700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:45.663 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.657+0000 7f5ea259c640 1 --2- 172.21.15.76:0/49185838 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ea406d9c0 0x7f5ea4145700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.657+0000 7f5ea259c640 1 -- 172.21.15.76:0/49185838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ea40ade00 msgr2=0x7f5ea41f2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:45.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.657+0000 7f5ea259c640 1 --2- 172.21.15.76:0/49185838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ea40ade00 0x7f5ea41f2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.657+0000 7f5ea259c640 1 -- 172.21.15.76:0/49185838 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e940094d0 con 0x7f5ea406c5f0 2023-11-09T03:55:45.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.658+0000 7f5ea259c640 1 --2- 172.21.15.76:0/49185838 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea406c5f0 0x7f5ea4158a10 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f5e94000c00 tx=0x7f5e940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:45.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.658+0000 7f5e92ffd640 1 -- 172.21.15.76:0/49185838 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e940329c0 con 0x7f5ea406c5f0 2023-11-09T03:55:45.664 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.658+0000 7f5e92ffd640 1 -- 172.21.15.76:0/49185838 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e94032b40 con 0x7f5ea406c5f0 2023-11-09T03:55:45.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.658+0000 7f5ea88af640 1 -- 172.21.15.76:0/49185838 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea41f39b0 con 0x7f5ea406c5f0 2023-11-09T03:55:45.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.658+0000 7f5e92ffd640 1 -- 172.21.15.76:0/49185838 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e940453f0 con 0x7f5ea406c5f0 2023-11-09T03:55:45.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.658+0000 7f5ea88af640 1 -- 172.21.15.76:0/49185838 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea4145c40 con 0x7f5ea406c5f0 2023-11-09T03:55:45.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.659+0000 7f5ea88af640 1 -- 172.21.15.76:0/49185838 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea4144d60 con 0x7f5ea406c5f0 2023-11-09T03:55:45.665 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.659+0000 7f5e92ffd640 1 -- 172.21.15.76:0/49185838 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5e9404d3f0 con 0x7f5ea406c5f0 2023-11-09T03:55:45.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.660+0000 7f5e92ffd640 1 --2- 172.21.15.76:0/49185838 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5e8406ae70 0x7f5e8406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:45.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.660+0000 7f5e92ffd640 1 -- 172.21.15.76:0/49185838 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8554+0+0 (secure 0 0 0) 0x7f5e940ac090 con 0x7f5ea406c5f0 2023-11-09T03:55:45.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.660+0000 7f5ea1d9b640 1 --2- 172.21.15.76:0/49185838 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5e8406ae70 0x7f5e8406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:45.666 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.660+0000 7f5ea1d9b640 1 --2- 172.21.15.76:0/49185838 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5e8406ae70 0x7f5e8406d330 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4154700 tx=0x7f5e980079c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:45.667 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.664+0000 7f5e92ffd640 1 -- 172.21.15.76:0/49185838 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ea4144d60 con 0x7f5ea406c5f0 2023-11-09T03:55:45.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.833+0000 7f5ea88af640 1 -- 172.21.15.76:0/49185838 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5ea4144d60 con 0x7f5ea406c5f0 2023-11-09T03:55:45.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.833+0000 7f5e92ffd640 1 -- 172.21.15.76:0/49185838 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v120) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5ea4144d60 con 0x7f5ea406c5f0 2023-11-09T03:55:45.834 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:55:45.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.835+0000 7f5ea88af640 1 -- 172.21.15.76:0/49185838 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5e8406ae70 msgr2=0x7f5e8406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:45.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.835+0000 7f5ea88af640 1 --2- 172.21.15.76:0/49185838 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5e8406ae70 0x7f5e8406d330 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4154700 tx=0x7f5e980079c0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.835+0000 7f5ea88af640 1 -- 172.21.15.76:0/49185838 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea406c5f0 msgr2=0x7f5ea4158a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:45.836 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.835+0000 7f5ea88af640 1 --2- 172.21.15.76:0/49185838 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea406c5f0 0x7f5ea4158a10 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f5e94000c00 tx=0x7f5e940046c0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.835+0000 7f5ea88af640 1 -- 172.21.15.76:0/49185838 shutdown_connections 2023-11-09T03:55:45.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.835+0000 7f5ea88af640 1 --2- 172.21.15.76:0/49185838 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5e8406ae70 0x7f5e8406d330 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.835+0000 7f5ea88af640 1 --2- 172.21.15.76:0/49185838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ea40ade00 0x7f5ea41f2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.835+0000 7f5ea88af640 1 --2- 172.21.15.76:0/49185838 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ea406d9c0 0x7f5ea4145700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.835+0000 7f5ea88af640 1 --2- 172.21.15.76:0/49185838 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea406c5f0 0x7f5ea4158a10 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.835+0000 7f5ea88af640 1 -- 172.21.15.76:0/49185838 >> 172.21.15.76:0/49185838 conn(0x7f5ea40cb530 msgr2=0x7f5ea4071090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:45.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.835+0000 7f5ea88af640 1 -- 172.21.15.76:0/49185838 shutdown_connections 2023-11-09T03:55:45.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.835+0000 7f5ea88af640 1 -- 172.21.15.76:0/49185838 wait complete. 2023-11-09T03:55:45.846 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-09T03:55:45.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.945+0000 7f92cf887640 1 Processor -- start 2023-11-09T03:55:45.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.946+0000 7f92cf887640 1 -- start start 2023-11-09T03:55:45.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.946+0000 7f92cf887640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f92c815d4e0 0x7f92c80d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:45.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.946+0000 7f92cf887640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92c80b32b0 con 0x7f92c80d3b60 2023-11-09T03:55:45.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.946+0000 7f92cf887640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92c80cee10 con 0x7f92c815d100 2023-11-09T03:55:45.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.946+0000 7f92cf887640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c80cec20 con 0x7f92c815d4e0 2023-11-09T03:55:45.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.946+0000 7f92cce22640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f92c815d4e0 0x7f92c80d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:45.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.946+0000 7f92cde24640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f92c80d3b60 0x7f92c80b3740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37704/0 (socket says 172.21.15.76:37704) 2023-11-09T03:55:45.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.947+0000 7f92cde24640 1 -- 172.21.15.76:0/1387880604 learned_addr learned my addr 172.21.15.76:0/1387880604 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:55:45.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.947+0000 7f92cce22640 1 -- 172.21.15.76:0/1387880604 >> v1:172.21.15.113:6789/0 conn(0x7f92c815d100 legacy=0x7f92c80ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:45.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.947+0000 7f92cce22640 1 -- 172.21.15.76:0/1387880604 >> v1:172.21.15.76:6789/0 conn(0x7f92c80d3b60 legacy=0x7f92c80b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:45.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.947+0000 7f92cce22640 1 -- 172.21.15.76:0/1387880604 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c80b2980 con 0x7f92c815d4e0 2023-11-09T03:55:45.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.947+0000 7f92bffff640 1 -- 172.21.15.76:0/1387880604 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 550526483 0 0) 0x7f92c80b32b0 con 0x7f92c80d3b60 2023-11-09T03:55:45.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.948+0000 7f92cce22640 1 --2- 172.21.15.76:0/1387880604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92c815d4e0 0x7f92c80d14d0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f92b80097a0 tx=0x7f92b802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109971465152248f server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:45.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.948+0000 7f92bffff640 1 -- 172.21.15.76:0/1387880604 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b8030610 con 0x7f92c815d4e0 2023-11-09T03:55:45.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.948+0000 7f92bffff640 1 -- 172.21.15.76:0/1387880604 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b8030770 con 0x7f92c815d4e0 2023-11-09T03:55:45.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.948+0000 7f92bffff640 1 -- 172.21.15.76:0/1387880604 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b8030b00 con 0x7f92c815d4e0 2023-11-09T03:55:45.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.948+0000 7f92cf887640 1 -- 172.21.15.76:0/1387880604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92c815d4e0 msgr2=0x7f92c80d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:45.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.948+0000 7f92cf887640 1 --2- 172.21.15.76:0/1387880604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92c815d4e0 0x7f92c80d14d0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f92b80097a0 tx=0x7f92b802b960 comp rx=0 tx=0).stop 2023-11-09T03:55:45.953 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.948+0000 7f92cf887640 1 -- 172.21.15.76:0/1387880604 shutdown_connections 2023-11-09T03:55:45.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.948+0000 7f92cf887640 1 --2- 172.21.15.76:0/1387880604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92c815d4e0 0x7f92c80d14d0 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.948+0000 7f92cf887640 1 -- 172.21.15.76:0/1387880604 >> 172.21.15.76:0/1387880604 conn(0x7f92c80d2ef0 msgr2=0x7f92c80d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:45.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.949+0000 7f92cf887640 1 -- 172.21.15.76:0/1387880604 shutdown_connections 2023-11-09T03:55:45.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.949+0000 7f92cf887640 1 -- 172.21.15.76:0/1387880604 wait complete. 2023-11-09T03:55:45.954 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.950+0000 7f92cf887640 1 Processor -- start 2023-11-09T03:55:45.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.950+0000 7f92cf887640 1 -- start start 2023-11-09T03:55:45.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.950+0000 7f92cf887640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92c80d3b60 0x7f92c8151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:45.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.950+0000 7f92cf887640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92c815d100 0x7f92c815fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:45.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.950+0000 7f92cf887640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92c815d4e0 0x7f92c8160200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:45.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.950+0000 7f92cf887640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c815f380 con 0x7f92c80d3b60 2023-11-09T03:55:45.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.950+0000 7f92cf887640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c815e770 con 0x7f92c815d4e0 2023-11-09T03:55:45.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.950+0000 7f92cf887640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c815ea50 con 0x7f92c815d100 2023-11-09T03:55:45.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.950+0000 7f92cd623640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92c80d3b60 0x7f92c8151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:45.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.951+0000 7f92cd623640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92c80d3b60 0x7f92c8151a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49760/0 (socket says 172.21.15.76:49760) 2023-11-09T03:55:45.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.951+0000 7f92cd623640 1 -- 172.21.15.76:0/1474980587 learned_addr learned my addr 172.21.15.76:0/1474980587 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:45.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.951+0000 7f92cde24640 1 --2- 172.21.15.76:0/1474980587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92c815d4e0 0x7f92c8160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:45.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.951+0000 7f92cce22640 1 --2- 172.21.15.76:0/1474980587 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92c815d100 0x7f92c815fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:45.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.951+0000 7f92cd623640 1 -- 172.21.15.76:0/1474980587 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92c815d100 msgr2=0x7f92c815fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:45.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.951+0000 7f92cd623640 1 --2- 172.21.15.76:0/1474980587 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92c815d100 0x7f92c815fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.951+0000 7f92cd623640 1 -- 172.21.15.76:0/1474980587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92c815d4e0 msgr2=0x7f92c8160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:45.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.951+0000 7f92cd623640 1 --2- 172.21.15.76:0/1474980587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92c815d4e0 0x7f92c8160200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:45.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.951+0000 7f92cd623640 1 -- 172.21.15.76:0/1474980587 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b8009400 con 0x7f92c80d3b60 2023-11-09T03:55:45.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.951+0000 7f92cd623640 1 --2- 172.21.15.76:0/1474980587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92c80d3b60 0x7f92c8151a90 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f92c400c450 tx=0x7f92c400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:45.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.951+0000 7f92bdffb640 1 -- 172.21.15.76:0/1474980587 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c40052d0 con 0x7f92c80d3b60 2023-11-09T03:55:45.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.951+0000 7f92bdffb640 1 -- 172.21.15.76:0/1474980587 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c4005450 con 0x7f92c80d3b60 2023-11-09T03:55:45.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.951+0000 7f92cf887640 1 -- 172.21.15.76:0/1474980587 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c815ed30 con 0x7f92c80d3b60 2023-11-09T03:55:45.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.951+0000 7f92bdffb640 1 -- 172.21.15.76:0/1474980587 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c4003880 con 0x7f92c80d3b60 2023-11-09T03:55:45.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.951+0000 7f92cf887640 1 -- 172.21.15.76:0/1474980587 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c8151fd0 con 0x7f92c80d3b60 2023-11-09T03:55:45.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.952+0000 7f92cf887640 1 -- 172.21.15.76:0/1474980587 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c815f4f0 con 0x7f92c80d3b60 2023-11-09T03:55:45.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.953+0000 7f92bdffb640 1 -- 172.21.15.76:0/1474980587 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f92c40039e0 con 0x7f92c80d3b60 2023-11-09T03:55:45.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.953+0000 7f92bdffb640 1 --2- 172.21.15.76:0/1474980587 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f92ac06ae70 0x7f92ac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:45.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.953+0000 7f92bdffb640 1 -- 172.21.15.76:0/1474980587 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8554+0+0 (secure 0 0 0) 0x7f92c4089c60 con 0x7f92c80d3b60 2023-11-09T03:55:45.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.953+0000 7f92cce22640 1 --2- 172.21.15.76:0/1474980587 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f92ac06ae70 0x7f92ac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:45.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.954+0000 7f92cce22640 1 --2- 172.21.15.76:0/1474980587 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f92ac06ae70 0x7f92ac06d330 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f92b8036000 tx=0x7f92b8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:45.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:45.957+0000 7f92bdffb640 1 -- 172.21.15.76:0/1474980587 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f92c40541c0 con 0x7f92c80d3b60 2023-11-09T03:55:46.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.136+0000 7f92cf887640 1 -- 172.21.15.76:0/1474980587 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f92c80684b0 con 0x7f92c80d3b60 2023-11-09T03:55:46.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.136+0000 7f92bdffb640 1 -- 172.21.15.76:0/1474980587 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 32 entries v120) v1 ==== 81+0+1923 (secure 0 0 0) 0x7f92c40577e0 con 0x7f92c80d3b60 2023-11-09T03:55:46.137 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/497547142 2023-11-09T04:55:40.451393+0000 2023-11-09T03:55:46.137 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/757529522 2023-11-10T03:55:31.678438+0000 2023-11-09T03:55:46.138 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6832/3697850101 2023-11-10T03:51:39.442929+0000 2023-11-09T03:55:46.138 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/3449910842 2023-11-10T03:50:59.611256+0000 2023-11-09T03:55:46.138 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6832/1536527256 2023-11-10T03:52:34.213804+0000 2023-11-09T03:55:46.138 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/3229239133 2023-11-10T03:51:32.624856+0000 2023-11-09T03:55:46.138 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/445306598 2023-11-10T03:52:27.406838+0000 2023-11-09T03:55:46.139 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/1663988060 2023-11-10T03:50:33.665092+0000 2023-11-09T03:55:46.139 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/1663988060 2023-11-10T03:50:33.665092+0000 2023-11-09T03:55:46.139 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/3280098776 2023-11-10T03:50:07.616780+0000 2023-11-09T03:55:46.139 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/2519773110 2023-11-09T04:54:14.823765+0000 2023-11-09T03:55:46.140 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/4011275120 2023-11-10T03:49:42.029057+0000 2023-11-09T03:55:46.140 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/757529522 2023-11-10T03:55:31.678438+0000 2023-11-09T03:55:46.140 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/4011275120 2023-11-10T03:49:42.029057+0000 2023-11-09T03:55:46.140 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/3462040782 2023-11-10T03:54:45.345437+0000 2023-11-09T03:55:46.140 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/503424885 2023-11-09T04:51:47.727422+0000 2023-11-09T03:55:46.141 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/3229239133 2023-11-10T03:51:32.624856+0000 2023-11-09T03:55:46.141 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/445306598 2023-11-10T03:52:27.406838+0000 2023-11-09T03:55:46.141 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/3280098776 2023-11-10T03:50:07.616780+0000 2023-11-09T03:55:46.141 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/3087804343 2023-11-09T04:52:42.513556+0000 2023-11-09T03:55:46.141 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/546338722 2023-11-10T03:53:17.470106+0000 2023-11-09T03:55:46.142 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6833/1536527256 2023-11-10T03:52:34.213804+0000 2023-11-09T03:55:46.142 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/546338722 2023-11-10T03:53:17.470106+0000 2023-11-09T03:55:46.142 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/7360844 2023-11-10T03:53:24.718008+0000 2023-11-09T03:55:46.142 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/7360844 2023-11-10T03:53:24.718008+0000 2023-11-09T03:55:46.143 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/2075292158 2023-11-10T03:54:06.568191+0000 2023-11-09T03:55:46.143 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/334503200 2023-11-09T04:53:32.572840+0000 2023-11-09T03:55:46.143 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/2075292158 2023-11-10T03:54:06.568191+0000 2023-11-09T03:55:46.143 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6833/3697850101 2023-11-10T03:51:39.442929+0000 2023-11-09T03:55:46.143 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/3462040782 2023-11-10T03:54:45.345437+0000 2023-11-09T03:55:46.144 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/3449910842 2023-11-10T03:50:59.611256+0000 2023-11-09T03:55:46.144 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/3870807407 2023-11-09T04:54:56.671385+0000 2023-11-09T03:55:46.145 INFO:teuthology.orchestra.run.smithi076.stderr:listed 32 entries 2023-11-09T03:55:46.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.138+0000 7f92cf887640 1 -- 172.21.15.76:0/1474980587 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f92ac06ae70 msgr2=0x7f92ac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:46.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.138+0000 7f92cf887640 1 --2- 172.21.15.76:0/1474980587 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f92ac06ae70 0x7f92ac06d330 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f92b8036000 tx=0x7f92b8008030 comp rx=0 tx=0).stop 2023-11-09T03:55:46.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.138+0000 7f92cf887640 1 -- 172.21.15.76:0/1474980587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92c80d3b60 msgr2=0x7f92c8151a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:46.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.138+0000 7f92cf887640 1 --2- 172.21.15.76:0/1474980587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92c80d3b60 0x7f92c8151a90 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f92c400c450 tx=0x7f92c400c910 comp rx=0 tx=0).stop 2023-11-09T03:55:46.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.138+0000 7f92cf887640 1 -- 172.21.15.76:0/1474980587 shutdown_connections 2023-11-09T03:55:46.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.138+0000 7f92cf887640 1 --2- 172.21.15.76:0/1474980587 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f92ac06ae70 0x7f92ac06d330 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:46.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.138+0000 7f92cf887640 1 --2- 172.21.15.76:0/1474980587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92c815d4e0 0x7f92c8160200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:46.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.138+0000 7f92cf887640 1 --2- 172.21.15.76:0/1474980587 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92c815d100 0x7f92c815fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:46.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.138+0000 7f92cf887640 1 --2- 172.21.15.76:0/1474980587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92c80d3b60 0x7f92c8151a90 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:46.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.138+0000 7f92cf887640 1 -- 172.21.15.76:0/1474980587 >> 172.21.15.76:0/1474980587 conn(0x7f92c80d2ef0 msgr2=0x7f92c80ce4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:46.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.138+0000 7f92cf887640 1 -- 172.21.15.76:0/1474980587 shutdown_connections 2023-11-09T03:55:46.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.138+0000 7f92cf887640 1 -- 172.21.15.76:0/1474980587 wait complete. 2023-11-09T03:55:46.149 DEBUG:tasks.cephfs.mount:Force/lazy unmounting on client.0 2023-11-09T03:55:46.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:55:46.150 DEBUG:teuthology.orchestra.run.smithi076:> sudo umount --lazy --force /home/ubuntu/cephtest/mnt.0 2023-11-09T03:55:46.176 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[131924]: fuse finished with error 0 and tester_r 0 2023-11-09T03:55:46.177 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T03:55:46.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:55:46.178 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-09T03:55:46.232 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:55:46.233 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:55:46.287 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056381,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14761 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248868k,nr_inodes=812217,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-09T03:55:46.288 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-09T03:55:46.401 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:55:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:55:46.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.428+0000 7f95d1ed4640 1 Processor -- start 2023-11-09T03:55:46.429 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.428+0000 7f95d1ed4640 1 -- start start 2023-11-09T03:55:46.432 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.429+0000 7f95d1ed4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95cc06d9c0 0x7f95cc0710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:46.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.429+0000 7f95d1ed4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f95cc06c5f0 0x7f95cc071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:46.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.429+0000 7f95d1ed4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95cc06cc90 con 0x7f95cc0ade00 2023-11-09T03:55:46.433 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.429+0000 7f95d1ed4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95cc0b0c40 con 0x7f95cc06c5f0 2023-11-09T03:55:46.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.429+0000 7f95d1ed4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc071b50 con 0x7f95cc06d9c0 2023-11-09T03:55:46.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.429+0000 7f95cbfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f95cc06c5f0 0x7f95cc071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:46.434 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.429+0000 7f95cbfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f95cc06c5f0 0x7f95cc071610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49762/0 (socket says 172.21.15.76:49762) 2023-11-09T03:55:46.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.429+0000 7f95cbfff640 1 -- 172.21.15.76:0/4220808762 learned_addr learned my addr 172.21.15.76:0/4220808762 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:46.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.429+0000 7f95caffd640 1 --2- 172.21.15.76:0/4220808762 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95cc06d9c0 0x7f95cc0710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:46.435 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.429+0000 7f95cbfff640 1 -- 172.21.15.76:0/4220808762 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95cc06d9c0 msgr2=0x7f95cc0710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:46.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.429+0000 7f95cbfff640 1 --2- 172.21.15.76:0/4220808762 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95cc06d9c0 0x7f95cc0710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:46.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.429+0000 7f95cbfff640 1 -- 172.21.15.76:0/4220808762 >> v1:172.21.15.113:6789/0 conn(0x7f95cc0ade00 legacy=0x7f95cc0b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:46.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.429+0000 7f95cbfff640 1 -- 172.21.15.76:0/4220808762 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95cc0b0e60 con 0x7f95cc06c5f0 2023-11-09T03:55:46.436 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.430+0000 7f95cbfff640 1 --2- 172.21.15.76:0/4220808762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95cc06c5f0 0x7f95cc071610 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f95c4009e50 tx=0x7f95c402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec44dcda88ef6b26 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:46.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.430+0000 7f95ca7fc640 1 -- 172.21.15.76:0/4220808762 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c402c9e0 con 0x7f95cc06c5f0 2023-11-09T03:55:46.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.430+0000 7f95ca7fc640 1 -- 172.21.15.76:0/4220808762 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c402cb40 con 0x7f95cc06c5f0 2023-11-09T03:55:46.437 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.430+0000 7f95ca7fc640 1 -- 172.21.15.76:0/4220808762 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c40343f0 con 0x7f95cc06c5f0 2023-11-09T03:55:46.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.431+0000 7f95d1ed4640 1 -- 172.21.15.76:0/4220808762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95cc06c5f0 msgr2=0x7f95cc071610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:46.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.431+0000 7f95d1ed4640 1 --2- 172.21.15.76:0/4220808762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95cc06c5f0 0x7f95cc071610 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f95c4009e50 tx=0x7f95c402be40 comp rx=0 tx=0).stop 2023-11-09T03:55:46.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.431+0000 7f95d1ed4640 1 -- 172.21.15.76:0/4220808762 shutdown_connections 2023-11-09T03:55:46.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.431+0000 7f95d1ed4640 1 --2- 172.21.15.76:0/4220808762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95cc06c5f0 0x7f95cc071610 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:46.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.431+0000 7f95d1ed4640 1 --2- 172.21.15.76:0/4220808762 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95cc06d9c0 0x7f95cc0710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:46.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.431+0000 7f95d1ed4640 1 -- 172.21.15.76:0/4220808762 >> 172.21.15.76:0/4220808762 conn(0x7f95cc0cb530 msgr2=0x7f95cc0cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:46.439 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.431+0000 7f95d1ed4640 1 -- 172.21.15.76:0/4220808762 shutdown_connections 2023-11-09T03:55:46.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.431+0000 7f95d1ed4640 1 -- 172.21.15.76:0/4220808762 wait complete. 2023-11-09T03:55:46.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.432+0000 7f95d1ed4640 1 Processor -- start 2023-11-09T03:55:46.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.432+0000 7f95d1ed4640 1 -- start start 2023-11-09T03:55:46.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.433+0000 7f95d1ed4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95cc06c5f0 0x7f95cc157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:46.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.433+0000 7f95d1ed4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95cc06d9c0 0x7f95cc14ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:46.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.433+0000 7f95d1ed4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95cc0ade00 0x7f95cc0c72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:46.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.433+0000 7f95d1ed4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc1578b0 con 0x7f95cc06c5f0 2023-11-09T03:55:46.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.433+0000 7f95d1ed4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc157680 con 0x7f95cc06d9c0 2023-11-09T03:55:46.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.433+0000 7f95d1ed4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc14b3c0 con 0x7f95cc0ade00 2023-11-09T03:55:46.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.433+0000 7f95cb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95cc06c5f0 0x7f95cc157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:46.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.433+0000 7f95cb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95cc06c5f0 0x7f95cc157140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49778/0 (socket says 172.21.15.76:49778) 2023-11-09T03:55:46.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.433+0000 7f95cb7fe640 1 -- 172.21.15.76:0/526975447 learned_addr learned my addr 172.21.15.76:0/526975447 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:46.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.433+0000 7f95caffd640 1 --2- 172.21.15.76:0/526975447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95cc06d9c0 0x7f95cc14ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:46.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.433+0000 7f95cbfff640 1 --2- 172.21.15.76:0/526975447 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95cc0ade00 0x7f95cc0c72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:46.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.433+0000 7f95cb7fe640 1 -- 172.21.15.76:0/526975447 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95cc0ade00 msgr2=0x7f95cc0c72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:46.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.433+0000 7f95cb7fe640 1 --2- 172.21.15.76:0/526975447 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95cc0ade00 0x7f95cc0c72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:46.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.433+0000 7f95cb7fe640 1 -- 172.21.15.76:0/526975447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95cc06d9c0 msgr2=0x7f95cc14ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:46.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.433+0000 7f95cb7fe640 1 --2- 172.21.15.76:0/526975447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95cc06d9c0 0x7f95cc14ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:46.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.433+0000 7f95cb7fe640 1 -- 172.21.15.76:0/526975447 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c4009b30 con 0x7f95cc06c5f0 2023-11-09T03:55:46.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.433+0000 7f95cb7fe640 1 --2- 172.21.15.76:0/526975447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95cc06c5f0 0x7f95cc157140 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00c450 tx=0x7f95bc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:46.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.433+0000 7f95b3fff640 1 -- 172.21.15.76:0/526975447 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc0052d0 con 0x7f95cc06c5f0 2023-11-09T03:55:46.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.433+0000 7f95b3fff640 1 -- 172.21.15.76:0/526975447 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95bc005430 con 0x7f95cc06c5f0 2023-11-09T03:55:46.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.433+0000 7f95b3fff640 1 -- 172.21.15.76:0/526975447 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc012870 con 0x7f95cc06c5f0 2023-11-09T03:55:46.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.433+0000 7f95d1ed4640 1 -- 172.21.15.76:0/526975447 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95cc154600 con 0x7f95cc06c5f0 2023-11-09T03:55:46.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.433+0000 7f95d1ed4640 1 -- 172.21.15.76:0/526975447 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95cc0c8380 con 0x7f95cc06c5f0 2023-11-09T03:55:46.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.434+0000 7f95d1ed4640 1 -- 172.21.15.76:0/526975447 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95cc0684b0 con 0x7f95cc06c5f0 2023-11-09T03:55:46.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.435+0000 7f95b3fff640 1 -- 172.21.15.76:0/526975447 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f95bc0129d0 con 0x7f95cc06c5f0 2023-11-09T03:55:46.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.435+0000 7f95b3fff640 1 --2- 172.21.15.76:0/526975447 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f95b806ae70 0x7f95b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:46.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.435+0000 7f95b3fff640 1 -- 172.21.15.76:0/526975447 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8554+0+0 (secure 0 0 0) 0x7f95bc089c30 con 0x7f95cc06c5f0 2023-11-09T03:55:46.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.435+0000 7f95caffd640 1 --2- 172.21.15.76:0/526975447 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f95b806ae70 0x7f95b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:46.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.436+0000 7f95caffd640 1 --2- 172.21.15.76:0/526975447 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f95b806ae70 0x7f95b806d330 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f95c0006fd0 tx=0x7f95c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:46.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.439+0000 7f95b3fff640 1 -- 172.21.15.76:0/526975447 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f95bc054200 con 0x7f95cc06c5f0 2023-11-09T03:55:46.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.642+0000 7f95d1ed4640 1 -- 172.21.15.76:0/526975447 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f95cc0cddc0 con 0x7f95cc06c5f0 2023-11-09T03:55:46.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.642+0000 7f95b3fff640 1 -- 172.21.15.76:0/526975447 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v141) v1 ==== 53+0+73 (secure 0 0 0) 0x7f95bc057820 con 0x7f95cc06c5f0 2023-11-09T03:55:46.643 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-09T03:55:46.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.644+0000 7f95d1ed4640 1 -- 172.21.15.76:0/526975447 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f95b806ae70 msgr2=0x7f95b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:46.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.644+0000 7f95d1ed4640 1 --2- 172.21.15.76:0/526975447 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f95b806ae70 0x7f95b806d330 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f95c0006fd0 tx=0x7f95c0008040 comp rx=0 tx=0).stop 2023-11-09T03:55:46.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.644+0000 7f95d1ed4640 1 -- 172.21.15.76:0/526975447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95cc06c5f0 msgr2=0x7f95cc157140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:46.646 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.644+0000 7f95d1ed4640 1 --2- 172.21.15.76:0/526975447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95cc06c5f0 0x7f95cc157140 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00c450 tx=0x7f95bc00c910 comp rx=0 tx=0).stop 2023-11-09T03:55:46.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.644+0000 7f95d1ed4640 1 -- 172.21.15.76:0/526975447 shutdown_connections 2023-11-09T03:55:46.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.644+0000 7f95d1ed4640 1 --2- 172.21.15.76:0/526975447 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f95b806ae70 0x7f95b806d330 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:46.647 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.644+0000 7f95d1ed4640 1 --2- 172.21.15.76:0/526975447 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95cc0ade00 0x7f95cc0c72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:46.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.644+0000 7f95d1ed4640 1 --2- 172.21.15.76:0/526975447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95cc06d9c0 0x7f95cc14ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:46.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.644+0000 7f95d1ed4640 1 --2- 172.21.15.76:0/526975447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95cc06c5f0 0x7f95cc157140 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:46.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.644+0000 7f95d1ed4640 1 -- 172.21.15.76:0/526975447 >> 172.21.15.76:0/526975447 conn(0x7f95cc0cb530 msgr2=0x7f95cc0b01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:46.648 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.644+0000 7f95d1ed4640 1 -- 172.21.15.76:0/526975447 shutdown_connections 2023-11-09T03:55:46.649 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:46.644+0000 7f95d1ed4640 1 -- 172.21.15.76:0/526975447 wait complete. 2023-11-09T03:55:46.656 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-09T03:55:46.656 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-09T03:55:46.657 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2023-11-09T03:55:46.657 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-09T03:55:46.657 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-09T03:55:46.658 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-09T03:55:46.658 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-09T03:55:46.659 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-09T03:55:46.659 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:55:46.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:55:46.660 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2023-11-09T03:55:46.712 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-09T03:55:46.713 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-09T03:55:46.713 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2023-11-09T03:55:46.713 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:55:46.713 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2023-11-09T03:55:46.714 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:55:46.714 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:55:46.714 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:55:46.715 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2023-11-09T03:55:46.715 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:55:46.715 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:55:46.716 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2023-11-09T03:55:46.716 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-09T03:55:46.716 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:55:46.717 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-09T03:55:46.717 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2023-11-09T03:55:46.717 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-09T03:55:46.717 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42002sec preferred_lft 42002sec 2023-11-09T03:55:46.718 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2023-11-09T03:55:46.718 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:55:46.718 INFO:teuthology.orchestra.run.smithi076.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-09T03:55:46.719 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2e:15:bd:03:a8:2e brd ff:ff:ff:ff:ff:ff 2023-11-09T03:55:46.719 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-09T03:55:46.719 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:55:46.719 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::bcda:38ff:fed4:fdb0/64 scope link 2023-11-09T03:55:46.720 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:55:46.720 INFO:teuthology.orchestra.run.smithi076.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-09T03:55:46.720 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 6a:8d:94:d7:e5:00 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:55:46.721 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::688d:94ff:fed7:e500/64 scope link 2023-11-09T03:55:46.721 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:55:46.721 INFO:teuthology.orchestra.run.smithi076.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-09T03:55:46.721 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2e:15:bd:03:a8:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:55:46.722 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::2c15:bdff:fe03:a82e/64 scope link 2023-11-09T03:55:46.722 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-09T03:55:46.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:55:46.723 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-09T03:55:46.779 INFO:teuthology.orchestra.run.smithi076.stdout:1 2023-11-09T03:55:46.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:55:46.781 DEBUG:teuthology.orchestra.run.smithi076:> route 2023-11-09T03:55:46.837 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2023-11-09T03:55:46.837 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-09T03:55:46.838 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-09T03:55:46.838 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-09T03:55:46.838 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-09T03:55:46.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:55:46.839 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-09T03:55:46.840 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-09T03:55:46.840 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-09T03:55:46.841 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-09T03:55:46.841 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-09T03:55:46.842 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-09T03:55:46.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:55:46.933 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2023-11-09T03:55:46.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-09T03:55:46.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-09T03:55:46.949 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:55:46.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:55:46.950 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2023-11-09T03:55:47.015 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-09T03:55:47.016 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:55:47.016 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-09T03:55:47.070 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:55:47.071 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-09T03:55:47.071 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-09T03:55:47.125 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:55:47.181 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:55:47.181 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:55:47.181 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:55:47.182 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:55:47.182 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:55:47.182 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:55:47.182 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:55:47.183 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:55:47.183 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:55:47.183 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:55:47.183 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:55:47.184 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:55:47.184 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:55:47.184 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:55:47.184 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:55:47.185 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:55:47.185 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:55:47.185 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:55:47.185 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:55:47.185 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:55:47.186 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:55:47.186 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:55:47.186 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:55:47.186 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:55:47.187 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:55:47.187 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:55:47.187 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:55:47.187 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:55:47.188 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:55:47.188 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:55:47.188 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:55:47.188 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:55:47.188 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:55:47.189 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:55:47.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:55:47.189 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:55:47.236 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-09T03:55:47.236 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-09T03:55:47.279 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:55:47.295 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:55:47.295 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:55:47.295 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:55:47.295 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:55:47.295 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:55:47.296 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:55:47.296 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:55:47.296 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:55:47.296 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:55:47.296 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:55:47.297 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:55:47.297 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:55:47.297 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:55:47.297 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:55:47.297 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:55:47.298 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:55:47.298 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:55:47.298 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:55:47.298 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:55:47.299 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:55:47.299 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:55:47.299 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:55:47.299 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:55:47.299 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:55:47.300 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:55:47.300 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:55:47.300 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:55:47.300 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:55:47.300 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:55:47.301 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:55:47.301 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:55:47.301 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:55:47.301 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:55:47.301 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:55:47.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:55:47.302 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:55:47.369 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:55:47.368+0000 7f331d985480 -1 init, newargv = 0x561a1bab4990 newargc=15 2023-11-09T03:55:47.369 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[133888]: starting ceph client 2023-11-09T03:55:47.376 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[133888]: starting fuse 2023-11-09T03:55:48.452 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2023-11-09T03:55:48.472 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2023-11-09T03:55:48.472 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2023-11-09T03:55:48.473 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2023-11-09T03:55:48.473 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2023-11-09T03:55:48.473 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2023-11-09T03:55:48.473 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2023-11-09T03:55:48.473 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2023-11-09T03:55:48.474 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2023-11-09T03:55:48.474 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2023-11-09T03:55:48.474 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2023-11-09T03:55:48.474 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2023-11-09T03:55:48.475 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2023-11-09T03:55:48.475 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2023-11-09T03:55:48.475 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2023-11-09T03:55:48.475 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2023-11-09T03:55:48.475 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2023-11-09T03:55:48.476 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2023-11-09T03:55:48.476 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-09T03:55:48.476 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2023-11-09T03:55:48.476 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2023-11-09T03:55:48.477 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2023-11-09T03:55:48.477 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-09T03:55:48.477 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2023-11-09T03:55:48.477 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2023-11-09T03:55:48.477 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2023-11-09T03:55:48.478 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2023-11-09T03:55:48.478 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2023-11-09T03:55:48.478 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2023-11-09T03:55:48.478 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2023-11-09T03:55:48.478 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2023-11-09T03:55:48.479 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:55:48.479 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T03:55:48.479 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:55:48.479 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T03:55:48.480 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-09T03:55:48.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:55:48.480 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2023-11-09T03:55:48.530 INFO:teuthology.orchestra.run.smithi076.stdout:43 2023-11-09T03:55:48.530 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-09T03:55:48.531 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer -- python3 -c ' 2023-11-09T03:55:48.531 DEBUG:teuthology.orchestra.run.smithi076:> import glob 2023-11-09T03:55:48.531 DEBUG:teuthology.orchestra.run.smithi076:> import re 2023-11-09T03:55:48.532 DEBUG:teuthology.orchestra.run.smithi076:> import os 2023-11-09T03:55:48.532 DEBUG:teuthology.orchestra.run.smithi076:> import subprocess 2023-11-09T03:55:48.532 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:55:48.533 DEBUG:teuthology.orchestra.run.smithi076:> def _find_admin_socket(client_name): 2023-11-09T03:55:48.533 DEBUG:teuthology.orchestra.run.smithi076:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-09T03:55:48.534 DEBUG:teuthology.orchestra.run.smithi076:> files = glob.glob(asok_path) 2023-11-09T03:55:48.534 DEBUG:teuthology.orchestra.run.smithi076:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-09T03:55:48.534 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:55:48.535 DEBUG:teuthology.orchestra.run.smithi076:> # Given a non-glob path, it better be there 2023-11-09T03:55:48.535 DEBUG:teuthology.orchestra.run.smithi076:> if "*" not in asok_path: 2023-11-09T03:55:48.536 DEBUG:teuthology.orchestra.run.smithi076:> assert(len(files) == 1) 2023-11-09T03:55:48.536 DEBUG:teuthology.orchestra.run.smithi076:> return files[0] 2023-11-09T03:55:48.536 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:55:48.537 DEBUG:teuthology.orchestra.run.smithi076:> for f in files: 2023-11-09T03:55:48.537 DEBUG:teuthology.orchestra.run.smithi076:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-09T03:55:48.537 DEBUG:teuthology.orchestra.run.smithi076:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-09T03:55:48.538 DEBUG:teuthology.orchestra.run.smithi076:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-09T03:55:48.538 DEBUG:teuthology.orchestra.run.smithi076:> contents = proc_f.read() 2023-11-09T03:55:48.539 DEBUG:teuthology.orchestra.run.smithi076:> if mountpoint in contents: 2023-11-09T03:55:48.539 DEBUG:teuthology.orchestra.run.smithi076:> return f 2023-11-09T03:55:48.539 DEBUG:teuthology.orchestra.run.smithi076:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-09T03:55:48.540 DEBUG:teuthology.orchestra.run.smithi076:> 2023-11-09T03:55:48.540 DEBUG:teuthology.orchestra.run.smithi076:> print(_find_admin_socket("client.0")) 2023-11-09T03:55:48.541 DEBUG:teuthology.orchestra.run.smithi076:> ' 2023-11-09T03:55:48.656 INFO:teuthology.orchestra.run.smithi076.stdout:/var/run/ceph/ceph-client.0.133888.asok 2023-11-09T03:55:48.659 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:55:48.667 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.133888.asok 2023-11-09T03:55:48.667 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:55:48.668 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.133888.asok status 2023-11-09T03:55:48.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:55:48.837 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:55:48.837 INFO:teuthology.orchestra.run.smithi076.stdout: "metadata": { 2023-11-09T03:55:48.837 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_sha1": "7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4", 2023-11-09T03:55:48.837 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev)", 2023-11-09T03:55:48.837 INFO:teuthology.orchestra.run.smithi076.stdout: "entity_id": "0", 2023-11-09T03:55:48.838 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": "smithi076", 2023-11-09T03:55:48.838 INFO:teuthology.orchestra.run.smithi076.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-09T03:55:48.838 INFO:teuthology.orchestra.run.smithi076.stdout: "pid": "133888", 2023-11-09T03:55:48.838 INFO:teuthology.orchestra.run.smithi076.stdout: "root": "/" 2023-11-09T03:55:48.838 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:55:48.839 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_count": 0, 2023-11-09T03:55:48.839 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_pinned_count": 0, 2023-11-09T03:55:48.839 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 9675, 2023-11-09T03:55:48.839 INFO:teuthology.orchestra.run.smithi076.stdout: "inst": { 2023-11-09T03:55:48.839 INFO:teuthology.orchestra.run.smithi076.stdout: "name": { 2023-11-09T03:55:48.840 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "client", 2023-11-09T03:55:48.840 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 9675 2023-11-09T03:55:48.840 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:55:48.840 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2023-11-09T03:55:48.840 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2023-11-09T03:55:48.841 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2023-11-09T03:55:48.841 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3178019461 2023-11-09T03:55:48.841 INFO:teuthology.orchestra.run.smithi076.stdout: } 2023-11-09T03:55:48.841 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:55:48.841 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2023-11-09T03:55:48.842 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2023-11-09T03:55:48.842 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2023-11-09T03:55:48.842 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3178019461 2023-11-09T03:55:48.842 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2023-11-09T03:55:48.842 INFO:teuthology.orchestra.run.smithi076.stdout: "inst_str": "client.9675 192.168.0.1:0/3178019461", 2023-11-09T03:55:48.843 INFO:teuthology.orchestra.run.smithi076.stdout: "addr_str": "192.168.0.1:0/3178019461", 2023-11-09T03:55:48.843 INFO:teuthology.orchestra.run.smithi076.stdout: "inode_count": 1, 2023-11-09T03:55:48.843 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_epoch": 141, 2023-11-09T03:55:48.843 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch": 120, 2023-11-09T03:55:48.843 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch_barrier": 0, 2023-11-09T03:55:48.843 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklisted": false, 2023-11-09T03:55:48.844 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs" 2023-11-09T03:55:48.844 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:55:48.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:55:48.851 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:55:48.851 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-11-09T03:55:48.852 DEBUG:teuthology.orchestra.run.smithi076:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-09T03:55:48.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:55:48.912 INFO:teuthology.orchestra.run.smithi076.stdout:fuseblk 2023-11-09T03:55:48.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:55:48.918 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-09T03:55:48.919 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:55:48.919 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-09T03:55:48.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:55:48.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:55:48.985 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 flush journal 2023-11-09T03:55:49.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.086+0000 7fae3dab4640 1 Processor -- start 2023-11-09T03:55:49.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.086+0000 7fae3dab4640 1 -- start start 2023-11-09T03:55:49.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.087+0000 7fae3dab4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fae380ade00 0x7fae380b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.087+0000 7fae3dab4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fae3806c5f0 0x7fae380cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.087+0000 7fae3dab4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae3806cc90 con 0x7fae3806d9c0 2023-11-09T03:55:49.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.087+0000 7fae3dab4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae380b0e60 con 0x7fae380ade00 2023-11-09T03:55:49.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.087+0000 7fae3dab4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae3806cee0 con 0x7fae3806c5f0 2023-11-09T03:55:49.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.087+0000 7fae377fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fae380ade00 0x7fae380b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.087+0000 7fae377fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fae380ade00 0x7fae380b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49784/0 (socket says 172.21.15.76:49784) 2023-11-09T03:55:49.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.087+0000 7fae377fe640 1 -- 172.21.15.76:0/3067979954 learned_addr learned my addr 172.21.15.76:0/3067979954 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:49.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.087+0000 7fae37fff640 1 --2- 172.21.15.76:0/3067979954 >> v2:172.21.15.113:3300/0 conn(0x7fae3806c5f0 0x7fae380cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.087+0000 7fae377fe640 1 -- 172.21.15.76:0/3067979954 >> v2:172.21.15.113:3300/0 conn(0x7fae3806c5f0 msgr2=0x7fae380cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:49.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.087+0000 7fae377fe640 1 --2- 172.21.15.76:0/3067979954 >> v2:172.21.15.113:3300/0 conn(0x7fae3806c5f0 0x7fae380cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.087+0000 7fae377fe640 1 -- 172.21.15.76:0/3067979954 >> v1:172.21.15.113:6789/0 conn(0x7fae3806d9c0 legacy=0x7fae380710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:49.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.087+0000 7fae377fe640 1 -- 172.21.15.76:0/3067979954 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3806d1c0 con 0x7fae380ade00 2023-11-09T03:55:49.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.087+0000 7fae37fff640 1 --2- 172.21.15.76:0/3067979954 >> v2:172.21.15.113:3300/0 conn(0x7fae3806c5f0 0x7fae380cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:55:49.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.088+0000 7fae377fe640 1 --2- 172.21.15.76:0/3067979954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae380ade00 0x7fae380b0530 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fae380b44f0 tx=0x7fae2802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195dce6555bbf583 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:49.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.088+0000 7fae367fc640 1 -- 172.21.15.76:0/3067979954 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae28031750 con 0x7fae380ade00 2023-11-09T03:55:49.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.088+0000 7fae367fc640 1 -- 172.21.15.76:0/3067979954 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae280318b0 con 0x7fae380ade00 2023-11-09T03:55:49.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.088+0000 7fae367fc640 1 -- 172.21.15.76:0/3067979954 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae28031c40 con 0x7fae380ade00 2023-11-09T03:55:49.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.089+0000 7fae3dab4640 1 -- 172.21.15.76:0/3067979954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae380ade00 msgr2=0x7fae380b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:49.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.089+0000 7fae3dab4640 1 --2- 172.21.15.76:0/3067979954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae380ade00 0x7fae380b0530 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fae380b44f0 tx=0x7fae2802b7f0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.089+0000 7fae3dab4640 1 -- 172.21.15.76:0/3067979954 shutdown_connections 2023-11-09T03:55:49.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.089+0000 7fae3dab4640 1 --2- 172.21.15.76:0/3067979954 >> v2:172.21.15.113:3300/0 conn(0x7fae3806c5f0 0x7fae380cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.089+0000 7fae3dab4640 1 --2- 172.21.15.76:0/3067979954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae380ade00 0x7fae380b0530 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.089+0000 7fae3dab4640 1 -- 172.21.15.76:0/3067979954 >> 172.21.15.76:0/3067979954 conn(0x7fae380cb530 msgr2=0x7fae380cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:49.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.089+0000 7fae3dab4640 1 -- 172.21.15.76:0/3067979954 shutdown_connections 2023-11-09T03:55:49.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.089+0000 7fae3dab4640 1 -- 172.21.15.76:0/3067979954 wait complete. 2023-11-09T03:55:49.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.090+0000 7fae3dab4640 1 Processor -- start 2023-11-09T03:55:49.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.090+0000 7fae3dab4640 1 -- start start 2023-11-09T03:55:49.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.091+0000 7fae3dab4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae3806c5f0 0x7fae38154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.091+0000 7fae3dab4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae3806d9c0 0x7fae38156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.091+0000 7fae3dab4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae380ade00 0x7fae3815a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.091+0000 7fae3dab4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae381545c0 con 0x7fae3806d9c0 2023-11-09T03:55:49.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.091+0000 7fae3dab4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae38156c50 con 0x7fae3806c5f0 2023-11-09T03:55:49.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.091+0000 7fae3dab4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae38156f30 con 0x7fae380ade00 2023-11-09T03:55:49.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.091+0000 7fae36ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae3806d9c0 0x7fae38156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.091+0000 7fae36ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae3806d9c0 0x7fae38156420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49800/0 (socket says 172.21.15.76:49800) 2023-11-09T03:55:49.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.091+0000 7fae36ffd640 1 -- 172.21.15.76:0/2402739093 learned_addr learned my addr 172.21.15.76:0/2402739093 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:49.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.091+0000 7fae377fe640 1 --2- 172.21.15.76:0/2402739093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae3806c5f0 0x7fae38154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.091+0000 7fae37fff640 1 --2- 172.21.15.76:0/2402739093 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae380ade00 0x7fae3815a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.091+0000 7fae36ffd640 1 -- 172.21.15.76:0/2402739093 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae380ade00 msgr2=0x7fae3815a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:49.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.091+0000 7fae36ffd640 1 --2- 172.21.15.76:0/2402739093 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae380ade00 0x7fae3815a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.091+0000 7fae36ffd640 1 -- 172.21.15.76:0/2402739093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae3806c5f0 msgr2=0x7fae38154080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:49.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.091+0000 7fae36ffd640 1 --2- 172.21.15.76:0/2402739093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae3806c5f0 0x7fae38154080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.091+0000 7fae36ffd640 1 -- 172.21.15.76:0/2402739093 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae280094d0 con 0x7fae3806d9c0 2023-11-09T03:55:49.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.091+0000 7fae36ffd640 1 --2- 172.21.15.76:0/2402739093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae3806d9c0 0x7fae38156420 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00b410 tx=0x7fae2c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:49.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.091+0000 7fae1ffff640 1 -- 172.21.15.76:0/2402739093 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c011830 con 0x7fae3806d9c0 2023-11-09T03:55:49.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.091+0000 7fae1ffff640 1 -- 172.21.15.76:0/2402739093 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2c011990 con 0x7fae3806d9c0 2023-11-09T03:55:49.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.091+0000 7fae3dab4640 1 -- 172.21.15.76:0/2402739093 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae38157210 con 0x7fae3806d9c0 2023-11-09T03:55:49.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.091+0000 7fae1ffff640 1 -- 172.21.15.76:0/2402739093 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c011d40 con 0x7fae3806d9c0 2023-11-09T03:55:49.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.091+0000 7fae3dab4640 1 -- 172.21.15.76:0/2402739093 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae38156960 con 0x7fae3806d9c0 2023-11-09T03:55:49.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.093+0000 7fae1ffff640 1 -- 172.21.15.76:0/2402739093 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fae2c01a400 con 0x7fae3806d9c0 2023-11-09T03:55:49.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.094+0000 7fae1ffff640 1 --2- 172.21.15.76:0/2402739093 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae2406ae70 0x7fae2406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.094+0000 7fae1ffff640 1 -- 172.21.15.76:0/2402739093 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8554+0+0 (secure 0 0 0) 0x7fae2c08a070 con 0x7fae3806d9c0 2023-11-09T03:55:49.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.094+0000 7fae377fe640 1 --2- 172.21.15.76:0/2402739093 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae2406ae70 0x7fae2406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.094+0000 7fae40646740 1 Processor -- start 2023-11-09T03:55:49.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.094+0000 7fae377fe640 1 --2- 172.21.15.76:0/2402739093 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae2406ae70 0x7fae2406d330 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fae28035000 tx=0x7fae28008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:49.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.094+0000 7fae40646740 1 -- start start 2023-11-09T03:55:49.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.095+0000 7fae40646740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55dd12b571f0 0x55dd12b392d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.095+0000 7fae40646740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd12b54d10 0x55dd12b39810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.095+0000 7fae40646740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd12b36900 0x55dd12b0d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.095+0000 7fae40646740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd12a7cd70 con 0x55dd12b36900 2023-11-09T03:55:49.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.095+0000 7fae40646740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd12a7a460 con 0x55dd12b54d10 2023-11-09T03:55:49.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.095+0000 7fae40646740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dd12a75d50 con 0x55dd12b571f0 2023-11-09T03:55:49.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.095+0000 7fae37fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd12b36900 0x55dd12b0d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.096+0000 7fae37fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd12b36900 0x55dd12b0d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49810/0 (socket says 172.21.15.76:49810) 2023-11-09T03:55:49.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.096+0000 7fae37fff640 1 -- 172.21.15.76:0/2570838665 learned_addr learned my addr 172.21.15.76:0/2570838665 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:49.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.096+0000 7fae377fe640 1 --2- 172.21.15.76:0/2570838665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55dd12b571f0 0x55dd12b392d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.096+0000 7fae36ffd640 1 --2- 172.21.15.76:0/2570838665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd12b54d10 0x55dd12b39810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.096+0000 7fae37fff640 1 -- 172.21.15.76:0/2570838665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55dd12b571f0 msgr2=0x55dd12b392d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:49.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.096+0000 7fae37fff640 1 --2- 172.21.15.76:0/2570838665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55dd12b571f0 0x55dd12b392d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.096+0000 7fae37fff640 1 -- 172.21.15.76:0/2570838665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd12b54d10 msgr2=0x55dd12b39810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:49.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.096+0000 7fae37fff640 1 --2- 172.21.15.76:0/2570838665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd12b54d10 0x55dd12b39810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.096+0000 7fae37fff640 1 -- 172.21.15.76:0/2570838665 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd12a38e30 con 0x55dd12b36900 2023-11-09T03:55:49.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.096+0000 7fae37fff640 1 --2- 172.21.15.76:0/2570838665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd12b36900 0x55dd12b0d040 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fae3000f4f0 tx=0x7fae3000f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93605b1ae8b8d38 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:49.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.096+0000 7fae1d7fa640 1 -- 172.21.15.76:0/2570838665 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30007b90 con 0x55dd12b36900 2023-11-09T03:55:49.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.096+0000 7fae1d7fa640 1 -- 172.21.15.76:0/2570838665 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3000d780 con 0x55dd12b36900 2023-11-09T03:55:49.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.096+0000 7fae1d7fa640 1 -- 172.21.15.76:0/2570838665 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3000daf0 con 0x55dd12b36900 2023-11-09T03:55:49.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.097+0000 7fae40646740 1 -- 172.21.15.76:0/2570838665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd12b36900 msgr2=0x55dd12b0d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:49.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.097+0000 7fae40646740 1 --2- 172.21.15.76:0/2570838665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd12b36900 0x55dd12b0d040 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fae3000f4f0 tx=0x7fae3000f9b0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.097+0000 7fae40646740 1 -- 172.21.15.76:0/2570838665 shutdown_connections 2023-11-09T03:55:49.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.097+0000 7fae40646740 1 --2- 172.21.15.76:0/2570838665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd12b36900 0x55dd12b0d040 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.097+0000 7fae40646740 1 --2- 172.21.15.76:0/2570838665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd12b54d10 0x55dd12b39810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.097+0000 7fae40646740 1 --2- 172.21.15.76:0/2570838665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55dd12b571f0 0x55dd12b392d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.097+0000 7fae40646740 1 -- 172.21.15.76:0/2570838665 >> 172.21.15.76:0/2570838665 conn(0x55dd12b0b9e0 msgr2=0x55dd12b542d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:49.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.097+0000 7fae40646740 1 -- 172.21.15.76:0/2570838665 shutdown_connections 2023-11-09T03:55:49.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.097+0000 7fae40646740 1 -- 172.21.15.76:0/2570838665 wait complete. 2023-11-09T03:55:49.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.097+0000 7fae40646740 1 -- start start 2023-11-09T03:55:49.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.097+0000 7fae40646740 1 Processor -- start 2023-11-09T03:55:49.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.098+0000 7fae40646740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd12b54d10 0x55dd12b304b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.098+0000 7fae40646740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55dd12b571f0 0x55dd12b55ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.098+0000 7fae40646740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd12b561e0 0x55dd12b3bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.112 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.098+0000 7fae40646740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd12a2fee0 con 0x55dd12b54d10 2023-11-09T03:55:49.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.098+0000 7fae40646740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd12a2fb80 con 0x55dd12b561e0 2023-11-09T03:55:49.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.098+0000 7fae40646740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dd129f53a0 con 0x55dd12b571f0 2023-11-09T03:55:49.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.098+0000 7fae40646740 10 client.0 _ll_register_callbacks cb 0x55dd129f98b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:55:49.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.098+0000 7fae377fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd12b54d10 0x55dd12b304b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.098+0000 7fae377fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd12b54d10 0x55dd12b304b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49822/0 (socket says 172.21.15.76:49822) 2023-11-09T03:55:49.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.098+0000 7fae377fe640 1 -- 172.21.15.76:0/3150606654 learned_addr learned my addr 172.21.15.76:0/3150606654 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:49.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.098+0000 7fae37fff640 1 --2- 172.21.15.76:0/3150606654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd12b561e0 0x55dd12b3bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.098+0000 7fae36ffd640 1 --2- 172.21.15.76:0/3150606654 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55dd12b571f0 0x55dd12b55ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.099+0000 7fae377fe640 1 -- 172.21.15.76:0/3150606654 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55dd12b571f0 msgr2=0x55dd12b55ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:49.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.099+0000 7fae377fe640 1 --2- 172.21.15.76:0/3150606654 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55dd12b571f0 0x55dd12b55ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.099+0000 7fae377fe640 1 -- 172.21.15.76:0/3150606654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd12b561e0 msgr2=0x55dd12b3bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:49.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.099+0000 7fae377fe640 1 --2- 172.21.15.76:0/3150606654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd12b561e0 0x55dd12b3bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.099+0000 7fae377fe640 1 -- 172.21.15.76:0/3150606654 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae30007cf0 con 0x55dd12b54d10 2023-11-09T03:55:49.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.099+0000 7fae377fe640 1 --2- 172.21.15.76:0/3150606654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd12b54d10 0x55dd12b304b0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fae28009960 tx=0x7fae2802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b1a6d1b9b7aa49d server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:49.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.099+0000 7fae167fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:55:49.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.099+0000 7fae167fc640 1 -- 172.21.15.76:0/3150606654 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae28033ad0 con 0x55dd12b54d10 2023-11-09T03:55:49.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.099+0000 7fae167fc640 1 -- 172.21.15.76:0/3150606654 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8554+0+0 (secure 0 0 0) 0x7fae2804c070 con 0x55dd12b54d10 2023-11-09T03:55:49.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.099+0000 7fae40646740 1 -- 172.21.15.76:0/3150606654 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd12a6ed30 con 0x55dd12b54d10 2023-11-09T03:55:49.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.099+0000 7fae167fc640 1 -- 172.21.15.76:0/3150606654 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=141) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae28033c30 con 0x55dd12b54d10 2023-11-09T03:55:49.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.099+0000 7fae40646740 10 client.9687 fetch_fsmap learned FSMap version 141 2023-11-09T03:55:49.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.099+0000 7fae40646740 1 -- 172.21.15.76:0/3150606654 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=141,monmap=2+}) v3 -- 0x55dd12a743d0 con 0x55dd12b54d10 2023-11-09T03:55:49.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.099+0000 7fae167fc640 1 -- 172.21.15.76:0/3150606654 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2805f020 con 0x55dd12b54d10 2023-11-09T03:55:49.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.099+0000 7fae167fc640 1 -- 172.21.15.76:0/3150606654 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 141) v1 ==== 3306+0+0 (secure 0 0 0) 0x7fae28045040 con 0x55dd12b54d10 2023-11-09T03:55:49.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.099+0000 7fae40646740 10 client.9687 fetch_fsmap finished waiting for FSMap version 141 2023-11-09T03:55:49.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.099+0000 7fae40646740 10 client.9687 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2023-11-09T03:55:49.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.099+0000 7fae40646740 20 client.9687 populate_metadata read hostname 'smithi076' 2023-11-09T03:55:49.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.099+0000 7fae40646740 1 --2- 172.21.15.76:0/3150606654 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x55dd12b33480 0x55dd12b33860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.099+0000 7fae40646740 4 client.9687 mds_command: new command op to 8568 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-09T03:55:49.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.099+0000 7fae40646740 1 -- 172.21.15.76:0/3150606654 --> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd12a6ea50 con 0x55dd12b33480 2023-11-09T03:55:49.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.099+0000 7fae36ffd640 1 --2- 172.21.15.76:0/3150606654 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x55dd12b33480 0x55dd12b33860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.100+0000 7fae167fc640 10 client.9687 ms_handle_connect on v2:172.21.15.76:6832/153465374 2023-11-09T03:55:49.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.100+0000 7fae36ffd640 1 --2- 172.21.15.76:0/3150606654 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x55dd12b33480 0x55dd12b33860 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25de50be35953b10 server_cookie=677f75db2428d4b6 in_seq=0 out_seq=0 2023-11-09T03:55:49.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.100+0000 7fae167fc640 1 -- 172.21.15.76:0/3150606654 <== mds.0 v2:172.21.15.76:6832/153465374 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dd12a6ea50 con 0x55dd12b33480 2023-11-09T03:55:49.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.100+0000 7fae167fc640 10 client.9687 handle_command_reply: tid=0 2023-11-09T03:55:49.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.100+0000 7fae40646740 1 client.9687 shutdown 2023-11-09T03:55:49.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.100+0000 7fae40646740 1 -- 172.21.15.76:0/3150606654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd12b54d10 msgr2=0x55dd12b304b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:49.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.100+0000 7fae40646740 1 --2- 172.21.15.76:0/3150606654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd12b54d10 0x55dd12b304b0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fae28009960 tx=0x7fae2802e000 comp rx=0 tx=0).stop 2023-11-09T03:55:49.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.100+0000 7fae40646740 1 -- 172.21.15.76:0/3150606654 shutdown_connections 2023-11-09T03:55:49.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.100+0000 7fae40646740 1 --2- 172.21.15.76:0/3150606654 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x55dd12b33480 0x55dd12b33860 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.100+0000 7fae40646740 1 --2- 172.21.15.76:0/3150606654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd12b561e0 0x55dd12b3bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.100+0000 7fae167fc640 0 client.9687 ms_handle_reset on v2:172.21.15.76:6832/153465374 2023-11-09T03:55:49.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.100+0000 7fae40646740 1 --2- 172.21.15.76:0/3150606654 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55dd12b571f0 0x55dd12b55ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.100+0000 7fae40646740 1 --2- 172.21.15.76:0/3150606654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd12b54d10 0x55dd12b304b0 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.100+0000 7fae40646740 1 -- 172.21.15.76:0/3150606654 >> 172.21.15.76:0/3150606654 conn(0x55dd12b36900 msgr2=0x55dd12b320e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:49.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.100+0000 7fae40646740 1 -- 172.21.15.76:0/3150606654 shutdown_connections 2023-11-09T03:55:49.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.100+0000 7fae40646740 1 -- 172.21.15.76:0/3150606654 wait complete. 2023-11-09T03:55:49.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.101+0000 7fae40646740 20 client.9687 trim_cache size 0 max 16384 2023-11-09T03:55:49.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.113+0000 7fae40646740 1 Processor -- start 2023-11-09T03:55:49.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.114+0000 7fae40646740 1 -- start start 2023-11-09T03:55:49.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.114+0000 7fae40646740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd12b54d10 0x55dd12b391c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.114+0000 7fae40646740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55dd12b561e0 0x55dd12b39700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.114+0000 7fae40646740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd12b571f0 0x55dd12b0df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.114+0000 7fae40646740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd12a0ced0 con 0x55dd12b54d10 2023-11-09T03:55:49.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.114+0000 7fae40646740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd12b566d0 con 0x55dd12b571f0 2023-11-09T03:55:49.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.114+0000 7fae40646740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dd12b60890 con 0x55dd12b561e0 2023-11-09T03:55:49.126 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.114+0000 7fae377fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd12b54d10 0x55dd12b391c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.114+0000 7fae377fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd12b54d10 0x55dd12b391c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49828/0 (socket says 172.21.15.76:49828) 2023-11-09T03:55:49.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.114+0000 7fae377fe640 1 -- 172.21.15.76:0/3995700405 learned_addr learned my addr 172.21.15.76:0/3995700405 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:49.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.115+0000 7fae37fff640 1 --2- 172.21.15.76:0/3995700405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd12b571f0 0x55dd12b0df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.115+0000 7fae36ffd640 1 --2- 172.21.15.76:0/3995700405 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55dd12b561e0 0x55dd12b39700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.127 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.115+0000 7fae377fe640 1 -- 172.21.15.76:0/3995700405 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55dd12b561e0 msgr2=0x55dd12b39700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:49.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.115+0000 7fae377fe640 1 --2- 172.21.15.76:0/3995700405 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55dd12b561e0 0x55dd12b39700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.115+0000 7fae377fe640 1 -- 172.21.15.76:0/3995700405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd12b571f0 msgr2=0x55dd12b0df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:49.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.115+0000 7fae377fe640 1 --2- 172.21.15.76:0/3995700405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd12b571f0 0x55dd12b0df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.115+0000 7fae377fe640 1 -- 172.21.15.76:0/3995700405 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd12a11ea0 con 0x55dd12b54d10 2023-11-09T03:55:49.128 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.115+0000 7fae377fe640 1 --2- 172.21.15.76:0/3995700405 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd12b54d10 0x55dd12b391c0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fae28036040 tx=0x7fae2803f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad3c98541579c32b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:49.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.115+0000 7fae1dffb640 1 -- 172.21.15.76:0/3995700405 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae28052870 con 0x55dd12b54d10 2023-11-09T03:55:49.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.115+0000 7fae1dffb640 1 -- 172.21.15.76:0/3995700405 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2805f040 con 0x55dd12b54d10 2023-11-09T03:55:49.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.115+0000 7fae1dffb640 1 -- 172.21.15.76:0/3995700405 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae28052bd0 con 0x55dd12b54d10 2023-11-09T03:55:49.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.115+0000 7fae40646740 1 -- 172.21.15.76:0/3995700405 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd12b54d10 msgr2=0x55dd12b391c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:49.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.115+0000 7fae40646740 1 --2- 172.21.15.76:0/3995700405 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd12b54d10 0x55dd12b391c0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fae28036040 tx=0x7fae2803f070 comp rx=0 tx=0).stop 2023-11-09T03:55:49.129 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.116+0000 7fae40646740 1 -- 172.21.15.76:0/3995700405 shutdown_connections 2023-11-09T03:55:49.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.116+0000 7fae40646740 1 --2- 172.21.15.76:0/3995700405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd12b571f0 0x55dd12b0df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.116+0000 7fae40646740 1 --2- 172.21.15.76:0/3995700405 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55dd12b561e0 0x55dd12b39700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.116+0000 7fae40646740 1 --2- 172.21.15.76:0/3995700405 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd12b54d10 0x55dd12b391c0 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.116+0000 7fae40646740 1 -- 172.21.15.76:0/3995700405 >> 172.21.15.76:0/3995700405 conn(0x55dd12b36900 msgr2=0x55dd12b5dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:49.130 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.116+0000 7fae40646740 1 -- 172.21.15.76:0/3995700405 shutdown_connections 2023-11-09T03:55:49.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.116+0000 7fae40646740 1 -- 172.21.15.76:0/3995700405 wait complete. 2023-11-09T03:55:49.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.116+0000 7fae40646740 1 -- start start 2023-11-09T03:55:49.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.116+0000 7fae40646740 1 Processor -- start 2023-11-09T03:55:49.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.117+0000 7fae40646740 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55dd12b54d10 0x55dd12b391c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.117+0000 7fae40646740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd12b561e0 0x55dd12b39700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.117+0000 7fae40646740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd12b571f0 0x55dd12b66fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.117+0000 7fae40646740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd12bc7760 con 0x55dd12b561e0 2023-11-09T03:55:49.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.117+0000 7fae40646740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd12b3cc00 con 0x55dd12b571f0 2023-11-09T03:55:49.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.117+0000 7fae40646740 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dd12b2f9b0 con 0x55dd12b54d10 2023-11-09T03:55:49.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.117+0000 7fae40646740 10 client.0 _ll_register_callbacks cb 0x7fae30007ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-09T03:55:49.132 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.117+0000 7fae36ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd12b561e0 0x55dd12b39700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.117+0000 7fae36ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd12b561e0 0x55dd12b39700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49840/0 (socket says 172.21.15.76:49840) 2023-11-09T03:55:49.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.117+0000 7fae36ffd640 1 -- 172.21.15.76:0/1391641977 learned_addr learned my addr 172.21.15.76:0/1391641977 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:49.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.117+0000 7fae377fe640 1 --2- 172.21.15.76:0/1391641977 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55dd12b54d10 0x55dd12b391c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.117+0000 7fae37fff640 1 --2- 172.21.15.76:0/1391641977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd12b571f0 0x55dd12b66fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.117+0000 7fae36ffd640 1 -- 172.21.15.76:0/1391641977 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55dd12b54d10 msgr2=0x55dd12b391c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:49.133 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.117+0000 7fae36ffd640 1 --2- 172.21.15.76:0/1391641977 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55dd12b54d10 0x55dd12b391c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.117+0000 7fae36ffd640 1 -- 172.21.15.76:0/1391641977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd12b571f0 msgr2=0x55dd12b66fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:49.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.117+0000 7fae36ffd640 1 --2- 172.21.15.76:0/1391641977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd12b571f0 0x55dd12b66fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.117+0000 7fae36ffd640 1 -- 172.21.15.76:0/1391641977 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd12a743d0 con 0x55dd12b561e0 2023-11-09T03:55:49.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.117+0000 7fae36ffd640 1 --2- 172.21.15.76:0/1391641977 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd12b561e0 0x55dd12b39700 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00bdd0 tx=0x7fae2c089040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61faf570be2530c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:49.134 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.117+0000 7fae167fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2023-11-09T03:55:49.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.118+0000 7fae167fc640 1 -- 172.21.15.76:0/1391641977 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c06f040 con 0x55dd12b561e0 2023-11-09T03:55:49.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.118+0000 7fae167fc640 1 -- 172.21.15.76:0/1391641977 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8554+0+0 (secure 0 0 0) 0x7fae2c05e040 con 0x55dd12b561e0 2023-11-09T03:55:49.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.118+0000 7fae40646740 1 -- 172.21.15.76:0/1391641977 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae3000d780 con 0x55dd12b561e0 2023-11-09T03:55:49.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.118+0000 7fae167fc640 1 -- 172.21.15.76:0/1391641977 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=141) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae2c054ac0 con 0x55dd12b561e0 2023-11-09T03:55:49.135 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.118+0000 7fae40646740 10 client.9693 fetch_fsmap learned FSMap version 141 2023-11-09T03:55:49.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.118+0000 7fae40646740 1 -- 172.21.15.76:0/1391641977 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=141,monmap=2+}) v3 -- 0x55dd12a37690 con 0x55dd12b561e0 2023-11-09T03:55:49.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.118+0000 7fae167fc640 1 -- 172.21.15.76:0/1391641977 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3000d780 con 0x55dd12b561e0 2023-11-09T03:55:49.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.118+0000 7fae167fc640 1 -- 172.21.15.76:0/1391641977 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 141) v1 ==== 3306+0+0 (secure 0 0 0) 0x7fae2c060170 con 0x55dd12b561e0 2023-11-09T03:55:49.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.118+0000 7fae40646740 10 client.9693 fetch_fsmap finished waiting for FSMap version 141 2023-11-09T03:55:49.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.118+0000 7fae40646740 10 client.9693 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2023-11-09T03:55:49.136 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.118+0000 7fae40646740 20 client.9693 populate_metadata read hostname 'smithi076' 2023-11-09T03:55:49.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.118+0000 7fae40646740 1 --2- 172.21.15.76:0/1391641977 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x55dd12bacb20 0x55dd12b67500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.118+0000 7fae40646740 4 client.9693 mds_command: new command op to 8568 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-11-09T03:55:49.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.118+0000 7fae40646740 1 -- 172.21.15.76:0/1391641977 --> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dd127767e0 con 0x55dd12bacb20 2023-11-09T03:55:49.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.118+0000 7fae377fe640 1 --2- 172.21.15.76:0/1391641977 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x55dd12bacb20 0x55dd12b67500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.118+0000 7fae167fc640 10 client.9693 ms_handle_connect on v2:172.21.15.76:6832/153465374 2023-11-09T03:55:49.137 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.118+0000 7fae377fe640 1 --2- 172.21.15.76:0/1391641977 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x55dd12bacb20 0x55dd12b67500 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9107cfe0ebd7a27 server_cookie=c8849abf5187d661 in_seq=0 out_seq=0 2023-11-09T03:55:49.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.130+0000 7fae167fc640 1 -- 172.21.15.76:0/1391641977 <== mds.0 v2:172.21.15.76:6832/153465374 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dd127767e0 con 0x55dd12bacb20 2023-11-09T03:55:49.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.130+0000 7fae167fc640 10 client.9693 handle_command_reply: tid=0 2023-11-09T03:55:49.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.130+0000 7fae40646740 1 client.9693 shutdown 2023-11-09T03:55:49.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.130+0000 7fae40646740 1 -- 172.21.15.76:0/1391641977 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd12b561e0 msgr2=0x55dd12b39700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:49.139 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.130+0000 7fae40646740 1 --2- 172.21.15.76:0/1391641977 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd12b561e0 0x55dd12b39700 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00bdd0 tx=0x7fae2c089040 comp rx=0 tx=0).stop 2023-11-09T03:55:49.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.130+0000 7fae40646740 1 -- 172.21.15.76:0/1391641977 shutdown_connections 2023-11-09T03:55:49.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.130+0000 7fae40646740 1 --2- 172.21.15.76:0/1391641977 >> [v2:172.21.15.76:6832/153465374,v1:172.21.15.76:6833/153465374] conn(0x55dd12bacb20 0x55dd12b67500 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.130+0000 7fae40646740 1 --2- 172.21.15.76:0/1391641977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dd12b571f0 0x55dd12b66fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.130+0000 7fae40646740 1 --2- 172.21.15.76:0/1391641977 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dd12b561e0 0x55dd12b39700 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.140 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.130+0000 7fae40646740 1 --2- 172.21.15.76:0/1391641977 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55dd12b54d10 0x55dd12b391c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.130+0000 7fae167fc640 0 client.9693 ms_handle_reset on v2:172.21.15.76:6832/153465374 2023-11-09T03:55:49.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.130+0000 7fae40646740 1 -- 172.21.15.76:0/1391641977 >> 172.21.15.76:0/1391641977 conn(0x55dd12b36900 msgr2=0x55dd12b5dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:49.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.130+0000 7fae40646740 1 -- 172.21.15.76:0/1391641977 shutdown_connections 2023-11-09T03:55:49.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.130+0000 7fae40646740 1 -- 172.21.15.76:0/1391641977 wait complete. 2023-11-09T03:55:49.141 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.130+0000 7fae40646740 20 client.9693 trim_cache size 0 max 16384 2023-11-09T03:55:49.142 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-09T03:55:49.142 INFO:teuthology.orchestra.run.smithi076.stdout: "message": "", 2023-11-09T03:55:49.142 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2023-11-09T03:55:49.142 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-09T03:55:49.143 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.131+0000 7fae3dab4640 1 -- 172.21.15.76:0/2402739093 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae2406ae70 msgr2=0x7fae2406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:49.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.131+0000 7fae3dab4640 1 --2- 172.21.15.76:0/2402739093 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae2406ae70 0x7fae2406d330 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fae28035000 tx=0x7fae28008030 comp rx=0 tx=0).stop 2023-11-09T03:55:49.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.131+0000 7fae3dab4640 1 -- 172.21.15.76:0/2402739093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae3806d9c0 msgr2=0x7fae38156420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:49.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.131+0000 7fae3dab4640 1 --2- 172.21.15.76:0/2402739093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae3806d9c0 0x7fae38156420 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00b410 tx=0x7fae2c00b8d0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.131+0000 7fae3dab4640 1 -- 172.21.15.76:0/2402739093 shutdown_connections 2023-11-09T03:55:49.144 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.131+0000 7fae3dab4640 1 --2- 172.21.15.76:0/2402739093 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae2406ae70 0x7fae2406d330 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.131+0000 7fae3dab4640 1 --2- 172.21.15.76:0/2402739093 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae380ade00 0x7fae3815a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.131+0000 7fae3dab4640 1 --2- 172.21.15.76:0/2402739093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae3806d9c0 0x7fae38156420 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.131+0000 7fae3dab4640 1 --2- 172.21.15.76:0/2402739093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae3806c5f0 0x7fae38154080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.131+0000 7fae3dab4640 1 -- 172.21.15.76:0/2402739093 >> 172.21.15.76:0/2402739093 conn(0x7fae380cb530 msgr2=0x7fae38071090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:49.145 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.131+0000 7fae3dab4640 1 -- 172.21.15.76:0/2402739093 shutdown_connections 2023-11-09T03:55:49.146 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.131+0000 7fae3dab4640 1 -- 172.21.15.76:0/2402739093 wait complete. 2023-11-09T03:55:49.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:55:49.146 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir d1/dir/.snap/*) 2023-11-09T03:55:49.176 INFO:tasks.cephfs.test_snapshots:testing snapclient mds_kill_mdstable_at=3, snapserver mds_kill_mdstable_at=8 2023-11-09T03:55:49.176 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:55:49.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.317+0000 7f956a07c640 1 Processor -- start 2023-11-09T03:55:49.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.317+0000 7f956a07c640 1 -- start start 2023-11-09T03:55:49.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.318+0000 7f956a07c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f95640ade00 0x7f95640b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.318+0000 7f956a07c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f956406c5f0 0x7f95640cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.318+0000 7f956a07c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f956406cc90 con 0x7f956406d9c0 2023-11-09T03:55:49.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.318+0000 7f956a07c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95640b0e60 con 0x7f95640ade00 2023-11-09T03:55:49.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.318+0000 7f956a07c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956406cee0 con 0x7f956406c5f0 2023-11-09T03:55:49.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.318+0000 7f95637fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f95640ade00 0x7f95640b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.318+0000 7f9562ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f956406d9c0 0x7f95640710d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37710/0 (socket says 172.21.15.76:37710) 2023-11-09T03:55:49.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.318+0000 7f9562ffd640 1 -- 172.21.15.76:0/4048055548 learned_addr learned my addr 172.21.15.76:0/4048055548 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:55:49.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.318+0000 7f95637fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f95640ade00 0x7f95640b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49852/0 (socket says 172.21.15.76:49852) 2023-11-09T03:55:49.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.318+0000 7f9563fff640 1 --2- 172.21.15.76:0/4048055548 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f956406c5f0 0x7f95640cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.318+0000 7f95637fe640 1 -- 172.21.15.76:0/4048055548 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f956406c5f0 msgr2=0x7f95640cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:49.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.318+0000 7f95637fe640 1 --2- 172.21.15.76:0/4048055548 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f956406c5f0 0x7f95640cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.318+0000 7f95637fe640 1 -- 172.21.15.76:0/4048055548 >> v1:172.21.15.76:6789/0 conn(0x7f956406d9c0 legacy=0x7f95640710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:49.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.318+0000 7f95637fe640 1 -- 172.21.15.76:0/4048055548 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956406d1c0 con 0x7f95640ade00 2023-11-09T03:55:49.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.318+0000 7f95627fc640 1 -- 172.21.15.76:0/4048055548 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2984927805 0 0) 0x7f956406cc90 con 0x7f956406d9c0 2023-11-09T03:55:49.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.319+0000 7f95637fe640 1 --2- 172.21.15.76:0/4048055548 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95640ade00 0x7f95640b0530 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f95540097a0 tx=0x7f955402b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935552973fefc8f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:49.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.319+0000 7f95627fc640 1 -- 172.21.15.76:0/4048055548 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9554031710 con 0x7f95640ade00 2023-11-09T03:55:49.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.319+0000 7f95627fc640 1 -- 172.21.15.76:0/4048055548 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955402f050 con 0x7f95640ade00 2023-11-09T03:55:49.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.319+0000 7f95627fc640 1 -- 172.21.15.76:0/4048055548 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9554031a90 con 0x7f95640ade00 2023-11-09T03:55:49.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.320+0000 7f956a07c640 1 -- 172.21.15.76:0/4048055548 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95640ade00 msgr2=0x7f95640b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:49.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.320+0000 7f956a07c640 1 --2- 172.21.15.76:0/4048055548 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95640ade00 0x7f95640b0530 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f95540097a0 tx=0x7f955402b7b0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.320+0000 7f956a07c640 1 -- 172.21.15.76:0/4048055548 shutdown_connections 2023-11-09T03:55:49.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.320+0000 7f956a07c640 1 --2- 172.21.15.76:0/4048055548 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f956406c5f0 0x7f95640cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.320+0000 7f956a07c640 1 --2- 172.21.15.76:0/4048055548 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95640ade00 0x7f95640b0530 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.320+0000 7f956a07c640 1 -- 172.21.15.76:0/4048055548 >> 172.21.15.76:0/4048055548 conn(0x7f95640cb530 msgr2=0x7f95640cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:49.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.320+0000 7f956a07c640 1 -- 172.21.15.76:0/4048055548 shutdown_connections 2023-11-09T03:55:49.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.320+0000 7f956a07c640 1 -- 172.21.15.76:0/4048055548 wait complete. 2023-11-09T03:55:49.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.321+0000 7f956a07c640 1 Processor -- start 2023-11-09T03:55:49.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.321+0000 7f956a07c640 1 -- start start 2023-11-09T03:55:49.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.322+0000 7f956a07c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f956406c5f0 0x7f95640c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.322+0000 7f956a07c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f956406d9c0 0x7f956415ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.322+0000 7f956a07c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95640ade00 0x7f95640ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.322+0000 7f956a07c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95640ca3f0 con 0x7f95640ade00 2023-11-09T03:55:49.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.322+0000 7f956a07c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95640c9640 con 0x7f956406c5f0 2023-11-09T03:55:49.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.322+0000 7f956a07c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95640c9870 con 0x7f956406d9c0 2023-11-09T03:55:49.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.322+0000 7f9563fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95640ade00 0x7f95640ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.322+0000 7f9563fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95640ade00 0x7f95640ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49856/0 (socket says 172.21.15.76:49856) 2023-11-09T03:55:49.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.322+0000 7f9563fff640 1 -- 172.21.15.76:0/2739871223 learned_addr learned my addr 172.21.15.76:0/2739871223 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:49.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.322+0000 7f9562ffd640 1 --2- 172.21.15.76:0/2739871223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f956406d9c0 0x7f956415ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.322+0000 7f9563fff640 1 -- 172.21.15.76:0/2739871223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f956406d9c0 msgr2=0x7f956415ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:49.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.322+0000 7f9563fff640 1 --2- 172.21.15.76:0/2739871223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f956406d9c0 0x7f956415ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.322+0000 7f9563fff640 1 -- 172.21.15.76:0/2739871223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f956406c5f0 msgr2=0x7f95640c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:49.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.322+0000 7f95637fe640 1 --2- 172.21.15.76:0/2739871223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f956406c5f0 0x7f95640c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.322+0000 7f9563fff640 1 --2- 172.21.15.76:0/2739871223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f956406c5f0 0x7f95640c9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.322+0000 7f9563fff640 1 -- 172.21.15.76:0/2739871223 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9554009400 con 0x7f95640ade00 2023-11-09T03:55:49.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.322+0000 7f9563fff640 1 --2- 172.21.15.76:0/2739871223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95640ade00 0x7f95640ac8d0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f955c007e60 tx=0x7f955c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:49.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.322+0000 7f954bfff640 1 -- 172.21.15.76:0/2739871223 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c014420 con 0x7f95640ade00 2023-11-09T03:55:49.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.323+0000 7f954bfff640 1 -- 172.21.15.76:0/2739871223 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955c0145a0 con 0x7f95640ade00 2023-11-09T03:55:49.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.323+0000 7f956a07c640 1 -- 172.21.15.76:0/2739871223 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95640c9b50 con 0x7f95640ade00 2023-11-09T03:55:49.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.323+0000 7f954bfff640 1 -- 172.21.15.76:0/2739871223 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c014950 con 0x7f95640ade00 2023-11-09T03:55:49.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.323+0000 7f956a07c640 1 -- 172.21.15.76:0/2739871223 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95640ca110 con 0x7f95640ade00 2023-11-09T03:55:49.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.324+0000 7f956a07c640 1 -- 172.21.15.76:0/2739871223 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95640b1590 con 0x7f95640ade00 2023-11-09T03:55:49.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.324+0000 7f954bfff640 1 -- 172.21.15.76:0/2739871223 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f955c014ab0 con 0x7f95640ade00 2023-11-09T03:55:49.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.324+0000 7f954bfff640 1 --2- 172.21.15.76:0/2739871223 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f955006ae70 0x7f955006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.324+0000 7f954bfff640 1 -- 172.21.15.76:0/2739871223 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8554+0+0 (secure 0 0 0) 0x7f955c08aa50 con 0x7f95640ade00 2023-11-09T03:55:49.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.324+0000 7f95637fe640 1 --2- 172.21.15.76:0/2739871223 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f955006ae70 0x7f955006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.325+0000 7f95637fe640 1 --2- 172.21.15.76:0/2739871223 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f955006ae70 0x7f955006d330 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f9554035000 tx=0x7f9554008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:49.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.329+0000 7f954bfff640 1 -- 172.21.15.76:0/2739871223 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f95640b1590 con 0x7f95640ade00 2023-11-09T03:55:49.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.533+0000 7f956a07c640 1 -- 172.21.15.76:0/2739871223 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95640b1590 con 0x7f95640ade00 2023-11-09T03:55:49.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.534+0000 7f954bfff640 1 -- 172.21.15.76:0/2739871223 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+4935 (secure 0 0 0) 0x7f95640b1590 con 0x7f95640ade00 2023-11-09T03:55:49.535 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:55:49.535 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":141,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":9525,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1713748386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1713748386},{"type":"v1","addr":"172.21.15.76:6835","nonce":1713748386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":138}],"filesystems":[{"mdsmap":{"epoch":141,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:55:41.732323+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":6698},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6698":{"gid":6698,"name":"b","rank":1,"incarnation":137,"state":"up:active","state_seq":18,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:55:49.535 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 141 2023-11-09T03:55:49.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.536+0000 7f956a07c640 1 -- 172.21.15.76:0/2739871223 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f955006ae70 msgr2=0x7f955006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:49.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.536+0000 7f956a07c640 1 --2- 172.21.15.76:0/2739871223 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f955006ae70 0x7f955006d330 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f9554035000 tx=0x7f9554008030 comp rx=0 tx=0).stop 2023-11-09T03:55:49.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.536+0000 7f956a07c640 1 -- 172.21.15.76:0/2739871223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95640ade00 msgr2=0x7f95640ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:49.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.536+0000 7f956a07c640 1 --2- 172.21.15.76:0/2739871223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95640ade00 0x7f95640ac8d0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f955c007e60 tx=0x7f955c00e4b0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.536+0000 7f956a07c640 1 -- 172.21.15.76:0/2739871223 shutdown_connections 2023-11-09T03:55:49.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.536+0000 7f956a07c640 1 --2- 172.21.15.76:0/2739871223 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f955006ae70 0x7f955006d330 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.536+0000 7f956a07c640 1 --2- 172.21.15.76:0/2739871223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95640ade00 0x7f95640ac8d0 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.536+0000 7f956a07c640 1 --2- 172.21.15.76:0/2739871223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f956406d9c0 0x7f956415ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.536+0000 7f956a07c640 1 --2- 172.21.15.76:0/2739871223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f956406c5f0 0x7f95640c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.536+0000 7f956a07c640 1 -- 172.21.15.76:0/2739871223 >> 172.21.15.76:0/2739871223 conn(0x7f95640cb530 msgr2=0x7f95640cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:49.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.536+0000 7f956a07c640 1 -- 172.21.15.76:0/2739871223 shutdown_connections 2023-11-09T03:55:49.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.536+0000 7f956a07c640 1 -- 172.21.15.76:0/2739871223 wait complete. 2023-11-09T03:55:49.548 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2023-11-09T03:55:49.663 INFO:teuthology.orchestra.run.smithi076.stdout:{"last_snap":19,"last_created":17,"last_destroyed":19,"pending_noop":[],"snaps":[],"need_to_purge":{"17":[16,17,18,19]},"pending_update":[],"pending_destroy":[]} 2023-11-09T03:55:49.664 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 17, "last_destroyed": 19, "last_snap": 19, "need_to_purge": { "17": [ 16, 17, 18, 19 ] }, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [] } 2023-11-09T03:55:49.664 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:55:49.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.767+0000 7f784cfb6640 1 Processor -- start 2023-11-09T03:55:49.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.767+0000 7f784cfb6640 1 -- start start 2023-11-09T03:55:49.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.768+0000 7f784cfb6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f784815d4e0 0x7f78480d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.768+0000 7f784cfb6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f78480d3b60 0x7f78480d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.768+0000 7f784cfb6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78480b32b0 con 0x7f784815d100 2023-11-09T03:55:49.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.768+0000 7f784cfb6640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78480cec20 con 0x7f78480d3b60 2023-11-09T03:55:49.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.768+0000 7f784cfb6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78480cee10 con 0x7f784815d4e0 2023-11-09T03:55:49.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.768+0000 7f7846d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f78480d3b60 0x7f78480d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.768+0000 7f7846d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f78480d3b60 0x7f78480d1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49866/0 (socket says 172.21.15.76:49866) 2023-11-09T03:55:49.772 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.768+0000 7f7846d9d640 1 -- 172.21.15.76:0/48812858 learned_addr learned my addr 172.21.15.76:0/48812858 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:49.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.768+0000 7f783dd9b640 1 --2- 172.21.15.76:0/48812858 >> v2:172.21.15.113:3300/0 conn(0x7f784815d4e0 0x7f78480d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.768+0000 7f7846d9d640 1 -- 172.21.15.76:0/48812858 >> v2:172.21.15.113:3300/0 conn(0x7f784815d4e0 msgr2=0x7f78480d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:49.773 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.768+0000 7f7846d9d640 1 --2- 172.21.15.76:0/48812858 >> v2:172.21.15.113:3300/0 conn(0x7f784815d4e0 0x7f78480d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.768+0000 7f7846d9d640 1 -- 172.21.15.76:0/48812858 >> v1:172.21.15.113:6789/0 conn(0x7f784815d100 legacy=0x7f78480ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:49.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.768+0000 7f7846d9d640 1 -- 172.21.15.76:0/48812858 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78480b4750 con 0x7f78480d3b60 2023-11-09T03:55:49.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.768+0000 7f783dd9b640 1 --2- 172.21.15.76:0/48812858 >> v2:172.21.15.113:3300/0 conn(0x7f784815d4e0 0x7f78480d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:55:49.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.769+0000 7f7846d9d640 1 --2- 172.21.15.76:0/48812858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78480d3b60 0x7f78480d1a10 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f7848160da0 tx=0x7f784002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3af294372c22d1a server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:49.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.769+0000 7f7845d9b640 1 -- 172.21.15.76:0/48812858 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784002cb20 con 0x7f78480d3b60 2023-11-09T03:55:49.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.769+0000 7f7845d9b640 1 -- 172.21.15.76:0/48812858 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784002cc80 con 0x7f78480d3b60 2023-11-09T03:55:49.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.769+0000 7f7845d9b640 1 -- 172.21.15.76:0/48812858 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78400343f0 con 0x7f78480d3b60 2023-11-09T03:55:49.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.769+0000 7f784cfb6640 1 -- 172.21.15.76:0/48812858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78480d3b60 msgr2=0x7f78480d1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:49.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.769+0000 7f784cfb6640 1 --2- 172.21.15.76:0/48812858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78480d3b60 0x7f78480d1a10 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f7848160da0 tx=0x7f784002bf80 comp rx=0 tx=0).stop 2023-11-09T03:55:49.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.769+0000 7f784cfb6640 1 -- 172.21.15.76:0/48812858 shutdown_connections 2023-11-09T03:55:49.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.769+0000 7f784cfb6640 1 --2- 172.21.15.76:0/48812858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78480d3b60 0x7f78480d1a10 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.769+0000 7f784cfb6640 1 --2- 172.21.15.76:0/48812858 >> v2:172.21.15.113:3300/0 conn(0x7f784815d4e0 0x7f78480d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.769+0000 7f784cfb6640 1 -- 172.21.15.76:0/48812858 >> 172.21.15.76:0/48812858 conn(0x7f78480d2ef0 msgr2=0x7f78480d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:49.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.769+0000 7f784cfb6640 1 -- 172.21.15.76:0/48812858 shutdown_connections 2023-11-09T03:55:49.777 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.769+0000 7f784cfb6640 1 -- 172.21.15.76:0/48812858 wait complete. 2023-11-09T03:55:49.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.771+0000 7f784cfb6640 1 Processor -- start 2023-11-09T03:55:49.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.771+0000 7f784cfb6640 1 -- start start 2023-11-09T03:55:49.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.772+0000 7f784cfb6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78480d3b60 0x7f78481f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.780 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.772+0000 7f784cfb6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f784815d100 0x7f784815b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.772+0000 7f784cfb6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f784815d4e0 0x7f78480b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.772+0000 7f784cfb6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78480cb2f0 con 0x7f784815d100 2023-11-09T03:55:49.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.772+0000 7f784cfb6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78481f3800 con 0x7f784815d4e0 2023-11-09T03:55:49.781 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.772+0000 7f784cfb6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78480b5a70 con 0x7f78480d3b60 2023-11-09T03:55:49.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.772+0000 7f783dd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f784815d100 0x7f784815b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.772+0000 7f783dd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f784815d100 0x7f784815b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49882/0 (socket says 172.21.15.76:49882) 2023-11-09T03:55:49.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.772+0000 7f783dd9b640 1 -- 172.21.15.76:0/3957847417 learned_addr learned my addr 172.21.15.76:0/3957847417 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:49.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.772+0000 7f7846d9d640 1 --2- 172.21.15.76:0/3957847417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f784815d4e0 0x7f78480b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.782 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.772+0000 7f784659c640 1 --2- 172.21.15.76:0/3957847417 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78480d3b60 0x7f78481f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.772+0000 7f783dd9b640 1 -- 172.21.15.76:0/3957847417 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78480d3b60 msgr2=0x7f78481f32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:49.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.772+0000 7f783dd9b640 1 --2- 172.21.15.76:0/3957847417 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78480d3b60 0x7f78481f32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.772+0000 7f783dd9b640 1 -- 172.21.15.76:0/3957847417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f784815d4e0 msgr2=0x7f78480b64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:49.783 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.772+0000 7f783dd9b640 1 --2- 172.21.15.76:0/3957847417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f784815d4e0 0x7f78480b64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.772+0000 7f783dd9b640 1 -- 172.21.15.76:0/3957847417 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7840009b30 con 0x7f784815d100 2023-11-09T03:55:49.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.772+0000 7f783dd9b640 1 --2- 172.21.15.76:0/3957847417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f784815d100 0x7f784815b380 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f783000d800 tx=0x7f783000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:49.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.772+0000 7f783f7fe640 1 -- 172.21.15.76:0/3957847417 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7830004030 con 0x7f784815d100 2023-11-09T03:55:49.784 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.772+0000 7f783f7fe640 1 -- 172.21.15.76:0/3957847417 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7830004190 con 0x7f784815d100 2023-11-09T03:55:49.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.772+0000 7f784cfb6640 1 -- 172.21.15.76:0/3957847417 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78480b5d10 con 0x7f784815d100 2023-11-09T03:55:49.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.772+0000 7f783f7fe640 1 -- 172.21.15.76:0/3957847417 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7830004520 con 0x7f784815d100 2023-11-09T03:55:49.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.772+0000 7f784cfb6640 1 -- 172.21.15.76:0/3957847417 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78480cb500 con 0x7f784815d100 2023-11-09T03:55:49.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.774+0000 7f784cfb6640 1 -- 172.21.15.76:0/3957847417 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78480684b0 con 0x7f784815d100 2023-11-09T03:55:49.785 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.774+0000 7f783f7fe640 1 -- 172.21.15.76:0/3957847417 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f78300096a0 con 0x7f784815d100 2023-11-09T03:55:49.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.774+0000 7f783f7fe640 1 --2- 172.21.15.76:0/3957847417 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f782806ae70 0x7f782806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:49.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.774+0000 7f783f7fe640 1 -- 172.21.15.76:0/3957847417 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8642+0+0 (secure 0 0 0) 0x7f783008a260 con 0x7f784815d100 2023-11-09T03:55:49.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.775+0000 7f784659c640 1 --2- 172.21.15.76:0/3957847417 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f782806ae70 0x7f782806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:49.786 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.775+0000 7f784659c640 1 --2- 172.21.15.76:0/3957847417 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f782806ae70 0x7f782806d330 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f78480a0d10 tx=0x7f7838009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:49.787 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.779+0000 7f783f7fe640 1 -- 172.21.15.76:0/3957847417 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f78300547f0 con 0x7f784815d100 2023-11-09T03:55:49.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.981+0000 7f784cfb6640 1 -- 172.21.15.76:0/3957847417 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78480d2cd0 con 0x7f784815d100 2023-11-09T03:55:49.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.982+0000 7f783f7fe640 1 -- 172.21.15.76:0/3957847417 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+4935 (secure 0 0 0) 0x7f7830057e10 con 0x7f784815d100 2023-11-09T03:55:49.983 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:55:49.984 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":141,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":9525,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1713748386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1713748386},{"type":"v1","addr":"172.21.15.76:6835","nonce":1713748386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":138}],"filesystems":[{"mdsmap":{"epoch":141,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:55:41.732323+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":6698},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6698":{"gid":6698,"name":"b","rank":1,"incarnation":137,"state":"up:active","state_seq":18,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:55:49.985 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 141 2023-11-09T03:55:49.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.984+0000 7f784cfb6640 1 -- 172.21.15.76:0/3957847417 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f782806ae70 msgr2=0x7f782806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:49.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.984+0000 7f784cfb6640 1 --2- 172.21.15.76:0/3957847417 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f782806ae70 0x7f782806d330 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f78480a0d10 tx=0x7f7838009040 comp rx=0 tx=0).stop 2023-11-09T03:55:49.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.984+0000 7f784cfb6640 1 -- 172.21.15.76:0/3957847417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f784815d100 msgr2=0x7f784815b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:49.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.984+0000 7f784cfb6640 1 --2- 172.21.15.76:0/3957847417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f784815d100 0x7f784815b380 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f783000d800 tx=0x7f783000dcc0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.984+0000 7f784cfb6640 1 -- 172.21.15.76:0/3957847417 shutdown_connections 2023-11-09T03:55:49.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.984+0000 7f784cfb6640 1 --2- 172.21.15.76:0/3957847417 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f782806ae70 0x7f782806d330 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.984+0000 7f784cfb6640 1 --2- 172.21.15.76:0/3957847417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f784815d4e0 0x7f78480b64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.984+0000 7f784cfb6640 1 --2- 172.21.15.76:0/3957847417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f784815d100 0x7f784815b380 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.984+0000 7f784cfb6640 1 --2- 172.21.15.76:0/3957847417 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78480d3b60 0x7f78481f32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:49.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.984+0000 7f784cfb6640 1 -- 172.21.15.76:0/3957847417 >> 172.21.15.76:0/3957847417 conn(0x7f78480d2ef0 msgr2=0x7f784806bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:49.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.984+0000 7f784cfb6640 1 -- 172.21.15.76:0/3957847417 shutdown_connections 2023-11-09T03:55:49.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:49.984+0000 7f784cfb6640 1 -- 172.21.15.76:0/3957847417 wait complete. 2023-11-09T03:55:49.996 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:0 true 2023-11-09T03:55:50.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.096+0000 7fd643bcc640 1 Processor -- start 2023-11-09T03:55:50.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.097+0000 7fd643bcc640 1 -- start start 2023-11-09T03:55:50.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.097+0000 7fd643bcc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd63c0addd0 0x7fd63c0cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:50.101 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.097+0000 7fd643bcc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd63c0cc680 0x7fd63c0ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:50.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.097+0000 7fd643bcc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd63c06bb10 0x7fd63c06bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:50.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.097+0000 7fd643bcc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd63c06c4c0 con 0x7fd63c06bb10 2023-11-09T03:55:50.102 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.097+0000 7fd643bcc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd63c0cd720 con 0x7fd63c0addd0 2023-11-09T03:55:50.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.097+0000 7fd643bcc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c0ae1b0 con 0x7fd63c0cc680 2023-11-09T03:55:50.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.098+0000 7fd642169640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd63c06bb10 0x7fd63c06bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:50.103 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.098+0000 7fd642169640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd63c06bb10 0x7fd63c06bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49890/0 (socket says 172.21.15.76:49890) 2023-11-09T03:55:50.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.098+0000 7fd642169640 1 -- 172.21.15.76:0/727896897 learned_addr learned my addr 172.21.15.76:0/727896897 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:50.104 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.098+0000 7fd642169640 1 -- 172.21.15.76:0/727896897 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd63c0cc680 msgr2=0x7fd63c0ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:50.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.098+0000 7fd641167640 1 --2- 172.21.15.76:0/727896897 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd63c0cc680 0x7fd63c0ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:50.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.098+0000 7fd641968640 1 --2- 172.21.15.76:0/727896897 >> v2:172.21.15.113:3300/0 conn(0x7fd63c0addd0 0x7fd63c0cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:50.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.098+0000 7fd642169640 1 --2- 172.21.15.76:0/727896897 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd63c0cc680 0x7fd63c0ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:50.105 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.098+0000 7fd642169640 1 -- 172.21.15.76:0/727896897 >> v2:172.21.15.113:3300/0 conn(0x7fd63c0addd0 msgr2=0x7fd63c0cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:50.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.098+0000 7fd642169640 1 --2- 172.21.15.76:0/727896897 >> v2:172.21.15.113:3300/0 conn(0x7fd63c0addd0 0x7fd63c0cdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:50.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.098+0000 7fd642169640 1 -- 172.21.15.76:0/727896897 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c0ccb10 con 0x7fd63c06bb10 2023-11-09T03:55:50.106 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.098+0000 7fd641167640 1 --2- 172.21.15.76:0/727896897 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd63c0cc680 0x7fd63c0ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:55:50.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.098+0000 7fd641968640 1 --2- 172.21.15.76:0/727896897 >> v2:172.21.15.113:3300/0 conn(0x7fd63c0addd0 0x7fd63c0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:55:50.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.099+0000 7fd642169640 1 --2- 172.21.15.76:0/727896897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c06bb10 0x7fd63c06bf80 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0cc630 tx=0x7fd63402cfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecaf28738c5890cd server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:50.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.099+0000 7fd640966640 1 -- 172.21.15.76:0/727896897 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63402d980 con 0x7fd63c06bb10 2023-11-09T03:55:50.107 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.099+0000 7fd640966640 1 -- 172.21.15.76:0/727896897 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63402dae0 con 0x7fd63c06bb10 2023-11-09T03:55:50.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.099+0000 7fd640966640 1 -- 172.21.15.76:0/727896897 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63402de70 con 0x7fd63c06bb10 2023-11-09T03:55:50.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.099+0000 7fd643bcc640 1 -- 172.21.15.76:0/727896897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c06bb10 msgr2=0x7fd63c06bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:50.108 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.099+0000 7fd643bcc640 1 --2- 172.21.15.76:0/727896897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c06bb10 0x7fd63c06bf80 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0cc630 tx=0x7fd63402cfe0 comp rx=0 tx=0).stop 2023-11-09T03:55:50.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.099+0000 7fd643bcc640 1 -- 172.21.15.76:0/727896897 shutdown_connections 2023-11-09T03:55:50.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.099+0000 7fd643bcc640 1 --2- 172.21.15.76:0/727896897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c06bb10 0x7fd63c06bf80 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:50.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.099+0000 7fd643bcc640 1 --2- 172.21.15.76:0/727896897 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd63c0cc680 0x7fd63c0ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:50.109 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.099+0000 7fd643bcc640 1 --2- 172.21.15.76:0/727896897 >> v2:172.21.15.113:3300/0 conn(0x7fd63c0addd0 0x7fd63c0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:50.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.099+0000 7fd643bcc640 1 -- 172.21.15.76:0/727896897 >> 172.21.15.76:0/727896897 conn(0x7fd63c0cb530 msgr2=0x7fd63c0cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:50.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.099+0000 7fd643bcc640 1 -- 172.21.15.76:0/727896897 shutdown_connections 2023-11-09T03:55:50.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.099+0000 7fd643bcc640 1 -- 172.21.15.76:0/727896897 wait complete. 2023-11-09T03:55:50.110 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.100+0000 7fd643bcc640 1 Processor -- start 2023-11-09T03:55:50.111 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.100+0000 7fd643bcc640 1 -- start start 2023-11-09T03:55:50.113 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.101+0000 7fd643bcc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd63c06bb10 0x7fd63c15ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:50.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.101+0000 7fd643bcc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd63c0addd0 0x7fd63c13d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:50.114 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.101+0000 7fd643bcc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c0c8c40 0x7fd63c13de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:50.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.101+0000 7fd643bcc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c0c4f00 con 0x7fd63c0c8c40 2023-11-09T03:55:50.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.101+0000 7fd643bcc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c151260 con 0x7fd63c0addd0 2023-11-09T03:55:50.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.101+0000 7fd643bcc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c15a990 con 0x7fd63c06bb10 2023-11-09T03:55:50.115 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.101+0000 7fd642169640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c0c8c40 0x7fd63c13de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:50.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.101+0000 7fd642169640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c0c8c40 0x7fd63c13de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49906/0 (socket says 172.21.15.76:49906) 2023-11-09T03:55:50.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.101+0000 7fd642169640 1 -- 172.21.15.76:0/3922157037 learned_addr learned my addr 172.21.15.76:0/3922157037 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:50.116 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.101+0000 7fd642169640 1 -- 172.21.15.76:0/3922157037 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd63c06bb10 msgr2=0x7fd63c15ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:50.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.101+0000 7fd642169640 1 --2- 172.21.15.76:0/3922157037 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd63c06bb10 0x7fd63c15ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:50.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.102+0000 7fd642169640 1 -- 172.21.15.76:0/3922157037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd63c0addd0 msgr2=0x7fd63c13d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:50.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.102+0000 7fd642169640 1 --2- 172.21.15.76:0/3922157037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd63c0addd0 0x7fd63c13d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:50.117 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.102+0000 7fd642169640 1 -- 172.21.15.76:0/3922157037 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63400ac40 con 0x7fd63c0c8c40 2023-11-09T03:55:50.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.102+0000 7fd642169640 1 --2- 172.21.15.76:0/3922157037 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c0c8c40 0x7fd63c13de50 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7fd634037040 tx=0x7fd634004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:50.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.102+0000 7fd6327fc640 1 -- 172.21.15.76:0/3922157037 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634009cd0 con 0x7fd63c0c8c40 2023-11-09T03:55:50.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.102+0000 7fd6327fc640 1 -- 172.21.15.76:0/3922157037 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd634009e30 con 0x7fd63c0c8c40 2023-11-09T03:55:50.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.102+0000 7fd643bcc640 1 -- 172.21.15.76:0/3922157037 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63c15ac70 con 0x7fd63c0c8c40 2023-11-09T03:55:50.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.102+0000 7fd6327fc640 1 -- 172.21.15.76:0/3922157037 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6340406b0 con 0x7fd63c0c8c40 2023-11-09T03:55:50.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.102+0000 7fd643bcc640 1 -- 172.21.15.76:0/3922157037 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63c15b230 con 0x7fd63c0c8c40 2023-11-09T03:55:50.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.103+0000 7fd643bcc640 1 -- 172.21.15.76:0/3922157037 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd63c0684b0 con 0x7fd63c0c8c40 2023-11-09T03:55:50.119 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.103+0000 7fd6327fc640 1 -- 172.21.15.76:0/3922157037 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd634009020 con 0x7fd63c0c8c40 2023-11-09T03:55:50.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.104+0000 7fd6327fc640 1 --2- 172.21.15.76:0/3922157037 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd62006ae70 0x7fd62006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:50.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.104+0000 7fd6327fc640 1 -- 172.21.15.76:0/3922157037 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8642+0+0 (secure 0 0 0) 0x7fd6340ad250 con 0x7fd63c0c8c40 2023-11-09T03:55:50.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.104+0000 7fd641968640 1 --2- 172.21.15.76:0/3922157037 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd62006ae70 0x7fd62006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:50.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.104+0000 7fd641968640 1 --2- 172.21.15.76:0/3922157037 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd62006ae70 0x7fd62006d330 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0c64a0 tx=0x7fd62c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:50.121 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.108+0000 7fd6327fc640 1 -- 172.21.15.76:0/3922157037 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd634077900 con 0x7fd63c0c8c40 2023-11-09T03:55:50.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.341+0000 7fd643bcc640 1 -- 172.21.15.76:0/3922157037 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"} v 0) v1 -- 0x7fd63c0b4a30 con 0x7fd63c0c8c40 2023-11-09T03:55:50.764 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.763+0000 7fd6327fc640 1 -- 172.21.15.76:0/3922157037 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"}]=0 freezing mds.8568 v142) v1 ==== 112+0+0 (secure 0 0 0) 0x7fd634044020 con 0x7fd63c0c8c40 2023-11-09T03:55:50.764 INFO:teuthology.orchestra.run.smithi076.stderr:freezing mds.8568 2023-11-09T03:55:50.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.765+0000 7fd643bcc640 1 -- 172.21.15.76:0/3922157037 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd62006ae70 msgr2=0x7fd62006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:50.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.765+0000 7fd643bcc640 1 --2- 172.21.15.76:0/3922157037 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd62006ae70 0x7fd62006d330 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0c64a0 tx=0x7fd62c008040 comp rx=0 tx=0).stop 2023-11-09T03:55:50.767 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.765+0000 7fd643bcc640 1 -- 172.21.15.76:0/3922157037 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c0c8c40 msgr2=0x7fd63c13de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:50.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.765+0000 7fd643bcc640 1 --2- 172.21.15.76:0/3922157037 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c0c8c40 0x7fd63c13de50 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7fd634037040 tx=0x7fd634004a40 comp rx=0 tx=0).stop 2023-11-09T03:55:50.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.765+0000 7fd643bcc640 1 -- 172.21.15.76:0/3922157037 shutdown_connections 2023-11-09T03:55:50.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.765+0000 7fd643bcc640 1 --2- 172.21.15.76:0/3922157037 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fd62006ae70 0x7fd62006d330 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:50.768 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.765+0000 7fd643bcc640 1 --2- 172.21.15.76:0/3922157037 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c0c8c40 0x7fd63c13de50 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:50.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.765+0000 7fd643bcc640 1 --2- 172.21.15.76:0/3922157037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd63c0addd0 0x7fd63c13d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:50.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.765+0000 7fd643bcc640 1 --2- 172.21.15.76:0/3922157037 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd63c06bb10 0x7fd63c15ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:50.769 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.765+0000 7fd643bcc640 1 -- 172.21.15.76:0/3922157037 >> 172.21.15.76:0/3922157037 conn(0x7fd63c0cb530 msgr2=0x7fd63c0c9620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:50.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.765+0000 7fd643bcc640 1 -- 172.21.15.76:0/3922157037 shutdown_connections 2023-11-09T03:55:50.770 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.765+0000 7fd643bcc640 1 -- 172.21.15.76:0/3922157037 wait complete. 2023-11-09T03:55:50.777 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:1 true 2023-11-09T03:55:50.882 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.881+0000 7f4e0e3c9640 1 Processor -- start 2023-11-09T03:55:50.883 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.882+0000 7f4e0e3c9640 1 -- start start 2023-11-09T03:55:50.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.882+0000 7f4e0e3c9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4e0815b200 0x7f4e080ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:50.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.882+0000 7f4e0e3c9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4e080afda0 0x7f4e080cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:50.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.882+0000 7f4e0e3c9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e080cdc20 con 0x7f4e080cd7f0 2023-11-09T03:55:50.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.882+0000 7f4e0e3c9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e080cde80 con 0x7f4e080afda0 2023-11-09T03:55:50.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.882+0000 7f4e0e3c9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e080cf670 con 0x7f4e0815b200 2023-11-09T03:55:50.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.882+0000 7f4e077fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4e080afda0 0x7f4e080cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:50.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.882+0000 7f4e0c966640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f4e080cd7f0 0x7f4e080cb840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37724/0 (socket says 172.21.15.76:37724) 2023-11-09T03:55:50.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.883+0000 7f4e0c966640 1 -- 172.21.15.76:0/2748153302 learned_addr learned my addr 172.21.15.76:0/2748153302 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:55:50.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.883+0000 7f4e077fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4e080afda0 0x7f4e080cd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49918/0 (socket says 172.21.15.76:49918) 2023-11-09T03:55:50.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.883+0000 7f4e07fff640 1 --2- 172.21.15.76:0/2748153302 >> v2:172.21.15.113:3300/0 conn(0x7f4e0815b200 0x7f4e080ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:50.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.883+0000 7f4e077fe640 1 -- 172.21.15.76:0/2748153302 >> v2:172.21.15.113:3300/0 conn(0x7f4e0815b200 msgr2=0x7f4e080ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:50.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.883+0000 7f4e077fe640 1 --2- 172.21.15.76:0/2748153302 >> v2:172.21.15.113:3300/0 conn(0x7f4e0815b200 0x7f4e080ceb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:50.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.883+0000 7f4e077fe640 1 -- 172.21.15.76:0/2748153302 >> v1:172.21.15.76:6789/0 conn(0x7f4e080cd7f0 legacy=0x7f4e080cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:50.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.883+0000 7f4e077fe640 1 -- 172.21.15.76:0/2748153302 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e080cf950 con 0x7f4e080afda0 2023-11-09T03:55:50.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.883+0000 7f4e06ffd640 1 -- 172.21.15.76:0/2748153302 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2141888428 0 0) 0x7f4e080cdc20 con 0x7f4e080cd7f0 2023-11-09T03:55:50.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.884+0000 7f4e077fe640 1 --2- 172.21.15.76:0/2748153302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e080afda0 0x7f4e080cd2b0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f4e08070130 tx=0x7f4dfc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe7539418a595fe server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:50.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.884+0000 7f4e06ffd640 1 -- 172.21.15.76:0/2748153302 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc02e070 con 0x7f4e080afda0 2023-11-09T03:55:50.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.884+0000 7f4e06ffd640 1 -- 172.21.15.76:0/2748153302 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dfc0315a0 con 0x7f4e080afda0 2023-11-09T03:55:50.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.884+0000 7f4e06ffd640 1 -- 172.21.15.76:0/2748153302 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc031930 con 0x7f4e080afda0 2023-11-09T03:55:50.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.884+0000 7f4e0e3c9640 1 -- 172.21.15.76:0/2748153302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e080afda0 msgr2=0x7f4e080cd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:50.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.884+0000 7f4e0e3c9640 1 --2- 172.21.15.76:0/2748153302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e080afda0 0x7f4e080cd2b0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f4e08070130 tx=0x7f4dfc02b840 comp rx=0 tx=0).stop 2023-11-09T03:55:50.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.884+0000 7f4e0e3c9640 1 -- 172.21.15.76:0/2748153302 shutdown_connections 2023-11-09T03:55:50.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.884+0000 7f4e0e3c9640 1 --2- 172.21.15.76:0/2748153302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e080afda0 0x7f4e080cd2b0 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:50.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.884+0000 7f4e0e3c9640 1 --2- 172.21.15.76:0/2748153302 >> v2:172.21.15.113:3300/0 conn(0x7f4e0815b200 0x7f4e080ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:50.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.884+0000 7f4e0e3c9640 1 -- 172.21.15.76:0/2748153302 >> 172.21.15.76:0/2748153302 conn(0x7f4e080d0c30 msgr2=0x7f4e080d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:50.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.884+0000 7f4e0e3c9640 1 -- 172.21.15.76:0/2748153302 shutdown_connections 2023-11-09T03:55:50.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.884+0000 7f4e0e3c9640 1 -- 172.21.15.76:0/2748153302 wait complete. 2023-11-09T03:55:50.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.885+0000 7f4e0e3c9640 1 Processor -- start 2023-11-09T03:55:50.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.886+0000 7f4e0e3c9640 1 -- start start 2023-11-09T03:55:50.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.886+0000 7f4e0e3c9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e080afda0 0x7f4e080ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:50.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.886+0000 7f4e0e3c9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e080cd7f0 0x7f4e081439b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:50.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.886+0000 7f4e0e3c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e0815b200 0x7f4e08143ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:50.897 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.886+0000 7f4e0e3c9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e08144430 con 0x7f4e0815b200 2023-11-09T03:55:50.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.886+0000 7f4e0e3c9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e08144730 con 0x7f4e080afda0 2023-11-09T03:55:50.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.886+0000 7f4e0e3c9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e081e5500 con 0x7f4e080cd7f0 2023-11-09T03:55:50.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.886+0000 7f4e0c966640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e0815b200 0x7f4e08143ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:50.898 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.887+0000 7f4e0c966640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e0815b200 0x7f4e08143ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49928/0 (socket says 172.21.15.76:49928) 2023-11-09T03:55:50.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.887+0000 7f4e0c966640 1 -- 172.21.15.76:0/2430501780 learned_addr learned my addr 172.21.15.76:0/2430501780 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:50.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.887+0000 7f4e07fff640 1 --2- 172.21.15.76:0/2430501780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e080afda0 0x7f4e080ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:50.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.887+0000 7f4e077fe640 1 --2- 172.21.15.76:0/2430501780 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e080cd7f0 0x7f4e081439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:50.899 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.887+0000 7f4e0c966640 1 -- 172.21.15.76:0/2430501780 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e080cd7f0 msgr2=0x7f4e081439b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:50.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.887+0000 7f4e0c966640 1 --2- 172.21.15.76:0/2430501780 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e080cd7f0 0x7f4e081439b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:50.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.887+0000 7f4e0c966640 1 -- 172.21.15.76:0/2430501780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e080afda0 msgr2=0x7f4e080ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:50.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.887+0000 7f4e0c966640 1 --2- 172.21.15.76:0/2430501780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e080afda0 0x7f4e080ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:50.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.887+0000 7f4e0c966640 1 -- 172.21.15.76:0/2430501780 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dfc009400 con 0x7f4e0815b200 2023-11-09T03:55:50.900 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.887+0000 7f4e07fff640 1 --2- 172.21.15.76:0/2430501780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e080afda0 0x7f4e080ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:55:50.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.887+0000 7f4e0c966640 1 --2- 172.21.15.76:0/2430501780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e0815b200 0x7f4e08143ef0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000cb60 tx=0x7f4e0000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:50.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.887+0000 7f4e04ff9640 1 -- 172.21.15.76:0/2430501780 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0000ad40 con 0x7f4e0815b200 2023-11-09T03:55:50.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.887+0000 7f4e04ff9640 1 -- 172.21.15.76:0/2430501780 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e00012400 con 0x7f4e0815b200 2023-11-09T03:55:50.901 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.887+0000 7f4e0e3c9640 1 -- 172.21.15.76:0/2430501780 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e081449a0 con 0x7f4e0815b200 2023-11-09T03:55:50.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.887+0000 7f4e04ff9640 1 -- 172.21.15.76:0/2430501780 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e000127b0 con 0x7f4e0815b200 2023-11-09T03:55:50.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.887+0000 7f4e0e3c9640 1 -- 172.21.15.76:0/2430501780 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e08144d30 con 0x7f4e0815b200 2023-11-09T03:55:50.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.888+0000 7f4e0e3c9640 1 -- 172.21.15.76:0/2430501780 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e080684b0 con 0x7f4e0815b200 2023-11-09T03:55:50.902 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.889+0000 7f4e04ff9640 1 -- 172.21.15.76:0/2430501780 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4e00007070 con 0x7f4e0815b200 2023-11-09T03:55:50.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.889+0000 7f4e04ff9640 1 --2- 172.21.15.76:0/2430501780 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4df406ae70 0x7f4df406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:50.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.889+0000 7f4e04ff9640 1 -- 172.21.15.76:0/2430501780 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8626+0+0 (secure 0 0 0) 0x7f4e0008a2b0 con 0x7f4e0815b200 2023-11-09T03:55:50.903 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.889+0000 7f4e07fff640 1 --2- 172.21.15.76:0/2430501780 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4df406ae70 0x7f4df406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:50.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.890+0000 7f4e07fff640 1 --2- 172.21.15.76:0/2430501780 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4df406ae70 0x7f4df406d330 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4df8009780 tx=0x7f4df8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:50.904 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:50.893+0000 7f4e04ff9640 1 -- 172.21.15.76:0/2430501780 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4e00054710 con 0x7f4e0815b200 2023-11-09T03:55:51.131 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:51.130+0000 7f4e0e3c9640 1 -- 172.21.15.76:0/2430501780 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"} v 0) v1 -- 0x7f4e080c4490 con 0x7f4e0815b200 2023-11-09T03:55:51.771 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:51.770+0000 7f4e04ff9640 1 -- 172.21.15.76:0/2430501780 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"}]=0 freezing mds.6698 v143) v1 ==== 112+0+0 (secure 0 0 0) 0x7f4e00017070 con 0x7f4e0815b200 2023-11-09T03:55:51.771 INFO:teuthology.orchestra.run.smithi076.stderr:freezing mds.6698 2023-11-09T03:55:51.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:51.772+0000 7f4e0e3c9640 1 -- 172.21.15.76:0/2430501780 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4df406ae70 msgr2=0x7f4df406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:51.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:51.772+0000 7f4e0e3c9640 1 --2- 172.21.15.76:0/2430501780 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4df406ae70 0x7f4df406d330 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4df8009780 tx=0x7f4df8008040 comp rx=0 tx=0).stop 2023-11-09T03:55:51.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:51.772+0000 7f4e0e3c9640 1 -- 172.21.15.76:0/2430501780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e0815b200 msgr2=0x7f4e08143ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:51.774 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:51.772+0000 7f4e0e3c9640 1 --2- 172.21.15.76:0/2430501780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e0815b200 0x7f4e08143ef0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000cb60 tx=0x7f4e0000a430 comp rx=0 tx=0).stop 2023-11-09T03:55:51.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:51.772+0000 7f4e0e3c9640 1 -- 172.21.15.76:0/2430501780 shutdown_connections 2023-11-09T03:55:51.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:51.772+0000 7f4e0e3c9640 1 --2- 172.21.15.76:0/2430501780 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f4df406ae70 0x7f4df406d330 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:51.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:51.772+0000 7f4e0e3c9640 1 --2- 172.21.15.76:0/2430501780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e0815b200 0x7f4e08143ef0 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:51.775 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:51.772+0000 7f4e0e3c9640 1 --2- 172.21.15.76:0/2430501780 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e080cd7f0 0x7f4e081439b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:51.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:51.772+0000 7f4e0e3c9640 1 --2- 172.21.15.76:0/2430501780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e080afda0 0x7f4e080ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:51.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:51.772+0000 7f4e0e3c9640 1 -- 172.21.15.76:0/2430501780 >> 172.21.15.76:0/2430501780 conn(0x7f4e080d0c30 msgr2=0x7f4e0806f100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:51.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:51.773+0000 7f4e0e3c9640 1 -- 172.21.15.76:0/2430501780 shutdown_connections 2023-11-09T03:55:51.776 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:51.773+0000 7f4e0e3c9640 1 -- 172.21.15.76:0/2430501780 wait complete. 2023-11-09T03:55:51.786 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_kill_mdstable_at 8 2023-11-09T03:55:51.900 INFO:teuthology.orchestra.run.smithi076.stdout:{"success":"mds_kill_mdstable_at = '8' (not observed, change may require restart) "} 2023-11-09T03:55:51.901 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '8' (not observed, change may require restart) " } 2023-11-09T03:55:51.901 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_kill_mdstable_at 3 2023-11-09T03:55:52.025 INFO:teuthology.orchestra.run.smithi113.stdout:{"success":"mds_kill_mdstable_at = '3' (not observed, change may require restart) "} 2023-11-09T03:55:52.025 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '3' (not observed, change may require restart) " } 2023-11-09T03:55:52.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:55:52.026 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s4) 2023-11-09T03:55:52.029 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2023-11-09T03:55:52.045 INFO:tasks.ceph.mds.b.smithi113.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::handle_request(ceph::cref_t&)' thread 7fdf7aa21640 time 2023-11-09T03:55:52.047031+0000 2023-11-09T03:55:52.046 INFO:tasks.ceph.mds.b.smithi113.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: 76: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 3) 2023-11-09T03:55:52.047 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:55:52.047 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fdf8056efaf] 2023-11-09T03:55:52.047 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fdf8056f16d] 2023-11-09T03:55:52.047 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x1187) [0x55a620c841b7] 2023-11-09T03:55:52.048 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xd55) [0x55a620a11d15] 2023-11-09T03:55:52.048 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55a620a13b2b] 2023-11-09T03:55:52.048 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a620a1401c] 2023-11-09T03:55:52.048 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55a6209fd325] 2023-11-09T03:55:52.048 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7fdf8075ff9a] 2023-11-09T03:55:52.049 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7fdf807f1741] 2023-11-09T03:55:52.049 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7fdf7fc9f802] 2023-11-09T03:55:52.049 INFO:tasks.ceph.mds.b.smithi113.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7fdf7fc3f450] 2023-11-09T03:55:52.049 INFO:tasks.ceph.mds.b.smithi113.stderr:*** Caught signal (Aborted) ** 2023-11-09T03:55:52.049 INFO:tasks.ceph.mds.b.smithi113.stderr: in thread 7fdf7aa21640 thread_name:ms_dispatch 2023-11-09T03:55:52.050 INFO:tasks.ceph.mds.b.smithi113.stderr:2023-11-09T03:55:52.046+0000 7fdf7aa21640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::handle_request(ceph::cref_t&)' thread 7fdf7aa21640 time 2023-11-09T03:55:52.047031+0000 2023-11-09T03:55:52.050 INFO:tasks.ceph.mds.b.smithi113.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: 76: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 3) 2023-11-09T03:55:52.050 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:55:52.050 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:55:52.051 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fdf8056efaf] 2023-11-09T03:55:52.051 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fdf8056f16d] 2023-11-09T03:55:52.051 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x1187) [0x55a620c841b7] 2023-11-09T03:55:52.051 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xd55) [0x55a620a11d15] 2023-11-09T03:55:52.051 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55a620a13b2b] 2023-11-09T03:55:52.052 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a620a1401c] 2023-11-09T03:55:52.052 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55a6209fd325] 2023-11-09T03:55:52.052 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7fdf8075ff9a] 2023-11-09T03:55:52.052 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7fdf807f1741] 2023-11-09T03:55:52.052 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7fdf7fc9f802] 2023-11-09T03:55:52.053 INFO:tasks.ceph.mds.b.smithi113.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7fdf7fc3f450] 2023-11-09T03:55:52.053 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:55:52.053 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:55:52.053 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fdf7fc54db0] 2023-11-09T03:55:52.053 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fdf7fca154c] 2023-11-09T03:55:52.054 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: raise() 2023-11-09T03:55:52.054 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: abort() 2023-11-09T03:55:52.054 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fdf8056f009] 2023-11-09T03:55:52.054 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fdf8056f16d] 2023-11-09T03:55:52.055 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x1187) [0x55a620c841b7] 2023-11-09T03:55:52.055 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xd55) [0x55a620a11d15] 2023-11-09T03:55:52.055 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55a620a13b2b] 2023-11-09T03:55:52.055 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a620a1401c] 2023-11-09T03:55:52.055 INFO:tasks.ceph.mds.b.smithi113.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55a6209fd325] 2023-11-09T03:55:52.056 INFO:tasks.ceph.mds.b.smithi113.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7fdf8075ff9a] 2023-11-09T03:55:52.056 INFO:tasks.ceph.mds.b.smithi113.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7fdf807f1741] 2023-11-09T03:55:52.056 INFO:tasks.ceph.mds.b.smithi113.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7fdf7fc9f802] 2023-11-09T03:55:52.056 INFO:tasks.ceph.mds.b.smithi113.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7fdf7fc3f450] 2023-11-09T03:55:52.056 INFO:tasks.ceph.mds.b.smithi113.stderr:2023-11-09T03:55:52.046+0000 7fdf7aa21640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:55:52.057 INFO:tasks.ceph.mds.b.smithi113.stderr: in thread 7fdf7aa21640 thread_name:ms_dispatch 2023-11-09T03:55:52.057 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:55:52.057 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:55:52.057 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fdf7fc54db0] 2023-11-09T03:55:52.057 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fdf7fca154c] 2023-11-09T03:55:52.058 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: raise() 2023-11-09T03:55:52.058 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: abort() 2023-11-09T03:55:52.058 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fdf8056f009] 2023-11-09T03:55:52.058 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fdf8056f16d] 2023-11-09T03:55:52.058 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x1187) [0x55a620c841b7] 2023-11-09T03:55:52.059 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xd55) [0x55a620a11d15] 2023-11-09T03:55:52.059 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55a620a13b2b] 2023-11-09T03:55:52.059 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a620a1401c] 2023-11-09T03:55:52.059 INFO:tasks.ceph.mds.b.smithi113.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55a6209fd325] 2023-11-09T03:55:52.060 INFO:tasks.ceph.mds.b.smithi113.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7fdf8075ff9a] 2023-11-09T03:55:52.060 INFO:tasks.ceph.mds.b.smithi113.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7fdf807f1741] 2023-11-09T03:55:52.060 INFO:tasks.ceph.mds.b.smithi113.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7fdf7fc9f802] 2023-11-09T03:55:52.060 INFO:tasks.ceph.mds.b.smithi113.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7fdf7fc3f450] 2023-11-09T03:55:52.060 INFO:tasks.ceph.mds.b.smithi113.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:55:52.061 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:55:52.061 INFO:tasks.ceph.mds.b.smithi113.stderr: -1> 2023-11-09T03:55:52.046+0000 7fdf7aa21640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::handle_request(ceph::cref_t&)' thread 7fdf7aa21640 time 2023-11-09T03:55:52.047031+0000 2023-11-09T03:55:52.062 INFO:tasks.ceph.mds.b.smithi113.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: 76: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 3) 2023-11-09T03:55:52.062 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:55:52.062 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:55:52.062 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fdf8056efaf] 2023-11-09T03:55:52.063 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fdf8056f16d] 2023-11-09T03:55:52.063 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x1187) [0x55a620c841b7] 2023-11-09T03:55:52.063 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xd55) [0x55a620a11d15] 2023-11-09T03:55:52.063 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55a620a13b2b] 2023-11-09T03:55:52.063 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a620a1401c] 2023-11-09T03:55:52.064 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55a6209fd325] 2023-11-09T03:55:52.064 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7fdf8075ff9a] 2023-11-09T03:55:52.064 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7fdf807f1741] 2023-11-09T03:55:52.064 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7fdf7fc9f802] 2023-11-09T03:55:52.064 INFO:tasks.ceph.mds.b.smithi113.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7fdf7fc3f450] 2023-11-09T03:55:52.065 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:55:52.065 INFO:tasks.ceph.mds.b.smithi113.stderr: 0> 2023-11-09T03:55:52.046+0000 7fdf7aa21640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:55:52.065 INFO:tasks.ceph.mds.b.smithi113.stderr: in thread 7fdf7aa21640 thread_name:ms_dispatch 2023-11-09T03:55:52.065 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:55:52.066 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:55:52.066 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fdf7fc54db0] 2023-11-09T03:55:52.066 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fdf7fca154c] 2023-11-09T03:55:52.066 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: raise() 2023-11-09T03:55:52.066 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: abort() 2023-11-09T03:55:52.067 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fdf8056f009] 2023-11-09T03:55:52.067 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fdf8056f16d] 2023-11-09T03:55:52.067 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x1187) [0x55a620c841b7] 2023-11-09T03:55:52.067 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xd55) [0x55a620a11d15] 2023-11-09T03:55:52.067 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55a620a13b2b] 2023-11-09T03:55:52.068 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a620a1401c] 2023-11-09T03:55:52.068 INFO:tasks.ceph.mds.b.smithi113.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55a6209fd325] 2023-11-09T03:55:52.068 INFO:tasks.ceph.mds.b.smithi113.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7fdf8075ff9a] 2023-11-09T03:55:52.068 INFO:tasks.ceph.mds.b.smithi113.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7fdf807f1741] 2023-11-09T03:55:52.068 INFO:tasks.ceph.mds.b.smithi113.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7fdf7fc9f802] 2023-11-09T03:55:52.069 INFO:tasks.ceph.mds.b.smithi113.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7fdf7fc3f450] 2023-11-09T03:55:52.069 INFO:tasks.ceph.mds.b.smithi113.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:55:52.069 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:55:52.069 INFO:tasks.ceph.mds.b.smithi113.stderr: -1> 2023-11-09T03:55:52.046+0000 7fdf7aa21640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::handle_request(ceph::cref_t&)' thread 7fdf7aa21640 time 2023-11-09T03:55:52.047031+0000 2023-11-09T03:55:52.069 INFO:tasks.ceph.mds.b.smithi113.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableClient.cc: 76: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 3) 2023-11-09T03:55:52.070 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:55:52.070 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:55:52.070 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fdf8056efaf] 2023-11-09T03:55:52.070 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fdf8056f16d] 2023-11-09T03:55:52.070 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x1187) [0x55a620c841b7] 2023-11-09T03:55:52.071 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xd55) [0x55a620a11d15] 2023-11-09T03:55:52.071 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55a620a13b2b] 2023-11-09T03:55:52.071 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a620a1401c] 2023-11-09T03:55:52.071 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55a6209fd325] 2023-11-09T03:55:52.072 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7fdf8075ff9a] 2023-11-09T03:55:52.072 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7fdf807f1741] 2023-11-09T03:55:52.072 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7fdf7fc9f802] 2023-11-09T03:55:52.072 INFO:tasks.ceph.mds.b.smithi113.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7fdf7fc3f450] 2023-11-09T03:55:52.072 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:55:52.073 INFO:tasks.ceph.mds.b.smithi113.stderr: 0> 2023-11-09T03:55:52.046+0000 7fdf7aa21640 -1 *** Caught signal (Aborted) ** 2023-11-09T03:55:52.073 INFO:tasks.ceph.mds.b.smithi113.stderr: in thread 7fdf7aa21640 thread_name:ms_dispatch 2023-11-09T03:55:52.073 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:55:52.073 INFO:tasks.ceph.mds.b.smithi113.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-09T03:55:52.074 INFO:tasks.ceph.mds.b.smithi113.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fdf7fc54db0] 2023-11-09T03:55:52.074 INFO:tasks.ceph.mds.b.smithi113.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fdf7fca154c] 2023-11-09T03:55:52.074 INFO:tasks.ceph.mds.b.smithi113.stderr: 3: raise() 2023-11-09T03:55:52.074 INFO:tasks.ceph.mds.b.smithi113.stderr: 4: abort() 2023-11-09T03:55:52.074 INFO:tasks.ceph.mds.b.smithi113.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fdf8056f009] 2023-11-09T03:55:52.075 INFO:tasks.ceph.mds.b.smithi113.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fdf8056f16d] 2023-11-09T03:55:52.075 INFO:tasks.ceph.mds.b.smithi113.stderr: 7: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x1187) [0x55a620c841b7] 2023-11-09T03:55:52.075 INFO:tasks.ceph.mds.b.smithi113.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xd55) [0x55a620a11d15] 2023-11-09T03:55:52.075 INFO:tasks.ceph.mds.b.smithi113.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55a620a13b2b] 2023-11-09T03:55:52.075 INFO:tasks.ceph.mds.b.smithi113.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a620a1401c] 2023-11-09T03:55:52.076 INFO:tasks.ceph.mds.b.smithi113.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55a6209fd325] 2023-11-09T03:55:52.076 INFO:tasks.ceph.mds.b.smithi113.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7fdf8075ff9a] 2023-11-09T03:55:52.076 INFO:tasks.ceph.mds.b.smithi113.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7fdf807f1741] 2023-11-09T03:55:52.076 INFO:tasks.ceph.mds.b.smithi113.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7fdf7fc9f802] 2023-11-09T03:55:52.076 INFO:tasks.ceph.mds.b.smithi113.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7fdf7fc3f450] 2023-11-09T03:55:52.077 INFO:tasks.ceph.mds.b.smithi113.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-09T03:55:52.077 INFO:tasks.ceph.mds.b.smithi113.stderr: 2023-11-09T03:55:52.175 INFO:teuthology.orchestra.run.smithi076.stdout:{"mds_beacon_grace":"15.000000"} 2023-11-09T03:55:52.176 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2023-11-09T03:55:52.176 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:55:52.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.274+0000 7fe43eb16640 1 Processor -- start 2023-11-09T03:55:52.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.274+0000 7fe43eb16640 1 -- start start 2023-11-09T03:55:52.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.275+0000 7fe43eb16640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe43806c220 0x7fe4380cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:52.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.275+0000 7fe43eb16640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4380cd720 con 0x7fe4380cc680 2023-11-09T03:55:52.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.275+0000 7fe43eb16640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4380ccb10 con 0x7fe4380addd0 2023-11-09T03:55:52.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.275+0000 7fe43eb16640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4380ae250 con 0x7fe43806c220 2023-11-09T03:55:52.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.275+0000 7fe43d0b3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe43806c220 0x7fe4380cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:52.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.275+0000 7fe43c8b2640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe4380cc680 0x7fe4380b05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37738/0 (socket says 172.21.15.76:37738) 2023-11-09T03:55:52.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.275+0000 7fe43d0b3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe43806c220 0x7fe4380cdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49936/0 (socket says 172.21.15.76:49936) 2023-11-09T03:55:52.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.275+0000 7fe43c8b2640 1 -- 172.21.15.76:0/233732967 learned_addr learned my addr 172.21.15.76:0/233732967 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:55:52.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.276+0000 7fe43d0b3640 1 -- 172.21.15.76:0/233732967 >> v1:172.21.15.113:6789/0 conn(0x7fe4380addd0 legacy=0x7fe43806bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:52.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.276+0000 7fe43d0b3640 1 -- 172.21.15.76:0/233732967 >> v1:172.21.15.76:6789/0 conn(0x7fe4380cc680 legacy=0x7fe4380b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:52.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.276+0000 7fe43d0b3640 1 -- 172.21.15.76:0/233732967 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4380ccdf0 con 0x7fe43806c220 2023-11-09T03:55:52.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.276+0000 7fe4377fe640 1 -- 172.21.15.76:0/233732967 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3198155233 0 0) 0x7fe4380cd720 con 0x7fe4380cc680 2023-11-09T03:55:52.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.276+0000 7fe43d0b3640 1 --2- 172.21.15.76:0/233732967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe43806c220 0x7fe4380cdbb0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fe430009d90 tx=0x7fe43002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3a0c923b3a0f50 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:52.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.276+0000 7fe4377fe640 1 -- 172.21.15.76:0/233732967 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43002cc60 con 0x7fe43806c220 2023-11-09T03:55:52.283 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.277+0000 7fe4377fe640 1 -- 172.21.15.76:0/233732967 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe43002cdc0 con 0x7fe43806c220 2023-11-09T03:55:52.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.277+0000 7fe4377fe640 1 -- 172.21.15.76:0/233732967 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe430034660 con 0x7fe43806c220 2023-11-09T03:55:52.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.277+0000 7fe43eb16640 1 -- 172.21.15.76:0/233732967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe43806c220 msgr2=0x7fe4380cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:52.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.277+0000 7fe43eb16640 1 --2- 172.21.15.76:0/233732967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe43806c220 0x7fe4380cdbb0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fe430009d90 tx=0x7fe43002beb0 comp rx=0 tx=0).stop 2023-11-09T03:55:52.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.277+0000 7fe43eb16640 1 -- 172.21.15.76:0/233732967 shutdown_connections 2023-11-09T03:55:52.284 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.277+0000 7fe43eb16640 1 --2- 172.21.15.76:0/233732967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe43806c220 0x7fe4380cdbb0 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:52.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.277+0000 7fe43eb16640 1 -- 172.21.15.76:0/233732967 >> 172.21.15.76:0/233732967 conn(0x7fe4380cb530 msgr2=0x7fe4380cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:52.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.277+0000 7fe43eb16640 1 -- 172.21.15.76:0/233732967 shutdown_connections 2023-11-09T03:55:52.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.277+0000 7fe43eb16640 1 -- 172.21.15.76:0/233732967 wait complete. 2023-11-09T03:55:52.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.278+0000 7fe43eb16640 1 Processor -- start 2023-11-09T03:55:52.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.278+0000 7fe43eb16640 1 -- start start 2023-11-09T03:55:52.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.279+0000 7fe43eb16640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe43806c220 0x7fe4381efae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:52.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.279+0000 7fe43eb16640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4380addd0 0x7fe4380c39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:52.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.279+0000 7fe43eb16640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4380c7b70 0x7fe43815e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:52.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.279+0000 7fe43eb16640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4381f0020 con 0x7fe4380c7b70 2023-11-09T03:55:52.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.279+0000 7fe43eb16640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4381528c0 con 0x7fe43806c220 2023-11-09T03:55:52.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.279+0000 7fe43eb16640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4381f02a0 con 0x7fe4380addd0 2023-11-09T03:55:52.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.279+0000 7fe43d0b3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4380c7b70 0x7fe43815e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:52.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.279+0000 7fe43d0b3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4380c7b70 0x7fe43815e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49940/0 (socket says 172.21.15.76:49940) 2023-11-09T03:55:52.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.279+0000 7fe43d0b3640 1 -- 172.21.15.76:0/700993267 learned_addr learned my addr 172.21.15.76:0/700993267 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:52.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.279+0000 7fe43c8b2640 1 --2- 172.21.15.76:0/700993267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe43806c220 0x7fe4381efae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:52.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.279+0000 7fe437fff640 1 --2- 172.21.15.76:0/700993267 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4380addd0 0x7fe4380c39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:52.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.280+0000 7fe43d0b3640 1 -- 172.21.15.76:0/700993267 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4380addd0 msgr2=0x7fe4380c39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:52.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.280+0000 7fe43d0b3640 1 --2- 172.21.15.76:0/700993267 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4380addd0 0x7fe4380c39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:52.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.280+0000 7fe43d0b3640 1 -- 172.21.15.76:0/700993267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe43806c220 msgr2=0x7fe4381efae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:52.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.280+0000 7fe43d0b3640 1 --2- 172.21.15.76:0/700993267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe43806c220 0x7fe4381efae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:52.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.280+0000 7fe43d0b3640 1 -- 172.21.15.76:0/700993267 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe430009a70 con 0x7fe4380c7b70 2023-11-09T03:55:52.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.280+0000 7fe43d0b3640 1 --2- 172.21.15.76:0/700993267 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4380c7b70 0x7fe43815e320 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fe430009eb0 tx=0x7fe430004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:52.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.280+0000 7fe4357fa640 1 -- 172.21.15.76:0/700993267 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43003e610 con 0x7fe4380c7b70 2023-11-09T03:55:52.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.280+0000 7fe4357fa640 1 -- 172.21.15.76:0/700993267 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe43003e790 con 0x7fe4380c7b70 2023-11-09T03:55:52.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.280+0000 7fe43eb16640 1 -- 172.21.15.76:0/700993267 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4380c8960 con 0x7fe4380c7b70 2023-11-09T03:55:52.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.280+0000 7fe4357fa640 1 -- 172.21.15.76:0/700993267 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43003eb40 con 0x7fe4380c7b70 2023-11-09T03:55:52.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.280+0000 7fe43eb16640 1 -- 172.21.15.76:0/700993267 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4380c83a0 con 0x7fe4380c7b70 2023-11-09T03:55:52.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.281+0000 7fe43eb16640 1 -- 172.21.15.76:0/700993267 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4380684b0 con 0x7fe4380c7b70 2023-11-09T03:55:52.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.281+0000 7fe4357fa640 1 -- 172.21.15.76:0/700993267 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe43003b050 con 0x7fe4380c7b70 2023-11-09T03:55:52.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.282+0000 7fe4357fa640 1 --2- 172.21.15.76:0/700993267 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe42406ae70 0x7fe42406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:52.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.282+0000 7fe4357fa640 1 -- 172.21.15.76:0/700993267 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8626+0+0 (secure 0 0 0) 0x7fe4300acb80 con 0x7fe4380c7b70 2023-11-09T03:55:52.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.282+0000 7fe43c8b2640 1 --2- 172.21.15.76:0/700993267 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe42406ae70 0x7fe42406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:52.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.282+0000 7fe43c8b2640 1 --2- 172.21.15.76:0/700993267 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe42406ae70 0x7fe42406d330 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe428007fd0 tx=0x7fe428007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:52.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.286+0000 7fe4357fa640 1 -- 172.21.15.76:0/700993267 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe430077120 con 0x7fe4380c7b70 2023-11-09T03:55:52.318 INFO:tasks.ceph.mds.b.smithi113.stderr:daemon-helper: command crashed with signal 6 2023-11-09T03:55:52.488 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.488+0000 7fe43eb16640 1 -- 172.21.15.76:0/700993267 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4381f03e0 con 0x7fe4380c7b70 2023-11-09T03:55:52.489 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.488+0000 7fe4357fa640 1 -- 172.21.15.76:0/700993267 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) v1 ==== 95+0+4935 (secure 0 0 0) 0x7fe43004e090 con 0x7fe4380c7b70 2023-11-09T03:55:52.489 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:55:52.489 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":143,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":9525,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1713748386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1713748386},{"type":"v1","addr":"172.21.15.76:6835","nonce":1713748386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":138}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:55:51.132628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":6698},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6698":{"gid":6698,"name":"b","rank":1,"incarnation":137,"state":"up:active","state_seq":18,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:55:52.490 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 143 2023-11-09T03:55:52.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.490+0000 7fe43eb16640 1 -- 172.21.15.76:0/700993267 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe42406ae70 msgr2=0x7fe42406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:52.491 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.490+0000 7fe43eb16640 1 --2- 172.21.15.76:0/700993267 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe42406ae70 0x7fe42406d330 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe428007fd0 tx=0x7fe428007de0 comp rx=0 tx=0).stop 2023-11-09T03:55:52.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.490+0000 7fe43eb16640 1 -- 172.21.15.76:0/700993267 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4380c7b70 msgr2=0x7fe43815e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:52.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.490+0000 7fe43eb16640 1 --2- 172.21.15.76:0/700993267 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4380c7b70 0x7fe43815e320 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fe430009eb0 tx=0x7fe430004a40 comp rx=0 tx=0).stop 2023-11-09T03:55:52.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.490+0000 7fe43eb16640 1 -- 172.21.15.76:0/700993267 shutdown_connections 2023-11-09T03:55:52.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.490+0000 7fe43eb16640 1 --2- 172.21.15.76:0/700993267 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fe42406ae70 0x7fe42406d330 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:52.492 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.490+0000 7fe43eb16640 1 --2- 172.21.15.76:0/700993267 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4380c7b70 0x7fe43815e320 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:52.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.490+0000 7fe43eb16640 1 --2- 172.21.15.76:0/700993267 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4380addd0 0x7fe4380c39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:52.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.490+0000 7fe43eb16640 1 --2- 172.21.15.76:0/700993267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe43806c220 0x7fe4381efae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:52.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.490+0000 7fe43eb16640 1 -- 172.21.15.76:0/700993267 >> 172.21.15.76:0/700993267 conn(0x7fe4380cb530 msgr2=0x7fe4380c9600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:52.493 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.490+0000 7fe43eb16640 1 -- 172.21.15.76:0/700993267 shutdown_connections 2023-11-09T03:55:52.494 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:52.490+0000 7fe43eb16640 1 -- 172.21.15.76:0/700993267 wait complete. 2023-11-09T03:55:52.502 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:55:57.503 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:55:57.612 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.612+0000 7ff29b325640 1 Processor -- start 2023-11-09T03:55:57.613 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.612+0000 7ff29b325640 1 -- start start 2023-11-09T03:55:57.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.613+0000 7ff29b325640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff29406c5f0 0x7ff294165ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:57.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.613+0000 7ff29b325640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff29406cc90 con 0x7ff2940ade00 2023-11-09T03:55:57.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.613+0000 7ff29b325640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2940b0e60 con 0x7ff29406d9c0 2023-11-09T03:55:57.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.613+0000 7ff29b325640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2940b0c40 con 0x7ff29406c5f0 2023-11-09T03:55:57.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.613+0000 7ff2998c2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff29406c5f0 0x7ff294165ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:57.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.613+0000 7ff2990c1640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff2940ade00 0x7ff2940b0530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43252/0 (socket says 172.21.15.76:43252) 2023-11-09T03:55:57.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.613+0000 7ff2998c2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff29406c5f0 0x7ff294165ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35082/0 (socket says 172.21.15.76:35082) 2023-11-09T03:55:57.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.613+0000 7ff2990c1640 1 -- 172.21.15.76:0/2775964556 learned_addr learned my addr 172.21.15.76:0/2775964556 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:55:57.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.613+0000 7ff2998c2640 1 -- 172.21.15.76:0/2775964556 >> v1:172.21.15.113:6789/0 conn(0x7ff29406d9c0 legacy=0x7ff2940710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:55:57.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.613+0000 7ff2998c2640 1 -- 172.21.15.76:0/2775964556 >> v1:172.21.15.76:6789/0 conn(0x7ff2940ade00 legacy=0x7ff2940b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:57.617 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.613+0000 7ff28bfff640 1 -- 172.21.15.76:0/2775964556 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1560988432 0 0) 0x7ff29406cc90 con 0x7ff2940ade00 2023-11-09T03:55:57.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.613+0000 7ff2998c2640 1 -- 172.21.15.76:0/2775964556 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29406cee0 con 0x7ff29406c5f0 2023-11-09T03:55:57.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.614+0000 7ff2998c2640 1 --2- 172.21.15.76:0/2775964556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff29406c5f0 0x7ff294165ce0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009d90 tx=0x7ff28c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fefa87ca67ebb7b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:57.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.614+0000 7ff28bfff640 1 -- 172.21.15.76:0/2775964556 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c02cbd0 con 0x7ff29406c5f0 2023-11-09T03:55:57.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.614+0000 7ff28bfff640 1 -- 172.21.15.76:0/2775964556 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28c02cd30 con 0x7ff29406c5f0 2023-11-09T03:55:57.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.614+0000 7ff28bfff640 1 -- 172.21.15.76:0/2775964556 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c02cea0 con 0x7ff29406c5f0 2023-11-09T03:55:57.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.615+0000 7ff29b325640 1 -- 172.21.15.76:0/2775964556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff29406c5f0 msgr2=0x7ff294165ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:57.623 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.615+0000 7ff29b325640 1 --2- 172.21.15.76:0/2775964556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff29406c5f0 0x7ff294165ce0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009d90 tx=0x7ff28c02be20 comp rx=0 tx=0).stop 2023-11-09T03:55:57.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.615+0000 7ff29b325640 1 -- 172.21.15.76:0/2775964556 shutdown_connections 2023-11-09T03:55:57.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.615+0000 7ff29b325640 1 --2- 172.21.15.76:0/2775964556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff29406c5f0 0x7ff294165ce0 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:57.624 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.615+0000 7ff29b325640 1 -- 172.21.15.76:0/2775964556 >> 172.21.15.76:0/2775964556 conn(0x7ff2940cb530 msgr2=0x7ff2940cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:57.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.615+0000 7ff29b325640 1 -- 172.21.15.76:0/2775964556 shutdown_connections 2023-11-09T03:55:57.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.615+0000 7ff29b325640 1 -- 172.21.15.76:0/2775964556 wait complete. 2023-11-09T03:55:57.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.616+0000 7ff29b325640 1 Processor -- start 2023-11-09T03:55:57.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.616+0000 7ff29b325640 1 -- start start 2023-11-09T03:55:57.625 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.617+0000 7ff29b325640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff29406c5f0 0x7ff294158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:57.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.617+0000 7ff29b325640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff29406d9c0 0x7ff2941f3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:57.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.617+0000 7ff29b325640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2940ade00 0x7ff2941f4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:57.626 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.617+0000 7ff29b325640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2941f1ce0 con 0x7ff29406d9c0 2023-11-09T03:55:57.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.617+0000 7ff29b325640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2941f19b0 con 0x7ff2940ade00 2023-11-09T03:55:57.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.617+0000 7ff29b325640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29415a190 con 0x7ff29406c5f0 2023-11-09T03:55:57.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.617+0000 7ff2988c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff29406d9c0 0x7ff2941f3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:57.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.617+0000 7ff2988c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff29406d9c0 0x7ff2941f3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35088/0 (socket says 172.21.15.76:35088) 2023-11-09T03:55:57.627 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.617+0000 7ff2988c0640 1 -- 172.21.15.76:0/868211591 learned_addr learned my addr 172.21.15.76:0/868211591 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:55:57.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.617+0000 7ff2998c2640 1 --2- 172.21.15.76:0/868211591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2940ade00 0x7ff2941f4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:57.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.617+0000 7ff2990c1640 1 --2- 172.21.15.76:0/868211591 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff29406c5f0 0x7ff294158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:57.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.617+0000 7ff2988c0640 1 -- 172.21.15.76:0/868211591 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff29406c5f0 msgr2=0x7ff294158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:57.628 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.617+0000 7ff2988c0640 1 --2- 172.21.15.76:0/868211591 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff29406c5f0 0x7ff294158300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:57.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.617+0000 7ff2988c0640 1 -- 172.21.15.76:0/868211591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2940ade00 msgr2=0x7ff2941f4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:57.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.617+0000 7ff2988c0640 1 --2- 172.21.15.76:0/868211591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2940ade00 0x7ff2941f4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:57.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.617+0000 7ff2988c0640 1 -- 172.21.15.76:0/868211591 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28c009a70 con 0x7ff29406d9c0 2023-11-09T03:55:57.629 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.617+0000 7ff2990c1640 1 --2- 172.21.15.76:0/868211591 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff29406c5f0 0x7ff294158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:55:57.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.617+0000 7ff2988c0640 1 --2- 172.21.15.76:0/868211591 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff29406d9c0 0x7ff2941f3d40 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7ff28400c3c0 tx=0x7ff28400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:57.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.617+0000 7ff289ffb640 1 -- 172.21.15.76:0/868211591 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2840052d0 con 0x7ff29406d9c0 2023-11-09T03:55:57.630 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.617+0000 7ff289ffb640 1 -- 172.21.15.76:0/868211591 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff284005430 con 0x7ff29406d9c0 2023-11-09T03:55:57.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.617+0000 7ff29b325640 1 -- 172.21.15.76:0/868211591 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff29415a470 con 0x7ff29406d9c0 2023-11-09T03:55:57.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.617+0000 7ff289ffb640 1 -- 172.21.15.76:0/868211591 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff284003830 con 0x7ff29406d9c0 2023-11-09T03:55:57.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.618+0000 7ff29b325640 1 -- 172.21.15.76:0/868211591 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff294159430 con 0x7ff29406d9c0 2023-11-09T03:55:57.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.618+0000 7ff29b325640 1 -- 172.21.15.76:0/868211591 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2940684b0 con 0x7ff29406d9c0 2023-11-09T03:55:57.631 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.619+0000 7ff289ffb640 1 -- 172.21.15.76:0/868211591 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff284003990 con 0x7ff29406d9c0 2023-11-09T03:55:57.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.619+0000 7ff289ffb640 1 --2- 172.21.15.76:0/868211591 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff27806ae70 0x7ff27806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:55:57.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.619+0000 7ff289ffb640 1 -- 172.21.15.76:0/868211591 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8582+0+0 (secure 0 0 0) 0x7ff2840895a0 con 0x7ff29406d9c0 2023-11-09T03:55:57.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.619+0000 7ff2990c1640 1 --2- 172.21.15.76:0/868211591 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff27806ae70 0x7ff27806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:55:57.632 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.620+0000 7ff2990c1640 1 --2- 172.21.15.76:0/868211591 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff27806ae70 0x7ff27806d330 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ff290007fd0 tx=0x7ff290007dc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:55:57.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.624+0000 7ff289ffb640 1 -- 172.21.15.76:0/868211591 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff284053b70 con 0x7ff29406d9c0 2023-11-09T03:55:57.827 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.826+0000 7ff29b325640 1 -- 172.21.15.76:0/868211591 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff29406d4a0 con 0x7ff29406d9c0 2023-11-09T03:55:57.828 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.827+0000 7ff289ffb640 1 -- 172.21.15.76:0/868211591 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) v1 ==== 95+0+4935 (secure 0 0 0) 0x7ff284017090 con 0x7ff29406d9c0 2023-11-09T03:55:57.828 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:55:57.828 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":143,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":9525,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1713748386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1713748386},{"type":"v1","addr":"172.21.15.76:6835","nonce":1713748386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":138}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:55:51.132628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":6698},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6698":{"gid":6698,"name":"b","rank":1,"incarnation":137,"state":"up:active","state_seq":18,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:55:57.829 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 143 2023-11-09T03:55:57.830 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.829+0000 7ff29b325640 1 -- 172.21.15.76:0/868211591 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff27806ae70 msgr2=0x7ff27806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:57.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.829+0000 7ff29b325640 1 --2- 172.21.15.76:0/868211591 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff27806ae70 0x7ff27806d330 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ff290007fd0 tx=0x7ff290007dc0 comp rx=0 tx=0).stop 2023-11-09T03:55:57.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.829+0000 7ff29b325640 1 -- 172.21.15.76:0/868211591 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff29406d9c0 msgr2=0x7ff2941f3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:55:57.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.829+0000 7ff29b325640 1 --2- 172.21.15.76:0/868211591 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff29406d9c0 0x7ff2941f3d40 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7ff28400c3c0 tx=0x7ff28400c880 comp rx=0 tx=0).stop 2023-11-09T03:55:57.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.829+0000 7ff29b325640 1 -- 172.21.15.76:0/868211591 shutdown_connections 2023-11-09T03:55:57.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.829+0000 7ff29b325640 1 --2- 172.21.15.76:0/868211591 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff27806ae70 0x7ff27806d330 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:57.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.829+0000 7ff29b325640 1 --2- 172.21.15.76:0/868211591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2940ade00 0x7ff2941f4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:57.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.829+0000 7ff29b325640 1 --2- 172.21.15.76:0/868211591 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff29406d9c0 0x7ff2941f3d40 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:57.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.829+0000 7ff29b325640 1 --2- 172.21.15.76:0/868211591 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff29406c5f0 0x7ff294158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:55:57.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.829+0000 7ff29b325640 1 -- 172.21.15.76:0/868211591 >> 172.21.15.76:0/868211591 conn(0x7ff2940cb530 msgr2=0x7ff29415be10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:55:57.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.829+0000 7ff29b325640 1 -- 172.21.15.76:0/868211591 shutdown_connections 2023-11-09T03:55:57.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:55:57.829+0000 7ff29b325640 1 -- 172.21.15.76:0/868211591 wait complete. 2023-11-09T03:55:57.841 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:55:58.791 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-11-09T03:56:02.843 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:56:02.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.958+0000 7f40c7ed4640 1 Processor -- start 2023-11-09T03:56:02.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.959+0000 7f40c7ed4640 1 -- start start 2023-11-09T03:56:02.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.959+0000 7f40c7ed4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f40c006d9c0 0x7f40c00710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:02.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.959+0000 7f40c7ed4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f40c006c5f0 0x7f40c0071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:02.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.959+0000 7f40c7ed4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c006cc90 con 0x7f40c00ade00 2023-11-09T03:56:02.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.959+0000 7f40c7ed4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c00b0c40 con 0x7f40c006c5f0 2023-11-09T03:56:02.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.959+0000 7f40c7ed4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c0071b50 con 0x7f40c006d9c0 2023-11-09T03:56:02.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.960+0000 7f40c6471640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f40c006c5f0 0x7f40c0071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:02.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.960+0000 7f40c5c70640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f40c00ade00 0x7f40c00b0530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47784/0 (socket says 172.21.15.76:47784) 2023-11-09T03:56:02.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.960+0000 7f40c5c70640 1 -- 172.21.15.76:0/1676784132 learned_addr learned my addr 172.21.15.76:0/1676784132 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:56:02.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.960+0000 7f40c546f640 1 --2- 172.21.15.76:0/1676784132 >> v2:172.21.15.113:3300/0 conn(0x7f40c006d9c0 0x7f40c00710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:02.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.960+0000 7f40c6471640 1 -- 172.21.15.76:0/1676784132 >> v2:172.21.15.113:3300/0 conn(0x7f40c006d9c0 msgr2=0x7f40c00710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:02.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.960+0000 7f40c6471640 1 --2- 172.21.15.76:0/1676784132 >> v2:172.21.15.113:3300/0 conn(0x7f40c006d9c0 0x7f40c00710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:02.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.960+0000 7f40c6471640 1 -- 172.21.15.76:0/1676784132 >> v1:172.21.15.76:6789/0 conn(0x7f40c00ade00 legacy=0x7f40c00b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:02.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.960+0000 7f40c4c6e640 1 -- 172.21.15.76:0/1676784132 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1099182627 0 0) 0x7f40c006cc90 con 0x7f40c00ade00 2023-11-09T03:56:02.962 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.960+0000 7f40c6471640 1 -- 172.21.15.76:0/1676784132 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c00b0e60 con 0x7f40c006c5f0 2023-11-09T03:56:02.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.961+0000 7f40c6471640 1 --2- 172.21.15.76:0/1676784132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40c006c5f0 0x7f40c0071610 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f40c00c9080 tx=0x7f40b802bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7270e06ed7a6c15 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:02.965 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.961+0000 7f40c4c6e640 1 -- 172.21.15.76:0/1676784132 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b802cb60 con 0x7f40c006c5f0 2023-11-09T03:56:02.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.961+0000 7f40c4c6e640 1 -- 172.21.15.76:0/1676784132 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b802ccc0 con 0x7f40c006c5f0 2023-11-09T03:56:02.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.961+0000 7f40c4c6e640 1 -- 172.21.15.76:0/1676784132 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8034660 con 0x7f40c006c5f0 2023-11-09T03:56:02.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.961+0000 7f40c7ed4640 1 -- 172.21.15.76:0/1676784132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40c006c5f0 msgr2=0x7f40c0071610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:02.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.961+0000 7f40c7ed4640 1 --2- 172.21.15.76:0/1676784132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40c006c5f0 0x7f40c0071610 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f40c00c9080 tx=0x7f40b802bdb0 comp rx=0 tx=0).stop 2023-11-09T03:56:02.966 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.962+0000 7f40c7ed4640 1 -- 172.21.15.76:0/1676784132 shutdown_connections 2023-11-09T03:56:02.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.962+0000 7f40c7ed4640 1 --2- 172.21.15.76:0/1676784132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40c006c5f0 0x7f40c0071610 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:02.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.962+0000 7f40c7ed4640 1 --2- 172.21.15.76:0/1676784132 >> v2:172.21.15.113:3300/0 conn(0x7f40c006d9c0 0x7f40c00710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:02.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.962+0000 7f40c7ed4640 1 -- 172.21.15.76:0/1676784132 >> 172.21.15.76:0/1676784132 conn(0x7f40c00cb530 msgr2=0x7f40c00cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:02.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.962+0000 7f40c7ed4640 1 -- 172.21.15.76:0/1676784132 shutdown_connections 2023-11-09T03:56:02.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.962+0000 7f40c7ed4640 1 -- 172.21.15.76:0/1676784132 wait complete. 2023-11-09T03:56:02.967 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.963+0000 7f40c7ed4640 1 Processor -- start 2023-11-09T03:56:02.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.963+0000 7f40c7ed4640 1 -- start start 2023-11-09T03:56:02.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.964+0000 7f40c7ed4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c006c5f0 0x7f40c01591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:02.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.964+0000 7f40c7ed4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c006d9c0 0x7f40c00af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:02.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.964+0000 7f40c7ed4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40c00ade00 0x7f40c00af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:02.968 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.964+0000 7f40c7ed4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c00c3e50 con 0x7f40c00ade00 2023-11-09T03:56:02.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.964+0000 7f40c7ed4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c00c4130 con 0x7f40c006d9c0 2023-11-09T03:56:02.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.964+0000 7f40c7ed4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c00c4410 con 0x7f40c006c5f0 2023-11-09T03:56:02.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.964+0000 7f40c6471640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40c00ade00 0x7f40c00af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:02.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.964+0000 7f40c6471640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40c00ade00 0x7f40c00af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40300/0 (socket says 172.21.15.76:40300) 2023-11-09T03:56:02.969 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.964+0000 7f40c6471640 1 -- 172.21.15.76:0/1622050035 learned_addr learned my addr 172.21.15.76:0/1622050035 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:56:02.970 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.964+0000 7f40c5c70640 1 --2- 172.21.15.76:0/1622050035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c006c5f0 0x7f40c01591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:02.970 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.964+0000 7f40c546f640 1 --2- 172.21.15.76:0/1622050035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c006d9c0 0x7f40c00af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:02.970 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.964+0000 7f40c6471640 1 -- 172.21.15.76:0/1622050035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c006c5f0 msgr2=0x7f40c01591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:02.970 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.964+0000 7f40c6471640 1 --2- 172.21.15.76:0/1622050035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c006c5f0 0x7f40c01591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:02.970 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.964+0000 7f40c6471640 1 -- 172.21.15.76:0/1622050035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c006d9c0 msgr2=0x7f40c00af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:02.971 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.964+0000 7f40c6471640 1 --2- 172.21.15.76:0/1622050035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c006d9c0 0x7f40c00af1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:02.971 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.964+0000 7f40c6471640 1 -- 172.21.15.76:0/1622050035 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b8009a70 con 0x7f40c00ade00 2023-11-09T03:56:02.971 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.964+0000 7f40c5c70640 1 --2- 172.21.15.76:0/1622050035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c006c5f0 0x7f40c01591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:56:02.971 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.964+0000 7f40c6471640 1 --2- 172.21.15.76:0/1622050035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40c00ade00 0x7f40c00af6e0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f40b8037040 tx=0x7f40b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:02.971 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.964+0000 7f40b27fc640 1 -- 172.21.15.76:0/1622050035 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8040650 con 0x7f40c00ade00 2023-11-09T03:56:02.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.964+0000 7f40b27fc640 1 -- 172.21.15.76:0/1622050035 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b80407b0 con 0x7f40c00ade00 2023-11-09T03:56:02.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.964+0000 7f40c7ed4640 1 -- 172.21.15.76:0/1622050035 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c00c46f0 con 0x7f40c00ade00 2023-11-09T03:56:02.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.964+0000 7f40b27fc640 1 -- 172.21.15.76:0/1622050035 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8040b40 con 0x7f40c00ade00 2023-11-09T03:56:02.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.964+0000 7f40c7ed4640 1 -- 172.21.15.76:0/1622050035 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c0159c90 con 0x7f40c00ade00 2023-11-09T03:56:02.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.966+0000 7f40c7ed4640 1 -- 172.21.15.76:0/1622050035 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c00684b0 con 0x7f40c00ade00 2023-11-09T03:56:02.972 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.966+0000 7f40b27fc640 1 -- 172.21.15.76:0/1622050035 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f40b8040ca0 con 0x7f40c00ade00 2023-11-09T03:56:02.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.966+0000 7f40b27fc640 1 --2- 172.21.15.76:0/1622050035 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f40a406ae70 0x7f40a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:02.973 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.966+0000 7f40b27fc640 1 -- 172.21.15.76:0/1622050035 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8582+0+0 (secure 0 0 0) 0x7f40b80acf60 con 0x7f40c00ade00 2023-11-09T03:56:02.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.966+0000 7f40c5c70640 1 --2- 172.21.15.76:0/1622050035 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f40a406ae70 0x7f40a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:02.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.967+0000 7f40c5c70640 1 --2- 172.21.15.76:0/1622050035 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f40a406ae70 0x7f40a406d330 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f40b4000f60 tx=0x7f40b4007c10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:02.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:02.971+0000 7f40b27fc640 1 -- 172.21.15.76:0/1622050035 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f40b8077630 con 0x7f40c00ade00 2023-11-09T03:56:03.070 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:56:03.072 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:56:03.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:03.175+0000 7f40c7ed4640 1 -- 172.21.15.76:0/1622050035 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40c00c3f90 con 0x7f40c00ade00 2023-11-09T03:56:03.175 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:03.176+0000 7f40b27fc640 1 -- 172.21.15.76:0/1622050035 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) v1 ==== 95+0+4935 (secure 0 0 0) 0x7f40c00c3f90 con 0x7f40c00ade00 2023-11-09T03:56:03.176 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:56:03.176 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":143,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":9525,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1713748386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1713748386},{"type":"v1","addr":"172.21.15.76:6835","nonce":1713748386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":138}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:55:51.132628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":6698},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6698":{"gid":6698,"name":"b","rank":1,"incarnation":137,"state":"up:active","state_seq":18,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:56:03.176 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 143 2023-11-09T03:56:03.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:03.178+0000 7f40c7ed4640 1 -- 172.21.15.76:0/1622050035 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f40a406ae70 msgr2=0x7f40a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:03.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:03.178+0000 7f40c7ed4640 1 --2- 172.21.15.76:0/1622050035 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f40a406ae70 0x7f40a406d330 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f40b4000f60 tx=0x7f40b4007c10 comp rx=0 tx=0).stop 2023-11-09T03:56:03.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:03.178+0000 7f40c7ed4640 1 -- 172.21.15.76:0/1622050035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40c00ade00 msgr2=0x7f40c00af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:03.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:03.178+0000 7f40c7ed4640 1 --2- 172.21.15.76:0/1622050035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40c00ade00 0x7f40c00af6e0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f40b8037040 tx=0x7f40b8004a40 comp rx=0 tx=0).stop 2023-11-09T03:56:03.178 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:03.178+0000 7f40c7ed4640 1 -- 172.21.15.76:0/1622050035 shutdown_connections 2023-11-09T03:56:03.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:03.178+0000 7f40c7ed4640 1 --2- 172.21.15.76:0/1622050035 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f40a406ae70 0x7f40a406d330 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:03.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:03.178+0000 7f40c7ed4640 1 --2- 172.21.15.76:0/1622050035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40c00ade00 0x7f40c00af6e0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:03.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:03.178+0000 7f40c7ed4640 1 --2- 172.21.15.76:0/1622050035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c006d9c0 0x7f40c00af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:03.179 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:03.178+0000 7f40c7ed4640 1 --2- 172.21.15.76:0/1622050035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c006c5f0 0x7f40c01591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:03.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:03.178+0000 7f40c7ed4640 1 -- 172.21.15.76:0/1622050035 >> 172.21.15.76:0/1622050035 conn(0x7f40c00cb530 msgr2=0x7f40c00cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:03.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:03.178+0000 7f40c7ed4640 1 -- 172.21.15.76:0/1622050035 shutdown_connections 2023-11-09T03:56:03.180 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:03.178+0000 7f40c7ed4640 1 -- 172.21.15.76:0/1622050035 wait complete. 2023-11-09T03:56:03.188 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-09T03:56:05.401 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~7s 2023-11-09T03:56:08.190 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T03:56:08.300 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.300+0000 7f91795fb640 1 Processor -- start 2023-11-09T03:56:08.301 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.301+0000 7f91795fb640 1 -- start start 2023-11-09T03:56:08.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.301+0000 7f91795fb640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9174159e40 0x7f91740b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:08.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.301+0000 7f91795fb640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9174071500 con 0x7f9174070700 2023-11-09T03:56:08.308 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.301+0000 7f91795fb640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f917415dae0 con 0x7f917406bb60 2023-11-09T03:56:08.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.301+0000 7f91795fb640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9174070c60 con 0x7f9174159e40 2023-11-09T03:56:08.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.301+0000 7f91737fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f9174070700 0x7f91740cd790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47800/0 (socket says 172.21.15.76:47800) 2023-11-09T03:56:08.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.301+0000 7f91737fe640 1 -- 172.21.15.76:0/110082152 learned_addr learned my addr 172.21.15.76:0/110082152 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:56:08.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.302+0000 7f9172ffd640 1 --2- 172.21.15.76:0/110082152 >> v2:172.21.15.113:3300/0 conn(0x7f9174159e40 0x7f91740b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:08.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.302+0000 7f9171ffb640 1 -- 172.21.15.76:0/110082152 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2257377406 0 0) 0x7f9174071500 con 0x7f9174070700 2023-11-09T03:56:08.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.302+0000 7f9171ffb640 1 -- 172.21.15.76:0/110082152 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f915c003520 con 0x7f9174070700 2023-11-09T03:56:08.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.302+0000 7f9171ffb640 1 -- 172.21.15.76:0/110082152 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2726510592 0 0) 0x7f915c003520 con 0x7f9174070700 2023-11-09T03:56:08.310 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.302+0000 7f9171ffb640 1 -- 172.21.15.76:0/110082152 >> v2:172.21.15.113:3300/0 conn(0x7f9174159e40 msgr2=0x7f91740b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:08.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.302+0000 7f9171ffb640 1 --2- 172.21.15.76:0/110082152 >> v2:172.21.15.113:3300/0 conn(0x7f9174159e40 0x7f91740b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:08.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.302+0000 7f9171ffb640 1 -- 172.21.15.76:0/110082152 >> v1:172.21.15.113:6789/0 conn(0x7f917406bb60 legacy=0x7f917406fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:08.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.302+0000 7f9171ffb640 1 -- 172.21.15.76:0/110082152 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9174070f40 con 0x7f9174070700 2023-11-09T03:56:08.311 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.302+0000 7f9171ffb640 1 -- 172.21.15.76:0/110082152 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f916c0019f0 con 0x7f9174070700 2023-11-09T03:56:08.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.302+0000 7f9171ffb640 1 -- 172.21.15.76:0/110082152 >> v1:172.21.15.76:6789/0 conn(0x7f9174070700 legacy=0x7f91740cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:08.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.302+0000 7f9171ffb640 1 --2- 172.21.15.76:0/110082152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f915c003970 0x7f915c003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:08.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.302+0000 7f9171ffb640 1 --2- 172.21.15.76:0/110082152 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f915c0047c0 0x7f915c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:08.312 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.302+0000 7f9171ffb640 1 --2- 172.21.15.76:0/110082152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f915c005180 0x7f915c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:08.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.302+0000 7f9171ffb640 1 -- 172.21.15.76:0/110082152 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9174070c60 con 0x7f915c005180 2023-11-09T03:56:08.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.302+0000 7f9171ffb640 1 -- 172.21.15.76:0/110082152 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915c00c0d0 con 0x7f915c003970 2023-11-09T03:56:08.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.302+0000 7f9171ffb640 1 -- 172.21.15.76:0/110082152 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f915c00c290 con 0x7f915c0047c0 2023-11-09T03:56:08.313 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.302+0000 7f91737fe640 1 --2- 172.21.15.76:0/110082152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f915c005180 0x7f915c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:08.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.302+0000 7f9172ffd640 1 --2- 172.21.15.76:0/110082152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f915c003970 0x7f915c003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:08.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.302+0000 7f91727fc640 1 --2- 172.21.15.76:0/110082152 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f915c0047c0 0x7f915c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:08.314 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.303+0000 7f91737fe640 1 -- 172.21.15.76:0/110082152 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f915c0047c0 msgr2=0x7f915c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:08.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.303+0000 7f91737fe640 1 --2- 172.21.15.76:0/110082152 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f915c0047c0 0x7f915c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:08.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.303+0000 7f91737fe640 1 -- 172.21.15.76:0/110082152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f915c003970 msgr2=0x7f915c003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:08.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.303+0000 7f91737fe640 1 --2- 172.21.15.76:0/110082152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f915c003970 0x7f915c003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:08.315 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.303+0000 7f91737fe640 1 -- 172.21.15.76:0/110082152 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f915c003520 con 0x7f915c005180 2023-11-09T03:56:08.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.303+0000 7f91737fe640 1 -- 172.21.15.76:0/110082152 reap_dead start 2023-11-09T03:56:08.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.303+0000 7f91737fe640 1 --2- 172.21.15.76:0/110082152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f915c005180 0x7f915c00b980 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f916c00a7d0 tx=0x7f915c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f501c1b8b8bdc76 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:08.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.303+0000 7f9171ffb640 1 -- 172.21.15.76:0/110082152 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c034630 con 0x7f915c005180 2023-11-09T03:56:08.316 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.303+0000 7f9171ffb640 1 -- 172.21.15.76:0/110082152 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916c02f070 con 0x7f915c005180 2023-11-09T03:56:08.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.303+0000 7f9171ffb640 1 -- 172.21.15.76:0/110082152 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c0349f0 con 0x7f915c005180 2023-11-09T03:56:08.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.304+0000 7f91795fb640 1 -- 172.21.15.76:0/110082152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f915c005180 msgr2=0x7f915c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:08.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.304+0000 7f91795fb640 1 --2- 172.21.15.76:0/110082152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f915c005180 0x7f915c00b980 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f916c00a7d0 tx=0x7f915c0057f0 comp rx=0 tx=0).stop 2023-11-09T03:56:08.317 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.304+0000 7f91795fb640 1 -- 172.21.15.76:0/110082152 shutdown_connections 2023-11-09T03:56:08.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.304+0000 7f91795fb640 1 --2- 172.21.15.76:0/110082152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f915c005180 0x7f915c00b980 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:08.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.304+0000 7f91795fb640 1 -- 172.21.15.76:0/110082152 >> 172.21.15.76:0/110082152 conn(0x7f91740cb530 msgr2=0x7f91740cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:08.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.304+0000 7f91795fb640 1 -- 172.21.15.76:0/110082152 shutdown_connections 2023-11-09T03:56:08.318 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.304+0000 7f91795fb640 1 -- 172.21.15.76:0/110082152 wait complete. 2023-11-09T03:56:08.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.305+0000 7f91795fb640 1 Processor -- start 2023-11-09T03:56:08.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.305+0000 7f91795fb640 1 -- start start 2023-11-09T03:56:08.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.306+0000 7f91795fb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f915c005180 0x7f917414eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:08.319 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.306+0000 7f91795fb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9174153f50 0x7f91740ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:08.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.306+0000 7f91795fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91740ca9c0 0x7f9174154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:08.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.306+0000 7f91795fb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91740cae40 con 0x7f91740ca9c0 2023-11-09T03:56:08.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.306+0000 7f91795fb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9174152eb0 con 0x7f9174153f50 2023-11-09T03:56:08.320 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.306+0000 7f91795fb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9174153190 con 0x7f915c005180 2023-11-09T03:56:08.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.306+0000 7f91737fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91740ca9c0 0x7f9174154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:08.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.306+0000 7f91737fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91740ca9c0 0x7f9174154850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40320/0 (socket says 172.21.15.76:40320) 2023-11-09T03:56:08.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.306+0000 7f91737fe640 1 -- 172.21.15.76:0/3893007911 learned_addr learned my addr 172.21.15.76:0/3893007911 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:56:08.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.306+0000 7f9172ffd640 1 --2- 172.21.15.76:0/3893007911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f915c005180 0x7f917414eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:08.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.306+0000 7f91727fc640 1 --2- 172.21.15.76:0/3893007911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9174153f50 0x7f91740ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:08.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.306+0000 7f91737fe640 1 -- 172.21.15.76:0/3893007911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f915c005180 msgr2=0x7f917414eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:08.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.306+0000 7f91737fe640 1 --2- 172.21.15.76:0/3893007911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f915c005180 0x7f917414eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:08.322 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.307+0000 7f91737fe640 1 -- 172.21.15.76:0/3893007911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9174153f50 msgr2=0x7f91740ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:08.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.307+0000 7f91737fe640 1 --2- 172.21.15.76:0/3893007911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9174153f50 0x7f91740ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:08.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.307+0000 7f91737fe640 1 -- 172.21.15.76:0/3893007911 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9174153470 con 0x7f91740ca9c0 2023-11-09T03:56:08.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.307+0000 7f9172ffd640 1 --2- 172.21.15.76:0/3893007911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f915c005180 0x7f917414eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:56:08.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.307+0000 7f91727fc640 1 --2- 172.21.15.76:0/3893007911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9174153f50 0x7f91740ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:56:08.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.307+0000 7f91737fe640 1 --2- 172.21.15.76:0/3893007911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91740ca9c0 0x7f9174154850 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f916c02cbc0 tx=0x7f916c001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:08.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.307+0000 7f915b7fe640 1 -- 172.21.15.76:0/3893007911 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c004d60 con 0x7f91740ca9c0 2023-11-09T03:56:08.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.307+0000 7f915b7fe640 1 -- 172.21.15.76:0/3893007911 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916c034430 con 0x7f91740ca9c0 2023-11-09T03:56:08.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.307+0000 7f91795fb640 1 -- 172.21.15.76:0/3893007911 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9174153750 con 0x7f91740ca9c0 2023-11-09T03:56:08.324 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.307+0000 7f915b7fe640 1 -- 172.21.15.76:0/3893007911 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c0347a0 con 0x7f91740ca9c0 2023-11-09T03:56:08.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.307+0000 7f91795fb640 1 -- 172.21.15.76:0/3893007911 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91741543d0 con 0x7f91740ca9c0 2023-11-09T03:56:08.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.308+0000 7f91795fb640 1 -- 172.21.15.76:0/3893007911 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9174153c70 con 0x7f91740ca9c0 2023-11-09T03:56:08.325 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.308+0000 7f915b7fe640 1 -- 172.21.15.76:0/3893007911 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f916c031030 con 0x7f91740ca9c0 2023-11-09T03:56:08.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.309+0000 7f915b7fe640 1 --2- 172.21.15.76:0/3893007911 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f916006aea0 0x7f916006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:08.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.309+0000 7f915b7fe640 1 -- 172.21.15.76:0/3893007911 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8582+0+0 (secure 0 0 0) 0x7f916c0abea0 con 0x7f91740ca9c0 2023-11-09T03:56:08.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.309+0000 7f9172ffd640 1 --2- 172.21.15.76:0/3893007911 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f916006aea0 0x7f916006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:08.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.309+0000 7f9172ffd640 1 --2- 172.21.15.76:0/3893007911 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f916006aea0 0x7f916006d360 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f9164009a40 tx=0x7f9164008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:08.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.313+0000 7f915b7fe640 1 -- 172.21.15.76:0/3893007911 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9174153c70 con 0x7f91740ca9c0 2023-11-09T03:56:08.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.515+0000 7f91795fb640 1 -- 172.21.15.76:0/3893007911 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9174153c70 con 0x7f91740ca9c0 2023-11-09T03:56:08.515 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.515+0000 7f915b7fe640 1 -- 172.21.15.76:0/3893007911 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 144 v144) v1 ==== 95+0+4983 (secure 0 0 0) 0x7f9174153c70 con 0x7f91740ca9c0 2023-11-09T03:56:08.516 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:56:08.516 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":144,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5741,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6834/387993570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":387993570},{"type":"v1","addr":"172.21.15.113:6834","nonce":387993570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":9525,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1713748386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1713748386},{"type":"v1","addr":"172.21.15.76:6835","nonce":1713748386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":138}],"filesystems":[{"mdsmap":{"epoch":144,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-09T03:48:57.383848+0000","modified":"2023-11-09T03:56:06.506266+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8568,"mds_1":6698},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6698":{"gid":6698,"name":"b","rank":1,"incarnation":137,"state":"up:active","state_seq":18,"addr":"172.21.15.113:6837/366683575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":366683575},{"type":"v1","addr":"172.21.15.113:6837","nonce":366683575}]},"join_fscid":-1,"laggy_since":"2023-11-09T03:56:06.506266+0000","export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8568":{"gid":8568,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":18,"addr":"172.21.15.76:6833/153465374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":153465374},{"type":"v1","addr":"172.21.15.76:6833","nonce":153465374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-09T03:56:08.517 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 144 2023-11-09T03:56:08.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.517+0000 7f91795fb640 1 -- 172.21.15.76:0/3893007911 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f916006aea0 msgr2=0x7f916006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:08.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.517+0000 7f91795fb640 1 --2- 172.21.15.76:0/3893007911 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f916006aea0 0x7f916006d360 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f9164009a40 tx=0x7f9164008040 comp rx=0 tx=0).stop 2023-11-09T03:56:08.518 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.517+0000 7f91795fb640 1 -- 172.21.15.76:0/3893007911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91740ca9c0 msgr2=0x7f9174154850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:08.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.517+0000 7f91795fb640 1 --2- 172.21.15.76:0/3893007911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91740ca9c0 0x7f9174154850 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f916c02cbc0 tx=0x7f916c001d00 comp rx=0 tx=0).stop 2023-11-09T03:56:08.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.517+0000 7f91795fb640 1 -- 172.21.15.76:0/3893007911 shutdown_connections 2023-11-09T03:56:08.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.517+0000 7f91795fb640 1 --2- 172.21.15.76:0/3893007911 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f916006aea0 0x7f916006d360 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:08.519 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.517+0000 7f91795fb640 1 --2- 172.21.15.76:0/3893007911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91740ca9c0 0x7f9174154850 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:08.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.517+0000 7f91795fb640 1 --2- 172.21.15.76:0/3893007911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9174153f50 0x7f91740ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:08.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.517+0000 7f91795fb640 1 --2- 172.21.15.76:0/3893007911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f915c005180 0x7f917414eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:08.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.517+0000 7f91795fb640 1 -- 172.21.15.76:0/3893007911 >> 172.21.15.76:0/3893007911 conn(0x7f91740cb530 msgr2=0x7f91740ad820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:08.520 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.517+0000 7f91795fb640 1 -- 172.21.15.76:0/3893007911 shutdown_connections 2023-11-09T03:56:08.521 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.517+0000 7f91795fb640 1 -- 172.21.15.76:0/3893007911 wait complete. 2023-11-09T03:56:08.528 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2023-11-09T03:56:08.529 DEBUG:teuthology.orchestra.run.smithi113:> sudo sysctl -n kernel.core_pattern 2023-11-09T03:56:08.560 INFO:teuthology.orchestra.run.smithi113.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-09T03:56:08.561 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2023-11-09T03:56:08.561 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2023-11-09T03:56:08.635 INFO:teuthology.orchestra.run.smithi113.stdout:1699502152.96518.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i b', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2023-11-09T03:56:08.637 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1699502152.96518.core'] 2023-11-09T03:56:08.638 INFO:tasks.cephfs.cephfs_test_case:Found core file 1699502152.96518.core, deleting it 2023-11-09T03:56:08.638 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1699502152.96518.core 2023-11-09T03:56:08.711 INFO:tasks.cephfs.mount:Killing connection on ubuntu@smithi076.front.sepia.ceph.com... 2023-11-09T03:56:08.711 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-09T03:56:08.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:56:08.712 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 down 2023-11-09T03:56:08.740 INFO:tasks.cephfs.mount:Cleaning up killed connection on ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T03:56:08.741 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:56:08.741 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:56:08.796 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:56:08.838 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T03:56:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:56:08.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.936+0000 7f659eeb4640 1 Processor -- start 2023-11-09T03:56:08.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.936+0000 7f659eeb4640 1 -- start start 2023-11-09T03:56:08.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.937+0000 7f659eeb4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f659815d100 0x7f65980ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:08.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.937+0000 7f659eeb4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65980b32b0 con 0x7f65980d1be0 2023-11-09T03:56:08.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.937+0000 7f659eeb4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65980cee10 con 0x7f659815d4e0 2023-11-09T03:56:08.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.937+0000 7f659eeb4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65980b2980 con 0x7f659815d100 2023-11-09T03:56:08.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.937+0000 7f659d451640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f65980d1be0 0x7f65980b3740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47812/0 (socket says 172.21.15.76:47812) 2023-11-09T03:56:08.942 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.937+0000 7f659d451640 1 -- 172.21.15.76:0/26592217 learned_addr learned my addr 172.21.15.76:0/26592217 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:56:08.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.938+0000 7f659cc50640 1 --2- 172.21.15.76:0/26592217 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f659815d100 0x7f65980ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:08.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.938+0000 7f65977fe640 1 -- 172.21.15.76:0/26592217 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1214276353 0 0) 0x7f65980b32b0 con 0x7f65980d1be0 2023-11-09T03:56:08.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.938+0000 7f65977fe640 1 -- 172.21.15.76:0/26592217 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6584003520 con 0x7f65980d1be0 2023-11-09T03:56:08.943 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.938+0000 7f65977fe640 1 -- 172.21.15.76:0/26592217 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3057812985 0 0) 0x7f65980cee10 con 0x7f659815d4e0 2023-11-09T03:56:08.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.938+0000 7f65977fe640 1 -- 172.21.15.76:0/26592217 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65980b32b0 con 0x7f659815d4e0 2023-11-09T03:56:08.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.938+0000 7f65977fe640 1 -- 172.21.15.76:0/26592217 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3139201719 0 0) 0x7f6584003520 con 0x7f65980d1be0 2023-11-09T03:56:08.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.938+0000 7f65977fe640 1 -- 172.21.15.76:0/26592217 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f659815d100 msgr2=0x7f65980ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:08.944 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.938+0000 7f65977fe640 1 --2- 172.21.15.76:0/26592217 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f659815d100 0x7f65980ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:08.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.938+0000 7f65977fe640 1 -- 172.21.15.76:0/26592217 >> v1:172.21.15.113:6789/0 conn(0x7f659815d4e0 legacy=0x7f65980d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:08.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.938+0000 7f65977fe640 1 -- 172.21.15.76:0/26592217 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65980b2c60 con 0x7f65980d1be0 2023-11-09T03:56:08.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.938+0000 7f65977fe640 1 -- 172.21.15.76:0/26592217 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f65900019f0 con 0x7f65980d1be0 2023-11-09T03:56:08.945 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.938+0000 7f659cc50640 1 --2- 172.21.15.76:0/26592217 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f659815d100 0x7f65980ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:56:08.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.938+0000 7f65977fe640 1 -- 172.21.15.76:0/26592217 >> v1:172.21.15.76:6789/0 conn(0x7f65980d1be0 legacy=0x7f65980b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:08.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.938+0000 7f65977fe640 1 --2- 172.21.15.76:0/26592217 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f659815d100 0x7f6584003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:08.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.938+0000 7f65977fe640 1 --2- 172.21.15.76:0/26592217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6584004070 0x7f65840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:08.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.938+0000 7f65977fe640 1 --2- 172.21.15.76:0/26592217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6584004d00 0x7f6584005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:08.946 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.938+0000 7f65977fe640 1 -- 172.21.15.76:0/26592217 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65980b2980 con 0x7f6584004d00 2023-11-09T03:56:08.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.938+0000 7f65977fe640 1 -- 172.21.15.76:0/26592217 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658400ba30 con 0x7f6584004070 2023-11-09T03:56:08.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.938+0000 7f65977fe640 1 -- 172.21.15.76:0/26592217 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f658400bbf0 con 0x7f659815d100 2023-11-09T03:56:08.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.938+0000 7f659d451640 1 --2- 172.21.15.76:0/26592217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6584004d00 0x7f6584005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:08.947 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.938+0000 7f659cc50640 1 --2- 172.21.15.76:0/26592217 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f659815d100 0x7f6584003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:08.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.938+0000 7f6597fff640 1 --2- 172.21.15.76:0/26592217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6584004070 0x7f65840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:08.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.938+0000 7f659d451640 1 -- 172.21.15.76:0/26592217 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f659815d100 msgr2=0x7f6584003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:08.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.938+0000 7f659d451640 1 --2- 172.21.15.76:0/26592217 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f659815d100 0x7f6584003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:08.948 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.938+0000 7f659d451640 1 -- 172.21.15.76:0/26592217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6584004070 msgr2=0x7f65840047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:08.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.938+0000 7f659d451640 1 --2- 172.21.15.76:0/26592217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6584004070 0x7f65840047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:08.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.938+0000 7f659d451640 1 -- 172.21.15.76:0/26592217 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65880093a0 con 0x7f6584004d00 2023-11-09T03:56:08.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.938+0000 7f6597fff640 1 --2- 172.21.15.76:0/26592217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6584004070 0x7f65840047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:56:08.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.939+0000 7f659d451640 1 --2- 172.21.15.76:0/26592217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6584004d00 0x7f6584005180 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f659808c730 tx=0x7f65840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f9f71603b26906 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:08.949 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.939+0000 7f65977fe640 1 -- 172.21.15.76:0/26592217 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6590034630 con 0x7f6584004d00 2023-11-09T03:56:08.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.939+0000 7f65977fe640 1 -- 172.21.15.76:0/26592217 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659002f070 con 0x7f6584004d00 2023-11-09T03:56:08.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.939+0000 7f65977fe640 1 -- 172.21.15.76:0/26592217 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65900349f0 con 0x7f6584004d00 2023-11-09T03:56:08.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.940+0000 7f659eeb4640 1 -- 172.21.15.76:0/26592217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6584004d00 msgr2=0x7f6584005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:08.950 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.940+0000 7f659eeb4640 1 --2- 172.21.15.76:0/26592217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6584004d00 0x7f6584005180 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f659808c730 tx=0x7f65840057f0 comp rx=0 tx=0).stop 2023-11-09T03:56:08.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.940+0000 7f659d451640 1 -- 172.21.15.76:0/26592217 reap_dead start 2023-11-09T03:56:08.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.940+0000 7f659eeb4640 1 -- 172.21.15.76:0/26592217 shutdown_connections 2023-11-09T03:56:08.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.940+0000 7f659eeb4640 1 -- 172.21.15.76:0/26592217 >> 172.21.15.76:0/26592217 conn(0x7f65980d2ef0 msgr2=0x7f65980d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:08.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.940+0000 7f659eeb4640 1 -- 172.21.15.76:0/26592217 shutdown_connections 2023-11-09T03:56:08.951 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.940+0000 7f659eeb4640 1 -- 172.21.15.76:0/26592217 wait complete. 2023-11-09T03:56:08.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.941+0000 7f659eeb4640 1 Processor -- start 2023-11-09T03:56:08.952 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.941+0000 7f659eeb4640 1 -- start start 2023-11-09T03:56:08.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.942+0000 7f659eeb4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6584004d00 0x7f659814f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:08.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.942+0000 7f659eeb4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65980b6df0 0x7f65980b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:08.955 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.942+0000 7f659eeb4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f659814fda0 0x7f6598150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:08.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.942+0000 7f659eeb4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65980b5a70 con 0x7f65980b6df0 2023-11-09T03:56:08.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.942+0000 7f659eeb4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65980b5d50 con 0x7f659814fda0 2023-11-09T03:56:08.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.942+0000 7f659eeb4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65980b6030 con 0x7f6584004d00 2023-11-09T03:56:08.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.942+0000 7f6597fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65980b6df0 0x7f65980b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:08.956 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.942+0000 7f6597fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65980b6df0 0x7f65980b7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40344/0 (socket says 172.21.15.76:40344) 2023-11-09T03:56:08.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.942+0000 7f6597fff640 1 -- 172.21.15.76:0/248443954 learned_addr learned my addr 172.21.15.76:0/248443954 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:56:08.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.942+0000 7f659d451640 1 --2- 172.21.15.76:0/248443954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f659814fda0 0x7f6598150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:08.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.942+0000 7f659cc50640 1 --2- 172.21.15.76:0/248443954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6584004d00 0x7f659814f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:08.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.942+0000 7f6597fff640 1 -- 172.21.15.76:0/248443954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6584004d00 msgr2=0x7f659814f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:08.957 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.942+0000 7f6597fff640 1 --2- 172.21.15.76:0/248443954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6584004d00 0x7f659814f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:08.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.942+0000 7f6597fff640 1 -- 172.21.15.76:0/248443954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f659814fda0 msgr2=0x7f6598150220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:08.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.942+0000 7f6597fff640 1 --2- 172.21.15.76:0/248443954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f659814fda0 0x7f6598150220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:08.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.942+0000 7f6597fff640 1 -- 172.21.15.76:0/248443954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65980b6310 con 0x7f65980b6df0 2023-11-09T03:56:08.958 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.943+0000 7f6597fff640 1 --2- 172.21.15.76:0/248443954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65980b6df0 0x7f65980b7250 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f658000e780 tx=0x7f658000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:08.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.943+0000 7f65957fa640 1 -- 172.21.15.76:0/248443954 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6580017070 con 0x7f65980b6df0 2023-11-09T03:56:08.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.943+0000 7f65957fa640 1 -- 172.21.15.76:0/248443954 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65800050d0 con 0x7f65980b6df0 2023-11-09T03:56:08.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.943+0000 7f659eeb4640 1 -- 172.21.15.76:0/248443954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65980b65f0 con 0x7f65980b6df0 2023-11-09T03:56:08.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.943+0000 7f65957fa640 1 -- 172.21.15.76:0/248443954 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6580005240 con 0x7f65980b6df0 2023-11-09T03:56:08.959 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.943+0000 7f659eeb4640 1 -- 172.21.15.76:0/248443954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f659814f760 con 0x7f65980b6df0 2023-11-09T03:56:08.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.944+0000 7f659eeb4640 1 -- 172.21.15.76:0/248443954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65980b6b10 con 0x7f65980b6df0 2023-11-09T03:56:08.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.944+0000 7f65957fa640 1 -- 172.21.15.76:0/248443954 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6580007020 con 0x7f65980b6df0 2023-11-09T03:56:08.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.944+0000 7f65957fa640 1 --2- 172.21.15.76:0/248443954 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f657806ae70 0x7f657806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:08.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.944+0000 7f65957fa640 1 -- 172.21.15.76:0/248443954 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8582+0+0 (secure 0 0 0) 0x7f6580089a30 con 0x7f65980b6df0 2023-11-09T03:56:08.960 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.945+0000 7f659cc50640 1 --2- 172.21.15.76:0/248443954 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f657806ae70 0x7f657806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:08.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.945+0000 7f659cc50640 1 --2- 172.21.15.76:0/248443954 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f657806ae70 0x7f657806d330 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f6588009c80 tx=0x7f6588008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:08.961 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:08.949+0000 7f65957fa640 1 -- 172.21.15.76:0/248443954 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f65980b6b10 con 0x7f65980b6df0 2023-11-09T03:56:09.118 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.118+0000 7f659eeb4640 1 -- 172.21.15.76:0/248443954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65980b6b10 con 0x7f65980b6df0 2023-11-09T03:56:09.120 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.120+0000 7f65957fa640 1 -- 172.21.15.76:0/248443954 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+24268 (secure 0 0 0) 0x7f6580053f50 con 0x7f65980b6df0 2023-11-09T03:56:09.120 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:56:09.120 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":123,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:55:56.504148+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-09T03:47:59.280601+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-09T03:47:59.742039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:48:55.490832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-09T03:48:56.352624+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":122,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.17","mappings":[{"from":0,"to":2}]},{"pgid":"17.36","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/497547142":"2023-11-09T04:55:40.451393+0000","172.21.15.76:6834/757529522":"2023-11-10T03:55:31.678438+0000","172.21.15.76:6832/3697850101":"2023-11-10T03:51:39.442929+0000","172.21.15.113:6837/3449910842":"2023-11-10T03:50:59.611256+0000","172.21.15.76:6832/1536527256":"2023-11-10T03:52:34.213804+0000","172.21.15.76:6835/3229239133":"2023-11-10T03:51:32.624856+0000","172.21.15.113:6836/445306598":"2023-11-10T03:52:27.406838+0000","172.21.15.76:6835/1663988060":"2023-11-10T03:50:33.665092+0000","172.21.15.76:6834/1663988060":"2023-11-10T03:50:33.665092+0000","172.21.15.113:6837/3280098776":"2023-11-10T03:50:07.616780+0000","192.168.0.1:0/2519773110":"2023-11-09T04:54:14.823765+0000","172.21.15.76:6835/4011275120":"2023-11-10T03:49:42.029057+0000","172.21.15.76:6835/757529522":"2023-11-10T03:55:31.678438+0000","172.21.15.76:6834/4011275120":"2023-11-10T03:49:42.029057+0000","172.21.15.113:6837/3462040782":"2023-11-10T03:54:45.345437+0000","192.168.0.1:0/503424885":"2023-11-09T04:51:47.727422+0000","172.21.15.76:6834/3229239133":"2023-11-10T03:51:32.624856+0000","172.21.15.113:6837/445306598":"2023-11-10T03:52:27.406838+0000","172.21.15.113:6836/3280098776":"2023-11-10T03:50:07.616780+0000","192.168.0.1:0/3087804343":"2023-11-09T04:52:42.513556+0000","172.21.15.76:6834/546338722":"2023-11-10T03:53:17.470106+0000","172.21.15.76:6833/1536527256":"2023-11-10T03:52:34.213804+0000","172.21.15.76:6835/546338722":"2023-11-10T03:53:17.470106+0000","172.21.15.113:6836/7360844":"2023-11-10T03:53:24.718008+0000","172.21.15.113:6837/7360844":"2023-11-10T03:53:24.718008+0000","172.21.15.76:6835/2075292158":"2023-11-10T03:54:06.568191+0000","192.168.0.1:0/334503200":"2023-11-09T04:53:32.572840+0000","172.21.15.76:6834/2075292158":"2023-11-10T03:54:06.568191+0000","172.21.15.76:6833/3697850101":"2023-11-10T03:51:39.442929+0000","172.21.15.113:6836/3462040782":"2023-11-10T03:54:45.345437+0000","172.21.15.113:6836/3449910842":"2023-11-10T03:50:59.611256+0000","192.168.0.1:0/3870807407":"2023-11-09T04:54:56.671385+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":17,"snaps":[{"begin":16,"length":4}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:56:09.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.122+0000 7f659eeb4640 1 -- 172.21.15.76:0/248443954 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f657806ae70 msgr2=0x7f657806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:09.122 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.122+0000 7f659eeb4640 1 --2- 172.21.15.76:0/248443954 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f657806ae70 0x7f657806d330 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f6588009c80 tx=0x7f6588008040 comp rx=0 tx=0).stop 2023-11-09T03:56:09.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.122+0000 7f659eeb4640 1 -- 172.21.15.76:0/248443954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65980b6df0 msgr2=0x7f65980b7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:09.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.122+0000 7f659eeb4640 1 --2- 172.21.15.76:0/248443954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65980b6df0 0x7f65980b7250 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f658000e780 tx=0x7f658000ec40 comp rx=0 tx=0).stop 2023-11-09T03:56:09.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.122+0000 7f659eeb4640 1 -- 172.21.15.76:0/248443954 shutdown_connections 2023-11-09T03:56:09.123 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.122+0000 7f659eeb4640 1 --2- 172.21.15.76:0/248443954 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f657806ae70 0x7f657806d330 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.122+0000 7f659eeb4640 1 --2- 172.21.15.76:0/248443954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f659814fda0 0x7f6598150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.122+0000 7f659eeb4640 1 --2- 172.21.15.76:0/248443954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65980b6df0 0x7f65980b7250 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.122+0000 7f659eeb4640 1 --2- 172.21.15.76:0/248443954 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6584004d00 0x7f659814f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.122+0000 7f659eeb4640 1 -- 172.21.15.76:0/248443954 >> 172.21.15.76:0/248443954 conn(0x7f65980d2ef0 msgr2=0x7f65980b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:09.124 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.122+0000 7f659eeb4640 1 -- 172.21.15.76:0/248443954 shutdown_connections 2023-11-09T03:56:09.125 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.122+0000 7f659eeb4640 1 -- 172.21.15.76:0/248443954 wait complete. 2023-11-09T03:56:09.133 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-09T03:47:00.036540+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_data_pool', 'create_time': '2023-11-09T03:47:59.280601+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 70, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-09T03:47:59.742039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:48:55.490832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:48:56.352624+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '122', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 122, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-09T03:56:09.134 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:56:09.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.238+0000 7f155504f640 1 Processor -- start 2023-11-09T03:56:09.238 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.238+0000 7f155504f640 1 -- start start 2023-11-09T03:56:09.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.239+0000 7f155504f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f155015d4e0 0x7f15500d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:09.241 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.239+0000 7f155504f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15500d3b60 0x7f15500d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:09.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.239+0000 7f155504f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15500b32b0 con 0x7f155015d100 2023-11-09T03:56:09.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.239+0000 7f155504f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15500cec20 con 0x7f155015d4e0 2023-11-09T03:56:09.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.239+0000 7f155504f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15500cee10 con 0x7f15500d3b60 2023-11-09T03:56:09.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.239+0000 7f154e59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f155015d4e0 0x7f15500d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:09.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.239+0000 7f154e59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f155015d4e0 0x7f15500d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40360/0 (socket says 172.21.15.76:40360) 2023-11-09T03:56:09.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.239+0000 7f154e59c640 1 -- 172.21.15.76:0/2960663438 learned_addr learned my addr 172.21.15.76:0/2960663438 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:56:09.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.239+0000 7f154f59e640 1 --2- 172.21.15.76:0/2960663438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15500d3b60 0x7f15500d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:09.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.240+0000 7f154e59c640 1 -- 172.21.15.76:0/2960663438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15500d3b60 msgr2=0x7f15500d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:09.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.240+0000 7f154e59c640 1 --2- 172.21.15.76:0/2960663438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15500d3b60 0x7f15500d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.240+0000 7f154e59c640 1 -- 172.21.15.76:0/2960663438 >> v1:172.21.15.113:6789/0 conn(0x7f155015d100 legacy=0x7f15500ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:09.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.240+0000 7f154e59c640 1 -- 172.21.15.76:0/2960663438 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15500b4750 con 0x7f155015d4e0 2023-11-09T03:56:09.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.240+0000 7f154e59c640 1 --2- 172.21.15.76:0/2960663438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f155015d4e0 0x7f15500d14d0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f1544009870 tx=0x7f154402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7575f55dd0a9c7e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:09.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.240+0000 7f154dd9b640 1 -- 172.21.15.76:0/2960663438 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1544030840 con 0x7f155015d4e0 2023-11-09T03:56:09.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.241+0000 7f154dd9b640 1 -- 172.21.15.76:0/2960663438 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15440309a0 con 0x7f155015d4e0 2023-11-09T03:56:09.245 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.241+0000 7f154dd9b640 1 -- 172.21.15.76:0/2960663438 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1544030d30 con 0x7f155015d4e0 2023-11-09T03:56:09.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.241+0000 7f155504f640 1 -- 172.21.15.76:0/2960663438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f155015d4e0 msgr2=0x7f15500d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:09.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.241+0000 7f155504f640 1 --2- 172.21.15.76:0/2960663438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f155015d4e0 0x7f15500d14d0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f1544009870 tx=0x7f154402b980 comp rx=0 tx=0).stop 2023-11-09T03:56:09.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.241+0000 7f155504f640 1 -- 172.21.15.76:0/2960663438 shutdown_connections 2023-11-09T03:56:09.246 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.241+0000 7f155504f640 1 --2- 172.21.15.76:0/2960663438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15500d3b60 0x7f15500d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.241+0000 7f155504f640 1 --2- 172.21.15.76:0/2960663438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f155015d4e0 0x7f15500d14d0 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.241+0000 7f155504f640 1 -- 172.21.15.76:0/2960663438 >> 172.21.15.76:0/2960663438 conn(0x7f15500d2ef0 msgr2=0x7f15500d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:09.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.241+0000 7f155504f640 1 -- 172.21.15.76:0/2960663438 shutdown_connections 2023-11-09T03:56:09.247 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.241+0000 7f155504f640 1 -- 172.21.15.76:0/2960663438 wait complete. 2023-11-09T03:56:09.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.243+0000 7f155504f640 1 Processor -- start 2023-11-09T03:56:09.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.243+0000 7f155504f640 1 -- start start 2023-11-09T03:56:09.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.243+0000 7f155504f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15500d3b60 0x7f155014fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:09.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.243+0000 7f155504f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f155015d100 0x7f15500cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:09.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.243+0000 7f155504f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f155015d4e0 0x7f15500cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:09.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.243+0000 7f155504f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155015eaa0 con 0x7f155015d100 2023-11-09T03:56:09.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.243+0000 7f155504f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155015de90 con 0x7f155015d4e0 2023-11-09T03:56:09.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.243+0000 7f155504f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f155015e170 con 0x7f15500d3b60 2023-11-09T03:56:09.253 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.243+0000 7f154e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f155015d100 0x7f15500cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:09.253 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.243+0000 7f154e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f155015d100 0x7f15500cc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40364/0 (socket says 172.21.15.76:40364) 2023-11-09T03:56:09.253 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.243+0000 7f154e59c640 1 -- 172.21.15.76:0/3633129006 learned_addr learned my addr 172.21.15.76:0/3633129006 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:56:09.253 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.244+0000 7f154f59e640 1 --2- 172.21.15.76:0/3633129006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f155015d4e0 0x7f15500cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:09.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.244+0000 7f154ed9d640 1 --2- 172.21.15.76:0/3633129006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15500d3b60 0x7f155014fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:09.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.244+0000 7f154e59c640 1 -- 172.21.15.76:0/3633129006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15500d3b60 msgr2=0x7f155014fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:09.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.244+0000 7f154e59c640 1 --2- 172.21.15.76:0/3633129006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15500d3b60 0x7f155014fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.254 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.244+0000 7f154e59c640 1 -- 172.21.15.76:0/3633129006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f155015d4e0 msgr2=0x7f15500cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:09.255 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.244+0000 7f154e59c640 1 --2- 172.21.15.76:0/3633129006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f155015d4e0 0x7f15500cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.255 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.244+0000 7f154e59c640 1 -- 172.21.15.76:0/3633129006 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15440094d0 con 0x7f155015d100 2023-11-09T03:56:09.255 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.244+0000 7f154e59c640 1 --2- 172.21.15.76:0/3633129006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f155015d100 0x7f15500cc3d0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f154402b9c0 tx=0x7f15440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:09.255 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.244+0000 7f15377fe640 1 -- 172.21.15.76:0/3633129006 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154403e070 con 0x7f155015d100 2023-11-09T03:56:09.256 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.244+0000 7f15377fe640 1 -- 172.21.15.76:0/3633129006 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1544039a10 con 0x7f155015d100 2023-11-09T03:56:09.256 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.244+0000 7f155504f640 1 -- 172.21.15.76:0/3633129006 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f155015e450 con 0x7f155015d100 2023-11-09T03:56:09.256 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.244+0000 7f15377fe640 1 -- 172.21.15.76:0/3633129006 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1544039de0 con 0x7f155015d100 2023-11-09T03:56:09.256 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.244+0000 7f155504f640 1 -- 172.21.15.76:0/3633129006 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1550150010 con 0x7f155015d100 2023-11-09T03:56:09.256 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.245+0000 7f155504f640 1 -- 172.21.15.76:0/3633129006 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15500684b0 con 0x7f155015d100 2023-11-09T03:56:09.257 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.246+0000 7f15377fe640 1 -- 172.21.15.76:0/3633129006 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f154404b400 con 0x7f155015d100 2023-11-09T03:56:09.257 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.246+0000 7f15377fe640 1 --2- 172.21.15.76:0/3633129006 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f153c06ae70 0x7f153c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:09.257 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.246+0000 7f15377fe640 1 -- 172.21.15.76:0/3633129006 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8582+0+0 (secure 0 0 0) 0x7f15440ac5a0 con 0x7f155015d100 2023-11-09T03:56:09.257 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.246+0000 7f154ed9d640 1 --2- 172.21.15.76:0/3633129006 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f153c06ae70 0x7f153c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:09.258 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.247+0000 7f154ed9d640 1 --2- 172.21.15.76:0/3633129006 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f153c06ae70 0x7f153c06d330 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f1540001330 tx=0x7f1540009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:09.258 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.250+0000 7f15377fe640 1 -- 172.21.15.76:0/3633129006 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1544076b70 con 0x7f155015d100 2023-11-09T03:56:09.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.416+0000 7f155504f640 1 -- 172.21.15.76:0/3633129006 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f15500cc1b0 con 0x7f155015d100 2023-11-09T03:56:09.416 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.416+0000 7f15377fe640 1 -- 172.21.15.76:0/3633129006 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1544034020 con 0x7f155015d100 2023-11-09T03:56:09.417 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:56:09.418 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.418+0000 7f155504f640 1 -- 172.21.15.76:0/3633129006 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f153c06ae70 msgr2=0x7f153c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:09.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.418+0000 7f155504f640 1 --2- 172.21.15.76:0/3633129006 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f153c06ae70 0x7f153c06d330 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f1540001330 tx=0x7f1540009040 comp rx=0 tx=0).stop 2023-11-09T03:56:09.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.418+0000 7f155504f640 1 -- 172.21.15.76:0/3633129006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f155015d100 msgr2=0x7f15500cc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:09.419 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.418+0000 7f155504f640 1 --2- 172.21.15.76:0/3633129006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f155015d100 0x7f15500cc3d0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f154402b9c0 tx=0x7f15440046c0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.418+0000 7f155504f640 1 -- 172.21.15.76:0/3633129006 shutdown_connections 2023-11-09T03:56:09.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.418+0000 7f155504f640 1 --2- 172.21.15.76:0/3633129006 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f153c06ae70 0x7f153c06d330 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.418+0000 7f155504f640 1 --2- 172.21.15.76:0/3633129006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f155015d4e0 0x7f15500cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.420 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.418+0000 7f155504f640 1 --2- 172.21.15.76:0/3633129006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f155015d100 0x7f15500cc3d0 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.418+0000 7f155504f640 1 --2- 172.21.15.76:0/3633129006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15500d3b60 0x7f155014fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.418+0000 7f155504f640 1 -- 172.21.15.76:0/3633129006 >> 172.21.15.76:0/3633129006 conn(0x7f15500d2ef0 msgr2=0x7f15500cd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:09.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.418+0000 7f155504f640 1 -- 172.21.15.76:0/3633129006 shutdown_connections 2023-11-09T03:56:09.421 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.418+0000 7f155504f640 1 -- 172.21.15.76:0/3633129006 wait complete. 2023-11-09T03:56:09.429 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-09T03:56:09.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.534+0000 7f2bb8b8e640 1 Processor -- start 2023-11-09T03:56:09.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.534+0000 7f2bb8b8e640 1 -- start start 2023-11-09T03:56:09.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.535+0000 7f2bb8b8e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bb40ade00 0x7f2bb40b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:09.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.535+0000 7f2bb8b8e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2bb406d9c0 0x7f2bb40710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:09.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.535+0000 7f2bb8b8e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb406cc90 con 0x7f2bb406c5f0 2023-11-09T03:56:09.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.535+0000 7f2bb8b8e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bb40b0e60 con 0x7f2bb406d9c0 2023-11-09T03:56:09.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.535+0000 7f2bb8b8e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb406cee0 con 0x7f2bb40ade00 2023-11-09T03:56:09.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.535+0000 7f2bb2d9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2bb406c5f0 0x7f2bb415e9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47822/0 (socket says 172.21.15.76:47822) 2023-11-09T03:56:09.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.535+0000 7f2bb2d9d640 1 -- 172.21.15.76:0/947140756 learned_addr learned my addr 172.21.15.76:0/947140756 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:56:09.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.535+0000 7f2bb1d9b640 1 --2- 172.21.15.76:0/947140756 >> v2:172.21.15.113:3300/0 conn(0x7f2bb406d9c0 0x7f2bb40710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:09.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.535+0000 7f2bb259c640 1 --2- 172.21.15.76:0/947140756 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bb40ade00 0x7f2bb40b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:09.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.536+0000 7f2bb1d9b640 1 -- 172.21.15.76:0/947140756 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bb40ade00 msgr2=0x7f2bb40b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:09.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.536+0000 7f2bb1d9b640 1 --2- 172.21.15.76:0/947140756 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bb40ade00 0x7f2bb40b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.536+0000 7f2bb1d9b640 1 -- 172.21.15.76:0/947140756 >> v1:172.21.15.76:6789/0 conn(0x7f2bb406c5f0 legacy=0x7f2bb415e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:09.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.536+0000 7f2bb1d9b640 1 -- 172.21.15.76:0/947140756 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb406d1c0 con 0x7f2bb406d9c0 2023-11-09T03:56:09.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.536+0000 7f2bb259c640 1 --2- 172.21.15.76:0/947140756 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bb40ade00 0x7f2bb40b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-09T03:56:09.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.536+0000 7f2bb1d9b640 1 --2- 172.21.15.76:0/947140756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb406d9c0 0x7f2bb40710d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2ba80097a0 tx=0x7f2ba802b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fe41902f0be336b server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:09.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.539+0000 7f2bb159a640 1 -- 172.21.15.76:0/947140756 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba802f050 con 0x7f2bb406d9c0 2023-11-09T03:56:09.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.539+0000 7f2bb159a640 1 -- 172.21.15.76:0/947140756 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba8031700 con 0x7f2bb406d9c0 2023-11-09T03:56:09.545 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.539+0000 7f2bb159a640 1 -- 172.21.15.76:0/947140756 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8031a90 con 0x7f2bb406d9c0 2023-11-09T03:56:09.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.540+0000 7f2bb8b8e640 1 -- 172.21.15.76:0/947140756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb406d9c0 msgr2=0x7f2bb40710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:09.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.540+0000 7f2bb8b8e640 1 --2- 172.21.15.76:0/947140756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb406d9c0 0x7f2bb40710d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2ba80097a0 tx=0x7f2ba802b7a0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.540+0000 7f2bb8b8e640 1 -- 172.21.15.76:0/947140756 shutdown_connections 2023-11-09T03:56:09.546 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.540+0000 7f2bb8b8e640 1 --2- 172.21.15.76:0/947140756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb406d9c0 0x7f2bb40710d0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.540+0000 7f2bb8b8e640 1 --2- 172.21.15.76:0/947140756 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bb40ade00 0x7f2bb40b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.540+0000 7f2bb8b8e640 1 -- 172.21.15.76:0/947140756 >> 172.21.15.76:0/947140756 conn(0x7f2bb40cb530 msgr2=0x7f2bb40cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:09.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.540+0000 7f2bb8b8e640 1 -- 172.21.15.76:0/947140756 shutdown_connections 2023-11-09T03:56:09.547 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.540+0000 7f2bb8b8e640 1 -- 172.21.15.76:0/947140756 wait complete. 2023-11-09T03:56:09.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.541+0000 7f2bb8b8e640 1 Processor -- start 2023-11-09T03:56:09.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.541+0000 7f2bb8b8e640 1 -- start start 2023-11-09T03:56:09.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.542+0000 7f2bb8b8e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bb406c5f0 0x7f2bb41f0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:09.548 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.542+0000 7f2bb8b8e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bb406d9c0 0x7f2bb40b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:09.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.542+0000 7f2bb8b8e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb40ade00 0x7f2bb40af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:09.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.542+0000 7f2bb8b8e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb40b01e0 con 0x7f2bb406c5f0 2023-11-09T03:56:09.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.542+0000 7f2bb8b8e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb41e4060 con 0x7f2bb40ade00 2023-11-09T03:56:09.549 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.542+0000 7f2bb8b8e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb415d3d0 con 0x7f2bb406d9c0 2023-11-09T03:56:09.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.542+0000 7f2bb259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bb406c5f0 0x7f2bb41f0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:09.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.542+0000 7f2bb259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bb406c5f0 0x7f2bb41f0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40380/0 (socket says 172.21.15.76:40380) 2023-11-09T03:56:09.550 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.542+0000 7f2bb259c640 1 -- 172.21.15.76:0/694497801 learned_addr learned my addr 172.21.15.76:0/694497801 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:56:09.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.542+0000 7f2bb2d9d640 1 --2- 172.21.15.76:0/694497801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb40ade00 0x7f2bb40af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:09.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.542+0000 7f2bb1d9b640 1 --2- 172.21.15.76:0/694497801 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bb406d9c0 0x7f2bb40b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:09.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.542+0000 7f2bb259c640 1 -- 172.21.15.76:0/694497801 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bb406d9c0 msgr2=0x7f2bb40b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:09.551 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.542+0000 7f2bb259c640 1 --2- 172.21.15.76:0/694497801 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bb406d9c0 0x7f2bb40b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.542+0000 7f2bb259c640 1 -- 172.21.15.76:0/694497801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb40ade00 msgr2=0x7f2bb40af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:09.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.542+0000 7f2bb259c640 1 --2- 172.21.15.76:0/694497801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb40ade00 0x7f2bb40af690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.542+0000 7f2bb259c640 1 -- 172.21.15.76:0/694497801 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba4009a70 con 0x7f2bb406c5f0 2023-11-09T03:56:09.552 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.542+0000 7f2bb1d9b640 1 --2- 172.21.15.76:0/694497801 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bb406d9c0 0x7f2bb40b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:56:09.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.543+0000 7f2bb259c640 1 --2- 172.21.15.76:0/694497801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bb406c5f0 0x7f2bb41f0970 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009ec0 tx=0x7f2ba400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:09.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.543+0000 7f2ba2ffd640 1 -- 172.21.15.76:0/694497801 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4014680 con 0x7f2bb406c5f0 2023-11-09T03:56:09.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.543+0000 7f2ba2ffd640 1 -- 172.21.15.76:0/694497801 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba400f040 con 0x7f2bb406c5f0 2023-11-09T03:56:09.553 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.543+0000 7f2bb8b8e640 1 -- 172.21.15.76:0/694497801 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ba8009400 con 0x7f2bb406c5f0 2023-11-09T03:56:09.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.543+0000 7f2ba2ffd640 1 -- 172.21.15.76:0/694497801 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4014a00 con 0x7f2bb406c5f0 2023-11-09T03:56:09.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.543+0000 7f2bb8b8e640 1 -- 172.21.15.76:0/694497801 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb41e3a10 con 0x7f2bb406c5f0 2023-11-09T03:56:09.554 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.544+0000 7f2bb8b8e640 1 -- 172.21.15.76:0/694497801 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb40684b0 con 0x7f2bb406c5f0 2023-11-09T03:56:09.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.544+0000 7f2ba2ffd640 1 -- 172.21.15.76:0/694497801 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2ba4014b60 con 0x7f2bb406c5f0 2023-11-09T03:56:09.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.545+0000 7f2ba2ffd640 1 --2- 172.21.15.76:0/694497801 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2b9406ae70 0x7f2b9406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:09.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.545+0000 7f2ba2ffd640 1 -- 172.21.15.76:0/694497801 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8582+0+0 (secure 0 0 0) 0x7f2ba408b0d0 con 0x7f2bb406c5f0 2023-11-09T03:56:09.555 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.545+0000 7f2bb1d9b640 1 --2- 172.21.15.76:0/694497801 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2b9406ae70 0x7f2b9406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:09.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.545+0000 7f2bb1d9b640 1 --2- 172.21.15.76:0/694497801 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2b9406ae70 0x7f2b9406d330 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8035000 tx=0x7f2ba8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:09.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.549+0000 7f2ba2ffd640 1 -- 172.21.15.76:0/694497801 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2ba40555f0 con 0x7f2bb406c5f0 2023-11-09T03:56:09.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.714+0000 7f2bb8b8e640 1 -- 172.21.15.76:0/694497801 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f2bb406d470 con 0x7f2bb406c5f0 2023-11-09T03:56:09.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.714+0000 7f2ba2ffd640 1 -- 172.21.15.76:0/694497801 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v123) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2ba4058c10 con 0x7f2bb406c5f0 2023-11-09T03:56:09.714 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:56:09.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.716+0000 7f2bb8b8e640 1 -- 172.21.15.76:0/694497801 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2b9406ae70 msgr2=0x7f2b9406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:09.716 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.716+0000 7f2bb8b8e640 1 --2- 172.21.15.76:0/694497801 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2b9406ae70 0x7f2b9406d330 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8035000 tx=0x7f2ba8008030 comp rx=0 tx=0).stop 2023-11-09T03:56:09.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.716+0000 7f2bb8b8e640 1 -- 172.21.15.76:0/694497801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bb406c5f0 msgr2=0x7f2bb41f0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:09.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.716+0000 7f2bb8b8e640 1 --2- 172.21.15.76:0/694497801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bb406c5f0 0x7f2bb41f0970 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009ec0 tx=0x7f2ba400c490 comp rx=0 tx=0).stop 2023-11-09T03:56:09.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.716+0000 7f2bb8b8e640 1 -- 172.21.15.76:0/694497801 shutdown_connections 2023-11-09T03:56:09.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.716+0000 7f2bb8b8e640 1 --2- 172.21.15.76:0/694497801 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f2b9406ae70 0x7f2b9406d330 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.717 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.716+0000 7f2bb8b8e640 1 --2- 172.21.15.76:0/694497801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb40ade00 0x7f2bb40af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.716+0000 7f2bb8b8e640 1 --2- 172.21.15.76:0/694497801 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bb406d9c0 0x7f2bb40b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.716+0000 7f2bb8b8e640 1 --2- 172.21.15.76:0/694497801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bb406c5f0 0x7f2bb41f0970 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.716+0000 7f2bb8b8e640 1 -- 172.21.15.76:0/694497801 >> 172.21.15.76:0/694497801 conn(0x7f2bb40cb530 msgr2=0x7f2bb40c81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:09.718 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.716+0000 7f2bb8b8e640 1 -- 172.21.15.76:0/694497801 shutdown_connections 2023-11-09T03:56:09.719 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.716+0000 7f2bb8b8e640 1 -- 172.21.15.76:0/694497801 wait complete. 2023-11-09T03:56:09.727 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-09T03:56:09.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.830+0000 7efdd51a9640 1 Processor -- start 2023-11-09T03:56:09.829 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.830+0000 7efdd51a9640 1 -- start start 2023-11-09T03:56:09.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.830+0000 7efdd51a9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7efdd00b30d0 0x7efdd00b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:09.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.830+0000 7efdd51a9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efdd00b1270 0x7efdd006a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:09.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.830+0000 7efdd51a9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdd006c950 con 0x7efdd015d1b0 2023-11-09T03:56:09.831 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.830+0000 7efdd51a9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdd00b3560 con 0x7efdd00b30d0 2023-11-09T03:56:09.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.830+0000 7efdd51a9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdd006cbe0 con 0x7efdd00b1270 2023-11-09T03:56:09.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.831+0000 7efdce59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7efdd00b30d0 0x7efdd00b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:09.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.831+0000 7efdced9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7efdd015d1b0 0x7efdd006c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47828/0 (socket says 172.21.15.76:47828) 2023-11-09T03:56:09.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.831+0000 7efdced9d640 1 -- 172.21.15.76:0/617882294 learned_addr learned my addr 172.21.15.76:0/617882294 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:56:09.832 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.831+0000 7efdce59c640 1 --2- 172.21.15.76:0/617882294 >> v2:172.21.15.76:3300/0 conn(0x7efdd00b30d0 0x7efdd00b0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40388/0 (socket says 172.21.15.76:40388) 2023-11-09T03:56:09.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.831+0000 7efdce59c640 1 -- 172.21.15.76:0/617882294 >> v2:172.21.15.113:3300/0 conn(0x7efdd00b1270 msgr2=0x7efdd006a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:09.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.831+0000 7efdce59c640 1 --2- 172.21.15.76:0/617882294 >> v2:172.21.15.113:3300/0 conn(0x7efdd00b1270 0x7efdd006a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.831+0000 7efdce59c640 1 -- 172.21.15.76:0/617882294 >> v1:172.21.15.76:6789/0 conn(0x7efdd015d1b0 legacy=0x7efdd006c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:09.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.831+0000 7efdce59c640 1 -- 172.21.15.76:0/617882294 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd00b2310 con 0x7efdd00b30d0 2023-11-09T03:56:09.834 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.831+0000 7efdcdd9b640 1 -- 172.21.15.76:0/617882294 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2116161905 0 0) 0x7efdd006c950 con 0x7efdd015d1b0 2023-11-09T03:56:09.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.832+0000 7efdce59c640 1 --2- 172.21.15.76:0/617882294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdd00b30d0 0x7efdd00b0d30 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7efdd00c2d90 tx=0x7efdc402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a27bd0a8562fc8 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:09.837 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.832+0000 7efdcdd9b640 1 -- 172.21.15.76:0/617882294 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc4030610 con 0x7efdd00b30d0 2023-11-09T03:56:09.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.832+0000 7efdcdd9b640 1 -- 172.21.15.76:0/617882294 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc4030770 con 0x7efdd00b30d0 2023-11-09T03:56:09.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.832+0000 7efdcdd9b640 1 -- 172.21.15.76:0/617882294 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc4030b00 con 0x7efdd00b30d0 2023-11-09T03:56:09.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.832+0000 7efdd51a9640 1 -- 172.21.15.76:0/617882294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdd00b30d0 msgr2=0x7efdd00b0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:09.838 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.832+0000 7efdd51a9640 1 --2- 172.21.15.76:0/617882294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdd00b30d0 0x7efdd00b0d30 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7efdd00c2d90 tx=0x7efdc402b960 comp rx=0 tx=0).stop 2023-11-09T03:56:09.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.832+0000 7efdd51a9640 1 -- 172.21.15.76:0/617882294 shutdown_connections 2023-11-09T03:56:09.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.832+0000 7efdd51a9640 1 --2- 172.21.15.76:0/617882294 >> v2:172.21.15.113:3300/0 conn(0x7efdd00b1270 0x7efdd006a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.832+0000 7efdd51a9640 1 --2- 172.21.15.76:0/617882294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdd00b30d0 0x7efdd00b0d30 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.832+0000 7efdd51a9640 1 -- 172.21.15.76:0/617882294 >> 172.21.15.76:0/617882294 conn(0x7efdd00cbd30 msgr2=0x7efdd00cc140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:09.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.832+0000 7efdd51a9640 1 -- 172.21.15.76:0/617882294 shutdown_connections 2023-11-09T03:56:09.839 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.832+0000 7efdd51a9640 1 -- 172.21.15.76:0/617882294 wait complete. 2023-11-09T03:56:09.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.834+0000 7efdd51a9640 1 Processor -- start 2023-11-09T03:56:09.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.834+0000 7efdd51a9640 1 -- start start 2023-11-09T03:56:09.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.834+0000 7efdd51a9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdd00b1270 0x7efdd00c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:09.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.834+0000 7efdd51a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdd00b30d0 0x7efdd00ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:09.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.834+0000 7efdd51a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdd015d1b0 0x7efdd00cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:09.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.834+0000 7efdd51a9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd009b3f0 con 0x7efdd00b1270 2023-11-09T03:56:09.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.834+0000 7efdd51a9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd00c7840 con 0x7efdd015d1b0 2023-11-09T03:56:09.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.834+0000 7efdd51a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd00c8b50 con 0x7efdd00b30d0 2023-11-09T03:56:09.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.835+0000 7efdced9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdd00b1270 0x7efdd00c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:09.841 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.835+0000 7efdced9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdd00b1270 0x7efdd00c9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40400/0 (socket says 172.21.15.76:40400) 2023-11-09T03:56:09.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.835+0000 7efdced9d640 1 -- 172.21.15.76:0/2701538169 learned_addr learned my addr 172.21.15.76:0/2701538169 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:56:09.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.835+0000 7efdced9d640 1 -- 172.21.15.76:0/2701538169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdd00b30d0 msgr2=0x7efdd00ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:09.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.835+0000 7efdced9d640 1 --2- 172.21.15.76:0/2701538169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdd00b30d0 0x7efdd00ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.835+0000 7efdcf59e640 1 --2- 172.21.15.76:0/2701538169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdd015d1b0 0x7efdd00cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:09.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.835+0000 7efdced9d640 1 -- 172.21.15.76:0/2701538169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdd015d1b0 msgr2=0x7efdd00cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:09.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.835+0000 7efdced9d640 1 --2- 172.21.15.76:0/2701538169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdd015d1b0 0x7efdd00cdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:09.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.835+0000 7efdced9d640 1 -- 172.21.15.76:0/2701538169 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc4009400 con 0x7efdd00b1270 2023-11-09T03:56:09.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.835+0000 7efdced9d640 1 --2- 172.21.15.76:0/2701538169 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdd00b1270 0x7efdd00c9ae0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7efdc000c410 tx=0x7efdc000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:09.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.835+0000 7efdb77fe640 1 -- 172.21.15.76:0/2701538169 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc00074e0 con 0x7efdd00b1270 2023-11-09T03:56:09.843 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.835+0000 7efdb77fe640 1 -- 172.21.15.76:0/2701538169 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc00057a0 con 0x7efdd00b1270 2023-11-09T03:56:09.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.835+0000 7efdd51a9640 1 -- 172.21.15.76:0/2701538169 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdd00c8e30 con 0x7efdd00b1270 2023-11-09T03:56:09.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.835+0000 7efdb77fe640 1 -- 172.21.15.76:0/2701538169 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc0007660 con 0x7efdd00b1270 2023-11-09T03:56:09.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.835+0000 7efdd51a9640 1 -- 172.21.15.76:0/2701538169 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd00c2de0 con 0x7efdd00b1270 2023-11-09T03:56:09.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.836+0000 7efdd51a9640 1 -- 172.21.15.76:0/2701538169 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdd009b150 con 0x7efdd00b1270 2023-11-09T03:56:09.844 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.837+0000 7efdb77fe640 1 -- 172.21.15.76:0/2701538169 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efdc0005bd0 con 0x7efdd00b1270 2023-11-09T03:56:09.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.837+0000 7efdb77fe640 1 --2- 172.21.15.76:0/2701538169 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efdbc06ae70 0x7efdbc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:09.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.837+0000 7efdb77fe640 1 -- 172.21.15.76:0/2701538169 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8582+0+0 (secure 0 0 0) 0x7efdc0089760 con 0x7efdd00b1270 2023-11-09T03:56:09.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.837+0000 7efdce59c640 1 --2- 172.21.15.76:0/2701538169 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efdbc06ae70 0x7efdbc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:09.845 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.838+0000 7efdce59c640 1 --2- 172.21.15.76:0/2701538169 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efdbc06ae70 0x7efdbc06d330 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7efdc4036000 tx=0x7efdc403b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:09.846 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:09.841+0000 7efdb77fe640 1 -- 172.21.15.76:0/2701538169 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efdd009b150 con 0x7efdd00b1270 2023-11-09T03:56:10.005 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.006+0000 7efdd51a9640 1 -- 172.21.15.76:0/2701538169 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7efdd009b530 con 0x7efdd00b1270 2023-11-09T03:56:10.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.006+0000 7efdb77fe640 1 -- 172.21.15.76:0/2701538169 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v123) v1 ==== 103+0+11 (secure 0 0 0) 0x7efdd009b530 con 0x7efdd00b1270 2023-11-09T03:56:10.040 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:56:10.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.008+0000 7efdd51a9640 1 -- 172.21.15.76:0/2701538169 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efdbc06ae70 msgr2=0x7efdbc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:10.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.008+0000 7efdd51a9640 1 --2- 172.21.15.76:0/2701538169 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efdbc06ae70 0x7efdbc06d330 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7efdc4036000 tx=0x7efdc403b040 comp rx=0 tx=0).stop 2023-11-09T03:56:10.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.008+0000 7efdd51a9640 1 -- 172.21.15.76:0/2701538169 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdd00b1270 msgr2=0x7efdd00c9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:10.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.008+0000 7efdd51a9640 1 --2- 172.21.15.76:0/2701538169 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdd00b1270 0x7efdd00c9ae0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7efdc000c410 tx=0x7efdc000c8d0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.008+0000 7efdd51a9640 1 -- 172.21.15.76:0/2701538169 shutdown_connections 2023-11-09T03:56:10.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.008+0000 7efdd51a9640 1 --2- 172.21.15.76:0/2701538169 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7efdbc06ae70 0x7efdbc06d330 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.008+0000 7efdd51a9640 1 --2- 172.21.15.76:0/2701538169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdd015d1b0 0x7efdd00cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.008+0000 7efdd51a9640 1 --2- 172.21.15.76:0/2701538169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdd00b30d0 0x7efdd00ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.008+0000 7efdd51a9640 1 --2- 172.21.15.76:0/2701538169 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdd00b1270 0x7efdd00c9ae0 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.008+0000 7efdd51a9640 1 -- 172.21.15.76:0/2701538169 >> 172.21.15.76:0/2701538169 conn(0x7efdd00cbd30 msgr2=0x7efdd00704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:10.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.008+0000 7efdd51a9640 1 -- 172.21.15.76:0/2701538169 shutdown_connections 2023-11-09T03:56:10.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.008+0000 7efdd51a9640 1 -- 172.21.15.76:0/2701538169 wait complete. 2023-11-09T03:56:10.044 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-09T03:56:10.147 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.148+0000 7faeb72da640 1 Processor -- start 2023-11-09T03:56:10.148 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.148+0000 7faeb72da640 1 -- start start 2023-11-09T03:56:10.150 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.149+0000 7faeb72da640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faeb0159e40 0x7faeb00b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:10.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.149+0000 7faeb72da640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faeb006bb60 0x7faeb00b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:10.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.149+0000 7faeb72da640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeb0071500 con 0x7faeb006fff0 2023-11-09T03:56:10.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.149+0000 7faeb72da640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeb015dae0 con 0x7faeb006bb60 2023-11-09T03:56:10.151 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.149+0000 7faeb72da640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb0070c60 con 0x7faeb0159e40 2023-11-09T03:56:10.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.149+0000 7faeb4875640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faeb006bb60 0x7faeb00b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:10.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.149+0000 7faeb5877640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7faeb006fff0 0x7faeb0070480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47842/0 (socket says 172.21.15.76:47842) 2023-11-09T03:56:10.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.149+0000 7faeb4875640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faeb006bb60 0x7faeb00b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40414/0 (socket says 172.21.15.76:40414) 2023-11-09T03:56:10.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.149+0000 7faeb5877640 1 -- 172.21.15.76:0/1195714779 learned_addr learned my addr 172.21.15.76:0/1195714779 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:56:10.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.149+0000 7faeb5076640 1 --2- 172.21.15.76:0/1195714779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faeb0159e40 0x7faeb00b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:10.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.149+0000 7faeb4875640 1 -- 172.21.15.76:0/1195714779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faeb0159e40 msgr2=0x7faeb00b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:10.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.149+0000 7faeb4875640 1 --2- 172.21.15.76:0/1195714779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faeb0159e40 0x7faeb00b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.149+0000 7faeb4875640 1 -- 172.21.15.76:0/1195714779 >> v1:172.21.15.76:6789/0 conn(0x7faeb006fff0 legacy=0x7faeb0070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:10.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.149+0000 7faeb4875640 1 -- 172.21.15.76:0/1195714779 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb0070f40 con 0x7faeb006bb60 2023-11-09T03:56:10.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.150+0000 7faeb4875640 1 --2- 172.21.15.76:0/1195714779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faeb006bb60 0x7faeb00b0a70 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7faea000ba40 tx=0x7faea002dff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84601c59724c51b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:10.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.150+0000 7faea7fff640 1 -- 172.21.15.76:0/1195714779 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea002eda0 con 0x7faeb006bb60 2023-11-09T03:56:10.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.150+0000 7faea7fff640 1 -- 172.21.15.76:0/1195714779 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea0036440 con 0x7faeb006bb60 2023-11-09T03:56:10.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.150+0000 7faea7fff640 1 -- 172.21.15.76:0/1195714779 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea00367d0 con 0x7faeb006bb60 2023-11-09T03:56:10.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.151+0000 7faeb72da640 1 -- 172.21.15.76:0/1195714779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faeb006bb60 msgr2=0x7faeb00b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:10.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.151+0000 7faeb72da640 1 --2- 172.21.15.76:0/1195714779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faeb006bb60 0x7faeb00b0a70 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7faea000ba40 tx=0x7faea002dff0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.151+0000 7faeb72da640 1 -- 172.21.15.76:0/1195714779 shutdown_connections 2023-11-09T03:56:10.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.151+0000 7faeb72da640 1 --2- 172.21.15.76:0/1195714779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faeb006bb60 0x7faeb00b0a70 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.151+0000 7faeb72da640 1 --2- 172.21.15.76:0/1195714779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faeb0159e40 0x7faeb00b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.151+0000 7faeb72da640 1 -- 172.21.15.76:0/1195714779 >> 172.21.15.76:0/1195714779 conn(0x7faeb00cb530 msgr2=0x7faeb00cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:10.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.151+0000 7faeb72da640 1 -- 172.21.15.76:0/1195714779 shutdown_connections 2023-11-09T03:56:10.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.151+0000 7faeb72da640 1 -- 172.21.15.76:0/1195714779 wait complete. 2023-11-09T03:56:10.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.152+0000 7faeb72da640 1 Processor -- start 2023-11-09T03:56:10.159 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.152+0000 7faeb72da640 1 -- start start 2023-11-09T03:56:10.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.152+0000 7faeb72da640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faeb006bb60 0x7faeb015ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:10.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.152+0000 7faeb72da640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faeb006fff0 0x7faeb00c1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:10.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.152+0000 7faeb72da640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faeb0159e40 0x7faeb00c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:10.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.152+0000 7faeb72da640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb015d210 con 0x7faeb006bb60 2023-11-09T03:56:10.160 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.153+0000 7faeb72da640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb00c2250 con 0x7faeb0159e40 2023-11-09T03:56:10.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.153+0000 7faeb72da640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb015b5b0 con 0x7faeb006fff0 2023-11-09T03:56:10.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.153+0000 7faeb5076640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faeb006bb60 0x7faeb015ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:10.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.153+0000 7faeb5076640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faeb006bb60 0x7faeb015ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40416/0 (socket says 172.21.15.76:40416) 2023-11-09T03:56:10.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.153+0000 7faeb5076640 1 -- 172.21.15.76:0/2756520170 learned_addr learned my addr 172.21.15.76:0/2756520170 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:56:10.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.153+0000 7faeb4875640 1 --2- 172.21.15.76:0/2756520170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faeb006fff0 0x7faeb00c1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:10.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.153+0000 7faeb5877640 1 --2- 172.21.15.76:0/2756520170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faeb0159e40 0x7faeb00c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:10.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.153+0000 7faeb5076640 1 -- 172.21.15.76:0/2756520170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faeb006fff0 msgr2=0x7faeb00c1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:10.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.153+0000 7faeb5076640 1 --2- 172.21.15.76:0/2756520170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faeb006fff0 0x7faeb00c1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.153+0000 7faeb5076640 1 -- 172.21.15.76:0/2756520170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faeb0159e40 msgr2=0x7faeb00c5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:10.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.153+0000 7faeb5076640 1 --2- 172.21.15.76:0/2756520170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faeb0159e40 0x7faeb00c5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.153+0000 7faeb5076640 1 -- 172.21.15.76:0/2756520170 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea000b6a0 con 0x7faeb006bb60 2023-11-09T03:56:10.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.153+0000 7faeb4875640 1 --2- 172.21.15.76:0/2756520170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faeb006fff0 0x7faeb00c1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:56:10.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.153+0000 7faeb5076640 1 --2- 172.21.15.76:0/2756520170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faeb006bb60 0x7faeb015ccd0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7faeac00d5f0 tx=0x7faeac00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:10.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.153+0000 7faea5ffb640 1 -- 172.21.15.76:0/2756520170 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac0125f0 con 0x7faeb006bb60 2023-11-09T03:56:10.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.153+0000 7faea5ffb640 1 -- 172.21.15.76:0/2756520170 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeac012750 con 0x7faeb006bb60 2023-11-09T03:56:10.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.153+0000 7faeb72da640 1 -- 172.21.15.76:0/2756520170 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb014ed50 con 0x7faeb006bb60 2023-11-09T03:56:10.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.153+0000 7faea5ffb640 1 -- 172.21.15.76:0/2756520170 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac012b00 con 0x7faeb006bb60 2023-11-09T03:56:10.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.153+0000 7faeb72da640 1 -- 172.21.15.76:0/2756520170 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb014f310 con 0x7faeb006bb60 2023-11-09T03:56:10.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.154+0000 7faeb72da640 1 -- 172.21.15.76:0/2756520170 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeb014f4a0 con 0x7faeb006bb60 2023-11-09T03:56:10.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.155+0000 7faea5ffb640 1 -- 172.21.15.76:0/2756520170 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faeac012c60 con 0x7faeb006bb60 2023-11-09T03:56:10.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.155+0000 7faea5ffb640 1 --2- 172.21.15.76:0/2756520170 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae9406aea0 0x7fae9406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:10.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.155+0000 7faea5ffb640 1 -- 172.21.15.76:0/2756520170 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8582+0+0 (secure 0 0 0) 0x7faeac08a5b0 con 0x7faeb006bb60 2023-11-09T03:56:10.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.155+0000 7faeb4875640 1 --2- 172.21.15.76:0/2756520170 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae9406aea0 0x7fae9406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:10.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.156+0000 7faeb4875640 1 --2- 172.21.15.76:0/2756520170 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae9406aea0 0x7fae9406d360 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7faea0038000 tx=0x7faea0032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:10.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.159+0000 7faea5ffb640 1 -- 172.21.15.76:0/2756520170 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faeac054b60 con 0x7faeb006bb60 2023-11-09T03:56:10.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.323+0000 7faeb72da640 1 -- 172.21.15.76:0/2756520170 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7faeb00684b0 con 0x7faeb006bb60 2023-11-09T03:56:10.323 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.324+0000 7faea5ffb640 1 -- 172.21.15.76:0/2756520170 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v123) v1 ==== 107+0+11 (secure 0 0 0) 0x7faeac058180 con 0x7faeb006bb60 2023-11-09T03:56:10.324 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:56:10.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.326+0000 7faeb72da640 1 -- 172.21.15.76:0/2756520170 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae9406aea0 msgr2=0x7fae9406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:10.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.326+0000 7faeb72da640 1 --2- 172.21.15.76:0/2756520170 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae9406aea0 0x7fae9406d360 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7faea0038000 tx=0x7faea0032000 comp rx=0 tx=0).stop 2023-11-09T03:56:10.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.326+0000 7faeb72da640 1 -- 172.21.15.76:0/2756520170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faeb006bb60 msgr2=0x7faeb015ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:10.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.326+0000 7faeb72da640 1 --2- 172.21.15.76:0/2756520170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faeb006bb60 0x7faeb015ccd0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7faeac00d5f0 tx=0x7faeac00dab0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.326+0000 7faeb72da640 1 -- 172.21.15.76:0/2756520170 shutdown_connections 2023-11-09T03:56:10.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.326+0000 7faeb72da640 1 --2- 172.21.15.76:0/2756520170 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fae9406aea0 0x7fae9406d360 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.326+0000 7faeb72da640 1 --2- 172.21.15.76:0/2756520170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faeb0159e40 0x7faeb00c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.326+0000 7faeb72da640 1 --2- 172.21.15.76:0/2756520170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faeb006fff0 0x7faeb00c1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.326+0000 7faeb72da640 1 --2- 172.21.15.76:0/2756520170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faeb006bb60 0x7faeb015ccd0 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.326+0000 7faeb72da640 1 -- 172.21.15.76:0/2756520170 >> 172.21.15.76:0/2756520170 conn(0x7faeb00cb530 msgr2=0x7faeb0159080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:10.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.326+0000 7faeb72da640 1 -- 172.21.15.76:0/2756520170 shutdown_connections 2023-11-09T03:56:10.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.326+0000 7faeb72da640 1 -- 172.21.15.76:0/2756520170 wait complete. 2023-11-09T03:56:10.336 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:56:10.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.438+0000 7f138d426640 1 Processor -- start 2023-11-09T03:56:10.438 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.438+0000 7f138d426640 1 -- start start 2023-11-09T03:56:10.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.439+0000 7f138d426640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f138815d100 0x7f13880ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:10.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.439+0000 7f138d426640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13880d1be0 0x7f13880b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:10.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.439+0000 7f138d426640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13880b32b0 con 0x7f138815d4e0 2023-11-09T03:56:10.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.439+0000 7f138d426640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13880cee10 con 0x7f138815d100 2023-11-09T03:56:10.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.439+0000 7f138d426640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13880b2980 con 0x7f13880d1be0 2023-11-09T03:56:10.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.439+0000 7f1386ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f138815d100 0x7f13880ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:10.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.439+0000 7f1386ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f138815d100 0x7f13880ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40432/0 (socket says 172.21.15.76:40432) 2023-11-09T03:56:10.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.439+0000 7f1386ffd640 1 -- 172.21.15.76:0/4162966879 learned_addr learned my addr 172.21.15.76:0/4162966879 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:56:10.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.439+0000 7f13867fc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f138815d4e0 0x7f13880d14d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47854/0 (socket says 172.21.15.76:47854) 2023-11-09T03:56:10.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.440+0000 7f13877fe640 1 --2- 172.21.15.76:0/4162966879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13880d1be0 0x7f13880b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:10.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.440+0000 7f1386ffd640 1 -- 172.21.15.76:0/4162966879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13880d1be0 msgr2=0x7f13880b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:10.442 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.440+0000 7f1386ffd640 1 --2- 172.21.15.76:0/4162966879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13880d1be0 0x7f13880b3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.440+0000 7f1386ffd640 1 -- 172.21.15.76:0/4162966879 >> v1:172.21.15.76:6789/0 conn(0x7f138815d4e0 legacy=0x7f13880d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:10.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.440+0000 7f1385ffb640 1 -- 172.21.15.76:0/4162966879 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1540105483 0 0) 0x7f13880b32b0 con 0x7f138815d4e0 2023-11-09T03:56:10.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.440+0000 7f1386ffd640 1 -- 172.21.15.76:0/4162966879 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13880b2c60 con 0x7f138815d100 2023-11-09T03:56:10.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.440+0000 7f1386ffd640 1 --2- 172.21.15.76:0/4162966879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f138815d100 0x7f13880ce510 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f1378009ea0 tx=0x7f137802bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be45ba4a2e7ec966 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:10.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.440+0000 7f1385ffb640 1 -- 172.21.15.76:0/4162966879 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137802cb40 con 0x7f138815d100 2023-11-09T03:56:10.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.441+0000 7f1385ffb640 1 -- 172.21.15.76:0/4162966879 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137802cca0 con 0x7f138815d100 2023-11-09T03:56:10.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.441+0000 7f1385ffb640 1 -- 172.21.15.76:0/4162966879 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1378034610 con 0x7f138815d100 2023-11-09T03:56:10.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.441+0000 7f138d426640 1 -- 172.21.15.76:0/4162966879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f138815d100 msgr2=0x7f13880ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:10.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.441+0000 7f138d426640 1 --2- 172.21.15.76:0/4162966879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f138815d100 0x7f13880ce510 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f1378009ea0 tx=0x7f137802bfa0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.441+0000 7f138d426640 1 -- 172.21.15.76:0/4162966879 shutdown_connections 2023-11-09T03:56:10.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.441+0000 7f138d426640 1 --2- 172.21.15.76:0/4162966879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13880d1be0 0x7f13880b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.441+0000 7f138d426640 1 --2- 172.21.15.76:0/4162966879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f138815d100 0x7f13880ce510 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.441+0000 7f138d426640 1 -- 172.21.15.76:0/4162966879 >> 172.21.15.76:0/4162966879 conn(0x7f13880d2ef0 msgr2=0x7f13880d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:10.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.441+0000 7f138d426640 1 -- 172.21.15.76:0/4162966879 shutdown_connections 2023-11-09T03:56:10.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.441+0000 7f138d426640 1 -- 172.21.15.76:0/4162966879 wait complete. 2023-11-09T03:56:10.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.442+0000 7f138d426640 1 Processor -- start 2023-11-09T03:56:10.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.442+0000 7f138d426640 1 -- start start 2023-11-09T03:56:10.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.443+0000 7f138d426640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13880d1be0 0x7f13880b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:10.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.443+0000 7f138d426640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f138815d100 0x7f13880b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:10.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.443+0000 7f138d426640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f138815d4e0 0x7f13880b6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:10.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.443+0000 7f138d426640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138815f540 con 0x7f13880d1be0 2023-11-09T03:56:10.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.443+0000 7f138d426640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138815f820 con 0x7f138815d100 2023-11-09T03:56:10.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.443+0000 7f138d426640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f138815fb00 con 0x7f138815d4e0 2023-11-09T03:56:10.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.443+0000 7f1386ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13880d1be0 0x7f13880b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:10.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.443+0000 7f1386ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13880d1be0 0x7f13880b5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40448/0 (socket says 172.21.15.76:40448) 2023-11-09T03:56:10.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.443+0000 7f1386ffd640 1 -- 172.21.15.76:0/3922986705 learned_addr learned my addr 172.21.15.76:0/3922986705 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:56:10.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.443+0000 7f13867fc640 1 --2- 172.21.15.76:0/3922986705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f138815d100 0x7f13880b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:10.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.443+0000 7f13877fe640 1 --2- 172.21.15.76:0/3922986705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f138815d4e0 0x7f13880b6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:10.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.444+0000 7f1386ffd640 1 -- 172.21.15.76:0/3922986705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f138815d4e0 msgr2=0x7f13880b6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:10.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.444+0000 7f1386ffd640 1 --2- 172.21.15.76:0/3922986705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f138815d4e0 0x7f13880b6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.444+0000 7f1386ffd640 1 -- 172.21.15.76:0/3922986705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f138815d100 msgr2=0x7f13880b6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:10.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.444+0000 7f1386ffd640 1 --2- 172.21.15.76:0/3922986705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f138815d100 0x7f13880b6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.444+0000 7f1386ffd640 1 -- 172.21.15.76:0/3922986705 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1378009b30 con 0x7f13880d1be0 2023-11-09T03:56:10.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.444+0000 7f1386ffd640 1 --2- 172.21.15.76:0/3922986705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13880d1be0 0x7f13880b5e10 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f1378036040 tx=0x7f1378004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:10.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.444+0000 7f136f7fe640 1 -- 172.21.15.76:0/3922986705 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137803e450 con 0x7f13880d1be0 2023-11-09T03:56:10.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.444+0000 7f136f7fe640 1 -- 172.21.15.76:0/3922986705 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137803e5b0 con 0x7f13880d1be0 2023-11-09T03:56:10.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.444+0000 7f136f7fe640 1 -- 172.21.15.76:0/3922986705 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137803e940 con 0x7f13880d1be0 2023-11-09T03:56:10.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.444+0000 7f138d426640 1 -- 172.21.15.76:0/3922986705 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f138815fde0 con 0x7f13880d1be0 2023-11-09T03:56:10.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.444+0000 7f138d426640 1 -- 172.21.15.76:0/3922986705 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13880b89e0 con 0x7f13880d1be0 2023-11-09T03:56:10.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.445+0000 7f138d426640 1 -- 172.21.15.76:0/3922986705 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13880cd0e0 con 0x7f13880d1be0 2023-11-09T03:56:10.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.445+0000 7f136f7fe640 1 -- 172.21.15.76:0/3922986705 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f137803eaa0 con 0x7f13880d1be0 2023-11-09T03:56:10.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.446+0000 7f136f7fe640 1 --2- 172.21.15.76:0/3922986705 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f137406ae70 0x7f137406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:10.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.446+0000 7f136f7fe640 1 -- 172.21.15.76:0/3922986705 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8582+0+0 (secure 0 0 0) 0x7f13780ad200 con 0x7f13880d1be0 2023-11-09T03:56:10.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.446+0000 7f13867fc640 1 --2- 172.21.15.76:0/3922986705 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f137406ae70 0x7f137406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:10.454 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.446+0000 7f13867fc640 1 --2- 172.21.15.76:0/3922986705 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f137406ae70 0x7f137406d330 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f13880ce5c0 tx=0x7f137c007b70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:10.455 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.450+0000 7f136f7fe640 1 -- 172.21.15.76:0/3922986705 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1378077720 con 0x7f13880d1be0 2023-11-09T03:56:10.615 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.616+0000 7f138d426640 1 -- 172.21.15.76:0/3922986705 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f13880b8b70 con 0x7f13880d1be0 2023-11-09T03:56:10.616 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.616+0000 7f136f7fe640 1 -- 172.21.15.76:0/3922986705 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1378043020 con 0x7f13880d1be0 2023-11-09T03:56:10.616 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:56:10.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.618+0000 7f138d426640 1 -- 172.21.15.76:0/3922986705 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f137406ae70 msgr2=0x7f137406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:10.618 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.618+0000 7f138d426640 1 --2- 172.21.15.76:0/3922986705 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f137406ae70 0x7f137406d330 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f13880ce5c0 tx=0x7f137c007b70 comp rx=0 tx=0).stop 2023-11-09T03:56:10.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.618+0000 7f138d426640 1 -- 172.21.15.76:0/3922986705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13880d1be0 msgr2=0x7f13880b5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:10.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.618+0000 7f138d426640 1 --2- 172.21.15.76:0/3922986705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13880d1be0 0x7f13880b5e10 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f1378036040 tx=0x7f1378004a40 comp rx=0 tx=0).stop 2023-11-09T03:56:10.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.618+0000 7f138d426640 1 -- 172.21.15.76:0/3922986705 shutdown_connections 2023-11-09T03:56:10.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.618+0000 7f138d426640 1 --2- 172.21.15.76:0/3922986705 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f137406ae70 0x7f137406d330 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.619 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.618+0000 7f138d426640 1 --2- 172.21.15.76:0/3922986705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f138815d4e0 0x7f13880b6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.618+0000 7f138d426640 1 --2- 172.21.15.76:0/3922986705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f138815d100 0x7f13880b6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.618+0000 7f138d426640 1 --2- 172.21.15.76:0/3922986705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13880d1be0 0x7f13880b5e10 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.618+0000 7f138d426640 1 -- 172.21.15.76:0/3922986705 >> 172.21.15.76:0/3922986705 conn(0x7f13880d2ef0 msgr2=0x7f13880b3740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:10.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.618+0000 7f138d426640 1 -- 172.21.15.76:0/3922986705 shutdown_connections 2023-11-09T03:56:10.620 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.618+0000 7f138d426640 1 -- 172.21.15.76:0/3922986705 wait complete. 2023-11-09T03:56:10.628 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:56:10.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.731+0000 7fdcbddf5640 1 Processor -- start 2023-11-09T03:56:10.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.731+0000 7fdcbddf5640 1 -- start start 2023-11-09T03:56:10.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.732+0000 7fdcbddf5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdcb80addd0 0x7fdcb80cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:10.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.732+0000 7fdcbddf5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcb80cc680 0x7fdcb80ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:10.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.732+0000 7fdcbddf5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcb80cd720 con 0x7fdcb806bb10 2023-11-09T03:56:10.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.732+0000 7fdcbddf5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcb80ae1b0 con 0x7fdcb80addd0 2023-11-09T03:56:10.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.732+0000 7fdcbddf5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb80ccb10 con 0x7fdcb80cc680 2023-11-09T03:56:10.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.732+0000 7fdcb77fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdcb80addd0 0x7fdcb80cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:10.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.732+0000 7fdcb77fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdcb80addd0 0x7fdcb80cdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40464/0 (socket says 172.21.15.76:40464) 2023-11-09T03:56:10.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.732+0000 7fdcb77fe640 1 -- 172.21.15.76:0/1960328407 learned_addr learned my addr 172.21.15.76:0/1960328407 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:56:10.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.732+0000 7fdcb77fe640 1 -- 172.21.15.76:0/1960328407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcb80cc680 msgr2=0x7fdcb80ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:10.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.732+0000 7fdcb77fe640 1 --2- 172.21.15.76:0/1960328407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcb80cc680 0x7fdcb80ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.732+0000 7fdcb77fe640 1 -- 172.21.15.76:0/1960328407 >> v1:172.21.15.113:6789/0 conn(0x7fdcb806bb10 legacy=0x7fdcb806f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:10.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.732+0000 7fdcb77fe640 1 -- 172.21.15.76:0/1960328407 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb80ccdf0 con 0x7fdcb80addd0 2023-11-09T03:56:10.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.733+0000 7fdcb77fe640 1 --2- 172.21.15.76:0/1960328407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcb80addd0 0x7fdcb80cdbb0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fdca800bb00 tx=0x7fdca802dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5f4143a7ef0763 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:10.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.733+0000 7fdcb67fc640 1 -- 172.21.15.76:0/1960328407 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca802ec50 con 0x7fdcb80addd0 2023-11-09T03:56:10.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.733+0000 7fdcb67fc640 1 -- 172.21.15.76:0/1960328407 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca8032070 con 0x7fdcb80addd0 2023-11-09T03:56:10.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.733+0000 7fdcb67fc640 1 -- 172.21.15.76:0/1960328407 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca8038400 con 0x7fdcb80addd0 2023-11-09T03:56:10.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.734+0000 7fdcbddf5640 1 -- 172.21.15.76:0/1960328407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcb80addd0 msgr2=0x7fdcb80cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:10.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.734+0000 7fdcbddf5640 1 --2- 172.21.15.76:0/1960328407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcb80addd0 0x7fdcb80cdbb0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fdca800bb00 tx=0x7fdca802dea0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.734+0000 7fdcbddf5640 1 -- 172.21.15.76:0/1960328407 shutdown_connections 2023-11-09T03:56:10.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.734+0000 7fdcbddf5640 1 --2- 172.21.15.76:0/1960328407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcb80cc680 0x7fdcb80ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.734+0000 7fdcbddf5640 1 --2- 172.21.15.76:0/1960328407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcb80addd0 0x7fdcb80cdbb0 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.734+0000 7fdcbddf5640 1 -- 172.21.15.76:0/1960328407 >> 172.21.15.76:0/1960328407 conn(0x7fdcb80cb530 msgr2=0x7fdcb80cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:10.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.734+0000 7fdcbddf5640 1 -- 172.21.15.76:0/1960328407 shutdown_connections 2023-11-09T03:56:10.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.734+0000 7fdcbddf5640 1 -- 172.21.15.76:0/1960328407 wait complete. 2023-11-09T03:56:10.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.735+0000 7fdcbddf5640 1 Processor -- start 2023-11-09T03:56:10.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.735+0000 7fdcbddf5640 1 -- start start 2023-11-09T03:56:10.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.736+0000 7fdcbddf5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcb806bb10 0x7fdcb8157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:10.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.736+0000 7fdcbddf5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcb80addd0 0x7fdcb80b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:10.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.736+0000 7fdcbddf5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcb815be20 0x7fdcb80c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:10.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.736+0000 7fdcbddf5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb80c2660 con 0x7fdcb80addd0 2023-11-09T03:56:10.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.736+0000 7fdcbddf5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb80c2900 con 0x7fdcb806bb10 2023-11-09T03:56:10.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.736+0000 7fdcbddf5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb80c2be0 con 0x7fdcb815be20 2023-11-09T03:56:10.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.736+0000 7fdcb6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcb80addd0 0x7fdcb80b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:10.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.736+0000 7fdcb6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcb80addd0 0x7fdcb80b38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40472/0 (socket says 172.21.15.76:40472) 2023-11-09T03:56:10.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.736+0000 7fdcb6ffd640 1 -- 172.21.15.76:0/2674114065 learned_addr learned my addr 172.21.15.76:0/2674114065 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:56:10.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.736+0000 7fdcb77fe640 1 --2- 172.21.15.76:0/2674114065 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcb806bb10 0x7fdcb8157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:10.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.736+0000 7fdcb7fff640 1 --2- 172.21.15.76:0/2674114065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcb815be20 0x7fdcb80c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:10.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.736+0000 7fdcb6ffd640 1 -- 172.21.15.76:0/2674114065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcb815be20 msgr2=0x7fdcb80c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:10.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.736+0000 7fdcb6ffd640 1 --2- 172.21.15.76:0/2674114065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcb815be20 0x7fdcb80c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.736+0000 7fdcb6ffd640 1 -- 172.21.15.76:0/2674114065 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcb806bb10 msgr2=0x7fdcb8157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:10.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.736+0000 7fdcb6ffd640 1 --2- 172.21.15.76:0/2674114065 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcb806bb10 0x7fdcb8157b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.736+0000 7fdcb6ffd640 1 -- 172.21.15.76:0/2674114065 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca800b760 con 0x7fdcb80addd0 2023-11-09T03:56:10.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.736+0000 7fdcb7fff640 1 --2- 172.21.15.76:0/2674114065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcb815be20 0x7fdcb80c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:56:10.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.736+0000 7fdcb6ffd640 1 --2- 172.21.15.76:0/2674114065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcb80addd0 0x7fdcb80b38a0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fdcac00b410 tx=0x7fdcac00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:10.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.737+0000 7fdc9ffff640 1 -- 172.21.15.76:0/2674114065 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac011830 con 0x7fdcb80addd0 2023-11-09T03:56:10.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.737+0000 7fdc9ffff640 1 -- 172.21.15.76:0/2674114065 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcac0119b0 con 0x7fdcb80addd0 2023-11-09T03:56:10.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.737+0000 7fdcbddf5640 1 -- 172.21.15.76:0/2674114065 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb80c2ec0 con 0x7fdcb80addd0 2023-11-09T03:56:10.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.737+0000 7fdc9ffff640 1 -- 172.21.15.76:0/2674114065 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac011d60 con 0x7fdcb80addd0 2023-11-09T03:56:10.746 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.737+0000 7fdcbddf5640 1 -- 172.21.15.76:0/2674114065 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb81580d0 con 0x7fdcb80addd0 2023-11-09T03:56:10.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.738+0000 7fdcbddf5640 1 -- 172.21.15.76:0/2674114065 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcb80c33e0 con 0x7fdcb80addd0 2023-11-09T03:56:10.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.738+0000 7fdc9ffff640 1 -- 172.21.15.76:0/2674114065 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdcac01a400 con 0x7fdcb80addd0 2023-11-09T03:56:10.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.738+0000 7fdc9ffff640 1 --2- 172.21.15.76:0/2674114065 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdca406ae70 0x7fdca406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:10.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.738+0000 7fdc9ffff640 1 -- 172.21.15.76:0/2674114065 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8582+0+0 (secure 0 0 0) 0x7fdcac08a370 con 0x7fdcb80addd0 2023-11-09T03:56:10.747 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.739+0000 7fdcb77fe640 1 --2- 172.21.15.76:0/2674114065 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdca406ae70 0x7fdca406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:10.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.739+0000 7fdcb77fe640 1 --2- 172.21.15.76:0/2674114065 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdca406ae70 0x7fdca406d330 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fdca8039000 tx=0x7fdca8034000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:10.748 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.742+0000 7fdc9ffff640 1 -- 172.21.15.76:0/2674114065 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdcb80c33e0 con 0x7fdcb80addd0 2023-11-09T03:56:10.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.907+0000 7fdcbddf5640 1 -- 172.21.15.76:0/2674114065 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdcb80c33e0 con 0x7fdcb80addd0 2023-11-09T03:56:10.907 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.907+0000 7fdc9ffff640 1 -- 172.21.15.76:0/2674114065 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdcb80c33e0 con 0x7fdcb80addd0 2023-11-09T03:56:10.907 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:56:10.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.909+0000 7fdcbddf5640 1 -- 172.21.15.76:0/2674114065 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdca406ae70 msgr2=0x7fdca406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:10.909 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.909+0000 7fdcbddf5640 1 --2- 172.21.15.76:0/2674114065 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdca406ae70 0x7fdca406d330 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fdca8039000 tx=0x7fdca8034000 comp rx=0 tx=0).stop 2023-11-09T03:56:10.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.909+0000 7fdcbddf5640 1 -- 172.21.15.76:0/2674114065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcb80addd0 msgr2=0x7fdcb80b38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:10.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.909+0000 7fdcbddf5640 1 --2- 172.21.15.76:0/2674114065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcb80addd0 0x7fdcb80b38a0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fdcac00b410 tx=0x7fdcac00b8d0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.910 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.909+0000 7fdcbddf5640 1 -- 172.21.15.76:0/2674114065 shutdown_connections 2023-11-09T03:56:10.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.909+0000 7fdcbddf5640 1 --2- 172.21.15.76:0/2674114065 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fdca406ae70 0x7fdca406d330 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.909+0000 7fdcbddf5640 1 --2- 172.21.15.76:0/2674114065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcb815be20 0x7fdcb80c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.909+0000 7fdcbddf5640 1 --2- 172.21.15.76:0/2674114065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcb80addd0 0x7fdcb80b38a0 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.911 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.909+0000 7fdcbddf5640 1 --2- 172.21.15.76:0/2674114065 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcb806bb10 0x7fdcb8157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:10.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.909+0000 7fdcbddf5640 1 -- 172.21.15.76:0/2674114065 >> 172.21.15.76:0/2674114065 conn(0x7fdcb80cb530 msgr2=0x7fdcb806f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:10.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.909+0000 7fdcbddf5640 1 -- 172.21.15.76:0/2674114065 shutdown_connections 2023-11-09T03:56:10.912 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:10.909+0000 7fdcbddf5640 1 -- 172.21.15.76:0/2674114065 wait complete. 2023-11-09T03:56:10.920 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-09T03:56:11.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.021+0000 7f35d4dbb640 1 Processor -- start 2023-11-09T03:56:11.021 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.021+0000 7f35d4dbb640 1 -- start start 2023-11-09T03:56:11.025 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.022+0000 7f35d4dbb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35d00ceda0 0x7f35d015cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:11.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.022+0000 7f35d4dbb640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d00b0150 con 0x7f35d00cf890 2023-11-09T03:56:11.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.022+0000 7f35d4dbb640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d00b1170 con 0x7f35d00b2210 2023-11-09T03:56:11.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.022+0000 7f35d4dbb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d00b1df0 con 0x7f35d00ceda0 2023-11-09T03:56:11.026 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.022+0000 7f35ced9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f35d00cf890 0x7f35d006b7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47870/0 (socket says 172.21.15.76:47870) 2023-11-09T03:56:11.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.022+0000 7f35ced9d640 1 -- 172.21.15.76:0/2256504985 learned_addr learned my addr 172.21.15.76:0/2256504985 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:56:11.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.023+0000 7f35ce59c640 1 --2- 172.21.15.76:0/2256504985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35d00ceda0 0x7f35d015cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:11.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.023+0000 7f35cd59a640 1 -- 172.21.15.76:0/2256504985 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3475861510 0 0) 0x7f35d00b0150 con 0x7f35d00cf890 2023-11-09T03:56:11.027 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.023+0000 7f35cd59a640 1 -- 172.21.15.76:0/2256504985 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35b8003520 con 0x7f35d00cf890 2023-11-09T03:56:11.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.023+0000 7f35cd59a640 1 -- 172.21.15.76:0/2256504985 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1492198483 0 0) 0x7f35b8003520 con 0x7f35d00cf890 2023-11-09T03:56:11.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.023+0000 7f35cd59a640 1 -- 172.21.15.76:0/2256504985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35d00ceda0 msgr2=0x7f35d015cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:11.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.023+0000 7f35cd59a640 1 --2- 172.21.15.76:0/2256504985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35d00ceda0 0x7f35d015cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.023+0000 7f35cd59a640 1 -- 172.21.15.76:0/2256504985 >> v1:172.21.15.113:6789/0 conn(0x7f35d00b2210 legacy=0x7f35d00cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:11.028 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.023+0000 7f35cd59a640 1 -- 172.21.15.76:0/2256504985 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d00b1450 con 0x7f35d00cf890 2023-11-09T03:56:11.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.023+0000 7f35cd59a640 1 -- 172.21.15.76:0/2256504985 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f35c8001670 con 0x7f35d00cf890 2023-11-09T03:56:11.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.023+0000 7f35cd59a640 1 -- 172.21.15.76:0/2256504985 >> v1:172.21.15.76:6789/0 conn(0x7f35d00cf890 legacy=0x7f35d006b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:11.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.023+0000 7f35cd59a640 1 --2- 172.21.15.76:0/2256504985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35d00ceda0 0x7f35b80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:11.029 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.023+0000 7f35cd59a640 1 --2- 172.21.15.76:0/2256504985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35b8003ee0 0x7f35b80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:11.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.023+0000 7f35cd59a640 1 --2- 172.21.15.76:0/2256504985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35b8004d00 0x7f35b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:11.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.023+0000 7f35cd59a640 1 -- 172.21.15.76:0/2256504985 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d00b1df0 con 0x7f35b8004d00 2023-11-09T03:56:11.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.023+0000 7f35cd59a640 1 -- 172.21.15.76:0/2256504985 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b800ba90 con 0x7f35b8003ee0 2023-11-09T03:56:11.030 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.023+0000 7f35cd59a640 1 -- 172.21.15.76:0/2256504985 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b800bc50 con 0x7f35d00ceda0 2023-11-09T03:56:11.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.023+0000 7f35ced9d640 1 --2- 172.21.15.76:0/2256504985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35b8004d00 0x7f35b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:11.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.023+0000 7f35ce59c640 1 --2- 172.21.15.76:0/2256504985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35d00ceda0 0x7f35b80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:11.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.023+0000 7f35cdd9b640 1 --2- 172.21.15.76:0/2256504985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35b8003ee0 0x7f35b80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:11.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.023+0000 7f35ced9d640 1 -- 172.21.15.76:0/2256504985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35d00ceda0 msgr2=0x7f35b80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:11.031 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.023+0000 7f35ced9d640 1 --2- 172.21.15.76:0/2256504985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35d00ceda0 0x7f35b80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.023+0000 7f35ced9d640 1 -- 172.21.15.76:0/2256504985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35b8003ee0 msgr2=0x7f35b80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:11.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.023+0000 7f35ced9d640 1 --2- 172.21.15.76:0/2256504985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35b8003ee0 0x7f35b80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.023+0000 7f35ced9d640 1 -- 172.21.15.76:0/2256504985 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b8003520 con 0x7f35b8004d00 2023-11-09T03:56:11.032 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.024+0000 7f35ced9d640 1 --2- 172.21.15.76:0/2256504985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35b8004d00 0x7f35b8005180 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f35c800c190 tx=0x7f35b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2eaa30c497aa2c server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:11.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.024+0000 7f35cd59a640 1 -- 172.21.15.76:0/2256504985 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c8005ad0 con 0x7f35b8004d00 2023-11-09T03:56:11.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.024+0000 7f35cd59a640 1 -- 172.21.15.76:0/2256504985 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c8005c50 con 0x7f35b8004d00 2023-11-09T03:56:11.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.024+0000 7f35cd59a640 1 -- 172.21.15.76:0/2256504985 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c802c850 con 0x7f35b8004d00 2023-11-09T03:56:11.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.025+0000 7f35d4dbb640 1 -- 172.21.15.76:0/2256504985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35b8004d00 msgr2=0x7f35b8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:11.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.025+0000 7f35d4dbb640 1 --2- 172.21.15.76:0/2256504985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35b8004d00 0x7f35b8005180 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f35c800c190 tx=0x7f35b80057f0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.025+0000 7f35ced9d640 1 -- 172.21.15.76:0/2256504985 reap_dead start 2023-11-09T03:56:11.034 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.025+0000 7f35d4dbb640 1 -- 172.21.15.76:0/2256504985 shutdown_connections 2023-11-09T03:56:11.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.025+0000 7f35d4dbb640 1 -- 172.21.15.76:0/2256504985 >> 172.21.15.76:0/2256504985 conn(0x7f35d00d0c30 msgr2=0x7f35d00d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:11.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.025+0000 7f35d4dbb640 1 -- 172.21.15.76:0/2256504985 shutdown_connections 2023-11-09T03:56:11.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.025+0000 7f35d4dbb640 1 -- 172.21.15.76:0/2256504985 wait complete. 2023-11-09T03:56:11.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.026+0000 7f35d4dbb640 1 Processor -- start 2023-11-09T03:56:11.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.026+0000 7f35d4dbb640 1 -- start start 2023-11-09T03:56:11.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.027+0000 7f35d4dbb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35b8004d00 0x7f35d0155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:11.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.027+0000 7f35d4dbb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d00cd740 0x7f35d00cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:11.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.027+0000 7f35d4dbb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35d00cb260 0x7f35d00cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:11.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.027+0000 7f35d4dbb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d00cfd20 con 0x7f35d00cd740 2023-11-09T03:56:11.039 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.027+0000 7f35d4dbb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d01556a0 con 0x7f35b8004d00 2023-11-09T03:56:11.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.027+0000 7f35d4dbb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d00ccba0 con 0x7f35d00cb260 2023-11-09T03:56:11.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.027+0000 7f35cdd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d00cd740 0x7f35d00cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:11.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.027+0000 7f35cdd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d00cd740 0x7f35d00cdbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40486/0 (socket says 172.21.15.76:40486) 2023-11-09T03:56:11.040 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.027+0000 7f35cdd9b640 1 -- 172.21.15.76:0/1962725063 learned_addr learned my addr 172.21.15.76:0/1962725063 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:56:11.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.027+0000 7f35ced9d640 1 --2- 172.21.15.76:0/1962725063 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35d00cb260 0x7f35d00cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:11.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.027+0000 7f35cdd9b640 1 -- 172.21.15.76:0/1962725063 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35d00cb260 msgr2=0x7f35d00cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:11.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.027+0000 7f35cdd9b640 1 --2- 172.21.15.76:0/1962725063 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35d00cb260 0x7f35d00cb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.041 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.027+0000 7f35ce59c640 1 --2- 172.21.15.76:0/1962725063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35b8004d00 0x7f35d0155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:11.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.027+0000 7f35cdd9b640 1 -- 172.21.15.76:0/1962725063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35b8004d00 msgr2=0x7f35d0155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:11.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.027+0000 7f35cdd9b640 1 --2- 172.21.15.76:0/1962725063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35b8004d00 0x7f35d0155160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.027+0000 7f35cdd9b640 1 -- 172.21.15.76:0/1962725063 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d00ce100 con 0x7f35d00cd740 2023-11-09T03:56:11.042 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.028+0000 7f35cdd9b640 1 --2- 172.21.15.76:0/1962725063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d00cd740 0x7f35d00cdbc0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f35c400e830 tx=0x7f35c400ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:11.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.028+0000 7f35beffd640 1 -- 172.21.15.76:0/1962725063 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c400eee0 con 0x7f35d00cd740 2023-11-09T03:56:11.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.028+0000 7f35beffd640 1 -- 172.21.15.76:0/1962725063 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c40052e0 con 0x7f35d00cd740 2023-11-09T03:56:11.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.028+0000 7f35beffd640 1 -- 172.21.15.76:0/1962725063 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c4007270 con 0x7f35d00cd740 2023-11-09T03:56:11.043 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.028+0000 7f35d4dbb640 1 -- 172.21.15.76:0/1962725063 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d00cbf90 con 0x7f35d00cd740 2023-11-09T03:56:11.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.028+0000 7f35d4dbb640 1 -- 172.21.15.76:0/1962725063 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d00cc550 con 0x7f35d00cd740 2023-11-09T03:56:11.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.029+0000 7f35d4dbb640 1 -- 172.21.15.76:0/1962725063 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d00ccce0 con 0x7f35d00cd740 2023-11-09T03:56:11.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.029+0000 7f35beffd640 1 -- 172.21.15.76:0/1962725063 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f35c40073d0 con 0x7f35d00cd740 2023-11-09T03:56:11.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.029+0000 7f35beffd640 1 --2- 172.21.15.76:0/1962725063 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f35b006ae70 0x7f35b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:11.044 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.029+0000 7f35beffd640 1 -- 172.21.15.76:0/1962725063 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8582+0+0 (secure 0 0 0) 0x7f35c4089bc0 con 0x7f35d00cd740 2023-11-09T03:56:11.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.030+0000 7f35ce59c640 1 --2- 172.21.15.76:0/1962725063 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f35b006ae70 0x7f35b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:11.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.030+0000 7f35ce59c640 1 --2- 172.21.15.76:0/1962725063 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f35b006ae70 0x7f35b006d330 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f35c0007440 tx=0x7f35c000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:11.045 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.034+0000 7f35beffd640 1 -- 172.21.15.76:0/1962725063 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f35d00ccce0 con 0x7f35d00cd740 2023-11-09T03:56:11.211 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.212+0000 7f35d4dbb640 1 -- 172.21.15.76:0/1962725063 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f35d00ccce0 con 0x7f35d00cd740 2023-11-09T03:56:11.212 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.212+0000 7f35beffd640 1 -- 172.21.15.76:0/1962725063 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 32 entries v123) v1 ==== 81+0+1923 (secure 0 0 0) 0x7f35d00ccce0 con 0x7f35d00cd740 2023-11-09T03:56:11.212 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/497547142 2023-11-09T04:55:40.451393+0000 2023-11-09T03:56:11.213 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/757529522 2023-11-10T03:55:31.678438+0000 2023-11-09T03:56:11.213 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6832/3697850101 2023-11-10T03:51:39.442929+0000 2023-11-09T03:56:11.213 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/3449910842 2023-11-10T03:50:59.611256+0000 2023-11-09T03:56:11.213 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6832/1536527256 2023-11-10T03:52:34.213804+0000 2023-11-09T03:56:11.214 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/3229239133 2023-11-10T03:51:32.624856+0000 2023-11-09T03:56:11.214 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/445306598 2023-11-10T03:52:27.406838+0000 2023-11-09T03:56:11.214 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/1663988060 2023-11-10T03:50:33.665092+0000 2023-11-09T03:56:11.214 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/1663988060 2023-11-10T03:50:33.665092+0000 2023-11-09T03:56:11.215 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/3280098776 2023-11-10T03:50:07.616780+0000 2023-11-09T03:56:11.215 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/2519773110 2023-11-09T04:54:14.823765+0000 2023-11-09T03:56:11.215 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/4011275120 2023-11-10T03:49:42.029057+0000 2023-11-09T03:56:11.215 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/757529522 2023-11-10T03:55:31.678438+0000 2023-11-09T03:56:11.215 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/4011275120 2023-11-10T03:49:42.029057+0000 2023-11-09T03:56:11.216 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/3462040782 2023-11-10T03:54:45.345437+0000 2023-11-09T03:56:11.216 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/503424885 2023-11-09T04:51:47.727422+0000 2023-11-09T03:56:11.216 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/3229239133 2023-11-10T03:51:32.624856+0000 2023-11-09T03:56:11.216 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/445306598 2023-11-10T03:52:27.406838+0000 2023-11-09T03:56:11.217 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/3280098776 2023-11-10T03:50:07.616780+0000 2023-11-09T03:56:11.217 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/3087804343 2023-11-09T04:52:42.513556+0000 2023-11-09T03:56:11.217 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/546338722 2023-11-10T03:53:17.470106+0000 2023-11-09T03:56:11.217 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6833/1536527256 2023-11-10T03:52:34.213804+0000 2023-11-09T03:56:11.217 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/546338722 2023-11-10T03:53:17.470106+0000 2023-11-09T03:56:11.218 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/7360844 2023-11-10T03:53:24.718008+0000 2023-11-09T03:56:11.218 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/7360844 2023-11-10T03:53:24.718008+0000 2023-11-09T03:56:11.218 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/2075292158 2023-11-10T03:54:06.568191+0000 2023-11-09T03:56:11.218 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/334503200 2023-11-09T04:53:32.572840+0000 2023-11-09T03:56:11.219 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/2075292158 2023-11-10T03:54:06.568191+0000 2023-11-09T03:56:11.219 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6833/3697850101 2023-11-10T03:51:39.442929+0000 2023-11-09T03:56:11.219 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/3462040782 2023-11-10T03:54:45.345437+0000 2023-11-09T03:56:11.219 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/3449910842 2023-11-10T03:50:59.611256+0000 2023-11-09T03:56:11.220 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/3870807407 2023-11-09T04:54:56.671385+0000 2023-11-09T03:56:11.221 INFO:teuthology.orchestra.run.smithi076.stderr:listed 32 entries 2023-11-09T03:56:11.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.214+0000 7f35d4dbb640 1 -- 172.21.15.76:0/1962725063 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f35b006ae70 msgr2=0x7f35b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:11.221 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.214+0000 7f35d4dbb640 1 --2- 172.21.15.76:0/1962725063 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f35b006ae70 0x7f35b006d330 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f35c0007440 tx=0x7f35c000b040 comp rx=0 tx=0).stop 2023-11-09T03:56:11.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.214+0000 7f35d4dbb640 1 -- 172.21.15.76:0/1962725063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d00cd740 msgr2=0x7f35d00cdbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:11.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.214+0000 7f35d4dbb640 1 --2- 172.21.15.76:0/1962725063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d00cd740 0x7f35d00cdbc0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f35c400e830 tx=0x7f35c400ecf0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.214+0000 7f35d4dbb640 1 -- 172.21.15.76:0/1962725063 shutdown_connections 2023-11-09T03:56:11.222 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.214+0000 7f35d4dbb640 1 --2- 172.21.15.76:0/1962725063 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f35b006ae70 0x7f35b006d330 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.214+0000 7f35d4dbb640 1 --2- 172.21.15.76:0/1962725063 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35d00cb260 0x7f35d00cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.214+0000 7f35d4dbb640 1 --2- 172.21.15.76:0/1962725063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d00cd740 0x7f35d00cdbc0 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.214+0000 7f35d4dbb640 1 --2- 172.21.15.76:0/1962725063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35b8004d00 0x7f35d0155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.214+0000 7f35d4dbb640 1 -- 172.21.15.76:0/1962725063 >> 172.21.15.76:0/1962725063 conn(0x7f35d00d0c30 msgr2=0x7f35d00b26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:11.223 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.214+0000 7f35d4dbb640 1 -- 172.21.15.76:0/1962725063 shutdown_connections 2023-11-09T03:56:11.224 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.214+0000 7f35d4dbb640 1 -- 172.21.15.76:0/1962725063 wait complete. 2023-11-09T03:56:11.225 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T03:56:11.225 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T03:56:11.241 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-09T03:56:11.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.380+0000 7ff4d2ae0640 1 Processor -- start 2023-11-09T03:56:11.380 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.380+0000 7ff4d2ae0640 1 -- start start 2023-11-09T03:56:11.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.381+0000 7ff4d2ae0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff4cc070700 0x7ff4cc164b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:11.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.381+0000 7ff4d2ae0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4cc071500 con 0x7ff4cc159e40 2023-11-09T03:56:11.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.381+0000 7ff4d2ae0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4cc15dae0 con 0x7ff4cc0b0c40 2023-11-09T03:56:11.382 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.381+0000 7ff4d2ae0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4cc070c60 con 0x7ff4cc070700 2023-11-09T03:56:11.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.381+0000 7ff4d107d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff4cc070700 0x7ff4cc164b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:11.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.381+0000 7ff4d087c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff4cc159e40 0x7ff4cc0b0530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47872/0 (socket says 172.21.15.76:47872) 2023-11-09T03:56:11.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.381+0000 7ff4d087c640 1 -- 172.21.15.76:0/2706255341 learned_addr learned my addr 172.21.15.76:0/2706255341 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:56:11.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.381+0000 7ff4d107d640 1 -- 172.21.15.76:0/2706255341 >> v1:172.21.15.113:6789/0 conn(0x7ff4cc0b0c40 legacy=0x7ff4cc06fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:11.383 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.381+0000 7ff4d107d640 1 -- 172.21.15.76:0/2706255341 >> v1:172.21.15.76:6789/0 conn(0x7ff4cc159e40 legacy=0x7ff4cc0b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:11.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.381+0000 7ff4cb7fe640 1 -- 172.21.15.76:0/2706255341 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3141798477 0 0) 0x7ff4cc071500 con 0x7ff4cc159e40 2023-11-09T03:56:11.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.381+0000 7ff4d107d640 1 -- 172.21.15.76:0/2706255341 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4cc070f40 con 0x7ff4cc070700 2023-11-09T03:56:11.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.382+0000 7ff4d107d640 1 --2- 172.21.15.76:0/2706255341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4cc070700 0x7ff4cc164b60 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009d90 tx=0x7ff4c402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=338d7306c8c08d34 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:11.384 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.382+0000 7ff4cb7fe640 1 -- 172.21.15.76:0/2706255341 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c402cc60 con 0x7ff4cc070700 2023-11-09T03:56:11.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.382+0000 7ff4cb7fe640 1 -- 172.21.15.76:0/2706255341 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c402cdc0 con 0x7ff4cc070700 2023-11-09T03:56:11.385 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.382+0000 7ff4cb7fe640 1 -- 172.21.15.76:0/2706255341 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4034660 con 0x7ff4cc070700 2023-11-09T03:56:11.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.383+0000 7ff4d2ae0640 1 -- 172.21.15.76:0/2706255341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4cc070700 msgr2=0x7ff4cc164b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:11.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.383+0000 7ff4d2ae0640 1 --2- 172.21.15.76:0/2706255341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4cc070700 0x7ff4cc164b60 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009d90 tx=0x7ff4c402beb0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.388 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.383+0000 7ff4d2ae0640 1 -- 172.21.15.76:0/2706255341 shutdown_connections 2023-11-09T03:56:11.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.383+0000 7ff4d2ae0640 1 --2- 172.21.15.76:0/2706255341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4cc070700 0x7ff4cc164b60 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.383+0000 7ff4d2ae0640 1 -- 172.21.15.76:0/2706255341 >> 172.21.15.76:0/2706255341 conn(0x7ff4cc0cb530 msgr2=0x7ff4cc0cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:11.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.383+0000 7ff4d2ae0640 1 -- 172.21.15.76:0/2706255341 shutdown_connections 2023-11-09T03:56:11.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.383+0000 7ff4d2ae0640 1 -- 172.21.15.76:0/2706255341 wait complete. 2023-11-09T03:56:11.389 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.384+0000 7ff4d2ae0640 1 Processor -- start 2023-11-09T03:56:11.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.384+0000 7ff4d2ae0640 1 -- start start 2023-11-09T03:56:11.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.385+0000 7ff4d2ae0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4cc070700 0x7ff4cc146d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:11.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.385+0000 7ff4d2ae0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4cc0b0c40 0x7ff4cc0c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:11.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.385+0000 7ff4d2ae0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4cc159e40 0x7ff4cc0c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:11.390 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.385+0000 7ff4d2ae0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc0c7250 con 0x7ff4cc0b0c40 2023-11-09T03:56:11.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.385+0000 7ff4d2ae0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc0c6640 con 0x7ff4cc159e40 2023-11-09T03:56:11.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.385+0000 7ff4d2ae0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc0c6920 con 0x7ff4cc070700 2023-11-09T03:56:11.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.385+0000 7ff4cbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4cc0b0c40 0x7ff4cc0c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:11.391 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.385+0000 7ff4cbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4cc0b0c40 0x7ff4cc0c76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40508/0 (socket says 172.21.15.76:40508) 2023-11-09T03:56:11.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.385+0000 7ff4cbfff640 1 -- 172.21.15.76:0/2839577469 learned_addr learned my addr 172.21.15.76:0/2839577469 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:56:11.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.385+0000 7ff4d107d640 1 --2- 172.21.15.76:0/2839577469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4cc159e40 0x7ff4cc0c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:11.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.385+0000 7ff4d087c640 1 --2- 172.21.15.76:0/2839577469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4cc070700 0x7ff4cc146d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:11.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.385+0000 7ff4cbfff640 1 -- 172.21.15.76:0/2839577469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4cc070700 msgr2=0x7ff4cc146d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:11.392 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.385+0000 7ff4cbfff640 1 --2- 172.21.15.76:0/2839577469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4cc070700 0x7ff4cc146d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.385+0000 7ff4cbfff640 1 -- 172.21.15.76:0/2839577469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4cc159e40 msgr2=0x7ff4cc0c5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:11.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.385+0000 7ff4cbfff640 1 --2- 172.21.15.76:0/2839577469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4cc159e40 0x7ff4cc0c5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.385+0000 7ff4cbfff640 1 -- 172.21.15.76:0/2839577469 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c4009a70 con 0x7ff4cc0b0c40 2023-11-09T03:56:11.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.385+0000 7ff4d107d640 1 --2- 172.21.15.76:0/2839577469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4cc159e40 0x7ff4cc0c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:56:11.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.386+0000 7ff4cbfff640 1 --2- 172.21.15.76:0/2839577469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4cc0b0c40 0x7ff4cc0c76e0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000c450 tx=0x7ff4c000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:11.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.386+0000 7ff4c97fa640 1 -- 172.21.15.76:0/2839577469 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c00052d0 con 0x7ff4cc0b0c40 2023-11-09T03:56:11.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.386+0000 7ff4c97fa640 1 -- 172.21.15.76:0/2839577469 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c0005430 con 0x7ff4cc0b0c40 2023-11-09T03:56:11.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.386+0000 7ff4d2ae0640 1 -- 172.21.15.76:0/2839577469 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4cc0c6c00 con 0x7ff4cc0b0c40 2023-11-09T03:56:11.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.386+0000 7ff4c97fa640 1 -- 172.21.15.76:0/2839577469 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0012870 con 0x7ff4cc0b0c40 2023-11-09T03:56:11.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.386+0000 7ff4d2ae0640 1 -- 172.21.15.76:0/2839577469 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4cc147250 con 0x7ff4cc0b0c40 2023-11-09T03:56:11.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.387+0000 7ff4d2ae0640 1 -- 172.21.15.76:0/2839577469 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4cc0c7400 con 0x7ff4cc0b0c40 2023-11-09T03:56:11.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.387+0000 7ff4c97fa640 1 -- 172.21.15.76:0/2839577469 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff4c00129d0 con 0x7ff4cc0b0c40 2023-11-09T03:56:11.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.387+0000 7ff4c97fa640 1 --2- 172.21.15.76:0/2839577469 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff4b806ae70 0x7ff4b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:11.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.387+0000 7ff4c97fa640 1 -- 172.21.15.76:0/2839577469 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8582+0+0 (secure 0 0 0) 0x7ff4c0089cd0 con 0x7ff4cc0b0c40 2023-11-09T03:56:11.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.388+0000 7ff4d087c640 1 --2- 172.21.15.76:0/2839577469 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff4b806ae70 0x7ff4b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:11.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.388+0000 7ff4d087c640 1 --2- 172.21.15.76:0/2839577469 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff4b806ae70 0x7ff4b806d330 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc000f60 tx=0x7ff4bc007d00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:11.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.392+0000 7ff4c97fa640 1 -- 172.21.15.76:0/2839577469 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff4c00541f0 con 0x7ff4cc0b0c40 2023-11-09T03:56:11.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.562+0000 7ff4d2ae0640 1 -- 172.21.15.76:0/2839577469 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4cc0684b0 con 0x7ff4cc0b0c40 2023-11-09T03:56:11.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.564+0000 7ff4c97fa640 1 -- 172.21.15.76:0/2839577469 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+24268 (secure 0 0 0) 0x7ff4c0057810 con 0x7ff4cc0b0c40 2023-11-09T03:56:11.564 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-09T03:56:11.564 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":123,"fsid":"c213b26f-d0da-45d5-8e34-bb75c096d4f2","created":"2023-11-09T03:45:06.616228+0000","modified":"2023-11-09T03:55:56.504148+0000","last_up_change":"2023-11-09T03:45:17.259656+0000","last_in_change":"2023-11-09T03:45:08.827625+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-09T03:45:18.792162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-09T03:47:00.036540+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-09T03:47:59.280601+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":70,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-09T03:47:59.742039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-09T03:48:55.490832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-09T03:48:56.352624+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":122,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9cd7b709-ad8a-46c9-a589-494542849bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6803","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6806","nonce":1864895740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6809","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6811","nonce":1864895740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6815","nonce":1864895740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1864895740},{"type":"v1","addr":"172.21.15.76:6813","nonce":1864895740}]},"public_addr":"172.21.15.76:6806/1864895740","cluster_addr":"172.21.15.76:6811/1864895740","heartbeat_back_addr":"172.21.15.76:6815/1864895740","heartbeat_front_addr":"172.21.15.76:6813/1864895740","state":["exists","up"]},{"osd":1,"uuid":"b4a945ee-af9f-4132-a93b-e81753a397f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6817","nonce":1166289696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6819","nonce":1166289696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6823","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6825","nonce":1166289696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1166289696},{"type":"v1","addr":"172.21.15.76:6821","nonce":1166289696}]},"public_addr":"172.21.15.76:6817/1166289696","cluster_addr":"172.21.15.76:6819/1166289696","heartbeat_back_addr":"172.21.15.76:6825/1166289696","heartbeat_front_addr":"172.21.15.76:6821/1166289696","state":["exists","up"]},{"osd":2,"uuid":"7b7745f1-856e-4a1c-a9ff-b13ccff6881a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6824","nonce":2582758863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6827","nonce":2582758863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6831","nonce":2582758863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2582758863},{"type":"v1","addr":"172.21.15.76:6829","nonce":2582758863}]},"public_addr":"172.21.15.76:6824/2582758863","cluster_addr":"172.21.15.76:6827/2582758863","heartbeat_back_addr":"172.21.15.76:6831/2582758863","heartbeat_front_addr":"172.21.15.76:6829/2582758863","state":["exists","up"]},{"osd":3,"uuid":"0d039f94-1da7-433b-aaeb-a2db8515d851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6801","nonce":276379883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6804","nonce":276379883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6810","nonce":276379883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":276379883},{"type":"v1","addr":"172.21.15.76:6807","nonce":276379883}]},"public_addr":"172.21.15.76:6801/276379883","cluster_addr":"172.21.15.76:6804/276379883","heartbeat_back_addr":"172.21.15.76:6810/276379883","heartbeat_front_addr":"172.21.15.76:6807/276379883","state":["exists","up"]},{"osd":4,"uuid":"c12b0e9b-1482-4980-8e43-5174eab87fa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6817","nonce":1824552967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6819","nonce":1824552967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6823","nonce":1824552967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":1824552967},{"type":"v1","addr":"172.21.15.113:6821","nonce":1824552967}]},"public_addr":"172.21.15.113:6817/1824552967","cluster_addr":"172.21.15.113:6819/1824552967","heartbeat_back_addr":"172.21.15.113:6823/1824552967","heartbeat_front_addr":"172.21.15.113:6821/1824552967","state":["exists","up"]},{"osd":5,"uuid":"57bdf820-32bd-4e3a-b9e3-d9669e0987ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6801","nonce":3754040883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6803","nonce":3754040883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6807","nonce":3754040883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3754040883},{"type":"v1","addr":"172.21.15.113:6805","nonce":3754040883}]},"public_addr":"172.21.15.113:6801/3754040883","cluster_addr":"172.21.15.113:6803/3754040883","heartbeat_back_addr":"172.21.15.113:6807/3754040883","heartbeat_front_addr":"172.21.15.113:6805/3754040883","state":["exists","up"]},{"osd":6,"uuid":"4211c9a9-c4f8-4519-8c8c-18a603d61651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6809","nonce":570146941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6811","nonce":570146941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6815","nonce":570146941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":570146941},{"type":"v1","addr":"172.21.15.113:6813","nonce":570146941}]},"public_addr":"172.21.15.113:6809/570146941","cluster_addr":"172.21.15.113:6811/570146941","heartbeat_back_addr":"172.21.15.113:6815/570146941","heartbeat_front_addr":"172.21.15.113:6813/570146941","state":["exists","up"]},{"osd":7,"uuid":"40f5a75c-ead7-401e-a2fa-03152ce75255","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6825","nonce":2785664875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6827","nonce":2785664875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6831","nonce":2785664875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":2785664875},{"type":"v1","addr":"172.21.15.113:6829","nonce":2785664875}]},"public_addr":"172.21.15.113:6825/2785664875","cluster_addr":"172.21.15.113:6827/2785664875","heartbeat_back_addr":"172.21.15.113:6831/2785664875","heartbeat_front_addr":"172.21.15.113:6829/2785664875","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.933432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:14.015622+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.999826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.975331+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.993199+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.970735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.944615+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-09T03:45:13.986235+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.17","mappings":[{"from":0,"to":2}]},{"pgid":"17.36","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/497547142":"2023-11-09T04:55:40.451393+0000","172.21.15.76:6834/757529522":"2023-11-10T03:55:31.678438+0000","172.21.15.76:6832/3697850101":"2023-11-10T03:51:39.442929+0000","172.21.15.113:6837/3449910842":"2023-11-10T03:50:59.611256+0000","172.21.15.76:6832/1536527256":"2023-11-10T03:52:34.213804+0000","172.21.15.76:6835/3229239133":"2023-11-10T03:51:32.624856+0000","172.21.15.113:6836/445306598":"2023-11-10T03:52:27.406838+0000","172.21.15.76:6835/1663988060":"2023-11-10T03:50:33.665092+0000","172.21.15.76:6834/1663988060":"2023-11-10T03:50:33.665092+0000","172.21.15.113:6837/3280098776":"2023-11-10T03:50:07.616780+0000","192.168.0.1:0/2519773110":"2023-11-09T04:54:14.823765+0000","172.21.15.76:6835/4011275120":"2023-11-10T03:49:42.029057+0000","172.21.15.76:6835/757529522":"2023-11-10T03:55:31.678438+0000","172.21.15.76:6834/4011275120":"2023-11-10T03:49:42.029057+0000","172.21.15.113:6837/3462040782":"2023-11-10T03:54:45.345437+0000","192.168.0.1:0/503424885":"2023-11-09T04:51:47.727422+0000","172.21.15.76:6834/3229239133":"2023-11-10T03:51:32.624856+0000","172.21.15.113:6837/445306598":"2023-11-10T03:52:27.406838+0000","172.21.15.113:6836/3280098776":"2023-11-10T03:50:07.616780+0000","192.168.0.1:0/3087804343":"2023-11-09T04:52:42.513556+0000","172.21.15.76:6834/546338722":"2023-11-10T03:53:17.470106+0000","172.21.15.76:6833/1536527256":"2023-11-10T03:52:34.213804+0000","172.21.15.76:6835/546338722":"2023-11-10T03:53:17.470106+0000","172.21.15.113:6836/7360844":"2023-11-10T03:53:24.718008+0000","172.21.15.113:6837/7360844":"2023-11-10T03:53:24.718008+0000","172.21.15.76:6835/2075292158":"2023-11-10T03:54:06.568191+0000","192.168.0.1:0/334503200":"2023-11-09T04:53:32.572840+0000","172.21.15.76:6834/2075292158":"2023-11-10T03:54:06.568191+0000","172.21.15.76:6833/3697850101":"2023-11-10T03:51:39.442929+0000","172.21.15.113:6836/3462040782":"2023-11-10T03:54:45.345437+0000","172.21.15.113:6836/3449910842":"2023-11-10T03:50:59.611256+0000","192.168.0.1:0/3870807407":"2023-11-09T04:54:56.671385+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":17,"snaps":[{"begin":16,"length":4}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-09T03:56:11.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.566+0000 7ff4d2ae0640 1 -- 172.21.15.76:0/2839577469 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff4b806ae70 msgr2=0x7ff4b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:11.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.566+0000 7ff4d2ae0640 1 --2- 172.21.15.76:0/2839577469 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff4b806ae70 0x7ff4b806d330 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc000f60 tx=0x7ff4bc007d00 comp rx=0 tx=0).stop 2023-11-09T03:56:11.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.566+0000 7ff4d2ae0640 1 -- 172.21.15.76:0/2839577469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4cc0b0c40 msgr2=0x7ff4cc0c76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:11.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.566+0000 7ff4d2ae0640 1 --2- 172.21.15.76:0/2839577469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4cc0b0c40 0x7ff4cc0c76e0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000c450 tx=0x7ff4c000c910 comp rx=0 tx=0).stop 2023-11-09T03:56:11.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.566+0000 7ff4d2ae0640 1 -- 172.21.15.76:0/2839577469 shutdown_connections 2023-11-09T03:56:11.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.566+0000 7ff4d2ae0640 1 --2- 172.21.15.76:0/2839577469 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7ff4b806ae70 0x7ff4b806d330 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.566+0000 7ff4d2ae0640 1 --2- 172.21.15.76:0/2839577469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4cc159e40 0x7ff4cc0c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.566+0000 7ff4d2ae0640 1 --2- 172.21.15.76:0/2839577469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4cc0b0c40 0x7ff4cc0c76e0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.566+0000 7ff4d2ae0640 1 --2- 172.21.15.76:0/2839577469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4cc070700 0x7ff4cc146d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.566+0000 7ff4d2ae0640 1 -- 172.21.15.76:0/2839577469 >> 172.21.15.76:0/2839577469 conn(0x7ff4cc0cb530 msgr2=0x7ff4cc0caa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:11.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.566+0000 7ff4d2ae0640 1 -- 172.21.15.76:0/2839577469 shutdown_connections 2023-11-09T03:56:11.568 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.566+0000 7ff4d2ae0640 1 -- 172.21.15.76:0/2839577469 wait complete. 2023-11-09T03:56:11.576 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-09T03:45:18.792162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-09T03:47:00.036540+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_data_pool', 'create_time': '2023-11-09T03:47:59.280601+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 70, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-09T03:47:59.742039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-09T03:48:55.490832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-11-09T03:48:56.352624+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '122', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 122, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-09T03:56:11.577 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-09T03:56:11.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.679+0000 7f6042f0f640 1 Processor -- start 2023-11-09T03:56:11.679 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.679+0000 7f6042f0f640 1 -- start start 2023-11-09T03:56:11.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.680+0000 7f6042f0f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f603c159e40 0x7f603c0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:11.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.680+0000 7f6042f0f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f603c06bb60 0x7f603c0b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:11.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.680+0000 7f6042f0f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f603c071500 con 0x7f603c06fff0 2023-11-09T03:56:11.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.680+0000 7f6042f0f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f603c15dae0 con 0x7f603c06bb60 2023-11-09T03:56:11.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.680+0000 7f6042f0f640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f603c070c60 con 0x7f603c159e40 2023-11-09T03:56:11.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.680+0000 7f603bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f603c06bb60 0x7f603c0b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:11.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.680+0000 7f603bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f603c06bb60 0x7f603c0b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40516/0 (socket says 172.21.15.76:40516) 2023-11-09T03:56:11.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.680+0000 7f603bfff640 1 -- 172.21.15.76:0/3512953288 learned_addr learned my addr 172.21.15.76:0/3512953288 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:56:11.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.680+0000 7f603bfff640 1 -- 172.21.15.76:0/3512953288 >> v2:172.21.15.113:3300/0 conn(0x7f603c159e40 msgr2=0x7f603c0b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:11.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.680+0000 7f603bfff640 1 --2- 172.21.15.76:0/3512953288 >> v2:172.21.15.113:3300/0 conn(0x7f603c159e40 0x7f603c0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.680+0000 7f603bfff640 1 -- 172.21.15.76:0/3512953288 >> v1:172.21.15.113:6789/0 conn(0x7f603c06fff0 legacy=0x7f603c070420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:11.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.680+0000 7f603bfff640 1 -- 172.21.15.76:0/3512953288 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f603c070f40 con 0x7f603c06bb60 2023-11-09T03:56:11.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.681+0000 7f603bfff640 1 --2- 172.21.15.76:0/3512953288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f603c06bb60 0x7f603c0b0a70 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f603c0ce7f0 tx=0x7f602802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de53b6bb1ab8393e server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:11.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.681+0000 7f603b7fe640 1 -- 172.21.15.76:0/3512953288 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6028031750 con 0x7f603c06bb60 2023-11-09T03:56:11.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.681+0000 7f603b7fe640 1 -- 172.21.15.76:0/3512953288 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60280318b0 con 0x7f603c06bb60 2023-11-09T03:56:11.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.681+0000 7f603b7fe640 1 -- 172.21.15.76:0/3512953288 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6028031c40 con 0x7f603c06bb60 2023-11-09T03:56:11.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.681+0000 7f6042f0f640 1 -- 172.21.15.76:0/3512953288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f603c06bb60 msgr2=0x7f603c0b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:11.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.681+0000 7f6042f0f640 1 --2- 172.21.15.76:0/3512953288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f603c06bb60 0x7f603c0b0a70 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f603c0ce7f0 tx=0x7f602802b7f0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.681+0000 7f6042f0f640 1 -- 172.21.15.76:0/3512953288 shutdown_connections 2023-11-09T03:56:11.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.681+0000 7f6042f0f640 1 --2- 172.21.15.76:0/3512953288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f603c06bb60 0x7f603c0b0a70 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.681+0000 7f6042f0f640 1 --2- 172.21.15.76:0/3512953288 >> v2:172.21.15.113:3300/0 conn(0x7f603c159e40 0x7f603c0b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.681+0000 7f6042f0f640 1 -- 172.21.15.76:0/3512953288 >> 172.21.15.76:0/3512953288 conn(0x7f603c0cb530 msgr2=0x7f603c0cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:11.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.681+0000 7f6042f0f640 1 -- 172.21.15.76:0/3512953288 shutdown_connections 2023-11-09T03:56:11.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.682+0000 7f6042f0f640 1 -- 172.21.15.76:0/3512953288 wait complete. 2023-11-09T03:56:11.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.683+0000 7f6042f0f640 1 Processor -- start 2023-11-09T03:56:11.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.683+0000 7f6042f0f640 1 -- start start 2023-11-09T03:56:11.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.683+0000 7f6042f0f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f603c06bb60 0x7f603c0c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:11.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.683+0000 7f6042f0f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f603c06fff0 0x7f603c142f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:11.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.684+0000 7f6042f0f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f603c159e40 0x7f603c144f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:11.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.684+0000 7f6042f0f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c145fc0 con 0x7f603c159e40 2023-11-09T03:56:11.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.684+0000 7f6042f0f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c156a80 con 0x7f603c06bb60 2023-11-09T03:56:11.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.684+0000 7f6042f0f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f603c145490 con 0x7f603c06fff0 2023-11-09T03:56:11.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.684+0000 7f60414ac640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f603c159e40 0x7f603c144f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:11.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.684+0000 7f60414ac640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f603c159e40 0x7f603c144f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40528/0 (socket says 172.21.15.76:40528) 2023-11-09T03:56:11.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.684+0000 7f60414ac640 1 -- 172.21.15.76:0/3653431304 learned_addr learned my addr 172.21.15.76:0/3653431304 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:56:11.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.684+0000 7f6040cab640 1 --2- 172.21.15.76:0/3653431304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f603c06bb60 0x7f603c0c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:11.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.684+0000 7f603bfff640 1 --2- 172.21.15.76:0/3653431304 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f603c06fff0 0x7f603c142f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:11.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.684+0000 7f60414ac640 1 -- 172.21.15.76:0/3653431304 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f603c06fff0 msgr2=0x7f603c142f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:11.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.684+0000 7f60414ac640 1 --2- 172.21.15.76:0/3653431304 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f603c06fff0 0x7f603c142f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.684+0000 7f60414ac640 1 -- 172.21.15.76:0/3653431304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f603c06bb60 msgr2=0x7f603c0c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:11.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.684+0000 7f60414ac640 1 --2- 172.21.15.76:0/3653431304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f603c06bb60 0x7f603c0c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.684+0000 7f60414ac640 1 -- 172.21.15.76:0/3653431304 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60280094d0 con 0x7f603c159e40 2023-11-09T03:56:11.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.684+0000 7f603bfff640 1 --2- 172.21.15.76:0/3653431304 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f603c06fff0 0x7f603c142f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:56:11.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.684+0000 7f60414ac640 1 --2- 172.21.15.76:0/3653431304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f603c159e40 0x7f603c144f50 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f603400c780 tx=0x7f603400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:11.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.684+0000 7f60397fa640 1 -- 172.21.15.76:0/3653431304 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6034003d80 con 0x7f603c159e40 2023-11-09T03:56:11.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.684+0000 7f60397fa640 1 -- 172.21.15.76:0/3653431304 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603400ce90 con 0x7f603c159e40 2023-11-09T03:56:11.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.684+0000 7f6042f0f640 1 -- 172.21.15.76:0/3653431304 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f603c156750 con 0x7f603c159e40 2023-11-09T03:56:11.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.684+0000 7f60397fa640 1 -- 172.21.15.76:0/3653431304 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60340052b0 con 0x7f603c159e40 2023-11-09T03:56:11.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.684+0000 7f6042f0f640 1 -- 172.21.15.76:0/3653431304 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603c145a00 con 0x7f603c159e40 2023-11-09T03:56:11.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.685+0000 7f6042f0f640 1 -- 172.21.15.76:0/3653431304 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f603c156bf0 con 0x7f603c159e40 2023-11-09T03:56:11.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.686+0000 7f60397fa640 1 -- 172.21.15.76:0/3653431304 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6034005410 con 0x7f603c159e40 2023-11-09T03:56:11.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.686+0000 7f60397fa640 1 --2- 172.21.15.76:0/3653431304 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f602006ae70 0x7f602006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:11.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.686+0000 7f60397fa640 1 -- 172.21.15.76:0/3653431304 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8582+0+0 (secure 0 0 0) 0x7f60340895e0 con 0x7f603c159e40 2023-11-09T03:56:11.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.686+0000 7f6040cab640 1 --2- 172.21.15.76:0/3653431304 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f602006ae70 0x7f602006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:11.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.687+0000 7f6040cab640 1 --2- 172.21.15.76:0/3653431304 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f602006ae70 0x7f602006d330 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f60300098e0 tx=0x7f6030009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:11.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.690+0000 7f60397fa640 1 -- 172.21.15.76:0/3653431304 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6034053b30 con 0x7f603c159e40 2023-11-09T03:56:11.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.859+0000 7f6042f0f640 1 -- 172.21.15.76:0/3653431304 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f603c0684b0 con 0x7f603c159e40 2023-11-09T03:56:11.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.859+0000 7f60397fa640 1 -- 172.21.15.76:0/3653431304 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6034057150 con 0x7f603c159e40 2023-11-09T03:56:11.859 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2023-11-09T03:56:11.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.861+0000 7f6042f0f640 1 -- 172.21.15.76:0/3653431304 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f602006ae70 msgr2=0x7f602006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:11.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.861+0000 7f6042f0f640 1 --2- 172.21.15.76:0/3653431304 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f602006ae70 0x7f602006d330 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f60300098e0 tx=0x7f6030009060 comp rx=0 tx=0).stop 2023-11-09T03:56:11.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.861+0000 7f6042f0f640 1 -- 172.21.15.76:0/3653431304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f603c159e40 msgr2=0x7f603c144f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:11.861 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.861+0000 7f6042f0f640 1 --2- 172.21.15.76:0/3653431304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f603c159e40 0x7f603c144f50 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f603400c780 tx=0x7f603400cc40 comp rx=0 tx=0).stop 2023-11-09T03:56:11.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.861+0000 7f6042f0f640 1 -- 172.21.15.76:0/3653431304 shutdown_connections 2023-11-09T03:56:11.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.861+0000 7f6042f0f640 1 --2- 172.21.15.76:0/3653431304 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f602006ae70 0x7f602006d330 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.861+0000 7f6042f0f640 1 --2- 172.21.15.76:0/3653431304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f603c159e40 0x7f603c144f50 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.861+0000 7f6042f0f640 1 --2- 172.21.15.76:0/3653431304 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f603c06fff0 0x7f603c142f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.861+0000 7f6042f0f640 1 --2- 172.21.15.76:0/3653431304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f603c06bb60 0x7f603c0c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.861+0000 7f6042f0f640 1 -- 172.21.15.76:0/3653431304 >> 172.21.15.76:0/3653431304 conn(0x7f603c0cb530 msgr2=0x7f603c070420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:11.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.861+0000 7f6042f0f640 1 -- 172.21.15.76:0/3653431304 shutdown_connections 2023-11-09T03:56:11.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.861+0000 7f6042f0f640 1 -- 172.21.15.76:0/3653431304 wait complete. 2023-11-09T03:56:11.871 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-09T03:56:11.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.974+0000 7f851bd65640 1 Processor -- start 2023-11-09T03:56:11.974 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.975+0000 7f851bd65640 1 -- start start 2023-11-09T03:56:11.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.975+0000 7f851bd65640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f851415d100 0x7f85140ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:11.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.975+0000 7f851bd65640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f851415d4e0 0x7f85140d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:11.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.975+0000 7f851bd65640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f85140d3b60 0x7f85140d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:11.976 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.975+0000 7f851bd65640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85140b32b0 con 0x7f851415d4e0 2023-11-09T03:56:11.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.975+0000 7f851bd65640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85140cee10 con 0x7f85140d3b60 2023-11-09T03:56:11.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.975+0000 7f851bd65640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85140b2980 con 0x7f851415d100 2023-11-09T03:56:11.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.976+0000 7f8519300640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f851415d4e0 0x7f85140d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:11.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.976+0000 7f8519300640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f851415d4e0 0x7f85140d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40534/0 (socket says 172.21.15.76:40534) 2023-11-09T03:56:11.977 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.976+0000 7f8519300640 1 -- 172.21.15.76:0/1695983471 learned_addr learned my addr 172.21.15.76:0/1695983471 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:56:11.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.976+0000 7f851a302640 1 --2- 172.21.15.76:0/1695983471 >> v2:172.21.15.113:3300/0 conn(0x7f85140d3b60 0x7f85140d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:11.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.976+0000 7f8519b01640 1 --2- 172.21.15.76:0/1695983471 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f851415d100 0x7f85140ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:11.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.976+0000 7f8519300640 1 -- 172.21.15.76:0/1695983471 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f851415d100 msgr2=0x7f85140ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:11.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.976+0000 7f8519300640 1 --2- 172.21.15.76:0/1695983471 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f851415d100 0x7f85140ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.978 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.976+0000 7f8519300640 1 -- 172.21.15.76:0/1695983471 >> v2:172.21.15.113:3300/0 conn(0x7f85140d3b60 msgr2=0x7f85140d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:11.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.976+0000 7f8519300640 1 --2- 172.21.15.76:0/1695983471 >> v2:172.21.15.113:3300/0 conn(0x7f85140d3b60 0x7f85140d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.976+0000 7f8519300640 1 -- 172.21.15.76:0/1695983471 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85140b2c60 con 0x7f851415d4e0 2023-11-09T03:56:11.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.976+0000 7f851a302640 1 --2- 172.21.15.76:0/1695983471 >> v2:172.21.15.113:3300/0 conn(0x7f85140d3b60 0x7f85140d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:56:11.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.977+0000 7f8519300640 1 --2- 172.21.15.76:0/1695983471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f851415d4e0 0x7f85140d14d0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f85140b27a0 tx=0x7f850802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7570113c4cd06f server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:11.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.977+0000 7f8518aff640 1 -- 172.21.15.76:0/1695983471 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8508031870 con 0x7f851415d4e0 2023-11-09T03:56:11.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.977+0000 7f8518aff640 1 -- 172.21.15.76:0/1695983471 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85080319d0 con 0x7f851415d4e0 2023-11-09T03:56:11.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.977+0000 7f8518aff640 1 -- 172.21.15.76:0/1695983471 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8508031d60 con 0x7f851415d4e0 2023-11-09T03:56:11.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.977+0000 7f851bd65640 1 -- 172.21.15.76:0/1695983471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f851415d4e0 msgr2=0x7f85140d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:11.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.977+0000 7f851bd65640 1 --2- 172.21.15.76:0/1695983471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f851415d4e0 0x7f85140d14d0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f85140b27a0 tx=0x7f850802b910 comp rx=0 tx=0).stop 2023-11-09T03:56:11.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.977+0000 7f851bd65640 1 -- 172.21.15.76:0/1695983471 shutdown_connections 2023-11-09T03:56:11.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.977+0000 7f851bd65640 1 --2- 172.21.15.76:0/1695983471 >> v2:172.21.15.113:3300/0 conn(0x7f85140d3b60 0x7f85140d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.977+0000 7f851bd65640 1 --2- 172.21.15.76:0/1695983471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f851415d4e0 0x7f85140d14d0 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.977+0000 7f851bd65640 1 --2- 172.21.15.76:0/1695983471 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f851415d100 0x7f85140ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.977+0000 7f851bd65640 1 -- 172.21.15.76:0/1695983471 >> 172.21.15.76:0/1695983471 conn(0x7f85140d2ef0 msgr2=0x7f85140d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:11.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.978+0000 7f851bd65640 1 -- 172.21.15.76:0/1695983471 shutdown_connections 2023-11-09T03:56:11.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.978+0000 7f851bd65640 1 -- 172.21.15.76:0/1695983471 wait complete. 2023-11-09T03:56:11.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.979+0000 7f851bd65640 1 Processor -- start 2023-11-09T03:56:11.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.979+0000 7f851bd65640 1 -- start start 2023-11-09T03:56:11.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.979+0000 7f851bd65640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85140d3b60 0x7f85140b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:11.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.979+0000 7f851bd65640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f851415d100 0x7f851409d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:11.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.979+0000 7f851bd65640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f851415d4e0 0x7f85140a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:11.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.979+0000 7f851bd65640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851409e3f0 con 0x7f85140d3b60 2023-11-09T03:56:11.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.979+0000 7f851bd65640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851409e6d0 con 0x7f851415d4e0 2023-11-09T03:56:11.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.979+0000 7f851bd65640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851409e9b0 con 0x7f851415d100 2023-11-09T03:56:11.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.979+0000 7f8519b01640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85140d3b60 0x7f85140b81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:11.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.980+0000 7f8519b01640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85140d3b60 0x7f85140b81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40538/0 (socket says 172.21.15.76:40538) 2023-11-09T03:56:11.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.980+0000 7f8519b01640 1 -- 172.21.15.76:0/2714599680 learned_addr learned my addr 172.21.15.76:0/2714599680 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:56:11.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.980+0000 7f8519300640 1 --2- 172.21.15.76:0/2714599680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f851415d100 0x7f851409d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:11.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.980+0000 7f851a302640 1 --2- 172.21.15.76:0/2714599680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f851415d4e0 0x7f85140a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:11.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.980+0000 7f8519b01640 1 -- 172.21.15.76:0/2714599680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f851415d100 msgr2=0x7f851409d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:11.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.980+0000 7f8519b01640 1 --2- 172.21.15.76:0/2714599680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f851415d100 0x7f851409d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.980+0000 7f8519b01640 1 -- 172.21.15.76:0/2714599680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f851415d4e0 msgr2=0x7f85140a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:11.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.980+0000 7f8519b01640 1 --2- 172.21.15.76:0/2714599680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f851415d4e0 0x7f85140a0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:11.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.980+0000 7f8519b01640 1 -- 172.21.15.76:0/2714599680 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85080094d0 con 0x7f85140d3b60 2023-11-09T03:56:11.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.980+0000 7f851a302640 1 --2- 172.21.15.76:0/2714599680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f851415d4e0 0x7f85140a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:56:11.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.980+0000 7f8519300640 1 --2- 172.21.15.76:0/2714599680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f851415d100 0x7f851409d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:56:11.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.980+0000 7f8519b01640 1 --2- 172.21.15.76:0/2714599680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85140d3b60 0x7f85140b81f0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f850c009fc0 tx=0x7f850c00e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:11.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.980+0000 7f85127fc640 1 -- 172.21.15.76:0/2714599680 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c014620 con 0x7f85140d3b60 2023-11-09T03:56:11.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.980+0000 7f85127fc640 1 -- 172.21.15.76:0/2714599680 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850c014780 con 0x7f85140d3b60 2023-11-09T03:56:11.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.980+0000 7f851bd65640 1 -- 172.21.15.76:0/2714599680 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f851409ec90 con 0x7f85140d3b60 2023-11-09T03:56:11.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.980+0000 7f85127fc640 1 -- 172.21.15.76:0/2714599680 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c014b30 con 0x7f85140d3b60 2023-11-09T03:56:11.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.980+0000 7f851bd65640 1 -- 172.21.15.76:0/2714599680 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8514152620 con 0x7f85140d3b60 2023-11-09T03:56:11.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.981+0000 7f851bd65640 1 -- 172.21.15.76:0/2714599680 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85140684b0 con 0x7f85140d3b60 2023-11-09T03:56:11.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.981+0000 7f85127fc640 1 -- 172.21.15.76:0/2714599680 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f850c014c90 con 0x7f85140d3b60 2023-11-09T03:56:11.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.982+0000 7f85127fc640 1 --2- 172.21.15.76:0/2714599680 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f850006ae70 0x7f850006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:11.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.982+0000 7f85127fc640 1 -- 172.21.15.76:0/2714599680 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8582+0+0 (secure 0 0 0) 0x7f850c08ab40 con 0x7f85140d3b60 2023-11-09T03:56:11.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.982+0000 7f8519300640 1 --2- 172.21.15.76:0/2714599680 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f850006ae70 0x7f850006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:11.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.983+0000 7f8519300640 1 --2- 172.21.15.76:0/2714599680 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f850006ae70 0x7f850006d330 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f8508036000 tx=0x7f850803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:11.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:11.986+0000 7f85127fc640 1 -- 172.21.15.76:0/2714599680 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f850c0550f0 con 0x7f85140d3b60 2023-11-09T03:56:12.012 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~13s 2023-11-09T03:56:12.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.152+0000 7f851bd65640 1 -- 172.21.15.76:0/2714599680 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f85140d2cd0 con 0x7f85140d3b60 2023-11-09T03:56:12.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.152+0000 7f85127fc640 1 -- 172.21.15.76:0/2714599680 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v123) v1 ==== 103+0+11 (secure 0 0 0) 0x7f850c058710 con 0x7f85140d3b60 2023-11-09T03:56:12.152 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:56:12.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.154+0000 7f851bd65640 1 -- 172.21.15.76:0/2714599680 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f850006ae70 msgr2=0x7f850006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:12.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.154+0000 7f851bd65640 1 --2- 172.21.15.76:0/2714599680 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f850006ae70 0x7f850006d330 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f8508036000 tx=0x7f850803c040 comp rx=0 tx=0).stop 2023-11-09T03:56:12.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.154+0000 7f851bd65640 1 -- 172.21.15.76:0/2714599680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85140d3b60 msgr2=0x7f85140b81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:12.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.154+0000 7f851bd65640 1 --2- 172.21.15.76:0/2714599680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85140d3b60 0x7f85140b81f0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f850c009fc0 tx=0x7f850c00e570 comp rx=0 tx=0).stop 2023-11-09T03:56:12.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.154+0000 7f851bd65640 1 -- 172.21.15.76:0/2714599680 shutdown_connections 2023-11-09T03:56:12.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.154+0000 7f851bd65640 1 --2- 172.21.15.76:0/2714599680 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f850006ae70 0x7f850006d330 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:12.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.154+0000 7f851bd65640 1 --2- 172.21.15.76:0/2714599680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f851415d4e0 0x7f85140a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:12.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.154+0000 7f851bd65640 1 --2- 172.21.15.76:0/2714599680 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f851415d100 0x7f851409d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:12.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.154+0000 7f851bd65640 1 --2- 172.21.15.76:0/2714599680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85140d3b60 0x7f85140b81f0 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:12.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.154+0000 7f851bd65640 1 -- 172.21.15.76:0/2714599680 >> 172.21.15.76:0/2714599680 conn(0x7f85140d2ef0 msgr2=0x7f851406bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:12.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.154+0000 7f851bd65640 1 -- 172.21.15.76:0/2714599680 shutdown_connections 2023-11-09T03:56:12.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.154+0000 7f851bd65640 1 -- 172.21.15.76:0/2714599680 wait complete. 2023-11-09T03:56:12.165 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-09T03:56:12.267 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.268+0000 7f3a26533640 1 Processor -- start 2023-11-09T03:56:12.268 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.268+0000 7f3a26533640 1 -- start start 2023-11-09T03:56:12.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.269+0000 7f3a26533640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3a20159e40 0x7f3a200b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:12.269 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.269+0000 7f3a26533640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a20071500 con 0x7f3a20070700 2023-11-09T03:56:12.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.269+0000 7f3a26533640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a2015dae0 con 0x7f3a2006bb60 2023-11-09T03:56:12.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.269+0000 7f3a26533640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a20070c60 con 0x7f3a20159e40 2023-11-09T03:56:12.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.269+0000 7f3a1ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3a20159e40 0x7f3a200b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:12.270 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.269+0000 7f3a24ad0640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f3a20070700 0x7f3a200cd790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47884/0 (socket says 172.21.15.76:47884) 2023-11-09T03:56:12.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.269+0000 7f3a24ad0640 1 -- 172.21.15.76:0/3571258953 learned_addr learned my addr 172.21.15.76:0/3571258953 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:56:12.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.269+0000 7f3a1ffff640 1 -- 172.21.15.76:0/3571258953 >> v1:172.21.15.113:6789/0 conn(0x7f3a2006bb60 legacy=0x7f3a2006fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:12.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.269+0000 7f3a1ffff640 1 -- 172.21.15.76:0/3571258953 >> v1:172.21.15.76:6789/0 conn(0x7f3a20070700 legacy=0x7f3a200cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:12.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.269+0000 7f3a1ffff640 1 -- 172.21.15.76:0/3571258953 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a20070f40 con 0x7f3a20159e40 2023-11-09T03:56:12.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.269+0000 7f3a1effd640 1 -- 172.21.15.76:0/3571258953 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2923485187 0 0) 0x7f3a20071500 con 0x7f3a20070700 2023-11-09T03:56:12.271 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.270+0000 7f3a1ffff640 1 --2- 172.21.15.76:0/3571258953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a20159e40 0x7f3a200b0530 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f3a100097a0 tx=0x7f3a1002b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cfd23c28a7a95c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:12.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.270+0000 7f3a1effd640 1 -- 172.21.15.76:0/3571258953 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1002f050 con 0x7f3a20159e40 2023-11-09T03:56:12.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.270+0000 7f3a1effd640 1 -- 172.21.15.76:0/3571258953 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a10031700 con 0x7f3a20159e40 2023-11-09T03:56:12.272 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.270+0000 7f3a1effd640 1 -- 172.21.15.76:0/3571258953 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a10031a90 con 0x7f3a20159e40 2023-11-09T03:56:12.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.270+0000 7f3a26533640 1 -- 172.21.15.76:0/3571258953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a20159e40 msgr2=0x7f3a200b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:12.275 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.270+0000 7f3a26533640 1 --2- 172.21.15.76:0/3571258953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a20159e40 0x7f3a200b0530 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f3a100097a0 tx=0x7f3a1002b7a0 comp rx=0 tx=0).stop 2023-11-09T03:56:12.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.270+0000 7f3a26533640 1 -- 172.21.15.76:0/3571258953 shutdown_connections 2023-11-09T03:56:12.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.270+0000 7f3a26533640 1 --2- 172.21.15.76:0/3571258953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a20159e40 0x7f3a200b0530 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:12.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.270+0000 7f3a26533640 1 -- 172.21.15.76:0/3571258953 >> 172.21.15.76:0/3571258953 conn(0x7f3a200cb530 msgr2=0x7f3a200cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:12.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.270+0000 7f3a26533640 1 -- 172.21.15.76:0/3571258953 shutdown_connections 2023-11-09T03:56:12.276 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.270+0000 7f3a26533640 1 -- 172.21.15.76:0/3571258953 wait complete. 2023-11-09T03:56:12.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.272+0000 7f3a26533640 1 Processor -- start 2023-11-09T03:56:12.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.272+0000 7f3a26533640 1 -- start start 2023-11-09T03:56:12.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.272+0000 7f3a26533640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a2006bb60 0x7f3a20156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:12.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.272+0000 7f3a26533640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a20070700 0x7f3a20156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:12.277 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.272+0000 7f3a26533640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a20159e40 0x7f3a200c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:12.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.272+0000 7f3a26533640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a201f06d0 con 0x7f3a20159e40 2023-11-09T03:56:12.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.272+0000 7f3a26533640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a200cd520 con 0x7f3a2006bb60 2023-11-09T03:56:12.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.272+0000 7f3a26533640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a200cd800 con 0x7f3a20070700 2023-11-09T03:56:12.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.273+0000 7f3a24ad0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a20159e40 0x7f3a200c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:12.278 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.273+0000 7f3a24ad0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a20159e40 0x7f3a200c7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40556/0 (socket says 172.21.15.76:40556) 2023-11-09T03:56:12.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.273+0000 7f3a24ad0640 1 -- 172.21.15.76:0/4026497847 learned_addr learned my addr 172.21.15.76:0/4026497847 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:56:12.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.273+0000 7f3a1ffff640 1 --2- 172.21.15.76:0/4026497847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a2006bb60 0x7f3a20156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:12.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.273+0000 7f3a1f7fe640 1 --2- 172.21.15.76:0/4026497847 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a20070700 0x7f3a20156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:12.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.273+0000 7f3a24ad0640 1 -- 172.21.15.76:0/4026497847 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a20070700 msgr2=0x7f3a20156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:12.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.273+0000 7f3a24ad0640 1 --2- 172.21.15.76:0/4026497847 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a20070700 0x7f3a20156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:12.279 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.273+0000 7f3a24ad0640 1 -- 172.21.15.76:0/4026497847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a2006bb60 msgr2=0x7f3a20156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:12.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.273+0000 7f3a24ad0640 1 --2- 172.21.15.76:0/4026497847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a2006bb60 0x7f3a20156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:12.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.273+0000 7f3a24ad0640 1 -- 172.21.15.76:0/4026497847 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a10009400 con 0x7f3a20159e40 2023-11-09T03:56:12.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.273+0000 7f3a24ad0640 1 --2- 172.21.15.76:0/4026497847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a20159e40 0x7f3a200c7e30 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800c7e0 tx=0x7f3a1800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:12.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.273+0000 7f3a1cff9640 1 -- 172.21.15.76:0/4026497847 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1800a640 con 0x7f3a20159e40 2023-11-09T03:56:12.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.273+0000 7f3a1cff9640 1 -- 172.21.15.76:0/4026497847 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1800a7c0 con 0x7f3a20159e40 2023-11-09T03:56:12.280 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.273+0000 7f3a1cff9640 1 -- 172.21.15.76:0/4026497847 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1800ab70 con 0x7f3a20159e40 2023-11-09T03:56:12.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.273+0000 7f3a26533640 1 -- 172.21.15.76:0/4026497847 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a200cdae0 con 0x7f3a20159e40 2023-11-09T03:56:12.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.273+0000 7f3a26533640 1 -- 172.21.15.76:0/4026497847 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a200ce450 con 0x7f3a20159e40 2023-11-09T03:56:12.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.274+0000 7f3a26533640 1 -- 172.21.15.76:0/4026497847 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a201f0840 con 0x7f3a20159e40 2023-11-09T03:56:12.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.274+0000 7f3a1cff9640 1 -- 172.21.15.76:0/4026497847 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3a1800acd0 con 0x7f3a20159e40 2023-11-09T03:56:12.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.275+0000 7f3a1cff9640 1 --2- 172.21.15.76:0/4026497847 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3a0c06ae70 0x7f3a0c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:12.281 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.275+0000 7f3a1cff9640 1 -- 172.21.15.76:0/4026497847 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8582+0+0 (secure 0 0 0) 0x7f3a18089730 con 0x7f3a20159e40 2023-11-09T03:56:12.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.275+0000 7f3a1ffff640 1 --2- 172.21.15.76:0/4026497847 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3a0c06ae70 0x7f3a0c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:12.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.276+0000 7f3a1ffff640 1 --2- 172.21.15.76:0/4026497847 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3a0c06ae70 0x7f3a0c06d330 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f3a10035000 tx=0x7f3a10008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:12.282 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.279+0000 7f3a1cff9640 1 -- 172.21.15.76:0/4026497847 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3a201f0840 con 0x7f3a20159e40 2023-11-09T03:56:12.443 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.444+0000 7f3a26533640 1 -- 172.21.15.76:0/4026497847 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7f3a201f0840 con 0x7f3a20159e40 2023-11-09T03:56:12.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.444+0000 7f3a1cff9640 1 -- 172.21.15.76:0/4026497847 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v123) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3a201f0840 con 0x7f3a20159e40 2023-11-09T03:56:12.444 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:56:12.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.446+0000 7f3a26533640 1 -- 172.21.15.76:0/4026497847 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3a0c06ae70 msgr2=0x7f3a0c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:12.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.446+0000 7f3a26533640 1 --2- 172.21.15.76:0/4026497847 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3a0c06ae70 0x7f3a0c06d330 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f3a10035000 tx=0x7f3a10008030 comp rx=0 tx=0).stop 2023-11-09T03:56:12.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.446+0000 7f3a26533640 1 -- 172.21.15.76:0/4026497847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a20159e40 msgr2=0x7f3a200c7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:12.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.446+0000 7f3a26533640 1 --2- 172.21.15.76:0/4026497847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a20159e40 0x7f3a200c7e30 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800c7e0 tx=0x7f3a1800cca0 comp rx=0 tx=0).stop 2023-11-09T03:56:12.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.446+0000 7f3a26533640 1 -- 172.21.15.76:0/4026497847 shutdown_connections 2023-11-09T03:56:12.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.446+0000 7f3a26533640 1 --2- 172.21.15.76:0/4026497847 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f3a0c06ae70 0x7f3a0c06d330 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:12.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.446+0000 7f3a26533640 1 --2- 172.21.15.76:0/4026497847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a20159e40 0x7f3a200c7e30 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:12.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.446+0000 7f3a26533640 1 --2- 172.21.15.76:0/4026497847 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a20070700 0x7f3a20156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:12.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.446+0000 7f3a26533640 1 --2- 172.21.15.76:0/4026497847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a2006bb60 0x7f3a20156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:12.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.446+0000 7f3a26533640 1 -- 172.21.15.76:0/4026497847 >> 172.21.15.76:0/4026497847 conn(0x7f3a200cb530 msgr2=0x7f3a200ae4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:12.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.446+0000 7f3a26533640 1 -- 172.21.15.76:0/4026497847 shutdown_connections 2023-11-09T03:56:12.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.446+0000 7f3a26533640 1 -- 172.21.15.76:0/4026497847 wait complete. 2023-11-09T03:56:12.456 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-09T03:56:12.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.559+0000 7f5f2b444640 1 Processor -- start 2023-11-09T03:56:12.559 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.560+0000 7f5f2b444640 1 -- start start 2023-11-09T03:56:12.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.560+0000 7f5f2b444640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f240d1be0 0x7f5f24167e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:12.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.560+0000 7f5f2b444640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f240b32b0 con 0x7f5f2415d4e0 2023-11-09T03:56:12.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.560+0000 7f5f2b444640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f240cee10 con 0x7f5f2415d100 2023-11-09T03:56:12.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.560+0000 7f5f2b444640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f240cec20 con 0x7f5f240d1be0 2023-11-09T03:56:12.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.561+0000 7f5f289df640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f5f2415d4e0 0x7f5f240d14d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50556/0 (socket says 172.21.15.76:50556) 2023-11-09T03:56:12.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.561+0000 7f5f289df640 1 -- 172.21.15.76:0/1344380473 learned_addr learned my addr 172.21.15.76:0/1344380473 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:56:12.562 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.561+0000 7f5f299e1640 1 --2- 172.21.15.76:0/1344380473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f240d1be0 0x7f5f24167e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:12.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.561+0000 7f5f1bfff640 1 -- 172.21.15.76:0/1344380473 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2082629147 0 0) 0x7f5f240b32b0 con 0x7f5f2415d4e0 2023-11-09T03:56:12.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.561+0000 7f5f1bfff640 1 -- 172.21.15.76:0/1344380473 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f10003520 con 0x7f5f2415d4e0 2023-11-09T03:56:12.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.561+0000 7f5f1bfff640 1 -- 172.21.15.76:0/1344380473 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3262522879 0 0) 0x7f5f10003520 con 0x7f5f2415d4e0 2023-11-09T03:56:12.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.561+0000 7f5f1bfff640 1 -- 172.21.15.76:0/1344380473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f240d1be0 msgr2=0x7f5f24167e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:12.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.561+0000 7f5f1bfff640 1 --2- 172.21.15.76:0/1344380473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f240d1be0 0x7f5f24167e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:12.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.561+0000 7f5f1bfff640 1 -- 172.21.15.76:0/1344380473 >> v1:172.21.15.113:6789/0 conn(0x7f5f2415d100 legacy=0x7f5f240ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:12.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.561+0000 7f5f1bfff640 1 -- 172.21.15.76:0/1344380473 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f240b4750 con 0x7f5f2415d4e0 2023-11-09T03:56:12.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.561+0000 7f5f1bfff640 1 -- 172.21.15.76:0/1344380473 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7f5f14001670 con 0x7f5f2415d4e0 2023-11-09T03:56:12.564 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.561+0000 7f5f1bfff640 1 -- 172.21.15.76:0/1344380473 >> v1:172.21.15.76:6789/0 conn(0x7f5f2415d4e0 legacy=0x7f5f240d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:12.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.561+0000 7f5f1bfff640 1 --2- 172.21.15.76:0/1344380473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f240d1be0 0x7f5f100039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:12.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.561+0000 7f5f1bfff640 1 --2- 172.21.15.76:0/1344380473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f10003f20 0x7f5f100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:12.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.561+0000 7f5f1bfff640 1 --2- 172.21.15.76:0/1344380473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f10004d00 0x7f5f10005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:12.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.561+0000 7f5f1bfff640 1 -- 172.21.15.76:0/1344380473 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1000ba30 con 0x7f5f10003f20 2023-11-09T03:56:12.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.561+0000 7f5f1bfff640 1 -- 172.21.15.76:0/1344380473 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1000bba0 con 0x7f5f10004d00 2023-11-09T03:56:12.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.561+0000 7f5f1bfff640 1 -- 172.21.15.76:0/1344380473 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1000bd60 con 0x7f5f240d1be0 2023-11-09T03:56:12.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.561+0000 7f5f289df640 1 --2- 172.21.15.76:0/1344380473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f10003f20 0x7f5f100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:12.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.561+0000 7f5f299e1640 1 --2- 172.21.15.76:0/1344380473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f10004d00 0x7f5f10005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:12.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.561+0000 7f5f291e0640 1 --2- 172.21.15.76:0/1344380473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f240d1be0 0x7f5f100039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:12.566 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.562+0000 7f5f289df640 1 -- 172.21.15.76:0/1344380473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f240d1be0 msgr2=0x7f5f100039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:12.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.562+0000 7f5f289df640 1 --2- 172.21.15.76:0/1344380473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f240d1be0 0x7f5f100039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:12.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.562+0000 7f5f289df640 1 -- 172.21.15.76:0/1344380473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f10004d00 msgr2=0x7f5f10005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:12.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.562+0000 7f5f289df640 1 --2- 172.21.15.76:0/1344380473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f10004d00 0x7f5f10005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:12.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.562+0000 7f5f289df640 1 -- 172.21.15.76:0/1344380473 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f10003520 con 0x7f5f10003f20 2023-11-09T03:56:12.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.562+0000 7f5f289df640 1 --2- 172.21.15.76:0/1344380473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f10003f20 0x7f5f100047c0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f5f14007fd0 tx=0x7f5f100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df71e8e9e136952 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:12.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.562+0000 7f5f1bfff640 1 -- 172.21.15.76:0/1344380473 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f14005ad0 con 0x7f5f10003f20 2023-11-09T03:56:12.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.562+0000 7f5f1bfff640 1 -- 172.21.15.76:0/1344380473 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f14005c50 con 0x7f5f10003f20 2023-11-09T03:56:12.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.562+0000 7f5f1bfff640 1 -- 172.21.15.76:0/1344380473 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1402c870 con 0x7f5f10003f20 2023-11-09T03:56:12.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.563+0000 7f5f2b444640 1 -- 172.21.15.76:0/1344380473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f10003f20 msgr2=0x7f5f100047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:12.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.563+0000 7f5f2b444640 1 --2- 172.21.15.76:0/1344380473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f10003f20 0x7f5f100047c0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f5f14007fd0 tx=0x7f5f100057f0 comp rx=0 tx=0).stop 2023-11-09T03:56:12.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.563+0000 7f5f299e1640 1 -- 172.21.15.76:0/1344380473 reap_dead start 2023-11-09T03:56:12.571 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.563+0000 7f5f2b444640 1 -- 172.21.15.76:0/1344380473 shutdown_connections 2023-11-09T03:56:12.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.563+0000 7f5f2b444640 1 -- 172.21.15.76:0/1344380473 >> 172.21.15.76:0/1344380473 conn(0x7f5f240d2ef0 msgr2=0x7f5f240d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:12.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.563+0000 7f5f2b444640 1 -- 172.21.15.76:0/1344380473 shutdown_connections 2023-11-09T03:56:12.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.563+0000 7f5f2b444640 1 -- 172.21.15.76:0/1344380473 wait complete. 2023-11-09T03:56:12.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.564+0000 7f5f2b444640 1 Processor -- start 2023-11-09T03:56:12.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.564+0000 7f5f2b444640 1 -- start start 2023-11-09T03:56:12.572 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.565+0000 7f5f2b444640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f10003f20 0x7f5f240ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:12.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.565+0000 7f5f2b444640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f241600d0 0x7f5f241f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:12.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.565+0000 7f5f2b444640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f241f6320 0x7f5f2415ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:12.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.565+0000 7f5f2b444640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f24160800 con 0x7f5f241600d0 2023-11-09T03:56:12.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.565+0000 7f5f2b444640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f240cb970 con 0x7f5f10003f20 2023-11-09T03:56:12.573 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.565+0000 7f5f2b444640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f241f67a0 con 0x7f5f241f6320 2023-11-09T03:56:12.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.565+0000 7f5f289df640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f241600d0 0x7f5f241f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:12.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.565+0000 7f5f289df640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f241600d0 0x7f5f241f5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40220/0 (socket says 172.21.15.76:40220) 2023-11-09T03:56:12.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.565+0000 7f5f289df640 1 -- 172.21.15.76:0/1795661875 learned_addr learned my addr 172.21.15.76:0/1795661875 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:56:12.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.566+0000 7f5f299e1640 1 --2- 172.21.15.76:0/1795661875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f241f6320 0x7f5f2415ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:12.574 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.566+0000 7f5f291e0640 1 --2- 172.21.15.76:0/1795661875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f10003f20 0x7f5f240ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:12.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.566+0000 7f5f289df640 1 -- 172.21.15.76:0/1795661875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f241f6320 msgr2=0x7f5f2415ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:12.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.566+0000 7f5f289df640 1 --2- 172.21.15.76:0/1795661875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f241f6320 0x7f5f2415ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:12.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.566+0000 7f5f289df640 1 -- 172.21.15.76:0/1795661875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f10003f20 msgr2=0x7f5f240ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:12.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.566+0000 7f5f289df640 1 --2- 172.21.15.76:0/1795661875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f10003f20 0x7f5f240ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:12.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.566+0000 7f5f289df640 1 -- 172.21.15.76:0/1795661875 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2415d940 con 0x7f5f241600d0 2023-11-09T03:56:12.575 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.566+0000 7f5f289df640 1 --2- 172.21.15.76:0/1795661875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f241600d0 0x7f5f241f5de0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f5f1402c4e0 tx=0x7f5f1402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:12.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.566+0000 7f5f19ffb640 1 -- 172.21.15.76:0/1795661875 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1403c040 con 0x7f5f241600d0 2023-11-09T03:56:12.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.566+0000 7f5f19ffb640 1 -- 172.21.15.76:0/1795661875 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1402cca0 con 0x7f5f241600d0 2023-11-09T03:56:12.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.566+0000 7f5f2b444640 1 -- 172.21.15.76:0/1795661875 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f24160550 con 0x7f5f241600d0 2023-11-09T03:56:12.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.566+0000 7f5f19ffb640 1 -- 172.21.15.76:0/1795661875 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f140072b0 con 0x7f5f241600d0 2023-11-09T03:56:12.576 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.566+0000 7f5f2b444640 1 -- 172.21.15.76:0/1795661875 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f240cbe40 con 0x7f5f241600d0 2023-11-09T03:56:12.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.567+0000 7f5f2b444640 1 -- 172.21.15.76:0/1795661875 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f240684b0 con 0x7f5f241600d0 2023-11-09T03:56:12.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.568+0000 7f5f19ffb640 1 -- 172.21.15.76:0/1795661875 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5f14032070 con 0x7f5f241600d0 2023-11-09T03:56:12.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.568+0000 7f5f19ffb640 1 --2- 172.21.15.76:0/1795661875 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5f0806aea0 0x7f5f0806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:12.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.568+0000 7f5f19ffb640 1 -- 172.21.15.76:0/1795661875 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8582+0+0 (secure 0 0 0) 0x7f5f140ab5e0 con 0x7f5f241600d0 2023-11-09T03:56:12.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.568+0000 7f5f291e0640 1 --2- 172.21.15.76:0/1795661875 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5f0806aea0 0x7f5f0806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:12.577 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.569+0000 7f5f291e0640 1 --2- 172.21.15.76:0/1795661875 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5f0806aea0 0x7f5f0806d360 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c007fd0 tx=0x7f5f1c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:12.578 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.572+0000 7f5f19ffb640 1 -- 172.21.15.76:0/1795661875 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5f14075bb0 con 0x7f5f241600d0 2023-11-09T03:56:12.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.739+0000 7f5f2b444640 1 -- 172.21.15.76:0/1795661875 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7f5f240d2cd0 con 0x7f5f241600d0 2023-11-09T03:56:12.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.739+0000 7f5f19ffb640 1 -- 172.21.15.76:0/1795661875 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v123) v1 ==== 107+0+11 (secure 0 0 0) 0x7f5f140791d0 con 0x7f5f241600d0 2023-11-09T03:56:12.739 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:56:12.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.741+0000 7f5f2b444640 1 -- 172.21.15.76:0/1795661875 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5f0806aea0 msgr2=0x7f5f0806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:12.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.741+0000 7f5f2b444640 1 --2- 172.21.15.76:0/1795661875 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5f0806aea0 0x7f5f0806d360 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c007fd0 tx=0x7f5f1c009040 comp rx=0 tx=0).stop 2023-11-09T03:56:12.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.741+0000 7f5f2b444640 1 -- 172.21.15.76:0/1795661875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f241600d0 msgr2=0x7f5f241f5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:12.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.741+0000 7f5f2b444640 1 --2- 172.21.15.76:0/1795661875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f241600d0 0x7f5f241f5de0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f5f1402c4e0 tx=0x7f5f1402e070 comp rx=0 tx=0).stop 2023-11-09T03:56:12.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.741+0000 7f5f2b444640 1 -- 172.21.15.76:0/1795661875 shutdown_connections 2023-11-09T03:56:12.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.741+0000 7f5f2b444640 1 --2- 172.21.15.76:0/1795661875 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f5f0806aea0 0x7f5f0806d360 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:12.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.741+0000 7f5f2b444640 1 --2- 172.21.15.76:0/1795661875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f241f6320 0x7f5f2415ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:12.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.741+0000 7f5f2b444640 1 --2- 172.21.15.76:0/1795661875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f241600d0 0x7f5f241f5de0 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:12.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.741+0000 7f5f2b444640 1 --2- 172.21.15.76:0/1795661875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f10003f20 0x7f5f240ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:12.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.741+0000 7f5f2b444640 1 -- 172.21.15.76:0/1795661875 >> 172.21.15.76:0/1795661875 conn(0x7f5f240d2ef0 msgr2=0x7f5f2406b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:12.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.741+0000 7f5f2b444640 1 -- 172.21.15.76:0/1795661875 shutdown_connections 2023-11-09T03:56:12.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.741+0000 7f5f2b444640 1 -- 172.21.15.76:0/1795661875 wait complete. 2023-11-09T03:56:12.751 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-09T03:56:12.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.855+0000 7f9cac6b4640 1 Processor -- start 2023-11-09T03:56:12.855 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.855+0000 7f9cac6b4640 1 -- start start 2023-11-09T03:56:12.856 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.856+0000 7f9cac6b4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9ca40b0c40 0x7f9ca406fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:12.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.856+0000 7f9cac6b4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca406bb60 0x7f9ca4070530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:12.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.856+0000 7f9cac6b4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca4071500 con 0x7f9ca4159e40 2023-11-09T03:56:12.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.856+0000 7f9cac6b4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca4070a70 con 0x7f9ca40b0c40 2023-11-09T03:56:12.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.856+0000 7f9cac6b4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca415dae0 con 0x7f9ca406bb60 2023-11-09T03:56:12.857 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.856+0000 7f9ca9c4f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9ca40b0c40 0x7f9ca406fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:12.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.856+0000 7f9ca9c4f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9ca40b0c40 0x7f9ca406fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40228/0 (socket says 172.21.15.76:40228) 2023-11-09T03:56:12.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.856+0000 7f9ca9c4f640 1 -- 172.21.15.76:0/193050560 learned_addr learned my addr 172.21.15.76:0/193050560 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:56:12.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.857+0000 7f9ca9c4f640 1 -- 172.21.15.76:0/193050560 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca406bb60 msgr2=0x7f9ca4070530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:12.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.857+0000 7f9ca9c4f640 1 --2- 172.21.15.76:0/193050560 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca406bb60 0x7f9ca4070530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:12.858 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.857+0000 7f9ca9c4f640 1 -- 172.21.15.76:0/193050560 >> v1:172.21.15.113:6789/0 conn(0x7f9ca4159e40 legacy=0x7f9ca40b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:12.859 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.857+0000 7f9ca9c4f640 1 -- 172.21.15.76:0/193050560 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca4070c60 con 0x7f9ca40b0c40 2023-11-09T03:56:12.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.857+0000 7f9ca9c4f640 1 --2- 172.21.15.76:0/193050560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ca40b0c40 0x7f9ca406fff0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009870 tx=0x7f9c9402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4e05c0b9823e36 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:12.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.857+0000 7f9ca944e640 1 -- 172.21.15.76:0/193050560 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c940317c0 con 0x7f9ca40b0c40 2023-11-09T03:56:12.862 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.857+0000 7f9ca944e640 1 -- 172.21.15.76:0/193050560 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c94031920 con 0x7f9ca40b0c40 2023-11-09T03:56:12.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.857+0000 7f9ca944e640 1 -- 172.21.15.76:0/193050560 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c94031cb0 con 0x7f9ca40b0c40 2023-11-09T03:56:12.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.858+0000 7f9cac6b4640 1 -- 172.21.15.76:0/193050560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ca40b0c40 msgr2=0x7f9ca406fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:12.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.858+0000 7f9cac6b4640 1 --2- 172.21.15.76:0/193050560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ca40b0c40 0x7f9ca406fff0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009870 tx=0x7f9c9402b860 comp rx=0 tx=0).stop 2023-11-09T03:56:12.863 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.858+0000 7f9cac6b4640 1 -- 172.21.15.76:0/193050560 shutdown_connections 2023-11-09T03:56:12.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.858+0000 7f9cac6b4640 1 --2- 172.21.15.76:0/193050560 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca406bb60 0x7f9ca4070530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:12.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.858+0000 7f9cac6b4640 1 --2- 172.21.15.76:0/193050560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ca40b0c40 0x7f9ca406fff0 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:12.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.858+0000 7f9cac6b4640 1 -- 172.21.15.76:0/193050560 >> 172.21.15.76:0/193050560 conn(0x7f9ca40cb530 msgr2=0x7f9ca40cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:12.864 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.858+0000 7f9cac6b4640 1 -- 172.21.15.76:0/193050560 shutdown_connections 2023-11-09T03:56:12.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.858+0000 7f9cac6b4640 1 -- 172.21.15.76:0/193050560 wait complete. 2023-11-09T03:56:12.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.859+0000 7f9cac6b4640 1 Processor -- start 2023-11-09T03:56:12.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.859+0000 7f9cac6b4640 1 -- start start 2023-11-09T03:56:12.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.860+0000 7f9cac6b4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ca406bb60 0x7f9ca414fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:12.865 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.860+0000 7f9cac6b4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ca40b0c40 0x7f9ca4147e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:12.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.860+0000 7f9cac6b4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca4159e40 0x7f9ca40c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:12.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.860+0000 7f9cac6b4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca4150580 con 0x7f9ca406bb60 2023-11-09T03:56:12.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.860+0000 7f9cac6b4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca4150860 con 0x7f9ca40b0c40 2023-11-09T03:56:12.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.860+0000 7f9cac6b4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca4150b40 con 0x7f9ca4159e40 2023-11-09T03:56:12.866 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.860+0000 7f9caa450640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ca406bb60 0x7f9ca414fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:12.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.860+0000 7f9caa450640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ca406bb60 0x7f9ca414fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40244/0 (socket says 172.21.15.76:40244) 2023-11-09T03:56:12.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.860+0000 7f9caa450640 1 -- 172.21.15.76:0/2217659918 learned_addr learned my addr 172.21.15.76:0/2217659918 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:56:12.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.860+0000 7f9caa450640 1 -- 172.21.15.76:0/2217659918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca4159e40 msgr2=0x7f9ca40c61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:12.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.860+0000 7f9caa450640 1 --2- 172.21.15.76:0/2217659918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca4159e40 0x7f9ca40c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:12.867 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.860+0000 7f9caa450640 1 -- 172.21.15.76:0/2217659918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ca40b0c40 msgr2=0x7f9ca4147e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:12.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.860+0000 7f9caa450640 1 --2- 172.21.15.76:0/2217659918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ca40b0c40 0x7f9ca4147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:12.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.860+0000 7f9caa450640 1 -- 172.21.15.76:0/2217659918 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c940094d0 con 0x7f9ca406bb60 2023-11-09T03:56:12.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.860+0000 7f9caa450640 1 --2- 172.21.15.76:0/2217659918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ca406bb60 0x7f9ca414fd30 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f9ca000b3f0 tx=0x7f9ca000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:12.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.860+0000 7f9c9affd640 1 -- 172.21.15.76:0/2217659918 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0011830 con 0x7f9ca406bb60 2023-11-09T03:56:12.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.861+0000 7f9c9affd640 1 -- 172.21.15.76:0/2217659918 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca0011990 con 0x7f9ca406bb60 2023-11-09T03:56:12.868 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.861+0000 7f9cac6b4640 1 -- 172.21.15.76:0/2217659918 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca4150e90 con 0x7f9ca406bb60 2023-11-09T03:56:12.869 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.861+0000 7f9c9affd640 1 -- 172.21.15.76:0/2217659918 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0011d40 con 0x7f9ca406bb60 2023-11-09T03:56:12.869 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.861+0000 7f9cac6b4640 1 -- 172.21.15.76:0/2217659918 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca4150270 con 0x7f9ca406bb60 2023-11-09T03:56:12.869 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.861+0000 7f9cac6b4640 1 -- 172.21.15.76:0/2217659918 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca41513b0 con 0x7f9ca406bb60 2023-11-09T03:56:12.869 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.862+0000 7f9c9affd640 1 -- 172.21.15.76:0/2217659918 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9ca001a400 con 0x7f9ca406bb60 2023-11-09T03:56:12.869 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.862+0000 7f9c9affd640 1 --2- 172.21.15.76:0/2217659918 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c8806aea0 0x7f9c8806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:12.870 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.862+0000 7f9c9affd640 1 -- 172.21.15.76:0/2217659918 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8582+0+0 (secure 0 0 0) 0x7f9ca008a340 con 0x7f9ca406bb60 2023-11-09T03:56:12.870 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.862+0000 7f9ca9c4f640 1 --2- 172.21.15.76:0/2217659918 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c8806aea0 0x7f9c8806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:12.870 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.863+0000 7f9ca9c4f640 1 --2- 172.21.15.76:0/2217659918 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c8806aea0 0x7f9c8806d360 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9c94036000 tx=0x7f9c94030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:12.871 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:12.866+0000 7f9c9affd640 1 -- 172.21.15.76:0/2217659918 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9ca41513b0 con 0x7f9ca406bb60 2023-11-09T03:56:13.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.033+0000 7f9cac6b4640 1 -- 172.21.15.76:0/2217659918 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9ca41513b0 con 0x7f9ca406bb60 2023-11-09T03:56:13.033 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.033+0000 7f9c9affd640 1 -- 172.21.15.76:0/2217659918 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9ca41513b0 con 0x7f9ca406bb60 2023-11-09T03:56:13.034 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2023-11-09T03:56:13.035 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.035+0000 7f9cac6b4640 1 -- 172.21.15.76:0/2217659918 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c8806aea0 msgr2=0x7f9c8806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:13.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.035+0000 7f9cac6b4640 1 --2- 172.21.15.76:0/2217659918 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c8806aea0 0x7f9c8806d360 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9c94036000 tx=0x7f9c94030000 comp rx=0 tx=0).stop 2023-11-09T03:56:13.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.035+0000 7f9cac6b4640 1 -- 172.21.15.76:0/2217659918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ca406bb60 msgr2=0x7f9ca414fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:13.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.035+0000 7f9cac6b4640 1 --2- 172.21.15.76:0/2217659918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ca406bb60 0x7f9ca414fd30 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f9ca000b3f0 tx=0x7f9ca000b8b0 comp rx=0 tx=0).stop 2023-11-09T03:56:13.036 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.035+0000 7f9cac6b4640 1 -- 172.21.15.76:0/2217659918 shutdown_connections 2023-11-09T03:56:13.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.035+0000 7f9cac6b4640 1 --2- 172.21.15.76:0/2217659918 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f9c8806aea0 0x7f9c8806d360 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:13.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.035+0000 7f9cac6b4640 1 --2- 172.21.15.76:0/2217659918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ca4159e40 0x7f9ca40c61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:13.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.035+0000 7f9cac6b4640 1 --2- 172.21.15.76:0/2217659918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ca40b0c40 0x7f9ca4147e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:13.037 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.035+0000 7f9cac6b4640 1 --2- 172.21.15.76:0/2217659918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ca406bb60 0x7f9ca414fd30 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:13.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.035+0000 7f9cac6b4640 1 -- 172.21.15.76:0/2217659918 >> 172.21.15.76:0/2217659918 conn(0x7f9ca40cb530 msgr2=0x7f9ca40cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:13.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.035+0000 7f9cac6b4640 1 -- 172.21.15.76:0/2217659918 shutdown_connections 2023-11-09T03:56:13.038 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.035+0000 7f9cac6b4640 1 -- 172.21.15.76:0/2217659918 wait complete. 2023-11-09T03:56:13.046 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-09T03:56:13.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.149+0000 7f124e290640 1 Processor -- start 2023-11-09T03:56:13.149 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.149+0000 7f124e290640 1 -- start start 2023-11-09T03:56:13.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.150+0000 7f124e290640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f124815d4e0 0x7f12480d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:13.152 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.150+0000 7f124e290640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f12480d3b60 0x7f12480d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:13.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.150+0000 7f124e290640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12480b32b0 con 0x7f124815d100 2023-11-09T03:56:13.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.150+0000 7f124e290640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12480cec20 con 0x7f124815d4e0 2023-11-09T03:56:13.153 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.150+0000 7f124e290640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12480cee10 con 0x7f12480d3b60 2023-11-09T03:56:13.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.150+0000 7f12477fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f124815d4e0 0x7f12480d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:13.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.150+0000 7f12477fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f124815d4e0 0x7f12480d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40258/0 (socket says 172.21.15.76:40258) 2023-11-09T03:56:13.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.150+0000 7f12477fe640 1 -- 172.21.15.76:0/1933934423 learned_addr learned my addr 172.21.15.76:0/1933934423 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:56:13.154 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.151+0000 7f124c82d640 1 --2- 172.21.15.76:0/1933934423 >> v2:172.21.15.113:3300/0 conn(0x7f12480d3b60 0x7f12480d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:13.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.151+0000 7f12477fe640 1 -- 172.21.15.76:0/1933934423 >> v2:172.21.15.113:3300/0 conn(0x7f12480d3b60 msgr2=0x7f12480d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:13.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.151+0000 7f12477fe640 1 --2- 172.21.15.76:0/1933934423 >> v2:172.21.15.113:3300/0 conn(0x7f12480d3b60 0x7f12480d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:13.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.151+0000 7f12477fe640 1 -- 172.21.15.76:0/1933934423 >> v1:172.21.15.113:6789/0 conn(0x7f124815d100 legacy=0x7f12480ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:13.155 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.151+0000 7f12477fe640 1 -- 172.21.15.76:0/1933934423 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12480b4750 con 0x7f124815d4e0 2023-11-09T03:56:13.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.151+0000 7f124c82d640 1 --2- 172.21.15.76:0/1933934423 >> v2:172.21.15.113:3300/0 conn(0x7f12480d3b60 0x7f12480d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-09T03:56:13.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.151+0000 7f12477fe640 1 --2- 172.21.15.76:0/1933934423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f124815d4e0 0x7f12480d14d0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f124808d6a0 tx=0x7f123c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7f24729669cf57 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:13.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.152+0000 7f1246ffd640 1 -- 172.21.15.76:0/1933934423 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c030860 con 0x7f124815d4e0 2023-11-09T03:56:13.156 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.152+0000 7f1246ffd640 1 -- 172.21.15.76:0/1933934423 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123c0309c0 con 0x7f124815d4e0 2023-11-09T03:56:13.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.152+0000 7f1246ffd640 1 -- 172.21.15.76:0/1933934423 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c030d50 con 0x7f124815d4e0 2023-11-09T03:56:13.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.152+0000 7f124e290640 1 -- 172.21.15.76:0/1933934423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f124815d4e0 msgr2=0x7f12480d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:13.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.152+0000 7f124e290640 1 --2- 172.21.15.76:0/1933934423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f124815d4e0 0x7f12480d14d0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f124808d6a0 tx=0x7f123c02b9a0 comp rx=0 tx=0).stop 2023-11-09T03:56:13.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.152+0000 7f124e290640 1 -- 172.21.15.76:0/1933934423 shutdown_connections 2023-11-09T03:56:13.157 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.152+0000 7f124e290640 1 --2- 172.21.15.76:0/1933934423 >> v2:172.21.15.113:3300/0 conn(0x7f12480d3b60 0x7f12480d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:13.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.152+0000 7f124e290640 1 --2- 172.21.15.76:0/1933934423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f124815d4e0 0x7f12480d14d0 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:13.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.152+0000 7f124e290640 1 -- 172.21.15.76:0/1933934423 >> 172.21.15.76:0/1933934423 conn(0x7f12480d2ef0 msgr2=0x7f12480d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:13.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.152+0000 7f124e290640 1 -- 172.21.15.76:0/1933934423 shutdown_connections 2023-11-09T03:56:13.158 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.152+0000 7f124e290640 1 -- 172.21.15.76:0/1933934423 wait complete. 2023-11-09T03:56:13.161 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.153+0000 7f124e290640 1 Processor -- start 2023-11-09T03:56:13.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.154+0000 7f124e290640 1 -- start start 2023-11-09T03:56:13.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.154+0000 7f124e290640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12480d3b60 0x7f124815aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:13.162 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.154+0000 7f124e290640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f124815d100 0x7f12481f2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:13.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.154+0000 7f124e290640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f124815d4e0 0x7f12480b0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:13.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.154+0000 7f124e290640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12480cb8a0 con 0x7f124815d100 2023-11-09T03:56:13.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.154+0000 7f124e290640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12481f32f0 con 0x7f12480d3b60 2023-11-09T03:56:13.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.154+0000 7f124e290640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12480b5e90 con 0x7f124815d4e0 2023-11-09T03:56:13.163 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.154+0000 7f12477fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f124815d100 0x7f12481f2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:13.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.154+0000 7f12477fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f124815d100 0x7f12481f2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40260/0 (socket says 172.21.15.76:40260) 2023-11-09T03:56:13.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.154+0000 7f12477fe640 1 -- 172.21.15.76:0/2756875163 learned_addr learned my addr 172.21.15.76:0/2756875163 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:56:13.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.155+0000 7f124c82d640 1 --2- 172.21.15.76:0/2756875163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f124815d4e0 0x7f12480b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:13.164 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.155+0000 7f1247fff640 1 --2- 172.21.15.76:0/2756875163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12480d3b60 0x7f124815aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:13.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.155+0000 7f12477fe640 1 -- 172.21.15.76:0/2756875163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f124815d4e0 msgr2=0x7f12480b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:13.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.155+0000 7f12477fe640 1 --2- 172.21.15.76:0/2756875163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f124815d4e0 0x7f12480b0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:13.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.155+0000 7f12477fe640 1 -- 172.21.15.76:0/2756875163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12480d3b60 msgr2=0x7f124815aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:13.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.155+0000 7f12477fe640 1 --2- 172.21.15.76:0/2756875163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12480d3b60 0x7f124815aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:13.165 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.155+0000 7f12477fe640 1 -- 172.21.15.76:0/2756875163 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f123c0094d0 con 0x7f124815d100 2023-11-09T03:56:13.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.155+0000 7f124c82d640 1 --2- 172.21.15.76:0/2756875163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f124815d4e0 0x7f12480b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:56:13.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.155+0000 7f1247fff640 1 --2- 172.21.15.76:0/2756875163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12480d3b60 0x7f124815aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:56:13.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.155+0000 7f12477fe640 1 --2- 172.21.15.76:0/2756875163 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f124815d100 0x7f12481f2db0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f123c02b9e0 tx=0x7f123c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:13.166 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.155+0000 7f1244ff9640 1 -- 172.21.15.76:0/2756875163 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c03e070 con 0x7f124815d100 2023-11-09T03:56:13.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.155+0000 7f1244ff9640 1 -- 172.21.15.76:0/2756875163 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123c039a60 con 0x7f124815d100 2023-11-09T03:56:13.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.155+0000 7f1244ff9640 1 -- 172.21.15.76:0/2756875163 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c039df0 con 0x7f124815d100 2023-11-09T03:56:13.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.155+0000 7f124e290640 1 -- 172.21.15.76:0/2756875163 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12480cba80 con 0x7f124815d100 2023-11-09T03:56:13.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.155+0000 7f124e290640 1 -- 172.21.15.76:0/2756875163 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12480cc040 con 0x7f124815d100 2023-11-09T03:56:13.167 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.156+0000 7f124e290640 1 -- 172.21.15.76:0/2756875163 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12480cc260 con 0x7f124815d100 2023-11-09T03:56:13.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.156+0000 7f1244ff9640 1 -- 172.21.15.76:0/2756875163 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f123c04b400 con 0x7f124815d100 2023-11-09T03:56:13.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.157+0000 7f1244ff9640 1 --2- 172.21.15.76:0/2756875163 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f123406ae70 0x7f123406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:13.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.157+0000 7f1244ff9640 1 -- 172.21.15.76:0/2756875163 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8582+0+0 (secure 0 0 0) 0x7f123c0ac870 con 0x7f124815d100 2023-11-09T03:56:13.168 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.157+0000 7f1247fff640 1 --2- 172.21.15.76:0/2756875163 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f123406ae70 0x7f123406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:13.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.158+0000 7f1247fff640 1 --2- 172.21.15.76:0/2756875163 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f123406ae70 0x7f123406d330 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1238000fb0 tx=0x7f1238009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:13.169 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.161+0000 7f1244ff9640 1 -- 172.21.15.76:0/2756875163 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f123c076d90 con 0x7f124815d100 2023-11-09T03:56:13.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.326+0000 7f124e290640 1 -- 172.21.15.76:0/2756875163 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f12480684b0 con 0x7f124815d100 2023-11-09T03:56:13.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.327+0000 7f1244ff9640 1 -- 172.21.15.76:0/2756875163 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) v1 ==== 100+0+11 (secure 0 0 0) 0x7f123c034070 con 0x7f124815d100 2023-11-09T03:56:13.327 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2023-11-09T03:56:13.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.329+0000 7f124e290640 1 -- 172.21.15.76:0/2756875163 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f123406ae70 msgr2=0x7f123406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:13.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.329+0000 7f124e290640 1 --2- 172.21.15.76:0/2756875163 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f123406ae70 0x7f123406d330 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1238000fb0 tx=0x7f1238009040 comp rx=0 tx=0).stop 2023-11-09T03:56:13.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.329+0000 7f124e290640 1 -- 172.21.15.76:0/2756875163 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f124815d100 msgr2=0x7f12481f2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:13.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.329+0000 7f124e290640 1 --2- 172.21.15.76:0/2756875163 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f124815d100 0x7f12481f2db0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f123c02b9e0 tx=0x7f123c0046c0 comp rx=0 tx=0).stop 2023-11-09T03:56:13.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.329+0000 7f124e290640 1 -- 172.21.15.76:0/2756875163 shutdown_connections 2023-11-09T03:56:13.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.329+0000 7f124e290640 1 --2- 172.21.15.76:0/2756875163 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7f123406ae70 0x7f123406d330 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:13.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.329+0000 7f124e290640 1 --2- 172.21.15.76:0/2756875163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f124815d4e0 0x7f12480b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:13.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.329+0000 7f124e290640 1 --2- 172.21.15.76:0/2756875163 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f124815d100 0x7f12481f2db0 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:13.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.329+0000 7f124e290640 1 --2- 172.21.15.76:0/2756875163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12480d3b60 0x7f124815aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:13.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.329+0000 7f124e290640 1 -- 172.21.15.76:0/2756875163 >> 172.21.15.76:0/2756875163 conn(0x7f12480d2ef0 msgr2=0x7f12480d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:13.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.329+0000 7f124e290640 1 -- 172.21.15.76:0/2756875163 shutdown_connections 2023-11-09T03:56:13.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.329+0000 7f124e290640 1 -- 172.21.15.76:0/2756875163 wait complete. 2023-11-09T03:56:13.339 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-09T03:56:13.440 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.441+0000 7fea2d505640 1 Processor -- start 2023-11-09T03:56:13.441 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.441+0000 7fea2d505640 1 -- start start 2023-11-09T03:56:13.444 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.442+0000 7fea2d505640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fea2815d1b0 0x7fea280d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:13.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.442+0000 7fea2d505640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea2815d590 con 0x7fea280d14d0 2023-11-09T03:56:13.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.442+0000 7fea2d505640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea280d1a50 con 0x7fea280b5050 2023-11-09T03:56:13.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.442+0000 7fea2d505640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea280ce6c0 con 0x7fea2815d1b0 2023-11-09T03:56:13.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.442+0000 7fea267fc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fea280d14d0 0x7fea280b4940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50572/0 (socket says 172.21.15.76:50572) 2023-11-09T03:56:13.445 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.442+0000 7fea267fc640 1 -- 172.21.15.76:0/2973177246 learned_addr learned my addr 172.21.15.76:0/2973177246 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-09T03:56:13.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.442+0000 7fea26ffd640 1 --2- 172.21.15.76:0/2973177246 >> v2:172.21.15.113:3300/0 conn(0x7fea2815d1b0 0x7fea280d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:13.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.442+0000 7fea25ffb640 1 -- 172.21.15.76:0/2973177246 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1863747727 0 0) 0x7fea1c001110 con 0x7fea280d14d0 2023-11-09T03:56:13.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.442+0000 7fea25ffb640 1 -- 172.21.15.76:0/2973177246 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea10003520 con 0x7fea280d14d0 2023-11-09T03:56:13.446 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.442+0000 7fea25ffb640 1 -- 172.21.15.76:0/2973177246 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1744793832 0 0) 0x7fea10003520 con 0x7fea280d14d0 2023-11-09T03:56:13.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.442+0000 7fea25ffb640 1 -- 172.21.15.76:0/2973177246 >> v2:172.21.15.113:3300/0 conn(0x7fea2815d1b0 msgr2=0x7fea280d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:13.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.442+0000 7fea25ffb640 1 --2- 172.21.15.76:0/2973177246 >> v2:172.21.15.113:3300/0 conn(0x7fea2815d1b0 0x7fea280d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:13.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.442+0000 7fea25ffb640 1 -- 172.21.15.76:0/2973177246 >> v1:172.21.15.113:6789/0 conn(0x7fea280b5050 legacy=0x7fea280b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:13.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.442+0000 7fea25ffb640 1 -- 172.21.15.76:0/2973177246 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea280d1d30 con 0x7fea280d14d0 2023-11-09T03:56:13.447 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.442+0000 7fea25ffb640 1 -- 172.21.15.76:0/2973177246 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1422027216 0 0) 0x7fea1c001800 con 0x7fea280d14d0 2023-11-09T03:56:13.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.442+0000 7fea25ffb640 1 -- 172.21.15.76:0/2973177246 >> v1:172.21.15.76:6789/0 conn(0x7fea280d14d0 legacy=0x7fea280b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:13.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.442+0000 7fea25ffb640 1 --2- 172.21.15.76:0/2973177246 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea10003970 0x7fea10003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:13.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.442+0000 7fea25ffb640 1 --2- 172.21.15.76:0/2973177246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea10004370 0x7fea10004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:13.448 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.442+0000 7fea25ffb640 1 --2- 172.21.15.76:0/2973177246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea10005040 0x7fea1000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:13.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.442+0000 7fea25ffb640 1 -- 172.21.15.76:0/2973177246 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea280ce6c0 con 0x7fea10004370 2023-11-09T03:56:13.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.442+0000 7fea25ffb640 1 -- 172.21.15.76:0/2973177246 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1000bfa0 con 0x7fea10005040 2023-11-09T03:56:13.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.442+0000 7fea25ffb640 1 -- 172.21.15.76:0/2973177246 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1000c160 con 0x7fea10003970 2023-11-09T03:56:13.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.442+0000 7fea267fc640 1 --2- 172.21.15.76:0/2973177246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea10004370 0x7fea10004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:13.449 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.443+0000 7fea26ffd640 1 --2- 172.21.15.76:0/2973177246 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea10003970 0x7fea10003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:13.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.443+0000 7fea277fe640 1 --2- 172.21.15.76:0/2973177246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea10005040 0x7fea1000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:13.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.443+0000 7fea267fc640 1 -- 172.21.15.76:0/2973177246 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea10003970 msgr2=0x7fea10003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:13.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.443+0000 7fea267fc640 1 --2- 172.21.15.76:0/2973177246 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea10003970 0x7fea10003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:13.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.443+0000 7fea267fc640 1 -- 172.21.15.76:0/2973177246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea10005040 msgr2=0x7fea1000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-09T03:56:13.450 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.443+0000 7fea267fc640 1 --2- 172.21.15.76:0/2973177246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea10005040 0x7fea1000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:13.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.443+0000 7fea277fe640 1 -- 172.21.15.76:0/2973177246 reap_dead start 2023-11-09T03:56:13.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.443+0000 7fea267fc640 1 -- 172.21.15.76:0/2973177246 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea10003520 con 0x7fea10004370 2023-11-09T03:56:13.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.443+0000 7fea267fc640 1 --2- 172.21.15.76:0/2973177246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea10004370 0x7fea10004840 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00c190 tx=0x7fea100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4580414ab87d4cec server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:13.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.444+0000 7fea25ffb640 1 -- 172.21.15.76:0/2973177246 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c005ad0 con 0x7fea10004370 2023-11-09T03:56:13.451 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.444+0000 7fea25ffb640 1 -- 172.21.15.76:0/2973177246 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1c005c50 con 0x7fea10004370 2023-11-09T03:56:13.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.444+0000 7fea25ffb640 1 -- 172.21.15.76:0/2973177246 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c02c830 con 0x7fea10004370 2023-11-09T03:56:13.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.444+0000 7fea2d505640 1 -- 172.21.15.76:0/2973177246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea10004370 msgr2=0x7fea10004840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:13.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.444+0000 7fea2d505640 1 --2- 172.21.15.76:0/2973177246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea10004370 0x7fea10004840 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00c190 tx=0x7fea100057f0 comp rx=0 tx=0).stop 2023-11-09T03:56:13.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.444+0000 7fea2d505640 1 -- 172.21.15.76:0/2973177246 shutdown_connections 2023-11-09T03:56:13.452 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.444+0000 7fea2d505640 1 --2- 172.21.15.76:0/2973177246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea10004370 0x7fea10004840 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:13.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.444+0000 7fea2d505640 1 -- 172.21.15.76:0/2973177246 >> 172.21.15.76:0/2973177246 conn(0x7fea280cf940 msgr2=0x7fea280cfd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:13.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.444+0000 7fea2d505640 1 -- 172.21.15.76:0/2973177246 shutdown_connections 2023-11-09T03:56:13.453 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.444+0000 7fea2d505640 1 -- 172.21.15.76:0/2973177246 wait complete. 2023-11-09T03:56:13.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.445+0000 7fea2d505640 1 Processor -- start 2023-11-09T03:56:13.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.445+0000 7fea2d505640 1 -- start start 2023-11-09T03:56:13.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.446+0000 7fea2d505640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea10004370 0x7fea2815d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:13.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.446+0000 7fea2d505640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea280c8eb0 0x7fea2815f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:13.456 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.446+0000 7fea2d505640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea2806bf50 0x7fea2806c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:13.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.446+0000 7fea2d505640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea280ca600 con 0x7fea2806bf50 2023-11-09T03:56:13.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.446+0000 7fea2d505640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2806c910 con 0x7fea10004370 2023-11-09T03:56:13.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.446+0000 7fea2d505640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea280cab10 con 0x7fea280c8eb0 2023-11-09T03:56:13.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.446+0000 7fea277fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea2806bf50 0x7fea2806c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:13.457 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.446+0000 7fea277fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea2806bf50 0x7fea2806c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40270/0 (socket says 172.21.15.76:40270) 2023-11-09T03:56:13.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.446+0000 7fea277fe640 1 -- 172.21.15.76:0/4046611412 learned_addr learned my addr 172.21.15.76:0/4046611412 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-09T03:56:13.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.446+0000 7fea267fc640 1 --2- 172.21.15.76:0/4046611412 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea280c8eb0 0x7fea2815f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:13.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.446+0000 7fea26ffd640 1 --2- 172.21.15.76:0/4046611412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea10004370 0x7fea2815d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:13.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.447+0000 7fea277fe640 1 -- 172.21.15.76:0/4046611412 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea280c8eb0 msgr2=0x7fea2815f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:13.458 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.447+0000 7fea277fe640 1 --2- 172.21.15.76:0/4046611412 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea280c8eb0 0x7fea2815f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:13.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.447+0000 7fea277fe640 1 -- 172.21.15.76:0/4046611412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea10004370 msgr2=0x7fea2815d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:13.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.447+0000 7fea277fe640 1 --2- 172.21.15.76:0/4046611412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea10004370 0x7fea2815d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:13.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.447+0000 7fea277fe640 1 -- 172.21.15.76:0/4046611412 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2815e030 con 0x7fea2806bf50 2023-11-09T03:56:13.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.447+0000 7fea26ffd640 1 --2- 172.21.15.76:0/4046611412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea10004370 0x7fea2815d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-09T03:56:13.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.447+0000 7fea277fe640 1 --2- 172.21.15.76:0/4046611412 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea2806bf50 0x7fea2806c3d0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fea280cf310 tx=0x7fea2000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:13.459 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.447+0000 7fea0f7fe640 1 -- 172.21.15.76:0/4046611412 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea20003970 con 0x7fea2806bf50 2023-11-09T03:56:13.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.447+0000 7fea0f7fe640 1 -- 172.21.15.76:0/4046611412 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea20003ad0 con 0x7fea2806bf50 2023-11-09T03:56:13.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.447+0000 7fea2d505640 1 -- 172.21.15.76:0/4046611412 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea2815e310 con 0x7fea2806bf50 2023-11-09T03:56:13.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.447+0000 7fea0f7fe640 1 -- 172.21.15.76:0/4046611412 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea20003e80 con 0x7fea2806bf50 2023-11-09T03:56:13.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.447+0000 7fea2d505640 1 -- 172.21.15.76:0/4046611412 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea280cad90 con 0x7fea2806bf50 2023-11-09T03:56:13.460 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.448+0000 7fea2d505640 1 -- 172.21.15.76:0/4046611412 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea2815e830 con 0x7fea2806bf50 2023-11-09T03:56:13.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.448+0000 7fea0f7fe640 1 -- 172.21.15.76:0/4046611412 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fea200050d0 con 0x7fea2806bf50 2023-11-09T03:56:13.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.449+0000 7fea0f7fe640 1 --2- 172.21.15.76:0/4046611412 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fea1406ae70 0x7fea1406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T03:56:13.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.449+0000 7fea0f7fe640 1 -- 172.21.15.76:0/4046611412 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8582+0+0 (secure 0 0 0) 0x7fea20089a80 con 0x7fea2806bf50 2023-11-09T03:56:13.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.449+0000 7fea26ffd640 1 --2- 172.21.15.76:0/4046611412 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fea1406ae70 0x7fea1406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-09T03:56:13.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.449+0000 7fea26ffd640 1 --2- 172.21.15.76:0/4046611412 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fea1406ae70 0x7fea1406d330 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fea2804fd90 tx=0x7fea18008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-09T03:56:13.462 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.453+0000 7fea0f7fe640 1 -- 172.21.15.76:0/4046611412 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fea2815e830 con 0x7fea2806bf50 2023-11-09T03:56:13.633 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.634+0000 7fea2d505640 1 -- 172.21.15.76:0/4046611412 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fea2815e830 con 0x7fea2806bf50 2023-11-09T03:56:13.634 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.634+0000 7fea0f7fe640 1 -- 172.21.15.76:0/4046611412 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 32 entries v123) v1 ==== 81+0+1923 (secure 0 0 0) 0x7fea2815e830 con 0x7fea2806bf50 2023-11-09T03:56:13.634 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/497547142 2023-11-09T04:55:40.451393+0000 2023-11-09T03:56:13.635 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/757529522 2023-11-10T03:55:31.678438+0000 2023-11-09T03:56:13.635 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6832/3697850101 2023-11-10T03:51:39.442929+0000 2023-11-09T03:56:13.635 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/3449910842 2023-11-10T03:50:59.611256+0000 2023-11-09T03:56:13.635 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6832/1536527256 2023-11-10T03:52:34.213804+0000 2023-11-09T03:56:13.635 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/3229239133 2023-11-10T03:51:32.624856+0000 2023-11-09T03:56:13.636 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/445306598 2023-11-10T03:52:27.406838+0000 2023-11-09T03:56:13.636 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/1663988060 2023-11-10T03:50:33.665092+0000 2023-11-09T03:56:13.636 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/1663988060 2023-11-10T03:50:33.665092+0000 2023-11-09T03:56:13.636 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/3280098776 2023-11-10T03:50:07.616780+0000 2023-11-09T03:56:13.637 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/2519773110 2023-11-09T04:54:14.823765+0000 2023-11-09T03:56:13.637 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/4011275120 2023-11-10T03:49:42.029057+0000 2023-11-09T03:56:13.637 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/757529522 2023-11-10T03:55:31.678438+0000 2023-11-09T03:56:13.637 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/4011275120 2023-11-10T03:49:42.029057+0000 2023-11-09T03:56:13.637 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/3462040782 2023-11-10T03:54:45.345437+0000 2023-11-09T03:56:13.638 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/503424885 2023-11-09T04:51:47.727422+0000 2023-11-09T03:56:13.638 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/3229239133 2023-11-10T03:51:32.624856+0000 2023-11-09T03:56:13.638 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/445306598 2023-11-10T03:52:27.406838+0000 2023-11-09T03:56:13.638 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/3280098776 2023-11-10T03:50:07.616780+0000 2023-11-09T03:56:13.638 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/3087804343 2023-11-09T04:52:42.513556+0000 2023-11-09T03:56:13.639 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/546338722 2023-11-10T03:53:17.470106+0000 2023-11-09T03:56:13.639 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6833/1536527256 2023-11-10T03:52:34.213804+0000 2023-11-09T03:56:13.639 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/546338722 2023-11-10T03:53:17.470106+0000 2023-11-09T03:56:13.639 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/7360844 2023-11-10T03:53:24.718008+0000 2023-11-09T03:56:13.639 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6837/7360844 2023-11-10T03:53:24.718008+0000 2023-11-09T03:56:13.640 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6835/2075292158 2023-11-10T03:54:06.568191+0000 2023-11-09T03:56:13.640 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/334503200 2023-11-09T04:53:32.572840+0000 2023-11-09T03:56:13.640 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6834/2075292158 2023-11-10T03:54:06.568191+0000 2023-11-09T03:56:13.640 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6833/3697850101 2023-11-10T03:51:39.442929+0000 2023-11-09T03:56:13.640 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/3462040782 2023-11-10T03:54:45.345437+0000 2023-11-09T03:56:13.640 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.113:6836/3449910842 2023-11-10T03:50:59.611256+0000 2023-11-09T03:56:13.641 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.1:0/3870807407 2023-11-09T04:54:56.671385+0000 2023-11-09T03:56:13.642 INFO:teuthology.orchestra.run.smithi076.stderr:listed 32 entries 2023-11-09T03:56:13.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.636+0000 7fea2d505640 1 -- 172.21.15.76:0/4046611412 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fea1406ae70 msgr2=0x7fea1406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:13.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.636+0000 7fea2d505640 1 --2- 172.21.15.76:0/4046611412 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fea1406ae70 0x7fea1406d330 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fea2804fd90 tx=0x7fea18008040 comp rx=0 tx=0).stop 2023-11-09T03:56:13.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.636+0000 7fea2d505640 1 -- 172.21.15.76:0/4046611412 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea2806bf50 msgr2=0x7fea2806c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-09T03:56:13.642 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.636+0000 7fea2d505640 1 --2- 172.21.15.76:0/4046611412 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea2806bf50 0x7fea2806c3d0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fea280cf310 tx=0x7fea2000eb50 comp rx=0 tx=0).stop 2023-11-09T03:56:13.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.636+0000 7fea2d505640 1 -- 172.21.15.76:0/4046611412 shutdown_connections 2023-11-09T03:56:13.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.636+0000 7fea2d505640 1 --2- 172.21.15.76:0/4046611412 >> [v2:172.21.15.113:6832/1264193042,v1:172.21.15.113:6833/1264193042] conn(0x7fea1406ae70 0x7fea1406d330 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:13.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.636+0000 7fea2d505640 1 --2- 172.21.15.76:0/4046611412 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea2806bf50 0x7fea2806c3d0 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:13.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.636+0000 7fea2d505640 1 --2- 172.21.15.76:0/4046611412 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea280c8eb0 0x7fea2815f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:13.643 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.636+0000 7fea2d505640 1 --2- 172.21.15.76:0/4046611412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea10004370 0x7fea2815d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T03:56:13.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.636+0000 7fea2d505640 1 -- 172.21.15.76:0/4046611412 >> 172.21.15.76:0/4046611412 conn(0x7fea280cf940 msgr2=0x7fea280b4930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-09T03:56:13.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.636+0000 7fea2d505640 1 -- 172.21.15.76:0/4046611412 shutdown_connections 2023-11-09T03:56:13.644 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13.636+0000 7fea2d505640 1 -- 172.21.15.76:0/4046611412 wait complete. 2023-11-09T03:56:13.647 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi076.front.sepia.ceph.com... 2023-11-09T03:56:13.647 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:56:13.647 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-09T03:56:13.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-09T03:56:13.706 INFO:teuthology.orchestra.run.smithi076.stderr:fusermount: failed to unmount /home/ubuntu/cephtest/mnt.0: Device or resource busy 2023-11-09T03:56:13.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:13 stdin-killer INFO: command exited with status 1: exiting normally with same code! 2023-11-09T03:56:13.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T03:56:13.713 INFO:tasks.cephfs.fuse_mount:Failed to unmount ceph-fuse on ubuntu@smithi076.front.sepia.ceph.com, aborting... 2023-11-09T03:56:13.713 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-09T03:56:13.714 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer --timeout=300 -- env PATH=/usr/sbin:$PATH lsof ; ps auxf 2023-11-09T03:56:14.583 INFO:teuthology.orchestra.run.smithi076.stdout:COMMAND PID TID TASKCMD USER FD TYPE DEVICE SIZE/OFF NODE NAME 2023-11-09T03:56:14.583 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.583 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.583 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root txt REG 8,1 102120 20353 /usr/lib/systemd/systemd 2023-11-09T03:56:14.584 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 596532 1027 /etc/selinux/targeted/contexts/files/file_contexts.bin 2023-11-09T03:56:14.584 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.584 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.584 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:14.584 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:14.585 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-11-09T03:56:14.585 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.585 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 21219 1168 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2023-11-09T03:56:14.585 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:14.585 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:14.586 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.586 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:14.586 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:14.586 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:14.586 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 3645472 49253 /usr/lib64/systemd/libsystemd-shared-252.so 2023-11-09T03:56:14.587 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 28552 9804 /usr/lib64/libattr.so.1.1.2501 2023-11-09T03:56:14.587 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:14.587 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-11-09T03:56:14.587 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:14.587 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:14.587 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 2174088 49251 /usr/lib64/systemd/libsystemd-core-252.so 2023-11-09T03:56:14.588 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:14.588 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root DEL REG 8,1 9806 /usr/lib64/libacl.so.1.1.2301 2023-11-09T03:56:14.588 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-11-09T03:56:14.588 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.588 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 107424 10180 /usr/lib64/libkmod.so.2.3.6 2023-11-09T03:56:14.589 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:14.589 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-11-09T03:56:14.589 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 127680 9972 /usr/lib64/libseccomp.so.2.5.2 2023-11-09T03:56:14.589 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:14.589 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.590 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root mem REG 0,19 0 19 /sys/fs/selinux/status 2023-11-09T03:56:14.590 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 0u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.590 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 1u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.590 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 2u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.590 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 3w CHR 1,11 0t0 10 /dev/kmsg 2023-11-09T03:56:14.591 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 4u a_inode 0,14 0 2077 [eventpoll:5,6,8,9,11,14,15,16,17,18,19,44,45,46,47,52,60,62,63,64,65,89,93,94,105,107,111,112,113,114,115,116...] 2023-11-09T03:56:14.591 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 5u a_inode 0,14 0 2077 [signalfd] 2023-11-09T03:56:14.591 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 6r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.591 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 7r DIR 0,25 0 1 /sys/fs/cgroup 2023-11-09T03:56:14.591 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 8u a_inode 0,14 0 2077 [timerfd] 2023-11-09T03:56:14.592 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 9u a_inode 0,14 0 2077 [eventpoll:10,13] 2023-11-09T03:56:14.592 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 10r REG 0,20 0 15639 /proc/1/mountinfo 2023-11-09T03:56:14.592 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 11r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.592 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 12u netlink 0t0 14757 AUDIT 2023-11-09T03:56:14.592 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 13r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.593 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 14r REG 0,20 0 4026532076 /proc/swaps 2023-11-09T03:56:14.593 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 15u netlink 0t0 14749 KOBJECT_UEVENT 2023-11-09T03:56:14.593 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 16u unix 0xffff959a47f09100 0t0 14751 /run/systemd/private type=STREAM (LISTEN) 2023-11-09T03:56:14.593 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 17r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.593 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 18u a_inode 0,14 0 2077 [timerfd] 2023-11-09T03:56:14.594 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 19r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.594 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 20u a_inode 0,14 0 2077 bpf-prog 2023-11-09T03:56:14.594 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 21u a_inode 0,14 0 2077 bpf-prog 2023-11-09T03:56:14.594 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 23u a_inode 0,14 0 2077 bpf-prog 2023-11-09T03:56:14.594 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 24u a_inode 0,14 0 2077 bpf-prog 2023-11-09T03:56:14.594 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 25u unix 0xffff959a61972ec0 0t0 19671 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.595 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 26u a_inode 0,14 0 2077 bpf-prog 2023-11-09T03:56:14.595 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 27r CHR 10,235 0t0 111 /dev/autofs 2023-11-09T03:56:14.595 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 28u a_inode 0,14 0 2077 bpf-prog 2023-11-09T03:56:14.595 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 29u a_inode 0,14 0 2077 bpf-prog 2023-11-09T03:56:14.595 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 30u a_inode 0,14 0 2077 bpf-prog 2023-11-09T03:56:14.596 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 31u a_inode 0,14 0 2077 bpf-prog 2023-11-09T03:56:14.596 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 32u a_inode 0,14 0 2077 bpf-prog 2023-11-09T03:56:14.596 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 44u unix 0xffff959a60328440 0t0 15647 /run/systemd/io.system.ManagedOOM type=STREAM (LISTEN) 2023-11-09T03:56:14.596 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 45u unix 0xffff959a6032b740 0t0 15646 /run/systemd/userdb/io.systemd.DynamicUser type=STREAM (LISTEN) 2023-11-09T03:56:14.596 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 46u unix 0xffff959a60325500 0t0 15641 /run/systemd/notify type=DGRAM (CONNECTED) 2023-11-09T03:56:14.596 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 47u unix 0xffff959a60322640 0t0 15642 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.597 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 48u unix 0xffff959a603250c0 0t0 15643 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.597 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 52u a_inode 0,14 0 2077 [timerfd] 2023-11-09T03:56:14.597 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 60u unix 0xffff959a60003fc0 0t0 22121 type=STREAM (CONNECTED) 2023-11-09T03:56:14.597 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 62u unix 0xffff959a60001540 0t0 22115 /run/.heim_org.h5l.kcm-socket type=STREAM (LISTEN) 2023-11-09T03:56:14.597 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 63u FIFO 0,24 0t0 447 /run/initctl 2023-11-09T03:56:14.598 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 64u FIFO 0,24 0t0 441 /run/dmeventd-server 2023-11-09T03:56:14.598 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 65u FIFO 0,24 0t0 442 /run/dmeventd-client 2023-11-09T03:56:14.598 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 67u unix 0xffff959a43261540 0t0 20507 /run/rpcbind.sock type=STREAM (LISTEN) 2023-11-09T03:56:14.598 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 71u IPv4 15769 0t0 TCP *:sunrpc (LISTEN) 2023-11-09T03:56:14.598 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 85u IPv4 19593 0t0 UDP *:sunrpc 2023-11-09T03:56:14.598 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 86u IPv6 15775 0t0 TCP *:sunrpc (LISTEN) 2023-11-09T03:56:14.599 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 87u IPv6 19599 0t0 UDP *:sunrpc 2023-11-09T03:56:14.599 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 88u netlink 0t0 20521 KOBJECT_UEVENT 2023-11-09T03:56:14.599 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 89u CHR 10,242 0t0 165 /dev/rfkill 2023-11-09T03:56:14.599 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 90u unix 0xffff959a60323300 0t0 15654 /run/systemd/journal/dev-log type=DGRAM (CONNECTED) 2023-11-09T03:56:14.599 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 91r FIFO 0,13 0t0 14761 pipe 2023-11-09T03:56:14.599 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 92u unix 0xffff959a60007700 0t0 22113 /run/dbus/system_bus_socket type=STREAM (LISTEN) 2023-11-09T03:56:14.600 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 93u unix 0xffff959a4325c840 0t0 20517 /run/systemd/coredump type=SEQPACKET (LISTEN) 2023-11-09T03:56:14.600 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 94u unix 0xffff959a43267700 0t0 20505 /run/lvm/lvmpolld.socket type=STREAM (LISTEN) 2023-11-09T03:56:14.600 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 95u unix 0xffff959a4325d940 0t0 20519 /run/udev/control type=SEQPACKET (LISTEN) 2023-11-09T03:56:14.600 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 97u unix 0xffff959a60324400 0t0 15656 /run/systemd/journal/socket type=DGRAM (CONNECTED) 2023-11-09T03:56:14.600 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 98u unix 0xffff959a60329540 0t0 15658 /run/systemd/journal/stdout type=STREAM (LISTEN) 2023-11-09T03:56:14.601 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 104u unix 0xffff959a6032aec0 0t0 163557 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-11-09T03:56:14.601 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 105u unix 0xffff959a6079d940 0t0 24077 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-11-09T03:56:14.601 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 107u unix 0xffff959a60349100 0t0 2570 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-11-09T03:56:14.601 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 108u unix 0xffff959a60001980 0t0 17774 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-11-09T03:56:14.601 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 109u unix 0xffff959a60006a40 0t0 17773 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-11-09T03:56:14.601 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 110u unix 0xffff959a6034ee80 0t0 17772 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-11-09T03:56:14.602 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 111u unix 0xffff959a47f0e1c0 0t0 17676 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-11-09T03:56:14.602 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 112u unix 0xffff959a605eaec0 0t0 21410 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-11-09T03:56:14.602 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 113u unix 0xffff959a605ed940 0t0 21409 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-11-09T03:56:14.602 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 114u unix 0xffff959a47f0cc80 0t0 21408 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-11-09T03:56:14.602 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 115u unix 0xffff959a605ebfc0 0t0 14880 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-11-09T03:56:14.602 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 116u unix 0xffff959a43263b80 0t0 17657 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-11-09T03:56:14.603 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 117u unix 0xffff959a60005500 0t0 19744 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-11-09T03:56:14.603 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 1 root 118u unix 0xffff959a605eb300 0t0 16621 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-11-09T03:56:14.603 INFO:teuthology.orchestra.run.smithi076.stdout:kthreadd 2 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.603 INFO:teuthology.orchestra.run.smithi076.stdout:kthreadd 2 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.603 INFO:teuthology.orchestra.run.smithi076.stdout:kthreadd 2 root txt unknown /proc/2/exe 2023-11-09T03:56:14.604 INFO:teuthology.orchestra.run.smithi076.stdout:rcu_gp 3 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.604 INFO:teuthology.orchestra.run.smithi076.stdout:rcu_gp 3 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.604 INFO:teuthology.orchestra.run.smithi076.stdout:rcu_gp 3 root txt unknown /proc/3/exe 2023-11-09T03:56:14.604 INFO:teuthology.orchestra.run.smithi076.stdout:rcu_par_g 4 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.604 INFO:teuthology.orchestra.run.smithi076.stdout:rcu_par_g 4 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.605 INFO:teuthology.orchestra.run.smithi076.stdout:rcu_par_g 4 root txt unknown /proc/4/exe 2023-11-09T03:56:14.605 INFO:teuthology.orchestra.run.smithi076.stdout:slub_flus 5 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.605 INFO:teuthology.orchestra.run.smithi076.stdout:slub_flus 5 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.605 INFO:teuthology.orchestra.run.smithi076.stdout:slub_flus 5 root txt unknown /proc/5/exe 2023-11-09T03:56:14.605 INFO:teuthology.orchestra.run.smithi076.stdout:netns 6 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.605 INFO:teuthology.orchestra.run.smithi076.stdout:netns 6 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.606 INFO:teuthology.orchestra.run.smithi076.stdout:netns 6 root txt unknown /proc/6/exe 2023-11-09T03:56:14.606 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/0 8 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.606 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/0 8 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.606 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/0 8 root txt unknown /proc/8/exe 2023-11-09T03:56:14.606 INFO:teuthology.orchestra.run.smithi076.stdout:mm_percpu 11 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.607 INFO:teuthology.orchestra.run.smithi076.stdout:mm_percpu 11 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.607 INFO:teuthology.orchestra.run.smithi076.stdout:mm_percpu 11 root txt unknown /proc/11/exe 2023-11-09T03:56:14.607 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/u 12 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.607 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/u 12 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.607 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/u 12 root txt unknown /proc/12/exe 2023-11-09T03:56:14.607 INFO:teuthology.orchestra.run.smithi076.stdout:rcu_tasks 13 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.608 INFO:teuthology.orchestra.run.smithi076.stdout:rcu_tasks 13 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.608 INFO:teuthology.orchestra.run.smithi076.stdout:rcu_tasks 13 root txt unknown /proc/13/exe 2023-11-09T03:56:14.608 INFO:teuthology.orchestra.run.smithi076.stdout:rcu_tasks 14 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.608 INFO:teuthology.orchestra.run.smithi076.stdout:rcu_tasks 14 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.608 INFO:teuthology.orchestra.run.smithi076.stdout:rcu_tasks 14 root txt unknown /proc/14/exe 2023-11-09T03:56:14.608 INFO:teuthology.orchestra.run.smithi076.stdout:rcu_tasks 15 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.609 INFO:teuthology.orchestra.run.smithi076.stdout:rcu_tasks 15 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.609 INFO:teuthology.orchestra.run.smithi076.stdout:rcu_tasks 15 root txt unknown /proc/15/exe 2023-11-09T03:56:14.609 INFO:teuthology.orchestra.run.smithi076.stdout:ksoftirqd 16 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.609 INFO:teuthology.orchestra.run.smithi076.stdout:ksoftirqd 16 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.609 INFO:teuthology.orchestra.run.smithi076.stdout:ksoftirqd 16 root txt unknown /proc/16/exe 2023-11-09T03:56:14.609 INFO:teuthology.orchestra.run.smithi076.stdout:pr/ttyS1 17 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.610 INFO:teuthology.orchestra.run.smithi076.stdout:pr/ttyS1 17 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.610 INFO:teuthology.orchestra.run.smithi076.stdout:pr/ttyS1 17 root txt unknown /proc/17/exe 2023-11-09T03:56:14.610 INFO:teuthology.orchestra.run.smithi076.stdout:rcu_preem 18 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.610 INFO:teuthology.orchestra.run.smithi076.stdout:rcu_preem 18 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.610 INFO:teuthology.orchestra.run.smithi076.stdout:rcu_preem 18 root txt unknown /proc/18/exe 2023-11-09T03:56:14.610 INFO:teuthology.orchestra.run.smithi076.stdout:migration 19 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.611 INFO:teuthology.orchestra.run.smithi076.stdout:migration 19 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.611 INFO:teuthology.orchestra.run.smithi076.stdout:migration 19 root txt unknown /proc/19/exe 2023-11-09T03:56:14.611 INFO:teuthology.orchestra.run.smithi076.stdout:idle_inje 20 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.611 INFO:teuthology.orchestra.run.smithi076.stdout:idle_inje 20 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.611 INFO:teuthology.orchestra.run.smithi076.stdout:idle_inje 20 root txt unknown /proc/20/exe 2023-11-09T03:56:14.611 INFO:teuthology.orchestra.run.smithi076.stdout:cpuhp/0 21 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.612 INFO:teuthology.orchestra.run.smithi076.stdout:cpuhp/0 21 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.612 INFO:teuthology.orchestra.run.smithi076.stdout:cpuhp/0 21 root txt unknown /proc/21/exe 2023-11-09T03:56:14.612 INFO:teuthology.orchestra.run.smithi076.stdout:cpuhp/1 22 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.612 INFO:teuthology.orchestra.run.smithi076.stdout:cpuhp/1 22 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.612 INFO:teuthology.orchestra.run.smithi076.stdout:cpuhp/1 22 root txt unknown /proc/22/exe 2023-11-09T03:56:14.612 INFO:teuthology.orchestra.run.smithi076.stdout:idle_inje 23 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.613 INFO:teuthology.orchestra.run.smithi076.stdout:idle_inje 23 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.613 INFO:teuthology.orchestra.run.smithi076.stdout:idle_inje 23 root txt unknown /proc/23/exe 2023-11-09T03:56:14.613 INFO:teuthology.orchestra.run.smithi076.stdout:migration 24 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.613 INFO:teuthology.orchestra.run.smithi076.stdout:migration 24 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.613 INFO:teuthology.orchestra.run.smithi076.stdout:migration 24 root txt unknown /proc/24/exe 2023-11-09T03:56:14.613 INFO:teuthology.orchestra.run.smithi076.stdout:ksoftirqd 25 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.614 INFO:teuthology.orchestra.run.smithi076.stdout:ksoftirqd 25 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.614 INFO:teuthology.orchestra.run.smithi076.stdout:ksoftirqd 25 root txt unknown /proc/25/exe 2023-11-09T03:56:14.614 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/1 27 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.614 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/1 27 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.614 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/1 27 root txt unknown /proc/27/exe 2023-11-09T03:56:14.615 INFO:teuthology.orchestra.run.smithi076.stdout:cpuhp/2 28 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.615 INFO:teuthology.orchestra.run.smithi076.stdout:cpuhp/2 28 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.615 INFO:teuthology.orchestra.run.smithi076.stdout:cpuhp/2 28 root txt unknown /proc/28/exe 2023-11-09T03:56:14.615 INFO:teuthology.orchestra.run.smithi076.stdout:idle_inje 29 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.615 INFO:teuthology.orchestra.run.smithi076.stdout:idle_inje 29 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.615 INFO:teuthology.orchestra.run.smithi076.stdout:idle_inje 29 root txt unknown /proc/29/exe 2023-11-09T03:56:14.616 INFO:teuthology.orchestra.run.smithi076.stdout:migration 30 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.616 INFO:teuthology.orchestra.run.smithi076.stdout:migration 30 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.616 INFO:teuthology.orchestra.run.smithi076.stdout:migration 30 root txt unknown /proc/30/exe 2023-11-09T03:56:14.616 INFO:teuthology.orchestra.run.smithi076.stdout:ksoftirqd 31 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.616 INFO:teuthology.orchestra.run.smithi076.stdout:ksoftirqd 31 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.616 INFO:teuthology.orchestra.run.smithi076.stdout:ksoftirqd 31 root txt unknown /proc/31/exe 2023-11-09T03:56:14.617 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/2 33 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.617 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/2 33 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.617 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/2 33 root txt unknown /proc/33/exe 2023-11-09T03:56:14.617 INFO:teuthology.orchestra.run.smithi076.stdout:cpuhp/3 34 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.617 INFO:teuthology.orchestra.run.smithi076.stdout:cpuhp/3 34 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.617 INFO:teuthology.orchestra.run.smithi076.stdout:cpuhp/3 34 root txt unknown /proc/34/exe 2023-11-09T03:56:14.617 INFO:teuthology.orchestra.run.smithi076.stdout:idle_inje 35 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.618 INFO:teuthology.orchestra.run.smithi076.stdout:idle_inje 35 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.618 INFO:teuthology.orchestra.run.smithi076.stdout:idle_inje 35 root txt unknown /proc/35/exe 2023-11-09T03:56:14.618 INFO:teuthology.orchestra.run.smithi076.stdout:migration 36 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.618 INFO:teuthology.orchestra.run.smithi076.stdout:migration 36 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.618 INFO:teuthology.orchestra.run.smithi076.stdout:migration 36 root txt unknown /proc/36/exe 2023-11-09T03:56:14.619 INFO:teuthology.orchestra.run.smithi076.stdout:ksoftirqd 37 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.619 INFO:teuthology.orchestra.run.smithi076.stdout:ksoftirqd 37 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.619 INFO:teuthology.orchestra.run.smithi076.stdout:ksoftirqd 37 root txt unknown /proc/37/exe 2023-11-09T03:56:14.619 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/3 39 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.619 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/3 39 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.619 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/3 39 root txt unknown /proc/39/exe 2023-11-09T03:56:14.620 INFO:teuthology.orchestra.run.smithi076.stdout:cpuhp/4 40 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.620 INFO:teuthology.orchestra.run.smithi076.stdout:cpuhp/4 40 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.620 INFO:teuthology.orchestra.run.smithi076.stdout:cpuhp/4 40 root txt unknown /proc/40/exe 2023-11-09T03:56:14.620 INFO:teuthology.orchestra.run.smithi076.stdout:idle_inje 41 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.620 INFO:teuthology.orchestra.run.smithi076.stdout:idle_inje 41 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.620 INFO:teuthology.orchestra.run.smithi076.stdout:idle_inje 41 root txt unknown /proc/41/exe 2023-11-09T03:56:14.621 INFO:teuthology.orchestra.run.smithi076.stdout:migration 42 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.621 INFO:teuthology.orchestra.run.smithi076.stdout:migration 42 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.621 INFO:teuthology.orchestra.run.smithi076.stdout:migration 42 root txt unknown /proc/42/exe 2023-11-09T03:56:14.621 INFO:teuthology.orchestra.run.smithi076.stdout:ksoftirqd 43 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.621 INFO:teuthology.orchestra.run.smithi076.stdout:ksoftirqd 43 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.621 INFO:teuthology.orchestra.run.smithi076.stdout:ksoftirqd 43 root txt unknown /proc/43/exe 2023-11-09T03:56:14.622 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/4 45 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.622 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/4 45 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.622 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/4 45 root txt unknown /proc/45/exe 2023-11-09T03:56:14.622 INFO:teuthology.orchestra.run.smithi076.stdout:cpuhp/5 46 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.622 INFO:teuthology.orchestra.run.smithi076.stdout:cpuhp/5 46 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.622 INFO:teuthology.orchestra.run.smithi076.stdout:cpuhp/5 46 root txt unknown /proc/46/exe 2023-11-09T03:56:14.623 INFO:teuthology.orchestra.run.smithi076.stdout:idle_inje 47 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.623 INFO:teuthology.orchestra.run.smithi076.stdout:idle_inje 47 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.623 INFO:teuthology.orchestra.run.smithi076.stdout:idle_inje 47 root txt unknown /proc/47/exe 2023-11-09T03:56:14.623 INFO:teuthology.orchestra.run.smithi076.stdout:migration 48 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.623 INFO:teuthology.orchestra.run.smithi076.stdout:migration 48 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.623 INFO:teuthology.orchestra.run.smithi076.stdout:migration 48 root txt unknown /proc/48/exe 2023-11-09T03:56:14.624 INFO:teuthology.orchestra.run.smithi076.stdout:ksoftirqd 49 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.624 INFO:teuthology.orchestra.run.smithi076.stdout:ksoftirqd 49 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.624 INFO:teuthology.orchestra.run.smithi076.stdout:ksoftirqd 49 root txt unknown /proc/49/exe 2023-11-09T03:56:14.624 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/5 51 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.624 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/5 51 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.624 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/5 51 root txt unknown /proc/51/exe 2023-11-09T03:56:14.625 INFO:teuthology.orchestra.run.smithi076.stdout:cpuhp/6 52 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.625 INFO:teuthology.orchestra.run.smithi076.stdout:cpuhp/6 52 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.625 INFO:teuthology.orchestra.run.smithi076.stdout:cpuhp/6 52 root txt unknown /proc/52/exe 2023-11-09T03:56:14.625 INFO:teuthology.orchestra.run.smithi076.stdout:idle_inje 53 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.625 INFO:teuthology.orchestra.run.smithi076.stdout:idle_inje 53 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.625 INFO:teuthology.orchestra.run.smithi076.stdout:idle_inje 53 root txt unknown /proc/53/exe 2023-11-09T03:56:14.626 INFO:teuthology.orchestra.run.smithi076.stdout:migration 54 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.626 INFO:teuthology.orchestra.run.smithi076.stdout:migration 54 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.626 INFO:teuthology.orchestra.run.smithi076.stdout:migration 54 root txt unknown /proc/54/exe 2023-11-09T03:56:14.626 INFO:teuthology.orchestra.run.smithi076.stdout:ksoftirqd 55 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.626 INFO:teuthology.orchestra.run.smithi076.stdout:ksoftirqd 55 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.626 INFO:teuthology.orchestra.run.smithi076.stdout:ksoftirqd 55 root txt unknown /proc/55/exe 2023-11-09T03:56:14.627 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/6 57 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.627 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/6 57 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.627 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/6 57 root txt unknown /proc/57/exe 2023-11-09T03:56:14.627 INFO:teuthology.orchestra.run.smithi076.stdout:cpuhp/7 58 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.627 INFO:teuthology.orchestra.run.smithi076.stdout:cpuhp/7 58 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.627 INFO:teuthology.orchestra.run.smithi076.stdout:cpuhp/7 58 root txt unknown /proc/58/exe 2023-11-09T03:56:14.628 INFO:teuthology.orchestra.run.smithi076.stdout:idle_inje 59 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.628 INFO:teuthology.orchestra.run.smithi076.stdout:idle_inje 59 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.628 INFO:teuthology.orchestra.run.smithi076.stdout:idle_inje 59 root txt unknown /proc/59/exe 2023-11-09T03:56:14.628 INFO:teuthology.orchestra.run.smithi076.stdout:migration 60 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.628 INFO:teuthology.orchestra.run.smithi076.stdout:migration 60 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.628 INFO:teuthology.orchestra.run.smithi076.stdout:migration 60 root txt unknown /proc/60/exe 2023-11-09T03:56:14.629 INFO:teuthology.orchestra.run.smithi076.stdout:ksoftirqd 61 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.629 INFO:teuthology.orchestra.run.smithi076.stdout:ksoftirqd 61 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.629 INFO:teuthology.orchestra.run.smithi076.stdout:ksoftirqd 61 root txt unknown /proc/61/exe 2023-11-09T03:56:14.629 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/7 63 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.629 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/7 63 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.629 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/7 63 root txt unknown /proc/63/exe 2023-11-09T03:56:14.630 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/u 67 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.630 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/u 67 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.630 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/u 67 root txt unknown /proc/67/exe 2023-11-09T03:56:14.630 INFO:teuthology.orchestra.run.smithi076.stdout:kdevtmpfs 71 root cwd DIR 0,5 3500 1 / 2023-11-09T03:56:14.630 INFO:teuthology.orchestra.run.smithi076.stdout:kdevtmpfs 71 root rtd DIR 0,5 3500 1 / 2023-11-09T03:56:14.630 INFO:teuthology.orchestra.run.smithi076.stdout:kdevtmpfs 71 root txt unknown /proc/71/exe 2023-11-09T03:56:14.631 INFO:teuthology.orchestra.run.smithi076.stdout:inet_frag 72 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.631 INFO:teuthology.orchestra.run.smithi076.stdout:inet_frag 72 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.631 INFO:teuthology.orchestra.run.smithi076.stdout:inet_frag 72 root txt unknown /proc/72/exe 2023-11-09T03:56:14.631 INFO:teuthology.orchestra.run.smithi076.stdout:kauditd 73 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.631 INFO:teuthology.orchestra.run.smithi076.stdout:kauditd 73 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.632 INFO:teuthology.orchestra.run.smithi076.stdout:kauditd 73 root txt unknown /proc/73/exe 2023-11-09T03:56:14.632 INFO:teuthology.orchestra.run.smithi076.stdout:khungtask 75 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.632 INFO:teuthology.orchestra.run.smithi076.stdout:khungtask 75 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.632 INFO:teuthology.orchestra.run.smithi076.stdout:khungtask 75 root txt unknown /proc/75/exe 2023-11-09T03:56:14.632 INFO:teuthology.orchestra.run.smithi076.stdout:oom_reape 76 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.633 INFO:teuthology.orchestra.run.smithi076.stdout:oom_reape 76 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.633 INFO:teuthology.orchestra.run.smithi076.stdout:oom_reape 76 root txt unknown /proc/76/exe 2023-11-09T03:56:14.633 INFO:teuthology.orchestra.run.smithi076.stdout:writeback 77 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.633 INFO:teuthology.orchestra.run.smithi076.stdout:writeback 77 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.633 INFO:teuthology.orchestra.run.smithi076.stdout:writeback 77 root txt unknown /proc/77/exe 2023-11-09T03:56:14.634 INFO:teuthology.orchestra.run.smithi076.stdout:kcompactd 78 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.634 INFO:teuthology.orchestra.run.smithi076.stdout:kcompactd 78 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.634 INFO:teuthology.orchestra.run.smithi076.stdout:kcompactd 78 root txt unknown /proc/78/exe 2023-11-09T03:56:14.634 INFO:teuthology.orchestra.run.smithi076.stdout:ksmd 79 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.634 INFO:teuthology.orchestra.run.smithi076.stdout:ksmd 79 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.635 INFO:teuthology.orchestra.run.smithi076.stdout:ksmd 79 root txt unknown /proc/79/exe 2023-11-09T03:56:14.635 INFO:teuthology.orchestra.run.smithi076.stdout:khugepage 80 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.635 INFO:teuthology.orchestra.run.smithi076.stdout:khugepage 80 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.635 INFO:teuthology.orchestra.run.smithi076.stdout:khugepage 80 root txt unknown /proc/80/exe 2023-11-09T03:56:14.635 INFO:teuthology.orchestra.run.smithi076.stdout:cryptd 81 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.635 INFO:teuthology.orchestra.run.smithi076.stdout:cryptd 81 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.636 INFO:teuthology.orchestra.run.smithi076.stdout:cryptd 81 root txt unknown /proc/81/exe 2023-11-09T03:56:14.636 INFO:teuthology.orchestra.run.smithi076.stdout:kintegrit 82 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.636 INFO:teuthology.orchestra.run.smithi076.stdout:kintegrit 82 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.636 INFO:teuthology.orchestra.run.smithi076.stdout:kintegrit 82 root txt unknown /proc/82/exe 2023-11-09T03:56:14.636 INFO:teuthology.orchestra.run.smithi076.stdout:kblockd 83 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.637 INFO:teuthology.orchestra.run.smithi076.stdout:kblockd 83 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.637 INFO:teuthology.orchestra.run.smithi076.stdout:kblockd 83 root txt unknown /proc/83/exe 2023-11-09T03:56:14.637 INFO:teuthology.orchestra.run.smithi076.stdout:blkcg_pun 84 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.637 INFO:teuthology.orchestra.run.smithi076.stdout:blkcg_pun 84 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.637 INFO:teuthology.orchestra.run.smithi076.stdout:blkcg_pun 84 root txt unknown /proc/84/exe 2023-11-09T03:56:14.638 INFO:teuthology.orchestra.run.smithi076.stdout:tpm_dev_w 85 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.638 INFO:teuthology.orchestra.run.smithi076.stdout:tpm_dev_w 85 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.638 INFO:teuthology.orchestra.run.smithi076.stdout:tpm_dev_w 85 root txt unknown /proc/85/exe 2023-11-09T03:56:14.638 INFO:teuthology.orchestra.run.smithi076.stdout:md 86 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.638 INFO:teuthology.orchestra.run.smithi076.stdout:md 86 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.639 INFO:teuthology.orchestra.run.smithi076.stdout:md 86 root txt unknown /proc/86/exe 2023-11-09T03:56:14.639 INFO:teuthology.orchestra.run.smithi076.stdout:md_bitmap 87 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.639 INFO:teuthology.orchestra.run.smithi076.stdout:md_bitmap 87 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.639 INFO:teuthology.orchestra.run.smithi076.stdout:md_bitmap 87 root txt unknown /proc/87/exe 2023-11-09T03:56:14.639 INFO:teuthology.orchestra.run.smithi076.stdout:edac-poll 88 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.639 INFO:teuthology.orchestra.run.smithi076.stdout:edac-poll 88 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.640 INFO:teuthology.orchestra.run.smithi076.stdout:edac-poll 88 root txt unknown /proc/88/exe 2023-11-09T03:56:14.640 INFO:teuthology.orchestra.run.smithi076.stdout:watchdogd 89 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.640 INFO:teuthology.orchestra.run.smithi076.stdout:watchdogd 89 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.640 INFO:teuthology.orchestra.run.smithi076.stdout:watchdogd 89 root txt unknown /proc/89/exe 2023-11-09T03:56:14.640 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/3 91 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.640 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/3 91 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.641 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/3 91 root txt unknown /proc/91/exe 2023-11-09T03:56:14.641 INFO:teuthology.orchestra.run.smithi076.stdout:kswapd0 93 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.641 INFO:teuthology.orchestra.run.smithi076.stdout:kswapd0 93 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.641 INFO:teuthology.orchestra.run.smithi076.stdout:kswapd0 93 root txt unknown /proc/93/exe 2023-11-09T03:56:14.641 INFO:teuthology.orchestra.run.smithi076.stdout:kthrotld 99 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.641 INFO:teuthology.orchestra.run.smithi076.stdout:kthrotld 99 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.642 INFO:teuthology.orchestra.run.smithi076.stdout:kthrotld 99 root txt unknown /proc/99/exe 2023-11-09T03:56:14.642 INFO:teuthology.orchestra.run.smithi076.stdout:acpi_ther 108 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.642 INFO:teuthology.orchestra.run.smithi076.stdout:acpi_ther 108 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.642 INFO:teuthology.orchestra.run.smithi076.stdout:acpi_ther 108 root txt unknown /proc/108/exe 2023-11-09T03:56:14.642 INFO:teuthology.orchestra.run.smithi076.stdout:kmpath_rd 109 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.642 INFO:teuthology.orchestra.run.smithi076.stdout:kmpath_rd 109 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.643 INFO:teuthology.orchestra.run.smithi076.stdout:kmpath_rd 109 root txt unknown /proc/109/exe 2023-11-09T03:56:14.643 INFO:teuthology.orchestra.run.smithi076.stdout:kaluad 110 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.643 INFO:teuthology.orchestra.run.smithi076.stdout:kaluad 110 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.643 INFO:teuthology.orchestra.run.smithi076.stdout:kaluad 110 root txt unknown /proc/110/exe 2023-11-09T03:56:14.643 INFO:teuthology.orchestra.run.smithi076.stdout:mld 111 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.643 INFO:teuthology.orchestra.run.smithi076.stdout:mld 111 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.644 INFO:teuthology.orchestra.run.smithi076.stdout:mld 111 root txt unknown /proc/111/exe 2023-11-09T03:56:14.644 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/6 112 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.644 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/6 112 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.644 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/6 112 root txt unknown /proc/112/exe 2023-11-09T03:56:14.644 INFO:teuthology.orchestra.run.smithi076.stdout:ipv6_addr 113 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.645 INFO:teuthology.orchestra.run.smithi076.stdout:ipv6_addr 113 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.645 INFO:teuthology.orchestra.run.smithi076.stdout:ipv6_addr 113 root txt unknown /proc/113/exe 2023-11-09T03:56:14.645 INFO:teuthology.orchestra.run.smithi076.stdout:kstrp 122 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.645 INFO:teuthology.orchestra.run.smithi076.stdout:kstrp 122 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.645 INFO:teuthology.orchestra.run.smithi076.stdout:kstrp 122 root txt unknown /proc/122/exe 2023-11-09T03:56:14.646 INFO:teuthology.orchestra.run.smithi076.stdout:zswap-shr 129 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.646 INFO:teuthology.orchestra.run.smithi076.stdout:zswap-shr 129 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.646 INFO:teuthology.orchestra.run.smithi076.stdout:zswap-shr 129 root txt unknown /proc/129/exe 2023-11-09T03:56:14.646 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/u 237 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.646 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/u 237 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.646 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/u 237 root txt unknown /proc/237/exe 2023-11-09T03:56:14.647 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/7 276 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.647 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/7 276 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.647 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/7 276 root txt unknown /proc/276/exe 2023-11-09T03:56:14.647 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/1 288 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.647 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/1 288 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.647 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/1 288 root txt unknown /proc/288/exe 2023-11-09T03:56:14.648 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/5 331 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.648 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/5 331 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.648 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/5 331 root txt unknown /proc/331/exe 2023-11-09T03:56:14.648 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/0 351 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.648 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/0 351 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.648 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/0 351 root txt unknown /proc/351/exe 2023-11-09T03:56:14.649 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/2 381 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.649 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/2 381 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.649 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/2 381 root txt unknown /proc/381/exe 2023-11-09T03:56:14.649 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/4 405 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.649 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/4 405 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.650 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/4 405 root txt unknown /proc/405/exe 2023-11-09T03:56:14.650 INFO:teuthology.orchestra.run.smithi076.stdout:nvme-wq 433 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.650 INFO:teuthology.orchestra.run.smithi076.stdout:nvme-wq 433 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.650 INFO:teuthology.orchestra.run.smithi076.stdout:nvme-wq 433 root txt unknown /proc/433/exe 2023-11-09T03:56:14.650 INFO:teuthology.orchestra.run.smithi076.stdout:nvme-rese 434 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.650 INFO:teuthology.orchestra.run.smithi076.stdout:nvme-rese 434 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.651 INFO:teuthology.orchestra.run.smithi076.stdout:nvme-rese 434 root txt unknown /proc/434/exe 2023-11-09T03:56:14.651 INFO:teuthology.orchestra.run.smithi076.stdout:ata_sff 436 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.651 INFO:teuthology.orchestra.run.smithi076.stdout:ata_sff 436 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.651 INFO:teuthology.orchestra.run.smithi076.stdout:ata_sff 436 root txt unknown /proc/436/exe 2023-11-09T03:56:14.651 INFO:teuthology.orchestra.run.smithi076.stdout:ixgbe 438 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.651 INFO:teuthology.orchestra.run.smithi076.stdout:ixgbe 438 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.652 INFO:teuthology.orchestra.run.smithi076.stdout:ixgbe 438 root txt unknown /proc/438/exe 2023-11-09T03:56:14.652 INFO:teuthology.orchestra.run.smithi076.stdout:nvme-dele 439 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.652 INFO:teuthology.orchestra.run.smithi076.stdout:nvme-dele 439 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.652 INFO:teuthology.orchestra.run.smithi076.stdout:nvme-dele 439 root txt unknown /proc/439/exe 2023-11-09T03:56:14.652 INFO:teuthology.orchestra.run.smithi076.stdout:nvme-auth 440 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.652 INFO:teuthology.orchestra.run.smithi076.stdout:nvme-auth 440 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.653 INFO:teuthology.orchestra.run.smithi076.stdout:nvme-auth 440 root txt unknown /proc/440/exe 2023-11-09T03:56:14.653 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_0 443 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.653 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_0 443 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.653 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_0 443 root txt unknown /proc/443/exe 2023-11-09T03:56:14.653 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 444 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.654 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 444 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.654 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 444 root txt unknown /proc/444/exe 2023-11-09T03:56:14.654 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_1 445 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.654 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_1 445 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.654 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_1 445 root txt unknown /proc/445/exe 2023-11-09T03:56:14.655 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 446 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.655 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 446 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.655 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 446 root txt unknown /proc/446/exe 2023-11-09T03:56:14.655 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_2 447 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.655 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_2 447 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.655 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_2 447 root txt unknown /proc/447/exe 2023-11-09T03:56:14.656 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 448 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.656 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 448 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.656 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 448 root txt unknown /proc/448/exe 2023-11-09T03:56:14.656 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_3 449 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.656 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_3 449 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.656 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_3 449 root txt unknown /proc/449/exe 2023-11-09T03:56:14.657 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 450 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.657 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 450 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.657 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 450 root txt unknown /proc/450/exe 2023-11-09T03:56:14.657 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_4 453 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.657 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_4 453 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.657 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_4 453 root txt unknown /proc/453/exe 2023-11-09T03:56:14.658 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 454 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.658 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 454 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.658 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 454 root txt unknown /proc/454/exe 2023-11-09T03:56:14.658 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_5 455 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.658 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_5 455 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.658 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_5 455 root txt unknown /proc/455/exe 2023-11-09T03:56:14.659 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 456 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.659 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 456 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.659 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 456 root txt unknown /proc/456/exe 2023-11-09T03:56:14.659 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_6 458 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.659 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_6 458 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.659 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_6 458 root txt unknown /proc/458/exe 2023-11-09T03:56:14.660 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 459 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.660 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 459 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.660 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 459 root txt unknown /proc/459/exe 2023-11-09T03:56:14.660 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_7 460 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.660 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_7 460 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.660 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_7 460 root txt unknown /proc/460/exe 2023-11-09T03:56:14.661 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 461 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.661 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 461 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.661 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 461 root txt unknown /proc/461/exe 2023-11-09T03:56:14.661 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_8 462 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.661 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_8 462 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.661 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_8 462 root txt unknown /proc/462/exe 2023-11-09T03:56:14.662 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 463 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.662 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 463 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.662 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 463 root txt unknown /proc/463/exe 2023-11-09T03:56:14.662 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_9 464 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.662 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_9 464 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.663 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_eh_9 464 root txt unknown /proc/464/exe 2023-11-09T03:56:14.663 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 465 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.663 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 465 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.663 INFO:teuthology.orchestra.run.smithi076.stdout:scsi_tmf_ 465 root txt unknown /proc/465/exe 2023-11-09T03:56:14.663 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/u 466 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.663 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/u 466 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.664 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/u 466 root txt unknown /proc/466/exe 2023-11-09T03:56:14.664 INFO:teuthology.orchestra.run.smithi076.stdout:jbd2/sda1 491 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.664 INFO:teuthology.orchestra.run.smithi076.stdout:jbd2/sda1 491 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.664 INFO:teuthology.orchestra.run.smithi076.stdout:jbd2/sda1 491 root txt unknown /proc/491/exe 2023-11-09T03:56:14.664 INFO:teuthology.orchestra.run.smithi076.stdout:ext4-rsv- 492 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.664 INFO:teuthology.orchestra.run.smithi076.stdout:ext4-rsv- 492 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.665 INFO:teuthology.orchestra.run.smithi076.stdout:ext4-rsv- 492 root txt unknown /proc/492/exe 2023-11-09T03:56:14.665 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.665 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.665 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root txt REG 8,1 185496 20360 /usr/lib/systemd/systemd-journald 2023-11-09T03:56:14.665 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 0,24 16777216 58 /run/log/journal/15949e3c88704300882482eddfd7baae/system.journal 2023-11-09T03:56:14.666 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:14.666 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:14.666 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.666 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.666 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:14.666 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:14.667 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-11-09T03:56:14.667 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.667 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:14.667 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:14.667 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 8,1 127680 9972 /usr/lib64/libseccomp.so.2.5.2 2023-11-09T03:56:14.667 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-11-09T03:56:14.668 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.668 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:14.668 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 8,1 28552 9804 /usr/lib64/libattr.so.1.1.2501 2023-11-09T03:56:14.668 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-11-09T03:56:14.668 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:14.668 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:14.669 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:14.669 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 8,1 3645472 49253 /usr/lib64/systemd/libsystemd-shared-252.so 2023-11-09T03:56:14.669 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-11-09T03:56:14.669 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 8,1 107424 10180 /usr/lib64/libkmod.so.2.3.6 2023-11-09T03:56:14.669 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:14.670 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:14.670 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root DEL REG 8,1 9806 /usr/lib64/libacl.so.1.1.2301 2023-11-09T03:56:14.670 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:14.670 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.670 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.670 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root mem REG 0,24 8 55 /run/systemd/journal/kernel-seqnum 2023-11-09T03:56:14.671 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.671 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 1w CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.671 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 2w CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.671 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 3u unix 0xffff959a60323300 0t0 15654 /run/systemd/journal/dev-log type=DGRAM (CONNECTED) 2023-11-09T03:56:14.671 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 4u unix 0xffff959a60324400 0t0 15656 /run/systemd/journal/socket type=DGRAM (CONNECTED) 2023-11-09T03:56:14.671 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 5u unix 0xffff959a60329540 0t0 15658 /run/systemd/journal/stdout type=STREAM (LISTEN) 2023-11-09T03:56:14.672 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 6w CHR 1,11 0t0 10 /dev/kmsg 2023-11-09T03:56:14.672 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 7u a_inode 0,14 0 2077 [eventpoll:3,4,5,8,9,10,11,12,13,15,17,18,19,20,21,22,23,24,25,27,28,29,30,32] 2023-11-09T03:56:14.672 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 8u CHR 1,11 0t0 10 /dev/kmsg 2023-11-09T03:56:14.672 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 9u unix 0xffff959a6034f700 0t0 19635 /run/systemd/journal/io.systemd.journal type=STREAM (LISTEN) 2023-11-09T03:56:14.672 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 10r REG 0,20 0 17439 /proc/sys/kernel/hostname 2023-11-09T03:56:14.672 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 11u a_inode 0,14 0 2077 [signalfd] 2023-11-09T03:56:14.673 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 12u a_inode 0,14 0 2077 [signalfd] 2023-11-09T03:56:14.673 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 13u a_inode 0,14 0 2077 [signalfd] 2023-11-09T03:56:14.673 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 14u unix 0xffff959a61972a80 0t0 19636 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.673 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 15u a_inode 0,14 0 2077 [timerfd] 2023-11-09T03:56:14.673 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 16u REG 0,24 16777216 58 /run/log/journal/15949e3c88704300882482eddfd7baae/system.journal 2023-11-09T03:56:14.674 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 17u unix 0xffff959a605eb300 0t0 16621 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-11-09T03:56:14.674 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 18u unix 0xffff959a43263b80 0t0 17657 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-11-09T03:56:14.674 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 19u unix 0xffff959a605ebfc0 0t0 14880 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-11-09T03:56:14.674 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 20u unix 0xffff959a60005500 0t0 19744 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-11-09T03:56:14.674 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 21u unix 0xffff959a6079d940 0t0 24077 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-11-09T03:56:14.675 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 22u unix 0xffff959a6032aec0 0t0 163557 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-11-09T03:56:14.675 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 23u unix 0xffff959a47f0cc80 0t0 21408 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-11-09T03:56:14.675 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 24u unix 0xffff959a605ed940 0t0 21409 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-11-09T03:56:14.675 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 25u unix 0xffff959a605eaec0 0t0 21410 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-11-09T03:56:14.675 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 27u unix 0xffff959a6034ee80 0t0 17772 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-11-09T03:56:14.675 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 28u unix 0xffff959a60006a40 0t0 17773 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-11-09T03:56:14.676 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 29u unix 0xffff959a47f0e1c0 0t0 17676 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-11-09T03:56:14.676 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 30u unix 0xffff959a60001980 0t0 17774 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-11-09T03:56:14.676 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-j 567 root 32u unix 0xffff959a60349100 0t0 2570 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-11-09T03:56:14.676 INFO:teuthology.orchestra.run.smithi076.stdout:ext4lazyi 573 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.676 INFO:teuthology.orchestra.run.smithi076.stdout:ext4lazyi 573 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.676 INFO:teuthology.orchestra.run.smithi076.stdout:ext4lazyi 573 root txt unknown /proc/573/exe 2023-11-09T03:56:14.677 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.677 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.677 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root txt REG 8,1 597744 11595 /usr/bin/udevadm 2023-11-09T03:56:14.677 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 11730661 64128 /etc/udev/hwdb.bin 2023-11-09T03:56:14.677 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 596532 1027 /etc/selinux/targeted/contexts/files/file_contexts.bin 2023-11-09T03:56:14.677 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 21219 1168 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2023-11-09T03:56:14.678 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 566395 265241 /usr/lib/modules/5.14.0-382.el9.x86_64/modules.symbols.bin 2023-11-09T03:56:14.678 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 821243 265238 /usr/lib/modules/5.14.0-382.el9.x86_64/modules.alias.bin 2023-11-09T03:56:14.678 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 383589 265247 /usr/lib/modules/5.14.0-382.el9.x86_64/modules.dep.bin 2023-11-09T03:56:14.678 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:14.678 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.678 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:14.679 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.679 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:14.679 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:14.679 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-11-09T03:56:14.679 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:14.680 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:14.680 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:14.680 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.680 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-11-09T03:56:14.680 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.680 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.681 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 127680 9972 /usr/lib64/libseccomp.so.2.5.2 2023-11-09T03:56:14.681 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-11-09T03:56:14.681 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-11-09T03:56:14.681 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:14.681 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:14.681 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 3645472 49253 /usr/lib64/systemd/libsystemd-shared-252.so 2023-11-09T03:56:14.682 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 28552 9804 /usr/lib64/libattr.so.1.1.2501 2023-11-09T03:56:14.682 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:14.682 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:14.682 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root DEL REG 8,1 9806 /usr/lib64/libacl.so.1.1.2301 2023-11-09T03:56:14.682 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:14.683 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:14.683 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 107424 10180 /usr/lib64/libkmod.so.2.3.6 2023-11-09T03:56:14.683 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 11812 265242 /usr/lib/modules/5.14.0-382.el9.x86_64/modules.builtin.bin 2023-11-09T03:56:14.683 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.683 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root mem REG 0,19 0 19 /sys/fs/selinux/status 2023-11-09T03:56:14.684 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.684 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root 1u unix 0xffff959a605ecc80 0t0 17557 type=STREAM (CONNECTED) 2023-11-09T03:56:14.684 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root 2u unix 0xffff959a605ecc80 0t0 17557 type=STREAM (CONNECTED) 2023-11-09T03:56:14.684 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root 3u unix 0xffff959a4325d940 0t0 20519 /run/udev/control type=SEQPACKET (LISTEN) 2023-11-09T03:56:14.684 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root 4u netlink 0t0 20521 KOBJECT_UEVENT 2023-11-09T03:56:14.684 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root 5u unix 0xffff959a6060dd80 0t0 17569 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.685 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root 6u unix 0xffff959a6060ee80 0t0 17573 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.685 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root 7u unix 0xffff959a6060e1c0 0t0 17574 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.685 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root 8r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.685 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root 9u a_inode 0,14 0 2077 [eventpoll:3,4,6,8,10,11,14] 2023-11-09T03:56:14.685 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root 10u a_inode 0,14 0 2077 [signalfd] 2023-11-09T03:56:14.686 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root 11u a_inode 0,14 0 2077 [timerfd] 2023-11-09T03:56:14.686 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root 12r REG 8,1 11730661 64128 /etc/udev/hwdb.bin 2023-11-09T03:56:14.686 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-u 582 root 14u a_inode 0,14 0 2077 [timerfd] 2023-11-09T03:56:14.686 INFO:teuthology.orchestra.run.smithi076.stdout:ipmi-msgh 631 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.686 INFO:teuthology.orchestra.run.smithi076.stdout:ipmi-msgh 631 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.686 INFO:teuthology.orchestra.run.smithi076.stdout:ipmi-msgh 631 root txt unknown /proc/631/exe 2023-11-09T03:56:14.687 INFO:teuthology.orchestra.run.smithi076.stdout:kipmi0 649 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.687 INFO:teuthology.orchestra.run.smithi076.stdout:kipmi0 649 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.687 INFO:teuthology.orchestra.run.smithi076.stdout:kipmi0 649 root txt unknown /proc/649/exe 2023-11-09T03:56:14.687 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.687 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.688 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc txt REG 8,1 61200 14999 /usr/bin/rpcbind 2023-11-09T03:56:14.688 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.688 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.688 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:14.688 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.688 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:14.689 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:14.689 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:14.689 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:14.689 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:14.689 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:14.689 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:14.690 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:14.690 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:14.690 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:14.690 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:14.690 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.691 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:14.691 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:14.691 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:14.691 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-11-09T03:56:14.691 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc mem REG 8,1 196112 11431 /usr/lib64/libtirpc.so.3.0.0 2023-11-09T03:56:14.691 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.692 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.692 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc 1u unix 0xffff959a60003b80 0t0 19119 type=STREAM (CONNECTED) 2023-11-09T03:56:14.692 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc 2u unix 0xffff959a60003b80 0t0 19119 type=STREAM (CONNECTED) 2023-11-09T03:56:14.692 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc 3u unix 0xffff959a43261540 0t0 20507 /run/rpcbind.sock type=STREAM (LISTEN) 2023-11-09T03:56:14.692 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc 4u IPv4 15769 0t0 TCP *:sunrpc (LISTEN) 2023-11-09T03:56:14.692 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc 5u IPv4 19593 0t0 UDP *:sunrpc 2023-11-09T03:56:14.693 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc 6u IPv6 15775 0t0 TCP *:sunrpc (LISTEN) 2023-11-09T03:56:14.693 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc 7u IPv6 19599 0t0 UDP *:sunrpc 2023-11-09T03:56:14.693 INFO:teuthology.orchestra.run.smithi076.stdout:rpcbind 651 rpc 8rW REG 0,24 0 1035 /run/rpcbind/rpcbind.lock 2023-11-09T03:56:14.693 INFO:teuthology.orchestra.run.smithi076.stdout:rpciod 658 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.693 INFO:teuthology.orchestra.run.smithi076.stdout:rpciod 658 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.694 INFO:teuthology.orchestra.run.smithi076.stdout:rpciod 658 root txt unknown /proc/658/exe 2023-11-09T03:56:14.694 INFO:teuthology.orchestra.run.smithi076.stdout:xprtiod 659 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.694 INFO:teuthology.orchestra.run.smithi076.stdout:xprtiod 659 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.694 INFO:teuthology.orchestra.run.smithi076.stdout:xprtiod 659 root txt unknown /proc/659/exe 2023-11-09T03:56:14.694 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.695 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.695 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root txt REG 8,1 144576 11776 /usr/sbin/auditd 2023-11-09T03:56:14.695 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:14.695 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.695 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.695 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.696 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:14.696 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:14.696 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:14.696 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:14.696 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:14.696 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:14.697 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.697 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:14.697 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:14.697 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root mem REG 8,1 143568 9730 /usr/lib64/libauparse.so.0.0.0 2023-11-09T03:56:14.697 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:14.698 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.698 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root 0u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.698 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root 1u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.698 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root 2u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.698 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root 3u netlink 0t0 2520 AUDIT 2023-11-09T03:56:14.698 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root 4w REG 8,1 54779 1946 /var/log/audit/audit.log 2023-11-09T03:56:14.699 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root 6u unix 0xffff959a60793fc0 0t0 2523 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.699 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root 7u unix 0xffff959a6079aa80 0t0 2522 type=STREAM (CONNECTED) 2023-11-09T03:56:14.699 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root 8u unix 0xffff959a6079b300 0t0 2524 type=STREAM (CONNECTED) 2023-11-09T03:56:14.699 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root 9u unix 0xffff959a6079cc80 0t0 2525 type=STREAM (CONNECTED) 2023-11-09T03:56:14.699 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 root 10u a_inode 0,14 0 2077 [eventfd:0] 2023-11-09T03:56:14.699 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.700 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.700 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root txt REG 8,1 144576 11776 /usr/sbin/auditd 2023-11-09T03:56:14.700 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:14.700 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.700 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.700 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.701 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:14.701 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:14.701 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:14.701 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:14.701 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:14.701 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:14.702 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.702 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:14.702 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:14.702 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root mem REG 8,1 143568 9730 /usr/lib64/libauparse.so.0.0.0 2023-11-09T03:56:14.702 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:14.703 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.703 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root 0u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.703 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root 1u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.703 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root 2u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.703 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root 3u netlink 0t0 2520 AUDIT 2023-11-09T03:56:14.704 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root 4w REG 8,1 54779 1946 /var/log/audit/audit.log 2023-11-09T03:56:14.704 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root 6u unix 0xffff959a60793fc0 0t0 2523 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.704 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root 7u unix 0xffff959a6079aa80 0t0 2522 type=STREAM (CONNECTED) 2023-11-09T03:56:14.704 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root 8u unix 0xffff959a6079b300 0t0 2524 type=STREAM (CONNECTED) 2023-11-09T03:56:14.704 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root 9u unix 0xffff959a6079cc80 0t0 2525 type=STREAM (CONNECTED) 2023-11-09T03:56:14.704 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 662 auditd root 10u a_inode 0,14 0 2077 [eventfd:0] 2023-11-09T03:56:14.705 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.705 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.705 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root txt REG 8,1 144576 11776 /usr/sbin/auditd 2023-11-09T03:56:14.705 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:14.705 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.705 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.706 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.706 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:14.706 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:14.706 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:14.706 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:14.706 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:14.707 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:14.707 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.707 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:14.707 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:14.707 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root mem REG 8,1 143568 9730 /usr/lib64/libauparse.so.0.0.0 2023-11-09T03:56:14.707 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:14.708 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.708 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root 0u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.708 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root 1u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.708 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root 2u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.708 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root 3u netlink 0t0 2520 AUDIT 2023-11-09T03:56:14.708 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root 4w REG 8,1 54779 1946 /var/log/audit/audit.log 2023-11-09T03:56:14.709 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root 6u unix 0xffff959a60793fc0 0t0 2523 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.709 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root 7u unix 0xffff959a6079aa80 0t0 2522 type=STREAM (CONNECTED) 2023-11-09T03:56:14.709 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root 8u unix 0xffff959a6079b300 0t0 2524 type=STREAM (CONNECTED) 2023-11-09T03:56:14.709 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root 9u unix 0xffff959a6079cc80 0t0 2525 type=STREAM (CONNECTED) 2023-11-09T03:56:14.709 INFO:teuthology.orchestra.run.smithi076.stdout:auditd 661 664 auditd root 10u a_inode 0,14 0 2077 [eventfd:0] 2023-11-09T03:56:14.709 INFO:teuthology.orchestra.run.smithi076.stdout:sedispatc 663 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.710 INFO:teuthology.orchestra.run.smithi076.stdout:sedispatc 663 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.710 INFO:teuthology.orchestra.run.smithi076.stdout:sedispatc 663 root txt REG 8,1 19624 11914 /usr/sbin/sedispatch 2023-11-09T03:56:14.710 INFO:teuthology.orchestra.run.smithi076.stdout:sedispatc 663 root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:14.710 INFO:teuthology.orchestra.run.smithi076.stdout:sedispatc 663 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:14.710 INFO:teuthology.orchestra.run.smithi076.stdout:sedispatc 663 root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:14.710 INFO:teuthology.orchestra.run.smithi076.stdout:sedispatc 663 root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:14.711 INFO:teuthology.orchestra.run.smithi076.stdout:sedispatc 663 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.711 INFO:teuthology.orchestra.run.smithi076.stdout:sedispatc 663 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.711 INFO:teuthology.orchestra.run.smithi076.stdout:sedispatc 663 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:14.711 INFO:teuthology.orchestra.run.smithi076.stdout:sedispatc 663 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:14.711 INFO:teuthology.orchestra.run.smithi076.stdout:sedispatc 663 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:14.711 INFO:teuthology.orchestra.run.smithi076.stdout:sedispatc 663 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:14.712 INFO:teuthology.orchestra.run.smithi076.stdout:sedispatc 663 root mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-11-09T03:56:14.712 INFO:teuthology.orchestra.run.smithi076.stdout:sedispatc 663 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.712 INFO:teuthology.orchestra.run.smithi076.stdout:sedispatc 663 root mem REG 8,1 143568 9730 /usr/lib64/libauparse.so.0.0.0 2023-11-09T03:56:14.712 INFO:teuthology.orchestra.run.smithi076.stdout:sedispatc 663 root mem REG 8,1 343752 10128 /usr/lib64/libdbus-1.so.3.19.13 2023-11-09T03:56:14.712 INFO:teuthology.orchestra.run.smithi076.stdout:sedispatc 663 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:14.712 INFO:teuthology.orchestra.run.smithi076.stdout:sedispatc 663 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.713 INFO:teuthology.orchestra.run.smithi076.stdout:sedispatc 663 root 0u unix 0xffff959a60799dc0 0t0 2521 type=STREAM (CONNECTED) 2023-11-09T03:56:14.713 INFO:teuthology.orchestra.run.smithi076.stdout:sedispatc 663 root 1u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.713 INFO:teuthology.orchestra.run.smithi076.stdout:sedispatc 663 root 2u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.713 INFO:teuthology.orchestra.run.smithi076.stdout:sedispatc 663 root 3u unix 0xffff959a60114840 0t0 24146 type=STREAM (CONNECTED) 2023-11-09T03:56:14.713 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.713 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.714 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus txt REG 8,1 131896 11213 /usr/bin/dbus-broker-launch 2023-11-09T03:56:14.714 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus DEL REG 0,1 2048 /memfd:dbus-broker-log 2023-11-09T03:56:14.714 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.714 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:14.714 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:14.715 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.715 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.715 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:14.715 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:14.715 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:14.715 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:14.716 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:14.716 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.716 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-11-09T03:56:14.716 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:14.716 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:14.716 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:14.717 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.717 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.717 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus 1u unix 0xffff959a43265500 0t0 21388 type=STREAM (CONNECTED) 2023-11-09T03:56:14.717 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus 2u unix 0xffff959a43265500 0t0 21388 type=STREAM (CONNECTED) 2023-11-09T03:56:14.717 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus 3u unix 0xffff959a60007700 0t0 22113 /run/dbus/system_bus_socket type=STREAM (LISTEN) 2023-11-09T03:56:14.718 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus 4u unix 0xffff959a43259100 0t0 21398 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.718 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus 5u a_inode 0,14 0 2077 [eventpoll:6,7,8,9,10,11] 2023-11-09T03:56:14.718 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus 6u a_inode 0,14 0 2077 [signalfd] 2023-11-09T03:56:14.718 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus 7r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.718 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus 8u unix 0xffff959a6032c400 0t0 15926 type=STREAM (CONNECTED) 2023-11-09T03:56:14.719 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus 9u unix 0xffff959a60112a80 0t0 16787 type=STREAM (CONNECTED) 2023-11-09T03:56:14.719 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus 10u a_inode 0,14 0 2077 [pidfd:703] 2023-11-09T03:56:14.719 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus 11u a_inode 0,14 0 2077 [timerfd] 2023-11-09T03:56:14.719 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 702 dbus 12u REG 0,1 2097152 2048 /memfd:dbus-broker-log (deleted) 2023-11-09T03:56:14.719 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.719 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.720 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus txt REG 8,1 236336 11212 /usr/bin/dbus-broker 2023-11-09T03:56:14.720 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.720 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.720 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:14.720 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.720 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:14.721 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:14.721 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.721 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.721 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus mem REG 0,19 0 19 /sys/fs/selinux/status 2023-11-09T03:56:14.721 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.722 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus 1u unix 0xffff959a43265500 0t0 21388 type=STREAM (CONNECTED) 2023-11-09T03:56:14.722 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus 2u unix 0xffff959a43265500 0t0 21388 type=STREAM (CONNECTED) 2023-11-09T03:56:14.722 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus 3u sock 0,8 0t0 14879 protocol: NETLINK 2023-11-09T03:56:14.722 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus 4u unix 0xffff959a43259100 0t0 21398 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.722 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus 5u a_inode 0,14 0 2077 [eventpoll:6,7,8,9,10,11,12,13,14,15,16,18] 2023-11-09T03:56:14.722 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus 6u a_inode 0,14 0 2077 [signalfd] 2023-11-09T03:56:14.723 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus 7u unix 0xffff959a60007700 0t0 22113 /run/dbus/system_bus_socket type=STREAM (LISTEN) 2023-11-09T03:56:14.723 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus 8u unix 0xffff959a60000000 0t0 14881 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2023-11-09T03:56:14.723 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus 9u unix 0xffff959a6032cc80 0t0 15927 type=STREAM (CONNECTED) 2023-11-09T03:56:14.723 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus 10u unix 0xffff959a60111dc0 0t0 19755 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2023-11-09T03:56:14.723 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus 11u unix 0xffff959a47f08440 0t0 19760 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2023-11-09T03:56:14.724 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus 12u unix 0xffff959a6034ea40 0t0 14970 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2023-11-09T03:56:14.724 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus 13u unix 0xffff959a605eee80 0t0 14979 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2023-11-09T03:56:14.724 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus 14u unix 0xffff959a60116e80 0t0 18113 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2023-11-09T03:56:14.724 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus 15u unix 0xffff959a60799540 0t0 15960 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2023-11-09T03:56:14.724 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus 16u unix 0xffff959a6079a200 0t0 18094 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2023-11-09T03:56:14.724 INFO:teuthology.orchestra.run.smithi076.stdout:dbus-brok 703 dbus 18u unix 0xffff959a47f0b300 0t0 16018 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2023-11-09T03:56:14.725 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.725 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.725 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root txt REG 8,1 3644720 11633 /usr/sbin/NetworkManager (deleted) 2023-11-09T03:56:14.725 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root DEL REG 8,1 52828 /usr/lib64/NetworkManager/1.44.0-2.el9/libnm-settings-plugin-ifcfg-rh.so 2023-11-09T03:56:14.725 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 343752 10128 /usr/lib64/libdbus-1.so.3.19.13 2023-11-09T03:56:14.725 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-11-09T03:56:14.726 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 62456 9782 /usr/lib64/libjansson.so.4.14.0 2023-11-09T03:56:14.726 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 31792 10954 /usr/lib64/libteamdctl.so.0.1.5 2023-11-09T03:56:14.726 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root DEL REG 8,1 52811 /usr/lib64/NetworkManager/1.44.0-2.el9/libnm-device-plugin-team.so 2023-11-09T03:56:14.726 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:14.726 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-11-09T03:56:14.726 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-11-09T03:56:14.727 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:14.727 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:14.727 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:14.727 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:14.727 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:14.727 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:14.728 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:14.728 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:14.728 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:14.728 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:14.728 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-11-09T03:56:14.728 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:14.729 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-11-09T03:56:14.729 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.729 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:14.729 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:14.729 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-11-09T03:56:14.729 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:14.730 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:14.730 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:14.730 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:14.730 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:14.730 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:14.731 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:14.731 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:14.731 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:14.731 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:14.731 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:14.731 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.732 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:14.732 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:14.732 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:14.732 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 626152 9989 /usr/lib64/libhogweed.so.6.5 2023-11-09T03:56:14.732 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:14.732 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.733 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:14.733 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:14.733 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-11-09T03:56:14.733 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 94112 9901 /usr/lib64/libtasn1.so.6.6.0 2023-11-09T03:56:14.733 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:14.734 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-11-09T03:56:14.734 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-11-09T03:56:14.734 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-11-09T03:56:14.734 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-11-09T03:56:14.734 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:14.735 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 2291576 9994 /usr/lib64/libgnutls.so.30.33.1 2023-11-09T03:56:14.735 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:14.735 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-11-09T03:56:14.735 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:14.735 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.735 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 32408 11334 /usr/lib64/libndp.so.0.2.0 2023-11-09T03:56:14.736 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-11-09T03:56:14.736 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:14.736 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-11-09T03:56:14.736 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-11-09T03:56:14.736 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-11-09T03:56:14.736 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-11-09T03:56:14.737 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-11-09T03:56:14.737 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-11-09T03:56:14.737 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.737 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-11-09T03:56:14.737 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.737 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root 1u unix 0xffff959a605e9540 0t0 17658 type=STREAM (CONNECTED) 2023-11-09T03:56:14.738 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root 2u unix 0xffff959a605e9540 0t0 17658 type=STREAM (CONNECTED) 2023-11-09T03:56:14.738 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root 3u a_inode 0,14 0 2077 [eventfd:6] 2023-11-09T03:56:14.738 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root 4u a_inode 0,14 0 2077 [eventfd:7] 2023-11-09T03:56:14.738 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root 5u unix 0xffff959a60609100 0t0 17695 type=DGRAM (UNCONNECTED) 2023-11-09T03:56:14.738 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root 6u unix 0xffff959a605ed0c0 0t0 17696 type=STREAM (CONNECTED) 2023-11-09T03:56:14.738 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root 7u a_inode 0,14 0 2077 [eventfd:8] 2023-11-09T03:56:14.739 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root 8r REG 0,4 0 4026531840 net 2023-11-09T03:56:14.739 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root 9r REG 0,4 0 4026532495 mnt 2023-11-09T03:56:14.739 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root 10u netlink 0t0 17700 KOBJECT_UEVENT 2023-11-09T03:56:14.739 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root 11u netlink 0t0 17701 GENERIC 2023-11-09T03:56:14.740 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root 12u netlink 0t0 17702 GENERIC 2023-11-09T03:56:14.740 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root 13u netlink 0t0 17703 ROUTE 2023-11-09T03:56:14.740 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root 14r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.740 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root 15r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.740 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root 16u netlink 0t0 19780 KOBJECT_UEVENT 2023-11-09T03:56:14.740 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root 17u a_inode 0,14 0 2077 [eventpoll] 2023-11-09T03:56:14.741 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root 18u unix 0xffff959a6079bfc0 0t0 2562 type=STREAM (CONNECTED) 2023-11-09T03:56:14.741 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root 19u a_inode 0,14 0 2077 [eventpoll:21,26] 2023-11-09T03:56:14.741 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root 20w FIFO 0,24 0t0 1159 /run/systemd/inhibit/1.ref 2023-11-09T03:56:14.741 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root 21u a_inode 0,14 0 2077 [timerfd] 2023-11-09T03:56:14.741 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root 23u a_inode 0,14 0 2077 [eventpoll:24,25] 2023-11-09T03:56:14.741 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root 24u a_inode 0,14 0 2077 [timerfd] 2023-11-09T03:56:14.742 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root 25u pack 17802 0t0 ARP type=SOCK_DGRAM 2023-11-09T03:56:14.742 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 root 26u IPv4 17803 0t0 UDP smithi076.front.sepia.ceph.com:bootpc->store01.front.sepia.ceph.com:bootps 2023-11-09T03:56:14.742 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.742 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.742 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root txt REG 8,1 3644720 11633 /usr/sbin/NetworkManager (deleted) 2023-11-09T03:56:14.742 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root DEL REG 8,1 52828 /usr/lib64/NetworkManager/1.44.0-2.el9/libnm-settings-plugin-ifcfg-rh.so 2023-11-09T03:56:14.743 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 343752 10128 /usr/lib64/libdbus-1.so.3.19.13 2023-11-09T03:56:14.743 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-11-09T03:56:14.743 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 62456 9782 /usr/lib64/libjansson.so.4.14.0 2023-11-09T03:56:14.743 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 31792 10954 /usr/lib64/libteamdctl.so.0.1.5 2023-11-09T03:56:14.743 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root DEL REG 8,1 52811 /usr/lib64/NetworkManager/1.44.0-2.el9/libnm-device-plugin-team.so 2023-11-09T03:56:14.743 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:14.744 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-11-09T03:56:14.744 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-11-09T03:56:14.744 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:14.744 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:14.744 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:14.744 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:14.745 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:14.745 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:14.745 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:14.745 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:14.745 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:14.745 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:14.746 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-11-09T03:56:14.746 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:14.746 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-11-09T03:56:14.746 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.746 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:14.747 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:14.747 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-11-09T03:56:14.747 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:14.747 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:14.747 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:14.748 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:14.748 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:14.748 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:14.748 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:14.748 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:14.748 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:14.749 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:14.749 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:14.749 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.749 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:14.749 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:14.749 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:14.750 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 626152 9989 /usr/lib64/libhogweed.so.6.5 2023-11-09T03:56:14.750 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:14.750 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.750 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:14.750 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:14.751 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-11-09T03:56:14.751 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 94112 9901 /usr/lib64/libtasn1.so.6.6.0 2023-11-09T03:56:14.751 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:14.751 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-11-09T03:56:14.751 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-11-09T03:56:14.751 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-11-09T03:56:14.752 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-11-09T03:56:14.752 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:14.752 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 2291576 9994 /usr/lib64/libgnutls.so.30.33.1 2023-11-09T03:56:14.752 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:14.752 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-11-09T03:56:14.752 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:14.753 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.753 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 32408 11334 /usr/lib64/libndp.so.0.2.0 2023-11-09T03:56:14.753 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-11-09T03:56:14.753 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:14.753 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-11-09T03:56:14.754 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-11-09T03:56:14.754 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-11-09T03:56:14.754 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-11-09T03:56:14.754 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-11-09T03:56:14.754 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-11-09T03:56:14.755 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.755 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-11-09T03:56:14.755 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.755 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root 1u unix 0xffff959a605e9540 0t0 17658 type=STREAM (CONNECTED) 2023-11-09T03:56:14.755 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root 2u unix 0xffff959a605e9540 0t0 17658 type=STREAM (CONNECTED) 2023-11-09T03:56:14.755 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root 3u a_inode 0,14 0 2077 [eventfd:6] 2023-11-09T03:56:14.756 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root 4u a_inode 0,14 0 2077 [eventfd:7] 2023-11-09T03:56:14.756 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root 5u unix 0xffff959a60609100 0t0 17695 type=DGRAM (UNCONNECTED) 2023-11-09T03:56:14.756 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root 6u unix 0xffff959a605ed0c0 0t0 17696 type=STREAM (CONNECTED) 2023-11-09T03:56:14.756 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root 7u a_inode 0,14 0 2077 [eventfd:8] 2023-11-09T03:56:14.756 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root 8r REG 0,4 0 4026531840 net 2023-11-09T03:56:14.756 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root 9r REG 0,4 0 4026532495 mnt 2023-11-09T03:56:14.757 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root 10u netlink 0t0 17700 KOBJECT_UEVENT 2023-11-09T03:56:14.757 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root 11u netlink 0t0 17701 GENERIC 2023-11-09T03:56:14.757 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root 12u netlink 0t0 17702 GENERIC 2023-11-09T03:56:14.757 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root 13u netlink 0t0 17703 ROUTE 2023-11-09T03:56:14.757 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root 14r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.758 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root 15r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.758 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root 16u netlink 0t0 19780 KOBJECT_UEVENT 2023-11-09T03:56:14.758 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root 17u a_inode 0,14 0 2077 [eventpoll] 2023-11-09T03:56:14.758 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root 18u unix 0xffff959a6079bfc0 0t0 2562 type=STREAM (CONNECTED) 2023-11-09T03:56:14.758 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root 19u a_inode 0,14 0 2077 [eventpoll:21,26] 2023-11-09T03:56:14.758 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root 20w FIFO 0,24 0t0 1159 /run/systemd/inhibit/1.ref 2023-11-09T03:56:14.759 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root 21u a_inode 0,14 0 2077 [timerfd] 2023-11-09T03:56:14.759 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root 23u a_inode 0,14 0 2077 [eventpoll:24,25] 2023-11-09T03:56:14.759 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root 24u a_inode 0,14 0 2077 [timerfd] 2023-11-09T03:56:14.759 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root 25u pack 17802 0t0 ARP type=SOCK_DGRAM 2023-11-09T03:56:14.759 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 732 gmain root 26u IPv4 17803 0t0 UDP smithi076.front.sepia.ceph.com:bootpc->store01.front.sepia.ceph.com:bootps 2023-11-09T03:56:14.759 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.760 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.760 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root txt REG 8,1 3644720 11633 /usr/sbin/NetworkManager (deleted) 2023-11-09T03:56:14.760 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root DEL REG 8,1 52828 /usr/lib64/NetworkManager/1.44.0-2.el9/libnm-settings-plugin-ifcfg-rh.so 2023-11-09T03:56:14.760 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 343752 10128 /usr/lib64/libdbus-1.so.3.19.13 2023-11-09T03:56:14.760 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-11-09T03:56:14.760 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 62456 9782 /usr/lib64/libjansson.so.4.14.0 2023-11-09T03:56:14.761 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 31792 10954 /usr/lib64/libteamdctl.so.0.1.5 2023-11-09T03:56:14.761 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root DEL REG 8,1 52811 /usr/lib64/NetworkManager/1.44.0-2.el9/libnm-device-plugin-team.so 2023-11-09T03:56:14.761 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:14.761 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-11-09T03:56:14.761 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-11-09T03:56:14.761 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:14.762 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:14.762 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:14.762 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:14.762 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:14.762 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:14.762 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:14.763 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:14.763 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:14.763 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:14.763 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-11-09T03:56:14.763 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:14.764 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-11-09T03:56:14.764 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.764 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:14.764 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:14.764 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-11-09T03:56:14.764 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:14.765 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:14.765 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:14.765 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:14.765 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:14.765 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:14.765 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:14.766 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:14.766 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:14.766 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:14.766 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:14.766 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.767 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:14.767 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:14.767 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:14.767 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 626152 9989 /usr/lib64/libhogweed.so.6.5 2023-11-09T03:56:14.767 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:14.767 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.768 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:14.768 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:14.768 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-11-09T03:56:14.768 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 94112 9901 /usr/lib64/libtasn1.so.6.6.0 2023-11-09T03:56:14.768 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:14.768 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-11-09T03:56:14.769 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-11-09T03:56:14.769 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-11-09T03:56:14.769 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-11-09T03:56:14.769 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:14.769 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 2291576 9994 /usr/lib64/libgnutls.so.30.33.1 2023-11-09T03:56:14.770 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:14.770 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-11-09T03:56:14.770 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:14.770 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.770 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 32408 11334 /usr/lib64/libndp.so.0.2.0 2023-11-09T03:56:14.770 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-11-09T03:56:14.771 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:14.771 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-11-09T03:56:14.771 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-11-09T03:56:14.771 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-11-09T03:56:14.771 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-11-09T03:56:14.771 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-11-09T03:56:14.772 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-11-09T03:56:14.772 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.772 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-11-09T03:56:14.772 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.772 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root 1u unix 0xffff959a605e9540 0t0 17658 type=STREAM (CONNECTED) 2023-11-09T03:56:14.772 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root 2u unix 0xffff959a605e9540 0t0 17658 type=STREAM (CONNECTED) 2023-11-09T03:56:14.773 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root 3u a_inode 0,14 0 2077 [eventfd:6] 2023-11-09T03:56:14.773 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root 4u a_inode 0,14 0 2077 [eventfd:7] 2023-11-09T03:56:14.773 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root 5u unix 0xffff959a60609100 0t0 17695 type=DGRAM (UNCONNECTED) 2023-11-09T03:56:14.773 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root 6u unix 0xffff959a605ed0c0 0t0 17696 type=STREAM (CONNECTED) 2023-11-09T03:56:14.773 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root 7u a_inode 0,14 0 2077 [eventfd:8] 2023-11-09T03:56:14.773 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root 8r REG 0,4 0 4026531840 net 2023-11-09T03:56:14.774 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root 9r REG 0,4 0 4026532495 mnt 2023-11-09T03:56:14.774 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root 10u netlink 0t0 17700 KOBJECT_UEVENT 2023-11-09T03:56:14.774 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root 11u netlink 0t0 17701 GENERIC 2023-11-09T03:56:14.774 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root 12u netlink 0t0 17702 GENERIC 2023-11-09T03:56:14.774 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root 13u netlink 0t0 17703 ROUTE 2023-11-09T03:56:14.775 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root 14r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.775 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root 15r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.775 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root 16u netlink 0t0 19780 KOBJECT_UEVENT 2023-11-09T03:56:14.775 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root 17u a_inode 0,14 0 2077 [eventpoll] 2023-11-09T03:56:14.775 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root 18u unix 0xffff959a6079bfc0 0t0 2562 type=STREAM (CONNECTED) 2023-11-09T03:56:14.775 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root 19u a_inode 0,14 0 2077 [eventpoll:21,26] 2023-11-09T03:56:14.776 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root 20w FIFO 0,24 0t0 1159 /run/systemd/inhibit/1.ref 2023-11-09T03:56:14.776 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root 21u a_inode 0,14 0 2077 [timerfd] 2023-11-09T03:56:14.776 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root 23u a_inode 0,14 0 2077 [eventpoll:24,25] 2023-11-09T03:56:14.776 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root 24u a_inode 0,14 0 2077 [timerfd] 2023-11-09T03:56:14.776 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root 25u pack 17802 0t0 ARP type=SOCK_DGRAM 2023-11-09T03:56:14.777 INFO:teuthology.orchestra.run.smithi076.stdout:NetworkMa 704 733 gdbus root 26u IPv4 17803 0t0 UDP smithi076.front.sepia.ceph.com:bootpc->store01.front.sepia.ceph.com:bootps 2023-11-09T03:56:14.777 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.777 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.777 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 root txt REG 8,1 61904 10507 /usr/sbin/irqbalance 2023-11-09T03:56:14.777 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 root mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-11-09T03:56:14.778 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.778 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.778 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 root DEL REG 8,1 9998 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:14.778 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 root mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-11-09T03:56:14.778 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:14.778 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.779 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.779 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 root 1u unix 0xffff959a47f0c400 0t0 14886 type=STREAM (CONNECTED) 2023-11-09T03:56:14.779 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 root 2u unix 0xffff959a47f0c400 0t0 14886 type=STREAM (CONNECTED) 2023-11-09T03:56:14.779 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 root 3u a_inode 0,14 0 2077 [eventfd:1] 2023-11-09T03:56:14.779 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 root 4u a_inode 0,14 0 2077 [eventfd:2] 2023-11-09T03:56:14.780 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 root 5u unix 0xffff959a60329dc0 0t0 15944 /run/irqbalance/irqbalance707.sock type=STREAM (LISTEN) 2023-11-09T03:56:14.780 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 root 6u unix 0xffff959a60792ec0 0t0 2786 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.780 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 714 gmain root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.780 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 714 gmain root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.780 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 714 gmain root txt REG 8,1 61904 10507 /usr/sbin/irqbalance 2023-11-09T03:56:14.780 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 714 gmain root mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-11-09T03:56:14.781 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 714 gmain root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.781 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 714 gmain root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.781 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 714 gmain root DEL REG 8,1 9998 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:14.781 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 714 gmain root mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-11-09T03:56:14.781 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 714 gmain root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:14.781 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 714 gmain root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.782 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 714 gmain root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.782 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 714 gmain root 1u unix 0xffff959a47f0c400 0t0 14886 type=STREAM (CONNECTED) 2023-11-09T03:56:14.782 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 714 gmain root 2u unix 0xffff959a47f0c400 0t0 14886 type=STREAM (CONNECTED) 2023-11-09T03:56:14.782 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 714 gmain root 3u a_inode 0,14 0 2077 [eventfd:1] 2023-11-09T03:56:14.782 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 714 gmain root 4u a_inode 0,14 0 2077 [eventfd:2] 2023-11-09T03:56:14.782 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 714 gmain root 5u unix 0xffff959a60329dc0 0t0 15944 /run/irqbalance/irqbalance707.sock type=STREAM (LISTEN) 2023-11-09T03:56:14.783 INFO:teuthology.orchestra.run.smithi076.stdout:irqbalanc 707 714 gmain root 6u unix 0xffff959a60792ec0 0t0 2786 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.783 INFO:teuthology.orchestra.run.smithi076.stdout:lsmd 708 libstoragemgmt cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.783 INFO:teuthology.orchestra.run.smithi076.stdout:lsmd 708 libstoragemgmt rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.783 INFO:teuthology.orchestra.run.smithi076.stdout:lsmd 708 libstoragemgmt txt REG 8,1 32280 11292 /usr/bin/lsmd 2023-11-09T03:56:14.783 INFO:teuthology.orchestra.run.smithi076.stdout:lsmd 708 libstoragemgmt mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.783 INFO:teuthology.orchestra.run.smithi076.stdout:lsmd 708 libstoragemgmt mem REG 8,1 56912 10811 /usr/lib64/libconfig.so.11.0.2 2023-11-09T03:56:14.784 INFO:teuthology.orchestra.run.smithi076.stdout:lsmd 708 libstoragemgmt mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.784 INFO:teuthology.orchestra.run.smithi076.stdout:lsmd 708 libstoragemgmt 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.784 INFO:teuthology.orchestra.run.smithi076.stdout:lsmd 708 libstoragemgmt 1u unix 0xffff959a605e8000 0t0 17667 type=STREAM (CONNECTED) 2023-11-09T03:56:14.784 INFO:teuthology.orchestra.run.smithi076.stdout:lsmd 708 libstoragemgmt 2u unix 0xffff959a605e8000 0t0 17667 type=STREAM (CONNECTED) 2023-11-09T03:56:14.784 INFO:teuthology.orchestra.run.smithi076.stdout:lsmd 708 libstoragemgmt 3uW REG 0,24 0 1130 /run/lsm/ipc/.lsmd-ipc-lock 2023-11-09T03:56:14.784 INFO:teuthology.orchestra.run.smithi076.stdout:lsmd 708 libstoragemgmt 5u unix 0xffff959a605ea200 0t0 17686 /var/run/lsm/ipc/simc type=STREAM (LISTEN) 2023-11-09T03:56:14.785 INFO:teuthology.orchestra.run.smithi076.stdout:lsmd 708 libstoragemgmt 6u unix 0xffff959a605e9980 0t0 17687 /var/run/lsm/ipc/sim type=STREAM (LISTEN) 2023-11-09T03:56:14.785 INFO:teuthology.orchestra.run.smithi076.stdout:mcelog 709 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.785 INFO:teuthology.orchestra.run.smithi076.stdout:mcelog 709 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.785 INFO:teuthology.orchestra.run.smithi076.stdout:mcelog 709 root txt REG 8,1 194568 12461 /usr/sbin/mcelog 2023-11-09T03:56:14.785 INFO:teuthology.orchestra.run.smithi076.stdout:mcelog 709 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.786 INFO:teuthology.orchestra.run.smithi076.stdout:mcelog 709 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.786 INFO:teuthology.orchestra.run.smithi076.stdout:mcelog 709 root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.786 INFO:teuthology.orchestra.run.smithi076.stdout:mcelog 709 root 1u unix 0xffff959a605ee600 0t0 17670 type=STREAM (CONNECTED) 2023-11-09T03:56:14.786 INFO:teuthology.orchestra.run.smithi076.stdout:mcelog 709 root 2u unix 0xffff959a605ee600 0t0 17670 type=STREAM (CONNECTED) 2023-11-09T03:56:14.786 INFO:teuthology.orchestra.run.smithi076.stdout:mcelog 709 root 3r CHR 10,227 0t0 142 /dev/mcelog 2023-11-09T03:56:14.786 INFO:teuthology.orchestra.run.smithi076.stdout:mcelog 709 root 4u unix 0xffff959a605e9100 0t0 17692 /var/run/mcelog-client type=STREAM (LISTEN) 2023-11-09T03:56:14.787 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.787 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.787 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd txt REG 8,1 145440 49355 /usr/lib/polkit-1/polkitd 2023-11-09T03:56:14.787 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.787 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.787 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:14.788 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.788 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:14.788 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:14.788 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:14.788 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:14.789 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:14.789 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:14.789 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd mem REG 8,1 29097776 36983 /usr/lib64/polkit-1/libmozjs-78.so 2023-11-09T03:56:14.789 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:14.789 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:14.790 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-11-09T03:56:14.790 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:14.790 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.790 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-11-09T03:56:14.790 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.790 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-11-09T03:56:14.791 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:14.791 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:14.791 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-11-09T03:56:14.791 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-11-09T03:56:14.791 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-11-09T03:56:14.791 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-11-09T03:56:14.792 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd mem REG 8,1 131296 10178 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-11-09T03:56:14.792 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.792 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.792 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.792 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.792 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd 3r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.793 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd 4u a_inode 0,14 0 2077 [eventfd:3] 2023-11-09T03:56:14.793 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd 5u a_inode 0,14 0 2077 [eventfd:4] 2023-11-09T03:56:14.793 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd 6u unix 0xffff959a6034dd80 0t0 19777 type=STREAM (CONNECTED) 2023-11-09T03:56:14.793 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd 7u a_inode 0,14 0 2077 [eventfd:5] 2023-11-09T03:56:14.793 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd 8r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.794 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd 9u a_inode 0,14 0 2077 [eventfd:10] 2023-11-09T03:56:14.794 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 polkitd 10u unix 0xffff959a61972640 0t0 19778 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.794 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.794 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.794 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd txt REG 8,1 145440 49355 /usr/lib/polkit-1/polkitd 2023-11-09T03:56:14.794 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.795 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.795 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:14.795 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.795 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:14.795 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:14.796 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:14.796 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:14.796 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:14.796 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:14.796 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 29097776 36983 /usr/lib64/polkit-1/libmozjs-78.so 2023-11-09T03:56:14.796 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:14.797 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:14.797 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-11-09T03:56:14.797 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:14.797 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.797 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-11-09T03:56:14.797 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.798 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-11-09T03:56:14.798 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:14.798 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:14.798 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-11-09T03:56:14.798 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-11-09T03:56:14.799 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-11-09T03:56:14.799 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-11-09T03:56:14.799 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 131296 10178 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-11-09T03:56:14.799 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.799 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.799 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.800 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.800 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd 3r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.800 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd 4u a_inode 0,14 0 2077 [eventfd:3] 2023-11-09T03:56:14.800 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd 5u a_inode 0,14 0 2077 [eventfd:4] 2023-11-09T03:56:14.800 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd 6u unix 0xffff959a6034dd80 0t0 19777 type=STREAM (CONNECTED) 2023-11-09T03:56:14.801 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd 7u a_inode 0,14 0 2077 [eventfd:5] 2023-11-09T03:56:14.801 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd 8r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.801 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd 9u a_inode 0,14 0 2077 [eventfd:10] 2023-11-09T03:56:14.801 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 722 gmain polkitd 10u unix 0xffff959a61972640 0t0 19778 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.801 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.801 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.802 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd txt REG 8,1 145440 49355 /usr/lib/polkit-1/polkitd 2023-11-09T03:56:14.802 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.802 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.802 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:14.802 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.802 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:14.803 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:14.803 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:14.803 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:14.803 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:14.803 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:14.804 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 29097776 36983 /usr/lib64/polkit-1/libmozjs-78.so 2023-11-09T03:56:14.804 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:14.804 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:14.804 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-11-09T03:56:14.804 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:14.804 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.805 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-11-09T03:56:14.805 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.805 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-11-09T03:56:14.805 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:14.805 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:14.805 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-11-09T03:56:14.806 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-11-09T03:56:14.806 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-11-09T03:56:14.806 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-11-09T03:56:14.806 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 131296 10178 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-11-09T03:56:14.806 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.807 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.807 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.807 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.807 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd 3r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.807 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd 4u a_inode 0,14 0 2077 [eventfd:3] 2023-11-09T03:56:14.808 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd 5u a_inode 0,14 0 2077 [eventfd:4] 2023-11-09T03:56:14.808 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd 6u unix 0xffff959a6034dd80 0t0 19777 type=STREAM (CONNECTED) 2023-11-09T03:56:14.808 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd 7u a_inode 0,14 0 2077 [eventfd:5] 2023-11-09T03:56:14.808 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd 8r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.808 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd 9u a_inode 0,14 0 2077 [eventfd:10] 2023-11-09T03:56:14.808 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 723 gdbus polkitd 10u unix 0xffff959a61972640 0t0 19778 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.809 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.809 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.809 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd txt REG 8,1 145440 49355 /usr/lib/polkit-1/polkitd 2023-11-09T03:56:14.809 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.809 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.809 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:14.810 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.810 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:14.810 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:14.810 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:14.810 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:14.810 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:14.811 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:14.811 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 29097776 36983 /usr/lib64/polkit-1/libmozjs-78.so 2023-11-09T03:56:14.811 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:14.811 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:14.811 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-11-09T03:56:14.812 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:14.812 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.812 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-11-09T03:56:14.812 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.812 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-11-09T03:56:14.813 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:14.813 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:14.813 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-11-09T03:56:14.813 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-11-09T03:56:14.813 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-11-09T03:56:14.813 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-11-09T03:56:14.814 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 131296 10178 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-11-09T03:56:14.814 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.814 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.814 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.814 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.815 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd 3r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.815 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd 4u a_inode 0,14 0 2077 [eventfd:3] 2023-11-09T03:56:14.815 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd 5u a_inode 0,14 0 2077 [eventfd:4] 2023-11-09T03:56:14.815 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd 6u unix 0xffff959a6034dd80 0t0 19777 type=STREAM (CONNECTED) 2023-11-09T03:56:14.815 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd 7u a_inode 0,14 0 2077 [eventfd:5] 2023-11-09T03:56:14.816 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd 8r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.816 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd 9u a_inode 0,14 0 2077 [eventfd:10] 2023-11-09T03:56:14.816 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 724 JS\x20Hel polkitd 10u unix 0xffff959a61972640 0t0 19778 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.816 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.816 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.816 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd txt REG 8,1 145440 49355 /usr/lib/polkit-1/polkitd 2023-11-09T03:56:14.817 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.817 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.817 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:14.817 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.817 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:14.818 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:14.818 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:14.818 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:14.818 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:14.818 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:14.819 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 29097776 36983 /usr/lib64/polkit-1/libmozjs-78.so 2023-11-09T03:56:14.819 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:14.819 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:14.819 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-11-09T03:56:14.819 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:14.819 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.820 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-11-09T03:56:14.820 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.820 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-11-09T03:56:14.820 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:14.820 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:14.820 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-11-09T03:56:14.821 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-11-09T03:56:14.821 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-11-09T03:56:14.821 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-11-09T03:56:14.821 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 131296 10178 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-11-09T03:56:14.821 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.821 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.822 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.822 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.822 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd 3r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.822 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd 4u a_inode 0,14 0 2077 [eventfd:3] 2023-11-09T03:56:14.822 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd 5u a_inode 0,14 0 2077 [eventfd:4] 2023-11-09T03:56:14.823 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd 6u unix 0xffff959a6034dd80 0t0 19777 type=STREAM (CONNECTED) 2023-11-09T03:56:14.823 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd 7u a_inode 0,14 0 2077 [eventfd:5] 2023-11-09T03:56:14.823 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd 8r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.823 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd 9u a_inode 0,14 0 2077 [eventfd:10] 2023-11-09T03:56:14.823 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 725 JS\x20Hel polkitd 10u unix 0xffff959a61972640 0t0 19778 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.823 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.824 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.824 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd txt REG 8,1 145440 49355 /usr/lib/polkit-1/polkitd 2023-11-09T03:56:14.824 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.824 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.824 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:14.824 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.825 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:14.825 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:14.825 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:14.825 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:14.825 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:14.826 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:14.826 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 29097776 36983 /usr/lib64/polkit-1/libmozjs-78.so 2023-11-09T03:56:14.826 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:14.826 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:14.826 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-11-09T03:56:14.826 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:14.827 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.827 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-11-09T03:56:14.827 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.827 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-11-09T03:56:14.827 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:14.827 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:14.828 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-11-09T03:56:14.828 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-11-09T03:56:14.828 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-11-09T03:56:14.828 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-11-09T03:56:14.828 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 131296 10178 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-11-09T03:56:14.828 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.829 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.829 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.829 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.829 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd 3r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.829 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd 4u a_inode 0,14 0 2077 [eventfd:3] 2023-11-09T03:56:14.830 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd 5u a_inode 0,14 0 2077 [eventfd:4] 2023-11-09T03:56:14.830 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd 6u unix 0xffff959a6034dd80 0t0 19777 type=STREAM (CONNECTED) 2023-11-09T03:56:14.830 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd 7u a_inode 0,14 0 2077 [eventfd:5] 2023-11-09T03:56:14.830 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd 8r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.830 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd 9u a_inode 0,14 0 2077 [eventfd:10] 2023-11-09T03:56:14.830 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 726 JS\x20Hel polkitd 10u unix 0xffff959a61972640 0t0 19778 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.831 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.831 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.831 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd txt REG 8,1 145440 49355 /usr/lib/polkit-1/polkitd 2023-11-09T03:56:14.831 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.831 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.831 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:14.832 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.832 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:14.832 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:14.832 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:14.832 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:14.833 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:14.833 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:14.833 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 29097776 36983 /usr/lib64/polkit-1/libmozjs-78.so 2023-11-09T03:56:14.833 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:14.833 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:14.833 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-11-09T03:56:14.834 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:14.834 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.834 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-11-09T03:56:14.834 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.834 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-11-09T03:56:14.835 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:14.835 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:14.835 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-11-09T03:56:14.835 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-11-09T03:56:14.835 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-11-09T03:56:14.835 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-11-09T03:56:14.836 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 131296 10178 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-11-09T03:56:14.836 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.836 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.836 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.836 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.836 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd 3r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.837 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd 4u a_inode 0,14 0 2077 [eventfd:3] 2023-11-09T03:56:14.837 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd 5u a_inode 0,14 0 2077 [eventfd:4] 2023-11-09T03:56:14.837 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd 6u unix 0xffff959a6034dd80 0t0 19777 type=STREAM (CONNECTED) 2023-11-09T03:56:14.837 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd 7u a_inode 0,14 0 2077 [eventfd:5] 2023-11-09T03:56:14.837 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd 8r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.837 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd 9u a_inode 0,14 0 2077 [eventfd:10] 2023-11-09T03:56:14.838 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 727 JS\x20Hel polkitd 10u unix 0xffff959a61972640 0t0 19778 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.838 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.838 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.838 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd txt REG 8,1 145440 49355 /usr/lib/polkit-1/polkitd 2023-11-09T03:56:14.838 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.839 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.839 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:14.839 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.839 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:14.839 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:14.839 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:14.840 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:14.840 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:14.840 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:14.840 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 29097776 36983 /usr/lib64/polkit-1/libmozjs-78.so 2023-11-09T03:56:14.840 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:14.841 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:14.841 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-11-09T03:56:14.841 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:14.841 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.841 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-11-09T03:56:14.841 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.842 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-11-09T03:56:14.842 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:14.842 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:14.842 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-11-09T03:56:14.842 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-11-09T03:56:14.842 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-11-09T03:56:14.843 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-11-09T03:56:14.843 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 131296 10178 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-11-09T03:56:14.843 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.843 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.843 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.844 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.844 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd 3r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.844 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd 4u a_inode 0,14 0 2077 [eventfd:3] 2023-11-09T03:56:14.844 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd 5u a_inode 0,14 0 2077 [eventfd:4] 2023-11-09T03:56:14.844 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd 6u unix 0xffff959a6034dd80 0t0 19777 type=STREAM (CONNECTED) 2023-11-09T03:56:14.844 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd 7u a_inode 0,14 0 2077 [eventfd:5] 2023-11-09T03:56:14.845 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd 8r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.845 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd 9u a_inode 0,14 0 2077 [eventfd:10] 2023-11-09T03:56:14.845 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 728 JS\x20Hel polkitd 10u unix 0xffff959a61972640 0t0 19778 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.845 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.845 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.846 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd txt REG 8,1 145440 49355 /usr/lib/polkit-1/polkitd 2023-11-09T03:56:14.846 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.846 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.846 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:14.846 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.847 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:14.847 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:14.847 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:14.847 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:14.847 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:14.847 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:14.848 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 29097776 36983 /usr/lib64/polkit-1/libmozjs-78.so 2023-11-09T03:56:14.848 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:14.848 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:14.848 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-11-09T03:56:14.848 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:14.848 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.849 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-11-09T03:56:14.849 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.849 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-11-09T03:56:14.849 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:14.849 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:14.849 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-11-09T03:56:14.850 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-11-09T03:56:14.850 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-11-09T03:56:14.850 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-11-09T03:56:14.850 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 131296 10178 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-11-09T03:56:14.850 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.851 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.851 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.851 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.851 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd 3r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.851 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd 4u a_inode 0,14 0 2077 [eventfd:3] 2023-11-09T03:56:14.851 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd 5u a_inode 0,14 0 2077 [eventfd:4] 2023-11-09T03:56:14.852 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd 6u unix 0xffff959a6034dd80 0t0 19777 type=STREAM (CONNECTED) 2023-11-09T03:56:14.852 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd 7u a_inode 0,14 0 2077 [eventfd:5] 2023-11-09T03:56:14.852 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd 8r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.852 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd 9u a_inode 0,14 0 2077 [eventfd:10] 2023-11-09T03:56:14.852 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 729 JS\x20Hel polkitd 10u unix 0xffff959a61972640 0t0 19778 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.852 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.853 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.853 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd txt REG 8,1 145440 49355 /usr/lib/polkit-1/polkitd 2023-11-09T03:56:14.853 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.853 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.853 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:14.854 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.854 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:14.854 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:14.854 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:14.854 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:14.855 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:14.855 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:14.855 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 29097776 36983 /usr/lib64/polkit-1/libmozjs-78.so 2023-11-09T03:56:14.855 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:14.855 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:14.855 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-11-09T03:56:14.856 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:14.856 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.856 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-11-09T03:56:14.856 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.856 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-11-09T03:56:14.856 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:14.857 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:14.857 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-11-09T03:56:14.857 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-11-09T03:56:14.857 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-11-09T03:56:14.857 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-11-09T03:56:14.857 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 131296 10178 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-11-09T03:56:14.858 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.858 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.858 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.858 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.858 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd 3r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.859 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd 4u a_inode 0,14 0 2077 [eventfd:3] 2023-11-09T03:56:14.859 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd 5u a_inode 0,14 0 2077 [eventfd:4] 2023-11-09T03:56:14.859 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd 6u unix 0xffff959a6034dd80 0t0 19777 type=STREAM (CONNECTED) 2023-11-09T03:56:14.859 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd 7u a_inode 0,14 0 2077 [eventfd:5] 2023-11-09T03:56:14.859 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd 8r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.859 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd 9u a_inode 0,14 0 2077 [eventfd:10] 2023-11-09T03:56:14.860 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 730 JS\x20Hel polkitd 10u unix 0xffff959a61972640 0t0 19778 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.860 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.860 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.860 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd txt REG 8,1 145440 49355 /usr/lib/polkit-1/polkitd 2023-11-09T03:56:14.860 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.861 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.861 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:14.861 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.861 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:14.861 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:14.861 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:14.862 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:14.862 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:14.862 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:14.862 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 29097776 36983 /usr/lib64/polkit-1/libmozjs-78.so 2023-11-09T03:56:14.862 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:14.862 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:14.863 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-11-09T03:56:14.863 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:14.863 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.863 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-11-09T03:56:14.863 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.864 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-11-09T03:56:14.864 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:14.864 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:14.864 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-11-09T03:56:14.864 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-11-09T03:56:14.864 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-11-09T03:56:14.865 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-11-09T03:56:14.865 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 131296 10178 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-11-09T03:56:14.865 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.865 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.865 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.866 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.866 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd 3r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.866 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd 4u a_inode 0,14 0 2077 [eventfd:3] 2023-11-09T03:56:14.866 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd 5u a_inode 0,14 0 2077 [eventfd:4] 2023-11-09T03:56:14.866 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd 6u unix 0xffff959a6034dd80 0t0 19777 type=STREAM (CONNECTED) 2023-11-09T03:56:14.867 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd 7u a_inode 0,14 0 2077 [eventfd:5] 2023-11-09T03:56:14.867 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd 8r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.867 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd 9u a_inode 0,14 0 2077 [eventfd:10] 2023-11-09T03:56:14.867 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 731 JS\x20Hel polkitd 10u unix 0xffff959a61972640 0t0 19778 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.867 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.868 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.868 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd txt REG 8,1 145440 49355 /usr/lib/polkit-1/polkitd 2023-11-09T03:56:14.868 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.868 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.868 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:14.869 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.869 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:14.869 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:14.869 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:14.869 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:14.869 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:14.870 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:14.870 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd mem REG 8,1 29097776 36983 /usr/lib64/polkit-1/libmozjs-78.so 2023-11-09T03:56:14.870 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:14.870 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:14.870 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-11-09T03:56:14.871 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:14.871 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.871 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-11-09T03:56:14.871 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.871 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-11-09T03:56:14.871 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:14.872 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:14.872 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-11-09T03:56:14.872 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-11-09T03:56:14.872 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-11-09T03:56:14.872 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-11-09T03:56:14.873 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd mem REG 8,1 131296 10178 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-11-09T03:56:14.873 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.873 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.873 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.873 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.873 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd 3r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.874 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd 4u a_inode 0,14 0 2077 [eventfd:3] 2023-11-09T03:56:14.874 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd 5u a_inode 0,14 0 2077 [eventfd:4] 2023-11-09T03:56:14.874 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd 6u unix 0xffff959a6034dd80 0t0 19777 type=STREAM (CONNECTED) 2023-11-09T03:56:14.874 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd 7u a_inode 0,14 0 2077 [eventfd:5] 2023-11-09T03:56:14.874 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd 8r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.875 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd 9u a_inode 0,14 0 2077 [eventfd:10] 2023-11-09T03:56:14.875 INFO:teuthology.orchestra.run.smithi076.stdout:polkitd 710 736 polkitd polkitd 10u unix 0xffff959a61972640 0t0 19778 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.875 INFO:teuthology.orchestra.run.smithi076.stdout:smartd 711 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.875 INFO:teuthology.orchestra.run.smithi076.stdout:smartd 711 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.875 INFO:teuthology.orchestra.run.smithi076.stdout:smartd 711 root txt REG 8,1 622048 12498 /usr/sbin/smartd 2023-11-09T03:56:14.876 INFO:teuthology.orchestra.run.smithi076.stdout:smartd 711 root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:14.876 INFO:teuthology.orchestra.run.smithi076.stdout:smartd 711 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.876 INFO:teuthology.orchestra.run.smithi076.stdout:smartd 711 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.876 INFO:teuthology.orchestra.run.smithi076.stdout:smartd 711 root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:14.876 INFO:teuthology.orchestra.run.smithi076.stdout:smartd 711 root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:14.876 INFO:teuthology.orchestra.run.smithi076.stdout:smartd 711 root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:14.876 INFO:teuthology.orchestra.run.smithi076.stdout:smartd 711 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:14.877 INFO:teuthology.orchestra.run.smithi076.stdout:smartd 711 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:14.877 INFO:teuthology.orchestra.run.smithi076.stdout:smartd 711 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:14.877 INFO:teuthology.orchestra.run.smithi076.stdout:smartd 711 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:14.877 INFO:teuthology.orchestra.run.smithi076.stdout:smartd 711 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.877 INFO:teuthology.orchestra.run.smithi076.stdout:smartd 711 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.878 INFO:teuthology.orchestra.run.smithi076.stdout:smartd 711 root mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-11-09T03:56:14.878 INFO:teuthology.orchestra.run.smithi076.stdout:smartd 711 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:14.878 INFO:teuthology.orchestra.run.smithi076.stdout:smartd 711 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.878 INFO:teuthology.orchestra.run.smithi076.stdout:smartd 711 root 0u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.878 INFO:teuthology.orchestra.run.smithi076.stdout:smartd 711 root 1u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.878 INFO:teuthology.orchestra.run.smithi076.stdout:smartd 711 root 2u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.879 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.879 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.879 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root txt REG 8,1 306896 20362 /usr/lib/systemd/systemd-logind 2023-11-09T03:56:14.879 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root DEL REG 8,1 1697 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2023-11-09T03:56:14.879 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root DEL REG 8,1 1143 /etc/selinux/targeted/contexts/files/file_contexts.bin 2023-11-09T03:56:14.879 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 8,1 357408 10120 /usr/lib64/libnss_systemd.so.2 2023-11-09T03:56:14.880 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:14.880 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.880 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-11-09T03:56:14.880 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:14.880 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:14.880 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:14.881 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.881 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:14.881 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:14.881 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.881 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:14.881 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.882 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 8,1 28552 9804 /usr/lib64/libattr.so.1.1.2501 2023-11-09T03:56:14.882 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 8,1 127680 9972 /usr/lib64/libseccomp.so.2.5.2 2023-11-09T03:56:14.882 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-11-09T03:56:14.882 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-11-09T03:56:14.882 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-11-09T03:56:14.882 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.883 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:14.883 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 8,1 107424 10180 /usr/lib64/libkmod.so.2.3.6 2023-11-09T03:56:14.883 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:14.883 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:14.883 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:14.884 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:14.884 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 8,1 3645472 49253 /usr/lib64/systemd/libsystemd-shared-252.so 2023-11-09T03:56:14.884 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root DEL REG 8,1 9806 /usr/lib64/libacl.so.1.1.2301 2023-11-09T03:56:14.884 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:14.884 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.884 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root mem REG 0,19 0 19 /sys/fs/selinux/status 2023-11-09T03:56:14.885 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.885 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root 1u unix 0xffff959a47f09dc0 0t0 14898 type=STREAM (CONNECTED) 2023-11-09T03:56:14.885 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root 2u unix 0xffff959a47f09dc0 0t0 14898 type=STREAM (CONNECTED) 2023-11-09T03:56:14.885 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root 3u unix 0xffff959a618fc840 0t0 14923 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.885 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root 4u a_inode 0,14 0 2077 [eventpoll:5,6,7,8,9,10,11,12,13,14,15,16,17,18,20] 2023-11-09T03:56:14.885 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root 5u a_inode 0,14 0 2077 [signalfd] 2023-11-09T03:56:14.886 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root 6u a_inode 0,14 0 2077 [timerfd] 2023-11-09T03:56:14.886 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root 7r REG 0,21 4096 22923 /sys/devices/virtual/tty/tty0/active 2023-11-09T03:56:14.886 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root 8u netlink 0t0 14934 KOBJECT_UEVENT 2023-11-09T03:56:14.886 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root 9r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.886 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root 10r FIFO 0,24 0t0 1159 /run/systemd/inhibit/1.ref 2023-11-09T03:56:14.886 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root 11u netlink 0t0 14935 KOBJECT_UEVENT 2023-11-09T03:56:14.887 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root 12u netlink 0t0 14936 KOBJECT_UEVENT 2023-11-09T03:56:14.887 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root 13u netlink 0t0 14937 KOBJECT_UEVENT 2023-11-09T03:56:14.887 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root 14u unix 0xffff959a47f0b740 0t0 14938 type=STREAM (CONNECTED) 2023-11-09T03:56:14.887 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root 15u a_inode 0,14 0 2077 [timerfd] 2023-11-09T03:56:14.887 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root 16u CHR 13,65 0t0 138 /dev/input/event1 2023-11-09T03:56:14.887 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root 17u CHR 13,64 0t0 137 /dev/input/event0 2023-11-09T03:56:14.888 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root 18u CHR 13,66 0t0 148 /dev/input/event2 2023-11-09T03:56:14.888 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root 19u CHR 4,6 0t0 25 /dev/tty6 2023-11-09T03:56:14.888 INFO:teuthology.orchestra.run.smithi076.stdout:systemd-l 713 root 20r FIFO 0,24 0t0 1604 /run/systemd/sessions/4.ref 2023-11-09T03:56:14.888 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.888 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.889 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root txt REG 8,1 968800 13744 /usr/sbin/sshd 2023-11-09T03:56:14.889 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:14.889 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.889 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:14.889 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:14.889 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.890 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:14.890 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:14.890 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.890 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:14.890 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:14.890 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:14.891 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:14.891 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:14.891 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:14.891 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:14.891 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-11-09T03:56:14.891 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:14.892 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:14.892 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:14.892 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:14.892 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.892 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:14.892 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.893 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-11-09T03:56:14.893 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-11-09T03:56:14.893 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:14.893 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.893 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.894 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root 1u unix 0xffff959a60006e80 0t0 22184 type=STREAM (CONNECTED) 2023-11-09T03:56:14.894 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root 2u unix 0xffff959a60006e80 0t0 22184 type=STREAM (CONNECTED) 2023-11-09T03:56:14.894 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root 3u IPv4 22197 0t0 TCP *:ssh (LISTEN) 2023-11-09T03:56:14.894 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 757 root 4u IPv6 22199 0t0 TCP *:ssh (LISTEN) 2023-11-09T03:56:14.894 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.894 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.895 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root txt REG 8,1 15448 10498 /usr/bin/python3.9 2023-11-09T03:56:14.895 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:14.895 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 130464 38212 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.895 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 107736 38215 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.895 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 204696 37013 /usr/lib64/girepository-1.0/GLib-2.0.typelib 2023-11-09T03:56:14.896 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.896 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:14.896 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.896 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-11-09T03:56:14.896 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-11-09T03:56:14.896 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-11-09T03:56:14.897 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:14.897 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-11-09T03:56:14.897 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 143200 10195 /usr/lib64/libgirepository-1.0.so.1.0.0 2023-11-09T03:56:14.897 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 357112 39324 /usr/lib64/python3.9/site-packages/gi/_gi.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.897 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-11-09T03:56:14.897 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-11-09T03:56:14.898 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 24736 39204 /usr/lib64/python3.9/site-packages/_dbus_glib_bindings.so 2023-11-09T03:56:14.898 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:14.898 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 71408 38249 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.898 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:14.898 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:14.899 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:14.899 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:14.899 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:14.899 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:14.899 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-11-09T03:56:14.899 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 343752 10128 /usr/lib64/libdbus-1.so.3.19.13 2023-11-09T03:56:14.900 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 171376 39203 /usr/lib64/python3.9/site-packages/_dbus_bindings.so 2023-11-09T03:56:14.900 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:14.900 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:14.900 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.900 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.901 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:14.901 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.901 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.901 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 50232 41979 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.901 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.901 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.902 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 69048 38220 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.902 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 24160 41992 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.902 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 15984 38200 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.902 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 124184 41989 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.902 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.902 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 68000 38241 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.903 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.903 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.903 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 15560 38227 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.903 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.903 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.904 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:14.904 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:14.904 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 101208 41993 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.904 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.904 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:14.905 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.905 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 20528 41991 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.905 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.905 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.905 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root 1u unix 0xffff959a60004840 0t0 22187 type=STREAM (CONNECTED) 2023-11-09T03:56:14.905 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root 2u unix 0xffff959a60004840 0t0 22187 type=STREAM (CONNECTED) 2023-11-09T03:56:14.906 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root 3w REG 8,1 10371 1216 /var/log/tuned/tuned.log 2023-11-09T03:56:14.906 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root 4u sock 0,8 0t0 16853 protocol: NETLINK 2023-11-09T03:56:14.906 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root 5u unix 0xffff959a47f0aa80 0t0 15012 type=STREAM (CONNECTED) 2023-11-09T03:56:14.906 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root 6u a_inode 0,14 0 2077 [eventfd:17] 2023-11-09T03:56:14.906 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root 7u unix 0xffff959a60113b80 0t0 16873 type=STREAM (CONNECTED) 2023-11-09T03:56:14.907 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 root 8u unix 0xffff959a601172c0 0t0 16874 type=STREAM (CONNECTED) 2023-11-09T03:56:14.907 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.907 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.907 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root txt REG 8,1 15448 10498 /usr/bin/python3.9 2023-11-09T03:56:14.907 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:14.907 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 130464 38212 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.908 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 107736 38215 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.908 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 204696 37013 /usr/lib64/girepository-1.0/GLib-2.0.typelib 2023-11-09T03:56:14.908 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.908 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:14.908 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.909 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-11-09T03:56:14.909 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-11-09T03:56:14.909 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-11-09T03:56:14.909 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:14.909 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-11-09T03:56:14.909 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 143200 10195 /usr/lib64/libgirepository-1.0.so.1.0.0 2023-11-09T03:56:14.910 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 357112 39324 /usr/lib64/python3.9/site-packages/gi/_gi.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.910 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-11-09T03:56:14.910 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-11-09T03:56:14.910 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 24736 39204 /usr/lib64/python3.9/site-packages/_dbus_glib_bindings.so 2023-11-09T03:56:14.910 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:14.911 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 71408 38249 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.911 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:14.911 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:14.911 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:14.911 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:14.912 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:14.912 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:14.912 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-11-09T03:56:14.912 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 343752 10128 /usr/lib64/libdbus-1.so.3.19.13 2023-11-09T03:56:14.912 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 171376 39203 /usr/lib64/python3.9/site-packages/_dbus_bindings.so 2023-11-09T03:56:14.912 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:14.913 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:14.913 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.913 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.913 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:14.913 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.914 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.914 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 50232 41979 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.914 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.914 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.914 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 69048 38220 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.914 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 24160 41992 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.915 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 15984 38200 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.915 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 124184 41989 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.915 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.915 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 68000 38241 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.915 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.916 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.916 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 15560 38227 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.916 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.916 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.916 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:14.917 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:14.917 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 101208 41993 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.917 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.917 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:14.917 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.917 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 20528 41991 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:14.918 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.918 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.918 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root 1u unix 0xffff959a60004840 0t0 22187 type=STREAM (CONNECTED) 2023-11-09T03:56:14.918 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root 2u unix 0xffff959a60004840 0t0 22187 type=STREAM (CONNECTED) 2023-11-09T03:56:14.918 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root 3w REG 8,1 10371 1216 /var/log/tuned/tuned.log 2023-11-09T03:56:14.919 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root 4u sock 0,8 0t0 16853 protocol: NETLINK 2023-11-09T03:56:14.919 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root 5u unix 0xffff959a47f0aa80 0t0 15012 type=STREAM (CONNECTED) 2023-11-09T03:56:14.919 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root 6u a_inode 0,14 0 2077 [eventfd:17] 2023-11-09T03:56:14.919 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root 7u unix 0xffff959a60113b80 0t0 16873 type=STREAM (CONNECTED) 2023-11-09T03:56:14.919 INFO:teuthology.orchestra.run.smithi076.stdout:tuned 758 852 tuned root 8u unix 0xffff959a601172c0 0t0 16874 type=STREAM (CONNECTED) 2023-11-09T03:56:14.919 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.920 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.920 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root txt REG 8,1 127680 15021 /usr/sbin/gssproxy 2023-11-09T03:56:14.920 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-11-09T03:56:14.920 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:14.921 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.921 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.921 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 70960 15011 /usr/lib64/libev.so.4.0.0 2023-11-09T03:56:14.921 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 16168 15018 /usr/lib64/libverto-libev.so.1.0.0 2023-11-09T03:56:14.921 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:14.921 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-11-09T03:56:14.922 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-11-09T03:56:14.922 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-11-09T03:56:14.922 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:14.922 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.922 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.923 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 15912 9764 /usr/lib64/libbasicobjects.so.0.1.0 2023-11-09T03:56:14.923 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 20304 9968 /usr/lib64/libpath_utils.so.1.0.1 2023-11-09T03:56:14.923 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 58392 9766 /usr/lib64/libcollection.so.4.1.1 2023-11-09T03:56:14.923 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:14.923 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:14.924 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:14.924 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:14.924 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:14.924 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 142168 11395 /usr/lib64/libgssrpc.so.4.2 2023-11-09T03:56:14.924 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.924 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 16056 9768 /usr/lib64/libref_array.so.1.2.1 2023-11-09T03:56:14.925 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 120640 9970 /usr/lib64/libini_config.so.5.2.1 2023-11-09T03:56:14.925 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 28472 11392 /usr/lib64/libverto.so.1.0.0 2023-11-09T03:56:14.925 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:14.925 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:14.925 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 58368 9689 /usr/lib64/libpopt.so.0.0.1 2023-11-09T03:56:14.925 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-11-09T03:56:14.926 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-11-09T03:56:14.926 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-11-09T03:56:14.926 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-11-09T03:56:14.926 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-11-09T03:56:14.926 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-11-09T03:56:14.926 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.927 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-11-09T03:56:14.927 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.927 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root 1u unix 0xffff959a6034c840 0t0 19847 type=STREAM (CONNECTED) 2023-11-09T03:56:14.927 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root 2u unix 0xffff959a6034c840 0t0 19847 type=STREAM (CONNECTED) 2023-11-09T03:56:14.927 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root 3u unix 0xffff959a60608440 0t0 17782 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.927 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root 4r FIFO 0,13 0t0 17787 pipe 2023-11-09T03:56:14.928 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root 5u a_inode 0,14 0 2077 [eventpoll:4,6,7,8] 2023-11-09T03:56:14.928 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root 6u a_inode 0,14 0 2077 [eventfd:16] 2023-11-09T03:56:14.928 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root 7u unix 0xffff959a605ec840 0t0 17783 /var/lib/gssproxy/default.sock type=STREAM (LISTEN) 2023-11-09T03:56:14.928 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root 8u unix 0xffff959a605eea40 0t0 17784 /run/gssproxy.sock type=STREAM (LISTEN) 2023-11-09T03:56:14.928 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 root 9w FIFO 0,13 0t0 17787 pipe 2023-11-09T03:56:14.928 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.929 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.929 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root txt REG 8,1 127680 15021 /usr/sbin/gssproxy 2023-11-09T03:56:14.929 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-11-09T03:56:14.929 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:14.929 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.929 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.930 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 70960 15011 /usr/lib64/libev.so.4.0.0 2023-11-09T03:56:14.930 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 16168 15018 /usr/lib64/libverto-libev.so.1.0.0 2023-11-09T03:56:14.930 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:14.930 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-11-09T03:56:14.930 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-11-09T03:56:14.930 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-11-09T03:56:14.931 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:14.931 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.931 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.931 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 15912 9764 /usr/lib64/libbasicobjects.so.0.1.0 2023-11-09T03:56:14.931 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 20304 9968 /usr/lib64/libpath_utils.so.1.0.1 2023-11-09T03:56:14.931 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 58392 9766 /usr/lib64/libcollection.so.4.1.1 2023-11-09T03:56:14.932 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:14.932 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:14.932 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:14.932 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:14.932 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:14.932 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 142168 11395 /usr/lib64/libgssrpc.so.4.2 2023-11-09T03:56:14.933 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.933 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 16056 9768 /usr/lib64/libref_array.so.1.2.1 2023-11-09T03:56:14.933 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 120640 9970 /usr/lib64/libini_config.so.5.2.1 2023-11-09T03:56:14.933 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 28472 11392 /usr/lib64/libverto.so.1.0.0 2023-11-09T03:56:14.933 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:14.934 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:14.934 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 58368 9689 /usr/lib64/libpopt.so.0.0.1 2023-11-09T03:56:14.934 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-11-09T03:56:14.934 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-11-09T03:56:14.934 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-11-09T03:56:14.935 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-11-09T03:56:14.935 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-11-09T03:56:14.935 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-11-09T03:56:14.935 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.935 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-11-09T03:56:14.935 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.936 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root 1u unix 0xffff959a6034c840 0t0 19847 type=STREAM (CONNECTED) 2023-11-09T03:56:14.936 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root 2u unix 0xffff959a6034c840 0t0 19847 type=STREAM (CONNECTED) 2023-11-09T03:56:14.936 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root 3u unix 0xffff959a60608440 0t0 17782 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.936 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root 4r FIFO 0,13 0t0 17787 pipe 2023-11-09T03:56:14.936 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root 5u a_inode 0,14 0 2077 [eventpoll:4,6,7,8] 2023-11-09T03:56:14.936 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root 6u a_inode 0,14 0 2077 [eventfd:16] 2023-11-09T03:56:14.937 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root 7u unix 0xffff959a605ec840 0t0 17783 /var/lib/gssproxy/default.sock type=STREAM (LISTEN) 2023-11-09T03:56:14.937 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root 8u unix 0xffff959a605eea40 0t0 17784 /run/gssproxy.sock type=STREAM (LISTEN) 2023-11-09T03:56:14.937 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 772 gssproxy root 9w FIFO 0,13 0t0 17787 pipe 2023-11-09T03:56:14.937 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.937 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.937 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root txt REG 8,1 127680 15021 /usr/sbin/gssproxy 2023-11-09T03:56:14.938 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-11-09T03:56:14.938 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:14.938 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.938 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.938 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 70960 15011 /usr/lib64/libev.so.4.0.0 2023-11-09T03:56:14.938 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 16168 15018 /usr/lib64/libverto-libev.so.1.0.0 2023-11-09T03:56:14.939 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:14.939 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-11-09T03:56:14.939 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-11-09T03:56:14.939 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-11-09T03:56:14.939 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:14.939 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.940 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.940 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 15912 9764 /usr/lib64/libbasicobjects.so.0.1.0 2023-11-09T03:56:14.940 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 20304 9968 /usr/lib64/libpath_utils.so.1.0.1 2023-11-09T03:56:14.940 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 58392 9766 /usr/lib64/libcollection.so.4.1.1 2023-11-09T03:56:14.940 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:14.941 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:14.941 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:14.941 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:14.941 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:14.941 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 142168 11395 /usr/lib64/libgssrpc.so.4.2 2023-11-09T03:56:14.942 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.942 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 16056 9768 /usr/lib64/libref_array.so.1.2.1 2023-11-09T03:56:14.942 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 120640 9970 /usr/lib64/libini_config.so.5.2.1 2023-11-09T03:56:14.942 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 28472 11392 /usr/lib64/libverto.so.1.0.0 2023-11-09T03:56:14.942 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:14.942 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:14.943 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 58368 9689 /usr/lib64/libpopt.so.0.0.1 2023-11-09T03:56:14.943 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-11-09T03:56:14.943 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-11-09T03:56:14.943 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-11-09T03:56:14.943 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-11-09T03:56:14.944 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-11-09T03:56:14.944 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-11-09T03:56:14.944 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.944 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-11-09T03:56:14.944 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.945 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root 1u unix 0xffff959a6034c840 0t0 19847 type=STREAM (CONNECTED) 2023-11-09T03:56:14.945 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root 2u unix 0xffff959a6034c840 0t0 19847 type=STREAM (CONNECTED) 2023-11-09T03:56:14.945 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root 3u unix 0xffff959a60608440 0t0 17782 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.945 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root 4r FIFO 0,13 0t0 17787 pipe 2023-11-09T03:56:14.945 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root 5u a_inode 0,14 0 2077 [eventpoll:4,6,7,8] 2023-11-09T03:56:14.945 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root 6u a_inode 0,14 0 2077 [eventfd:16] 2023-11-09T03:56:14.946 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root 7u unix 0xffff959a605ec840 0t0 17783 /var/lib/gssproxy/default.sock type=STREAM (LISTEN) 2023-11-09T03:56:14.946 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root 8u unix 0xffff959a605eea40 0t0 17784 /run/gssproxy.sock type=STREAM (LISTEN) 2023-11-09T03:56:14.946 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 773 gssproxy root 9w FIFO 0,13 0t0 17787 pipe 2023-11-09T03:56:14.946 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.946 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.947 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root txt REG 8,1 127680 15021 /usr/sbin/gssproxy 2023-11-09T03:56:14.947 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-11-09T03:56:14.947 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:14.947 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.947 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.947 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 70960 15011 /usr/lib64/libev.so.4.0.0 2023-11-09T03:56:14.948 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 16168 15018 /usr/lib64/libverto-libev.so.1.0.0 2023-11-09T03:56:14.948 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:14.948 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-11-09T03:56:14.948 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-11-09T03:56:14.948 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-11-09T03:56:14.949 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:14.949 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.949 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.949 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 15912 9764 /usr/lib64/libbasicobjects.so.0.1.0 2023-11-09T03:56:14.949 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 20304 9968 /usr/lib64/libpath_utils.so.1.0.1 2023-11-09T03:56:14.949 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 58392 9766 /usr/lib64/libcollection.so.4.1.1 2023-11-09T03:56:14.950 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:14.950 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:14.950 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:14.950 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:14.950 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:14.950 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 142168 11395 /usr/lib64/libgssrpc.so.4.2 2023-11-09T03:56:14.951 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.951 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 16056 9768 /usr/lib64/libref_array.so.1.2.1 2023-11-09T03:56:14.951 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 120640 9970 /usr/lib64/libini_config.so.5.2.1 2023-11-09T03:56:14.951 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 28472 11392 /usr/lib64/libverto.so.1.0.0 2023-11-09T03:56:14.951 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:14.951 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:14.952 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 58368 9689 /usr/lib64/libpopt.so.0.0.1 2023-11-09T03:56:14.952 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-11-09T03:56:14.952 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-11-09T03:56:14.952 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-11-09T03:56:14.952 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-11-09T03:56:14.953 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-11-09T03:56:14.953 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-11-09T03:56:14.953 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.953 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-11-09T03:56:14.953 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.954 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root 1u unix 0xffff959a6034c840 0t0 19847 type=STREAM (CONNECTED) 2023-11-09T03:56:14.954 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root 2u unix 0xffff959a6034c840 0t0 19847 type=STREAM (CONNECTED) 2023-11-09T03:56:14.954 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root 3u unix 0xffff959a60608440 0t0 17782 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.954 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root 4r FIFO 0,13 0t0 17787 pipe 2023-11-09T03:56:14.954 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root 5u a_inode 0,14 0 2077 [eventpoll:4,6,7,8] 2023-11-09T03:56:14.955 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root 6u a_inode 0,14 0 2077 [eventfd:16] 2023-11-09T03:56:14.955 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root 7u unix 0xffff959a605ec840 0t0 17783 /var/lib/gssproxy/default.sock type=STREAM (LISTEN) 2023-11-09T03:56:14.955 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root 8u unix 0xffff959a605eea40 0t0 17784 /run/gssproxy.sock type=STREAM (LISTEN) 2023-11-09T03:56:14.955 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 774 gssproxy root 9w FIFO 0,13 0t0 17787 pipe 2023-11-09T03:56:14.955 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.955 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.956 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root txt REG 8,1 127680 15021 /usr/sbin/gssproxy 2023-11-09T03:56:14.956 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-11-09T03:56:14.956 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:14.956 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.956 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.957 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 70960 15011 /usr/lib64/libev.so.4.0.0 2023-11-09T03:56:14.957 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 16168 15018 /usr/lib64/libverto-libev.so.1.0.0 2023-11-09T03:56:14.957 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:14.957 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-11-09T03:56:14.957 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-11-09T03:56:14.957 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-11-09T03:56:14.958 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:14.958 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.958 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.958 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 15912 9764 /usr/lib64/libbasicobjects.so.0.1.0 2023-11-09T03:56:14.958 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 20304 9968 /usr/lib64/libpath_utils.so.1.0.1 2023-11-09T03:56:14.959 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 58392 9766 /usr/lib64/libcollection.so.4.1.1 2023-11-09T03:56:14.959 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:14.959 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:14.959 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:14.959 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:14.959 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:14.960 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 142168 11395 /usr/lib64/libgssrpc.so.4.2 2023-11-09T03:56:14.960 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.960 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 16056 9768 /usr/lib64/libref_array.so.1.2.1 2023-11-09T03:56:14.960 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 120640 9970 /usr/lib64/libini_config.so.5.2.1 2023-11-09T03:56:14.960 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 28472 11392 /usr/lib64/libverto.so.1.0.0 2023-11-09T03:56:14.961 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:14.961 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:14.961 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 58368 9689 /usr/lib64/libpopt.so.0.0.1 2023-11-09T03:56:14.961 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-11-09T03:56:14.961 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-11-09T03:56:14.961 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-11-09T03:56:14.962 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-11-09T03:56:14.962 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-11-09T03:56:14.962 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-11-09T03:56:14.962 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.962 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-11-09T03:56:14.963 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.963 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root 1u unix 0xffff959a6034c840 0t0 19847 type=STREAM (CONNECTED) 2023-11-09T03:56:14.963 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root 2u unix 0xffff959a6034c840 0t0 19847 type=STREAM (CONNECTED) 2023-11-09T03:56:14.963 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root 3u unix 0xffff959a60608440 0t0 17782 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.963 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root 4r FIFO 0,13 0t0 17787 pipe 2023-11-09T03:56:14.964 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root 5u a_inode 0,14 0 2077 [eventpoll:4,6,7,8] 2023-11-09T03:56:14.964 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root 6u a_inode 0,14 0 2077 [eventfd:16] 2023-11-09T03:56:14.964 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root 7u unix 0xffff959a605ec840 0t0 17783 /var/lib/gssproxy/default.sock type=STREAM (LISTEN) 2023-11-09T03:56:14.964 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root 8u unix 0xffff959a605eea40 0t0 17784 /run/gssproxy.sock type=STREAM (LISTEN) 2023-11-09T03:56:14.964 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 775 gssproxy root 9w FIFO 0,13 0t0 17787 pipe 2023-11-09T03:56:14.965 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.965 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.965 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root txt REG 8,1 127680 15021 /usr/sbin/gssproxy 2023-11-09T03:56:14.965 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-11-09T03:56:14.965 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:14.965 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.966 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.966 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 70960 15011 /usr/lib64/libev.so.4.0.0 2023-11-09T03:56:14.966 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 16168 15018 /usr/lib64/libverto-libev.so.1.0.0 2023-11-09T03:56:14.966 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:14.966 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-11-09T03:56:14.966 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-11-09T03:56:14.967 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-11-09T03:56:14.967 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:14.967 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.967 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.967 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 15912 9764 /usr/lib64/libbasicobjects.so.0.1.0 2023-11-09T03:56:14.968 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 20304 9968 /usr/lib64/libpath_utils.so.1.0.1 2023-11-09T03:56:14.968 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 58392 9766 /usr/lib64/libcollection.so.4.1.1 2023-11-09T03:56:14.968 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:14.968 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:14.968 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:14.968 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:14.969 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:14.969 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 142168 11395 /usr/lib64/libgssrpc.so.4.2 2023-11-09T03:56:14.969 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.969 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 16056 9768 /usr/lib64/libref_array.so.1.2.1 2023-11-09T03:56:14.969 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 120640 9970 /usr/lib64/libini_config.so.5.2.1 2023-11-09T03:56:14.969 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 28472 11392 /usr/lib64/libverto.so.1.0.0 2023-11-09T03:56:14.970 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:14.970 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:14.970 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 58368 9689 /usr/lib64/libpopt.so.0.0.1 2023-11-09T03:56:14.970 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-11-09T03:56:14.970 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-11-09T03:56:14.971 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-11-09T03:56:14.971 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-11-09T03:56:14.971 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-11-09T03:56:14.971 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-11-09T03:56:14.971 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.971 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-11-09T03:56:14.972 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.972 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root 1u unix 0xffff959a6034c840 0t0 19847 type=STREAM (CONNECTED) 2023-11-09T03:56:14.972 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root 2u unix 0xffff959a6034c840 0t0 19847 type=STREAM (CONNECTED) 2023-11-09T03:56:14.972 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root 3u unix 0xffff959a60608440 0t0 17782 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.972 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root 4r FIFO 0,13 0t0 17787 pipe 2023-11-09T03:56:14.972 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root 5u a_inode 0,14 0 2077 [eventpoll:4,6,7,8] 2023-11-09T03:56:14.973 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root 6u a_inode 0,14 0 2077 [eventfd:16] 2023-11-09T03:56:14.973 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root 7u unix 0xffff959a605ec840 0t0 17783 /var/lib/gssproxy/default.sock type=STREAM (LISTEN) 2023-11-09T03:56:14.973 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root 8u unix 0xffff959a605eea40 0t0 17784 /run/gssproxy.sock type=STREAM (LISTEN) 2023-11-09T03:56:14.973 INFO:teuthology.orchestra.run.smithi076.stdout:gssproxy 771 776 gssproxy root 9w FIFO 0,13 0t0 17787 pipe 2023-11-09T03:56:14.973 INFO:teuthology.orchestra.run.smithi076.stdout:atd 778 root cwd DIR 8,1 4096 1657 /var/spool/at 2023-11-09T03:56:14.973 INFO:teuthology.orchestra.run.smithi076.stdout:atd 778 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.974 INFO:teuthology.orchestra.run.smithi076.stdout:atd 778 root txt REG 8,1 32680 12473 /usr/sbin/atd 2023-11-09T03:56:14.974 INFO:teuthology.orchestra.run.smithi076.stdout:atd 778 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.974 INFO:teuthology.orchestra.run.smithi076.stdout:atd 778 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:14.974 INFO:teuthology.orchestra.run.smithi076.stdout:atd 778 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.974 INFO:teuthology.orchestra.run.smithi076.stdout:atd 778 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.975 INFO:teuthology.orchestra.run.smithi076.stdout:atd 778 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-11-09T03:56:14.975 INFO:teuthology.orchestra.run.smithi076.stdout:atd 778 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:14.975 INFO:teuthology.orchestra.run.smithi076.stdout:atd 778 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.975 INFO:teuthology.orchestra.run.smithi076.stdout:atd 778 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-11-09T03:56:14.975 INFO:teuthology.orchestra.run.smithi076.stdout:atd 778 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.975 INFO:teuthology.orchestra.run.smithi076.stdout:atd 778 root 0u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.976 INFO:teuthology.orchestra.run.smithi076.stdout:atd 778 root 1u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.976 INFO:teuthology.orchestra.run.smithi076.stdout:atd 778 root 2u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.976 INFO:teuthology.orchestra.run.smithi076.stdout:atd 778 root 3uW REG 0,24 4 1181 /run/atd.pid 2023-11-09T03:56:14.976 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.976 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.976 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root txt REG 8,1 74552 11234 /usr/sbin/crond 2023-11-09T03:56:14.977 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:14.977 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-11-09T03:56:14.977 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.977 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.977 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.977 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-11-09T03:56:14.978 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-11-09T03:56:14.978 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-11-09T03:56:14.978 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:14.978 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:14.978 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-11-09T03:56:14.979 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:14.979 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-11-09T03:56:14.979 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.979 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-11-09T03:56:14.979 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-11-09T03:56:14.979 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-11-09T03:56:14.979 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-11-09T03:56:14.980 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-11-09T03:56:14.980 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-11-09T03:56:14.980 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.980 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-11-09T03:56:14.980 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.980 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root 1u unix 0xffff959a6034d0c0 0t0 19860 type=STREAM (CONNECTED) 2023-11-09T03:56:14.981 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root 2u unix 0xffff959a6034d0c0 0t0 19860 type=STREAM (CONNECTED) 2023-11-09T03:56:14.981 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root 3uW REG 0,24 4 1180 /run/crond.pid 2023-11-09T03:56:14.981 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root 4u unix 0xffff959a619750c0 0t0 19863 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.981 INFO:teuthology.orchestra.run.smithi076.stdout:crond 779 root 5r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.981 INFO:teuthology.orchestra.run.smithi076.stdout:nrpe 868 nrpe cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.982 INFO:teuthology.orchestra.run.smithi076.stdout:nrpe 868 nrpe rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.982 INFO:teuthology.orchestra.run.smithi076.stdout:nrpe 868 nrpe txt REG 8,1 80984 12959 /usr/sbin/nrpe 2023-11-09T03:56:14.982 INFO:teuthology.orchestra.run.smithi076.stdout:nrpe 868 nrpe mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.982 INFO:teuthology.orchestra.run.smithi076.stdout:nrpe 868 nrpe mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:14.982 INFO:teuthology.orchestra.run.smithi076.stdout:nrpe 868 nrpe mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.982 INFO:teuthology.orchestra.run.smithi076.stdout:nrpe 868 nrpe mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:14.983 INFO:teuthology.orchestra.run.smithi076.stdout:nrpe 868 nrpe mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.983 INFO:teuthology.orchestra.run.smithi076.stdout:nrpe 868 nrpe 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.983 INFO:teuthology.orchestra.run.smithi076.stdout:nrpe 868 nrpe 1w CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.983 INFO:teuthology.orchestra.run.smithi076.stdout:nrpe 868 nrpe 2w CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.983 INFO:teuthology.orchestra.run.smithi076.stdout:nrpe 868 nrpe 3u IPv4 15046 0t0 TCP *:nrpe (LISTEN) 2023-11-09T03:56:14.984 INFO:teuthology.orchestra.run.smithi076.stdout:nrpe 868 nrpe 4u unix 0xffff959a618f9100 0t0 15038 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.984 INFO:teuthology.orchestra.run.smithi076.stdout:nrpe 868 nrpe 5u IPv6 15047 0t0 TCP *:nrpe (LISTEN) 2023-11-09T03:56:14.984 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1302 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.984 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1302 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.984 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1302 root txt REG 8,1 58072 13690 /usr/sbin/agetty 2023-11-09T03:56:14.984 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1302 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:14.985 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1302 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.985 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1302 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:14.985 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1302 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.985 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1302 root 0u CHR 4,1 0t0 20 /dev/tty1 2023-11-09T03:56:14.985 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1302 root 1u CHR 4,1 0t0 20 /dev/tty1 2023-11-09T03:56:14.985 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1302 root 2u CHR 4,1 0t0 20 /dev/tty1 2023-11-09T03:56:14.986 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1302 root 4r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.986 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1304 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.986 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1304 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.986 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1304 root txt REG 8,1 58072 13690 /usr/sbin/agetty 2023-11-09T03:56:14.986 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1304 root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-11-09T03:56:14.987 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1304 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.987 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1304 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:14.987 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1304 root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-11-09T03:56:14.987 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1304 root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-11-09T03:56:14.987 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1304 root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-11-09T03:56:14.987 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1304 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:14.988 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1304 root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-11-09T03:56:14.988 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1304 root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-11-09T03:56:14.988 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1304 root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-11-09T03:56:14.988 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1304 root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-11-09T03:56:14.988 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1304 root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-11-09T03:56:14.988 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1304 root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-11-09T03:56:14.989 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1304 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.989 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1304 root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-11-09T03:56:14.989 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1304 root 0u CHR 4,65 0t0 114 /dev/ttyS1 2023-11-09T03:56:14.989 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1304 root 1u CHR 4,65 0t0 114 /dev/ttyS1 2023-11-09T03:56:14.989 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1304 root 2u CHR 4,65 0t0 114 /dev/ttyS1 2023-11-09T03:56:14.989 INFO:teuthology.orchestra.run.smithi076.stdout:agetty 1304 root 4r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.990 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu cwd DIR 8,1 4096 2 / 2023-11-09T03:56:14.990 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu rtd DIR 8,1 4096 2 / 2023-11-09T03:56:14.990 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu txt REG 8,1 102120 20353 /usr/lib/systemd/systemd 2023-11-09T03:56:14.990 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu DEL REG 8,1 1143 /etc/selinux/targeted/contexts/files/file_contexts.bin 2023-11-09T03:56:14.990 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:14.991 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:14.991 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:14.991 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:14.991 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu DEL REG 8,1 1697 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2023-11-09T03:56:14.991 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:14.992 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:14.992 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-11-09T03:56:14.992 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:14.992 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:14.992 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:14.992 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:14.993 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:14.993 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:14.993 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:14.993 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 8,1 3645472 49253 /usr/lib64/systemd/libsystemd-shared-252.so 2023-11-09T03:56:14.993 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 8,1 28552 9804 /usr/lib64/libattr.so.1.1.2501 2023-11-09T03:56:14.994 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:14.994 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-11-09T03:56:14.994 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-11-09T03:56:14.994 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:14.994 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 8,1 2174088 49251 /usr/lib64/systemd/libsystemd-core-252.so 2023-11-09T03:56:14.994 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:14.995 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu DEL REG 8,1 9806 /usr/lib64/libacl.so.1.1.2301 2023-11-09T03:56:14.995 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 8,1 107424 10180 /usr/lib64/libkmod.so.2.3.6 2023-11-09T03:56:14.995 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:14.995 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-11-09T03:56:14.995 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 8,1 127680 9972 /usr/lib64/libseccomp.so.2.5.2 2023-11-09T03:56:14.995 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:14.996 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:14.996 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu mem REG 0,19 0 19 /sys/fs/selinux/status 2023-11-09T03:56:14.996 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:14.996 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu 1u unix 0xffff959a60798000 0t0 24650 type=STREAM (CONNECTED) 2023-11-09T03:56:14.996 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu 2u unix 0xffff959a60798000 0t0 24650 type=STREAM (CONNECTED) 2023-11-09T03:56:14.997 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu 3u unix 0xffff959a60795500 0t0 24678 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.997 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu 4u a_inode 0,14 0 2077 [eventpoll:5,6,8,9,11,12,14,15,16,17,19,20,21] 2023-11-09T03:56:14.997 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu 5u a_inode 0,14 0 2077 [signalfd] 2023-11-09T03:56:14.997 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu 6r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.997 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu 7r DIR 0,25 0 3391 /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service 2023-11-09T03:56:14.997 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu 8u a_inode 0,14 0 2077 [timerfd] 2023-11-09T03:56:14.998 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu 9u a_inode 0,14 0 2077 [eventpoll:10,13] 2023-11-09T03:56:14.998 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu 10r REG 0,20 0 24667 /proc/4854/mountinfo 2023-11-09T03:56:14.998 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu 11r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.998 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu 12u unix 0xffff959a60799980 0t0 24700 /run/user/1000/bus type=STREAM (LISTEN) 2023-11-09T03:56:14.998 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu 13r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:14.998 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu 14r REG 0,20 0 4026532076 /proc/swaps 2023-11-09T03:56:14.999 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu 15u netlink 0t0 24682 KOBJECT_UEVENT 2023-11-09T03:56:14.999 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu 16u unix 0xffff959a60792640 0t0 24687 /run/user/1000/systemd/notify type=DGRAM (UNCONNECTED) 2023-11-09T03:56:14.999 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu 17u unix 0xffff959a607950c0 0t0 24688 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.999 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu 18u unix 0xffff959a60793300 0t0 24689 type=DGRAM (CONNECTED) 2023-11-09T03:56:14.999 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu 19u unix 0xffff959a60798880 0t0 24690 /run/user/1000/systemd/private type=STREAM (LISTEN) 2023-11-09T03:56:14.999 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu 20u unix 0xffff959a60798cc0 0t0 24692 type=STREAM (CONNECTED) 2023-11-09T03:56:15.000 INFO:teuthology.orchestra.run.smithi076.stdout:systemd 4854 ubuntu 21u a_inode 0,14 0 2077 [timerfd] 2023-11-09T03:56:15.000 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.000 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.000 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu txt REG 8,1 102120 20353 /usr/lib/systemd/systemd 2023-11-09T03:56:15.000 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 357408 10120 /usr/lib64/libnss_systemd.so.2 2023-11-09T03:56:15.000 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 14928 32270 /usr/lib64/security/pam_deny.so 2023-11-09T03:56:15.001 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 19600 10967 /usr/lib64/libpam_misc.so.0.82.1 2023-11-09T03:56:15.001 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 19400 32305 /usr/lib64/security/pam_succeed_if.so 2023-11-09T03:56:15.001 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 514280 32422 /usr/lib64/security/pam_systemd.so 2023-11-09T03:56:15.001 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 27536 32284 /usr/lib64/security/pam_limits.so 2023-11-09T03:56:15.001 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 44152 32291 /usr/lib64/security/pam_namespace.so 2023-11-09T03:56:15.001 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 15184 32309 /usr/lib64/security/pam_umask.so 2023-11-09T03:56:15.002 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 15352 32282 /usr/lib64/security/pam_keyinit.so 2023-11-09T03:56:15.002 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 15240 32287 /usr/lib64/security/pam_loginuid.so 2023-11-09T03:56:15.002 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.002 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.002 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:15.002 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.003 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 27616 32299 /usr/lib64/security/pam_selinux.so 2023-11-09T03:56:15.003 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu DEL REG 8,1 63926 /etc/selinux/targeted/contexts/files/file_contexts.bin 2023-11-09T03:56:15.003 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-11-09T03:56:15.003 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.003 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 56712 32310 /usr/lib64/security/pam_unix.so 2023-11-09T03:56:15.004 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:15.004 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:15.004 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.004 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:15.004 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.004 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:15.005 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 3645472 49253 /usr/lib64/systemd/libsystemd-shared-252.so 2023-11-09T03:56:15.005 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 28552 9804 /usr/lib64/libattr.so.1.1.2501 2023-11-09T03:56:15.005 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:15.005 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-11-09T03:56:15.005 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.005 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.006 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 2174088 49251 /usr/lib64/systemd/libsystemd-core-252.so 2023-11-09T03:56:15.006 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:15.006 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu DEL REG 8,1 9806 /usr/lib64/libacl.so.1.1.2301 2023-11-09T03:56:15.006 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-11-09T03:56:15.006 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.007 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 107424 10180 /usr/lib64/libkmod.so.2.3.6 2023-11-09T03:56:15.007 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:15.007 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-11-09T03:56:15.007 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 127680 9972 /usr/lib64/libseccomp.so.2.5.2 2023-11-09T03:56:15.007 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.007 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu DEL REG 8,1 63928 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2023-11-09T03:56:15.008 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.008 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu mem REG 0,19 0 19 /sys/fs/selinux/status 2023-11-09T03:56:15.008 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.008 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu 1u unix 0xffff959a60798000 0t0 24650 type=STREAM (CONNECTED) 2023-11-09T03:56:15.008 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu 2u unix 0xffff959a60798000 0t0 24650 type=STREAM (CONNECTED) 2023-11-09T03:56:15.008 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu 3u a_inode 0,14 0 2077 [eventfd:9] 2023-11-09T03:56:15.009 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu 4u a_inode 0,14 0 2077 [eventfd:11] 2023-11-09T03:56:15.009 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu 6w FIFO 0,13 0t0 24654 pipe 2023-11-09T03:56:15.009 INFO:teuthology.orchestra.run.smithi076.stdout:(sd-pam) 4856 ubuntu 7u unix 0xffff959a60790000 0t0 24670 type=DGRAM (CONNECTED) 2023-11-09T03:56:15.009 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.009 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.009 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root txt REG 8,1 769928 12237 /usr/sbin/rsyslogd 2023-11-09T03:56:15.010 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root mem REG 0,24 16777216 58 /run/log/journal/15949e3c88704300882482eddfd7baae/system.journal 2023-11-09T03:56:15.010 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root mem REG 8,1 33592 12207 /usr/lib64/rsyslog/imjournal.so 2023-11-09T03:56:15.010 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:15.010 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.010 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:15.010 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:15.011 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.011 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root mem REG 8,1 42248 12214 /usr/lib64/rsyslog/imuxsock.so 2023-11-09T03:56:15.013 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root mem REG 8,1 32808 12215 /usr/lib64/rsyslog/lmnet.so 2023-11-09T03:56:15.013 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:15.013 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:15.014 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.014 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:15.014 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root mem REG 8,1 36328 9712 /usr/lib64/libuuid.so.1.3.0 2023-11-09T03:56:15.014 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-11-09T03:56:15.014 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root mem REG 8,1 52856 12201 /usr/lib64/libfastjson.so.4.3.0 2023-11-09T03:56:15.014 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root mem REG 8,1 16032 12200 /usr/lib64/libestr.so.0.0.0 2023-11-09T03:56:15.015 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.015 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.015 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.015 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root 1w CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.015 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root 2w CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.015 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root 3r CHR 1,9 0t0 9 /dev/urandom 2023-11-09T03:56:15.016 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root 4u unix 0xffff959a4837c840 0t0 27753 type=DGRAM (CONNECTED) 2023-11-09T03:56:15.016 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root 5w REG 8,1 1151277 1642 /var/log/secure 2023-11-09T03:56:15.016 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root 6r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:15.016 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root 8w REG 8,1 1475705 1641 /var/log/messages 2023-11-09T03:56:15.016 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 root 9r REG 0,24 16777216 58 /run/log/journal/15949e3c88704300882482eddfd7baae/system.journal 2023-11-09T03:56:15.017 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.017 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.017 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root txt REG 8,1 769928 12237 /usr/sbin/rsyslogd 2023-11-09T03:56:15.017 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root mem REG 0,24 16777216 58 /run/log/journal/15949e3c88704300882482eddfd7baae/system.journal 2023-11-09T03:56:15.017 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root mem REG 8,1 33592 12207 /usr/lib64/rsyslog/imjournal.so 2023-11-09T03:56:15.017 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:15.018 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.018 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:15.018 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:15.018 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.018 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root mem REG 8,1 42248 12214 /usr/lib64/rsyslog/imuxsock.so 2023-11-09T03:56:15.019 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root mem REG 8,1 32808 12215 /usr/lib64/rsyslog/lmnet.so 2023-11-09T03:56:15.019 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:15.019 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:15.019 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.019 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:15.019 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root mem REG 8,1 36328 9712 /usr/lib64/libuuid.so.1.3.0 2023-11-09T03:56:15.020 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-11-09T03:56:15.020 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root mem REG 8,1 52856 12201 /usr/lib64/libfastjson.so.4.3.0 2023-11-09T03:56:15.020 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root mem REG 8,1 16032 12200 /usr/lib64/libestr.so.0.0.0 2023-11-09T03:56:15.020 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.020 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.020 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.021 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root 1w CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.021 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root 2w CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.021 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root 3r CHR 1,9 0t0 9 /dev/urandom 2023-11-09T03:56:15.021 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root 4u unix 0xffff959a4837c840 0t0 27753 type=DGRAM (CONNECTED) 2023-11-09T03:56:15.021 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root 5w REG 8,1 1151277 1642 /var/log/secure 2023-11-09T03:56:15.021 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root 6r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:15.022 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root 8w REG 8,1 1475705 1641 /var/log/messages 2023-11-09T03:56:15.022 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5121 in:imjour root 9r REG 0,24 16777216 58 /run/log/journal/15949e3c88704300882482eddfd7baae/system.journal 2023-11-09T03:56:15.022 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.022 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.022 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root txt REG 8,1 769928 12237 /usr/sbin/rsyslogd 2023-11-09T03:56:15.022 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root mem REG 0,24 16777216 58 /run/log/journal/15949e3c88704300882482eddfd7baae/system.journal 2023-11-09T03:56:15.023 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root mem REG 8,1 33592 12207 /usr/lib64/rsyslog/imjournal.so 2023-11-09T03:56:15.023 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:15.023 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.023 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:15.023 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:15.024 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.024 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root mem REG 8,1 42248 12214 /usr/lib64/rsyslog/imuxsock.so 2023-11-09T03:56:15.024 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root mem REG 8,1 32808 12215 /usr/lib64/rsyslog/lmnet.so 2023-11-09T03:56:15.024 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:15.024 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:15.024 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.025 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:15.025 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root mem REG 8,1 36328 9712 /usr/lib64/libuuid.so.1.3.0 2023-11-09T03:56:15.025 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-11-09T03:56:15.025 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root mem REG 8,1 52856 12201 /usr/lib64/libfastjson.so.4.3.0 2023-11-09T03:56:15.025 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root mem REG 8,1 16032 12200 /usr/lib64/libestr.so.0.0.0 2023-11-09T03:56:15.025 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.026 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.026 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.026 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root 1w CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.026 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root 2w CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.026 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root 3r CHR 1,9 0t0 9 /dev/urandom 2023-11-09T03:56:15.026 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root 4u unix 0xffff959a4837c840 0t0 27753 type=DGRAM (CONNECTED) 2023-11-09T03:56:15.027 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root 5w REG 8,1 1151277 1642 /var/log/secure 2023-11-09T03:56:15.027 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root 6r a_inode 0,14 0 2077 inotify 2023-11-09T03:56:15.027 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root 8w REG 8,1 1475705 1641 /var/log/messages 2023-11-09T03:56:15.027 INFO:teuthology.orchestra.run.smithi076.stdout:rsyslogd 5119 5122 rs:main root 9r REG 0,24 16777216 58 /run/log/journal/15949e3c88704300882482eddfd7baae/system.journal 2023-11-09T03:56:15.027 INFO:teuthology.orchestra.run.smithi076.stdout:tls-strp 60996 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.027 INFO:teuthology.orchestra.run.smithi076.stdout:tls-strp 60996 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.028 INFO:teuthology.orchestra.run.smithi076.stdout:tls-strp 60996 root txt unknown /proc/60996/exe 2023-11-09T03:56:15.028 INFO:teuthology.orchestra.run.smithi076.stdout:kdmflush/ 70282 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.028 INFO:teuthology.orchestra.run.smithi076.stdout:kdmflush/ 70282 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.028 INFO:teuthology.orchestra.run.smithi076.stdout:kdmflush/ 70282 root txt unknown /proc/70282/exe 2023-11-09T03:56:15.028 INFO:teuthology.orchestra.run.smithi076.stdout:kdmflush/ 70402 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.028 INFO:teuthology.orchestra.run.smithi076.stdout:kdmflush/ 70402 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.029 INFO:teuthology.orchestra.run.smithi076.stdout:kdmflush/ 70402 root txt unknown /proc/70402/exe 2023-11-09T03:56:15.029 INFO:teuthology.orchestra.run.smithi076.stdout:kdmflush/ 70522 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.029 INFO:teuthology.orchestra.run.smithi076.stdout:kdmflush/ 70522 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.029 INFO:teuthology.orchestra.run.smithi076.stdout:kdmflush/ 70522 root txt unknown /proc/70522/exe 2023-11-09T03:56:15.029 INFO:teuthology.orchestra.run.smithi076.stdout:kdmflush/ 70642 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.029 INFO:teuthology.orchestra.run.smithi076.stdout:kdmflush/ 70642 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.030 INFO:teuthology.orchestra.run.smithi076.stdout:kdmflush/ 70642 root txt unknown /proc/70642/exe 2023-11-09T03:56:15.030 INFO:teuthology.orchestra.run.smithi076.stdout:kdmflush/ 70762 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.030 INFO:teuthology.orchestra.run.smithi076.stdout:kdmflush/ 70762 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.030 INFO:teuthology.orchestra.run.smithi076.stdout:kdmflush/ 70762 root txt unknown /proc/70762/exe 2023-11-09T03:56:15.030 INFO:teuthology.orchestra.run.smithi076.stdout:xfsalloc 71655 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.030 INFO:teuthology.orchestra.run.smithi076.stdout:xfsalloc 71655 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.031 INFO:teuthology.orchestra.run.smithi076.stdout:xfsalloc 71655 root txt unknown /proc/71655/exe 2023-11-09T03:56:15.031 INFO:teuthology.orchestra.run.smithi076.stdout:xfs_mru_c 71656 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.031 INFO:teuthology.orchestra.run.smithi076.stdout:xfs_mru_c 71656 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.031 INFO:teuthology.orchestra.run.smithi076.stdout:xfs_mru_c 71656 root txt unknown /proc/71656/exe 2023-11-09T03:56:15.031 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-buf/d 71657 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.031 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-buf/d 71657 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.032 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-buf/d 71657 root txt unknown /proc/71657/exe 2023-11-09T03:56:15.032 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-conv/ 71658 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.032 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-conv/ 71658 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.032 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-conv/ 71658 root txt unknown /proc/71658/exe 2023-11-09T03:56:15.032 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-recla 71659 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.033 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-recla 71659 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.033 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-recla 71659 root txt unknown /proc/71659/exe 2023-11-09T03:56:15.033 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-block 71660 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.033 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-block 71660 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.033 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-block 71660 root txt unknown /proc/71660/exe 2023-11-09T03:56:15.033 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-inode 71661 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.034 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-inode 71661 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.034 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-inode 71661 root txt unknown /proc/71661/exe 2023-11-09T03:56:15.034 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-log/d 71662 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.034 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-log/d 71662 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.034 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-log/d 71662 root txt unknown /proc/71662/exe 2023-11-09T03:56:15.034 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-cil/d 71663 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.035 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-cil/d 71663 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.035 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-cil/d 71663 root txt unknown /proc/71663/exe 2023-11-09T03:56:15.035 INFO:teuthology.orchestra.run.smithi076.stdout:xfsaild/d 71664 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.035 INFO:teuthology.orchestra.run.smithi076.stdout:xfsaild/d 71664 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.035 INFO:teuthology.orchestra.run.smithi076.stdout:xfsaild/d 71664 root txt unknown /proc/71664/exe 2023-11-09T03:56:15.035 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.036 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.036 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root txt REG 8,1 968800 13744 /usr/sbin/sshd 2023-11-09T03:56:15.036 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 357408 10120 /usr/lib64/libnss_systemd.so.2 2023-11-09T03:56:15.036 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 19520 32283 /usr/lib64/security/pam_lastlog.so 2023-11-09T03:56:15.036 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 15184 32309 /usr/lib64/security/pam_umask.so 2023-11-09T03:56:15.036 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:15.037 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-11-09T03:56:15.037 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 19600 10967 /usr/lib64/libpam_misc.so.0.82.1 2023-11-09T03:56:15.037 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 19400 32305 /usr/lib64/security/pam_succeed_if.so 2023-11-09T03:56:15.037 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 514280 32422 /usr/lib64/security/pam_systemd.so 2023-11-09T03:56:15.037 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 27536 32284 /usr/lib64/security/pam_limits.so 2023-11-09T03:56:15.037 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 15264 32290 /usr/lib64/security/pam_motd.so 2023-11-09T03:56:15.038 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 15352 32282 /usr/lib64/security/pam_keyinit.so 2023-11-09T03:56:15.038 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 44152 32291 /usr/lib64/security/pam_namespace.so 2023-11-09T03:56:15.038 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 27616 32299 /usr/lib64/security/pam_selinux.so 2023-11-09T03:56:15.038 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 48392 10228 /usr/lib64/libcrack.so.2.9.0 2023-11-09T03:56:15.038 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 32608 10961 /usr/lib64/libpwquality.so.1.0.2 2023-11-09T03:56:15.038 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 15240 32287 /usr/lib64/security/pam_loginuid.so 2023-11-09T03:56:15.039 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 15840 32264 /usr/lib64/security/pam_pwquality.so 2023-11-09T03:56:15.039 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 15232 32292 /usr/lib64/security/pam_nologin.so 2023-11-09T03:56:15.039 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 19368 32301 /usr/lib64/security/pam_sepermit.so 2023-11-09T03:56:15.039 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 14928 32270 /usr/lib64/security/pam_deny.so 2023-11-09T03:56:15.039 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 56712 32310 /usr/lib64/security/pam_unix.so 2023-11-09T03:56:15.039 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 15184 32274 /usr/lib64/security/pam_faildelay.so 2023-11-09T03:56:15.040 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:15.040 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.040 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:15.040 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:15.040 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.040 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.041 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.041 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.041 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.041 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.041 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.042 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:15.042 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:15.042 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-11-09T03:56:15.042 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:15.043 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.043 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.043 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.043 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.043 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.044 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.044 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.044 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.044 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-11-09T03:56:15.044 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-11-09T03:56:15.044 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:15.045 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 19464 32272 /usr/lib64/security/pam_env.so 2023-11-09T03:56:15.045 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.045 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root mem REG 0,19 0 19 /sys/fs/selinux/status 2023-11-09T03:56:15.045 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root 0u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.045 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root 1u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.046 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root 2u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.046 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root 3u unix 0xffff959a43258880 0t0 161264 type=DGRAM (CONNECTED) 2023-11-09T03:56:15.046 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root 4u IPv4 163924 0t0 TCP smithi076.front.sepia.ceph.com:ssh->teuthology.front.sepia.ceph.com:46308 (ESTABLISHED) 2023-11-09T03:56:15.046 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root 6u unix 0xffff959a43260880 0t0 161239 type=STREAM (CONNECTED) 2023-11-09T03:56:15.046 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root 7w FIFO 0,24 0t0 1604 /run/systemd/sessions/4.ref 2023-11-09T03:56:15.047 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74686 root 8u unix 0xffff959a43263740 0t0 161268 type=STREAM (CONNECTED) 2023-11-09T03:56:15.047 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.047 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.047 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu txt REG 8,1 968800 13744 /usr/sbin/sshd 2023-11-09T03:56:15.047 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 357408 10120 /usr/lib64/libnss_systemd.so.2 2023-11-09T03:56:15.047 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 19520 32283 /usr/lib64/security/pam_lastlog.so 2023-11-09T03:56:15.048 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 15184 32309 /usr/lib64/security/pam_umask.so 2023-11-09T03:56:15.048 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:15.048 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-11-09T03:56:15.048 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 19600 10967 /usr/lib64/libpam_misc.so.0.82.1 2023-11-09T03:56:15.048 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 19400 32305 /usr/lib64/security/pam_succeed_if.so 2023-11-09T03:56:15.049 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 514280 32422 /usr/lib64/security/pam_systemd.so 2023-11-09T03:56:15.049 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 27536 32284 /usr/lib64/security/pam_limits.so 2023-11-09T03:56:15.049 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 15264 32290 /usr/lib64/security/pam_motd.so 2023-11-09T03:56:15.049 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 15352 32282 /usr/lib64/security/pam_keyinit.so 2023-11-09T03:56:15.049 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 44152 32291 /usr/lib64/security/pam_namespace.so 2023-11-09T03:56:15.050 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 27616 32299 /usr/lib64/security/pam_selinux.so 2023-11-09T03:56:15.050 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 48392 10228 /usr/lib64/libcrack.so.2.9.0 2023-11-09T03:56:15.050 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 32608 10961 /usr/lib64/libpwquality.so.1.0.2 2023-11-09T03:56:15.050 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 15240 32287 /usr/lib64/security/pam_loginuid.so 2023-11-09T03:56:15.050 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 15840 32264 /usr/lib64/security/pam_pwquality.so 2023-11-09T03:56:15.051 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 15232 32292 /usr/lib64/security/pam_nologin.so 2023-11-09T03:56:15.051 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 19368 32301 /usr/lib64/security/pam_sepermit.so 2023-11-09T03:56:15.051 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 14928 32270 /usr/lib64/security/pam_deny.so 2023-11-09T03:56:15.051 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 56712 32310 /usr/lib64/security/pam_unix.so 2023-11-09T03:56:15.051 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 15184 32274 /usr/lib64/security/pam_faildelay.so 2023-11-09T03:56:15.051 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-11-09T03:56:15.052 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.052 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-11-09T03:56:15.052 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-11-09T03:56:15.052 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.052 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.053 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.053 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.053 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.053 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.053 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.053 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:15.054 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:15.054 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-11-09T03:56:15.054 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:15.054 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.054 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.055 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.055 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.055 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.055 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.055 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.055 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.056 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-11-09T03:56:15.056 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-11-09T03:56:15.056 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:15.056 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 19464 32272 /usr/lib64/security/pam_env.so 2023-11-09T03:56:15.056 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.057 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu mem REG 0,19 0 19 /sys/fs/selinux/status 2023-11-09T03:56:15.057 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 0u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.057 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 1u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.057 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 2u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.057 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 3u unix 0xffff959a43258880 0t0 161264 type=DGRAM (CONNECTED) 2023-11-09T03:56:15.057 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 4u IPv4 163924 0t0 TCP smithi076.front.sepia.ceph.com:ssh->teuthology.front.sepia.ceph.com:46308 (ESTABLISHED) 2023-11-09T03:56:15.058 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 5u unix 0xffff959a43260440 0t0 161267 type=STREAM (CONNECTED) 2023-11-09T03:56:15.058 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 6u unix 0xffff959a43260880 0t0 161239 type=STREAM (CONNECTED) 2023-11-09T03:56:15.058 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 7w FIFO 0,24 0t0 1604 /run/systemd/sessions/4.ref 2023-11-09T03:56:15.058 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 11w FIFO 0,13 0t0 180521 pipe 2023-11-09T03:56:15.058 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 12r FIFO 0,13 0t0 180522 pipe 2023-11-09T03:56:15.058 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 13w FIFO 0,13 0t0 180529 pipe 2023-11-09T03:56:15.059 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 14r FIFO 0,13 0t0 180523 pipe 2023-11-09T03:56:15.059 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 15r FIFO 0,13 0t0 180530 pipe 2023-11-09T03:56:15.059 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 16w FIFO 0,13 0t0 182400 pipe 2023-11-09T03:56:15.059 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 17r FIFO 0,13 0t0 180531 pipe 2023-11-09T03:56:15.059 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 18r FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:15.060 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 19w FIFO 0,13 0t0 182404 pipe 2023-11-09T03:56:15.060 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 20r FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:15.060 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 21r FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:15.060 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 22w FIFO 0,13 0t0 182408 pipe 2023-11-09T03:56:15.060 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 23r FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:15.060 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 24r FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:15.061 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 25w FIFO 0,13 0t0 183332 pipe 2023-11-09T03:56:15.061 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 26r FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:15.061 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 27r FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:15.061 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 28w FIFO 0,13 0t0 248572 pipe 2023-11-09T03:56:15.061 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 29r FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:15.061 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 30r FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:15.062 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 31w FIFO 0,13 0t0 265997 pipe 2023-11-09T03:56:15.062 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 32r FIFO 0,13 0t0 265998 pipe 2023-11-09T03:56:15.062 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 33w FIFO 0,13 0t0 277656 pipe 2023-11-09T03:56:15.062 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 34r FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:15.062 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 35r FIFO 0,13 0t0 277657 pipe 2023-11-09T03:56:15.062 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 36r FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:15.063 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 37r FIFO 0,13 0t0 265999 pipe 2023-11-09T03:56:15.063 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 38r FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:15.063 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 39r FIFO 0,13 0t0 272767 pipe 2023-11-09T03:56:15.063 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 41r FIFO 0,13 0t0 272768 pipe 2023-11-09T03:56:15.063 INFO:teuthology.orchestra.run.smithi076.stdout:sshd 74689 ubuntu 42r FIFO 0,13 0t0 277658 pipe 2023-11-09T03:56:15.063 INFO:teuthology.orchestra.run.smithi076.stdout:chronyd 74749 chrony cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.064 INFO:teuthology.orchestra.run.smithi076.stdout:chronyd 74749 chrony rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.064 INFO:teuthology.orchestra.run.smithi076.stdout:chronyd 74749 chrony txt REG 8,1 353408 12490 /usr/sbin/chronyd 2023-11-09T03:56:15.064 INFO:teuthology.orchestra.run.smithi076.stdout:chronyd 74749 chrony mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.064 INFO:teuthology.orchestra.run.smithi076.stdout:chronyd 74749 chrony mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.064 INFO:teuthology.orchestra.run.smithi076.stdout:chronyd 74749 chrony mem REG 8,1 626152 9989 /usr/lib64/libhogweed.so.6.5 2023-11-09T03:56:15.065 INFO:teuthology.orchestra.run.smithi076.stdout:chronyd 74749 chrony mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-11-09T03:56:15.065 INFO:teuthology.orchestra.run.smithi076.stdout:chronyd 74749 chrony mem REG 8,1 2291576 9994 /usr/lib64/libgnutls.so.30.33.1 2023-11-09T03:56:15.065 INFO:teuthology.orchestra.run.smithi076.stdout:chronyd 74749 chrony mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:15.065 INFO:teuthology.orchestra.run.smithi076.stdout:chronyd 74749 chrony mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-11-09T03:56:15.065 INFO:teuthology.orchestra.run.smithi076.stdout:chronyd 74749 chrony mem REG 8,1 94112 9901 /usr/lib64/libtasn1.so.6.6.0 2023-11-09T03:56:15.065 INFO:teuthology.orchestra.run.smithi076.stdout:chronyd 74749 chrony mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.066 INFO:teuthology.orchestra.run.smithi076.stdout:chronyd 74749 chrony mem REG 8,1 127680 9972 /usr/lib64/libseccomp.so.2.5.2 2023-11-09T03:56:15.066 INFO:teuthology.orchestra.run.smithi076.stdout:chronyd 74749 chrony mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:15.066 INFO:teuthology.orchestra.run.smithi076.stdout:chronyd 74749 chrony mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.066 INFO:teuthology.orchestra.run.smithi076.stdout:chronyd 74749 chrony mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.066 INFO:teuthology.orchestra.run.smithi076.stdout:chronyd 74749 chrony 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.066 INFO:teuthology.orchestra.run.smithi076.stdout:chronyd 74749 chrony 1w CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.067 INFO:teuthology.orchestra.run.smithi076.stdout:chronyd 74749 chrony 2u CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.067 INFO:teuthology.orchestra.run.smithi076.stdout:chronyd 74749 chrony 3u unix 0xffff959a600f72c0 0t0 163174 type=DGRAM (CONNECTED) 2023-11-09T03:56:15.067 INFO:teuthology.orchestra.run.smithi076.stdout:chronyd 74749 chrony 5u IPv4 163179 0t0 UDP localhost:323 2023-11-09T03:56:15.067 INFO:teuthology.orchestra.run.smithi076.stdout:chronyd 74749 chrony 6u IPv6 163180 0t0 UDP localhost:323 2023-11-09T03:56:15.067 INFO:teuthology.orchestra.run.smithi076.stdout:chronyd 74749 chrony 7u unix 0xffff959a600f3740 0t0 163181 /run/chrony/chronyd.sock type=DGRAM (CONNECTED) 2023-11-09T03:56:15.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-cras 74992 ceph cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-cras 74992 ceph rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-cras 74992 ceph txt REG 8,1 15448 10498 /usr/bin/python3.9 2023-11-09T03:56:15.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-cras 74992 ceph mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:15.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-cras 74992 ceph mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-cras 74992 ceph mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:15.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-cras 74992 ceph mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-cras 74992 ceph mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-cras 74992 ceph mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-cras 74992 ceph mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-cras 74992 ceph mem REG 8,1 68000 38241 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-cras 74992 ceph mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-cras 74992 ceph mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-cras 74992 ceph mem REG 8,1 101208 41993 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-cras 74992 ceph mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-cras 74992 ceph mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-cras 74992 ceph mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:15.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-cras 74992 ceph mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-cras 74992 ceph mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:15.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-cras 74992 ceph mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-cras 74992 ceph mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:15.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-cras 74992 ceph mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-cras 74992 ceph 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-cras 74992 ceph 1u unix 0xffff959a6032f700 0t0 165085 type=STREAM (CONNECTED) 2023-11-09T03:56:15.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-cras 74992 ceph 2u unix 0xffff959a6032f700 0t0 165085 type=STREAM (CONNECTED) 2023-11-09T03:56:15.072 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/u 75669 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.072 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/u 75669 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.072 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/u 75669 root txt unknown /proc/75669/exe 2023-11-09T03:56:15.072 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/3 76435 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.072 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/3 76435 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.073 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/3 76435 root txt unknown /proc/76435/exe 2023-11-09T03:56:15.073 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/2 76857 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.073 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/2 76857 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.073 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/2 76857 root txt unknown /proc/76857/exe 2023-11-09T03:56:15.073 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/2 77087 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.074 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/2 77087 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.074 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/2 77087 root txt unknown /proc/77087/exe 2023-11-09T03:56:15.074 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-buf/d 82091 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.074 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-buf/d 82091 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.074 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-buf/d 82091 root txt unknown /proc/82091/exe 2023-11-09T03:56:15.075 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-conv/ 82092 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.075 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-conv/ 82092 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.075 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-conv/ 82092 root txt unknown /proc/82092/exe 2023-11-09T03:56:15.075 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-recla 82093 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.075 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-recla 82093 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.075 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-recla 82093 root txt unknown /proc/82093/exe 2023-11-09T03:56:15.076 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-block 82094 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.076 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-block 82094 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.076 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-block 82094 root txt unknown /proc/82094/exe 2023-11-09T03:56:15.076 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-inode 82095 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.076 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-inode 82095 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.076 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-inode 82095 root txt unknown /proc/82095/exe 2023-11-09T03:56:15.077 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-log/d 82096 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.077 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-log/d 82096 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.077 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-log/d 82096 root txt unknown /proc/82096/exe 2023-11-09T03:56:15.077 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-cil/d 82097 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.077 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-cil/d 82097 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.077 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-cil/d 82097 root txt unknown /proc/82097/exe 2023-11-09T03:56:15.078 INFO:teuthology.orchestra.run.smithi076.stdout:xfsaild/d 82098 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.078 INFO:teuthology.orchestra.run.smithi076.stdout:xfsaild/d 82098 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.078 INFO:teuthology.orchestra.run.smithi076.stdout:xfsaild/d 82098 root txt unknown /proc/82098/exe 2023-11-09T03:56:15.078 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-buf/d 82175 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.078 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-buf/d 82175 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.078 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-buf/d 82175 root txt unknown /proc/82175/exe 2023-11-09T03:56:15.079 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-conv/ 82176 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.079 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-conv/ 82176 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.079 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-conv/ 82176 root txt unknown /proc/82176/exe 2023-11-09T03:56:15.079 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-recla 82177 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.079 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-recla 82177 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.079 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-recla 82177 root txt unknown /proc/82177/exe 2023-11-09T03:56:15.080 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-block 82178 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.080 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-block 82178 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.080 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-block 82178 root txt unknown /proc/82178/exe 2023-11-09T03:56:15.080 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-inode 82179 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.080 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-inode 82179 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.080 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-inode 82179 root txt unknown /proc/82179/exe 2023-11-09T03:56:15.081 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-log/d 82180 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.081 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-log/d 82180 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.081 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-log/d 82180 root txt unknown /proc/82180/exe 2023-11-09T03:56:15.081 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-cil/d 82181 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.081 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-cil/d 82181 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.081 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-cil/d 82181 root txt unknown /proc/82181/exe 2023-11-09T03:56:15.081 INFO:teuthology.orchestra.run.smithi076.stdout:xfsaild/d 82182 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.082 INFO:teuthology.orchestra.run.smithi076.stdout:xfsaild/d 82182 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.082 INFO:teuthology.orchestra.run.smithi076.stdout:xfsaild/d 82182 root txt unknown /proc/82182/exe 2023-11-09T03:56:15.082 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-buf/d 82259 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.082 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-buf/d 82259 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.082 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-buf/d 82259 root txt unknown /proc/82259/exe 2023-11-09T03:56:15.082 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-conv/ 82260 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.083 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-conv/ 82260 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.083 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-conv/ 82260 root txt unknown /proc/82260/exe 2023-11-09T03:56:15.083 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-recla 82261 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.083 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-recla 82261 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.083 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-recla 82261 root txt unknown /proc/82261/exe 2023-11-09T03:56:15.084 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-block 82262 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.084 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-block 82262 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.084 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-block 82262 root txt unknown /proc/82262/exe 2023-11-09T03:56:15.084 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-inode 82263 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.084 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-inode 82263 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.084 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-inode 82263 root txt unknown /proc/82263/exe 2023-11-09T03:56:15.084 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-log/d 82264 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.085 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-log/d 82264 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.085 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-log/d 82264 root txt unknown /proc/82264/exe 2023-11-09T03:56:15.085 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-cil/d 82265 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.085 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-cil/d 82265 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.085 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-cil/d 82265 root txt unknown /proc/82265/exe 2023-11-09T03:56:15.085 INFO:teuthology.orchestra.run.smithi076.stdout:xfsaild/d 82266 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.086 INFO:teuthology.orchestra.run.smithi076.stdout:xfsaild/d 82266 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.086 INFO:teuthology.orchestra.run.smithi076.stdout:xfsaild/d 82266 root txt unknown /proc/82266/exe 2023-11-09T03:56:15.086 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-buf/d 82343 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.086 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-buf/d 82343 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.086 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-buf/d 82343 root txt unknown /proc/82343/exe 2023-11-09T03:56:15.086 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-conv/ 82344 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.087 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-conv/ 82344 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.087 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-conv/ 82344 root txt unknown /proc/82344/exe 2023-11-09T03:56:15.087 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-recla 82345 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.087 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-recla 82345 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.087 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-recla 82345 root txt unknown /proc/82345/exe 2023-11-09T03:56:15.087 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-block 82346 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.088 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-block 82346 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.088 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-block 82346 root txt unknown /proc/82346/exe 2023-11-09T03:56:15.088 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-inode 82347 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.088 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-inode 82347 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.088 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-inode 82347 root txt unknown /proc/82347/exe 2023-11-09T03:56:15.088 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-log/d 82348 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.089 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-log/d 82348 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.089 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-log/d 82348 root txt unknown /proc/82348/exe 2023-11-09T03:56:15.089 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-cil/d 82349 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.089 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-cil/d 82349 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.089 INFO:teuthology.orchestra.run.smithi076.stdout:xfs-cil/d 82349 root txt unknown /proc/82349/exe 2023-11-09T03:56:15.090 INFO:teuthology.orchestra.run.smithi076.stdout:xfsaild/d 82350 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.090 INFO:teuthology.orchestra.run.smithi076.stdout:xfsaild/d 82350 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.090 INFO:teuthology.orchestra.run.smithi076.stdout:xfsaild/d 82350 root txt unknown /proc/82350/exe 2023-11-09T03:56:15.090 INFO:teuthology.orchestra.run.smithi076.stdout:dio/dm-0 82396 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.090 INFO:teuthology.orchestra.run.smithi076.stdout:dio/dm-0 82396 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.090 INFO:teuthology.orchestra.run.smithi076.stdout:dio/dm-0 82396 root txt unknown /proc/82396/exe 2023-11-09T03:56:15.091 INFO:teuthology.orchestra.run.smithi076.stdout:dio/dm-1 83558 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.091 INFO:teuthology.orchestra.run.smithi076.stdout:dio/dm-1 83558 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.091 INFO:teuthology.orchestra.run.smithi076.stdout:dio/dm-1 83558 root txt unknown /proc/83558/exe 2023-11-09T03:56:15.091 INFO:teuthology.orchestra.run.smithi076.stdout:dio/dm-2 84720 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.091 INFO:teuthology.orchestra.run.smithi076.stdout:dio/dm-2 84720 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.091 INFO:teuthology.orchestra.run.smithi076.stdout:dio/dm-2 84720 root txt unknown /proc/84720/exe 2023-11-09T03:56:15.092 INFO:teuthology.orchestra.run.smithi076.stdout:dio/dm-3 85882 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:15.092 INFO:teuthology.orchestra.run.smithi076.stdout:dio/dm-3 85882 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.092 INFO:teuthology.orchestra.run.smithi076.stdout:dio/dm-3 85882 root txt unknown /proc/85882/exe 2023-11-09T03:56:15.092 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.092 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.092 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root txt REG 8,1 185296 11759 /usr/bin/sudo 2023-11-09T03:56:15.093 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 357408 10120 /usr/lib64/libnss_systemd.so.2 2023-11-09T03:56:15.093 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:15.093 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.093 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-11-09T03:56:15.093 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:15.093 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 19600 10967 /usr/lib64/libpam_misc.so.0.82.1 2023-11-09T03:56:15.094 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 19400 32305 /usr/lib64/security/pam_succeed_if.so 2023-11-09T03:56:15.094 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 514280 32422 /usr/lib64/security/pam_systemd.so 2023-11-09T03:56:15.094 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 27536 32284 /usr/lib64/security/pam_limits.so 2023-11-09T03:56:15.094 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 48392 10228 /usr/lib64/libcrack.so.2.9.0 2023-11-09T03:56:15.094 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 32608 10961 /usr/lib64/libpwquality.so.1.0.2 2023-11-09T03:56:15.094 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 15352 32282 /usr/lib64/security/pam_keyinit.so 2023-11-09T03:56:15.095 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 15840 32264 /usr/lib64/security/pam_pwquality.so 2023-11-09T03:56:15.095 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 56712 32310 /usr/lib64/security/pam_unix.so 2023-11-09T03:56:15.095 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.095 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.095 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.095 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.096 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.096 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.096 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.096 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.096 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.096 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.097 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.097 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.097 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.097 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.097 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.097 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-11-09T03:56:15.098 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 14928 32270 /usr/lib64/security/pam_deny.so 2023-11-09T03:56:15.098 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-11-09T03:56:15.098 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.098 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 472568 54513 /usr/libexec/sudo/sudoers.so 2023-11-09T03:56:15.098 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.099 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.099 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 15184 32274 /usr/lib64/security/pam_faildelay.so 2023-11-09T03:56:15.099 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-11-09T03:56:15.099 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 19464 32272 /usr/lib64/security/pam_env.so 2023-11-09T03:56:15.099 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:15.099 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-11-09T03:56:15.100 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-11-09T03:56:15.100 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-11-09T03:56:15.100 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-11-09T03:56:15.100 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-11-09T03:56:15.100 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-11-09T03:56:15.100 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-11-09T03:56:15.101 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-11-09T03:56:15.101 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-11-09T03:56:15.101 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:15.101 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 119832 54509 /usr/libexec/sudo/libsudo_util.so.0.0.0 2023-11-09T03:56:15.101 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.101 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:15.102 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:15.102 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.102 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-11-09T03:56:15.102 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root 0r FIFO 0,13 0t0 180521 pipe 2023-11-09T03:56:15.102 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root 1w FIFO 0,13 0t0 180522 pipe 2023-11-09T03:56:15.102 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root 2w FIFO 0,13 0t0 180523 pipe 2023-11-09T03:56:15.103 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root 3r FIFO 0,13 0t0 177842 pipe 2023-11-09T03:56:15.103 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root 4w FIFO 0,13 0t0 177842 pipe 2023-11-09T03:56:15.103 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root 5u netlink 0t0 177847 AUDIT 2023-11-09T03:56:15.103 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87693 root 6u unix 0xffff959a619772c0 0t0 177848 type=DGRAM (CONNECTED) 2023-11-09T03:56:15.103 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87710 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.104 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87710 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.104 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87710 root txt REG 8,1 15448 10498 /usr/bin/python3.9 2023-11-09T03:56:15.104 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87710 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:15.104 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87710 root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:15.105 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87710 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.105 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87710 root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.105 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87710 root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.105 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87710 root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.106 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87710 root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.106 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87710 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.106 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87710 root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.106 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87710 root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.106 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87710 root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.107 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87710 root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.107 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87710 root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.107 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87710 root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:15.107 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87710 root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.107 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87710 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:15.108 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87710 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.108 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87710 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:15.108 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87710 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.108 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87710 root 0r FIFO 0,13 0t0 180521 pipe 2023-11-09T03:56:15.108 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87710 root 1w FIFO 0,13 0t0 180522 pipe 2023-11-09T03:56:15.109 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87710 root 2w FIFO 0,13 0t0 180523 pipe 2023-11-09T03:56:15.109 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.109 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.109 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root txt REG 8,1 185296 11759 /usr/bin/sudo 2023-11-09T03:56:15.110 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 357408 10120 /usr/lib64/libnss_systemd.so.2 2023-11-09T03:56:15.110 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:15.110 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.111 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-11-09T03:56:15.111 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:15.111 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 19600 10967 /usr/lib64/libpam_misc.so.0.82.1 2023-11-09T03:56:15.112 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 19400 32305 /usr/lib64/security/pam_succeed_if.so 2023-11-09T03:56:15.112 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 514280 32422 /usr/lib64/security/pam_systemd.so 2023-11-09T03:56:15.112 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 27536 32284 /usr/lib64/security/pam_limits.so 2023-11-09T03:56:15.112 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 48392 10228 /usr/lib64/libcrack.so.2.9.0 2023-11-09T03:56:15.113 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 32608 10961 /usr/lib64/libpwquality.so.1.0.2 2023-11-09T03:56:15.113 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 15352 32282 /usr/lib64/security/pam_keyinit.so 2023-11-09T03:56:15.113 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 15840 32264 /usr/lib64/security/pam_pwquality.so 2023-11-09T03:56:15.113 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 14928 32270 /usr/lib64/security/pam_deny.so 2023-11-09T03:56:15.113 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 56712 32310 /usr/lib64/security/pam_unix.so 2023-11-09T03:56:15.113 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 15184 32274 /usr/lib64/security/pam_faildelay.so 2023-11-09T03:56:15.114 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.114 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.114 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.114 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.115 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.115 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.115 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.115 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.116 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.116 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.116 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.116 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.116 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.117 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-11-09T03:56:15.117 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.117 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.117 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 472568 54513 /usr/libexec/sudo/sudoers.so 2023-11-09T03:56:15.118 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:15.118 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.118 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.118 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-11-09T03:56:15.119 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.119 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-11-09T03:56:15.119 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 19464 32272 /usr/lib64/security/pam_env.so 2023-11-09T03:56:15.119 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-11-09T03:56:15.120 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-11-09T03:56:15.120 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-11-09T03:56:15.120 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-11-09T03:56:15.120 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-11-09T03:56:15.120 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-11-09T03:56:15.121 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-11-09T03:56:15.121 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-11-09T03:56:15.121 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-11-09T03:56:15.121 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:15.121 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 119832 54509 /usr/libexec/sudo/libsudo_util.so.0.0.0 2023-11-09T03:56:15.122 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.122 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:15.122 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:15.122 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.123 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-11-09T03:56:15.123 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root 0r FIFO 0,13 0t0 180529 pipe 2023-11-09T03:56:15.123 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root 1w FIFO 0,13 0t0 180530 pipe 2023-11-09T03:56:15.123 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root 2w FIFO 0,13 0t0 180531 pipe 2023-11-09T03:56:15.123 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root 3r FIFO 0,13 0t0 180539 pipe 2023-11-09T03:56:15.124 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root 4w FIFO 0,13 0t0 180539 pipe 2023-11-09T03:56:15.124 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root 5u netlink 0t0 180544 AUDIT 2023-11-09T03:56:15.124 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87712 root 6u unix 0xffff959a60323740 0t0 180545 type=DGRAM (CONNECTED) 2023-11-09T03:56:15.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root txt REG 8,1 12250256 3112 /usr/bin/ceph-mon 2023-11-09T03:56:15.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:15.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:15.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:15.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 1w FIFO 0,13 0t0 180522 pipe 2023-11-09T03:56:15.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 2w FIFO 0,13 0t0 180523 pipe 2023-11-09T03:56:15.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 3r FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 4w FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 5w REG 8,1 68063672 265319 /var/log/ceph/ceph-mon.a.log 2023-11-09T03:56:15.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 6uW REG 0,24 6 1732 /run/ceph/ceph-mon.a.pid 2023-11-09T03:56:15.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 7r FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 8w FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 9u unix 0xffff959a6034b740 0t0 177870 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 10r FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 11w FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 12r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 13uW REG 253,4 0 33816711 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2023-11-09T03:56:15.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 14r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 15u IPv4 250312 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60028 (ESTABLISHED) 2023-11-09T03:56:15.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 16w REG 253,4 1820304 33816715 /var/lib/ceph/mon/ceph-a/store.db/000019.log 2023-11-09T03:56:15.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 17w REG 253,4 898 33816719 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-09T03:56:15.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 18u a_inode 0,14 0 2077 [eventpoll:19,27,28,29,41,44,47] 2023-11-09T03:56:15.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 19r FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 20w FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 21u a_inode 0,14 0 2077 [eventpoll:15,22,38,42] 2023-11-09T03:56:15.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 22r FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 23w FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 24u a_inode 0,14 0 2077 [eventpoll:25,32,40,43,46] 2023-11-09T03:56:15.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 25r FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 26w FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 27u IPv4 178242 0t0 TCP smithi076.front.sepia.ceph.com:3300 (LISTEN) 2023-11-09T03:56:15.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 28u IPv4 178243 0t0 TCP smithi076.front.sepia.ceph.com:smc-https (LISTEN) 2023-11-09T03:56:15.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 29u IPv4 180558 0t0 TCP smithi076.front.sepia.ceph.com:52700->smithi113.front.sepia.ceph.com:3301 (ESTABLISHED) 2023-11-09T03:56:15.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 30r FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 31w FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 32u IPv4 176127 0t0 TCP smithi076.front.sepia.ceph.com:48276->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 33r FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 34w FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 35r FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 36w FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 37w REG 8,1 125211 265320 /var/log/ceph/ceph.log 2023-11-09T03:56:15.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 38u IPv4 182390 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:46708 (ESTABLISHED) 2023-11-09T03:56:15.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 39w REG 8,1 209797 265351 /var/log/ceph/ceph.audit.log 2023-11-09T03:56:15.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 40u IPv4 183510 0t0 TCP smithi076.front.sepia.ceph.com:48946->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 41u IPv4 179814 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60322 (ESTABLISHED) 2023-11-09T03:56:15.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 42u IPv4 179820 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60330 (ESTABLISHED) 2023-11-09T03:56:15.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 43u IPv4 181612 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60338 (ESTABLISHED) 2023-11-09T03:56:15.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 44u IPv4 181618 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60350 (ESTABLISHED) 2023-11-09T03:56:15.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 45r REG 253,4 4632438 33816724 /var/lib/ceph/mon/ceph-a/store.db/000021.sst 2023-11-09T03:56:15.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 46u IPv4 267111 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:50182 (ESTABLISHED) 2023-11-09T03:56:15.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 root 47u IPv4 271047 0t0 TCP smithi076.front.sepia.ceph.com:3300->192.168.0.1:37190 (ESTABLISHED) 2023-11-09T03:56:15.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root txt REG 8,1 12250256 3112 /usr/bin/ceph-mon 2023-11-09T03:56:15.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:15.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:15.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:15.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 1w FIFO 0,13 0t0 180522 pipe 2023-11-09T03:56:15.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 2w FIFO 0,13 0t0 180523 pipe 2023-11-09T03:56:15.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 3r FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 4w FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 5w REG 8,1 68063672 265319 /var/log/ceph/ceph-mon.a.log 2023-11-09T03:56:15.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 6uW REG 0,24 6 1732 /run/ceph/ceph-mon.a.pid 2023-11-09T03:56:15.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 7r FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 8w FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 9u unix 0xffff959a6034b740 0t0 177870 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 10r FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 11w FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 12r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 13uW REG 253,4 0 33816711 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2023-11-09T03:56:15.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 14r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 15u IPv4 250312 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60028 (ESTABLISHED) 2023-11-09T03:56:15.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 16w REG 253,4 1820304 33816715 /var/lib/ceph/mon/ceph-a/store.db/000019.log 2023-11-09T03:56:15.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 17w REG 253,4 898 33816719 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-09T03:56:15.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 18u a_inode 0,14 0 2077 [eventpoll:19,27,28,29,41,44,47] 2023-11-09T03:56:15.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 19r FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 20w FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 21u a_inode 0,14 0 2077 [eventpoll:15,22,38,42] 2023-11-09T03:56:15.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 22r FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 23w FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 24u a_inode 0,14 0 2077 [eventpoll:25,32,40,43,46] 2023-11-09T03:56:15.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 25r FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 26w FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 27u IPv4 178242 0t0 TCP smithi076.front.sepia.ceph.com:3300 (LISTEN) 2023-11-09T03:56:15.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 28u IPv4 178243 0t0 TCP smithi076.front.sepia.ceph.com:smc-https (LISTEN) 2023-11-09T03:56:15.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 29u IPv4 180558 0t0 TCP smithi076.front.sepia.ceph.com:52700->smithi113.front.sepia.ceph.com:3301 (ESTABLISHED) 2023-11-09T03:56:15.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 30r FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 31w FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 32u IPv4 176127 0t0 TCP smithi076.front.sepia.ceph.com:48276->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 33r FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 34w FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 35r FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 36w FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 37w REG 8,1 125211 265320 /var/log/ceph/ceph.log 2023-11-09T03:56:15.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 38u IPv4 182390 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:46708 (ESTABLISHED) 2023-11-09T03:56:15.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 39w REG 8,1 209797 265351 /var/log/ceph/ceph.audit.log 2023-11-09T03:56:15.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 40u IPv4 183510 0t0 TCP smithi076.front.sepia.ceph.com:48946->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 41u IPv4 179814 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60322 (ESTABLISHED) 2023-11-09T03:56:15.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 42u IPv4 179820 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60330 (ESTABLISHED) 2023-11-09T03:56:15.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 43u IPv4 181612 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60338 (ESTABLISHED) 2023-11-09T03:56:15.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 44u IPv4 181618 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60350 (ESTABLISHED) 2023-11-09T03:56:15.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 45r REG 253,4 4632438 33816724 /var/lib/ceph/mon/ceph-a/store.db/000021.sst 2023-11-09T03:56:15.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 46u IPv4 267111 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:50182 (ESTABLISHED) 2023-11-09T03:56:15.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87749 log root 47u IPv4 271047 0t0 TCP smithi076.front.sepia.ceph.com:3300->192.168.0.1:37190 (ESTABLISHED) 2023-11-09T03:56:15.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root txt REG 8,1 12250256 3112 /usr/bin/ceph-mon 2023-11-09T03:56:15.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:15.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:15.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:15.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 1w FIFO 0,13 0t0 180522 pipe 2023-11-09T03:56:15.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 2w FIFO 0,13 0t0 180523 pipe 2023-11-09T03:56:15.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 3r FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 4w FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 5w REG 8,1 68063672 265319 /var/log/ceph/ceph-mon.a.log 2023-11-09T03:56:15.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 6uW REG 0,24 6 1732 /run/ceph/ceph-mon.a.pid 2023-11-09T03:56:15.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 7r FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 8w FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 9u unix 0xffff959a6034b740 0t0 177870 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 10r FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 11w FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 12r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 13uW REG 253,4 0 33816711 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2023-11-09T03:56:15.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 14r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 15u IPv4 250312 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60028 (ESTABLISHED) 2023-11-09T03:56:15.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 16w REG 253,4 1820304 33816715 /var/lib/ceph/mon/ceph-a/store.db/000019.log 2023-11-09T03:56:15.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 17w REG 253,4 898 33816719 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-09T03:56:15.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 18u a_inode 0,14 0 2077 [eventpoll:19,27,28,29,41,44,47] 2023-11-09T03:56:15.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 19r FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 20w FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 21u a_inode 0,14 0 2077 [eventpoll:15,22,38,42] 2023-11-09T03:56:15.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 22r FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 23w FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 24u a_inode 0,14 0 2077 [eventpoll:25,32,40,43,46] 2023-11-09T03:56:15.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 25r FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 26w FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 27u IPv4 178242 0t0 TCP smithi076.front.sepia.ceph.com:3300 (LISTEN) 2023-11-09T03:56:15.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 28u IPv4 178243 0t0 TCP smithi076.front.sepia.ceph.com:smc-https (LISTEN) 2023-11-09T03:56:15.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 29u IPv4 180558 0t0 TCP smithi076.front.sepia.ceph.com:52700->smithi113.front.sepia.ceph.com:3301 (ESTABLISHED) 2023-11-09T03:56:15.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 30r FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 31w FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 32u IPv4 176127 0t0 TCP smithi076.front.sepia.ceph.com:48276->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 33r FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 34w FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 35r FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 36w FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 37w REG 8,1 125211 265320 /var/log/ceph/ceph.log 2023-11-09T03:56:15.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 38u IPv4 182390 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:46708 (ESTABLISHED) 2023-11-09T03:56:15.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 39w REG 8,1 209797 265351 /var/log/ceph/ceph.audit.log 2023-11-09T03:56:15.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 40u IPv4 183510 0t0 TCP smithi076.front.sepia.ceph.com:48946->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 41u IPv4 179814 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60322 (ESTABLISHED) 2023-11-09T03:56:15.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 42u IPv4 179820 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60330 (ESTABLISHED) 2023-11-09T03:56:15.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 43u IPv4 181612 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60338 (ESTABLISHED) 2023-11-09T03:56:15.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 44u IPv4 181618 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60350 (ESTABLISHED) 2023-11-09T03:56:15.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 45r REG 253,4 4632438 33816724 /var/lib/ceph/mon/ceph-a/store.db/000021.sst 2023-11-09T03:56:15.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 46u IPv4 267111 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:50182 (ESTABLISHED) 2023-11-09T03:56:15.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87750 service root 47u IPv4 271047 0t0 TCP smithi076.front.sepia.ceph.com:3300->192.168.0.1:37190 (ESTABLISHED) 2023-11-09T03:56:15.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root txt REG 8,1 12250256 3112 /usr/bin/ceph-mon 2023-11-09T03:56:15.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:15.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:15.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:15.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 1w FIFO 0,13 0t0 180522 pipe 2023-11-09T03:56:15.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 2w FIFO 0,13 0t0 180523 pipe 2023-11-09T03:56:15.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 3r FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 4w FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 5w REG 8,1 68063672 265319 /var/log/ceph/ceph-mon.a.log 2023-11-09T03:56:15.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 6uW REG 0,24 6 1732 /run/ceph/ceph-mon.a.pid 2023-11-09T03:56:15.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 7r FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 8w FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 9u unix 0xffff959a6034b740 0t0 177870 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 10r FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 11w FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 12r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 13uW REG 253,4 0 33816711 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2023-11-09T03:56:15.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 14r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 15u IPv4 250312 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60028 (ESTABLISHED) 2023-11-09T03:56:15.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 16w REG 253,4 1820304 33816715 /var/lib/ceph/mon/ceph-a/store.db/000019.log 2023-11-09T03:56:15.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 17w REG 253,4 898 33816719 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-09T03:56:15.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 18u a_inode 0,14 0 2077 [eventpoll:19,27,28,29,41,44,47] 2023-11-09T03:56:15.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 19r FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 20w FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 21u a_inode 0,14 0 2077 [eventpoll:15,22,38,42] 2023-11-09T03:56:15.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 22r FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 23w FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 24u a_inode 0,14 0 2077 [eventpoll:25,32,40,43,46] 2023-11-09T03:56:15.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 25r FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 26w FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 27u IPv4 178242 0t0 TCP smithi076.front.sepia.ceph.com:3300 (LISTEN) 2023-11-09T03:56:15.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 28u IPv4 178243 0t0 TCP smithi076.front.sepia.ceph.com:smc-https (LISTEN) 2023-11-09T03:56:15.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 29u IPv4 180558 0t0 TCP smithi076.front.sepia.ceph.com:52700->smithi113.front.sepia.ceph.com:3301 (ESTABLISHED) 2023-11-09T03:56:15.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 30r FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 31w FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 32u IPv4 176127 0t0 TCP smithi076.front.sepia.ceph.com:48276->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 33r FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 34w FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 35r FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 36w FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 37w REG 8,1 125211 265320 /var/log/ceph/ceph.log 2023-11-09T03:56:15.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 38u IPv4 182390 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:46708 (ESTABLISHED) 2023-11-09T03:56:15.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 39w REG 8,1 209797 265351 /var/log/ceph/ceph.audit.log 2023-11-09T03:56:15.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 40u IPv4 183510 0t0 TCP smithi076.front.sepia.ceph.com:48946->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 41u IPv4 179814 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60322 (ESTABLISHED) 2023-11-09T03:56:15.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 42u IPv4 179820 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60330 (ESTABLISHED) 2023-11-09T03:56:15.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 43u IPv4 181612 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60338 (ESTABLISHED) 2023-11-09T03:56:15.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 44u IPv4 181618 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60350 (ESTABLISHED) 2023-11-09T03:56:15.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 45r REG 253,4 4632438 33816724 /var/lib/ceph/mon/ceph-a/store.db/000021.sst 2023-11-09T03:56:15.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 46u IPv4 267111 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:50182 (ESTABLISHED) 2023-11-09T03:56:15.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87751 admin_soc root 47u IPv4 271047 0t0 TCP smithi076.front.sepia.ceph.com:3300->192.168.0.1:37190 (ESTABLISHED) 2023-11-09T03:56:15.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root txt REG 8,1 12250256 3112 /usr/bin/ceph-mon 2023-11-09T03:56:15.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:15.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:15.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:15.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 1w FIFO 0,13 0t0 180522 pipe 2023-11-09T03:56:15.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 2w FIFO 0,13 0t0 180523 pipe 2023-11-09T03:56:15.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 3r FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 4w FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 5w REG 8,1 68063672 265319 /var/log/ceph/ceph-mon.a.log 2023-11-09T03:56:15.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 6uW REG 0,24 6 1732 /run/ceph/ceph-mon.a.pid 2023-11-09T03:56:15.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 7r FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 8w FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 9u unix 0xffff959a6034b740 0t0 177870 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 10r FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 11w FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 12r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 13uW REG 253,4 0 33816711 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2023-11-09T03:56:15.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 14r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 15u IPv4 250312 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60028 (ESTABLISHED) 2023-11-09T03:56:15.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 16w REG 253,4 1820304 33816715 /var/lib/ceph/mon/ceph-a/store.db/000019.log 2023-11-09T03:56:15.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 17w REG 253,4 898 33816719 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-09T03:56:15.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 18u a_inode 0,14 0 2077 [eventpoll:19,27,28,29,41,44,47] 2023-11-09T03:56:15.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 19r FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 20w FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 21u a_inode 0,14 0 2077 [eventpoll:15,22,38,42] 2023-11-09T03:56:15.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 22r FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 23w FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 24u a_inode 0,14 0 2077 [eventpoll:25,32,40,43,46] 2023-11-09T03:56:15.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 25r FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 26w FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 27u IPv4 178242 0t0 TCP smithi076.front.sepia.ceph.com:3300 (LISTEN) 2023-11-09T03:56:15.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 28u IPv4 178243 0t0 TCP smithi076.front.sepia.ceph.com:smc-https (LISTEN) 2023-11-09T03:56:15.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 29u IPv4 180558 0t0 TCP smithi076.front.sepia.ceph.com:52700->smithi113.front.sepia.ceph.com:3301 (ESTABLISHED) 2023-11-09T03:56:15.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 30r FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 31w FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 32u IPv4 176127 0t0 TCP smithi076.front.sepia.ceph.com:48276->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 33r FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 34w FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 35r FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 36w FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 37w REG 8,1 125211 265320 /var/log/ceph/ceph.log 2023-11-09T03:56:15.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 38u IPv4 182390 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:46708 (ESTABLISHED) 2023-11-09T03:56:15.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 39w REG 8,1 209797 265351 /var/log/ceph/ceph.audit.log 2023-11-09T03:56:15.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 40u IPv4 183510 0t0 TCP smithi076.front.sepia.ceph.com:48946->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 41u IPv4 179814 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60322 (ESTABLISHED) 2023-11-09T03:56:15.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 42u IPv4 179820 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60330 (ESTABLISHED) 2023-11-09T03:56:15.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 43u IPv4 181612 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60338 (ESTABLISHED) 2023-11-09T03:56:15.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 44u IPv4 181618 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60350 (ESTABLISHED) 2023-11-09T03:56:15.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 45r REG 253,4 4632438 33816724 /var/lib/ceph/mon/ceph-a/store.db/000021.sst 2023-11-09T03:56:15.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 46u IPv4 267111 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:50182 (ESTABLISHED) 2023-11-09T03:56:15.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87753 signal_ha root 47u IPv4 271047 0t0 TCP smithi076.front.sepia.ceph.com:3300->192.168.0.1:37190 (ESTABLISHED) 2023-11-09T03:56:15.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root txt REG 8,1 12250256 3112 /usr/bin/ceph-mon 2023-11-09T03:56:15.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:15.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:15.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:15.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 1w FIFO 0,13 0t0 180522 pipe 2023-11-09T03:56:15.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 2w FIFO 0,13 0t0 180523 pipe 2023-11-09T03:56:15.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 3r FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 4w FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 5w REG 8,1 68063672 265319 /var/log/ceph/ceph-mon.a.log 2023-11-09T03:56:15.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 6uW REG 0,24 6 1732 /run/ceph/ceph-mon.a.pid 2023-11-09T03:56:15.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 7r FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 8w FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 9u unix 0xffff959a6034b740 0t0 177870 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 10r FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 11w FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 12r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 13uW REG 253,4 0 33816711 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2023-11-09T03:56:15.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 14r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 15u IPv4 250312 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60028 (ESTABLISHED) 2023-11-09T03:56:15.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 16w REG 253,4 1820304 33816715 /var/lib/ceph/mon/ceph-a/store.db/000019.log 2023-11-09T03:56:15.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 17w REG 253,4 898 33816719 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-09T03:56:15.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 18u a_inode 0,14 0 2077 [eventpoll:19,27,28,29,41,44,47] 2023-11-09T03:56:15.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 19r FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 20w FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 21u a_inode 0,14 0 2077 [eventpoll:15,22,38,42] 2023-11-09T03:56:15.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 22r FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 23w FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 24u a_inode 0,14 0 2077 [eventpoll:25,32,40,43,46] 2023-11-09T03:56:15.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 25r FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 26w FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 27u IPv4 178242 0t0 TCP smithi076.front.sepia.ceph.com:3300 (LISTEN) 2023-11-09T03:56:15.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 28u IPv4 178243 0t0 TCP smithi076.front.sepia.ceph.com:smc-https (LISTEN) 2023-11-09T03:56:15.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 29u IPv4 180558 0t0 TCP smithi076.front.sepia.ceph.com:52700->smithi113.front.sepia.ceph.com:3301 (ESTABLISHED) 2023-11-09T03:56:15.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 30r FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 31w FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 32u IPv4 176127 0t0 TCP smithi076.front.sepia.ceph.com:48276->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 33r FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 34w FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 35r FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 36w FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 37w REG 8,1 125211 265320 /var/log/ceph/ceph.log 2023-11-09T03:56:15.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 38u IPv4 182390 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:46708 (ESTABLISHED) 2023-11-09T03:56:15.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 39w REG 8,1 209797 265351 /var/log/ceph/ceph.audit.log 2023-11-09T03:56:15.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 40u IPv4 183510 0t0 TCP smithi076.front.sepia.ceph.com:48946->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 41u IPv4 179814 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60322 (ESTABLISHED) 2023-11-09T03:56:15.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 42u IPv4 179820 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60330 (ESTABLISHED) 2023-11-09T03:56:15.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 43u IPv4 181612 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60338 (ESTABLISHED) 2023-11-09T03:56:15.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 44u IPv4 181618 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60350 (ESTABLISHED) 2023-11-09T03:56:15.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 45r REG 253,4 4632438 33816724 /var/lib/ceph/mon/ceph-a/store.db/000021.sst 2023-11-09T03:56:15.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 46u IPv4 267111 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:50182 (ESTABLISHED) 2023-11-09T03:56:15.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87754 rocksdb:l root 47u IPv4 271047 0t0 TCP smithi076.front.sepia.ceph.com:3300->192.168.0.1:37190 (ESTABLISHED) 2023-11-09T03:56:15.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root txt REG 8,1 12250256 3112 /usr/bin/ceph-mon 2023-11-09T03:56:15.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:15.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:15.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:15.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 1w FIFO 0,13 0t0 180522 pipe 2023-11-09T03:56:15.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 2w FIFO 0,13 0t0 180523 pipe 2023-11-09T03:56:15.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 3r FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 4w FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 5w REG 8,1 68063672 265319 /var/log/ceph/ceph-mon.a.log 2023-11-09T03:56:15.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 6uW REG 0,24 6 1732 /run/ceph/ceph-mon.a.pid 2023-11-09T03:56:15.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 7r FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 8w FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 9u unix 0xffff959a6034b740 0t0 177870 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 10r FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 11w FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 12r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 13uW REG 253,4 0 33816711 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2023-11-09T03:56:15.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 14r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 15u IPv4 250312 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60028 (ESTABLISHED) 2023-11-09T03:56:15.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 16w REG 253,4 1820304 33816715 /var/lib/ceph/mon/ceph-a/store.db/000019.log 2023-11-09T03:56:15.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 17w REG 253,4 898 33816719 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-09T03:56:15.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 18u a_inode 0,14 0 2077 [eventpoll:19,27,28,29,41,44,47] 2023-11-09T03:56:15.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 19r FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 20w FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 21u a_inode 0,14 0 2077 [eventpoll:15,22,38,42] 2023-11-09T03:56:15.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 22r FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 23w FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 24u a_inode 0,14 0 2077 [eventpoll:25,32,40,43,46] 2023-11-09T03:56:15.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 25r FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 26w FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 27u IPv4 178242 0t0 TCP smithi076.front.sepia.ceph.com:3300 (LISTEN) 2023-11-09T03:56:15.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 28u IPv4 178243 0t0 TCP smithi076.front.sepia.ceph.com:smc-https (LISTEN) 2023-11-09T03:56:15.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 29u IPv4 180558 0t0 TCP smithi076.front.sepia.ceph.com:52700->smithi113.front.sepia.ceph.com:3301 (ESTABLISHED) 2023-11-09T03:56:15.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 30r FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 31w FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 32u IPv4 176127 0t0 TCP smithi076.front.sepia.ceph.com:48276->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 33r FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 34w FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 35r FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 36w FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 37w REG 8,1 125211 265320 /var/log/ceph/ceph.log 2023-11-09T03:56:15.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 38u IPv4 182390 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:46708 (ESTABLISHED) 2023-11-09T03:56:15.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 39w REG 8,1 209797 265351 /var/log/ceph/ceph.audit.log 2023-11-09T03:56:15.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 40u IPv4 183510 0t0 TCP smithi076.front.sepia.ceph.com:48946->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 41u IPv4 179814 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60322 (ESTABLISHED) 2023-11-09T03:56:15.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 42u IPv4 179820 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60330 (ESTABLISHED) 2023-11-09T03:56:15.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 43u IPv4 181612 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60338 (ESTABLISHED) 2023-11-09T03:56:15.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 44u IPv4 181618 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60350 (ESTABLISHED) 2023-11-09T03:56:15.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 45r REG 253,4 4632438 33816724 /var/lib/ceph/mon/ceph-a/store.db/000021.sst 2023-11-09T03:56:15.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 46u IPv4 267111 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:50182 (ESTABLISHED) 2023-11-09T03:56:15.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87755 rocksdb:h root 47u IPv4 271047 0t0 TCP smithi076.front.sepia.ceph.com:3300->192.168.0.1:37190 (ESTABLISHED) 2023-11-09T03:56:15.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root txt REG 8,1 12250256 3112 /usr/bin/ceph-mon 2023-11-09T03:56:15.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:15.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:15.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:15.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 1w FIFO 0,13 0t0 180522 pipe 2023-11-09T03:56:15.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 2w FIFO 0,13 0t0 180523 pipe 2023-11-09T03:56:15.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 3r FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 4w FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 5w REG 8,1 68063672 265319 /var/log/ceph/ceph-mon.a.log 2023-11-09T03:56:15.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 6uW REG 0,24 6 1732 /run/ceph/ceph-mon.a.pid 2023-11-09T03:56:15.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 7r FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 8w FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 9u unix 0xffff959a6034b740 0t0 177870 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 10r FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 11w FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 12r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 13uW REG 253,4 0 33816711 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2023-11-09T03:56:15.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 14r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 15u IPv4 250312 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60028 (ESTABLISHED) 2023-11-09T03:56:15.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 16w REG 253,4 1820304 33816715 /var/lib/ceph/mon/ceph-a/store.db/000019.log 2023-11-09T03:56:15.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 17w REG 253,4 898 33816719 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-09T03:56:15.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 18u a_inode 0,14 0 2077 [eventpoll:19,27,28,29,41,44,47] 2023-11-09T03:56:15.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 19r FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 20w FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 21u a_inode 0,14 0 2077 [eventpoll:15,22,38,42] 2023-11-09T03:56:15.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 22r FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 23w FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 24u a_inode 0,14 0 2077 [eventpoll:25,32,40,43,46] 2023-11-09T03:56:15.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 25r FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 26w FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 27u IPv4 178242 0t0 TCP smithi076.front.sepia.ceph.com:3300 (LISTEN) 2023-11-09T03:56:15.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 28u IPv4 178243 0t0 TCP smithi076.front.sepia.ceph.com:smc-https (LISTEN) 2023-11-09T03:56:15.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 29u IPv4 180558 0t0 TCP smithi076.front.sepia.ceph.com:52700->smithi113.front.sepia.ceph.com:3301 (ESTABLISHED) 2023-11-09T03:56:15.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 30r FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 31w FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 32u IPv4 176127 0t0 TCP smithi076.front.sepia.ceph.com:48276->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 33r FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 34w FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 35r FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 36w FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 37w REG 8,1 125211 265320 /var/log/ceph/ceph.log 2023-11-09T03:56:15.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 38u IPv4 182390 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:46708 (ESTABLISHED) 2023-11-09T03:56:15.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 39w REG 8,1 209797 265351 /var/log/ceph/ceph.audit.log 2023-11-09T03:56:15.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 40u IPv4 183510 0t0 TCP smithi076.front.sepia.ceph.com:48946->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 41u IPv4 179814 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60322 (ESTABLISHED) 2023-11-09T03:56:15.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 42u IPv4 179820 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60330 (ESTABLISHED) 2023-11-09T03:56:15.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 43u IPv4 181612 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60338 (ESTABLISHED) 2023-11-09T03:56:15.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 44u IPv4 181618 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60350 (ESTABLISHED) 2023-11-09T03:56:15.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 45r REG 253,4 4632438 33816724 /var/lib/ceph/mon/ceph-a/store.db/000021.sst 2023-11-09T03:56:15.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 46u IPv4 267111 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:50182 (ESTABLISHED) 2023-11-09T03:56:15.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87771 ceph-mon root 47u IPv4 271047 0t0 TCP smithi076.front.sepia.ceph.com:3300->192.168.0.1:37190 (ESTABLISHED) 2023-11-09T03:56:15.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root txt REG 8,1 12250256 3112 /usr/bin/ceph-mon 2023-11-09T03:56:15.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:15.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:15.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:15.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 1w FIFO 0,13 0t0 180522 pipe 2023-11-09T03:56:15.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 2w FIFO 0,13 0t0 180523 pipe 2023-11-09T03:56:15.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 3r FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 4w FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 5w REG 8,1 68063672 265319 /var/log/ceph/ceph-mon.a.log 2023-11-09T03:56:15.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 6uW REG 0,24 6 1732 /run/ceph/ceph-mon.a.pid 2023-11-09T03:56:15.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 7r FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 8w FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 9u unix 0xffff959a6034b740 0t0 177870 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 10r FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 11w FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 12r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 13uW REG 253,4 0 33816711 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2023-11-09T03:56:15.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 14r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 15u IPv4 250312 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60028 (ESTABLISHED) 2023-11-09T03:56:15.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 16w REG 253,4 1820304 33816715 /var/lib/ceph/mon/ceph-a/store.db/000019.log 2023-11-09T03:56:15.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 17w REG 253,4 898 33816719 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-09T03:56:15.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 18u a_inode 0,14 0 2077 [eventpoll:19,27,28,29,41,44,47] 2023-11-09T03:56:15.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 19r FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 20w FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 21u a_inode 0,14 0 2077 [eventpoll:15,22,38,42] 2023-11-09T03:56:15.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 22r FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 23w FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 24u a_inode 0,14 0 2077 [eventpoll:25,32,40,43,46] 2023-11-09T03:56:15.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 25r FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 26w FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 27u IPv4 178242 0t0 TCP smithi076.front.sepia.ceph.com:3300 (LISTEN) 2023-11-09T03:56:15.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 28u IPv4 178243 0t0 TCP smithi076.front.sepia.ceph.com:smc-https (LISTEN) 2023-11-09T03:56:15.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 29u IPv4 180558 0t0 TCP smithi076.front.sepia.ceph.com:52700->smithi113.front.sepia.ceph.com:3301 (ESTABLISHED) 2023-11-09T03:56:15.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 30r FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 31w FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 32u IPv4 176127 0t0 TCP smithi076.front.sepia.ceph.com:48276->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 33r FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 34w FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 35r FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 36w FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 37w REG 8,1 125211 265320 /var/log/ceph/ceph.log 2023-11-09T03:56:15.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 38u IPv4 182390 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:46708 (ESTABLISHED) 2023-11-09T03:56:15.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 39w REG 8,1 209797 265351 /var/log/ceph/ceph.audit.log 2023-11-09T03:56:15.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 40u IPv4 183510 0t0 TCP smithi076.front.sepia.ceph.com:48946->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 41u IPv4 179814 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60322 (ESTABLISHED) 2023-11-09T03:56:15.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 42u IPv4 179820 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60330 (ESTABLISHED) 2023-11-09T03:56:15.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 43u IPv4 181612 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60338 (ESTABLISHED) 2023-11-09T03:56:15.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 44u IPv4 181618 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60350 (ESTABLISHED) 2023-11-09T03:56:15.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 45r REG 253,4 4632438 33816724 /var/lib/ceph/mon/ceph-a/store.db/000021.sst 2023-11-09T03:56:15.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 46u IPv4 267111 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:50182 (ESTABLISHED) 2023-11-09T03:56:15.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87772 fn_monsto root 47u IPv4 271047 0t0 TCP smithi076.front.sepia.ceph.com:3300->192.168.0.1:37190 (ESTABLISHED) 2023-11-09T03:56:15.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root txt REG 8,1 12250256 3112 /usr/bin/ceph-mon 2023-11-09T03:56:15.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:15.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:15.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:15.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 1w FIFO 0,13 0t0 180522 pipe 2023-11-09T03:56:15.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 2w FIFO 0,13 0t0 180523 pipe 2023-11-09T03:56:15.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 3r FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 4w FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 5w REG 8,1 68063672 265319 /var/log/ceph/ceph-mon.a.log 2023-11-09T03:56:15.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 6uW REG 0,24 6 1732 /run/ceph/ceph-mon.a.pid 2023-11-09T03:56:15.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 7r FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 8w FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 9u unix 0xffff959a6034b740 0t0 177870 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 10r FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 11w FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 12r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 13uW REG 253,4 0 33816711 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2023-11-09T03:56:15.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 14r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 15u IPv4 250312 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60028 (ESTABLISHED) 2023-11-09T03:56:15.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 16w REG 253,4 1820304 33816715 /var/lib/ceph/mon/ceph-a/store.db/000019.log 2023-11-09T03:56:15.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 17w REG 253,4 898 33816719 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-09T03:56:15.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 18u a_inode 0,14 0 2077 [eventpoll:19,27,28,29,41,44,47] 2023-11-09T03:56:15.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 19r FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 20w FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 21u a_inode 0,14 0 2077 [eventpoll:15,22,38,42] 2023-11-09T03:56:15.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 22r FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 23w FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 24u a_inode 0,14 0 2077 [eventpoll:25,32,40,43,46] 2023-11-09T03:56:15.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 25r FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 26w FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 27u IPv4 178242 0t0 TCP smithi076.front.sepia.ceph.com:3300 (LISTEN) 2023-11-09T03:56:15.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 28u IPv4 178243 0t0 TCP smithi076.front.sepia.ceph.com:smc-https (LISTEN) 2023-11-09T03:56:15.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 29u IPv4 180558 0t0 TCP smithi076.front.sepia.ceph.com:52700->smithi113.front.sepia.ceph.com:3301 (ESTABLISHED) 2023-11-09T03:56:15.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 30r FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 31w FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 32u IPv4 176127 0t0 TCP smithi076.front.sepia.ceph.com:48276->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 33r FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 34w FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 35r FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 36w FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 37w REG 8,1 125211 265320 /var/log/ceph/ceph.log 2023-11-09T03:56:15.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 38u IPv4 182390 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:46708 (ESTABLISHED) 2023-11-09T03:56:15.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 39w REG 8,1 209797 265351 /var/log/ceph/ceph.audit.log 2023-11-09T03:56:15.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 40u IPv4 183510 0t0 TCP smithi076.front.sepia.ceph.com:48946->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 41u IPv4 179814 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60322 (ESTABLISHED) 2023-11-09T03:56:15.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 42u IPv4 179820 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60330 (ESTABLISHED) 2023-11-09T03:56:15.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 43u IPv4 181612 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60338 (ESTABLISHED) 2023-11-09T03:56:15.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 44u IPv4 181618 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60350 (ESTABLISHED) 2023-11-09T03:56:15.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 45r REG 253,4 4632438 33816724 /var/lib/ceph/mon/ceph-a/store.db/000021.sst 2023-11-09T03:56:15.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 46u IPv4 267111 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:50182 (ESTABLISHED) 2023-11-09T03:56:15.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87773 msgr-work root 47u IPv4 271047 0t0 TCP smithi076.front.sepia.ceph.com:3300->192.168.0.1:37190 (ESTABLISHED) 2023-11-09T03:56:15.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root txt REG 8,1 12250256 3112 /usr/bin/ceph-mon 2023-11-09T03:56:15.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:15.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:15.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:15.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 1w FIFO 0,13 0t0 180522 pipe 2023-11-09T03:56:15.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 2w FIFO 0,13 0t0 180523 pipe 2023-11-09T03:56:15.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 3r FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 4w FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 5w REG 8,1 68063672 265319 /var/log/ceph/ceph-mon.a.log 2023-11-09T03:56:15.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 6uW REG 0,24 6 1732 /run/ceph/ceph-mon.a.pid 2023-11-09T03:56:15.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 7r FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 8w FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 9u unix 0xffff959a6034b740 0t0 177870 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 10r FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 11w FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 12r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 13uW REG 253,4 0 33816711 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2023-11-09T03:56:15.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 14r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 15u IPv4 250312 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60028 (ESTABLISHED) 2023-11-09T03:56:15.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 16w REG 253,4 1820304 33816715 /var/lib/ceph/mon/ceph-a/store.db/000019.log 2023-11-09T03:56:15.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 17w REG 253,4 898 33816719 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-09T03:56:15.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 18u a_inode 0,14 0 2077 [eventpoll:19,27,28,29,41,44,47] 2023-11-09T03:56:15.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 19r FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 20w FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 21u a_inode 0,14 0 2077 [eventpoll:15,22,38,42] 2023-11-09T03:56:15.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 22r FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 23w FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 24u a_inode 0,14 0 2077 [eventpoll:25,32,40,43,46] 2023-11-09T03:56:15.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 25r FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 26w FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 27u IPv4 178242 0t0 TCP smithi076.front.sepia.ceph.com:3300 (LISTEN) 2023-11-09T03:56:15.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 28u IPv4 178243 0t0 TCP smithi076.front.sepia.ceph.com:smc-https (LISTEN) 2023-11-09T03:56:15.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 29u IPv4 180558 0t0 TCP smithi076.front.sepia.ceph.com:52700->smithi113.front.sepia.ceph.com:3301 (ESTABLISHED) 2023-11-09T03:56:15.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 30r FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 31w FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 32u IPv4 176127 0t0 TCP smithi076.front.sepia.ceph.com:48276->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 33r FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 34w FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 35r FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 36w FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 37w REG 8,1 125211 265320 /var/log/ceph/ceph.log 2023-11-09T03:56:15.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 38u IPv4 182390 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:46708 (ESTABLISHED) 2023-11-09T03:56:15.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 39w REG 8,1 209797 265351 /var/log/ceph/ceph.audit.log 2023-11-09T03:56:15.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 40u IPv4 183510 0t0 TCP smithi076.front.sepia.ceph.com:48946->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 41u IPv4 179814 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60322 (ESTABLISHED) 2023-11-09T03:56:15.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 42u IPv4 179820 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60330 (ESTABLISHED) 2023-11-09T03:56:15.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 43u IPv4 181612 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60338 (ESTABLISHED) 2023-11-09T03:56:15.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 44u IPv4 181618 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60350 (ESTABLISHED) 2023-11-09T03:56:15.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 45r REG 253,4 4632438 33816724 /var/lib/ceph/mon/ceph-a/store.db/000021.sst 2023-11-09T03:56:15.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 46u IPv4 267111 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:50182 (ESTABLISHED) 2023-11-09T03:56:15.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87774 msgr-work root 47u IPv4 271047 0t0 TCP smithi076.front.sepia.ceph.com:3300->192.168.0.1:37190 (ESTABLISHED) 2023-11-09T03:56:15.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root txt REG 8,1 12250256 3112 /usr/bin/ceph-mon 2023-11-09T03:56:15.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:15.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:15.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:15.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 1w FIFO 0,13 0t0 180522 pipe 2023-11-09T03:56:15.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 2w FIFO 0,13 0t0 180523 pipe 2023-11-09T03:56:15.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 3r FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 4w FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 5w REG 8,1 68063672 265319 /var/log/ceph/ceph-mon.a.log 2023-11-09T03:56:15.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 6uW REG 0,24 6 1732 /run/ceph/ceph-mon.a.pid 2023-11-09T03:56:15.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 7r FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 8w FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 9u unix 0xffff959a6034b740 0t0 177870 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 10r FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 11w FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 12r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 13uW REG 253,4 0 33816711 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2023-11-09T03:56:15.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 14r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 15u IPv4 250312 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60028 (ESTABLISHED) 2023-11-09T03:56:15.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 16w REG 253,4 1820304 33816715 /var/lib/ceph/mon/ceph-a/store.db/000019.log 2023-11-09T03:56:15.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 17w REG 253,4 898 33816719 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-09T03:56:15.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 18u a_inode 0,14 0 2077 [eventpoll:19,27,28,29,41,44,47] 2023-11-09T03:56:15.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 19r FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 20w FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 21u a_inode 0,14 0 2077 [eventpoll:15,22,38,42] 2023-11-09T03:56:15.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 22r FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 23w FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 24u a_inode 0,14 0 2077 [eventpoll:25,32,40,43,46] 2023-11-09T03:56:15.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 25r FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 26w FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 27u IPv4 178242 0t0 TCP smithi076.front.sepia.ceph.com:3300 (LISTEN) 2023-11-09T03:56:15.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 28u IPv4 178243 0t0 TCP smithi076.front.sepia.ceph.com:smc-https (LISTEN) 2023-11-09T03:56:15.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 29u IPv4 180558 0t0 TCP smithi076.front.sepia.ceph.com:52700->smithi113.front.sepia.ceph.com:3301 (ESTABLISHED) 2023-11-09T03:56:15.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 30r FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 31w FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 32u IPv4 176127 0t0 TCP smithi076.front.sepia.ceph.com:48276->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 33r FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 34w FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 35r FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 36w FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 37w REG 8,1 125211 265320 /var/log/ceph/ceph.log 2023-11-09T03:56:15.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 38u IPv4 182390 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:46708 (ESTABLISHED) 2023-11-09T03:56:15.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 39w REG 8,1 209797 265351 /var/log/ceph/ceph.audit.log 2023-11-09T03:56:15.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 40u IPv4 183510 0t0 TCP smithi076.front.sepia.ceph.com:48946->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 41u IPv4 179814 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60322 (ESTABLISHED) 2023-11-09T03:56:15.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 42u IPv4 179820 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60330 (ESTABLISHED) 2023-11-09T03:56:15.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 43u IPv4 181612 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60338 (ESTABLISHED) 2023-11-09T03:56:15.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 44u IPv4 181618 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60350 (ESTABLISHED) 2023-11-09T03:56:15.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 45r REG 253,4 4632438 33816724 /var/lib/ceph/mon/ceph-a/store.db/000021.sst 2023-11-09T03:56:15.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 46u IPv4 267111 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:50182 (ESTABLISHED) 2023-11-09T03:56:15.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87775 msgr-work root 47u IPv4 271047 0t0 TCP smithi076.front.sepia.ceph.com:3300->192.168.0.1:37190 (ESTABLISHED) 2023-11-09T03:56:15.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root txt REG 8,1 12250256 3112 /usr/bin/ceph-mon 2023-11-09T03:56:15.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:15.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:15.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:15.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 1w FIFO 0,13 0t0 180522 pipe 2023-11-09T03:56:15.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 2w FIFO 0,13 0t0 180523 pipe 2023-11-09T03:56:15.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 3r FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 4w FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 5w REG 8,1 68063672 265319 /var/log/ceph/ceph-mon.a.log 2023-11-09T03:56:15.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 6uW REG 0,24 6 1732 /run/ceph/ceph-mon.a.pid 2023-11-09T03:56:15.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 7r FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 8w FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 9u unix 0xffff959a6034b740 0t0 177870 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 10r FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 11w FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 12r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 13uW REG 253,4 0 33816711 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2023-11-09T03:56:15.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 14r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 15u IPv4 250312 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60028 (ESTABLISHED) 2023-11-09T03:56:15.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 16w REG 253,4 1820304 33816715 /var/lib/ceph/mon/ceph-a/store.db/000019.log 2023-11-09T03:56:15.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 17w REG 253,4 898 33816719 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-09T03:56:15.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 18u a_inode 0,14 0 2077 [eventpoll:19,27,28,29,41,44,47] 2023-11-09T03:56:15.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 19r FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 20w FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 21u a_inode 0,14 0 2077 [eventpoll:15,22,38,42] 2023-11-09T03:56:15.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 22r FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 23w FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 24u a_inode 0,14 0 2077 [eventpoll:25,32,40,43,46] 2023-11-09T03:56:15.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 25r FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 26w FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 27u IPv4 178242 0t0 TCP smithi076.front.sepia.ceph.com:3300 (LISTEN) 2023-11-09T03:56:15.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 28u IPv4 178243 0t0 TCP smithi076.front.sepia.ceph.com:smc-https (LISTEN) 2023-11-09T03:56:15.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 29u IPv4 180558 0t0 TCP smithi076.front.sepia.ceph.com:52700->smithi113.front.sepia.ceph.com:3301 (ESTABLISHED) 2023-11-09T03:56:15.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 30r FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 31w FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 32u IPv4 176127 0t0 TCP smithi076.front.sepia.ceph.com:48276->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 33r FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 34w FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 35r FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 36w FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 37w REG 8,1 125211 265320 /var/log/ceph/ceph.log 2023-11-09T03:56:15.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 38u IPv4 182390 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:46708 (ESTABLISHED) 2023-11-09T03:56:15.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 39w REG 8,1 209797 265351 /var/log/ceph/ceph.audit.log 2023-11-09T03:56:15.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 40u IPv4 183510 0t0 TCP smithi076.front.sepia.ceph.com:48946->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 41u IPv4 179814 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60322 (ESTABLISHED) 2023-11-09T03:56:15.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 42u IPv4 179820 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60330 (ESTABLISHED) 2023-11-09T03:56:15.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 43u IPv4 181612 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60338 (ESTABLISHED) 2023-11-09T03:56:15.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 44u IPv4 181618 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60350 (ESTABLISHED) 2023-11-09T03:56:15.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 45r REG 253,4 4632438 33816724 /var/lib/ceph/mon/ceph-a/store.db/000021.sst 2023-11-09T03:56:15.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 46u IPv4 267111 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:50182 (ESTABLISHED) 2023-11-09T03:56:15.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87776 OpHistory root 47u IPv4 271047 0t0 TCP smithi076.front.sepia.ceph.com:3300->192.168.0.1:37190 (ESTABLISHED) 2023-11-09T03:56:15.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root txt REG 8,1 12250256 3112 /usr/bin/ceph-mon 2023-11-09T03:56:15.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:15.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:15.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:15.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 1w FIFO 0,13 0t0 180522 pipe 2023-11-09T03:56:15.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 2w FIFO 0,13 0t0 180523 pipe 2023-11-09T03:56:15.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 3r FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 4w FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 5w REG 8,1 68063672 265319 /var/log/ceph/ceph-mon.a.log 2023-11-09T03:56:15.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 6uW REG 0,24 6 1732 /run/ceph/ceph-mon.a.pid 2023-11-09T03:56:15.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 7r FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 8w FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 9u unix 0xffff959a6034b740 0t0 177870 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 10r FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 11w FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 12r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 13uW REG 253,4 0 33816711 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2023-11-09T03:56:15.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 14r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 15u IPv4 250312 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60028 (ESTABLISHED) 2023-11-09T03:56:15.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 16w REG 253,4 1820304 33816715 /var/lib/ceph/mon/ceph-a/store.db/000019.log 2023-11-09T03:56:15.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 17w REG 253,4 898 33816719 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-09T03:56:15.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 18u a_inode 0,14 0 2077 [eventpoll:19,27,28,29,41,44,47] 2023-11-09T03:56:15.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 19r FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 20w FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 21u a_inode 0,14 0 2077 [eventpoll:15,22,38,42] 2023-11-09T03:56:15.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 22r FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 23w FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 24u a_inode 0,14 0 2077 [eventpoll:25,32,40,43,46] 2023-11-09T03:56:15.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 25r FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 26w FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 27u IPv4 178242 0t0 TCP smithi076.front.sepia.ceph.com:3300 (LISTEN) 2023-11-09T03:56:15.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 28u IPv4 178243 0t0 TCP smithi076.front.sepia.ceph.com:smc-https (LISTEN) 2023-11-09T03:56:15.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 29u IPv4 180558 0t0 TCP smithi076.front.sepia.ceph.com:52700->smithi113.front.sepia.ceph.com:3301 (ESTABLISHED) 2023-11-09T03:56:15.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 30r FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 31w FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 32u IPv4 176127 0t0 TCP smithi076.front.sepia.ceph.com:48276->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 33r FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 34w FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 35r FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 36w FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 37w REG 8,1 125211 265320 /var/log/ceph/ceph.log 2023-11-09T03:56:15.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 38u IPv4 182390 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:46708 (ESTABLISHED) 2023-11-09T03:56:15.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 39w REG 8,1 209797 265351 /var/log/ceph/ceph.audit.log 2023-11-09T03:56:15.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 40u IPv4 183510 0t0 TCP smithi076.front.sepia.ceph.com:48946->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 41u IPv4 179814 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60322 (ESTABLISHED) 2023-11-09T03:56:15.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 42u IPv4 179820 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60330 (ESTABLISHED) 2023-11-09T03:56:15.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 43u IPv4 181612 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60338 (ESTABLISHED) 2023-11-09T03:56:15.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 44u IPv4 181618 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60350 (ESTABLISHED) 2023-11-09T03:56:15.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 45r REG 253,4 4632438 33816724 /var/lib/ceph/mon/ceph-a/store.db/000021.sst 2023-11-09T03:56:15.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 46u IPv4 267111 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:50182 (ESTABLISHED) 2023-11-09T03:56:15.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87777 fin root 47u IPv4 271047 0t0 TCP smithi076.front.sepia.ceph.com:3300->192.168.0.1:37190 (ESTABLISHED) 2023-11-09T03:56:15.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root txt REG 8,1 12250256 3112 /usr/bin/ceph-mon 2023-11-09T03:56:15.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:15.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:15.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:15.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 1w FIFO 0,13 0t0 180522 pipe 2023-11-09T03:56:15.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 2w FIFO 0,13 0t0 180523 pipe 2023-11-09T03:56:15.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 3r FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 4w FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 5w REG 8,1 68063672 265319 /var/log/ceph/ceph-mon.a.log 2023-11-09T03:56:15.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 6uW REG 0,24 6 1732 /run/ceph/ceph-mon.a.pid 2023-11-09T03:56:15.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 7r FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 8w FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 9u unix 0xffff959a6034b740 0t0 177870 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 10r FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 11w FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 12r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 13uW REG 253,4 0 33816711 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2023-11-09T03:56:15.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 14r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 15u IPv4 250312 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60028 (ESTABLISHED) 2023-11-09T03:56:15.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 16w REG 253,4 1820304 33816715 /var/lib/ceph/mon/ceph-a/store.db/000019.log 2023-11-09T03:56:15.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 17w REG 253,4 898 33816719 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-09T03:56:15.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 18u a_inode 0,14 0 2077 [eventpoll:19,27,28,29,41,44,47] 2023-11-09T03:56:15.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 19r FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 20w FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 21u a_inode 0,14 0 2077 [eventpoll:15,22,38,42] 2023-11-09T03:56:15.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 22r FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 23w FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 24u a_inode 0,14 0 2077 [eventpoll:25,32,40,43,46] 2023-11-09T03:56:15.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 25r FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 26w FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 27u IPv4 178242 0t0 TCP smithi076.front.sepia.ceph.com:3300 (LISTEN) 2023-11-09T03:56:15.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 28u IPv4 178243 0t0 TCP smithi076.front.sepia.ceph.com:smc-https (LISTEN) 2023-11-09T03:56:15.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 29u IPv4 180558 0t0 TCP smithi076.front.sepia.ceph.com:52700->smithi113.front.sepia.ceph.com:3301 (ESTABLISHED) 2023-11-09T03:56:15.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 30r FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 31w FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 32u IPv4 176127 0t0 TCP smithi076.front.sepia.ceph.com:48276->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 33r FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 34w FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 35r FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 36w FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 37w REG 8,1 125211 265320 /var/log/ceph/ceph.log 2023-11-09T03:56:15.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 38u IPv4 182390 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:46708 (ESTABLISHED) 2023-11-09T03:56:15.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 39w REG 8,1 209797 265351 /var/log/ceph/ceph.audit.log 2023-11-09T03:56:15.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 40u IPv4 183510 0t0 TCP smithi076.front.sepia.ceph.com:48946->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 41u IPv4 179814 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60322 (ESTABLISHED) 2023-11-09T03:56:15.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 42u IPv4 179820 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60330 (ESTABLISHED) 2023-11-09T03:56:15.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 43u IPv4 181612 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60338 (ESTABLISHED) 2023-11-09T03:56:15.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 44u IPv4 181618 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60350 (ESTABLISHED) 2023-11-09T03:56:15.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 45r REG 253,4 4632438 33816724 /var/lib/ceph/mon/ceph-a/store.db/000021.sst 2023-11-09T03:56:15.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 46u IPv4 267111 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:50182 (ESTABLISHED) 2023-11-09T03:56:15.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87778 safe_time root 47u IPv4 271047 0t0 TCP smithi076.front.sepia.ceph.com:3300->192.168.0.1:37190 (ESTABLISHED) 2023-11-09T03:56:15.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root txt REG 8,1 12250256 3112 /usr/bin/ceph-mon 2023-11-09T03:56:15.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:15.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:15.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:15.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 1w FIFO 0,13 0t0 180522 pipe 2023-11-09T03:56:15.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 2w FIFO 0,13 0t0 180523 pipe 2023-11-09T03:56:15.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 3r FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 4w FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 5w REG 8,1 68063672 265319 /var/log/ceph/ceph-mon.a.log 2023-11-09T03:56:15.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 6uW REG 0,24 6 1732 /run/ceph/ceph-mon.a.pid 2023-11-09T03:56:15.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 7r FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 8w FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 9u unix 0xffff959a6034b740 0t0 177870 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 10r FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 11w FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 12r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 13uW REG 253,4 0 33816711 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2023-11-09T03:56:15.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 14r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 15u IPv4 250312 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60028 (ESTABLISHED) 2023-11-09T03:56:15.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 16w REG 253,4 1820304 33816715 /var/lib/ceph/mon/ceph-a/store.db/000019.log 2023-11-09T03:56:15.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 17w REG 253,4 898 33816719 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-09T03:56:15.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 18u a_inode 0,14 0 2077 [eventpoll:19,27,28,29,41,44,47] 2023-11-09T03:56:15.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 19r FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 20w FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 21u a_inode 0,14 0 2077 [eventpoll:15,22,38,42] 2023-11-09T03:56:15.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 22r FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 23w FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 24u a_inode 0,14 0 2077 [eventpoll:25,32,40,43,46] 2023-11-09T03:56:15.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 25r FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 26w FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 27u IPv4 178242 0t0 TCP smithi076.front.sepia.ceph.com:3300 (LISTEN) 2023-11-09T03:56:15.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 28u IPv4 178243 0t0 TCP smithi076.front.sepia.ceph.com:smc-https (LISTEN) 2023-11-09T03:56:15.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 29u IPv4 180558 0t0 TCP smithi076.front.sepia.ceph.com:52700->smithi113.front.sepia.ceph.com:3301 (ESTABLISHED) 2023-11-09T03:56:15.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 30r FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 31w FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 32u IPv4 176127 0t0 TCP smithi076.front.sepia.ceph.com:48276->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 33r FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 34w FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 35r FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 36w FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 37w REG 8,1 125211 265320 /var/log/ceph/ceph.log 2023-11-09T03:56:15.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 38u IPv4 182390 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:46708 (ESTABLISHED) 2023-11-09T03:56:15.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 39w REG 8,1 209797 265351 /var/log/ceph/ceph.audit.log 2023-11-09T03:56:15.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 40u IPv4 183510 0t0 TCP smithi076.front.sepia.ceph.com:48946->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 41u IPv4 179814 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60322 (ESTABLISHED) 2023-11-09T03:56:15.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 42u IPv4 179820 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60330 (ESTABLISHED) 2023-11-09T03:56:15.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 43u IPv4 181612 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60338 (ESTABLISHED) 2023-11-09T03:56:15.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 44u IPv4 181618 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60350 (ESTABLISHED) 2023-11-09T03:56:15.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 45r REG 253,4 4632438 33816724 /var/lib/ceph/mon/ceph-a/store.db/000021.sst 2023-11-09T03:56:15.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 46u IPv4 267111 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:50182 (ESTABLISHED) 2023-11-09T03:56:15.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87779 cpu_tp root 47u IPv4 271047 0t0 TCP smithi076.front.sepia.ceph.com:3300->192.168.0.1:37190 (ESTABLISHED) 2023-11-09T03:56:15.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root txt REG 8,1 12250256 3112 /usr/bin/ceph-mon 2023-11-09T03:56:15.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:15.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:15.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:15.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 1w FIFO 0,13 0t0 180522 pipe 2023-11-09T03:56:15.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 2w FIFO 0,13 0t0 180523 pipe 2023-11-09T03:56:15.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 3r FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 4w FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 5w REG 8,1 68063672 265319 /var/log/ceph/ceph-mon.a.log 2023-11-09T03:56:15.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 6uW REG 0,24 6 1732 /run/ceph/ceph-mon.a.pid 2023-11-09T03:56:15.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 7r FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 8w FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 9u unix 0xffff959a6034b740 0t0 177870 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 10r FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 11w FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 12r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 13uW REG 253,4 0 33816711 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2023-11-09T03:56:15.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 14r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 15u IPv4 250312 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60028 (ESTABLISHED) 2023-11-09T03:56:15.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 16w REG 253,4 1820304 33816715 /var/lib/ceph/mon/ceph-a/store.db/000019.log 2023-11-09T03:56:15.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 17w REG 253,4 898 33816719 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-09T03:56:15.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 18u a_inode 0,14 0 2077 [eventpoll:19,27,28,29,41,44,47] 2023-11-09T03:56:15.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 19r FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 20w FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 21u a_inode 0,14 0 2077 [eventpoll:15,22,38,42] 2023-11-09T03:56:15.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 22r FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 23w FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 24u a_inode 0,14 0 2077 [eventpoll:25,32,40,43,46] 2023-11-09T03:56:15.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 25r FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 26w FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 27u IPv4 178242 0t0 TCP smithi076.front.sepia.ceph.com:3300 (LISTEN) 2023-11-09T03:56:15.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 28u IPv4 178243 0t0 TCP smithi076.front.sepia.ceph.com:smc-https (LISTEN) 2023-11-09T03:56:15.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 29u IPv4 180558 0t0 TCP smithi076.front.sepia.ceph.com:52700->smithi113.front.sepia.ceph.com:3301 (ESTABLISHED) 2023-11-09T03:56:15.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 30r FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 31w FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 32u IPv4 176127 0t0 TCP smithi076.front.sepia.ceph.com:48276->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 33r FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 34w FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 35r FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 36w FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 37w REG 8,1 125211 265320 /var/log/ceph/ceph.log 2023-11-09T03:56:15.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 38u IPv4 182390 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:46708 (ESTABLISHED) 2023-11-09T03:56:15.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 39w REG 8,1 209797 265351 /var/log/ceph/ceph.audit.log 2023-11-09T03:56:15.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 40u IPv4 183510 0t0 TCP smithi076.front.sepia.ceph.com:48946->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 41u IPv4 179814 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60322 (ESTABLISHED) 2023-11-09T03:56:15.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 42u IPv4 179820 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60330 (ESTABLISHED) 2023-11-09T03:56:15.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 43u IPv4 181612 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60338 (ESTABLISHED) 2023-11-09T03:56:15.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 44u IPv4 181618 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60350 (ESTABLISHED) 2023-11-09T03:56:15.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 45r REG 253,4 4632438 33816724 /var/lib/ceph/mon/ceph-a/store.db/000021.sst 2023-11-09T03:56:15.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 46u IPv4 267111 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:50182 (ESTABLISHED) 2023-11-09T03:56:15.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87780 cpu_tp root 47u IPv4 271047 0t0 TCP smithi076.front.sepia.ceph.com:3300->192.168.0.1:37190 (ESTABLISHED) 2023-11-09T03:56:15.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root txt REG 8,1 12250256 3112 /usr/bin/ceph-mon 2023-11-09T03:56:15.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:15.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:15.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:15.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 1w FIFO 0,13 0t0 180522 pipe 2023-11-09T03:56:15.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 2w FIFO 0,13 0t0 180523 pipe 2023-11-09T03:56:15.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 3r FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 4w FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 5w REG 8,1 68063672 265319 /var/log/ceph/ceph-mon.a.log 2023-11-09T03:56:15.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 6uW REG 0,24 6 1732 /run/ceph/ceph-mon.a.pid 2023-11-09T03:56:15.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 7r FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 8w FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 9u unix 0xffff959a6034b740 0t0 177870 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 10r FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 11w FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 12r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 13uW REG 253,4 0 33816711 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2023-11-09T03:56:15.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 14r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 15u IPv4 250312 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60028 (ESTABLISHED) 2023-11-09T03:56:15.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 16w REG 253,4 1820304 33816715 /var/lib/ceph/mon/ceph-a/store.db/000019.log 2023-11-09T03:56:15.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 17w REG 253,4 898 33816719 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-09T03:56:15.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 18u a_inode 0,14 0 2077 [eventpoll:19,27,28,29,41,44,47] 2023-11-09T03:56:15.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 19r FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 20w FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 21u a_inode 0,14 0 2077 [eventpoll:15,22,38,42] 2023-11-09T03:56:15.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 22r FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 23w FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 24u a_inode 0,14 0 2077 [eventpoll:25,32,40,43,46] 2023-11-09T03:56:15.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 25r FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 26w FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 27u IPv4 178242 0t0 TCP smithi076.front.sepia.ceph.com:3300 (LISTEN) 2023-11-09T03:56:15.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 28u IPv4 178243 0t0 TCP smithi076.front.sepia.ceph.com:smc-https (LISTEN) 2023-11-09T03:56:15.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 29u IPv4 180558 0t0 TCP smithi076.front.sepia.ceph.com:52700->smithi113.front.sepia.ceph.com:3301 (ESTABLISHED) 2023-11-09T03:56:15.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 30r FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 31w FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 32u IPv4 176127 0t0 TCP smithi076.front.sepia.ceph.com:48276->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 33r FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 34w FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 35r FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 36w FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 37w REG 8,1 125211 265320 /var/log/ceph/ceph.log 2023-11-09T03:56:15.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 38u IPv4 182390 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:46708 (ESTABLISHED) 2023-11-09T03:56:15.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 39w REG 8,1 209797 265351 /var/log/ceph/ceph.audit.log 2023-11-09T03:56:15.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 40u IPv4 183510 0t0 TCP smithi076.front.sepia.ceph.com:48946->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 41u IPv4 179814 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60322 (ESTABLISHED) 2023-11-09T03:56:15.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 42u IPv4 179820 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60330 (ESTABLISHED) 2023-11-09T03:56:15.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 43u IPv4 181612 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60338 (ESTABLISHED) 2023-11-09T03:56:15.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 44u IPv4 181618 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60350 (ESTABLISHED) 2023-11-09T03:56:15.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 45r REG 253,4 4632438 33816724 /var/lib/ceph/mon/ceph-a/store.db/000021.sst 2023-11-09T03:56:15.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 46u IPv4 267111 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:50182 (ESTABLISHED) 2023-11-09T03:56:15.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87781 cpu_tp root 47u IPv4 271047 0t0 TCP smithi076.front.sepia.ceph.com:3300->192.168.0.1:37190 (ESTABLISHED) 2023-11-09T03:56:15.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root txt REG 8,1 12250256 3112 /usr/bin/ceph-mon 2023-11-09T03:56:15.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:15.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:15.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:15.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 1w FIFO 0,13 0t0 180522 pipe 2023-11-09T03:56:15.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 2w FIFO 0,13 0t0 180523 pipe 2023-11-09T03:56:15.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 3r FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 4w FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 5w REG 8,1 68063672 265319 /var/log/ceph/ceph-mon.a.log 2023-11-09T03:56:15.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 6uW REG 0,24 6 1732 /run/ceph/ceph-mon.a.pid 2023-11-09T03:56:15.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 7r FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 8w FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 9u unix 0xffff959a6034b740 0t0 177870 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 10r FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 11w FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 12r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 13uW REG 253,4 0 33816711 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2023-11-09T03:56:15.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 14r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 15u IPv4 250312 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60028 (ESTABLISHED) 2023-11-09T03:56:15.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 16w REG 253,4 1820304 33816715 /var/lib/ceph/mon/ceph-a/store.db/000019.log 2023-11-09T03:56:15.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 17w REG 253,4 898 33816719 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-09T03:56:15.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 18u a_inode 0,14 0 2077 [eventpoll:19,27,28,29,41,44,47] 2023-11-09T03:56:15.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 19r FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 20w FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 21u a_inode 0,14 0 2077 [eventpoll:15,22,38,42] 2023-11-09T03:56:15.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 22r FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 23w FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 24u a_inode 0,14 0 2077 [eventpoll:25,32,40,43,46] 2023-11-09T03:56:15.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 25r FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 26w FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 27u IPv4 178242 0t0 TCP smithi076.front.sepia.ceph.com:3300 (LISTEN) 2023-11-09T03:56:15.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 28u IPv4 178243 0t0 TCP smithi076.front.sepia.ceph.com:smc-https (LISTEN) 2023-11-09T03:56:15.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 29u IPv4 180558 0t0 TCP smithi076.front.sepia.ceph.com:52700->smithi113.front.sepia.ceph.com:3301 (ESTABLISHED) 2023-11-09T03:56:15.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 30r FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 31w FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 32u IPv4 176127 0t0 TCP smithi076.front.sepia.ceph.com:48276->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 33r FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 34w FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 35r FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 36w FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 37w REG 8,1 125211 265320 /var/log/ceph/ceph.log 2023-11-09T03:56:15.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 38u IPv4 182390 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:46708 (ESTABLISHED) 2023-11-09T03:56:15.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 39w REG 8,1 209797 265351 /var/log/ceph/ceph.audit.log 2023-11-09T03:56:15.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 40u IPv4 183510 0t0 TCP smithi076.front.sepia.ceph.com:48946->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 41u IPv4 179814 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60322 (ESTABLISHED) 2023-11-09T03:56:15.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 42u IPv4 179820 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60330 (ESTABLISHED) 2023-11-09T03:56:15.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 43u IPv4 181612 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60338 (ESTABLISHED) 2023-11-09T03:56:15.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 44u IPv4 181618 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60350 (ESTABLISHED) 2023-11-09T03:56:15.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 45r REG 253,4 4632438 33816724 /var/lib/ceph/mon/ceph-a/store.db/000021.sst 2023-11-09T03:56:15.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 46u IPv4 267111 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:50182 (ESTABLISHED) 2023-11-09T03:56:15.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87782 cpu_tp root 47u IPv4 271047 0t0 TCP smithi076.front.sepia.ceph.com:3300->192.168.0.1:37190 (ESTABLISHED) 2023-11-09T03:56:15.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root txt REG 8,1 12250256 3112 /usr/bin/ceph-mon 2023-11-09T03:56:15.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:15.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:15.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:15.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 1w FIFO 0,13 0t0 180522 pipe 2023-11-09T03:56:15.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 2w FIFO 0,13 0t0 180523 pipe 2023-11-09T03:56:15.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 3r FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 4w FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 5w REG 8,1 68063672 265319 /var/log/ceph/ceph-mon.a.log 2023-11-09T03:56:15.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 6uW REG 0,24 6 1732 /run/ceph/ceph-mon.a.pid 2023-11-09T03:56:15.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 7r FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 8w FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 9u unix 0xffff959a6034b740 0t0 177870 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 10r FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 11w FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 12r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 13uW REG 253,4 0 33816711 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2023-11-09T03:56:15.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 14r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 15u IPv4 250312 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60028 (ESTABLISHED) 2023-11-09T03:56:15.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 16w REG 253,4 1820304 33816715 /var/lib/ceph/mon/ceph-a/store.db/000019.log 2023-11-09T03:56:15.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 17w REG 253,4 898 33816719 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-09T03:56:15.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 18u a_inode 0,14 0 2077 [eventpoll:19,27,28,29,41,44,47] 2023-11-09T03:56:15.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 19r FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 20w FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 21u a_inode 0,14 0 2077 [eventpoll:15,22,38,42] 2023-11-09T03:56:15.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 22r FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 23w FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 24u a_inode 0,14 0 2077 [eventpoll:25,32,40,43,46] 2023-11-09T03:56:15.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 25r FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 26w FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 27u IPv4 178242 0t0 TCP smithi076.front.sepia.ceph.com:3300 (LISTEN) 2023-11-09T03:56:15.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 28u IPv4 178243 0t0 TCP smithi076.front.sepia.ceph.com:smc-https (LISTEN) 2023-11-09T03:56:15.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 29u IPv4 180558 0t0 TCP smithi076.front.sepia.ceph.com:52700->smithi113.front.sepia.ceph.com:3301 (ESTABLISHED) 2023-11-09T03:56:15.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 30r FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 31w FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 32u IPv4 176127 0t0 TCP smithi076.front.sepia.ceph.com:48276->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 33r FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 34w FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 35r FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 36w FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 37w REG 8,1 125211 265320 /var/log/ceph/ceph.log 2023-11-09T03:56:15.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 38u IPv4 182390 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:46708 (ESTABLISHED) 2023-11-09T03:56:15.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 39w REG 8,1 209797 265351 /var/log/ceph/ceph.audit.log 2023-11-09T03:56:15.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 40u IPv4 183510 0t0 TCP smithi076.front.sepia.ceph.com:48946->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 41u IPv4 179814 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60322 (ESTABLISHED) 2023-11-09T03:56:15.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 42u IPv4 179820 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60330 (ESTABLISHED) 2023-11-09T03:56:15.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 43u IPv4 181612 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60338 (ESTABLISHED) 2023-11-09T03:56:15.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 44u IPv4 181618 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60350 (ESTABLISHED) 2023-11-09T03:56:15.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 45r REG 253,4 4632438 33816724 /var/lib/ceph/mon/ceph-a/store.db/000021.sst 2023-11-09T03:56:15.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 46u IPv4 267111 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:50182 (ESTABLISHED) 2023-11-09T03:56:15.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87783 ms_dispat root 47u IPv4 271047 0t0 TCP smithi076.front.sepia.ceph.com:3300->192.168.0.1:37190 (ESTABLISHED) 2023-11-09T03:56:15.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root txt REG 8,1 12250256 3112 /usr/bin/ceph-mon 2023-11-09T03:56:15.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:15.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:15.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:15.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 1w FIFO 0,13 0t0 180522 pipe 2023-11-09T03:56:15.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 2w FIFO 0,13 0t0 180523 pipe 2023-11-09T03:56:15.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 3r FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 4w FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 5w REG 8,1 68063672 265319 /var/log/ceph/ceph-mon.a.log 2023-11-09T03:56:15.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 6uW REG 0,24 6 1732 /run/ceph/ceph-mon.a.pid 2023-11-09T03:56:15.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 7r FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 8w FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 9u unix 0xffff959a6034b740 0t0 177870 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 10r FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 11w FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 12r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 13uW REG 253,4 0 33816711 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2023-11-09T03:56:15.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 14r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 15u IPv4 250312 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60028 (ESTABLISHED) 2023-11-09T03:56:15.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 16w REG 253,4 1820304 33816715 /var/lib/ceph/mon/ceph-a/store.db/000019.log 2023-11-09T03:56:15.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 17w REG 253,4 898 33816719 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-09T03:56:15.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 18u a_inode 0,14 0 2077 [eventpoll:19,27,28,29,41,44,47] 2023-11-09T03:56:15.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 19r FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 20w FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 21u a_inode 0,14 0 2077 [eventpoll:15,22,38,42] 2023-11-09T03:56:15.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 22r FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 23w FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 24u a_inode 0,14 0 2077 [eventpoll:25,32,40,43,46] 2023-11-09T03:56:15.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 25r FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 26w FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 27u IPv4 178242 0t0 TCP smithi076.front.sepia.ceph.com:3300 (LISTEN) 2023-11-09T03:56:15.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 28u IPv4 178243 0t0 TCP smithi076.front.sepia.ceph.com:smc-https (LISTEN) 2023-11-09T03:56:15.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 29u IPv4 180558 0t0 TCP smithi076.front.sepia.ceph.com:52700->smithi113.front.sepia.ceph.com:3301 (ESTABLISHED) 2023-11-09T03:56:15.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 30r FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 31w FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 32u IPv4 176127 0t0 TCP smithi076.front.sepia.ceph.com:48276->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 33r FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 34w FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 35r FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 36w FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 37w REG 8,1 125211 265320 /var/log/ceph/ceph.log 2023-11-09T03:56:15.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 38u IPv4 182390 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:46708 (ESTABLISHED) 2023-11-09T03:56:15.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 39w REG 8,1 209797 265351 /var/log/ceph/ceph.audit.log 2023-11-09T03:56:15.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 40u IPv4 183510 0t0 TCP smithi076.front.sepia.ceph.com:48946->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 41u IPv4 179814 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60322 (ESTABLISHED) 2023-11-09T03:56:15.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 42u IPv4 179820 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60330 (ESTABLISHED) 2023-11-09T03:56:15.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 43u IPv4 181612 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60338 (ESTABLISHED) 2023-11-09T03:56:15.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 44u IPv4 181618 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60350 (ESTABLISHED) 2023-11-09T03:56:15.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 45r REG 253,4 4632438 33816724 /var/lib/ceph/mon/ceph-a/store.db/000021.sst 2023-11-09T03:56:15.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 46u IPv4 267111 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:50182 (ESTABLISHED) 2023-11-09T03:56:15.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87784 ms_local root 47u IPv4 271047 0t0 TCP smithi076.front.sepia.ceph.com:3300->192.168.0.1:37190 (ESTABLISHED) 2023-11-09T03:56:15.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root txt REG 8,1 12250256 3112 /usr/bin/ceph-mon 2023-11-09T03:56:15.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:15.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:15.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:15.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 1w FIFO 0,13 0t0 180522 pipe 2023-11-09T03:56:15.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 2w FIFO 0,13 0t0 180523 pipe 2023-11-09T03:56:15.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 3r FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 4w FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 5w REG 8,1 68063672 265319 /var/log/ceph/ceph-mon.a.log 2023-11-09T03:56:15.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 6uW REG 0,24 6 1732 /run/ceph/ceph-mon.a.pid 2023-11-09T03:56:15.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 7r FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 8w FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 9u unix 0xffff959a6034b740 0t0 177870 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 10r FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 11w FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 12r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 13uW REG 253,4 0 33816711 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2023-11-09T03:56:15.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 14r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 15u IPv4 250312 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60028 (ESTABLISHED) 2023-11-09T03:56:15.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 16w REG 253,4 1820304 33816715 /var/lib/ceph/mon/ceph-a/store.db/000019.log 2023-11-09T03:56:15.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 17w REG 253,4 898 33816719 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-09T03:56:15.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 18u a_inode 0,14 0 2077 [eventpoll:19,27,28,29,41,44,47] 2023-11-09T03:56:15.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 19r FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 20w FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 21u a_inode 0,14 0 2077 [eventpoll:15,22,38,42] 2023-11-09T03:56:15.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 22r FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 23w FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 24u a_inode 0,14 0 2077 [eventpoll:25,32,40,43,46] 2023-11-09T03:56:15.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 25r FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 26w FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 27u IPv4 178242 0t0 TCP smithi076.front.sepia.ceph.com:3300 (LISTEN) 2023-11-09T03:56:15.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 28u IPv4 178243 0t0 TCP smithi076.front.sepia.ceph.com:smc-https (LISTEN) 2023-11-09T03:56:15.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 29u IPv4 180558 0t0 TCP smithi076.front.sepia.ceph.com:52700->smithi113.front.sepia.ceph.com:3301 (ESTABLISHED) 2023-11-09T03:56:15.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 30r FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 31w FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 32u IPv4 176127 0t0 TCP smithi076.front.sepia.ceph.com:48276->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 33r FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 34w FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 35r FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 36w FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 37w REG 8,1 125211 265320 /var/log/ceph/ceph.log 2023-11-09T03:56:15.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 38u IPv4 182390 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:46708 (ESTABLISHED) 2023-11-09T03:56:15.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 39w REG 8,1 209797 265351 /var/log/ceph/ceph.audit.log 2023-11-09T03:56:15.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 40u IPv4 183510 0t0 TCP smithi076.front.sepia.ceph.com:48946->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 41u IPv4 179814 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60322 (ESTABLISHED) 2023-11-09T03:56:15.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 42u IPv4 179820 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60330 (ESTABLISHED) 2023-11-09T03:56:15.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 43u IPv4 181612 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60338 (ESTABLISHED) 2023-11-09T03:56:15.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 44u IPv4 181618 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60350 (ESTABLISHED) 2023-11-09T03:56:15.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 45r REG 253,4 4632438 33816724 /var/lib/ceph/mon/ceph-a/store.db/000021.sst 2023-11-09T03:56:15.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 46u IPv4 267111 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:50182 (ESTABLISHED) 2023-11-09T03:56:15.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87785 safe_time root 47u IPv4 271047 0t0 TCP smithi076.front.sepia.ceph.com:3300->192.168.0.1:37190 (ESTABLISHED) 2023-11-09T03:56:15.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root txt REG 8,1 12250256 3112 /usr/bin/ceph-mon 2023-11-09T03:56:15.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:15.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:15.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:15.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 1w FIFO 0,13 0t0 180522 pipe 2023-11-09T03:56:15.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 2w FIFO 0,13 0t0 180523 pipe 2023-11-09T03:56:15.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 3r FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 4w FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 5w REG 8,1 68063672 265319 /var/log/ceph/ceph-mon.a.log 2023-11-09T03:56:15.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 6uW REG 0,24 6 1732 /run/ceph/ceph-mon.a.pid 2023-11-09T03:56:15.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 7r FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 8w FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 9u unix 0xffff959a6034b740 0t0 177870 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 10r FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 11w FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 12r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 13uW REG 253,4 0 33816711 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2023-11-09T03:56:15.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 14r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 15u IPv4 250312 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60028 (ESTABLISHED) 2023-11-09T03:56:15.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 16w REG 253,4 1820304 33816715 /var/lib/ceph/mon/ceph-a/store.db/000019.log 2023-11-09T03:56:15.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 17w REG 253,4 898 33816719 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-09T03:56:15.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 18u a_inode 0,14 0 2077 [eventpoll:19,27,28,29,41,44,47] 2023-11-09T03:56:15.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 19r FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 20w FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 21u a_inode 0,14 0 2077 [eventpoll:15,22,38,42] 2023-11-09T03:56:15.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 22r FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 23w FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 24u a_inode 0,14 0 2077 [eventpoll:25,32,40,43,46] 2023-11-09T03:56:15.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 25r FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 26w FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 27u IPv4 178242 0t0 TCP smithi076.front.sepia.ceph.com:3300 (LISTEN) 2023-11-09T03:56:15.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 28u IPv4 178243 0t0 TCP smithi076.front.sepia.ceph.com:smc-https (LISTEN) 2023-11-09T03:56:15.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 29u IPv4 180558 0t0 TCP smithi076.front.sepia.ceph.com:52700->smithi113.front.sepia.ceph.com:3301 (ESTABLISHED) 2023-11-09T03:56:15.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 30r FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 31w FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 32u IPv4 176127 0t0 TCP smithi076.front.sepia.ceph.com:48276->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 33r FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 34w FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 35r FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 36w FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 37w REG 8,1 125211 265320 /var/log/ceph/ceph.log 2023-11-09T03:56:15.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 38u IPv4 182390 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:46708 (ESTABLISHED) 2023-11-09T03:56:15.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 39w REG 8,1 209797 265351 /var/log/ceph/ceph.audit.log 2023-11-09T03:56:15.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 40u IPv4 183510 0t0 TCP smithi076.front.sepia.ceph.com:48946->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 41u IPv4 179814 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60322 (ESTABLISHED) 2023-11-09T03:56:15.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 42u IPv4 179820 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60330 (ESTABLISHED) 2023-11-09T03:56:15.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 43u IPv4 181612 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60338 (ESTABLISHED) 2023-11-09T03:56:15.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 44u IPv4 181618 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60350 (ESTABLISHED) 2023-11-09T03:56:15.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 45r REG 253,4 4632438 33816724 /var/lib/ceph/mon/ceph-a/store.db/000021.sst 2023-11-09T03:56:15.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 46u IPv4 267111 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:50182 (ESTABLISHED) 2023-11-09T03:56:15.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87786 ms_dispat root 47u IPv4 271047 0t0 TCP smithi076.front.sepia.ceph.com:3300->192.168.0.1:37190 (ESTABLISHED) 2023-11-09T03:56:15.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root txt REG 8,1 12250256 3112 /usr/bin/ceph-mon 2023-11-09T03:56:15.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:15.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:15.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:15.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 1w FIFO 0,13 0t0 180522 pipe 2023-11-09T03:56:15.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 2w FIFO 0,13 0t0 180523 pipe 2023-11-09T03:56:15.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 3r FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 4w FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 5w REG 8,1 68063672 265319 /var/log/ceph/ceph-mon.a.log 2023-11-09T03:56:15.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 6uW REG 0,24 6 1732 /run/ceph/ceph-mon.a.pid 2023-11-09T03:56:15.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 7r FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 8w FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 9u unix 0xffff959a6034b740 0t0 177870 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 10r FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 11w FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 12r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 13uW REG 253,4 0 33816711 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2023-11-09T03:56:15.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 14r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 15u IPv4 250312 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60028 (ESTABLISHED) 2023-11-09T03:56:15.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 16w REG 253,4 1820304 33816715 /var/lib/ceph/mon/ceph-a/store.db/000019.log 2023-11-09T03:56:15.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 17w REG 253,4 898 33816719 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-09T03:56:15.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 18u a_inode 0,14 0 2077 [eventpoll:19,27,28,29,41,44,47] 2023-11-09T03:56:15.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 19r FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 20w FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 21u a_inode 0,14 0 2077 [eventpoll:15,22,38,42] 2023-11-09T03:56:15.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 22r FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 23w FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 24u a_inode 0,14 0 2077 [eventpoll:25,32,40,43,46] 2023-11-09T03:56:15.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 25r FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 26w FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 27u IPv4 178242 0t0 TCP smithi076.front.sepia.ceph.com:3300 (LISTEN) 2023-11-09T03:56:15.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 28u IPv4 178243 0t0 TCP smithi076.front.sepia.ceph.com:smc-https (LISTEN) 2023-11-09T03:56:15.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 29u IPv4 180558 0t0 TCP smithi076.front.sepia.ceph.com:52700->smithi113.front.sepia.ceph.com:3301 (ESTABLISHED) 2023-11-09T03:56:15.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 30r FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 31w FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 32u IPv4 176127 0t0 TCP smithi076.front.sepia.ceph.com:48276->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 33r FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 34w FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 35r FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 36w FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 37w REG 8,1 125211 265320 /var/log/ceph/ceph.log 2023-11-09T03:56:15.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 38u IPv4 182390 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:46708 (ESTABLISHED) 2023-11-09T03:56:15.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 39w REG 8,1 209797 265351 /var/log/ceph/ceph.audit.log 2023-11-09T03:56:15.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 40u IPv4 183510 0t0 TCP smithi076.front.sepia.ceph.com:48946->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 41u IPv4 179814 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60322 (ESTABLISHED) 2023-11-09T03:56:15.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 42u IPv4 179820 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60330 (ESTABLISHED) 2023-11-09T03:56:15.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 43u IPv4 181612 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60338 (ESTABLISHED) 2023-11-09T03:56:15.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 44u IPv4 181618 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60350 (ESTABLISHED) 2023-11-09T03:56:15.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 45r REG 253,4 4632438 33816724 /var/lib/ceph/mon/ceph-a/store.db/000021.sst 2023-11-09T03:56:15.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 46u IPv4 267111 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:50182 (ESTABLISHED) 2023-11-09T03:56:15.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 87787 ms_local root 47u IPv4 271047 0t0 TCP smithi076.front.sepia.ceph.com:3300->192.168.0.1:37190 (ESTABLISHED) 2023-11-09T03:56:15.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root txt REG 8,1 12250256 3112 /usr/bin/ceph-mon 2023-11-09T03:56:15.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:15.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:15.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:15.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:15.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 1w FIFO 0,13 0t0 180522 pipe 2023-11-09T03:56:15.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 2w FIFO 0,13 0t0 180523 pipe 2023-11-09T03:56:15.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 3r FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 4w FIFO 0,13 0t0 177867 pipe 2023-11-09T03:56:15.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 5w REG 8,1 68063672 265319 /var/log/ceph/ceph-mon.a.log 2023-11-09T03:56:15.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 6uW REG 0,24 6 1732 /run/ceph/ceph-mon.a.pid 2023-11-09T03:56:15.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 7r FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 8w FIFO 0,13 0t0 177869 pipe 2023-11-09T03:56:15.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 9u unix 0xffff959a6034b740 0t0 177870 /var/run/ceph/ceph-mon.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 10r FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 11w FIFO 0,13 0t0 177871 pipe 2023-11-09T03:56:15.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 12r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 13uW REG 253,4 0 33816711 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2023-11-09T03:56:15.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 14r DIR 253,4 4096 33816710 /var/lib/ceph/mon/ceph-a/store.db 2023-11-09T03:56:15.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 15u IPv4 250312 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60028 (ESTABLISHED) 2023-11-09T03:56:15.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 16w REG 253,4 1820304 33816715 /var/lib/ceph/mon/ceph-a/store.db/000019.log 2023-11-09T03:56:15.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 17w REG 253,4 898 33816719 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-09T03:56:15.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 18u a_inode 0,14 0 2077 [eventpoll:19,27,28,29,41,44,47] 2023-11-09T03:56:15.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 19r FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 20w FIFO 0,13 0t0 177877 pipe 2023-11-09T03:56:15.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 21u a_inode 0,14 0 2077 [eventpoll:15,22,38,42] 2023-11-09T03:56:15.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 22r FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 23w FIFO 0,13 0t0 177878 pipe 2023-11-09T03:56:15.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 24u a_inode 0,14 0 2077 [eventpoll:25,32,40,43,46] 2023-11-09T03:56:15.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 25r FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 26w FIFO 0,13 0t0 177879 pipe 2023-11-09T03:56:15.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 27u IPv4 178242 0t0 TCP smithi076.front.sepia.ceph.com:3300 (LISTEN) 2023-11-09T03:56:15.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 28u IPv4 178243 0t0 TCP smithi076.front.sepia.ceph.com:smc-https (LISTEN) 2023-11-09T03:56:15.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 29u IPv4 180558 0t0 TCP smithi076.front.sepia.ceph.com:52700->smithi113.front.sepia.ceph.com:3301 (ESTABLISHED) 2023-11-09T03:56:15.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 30r FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 31w FIFO 0,13 0t0 176123 pipe 2023-11-09T03:56:15.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 32u IPv4 176127 0t0 TCP smithi076.front.sepia.ceph.com:48276->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 33r FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 34w FIFO 0,13 0t0 176124 pipe 2023-11-09T03:56:15.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 35r FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 36w FIFO 0,13 0t0 176125 pipe 2023-11-09T03:56:15.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 37w REG 8,1 125211 265320 /var/log/ceph/ceph.log 2023-11-09T03:56:15.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 38u IPv4 182390 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:46708 (ESTABLISHED) 2023-11-09T03:56:15.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 39w REG 8,1 209797 265351 /var/log/ceph/ceph.audit.log 2023-11-09T03:56:15.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 40u IPv4 183510 0t0 TCP smithi076.front.sepia.ceph.com:48946->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 41u IPv4 179814 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60322 (ESTABLISHED) 2023-11-09T03:56:15.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 42u IPv4 179820 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60330 (ESTABLISHED) 2023-11-09T03:56:15.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 43u IPv4 181612 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60338 (ESTABLISHED) 2023-11-09T03:56:15.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 44u IPv4 181618 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:60350 (ESTABLISHED) 2023-11-09T03:56:15.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 45r REG 253,4 4632438 33816724 /var/lib/ceph/mon/ceph-a/store.db/000021.sst 2023-11-09T03:56:15.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 46u IPv4 267111 0t0 TCP smithi076.front.sepia.ceph.com:3300->smithi076.front.sepia.ceph.com:50182 (ESTABLISHED) 2023-11-09T03:56:15.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mon 87729 121891 rstore_co root 47u IPv4 271047 0t0 TCP smithi076.front.sepia.ceph.com:3300->192.168.0.1:37190 (ESTABLISHED) 2023-11-09T03:56:15.607 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87747 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.607 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87747 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.608 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87747 root txt REG 8,1 15448 10498 /usr/bin/python3.9 2023-11-09T03:56:15.608 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87747 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:15.608 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87747 root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:15.608 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87747 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.608 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87747 root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.608 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87747 root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.609 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87747 root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.609 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87747 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.609 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87747 root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.609 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87747 root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.609 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87747 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.609 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87747 root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:15.610 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87747 root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.610 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87747 root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.610 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87747 root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.610 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87747 root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.610 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87747 root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.610 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87747 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:15.611 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87747 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:15.611 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87747 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.611 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87747 root 0r FIFO 0,13 0t0 180529 pipe 2023-11-09T03:56:15.611 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87747 root 1w FIFO 0,13 0t0 180530 pipe 2023-11-09T03:56:15.611 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87747 root 2w FIFO 0,13 0t0 180531 pipe 2023-11-09T03:56:15.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root txt REG 8,1 4625312 2939 /usr/bin/ceph-mgr 2023-11-09T03:56:15.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-11-09T03:56:15.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 64192 38198 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 65240 274077 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 783584 273412 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 87608 273415 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 331936 273390 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 654400 273423 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 41112936 2759 /usr/lib64/libopenblaso-r0.3.21.so 2023-11-09T03:56:15.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 60264 273417 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 96304 273413 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 237384 273392 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 177920 273418 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 7042240 272781 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2023-11-09T03:56:15.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 70784 273414 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 73976 273198 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 155704 273300 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 2883920 2473 /usr/lib64/libgfortran.so.5.0.0 2023-11-09T03:56:15.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 3870704 2771 /usr/lib64/libflexiblas.so.3.0 2023-11-09T03:56:15.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 3784896 272831 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 132136 272830 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 294392 9898 /usr/lib64/libgomp.so.1.0.0 2023-11-09T03:56:15.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 382824 272782 /usr/lib64/flexiblas/libflexiblas_netlib.so 2023-11-09T03:56:15.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 299952 2469 /usr/lib64/libquadmath.so.0.0.0 2023-11-09T03:56:15.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 8003360 1775 /usr/lib64/librbd.so.1.18.0 2023-11-09T03:56:15.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 31984 42006 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 15336 38209 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 20504 273301 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 40328 272779 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2023-11-09T03:56:15.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 78840 41984 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 75928 9784 /usr/lib64/libjson-c.so.5.0.0 2023-11-09T03:56:15.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 384592 11464 /usr/lib64/libdevmapper.so.1.02 2023-11-09T03:56:15.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 554440 11254 /usr/lib64/libcryptsetup.so.12.9.0 2023-11-09T03:56:15.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 1129568 2920 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 130464 38212 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:15.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 71408 38249 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 38744 38211 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 136048 11808 /usr/lib64/libyaml-0.so.2.0.9 2023-11-09T03:56:15.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 20832 38226 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 246392 59769 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 119688 36903 /usr/lib64/ossl-modules/legacy.so 2023-11-09T03:56:15.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 721976 270244 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2023-11-09T03:56:15.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 2154416 270246 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2023-11-09T03:56:15.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 1116856 42007 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 124184 41989 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 16192 265939 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 283592 38217 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 213376 41995 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 20528 41991 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 41384 38243 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 24160 41992 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:15.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:15.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 2103760 2040 /usr/lib64/libcephfs.so.2.0.0 2023-11-09T03:56:15.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 515320 2482 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:15.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 198136 2576 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 49328 270783 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2023-11-09T03:56:15.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 50232 41979 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 69048 38220 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 36328 9712 /usr/lib64/libuuid.so.1.3.0 2023-11-09T03:56:15.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 15984 38200 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 96936 41994 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 68000 38241 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 101208 41993 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 107736 38215 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 1129032 2042 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 50048 41986 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:15.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 15504 41997 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:15.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 1892920 1771 /usr/lib64/librados.so.2.0.0 2023-11-09T03:56:15.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:15.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 1294848 9721 /usr/lib64/libsqlite3.so.0.8.6 2023-11-09T03:56:15.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 435816 2332 /usr/lib64/libcephsqlite.so 2023-11-09T03:56:15.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 15560 38227 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root 1w FIFO 0,13 0t0 180530 pipe 2023-11-09T03:56:15.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root 2w FIFO 0,13 0t0 180531 pipe 2023-11-09T03:56:15.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root 3r FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:15.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root 4w FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:15.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root 5u a_inode 0,14 0 2077 [eventpoll:6,25] 2023-11-09T03:56:15.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root 6r FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:15.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root 7w FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:15.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root 8u a_inode 0,14 0 2077 [eventpoll:9,23] 2023-11-09T03:56:15.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root 9r FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:15.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root 10w FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:15.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root 11u a_inode 0,14 0 2077 [eventpoll:12] 2023-11-09T03:56:15.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root 12r FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:15.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root 13w FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:15.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root 14w REG 8,1 809589 265321 /var/log/ceph/ceph-mgr.y.log 2023-11-09T03:56:15.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root 15uW REG 0,24 6 1735 /run/ceph/ceph-mgr.y.pid 2023-11-09T03:56:15.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root 16r FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:15.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root 17w FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:15.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root 18u unix 0xffff959a60349540 0t0 177902 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root 19r FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:15.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root 20w FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:15.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root 21r FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:15.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root 22w FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:15.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root 23u IPv4 185569 0t0 TCP smithi076.front.sepia.ceph.com:44172->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 root 25u IPv4 183326 0t0 TCP smithi076.front.sepia.ceph.com:46708->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root txt REG 8,1 4625312 2939 /usr/bin/ceph-mgr 2023-11-09T03:56:15.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-11-09T03:56:15.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 64192 38198 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 65240 274077 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 783584 273412 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 87608 273415 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 331936 273390 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 654400 273423 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 41112936 2759 /usr/lib64/libopenblaso-r0.3.21.so 2023-11-09T03:56:15.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 60264 273417 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 96304 273413 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 237384 273392 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 177920 273418 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 7042240 272781 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2023-11-09T03:56:15.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 70784 273414 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 73976 273198 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 155704 273300 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 2883920 2473 /usr/lib64/libgfortran.so.5.0.0 2023-11-09T03:56:15.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 3870704 2771 /usr/lib64/libflexiblas.so.3.0 2023-11-09T03:56:15.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 3784896 272831 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 132136 272830 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 294392 9898 /usr/lib64/libgomp.so.1.0.0 2023-11-09T03:56:15.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 382824 272782 /usr/lib64/flexiblas/libflexiblas_netlib.so 2023-11-09T03:56:15.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 299952 2469 /usr/lib64/libquadmath.so.0.0.0 2023-11-09T03:56:15.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 8003360 1775 /usr/lib64/librbd.so.1.18.0 2023-11-09T03:56:15.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 31984 42006 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 15336 38209 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 20504 273301 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 40328 272779 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2023-11-09T03:56:15.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 78840 41984 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 75928 9784 /usr/lib64/libjson-c.so.5.0.0 2023-11-09T03:56:15.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 384592 11464 /usr/lib64/libdevmapper.so.1.02 2023-11-09T03:56:15.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 554440 11254 /usr/lib64/libcryptsetup.so.12.9.0 2023-11-09T03:56:15.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 1129568 2920 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 130464 38212 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:15.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 71408 38249 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 38744 38211 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 136048 11808 /usr/lib64/libyaml-0.so.2.0.9 2023-11-09T03:56:15.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 20832 38226 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 246392 59769 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 119688 36903 /usr/lib64/ossl-modules/legacy.so 2023-11-09T03:56:15.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 721976 270244 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2023-11-09T03:56:15.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 2154416 270246 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2023-11-09T03:56:15.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 1116856 42007 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 124184 41989 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 16192 265939 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 283592 38217 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 213376 41995 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 20528 41991 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 41384 38243 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 24160 41992 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:15.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:15.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 2103760 2040 /usr/lib64/libcephfs.so.2.0.0 2023-11-09T03:56:15.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 515320 2482 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:15.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 198136 2576 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 49328 270783 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2023-11-09T03:56:15.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 50232 41979 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 69048 38220 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 36328 9712 /usr/lib64/libuuid.so.1.3.0 2023-11-09T03:56:15.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 15984 38200 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 96936 41994 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 68000 38241 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 101208 41993 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 107736 38215 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 1129032 2042 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 50048 41986 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:15.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 15504 41997 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:15.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 1892920 1771 /usr/lib64/librados.so.2.0.0 2023-11-09T03:56:15.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:15.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 1294848 9721 /usr/lib64/libsqlite3.so.0.8.6 2023-11-09T03:56:15.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 435816 2332 /usr/lib64/libcephsqlite.so 2023-11-09T03:56:15.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 15560 38227 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root 1w FIFO 0,13 0t0 180530 pipe 2023-11-09T03:56:15.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root 2w FIFO 0,13 0t0 180531 pipe 2023-11-09T03:56:15.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root 3r FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:15.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root 4w FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:15.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root 5u a_inode 0,14 0 2077 [eventpoll:6,25] 2023-11-09T03:56:15.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root 6r FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:15.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root 7w FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:15.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root 8u a_inode 0,14 0 2077 [eventpoll:9,23] 2023-11-09T03:56:15.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root 9r FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:15.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root 10w FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:15.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root 11u a_inode 0,14 0 2077 [eventpoll:12] 2023-11-09T03:56:15.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root 12r FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:15.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root 13w FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:15.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root 14w REG 8,1 809589 265321 /var/log/ceph/ceph-mgr.y.log 2023-11-09T03:56:15.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root 15uW REG 0,24 6 1735 /run/ceph/ceph-mgr.y.pid 2023-11-09T03:56:15.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root 16r FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:15.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root 17w FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:15.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root 18u unix 0xffff959a60349540 0t0 177902 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root 19r FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:15.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root 20w FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:15.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root 21r FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:15.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root 22w FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:15.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root 23u IPv4 185569 0t0 TCP smithi076.front.sepia.ceph.com:44172->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87789 log root 25u IPv4 183326 0t0 TCP smithi076.front.sepia.ceph.com:46708->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root txt REG 8,1 4625312 2939 /usr/bin/ceph-mgr 2023-11-09T03:56:15.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-11-09T03:56:15.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 64192 38198 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 65240 274077 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 783584 273412 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 87608 273415 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 331936 273390 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 654400 273423 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 41112936 2759 /usr/lib64/libopenblaso-r0.3.21.so 2023-11-09T03:56:15.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 60264 273417 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 96304 273413 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 237384 273392 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 177920 273418 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 7042240 272781 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2023-11-09T03:56:15.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 70784 273414 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 73976 273198 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 155704 273300 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 2883920 2473 /usr/lib64/libgfortran.so.5.0.0 2023-11-09T03:56:15.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 3870704 2771 /usr/lib64/libflexiblas.so.3.0 2023-11-09T03:56:15.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 3784896 272831 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 132136 272830 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 294392 9898 /usr/lib64/libgomp.so.1.0.0 2023-11-09T03:56:15.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 382824 272782 /usr/lib64/flexiblas/libflexiblas_netlib.so 2023-11-09T03:56:15.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 299952 2469 /usr/lib64/libquadmath.so.0.0.0 2023-11-09T03:56:15.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 8003360 1775 /usr/lib64/librbd.so.1.18.0 2023-11-09T03:56:15.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 31984 42006 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 15336 38209 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 20504 273301 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 40328 272779 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2023-11-09T03:56:15.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 78840 41984 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 75928 9784 /usr/lib64/libjson-c.so.5.0.0 2023-11-09T03:56:15.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 384592 11464 /usr/lib64/libdevmapper.so.1.02 2023-11-09T03:56:15.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 554440 11254 /usr/lib64/libcryptsetup.so.12.9.0 2023-11-09T03:56:15.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 1129568 2920 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 130464 38212 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:15.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 71408 38249 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 38744 38211 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 136048 11808 /usr/lib64/libyaml-0.so.2.0.9 2023-11-09T03:56:15.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 20832 38226 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 246392 59769 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 119688 36903 /usr/lib64/ossl-modules/legacy.so 2023-11-09T03:56:15.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 721976 270244 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2023-11-09T03:56:15.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 2154416 270246 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2023-11-09T03:56:15.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 1116856 42007 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 124184 41989 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 16192 265939 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 283592 38217 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 213376 41995 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 20528 41991 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 41384 38243 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 24160 41992 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:15.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:15.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 2103760 2040 /usr/lib64/libcephfs.so.2.0.0 2023-11-09T03:56:15.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 515320 2482 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:15.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 198136 2576 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 49328 270783 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2023-11-09T03:56:15.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 50232 41979 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 69048 38220 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 36328 9712 /usr/lib64/libuuid.so.1.3.0 2023-11-09T03:56:15.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 15984 38200 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 96936 41994 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 68000 38241 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 101208 41993 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 107736 38215 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 1129032 2042 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 50048 41986 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:15.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 15504 41997 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:15.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 1892920 1771 /usr/lib64/librados.so.2.0.0 2023-11-09T03:56:15.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:15.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 1294848 9721 /usr/lib64/libsqlite3.so.0.8.6 2023-11-09T03:56:15.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 435816 2332 /usr/lib64/libcephsqlite.so 2023-11-09T03:56:15.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 15560 38227 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root 1w FIFO 0,13 0t0 180530 pipe 2023-11-09T03:56:15.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root 2w FIFO 0,13 0t0 180531 pipe 2023-11-09T03:56:15.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root 3r FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:15.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root 4w FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:15.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root 5u a_inode 0,14 0 2077 [eventpoll:6,25] 2023-11-09T03:56:15.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root 6r FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:15.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root 7w FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:15.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root 8u a_inode 0,14 0 2077 [eventpoll:9,23] 2023-11-09T03:56:15.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root 9r FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:15.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root 10w FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:15.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root 11u a_inode 0,14 0 2077 [eventpoll:12] 2023-11-09T03:56:15.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root 12r FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:15.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root 13w FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:15.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root 14w REG 8,1 809589 265321 /var/log/ceph/ceph-mgr.y.log 2023-11-09T03:56:15.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root 15uW REG 0,24 6 1735 /run/ceph/ceph-mgr.y.pid 2023-11-09T03:56:15.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root 16r FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:15.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root 17w FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:15.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root 18u unix 0xffff959a60349540 0t0 177902 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root 19r FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:15.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root 20w FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:15.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root 21r FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:15.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root 22w FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:15.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root 23u IPv4 185569 0t0 TCP smithi076.front.sepia.ceph.com:44172->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87791 msgr-work root 25u IPv4 183326 0t0 TCP smithi076.front.sepia.ceph.com:46708->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root txt REG 8,1 4625312 2939 /usr/bin/ceph-mgr 2023-11-09T03:56:15.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-11-09T03:56:15.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 64192 38198 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 65240 274077 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 783584 273412 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 87608 273415 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 331936 273390 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 654400 273423 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 41112936 2759 /usr/lib64/libopenblaso-r0.3.21.so 2023-11-09T03:56:15.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 60264 273417 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 96304 273413 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 237384 273392 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 177920 273418 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 7042240 272781 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2023-11-09T03:56:15.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 70784 273414 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 73976 273198 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 155704 273300 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 2883920 2473 /usr/lib64/libgfortran.so.5.0.0 2023-11-09T03:56:15.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 3870704 2771 /usr/lib64/libflexiblas.so.3.0 2023-11-09T03:56:15.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 3784896 272831 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 132136 272830 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 294392 9898 /usr/lib64/libgomp.so.1.0.0 2023-11-09T03:56:15.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 382824 272782 /usr/lib64/flexiblas/libflexiblas_netlib.so 2023-11-09T03:56:15.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 299952 2469 /usr/lib64/libquadmath.so.0.0.0 2023-11-09T03:56:15.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 8003360 1775 /usr/lib64/librbd.so.1.18.0 2023-11-09T03:56:15.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 31984 42006 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 15336 38209 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 20504 273301 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 40328 272779 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2023-11-09T03:56:15.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 78840 41984 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 75928 9784 /usr/lib64/libjson-c.so.5.0.0 2023-11-09T03:56:15.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 384592 11464 /usr/lib64/libdevmapper.so.1.02 2023-11-09T03:56:15.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 554440 11254 /usr/lib64/libcryptsetup.so.12.9.0 2023-11-09T03:56:15.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 1129568 2920 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 130464 38212 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:15.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 71408 38249 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 38744 38211 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 136048 11808 /usr/lib64/libyaml-0.so.2.0.9 2023-11-09T03:56:15.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 20832 38226 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 246392 59769 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 119688 36903 /usr/lib64/ossl-modules/legacy.so 2023-11-09T03:56:15.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 721976 270244 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2023-11-09T03:56:15.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 2154416 270246 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2023-11-09T03:56:15.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 1116856 42007 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 124184 41989 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 16192 265939 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 283592 38217 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 213376 41995 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 20528 41991 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 41384 38243 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 24160 41992 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:15.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:15.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 2103760 2040 /usr/lib64/libcephfs.so.2.0.0 2023-11-09T03:56:15.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 515320 2482 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:15.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 198136 2576 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 49328 270783 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2023-11-09T03:56:15.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 50232 41979 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 69048 38220 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 36328 9712 /usr/lib64/libuuid.so.1.3.0 2023-11-09T03:56:15.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 15984 38200 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 96936 41994 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 68000 38241 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 101208 41993 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 107736 38215 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 1129032 2042 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 50048 41986 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:15.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 15504 41997 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:15.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 1892920 1771 /usr/lib64/librados.so.2.0.0 2023-11-09T03:56:15.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:15.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 1294848 9721 /usr/lib64/libsqlite3.so.0.8.6 2023-11-09T03:56:15.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 435816 2332 /usr/lib64/libcephsqlite.so 2023-11-09T03:56:15.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 15560 38227 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root 1w FIFO 0,13 0t0 180530 pipe 2023-11-09T03:56:15.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root 2w FIFO 0,13 0t0 180531 pipe 2023-11-09T03:56:15.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root 3r FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:15.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root 4w FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:15.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root 5u a_inode 0,14 0 2077 [eventpoll:6,25] 2023-11-09T03:56:15.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root 6r FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:15.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root 7w FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:15.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root 8u a_inode 0,14 0 2077 [eventpoll:9,23] 2023-11-09T03:56:15.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root 9r FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:15.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root 10w FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:15.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root 11u a_inode 0,14 0 2077 [eventpoll:12] 2023-11-09T03:56:15.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root 12r FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:15.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root 13w FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:15.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root 14w REG 8,1 809589 265321 /var/log/ceph/ceph-mgr.y.log 2023-11-09T03:56:15.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root 15uW REG 0,24 6 1735 /run/ceph/ceph-mgr.y.pid 2023-11-09T03:56:15.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root 16r FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:15.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root 17w FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:15.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root 18u unix 0xffff959a60349540 0t0 177902 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root 19r FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:15.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root 20w FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:15.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root 21r FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:15.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root 22w FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:15.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root 23u IPv4 185569 0t0 TCP smithi076.front.sepia.ceph.com:44172->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87792 msgr-work root 25u IPv4 183326 0t0 TCP smithi076.front.sepia.ceph.com:46708->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root txt REG 8,1 4625312 2939 /usr/bin/ceph-mgr 2023-11-09T03:56:15.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-11-09T03:56:15.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 64192 38198 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 65240 274077 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 783584 273412 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 87608 273415 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 331936 273390 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 654400 273423 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 41112936 2759 /usr/lib64/libopenblaso-r0.3.21.so 2023-11-09T03:56:15.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 60264 273417 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 96304 273413 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 237384 273392 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 177920 273418 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 7042240 272781 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2023-11-09T03:56:15.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 70784 273414 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 73976 273198 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 155704 273300 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 2883920 2473 /usr/lib64/libgfortran.so.5.0.0 2023-11-09T03:56:15.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 3870704 2771 /usr/lib64/libflexiblas.so.3.0 2023-11-09T03:56:15.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 3784896 272831 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 132136 272830 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 294392 9898 /usr/lib64/libgomp.so.1.0.0 2023-11-09T03:56:15.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 382824 272782 /usr/lib64/flexiblas/libflexiblas_netlib.so 2023-11-09T03:56:15.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 299952 2469 /usr/lib64/libquadmath.so.0.0.0 2023-11-09T03:56:15.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 8003360 1775 /usr/lib64/librbd.so.1.18.0 2023-11-09T03:56:15.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 31984 42006 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 15336 38209 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 20504 273301 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 40328 272779 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2023-11-09T03:56:15.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 78840 41984 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 75928 9784 /usr/lib64/libjson-c.so.5.0.0 2023-11-09T03:56:15.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 384592 11464 /usr/lib64/libdevmapper.so.1.02 2023-11-09T03:56:15.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 554440 11254 /usr/lib64/libcryptsetup.so.12.9.0 2023-11-09T03:56:15.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 1129568 2920 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 130464 38212 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:15.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 71408 38249 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 38744 38211 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 136048 11808 /usr/lib64/libyaml-0.so.2.0.9 2023-11-09T03:56:15.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 20832 38226 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 246392 59769 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 119688 36903 /usr/lib64/ossl-modules/legacy.so 2023-11-09T03:56:15.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 721976 270244 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2023-11-09T03:56:15.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 2154416 270246 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2023-11-09T03:56:15.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 1116856 42007 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 124184 41989 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 16192 265939 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 283592 38217 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 213376 41995 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 20528 41991 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 41384 38243 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 24160 41992 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:15.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:15.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 2103760 2040 /usr/lib64/libcephfs.so.2.0.0 2023-11-09T03:56:15.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 515320 2482 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:15.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 198136 2576 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 49328 270783 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2023-11-09T03:56:15.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 50232 41979 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 69048 38220 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 36328 9712 /usr/lib64/libuuid.so.1.3.0 2023-11-09T03:56:15.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 15984 38200 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 96936 41994 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 68000 38241 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 101208 41993 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 107736 38215 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 1129032 2042 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 50048 41986 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:15.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 15504 41997 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:15.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 1892920 1771 /usr/lib64/librados.so.2.0.0 2023-11-09T03:56:15.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:15.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 1294848 9721 /usr/lib64/libsqlite3.so.0.8.6 2023-11-09T03:56:15.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 435816 2332 /usr/lib64/libcephsqlite.so 2023-11-09T03:56:15.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 15560 38227 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root 1w FIFO 0,13 0t0 180530 pipe 2023-11-09T03:56:15.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root 2w FIFO 0,13 0t0 180531 pipe 2023-11-09T03:56:15.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root 3r FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:15.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root 4w FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:15.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root 5u a_inode 0,14 0 2077 [eventpoll:6,25] 2023-11-09T03:56:15.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root 6r FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:15.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root 7w FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:15.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root 8u a_inode 0,14 0 2077 [eventpoll:9,23] 2023-11-09T03:56:15.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root 9r FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:15.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root 10w FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:15.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root 11u a_inode 0,14 0 2077 [eventpoll:12] 2023-11-09T03:56:15.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root 12r FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:15.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root 13w FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:15.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root 14w REG 8,1 809589 265321 /var/log/ceph/ceph-mgr.y.log 2023-11-09T03:56:15.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root 15uW REG 0,24 6 1735 /run/ceph/ceph-mgr.y.pid 2023-11-09T03:56:15.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root 16r FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:15.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root 17w FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:15.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root 18u unix 0xffff959a60349540 0t0 177902 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root 19r FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:15.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root 20w FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:15.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root 21r FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:15.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root 22w FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:15.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root 23u IPv4 185569 0t0 TCP smithi076.front.sepia.ceph.com:44172->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87793 msgr-work root 25u IPv4 183326 0t0 TCP smithi076.front.sepia.ceph.com:46708->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root txt REG 8,1 4625312 2939 /usr/bin/ceph-mgr 2023-11-09T03:56:15.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-11-09T03:56:15.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 64192 38198 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 65240 274077 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 783584 273412 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 87608 273415 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 331936 273390 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 654400 273423 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 41112936 2759 /usr/lib64/libopenblaso-r0.3.21.so 2023-11-09T03:56:15.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 60264 273417 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 96304 273413 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 237384 273392 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 177920 273418 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 7042240 272781 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2023-11-09T03:56:15.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 70784 273414 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 73976 273198 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 155704 273300 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 2883920 2473 /usr/lib64/libgfortran.so.5.0.0 2023-11-09T03:56:15.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 3870704 2771 /usr/lib64/libflexiblas.so.3.0 2023-11-09T03:56:15.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 3784896 272831 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 132136 272830 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 294392 9898 /usr/lib64/libgomp.so.1.0.0 2023-11-09T03:56:15.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 382824 272782 /usr/lib64/flexiblas/libflexiblas_netlib.so 2023-11-09T03:56:15.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 299952 2469 /usr/lib64/libquadmath.so.0.0.0 2023-11-09T03:56:15.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 8003360 1775 /usr/lib64/librbd.so.1.18.0 2023-11-09T03:56:15.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 31984 42006 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 15336 38209 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 20504 273301 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 40328 272779 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2023-11-09T03:56:15.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 78840 41984 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 75928 9784 /usr/lib64/libjson-c.so.5.0.0 2023-11-09T03:56:15.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 384592 11464 /usr/lib64/libdevmapper.so.1.02 2023-11-09T03:56:15.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 554440 11254 /usr/lib64/libcryptsetup.so.12.9.0 2023-11-09T03:56:15.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 1129568 2920 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 130464 38212 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:15.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 71408 38249 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 38744 38211 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 136048 11808 /usr/lib64/libyaml-0.so.2.0.9 2023-11-09T03:56:15.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 20832 38226 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 246392 59769 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 119688 36903 /usr/lib64/ossl-modules/legacy.so 2023-11-09T03:56:15.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 721976 270244 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2023-11-09T03:56:15.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 2154416 270246 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2023-11-09T03:56:15.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 1116856 42007 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 124184 41989 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 16192 265939 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 283592 38217 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 213376 41995 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 20528 41991 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 41384 38243 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 24160 41992 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:15.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:15.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 2103760 2040 /usr/lib64/libcephfs.so.2.0.0 2023-11-09T03:56:15.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 515320 2482 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:15.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 198136 2576 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 49328 270783 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2023-11-09T03:56:15.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 50232 41979 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 69048 38220 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 36328 9712 /usr/lib64/libuuid.so.1.3.0 2023-11-09T03:56:15.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 15984 38200 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 96936 41994 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 68000 38241 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 101208 41993 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 107736 38215 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 1129032 2042 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 50048 41986 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:15.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 15504 41997 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:15.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 1892920 1771 /usr/lib64/librados.so.2.0.0 2023-11-09T03:56:15.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:15.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 1294848 9721 /usr/lib64/libsqlite3.so.0.8.6 2023-11-09T03:56:15.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 435816 2332 /usr/lib64/libcephsqlite.so 2023-11-09T03:56:15.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 15560 38227 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root 1w FIFO 0,13 0t0 180530 pipe 2023-11-09T03:56:15.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root 2w FIFO 0,13 0t0 180531 pipe 2023-11-09T03:56:15.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root 3r FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:15.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root 4w FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:15.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root 5u a_inode 0,14 0 2077 [eventpoll:6,25] 2023-11-09T03:56:15.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root 6r FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:15.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root 7w FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:15.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root 8u a_inode 0,14 0 2077 [eventpoll:9,23] 2023-11-09T03:56:15.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root 9r FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:15.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root 10w FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:15.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root 11u a_inode 0,14 0 2077 [eventpoll:12] 2023-11-09T03:56:15.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root 12r FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:15.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root 13w FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:15.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root 14w REG 8,1 809589 265321 /var/log/ceph/ceph-mgr.y.log 2023-11-09T03:56:15.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root 15uW REG 0,24 6 1735 /run/ceph/ceph-mgr.y.pid 2023-11-09T03:56:15.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root 16r FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:15.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root 17w FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:15.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root 18u unix 0xffff959a60349540 0t0 177902 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root 19r FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:15.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root 20w FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:15.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root 21r FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:15.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root 22w FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:15.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root 23u IPv4 185569 0t0 TCP smithi076.front.sepia.ceph.com:44172->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87891 service root 25u IPv4 183326 0t0 TCP smithi076.front.sepia.ceph.com:46708->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root txt REG 8,1 4625312 2939 /usr/bin/ceph-mgr 2023-11-09T03:56:15.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-11-09T03:56:15.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 64192 38198 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 65240 274077 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 783584 273412 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 87608 273415 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 331936 273390 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 654400 273423 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 41112936 2759 /usr/lib64/libopenblaso-r0.3.21.so 2023-11-09T03:56:15.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 60264 273417 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 96304 273413 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 237384 273392 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 177920 273418 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 7042240 272781 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2023-11-09T03:56:15.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 70784 273414 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 73976 273198 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 155704 273300 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 2883920 2473 /usr/lib64/libgfortran.so.5.0.0 2023-11-09T03:56:15.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 3870704 2771 /usr/lib64/libflexiblas.so.3.0 2023-11-09T03:56:15.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 3784896 272831 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 132136 272830 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 294392 9898 /usr/lib64/libgomp.so.1.0.0 2023-11-09T03:56:15.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 382824 272782 /usr/lib64/flexiblas/libflexiblas_netlib.so 2023-11-09T03:56:15.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 299952 2469 /usr/lib64/libquadmath.so.0.0.0 2023-11-09T03:56:15.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 8003360 1775 /usr/lib64/librbd.so.1.18.0 2023-11-09T03:56:15.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 31984 42006 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 15336 38209 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 20504 273301 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 40328 272779 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2023-11-09T03:56:15.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 78840 41984 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 75928 9784 /usr/lib64/libjson-c.so.5.0.0 2023-11-09T03:56:15.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 384592 11464 /usr/lib64/libdevmapper.so.1.02 2023-11-09T03:56:15.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 554440 11254 /usr/lib64/libcryptsetup.so.12.9.0 2023-11-09T03:56:15.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 1129568 2920 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 130464 38212 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:15.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 71408 38249 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 38744 38211 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 136048 11808 /usr/lib64/libyaml-0.so.2.0.9 2023-11-09T03:56:15.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 20832 38226 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 246392 59769 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 119688 36903 /usr/lib64/ossl-modules/legacy.so 2023-11-09T03:56:15.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 721976 270244 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2023-11-09T03:56:15.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 2154416 270246 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2023-11-09T03:56:15.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 1116856 42007 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 124184 41989 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 16192 265939 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 283592 38217 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 213376 41995 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 20528 41991 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 41384 38243 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 24160 41992 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:15.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:15.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 2103760 2040 /usr/lib64/libcephfs.so.2.0.0 2023-11-09T03:56:15.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 515320 2482 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:15.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 198136 2576 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 49328 270783 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2023-11-09T03:56:15.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 50232 41979 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 69048 38220 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 36328 9712 /usr/lib64/libuuid.so.1.3.0 2023-11-09T03:56:15.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 15984 38200 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 96936 41994 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 68000 38241 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 101208 41993 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 107736 38215 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 1129032 2042 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 50048 41986 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:15.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 15504 41997 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:15.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 1892920 1771 /usr/lib64/librados.so.2.0.0 2023-11-09T03:56:15.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:15.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 1294848 9721 /usr/lib64/libsqlite3.so.0.8.6 2023-11-09T03:56:15.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 435816 2332 /usr/lib64/libcephsqlite.so 2023-11-09T03:56:15.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 15560 38227 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root 1w FIFO 0,13 0t0 180530 pipe 2023-11-09T03:56:15.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root 2w FIFO 0,13 0t0 180531 pipe 2023-11-09T03:56:15.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root 3r FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:15.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root 4w FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:15.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root 5u a_inode 0,14 0 2077 [eventpoll:6,25] 2023-11-09T03:56:15.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root 6r FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:15.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root 7w FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:15.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root 8u a_inode 0,14 0 2077 [eventpoll:9,23] 2023-11-09T03:56:15.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root 9r FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:15.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root 10w FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:15.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root 11u a_inode 0,14 0 2077 [eventpoll:12] 2023-11-09T03:56:15.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root 12r FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:15.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root 13w FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:15.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root 14w REG 8,1 809589 265321 /var/log/ceph/ceph-mgr.y.log 2023-11-09T03:56:15.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root 15uW REG 0,24 6 1735 /run/ceph/ceph-mgr.y.pid 2023-11-09T03:56:15.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root 16r FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:15.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root 17w FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:15.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root 18u unix 0xffff959a60349540 0t0 177902 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root 19r FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:15.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root 20w FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:15.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root 21r FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:15.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root 22w FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:15.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root 23u IPv4 185569 0t0 TCP smithi076.front.sepia.ceph.com:44172->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87892 admin_soc root 25u IPv4 183326 0t0 TCP smithi076.front.sepia.ceph.com:46708->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root txt REG 8,1 4625312 2939 /usr/bin/ceph-mgr 2023-11-09T03:56:15.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-11-09T03:56:15.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 64192 38198 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 65240 274077 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 783584 273412 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 87608 273415 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 331936 273390 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 654400 273423 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 41112936 2759 /usr/lib64/libopenblaso-r0.3.21.so 2023-11-09T03:56:15.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 60264 273417 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 96304 273413 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 237384 273392 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 177920 273418 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 7042240 272781 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2023-11-09T03:56:15.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 70784 273414 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 73976 273198 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 155704 273300 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 2883920 2473 /usr/lib64/libgfortran.so.5.0.0 2023-11-09T03:56:15.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 3870704 2771 /usr/lib64/libflexiblas.so.3.0 2023-11-09T03:56:15.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 3784896 272831 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 132136 272830 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 294392 9898 /usr/lib64/libgomp.so.1.0.0 2023-11-09T03:56:15.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 382824 272782 /usr/lib64/flexiblas/libflexiblas_netlib.so 2023-11-09T03:56:15.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 299952 2469 /usr/lib64/libquadmath.so.0.0.0 2023-11-09T03:56:15.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 8003360 1775 /usr/lib64/librbd.so.1.18.0 2023-11-09T03:56:15.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 31984 42006 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 15336 38209 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 20504 273301 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 40328 272779 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2023-11-09T03:56:15.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 78840 41984 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 75928 9784 /usr/lib64/libjson-c.so.5.0.0 2023-11-09T03:56:15.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 384592 11464 /usr/lib64/libdevmapper.so.1.02 2023-11-09T03:56:15.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 554440 11254 /usr/lib64/libcryptsetup.so.12.9.0 2023-11-09T03:56:15.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 1129568 2920 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 130464 38212 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:15.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 71408 38249 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 38744 38211 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 136048 11808 /usr/lib64/libyaml-0.so.2.0.9 2023-11-09T03:56:15.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 20832 38226 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 246392 59769 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 119688 36903 /usr/lib64/ossl-modules/legacy.so 2023-11-09T03:56:15.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 721976 270244 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2023-11-09T03:56:15.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 2154416 270246 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2023-11-09T03:56:15.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 1116856 42007 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 124184 41989 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 16192 265939 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 283592 38217 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 213376 41995 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 20528 41991 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 41384 38243 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 24160 41992 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:15.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:15.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 2103760 2040 /usr/lib64/libcephfs.so.2.0.0 2023-11-09T03:56:15.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 515320 2482 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:15.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 198136 2576 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 49328 270783 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2023-11-09T03:56:15.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 50232 41979 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 69048 38220 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 36328 9712 /usr/lib64/libuuid.so.1.3.0 2023-11-09T03:56:15.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 15984 38200 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 96936 41994 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 68000 38241 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 101208 41993 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 107736 38215 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 1129032 2042 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 50048 41986 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:15.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 15504 41997 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:15.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 1892920 1771 /usr/lib64/librados.so.2.0.0 2023-11-09T03:56:15.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:15.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 1294848 9721 /usr/lib64/libsqlite3.so.0.8.6 2023-11-09T03:56:15.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 435816 2332 /usr/lib64/libcephsqlite.so 2023-11-09T03:56:15.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 15560 38227 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root 1w FIFO 0,13 0t0 180530 pipe 2023-11-09T03:56:15.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root 2w FIFO 0,13 0t0 180531 pipe 2023-11-09T03:56:15.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root 3r FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:15.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root 4w FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:15.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root 5u a_inode 0,14 0 2077 [eventpoll:6,25] 2023-11-09T03:56:15.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root 6r FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:15.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root 7w FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:15.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root 8u a_inode 0,14 0 2077 [eventpoll:9,23] 2023-11-09T03:56:15.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root 9r FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:15.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root 10w FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:15.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root 11u a_inode 0,14 0 2077 [eventpoll:12] 2023-11-09T03:56:15.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root 12r FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:15.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root 13w FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:15.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root 14w REG 8,1 809589 265321 /var/log/ceph/ceph-mgr.y.log 2023-11-09T03:56:15.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root 15uW REG 0,24 6 1735 /run/ceph/ceph-mgr.y.pid 2023-11-09T03:56:15.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root 16r FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:15.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root 17w FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:15.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root 18u unix 0xffff959a60349540 0t0 177902 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root 19r FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:15.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root 20w FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:15.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root 21r FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:15.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root 22w FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:15.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root 23u IPv4 185569 0t0 TCP smithi076.front.sepia.ceph.com:44172->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87893 ceph_time root 25u IPv4 183326 0t0 TCP smithi076.front.sepia.ceph.com:46708->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root txt REG 8,1 4625312 2939 /usr/bin/ceph-mgr 2023-11-09T03:56:15.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-11-09T03:56:15.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 64192 38198 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 65240 274077 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 783584 273412 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 87608 273415 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 331936 273390 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 654400 273423 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 41112936 2759 /usr/lib64/libopenblaso-r0.3.21.so 2023-11-09T03:56:15.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 60264 273417 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 96304 273413 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 237384 273392 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 177920 273418 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 7042240 272781 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2023-11-09T03:56:15.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 70784 273414 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 73976 273198 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 155704 273300 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 2883920 2473 /usr/lib64/libgfortran.so.5.0.0 2023-11-09T03:56:15.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 3870704 2771 /usr/lib64/libflexiblas.so.3.0 2023-11-09T03:56:15.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 3784896 272831 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 132136 272830 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 294392 9898 /usr/lib64/libgomp.so.1.0.0 2023-11-09T03:56:15.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 382824 272782 /usr/lib64/flexiblas/libflexiblas_netlib.so 2023-11-09T03:56:15.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 299952 2469 /usr/lib64/libquadmath.so.0.0.0 2023-11-09T03:56:15.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 8003360 1775 /usr/lib64/librbd.so.1.18.0 2023-11-09T03:56:15.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 31984 42006 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 15336 38209 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 20504 273301 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 40328 272779 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2023-11-09T03:56:15.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 78840 41984 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 75928 9784 /usr/lib64/libjson-c.so.5.0.0 2023-11-09T03:56:15.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 384592 11464 /usr/lib64/libdevmapper.so.1.02 2023-11-09T03:56:15.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 554440 11254 /usr/lib64/libcryptsetup.so.12.9.0 2023-11-09T03:56:15.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 1129568 2920 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 130464 38212 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:15.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 71408 38249 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 38744 38211 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 136048 11808 /usr/lib64/libyaml-0.so.2.0.9 2023-11-09T03:56:15.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 20832 38226 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 246392 59769 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 119688 36903 /usr/lib64/ossl-modules/legacy.so 2023-11-09T03:56:15.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 721976 270244 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2023-11-09T03:56:15.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 2154416 270246 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2023-11-09T03:56:15.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 1116856 42007 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 124184 41989 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 16192 265939 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 283592 38217 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 213376 41995 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 20528 41991 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 41384 38243 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 24160 41992 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:15.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:15.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 2103760 2040 /usr/lib64/libcephfs.so.2.0.0 2023-11-09T03:56:15.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 515320 2482 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:15.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 198136 2576 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 49328 270783 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2023-11-09T03:56:15.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 50232 41979 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 69048 38220 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 36328 9712 /usr/lib64/libuuid.so.1.3.0 2023-11-09T03:56:15.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 15984 38200 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 96936 41994 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 68000 38241 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 101208 41993 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 107736 38215 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 1129032 2042 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 50048 41986 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:15.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 15504 41997 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:15.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 1892920 1771 /usr/lib64/librados.so.2.0.0 2023-11-09T03:56:15.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:15.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 1294848 9721 /usr/lib64/libsqlite3.so.0.8.6 2023-11-09T03:56:15.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 435816 2332 /usr/lib64/libcephsqlite.so 2023-11-09T03:56:15.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 15560 38227 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root 1w FIFO 0,13 0t0 180530 pipe 2023-11-09T03:56:15.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root 2w FIFO 0,13 0t0 180531 pipe 2023-11-09T03:56:15.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root 3r FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:15.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root 4w FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:15.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root 5u a_inode 0,14 0 2077 [eventpoll:6,25] 2023-11-09T03:56:15.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root 6r FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:15.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root 7w FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:15.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root 8u a_inode 0,14 0 2077 [eventpoll:9,23] 2023-11-09T03:56:15.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root 9r FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:15.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root 10w FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:15.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root 11u a_inode 0,14 0 2077 [eventpoll:12] 2023-11-09T03:56:15.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root 12r FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:15.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root 13w FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:15.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root 14w REG 8,1 809589 265321 /var/log/ceph/ceph-mgr.y.log 2023-11-09T03:56:15.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root 15uW REG 0,24 6 1735 /run/ceph/ceph-mgr.y.pid 2023-11-09T03:56:15.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root 16r FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:15.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root 17w FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:15.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root 18u unix 0xffff959a60349540 0t0 177902 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root 19r FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:15.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root 20w FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:15.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root 21r FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:15.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root 22w FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:15.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root 23u IPv4 185569 0t0 TCP smithi076.front.sepia.ceph.com:44172->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87894 fn_anonym root 25u IPv4 183326 0t0 TCP smithi076.front.sepia.ceph.com:46708->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root txt REG 8,1 4625312 2939 /usr/bin/ceph-mgr 2023-11-09T03:56:15.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-11-09T03:56:15.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 64192 38198 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 65240 274077 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 783584 273412 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 87608 273415 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 331936 273390 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 654400 273423 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 41112936 2759 /usr/lib64/libopenblaso-r0.3.21.so 2023-11-09T03:56:15.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 60264 273417 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 96304 273413 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 237384 273392 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 177920 273418 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 7042240 272781 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2023-11-09T03:56:15.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 70784 273414 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 73976 273198 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 155704 273300 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 2883920 2473 /usr/lib64/libgfortran.so.5.0.0 2023-11-09T03:56:15.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 3870704 2771 /usr/lib64/libflexiblas.so.3.0 2023-11-09T03:56:15.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 3784896 272831 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 132136 272830 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 294392 9898 /usr/lib64/libgomp.so.1.0.0 2023-11-09T03:56:15.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 382824 272782 /usr/lib64/flexiblas/libflexiblas_netlib.so 2023-11-09T03:56:15.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 299952 2469 /usr/lib64/libquadmath.so.0.0.0 2023-11-09T03:56:15.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 8003360 1775 /usr/lib64/librbd.so.1.18.0 2023-11-09T03:56:15.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 31984 42006 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 15336 38209 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 20504 273301 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 40328 272779 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2023-11-09T03:56:15.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 78840 41984 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 75928 9784 /usr/lib64/libjson-c.so.5.0.0 2023-11-09T03:56:15.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 384592 11464 /usr/lib64/libdevmapper.so.1.02 2023-11-09T03:56:15.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 554440 11254 /usr/lib64/libcryptsetup.so.12.9.0 2023-11-09T03:56:15.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 1129568 2920 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 130464 38212 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:15.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 71408 38249 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 38744 38211 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 136048 11808 /usr/lib64/libyaml-0.so.2.0.9 2023-11-09T03:56:15.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 20832 38226 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 246392 59769 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 119688 36903 /usr/lib64/ossl-modules/legacy.so 2023-11-09T03:56:15.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 721976 270244 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2023-11-09T03:56:15.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 2154416 270246 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2023-11-09T03:56:15.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 1116856 42007 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 124184 41989 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 16192 265939 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 283592 38217 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 213376 41995 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 20528 41991 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 41384 38243 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 24160 41992 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:15.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:15.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 2103760 2040 /usr/lib64/libcephfs.so.2.0.0 2023-11-09T03:56:15.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 515320 2482 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:15.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 198136 2576 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 49328 270783 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2023-11-09T03:56:15.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 50232 41979 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 69048 38220 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 36328 9712 /usr/lib64/libuuid.so.1.3.0 2023-11-09T03:56:15.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 15984 38200 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 96936 41994 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 68000 38241 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 101208 41993 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 107736 38215 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 1129032 2042 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 50048 41986 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:15.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 15504 41997 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:15.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 1892920 1771 /usr/lib64/librados.so.2.0.0 2023-11-09T03:56:15.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:15.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 1294848 9721 /usr/lib64/libsqlite3.so.0.8.6 2023-11-09T03:56:15.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 435816 2332 /usr/lib64/libcephsqlite.so 2023-11-09T03:56:15.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 15560 38227 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root 1w FIFO 0,13 0t0 180530 pipe 2023-11-09T03:56:15.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root 2w FIFO 0,13 0t0 180531 pipe 2023-11-09T03:56:15.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root 3r FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:15.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root 4w FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:15.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root 5u a_inode 0,14 0 2077 [eventpoll:6,25] 2023-11-09T03:56:15.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root 6r FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:15.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root 7w FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:15.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root 8u a_inode 0,14 0 2077 [eventpoll:9,23] 2023-11-09T03:56:15.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root 9r FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:15.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root 10w FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:15.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root 11u a_inode 0,14 0 2077 [eventpoll:12] 2023-11-09T03:56:15.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root 12r FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:15.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root 13w FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:15.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root 14w REG 8,1 809589 265321 /var/log/ceph/ceph-mgr.y.log 2023-11-09T03:56:15.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root 15uW REG 0,24 6 1735 /run/ceph/ceph-mgr.y.pid 2023-11-09T03:56:15.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root 16r FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:15.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root 17w FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:15.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root 18u unix 0xffff959a60349540 0t0 177902 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root 19r FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:15.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root 20w FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:15.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root 21r FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:15.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root 22w FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:15.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root 23u IPv4 185569 0t0 TCP smithi076.front.sepia.ceph.com:44172->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87895 signal_ha root 25u IPv4 183326 0t0 TCP smithi076.front.sepia.ceph.com:46708->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root txt REG 8,1 4625312 2939 /usr/bin/ceph-mgr 2023-11-09T03:56:15.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-11-09T03:56:15.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 64192 38198 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 65240 274077 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 783584 273412 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 87608 273415 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 331936 273390 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 654400 273423 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 41112936 2759 /usr/lib64/libopenblaso-r0.3.21.so 2023-11-09T03:56:15.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 60264 273417 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 96304 273413 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 237384 273392 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 177920 273418 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 7042240 272781 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2023-11-09T03:56:15.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 70784 273414 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 73976 273198 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 155704 273300 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 2883920 2473 /usr/lib64/libgfortran.so.5.0.0 2023-11-09T03:56:15.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 3870704 2771 /usr/lib64/libflexiblas.so.3.0 2023-11-09T03:56:15.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 3784896 272831 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 132136 272830 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 294392 9898 /usr/lib64/libgomp.so.1.0.0 2023-11-09T03:56:15.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 382824 272782 /usr/lib64/flexiblas/libflexiblas_netlib.so 2023-11-09T03:56:15.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 299952 2469 /usr/lib64/libquadmath.so.0.0.0 2023-11-09T03:56:15.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 8003360 1775 /usr/lib64/librbd.so.1.18.0 2023-11-09T03:56:15.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 31984 42006 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 15336 38209 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 20504 273301 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 40328 272779 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2023-11-09T03:56:15.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 78840 41984 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 75928 9784 /usr/lib64/libjson-c.so.5.0.0 2023-11-09T03:56:15.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 384592 11464 /usr/lib64/libdevmapper.so.1.02 2023-11-09T03:56:15.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 554440 11254 /usr/lib64/libcryptsetup.so.12.9.0 2023-11-09T03:56:15.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 1129568 2920 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 130464 38212 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:15.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 71408 38249 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 38744 38211 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 136048 11808 /usr/lib64/libyaml-0.so.2.0.9 2023-11-09T03:56:15.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 20832 38226 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 246392 59769 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 119688 36903 /usr/lib64/ossl-modules/legacy.so 2023-11-09T03:56:15.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 721976 270244 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2023-11-09T03:56:15.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 2154416 270246 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2023-11-09T03:56:15.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 1116856 42007 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 124184 41989 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 16192 265939 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 283592 38217 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 213376 41995 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 20528 41991 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 41384 38243 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 24160 41992 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:15.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:15.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 2103760 2040 /usr/lib64/libcephfs.so.2.0.0 2023-11-09T03:56:15.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 515320 2482 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:15.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 198136 2576 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 49328 270783 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2023-11-09T03:56:15.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 50232 41979 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 69048 38220 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 36328 9712 /usr/lib64/libuuid.so.1.3.0 2023-11-09T03:56:15.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 15984 38200 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 96936 41994 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 68000 38241 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 101208 41993 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 107736 38215 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 1129032 2042 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 50048 41986 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:15.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 15504 41997 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:15.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 1892920 1771 /usr/lib64/librados.so.2.0.0 2023-11-09T03:56:15.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:15.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 1294848 9721 /usr/lib64/libsqlite3.so.0.8.6 2023-11-09T03:56:15.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 435816 2332 /usr/lib64/libcephsqlite.so 2023-11-09T03:56:15.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 15560 38227 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root 1w FIFO 0,13 0t0 180530 pipe 2023-11-09T03:56:15.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root 2w FIFO 0,13 0t0 180531 pipe 2023-11-09T03:56:15.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root 3r FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:15.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root 4w FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:15.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root 5u a_inode 0,14 0 2077 [eventpoll:6,25] 2023-11-09T03:56:15.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root 6r FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:15.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root 7w FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:15.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root 8u a_inode 0,14 0 2077 [eventpoll:9,23] 2023-11-09T03:56:15.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root 9r FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:15.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root 10w FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:15.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root 11u a_inode 0,14 0 2077 [eventpoll:12] 2023-11-09T03:56:15.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root 12r FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:15.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root 13w FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:15.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root 14w REG 8,1 809589 265321 /var/log/ceph/ceph-mgr.y.log 2023-11-09T03:56:15.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root 15uW REG 0,24 6 1735 /run/ceph/ceph-mgr.y.pid 2023-11-09T03:56:15.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root 16r FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:15.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root 17w FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:15.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root 18u unix 0xffff959a60349540 0t0 177902 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root 19r FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:15.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root 20w FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:15.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root 21r FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:15.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root 22w FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:15.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root 23u IPv4 185569 0t0 TCP smithi076.front.sepia.ceph.com:44172->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87896 mgrsb-fin root 25u IPv4 183326 0t0 TCP smithi076.front.sepia.ceph.com:46708->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root txt REG 8,1 4625312 2939 /usr/bin/ceph-mgr 2023-11-09T03:56:15.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-11-09T03:56:15.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 64192 38198 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 65240 274077 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 783584 273412 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 87608 273415 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 331936 273390 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 654400 273423 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 41112936 2759 /usr/lib64/libopenblaso-r0.3.21.so 2023-11-09T03:56:15.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 60264 273417 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 96304 273413 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 237384 273392 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 177920 273418 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 7042240 272781 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2023-11-09T03:56:15.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 70784 273414 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 73976 273198 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 155704 273300 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 2883920 2473 /usr/lib64/libgfortran.so.5.0.0 2023-11-09T03:56:15.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 3870704 2771 /usr/lib64/libflexiblas.so.3.0 2023-11-09T03:56:15.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 3784896 272831 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 132136 272830 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 294392 9898 /usr/lib64/libgomp.so.1.0.0 2023-11-09T03:56:15.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 382824 272782 /usr/lib64/flexiblas/libflexiblas_netlib.so 2023-11-09T03:56:15.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 299952 2469 /usr/lib64/libquadmath.so.0.0.0 2023-11-09T03:56:15.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 8003360 1775 /usr/lib64/librbd.so.1.18.0 2023-11-09T03:56:15.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 31984 42006 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 15336 38209 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 20504 273301 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 40328 272779 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2023-11-09T03:56:15.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 78840 41984 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 75928 9784 /usr/lib64/libjson-c.so.5.0.0 2023-11-09T03:56:15.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 384592 11464 /usr/lib64/libdevmapper.so.1.02 2023-11-09T03:56:15.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 554440 11254 /usr/lib64/libcryptsetup.so.12.9.0 2023-11-09T03:56:15.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 1129568 2920 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 130464 38212 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:15.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 71408 38249 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 38744 38211 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 136048 11808 /usr/lib64/libyaml-0.so.2.0.9 2023-11-09T03:56:15.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 20832 38226 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 246392 59769 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 119688 36903 /usr/lib64/ossl-modules/legacy.so 2023-11-09T03:56:15.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 721976 270244 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2023-11-09T03:56:15.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 2154416 270246 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2023-11-09T03:56:15.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 1116856 42007 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 124184 41989 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 16192 265939 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 283592 38217 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 213376 41995 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 20528 41991 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 41384 38243 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 24160 41992 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:15.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:15.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 2103760 2040 /usr/lib64/libcephfs.so.2.0.0 2023-11-09T03:56:15.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 515320 2482 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:15.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 198136 2576 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 49328 270783 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2023-11-09T03:56:15.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 50232 41979 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 69048 38220 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 36328 9712 /usr/lib64/libuuid.so.1.3.0 2023-11-09T03:56:15.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 15984 38200 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 96936 41994 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 68000 38241 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 101208 41993 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 107736 38215 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 1129032 2042 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 50048 41986 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:15.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 15504 41997 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:15.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 1892920 1771 /usr/lib64/librados.so.2.0.0 2023-11-09T03:56:15.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:15.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 1294848 9721 /usr/lib64/libsqlite3.so.0.8.6 2023-11-09T03:56:15.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 435816 2332 /usr/lib64/libcephsqlite.so 2023-11-09T03:56:15.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 15560 38227 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root 1w FIFO 0,13 0t0 180530 pipe 2023-11-09T03:56:15.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root 2w FIFO 0,13 0t0 180531 pipe 2023-11-09T03:56:15.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root 3r FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:15.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root 4w FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:15.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,25] 2023-11-09T03:56:15.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root 6r FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:15.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root 7w FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:15.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,23] 2023-11-09T03:56:15.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root 9r FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:15.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root 10w FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:15.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12] 2023-11-09T03:56:15.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root 12r FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:15.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root 13w FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:15.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root 14w REG 8,1 809589 265321 /var/log/ceph/ceph-mgr.y.log 2023-11-09T03:56:15.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root 15uW REG 0,24 6 1735 /run/ceph/ceph-mgr.y.pid 2023-11-09T03:56:15.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root 16r FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:15.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root 17w FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:15.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root 18u unix 0xffff959a60349540 0t0 177902 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root 19r FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:15.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root 20w FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:15.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root 21r FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:15.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root 22w FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:15.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root 23u IPv4 185569 0t0 TCP smithi076.front.sepia.ceph.com:44172->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87897 ms_dispat root 25u IPv4 183326 0t0 TCP smithi076.front.sepia.ceph.com:46708->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root txt REG 8,1 4625312 2939 /usr/bin/ceph-mgr 2023-11-09T03:56:15.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-11-09T03:56:15.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 64192 38198 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 65240 274077 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 783584 273412 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 87608 273415 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 331936 273390 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 654400 273423 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 41112936 2759 /usr/lib64/libopenblaso-r0.3.21.so 2023-11-09T03:56:15.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 60264 273417 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 96304 273413 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 237384 273392 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 177920 273418 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 7042240 272781 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2023-11-09T03:56:15.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 70784 273414 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 73976 273198 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 155704 273300 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 2883920 2473 /usr/lib64/libgfortran.so.5.0.0 2023-11-09T03:56:15.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 3870704 2771 /usr/lib64/libflexiblas.so.3.0 2023-11-09T03:56:15.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 3784896 272831 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 132136 272830 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 294392 9898 /usr/lib64/libgomp.so.1.0.0 2023-11-09T03:56:15.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 382824 272782 /usr/lib64/flexiblas/libflexiblas_netlib.so 2023-11-09T03:56:15.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 299952 2469 /usr/lib64/libquadmath.so.0.0.0 2023-11-09T03:56:15.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 8003360 1775 /usr/lib64/librbd.so.1.18.0 2023-11-09T03:56:15.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 31984 42006 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 15336 38209 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 20504 273301 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 40328 272779 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2023-11-09T03:56:15.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 78840 41984 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 75928 9784 /usr/lib64/libjson-c.so.5.0.0 2023-11-09T03:56:15.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 384592 11464 /usr/lib64/libdevmapper.so.1.02 2023-11-09T03:56:15.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 554440 11254 /usr/lib64/libcryptsetup.so.12.9.0 2023-11-09T03:56:15.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 1129568 2920 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 130464 38212 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:15.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 71408 38249 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 38744 38211 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 136048 11808 /usr/lib64/libyaml-0.so.2.0.9 2023-11-09T03:56:15.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 20832 38226 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 246392 59769 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 119688 36903 /usr/lib64/ossl-modules/legacy.so 2023-11-09T03:56:15.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 721976 270244 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2023-11-09T03:56:15.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 2154416 270246 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2023-11-09T03:56:15.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 1116856 42007 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 124184 41989 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 16192 265939 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 283592 38217 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 213376 41995 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 20528 41991 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 41384 38243 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 24160 41992 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:15.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:15.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 2103760 2040 /usr/lib64/libcephfs.so.2.0.0 2023-11-09T03:56:15.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 515320 2482 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:15.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 198136 2576 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 49328 270783 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2023-11-09T03:56:15.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 50232 41979 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 69048 38220 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 36328 9712 /usr/lib64/libuuid.so.1.3.0 2023-11-09T03:56:15.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 15984 38200 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 96936 41994 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 68000 38241 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 101208 41993 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 107736 38215 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 1129032 2042 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 50048 41986 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:15.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 15504 41997 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:15.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 1892920 1771 /usr/lib64/librados.so.2.0.0 2023-11-09T03:56:15.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:15.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:15.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:15.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 1294848 9721 /usr/lib64/libsqlite3.so.0.8.6 2023-11-09T03:56:15.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 435816 2332 /usr/lib64/libcephsqlite.so 2023-11-09T03:56:15.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 15560 38227 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:15.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:15.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root 1w FIFO 0,13 0t0 180530 pipe 2023-11-09T03:56:15.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root 2w FIFO 0,13 0t0 180531 pipe 2023-11-09T03:56:15.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root 3r FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:15.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root 4w FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:15.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,25] 2023-11-09T03:56:15.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root 6r FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:15.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root 7w FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:15.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,23] 2023-11-09T03:56:15.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root 9r FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:15.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root 10w FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:15.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12] 2023-11-09T03:56:15.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root 12r FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:15.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root 13w FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:15.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root 14w REG 8,1 809589 265321 /var/log/ceph/ceph-mgr.y.log 2023-11-09T03:56:15.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root 15uW REG 0,24 6 1735 /run/ceph/ceph-mgr.y.pid 2023-11-09T03:56:15.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root 16r FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:15.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root 17w FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:15.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root 18u unix 0xffff959a60349540 0t0 177902 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2023-11-09T03:56:15.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root 19r FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:15.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root 20w FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:15.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root 21r FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:15.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root 22w FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:15.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root 23u IPv4 185569 0t0 TCP smithi076.front.sepia.ceph.com:44172->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:15.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87898 ms_local root 25u IPv4 183326 0t0 TCP smithi076.front.sepia.ceph.com:46708->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:15.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:15.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:15.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root txt REG 8,1 4625312 2939 /usr/bin/ceph-mgr 2023-11-09T03:56:15.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-11-09T03:56:15.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 64192 38198 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 65240 274077 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 783584 273412 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 87608 273415 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 331936 273390 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 654400 273423 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 41112936 2759 /usr/lib64/libopenblaso-r0.3.21.so 2023-11-09T03:56:15.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 60264 273417 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 96304 273413 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 237384 273392 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 177920 273418 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 7042240 272781 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2023-11-09T03:56:15.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 70784 273414 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 73976 273198 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 155704 273300 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 2883920 2473 /usr/lib64/libgfortran.so.5.0.0 2023-11-09T03:56:15.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 3870704 2771 /usr/lib64/libflexiblas.so.3.0 2023-11-09T03:56:15.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 3784896 272831 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 132136 272830 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 294392 9898 /usr/lib64/libgomp.so.1.0.0 2023-11-09T03:56:15.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 382824 272782 /usr/lib64/flexiblas/libflexiblas_netlib.so 2023-11-09T03:56:15.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 299952 2469 /usr/lib64/libquadmath.so.0.0.0 2023-11-09T03:56:15.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 8003360 1775 /usr/lib64/librbd.so.1.18.0 2023-11-09T03:56:15.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 31984 42006 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 15336 38209 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 20504 273301 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 40328 272779 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2023-11-09T03:56:15.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 78840 41984 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 75928 9784 /usr/lib64/libjson-c.so.5.0.0 2023-11-09T03:56:15.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 384592 11464 /usr/lib64/libdevmapper.so.1.02 2023-11-09T03:56:15.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 554440 11254 /usr/lib64/libcryptsetup.so.12.9.0 2023-11-09T03:56:15.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 1129568 2920 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 130464 38212 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:15.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 71408 38249 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 38744 38211 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 136048 11808 /usr/lib64/libyaml-0.so.2.0.9 2023-11-09T03:56:15.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 20832 38226 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 246392 59769 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 119688 36903 /usr/lib64/ossl-modules/legacy.so 2023-11-09T03:56:15.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 721976 270244 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2023-11-09T03:56:15.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 2154416 270246 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2023-11-09T03:56:15.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 1116856 42007 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 124184 41989 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 16192 265939 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 283592 38217 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 213376 41995 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 20528 41991 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 41384 38243 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 24160 41992 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:15.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:15.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 2103760 2040 /usr/lib64/libcephfs.so.2.0.0 2023-11-09T03:56:15.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 515320 2482 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:15.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 198136 2576 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 49328 270783 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2023-11-09T03:56:15.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 50232 41979 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 69048 38220 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 36328 9712 /usr/lib64/libuuid.so.1.3.0 2023-11-09T03:56:15.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 15984 38200 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 96936 41994 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 68000 38241 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 101208 41993 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 107736 38215 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 1129032 2042 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 50048 41986 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:15.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:15.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:15.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:15.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:15.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:15.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:15.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:15.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:15.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:15.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:15.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:15.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:15.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:15.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:15.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:15.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:15.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:15.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:15.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 15504 41997 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:15.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:15.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:15.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:15.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:15.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:15.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:15.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:15.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:15.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:15.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:15.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:15.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:15.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:15.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:15.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:15.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:15.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:15.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:15.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:15.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 1892920 1771 /usr/lib64/librados.so.2.0.0 2023-11-09T03:56:15.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:15.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:15.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:15.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:15.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:15.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:15.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:15.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 1294848 9721 /usr/lib64/libsqlite3.so.0.8.6 2023-11-09T03:56:16.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 435816 2332 /usr/lib64/libcephsqlite.so 2023-11-09T03:56:16.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 15560 38227 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root 1w FIFO 0,13 0t0 180530 pipe 2023-11-09T03:56:16.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root 2w FIFO 0,13 0t0 180531 pipe 2023-11-09T03:56:16.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root 3r FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:16.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root 4w FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:16.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root 5u a_inode 0,14 0 2077 [eventpoll:6,25] 2023-11-09T03:56:16.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root 6r FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:16.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root 7w FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:16.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root 8u a_inode 0,14 0 2077 [eventpoll:9,23] 2023-11-09T03:56:16.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root 9r FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:16.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root 10w FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:16.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root 11u a_inode 0,14 0 2077 [eventpoll:12] 2023-11-09T03:56:16.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root 12r FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:16.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root 13w FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:16.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root 14w REG 8,1 809589 265321 /var/log/ceph/ceph-mgr.y.log 2023-11-09T03:56:16.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root 15uW REG 0,24 6 1735 /run/ceph/ceph-mgr.y.pid 2023-11-09T03:56:16.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root 16r FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:16.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root 17w FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:16.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root 18u unix 0xffff959a60349540 0t0 177902 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root 19r FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:16.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root 20w FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:16.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root 21r FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:16.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root 22w FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:16.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root 23u IPv4 185569 0t0 TCP smithi076.front.sepia.ceph.com:44172->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87899 io_contex root 25u IPv4 183326 0t0 TCP smithi076.front.sepia.ceph.com:46708->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root txt REG 8,1 4625312 2939 /usr/bin/ceph-mgr 2023-11-09T03:56:16.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-11-09T03:56:16.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 64192 38198 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 65240 274077 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 783584 273412 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 87608 273415 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 331936 273390 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 654400 273423 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 41112936 2759 /usr/lib64/libopenblaso-r0.3.21.so 2023-11-09T03:56:16.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 60264 273417 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 96304 273413 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 237384 273392 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 177920 273418 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 7042240 272781 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2023-11-09T03:56:16.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 70784 273414 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 73976 273198 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 155704 273300 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 2883920 2473 /usr/lib64/libgfortran.so.5.0.0 2023-11-09T03:56:16.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 3870704 2771 /usr/lib64/libflexiblas.so.3.0 2023-11-09T03:56:16.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 3784896 272831 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 132136 272830 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 294392 9898 /usr/lib64/libgomp.so.1.0.0 2023-11-09T03:56:16.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 382824 272782 /usr/lib64/flexiblas/libflexiblas_netlib.so 2023-11-09T03:56:16.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 299952 2469 /usr/lib64/libquadmath.so.0.0.0 2023-11-09T03:56:16.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 8003360 1775 /usr/lib64/librbd.so.1.18.0 2023-11-09T03:56:16.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 31984 42006 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 15336 38209 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 20504 273301 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 40328 272779 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2023-11-09T03:56:16.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 78840 41984 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 75928 9784 /usr/lib64/libjson-c.so.5.0.0 2023-11-09T03:56:16.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 384592 11464 /usr/lib64/libdevmapper.so.1.02 2023-11-09T03:56:16.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 554440 11254 /usr/lib64/libcryptsetup.so.12.9.0 2023-11-09T03:56:16.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 1129568 2920 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 130464 38212 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:16.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 71408 38249 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 38744 38211 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 136048 11808 /usr/lib64/libyaml-0.so.2.0.9 2023-11-09T03:56:16.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 20832 38226 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 246392 59769 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 119688 36903 /usr/lib64/ossl-modules/legacy.so 2023-11-09T03:56:16.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 721976 270244 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2023-11-09T03:56:16.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 2154416 270246 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2023-11-09T03:56:16.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 1116856 42007 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 124184 41989 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 16192 265939 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 283592 38217 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 213376 41995 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 20528 41991 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 41384 38243 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 24160 41992 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:16.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:16.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 2103760 2040 /usr/lib64/libcephfs.so.2.0.0 2023-11-09T03:56:16.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 515320 2482 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:16.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 198136 2576 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 49328 270783 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2023-11-09T03:56:16.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 50232 41979 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 69048 38220 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 36328 9712 /usr/lib64/libuuid.so.1.3.0 2023-11-09T03:56:16.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 15984 38200 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 96936 41994 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 68000 38241 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 101208 41993 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 107736 38215 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 1129032 2042 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 50048 41986 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:16.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 15504 41997 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:16.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:16.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 1892920 1771 /usr/lib64/librados.so.2.0.0 2023-11-09T03:56:16.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:16.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 1294848 9721 /usr/lib64/libsqlite3.so.0.8.6 2023-11-09T03:56:16.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 435816 2332 /usr/lib64/libcephsqlite.so 2023-11-09T03:56:16.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 15560 38227 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root 1w FIFO 0,13 0t0 180530 pipe 2023-11-09T03:56:16.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root 2w FIFO 0,13 0t0 180531 pipe 2023-11-09T03:56:16.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root 3r FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:16.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root 4w FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:16.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root 5u a_inode 0,14 0 2077 [eventpoll:6,25] 2023-11-09T03:56:16.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root 6r FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:16.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root 7w FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:16.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root 8u a_inode 0,14 0 2077 [eventpoll:9,23] 2023-11-09T03:56:16.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root 9r FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:16.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root 10w FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:16.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root 11u a_inode 0,14 0 2077 [eventpoll:12] 2023-11-09T03:56:16.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root 12r FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:16.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root 13w FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:16.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root 14w REG 8,1 809589 265321 /var/log/ceph/ceph-mgr.y.log 2023-11-09T03:56:16.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root 15uW REG 0,24 6 1735 /run/ceph/ceph-mgr.y.pid 2023-11-09T03:56:16.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root 16r FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:16.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root 17w FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:16.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root 18u unix 0xffff959a60349540 0t0 177902 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root 19r FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:16.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root 20w FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:16.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root 21r FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:16.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root 22w FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:16.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root 23u IPv4 185569 0t0 TCP smithi076.front.sepia.ceph.com:44172->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87900 io_contex root 25u IPv4 183326 0t0 TCP smithi076.front.sepia.ceph.com:46708->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root txt REG 8,1 4625312 2939 /usr/bin/ceph-mgr 2023-11-09T03:56:16.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-11-09T03:56:16.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 64192 38198 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 65240 274077 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 783584 273412 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 87608 273415 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 331936 273390 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 654400 273423 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 41112936 2759 /usr/lib64/libopenblaso-r0.3.21.so 2023-11-09T03:56:16.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 60264 273417 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 96304 273413 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 237384 273392 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 177920 273418 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 7042240 272781 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2023-11-09T03:56:16.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 70784 273414 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 73976 273198 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 155704 273300 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 2883920 2473 /usr/lib64/libgfortran.so.5.0.0 2023-11-09T03:56:16.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 3870704 2771 /usr/lib64/libflexiblas.so.3.0 2023-11-09T03:56:16.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 3784896 272831 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 132136 272830 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 294392 9898 /usr/lib64/libgomp.so.1.0.0 2023-11-09T03:56:16.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 382824 272782 /usr/lib64/flexiblas/libflexiblas_netlib.so 2023-11-09T03:56:16.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 299952 2469 /usr/lib64/libquadmath.so.0.0.0 2023-11-09T03:56:16.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 8003360 1775 /usr/lib64/librbd.so.1.18.0 2023-11-09T03:56:16.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 31984 42006 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 15336 38209 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 20504 273301 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 40328 272779 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2023-11-09T03:56:16.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 78840 41984 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 75928 9784 /usr/lib64/libjson-c.so.5.0.0 2023-11-09T03:56:16.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 384592 11464 /usr/lib64/libdevmapper.so.1.02 2023-11-09T03:56:16.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 554440 11254 /usr/lib64/libcryptsetup.so.12.9.0 2023-11-09T03:56:16.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 1129568 2920 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 130464 38212 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:16.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 71408 38249 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 38744 38211 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 136048 11808 /usr/lib64/libyaml-0.so.2.0.9 2023-11-09T03:56:16.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 20832 38226 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 246392 59769 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 119688 36903 /usr/lib64/ossl-modules/legacy.so 2023-11-09T03:56:16.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 721976 270244 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2023-11-09T03:56:16.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 2154416 270246 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2023-11-09T03:56:16.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 1116856 42007 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 124184 41989 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 16192 265939 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 283592 38217 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 213376 41995 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 20528 41991 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 41384 38243 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 24160 41992 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:16.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:16.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 2103760 2040 /usr/lib64/libcephfs.so.2.0.0 2023-11-09T03:56:16.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 515320 2482 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:16.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 198136 2576 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 49328 270783 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2023-11-09T03:56:16.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 50232 41979 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 69048 38220 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 36328 9712 /usr/lib64/libuuid.so.1.3.0 2023-11-09T03:56:16.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 15984 38200 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 96936 41994 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 68000 38241 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 101208 41993 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 107736 38215 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 1129032 2042 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 50048 41986 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:16.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 15504 41997 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:16.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:16.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 1892920 1771 /usr/lib64/librados.so.2.0.0 2023-11-09T03:56:16.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:16.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 1294848 9721 /usr/lib64/libsqlite3.so.0.8.6 2023-11-09T03:56:16.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 435816 2332 /usr/lib64/libcephsqlite.so 2023-11-09T03:56:16.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 15560 38227 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root 1w FIFO 0,13 0t0 180530 pipe 2023-11-09T03:56:16.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root 2w FIFO 0,13 0t0 180531 pipe 2023-11-09T03:56:16.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root 3r FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:16.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root 4w FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:16.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,25] 2023-11-09T03:56:16.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root 6r FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:16.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root 7w FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:16.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,23] 2023-11-09T03:56:16.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root 9r FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:16.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root 10w FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:16.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12] 2023-11-09T03:56:16.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root 12r FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:16.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root 13w FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:16.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root 14w REG 8,1 809589 265321 /var/log/ceph/ceph-mgr.y.log 2023-11-09T03:56:16.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root 15uW REG 0,24 6 1735 /run/ceph/ceph-mgr.y.pid 2023-11-09T03:56:16.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root 16r FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:16.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root 17w FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:16.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root 18u unix 0xffff959a60349540 0t0 177902 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root 19r FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:16.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root 20w FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:16.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root 21r FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:16.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root 22w FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:16.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root 23u IPv4 185569 0t0 TCP smithi076.front.sepia.ceph.com:44172->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87901 safe_time root 25u IPv4 183326 0t0 TCP smithi076.front.sepia.ceph.com:46708->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root txt REG 8,1 4625312 2939 /usr/bin/ceph-mgr 2023-11-09T03:56:16.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-11-09T03:56:16.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 64192 38198 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 65240 274077 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 783584 273412 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 87608 273415 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 331936 273390 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 654400 273423 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 41112936 2759 /usr/lib64/libopenblaso-r0.3.21.so 2023-11-09T03:56:16.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 60264 273417 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 96304 273413 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 237384 273392 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 177920 273418 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 7042240 272781 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2023-11-09T03:56:16.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 70784 273414 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 73976 273198 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 155704 273300 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 2883920 2473 /usr/lib64/libgfortran.so.5.0.0 2023-11-09T03:56:16.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 3870704 2771 /usr/lib64/libflexiblas.so.3.0 2023-11-09T03:56:16.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 3784896 272831 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 132136 272830 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 294392 9898 /usr/lib64/libgomp.so.1.0.0 2023-11-09T03:56:16.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 382824 272782 /usr/lib64/flexiblas/libflexiblas_netlib.so 2023-11-09T03:56:16.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 299952 2469 /usr/lib64/libquadmath.so.0.0.0 2023-11-09T03:56:16.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 8003360 1775 /usr/lib64/librbd.so.1.18.0 2023-11-09T03:56:16.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 31984 42006 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 15336 38209 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 20504 273301 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 40328 272779 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2023-11-09T03:56:16.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 78840 41984 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 75928 9784 /usr/lib64/libjson-c.so.5.0.0 2023-11-09T03:56:16.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 384592 11464 /usr/lib64/libdevmapper.so.1.02 2023-11-09T03:56:16.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 554440 11254 /usr/lib64/libcryptsetup.so.12.9.0 2023-11-09T03:56:16.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 1129568 2920 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 130464 38212 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:16.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 71408 38249 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 38744 38211 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 136048 11808 /usr/lib64/libyaml-0.so.2.0.9 2023-11-09T03:56:16.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 20832 38226 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 246392 59769 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 119688 36903 /usr/lib64/ossl-modules/legacy.so 2023-11-09T03:56:16.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 721976 270244 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2023-11-09T03:56:16.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 2154416 270246 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2023-11-09T03:56:16.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 1116856 42007 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 124184 41989 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 16192 265939 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 283592 38217 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 213376 41995 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 20528 41991 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 41384 38243 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 24160 41992 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:16.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:16.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 2103760 2040 /usr/lib64/libcephfs.so.2.0.0 2023-11-09T03:56:16.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 515320 2482 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:16.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 198136 2576 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 49328 270783 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2023-11-09T03:56:16.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 50232 41979 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 69048 38220 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 36328 9712 /usr/lib64/libuuid.so.1.3.0 2023-11-09T03:56:16.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 15984 38200 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 96936 41994 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 68000 38241 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 101208 41993 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 107736 38215 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 1129032 2042 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 50048 41986 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:16.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 15504 41997 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:16.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:16.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 1892920 1771 /usr/lib64/librados.so.2.0.0 2023-11-09T03:56:16.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:16.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 1294848 9721 /usr/lib64/libsqlite3.so.0.8.6 2023-11-09T03:56:16.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 435816 2332 /usr/lib64/libcephsqlite.so 2023-11-09T03:56:16.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 15560 38227 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root 1w FIFO 0,13 0t0 180530 pipe 2023-11-09T03:56:16.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root 2w FIFO 0,13 0t0 180531 pipe 2023-11-09T03:56:16.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root 3r FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:16.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root 4w FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:16.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,25] 2023-11-09T03:56:16.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root 6r FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:16.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root 7w FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:16.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,23] 2023-11-09T03:56:16.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root 9r FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:16.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root 10w FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:16.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12] 2023-11-09T03:56:16.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root 12r FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:16.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root 13w FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:16.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root 14w REG 8,1 809589 265321 /var/log/ceph/ceph-mgr.y.log 2023-11-09T03:56:16.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root 15uW REG 0,24 6 1735 /run/ceph/ceph-mgr.y.pid 2023-11-09T03:56:16.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root 16r FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:16.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root 17w FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:16.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root 18u unix 0xffff959a60349540 0t0 177902 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root 19r FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:16.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root 20w FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:16.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root 21r FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:16.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root 22w FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:16.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root 23u IPv4 185569 0t0 TCP smithi076.front.sepia.ceph.com:44172->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87902 safe_time root 25u IPv4 183326 0t0 TCP smithi076.front.sepia.ceph.com:46708->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root txt REG 8,1 4625312 2939 /usr/bin/ceph-mgr 2023-11-09T03:56:16.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-11-09T03:56:16.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 64192 38198 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 65240 274077 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 783584 273412 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 87608 273415 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 331936 273390 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 654400 273423 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 41112936 2759 /usr/lib64/libopenblaso-r0.3.21.so 2023-11-09T03:56:16.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 60264 273417 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 96304 273413 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 237384 273392 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 177920 273418 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 7042240 272781 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2023-11-09T03:56:16.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 70784 273414 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 73976 273198 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 155704 273300 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 2883920 2473 /usr/lib64/libgfortran.so.5.0.0 2023-11-09T03:56:16.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 3870704 2771 /usr/lib64/libflexiblas.so.3.0 2023-11-09T03:56:16.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 3784896 272831 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 132136 272830 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 294392 9898 /usr/lib64/libgomp.so.1.0.0 2023-11-09T03:56:16.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 382824 272782 /usr/lib64/flexiblas/libflexiblas_netlib.so 2023-11-09T03:56:16.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 299952 2469 /usr/lib64/libquadmath.so.0.0.0 2023-11-09T03:56:16.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 8003360 1775 /usr/lib64/librbd.so.1.18.0 2023-11-09T03:56:16.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 31984 42006 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 15336 38209 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 20504 273301 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 40328 272779 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2023-11-09T03:56:16.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 78840 41984 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 75928 9784 /usr/lib64/libjson-c.so.5.0.0 2023-11-09T03:56:16.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 384592 11464 /usr/lib64/libdevmapper.so.1.02 2023-11-09T03:56:16.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 554440 11254 /usr/lib64/libcryptsetup.so.12.9.0 2023-11-09T03:56:16.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 1129568 2920 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 130464 38212 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:16.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 71408 38249 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 38744 38211 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 136048 11808 /usr/lib64/libyaml-0.so.2.0.9 2023-11-09T03:56:16.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 20832 38226 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 246392 59769 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 119688 36903 /usr/lib64/ossl-modules/legacy.so 2023-11-09T03:56:16.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 721976 270244 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2023-11-09T03:56:16.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 2154416 270246 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2023-11-09T03:56:16.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 1116856 42007 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 124184 41989 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 16192 265939 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 283592 38217 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 213376 41995 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 20528 41991 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 41384 38243 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 24160 41992 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:16.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:16.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 2103760 2040 /usr/lib64/libcephfs.so.2.0.0 2023-11-09T03:56:16.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 515320 2482 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:16.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 198136 2576 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 49328 270783 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2023-11-09T03:56:16.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 50232 41979 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 69048 38220 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 36328 9712 /usr/lib64/libuuid.so.1.3.0 2023-11-09T03:56:16.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 15984 38200 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 96936 41994 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 68000 38241 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 101208 41993 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 107736 38215 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 1129032 2042 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 50048 41986 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:16.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 15504 41997 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:16.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:16.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 1892920 1771 /usr/lib64/librados.so.2.0.0 2023-11-09T03:56:16.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:16.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 1294848 9721 /usr/lib64/libsqlite3.so.0.8.6 2023-11-09T03:56:16.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 435816 2332 /usr/lib64/libcephsqlite.so 2023-11-09T03:56:16.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 15560 38227 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root 1w FIFO 0,13 0t0 180530 pipe 2023-11-09T03:56:16.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root 2w FIFO 0,13 0t0 180531 pipe 2023-11-09T03:56:16.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root 3r FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:16.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root 4w FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:16.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,25] 2023-11-09T03:56:16.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root 6r FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:16.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root 7w FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:16.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,23] 2023-11-09T03:56:16.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root 9r FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:16.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root 10w FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:16.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12] 2023-11-09T03:56:16.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root 12r FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:16.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root 13w FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:16.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root 14w REG 8,1 809589 265321 /var/log/ceph/ceph-mgr.y.log 2023-11-09T03:56:16.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root 15uW REG 0,24 6 1735 /run/ceph/ceph-mgr.y.pid 2023-11-09T03:56:16.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root 16r FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:16.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root 17w FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:16.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root 18u unix 0xffff959a60349540 0t0 177902 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root 19r FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:16.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root 20w FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:16.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root 21r FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:16.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root 22w FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:16.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root 23u IPv4 185569 0t0 TCP smithi076.front.sepia.ceph.com:44172->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87903 safe_time root 25u IPv4 183326 0t0 TCP smithi076.front.sepia.ceph.com:46708->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root txt REG 8,1 4625312 2939 /usr/bin/ceph-mgr 2023-11-09T03:56:16.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-11-09T03:56:16.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 64192 38198 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 65240 274077 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 783584 273412 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 87608 273415 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 331936 273390 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 654400 273423 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 41112936 2759 /usr/lib64/libopenblaso-r0.3.21.so 2023-11-09T03:56:16.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 60264 273417 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 96304 273413 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 237384 273392 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 177920 273418 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 7042240 272781 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2023-11-09T03:56:16.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 70784 273414 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 73976 273198 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 155704 273300 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 2883920 2473 /usr/lib64/libgfortran.so.5.0.0 2023-11-09T03:56:16.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 3870704 2771 /usr/lib64/libflexiblas.so.3.0 2023-11-09T03:56:16.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 3784896 272831 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 132136 272830 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 294392 9898 /usr/lib64/libgomp.so.1.0.0 2023-11-09T03:56:16.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 382824 272782 /usr/lib64/flexiblas/libflexiblas_netlib.so 2023-11-09T03:56:16.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 299952 2469 /usr/lib64/libquadmath.so.0.0.0 2023-11-09T03:56:16.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 8003360 1775 /usr/lib64/librbd.so.1.18.0 2023-11-09T03:56:16.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 31984 42006 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 15336 38209 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 20504 273301 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 40328 272779 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2023-11-09T03:56:16.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 78840 41984 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 75928 9784 /usr/lib64/libjson-c.so.5.0.0 2023-11-09T03:56:16.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 384592 11464 /usr/lib64/libdevmapper.so.1.02 2023-11-09T03:56:16.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 554440 11254 /usr/lib64/libcryptsetup.so.12.9.0 2023-11-09T03:56:16.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 1129568 2920 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 130464 38212 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:16.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 71408 38249 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 38744 38211 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 136048 11808 /usr/lib64/libyaml-0.so.2.0.9 2023-11-09T03:56:16.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 20832 38226 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 246392 59769 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 119688 36903 /usr/lib64/ossl-modules/legacy.so 2023-11-09T03:56:16.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 721976 270244 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2023-11-09T03:56:16.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 2154416 270246 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2023-11-09T03:56:16.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 1116856 42007 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 124184 41989 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 16192 265939 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 283592 38217 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 213376 41995 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 20528 41991 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 41384 38243 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 24160 41992 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:16.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:16.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 2103760 2040 /usr/lib64/libcephfs.so.2.0.0 2023-11-09T03:56:16.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 515320 2482 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:16.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 198136 2576 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 49328 270783 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2023-11-09T03:56:16.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 50232 41979 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 69048 38220 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 36328 9712 /usr/lib64/libuuid.so.1.3.0 2023-11-09T03:56:16.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 15984 38200 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 96936 41994 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 68000 38241 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 101208 41993 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 107736 38215 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 1129032 2042 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 50048 41986 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:16.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 15504 41997 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:16.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:16.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 1892920 1771 /usr/lib64/librados.so.2.0.0 2023-11-09T03:56:16.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:16.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 1294848 9721 /usr/lib64/libsqlite3.so.0.8.6 2023-11-09T03:56:16.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 435816 2332 /usr/lib64/libcephsqlite.so 2023-11-09T03:56:16.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 15560 38227 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root 1w FIFO 0,13 0t0 180530 pipe 2023-11-09T03:56:16.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root 2w FIFO 0,13 0t0 180531 pipe 2023-11-09T03:56:16.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root 3r FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:16.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root 4w FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:16.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root 5u a_inode 0,14 0 2077 [eventpoll:6,25] 2023-11-09T03:56:16.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root 6r FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:16.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root 7w FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:16.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root 8u a_inode 0,14 0 2077 [eventpoll:9,23] 2023-11-09T03:56:16.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root 9r FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:16.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root 10w FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:16.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root 11u a_inode 0,14 0 2077 [eventpoll:12] 2023-11-09T03:56:16.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root 12r FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:16.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root 13w FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:16.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root 14w REG 8,1 809589 265321 /var/log/ceph/ceph-mgr.y.log 2023-11-09T03:56:16.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root 15uW REG 0,24 6 1735 /run/ceph/ceph-mgr.y.pid 2023-11-09T03:56:16.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root 16r FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:16.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root 17w FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:16.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root 18u unix 0xffff959a60349540 0t0 177902 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root 19r FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:16.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root 20w FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:16.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root 21r FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:16.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root 22w FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:16.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root 23u IPv4 185569 0t0 TCP smithi076.front.sepia.ceph.com:44172->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87904 fn_anonym root 25u IPv4 183326 0t0 TCP smithi076.front.sepia.ceph.com:46708->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root txt REG 8,1 4625312 2939 /usr/bin/ceph-mgr 2023-11-09T03:56:16.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-11-09T03:56:16.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 64192 38198 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 65240 274077 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 783584 273412 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 87608 273415 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 331936 273390 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 654400 273423 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 41112936 2759 /usr/lib64/libopenblaso-r0.3.21.so 2023-11-09T03:56:16.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 60264 273417 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 96304 273413 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 237384 273392 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 177920 273418 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 7042240 272781 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2023-11-09T03:56:16.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 70784 273414 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 73976 273198 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 155704 273300 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 2883920 2473 /usr/lib64/libgfortran.so.5.0.0 2023-11-09T03:56:16.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 3870704 2771 /usr/lib64/libflexiblas.so.3.0 2023-11-09T03:56:16.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 3784896 272831 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 132136 272830 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 294392 9898 /usr/lib64/libgomp.so.1.0.0 2023-11-09T03:56:16.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 382824 272782 /usr/lib64/flexiblas/libflexiblas_netlib.so 2023-11-09T03:56:16.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 299952 2469 /usr/lib64/libquadmath.so.0.0.0 2023-11-09T03:56:16.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 8003360 1775 /usr/lib64/librbd.so.1.18.0 2023-11-09T03:56:16.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 31984 42006 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 15336 38209 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 20504 273301 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 40328 272779 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2023-11-09T03:56:16.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 78840 41984 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 75928 9784 /usr/lib64/libjson-c.so.5.0.0 2023-11-09T03:56:16.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 384592 11464 /usr/lib64/libdevmapper.so.1.02 2023-11-09T03:56:16.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 554440 11254 /usr/lib64/libcryptsetup.so.12.9.0 2023-11-09T03:56:16.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 1129568 2920 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 130464 38212 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:16.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 71408 38249 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 38744 38211 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 136048 11808 /usr/lib64/libyaml-0.so.2.0.9 2023-11-09T03:56:16.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 20832 38226 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 246392 59769 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 119688 36903 /usr/lib64/ossl-modules/legacy.so 2023-11-09T03:56:16.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 721976 270244 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2023-11-09T03:56:16.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 2154416 270246 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2023-11-09T03:56:16.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 1116856 42007 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 124184 41989 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 16192 265939 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 283592 38217 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 213376 41995 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 20528 41991 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 41384 38243 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 24160 41992 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:16.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:16.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 2103760 2040 /usr/lib64/libcephfs.so.2.0.0 2023-11-09T03:56:16.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 515320 2482 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:16.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 198136 2576 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 49328 270783 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2023-11-09T03:56:16.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 50232 41979 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 69048 38220 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 36328 9712 /usr/lib64/libuuid.so.1.3.0 2023-11-09T03:56:16.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 15984 38200 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 96936 41994 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 68000 38241 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 101208 41993 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 107736 38215 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 1129032 2042 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 50048 41986 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:16.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 15504 41997 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:16.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:16.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 1892920 1771 /usr/lib64/librados.so.2.0.0 2023-11-09T03:56:16.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:16.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 1294848 9721 /usr/lib64/libsqlite3.so.0.8.6 2023-11-09T03:56:16.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 435816 2332 /usr/lib64/libcephsqlite.so 2023-11-09T03:56:16.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 15560 38227 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root 1w FIFO 0,13 0t0 180530 pipe 2023-11-09T03:56:16.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root 2w FIFO 0,13 0t0 180531 pipe 2023-11-09T03:56:16.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root 3r FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:16.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root 4w FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:16.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root 5u a_inode 0,14 0 2077 [eventpoll:6,25] 2023-11-09T03:56:16.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root 6r FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:16.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root 7w FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:16.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root 8u a_inode 0,14 0 2077 [eventpoll:9,23] 2023-11-09T03:56:16.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root 9r FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:16.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root 10w FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:16.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root 11u a_inode 0,14 0 2077 [eventpoll:12] 2023-11-09T03:56:16.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root 12r FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:16.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root 13w FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:16.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root 14w REG 8,1 809589 265321 /var/log/ceph/ceph-mgr.y.log 2023-11-09T03:56:16.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root 15uW REG 0,24 6 1735 /run/ceph/ceph-mgr.y.pid 2023-11-09T03:56:16.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root 16r FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:16.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root 17w FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:16.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root 18u unix 0xffff959a60349540 0t0 177902 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root 19r FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:16.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root 20w FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:16.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root 21r FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:16.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root 22w FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:16.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root 23u IPv4 185569 0t0 TCP smithi076.front.sepia.ceph.com:44172->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87905 flusher root 25u IPv4 183326 0t0 TCP smithi076.front.sepia.ceph.com:46708->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root txt REG 8,1 4625312 2939 /usr/bin/ceph-mgr 2023-11-09T03:56:16.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-11-09T03:56:16.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 64192 38198 /usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 65240 274077 /usr/lib64/python3.9/site-packages/scipy/_lib/_ccallback_c.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 783584 273412 /usr/lib64/python3.9/site-packages/numpy/random/_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 87608 273415 /usr/lib64/python3.9/site-packages/numpy/random/_philox.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 331936 273390 /usr/lib64/python3.9/site-packages/numpy/random/_bounded_integers.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 654400 273423 /usr/lib64/python3.9/site-packages/numpy/random/mtrand.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 41112936 2759 /usr/lib64/libopenblaso-r0.3.21.so 2023-11-09T03:56:16.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 60264 273417 /usr/lib64/python3.9/site-packages/numpy/random/_sfc64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 96304 273413 /usr/lib64/python3.9/site-packages/numpy/random/_mt19937.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 237384 273392 /usr/lib64/python3.9/site-packages/numpy/random/_common.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 177920 273418 /usr/lib64/python3.9/site-packages/numpy/random/bit_generator.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 7042240 272781 /usr/lib64/flexiblas/libflexiblas_fallback_lapack.so 2023-11-09T03:56:16.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 70784 273414 /usr/lib64/python3.9/site-packages/numpy/random/_pcg64.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 73976 273198 /usr/lib64/python3.9/site-packages/numpy/fft/_pocketfft_internal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 155704 273300 /usr/lib64/python3.9/site-packages/numpy/linalg/_umath_linalg.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 2883920 2473 /usr/lib64/libgfortran.so.5.0.0 2023-11-09T03:56:16.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 3870704 2771 /usr/lib64/libflexiblas.so.3.0 2023-11-09T03:56:16.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 3784896 272831 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 132136 272830 /usr/lib64/python3.9/site-packages/numpy/core/_multiarray_tests.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 294392 9898 /usr/lib64/libgomp.so.1.0.0 2023-11-09T03:56:16.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 382824 272782 /usr/lib64/flexiblas/libflexiblas_netlib.so 2023-11-09T03:56:16.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 299952 2469 /usr/lib64/libquadmath.so.0.0.0 2023-11-09T03:56:16.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 8003360 1775 /usr/lib64/librbd.so.1.18.0 2023-11-09T03:56:16.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 31984 42006 /usr/lib64/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 15336 38209 /usr/lib64/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 20504 273301 /usr/lib64/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 40328 272779 /usr/lib64/flexiblas/libflexiblas_openblas-openmp.so 2023-11-09T03:56:16.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 78840 41984 /usr/lib64/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 75928 9784 /usr/lib64/libjson-c.so.5.0.0 2023-11-09T03:56:16.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 384592 11464 /usr/lib64/libdevmapper.so.1.02 2023-11-09T03:56:16.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 554440 11254 /usr/lib64/libcryptsetup.so.12.9.0 2023-11-09T03:56:16.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 1129568 2920 /usr/lib64/python3.9/site-packages/rbd.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 130464 38212 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-11-09T03:56:16.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 71408 38249 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 38744 38211 /usr/lib64/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 136048 11808 /usr/lib64/libyaml-0.so.2.0.9 2023-11-09T03:56:16.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 20832 38226 /usr/lib64/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 246392 59769 /usr/lib64/python3.9/site-packages/yaml/_yaml.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 119688 36903 /usr/lib64/ossl-modules/legacy.so 2023-11-09T03:56:16.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 721976 270244 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2023-11-09T03:56:16.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 2154416 270246 /usr/lib64/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so 2023-11-09T03:56:16.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 1116856 42007 /usr/lib64/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 124184 41989 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 16192 265939 /usr/lib64/python3.9/site-packages/markupsafe/_speedups.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 283592 38217 /usr/lib64/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 213376 41995 /usr/lib64/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 20528 41991 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 41384 38243 /usr/lib64/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 24160 41992 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:16.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:16.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 2103760 2040 /usr/lib64/libcephfs.so.2.0.0 2023-11-09T03:56:16.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 515320 2482 /usr/lib64/python3.9/site-packages/cephfs.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:16.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 198136 2576 /usr/lib64/python3.9/site-packages/_cffi_backend.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 49328 270783 /usr/lib64/python3.9/site-packages/bcrypt/_bcrypt.abi3.so 2023-11-09T03:56:16.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 50232 41979 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 69048 38220 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 36328 9712 /usr/lib64/libuuid.so.1.3.0 2023-11-09T03:56:16.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 15984 38200 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 96936 41994 /usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 68000 38241 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 101208 41993 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 107736 38215 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 1129032 2042 /usr/lib64/python3.9/site-packages/rados.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 50048 41986 /usr/lib64/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:16.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 15504 41997 /usr/lib64/python3.9/lib-dynload/_uuid.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:16.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:16.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 1892920 1771 /usr/lib64/librados.so.2.0.0 2023-11-09T03:56:16.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:16.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 1294848 9721 /usr/lib64/libsqlite3.so.0.8.6 2023-11-09T03:56:16.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 435816 2332 /usr/lib64/libcephsqlite.so 2023-11-09T03:56:16.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 15560 38227 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root 1w FIFO 0,13 0t0 180530 pipe 2023-11-09T03:56:16.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root 2w FIFO 0,13 0t0 180531 pipe 2023-11-09T03:56:16.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root 3r FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:16.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root 4w FIFO 0,13 0t0 177888 pipe 2023-11-09T03:56:16.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,25] 2023-11-09T03:56:16.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root 6r FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:16.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root 7w FIFO 0,13 0t0 177890 pipe 2023-11-09T03:56:16.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,23] 2023-11-09T03:56:16.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root 9r FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:16.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root 10w FIFO 0,13 0t0 177891 pipe 2023-11-09T03:56:16.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12] 2023-11-09T03:56:16.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root 12r FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:16.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root 13w FIFO 0,13 0t0 177892 pipe 2023-11-09T03:56:16.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root 14w REG 8,1 809589 265321 /var/log/ceph/ceph-mgr.y.log 2023-11-09T03:56:16.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root 15uW REG 0,24 6 1735 /run/ceph/ceph-mgr.y.pid 2023-11-09T03:56:16.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root 16r FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:16.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root 17w FIFO 0,13 0t0 177901 pipe 2023-11-09T03:56:16.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root 18u unix 0xffff959a60349540 0t0 177902 /var/run/ceph/ceph-mgr.y.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root 19r FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:16.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root 20w FIFO 0,13 0t0 177906 pipe 2023-11-09T03:56:16.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root 21r FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:16.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root 22w FIFO 0,13 0t0 177907 pipe 2023-11-09T03:56:16.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root 23u IPv4 185569 0t0 TCP smithi076.front.sepia.ceph.com:44172->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mgr 87788 87906 safe_time root 25u IPv4 183326 0t0 TCP smithi076.front.sepia.ceph.com:46708->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.205 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.205 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.205 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root txt REG 8,1 185296 11759 /usr/bin/sudo 2023-11-09T03:56:16.205 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 357408 10120 /usr/lib64/libnss_systemd.so.2 2023-11-09T03:56:16.206 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:16.206 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.206 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-11-09T03:56:16.206 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:16.206 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 19600 10967 /usr/lib64/libpam_misc.so.0.82.1 2023-11-09T03:56:16.206 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 19400 32305 /usr/lib64/security/pam_succeed_if.so 2023-11-09T03:56:16.207 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 514280 32422 /usr/lib64/security/pam_systemd.so 2023-11-09T03:56:16.207 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 27536 32284 /usr/lib64/security/pam_limits.so 2023-11-09T03:56:16.207 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 48392 10228 /usr/lib64/libcrack.so.2.9.0 2023-11-09T03:56:16.207 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 32608 10961 /usr/lib64/libpwquality.so.1.0.2 2023-11-09T03:56:16.207 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 15352 32282 /usr/lib64/security/pam_keyinit.so 2023-11-09T03:56:16.207 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 15840 32264 /usr/lib64/security/pam_pwquality.so 2023-11-09T03:56:16.208 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 14928 32270 /usr/lib64/security/pam_deny.so 2023-11-09T03:56:16.208 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 56712 32310 /usr/lib64/security/pam_unix.so 2023-11-09T03:56:16.208 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 15184 32274 /usr/lib64/security/pam_faildelay.so 2023-11-09T03:56:16.208 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.208 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.208 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.209 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.209 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.209 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.209 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.209 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.209 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.210 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.210 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.210 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.210 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-11-09T03:56:16.210 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.211 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.211 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-11-09T03:56:16.211 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.211 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.211 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-11-09T03:56:16.211 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 19464 32272 /usr/lib64/security/pam_env.so 2023-11-09T03:56:16.212 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 472568 54513 /usr/libexec/sudo/sudoers.so 2023-11-09T03:56:16.212 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.212 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.212 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:16.212 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-11-09T03:56:16.212 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-11-09T03:56:16.213 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-11-09T03:56:16.213 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-11-09T03:56:16.213 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-11-09T03:56:16.213 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-11-09T03:56:16.213 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-11-09T03:56:16.214 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-11-09T03:56:16.214 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-11-09T03:56:16.214 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:16.214 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 119832 54509 /usr/libexec/sudo/libsudo_util.so.0.0.0 2023-11-09T03:56:16.214 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.215 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:16.215 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:16.215 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.215 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-11-09T03:56:16.215 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root 0r FIFO 0,13 0t0 182400 pipe 2023-11-09T03:56:16.215 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:16.216 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:16.216 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root 3r FIFO 0,13 0t0 184340 pipe 2023-11-09T03:56:16.216 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root 4w FIFO 0,13 0t0 184340 pipe 2023-11-09T03:56:16.216 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root 5u netlink 0t0 184345 AUDIT 2023-11-09T03:56:16.216 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87907 root 6u unix 0xffff959a618fbb80 0t0 184346 type=DGRAM (CONNECTED) 2023-11-09T03:56:16.216 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.217 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.217 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root txt REG 8,1 185296 11759 /usr/bin/sudo 2023-11-09T03:56:16.217 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 357408 10120 /usr/lib64/libnss_systemd.so.2 2023-11-09T03:56:16.217 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:16.217 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.217 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-11-09T03:56:16.218 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:16.218 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 19600 10967 /usr/lib64/libpam_misc.so.0.82.1 2023-11-09T03:56:16.218 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 19400 32305 /usr/lib64/security/pam_succeed_if.so 2023-11-09T03:56:16.218 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 514280 32422 /usr/lib64/security/pam_systemd.so 2023-11-09T03:56:16.218 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 27536 32284 /usr/lib64/security/pam_limits.so 2023-11-09T03:56:16.219 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 48392 10228 /usr/lib64/libcrack.so.2.9.0 2023-11-09T03:56:16.219 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 32608 10961 /usr/lib64/libpwquality.so.1.0.2 2023-11-09T03:56:16.219 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 15352 32282 /usr/lib64/security/pam_keyinit.so 2023-11-09T03:56:16.219 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 15840 32264 /usr/lib64/security/pam_pwquality.so 2023-11-09T03:56:16.219 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 14928 32270 /usr/lib64/security/pam_deny.so 2023-11-09T03:56:16.219 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 56712 32310 /usr/lib64/security/pam_unix.so 2023-11-09T03:56:16.220 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.220 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.220 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.220 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.220 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 15184 32274 /usr/lib64/security/pam_faildelay.so 2023-11-09T03:56:16.220 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.221 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.221 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.221 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.221 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.221 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.221 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-11-09T03:56:16.222 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.222 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.222 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.222 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-11-09T03:56:16.222 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.223 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.223 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.223 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.223 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-11-09T03:56:16.223 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 19464 32272 /usr/lib64/security/pam_env.so 2023-11-09T03:56:16.224 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 472568 54513 /usr/libexec/sudo/sudoers.so 2023-11-09T03:56:16.224 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:16.224 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-11-09T03:56:16.224 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-11-09T03:56:16.224 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-11-09T03:56:16.225 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-11-09T03:56:16.225 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-11-09T03:56:16.225 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-11-09T03:56:16.225 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-11-09T03:56:16.225 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-11-09T03:56:16.226 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-11-09T03:56:16.226 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.226 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:16.226 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 119832 54509 /usr/libexec/sudo/libsudo_util.so.0.0.0 2023-11-09T03:56:16.226 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.226 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:16.227 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:16.227 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.227 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-11-09T03:56:16.227 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root 0r FIFO 0,13 0t0 182404 pipe 2023-11-09T03:56:16.227 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:16.227 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:16.228 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root 3r FIFO 0,13 0t0 182426 pipe 2023-11-09T03:56:16.228 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root 4w FIFO 0,13 0t0 182426 pipe 2023-11-09T03:56:16.228 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root 5u netlink 0t0 182438 AUDIT 2023-11-09T03:56:16.228 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87911 root 6u unix 0xffff959a60608000 0t0 182439 type=DGRAM (CONNECTED) 2023-11-09T03:56:16.228 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.228 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.229 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root txt REG 8,1 185296 11759 /usr/bin/sudo 2023-11-09T03:56:16.229 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 357408 10120 /usr/lib64/libnss_systemd.so.2 2023-11-09T03:56:16.229 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:16.229 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.229 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-11-09T03:56:16.229 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:16.230 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 19600 10967 /usr/lib64/libpam_misc.so.0.82.1 2023-11-09T03:56:16.230 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 19400 32305 /usr/lib64/security/pam_succeed_if.so 2023-11-09T03:56:16.230 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 514280 32422 /usr/lib64/security/pam_systemd.so 2023-11-09T03:56:16.230 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 27536 32284 /usr/lib64/security/pam_limits.so 2023-11-09T03:56:16.230 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 48392 10228 /usr/lib64/libcrack.so.2.9.0 2023-11-09T03:56:16.230 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 32608 10961 /usr/lib64/libpwquality.so.1.0.2 2023-11-09T03:56:16.231 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 15352 32282 /usr/lib64/security/pam_keyinit.so 2023-11-09T03:56:16.231 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 15840 32264 /usr/lib64/security/pam_pwquality.so 2023-11-09T03:56:16.231 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.231 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 14928 32270 /usr/lib64/security/pam_deny.so 2023-11-09T03:56:16.231 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 56712 32310 /usr/lib64/security/pam_unix.so 2023-11-09T03:56:16.232 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.232 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.232 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.232 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.232 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.232 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.233 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.233 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.233 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.233 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.234 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.234 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-11-09T03:56:16.234 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.234 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-11-09T03:56:16.234 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.235 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.235 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.235 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-11-09T03:56:16.235 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 15184 32274 /usr/lib64/security/pam_faildelay.so 2023-11-09T03:56:16.235 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 472568 54513 /usr/libexec/sudo/sudoers.so 2023-11-09T03:56:16.235 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:16.237 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.237 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 19464 32272 /usr/lib64/security/pam_env.so 2023-11-09T03:56:16.237 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-11-09T03:56:16.237 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-11-09T03:56:16.237 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-11-09T03:56:16.237 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-11-09T03:56:16.238 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-11-09T03:56:16.238 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-11-09T03:56:16.238 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-11-09T03:56:16.238 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-11-09T03:56:16.238 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-11-09T03:56:16.238 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.239 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:16.239 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 119832 54509 /usr/libexec/sudo/libsudo_util.so.0.0.0 2023-11-09T03:56:16.239 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.239 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:16.239 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:16.239 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.240 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-11-09T03:56:16.240 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root 0r FIFO 0,13 0t0 182408 pipe 2023-11-09T03:56:16.240 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:16.240 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:16.240 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root 3r FIFO 0,13 0t0 180587 pipe 2023-11-09T03:56:16.240 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root 4w FIFO 0,13 0t0 180587 pipe 2023-11-09T03:56:16.241 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root 5u netlink 0t0 177933 AUDIT 2023-11-09T03:56:16.241 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87918 root 6u unix 0xffff959a61973740 0t0 177934 type=DGRAM (CONNECTED) 2023-11-09T03:56:16.241 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.241 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.241 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root txt REG 8,1 185296 11759 /usr/bin/sudo 2023-11-09T03:56:16.241 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 357408 10120 /usr/lib64/libnss_systemd.so.2 2023-11-09T03:56:16.242 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:16.242 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.242 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-11-09T03:56:16.242 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:16.242 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 19600 10967 /usr/lib64/libpam_misc.so.0.82.1 2023-11-09T03:56:16.243 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 19400 32305 /usr/lib64/security/pam_succeed_if.so 2023-11-09T03:56:16.243 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 514280 32422 /usr/lib64/security/pam_systemd.so 2023-11-09T03:56:16.243 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 27536 32284 /usr/lib64/security/pam_limits.so 2023-11-09T03:56:16.243 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 48392 10228 /usr/lib64/libcrack.so.2.9.0 2023-11-09T03:56:16.243 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 32608 10961 /usr/lib64/libpwquality.so.1.0.2 2023-11-09T03:56:16.243 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 15352 32282 /usr/lib64/security/pam_keyinit.so 2023-11-09T03:56:16.244 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 15840 32264 /usr/lib64/security/pam_pwquality.so 2023-11-09T03:56:16.244 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 14928 32270 /usr/lib64/security/pam_deny.so 2023-11-09T03:56:16.244 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 56712 32310 /usr/lib64/security/pam_unix.so 2023-11-09T03:56:16.244 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.244 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.244 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.245 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.245 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.245 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.245 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.245 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.246 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-11-09T03:56:16.246 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.246 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.246 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.246 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.246 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.247 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.247 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 472568 54513 /usr/libexec/sudo/sudoers.so 2023-11-09T03:56:16.247 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.247 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 15184 32274 /usr/lib64/security/pam_faildelay.so 2023-11-09T03:56:16.247 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-11-09T03:56:16.247 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.248 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.248 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-11-09T03:56:16.248 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 19464 32272 /usr/lib64/security/pam_env.so 2023-11-09T03:56:16.248 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:16.248 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-11-09T03:56:16.248 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-11-09T03:56:16.249 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-11-09T03:56:16.249 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-11-09T03:56:16.249 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-11-09T03:56:16.249 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-11-09T03:56:16.249 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-11-09T03:56:16.249 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-11-09T03:56:16.250 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-11-09T03:56:16.250 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.250 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:16.250 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 119832 54509 /usr/libexec/sudo/libsudo_util.so.0.0.0 2023-11-09T03:56:16.250 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.250 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:16.251 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:16.251 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.251 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-11-09T03:56:16.251 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root 0r FIFO 0,13 0t0 183332 pipe 2023-11-09T03:56:16.251 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:16.251 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:16.252 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root 3r FIFO 0,13 0t0 183351 pipe 2023-11-09T03:56:16.252 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root 4w FIFO 0,13 0t0 183351 pipe 2023-11-09T03:56:16.252 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root 5u netlink 0t0 181496 AUDIT 2023-11-09T03:56:16.252 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 87931 root 6u unix 0xffff959a48378000 0t0 181497 type=DGRAM (CONNECTED) 2023-11-09T03:56:16.252 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87974 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.252 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87974 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.253 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87974 root txt REG 8,1 15448 10498 /usr/bin/python3.9 2023-11-09T03:56:16.253 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87974 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:16.253 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87974 root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:16.253 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87974 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.253 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87974 root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.253 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87974 root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.254 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87974 root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.254 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87974 root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.254 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87974 root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.254 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87974 root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.254 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87974 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.255 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87974 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:16.255 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87974 root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:16.255 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87974 root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.255 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87974 root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.255 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87974 root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.256 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87974 root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.256 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87974 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.256 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87974 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:16.256 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87974 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.256 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87974 root 0r FIFO 0,13 0t0 182400 pipe 2023-11-09T03:56:16.256 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87974 root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:16.257 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87974 root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:16.257 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87982 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.257 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87982 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.257 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87982 root txt REG 8,1 15448 10498 /usr/bin/python3.9 2023-11-09T03:56:16.257 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87982 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:16.257 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87982 root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:16.258 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87982 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.258 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87982 root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.258 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87982 root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.258 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87982 root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.258 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87982 root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.259 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87982 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.259 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87982 root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.259 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87982 root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.259 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87982 root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.259 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87982 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.259 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87982 root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:16.260 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87982 root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.260 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87982 root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.260 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87982 root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.260 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87982 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:16.260 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87982 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:16.260 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87982 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.261 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87982 root 0r FIFO 0,13 0t0 182404 pipe 2023-11-09T03:56:16.261 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87982 root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:16.261 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87982 root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:16.261 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87993 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.261 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87993 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.261 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87993 root txt REG 8,1 15448 10498 /usr/bin/python3.9 2023-11-09T03:56:16.262 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87993 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:16.262 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87993 root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:16.262 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87993 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.262 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87993 root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.262 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87993 root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.262 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87993 root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.263 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87993 root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.263 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87993 root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.263 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87993 root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.263 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87993 root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.263 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87993 root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.263 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87993 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.264 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87993 root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.264 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87993 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:16.264 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87993 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.264 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87993 root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:16.264 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87993 root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.265 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87993 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:16.265 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87993 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.265 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87993 root 0r FIFO 0,13 0t0 183332 pipe 2023-11-09T03:56:16.265 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87993 root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:16.265 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87993 root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:16.265 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87994 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.266 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87994 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.266 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87994 root txt REG 8,1 15448 10498 /usr/bin/python3.9 2023-11-09T03:56:16.266 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87994 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:16.266 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87994 root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:16.266 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87994 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.267 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87994 root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.267 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87994 root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.267 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87994 root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.267 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87994 root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.267 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87994 root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.267 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87994 root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.268 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87994 root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.268 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87994 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.268 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87994 root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.268 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87994 root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.268 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87994 root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:16.268 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87994 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.269 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87994 root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:16.269 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87994 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:16.269 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87994 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:16.269 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87994 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.269 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87994 root 0r FIFO 0,13 0t0 182408 pipe 2023-11-09T03:56:16.269 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87994 root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:16.270 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 87994 root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:16.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:16.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:16.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:16.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:16.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:16.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:16.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:16.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:16.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:16.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:16.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:16.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:16.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:16.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:16.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:16.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:16.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:16.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:16.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:16.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:16.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:16.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:16.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:16.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:16.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:16.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:16.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:16.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:16.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:16.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:16.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:16.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:16.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:16.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:16.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:16.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:16.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:16.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 14w REG 8,1 160176902 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:16.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:16.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:16.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:16.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:16.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:16.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:16.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:16.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:16.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:16.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:16.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:16.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:16.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:16.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:16.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:16.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:16.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:16.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:16.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:16.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:16.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:16.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:16.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:16.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:16.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:16.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:16.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:16.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:16.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:16.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:16.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:16.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:16.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:16.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:16.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:16.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:16.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:16.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:16.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:16.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:16.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:16.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:16.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:16.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:16.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:16.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:16.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:16.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:16.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:16.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:16.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:16.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:16.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:16.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:16.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:16.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:16.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:16.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:16.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:16.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:16.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:16.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:16.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:16.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:16.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:16.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:16.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:16.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:16.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:16.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:16.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:16.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:16.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:16.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:16.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:16.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:16.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:16.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:16.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:16.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:16.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 14w REG 8,1 160176336 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:16.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:16.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:16.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:16.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:16.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:16.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:16.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:16.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:16.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:16.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:16.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:16.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:16.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:16.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:16.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:16.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:16.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:16.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:16.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:16.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:16.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:16.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:16.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:16.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:16.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:16.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:16.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:16.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:16.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:16.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:16.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:16.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:16.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:16.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:16.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:16.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:16.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:16.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:16.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:16.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:16.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:16.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:16.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88007 log root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:16.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:16.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:16.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:16.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:16.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:16.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:16.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:16.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:16.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:16.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:16.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:16.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:16.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:16.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:16.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:16.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:16.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:16.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:16.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:16.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:16.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:16.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:16.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:16.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:16.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:16.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:16.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:16.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:16.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:16.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:16.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:16.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:16.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:16.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:16.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:16.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:16.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:16.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 14w REG 8,1 160176336 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:16.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:16.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:16.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:16.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:16.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:16.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:16.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:16.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:16.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:16.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:16.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:16.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:16.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:16.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:16.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:16.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:16.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:16.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:16.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:16.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:16.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:16.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:16.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:16.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:16.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:16.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:16.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:16.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:16.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:16.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:16.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:16.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:16.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:16.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:16.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:16.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:16.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:16.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:16.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:16.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:16.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:16.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:16.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88009 msgr-work root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:16.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:16.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:16.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:16.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:16.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:16.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:16.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:16.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:16.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:16.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:16.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:16.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:16.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:16.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:16.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:16.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:16.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:16.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:16.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:16.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:16.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:16.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:16.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:16.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:16.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:16.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:16.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:16.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:16.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:16.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:16.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:16.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:16.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:16.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:16.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:16.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:16.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:16.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 14w REG 8,1 160176336 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:16.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:16.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:16.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:16.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:16.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:16.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:16.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:16.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:16.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:16.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:16.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:16.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:16.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:16.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:16.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:16.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:16.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:16.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:16.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:16.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:16.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:16.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:16.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:16.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:16.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:16.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:16.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:16.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:16.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:16.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:16.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:16.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:16.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:16.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:16.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:16.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:16.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:16.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:16.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:16.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:16.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:16.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:16.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88010 msgr-work root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:16.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:16.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:16.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:16.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:16.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:16.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:16.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:16.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:16.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:16.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:16.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:16.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:16.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:16.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:16.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:16.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:16.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:16.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:16.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:16.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:16.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:16.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:16.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:16.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:16.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:16.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:16.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:16.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:16.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:16.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:16.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:16.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:16.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:16.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:16.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:16.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:16.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:16.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 14w REG 8,1 160176336 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:16.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:16.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:16.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:16.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:16.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:16.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:16.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:16.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:16.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:16.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:16.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:16.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:16.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:16.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:16.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:16.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:16.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:16.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:16.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:16.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:16.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:16.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:16.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:16.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:16.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:16.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:16.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:16.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:16.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:16.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:16.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:16.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:16.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:16.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:16.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:16.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:16.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:16.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:16.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:16.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:16.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:16.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:16.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88011 msgr-work root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:16.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:16.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:16.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:16.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:16.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:16.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:16.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:16.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:16.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:16.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:16.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:16.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:16.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:16.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:16.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:16.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:16.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:16.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:16.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:16.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:16.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:16.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:16.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:16.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:16.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:16.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:16.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:16.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:16.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:16.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:16.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:16.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:16.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:16.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:16.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:16.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:16.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:16.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 14w REG 8,1 160176336 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:16.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:16.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:16.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:16.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:16.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:16.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:16.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:16.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:16.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:16.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:16.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:16.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:16.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:16.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:16.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:16.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:16.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:16.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:16.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:16.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:16.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:16.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:16.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:16.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:16.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:16.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:16.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:16.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:16.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:16.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:16.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:16.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:16.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:16.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:16.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:16.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:16.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:16.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:16.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:16.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:16.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:16.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:16.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88015 service root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:16.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:16.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:16.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:16.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:16.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:16.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:16.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:16.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:16.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:16.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:16.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:16.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:16.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:16.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:16.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:16.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:16.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:16.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:16.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:16.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:16.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:16.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:16.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:16.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:16.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:16.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:16.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:16.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:16.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:16.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:16.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:16.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:16.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:16.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:16.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:16.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:16.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:16.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 14w REG 8,1 160176336 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:16.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:16.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:16.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:16.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:16.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:16.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:16.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:16.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:16.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:16.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:16.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:16.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:16.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:16.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:16.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:16.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:16.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:16.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:16.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:16.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:16.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:16.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:16.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:16.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:16.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:16.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:16.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:16.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:16.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:16.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:16.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:16.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:16.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:16.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:16.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:16.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:16.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:16.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:16.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:16.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:16.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:16.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:16.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88016 admin_soc root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:16.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:16.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:16.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:16.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:16.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:16.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:16.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:16.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:16.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:16.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:16.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:16.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:16.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:16.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:16.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:16.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:16.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:16.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:16.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:16.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:16.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:16.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:16.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:16.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:16.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:16.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:16.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:16.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:16.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:16.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:16.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:16.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:16.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:16.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:16.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:16.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:16.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:16.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 14w REG 8,1 160176336 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:16.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:16.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:16.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:16.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:16.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:16.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:16.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:16.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:16.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:16.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:16.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:16.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:16.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:16.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:16.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:16.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:16.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:16.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:16.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:16.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:16.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:16.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:16.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:16.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:16.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:16.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:16.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:16.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:16.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:16.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:16.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:16.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:16.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:16.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:16.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:16.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:16.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:16.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:16.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:16.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:16.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:16.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:16.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88080 signal_ha root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:16.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:16.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:16.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:16.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:16.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:16.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:16.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:16.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:16.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:16.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:16.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:16.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:16.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:16.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:16.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:16.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:16.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:16.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:16.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:16.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:16.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:16.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:16.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:16.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:16.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:16.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:16.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:16.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:16.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:16.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:16.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:16.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:16.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:16.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:16.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:16.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:16.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:16.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 14w REG 8,1 160176336 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:16.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:16.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:16.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:16.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:16.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:16.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:16.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:16.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:16.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:16.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:16.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:16.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:16.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:16.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:16.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:16.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:16.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:16.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:16.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:16.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:16.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:16.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:16.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:16.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:16.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:16.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:16.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:16.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:16.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:16.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:16.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:16.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:16.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:16.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:16.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:16.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:16.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:16.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:16.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:16.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:16.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:16.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:16.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88081 io_contex root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:16.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:16.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:16.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:16.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:16.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:16.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:16.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:16.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:16.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:16.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:16.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:16.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:16.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:16.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:16.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:16.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:16.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:16.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:16.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:16.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:16.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:16.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:16.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:16.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:16.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:16.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:16.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:16.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:16.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:16.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:16.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:16.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:16.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:16.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:16.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:16.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:16.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:16.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 14w REG 8,1 160176336 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:16.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:16.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:16.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:16.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:16.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:16.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:16.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:16.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:16.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:16.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:16.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:16.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:16.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:16.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:16.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:16.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:16.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:16.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:16.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:16.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:16.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:16.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:16.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:16.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:16.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:16.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:16.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:16.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:16.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:16.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:16.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:16.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:16.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:16.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:16.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:16.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:16.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:16.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:16.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:16.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:16.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:16.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:16.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88082 io_contex root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:16.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:16.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:16.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:16.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:16.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:16.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:16.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:16.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:16.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:16.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:16.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:16.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:16.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:16.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:16.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:16.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:16.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:16.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:16.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:16.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:16.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:16.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:16.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:16.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:16.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:16.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:16.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:16.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:16.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:16.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:16.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:16.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:16.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:16.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:16.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:16.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:16.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:16.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 14w REG 8,1 160176336 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:16.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:16.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:16.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:16.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:16.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:16.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:16.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:16.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:16.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:16.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:16.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:16.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:16.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:16.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:16.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:16.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:16.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:16.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:16.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:16.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:16.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:16.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:16.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:16.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:16.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:16.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:16.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:16.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:16.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:16.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:16.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:16.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:16.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:16.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:16.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:16.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:16.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:16.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:16.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:16.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:16.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:16.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:16.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88096 OpHistory root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:16.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:16.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:16.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:16.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:16.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:16.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:16.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:16.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:16.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:16.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:16.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:16.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:16.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:16.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:16.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:16.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:16.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:16.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:16.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:16.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:16.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:16.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:16.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:16.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:16.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:16.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:16.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:16.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:16.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:16.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:16.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:16.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:16.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:16.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:16.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:16.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:16.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:16.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 14w REG 8,1 160176336 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:16.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:16.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:16.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:16.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:16.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:16.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:16.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:16.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:16.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:16.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:16.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:16.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:16.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:16.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:16.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:16.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:16.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:16.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:16.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:16.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:16.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:16.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:16.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:16.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:16.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:16.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:16.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:16.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:16.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:16.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:16.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:16.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:16.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:16.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:16.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:16.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:16.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:16.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:16.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:16.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:16.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:16.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:16.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88097 ceph_time root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:16.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:16.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:16.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:16.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:16.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:16.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:16.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:16.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:16.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:16.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:16.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:16.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:16.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:16.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:16.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:16.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:16.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:16.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:16.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:16.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:16.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:16.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:16.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:16.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:16.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:16.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:16.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:16.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:16.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:16.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:16.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:16.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:16.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:16.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:16.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:16.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:16.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:16.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 14w REG 8,1 160176336 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:16.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:16.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:16.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:16.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:16.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:16.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:16.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:16.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:16.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:16.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:16.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:16.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:16.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:16.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:16.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:16.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:16.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:16.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:16.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:16.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:16.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:16.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:16.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:16.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:16.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:16.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:16.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:16.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:16.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:16.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:16.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:16.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:16.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:16.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:16.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:16.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:16.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:16.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:16.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:16.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:16.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:16.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:16.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88144 ceph-osd root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:16.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:16.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:16.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:16.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:16.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:16.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:16.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:16.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:16.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:16.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:16.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:16.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:16.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:16.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:16.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:16.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:16.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:16.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:16.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:16.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:16.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:16.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:16.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:16.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:16.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:16.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:16.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:16.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:16.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:16.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:16.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:16.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:16.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:16.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:16.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:16.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:16.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:16.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 14w REG 8,1 160176336 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:16.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:16.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:16.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:16.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:16.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:16.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:16.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:16.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:16.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:16.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:16.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:16.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:16.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:16.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:16.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:16.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:16.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:16.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:16.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:16.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:16.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:16.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:16.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:16.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:16.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:16.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:16.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:16.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:16.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:16.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:16.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:16.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:16.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:16.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:16.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:16.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:16.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:16.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:16.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:16.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:16.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:16.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:16.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88158 ceph-osd root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:16.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:16.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:16.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:16.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:16.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:16.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:16.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:16.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:16.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:16.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:16.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:16.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:16.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:16.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:16.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:16.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:16.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:16.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:16.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:16.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:16.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:16.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:16.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:16.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:16.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:16.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:16.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:16.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:16.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:16.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:16.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:16.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:16.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:16.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:16.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:16.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:16.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:16.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 14w REG 8,1 160176336 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:16.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:16.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:16.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:16.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:16.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:16.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:16.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:16.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:16.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:16.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:16.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:16.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:16.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:16.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:16.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:16.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:16.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:16.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:16.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:16.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:16.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:16.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:16.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:16.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:16.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:16.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:16.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:16.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:16.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:16.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:16.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:16.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:16.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:16.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:16.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:16.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:16.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:16.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:16.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:16.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:16.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:16.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:16.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88173 ceph-osd root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:16.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:16.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:16.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:16.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:16.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:16.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:16.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:16.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:16.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:16.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:16.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:16.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:16.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:16.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:16.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:16.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:16.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:16.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:16.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:16.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:16.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:16.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:16.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:16.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:16.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:16.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:16.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:16.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:16.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:16.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:16.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:16.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:16.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:16.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:16.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:16.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:16.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:16.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 14w REG 8,1 160176336 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:16.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:16.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:16.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:16.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:16.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:16.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:16.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:16.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:16.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:16.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:16.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:16.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:16.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:16.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:16.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:16.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:16.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:16.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:16.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:16.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:16.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:16.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:16.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:16.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:16.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:16.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:16.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:16.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:16.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:16.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:16.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:16.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:16.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:16.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:16.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:16.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:16.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:16.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:16.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:16.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:16.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:16.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:16.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88181 ceph-osd root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:16.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:16.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:16.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:16.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:16.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:16.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:16.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:16.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:16.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:16.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:16.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:16.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:16.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:16.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:16.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:16.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:16.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:16.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:16.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:16.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:16.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:16.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:16.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:16.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:16.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:16.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:16.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:16.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:16.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:16.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:16.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:16.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:16.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:16.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:16.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:16.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:16.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:16.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 14w REG 8,1 160176336 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:16.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:16.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:16.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:16.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:16.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:16.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:16.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:16.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:16.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:16.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:16.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:16.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:16.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:16.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:16.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:16.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:16.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:16.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:16.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:16.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:16.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:16.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:16.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:16.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:16.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:16.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:16.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:16.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:16.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:16.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:16.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:16.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:16.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:16.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:16.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:16.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:16.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:16.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:16.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:16.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:16.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:16.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:16.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88189 ceph-osd root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:16.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:16.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:16.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:16.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:16.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:16.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:16.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:16.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:16.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:16.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:16.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:16.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:16.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:16.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:16.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:16.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:16.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:16.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:16.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:16.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:16.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:16.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:16.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:16.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:16.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:16.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:16.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:16.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:16.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:16.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:16.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:16.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:16.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:16.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:16.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:16.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:16.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:16.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 14w REG 8,1 160176336 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:16.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:16.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:16.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:16.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:16.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:16.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:16.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:16.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:16.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:16.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:16.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:16.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:16.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:16.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:16.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:16.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:16.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:16.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:16.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:16.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:16.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:16.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:16.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:16.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:16.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:16.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:16.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:16.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:16.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:16.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:16.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:16.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:16.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:16.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:16.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:16.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:16.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:16.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:16.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:16.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:16.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:16.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:16.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88197 ceph-osd root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:16.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:16.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:16.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:16.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:16.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:16.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:16.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:16.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:16.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:16.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:16.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:16.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:16.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:16.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:16.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:16.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:16.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:16.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:16.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:16.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:16.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:16.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:16.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:16.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:16.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:16.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:16.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:16.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:16.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:16.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:16.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:16.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:16.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:16.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:16.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:16.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:16.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:16.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 14w REG 8,1 160176336 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:16.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:16.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:16.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:16.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:16.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:16.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:16.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:16.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:16.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:16.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:16.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:16.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:16.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:16.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:16.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:16.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:16.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:16.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:16.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:16.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:16.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:16.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:16.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:16.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:16.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:16.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:16.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:16.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:16.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:16.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:16.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:16.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:16.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:16.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:16.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:16.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:16.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:16.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:16.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:16.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:16.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:16.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:16.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88205 ceph-osd root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:16.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:16.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:16.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:16.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:16.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:16.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:16.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:16.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:16.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:16.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:16.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:16.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:16.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:16.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:16.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:16.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:16.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:16.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:16.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:16.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:16.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:16.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:16.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:16.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:16.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:16.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:16.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:16.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:16.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:16.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:16.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:16.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:16.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:16.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:16.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:16.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:16.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:16.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 14w REG 8,1 160176336 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:16.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:16.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:16.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:16.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:16.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:16.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:16.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:16.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:16.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:16.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:16.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:16.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:16.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:16.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:16.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:16.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:16.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:16.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:16.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:16.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:16.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:16.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:16.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:16.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:16.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:16.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:16.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:16.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:16.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:16.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:16.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:16.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:16.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:16.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:16.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:16.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:16.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:16.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:16.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:16.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:16.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:16.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:16.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88220 ceph-osd root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:16.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:16.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:16.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:16.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:16.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:16.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:16.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:16.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:16.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:16.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:16.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:16.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:16.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:16.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:16.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:16.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:16.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:16.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:16.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:16.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:16.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:16.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:16.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:16.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:16.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:16.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:16.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:16.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:16.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:16.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:16.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:16.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:16.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:16.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:16.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:16.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:16.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:16.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 14w REG 8,1 160176336 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:16.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:16.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:16.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:16.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:16.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:16.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:16.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:16.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:16.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:16.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:16.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:16.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:16.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:16.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:16.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:16.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:16.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:16.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:16.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:16.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:16.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:16.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:16.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:16.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:16.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:16.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:16.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:16.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:16.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:16.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:16.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:16.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:16.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:16.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:16.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:16.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:16.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:16.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:16.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:16.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:16.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:16.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:16.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88221 ceph-osd root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:16.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:16.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:16.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:16.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:16.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:16.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:16.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:16.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:16.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:16.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:16.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:16.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:16.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:16.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:16.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:16.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:16.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:16.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:16.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:16.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:16.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:16.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:16.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:16.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:16.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:16.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:16.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:16.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:16.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:16.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:16.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:16.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:16.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:16.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:16.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:16.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:16.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:16.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 14w REG 8,1 160176336 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:16.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:16.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:16.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:16.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:16.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:16.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:16.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:16.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:16.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:16.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:16.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:16.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:16.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:16.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:16.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:16.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:16.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:16.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:16.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:16.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:16.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:16.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:16.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:16.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:16.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:16.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:16.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:16.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:16.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:16.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:16.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:16.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:16.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:16.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:16.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:16.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:16.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:16.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:16.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:16.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:16.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:16.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:16.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88222 safe_time root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:16.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:16.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:16.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:16.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:16.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:16.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:16.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:16.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:16.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:16.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:16.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:16.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:16.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:16.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:16.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:16.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:16.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:16.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:16.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:16.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:16.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:16.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:16.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:16.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:16.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:16.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:16.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:16.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:16.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:16.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:16.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:16.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:16.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:16.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:16.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:16.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:16.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:16.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:16.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:16.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:16.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:16.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:16.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:16.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:16.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:16.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:16.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:16.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:16.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:16.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:16.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:16.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:16.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:16.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:16.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:16.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:16.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:16.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:16.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:16.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:16.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:16.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:16.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:16.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:16.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:16.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:16.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:16.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:16.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:16.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:16.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:16.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:16.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:16.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:16.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 14w REG 8,1 160176336 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:16.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:16.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:16.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:16.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:16.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:16.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:16.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:16.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:16.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:16.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:16.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:16.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:16.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:16.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:16.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:16.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:16.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:16.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:16.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:16.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:16.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:16.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:16.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:16.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:16.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:16.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:16.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:16.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:16.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:16.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:16.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:16.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:16.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:16.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:16.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:16.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:16.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:16.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:16.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:16.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:16.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:16.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:16.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:16.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:16.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:16.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:16.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:16.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:16.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:16.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:16.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:16.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:16.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:16.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:16.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88223 safe_time root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:16.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:16.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:16.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:16.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:16.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:16.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:16.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:16.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:16.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:16.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:16.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:16.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:16.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:16.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:16.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:16.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:16.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:16.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:16.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:16.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:16.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:16.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:16.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:16.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:16.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:16.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:16.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:16.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:16.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:16.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:16.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:16.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:16.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:16.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:16.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:16.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:16.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:16.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:16.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:16.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:16.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:16.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 14w REG 8,1 160176336 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88224 safe_time root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 14w REG 8,1 160176336 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88225 safe_time root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 14w REG 8,1 160176336 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88226 fn_anonym root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 14w REG 8,1 160176619 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88396 rocksdb:l root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 14w REG 8,1 160176619 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88399 rocksdb:l root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 14w REG 8,1 160176619 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88401 rocksdb:l root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 14w REG 8,1 160176619 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 88405 rocksdb:h root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 14w REG 8,1 160176619 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 89893 bstore_ai root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 14w REG 8,1 160176619 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90664 bstore_ai root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 14w REG 8,1 160176619 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90845 ceph-osd root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 14w REG 8,1 160176619 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90846 cfin root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 14w REG 8,1 160176619 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90847 bstore_kv root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 14w REG 8,1 160176619 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90848 bstore_kv root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 14w REG 8,1 160176619 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90849 bstore_me root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 14w REG 8,1 160176619 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90851 ms_dispat root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 14w REG 8,1 160176619 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90852 ms_local root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 14w REG 8,1 160176619 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90855 safe_time root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 14w REG 8,1 160176619 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90857 safe_time root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 14w REG 8,1 160176619 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90860 ms_dispat root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 14w REG 8,1 160176619 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90862 ms_local root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 14w REG 8,1 160176619 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90865 ms_dispat root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 14w REG 8,1 160176619 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90866 ms_local root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 14w REG 8,1 160176619 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90869 ms_dispat root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 14w REG 8,1 160176619 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90871 ms_local root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 14w REG 8,1 160176619 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90873 ms_dispat root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 14w REG 8,1 160176619 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90875 ms_local root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 14w REG 8,1 160176619 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90878 ms_dispat root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 14w REG 8,1 160176619 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90879 ms_local root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 14w REG 8,1 160176902 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90882 ms_dispat root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 14w REG 8,1 160176902 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90885 ms_local root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:17.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:17.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:17.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:17.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:17.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:17.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:17.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:17.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:17.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:17.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:17.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:17.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:17.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:17.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:17.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:17.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:17.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:17.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:17.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:17.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:17.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:17.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:17.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:17.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:17.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:17.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:17.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:17.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:17.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:17.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:17.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:17.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:17.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:17.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:17.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:17.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:17.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:17.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:17.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:17.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:17.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:17.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:17.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:17.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:17.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:17.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:17.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:17.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:17.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:17.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:17.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:17.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:17.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:17.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:17.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:17.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:17.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:17.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:17.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:17.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:17.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:17.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:17.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:17.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:17.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:17.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:17.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:17.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:17.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:17.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:17.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:17.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:17.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:17.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:17.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:17.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:17.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:17.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:17.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:17.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:17.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:17.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:17.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:17.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:17.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:17.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 14w REG 8,1 160176902 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:17.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:17.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:17.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:17.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:17.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:17.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:17.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:17.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:17.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:17.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:17.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:17.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:17.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:17.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:17.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:17.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:17.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:17.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:17.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:17.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:17.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:17.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:17.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:17.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:17.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:17.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:17.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:17.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:17.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:17.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:17.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:17.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:17.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:17.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:17.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:17.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:17.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:17.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:17.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:17.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:17.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:17.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:17.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:17.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:17.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:17.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:17.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:17.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:17.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:17.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:17.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:17.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:17.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:17.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:17.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90887 fn_anonym root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:17.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:17.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:17.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:18.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:18.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:18.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:18.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:18.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:18.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:18.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 14w REG 8,1 160176902 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:18.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:18.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:18.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:18.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:18.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:18.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:18.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:18.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:18.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:18.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:18.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:18.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:18.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:18.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:18.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:18.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:18.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:18.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:18.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:18.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:18.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:18.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:18.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:18.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:18.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:18.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:18.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:18.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:18.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:18.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:18.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:18.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:18.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:18.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:18.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90889 finisher root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:18.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:18.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:18.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:18.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:18.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:18.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:18.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:18.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 14w REG 8,1 160176902 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:18.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:18.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:18.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:18.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:18.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:18.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:18.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:18.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:18.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:18.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:18.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:18.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:18.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:18.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:18.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:18.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:18.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:18.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:18.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:18.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:18.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:18.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:18.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:18.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:18.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:18.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:18.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:18.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:18.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:18.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:18.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:18.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:18.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:18.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:18.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90891 safe_time root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:18.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:18.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:18.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:18.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:18.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:18.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:18.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:18.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 14w REG 8,1 160176902 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:18.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:18.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:18.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:18.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:18.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:18.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:18.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:18.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:18.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:18.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:18.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:18.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:18.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:18.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:18.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:18.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:18.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:18.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:18.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:18.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:18.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:18.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:18.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:18.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:18.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:18.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:18.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:18.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:18.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:18.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:18.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:18.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:18.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:18.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:18.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90893 safe_time root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:18.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:18.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:18.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:18.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:18.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:18.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:18.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:18.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 14w REG 8,1 160176902 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:18.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:18.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:18.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:18.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:18.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:18.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:18.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:18.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:18.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:18.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:18.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:18.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:18.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:18.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:18.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:18.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:18.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:18.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:18.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:18.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:18.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:18.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:18.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:18.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:18.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:18.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:18.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:18.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:18.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:18.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:18.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:18.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:18.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:18.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:18.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90895 ceph-osd root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:18.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:18.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:18.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:18.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:18.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:18.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:18.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:18.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 14w REG 8,1 160176902 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:18.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:18.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:18.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:18.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:18.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:18.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:18.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:18.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:18.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:18.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:18.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:18.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:18.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:18.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:18.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:18.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:18.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:18.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:18.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:18.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:18.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:18.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:18.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:18.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:18.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:18.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:18.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:18.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:18.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:18.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:18.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:18.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:18.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:18.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:18.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90897 osd_srv_a root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:18.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:18.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:18.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:18.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:18.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:18.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:18.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:18.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 14w REG 8,1 160176902 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:18.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:18.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:18.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:18.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:18.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:18.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:18.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:18.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:18.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:18.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:18.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:18.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:18.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:18.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:18.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:18.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:18.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:18.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:18.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:18.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:18.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:18.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:18.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:18.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:18.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:18.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:18.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:18.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:18.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:18.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:18.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:18.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:18.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:18.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:18.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90899 tp_osd_tp root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:18.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:18.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:18.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:18.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:18.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:18.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:18.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:18.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 14w REG 8,1 160176902 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:18.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:18.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:18.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:18.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:18.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:18.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:18.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:18.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:18.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:18.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:18.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:18.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:18.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:18.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:18.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:18.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:18.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:18.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:18.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:18.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:18.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:18.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:18.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:18.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:18.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:18.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:18.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:18.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:18.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:18.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:18.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:18.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:18.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:18.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:18.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90901 tp_osd_tp root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:18.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:18.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:18.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:18.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:18.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:18.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:18.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:18.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 14w REG 8,1 160176902 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:18.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:18.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:18.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:18.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:18.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:18.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:18.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:18.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:18.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:18.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:18.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:18.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:18.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:18.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:18.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:18.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:18.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:18.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:18.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:18.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:18.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:18.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:18.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:18.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:18.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:18.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:18.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:18.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:18.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:18.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:18.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:18.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:18.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:18.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:18.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90904 tp_osd_tp root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:18.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:18.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:18.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:18.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:18.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:18.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:18.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:18.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 14w REG 8,1 160176902 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:18.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:18.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:18.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:18.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:18.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:18.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:18.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:18.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:18.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:18.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:18.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:18.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:18.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:18.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:18.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:18.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:18.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:18.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:18.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:18.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:18.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:18.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:18.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:18.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:18.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:18.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:18.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:18.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:18.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:18.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:18.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:18.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:18.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:18.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:18.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90906 tp_osd_tp root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:18.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:18.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:18.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:18.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:18.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:18.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:18.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:18.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 14w REG 8,1 160176902 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:18.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:18.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:18.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:18.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:18.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:18.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:18.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:18.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:18.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:18.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:18.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:18.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:18.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:18.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:18.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:18.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:18.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:18.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:18.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:18.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:18.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:18.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:18.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:18.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:18.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:18.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:18.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:18.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:18.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:18.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:18.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:18.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:18.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:18.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:18.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90907 tp_osd_tp root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:18.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:18.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:18.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:18.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:18.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:18.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:18.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:18.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 14w REG 8,1 160176902 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:18.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:18.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:18.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:18.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:18.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:18.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:18.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:18.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:18.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:18.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:18.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:18.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:18.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:18.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:18.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:18.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:18.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:18.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:18.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:18.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:18.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:18.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:18.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:18.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:18.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:18.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:18.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:18.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:18.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:18.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:18.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:18.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:18.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:18.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:18.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90908 tp_osd_tp root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:18.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:18.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:18.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:18.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:18.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:18.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:18.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:18.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 14w REG 8,1 160176902 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:18.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:18.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:18.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:18.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:18.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:18.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:18.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:18.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:18.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:18.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:18.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:18.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:18.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:18.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:18.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:18.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:18.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:18.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:18.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:18.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:18.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:18.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:18.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:18.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:18.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:18.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:18.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:18.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:18.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:18.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:18.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:18.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:18.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:18.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:18.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90911 tp_osd_tp root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:18.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:18.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:18.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:18.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:18.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:18.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:18.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:18.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 14w REG 8,1 160176902 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:18.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:18.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:18.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:18.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:18.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:18.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:18.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:18.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:18.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:18.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:18.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:18.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:18.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:18.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:18.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:18.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:18.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:18.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:18.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:18.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:18.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:18.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:18.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:18.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:18.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:18.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:18.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:18.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:18.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:18.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:18.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:18.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:18.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:18.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:18.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90914 tp_osd_tp root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:18.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:18.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:18.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:18.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:18.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:18.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:18.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:18.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 14w REG 8,1 160176902 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:18.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:18.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:18.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:18.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:18.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:18.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:18.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:18.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:18.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:18.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:18.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:18.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:18.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:18.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:18.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:18.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:18.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:18.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:18.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:18.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:18.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:18.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:18.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:18.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:18.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:18.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:18.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:18.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:18.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:18.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:18.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:18.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:18.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:18.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:18.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90916 tp_osd_tp root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:18.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:18.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:18.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:18.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:18.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:18.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:18.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:18.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 14w REG 8,1 160176902 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:18.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:18.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:18.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:18.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:18.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:18.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:18.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:18.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:18.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:18.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:18.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:18.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:18.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:18.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:18.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:18.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:18.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:18.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:18.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:18.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:18.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:18.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:18.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:18.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:18.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:18.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:18.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:18.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:18.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:18.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:18.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:18.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:18.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:18.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:18.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90919 tp_osd_tp root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:18.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:18.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:18.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:18.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:18.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:18.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:18.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:18.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 14w REG 8,1 160176902 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:18.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:18.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:18.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:18.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:18.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:18.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:18.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:18.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:18.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:18.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:18.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:18.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:18.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:18.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:18.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:18.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:18.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:18.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:18.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:18.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:18.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:18.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:18.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:18.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:18.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:18.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:18.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:18.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:18.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:18.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:18.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:18.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:18.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:18.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:18.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90921 tp_osd_tp root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:18.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:18.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:18.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:18.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:18.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:18.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:18.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:18.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 14w REG 8,1 160176902 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:18.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:18.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:18.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:18.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:18.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:18.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:18.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:18.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:18.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:18.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:18.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:18.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:18.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:18.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:18.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:18.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:18.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:18.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:18.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:18.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:18.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:18.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:18.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:18.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:18.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:18.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:18.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:18.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:18.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:18.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:18.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:18.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:18.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:18.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:18.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90923 tp_osd_tp root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:18.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:18.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:18.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:18.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:18.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:18.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:18.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:18.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 14w REG 8,1 160176902 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:18.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:18.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:18.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:18.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:18.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:18.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:18.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:18.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:18.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:18.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:18.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:18.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:18.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:18.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:18.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:18.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:18.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:18.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:18.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:18.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:18.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:18.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:18.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:18.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:18.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:18.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:18.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:18.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:18.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:18.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:18.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:18.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:18.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:18.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:18.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90925 tp_osd_tp root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:18.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:18.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:18.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:18.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:18.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:18.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:18.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:18.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 14w REG 8,1 160176902 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:18.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:18.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:18.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:18.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:18.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:18.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:18.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:18.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:18.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:18.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:18.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:18.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:18.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:18.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:18.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:18.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:18.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:18.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:18.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:18.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:18.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:18.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:18.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:18.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:18.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:18.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:18.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:18.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:18.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:18.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:18.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:18.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:18.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:18.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:18.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90926 tp_osd_tp root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:18.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:18.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:18.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:18.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:18.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:18.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:18.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:18.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 14w REG 8,1 160176902 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:18.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:18.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:18.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:18.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:18.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:18.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:18.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:18.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:18.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:18.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:18.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:18.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:18.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:18.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:18.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:18.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:18.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:18.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:18.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:18.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:18.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:18.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:18.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:18.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:18.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:18.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:18.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:18.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:18.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:18.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:18.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:18.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:18.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:18.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:18.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90928 tp_osd_tp root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:18.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:18.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:18.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:18.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:18.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:18.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:18.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:18.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 14w REG 8,1 160176902 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:18.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:18.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:18.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:18.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:18.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:18.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:18.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:18.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:18.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:18.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:18.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:18.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:18.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:18.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:18.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:18.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:18.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:18.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:18.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:18.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:18.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:18.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:18.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:18.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:18.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:18.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:18.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:18.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:18.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:18.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:18.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:18.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:18.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:18.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:18.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90930 tp_osd_tp root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:18.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root DEL REG 0,17 183514 /[aio] 2023-11-09T03:56:18.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root DEL REG 0,17 183511 /[aio] 2023-11-09T03:56:18.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 1w FIFO 0,13 0t0 182401 pipe 2023-11-09T03:56:18.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 2w FIFO 0,13 0t0 182402 pipe 2023-11-09T03:56:18.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 3r FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 4w FIFO 0,13 0t0 183395 pipe 2023-11-09T03:56:18.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,78,79,80,81,82,83,88,91,94,97,100] 2023-11-09T03:56:18.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 6r FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 7w FIFO 0,13 0t0 183397 pipe 2023-11-09T03:56:18.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,69,70,76,84,85,86,89,92,95,101] 2023-11-09T03:56:18.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 9r FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 10w FIFO 0,13 0t0 183398 pipe 2023-11-09T03:56:18.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,71,72,73,74,75,77,87,90,93,96,102] 2023-11-09T03:56:18.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 12r FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 13w FIFO 0,13 0t0 183399 pipe 2023-11-09T03:56:18.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 14w REG 8,1 160176902 265315 /var/log/ceph/ceph-osd.0.log 2023-11-09T03:56:18.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 15uW REG 0,24 6 1737 /run/ceph/ceph-osd.0.pid 2023-11-09T03:56:18.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 16r FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 17w FIFO 0,13 0t0 181508 pipe 2023-11-09T03:56:18.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 18u unix 0xffff959a600061c0 0t0 181509 /var/run/ceph/ceph-osd.0.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 19u IPv4 180627 0t0 TCP *:6803 (LISTEN) 2023-11-09T03:56:18.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 20u IPv4 180634 0t0 TCP *:6806 (LISTEN) 2023-11-09T03:56:18.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 21u IPv4 180644 0t0 TCP *:6809 (LISTEN) 2023-11-09T03:56:18.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 22u IPv4 180656 0t0 TCP *:6811 (LISTEN) 2023-11-09T03:56:18.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 23u IPv4 180669 0t0 TCP *:6812 (LISTEN) 2023-11-09T03:56:18.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 24u IPv4 180683 0t0 TCP *:6813 (LISTEN) 2023-11-09T03:56:18.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 25u IPv4 180698 0t0 TCP *:6814 (LISTEN) 2023-11-09T03:56:18.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 26u IPv4 180714 0t0 TCP *:6815 (LISTEN) 2023-11-09T03:56:18.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 27r FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 28w FIFO 0,13 0t0 181538 pipe 2023-11-09T03:56:18.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 29r FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 30w FIFO 0,13 0t0 181539 pipe 2023-11-09T03:56:18.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 31u sock 0,8 0t0 181592 protocol: UDPv6 2023-11-09T03:56:18.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 32r DIR 253,0 357 64 /var/lib/ceph/osd/ceph-0 2023-11-09T03:56:18.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 33uW REG 253,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2023-11-09T03:56:18.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 34u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 35u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 36u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 37u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 38u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 39u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 40u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 41u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 42u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 43u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 44u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 45u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 46u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 47u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 48u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 49u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 50u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 51u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 52u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 53u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 54u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 55u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 56u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 57u REG 253,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2023-11-09T03:56:18.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 58u IPv4 185570 0t0 TCP smithi076.front.sepia.ceph.com:60330->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 59r FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 60w FIFO 0,13 0t0 181629 pipe 2023-11-09T03:56:18.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 61u IPv4 182591 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35192 (ESTABLISHED) 2023-11-09T03:56:18.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 62r FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 63w FIFO 0,13 0t0 181630 pipe 2023-11-09T03:56:18.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 64u IPv4 182594 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42348 (ESTABLISHED) 2023-11-09T03:56:18.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 65u IPv4 179836 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48456 (ESTABLISHED) 2023-11-09T03:56:18.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 66u IPv4 179839 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38140 (ESTABLISHED) 2023-11-09T03:56:18.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 67u IPv4 179842 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48460 (ESTABLISHED) 2023-11-09T03:56:18.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 68u IPv4 185580 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38142 (ESTABLISHED) 2023-11-09T03:56:18.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 69u IPv4 178366 0t0 TCP smithi076.front.sepia.ceph.com:44526->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 70u IPv4 178367 0t0 TCP smithi076.front.sepia.ceph.com:53964->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 71u IPv4 185583 0t0 TCP smithi076.front.sepia.ceph.com:35828->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 72u IPv4 185584 0t0 TCP smithi076.front.sepia.ceph.com:42966->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 73u IPv4 185585 0t0 TCP smithi076.front.sepia.ceph.com:41854->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 74u IPv4 185586 0t0 TCP smithi076.front.sepia.ceph.com:59310->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 75u IPv4 185587 0t0 TCP smithi076.front.sepia.ceph.com:44700->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 76u IPv4 185588 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35206 (ESTABLISHED) 2023-11-09T03:56:18.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 77u IPv4 185591 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42356 (ESTABLISHED) 2023-11-09T03:56:18.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 78u IPv4 185592 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi076.front.sepia.ceph.com:42360 (ESTABLISHED) 2023-11-09T03:56:18.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 79u IPv4 185595 0t0 TCP smithi076.front.sepia.ceph.com:51586->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 80u IPv4 185596 0t0 TCP smithi076.front.sepia.ceph.com:39630->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 81u IPv4 185597 0t0 TCP smithi076.front.sepia.ceph.com:32872->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 82u IPv4 185598 0t0 TCP smithi076.front.sepia.ceph.com:51912->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 83u IPv4 185599 0t0 TCP smithi076.front.sepia.ceph.com:52076->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 84u IPv4 183530 0t0 TCP smithi076.front.sepia.ceph.com:53788->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 85u IPv4 183531 0t0 TCP smithi076.front.sepia.ceph.com:39434->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 86u IPv4 184463 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi076.front.sepia.ceph.com:35220 (ESTABLISHED) 2023-11-09T03:56:18.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 87u IPv4 181644 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48472 (ESTABLISHED) 2023-11-09T03:56:18.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 88u IPv4 181647 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38156 (ESTABLISHED) 2023-11-09T03:56:18.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 89u IPv4 181656 0t0 TCP smithi076.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:48488 (ESTABLISHED) 2023-11-09T03:56:18.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 90u IPv4 181659 0t0 TCP smithi076.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:38166 (ESTABLISHED) 2023-11-09T03:56:18.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 91u IPv4 184473 0t0 TCP smithi076.front.sepia.ceph.com:pmcdproxy->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 92u IPv4 180763 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54314 (ESTABLISHED) 2023-11-09T03:56:18.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 93u IPv4 182640 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi113.front.sepia.ceph.com:57114 (ESTABLISHED) 2023-11-09T03:56:18.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 94u IPv4 178397 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54328 (ESTABLISHED) 2023-11-09T03:56:18.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 95u IPv4 181669 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44906 (ESTABLISHED) 2023-11-09T03:56:18.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 96u IPv4 182630 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi076.front.sepia.ceph.com:54334 (ESTABLISHED) 2023-11-09T03:56:18.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 97u IPv4 253438 0t0 TCP smithi076.front.sepia.ceph.com:6803->smithi076.front.sepia.ceph.com:33348 (ESTABLISHED) 2023-11-09T03:56:18.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 100u IPv4 182634 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44920 (ESTABLISHED) 2023-11-09T03:56:18.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 101u IPv4 182635 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44922 (ESTABLISHED) 2023-11-09T03:56:18.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88003 90932 osd_srv_h root 102u IPv4 182636 0t0 TCP smithi076.front.sepia.ceph.com:6809->smithi113.front.sepia.ceph.com:44936 (ESTABLISHED) 2023-11-09T03:56:18.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:18.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:18.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:18.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:18.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:18.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:18.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:18.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:18.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:18.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:18.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:18.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:18.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:18.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:18.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:18.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 14w REG 8,1 133446126 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:18.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:18.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:18.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:18.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:18.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:18.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:18.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:18.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:18.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:18.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:18.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:18.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:18.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:18.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:18.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:18.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:18.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:18.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:18.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:18.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:18.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:18.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:18.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:18.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:18.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:18.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:18.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:18.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:18.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:18.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:18.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:18.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:18.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:18.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:18.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:18.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:18.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:18.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:18.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:18.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:18.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:18.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:18.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:18.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:18.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:18.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:18.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:18.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:18.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:18.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:18.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:18.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:18.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:18.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:18.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:18.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:18.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:18.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:18.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:18.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:18.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 14w REG 8,1 133445277 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:18.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:18.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:18.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:18.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:18.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:18.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:18.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:18.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:18.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:18.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:18.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:18.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:18.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:18.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:18.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:18.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:18.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:18.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:18.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:18.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:18.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:18.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:18.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:18.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:18.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:18.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:18.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:18.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:18.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:18.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:18.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:18.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:18.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:18.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:18.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:18.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:18.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:18.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:18.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:18.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:18.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:18.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:18.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:18.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:18.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88019 log root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:18.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:18.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:18.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:18.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:18.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:18.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:18.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:18.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:18.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:18.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:18.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:18.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:18.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:18.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:18.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:18.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 14w REG 8,1 133445277 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:18.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:18.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:18.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:18.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:18.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:18.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:18.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:18.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:18.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:18.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:18.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:18.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:18.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:18.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:18.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:18.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:18.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:18.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:18.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:18.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:18.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:18.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:18.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:18.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:18.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:18.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:18.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:18.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:18.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:18.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:18.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:18.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:18.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:18.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:18.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:18.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:18.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:18.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:18.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:18.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:18.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:18.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:18.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:18.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:18.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88029 msgr-work root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:18.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:18.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:18.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:18.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:18.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:18.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:18.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:18.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:18.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:18.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:18.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:18.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:18.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:18.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:18.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:18.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 14w REG 8,1 133445277 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:18.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:18.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:18.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:18.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:18.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:18.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:18.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:18.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:18.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:18.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:18.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:18.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:18.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:18.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:18.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:18.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:18.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:18.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:18.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:18.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:18.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:18.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:18.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:18.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:18.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:18.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:18.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:18.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:18.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:18.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:18.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:18.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:18.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:18.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:18.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:18.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:18.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:18.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:18.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:18.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:18.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:18.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:18.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:18.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:18.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88031 msgr-work root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:18.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:18.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:18.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:18.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:18.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:18.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:18.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:18.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:18.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:18.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:18.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:18.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:18.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:18.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:18.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:18.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 14w REG 8,1 133445277 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:18.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:18.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:18.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:18.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:18.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:18.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:18.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:18.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:18.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:18.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:18.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:18.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:18.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:18.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:18.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:18.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:18.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:18.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:18.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:18.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:18.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:18.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:18.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:18.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:18.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:18.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:18.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:18.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:18.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:18.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:18.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:18.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:18.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:18.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:18.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:18.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:18.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:18.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:18.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:18.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:18.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:18.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:18.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:18.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:18.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88032 msgr-work root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:18.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:18.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:18.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:18.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:18.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:18.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:18.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:18.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:18.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:18.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:18.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:18.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:18.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:18.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:18.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:18.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 14w REG 8,1 133445277 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:18.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:18.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:18.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:18.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:18.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:18.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:18.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:18.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:18.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:18.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:18.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:18.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:18.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:18.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:18.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:18.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:18.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:18.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:18.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:18.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:18.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:18.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:18.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:18.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:18.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:18.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:18.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:18.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:18.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:18.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:18.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:18.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:18.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:18.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:18.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:18.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:18.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:18.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:18.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:18.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:18.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:18.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:18.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:18.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:18.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88049 service root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:18.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:18.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:18.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:18.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:18.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:18.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:18.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:18.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:18.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:18.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:18.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:18.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:18.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:18.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:18.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:18.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 14w REG 8,1 133445277 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:18.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:18.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:18.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:18.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:18.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:18.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:18.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:18.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:18.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:18.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:18.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:18.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:18.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:18.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:18.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:18.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:18.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:18.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:18.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:18.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:18.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:18.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:18.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:18.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:18.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:18.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:18.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:18.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:18.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:18.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:18.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:18.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:18.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:18.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:18.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:18.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:18.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:18.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:18.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:18.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:18.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:18.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:18.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:18.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:18.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88050 admin_soc root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:18.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:18.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:18.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:18.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:18.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:18.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:18.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:18.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:18.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:18.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:18.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:18.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:18.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:18.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:18.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:18.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 14w REG 8,1 133445277 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:18.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:18.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:18.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:18.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:18.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:18.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:18.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:18.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:18.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:18.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:18.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:18.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:18.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:18.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:18.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:18.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:18.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:18.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:18.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:18.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:18.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:18.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:18.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:18.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:18.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:18.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:18.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:18.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:18.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:18.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:18.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:18.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:18.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:18.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:18.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:18.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:18.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:18.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:18.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:18.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:18.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:18.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:18.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:18.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:18.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88088 signal_ha root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:18.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:18.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:18.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:18.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:18.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:18.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:18.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:18.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:18.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:18.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:18.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:18.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:18.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:18.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:18.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:18.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:18.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:18.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:18.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:18.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:18.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:18.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:18.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:18.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:18.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:18.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:18.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:18.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:18.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:18.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:18.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:18.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:18.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:18.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:18.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:18.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:18.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:18.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:18.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:18.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:18.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:18.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:18.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:18.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:18.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:18.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:18.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:18.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:18.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:18.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:18.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:18.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:18.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:18.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:18.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:18.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:18.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:18.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:18.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:18.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:18.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:18.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:18.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:18.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 14w REG 8,1 133445277 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:18.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:18.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:18.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:18.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:18.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:18.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:18.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:18.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:18.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:18.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:18.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:18.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:18.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:18.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:18.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:18.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:18.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:18.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:18.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:18.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:18.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:18.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:18.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:18.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:18.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:18.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:18.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:18.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:18.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:18.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:18.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:18.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:18.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:18.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:18.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:18.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:18.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:18.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:18.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:18.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:18.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:18.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:18.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:18.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:18.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:18.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:18.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:18.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:18.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:18.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:18.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:18.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:18.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:18.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:18.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:18.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:18.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:18.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:18.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:18.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88089 io_contex root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:18.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:18.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:18.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:18.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:18.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:18.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:18.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:18.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:18.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:18.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:18.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:18.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:18.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:18.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:18.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:18.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:18.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:18.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:18.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:18.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:18.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:18.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:18.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:18.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:18.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:18.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:18.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:18.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:18.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:18.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:18.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:18.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 14w REG 8,1 133445277 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88090 io_contex root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 14w REG 8,1 133445277 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88102 OpHistory root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 14w REG 8,1 133445277 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88103 ceph_time root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 14w REG 8,1 133445277 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88146 ceph-osd root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 14w REG 8,1 133445277 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88160 ceph-osd root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 14w REG 8,1 133445560 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88177 ceph-osd root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 14w REG 8,1 133445560 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88185 ceph-osd root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 14w REG 8,1 133445560 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88193 ceph-osd root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 14w REG 8,1 133445560 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88201 ceph-osd root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 14w REG 8,1 133445560 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88209 ceph-osd root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 14w REG 8,1 133445560 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88790 ceph-osd root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 14w REG 8,1 133445560 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88791 ceph-osd root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 14w REG 8,1 133445560 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88792 safe_time root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 14w REG 8,1 133445560 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88793 safe_time root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 14w REG 8,1 133445560 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88794 safe_time root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 14w REG 8,1 133445560 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88795 safe_time root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 14w REG 8,1 133445560 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88796 fn_anonym root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 14w REG 8,1 133445560 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88799 rocksdb:l root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 14w REG 8,1 133445560 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88800 rocksdb:l root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 14w REG 8,1 133445560 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88801 rocksdb:l root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 14w REG 8,1 133445560 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 88802 rocksdb:h root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 14w REG 8,1 133445560 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 90257 bstore_ai root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 14w REG 8,1 133445560 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91070 bstore_ai root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 14w REG 8,1 133445560 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91256 ceph-osd root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 14w REG 8,1 133445560 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91257 cfin root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 14w REG 8,1 133445560 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91258 bstore_kv root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 14w REG 8,1 133445560 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91259 bstore_kv root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 14w REG 8,1 133445560 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91260 bstore_me root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 14w REG 8,1 133445560 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91300 ms_dispat root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 14w REG 8,1 133445560 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:19.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:19.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:19.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:19.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:19.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:19.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:19.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:19.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:19.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:19.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:19.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:19.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:19.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:19.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:19.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:19.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:19.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:19.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:19.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:19.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:19.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:19.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:19.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:19.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:19.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:19.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:19.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:19.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:19.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:19.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:19.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:19.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:19.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:19.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:19.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:19.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:19.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:19.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:19.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:19.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:19.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:19.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:19.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:19.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:19.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:19.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:19.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91301 ms_local root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:19.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:19.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:19.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:19.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:19.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:19.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:19.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:19.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:19.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:19.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:19.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:19.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:19.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:19.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:19.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:19.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:19.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:19.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:19.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:19.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:19.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:19.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:19.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:19.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:19.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:19.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:19.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:19.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:19.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:19.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:19.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:19.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:19.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:19.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:19.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:19.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:19.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:19.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:19.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:19.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:19.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:19.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:19.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:19.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:19.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:19.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:19.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:19.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:19.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:19.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:19.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:19.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:19.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:19.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:19.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:19.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:19.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:19.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:19.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:19.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:19.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:19.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:19.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:19.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:19.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:19.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:19.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:19.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:19.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:19.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:19.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:19.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:19.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:19.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:19.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:19.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:19.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:19.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:19.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:19.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:19.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:19.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:19.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:19.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:19.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:19.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:19.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:19.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:19.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 14w REG 8,1 133445843 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:19.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:19.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:19.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:19.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:19.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:19.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:19.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:19.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91302 safe_time root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 14w REG 8,1 133445843 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91303 safe_time root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 14w REG 8,1 133445843 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91304 ms_dispat root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 14w REG 8,1 133445843 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91305 ms_local root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 14w REG 8,1 133445843 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91306 ms_dispat root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 14w REG 8,1 133445843 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91307 ms_local root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 14w REG 8,1 133445843 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91308 ms_dispat root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 14w REG 8,1 133445843 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91309 ms_local root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 14w REG 8,1 133445843 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91310 ms_dispat root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 14w REG 8,1 133445843 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91311 ms_local root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 14w REG 8,1 133445843 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91312 ms_dispat root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 14w REG 8,1 133445843 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91313 ms_local root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 14w REG 8,1 133445843 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91314 ms_dispat root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 14w REG 8,1 133445843 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91315 ms_local root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 14w REG 8,1 133445843 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91316 fn_anonym root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 14w REG 8,1 133445843 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91317 finisher root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 14w REG 8,1 133445843 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91318 safe_time root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 14w REG 8,1 133445843 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91319 safe_time root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 14w REG 8,1 133445843 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91320 ceph-osd root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 14w REG 8,1 133445843 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91321 osd_srv_a root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 14w REG 8,1 133445843 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91322 tp_osd_tp root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 14w REG 8,1 133445843 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91323 tp_osd_tp root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 14w REG 8,1 133445843 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91324 tp_osd_tp root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 14w REG 8,1 133445843 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91325 tp_osd_tp root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 14w REG 8,1 133445843 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91326 tp_osd_tp root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 14w REG 8,1 133445843 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91327 tp_osd_tp root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 14w REG 8,1 133446126 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91328 tp_osd_tp root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 14w REG 8,1 133446126 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91329 tp_osd_tp root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 14w REG 8,1 133446126 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91330 tp_osd_tp root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 14w REG 8,1 133446126 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91331 tp_osd_tp root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 14w REG 8,1 133446126 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:20.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:20.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:20.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:20.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:20.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:20.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:20.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:20.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:20.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:20.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:20.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:20.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:20.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:20.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:20.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:20.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:20.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:20.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:20.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:20.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:20.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:20.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:20.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:20.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:20.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:20.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:20.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:20.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:20.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:20.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:20.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:20.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:20.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:20.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:20.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:20.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:20.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:20.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:20.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:20.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:20.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:20.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:20.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:20.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:20.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:20.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:20.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91332 tp_osd_tp root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:20.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:20.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:20.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:20.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:20.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:20.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:20.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:20.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:20.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:20.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:20.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:20.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:20.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:20.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:20.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:20.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:20.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:20.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:20.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:20.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:20.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:20.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:20.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:20.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:20.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:20.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:20.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:20.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:20.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:20.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:20.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:20.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:20.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:20.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:20.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:20.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:20.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:20.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:20.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:20.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:20.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:20.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:20.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:20.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:20.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:20.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:20.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:20.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:20.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:20.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:20.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:20.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:20.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:20.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:20.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:20.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:20.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:20.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:20.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:20.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:20.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:20.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:20.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:20.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:20.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:20.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:20.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:20.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:20.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:20.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:20.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:20.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:20.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:20.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:20.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:20.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:20.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:20.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:20.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:20.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:20.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:20.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:20.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:20.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:20.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:20.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:20.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:20.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:20.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 14w REG 8,1 133446126 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:20.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:20.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:20.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:20.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:20.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:20.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:20.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:20.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:21.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:21.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:21.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:21.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:21.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:21.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:21.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:21.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:21.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:21.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:21.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:21.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:21.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:21.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:21.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:21.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:21.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:21.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:21.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:21.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:21.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:21.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:21.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:21.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:21.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:21.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:21.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:21.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:21.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:21.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:21.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:21.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:21.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:21.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:21.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:21.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91333 tp_osd_tp root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:21.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:21.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:21.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:21.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:21.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:21.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:21.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:21.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:21.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:21.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:21.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:21.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:21.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:21.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:21.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:21.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 14w REG 8,1 133446126 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:21.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:21.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:21.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:21.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:21.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:21.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:21.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:21.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:21.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:21.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:21.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:21.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:21.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:21.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:21.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:21.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:21.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:21.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:21.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:21.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:21.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:21.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:21.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:21.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:21.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:21.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:21.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:21.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:21.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:21.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:21.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:21.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:21.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:21.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:21.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:21.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:21.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:21.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:21.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:21.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:21.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:21.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:21.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:21.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91334 tp_osd_tp root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:21.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:21.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:21.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:21.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:21.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:21.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:21.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:21.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:21.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:21.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:21.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:21.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:21.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:21.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:21.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:21.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 14w REG 8,1 133446126 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:21.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:21.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:21.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:21.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:21.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:21.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:21.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:21.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:21.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:21.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:21.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:21.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:21.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:21.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:21.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:21.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:21.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:21.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:21.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:21.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:21.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:21.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:21.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:21.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:21.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:21.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:21.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:21.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:21.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:21.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:21.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:21.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:21.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:21.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:21.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:21.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:21.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:21.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:21.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:21.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:21.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:21.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:21.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:21.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91335 tp_osd_tp root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:21.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:21.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:21.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:21.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:21.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:21.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:21.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:21.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:21.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:21.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:21.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:21.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:21.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:21.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:21.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:21.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 14w REG 8,1 133446126 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:21.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:21.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:21.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:21.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:21.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:21.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:21.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:21.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:21.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:21.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:21.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:21.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:21.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:21.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:21.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:21.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:21.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:21.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:21.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:21.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:21.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:21.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:21.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:21.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:21.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:21.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:21.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:21.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:21.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:21.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:21.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:21.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:21.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:21.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:21.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:21.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:21.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:21.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:21.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:21.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:21.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:21.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:21.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:21.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91336 tp_osd_tp root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:21.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:21.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:21.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:21.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:21.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:21.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:21.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:21.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:21.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:21.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:21.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:21.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:21.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:21.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:21.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:21.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 14w REG 8,1 133446126 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:21.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:21.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:21.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:21.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:21.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:21.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:21.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:21.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:21.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:21.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:21.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:21.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:21.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:21.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:21.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:21.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:21.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:21.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:21.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:21.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:21.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:21.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:21.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:21.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:21.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:21.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:21.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:21.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:21.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:21.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:21.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:21.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:21.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:21.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:21.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:21.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:21.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:21.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:21.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:21.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:21.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:21.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:21.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:21.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91337 tp_osd_tp root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:21.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root DEL REG 0,17 178348 /[aio] 2023-11-09T03:56:21.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root DEL REG 0,17 178345 /[aio] 2023-11-09T03:56:21.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 1w FIFO 0,13 0t0 182409 pipe 2023-11-09T03:56:21.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 2w FIFO 0,13 0t0 182410 pipe 2023-11-09T03:56:21.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 3r FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:21.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 4w FIFO 0,13 0t0 180615 pipe 2023-11-09T03:56:21.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,60,65,68,71,74,79,82,85,88,91,92,94,103] 2023-11-09T03:56:21.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 6r FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:21.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 7w FIFO 0,13 0t0 180617 pipe 2023-11-09T03:56:21.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 8u a_inode 0,14 0 2077 [eventpoll:9,63,67,70,75,76,77,80,83,86,89,95,101,104] 2023-11-09T03:56:21.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 9r FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:21.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 10w FIFO 0,13 0t0 180618 pipe 2023-11-09T03:56:21.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 11u a_inode 0,14 0 2077 [eventpoll:12,64,66,69,72,73,78,81,84,87,90,93,96,102] 2023-11-09T03:56:21.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 12r FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:21.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 13w FIFO 0,13 0t0 180619 pipe 2023-11-09T03:56:21.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 14w REG 8,1 133446126 265317 /var/log/ceph/ceph-osd.2.log 2023-11-09T03:56:21.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 15uW REG 0,24 6 1743 /run/ceph/ceph-osd.2.pid 2023-11-09T03:56:21.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 16r FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:21.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 17w FIFO 0,13 0t0 183420 pipe 2023-11-09T03:56:21.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 18u unix 0xffff959a60117700 0t0 183421 /var/run/ceph/ceph-osd.2.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 19u IPv4 178081 0t0 TCP *:6822 (LISTEN) 2023-11-09T03:56:21.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 20u IPv4 178106 0t0 TCP *:6824 (LISTEN) 2023-11-09T03:56:21.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 21u IPv4 178133 0t0 TCP *:6826 (LISTEN) 2023-11-09T03:56:21.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 22u IPv4 178161 0t0 TCP *:6827 (LISTEN) 2023-11-09T03:56:21.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 23u IPv4 185358 0t0 TCP *:6828 (LISTEN) 2023-11-09T03:56:21.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 24u IPv4 185388 0t0 TCP *:6829 (LISTEN) 2023-11-09T03:56:21.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 25u IPv4 185419 0t0 TCP *:6830 (LISTEN) 2023-11-09T03:56:21.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 26u IPv4 185451 0t0 TCP *:ambit-lm (LISTEN) 2023-11-09T03:56:21.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 27r FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:21.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 28w FIFO 0,13 0t0 183476 pipe 2023-11-09T03:56:21.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 29r FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:21.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 30w FIFO 0,13 0t0 183477 pipe 2023-11-09T03:56:21.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 31u sock 0,8 0t0 185557 protocol: UDPv6 2023-11-09T03:56:21.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 32r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-2 2023-11-09T03:56:21.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 33uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2023-11-09T03:56:21.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 55u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 56u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 57u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2023-11-09T03:56:21.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 58r FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:21.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 59w FIFO 0,13 0t0 184423 pipe 2023-11-09T03:56:21.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 60u IPv4 182570 0t0 TCP smithi076.front.sepia.ceph.com:60350->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 61r FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:21.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 62w FIFO 0,13 0t0 184424 pipe 2023-11-09T03:56:21.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 63u IPv4 180726 0t0 TCP smithi076.front.sepia.ceph.com:35824->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:21.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 64u IPv4 185575 0t0 TCP smithi076.front.sepia.ceph.com:51582->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 65u IPv4 178350 0t0 TCP smithi076.front.sepia.ceph.com:44676->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 66u IPv4 185576 0t0 TCP smithi076.front.sepia.ceph.com:59296->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 67u IPv4 180727 0t0 TCP smithi076.front.sepia.ceph.com:53758->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 68u IPv4 178351 0t0 TCP smithi076.front.sepia.ceph.com:39396->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 69u IPv4 185577 0t0 TCP smithi076.front.sepia.ceph.com:51894->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 70u IPv4 180728 0t0 TCP smithi076.front.sepia.ceph.com:52062->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 71u IPv4 179834 0t0 TCP smithi076.front.sepia.ceph.com:39610->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 72u IPv4 185578 0t0 TCP smithi076.front.sepia.ceph.com:42948->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 73u IPv4 185579 0t0 TCP smithi076.front.sepia.ceph.com:35192->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 74u IPv4 179835 0t0 TCP smithi076.front.sepia.ceph.com:42348->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 75u IPv4 180729 0t0 TCP smithi076.front.sepia.ceph.com:44520->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 76u IPv4 180730 0t0 TCP smithi076.front.sepia.ceph.com:53950->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 77u IPv4 184452 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60000 (ESTABLISHED) 2023-11-09T03:56:21.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 78u IPv4 184455 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37126 (ESTABLISHED) 2023-11-09T03:56:21.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 79u IPv4 178352 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37132 (ESTABLISHED) 2023-11-09T03:56:21.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 80u IPv4 178355 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60004 (ESTABLISHED) 2023-11-09T03:56:21.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 81u IPv4 182597 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41846 (ESTABLISHED) 2023-11-09T03:56:21.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 82u IPv4 182600 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41854 (ESTABLISHED) 2023-11-09T03:56:21.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 83u IPv4 182601 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi076.front.sepia.ceph.com:41868 (ESTABLISHED) 2023-11-09T03:56:21.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 84u IPv4 182604 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32860 (ESTABLISHED) 2023-11-09T03:56:21.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 85u IPv4 180732 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32848 (ESTABLISHED) 2023-11-09T03:56:21.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 86u IPv4 180735 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi076.front.sepia.ceph.com:32872 (ESTABLISHED) 2023-11-09T03:56:21.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 87u IPv4 181650 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37146 (ESTABLISHED) 2023-11-09T03:56:21.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 88u IPv4 181653 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60008 (ESTABLISHED) 2023-11-09T03:56:21.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 89u IPv4 181662 0t0 TCP smithi076.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37150 (ESTABLISHED) 2023-11-09T03:56:21.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 90u IPv4 182607 0t0 TCP smithi076.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60020 (ESTABLISHED) 2023-11-09T03:56:21.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 91u IPv4 182624 0t0 TCP smithi076.front.sepia.ceph.com:44346->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 92u IPv4 182611 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi076.front.sepia.ceph.com:45842 (ESTABLISHED) 2023-11-09T03:56:21.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 93u IPv4 180779 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:57160 (ESTABLISHED) 2023-11-09T03:56:21.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 94u IPv4 250775 0t0 TCP smithi076.front.sepia.ceph.com:6822->smithi076.front.sepia.ceph.com:48810 (ESTABLISHED) 2023-11-09T03:56:21.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 95u IPv4 179850 0t0 TCP smithi076.front.sepia.ceph.com:54314->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 96u IPv4 185600 0t0 TCP smithi076.front.sepia.ceph.com:59766->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 101u IPv4 182618 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45154 (ESTABLISHED) 2023-11-09T03:56:21.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 102u IPv4 182619 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45166 (ESTABLISHED) 2023-11-09T03:56:21.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 103u IPv4 182620 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45158 (ESTABLISHED) 2023-11-09T03:56:21.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88004 91338 osd_srv_h root 104u IPv4 182621 0t0 TCP smithi076.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:45180 (ESTABLISHED) 2023-11-09T03:56:21.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:21.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:21.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:21.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:21.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:21.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:21.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:21.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 14w REG 8,1 142226174 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:21.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:21.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:21.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:21.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:21.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:21.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:21.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:21.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:21.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:21.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:21.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:21.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:21.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:21.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:21.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:21.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:21.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:21.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:21.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:21.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:21.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:21.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:21.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:21.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:21.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:21.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:21.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:21.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:21.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:21.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:21.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:21.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:21.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:21.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:21.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:21.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:21.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:21.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 14w REG 8,1 142225325 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:21.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:21.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:21.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:21.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:21.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:21.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:21.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:21.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:21.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:21.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:21.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:21.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:21.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:21.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:21.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:21.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:21.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:21.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:21.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:21.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:21.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:21.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:21.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:21.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:21.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:21.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:21.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:21.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:21.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:21.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:21.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88023 log root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:21.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:21.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:21.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:21.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:21.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:21.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:21.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 14w REG 8,1 142225325 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:21.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:21.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:21.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:21.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:21.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:21.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:21.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:21.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:21.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:21.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:21.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:21.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:21.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:21.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:21.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:21.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:21.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:21.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:21.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:21.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:21.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:21.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:21.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:21.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:21.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:21.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:21.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:21.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:21.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:21.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:21.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88035 msgr-work root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:21.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:21.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:21.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:21.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:21.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:21.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:21.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 14w REG 8,1 142225325 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:21.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:21.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:21.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:21.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:21.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:21.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:21.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:21.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:21.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:21.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:21.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:21.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:21.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:21.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:21.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:21.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:21.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:21.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:21.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:21.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:21.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:21.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:21.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:21.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:21.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:21.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:21.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:21.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:21.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:21.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:21.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88037 msgr-work root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:21.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:21.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:21.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:21.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:21.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:21.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:21.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 14w REG 8,1 142225325 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:21.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:21.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:21.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:21.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:21.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:21.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:21.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:21.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:21.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:21.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:21.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:21.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:21.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:21.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:21.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:21.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:21.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:21.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:21.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:21.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:21.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:21.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:21.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:21.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:21.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:21.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:21.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:21.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:21.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:21.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:21.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88038 msgr-work root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:21.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:21.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:21.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:21.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:21.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:21.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:21.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 14w REG 8,1 142225325 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:21.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:21.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:21.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:21.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:21.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:21.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:21.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:21.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:21.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:21.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:21.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:21.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:21.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:21.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:21.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:21.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:21.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:21.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:21.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:21.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:21.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:21.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:21.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:21.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:21.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:21.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:21.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:21.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:21.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:21.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:21.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88046 service root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:21.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:21.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:21.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:21.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:21.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:21.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:21.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 14w REG 8,1 142225325 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:21.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:21.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:21.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:21.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:21.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:21.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:21.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:21.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:21.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:21.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:21.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:21.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:21.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:21.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:21.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:21.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:21.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:21.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:21.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:21.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:21.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:21.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:21.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:21.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:21.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:21.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:21.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:21.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:21.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:21.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:21.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88047 admin_soc root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:21.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:21.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:21.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:21.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:21.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:21.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:21.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 14w REG 8,1 142225325 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:21.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:21.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:21.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:21.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:21.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:21.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:21.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:21.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:21.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:21.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:21.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:21.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:21.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:21.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:21.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:21.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:21.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:21.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:21.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:21.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:21.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:21.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:21.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:21.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:21.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:21.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:21.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:21.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:21.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:21.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:21.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88085 signal_ha root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:21.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:21.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:21.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:21.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:21.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:21.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:21.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 14w REG 8,1 142225325 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:21.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:21.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:21.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:21.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:21.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:21.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:21.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:21.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:21.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:21.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:21.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:21.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:21.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:21.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:21.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:21.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:21.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:21.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:21.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:21.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:21.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:21.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:21.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:21.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:21.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:21.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:21.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:21.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:21.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:21.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:21.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88086 io_contex root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:21.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:21.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:21.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:21.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:21.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:21.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:21.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 14w REG 8,1 142225325 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:21.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:21.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:21.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:21.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:21.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:21.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:21.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:21.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:21.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:21.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:21.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:21.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:21.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:21.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:21.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:21.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:21.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:21.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:21.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:21.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:21.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:21.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:21.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:21.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:21.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:21.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:21.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:21.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:21.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:21.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:21.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88087 io_contex root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:21.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:21.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:21.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:21.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:21.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:21.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:21.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 14w REG 8,1 142225325 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:21.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:21.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:21.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:21.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:21.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:21.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:21.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:21.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:21.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:21.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:21.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:21.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:21.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:21.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:21.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:21.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:21.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:21.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:21.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:21.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:21.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:21.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:21.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:21.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:21.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:21.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:21.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:21.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:21.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:21.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:21.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88099 OpHistory root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:21.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:21.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:21.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:21.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:21.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:21.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:21.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 14w REG 8,1 142225325 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:21.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:21.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:21.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:21.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:21.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:21.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:21.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:21.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:21.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:21.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:21.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:21.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:21.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:21.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:21.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:21.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:21.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:21.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:21.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:21.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:21.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:21.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:21.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:21.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:21.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:21.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:21.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:21.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:21.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:21.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:21.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88100 ceph_time root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:21.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:21.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:21.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:21.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:21.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:21.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:21.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 14w REG 8,1 142225325 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:21.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:21.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:21.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:21.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:21.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:21.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:21.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:21.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:21.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:21.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:21.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:21.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:21.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:21.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:21.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:21.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:21.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:21.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:21.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:21.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:21.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:21.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:21.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:21.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:21.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:21.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:21.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:21.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:21.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:21.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:21.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88148 ceph-osd root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:21.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:21.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:21.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:21.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:21.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:21.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:21.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 14w REG 8,1 142225325 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:21.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:21.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:21.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:21.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:21.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:21.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:21.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:21.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:21.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:21.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:21.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:21.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:21.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:21.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:21.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:21.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:21.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:21.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:21.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:21.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:21.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:21.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:21.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:21.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:21.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:21.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:21.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:21.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:21.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:21.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:21.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88161 ceph-osd root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:21.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:21.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:21.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:21.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:21.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:21.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:21.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 14w REG 8,1 142225325 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:21.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:21.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:21.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:21.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:21.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:21.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:21.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:21.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:21.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:21.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:21.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:21.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:21.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:21.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:21.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:21.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:21.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:21.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:21.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:21.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:21.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:21.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:21.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:21.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:21.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:21.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:21.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:21.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:21.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:21.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:21.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88175 ceph-osd root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:21.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:21.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:21.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:21.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:21.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:21.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:21.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 14w REG 8,1 142225608 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:21.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:21.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:21.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:21.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:21.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:21.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:21.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:21.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:21.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:21.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:21.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:21.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:21.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:21.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:21.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:21.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:21.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:21.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:21.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:21.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:21.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:21.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:21.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:21.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:21.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:21.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:21.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:21.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:21.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:21.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:21.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88183 ceph-osd root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:21.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:21.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:21.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:21.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:21.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:21.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:21.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 14w REG 8,1 142225608 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:21.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:21.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:21.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:21.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:21.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:21.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:21.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:21.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:21.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:21.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:21.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:21.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:21.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:21.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:21.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:21.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:21.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:21.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:21.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:21.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:21.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:21.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:21.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:21.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:21.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:21.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:21.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:21.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:21.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:21.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:21.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88191 ceph-osd root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:21.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:21.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:21.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:21.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:21.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:21.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:21.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 14w REG 8,1 142225608 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:21.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:21.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:21.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:21.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:21.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:21.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:21.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:21.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:21.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:21.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:21.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:21.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:21.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:21.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:21.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:21.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:21.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:21.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:21.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:21.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:21.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:21.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:21.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:21.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:21.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:21.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:21.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:21.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:21.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:21.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:21.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88199 ceph-osd root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:21.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:21.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:21.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:21.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:21.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:21.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:21.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 14w REG 8,1 142225608 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:21.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:21.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:21.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:21.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:21.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:21.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:21.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:21.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:21.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:21.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:21.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:21.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:21.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:21.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:21.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:21.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:21.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:21.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:21.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:21.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:21.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:21.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:21.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:21.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:21.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:21.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:21.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:21.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:21.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:21.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:21.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88207 ceph-osd root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:21.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:21.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:21.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:21.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:21.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:21.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:21.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 14w REG 8,1 142225608 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:21.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:21.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:21.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:21.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:21.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:21.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:21.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:21.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:21.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:21.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:21.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:21.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:21.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:21.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:21.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:21.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:21.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:21.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:21.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:21.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:21.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:21.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:21.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:21.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:21.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:21.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:21.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:21.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:21.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:21.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:21.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88597 ceph-osd root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:21.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:21.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:21.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:21.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:21.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:21.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:21.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 14w REG 8,1 142225608 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:21.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:21.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:21.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:21.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:21.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:21.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:21.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:21.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:21.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:21.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:21.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:21.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:21.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:21.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:21.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:21.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:21.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:21.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:21.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:21.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:21.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:21.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:21.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:21.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:21.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:21.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:21.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:21.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:21.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:21.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:21.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88598 ceph-osd root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:21.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:21.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:21.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:21.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:21.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:21.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:21.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 14w REG 8,1 142225608 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:21.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:21.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:21.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:21.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:21.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:21.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:21.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:21.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:21.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:21.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:21.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:21.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:21.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:21.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:21.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:21.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:21.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:21.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:21.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:21.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:21.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:21.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:21.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:21.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:21.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:21.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:21.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:21.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:21.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:21.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:21.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88599 safe_time root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:21.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:21.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:21.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:21.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:21.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:21.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:21.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 14w REG 8,1 142225608 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:21.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:21.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:21.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:21.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:21.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:21.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:21.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:21.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:21.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:21.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:21.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:21.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:21.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:21.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:21.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:21.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:21.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:21.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:21.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:21.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:21.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:21.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:21.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:21.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:21.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:21.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:21.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:21.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:21.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:21.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:21.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88600 safe_time root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:21.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:21.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:21.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:21.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:21.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:21.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:21.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 14w REG 8,1 142225608 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:21.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:21.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:21.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:21.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:21.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:21.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:21.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:21.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:21.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:21.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:21.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:21.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:21.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:21.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:21.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:21.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:21.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:21.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:21.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:21.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:21.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:21.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:21.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:21.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:21.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:21.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:21.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:21.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:21.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:21.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:21.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88601 safe_time root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:21.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:21.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:21.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:21.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:21.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:21.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:21.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:21.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:21.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:21.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:21.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:21.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:21.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:21.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:21.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:21.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:21.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:21.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:21.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:21.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:21.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:21.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:21.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:21.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:21.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:21.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:21.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:21.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:21.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:21.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:21.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:21.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:21.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:21.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:21.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:21.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:21.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:21.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:21.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:21.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:21.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:21.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:21.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:21.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:21.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:21.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:21.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:21.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:21.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:21.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:21.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:21.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:21.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:21.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:21.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:21.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:21.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:21.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:21.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:21.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:21.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:21.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:21.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:21.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:21.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 14w REG 8,1 142225608 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:21.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:21.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:21.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:21.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:21.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:21.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:21.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:21.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:21.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:21.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:21.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:21.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:21.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:21.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:21.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:21.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:21.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:21.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:21.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:21.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:21.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:21.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:21.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:21.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:21.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:21.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:21.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:21.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:21.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:21.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:21.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:21.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:21.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:21.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:21.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:21.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:21.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:21.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:21.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:21.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:21.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:21.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:21.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:21.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:21.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:21.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:21.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:21.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:21.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:21.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:21.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:21.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88602 safe_time root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:21.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:21.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:21.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:21.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:21.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:21.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:21.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:21.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:21.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:21.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:21.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:21.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:21.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:21.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:21.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:21.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:21.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:21.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:21.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:21.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:21.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:21.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:21.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:21.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 14w REG 8,1 142225608 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88603 fn_anonym root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 14w REG 8,1 142225608 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88606 rocksdb:l root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 14w REG 8,1 142225608 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88607 rocksdb:l root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 14w REG 8,1 142225608 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88608 rocksdb:l root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 14w REG 8,1 142225608 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 88609 rocksdb:h root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 14w REG 8,1 142225608 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90075 bstore_ai root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 14w REG 8,1 142225608 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 90850 bstore_ai root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 14w REG 8,1 142225608 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91226 ceph-osd root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 14w REG 8,1 142225608 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91252 cfin root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 14w REG 8,1 142225608 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91253 bstore_kv root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 14w REG 8,1 142225608 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91254 bstore_kv root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 14w REG 8,1 142225608 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91255 bstore_me root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 14w REG 8,1 142225608 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91261 ms_dispat root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 14w REG 8,1 142225608 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91262 ms_local root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 14w REG 8,1 142225608 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91263 safe_time root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 14w REG 8,1 142225891 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91264 safe_time root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 14w REG 8,1 142225891 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91265 ms_dispat root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 14w REG 8,1 142225891 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91266 ms_local root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 14w REG 8,1 142225891 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91267 ms_dispat root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 14w REG 8,1 142225891 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91268 ms_local root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 14w REG 8,1 142225891 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91269 ms_dispat root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 14w REG 8,1 142225891 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91270 ms_local root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 14w REG 8,1 142225891 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91271 ms_dispat root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 14w REG 8,1 142225891 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91272 ms_local root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 14w REG 8,1 142225891 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91273 ms_dispat root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 14w REG 8,1 142225891 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91274 ms_local root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 14w REG 8,1 142225891 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91275 ms_dispat root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 14w REG 8,1 142225891 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91276 ms_local root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 14w REG 8,1 142225891 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91277 fn_anonym root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:22.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:22.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:22.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:22.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:22.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:22.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:22.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:22.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:22.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:22.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:22.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:22.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:22.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:22.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:22.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:22.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:22.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:22.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:22.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:22.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:22.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:22.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:22.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:22.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:22.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:22.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:22.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:22.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:22.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:22.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:22.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:22.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:22.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:22.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:22.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 14w REG 8,1 142225891 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:22.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:22.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:22.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:22.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:22.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:22.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:22.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:22.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:22.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:22.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:22.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:22.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:22.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:22.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:22.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:22.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:22.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:22.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:22.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:22.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:22.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:22.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:22.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:22.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:22.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:22.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:22.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:22.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:22.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:22.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:22.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:22.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:22.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:22.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:22.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:22.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:22.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:22.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:22.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:22.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:22.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:22.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:22.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:22.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:22.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:22.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:22.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:22.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:22.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:22.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:22.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:22.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91278 finisher root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:22.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:22.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:22.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:22.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:22.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:22.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:22.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:22.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:22.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:22.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:22.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:22.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:22.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:22.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:22.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:22.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:22.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:22.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:22.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:22.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:22.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:22.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:22.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:22.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:22.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:22.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:22.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:22.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:22.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:22.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:22.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:22.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:22.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:22.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:22.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:22.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:22.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:22.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:22.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:22.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:22.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:22.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:22.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:22.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:22.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:22.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:22.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:22.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:22.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:22.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:22.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:22.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:22.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:22.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:23.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:23.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:23.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:23.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:23.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 14w REG 8,1 142225891 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:23.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:23.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:23.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:23.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:23.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:23.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:23.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:23.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:23.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:23.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:23.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:23.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:23.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:23.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:23.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:23.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:23.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:23.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:23.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:23.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:23.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:23.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:23.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:23.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:23.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:23.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:23.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:23.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:23.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:23.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:23.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:23.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:23.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91279 safe_time root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:23.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:23.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:23.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:23.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:23.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:23.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:23.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 14w REG 8,1 142225891 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:23.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:23.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:23.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:23.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:23.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:23.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:23.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:23.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:23.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:23.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:23.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:23.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:23.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:23.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:23.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:23.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:23.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:23.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:23.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:23.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:23.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:23.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:23.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:23.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:23.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:23.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:23.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:23.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:23.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:23.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:23.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:23.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:23.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91280 safe_time root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:23.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:23.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:23.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:23.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:23.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:23.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:23.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 14w REG 8,1 142225891 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:23.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:23.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:23.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:23.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:23.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:23.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:23.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:23.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:23.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:23.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:23.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:23.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:23.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:23.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:23.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:23.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:23.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:23.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:23.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:23.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:23.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:23.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:23.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:23.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:23.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:23.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:23.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:23.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:23.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:23.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:23.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:23.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:23.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91281 ceph-osd root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:23.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:23.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:23.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:23.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:23.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:23.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:23.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 14w REG 8,1 142225891 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:23.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:23.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:23.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:23.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:23.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:23.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:23.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:23.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:23.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:23.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:23.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:23.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:23.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:23.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:23.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:23.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:23.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:23.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:23.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:23.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:23.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:23.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:23.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:23.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:23.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:23.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:23.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:23.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:23.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:23.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:23.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:23.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:23.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91282 osd_srv_a root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:23.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:23.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:23.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:23.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:23.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:23.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:23.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 14w REG 8,1 142225891 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:23.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:23.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:23.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:23.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:23.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:23.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:23.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:23.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:23.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:23.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:23.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:23.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:23.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:23.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:23.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:23.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:23.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:23.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:23.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:23.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:23.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:23.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:23.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:23.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:23.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:23.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:23.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:23.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:23.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:23.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:23.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:23.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:23.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91283 tp_osd_tp root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:23.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:23.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:23.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:23.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:23.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:23.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:23.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 14w REG 8,1 142225891 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:23.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:23.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:23.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:23.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:23.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:23.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:23.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:23.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:23.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:23.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:23.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:23.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:23.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:23.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:23.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:23.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:23.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:23.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:23.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:23.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:23.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:23.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:23.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:23.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:23.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:23.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:23.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:23.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:23.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:23.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:23.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:23.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:23.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91284 tp_osd_tp root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:23.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:23.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:23.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:23.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:23.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:23.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:23.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 14w REG 8,1 142225891 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:23.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:23.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:23.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:23.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:23.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:23.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:23.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:23.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:23.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:23.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:23.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:23.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:23.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:23.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:23.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:23.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:23.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:23.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:23.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:23.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:23.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:23.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:23.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:23.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:23.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:23.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:23.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:23.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:23.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:23.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:23.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:23.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:23.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91285 tp_osd_tp root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:23.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:23.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:23.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:23.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:23.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:23.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:23.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 14w REG 8,1 142225891 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:23.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:23.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:23.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:23.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:23.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:23.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:23.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:23.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:23.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:23.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:23.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:23.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:23.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:23.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:23.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:23.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:23.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:23.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:23.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:23.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:23.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:23.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:23.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:23.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:23.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:23.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:23.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:23.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:23.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:23.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:23.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:23.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:23.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91286 tp_osd_tp root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:23.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:23.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:23.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:23.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:23.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:23.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:23.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 14w REG 8,1 142225891 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:23.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:23.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:23.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:23.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:23.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:23.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:23.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:23.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:23.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:23.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:23.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:23.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:23.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:23.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:23.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:23.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:23.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:23.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:23.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:23.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:23.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:23.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:23.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:23.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:23.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:23.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:23.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:23.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:23.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:23.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:23.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:23.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:23.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91287 tp_osd_tp root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:23.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:23.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:23.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:23.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:23.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:23.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:23.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 14w REG 8,1 142225891 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:23.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:23.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:23.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:23.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:23.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:23.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:23.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:23.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:23.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:23.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:23.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:23.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:23.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:23.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:23.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:23.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:23.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:23.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:23.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:23.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:23.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:23.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:23.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:23.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:23.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:23.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:23.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:23.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:23.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:23.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:23.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:23.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:23.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91288 tp_osd_tp root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:23.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:23.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:23.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:23.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:23.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:23.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:23.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 14w REG 8,1 142225891 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:23.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:23.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:23.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:23.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:23.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:23.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:23.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:23.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:23.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:23.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:23.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:23.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:23.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:23.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:23.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:23.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:23.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:23.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:23.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:23.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:23.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:23.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:23.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:23.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:23.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:23.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:23.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:23.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:23.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:23.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:23.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:23.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:23.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91289 tp_osd_tp root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:23.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:23.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:23.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:23.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:23.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:23.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:23.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 14w REG 8,1 142226174 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:23.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:23.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:23.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:23.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:23.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:23.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:23.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:23.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:23.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:23.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:23.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:23.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:23.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:23.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:23.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:23.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:23.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:23.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:23.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:23.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:23.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:23.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:23.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:23.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:23.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:23.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:23.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:23.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:23.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:23.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:23.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:23.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:23.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91290 tp_osd_tp root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:23.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:23.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:23.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:23.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:23.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:23.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:23.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 14w REG 8,1 142226174 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:23.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:23.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:23.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:23.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:23.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:23.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:23.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:23.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:23.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:23.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:23.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:23.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:23.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:23.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:23.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:23.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:23.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:23.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:23.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:23.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:23.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:23.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:23.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:23.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:23.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:23.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:23.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:23.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:23.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:23.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:23.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:23.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:23.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91291 tp_osd_tp root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:23.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:23.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:23.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:23.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:23.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:23.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:23.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 14w REG 8,1 142226174 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:23.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:23.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:23.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:23.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:23.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:23.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:23.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:23.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:23.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:23.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:23.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:23.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:23.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:23.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:23.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:23.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:23.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:23.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:23.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:23.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:23.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:23.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:23.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:23.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:23.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:23.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:23.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:23.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:23.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:23.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:23.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:23.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:23.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91292 tp_osd_tp root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:23.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:23.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:23.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:23.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:23.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:23.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:23.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 14w REG 8,1 142226174 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:23.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:23.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:23.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:23.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:23.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:23.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:23.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:23.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:23.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:23.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:23.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:23.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:23.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:23.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:23.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:23.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:23.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:23.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:23.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:23.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:23.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:23.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:23.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:23.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:23.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:23.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:23.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:23.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:23.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:23.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:23.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:23.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:23.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91293 tp_osd_tp root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:23.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:23.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:23.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:23.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:23.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:23.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:23.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 14w REG 8,1 142226174 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:23.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:23.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:23.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:23.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:23.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:23.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:23.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:23.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:23.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:23.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:23.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:23.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:23.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:23.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:23.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:23.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:23.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:23.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:23.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:23.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:23.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:23.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:23.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:23.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:23.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:23.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:23.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:23.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:23.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:23.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:23.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:23.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:23.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91294 tp_osd_tp root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:23.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:23.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:23.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:23.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:23.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:23.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:23.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 14w REG 8,1 142226174 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:23.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:23.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:23.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:23.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:23.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:23.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:23.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:23.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:23.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:23.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:23.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:23.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:23.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:23.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:23.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:23.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:23.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:23.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:23.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:23.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:23.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:23.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:23.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:23.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:23.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:23.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:23.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:23.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:23.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:23.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:23.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:23.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:23.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91295 tp_osd_tp root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:23.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:23.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:23.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:23.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:23.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:23.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:23.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 14w REG 8,1 142226174 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:23.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:23.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:23.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:23.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:23.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:23.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:23.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:23.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:23.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:23.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:23.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:23.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:23.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:23.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:23.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:23.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:23.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:23.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:23.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:23.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:23.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:23.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:23.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:23.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:23.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:23.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:23.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:23.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:23.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:23.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:23.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:23.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:23.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91296 tp_osd_tp root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:23.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:23.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:23.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:23.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:23.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:23.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:23.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 14w REG 8,1 142226174 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:23.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:23.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:23.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:23.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:23.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:23.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:23.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:23.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:23.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:23.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:23.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:23.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:23.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:23.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:23.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:23.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:23.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:23.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:23.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:23.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:23.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:23.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:23.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:23.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:23.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:23.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:23.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:23.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:23.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:23.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:23.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:23.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:23.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91297 tp_osd_tp root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:23.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:23.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:23.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:23.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:23.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:23.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:23.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 14w REG 8,1 142226174 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:23.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:23.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:23.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:23.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:23.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:23.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:23.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:23.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:23.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:23.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:23.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:23.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:23.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:23.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:23.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:23.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:23.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:23.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:23.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:23.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:23.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:23.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:23.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:23.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:23.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:23.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:23.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:23.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:23.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:23.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:23.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:23.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:23.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91298 tp_osd_tp root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root DEL REG 0,17 184421 /[aio] 2023-11-09T03:56:23.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root DEL REG 0,17 184419 /[aio] 2023-11-09T03:56:23.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 1w FIFO 0,13 0t0 182405 pipe 2023-11-09T03:56:23.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 2w FIFO 0,13 0t0 182406 pipe 2023-11-09T03:56:23.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 3r FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 4w FIFO 0,13 0t0 184366 pipe 2023-11-09T03:56:23.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,80,81,82,83,84,85,88,91,94,99,101] 2023-11-09T03:56:23.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 6r FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 7w FIFO 0,13 0t0 184368 pipe 2023-11-09T03:56:23.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 8u a_inode 0,14 0 2077 [eventpoll:9,58,65,68,70,72,74,75,76,86,89,92,95,97] 2023-11-09T03:56:23.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 9r FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 10w FIFO 0,13 0t0 184369 pipe 2023-11-09T03:56:23.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 11u a_inode 0,14 0 2077 [eventpoll:12,61,66,69,71,73,77,78,79,87,90,93,96,98] 2023-11-09T03:56:23.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 12r FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 13w FIFO 0,13 0t0 184370 pipe 2023-11-09T03:56:23.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 14w REG 8,1 142226174 265316 /var/log/ceph/ceph-osd.1.log 2023-11-09T03:56:23.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 15uW REG 0,24 6 1741 /run/ceph/ceph-osd.1.pid 2023-11-09T03:56:23.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 16r FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 17w FIFO 0,13 0t0 184373 pipe 2023-11-09T03:56:23.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 18u unix 0xffff959a47f0d0c0 0t0 184374 /var/run/ceph/ceph-osd.1.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 19u IPv4 177969 0t0 TCP *:6816 (LISTEN) 2023-11-09T03:56:23.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 20u IPv4 177987 0t0 TCP *:pentbox-sim (LISTEN) 2023-11-09T03:56:23.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 21u IPv4 178006 0t0 TCP *:6818 (LISTEN) 2023-11-09T03:56:23.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 22u IPv4 178026 0t0 TCP *:6819 (LISTEN) 2023-11-09T03:56:23.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 23u IPv4 178047 0t0 TCP *:6820 (LISTEN) 2023-11-09T03:56:23.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 24u IPv4 179753 0t0 TCP *:6821 (LISTEN) 2023-11-09T03:56:23.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 25u IPv4 179777 0t0 TCP *:6823 (LISTEN) 2023-11-09T03:56:23.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 26u IPv4 179803 0t0 TCP *:6825 (LISTEN) 2023-11-09T03:56:23.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 27r FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 28w FIFO 0,13 0t0 184380 pipe 2023-11-09T03:56:23.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 29r FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 30w FIFO 0,13 0t0 184381 pipe 2023-11-09T03:56:23.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 31u sock 0,8 0t0 184407 protocol: UDPv6 2023-11-09T03:56:23.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 32r DIR 253,1 357 64 /var/lib/ceph/osd/ceph-1 2023-11-09T03:56:23.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 33uW REG 253,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2023-11-09T03:56:23.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 34u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 35u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 36u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 37u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 38u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 39u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 40u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 41u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 42u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 43u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 44u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 45u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 46u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 47u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 48u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 49u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 50u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 51u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 52u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 53u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 54u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 55u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 56u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 57u REG 253,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2023-11-09T03:56:23.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 58u IPv4 183515 0t0 TCP smithi076.front.sepia.ceph.com:60338->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 59r FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 60w FIFO 0,13 0t0 182572 pipe 2023-11-09T03:56:23.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 61u IPv4 179830 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51582 (ESTABLISHED) 2023-11-09T03:56:23.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 62r FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 63w FIFO 0,13 0t0 182573 pipe 2023-11-09T03:56:23.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 64u IPv4 179833 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35824 (ESTABLISHED) 2023-11-09T03:56:23.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 65u IPv4 181632 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46006 (ESTABLISHED) 2023-11-09T03:56:23.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 66u IPv4 181635 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44974 (ESTABLISHED) 2023-11-09T03:56:23.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 67u IPv4 181638 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46012 (ESTABLISHED) 2023-11-09T03:56:23.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 68u IPv4 179847 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44986 (ESTABLISHED) 2023-11-09T03:56:23.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 69u IPv4 181639 0t0 TCP smithi076.front.sepia.ceph.com:35206->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 70u IPv4 183526 0t0 TCP smithi076.front.sepia.ceph.com:41868->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 71u IPv4 181640 0t0 TCP smithi076.front.sepia.ceph.com:32860->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 72u IPv4 183527 0t0 TCP smithi076.front.sepia.ceph.com:39424->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 73u IPv4 181641 0t0 TCP smithi076.front.sepia.ceph.com:53772->smithi076.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:23.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 74u IPv4 183528 0t0 TCP smithi076.front.sepia.ceph.com:44538->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 75u IPv4 183529 0t0 TCP smithi076.front.sepia.ceph.com:52070->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 76u IPv4 178372 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35828 (ESTABLISHED) 2023-11-09T03:56:23.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 77u IPv4 178373 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi076.front.sepia.ceph.com:35834 (ESTABLISHED) 2023-11-09T03:56:23.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 78u IPv4 181642 0t0 TCP smithi076.front.sepia.ceph.com:42982->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 79u IPv4 181643 0t0 TCP smithi076.front.sepia.ceph.com:39624->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 80u IPv4 178376 0t0 TCP smithi076.front.sepia.ceph.com:42360->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 81u IPv4 178377 0t0 TCP smithi076.front.sepia.ceph.com:51904->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 82u IPv4 178378 0t0 TCP smithi076.front.sepia.ceph.com:59320->smithi076.front.sepia.ceph.com:6805 (ESTABLISHED) 2023-11-09T03:56:23.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 83u IPv4 178379 0t0 TCP smithi076.front.sepia.ceph.com:44710->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 84u IPv4 178380 0t0 TCP smithi076.front.sepia.ceph.com:53972->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 85u IPv4 178381 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51586 (ESTABLISHED) 2023-11-09T03:56:23.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 86u IPv4 178384 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi076.front.sepia.ceph.com:51598 (ESTABLISHED) 2023-11-09T03:56:23.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 87u IPv4 180750 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:44992 (ESTABLISHED) 2023-11-09T03:56:23.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 88u IPv4 180753 0t0 TCP smithi076.front.sepia.ceph.com:6823->smithi113.front.sepia.ceph.com:45006 (ESTABLISHED) 2023-11-09T03:56:23.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 89u IPv4 180756 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46016 (ESTABLISHED) 2023-11-09T03:56:23.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 90u IPv4 180759 0t0 TCP smithi076.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:46024 (ESTABLISHED) 2023-11-09T03:56:23.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 91u IPv4 184469 0t0 TCP smithi076.front.sepia.ceph.com:45842->smithi076.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 92u IPv4 178396 0t0 TCP smithi076.front.sepia.ceph.com:44336->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 93u IPv4 185602 0t0 TCP smithi076.front.sepia.ceph.com:59772->smithi076.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 94u IPv4 178400 0t0 TCP smithi076.front.sepia.ceph.com:54334->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 95u IPv4 182627 0t0 TCP smithi076.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:44208 (ESTABLISHED) 2023-11-09T03:56:23.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 96u IPv4 184475 0t0 TCP smithi076.front.sepia.ceph.com:34680->smithi113.front.sepia.ceph.com:6810 (ESTABLISHED) 2023-11-09T03:56:23.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 97u IPv4 182639 0t0 TCP smithi076.front.sepia.ceph.com:59164->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2023-11-09T03:56:23.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 98u IPv4 178404 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi113.front.sepia.ceph.com:58948 (ESTABLISHED) 2023-11-09T03:56:23.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 99u IPv4 249698 0t0 TCP smithi076.front.sepia.ceph.com:6816->smithi076.front.sepia.ceph.com:38604 (ESTABLISHED) 2023-11-09T03:56:23.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88005 91299 osd_srv_h root 101u IPv4 180774 0t0 TCP smithi076.front.sepia.ceph.com:50034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2023-11-09T03:56:23.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:23.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:23.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:23.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:23.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:23.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:23.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:23.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:23.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:23.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:23.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:23.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:23.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:23.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:23.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:23.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 14w REG 8,1 146031213 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:23.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:23.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:23.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:23.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:23.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:23.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:23.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:23.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:23.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:23.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:23.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:23.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:23.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:23.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:23.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:23.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:23.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:23.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:23.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:23.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:23.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:23.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:23.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:23.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:23.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:23.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:23.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:23.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:23.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:23.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:23.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:23.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:23.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:23.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:23.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:23.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:23.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:23.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:23.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:23.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:23.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:23.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:23.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:23.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:23.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:23.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:23.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:23.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:23.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:23.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:23.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:23.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:23.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:23.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:23.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:23.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:23.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:23.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:23.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:23.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:23.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 14w REG 8,1 146026927 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:23.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:23.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:23.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:23.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:23.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:23.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:23.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:23.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:23.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:23.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:23.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:23.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:23.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:23.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:23.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:23.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:23.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:23.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:23.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:23.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:23.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:23.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:23.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:23.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:23.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:23.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:23.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:23.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:23.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:23.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:23.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:23.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:23.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:23.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:23.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:23.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:23.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:23.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:23.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:23.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:23.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:23.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:23.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:23.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:23.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88021 log root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:23.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:23.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:23.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:23.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:23.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:23.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:23.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:23.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:23.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:23.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:23.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:23.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:23.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:23.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:23.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:23.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 14w REG 8,1 146026927 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:23.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:23.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:23.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:23.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:23.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:23.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:23.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:23.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:23.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:23.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:23.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:23.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:23.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:23.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:23.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:23.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:23.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:23.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:23.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:23.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:23.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:23.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:23.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:23.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:23.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:23.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:23.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:23.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:23.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:23.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:23.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:23.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:23.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:23.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:23.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:23.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:23.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:23.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:23.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:23.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:23.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:23.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:23.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:23.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:23.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88025 msgr-work root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:23.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:23.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:23.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:23.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:23.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:23.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:23.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:23.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:23.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:23.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:23.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:23.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:23.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:23.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:23.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:23.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 14w REG 8,1 146026927 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:23.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:23.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:23.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:23.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:23.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:23.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:23.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:23.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:23.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:23.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:23.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:23.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:23.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:23.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:23.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:23.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:23.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:23.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:23.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:23.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:23.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:23.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:23.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:23.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:23.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:23.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:23.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:23.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:23.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:23.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:23.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:23.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:23.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:23.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:23.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:23.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:23.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:23.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:23.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:23.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:23.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:23.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:23.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:23.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:23.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88026 msgr-work root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:23.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:23.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:23.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:23.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:23.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:23.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:23.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:23.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:23.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:23.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:23.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:23.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:23.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:23.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:23.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:23.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 14w REG 8,1 146026927 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:23.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:23.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:23.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:23.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:23.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:23.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:23.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:23.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:23.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:23.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:23.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:23.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:23.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:23.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:23.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:23.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:23.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:23.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:23.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:23.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:23.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:23.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:23.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:23.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:23.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:23.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:23.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:23.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:23.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:23.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:23.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:23.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:23.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:23.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:23.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:23.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:23.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:23.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:23.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:23.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:23.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:23.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:23.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:23.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:23.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88027 msgr-work root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:23.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:23.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:23.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:23.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:23.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:23.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:23.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:23.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:23.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:23.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:23.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:23.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:23.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:23.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:23.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:23.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 14w REG 8,1 146026927 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:23.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:23.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:23.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:23.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:23.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:23.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:23.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:23.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:23.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:23.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:23.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:23.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:23.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:23.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:23.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:23.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:23.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:23.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:23.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:23.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:23.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:23.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:23.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:23.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:23.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:23.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:23.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:23.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:23.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:23.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:23.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:23.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:23.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:23.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:23.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:23.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:23.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:23.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:23.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:23.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:23.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:23.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:23.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:23.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:23.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88043 service root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:23.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:23.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:23.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:23.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:23.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:23.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:23.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:23.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:23.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:23.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:23.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:23.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:23.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:23.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:23.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:23.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 14w REG 8,1 146026927 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:23.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:23.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:23.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:23.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:23.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:23.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:23.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:23.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:23.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:23.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:23.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:23.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:23.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:23.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:23.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:23.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:23.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:23.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:23.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:23.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:23.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:23.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:23.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:23.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:23.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:23.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:23.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:23.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:23.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:23.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:23.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:23.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:23.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:23.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:23.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:23.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:23.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:23.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:23.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:23.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:23.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:23.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:23.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:23.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:23.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88044 admin_soc root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:23.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:23.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:23.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:23.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:23.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:23.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:23.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:23.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:23.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:23.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:23.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:23.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:23.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:23.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:23.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:23.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 14w REG 8,1 146026927 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:23.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:23.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:23.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:23.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:23.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:23.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:23.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:23.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:23.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:23.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:23.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:23.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:23.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:23.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:23.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:23.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:23.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:23.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:23.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:23.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:23.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:23.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:23.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:23.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:23.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:23.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:23.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:23.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:23.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:23.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:23.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:23.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:23.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:23.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:23.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:23.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:23.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:23.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:23.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:23.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:23.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:23.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:23.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:23.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:23.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88077 signal_ha root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:23.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:23.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:23.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:23.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:23.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:23.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:23.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:23.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:23.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:23.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:23.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:23.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:23.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:23.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:23.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:23.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 14w REG 8,1 146026927 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:23.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:23.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:23.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:23.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:23.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:23.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:23.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:23.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:23.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:23.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:23.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:23.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:23.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:23.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:23.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:23.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:23.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:23.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:23.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:23.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:23.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:23.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:23.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:23.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:23.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:23.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:23.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:23.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:23.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:23.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:23.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:23.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:23.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:23.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:23.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:23.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:23.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:23.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:23.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:23.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:23.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:23.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:23.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:23.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:23.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88078 io_contex root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:23.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:23.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:23.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:23.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:23.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:23.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:23.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:23.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:23.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:23.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:23.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:23.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:23.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:23.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:23.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:23.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 14w REG 8,1 146026927 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:23.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:23.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:23.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:23.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:23.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:23.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:23.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:23.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:23.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:23.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:23.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:23.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:23.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:23.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:23.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:23.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:23.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:23.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:23.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:23.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:23.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:23.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:23.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:23.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:23.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:23.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:23.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:23.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:23.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:23.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:23.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:23.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:23.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:23.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:23.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:23.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:23.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:23.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:23.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:23.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:23.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:23.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:23.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:23.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:23.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88079 io_contex root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:23.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:23.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:23.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:23.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:23.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:23.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:23.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:23.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:23.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:23.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:23.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:23.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:23.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:23.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:23.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:23.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:23.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:23.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:23.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:23.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:23.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:23.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:23.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:23.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:23.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:23.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:23.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:23.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:23.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:23.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:23.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:23.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:23.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:23.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:23.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:23.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:23.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:23.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:23.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:23.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:23.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:23.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:23.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:23.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:23.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:23.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:23.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:23.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:23.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:23.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:23.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:23.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:23.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:23.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:23.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:23.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:23.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:23.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:23.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:23.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:23.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:23.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:23.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:23.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:23.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:23.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:23.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:23.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:23.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 14w REG 8,1 146026927 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:23.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:23.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:23.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:23.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:23.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:23.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:23.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:23.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:23.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:23.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:23.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:23.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:23.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:23.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:23.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:23.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:23.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:23.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:23.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:23.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:23.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:23.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:23.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:23.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:23.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:23.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:23.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:23.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:23.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:23.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:23.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:23.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:23.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:23.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:23.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:23.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:23.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:23.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:23.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:23.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:23.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:23.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:23.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:23.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:23.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:23.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:23.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:23.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:23.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:23.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:23.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:23.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:23.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:23.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:23.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:23.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:23.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:23.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88093 OpHistory root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:23.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:23.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:23.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:23.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:23.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:23.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:23.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:23.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:23.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:23.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:23.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:23.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:23.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:23.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:23.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:23.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:23.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:23.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:23.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:23.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:23.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:23.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:23.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:23.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:23.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:23.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:23.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 14w REG 8,1 146026927 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88094 ceph_time root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 14w REG 8,1 146026927 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88142 ceph-osd root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 14w REG 8,1 146026927 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88156 ceph-osd root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 14w REG 8,1 146026927 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88171 ceph-osd root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 14w REG 8,1 146026927 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88179 ceph-osd root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 14w REG 8,1 146026927 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88187 ceph-osd root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 14w REG 8,1 146026927 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88195 ceph-osd root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 14w REG 8,1 146026927 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88203 ceph-osd root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 14w REG 8,1 146026927 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88211 ceph-osd root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 14w REG 8,1 146026927 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88212 ceph-osd root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 14w REG 8,1 146026927 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88213 safe_time root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 14w REG 8,1 146026927 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88214 safe_time root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 14w REG 8,1 146026927 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88215 safe_time root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 14w REG 8,1 146027756 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88216 safe_time root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 14w REG 8,1 146028039 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88217 fn_anonym root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 14w REG 8,1 146028039 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88227 rocksdb:l root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 14w REG 8,1 146028039 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88228 rocksdb:l root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 14w REG 8,1 146028039 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88229 rocksdb:l root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 14w REG 8,1 146028039 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 88230 rocksdb:h root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 14w REG 8,1 146028039 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 89711 bstore_ai root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 14w REG 8,1 146028039 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90439 bstore_ai root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 14w REG 8,1 146028039 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90620 ceph-osd root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 14w REG 8,1 146028039 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90621 cfin root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 14w REG 8,1 146028039 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90622 bstore_kv root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 14w REG 8,1 146028039 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90623 bstore_kv root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 14w REG 8,1 146028039 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90624 bstore_me root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 14w REG 8,1 146028039 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90625 ms_dispat root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 14w REG 8,1 146028039 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90626 ms_local root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 14w REG 8,1 146028039 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90627 safe_time root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 14w REG 8,1 146028039 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:24.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:24.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:24.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:24.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:24.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:24.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:24.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:24.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:24.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:24.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:24.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:24.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:24.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:24.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:24.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:24.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:24.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:24.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:24.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:24.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:24.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:24.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:24.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:24.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:24.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:24.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:24.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:24.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:24.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:24.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:24.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:24.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:24.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:24.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:24.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:24.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:24.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:24.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:24.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:24.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:24.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:24.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:24.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:24.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:24.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:24.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:24.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:24.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:24.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:24.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:24.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90628 safe_time root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:24.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:24.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:24.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:24.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:24.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:24.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:24.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:24.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:24.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:24.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:24.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:24.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:24.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:24.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:24.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:24.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:24.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:24.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:24.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:24.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:24.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:24.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:24.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:24.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:24.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:24.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:24.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:24.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:24.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:24.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:24.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:24.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:24.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:24.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:24.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:24.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:24.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:24.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:24.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:24.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:24.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:24.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:24.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:24.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:24.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:24.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:24.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:24.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:24.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:24.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:24.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:24.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:24.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:24.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:24.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:24.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:24.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:24.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:24.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:24.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:24.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:24.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:24.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:24.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:24.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:24.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:24.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:24.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:24.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:24.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:24.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:24.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:24.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:24.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:24.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:24.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:24.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:24.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:24.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:24.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:24.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:24.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:24.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:24.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:24.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:24.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:24.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:24.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:24.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 14w REG 8,1 146028039 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:24.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:24.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90629 ms_dispat root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 14w REG 8,1 146028039 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90630 ms_local root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 14w REG 8,1 146028039 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90631 ms_dispat root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 14w REG 8,1 146028039 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90632 ms_local root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 14w REG 8,1 146028039 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90633 ms_dispat root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 14w REG 8,1 146028039 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90634 ms_local root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 14w REG 8,1 146028039 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90635 ms_dispat root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 14w REG 8,1 146030647 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90636 ms_local root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 14w REG 8,1 146030647 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90637 ms_dispat root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 14w REG 8,1 146030930 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90638 ms_local root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 14w REG 8,1 146030930 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90639 ms_dispat root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 14w REG 8,1 146030930 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90640 ms_local root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 14w REG 8,1 146030930 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90641 fn_anonym root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 14w REG 8,1 146030930 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90642 finisher root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 14w REG 8,1 146030930 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90643 safe_time root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 14w REG 8,1 146030930 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90644 safe_time root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 14w REG 8,1 146030930 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.520 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90645 ceph-osd root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.521 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.522 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.523 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.524 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.525 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.526 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.527 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.528 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.529 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.530 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.531 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.532 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.533 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.534 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.535 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 14w REG 8,1 146030930 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90646 osd_srv_a root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 14w REG 8,1 146030930 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90647 tp_osd_tp root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 14w REG 8,1 146030930 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90648 tp_osd_tp root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.620 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 14w REG 8,1 146030930 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90649 tp_osd_tp root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 14w REG 8,1 146030930 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90650 tp_osd_tp root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 14w REG 8,1 146030930 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90651 tp_osd_tp root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 14w REG 8,1 146030930 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90652 tp_osd_tp root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 14w REG 8,1 146030930 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.760 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.761 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.763 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.764 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.765 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.766 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.767 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.768 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.769 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.770 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.771 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.772 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.773 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.774 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90653 tp_osd_tp root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.775 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.776 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.777 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.778 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.779 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.780 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.781 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.782 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.783 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.784 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.785 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.786 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.788 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.789 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.790 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 14w REG 8,1 146030930 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90654 tp_osd_tp root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 14w REG 8,1 146030930 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90655 tp_osd_tp root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 14w REG 8,1 146030930 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90656 tp_osd_tp root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 14w REG 8,1 146030930 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90657 tp_osd_tp root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 14w REG 8,1 146030930 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90658 tp_osd_tp root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 14w REG 8,1 146030930 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.957 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.958 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.959 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90659 tp_osd_tp root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:25.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:25.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:25.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:25.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:25.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:25.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:25.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:25.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:25.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:25.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:25.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:25.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:25.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:25.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:25.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:25.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:25.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:25.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:25.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:25.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:25.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:25.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:25.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:25.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:25.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:25.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:25.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:25.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:25.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:25.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:25.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:25.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:25.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:25.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:25.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:25.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:25.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:25.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:25.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:25.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:25.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:25.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:25.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:25.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:25.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:25.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:25.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:25.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:25.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:25.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:25.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:25.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:25.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:25.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:25.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:25.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:25.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 14w REG 8,1 146030930 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:25.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:25.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:25.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:25.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:25.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:25.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:25.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:25.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:25.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:25.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:25.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:25.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:25.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:25.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:25.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:25.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:25.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:25.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:25.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:25.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:25.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:25.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:25.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:25.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:25.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:25.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:25.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:25.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:25.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:25.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:25.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:25.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:25.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:25.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:25.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:25.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:25.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:25.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:25.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:25.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:25.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:25.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:25.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:25.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:25.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:25.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:25.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:25.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:25.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:25.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:25.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:25.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:25.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90660 tp_osd_tp root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:25.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:25.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:25.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:25.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:25.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:25.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:25.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:25.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:25.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:25.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:25.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:25.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:25.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:25.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:25.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:25.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:25.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:25.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:25.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:25.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:25.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:25.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:25.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:25.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:25.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:25.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:25.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:25.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:25.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:25.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:25.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:25.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:26.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:26.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:26.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:26.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:26.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:26.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:26.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:26.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:26.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:26.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:26.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:26.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:26.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:26.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:26.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:26.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:26.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:26.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 14w REG 8,1 146030930 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:26.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:26.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:26.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:26.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:26.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:26.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:26.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:26.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:26.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:26.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:26.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:26.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:26.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:26.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:26.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:26.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:26.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:26.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:26.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:26.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:26.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:26.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:26.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:26.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:26.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:26.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:26.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:26.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:26.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:26.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:26.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:26.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:26.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:26.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:26.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:26.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:26.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:26.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:26.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:26.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:26.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:26.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:26.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:26.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:26.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:26.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:26.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:26.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:26.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:26.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:26.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:26.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:26.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:26.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:26.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:26.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:26.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:26.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90661 tp_osd_tp root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:26.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:26.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:26.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:26.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:26.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:26.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:26.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:26.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:26.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:26.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:26.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:26.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:26.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:26.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:26.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:26.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:26.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:26.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:26.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:26.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:26.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:26.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:26.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:26.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:26.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:26.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:26.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:26.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:26.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:26.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:26.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:26.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:26.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:26.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:26.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:26.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:26.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:26.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:26.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:26.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:26.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:26.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:26.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:26.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:26.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:26.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 14w REG 8,1 146030930 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:26.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:26.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:26.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:26.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:26.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:26.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:26.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:26.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:26.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:26.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:26.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:26.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:26.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:26.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:26.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:26.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:26.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:26.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:26.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:26.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:26.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:26.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:26.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:26.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:26.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:26.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:26.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:26.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:26.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:26.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:26.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:26.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:26.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:26.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:26.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:26.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:26.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:26.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:26.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:26.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:26.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:26.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:26.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:26.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:26.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:26.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:26.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:26.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:26.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:26.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:26.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:26.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:26.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:26.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:26.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:26.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:26.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:26.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90662 tp_osd_tp root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:26.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root txt REG 8,1 25615776 3093 /usr/bin/ceph-osd 2023-11-09T03:56:26.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 74496 276162 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-11-09T03:56:26.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 116440 276189 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-11-09T03:56:26.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 128888 276141 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-11-09T03:56:26.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 91248 276177 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-11-09T03:56:26.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 83008 276159 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-11-09T03:56:26.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 49424 276165 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-11-09T03:56:26.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 1350912 276186 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-11-09T03:56:26.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 224600 276153 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-11-09T03:56:26.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 864280 276183 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-11-09T03:56:26.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 45176 276198 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-11-09T03:56:26.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 65968 276201 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-11-09T03:56:26.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 74936 276144 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-11-09T03:56:26.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 41168 276147 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-11-09T03:56:26.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 36976 276204 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-11-09T03:56:26.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 70536 276150 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-11-09T03:56:26.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 69736 2466 /usr/lib64/liboath.so.0.1.3 2023-11-09T03:56:26.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 70424 276174 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-11-09T03:56:26.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 409304 276180 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-11-09T03:56:26.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 37064 276156 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-11-09T03:56:26.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 32512 276171 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-11-09T03:56:26.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 32664 276112 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-11-09T03:56:26.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root DEL REG 0,17 182567 /[aio] 2023-11-09T03:56:26.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root DEL REG 0,17 182565 /[aio] 2023-11-09T03:56:26.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 1469128 276131 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-11-09T03:56:26.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 353384 276127 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-11-09T03:56:26.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 74672 276137 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-11-09T03:56:26.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:26.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-11-09T03:56:26.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-11-09T03:56:26.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-11-09T03:56:26.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:26.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 1w FIFO 0,13 0t0 183333 pipe 2023-11-09T03:56:26.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 2w FIFO 0,13 0t0 183334 pipe 2023-11-09T03:56:26.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 3r FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:26.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 4w FIFO 0,13 0t0 182454 pipe 2023-11-09T03:56:26.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,21,22,23,24,25,26,64,67,70,81,83,84,85,86,88,91,95,96,101] 2023-11-09T03:56:26.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 6r FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:26.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 7w FIFO 0,13 0t0 182456 pipe 2023-11-09T03:56:26.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 8u a_inode 0,14 0 2077 [eventpoll:9,63,65,68,73,75,76,77,79,82,89,92,94,99] 2023-11-09T03:56:26.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 9r FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:26.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 10w FIFO 0,13 0t0 182457 pipe 2023-11-09T03:56:26.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 11u a_inode 0,14 0 2077 [eventpoll:12,59,66,69,71,72,74,78,80,87,90,93,97,100] 2023-11-09T03:56:26.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 12r FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:26.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 13w FIFO 0,13 0t0 182458 pipe 2023-11-09T03:56:26.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 14w REG 8,1 146030930 265318 /var/log/ceph/ceph-osd.3.log 2023-11-09T03:56:26.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 15uW REG 0,24 6 1739 /run/ceph/ceph-osd.3.pid 2023-11-09T03:56:26.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 16r FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:26.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 17w FIFO 0,13 0t0 177948 pipe 2023-11-09T03:56:26.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 18u unix 0xffff959a60348880 0t0 177949 /var/run/ceph/ceph-osd.3.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 19u IPv4 183428 0t0 TCP *:6800 (LISTEN) 2023-11-09T03:56:26.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 20u IPv4 183430 0t0 TCP *:acnet (LISTEN) 2023-11-09T03:56:26.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 21u IPv4 183433 0t0 TCP *:6802 (LISTEN) 2023-11-09T03:56:26.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 22u IPv4 183438 0t0 TCP *:6804 (LISTEN) 2023-11-09T03:56:26.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 23u IPv4 183444 0t0 TCP *:6805 (LISTEN) 2023-11-09T03:56:26.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 24u IPv4 183452 0t0 TCP *:6807 (LISTEN) 2023-11-09T03:56:26.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 25u IPv4 183461 0t0 TCP *:6808 (LISTEN) 2023-11-09T03:56:26.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 26u IPv4 183472 0t0 TCP *:6810 (LISTEN) 2023-11-09T03:56:26.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 27r FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:26.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 28w FIFO 0,13 0t0 182467 pipe 2023-11-09T03:56:26.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 29r FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:26.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 30w FIFO 0,13 0t0 182468 pipe 2023-11-09T03:56:26.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 31u sock 0,8 0t0 182553 protocol: UDPv6 2023-11-09T03:56:26.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 32r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-3 2023-11-09T03:56:26.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 33uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2023-11-09T03:56:26.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 55u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 56u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 57u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2023-11-09T03:56:26.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 58r FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:26.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 59u IPv4 181610 0t0 TCP smithi076.front.sepia.ceph.com:60322->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 60w FIFO 0,13 0t0 180719 pipe 2023-11-09T03:56:26.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 61r FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:26.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 62w FIFO 0,13 0t0 180720 pipe 2023-11-09T03:56:26.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 63u IPv4 184446 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59296 (ESTABLISHED) 2023-11-09T03:56:26.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 64u IPv4 184449 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53758 (ESTABLISHED) 2023-11-09T03:56:26.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 65u IPv4 178358 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42276 (ESTABLISHED) 2023-11-09T03:56:26.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 66u IPv4 178359 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42292 (ESTABLISHED) 2023-11-09T03:56:26.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 67u IPv4 178362 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54560 (ESTABLISHED) 2023-11-09T03:56:26.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 68u IPv4 178363 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54566 (ESTABLISHED) 2023-11-09T03:56:26.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 69u IPv4 184458 0t0 TCP smithi076.front.sepia.ceph.com:41846->smithi076.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:26.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 70u IPv4 180731 0t0 TCP smithi076.front.sepia.ceph.com:32848->smithi076.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:26.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 71u IPv4 184459 0t0 TCP smithi076.front.sepia.ceph.com:39412->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:26.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 72u IPv4 184460 0t0 TCP smithi076.front.sepia.ceph.com:44692->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:26.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 73u IPv4 178368 0t0 TCP smithi076.front.sepia.ceph.com:51902->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 74u IPv4 184461 0t0 TCP smithi076.front.sepia.ceph.com:53968->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2023-11-09T03:56:26.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 75u IPv4 178369 0t0 TCP smithi076.front.sepia.ceph.com:42960->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2023-11-09T03:56:26.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 76u IPv4 178370 0t0 TCP smithi076.front.sepia.ceph.com:39612->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2023-11-09T03:56:26.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 77u IPv4 178371 0t0 TCP smithi076.front.sepia.ceph.com:42356->smithi076.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:26.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 78u IPv4 184462 0t0 TCP smithi076.front.sepia.ceph.com:35834->smithi076.front.sepia.ceph.com:6823 (ESTABLISHED) 2023-11-09T03:56:26.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 79u IPv4 180738 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59310 (ESTABLISHED) 2023-11-09T03:56:26.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 80u IPv4 180739 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi076.front.sepia.ceph.com:59320 (ESTABLISHED) 2023-11-09T03:56:26.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 81u IPv4 180742 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53772 (ESTABLISHED) 2023-11-09T03:56:26.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 82u IPv4 180743 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi076.front.sepia.ceph.com:53788 (ESTABLISHED) 2023-11-09T03:56:26.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 83u IPv4 180746 0t0 TCP smithi076.front.sepia.ceph.com:44552->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-11-09T03:56:26.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 84u IPv4 180747 0t0 TCP smithi076.front.sepia.ceph.com:52082->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2023-11-09T03:56:26.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 85u IPv4 180748 0t0 TCP smithi076.front.sepia.ceph.com:35220->smithi076.front.sepia.ceph.com:6814 (ESTABLISHED) 2023-11-09T03:56:26.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 86u IPv4 180749 0t0 TCP smithi076.front.sepia.ceph.com:51598->smithi076.front.sepia.ceph.com:6820 (ESTABLISHED) 2023-11-09T03:56:26.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 87u IPv4 178387 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54574 (ESTABLISHED) 2023-11-09T03:56:26.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 88u IPv4 178390 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42294 (ESTABLISHED) 2023-11-09T03:56:26.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 89u IPv4 184466 0t0 TCP smithi076.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54576 (ESTABLISHED) 2023-11-09T03:56:26.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 90u IPv4 178393 0t0 TCP smithi076.front.sepia.ceph.com:6805->smithi113.front.sepia.ceph.com:42306 (ESTABLISHED) 2023-11-09T03:56:26.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 91u IPv4 179852 0t0 TCP smithi076.front.sepia.ceph.com:44312->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 92u IPv4 181666 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59766 (ESTABLISHED) 2023-11-09T03:56:26.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 93u IPv4 182625 0t0 TCP smithi076.front.sepia.ceph.com:54328->smithi076.front.sepia.ceph.com:6809 (ESTABLISHED) 2023-11-09T03:56:26.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 94u IPv4 179853 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi076.front.sepia.ceph.com:59772 (ESTABLISHED) 2023-11-09T03:56:26.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 95u IPv4 181697 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:36294 (ESTABLISHED) 2023-11-09T03:56:26.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 96u IPv4 257102 0t0 TCP smithi076.front.sepia.ceph.com:6800->smithi076.front.sepia.ceph.com:60920 (ESTABLISHED) 2023-11-09T03:56:26.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 97u IPv4 179858 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55334 (ESTABLISHED) 2023-11-09T03:56:26.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 99u IPv4 179859 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55344 (ESTABLISHED) 2023-11-09T03:56:26.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 100u IPv4 179860 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55358 (ESTABLISHED) 2023-11-09T03:56:26.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-osd 88006 90663 osd_srv_h root 101u IPv4 179861 0t0 TCP smithi076.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:55374 (ESTABLISHED) 2023-11-09T03:56:26.089 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/3 91620 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:26.090 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/3 91620 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.090 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/3 91620 root txt unknown /proc/91620/exe 2023-11-09T03:56:26.090 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/4 91634 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:26.090 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/4 91634 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.090 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/4 91634 root txt unknown /proc/91634/exe 2023-11-09T03:56:26.090 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/5 91777 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:26.091 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/5 91777 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.091 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/5 91777 root txt unknown /proc/91777/exe 2023-11-09T03:56:26.091 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/7 91791 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:26.091 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/7 91791 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.091 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/7 91791 root txt unknown /proc/91791/exe 2023-11-09T03:56:26.092 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/0 91938 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:26.092 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/0 91938 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.092 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/0 91938 root txt unknown /proc/91938/exe 2023-11-09T03:56:26.092 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/5 91982 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:26.092 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/5 91982 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.092 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/5 91982 root txt unknown /proc/91982/exe 2023-11-09T03:56:26.093 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/0 91991 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:26.093 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/0 91991 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.093 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/0 91991 root txt unknown /proc/91991/exe 2023-11-09T03:56:26.093 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/7 92133 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:26.093 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/7 92133 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.094 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/7 92133 root txt unknown /proc/92133/exe 2023-11-09T03:56:26.094 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/6 92518 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:26.094 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/6 92518 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.094 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/6 92518 root txt unknown /proc/92518/exe 2023-11-09T03:56:26.094 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/6 92519 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:26.095 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/6 92519 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.095 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/6 92519 root txt unknown /proc/92519/exe 2023-11-09T03:56:26.095 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/1 92747 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:26.095 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/1 92747 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.095 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/1 92747 root txt unknown /proc/92747/exe 2023-11-09T03:56:26.095 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/1 92763 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:26.096 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/1 92763 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.096 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/1 92763 root txt unknown /proc/92763/exe 2023-11-09T03:56:26.096 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/4 92974 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:26.096 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/4 92974 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.096 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/4 92974 root txt unknown /proc/92974/exe 2023-11-09T03:56:26.097 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/u 120230 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:26.097 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/u 120230 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.097 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/u 120230 root txt unknown /proc/120230/exe 2023-11-09T03:56:26.097 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/7 120335 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:26.097 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/7 120335 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.097 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/7 120335 root txt unknown /proc/120335/exe 2023-11-09T03:56:26.098 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/3 121726 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:26.098 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/3 121726 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.098 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/3 121726 root txt unknown /proc/121726/exe 2023-11-09T03:56:26.098 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/5 121806 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:26.098 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/5 121806 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.098 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/5 121806 root txt unknown /proc/121806/exe 2023-11-09T03:56:26.099 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/4 122113 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:26.099 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/4 122113 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.099 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/4 122113 root txt unknown /proc/122113/exe 2023-11-09T03:56:26.099 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/4 124193 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:26.099 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/4 124193 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.100 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/4 124193 root txt unknown /proc/124193/exe 2023-11-09T03:56:26.100 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.100 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.100 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root txt REG 8,1 185296 11759 /usr/bin/sudo 2023-11-09T03:56:26.100 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 357408 10120 /usr/lib64/libnss_systemd.so.2 2023-11-09T03:56:26.100 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:26.101 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.101 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-11-09T03:56:26.101 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:26.101 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 19600 10967 /usr/lib64/libpam_misc.so.0.82.1 2023-11-09T03:56:26.101 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 19400 32305 /usr/lib64/security/pam_succeed_if.so 2023-11-09T03:56:26.101 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 514280 32422 /usr/lib64/security/pam_systemd.so 2023-11-09T03:56:26.102 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 27536 32284 /usr/lib64/security/pam_limits.so 2023-11-09T03:56:26.102 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 48392 10228 /usr/lib64/libcrack.so.2.9.0 2023-11-09T03:56:26.102 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 32608 10961 /usr/lib64/libpwquality.so.1.0.2 2023-11-09T03:56:26.102 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 15352 32282 /usr/lib64/security/pam_keyinit.so 2023-11-09T03:56:26.102 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 15840 32264 /usr/lib64/security/pam_pwquality.so 2023-11-09T03:56:26.102 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 14928 32270 /usr/lib64/security/pam_deny.so 2023-11-09T03:56:26.103 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 56712 32310 /usr/lib64/security/pam_unix.so 2023-11-09T03:56:26.103 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.103 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.103 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.103 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.104 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.104 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.104 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.104 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.104 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.104 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.105 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.105 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.105 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-11-09T03:56:26.105 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.105 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.105 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-11-09T03:56:26.105 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 472568 54513 /usr/libexec/sudo/sudoers.so 2023-11-09T03:56:26.106 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:26.106 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.106 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.106 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 15184 32274 /usr/lib64/security/pam_faildelay.so 2023-11-09T03:56:26.106 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.107 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.107 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-11-09T03:56:26.107 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 19464 32272 /usr/lib64/security/pam_env.so 2023-11-09T03:56:26.107 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-11-09T03:56:26.107 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-11-09T03:56:26.107 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-11-09T03:56:26.108 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-11-09T03:56:26.108 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-11-09T03:56:26.108 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-11-09T03:56:26.108 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-11-09T03:56:26.108 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-11-09T03:56:26.108 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-11-09T03:56:26.109 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:26.109 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 119832 54509 /usr/libexec/sudo/libsudo_util.so.0.0.0 2023-11-09T03:56:26.109 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.109 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:26.109 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:26.109 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.110 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-11-09T03:56:26.110 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root 0r FIFO 0,13 0t0 248572 pipe 2023-11-09T03:56:26.110 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root 1w FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:26.110 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root 2w FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:26.110 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root 3r FIFO 0,13 0t0 249279 pipe 2023-11-09T03:56:26.110 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root 4w FIFO 0,13 0t0 249279 pipe 2023-11-09T03:56:26.111 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root 5u netlink 0t0 249284 AUDIT 2023-11-09T03:56:26.111 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 124494 root 6u unix 0xffff959a618fe1c0 0t0 249285 type=DGRAM (CONNECTED) 2023-11-09T03:56:26.111 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 124528 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.111 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 124528 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.111 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 124528 root txt REG 8,1 15448 10498 /usr/bin/python3.9 2023-11-09T03:56:26.112 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 124528 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:26.112 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 124528 root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:26.112 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 124528 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.112 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 124528 root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.112 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 124528 root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.112 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 124528 root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.113 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 124528 root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.113 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 124528 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.113 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 124528 root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.113 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 124528 root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.113 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 124528 root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.113 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 124528 root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.114 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 124528 root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.114 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 124528 root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.114 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 124528 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:26.114 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 124528 root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:26.114 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 124528 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.114 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 124528 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:26.115 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 124528 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.115 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 124528 root 0r FIFO 0,13 0t0 248572 pipe 2023-11-09T03:56:26.115 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 124528 root 1w FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:26.115 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 124528 root 2w FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:26.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 1w FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:26.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 2w FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:26.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 3r FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 4w FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,31,34,37] 2023-11-09T03:56:26.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 6r FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 7w FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 8u a_inode 0,14 0 2077 [eventpoll:9,25,32,35] 2023-11-09T03:56:26.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 9r FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 10w FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 11u a_inode 0,14 0 2077 [eventpoll:12,30,33,36,39] 2023-11-09T03:56:26.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 12r FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 13w FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 14w REG 8,1 9139477 268502 /var/log/ceph/ceph-mds.c.log 2023-11-09T03:56:26.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 15uW REG 0,24 7 2398 /run/ceph/ceph-mds.c.pid 2023-11-09T03:56:26.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 16r FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 17w FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 18u unix 0xffff959a60112ec0 0t0 252220 /var/run/ceph/ceph-mds.c.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 19u IPv4 251300 0t0 TCP *:6832 (LISTEN) 2023-11-09T03:56:26.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 20u IPv4 251334 0t0 TCP *:6833 (LISTEN) 2023-11-09T03:56:26.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 21r FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 22w FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 23r FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 24w FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 25u IPv4 247722 0t0 TCP smithi076.front.sepia.ceph.com:60028->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 26r FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 27w FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 28r FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 29w FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 30u IPv4 252224 0t0 TCP smithi076.front.sepia.ceph.com:59706->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 31u IPv4 250772 0t0 TCP smithi076.front.sepia.ceph.com:38604->smithi076.front.sepia.ceph.com:6816 (ESTABLISHED) 2023-11-09T03:56:26.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 32u IPv4 256430 0t0 TCP smithi076.front.sepia.ceph.com:33348->smithi076.front.sepia.ceph.com:6803 (ESTABLISHED) 2023-11-09T03:56:26.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 33u IPv4 249701 0t0 TCP smithi076.front.sepia.ceph.com:56048->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:26.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 34u IPv4 249702 0t0 TCP smithi076.front.sepia.ceph.com:48810->smithi076.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 35u IPv4 249703 0t0 TCP smithi076.front.sepia.ceph.com:37152->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 36u IPv4 256431 0t0 TCP smithi076.front.sepia.ceph.com:60920->smithi076.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 37u IPv4 257101 0t0 TCP smithi076.front.sepia.ceph.com:50856->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2023-11-09T03:56:26.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 root 39u IPv4 272562 0t0 TCP smithi076.front.sepia.ceph.com:6832->192.168.0.1:35628 (ESTABLISHED) 2023-11-09T03:56:26.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 1w FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:26.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 2w FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:26.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 3r FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 4w FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,31,34,37] 2023-11-09T03:56:26.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 6r FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 7w FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 8u a_inode 0,14 0 2077 [eventpoll:9,25,32,35] 2023-11-09T03:56:26.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 9r FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 10w FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 11u a_inode 0,14 0 2077 [eventpoll:12,30,33,36,39] 2023-11-09T03:56:26.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 12r FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 13w FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 14w REG 8,1 9138594 268502 /var/log/ceph/ceph-mds.c.log 2023-11-09T03:56:26.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 15uW REG 0,24 7 2398 /run/ceph/ceph-mds.c.pid 2023-11-09T03:56:26.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 16r FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 17w FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 18u unix 0xffff959a60112ec0 0t0 252220 /var/run/ceph/ceph-mds.c.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 19u IPv4 251300 0t0 TCP *:6832 (LISTEN) 2023-11-09T03:56:26.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 20u IPv4 251334 0t0 TCP *:6833 (LISTEN) 2023-11-09T03:56:26.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 21r FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 22w FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 23r FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 24w FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 25u IPv4 247722 0t0 TCP smithi076.front.sepia.ceph.com:60028->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 26r FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 27w FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 28r FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 29w FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 30u IPv4 252224 0t0 TCP smithi076.front.sepia.ceph.com:59706->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 31u IPv4 250772 0t0 TCP smithi076.front.sepia.ceph.com:38604->smithi076.front.sepia.ceph.com:6816 (ESTABLISHED) 2023-11-09T03:56:26.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 32u IPv4 256430 0t0 TCP smithi076.front.sepia.ceph.com:33348->smithi076.front.sepia.ceph.com:6803 (ESTABLISHED) 2023-11-09T03:56:26.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 33u IPv4 249701 0t0 TCP smithi076.front.sepia.ceph.com:56048->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:26.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 34u IPv4 249702 0t0 TCP smithi076.front.sepia.ceph.com:48810->smithi076.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 35u IPv4 249703 0t0 TCP smithi076.front.sepia.ceph.com:37152->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.146 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 36u IPv4 256431 0t0 TCP smithi076.front.sepia.ceph.com:60920->smithi076.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 37u IPv4 257101 0t0 TCP smithi076.front.sepia.ceph.com:50856->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2023-11-09T03:56:26.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124534 log root 39u IPv4 272562 0t0 TCP smithi076.front.sepia.ceph.com:6832->192.168.0.1:35628 (ESTABLISHED) 2023-11-09T03:56:26.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.147 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.148 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.149 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.150 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.151 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.152 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.153 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.154 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.155 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 1w FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:26.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 2w FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:26.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 3r FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 4w FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,31,34,37] 2023-11-09T03:56:26.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 6r FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.156 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 7w FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 8u a_inode 0,14 0 2077 [eventpoll:9,25,32,35] 2023-11-09T03:56:26.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 9r FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 10w FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 11u a_inode 0,14 0 2077 [eventpoll:12,30,33,36,39] 2023-11-09T03:56:26.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 12r FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.157 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 13w FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 14w REG 8,1 9138594 268502 /var/log/ceph/ceph-mds.c.log 2023-11-09T03:56:26.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 15uW REG 0,24 7 2398 /run/ceph/ceph-mds.c.pid 2023-11-09T03:56:26.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 16r FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 17w FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.158 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 18u unix 0xffff959a60112ec0 0t0 252220 /var/run/ceph/ceph-mds.c.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 19u IPv4 251300 0t0 TCP *:6832 (LISTEN) 2023-11-09T03:56:26.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 20u IPv4 251334 0t0 TCP *:6833 (LISTEN) 2023-11-09T03:56:26.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 21r FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 22w FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 23r FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 24w FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 25u IPv4 247722 0t0 TCP smithi076.front.sepia.ceph.com:60028->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 26r FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 27w FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 28r FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 29w FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.160 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 30u IPv4 252224 0t0 TCP smithi076.front.sepia.ceph.com:59706->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 31u IPv4 250772 0t0 TCP smithi076.front.sepia.ceph.com:38604->smithi076.front.sepia.ceph.com:6816 (ESTABLISHED) 2023-11-09T03:56:26.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 32u IPv4 256430 0t0 TCP smithi076.front.sepia.ceph.com:33348->smithi076.front.sepia.ceph.com:6803 (ESTABLISHED) 2023-11-09T03:56:26.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 33u IPv4 249701 0t0 TCP smithi076.front.sepia.ceph.com:56048->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:26.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 34u IPv4 249702 0t0 TCP smithi076.front.sepia.ceph.com:48810->smithi076.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 35u IPv4 249703 0t0 TCP smithi076.front.sepia.ceph.com:37152->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.161 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 36u IPv4 256431 0t0 TCP smithi076.front.sepia.ceph.com:60920->smithi076.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 37u IPv4 257101 0t0 TCP smithi076.front.sepia.ceph.com:50856->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2023-11-09T03:56:26.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124536 msgr-work root 39u IPv4 272562 0t0 TCP smithi076.front.sepia.ceph.com:6832->192.168.0.1:35628 (ESTABLISHED) 2023-11-09T03:56:26.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.162 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.163 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.164 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.165 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.166 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.167 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.168 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.169 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.170 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 1w FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:26.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 2w FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:26.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 3r FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 4w FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,31,34,37] 2023-11-09T03:56:26.171 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 6r FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 7w FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 8u a_inode 0,14 0 2077 [eventpoll:9,25,32,35] 2023-11-09T03:56:26.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 9r FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 10w FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 11u a_inode 0,14 0 2077 [eventpoll:12,30,33,36,39] 2023-11-09T03:56:26.172 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 12r FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 13w FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 14w REG 8,1 9138594 268502 /var/log/ceph/ceph-mds.c.log 2023-11-09T03:56:26.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 15uW REG 0,24 7 2398 /run/ceph/ceph-mds.c.pid 2023-11-09T03:56:26.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 16r FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 17w FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.173 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 18u unix 0xffff959a60112ec0 0t0 252220 /var/run/ceph/ceph-mds.c.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 19u IPv4 251300 0t0 TCP *:6832 (LISTEN) 2023-11-09T03:56:26.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 20u IPv4 251334 0t0 TCP *:6833 (LISTEN) 2023-11-09T03:56:26.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 21r FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 22w FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.174 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 23r FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 24w FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 25u IPv4 247722 0t0 TCP smithi076.front.sepia.ceph.com:60028->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 26r FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 27w FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 28r FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.175 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 29w FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 30u IPv4 252224 0t0 TCP smithi076.front.sepia.ceph.com:59706->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 31u IPv4 250772 0t0 TCP smithi076.front.sepia.ceph.com:38604->smithi076.front.sepia.ceph.com:6816 (ESTABLISHED) 2023-11-09T03:56:26.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 32u IPv4 256430 0t0 TCP smithi076.front.sepia.ceph.com:33348->smithi076.front.sepia.ceph.com:6803 (ESTABLISHED) 2023-11-09T03:56:26.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 33u IPv4 249701 0t0 TCP smithi076.front.sepia.ceph.com:56048->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:26.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 34u IPv4 249702 0t0 TCP smithi076.front.sepia.ceph.com:48810->smithi076.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.176 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 35u IPv4 249703 0t0 TCP smithi076.front.sepia.ceph.com:37152->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 36u IPv4 256431 0t0 TCP smithi076.front.sepia.ceph.com:60920->smithi076.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 37u IPv4 257101 0t0 TCP smithi076.front.sepia.ceph.com:50856->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2023-11-09T03:56:26.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124537 msgr-work root 39u IPv4 272562 0t0 TCP smithi076.front.sepia.ceph.com:6832->192.168.0.1:35628 (ESTABLISHED) 2023-11-09T03:56:26.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.177 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.178 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.179 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.180 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.181 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.182 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.183 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.184 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.185 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 1w FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:26.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 2w FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:26.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 3r FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.186 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 4w FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,31,34,37] 2023-11-09T03:56:26.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 6r FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 7w FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 8u a_inode 0,14 0 2077 [eventpoll:9,25,32,35] 2023-11-09T03:56:26.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 9r FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.187 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 10w FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 11u a_inode 0,14 0 2077 [eventpoll:12,30,33,36,39] 2023-11-09T03:56:26.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 12r FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 13w FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 14w REG 8,1 9138594 268502 /var/log/ceph/ceph-mds.c.log 2023-11-09T03:56:26.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 15uW REG 0,24 7 2398 /run/ceph/ceph-mds.c.pid 2023-11-09T03:56:26.188 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 16r FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 17w FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 18u unix 0xffff959a60112ec0 0t0 252220 /var/run/ceph/ceph-mds.c.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 19u IPv4 251300 0t0 TCP *:6832 (LISTEN) 2023-11-09T03:56:26.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 20u IPv4 251334 0t0 TCP *:6833 (LISTEN) 2023-11-09T03:56:26.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 21r FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.189 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 22w FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 23r FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 24w FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 25u IPv4 247722 0t0 TCP smithi076.front.sepia.ceph.com:60028->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 26r FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 27w FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.190 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 28r FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 29w FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 30u IPv4 252224 0t0 TCP smithi076.front.sepia.ceph.com:59706->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 31u IPv4 250772 0t0 TCP smithi076.front.sepia.ceph.com:38604->smithi076.front.sepia.ceph.com:6816 (ESTABLISHED) 2023-11-09T03:56:26.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 32u IPv4 256430 0t0 TCP smithi076.front.sepia.ceph.com:33348->smithi076.front.sepia.ceph.com:6803 (ESTABLISHED) 2023-11-09T03:56:26.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 33u IPv4 249701 0t0 TCP smithi076.front.sepia.ceph.com:56048->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:26.191 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 34u IPv4 249702 0t0 TCP smithi076.front.sepia.ceph.com:48810->smithi076.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 35u IPv4 249703 0t0 TCP smithi076.front.sepia.ceph.com:37152->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 36u IPv4 256431 0t0 TCP smithi076.front.sepia.ceph.com:60920->smithi076.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 37u IPv4 257101 0t0 TCP smithi076.front.sepia.ceph.com:50856->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2023-11-09T03:56:26.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124538 msgr-work root 39u IPv4 272562 0t0 TCP smithi076.front.sepia.ceph.com:6832->192.168.0.1:35628 (ESTABLISHED) 2023-11-09T03:56:26.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.192 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.193 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.194 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.195 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.196 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.197 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.198 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.199 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.200 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 1w FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:26.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 2w FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:26.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 3r FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 4w FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,31,34,37] 2023-11-09T03:56:26.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 6r FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 7w FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 8u a_inode 0,14 0 2077 [eventpoll:9,25,32,35] 2023-11-09T03:56:26.202 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 9r FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 10w FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 11u a_inode 0,14 0 2077 [eventpoll:12,30,33,36,39] 2023-11-09T03:56:26.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 12r FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 13w FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.203 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 14w REG 8,1 9138594 268502 /var/log/ceph/ceph-mds.c.log 2023-11-09T03:56:26.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 15uW REG 0,24 7 2398 /run/ceph/ceph-mds.c.pid 2023-11-09T03:56:26.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 16r FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 17w FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 18u unix 0xffff959a60112ec0 0t0 252220 /var/run/ceph/ceph-mds.c.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.204 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 19u IPv4 251300 0t0 TCP *:6832 (LISTEN) 2023-11-09T03:56:26.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 20u IPv4 251334 0t0 TCP *:6833 (LISTEN) 2023-11-09T03:56:26.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 21r FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 22w FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 23r FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 24w FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.205 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 25u IPv4 247722 0t0 TCP smithi076.front.sepia.ceph.com:60028->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 26r FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 27w FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 28r FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 29w FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.206 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 30u IPv4 252224 0t0 TCP smithi076.front.sepia.ceph.com:59706->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 31u IPv4 250772 0t0 TCP smithi076.front.sepia.ceph.com:38604->smithi076.front.sepia.ceph.com:6816 (ESTABLISHED) 2023-11-09T03:56:26.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 32u IPv4 256430 0t0 TCP smithi076.front.sepia.ceph.com:33348->smithi076.front.sepia.ceph.com:6803 (ESTABLISHED) 2023-11-09T03:56:26.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 33u IPv4 249701 0t0 TCP smithi076.front.sepia.ceph.com:56048->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:26.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 34u IPv4 249702 0t0 TCP smithi076.front.sepia.ceph.com:48810->smithi076.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 35u IPv4 249703 0t0 TCP smithi076.front.sepia.ceph.com:37152->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.207 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 36u IPv4 256431 0t0 TCP smithi076.front.sepia.ceph.com:60920->smithi076.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 37u IPv4 257101 0t0 TCP smithi076.front.sepia.ceph.com:50856->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2023-11-09T03:56:26.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124542 service root 39u IPv4 272562 0t0 TCP smithi076.front.sepia.ceph.com:6832->192.168.0.1:35628 (ESTABLISHED) 2023-11-09T03:56:26.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.208 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.209 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.210 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.211 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.212 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.214 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.215 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.216 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 1w FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:26.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 2w FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:26.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 3r FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.217 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 4w FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,31,34,37] 2023-11-09T03:56:26.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 6r FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 7w FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 8u a_inode 0,14 0 2077 [eventpoll:9,25,32,35] 2023-11-09T03:56:26.218 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 9r FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 10w FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 11u a_inode 0,14 0 2077 [eventpoll:12,30,33,36,39] 2023-11-09T03:56:26.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 12r FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 13w FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 14w REG 8,1 9138594 268502 /var/log/ceph/ceph-mds.c.log 2023-11-09T03:56:26.219 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 15uW REG 0,24 7 2398 /run/ceph/ceph-mds.c.pid 2023-11-09T03:56:26.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 16r FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 17w FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 18u unix 0xffff959a60112ec0 0t0 252220 /var/run/ceph/ceph-mds.c.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 19u IPv4 251300 0t0 TCP *:6832 (LISTEN) 2023-11-09T03:56:26.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 20u IPv4 251334 0t0 TCP *:6833 (LISTEN) 2023-11-09T03:56:26.220 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 21r FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 22w FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 23r FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 24w FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 25u IPv4 247722 0t0 TCP smithi076.front.sepia.ceph.com:60028->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 26r FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 27w FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 28r FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 29w FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 30u IPv4 252224 0t0 TCP smithi076.front.sepia.ceph.com:59706->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 31u IPv4 250772 0t0 TCP smithi076.front.sepia.ceph.com:38604->smithi076.front.sepia.ceph.com:6816 (ESTABLISHED) 2023-11-09T03:56:26.222 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 32u IPv4 256430 0t0 TCP smithi076.front.sepia.ceph.com:33348->smithi076.front.sepia.ceph.com:6803 (ESTABLISHED) 2023-11-09T03:56:26.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 33u IPv4 249701 0t0 TCP smithi076.front.sepia.ceph.com:56048->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:26.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 34u IPv4 249702 0t0 TCP smithi076.front.sepia.ceph.com:48810->smithi076.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 35u IPv4 249703 0t0 TCP smithi076.front.sepia.ceph.com:37152->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 36u IPv4 256431 0t0 TCP smithi076.front.sepia.ceph.com:60920->smithi076.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 37u IPv4 257101 0t0 TCP smithi076.front.sepia.ceph.com:50856->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2023-11-09T03:56:26.223 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124543 admin_soc root 39u IPv4 272562 0t0 TCP smithi076.front.sepia.ceph.com:6832->192.168.0.1:35628 (ESTABLISHED) 2023-11-09T03:56:26.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.224 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.226 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.227 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.228 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.229 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.230 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.231 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 1w FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:26.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 2w FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:26.232 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 3r FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 4w FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,31,34,37] 2023-11-09T03:56:26.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 6r FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 7w FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 8u a_inode 0,14 0 2077 [eventpoll:9,25,32,35] 2023-11-09T03:56:26.233 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 9r FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 10w FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 11u a_inode 0,14 0 2077 [eventpoll:12,30,33,36,39] 2023-11-09T03:56:26.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 12r FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 13w FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.234 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 14w REG 8,1 9138594 268502 /var/log/ceph/ceph-mds.c.log 2023-11-09T03:56:26.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 15uW REG 0,24 7 2398 /run/ceph/ceph-mds.c.pid 2023-11-09T03:56:26.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 16r FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 17w FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 18u unix 0xffff959a60112ec0 0t0 252220 /var/run/ceph/ceph-mds.c.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 19u IPv4 251300 0t0 TCP *:6832 (LISTEN) 2023-11-09T03:56:26.235 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 20u IPv4 251334 0t0 TCP *:6833 (LISTEN) 2023-11-09T03:56:26.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 21r FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 22w FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 23r FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 24w FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 25u IPv4 247722 0t0 TCP smithi076.front.sepia.ceph.com:60028->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.236 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 26r FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 27w FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 28r FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 29w FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 30u IPv4 252224 0t0 TCP smithi076.front.sepia.ceph.com:59706->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 31u IPv4 250772 0t0 TCP smithi076.front.sepia.ceph.com:38604->smithi076.front.sepia.ceph.com:6816 (ESTABLISHED) 2023-11-09T03:56:26.237 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 32u IPv4 256430 0t0 TCP smithi076.front.sepia.ceph.com:33348->smithi076.front.sepia.ceph.com:6803 (ESTABLISHED) 2023-11-09T03:56:26.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 33u IPv4 249701 0t0 TCP smithi076.front.sepia.ceph.com:56048->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:26.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 34u IPv4 249702 0t0 TCP smithi076.front.sepia.ceph.com:48810->smithi076.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 35u IPv4 249703 0t0 TCP smithi076.front.sepia.ceph.com:37152->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 36u IPv4 256431 0t0 TCP smithi076.front.sepia.ceph.com:60920->smithi076.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.238 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 37u IPv4 257101 0t0 TCP smithi076.front.sepia.ceph.com:50856->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2023-11-09T03:56:26.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124544 signal_ha root 39u IPv4 272562 0t0 TCP smithi076.front.sepia.ceph.com:6832->192.168.0.1:35628 (ESTABLISHED) 2023-11-09T03:56:26.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.239 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.240 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.241 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.242 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.243 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.244 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.245 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.247 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.248 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 1w FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:26.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 2w FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:26.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 3r FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 4w FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.249 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,31,34,37] 2023-11-09T03:56:26.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 6r FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 7w FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 8u a_inode 0,14 0 2077 [eventpoll:9,25,32,35] 2023-11-09T03:56:26.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 9r FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 10w FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.250 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 11u a_inode 0,14 0 2077 [eventpoll:12,30,33,36,39] 2023-11-09T03:56:26.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 12r FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 13w FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 14w REG 8,1 9138594 268502 /var/log/ceph/ceph-mds.c.log 2023-11-09T03:56:26.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 15uW REG 0,24 7 2398 /run/ceph/ceph-mds.c.pid 2023-11-09T03:56:26.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 16r FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.251 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 17w FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 18u unix 0xffff959a60112ec0 0t0 252220 /var/run/ceph/ceph-mds.c.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 19u IPv4 251300 0t0 TCP *:6832 (LISTEN) 2023-11-09T03:56:26.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 20u IPv4 251334 0t0 TCP *:6833 (LISTEN) 2023-11-09T03:56:26.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 21r FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 22w FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.252 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 23r FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 24w FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 25u IPv4 247722 0t0 TCP smithi076.front.sepia.ceph.com:60028->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 26r FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 27w FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.253 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 28r FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 29w FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 30u IPv4 252224 0t0 TCP smithi076.front.sepia.ceph.com:59706->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 31u IPv4 250772 0t0 TCP smithi076.front.sepia.ceph.com:38604->smithi076.front.sepia.ceph.com:6816 (ESTABLISHED) 2023-11-09T03:56:26.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 32u IPv4 256430 0t0 TCP smithi076.front.sepia.ceph.com:33348->smithi076.front.sepia.ceph.com:6803 (ESTABLISHED) 2023-11-09T03:56:26.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 33u IPv4 249701 0t0 TCP smithi076.front.sepia.ceph.com:56048->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:26.254 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 34u IPv4 249702 0t0 TCP smithi076.front.sepia.ceph.com:48810->smithi076.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 35u IPv4 249703 0t0 TCP smithi076.front.sepia.ceph.com:37152->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 36u IPv4 256431 0t0 TCP smithi076.front.sepia.ceph.com:60920->smithi076.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 37u IPv4 257101 0t0 TCP smithi076.front.sepia.ceph.com:50856->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2023-11-09T03:56:26.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124545 io_contex root 39u IPv4 272562 0t0 TCP smithi076.front.sepia.ceph.com:6832->192.168.0.1:35628 (ESTABLISHED) 2023-11-09T03:56:26.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.255 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.256 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.257 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.258 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.260 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.261 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.262 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.263 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 1w FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:26.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 2w FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:26.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 3r FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.264 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 4w FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,31,34,37] 2023-11-09T03:56:26.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 6r FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 7w FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 8u a_inode 0,14 0 2077 [eventpoll:9,25,32,35] 2023-11-09T03:56:26.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 9r FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.265 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 10w FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 11u a_inode 0,14 0 2077 [eventpoll:12,30,33,36,39] 2023-11-09T03:56:26.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 12r FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 13w FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 14w REG 8,1 9138594 268502 /var/log/ceph/ceph-mds.c.log 2023-11-09T03:56:26.266 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 15uW REG 0,24 7 2398 /run/ceph/ceph-mds.c.pid 2023-11-09T03:56:26.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 16r FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 17w FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 18u unix 0xffff959a60112ec0 0t0 252220 /var/run/ceph/ceph-mds.c.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 19u IPv4 251300 0t0 TCP *:6832 (LISTEN) 2023-11-09T03:56:26.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 20u IPv4 251334 0t0 TCP *:6833 (LISTEN) 2023-11-09T03:56:26.267 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 21r FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 22w FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 23r FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 24w FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 25u IPv4 247722 0t0 TCP smithi076.front.sepia.ceph.com:60028->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.268 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 26r FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 27w FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 28r FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 29w FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 30u IPv4 252224 0t0 TCP smithi076.front.sepia.ceph.com:59706->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 31u IPv4 250772 0t0 TCP smithi076.front.sepia.ceph.com:38604->smithi076.front.sepia.ceph.com:6816 (ESTABLISHED) 2023-11-09T03:56:26.269 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 32u IPv4 256430 0t0 TCP smithi076.front.sepia.ceph.com:33348->smithi076.front.sepia.ceph.com:6803 (ESTABLISHED) 2023-11-09T03:56:26.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 33u IPv4 249701 0t0 TCP smithi076.front.sepia.ceph.com:56048->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:26.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 34u IPv4 249702 0t0 TCP smithi076.front.sepia.ceph.com:48810->smithi076.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 35u IPv4 249703 0t0 TCP smithi076.front.sepia.ceph.com:37152->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 36u IPv4 256431 0t0 TCP smithi076.front.sepia.ceph.com:60920->smithi076.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 37u IPv4 257101 0t0 TCP smithi076.front.sepia.ceph.com:50856->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2023-11-09T03:56:26.270 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124546 io_contex root 39u IPv4 272562 0t0 TCP smithi076.front.sepia.ceph.com:6832->192.168.0.1:35628 (ESTABLISHED) 2023-11-09T03:56:26.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.271 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.272 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.273 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.274 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.275 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.276 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.277 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.278 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.279 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 1w FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:26.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 2w FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:26.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 3r FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 4w FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,31,34,37] 2023-11-09T03:56:26.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 6r FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.280 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 7w FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,25,32,35] 2023-11-09T03:56:26.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 9r FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 10w FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,30,33,36,39] 2023-11-09T03:56:26.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 12r FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 13w FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 14w REG 8,1 9138594 268502 /var/log/ceph/ceph-mds.c.log 2023-11-09T03:56:26.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 15uW REG 0,24 7 2398 /run/ceph/ceph-mds.c.pid 2023-11-09T03:56:26.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 16r FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.282 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 17w FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 18u unix 0xffff959a60112ec0 0t0 252220 /var/run/ceph/ceph-mds.c.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 19u IPv4 251300 0t0 TCP *:6832 (LISTEN) 2023-11-09T03:56:26.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 20u IPv4 251334 0t0 TCP *:6833 (LISTEN) 2023-11-09T03:56:26.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 21r FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 22w FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.283 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 23r FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 24w FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 25u IPv4 247722 0t0 TCP smithi076.front.sepia.ceph.com:60028->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 26r FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 27w FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.284 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 28r FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 29w FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 30u IPv4 252224 0t0 TCP smithi076.front.sepia.ceph.com:59706->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 31u IPv4 250772 0t0 TCP smithi076.front.sepia.ceph.com:38604->smithi076.front.sepia.ceph.com:6816 (ESTABLISHED) 2023-11-09T03:56:26.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 32u IPv4 256430 0t0 TCP smithi076.front.sepia.ceph.com:33348->smithi076.front.sepia.ceph.com:6803 (ESTABLISHED) 2023-11-09T03:56:26.285 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 33u IPv4 249701 0t0 TCP smithi076.front.sepia.ceph.com:56048->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:26.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 34u IPv4 249702 0t0 TCP smithi076.front.sepia.ceph.com:48810->smithi076.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 35u IPv4 249703 0t0 TCP smithi076.front.sepia.ceph.com:37152->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 36u IPv4 256431 0t0 TCP smithi076.front.sepia.ceph.com:60920->smithi076.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 37u IPv4 257101 0t0 TCP smithi076.front.sepia.ceph.com:50856->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2023-11-09T03:56:26.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124547 ms_dispat root 39u IPv4 272562 0t0 TCP smithi076.front.sepia.ceph.com:6832->192.168.0.1:35628 (ESTABLISHED) 2023-11-09T03:56:26.286 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.287 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.288 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.289 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.290 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.291 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.292 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.293 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.294 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 1w FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:26.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 2w FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:26.295 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 3r FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 4w FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,31,34,37] 2023-11-09T03:56:26.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 6r FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 7w FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.296 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,25,32,35] 2023-11-09T03:56:26.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 9r FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 10w FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,30,33,36,39] 2023-11-09T03:56:26.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 12r FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.297 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 13w FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 14w REG 8,1 9138594 268502 /var/log/ceph/ceph-mds.c.log 2023-11-09T03:56:26.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 15uW REG 0,24 7 2398 /run/ceph/ceph-mds.c.pid 2023-11-09T03:56:26.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 16r FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 17w FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.298 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 18u unix 0xffff959a60112ec0 0t0 252220 /var/run/ceph/ceph-mds.c.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 19u IPv4 251300 0t0 TCP *:6832 (LISTEN) 2023-11-09T03:56:26.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 20u IPv4 251334 0t0 TCP *:6833 (LISTEN) 2023-11-09T03:56:26.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 21r FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 22w FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.299 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 23r FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 24w FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 25u IPv4 247722 0t0 TCP smithi076.front.sepia.ceph.com:60028->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 26r FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 27w FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 28r FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.300 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 29w FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 30u IPv4 252224 0t0 TCP smithi076.front.sepia.ceph.com:59706->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 31u IPv4 250772 0t0 TCP smithi076.front.sepia.ceph.com:38604->smithi076.front.sepia.ceph.com:6816 (ESTABLISHED) 2023-11-09T03:56:26.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 32u IPv4 256430 0t0 TCP smithi076.front.sepia.ceph.com:33348->smithi076.front.sepia.ceph.com:6803 (ESTABLISHED) 2023-11-09T03:56:26.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 33u IPv4 249701 0t0 TCP smithi076.front.sepia.ceph.com:56048->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:26.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 34u IPv4 249702 0t0 TCP smithi076.front.sepia.ceph.com:48810->smithi076.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.301 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 35u IPv4 249703 0t0 TCP smithi076.front.sepia.ceph.com:37152->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 36u IPv4 256431 0t0 TCP smithi076.front.sepia.ceph.com:60920->smithi076.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 37u IPv4 257101 0t0 TCP smithi076.front.sepia.ceph.com:50856->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2023-11-09T03:56:26.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124548 ms_local root 39u IPv4 272562 0t0 TCP smithi076.front.sepia.ceph.com:6832->192.168.0.1:35628 (ESTABLISHED) 2023-11-09T03:56:26.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.302 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.303 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.304 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.305 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.306 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.307 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.308 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.309 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.310 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 1w FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:26.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 2w FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:26.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 3r FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 4w FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,31,34,37] 2023-11-09T03:56:26.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 6r FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 7w FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,25,32,35] 2023-11-09T03:56:26.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 9r FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 10w FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.312 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,30,33,36,39] 2023-11-09T03:56:26.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 12r FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 13w FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 14w REG 8,1 9138594 268502 /var/log/ceph/ceph-mds.c.log 2023-11-09T03:56:26.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 15uW REG 0,24 7 2398 /run/ceph/ceph-mds.c.pid 2023-11-09T03:56:26.313 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 16r FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 17w FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 18u unix 0xffff959a60112ec0 0t0 252220 /var/run/ceph/ceph-mds.c.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 19u IPv4 251300 0t0 TCP *:6832 (LISTEN) 2023-11-09T03:56:26.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 20u IPv4 251334 0t0 TCP *:6833 (LISTEN) 2023-11-09T03:56:26.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 21r FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.314 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 22w FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 23r FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 24w FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 25u IPv4 247722 0t0 TCP smithi076.front.sepia.ceph.com:60028->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 26r FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.315 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 27w FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 28r FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 29w FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 30u IPv4 252224 0t0 TCP smithi076.front.sepia.ceph.com:59706->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 31u IPv4 250772 0t0 TCP smithi076.front.sepia.ceph.com:38604->smithi076.front.sepia.ceph.com:6816 (ESTABLISHED) 2023-11-09T03:56:26.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 32u IPv4 256430 0t0 TCP smithi076.front.sepia.ceph.com:33348->smithi076.front.sepia.ceph.com:6803 (ESTABLISHED) 2023-11-09T03:56:26.316 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 33u IPv4 249701 0t0 TCP smithi076.front.sepia.ceph.com:56048->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:26.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 34u IPv4 249702 0t0 TCP smithi076.front.sepia.ceph.com:48810->smithi076.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 35u IPv4 249703 0t0 TCP smithi076.front.sepia.ceph.com:37152->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 36u IPv4 256431 0t0 TCP smithi076.front.sepia.ceph.com:60920->smithi076.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 37u IPv4 257101 0t0 TCP smithi076.front.sepia.ceph.com:50856->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2023-11-09T03:56:26.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124549 safe_time root 39u IPv4 272562 0t0 TCP smithi076.front.sepia.ceph.com:6832->192.168.0.1:35628 (ESTABLISHED) 2023-11-09T03:56:26.317 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.318 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.319 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.320 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.321 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.322 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.323 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.324 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.325 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 1w FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:26.326 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 2w FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:26.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 3r FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 4w FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,31,34,37] 2023-11-09T03:56:26.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 6r FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 7w FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.327 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,25,32,35] 2023-11-09T03:56:26.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 9r FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 10w FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,30,33,36,39] 2023-11-09T03:56:26.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 12r FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 13w FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.328 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 14w REG 8,1 9138594 268502 /var/log/ceph/ceph-mds.c.log 2023-11-09T03:56:26.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 15uW REG 0,24 7 2398 /run/ceph/ceph-mds.c.pid 2023-11-09T03:56:26.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 16r FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 17w FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 18u unix 0xffff959a60112ec0 0t0 252220 /var/run/ceph/ceph-mds.c.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 19u IPv4 251300 0t0 TCP *:6832 (LISTEN) 2023-11-09T03:56:26.329 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 20u IPv4 251334 0t0 TCP *:6833 (LISTEN) 2023-11-09T03:56:26.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 21r FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 22w FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 23r FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 24w FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 25u IPv4 247722 0t0 TCP smithi076.front.sepia.ceph.com:60028->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.330 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 26r FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 27w FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 28r FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 29w FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 30u IPv4 252224 0t0 TCP smithi076.front.sepia.ceph.com:59706->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 31u IPv4 250772 0t0 TCP smithi076.front.sepia.ceph.com:38604->smithi076.front.sepia.ceph.com:6816 (ESTABLISHED) 2023-11-09T03:56:26.331 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 32u IPv4 256430 0t0 TCP smithi076.front.sepia.ceph.com:33348->smithi076.front.sepia.ceph.com:6803 (ESTABLISHED) 2023-11-09T03:56:26.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 33u IPv4 249701 0t0 TCP smithi076.front.sepia.ceph.com:56048->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:26.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 34u IPv4 249702 0t0 TCP smithi076.front.sepia.ceph.com:48810->smithi076.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 35u IPv4 249703 0t0 TCP smithi076.front.sepia.ceph.com:37152->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 36u IPv4 256431 0t0 TCP smithi076.front.sepia.ceph.com:60920->smithi076.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 37u IPv4 257101 0t0 TCP smithi076.front.sepia.ceph.com:50856->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2023-11-09T03:56:26.332 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124550 safe_time root 39u IPv4 272562 0t0 TCP smithi076.front.sepia.ceph.com:6832->192.168.0.1:35628 (ESTABLISHED) 2023-11-09T03:56:26.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.333 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.334 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.335 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.336 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.338 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.339 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.340 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.341 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 1w FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:26.342 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 2w FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:26.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 3r FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 4w FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,31,34,37] 2023-11-09T03:56:26.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 6r FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 7w FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.343 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 8u a_inode 0,14 0 2077 [eventpoll:9,25,32,35] 2023-11-09T03:56:26.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 9r FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 10w FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 11u a_inode 0,14 0 2077 [eventpoll:12,30,33,36,39] 2023-11-09T03:56:26.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 12r FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.344 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 13w FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 14w REG 8,1 9138594 268502 /var/log/ceph/ceph-mds.c.log 2023-11-09T03:56:26.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 15uW REG 0,24 7 2398 /run/ceph/ceph-mds.c.pid 2023-11-09T03:56:26.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 16r FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 17w FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.345 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 18u unix 0xffff959a60112ec0 0t0 252220 /var/run/ceph/ceph-mds.c.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 19u IPv4 251300 0t0 TCP *:6832 (LISTEN) 2023-11-09T03:56:26.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 20u IPv4 251334 0t0 TCP *:6833 (LISTEN) 2023-11-09T03:56:26.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 21r FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 22w FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.346 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 23r FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 24w FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 25u IPv4 247722 0t0 TCP smithi076.front.sepia.ceph.com:60028->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 26r FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 27w FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 28r FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.347 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 29w FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 30u IPv4 252224 0t0 TCP smithi076.front.sepia.ceph.com:59706->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 31u IPv4 250772 0t0 TCP smithi076.front.sepia.ceph.com:38604->smithi076.front.sepia.ceph.com:6816 (ESTABLISHED) 2023-11-09T03:56:26.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 32u IPv4 256430 0t0 TCP smithi076.front.sepia.ceph.com:33348->smithi076.front.sepia.ceph.com:6803 (ESTABLISHED) 2023-11-09T03:56:26.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 33u IPv4 249701 0t0 TCP smithi076.front.sepia.ceph.com:56048->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:26.348 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 34u IPv4 249702 0t0 TCP smithi076.front.sepia.ceph.com:48810->smithi076.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 35u IPv4 249703 0t0 TCP smithi076.front.sepia.ceph.com:37152->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 36u IPv4 256431 0t0 TCP smithi076.front.sepia.ceph.com:60920->smithi076.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 37u IPv4 257101 0t0 TCP smithi076.front.sepia.ceph.com:50856->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2023-11-09T03:56:26.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124551 ceph-mds root 39u IPv4 272562 0t0 TCP smithi076.front.sepia.ceph.com:6832->192.168.0.1:35628 (ESTABLISHED) 2023-11-09T03:56:26.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.349 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.350 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.351 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.352 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.353 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.354 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.355 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.356 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.357 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 1w FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:26.358 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 2w FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:26.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 3r FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 4w FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,31,34,37] 2023-11-09T03:56:26.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 6r FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 7w FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.359 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,25,32,35] 2023-11-09T03:56:26.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 9r FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 10w FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,30,33,36,39] 2023-11-09T03:56:26.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 12r FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.360 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 13w FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 14w REG 8,1 9138594 268502 /var/log/ceph/ceph-mds.c.log 2023-11-09T03:56:26.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 15uW REG 0,24 7 2398 /run/ceph/ceph-mds.c.pid 2023-11-09T03:56:26.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 16r FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 17w FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 18u unix 0xffff959a60112ec0 0t0 252220 /var/run/ceph/ceph-mds.c.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.361 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 19u IPv4 251300 0t0 TCP *:6832 (LISTEN) 2023-11-09T03:56:26.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 20u IPv4 251334 0t0 TCP *:6833 (LISTEN) 2023-11-09T03:56:26.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 21r FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 22w FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 23r FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.362 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 24w FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 25u IPv4 247722 0t0 TCP smithi076.front.sepia.ceph.com:60028->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 26r FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 27w FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 28r FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 29w FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.363 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 30u IPv4 252224 0t0 TCP smithi076.front.sepia.ceph.com:59706->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 31u IPv4 250772 0t0 TCP smithi076.front.sepia.ceph.com:38604->smithi076.front.sepia.ceph.com:6816 (ESTABLISHED) 2023-11-09T03:56:26.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 32u IPv4 256430 0t0 TCP smithi076.front.sepia.ceph.com:33348->smithi076.front.sepia.ceph.com:6803 (ESTABLISHED) 2023-11-09T03:56:26.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 33u IPv4 249701 0t0 TCP smithi076.front.sepia.ceph.com:56048->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:26.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 34u IPv4 249702 0t0 TCP smithi076.front.sepia.ceph.com:48810->smithi076.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 35u IPv4 249703 0t0 TCP smithi076.front.sepia.ceph.com:37152->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.364 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 36u IPv4 256431 0t0 TCP smithi076.front.sepia.ceph.com:60920->smithi076.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 37u IPv4 257101 0t0 TCP smithi076.front.sepia.ceph.com:50856->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2023-11-09T03:56:26.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 124574 safe_time root 39u IPv4 272562 0t0 TCP smithi076.front.sepia.ceph.com:6832->192.168.0.1:35628 (ESTABLISHED) 2023-11-09T03:56:26.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.365 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.366 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.367 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.368 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.369 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.370 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.371 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.372 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.373 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 1w FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:26.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 2w FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:26.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 3r FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 4w FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,31,34,37] 2023-11-09T03:56:26.374 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 6r FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 7w FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 8u a_inode 0,14 0 2077 [eventpoll:9,25,32,35] 2023-11-09T03:56:26.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 9r FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 10w FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 11u a_inode 0,14 0 2077 [eventpoll:12,30,33,36,39] 2023-11-09T03:56:26.375 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 12r FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 13w FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 14w REG 8,1 9138594 268502 /var/log/ceph/ceph-mds.c.log 2023-11-09T03:56:26.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 15uW REG 0,24 7 2398 /run/ceph/ceph-mds.c.pid 2023-11-09T03:56:26.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 16r FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 17w FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.376 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 18u unix 0xffff959a60112ec0 0t0 252220 /var/run/ceph/ceph-mds.c.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 19u IPv4 251300 0t0 TCP *:6832 (LISTEN) 2023-11-09T03:56:26.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 20u IPv4 251334 0t0 TCP *:6833 (LISTEN) 2023-11-09T03:56:26.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 21r FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 22w FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 23r FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.377 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 24w FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 25u IPv4 247722 0t0 TCP smithi076.front.sepia.ceph.com:60028->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 26r FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 27w FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 28r FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 29w FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.378 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 30u IPv4 252224 0t0 TCP smithi076.front.sepia.ceph.com:59706->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 31u IPv4 250772 0t0 TCP smithi076.front.sepia.ceph.com:38604->smithi076.front.sepia.ceph.com:6816 (ESTABLISHED) 2023-11-09T03:56:26.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 32u IPv4 256430 0t0 TCP smithi076.front.sepia.ceph.com:33348->smithi076.front.sepia.ceph.com:6803 (ESTABLISHED) 2023-11-09T03:56:26.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 33u IPv4 249701 0t0 TCP smithi076.front.sepia.ceph.com:56048->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:26.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 34u IPv4 249702 0t0 TCP smithi076.front.sepia.ceph.com:48810->smithi076.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 35u IPv4 249703 0t0 TCP smithi076.front.sepia.ceph.com:37152->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.379 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 36u IPv4 256431 0t0 TCP smithi076.front.sepia.ceph.com:60920->smithi076.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 37u IPv4 257101 0t0 TCP smithi076.front.sepia.ceph.com:50856->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2023-11-09T03:56:26.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126453 ceph_time root 39u IPv4 272562 0t0 TCP smithi076.front.sepia.ceph.com:6832->192.168.0.1:35628 (ESTABLISHED) 2023-11-09T03:56:26.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.380 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.382 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.383 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.384 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.385 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.386 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.387 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.388 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 1w FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:26.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 2w FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:26.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 3r FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 4w FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,31,34,37] 2023-11-09T03:56:26.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 6r FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.389 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 7w FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 8u a_inode 0,14 0 2077 [eventpoll:9,25,32,35] 2023-11-09T03:56:26.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 9r FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 10w FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 11u a_inode 0,14 0 2077 [eventpoll:12,30,33,36,39] 2023-11-09T03:56:26.390 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 12r FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 13w FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 14w REG 8,1 9138594 268502 /var/log/ceph/ceph-mds.c.log 2023-11-09T03:56:26.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 15uW REG 0,24 7 2398 /run/ceph/ceph-mds.c.pid 2023-11-09T03:56:26.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 16r FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 17w FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.391 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 18u unix 0xffff959a60112ec0 0t0 252220 /var/run/ceph/ceph-mds.c.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 19u IPv4 251300 0t0 TCP *:6832 (LISTEN) 2023-11-09T03:56:26.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 20u IPv4 251334 0t0 TCP *:6833 (LISTEN) 2023-11-09T03:56:26.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 21r FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 22w FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 23r FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.392 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 24w FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 25u IPv4 247722 0t0 TCP smithi076.front.sepia.ceph.com:60028->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 26r FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 27w FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 28r FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 29w FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.393 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 30u IPv4 252224 0t0 TCP smithi076.front.sepia.ceph.com:59706->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 31u IPv4 250772 0t0 TCP smithi076.front.sepia.ceph.com:38604->smithi076.front.sepia.ceph.com:6816 (ESTABLISHED) 2023-11-09T03:56:26.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 32u IPv4 256430 0t0 TCP smithi076.front.sepia.ceph.com:33348->smithi076.front.sepia.ceph.com:6803 (ESTABLISHED) 2023-11-09T03:56:26.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 33u IPv4 249701 0t0 TCP smithi076.front.sepia.ceph.com:56048->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:26.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 34u IPv4 249702 0t0 TCP smithi076.front.sepia.ceph.com:48810->smithi076.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 35u IPv4 249703 0t0 TCP smithi076.front.sepia.ceph.com:37152->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.394 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 36u IPv4 256431 0t0 TCP smithi076.front.sepia.ceph.com:60920->smithi076.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 37u IPv4 257101 0t0 TCP smithi076.front.sepia.ceph.com:50856->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2023-11-09T03:56:26.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126454 OpHistory root 39u IPv4 272562 0t0 TCP smithi076.front.sepia.ceph.com:6832->192.168.0.1:35628 (ESTABLISHED) 2023-11-09T03:56:26.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.395 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.396 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.397 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.398 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.399 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.400 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.401 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.402 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.403 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 1w FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:26.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 2w FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:26.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 3r FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 4w FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,31,34,37] 2023-11-09T03:56:26.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 6r FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.404 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 7w FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,25,32,35] 2023-11-09T03:56:26.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 9r FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 10w FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,30,33,36,39] 2023-11-09T03:56:26.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 12r FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.405 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 13w FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 14w REG 8,1 9138594 268502 /var/log/ceph/ceph-mds.c.log 2023-11-09T03:56:26.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 15uW REG 0,24 7 2398 /run/ceph/ceph-mds.c.pid 2023-11-09T03:56:26.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 16r FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 17w FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 18u unix 0xffff959a60112ec0 0t0 252220 /var/run/ceph/ceph-mds.c.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.406 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 19u IPv4 251300 0t0 TCP *:6832 (LISTEN) 2023-11-09T03:56:26.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 20u IPv4 251334 0t0 TCP *:6833 (LISTEN) 2023-11-09T03:56:26.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 21r FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 22w FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 23r FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 24w FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.407 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 25u IPv4 247722 0t0 TCP smithi076.front.sepia.ceph.com:60028->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 26r FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 27w FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 28r FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 29w FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 30u IPv4 252224 0t0 TCP smithi076.front.sepia.ceph.com:59706->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.408 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 31u IPv4 250772 0t0 TCP smithi076.front.sepia.ceph.com:38604->smithi076.front.sepia.ceph.com:6816 (ESTABLISHED) 2023-11-09T03:56:26.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 32u IPv4 256430 0t0 TCP smithi076.front.sepia.ceph.com:33348->smithi076.front.sepia.ceph.com:6803 (ESTABLISHED) 2023-11-09T03:56:26.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 33u IPv4 249701 0t0 TCP smithi076.front.sepia.ceph.com:56048->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:26.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 34u IPv4 249702 0t0 TCP smithi076.front.sepia.ceph.com:48810->smithi076.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 35u IPv4 249703 0t0 TCP smithi076.front.sepia.ceph.com:37152->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.409 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 36u IPv4 256431 0t0 TCP smithi076.front.sepia.ceph.com:60920->smithi076.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 37u IPv4 257101 0t0 TCP smithi076.front.sepia.ceph.com:50856->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2023-11-09T03:56:26.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126455 ms_dispat root 39u IPv4 272562 0t0 TCP smithi076.front.sepia.ceph.com:6832->192.168.0.1:35628 (ESTABLISHED) 2023-11-09T03:56:26.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.410 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.411 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.412 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.413 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.414 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.415 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.416 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.417 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.418 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 1w FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:26.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 2w FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:26.419 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 3r FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 4w FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,31,34,37] 2023-11-09T03:56:26.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 6r FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 7w FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 8u a_inode 0,14 0 2077 [eventpoll:9,25,32,35] 2023-11-09T03:56:26.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 9r FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 10w FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 11u a_inode 0,14 0 2077 [eventpoll:12,30,33,36,39] 2023-11-09T03:56:26.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 12r FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.421 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 13w FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 14w REG 8,1 9138594 268502 /var/log/ceph/ceph-mds.c.log 2023-11-09T03:56:26.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 15uW REG 0,24 7 2398 /run/ceph/ceph-mds.c.pid 2023-11-09T03:56:26.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 16r FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 17w FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 18u unix 0xffff959a60112ec0 0t0 252220 /var/run/ceph/ceph-mds.c.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.422 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 19u IPv4 251300 0t0 TCP *:6832 (LISTEN) 2023-11-09T03:56:26.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 20u IPv4 251334 0t0 TCP *:6833 (LISTEN) 2023-11-09T03:56:26.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 21r FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 22w FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 23r FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.423 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 24w FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 25u IPv4 247722 0t0 TCP smithi076.front.sepia.ceph.com:60028->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 26r FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 27w FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 28r FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.424 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 29w FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 30u IPv4 252224 0t0 TCP smithi076.front.sepia.ceph.com:59706->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 31u IPv4 250772 0t0 TCP smithi076.front.sepia.ceph.com:38604->smithi076.front.sepia.ceph.com:6816 (ESTABLISHED) 2023-11-09T03:56:26.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 32u IPv4 256430 0t0 TCP smithi076.front.sepia.ceph.com:33348->smithi076.front.sepia.ceph.com:6803 (ESTABLISHED) 2023-11-09T03:56:26.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 33u IPv4 249701 0t0 TCP smithi076.front.sepia.ceph.com:56048->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:26.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 34u IPv4 249702 0t0 TCP smithi076.front.sepia.ceph.com:48810->smithi076.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 35u IPv4 249703 0t0 TCP smithi076.front.sepia.ceph.com:37152->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 36u IPv4 256431 0t0 TCP smithi076.front.sepia.ceph.com:60920->smithi076.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 37u IPv4 257101 0t0 TCP smithi076.front.sepia.ceph.com:50856->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2023-11-09T03:56:26.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126456 mds_rank_ root 39u IPv4 272562 0t0 TCP smithi076.front.sepia.ceph.com:6832->192.168.0.1:35628 (ESTABLISHED) 2023-11-09T03:56:26.426 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.427 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.428 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.429 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.430 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.431 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.432 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.433 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.434 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.435 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 1w FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:26.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 2w FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:26.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 3r FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 4w FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,31,34,37] 2023-11-09T03:56:26.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 6r FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.436 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 7w FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 8u a_inode 0,14 0 2077 [eventpoll:9,25,32,35] 2023-11-09T03:56:26.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 9r FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 10w FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 11u a_inode 0,14 0 2077 [eventpoll:12,30,33,36,39] 2023-11-09T03:56:26.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 12r FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.437 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 13w FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 14w REG 8,1 9138594 268502 /var/log/ceph/ceph-mds.c.log 2023-11-09T03:56:26.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 15uW REG 0,24 7 2398 /run/ceph/ceph-mds.c.pid 2023-11-09T03:56:26.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 16r FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 17w FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.438 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 18u unix 0xffff959a60112ec0 0t0 252220 /var/run/ceph/ceph-mds.c.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 19u IPv4 251300 0t0 TCP *:6832 (LISTEN) 2023-11-09T03:56:26.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 20u IPv4 251334 0t0 TCP *:6833 (LISTEN) 2023-11-09T03:56:26.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 21r FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 22w FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 23r FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.439 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 24w FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 25u IPv4 247722 0t0 TCP smithi076.front.sepia.ceph.com:60028->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 26r FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 27w FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 28r FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 29w FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.440 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 30u IPv4 252224 0t0 TCP smithi076.front.sepia.ceph.com:59706->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 31u IPv4 250772 0t0 TCP smithi076.front.sepia.ceph.com:38604->smithi076.front.sepia.ceph.com:6816 (ESTABLISHED) 2023-11-09T03:56:26.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 32u IPv4 256430 0t0 TCP smithi076.front.sepia.ceph.com:33348->smithi076.front.sepia.ceph.com:6803 (ESTABLISHED) 2023-11-09T03:56:26.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 33u IPv4 249701 0t0 TCP smithi076.front.sepia.ceph.com:56048->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:26.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 34u IPv4 249702 0t0 TCP smithi076.front.sepia.ceph.com:48810->smithi076.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 35u IPv4 249703 0t0 TCP smithi076.front.sepia.ceph.com:37152->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.441 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 36u IPv4 256431 0t0 TCP smithi076.front.sepia.ceph.com:60920->smithi076.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 37u IPv4 257101 0t0 TCP smithi076.front.sepia.ceph.com:50856->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2023-11-09T03:56:26.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126457 PQ_Finish root 39u IPv4 272562 0t0 TCP smithi076.front.sepia.ceph.com:6832->192.168.0.1:35628 (ESTABLISHED) 2023-11-09T03:56:26.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.442 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.443 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.444 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.445 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.446 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.447 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.448 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.449 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.450 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 1w FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:26.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 2w FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:26.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 3r FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 4w FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,31,34,37] 2023-11-09T03:56:26.451 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 6r FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 7w FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,25,32,35] 2023-11-09T03:56:26.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 9r FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 10w FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,30,33,36,39] 2023-11-09T03:56:26.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 12r FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 13w FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 14w REG 8,1 9138594 268502 /var/log/ceph/ceph-mds.c.log 2023-11-09T03:56:26.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 15uW REG 0,24 7 2398 /run/ceph/ceph-mds.c.pid 2023-11-09T03:56:26.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 16r FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.453 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 17w FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 18u unix 0xffff959a60112ec0 0t0 252220 /var/run/ceph/ceph-mds.c.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 19u IPv4 251300 0t0 TCP *:6832 (LISTEN) 2023-11-09T03:56:26.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 20u IPv4 251334 0t0 TCP *:6833 (LISTEN) 2023-11-09T03:56:26.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 21r FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 22w FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.454 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 23r FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 24w FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 25u IPv4 247722 0t0 TCP smithi076.front.sepia.ceph.com:60028->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 26r FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 27w FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 28r FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.455 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 29w FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 30u IPv4 252224 0t0 TCP smithi076.front.sepia.ceph.com:59706->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 31u IPv4 250772 0t0 TCP smithi076.front.sepia.ceph.com:38604->smithi076.front.sepia.ceph.com:6816 (ESTABLISHED) 2023-11-09T03:56:26.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 32u IPv4 256430 0t0 TCP smithi076.front.sepia.ceph.com:33348->smithi076.front.sepia.ceph.com:6803 (ESTABLISHED) 2023-11-09T03:56:26.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 33u IPv4 249701 0t0 TCP smithi076.front.sepia.ceph.com:56048->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:26.456 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 34u IPv4 249702 0t0 TCP smithi076.front.sepia.ceph.com:48810->smithi076.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 35u IPv4 249703 0t0 TCP smithi076.front.sepia.ceph.com:37152->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 36u IPv4 256431 0t0 TCP smithi076.front.sepia.ceph.com:60920->smithi076.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 37u IPv4 257101 0t0 TCP smithi076.front.sepia.ceph.com:50856->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2023-11-09T03:56:26.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126458 safe_time root 39u IPv4 272562 0t0 TCP smithi076.front.sepia.ceph.com:6832->192.168.0.1:35628 (ESTABLISHED) 2023-11-09T03:56:26.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.457 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.458 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.459 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.460 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.461 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.462 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.463 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.464 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 1w FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:26.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 2w FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:26.466 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 3r FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 4w FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,31,34,37] 2023-11-09T03:56:26.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 6r FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 7w FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 8u a_inode 0,14 0 2077 [eventpoll:9,25,32,35] 2023-11-09T03:56:26.467 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 9r FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 10w FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 11u a_inode 0,14 0 2077 [eventpoll:12,30,33,36,39] 2023-11-09T03:56:26.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 12r FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 13w FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 14w REG 8,1 9138594 268502 /var/log/ceph/ceph-mds.c.log 2023-11-09T03:56:26.468 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 15uW REG 0,24 7 2398 /run/ceph/ceph-mds.c.pid 2023-11-09T03:56:26.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 16r FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 17w FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 18u unix 0xffff959a60112ec0 0t0 252220 /var/run/ceph/ceph-mds.c.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 19u IPv4 251300 0t0 TCP *:6832 (LISTEN) 2023-11-09T03:56:26.469 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 20u IPv4 251334 0t0 TCP *:6833 (LISTEN) 2023-11-09T03:56:26.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 21r FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 22w FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 23r FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 24w FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 25u IPv4 247722 0t0 TCP smithi076.front.sepia.ceph.com:60028->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.470 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 26r FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 27w FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 28r FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 29w FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 30u IPv4 252224 0t0 TCP smithi076.front.sepia.ceph.com:59706->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.471 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 31u IPv4 250772 0t0 TCP smithi076.front.sepia.ceph.com:38604->smithi076.front.sepia.ceph.com:6816 (ESTABLISHED) 2023-11-09T03:56:26.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 32u IPv4 256430 0t0 TCP smithi076.front.sepia.ceph.com:33348->smithi076.front.sepia.ceph.com:6803 (ESTABLISHED) 2023-11-09T03:56:26.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 33u IPv4 249701 0t0 TCP smithi076.front.sepia.ceph.com:56048->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:26.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 34u IPv4 249702 0t0 TCP smithi076.front.sepia.ceph.com:48810->smithi076.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 35u IPv4 249703 0t0 TCP smithi076.front.sepia.ceph.com:37152->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.472 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 36u IPv4 256431 0t0 TCP smithi076.front.sepia.ceph.com:60920->smithi076.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 37u IPv4 257101 0t0 TCP smithi076.front.sepia.ceph.com:50856->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2023-11-09T03:56:26.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126459 MR_Finish root 39u IPv4 272562 0t0 TCP smithi076.front.sepia.ceph.com:6832->192.168.0.1:35628 (ESTABLISHED) 2023-11-09T03:56:26.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.473 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.474 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.475 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.476 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.477 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.478 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.479 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.480 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.481 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 1w FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:26.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 2w FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:26.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 3r FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 4w FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.482 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,31,34,37] 2023-11-09T03:56:26.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 6r FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 7w FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 8u a_inode 0,14 0 2077 [eventpoll:9,25,32,35] 2023-11-09T03:56:26.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 9r FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.483 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 10w FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 11u a_inode 0,14 0 2077 [eventpoll:12,30,33,36,39] 2023-11-09T03:56:26.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 12r FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 13w FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 14w REG 8,1 9138594 268502 /var/log/ceph/ceph-mds.c.log 2023-11-09T03:56:26.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 15uW REG 0,24 7 2398 /run/ceph/ceph-mds.c.pid 2023-11-09T03:56:26.484 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 16r FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 17w FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 18u unix 0xffff959a60112ec0 0t0 252220 /var/run/ceph/ceph-mds.c.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 19u IPv4 251300 0t0 TCP *:6832 (LISTEN) 2023-11-09T03:56:26.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 20u IPv4 251334 0t0 TCP *:6833 (LISTEN) 2023-11-09T03:56:26.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 21r FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.485 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 22w FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 23r FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 24w FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 25u IPv4 247722 0t0 TCP smithi076.front.sepia.ceph.com:60028->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 26r FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 27w FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.486 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 28r FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 29w FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 30u IPv4 252224 0t0 TCP smithi076.front.sepia.ceph.com:59706->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 31u IPv4 250772 0t0 TCP smithi076.front.sepia.ceph.com:38604->smithi076.front.sepia.ceph.com:6816 (ESTABLISHED) 2023-11-09T03:56:26.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 32u IPv4 256430 0t0 TCP smithi076.front.sepia.ceph.com:33348->smithi076.front.sepia.ceph.com:6803 (ESTABLISHED) 2023-11-09T03:56:26.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 33u IPv4 249701 0t0 TCP smithi076.front.sepia.ceph.com:56048->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:26.487 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 34u IPv4 249702 0t0 TCP smithi076.front.sepia.ceph.com:48810->smithi076.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 35u IPv4 249703 0t0 TCP smithi076.front.sepia.ceph.com:37152->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 36u IPv4 256431 0t0 TCP smithi076.front.sepia.ceph.com:60920->smithi076.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 37u IPv4 257101 0t0 TCP smithi076.front.sepia.ceph.com:50856->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2023-11-09T03:56:26.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 126461 md_submit root 39u IPv4 272562 0t0 TCP smithi076.front.sepia.ceph.com:6832->192.168.0.1:35628 (ESTABLISHED) 2023-11-09T03:56:26.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.488 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.489 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.490 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.491 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.493 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.494 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.495 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.496 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 1w FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:26.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 2w FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:26.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 3r FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 4w FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.497 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,31,34,37] 2023-11-09T03:56:26.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 6r FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 7w FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,25,32,35] 2023-11-09T03:56:26.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 9r FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 10w FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,30,33,36,39] 2023-11-09T03:56:26.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 12r FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 13w FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 14w REG 8,1 9139477 268502 /var/log/ceph/ceph-mds.c.log 2023-11-09T03:56:26.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 15uW REG 0,24 7 2398 /run/ceph/ceph-mds.c.pid 2023-11-09T03:56:26.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 16r FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.499 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 17w FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 18u unix 0xffff959a60112ec0 0t0 252220 /var/run/ceph/ceph-mds.c.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 19u IPv4 251300 0t0 TCP *:6832 (LISTEN) 2023-11-09T03:56:26.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 20u IPv4 251334 0t0 TCP *:6833 (LISTEN) 2023-11-09T03:56:26.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 21r FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 22w FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.500 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 23r FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 24w FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 25u IPv4 247722 0t0 TCP smithi076.front.sepia.ceph.com:60028->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 26r FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 27w FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 28r FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.501 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 29w FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 30u IPv4 252224 0t0 TCP smithi076.front.sepia.ceph.com:59706->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 31u IPv4 250772 0t0 TCP smithi076.front.sepia.ceph.com:38604->smithi076.front.sepia.ceph.com:6816 (ESTABLISHED) 2023-11-09T03:56:26.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 32u IPv4 256430 0t0 TCP smithi076.front.sepia.ceph.com:33348->smithi076.front.sepia.ceph.com:6803 (ESTABLISHED) 2023-11-09T03:56:26.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 33u IPv4 249701 0t0 TCP smithi076.front.sepia.ceph.com:56048->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:26.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 34u IPv4 249702 0t0 TCP smithi076.front.sepia.ceph.com:48810->smithi076.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.502 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 35u IPv4 249703 0t0 TCP smithi076.front.sepia.ceph.com:37152->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 36u IPv4 256431 0t0 TCP smithi076.front.sepia.ceph.com:60920->smithi076.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 37u IPv4 257101 0t0 TCP smithi076.front.sepia.ceph.com:50856->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2023-11-09T03:56:26.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127075 ms_dispat root 39u IPv4 272562 0t0 TCP smithi076.front.sepia.ceph.com:6832->192.168.0.1:35628 (ESTABLISHED) 2023-11-09T03:56:26.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.503 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.504 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.505 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.506 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.507 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.508 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.509 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.510 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.511 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 1w FIFO 0,13 0t0 248573 pipe 2023-11-09T03:56:26.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 2w FIFO 0,13 0t0 248574 pipe 2023-11-09T03:56:26.512 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 3r FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 4w FIFO 0,13 0t0 247717 pipe 2023-11-09T03:56:26.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20,31,34,37] 2023-11-09T03:56:26.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 6r FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 7w FIFO 0,13 0t0 247719 pipe 2023-11-09T03:56:26.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,25,32,35] 2023-11-09T03:56:26.513 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 9r FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 10w FIFO 0,13 0t0 247720 pipe 2023-11-09T03:56:26.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,30,33,36,39] 2023-11-09T03:56:26.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 12r FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 13w FIFO 0,13 0t0 247721 pipe 2023-11-09T03:56:26.514 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 14w REG 8,1 9139477 268502 /var/log/ceph/ceph-mds.c.log 2023-11-09T03:56:26.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 15uW REG 0,24 7 2398 /run/ceph/ceph-mds.c.pid 2023-11-09T03:56:26.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 16r FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 17w FIFO 0,13 0t0 252219 pipe 2023-11-09T03:56:26.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 18u unix 0xffff959a60112ec0 0t0 252220 /var/run/ceph/ceph-mds.c.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 19u IPv4 251300 0t0 TCP *:6832 (LISTEN) 2023-11-09T03:56:26.515 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 20u IPv4 251334 0t0 TCP *:6833 (LISTEN) 2023-11-09T03:56:26.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 21r FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 22w FIFO 0,13 0t0 252222 pipe 2023-11-09T03:56:26.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 23r FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 24w FIFO 0,13 0t0 252223 pipe 2023-11-09T03:56:26.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 25u IPv4 247722 0t0 TCP smithi076.front.sepia.ceph.com:60028->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.516 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 26r FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 27w FIFO 0,13 0t0 247723 pipe 2023-11-09T03:56:26.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 28r FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 29w FIFO 0,13 0t0 247724 pipe 2023-11-09T03:56:26.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 30u IPv4 252224 0t0 TCP smithi076.front.sepia.ceph.com:59706->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 31u IPv4 250772 0t0 TCP smithi076.front.sepia.ceph.com:38604->smithi076.front.sepia.ceph.com:6816 (ESTABLISHED) 2023-11-09T03:56:26.517 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 32u IPv4 256430 0t0 TCP smithi076.front.sepia.ceph.com:33348->smithi076.front.sepia.ceph.com:6803 (ESTABLISHED) 2023-11-09T03:56:26.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 33u IPv4 249701 0t0 TCP smithi076.front.sepia.ceph.com:56048->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2023-11-09T03:56:26.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 34u IPv4 249702 0t0 TCP smithi076.front.sepia.ceph.com:48810->smithi076.front.sepia.ceph.com:6822 (ESTABLISHED) 2023-11-09T03:56:26.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 35u IPv4 249703 0t0 TCP smithi076.front.sepia.ceph.com:37152->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 36u IPv4 256431 0t0 TCP smithi076.front.sepia.ceph.com:60920->smithi076.front.sepia.ceph.com:6800 (ESTABLISHED) 2023-11-09T03:56:26.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 37u IPv4 257101 0t0 TCP smithi076.front.sepia.ceph.com:50856->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2023-11-09T03:56:26.518 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 124533 127076 ms_dispat root 39u IPv4 272562 0t0 TCP smithi076.front.sepia.ceph.com:6832->192.168.0.1:35628 (ESTABLISHED) 2023-11-09T03:56:26.519 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/2 126410 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:26.519 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/2 126410 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.519 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/2 126410 root txt unknown /proc/126410/exe 2023-11-09T03:56:26.519 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/2 128551 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:26.519 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/2 128551 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.519 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/2 128551 root txt unknown /proc/128551/exe 2023-11-09T03:56:26.520 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/5 130499 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:26.520 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/5 130499 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.520 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/5 130499 root txt unknown /proc/130499/exe 2023-11-09T03:56:26.520 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.520 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.520 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root txt REG 8,1 185296 11759 /usr/bin/sudo 2023-11-09T03:56:26.521 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 357408 10120 /usr/lib64/libnss_systemd.so.2 2023-11-09T03:56:26.521 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:26.521 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.521 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-11-09T03:56:26.521 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:26.521 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 19600 10967 /usr/lib64/libpam_misc.so.0.82.1 2023-11-09T03:56:26.522 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 19400 32305 /usr/lib64/security/pam_succeed_if.so 2023-11-09T03:56:26.522 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 514280 32422 /usr/lib64/security/pam_systemd.so 2023-11-09T03:56:26.522 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 48392 10228 /usr/lib64/libcrack.so.2.9.0 2023-11-09T03:56:26.522 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.522 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.522 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 27536 32284 /usr/lib64/security/pam_limits.so 2023-11-09T03:56:26.523 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 32608 10961 /usr/lib64/libpwquality.so.1.0.2 2023-11-09T03:56:26.523 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 15352 32282 /usr/lib64/security/pam_keyinit.so 2023-11-09T03:56:26.523 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 15840 32264 /usr/lib64/security/pam_pwquality.so 2023-11-09T03:56:26.523 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 56712 32310 /usr/lib64/security/pam_unix.so 2023-11-09T03:56:26.523 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.523 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.524 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.524 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.524 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.524 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.524 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-11-09T03:56:26.525 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 14928 32270 /usr/lib64/security/pam_deny.so 2023-11-09T03:56:26.525 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.525 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.525 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.525 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.525 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.526 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 15184 32274 /usr/lib64/security/pam_faildelay.so 2023-11-09T03:56:26.526 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.526 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.526 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-11-09T03:56:26.526 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.526 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.527 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-11-09T03:56:26.527 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 19464 32272 /usr/lib64/security/pam_env.so 2023-11-09T03:56:26.527 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 472568 54513 /usr/libexec/sudo/sudoers.so 2023-11-09T03:56:26.527 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:26.527 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-11-09T03:56:26.527 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-11-09T03:56:26.528 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-11-09T03:56:26.528 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-11-09T03:56:26.528 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-11-09T03:56:26.528 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-11-09T03:56:26.528 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-11-09T03:56:26.528 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-11-09T03:56:26.529 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-11-09T03:56:26.529 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.529 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:26.529 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 119832 54509 /usr/libexec/sudo/libsudo_util.so.0.0.0 2023-11-09T03:56:26.529 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.529 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:26.530 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:26.530 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.530 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-11-09T03:56:26.530 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root 0r FIFO 0,13 0t0 265997 pipe 2023-11-09T03:56:26.530 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root 1w FIFO 0,13 0t0 265998 pipe 2023-11-09T03:56:26.530 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root 2w FIFO 0,13 0t0 265999 pipe 2023-11-09T03:56:26.531 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root 3r FIFO 0,13 0t0 261969 pipe 2023-11-09T03:56:26.531 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root 4w FIFO 0,13 0t0 261969 pipe 2023-11-09T03:56:26.531 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root 5u netlink 0t0 261974 AUDIT 2023-11-09T03:56:26.531 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 132747 root 6u unix 0xffff959a4837ee80 0t0 261975 type=DGRAM (CONNECTED) 2023-11-09T03:56:26.531 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 132781 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.532 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 132781 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.532 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 132781 root txt REG 8,1 15448 10498 /usr/bin/python3.9 2023-11-09T03:56:26.532 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 132781 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:26.532 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 132781 root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:26.532 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 132781 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.532 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 132781 root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.533 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 132781 root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.533 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 132781 root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.533 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 132781 root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.533 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 132781 root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.533 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 132781 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.533 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 132781 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:26.534 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 132781 root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:26.534 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 132781 root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.534 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 132781 root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.534 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 132781 root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.534 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 132781 root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.534 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 132781 root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.535 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 132781 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.535 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 132781 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:26.535 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 132781 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.535 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 132781 root 0r FIFO 0,13 0t0 265997 pipe 2023-11-09T03:56:26.535 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 132781 root 1w FIFO 0,13 0t0 265998 pipe 2023-11-09T03:56:26.536 INFO:teuthology.orchestra.run.smithi076.stdout:daemon-he 132781 root 2w FIFO 0,13 0t0 265999 pipe 2023-11-09T03:56:26.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.536 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.537 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.538 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.539 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.540 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.541 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.542 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.543 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.544 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 1w FIFO 0,13 0t0 265998 pipe 2023-11-09T03:56:26.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 2w FIFO 0,13 0t0 265999 pipe 2023-11-09T03:56:26.545 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 3r FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 4w FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20] 2023-11-09T03:56:26.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 6r FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 7w FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 8u a_inode 0,14 0 2077 [eventpoll:9,30] 2023-11-09T03:56:26.546 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 9r FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 10w FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 11u a_inode 0,14 0 2077 [eventpoll:12,26] 2023-11-09T03:56:26.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 12r FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 13w FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.547 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 14w REG 8,1 6800029 268501 /var/log/ceph/ceph-mds.a.log 2023-11-09T03:56:26.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 15uW REG 0,24 7 2583 /run/ceph/ceph-mds.a.pid 2023-11-09T03:56:26.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 16r FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 17w FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 18u unix 0xffff959a60117b40 0t0 268076 /var/run/ceph/ceph-mds.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 19u IPv4 270633 0t0 TCP *:6834 (LISTEN) 2023-11-09T03:56:26.548 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 20u IPv4 270669 0t0 TCP *:6835 (LISTEN) 2023-11-09T03:56:26.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 21r FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 22w FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 23r FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 24w FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 25r FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.549 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 26u IPv4 265160 0t0 TCP smithi076.front.sepia.ceph.com:50182->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 27w FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 28r FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 29w FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 root 30u IPv4 269842 0t0 TCP smithi076.front.sepia.ceph.com:52512->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.550 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.551 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.552 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.553 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.554 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.555 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.557 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.558 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 1w FIFO 0,13 0t0 265998 pipe 2023-11-09T03:56:26.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 2w FIFO 0,13 0t0 265999 pipe 2023-11-09T03:56:26.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 3r FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 4w FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.559 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20] 2023-11-09T03:56:26.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 6r FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 7w FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 8u a_inode 0,14 0 2077 [eventpoll:9,30] 2023-11-09T03:56:26.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 9r FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.560 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 10w FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 11u a_inode 0,14 0 2077 [eventpoll:12,26] 2023-11-09T03:56:26.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 12r FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 13w FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 14w REG 8,1 6800029 268501 /var/log/ceph/ceph-mds.a.log 2023-11-09T03:56:26.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 15uW REG 0,24 7 2583 /run/ceph/ceph-mds.a.pid 2023-11-09T03:56:26.561 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 16r FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 17w FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 18u unix 0xffff959a60117b40 0t0 268076 /var/run/ceph/ceph-mds.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 19u IPv4 270633 0t0 TCP *:6834 (LISTEN) 2023-11-09T03:56:26.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 20u IPv4 270669 0t0 TCP *:6835 (LISTEN) 2023-11-09T03:56:26.562 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 21r FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 22w FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 23r FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 24w FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 25r FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 26u IPv4 265160 0t0 TCP smithi076.front.sepia.ceph.com:50182->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.563 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 27w FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 28r FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 29w FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132787 log root 30u IPv4 269842 0t0 TCP smithi076.front.sepia.ceph.com:52512->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.564 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.565 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.567 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.568 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.569 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.570 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.571 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.572 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 1w FIFO 0,13 0t0 265998 pipe 2023-11-09T03:56:26.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 2w FIFO 0,13 0t0 265999 pipe 2023-11-09T03:56:26.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 3r FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 4w FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20] 2023-11-09T03:56:26.573 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 6r FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 7w FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 8u a_inode 0,14 0 2077 [eventpoll:9,30] 2023-11-09T03:56:26.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 9r FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 10w FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 11u a_inode 0,14 0 2077 [eventpoll:12,26] 2023-11-09T03:56:26.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 12r FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 13w FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 14w REG 8,1 6800029 268501 /var/log/ceph/ceph-mds.a.log 2023-11-09T03:56:26.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 15uW REG 0,24 7 2583 /run/ceph/ceph-mds.a.pid 2023-11-09T03:56:26.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 16r FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.575 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 17w FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 18u unix 0xffff959a60117b40 0t0 268076 /var/run/ceph/ceph-mds.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 19u IPv4 270633 0t0 TCP *:6834 (LISTEN) 2023-11-09T03:56:26.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 20u IPv4 270669 0t0 TCP *:6835 (LISTEN) 2023-11-09T03:56:26.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 21r FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 22w FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 23r FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 24w FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 25r FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 26u IPv4 265160 0t0 TCP smithi076.front.sepia.ceph.com:50182->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 27w FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 28r FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 29w FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132789 msgr-work root 30u IPv4 269842 0t0 TCP smithi076.front.sepia.ceph.com:52512->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.578 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.579 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.580 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.581 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.582 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.583 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.584 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.585 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.586 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 1w FIFO 0,13 0t0 265998 pipe 2023-11-09T03:56:26.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 2w FIFO 0,13 0t0 265999 pipe 2023-11-09T03:56:26.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 3r FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 4w FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20] 2023-11-09T03:56:26.587 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 6r FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 7w FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 8u a_inode 0,14 0 2077 [eventpoll:9,30] 2023-11-09T03:56:26.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 9r FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 10w FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 11u a_inode 0,14 0 2077 [eventpoll:12,26] 2023-11-09T03:56:26.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 12r FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 13w FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 14w REG 8,1 6800029 268501 /var/log/ceph/ceph-mds.a.log 2023-11-09T03:56:26.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 15uW REG 0,24 7 2583 /run/ceph/ceph-mds.a.pid 2023-11-09T03:56:26.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 16r FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 17w FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 18u unix 0xffff959a60117b40 0t0 268076 /var/run/ceph/ceph-mds.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 19u IPv4 270633 0t0 TCP *:6834 (LISTEN) 2023-11-09T03:56:26.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 20u IPv4 270669 0t0 TCP *:6835 (LISTEN) 2023-11-09T03:56:26.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 21r FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 22w FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 23r FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.590 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 24w FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 25r FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 26u IPv4 265160 0t0 TCP smithi076.front.sepia.ceph.com:50182->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 27w FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 28r FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 29w FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.591 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132790 msgr-work root 30u IPv4 269842 0t0 TCP smithi076.front.sepia.ceph.com:52512->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.592 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.593 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.594 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.595 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.596 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.597 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.598 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.599 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 1w FIFO 0,13 0t0 265998 pipe 2023-11-09T03:56:26.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 2w FIFO 0,13 0t0 265999 pipe 2023-11-09T03:56:26.600 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 3r FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 4w FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20] 2023-11-09T03:56:26.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 6r FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 7w FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.601 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 8u a_inode 0,14 0 2077 [eventpoll:9,30] 2023-11-09T03:56:26.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 9r FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 10w FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 11u a_inode 0,14 0 2077 [eventpoll:12,26] 2023-11-09T03:56:26.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 12r FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 13w FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.602 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 14w REG 8,1 6800029 268501 /var/log/ceph/ceph-mds.a.log 2023-11-09T03:56:26.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 15uW REG 0,24 7 2583 /run/ceph/ceph-mds.a.pid 2023-11-09T03:56:26.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 16r FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 17w FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 18u unix 0xffff959a60117b40 0t0 268076 /var/run/ceph/ceph-mds.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.603 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 19u IPv4 270633 0t0 TCP *:6834 (LISTEN) 2023-11-09T03:56:26.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 20u IPv4 270669 0t0 TCP *:6835 (LISTEN) 2023-11-09T03:56:26.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 21r FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 22w FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 23r FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 24w FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.604 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 25r FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 26u IPv4 265160 0t0 TCP smithi076.front.sepia.ceph.com:50182->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 27w FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 28r FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 29w FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132791 msgr-work root 30u IPv4 269842 0t0 TCP smithi076.front.sepia.ceph.com:52512->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.606 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.608 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.609 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.611 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.612 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.613 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 1w FIFO 0,13 0t0 265998 pipe 2023-11-09T03:56:26.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 2w FIFO 0,13 0t0 265999 pipe 2023-11-09T03:56:26.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 3r FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.614 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 4w FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20] 2023-11-09T03:56:26.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 6r FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 7w FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 8u a_inode 0,14 0 2077 [eventpoll:9,30] 2023-11-09T03:56:26.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 9r FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.615 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 10w FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 11u a_inode 0,14 0 2077 [eventpoll:12,26] 2023-11-09T03:56:26.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 12r FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 13w FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 14w REG 8,1 6800029 268501 /var/log/ceph/ceph-mds.a.log 2023-11-09T03:56:26.616 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 15uW REG 0,24 7 2583 /run/ceph/ceph-mds.a.pid 2023-11-09T03:56:26.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 16r FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 17w FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 18u unix 0xffff959a60117b40 0t0 268076 /var/run/ceph/ceph-mds.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 19u IPv4 270633 0t0 TCP *:6834 (LISTEN) 2023-11-09T03:56:26.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 20u IPv4 270669 0t0 TCP *:6835 (LISTEN) 2023-11-09T03:56:26.617 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 21r FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 22w FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 23r FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 24w FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 25r FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.618 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 26u IPv4 265160 0t0 TCP smithi076.front.sepia.ceph.com:50182->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 27w FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 28r FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 29w FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132795 service root 30u IPv4 269842 0t0 TCP smithi076.front.sepia.ceph.com:52512->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.619 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.621 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.622 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.623 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.624 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.625 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.626 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.627 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.628 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.629 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 1w FIFO 0,13 0t0 265998 pipe 2023-11-09T03:56:26.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 2w FIFO 0,13 0t0 265999 pipe 2023-11-09T03:56:26.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 3r FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 4w FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20] 2023-11-09T03:56:26.630 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 6r FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 7w FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 8u a_inode 0,14 0 2077 [eventpoll:9,30] 2023-11-09T03:56:26.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 9r FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 10w FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.631 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 11u a_inode 0,14 0 2077 [eventpoll:12,26] 2023-11-09T03:56:26.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 12r FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 13w FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 14w REG 8,1 6800029 268501 /var/log/ceph/ceph-mds.a.log 2023-11-09T03:56:26.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 15uW REG 0,24 7 2583 /run/ceph/ceph-mds.a.pid 2023-11-09T03:56:26.632 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 16r FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 17w FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 18u unix 0xffff959a60117b40 0t0 268076 /var/run/ceph/ceph-mds.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 19u IPv4 270633 0t0 TCP *:6834 (LISTEN) 2023-11-09T03:56:26.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 20u IPv4 270669 0t0 TCP *:6835 (LISTEN) 2023-11-09T03:56:26.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 21r FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.633 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 22w FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 23r FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 24w FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 25r FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 26u IPv4 265160 0t0 TCP smithi076.front.sepia.ceph.com:50182->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.634 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 27w FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 28r FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 29w FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132796 admin_soc root 30u IPv4 269842 0t0 TCP smithi076.front.sepia.ceph.com:52512->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.635 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.636 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.637 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.638 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.639 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.640 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.641 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.642 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.643 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 1w FIFO 0,13 0t0 265998 pipe 2023-11-09T03:56:26.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 2w FIFO 0,13 0t0 265999 pipe 2023-11-09T03:56:26.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 3r FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.644 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 4w FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20] 2023-11-09T03:56:26.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 6r FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 7w FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 8u a_inode 0,14 0 2077 [eventpoll:9,30] 2023-11-09T03:56:26.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 9r FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.645 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 10w FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 11u a_inode 0,14 0 2077 [eventpoll:12,26] 2023-11-09T03:56:26.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 12r FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 13w FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 14w REG 8,1 6800029 268501 /var/log/ceph/ceph-mds.a.log 2023-11-09T03:56:26.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 15uW REG 0,24 7 2583 /run/ceph/ceph-mds.a.pid 2023-11-09T03:56:26.646 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 16r FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 17w FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 18u unix 0xffff959a60117b40 0t0 268076 /var/run/ceph/ceph-mds.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 19u IPv4 270633 0t0 TCP *:6834 (LISTEN) 2023-11-09T03:56:26.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 20u IPv4 270669 0t0 TCP *:6835 (LISTEN) 2023-11-09T03:56:26.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 21r FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.647 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 22w FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 23r FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 24w FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 25r FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 26u IPv4 265160 0t0 TCP smithi076.front.sepia.ceph.com:50182->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.648 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 27w FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 28r FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 29w FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132797 signal_ha root 30u IPv4 269842 0t0 TCP smithi076.front.sepia.ceph.com:52512->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.649 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.650 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.651 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.652 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.653 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.654 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.655 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.656 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.657 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 1w FIFO 0,13 0t0 265998 pipe 2023-11-09T03:56:26.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 2w FIFO 0,13 0t0 265999 pipe 2023-11-09T03:56:26.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 3r FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 4w FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.658 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20] 2023-11-09T03:56:26.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 6r FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 7w FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 8u a_inode 0,14 0 2077 [eventpoll:9,30] 2023-11-09T03:56:26.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 9r FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 10w FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.659 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 11u a_inode 0,14 0 2077 [eventpoll:12,26] 2023-11-09T03:56:26.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 12r FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 13w FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 14w REG 8,1 6800029 268501 /var/log/ceph/ceph-mds.a.log 2023-11-09T03:56:26.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 15uW REG 0,24 7 2583 /run/ceph/ceph-mds.a.pid 2023-11-09T03:56:26.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 16r FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.660 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 17w FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 18u unix 0xffff959a60117b40 0t0 268076 /var/run/ceph/ceph-mds.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 19u IPv4 270633 0t0 TCP *:6834 (LISTEN) 2023-11-09T03:56:26.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 20u IPv4 270669 0t0 TCP *:6835 (LISTEN) 2023-11-09T03:56:26.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 21r FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 22w FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.661 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 23r FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 24w FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 25r FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 26u IPv4 265160 0t0 TCP smithi076.front.sepia.ceph.com:50182->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 27w FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.662 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 28r FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 29w FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132798 io_contex root 30u IPv4 269842 0t0 TCP smithi076.front.sepia.ceph.com:52512->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.663 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.664 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.665 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.666 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.667 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.668 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.669 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.670 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 1w FIFO 0,13 0t0 265998 pipe 2023-11-09T03:56:26.671 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 2w FIFO 0,13 0t0 265999 pipe 2023-11-09T03:56:26.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 3r FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 4w FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20] 2023-11-09T03:56:26.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 6r FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 7w FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.672 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 8u a_inode 0,14 0 2077 [eventpoll:9,30] 2023-11-09T03:56:26.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 9r FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 10w FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 11u a_inode 0,14 0 2077 [eventpoll:12,26] 2023-11-09T03:56:26.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 12r FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.673 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 13w FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 14w REG 8,1 6800029 268501 /var/log/ceph/ceph-mds.a.log 2023-11-09T03:56:26.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 15uW REG 0,24 7 2583 /run/ceph/ceph-mds.a.pid 2023-11-09T03:56:26.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 16r FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 17w FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.674 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 18u unix 0xffff959a60117b40 0t0 268076 /var/run/ceph/ceph-mds.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 19u IPv4 270633 0t0 TCP *:6834 (LISTEN) 2023-11-09T03:56:26.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 20u IPv4 270669 0t0 TCP *:6835 (LISTEN) 2023-11-09T03:56:26.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 21r FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 22w FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 23r FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.675 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 24w FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 25r FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 26u IPv4 265160 0t0 TCP smithi076.front.sepia.ceph.com:50182->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 27w FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 28r FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 29w FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132799 io_contex root 30u IPv4 269842 0t0 TCP smithi076.front.sepia.ceph.com:52512->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.676 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.677 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.678 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.679 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.680 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.681 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.682 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 1w FIFO 0,13 0t0 265998 pipe 2023-11-09T03:56:26.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 2w FIFO 0,13 0t0 265999 pipe 2023-11-09T03:56:26.685 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 3r FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 4w FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20] 2023-11-09T03:56:26.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 6r FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 7w FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 8u a_inode 0,14 0 2077 [eventpoll:9,30] 2023-11-09T03:56:26.686 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 9r FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 10w FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 11u a_inode 0,14 0 2077 [eventpoll:12,26] 2023-11-09T03:56:26.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 12r FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 13w FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 14w REG 8,1 6800029 268501 /var/log/ceph/ceph-mds.a.log 2023-11-09T03:56:26.687 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 15uW REG 0,24 7 2583 /run/ceph/ceph-mds.a.pid 2023-11-09T03:56:26.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 16r FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 17w FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 18u unix 0xffff959a60117b40 0t0 268076 /var/run/ceph/ceph-mds.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 19u IPv4 270633 0t0 TCP *:6834 (LISTEN) 2023-11-09T03:56:26.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 20u IPv4 270669 0t0 TCP *:6835 (LISTEN) 2023-11-09T03:56:26.688 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 21r FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 22w FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 23r FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 24w FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 25r FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 26u IPv4 265160 0t0 TCP smithi076.front.sepia.ceph.com:50182->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.689 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 27w FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 28r FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 29w FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132800 ms_dispat root 30u IPv4 269842 0t0 TCP smithi076.front.sepia.ceph.com:52512->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.690 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.691 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.692 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.693 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.694 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.695 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.696 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.697 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.698 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 1w FIFO 0,13 0t0 265998 pipe 2023-11-09T03:56:26.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 2w FIFO 0,13 0t0 265999 pipe 2023-11-09T03:56:26.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 3r FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 4w FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20] 2023-11-09T03:56:26.699 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 6r FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 7w FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 8u a_inode 0,14 0 2077 [eventpoll:9,30] 2023-11-09T03:56:26.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 9r FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 10w FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.700 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 11u a_inode 0,14 0 2077 [eventpoll:12,26] 2023-11-09T03:56:26.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 12r FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 13w FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.701 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 14w REG 8,1 6800029 268501 /var/log/ceph/ceph-mds.a.log 2023-11-09T03:56:26.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 15uW REG 0,24 7 2583 /run/ceph/ceph-mds.a.pid 2023-11-09T03:56:26.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 16r FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 17w FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 18u unix 0xffff959a60117b40 0t0 268076 /var/run/ceph/ceph-mds.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 19u IPv4 270633 0t0 TCP *:6834 (LISTEN) 2023-11-09T03:56:26.702 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 20u IPv4 270669 0t0 TCP *:6835 (LISTEN) 2023-11-09T03:56:26.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 21r FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 22w FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 23r FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 24w FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.703 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 25r FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 26u IPv4 265160 0t0 TCP smithi076.front.sepia.ceph.com:50182->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 27w FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 28r FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 29w FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132801 ms_local root 30u IPv4 269842 0t0 TCP smithi076.front.sepia.ceph.com:52512->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.704 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.705 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.706 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.707 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.708 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.709 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.710 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.711 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.712 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 1w FIFO 0,13 0t0 265998 pipe 2023-11-09T03:56:26.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 2w FIFO 0,13 0t0 265999 pipe 2023-11-09T03:56:26.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 3r FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 4w FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.713 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20] 2023-11-09T03:56:26.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 6r FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 7w FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,30] 2023-11-09T03:56:26.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 9r FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.714 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 10w FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,26] 2023-11-09T03:56:26.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 12r FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 13w FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 14w REG 8,1 6800029 268501 /var/log/ceph/ceph-mds.a.log 2023-11-09T03:56:26.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 15uW REG 0,24 7 2583 /run/ceph/ceph-mds.a.pid 2023-11-09T03:56:26.715 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 16r FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 17w FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 18u unix 0xffff959a60117b40 0t0 268076 /var/run/ceph/ceph-mds.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 19u IPv4 270633 0t0 TCP *:6834 (LISTEN) 2023-11-09T03:56:26.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 20u IPv4 270669 0t0 TCP *:6835 (LISTEN) 2023-11-09T03:56:26.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 21r FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.716 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 22w FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 23r FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 24w FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 25r FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 26u IPv4 265160 0t0 TCP smithi076.front.sepia.ceph.com:50182->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 27w FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.717 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 28r FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 29w FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132802 safe_time root 30u IPv4 269842 0t0 TCP smithi076.front.sepia.ceph.com:52512->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.718 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.719 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.720 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.721 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.722 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.723 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.724 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.725 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.726 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 1w FIFO 0,13 0t0 265998 pipe 2023-11-09T03:56:26.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 2w FIFO 0,13 0t0 265999 pipe 2023-11-09T03:56:26.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 3r FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 4w FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20] 2023-11-09T03:56:26.727 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 6r FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 7w FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,30] 2023-11-09T03:56:26.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 9r FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 10w FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,26] 2023-11-09T03:56:26.728 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 12r FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 13w FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 14w REG 8,1 6800029 268501 /var/log/ceph/ceph-mds.a.log 2023-11-09T03:56:26.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 15uW REG 0,24 7 2583 /run/ceph/ceph-mds.a.pid 2023-11-09T03:56:26.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 16r FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 17w FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 18u unix 0xffff959a60117b40 0t0 268076 /var/run/ceph/ceph-mds.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.729 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 19u IPv4 270633 0t0 TCP *:6834 (LISTEN) 2023-11-09T03:56:26.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 20u IPv4 270669 0t0 TCP *:6835 (LISTEN) 2023-11-09T03:56:26.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 21r FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 22w FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 23r FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.730 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 24w FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 25r FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 26u IPv4 265160 0t0 TCP smithi076.front.sepia.ceph.com:50182->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 27w FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 28r FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 29w FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.731 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132803 safe_time root 30u IPv4 269842 0t0 TCP smithi076.front.sepia.ceph.com:52512->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.732 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.733 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.734 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.735 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.736 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.737 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.738 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.739 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 1w FIFO 0,13 0t0 265998 pipe 2023-11-09T03:56:26.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 2w FIFO 0,13 0t0 265999 pipe 2023-11-09T03:56:26.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 3r FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.740 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 4w FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20] 2023-11-09T03:56:26.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 6r FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 7w FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 8u a_inode 0,14 0 2077 [eventpoll:9,30] 2023-11-09T03:56:26.741 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 9r FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 10w FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 11u a_inode 0,14 0 2077 [eventpoll:12,26] 2023-11-09T03:56:26.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 12r FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 13w FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 14w REG 8,1 6800029 268501 /var/log/ceph/ceph-mds.a.log 2023-11-09T03:56:26.742 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 15uW REG 0,24 7 2583 /run/ceph/ceph-mds.a.pid 2023-11-09T03:56:26.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 16r FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 17w FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 18u unix 0xffff959a60117b40 0t0 268076 /var/run/ceph/ceph-mds.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 19u IPv4 270633 0t0 TCP *:6834 (LISTEN) 2023-11-09T03:56:26.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 20u IPv4 270669 0t0 TCP *:6835 (LISTEN) 2023-11-09T03:56:26.743 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 21r FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 22w FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 23r FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 24w FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 25r FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 26u IPv4 265160 0t0 TCP smithi076.front.sepia.ceph.com:50182->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.744 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 27w FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 28r FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 29w FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132804 ceph-mds root 30u IPv4 269842 0t0 TCP smithi076.front.sepia.ceph.com:52512->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.745 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root txt REG 8,1 6696672 3106 /usr/bin/ceph-mds 2023-11-09T03:56:26.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.746 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.747 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.748 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.749 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.750 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.751 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 326760 2537 /usr/lib64/libtcmalloc.so.4.5.9 2023-11-09T03:56:26.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.752 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 61880 2527 /usr/lib64/libunwind.so.8.0.1 2023-11-09T03:56:26.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 287296 9996 /usr/lib64/liblua-5.4.so 2023-11-09T03:56:26.753 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-11-09T03:56:26.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 1w FIFO 0,13 0t0 265998 pipe 2023-11-09T03:56:26.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 2w FIFO 0,13 0t0 265999 pipe 2023-11-09T03:56:26.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 3r FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.754 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 4w FIFO 0,13 0t0 268069 pipe 2023-11-09T03:56:26.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 5u a_inode 0,14 0 2077 [eventpoll:6,19,20] 2023-11-09T03:56:26.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 6r FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 7w FIFO 0,13 0t0 268071 pipe 2023-11-09T03:56:26.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 8u a_inode 0,14 0 2077 [eventpoll:9,30] 2023-11-09T03:56:26.755 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 9r FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 10w FIFO 0,13 0t0 268072 pipe 2023-11-09T03:56:26.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 11u a_inode 0,14 0 2077 [eventpoll:12,26] 2023-11-09T03:56:26.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 12r FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 13w FIFO 0,13 0t0 268073 pipe 2023-11-09T03:56:26.756 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 14w REG 8,1 6800029 268501 /var/log/ceph/ceph-mds.a.log 2023-11-09T03:56:26.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 15uW REG 0,24 7 2583 /run/ceph/ceph-mds.a.pid 2023-11-09T03:56:26.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 16r FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 17w FIFO 0,13 0t0 268075 pipe 2023-11-09T03:56:26.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 18u unix 0xffff959a60117b40 0t0 268076 /var/run/ceph/ceph-mds.a.asok type=STREAM (LISTEN) 2023-11-09T03:56:26.757 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 19u IPv4 270633 0t0 TCP *:6834 (LISTEN) 2023-11-09T03:56:26.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 20u IPv4 270669 0t0 TCP *:6835 (LISTEN) 2023-11-09T03:56:26.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 21r FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 22w FIFO 0,13 0t0 268078 pipe 2023-11-09T03:56:26.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 23r FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.758 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 24w FIFO 0,13 0t0 268079 pipe 2023-11-09T03:56:26.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 25r FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 26u IPv4 265160 0t0 TCP smithi076.front.sepia.ceph.com:50182->smithi076.front.sepia.ceph.com:3300 (ESTABLISHED) 2023-11-09T03:56:26.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 27w FIFO 0,13 0t0 270671 pipe 2023-11-09T03:56:26.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 28r FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 29w FIFO 0,13 0t0 270672 pipe 2023-11-09T03:56:26.759 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-mds 132786 132827 safe_time root 30u IPv4 269842 0t0 TCP smithi076.front.sepia.ceph.com:52512->smithi113.front.sepia.ceph.com:6832 (ESTABLISHED) 2023-11-09T03:56:26.760 INFO:teuthology.orchestra.run.smithi076.stdout:bash 133832 ubuntu cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:26.760 INFO:teuthology.orchestra.run.smithi076.stdout:bash 133832 ubuntu rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.760 INFO:teuthology.orchestra.run.smithi076.stdout:bash 133832 ubuntu txt REG 8,1 1388920 9667 /usr/bin/bash 2023-11-09T03:56:26.760 INFO:teuthology.orchestra.run.smithi076.stdout:bash 133832 ubuntu mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-11-09T03:56:26.760 INFO:teuthology.orchestra.run.smithi076.stdout:bash 133832 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.760 INFO:teuthology.orchestra.run.smithi076.stdout:bash 133832 ubuntu mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:26.761 INFO:teuthology.orchestra.run.smithi076.stdout:bash 133832 ubuntu mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-11-09T03:56:26.761 INFO:teuthology.orchestra.run.smithi076.stdout:bash 133832 ubuntu mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-11-09T03:56:26.761 INFO:teuthology.orchestra.run.smithi076.stdout:bash 133832 ubuntu mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-11-09T03:56:26.761 INFO:teuthology.orchestra.run.smithi076.stdout:bash 133832 ubuntu mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-11-09T03:56:26.761 INFO:teuthology.orchestra.run.smithi076.stdout:bash 133832 ubuntu mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-11-09T03:56:26.762 INFO:teuthology.orchestra.run.smithi076.stdout:bash 133832 ubuntu mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-11-09T03:56:26.762 INFO:teuthology.orchestra.run.smithi076.stdout:bash 133832 ubuntu mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-11-09T03:56:26.762 INFO:teuthology.orchestra.run.smithi076.stdout:bash 133832 ubuntu mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-11-09T03:56:26.762 INFO:teuthology.orchestra.run.smithi076.stdout:bash 133832 ubuntu mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-11-09T03:56:26.762 INFO:teuthology.orchestra.run.smithi076.stdout:bash 133832 ubuntu mem REG 8,1 195080 9664 /usr/lib64/libtinfo.so.6.2 2023-11-09T03:56:26.762 INFO:teuthology.orchestra.run.smithi076.stdout:bash 133832 ubuntu mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:26.763 INFO:teuthology.orchestra.run.smithi076.stdout:bash 133832 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.763 INFO:teuthology.orchestra.run.smithi076.stdout:bash 133832 ubuntu mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-11-09T03:56:26.763 INFO:teuthology.orchestra.run.smithi076.stdout:bash 133832 ubuntu 0r FIFO 0,13 0t0 274487 pipe 2023-11-09T03:56:26.763 INFO:teuthology.orchestra.run.smithi076.stdout:bash 133832 ubuntu 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:26.763 INFO:teuthology.orchestra.run.smithi076.stdout:bash 133832 ubuntu 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:26.764 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:26.764 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.764 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root txt REG 8,1 185296 11759 /usr/bin/sudo 2023-11-09T03:56:26.764 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 357408 10120 /usr/lib64/libnss_systemd.so.2 2023-11-09T03:56:26.764 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:26.764 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.765 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-11-09T03:56:26.765 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:26.765 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 19600 10967 /usr/lib64/libpam_misc.so.0.82.1 2023-11-09T03:56:26.765 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 19400 32305 /usr/lib64/security/pam_succeed_if.so 2023-11-09T03:56:26.765 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 514280 32422 /usr/lib64/security/pam_systemd.so 2023-11-09T03:56:26.765 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 27536 32284 /usr/lib64/security/pam_limits.so 2023-11-09T03:56:26.766 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 48392 10228 /usr/lib64/libcrack.so.2.9.0 2023-11-09T03:56:26.766 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 32608 10961 /usr/lib64/libpwquality.so.1.0.2 2023-11-09T03:56:26.766 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 15352 32282 /usr/lib64/security/pam_keyinit.so 2023-11-09T03:56:26.766 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 15840 32264 /usr/lib64/security/pam_pwquality.so 2023-11-09T03:56:26.766 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 14928 32270 /usr/lib64/security/pam_deny.so 2023-11-09T03:56:26.767 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 56712 32310 /usr/lib64/security/pam_unix.so 2023-11-09T03:56:26.767 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.767 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.767 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.767 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.767 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.768 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.768 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.768 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.768 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.768 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.769 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.769 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.769 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-11-09T03:56:26.769 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.769 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.769 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 472568 54513 /usr/libexec/sudo/sudoers.so 2023-11-09T03:56:26.770 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:26.770 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.770 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.770 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 15184 32274 /usr/lib64/security/pam_faildelay.so 2023-11-09T03:56:26.770 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.771 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-11-09T03:56:26.771 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.771 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-11-09T03:56:26.771 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 19464 32272 /usr/lib64/security/pam_env.so 2023-11-09T03:56:26.771 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-11-09T03:56:26.771 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-11-09T03:56:26.772 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-11-09T03:56:26.772 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-11-09T03:56:26.772 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-11-09T03:56:26.772 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-11-09T03:56:26.772 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-11-09T03:56:26.772 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-11-09T03:56:26.773 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-11-09T03:56:26.773 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:26.773 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 119832 54509 /usr/libexec/sudo/libsudo_util.so.0.0.0 2023-11-09T03:56:26.773 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.773 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:26.773 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:26.774 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.774 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-11-09T03:56:26.774 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root 0r FIFO 0,13 0t0 274487 pipe 2023-11-09T03:56:26.774 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:26.774 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:26.775 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root 3r FIFO 0,13 0t0 268987 pipe 2023-11-09T03:56:26.775 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root 4w FIFO 0,13 0t0 268987 pipe 2023-11-09T03:56:26.775 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root 5u netlink 0t0 268993 AUDIT 2023-11-09T03:56:26.775 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133862 root 6u unix 0xffff959a618faa80 0t0 268994 type=DGRAM (CONNECTED) 2023-11-09T03:56:26.775 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:26.775 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.776 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root txt REG 8,1 185296 11759 /usr/bin/sudo 2023-11-09T03:56:26.776 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 357408 10120 /usr/lib64/libnss_systemd.so.2 2023-11-09T03:56:26.776 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:26.776 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.776 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-11-09T03:56:26.776 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:26.777 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 19600 10967 /usr/lib64/libpam_misc.so.0.82.1 2023-11-09T03:56:26.777 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 19400 32305 /usr/lib64/security/pam_succeed_if.so 2023-11-09T03:56:26.777 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 514280 32422 /usr/lib64/security/pam_systemd.so 2023-11-09T03:56:26.777 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 27536 32284 /usr/lib64/security/pam_limits.so 2023-11-09T03:56:26.777 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 48392 10228 /usr/lib64/libcrack.so.2.9.0 2023-11-09T03:56:26.777 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 32608 10961 /usr/lib64/libpwquality.so.1.0.2 2023-11-09T03:56:26.778 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 15352 32282 /usr/lib64/security/pam_keyinit.so 2023-11-09T03:56:26.778 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.778 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 15840 32264 /usr/lib64/security/pam_pwquality.so 2023-11-09T03:56:26.778 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 14928 32270 /usr/lib64/security/pam_deny.so 2023-11-09T03:56:26.778 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 56712 32310 /usr/lib64/security/pam_unix.so 2023-11-09T03:56:26.779 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 15184 32274 /usr/lib64/security/pam_faildelay.so 2023-11-09T03:56:26.779 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.779 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.779 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.779 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.779 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.779 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.780 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.780 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.780 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.780 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.780 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-11-09T03:56:26.781 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.781 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.781 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-11-09T03:56:26.781 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.781 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.781 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.782 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-11-09T03:56:26.782 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 472568 54513 /usr/libexec/sudo/sudoers.so 2023-11-09T03:56:26.782 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:26.782 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.782 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 19464 32272 /usr/lib64/security/pam_env.so 2023-11-09T03:56:26.782 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-11-09T03:56:26.783 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-11-09T03:56:26.783 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-11-09T03:56:26.783 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-11-09T03:56:26.783 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-11-09T03:56:26.783 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-11-09T03:56:26.783 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-11-09T03:56:26.784 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-11-09T03:56:26.784 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-11-09T03:56:26.784 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.784 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:26.784 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 119832 54509 /usr/libexec/sudo/libsudo_util.so.0.0.0 2023-11-09T03:56:26.785 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.785 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:26.785 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:26.785 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.785 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-11-09T03:56:26.785 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root 0r FIFO 0,13 0t0 274487 pipe 2023-11-09T03:56:26.785 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:26.786 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:26.786 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root 3r FIFO 0,13 0t0 271028 pipe 2023-11-09T03:56:26.786 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root 4w FIFO 0,13 0t0 271028 pipe 2023-11-09T03:56:26.786 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root 5u sock 0,8 0t0 271034 protocol: NETLINK 2023-11-09T03:56:26.786 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 133868 root 6u sock 0,8 0t0 271035 protocol: UNIX 2023-11-09T03:56:26.787 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 133870 root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:26.787 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 133870 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.787 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 133870 root txt REG 8,1 15448 10498 /usr/bin/python3.9 2023-11-09T03:56:26.787 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 133870 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:26.787 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 133870 root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.787 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 133870 root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:26.788 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 133870 root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.788 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 133870 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.788 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 133870 root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.788 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 133870 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.788 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 133870 root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.788 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 133870 root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.789 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 133870 root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:26.789 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 133870 root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.789 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 133870 root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.789 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 133870 root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.789 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 133870 root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.789 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 133870 root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:26.790 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 133870 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:26.790 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 133870 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.790 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 133870 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:26.790 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 133870 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.790 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 133870 root 0r FIFO 0,13 0t0 274487 pipe 2023-11-09T03:56:26.790 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 133870 root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:26.791 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 133870 root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:26.791 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 133870 root 3r FIFO 0,13 0t0 273482 pipe 2023-11-09T03:56:26.791 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 133870 root 4w FIFO 0,13 0t0 273482 pipe 2023-11-09T03:56:26.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:26.791 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root txt REG 8,1 2635024 20952 /usr/bin/ceph-fuse 2023-11-09T03:56:26.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.792 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.793 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.794 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.795 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.796 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.797 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.798 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:26.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 0r FIFO 0,13 0t0 273483 pipe 2023-11-09T03:56:26.799 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:26.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:26.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 3w REG 8,1 298313 278434 /var/log/ceph/ceph-client.0.133888.log 2023-11-09T03:56:26.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 4u a_inode 0,14 0 2077 [eventpoll:5,22] 2023-11-09T03:56:26.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 5r FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 6w FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.800 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 7u a_inode 0,14 0 2077 [eventpoll:8,20] 2023-11-09T03:56:26.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 8r FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 9w FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 10u a_inode 0,14 0 2077 [eventpoll:11] 2023-11-09T03:56:26.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 11r FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 12w FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.801 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 13r FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 14w FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 15u sock 0,8 0t0 272556 protocol: UNIX-STREAM 2023-11-09T03:56:26.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 16r FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 17w FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 18r FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 19w FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 20u sock 0,8 0t0 274516 protocol: TCP 2023-11-09T03:56:26.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 22u sock 0,8 0t0 271558 protocol: TCP 2023-11-09T03:56:26.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 23u CHR 10,229 0t0 161 /dev/fuse 2023-11-09T03:56:26.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 24r FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.803 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 25w FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 26r FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 27w FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 29r FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 root 30w FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.804 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:26.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root txt REG 8,1 2635024 20952 /usr/bin/ceph-fuse 2023-11-09T03:56:26.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.805 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.808 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.809 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.810 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.811 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:26.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 0r FIFO 0,13 0t0 273483 pipe 2023-11-09T03:56:26.812 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:26.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:26.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 3w REG 8,1 298313 278434 /var/log/ceph/ceph-client.0.133888.log 2023-11-09T03:56:26.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 4u a_inode 0,14 0 2077 [eventpoll:5,22] 2023-11-09T03:56:26.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 5r FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.813 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 6w FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 7u a_inode 0,14 0 2077 [eventpoll:8,20] 2023-11-09T03:56:26.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 8r FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 9w FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 10u a_inode 0,14 0 2077 [eventpoll:11] 2023-11-09T03:56:26.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 11r FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.814 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 12w FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 13r FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 14w FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 15u sock 0,8 0t0 272556 protocol: UNIX-STREAM 2023-11-09T03:56:26.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 16r FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 17w FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.815 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 18r FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 19w FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 20u sock 0,8 0t0 274516 protocol: TCP 2023-11-09T03:56:26.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 22u sock 0,8 0t0 271558 protocol: TCP 2023-11-09T03:56:26.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 23u CHR 10,229 0t0 161 /dev/fuse 2023-11-09T03:56:26.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 24r FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.816 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 25w FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 26r FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 27w FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 29r FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133889 log root 30w FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:26.817 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root txt REG 8,1 2635024 20952 /usr/bin/ceph-fuse 2023-11-09T03:56:26.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.818 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.819 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.820 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.821 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.822 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.823 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.824 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:26.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 0r FIFO 0,13 0t0 273483 pipe 2023-11-09T03:56:26.825 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:26.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:26.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 3w REG 8,1 298313 278434 /var/log/ceph/ceph-client.0.133888.log 2023-11-09T03:56:26.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 4u a_inode 0,14 0 2077 [eventpoll:5,22] 2023-11-09T03:56:26.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 5r FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 6w FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.826 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 7u a_inode 0,14 0 2077 [eventpoll:8,20] 2023-11-09T03:56:26.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 8r FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 9w FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 10u a_inode 0,14 0 2077 [eventpoll:11] 2023-11-09T03:56:26.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 11r FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 12w FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.827 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 13r FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 14w FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 15u sock 0,8 0t0 272556 protocol: UNIX-STREAM 2023-11-09T03:56:26.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 16r FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 17w FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 18r FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.828 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 19w FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 20u sock 0,8 0t0 274516 protocol: TCP 2023-11-09T03:56:26.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 22u sock 0,8 0t0 271558 protocol: TCP 2023-11-09T03:56:26.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 23u CHR 10,229 0t0 161 /dev/fuse 2023-11-09T03:56:26.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 24r FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 25w FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.829 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 26r FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 27w FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 29r FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133891 msgr-work root 30w FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:26.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.830 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root txt REG 8,1 2635024 20952 /usr/bin/ceph-fuse 2023-11-09T03:56:26.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.831 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.832 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.833 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.835 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.836 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.837 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:26.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.838 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 0r FIFO 0,13 0t0 273483 pipe 2023-11-09T03:56:26.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:26.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:26.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 3w REG 8,1 298313 278434 /var/log/ceph/ceph-client.0.133888.log 2023-11-09T03:56:26.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 4u a_inode 0,14 0 2077 [eventpoll:5,22] 2023-11-09T03:56:26.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 5r FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.839 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 6w FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 7u a_inode 0,14 0 2077 [eventpoll:8,20] 2023-11-09T03:56:26.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 8r FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 9w FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 10u a_inode 0,14 0 2077 [eventpoll:11] 2023-11-09T03:56:26.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 11r FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.840 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 12w FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 13r FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 14w FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 15u sock 0,8 0t0 272556 protocol: UNIX-STREAM 2023-11-09T03:56:26.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 16r FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.841 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 17w FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 18r FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 19w FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 20u sock 0,8 0t0 274516 protocol: TCP 2023-11-09T03:56:26.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 22u sock 0,8 0t0 271558 protocol: TCP 2023-11-09T03:56:26.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 23u CHR 10,229 0t0 161 /dev/fuse 2023-11-09T03:56:26.842 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 24r FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 25w FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 26r FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 27w FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 29r FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.843 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133892 msgr-work root 30w FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:26.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root txt REG 8,1 2635024 20952 /usr/bin/ceph-fuse 2023-11-09T03:56:26.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.844 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.845 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.846 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.847 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.848 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.849 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.850 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.851 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:26.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 0r FIFO 0,13 0t0 273483 pipe 2023-11-09T03:56:26.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:26.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:26.852 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 3w REG 8,1 298313 278434 /var/log/ceph/ceph-client.0.133888.log 2023-11-09T03:56:26.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 4u a_inode 0,14 0 2077 [eventpoll:5,22] 2023-11-09T03:56:26.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 5r FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 6w FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 7u a_inode 0,14 0 2077 [eventpoll:8,20] 2023-11-09T03:56:26.853 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 8r FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 9w FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 10u a_inode 0,14 0 2077 [eventpoll:11] 2023-11-09T03:56:26.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 11r FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 12w FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 13r FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.854 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 14w FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 15u sock 0,8 0t0 272556 protocol: UNIX-STREAM 2023-11-09T03:56:26.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 16r FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 17w FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 18r FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 19w FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.855 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 20u sock 0,8 0t0 274516 protocol: TCP 2023-11-09T03:56:26.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 22u sock 0,8 0t0 271558 protocol: TCP 2023-11-09T03:56:26.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 23u CHR 10,229 0t0 161 /dev/fuse 2023-11-09T03:56:26.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 24r FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 25w FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.856 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 26r FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 27w FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 29r FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133893 msgr-work root 30w FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:26.857 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root txt REG 8,1 2635024 20952 /usr/bin/ceph-fuse 2023-11-09T03:56:26.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.858 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.859 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.860 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.861 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.862 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.863 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.864 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.865 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:26.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 0r FIFO 0,13 0t0 273483 pipe 2023-11-09T03:56:26.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:26.866 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:26.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 3w REG 8,1 298313 278434 /var/log/ceph/ceph-client.0.133888.log 2023-11-09T03:56:26.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 4u a_inode 0,14 0 2077 [eventpoll:5,22] 2023-11-09T03:56:26.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 5r FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 6w FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 7u a_inode 0,14 0 2077 [eventpoll:8,20] 2023-11-09T03:56:26.867 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 8r FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 9w FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 10u a_inode 0,14 0 2077 [eventpoll:11] 2023-11-09T03:56:26.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 11r FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 12w FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 13r FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.868 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 14w FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 15u sock 0,8 0t0 272556 protocol: UNIX-STREAM 2023-11-09T03:56:26.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 16r FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 17w FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 18r FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 19w FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.869 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 20u sock 0,8 0t0 274516 protocol: TCP 2023-11-09T03:56:26.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 22u sock 0,8 0t0 271558 protocol: TCP 2023-11-09T03:56:26.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 23u CHR 10,229 0t0 161 /dev/fuse 2023-11-09T03:56:26.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 24r FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 25w FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 26r FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.870 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 27w FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 29r FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133897 service root 30w FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:26.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.871 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root txt REG 8,1 2635024 20952 /usr/bin/ceph-fuse 2023-11-09T03:56:26.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.873 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.874 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.875 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.876 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.877 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.878 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:26.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 0r FIFO 0,13 0t0 273483 pipe 2023-11-09T03:56:26.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:26.879 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:26.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 3w REG 8,1 298313 278434 /var/log/ceph/ceph-client.0.133888.log 2023-11-09T03:56:26.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 4u a_inode 0,14 0 2077 [eventpoll:5,22] 2023-11-09T03:56:26.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 5r FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 6w FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 7u a_inode 0,14 0 2077 [eventpoll:8,20] 2023-11-09T03:56:26.880 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 8r FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 9w FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 10u a_inode 0,14 0 2077 [eventpoll:11] 2023-11-09T03:56:26.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 11r FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 12w FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 13r FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.881 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 14w FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 15u sock 0,8 0t0 272556 protocol: UNIX-STREAM 2023-11-09T03:56:26.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 16r FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 17w FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 18r FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 19w FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.882 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 20u sock 0,8 0t0 274516 protocol: TCP 2023-11-09T03:56:26.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 22u sock 0,8 0t0 271558 protocol: TCP 2023-11-09T03:56:26.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 23u CHR 10,229 0t0 161 /dev/fuse 2023-11-09T03:56:26.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 24r FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 25w FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 26r FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.883 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 27w FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 29r FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133898 admin_soc root 30w FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:26.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root txt REG 8,1 2635024 20952 /usr/bin/ceph-fuse 2023-11-09T03:56:26.884 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.885 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.886 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.887 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.888 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.889 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.890 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.891 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:26.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 0r FIFO 0,13 0t0 273483 pipe 2023-11-09T03:56:26.892 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:26.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:26.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 3w REG 8,1 298313 278434 /var/log/ceph/ceph-client.0.133888.log 2023-11-09T03:56:26.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 4u a_inode 0,14 0 2077 [eventpoll:5,22] 2023-11-09T03:56:26.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 5r FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.893 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 6w FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 7u a_inode 0,14 0 2077 [eventpoll:8,20] 2023-11-09T03:56:26.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 8r FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 9w FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 10u a_inode 0,14 0 2077 [eventpoll:11] 2023-11-09T03:56:26.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 11r FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.894 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 12w FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 13r FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 14w FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 15u sock 0,8 0t0 272556 protocol: UNIX-STREAM 2023-11-09T03:56:26.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 16r FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 17w FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.895 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 18r FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 19w FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 20u sock 0,8 0t0 274516 protocol: TCP 2023-11-09T03:56:26.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 22u sock 0,8 0t0 271558 protocol: TCP 2023-11-09T03:56:26.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 23u CHR 10,229 0t0 161 /dev/fuse 2023-11-09T03:56:26.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 24r FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.896 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 25w FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 26r FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 27w FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 29r FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133899 signal_ha root 30w FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:26.897 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root txt REG 8,1 2635024 20952 /usr/bin/ceph-fuse 2023-11-09T03:56:26.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.898 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.899 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.900 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.901 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.902 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.903 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.904 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:26.905 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 0r FIFO 0,13 0t0 273483 pipe 2023-11-09T03:56:26.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:26.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:26.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 3w REG 8,1 298313 278434 /var/log/ceph/ceph-client.0.133888.log 2023-11-09T03:56:26.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 4u a_inode 0,14 0 2077 [eventpoll:5,22] 2023-11-09T03:56:26.906 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 5r FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 6w FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 7u a_inode 0,14 0 2077 [eventpoll:8,20] 2023-11-09T03:56:26.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 8r FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 9w FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 10u a_inode 0,14 0 2077 [eventpoll:11] 2023-11-09T03:56:26.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 11r FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 12w FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 13r FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 14w FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 15u sock 0,8 0t0 272556 protocol: UNIX-STREAM 2023-11-09T03:56:26.908 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 16r FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 17w FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 18r FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 19w FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 20u sock 0,8 0t0 274516 protocol: TCP 2023-11-09T03:56:26.909 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 22u sock 0,8 0t0 271558 protocol: TCP 2023-11-09T03:56:26.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 23u CHR 10,229 0t0 161 /dev/fuse 2023-11-09T03:56:26.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 24r FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 25w FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 26r FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 27w FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.910 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 29r FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133900 io_contex root 30w FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:26.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root txt REG 8,1 2635024 20952 /usr/bin/ceph-fuse 2023-11-09T03:56:26.911 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.912 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.913 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.914 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.915 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.916 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.917 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.918 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:26.919 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 0r FIFO 0,13 0t0 273483 pipe 2023-11-09T03:56:26.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:26.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:26.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 3w REG 8,1 298313 278434 /var/log/ceph/ceph-client.0.133888.log 2023-11-09T03:56:26.920 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 4u a_inode 0,14 0 2077 [eventpoll:5,22] 2023-11-09T03:56:26.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 5r FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 6w FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 7u a_inode 0,14 0 2077 [eventpoll:8,20] 2023-11-09T03:56:26.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 8r FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.921 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 9w FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 10u a_inode 0,14 0 2077 [eventpoll:11] 2023-11-09T03:56:26.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 11r FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 12w FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 13r FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 14w FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.922 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 15u sock 0,8 0t0 272556 protocol: UNIX-STREAM 2023-11-09T03:56:26.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 16r FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 17w FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 18r FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 19w FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.923 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 20u sock 0,8 0t0 274516 protocol: TCP 2023-11-09T03:56:26.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 22u sock 0,8 0t0 271558 protocol: TCP 2023-11-09T03:56:26.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 23u CHR 10,229 0t0 161 /dev/fuse 2023-11-09T03:56:26.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 24r FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 25w FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 26r FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.924 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 27w FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 29r FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133901 io_contex root 30w FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:26.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.925 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root txt REG 8,1 2635024 20952 /usr/bin/ceph-fuse 2023-11-09T03:56:26.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.926 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.927 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.928 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.929 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.930 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.931 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.932 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:26.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.933 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 0r FIFO 0,13 0t0 273483 pipe 2023-11-09T03:56:26.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:26.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:26.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 3w REG 8,1 298313 278434 /var/log/ceph/ceph-client.0.133888.log 2023-11-09T03:56:26.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 4u a_inode 0,14 0 2077 [eventpoll:5,22] 2023-11-09T03:56:26.934 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 5r FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 6w FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 7u a_inode 0,14 0 2077 [eventpoll:8,20] 2023-11-09T03:56:26.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 8r FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 9w FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 10u a_inode 0,14 0 2077 [eventpoll:11] 2023-11-09T03:56:26.935 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 11r FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 12w FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 13r FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 14w FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 15u sock 0,8 0t0 272556 protocol: UNIX-STREAM 2023-11-09T03:56:26.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 16r FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.936 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 17w FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 18r FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 19w FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 20u sock 0,8 0t0 274516 protocol: TCP 2023-11-09T03:56:26.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 22u sock 0,8 0t0 271558 protocol: TCP 2023-11-09T03:56:26.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 23u CHR 10,229 0t0 161 /dev/fuse 2023-11-09T03:56:26.937 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 24r FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 25w FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 26r FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 27w FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 29r FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.938 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133902 ceph_time root 30w FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:26.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root txt REG 8,1 2635024 20952 /usr/bin/ceph-fuse 2023-11-09T03:56:26.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.939 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.940 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.941 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.942 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.943 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.944 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.945 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.946 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:26.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 0r FIFO 0,13 0t0 273483 pipe 2023-11-09T03:56:26.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:26.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:26.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 3w REG 8,1 298313 278434 /var/log/ceph/ceph-client.0.133888.log 2023-11-09T03:56:26.947 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 4u a_inode 0,14 0 2077 [eventpoll:5,22] 2023-11-09T03:56:26.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 5r FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 6w FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 7u a_inode 0,14 0 2077 [eventpoll:8,20] 2023-11-09T03:56:26.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 8r FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 9w FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.948 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 10u a_inode 0,14 0 2077 [eventpoll:11] 2023-11-09T03:56:26.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 11r FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 12w FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 13r FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 14w FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 15u sock 0,8 0t0 272556 protocol: UNIX-STREAM 2023-11-09T03:56:26.949 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 16r FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 17w FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 18r FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 19w FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 20u sock 0,8 0t0 274516 protocol: TCP 2023-11-09T03:56:26.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 22u sock 0,8 0t0 271558 protocol: TCP 2023-11-09T03:56:26.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 23u CHR 10,229 0t0 161 /dev/fuse 2023-11-09T03:56:26.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 24r FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 25w FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 26r FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.951 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 27w FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 29r FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133903 fn_anonym root 30w FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:26.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root txt REG 8,1 2635024 20952 /usr/bin/ceph-fuse 2023-11-09T03:56:26.952 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.953 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.954 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.955 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.956 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.960 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.961 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.962 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.963 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:26.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 0r FIFO 0,13 0t0 273483 pipe 2023-11-09T03:56:26.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:26.964 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:26.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 3w REG 8,1 298313 278434 /var/log/ceph/ceph-client.0.133888.log 2023-11-09T03:56:26.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 4u a_inode 0,14 0 2077 [eventpoll:5,22] 2023-11-09T03:56:26.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 5r FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 6w FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.965 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 7u a_inode 0,14 0 2077 [eventpoll:8,20] 2023-11-09T03:56:26.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 8r FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 9w FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 10u a_inode 0,14 0 2077 [eventpoll:11] 2023-11-09T03:56:26.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 11r FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.966 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 12w FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 13r FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 14w FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 15u sock 0,8 0t0 272556 protocol: UNIX-STREAM 2023-11-09T03:56:26.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 16r FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 17w FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.967 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 18r FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 19w FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 20u sock 0,8 0t0 274516 protocol: TCP 2023-11-09T03:56:26.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 22u sock 0,8 0t0 271558 protocol: TCP 2023-11-09T03:56:26.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 23u CHR 10,229 0t0 161 /dev/fuse 2023-11-09T03:56:26.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 24r FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.968 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 25w FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 26r FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 27w FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 29r FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133904 fn_anonym root 30w FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.969 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:26.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root txt REG 8,1 2635024 20952 /usr/bin/ceph-fuse 2023-11-09T03:56:26.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.970 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.971 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.972 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.973 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.974 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.975 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.976 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:26.977 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 0r FIFO 0,13 0t0 273483 pipe 2023-11-09T03:56:26.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:26.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:26.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 3w REG 8,1 298313 278434 /var/log/ceph/ceph-client.0.133888.log 2023-11-09T03:56:26.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 4u a_inode 0,14 0 2077 [eventpoll:5,22] 2023-11-09T03:56:26.978 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 5r FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 6w FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 7u a_inode 0,14 0 2077 [eventpoll:8,20] 2023-11-09T03:56:26.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 8r FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 9w FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 10u a_inode 0,14 0 2077 [eventpoll:11] 2023-11-09T03:56:26.979 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 11r FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 12w FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 13r FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 14w FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 15u sock 0,8 0t0 272556 protocol: UNIX-STREAM 2023-11-09T03:56:26.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 16r FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.980 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 17w FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 18r FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 19w FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 20u sock 0,8 0t0 274516 protocol: TCP 2023-11-09T03:56:26.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 22u sock 0,8 0t0 271558 protocol: TCP 2023-11-09T03:56:26.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 23u CHR 10,229 0t0 161 /dev/fuse 2023-11-09T03:56:26.981 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 24r FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 25w FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 26r FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 27w FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 29r FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133905 fn_anonym root 30w FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.982 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:26.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root txt REG 8,1 2635024 20952 /usr/bin/ceph-fuse 2023-11-09T03:56:26.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.983 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.984 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.985 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:26.986 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:26.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:26.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:26.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:26.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:26.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:26.987 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:26.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:26.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:26.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:26.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:26.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:26.988 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:26.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:26.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:26.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:26.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:26.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:26.989 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:26.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:26.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:26.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:26.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:26.990 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:26.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 0r FIFO 0,13 0t0 273483 pipe 2023-11-09T03:56:26.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:26.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:26.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 3w REG 8,1 298313 278434 /var/log/ceph/ceph-client.0.133888.log 2023-11-09T03:56:26.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 4u a_inode 0,14 0 2077 [eventpoll:5,22] 2023-11-09T03:56:26.991 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 5r FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 6w FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:26.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 7u a_inode 0,14 0 2077 [eventpoll:8,20] 2023-11-09T03:56:26.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 8r FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 9w FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:26.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 10u a_inode 0,14 0 2077 [eventpoll:11] 2023-11-09T03:56:26.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 11r FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.992 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 12w FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:26.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 13r FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 14w FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:26.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 15u sock 0,8 0t0 272556 protocol: UNIX-STREAM 2023-11-09T03:56:26.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 16r FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.993 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 17w FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:26.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 18r FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 19w FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:26.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 20u sock 0,8 0t0 274516 protocol: TCP 2023-11-09T03:56:26.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 22u sock 0,8 0t0 271558 protocol: TCP 2023-11-09T03:56:26.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 23u CHR 10,229 0t0 161 /dev/fuse 2023-11-09T03:56:26.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 24r FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 25w FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:26.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 26r FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 27w FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:26.995 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 29r FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133906 fn_anonym root 30w FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:26.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:26.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:26.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root txt REG 8,1 2635024 20952 /usr/bin/ceph-fuse 2023-11-09T03:56:26.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:26.996 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:26.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:26.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:26.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:26.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:26.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:26.997 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:26.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:26.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:26.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:26.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:26.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:26.998 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:26.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:26.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:26.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:26.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:26.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:26.999 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:27.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:27.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:27.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:27.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:27.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:27.000 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:27.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:27.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:27.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:27.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:27.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:27.001 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:27.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:27.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:27.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:27.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:27.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:27.002 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:27.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:27.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:27.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:27.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:27.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:27.003 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:27.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 0r FIFO 0,13 0t0 273483 pipe 2023-11-09T03:56:27.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:27.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:27.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 3w REG 8,1 298313 278434 /var/log/ceph/ceph-client.0.133888.log 2023-11-09T03:56:27.004 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 4u a_inode 0,14 0 2077 [eventpoll:5,22] 2023-11-09T03:56:27.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 5r FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:27.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 6w FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:27.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 7u a_inode 0,14 0 2077 [eventpoll:8,20] 2023-11-09T03:56:27.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 8r FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:27.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 9w FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:27.005 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 10u a_inode 0,14 0 2077 [eventpoll:11] 2023-11-09T03:56:27.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 11r FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:27.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 12w FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:27.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 13r FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:27.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 14w FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:27.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 15u sock 0,8 0t0 272556 protocol: UNIX-STREAM 2023-11-09T03:56:27.006 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 16r FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:27.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 17w FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:27.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 18r FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:27.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 19w FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:27.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 20u sock 0,8 0t0 274516 protocol: TCP 2023-11-09T03:56:27.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 22u sock 0,8 0t0 271558 protocol: TCP 2023-11-09T03:56:27.007 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 23u CHR 10,229 0t0 161 /dev/fuse 2023-11-09T03:56:27.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 24r FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:27.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 25w FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:27.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 26r FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:27.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 27w FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:27.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 29r FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:27.008 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133907 fn_anonym root 30w FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:27.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:27.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:27.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root txt REG 8,1 2635024 20952 /usr/bin/ceph-fuse 2023-11-09T03:56:27.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:27.009 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:27.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:27.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:27.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:27.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:27.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:27.010 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:27.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:27.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:27.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:27.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:27.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:27.011 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:27.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:27.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:27.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:27.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:27.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:27.012 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:27.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:27.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:27.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:27.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:27.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:27.013 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:27.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:27.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:27.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:27.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:27.014 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:27.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:27.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:27.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:27.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:27.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:27.015 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:27.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:27.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:27.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:27.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:27.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:27.016 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:27.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:27.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 0r FIFO 0,13 0t0 273483 pipe 2023-11-09T03:56:27.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:27.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:27.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 3w REG 8,1 298313 278434 /var/log/ceph/ceph-client.0.133888.log 2023-11-09T03:56:27.017 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 4u a_inode 0,14 0 2077 [eventpoll:5,22] 2023-11-09T03:56:27.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 5r FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:27.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 6w FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:27.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 7u a_inode 0,14 0 2077 [eventpoll:8,20] 2023-11-09T03:56:27.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 8r FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:27.018 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 9w FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:27.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 10u a_inode 0,14 0 2077 [eventpoll:11] 2023-11-09T03:56:27.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 11r FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:27.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 12w FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:27.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 13r FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:27.019 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 14w FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:27.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 15u sock 0,8 0t0 272556 protocol: UNIX-STREAM 2023-11-09T03:56:27.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 16r FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:27.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 17w FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:27.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 18r FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:27.020 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 19w FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:27.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 20u sock 0,8 0t0 274516 protocol: TCP 2023-11-09T03:56:27.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 22u sock 0,8 0t0 271558 protocol: TCP 2023-11-09T03:56:27.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 23u CHR 10,229 0t0 161 /dev/fuse 2023-11-09T03:56:27.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 24r FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:27.021 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 25w FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:27.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 26r FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:27.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 27w FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:27.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 29r FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:27.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133908 safe_time root 30w FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:27.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:27.022 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:27.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root txt REG 8,1 2635024 20952 /usr/bin/ceph-fuse 2023-11-09T03:56:27.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:27.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:27.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:27.023 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:27.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:27.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:27.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:27.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:27.024 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:27.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:27.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:27.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:27.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:27.025 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:27.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:27.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:27.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:27.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:27.026 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:27.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:27.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:27.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:27.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:27.027 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:27.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:27.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:27.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:27.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:27.028 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:27.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:27.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:27.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:27.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:27.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:27.029 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:27.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:27.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:27.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:27.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:27.030 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:27.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:27.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:27.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:27.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:27.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 0r FIFO 0,13 0t0 273483 pipe 2023-11-09T03:56:27.031 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:27.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:27.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 3w REG 8,1 298313 278434 /var/log/ceph/ceph-client.0.133888.log 2023-11-09T03:56:27.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 4u a_inode 0,14 0 2077 [eventpoll:5,22] 2023-11-09T03:56:27.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 5r FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:27.032 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 6w FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:27.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 7u a_inode 0,14 0 2077 [eventpoll:8,20] 2023-11-09T03:56:27.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 8r FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:27.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 9w FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:27.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 10u a_inode 0,14 0 2077 [eventpoll:11] 2023-11-09T03:56:27.033 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 11r FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:27.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 12w FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:27.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 13r FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:27.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 14w FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:27.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 15u sock 0,8 0t0 272556 protocol: UNIX-STREAM 2023-11-09T03:56:27.034 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 16r FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:27.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 17w FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:27.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 18r FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:27.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 19w FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:27.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 20u sock 0,8 0t0 274516 protocol: TCP 2023-11-09T03:56:27.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 22u sock 0,8 0t0 271558 protocol: TCP 2023-11-09T03:56:27.035 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 23u CHR 10,229 0t0 161 /dev/fuse 2023-11-09T03:56:27.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 24r FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:27.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 25w FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:27.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 26r FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:27.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 27w FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:27.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 29r FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:27.036 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133909 fn_anonym root 30w FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:27.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:27.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:27.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root txt REG 8,1 2635024 20952 /usr/bin/ceph-fuse 2023-11-09T03:56:27.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:27.037 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:27.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:27.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:27.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:27.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:27.038 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:27.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:27.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:27.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:27.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:27.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:27.039 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:27.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:27.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:27.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:27.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:27.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:27.040 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:27.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:27.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:27.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:27.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:27.041 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:27.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:27.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:27.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:27.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:27.042 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:27.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:27.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:27.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:27.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:27.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:27.043 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:27.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:27.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:27.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:27.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:27.044 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:27.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:27.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:27.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:27.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:27.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 0r FIFO 0,13 0t0 273483 pipe 2023-11-09T03:56:27.045 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:27.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:27.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 3w REG 8,1 298313 278434 /var/log/ceph/ceph-client.0.133888.log 2023-11-09T03:56:27.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 4u a_inode 0,14 0 2077 [eventpoll:5,22] 2023-11-09T03:56:27.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 5r FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:27.046 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 6w FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:27.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 7u a_inode 0,14 0 2077 [eventpoll:8,20] 2023-11-09T03:56:27.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 8r FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:27.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 9w FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:27.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 10u a_inode 0,14 0 2077 [eventpoll:11] 2023-11-09T03:56:27.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 11r FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:27.047 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 12w FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:27.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 13r FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:27.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 14w FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:27.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 15u sock 0,8 0t0 272556 protocol: UNIX-STREAM 2023-11-09T03:56:27.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 16r FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:27.048 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 17w FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:27.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 18r FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:27.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 19w FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:27.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 20u sock 0,8 0t0 274516 protocol: TCP 2023-11-09T03:56:27.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 22u sock 0,8 0t0 271558 protocol: TCP 2023-11-09T03:56:27.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 23u CHR 10,229 0t0 161 /dev/fuse 2023-11-09T03:56:27.049 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 24r FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:27.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 25w FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:27.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 26r FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:27.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 27w FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:27.050 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 29r FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:27.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133910 flusher root 30w FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:27.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:27.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:27.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root txt REG 8,1 2635024 20952 /usr/bin/ceph-fuse 2023-11-09T03:56:27.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:27.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:27.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:27.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:27.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:27.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:27.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:27.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:27.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:27.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:27.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:27.053 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:27.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:27.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:27.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:27.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:27.054 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:27.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:27.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:27.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:27.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:27.055 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:27.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:27.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:27.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:27.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:27.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:27.056 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:27.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:27.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:27.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:27.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:27.057 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:27.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:27.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:27.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:27.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:27.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:27.058 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:27.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:27.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:27.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:27.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:27.059 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:27.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 0r FIFO 0,13 0t0 273483 pipe 2023-11-09T03:56:27.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:27.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:27.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 3w REG 8,1 298313 278434 /var/log/ceph/ceph-client.0.133888.log 2023-11-09T03:56:27.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 4u a_inode 0,14 0 2077 [eventpoll:5,22] 2023-11-09T03:56:27.060 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 5r FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:27.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 6w FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:27.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 7u a_inode 0,14 0 2077 [eventpoll:8,20] 2023-11-09T03:56:27.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 8r FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:27.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 9w FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:27.061 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 10u a_inode 0,14 0 2077 [eventpoll:11] 2023-11-09T03:56:27.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 11r FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:27.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 12w FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:27.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 13r FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:27.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 14w FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:27.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 15u sock 0,8 0t0 272556 protocol: UNIX-STREAM 2023-11-09T03:56:27.062 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 16r FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:27.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 17w FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:27.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 18r FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:27.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 19w FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:27.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 20u sock 0,8 0t0 274516 protocol: TCP 2023-11-09T03:56:27.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 22u sock 0,8 0t0 271558 protocol: TCP 2023-11-09T03:56:27.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 23u CHR 10,229 0t0 161 /dev/fuse 2023-11-09T03:56:27.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 24r FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:27.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 25w FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:27.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 26r FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:27.064 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 27w FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:27.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 29r FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:27.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133911 ms_dispat root 30w FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:27.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:27.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:27.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root txt REG 8,1 2635024 20952 /usr/bin/ceph-fuse 2023-11-09T03:56:27.065 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:27.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:27.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:27.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:27.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:27.066 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:27.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:27.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:27.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:27.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:27.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:27.067 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:27.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:27.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:27.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:27.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:27.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:27.068 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:27.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:27.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:27.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:27.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:27.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:27.069 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:27.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:27.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:27.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:27.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:27.070 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:27.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:27.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:27.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:27.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:27.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:27.071 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:27.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:27.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:27.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:27.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:27.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:27.072 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:27.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:27.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:27.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:27.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 0r FIFO 0,13 0t0 273483 pipe 2023-11-09T03:56:27.073 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:27.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:27.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 3w REG 8,1 298313 278434 /var/log/ceph/ceph-client.0.133888.log 2023-11-09T03:56:27.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 4u a_inode 0,14 0 2077 [eventpoll:5,22] 2023-11-09T03:56:27.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 5r FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:27.074 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 6w FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:27.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 7u a_inode 0,14 0 2077 [eventpoll:8,20] 2023-11-09T03:56:27.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 8r FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:27.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 9w FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:27.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 10u a_inode 0,14 0 2077 [eventpoll:11] 2023-11-09T03:56:27.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 11r FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:27.075 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 12w FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:27.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 13r FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:27.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 14w FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:27.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 15u sock 0,8 0t0 272556 protocol: UNIX-STREAM 2023-11-09T03:56:27.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 16r FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:27.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 17w FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:27.076 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 18r FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:27.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 19w FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:27.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 20u sock 0,8 0t0 274516 protocol: TCP 2023-11-09T03:56:27.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 22u sock 0,8 0t0 271558 protocol: TCP 2023-11-09T03:56:27.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 23u CHR 10,229 0t0 161 /dev/fuse 2023-11-09T03:56:27.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 24r FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:27.077 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 25w FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:27.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 26r FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:27.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 27w FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:27.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 29r FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:27.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133912 ms_local root 30w FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:27.078 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:27.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:27.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root txt REG 8,1 2635024 20952 /usr/bin/ceph-fuse 2023-11-09T03:56:27.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:27.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:27.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:27.079 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:27.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:27.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:27.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:27.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:27.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:27.080 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:27.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:27.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:27.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:27.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:27.081 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:27.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:27.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:27.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:27.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:27.082 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:27.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:27.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:27.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:27.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:27.083 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:27.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:27.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:27.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:27.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:27.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:27.084 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:27.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:27.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:27.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:27.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:27.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:27.085 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:27.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:27.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:27.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:27.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:27.086 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:27.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:27.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:27.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 0r FIFO 0,13 0t0 273483 pipe 2023-11-09T03:56:27.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:27.087 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:27.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 3w REG 8,1 298313 278434 /var/log/ceph/ceph-client.0.133888.log 2023-11-09T03:56:27.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 4u a_inode 0,14 0 2077 [eventpoll:5,22] 2023-11-09T03:56:27.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 5r FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:27.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 6w FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:27.088 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 7u a_inode 0,14 0 2077 [eventpoll:8,20] 2023-11-09T03:56:27.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 8r FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:27.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 9w FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:27.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 10u a_inode 0,14 0 2077 [eventpoll:11] 2023-11-09T03:56:27.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 11r FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:27.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 12w FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:27.089 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 13r FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:27.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 14w FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:27.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 15u sock 0,8 0t0 272556 protocol: UNIX-STREAM 2023-11-09T03:56:27.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 16r FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:27.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 17w FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:27.090 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 18r FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:27.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 19w FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:27.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 20u sock 0,8 0t0 274516 protocol: TCP 2023-11-09T03:56:27.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 22u sock 0,8 0t0 271558 protocol: TCP 2023-11-09T03:56:27.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 23u CHR 10,229 0t0 161 /dev/fuse 2023-11-09T03:56:27.091 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 24r FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:27.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 25w FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:27.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 26r FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:27.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 27w FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:27.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 29r FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:27.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133913 safe_time root 30w FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:27.092 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:27.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:27.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root txt REG 8,1 2635024 20952 /usr/bin/ceph-fuse 2023-11-09T03:56:27.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:27.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:27.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:27.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:27.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:27.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:27.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:27.094 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:27.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:27.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:27.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:27.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:27.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:27.095 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:27.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:27.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:27.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:27.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:27.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:27.096 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:27.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:27.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:27.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:27.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:27.097 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:27.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:27.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:27.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:27.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:27.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:27.098 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:27.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:27.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:27.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:27.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:27.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:27.099 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:27.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:27.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:27.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:27.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:27.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:27.100 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:27.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:27.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 0r FIFO 0,13 0t0 273483 pipe 2023-11-09T03:56:27.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:27.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:27.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 3w REG 8,1 298313 278434 /var/log/ceph/ceph-client.0.133888.log 2023-11-09T03:56:27.101 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 4u a_inode 0,14 0 2077 [eventpoll:5,22] 2023-11-09T03:56:27.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 5r FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:27.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 6w FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:27.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 7u a_inode 0,14 0 2077 [eventpoll:8,20] 2023-11-09T03:56:27.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 8r FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:27.102 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 9w FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:27.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 10u a_inode 0,14 0 2077 [eventpoll:11] 2023-11-09T03:56:27.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 11r FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:27.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 12w FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:27.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 13r FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:27.103 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 14w FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:27.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 15u sock 0,8 0t0 272556 protocol: UNIX-STREAM 2023-11-09T03:56:27.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 16r FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:27.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 17w FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:27.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 18r FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:27.104 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 19w FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:27.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 20u sock 0,8 0t0 274516 protocol: TCP 2023-11-09T03:56:27.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 22u sock 0,8 0t0 271558 protocol: TCP 2023-11-09T03:56:27.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 23u CHR 10,229 0t0 161 /dev/fuse 2023-11-09T03:56:27.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 24r FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:27.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 25w FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:27.105 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 26r FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:27.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 27w FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:27.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 29r FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:27.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133914 ceph-fuse root 30w FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:27.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:27.106 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:27.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root txt REG 8,1 2635024 20952 /usr/bin/ceph-fuse 2023-11-09T03:56:27.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:27.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:27.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:27.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:27.107 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:27.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:27.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:27.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:27.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:27.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:27.108 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:27.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:27.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:27.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:27.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:27.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:27.109 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:27.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:27.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:27.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:27.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:27.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:27.110 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:27.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:27.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:27.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:27.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:27.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:27.111 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:27.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:27.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:27.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:27.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:27.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:27.112 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:27.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:27.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:27.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:27.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:27.113 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:27.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:27.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:27.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:27.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:27.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 0r FIFO 0,13 0t0 273483 pipe 2023-11-09T03:56:27.114 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:27.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:27.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 3w REG 8,1 298313 278434 /var/log/ceph/ceph-client.0.133888.log 2023-11-09T03:56:27.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 4u a_inode 0,14 0 2077 [eventpoll:5,22] 2023-11-09T03:56:27.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 5r FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:27.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 6w FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:27.115 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 7u a_inode 0,14 0 2077 [eventpoll:8,20] 2023-11-09T03:56:27.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 8r FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:27.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 9w FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:27.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 10u a_inode 0,14 0 2077 [eventpoll:11] 2023-11-09T03:56:27.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 11r FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:27.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 12w FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:27.116 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 13r FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:27.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 14w FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:27.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 15u sock 0,8 0t0 272556 protocol: UNIX-STREAM 2023-11-09T03:56:27.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 16r FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:27.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 17w FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:27.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 18r FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:27.117 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 19w FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:27.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 20u sock 0,8 0t0 274516 protocol: TCP 2023-11-09T03:56:27.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 22u sock 0,8 0t0 271558 protocol: TCP 2023-11-09T03:56:27.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 23u CHR 10,229 0t0 161 /dev/fuse 2023-11-09T03:56:27.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 24r FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:27.118 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 25w FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:27.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 26r FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:27.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 27w FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:27.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 29r FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:27.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133916 ceph-fuse root 30w FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:27.119 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:27.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:27.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root txt REG 8,1 2635024 20952 /usr/bin/ceph-fuse 2023-11-09T03:56:27.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:27.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:27.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:27.120 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:27.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:27.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:27.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:27.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:27.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:27.121 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:27.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:27.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:27.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:27.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:27.122 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:27.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:27.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:27.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:27.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:27.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:27.123 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:27.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:27.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:27.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:27.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:27.124 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:27.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:27.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:27.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:27.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:27.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:27.125 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:27.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:27.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:27.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:27.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:27.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:27.126 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:27.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:27.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:27.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:27.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:27.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:27.127 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:27.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 0r FIFO 0,13 0t0 273483 pipe 2023-11-09T03:56:27.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:27.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:27.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 3w REG 8,1 298313 278434 /var/log/ceph/ceph-client.0.133888.log 2023-11-09T03:56:27.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 4u a_inode 0,14 0 2077 [eventpoll:5,22] 2023-11-09T03:56:27.128 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 5r FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:27.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 6w FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:27.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 7u a_inode 0,14 0 2077 [eventpoll:8,20] 2023-11-09T03:56:27.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 8r FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:27.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 9w FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:27.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 10u a_inode 0,14 0 2077 [eventpoll:11] 2023-11-09T03:56:27.129 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 11r FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:27.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 12w FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:27.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 13r FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:27.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 14w FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:27.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 15u sock 0,8 0t0 272556 protocol: UNIX-STREAM 2023-11-09T03:56:27.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 16r FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:27.130 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 17w FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:27.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 18r FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:27.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 19w FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:27.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 20u sock 0,8 0t0 274516 protocol: TCP 2023-11-09T03:56:27.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 22u sock 0,8 0t0 271558 protocol: TCP 2023-11-09T03:56:27.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 23u CHR 10,229 0t0 161 /dev/fuse 2023-11-09T03:56:27.131 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 24r FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:27.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 25w FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:27.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 26r FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:27.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 27w FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:27.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 29r FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:27.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 133918 ceph-fuse root 30w FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:27.132 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root cwd DIR 8,1 4096 262166 /home/ubuntu/cephtest 2023-11-09T03:56:27.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:27.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root txt REG 8,1 2635024 20952 /usr/bin/ceph-fuse 2023-11-09T03:56:27.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-11-09T03:56:27.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-11-09T03:56:27.133 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-11-09T03:56:27.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 53816 1695 /usr/lib64/libnuma.so.1.0.0 2023-11-09T03:56:27.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 53272 2022 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-11-09T03:56:27.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:27.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:27.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:27.134 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-11-09T03:56:27.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:27.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:27.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:27.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-11-09T03:56:27.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:27.135 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:27.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:27.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-11-09T03:56:27.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:27.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:27.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:27.136 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:27.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-11-09T03:56:27.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-11-09T03:56:27.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:27.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-11-09T03:56:27.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-11-09T03:56:27.137 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 639224 1986 /usr/lib64/libthrift-0.15.0.so 2023-11-09T03:56:27.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-11-09T03:56:27.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-11-09T03:56:27.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-11-09T03:56:27.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-11-09T03:56:27.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:27.138 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-11-09T03:56:27.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-11-09T03:56:27.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-11-09T03:56:27.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-11-09T03:56:27.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:27.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 9894040 265483 /usr/lib64/ceph/libceph-common.so.2 2023-11-09T03:56:27.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:27.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:27.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:27.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:27.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-11-09T03:56:27.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:27.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 0r FIFO 0,13 0t0 273483 pipe 2023-11-09T03:56:27.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 1w FIFO 0,13 0t0 274488 pipe 2023-11-09T03:56:27.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 2w FIFO 0,13 0t0 274489 pipe 2023-11-09T03:56:27.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 3w REG 8,1 298313 278434 /var/log/ceph/ceph-client.0.133888.log 2023-11-09T03:56:27.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 4u a_inode 0,14 0 2077 [eventpoll:5,22] 2023-11-09T03:56:27.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 5r FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:27.141 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 6w FIFO 0,13 0t0 272552 pipe 2023-11-09T03:56:27.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 7u a_inode 0,14 0 2077 [eventpoll:8,20] 2023-11-09T03:56:27.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 8r FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:27.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 9w FIFO 0,13 0t0 272553 pipe 2023-11-09T03:56:27.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 10u a_inode 0,14 0 2077 [eventpoll:11] 2023-11-09T03:56:27.142 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 11r FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:27.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 12w FIFO 0,13 0t0 272554 pipe 2023-11-09T03:56:27.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 13r FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:27.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 14w FIFO 0,13 0t0 272555 pipe 2023-11-09T03:56:27.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 15u sock 0,8 0t0 272556 protocol: UNIX-STREAM 2023-11-09T03:56:27.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 16r FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:27.143 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 17w FIFO 0,13 0t0 272557 pipe 2023-11-09T03:56:27.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 18r FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:27.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 19w FIFO 0,13 0t0 272558 pipe 2023-11-09T03:56:27.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 20u sock 0,8 0t0 274516 protocol: TCP 2023-11-09T03:56:27.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 22u sock 0,8 0t0 271558 protocol: TCP 2023-11-09T03:56:27.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 23u CHR 10,229 0t0 161 /dev/fuse 2023-11-09T03:56:27.144 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 24r FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:27.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 25w FIFO 0,13 0t0 271050 pipe 2023-11-09T03:56:27.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 26r FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:27.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 27w FIFO 0,13 0t0 270143 pipe 2023-11-09T03:56:27.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 29r FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:27.145 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-fuse 133888 134113 ceph-fuse root 30w FIFO 0,13 0t0 274557 pipe 2023-11-09T03:56:27.145 INFO:teuthology.orchestra.run.smithi076.stdout:bash 134322 ubuntu cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:27.146 INFO:teuthology.orchestra.run.smithi076.stdout:bash 134322 ubuntu rtd DIR 8,1 4096 2 / 2023-11-09T03:56:27.146 INFO:teuthology.orchestra.run.smithi076.stdout:bash 134322 ubuntu txt REG 8,1 1388920 9667 /usr/bin/bash 2023-11-09T03:56:27.146 INFO:teuthology.orchestra.run.smithi076.stdout:bash 134322 ubuntu mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-11-09T03:56:27.146 INFO:teuthology.orchestra.run.smithi076.stdout:bash 134322 ubuntu mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:27.146 INFO:teuthology.orchestra.run.smithi076.stdout:bash 134322 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:27.146 INFO:teuthology.orchestra.run.smithi076.stdout:bash 134322 ubuntu mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-11-09T03:56:27.147 INFO:teuthology.orchestra.run.smithi076.stdout:bash 134322 ubuntu mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-11-09T03:56:27.147 INFO:teuthology.orchestra.run.smithi076.stdout:bash 134322 ubuntu mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-11-09T03:56:27.147 INFO:teuthology.orchestra.run.smithi076.stdout:bash 134322 ubuntu mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-11-09T03:56:27.147 INFO:teuthology.orchestra.run.smithi076.stdout:bash 134322 ubuntu mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-11-09T03:56:27.147 INFO:teuthology.orchestra.run.smithi076.stdout:bash 134322 ubuntu mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-11-09T03:56:27.147 INFO:teuthology.orchestra.run.smithi076.stdout:bash 134322 ubuntu mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-11-09T03:56:27.148 INFO:teuthology.orchestra.run.smithi076.stdout:bash 134322 ubuntu mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-11-09T03:56:27.148 INFO:teuthology.orchestra.run.smithi076.stdout:bash 134322 ubuntu mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-11-09T03:56:27.148 INFO:teuthology.orchestra.run.smithi076.stdout:bash 134322 ubuntu mem REG 8,1 195080 9664 /usr/lib64/libtinfo.so.6.2 2023-11-09T03:56:27.148 INFO:teuthology.orchestra.run.smithi076.stdout:bash 134322 ubuntu mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:27.148 INFO:teuthology.orchestra.run.smithi076.stdout:bash 134322 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:27.148 INFO:teuthology.orchestra.run.smithi076.stdout:bash 134322 ubuntu mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-11-09T03:56:27.149 INFO:teuthology.orchestra.run.smithi076.stdout:bash 134322 ubuntu 0r FIFO 0,13 0t0 272766 pipe 2023-11-09T03:56:27.149 INFO:teuthology.orchestra.run.smithi076.stdout:bash 134322 ubuntu 1w FIFO 0,13 0t0 272767 pipe 2023-11-09T03:56:27.149 INFO:teuthology.orchestra.run.smithi076.stdout:bash 134322 ubuntu 2w FIFO 0,13 0t0 272768 pipe 2023-11-09T03:56:27.149 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir 134338 ubuntu cwd DIR 0,43 0 1 /home/ubuntu/cephtest/mnt.0 2023-11-09T03:56:27.149 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir 134338 ubuntu rtd DIR 8,1 4096 2 / 2023-11-09T03:56:27.149 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir 134338 ubuntu txt REG 8,1 69752 10054 /usr/bin/mkdir 2023-11-09T03:56:27.150 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir 134338 ubuntu mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-11-09T03:56:27.150 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir 134338 ubuntu mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:27.150 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir 134338 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:27.150 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir 134338 ubuntu mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:27.150 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir 134338 ubuntu mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-11-09T03:56:27.151 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir 134338 ubuntu mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-11-09T03:56:27.151 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir 134338 ubuntu mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-11-09T03:56:27.151 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir 134338 ubuntu mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-11-09T03:56:27.151 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir 134338 ubuntu mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-11-09T03:56:27.151 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir 134338 ubuntu mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-11-09T03:56:27.151 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir 134338 ubuntu mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-11-09T03:56:27.152 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir 134338 ubuntu mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-11-09T03:56:27.152 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir 134338 ubuntu mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-11-09T03:56:27.152 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir 134338 ubuntu mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:27.152 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir 134338 ubuntu mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:27.152 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir 134338 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:27.152 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir 134338 ubuntu mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-11-09T03:56:27.153 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir 134338 ubuntu 0r FIFO 0,13 0t0 272766 pipe 2023-11-09T03:56:27.153 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir 134338 ubuntu 1w FIFO 0,13 0t0 272767 pipe 2023-11-09T03:56:27.153 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir 134338 ubuntu 2w FIFO 0,13 0t0 272768 pipe 2023-11-09T03:56:27.153 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/u 134443 root cwd DIR 8,1 4096 2 / 2023-11-09T03:56:27.153 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/u 134443 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:27.153 INFO:teuthology.orchestra.run.smithi076.stdout:kworker/u 134443 root txt unknown /proc/134443/exe 2023-11-09T03:56:27.154 INFO:teuthology.orchestra.run.smithi076.stdout:bash 135287 ubuntu cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:27.154 INFO:teuthology.orchestra.run.smithi076.stdout:bash 135287 ubuntu rtd DIR 8,1 4096 2 / 2023-11-09T03:56:27.154 INFO:teuthology.orchestra.run.smithi076.stdout:bash 135287 ubuntu txt REG 8,1 1388920 9667 /usr/bin/bash 2023-11-09T03:56:27.154 INFO:teuthology.orchestra.run.smithi076.stdout:bash 135287 ubuntu mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-11-09T03:56:27.154 INFO:teuthology.orchestra.run.smithi076.stdout:bash 135287 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:27.155 INFO:teuthology.orchestra.run.smithi076.stdout:bash 135287 ubuntu mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:27.155 INFO:teuthology.orchestra.run.smithi076.stdout:bash 135287 ubuntu mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-11-09T03:56:27.155 INFO:teuthology.orchestra.run.smithi076.stdout:bash 135287 ubuntu mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-11-09T03:56:27.155 INFO:teuthology.orchestra.run.smithi076.stdout:bash 135287 ubuntu mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-11-09T03:56:27.155 INFO:teuthology.orchestra.run.smithi076.stdout:bash 135287 ubuntu mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-11-09T03:56:27.155 INFO:teuthology.orchestra.run.smithi076.stdout:bash 135287 ubuntu mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-11-09T03:56:27.156 INFO:teuthology.orchestra.run.smithi076.stdout:bash 135287 ubuntu mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-11-09T03:56:27.156 INFO:teuthology.orchestra.run.smithi076.stdout:bash 135287 ubuntu mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-11-09T03:56:27.156 INFO:teuthology.orchestra.run.smithi076.stdout:bash 135287 ubuntu mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-11-09T03:56:27.156 INFO:teuthology.orchestra.run.smithi076.stdout:bash 135287 ubuntu mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-11-09T03:56:27.156 INFO:teuthology.orchestra.run.smithi076.stdout:bash 135287 ubuntu mem REG 8,1 195080 9664 /usr/lib64/libtinfo.so.6.2 2023-11-09T03:56:27.156 INFO:teuthology.orchestra.run.smithi076.stdout:bash 135287 ubuntu mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:27.157 INFO:teuthology.orchestra.run.smithi076.stdout:bash 135287 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:27.157 INFO:teuthology.orchestra.run.smithi076.stdout:bash 135287 ubuntu mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-11-09T03:56:27.157 INFO:teuthology.orchestra.run.smithi076.stdout:bash 135287 ubuntu 0r FIFO 0,13 0t0 277656 pipe 2023-11-09T03:56:27.157 INFO:teuthology.orchestra.run.smithi076.stdout:bash 135287 ubuntu 1w FIFO 0,13 0t0 277657 pipe 2023-11-09T03:56:27.157 INFO:teuthology.orchestra.run.smithi076.stdout:bash 135287 ubuntu 2w FIFO 0,13 0t0 277658 pipe 2023-11-09T03:56:27.157 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:27.158 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:27.158 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root txt REG 8,1 185296 11759 /usr/bin/sudo 2023-11-09T03:56:27.158 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 357408 10120 /usr/lib64/libnss_systemd.so.2 2023-11-09T03:56:27.158 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-11-09T03:56:27.158 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-11-09T03:56:27.158 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-11-09T03:56:27.159 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-11-09T03:56:27.159 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 19600 10967 /usr/lib64/libpam_misc.so.0.82.1 2023-11-09T03:56:27.159 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 19400 32305 /usr/lib64/security/pam_succeed_if.so 2023-11-09T03:56:27.159 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 514280 32422 /usr/lib64/security/pam_systemd.so 2023-11-09T03:56:27.159 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 27536 32284 /usr/lib64/security/pam_limits.so 2023-11-09T03:56:27.159 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 48392 10228 /usr/lib64/libcrack.so.2.9.0 2023-11-09T03:56:27.160 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 32608 10961 /usr/lib64/libpwquality.so.1.0.2 2023-11-09T03:56:27.160 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 15352 32282 /usr/lib64/security/pam_keyinit.so 2023-11-09T03:56:27.160 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 15840 32264 /usr/lib64/security/pam_pwquality.so 2023-11-09T03:56:27.160 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 14928 32270 /usr/lib64/security/pam_deny.so 2023-11-09T03:56:27.160 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 56712 32310 /usr/lib64/security/pam_unix.so 2023-11-09T03:56:27.160 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:27.161 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:27.161 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:27.161 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:27.161 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:27.161 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:27.161 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:27.162 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:27.162 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-11-09T03:56:27.162 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-11-09T03:56:27.162 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-11-09T03:56:27.162 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-11-09T03:56:27.162 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:27.163 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:27.163 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-11-09T03:56:27.163 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-11-09T03:56:27.163 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 15184 32274 /usr/lib64/security/pam_faildelay.so 2023-11-09T03:56:27.163 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-11-09T03:56:27.164 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-11-09T03:56:27.164 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 472568 54513 /usr/libexec/sudo/sudoers.so 2023-11-09T03:56:27.164 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:27.164 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:27.164 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-11-09T03:56:27.164 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 19464 32272 /usr/lib64/security/pam_env.so 2023-11-09T03:56:27.165 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:27.165 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-11-09T03:56:27.165 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-11-09T03:56:27.165 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-11-09T03:56:27.165 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-11-09T03:56:27.165 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-11-09T03:56:27.166 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-11-09T03:56:27.166 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-11-09T03:56:27.166 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-11-09T03:56:27.166 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-11-09T03:56:27.166 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-11-09T03:56:27.166 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 119832 54509 /usr/libexec/sudo/libsudo_util.so.0.0.0 2023-11-09T03:56:27.167 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:27.167 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-11-09T03:56:27.167 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:27.167 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:27.167 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-11-09T03:56:27.167 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root 0r FIFO 0,13 0t0 277656 pipe 2023-11-09T03:56:27.168 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root 1w FIFO 0,13 0t0 277657 pipe 2023-11-09T03:56:27.168 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root 2w FIFO 0,13 0t0 277658 pipe 2023-11-09T03:56:27.168 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root 3r FIFO 0,13 0t0 275697 pipe 2023-11-09T03:56:27.168 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root 4w FIFO 0,13 0t0 275697 pipe 2023-11-09T03:56:27.168 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root 5u netlink 0t0 275703 AUDIT 2023-11-09T03:56:27.168 INFO:teuthology.orchestra.run.smithi076.stdout:sudo 135303 root 6u unix 0xffff959a4837d500 0t0 275704 type=DGRAM (CONNECTED) 2023-11-09T03:56:27.169 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 135305 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:27.169 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 135305 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:27.169 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 135305 root txt REG 8,1 15448 10498 /usr/bin/python3.9 2023-11-09T03:56:27.169 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 135305 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-11-09T03:56:27.169 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 135305 root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:27.169 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 135305 root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-11-09T03:56:27.170 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 135305 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:27.170 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 135305 root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:27.170 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 135305 root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:27.170 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 135305 root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:27.170 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 135305 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:27.170 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 135305 root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:27.171 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 135305 root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:27.171 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 135305 root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:27.171 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 135305 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-11-09T03:56:27.171 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 135305 root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-11-09T03:56:27.171 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 135305 root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:27.171 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 135305 root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:27.172 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 135305 root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-11-09T03:56:27.172 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 135305 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:27.172 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 135305 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:27.172 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 135305 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:27.172 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 135305 root 0r FIFO 0,13 0t0 277656 pipe 2023-11-09T03:56:27.172 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 135305 root 1w FIFO 0,13 0t0 277657 pipe 2023-11-09T03:56:27.173 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 135305 root 2w FIFO 0,13 0t0 277658 pipe 2023-11-09T03:56:27.173 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 135305 root 3r FIFO 0,13 0t0 273806 pipe 2023-11-09T03:56:27.173 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 135305 root 4w FIFO 0,13 0t0 273806 pipe 2023-11-09T03:56:27.173 INFO:teuthology.orchestra.run.smithi076.stdout:stdin-kil 135305 root 6w FIFO 0,13 0t0 273807 pipe 2023-11-09T03:56:27.173 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135306 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:27.173 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135306 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:27.174 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135306 root txt REG 8,1 179232 12343 /usr/bin/lsof 2023-11-09T03:56:27.174 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135306 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:27.174 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135306 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:27.174 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135306 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:27.174 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135306 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:27.175 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135306 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:27.175 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135306 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:27.175 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135306 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:27.175 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135306 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:27.175 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135306 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:27.175 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135306 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:27.176 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135306 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:27.176 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135306 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:27.176 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135306 root mem REG 8,1 196112 11431 /usr/lib64/libtirpc.so.3.0.0 2023-11-09T03:56:27.176 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135306 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:27.176 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135306 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:27.176 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135306 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:27.177 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135306 root 0r FIFO 0,13 0t0 273807 pipe 2023-11-09T03:56:27.177 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135306 root 1w FIFO 0,13 0t0 277657 pipe 2023-11-09T03:56:27.177 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135306 root 2w FIFO 0,13 0t0 277658 pipe 2023-11-09T03:56:27.177 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135306 root 3r DIR 0,20 0 1 /proc 2023-11-09T03:56:27.177 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135306 root 4r DIR 0,20 0 273135 /proc/135306/fd 2023-11-09T03:56:27.177 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135306 root 5w FIFO 0,13 0t0 273147 pipe 2023-11-09T03:56:27.178 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135306 root 6r FIFO 0,13 0t0 273148 pipe 2023-11-09T03:56:27.178 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135307 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-11-09T03:56:27.178 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135307 root rtd DIR 8,1 4096 2 / 2023-11-09T03:56:27.178 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135307 root txt REG 8,1 179232 12343 /usr/bin/lsof 2023-11-09T03:56:27.178 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135307 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-11-09T03:56:27.178 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135307 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-11-09T03:56:27.179 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135307 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-11-09T03:56:27.179 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135307 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-11-09T03:56:27.179 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135307 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-11-09T03:56:27.179 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135307 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-11-09T03:56:27.179 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135307 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-11-09T03:56:27.179 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135307 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-11-09T03:56:27.180 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135307 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-11-09T03:56:27.180 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135307 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-11-09T03:56:27.180 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135307 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-11-09T03:56:27.180 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135307 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-11-09T03:56:27.180 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135307 root mem REG 8,1 196112 11431 /usr/lib64/libtirpc.so.3.0.0 2023-11-09T03:56:27.181 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135307 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-11-09T03:56:27.181 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135307 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-11-09T03:56:27.181 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135307 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-11-09T03:56:27.181 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135307 root 4r FIFO 0,13 0t0 273147 pipe 2023-11-09T03:56:27.181 INFO:teuthology.orchestra.run.smithi076.stdout:lsof 135307 root 7w FIFO 0,13 0t0 273148 pipe 2023-11-09T03:56:27.182 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:56:27.184 INFO:teuthology.orchestra.run.smithi076.stdout:USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND 2023-11-09T03:56:27.184 INFO:teuthology.orchestra.run.smithi076.stdout:root 2 0.0 0.0 0 0 ? S 03:35 0:00 [kthreadd] 2023-11-09T03:56:27.184 INFO:teuthology.orchestra.run.smithi076.stdout:root 3 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [rcu_gp] 2023-11-09T03:56:27.184 INFO:teuthology.orchestra.run.smithi076.stdout:root 4 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [rcu_par_gp] 2023-11-09T03:56:27.184 INFO:teuthology.orchestra.run.smithi076.stdout:root 5 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [slub_flushwq] 2023-11-09T03:56:27.185 INFO:teuthology.orchestra.run.smithi076.stdout:root 6 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [netns] 2023-11-09T03:56:27.185 INFO:teuthology.orchestra.run.smithi076.stdout:root 8 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [kworker/0:0H-events_highpri] 2023-11-09T03:56:27.185 INFO:teuthology.orchestra.run.smithi076.stdout:root 11 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [mm_percpu_wq] 2023-11-09T03:56:27.185 INFO:teuthology.orchestra.run.smithi076.stdout:root 12 0.0 0.0 0 0 ? I 03:35 0:00 \_ [kworker/u16:1-flush-253:1] 2023-11-09T03:56:27.185 INFO:teuthology.orchestra.run.smithi076.stdout:root 13 0.0 0.0 0 0 ? I 03:35 0:00 \_ [rcu_tasks_kthre] 2023-11-09T03:56:27.186 INFO:teuthology.orchestra.run.smithi076.stdout:root 14 0.0 0.0 0 0 ? I 03:35 0:00 \_ [rcu_tasks_rude_] 2023-11-09T03:56:27.186 INFO:teuthology.orchestra.run.smithi076.stdout:root 15 0.0 0.0 0 0 ? I 03:35 0:00 \_ [rcu_tasks_trace] 2023-11-09T03:56:27.186 INFO:teuthology.orchestra.run.smithi076.stdout:root 16 0.0 0.0 0 0 ? S 03:35 0:00 \_ [ksoftirqd/0] 2023-11-09T03:56:27.186 INFO:teuthology.orchestra.run.smithi076.stdout:root 17 0.4 0.0 0 0 ? S 03:35 0:05 \_ [pr/ttyS1] 2023-11-09T03:56:27.186 INFO:teuthology.orchestra.run.smithi076.stdout:root 18 0.0 0.0 0 0 ? I 03:35 0:00 \_ [rcu_preempt] 2023-11-09T03:56:27.187 INFO:teuthology.orchestra.run.smithi076.stdout:root 19 0.0 0.0 0 0 ? S 03:35 0:00 \_ [migration/0] 2023-11-09T03:56:27.187 INFO:teuthology.orchestra.run.smithi076.stdout:root 20 0.0 0.0 0 0 ? S 03:35 0:00 \_ [idle_inject/0] 2023-11-09T03:56:27.187 INFO:teuthology.orchestra.run.smithi076.stdout:root 21 0.0 0.0 0 0 ? S 03:35 0:00 \_ [cpuhp/0] 2023-11-09T03:56:27.187 INFO:teuthology.orchestra.run.smithi076.stdout:root 22 0.0 0.0 0 0 ? S 03:35 0:00 \_ [cpuhp/1] 2023-11-09T03:56:27.187 INFO:teuthology.orchestra.run.smithi076.stdout:root 23 0.0 0.0 0 0 ? S 03:35 0:00 \_ [idle_inject/1] 2023-11-09T03:56:27.187 INFO:teuthology.orchestra.run.smithi076.stdout:root 24 0.0 0.0 0 0 ? S 03:35 0:00 \_ [migration/1] 2023-11-09T03:56:27.188 INFO:teuthology.orchestra.run.smithi076.stdout:root 25 0.0 0.0 0 0 ? S 03:35 0:00 \_ [ksoftirqd/1] 2023-11-09T03:56:27.188 INFO:teuthology.orchestra.run.smithi076.stdout:root 27 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [kworker/1:0H-events_highpri] 2023-11-09T03:56:27.188 INFO:teuthology.orchestra.run.smithi076.stdout:root 28 0.0 0.0 0 0 ? S 03:35 0:00 \_ [cpuhp/2] 2023-11-09T03:56:27.188 INFO:teuthology.orchestra.run.smithi076.stdout:root 29 0.0 0.0 0 0 ? S 03:35 0:00 \_ [idle_inject/2] 2023-11-09T03:56:27.188 INFO:teuthology.orchestra.run.smithi076.stdout:root 30 0.0 0.0 0 0 ? S 03:35 0:00 \_ [migration/2] 2023-11-09T03:56:27.189 INFO:teuthology.orchestra.run.smithi076.stdout:root 31 0.0 0.0 0 0 ? S 03:35 0:00 \_ [ksoftirqd/2] 2023-11-09T03:56:27.189 INFO:teuthology.orchestra.run.smithi076.stdout:root 33 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [kworker/2:0H-events_highpri] 2023-11-09T03:56:27.189 INFO:teuthology.orchestra.run.smithi076.stdout:root 34 0.0 0.0 0 0 ? S 03:35 0:00 \_ [cpuhp/3] 2023-11-09T03:56:27.189 INFO:teuthology.orchestra.run.smithi076.stdout:root 35 0.0 0.0 0 0 ? S 03:35 0:00 \_ [idle_inject/3] 2023-11-09T03:56:27.189 INFO:teuthology.orchestra.run.smithi076.stdout:root 36 0.0 0.0 0 0 ? S 03:35 0:00 \_ [migration/3] 2023-11-09T03:56:27.189 INFO:teuthology.orchestra.run.smithi076.stdout:root 37 0.0 0.0 0 0 ? S 03:35 0:00 \_ [ksoftirqd/3] 2023-11-09T03:56:27.190 INFO:teuthology.orchestra.run.smithi076.stdout:root 39 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [kworker/3:0H-events_highpri] 2023-11-09T03:56:27.190 INFO:teuthology.orchestra.run.smithi076.stdout:root 40 0.0 0.0 0 0 ? S 03:35 0:00 \_ [cpuhp/4] 2023-11-09T03:56:27.190 INFO:teuthology.orchestra.run.smithi076.stdout:root 41 0.0 0.0 0 0 ? S 03:35 0:00 \_ [idle_inject/4] 2023-11-09T03:56:27.190 INFO:teuthology.orchestra.run.smithi076.stdout:root 42 0.0 0.0 0 0 ? S 03:35 0:00 \_ [migration/4] 2023-11-09T03:56:27.190 INFO:teuthology.orchestra.run.smithi076.stdout:root 43 0.0 0.0 0 0 ? S 03:35 0:00 \_ [ksoftirqd/4] 2023-11-09T03:56:27.190 INFO:teuthology.orchestra.run.smithi076.stdout:root 45 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [kworker/4:0H-events_highpri] 2023-11-09T03:56:27.191 INFO:teuthology.orchestra.run.smithi076.stdout:root 46 0.0 0.0 0 0 ? S 03:35 0:00 \_ [cpuhp/5] 2023-11-09T03:56:27.191 INFO:teuthology.orchestra.run.smithi076.stdout:root 47 0.0 0.0 0 0 ? S 03:35 0:00 \_ [idle_inject/5] 2023-11-09T03:56:27.191 INFO:teuthology.orchestra.run.smithi076.stdout:root 48 0.0 0.0 0 0 ? S 03:35 0:00 \_ [migration/5] 2023-11-09T03:56:27.191 INFO:teuthology.orchestra.run.smithi076.stdout:root 49 0.0 0.0 0 0 ? S 03:35 0:00 \_ [ksoftirqd/5] 2023-11-09T03:56:27.191 INFO:teuthology.orchestra.run.smithi076.stdout:root 51 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [kworker/5:0H-events_highpri] 2023-11-09T03:56:27.192 INFO:teuthology.orchestra.run.smithi076.stdout:root 52 0.0 0.0 0 0 ? S 03:35 0:00 \_ [cpuhp/6] 2023-11-09T03:56:27.192 INFO:teuthology.orchestra.run.smithi076.stdout:root 53 0.0 0.0 0 0 ? S 03:35 0:00 \_ [idle_inject/6] 2023-11-09T03:56:27.192 INFO:teuthology.orchestra.run.smithi076.stdout:root 54 0.0 0.0 0 0 ? S 03:35 0:00 \_ [migration/6] 2023-11-09T03:56:27.192 INFO:teuthology.orchestra.run.smithi076.stdout:root 55 0.0 0.0 0 0 ? S 03:35 0:00 \_ [ksoftirqd/6] 2023-11-09T03:56:27.192 INFO:teuthology.orchestra.run.smithi076.stdout:root 57 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [kworker/6:0H-events_highpri] 2023-11-09T03:56:27.192 INFO:teuthology.orchestra.run.smithi076.stdout:root 58 0.0 0.0 0 0 ? S 03:35 0:00 \_ [cpuhp/7] 2023-11-09T03:56:27.193 INFO:teuthology.orchestra.run.smithi076.stdout:root 59 0.0 0.0 0 0 ? S 03:35 0:00 \_ [idle_inject/7] 2023-11-09T03:56:27.193 INFO:teuthology.orchestra.run.smithi076.stdout:root 60 0.0 0.0 0 0 ? S 03:35 0:00 \_ [migration/7] 2023-11-09T03:56:27.193 INFO:teuthology.orchestra.run.smithi076.stdout:root 61 0.0 0.0 0 0 ? S 03:35 0:00 \_ [ksoftirqd/7] 2023-11-09T03:56:27.193 INFO:teuthology.orchestra.run.smithi076.stdout:root 63 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [kworker/7:0H-events_highpri] 2023-11-09T03:56:27.193 INFO:teuthology.orchestra.run.smithi076.stdout:root 67 0.0 0.0 0 0 ? I 03:35 0:00 \_ [kworker/u16:4-events_unbound] 2023-11-09T03:56:27.193 INFO:teuthology.orchestra.run.smithi076.stdout:root 71 0.0 0.0 0 0 ? S 03:35 0:00 \_ [kdevtmpfs] 2023-11-09T03:56:27.194 INFO:teuthology.orchestra.run.smithi076.stdout:root 72 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [inet_frag_wq] 2023-11-09T03:56:27.194 INFO:teuthology.orchestra.run.smithi076.stdout:root 73 0.0 0.0 0 0 ? S 03:35 0:00 \_ [kauditd] 2023-11-09T03:56:27.194 INFO:teuthology.orchestra.run.smithi076.stdout:root 75 0.0 0.0 0 0 ? S 03:35 0:00 \_ [khungtaskd] 2023-11-09T03:56:27.194 INFO:teuthology.orchestra.run.smithi076.stdout:root 76 0.0 0.0 0 0 ? S 03:35 0:00 \_ [oom_reaper] 2023-11-09T03:56:27.194 INFO:teuthology.orchestra.run.smithi076.stdout:root 77 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [writeback] 2023-11-09T03:56:27.195 INFO:teuthology.orchestra.run.smithi076.stdout:root 78 0.0 0.0 0 0 ? S 03:35 0:00 \_ [kcompactd0] 2023-11-09T03:56:27.195 INFO:teuthology.orchestra.run.smithi076.stdout:root 79 0.0 0.0 0 0 ? SN 03:35 0:00 \_ [ksmd] 2023-11-09T03:56:27.195 INFO:teuthology.orchestra.run.smithi076.stdout:root 80 0.0 0.0 0 0 ? SN 03:35 0:00 \_ [khugepaged] 2023-11-09T03:56:27.195 INFO:teuthology.orchestra.run.smithi076.stdout:root 81 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [cryptd] 2023-11-09T03:56:27.195 INFO:teuthology.orchestra.run.smithi076.stdout:root 82 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [kintegrityd] 2023-11-09T03:56:27.195 INFO:teuthology.orchestra.run.smithi076.stdout:root 83 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [kblockd] 2023-11-09T03:56:27.196 INFO:teuthology.orchestra.run.smithi076.stdout:root 84 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [blkcg_punt_bio] 2023-11-09T03:56:27.196 INFO:teuthology.orchestra.run.smithi076.stdout:root 85 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [tpm_dev_wq] 2023-11-09T03:56:27.196 INFO:teuthology.orchestra.run.smithi076.stdout:root 86 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [md] 2023-11-09T03:56:27.196 INFO:teuthology.orchestra.run.smithi076.stdout:root 87 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [md_bitmap] 2023-11-09T03:56:27.196 INFO:teuthology.orchestra.run.smithi076.stdout:root 88 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [edac-poller] 2023-11-09T03:56:27.196 INFO:teuthology.orchestra.run.smithi076.stdout:root 89 0.0 0.0 0 0 ? S 03:35 0:00 \_ [watchdogd] 2023-11-09T03:56:27.197 INFO:teuthology.orchestra.run.smithi076.stdout:root 91 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [kworker/3:1H-kblockd] 2023-11-09T03:56:27.197 INFO:teuthology.orchestra.run.smithi076.stdout:root 93 0.0 0.0 0 0 ? S 03:35 0:00 \_ [kswapd0] 2023-11-09T03:56:27.197 INFO:teuthology.orchestra.run.smithi076.stdout:root 99 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [kthrotld] 2023-11-09T03:56:27.197 INFO:teuthology.orchestra.run.smithi076.stdout:root 108 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [acpi_thermal_pm] 2023-11-09T03:56:27.197 INFO:teuthology.orchestra.run.smithi076.stdout:root 109 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [kmpath_rdacd] 2023-11-09T03:56:27.197 INFO:teuthology.orchestra.run.smithi076.stdout:root 110 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [kaluad] 2023-11-09T03:56:27.198 INFO:teuthology.orchestra.run.smithi076.stdout:root 111 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [mld] 2023-11-09T03:56:27.198 INFO:teuthology.orchestra.run.smithi076.stdout:root 112 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [kworker/6:1H-kblockd] 2023-11-09T03:56:27.198 INFO:teuthology.orchestra.run.smithi076.stdout:root 113 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [ipv6_addrconf] 2023-11-09T03:56:27.198 INFO:teuthology.orchestra.run.smithi076.stdout:root 122 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [kstrp] 2023-11-09T03:56:27.198 INFO:teuthology.orchestra.run.smithi076.stdout:root 129 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [zswap-shrink] 2023-11-09T03:56:27.198 INFO:teuthology.orchestra.run.smithi076.stdout:root 237 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [kworker/u17:0] 2023-11-09T03:56:27.199 INFO:teuthology.orchestra.run.smithi076.stdout:root 276 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [kworker/7:1H-xfs-log/dm-4] 2023-11-09T03:56:27.199 INFO:teuthology.orchestra.run.smithi076.stdout:root 288 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [kworker/1:1H-xfs-log/dm-4] 2023-11-09T03:56:27.199 INFO:teuthology.orchestra.run.smithi076.stdout:root 331 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [kworker/5:1H-kblockd] 2023-11-09T03:56:27.199 INFO:teuthology.orchestra.run.smithi076.stdout:root 351 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [kworker/0:1H-xfs-log/dm-4] 2023-11-09T03:56:27.199 INFO:teuthology.orchestra.run.smithi076.stdout:root 381 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [kworker/2:1H-xfs-log/dm-4] 2023-11-09T03:56:27.199 INFO:teuthology.orchestra.run.smithi076.stdout:root 405 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [kworker/4:1H-kblockd] 2023-11-09T03:56:27.200 INFO:teuthology.orchestra.run.smithi076.stdout:root 433 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [nvme-wq] 2023-11-09T03:56:27.200 INFO:teuthology.orchestra.run.smithi076.stdout:root 434 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [nvme-reset-wq] 2023-11-09T03:56:27.200 INFO:teuthology.orchestra.run.smithi076.stdout:root 436 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [ata_sff] 2023-11-09T03:56:27.200 INFO:teuthology.orchestra.run.smithi076.stdout:root 438 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [ixgbe] 2023-11-09T03:56:27.200 INFO:teuthology.orchestra.run.smithi076.stdout:root 439 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [nvme-delete-wq] 2023-11-09T03:56:27.200 INFO:teuthology.orchestra.run.smithi076.stdout:root 440 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [nvme-auth-wq] 2023-11-09T03:56:27.201 INFO:teuthology.orchestra.run.smithi076.stdout:root 443 0.0 0.0 0 0 ? S 03:35 0:00 \_ [scsi_eh_0] 2023-11-09T03:56:27.201 INFO:teuthology.orchestra.run.smithi076.stdout:root 444 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [scsi_tmf_0] 2023-11-09T03:56:27.201 INFO:teuthology.orchestra.run.smithi076.stdout:root 445 0.0 0.0 0 0 ? S 03:35 0:00 \_ [scsi_eh_1] 2023-11-09T03:56:27.201 INFO:teuthology.orchestra.run.smithi076.stdout:root 446 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [scsi_tmf_1] 2023-11-09T03:56:27.201 INFO:teuthology.orchestra.run.smithi076.stdout:root 447 0.0 0.0 0 0 ? S 03:35 0:00 \_ [scsi_eh_2] 2023-11-09T03:56:27.201 INFO:teuthology.orchestra.run.smithi076.stdout:root 448 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [scsi_tmf_2] 2023-11-09T03:56:27.202 INFO:teuthology.orchestra.run.smithi076.stdout:root 449 0.0 0.0 0 0 ? S 03:35 0:00 \_ [scsi_eh_3] 2023-11-09T03:56:27.202 INFO:teuthology.orchestra.run.smithi076.stdout:root 450 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [scsi_tmf_3] 2023-11-09T03:56:27.202 INFO:teuthology.orchestra.run.smithi076.stdout:root 453 0.0 0.0 0 0 ? S 03:35 0:00 \_ [scsi_eh_4] 2023-11-09T03:56:27.202 INFO:teuthology.orchestra.run.smithi076.stdout:root 454 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [scsi_tmf_4] 2023-11-09T03:56:27.202 INFO:teuthology.orchestra.run.smithi076.stdout:root 455 0.0 0.0 0 0 ? S 03:35 0:00 \_ [scsi_eh_5] 2023-11-09T03:56:27.202 INFO:teuthology.orchestra.run.smithi076.stdout:root 456 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [scsi_tmf_5] 2023-11-09T03:56:27.203 INFO:teuthology.orchestra.run.smithi076.stdout:root 458 0.0 0.0 0 0 ? S 03:35 0:00 \_ [scsi_eh_6] 2023-11-09T03:56:27.203 INFO:teuthology.orchestra.run.smithi076.stdout:root 459 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [scsi_tmf_6] 2023-11-09T03:56:27.203 INFO:teuthology.orchestra.run.smithi076.stdout:root 460 0.0 0.0 0 0 ? S 03:35 0:00 \_ [scsi_eh_7] 2023-11-09T03:56:27.203 INFO:teuthology.orchestra.run.smithi076.stdout:root 461 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [scsi_tmf_7] 2023-11-09T03:56:27.203 INFO:teuthology.orchestra.run.smithi076.stdout:root 462 0.0 0.0 0 0 ? S 03:35 0:00 \_ [scsi_eh_8] 2023-11-09T03:56:27.204 INFO:teuthology.orchestra.run.smithi076.stdout:root 463 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [scsi_tmf_8] 2023-11-09T03:56:27.204 INFO:teuthology.orchestra.run.smithi076.stdout:root 464 0.0 0.0 0 0 ? S 03:35 0:00 \_ [scsi_eh_9] 2023-11-09T03:56:27.204 INFO:teuthology.orchestra.run.smithi076.stdout:root 465 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [scsi_tmf_9] 2023-11-09T03:56:27.204 INFO:teuthology.orchestra.run.smithi076.stdout:root 466 0.0 0.0 0 0 ? I 03:35 0:00 \_ [kworker/u16:8-flush-8:0] 2023-11-09T03:56:27.204 INFO:teuthology.orchestra.run.smithi076.stdout:root 491 0.0 0.0 0 0 ? S 03:35 0:00 \_ [jbd2/sda1-8] 2023-11-09T03:56:27.204 INFO:teuthology.orchestra.run.smithi076.stdout:root 492 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [ext4-rsv-conver] 2023-11-09T03:56:27.205 INFO:teuthology.orchestra.run.smithi076.stdout:root 573 0.0 0.0 0 0 ? S 03:35 0:00 \_ [ext4lazyinit] 2023-11-09T03:56:27.205 INFO:teuthology.orchestra.run.smithi076.stdout:root 631 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [ipmi-msghandler] 2023-11-09T03:56:27.205 INFO:teuthology.orchestra.run.smithi076.stdout:root 649 0.0 0.0 0 0 ? SN 03:35 0:00 \_ [kipmi0] 2023-11-09T03:56:27.205 INFO:teuthology.orchestra.run.smithi076.stdout:root 658 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [rpciod] 2023-11-09T03:56:27.205 INFO:teuthology.orchestra.run.smithi076.stdout:root 659 0.0 0.0 0 0 ? I< 03:35 0:00 \_ [xprtiod] 2023-11-09T03:56:27.205 INFO:teuthology.orchestra.run.smithi076.stdout:root 60996 0.0 0.0 0 0 ? I< 03:40 0:00 \_ [tls-strp] 2023-11-09T03:56:27.206 INFO:teuthology.orchestra.run.smithi076.stdout:root 70282 0.0 0.0 0 0 ? I< 03:41 0:00 \_ [kdmflush/253:0] 2023-11-09T03:56:27.206 INFO:teuthology.orchestra.run.smithi076.stdout:root 70402 0.0 0.0 0 0 ? I< 03:41 0:00 \_ [kdmflush/253:1] 2023-11-09T03:56:27.206 INFO:teuthology.orchestra.run.smithi076.stdout:root 70522 0.0 0.0 0 0 ? I< 03:41 0:00 \_ [kdmflush/253:2] 2023-11-09T03:56:27.206 INFO:teuthology.orchestra.run.smithi076.stdout:root 70642 0.0 0.0 0 0 ? I< 03:41 0:00 \_ [kdmflush/253:3] 2023-11-09T03:56:27.206 INFO:teuthology.orchestra.run.smithi076.stdout:root 70762 0.0 0.0 0 0 ? I< 03:41 0:00 \_ [kdmflush/253:4] 2023-11-09T03:56:27.206 INFO:teuthology.orchestra.run.smithi076.stdout:root 71655 0.0 0.0 0 0 ? I< 03:41 0:00 \_ [xfsalloc] 2023-11-09T03:56:27.207 INFO:teuthology.orchestra.run.smithi076.stdout:root 71656 0.0 0.0 0 0 ? I< 03:41 0:00 \_ [xfs_mru_cache] 2023-11-09T03:56:27.207 INFO:teuthology.orchestra.run.smithi076.stdout:root 71657 0.0 0.0 0 0 ? I< 03:41 0:00 \_ [xfs-buf/dm-4] 2023-11-09T03:56:27.207 INFO:teuthology.orchestra.run.smithi076.stdout:root 71658 0.0 0.0 0 0 ? I< 03:41 0:00 \_ [xfs-conv/dm-4] 2023-11-09T03:56:27.207 INFO:teuthology.orchestra.run.smithi076.stdout:root 71659 0.0 0.0 0 0 ? I< 03:41 0:00 \_ [xfs-reclaim/dm-] 2023-11-09T03:56:27.207 INFO:teuthology.orchestra.run.smithi076.stdout:root 71660 0.0 0.0 0 0 ? I< 03:41 0:00 \_ [xfs-blockgc/dm-] 2023-11-09T03:56:27.207 INFO:teuthology.orchestra.run.smithi076.stdout:root 71661 0.0 0.0 0 0 ? I< 03:41 0:00 \_ [xfs-inodegc/dm-] 2023-11-09T03:56:27.208 INFO:teuthology.orchestra.run.smithi076.stdout:root 71662 0.0 0.0 0 0 ? I< 03:41 0:00 \_ [xfs-log/dm-4] 2023-11-09T03:56:27.208 INFO:teuthology.orchestra.run.smithi076.stdout:root 71663 0.0 0.0 0 0 ? I< 03:41 0:00 \_ [xfs-cil/dm-4] 2023-11-09T03:56:27.208 INFO:teuthology.orchestra.run.smithi076.stdout:root 71664 0.0 0.0 0 0 ? S 03:41 0:00 \_ [xfsaild/dm-4] 2023-11-09T03:56:27.208 INFO:teuthology.orchestra.run.smithi076.stdout:root 75669 0.0 0.0 0 0 ? I 03:44 0:00 \_ [kworker/u16:3-events_unbound] 2023-11-09T03:56:27.208 INFO:teuthology.orchestra.run.smithi076.stdout:root 76435 0.0 0.0 0 0 ? I 03:44 0:00 \_ [kworker/3:46-events] 2023-11-09T03:56:27.208 INFO:teuthology.orchestra.run.smithi076.stdout:root 76857 0.0 0.0 0 0 ? I 03:44 0:00 \_ [kworker/2:16-events] 2023-11-09T03:56:27.209 INFO:teuthology.orchestra.run.smithi076.stdout:root 77087 0.0 0.0 0 0 ? I 03:44 0:00 \_ [kworker/2:50-rcu_gp] 2023-11-09T03:56:27.209 INFO:teuthology.orchestra.run.smithi076.stdout:root 82091 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [xfs-buf/dm-0] 2023-11-09T03:56:27.209 INFO:teuthology.orchestra.run.smithi076.stdout:root 82092 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [xfs-conv/dm-0] 2023-11-09T03:56:27.209 INFO:teuthology.orchestra.run.smithi076.stdout:root 82093 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [xfs-reclaim/dm-] 2023-11-09T03:56:27.209 INFO:teuthology.orchestra.run.smithi076.stdout:root 82094 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [xfs-blockgc/dm-] 2023-11-09T03:56:27.209 INFO:teuthology.orchestra.run.smithi076.stdout:root 82095 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [xfs-inodegc/dm-] 2023-11-09T03:56:27.210 INFO:teuthology.orchestra.run.smithi076.stdout:root 82096 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [xfs-log/dm-0] 2023-11-09T03:56:27.210 INFO:teuthology.orchestra.run.smithi076.stdout:root 82097 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [xfs-cil/dm-0] 2023-11-09T03:56:27.210 INFO:teuthology.orchestra.run.smithi076.stdout:root 82098 0.0 0.0 0 0 ? S 03:44 0:00 \_ [xfsaild/dm-0] 2023-11-09T03:56:27.210 INFO:teuthology.orchestra.run.smithi076.stdout:root 82175 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [xfs-buf/dm-1] 2023-11-09T03:56:27.210 INFO:teuthology.orchestra.run.smithi076.stdout:root 82176 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [xfs-conv/dm-1] 2023-11-09T03:56:27.210 INFO:teuthology.orchestra.run.smithi076.stdout:root 82177 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [xfs-reclaim/dm-] 2023-11-09T03:56:27.211 INFO:teuthology.orchestra.run.smithi076.stdout:root 82178 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [xfs-blockgc/dm-] 2023-11-09T03:56:27.211 INFO:teuthology.orchestra.run.smithi076.stdout:root 82179 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [xfs-inodegc/dm-] 2023-11-09T03:56:27.211 INFO:teuthology.orchestra.run.smithi076.stdout:root 82180 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [xfs-log/dm-1] 2023-11-09T03:56:27.211 INFO:teuthology.orchestra.run.smithi076.stdout:root 82181 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [xfs-cil/dm-1] 2023-11-09T03:56:27.211 INFO:teuthology.orchestra.run.smithi076.stdout:root 82182 0.0 0.0 0 0 ? S 03:44 0:00 \_ [xfsaild/dm-1] 2023-11-09T03:56:27.211 INFO:teuthology.orchestra.run.smithi076.stdout:root 82259 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [xfs-buf/dm-2] 2023-11-09T03:56:27.212 INFO:teuthology.orchestra.run.smithi076.stdout:root 82260 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [xfs-conv/dm-2] 2023-11-09T03:56:27.212 INFO:teuthology.orchestra.run.smithi076.stdout:root 82261 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [xfs-reclaim/dm-] 2023-11-09T03:56:27.212 INFO:teuthology.orchestra.run.smithi076.stdout:root 82262 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [xfs-blockgc/dm-] 2023-11-09T03:56:27.212 INFO:teuthology.orchestra.run.smithi076.stdout:root 82263 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [xfs-inodegc/dm-] 2023-11-09T03:56:27.212 INFO:teuthology.orchestra.run.smithi076.stdout:root 82264 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [xfs-log/dm-2] 2023-11-09T03:56:27.212 INFO:teuthology.orchestra.run.smithi076.stdout:root 82265 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [xfs-cil/dm-2] 2023-11-09T03:56:27.213 INFO:teuthology.orchestra.run.smithi076.stdout:root 82266 0.0 0.0 0 0 ? S 03:44 0:00 \_ [xfsaild/dm-2] 2023-11-09T03:56:27.213 INFO:teuthology.orchestra.run.smithi076.stdout:root 82343 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [xfs-buf/dm-3] 2023-11-09T03:56:27.213 INFO:teuthology.orchestra.run.smithi076.stdout:root 82344 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [xfs-conv/dm-3] 2023-11-09T03:56:27.213 INFO:teuthology.orchestra.run.smithi076.stdout:root 82345 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [xfs-reclaim/dm-] 2023-11-09T03:56:27.213 INFO:teuthology.orchestra.run.smithi076.stdout:root 82346 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [xfs-blockgc/dm-] 2023-11-09T03:56:27.214 INFO:teuthology.orchestra.run.smithi076.stdout:root 82347 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [xfs-inodegc/dm-] 2023-11-09T03:56:27.214 INFO:teuthology.orchestra.run.smithi076.stdout:root 82348 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [xfs-log/dm-3] 2023-11-09T03:56:27.214 INFO:teuthology.orchestra.run.smithi076.stdout:root 82349 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [xfs-cil/dm-3] 2023-11-09T03:56:27.214 INFO:teuthology.orchestra.run.smithi076.stdout:root 82350 0.0 0.0 0 0 ? S 03:44 0:00 \_ [xfsaild/dm-3] 2023-11-09T03:56:27.214 INFO:teuthology.orchestra.run.smithi076.stdout:root 82396 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [dio/dm-0] 2023-11-09T03:56:27.214 INFO:teuthology.orchestra.run.smithi076.stdout:root 83558 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [dio/dm-1] 2023-11-09T03:56:27.215 INFO:teuthology.orchestra.run.smithi076.stdout:root 84720 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [dio/dm-2] 2023-11-09T03:56:27.215 INFO:teuthology.orchestra.run.smithi076.stdout:root 85882 0.0 0.0 0 0 ? I< 03:44 0:00 \_ [dio/dm-3] 2023-11-09T03:56:27.215 INFO:teuthology.orchestra.run.smithi076.stdout:root 91634 0.0 0.0 0 0 ? I 03:45 0:00 \_ [kworker/4:113-events] 2023-11-09T03:56:27.215 INFO:teuthology.orchestra.run.smithi076.stdout:root 91777 0.0 0.0 0 0 ? I 03:45 0:00 \_ [kworker/5:26-events] 2023-11-09T03:56:27.215 INFO:teuthology.orchestra.run.smithi076.stdout:root 91791 0.0 0.0 0 0 ? I 03:45 0:00 \_ [kworker/7:27-mm_percpu_wq] 2023-11-09T03:56:27.215 INFO:teuthology.orchestra.run.smithi076.stdout:root 91938 0.0 0.0 0 0 ? I 03:45 0:00 \_ [kworker/0:110-events] 2023-11-09T03:56:27.216 INFO:teuthology.orchestra.run.smithi076.stdout:root 91982 0.0 0.0 0 0 ? I 03:45 0:00 \_ [kworker/5:105-rcu_gp] 2023-11-09T03:56:27.216 INFO:teuthology.orchestra.run.smithi076.stdout:root 91991 0.0 0.0 0 0 ? I 03:45 0:00 \_ [kworker/0:128-rcu_gp] 2023-11-09T03:56:27.216 INFO:teuthology.orchestra.run.smithi076.stdout:root 92133 0.0 0.0 0 0 ? I 03:45 0:00 \_ [kworker/7:142-rcu_gp] 2023-11-09T03:56:27.216 INFO:teuthology.orchestra.run.smithi076.stdout:root 92518 0.0 0.0 0 0 ? I 03:45 0:00 \_ [kworker/6:89-dio/dm-3] 2023-11-09T03:56:27.216 INFO:teuthology.orchestra.run.smithi076.stdout:root 92519 0.0 0.0 0 0 ? I 03:45 0:00 \_ [kworker/6:90-mm_percpu_wq] 2023-11-09T03:56:27.216 INFO:teuthology.orchestra.run.smithi076.stdout:root 92747 0.0 0.0 0 0 ? I 03:45 0:00 \_ [kworker/1:143-mm_percpu_wq] 2023-11-09T03:56:27.217 INFO:teuthology.orchestra.run.smithi076.stdout:root 92763 0.0 0.0 0 0 ? I 03:45 0:00 \_ [kworker/1:159-events] 2023-11-09T03:56:27.217 INFO:teuthology.orchestra.run.smithi076.stdout:root 92974 0.0 0.0 0 0 ? I 03:45 0:00 \_ [kworker/4:265-rcu_gp] 2023-11-09T03:56:27.217 INFO:teuthology.orchestra.run.smithi076.stdout:root 120230 0.0 0.0 0 0 ? I 03:50 0:00 \_ [kworker/u16:5-flush-253:3] 2023-11-09T03:56:27.217 INFO:teuthology.orchestra.run.smithi076.stdout:root 120335 0.0 0.0 0 0 ? I 03:50 0:00 \_ [kworker/7:0-mm_percpu_wq] 2023-11-09T03:56:27.217 INFO:teuthology.orchestra.run.smithi076.stdout:root 121726 0.0 0.0 0 0 ? I 03:51 0:00 \_ [kworker/3:0-mm_percpu_wq] 2023-11-09T03:56:27.217 INFO:teuthology.orchestra.run.smithi076.stdout:root 121806 0.0 0.0 0 0 ? I 03:51 0:00 \_ [kworker/5:0-events] 2023-11-09T03:56:27.218 INFO:teuthology.orchestra.run.smithi076.stdout:root 122113 0.0 0.0 0 0 ? I 03:51 0:00 \_ [kworker/4:0-mm_percpu_wq] 2023-11-09T03:56:27.218 INFO:teuthology.orchestra.run.smithi076.stdout:root 124193 0.0 0.0 0 0 ? I 03:52 0:00 \_ [kworker/4:1] 2023-11-09T03:56:27.218 INFO:teuthology.orchestra.run.smithi076.stdout:root 126410 0.0 0.0 0 0 ? I 03:53 0:00 \_ [kworker/2:0-events] 2023-11-09T03:56:27.218 INFO:teuthology.orchestra.run.smithi076.stdout:root 128551 0.0 0.0 0 0 ? I 03:54 0:00 \_ [kworker/2:1-mm_percpu_wq] 2023-11-09T03:56:27.218 INFO:teuthology.orchestra.run.smithi076.stdout:root 130499 0.0 0.0 0 0 ? I 03:54 0:00 \_ [kworker/5:1-mm_percpu_wq] 2023-11-09T03:56:27.218 INFO:teuthology.orchestra.run.smithi076.stdout:root 134443 0.0 0.0 0 0 ? I 03:56 0:00 \_ [kworker/u16:0-xfs-cil/dm-4] 2023-11-09T03:56:27.219 INFO:teuthology.orchestra.run.smithi076.stdout:root 1 0.2 0.0 176168 19920 ? Ss 03:35 0:02 /usr/lib/systemd/systemd --switched-root --system --deserialize 31 2023-11-09T03:56:27.219 INFO:teuthology.orchestra.run.smithi076.stdout:root 567 0.1 0.0 35020 18816 ? Ss 03:35 0:01 /usr/lib/systemd/systemd-journald 2023-11-09T03:56:27.219 INFO:teuthology.orchestra.run.smithi076.stdout:root 582 0.0 0.0 37652 15600 ? Ss 03:35 0:00 /usr/lib/systemd/systemd-udevd 2023-11-09T03:56:27.219 INFO:teuthology.orchestra.run.smithi076.stdout:rpc 651 0.0 0.0 13244 5504 ? Ss 03:35 0:00 /usr/bin/rpcbind -w -f 2023-11-09T03:56:27.219 INFO:teuthology.orchestra.run.smithi076.stdout:root 661 0.1 0.0 91928 4360 ? S stdin-killer -- python3 -c ' 2023-11-09T03:56:27.231 DEBUG:teuthology.orchestra.run.smithi076:> import os 2023-11-09T03:56:27.231 DEBUG:teuthology.orchestra.run.smithi076:> path = "/sys/fs/fuse/connections/43/abort" 2023-11-09T03:56:27.231 DEBUG:teuthology.orchestra.run.smithi076:> if os.path.exists(path): 2023-11-09T03:56:27.232 DEBUG:teuthology.orchestra.run.smithi076:> open(path, "w").write("1") 2023-11-09T03:56:27.232 DEBUG:teuthology.orchestra.run.smithi076:> ' 2023-11-09T03:56:27.309 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T03:56:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-09T03:56:27.316 DEBUG:tasks.cephfs.mount:Force/lazy unmounting on client.0 2023-11-09T03:56:27.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T03:56:27.317 DEBUG:teuthology.orchestra.run.smithi076:> sudo umount --lazy --force /home/ubuntu/cephtest/mnt.0 2023-11-09T03:56:27.385 INFO:teuthology.orchestra.run.smithi076.stderr:mkdir: cannot create directory ‘d1/dir/.snap/s4’: Software caused connection abort 2023-11-09T03:56:27.386 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T03:56:28.288 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~20s 2023-11-09T03:56:33.626 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:56:33.629 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:56:34.901 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~36s 2023-11-09T03:56:41.513 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~43s 2023-11-09T03:56:48.126 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~49s 2023-11-09T03:56:54.739 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~56s 2023-11-09T03:57:01.351 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~63s 2023-11-09T03:57:03.696 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:57:03.700 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:57:07.962 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~69s 2023-11-09T03:57:14.572 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~76s 2023-11-09T03:57:21.187 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~82s 2023-11-09T03:57:27.800 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~89s 2023-11-09T03:57:33.861 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:57:33.865 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:57:34.413 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~96s 2023-11-09T03:57:41.026 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~102s 2023-11-09T03:57:47.638 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~109s 2023-11-09T03:57:54.248 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~115s 2023-11-09T03:58:00.861 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~122s 2023-11-09T03:58:03.919 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:58:03.922 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:58:07.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~129s 2023-11-09T03:58:14.082 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~135s 2023-11-09T03:58:20.693 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~142s 2023-11-09T03:58:27.305 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~149s 2023-11-09T03:58:33.915 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~155s 2023-11-09T03:58:34.064 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:58:34.068 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:58:40.529 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~162s 2023-11-09T03:58:47.144 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~168s 2023-11-09T03:58:53.751 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~175s 2023-11-09T03:59:00.361 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~182s 2023-11-09T03:59:04.125 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:59:04.137 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:59:06.971 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~188s 2023-11-09T03:59:13.585 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~195s 2023-11-09T03:59:20.196 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~201s 2023-11-09T03:59:26.807 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~208s 2023-11-09T03:59:33.421 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~215s 2023-11-09T03:59:34.268 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:59:34.272 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T03:59:40.034 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~221s 2023-11-09T03:59:46.644 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~228s 2023-11-09T03:59:53.255 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~234s 2023-11-09T03:59:59.865 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~241s 2023-11-09T04:00:04.333 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:00:04.336 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:00:06.475 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~248s 2023-11-09T04:00:13.087 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~254s 2023-11-09T04:00:19.699 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~261s 2023-11-09T04:00:26.310 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~268s 2023-11-09T04:00:32.923 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~274s 2023-11-09T04:00:34.431 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:00:34.435 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:00:39.535 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~281s 2023-11-09T04:00:46.149 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~287s 2023-11-09T04:00:52.758 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~294s 2023-11-09T04:00:59.370 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~301s 2023-11-09T04:00:59.371 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2023-11-09T04:00:59.372 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T04:00:59.372 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T04:00:59.393 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056381,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14761 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248868k,nr_inodes=812217,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-09T04:00:59.393 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi076.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-11-09T04:00:59.394 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T04:00:59.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T04:00:59.395 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-09T04:00:59.452 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T04:00:59.452 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T04:00:59.509 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056381,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14761 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248868k,nr_inodes=812217,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-09T04:00:59.510 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.2 is not mounted at ubuntu@smithi076.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.2 2023-11-09T04:00:59.510 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T04:00:59.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T04:00:59.511 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-09T04:00:59.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T04:00:59.567 INFO:teuthology.orchestra.run.smithi076.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-09T04:00:59.568 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T04:00:59.568 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T04:00:59.587 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2440 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-09T04:00:59.587 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.1 is not mounted at ubuntu@smithi113.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1 2023-11-09T04:00:59.587 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2023-11-09T04:00:59.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T04:00:59.588 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-09T04:00:59.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T04:00:59.645 INFO:teuthology.orchestra.run.smithi113.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-09T04:00:59.646 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T04:00:59.646 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T04:00:59.704 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2440 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-09T04:00:59.705 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.3 is not mounted at ubuntu@smithi113.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.3 2023-11-09T04:00:59.705 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2023-11-09T04:00:59.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T04:00:59.705 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-09T04:00:59.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T04:00:59.762 INFO:teuthology.orchestra.run.smithi113.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-09T04:01:01.369 INFO:tasks.ceph.osd.0:Sent signal 15 2023-11-09T04:01:01.370 INFO:tasks.ceph.osd.1:Sent signal 15 2023-11-09T04:01:01.370 INFO:tasks.ceph.osd.2:Sent signal 15 2023-11-09T04:01:01.371 INFO:tasks.ceph.osd.3:Sent signal 15 2023-11-09T04:01:01.371 INFO:tasks.ceph.osd.4:Sent signal 15 2023-11-09T04:01:01.371 INFO:tasks.ceph.osd.5:Sent signal 15 2023-11-09T04:01:01.372 INFO:tasks.ceph.osd.6:Sent signal 15 2023-11-09T04:01:01.372 INFO:tasks.ceph.osd.7:Sent signal 15 2023-11-09T04:01:01.372 INFO:tasks.ceph.mds.a:Sent signal 15 2023-11-09T04:01:01.373 INFO:tasks.ceph.mds.c:Sent signal 15 2023-11-09T04:01:01.373 INFO:tasks.ceph.mds.d:Sent signal 15 2023-11-09T04:01:01.374 INFO:tasks.ceph.mon.a:Sent signal 15 2023-11-09T04:01:01.374 INFO:tasks.ceph.mon.b:Sent signal 15 2023-11-09T04:01:01.375 INFO:tasks.ceph.mon.c:Sent signal 15 2023-11-09T04:01:01.375 INFO:tasks.ceph.mgr.y:Sent signal 15 2023-11-09T04:01:01.375 INFO:tasks.ceph.mgr.x:Sent signal 15 2023-11-09T04:01:01.376 INFO:tasks.ceph.osd.4.smithi113.stderr:2023-11-09T04:01:01.372+0000 7f8da9f7d640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 88438) UID: 0 2023-11-09T04:01:01.376 INFO:tasks.ceph.osd.4.smithi113.stderr:2023-11-09T04:01:01.372+0000 7f8da9f7d640 -1 osd.4 123 *** Got signal Terminated *** 2023-11-09T04:01:01.377 INFO:tasks.ceph.osd.4.smithi113.stderr:2023-11-09T04:01:01.372+0000 7f8da9f7d640 -1 osd.4 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-09T04:01:01.377 INFO:tasks.ceph.osd.5.smithi113.stderr:2023-11-09T04:01:01.373+0000 7f03995cd640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 88440) UID: 0 2023-11-09T04:01:01.377 INFO:tasks.ceph.osd.5.smithi113.stderr:2023-11-09T04:01:01.373+0000 7f03995cd640 -1 osd.5 123 *** Got signal Terminated *** 2023-11-09T04:01:01.377 INFO:tasks.ceph.osd.5.smithi113.stderr:2023-11-09T04:01:01.373+0000 7f03995cd640 -1 osd.5 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-09T04:01:01.378 INFO:tasks.ceph.osd.6.smithi113.stderr:2023-11-09T04:01:01.373+0000 7f2843dbe640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 88443) UID: 0 2023-11-09T04:01:01.378 INFO:tasks.ceph.osd.6.smithi113.stderr:2023-11-09T04:01:01.373+0000 7f2843dbe640 -1 osd.6 123 *** Got signal Terminated *** 2023-11-09T04:01:01.378 INFO:tasks.ceph.osd.6.smithi113.stderr:2023-11-09T04:01:01.373+0000 7f2843dbe640 -1 osd.6 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-09T04:01:01.379 INFO:tasks.ceph.osd.7.smithi113.stderr:2023-11-09T04:01:01.373+0000 7f8a0dabd640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 88445) UID: 0 2023-11-09T04:01:01.379 INFO:tasks.ceph.osd.7.smithi113.stderr:2023-11-09T04:01:01.373+0000 7f8a0dabd640 -1 osd.7 123 *** Got signal Terminated *** 2023-11-09T04:01:01.379 INFO:tasks.ceph.osd.7.smithi113.stderr:2023-11-09T04:01:01.373+0000 7f8a0dabd640 -1 osd.7 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-09T04:01:01.380 INFO:tasks.ceph.mds.d.smithi113.stderr:2023-11-09T04:01:01.375+0000 7f1d4fda7640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i d (PID: 95539) UID: 0 2023-11-09T04:01:01.380 INFO:tasks.ceph.mds.d.smithi113.stderr:2023-11-09T04:01:01.375+0000 7f1d4fda7640 -1 mds.d *** got signal Terminated *** 2023-11-09T04:01:01.381 INFO:tasks.ceph.mon.b.smithi113.stderr:2023-11-09T04:01:01.376+0000 7f525bd19640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i b (PID: 87869) UID: 0 2023-11-09T04:01:01.381 INFO:tasks.ceph.mon.b.smithi113.stderr:2023-11-09T04:01:01.376+0000 7f525bd19640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2023-11-09T04:01:01.381 INFO:tasks.ceph.mon.c.smithi113.stderr:2023-11-09T04:01:01.376+0000 7f1f88ad1640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i c (PID: 87871) UID: 0 2023-11-09T04:01:01.381 INFO:tasks.ceph.mon.c.smithi113.stderr:2023-11-09T04:01:01.376+0000 7f1f88ad1640 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2023-11-09T04:01:01.382 INFO:tasks.ceph.osd.0.smithi076.stderr:2023-11-09T04:01:01.371+0000 7f12c2c13640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 87974) UID: 0 2023-11-09T04:01:01.382 INFO:tasks.ceph.osd.0.smithi076.stderr:2023-11-09T04:01:01.371+0000 7f12c2c13640 -1 osd.0 123 *** Got signal Terminated *** 2023-11-09T04:01:01.383 INFO:tasks.ceph.osd.0.smithi076.stderr:2023-11-09T04:01:01.371+0000 7f12c2c13640 -1 osd.0 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-09T04:01:01.383 INFO:tasks.ceph.osd.1.smithi076.stderr:2023-11-09T04:01:01.371+0000 7fcb294bd640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 87982) UID: 0 2023-11-09T04:01:01.383 INFO:tasks.ceph.osd.1.smithi076.stderr:2023-11-09T04:01:01.371+0000 7fcb294bd640 -1 osd.1 123 *** Got signal Terminated *** 2023-11-09T04:01:01.383 INFO:tasks.ceph.osd.1.smithi076.stderr:2023-11-09T04:01:01.371+0000 7fcb294bd640 -1 osd.1 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-09T04:01:01.384 INFO:tasks.ceph.osd.2.smithi076.stderr:2023-11-09T04:01:01.371+0000 7f8f08c72640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 87994) UID: 0 2023-11-09T04:01:01.384 INFO:tasks.ceph.osd.2.smithi076.stderr:2023-11-09T04:01:01.371+0000 7f8f08c72640 -1 osd.2 123 *** Got signal Terminated *** 2023-11-09T04:01:01.384 INFO:tasks.ceph.osd.2.smithi076.stderr:2023-11-09T04:01:01.371+0000 7f8f08c72640 -1 osd.2 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-09T04:01:01.385 INFO:tasks.ceph.osd.3.smithi076.stderr:2023-11-09T04:01:01.372+0000 7fd71a8c7640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 87993) UID: 0 2023-11-09T04:01:01.385 INFO:tasks.ceph.osd.3.smithi076.stderr:2023-11-09T04:01:01.372+0000 7fd71a8c7640 -1 osd.3 123 *** Got signal Terminated *** 2023-11-09T04:01:01.385 INFO:tasks.ceph.osd.3.smithi076.stderr:2023-11-09T04:01:01.372+0000 7fd71a8c7640 -1 osd.3 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-09T04:01:01.386 INFO:tasks.ceph.mds.a.smithi076.stderr:2023-11-09T04:01:01.374+0000 7f8069504640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i a (PID: 132781) UID: 0 2023-11-09T04:01:01.386 INFO:tasks.ceph.mds.a.smithi076.stderr:2023-11-09T04:01:01.374+0000 7f8069504640 -1 mds.a *** got signal Terminated *** 2023-11-09T04:01:01.386 INFO:tasks.ceph.mds.c.smithi076.stderr:2023-11-09T04:01:01.374+0000 7f1aa0217640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i c (PID: 124528) UID: 0 2023-11-09T04:01:01.387 INFO:tasks.ceph.mds.c.smithi076.stderr:2023-11-09T04:01:01.374+0000 7f1aa0217640 -1 mds.c *** got signal Terminated *** 2023-11-09T04:01:01.387 INFO:tasks.ceph.mon.a.smithi076.stderr:2023-11-09T04:01:01.375+0000 7fe4a9148640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 87710) UID: 0 2023-11-09T04:01:01.387 INFO:tasks.ceph.mon.a.smithi076.stderr:2023-11-09T04:01:01.375+0000 7fe4a9148640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-11-09T04:01:01.577 INFO:tasks.ceph.mgr.y.smithi076.stderr:daemon-helper: command crashed with signal 15 2023-11-09T04:01:04.572 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:01:04.575 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:01:08.937 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2023-11-09T04:01:08 stdin-killer ERROR: timeout expired: sending signal 9 to command and myself 2023-11-09T04:01:08.941 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:bash: line 1: 133862 Killed ( cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 ) 2023-11-09T04:01:14.153 DEBUG:teuthology.orchestra.run:got remote process result: 137 2023-11-09T04:01:14.153 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T04:01:14.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T04:01:14.154 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-09T04:01:14.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T04:01:14.173 INFO:teuthology.orchestra.run.smithi076.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-09T04:01:14.174 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T04:01:14.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:14.326+0000 7f1cef190640 1 Processor -- start 2023-11-09T04:01:14.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:14.326+0000 7f1cef190640 1 -- start start 2023-11-09T04:01:14.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:14.326+0000 7f1cef190640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1ce815d100 0x7f1ce80ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:01:14.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:14.327+0000 7f1cef190640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b32b0 con 0x7f1ce815d4e0 2023-11-09T04:01:14.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:14.327+0000 7f1cef190640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80cee10 con 0x7f1ce80d1be0 2023-11-09T04:01:14.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:14.327+0000 7f1cef190640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1ce815d100 2023-11-09T04:01:14.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:14.327+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1ce815d4e0 legacy=0x7f1ce80d14d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:14.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:14.327+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1ce815d100 msgr2=0x7f1ce80ce510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:14.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:14.327+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1ce815d100 0x7f1ce80ce510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:01:14.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:14.327+0000 7f1ced72d640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1ce80d1be0 legacy=0x7f1ce80b3740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:14.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:14.528+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1ce815d4e0 legacy=0x7f1ce80d14d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:14.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:14.528+0000 7f1ced72d640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1ce80d1be0 legacy=0x7f1ce80b3740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:14.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:14.528+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1ce815d100 msgr2=0x7f1ce80ce510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:14.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:14.528+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1ce815d100 0x7f1ce80ce510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:01:14.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:14.929+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1ce815d4e0 legacy=0x7f1ce80d14d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:14.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:14.929+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1ce815d100 msgr2=0x7f1ce80ce510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:14.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:14.929+0000 7f1ced72d640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1ce80d1be0 legacy=0x7f1ce80b3740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:14.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:14.929+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1ce815d100 0x7f1ce80ce510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:01:15.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:15.730+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1ce815d4e0 legacy=0x7f1ce80d14d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:15.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:15.730+0000 7f1ced72d640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1ce80d1be0 legacy=0x7f1ce80b3740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:15.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:15.730+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1ce815d100 msgr2=0x7f1ce80ce510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:15.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:15.730+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1ce815d100 0x7f1ce80ce510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:01:17.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:17.327+0000 7f1ce67fc640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1ce815d100 msgr2=0x7f1ce80ce510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:17.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:17.327+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1ce815d100 0x7f1ce80ce510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:01:17.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:17.327+0000 7f1ce67fc640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1ce80d1be0 legacy=0x7f1ce80b3740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:17.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:17.327+0000 7f1ce67fc640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1ce815d4e0 legacy=0x7f1ce80d14d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:17.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:17.327+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80036a0 0x7f1cc8005af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:01:17.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:17.327+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006130 0x7f1cc80085f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:01:17.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:17.327+0000 7f1ce67fc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc8006130 2023-11-09T04:01:17.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:17.327+0000 7f1ce67fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc8008ca0 con 0x7f1cc80036a0 2023-11-09T04:01:17.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:17.327+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006130 msgr2=0x7f1cc80085f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:17.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:17.327+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006130 0x7f1cc80085f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:01:17.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:17.327+0000 7f1cecf2c640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80036a0 msgr2=0x7f1cc8005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:17.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:17.327+0000 7f1cecf2c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80036a0 0x7f1cc8005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:01:17.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:17.528+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006130 msgr2=0x7f1cc80085f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:17.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:17.528+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006130 0x7f1cc80085f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:01:17.528 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:17.528+0000 7f1cecf2c640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80036a0 msgr2=0x7f1cc8005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:17.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:17.528+0000 7f1cecf2c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80036a0 0x7f1cc8005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:01:17.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:17.929+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006130 msgr2=0x7f1cc80085f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:17.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:17.929+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006130 0x7f1cc80085f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:01:17.929 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:17.929+0000 7f1cecf2c640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80036a0 msgr2=0x7f1cc8005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:17.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:17.929+0000 7f1cecf2c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80036a0 0x7f1cc8005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:01:18.730 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:18.730+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006130 msgr2=0x7f1cc80085f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:18.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:18.730+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006130 0x7f1cc80085f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:01:18.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:18.730+0000 7f1cecf2c640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80036a0 msgr2=0x7f1cc8005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:18.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:18.730+0000 7f1cecf2c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80036a0 0x7f1cc8005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:01:20.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:20.328+0000 7f1ce67fc640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80036a0 msgr2=0x7f1cc8005af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:20.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:20.328+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80036a0 0x7f1cc8005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:01:20.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:20.328+0000 7f1ce67fc640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006130 msgr2=0x7f1cc80085f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:20.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:20.328+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006130 0x7f1cc80085f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:01:20.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:20.328+0000 7f1ced72d640 1 -- reap_dead start 2023-11-09T04:01:20.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:20.328+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8007d00 0x7f1cc8008190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:01:20.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:20.328+0000 7f1ce67fc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b2c60 con 0x7f1cc8003ae0 2023-11-09T04:01:20.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:20.328+0000 7f1ce67fc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b32b0 con 0x7f1ce80d1be0 2023-11-09T04:01:20.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:20.328+0000 7f1ce67fc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc8007d00 2023-11-09T04:01:20.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:20.328+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8003ae0 legacy=0x7f1cc80065e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:20.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:20.328+0000 7f1ced72d640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8007d00 msgr2=0x7f1cc8008190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:20.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:20.328+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1ce80d1be0 legacy=0x7f1cc800af90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:20.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:20.328+0000 7f1ced72d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8007d00 0x7f1cc8008190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:01:20.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:20.529+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8003ae0 legacy=0x7f1cc80065e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:20.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:20.529+0000 7f1ced72d640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8007d00 msgr2=0x7f1cc8008190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:20.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:20.529+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1ce80d1be0 legacy=0x7f1cc800af90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:20.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:20.529+0000 7f1ced72d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8007d00 0x7f1cc8008190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:01:20.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:20.930+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8003ae0 legacy=0x7f1cc80065e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:20.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:20.930+0000 7f1ced72d640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8007d00 msgr2=0x7f1cc8008190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:20.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:20.930+0000 7f1ced72d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8007d00 0x7f1cc8008190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:01:20.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:20.930+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1ce80d1be0 legacy=0x7f1cc800af90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:21.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:21.732+0000 7f1ced72d640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8007d00 msgr2=0x7f1cc8008190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:21.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:21.732+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8003ae0 legacy=0x7f1cc80065e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:21.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:21.732+0000 7f1ced72d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8007d00 0x7f1cc8008190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:01:21.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:21.732+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1ce80d1be0 legacy=0x7f1cc800af90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:23.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:23.328+0000 7f1ce67fc640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8007d00 msgr2=0x7f1cc8008190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:23.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:23.328+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8007d00 0x7f1cc8008190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:01:23.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:23.328+0000 7f1ce67fc640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1ce80d1be0 legacy=0x7f1cc800af90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:23.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:23.328+0000 7f1ce67fc640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8003ae0 legacy=0x7f1cc80065e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:23.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:23.328+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800e6c0 0x7f1cc8010ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:01:23.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:23.328+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ff0 0x7f1cc80133e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:01:23.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:23.328+0000 7f1ce67fc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc800e6c0 2023-11-09T04:01:23.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:23.328+0000 7f1ce67fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc8008ca0 con 0x7f1cc8010ff0 2023-11-09T04:01:23.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:23.329+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800e6c0 msgr2=0x7f1cc8010ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:23.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:23.329+0000 7f1ce7fff640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ff0 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:23.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:23.329+0000 7f1ce7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ff0 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:01:23.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:23.329+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800e6c0 0x7f1cc8010ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:01:23.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:23.530+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800e6c0 msgr2=0x7f1cc8010ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:23.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:23.530+0000 7f1ce7fff640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ff0 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:23.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:23.530+0000 7f1ce7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ff0 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:01:23.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:23.530+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800e6c0 0x7f1cc8010ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:01:23.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:23.931+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800e6c0 msgr2=0x7f1cc8010ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:23.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:23.931+0000 7f1ce7fff640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ff0 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:23.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:23.931+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800e6c0 0x7f1cc8010ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:01:23.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:23.931+0000 7f1ce7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ff0 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:01:24.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:24.732+0000 7f1ce7fff640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ff0 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:24.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:24.732+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800e6c0 msgr2=0x7f1cc8010ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:24.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:24.732+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800e6c0 0x7f1cc8010ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:01:24.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:24.732+0000 7f1ce7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ff0 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:01:26.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:26.329+0000 7f1ce67fc640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ff0 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:26.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:26.329+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ff0 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:01:26.329 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:26.329+0000 7f1ce67fc640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800e6c0 msgr2=0x7f1cc8010ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:26.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:26.329+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800e6c0 0x7f1cc8010ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:01:26.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:26.329+0000 7f1ced72d640 1 -- reap_dead start 2023-11-09T04:01:26.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:26.329+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 0x7f1cc8007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:01:26.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:26.329+0000 7f1ce67fc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80cee10 con 0x7f1cc8003ae0 2023-11-09T04:01:26.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:26.329+0000 7f1ce67fc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b2c60 con 0x7f1cc8008190 2023-11-09T04:01:26.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:26.329+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8003ae0 legacy=0x7f1cc8006600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:26.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:26.329+0000 7f1ce67fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc8010ab0 2023-11-09T04:01:26.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:26.329+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8008190 legacy=0x7f1cc8006d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:26.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:26.329+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 msgr2=0x7f1cc8007420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:26.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:26.329+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 0x7f1cc8007420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:01:26.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:26.530+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8003ae0 legacy=0x7f1cc8006600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:26.529 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:26.530+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8008190 legacy=0x7f1cc8006d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:26.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:26.530+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 msgr2=0x7f1cc8007420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:26.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:26.530+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 0x7f1cc8007420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:01:26.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:26.931+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8003ae0 legacy=0x7f1cc8006600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:26.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:26.931+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8008190 legacy=0x7f1cc8006d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:26.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:26.931+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 msgr2=0x7f1cc8007420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:26.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:26.931+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 0x7f1cc8007420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:01:27.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:27.732+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8003ae0 legacy=0x7f1cc8006600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:27.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:27.732+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8008190 legacy=0x7f1cc8006d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:27.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:27.732+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 msgr2=0x7f1cc8007420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:27.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:27.732+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 0x7f1cc8007420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:01:29.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:29.329+0000 7f1ce67fc640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 msgr2=0x7f1cc8007420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:29.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:29.329+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 0x7f1cc8007420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:01:29.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:29.329+0000 7f1ce67fc640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8008190 legacy=0x7f1cc8006d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:29.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:29.329+0000 7f1ce67fc640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8003ae0 legacy=0x7f1cc8006600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:29.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:29.329+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8007960 0x7f1cc80123e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:01:29.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:29.329+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8012920 0x7f1cc8012d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:01:29.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:29.329+0000 7f1ce67fc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc8012920 2023-11-09T04:01:29.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:29.329+0000 7f1ce67fc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cc8008ca0 con 0x7f1cc8007960 2023-11-09T04:01:29.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:29.329+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8012920 msgr2=0x7f1cc8012d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:29.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:29.329+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8012920 0x7f1cc8012d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:01:29.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:29.329+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8007960 msgr2=0x7f1cc80123e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:29.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:29.329+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8007960 0x7f1cc80123e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:01:29.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:29.530+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8012920 msgr2=0x7f1cc8012d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:29.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:29.530+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8012920 0x7f1cc8012d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:01:29.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:29.530+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8007960 msgr2=0x7f1cc80123e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:29.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:29.530+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8007960 0x7f1cc80123e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:01:29.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:29.931+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8012920 msgr2=0x7f1cc8012d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:29.930 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:29.931+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8012920 0x7f1cc8012d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:01:29.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:29.931+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8007960 msgr2=0x7f1cc80123e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:29.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:29.931+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8007960 0x7f1cc80123e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:01:30.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:30.732+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8012920 msgr2=0x7f1cc8012d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:30.731 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:30.732+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8012920 0x7f1cc8012d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:01:30.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:30.732+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8007960 msgr2=0x7f1cc80123e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:30.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:30.732+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8007960 0x7f1cc80123e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:01:32.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:32.330+0000 7f1ce67fc640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8007960 msgr2=0x7f1cc80123e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:32.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:32.330+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8007960 0x7f1cc80123e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:01:32.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:32.330+0000 7f1ce67fc640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8012920 msgr2=0x7f1cc8012d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:32.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:32.330+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8012920 0x7f1cc8012d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:01:32.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:32.330+0000 7f1ced72d640 1 -- reap_dead start 2023-11-09T04:01:32.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:32.330+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 0x7f1cc8007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:01:32.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:32.330+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8008190 0x7f1cc80133e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:01:32.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:32.330+0000 7f1ce67fc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b32b0 con 0x7f1cc8007960 2023-11-09T04:01:32.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:32.330+0000 7f1ce67fc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc8008190 2023-11-09T04:01:32.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:32.330+0000 7f1ce67fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc8008ca0 con 0x7f1cc8010ab0 2023-11-09T04:01:32.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:32.330+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8007960 legacy=0x7f1cc8006600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:32.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:32.330+0000 7f1cecf2c640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 msgr2=0x7f1cc8007420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:32.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:32.330+0000 7f1cecf2c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 0x7f1cc8007420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:01:32.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:32.330+0000 7f1ced72d640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8008190 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:32.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:32.330+0000 7f1ced72d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8008190 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:01:32.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:32.531+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8007960 legacy=0x7f1cc8006600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:32.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:32.531+0000 7f1cecf2c640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 msgr2=0x7f1cc8007420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:32.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:32.531+0000 7f1ced72d640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8008190 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:32.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:32.531+0000 7f1cecf2c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 0x7f1cc8007420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:01:32.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:32.531+0000 7f1ced72d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8008190 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:01:32.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:32.932+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8007960 legacy=0x7f1cc8006600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:32.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:32.932+0000 7f1cecf2c640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 msgr2=0x7f1cc8007420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:32.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:32.932+0000 7f1cecf2c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 0x7f1cc8007420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:01:32.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:32.932+0000 7f1ced72d640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8008190 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:32.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:32.932+0000 7f1ced72d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8008190 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:01:33.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:33.733+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8007960 legacy=0x7f1cc8006600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:33.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:33.733+0000 7f1cecf2c640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 msgr2=0x7f1cc8007420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:33.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:33.733+0000 7f1cecf2c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 0x7f1cc8007420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:01:33.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:33.733+0000 7f1ced72d640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8008190 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:33.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:33.733+0000 7f1ced72d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8008190 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:01:34.708 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:01:34.711 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:01:35.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:35.330+0000 7f1ce67fc640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 msgr2=0x7f1cc8007420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:35.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:35.330+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 0x7f1cc8007420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:01:35.330 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:35.330+0000 7f1ce67fc640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8008190 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:35.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:35.330+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8008190 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:01:35.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:35.330+0000 7f1ce67fc640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8007960 legacy=0x7f1cc8006600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:35.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:35.330+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006d10 0x7f1cc8004af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:01:35.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:35.330+0000 7f1ce67fc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80cee10 con 0x7f1cc8005030 2023-11-09T04:01:35.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:35.330+0000 7f1ce67fc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc8006d10 2023-11-09T04:01:35.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:35.330+0000 7f1cecf2c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006d10 msgr2=0x7f1cc8004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:35.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:35.330+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006d10 0x7f1cc8004af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:01:35.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:35.330+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8005030 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:35.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:35.531+0000 7f1cecf2c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006d10 msgr2=0x7f1cc8004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:35.530 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:35.531+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006d10 0x7f1cc8004af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:01:35.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:35.531+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8005030 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:35.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:35.932+0000 7f1cecf2c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006d10 msgr2=0x7f1cc8004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:35.931 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:35.932+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006d10 0x7f1cc8004af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:01:35.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:35.932+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8005030 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:36.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:36.733+0000 7f1cecf2c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006d10 msgr2=0x7f1cc8004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:36.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:36.733+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006d10 0x7f1cc8004af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:01:36.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:36.733+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8005030 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:38.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:38.330+0000 7f1ce67fc640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006d10 msgr2=0x7f1cc8004af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:38.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:38.330+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006d10 0x7f1cc8004af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:01:38.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:38.330+0000 7f1ce67fc640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8005030 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:38.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:38.330+0000 7f1ced72d640 1 -- reap_dead start 2023-11-09T04:01:38.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:38.330+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 0x7f1cc8006600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:01:38.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:38.330+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8006b40 0x7f1cc8007240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:01:38.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:38.330+0000 7f1ce67fc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b32b0 con 0x7f1cc800dfb0 2023-11-09T04:01:38.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:38.330+0000 7f1ce67fc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc8008190 2023-11-09T04:01:38.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:38.330+0000 7f1ce67fc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cc8008ca0 con 0x7f1cc8006b40 2023-11-09T04:01:38.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:38.330+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 msgr2=0x7f1cc8006600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:38.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:38.330+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 0x7f1cc8006600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:01:38.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:38.330+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800dfb0 legacy=0x7f1cc8017ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:38.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:38.330+0000 7f1ced72d640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8006b40 msgr2=0x7f1cc8007240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:38.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:38.330+0000 7f1ced72d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8006b40 0x7f1cc8007240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:01:38.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:38.531+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 msgr2=0x7f1cc8006600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:38.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:38.531+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 0x7f1cc8006600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:01:38.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:38.531+0000 7f1ced72d640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8006b40 msgr2=0x7f1cc8007240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:38.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:38.531+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800dfb0 legacy=0x7f1cc8017ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:38.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:38.531+0000 7f1ced72d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8006b40 0x7f1cc8007240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:01:38.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:38.932+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 msgr2=0x7f1cc8006600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:38.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:38.932+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 0x7f1cc8006600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:01:38.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:38.932+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800dfb0 legacy=0x7f1cc8017ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:38.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:38.932+0000 7f1ced72d640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8006b40 msgr2=0x7f1cc8007240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:38.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:38.932+0000 7f1ced72d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8006b40 0x7f1cc8007240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:01:39.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:39.733+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 msgr2=0x7f1cc8006600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:39.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:39.733+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 0x7f1cc8006600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:01:39.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:39.733+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800dfb0 legacy=0x7f1cc8017ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:39.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:39.733+0000 7f1ced72d640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8006b40 msgr2=0x7f1cc8007240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:39.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:39.733+0000 7f1ced72d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8006b40 0x7f1cc8007240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:01:41.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:41.330+0000 7f1ce67fc640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8006b40 msgr2=0x7f1cc8007240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:41.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:41.330+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8006b40 0x7f1cc8007240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:01:41.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:41.330+0000 7f1ce67fc640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 msgr2=0x7f1cc8006600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:41.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:41.330+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 0x7f1cc8006600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:01:41.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:41.330+0000 7f1ce67fc640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800dfb0 legacy=0x7f1cc8017ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:41.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:41.330+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc80133e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:01:41.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:41.330+0000 7f1ce67fc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80cee10 con 0x7f1cc8010ab0 2023-11-09T04:01:41.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:41.330+0000 7f1ce67fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc8005b50 2023-11-09T04:01:41.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:41.330+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8010ab0 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:41.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:41.330+0000 7f1ce7fff640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:41.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:41.330+0000 7f1ce7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:01:41.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:41.531+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8010ab0 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:41.531 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:41.531+0000 7f1ce7fff640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:41.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:41.531+0000 7f1ce7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:01:41.932 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:41.932+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8010ab0 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:41.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:41.932+0000 7f1ce7fff640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:41.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:41.932+0000 7f1ce7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:01:42.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:42.733+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8010ab0 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:42.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:42.733+0000 7f1ce7fff640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:42.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:42.733+0000 7f1ce7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:01:44.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:44.330+0000 7f1ce67fc640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:44.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:44.330+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:01:44.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:44.330+0000 7f1ce67fc640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8010ab0 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:44.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:44.330+0000 7f1ced72d640 1 -- reap_dead start 2023-11-09T04:01:44.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:44.330+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8006b40 0x7f1cc8004ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:01:44.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:44.330+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 0x7f1cc8006600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:01:44.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:44.330+0000 7f1ce67fc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b32b0 con 0x7f1cc8008190 2023-11-09T04:01:44.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:44.330+0000 7f1ce67fc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc8006b40 2023-11-09T04:01:44.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:44.330+0000 7f1ce67fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc8008ca0 con 0x7f1cc8010ab0 2023-11-09T04:01:44.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:44.330+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8008190 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:44.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:44.330+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8006b40 msgr2=0x7f1cc8004ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:44.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:44.330+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8006b40 0x7f1cc8004ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:01:44.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:44.330+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 msgr2=0x7f1cc8006600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:44.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:44.330+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 0x7f1cc8006600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:01:44.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:44.531+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8008190 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:44.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:44.531+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8006b40 msgr2=0x7f1cc8004ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:44.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:44.531+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8006b40 0x7f1cc8004ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:01:44.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:44.531+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 msgr2=0x7f1cc8006600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:44.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:44.531+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 0x7f1cc8006600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:01:44.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:44.932+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8008190 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:44.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:44.932+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8006b40 msgr2=0x7f1cc8004ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:44.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:44.932+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8006b40 0x7f1cc8004ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:01:44.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:44.932+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 msgr2=0x7f1cc8006600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:44.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:44.932+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 0x7f1cc8006600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:01:45.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:45.733+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8008190 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:45.733 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:45.733+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8006b40 msgr2=0x7f1cc8004ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:45.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:45.733+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8006b40 0x7f1cc8004ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:01:45.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:45.733+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 msgr2=0x7f1cc8006600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:45.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:45.733+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 0x7f1cc8006600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:01:47.331 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:47.331+0000 7f1ce67fc640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 msgr2=0x7f1cc8006600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:47.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:47.331+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8010ab0 0x7f1cc8006600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:01:47.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:47.331+0000 7f1ce67fc640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8006b40 msgr2=0x7f1cc8004ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:47.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:47.331+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8006b40 0x7f1cc8004ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:01:47.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:47.331+0000 7f1ce67fc640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8008190 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:47.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:47.331+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8005b50 0x7f1cc80133e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:01:47.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:47.331+0000 7f1ce67fc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80cee10 con 0x7f1cc8007190 2023-11-09T04:01:47.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:47.331+0000 7f1ce67fc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc8005b50 2023-11-09T04:01:47.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:47.331+0000 7f1cecf2c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8005b50 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:47.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:47.331+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8005b50 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:01:47.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:47.331+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8007190 legacy=0x7f1cc80075a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:47.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:47.532+0000 7f1cecf2c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8005b50 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:47.532 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:47.532+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8005b50 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:01:47.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:47.532+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8007190 legacy=0x7f1cc80075a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:47.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:47.933+0000 7f1cecf2c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8005b50 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:47.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:47.933+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8005b50 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:01:47.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:47.933+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8007190 legacy=0x7f1cc80075a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:48.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:48.734+0000 7f1cecf2c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8005b50 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:48.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:48.734+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8005b50 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:01:48.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:48.734+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8007190 legacy=0x7f1cc80075a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:50.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:50.331+0000 7f1ce67fc640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8005b50 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:50.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:50.331+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8005b50 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:01:50.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:50.331+0000 7f1ce67fc640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8007190 legacy=0x7f1cc80075a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:50.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:50.331+0000 7f1ced72d640 1 -- reap_dead start 2023-11-09T04:01:50.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:50.331+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8008190 0x7f1cc80075a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:01:50.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:50.331+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006600 0x7f1cc80069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:01:50.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:50.331+0000 7f1ce67fc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b32b0 con 0x7f1cc800e050 2023-11-09T04:01:50.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:50.331+0000 7f1ce67fc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc8006600 2023-11-09T04:01:50.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:50.331+0000 7f1ce67fc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cc8008ca0 con 0x7f1cc8008190 2023-11-09T04:01:50.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:50.331+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800e050 legacy=0x7f1cc8017ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:50.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:50.331+0000 7f1ce7fff640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8008190 msgr2=0x7f1cc80075a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:50.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:50.331+0000 7f1ced72d640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006600 msgr2=0x7f1cc80069e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:50.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:50.331+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8008190 0x7f1cc80075a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:01:50.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:50.331+0000 7f1ced72d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006600 0x7f1cc80069e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:01:50.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:50.532+0000 7f1ced72d640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006600 msgr2=0x7f1cc80069e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:50.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:50.532+0000 7f1ced72d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006600 0x7f1cc80069e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:01:50.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:50.532+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800e050 legacy=0x7f1cc8017ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:50.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:50.532+0000 7f1ce7fff640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8008190 msgr2=0x7f1cc80075a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:50.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:50.532+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8008190 0x7f1cc80075a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:01:50.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:50.933+0000 7f1ced72d640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006600 msgr2=0x7f1cc80069e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:50.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:50.933+0000 7f1ced72d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006600 0x7f1cc80069e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:01:50.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:50.933+0000 7f1ce7fff640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8008190 msgr2=0x7f1cc80075a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:50.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:50.933+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800e050 legacy=0x7f1cc8017ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:50.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:50.933+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8008190 0x7f1cc80075a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:01:51.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:51.734+0000 7f1ced72d640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006600 msgr2=0x7f1cc80069e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:51.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:51.734+0000 7f1ced72d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006600 0x7f1cc80069e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:01:51.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:51.734+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800e050 legacy=0x7f1cc8017ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:51.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:51.734+0000 7f1ce7fff640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8008190 msgr2=0x7f1cc80075a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:51.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:51.734+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8008190 0x7f1cc80075a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:01:53.332 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:53.332+0000 7f1ce67fc640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8008190 msgr2=0x7f1cc80075a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:53.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:53.332+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8008190 0x7f1cc80075a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:01:53.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:53.332+0000 7f1ce67fc640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006600 msgr2=0x7f1cc80069e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:53.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:53.332+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8006600 0x7f1cc80069e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:01:53.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:53.332+0000 7f1ce67fc640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800e050 legacy=0x7f1cc8017ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:53.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:53.332+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc80133e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:01:53.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:53.332+0000 7f1ce67fc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80cee10 con 0x7f1cc8010ab0 2023-11-09T04:01:53.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:53.332+0000 7f1ce67fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc8005b50 2023-11-09T04:01:53.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:53.332+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8010ab0 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:53.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:53.332+0000 7f1ce7fff640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:53.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:53.332+0000 7f1ce7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:01:53.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:53.533+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8010ab0 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:53.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:53.533+0000 7f1ce7fff640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:53.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:53.533+0000 7f1ce7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:01:53.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:53.934+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8010ab0 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:53.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:53.934+0000 7f1ce7fff640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:53.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:53.934+0000 7f1ce7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:01:54.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:54.735+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8010ab0 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:54.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:54.735+0000 7f1ce7fff640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:54.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:54.735+0000 7f1ce7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:01:56.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:56.332+0000 7f1ce67fc640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:56.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:56.332+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:01:56.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:56.332+0000 7f1ce67fc640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8010ab0 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:56.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:56.332+0000 7f1ced72d640 1 -- reap_dead start 2023-11-09T04:01:56.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:56.332+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 0x7f1cc80075a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:01:56.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:56.332+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:01:56.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:56.332+0000 7f1ce67fc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b32b0 con 0x7f1cc8006600 2023-11-09T04:01:56.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:56.332+0000 7f1ce67fc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc8008190 2023-11-09T04:01:56.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:56.332+0000 7f1ce67fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc8008ca0 con 0x7f1cc8005b50 2023-11-09T04:01:56.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:56.332+0000 7f1cecf2c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 msgr2=0x7f1cc80075a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:56.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:56.332+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 0x7f1cc80075a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:01:56.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:56.332+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8006600 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:56.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:56.332+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:56.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:56.332+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:01:56.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:56.533+0000 7f1cecf2c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 msgr2=0x7f1cc80075a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:56.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:56.533+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 0x7f1cc80075a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:01:56.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:56.533+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8006600 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:56.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:56.533+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:56.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:56.533+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:01:56.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:56.934+0000 7f1cecf2c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 msgr2=0x7f1cc80075a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:56.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:56.934+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 0x7f1cc80075a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:01:56.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:56.934+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8006600 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:56.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:56.934+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:56.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:56.934+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:01:57.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:57.735+0000 7f1cecf2c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 msgr2=0x7f1cc80075a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:01:57.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:57.735+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 0x7f1cc80075a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:01:57.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:57.735+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8006600 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:01:57.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:57.735+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:01:57.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:57.735+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:01:59.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:59.332+0000 7f1ce67fc640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8004a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:59.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:59.332+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:01:59.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:59.332+0000 7f1ce67fc640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 msgr2=0x7f1cc80075a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:59.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:59.332+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 0x7f1cc80075a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:01:59.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:59.332+0000 7f1ce67fc640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8006600 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:01:59.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:59.332+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80069e0 0x7f1cc80133e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:01:59.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:59.332+0000 7f1ce67fc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80cee10 con 0x7f1cc8010ab0 2023-11-09T04:01:59.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:59.332+0000 7f1ce67fc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc80069e0 2023-11-09T04:01:59.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:59.332+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8010ab0 legacy=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:59.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:59.333+0000 7f1ce7fff640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80069e0 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:59.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:59.333+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80069e0 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:01:59.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:59.533+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8010ab0 legacy=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:59.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:59.534+0000 7f1ce7fff640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80069e0 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:59.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:59.534+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80069e0 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:01:59.933 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:59.934+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8010ab0 legacy=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:01:59.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:59.935+0000 7f1ce7fff640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80069e0 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:01:59.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:01:59.935+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80069e0 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:00.734 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:00.735+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8010ab0 legacy=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:00.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:00.736+0000 7f1ce7fff640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80069e0 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:00.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:00.736+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80069e0 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:02.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:02.333+0000 7f1ce67fc640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80069e0 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:02.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:02.333+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80069e0 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:02.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:02.333+0000 7f1ce67fc640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8010ab0 legacy=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:02.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:02.333+0000 7f1ced72d640 1 -- reap_dead start 2023-11-09T04:02:02.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:02.333+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800d750 0x7f1cc80133e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:02.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:02.333+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:02.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:02.333+0000 7f1ce67fc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b32b0 con 0x7f1cc8006600 2023-11-09T04:02:02.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:02.333+0000 7f1ce67fc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc800d750 2023-11-09T04:02:02.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:02.333+0000 7f1ce67fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc8008ca0 con 0x7f1cc8005b50 2023-11-09T04:02:02.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:02.333+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8006600 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:02.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:02.333+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800d750 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:02.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:02.333+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:02.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:02.333+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800d750 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:02.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:02.333+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:02.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:02.534+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800d750 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:02.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:02.534+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8006600 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:02.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:02.534+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:02.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:02.534+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800d750 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:02.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:02.534+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:02.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:02.935+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8006600 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:02.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:02.935+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800d750 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:02.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:02.935+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:02.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:02.935+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800d750 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:02.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:02.935+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:03.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:03.736+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8006600 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:03.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:03.736+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800d750 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:03.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:03.736+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:03.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:03.736+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800d750 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:03.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:03.736+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:05.000 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:02:05.003 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:02:05.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:05.333+0000 7f1ce67fc640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8004a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:05.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:05.333+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:05.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:05.333+0000 7f1ce67fc640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800d750 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:05.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:05.333+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800d750 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:05.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:05.333+0000 7f1ce67fc640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8006600 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:05.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:05.333+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 0x7f1cc80075a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:05.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:05.333+0000 7f1ce67fc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80cee10 con 0x7f1cc8010ab0 2023-11-09T04:02:05.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:05.333+0000 7f1ce67fc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc8008190 2023-11-09T04:02:05.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:05.333+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 msgr2=0x7f1cc80075a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:05.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:05.333+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8010ab0 legacy=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:05.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:05.333+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 0x7f1cc80075a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:05.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:05.534+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 msgr2=0x7f1cc80075a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:05.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:05.534+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8010ab0 legacy=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:05.533 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:05.534+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 0x7f1cc80075a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:05.934 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:05.935+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8010ab0 legacy=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:05.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:05.935+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 msgr2=0x7f1cc80075a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:05.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:05.935+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 0x7f1cc80075a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:06.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:06.736+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 msgr2=0x7f1cc80075a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:06.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:06.736+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8010ab0 legacy=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:06.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:06.736+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 0x7f1cc80075a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:08.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:08.333+0000 7f1ce67fc640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 msgr2=0x7f1cc80075a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:08.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:08.333+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8008190 0x7f1cc80075a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:08.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:08.333+0000 7f1ce67fc640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8010ab0 legacy=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:08.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:08.333+0000 7f1ced72d640 1 -- reap_dead start 2023-11-09T04:02:08.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:08.333+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 0x7f1cc800df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:08.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:08.333+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:08.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:08.333+0000 7f1ce67fc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b32b0 con 0x7f1cc8006600 2023-11-09T04:02:08.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:08.333+0000 7f1ce67fc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc800db30 2023-11-09T04:02:08.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:08.333+0000 7f1ce67fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc8008ca0 con 0x7f1cc8005b50 2023-11-09T04:02:08.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:08.333+0000 7f1cecf2c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 msgr2=0x7f1cc800df70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:08.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:08.333+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 0x7f1cc800df70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:08.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:08.333+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8006600 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:08.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:08.333+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:08.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:08.333+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:08.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:08.534+0000 7f1cecf2c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 msgr2=0x7f1cc800df70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:08.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:08.534+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 0x7f1cc800df70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:08.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:08.534+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:08.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:08.534+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8006600 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:08.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:08.534+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:08.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:08.935+0000 7f1cecf2c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 msgr2=0x7f1cc800df70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:08.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:08.935+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 0x7f1cc800df70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:08.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:08.935+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8006600 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:08.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:08.935+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:08.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:08.935+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:09.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:09.736+0000 7f1cecf2c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 msgr2=0x7f1cc800df70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:09.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:09.736+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 0x7f1cc800df70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:09.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:09.736+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8006600 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:09.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:09.736+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:09.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:09.736+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:11.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:11.333+0000 7f1ce67fc640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8004a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:11.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:11.333+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:11.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:11.333+0000 7f1ce67fc640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 msgr2=0x7f1cc800df70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:11.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:11.333+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 0x7f1cc800df70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:11.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:11.333+0000 7f1ce67fc640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8006600 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:11.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:11.333+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800d750 0x7f1cc80133e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:11.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:11.333+0000 7f1ce67fc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80cee10 con 0x7f1cc8010ab0 2023-11-09T04:02:11.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:11.333+0000 7f1ce67fc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc800d750 2023-11-09T04:02:11.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:11.333+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8010ab0 legacy=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:11.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:11.333+0000 7f1ce7fff640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800d750 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:11.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:11.333+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800d750 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:11.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:11.534+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8010ab0 legacy=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:11.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:11.534+0000 7f1ce7fff640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800d750 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:11.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:11.534+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800d750 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:11.935 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:11.935+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8010ab0 legacy=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:11.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:11.935+0000 7f1ce7fff640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800d750 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:11.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:11.935+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800d750 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:12.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:12.736+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8010ab0 legacy=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:12.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:12.736+0000 7f1ce7fff640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800d750 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:12.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:12.736+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800d750 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:14.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:14.333+0000 7f1ce67fc640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800d750 msgr2=0x7f1cc80133e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:14.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:14.333+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc800d750 0x7f1cc80133e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:14.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:14.333+0000 7f1ce67fc640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8010ab0 legacy=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:14.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:14.333+0000 7f1ced72d640 1 -- reap_dead start 2023-11-09T04:02:14.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:14.333+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 0x7f1cc800df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:14.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:14.333+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:14.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:14.333+0000 7f1ce67fc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b32b0 con 0x7f1cc80075a0 2023-11-09T04:02:14.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:14.333+0000 7f1ce67fc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc800db30 2023-11-09T04:02:14.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:14.333+0000 7f1ce67fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc8008ca0 con 0x7f1cc8005b50 2023-11-09T04:02:14.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:14.333+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc80075a0 legacy=0x7f1cc8014a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:14.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:14.333+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 msgr2=0x7f1cc800df40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:14.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:14.333+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 0x7f1cc800df40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:14.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:14.333+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:14.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:14.333+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:14.534 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:14.534+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc80075a0 legacy=0x7f1cc8014a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:14.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:14.534+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 msgr2=0x7f1cc800df40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:14.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:14.534+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 0x7f1cc800df40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:14.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:14.534+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:14.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:14.534+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:14.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:14.935+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc80075a0 legacy=0x7f1cc8014a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:14.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:14.935+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 msgr2=0x7f1cc800df40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:14.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:14.935+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 0x7f1cc800df40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:14.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:14.935+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:14.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:14.935+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:15.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:15.736+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc80075a0 legacy=0x7f1cc8014a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:15.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:15.736+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 msgr2=0x7f1cc800df40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:15.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:15.736+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 0x7f1cc800df40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:15.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:15.736+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:15.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:15.736+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:17.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:17.334+0000 7f1ce67fc640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8004a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:17.334 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:17.334+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:17.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:17.334+0000 7f1ce67fc640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 msgr2=0x7f1cc800df40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:17.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:17.334+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 0x7f1cc800df40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:17.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:17.334+0000 7f1ce67fc640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc80075a0 legacy=0x7f1cc8014a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:17.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:17.334+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 0x7f1cc8015110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:17.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:17.334+0000 7f1ce67fc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80cee10 con 0x7f1cc8006600 2023-11-09T04:02:17.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:17.334+0000 7f1ce67fc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc80133e0 2023-11-09T04:02:17.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:17.334+0000 7f1ce7fff640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 msgr2=0x7f1cc8015110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:17.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:17.334+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8006600 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:17.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:17.334+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 0x7f1cc8015110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:17.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:17.535+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8006600 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:17.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:17.535+0000 7f1ce7fff640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 msgr2=0x7f1cc8015110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:17.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:17.535+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 0x7f1cc8015110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:17.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:17.936+0000 7f1ce7fff640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 msgr2=0x7f1cc8015110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:17.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:17.936+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8006600 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:17.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:17.936+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 0x7f1cc8015110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:18.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:18.737+0000 7f1ce7fff640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 msgr2=0x7f1cc8015110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:18.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:18.737+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8006600 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:18.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:18.737+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 0x7f1cc8015110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:20.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:20.334+0000 7f1ce67fc640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 msgr2=0x7f1cc8015110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:20.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:20.334+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 0x7f1cc8015110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:20.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:20.334+0000 7f1ce67fc640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8006600 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:20.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:20.334+0000 7f1ced72d640 1 -- reap_dead start 2023-11-09T04:02:20.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:20.334+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:20.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:20.334+0000 7f1ce67fc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b32b0 con 0x7f1cc80075a0 2023-11-09T04:02:20.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:20.334+0000 7f1ce67fc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80cee10 con 0x7f1cc800d040 2023-11-09T04:02:20.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:20.334+0000 7f1ce67fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc8005b50 2023-11-09T04:02:20.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:20.334+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc80075a0 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:20.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:20.334+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800d040 legacy=0x7f1cc800d420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:20.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:20.334+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:20.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:20.334+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:20.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:20.535+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc80075a0 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:20.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:20.535+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800d040 legacy=0x7f1cc800d420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:20.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:20.535+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:20.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:20.535+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:20.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:20.936+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc80075a0 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:20.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:20.936+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800d040 legacy=0x7f1cc800d420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:20.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:20.936+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:20.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:20.936+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:21.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:21.737+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc80075a0 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:21.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:21.737+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800d040 legacy=0x7f1cc800d420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:21.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:21.737+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8004a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:21.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:21.737+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:23.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:23.334+0000 7f1ce67fc640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8004a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:23.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:23.334+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8004a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:23.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:23.334+0000 7f1ce67fc640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800d040 legacy=0x7f1cc800d420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:23.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:23.334+0000 7f1ce67fc640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc80075a0 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:23.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:23.334+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 0x7f1cc800df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:23.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:23.334+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 0x7f1cc8014a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:23.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:23.334+0000 7f1ce67fc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc800db30 2023-11-09T04:02:23.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:23.334+0000 7f1ce67fc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cc8008ca0 con 0x7f1cc80133e0 2023-11-09T04:02:23.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:23.334+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 msgr2=0x7f1cc8014a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:23.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:23.334+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 0x7f1cc8014a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:23.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:23.334+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 msgr2=0x7f1cc800df40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:23.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:23.334+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 0x7f1cc800df40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:23.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:23.535+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 msgr2=0x7f1cc8014a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:23.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:23.535+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 0x7f1cc8014a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:23.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:23.535+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 msgr2=0x7f1cc800df40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:23.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:23.535+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 0x7f1cc800df40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:23.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:23.936+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 msgr2=0x7f1cc8014a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:23.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:23.936+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 0x7f1cc8014a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:23.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:23.936+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 msgr2=0x7f1cc800df40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:23.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:23.936+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 0x7f1cc800df40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:24.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:24.737+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 msgr2=0x7f1cc8014a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:24.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:24.737+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 0x7f1cc8014a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:24.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:24.737+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 msgr2=0x7f1cc800df40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:24.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:24.737+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 0x7f1cc800df40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:26.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:26.335+0000 7f1ce67fc640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 msgr2=0x7f1cc800df40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:26.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:26.335+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 0x7f1cc800df40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:26.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:26.335+0000 7f1ce67fc640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 msgr2=0x7f1cc8014a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:26.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:26.335+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800db30 0x7f1cc8014a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:26.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:26.335+0000 7f1ced72d640 1 -- reap_dead start 2023-11-09T04:02:26.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:26.335+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8014f40 0x7f1cc8015360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:26.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:26.335+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 0x7f1cc800df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:26.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:26.335+0000 7f1ce67fc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b2c60 con 0x7f1cc800d040 2023-11-09T04:02:26.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:26.335+0000 7f1ce67fc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc8014f40 2023-11-09T04:02:26.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:26.335+0000 7f1ce67fc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cc8008ca0 con 0x7f1cc80133e0 2023-11-09T04:02:26.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:26.335+0000 7f1cecf2c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8014f40 msgr2=0x7f1cc8015360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:26.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:26.335+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8014f40 0x7f1cc8015360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:26.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:26.335+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800d040 legacy=0x7f1cc800d420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:26.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:26.335+0000 7f1ced72d640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 msgr2=0x7f1cc800df40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:26.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:26.335+0000 7f1ced72d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 0x7f1cc800df40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:26.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:26.536+0000 7f1cecf2c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8014f40 msgr2=0x7f1cc8015360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:26.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:26.536+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8014f40 0x7f1cc8015360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:26.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:26.536+0000 7f1ced72d640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 msgr2=0x7f1cc800df40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:26.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:26.536+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800d040 legacy=0x7f1cc800d420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:26.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:26.536+0000 7f1ced72d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 0x7f1cc800df40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:26.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:26.937+0000 7f1cecf2c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8014f40 msgr2=0x7f1cc8015360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:26.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:26.937+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8014f40 0x7f1cc8015360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:26.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:26.937+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800d040 legacy=0x7f1cc800d420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:26.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:26.937+0000 7f1ced72d640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 msgr2=0x7f1cc800df40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:26.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:26.937+0000 7f1ced72d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 0x7f1cc800df40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:27.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:27.738+0000 7f1cecf2c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8014f40 msgr2=0x7f1cc8015360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:27.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:27.738+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8014f40 0x7f1cc8015360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:27.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:27.738+0000 7f1ced72d640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 msgr2=0x7f1cc800df40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:27.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:27.738+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800d040 legacy=0x7f1cc800d420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:27.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:27.738+0000 7f1ced72d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 0x7f1cc800df40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:29.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:29.335+0000 7f1ce67fc640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 msgr2=0x7f1cc800df40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:29.335 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:29.335+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc80133e0 0x7f1cc800df40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:29.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:29.335+0000 7f1ce67fc640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8014f40 msgr2=0x7f1cc8015360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:29.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:29.335+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8014f40 0x7f1cc8015360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:29.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:29.335+0000 7f1ce67fc640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800d040 legacy=0x7f1cc800d420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:29.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:29.335+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8014a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:29.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:29.335+0000 7f1ce67fc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b32b0 con 0x7f1cc8004a90 2023-11-09T04:02:29.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:29.335+0000 7f1ce67fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc8005b50 2023-11-09T04:02:29.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:29.335+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8004a90 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:29.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:29.335+0000 7f1ce7fff640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8014a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:29.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:29.335+0000 7f1ce7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8014a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:29.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:29.536+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8004a90 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:29.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:29.536+0000 7f1ce7fff640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8014a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:29.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:29.536+0000 7f1ce7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8014a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:29.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:29.937+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8004a90 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:29.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:29.937+0000 7f1ce7fff640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8014a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:29.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:29.937+0000 7f1ce7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8014a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:30.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:30.738+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8004a90 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:30.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:30.738+0000 7f1ce7fff640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8014a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:30.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:30.738+0000 7f1ce7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8014a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:32.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:32.335+0000 7f1ce67fc640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 msgr2=0x7f1cc8014a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:32.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:32.335+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8005b50 0x7f1cc8014a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:32.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:32.335+0000 7f1ce67fc640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8004a90 legacy=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:32.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:32.335+0000 7f1ced72d640 1 -- reap_dead start 2023-11-09T04:02:32.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:32.335+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8004a90 0x7f1cc800df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:32.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:32.335+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 0x7f1cc8005440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:32.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:32.335+0000 7f1ce67fc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b2c60 con 0x7f1cc800d040 2023-11-09T04:02:32.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:32.335+0000 7f1ce67fc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc8004a90 2023-11-09T04:02:32.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:32.335+0000 7f1ce67fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc8008ca0 con 0x7f1cc80075a0 2023-11-09T04:02:32.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:32.335+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8004a90 msgr2=0x7f1cc800df40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:32.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:32.335+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8004a90 0x7f1cc800df40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:32.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:32.336+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 msgr2=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:32.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:32.336+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800d040 legacy=0x7f1cc800d420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:32.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:32.336+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 0x7f1cc8005440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:32.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:32.536+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8004a90 msgr2=0x7f1cc800df40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:32.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:32.536+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8004a90 0x7f1cc800df40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:32.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:32.537+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800d040 legacy=0x7f1cc800d420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:32.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:32.537+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 msgr2=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:32.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:32.537+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 0x7f1cc8005440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:32.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:32.937+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8004a90 msgr2=0x7f1cc800df40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:32.936 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:32.937+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8004a90 0x7f1cc800df40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:32.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:32.938+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800d040 legacy=0x7f1cc800d420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:32.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:32.938+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 msgr2=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:32.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:32.938+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 0x7f1cc8005440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:33.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:33.738+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8004a90 msgr2=0x7f1cc800df40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:33.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:33.738+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8004a90 0x7f1cc800df40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:33.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:33.739+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800d040 legacy=0x7f1cc800d420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:33.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:33.739+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 msgr2=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:33.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:33.739+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 0x7f1cc8005440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:35.159 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:02:35.162 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:02:35.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:35.336+0000 7f1ce67fc640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 msgr2=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:35.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:35.336+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 0x7f1cc8005440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:35.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:35.336+0000 7f1ce67fc640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8004a90 msgr2=0x7f1cc800df40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:35.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:35.336+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8004a90 0x7f1cc800df40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:35.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:35.336+0000 7f1ce67fc640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800d040 legacy=0x7f1cc800d420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:35.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:35.336+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:35.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:35.336+0000 7f1ce67fc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b32b0 con 0x7f1cc80133e0 2023-11-09T04:02:35.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:35.336+0000 7f1ce67fc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc8005980 2023-11-09T04:02:35.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:35.336+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc80133e0 legacy=0x7f1cc8013f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:35.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:35.336+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:35.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:35.336+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:35.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:35.537+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc80133e0 legacy=0x7f1cc8013f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:35.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:35.537+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:35.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:35.537+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:35.937 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:35.938+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc80133e0 legacy=0x7f1cc8013f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:35.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:35.938+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:35.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:35.938+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:36.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:36.739+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc80133e0 legacy=0x7f1cc8013f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:36.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:36.739+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:36.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:36.739+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:38.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:38.336+0000 7f1ce67fc640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:38.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:38.336+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:38.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:38.336+0000 7f1ce67fc640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc80133e0 legacy=0x7f1cc8013f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:38.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:38.336+0000 7f1ced72d640 1 -- reap_dead start 2023-11-09T04:02:38.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:38.336+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:38.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:38.336+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 0x7f1cc8005440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:38.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:38.336+0000 7f1ce67fc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b2c60 con 0x7f1cc8017ac0 2023-11-09T04:02:38.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:38.336+0000 7f1ce67fc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc8005980 2023-11-09T04:02:38.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:38.336+0000 7f1ce67fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc8008ca0 con 0x7f1cc80075a0 2023-11-09T04:02:38.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:38.336+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8017ac0 legacy=0x7f1cc8019eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:38.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:38.336+0000 7f1ce7fff640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:38.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:38.336+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:38.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:38.336+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 msgr2=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:38.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:38.336+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 0x7f1cc8005440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:38.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:38.537+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8017ac0 legacy=0x7f1cc8019eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:38.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:38.537+0000 7f1ce7fff640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:38.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:38.537+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 msgr2=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:38.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:38.537+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:38.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:38.537+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 0x7f1cc8005440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:38.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:38.938+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8017ac0 legacy=0x7f1cc8019eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:38.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:38.938+0000 7f1ce7fff640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:38.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:38.938+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 msgr2=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:38.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:38.938+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:38.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:38.938+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 0x7f1cc8005440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:39.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:39.739+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8017ac0 legacy=0x7f1cc8019eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:39.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:39.739+0000 7f1ce7fff640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:39.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:39.739+0000 7f1ced72d640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 msgr2=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:39.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:39.739+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:39.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:39.739+0000 7f1ced72d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 0x7f1cc8005440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:41.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:41.337+0000 7f1ce67fc640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 msgr2=0x7f1cc8005440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:41.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:41.337+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 0x7f1cc8005440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:41.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:41.337+0000 7f1ce67fc640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:41.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:41.337+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:41.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:41.337+0000 7f1ce67fc640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8017ac0 legacy=0x7f1cc8019eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:41.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:41.337+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:41.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:41.337+0000 7f1ce67fc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b32b0 con 0x7f1cc800d580 2023-11-09T04:02:41.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:41.337+0000 7f1ce67fc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc800df40 2023-11-09T04:02:41.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:41.337+0000 7f1cecf2c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:41.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:41.337+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:41.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:41.337+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800d580 legacy=0x7f1cc8003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:41.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:41.538+0000 7f1cecf2c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:41.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:41.538+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:41.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:41.538+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800d580 legacy=0x7f1cc8003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:41.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:41.939+0000 7f1cecf2c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:41.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:41.939+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:41.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:41.939+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800d580 legacy=0x7f1cc8003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:42.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:42.739+0000 7f1cecf2c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:42.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:42.739+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:42.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:42.739+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800d580 legacy=0x7f1cc8003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:44.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:44.336+0000 7f1ce67fc640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:44.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:44.336+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:44.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:44.336+0000 7f1ce67fc640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc800d580 legacy=0x7f1cc8003a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:44.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:44.336+0000 7f1ced72d640 1 -- reap_dead start 2023-11-09T04:02:44.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:44.336+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:44.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:44.336+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:44.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:44.336+0000 7f1ce67fc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b2c60 con 0x7f1cc8017ac0 2023-11-09T04:02:44.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:44.336+0000 7f1ce67fc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc800df40 2023-11-09T04:02:44.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:44.336+0000 7f1ce67fc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cc8008ca0 con 0x7f1cc8005980 2023-11-09T04:02:44.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:44.336+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8017ac0 legacy=0x7f1cc8014f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:44.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:44.336+0000 7f1ced72d640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:44.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:44.336+0000 7f1ced72d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:44.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:44.336+0000 7f1ce7fff640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:44.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:44.336+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:44.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:44.537+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8017ac0 legacy=0x7f1cc8014f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:44.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:44.537+0000 7f1ced72d640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:44.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:44.537+0000 7f1ced72d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:44.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:44.537+0000 7f1ce7fff640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:44.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:44.537+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:44.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:44.938+0000 7f1ced72d640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:44.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:44.938+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8017ac0 legacy=0x7f1cc8014f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:44.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:44.938+0000 7f1ced72d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:44.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:44.938+0000 7f1ce7fff640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:44.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:44.938+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:45.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:45.739+0000 7f1ced72d640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:45.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:45.739+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8017ac0 legacy=0x7f1cc8014f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:45.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:45.739+0000 7f1ced72d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:45.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:45.739+0000 7f1ce7fff640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:45.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:45.739+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:47.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:47.336+0000 7f1ce67fc640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:47.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:47.336+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:47.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:47.336+0000 7f1ce67fc640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:47.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:47.336+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:47.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:47.336+0000 7f1ce67fc640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8017ac0 legacy=0x7f1cc8014f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:47.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:47.336+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8004a90 0x7f1cc800d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:47.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:47.336+0000 7f1ce67fc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b32b0 con 0x7f1cc8005440 2023-11-09T04:02:47.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:47.336+0000 7f1ce67fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc8004a90 2023-11-09T04:02:47.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:47.336+0000 7f1ce7fff640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8004a90 msgr2=0x7f1cc800d580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:47.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:47.336+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8005440 legacy=0x7f1cc8013f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:47.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:47.336+0000 7f1ce7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8004a90 0x7f1cc800d580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:47.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:47.537+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8005440 legacy=0x7f1cc8013f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:47.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:47.537+0000 7f1ce7fff640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8004a90 msgr2=0x7f1cc800d580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:47.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:47.537+0000 7f1ce7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8004a90 0x7f1cc800d580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:47.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:47.938+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8005440 legacy=0x7f1cc8013f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:47.938 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:47.938+0000 7f1ce7fff640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8004a90 msgr2=0x7f1cc800d580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:47.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:47.938+0000 7f1ce7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8004a90 0x7f1cc800d580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:48.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:48.739+0000 7f1ce7fff640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8004a90 msgr2=0x7f1cc800d580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:48.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:48.739+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8005440 legacy=0x7f1cc8013f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:48.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:48.739+0000 7f1ce7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8004a90 0x7f1cc800d580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:50.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:50.337+0000 7f1ce67fc640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8004a90 msgr2=0x7f1cc800d580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:50.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:50.337+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8004a90 0x7f1cc800d580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:50.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:50.337+0000 7f1ce67fc640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8005440 legacy=0x7f1cc8013f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:50.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:50.337+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:50.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:50.337+0000 7f1ced72d640 1 -- reap_dead start 2023-11-09T04:02:50.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:50.337+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:50.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:50.337+0000 7f1ce67fc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b2c60 con 0x7f1cc80075a0 2023-11-09T04:02:50.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:50.337+0000 7f1ce67fc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc800df40 2023-11-09T04:02:50.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:50.337+0000 7f1ce67fc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cc8008ca0 con 0x7f1cc8005980 2023-11-09T04:02:50.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:50.337+0000 7f1ced72d640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:50.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:50.337+0000 7f1ced72d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:50.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:50.337+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:50.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:50.337+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc80075a0 legacy=0x7f1cc8014610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:50.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:50.337+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:50.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:50.538+0000 7f1ced72d640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:50.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:50.538+0000 7f1ced72d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:50.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:50.538+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:50.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:50.538+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc80075a0 legacy=0x7f1cc8014610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:50.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:50.538+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:50.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:50.939+0000 7f1ced72d640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:50.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:50.939+0000 7f1ced72d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:50.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:50.939+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc80075a0 legacy=0x7f1cc8014610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:50.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:50.939+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:50.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:50.939+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:51.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:51.740+0000 7f1ced72d640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:51.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:51.740+0000 7f1ced72d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:51.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:51.740+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc80075a0 legacy=0x7f1cc8014610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:51.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:51.740+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:51.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:51.740+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:53.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:53.337+0000 7f1ce67fc640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:53.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:53.337+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:53.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:53.337+0000 7f1ce67fc640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:53.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:53.337+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:53.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:53.337+0000 7f1ce67fc640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc80075a0 legacy=0x7f1cc8014610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:53.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:53.337+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc800d580 0x7f1cc8013f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:53.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:53.337+0000 7f1ce67fc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b32b0 con 0x7f1cc8014d20 2023-11-09T04:02:53.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:53.337+0000 7f1ce67fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc800d580 2023-11-09T04:02:53.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:53.337+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8014d20 legacy=0x7f1cc8015100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:53.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:53.337+0000 7f1cecf2c640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc800d580 msgr2=0x7f1cc8013f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:53.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:53.337+0000 7f1cecf2c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc800d580 0x7f1cc8013f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:53.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:53.538+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8014d20 legacy=0x7f1cc8015100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:53.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:53.538+0000 7f1cecf2c640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc800d580 msgr2=0x7f1cc8013f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:53.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:53.538+0000 7f1cecf2c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc800d580 0x7f1cc8013f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:53.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:53.939+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8014d20 legacy=0x7f1cc8015100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:53.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:53.939+0000 7f1cecf2c640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc800d580 msgr2=0x7f1cc8013f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:53.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:53.939+0000 7f1cecf2c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc800d580 0x7f1cc8013f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:54.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:54.740+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8014d20 legacy=0x7f1cc8015100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:54.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:54.740+0000 7f1cecf2c640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc800d580 msgr2=0x7f1cc8013f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:54.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:54.740+0000 7f1cecf2c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc800d580 0x7f1cc8013f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:56.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:56.337+0000 7f1ce67fc640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc800d580 msgr2=0x7f1cc8013f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:56.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:56.337+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc800d580 0x7f1cc8013f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:56.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:56.337+0000 7f1ce67fc640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8014d20 legacy=0x7f1cc8015100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:56.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:56.337+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:56.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:56.337+0000 7f1ced72d640 1 -- reap_dead start 2023-11-09T04:02:56.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:56.337+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:56.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:56.337+0000 7f1ce67fc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b2c60 con 0x7f1cc80075a0 2023-11-09T04:02:56.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:56.337+0000 7f1ce67fc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc800df40 2023-11-09T04:02:56.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:56.337+0000 7f1ce67fc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cc8008ca0 con 0x7f1cc8005980 2023-11-09T04:02:56.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:56.337+0000 7f1ced72d640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:56.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:56.337+0000 7f1ced72d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:56.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:56.337+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:56.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:56.337+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc80075a0 legacy=0x7f1cc8014440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:56.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:56.337+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:56.538 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:56.538+0000 7f1ced72d640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:56.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:56.538+0000 7f1ced72d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:56.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:56.538+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:56.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:56.538+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc80075a0 legacy=0x7f1cc8014440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:56.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:56.538+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:56.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:56.939+0000 7f1ced72d640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:56.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:56.939+0000 7f1ced72d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:56.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:56.939+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:56.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:56.939+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc80075a0 legacy=0x7f1cc8014440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:56.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:56.939+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:02:57.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:57.740+0000 7f1ced72d640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:02:57.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:57.740+0000 7f1ced72d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:57.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:57.740+0000 7f1cecf2c640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:02:57.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:57.740+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc80075a0 legacy=0x7f1cc8014440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:02:57.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:57.740+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:02:59.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:59.338+0000 7f1ce67fc640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:59.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:59.338+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:59.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:59.338+0000 7f1ce67fc640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:59.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:59.338+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:02:59.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:59.338+0000 7f1ce67fc640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc80075a0 legacy=0x7f1cc8014440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:02:59.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:59.338+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8004a90 0x7f1cc8013f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:02:59.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:59.338+0000 7f1ce67fc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b32b0 con 0x7f1cc800d580 2023-11-09T04:02:59.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:59.338+0000 7f1ce67fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc8004a90 2023-11-09T04:02:59.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:59.338+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc800d580 legacy=0x7f1cc8014b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:59.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:59.338+0000 7f1ce7fff640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8004a90 msgr2=0x7f1cc8013f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:59.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:59.338+0000 7f1ce7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8004a90 0x7f1cc8013f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:02:59.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:59.539+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc800d580 legacy=0x7f1cc8014b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:59.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:59.539+0000 7f1ce7fff640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8004a90 msgr2=0x7f1cc8013f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:59.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:59.539+0000 7f1ce7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8004a90 0x7f1cc8013f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:02:59.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:59.940+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc800d580 legacy=0x7f1cc8014b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:02:59.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:59.940+0000 7f1ce7fff640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8004a90 msgr2=0x7f1cc8013f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:02:59.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:02:59.940+0000 7f1ce7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8004a90 0x7f1cc8013f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:00.740 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:00.741+0000 7f1cecf2c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc800d580 legacy=0x7f1cc8014b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:00.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:00.741+0000 7f1ce7fff640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8004a90 msgr2=0x7f1cc8013f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:00.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:00.741+0000 7f1ce7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8004a90 0x7f1cc8013f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:02.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:02.338+0000 7f1ce67fc640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8004a90 msgr2=0x7f1cc8013f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:02.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:02.338+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc8004a90 0x7f1cc8013f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:03:02.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:02.338+0000 7f1ce67fc640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc800d580 legacy=0x7f1cc8014b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:02.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:02.338+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:02.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:02.338+0000 7f1ced72d640 1 -- reap_dead start 2023-11-09T04:03:02.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:02.338+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:02.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:02.338+0000 7f1ce67fc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b2c60 con 0x7f1cc8015260 2023-11-09T04:03:02.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:02.338+0000 7f1ce67fc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc800df40 2023-11-09T04:03:02.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:02.338+0000 7f1ce67fc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cc8008ca0 con 0x7f1cc8005980 2023-11-09T04:03:02.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:02.338+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8015260 legacy=0x7f1cc800bfa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:02.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:02.338+0000 7f1cecf2c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:02.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:02.338+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:02.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:02.338+0000 7f1ced72d640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:02.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:02.338+0000 7f1ced72d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:02.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:02.539+0000 7f1cecf2c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:02.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:02.539+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8015260 legacy=0x7f1cc800bfa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:02.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:02.539+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:02.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:02.539+0000 7f1ced72d640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:02.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:02.539+0000 7f1ced72d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:02.939 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:02.940+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8015260 legacy=0x7f1cc800bfa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:02.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:02.940+0000 7f1cecf2c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:02.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:02.940+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:02.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:02.940+0000 7f1ced72d640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:02.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:02.940+0000 7f1ced72d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:03.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:03.741+0000 7f1cecf2c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:03.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:03.741+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8015260 legacy=0x7f1cc800bfa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:03.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:03.741+0000 7f1cecf2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:03.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:03.741+0000 7f1ced72d640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:03.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:03.741+0000 7f1ced72d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:05.285 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:03:05.288 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:03:05.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:05.338+0000 7f1ce67fc640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:05.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:05.338+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:03:05.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:05.338+0000 7f1ce67fc640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:05.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:05.338+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:03:05.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:05.338+0000 7f1ce67fc640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8015260 legacy=0x7f1cc800bfa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:05.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:05.338+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 0x7f1cc800c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:05.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:05.338+0000 7f1ce67fc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b32b0 con 0x7f1cc8013f00 2023-11-09T04:03:05.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:05.338+0000 7f1ce67fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc80075a0 2023-11-09T04:03:05.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:05.339+0000 7f1cecf2c640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 msgr2=0x7f1cc800c6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:05.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:05.339+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8013f00 legacy=0x7f1cc80142e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:05.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:05.339+0000 7f1cecf2c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 0x7f1cc800c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:05.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:05.540+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8013f00 legacy=0x7f1cc80142e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:05.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:05.540+0000 7f1cecf2c640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 msgr2=0x7f1cc800c6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:05.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:05.540+0000 7f1cecf2c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 0x7f1cc800c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:05.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:05.941+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8013f00 legacy=0x7f1cc80142e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:05.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:05.941+0000 7f1cecf2c640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 msgr2=0x7f1cc800c6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:05.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:05.941+0000 7f1cecf2c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 0x7f1cc800c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:06.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:06.742+0000 7f1ce7fff640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8013f00 legacy=0x7f1cc80142e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:06.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:06.742+0000 7f1cecf2c640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 msgr2=0x7f1cc800c6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:06.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:06.742+0000 7f1cecf2c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 0x7f1cc800c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:08.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:08.339+0000 7f1ce67fc640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 msgr2=0x7f1cc800c6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:08.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:08.339+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc80075a0 0x7f1cc800c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:03:08.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:08.339+0000 7f1ce67fc640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8013f00 legacy=0x7f1cc80142e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:08.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:08.339+0000 7f1ced72d640 1 -- reap_dead start 2023-11-09T04:03:08.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:08.339+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:08.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:08.339+0000 7f1ce67fc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b2c60 con 0x7f1cc8015260 2023-11-09T04:03:08.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:08.339+0000 7f1ce67fc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b32b0 con 0x7f1cc8004a90 2023-11-09T04:03:08.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:08.339+0000 7f1ce67fc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc8005980 2023-11-09T04:03:08.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:08.339+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8015260 legacy=0x7f1cc80149f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:08.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:08.339+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8004a90 legacy=0x7f1cc80142e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:08.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:08.339+0000 7f1ced72d640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:08.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:08.339+0000 7f1ced72d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:08.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:08.540+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8015260 legacy=0x7f1cc80149f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:08.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:08.540+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8004a90 legacy=0x7f1cc80142e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:08.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:08.540+0000 7f1ced72d640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:08.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:08.540+0000 7f1ced72d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:08.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:08.941+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8015260 legacy=0x7f1cc80149f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:08.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:08.941+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8004a90 legacy=0x7f1cc80142e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:08.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:08.941+0000 7f1ced72d640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:08.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:08.941+0000 7f1ced72d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:09.741 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:09.742+0000 7f1ce7fff640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8015260 legacy=0x7f1cc80149f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:09.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:09.742+0000 7f1cecf2c640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8004a90 legacy=0x7f1cc80142e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:09.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:09.742+0000 7f1ced72d640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:09.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:09.742+0000 7f1ced72d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:11.339 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:11.339+0000 7f1ce67fc640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 msgr2=0x7f1cc80139c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:11.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:11.339+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cc8005980 0x7f1cc80139c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:03:11.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:11.339+0000 7f1ce67fc640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1cc8004a90 legacy=0x7f1cc80142e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:11.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:11.339+0000 7f1ce67fc640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f1cc8015260 legacy=0x7f1cc80149f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:11.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:11.339+0000 7f1ce67fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:11.340 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:11.339+0000 7f1ce67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8015640 0x7f1cc80075a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:11.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:11.339+0000 7f1ce67fc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b2980 con 0x7f1cc8015640 2023-11-09T04:03:11.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:11.339+0000 7f1ce67fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc8008ca0 con 0x7f1cc800df40 2023-11-09T04:03:11.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:11.339+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8015640 msgr2=0x7f1cc80075a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:11.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:11.339+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8015640 0x7f1cc80075a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:11.341 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:11.339+0000 7f1cecf2c640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:11.342 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:11.339+0000 7f1cecf2c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:11.539 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:11.540+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8015640 msgr2=0x7f1cc80075a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:11.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:11.540+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8015640 0x7f1cc80075a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:11.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:11.540+0000 7f1cecf2c640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:11.540 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:11.540+0000 7f1cecf2c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:11.940 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:11.941+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8015640 msgr2=0x7f1cc80075a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:11.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:11.941+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8015640 0x7f1cc80075a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:11.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:11.941+0000 7f1cecf2c640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:11.941 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:11.941+0000 7f1cecf2c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:12.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:12.742+0000 7f1ce7fff640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8015640 msgr2=0x7f1cc80075a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:12.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:12.742+0000 7f1ce7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cc8015640 0x7f1cc80075a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:12.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:12.742+0000 7f1cecf2c640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc800df40 msgr2=0x7f1cc800d040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:12.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:12.742+0000 7f1cecf2c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1cc800df40 0x7f1cc800d040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:14.250 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-11-09T04:03:14.252 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-09T04:03:14.253 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-09T04:03:14.302 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-09T04:03:14.362 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-09T04:03:14.363 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T04:03:14.363 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T04:03:14.383 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056381,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14761 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248868k,nr_inodes=812217,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-09T04:03:14.383 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T04:03:14.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T04:03:14.384 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-09T04:03:14.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T04:03:14.442 INFO:teuthology.orchestra.run.smithi076.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-09T04:03:14.443 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T04:03:14.443 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T04:03:14.463 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2440 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-09T04:03:14.463 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2023-11-09T04:03:14.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T04:03:14.464 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-09T04:03:14.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T04:03:14.521 INFO:teuthology.orchestra.run.smithi113.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-09T04:03:14.521 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T04:03:14.522 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T04:03:14.541 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056381,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14761 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248868k,nr_inodes=812217,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-09T04:03:14.541 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T04:03:14.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T04:03:14.542 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-09T04:03:14.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T04:03:14.602 INFO:teuthology.orchestra.run.smithi076.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-09T04:03:14.602 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T04:03:14.603 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T04:03:14.621 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2440 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-09T04:03:14.622 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2023-11-09T04:03:14.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T04:03:14.623 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-09T04:03:14.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T04:03:14.679 INFO:teuthology.orchestra.run.smithi113.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-09T04:03:14.680 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-09T04:03:14.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:14.789+0000 7fc9739c9640 1 Processor -- start 2023-11-09T04:03:14.790 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:14.790+0000 7fc9739c9640 1 -- start start 2023-11-09T04:03:14.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:14.791+0000 7fc9739c9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 0x7fc96c0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:14.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:14.791+0000 7fc9739c9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c071500 con 0x7fc96c06bb60 2023-11-09T04:03:14.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:14.791+0000 7fc9739c9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c15dae0 con 0x7fc96c070700 2023-11-09T04:03:14.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:14.791+0000 7fc9739c9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc96c159e40 2023-11-09T04:03:14.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:14.791+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc96c06bb60 legacy=0x7fc96c06fff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:14.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:14.791+0000 7fc971f66640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc96c070700 legacy=0x7fc96c0cd790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:14.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:14.791+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 msgr2=0x7fc96c0b0530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:14.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:14.791+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 0x7fc96c0b0530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:14.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:14.992+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc96c06bb60 legacy=0x7fc96c06fff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:14.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:14.992+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 msgr2=0x7fc96c0b0530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:14.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:14.992+0000 7fc971f66640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc96c070700 legacy=0x7fc96c0cd790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:14.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:14.992+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 0x7fc96c0b0530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:15.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:15.393+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc96c06bb60 legacy=0x7fc96c06fff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:15.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:15.393+0000 7fc971f66640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc96c070700 legacy=0x7fc96c0cd790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:15.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:15.393+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 msgr2=0x7fc96c0b0530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:15.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:15.393+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 0x7fc96c0b0530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:16.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:16.194+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc96c06bb60 legacy=0x7fc96c06fff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:16.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:16.194+0000 7fc971f66640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc96c070700 legacy=0x7fc96c0cd790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:16.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:16.194+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 msgr2=0x7fc96c0b0530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:16.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:16.194+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 0x7fc96c0b0530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:17.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:17.791+0000 7fc95effd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 msgr2=0x7fc96c0b0530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:17.792 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:17.791+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 0x7fc96c0b0530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:03:17.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:17.791+0000 7fc95effd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc96c070700 legacy=0x7fc96c0cd790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:17.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:17.791+0000 7fc95effd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc96c06bb60 legacy=0x7fc96c06fff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:17.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:17.791+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500058b0 0x7fc950007d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:17.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:17.791+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500083e0 0x7fc95000a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:17.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:17.791+0000 7fc95effd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc9500083e0 2023-11-09T04:03:17.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:17.791+0000 7fc95effd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95000af60 con 0x7fc9500058b0 2023-11-09T04:03:17.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:17.791+0000 7fc970f64640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc9500083e0 msgr2=0x7fc95000a8a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:17.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:17.791+0000 7fc970f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500083e0 0x7fc95000a8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:17.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:17.791+0000 7fc971765640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500058b0 msgr2=0x7fc950007d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:17.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:17.791+0000 7fc971765640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500058b0 0x7fc950007d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:17.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:17.992+0000 7fc970f64640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc9500083e0 msgr2=0x7fc95000a8a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:17.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:17.992+0000 7fc970f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500083e0 0x7fc95000a8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:17.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:17.992+0000 7fc971765640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500058b0 msgr2=0x7fc950007d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:17.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:17.992+0000 7fc971765640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500058b0 0x7fc950007d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:18.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:18.393+0000 7fc970f64640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc9500083e0 msgr2=0x7fc95000a8a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:18.393 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:18.393+0000 7fc970f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500083e0 0x7fc95000a8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:18.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:18.393+0000 7fc971765640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500058b0 msgr2=0x7fc950007d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:18.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:18.393+0000 7fc971765640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500058b0 0x7fc950007d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:19.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:19.194+0000 7fc970f64640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc9500083e0 msgr2=0x7fc95000a8a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:19.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:19.194+0000 7fc970f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500083e0 0x7fc95000a8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:19.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:19.194+0000 7fc971765640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500058b0 msgr2=0x7fc950007d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:19.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:19.194+0000 7fc971765640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500058b0 0x7fc950007d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:20.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:20.792+0000 7fc95effd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500058b0 msgr2=0x7fc950007d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:20.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:20.792+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500058b0 0x7fc950007d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:03:20.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:20.792+0000 7fc95effd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc9500083e0 msgr2=0x7fc95000a8a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:20.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:20.792+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500083e0 0x7fc95000a8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:03:20.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:20.792+0000 7fc971f66640 1 -- reap_dead start 2023-11-09T04:03:20.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:20.792+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 0x7fc95000d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:20.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:20.792+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950005cf0 0x7fc950008890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:20.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:20.792+0000 7fc95effd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c070f40 con 0x7fc96c070700 2023-11-09T04:03:20.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:20.792+0000 7fc95effd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc950005cf0 2023-11-09T04:03:20.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:20.792+0000 7fc95effd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc95000af60 con 0x7fc96c159e40 2023-11-09T04:03:20.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:20.792+0000 7fc971f66640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950005cf0 msgr2=0x7fc950008890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:20.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:20.792+0000 7fc971f66640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950005cf0 0x7fc950008890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:20.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:20.792+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 msgr2=0x7fc95000d250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:20.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:20.792+0000 7fc970f64640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc96c070700 legacy=0x7fc95000f7a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:20.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:20.792+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 0x7fc95000d250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:20.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:20.993+0000 7fc971f66640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950005cf0 msgr2=0x7fc950008890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:20.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:20.993+0000 7fc971f66640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950005cf0 0x7fc950008890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:20.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:20.993+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 msgr2=0x7fc95000d250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:20.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:20.993+0000 7fc970f64640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc96c070700 legacy=0x7fc95000f7a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:20.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:20.993+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 0x7fc95000d250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:21.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:21.394+0000 7fc971f66640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950005cf0 msgr2=0x7fc950008890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:21.394 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:21.394+0000 7fc971f66640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950005cf0 0x7fc950008890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:21.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:21.394+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 msgr2=0x7fc95000d250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:21.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:21.394+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 0x7fc95000d250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:21.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:21.394+0000 7fc970f64640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc96c070700 legacy=0x7fc95000f7a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:22.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:22.195+0000 7fc971f66640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950005cf0 msgr2=0x7fc950008890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:22.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:22.195+0000 7fc971f66640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950005cf0 0x7fc950008890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:22.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:22.195+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 msgr2=0x7fc95000d250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:22.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:22.195+0000 7fc970f64640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc96c070700 legacy=0x7fc95000f7a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:22.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:22.195+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 0x7fc95000d250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:23.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:23.792+0000 7fc95effd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 msgr2=0x7fc95000d250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:23.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:23.792+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 0x7fc95000d250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:03:23.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:23.792+0000 7fc95effd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950005cf0 msgr2=0x7fc950008890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:23.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:23.792+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950005cf0 0x7fc950008890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:03:23.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:23.792+0000 7fc95effd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc96c070700 legacy=0x7fc95000f7a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:23.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:23.792+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950009f30 0x7fc95000a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:23.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:23.792+0000 7fc95effd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c071500 con 0x7fc95000a980 2023-11-09T04:03:23.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:23.792+0000 7fc95effd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc950009f30 2023-11-09T04:03:23.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:23.792+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc95000a980 legacy=0x7fc950008dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:23.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:23.792+0000 7fc971765640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950009f30 msgr2=0x7fc95000a310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:23.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:23.792+0000 7fc971765640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950009f30 0x7fc95000a310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:23.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:23.993+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc95000a980 legacy=0x7fc950008dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:23.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:23.993+0000 7fc971765640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950009f30 msgr2=0x7fc95000a310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:23.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:23.993+0000 7fc971765640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950009f30 0x7fc95000a310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:24.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:24.394+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc95000a980 legacy=0x7fc950008dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:24.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:24.394+0000 7fc971765640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950009f30 msgr2=0x7fc95000a310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:24.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:24.394+0000 7fc971765640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950009f30 0x7fc95000a310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:25.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:25.195+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc95000a980 legacy=0x7fc950008dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:25.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:25.195+0000 7fc971765640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950009f30 msgr2=0x7fc95000a310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:25.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:25.195+0000 7fc971765640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950009f30 0x7fc95000a310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:26.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:26.793+0000 7fc95effd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950009f30 msgr2=0x7fc95000a310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:26.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:26.793+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950009f30 0x7fc95000a310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:03:26.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:26.793+0000 7fc95effd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc95000a980 legacy=0x7fc950008dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:26.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:26.793+0000 7fc971f66640 1 -- reap_dead start 2023-11-09T04:03:26.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:26.793+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950005cf0 0x7fc950008890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:26.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:26.793+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 0x7fc95000d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:26.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:26.793+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950009f30 0x7fc95000a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:26.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:26.793+0000 7fc95effd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc950005cf0 2023-11-09T04:03:26.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:26.793+0000 7fc95effd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc95000af60 con 0x7fc96c159e40 2023-11-09T04:03:26.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:26.793+0000 7fc95effd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc950008dd0 con 0x7fc950009f30 2023-11-09T04:03:26.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:26.793+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950005cf0 msgr2=0x7fc950008890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:26.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:26.793+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950005cf0 0x7fc950008890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:26.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:26.793+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 msgr2=0x7fc95000d230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:26.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:26.793+0000 7fc971f66640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950009f30 msgr2=0x7fc95000a310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:26.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:26.793+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 0x7fc95000d230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:26.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:26.793+0000 7fc971f66640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950009f30 0x7fc95000a310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:26.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:26.994+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950005cf0 msgr2=0x7fc950008890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:26.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:26.994+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950005cf0 0x7fc950008890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:26.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:26.994+0000 7fc971f66640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950009f30 msgr2=0x7fc95000a310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:26.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:26.994+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 msgr2=0x7fc95000d230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:26.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:26.994+0000 7fc971f66640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950009f30 0x7fc95000a310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:26.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:26.994+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 0x7fc95000d230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:27.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:27.395+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950005cf0 msgr2=0x7fc950008890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:27.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:27.395+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950005cf0 0x7fc950008890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:27.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:27.395+0000 7fc971f66640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950009f30 msgr2=0x7fc95000a310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:27.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:27.395+0000 7fc971f66640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950009f30 0x7fc95000a310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:27.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:27.395+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 msgr2=0x7fc95000d230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:27.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:27.395+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 0x7fc95000d230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:28.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:28.196+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950005cf0 msgr2=0x7fc950008890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:28.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:28.196+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950005cf0 0x7fc950008890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:28.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:28.196+0000 7fc971f66640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950009f30 msgr2=0x7fc95000a310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:28.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:28.196+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 msgr2=0x7fc95000d230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:28.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:28.196+0000 7fc971f66640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950009f30 0x7fc95000a310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:28.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:28.196+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 0x7fc95000d230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:29.793 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:29.793+0000 7fc95effd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950009f30 msgr2=0x7fc95000a310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:29.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:29.793+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950009f30 0x7fc95000a310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:03:29.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:29.793+0000 7fc95effd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 msgr2=0x7fc95000d230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:29.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:29.793+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc96c159e40 0x7fc95000d230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:03:29.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:29.793+0000 7fc95effd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950005cf0 msgr2=0x7fc950008890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:29.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:29.793+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950005cf0 0x7fc950008890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:03:29.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:29.793+0000 7fc95effd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c070f40 con 0x7fc9500107b0 2023-11-09T04:03:29.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:29.793+0000 7fc95effd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c071500 con 0x7fc950008fe0 2023-11-09T04:03:29.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:29.793+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc9500107b0 legacy=0x7fc950014ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:29.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:29.793+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950008fe0 legacy=0x7fc9500096a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:29.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:29.994+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc9500107b0 legacy=0x7fc950014ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:29.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:29.994+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950008fe0 legacy=0x7fc9500096a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:30.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:30.395+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc9500107b0 legacy=0x7fc950014ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:30.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:30.395+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950008fe0 legacy=0x7fc9500096a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:31.195 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:31.196+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc9500107b0 legacy=0x7fc950014ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:31.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:31.196+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950008fe0 legacy=0x7fc9500096a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:32.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:32.793+0000 7fc95effd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950008fe0 legacy=0x7fc9500096a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:32.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:32.793+0000 7fc95effd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc9500107b0 legacy=0x7fc950014ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:32.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:32.793+0000 7fc971f66640 1 -- reap_dead start 2023-11-09T04:03:32.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:32.793+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 0x7fc950008890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:32.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:32.793+0000 7fc95effd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c15dae0 con 0x7fc9500096a0 2023-11-09T04:03:32.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:32.793+0000 7fc95effd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c070f40 con 0x7fc950016600 2023-11-09T04:03:32.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:32.793+0000 7fc95effd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc950005cf0 2023-11-09T04:03:32.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:32.793+0000 7fc971f66640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc9500096a0 legacy=0x7fc950009f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:32.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:32.793+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 msgr2=0x7fc950008890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:32.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:32.793+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950016600 legacy=0x7fc9500189f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:32.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:32.793+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 0x7fc950008890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:32.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:32.994+0000 7fc971f66640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc9500096a0 legacy=0x7fc950009f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:32.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:32.994+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950016600 legacy=0x7fc9500189f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:32.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:32.994+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 msgr2=0x7fc950008890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:32.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:32.994+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 0x7fc950008890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:33.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:33.395+0000 7fc971f66640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc9500096a0 legacy=0x7fc950009f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:33.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:33.395+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950016600 legacy=0x7fc9500189f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:33.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:33.395+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 msgr2=0x7fc950008890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:33.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:33.395+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 0x7fc950008890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:34.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:34.196+0000 7fc971f66640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc9500096a0 legacy=0x7fc950009f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:34.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:34.196+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 msgr2=0x7fc950008890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:34.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:34.196+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950016600 legacy=0x7fc9500189f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:34.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:34.196+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 0x7fc950008890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:35.375 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:03:35.379 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:03:35.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:35.794+0000 7fc95effd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 msgr2=0x7fc950008890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:35.794 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:35.794+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 0x7fc950008890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:03:35.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:35.794+0000 7fc95effd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950016600 legacy=0x7fc9500189f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:35.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:35.794+0000 7fc95effd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc9500096a0 legacy=0x7fc950009f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:35.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:35.794+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc95000d230 0x7fc950010df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:35.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:35.794+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950011330 0x7fc950013750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:35.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:35.794+0000 7fc95effd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc950011330 2023-11-09T04:03:35.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:35.794+0000 7fc95effd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc95000af60 con 0x7fc95000d230 2023-11-09T04:03:35.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:35.794+0000 7fc970f64640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950011330 msgr2=0x7fc950013750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:35.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:35.794+0000 7fc970f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950011330 0x7fc950013750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:35.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:35.794+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc95000d230 msgr2=0x7fc950010df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:35.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:35.794+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc95000d230 0x7fc950010df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:35.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:35.995+0000 7fc970f64640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950011330 msgr2=0x7fc950013750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:35.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:35.995+0000 7fc970f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950011330 0x7fc950013750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:35.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:35.995+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc95000d230 msgr2=0x7fc950010df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:35.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:35.995+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc95000d230 0x7fc950010df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:36.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:36.396+0000 7fc970f64640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950011330 msgr2=0x7fc950013750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:36.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:36.396+0000 7fc970f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950011330 0x7fc950013750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:36.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:36.396+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc95000d230 msgr2=0x7fc950010df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:36.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:36.396+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc95000d230 0x7fc950010df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:37.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:37.197+0000 7fc970f64640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950011330 msgr2=0x7fc950013750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:37.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:37.197+0000 7fc970f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950011330 0x7fc950013750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:37.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:37.197+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc95000d230 msgr2=0x7fc950010df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:37.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:37.197+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc95000d230 0x7fc950010df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:38.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:38.794+0000 7fc95effd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc95000d230 msgr2=0x7fc950010df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:38.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:38.794+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc95000d230 0x7fc950010df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:03:38.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:38.794+0000 7fc95effd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950011330 msgr2=0x7fc950013750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:38.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:38.794+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950011330 0x7fc950013750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:03:38.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:38.794+0000 7fc971f66640 1 -- reap_dead start 2023-11-09T04:03:38.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:38.794+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 0x7fc950008850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:38.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:38.794+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc95000d230 0x7fc950010ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:38.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:38.794+0000 7fc95effd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c071500 con 0x7fc9500096a0 2023-11-09T04:03:38.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:38.794+0000 7fc95effd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc95000d230 2023-11-09T04:03:38.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:38.794+0000 7fc95effd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95000af60 con 0x7fc950005cf0 2023-11-09T04:03:38.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:38.794+0000 7fc971f66640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc95000d230 msgr2=0x7fc950010ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:38.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:38.794+0000 7fc971f66640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc95000d230 0x7fc950010ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:38.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:38.794+0000 7fc971765640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 msgr2=0x7fc950008850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:38.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:38.794+0000 7fc970f64640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500096a0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:38.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:38.794+0000 7fc971765640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 0x7fc950008850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:38.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:38.995+0000 7fc971f66640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc95000d230 msgr2=0x7fc950010ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:38.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:38.995+0000 7fc971f66640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc95000d230 0x7fc950010ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:38.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:38.995+0000 7fc970f64640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500096a0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:38.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:38.995+0000 7fc971765640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 msgr2=0x7fc950008850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:38.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:38.995+0000 7fc971765640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 0x7fc950008850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:39.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:39.396+0000 7fc971f66640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc95000d230 msgr2=0x7fc950010ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:39.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:39.396+0000 7fc971f66640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc95000d230 0x7fc950010ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:39.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:39.396+0000 7fc971765640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 msgr2=0x7fc950008850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:39.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:39.396+0000 7fc970f64640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500096a0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:39.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:39.396+0000 7fc971765640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 0x7fc950008850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:40.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:40.197+0000 7fc971f66640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc95000d230 msgr2=0x7fc950010ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:40.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:40.197+0000 7fc971f66640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc95000d230 0x7fc950010ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:40.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:40.197+0000 7fc971765640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 msgr2=0x7fc950008850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:40.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:40.197+0000 7fc970f64640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500096a0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:40.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:40.197+0000 7fc971765640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 0x7fc950008850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:41.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:41.794+0000 7fc95effd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 msgr2=0x7fc950008850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:41.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:41.794+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 0x7fc950008850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:03:41.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:41.794+0000 7fc95effd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc95000d230 msgr2=0x7fc950010ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:41.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:41.794+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc95000d230 0x7fc950010ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:03:41.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:41.794+0000 7fc95effd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500096a0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:41.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:41.794+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950011220 0x7fc950013610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:41.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:41.794+0000 7fc95effd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c15dae0 con 0x7fc950013b50 2023-11-09T04:03:41.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:41.794+0000 7fc95effd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc950011220 2023-11-09T04:03:41.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:41.794+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013b50 legacy=0x7fc950015f40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:41.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:41.794+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950011220 msgr2=0x7fc950013610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:41.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:41.795+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950011220 0x7fc950013610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:41.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:41.995+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013b50 legacy=0x7fc950015f40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:41.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:41.996+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950011220 msgr2=0x7fc950013610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:41.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:41.996+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950011220 0x7fc950013610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:42.395 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:42.396+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013b50 legacy=0x7fc950015f40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:42.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:42.397+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950011220 msgr2=0x7fc950013610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:42.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:42.397+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950011220 0x7fc950013610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:43.196 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:43.197+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013b50 legacy=0x7fc950015f40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:43.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:43.198+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950011220 msgr2=0x7fc950013610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:43.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:43.198+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950011220 0x7fc950013610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:44.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:44.795+0000 7fc95effd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950011220 msgr2=0x7fc950013610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:44.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:44.795+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950011220 0x7fc950013610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:03:44.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:44.795+0000 7fc95effd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013b50 legacy=0x7fc950015f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:44.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:44.795+0000 7fc971f66640 1 -- reap_dead start 2023-11-09T04:03:44.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:44.795+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc95000d230 0x7fc950010ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:44.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:44.795+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 0x7fc950008850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:44.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:44.795+0000 7fc95effd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c071500 con 0x7fc9500096a0 2023-11-09T04:03:44.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:44.795+0000 7fc95effd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc95000d230 2023-11-09T04:03:44.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:44.795+0000 7fc95effd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95000af60 con 0x7fc950005cf0 2023-11-09T04:03:44.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:44.795+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc95000d230 msgr2=0x7fc950010ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:44.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:44.795+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc95000d230 0x7fc950010ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:44.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:44.795+0000 7fc970f64640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500096a0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:44.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:44.795+0000 7fc971f66640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 msgr2=0x7fc950008850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:44.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:44.795+0000 7fc971f66640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 0x7fc950008850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:44.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:44.996+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc95000d230 msgr2=0x7fc950010ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:44.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:44.996+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc95000d230 0x7fc950010ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:44.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:44.996+0000 7fc971f66640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 msgr2=0x7fc950008850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:44.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:44.996+0000 7fc970f64640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500096a0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:44.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:44.996+0000 7fc971f66640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 0x7fc950008850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:45.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:45.397+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc95000d230 msgr2=0x7fc950010ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:45.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:45.397+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc95000d230 0x7fc950010ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:45.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:45.397+0000 7fc971f66640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 msgr2=0x7fc950008850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:45.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:45.397+0000 7fc970f64640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500096a0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:45.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:45.397+0000 7fc971f66640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 0x7fc950008850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:46.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:46.197+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc95000d230 msgr2=0x7fc950010ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:46.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:46.197+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc95000d230 0x7fc950010ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:46.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:46.197+0000 7fc970f64640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500096a0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:46.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:46.197+0000 7fc971f66640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 msgr2=0x7fc950008850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:46.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:46.197+0000 7fc971f66640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 0x7fc950008850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:47.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:47.794+0000 7fc95effd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 msgr2=0x7fc950008850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:47.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:47.794+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950005cf0 0x7fc950008850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:03:47.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:47.794+0000 7fc95effd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc95000d230 msgr2=0x7fc950010ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:47.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:47.794+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc95000d230 0x7fc950010ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:03:47.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:47.794+0000 7fc95effd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500096a0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:47.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:47.794+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950014dd0 0x7fc9500171c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:47.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:47.794+0000 7fc95effd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c15dae0 con 0x7fc950011220 2023-11-09T04:03:47.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:47.794+0000 7fc95effd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc950014dd0 2023-11-09T04:03:47.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:47.794+0000 7fc971765640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950011220 legacy=0x7fc950013610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:47.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:47.794+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950014dd0 msgr2=0x7fc9500171c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:47.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:47.794+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950014dd0 0x7fc9500171c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:47.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:47.995+0000 7fc971765640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950011220 legacy=0x7fc950013610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:47.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:47.995+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950014dd0 msgr2=0x7fc9500171c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:47.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:47.995+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950014dd0 0x7fc9500171c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:48.396 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:48.396+0000 7fc971765640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950011220 legacy=0x7fc950013610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:48.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:48.396+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950014dd0 msgr2=0x7fc9500171c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:48.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:48.396+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950014dd0 0x7fc9500171c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:49.197 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:49.197+0000 7fc971765640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950011220 legacy=0x7fc950013610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:49.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:49.197+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950014dd0 msgr2=0x7fc9500171c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:49.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:49.197+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950014dd0 0x7fc9500171c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:50.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:50.795+0000 7fc95effd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950014dd0 msgr2=0x7fc9500171c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:50.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:50.795+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950014dd0 0x7fc9500171c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:03:50.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:50.795+0000 7fc95effd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950011220 legacy=0x7fc950013610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:50.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:50.795+0000 7fc971f66640 1 -- reap_dead start 2023-11-09T04:03:50.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:50.795+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc9500096a0 0x7fc950008850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:50.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:50.795+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:50.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:50.795+0000 7fc95effd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c071500 con 0x7fc9500177a0 2023-11-09T04:03:50.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:50.795+0000 7fc95effd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc950007d00 2023-11-09T04:03:50.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:50.795+0000 7fc95effd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc95000af60 con 0x7fc9500096a0 2023-11-09T04:03:50.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:50.795+0000 7fc971765640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc9500177a0 legacy=0x7fc950019b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:50.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:50.795+0000 7fc971f66640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:50.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:50.795+0000 7fc971f66640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:50.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:50.795+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc9500096a0 msgr2=0x7fc950008850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:50.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:50.795+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc9500096a0 0x7fc950008850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:50.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:50.996+0000 7fc971765640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc9500177a0 legacy=0x7fc950019b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:50.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:50.996+0000 7fc971f66640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:50.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:50.996+0000 7fc971f66640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:50.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:50.996+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc9500096a0 msgr2=0x7fc950008850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:50.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:50.996+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc9500096a0 0x7fc950008850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:51.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:51.397+0000 7fc971765640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc9500177a0 legacy=0x7fc950019b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:51.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:51.397+0000 7fc971f66640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:51.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:51.397+0000 7fc971f66640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:51.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:51.397+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc9500096a0 msgr2=0x7fc950008850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:51.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:51.397+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc9500096a0 0x7fc950008850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:52.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:52.198+0000 7fc971f66640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:52.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:52.198+0000 7fc971765640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc9500177a0 legacy=0x7fc950019b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:52.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:52.198+0000 7fc971f66640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:52.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:52.198+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc9500096a0 msgr2=0x7fc950008850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:52.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:52.198+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc9500096a0 0x7fc950008850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:53.795 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:53.795+0000 7fc95effd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc9500096a0 msgr2=0x7fc950008850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:53.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:53.795+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc9500096a0 0x7fc950008850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:03:53.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:53.795+0000 7fc95effd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:53.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:53.795+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:03:53.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:53.795+0000 7fc95effd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc9500177a0 legacy=0x7fc950019b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:53.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:53.795+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950013dc0 0x7fc9500161e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:53.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:53.795+0000 7fc95effd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c15dae0 con 0x7fc950013610 2023-11-09T04:03:53.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:53.795+0000 7fc95effd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc950013dc0 2023-11-09T04:03:53.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:53.795+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950013610 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:53.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:53.795+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950013dc0 msgr2=0x7fc9500161e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:53.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:53.795+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950013dc0 0x7fc9500161e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:53.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:53.996+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950013dc0 msgr2=0x7fc9500161e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:53.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:53.996+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950013610 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:53.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:53.996+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950013dc0 0x7fc9500161e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:54.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:54.397+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950013610 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:54.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:54.397+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950013dc0 msgr2=0x7fc9500161e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:54.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:54.397+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950013dc0 0x7fc9500161e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:55.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:55.198+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950013dc0 msgr2=0x7fc9500161e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:55.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:55.198+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950013610 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:55.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:55.198+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950013dc0 0x7fc9500161e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:56.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:56.795+0000 7fc95effd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950013dc0 msgr2=0x7fc9500161e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:56.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:56.795+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950013dc0 0x7fc9500161e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:03:56.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:56.795+0000 7fc95effd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950013610 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:56.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:56.795+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:56.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:56.795+0000 7fc971f66640 1 -- reap_dead start 2023-11-09T04:03:56.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:56.795+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 0x7fc950008830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:56.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:56.795+0000 7fc95effd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c071500 con 0x7fc950013dc0 2023-11-09T04:03:56.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:56.795+0000 7fc95effd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc950007d00 2023-11-09T04:03:56.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:56.795+0000 7fc95effd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc95000af60 con 0x7fc950013610 2023-11-09T04:03:56.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:56.795+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:56.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:56.795+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:56.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:56.795+0000 7fc971f66640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:56.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:56.795+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 msgr2=0x7fc950008830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:56.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:56.795+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 0x7fc950008830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:56.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:56.996+0000 7fc971f66640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:56.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:56.996+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:56.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:56.996+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:56.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:56.996+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 msgr2=0x7fc950008830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:56.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:56.996+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 0x7fc950008830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:03:57.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:57.397+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:57.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:57.397+0000 7fc971f66640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:57.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:57.397+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:57.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:57.397+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 msgr2=0x7fc950008830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:57.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:57.397+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 0x7fc950008830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:03:58.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:58.198+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:03:58.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:58.198+0000 7fc971f66640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:03:58.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:58.198+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:58.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:58.198+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 msgr2=0x7fc950008830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:03:58.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:58.198+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 0x7fc950008830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:03:59.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:59.796+0000 7fc95effd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 msgr2=0x7fc950008830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:59.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:59.796+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 0x7fc950008830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:03:59.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:59.796+0000 7fc95effd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:59.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:59.796+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:03:59.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:59.796+0000 7fc95effd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:03:59.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:59.796+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950017900 0x7fc950019cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:03:59.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:59.796+0000 7fc95effd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c15dae0 con 0x7fc9500096a0 2023-11-09T04:03:59.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:59.796+0000 7fc95effd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc950017900 2023-11-09T04:03:59.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:59.796+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500096a0 legacy=0x7fc9500161e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:59.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:59.796+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950017900 msgr2=0x7fc950019cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:59.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:59.796+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950017900 0x7fc950019cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:03:59.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:59.997+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500096a0 legacy=0x7fc9500161e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:03:59.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:59.997+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950017900 msgr2=0x7fc950019cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:03:59.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:03:59.997+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950017900 0x7fc950019cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:00.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:00.398+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500096a0 legacy=0x7fc9500161e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:00.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:00.398+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950017900 msgr2=0x7fc950019cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:00.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:00.398+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950017900 0x7fc950019cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:01.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:01.200+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500096a0 legacy=0x7fc9500161e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:01.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:01.200+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950017900 msgr2=0x7fc950019cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:01.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:01.200+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950017900 0x7fc950019cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:02.796 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:02.796+0000 7fc95effd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950017900 msgr2=0x7fc950019cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:02.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:02.796+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950017900 0x7fc950019cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:02.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:02.796+0000 7fc95effd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500096a0 legacy=0x7fc9500161e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:02.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:02.796+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:02.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:02.796+0000 7fc971f66640 1 -- reap_dead start 2023-11-09T04:04:02.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:02.796+0000 7fc95effd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c071500 con 0x7fc950013dc0 2023-11-09T04:04:02.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:02.796+0000 7fc95effd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c15dae0 con 0x7fc95001a230 2023-11-09T04:04:02.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:02.796+0000 7fc95effd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc950007d00 2023-11-09T04:04:02.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:02.796+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:02.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:02.796+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:02.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:02.796+0000 7fc971f66640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:02.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:02.797+0000 7fc970f64640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc95001a230 legacy=0x7fc95001c620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:02.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:02.997+0000 7fc971f66640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:02.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:02.997+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:02.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:02.997+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:02.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:02.998+0000 7fc970f64640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc95001a230 legacy=0x7fc95001c620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:03.397 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:03.398+0000 7fc971f66640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:03.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:03.398+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:03.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:03.398+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:03.398 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:03.399+0000 7fc970f64640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc95001a230 legacy=0x7fc95001c620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:04.198 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:04.199+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:04.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:04.199+0000 7fc971f66640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:04.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:04.199+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:04.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:04.200+0000 7fc970f64640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc95001a230 legacy=0x7fc95001c620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:05.522 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:04:05.525 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:04:05.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:05.796+0000 7fc95effd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:05.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:05.797+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:05.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:05.797+0000 7fc95effd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc95001a230 legacy=0x7fc95001c620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:05.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:05.797+0000 7fc95effd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:05.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:05.797+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 0x7fc950008830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:05.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:05.797+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 0x7fc9500165c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:05.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:05.797+0000 7fc95effd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc950013610 2023-11-09T04:04:05.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:05.797+0000 7fc95effd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95000af60 con 0x7fc9500141a0 2023-11-09T04:04:05.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:05.797+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 msgr2=0x7fc9500165c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:05.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:05.797+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 msgr2=0x7fc950008830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:05.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:05.797+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 0x7fc9500165c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:05.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:05.797+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 0x7fc950008830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:05.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:05.998+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 msgr2=0x7fc9500165c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:05.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:05.998+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 msgr2=0x7fc950008830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:05.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:05.998+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 0x7fc9500165c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:05.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:05.998+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 0x7fc950008830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:06.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:06.399+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 msgr2=0x7fc9500165c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:06.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:06.399+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 0x7fc9500165c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:06.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:06.399+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 msgr2=0x7fc950008830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:06.461 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:06.399+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 0x7fc950008830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:07.199 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:07.200+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 msgr2=0x7fc9500165c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:07.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:07.200+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 msgr2=0x7fc950008830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:07.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:07.200+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 0x7fc950008830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:07.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:07.200+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 0x7fc9500165c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:08.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:08.797+0000 7fc95effd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 msgr2=0x7fc9500165c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:08.797 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:08.797+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 0x7fc9500165c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:08.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:08.797+0000 7fc95effd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 msgr2=0x7fc950008830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:08.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:08.797+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 0x7fc950008830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:08.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:08.797+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 0x7fc9500165c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:08.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:08.797+0000 7fc971f66640 1 -- reap_dead start 2023-11-09T04:04:08.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:08.797+0000 7fc95effd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c070f40 con 0x7fc950013dc0 2023-11-09T04:04:08.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:08.797+0000 7fc95effd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c071500 con 0x7fc95001a230 2023-11-09T04:04:08.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:08.797+0000 7fc95effd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc9500141a0 2023-11-09T04:04:08.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:08.797+0000 7fc971f66640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:08.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:08.797+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 msgr2=0x7fc9500165c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:08.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:08.797+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc95001a230 legacy=0x7fc950018b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:08.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:08.797+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 0x7fc9500165c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:08.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:08.998+0000 7fc971f66640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:08.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:08.998+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc95001a230 legacy=0x7fc950018b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:08.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:08.998+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 msgr2=0x7fc9500165c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:08.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:08.998+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 0x7fc9500165c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:09.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:09.399+0000 7fc971f66640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:09.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:09.399+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc95001a230 legacy=0x7fc950018b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:09.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:09.399+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 msgr2=0x7fc9500165c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:09.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:09.399+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 0x7fc9500165c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:10.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:10.200+0000 7fc971f66640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:10.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:10.200+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 msgr2=0x7fc9500165c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:10.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:10.200+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc95001a230 legacy=0x7fc950018b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:10.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:10.200+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 0x7fc9500165c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:11.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:11.797+0000 7fc95effd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 msgr2=0x7fc9500165c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:11.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:11.797+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 0x7fc9500165c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:11.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:11.797+0000 7fc95effd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc95001a230 legacy=0x7fc950018b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:11.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:11.797+0000 7fc95effd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:11.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:11.797+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:11.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:11.797+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 0x7fc95001a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:11.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:11.797+0000 7fc95effd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc9500087f0 2023-11-09T04:04:11.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:11.797+0000 7fc95effd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc95000af60 con 0x7fc950007d00 2023-11-09T04:04:11.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:11.797+0000 7fc970f64640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 msgr2=0x7fc95001a610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:11.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:11.797+0000 7fc970f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 0x7fc95001a610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:11.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:11.797+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950007d00 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:11.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:11.797+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:11.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:11.998+0000 7fc970f64640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 msgr2=0x7fc95001a610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:11.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:11.998+0000 7fc970f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 0x7fc95001a610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:11.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:11.998+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950007d00 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:11.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:11.998+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:12.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:12.399+0000 7fc970f64640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 msgr2=0x7fc95001a610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:12.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:12.399+0000 7fc970f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 0x7fc95001a610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:12.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:12.399+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950007d00 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:12.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:12.399+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:13.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:13.200+0000 7fc970f64640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 msgr2=0x7fc95001a610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:13.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:13.200+0000 7fc970f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 0x7fc95001a610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:13.200 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:13.200+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950007d00 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:13.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:13.200+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:14.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:14.798+0000 7fc95effd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950007d00 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:14.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:14.798+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:14.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:14.798+0000 7fc95effd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 msgr2=0x7fc95001a610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:14.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:14.798+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 0x7fc95001a610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:14.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:14.798+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 0x7fc950016590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:14.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:14.798+0000 7fc971f66640 1 -- reap_dead start 2023-11-09T04:04:14.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:14.798+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:14.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:14.798+0000 7fc95effd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c15dae0 con 0x7fc950013dc0 2023-11-09T04:04:14.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:14.798+0000 7fc95effd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc950007d00 2023-11-09T04:04:14.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:14.798+0000 7fc95effd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95000af60 con 0x7fc9500141a0 2023-11-09T04:04:14.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:14.798+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:14.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:14.798+0000 7fc971765640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 msgr2=0x7fc950016590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:14.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:14.798+0000 7fc971f66640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950007d00 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:14.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:14.798+0000 7fc971f66640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:14.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:14.798+0000 7fc971765640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 0x7fc950016590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:14.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:14.999+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:14.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:14.999+0000 7fc971765640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 msgr2=0x7fc950016590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:14.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:14.999+0000 7fc971f66640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950007d00 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:14.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:14.999+0000 7fc971765640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 0x7fc950016590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:15.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:14.999+0000 7fc971f66640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:15.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:15.400+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:15.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:15.400+0000 7fc971f66640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950007d00 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:15.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:15.400+0000 7fc971765640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 msgr2=0x7fc950016590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:15.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:15.400+0000 7fc971f66640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:15.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:15.400+0000 7fc971765640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 0x7fc950016590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:16.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:16.201+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:16.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:16.201+0000 7fc971f66640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950007d00 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:16.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:16.201+0000 7fc971765640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 msgr2=0x7fc950016590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:16.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:16.201+0000 7fc971f66640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:16.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:16.201+0000 7fc971765640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 0x7fc950016590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:17.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:17.797+0000 7fc95effd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 msgr2=0x7fc950016590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:17.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:17.797+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 0x7fc950016590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:17.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:17.797+0000 7fc95effd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950007d00 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:17.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:17.797+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950007d00 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:17.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:17.797+0000 7fc95effd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:17.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:17.797+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950013610 0x7fc95001a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:17.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:17.797+0000 7fc95effd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c070f40 con 0x7fc9500087f0 2023-11-09T04:04:17.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:17.797+0000 7fc95effd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc950013610 2023-11-09T04:04:17.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:17.797+0000 7fc970f64640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950013610 msgr2=0x7fc95001a200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:17.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:17.797+0000 7fc970f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950013610 0x7fc95001a200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:17.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:17.797+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500087f0 legacy=0x7fc950016ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:17.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:17.998+0000 7fc970f64640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950013610 msgr2=0x7fc95001a200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:17.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:17.998+0000 7fc970f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950013610 0x7fc95001a200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:17.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:17.998+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500087f0 legacy=0x7fc950016ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:18.399 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:18.399+0000 7fc970f64640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950013610 msgr2=0x7fc95001a200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:18.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:18.399+0000 7fc970f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950013610 0x7fc95001a200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:18.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:18.399+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500087f0 legacy=0x7fc950016ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:19.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:19.200+0000 7fc970f64640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950013610 msgr2=0x7fc95001a200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:19.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:19.200+0000 7fc970f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950013610 0x7fc95001a200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:19.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:19.200+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500087f0 legacy=0x7fc950016ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:20.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:20.797+0000 7fc95effd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950013610 msgr2=0x7fc95001a200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:20.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:20.797+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950013610 0x7fc95001a200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:20.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:20.797+0000 7fc95effd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500087f0 legacy=0x7fc950016ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:20.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:20.797+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 0x7fc950016590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:20.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:20.797+0000 7fc971f66640 1 -- reap_dead start 2023-11-09T04:04:20.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:20.797+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 0x7fc950010bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:20.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:20.797+0000 7fc95effd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c15dae0 con 0x7fc950013dc0 2023-11-09T04:04:20.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:20.797+0000 7fc95effd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc9500087f0 2023-11-09T04:04:20.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:20.797+0000 7fc95effd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95000af60 con 0x7fc9500141a0 2023-11-09T04:04:20.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:20.797+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:20.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:20.797+0000 7fc971f66640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:20.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:20.797+0000 7fc971f66640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:20.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:20.797+0000 7fc971765640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 msgr2=0x7fc950016590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:20.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:20.797+0000 7fc971765640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 0x7fc950016590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:20.998 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:20.998+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:20.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:20.998+0000 7fc971f66640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:20.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:20.998+0000 7fc971f66640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:20.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:20.998+0000 7fc971765640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 msgr2=0x7fc950016590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:20.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:20.998+0000 7fc971765640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 0x7fc950016590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:21.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:21.399+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:21.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:21.399+0000 7fc971f66640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:21.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:21.399+0000 7fc971f66640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:21.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:21.399+0000 7fc971765640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 msgr2=0x7fc950016590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:21.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:21.399+0000 7fc971765640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 0x7fc950016590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:22.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:22.200+0000 7fc971f66640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:22.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:22.200+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:22.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:22.200+0000 7fc971f66640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:22.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:22.200+0000 7fc971765640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 msgr2=0x7fc950016590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:22.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:22.200+0000 7fc971765640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 0x7fc950016590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:23.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:23.798+0000 7fc95effd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 msgr2=0x7fc950016590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:23.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:23.798+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500141a0 0x7fc950016590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:23.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:23.798+0000 7fc95effd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:23.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:23.798+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:23.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:23.798+0000 7fc95effd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:23.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:23.798+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 0x7fc95001a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:23.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:23.798+0000 7fc95effd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c070f40 con 0x7fc950007ce0 2023-11-09T04:04:23.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:23.798+0000 7fc95effd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc950013610 2023-11-09T04:04:23.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:23.798+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950007ce0 legacy=0x7fc950016ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:23.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:23.798+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 msgr2=0x7fc95001a200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:23.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:23.798+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 0x7fc95001a200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:24.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:23.999+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950007ce0 legacy=0x7fc950016ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:24.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:23.999+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 msgr2=0x7fc95001a200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:24.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:23.999+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 0x7fc95001a200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:24.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:24.400+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950007ce0 legacy=0x7fc950016ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:24.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:24.400+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 msgr2=0x7fc95001a200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:24.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:24.400+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 0x7fc95001a200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:25.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:25.201+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950007ce0 legacy=0x7fc950016ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:25.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:25.201+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 msgr2=0x7fc95001a200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:25.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:25.201+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 0x7fc95001a200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:26.798 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:26.798+0000 7fc95effd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 msgr2=0x7fc95001a200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:26.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:26.798+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950013610 0x7fc95001a200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:26.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:26.798+0000 7fc95effd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950007ce0 legacy=0x7fc950016ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:26.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:26.798+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc95001a740 0x7fc95001cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:26.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:26.798+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 0x7fc950010bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:26.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:26.798+0000 7fc95effd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c15dae0 con 0x7fc950013dc0 2023-11-09T04:04:26.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:26.798+0000 7fc971f66640 1 -- reap_dead start 2023-11-09T04:04:26.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:26.798+0000 7fc95effd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc9500087f0 2023-11-09T04:04:26.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:26.798+0000 7fc95effd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc95000af60 con 0x7fc95001a740 2023-11-09T04:04:26.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:26.798+0000 7fc970f64640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:26.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:26.798+0000 7fc971f66640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:26.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:26.798+0000 7fc970f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:26.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:26.798+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc95001a740 msgr2=0x7fc95001cb30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:26.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:26.798+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc95001a740 0x7fc95001cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:26.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:26.999+0000 7fc971f66640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:27.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:26.999+0000 7fc970f64640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:27.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:26.999+0000 7fc970f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:27.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:26.999+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc95001a740 msgr2=0x7fc95001cb30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:27.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:26.999+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc95001a740 0x7fc95001cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:27.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:27.400+0000 7fc970f64640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:27.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:27.400+0000 7fc971f66640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:27.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:27.400+0000 7fc970f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:27.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:27.400+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc95001a740 msgr2=0x7fc95001cb30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:27.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:27.400+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc95001a740 0x7fc95001cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:28.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:28.201+0000 7fc970f64640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:28.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:28.201+0000 7fc971f66640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:28.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:28.201+0000 7fc970f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:28.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:28.201+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc95001a740 msgr2=0x7fc95001cb30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:28.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:28.201+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc95001a740 0x7fc95001cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:29.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:29.798+0000 7fc95effd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc95001a740 msgr2=0x7fc95001cb30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:29.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:29.798+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc95001a740 0x7fc95001cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:29.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:29.798+0000 7fc95effd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:29.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:29.798+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:29.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:29.798+0000 7fc95effd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:29.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:29.798+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500158c0 0x7fc950017cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:29.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:29.798+0000 7fc95effd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c070f40 con 0x7fc950007ce0 2023-11-09T04:04:29.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:29.798+0000 7fc95effd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc9500158c0 2023-11-09T04:04:29.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:29.798+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500158c0 msgr2=0x7fc950017cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:29.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:29.798+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950007ce0 legacy=0x7fc9500141a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:29.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:29.798+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500158c0 0x7fc950017cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:29.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:29.999+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950007ce0 legacy=0x7fc9500141a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:29.999 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:29.999+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500158c0 msgr2=0x7fc950017cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:30.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:29.999+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500158c0 0x7fc950017cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:30.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:30.400+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950007ce0 legacy=0x7fc9500141a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:30.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:30.400+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500158c0 msgr2=0x7fc950017cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:30.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:30.400+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500158c0 0x7fc950017cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:31.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:31.201+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950007ce0 legacy=0x7fc9500141a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:31.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:31.201+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500158c0 msgr2=0x7fc950017cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:31.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:31.201+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500158c0 0x7fc950017cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:32.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:32.799+0000 7fc95effd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500158c0 msgr2=0x7fc950017cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:32.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:32.799+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc9500158c0 0x7fc950017cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:32.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:32.799+0000 7fc95effd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950007ce0 legacy=0x7fc9500141a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:32.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:32.799+0000 7fc971f66640 1 -- reap_dead start 2023-11-09T04:04:32.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:32.799+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 0x7fc950010bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:32.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:32.799+0000 7fc95effd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c15dae0 con 0x7fc950013dc0 2023-11-09T04:04:32.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:32.799+0000 7fc95effd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c070f40 con 0x7fc9500181f0 2023-11-09T04:04:32.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:32.799+0000 7fc95effd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc9500087f0 2023-11-09T04:04:32.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:32.799+0000 7fc970f64640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:32.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:32.799+0000 7fc971f66640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:32.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:32.799+0000 7fc970f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:32.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:32.799+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500181f0 legacy=0x7fc95001d070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:33.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:33.001+0000 7fc971f66640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:33.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:33.001+0000 7fc970f64640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:33.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:33.001+0000 7fc970f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:33.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:33.001+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500181f0 legacy=0x7fc95001d070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:33.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:33.402+0000 7fc971f66640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:33.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:33.402+0000 7fc970f64640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:33.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:33.402+0000 7fc970f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:33.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:33.402+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500181f0 legacy=0x7fc95001d070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:34.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:34.203+0000 7fc971f66640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:34.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:34.203+0000 7fc970f64640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:34.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:34.203+0000 7fc970f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:34.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:34.203+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500181f0 legacy=0x7fc95001d070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:35.586 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:04:35.589 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:04:35.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:35.799+0000 7fc95effd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:35.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:35.799+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9500087f0 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:35.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:35.799+0000 7fc95effd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500181f0 legacy=0x7fc95001d070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:35.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:35.799+0000 7fc95effd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:35.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:35.799+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950011600 0x7fc95001c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:35.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:35.799+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc95001cbd0 0x7fc9500119e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:35.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:35.799+0000 7fc95effd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc950011600 2023-11-09T04:04:35.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:35.799+0000 7fc95effd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95000af60 con 0x7fc95001cbd0 2023-11-09T04:04:35.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:35.799+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950011600 msgr2=0x7fc95001c690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:35.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:35.799+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc95001cbd0 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:35.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:35.799+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950011600 0x7fc95001c690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:35.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:35.799+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc95001cbd0 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:36.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:36.000+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950011600 msgr2=0x7fc95001c690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:36.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:36.000+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc95001cbd0 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:36.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:36.000+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950011600 0x7fc95001c690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:36.000 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:36.000+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc95001cbd0 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:36.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:36.401+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950011600 msgr2=0x7fc95001c690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:36.400 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:36.401+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc95001cbd0 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:36.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:36.401+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950011600 0x7fc95001c690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:36.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:36.401+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc95001cbd0 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:37.201 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:37.202+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc95001cbd0 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:37.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:37.202+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950011600 msgr2=0x7fc95001c690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:37.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:37.202+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950011600 0x7fc95001c690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:37.202 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:37.202+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc95001cbd0 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:38.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:38.799+0000 7fc95effd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc95001cbd0 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:38.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:38.799+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc95001cbd0 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:38.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:38.799+0000 7fc95effd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950011600 msgr2=0x7fc95001c690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:38.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:38.799+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950011600 0x7fc95001c690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:38.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:38.799+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc95001cbd0 0x7fc9500119e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:38.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:38.799+0000 7fc971f66640 1 -- reap_dead start 2023-11-09T04:04:38.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:38.800+0000 7fc95effd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c071500 con 0x7fc950013dc0 2023-11-09T04:04:38.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:38.800+0000 7fc95effd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c15dae0 con 0x7fc9500087f0 2023-11-09T04:04:38.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:38.800+0000 7fc95effd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc95001cbd0 2023-11-09T04:04:38.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:38.800+0000 7fc971765640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:38.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:38.800+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc95001cbd0 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:38.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:38.800+0000 7fc971f66640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500087f0 legacy=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:38.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:38.800+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc95001cbd0 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:39.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:39.001+0000 7fc971765640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:39.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:39.001+0000 7fc971f66640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500087f0 legacy=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:39.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:39.001+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc95001cbd0 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:39.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:39.001+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc95001cbd0 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:39.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:39.402+0000 7fc971765640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:39.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:39.402+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc95001cbd0 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:39.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:39.402+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc95001cbd0 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:39.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:39.402+0000 7fc971f66640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500087f0 legacy=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:40.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:40.203+0000 7fc971765640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:40.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:40.203+0000 7fc971f66640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500087f0 legacy=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:40.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:40.203+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc95001cbd0 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:40.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:40.203+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc95001cbd0 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:41.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:41.800+0000 7fc95effd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc95001cbd0 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:41.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:41.800+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc95001cbd0 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:41.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:41.800+0000 7fc95effd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500087f0 legacy=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:41.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:41.800+0000 7fc95effd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:41.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:41.800+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950011600 0x7fc950010bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:41.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:41.800+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012630 0x7fc950012a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:41.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:41.800+0000 7fc95effd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc950011600 2023-11-09T04:04:41.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:41.800+0000 7fc95effd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc95000af60 con 0x7fc950012630 2023-11-09T04:04:41.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:41.800+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950011600 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:41.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:41.800+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950011600 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:41.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:41.800+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012630 msgr2=0x7fc950012a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:41.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:41.800+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012630 0x7fc950012a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:42.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:42.001+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950011600 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:42.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:42.001+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950011600 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:42.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:42.001+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012630 msgr2=0x7fc950012a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:42.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:42.001+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012630 0x7fc950012a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:42.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:42.402+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950011600 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:42.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:42.402+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950011600 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:42.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:42.402+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012630 msgr2=0x7fc950012a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:42.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:42.402+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012630 0x7fc950012a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:43.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:43.203+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950011600 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:43.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:43.203+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950011600 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:43.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:43.203+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012630 msgr2=0x7fc950012a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:43.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:43.203+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012630 0x7fc950012a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:44.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:44.800+0000 7fc95effd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012630 msgr2=0x7fc950012a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:44.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:44.800+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012630 0x7fc950012a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:44.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:44.800+0000 7fc95effd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950011600 msgr2=0x7fc950010bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:44.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:44.800+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950011600 0x7fc950010bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:44.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:44.800+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 0x7fc95001b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:44.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:44.800+0000 7fc971f66640 1 -- reap_dead start 2023-11-09T04:04:44.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:44.800+0000 7fc95effd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c070f40 con 0x7fc950013dc0 2023-11-09T04:04:44.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:44.800+0000 7fc95effd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c071500 con 0x7fc9500087f0 2023-11-09T04:04:44.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:44.800+0000 7fc95effd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc950012fb0 2023-11-09T04:04:44.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:44.800+0000 7fc971f66640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:44.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:44.800+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500087f0 legacy=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:44.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:44.800+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 msgr2=0x7fc95001b1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:44.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:44.800+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 0x7fc95001b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:45.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:45.001+0000 7fc971f66640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:45.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:45.001+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500087f0 legacy=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:45.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:45.001+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 msgr2=0x7fc95001b1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:45.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:45.001+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 0x7fc95001b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:45.401 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:45.402+0000 7fc971f66640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:45.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:45.402+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 msgr2=0x7fc95001b1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:45.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:45.402+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500087f0 legacy=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:45.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:45.402+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 0x7fc95001b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:46.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:46.203+0000 7fc971f66640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:46.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:46.203+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 msgr2=0x7fc95001b1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:46.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:46.203+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500087f0 legacy=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:46.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:46.203+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 0x7fc95001b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:47.800 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:47.801+0000 7fc95effd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 msgr2=0x7fc95001b1e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:47.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:47.801+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 0x7fc95001b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:47.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:47.801+0000 7fc95effd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500087f0 legacy=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:47.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:47.801+0000 7fc95effd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:47.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:47.801+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 0x7fc950012a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:47.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:47.801+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:47.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:47.801+0000 7fc95effd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc950010bc0 2023-11-09T04:04:47.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:47.801+0000 7fc95effd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95000af60 con 0x7fc950011600 2023-11-09T04:04:47.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:47.801+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 msgr2=0x7fc950012a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:47.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:47.801+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 0x7fc950012a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:47.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:47.801+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:47.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:47.801+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:48.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:48.002+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 msgr2=0x7fc950012a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:48.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:48.002+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 0x7fc950012a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:48.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:48.002+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:48.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:48.002+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:48.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:48.403+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 msgr2=0x7fc950012a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:48.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:48.403+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 0x7fc950012a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:48.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:48.403+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:48.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:48.403+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:49.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:49.204+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 msgr2=0x7fc950012a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:49.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:49.204+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 0x7fc950012a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:49.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:49.204+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:49.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:49.204+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:50.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:50.800+0000 7fc95effd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:50.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:50.800+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:50.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:50.800+0000 7fc95effd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 msgr2=0x7fc950012a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:50.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:50.800+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 0x7fc950012a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:50.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:50.800+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:50.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:50.800+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 0x7fc950012a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:50.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:50.800+0000 7fc95effd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c15dae0 con 0x7fc9500087f0 2023-11-09T04:04:50.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:50.800+0000 7fc971f66640 1 -- reap_dead start 2023-11-09T04:04:50.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:50.800+0000 7fc95effd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc950010bc0 2023-11-09T04:04:50.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:50.800+0000 7fc95effd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95000af60 con 0x7fc950011600 2023-11-09T04:04:50.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:50.800+0000 7fc971f66640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 msgr2=0x7fc950012a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:50.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:50.800+0000 7fc971f66640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 0x7fc950012a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:50.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:50.800+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500087f0 legacy=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:50.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:50.800+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:50.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:50.800+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:51.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:51.001+0000 7fc971f66640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 msgr2=0x7fc950012a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:51.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:51.001+0000 7fc971f66640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 0x7fc950012a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:51.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:51.001+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:51.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:51.001+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500087f0 legacy=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:51.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:51.001+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:51.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:51.402+0000 7fc971f66640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 msgr2=0x7fc950012a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:51.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:51.402+0000 7fc971f66640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 0x7fc950012a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:51.402 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:51.402+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:51.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:51.402+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500087f0 legacy=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:51.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:51.402+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:52.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:52.203+0000 7fc971f66640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 msgr2=0x7fc950012a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:52.203 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:52.203+0000 7fc971f66640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 0x7fc950012a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:52.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:52.203+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500087f0 legacy=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:52.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:52.203+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:52.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:52.203+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:53.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:53.800+0000 7fc95effd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:53.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:53.800+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:53.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:53.800+0000 7fc95effd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 msgr2=0x7fc950012a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:53.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:53.800+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 0x7fc950012a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:53.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:53.800+0000 7fc95effd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500087f0 legacy=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:53.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:53.800+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 0x7fc95001b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:53.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:53.800+0000 7fc95effd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c070f40 con 0x7fc950013dc0 2023-11-09T04:04:53.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:53.800+0000 7fc95effd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc950012fb0 2023-11-09T04:04:53.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:53.800+0000 7fc971765640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:53.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:53.800+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 msgr2=0x7fc95001b1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:53.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:53.800+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 0x7fc95001b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:54.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:54.001+0000 7fc971765640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:54.001 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:54.001+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 msgr2=0x7fc95001b1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:54.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:54.001+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 0x7fc95001b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:54.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:54.402+0000 7fc971765640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:54.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:54.402+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 msgr2=0x7fc95001b1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:54.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:54.402+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 0x7fc95001b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:55.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:55.203+0000 7fc971765640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:55.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:55.203+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 msgr2=0x7fc95001b1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:55.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:55.203+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 0x7fc95001b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:56.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:56.801+0000 7fc95effd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 msgr2=0x7fc95001b1e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:56.801 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:56.801+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 0x7fc95001b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:56.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:56.801+0000 7fc95effd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc950013dc0 legacy=0x7fc950009eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:56.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:56.801+0000 7fc971f66640 1 -- reap_dead start 2023-11-09T04:04:56.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:56.801+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 0x7fc950012a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:56.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:56.801+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:56.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:56.801+0000 7fc95effd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c15dae0 con 0x7fc95001b720 2023-11-09T04:04:56.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:56.801+0000 7fc95effd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc950010bc0 2023-11-09T04:04:56.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:56.801+0000 7fc95effd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95000af60 con 0x7fc950011600 2023-11-09T04:04:56.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:56.801+0000 7fc971765640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc95001b720 legacy=0x7fc95001db10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:56.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:56.801+0000 7fc970f64640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 msgr2=0x7fc950012a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:56.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:56.801+0000 7fc970f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 0x7fc950012a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:56.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:56.801+0000 7fc971f66640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:56.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:56.801+0000 7fc971f66640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:04:57.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:57.002+0000 7fc970f64640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 msgr2=0x7fc950012a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:57.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:57.002+0000 7fc971765640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc95001b720 legacy=0x7fc95001db10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:57.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:57.002+0000 7fc970f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 0x7fc950012a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:57.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:57.002+0000 7fc971f66640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:57.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:57.002+0000 7fc971f66640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:04:57.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:57.403+0000 7fc970f64640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 msgr2=0x7fc950012a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:57.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:57.403+0000 7fc971765640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc95001b720 legacy=0x7fc95001db10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:57.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:57.403+0000 7fc970f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 0x7fc950012a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:57.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:57.403+0000 7fc971f66640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:57.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:57.403+0000 7fc971f66640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:04:58.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:58.204+0000 7fc971765640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc95001b720 legacy=0x7fc95001db10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:04:58.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:58.204+0000 7fc970f64640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 msgr2=0x7fc950012a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:04:58.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:58.204+0000 7fc970f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 0x7fc950012a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:58.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:58.204+0000 7fc971f66640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:04:58.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:58.204+0000 7fc971f66640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:04:59.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:59.801+0000 7fc95effd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:59.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:59.801+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:59.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:59.801+0000 7fc95effd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 msgr2=0x7fc950012a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:59.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:59.801+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 0x7fc950012a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:04:59.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:59.801+0000 7fc95effd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc95001b720 legacy=0x7fc95001db10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:04:59.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:59.801+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 0x7fc95001b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:04:59.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:59.801+0000 7fc95effd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c070f40 con 0x7fc9500087f0 2023-11-09T04:04:59.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:59.801+0000 7fc95effd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc950012fb0 2023-11-09T04:04:59.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:59.801+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 msgr2=0x7fc95001b1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:04:59.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:59.801+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500087f0 legacy=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:04:59.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:04:59.801+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 0x7fc95001b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:00.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:00.002+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500087f0 legacy=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:00.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:00.002+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 msgr2=0x7fc95001b1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:00.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:00.002+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 0x7fc95001b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:00.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:00.403+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 msgr2=0x7fc95001b1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:00.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:00.403+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500087f0 legacy=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:00.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:00.403+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 0x7fc95001b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:01.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:01.204+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500087f0 legacy=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:01.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:01.204+0000 7fc970f64640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 msgr2=0x7fc95001b1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:01.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:01.204+0000 7fc970f64640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 0x7fc95001b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:02.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:02.801+0000 7fc95effd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 msgr2=0x7fc95001b1e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:02.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:02.801+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 0x7fc95001b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:05:02.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:02.801+0000 7fc95effd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc9500087f0 legacy=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:02.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:02.801+0000 7fc971f66640 1 -- reap_dead start 2023-11-09T04:05:02.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:02.801+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 0x7fc950011f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:02.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:02.801+0000 7fc95effd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c15dae0 con 0x7fc95001b720 2023-11-09T04:05:02.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:02.801+0000 7fc95effd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c070f40 con 0x7fc950009eb0 2023-11-09T04:05:02.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:02.801+0000 7fc95effd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc950012fb0 2023-11-09T04:05:02.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:02.801+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc95001b720 legacy=0x7fc9500168c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:02.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:02.801+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950009eb0 legacy=0x7fc9500161b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:02.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:02.801+0000 7fc971f66640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 msgr2=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:02.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:02.801+0000 7fc971f66640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 0x7fc950011f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:03.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:03.002+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc95001b720 legacy=0x7fc9500168c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:03.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:03.002+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950009eb0 legacy=0x7fc9500161b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:03.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:03.002+0000 7fc971f66640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 msgr2=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:03.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:03.002+0000 7fc971f66640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 0x7fc950011f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:03.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:03.403+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc95001b720 legacy=0x7fc9500168c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:03.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:03.403+0000 7fc971f66640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 msgr2=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:03.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:03.403+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950009eb0 legacy=0x7fc9500161b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:03.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:03.403+0000 7fc971f66640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 0x7fc950011f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:04.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:04.204+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc95001b720 legacy=0x7fc9500168c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:04.204 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:04.204+0000 7fc971f66640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 msgr2=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:04.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:04.204+0000 7fc971765640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950009eb0 legacy=0x7fc9500161b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:04.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:04.204+0000 7fc971f66640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 0x7fc950011f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:05.727 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:05:05.731 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:05:05.802 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:05.802+0000 7fc95effd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 msgr2=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:05.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:05.802+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012fb0 0x7fc950011f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:05:05.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:05.802+0000 7fc95effd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950009eb0 legacy=0x7fc9500161b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:05.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:05.802+0000 7fc95effd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc95001b720 legacy=0x7fc9500168c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:05.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:05.802+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 0x7fc950012a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:05.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:05.802+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:05.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:05.802+0000 7fc95effd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc950010bc0 2023-11-09T04:05:05.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:05.802+0000 7fc95effd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95000af60 con 0x7fc950011600 2023-11-09T04:05:05.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:05.802+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 msgr2=0x7fc950012a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:05.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:05.802+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 0x7fc950012a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:05.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:05.802+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:05.807 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:05.802+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:06.002 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:06.003+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 msgr2=0x7fc950012a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:06.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:06.003+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 0x7fc950012a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:06.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:06.003+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:06.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:06.003+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:06.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:06.404+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 msgr2=0x7fc950012a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:06.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:06.404+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 0x7fc950012a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:06.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:06.404+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:06.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:06.404+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:07.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:07.205+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 msgr2=0x7fc950012a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:07.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:07.205+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 0x7fc950012a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:07.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:07.205+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:07.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:07.205+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:08.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:08.802+0000 7fc95effd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:08.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:08.802+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:05:08.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:08.802+0000 7fc95effd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 msgr2=0x7fc950012a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:08.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:08.802+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc950010bc0 0x7fc950012a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:05:08.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:08.802+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc95001db10 0x7fc95001aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:08.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:08.802+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:08.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:08.802+0000 7fc971f66640 1 -- reap_dead start 2023-11-09T04:05:08.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:08.802+0000 7fc95effd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c071500 con 0x7fc950012fb0 2023-11-09T04:05:08.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:08.802+0000 7fc95effd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc95001db10 2023-11-09T04:05:08.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:08.802+0000 7fc95effd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95000af60 con 0x7fc950011600 2023-11-09T04:05:08.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:08.802+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc95001db10 msgr2=0x7fc95001aff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:08.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:08.802+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc95001db10 0x7fc95001aff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:08.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:08.802+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:08.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:08.802+0000 7fc971f66640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950012fb0 legacy=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:08.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:08.802+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:09.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:09.003+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc95001db10 msgr2=0x7fc95001aff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:09.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:09.003+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc95001db10 0x7fc95001aff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:09.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:09.003+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:09.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:09.003+0000 7fc971f66640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950012fb0 legacy=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:09.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:09.003+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:09.403 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:09.404+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc95001db10 msgr2=0x7fc95001aff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:09.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:09.404+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc95001db10 0x7fc95001aff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:09.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:09.404+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:09.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:09.404+0000 7fc971f66640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950012fb0 legacy=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:09.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:09.404+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:10.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:10.205+0000 7fc971765640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc95001db10 msgr2=0x7fc95001aff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:10.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:10.205+0000 7fc971765640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc95001db10 0x7fc95001aff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:10.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:10.205+0000 7fc971f66640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950012fb0 legacy=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:10.205 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:10.205+0000 7fc970f64640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:10.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:10.205+0000 7fc970f64640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:11.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:11.803+0000 7fc95effd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 msgr2=0x7fc9500119e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:11.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:11.803+0000 7fc95effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc950011600 0x7fc9500119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:05:11.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:11.803+0000 7fc95effd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fc95001db10 msgr2=0x7fc95001aff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:11.803 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:11.803+0000 7fc95effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc95001db10 0x7fc95001aff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:05:11.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:11.803+0000 7fc95effd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc950012fb0 legacy=0x7fc950011f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:11.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:11.803+0000 7fc95effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012a50 0x7fc950009eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:11.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:11.803+0000 7fc95effd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c15dae0 con 0x7fc95001b530 2023-11-09T04:05:11.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:11.803+0000 7fc95effd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c070c60 con 0x7fc950012a50 2023-11-09T04:05:11.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:11.803+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc95001b530 legacy=0x7fc9500141a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:11.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:11.803+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012a50 msgr2=0x7fc950009eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:11.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:11.803+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012a50 0x7fc950009eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:12.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:12.004+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc95001b530 legacy=0x7fc9500141a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:12.003 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:12.004+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012a50 msgr2=0x7fc950009eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:12.004 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:12.004+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012a50 0x7fc950009eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:12.404 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:12.405+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc95001b530 legacy=0x7fc9500141a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:12.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:12.405+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012a50 msgr2=0x7fc950009eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:12.405 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:12.405+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012a50 0x7fc950009eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:13.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:13.206+0000 7fc970f64640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fc95001b530 legacy=0x7fc9500141a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:13.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:13.206+0000 7fc971765640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc950012a50 msgr2=0x7fc950009eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:13.206 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:13.206+0000 7fc971765640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc950012a50 0x7fc950009eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:14.717 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-11-09T04:05:14.719 INFO:tasks.cephfs_test_runner:check snaptable transcation ... ERROR 2023-11-09T04:05:14.719 INFO:tasks.cephfs_test_runner:ERROR 2023-11-09T04:05:14.720 INFO:tasks.cephfs_test_runner: 2023-11-09T04:05:14.721 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-09T04:05:14.721 INFO:tasks.cephfs_test_runner:ERROR: test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots) 2023-11-09T04:05:14.721 INFO:tasks.cephfs_test_runner:check snaptable transcation 2023-11-09T04:05:14.721 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-09T04:05:14.722 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-09T04:05:14.722 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/cephfs/test_snapshots.py", line 224, in test_kill_mdstable 2023-11-09T04:05:14.722 INFO:tasks.cephfs_test_runner: self.assertEqual(len(self._get_pending_snap_update(rank=0)), 1) 2023-11-09T04:05:14.723 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/cephfs/test_snapshots.py", line 38, in _get_pending_snap_update 2023-11-09T04:05:14.723 INFO:tasks.cephfs_test_runner: return self._get_snapserver_dump(rank,status=status)["pending_update"] 2023-11-09T04:05:14.723 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/cephfs/test_snapshots.py", line 29, in _get_snapserver_dump 2023-11-09T04:05:14.723 INFO:tasks.cephfs_test_runner: return self.fs.rank_asok(["dump", "snaps", "--server"], rank=rank, status=status) 2023-11-09T04:05:14.724 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/cephfs/filesystem.py", line 1295, in rank_asok 2023-11-09T04:05:14.724 INFO:tasks.cephfs_test_runner: info = self.get_rank(rank=rank, status=status) 2023-11-09T04:05:14.724 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/cephfs/filesystem.py", line 1101, in get_rank 2023-11-09T04:05:14.725 INFO:tasks.cephfs_test_runner: status = self.getinfo() 2023-11-09T04:05:14.725 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/cephfs/filesystem.py", line 565, in getinfo 2023-11-09T04:05:14.725 INFO:tasks.cephfs_test_runner: status = self.status() 2023-11-09T04:05:14.725 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/cephfs/filesystem.py", line 396, in status 2023-11-09T04:05:14.726 INFO:tasks.cephfs_test_runner: return FSStatus(self.mon_manager, epoch) 2023-11-09T04:05:14.726 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/cephfs/filesystem.py", line 88, in __init__ 2023-11-09T04:05:14.726 INFO:tasks.cephfs_test_runner: self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) 2023-11-09T04:05:14.726 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2023-11-09T04:05:14.727 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2023-11-09T04:05:14.727 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2023-11-09T04:05:14.727 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2023-11-09T04:05:14.728 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd 2023-11-09T04:05:14.728 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-11-09T04:05:14.728 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/remote.py", line 522, in run 2023-11-09T04:05:14.728 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-11-09T04:05:14.729 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 455, in run 2023-11-09T04:05:14.729 INFO:tasks.cephfs_test_runner: r.wait() 2023-11-09T04:05:14.729 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait 2023-11-09T04:05:14.729 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-11-09T04:05:14.730 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-11-09T04:05:14.730 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-11-09T04:05:14.730 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi076 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' 2023-11-09T04:05:14.730 INFO:tasks.cephfs_test_runner: 2023-11-09T04:05:14.731 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-09T04:05:14.731 INFO:tasks.cephfs_test_runner:ERROR: test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots) 2023-11-09T04:05:14.731 INFO:tasks.cephfs_test_runner:check snaptable transcation 2023-11-09T04:05:14.732 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-09T04:05:14.732 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-09T04:05:14.732 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/cephfs/cephfs_test_case.py", line 208, in tearDown 2023-11-09T04:05:14.732 INFO:tasks.cephfs_test_runner: self.mds_cluster.delete_all_filesystems() 2023-11-09T04:05:14.733 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/cephfs/filesystem.py", line 493, in delete_all_filesystems 2023-11-09T04:05:14.733 INFO:tasks.cephfs_test_runner: for fs in self.status().get_filesystems(): 2023-11-09T04:05:14.733 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/cephfs/filesystem.py", line 396, in status 2023-11-09T04:05:14.734 INFO:tasks.cephfs_test_runner: return FSStatus(self.mon_manager, epoch) 2023-11-09T04:05:14.734 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/cephfs/filesystem.py", line 88, in __init__ 2023-11-09T04:05:14.734 INFO:tasks.cephfs_test_runner: self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) 2023-11-09T04:05:14.734 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2023-11-09T04:05:14.735 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2023-11-09T04:05:14.735 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2023-11-09T04:05:14.735 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2023-11-09T04:05:14.735 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd 2023-11-09T04:05:14.736 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-11-09T04:05:14.736 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/remote.py", line 522, in run 2023-11-09T04:05:14.736 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-11-09T04:05:14.736 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 455, in run 2023-11-09T04:05:14.737 INFO:tasks.cephfs_test_runner: r.wait() 2023-11-09T04:05:14.737 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait 2023-11-09T04:05:14.737 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-11-09T04:05:14.738 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-11-09T04:05:14.738 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-11-09T04:05:14.738 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi076 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' 2023-11-09T04:05:14.738 INFO:tasks.cephfs_test_runner: 2023-11-09T04:05:14.739 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-09T04:05:14.739 INFO:tasks.cephfs_test_runner:Ran 5 tests in 1135.069s 2023-11-09T04:05:14.739 INFO:tasks.cephfs_test_runner: 2023-11-09T04:05:14.739 INFO:tasks.cephfs_test_runner:FAILED (errors=2) 2023-11-09T04:05:14.740 INFO:tasks.cephfs_test_runner: 2023-11-09T04:05:14.740 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-09T04:05:14.740 INFO:tasks.cephfs_test_runner:ERROR: test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots) 2023-11-09T04:05:14.741 INFO:tasks.cephfs_test_runner:check snaptable transcation 2023-11-09T04:05:14.741 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-09T04:05:14.741 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-09T04:05:14.741 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/cephfs/test_snapshots.py", line 224, in test_kill_mdstable 2023-11-09T04:05:14.742 INFO:tasks.cephfs_test_runner: self.assertEqual(len(self._get_pending_snap_update(rank=0)), 1) 2023-11-09T04:05:14.742 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/cephfs/test_snapshots.py", line 38, in _get_pending_snap_update 2023-11-09T04:05:14.742 INFO:tasks.cephfs_test_runner: return self._get_snapserver_dump(rank,status=status)["pending_update"] 2023-11-09T04:05:14.742 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/cephfs/test_snapshots.py", line 29, in _get_snapserver_dump 2023-11-09T04:05:14.743 INFO:tasks.cephfs_test_runner: return self.fs.rank_asok(["dump", "snaps", "--server"], rank=rank, status=status) 2023-11-09T04:05:14.743 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/cephfs/filesystem.py", line 1295, in rank_asok 2023-11-09T04:05:14.743 INFO:tasks.cephfs_test_runner: info = self.get_rank(rank=rank, status=status) 2023-11-09T04:05:14.744 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/cephfs/filesystem.py", line 1101, in get_rank 2023-11-09T04:05:14.744 INFO:tasks.cephfs_test_runner: status = self.getinfo() 2023-11-09T04:05:14.744 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/cephfs/filesystem.py", line 565, in getinfo 2023-11-09T04:05:14.744 INFO:tasks.cephfs_test_runner: status = self.status() 2023-11-09T04:05:14.745 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/cephfs/filesystem.py", line 396, in status 2023-11-09T04:05:14.745 INFO:tasks.cephfs_test_runner: return FSStatus(self.mon_manager, epoch) 2023-11-09T04:05:14.745 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/cephfs/filesystem.py", line 88, in __init__ 2023-11-09T04:05:14.745 INFO:tasks.cephfs_test_runner: self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) 2023-11-09T04:05:14.746 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2023-11-09T04:05:14.746 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2023-11-09T04:05:14.746 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2023-11-09T04:05:14.746 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2023-11-09T04:05:14.747 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd 2023-11-09T04:05:14.747 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-11-09T04:05:14.747 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/remote.py", line 522, in run 2023-11-09T04:05:14.747 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-11-09T04:05:14.748 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 455, in run 2023-11-09T04:05:14.748 INFO:tasks.cephfs_test_runner: r.wait() 2023-11-09T04:05:14.748 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait 2023-11-09T04:05:14.748 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-11-09T04:05:14.749 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-11-09T04:05:14.749 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-11-09T04:05:14.749 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi076 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' 2023-11-09T04:05:14.749 INFO:tasks.cephfs_test_runner: 2023-11-09T04:05:14.750 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-09T04:05:14.750 INFO:tasks.cephfs_test_runner:ERROR: test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots) 2023-11-09T04:05:14.750 INFO:tasks.cephfs_test_runner:check snaptable transcation 2023-11-09T04:05:14.750 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-09T04:05:14.751 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-09T04:05:14.751 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/cephfs/cephfs_test_case.py", line 208, in tearDown 2023-11-09T04:05:14.751 INFO:tasks.cephfs_test_runner: self.mds_cluster.delete_all_filesystems() 2023-11-09T04:05:14.752 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/cephfs/filesystem.py", line 493, in delete_all_filesystems 2023-11-09T04:05:14.752 INFO:tasks.cephfs_test_runner: for fs in self.status().get_filesystems(): 2023-11-09T04:05:14.752 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/cephfs/filesystem.py", line 396, in status 2023-11-09T04:05:14.752 INFO:tasks.cephfs_test_runner: return FSStatus(self.mon_manager, epoch) 2023-11-09T04:05:14.752 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/cephfs/filesystem.py", line 88, in __init__ 2023-11-09T04:05:14.753 INFO:tasks.cephfs_test_runner: self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) 2023-11-09T04:05:14.753 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2023-11-09T04:05:14.753 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2023-11-09T04:05:14.754 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2023-11-09T04:05:14.754 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2023-11-09T04:05:14.754 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd 2023-11-09T04:05:14.754 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-11-09T04:05:14.755 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/remote.py", line 522, in run 2023-11-09T04:05:14.755 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-11-09T04:05:14.755 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 455, in run 2023-11-09T04:05:14.755 INFO:tasks.cephfs_test_runner: r.wait() 2023-11-09T04:05:14.756 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait 2023-11-09T04:05:14.756 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-11-09T04:05:14.756 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-11-09T04:05:14.756 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-11-09T04:05:14.757 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi076 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' 2023-11-09T04:05:14.757 INFO:tasks.cephfs_test_runner: 2023-11-09T04:05:14.757 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots), test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots) 2023-11-09T04:05:14.827 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=5f99bffc4cc44425a7516634a1672ce5 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots), test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots) 2023-11-09T04:05:14.829 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-09T04:05:14.838 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-11-09T04:05:14.867 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-11-09T04:05:14.867 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T04:05:14.868 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T04:05:14.899 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056381,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14761 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248868k,nr_inodes=812217,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-09T04:05:14.899 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-09T04:05:14.900 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T04:05:14.957 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056381,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14761 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248868k,nr_inodes=812217,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497740k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-09T04:05:14.958 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T04:05:14.958 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T04:05:14.976 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2440 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-09T04:05:14.976 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2023-11-09T04:05:14.977 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-09T04:05:15.033 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2440 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248860k,nr_inodes=812215,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497724k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-09T04:05:15.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T04:05:15.034 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2023-11-09T04:05:15.051 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-09T04:05:15.052 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-09T04:05:15.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T04:05:15.053 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-09T04:05:15.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T04:05:15.122 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-09T04:05:15.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T04:05:15.191 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link delete ceph-brx 2023-11-09T04:05:15.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T04:05:15.292 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2023-11-09T04:05:15.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-09T04:05:15.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-09T04:05:15.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T04:05:15.311 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-09T04:05:15.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T04:05:15.383 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-09T04:05:15.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-09T04:05:15.453 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link delete ceph-brx 2023-11-09T04:05:15.567 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-09T04:05:15.577 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-09T04:05:15.578 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-09T04:05:15.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:15.682+0000 7f6b5f47c640 1 Processor -- start 2023-11-09T04:05:15.682 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:15.683+0000 7f6b5f47c640 1 -- start start 2023-11-09T04:05:15.683 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:15.683+0000 7f6b5f47c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b58070700 0x7f6b58164b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:15.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:15.683+0000 7f6b5f47c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b58071500 con 0x7f6b580b0c40 2023-11-09T04:05:15.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:15.683+0000 7f6b5f47c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5815dae0 con 0x7f6b58159e40 2023-11-09T04:05:15.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:15.683+0000 7f6b5f47c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b58070700 2023-11-09T04:05:15.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:15.684+0000 7f6b5ca17640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b580b0c40 legacy=0x7f6b5806fff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:15.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:15.684+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b58159e40 legacy=0x7f6b580b0530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:15.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:15.684+0000 7f6b5da19640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b58070700 msgr2=0x7f6b58164b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:15.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:15.684+0000 7f6b5da19640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b58070700 0x7f6b58164b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:15.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:15.885+0000 7f6b5ca17640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b580b0c40 legacy=0x7f6b5806fff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:15.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:15.885+0000 7f6b5da19640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b58070700 msgr2=0x7f6b58164b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:15.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:15.885+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b58159e40 legacy=0x7f6b580b0530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:15.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:15.885+0000 7f6b5da19640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b58070700 0x7f6b58164b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:16.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:16.286+0000 7f6b5ca17640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b580b0c40 legacy=0x7f6b5806fff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:16.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:16.286+0000 7f6b5da19640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b58070700 msgr2=0x7f6b58164b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:16.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:16.286+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b58159e40 legacy=0x7f6b580b0530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:16.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:16.286+0000 7f6b5da19640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b58070700 0x7f6b58164b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:17.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:17.087+0000 7f6b5ca17640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b580b0c40 legacy=0x7f6b5806fff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:17.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:17.087+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b58159e40 legacy=0x7f6b580b0530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:17.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:17.087+0000 7f6b5da19640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b58070700 msgr2=0x7f6b58164b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:17.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:17.087+0000 7f6b5da19640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b58070700 0x7f6b58164b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:18.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:18.683+0000 7f6b4affd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b58070700 msgr2=0x7f6b58164b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:18.684 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:18.684+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b58070700 0x7f6b58164b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:05:18.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:18.684+0000 7f6b4affd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b58159e40 legacy=0x7f6b580b0530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:18.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:18.684+0000 7f6b4affd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b580b0c40 legacy=0x7f6b5806fff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:18.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:18.684+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c005b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:18.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:18.684+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0061a0 0x7f6b3c008660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:18.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:18.684+0000 7f6b4affd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c0061a0 2023-11-09T04:05:18.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:18.684+0000 7f6b4affd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b3c008d20 con 0x7f6b3c0036d0 2023-11-09T04:05:18.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:18.684+0000 7f6b5ca17640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0061a0 msgr2=0x7f6b3c008660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:18.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:18.684+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0061a0 0x7f6b3c008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:18.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:18.684+0000 7f6b5d218640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 msgr2=0x7f6b3c005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:18.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:18.684+0000 7f6b5d218640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:18.884 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:18.885+0000 7f6b5ca17640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0061a0 msgr2=0x7f6b3c008660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:18.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:18.885+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0061a0 0x7f6b3c008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:18.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:18.885+0000 7f6b5d218640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 msgr2=0x7f6b3c005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:18.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:18.885+0000 7f6b5d218640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:19.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:19.286+0000 7f6b5ca17640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0061a0 msgr2=0x7f6b3c008660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:19.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:19.286+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0061a0 0x7f6b3c008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:19.285 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:19.286+0000 7f6b5d218640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 msgr2=0x7f6b3c005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:19.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:19.286+0000 7f6b5d218640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:20.086 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:20.087+0000 7f6b5ca17640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0061a0 msgr2=0x7f6b3c008660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:20.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:20.087+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0061a0 0x7f6b3c008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:20.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:20.087+0000 7f6b5d218640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 msgr2=0x7f6b3c005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:20.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:20.087+0000 7f6b5d218640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:21.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:21.683+0000 7f6b4affd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 msgr2=0x7f6b3c005b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:21.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:21.683+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:05:21.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:21.683+0000 7f6b4affd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0061a0 msgr2=0x7f6b3c008660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:21.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:21.683+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0061a0 0x7f6b3c008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:05:21.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:21.683+0000 7f6b5da19640 1 -- reap_dead start 2023-11-09T04:05:21.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:21.683+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c005b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:21.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:21.683+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c006650 0x7f6b3c008f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:21.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:21.683+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0094d0 0x7f6b3c00b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:21.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:21.683+0000 7f6b4affd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c0094d0 2023-11-09T04:05:21.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:21.683+0000 7f6b4affd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b3c008d20 con 0x7f6b3c0036d0 2023-11-09T04:05:21.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:21.683+0000 7f6b4affd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c00bf60 con 0x7f6b3c006650 2023-11-09T04:05:21.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:21.683+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0094d0 msgr2=0x7f6b3c00b8c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:21.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:21.683+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0094d0 0x7f6b3c00b8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:21.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:21.683+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c006650 msgr2=0x7f6b3c008f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:21.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:21.683+0000 7f6b5d218640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 msgr2=0x7f6b3c005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:21.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:21.683+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c006650 0x7f6b3c008f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:21.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:21.683+0000 7f6b5d218640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:21.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:21.884+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0094d0 msgr2=0x7f6b3c00b8c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:21.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:21.884+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0094d0 0x7f6b3c00b8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:21.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:21.884+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c006650 msgr2=0x7f6b3c008f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:21.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:21.884+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c006650 0x7f6b3c008f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:21.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:21.884+0000 7f6b5d218640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 msgr2=0x7f6b3c005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:21.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:21.884+0000 7f6b5d218640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:22.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:22.285+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0094d0 msgr2=0x7f6b3c00b8c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:22.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:22.285+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0094d0 0x7f6b3c00b8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:22.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:22.285+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c006650 msgr2=0x7f6b3c008f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:22.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:22.285+0000 7f6b5d218640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 msgr2=0x7f6b3c005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:22.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:22.285+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c006650 0x7f6b3c008f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:22.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:22.285+0000 7f6b5d218640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:23.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:23.086+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0094d0 msgr2=0x7f6b3c00b8c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:23.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:23.086+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0094d0 0x7f6b3c00b8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:23.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:23.086+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c006650 msgr2=0x7f6b3c008f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:23.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:23.086+0000 7f6b5d218640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 msgr2=0x7f6b3c005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:23.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:23.086+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c006650 0x7f6b3c008f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:23.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:23.086+0000 7f6b5d218640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:24.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:24.684+0000 7f6b4affd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c006650 msgr2=0x7f6b3c008f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:24.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:24.684+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c006650 0x7f6b3c008f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:05:24.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:24.684+0000 7f6b4affd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 msgr2=0x7f6b3c005b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:24.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:24.684+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:05:24.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:24.684+0000 7f6b4affd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0094d0 msgr2=0x7f6b3c00b8c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:24.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:24.684+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0094d0 0x7f6b3c00b8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:05:24.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:24.684+0000 7f6b4affd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b58070f40 con 0x7f6b3c00ff00 2023-11-09T04:05:24.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:24.684+0000 7f6b4affd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b58071500 con 0x7f6b3c00c260 2023-11-09T04:05:24.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:24.684+0000 7f6b5ca17640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c00ff00 legacy=0x7f6b3c012320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:24.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:24.684+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00c260 legacy=0x7f6b3c00e680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:24.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:24.885+0000 7f6b5ca17640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c00ff00 legacy=0x7f6b3c012320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:24.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:24.885+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00c260 legacy=0x7f6b3c00e680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:25.286 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:25.286+0000 7f6b5ca17640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c00ff00 legacy=0x7f6b3c012320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:25.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:25.286+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00c260 legacy=0x7f6b3c00e680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:26.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:26.087+0000 7f6b5ca17640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c00ff00 legacy=0x7f6b3c012320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:26.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:26.087+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00c260 legacy=0x7f6b3c00e680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:27.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:27.684+0000 7f6b4affd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00c260 legacy=0x7f6b3c00e680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:27.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:27.684+0000 7f6b4affd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c00ff00 legacy=0x7f6b3c012320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:27.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:27.684+0000 7f6b5da19640 1 -- reap_dead start 2023-11-09T04:05:27.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:27.684+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0036d0 0x7f6b3c008f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:27.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:27.684+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c008660 0x7f6b3c005b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:27.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:27.684+0000 7f6b4affd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5815dae0 con 0x7f6b3c014bc0 2023-11-09T04:05:27.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:27.684+0000 7f6b4affd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c008660 2023-11-09T04:05:27.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:27.684+0000 7f6b4affd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c008d20 con 0x7f6b3c0036d0 2023-11-09T04:05:27.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:27.684+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c008660 msgr2=0x7f6b3c005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:27.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:27.684+0000 7f6b5ca17640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c014bc0 legacy=0x7f6b3c014fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:27.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:27.684+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c008660 0x7f6b3c005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:27.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:27.684+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0036d0 msgr2=0x7f6b3c008f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:27.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:27.684+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0036d0 0x7f6b3c008f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:27.885 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:27.885+0000 7f6b5ca17640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c014bc0 legacy=0x7f6b3c014fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:27.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:27.885+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c008660 msgr2=0x7f6b3c005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:27.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:27.885+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c008660 0x7f6b3c005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:27.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:27.885+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0036d0 msgr2=0x7f6b3c008f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:27.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:27.885+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0036d0 0x7f6b3c008f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:28.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:28.286+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c008660 msgr2=0x7f6b3c005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:28.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:28.286+0000 7f6b5ca17640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c014bc0 legacy=0x7f6b3c014fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:28.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:28.286+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c008660 0x7f6b3c005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:28.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:28.286+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0036d0 msgr2=0x7f6b3c008f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:28.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:28.286+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0036d0 0x7f6b3c008f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:29.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:29.087+0000 7f6b5ca17640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c014bc0 legacy=0x7f6b3c014fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:29.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:29.087+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c008660 msgr2=0x7f6b3c005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:29.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:29.087+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c008660 0x7f6b3c005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:29.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:29.087+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0036d0 msgr2=0x7f6b3c008f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:29.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:29.087+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0036d0 0x7f6b3c008f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:30.685 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:30.685+0000 7f6b4affd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0036d0 msgr2=0x7f6b3c008f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:30.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:30.685+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0036d0 0x7f6b3c008f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:05:30.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:30.685+0000 7f6b4affd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c008660 msgr2=0x7f6b3c005b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:30.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:30.685+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c008660 0x7f6b3c005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:05:30.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:30.685+0000 7f6b4affd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c014bc0 legacy=0x7f6b3c014fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:30.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:30.685+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011a00 0x7f6b3c00e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:30.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:30.685+0000 7f6b4affd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b58070f40 con 0x7f6b3c0098b0 2023-11-09T04:05:30.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:30.685+0000 7f6b4affd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c011a00 2023-11-09T04:05:30.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:30.685+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0098b0 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:30.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:30.685+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011a00 msgr2=0x7f6b3c00e650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:30.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:30.685+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011a00 0x7f6b3c00e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:30.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:30.886+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0098b0 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:30.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:30.886+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011a00 msgr2=0x7f6b3c00e650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:30.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:30.886+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011a00 0x7f6b3c00e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:31.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:31.287+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011a00 msgr2=0x7f6b3c00e650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:31.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:31.287+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0098b0 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:31.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:31.287+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011a00 0x7f6b3c00e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:32.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:32.088+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0098b0 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:32.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:32.088+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011a00 msgr2=0x7f6b3c00e650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:32.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:32.088+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011a00 0x7f6b3c00e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:33.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:33.685+0000 7f6b4affd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011a00 msgr2=0x7f6b3c00e650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:33.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:33.685+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011a00 0x7f6b3c00e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:05:33.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:33.685+0000 7f6b4affd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0098b0 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:33.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:33.685+0000 7f6b5da19640 1 -- reap_dead start 2023-11-09T04:05:33.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:33.685+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 0x7f6b3c0121f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:33.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:33.685+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c008f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:33.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:33.685+0000 7f6b4affd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5815dae0 con 0x7f6b3c0098b0 2023-11-09T04:05:33.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:33.685+0000 7f6b4affd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c0036d0 2023-11-09T04:05:33.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:33.685+0000 7f6b4affd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b3c008d20 con 0x7f6b3c011e10 2023-11-09T04:05:33.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:33.685+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0036d0 msgr2=0x7f6b3c008f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:33.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:33.685+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c008f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:33.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:33.685+0000 7f6b5d218640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 msgr2=0x7f6b3c0121f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:33.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:33.685+0000 7f6b5ca17640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0098b0 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:33.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:33.685+0000 7f6b5d218640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 0x7f6b3c0121f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:33.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:33.886+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0036d0 msgr2=0x7f6b3c008f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:33.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:33.886+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c008f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:33.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:33.886+0000 7f6b5ca17640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0098b0 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:33.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:33.886+0000 7f6b5d218640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 msgr2=0x7f6b3c0121f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:33.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:33.886+0000 7f6b5d218640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 0x7f6b3c0121f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:34.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:34.287+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0036d0 msgr2=0x7f6b3c008f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:34.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:34.287+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c008f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:34.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:34.287+0000 7f6b5ca17640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0098b0 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:34.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:34.287+0000 7f6b5d218640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 msgr2=0x7f6b3c0121f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:34.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:34.287+0000 7f6b5d218640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 0x7f6b3c0121f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:35.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:35.088+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0036d0 msgr2=0x7f6b3c008f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:35.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:35.088+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c008f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:35.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:35.088+0000 7f6b5d218640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 msgr2=0x7f6b3c0121f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:35.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:35.088+0000 7f6b5ca17640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0098b0 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:35.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:35.088+0000 7f6b5d218640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 0x7f6b3c0121f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:35.797 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:05:35.801 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:05:36.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:36.686+0000 7f6b4affd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 msgr2=0x7f6b3c0121f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:36.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:36.686+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 0x7f6b3c0121f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:05:36.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:36.686+0000 7f6b4affd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0036d0 msgr2=0x7f6b3c008f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:36.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:36.686+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c008f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:05:36.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:36.686+0000 7f6b4affd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0098b0 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:36.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:36.686+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014bc0 0x7f6b3c014fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:36.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:36.686+0000 7f6b4affd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b58070f40 con 0x7f6b3c008640 2023-11-09T04:05:36.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:36.686+0000 7f6b4affd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c014bc0 2023-11-09T04:05:36.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:36.686+0000 7f6b5ca17640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008640 legacy=0x7f6b3c00ed90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:36.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:36.686+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014bc0 msgr2=0x7f6b3c014fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:36.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:36.686+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014bc0 0x7f6b3c014fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:36.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:36.887+0000 7f6b5ca17640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008640 legacy=0x7f6b3c00ed90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:36.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:36.887+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014bc0 msgr2=0x7f6b3c014fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:36.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:36.887+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014bc0 0x7f6b3c014fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:37.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:37.288+0000 7f6b5ca17640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008640 legacy=0x7f6b3c00ed90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:37.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:37.288+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014bc0 msgr2=0x7f6b3c014fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:37.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:37.288+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014bc0 0x7f6b3c014fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:38.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:38.089+0000 7f6b5ca17640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008640 legacy=0x7f6b3c00ed90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:38.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:38.089+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014bc0 msgr2=0x7f6b3c014fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:38.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:38.089+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014bc0 0x7f6b3c014fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:39.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:39.686+0000 7f6b4affd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014bc0 msgr2=0x7f6b3c014fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:39.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:39.686+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014bc0 0x7f6b3c014fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:05:39.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:39.686+0000 7f6b4affd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008640 legacy=0x7f6b3c00ed90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:39.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:39.686+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c008f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:39.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:39.686+0000 7f6b5da19640 1 -- reap_dead start 2023-11-09T04:05:39.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:39.686+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 0x7f6b3c0121f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:39.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:39.686+0000 7f6b4affd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5815dae0 con 0x7f6b3c0098b0 2023-11-09T04:05:39.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:39.686+0000 7f6b4affd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c0036d0 2023-11-09T04:05:39.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:39.686+0000 7f6b4affd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b3c008d20 con 0x7f6b3c011e10 2023-11-09T04:05:39.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:39.686+0000 7f6b5d218640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0036d0 msgr2=0x7f6b3c008f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:39.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:39.686+0000 7f6b5d218640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c008f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:39.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:39.686+0000 7f6b5da19640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0098b0 legacy=0x7f6b3c00f4a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:39.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:39.686+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 msgr2=0x7f6b3c0121f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:39.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:39.686+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 0x7f6b3c0121f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:39.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:39.887+0000 7f6b5d218640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0036d0 msgr2=0x7f6b3c008f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:39.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:39.887+0000 7f6b5d218640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c008f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:39.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:39.887+0000 7f6b5da19640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0098b0 legacy=0x7f6b3c00f4a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:39.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:39.887+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 msgr2=0x7f6b3c0121f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:39.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:39.887+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 0x7f6b3c0121f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:40.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:40.288+0000 7f6b5d218640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0036d0 msgr2=0x7f6b3c008f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:40.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:40.288+0000 7f6b5d218640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c008f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:40.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:40.288+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 msgr2=0x7f6b3c0121f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:40.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:40.288+0000 7f6b5da19640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0098b0 legacy=0x7f6b3c00f4a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:40.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:40.288+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 0x7f6b3c0121f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:41.088 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:41.089+0000 7f6b5d218640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0036d0 msgr2=0x7f6b3c008f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:41.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:41.089+0000 7f6b5d218640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c008f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:41.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:41.089+0000 7f6b5da19640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0098b0 legacy=0x7f6b3c00f4a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:41.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:41.089+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 msgr2=0x7f6b3c0121f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:41.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:41.089+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 0x7f6b3c0121f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:42.686 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:42.686+0000 7f6b4affd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 msgr2=0x7f6b3c0121f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:42.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:42.686+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 0x7f6b3c0121f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:05:42.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:42.686+0000 7f6b4affd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0036d0 msgr2=0x7f6b3c008f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:42.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:42.686+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c008f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:05:42.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:42.686+0000 7f6b4affd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0098b0 legacy=0x7f6b3c00f4a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:42.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:42.686+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014bc0 0x7f6b3c00e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:42.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:42.686+0000 7f6b4affd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b58070f40 con 0x7f6b3c00ed90 2023-11-09T04:05:42.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:42.686+0000 7f6b4affd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c014bc0 2023-11-09T04:05:42.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:42.686+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c00ed90 legacy=0x7f6b3c0112f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:42.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:42.686+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014bc0 msgr2=0x7f6b3c00e650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:42.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:42.686+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014bc0 0x7f6b3c00e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:42.886 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:42.887+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c00ed90 legacy=0x7f6b3c0112f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:42.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:42.887+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014bc0 msgr2=0x7f6b3c00e650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:42.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:42.887+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014bc0 0x7f6b3c00e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:43.287 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:43.288+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c00ed90 legacy=0x7f6b3c0112f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:43.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:43.288+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014bc0 msgr2=0x7f6b3c00e650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:43.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:43.288+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014bc0 0x7f6b3c00e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:44.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:44.089+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c00ed90 legacy=0x7f6b3c0112f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:44.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:44.089+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014bc0 msgr2=0x7f6b3c00e650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:44.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:44.089+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014bc0 0x7f6b3c00e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:45.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:45.687+0000 7f6b4affd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014bc0 msgr2=0x7f6b3c00e650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:45.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:45.687+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014bc0 0x7f6b3c00e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:05:45.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:45.687+0000 7f6b4affd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c00ed90 legacy=0x7f6b3c0112f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:45.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:45.687+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 0x7f6b3c0121f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:45.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:45.687+0000 7f6b5da19640 1 -- reap_dead start 2023-11-09T04:05:45.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:45.687+0000 7f6b4affd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5815dae0 con 0x7f6b3c014fd0 2023-11-09T04:05:45.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:45.687+0000 7f6b4affd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b58070f40 con 0x7f6b3c0036d0 2023-11-09T04:05:45.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:45.687+0000 7f6b4affd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c011e10 2023-11-09T04:05:45.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:45.687+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c014fd0 legacy=0x7f6b3c007600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:45.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:45.687+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 msgr2=0x7f6b3c0121f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:45.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:45.687+0000 7f6b5da19640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0036d0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:45.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:45.687+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 0x7f6b3c0121f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:45.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:45.888+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c014fd0 legacy=0x7f6b3c007600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:45.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:45.888+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 msgr2=0x7f6b3c0121f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:45.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:45.888+0000 7f6b5da19640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0036d0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:45.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:45.888+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 0x7f6b3c0121f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:46.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:46.289+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c014fd0 legacy=0x7f6b3c007600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:46.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:46.289+0000 7f6b5da19640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0036d0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:46.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:46.289+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 msgr2=0x7f6b3c0121f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:46.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:46.289+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 0x7f6b3c0121f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:47.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:47.090+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c014fd0 legacy=0x7f6b3c007600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:47.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:47.090+0000 7f6b5da19640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0036d0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:47.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:47.090+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 msgr2=0x7f6b3c0121f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:47.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:47.090+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 0x7f6b3c0121f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:48.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:48.687+0000 7f6b4affd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 msgr2=0x7f6b3c0121f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:48.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:48.687+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c011e10 0x7f6b3c0121f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:05:48.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:48.687+0000 7f6b4affd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0036d0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:48.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:48.687+0000 7f6b4affd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c014fd0 legacy=0x7f6b3c007600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:48.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:48.687+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0098b0 0x7f6b3c008f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:48.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:48.687+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c014bc0 0x7f6b3c005b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:48.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:48.687+0000 7f6b4affd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c014bc0 2023-11-09T04:05:48.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:48.687+0000 7f6b4affd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c008d20 con 0x7f6b3c0098b0 2023-11-09T04:05:48.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:48.687+0000 7f6b5ca17640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c014bc0 msgr2=0x7f6b3c005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:48.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:48.687+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c014bc0 0x7f6b3c005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:48.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:48.687+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0098b0 msgr2=0x7f6b3c008f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:48.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:48.687+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0098b0 0x7f6b3c008f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:48.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:48.888+0000 7f6b5ca17640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c014bc0 msgr2=0x7f6b3c005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:48.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:48.888+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c014bc0 0x7f6b3c005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:48.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:48.888+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0098b0 msgr2=0x7f6b3c008f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:48.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:48.888+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0098b0 0x7f6b3c008f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:49.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:49.289+0000 7f6b5ca17640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c014bc0 msgr2=0x7f6b3c005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:49.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:49.289+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c014bc0 0x7f6b3c005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:49.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:49.289+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0098b0 msgr2=0x7f6b3c008f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:49.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:49.289+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0098b0 0x7f6b3c008f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:50.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:50.090+0000 7f6b5ca17640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c014bc0 msgr2=0x7f6b3c005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:50.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:50.090+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c014bc0 0x7f6b3c005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:50.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:50.090+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0098b0 msgr2=0x7f6b3c008f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:50.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:50.090+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0098b0 0x7f6b3c008f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:51.687 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:51.687+0000 7f6b4affd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0098b0 msgr2=0x7f6b3c008f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:51.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:51.687+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0098b0 0x7f6b3c008f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:05:51.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:51.687+0000 7f6b4affd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c014bc0 msgr2=0x7f6b3c005b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:51.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:51.687+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c014bc0 0x7f6b3c005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:05:51.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:51.687+0000 7f6b5da19640 1 -- reap_dead start 2023-11-09T04:05:51.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:51.687+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 0x7f6b3c00e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:51.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:51.687+0000 7f6b4affd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b58071500 con 0x7f6b3c0098b0 2023-11-09T04:05:51.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:51.687+0000 7f6b4affd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5815dae0 con 0x7f6b3c0036d0 2023-11-09T04:05:51.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:51.687+0000 7f6b5d218640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 msgr2=0x7f6b3c00e650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:51.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:51.687+0000 7f6b4affd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c00f880 2023-11-09T04:05:51.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:51.687+0000 7f6b5da19640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0098b0 legacy=0x7f6b3c007600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:51.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:51.687+0000 7f6b5d218640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 0x7f6b3c00e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:51.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:51.687+0000 7f6b5ca17640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0036d0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:51.887 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:51.888+0000 7f6b5da19640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0098b0 legacy=0x7f6b3c007600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:51.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:51.888+0000 7f6b5d218640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 msgr2=0x7f6b3c00e650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:51.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:51.888+0000 7f6b5d218640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 0x7f6b3c00e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:51.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:51.888+0000 7f6b5ca17640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0036d0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:52.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:52.289+0000 7f6b5da19640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0098b0 legacy=0x7f6b3c007600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:52.288 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:52.289+0000 7f6b5d218640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 msgr2=0x7f6b3c00e650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:52.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:52.289+0000 7f6b5d218640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 0x7f6b3c00e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:52.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:52.289+0000 7f6b5ca17640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0036d0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:53.089 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:53.090+0000 7f6b5d218640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 msgr2=0x7f6b3c00e650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:53.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:53.090+0000 7f6b5da19640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0098b0 legacy=0x7f6b3c007600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:53.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:53.090+0000 7f6b5d218640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 0x7f6b3c00e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:53.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:53.090+0000 7f6b5ca17640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0036d0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:05:54.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:54.687+0000 7f6b4affd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 msgr2=0x7f6b3c00e650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:54.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:54.687+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 0x7f6b3c00e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:05:54.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:54.687+0000 7f6b4affd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0036d0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:54.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:54.687+0000 7f6b4affd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0098b0 legacy=0x7f6b3c007600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:54.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:54.687+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014fd0 0x7f6b3c008f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:54.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:54.687+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0112f0 0x7f6b3c005af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:54.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:54.687+0000 7f6b4affd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c0112f0 2023-11-09T04:05:54.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:54.687+0000 7f6b4affd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c008d20 con 0x7f6b3c014fd0 2023-11-09T04:05:54.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:54.687+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014fd0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:54.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:54.687+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0112f0 msgr2=0x7f6b3c005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:54.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:54.687+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014fd0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:54.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:54.687+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0112f0 0x7f6b3c005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:54.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:54.888+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014fd0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:54.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:54.888+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0112f0 msgr2=0x7f6b3c005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:54.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:54.888+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014fd0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:54.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:54.888+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0112f0 0x7f6b3c005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:55.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:55.289+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0112f0 msgr2=0x7f6b3c005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:55.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:55.289+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014fd0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:55.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:55.289+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014fd0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:55.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:55.289+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0112f0 0x7f6b3c005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:56.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:56.090+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014fd0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:56.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:56.090+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0112f0 msgr2=0x7f6b3c005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:05:56.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:56.090+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014fd0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:56.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:56.090+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0112f0 0x7f6b3c005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:57.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:57.687+0000 7f6b4affd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014fd0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:57.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:57.687+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c014fd0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:05:57.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:57.687+0000 7f6b4affd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0112f0 msgr2=0x7f6b3c005af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:05:57.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:57.687+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0112f0 0x7f6b3c005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:05:57.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:57.687+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 0x7f6b3c00e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:57.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:57.687+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c007d10 0x7f6b3c008150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:05:57.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:57.687+0000 7f6b5da19640 1 -- reap_dead start 2023-11-09T04:05:57.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:57.687+0000 7f6b4affd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b58070f40 con 0x7f6b3c0098b0 2023-11-09T04:05:57.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:57.687+0000 7f6b4affd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c00f880 2023-11-09T04:05:57.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:57.687+0000 7f6b4affd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c008d20 con 0x7f6b3c007d10 2023-11-09T04:05:57.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:57.687+0000 7f6b5da19640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0098b0 legacy=0x7f6b3c007600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:57.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:57.687+0000 7f6b5d218640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 msgr2=0x7f6b3c00e650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:57.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:57.687+0000 7f6b5d218640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 0x7f6b3c00e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:57.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:57.687+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c007d10 msgr2=0x7f6b3c008150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:57.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:57.687+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c007d10 0x7f6b3c008150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:05:57.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:57.888+0000 7f6b5d218640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 msgr2=0x7f6b3c00e650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:57.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:57.888+0000 7f6b5da19640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0098b0 legacy=0x7f6b3c007600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:57.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:57.888+0000 7f6b5d218640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 0x7f6b3c00e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:57.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:57.888+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c007d10 msgr2=0x7f6b3c008150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:57.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:57.888+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c007d10 0x7f6b3c008150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:05:58.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:58.289+0000 7f6b5da19640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0098b0 legacy=0x7f6b3c007600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:58.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:58.289+0000 7f6b5d218640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 msgr2=0x7f6b3c00e650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:58.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:58.289+0000 7f6b5d218640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 0x7f6b3c00e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:58.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:58.289+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c007d10 msgr2=0x7f6b3c008150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:58.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:58.289+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c007d10 0x7f6b3c008150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:05:59.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:59.090+0000 7f6b5d218640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 msgr2=0x7f6b3c00e650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:05:59.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:59.090+0000 7f6b5da19640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0098b0 legacy=0x7f6b3c007600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:05:59.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:59.090+0000 7f6b5d218640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 0x7f6b3c00e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:05:59.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:59.090+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c007d10 msgr2=0x7f6b3c008150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:05:59.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:05:59.090+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c007d10 0x7f6b3c008150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:00.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:00.687+0000 7f6b4affd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c007d10 msgr2=0x7f6b3c008150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:00.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:00.687+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c007d10 0x7f6b3c008150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:00.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:00.687+0000 7f6b4affd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 msgr2=0x7f6b3c00e650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:00.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:00.687+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 0x7f6b3c00e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:00.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:00.687+0000 7f6b4affd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0098b0 legacy=0x7f6b3c007600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:00.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:00.687+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c008f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:00.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:00.687+0000 7f6b4affd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b58071500 con 0x7f6b3c0112f0 2023-11-09T04:06:00.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:00.687+0000 7f6b4affd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c0036d0 2023-11-09T04:06:00.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:00.687+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0112f0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:00.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:00.687+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:00.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:00.687+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:00.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:00.888+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0112f0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:00.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:00.888+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:00.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:00.888+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:01.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:01.289+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0112f0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:01.289 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:01.289+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:01.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:01.289+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:02.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:02.090+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0112f0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:02.090 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:02.090+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:02.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:02.090+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:03.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:03.688+0000 7f6b4affd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:03.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:03.688+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0036d0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:03.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:03.688+0000 7f6b4affd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0112f0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:03.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:03.688+0000 7f6b5da19640 1 -- reap_dead start 2023-11-09T04:06:03.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:03.688+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c005af0 0x7f6b3c014200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:03.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:03.688+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0098b0 0x7f6b3c008f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:03.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:03.688+0000 7f6b4affd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b58070f40 con 0x7f6b3c013e20 2023-11-09T04:06:03.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:03.688+0000 7f6b4affd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c005af0 2023-11-09T04:06:03.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:03.688+0000 7f6b4affd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c008d20 con 0x7f6b3c0098b0 2023-11-09T04:06:03.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:03.688+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c013e20 legacy=0x7f6b3c01e0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:03.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:03.688+0000 7f6b5da19640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0098b0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:03.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:03.688+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c005af0 msgr2=0x7f6b3c014200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:03.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:03.688+0000 7f6b5da19640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0098b0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:03.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:03.688+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c005af0 0x7f6b3c014200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:03.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:03.889+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c013e20 legacy=0x7f6b3c01e0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:03.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:03.889+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c005af0 msgr2=0x7f6b3c014200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:03.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:03.889+0000 7f6b5da19640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0098b0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:03.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:03.889+0000 7f6b5da19640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0098b0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:03.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:03.889+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c005af0 0x7f6b3c014200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:04.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:04.290+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c005af0 msgr2=0x7f6b3c014200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:04.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:04.290+0000 7f6b5da19640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0098b0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:04.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:04.290+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c013e20 legacy=0x7f6b3c01e0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:04.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:04.290+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c005af0 0x7f6b3c014200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:04.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:04.290+0000 7f6b5da19640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0098b0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:05.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:05.091+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c005af0 msgr2=0x7f6b3c014200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:05.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:05.091+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c013e20 legacy=0x7f6b3c01e0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:05.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:05.091+0000 7f6b5da19640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0098b0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:05.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:05.091+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c005af0 0x7f6b3c014200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:05.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:05.091+0000 7f6b5da19640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0098b0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:05.937 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:06:05.940 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:06:06.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:06.688+0000 7f6b4affd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0098b0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:06.688 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:06.688+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0098b0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:06.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:06.688+0000 7f6b4affd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c005af0 msgr2=0x7f6b3c014200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:06.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:06.688+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c005af0 0x7f6b3c014200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:06.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:06.688+0000 7f6b4affd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c013e20 legacy=0x7f6b3c01e0d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:06.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:06.688+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 0x7f6b3c0065d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:06.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:06.688+0000 7f6b4affd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b58071500 con 0x7f6b3c008130 2023-11-09T04:06:06.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:06.688+0000 7f6b4affd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c00f880 2023-11-09T04:06:06.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:06.688+0000 7f6b5ca17640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 msgr2=0x7f6b3c0065d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:06.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:06.688+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008130 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:06.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:06.688+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 0x7f6b3c0065d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:06.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:06.889+0000 7f6b5ca17640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 msgr2=0x7f6b3c0065d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:06.888 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:06.889+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008130 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:06.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:06.889+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 0x7f6b3c0065d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:07.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:07.290+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008130 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:07.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:07.290+0000 7f6b5ca17640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 msgr2=0x7f6b3c0065d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:07.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:07.290+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 0x7f6b3c0065d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:08.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:08.091+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008130 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:08.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:08.091+0000 7f6b5ca17640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 msgr2=0x7f6b3c0065d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:08.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:08.091+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 0x7f6b3c0065d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:09.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:09.688+0000 7f6b4affd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 msgr2=0x7f6b3c0065d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:09.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:09.688+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 0x7f6b3c0065d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:09.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:09.688+0000 7f6b4affd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008130 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:09.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:09.688+0000 7f6b5da19640 1 -- reap_dead start 2023-11-09T04:06:09.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:09.688+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c008f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:09.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:09.689+0000 7f6b4affd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b58070f40 con 0x7f6b3c008510 2023-11-09T04:06:09.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:09.689+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008510 legacy=0x7f6b3c006b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:09.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:09.689+0000 7f6b4affd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b58071500 con 0x7f6b3c0065d0 2023-11-09T04:06:09.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:09.689+0000 7f6b4affd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c0098b0 2023-11-09T04:06:09.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:09.689+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:09.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:09.689+0000 7f6b5da19640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0065d0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:09.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:09.689+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:09.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:09.890+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008510 legacy=0x7f6b3c006b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:09.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:09.890+0000 7f6b5da19640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0065d0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:09.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:09.890+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:09.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:09.890+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:10.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:10.291+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008510 legacy=0x7f6b3c006b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:10.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:10.291+0000 7f6b5da19640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0065d0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:10.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:10.291+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:10.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:10.291+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:11.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:11.092+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008510 legacy=0x7f6b3c006b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:11.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:11.092+0000 7f6b5da19640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0065d0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:11.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:11.092+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:11.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:11.092+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:12.689 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:12.689+0000 7f6b4affd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:12.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:12.689+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:12.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:12.689+0000 7f6b4affd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0065d0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:12.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:12.689+0000 7f6b4affd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008510 legacy=0x7f6b3c006b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:12.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:12.689+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c00f880 0x7f6b3c007220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:12.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:12.689+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c005af0 0x7f6b3c013e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:12.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:12.689+0000 7f6b4affd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c005af0 2023-11-09T04:06:12.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:12.689+0000 7f6b4affd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c008d20 con 0x7f6b3c00f880 2023-11-09T04:06:12.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:12.689+0000 7f6b5ca17640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c005af0 msgr2=0x7f6b3c013e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:12.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:12.689+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c005af0 0x7f6b3c013e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:12.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:12.689+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c00f880 msgr2=0x7f6b3c007220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:12.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:12.689+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c00f880 0x7f6b3c007220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:12.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:12.890+0000 7f6b5ca17640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c005af0 msgr2=0x7f6b3c013e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:12.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:12.890+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c005af0 0x7f6b3c013e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:12.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:12.890+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c00f880 msgr2=0x7f6b3c007220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:12.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:12.890+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c00f880 0x7f6b3c007220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:13.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:13.291+0000 7f6b5ca17640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c005af0 msgr2=0x7f6b3c013e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:13.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:13.291+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c005af0 0x7f6b3c013e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:13.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:13.291+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c00f880 msgr2=0x7f6b3c007220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:13.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:13.291+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c00f880 0x7f6b3c007220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:14.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:14.092+0000 7f6b5ca17640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c005af0 msgr2=0x7f6b3c013e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:14.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:14.092+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c005af0 0x7f6b3c013e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:14.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:14.092+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c00f880 msgr2=0x7f6b3c007220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:14.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:14.092+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c00f880 0x7f6b3c007220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:15.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:15.689+0000 7f6b4affd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c00f880 msgr2=0x7f6b3c007220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:15.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:15.689+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c00f880 0x7f6b3c007220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:15.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:15.689+0000 7f6b4affd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c005af0 msgr2=0x7f6b3c013e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:15.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:15.689+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c005af0 0x7f6b3c013e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:15.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:15.689+0000 7f6b5da19640 1 -- reap_dead start 2023-11-09T04:06:15.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:15.689+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c008f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:15.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:15.689+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 0x7f6b3c0069b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:15.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:15.689+0000 7f6b4affd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5815dae0 con 0x7f6b3c0065d0 2023-11-09T04:06:15.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:15.689+0000 7f6b4affd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c00f880 2023-11-09T04:06:15.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:15.689+0000 7f6b4affd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b3c008d20 con 0x7f6b3c0098b0 2023-11-09T04:06:15.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:15.689+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 msgr2=0x7f6b3c0069b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:15.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:15.689+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 0x7f6b3c0069b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:15.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:15.689+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:15.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:15.689+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0065d0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:15.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:15.689+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:15.889 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:15.890+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 msgr2=0x7f6b3c0069b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:15.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:15.890+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 0x7f6b3c0069b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:15.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:15.890+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0065d0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:15.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:15.890+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:15.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:15.890+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:16.290 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:16.291+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 msgr2=0x7f6b3c0069b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:16.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:16.291+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 0x7f6b3c0069b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:16.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:16.291+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:16.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:16.291+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0065d0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:16.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:16.291+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:17.091 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:17.092+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 msgr2=0x7f6b3c0069b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:17.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:17.092+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 0x7f6b3c0069b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:17.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:17.092+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:17.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:17.092+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:17.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:17.092+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0065d0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:18.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:18.690+0000 7f6b4affd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:18.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:18.690+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:18.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:18.690+0000 7f6b4affd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 msgr2=0x7f6b3c0069b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:18.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:18.690+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c00f880 0x7f6b3c0069b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:18.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:18.690+0000 7f6b4affd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0065d0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:18.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:18.690+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 0x7f6b3c006ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:18.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:18.690+0000 7f6b4affd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b58070f40 con 0x7f6b3c005af0 2023-11-09T04:06:18.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:18.690+0000 7f6b4affd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c008510 2023-11-09T04:06:18.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:18.690+0000 7f6b5ca17640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c005af0 legacy=0x7f6b3c007430 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:18.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:18.690+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 msgr2=0x7f6b3c006ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:18.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:18.690+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 0x7f6b3c006ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:18.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:18.891+0000 7f6b5ca17640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c005af0 legacy=0x7f6b3c007430 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:18.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:18.891+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 msgr2=0x7f6b3c006ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:18.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:18.891+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 0x7f6b3c006ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:19.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:19.292+0000 7f6b5ca17640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c005af0 legacy=0x7f6b3c007430 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:19.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:19.292+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 msgr2=0x7f6b3c006ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:19.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:19.292+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 0x7f6b3c006ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:20.092 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:20.093+0000 7f6b5ca17640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c005af0 legacy=0x7f6b3c007430 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:20.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:20.093+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 msgr2=0x7f6b3c006ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:20.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:20.093+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 0x7f6b3c006ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:21.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:21.690+0000 7f6b4affd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 msgr2=0x7f6b3c006ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:21.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:21.690+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 0x7f6b3c006ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:21.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:21.690+0000 7f6b4affd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c005af0 legacy=0x7f6b3c007430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:21.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:21.690+0000 7f6b5da19640 1 -- reap_dead start 2023-11-09T04:06:21.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:21.690+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c008f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:21.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:21.690+0000 7f6b4affd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5815dae0 con 0x7f6b3c008510 2023-11-09T04:06:21.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:21.690+0000 7f6b4affd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b58070f40 con 0x7f6b3c0065d0 2023-11-09T04:06:21.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:21.690+0000 7f6b4affd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c0098b0 2023-11-09T04:06:21.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:21.690+0000 7f6b5ca17640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008510 legacy=0x7f6b3c0069b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:21.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:21.690+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:21.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:21.690+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0065d0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:21.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:21.690+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:21.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:21.891+0000 7f6b5ca17640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008510 legacy=0x7f6b3c0069b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:21.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:21.891+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:21.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:21.891+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:21.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:21.891+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0065d0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:22.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:22.292+0000 7f6b5ca17640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008510 legacy=0x7f6b3c0069b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:22.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:22.292+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:22.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:22.292+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:22.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:22.292+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0065d0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:23.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:23.093+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:23.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:23.093+0000 7f6b5ca17640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008510 legacy=0x7f6b3c0069b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:23.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:23.093+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:23.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:23.093+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0065d0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:24.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:24.690+0000 7f6b4affd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:24.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:24.690+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:24.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:24.690+0000 7f6b4affd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c0065d0 legacy=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:24.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:24.690+0000 7f6b4affd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008510 legacy=0x7f6b3c0069b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:24.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:24.690+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c00f880 0x7f6b3c00c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:24.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:24.690+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c005ac0 0x7f6b3c011e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:24.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:24.690+0000 7f6b4affd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c005ac0 2023-11-09T04:06:24.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:24.690+0000 7f6b4affd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c008d20 con 0x7f6b3c00f880 2023-11-09T04:06:24.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:24.690+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c005ac0 msgr2=0x7f6b3c011e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:24.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:24.690+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c00f880 msgr2=0x7f6b3c00c640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:24.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:24.690+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c005ac0 0x7f6b3c011e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:24.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:24.690+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c00f880 0x7f6b3c00c640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:24.890 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:24.890+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c005ac0 msgr2=0x7f6b3c011e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:24.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:24.890+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c00f880 msgr2=0x7f6b3c00c640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:24.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:24.890+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c00f880 0x7f6b3c00c640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:24.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:24.890+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c005ac0 0x7f6b3c011e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:25.291 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:25.291+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c005ac0 msgr2=0x7f6b3c011e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:25.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:25.291+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c005ac0 0x7f6b3c011e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:25.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:25.291+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c00f880 msgr2=0x7f6b3c00c640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:25.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:25.291+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c00f880 0x7f6b3c00c640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:26.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:26.092+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c005ac0 msgr2=0x7f6b3c011e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:26.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:26.092+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c00f880 msgr2=0x7f6b3c00c640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:26.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:26.092+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c005ac0 0x7f6b3c011e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:26.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:26.092+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c00f880 0x7f6b3c00c640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:27.690 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:27.690+0000 7f6b4affd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c00f880 msgr2=0x7f6b3c00c640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:27.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:27.690+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c00f880 0x7f6b3c00c640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:27.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:27.690+0000 7f6b4affd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c005ac0 msgr2=0x7f6b3c011e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:27.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:27.690+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c005ac0 0x7f6b3c011e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:27.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:27.690+0000 7f6b5da19640 1 -- reap_dead start 2023-11-09T04:06:27.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:27.690+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0065d0 0x7f6b3c008f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:27.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:27.690+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c00f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:27.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:27.690+0000 7f6b4affd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b58071500 con 0x7f6b3c008510 2023-11-09T04:06:27.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:27.690+0000 7f6b4affd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c0098b0 2023-11-09T04:06:27.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:27.690+0000 7f6b4affd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c008d20 con 0x7f6b3c0065d0 2023-11-09T04:06:27.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:27.690+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008510 legacy=0x7f6b3c0069b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:27.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:27.690+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:27.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:27.690+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:27.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:27.690+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0065d0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:27.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:27.690+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0065d0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:27.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:27.891+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008510 legacy=0x7f6b3c0069b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:27.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:27.891+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:27.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:27.891+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:27.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:27.891+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0065d0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:27.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:27.891+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0065d0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:28.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:28.292+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:28.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:28.292+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008510 legacy=0x7f6b3c0069b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:28.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:28.292+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:28.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:28.292+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0065d0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:28.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:28.292+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0065d0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:29.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:29.093+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008510 legacy=0x7f6b3c0069b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:29.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:29.093+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:29.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:29.093+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:29.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:29.093+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0065d0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:29.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:29.093+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0065d0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:30.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:30.690+0000 7f6b4affd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0065d0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:30.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:30.690+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0065d0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:30.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:30.690+0000 7f6b4affd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:30.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:30.690+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:30.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:30.690+0000 7f6b4affd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008510 legacy=0x7f6b3c0069b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:30.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:30.690+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0147e0 0x7f6b3c0109f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:30.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:30.690+0000 7f6b4affd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5815dae0 con 0x7f6b3c00f6b0 2023-11-09T04:06:30.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:30.690+0000 7f6b4affd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c0147e0 2023-11-09T04:06:30.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:30.690+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0147e0 msgr2=0x7f6b3c0109f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:30.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:30.690+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00f6b0 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:30.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:30.690+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0147e0 0x7f6b3c0109f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:30.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:30.891+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00f6b0 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:30.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:30.891+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0147e0 msgr2=0x7f6b3c0109f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:30.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:30.891+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0147e0 0x7f6b3c0109f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:31.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:31.292+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0147e0 msgr2=0x7f6b3c0109f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:31.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:31.292+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00f6b0 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:31.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:31.292+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0147e0 0x7f6b3c0109f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:32.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:32.093+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00f6b0 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:32.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:32.093+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0147e0 msgr2=0x7f6b3c0109f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:32.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:32.093+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0147e0 0x7f6b3c0109f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:33.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:33.690+0000 7f6b4affd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0147e0 msgr2=0x7f6b3c0109f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:33.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:33.690+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0147e0 0x7f6b3c0109f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:33.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:33.690+0000 7f6b4affd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00f6b0 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:33.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:33.690+0000 7f6b5da19640 1 -- reap_dead start 2023-11-09T04:06:33.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:33.690+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 0x7f6b3c008f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:33.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:33.690+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c00f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:33.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:33.690+0000 7f6b4affd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b58071500 con 0x7f6b3c005ac0 2023-11-09T04:06:33.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:33.690+0000 7f6b4affd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c0098b0 2023-11-09T04:06:33.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:33.690+0000 7f6b4affd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c008d20 con 0x7f6b3c008510 2023-11-09T04:06:33.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:33.690+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:33.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:33.690+0000 7f6b5da19640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:33.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:33.690+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c005ac0 legacy=0x7f6b3c013e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:33.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:33.690+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:33.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:33.690+0000 7f6b5da19640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:33.891 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:33.891+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:33.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:33.891+0000 7f6b5da19640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:33.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:33.891+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c005ac0 legacy=0x7f6b3c013e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:33.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:33.891+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:33.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:33.891+0000 7f6b5da19640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:34.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:34.292+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:34.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:34.292+0000 7f6b5da19640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:34.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:34.292+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c005ac0 legacy=0x7f6b3c013e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:34.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:34.292+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:34.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:34.292+0000 7f6b5da19640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:35.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:35.093+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:35.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:35.093+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c005ac0 legacy=0x7f6b3c013e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:35.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:35.093+0000 7f6b5da19640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:35.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:35.093+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:35.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:35.093+0000 7f6b5da19640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:36.077 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:06:36.080 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:06:36.691 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:36.691+0000 7f6b4affd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:36.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:36.691+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:36.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:36.691+0000 7f6b4affd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:36.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:36.691+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:36.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:36.691+0000 7f6b4affd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c005ac0 legacy=0x7f6b3c013e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:36.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:36.691+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0147e0 0x7f6b3c0102e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:36.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:36.691+0000 7f6b4affd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5815dae0 con 0x7f6b3c00e660 2023-11-09T04:06:36.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:36.691+0000 7f6b4affd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c0147e0 2023-11-09T04:06:36.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:36.691+0000 7f6b5ca17640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c00e660 legacy=0x7f6b3c010820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:36.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:36.691+0000 7f6b5d218640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0147e0 msgr2=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:36.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:36.691+0000 7f6b5d218640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0147e0 0x7f6b3c0102e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:36.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:36.892+0000 7f6b5d218640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0147e0 msgr2=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:36.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:36.892+0000 7f6b5ca17640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c00e660 legacy=0x7f6b3c010820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:36.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:36.892+0000 7f6b5d218640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0147e0 0x7f6b3c0102e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:37.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:37.293+0000 7f6b5d218640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0147e0 msgr2=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:37.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:37.293+0000 7f6b5ca17640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c00e660 legacy=0x7f6b3c010820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:37.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:37.293+0000 7f6b5d218640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0147e0 0x7f6b3c0102e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:38.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:38.094+0000 7f6b5ca17640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c00e660 legacy=0x7f6b3c010820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:38.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:38.094+0000 7f6b5d218640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0147e0 msgr2=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:38.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:38.094+0000 7f6b5d218640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0147e0 0x7f6b3c0102e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:39.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:39.691+0000 7f6b4affd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0147e0 msgr2=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:39.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:39.691+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0147e0 0x7f6b3c0102e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:39.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:39.691+0000 7f6b4affd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c00e660 legacy=0x7f6b3c010820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:39.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:39.691+0000 7f6b5da19640 1 -- reap_dead start 2023-11-09T04:06:39.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:39.691+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c00f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:39.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:39.691+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c005ac0 0x7f6b3c008f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:39.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:39.691+0000 7f6b4affd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b58071500 con 0x7f6b3c0147e0 2023-11-09T04:06:39.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:39.691+0000 7f6b4affd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c0098b0 2023-11-09T04:06:39.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:39.691+0000 7f6b4affd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c008d20 con 0x7f6b3c005ac0 2023-11-09T04:06:39.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:39.691+0000 7f6b5da19640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0147e0 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:39.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:39.691+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c005ac0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:39.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:39.691+0000 7f6b5d218640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:39.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:39.691+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c005ac0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:39.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:39.691+0000 7f6b5d218640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:39.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:39.892+0000 7f6b5da19640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0147e0 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:39.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:39.892+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c005ac0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:39.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:39.892+0000 7f6b5d218640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:39.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:39.892+0000 7f6b5d218640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:39.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:39.892+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c005ac0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:40.292 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:40.293+0000 7f6b5da19640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0147e0 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:40.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:40.293+0000 7f6b5d218640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:40.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:40.293+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c005ac0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:40.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:40.293+0000 7f6b5d218640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:40.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:40.293+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c005ac0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:41.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:41.094+0000 7f6b5da19640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0147e0 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:41.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:41.094+0000 7f6b5d218640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:41.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:41.094+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c005ac0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:41.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:41.094+0000 7f6b5d218640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:41.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:41.094+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c005ac0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:42.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:42.692+0000 7f6b4affd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c005ac0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:42.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:42.692+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c005ac0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:42.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:42.692+0000 7f6b4affd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:42.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:42.692+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:42.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:42.692+0000 7f6b4affd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0147e0 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:42.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:42.692+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 0x7f6b3c0079c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:42.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:42.692+0000 7f6b4affd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5815dae0 con 0x7f6b3c008510 2023-11-09T04:06:42.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:42.692+0000 7f6b4affd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c0075b0 2023-11-09T04:06:42.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:42.692+0000 7f6b5ca17640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 msgr2=0x7f6b3c0079c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:42.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:42.692+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 0x7f6b3c0079c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:42.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:42.692+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c008510 legacy=0x7f6b3c0109f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:42.892 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:42.893+0000 7f6b5ca17640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 msgr2=0x7f6b3c0079c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:42.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:42.893+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 0x7f6b3c0079c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:42.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:42.893+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c008510 legacy=0x7f6b3c0109f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:43.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:43.294+0000 7f6b5ca17640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 msgr2=0x7f6b3c0079c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:43.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:43.294+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 0x7f6b3c0079c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:43.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:43.294+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c008510 legacy=0x7f6b3c0109f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:44.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:44.095+0000 7f6b5ca17640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 msgr2=0x7f6b3c0079c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:44.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:44.095+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 0x7f6b3c0079c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:44.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:44.095+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c008510 legacy=0x7f6b3c0109f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:45.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:45.692+0000 7f6b4affd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 msgr2=0x7f6b3c0079c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:45.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:45.692+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 0x7f6b3c0079c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:45.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:45.692+0000 7f6b4affd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c008510 legacy=0x7f6b3c0109f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:45.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:45.692+0000 7f6b5da19640 1 -- reap_dead start 2023-11-09T04:06:45.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:45.692+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c00f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:45.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:45.692+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 0x7f6b3c0079c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:45.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:45.692+0000 7f6b4affd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b58071500 con 0x7f6b3c0147e0 2023-11-09T04:06:45.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:45.692+0000 7f6b4affd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c0075b0 2023-11-09T04:06:45.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:45.692+0000 7f6b4affd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b3c008d20 con 0x7f6b3c0098b0 2023-11-09T04:06:45.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:45.692+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0147e0 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:45.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:45.692+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 msgr2=0x7f6b3c0079c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:45.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:45.692+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 0x7f6b3c0079c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:45.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:45.692+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:45.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:45.692+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:45.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:45.893+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 msgr2=0x7f6b3c0079c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:45.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:45.893+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0147e0 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:45.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:45.893+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 0x7f6b3c0079c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:45.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:45.893+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:45.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:45.893+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:46.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:46.294+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 msgr2=0x7f6b3c0079c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:46.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:46.294+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0147e0 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:46.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:46.294+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 0x7f6b3c0079c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:46.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:46.294+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:46.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:46.294+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:47.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:47.095+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0147e0 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:47.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:47.095+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 msgr2=0x7f6b3c0079c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:47.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:47.095+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 0x7f6b3c0079c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:47.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:47.095+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:47.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:47.095+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:48.692 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:48.692+0000 7f6b4affd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:48.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:48.692+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c0098b0 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:48.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:48.692+0000 7f6b4affd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 msgr2=0x7f6b3c0079c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:48.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:48.692+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 0x7f6b3c0079c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:48.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:48.692+0000 7f6b4affd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0147e0 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:48.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:48.692+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 0x7f6b3c008f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:48.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:48.692+0000 7f6b4affd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5815dae0 con 0x7f6b3c005ac0 2023-11-09T04:06:48.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:48.692+0000 7f6b4affd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c008510 2023-11-09T04:06:48.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:48.692+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c005ac0 legacy=0x7f6b3c0109f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:48.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:48.692+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:48.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:48.692+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:48.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:48.893+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c005ac0 legacy=0x7f6b3c0109f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:48.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:48.893+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:48.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:48.893+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:49.293 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:49.294+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:49.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:49.294+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c005ac0 legacy=0x7f6b3c0109f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:49.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:49.294+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:50.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:50.095+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c005ac0 legacy=0x7f6b3c0109f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:50.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:50.095+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:50.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:50.095+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:51.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:51.693+0000 7f6b4affd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:51.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:51.693+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c008510 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:51.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:51.693+0000 7f6b4affd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c005ac0 legacy=0x7f6b3c0109f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:51.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:51.693+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 0x7f6b3c0079c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:51.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:51.693+0000 7f6b5da19640 1 -- reap_dead start 2023-11-09T04:06:51.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:51.693+0000 7f6b4affd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b58071500 con 0x7f6b3c008510 2023-11-09T04:06:51.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:51.693+0000 7f6b4affd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5815dae0 con 0x7f6b3c00e650 2023-11-09T04:06:51.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:51.693+0000 7f6b4affd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c0075b0 2023-11-09T04:06:51.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:51.693+0000 7f6b5ca17640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 msgr2=0x7f6b3c0079c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:51.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:51.693+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 0x7f6b3c0079c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:51.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:51.693+0000 7f6b5da19640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008510 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:51.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:51.693+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00e650 legacy=0x7f6b3c013e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:51.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:51.894+0000 7f6b5da19640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008510 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:51.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:51.894+0000 7f6b5ca17640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 msgr2=0x7f6b3c0079c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:51.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:51.894+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 0x7f6b3c0079c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:51.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:51.894+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00e650 legacy=0x7f6b3c013e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:52.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:52.295+0000 7f6b5da19640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008510 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:52.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:52.295+0000 7f6b5ca17640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 msgr2=0x7f6b3c0079c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:52.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:52.295+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 0x7f6b3c0079c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:52.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:52.295+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00e650 legacy=0x7f6b3c013e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:53.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:53.096+0000 7f6b5ca17640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 msgr2=0x7f6b3c0079c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:53.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:53.096+0000 7f6b5da19640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008510 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:06:53.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:53.096+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 0x7f6b3c0079c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:53.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:53.096+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00e650 legacy=0x7f6b3c013e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:54.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:54.693+0000 7f6b4affd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 msgr2=0x7f6b3c0079c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:54.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:54.693+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0075b0 0x7f6b3c0079c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:54.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:54.693+0000 7f6b4affd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00e650 legacy=0x7f6b3c013e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:54.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:54.693+0000 7f6b4affd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c008510 legacy=0x7f6b3c0102e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:54.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:54.693+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0147e0 0x7f6b3c00f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:54.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:54.693+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008f70 0x7f6b3c0109f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:54.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:54.693+0000 7f6b4affd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c008f70 2023-11-09T04:06:54.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:54.693+0000 7f6b4affd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c008d20 con 0x7f6b3c0147e0 2023-11-09T04:06:54.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:54.693+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0147e0 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:54.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:54.693+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008f70 msgr2=0x7f6b3c0109f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:54.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:54.693+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0147e0 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:54.699 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:54.693+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008f70 0x7f6b3c0109f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:54.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:54.894+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008f70 msgr2=0x7f6b3c0109f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:54.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:54.894+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0147e0 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:54.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:54.894+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008f70 0x7f6b3c0109f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:54.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:54.894+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0147e0 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:55.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:55.295+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0147e0 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:55.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:55.295+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008f70 msgr2=0x7f6b3c0109f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:55.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:55.295+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0147e0 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:55.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:55.295+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008f70 0x7f6b3c0109f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:56.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:56.096+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008f70 msgr2=0x7f6b3c0109f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:56.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:56.096+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0147e0 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:06:56.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:56.096+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008f70 0x7f6b3c0109f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:56.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:56.096+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0147e0 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:57.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:57.692+0000 7f6b4affd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0147e0 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:57.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:57.692+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0147e0 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:57.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:57.692+0000 7f6b4affd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008f70 msgr2=0x7f6b3c0109f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:06:57.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:57.692+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008f70 0x7f6b3c0109f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:06:57.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:57.692+0000 7f6b5da19640 1 -- reap_dead start 2023-11-09T04:06:57.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:57.692+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008510 0x7f6b3c00f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:57.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:57.692+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0147a0 0x7f6b3c008f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:06:57.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:57.692+0000 7f6b4affd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b58070f40 con 0x7f6b3c00e650 2023-11-09T04:06:57.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:57.692+0000 7f6b4affd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c0147a0 2023-11-09T04:06:57.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:57.692+0000 7f6b4affd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b3c008d20 con 0x7f6b3c008510 2023-11-09T04:06:57.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:57.692+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0147a0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:57.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:57.692+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0147a0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:57.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:57.692+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008510 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:57.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:57.692+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008510 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:06:57.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:57.692+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00e650 legacy=0x7f6b3c013e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:57.893 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:57.893+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0147a0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:57.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:57.893+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0147a0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:57.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:57.893+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008510 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:57.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:57.893+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00e650 legacy=0x7f6b3c013e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:57.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:57.893+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008510 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:06:58.294 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:58.294+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0147a0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:58.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:58.294+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0147a0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:58.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:58.294+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00e650 legacy=0x7f6b3c013e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:58.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:58.294+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008510 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:58.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:58.294+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008510 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:06:59.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:59.095+0000 7f6b5da19640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0147a0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:06:59.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:59.095+0000 7f6b5da19640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0147a0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:06:59.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:59.095+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00e650 legacy=0x7f6b3c013e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:06:59.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:59.095+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008510 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:06:59.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:06:59.095+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008510 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:07:00.693 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:00.693+0000 7f6b4affd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008510 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:07:00.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:00.693+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008510 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:07:00.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:00.693+0000 7f6b4affd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0147a0 msgr2=0x7f6b3c008f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:07:00.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:00.693+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0147a0 0x7f6b3c008f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:07:00.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:00.693+0000 7f6b4affd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00e650 legacy=0x7f6b3c013e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:07:00.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:00.693+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c010e60 0x7f6b3c0065a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:07:00.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:00.693+0000 7f6b4affd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b58071500 con 0x7f6b3c0102e0 2023-11-09T04:07:00.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:00.693+0000 7f6b4affd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c010e60 2023-11-09T04:07:00.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:00.693+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0102e0 legacy=0x7f6b3c010750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:07:00.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:00.693+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c010e60 msgr2=0x7f6b3c0065a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:07:00.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:00.693+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c010e60 0x7f6b3c0065a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:07:00.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:00.894+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0102e0 legacy=0x7f6b3c010750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:07:00.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:00.894+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c010e60 msgr2=0x7f6b3c0065a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:07:00.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:00.894+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c010e60 0x7f6b3c0065a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:07:01.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:01.295+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0102e0 legacy=0x7f6b3c010750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:07:01.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:01.295+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c010e60 msgr2=0x7f6b3c0065a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:07:01.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:01.295+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c010e60 0x7f6b3c0065a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:07:02.096 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:02.096+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0102e0 legacy=0x7f6b3c010750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:07:02.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:02.096+0000 7f6b5ca17640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c010e60 msgr2=0x7f6b3c0065a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:07:02.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:02.096+0000 7f6b5ca17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c010e60 0x7f6b3c0065a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:07:03.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:03.693+0000 7f6b4affd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c010e60 msgr2=0x7f6b3c0065a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:07:03.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:03.693+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c010e60 0x7f6b3c0065a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:07:03.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:03.693+0000 7f6b4affd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0102e0 legacy=0x7f6b3c010750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:07:03.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:03.693+0000 7f6b5da19640 1 -- reap_dead start 2023-11-09T04:07:03.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:03.693+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c008510 0x7f6b3c00f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:07:03.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:03.693+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0102e0 0x7f6b3c008f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:07:03.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:03.693+0000 7f6b4affd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b58070f40 con 0x7f6b3c00e650 2023-11-09T04:07:03.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:03.693+0000 7f6b4affd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c008510 2023-11-09T04:07:03.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:03.693+0000 7f6b4affd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c008d20 con 0x7f6b3c0102e0 2023-11-09T04:07:03.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:03.693+0000 7f6b5ca17640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c008510 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:07:03.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:03.693+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c008510 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:07:03.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:03.693+0000 7f6b5da19640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0102e0 msgr2=0x7f6b3c008f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:07:03.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:03.693+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00e650 legacy=0x7f6b3c013e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:07:03.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:03.693+0000 7f6b5da19640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0102e0 0x7f6b3c008f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:07:03.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:03.894+0000 7f6b5ca17640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c008510 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:07:03.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:03.894+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c008510 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:07:03.894 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:03.894+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00e650 legacy=0x7f6b3c013e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:07:03.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:03.894+0000 7f6b5da19640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0102e0 msgr2=0x7f6b3c008f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:07:03.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:03.894+0000 7f6b5da19640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0102e0 0x7f6b3c008f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:07:04.295 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:04.295+0000 7f6b5ca17640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c008510 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:07:04.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:04.295+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c008510 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:07:04.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:04.295+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00e650 legacy=0x7f6b3c013e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:07:04.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:04.295+0000 7f6b5da19640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0102e0 msgr2=0x7f6b3c008f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:07:04.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:04.295+0000 7f6b5da19640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0102e0 0x7f6b3c008f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:07:05.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:05.096+0000 7f6b5ca17640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c008510 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:07:05.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:05.096+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c008510 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:07:05.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:05.096+0000 7f6b5d218640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00e650 legacy=0x7f6b3c013e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:07:05.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:05.096+0000 7f6b5da19640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0102e0 msgr2=0x7f6b3c008f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:07:05.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:05.096+0000 7f6b5da19640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0102e0 0x7f6b3c008f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:07:06.224 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:07:06.228 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:07:06.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:06.694+0000 7f6b4affd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0102e0 msgr2=0x7f6b3c008f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:07:06.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:06.694+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0102e0 0x7f6b3c008f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:07:06.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:06.694+0000 7f6b4affd640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c008510 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:07:06.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:06.694+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c008510 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:07:06.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:06.694+0000 7f6b4affd640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00e650 legacy=0x7f6b3c013e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:07:06.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:06.694+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c010e60 0x7f6b3c006580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:07:06.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:06.694+0000 7f6b4affd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b58071500 con 0x7f6b3c0147a0 2023-11-09T04:07:06.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:06.694+0000 7f6b4affd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c010e60 2023-11-09T04:07:06.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:06.694+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0147a0 legacy=0x7f6b3c010750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:07:06.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:06.694+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c010e60 msgr2=0x7f6b3c006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:07:06.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:06.694+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c010e60 0x7f6b3c006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:07:06.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:06.895+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0147a0 legacy=0x7f6b3c010750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:07:06.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:06.895+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c010e60 msgr2=0x7f6b3c006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:07:06.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:06.895+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c010e60 0x7f6b3c006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:07:07.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:07.296+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0147a0 legacy=0x7f6b3c010750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:07:07.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:07.296+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c010e60 msgr2=0x7f6b3c006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:07:07.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:07.296+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c010e60 0x7f6b3c006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:07:08.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:08.097+0000 7f6b5d218640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0147a0 legacy=0x7f6b3c010750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:07:08.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:08.097+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c010e60 msgr2=0x7f6b3c006580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:07:08.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:08.097+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c010e60 0x7f6b3c006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:07:09.694 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:09.694+0000 7f6b4affd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c010e60 msgr2=0x7f6b3c006580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:07:09.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:09.694+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c010e60 0x7f6b3c006580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:07:09.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:09.694+0000 7f6b4affd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0147a0 legacy=0x7f6b3c010750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:07:09.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:09.694+0000 7f6b5da19640 1 -- reap_dead start 2023-11-09T04:07:09.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:09.694+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0102e0 0x7f6b3c008f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:07:09.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:09.694+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008510 0x7f6b3c00f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:07:09.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:09.694+0000 7f6b4affd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b58070f40 con 0x7f6b3c0147a0 2023-11-09T04:07:09.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:09.694+0000 7f6b4affd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c008510 2023-11-09T04:07:09.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:09.694+0000 7f6b4affd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c008d20 con 0x7f6b3c0102e0 2023-11-09T04:07:09.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:09.694+0000 7f6b5da19640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0147a0 legacy=0x7f6b3c010750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:07:09.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:09.694+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008510 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:07:09.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:09.694+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0102e0 msgr2=0x7f6b3c008f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:07:09.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:09.694+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008510 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:07:09.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:09.694+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0102e0 0x7f6b3c008f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:07:09.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:09.895+0000 7f6b5da19640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0147a0 legacy=0x7f6b3c010750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:07:09.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:09.895+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008510 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:07:09.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:09.895+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0102e0 msgr2=0x7f6b3c008f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:07:09.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:09.895+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008510 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:07:09.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:09.895+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0102e0 0x7f6b3c008f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:07:10.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:10.296+0000 7f6b5da19640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0147a0 legacy=0x7f6b3c010750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:07:10.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:10.296+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0102e0 msgr2=0x7f6b3c008f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:07:10.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:10.296+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008510 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:07:10.298 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:10.296+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0102e0 0x7f6b3c008f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:07:10.298 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:10.296+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008510 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:07:11.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:11.097+0000 7f6b5da19640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0147a0 legacy=0x7f6b3c010750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2023-11-09T04:07:11.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:11.097+0000 7f6b5ca17640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008510 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2023-11-09T04:07:11.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:11.097+0000 7f6b5d218640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0102e0 msgr2=0x7f6b3c008f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3301/0 2023-11-09T04:07:11.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:11.097+0000 7f6b5ca17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008510 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:07:11.099 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:11.097+0000 7f6b5d218640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0102e0 0x7f6b3c008f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:07:12.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:12.694+0000 7f6b4affd640 1 -- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0102e0 msgr2=0x7f6b3c008f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:07:12.695 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:12.694+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b3c0102e0 0x7f6b3c008f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:07:12.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:12.694+0000 7f6b4affd640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008510 msgr2=0x7f6b3c00f170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:07:12.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:12.694+0000 7f6b4affd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b3c008510 0x7f6b3c00f170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-09T04:07:12.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:12.695+0000 7f6b4affd640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f6b3c0147a0 legacy=0x7f6b3c010750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-09T04:07:12.696 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:12.695+0000 7f6b4affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c010e60 0x7f6b3c007590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-09T04:07:12.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:12.695+0000 7f6b4affd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b58071500 con 0x7f6b3c00e650 2023-11-09T04:07:12.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:12.695+0000 7f6b4affd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58070c60 con 0x7f6b3c010e60 2023-11-09T04:07:12.697 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:12.695+0000 7f6b5d218640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c010e60 msgr2=0x7f6b3c007590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:07:12.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:12.695+0000 7f6b5d218640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c010e60 0x7f6b3c007590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-09T04:07:12.698 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:12.695+0000 7f6b5ca17640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00e650 legacy=0x7f6b3c011e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:07:12.895 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:12.896+0000 7f6b5d218640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c010e60 msgr2=0x7f6b3c007590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:07:12.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:12.896+0000 7f6b5d218640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c010e60 0x7f6b3c007590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-09T04:07:12.896 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:12.896+0000 7f6b5ca17640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00e650 legacy=0x7f6b3c011e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:07:13.296 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:13.297+0000 7f6b5d218640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c010e60 msgr2=0x7f6b3c007590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:07:13.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:13.297+0000 7f6b5d218640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c010e60 0x7f6b3c007590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-09T04:07:13.297 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:13.297+0000 7f6b5ca17640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00e650 legacy=0x7f6b3c011e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:07:14.097 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:14.098+0000 7f6b5d218640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c010e60 msgr2=0x7f6b3c007590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2023-11-09T04:07:14.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:14.098+0000 7f6b5d218640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c010e60 0x7f6b3c007590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-09T04:07:14.098 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-09T04:07:14.098+0000 7f6b5ca17640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f6b3c00e650 legacy=0x7f6b3c011e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2023-11-09T04:07:15.612 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-11-09T04:07:15.613 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 2758, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 2590, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 2356, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 1611, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi076 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-11-09T04:07:15.623 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-09T04:07:15.624 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-09T04:07:15.624 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T04:07:15.627 INFO:tasks.ceph.mds.a:Stopped 2023-11-09T04:07:15.627 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-09T04:07:15.628 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T04:07:15.628 INFO:tasks.ceph.mds.c:Stopped 2023-11-09T04:07:15.629 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-09T04:07:15.629 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T04:07:15.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T04:07:15.630 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 2758, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 2590, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 2356, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 1611, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi076 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi113 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b' 2023-11-09T04:07:15.642 INFO:tasks.ceph.mds.b:Stopped 2023-11-09T04:07:15.642 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-09T04:07:15.643 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T04:07:15.643 INFO:tasks.ceph.mds.d:Stopped 2023-11-09T04:07:15.644 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-09T04:07:15.644 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-09T04:07:15.645 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T04:07:15.645 INFO:tasks.ceph.osd.0:Stopped 2023-11-09T04:07:15.646 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-09T04:07:15.646 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T04:07:15.647 INFO:tasks.ceph.osd.1:Stopped 2023-11-09T04:07:15.647 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-09T04:07:15.647 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T04:07:15.648 INFO:tasks.ceph.osd.2:Stopped 2023-11-09T04:07:15.648 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-09T04:07:15.649 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T04:07:15.649 INFO:tasks.ceph.osd.3:Stopped 2023-11-09T04:07:15.650 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-09T04:07:15.650 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T04:07:15.651 INFO:tasks.ceph.osd.4:Stopped 2023-11-09T04:07:15.651 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-09T04:07:15.652 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T04:07:15.652 INFO:tasks.ceph.osd.5:Stopped 2023-11-09T04:07:15.653 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-09T04:07:15.653 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T04:07:15.654 INFO:tasks.ceph.osd.6:Stopped 2023-11-09T04:07:15.654 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-09T04:07:15.655 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T04:07:15.655 INFO:tasks.ceph.osd.7:Stopped 2023-11-09T04:07:15.656 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-09T04:07:15.656 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-09T04:07:15.656 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T04:07:15.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T04:07:15.657 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 2758, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 2590, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 2356, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 1611, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi076 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi076 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y' 2023-11-09T04:07:15.667 INFO:tasks.ceph.mgr.y:Stopped 2023-11-09T04:07:15.667 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-09T04:07:15.668 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T04:07:15.668 INFO:tasks.ceph.mgr.x:Stopped 2023-11-09T04:07:15.669 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-09T04:07:15.669 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-09T04:07:15.670 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T04:07:15.670 INFO:tasks.ceph.mon.a:Stopped 2023-11-09T04:07:15.671 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-09T04:07:15.671 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T04:07:15.672 INFO:tasks.ceph.mon.b:Stopped 2023-11-09T04:07:15.672 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-09T04:07:15.673 INFO:teuthology.orchestra.run:waiting for 300 2023-11-09T04:07:15.673 INFO:tasks.ceph.mon.c:Stopped 2023-11-09T04:07:15.674 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-09T04:07:15.674 DEBUG:teuthology.orchestra.run.smithi076:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'evicting unresponsive client' | egrep -v RECENT_CRASH | head -n 1 2023-11-09T04:07:15.774 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T04:07:15.774 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-09T04:07:15.897 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T04:07:15.898 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-09T04:07:15.976 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T04:07:15.977 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-09T04:07:16.034 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T04:07:16.034 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-09T04:07:16.108 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi113.front.sepia.ceph.com 2023-11-09T04:07:16.109 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-09T04:07:16.304 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi113.front.sepia.ceph.com 2023-11-09T04:07:16.305 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-09T04:07:16.352 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi113.front.sepia.ceph.com 2023-11-09T04:07:16.353 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-09T04:07:16.403 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi113.front.sepia.ceph.com 2023-11-09T04:07:16.404 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-09T04:07:16.435 INFO:tasks.ceph:Archiving mon data... 2023-11-09T04:07:16.436 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2023-11-09_03:21:01-fs:functional-main-distro-default-smithi/7452464/data/mon.a.tgz 2023-11-09T04:07:16.437 DEBUG:teuthology.orchestra.run.smithi076:> mktemp 2023-11-09T04:07:16.453 INFO:teuthology.orchestra.run.smithi076.stdout:/tmp/tmp.uxjRMyS8Cc 2023-11-09T04:07:16.454 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.uxjRMyS8Cc 2023-11-09T04:07:16.623 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0666 /tmp/tmp.uxjRMyS8Cc 2023-11-09T04:07:16.707 DEBUG:teuthology.orchestra.remote:smithi076:/tmp/tmp.uxjRMyS8Cc is 475KB 2023-11-09T04:07:16.804 DEBUG:teuthology.orchestra.run.smithi076:> rm -fr /tmp/tmp.uxjRMyS8Cc 2023-11-09T04:07:16.819 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2023-11-09_03:21:01-fs:functional-main-distro-default-smithi/7452464/data/mon.b.tgz 2023-11-09T04:07:16.819 DEBUG:teuthology.orchestra.run.smithi113:> mktemp 2023-11-09T04:07:16.836 INFO:teuthology.orchestra.run.smithi113.stdout:/tmp/tmp.yCLWgEYlqM 2023-11-09T04:07:16.837 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.yCLWgEYlqM 2023-11-09T04:07:17.022 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0666 /tmp/tmp.yCLWgEYlqM 2023-11-09T04:07:17.107 DEBUG:teuthology.orchestra.remote:smithi113:/tmp/tmp.yCLWgEYlqM is 489KB 2023-11-09T04:07:17.204 DEBUG:teuthology.orchestra.run.smithi113:> rm -fr /tmp/tmp.yCLWgEYlqM 2023-11-09T04:07:17.218 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2023-11-09_03:21:01-fs:functional-main-distro-default-smithi/7452464/data/mon.c.tgz 2023-11-09T04:07:17.219 DEBUG:teuthology.orchestra.run.smithi113:> mktemp 2023-11-09T04:07:17.272 INFO:teuthology.orchestra.run.smithi113.stdout:/tmp/tmp.2WccKc2lC0 2023-11-09T04:07:17.272 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.2WccKc2lC0 2023-11-09T04:07:17.436 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0666 /tmp/tmp.2WccKc2lC0 2023-11-09T04:07:17.521 DEBUG:teuthology.orchestra.remote:smithi113:/tmp/tmp.2WccKc2lC0 is 489KB 2023-11-09T04:07:17.618 DEBUG:teuthology.orchestra.run.smithi113:> rm -fr /tmp/tmp.2WccKc2lC0 2023-11-09T04:07:17.632 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-09T04:07:17.633 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-09T04:07:17.635 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-09T04:07:17.699 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-09T04:07:17.699 DEBUG:teuthology.orchestra.run.smithi076:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-09T04:07:17.705 DEBUG:teuthology.orchestra.run.smithi113:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-09T04:07:17.733 INFO:teuthology.orchestra.run.smithi076.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-09T04:07:17.768 INFO:teuthology.orchestra.run.smithi113.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-09T04:07:17.772 INFO:tasks.ceph:Archiving crash dumps... 2023-11-09T04:07:17.774 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2023-11-09_03:21:01-fs:functional-main-distro-default-smithi/7452464/remote/smithi076/crash 2023-11-09T04:07:17.776 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-09T04:07:17.894 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2023-11-09_03:21:01-fs:functional-main-distro-default-smithi/7452464/remote/smithi113/crash 2023-11-09T04:07:17.896 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-09T04:07:17.988 INFO:tasks.ceph:Shutting down logrotate 2023-11-09T04:07:17.989 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:07:17.991 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-09T04:07:18.080 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-09T04:07:18.107 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-09T04:07:18.133 INFO:tasks.ceph:Compressing logs... 2023-11-09T04:07:18.134 DEBUG:teuthology.orchestra.run.smithi076:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-09T04:07:18.150 DEBUG:teuthology.orchestra.run.smithi113:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-09T04:07:18.226 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-11-09T04:07:18.226 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.93700.log 2023-11-09T04:07:18.227 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.94152.log 2023-11-09T04:07:18.227 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.1.93700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88108.log 2023-11-09T04:07:18.227 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.3.94152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88188.log 2023-11-09T04:07:18.227 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-09T04:07:18.228 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88108.log: /var/log/ceph/ceph-client.admin.88188.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-11-09T04:07:18.228 INFO:teuthology.orchestra.run.smithi113.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.1.93700.log.gz 2023-11-09T04:07:18.228 INFO:teuthology.orchestra.run.smithi113.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88108.log.gz 2023-11-09T04:07:18.228 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph.audit.log: 91.4% -- replaced with /var/log/ceph/ceph-client.3.94152.log.gz 2023-11-09T04:07:18.229 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88188.log.gz 2023-11-09T04:07:18.229 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-11-09T04:07:18.229 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-11-09T04:07:18.230 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88348.log 2023-11-09T04:07:18.230 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-11-09T04:07:18.230 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88228.log 2023-11-09T04:07:18.230 INFO:teuthology.orchestra.run.smithi113.stderr: 88.0%/var/log/ceph/ceph-osd.5.log: -- replaced with /var/log/ceph/ceph-client.admin.88348.log.gz 2023-11-09T04:07:18.231 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88068.log 2023-11-09T04:07:18.231 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88228.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-11-09T04:07:18.231 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88228.log.gz 2023-11-09T04:07:18.231 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88148.log 2023-11-09T04:07:18.232 INFO:teuthology.orchestra.run.smithi113.stderr: 87.5%/var/log/ceph/ceph-mgr.x.log: -- replaced with /var/log/ceph/ceph-client.admin.88068.log.gz 2023-11-09T04:07:18.232 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88268.log 2023-11-09T04:07:18.232 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88148.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.88148.log.gz 2023-11-09T04:07:18.233 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-09T04:07:18.233 INFO:teuthology.orchestra.run.smithi113.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-09T04:07:18.233 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88268.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-11-09T04:07:18.233 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.88268.log.gz 2023-11-09T04:07:18.234 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-11-09T04:07:18.234 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mon.b.log: 93.2% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-09T04:07:18.234 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88308.log 2023-11-09T04:07:18.234 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mds.d.log: 93.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-11-09T04:07:18.235 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-11-09T04:07:18.235 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.88308.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.88308.log.gz 2023-11-09T04:07:18.336 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mon.c.log: 91.3% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-11-09T04:07:18.404 INFO:teuthology.orchestra.run.smithi113.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-11-09T04:07:18.484 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101176.log 2023-11-09T04:07:18.484 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93583.log 2023-11-09T04:07:18.485 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.101176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101914.log 2023-11-09T04:07:18.485 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.93583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128124.log 2023-11-09T04:07:18.485 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101176.log.gz 2023-11-09T04:07:18.485 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.101914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96937.log 2023-11-09T04:07:18.486 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.93583.log.gz 2023-11-09T04:07:18.486 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101914.log.gz 2023-11-09T04:07:18.486 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.128124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119743.log 2023-11-09T04:07:18.486 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.128124.log.gz 2023-11-09T04:07:18.487 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.96937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103043.log 2023-11-09T04:07:18.487 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.119743.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.96937.log.gz 2023-11-09T04:07:18.487 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130885.log 2023-11-09T04:07:18.487 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.103043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114204.log 2023-11-09T04:07:18.488 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.119743.log.gz 2023-11-09T04:07:18.488 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.130885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130633.log 2023-11-09T04:07:18.488 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103043.log.gz 2023-11-09T04:07:18.488 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.114204.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.130885.log.gz 2023-11-09T04:07:18.489 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110499.log 2023-11-09T04:07:18.489 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.130633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121783.log 2023-11-09T04:07:18.489 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114204.log.gz 2023-11-09T04:07:18.489 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.110499.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.130633.log.gz 2023-11-09T04:07:18.490 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132907.log 2023-11-09T04:07:18.490 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.121783.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110499.log.gz -5 2023-11-09T04:07:18.490 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.100803.log 2023-11-09T04:07:18.491 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.132907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106911.log 2023-11-09T04:07:18.491 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.121783.log.gz 2023-11-09T04:07:18.491 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.100803.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.132907.log.gz 2023-11-09T04:07:18.491 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101688.log 2023-11-09T04:07:18.492 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.106911.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100803.log.gz 2023-11-09T04:07:18.492 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113530.log 2023-11-09T04:07:18.492 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.101688.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.106911.log.gz 2023-11-09T04:07:18.493 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109855.log 2023-11-09T04:07:18.493 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.113530.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101688.log.gz 2023-11-09T04:07:18.493 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-11-09T04:07:18.494 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.109855.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.113530.log.gz 2023-11-09T04:07:18.494 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125216.log 2023-11-09T04:07:18.494 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-mon.a.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.109855.log.gz 2023-11-09T04:07:18.494 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119124.log 2023-11-09T04:07:18.495 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.125216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99094.log 2023-11-09T04:07:18.495 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.119124.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.125216.log.gz 2023-11-09T04:07:18.495 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126501.log 2023-11-09T04:07:18.496 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.119124.log.gz 2023-11-09T04:07:18.496 INFO:teuthology.orchestra.run.smithi076.stderr:gzip/var/log/ceph/ceph-client.admin.99094.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.120066.log 2023-11-09T04:07:18.496 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.126501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94027.log 2023-11-09T04:07:18.497 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99094.log.gz 2023-11-09T04:07:18.497 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.126501.log.gz 2023-11-09T04:07:18.497 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.120066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97444.log 2023-11-09T04:07:18.498 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.94027.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.120066.log.gzgzip 2023-11-09T04:07:18.498 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.114461.log 2023-11-09T04:07:18.498 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.97444.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.94027.log.gz 2023-11-09T04:07:18.498 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.105043.log 2023-11-09T04:07:18.499 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97444.log.gz 2023-11-09T04:07:18.499 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.0.114461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111772.log 2023-11-09T04:07:18.499 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.105043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132662.log 2023-11-09T04:07:18.499 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.111772.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.105043.log.gz 2023-11-09T04:07:18.500 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128798.log 2023-11-09T04:07:18.500 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94111.log 2023-11-09T04:07:18.500 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111772.log.gz 2023-11-09T04:07:18.500 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.132662.log: /var/log/ceph/ceph-client.admin.128798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128451.log 2023-11-09T04:07:18.501 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.132662.log.gz 2023-11-09T04:07:18.501 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9%/var/log/ceph/ceph-client.admin.94111.log: -- replaced with /var/log/ceph/ceph-client.admin.128798.log.gz 2023-11-09T04:07:18.501 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131405.log 2023-11-09T04:07:18.501 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.128451.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94111.log.gz 2023-11-09T04:07:18.502 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132255.log 2023-11-09T04:07:18.502 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.128451.log.gz 2023-11-09T04:07:18.502 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108655.log 2023-11-09T04:07:18.503 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.131405.log: /var/log/ceph/ceph-client.admin.132255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99588.log 2023-11-09T04:07:18.503 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.131405.log.gz 2023-11-09T04:07:18.503 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.108655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129342.log 2023-11-09T04:07:18.503 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.132255.log.gz 2023-11-09T04:07:18.504 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.108655.log.gz 2023-11-09T04:07:18.504 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.99588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122810.log 2023-11-09T04:07:18.504 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.129342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111288.log 2023-11-09T04:07:18.504 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99588.log.gz 2023-11-09T04:07:18.505 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.122810.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.129342.log.gz 2023-11-09T04:07:18.505 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117454.log 2023-11-09T04:07:18.505 INFO:teuthology.orchestra.run.smithi076.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.0.114461.log.gz 2023-11-09T04:07:18.505 INFO:teuthology.orchestra.run.smithi076.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.122810.log.gz 2023-11-09T04:07:18.506 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.111288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132949.log 2023-11-09T04:07:18.506 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.117454.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.111288.log.gz 2023-11-09T04:07:18.506 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99714.log 2023-11-09T04:07:18.506 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.132949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113572.log 2023-11-09T04:07:18.507 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.99714.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.117454.log.gz 2023-11-09T04:07:18.507 INFO:teuthology.orchestra.run.smithi076.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.132949.log.gz 2023-11-09T04:07:18.507 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93711.log 2023-11-09T04:07:18.507 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99714.log.gz 2023-11-09T04:07:18.508 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.113572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129384.log 2023-11-09T04:07:18.508 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.93711.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.113572.log.gz 2023-11-09T04:07:18.508 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108172.log 2023-11-09T04:07:18.508 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.129384.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93711.log.gz 2023-11-09T04:07:18.509 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129300.log 2023-11-09T04:07:18.509 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.108172.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.129384.log.gz 2023-11-09T04:07:18.509 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105541.log 2023-11-09T04:07:18.509 INFO:teuthology.orchestra.run.smithi076.stderr: 87.2%/var/log/ceph/ceph-client.admin.129300.log: -- replaced with /var/log/ceph/ceph-client.admin.108172.log.gz 2023-11-09T04:07:18.510 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100492.log 2023-11-09T04:07:18.510 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.129300.log.gz 2023-11-09T04:07:18.510 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.105541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102875.log 2023-11-09T04:07:18.510 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.100492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97486.log 2023-11-09T04:07:18.511 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105541.log.gz 2023-11-09T04:07:18.511 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.102875.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100492.log.gz 2023-11-09T04:07:18.511 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117098.log 2023-11-09T04:07:18.511 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102875.log.gz 2023-11-09T04:07:18.512 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.97486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118136.log 2023-11-09T04:07:18.512 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.117098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106783.log 2023-11-09T04:07:18.512 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97486.log.gz 2023-11-09T04:07:18.512 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.118136.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.117098.log.gz 2023-11-09T04:07:18.513 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118571.log 2023-11-09T04:07:18.513 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.106783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126184.log 2023-11-09T04:07:18.513 INFO:teuthology.orchestra.run.smithi076.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.118136.log.gz 2023-11-09T04:07:18.514 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.118571.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.106783.log.gz 2023-11-09T04:07:18.514 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115508.log 2023-11-09T04:07:18.514 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107175.log 2023-11-09T04:07:18.514 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118571.log.gz 2023-11-09T04:07:18.515 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.126184.log: /var/log/ceph/ceph-client.admin.115508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97276.log 2023-11-09T04:07:18.515 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.126184.log.gz 2023-11-09T04:07:18.515 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.107175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134851.log 2023-11-09T04:07:18.515 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115508.log.gz 2023-11-09T04:07:18.516 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.97276.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.107175.log.gz 2023-11-09T04:07:18.516 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105625.log 2023-11-09T04:07:18.516 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97276.log.gz 2023-11-09T04:07:18.516 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.134851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121002.log 2023-11-09T04:07:18.517 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 88.7% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.134851.log.gz 2023-11-09T04:07:18.517 INFO:teuthology.orchestra.run.smithi076.stderr: /var/log/ceph/ceph-client.admin.103835.log 2023-11-09T04:07:18.517 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.105625.log: /var/log/ceph/ceph-client.admin.121002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115298.log 2023-11-09T04:07:18.517 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105625.log.gz 2023-11-09T04:07:18.518 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.103835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113742.log 2023-11-09T04:07:18.518 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.121002.log.gz 2023-11-09T04:07:18.518 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.115298.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103835.log.gz 2023-11-09T04:07:18.518 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105835.log 2023-11-09T04:07:18.519 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.115298.log.gz 2023-11-09T04:07:18.519 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.113742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116420.log 2023-11-09T04:07:18.519 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.105835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130416.log 2023-11-09T04:07:18.519 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.113742.log.gz 2023-11-09T04:07:18.520 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.116420.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105835.log.gz 2023-11-09T04:07:18.520 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113098.log 2023-11-09T04:07:18.520 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116420.log.gz 2023-11-09T04:07:18.520 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.130416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111026.log 2023-11-09T04:07:18.521 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.113098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122298.log 2023-11-09T04:07:18.521 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.130416.log.gz 2023-11-09T04:07:18.521 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0%/var/log/ceph/ceph-client.admin.111026.log: -- replaced with /var/log/ceph/ceph-client.admin.113098.log.gz 2023-11-09T04:07:18.521 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103001.log 2023-11-09T04:07:18.522 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111026.log.gz 2023-11-09T04:07:18.522 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.122298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102707.log 2023-11-09T04:07:18.522 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.103001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106741.log 2023-11-09T04:07:18.523 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.122298.log.gz 2023-11-09T04:07:18.523 INFO:teuthology.orchestra.run.smithi076.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.103001.log.gz 2023-11-09T04:07:18.523 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.102707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119382.log 2023-11-09T04:07:18.523 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.106741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105261.log 2023-11-09T04:07:18.524 INFO:teuthology.orchestra.run.smithi076.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.102707.log.gz 2023-11-09T04:07:18.524 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.119382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103085.log 2023-11-09T04:07:18.524 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106741.log.gz 2023-11-09T04:07:18.524 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.105261.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.119382.log.gz 2023-11-09T04:07:18.525 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108781.log 2023-11-09T04:07:18.525 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.103085.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105261.log.gz 2023-11-09T04:07:18.525 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96938.log 2023-11-09T04:07:18.526 INFO:teuthology.orchestra.run.smithi076.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103085.log.gz 2023-11-09T04:07:18.526 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.108781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93237.log 2023-11-09T04:07:18.526 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.96938.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.108781.log.gz 2023-11-09T04:07:18.526 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129098.log 2023-11-09T04:07:18.527 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.93237.log: gzip -5 89.1% --verbose -- replaced with /var/log/ceph/ceph-client.admin.96938.log.gz -- 2023-11-09T04:07:18.527 INFO:teuthology.orchestra.run.smithi076.stderr: /var/log/ceph/ceph-client.admin.104922.log 2023-11-09T04:07:18.527 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.129098.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.93237.log.gz 2023-11-09T04:07:18.527 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.110864.log 2023-11-09T04:07:18.528 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.129098.log.gz 2023-11-09T04:07:18.528 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.104922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116241.log 2023-11-09T04:07:18.528 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.0.110864.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.104922.log.gz 2023-11-09T04:07:18.528 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113182.log 2023-11-09T04:07:18.529 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.116241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119830.log 2023-11-09T04:07:18.529 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2%/var/log/ceph/ceph-client.admin.113182.log: -- replaced with /var/log/ceph/ceph-client.admin.116241.log.gz 2023-11-09T04:07:18.529 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114934.log 2023-11-09T04:07:18.530 INFO:teuthology.orchestra.run.smithi076.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.0.110864.log.gz 2023-11-09T04:07:18.530 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.113182.log.gz 2023-11-09T04:07:18.530 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102791.log 2023-11-09T04:07:18.530 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.119830.log: /var/log/ceph/ceph-client.admin.114934.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.119830.log.gz -5 2023-11-09T04:07:18.531 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.98094.log 2023-11-09T04:07:18.531 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.102791.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114934.log.gz 2023-11-09T04:07:18.531 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124471.log 2023-11-09T04:07:18.531 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.98094.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.102791.log.gz 2023-11-09T04:07:18.532 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122453.log 2023-11-09T04:07:18.532 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.124471.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98094.log.gz 2023-11-09T04:07:18.532 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111247.log 2023-11-09T04:07:18.532 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.124471.log.gz 2023-11-09T04:07:18.533 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.122453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127214.log 2023-11-09T04:07:18.533 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.111247.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.122453.log.gz 2023-11-09T04:07:18.533 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119503.log 2023-11-09T04:07:18.534 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.127214.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.111247.log.gz 2023-11-09T04:07:18.534 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106021.log 2023-11-09T04:07:18.534 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.127214.log.gz 2023-11-09T04:07:18.534 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.119503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113350.log 2023-11-09T04:07:18.535 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.106021.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.119503.log.gz 2023-11-09T04:07:18.535 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97966.log 2023-11-09T04:07:18.535 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.113350.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106021.log.gz 2023-11-09T04:07:18.535 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103169.log 2023-11-09T04:07:18.536 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.113350.log.gz 2023-11-09T04:07:18.536 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.97966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132600.log 2023-11-09T04:07:18.536 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.103169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113308.log 2023-11-09T04:07:18.536 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103169.log.gz/var/log/ceph/ceph-client.admin.132600.log: 2023-11-09T04:07:18.537 INFO:teuthology.orchestra.run.smithi076.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.97966.log.gz 2023-11-09T04:07:18.537 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122340.log 2023-11-09T04:07:18.537 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.132600.log.gz 2023-11-09T04:07:18.537 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.113308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94195.log 2023-11-09T04:07:18.538 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.122340.log: gzip -5 --verbose 88.7% -- -- replaced with /var/log/ceph/ceph-client.admin.113308.log.gz /var/log/ceph/ceph-client.admin.111070.log 2023-11-09T04:07:18.538 INFO:teuthology.orchestra.run.smithi076.stderr: 2023-11-09T04:07:18.538 INFO:teuthology.orchestra.run.smithi076.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.122340.log.gz/var/log/ceph/ceph-client.admin.94195.log: 2023-11-09T04:07:18.539 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100703.log 2023-11-09T04:07:18.539 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.111070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109304.log 2023-11-09T04:07:18.539 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.100703.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.111070.log.gz 2023-11-09T04:07:18.539 INFO:teuthology.orchestra.run.smithi076.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.94195.log.gz 2023-11-09T04:07:18.540 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106316.log 2023-11-09T04:07:18.540 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100703.log.gz 2023-11-09T04:07:18.540 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.109304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115018.log 2023-11-09T04:07:18.540 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.106316.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109304.log.gz 2023-11-09T04:07:18.541 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108823.log 2023-11-09T04:07:18.541 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.115018.log: 89.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.106316.log.gz 2023-11-09T04:07:18.541 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.95253.log 2023-11-09T04:07:18.541 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.108823.log: 89.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.115018.log.gz -5 2023-11-09T04:07:18.542 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.106106.log 2023-11-09T04:07:18.542 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.95253.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108823.log.gz 2023-11-09T04:07:18.542 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100134.log 2023-11-09T04:07:18.543 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95253.log.gz 2023-11-09T04:07:18.543 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.106106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110025.log 2023-11-09T04:07:18.543 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.106106.log.gz 2023-11-09T04:07:18.543 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.100134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118754.log 2023-11-09T04:07:18.544 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.110025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94664.log 2023-11-09T04:07:18.544 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.100134.log.gz 2023-11-09T04:07:18.544 INFO:teuthology.orchestra.run.smithi076.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.110025.log.gz 2023-11-09T04:07:18.544 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.118754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126563.log 2023-11-09T04:07:18.545 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.118754.log.gz 2023-11-09T04:07:18.545 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.94664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96699.log 2023-11-09T04:07:18.545 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.126563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116665.log 2023-11-09T04:07:18.545 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.126563.log.gz 2023-11-09T04:07:18.546 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.96699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120960.log 2023-11-09T04:07:18.546 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96346.log 2023-11-09T04:07:18.546 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.116665.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.96699.log.gz 2023-11-09T04:07:18.547 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.120960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121129.log 2023-11-09T04:07:18.547 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116665.log.gz 2023-11-09T04:07:18.547 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.96346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135077.log 2023-11-09T04:07:18.547 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120960.log.gz 2023-11-09T04:07:18.547 INFO:teuthology.orchestra.run.smithi076.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.94664.log.gz 2023-11-09T04:07:18.548 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.121129.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96346.log.gz 2023-11-09T04:07:18.548 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120731.log 2023-11-09T04:07:18.548 INFO:teuthology.orchestra.run.smithi076.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.121129.log.gz 2023-11-09T04:07:18.549 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.131924.log 2023-11-09T04:07:18.549 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.135077.log: /var/log/ceph/ceph-client.admin.120731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134641.log 2023-11-09T04:07:18.549 INFO:teuthology.orchestra.run.smithi076.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.135077.log.gz 2023-11-09T04:07:18.549 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.0.131924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126823.log 2023-11-09T04:07:18.550 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.134641.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.120731.log.gz 2023-11-09T04:07:18.550 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134993.log 2023-11-09T04:07:18.550 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.126823.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.134641.log.gz 2023-11-09T04:07:18.550 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113140.log 2023-11-09T04:07:18.551 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.126823.log.gz 2023-11-09T04:07:18.551 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112421.log 2023-11-09T04:07:18.551 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.134993.log: /var/log/ceph/ceph-client.admin.113140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99378.log 2023-11-09T04:07:18.551 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.134993.log.gz 2023-11-09T04:07:18.552 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.112421.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113140.log.gz 2023-11-09T04:07:18.552 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116876.log 2023-11-09T04:07:18.552 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.99378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99630.log 2023-11-09T04:07:18.552 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.112421.log.gz 2023-11-09T04:07:18.553 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.116876.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99378.log.gz 2023-11-09T04:07:18.553 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106699.log 2023-11-09T04:07:18.553 INFO:teuthology.orchestra.run.smithi076.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.0.131924.log.gz 2023-11-09T04:07:18.553 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.116876.log.gz 2023-11-09T04:07:18.554 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118898.log 2023-11-09T04:07:18.554 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.99630.log: /var/log/ceph/ceph-client.admin.106699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113436.log 2023-11-09T04:07:18.554 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99630.log.gz 2023-11-09T04:07:18.554 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.118898.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.106699.log.gzgzip 2023-11-09T04:07:18.555 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.133325.log 2023-11-09T04:07:18.555 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.113436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132724.log 2023-11-09T04:07:18.555 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118898.log.gz 2023-11-09T04:07:18.555 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.133325.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.113436.log.gz 2023-11-09T04:07:18.556 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117412.log 2023-11-09T04:07:18.556 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.133325.log.gz 2023-11-09T04:07:18.556 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.132724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94901.log 2023-11-09T04:07:18.556 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.117412.log: gzip -5 --verbose 88.7% -- /var/log/ceph/ceph-client.admin.102329.log -- replaced with /var/log/ceph/ceph-client.admin.132724.log.gz 2023-11-09T04:07:18.557 INFO:teuthology.orchestra.run.smithi076.stderr: 2023-11-09T04:07:18.557 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.94901.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117412.log.gz 2023-11-09T04:07:18.557 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116920.log 2023-11-09T04:07:18.557 INFO:teuthology.orchestra.run.smithi076.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.94901.log.gz 2023-11-09T04:07:18.558 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.102329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119690.log 2023-11-09T04:07:18.558 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.116920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134279.log 2023-11-09T04:07:18.558 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.102329.log.gz 2023-11-09T04:07:18.558 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.119690.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.116920.log.gz 2023-11-09T04:07:18.559 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109897.log 2023-11-09T04:07:18.559 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.119690.log.gz 2023-11-09T04:07:18.559 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.134279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133411.log 2023-11-09T04:07:18.559 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.109897.log: gzip -5 --verbose -- 88.1% /var/log/ceph/ceph-client.admin.95337.log -- replaced with /var/log/ceph/ceph-client.admin.134279.log.gz 2023-11-09T04:07:18.560 INFO:teuthology.orchestra.run.smithi076.stderr: 2023-11-09T04:07:18.560 INFO:teuthology.orchestra.run.smithi076.stderr:gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.109897.log.gz -5 2023-11-09T04:07:18.560 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.111492.log 2023-11-09T04:07:18.560 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.133411.log: /var/log/ceph/ceph-client.admin.95337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120207.log 2023-11-09T04:07:18.561 INFO:teuthology.orchestra.run.smithi076.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.133411.log.gz 2023-11-09T04:07:18.561 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.111492.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95337.log.gz 2023-11-09T04:07:18.561 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120108.log 2023-11-09T04:07:18.561 INFO:teuthology.orchestra.run.smithi076.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.111492.log.gz 2023-11-09T04:07:18.562 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.120207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133537.log 2023-11-09T04:07:18.562 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.120108.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.120207.log.gz 2023-11-09T04:07:18.562 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121086.log 2023-11-09T04:07:18.562 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.133537.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120108.log.gz 2023-11-09T04:07:18.563 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120839.log 2023-11-09T04:07:18.563 INFO:teuthology.orchestra.run.smithi076.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.133537.log.gz 2023-11-09T04:07:18.563 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.121086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124848.log 2023-11-09T04:07:18.563 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.120839.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.121086.log.gz 2023-11-09T04:07:18.564 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123778.log 2023-11-09T04:07:18.564 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.124848.log: gzip -5 88.9% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.120839.log.gz 2023-11-09T04:07:18.564 INFO:teuthology.orchestra.run.smithi076.stderr: /var/log/ceph/ceph-client.admin.96696.log 2023-11-09T04:07:18.565 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.124848.log.gz 2023-11-09T04:07:18.565 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.123778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108459.log 2023-11-09T04:07:18.565 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.96696.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.123778.log.gz 2023-11-09T04:07:18.565 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119340.log 2023-11-09T04:07:18.566 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.108459.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96696.log.gz 2023-11-09T04:07:18.566 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109220.log 2023-11-09T04:07:18.566 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108459.log.gz 2023-11-09T04:07:18.566 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.119340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88105.log 2023-11-09T04:07:18.567 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.109220.log: gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.119340.log.gz -5 2023-11-09T04:07:18.567 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.115424.log 2023-11-09T04:07:18.567 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.88105.log: gzip -5 --verbose 88.8% -- -- replaced with /var/log/ceph/ceph-client.admin.109220.log.gz 2023-11-09T04:07:18.567 INFO:teuthology.orchestra.run.smithi076.stderr: /var/log/ceph/ceph-client.admin.103497.log 2023-11-09T04:07:18.568 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.88105.log.gz 2023-11-09T04:07:18.568 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.115424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128633.log 2023-11-09T04:07:18.568 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.103497.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.115424.log.gzgzip 2023-11-09T04:07:18.569 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.125812.log 2023-11-09T04:07:18.569 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.128633.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103497.log.gz 2023-11-09T04:07:18.569 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114976.log 2023-11-09T04:07:18.569 INFO:teuthology.orchestra.run.smithi076.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.128633.log.gz 2023-11-09T04:07:18.570 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.125812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134133.log 2023-11-09T04:07:18.570 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.114976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109771.log 2023-11-09T04:07:18.570 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.134133.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.125812.log.gz 2023-11-09T04:07:18.570 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.114976.log.gz 2023-11-09T04:07:18.571 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126124.log 2023-11-09T04:07:18.571 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.134133.log.gz 2023-11-09T04:07:18.571 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.109771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103877.log 2023-11-09T04:07:18.571 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.126124.log: gzip -5 89.3% --verbose -- replaced with /var/log/ceph/ceph-client.admin.109771.log.gz -- 2023-11-09T04:07:18.572 INFO:teuthology.orchestra.run.smithi076.stderr: /var/log/ceph/ceph-client.admin.124636.log 2023-11-09T04:07:18.572 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.103877.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.126124.log.gz -5 2023-11-09T04:07:18.572 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.111982.log 2023-11-09T04:07:18.572 INFO:teuthology.orchestra.run.smithi076.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103877.log.gz 2023-11-09T04:07:18.573 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.124636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99882.log 2023-11-09T04:07:18.573 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.111982.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.124636.log.gz 2023-11-09T04:07:18.573 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113868.log 2023-11-09T04:07:18.573 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.99882.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.111982.log.gz 2023-11-09T04:07:18.574 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133075.log 2023-11-09T04:07:18.574 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.113868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104269.log 2023-11-09T04:07:18.574 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99882.log.gz 2023-11-09T04:07:18.575 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.133075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130519.log 2023-11-09T04:07:18.575 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.113868.log.gz 2023-11-09T04:07:18.575 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.104269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103253.log 2023-11-09T04:07:18.575 INFO:teuthology.orchestra.run.smithi076.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.133075.log.gz 2023-11-09T04:07:18.576 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104269.log.gz 2023-11-09T04:07:18.576 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.130519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102371.log 2023-11-09T04:07:18.576 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.103253.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.130519.log.gz 2023-11-09T04:07:18.576 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109514.log 2023-11-09T04:07:18.577 INFO:teuthology.orchestra.run.smithi076.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103253.log.gz 2023-11-09T04:07:18.577 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.102371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100260.log 2023-11-09T04:07:18.577 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102371.log.gz 2023-11-09T04:07:18.577 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.109514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126082.log 2023-11-09T04:07:18.578 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.100260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121683.log 2023-11-09T04:07:18.578 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109514.log.gz 2023-11-09T04:07:18.578 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.126082.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100260.log.gz 2023-11-09T04:07:18.578 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95752.log 2023-11-09T04:07:18.579 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.126082.log.gz 2023-11-09T04:07:18.579 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.121683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104503.log 2023-11-09T04:07:18.579 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.95752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100971.log 2023-11-09T04:07:18.579 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.121683.log.gz 2023-11-09T04:07:18.580 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.104503.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95752.log.gz 2023-11-09T04:07:18.580 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135655.log 2023-11-09T04:07:18.580 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104503.log.gz 2023-11-09T04:07:18.580 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.100971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102040.log 2023-11-09T04:07:18.581 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.135655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.107936.log 2023-11-09T04:07:18.581 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100971.log.gz 2023-11-09T04:07:18.581 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.102040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118613.log 2023-11-09T04:07:18.581 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.102040.log.gz 2023-11-09T04:07:18.582 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.0.107936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102287.log 2023-11-09T04:07:18.582 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134809.log 2023-11-09T04:07:18.582 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.118613.log: /var/log/ceph/ceph-client.admin.102287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107511.log 2023-11-09T04:07:18.582 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.118613.log.gz 2023-11-09T04:07:18.583 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.0.107936.log.gz 2023-11-09T04:07:18.583 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.134809.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102287.log.gz 2023-11-09T04:07:18.583 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125982.log 2023-11-09T04:07:18.583 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.134809.log.gz 2023-11-09T04:07:18.584 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.107511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88002.log 2023-11-09T04:07:18.584 INFO:teuthology.orchestra.run.smithi076.stderr: 95.6% -- replaced with /var/log/ceph/ceph-client.admin.135655.log.gz 2023-11-09T04:07:18.584 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.125982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101134.log 2023-11-09T04:07:18.585 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.107511.log.gz 2023-11-09T04:07:18.585 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.125982.log.gz 2023-11-09T04:07:18.585 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.88002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105219.log 2023-11-09T04:07:18.585 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.101134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110331.log 87.5% 2023-11-09T04:07:18.586 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.88002.log.gz 2023-11-09T04:07:18.586 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.105219.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101134.log.gz 2023-11-09T04:07:18.586 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133453.log 2023-11-09T04:07:18.586 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.105219.log.gz 2023-11-09T04:07:18.587 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.110331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129258.log 2023-11-09T04:07:18.587 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.133453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126949.log 2023-11-09T04:07:18.587 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110331.log.gz 2023-11-09T04:07:18.587 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.129258.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.133453.log.gz 2023-11-09T04:07:18.588 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108417.log 2023-11-09T04:07:18.588 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.129258.log.gz 2023-11-09T04:07:18.588 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.126949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106657.log 2023-11-09T04:07:18.588 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.108417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101092.log 2023-11-09T04:07:18.589 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.126949.log.gz 2023-11-09T04:07:18.589 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.106657.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108417.log.gz 2023-11-09T04:07:18.589 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109939.log 2023-11-09T04:07:18.589 INFO:teuthology.orchestra.run.smithi076.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.106657.log.gz 2023-11-09T04:07:18.589 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.101092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97048.log 2023-11-09T04:07:18.590 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.109939.log: gzip -5 --verbose 88.4% -- /var/log/ceph/ceph-client.admin.108613.log -- replaced with /var/log/ceph/ceph-client.admin.101092.log.gz 2023-11-09T04:07:18.590 INFO:teuthology.orchestra.run.smithi076.stderr: 2023-11-09T04:07:18.590 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.97048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117520.log 2023-11-09T04:07:18.591 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.109939.log.gz 2023-11-09T04:07:18.591 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.108613.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97048.log.gz 2023-11-09T04:07:18.591 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133241.log 2023-11-09T04:07:18.591 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108613.log.gz 2023-11-09T04:07:18.592 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.117520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126605.log 2023-11-09T04:07:18.592 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.133241.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.117520.log.gz -5 2023-11-09T04:07:18.592 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.110457.log 2023-11-09T04:07:18.592 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.126605.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.133241.log.gz 2023-11-09T04:07:18.593 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132481.log 2023-11-09T04:07:18.593 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.126605.log.gz 2023-11-09T04:07:18.593 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.110457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96013.log 2023-11-09T04:07:18.593 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.132481.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.110457.log.gz 2023-11-09T04:07:18.594 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113658.log 2023-11-09T04:07:18.594 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.96013.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.132481.log.gz 2023-11-09T04:07:18.594 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124804.log 2023-11-09T04:07:18.594 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96013.log.gz 2023-11-09T04:07:18.595 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.113658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93333.log 2023-11-09T04:07:18.595 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.124804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126771.log 2023-11-09T04:07:18.595 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.113658.log.gz 2023-11-09T04:07:18.595 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.93333.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.124804.log.gz 2023-11-09T04:07:18.596 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95626.log 2023-11-09T04:07:18.596 INFO:teuthology.orchestra.run.smithi076.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.93333.log.gz 2023-11-09T04:07:18.596 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.126771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111618.log 2023-11-09T04:07:18.596 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.95626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112799.log 2023-11-09T04:07:18.597 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.126771.log.gz 2023-11-09T04:07:18.597 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.111618.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.95626.log.gz 2023-11-09T04:07:18.597 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125132.log 2023-11-09T04:07:18.597 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111618.log.gz 2023-11-09T04:07:18.598 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.112799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103455.log 2023-11-09T04:07:18.598 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.125132.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.112799.log.gzgzip 2023-11-09T04:07:18.598 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.112715.log 2023-11-09T04:07:18.599 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.103455.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.125132.log.gz 2023-11-09T04:07:18.599 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118365.log 2023-11-09T04:07:18.599 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0%/var/log/ceph/ceph-client.admin.112715.log: -- replaced with /var/log/ceph/ceph-client.admin.103455.log.gz 2023-11-09T04:07:18.599 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118796.log 2023-11-09T04:07:18.600 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.118365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93414.log 2023-11-09T04:07:18.600 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112715.log.gz 2023-11-09T04:07:18.600 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.118796.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.118365.log.gz 2023-11-09T04:07:18.600 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99756.log 2023-11-09T04:07:18.600 INFO:teuthology.orchestra.run.smithi076.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.118796.log.gz 2023-11-09T04:07:18.601 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.93414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.123533.log 2023-11-09T04:07:18.601 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9%/var/log/ceph/ceph-client.admin.99756.log: -- replaced with /var/log/ceph/ceph-client.admin.93414.log.gz 2023-11-09T04:07:18.601 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106148.log 2023-11-09T04:07:18.601 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.0.123533.log: 87.3%gzip -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.99756.log.gz 2023-11-09T04:07:18.602 INFO:teuthology.orchestra.run.smithi076.stderr: -- /var/log/ceph/ceph-client.admin.127466.log 2023-11-09T04:07:18.602 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.106148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116157.log 2023-11-09T04:07:18.602 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106148.log.gz 2023-11-09T04:07:18.602 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.127466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121298.log 2023-11-09T04:07:18.603 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124910.log 2023-11-09T04:07:18.603 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.127466.log.gz 2023-11-09T04:07:18.603 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.116157.log: /var/log/ceph/ceph-client.admin.121298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111814.log 2023-11-09T04:07:18.603 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116157.log.gz 2023-11-09T04:07:18.604 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124720.log 2023-11-09T04:07:18.604 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.124910.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.121298.log.gz 2023-11-09T04:07:18.604 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.111814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132865.log 88.9% 2023-11-09T04:07:18.605 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.124910.log.gz 2023-11-09T04:07:18.605 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105099.log 2023-11-09T04:07:18.605 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.124720.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111814.log.gz 2023-11-09T04:07:18.605 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.132865.log: gzip 88.2% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.124720.log.gz 2023-11-09T04:07:18.606 INFO:teuthology.orchestra.run.smithi076.stderr: -- /var/log/ceph/ceph-client.admin.116834.log 2023-11-09T04:07:18.606 INFO:teuthology.orchestra.run.smithi076.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.0.123533.log.gz 2023-11-09T04:07:18.606 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.132865.log.gz 2023-11-09T04:07:18.606 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.105099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110289.log 2023-11-09T04:07:18.607 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.116834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134683.log 2023-11-09T04:07:18.607 INFO:teuthology.orchestra.run.smithi076.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.105099.log.gz 2023-11-09T04:07:18.607 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8%/var/log/ceph/ceph-client.admin.110289.log: -- replaced with /var/log/ceph/ceph-client.admin.116834.log.gz 2023-11-09T04:07:18.607 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125878.log 2023-11-09T04:07:18.608 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.134683.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.110289.log.gz 2023-11-09T04:07:18.608 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104143.log 2023-11-09T04:07:18.608 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.134683.log.gz 2023-11-09T04:07:18.608 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.125878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117142.log 2023-11-09T04:07:18.609 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.125878.log.gz 2023-11-09T04:07:18.609 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.104143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93985.log 2023-11-09T04:07:18.609 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.117142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115550.log 2023-11-09T04:07:18.609 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104143.log.gz 2023-11-09T04:07:18.610 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.93985.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117142.log.gz 2023-11-09T04:07:18.610 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119872.log 2023-11-09T04:07:18.610 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.93985.log.gz 2023-11-09T04:07:18.610 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.115550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104099.log 2023-11-09T04:07:18.611 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.119872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115102.log 2023-11-09T04:07:18.611 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115550.log.gz 2023-11-09T04:07:18.611 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.104099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121340.log 2023-11-09T04:07:18.611 INFO:teuthology.orchestra.run.smithi076.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.119872.log.gz 2023-11-09T04:07:18.611 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.115102.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.104099.log.gz 2023-11-09T04:07:18.612 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118094.log 2023-11-09T04:07:18.612 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.115102.log.gz 2023-11-09T04:07:18.612 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.121340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94943.log 2023-11-09T04:07:18.612 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.118094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115256.log 2023-11-09T04:07:18.613 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.94943.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118094.log.gz 2023-11-09T04:07:18.613 INFO:teuthology.orchestra.run.smithi076.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.121340.log.gz 2023-11-09T04:07:18.613 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132439.log 2023-11-09T04:07:18.613 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.115256.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94943.log.gz 2023-11-09T04:07:18.614 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116031.log 2023-11-09T04:07:18.614 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.132439.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115256.log.gz 2023-11-09T04:07:18.614 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96011.log 2023-11-09T04:07:18.615 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.132439.log.gz 2023-11-09T04:07:18.615 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.116031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106232.log 2023-11-09T04:07:18.615 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.96011.log: 89.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.116031.log.gz -5 2023-11-09T04:07:18.615 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.111576.log 2023-11-09T04:07:18.616 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96011.log.gz 2023-11-09T04:07:18.616 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.106232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118427.log 2023-11-09T04:07:18.616 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.111576.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.106232.log.gz 2023-11-09T04:07:18.616 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120292.log 2023-11-09T04:07:18.617 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.118427.log: gzip -5 --verbose -- 88.3% /var/log/ceph/ceph-client.admin.102161.log -- replaced with /var/log/ceph/ceph-client.admin.111576.log.gz 2023-11-09T04:07:18.617 INFO:teuthology.orchestra.run.smithi076.stderr: 2023-11-09T04:07:18.617 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.120292.log: gzip -5 --verbose -- 88.6% /var/log/ceph/ceph-client.admin.103709.log 2023-11-09T04:07:18.617 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.118427.log.gz 2023-11-09T04:07:18.617 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.102161.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120292.log.gz 2023-11-09T04:07:18.618 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101586.log 2023-11-09T04:07:18.618 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102161.log.gz 2023-11-09T04:07:18.618 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.103709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97780.log 2023-11-09T04:07:18.619 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.101586.log: gzip -5 87.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.103709.log.gz 2023-11-09T04:07:18.619 INFO:teuthology.orchestra.run.smithi076.stderr: /var/log/ceph/ceph-client.admin.93625.log 2023-11-09T04:07:18.619 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.97780.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.101586.log.gz 2023-11-09T04:07:18.619 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101260.log 2023-11-09T04:07:18.620 INFO:teuthology.orchestra.run.smithi076.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97780.log.gz 2023-11-09T04:07:18.620 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.93625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103625.log 2023-11-09T04:07:18.620 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.101260.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.93625.log.gz 2023-11-09T04:07:18.620 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108333.log 2023-11-09T04:07:18.621 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.103625.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101260.log.gz -5 2023-11-09T04:07:18.621 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.130458.log 2023-11-09T04:07:18.621 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103625.log.gz 2023-11-09T04:07:18.621 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.108333.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-09T04:07:18.622 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108333.log.gz 2023-11-09T04:07:18.622 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.130458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108291.log 2023-11-09T04:07:18.622 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- 88.3% /var/log/ceph/ceph-client.admin.134378.log 2023-11-09T04:07:18.622 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.130458.log.gz 2023-11-09T04:07:18.623 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.108291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130192.log 2023-11-09T04:07:18.623 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108291.log.gz 2023-11-09T04:07:18.623 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.134378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134463.log 2023-11-09T04:07:18.623 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103375.log 2023-11-09T04:07:18.624 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.130192.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.134378.log.gz 2023-11-09T04:07:18.624 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.134463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97402.log 2023-11-09T04:07:18.624 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.130192.log.gz 2023-11-09T04:07:18.624 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118001.log 2023-11-09T04:07:18.625 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.134463.log.gz 2023-11-09T04:07:18.625 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.103375.log: /var/log/ceph/ceph-client.admin.97402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100218.log 2023-11-09T04:07:18.625 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103375.log.gz 2023-11-09T04:07:18.625 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95211.log 2023-11-09T04:07:18.626 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.118001.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97402.log.gz 2023-11-09T04:07:18.626 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.100218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108949.log 2023-11-09T04:07:18.626 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.118001.log.gz 2023-11-09T04:07:18.626 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135161.log 2023-11-09T04:07:18.627 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.95211.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100218.log.gz 2023-11-09T04:07:18.627 INFO:teuthology.orchestra.run.smithi076.stderr: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-09T04:07:18.627 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.108949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110541.log 2023-11-09T04:07:18.627 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.95211.log.gz 2023-11-09T04:07:18.628 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.135161.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.108949.log.gz 2023-11-09T04:07:18.628 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122557.log 2023-11-09T04:07:18.628 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4%/var/log/ceph/ceph-client.admin.110541.log: -- replaced with /var/log/ceph/ceph-client.admin.135161.log.gz 2023-11-09T04:07:18.628 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101428.log 2023-11-09T04:07:18.629 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.122557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100092.log 2023-11-09T04:07:18.629 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.101428.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.110541.log.gz 2023-11-09T04:07:18.629 INFO:teuthology.orchestra.run.smithi076.stderr:gzip 89.3% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.122557.log.gz -- 2023-11-09T04:07:18.629 INFO:teuthology.orchestra.run.smithi076.stderr: /var/log/ceph/ceph-client.admin.119588.log 2023-11-09T04:07:18.630 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.101428.log.gz 2023-11-09T04:07:18.630 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.100092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116115.log 2023-11-09T04:07:18.630 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.119588.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.100092.log.gz 2023-11-09T04:07:18.630 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128966.log 2023-11-09T04:07:18.631 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.116115.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.119588.log.gz 2023-11-09T04:07:18.631 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97738.log 2023-11-09T04:07:18.631 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.128966.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116115.log.gz 2023-11-09T04:07:18.631 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100761.log 2023-11-09T04:07:18.632 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.97738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120250.log 2023-11-09T04:07:18.632 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97738.log.gz 2023-11-09T04:07:18.632 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.100761.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.128966.log.gz 2023-11-09T04:07:18.632 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116708.log 2023-11-09T04:07:18.633 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100761.log.gz 2023-11-09T04:07:18.633 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.120250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128349.log 2023-11-09T04:07:18.633 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.116708.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.120250.log.gz 2023-11-09T04:07:18.633 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113700.log 2023-11-09T04:07:18.634 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.128349.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.116708.log.gz 2023-11-09T04:07:18.634 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95500.log 2023-11-09T04:07:18.634 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.113700.log: gzip 87.5% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.128349.log.gz -- 2023-11-09T04:07:18.634 INFO:teuthology.orchestra.run.smithi076.stderr: /var/log/ceph/ceph-client.admin.93815.log 2023-11-09T04:07:18.635 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113700.log.gz 2023-11-09T04:07:18.635 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.95500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111534.log 2023-11-09T04:07:18.635 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95500.log.gz 2023-11-09T04:07:18.635 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.93815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127096.log 2023-11-09T04:07:18.636 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.111534.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.93815.log.gz -5 2023-11-09T04:07:18.636 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.119424.log 2023-11-09T04:07:18.636 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.127096.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.111534.log.gz 2023-11-09T04:07:18.636 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108991.log 2023-11-09T04:07:18.637 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.127096.log.gz 2023-11-09T04:07:18.637 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.119424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125258.log 2023-11-09T04:07:18.637 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.108991.log: gzip -5 --verbose -- 89.2% /var/log/ceph/ceph-client.admin.122133.log 2023-11-09T04:07:18.637 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.119424.log.gz 2023-11-09T04:07:18.638 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.125258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99336.log 2023-11-09T04:07:18.638 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.108991.log.gz 2023-11-09T04:07:18.638 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.122133.log: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.125258.log.gz -5 2023-11-09T04:07:18.638 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.130314.log 2023-11-09T04:07:18.639 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.122133.log.gz 2023-11-09T04:07:18.639 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.99336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123820.log 2023-11-09T04:07:18.639 INFO:teuthology.orchestra.run.smithi076.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99336.log.gz 2023-11-09T04:07:18.640 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.130314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131447.log 2023-11-09T04:07:18.640 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.123820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112108.log 88.1% 2023-11-09T04:07:18.640 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.130314.log.gz 2023-11-09T04:07:18.640 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.131447.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.123820.log.gz 2023-11-09T04:07:18.641 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134523.log 2023-11-09T04:07:18.641 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.131447.log.gz 2023-11-09T04:07:18.641 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.112108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132355.log 2023-11-09T04:07:18.641 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.134523.log: gzip 89.2% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.112108.log.gz -- 2023-11-09T04:07:18.641 INFO:teuthology.orchestra.run.smithi076.stderr: /var/log/ceph/ceph-client.admin.109981.log 2023-11-09T04:07:18.642 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.132355.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.134523.log.gz 2023-11-09T04:07:18.642 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.98971.log 2023-11-09T04:07:18.642 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.132355.log.gz 2023-11-09T04:07:18.643 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.109981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99136.log 2023-11-09T04:07:18.643 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.2.98971.log: gzip -5 88.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.109981.log.gz /var/log/ceph/ceph-client.admin.110607.log 2023-11-09T04:07:18.643 INFO:teuthology.orchestra.run.smithi076.stderr: 2023-11-09T04:07:18.643 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.99136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101218.log 2023-11-09T04:07:18.644 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.99136.log.gz 2023-11-09T04:07:18.644 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.110607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126647.log 2023-11-09T04:07:18.644 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.110607.log.gz 2023-11-09T04:07:18.644 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107301.log 2023-11-09T04:07:18.645 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.101218.log: 91.9% -- replaced with /var/log/ceph/ceph-client.2.98971.log.gz 2023-11-09T04:07:18.645 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.126647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122683.log 2023-11-09T04:07:18.645 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101218.log.gz 2023-11-09T04:07:18.645 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.107301.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.126647.log.gz 2023-11-09T04:07:18.646 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112378.log 2023-11-09T04:07:18.646 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.122683.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.107301.log.gz 2023-11-09T04:07:18.646 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135203.log 2023-11-09T04:07:18.646 INFO:teuthology.orchestra.run.smithi076.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.122683.log.gz 2023-11-09T04:07:18.647 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.112378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133283.log 2023-11-09T04:07:18.647 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.135203.log: gzip -5 --verbose -- 88.9% /var/log/ceph/ceph-client.admin.128756.log 2023-11-09T04:07:18.647 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.112378.log.gz 2023-11-09T04:07:18.647 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.133283.log: gzip -5 88.8% --verbose -- replaced with /var/log/ceph/ceph-client.admin.135203.log.gz -- 2023-11-09T04:07:18.648 INFO:teuthology.orchestra.run.smithi076.stderr: /var/log/ceph/ceph-client.admin.119166.log 2023-11-09T04:07:18.648 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.128756.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.133283.log.gz 2023-11-09T04:07:18.648 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120623.log 2023-11-09T04:07:18.648 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.128756.log.gz 2023-11-09T04:07:18.649 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.119166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97906.log 2023-11-09T04:07:18.649 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.120623.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.119166.log.gzgzip 2023-11-09T04:07:18.649 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.115382.log 2023-11-09T04:07:18.649 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.97906.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.120623.log.gz 2023-11-09T04:07:18.650 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115947.log 2023-11-09T04:07:18.650 INFO:teuthology.orchestra.run.smithi076.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.97906.log.gz 2023-11-09T04:07:18.650 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.115382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116199.log 2023-11-09T04:07:18.650 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.115947.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115382.log.gz 2023-11-09T04:07:18.651 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115905.log 2023-11-09T04:07:18.651 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.116199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112318.log 2023-11-09T04:07:18.651 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115947.log.gz 2023-11-09T04:07:18.651 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.115905.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116199.log.gz 2023-11-09T04:07:18.652 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111408.log 2023-11-09T04:07:18.652 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.115905.log.gz 2023-11-09T04:07:18.652 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.112318.log: /var/log/ceph/ceph-client.admin.102959.log 2023-11-09T04:07:18.652 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.111408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94261.log 2023-11-09T04:07:18.653 INFO:teuthology.orchestra.run.smithi076.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.112318.log.gz 2023-11-09T04:07:18.653 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.102959.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111408.log.gz 2023-11-09T04:07:18.653 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131531.log 2023-11-09T04:07:18.653 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102959.log.gz 2023-11-09T04:07:18.654 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.94261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115340.log 2023-11-09T04:07:18.654 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94261.log.gz 2023-11-09T04:07:18.654 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.131531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95710.log 2023-11-09T04:07:18.655 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.115340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102917.log 2023-11-09T04:07:18.655 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.131531.log.gz 2023-11-09T04:07:18.655 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3%/var/log/ceph/ceph-client.admin.95710.log: -- replaced with /var/log/ceph/ceph-client.admin.115340.log.gz 2023-11-09T04:07:18.655 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104311.log 2023-11-09T04:07:18.656 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.102917.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.95710.log.gzgzip 2023-11-09T04:07:18.656 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.112841.log 2023-11-09T04:07:18.656 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.104311.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.102917.log.gz 2023-11-09T04:07:18.656 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113910.log 2023-11-09T04:07:18.657 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104311.log.gz 2023-11-09T04:07:18.657 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.112841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100887.log 2023-11-09T04:07:18.657 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.113910.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.112841.log.gz 2023-11-09T04:07:18.657 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121826.log 2023-11-09T04:07:18.658 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.100887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129646.log 2023-11-09T04:07:18.658 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.113910.log.gz 2023-11-09T04:07:18.658 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.121826.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100887.log.gz 2023-11-09T04:07:18.658 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111095.log 2023-11-09T04:07:18.659 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.121826.log.gz 2023-11-09T04:07:18.659 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.129646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119938.log 2023-11-09T04:07:18.659 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.111095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122768.log 2023-11-09T04:07:18.659 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.129646.log.gz 2023-11-09T04:07:18.660 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.119938.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.111095.log.gz 2023-11-09T04:07:18.660 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104015.log 2023-11-09T04:07:18.660 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.122768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130581.log 2023-11-09T04:07:18.660 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.104015.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.122768.log.gz 2023-11-09T04:07:18.661 INFO:teuthology.orchestra.run.smithi076.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.119938.log.gz 2023-11-09T04:07:18.661 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107427.log 2023-11-09T04:07:18.661 INFO:teuthology.orchestra.run.smithi076.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.104015.log.gz 2023-11-09T04:07:18.661 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.130581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121953.log 2023-11-09T04:07:18.662 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.130581.log.gz 2023-11-09T04:07:18.662 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.107427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100577.log 2023-11-09T04:07:18.662 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.121953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107217.log 2023-11-09T04:07:18.662 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107427.log.gz 2023-11-09T04:07:18.663 INFO:teuthology.orchestra.run.smithi076.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.121953.log.gz 2023-11-09T04:07:18.663 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.100577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96015.log 2023-11-09T04:07:18.663 INFO:teuthology.orchestra.run.smithi076.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100577.log.gz 2023-11-09T04:07:18.663 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107091.log 2023-11-09T04:07:18.664 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.107217.log: /var/log/ceph/ceph-client.admin.96015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96352.log 2023-11-09T04:07:18.664 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.107217.log.gz 2023-11-09T04:07:18.664 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.107091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130675.log 2023-11-09T04:07:18.664 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.96015.log.gz 2023-11-09T04:07:18.665 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.96352.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107091.log.gz 2023-11-09T04:07:18.665 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121472.log 2023-11-09T04:07:18.665 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.130675.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.96352.log.gz 2023-11-09T04:07:18.666 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113392.log 2023-11-09T04:07:18.666 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.130675.log.gz 2023-11-09T04:07:18.666 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.121472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119545.log 2023-11-09T04:07:18.666 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.121472.log.gz 2023-11-09T04:07:18.667 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.113392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112936.log 2023-11-09T04:07:18.667 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.119545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125300.log 2023-11-09T04:07:18.667 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.113392.log.gz 2023-11-09T04:07:18.668 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.119545.log.gz 2023-11-09T04:07:18.668 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.112936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117959.log 2023-11-09T04:07:18.668 INFO:teuthology.orchestra.run.smithi076.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.112936.log.gz 2023-11-09T04:07:18.668 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.125300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134195.log 2023-11-09T04:07:18.669 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.117959.log: gzip -5 --verbose -- 88.6% /var/log/ceph/ceph-client.admin.96016.log 2023-11-09T04:07:18.669 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.125300.log.gz 2023-11-09T04:07:18.669 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.134195.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.117959.log.gz 2023-11-09T04:07:18.669 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135245.log 2023-11-09T04:07:18.670 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.134195.log.gz 2023-11-09T04:07:18.670 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.96016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113784.log 2023-11-09T04:07:18.670 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.135245.log: gzip 88.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.115862.log -- replaced with /var/log/ceph/ceph-client.admin.96016.log.gz 2023-11-09T04:07:18.671 INFO:teuthology.orchestra.run.smithi076.stderr: 2023-11-09T04:07:18.671 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.113784.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-11-09T04:07:18.671 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.135245.log.gz 2023-11-09T04:07:18.671 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.115862.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.113784.log.gz 2023-11-09T04:07:18.672 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97318.log 2023-11-09T04:07:18.672 INFO:teuthology.orchestra.run.smithi076.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.115862.log.gz 2023-11-09T04:07:18.672 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121190.log 2023-11-09T04:07:18.672 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.97318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128840.log 2023-11-09T04:07:18.673 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.121190.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.97318.log.gz 2023-11-09T04:07:18.673 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125394.log 2023-11-09T04:07:18.673 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.128840.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.121190.log.gz 2023-11-09T04:07:18.674 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95069.log 2023-11-09T04:07:18.674 INFO:teuthology.orchestra.run.smithi076.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.128840.log.gz 2023-11-09T04:07:18.674 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.125394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127382.log 2023-11-09T04:07:18.674 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.125394.log.gz 2023-11-09T04:07:18.675 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.95069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102539.log 2023-11-09T04:07:18.675 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95069.log.gz 2023-11-09T04:07:18.675 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.127382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128062.log 2023-11-09T04:07:18.675 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.127382.log.gz 2023-11-09T04:07:18.676 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.102539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94859.log 2023-11-09T04:07:18.676 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96017.log 2023-11-09T04:07:18.676 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102539.log.gz 2023-11-09T04:07:18.676 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.128062.log: /var/log/ceph/ceph-client.admin.94859.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.128062.log.gz 2023-11-09T04:07:18.677 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124762.log 2023-11-09T04:07:18.677 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115466.log 2023-11-09T04:07:18.677 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.96017.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94859.log.gz 2023-11-09T04:07:18.677 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.124762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135844.log 2023-11-09T04:07:18.678 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96017.log.gz 2023-11-09T04:07:18.678 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100302.log 2023-11-09T04:07:18.678 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.115466.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.124762.log.gz 2023-11-09T04:07:18.679 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.135844.log: gzip -5 --verbose 88.8% -- -- replaced with /var/log/ceph/ceph-client.admin.115466.log.gz /var/log/ceph/ceph-client.admin.99294.log 2023-11-09T04:07:18.679 INFO:teuthology.orchestra.run.smithi076.stderr: 2023-11-09T04:07:18.679 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102203.log 2023-11-09T04:07:18.679 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.100302.log: /var/log/ceph/ceph-client.admin.99294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123056.log 2023-11-09T04:07:18.680 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100302.log.gz 2023-11-09T04:07:18.680 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.102203.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99294.log.gz 2023-11-09T04:07:18.680 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124325.log 2023-11-09T04:07:18.680 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102203.log.gz 2023-11-09T04:07:18.681 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.123056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93456.log 2023-11-09T04:07:18.681 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116283.log 2023-11-09T04:07:18.681 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.124325.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.123056.log.gz 2023-11-09T04:07:18.681 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.93456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101998.log 2023-11-09T04:07:18.682 INFO:teuthology.orchestra.run.smithi076.stderr: 95.7% -- replaced with /var/log/ceph/ceph-client.admin.135844.log.gz 2023-11-09T04:07:18.682 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.124325.log.gz 2023-11-09T04:07:18.682 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.116283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117310.log 2023-11-09T04:07:18.682 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.93456.log.gz 2023-11-09T04:07:18.683 INFO:teuthology.orchestra.run.smithi076.stderr:gzip/var/log/ceph/ceph-client.admin.101998.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.120581.log 2023-11-09T04:07:18.683 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116283.log.gz 2023-11-09T04:07:18.683 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.117310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107385.log 2023-11-09T04:07:18.684 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.101998.log.gz 2023-11-09T04:07:18.684 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.117310.log.gz 2023-11-09T04:07:18.684 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.120581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121868.log 2023-11-09T04:07:18.684 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112631.log 2023-11-09T04:07:18.685 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.107385.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.120581.log.gz 2023-11-09T04:07:18.685 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.121868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134599.log 2023-11-09T04:07:18.685 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107385.log.gz 2023-11-09T04:07:18.685 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109346.log 2023-11-09T04:07:18.686 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.112631.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.121868.log.gz 2023-11-09T04:07:18.686 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.134599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126907.log 2023-11-09T04:07:18.686 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112631.log.gz 2023-11-09T04:07:18.686 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103667.log 2023-11-09T04:07:18.687 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.109346.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.134599.log.gz 2023-11-09T04:07:18.687 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.126907.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.109346.log.gz 2023-11-09T04:07:18.687 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122972.log 2023-11-09T04:07:18.687 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100845.log 2023-11-09T04:07:18.688 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.103667.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.126907.log.gz 2023-11-09T04:07:18.688 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.122972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99924.log 2023-11-09T04:07:18.688 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.103667.log.gz 2023-11-09T04:07:18.688 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112150.log 2023-11-09T04:07:18.689 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.100845.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.122972.log.gz 2023-11-09T04:07:18.689 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.99924.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100845.log.gz 2023-11-09T04:07:18.689 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114660.log 2023-11-09T04:07:18.689 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121911.log 2023-11-09T04:07:18.690 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.112150.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99924.log.gz 2023-11-09T04:07:18.690 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.114660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116792.log 2023-11-09T04:07:18.690 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.112150.log.gz 2023-11-09T04:07:18.690 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.121911.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.114660.log.gz 2023-11-09T04:07:18.691 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99672.log 2023-11-09T04:07:18.691 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.116792.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.121911.log.gz 2023-11-09T04:07:18.691 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122382.log 2023-11-09T04:07:18.691 INFO:teuthology.orchestra.run.smithi076.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.116792.log.gz 2023-11-09T04:07:18.692 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132785.log 2023-11-09T04:07:18.692 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.99672.log: /var/log/ceph/ceph-client.admin.122382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99420.log 2023-11-09T04:07:18.692 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99672.log.gz 2023-11-09T04:07:18.692 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.132785.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.122382.log.gz 2023-11-09T04:07:18.693 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102245.log 2023-11-09T04:07:18.693 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103583.log 2023-11-09T04:07:18.693 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.99420.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.132785.log.gz 2023-11-09T04:07:18.693 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.102245.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99420.log.gz 2023-11-09T04:07:18.694 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122256.log 2023-11-09T04:07:18.694 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.102245.log.gz 2023-11-09T04:07:18.694 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96697.log 2023-11-09T04:07:18.695 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.103583.log: /var/log/ceph/ceph-client.admin.122256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108213.log 2023-11-09T04:07:18.695 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103583.log.gz 2023-11-09T04:07:18.695 INFO:teuthology.orchestra.run.smithi076.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.122256.log.gz 2023-11-09T04:07:18.695 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.96697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123098.log 2023-11-09T04:07:18.696 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96697.log.gz 2023-11-09T04:07:18.696 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124367.log 2023-11-09T04:07:18.696 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.108213.log: /var/log/ceph/ceph-client.admin.123098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96580.log 2023-11-09T04:07:18.696 INFO:teuthology.orchestra.run.smithi076.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.108213.log.gz 2023-11-09T04:07:18.697 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.124367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97234.log 2023-11-09T04:07:18.697 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.123098.log.gz 2023-11-09T04:07:18.697 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.124367.log.gz 2023-11-09T04:07:18.697 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.96580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109609.log 2023-11-09T04:07:18.698 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96580.log.gz 2023-11-09T04:07:18.698 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122599.log 2023-11-09T04:07:18.698 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.97234.log: /var/log/ceph/ceph-client.admin.109609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111660.log 2023-11-09T04:07:18.698 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97234.log.gz 2023-11-09T04:07:18.699 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.109609.log.gz 2023-11-09T04:07:18.699 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.122599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115802.log 2023-11-09T04:07:18.699 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.111660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115760.log 2023-11-09T04:07:18.699 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.122599.log.gz 2023-11-09T04:07:18.700 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.115802.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111660.log.gz 2023-11-09T04:07:18.700 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95416.log 2023-11-09T04:07:18.700 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95668.log 2023-11-09T04:07:18.700 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.115760.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115802.log.gz 2023-11-09T04:07:18.701 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.95416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127256.log 2023-11-09T04:07:18.701 INFO:teuthology.orchestra.run.smithi076.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.115760.log.gz 2023-11-09T04:07:18.701 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95416.log.gz 2023-11-09T04:07:18.701 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116500.log 2023-11-09T04:07:18.702 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.95668.log: /var/log/ceph/ceph-client.admin.127256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96898.log 2023-11-09T04:07:18.702 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95668.log.gz 2023-11-09T04:07:18.702 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.116500.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.127256.log.gz 2023-11-09T04:07:18.703 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118051.log 2023-11-09T04:07:18.703 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.96898.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116500.log.gz 2023-11-09T04:07:18.703 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132213.log 2023-11-09T04:07:18.703 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96898.log.gz 2023-11-09T04:07:18.704 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.118051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105877.log 2023-11-09T04:07:18.704 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.118051.log.gz 2023-11-09T04:07:18.704 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134049.log 2023-11-09T04:07:18.704 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.132213.log: /var/log/ceph/ceph-client.admin.105877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108079.log 2023-11-09T04:07:18.705 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.132213.log.gz 2023-11-09T04:07:18.705 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105877.log.gz 2023-11-09T04:07:18.705 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.134049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.127808.log 2023-11-09T04:07:18.706 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97037.log 2023-11-09T04:07:18.706 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.108079.log: /var/log/ceph/ceph-client.0.127808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109033.log 2023-11-09T04:07:18.706 INFO:teuthology.orchestra.run.smithi076.stderr: 91.1% 90.2% -- replaced with /var/log/ceph/ceph-client.admin.108079.log.gz 2023-11-09T04:07:18.706 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.134049.log.gz 2023-11-09T04:07:18.707 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.97037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105583.log 2023-11-09T04:07:18.707 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131102.log 2023-11-09T04:07:18.707 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.109033.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.97037.log.gz 2023-11-09T04:07:18.707 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.105583.log: gzip -5 88.9% -- replaced with /var/log/ceph/ceph-client.admin.109033.log.gz 2023-11-09T04:07:18.708 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.0.98478.log 2023-11-09T04:07:18.708 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.131102.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.105583.log.gz 2023-11-09T04:07:18.708 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105961.log 2023-11-09T04:07:18.708 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.131102.log.gz 2023-11-09T04:07:18.709 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.0.98478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120518.log 2023-11-09T04:07:18.709 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.105961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93857.log 2023-11-09T04:07:18.709 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.120518.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.105961.log.gz 2023-11-09T04:07:18.709 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96014.log 2023-11-09T04:07:18.710 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.120518.log.gz 2023-11-09T04:07:18.710 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.93857.log: 91.2% -- replaced with /var/log/ceph/ceph-client.0.98478.log.gz 2023-11-09T04:07:18.710 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107343.log 2023-11-09T04:07:18.710 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.96014.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93857.log.gz 2023-11-09T04:07:18.711 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131363.log 2023-11-09T04:07:18.711 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96014.log.gz 2023-11-09T04:07:18.711 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.107343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95458.log 2023-11-09T04:07:18.711 INFO:teuthology.orchestra.run.smithi076.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.107343.log.gz 2023-11-09T04:07:18.712 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122641.log 2023-11-09T04:07:18.712 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.131363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127051.log 2023-11-09T04:07:18.712 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.95458.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.131363.log.gz 2023-11-09T04:07:18.712 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.122641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97570.log 2023-11-09T04:07:18.713 INFO:teuthology.orchestra.run.smithi076.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.127808.log.gz 2023-11-09T04:07:18.713 INFO:teuthology.orchestra.run.smithi076.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.95458.log.gz 2023-11-09T04:07:18.713 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.127051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105387.log 2023-11-09T04:07:18.713 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.122641.log.gz 2023-11-09T04:07:18.714 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.97570.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.127051.log.gz 2023-11-09T04:07:18.714 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.99462.log 2023-11-09T04:07:18.714 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.105387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111898.log 2023-11-09T04:07:18.714 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.97570.log.gz 2023-11-09T04:07:18.715 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.99462.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105387.log.gz 2023-11-09T04:07:18.715 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134237.log 2023-11-09T04:07:18.715 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99462.log.gzgzip 2023-11-09T04:07:18.715 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-11-09T04:07:18.716 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.111898.log: /var/log/ceph/ceph-client.admin.134237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110161.log 2023-11-09T04:07:18.716 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111898.log.gz 2023-11-09T04:07:18.716 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-osd.3.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.134237.log.gz 2023-11-09T04:07:18.716 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101344.log 2023-11-09T04:07:18.717 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.110161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101814.log 2023-11-09T04:07:18.717 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.101344.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.110161.log.gz 2023-11-09T04:07:18.717 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103211.log 2023-11-09T04:07:18.717 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.101814.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101344.log.gz 2023-11-09T04:07:18.718 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109689.log 2023-11-09T04:07:18.718 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.103211.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.101814.log.gz 2023-11-09T04:07:18.718 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109178.log 2023-11-09T04:07:18.718 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.109689.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.103211.log.gz -5 2023-11-09T04:07:18.719 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.126709.log 2023-11-09T04:07:18.719 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.109689.log.gz 2023-11-09T04:07:18.719 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.109178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97040.log 2023-11-09T04:07:18.719 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112505.log 2023-11-09T04:07:18.719 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.126709.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.109178.log.gz 2023-11-09T04:07:18.720 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.97040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118856.log 2023-11-09T04:07:18.720 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.126709.log.gz 2023-11-09T04:07:18.720 INFO:teuthology.orchestra.run.smithi076.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97040.log.gz 2023-11-09T04:07:18.720 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.112505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112276.log 2023-11-09T04:07:18.721 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106358.log 2023-11-09T04:07:18.721 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.118856.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112505.log.gz 2023-11-09T04:07:18.721 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.112276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105021.log 2023-11-09T04:07:18.721 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.118856.log.gz 2023-11-09T04:07:18.722 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.106358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96353.log 2023-11-09T04:07:18.722 INFO:teuthology.orchestra.run.smithi076.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.112276.log.gz 2023-11-09T04:07:18.722 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.105021.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106358.log.gz 2023-11-09T04:07:18.722 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95836.log 2023-11-09T04:07:18.723 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134951.log 2023-11-09T04:07:18.723 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.96353.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.105021.log.gz 2023-11-09T04:07:18.723 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.95836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123182.log 2023-11-09T04:07:18.724 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.96353.log.gz 2023-11-09T04:07:18.724 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95836.log.gz 2023-11-09T04:07:18.724 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.134951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101872.log 2023-11-09T04:07:18.724 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114720.log 2023-11-09T04:07:18.724 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.134951.log.gz 2023-11-09T04:07:18.725 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.123182.log: /var/log/ceph/ceph-client.admin.101872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131279.log 2023-11-09T04:07:18.725 INFO:teuthology.orchestra.run.smithi076.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.123182.log.gz 2023-11-09T04:07:18.725 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.114720.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101872.log.gzgzip 2023-11-09T04:07:18.725 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.127009.log 2023-11-09T04:07:18.726 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.114720.log.gz 2023-11-09T04:07:18.726 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123694.log 2023-11-09T04:07:18.726 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.131279.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.127009.log: /var/log/ceph/ceph-client.admin.100395.log 2023-11-09T04:07:18.726 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.123694.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.131279.log.gz 2023-11-09T04:07:18.727 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124068.log 2023-11-09T04:07:18.727 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.127009.log.gz 2023-11-09T04:07:18.727 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.100395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110247.log 2023-11-09T04:07:18.727 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.124068.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100395.log.gz 2023-11-09T04:07:18.728 INFO:teuthology.orchestra.run.smithi076.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.123694.log.gz 2023-11-09T04:07:18.728 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110203.log 2023-11-09T04:07:18.728 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.124068.log.gz 2023-11-09T04:07:18.729 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109075.log 2023-11-09T04:07:18.729 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.110247.log: /var/log/ceph/ceph-client.admin.110203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104965.log 2023-11-09T04:07:18.729 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110247.log.gz 2023-11-09T04:07:18.729 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.109075.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110203.log.gz 2023-11-09T04:07:18.730 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110069.log 2023-11-09T04:07:18.730 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109075.log.gz 2023-11-09T04:07:18.730 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.104965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115989.log 2023-11-09T04:07:18.730 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.110069.log: -- /var/log/ceph/ceph-client.admin.109388.log 2023-11-09T04:07:18.731 INFO:teuthology.orchestra.run.smithi076.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.104965.log.gz 2023-11-09T04:07:18.731 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.115989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133141.log 2023-11-09T04:07:18.731 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.110069.log.gz 2023-11-09T04:07:18.731 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.109388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121406.log 2023-11-09T04:07:18.732 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.115989.log.gz 2023-11-09T04:07:18.732 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 89.4% -- replaced with /var/log/ceph/ceph-client.admin.109388.log.gz 2023-11-09T04:07:18.732 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.133141.log: --verbose -- /var/log/ceph/ceph-client.admin.112547.log 2023-11-09T04:07:18.732 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.121406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116750.log 2023-11-09T04:07:18.733 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.112547.log: gzip 91.1% -- replaced with /var/log/ceph/ceph-client.admin.133141.log.gz 2023-11-09T04:07:18.733 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.112547.log.gz 2023-11-09T04:07:18.733 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.120797.log 2023-11-09T04:07:18.733 INFO:teuthology.orchestra.run.smithi076.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.121406.log.gz 2023-11-09T04:07:18.734 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.116750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128265.log 2023-11-09T04:07:18.734 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116750.log.gz 2023-11-09T04:07:18.734 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.120797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103965.log 2023-11-09T04:07:18.734 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.120797.log.gz 2023-11-09T04:07:18.735 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.128265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116581.log 2023-11-09T04:07:18.735 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95794.log 2023-11-09T04:07:18.735 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.103965.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.128265.log.gz 2023-11-09T04:07:18.735 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.116581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133631.log 2023-11-09T04:07:18.736 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103965.log.gz 2023-11-09T04:07:18.736 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.95794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96341.log 2023-11-09T04:07:18.736 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.116581.log.gz 2023-11-09T04:07:18.736 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.133631.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95794.log.gz 2023-11-09T04:07:18.737 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95295.log 2023-11-09T04:07:18.737 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116073.log 2023-11-09T04:07:18.737 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.96341.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.133631.log.gz 2023-11-09T04:07:18.737 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.95295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119788.log 89.0% 2023-11-09T04:07:18.738 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.96341.log.gz 2023-11-09T04:07:18.738 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.116073.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.95295.log.gz 2023-11-09T04:07:18.738 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119274.log 2023-11-09T04:07:18.738 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.116073.log.gz 2023-11-09T04:07:18.739 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.119788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97039.log 2023-11-09T04:07:18.739 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.119274.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.119788.log.gz 2023-11-09T04:07:18.739 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121579.log 2023-11-09T04:07:18.739 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.97039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125048.log 2023-11-09T04:07:18.740 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.121579.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.97039.log.gz 2023-11-09T04:07:18.740 INFO:teuthology.orchestra.run.smithi076.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.119274.log.gz 2023-11-09T04:07:18.740 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101772.log 88.6% -- replaced with /var/log/ceph/ceph-client.admin.121579.log.gz 2023-11-09T04:07:18.740 INFO:teuthology.orchestra.run.smithi076.stderr: 2023-11-09T04:07:18.741 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.125048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118712.log 2023-11-09T04:07:18.741 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.101772.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.125048.log.gz 2023-11-09T04:07:18.741 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95584.log 2023-11-09T04:07:18.741 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101772.log.gz 2023-11-09T04:07:18.742 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.118712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105499.log 2023-11-09T04:07:18.742 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.95584.log: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112673.log 2023-11-09T04:07:18.742 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.118712.log.gz 2023-11-09T04:07:18.742 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95584.log.gz 2023-11-09T04:07:18.743 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102581.log 2023-11-09T04:07:18.743 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.105499.log: /var/log/ceph/ceph-client.admin.112673.log: gzip -5 --verbose 87.8% -- replaced with /var/log/ceph/ceph-client.admin.105499.log.gz 2023-11-09T04:07:18.743 INFO:teuthology.orchestra.run.smithi076.stderr: -- /var/log/ceph/ceph-client.admin.93943.log 2023-11-09T04:07:18.744 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112673.log.gz 2023-11-09T04:07:18.744 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.102581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119061.log 2023-11-09T04:07:18.744 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.93943.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102581.log.gz 2023-11-09T04:07:18.744 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110119.log 2023-11-09T04:07:18.745 INFO:teuthology.orchestra.run.smithi076.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.93943.log.gz 2023-11-09T04:07:18.745 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.119061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129426.log 2023-11-09T04:07:18.745 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.110119.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.119061.log.gz 2023-11-09T04:07:18.745 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112757.log 2023-11-09T04:07:18.746 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.110119.log.gz 2023-11-09T04:07:18.746 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.129426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130928.log 2023-11-09T04:07:18.746 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.112757.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.129426.log.gz 2023-11-09T04:07:18.747 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96700.log 2023-11-09T04:07:18.747 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107047.log 2023-11-09T04:07:18.747 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.130928.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112757.log.gz 2023-11-09T04:07:18.747 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.96700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128166.log 2023-11-09T04:07:18.748 INFO:teuthology.orchestra.run.smithi076.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.130928.log.gz 2023-11-09T04:07:18.748 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.107047.log: gzip -5 88.7% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.96700.log.gz /var/log/ceph/ceph-client.admin.107469.log 2023-11-09T04:07:18.748 INFO:teuthology.orchestra.run.smithi076.stderr: 2023-11-09T04:07:18.748 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107047.log.gz 2023-11-09T04:07:18.749 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124026.log 2023-11-09T04:07:18.749 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.128166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106274.log 2023-11-09T04:07:18.749 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.107469.log: gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.128166.log.gz 2023-11-09T04:07:18.749 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.124026.log: 87.1% -5 --verbose -- /var/log/ceph/ceph-client.admin.99798.log 2023-11-09T04:07:18.750 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.107469.log.gz 2023-11-09T04:07:18.750 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.106274.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.124026.log.gz 2023-11-09T04:07:18.750 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129510.log 2023-11-09T04:07:18.751 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106274.log.gz 2023-11-09T04:07:18.751 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.99798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105177.log 2023-11-09T04:07:18.751 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108907.log 2023-11-09T04:07:18.751 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.129510.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99798.log.gz 2023-11-09T04:07:18.752 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.105177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122194.log 2023-11-09T04:07:18.752 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.129510.log.gz 2023-11-09T04:07:18.752 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.108907.log: gzip -5 --verbose 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105177.log.gz 2023-11-09T04:07:18.752 INFO:teuthology.orchestra.run.smithi076.stderr: -- /var/log/ceph/ceph-client.admin.96351.log 2023-11-09T04:07:18.753 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.122194.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.108907.log.gz 2023-11-09T04:07:18.753 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115144.log 2023-11-09T04:07:18.753 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.122194.log.gz 2023-11-09T04:07:18.754 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.96351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96012.log 2023-11-09T04:07:18.754 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.115144.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96351.log.gz 2023-11-09T04:07:18.754 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108375.log 2023-11-09T04:07:18.754 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.115144.log.gz 2023-11-09T04:07:18.755 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.96012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129160.log 2023-11-09T04:07:18.755 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.108375.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.96012.log.gz 2023-11-09T04:07:18.755 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109472.log 2023-11-09T04:07:18.755 INFO:teuthology.orchestra.run.smithi076.stderr: 89.5%/var/log/ceph/ceph-client.admin.129160.log: -- replaced with /var/log/ceph/ceph-client.admin.108375.log.gz 2023-11-09T04:07:18.756 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118202.log 2023-11-09T04:07:18.756 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.109472.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114850.log 2023-11-09T04:07:18.756 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.129160.log.gz 2023-11-09T04:07:18.756 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.118202.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.109472.log.gz 2023-11-09T04:07:18.757 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130801.log 2023-11-09T04:07:18.757 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.114850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102665.log 2023-11-09T04:07:18.757 INFO:teuthology.orchestra.run.smithi076.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.118202.log.gz 2023-11-09T04:07:18.757 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.130801.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100929.log 2023-11-09T04:07:18.758 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.114850.log.gz 2023-11-09T04:07:18.758 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.133888.log 2023-11-09T04:07:18.758 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.130801.log.gz 2023-11-09T04:07:18.759 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.102665.log: /var/log/ceph/ceph-client.admin.100929.log: gzip -5 --verbose 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102665.log.gz 2023-11-09T04:07:18.759 INFO:teuthology.orchestra.run.smithi076.stderr: -- /var/log/ceph/ceph-client.admin.131489.log 2023-11-09T04:07:18.759 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100929.log.gz 2023-11-09T04:07:18.759 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.0.133888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122726.log 2023-11-09T04:07:18.760 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.131489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101470.log 2023-11-09T04:07:18.760 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.131489.log.gz 2023-11-09T04:07:18.760 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.122726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102497.log 2023-11-09T04:07:18.760 INFO:teuthology.orchestra.run.smithi076.stderr: 89.3%/var/log/ceph/ceph-client.admin.101470.log: -- replaced with /var/log/ceph/ceph-client.admin.122726.log.gz 2023-11-09T04:07:18.761 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101730.log 2023-11-09T04:07:18.761 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.102497.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.101470.log.gz 2023-11-09T04:07:18.761 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131667.log 2023-11-09T04:07:18.761 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.101730.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102497.log.gz 2023-11-09T04:07:18.762 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.111366.log 2023-11-09T04:07:18.762 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.131667.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.101730.log.gz 2023-11-09T04:07:18.762 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100008.log 2023-11-09T04:07:18.763 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.131667.log.gz 2023-11-09T04:07:18.763 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.111366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111169.log 2023-11-09T04:07:18.763 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.100008.log: gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111366.log.gz 2023-11-09T04:07:18.763 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.123014.log 2023-11-09T04:07:18.764 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.111169.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100008.log.gz 2023-11-09T04:07:18.764 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120881.log 2023-11-09T04:07:18.764 INFO:teuthology.orchestra.run.smithi076.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.111169.log.gz 2023-11-09T04:07:18.764 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5/var/log/ceph/ceph-client.admin.123014.log: --verbose -- /var/log/ceph/ceph-client.admin.95127.log 2023-11-09T04:07:18.765 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.120881.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.123014.log.gz 2023-11-09T04:07:18.765 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-11-09T04:07:18.765 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.120881.log.gz 2023-11-09T04:07:18.765 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122930.log 2023-11-09T04:07:18.766 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.95127.log: gzip/var/log/ceph/ceph-mgr.y.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.100345.log 2023-11-09T04:07:18.766 INFO:teuthology.orchestra.run.smithi076.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.95127.log.gz 2023-11-09T04:07:18.766 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.122930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.117777.log 2023-11-09T04:07:18.766 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97612.log 2023-11-09T04:07:18.767 INFO:teuthology.orchestra.run.smithi076.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.122930.log.gz 2023-11-09T04:07:18.767 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.100345.log: /var/log/ceph/ceph-client.0.117777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112589.log 2023-11-09T04:07:18.767 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.97612.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100345.log.gz 2023-11-09T04:07:18.767 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97612.log.gz 2023-11-09T04:07:18.768 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120355.log 2023-11-09T04:07:18.768 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104353.log 2023-11-09T04:07:18.768 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.120355.log: /var/log/ceph/ceph-client.admin.112589.log: 88.1% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.120355.log.gz 2023-11-09T04:07:18.768 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.112589.log.gz 2023-11-09T04:07:18.769 INFO:teuthology.orchestra.run.smithi076.stderr: 95.2% -- replaced with /var/log/ceph/ceph-client.0.133888.log.gz 2023-11-09T04:07:18.769 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100661.log 2023-11-09T04:07:18.769 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.104353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113952.log 2023-11-09T04:07:18.769 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.100661.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.104353.log.gz 2023-11-09T04:07:18.770 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127298.log 2023-11-09T04:07:18.770 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100661.log.gz 2023-11-09T04:07:18.770 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120665.log 2023-11-09T04:07:18.770 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.113952.log: /var/log/ceph/ceph-client.admin.127298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105919.log 2023-11-09T04:07:18.771 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.120665.log: 87.2% 88.1% -- replaced with /var/log/ceph/ceph-client.admin.113952.log.gz 2023-11-09T04:07:18.771 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.127298.log.gz 2023-11-09T04:07:18.771 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110415.log 2023-11-09T04:07:18.771 INFO:teuthology.orchestra.run.smithi076.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.120665.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109135.log 2023-11-09T04:07:18.772 INFO:teuthology.orchestra.run.smithi076.stderr: 2023-11-09T04:07:18.772 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.105919.log: /var/log/ceph/ceph-client.admin.110415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128391.log 2023-11-09T04:07:18.772 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.109135.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.105919.log.gz 2023-11-09T04:07:18.772 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110415.log.gz 2023-11-09T04:07:18.773 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109135.log.gz 2023-11-09T04:07:18.773 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130374.log 2023-11-09T04:07:18.773 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131164.log/var/log/ceph/ceph-client.admin.128391.log: 2023-11-09T04:07:18.773 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.130374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103539.log 2023-11-09T04:07:18.774 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.128391.log.gz 2023-11-09T04:07:18.774 INFO:teuthology.orchestra.run.smithi076.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.130374.log.gz 2023-11-09T04:07:18.774 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.131164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105303.log 2023-11-09T04:07:18.774 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.103539.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.131164.log.gz 2023-11-09T04:07:18.775 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106955.log 2023-11-09T04:07:18.775 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103539.log.gz 2023-11-09T04:07:18.775 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102833.log 2023-11-09T04:07:18.776 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.105303.log: /var/log/ceph/ceph-client.admin.106955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123862.log 2023-11-09T04:07:18.776 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.102833.log: 88.5% 87.4% -- replaced with /var/log/ceph/ceph-client.admin.106955.log.gz 2023-11-09T04:07:18.776 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.105303.log.gz 2023-11-09T04:07:18.776 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105345.log 2023-11-09T04:07:18.777 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.102833.log.gz 2023-11-09T04:07:18.777 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.123862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126369.log 2023-11-09T04:07:18.777 INFO:teuthology.orchestra.run.smithi076.stderr: 94.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-11-09T04:07:18.777 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125940.log 2023-11-09T04:07:18.778 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.123862.log.gz 2023-11-09T04:07:18.778 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.105345.log: /var/log/ceph/ceph-client.admin.126369.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.126369.log.gz 2023-11-09T04:07:18.778 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.105345.log.gz 2023-11-09T04:07:18.778 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103793.log 2023-11-09T04:07:18.779 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.125940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124110.log 2023-11-09T04:07:18.779 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.103793.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.125940.log.gz 2023-11-09T04:07:18.779 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103751.log 2023-11-09T04:07:18.780 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.124110.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.103793.log.gz 2023-11-09T04:07:18.780 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134767.log 2023-11-09T04:07:18.780 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.103751.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.124110.log.gz 2023-11-09T04:07:18.780 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135119.log 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103751.log.gz 2023-11-09T04:07:18.781 INFO:teuthology.orchestra.run.smithi076.stderr: 2023-11-09T04:07:18.781 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.134767.log: 93.1% -- replaced with /var/log/ceph/ceph-client.0.117777.log.gz 2023-11-09T04:07:18.781 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103127.log 2023-11-09T04:07:18.781 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.134767.log.gz 2023-11-09T04:07:18.782 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.135119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128924.log 2023-11-09T04:07:18.782 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.135119.log.gz 2023-11-09T04:07:18.782 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.103127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112234.log 2023-11-09T04:07:18.782 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.128924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129552.log 2023-11-09T04:07:18.783 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103127.log.gz 2023-11-09T04:07:18.783 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.112234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96354.log 2023-11-09T04:07:18.783 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.128924.log.gz 2023-11-09T04:07:18.783 INFO:teuthology.orchestra.run.smithi076.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112234.log.gz 2023-11-09T04:07:18.784 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97035.log 2023-11-09T04:07:18.784 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.129552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118303.log 2023-11-09T04:07:18.784 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.96354.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.129552.log.gz 2023-11-09T04:07:18.784 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.97035.log: gzip -5 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96354.log.gz 2023-11-09T04:07:18.785 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.108865.log 2023-11-09T04:07:18.785 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.118303.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.97035.log.gz 2023-11-09T04:07:18.785 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103921.log 2023-11-09T04:07:18.785 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.118303.log.gz 2023-11-09T04:07:18.786 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117226.log 2023-11-09T04:07:18.786 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.108865.log: /var/log/ceph/ceph-client.admin.103921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125090.log 2023-11-09T04:07:18.786 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108865.log.gz 2023-11-09T04:07:18.787 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103921.log.gz 2023-11-09T04:07:18.787 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.117226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94069.log 2023-11-09T04:07:18.787 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117370.log 2023-11-09T04:07:18.787 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.125090.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.117226.log.gz 2023-11-09T04:07:18.788 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.94069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108739.log 2023-11-09T04:07:18.788 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.125090.log.gz 2023-11-09T04:07:18.788 INFO:teuthology.orchestra.run.smithi076.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.94069.log.gz 2023-11-09T04:07:18.788 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.117370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123984.log 2023-11-09T04:07:18.788 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.108739.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.117370.log.gz 2023-11-09T04:07:18.789 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96348.log 2023-11-09T04:07:18.789 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.108739.log.gz 2023-11-09T04:07:18.789 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.123984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87752.log 2023-11-09T04:07:18.789 INFO:teuthology.orchestra.run.smithi076.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.123984.log.gz 2023-11-09T04:07:18.790 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.96348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108501.log 2023-11-09T04:07:18.790 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.96348.log.gz 2023-11-09T04:07:18.790 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.87752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132397.log 2023-11-09T04:07:18.791 INFO:teuthology.orchestra.run.smithi076.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.87752.log.gz 2023-11-09T04:07:18.791 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.108501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104227.log 2023-11-09T04:07:18.791 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.108501.log.gz 2023-11-09T04:07:18.791 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.132397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123276.log 2023-11-09T04:07:18.792 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.104227.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.132397.log.gz 2023-11-09T04:07:18.792 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99252.log 2023-11-09T04:07:18.792 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104227.log.gz 2023-11-09T04:07:18.792 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.123276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134893.log 2023-11-09T04:07:18.793 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.123276.log.gz 2023-11-09T04:07:18.793 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.99252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107005.log 2023-11-09T04:07:18.793 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.134893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114054.log 2023-11-09T04:07:18.793 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99252.log.gz 2023-11-09T04:07:18.794 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.107005.log: gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.134893.log.gz 2023-11-09T04:07:18.794 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.104395.log 2023-11-09T04:07:18.794 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.114054.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.107005.log.gz 2023-11-09T04:07:18.795 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127424.log 2023-11-09T04:07:18.795 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.104395.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.114054.log.gz 2023-11-09T04:07:18.795 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-11-09T04:07:18.795 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104395.log.gz 2023-11-09T04:07:18.796 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.127424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126865.log 2023-11-09T04:07:18.796 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose 89.0% -- replaced with /var/log/ceph/ceph-client.admin.127424.log.gz 2023-11-09T04:07:18.796 INFO:teuthology.orchestra.run.smithi076.stderr: -- /var/log/ceph/ceph-client.admin.107595.log 2023-11-09T04:07:18.796 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.126865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124678.log 2023-11-09T04:07:18.797 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.126865.log.gz 2023-11-09T04:07:18.797 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.107595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102623.log 2023-11-09T04:07:18.797 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.124678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128882.log 2023-11-09T04:07:18.797 INFO:teuthology.orchestra.run.smithi076.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.107595.log.gz 2023-11-09T04:07:18.798 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.102623.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.124678.log.gz 2023-11-09T04:07:18.798 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105667.log 2023-11-09T04:07:18.798 INFO:teuthology.orchestra.run.smithi076.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102623.log.gz 2023-11-09T04:07:18.798 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117056.log 2023-11-09T04:07:18.799 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.128882.log: /var/log/ceph/ceph-client.admin.105667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97696.log 2023-11-09T04:07:18.799 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.128882.log.gz 2023-11-09T04:07:18.799 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.117056.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105667.log.gz 2023-11-09T04:07:18.799 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97654.log 2023-11-09T04:07:18.800 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.117056.log.gz 2023-11-09T04:07:18.800 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.97696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115718.log 2023-11-09T04:07:18.800 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97696.log.gz 2023-11-09T04:07:18.800 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.97654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99966.log 2023-11-09T04:07:18.801 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.115718.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97654.log.gzgzip 2023-11-09T04:07:18.801 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.132151.log 2023-11-09T04:07:18.801 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.99966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131036.log 2023-11-09T04:07:18.801 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115718.log.gz 2023-11-09T04:07:18.802 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100176.log 2023-11-09T04:07:18.802 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99966.log.gz 2023-11-09T04:07:18.802 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.132151.log: /var/log/ceph/ceph-client.admin.131036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93499.log 2023-11-09T04:07:18.802 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.132151.log.gz/var/log/ceph/ceph-client.admin.100176.log: 2023-11-09T04:07:18.803 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97822.log 2023-11-09T04:07:18.803 INFO:teuthology.orchestra.run.smithi076.stderr: 87.5% 91.3% -- replaced with /var/log/ceph/ceph-client.admin.131036.log.gz 2023-11-09T04:07:18.803 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.93499.log: -- replaced with /var/log/ceph/ceph-client.admin.100176.log.gz 2023-11-09T04:07:18.803 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118489.log 2023-11-09T04:07:18.804 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93499.log.gz 2023-11-09T04:07:18.804 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.97822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101386.log 2023-11-09T04:07:18.804 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.97822.log.gz 2023-11-09T04:07:18.804 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.118489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113826.log 2023-11-09T04:07:18.805 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.101386.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.118489.log.gz 2023-11-09T04:07:18.805 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131321.log 2023-11-09T04:07:18.805 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101386.log.gz 2023-11-09T04:07:18.805 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.113826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111856.log 2023-11-09T04:07:18.806 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.113826.log.gz 2023-11-09T04:07:18.806 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.131321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102455.log 2023-11-09T04:07:18.806 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.111856.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.131321.log.gz 2023-11-09T04:07:18.806 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108697.log 2023-11-09T04:07:18.807 INFO:teuthology.orchestra.run.smithi076.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.111856.log.gz 2023-11-09T04:07:18.807 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.102455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124594.log 2023-11-09T04:07:18.807 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102455.log.gz 2023-11-09T04:07:18.807 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.108697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104057.log 2023-11-09T04:07:18.808 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.124594.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108697.log.gz 2023-11-09T04:07:18.808 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109813.log 2023-11-09T04:07:18.808 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.124594.log.gz 2023-11-09T04:07:18.808 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.104057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96581.log 2023-11-09T04:07:18.809 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104057.log.gz 2023-11-09T04:07:18.809 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.109813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97864.log 2023-11-09T04:07:18.809 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.96581.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109813.log.gz 2023-11-09T04:07:18.809 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125174.log 2023-11-09T04:07:18.810 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96581.log.gz 2023-11-09T04:07:18.810 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.97864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104437.log 2023-11-09T04:07:18.810 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97864.log.gz 2023-11-09T04:07:18.810 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.125174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97360.log 2023-11-09T04:07:18.811 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.104437.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.125174.log.gz 2023-11-09T04:07:18.811 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-09T04:07:18.811 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.97360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101956.log 90.8% -- replaced with /var/log/ceph/ceph-client.admin.104437.log.gz 2023-11-09T04:07:18.811 INFO:teuthology.orchestra.run.smithi076.stderr: 2023-11-09T04:07:18.812 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.97360.log.gz 2023-11-09T04:07:18.812 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106190.log 2023-11-09T04:07:18.812 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.101956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115676.log 2023-11-09T04:07:18.812 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.106190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95027.log 2023-11-09T04:07:18.813 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101956.log.gz 2023-11-09T04:07:18.813 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.115676.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106190.log.gz 2023-11-09T04:07:18.813 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111191.log 2023-11-09T04:07:18.813 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.95027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113016.log 2023-11-09T04:07:18.814 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.115676.log.gz 2023-11-09T04:07:18.814 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.95027.log.gz 2023-11-09T04:07:18.814 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.111191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104185.log 2023-11-09T04:07:18.814 INFO:teuthology.orchestra.run.smithi076.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.111191.log.gz 2023-11-09T04:07:18.815 INFO:teuthology.orchestra.run.smithi076.stderr: 87.7%/var/log/ceph/ceph-client.admin.113016.log: -- replaced with /var/log/ceph/ceph.log.gz 2023-11-09T04:07:18.815 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130843.log 2023-11-09T04:07:18.815 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.113016.log.gz 2023-11-09T04:07:18.816 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.104185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127172.log 2023-11-09T04:07:18.816 INFO:teuthology.orchestra.run.smithi076.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.104185.log.gz 2023-11-09T04:07:18.816 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.130843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114096.log 2023-11-09T04:07:18.816 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.127172.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.130843.log.gz 2023-11-09T04:07:18.817 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115592.log 2023-11-09T04:07:18.817 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.127172.log.gz 2023-11-09T04:07:18.817 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.114096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128694.log 2023-11-09T04:07:18.817 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.114096.log.gz 2023-11-09T04:07:18.818 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.115592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125006.log 2023-11-09T04:07:18.818 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.128694.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115592.log.gz 2023-11-09T04:07:18.818 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98601.log 2023-11-09T04:07:18.818 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.125006.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.128694.log.gz 2023-11-09T04:07:18.819 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121044.log 2023-11-09T04:07:18.819 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.125006.log.gz 2023-11-09T04:07:18.819 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.98601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126268.log 2023-11-09T04:07:18.819 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98601.log.gz 2023-11-09T04:07:18.820 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.121044.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-11-09T04:07:18.820 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.121044.log.gz 2023-11-09T04:07:18.820 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100050.log 2023-11-09T04:07:18.821 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.126268.log: /var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93773.log 2023-11-09T04:07:18.821 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.100050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124265.log 2023-11-09T04:07:18.821 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.126268.log.gz 2023-11-09T04:07:18.821 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.93773.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128307.log 2023-11-09T04:07:18.822 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.100050.log.gz 2023-11-09T04:07:18.822 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93773.log.gz 2023-11-09T04:07:18.822 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.124265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107661.log 2023-11-09T04:07:18.822 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.128307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114892.log 2023-11-09T04:07:18.823 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.124265.log.gz 2023-11-09T04:07:18.823 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.107661.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.128307.log.gz 2023-11-09T04:07:18.823 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-11-09T04:07:18.824 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.114892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112066.log 2023-11-09T04:07:18.824 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.107661.log.gz 2023-11-09T04:07:18.824 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.114892.log.gz 2023-11-09T04:07:18.824 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116964.log 2023-11-09T04:07:18.825 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.112066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129032.log 2023-11-09T04:07:18.825 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.116964.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.112066.log.gz 2023-11-09T04:07:18.825 INFO:teuthology.orchestra.run.smithi076.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.116964.log.gz 2023-11-09T04:07:18.825 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127340.log 2023-11-09T04:07:18.826 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.129032.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.129032.log.gz 2023-11-09T04:07:18.826 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113480.log 2023-11-09T04:07:18.826 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.127340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116325.log 2023-11-09T04:07:18.826 INFO:teuthology.orchestra.run.smithi076.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.127340.log.gz 2023-11-09T04:07:18.827 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.113480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130970.log 2023-11-09T04:07:18.827 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.116325.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.113480.log.gz 2023-11-09T04:07:18.827 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116325.log.gz 2023-11-09T04:07:18.827 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134420.log 2023-11-09T04:07:18.828 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131573.log 2023-11-09T04:07:18.828 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.134420.log: /var/log/ceph/ceph-client.admin.130970.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.134420.log.gz 2023-11-09T04:07:18.828 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119208.log 2023-11-09T04:07:18.828 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.131573.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.130970.log.gz 2023-11-09T04:07:18.829 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.131573.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.116623.log 2023-11-09T04:07:18.829 INFO:teuthology.orchestra.run.smithi076.stderr: 2023-11-09T04:07:18.829 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105709.log 2023-11-09T04:07:18.829 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.119208.log: /var/log/ceph/ceph-client.admin.116623.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116623.log.gz 2023-11-09T04:07:18.830 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93667.log 2023-11-09T04:07:18.830 INFO:teuthology.orchestra.run.smithi076.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.119208.log.gz 2023-11-09T04:07:18.830 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.105709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106867.log 2023-11-09T04:07:18.830 INFO:teuthology.orchestra.run.smithi076.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.105709.log.gz 2023-11-09T04:07:18.831 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.93667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107553.log 2023-11-09T04:07:18.831 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.106867.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.93667.log.gz 2023-11-09T04:07:18.831 INFO:teuthology.orchestra.run.smithi076.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.106867.log.gz 2023-11-09T04:07:18.831 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110373.log 2023-11-09T04:07:18.832 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96349.log 2023-11-09T04:07:18.832 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.107553.log: /var/log/ceph/ceph-client.admin.110373.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107553.log.gz 2023-11-09T04:07:18.832 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110373.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113994.log 2023-11-09T04:07:18.832 INFO:teuthology.orchestra.run.smithi076.stderr: 2023-11-09T04:07:18.833 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119648.log 2023-11-09T04:07:18.833 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.96349.log: /var/log/ceph/ceph-client.admin.113994.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.113994.log.gz 2023-11-09T04:07:18.833 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.96349.log.gz 2023-11-09T04:07:18.833 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.81722.log 2023-11-09T04:07:18.834 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95542.log 2023-11-09T04:07:18.834 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.119648.log: /var/log/ceph/ceph.tmp-client.admin.81722.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.81722.log.gz 2023-11-09T04:07:18.834 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.119648.log.gz 2023-11-09T04:07:18.834 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93541.log 2023-11-09T04:07:18.835 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.95542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134725.log 2023-11-09T04:07:18.835 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.95542.log.gz 2023-11-09T04:07:18.835 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.93541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111940.log 2023-11-09T04:07:18.835 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.134725.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93541.log.gz 2023-11-09T04:07:18.836 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96701.log 2023-11-09T04:07:18.836 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.134725.log.gz 2023-11-09T04:07:18.836 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.111940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120456.log 2023-11-09T04:07:18.836 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.96701.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111940.log.gz 2023-11-09T04:07:18.837 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96701.log.gz 2023-11-09T04:07:18.837 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123942.log 2023-11-09T04:07:18.837 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95169.log 2023-11-09T04:07:18.837 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.120456.log: /var/log/ceph/ceph-client.admin.123942.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.120456.log.gz 2023-11-09T04:07:18.838 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.123942.log.gz 2023-11-09T04:07:18.838 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94985.log 2023-11-09T04:07:18.838 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133367.log 2023-11-09T04:07:18.838 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.95169.log: /var/log/ceph/ceph-client.admin.94985.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95169.log.gz 2023-11-09T04:07:18.839 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94985.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102413.log 2023-11-09T04:07:18.839 INFO:teuthology.orchestra.run.smithi076.stderr: 2023-11-09T04:07:18.839 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100535.log 2023-11-09T04:07:18.839 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.133367.log: /var/log/ceph/ceph-client.admin.102413.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.133367.log.gz 2023-11-09T04:07:18.840 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102413.log.gz 2023-11-09T04:07:18.840 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117184.log 2023-11-09T04:07:18.840 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107259.log 2023-11-09T04:07:18.840 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.100535.log: /var/log/ceph/ceph-client.admin.117184.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100535.log.gz 2023-11-09T04:07:18.841 INFO:teuthology.orchestra.run.smithi076.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.117184.log.gz 2023-11-09T04:07:18.841 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122888.log 2023-11-09T04:07:18.841 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106495.log 2023-11-09T04:07:18.841 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.107259.log: /var/log/ceph/ceph-client.admin.122888.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.107259.log.gz 2023-11-09T04:07:18.842 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.122888.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99504.log 2023-11-09T04:07:18.842 INFO:teuthology.orchestra.run.smithi076.stderr: 2023-11-09T04:07:18.842 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120004.log 2023-11-09T04:07:18.842 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.106495.log: /var/log/ceph/ceph-client.admin.99504.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106495.log.gz 2023-11-09T04:07:18.843 INFO:teuthology.orchestra.run.smithi076.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106064.log 2023-11-09T04:07:18.843 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99504.log.gz 2023-11-09T04:07:18.843 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121256.log 2023-11-09T04:07:18.843 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.120004.log: /var/log/ceph/ceph-client.admin.106064.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.120004.log.gz 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106064.log.gz 2023-11-09T04:07:18.844 INFO:teuthology.orchestra.run.smithi076.stderr: 2023-11-09T04:07:18.844 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123140.log 2023-11-09T04:07:18.844 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.121256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122054.log 2023-11-09T04:07:18.844 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.121256.log.gz 2023-11-09T04:07:18.845 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.123140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112192.log 2023-11-09T04:07:18.845 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.122054.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.123140.log.gz 2023-11-09T04:07:18.845 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.122054.log.gz 2023-11-09T04:07:18.845 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122515.log 2023-11-09T04:07:18.846 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.112192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109262.log 2023-11-09T04:07:18.846 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.112192.log.gz 2023-11-09T04:07:18.846 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.122515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.125642.log 2023-11-09T04:07:18.846 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.109262.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.122515.log.gz 2023-11-09T04:07:18.847 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.109262.log.gz 2023-11-09T04:07:18.847 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96690.log 2023-11-09T04:07:18.847 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.0.125642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130064.log 2023-11-09T04:07:18.847 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.96690.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96690.log.gz 2023-11-09T04:07:18.848 INFO:teuthology.orchestra.run.smithi076.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.0.125642.log.gz 2023-11-09T04:07:18.848 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130717.log 2023-11-09T04:07:18.848 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.130064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130130.log 2023-11-09T04:07:18.848 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.130717.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.130064.log.gz 2023-11-09T04:07:18.849 INFO:teuthology.orchestra.run.smithi076.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.130717.log.gz 2023-11-09T04:07:18.849 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102749.log 2023-11-09T04:07:18.849 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.130130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118999.log 2023-11-09T04:07:18.849 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.130130.log.gz 2023-11-09T04:07:18.850 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.102749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113266.log 2023-11-09T04:07:18.850 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.118999.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102749.log.gz 2023-11-09T04:07:18.850 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118999.log.gz 2023-11-09T04:07:18.850 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93156.log 2023-11-09T04:07:18.851 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.113266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135035.log 2023-11-09T04:07:18.851 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0%/var/log/ceph/ceph-client.admin.93156.log: -- replaced with /var/log/ceph/ceph-client.admin.113266.log.gz 2023-11-09T04:07:18.851 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127560.log 2023-11-09T04:07:18.851 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.93156.log.gz 2023-11-09T04:07:18.852 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.135035.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.135035.log.gz 2023-11-09T04:07:18.852 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97528.log 2023-11-09T04:07:18.852 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.127560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99840.log 2023-11-09T04:07:18.852 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.127560.log.gz 2023-11-09T04:07:18.853 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.97528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132085.log 2023-11-09T04:07:18.853 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.97528.log.gz 2023-11-09T04:07:18.853 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.99840.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.99840.log.gz 2023-11-09T04:07:18.853 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114138.log 2023-11-09T04:07:18.854 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.132085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132991.log 2023-11-09T04:07:18.854 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.114138.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.132085.log.gz 2023-11-09T04:07:18.854 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128571.log 2023-11-09T04:07:18.854 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.132991.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.114138.log.gz 2023-11-09T04:07:18.855 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.132991.log.gz 2023-11-09T04:07:18.855 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107133.log 2023-11-09T04:07:18.855 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.128571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101512.log 2023-11-09T04:07:18.855 INFO:teuthology.orchestra.run.smithi076.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.128571.log.gz/var/log/ceph/ceph-client.admin.107133.log: 2023-11-09T04:07:18.856 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115634.log 2023-11-09T04:07:18.856 INFO:teuthology.orchestra.run.smithi076.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.107133.log.gz/var/log/ceph/ceph-client.admin.101512.log: 2023-11-09T04:07:18.856 INFO:teuthology.orchestra.run.smithi076.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.101512.log.gz 2023-11-09T04:07:18.857 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124409.log 2023-11-09T04:07:18.857 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.115634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124152.log 2023-11-09T04:07:18.857 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.115634.log.gz 2023-11-09T04:07:18.857 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.124409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100619.log 2023-11-09T04:07:18.858 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.124152.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.124409.log.gz 2023-11-09T04:07:18.858 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.124152.log.gz 2023-11-09T04:07:18.858 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113614.log 2023-11-09T04:07:18.859 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.100619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117268.log 2023-11-09T04:07:18.859 INFO:teuthology.orchestra.run.smithi076.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.100619.log.gz 2023-11-09T04:07:18.859 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.113614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127978.log 2023-11-09T04:07:18.859 INFO:teuthology.orchestra.run.smithi076.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.113614.log.gz 2023-11-09T04:07:18.860 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.117268.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117268.log.gz 2023-11-09T04:07:18.860 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133033.log 2023-11-09T04:07:18.860 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.127978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108116.log 2023-11-09T04:07:18.861 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.133033.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.127978.log.gz 2023-11-09T04:07:18.861 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.133033.log.gz 2023-11-09T04:07:18.861 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112463.log 2023-11-09T04:07:18.861 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.108116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103295.log 2023-11-09T04:07:18.862 INFO:teuthology.orchestra.run.smithi076.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.108116.log.gz/var/log/ceph/ceph-client.admin.112463.log: 2023-11-09T04:07:18.862 INFO:teuthology.orchestra.run.smithi076.stderr: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113224.log 2023-11-09T04:07:18.862 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.112463.log.gz 2023-11-09T04:07:18.862 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.103295.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.103295.log.gz 2023-11-09T04:07:18.863 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136060.log 2023-11-09T04:07:18.863 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.113224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126430.log 2023-11-09T04:07:18.863 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.113224.log.gz 2023-11-09T04:07:18.864 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.136060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111450.log 2023-11-09T04:07:18.864 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.126430.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.126430.log.gz 2023-11-09T04:07:18.864 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124213.log 2023-11-09T04:07:18.864 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.111450.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111450.log.gz 2023-11-09T04:07:18.865 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.104760.log 2023-11-09T04:07:18.865 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.124213.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.124213.log.gz 2023-11-09T04:07:18.865 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99546.log 2023-11-09T04:07:18.865 INFO:teuthology.orchestra.run.smithi076.stderr: 95.6% -- replaced with /var/log/ceph/ceph-client.admin.136060.log.gz/var/log/ceph/ceph-client.0.104760.log: 2023-11-09T04:07:18.866 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.0.104760.log.gz 2023-11-09T04:07:18.866 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106825.log 2023-11-09T04:07:18.866 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.99546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93901.log 2023-11-09T04:07:18.867 INFO:teuthology.orchestra.run.smithi076.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99546.log.gz 2023-11-09T04:07:18.867 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.106825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94153.log 2023-11-09T04:07:18.868 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.93901.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106825.log.gz 2023-11-09T04:07:18.868 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106575.log 2023-11-09T04:07:18.868 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.93901.log.gz 2023-11-09T04:07:18.868 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.94153.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94153.log.gz 2023-11-09T04:07:18.869 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96018.log 2023-11-09T04:07:18.869 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115060.log 2023-11-09T04:07:18.869 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.106575.log: /var/log/ceph/ceph-client.admin.96018.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.106575.log.gz 2023-11-09T04:07:18.869 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.96018.log.gz 2023-11-09T04:07:18.870 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112024.log 2023-11-09T04:07:18.870 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129468.log 2023-11-09T04:07:18.870 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.112024.log: /var/log/ceph/ceph-client.admin.115060.log: 88.2% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115060.log.gz 2023-11-09T04:07:18.871 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.112024.log.gz 2023-11-09T04:07:18.871 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.129903.log 2023-11-09T04:07:18.871 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101050.log 2023-11-09T04:07:18.871 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.129468.log: /var/log/ceph/ceph-client.0.129903.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.129468.log.gz 2023-11-09T04:07:18.872 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124532.log 2023-11-09T04:07:18.872 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.101050.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101050.log.gz 2023-11-09T04:07:18.872 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-11-09T04:07:18.872 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.124532.log: 92.5% -- replaced with /var/log/ceph/ceph-client.0.129903.log.gz 2023-11-09T04:07:18.873 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.124532.log.gz 2023-11-09T04:07:18.873 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121641.log 2023-11-09T04:07:18.873 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105793.log 2023-11-09T04:07:18.873 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.121641.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.121641.log.gz 2023-11-09T04:07:18.874 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130759.log 2023-11-09T04:07:18.874 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.105793.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105793.log.gz 2023-11-09T04:07:18.874 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133495.log 2023-11-09T04:07:18.875 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.130759.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.130759.log.gz 2023-11-09T04:07:18.875 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101628.log 2023-11-09T04:07:18.875 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.133495.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.133495.log.gz 2023-11-09T04:07:18.875 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109430.log 2023-11-09T04:07:18.876 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.101628.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.101628.log.gz 2023-11-09T04:07:18.876 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105751.log 2023-11-09T04:07:18.876 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.109430.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109430.log.gz 2023-11-09T04:07:18.876 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101302.log 2023-11-09T04:07:18.877 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.105751.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105751.log.gz 2023-11-09T04:07:18.877 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102082.log 2023-11-09T04:07:18.877 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.101302.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101302.log.gz 2023-11-09T04:07:18.877 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126226.log 2023-11-09T04:07:18.878 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.102082.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102082.log.gz 2023-11-09T04:07:18.878 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117014.log 2023-11-09T04:07:18.878 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.126226.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.126226.log.gz 2023-11-09T04:07:18.879 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130234.log 2023-11-09T04:07:18.879 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.117014.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117014.log.gz 2023-11-09T04:07:18.879 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106400.log 2023-11-09T04:07:18.879 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.130234.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.130234.log.gz 2023-11-09T04:07:18.880 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.106400.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106400.log.gz 2023-11-09T04:07:18.881 INFO:teuthology.orchestra.run.smithi076.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-11-09T04:07:18.881 INFO:teuthology.orchestra.run.smithi076.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-11-09T04:07:18.882 INFO:teuthology.orchestra.run.smithi113.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-11-09T04:07:18.882 INFO:teuthology.orchestra.run.smithi113.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-11-09T04:07:19.037 INFO:teuthology.orchestra.run.smithi076.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-11-09T04:07:19.567 INFO:teuthology.orchestra.run.smithi113.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-11-09T04:07:19.600 INFO:teuthology.orchestra.run.smithi113.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-11-09T04:07:19.642 INFO:teuthology.orchestra.run.smithi113.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-11-09T04:07:19.651 INFO:teuthology.orchestra.run.smithi076.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-11-09T04:07:19.666 INFO:teuthology.orchestra.run.smithi113.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-11-09T04:07:19.668 INFO:teuthology.orchestra.run.smithi113.stderr: 2023-11-09T04:07:19.669 INFO:teuthology.orchestra.run.smithi113.stderr:real 0m1.475s 2023-11-09T04:07:19.669 INFO:teuthology.orchestra.run.smithi113.stderr:user 0m6.507s 2023-11-09T04:07:19.669 INFO:teuthology.orchestra.run.smithi113.stderr:sys 0m0.243s 2023-11-09T04:07:19.832 INFO:teuthology.orchestra.run.smithi076.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-11-09T04:07:19.891 INFO:teuthology.orchestra.run.smithi076.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-11-09T04:07:19.892 INFO:teuthology.orchestra.run.smithi076.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-11-09T04:07:19.894 INFO:teuthology.orchestra.run.smithi076.stderr: 2023-11-09T04:07:19.895 INFO:teuthology.orchestra.run.smithi076.stderr:real 0m1.733s 2023-11-09T04:07:19.895 INFO:teuthology.orchestra.run.smithi076.stderr:user 0m7.227s 2023-11-09T04:07:19.895 INFO:teuthology.orchestra.run.smithi076.stderr:sys 0m0.654s 2023-11-09T04:07:19.896 INFO:tasks.ceph:Archiving logs... 2023-11-09T04:07:19.896 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-11-09_03:21:01-fs:functional-main-distro-default-smithi/7452464/remote/smithi076/log 2023-11-09T04:07:19.898 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-09T04:07:21.449 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-11-09_03:21:01-fs:functional-main-distro-default-smithi/7452464/remote/smithi113/log 2023-11-09T04:07:21.451 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-09T04:07:22.316 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 2758, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 2590, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 2356, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 1611, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi076 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-11-09T04:07:22.326 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-09T04:07:22.336 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 2758, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 2590, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 2356, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 1611, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_batrick_ceph_8dae6b9fa1760040e7156daa9d03f77642df4f90/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi076 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-11-09T04:07:22.340 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-09T04:07:22.341 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-09T04:07:22.346 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-09T04:07:22.423 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-09T04:07:22.432 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-09T04:07:22.433 DEBUG:teuthology.orchestra.run.smithi076:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-09T04:07:22.436 DEBUG:teuthology.orchestra.run.smithi113:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-09T04:07:22.450 INFO:teuthology.orchestra.run.smithi076.stderr:bash: line 1: ntpq: command not found 2023-11-09T04:07:22.451 INFO:teuthology.orchestra.run.smithi113.stderr:bash: line 1: ntpq: command not found 2023-11-09T04:07:22.454 INFO:teuthology.orchestra.run.smithi076.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-09T04:07:22.455 INFO:teuthology.orchestra.run.smithi076.stdout:=============================================================================== 2023-11-09T04:07:22.455 INFO:teuthology.orchestra.run.smithi076.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 25 +511us[ +511us] +/- 74ms 2023-11-09T04:07:22.455 INFO:teuthology.orchestra.run.smithi076.stdout:^* hv02.front.sepia.ceph.com 3 6 377 27 +579us[ +620us] +/- 58ms 2023-11-09T04:07:22.456 INFO:teuthology.orchestra.run.smithi076.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 -1074us[-1033us] +/- 67ms 2023-11-09T04:07:22.456 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-11-09T04:07:22.457 INFO:teuthology.orchestra.run.smithi113.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-09T04:07:22.458 INFO:teuthology.orchestra.run.smithi113.stdout:=============================================================================== 2023-11-09T04:07:22.458 INFO:teuthology.orchestra.run.smithi113.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 29 +498us[ +513us] +/- 74ms 2023-11-09T04:07:22.458 INFO:teuthology.orchestra.run.smithi113.stdout:^* hv02.front.sepia.ceph.com 3 6 377 27 +573us[ +588us] +/- 58ms 2023-11-09T04:07:22.459 INFO:teuthology.orchestra.run.smithi113.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 -1085us[-1070us] +/- 67ms 2023-11-09T04:07:22.459 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-11-09T04:07:22.459 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-09T04:07:22.469 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-09T04:07:22.470 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-09T04:07:22.480 DEBUG:teuthology.orchestra.run.smithi076:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-09T04:07:22.547 DEBUG:teuthology.orchestra.run.smithi113:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-09T04:07:22.643 DEBUG:teuthology.orchestra.run.smithi076:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-09T04:07:22.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T04:07:22.673 DEBUG:teuthology.orchestra.run.smithi113:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-09T04:07:22.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T04:07:22.705 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-09T04:07:22.715 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-09T04:07:22.726 INFO:teuthology.task.internal:Duration was 1852.460792 seconds 2023-11-09T04:07:22.726 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-09T04:07:22.755 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-09T04:07:22.755 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-09T04:07:22.758 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-09T04:07:22.801 INFO:teuthology.orchestra.run.smithi113.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-09T04:07:22.802 INFO:teuthology.orchestra.run.smithi076.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-09T04:07:23.012 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-09T04:07:23.012 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi076.front.sepia.ceph.com 2023-11-09T04:07:23.013 DEBUG:teuthology.orchestra.run.smithi076:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-09T04:07:23.034 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi113.front.sepia.ceph.com 2023-11-09T04:07:23.034 DEBUG:teuthology.orchestra.run.smithi113:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-09T04:07:23.063 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-09T04:07:23.063 DEBUG:teuthology.orchestra.run.smithi076:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-09T04:07:23.077 DEBUG:teuthology.orchestra.run.smithi113:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-09T04:07:23.131 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-09T04:07:23.132 DEBUG:teuthology.orchestra.run.smithi076:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-09T04:07:23.147 DEBUG:teuthology.orchestra.run.smithi113:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-09T04:07:23.281 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-09T04:07:23.291 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-09T04:07:23.292 DEBUG:teuthology.orchestra.run.smithi076:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-09T04:07:23.318 DEBUG:teuthology.orchestra.run.smithi113:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-09T04:07:23.345 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-09T04:07:23.355 DEBUG:teuthology.orchestra.run.smithi076:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-09T04:07:23.360 DEBUG:teuthology.orchestra.run.smithi113:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-09T04:07:23.384 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern = core 2023-11-09T04:07:23.410 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern = core 2023-11-09T04:07:23.426 DEBUG:teuthology.orchestra.run.smithi076:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-09T04:07:23.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T04:07:23.453 DEBUG:teuthology.orchestra.run.smithi113:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-09T04:07:23.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-09T04:07:23.482 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-09T04:07:23.492 INFO:teuthology.task.internal:Transferring archived files... 2023-11-09T04:07:23.493 DEBUG:teuthology.misc:Transferring archived files from smithi076:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-11-09_03:21:01-fs:functional-main-distro-default-smithi/7452464/remote/smithi076 2023-11-09T04:07:23.494 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-09T04:07:23.527 DEBUG:teuthology.misc:Transferring archived files from smithi113:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-11-09_03:21:01-fs:functional-main-distro-default-smithi/7452464/remote/smithi113 2023-11-09T04:07:23.528 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-09T04:07:23.564 INFO:teuthology.task.internal:Removing archive directory... 2023-11-09T04:07:23.564 DEBUG:teuthology.orchestra.run.smithi076:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-09T04:07:23.567 DEBUG:teuthology.orchestra.run.smithi113:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-09T04:07:23.613 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-09T04:07:23.624 INFO:teuthology.task.internal:Not uploading archives. 2023-11-09T04:07:23.625 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-09T04:07:23.634 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-09T04:07:23.634 DEBUG:teuthology.orchestra.run.smithi076:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-09T04:07:23.637 DEBUG:teuthology.orchestra.run.smithi113:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-09T04:07:23.649 INFO:teuthology.orchestra.run.smithi076.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 9 04:07 /home/ubuntu/cephtest 2023-11-09T04:07:23.667 INFO:teuthology.orchestra.run.smithi113.stdout: 265245 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 9 04:07 /home/ubuntu/cephtest 2023-11-09T04:07:23.668 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-09T04:07:23.678 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-09T04:07:23.689 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi076 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-11-09T04:07:23.689 INFO:teuthology.nuke:Checking targets against current locks 2023-11-09T04:07:23.717 DEBUG:teuthology.nuke:shortname: smithi076 2023-11-09T04:07:23.718 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-09T04:07:23.743 DEBUG:teuthology.nuke:shortname: smithi113 2023-11-09T04:07:23.744 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-09T04:07:23.747 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi076.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-11-09_03:21:01-fs:functional-main-distro-default-smithi/7452464', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-09 03:27:25.666687', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBSaOfkkKDMrAw8lm/2WHJKaNe6HUq/VfeTji9wjOUR84c6zvChUurbrU6h/utzarVeqdBjqZcioe/t0+54KMAU='} 2023-11-09T04:07:23.769 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi113.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-11-09_03:21:01-fs:functional-main-distro-default-smithi/7452464', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-09 03:27:25.668533', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK1EN9hrd6xzeNxf/hBCFS2ygei1vHqiKEq9qLM07K+yU9nQIGDVWXPapz/JUuXhIDcePlEle4hhoRzs9FOUUMg='} 2023-11-09T04:07:23.800 INFO:teuthology.orchestra.console.smithi076:Power off 2023-11-09T04:07:23.801 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-09T04:07:23.824 DEBUG:teuthology.orchestra.console.smithi076:power off output: Chassis Power Control: Down/Off 2023-11-09T04:07:23.824 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-09T04:07:23.843 INFO:teuthology.orchestra.console.smithi113:Power off 2023-11-09T04:07:23.843 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-09T04:07:23.853 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is on 2023-11-09T04:07:23.867 DEBUG:teuthology.orchestra.console.smithi113:power off output: Chassis Power Control: Down/Off 2023-11-09T04:07:23.867 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-09T04:07:23.889 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on 2023-11-09T04:07:27.854 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-09T04:07:27.891 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-09T04:07:27.964 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is on 2023-11-09T04:07:28.001 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on 2023-11-09T04:07:31.970 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-09T04:07:32.002 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-09T04:07:32.081 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is off 2023-11-09T04:07:32.112 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on 2023-11-09T04:07:32.182 INFO:teuthology.orchestra.console.smithi076:Power off completed 2023-11-09T04:07:36.117 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-09T04:07:36.226 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is off 2023-11-09T04:07:36.327 INFO:teuthology.orchestra.console.smithi113:Power off completed 2023-11-09T04:07:36.495 INFO:teuthology.run:Summary data: description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snapshots} duration: 1852.4607923030853 failure_reason: 'Test failure: test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots), test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots)' flavor: default owner: scheduled_pdonnell@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=5f99bffc4cc44425a7516634a1672ce5 status: fail success: false 2023-11-09T04:07:36.495 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-09T04:07:36.570 INFO:teuthology.run:FAIL